2023-12-01T01:18:47.969 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-01T01:18:47.969 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199 --name yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199/orig.config.yaml 2023-12-01T01:18:48.015 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-01T01:18:48.103 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199 branch: reef-release description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: yweinste@redhat.com first_in_suite: false job_id: '7473199' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef-release ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef-release sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 owner: scheduled_yuriw@teuthology priority: 55 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8832 sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: reef-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAG1ZN4VL+XG0I899Gyv9++bmYkxtxRfIGXaOnv+Y5GEYm47zUgmZY1Nos8WsFkWxPJZYnlnIPcgDlygBFmRhMI= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrWw7VN9mng46aNqeugrNH8siuKXNZ4jODSxOlLGm+vYJVed2n0zJzYcvmZ9uTx05qQgz+NtqqnVuk7wDuI/G8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: f2de108beb0a6dde7f3e6c96039071828a1f6269 timestamp: 2023-11-30_22:47:08 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-01T01:18:48.103 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa; will attempt to use it 2023-12-01T01:18:48.104 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa/tasks 2023-12-01T01:18:48.104 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-01T01:18:48.106 INFO:teuthology.task.internal:Checking packages... 2023-12-01T01:18:48.136 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a6f82997ac57b70a895455dfed6256360a1e4c32' 2023-12-01T01:18:48.136 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-01T01:18:48.136 INFO:teuthology.packaging:ref: None 2023-12-01T01:18:48.136 INFO:teuthology.packaging:tag: None 2023-12-01T01:18:48.136 INFO:teuthology.packaging:branch: reef-release 2023-12-01T01:18:48.137 INFO:teuthology.packaging:sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:18:48.137 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef-release 2023-12-01T01:18:48.284 INFO:teuthology.task.internal:Found packages for ceph version 18.2.0-1249.ga6f82997 2023-12-01T01:18:48.285 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-01T01:18:48.290 INFO:teuthology.task.internal:no buildpackages task found 2023-12-01T01:18:48.290 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-01T01:18:48.302 INFO:teuthology.task.internal:Saving configuration 2023-12-01T01:18:48.320 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-01T01:18:48.326 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-01T01:18:48.348 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 01:13:49.534310', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAG1ZN4VL+XG0I899Gyv9++bmYkxtxRfIGXaOnv+Y5GEYm47zUgmZY1Nos8WsFkWxPJZYnlnIPcgDlygBFmRhMI='} 2023-12-01T01:18:48.370 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 01:13:49.532312', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrWw7VN9mng46aNqeugrNH8siuKXNZ4jODSxOlLGm+vYJVed2n0zJzYcvmZ9uTx05qQgz+NtqqnVuk7wDuI/G8='} 2023-12-01T01:18:48.370 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-01T01:18:48.377 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-12-01T01:18:48.377 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-12-01T01:18:48.377 INFO:teuthology.run_tasks:Running task console_log... 2023-12-01T01:18:48.453 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fefe87b29d0>, signals=[15]) 2023-12-01T01:18:48.453 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-01T01:18:48.459 INFO:teuthology.task.internal:Opening connections... 2023-12-01T01:18:48.460 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:18:48.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:18:48.549 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T01:18:48.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:18:48.634 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-01T01:18:48.642 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2023-12-01T01:18:48.659 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2023-12-01T01:18:48.659 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2023-12-01T01:18:48.716 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-01T01:18:48.717 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-01T01:18:48.718 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2023-12-01T01:18:48.748 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2023-12-01T01:18:48.765 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2023-12-01T01:18:48.765 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2023-12-01T01:18:48.822 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2023-12-01T01:18:48.822 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2023-12-01T01:18:48.822 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2023-12-01T01:18:48.822 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2023-12-01T01:18:48.822 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2023-12-01T01:18:48.822 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2023-12-01T01:18:48.822 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-01T01:18:48.822 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2023-12-01T01:18:48.822 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2023-12-01T01:18:48.822 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-01T01:18:48.823 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2023-12-01T01:18:48.823 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-01T01:18:48.823 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-01T01:18:48.823 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-01T01:18:48.823 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2023-12-01T01:18:48.847 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-01T01:18:48.864 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-01T01:18:48.870 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-01T01:18:48.870 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2023-12-01T01:18:48.872 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2023-12-01T01:18:48.888 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-01T01:18:48.894 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-01T01:18:48.894 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2023-12-01T01:18:48.929 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2023-12-01T01:18:49.034 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-01T01:18:49.088 INFO:teuthology.run_tasks:Running task kernel... 2023-12-01T01:18:49.102 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-12-01T01:18:49.103 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-12-01T01:18:49.103 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-12-01T01:18:49.103 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-01T01:18:49.103 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-01T01:18:49.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:18:49.120 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-12-01T01:18:49.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:18:49.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-12-01T01:18:49.273 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-12-01T01:18:49.273 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-12-01T01:18:49.273 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2023-12-01T01:18:49.290 DEBUG:teuthology.task.kernel:sha1 for client.1 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-12-01T01:18:49.290 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-12-01T01:18:49.290 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2023-12-01T01:18:49.291 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-361.el9.x86_64 2023-12-01T01:18:49.291 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-12-01T01:18:49.292 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-01T01:18:49.292 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum upgrade -y linux-firmware 2023-12-01T01:18:49.309 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-361.el9.x86_64 2023-12-01T01:18:49.309 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-12-01T01:18:49.309 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-01T01:18:49.309 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum upgrade -y linux-firmware 2023-12-01T01:18:52.022 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - AppStream 5.9 MB/s | 6.4 MB 00:01 2023-12-01T01:18:52.033 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - AppStream 6.0 MB/s | 6.4 MB 00:01 2023-12-01T01:18:53.647 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2023-12-01T01:18:53.742 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2023-12-01T01:18:54.574 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 7.9 MB 00:00 2023-12-01T01:18:56.087 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 7.9 MB 00:02 2023-12-01T01:18:59.016 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 18 MB 00:03 2023-12-01T01:18:59.413 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 18 MB 00:02 2023-12-01T01:19:03.447 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 5.8 MB 00:00 2023-12-01T01:19:03.974 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 5.8 MB 00:01 2023-12-01T01:19:04.873 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 15 kB 00:00 2023-12-01T01:19:05.467 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 15 kB 00:00 2023-12-01T01:19:05.665 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 31 MB/s | 20 MB 00:00 2023-12-01T01:19:06.460 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 23 MB/s | 20 MB 00:00 2023-12-01T01:19:10.398 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2023-12-01T01:19:11.385 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-12-01T01:19:12.787 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2023-12-01T01:19:12.792 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2023-12-01T01:19:12.792 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2023-12-01T01:19:12.792 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2023-12-01T01:19:12.792 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2023-12-01T01:19:12.792 INFO:teuthology.orchestra.run.smithi142.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-12-01T01:19:12.792 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:19:12.792 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2023-12-01T01:19:12.792 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2023-12-01T01:19:12.792 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 1 Package 2023-12-01T01:19:12.793 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:19:12.793 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 328 M 2023-12-01T01:19:12.793 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2023-12-01T01:19:13.833 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2023-12-01T01:19:13.840 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2023-12-01T01:19:13.840 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2023-12-01T01:19:13.840 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2023-12-01T01:19:13.841 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2023-12-01T01:19:13.841 INFO:teuthology.orchestra.run.smithi115.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-12-01T01:19:13.841 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:19:13.841 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2023-12-01T01:19:13.841 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2023-12-01T01:19:13.841 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 1 Package 2023-12-01T01:19:13.842 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:19:13.842 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 328 M 2023-12-01T01:19:13.842 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2023-12-01T01:19:24.059 INFO:teuthology.orchestra.run.smithi115.stdout:linux-firmware-20231030-141.el9.noarch.rpm 32 MB/s | 328 MB 00:10 2023-12-01T01:19:24.060 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:19:24.060 INFO:teuthology.orchestra.run.smithi115.stdout:Total 32 MB/s | 328 MB 00:10 2023-12-01T01:19:24.061 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2023-12-01T01:19:24.148 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2023-12-01T01:19:24.148 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2023-12-01T01:19:24.915 INFO:teuthology.orchestra.run.smithi142.stdout:linux-firmware-20231030-141.el9.noarch.rpm 27 MB/s | 328 MB 00:12 2023-12-01T01:19:24.916 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:19:24.916 INFO:teuthology.orchestra.run.smithi142.stdout:Total 27 MB/s | 328 MB 00:12 2023-12-01T01:19:24.917 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2023-12-01T01:19:25.020 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2023-12-01T01:19:25.020 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2023-12-01T01:19:26.314 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2023-12-01T01:19:26.314 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2023-12-01T01:19:27.242 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2023-12-01T01:19:27.242 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2023-12-01T01:19:32.082 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2023-12-01T01:19:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-01T01:19:32.622 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-01T01:19:33.232 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2023-12-01T01:19:33.473 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-12-01T01:19:33.473 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-01T01:19:33.554 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-01T01:19:33.636 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-01T01:19:33.878 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-01T01:19:33.878 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:19:33.879 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2023-12-01T01:19:33.879 INFO:teuthology.orchestra.run.smithi115.stdout: linux-firmware-20231030-141.el9.noarch 2023-12-01T01:19:33.879 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:19:33.879 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2023-12-01T01:19:34.158 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-12-01T01:19:34.158 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-12-01T01:19:34.305 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-12-01T01:19:34.306 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-12-01T01:19:34.308 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi115.front.sepia.ceph.com to complete... 2023-12-01T01:19:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-12-01T01:19:34.593 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-01T01:19:34.967 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-01T01:19:34.968 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:19:34.968 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2023-12-01T01:19:34.968 INFO:teuthology.orchestra.run.smithi142.stdout: linux-firmware-20231030-141.el9.noarch 2023-12-01T01:19:34.968 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:19:34.968 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2023-12-01T01:19:35.246 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-12-01T01:19:35.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-12-01T01:19:35.388 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-12-01T01:19:35.388 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-12-01T01:19:35.391 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi142.front.sepia.ceph.com to complete... 2023-12-01T01:20:01.134 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01 01:20:01 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-01T01:20:01.135 INFO:teuthology.orchestra.run.smithi142.stderr:FINISHED --2023-12-01 01:20:01-- 2023-12-01T01:20:01.135 INFO:teuthology.orchestra.run.smithi142.stderr:Total wall clock time: 26s 2023-12-01T01:20:01.135 INFO:teuthology.orchestra.run.smithi142.stderr:Downloaded: 1 files, 306M in 25s (12.1 MB/s) 2023-12-01T01:20:01.135 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-12-01T01:20:01.135 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-01T01:20:02.976 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying... ######################################## 2023-12-01T01:20:03.058 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing... ######################################## 2023-12-01T01:20:03.138 INFO:teuthology.orchestra.run.smithi142.stdout:Updating / installing... 2023-12-01T01:20:06.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01 01:20:06 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-01T01:20:06.077 INFO:teuthology.orchestra.run.smithi115.stderr:FINISHED --2023-12-01 01:20:06-- 2023-12-01T01:20:06.077 INFO:teuthology.orchestra.run.smithi115.stderr:Total wall clock time: 32s 2023-12-01T01:20:06.077 INFO:teuthology.orchestra.run.smithi115.stderr:Downloaded: 1 files, 306M in 31s (9.78 MB/s) 2023-12-01T01:20:06.078 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-12-01T01:20:06.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-01T01:20:07.916 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying... ######################################## 2023-12-01T01:20:07.988 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing... ######################################## 2023-12-01T01:20:08.067 INFO:teuthology.orchestra.run.smithi115.stdout:Updating / installing... 2023-12-01T01:20:10.202 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-12-01T01:20:15.205 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-12-01T01:20:24.917 INFO:teuthology.orchestra.run.smithi142.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-01T01:20:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-01T01:20:45.881 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-01T01:20:45.881 DEBUG:teuthology.orchestra.run.smithi142:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-01T01:20:45.951 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-12-01T01:20:45.951 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-12-01T01:20:45.951 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-12-01T01:20:45.951 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-12-01T01:20:45.951 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-12-01T01:20:45.951 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-01T01:20:45.952 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-01T01:20:45.953 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-12-01T01:20:45.954 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-12-01T01:20:45.955 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-12-01T01:20:45.956 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-12-01T01:20:45.957 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-12-01T01:20:45.958 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-12-01T01:20:45.959 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-12-01T01:20:45.960 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-12-01T01:20:45.961 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-12-01T01:20:45.962 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-12-01T01:20:45.963 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-12-01T01:20:45.964 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-01T01:20:45.965 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-01T01:20:45.966 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-12-01T01:20:45.967 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-01T01:20:45.968 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-01T01:20:45.969 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-12-01T01:20:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-01T01:20:45.971 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-01T01:20:45.972 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-12-01T01:20:45.973 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-12-01T01:20:45.974 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-12-01T01:20:45.975 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-12-01T01:20:45.976 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-12-01T01:20:45.977 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-12-01T01:20:45.978 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-12-01T01:20:45.979 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-01T01:20:45.980 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-12-01T01:20:45.981 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-12-01T01:20:45.982 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-01T01:20:45.983 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-01T01:20:45.984 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-12-01T01:20:45.985 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-01T01:20:45.986 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-12-01T01:20:45.987 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-01T01:20:45.988 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-12-01T01:20:45.989 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-12-01T01:20:45.990 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-12-01T01:20:45.991 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-12-01T01:20:45.992 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-12-01T01:20:45.993 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-12-01T01:20:45.994 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-01T01:20:45.995 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-01T01:20:45.996 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-01T01:20:45.997 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-01T01:20:45.998 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-12-01T01:20:45.999 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-12-01T01:20:46.000 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-01T01:20:46.001 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-12-01T01:20:46.002 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-01T01:20:46.003 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-01T01:20:46.004 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-01T01:20:46.005 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-01T01:20:46.006 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-01T01:20:46.007 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-12-01T01:20:46.008 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-12-01T01:20:46.009 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-12-01T01:20:46.010 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-12-01T01:20:46.011 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-12-01T01:20:46.012 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-01T01:20:46.013 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-01T01:20:46.014 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-12-01T01:20:46.015 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-01T01:20:46.016 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-12-01T01:20:46.017 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-12-01T01:20:46.018 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-01T01:20:46.019 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-01T01:20:46.020 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-12-01T01:20:46.021 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-12-01T01:20:46.022 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-12-01T01:20:46.023 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-12-01T01:20:46.024 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-12-01T01:20:46.025 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-01T01:20:46.026 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-12-01T01:20:46.027 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-12-01T01:20:46.028 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-01T01:20:46.029 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-12-01T01:20:46.030 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-01T01:20:46.031 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-12-01T01:20:46.032 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-01T01:20:46.033 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-12-01T01:20:46.034 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-12-01T01:20:46.035 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-12-01T01:20:46.036 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-01T01:20:46.037 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-01T01:20:46.038 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-12-01T01:20:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-12-01T01:20:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-12-01T01:20:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-12-01T01:20:46.042 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-12-01T01:20:46.043 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-12-01T01:20:46.044 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-12-01T01:20:46.045 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-12-01T01:20:46.046 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-12-01T01:20:46.047 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-12-01T01:20:46.048 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-12-01T01:20:46.049 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-12-01T01:20:46.050 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-12-01T01:20:46.051 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-12-01T01:20:46.052 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-12-01T01:20:46.053 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-01T01:20:46.054 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-12-01T01:20:46.055 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-12-01T01:20:46.056 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-12-01T01:20:46.057 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-01T01:20:46.058 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-12-01T01:20:46.059 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-12-01T01:20:46.060 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-01T01:20:46.061 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-01T01:20:46.062 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-12-01T01:20:46.063 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-01T01:20:46.064 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-01T01:20:46.065 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-01T01:20:46.066 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-12-01T01:20:46.067 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-12-01T01:20:46.068 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-12-01T01:20:46.069 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-12-01T01:20:46.070 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-12-01T01:20:46.071 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-12-01T01:20:46.072 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-12-01T01:20:46.073 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-12-01T01:20:46.074 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-12-01T01:20:46.075 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-12-01T01:20:46.076 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-12-01T01:20:46.077 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-01T01:20:46.078 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-12-01T01:20:46.079 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-12-01T01:20:46.080 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-12-01T01:20:46.081 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-12-01T01:20:46.081 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-12-01T01:20:46.081 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-12-01T01:20:46.081 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-12-01T01:20:46.081 DEBUG:teuthology.orchestra.run.smithi142:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-01T01:20:46.104 INFO:teuthology.orchestra.run.smithi142.stdout:postinstall scriptlet (using /bin/sh): 2023-12-01T01:20:46.104 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-12-01T01:20:46.104 INFO:teuthology.orchestra.run.smithi142.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-12-01T01:20:46.104 INFO:teuthology.orchestra.run.smithi142.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-12-01T01:20:46.104 INFO:teuthology.orchestra.run.smithi142.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-12-01T01:20:46.104 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-12-01T01:20:46.104 INFO:teuthology.orchestra.run.smithi142.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-12-01T01:20:46.104 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2023-12-01T01:20:46.104 INFO:teuthology.orchestra.run.smithi142.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-01T01:20:46.104 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-01T01:20:46.105 INFO:teuthology.orchestra.run.smithi142.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-12-01T01:20:46.105 INFO:teuthology.orchestra.run.smithi142.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-01T01:20:46.105 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-12-01T01:20:46.105 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2023-12-01T01:20:46.105 INFO:teuthology.orchestra.run.smithi142.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-01T01:20:46.105 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-01T01:20:46.105 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-12-01T01:20:46.105 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2023-12-01T01:20:46.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -qi grub2-tools 2023-12-01T01:20:46.181 INFO:teuthology.orchestra.run.smithi142.stdout:Name : grub2-tools 2023-12-01T01:20:46.181 INFO:teuthology.orchestra.run.smithi142.stdout:Epoch : 1 2023-12-01T01:20:46.181 INFO:teuthology.orchestra.run.smithi142.stdout:Version : 2.06 2023-12-01T01:20:46.181 INFO:teuthology.orchestra.run.smithi142.stdout:Release : 68.el9 2023-12-01T01:20:46.181 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: x86_64 2023-12-01T01:20:46.181 INFO:teuthology.orchestra.run.smithi142.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-01T01:20:46.181 INFO:teuthology.orchestra.run.smithi142.stdout:Group : Unspecified 2023-12-01T01:20:46.182 INFO:teuthology.orchestra.run.smithi142.stdout:Size : 8264254 2023-12-01T01:20:46.182 INFO:teuthology.orchestra.run.smithi142.stdout:License : GPLv3+ 2023-12-01T01:20:46.182 INFO:teuthology.orchestra.run.smithi142.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-01T01:20:46.182 INFO:teuthology.orchestra.run.smithi142.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-01T01:20:46.182 INFO:teuthology.orchestra.run.smithi142.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-01T01:20:46.182 INFO:teuthology.orchestra.run.smithi142.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-01T01:20:46.182 INFO:teuthology.orchestra.run.smithi142.stdout:Packager : builder@centos.org 2023-12-01T01:20:46.182 INFO:teuthology.orchestra.run.smithi142.stdout:Vendor : CentOS 2023-12-01T01:20:46.183 INFO:teuthology.orchestra.run.smithi142.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-01T01:20:46.183 INFO:teuthology.orchestra.run.smithi142.stdout:Summary : Support tools for GRUB. 2023-12-01T01:20:46.183 INFO:teuthology.orchestra.run.smithi142.stdout:Description : 2023-12-01T01:20:46.183 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:20:46.183 INFO:teuthology.orchestra.run.smithi142.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-01T01:20:46.184 INFO:teuthology.orchestra.run.smithi142.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-01T01:20:46.184 INFO:teuthology.orchestra.run.smithi142.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-01T01:20:46.184 INFO:teuthology.orchestra.run.smithi142.stdout:hardware devices. 2023-12-01T01:20:46.184 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:20:46.184 INFO:teuthology.orchestra.run.smithi142.stdout:This subpackage provides tools for support of all platforms. 2023-12-01T01:20:46.185 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-01T01:20:46.185 INFO:teuthology.task.kernel:Updating grub on smithi142 to boot 6.6.0-g8247bbb5b470 2023-12-01T01:20:46.185 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-01T01:20:46.756 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2023-12-01T01:20:48.137 INFO:teuthology.orchestra.run.smithi142.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-01T01:20:48.166 INFO:teuthology.orchestra.run.smithi142.stderr:done 2023-12-01T01:20:48.168 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2023-12-01T01:20:48.185 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.HofNUnBUgB 2023-12-01T01:20:48.186 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HofNUnBUgB 2023-12-01T01:20:48.254 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.HofNUnBUgB 2023-12-01T01:20:48.388 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.HofNUnBUgB is 6KB 2023-12-01T01:20:48.436 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.HofNUnBUgB 2023-12-01T01:20:48.449 DEBUG:teuthology.orchestra.run.smithi142:> sudo /bin/ls /boot/loader/entries || true 2023-12-01T01:20:48.513 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-01T01:20:48.514 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-01T01:20:48.514 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-01T01:20:48.514 INFO:teuthology.orchestra.run.smithi142.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-12-01T01:20:48.515 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-12-01T01:20:48.661 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2023-12-01T01:20:50.067 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-01T01:20:50.067 DEBUG:teuthology.orchestra.run.smithi115:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-01T01:20:50.141 INFO:teuthology.orchestra.run.smithi115.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-12-01T01:20:50.141 INFO:teuthology.orchestra.run.smithi115.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-12-01T01:20:50.141 INFO:teuthology.orchestra.run.smithi115.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-12-01T01:20:50.141 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-12-01T01:20:50.141 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-01T01:20:50.142 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-01T01:20:50.143 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-12-01T01:20:50.144 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-12-01T01:20:50.145 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-12-01T01:20:50.146 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-12-01T01:20:50.147 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-01T01:20:50.148 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-12-01T01:20:50.149 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-12-01T01:20:50.150 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-01T01:20:50.151 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-12-01T01:20:50.152 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-12-01T01:20:50.153 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-01T01:20:50.154 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-12-01T01:20:50.155 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-12-01T01:20:50.156 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-01T01:20:50.157 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-12-01T01:20:50.158 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-12-01T01:20:50.159 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-12-01T01:20:50.160 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-12-01T01:20:50.161 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-01T01:20:50.162 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-01T01:20:50.163 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-12-01T01:20:50.164 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-12-01T01:20:50.165 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-12-01T01:20:50.166 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-12-01T01:20:50.167 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-12-01T01:20:50.168 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-12-01T01:20:50.169 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-12-01T01:20:50.170 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-12-01T01:20:50.170 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-12-01T01:20:50.170 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-12-01T01:20:50.170 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-12-01T01:20:50.170 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-12-01T01:20:50.170 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-12-01T01:20:50.170 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-12-01T01:20:50.170 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-12-01T01:20:50.170 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-12-01T01:20:50.170 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-12-01T01:20:50.171 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-12-01T01:20:50.172 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-12-01T01:20:50.173 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-12-01T01:20:50.174 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-12-01T01:20:50.175 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-01T01:20:50.176 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-12-01T01:20:50.177 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-12-01T01:20:50.178 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-01T01:20:50.179 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-12-01T01:20:50.180 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-12-01T01:20:50.181 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-01T01:20:50.182 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-12-01T01:20:50.183 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-01T01:20:50.184 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-12-01T01:20:50.185 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-12-01T01:20:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-01T01:20:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-12-01T01:20:50.188 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-12-01T01:20:50.189 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-12-01T01:20:50.190 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-01T01:20:50.191 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-01T01:20:50.192 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-12-01T01:20:50.193 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-12-01T01:20:50.194 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-01T01:20:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-01T01:20:50.196 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-01T01:20:50.197 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-12-01T01:20:50.198 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-01T01:20:50.199 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-12-01T01:20:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-01T01:20:50.201 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-12-01T01:20:50.202 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-12-01T01:20:50.203 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-12-01T01:20:50.203 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-12-01T01:20:50.203 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-01T01:20:50.203 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-01T01:20:50.203 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-12-01T01:20:50.203 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-12-01T01:20:50.203 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-01T01:20:50.203 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-01T01:20:50.203 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-12-01T01:20:50.204 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-12-01T01:20:50.205 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-01T01:20:50.206 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-12-01T01:20:50.207 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-01T01:20:50.208 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-01T01:20:50.209 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-01T01:20:50.210 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-12-01T01:20:50.211 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-01T01:20:50.212 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-01T01:20:50.213 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-12-01T01:20:50.214 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-12-01T01:20:50.215 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-12-01T01:20:50.216 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-12-01T01:20:50.217 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-12-01T01:20:50.218 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-12-01T01:20:50.219 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-12-01T01:20:50.220 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-12-01T01:20:50.221 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-01T01:20:50.222 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-12-01T01:20:50.223 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-12-01T01:20:50.224 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-12-01T01:20:50.225 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-12-01T01:20:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-12-01T01:20:50.227 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-12-01T01:20:50.228 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-01T01:20:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-01T01:20:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-01T01:20:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-12-01T01:20:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-01T01:20:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-01T01:20:50.234 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-01T01:20:50.235 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-12-01T01:20:50.236 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-12-01T01:20:50.237 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-12-01T01:20:50.238 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-12-01T01:20:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-12-01T01:20:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-12-01T01:20:50.241 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-12-01T01:20:50.242 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-12-01T01:20:50.243 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-01T01:20:50.244 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-12-01T01:20:50.245 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-12-01T01:20:50.246 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-12-01T01:20:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-12-01T01:20:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-12-01T01:20:50.249 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-12-01T01:20:50.250 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-01T01:20:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-01T01:20:50.252 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-12-01T01:20:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-12-01T01:20:50.254 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-01T01:20:50.255 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-12-01T01:20:50.256 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-12-01T01:20:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-01T01:20:50.258 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-01T01:20:50.259 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-12-01T01:20:50.260 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-01T01:20:50.261 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-01T01:20:50.262 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-01T01:20:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-12-01T01:20:50.264 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-12-01T01:20:50.265 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-01T01:20:50.266 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-12-01T01:20:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-12-01T01:20:50.268 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-12-01T01:20:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-12-01T01:20:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-12-01T01:20:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-12-01T01:20:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-12-01T01:20:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-12-01T01:20:50.274 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-12-01T01:20:50.275 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-12-01T01:20:50.276 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-12-01T01:20:50.277 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-12-01T01:20:50.278 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-12-01T01:20:50.278 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-12-01T01:20:50.278 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-12-01T01:20:50.278 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-12-01T01:20:50.278 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-12-01T01:20:50.278 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-12-01T01:20:50.278 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-12-01T01:20:50.278 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-12-01T01:20:50.278 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-12-01T01:20:50.278 INFO:teuthology.orchestra.run.smithi115.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-12-01T01:20:50.279 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-12-01T01:20:50.279 DEBUG:teuthology.orchestra.run.smithi115:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-01T01:20:50.300 INFO:teuthology.orchestra.run.smithi115.stdout:postinstall scriptlet (using /bin/sh): 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:fi 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-12-01T01:20:50.301 INFO:teuthology.orchestra.run.smithi115.stdout:fi 2023-12-01T01:20:50.302 INFO:teuthology.orchestra.run.smithi115.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-01T01:20:50.302 INFO:teuthology.orchestra.run.smithi115.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-01T01:20:50.302 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-12-01T01:20:50.302 INFO:teuthology.orchestra.run.smithi115.stdout:fi 2023-12-01T01:20:50.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2023-12-01T01:20:50.388 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2023-12-01T01:20:50.388 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2023-12-01T01:20:50.388 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.06 2023-12-01T01:20:50.388 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 68.el9 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Group : Unspecified 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 8264254 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : builder@centos.org 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:20:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-01T01:20:50.390 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-01T01:20:50.390 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-01T01:20:50.390 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2023-12-01T01:20:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:20:50.390 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2023-12-01T01:20:50.391 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-01T01:20:50.391 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 6.6.0-g8247bbb5b470 2023-12-01T01:20:50.391 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-01T01:20:50.979 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2023-12-01T01:20:52.387 INFO:teuthology.orchestra.run.smithi115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-01T01:20:52.425 INFO:teuthology.orchestra.run.smithi115.stderr:done 2023-12-01T01:20:52.427 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2023-12-01T01:20:52.442 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.tjwJYD8Q8Q 2023-12-01T01:20:52.442 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.tjwJYD8Q8Q 2023-12-01T01:20:52.515 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.tjwJYD8Q8Q 2023-12-01T01:20:52.654 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.tjwJYD8Q8Q is 6KB 2023-12-01T01:20:52.702 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.tjwJYD8Q8Q 2023-12-01T01:20:52.716 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2023-12-01T01:20:52.780 INFO:teuthology.orchestra.run.smithi115.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-01T01:20:52.781 INFO:teuthology.orchestra.run.smithi115.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-01T01:20:52.781 INFO:teuthology.orchestra.run.smithi115.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-01T01:20:52.781 INFO:teuthology.orchestra.run.smithi115.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-12-01T01:20:52.782 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-12-01T01:20:52.956 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2023-12-01T01:21:18.666 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-12-01T01:21:18.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-12-01T01:21:18.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:21:22.962 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-12-01T01:21:22.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-12-01T01:21:22.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:21:37.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-12-01T01:21:41.474 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2023-12-01T01:21:46.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-12-01T01:21:46.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:21:49.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2023-12-01T01:21:50.476 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-12-01T01:21:50.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:21:53.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2023-12-01T01:22:01.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-12-01T01:22:01.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:22:05.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-12-01T01:22:05.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:22:23.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2023-12-01T01:22:33.179 DEBUG:teuthology.orchestra.run.smithi142:> true 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-12-01T01:22:33.819 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-g8247bbb5b470"... 2023-12-01T01:22:33.820 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi142.stdout:6.6.0-g8247bbb5b470 2023-12-01T01:22:33.860 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-12-01T01:22:33.860 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-01T01:22:33.860 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-01T01:22:34.862 DEBUG:teuthology.parallel:result is None 2023-12-01T01:22:38.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-12-01T01:22:38.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:22:39.423 DEBUG:teuthology.orchestra.run.smithi115:> true 2023-12-01T01:22:40.064 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-12-01T01:22:40.064 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g8247bbb5b470"... 2023-12-01T01:22:40.065 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2023-12-01T01:22:40.105 INFO:teuthology.orchestra.run.smithi115.stdout:6.6.0-g8247bbb5b470 2023-12-01T01:22:40.106 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-12-01T01:22:40.106 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-01T01:22:40.106 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-01T01:22:41.107 DEBUG:teuthology.parallel:result is None 2023-12-01T01:22:41.107 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-01T01:22:41.122 INFO:teuthology.task.internal:Creating test directory... 2023-12-01T01:22:41.122 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-01T01:22:41.132 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-01T01:22:41.183 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-01T01:22:41.190 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-01T01:22:41.198 INFO:teuthology.task.internal:Creating archive directory... 2023-12-01T01:22:41.198 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-01T01:22:41.204 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-01T01:22:41.269 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-01T01:22:41.277 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-01T01:22:41.277 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-01T01:22:41.282 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-01T01:22:41.434 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T01:22:41.436 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T01:22:41.488 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T01:22:41.501 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T01:22:41.506 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-01T01:22:41.513 INFO:teuthology.task.internal:Configuring sudo... 2023-12-01T01:22:41.513 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-01T01:22:41.518 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-01T01:22:41.602 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-01T01:22:41.611 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-01T01:22:41.612 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-01T01:22:41.617 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-01T01:22:41.660 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T01:22:41.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T01:22:41.863 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T01:22:41.935 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T01:22:42.037 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:22:42.037 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-01T01:22:42.112 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T01:22:42.194 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T01:22:42.280 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T01:22:42.352 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T01:22:42.444 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:22:42.444 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-01T01:22:42.516 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2023-12-01T01:22:42.521 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2023-12-01T01:22:42.632 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-01T01:22:42.668 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-01T01:22:42.859 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-01T01:22:42.868 INFO:teuthology.task.internal:Starting timer... 2023-12-01T01:22:42.868 INFO:teuthology.run_tasks:Running task pcp... 2023-12-01T01:22:42.880 INFO:teuthology.run_tasks:Running task selinux... 2023-12-01T01:22:42.889 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2023-12-01T01:22:43.019 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2023-12-01T01:22:43.022 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2023-12-01T01:22:43.153 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2023-12-01T01:22:43.156 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-01T01:22:43.157 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2023-12-01T01:22:43.211 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2023-12-01T01:22:43.212 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2023-12-01T01:22:43.267 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2023-12-01T01:22:43.268 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive'} 2023-12-01T01:22:43.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-01T01:22:43.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:22:43.354 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-01T01:22:43.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:22:43.432 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-01T01:22:43.433 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2023-12-01T01:22:43.513 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2023-12-01T01:22:43.614 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-01T01:22:43.624 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-01T01:22:43.624 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-01T01:22:43.647 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-01T01:22:43.648 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi115.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-01T01:32:54.350 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2023-12-01T01:32:54.373 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-12-01T01:32:54.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:32:54.518 DEBUG:teuthology.orchestra.run.smithi115:> true 2023-12-01T01:32:54.678 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2023-12-01T01:32:54.679 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-12-01T01:32:54.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:32:54.809 DEBUG:teuthology.orchestra.run.smithi142:> true 2023-12-01T01:32:54.965 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2023-12-01T01:32:54.965 INFO:teuthology.run_tasks:Running task clock... 2023-12-01T01:32:54.975 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-01T01:32:54.975 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-01T01:32:54.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T01:32:54.985 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-01T01:32:54.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T01:32:55.086 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-01T01:32:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-01T01:32:55.137 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-01T01:32:55.140 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-01T01:32:55.229 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2023-12-01T01:32:55.231 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2023-12-01T01:32:55.267 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2023-12-01T01:32:55.270 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2023-12-01T01:32:55.315 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-01T01:32:55.321 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-01T01:32:55.365 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-01T01:32:55.373 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-01T01:32:55.485 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2023-12-01T01:32:55.493 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-01T01:32:55.494 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2023-12-01T01:32:55.494 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:32:55.494 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:32:55.494 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:32:55.494 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:32:55.496 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2023-12-01T01:32:55.505 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-01T01:32:55.505 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2023-12-01T01:32:55.505 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:32:55.505 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:32:55.505 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:32:55.505 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:32:55.507 INFO:teuthology.run_tasks:Running task install... 2023-12-01T01:32:55.516 DEBUG:teuthology.task.install:project ceph 2023-12-01T01:32:55.516 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32'}} 2023-12-01T01:32:55.517 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32'} 2023-12-01T01:32:55.517 INFO:teuthology.task.install:Using flavor: default 2023-12-01T01:32:55.523 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-01T01:32:55.523 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-01T01:32:55.524 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32', 'tag': None, 'wait_for_package': False} 2023-12-01T01:32:55.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:32:55.526 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32', 'tag': None, 'wait_for_package': False} 2023-12-01T01:32:55.526 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:32:55.676 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/9/flavors/default/ 2023-12-01T01:32:55.676 INFO:teuthology.task.install.rpm:Package version is 18.2.0-1249.ga6f82997 2023-12-01T01:32:55.743 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/9/flavors/default/ 2023-12-01T01:32:55.743 INFO:teuthology.task.install.rpm:Package version is 18.2.0-1249.ga6f82997 2023-12-01T01:32:55.887 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-01T01:32:55.887 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:32:55.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-01T01:32:55.981 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-01T01:32:55.981 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:32:55.981 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-01T01:32:55.988 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-01T01:32:55.989 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6f82997ac57b70a895455dfed6256360a1e4c32/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-01T01:32:56.081 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-01T01:32:56.081 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6f82997ac57b70a895455dfed6256360a1e4c32/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-01T01:32:56.127 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-01T01:32:56.218 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-01T01:32:56.260 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-01T01:32:56.352 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2023-12-01T01:32:56.353 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-01T01:32:56.356 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2023-12-01T01:32:56.437 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2023-12-01T01:32:56.441 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2023-12-01T01:32:56.721 INFO:teuthology.orchestra.run.smithi115.stdout:73 files removed 2023-12-01T01:32:56.758 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-01T01:32:56.803 INFO:teuthology.orchestra.run.smithi142.stdout:73 files removed 2023-12-01T01:32:56.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-01T01:32:57.723 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 211 kB/s | 77 kB 00:00 2023-12-01T01:32:57.789 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 146 kB/s | 77 kB 00:00 2023-12-01T01:32:58.274 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 21 kB/s | 9.5 kB 00:00 2023-12-01T01:32:58.372 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 19 kB/s | 9.5 kB 00:00 2023-12-01T01:32:58.799 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2023-12-01T01:32:58.923 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2023-12-01T01:32:59.703 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - AppStream 7.7 MB/s | 6.4 MB 00:00 2023-12-01T01:33:00.043 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - AppStream 6.1 MB/s | 6.4 MB 00:01 2023-12-01T01:33:01.859 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2023-12-01T01:33:02.021 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS-9 - BaseOS 1.9 MB/s | 2.0 MB 00:01 2023-12-01T01:33:03.958 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 7.9 MB 00:01 2023-12-01T01:33:05.272 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 7.9 MB 00:03 2023-12-01T01:33:06.858 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-12-01T01:33:13.176 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 5.8 MB 00:02 2023-12-01T01:33:14.755 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 2.2 MB/s | 18 MB 00:08 2023-12-01T01:33:14.860 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 15 kB 00:00 2023-12-01T01:33:16.554 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2023-12-01T01:33:20.063 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 5.8 MB 00:01 2023-12-01T01:33:21.656 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 15 kB 00:00 2023-12-01T01:33:22.162 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-12-01T01:33:23.391 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2023-12-01T01:33:25.055 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-01T01:33:25.057 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-01T01:33:25.078 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-01T01:33:25.080 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-01T01:33:25.081 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-01T01:33:25.082 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-01T01:33:25.086 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-01T01:33:25.089 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-01T01:33:25.090 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-01T01:33:25.169 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2023-12-01T01:33:25.188 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================================ 2023-12-01T01:33:25.188 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2023-12-01T01:33:25.188 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================================ 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 6.5 k 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 5.2 M 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 840 k 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 142 k 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 1.4 M 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 132 k 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 1.8 M 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 7.4 M 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 50 k 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 7.7 M 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 40 M 2023-12-01T01:33:25.189 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 262 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 221 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 219 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 18 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 31 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 658 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 126 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-01T01:33:25.190 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 162 k 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 320 k 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 299 k 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 99 k 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 86 k 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 3.0 M 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 171 k 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-01T01:33:25.191 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 3.3 M 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 3.0 M 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-01T01:33:25.192 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 18 M 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 23 k 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 2.1 M 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 242 k 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 4.4 M 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 18 M 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 15 k 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 24 k 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-01T01:33:25.193 INFO:teuthology.orchestra.run.smithi142.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 165 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 475 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-01T01:33:25.194 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 4.5 M 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 45 k 2023-12-01T01:33:25.195 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 124 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-01T01:33:25.196 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-01T01:33:25.197 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-01T01:33:25.198 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================================ 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout:Install 133 Packages 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 15 Packages 2023-12-01T01:33:25.199 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:33:25.200 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 200 M 2023-12-01T01:33:25.200 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2023-12-01T01:33:25.913 INFO:teuthology.orchestra.run.smithi142.stdout:(1/148): ceph-18.2.0-1249.ga6f82997.el9.x86_64. 27 kB/s | 6.5 kB 00:00 2023-12-01T01:33:26.138 INFO:teuthology.orchestra.run.smithi142.stdout:(2/148): ceph-fuse-18.2.0-1249.ga6f82997.el9.x8 3.7 MB/s | 840 kB 00:00 2023-12-01T01:33:26.197 INFO:teuthology.orchestra.run.smithi142.stdout:(3/148): ceph-immutable-object-cache-18.2.0-124 2.4 MB/s | 142 kB 00:00 2023-12-01T01:33:26.614 INFO:teuthology.orchestra.run.smithi142.stdout:(4/148): ceph-mds-18.2.0-1249.ga6f82997.el9.x86 5.1 MB/s | 2.1 MB 00:00 2023-12-01T01:33:26.723 INFO:teuthology.orchestra.run.smithi142.stdout:(5/148): ceph-base-18.2.0-1249.ga6f82997.el9.x8 4.9 MB/s | 5.2 MB 00:01 2023-12-01T01:33:26.889 INFO:teuthology.orchestra.run.smithi142.stdout:(6/148): ceph-mgr-18.2.0-1249.ga6f82997.el9.x86 5.3 MB/s | 1.4 MB 00:00 2023-12-01T01:33:27.382 INFO:teuthology.orchestra.run.smithi142.stdout:(7/148): ceph-mon-18.2.0-1249.ga6f82997.el9.x86 6.7 MB/s | 4.4 MB 00:00 2023-12-01T01:33:28.532 INFO:teuthology.orchestra.run.smithi142.stdout:(8/148): ceph-radosgw-18.2.0-1249.ga6f82997.el9 6.7 MB/s | 7.7 MB 00:01 2023-12-01T01:33:28.573 INFO:teuthology.orchestra.run.smithi142.stdout:(9/148): ceph-selinux-18.2.0-1249.ga6f82997.el9 593 kB/s | 24 kB 00:00 2023-12-01T01:33:28.973 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2023-12-01T01:33:30.074 INFO:teuthology.orchestra.run.smithi142.stdout:(10/148): ceph-osd-18.2.0-1249.ga6f82997.el9.x8 5.5 MB/s | 18 MB 00:03 2023-12-01T01:33:30.215 INFO:teuthology.orchestra.run.smithi142.stdout:(11/148): cephfs-mirror-18.2.0-1249.ga6f82997.e 1.5 MB/s | 219 kB 00:00 2023-12-01T01:33:30.308 INFO:teuthology.orchestra.run.smithi142.stdout:(12/148): libcephfs-devel-18.2.0-1249.ga6f82997 341 kB/s | 31 kB 00:00 2023-12-01T01:33:30.483 INFO:teuthology.orchestra.run.smithi142.stdout:(13/148): libcephfs2-18.2.0-1249.ga6f82997.el9. 3.7 MB/s | 658 kB 00:00 2023-12-01T01:33:30.574 INFO:teuthology.orchestra.run.smithi142.stdout:(14/148): libcephsqlite-18.2.0-1249.ga6f82997.e 1.8 MB/s | 165 kB 00:00 2023-12-01T01:33:30.658 INFO:teuthology.orchestra.run.smithi142.stdout:(15/148): librados-devel-18.2.0-1249.ga6f82997. 1.5 MB/s | 126 kB 00:00 2023-12-01T01:33:30.825 INFO:teuthology.orchestra.run.smithi142.stdout:(16/148): libradosstriper1-18.2.0-1249.ga6f8299 2.8 MB/s | 475 kB 00:00 2023-12-01T01:33:31.534 INFO:teuthology.orchestra.run.smithi142.stdout:(17/148): librgw2-18.2.0-1249.ga6f82997.el9.x86 6.3 MB/s | 4.5 MB 00:00 2023-12-01T01:33:31.601 INFO:teuthology.orchestra.run.smithi142.stdout:(18/148): python3-ceph-argparse-18.2.0-1249.ga6 687 kB/s | 45 kB 00:00 2023-12-01T01:33:31.699 INFO:teuthology.orchestra.run.smithi142.stdout:(19/148): python3-ceph-common-18.2.0-1249.ga6f8 1.2 MB/s | 124 kB 00:00 2023-12-01T01:33:31.768 INFO:teuthology.orchestra.run.smithi142.stdout:(20/148): python3-cephfs-18.2.0-1249.ga6f82997. 2.3 MB/s | 162 kB 00:00 2023-12-01T01:33:31.843 INFO:teuthology.orchestra.run.smithi142.stdout:(21/148): python3-rados-18.2.0-1249.ga6f82997.e 4.2 MB/s | 320 kB 00:00 2023-12-01T01:33:31.919 INFO:teuthology.orchestra.run.smithi142.stdout:(22/148): python3-rbd-18.2.0-1249.ga6f82997.el9 3.9 MB/s | 299 kB 00:00 2023-12-01T01:33:32.000 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-01T01:33:32.002 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-01T01:33:32.024 INFO:teuthology.orchestra.run.smithi115.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-01T01:33:32.025 INFO:teuthology.orchestra.run.smithi115.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-01T01:33:32.027 INFO:teuthology.orchestra.run.smithi115.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-01T01:33:32.028 INFO:teuthology.orchestra.run.smithi115.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-01T01:33:32.031 INFO:teuthology.orchestra.run.smithi115.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-01T01:33:32.035 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-01T01:33:32.036 INFO:teuthology.orchestra.run.smithi115.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-01T01:33:32.120 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2023-12-01T01:33:32.160 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================ 2023-12-01T01:33:32.160 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2023-12-01T01:33:32.160 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================ 2023-12-01T01:33:32.160 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 6.5 k 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 5.2 M 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 840 k 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 142 k 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 1.4 M 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 132 k 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 1.8 M 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 7.4 M 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 50 k 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 7.7 M 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 40 M 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 262 k 2023-12-01T01:33:32.161 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 221 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 219 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 18 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 31 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 658 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 126 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-01T01:33:32.162 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 162 k 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 320 k 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 299 k 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 99 k 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 86 k 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 3.0 M 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 171 k 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-01T01:33:32.163 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 3.3 M 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 3.0 M 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 18 M 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 23 k 2023-12-01T01:33:32.164 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 2.1 M 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 242 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 4.4 M 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 18 M 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:18.2.0-1249.ga6f82997.el9 ceph-noarch 15 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 24 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 165 k 2023-12-01T01:33:32.165 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 475 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 4.5 M 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-01T01:33:32.166 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 45 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:18.2.0-1249.ga6f82997.el9 ceph 124 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-01T01:33:32.167 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-01T01:33:32.168 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-01T01:33:32.169 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-01T01:33:32.170 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-01T01:33:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-01T01:33:32.171 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2023-12-01T01:33:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-01T01:33:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:33:32.171 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2023-12-01T01:33:32.171 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================================ 2023-12-01T01:33:32.171 INFO:teuthology.orchestra.run.smithi115.stdout:Install 133 Packages 2023-12-01T01:33:32.171 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 15 Packages 2023-12-01T01:33:32.171 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:33:32.171 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 200 M 2023-12-01T01:33:32.172 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2023-12-01T01:33:32.419 INFO:teuthology.orchestra.run.smithi142.stdout:(23/148): ceph-test-18.2.0-1249.ga6f82997.el9.x 10 MB/s | 40 MB 00:03 2023-12-01T01:33:32.453 INFO:teuthology.orchestra.run.smithi142.stdout:(24/148): python3-rgw-18.2.0-1249.ga6f82997.el9 187 kB/s | 99 kB 00:00 2023-12-01T01:33:32.478 INFO:teuthology.orchestra.run.smithi142.stdout:(25/148): rbd-fuse-18.2.0-1249.ga6f82997.el9.x8 1.5 MB/s | 86 kB 00:00 2023-12-01T01:33:32.528 INFO:teuthology.orchestra.run.smithi142.stdout:(26/148): rbd-nbd-18.2.0-1249.ga6f82997.el9.x86 3.4 MB/s | 171 kB 00:00 2023-12-01T01:33:32.578 INFO:teuthology.orchestra.run.smithi142.stdout:(27/148): ceph-grafana-dashboards-18.2.0-1249.g 472 kB/s | 23 kB 00:00 2023-12-01T01:33:32.646 INFO:teuthology.orchestra.run.smithi142.stdout:(28/148): ceph-mgr-cephadm-18.2.0-1249.ga6f8299 2.0 MB/s | 132 kB 00:00 2023-12-01T01:33:32.929 INFO:teuthology.orchestra.run.smithi142.stdout:(29/148): ceph-mgr-dashboard-18.2.0-1249.ga6f82 6.2 MB/s | 1.8 MB 00:00 2023-12-01T01:33:33.013 INFO:teuthology.orchestra.run.smithi142.stdout:(30/148): rbd-mirror-18.2.0-1249.ga6f82997.el9. 5.3 MB/s | 3.0 MB 00:00 2023-12-01T01:33:33.088 INFO:teuthology.orchestra.run.smithi142.stdout:(31/148): ceph-mgr-modules-core-18.2.0-1249.ga6 3.2 MB/s | 242 kB 00:00 2023-12-01T01:33:33.116 INFO:teuthology.orchestra.run.smithi115.stdout:(1/148): ceph-18.2.0-1249.ga6f82997.el9.x86_64. 18 kB/s | 6.5 kB 00:00 2023-12-01T01:33:33.155 INFO:teuthology.orchestra.run.smithi142.stdout:(32/148): ceph-mgr-rook-18.2.0-1249.ga6f82997.e 765 kB/s | 50 kB 00:00 2023-12-01T01:33:33.222 INFO:teuthology.orchestra.run.smithi142.stdout:(33/148): ceph-prometheus-alerts-18.2.0-1249.ga 223 kB/s | 15 kB 00:00 2023-12-01T01:33:33.305 INFO:teuthology.orchestra.run.smithi142.stdout:(34/148): ceph-volume-18.2.0-1249.ga6f82997.el9 3.1 MB/s | 262 kB 00:00 2023-12-01T01:33:33.381 INFO:teuthology.orchestra.run.smithi142.stdout:(35/148): cephadm-18.2.0-1249.ga6f82997.el9.noa 2.9 MB/s | 221 kB 00:00 2023-12-01T01:33:33.448 INFO:teuthology.orchestra.run.smithi142.stdout:(36/148): cephfs-top-18.2.0-1249.ga6f82997.el9. 275 kB/s | 18 kB 00:00 2023-12-01T01:33:33.590 INFO:teuthology.orchestra.run.smithi142.stdout:(37/148): ceph-mgr-diskprediction-local-18.2.0- 11 MB/s | 7.4 MB 00:00 2023-12-01T01:33:33.875 INFO:teuthology.orchestra.run.smithi115.stdout:(2/148): ceph-base-18.2.0-1249.ga6f82997.el9.x8 4.7 MB/s | 5.2 MB 00:01 2023-12-01T01:33:33.909 INFO:teuthology.orchestra.run.smithi115.stdout:(3/148): ceph-fuse-18.2.0-1249.ga6f82997.el9.x8 1.0 MB/s | 840 kB 00:00 2023-12-01T01:33:33.933 INFO:teuthology.orchestra.run.smithi115.stdout:(4/148): ceph-immutable-object-cache-18.2.0-124 2.4 MB/s | 142 kB 00:00 2023-12-01T01:33:34.040 INFO:teuthology.orchestra.run.smithi142.stdout:(38/148): ceph-common-18.2.0-1249.ga6f82997.el9 2.2 MB/s | 18 MB 00:08 2023-12-01T01:33:34.075 INFO:teuthology.orchestra.run.smithi115.stdout:(5/148): ceph-mgr-18.2.0-1249.ga6f82997.el9.x86 10 MB/s | 1.4 MB 00:00 2023-12-01T01:33:34.165 INFO:teuthology.orchestra.run.smithi142.stdout:(39/148): automake-1.16.2-8.el9.noarch.rpm 924 kB/s | 662 kB 00:00 2023-12-01T01:33:34.232 INFO:teuthology.orchestra.run.smithi142.stdout:(40/148): boost-program-options-1.75.0-8.el9.x8 567 kB/s | 106 kB 00:00 2023-12-01T01:33:34.258 INFO:teuthology.orchestra.run.smithi142.stdout:(41/148): elfutils-libelf-devel-0.190-1.el9.x86 239 kB/s | 21 kB 00:00 2023-12-01T01:33:34.366 INFO:teuthology.orchestra.run.smithi142.stdout:(42/148): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 307 kB 00:00 2023-12-01T01:33:34.367 INFO:teuthology.orchestra.run.smithi115.stdout:(6/148): ceph-mon-18.2.0-1249.ga6f82997.el9.x86 15 MB/s | 4.4 MB 00:00 2023-12-01T01:33:34.392 INFO:teuthology.orchestra.run.smithi142.stdout:(43/148): flexiblas-3.0.4-8.el9.x86_64.rpm 238 kB/s | 32 kB 00:00 2023-12-01T01:33:34.725 INFO:teuthology.orchestra.run.smithi142.stdout:(44/148): flexiblas-openblas-openmp-3.0.4-8.el9 52 kB/s | 17 kB 00:00 2023-12-01T01:33:34.817 INFO:teuthology.orchestra.run.smithi142.stdout:(45/148): bison-3.7.4-5.el9.x86_64.rpm 794 kB/s | 920 kB 00:01 2023-12-01T01:33:34.884 INFO:teuthology.orchestra.run.smithi115.stdout:(7/148): ceph-mds-18.2.0-1249.ga6f82997.el9.x86 2.2 MB/s | 2.1 MB 00:00 2023-12-01T01:33:35.009 INFO:teuthology.orchestra.run.smithi142.stdout:(46/148): jq-1.6-16.el9.x86_64.rpm 663 kB/s | 186 kB 00:00 2023-12-01T01:33:35.209 INFO:teuthology.orchestra.run.smithi142.stdout:(47/148): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.6 MB/s | 3.0 MB 00:00 2023-12-01T01:33:35.234 INFO:teuthology.orchestra.run.smithi142.stdout:(48/148): libaio-devel-0.3.111-13.el9.x86_64.rp 24 kB/s | 10 kB 00:00 2023-12-01T01:33:35.268 INFO:teuthology.orchestra.run.smithi142.stdout:(49/148): libpmemobj-1.12.1-1.el9.x86_64.rpm 620 kB/s | 159 kB 00:00 2023-12-01T01:33:35.301 INFO:teuthology.orchestra.run.smithi115.stdout:(8/148): ceph-osd-18.2.0-1249.ga6f82997.el9.x86 19 MB/s | 18 MB 00:00 2023-12-01T01:33:35.301 INFO:teuthology.orchestra.run.smithi142.stdout:(50/148): librabbitmq-0.11.0-7.el9.x86_64.rpm 685 kB/s | 44 kB 00:00 2023-12-01T01:33:35.343 INFO:teuthology.orchestra.run.smithi115.stdout:(9/148): ceph-selinux-18.2.0-1249.ga6f82997.el9 596 kB/s | 24 kB 00:00 2023-12-01T01:33:35.418 INFO:teuthology.orchestra.run.smithi142.stdout:(51/148): librdkafka-1.6.1-102.el9.x86_64.rpm 3.6 MB/s | 661 kB 00:00 2023-12-01T01:33:35.518 INFO:teuthology.orchestra.run.smithi142.stdout:(52/148): libtool-2.4.6-45.el9.x86_64.rpm 2.3 MB/s | 578 kB 00:00 2023-12-01T01:33:35.536 INFO:teuthology.orchestra.run.smithi142.stdout:(53/148): libuuid-devel-2.37.4-15.el9.x86_64.rp 105 kB/s | 24 kB 00:00 2023-12-01T01:33:35.552 INFO:teuthology.orchestra.run.smithi142.stdout:(54/148): libzstd-devel-1.5.1-2.el9.x86_64.rpm 352 kB/s | 46 kB 00:00 2023-12-01T01:33:35.639 INFO:teuthology.orchestra.run.smithi142.stdout:(55/148): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 282 kB 00:00 2023-12-01T01:33:35.661 INFO:teuthology.orchestra.run.smithi142.stdout:(56/148): openblas-0.3.21-2.el9.x86_64.rpm 321 kB/s | 34 kB 00:00 2023-12-01T01:33:35.695 INFO:teuthology.orchestra.run.smithi142.stdout:(57/148): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.5 MB/s | 217 kB 00:00 2023-12-01T01:33:36.070 INFO:teuthology.orchestra.run.smithi142.stdout:(58/148): postgresql-contrib-13.11-1.el9.x86_64 2.1 MB/s | 811 kB 00:00 2023-12-01T01:33:36.179 INFO:teuthology.orchestra.run.smithi142.stdout:(59/148): postgresql-13.11-1.el9.x86_64.rpm 3.0 MB/s | 1.5 MB 00:00 2023-12-01T01:33:36.312 INFO:teuthology.orchestra.run.smithi142.stdout:(60/148): openblas-openmp-0.3.21-2.el9.x86_64.r 7.1 MB/s | 4.7 MB 00:00 2023-12-01T01:33:36.355 INFO:teuthology.orchestra.run.smithi142.stdout:(61/148): postgresql-private-libs-13.11-1.el9.x 478 kB/s | 134 kB 00:00 2023-12-01T01:33:36.569 INFO:teuthology.orchestra.run.smithi142.stdout:(62/148): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2023-12-01T01:33:36.630 INFO:teuthology.orchestra.run.smithi142.stdout:(63/148): python3-jinja2-2.11.3-4.el9.noarch.rp 872 kB/s | 228 kB 00:00 2023-12-01T01:33:36.755 INFO:teuthology.orchestra.run.smithi142.stdout:(64/148): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2023-12-01T01:33:36.780 INFO:teuthology.orchestra.run.smithi142.stdout:(65/148): python3-jsonpatch-1.21-16.el9.noarch. 123 kB/s | 24 kB 00:00 2023-12-01T01:33:36.806 INFO:teuthology.orchestra.run.smithi142.stdout:(66/148): python3-jsonpointer-2.0-4.el9.noarch. 112 kB/s | 18 kB 00:00 2023-12-01T01:33:36.848 INFO:teuthology.orchestra.run.smithi142.stdout:(67/148): python3-mako-1.1.4-6.el9.noarch.rpm 3.5 MB/s | 144 kB 00:00 2023-12-01T01:33:36.873 INFO:teuthology.orchestra.run.smithi142.stdout:(68/148): python3-markupsafe-1.1.1-12.el9.x86_6 485 kB/s | 32 kB 00:00 2023-12-01T01:33:37.018 INFO:teuthology.orchestra.run.smithi115.stdout:(10/148): ceph-radosgw-18.2.0-1249.ga6f82997.el 3.6 MB/s | 7.7 MB 00:02 2023-12-01T01:33:37.065 INFO:teuthology.orchestra.run.smithi142.stdout:(69/148): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2023-12-01T01:33:37.085 INFO:teuthology.orchestra.run.smithi115.stdout:(11/148): cephfs-mirror-18.2.0-1249.ga6f82997.e 3.3 MB/s | 219 kB 00:00 2023-12-01T01:33:37.098 INFO:teuthology.orchestra.run.smithi142.stdout:(70/148): python3-oauthlib-3.1.1-5.el9.noarch.r 828 kB/s | 174 kB 00:00 2023-12-01T01:33:37.132 INFO:teuthology.orchestra.run.smithi142.stdout:(71/148): python3-numpy-f2py-1.20.1-5.el9.x86_6 890 kB/s | 217 kB 00:00 2023-12-01T01:33:37.157 INFO:teuthology.orchestra.run.smithi142.stdout:(72/148): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 41 kB 00:00 2023-12-01T01:33:37.182 INFO:teuthology.orchestra.run.smithi142.stdout:(73/148): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2023-12-01T01:33:37.208 INFO:teuthology.orchestra.run.smithi142.stdout:(74/148): python3-pytz-2021.1-5.el9.noarch.rpm 990 kB/s | 47 kB 00:00 2023-12-01T01:33:37.241 INFO:teuthology.orchestra.run.smithi142.stdout:(75/148): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2023-12-01T01:33:37.333 INFO:teuthology.orchestra.run.smithi142.stdout:(76/148): python3-toml-0.10.2-6.el9.noarch.rpm 423 kB/s | 39 kB 00:00 2023-12-01T01:33:37.375 INFO:teuthology.orchestra.run.smithi142.stdout:(77/148): python3-requests-oauthlib-1.3.0-12.el 223 kB/s | 43 kB 00:00 2023-12-01T01:33:37.539 INFO:teuthology.orchestra.run.smithi115.stdout:(12/148): ceph-test-18.2.0-1249.ga6f82997.el9.x 18 MB/s | 40 MB 00:02 2023-12-01T01:33:37.585 INFO:teuthology.orchestra.run.smithi115.stdout:(13/148): libcephfs-devel-18.2.0-1249.ga6f82997 62 kB/s | 31 kB 00:00 2023-12-01T01:33:37.642 INFO:teuthology.orchestra.run.smithi142.stdout:(78/148): uuid-1.6.2-55.el9.x86_64.rpm 223 kB/s | 56 kB 00:00 2023-12-01T01:33:37.652 INFO:teuthology.orchestra.run.smithi115.stdout:(14/148): libcephfs2-18.2.0-1249.ga6f82997.el9. 5.7 MB/s | 658 kB 00:00 2023-12-01T01:33:37.676 INFO:teuthology.orchestra.run.smithi142.stdout:(79/148): socat-1.7.4.1-5.el9.x86_64.rpm 916 kB/s | 300 kB 00:00 2023-12-01T01:33:37.711 INFO:teuthology.orchestra.run.smithi115.stdout:(15/148): librados-devel-18.2.0-1249.ga6f82997. 2.1 MB/s | 126 kB 00:00 2023-12-01T01:33:37.744 INFO:teuthology.orchestra.run.smithi115.stdout:(16/148): libcephsqlite-18.2.0-1249.ga6f82997.e 1.0 MB/s | 165 kB 00:00 2023-12-01T01:33:37.786 INFO:teuthology.orchestra.run.smithi115.stdout:(17/148): libradosstriper1-18.2.0-1249.ga6f8299 6.2 MB/s | 475 kB 00:00 2023-12-01T01:33:37.870 INFO:teuthology.orchestra.run.smithi115.stdout:(18/148): python3-ceph-argparse-18.2.0-1249.ga6 546 kB/s | 45 kB 00:00 2023-12-01T01:33:37.884 INFO:teuthology.orchestra.run.smithi142.stdout:(80/148): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2023-12-01T01:33:37.909 INFO:teuthology.orchestra.run.smithi142.stdout:(81/148): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 323 kB/s | 85 kB 00:00 2023-12-01T01:33:37.935 INFO:teuthology.orchestra.run.smithi142.stdout:(82/148): xmlstarlet-1.6.1-20.el9.x86_64.rpm 256 kB/s | 63 kB 00:00 2023-12-01T01:33:37.937 INFO:teuthology.orchestra.run.smithi115.stdout:(19/148): python3-ceph-common-18.2.0-1249.ga6f8 1.9 MB/s | 124 kB 00:00 2023-12-01T01:33:37.985 INFO:teuthology.orchestra.run.smithi142.stdout:(83/148): zlib-devel-1.2.11-40.el9.x86_64.rpm 598 kB/s | 44 kB 00:00 2023-12-01T01:33:38.010 INFO:teuthology.orchestra.run.smithi142.stdout:(84/148): fuse-2.9.9-15.el9.x86_64.rpm 797 kB/s | 79 kB 00:00 2023-12-01T01:33:38.012 INFO:teuthology.orchestra.run.smithi115.stdout:(20/148): python3-cephfs-18.2.0-1249.ga6f82997. 2.1 MB/s | 162 kB 00:00 2023-12-01T01:33:38.052 INFO:teuthology.orchestra.run.smithi142.stdout:(85/148): libgfortran-11.4.1-2.3.el9.x86_64.rpm 6.8 MB/s | 803 kB 00:00 2023-12-01T01:33:38.086 INFO:teuthology.orchestra.run.smithi142.stdout:(86/148): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.8 MB/s | 195 kB 00:00 2023-12-01T01:33:38.096 INFO:teuthology.orchestra.run.smithi115.stdout:(21/148): python3-rados-18.2.0-1249.ga6f82997.e 3.8 MB/s | 320 kB 00:00 2023-12-01T01:33:38.111 INFO:teuthology.orchestra.run.smithi142.stdout:(87/148): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 241 kB 00:00 2023-12-01T01:33:38.170 INFO:teuthology.orchestra.run.smithi142.stdout:(88/148): python3-cryptography-36.0.1-4.el9.x86 9.9 MB/s | 1.1 MB 00:00 2023-12-01T01:33:38.171 INFO:teuthology.orchestra.run.smithi115.stdout:(22/148): python3-rbd-18.2.0-1249.ga6f82997.el9 3.9 MB/s | 299 kB 00:00 2023-12-01T01:33:38.195 INFO:teuthology.orchestra.run.smithi142.stdout:(89/148): python3-pycparser-2.20-6.el9.noarch.r 1.7 MB/s | 124 kB 00:00 2023-12-01T01:33:38.220 INFO:teuthology.orchestra.run.smithi142.stdout:(90/148): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2023-12-01T01:33:38.245 INFO:teuthology.orchestra.run.smithi142.stdout:(91/148): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-12-01T01:33:38.246 INFO:teuthology.orchestra.run.smithi115.stdout:(23/148): python3-rgw-18.2.0-1249.ga6f82997.el9 1.3 MB/s | 99 kB 00:00 2023-12-01T01:33:38.279 INFO:teuthology.orchestra.run.smithi142.stdout:(92/148): python3-urllib3-1.26.5-4.el9.noarch.r 2.3 MB/s | 187 kB 00:00 2023-12-01T01:33:38.330 INFO:teuthology.orchestra.run.smithi115.stdout:(24/148): rbd-fuse-18.2.0-1249.ga6f82997.el9.x8 1.0 MB/s | 86 kB 00:00 2023-12-01T01:33:38.672 INFO:teuthology.orchestra.run.smithi115.stdout:(25/148): ceph-common-18.2.0-1249.ga6f82997.el9 3.1 MB/s | 18 MB 00:05 2023-12-01T01:33:38.730 INFO:teuthology.orchestra.run.smithi115.stdout:(26/148): rbd-nbd-18.2.0-1249.ga6f82997.el9.x86 2.9 MB/s | 171 kB 00:00 2023-12-01T01:33:38.831 INFO:teuthology.orchestra.run.smithi115.stdout:(27/148): librgw2-18.2.0-1249.ga6f82997.el9.x86 4.1 MB/s | 4.5 MB 00:01 2023-12-01T01:33:38.881 INFO:teuthology.orchestra.run.smithi115.stdout:(28/148): ceph-grafana-dashboards-18.2.0-1249.g 155 kB/s | 23 kB 00:00 2023-12-01T01:33:38.914 INFO:teuthology.orchestra.run.smithi115.stdout:(29/148): ceph-mgr-cephadm-18.2.0-1249.ga6f8299 1.6 MB/s | 132 kB 00:00 2023-12-01T01:33:38.979 INFO:teuthology.orchestra.run.smithi142.stdout:(93/148): fmt-8.1.1-5.el9.x86_64.rpm 158 kB/s | 111 kB 00:00 2023-12-01T01:33:38.998 INFO:teuthology.orchestra.run.smithi115.stdout:(30/148): rbd-mirror-18.2.0-1249.ga6f82997.el9. 4.5 MB/s | 3.0 MB 00:00 2023-12-01T01:33:39.073 INFO:teuthology.orchestra.run.smithi115.stdout:(31/148): ceph-mgr-dashboard-18.2.0-1249.ga6f82 9.2 MB/s | 1.8 MB 00:00 2023-12-01T01:33:39.107 INFO:teuthology.orchestra.run.smithi115.stdout:(32/148): ceph-mgr-modules-core-18.2.0-1249.ga6 2.2 MB/s | 242 kB 00:00 2023-12-01T01:33:39.113 INFO:teuthology.orchestra.run.smithi142.stdout:(94/148): python3-devel-3.9.17-2.el9.x86_64.rpm 284 kB/s | 245 kB 00:00 2023-12-01T01:33:39.149 INFO:teuthology.orchestra.run.smithi115.stdout:(33/148): ceph-mgr-rook-18.2.0-1249.ga6f82997.e 675 kB/s | 50 kB 00:00 2023-12-01T01:33:39.191 INFO:teuthology.orchestra.run.smithi115.stdout:(34/148): ceph-prometheus-alerts-18.2.0-1249.ga 177 kB/s | 15 kB 00:00 2023-12-01T01:33:39.224 INFO:teuthology.orchestra.run.smithi115.stdout:(35/148): ceph-volume-18.2.0-1249.ga6f82997.el9 3.5 MB/s | 262 kB 00:00 2023-12-01T01:33:39.258 INFO:teuthology.orchestra.run.smithi115.stdout:(36/148): cephadm-18.2.0-1249.ga6f82997.el9.noa 3.3 MB/s | 221 kB 00:00 2023-12-01T01:33:39.300 INFO:teuthology.orchestra.run.smithi115.stdout:(37/148): cephfs-top-18.2.0-1249.ga6f82997.el9. 243 kB/s | 18 kB 00:00 2023-12-01T01:33:39.304 INFO:teuthology.orchestra.run.smithi142.stdout:(95/148): gperftools-libs-2.9.1-2.el9.x86_64.rp 952 kB/s | 309 kB 00:00 2023-12-01T01:33:39.430 INFO:teuthology.orchestra.run.smithi142.stdout:(96/148): libarrow-doc-9.0.0-9.el9.noarch.rpm 195 kB/s | 24 kB 00:00 2023-12-01T01:33:39.542 INFO:teuthology.orchestra.run.smithi115.stdout:(38/148): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 920 kB 00:00 2023-12-01T01:33:39.564 INFO:teuthology.orchestra.run.smithi142.stdout:(97/148): liboath-2.6.7-2.el9.x86_64.rpm 366 kB/s | 49 kB 00:00 2023-12-01T01:33:39.575 INFO:teuthology.orchestra.run.smithi115.stdout:(39/148): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 662 kB 00:00 2023-12-01T01:33:39.601 INFO:teuthology.orchestra.run.smithi115.stdout:(40/148): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2023-12-01T01:33:39.643 INFO:teuthology.orchestra.run.smithi115.stdout:(41/148): elfutils-libelf-devel-0.190-1.el9.x86 330 kB/s | 21 kB 00:00 2023-12-01T01:33:39.697 INFO:teuthology.orchestra.run.smithi142.stdout:(98/148): libunwind-1.6.2-1.el9.x86_64.rpm 508 kB/s | 67 kB 00:00 2023-12-01T01:33:39.698 INFO:teuthology.orchestra.run.smithi115.stdout:(42/148): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 307 kB 00:00 2023-12-01T01:33:39.752 INFO:teuthology.orchestra.run.smithi115.stdout:(43/148): flexiblas-3.0.4-8.el9.x86_64.rpm 300 kB/s | 32 kB 00:00 2023-12-01T01:33:39.869 INFO:teuthology.orchestra.run.smithi115.stdout:(44/148): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2023-12-01T01:33:39.911 INFO:teuthology.orchestra.run.smithi115.stdout:(45/148): flexiblas-openblas-openmp-3.0.4-8.el9 113 kB/s | 17 kB 00:00 2023-12-01T01:33:39.969 INFO:teuthology.orchestra.run.smithi115.stdout:(46/148): libaio-devel-0.3.111-13.el9.x86_64.rp 176 kB/s | 10 kB 00:00 2023-12-01T01:33:39.997 INFO:teuthology.orchestra.run.smithi142.stdout:(99/148): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.7 MB/s | 837 kB 00:00 2023-12-01T01:33:40.003 INFO:teuthology.orchestra.run.smithi115.stdout:(47/148): jq-1.6-16.el9.x86_64.rpm 2.0 MB/s | 186 kB 00:00 2023-12-01T01:33:40.028 INFO:teuthology.orchestra.run.smithi115.stdout:(48/148): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.8 MB/s | 159 kB 00:00 2023-12-01T01:33:40.070 INFO:teuthology.orchestra.run.smithi115.stdout:(49/148): librabbitmq-0.11.0-7.el9.x86_64.rpm 682 kB/s | 44 kB 00:00 2023-12-01T01:33:40.112 INFO:teuthology.orchestra.run.smithi115.stdout:(50/148): librdkafka-1.6.1-102.el9.x86_64.rpm 7.9 MB/s | 661 kB 00:00 2023-12-01T01:33:40.154 INFO:teuthology.orchestra.run.smithi115.stdout:(51/148): libtool-2.4.6-45.el9.x86_64.rpm 7.3 MB/s | 578 kB 00:00 2023-12-01T01:33:40.173 INFO:teuthology.orchestra.run.smithi142.stdout:(100/148): python3-asyncssh-2.13.2-3.el9.noarch 3.1 MB/s | 548 kB 00:00 2023-12-01T01:33:40.196 INFO:teuthology.orchestra.run.smithi115.stdout:(52/148): libuuid-devel-2.37.4-15.el9.x86_64.rp 314 kB/s | 24 kB 00:00 2023-12-01T01:33:40.238 INFO:teuthology.orchestra.run.smithi115.stdout:(53/148): libzstd-devel-1.5.1-2.el9.x86_64.rpm 560 kB/s | 46 kB 00:00 2023-12-01T01:33:40.271 INFO:teuthology.orchestra.run.smithi115.stdout:(54/148): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2023-12-01T01:33:40.305 INFO:teuthology.orchestra.run.smithi115.stdout:(55/148): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-12-01T01:33:40.340 INFO:teuthology.orchestra.run.smithi142.stdout:(101/148): openssl-devel-3.0.7-25.el9.x86_64.rp 2.0 MB/s | 4.1 MB 00:02 2023-12-01T01:33:40.347 INFO:teuthology.orchestra.run.smithi115.stdout:(56/148): openblas-0.3.21-2.el9.x86_64.rpm 455 kB/s | 34 kB 00:00 2023-12-01T01:33:40.365 INFO:teuthology.orchestra.run.smithi142.stdout:(102/148): python3-bcrypt-3.2.2-1.el9.x86_64.rp 227 kB/s | 43 kB 00:00 2023-12-01T01:33:40.465 INFO:teuthology.orchestra.run.smithi142.stdout:(103/148): libarrow-9.0.0-9.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2023-12-01T01:33:40.489 INFO:teuthology.orchestra.run.smithi115.stdout:(57/148): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2023-12-01T01:33:40.556 INFO:teuthology.orchestra.run.smithi115.stdout:(58/148): postgresql-13.11-1.el9.x86_64.rpm 7.8 MB/s | 1.5 MB 00:00 2023-12-01T01:33:40.557 INFO:teuthology.orchestra.run.smithi142.stdout:(104/148): python3-certifi-2021.10.8-2.el9.noar 78 kB/s | 15 kB 00:00 2023-12-01T01:33:40.598 INFO:teuthology.orchestra.run.smithi115.stdout:(59/148): postgresql-contrib-13.11-1.el9.x86_64 8.3 MB/s | 811 kB 00:00 2023-12-01T01:33:40.623 INFO:teuthology.orchestra.run.smithi115.stdout:(60/148): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2023-12-01T01:33:40.691 INFO:teuthology.orchestra.run.smithi142.stdout:(105/148): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2023-12-01T01:33:40.725 INFO:teuthology.orchestra.run.smithi142.stdout:(106/148): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2023-12-01T01:33:40.757 INFO:teuthology.orchestra.run.smithi115.stdout:(61/148): ceph-mgr-diskprediction-local-18.2.0- 4.0 MB/s | 7.4 MB 00:01 2023-12-01T01:33:40.833 INFO:teuthology.orchestra.run.smithi142.stdout:(107/148): python3-google-auth-2.23.4-1.el9.noa 1.4 MB/s | 197 kB 00:00 2023-12-01T01:33:40.858 INFO:teuthology.orchestra.run.smithi142.stdout:(108/148): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2023-12-01T01:33:40.884 INFO:teuthology.orchestra.run.smithi142.stdout:(109/148): python3-cachetools-4.2.4-1.el9.noarc 59 kB/s | 32 kB 00:00 2023-12-01T01:33:40.901 INFO:teuthology.orchestra.run.smithi115.stdout:(62/148): postgresql-server-13.11-1.el9.x86_64. 19 MB/s | 5.7 MB 00:00 2023-12-01T01:33:40.984 INFO:teuthology.orchestra.run.smithi142.stdout:(110/148): python3-jaraco-classes-3.2.1-5.el9.n 118 kB/s | 18 kB 00:00 2023-12-01T01:33:41.009 INFO:teuthology.orchestra.run.smithi142.stdout:(111/148): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2023-12-01T01:33:41.032 INFO:teuthology.orchestra.run.smithi115.stdout:(63/148): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 5.8 MB 00:00 2023-12-01T01:33:41.034 INFO:teuthology.orchestra.run.smithi142.stdout:(112/148): python3-jaraco-functools-3.5.0-2.el9 130 kB/s | 19 kB 00:00 2023-12-01T01:33:41.057 INFO:teuthology.orchestra.run.smithi115.stdout:(64/148): python3-jinja2-2.11.3-4.el9.noarch.rp 796 kB/s | 228 kB 00:00 2023-12-01T01:33:41.107 INFO:teuthology.orchestra.run.smithi115.stdout:(65/148): python3-jsonpatch-1.21-16.el9.noarch. 125 kB/s | 24 kB 00:00 2023-12-01T01:33:41.126 INFO:teuthology.orchestra.run.smithi142.stdout:(113/148): python3-jwt+crypto-2.4.0-1.el9.noarc 78 kB/s | 9.0 kB 00:00 2023-12-01T01:33:41.152 INFO:teuthology.orchestra.run.smithi142.stdout:(114/148): python3-jaraco-text-3.2.0-6.el9.noar 118 kB/s | 20 kB 00:00 2023-12-01T01:33:41.183 INFO:teuthology.orchestra.run.smithi115.stdout:(66/148): python3-jsonpointer-2.0-4.el9.noarch. 246 kB/s | 18 kB 00:00 2023-12-01T01:33:41.233 INFO:teuthology.orchestra.run.smithi115.stdout:(67/148): python3-markupsafe-1.1.1-12.el9.x86_6 256 kB/s | 32 kB 00:00 2023-12-01T01:33:41.244 INFO:teuthology.orchestra.run.smithi142.stdout:(115/148): python3-jwt-2.4.0-1.el9.noarch.rpm 196 kB/s | 41 kB 00:00 2023-12-01T01:33:41.275 INFO:teuthology.orchestra.run.smithi115.stdout:(68/148): python3-mako-1.1.4-6.el9.noarch.rpm 872 kB/s | 144 kB 00:00 2023-12-01T01:33:41.302 INFO:teuthology.orchestra.run.smithi142.stdout:(116/148): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2023-12-01T01:33:41.328 INFO:teuthology.orchestra.run.smithi142.stdout:(117/148): python3-logutils-0.3.5-21.el9.noarch 265 kB/s | 46 kB 00:00 2023-12-01T01:33:41.400 INFO:teuthology.orchestra.run.smithi115.stdout:(69/148): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2023-12-01T01:33:41.425 INFO:teuthology.orchestra.run.smithi115.stdout:(70/148): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2023-12-01T01:33:41.427 INFO:teuthology.orchestra.run.smithi142.stdout:(118/148): python3-natsort-7.1.1-5.el9.noarch.r 464 kB/s | 58 kB 00:00 2023-12-01T01:33:41.459 INFO:teuthology.orchestra.run.smithi115.stdout:(71/148): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2023-12-01T01:33:41.486 INFO:teuthology.orchestra.run.smithi142.stdout:(119/148): python3-more-itertools-8.12.0-2.el9. 326 kB/s | 79 kB 00:00 2023-12-01T01:33:41.521 INFO:teuthology.orchestra.run.smithi142.stdout:(120/148): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2023-12-01T01:33:41.570 INFO:teuthology.orchestra.run.smithi142.stdout:(121/148): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2023-12-01T01:33:41.645 INFO:teuthology.orchestra.run.smithi142.stdout:(122/148): python3-pyOpenSSL-21.0.0-1.el9.noarc 569 kB/s | 90 kB 00:00 2023-12-01T01:33:41.651 INFO:teuthology.orchestra.run.smithi115.stdout:(72/148): python3-prettytable-0.7.2-27.el9.noar 185 kB/s | 41 kB 00:00 2023-12-01T01:33:41.676 INFO:teuthology.orchestra.run.smithi115.stdout:(73/148): python3-pyasn1-0.4.8-6.el9.noarch.rpm 534 kB/s | 132 kB 00:00 2023-12-01T01:33:41.691 INFO:teuthology.orchestra.run.smithi142.stdout:(123/148): python3-repoze-lru-0.7-16.el9.noarch 180 kB/s | 31 kB 00:00 2023-12-01T01:33:41.710 INFO:teuthology.orchestra.run.smithi115.stdout:(74/148): python3-pyasn1-modules-0.4.8-6.el9.no 844 kB/s | 211 kB 00:00 2023-12-01T01:33:41.729 INFO:teuthology.orchestra.run.smithi142.stdout:(124/148): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2023-12-01T01:33:41.752 INFO:teuthology.orchestra.run.smithi115.stdout:(75/148): python3-pytz-2021.1-5.el9.noarch.rpm 650 kB/s | 47 kB 00:00 2023-12-01T01:33:41.793 INFO:teuthology.orchestra.run.smithi115.stdout:(76/148): python3-requests-oauthlib-1.3.0-12.el 374 kB/s | 43 kB 00:00 2023-12-01T01:33:41.796 INFO:teuthology.orchestra.run.smithi142.stdout:(125/148): python3-rsa-4.9-2.el9.noarch.rpm 394 kB/s | 59 kB 00:00 2023-12-01T01:33:41.830 INFO:teuthology.orchestra.run.smithi142.stdout:(126/148): python3-tempora-5.0.0-2.el9.noarch.r 259 kB/s | 36 kB 00:00 2023-12-01T01:33:41.855 INFO:teuthology.orchestra.run.smithi142.stdout:(127/148): python3-typing-extensions-4.4.0-2.el 408 kB/s | 51 kB 00:00 2023-12-01T01:33:41.956 INFO:teuthology.orchestra.run.smithi142.stdout:(128/148): python3-websocket-client-1.2.3-2.el9 718 kB/s | 90 kB 00:00 2023-12-01T01:33:41.997 INFO:teuthology.orchestra.run.smithi142.stdout:(129/148): python3-werkzeug-2.0.3-3.el9.1.noarc 3.0 MB/s | 427 kB 00:00 2023-12-01T01:33:42.048 INFO:teuthology.orchestra.run.smithi142.stdout:(130/148): python3-webob-1.8.7-6.el9.noarch.rpm 917 kB/s | 230 kB 00:00 2023-12-01T01:33:42.098 INFO:teuthology.orchestra.run.smithi142.stdout:(131/148): python3-zc-lockfile-2.0-10.el9.noarc 142 kB/s | 20 kB 00:00 2023-12-01T01:33:42.161 INFO:teuthology.orchestra.run.smithi142.stdout:(132/148): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2023-12-01T01:33:42.327 INFO:teuthology.orchestra.run.smithi115.stdout:(77/148): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2023-12-01T01:33:42.369 INFO:teuthology.orchestra.run.smithi115.stdout:(78/148): python3-toml-0.10.2-6.el9.noarch.rpm 69 kB/s | 39 kB 00:00 2023-12-01T01:33:42.402 INFO:teuthology.orchestra.run.smithi115.stdout:(79/148): socat-1.7.4.1-5.el9.x86_64.rpm 505 kB/s | 300 kB 00:00 2023-12-01T01:33:42.444 INFO:teuthology.orchestra.run.smithi115.stdout:(80/148): uuid-1.6.2-55.el9.x86_64.rpm 498 kB/s | 56 kB 00:00 2023-12-01T01:33:42.469 INFO:teuthology.orchestra.run.smithi115.stdout:(81/148): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.3 MB/s | 85 kB 00:00 2023-12-01T01:33:42.511 INFO:teuthology.orchestra.run.smithi115.stdout:(82/148): xmlstarlet-1.6.1-20.el9.x86_64.rpm 585 kB/s | 63 kB 00:00 2023-12-01T01:33:42.532 INFO:teuthology.orchestra.run.smithi142.stdout:(133/148): thrift-0.15.0-2.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2023-12-01T01:33:42.553 INFO:teuthology.orchestra.run.smithi115.stdout:(83/148): zlib-devel-1.2.11-40.el9.x86_64.rpm 412 kB/s | 44 kB 00:00 2023-12-01T01:33:42.587 INFO:teuthology.orchestra.run.smithi115.stdout:(84/148): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2023-12-01T01:33:42.629 INFO:teuthology.orchestra.run.smithi115.stdout:(85/148): libgfortran-11.4.1-2.3.el9.x86_64.rpm 6.8 MB/s | 803 kB 00:00 2023-12-01T01:33:42.649 INFO:teuthology.orchestra.run.smithi142.stdout:(134/148): librbd1-18.2.0-1249.ga6f82997.el9.x8 6.2 MB/s | 3.0 MB 00:00 2023-12-01T01:33:42.662 INFO:teuthology.orchestra.run.smithi115.stdout:(86/148): libquadmath-11.4.1-2.3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2023-12-01T01:33:42.674 INFO:teuthology.orchestra.run.smithi142.stdout:(135/148): libacl-devel-2.3.1-4.el9.x86_64.rpm 479 kB/s | 68 kB 00:00 2023-12-01T01:33:42.699 INFO:teuthology.orchestra.run.smithi142.stdout:(136/148): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 69 kB 00:00 2023-12-01T01:33:42.704 INFO:teuthology.orchestra.run.smithi115.stdout:(87/148): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2023-12-01T01:33:42.716 INFO:teuthology.orchestra.run.smithi142.stdout:(137/148): elfutils-debuginfod-client-0.190-1.e 908 kB/s | 36 kB 00:00 2023-12-01T01:33:42.746 INFO:teuthology.orchestra.run.smithi115.stdout:(88/148): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.1 MB 00:00 2023-12-01T01:33:42.750 INFO:teuthology.orchestra.run.smithi142.stdout:(138/148): elfutils-libelf-0.190-1.el9.x86_64.r 3.9 MB/s | 192 kB 00:00 2023-12-01T01:33:42.771 INFO:teuthology.orchestra.run.smithi115.stdout:(89/148): python3-ply-3.11-14.el9.noarch.rpm 955 kB/s | 103 kB 00:00 2023-12-01T01:33:42.784 INFO:teuthology.orchestra.run.smithi142.stdout:(139/148): elfutils-libs-0.190-1.el9.x86_64.rpm 3.9 MB/s | 255 kB 00:00 2023-12-01T01:33:42.809 INFO:teuthology.orchestra.run.smithi142.stdout:(140/148): libacl-2.3.1-4.el9.x86_64.rpm 391 kB/s | 22 kB 00:00 2023-12-01T01:33:42.813 INFO:teuthology.orchestra.run.smithi115.stdout:(90/148): python3-requests-2.25.1-7.el9.noarch. 2.7 MB/s | 113 kB 00:00 2023-12-01T01:33:42.843 INFO:teuthology.orchestra.run.smithi142.stdout:(141/148): xfsdump-3.1.12-4.el9.x86_64.rpm 5.5 MB/s | 321 kB 00:00 2023-12-01T01:33:42.846 INFO:teuthology.orchestra.run.smithi115.stdout:(91/148): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 124 kB 00:00 2023-12-01T01:33:42.891 INFO:teuthology.orchestra.run.smithi115.stdout:(92/148): python3-urllib3-1.26.5-4.el9.noarch.r 1.6 MB/s | 187 kB 00:00 2023-12-01T01:33:42.926 INFO:teuthology.orchestra.run.smithi142.stdout:(142/148): librados2-18.2.0-1249.ga6f82997.el9. 4.0 MB/s | 3.3 MB 00:00 2023-12-01T01:33:42.989 INFO:teuthology.orchestra.run.smithi115.stdout:(93/148): python3-devel-3.9.17-2.el9.x86_64.rpm 2.5 MB/s | 245 kB 00:00 2023-12-01T01:33:43.043 INFO:teuthology.orchestra.run.smithi142.stdout:(143/148): NetworkManager-team-1.45.7-1.el9.x86 372 kB/s | 43 kB 00:00 2023-12-01T01:33:43.160 INFO:teuthology.orchestra.run.smithi142.stdout:(144/148): NetworkManager-tui-1.45.7-1.el9.x86_ 2.1 MB/s | 246 kB 00:00 2023-12-01T01:33:43.227 INFO:teuthology.orchestra.run.smithi142.stdout:(145/148): NetworkManager-libnm-1.45.7-1.el9.x8 4.6 MB/s | 1.8 MB 00:00 2023-12-01T01:33:43.256 INFO:teuthology.orchestra.run.smithi115.stdout:(94/148): openssl-devel-3.0.7-25.el9.x86_64.rpm 9.4 MB/s | 4.1 MB 00:00 2023-12-01T01:33:43.311 INFO:teuthology.orchestra.run.smithi142.stdout:(146/148): NetworkManager-1.45.7-1.el9.x86_64.r 4.5 MB/s | 2.3 MB 00:00 2023-12-01T01:33:43.339 INFO:teuthology.orchestra.run.smithi115.stdout:(95/148): fmt-8.1.1-5.el9.x86_64.rpm 248 kB/s | 111 kB 00:00 2023-12-01T01:33:43.378 INFO:teuthology.orchestra.run.smithi142.stdout:(147/148): openssl-3.0.7-25.el9.x86_64.rpm 5.6 MB/s | 1.2 MB 00:00 2023-12-01T01:33:43.444 INFO:teuthology.orchestra.run.smithi142.stdout:(148/148): openssl-libs-3.0.7-25.el9.x86_64.rpm 10 MB/s | 2.2 MB 00:00 2023-12-01T01:33:43.456 INFO:teuthology.orchestra.run.smithi115.stdout:(96/148): libarrow-doc-9.0.0-9.el9.noarch.rpm 210 kB/s | 24 kB 00:00 2023-12-01T01:33:43.459 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:33:43.459 INFO:teuthology.orchestra.run.smithi142.stdout:Total 11 MB/s | 200 MB 00:18 2023-12-01T01:33:43.531 INFO:teuthology.orchestra.run.smithi115.stdout:(97/148): gperftools-libs-2.9.1-2.el9.x86_64.rp 570 kB/s | 309 kB 00:00 2023-12-01T01:33:43.573 INFO:teuthology.orchestra.run.smithi115.stdout:(98/148): liboath-2.6.7-2.el9.x86_64.rpm 418 kB/s | 49 kB 00:00 2023-12-01T01:33:43.632 INFO:teuthology.orchestra.run.smithi115.stdout:(99/148): libunwind-1.6.2-1.el9.x86_64.rpm 677 kB/s | 67 kB 00:00 2023-12-01T01:33:43.832 INFO:teuthology.orchestra.run.smithi115.stdout:(100/148): python3-asyncssh-2.13.2-3.el9.noarch 2.7 MB/s | 548 kB 00:00 2023-12-01T01:33:43.874 INFO:teuthology.orchestra.run.smithi115.stdout:(101/148): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.7 MB/s | 837 kB 00:00 2023-12-01T01:33:43.877 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2023-12-01T01:33:43.966 INFO:teuthology.orchestra.run.smithi115.stdout:(102/148): python3-bcrypt-3.2.2-1.el9.x86_64.rp 327 kB/s | 43 kB 00:00 2023-12-01T01:33:44.016 INFO:teuthology.orchestra.run.smithi115.stdout:(103/148): python3-cachetools-4.2.4-1.el9.noarc 228 kB/s | 32 kB 00:00 2023-12-01T01:33:44.083 INFO:teuthology.orchestra.run.smithi115.stdout:(104/148): python3-certifi-2021.10.8-2.el9.noar 130 kB/s | 15 kB 00:00 2023-12-01T01:33:44.117 INFO:teuthology.orchestra.run.smithi115.stdout:(105/148): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2023-12-01T01:33:44.153 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2023-12-01T01:33:44.153 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2023-12-01T01:33:44.209 INFO:teuthology.orchestra.run.smithi115.stdout:(106/148): libarrow-9.0.0-9.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2023-12-01T01:33:44.242 INFO:teuthology.orchestra.run.smithi115.stdout:(107/148): python3-google-auth-2.23.4-1.el9.noa 1.5 MB/s | 197 kB 00:00 2023-12-01T01:33:44.278 INFO:teuthology.orchestra.run.smithi115.stdout:(108/148): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2023-12-01T01:33:44.329 INFO:teuthology.orchestra.run.smithi115.stdout:(109/148): python3-jaraco-8.2.1-3.el9.noarch.rp 90 kB/s | 11 kB 00:00 2023-12-01T01:33:44.379 INFO:teuthology.orchestra.run.smithi115.stdout:(110/148): python3-jaraco-classes-3.2.1-5.el9.n 130 kB/s | 18 kB 00:00 2023-12-01T01:33:44.429 INFO:teuthology.orchestra.run.smithi115.stdout:(111/148): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2023-12-01T01:33:44.479 INFO:teuthology.orchestra.run.smithi115.stdout:(112/148): python3-jaraco-functools-3.5.0-2.el9 130 kB/s | 19 kB 00:00 2023-12-01T01:33:44.529 INFO:teuthology.orchestra.run.smithi115.stdout:(113/148): python3-jaraco-text-3.2.0-6.el9.noar 131 kB/s | 20 kB 00:00 2023-12-01T01:33:44.580 INFO:teuthology.orchestra.run.smithi115.stdout:(114/148): python3-jwt+crypto-2.4.0-1.el9.noarc 60 kB/s | 9.0 kB 00:00 2023-12-01T01:33:44.630 INFO:teuthology.orchestra.run.smithi115.stdout:(115/148): python3-jwt-2.4.0-1.el9.noarch.rpm 273 kB/s | 41 kB 00:00 2023-12-01T01:33:44.705 INFO:teuthology.orchestra.run.smithi115.stdout:(116/148): python3-logutils-0.3.5-21.el9.noarch 372 kB/s | 46 kB 00:00 2023-12-01T01:33:44.731 INFO:teuthology.orchestra.run.smithi115.stdout:(117/148): python3-more-itertools-8.12.0-2.el9. 793 kB/s | 79 kB 00:00 2023-12-01T01:33:44.772 INFO:teuthology.orchestra.run.smithi115.stdout:(118/148): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2023-12-01T01:33:44.831 INFO:teuthology.orchestra.run.smithi115.stdout:(119/148): python3-natsort-7.1.1-5.el9.noarch.r 462 kB/s | 58 kB 00:00 2023-12-01T01:33:44.865 INFO:teuthology.orchestra.run.smithi115.stdout:(120/148): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2023-12-01T01:33:44.907 INFO:teuthology.orchestra.run.smithi115.stdout:(121/148): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2023-12-01T01:33:44.965 INFO:teuthology.orchestra.run.smithi115.stdout:(122/148): python3-pyOpenSSL-21.0.0-1.el9.noarc 675 kB/s | 90 kB 00:00 2023-12-01T01:33:45.009 INFO:teuthology.orchestra.run.smithi115.stdout:(123/148): python3-repoze-lru-0.7-16.el9.noarch 217 kB/s | 31 kB 00:00 2023-12-01T01:33:45.041 INFO:teuthology.orchestra.run.smithi115.stdout:(124/148): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2023-12-01T01:33:45.083 INFO:teuthology.orchestra.run.smithi115.stdout:(125/148): python3-rsa-4.9-2.el9.noarch.rpm 507 kB/s | 59 kB 00:00 2023-12-01T01:33:45.125 INFO:teuthology.orchestra.run.smithi115.stdout:(126/148): python3-tempora-5.0.0-2.el9.noarch.r 307 kB/s | 36 kB 00:00 2023-12-01T01:33:45.167 INFO:teuthology.orchestra.run.smithi115.stdout:(127/148): python3-typing-extensions-4.4.0-2.el 407 kB/s | 51 kB 00:00 2023-12-01T01:33:45.200 INFO:teuthology.orchestra.run.smithi115.stdout:(128/148): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2023-12-01T01:33:45.226 INFO:teuthology.orchestra.run.smithi115.stdout:(129/148): python3-websocket-client-1.2.3-2.el9 898 kB/s | 90 kB 00:00 2023-12-01T01:33:45.284 INFO:teuthology.orchestra.run.smithi115.stdout:(130/148): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2023-12-01T01:33:45.309 INFO:teuthology.orchestra.run.smithi115.stdout:(131/148): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2023-12-01T01:33:45.343 INFO:teuthology.orchestra.run.smithi115.stdout:(132/148): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2023-12-01T01:33:45.426 INFO:teuthology.orchestra.run.smithi115.stdout:(133/148): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2023-12-01T01:33:45.460 INFO:teuthology.orchestra.run.smithi115.stdout:(134/148): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-12-01T01:33:45.502 INFO:teuthology.orchestra.run.smithi115.stdout:(135/148): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 69 kB 00:00 2023-12-01T01:33:45.544 INFO:teuthology.orchestra.run.smithi115.stdout:(136/148): elfutils-debuginfod-client-0.190-1.e 887 kB/s | 36 kB 00:00 2023-12-01T01:33:45.586 INFO:teuthology.orchestra.run.smithi115.stdout:(137/148): elfutils-libelf-0.190-1.el9.x86_64.r 4.6 MB/s | 192 kB 00:00 2023-12-01T01:33:45.630 INFO:teuthology.orchestra.run.smithi115.stdout:(138/148): elfutils-libs-0.190-1.el9.x86_64.rpm 6.1 MB/s | 255 kB 00:00 2023-12-01T01:33:45.662 INFO:teuthology.orchestra.run.smithi115.stdout:(139/148): libacl-2.3.1-4.el9.x86_64.rpm 674 kB/s | 22 kB 00:00 2023-12-01T01:33:45.746 INFO:teuthology.orchestra.run.smithi115.stdout:(140/148): librbd1-18.2.0-1249.ga6f82997.el9.x8 7.5 MB/s | 3.0 MB 00:00 2023-12-01T01:33:45.779 INFO:teuthology.orchestra.run.smithi115.stdout:(141/148): xfsdump-3.1.12-4.el9.x86_64.rpm 2.7 MB/s | 321 kB 00:00 2023-12-01T01:33:45.938 INFO:teuthology.orchestra.run.smithi115.stdout:(142/148): librados2-18.2.0-1249.ga6f82997.el9. 5.2 MB/s | 3.3 MB 00:00 2023-12-01T01:33:46.088 INFO:teuthology.orchestra.run.smithi115.stdout:(143/148): NetworkManager-team-1.45.7-1.el9.x86 290 kB/s | 43 kB 00:00 2023-12-01T01:33:46.238 INFO:teuthology.orchestra.run.smithi115.stdout:(144/148): NetworkManager-tui-1.45.7-1.el9.x86_ 1.6 MB/s | 246 kB 00:00 2023-12-01T01:33:46.312 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2023-12-01T01:33:46.313 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2023-12-01T01:33:46.322 INFO:teuthology.orchestra.run.smithi115.stdout:(145/148): NetworkManager-libnm-1.45.7-1.el9.x8 3.3 MB/s | 1.8 MB 00:00 2023-12-01T01:33:46.439 INFO:teuthology.orchestra.run.smithi115.stdout:(146/148): openssl-3.0.7-25.el9.x86_64.rpm 6.1 MB/s | 1.2 MB 00:00 2023-12-01T01:33:46.505 INFO:teuthology.orchestra.run.smithi115.stdout:(147/148): NetworkManager-1.45.7-1.el9.x86_64.r 3.0 MB/s | 2.3 MB 00:00 2023-12-01T01:33:46.997 INFO:teuthology.orchestra.run.smithi115.stdout:(148/148): openssl-libs-3.0.7-25.el9.x86_64.rpm 3.2 MB/s | 2.2 MB 00:00 2023-12-01T01:33:47.006 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:33:47.006 INFO:teuthology.orchestra.run.smithi115.stdout:Total 13 MB/s | 200 MB 00:14 2023-12-01T01:33:47.409 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2023-12-01T01:33:47.690 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2023-12-01T01:33:47.690 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2023-12-01T01:33:49.825 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2023-12-01T01:33:49.825 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2023-12-01T01:33:52.855 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2023-12-01T01:33:53.034 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/163 2023-12-01T01:33:53.069 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/163 2023-12-01T01:33:53.139 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/163 2023-12-01T01:33:53.195 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/163 2023-12-01T01:33:53.218 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/163 2023-12-01T01:33:53.305 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/163 2023-12-01T01:33:53.722 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/163 2023-12-01T01:33:53.758 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:18.2.0-1249.ga6f82997.el9.x86_64 8/163 2023-12-01T01:33:53.917 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:18.2.0-1249.ga6f82997.el9.x86_64 8/163 2023-12-01T01:33:53.951 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:18.2.0-1249.ga6f82997.el9.x86_64 9/163 2023-12-01T01:33:54.075 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:18.2.0-1249.ga6f82997.el9.x86_64 9/163 2023-12-01T01:33:54.115 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:18.2.0-1249.ga6f82997.el9.x86_64 10/163 2023-12-01T01:33:54.188 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/163 2023-12-01T01:33:54.517 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/163 2023-12-01T01:33:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/163 2023-12-01T01:33:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:18.2.0-1249.ga6f82997.el9.x86_64 14/163 2023-12-01T01:33:54.771 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:18.2.0-1249.ga6f82997.el9.x86_64 14/163 2023-12-01T01:33:54.793 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:18.2.0-1249.ga6f82997.el9.x86 15/163 2023-12-01T01:33:55.130 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:18.2.0-1249.ga6f82997.el9.x86 15/163 2023-12-01T01:33:55.332 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/163 2023-12-01T01:33:55.419 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/163 2023-12-01T01:33:55.621 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/163 2023-12-01T01:33:55.684 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/163 2023-12-01T01:33:56.010 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/163 2023-12-01T01:33:56.167 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2023-12-01T01:33:56.189 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/163 2023-12-01T01:33:56.231 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/163 2023-12-01T01:33:56.344 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/163 2023-12-01T01:33:56.373 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/163 2023-12-01T01:33:56.379 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/163 2023-12-01T01:33:56.449 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/163 2023-12-01T01:33:56.465 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/163 2023-12-01T01:33:56.486 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/163 2023-12-01T01:33:56.528 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/163 2023-12-01T01:33:56.552 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/163 2023-12-01T01:33:56.571 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/163 2023-12-01T01:33:56.606 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/163 2023-12-01T01:33:56.671 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/163 2023-12-01T01:33:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/163 2023-12-01T01:33:56.763 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/163 2023-12-01T01:33:56.796 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/163 2023-12-01T01:33:56.897 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/163 2023-12-01T01:33:56.925 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:18.2.0-1249.ga6f82997.el9. 29/163 2023-12-01T01:33:56.967 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el 30/163 2023-12-01T01:33:57.067 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:18.2.0-1249.ga6f82997.el9.x86_6 31/163 2023-12-01T01:33:57.071 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/163 2023-12-01T01:33:57.111 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:18.2.0-1249.ga6f82997.el9.x86_64 8/163 2023-12-01T01:33:57.241 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/163 2023-12-01T01:33:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:18.2.0-1249.ga6f82997.el9.x86_64 8/163 2023-12-01T01:33:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:18.2.0-1249.ga6f82997.el9.x86_64 9/163 2023-12-01T01:33:57.309 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/163 2023-12-01T01:33:57.416 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:18.2.0-1249.ga6f82997.el9.x86_64 9/163 2023-12-01T01:33:57.455 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:18.2.0-1249.ga6f82997.el9.x86_64 10/163 2023-12-01T01:33:57.525 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/163 2023-12-01T01:33:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/163 2023-12-01T01:33:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/163 2023-12-01T01:33:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:18.2.0-1249.ga6f82997.el9.x86_64 14/163 2023-12-01T01:33:57.990 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/163 2023-12-01T01:33:58.042 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:18.2.0-1249.ga6f82997.el9.x86_64 14/163 2023-12-01T01:33:58.068 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:18.2.0-1249.ga6f82997.el9.x86 15/163 2023-12-01T01:33:58.080 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/163 2023-12-01T01:33:58.144 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/163 2023-12-01T01:33:58.201 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/163 2023-12-01T01:33:58.247 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:18.2.0-1249.ga6f82997.el9.x86_6 38/163 2023-12-01T01:33:58.312 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/163 2023-12-01T01:33:58.420 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:18.2.0-1249.ga6f82997.el9.x86 15/163 2023-12-01T01:33:58.508 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/163 2023-12-01T01:33:58.551 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/163 2023-12-01T01:33:58.581 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/163 2023-12-01T01:33:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/163 2023-12-01T01:33:58.716 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/163 2023-12-01T01:33:58.788 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/163 2023-12-01T01:33:58.870 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/163 2023-12-01T01:33:58.942 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/163 2023-12-01T01:33:59.008 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/163 2023-12-01T01:33:59.154 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/163 2023-12-01T01:33:59.199 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/163 2023-12-01T01:33:59.247 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/163 2023-12-01T01:33:59.273 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/163 2023-12-01T01:33:59.333 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/163 2023-12-01T01:33:59.396 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/163 2023-12-01T01:33:59.436 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/163 2023-12-01T01:33:59.458 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/163 2023-12-01T01:33:59.473 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/163 2023-12-01T01:33:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/163 2023-12-01T01:33:59.647 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/163 2023-12-01T01:33:59.661 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/163 2023-12-01T01:33:59.696 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/163 2023-12-01T01:33:59.701 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/163 2023-12-01T01:33:59.742 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/163 2023-12-01T01:33:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/163 2023-12-01T01:33:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/163 2023-12-01T01:33:59.866 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/163 2023-12-01T01:33:59.909 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/163 2023-12-01T01:33:59.910 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/163 2023-12-01T01:33:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/163 2023-12-01T01:33:59.961 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/163 2023-12-01T01:33:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/163 2023-12-01T01:34:00.011 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/163 2023-12-01T01:34:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/163 2023-12-01T01:34:00.095 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/163 2023-12-01T01:34:00.134 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:18.2.0-1249.ga6f82997.el9. 29/163 2023-12-01T01:34:00.184 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el 30/163 2023-12-01T01:34:00.219 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fmt-8.1.1-5.el9.x86_64 59/163 2023-12-01T01:34:00.265 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 60/163 2023-12-01T01:34:00.288 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:18.2.0-1249.ga6f82997.el9.x86_6 31/163 2023-12-01T01:34:00.334 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 61/163 2023-12-01T01:34:00.396 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 62/163 2023-12-01T01:34:00.462 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/163 2023-12-01T01:34:00.527 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/163 2023-12-01T01:34:00.692 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 63/163 2023-12-01T01:34:00.754 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 64/163 2023-12-01T01:34:00.925 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 65/163 2023-12-01T01:34:00.990 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 66/163 2023-12-01T01:34:01.014 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 67/163 2023-12-01T01:34:01.054 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 68/163 2023-12-01T01:34:01.097 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 69/163 2023-12-01T01:34:01.133 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-15.el9.x86_64 70/163 2023-12-01T01:34:01.177 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 71/163 2023-12-01T01:34:01.207 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/163 2023-12-01T01:34:01.233 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 72/163 2023-12-01T01:34:01.298 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/163 2023-12-01T01:34:01.300 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-55.el9.x86_64 73/163 2023-12-01T01:34:01.343 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/163 2023-12-01T01:34:01.359 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/163 2023-12-01T01:34:01.434 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/163 2023-12-01T01:34:01.499 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/163 2023-12-01T01:34:01.536 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 76/163 2023-12-01T01:34:01.572 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 77/163 2023-12-01T01:34:01.626 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/163 2023-12-01T01:34:01.658 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 79/163 2023-12-01T01:34:01.713 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:18.2.0-1249.ga6f82997.el9.x86_6 38/163 2023-12-01T01:34:01.787 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 80/163 2023-12-01T01:34:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/163 2023-12-01T01:34:01.855 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 81/163 2023-12-01T01:34:02.005 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/163 2023-12-01T01:34:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/163 2023-12-01T01:34:02.071 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/163 2023-12-01T01:34:02.072 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/163 2023-12-01T01:34:02.225 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/163 2023-12-01T01:34:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/163 2023-12-01T01:34:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/163 2023-12-01T01:34:02.425 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/163 2023-12-01T01:34:02.466 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/163 2023-12-01T01:34:02.505 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/163 2023-12-01T01:34:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/163 2023-12-01T01:34:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/163 2023-12-01T01:34:02.691 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/163 2023-12-01T01:34:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/163 2023-12-01T01:34:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/163 2023-12-01T01:34:02.967 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/163 2023-12-01T01:34:03.038 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/163 2023-12-01T01:34:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/163 2023-12-01T01:34:03.094 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/163 2023-12-01T01:34:03.128 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/163 2023-12-01T01:34:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/163 2023-12-01T01:34:03.161 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/163 2023-12-01T01:34:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/163 2023-12-01T01:34:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/163 2023-12-01T01:34:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/163 2023-12-01T01:34:03.396 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.21-2.el9.x86_64 88/163 2023-12-01T01:34:03.415 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fmt-8.1.1-5.el9.x86_64 59/163 2023-12-01T01:34:03.433 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 89/163 2023-12-01T01:34:03.460 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 60/163 2023-12-01T01:34:03.529 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 61/163 2023-12-01T01:34:03.530 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 90/163 2023-12-01T01:34:03.610 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 62/163 2023-12-01T01:34:03.864 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 63/163 2023-12-01T01:34:03.931 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 64/163 2023-12-01T01:34:04.102 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 65/163 2023-12-01T01:34:04.158 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 66/163 2023-12-01T01:34:04.183 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 67/163 2023-12-01T01:34:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 68/163 2023-12-01T01:34:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 69/163 2023-12-01T01:34:04.305 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 91/163 2023-12-01T01:34:04.311 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-15.el9.x86_64 70/163 2023-12-01T01:34:04.344 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 71/163 2023-12-01T01:34:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 72/163 2023-12-01T01:34:04.442 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 92/163 2023-12-01T01:34:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : uuid-1.6.2-55.el9.x86_64 73/163 2023-12-01T01:34:04.502 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/163 2023-12-01T01:34:04.628 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/163 2023-12-01T01:34:04.741 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 76/163 2023-12-01T01:34:04.784 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 77/163 2023-12-01T01:34:04.853 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/163 2023-12-01T01:34:04.896 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 79/163 2023-12-01T01:34:05.042 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 80/163 2023-12-01T01:34:05.092 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 81/163 2023-12-01T01:34:05.230 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/163 2023-12-01T01:34:05.289 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/163 2023-12-01T01:34:05.957 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 93/163 2023-12-01T01:34:06.071 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 94/163 2023-12-01T01:34:06.108 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 95/163 2023-12-01T01:34:06.139 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : jq-1.6-16.el9.x86_64 96/163 2023-12-01T01:34:06.183 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 97/163 2023-12-01T01:34:06.224 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 98/163 2023-12-01T01:34:06.257 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/163 2023-12-01T01:34:06.313 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/163 2023-12-01T01:34:06.347 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/163 2023-12-01T01:34:06.371 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/163 2023-12-01T01:34:06.576 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 99/163 2023-12-01T01:34:06.602 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.21-2.el9.x86_64 88/163 2023-12-01T01:34:06.611 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:18.2.0-1249.ga6f82997.el9.x86_64 100/163 2023-12-01T01:34:06.635 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 89/163 2023-12-01T01:34:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 90/163 2023-12-01T01:34:06.731 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:18.2.0-1249.ga6f82997.el9.x86_64 100/163 2023-12-01T01:34:06.767 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:18.2.0-1249.ga6f82997.el9.x86_64 101/163 2023-12-01T01:34:06.821 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 102/163 2023-12-01T01:34:07.265 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 103/163 2023-12-01T01:34:07.297 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:18.2.0-1249.ga6f82997.el9.x86_64 104/163 2023-12-01T01:34:07.446 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:18.2.0-1249.ga6f82997.el9.x86_64 104/163 2023-12-01T01:34:07.474 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:18.2.0-1249.ga6f82997.el9.x86_64 105/163 2023-12-01T01:34:07.511 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 91/163 2023-12-01T01:34:07.637 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 92/163 2023-12-01T01:34:09.167 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 93/163 2023-12-01T01:34:09.283 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 94/163 2023-12-01T01:34:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 95/163 2023-12-01T01:34:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : jq-1.6-16.el9.x86_64 96/163 2023-12-01T01:34:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 97/163 2023-12-01T01:34:09.422 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 98/163 2023-12-01T01:34:09.738 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 99/163 2023-12-01T01:34:09.773 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:18.2.0-1249.ga6f82997.el9.x86_64 100/163 2023-12-01T01:34:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:18.2.0-1249.ga6f82997.el9.x86_64 100/163 2023-12-01T01:34:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:18.2.0-1249.ga6f82997.el9.x86_64 101/163 2023-12-01T01:34:09.983 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 102/163 2023-12-01T01:34:10.122 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 106/163 2023-12-01T01:34:10.162 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 106/163 2023-12-01T01:34:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 103/163 2023-12-01T01:34:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:18.2.0-1249.ga6f82997.el9.x86_64 104/163 2023-12-01T01:34:10.558 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:18.2.0-1249.ga6f82997.el9.x86_64 104/163 2023-12-01T01:34:10.585 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:18.2.0-1249.ga6f82997.el9.x86_64 105/163 2023-12-01T01:34:10.782 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 106/163 2023-12-01T01:34:10.832 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:18.2.0-1249.ga6f82997.el9.x86_64 107/163 2023-12-01T01:34:11.022 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:18.2.0-1249.ga6f82997.el9.x86_64 107/163 2023-12-01T01:34:11.023 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-01T01:34:11.023 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-01T01:34:11.023 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:11.075 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:18.2.0-1249.ga6f82997.el9.x86_64 108/163 2023-12-01T01:34:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 106/163 2023-12-01T01:34:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 106/163 2023-12-01T01:34:13.627 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 106/163 2023-12-01T01:34:13.669 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:18.2.0-1249.ga6f82997.el9.x86_64 107/163 2023-12-01T01:34:13.851 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:18.2.0-1249.ga6f82997.el9.x86_64 107/163 2023-12-01T01:34:13.851 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-01T01:34:13.851 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-01T01:34:13.851 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:18.2.0-1249.ga6f82997.el9.x86_64 108/163 2023-12-01T01:34:23.377 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:18.2.0-1249.ga6f82997.el9.x86_64 108/163 2023-12-01T01:34:23.378 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2023-12-01T01:34:23.378 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2023-12-01T01:34:23.378 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2023-12-01T01:34:23.378 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2023-12-01T01:34:23.378 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2023-12-01T01:34:23.379 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2023-12-01T01:34:23.379 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2023-12-01T01:34:23.379 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:23.596 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:18.2.0-1249.ga6f82997.el9.noarch 109/163 2023-12-01T01:34:23.645 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:18.2.0-1249.ga6f82997.el9.noarch 109/163 2023-12-01T01:34:23.645 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:23.645 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-01T01:34:23.646 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:25.331 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:18.2.0-1249.ga6f82997.el9.x86_64 110/163 2023-12-01T01:34:25.411 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:18.2.0-1249.ga6f82997.el9.x86_64 110/163 2023-12-01T01:34:25.411 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:25.411 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-01T01:34:25.412 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-01T01:34:25.412 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-01T01:34:25.412 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:25.623 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:18.2.0-1249.ga6f82997.el9.x86_64 111/163 2023-12-01T01:34:25.698 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:18.2.0-1249.ga6f82997.el9.x86_64 111/163 2023-12-01T01:34:25.698 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:25.698 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-01T01:34:25.698 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-01T01:34:25.698 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-01T01:34:25.698 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:26.190 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:18.2.0-1249.ga6f82997.el9.x86_64 112/163 2023-12-01T01:34:26.269 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:18.2.0-1249.ga6f82997.el9.x86_64 112/163 2023-12-01T01:34:26.269 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:26.270 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-01T01:34:26.270 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-01T01:34:26.270 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-01T01:34:26.270 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:26.384 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.2-8.el9.noarch 113/163 2023-12-01T01:34:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:18.2.0-1249.ga6f82997.el9.x86_64 108/163 2023-12-01T01:34:26.400 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2023-12-01T01:34:26.400 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2023-12-01T01:34:26.400 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2023-12-01T01:34:26.400 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2023-12-01T01:34:26.400 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2023-12-01T01:34:26.400 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2023-12-01T01:34:26.400 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2023-12-01T01:34:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:26.624 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:18.2.0-1249.ga6f82997.el9.noarch 109/163 2023-12-01T01:34:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:18.2.0-1249.ga6f82997.el9.noarch 109/163 2023-12-01T01:34:26.669 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:26.669 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-01T01:34:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:27.269 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:18.2.0-1249.ga6f82997.el9.noarch 114/163 2023-12-01T01:34:27.311 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:18.2.0-1249.ga6f82997.el9.noarch 114/163 2023-12-01T01:34:27.348 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.e 115/163 2023-12-01T01:34:27.449 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997. 116/163 2023-12-01T01:34:27.481 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el9.noa 117/163 2023-12-01T01:34:27.798 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el9.noa 117/163 2023-12-01T01:34:27.840 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el9.n 118/163 2023-12-01T01:34:28.343 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:18.2.0-1249.ga6f82997.el9.x86_64 110/163 2023-12-01T01:34:28.422 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:18.2.0-1249.ga6f82997.el9.x86_64 110/163 2023-12-01T01:34:28.422 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:28.422 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-01T01:34:28.422 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-01T01:34:28.422 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-01T01:34:28.422 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:28.643 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:18.2.0-1249.ga6f82997.el9.x86_64 111/163 2023-12-01T01:34:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:18.2.0-1249.ga6f82997.el9.x86_64 111/163 2023-12-01T01:34:28.713 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:28.713 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-01T01:34:28.713 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-01T01:34:28.713 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-01T01:34:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:28.966 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el9.n 118/163 2023-12-01T01:34:28.999 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 119/163 2023-12-01T01:34:29.139 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 119/163 2023-12-01T01:34:29.211 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:18.2.0-1249.ga6f82997.el9.x86_64 112/163 2023-12-01T01:34:29.282 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:18.2.0-1249.ga6f82997.el9.x86_64 112/163 2023-12-01T01:34:29.282 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:29.282 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-01T01:34:29.283 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-01T01:34:29.283 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-01T01:34:29.283 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:29.321 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el 120/163 2023-12-01T01:34:29.350 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:18.2.0-1249.ga6f82997.el9.x86_64 121/163 2023-12-01T01:34:29.405 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : automake-1.16.2-8.el9.noarch 113/163 2023-12-01T01:34:29.433 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:18.2.0-1249.ga6f82997.el9.x86_64 121/163 2023-12-01T01:34:29.433 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:29.433 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-01T01:34:29.433 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-01T01:34:29.434 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-01T01:34:29.434 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:29.493 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el9.noarch 122/163 2023-12-01T01:34:29.534 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el9.noarch 122/163 2023-12-01T01:34:29.622 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:18.2.0-1249.ga6f82997.el9.x86_64 123/163 2023-12-01T01:34:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:18.2.0-1249.ga6f82997.el9.noarch 114/163 2023-12-01T01:34:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:18.2.0-1249.ga6f82997.el9.noarch 114/163 2023-12-01T01:34:30.103 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.e 115/163 2023-12-01T01:34:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997. 116/163 2023-12-01T01:34:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el9.noa 117/163 2023-12-01T01:34:30.472 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-45.el9.x86_64 124/163 2023-12-01T01:34:30.511 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:18.2.0-1249.ga6f82997.el9.x86_64 125/163 2023-12-01T01:34:30.595 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-1249.ga6f82997.el9.x86_64 125/163 2023-12-01T01:34:30.595 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:30.595 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-01T01:34:30.596 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-01T01:34:30.596 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-01T01:34:30.596 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el9.noa 117/163 2023-12-01T01:34:30.652 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el9.n 118/163 2023-12-01T01:34:30.662 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 126/163 2023-12-01T01:34:30.737 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 126/163 2023-12-01T01:34:30.737 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:30.738 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-01T01:34:30.738 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:30.830 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 127/163 2023-12-01T01:34:30.909 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 127/163 2023-12-01T01:34:30.910 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:30.910 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-01T01:34:30.910 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-01T01:34:30.910 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-01T01:34:30.910 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:31.297 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 128/163 2023-12-01T01:34:31.371 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 128/163 2023-12-01T01:34:31.372 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:31.372 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-01T01:34:31.372 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-01T01:34:31.372 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-01T01:34:31.372 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:31.773 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el9.n 118/163 2023-12-01T01:34:31.802 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 119/163 2023-12-01T01:34:31.945 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 119/163 2023-12-01T01:34:32.129 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el 120/163 2023-12-01T01:34:32.187 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:18.2.0-1249.ga6f82997.el9.x86_64 121/163 2023-12-01T01:34:32.262 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:18.2.0-1249.ga6f82997.el9.x86_64 121/163 2023-12-01T01:34:32.262 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:32.262 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-01T01:34:32.262 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-01T01:34:32.262 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-01T01:34:32.263 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:32.313 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el9.noarch 122/163 2023-12-01T01:34:32.353 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el9.noarch 122/163 2023-12-01T01:34:32.445 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:18.2.0-1249.ga6f82997.el9.x86_64 123/163 2023-12-01T01:34:33.232 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libtool-2.4.6-45.el9.x86_64 124/163 2023-12-01T01:34:33.290 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:18.2.0-1249.ga6f82997.el9.x86_64 125/163 2023-12-01T01:34:33.366 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-1249.ga6f82997.el9.x86_64 125/163 2023-12-01T01:34:33.366 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:33.366 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-01T01:34:33.366 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-01T01:34:33.366 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-01T01:34:33.366 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:33.433 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 126/163 2023-12-01T01:34:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 126/163 2023-12-01T01:34:33.507 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:33.507 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-01T01:34:33.507 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:33.592 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 127/163 2023-12-01T01:34:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 127/163 2023-12-01T01:34:33.698 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:33.698 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-01T01:34:33.698 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-01T01:34:33.699 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-01T01:34:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 128/163 2023-12-01T01:34:34.114 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 128/163 2023-12-01T01:34:34.114 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-01T01:34:34.114 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-01T01:34:34.114 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-01T01:34:34.114 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-01T01:34:34.114 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:35.376 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:18.2.0-1249.ga6f82997.el9.x86_64 129/163 2023-12-01T01:34:35.420 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 130/163 2023-12-01T01:34:35.474 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:18.2.0-1249.ga6f82997.el9.x86_64 131/163 2023-12-01T01:34:35.508 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 132/163 2023-12-01T01:34:35.758 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 133/163 2023-12-01T01:34:35.883 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 134/163 2023-12-01T01:34:35.921 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 135/163 2023-12-01T01:34:35.965 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:18.2.0-1249.ga6f82997.el9.x86_ 136/163 2023-12-01T01:34:36.060 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 137/163 2023-12-01T01:34:36.167 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 138/163 2023-12-01T01:34:36.209 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 139/163 2023-12-01T01:34:36.261 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 140/163 2023-12-01T01:34:36.316 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 141/163 2023-12-01T01:34:37.411 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 142/163 2023-12-01T01:34:37.455 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 142/163 2023-12-01T01:34:37.516 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 142/163 2023-12-01T01:34:38.097 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:18.2.0-1249.ga6f82997.el9.x86_64 129/163 2023-12-01T01:34:38.143 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 130/163 2023-12-01T01:34:38.198 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:18.2.0-1249.ga6f82997.el9.x86_64 131/163 2023-12-01T01:34:38.238 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 132/163 2023-12-01T01:34:38.508 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 133/163 2023-12-01T01:34:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 134/163 2023-12-01T01:34:38.667 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 135/163 2023-12-01T01:34:38.701 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:18.2.0-1249.ga6f82997.el9.x86_ 136/163 2023-12-01T01:34:38.794 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 137/163 2023-12-01T01:34:38.846 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:18.2.0-1249.ga6f82997.el9.noarch 143/163 2023-12-01T01:34:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 138/163 2023-12-01T01:34:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 139/163 2023-12-01T01:34:38.974 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 140/163 2023-12-01T01:34:38.975 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 144/163 2023-12-01T01:34:39.028 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 141/163 2023-12-01T01:34:39.051 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 145/163 2023-12-01T01:34:39.125 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 146/163 2023-12-01T01:34:39.277 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.4-9.el9.x86_64 147/163 2023-12-01T01:34:39.313 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.7.4-5.el9.x86_64 148/163 2023-12-01T01:34:39.346 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 149/163 2023-12-01T01:34:39.406 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/163 2023-12-01T01:34:39.449 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 151/163 2023-12-01T01:34:39.449 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 152/163 2023-12-01T01:34:39.565 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 152/163 2023-12-01T01:34:39.616 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 153/163 2023-12-01T01:34:39.663 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 154/163 2023-12-01T01:34:39.686 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 155/163 2023-12-01T01:34:39.687 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 156/163 2023-12-01T01:34:39.815 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 156/163 2023-12-01T01:34:39.871 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 157/163 2023-12-01T01:34:39.910 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 158/163 2023-12-01T01:34:39.910 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 158/163 2023-12-01T01:34:39.981 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 142/163 2023-12-01T01:34:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 142/163 2023-12-01T01:34:40.084 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 142/163 2023-12-01T01:34:40.237 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 158/163 2023-12-01T01:34:40.237 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 159/163 2023-12-01T01:34:40.387 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 159/163 2023-12-01T01:34:40.423 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 160/163 2023-12-01T01:34:40.457 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 161/163 2023-12-01T01:34:40.483 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 162/163 2023-12-01T01:34:40.609 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/163 2023-12-01T01:34:40.858 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 163/163 2023-12-01T01:34:40.858 INFO:teuthology.orchestra.run.smithi142.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-01T01:34:40.859 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:41.433 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephfs-top-2:18.2.0-1249.ga6f82997.el9.noarch 143/163 2023-12-01T01:34:41.576 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 144/163 2023-12-01T01:34:41.641 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 145/163 2023-12-01T01:34:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 146/163 2023-12-01T01:34:41.864 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flex-2.6.4-9.el9.x86_64 147/163 2023-12-01T01:34:41.912 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : bison-3.7.4-5.el9.x86_64 148/163 2023-12-01T01:34:41.945 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 149/163 2023-12-01T01:34:41.979 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/163 2023-12-01T01:34:42.021 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 151/163 2023-12-01T01:34:42.021 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 152/163 2023-12-01T01:34:42.204 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 152/163 2023-12-01T01:34:42.255 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 153/163 2023-12-01T01:34:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 154/163 2023-12-01T01:34:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 155/163 2023-12-01T01:34:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 156/163 2023-12-01T01:34:42.462 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 156/163 2023-12-01T01:34:42.493 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 157/163 2023-12-01T01:34:42.533 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 158/163 2023-12-01T01:34:42.533 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 158/163 2023-12-01T01:34:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 158/163 2023-12-01T01:34:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 159/163 2023-12-01T01:34:43.009 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 159/163 2023-12-01T01:34:43.053 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 160/163 2023-12-01T01:34:43.096 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 161/163 2023-12-01T01:34:43.131 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 162/163 2023-12-01T01:34:43.257 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 163/163 2023-12-01T01:34:43.552 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 163/163 2023-12-01T01:34:43.552 INFO:teuthology.orchestra.run.smithi115.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-01T01:34:43.552 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:45.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:18.2.0-1249.ga6f82997.el9.x86_64 1/163 2023-12-01T01:34:45.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:18.2.0-1249.ga6f82997.el9.x86_64 2/163 2023-12-01T01:34:45.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 3/163 2023-12-01T01:34:45.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 4/163 2023-12-01T01:34:45.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 5/163 2023-12-01T01:34:45.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:18.2.0-1249.ga6f82997.el9.x86_64 6/163 2023-12-01T01:34:45.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:18.2.0-1249.ga6f82997.el9.x86_64 7/163 2023-12-01T01:34:45.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:18.2.0-1249.ga6f82997.el9.x86_64 8/163 2023-12-01T01:34:45.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:18.2.0-1249.ga6f82997.el9.x86_64 9/163 2023-12-01T01:34:45.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:18.2.0-1249.ga6f82997.el9.x86_64 10/163 2023-12-01T01:34:45.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:18.2.0-1249.ga6f82997.el9.x86_64 11/163 2023-12-01T01:34:45.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:18.2.0-1249.ga6f82997.el9.x86_64 12/163 2023-12-01T01:34:45.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 13/163 2023-12-01T01:34:45.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:18.2.0-1249.ga6f82997.el9.x86_ 14/163 2023-12-01T01:34:45.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:18.2.0-1249.ga6f82997.el9.x86_64 15/163 2023-12-01T01:34:45.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:18.2.0-1249.ga6f82997.el9.x86_64 16/163 2023-12-01T01:34:45.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:18.2.0-1249.ga6f82997.el9.x86_6 17/163 2023-12-01T01:34:45.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:18.2.0-1249.ga6f82997.el9.x86 18/163 2023-12-01T01:34:45.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:18.2.0-1249.ga6f82997.el9.x86_64 19/163 2023-12-01T01:34:45.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el 20/163 2023-12-01T01:34:45.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:18.2.0-1249.ga6f82997.el9. 21/163 2023-12-01T01:34:45.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:18.2.0-1249.ga6f82997.el9.x86_6 22/163 2023-12-01T01:34:45.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:18.2.0-1249.ga6f82997.el9.x86_64 23/163 2023-12-01T01:34:45.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:18.2.0-1249.ga6f82997.el9.x86_64 24/163 2023-12-01T01:34:45.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:18.2.0-1249.ga6f82997.el9.x86_64 25/163 2023-12-01T01:34:45.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 26/163 2023-12-01T01:34:45.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 27/163 2023-12-01T01:34:45.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:18.2.0-1249.ga6f82997.el9.x86_64 28/163 2023-12-01T01:34:45.532 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997. 29/163 2023-12-01T01:34:45.532 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el9.noa 30/163 2023-12-01T01:34:45.532 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el9.n 31/163 2023-12-01T01:34:45.532 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 32/163 2023-12-01T01:34:45.532 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el 33/163 2023-12-01T01:34:45.532 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el9.noarch 34/163 2023-12-01T01:34:45.532 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.e 35/163 2023-12-01T01:34:45.532 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:18.2.0-1249.ga6f82997.el9.noarch 36/163 2023-12-01T01:34:45.532 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:18.2.0-1249.ga6f82997.el9.noarch 37/163 2023-12-01T01:34:45.533 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:18.2.0-1249.ga6f82997.el9.noarch 38/163 2023-12-01T01:34:45.533 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/163 2023-12-01T01:34:45.533 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/163 2023-12-01T01:34:45.533 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/163 2023-12-01T01:34:45.533 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/163 2023-12-01T01:34:45.533 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/163 2023-12-01T01:34:45.533 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/163 2023-12-01T01:34:45.533 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/163 2023-12-01T01:34:45.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/163 2023-12-01T01:34:45.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : jq-1.6-16.el9.x86_64 47/163 2023-12-01T01:34:45.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/163 2023-12-01T01:34:45.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/163 2023-12-01T01:34:45.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/163 2023-12-01T01:34:45.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/163 2023-12-01T01:34:45.534 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/163 2023-12-01T01:34:45.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/163 2023-12-01T01:34:45.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/163 2023-12-01T01:34:45.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/163 2023-12-01T01:34:45.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/163 2023-12-01T01:34:45.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/163 2023-12-01T01:34:45.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/163 2023-12-01T01:34:45.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/163 2023-12-01T01:34:45.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/163 2023-12-01T01:34:45.535 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/163 2023-12-01T01:34:45.536 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/163 2023-12-01T01:34:45.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/163 2023-12-01T01:34:45.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/163 2023-12-01T01:34:45.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/163 2023-12-01T01:34:45.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/163 2023-12-01T01:34:45.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/163 2023-12-01T01:34:45.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/163 2023-12-01T01:34:45.537 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/163 2023-12-01T01:34:45.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/163 2023-12-01T01:34:45.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/163 2023-12-01T01:34:45.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/163 2023-12-01T01:34:45.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/163 2023-12-01T01:34:45.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/163 2023-12-01T01:34:45.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/163 2023-12-01T01:34:45.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/163 2023-12-01T01:34:45.538 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/163 2023-12-01T01:34:45.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/163 2023-12-01T01:34:45.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/163 2023-12-01T01:34:45.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/163 2023-12-01T01:34:45.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/163 2023-12-01T01:34:45.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/163 2023-12-01T01:34:45.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/163 2023-12-01T01:34:45.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/163 2023-12-01T01:34:45.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/163 2023-12-01T01:34:45.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/163 2023-12-01T01:34:45.539 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 108/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/163 2023-12-01T01:34:45.540 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/163 2023-12-01T01:34:45.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/163 2023-12-01T01:34:45.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/163 2023-12-01T01:34:45.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/163 2023-12-01T01:34:45.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/163 2023-12-01T01:34:45.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/163 2023-12-01T01:34:45.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/163 2023-12-01T01:34:45.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/163 2023-12-01T01:34:45.541 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/163 2023-12-01T01:34:45.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/163 2023-12-01T01:34:45.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/163 2023-12-01T01:34:45.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/163 2023-12-01T01:34:45.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/163 2023-12-01T01:34:45.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/163 2023-12-01T01:34:45.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/163 2023-12-01T01:34:45.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/163 2023-12-01T01:34:45.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/163 2023-12-01T01:34:45.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/163 2023-12-01T01:34:45.542 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:18.2.0-1249.ga6f82997.el9.x86_64 134/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:18.2.0-1249.ga6f82997.el9.x86_64 136/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : acl-2.3.1-4.el9.x86_64 140/163 2023-12-01T01:34:45.543 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : acl-2.3.1-3.el9.x86_64 141/163 2023-12-01T01:34:45.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 142/163 2023-12-01T01:34:45.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 143/163 2023-12-01T01:34:45.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 144/163 2023-12-01T01:34:45.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 145/163 2023-12-01T01:34:45.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 146/163 2023-12-01T01:34:45.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 147/163 2023-12-01T01:34:45.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 148/163 2023-12-01T01:34:45.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 149/163 2023-12-01T01:34:45.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 150/163 2023-12-01T01:34:45.544 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 151/163 2023-12-01T01:34:45.545 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 152/163 2023-12-01T01:34:45.545 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 153/163 2023-12-01T01:34:45.545 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 154/163 2023-12-01T01:34:45.545 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 155/163 2023-12-01T01:34:45.545 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 156/163 2023-12-01T01:34:45.545 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 157/163 2023-12-01T01:34:45.545 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 158/163 2023-12-01T01:34:45.545 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 159/163 2023-12-01T01:34:45.545 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 160/163 2023-12-01T01:34:45.546 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 161/163 2023-12-01T01:34:45.546 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 162/163 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 163/163 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout: acl-2.3.1-4.el9.x86_64 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-01T01:34:47.720 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.2-8.el9.noarch 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.7.4-5.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.721 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.722 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.4-9.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: fmt-8.1.1-5.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-01T01:34:47.723 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-01T01:34:47.724 INFO:teuthology.orchestra.run.smithi142.stdout: jq-1.6-16.el9.x86_64 2023-12-01T01:34:47.724 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-01T01:34:47.724 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-9.el9.x86_64 2023-12-01T01:34:47.724 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-12-01T01:34:47.724 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.724 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.724 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.724 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-01T01:34:47.725 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-13.11-1.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-01T01:34:47.726 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-01T01:34:47.727 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-01T01:34:47.728 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-01T01:34:47.729 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.730 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.731 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:47.731 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-3.el9.x86_64 2023-12-01T01:34:47.731 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-01T01:34:47.731 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-01T01:34:47.731 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-01T01:34:47.731 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-01T01:34:47.731 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-01T01:34:47.731 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-01T01:34:47.731 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:47.731 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2023-12-01T01:34:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:18.2.0-1249.ga6f82997.el9.x86_64 1/163 2023-12-01T01:34:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:18.2.0-1249.ga6f82997.el9.x86_64 2/163 2023-12-01T01:34:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 3/163 2023-12-01T01:34:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 4/163 2023-12-01T01:34:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 5/163 2023-12-01T01:34:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:18.2.0-1249.ga6f82997.el9.x86_64 6/163 2023-12-01T01:34:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:18.2.0-1249.ga6f82997.el9.x86_64 7/163 2023-12-01T01:34:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:18.2.0-1249.ga6f82997.el9.x86_64 8/163 2023-12-01T01:34:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:18.2.0-1249.ga6f82997.el9.x86_64 9/163 2023-12-01T01:34:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:18.2.0-1249.ga6f82997.el9.x86_64 10/163 2023-12-01T01:34:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:18.2.0-1249.ga6f82997.el9.x86_64 11/163 2023-12-01T01:34:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:18.2.0-1249.ga6f82997.el9.x86_64 12/163 2023-12-01T01:34:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 13/163 2023-12-01T01:34:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:18.2.0-1249.ga6f82997.el9.x86_ 14/163 2023-12-01T01:34:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:18.2.0-1249.ga6f82997.el9.x86_64 15/163 2023-12-01T01:34:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:18.2.0-1249.ga6f82997.el9.x86_64 16/163 2023-12-01T01:34:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:18.2.0-1249.ga6f82997.el9.x86_6 17/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:18.2.0-1249.ga6f82997.el9.x86 18/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:18.2.0-1249.ga6f82997.el9.x86_64 19/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el 20/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:18.2.0-1249.ga6f82997.el9. 21/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:18.2.0-1249.ga6f82997.el9.x86_6 22/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:18.2.0-1249.ga6f82997.el9.x86_64 23/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:18.2.0-1249.ga6f82997.el9.x86_64 24/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:18.2.0-1249.ga6f82997.el9.x86_64 25/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 26/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 27/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:18.2.0-1249.ga6f82997.el9.x86_64 28/163 2023-12-01T01:34:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997. 29/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el9.noa 30/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el9.n 31/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 32/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el 33/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el9.noarch 34/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.e 35/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:18.2.0-1249.ga6f82997.el9.noarch 36/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:18.2.0-1249.ga6f82997.el9.noarch 37/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephfs-top-2:18.2.0-1249.ga6f82997.el9.noarch 38/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/163 2023-12-01T01:34:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/163 2023-12-01T01:34:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/163 2023-12-01T01:34:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/163 2023-12-01T01:34:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/163 2023-12-01T01:34:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : jq-1.6-16.el9.x86_64 47/163 2023-12-01T01:34:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/163 2023-12-01T01:34:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/163 2023-12-01T01:34:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/163 2023-12-01T01:34:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/163 2023-12-01T01:34:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/163 2023-12-01T01:34:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/163 2023-12-01T01:34:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 108/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/163 2023-12-01T01:34:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/163 2023-12-01T01:34:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:18.2.0-1249.ga6f82997.el9.x86_64 134/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:18.2.0-1249.ga6f82997.el9.x86_64 136/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : acl-2.3.1-4.el9.x86_64 140/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : acl-2.3.1-3.el9.x86_64 141/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 142/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 143/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 144/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 145/163 2023-12-01T01:34:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 146/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 147/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 148/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 149/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 150/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 151/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 152/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 153/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 154/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 155/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 156/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 157/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 158/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 159/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 160/163 2023-12-01T01:34:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 161/163 2023-12-01T01:34:48.141 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 162/163 2023-12-01T01:34:48.293 DEBUG:teuthology.parallel:result is None 2023-12-01T01:34:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 163/163 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: acl-2.3.1-4.el9.x86_64 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-01T01:34:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: automake-1.16.2-8.el9.noarch 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: bison-3.7.4-5.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: cephfs-top-2:18.2.0-1249.ga6f82997.el9.noarch 2023-12-01T01:34:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-01T01:34:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: flex-2.6.4-9.el9.x86_64 2023-12-01T01:34:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-01T01:34:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-01T01:34:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-01T01:34:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: fmt-8.1.1-5.el9.x86_64 2023-12-01T01:34:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-01T01:34:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-01T01:34:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: jq-1.6-16.el9.x86_64 2023-12-01T01:34:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-01T01:34:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-9.el9.x86_64 2023-12-01T01:34:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-12-01T01:34:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-01T01:34:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-01T01:34:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-01T01:34:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-01T01:34:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-01T01:34:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-12-01T01:34:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-13.11-1.el9.x86_64 2023-12-01T01:34:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-01T01:34:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-01T01:34:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-01T01:34:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-01T01:34:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-01T01:34:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-01T01:34:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-01T01:34:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-01T01:34:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-01T01:34:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-01T01:34:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-01T01:34:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-01T01:34:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-01T01:34:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-01T01:34:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-01T01:34:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-01T01:34:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-01T01:34:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-01T01:34:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:18.2.0-1249.ga6f82997.el9.x86_64 2023-12-01T01:34:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-3.el9.x86_64 2023-12-01T01:34:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-01T01:34:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-01T01:34:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-01T01:34:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-01T01:34:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-01T01:34:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-01T01:34:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:50.263 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2023-12-01T01:34:50.839 DEBUG:teuthology.parallel:result is None 2023-12-01T01:34:50.839 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:34:51.084 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-01T01:34:51.153 INFO:teuthology.orchestra.run.smithi115.stdout:18.2.0-1249.ga6f82997.el9 2023-12-01T01:34:51.153 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1249.ga6f82997.el9 2023-12-01T01:34:51.154 INFO:teuthology.task.install:The correct ceph version 18.2.0-1249.ga6f82997 is installed. 2023-12-01T01:34:51.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:34:51.293 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-01T01:34:51.357 INFO:teuthology.orchestra.run.smithi142.stdout:18.2.0-1249.ga6f82997.el9 2023-12-01T01:34:51.357 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1249.ga6f82997.el9 2023-12-01T01:34:51.357 INFO:teuthology.task.install:The correct ceph version 18.2.0-1249.ga6f82997 is installed. 2023-12-01T01:34:51.357 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-01T01:34:51.358 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:51.358 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-01T01:34:51.442 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:34:51.442 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-01T01:34:51.525 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-01T01:34:51.526 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:51.526 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2023-12-01T01:34:51.596 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-01T01:34:51.712 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:34:51.713 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2023-12-01T01:34:51.789 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-01T01:34:51.904 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-01T01:34:51.904 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:51.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-01T01:34:51.978 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-01T01:34:52.095 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:34:52.095 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-01T01:34:52.174 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-01T01:34:52.291 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-01T01:34:52.292 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:52.292 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2023-12-01T01:34:52.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-01T01:34:52.498 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:34:52.498 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2023-12-01T01:34:52.587 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-01T01:34:52.657 INFO:teuthology.run_tasks:Running task ceph... 2023-12-01T01:34:52.732 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-01T01:34:52.732 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /var/log/ceph 2023-12-01T01:34:52.739 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /var/log/ceph 2023-12-01T01:34:52.813 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-01T01:34:52.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-01T01:34:52.819 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-01T01:34:52.889 INFO:tasks.ceph:Creating extra log directories... 2023-12-01T01:34:52.890 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-01T01:34:52.894 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-01T01:34:52.992 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-12-01T01:34:52.992 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-12-01T01:34:52.992 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-12-01T01:34:52.993 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:52.993 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-01T01:34:53.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-01T01:34:53.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-01T01:34:53.329 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:34:53.329 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-01T01:34:53.376 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-01T01:34:53.591 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-01T01:34:53.669 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-01T01:34:53.669 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-01T01:34:53.670 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199', 'branch': 'reef-release', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7473199', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef-release'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef-release', 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 55, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8832, 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'reef-release', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi115.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAG1ZN4VL+XG0I899Gyv9++bmYkxtxRfIGXaOnv+Y5GEYm47zUgmZY1Nos8WsFkWxPJZYnlnIPcgDlygBFmRhMI=', 'smithi142.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrWw7VN9mng46aNqeugrNH8siuKXNZ4jODSxOlLGm+vYJVed2n0zJzYcvmZ9uTx05qQgz+NtqqnVuk7wDuI/G8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'f2de108beb0a6dde7f3e6c96039071828a1f6269', 'timestamp': '2023-11-30_22:47:08', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-12-01T01:34:53.670 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-01T01:34:53.676 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-01T01:34:53.717 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/run/ceph 2023-12-01T01:34:53.763 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/run/ceph 2023-12-01T01:34:53.841 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:53.841 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2023-12-01T01:34:53.880 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-01T01:34:53.881 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2023-12-01T01:34:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-01T01:34:53.962 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:34:53.962 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 632 Links: 1 2023-12-01T01:34:53.962 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:34:53.962 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:34:53.962 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-12-01 01:34:43.979127729 +0000 2023-12-01T01:34:53.962 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-12-01 01:32:04.798946558 +0000 2023-12-01T01:34:53.962 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-12-01 01:32:04.798946558 +0000 2023-12-01T01:34:53.962 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2023-12-01 01:32:04.798946558 +0000 2023-12-01T01:34:53.962 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-01T01:34:54.074 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-12-01T01:34:54.074 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-12-01T01:34:54.074 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000371506 s, 1.4 MB/s 2023-12-01T01:34:54.077 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-01T01:34:54.119 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2023-12-01T01:34:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-01T01:34:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:34:54.197 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-12-01T01:34:54.197 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:34:54.197 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:34:54.197 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-12-01 01:34:43.980127710 +0000 2023-12-01T01:34:54.198 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-12-01 01:32:05.836927121 +0000 2023-12-01T01:34:54.198 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-12-01 01:32:05.836927121 +0000 2023-12-01T01:34:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2023-12-01 01:32:05.836927121 +0000 2023-12-01T01:34:54.198 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-01T01:34:54.311 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-12-01T01:34:54.311 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-12-01T01:34:54.311 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000278338 s, 1.8 MB/s 2023-12-01T01:34:54.313 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-01T01:34:54.353 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2023-12-01T01:34:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-01T01:34:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:34:54.435 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-12-01T01:34:54.435 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:34:54.435 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:34:54.435 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-12-01 01:34:43.981127692 +0000 2023-12-01T01:34:54.435 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-12-01 01:32:06.860907947 +0000 2023-12-01T01:34:54.435 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-12-01 01:32:06.860907947 +0000 2023-12-01T01:34:54.435 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2023-12-01 01:32:06.860907947 +0000 2023-12-01T01:34:54.435 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-01T01:34:54.547 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-12-01T01:34:54.547 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-12-01T01:34:54.547 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000289774 s, 1.8 MB/s 2023-12-01T01:34:54.549 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-01T01:34:54.592 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2023-12-01T01:34:54.673 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-01T01:34:54.673 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:34:54.673 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 691 Links: 1 2023-12-01T01:34:54.673 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:34:54.673 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:34:54.673 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2023-12-01 01:34:43.982127673 +0000 2023-12-01T01:34:54.673 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2023-12-01 01:32:07.884888773 +0000 2023-12-01T01:34:54.673 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2023-12-01 01:32:07.884888773 +0000 2023-12-01T01:34:54.673 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2023-12-01 01:32:07.884888773 +0000 2023-12-01T01:34:54.674 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-01T01:34:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2023-12-01T01:34:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2023-12-01T01:34:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000290128 s, 1.8 MB/s 2023-12-01T01:34:54.792 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-01T01:34:54.838 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-01T01:34:54.838 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:34:54.838 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2023-12-01T01:34:54.891 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-01T01:34:54.891 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2023-12-01T01:34:54.983 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-01T01:34:54.983 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:34:54.983 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 631 Links: 1 2023-12-01T01:34:54.983 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:34:54.983 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:34:54.983 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-12-01 01:34:41.303181699 +0000 2023-12-01T01:34:54.983 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-12-01 01:32:04.797448662 +0000 2023-12-01T01:34:54.984 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-12-01 01:32:04.797448662 +0000 2023-12-01T01:34:54.984 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2023-12-01 01:32:04.797448662 +0000 2023-12-01T01:34:54.984 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-01T01:34:55.058 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-12-01T01:34:55.058 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-12-01T01:34:55.058 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000387503 s, 1.3 MB/s 2023-12-01T01:34:55.061 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-01T01:34:55.112 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2023-12-01T01:34:55.203 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-01T01:34:55.203 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:34:55.203 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-12-01T01:34:55.203 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:34:55.204 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:34:55.204 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-12-01 01:34:41.303181699 +0000 2023-12-01T01:34:55.204 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-12-01 01:32:05.839429241 +0000 2023-12-01T01:34:55.204 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-12-01 01:32:05.839429241 +0000 2023-12-01T01:34:55.204 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2023-12-01 01:32:05.839429241 +0000 2023-12-01T01:34:55.204 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-01T01:34:55.332 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-12-01T01:34:55.332 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-12-01T01:34:55.332 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00029826 s, 1.7 MB/s 2023-12-01T01:34:55.335 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-01T01:34:55.381 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2023-12-01T01:34:55.472 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-01T01:34:55.472 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:34:55.472 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 670 Links: 1 2023-12-01T01:34:55.472 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:34:55.472 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:34:55.472 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-12-01 01:34:41.304181680 +0000 2023-12-01T01:34:55.472 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-12-01 01:32:06.857410266 +0000 2023-12-01T01:34:55.472 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-12-01 01:32:06.857410266 +0000 2023-12-01T01:34:55.472 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2023-12-01 01:32:06.857410266 +0000 2023-12-01T01:34:55.473 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-01T01:34:55.551 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-12-01T01:34:55.553 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-12-01T01:34:55.553 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000297357 s, 1.7 MB/s 2023-12-01T01:34:55.555 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-01T01:34:55.608 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2023-12-01T01:34:55.648 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-01T01:34:55.648 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:34:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-12-01T01:34:55.648 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:34:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:34:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2023-12-01 01:34:41.304181680 +0000 2023-12-01T01:34:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2023-12-01 01:32:07.876391273 +0000 2023-12-01T01:34:55.649 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2023-12-01 01:32:07.876391273 +0000 2023-12-01T01:34:55.649 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2023-12-01 01:32:07.876391273 +0000 2023-12-01T01:34:55.649 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-01T01:34:55.779 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2023-12-01T01:34:55.779 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2023-12-01T01:34:55.779 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000332228 s, 1.5 MB/s 2023-12-01T01:34:55.785 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-01T01:34:55.838 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-01T01:34:55.838 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-01T01:34:55.838 INFO:tasks.ceph:Generating config... 2023-12-01T01:34:55.839 INFO:tasks.ceph:[client] client mount timeout = 600 2023-12-01T01:34:55.839 INFO:tasks.ceph:[client] debug client = 20 2023-12-01T01:34:55.840 INFO:tasks.ceph:[client] debug ms = 1 2023-12-01T01:34:55.840 INFO:tasks.ceph:[client] pid file = 2023-12-01T01:34:55.840 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-12-01T01:34:55.840 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-12-01T01:34:55.840 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-12-01T01:34:55.840 INFO:tasks.ceph:[mds] debug mds = 20 2023-12-01T01:34:55.840 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-12-01T01:34:55.840 INFO:tasks.ceph:[mds] debug ms = 1 2023-12-01T01:34:55.840 INFO:tasks.ceph:[mds] mds debug frag = True 2023-12-01T01:34:55.840 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-12-01T01:34:55.840 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-12-01T01:34:55.840 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-12-01T01:34:55.840 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-12-01T01:34:55.840 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-12-01T01:34:55.840 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-12-01T01:34:55.841 INFO:tasks.ceph:[mgr] debug client = 20 2023-12-01T01:34:55.841 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-12-01T01:34:55.841 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-01T01:34:55.841 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-01T01:34:55.841 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-01T01:34:55.841 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-01T01:34:55.841 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-01T01:34:55.841 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-12-01T01:34:55.841 INFO:tasks.ceph:[osd] bdev async discard = True 2023-12-01T01:34:55.841 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-12-01T01:34:55.841 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-12-01T01:34:55.841 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-01T01:34:55.841 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-01T01:34:55.841 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-12-01T01:34:55.841 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-12-01T01:34:55.841 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-01T01:34:55.841 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-01T01:34:55.842 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-12-01T01:34:55.842 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-01T01:34:55.842 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-01T01:34:55.842 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-01T01:34:55.842 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-12-01T01:34:55.842 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-12-01T01:34:55.842 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-12-01T01:34:55.842 INFO:tasks.ceph:Setting up mon.a... 2023-12-01T01:34:55.842 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-01T01:34:56.010 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.keyring 2023-12-01T01:34:56.018 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-01T01:34:56.146 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-01T01:34:56.240 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.115'), ('mon.b', '172.21.15.142'), ('mon.c', '[v2:172.21.15.142:3301,v1:172.21.15.142:6790]')] 2023-12-01T01:34:56.240 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.115,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-12-01T01:34:56.241 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:56.241 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-01T01:34:56.292 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.115 --add b 172.21.15.142 --addv c '[v2:172.21.15.142:3301,v1:172.21.15.142:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:34:56.427 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:34:56.427 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: generated fsid 5ba3726b-7abf-4a38-892b-ec2cd5ba09fc 2023-12-01T01:34:56.427 INFO:teuthology.orchestra.run.smithi115.stdout:setting min_mon_release = pacific 2023-12-01T01:34:56.427 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 0 2023-12-01T01:34:56.427 INFO:teuthology.orchestra.run.smithi115.stdout:fsid 5ba3726b-7abf-4a38-892b-ec2cd5ba09fc 2023-12-01T01:34:56.427 INFO:teuthology.orchestra.run.smithi115.stdout:last_changed 2023-12-01T01:34:56.426830+0000 2023-12-01T01:34:56.427 INFO:teuthology.orchestra.run.smithi115.stdout:created 2023-12-01T01:34:56.426830+0000 2023-12-01T01:34:56.427 INFO:teuthology.orchestra.run.smithi115.stdout:min_mon_release 16 (pacific) 2023-12-01T01:34:56.427 INFO:teuthology.orchestra.run.smithi115.stdout:election_strategy: 1 2023-12-01T01:34:56.427 INFO:teuthology.orchestra.run.smithi115.stdout:0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2023-12-01T01:34:56.428 INFO:teuthology.orchestra.run.smithi115.stdout:1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2023-12-01T01:34:56.428 INFO:teuthology.orchestra.run.smithi115.stdout:2: [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] mon.c 2023-12-01T01:34:56.428 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-01T01:34:56.430 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-01T01:34:56.475 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5ba3726b-7abf-4a38-892b-ec2cd5ba09fc... 2023-12-01T01:34:56.476 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-01T01:34:56.523 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-01T01:34:56.672 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2023-12-01T01:34:56.672 INFO:teuthology.orchestra.run.smithi115.stdout: chdir = "" 2023-12-01T01:34:56.672 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-01T01:34:56.672 INFO:teuthology.orchestra.run.smithi115.stdout: auth supported = cephx 2023-12-01T01:34:56.672 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: filestore xattr use omap = true 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: mon clock drift allowed = 1.000 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: osd crush chooseleaf type = 0 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: auth debug = true 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on old message = true 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on bug = true 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: mon pg warn max object skew = 0 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: # disable pg_autoscaler by default for new pools 2023-12-01T01:34:56.673 INFO:teuthology.orchestra.run.smithi115.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default size = 2 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow primary affinity = true 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow pg remap = true 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on legacy crush tunables = false 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on crush straw calc version zero = false 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on no sortbitwise = false 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on osd down out interval zero = false 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on too few osds = false 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-01T01:34:56.674 INFO:teuthology.orchestra.run.smithi115.stdout: mon_allow_pool_size_one = true 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: osd default data pool replay window = 5 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: mon allow pool delete = true 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: mon cluster log file level = debug 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: debug asserts on shutdown = true 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: mon health detail to clog = false 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: mon host = "172.21.15.115,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on skipped message = False 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = 5ba3726b-7abf-4a38-892b-ec2cd5ba09fc 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.675 INFO:teuthology.orchestra.run.smithi115.stdout:[osd] 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: osd journal size = 100 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub load threshold = 5.0 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub max interval = 600 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock profile = high_recovery_ops 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: osd recover clone overlap = true 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: osd recovery max chunk = 1048576 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug shutdown = true 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug op order = true 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify stray on activate = true 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.676 INFO:teuthology.orchestra.run.smithi115.stdout: osd open classes on start = true 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug pg log writeout = true 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: osd deep scrub update digest min age = 30 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: osd map max advance = 10 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: journal zero on create = true 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: filestore ondisk finisher threads = 3 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: filestore apply finisher threads = 3 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: bdev debug aio = true 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug misdirected ops = true 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: bdev async discard = True 2023-12-01T01:34:56.677 INFO:teuthology.orchestra.run.smithi115.stdout: bdev enable discard = True 2023-12-01T01:34:56.678 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore allocator = bitmap 2023-12-01T01:34:56.678 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore block size = 96636764160 2023-12-01T01:34:56.678 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore fsck on mount = True 2023-12-01T01:34:56.678 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluefs = 1/20 2023-12-01T01:34:56.678 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluestore = 1/20 2023-12-01T01:34:56.678 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: debug osd = 20 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: debug rocksdb = 4/10 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd backfillfull_ratio = 0.85 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd full ratio = 0.9 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd nearfull ratio = 0.8 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: osd failsafe full ratio = 0.95 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: osd objectstore = bluestore 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: osd op complaint time = 180 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr] 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: debug mgr = 20 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2023-12-01T01:34:56.679 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: mgr/telemetry/nag = false 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: debug client = 20 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: debug finisher = 20 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout:[mon] 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: debug paxos = 20 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: mon data avail warn = 5 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: mon mgr mkfs grace = 240 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd reporter subtree level = osd 2023-12-01T01:34:56.680 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd prime pg temp = true 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: auth mon ticket ttl = 660 # 11m 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: auth service ticket ttl = 240 # 4m 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: # don't complain about insecure global_id in the test suite 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: # 1m isn't quite enough 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: mon_down_mkfs_grace = 2m 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_filestore_osds = false 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: mon op complaint time = 120 2023-12-01T01:34:56.681 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout:[client] 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: rgw cache enabled = true 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable ops log = true 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable usage log = true 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: client mount timeout = 600 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: debug client = 20 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = "" 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: rados mon op timeout = 900 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout: rados osd op timeout = 900 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.a] 2023-12-01T01:34:56.682 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.b] 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.c] 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.a] 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.c] 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.b] 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.d] 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout:[mds] 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout: debug mds = 20 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout: debug mds balancer = 20 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout: mds debug frag = True 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout: mds debug scatterstat = True 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout: mds op complaint time = 180 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout: mds verify scatter = True 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout: osd op complaint time = 180 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout: rados mon op timeout = 900 2023-12-01T01:34:56.683 INFO:teuthology.orchestra.run.smithi115.stdout: rados osd op timeout = 900 2023-12-01T01:34:56.701 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.702 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2023-12-01T01:34:56.703 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2023-12-01T01:34:56.704 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.115,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on skipped message = False 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 5ba3726b-7abf-4a38-892b-ec2cd5ba09fc 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.705 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2023-12-01T01:34:56.706 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: bdev async discard = True 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: bdev enable discard = True 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore allocator = bitmap 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 1/20 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 1/20 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 4/10 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2023-12-01T01:34:56.707 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: debug finisher = 20 2023-12-01T01:34:56.708 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2023-12-01T01:34:56.709 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2023-12-01T01:34:56.710 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2023-12-01T01:34:56.711 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2023-12-01T01:34:56.712 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2023-12-01T01:34:56.746 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-01T01:34:56.747 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-01T01:34:56.877 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-01T01:34:56.877 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:56.877 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-01T01:34:56.917 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:56.918 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-01T01:34:57.006 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:34:57.006 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:57.006 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-01T01:34:57.006 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-01T01:34:57.177 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:34:57.178 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:34:57.265 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T01:34:57.266 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:34:57.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-01T01:34:57.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-01T01:34:57.428 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:34:57.428 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:34:57.527 INFO:tasks.ceph:Setting up mon nodes... 2023-12-01T01:34:57.527 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-01T01:34:57.527 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-01T01:34:57.709 INFO:teuthology.orchestra.run.smithi115.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-01T01:34:57.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-01T01:34:57.919 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-01T01:34:57.930 INFO:tasks.ceph:Setting up mds nodes... 2023-12-01T01:34:57.931 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-12-01T01:34:58.118 INFO:teuthology.orchestra.run.smithi115.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-12-01T01:34:58.127 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-12-01T01:34:58.222 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-12-01T01:34:58.406 INFO:teuthology.orchestra.run.smithi115.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-12-01T01:34:58.415 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-12-01T01:34:58.515 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-12-01T01:34:58.706 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-12-01T01:34:58.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-12-01T01:34:58.812 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-12-01T01:34:59.000 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-12-01T01:34:59.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-12-01T01:34:59.097 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-01T01:34:59.098 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-01T01:34:59.228 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-01T01:34:59.293 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-12-01T01:34:59.418 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-12-01T01:34:59.483 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-01T01:34:59.618 INFO:teuthology.orchestra.run.smithi142.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-12-01T01:34:59.670 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-12-01T01:34:59.794 INFO:teuthology.orchestra.run.smithi142.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-12-01T01:34:59.854 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-01T01:34:59.854 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-01T01:34:59.854 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-01T01:34:59.960 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-01T01:34:59.960 INFO:tasks.ceph:role: osd.0 2023-12-01T01:34:59.960 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:34:59.960 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-01T01:35:00.069 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-01T01:35:00.069 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-01T01:35:00.070 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-01T01:35:00.070 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-01T01:35:00.070 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-01T01:35:00.070 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2023-12-01T01:35:00.070 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-01T01:35:00.070 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-01T01:35:00.070 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-01T01:35:00.070 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-01T01:35:00.579 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2023-12-01T01:35:00.586 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2023-12-01T01:35:00.587 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-01T01:35:00.781 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-01T01:35:00.893 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-01T01:35:00.983 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-01T01:35:00.983 INFO:tasks.ceph:role: osd.1 2023-12-01T01:35:00.983 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:35:00.983 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-01T01:35:01.080 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-01T01:35:01.080 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-01T01:35:01.081 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-01T01:35:01.081 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-01T01:35:01.081 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-01T01:35:01.081 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2023-12-01T01:35:01.081 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-01T01:35:01.081 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-01T01:35:01.081 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-01T01:35:01.081 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-01T01:35:01.562 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2023-12-01T01:35:01.568 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2023-12-01T01:35:01.568 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-01T01:35:01.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-01T01:35:01.812 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-01T01:35:01.906 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-01T01:35:01.906 INFO:tasks.ceph:role: osd.2 2023-12-01T01:35:01.906 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:35:01.906 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-01T01:35:02.004 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-01T01:35:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-01T01:35:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-01T01:35:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-01T01:35:02.005 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-01T01:35:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2023-12-01T01:35:02.005 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-01T01:35:02.005 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-01T01:35:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-01T01:35:02.005 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-01T01:35:02.521 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2023-12-01T01:35:02.527 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2023-12-01T01:35:02.528 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-01T01:35:02.736 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-01T01:35:02.850 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-01T01:35:02.973 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-01T01:35:02.973 INFO:tasks.ceph:role: osd.3 2023-12-01T01:35:02.973 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:35:02.973 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-01T01:35:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-01T01:35:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-01T01:35:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-01T01:35:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-01T01:35:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-01T01:35:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2023-12-01T01:35:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-01T01:35:03.074 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-01T01:35:03.075 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-01T01:35:03.075 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-01T01:35:03.547 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2023-12-01T01:35:03.554 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2023-12-01T01:35:03.554 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-01T01:35:03.705 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-01T01:35:03.804 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:35:03.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:03.950+0000 7fb9b05a9500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-01T01:35:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:03.956+0000 7fb9b05a9500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-01T01:35:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:03.956+0000 7fb9b05a9500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-01T01:35:03.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:03.956+0000 7fb9b05a9500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-01T01:35:03.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:03.956+0000 7fb9b05a9500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-01T01:35:03.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:03.966+0000 7fb9b05a9500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-12-01T01:35:06.913 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-01T01:35:07.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:35:07.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:07.151+0000 7f2278810500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-01T01:35:07.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:07.152+0000 7f2278810500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-01T01:35:07.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:07.152+0000 7f2278810500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-01T01:35:07.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:07.152+0000 7f2278810500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-01T01:35:07.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:07.152+0000 7f2278810500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-01T01:35:07.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:07.154+0000 7f2278810500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-12-01T01:35:10.334 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-01T01:35:10.434 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:35:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:10.568+0000 7f69a3f8b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-01T01:35:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:10.569+0000 7f69a3f8b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-01T01:35:10.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:10.569+0000 7f69a3f8b500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-01T01:35:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:10.569+0000 7f69a3f8b500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-01T01:35:10.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:10.569+0000 7f69a3f8b500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-01T01:35:10.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:10.571+0000 7f69a3f8b500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-12-01T01:35:13.490 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-01T01:35:13.589 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:35:13.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:13.730+0000 7f4e96b8b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-01T01:35:13.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:13.731+0000 7f4e96b8b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-01T01:35:13.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:13.731+0000 7f4e96b8b500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-01T01:35:13.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:13.731+0000 7f4e96b8b500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-01T01:35:13.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:13.731+0000 7f4e96b8b500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-01T01:35:13.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:13.733+0000 7f4e96b8b500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-12-01T01:35:16.912 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-01T01:35:17.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-01T01:35:17.109 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-01T01:35:17.109 INFO:tasks.ceph:role: osd.4 2023-12-01T01:35:17.109 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T01:35:17.110 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-01T01:35:17.208 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-01T01:35:17.208 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-01T01:35:17.208 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-01T01:35:17.208 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-01T01:35:17.208 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-01T01:35:17.209 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2023-12-01T01:35:17.209 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-01T01:35:17.209 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-01T01:35:17.209 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-01T01:35:17.209 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-01T01:35:17.729 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2023-12-01T01:35:17.735 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2023-12-01T01:35:17.735 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-01T01:35:17.906 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-01T01:35:18.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-01T01:35:18.097 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-01T01:35:18.097 INFO:tasks.ceph:role: osd.5 2023-12-01T01:35:18.097 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T01:35:18.097 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-01T01:35:18.189 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-01T01:35:18.189 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-01T01:35:18.189 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-01T01:35:18.190 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-01T01:35:18.190 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-01T01:35:18.190 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2023-12-01T01:35:18.190 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-01T01:35:18.190 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-01T01:35:18.190 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-01T01:35:18.190 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-01T01:35:18.666 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2023-12-01T01:35:18.671 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2023-12-01T01:35:18.671 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-01T01:35:18.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-01T01:35:18.870 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-01T01:35:18.951 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-01T01:35:18.951 INFO:tasks.ceph:role: osd.6 2023-12-01T01:35:18.951 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T01:35:18.951 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-01T01:35:19.032 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-01T01:35:19.032 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-01T01:35:19.032 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-01T01:35:19.033 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-01T01:35:19.033 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-01T01:35:19.033 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2023-12-01T01:35:19.033 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-01T01:35:19.033 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-01T01:35:19.033 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-01T01:35:19.034 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-01T01:35:19.557 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2023-12-01T01:35:19.562 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2023-12-01T01:35:19.562 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-01T01:35:19.740 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-01T01:35:19.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-01T01:35:19.939 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-01T01:35:19.939 INFO:tasks.ceph:role: osd.7 2023-12-01T01:35:19.939 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T01:35:19.940 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-01T01:35:20.038 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-01T01:35:20.038 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-01T01:35:20.038 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-01T01:35:20.038 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-01T01:35:20.038 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-01T01:35:20.038 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2023-12-01T01:35:20.038 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-01T01:35:20.039 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-01T01:35:20.039 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-01T01:35:20.039 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-01T01:35:20.486 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2023-12-01T01:35:20.492 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2023-12-01T01:35:20.493 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-01T01:35:20.622 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-01T01:35:20.704 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:35:20.841 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:20.840+0000 7faf78cc0500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-01T01:35:20.842 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:20.841+0000 7faf78cc0500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-01T01:35:20.842 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:20.841+0000 7faf78cc0500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-01T01:35:20.842 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:20.841+0000 7faf78cc0500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-01T01:35:20.842 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:20.841+0000 7faf78cc0500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-01T01:35:20.845 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:20.843+0000 7faf78cc0500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-12-01T01:35:23.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:35:23.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:35:23.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-01T01:35:23.886 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:35:24.066 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:24.065+0000 7fbd481e2500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-01T01:35:24.066 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:24.066+0000 7fbd481e2500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-01T01:35:24.067 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:24.066+0000 7fbd481e2500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-01T01:35:24.067 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:24.066+0000 7fbd481e2500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-01T01:35:24.067 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:24.066+0000 7fbd481e2500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-01T01:35:24.069 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:24.068+0000 7fbd481e2500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-12-01T01:35:27.261 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-01T01:35:27.362 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:35:27.506 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:27.505+0000 7f47b9374500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-01T01:35:27.507 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:27.506+0000 7f47b9374500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-01T01:35:27.507 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:27.506+0000 7f47b9374500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-01T01:35:27.507 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:27.506+0000 7f47b9374500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-01T01:35:27.507 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:27.506+0000 7f47b9374500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-01T01:35:27.509 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:27.508+0000 7f47b9374500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-12-01T01:35:30.691 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-01T01:35:30.782 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:35:30.923 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:30.922+0000 7f4bd1b74500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-01T01:35:30.924 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:30.923+0000 7f4bd1b74500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-01T01:35:30.924 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:30.923+0000 7f4bd1b74500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-01T01:35:30.924 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:30.923+0000 7f4bd1b74500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-01T01:35:30.924 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:30.923+0000 7f4bd1b74500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-01T01:35:30.925 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:30.925+0000 7f4bd1b74500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-12-01T01:35:34.101 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-01T01:35:34.198 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-01T01:35:34.198 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:34.198 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-01T01:35:34.300 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:34.300 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-12-01T01:35:34.396 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:34.397 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-12-01T01:35:34.491 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:34.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-01T01:35:34.589 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:34.589 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-01T01:35:34.689 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:34.689 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-01T01:35:34.779 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:34.780 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-01T01:35:34.922 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:34.922 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-01T01:35:35.027 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:35.027 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-12-01T01:35:35.118 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:35.118 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-12-01T01:35:35.210 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:35.210 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-01T01:35:35.304 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:35.304 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-01T01:35:35.397 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:35.397 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-01T01:35:35.492 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:35.492 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-01T01:35:35.591 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:35.591 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-01T01:35:35.649 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:35.649 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-12-01T01:35:35.696 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:35.696 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-12-01T01:35:35.757 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:35.758 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-12-01T01:35:35.808 INFO:tasks.ceph:Adding keys to all mons... 2023-12-01T01:35:35.808 DEBUG:teuthology.orchestra.run.smithi115:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-01T01:35:35.817 DEBUG:teuthology.orchestra.run.smithi142:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-01T01:35:35.906 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.y] 2023-12-01T01:35:35.906 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA== 2023-12-01T01:35:35.906 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.a] 2023-12-01T01:35:35.906 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ== 2023-12-01T01:35:35.906 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.c] 2023-12-01T01:35:35.906 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw== 2023-12-01T01:35:35.906 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.0] 2023-12-01T01:35:35.906 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw== 2023-12-01T01:35:35.906 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.1] 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig== 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.2] 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A== 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.3] 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg== 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.x] 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg== 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.b] 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg== 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout:[mds.d] 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q== 2023-12-01T01:35:35.907 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.4] 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg== 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.5] 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA== 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.6] 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw== 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.7] 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w== 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg== 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout:[client.2] 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g== 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout:[client.1] 2023-12-01T01:35:35.908 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A== 2023-12-01T01:35:35.909 INFO:teuthology.orchestra.run.smithi115.stdout:[client.3] 2023-12-01T01:35:35.909 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g== 2023-12-01T01:35:35.909 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr.y] 2023-12-01T01:35:35.909 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA== 2023-12-01T01:35:35.909 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2023-12-01T01:35:35.909 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ== 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw== 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.0] 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw== 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.1] 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig== 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.2] 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A== 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.3] 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg== 2023-12-01T01:35:35.910 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr.x] 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg== 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg== 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q== 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.4] 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg== 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.5] 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA== 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.6] 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw== 2023-12-01T01:35:35.911 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.7] 2023-12-01T01:35:35.912 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w== 2023-12-01T01:35:35.912 INFO:teuthology.orchestra.run.smithi142.stdout:[client.0] 2023-12-01T01:35:35.912 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg== 2023-12-01T01:35:35.912 INFO:teuthology.orchestra.run.smithi142.stdout:[client.2] 2023-12-01T01:35:35.912 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g== 2023-12-01T01:35:35.912 INFO:teuthology.orchestra.run.smithi142.stdout:[client.1] 2023-12-01T01:35:35.912 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A== 2023-12-01T01:35:35.912 INFO:teuthology.orchestra.run.smithi142.stdout:[client.3] 2023-12-01T01:35:35.912 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g== 2023-12-01T01:35:35.914 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-01T01:35:35.923 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-01T01:35:36.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-01T01:35:36.082 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-01T01:35:36.227 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-01T01:35:36.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-01T01:35:36.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:36.390 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:36.529 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:36.535 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:36.672 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:36.679 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:36.819 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:36.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:36.965 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-01T01:35:36.971 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-01T01:35:37.111 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-01T01:35:37.116 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-01T01:35:37.258 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-01T01:35:37.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-01T01:35:37.413 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:37.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:37.565 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:37.570 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:37.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:37.721 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:37.857 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:37.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-01T01:35:37.998 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-01T01:35:38.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-01T01:35:38.153 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-01T01:35:38.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-01T01:35:38.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-01T01:35:38.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-01T01:35:38.472 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-01T01:35:38.478 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-01T01:35:38.621 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-01T01:35:38.622 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-01T01:35:38.718 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-01T01:35:38.880 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-01T01:35:38.966 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-01T01:35:39.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-01T01:35:39.226 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-01T01:35:39.308 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-01T01:35:39.400 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-01T01:35:39.547 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-01T01:35:39.630 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:35:39.639 DEBUG:teuthology.orchestra.run.smithi142:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-01T01:35:39.689 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-01T01:35:39.690 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-01T01:35:39.690 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-12-01T01:35:39.696 INFO:tasks.ceph.mon.a:Started 2023-12-01T01:35:39.696 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-01T01:35:39.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-12-01T01:35:39.701 INFO:tasks.ceph.mon.b:Started 2023-12-01T01:35:39.701 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-01T01:35:39.701 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-12-01T01:35:39.706 INFO:tasks.ceph.mon.c:Started 2023-12-01T01:35:39.706 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-01T01:35:39.706 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-01T01:35:39.706 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-01T01:35:39.712 INFO:tasks.ceph.mgr.y:Started 2023-12-01T01:35:39.712 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-01T01:35:39.712 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-01T01:35:39.717 INFO:tasks.ceph.mgr.x:Started 2023-12-01T01:35:39.717 INFO:tasks.ceph:Setting crush tunables to default 2023-12-01T01:35:39.717 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd crush tunables default 2023-12-01T01:35:39.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.896+0000 7f38f4a28640 1 Processor -- start 2023-12-01T01:35:39.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.896+0000 7f38f4a28640 1 -- start start 2023-12-01T01:35:39.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.897+0000 7f38f4a28640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:39.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.897+0000 7f38f4a28640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f00b7760 con 0x7f38f015cf70 2023-12-01T01:35:39.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.897+0000 7f38f4a28640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f00b8b70 con 0x7f38f00d0260 2023-12-01T01:35:39.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.897+0000 7f38f4a28640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f00b2170 con 0x7f38f00b18f0 2023-12-01T01:35:39.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.897+0000 7f38ee59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f38f015cf70 0x7f38f00cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46812/0 (socket says 172.21.15.115:46812) 2023-12-01T01:35:39.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.897+0000 7f38ee59c640 1 -- 172.21.15.115:0/3963191837 learned_addr learned my addr 172.21.15.115:0/3963191837 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:35:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.898+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.898+0000 7f38eed9d640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 msgr2=0x7f38f00b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-01T01:35:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.898+0000 7f38eed9d640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 msgr2=0x7f38f00b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.898+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:39.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:39.898+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:35:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:40.099+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:40.099+0000 7f38eed9d640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 msgr2=0x7f38f00b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-01T01:35:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:40.099+0000 7f38eed9d640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 msgr2=0x7f38f00b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:40.099+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:40.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:40.100+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:35:40.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:40.501+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:40.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:40.501+0000 7f38eed9d640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 msgr2=0x7f38f00b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-01T01:35:40.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:40.502+0000 7f38eed9d640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 msgr2=0x7f38f00b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:40.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:40.502+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:40.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:40.502+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:35:41.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:41.303+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:41.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:41.303+0000 7f38eed9d640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 msgr2=0x7f38f00b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-01T01:35:41.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:41.304+0000 7f38eed9d640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 msgr2=0x7f38f00b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:41.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:41.304+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:41.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:41.304+0000 7f38eed9d640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:35:42.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.897+0000 7f38dffff640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 msgr2=0x7f38f00b8470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:35:42.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.897+0000 7f38dffff640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.115:3300/0 conn(0x7f38f00b18f0 0x7f38f00b8470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:42.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.897+0000 7f38dffff640 1 -- 172.21.15.115:0/3963191837 >> v1:172.21.15.142:6789/0 conn(0x7f38f00d0260 legacy=0x7f38f00d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:42.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.897+0000 7f38dffff640 1 -- 172.21.15.115:0/3963191837 >> v1:172.21.15.115:6789/0 conn(0x7f38f015cf70 legacy=0x7f38f00cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:42.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.897+0000 7f38dffff640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:42.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.897+0000 7f38dffff640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:42.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.897+0000 7f38dffff640 1 -- 172.21.15.115:0/3963191837 --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f00b2170 con 0x7f38d00036e0 2023-12-01T01:35:42.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.898+0000 7f38dffff640 1 -- 172.21.15.115:0/3963191837 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d0008d60 con 0x7f38d0006180 2023-12-01T01:35:42.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.898+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:42.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.898+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:42.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.899+0000 7f38ee59c640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 msgr2=0x7f38d0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-01T01:35:42.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.899+0000 7f38ee59c640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 msgr2=0x7f38d0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:42.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.899+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:42.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.899+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:35:42.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.899+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 msgr2=0x7f38d0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-01T01:35:42.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.899+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 msgr2=0x7f38d0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:42.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.899+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:42.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:42.899+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:35:43.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.100+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:43.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.100+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:43.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.101+0000 7f38ee59c640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 msgr2=0x7f38d0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-01T01:35:43.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.101+0000 7f38ee59c640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 msgr2=0x7f38d0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:43.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.101+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:43.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.101+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 msgr2=0x7f38d0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-01T01:35:43.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.101+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 msgr2=0x7f38d0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:43.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.101+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:43.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.101+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:35:43.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.101+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:35:43.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.502+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:43.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.503+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:43.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.503+0000 7f38ee59c640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 msgr2=0x7f38d0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-01T01:35:43.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.503+0000 7f38ee59c640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 msgr2=0x7f38d0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:43.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.503+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:43.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.503+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:35:43.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.503+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 msgr2=0x7f38d0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-01T01:35:43.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.503+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 msgr2=0x7f38d0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:43.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.503+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:43.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:43.503+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:35:44.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:44.305+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:44.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:44.305+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:44.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:44.305+0000 7f38ee59c640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 msgr2=0x7f38d0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-01T01:35:44.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:44.305+0000 7f38ee59c640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 msgr2=0x7f38d0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:44.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:44.305+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:44.305+0000 7f38ee59c640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:35:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:44.306+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 msgr2=0x7f38d0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-01T01:35:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:44.306+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 msgr2=0x7f38d0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-01T01:35:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:44.306+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:35:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:44.306+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:35:45.749 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:45.747+0000 7f1908018040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:35:45.812 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:45.811+0000 7f1908018040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:35:45.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.897+0000 7f38dffff640 1 -- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 msgr2=0x7f38d0008640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:35:45.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.897+0000 7f38dffff640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0006180 0x7f38d0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:45.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.897+0000 7f38dffff640 1 -- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 msgr2=0x7f38d0005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:35:45.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.897+0000 7f38dffff640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d00036e0 0x7f38d0005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:45.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.897+0000 7f38eed9d640 1 -- 172.21.15.115:0/3963191837 reap_dead start 2023-12-01T01:35:45.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.897+0000 7f38dffff640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d0003b30 0x7f38d0006630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:45.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.897+0000 7f38dffff640 1 -- 172.21.15.115:0/3963191837 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f00b6e30 con 0x7f38f015cf70 2023-12-01T01:35:45.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.897+0000 7f38dffff640 1 -- 172.21.15.115:0/3963191837 --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d0008840 con 0x7f38d0006b70 2023-12-01T01:35:45.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.897+0000 7f38dffff640 1 -- 172.21.15.115:0/3963191837 --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f00b2170 con 0x7f38d0003b30 2023-12-01T01:35:45.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.898+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> v2:172.21.15.142:3300/0 conn(0x7f38d0003b30 0x7f38d0006630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:45.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.898+0000 7f38ed59a640 1 -- 172.21.15.115:0/3963191837 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123312804 0 0) 0x7f38f00b6e30 con 0x7f38f015cf70 2023-12-01T01:35:45.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.898+0000 7f38ed59a640 1 -- 172.21.15.115:0/3963191837 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d8003520 con 0x7f38f015cf70 2023-12-01T01:35:45.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.899+0000 7f38ed59a640 1 -- 172.21.15.115:0/3963191837 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275369565 0 0) 0x7f38d0008840 con 0x7f38d0006b70 2023-12-01T01:35:45.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.899+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 >> v1:172.21.15.142:6789/0 conn(0x7f38d0006b70 legacy=0x7f38d0007000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:45.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.899+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 >> v1:172.21.15.115:6789/0 conn(0x7f38f015cf70 legacy=0x7f38d000b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:45.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.899+0000 7f38edd9b640 1 -- 172.21.15.115:0/3963191837 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00108c0 con 0x7f38d0003b30 2023-12-01T01:35:45.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.901+0000 7f38edd9b640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38d0003b30 0x7f38d0006630 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f38f00d0680 tx=0x7f38e402c100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d49e109c584912e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:45.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.902+0000 7f38ed59a640 1 -- 172.21.15.115:0/3963191837 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e402ccb0 con 0x7f38d0003b30 2023-12-01T01:35:45.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.902+0000 7f38ed59a640 1 -- 172.21.15.115:0/3963191837 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e4030070 con 0x7f38d0003b30 2023-12-01T01:35:45.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.902+0000 7f38ed59a640 1 -- 172.21.15.115:0/3963191837 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4036650 con 0x7f38d0003b30 2023-12-01T01:35:45.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.903+0000 7f38f4a28640 1 -- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38d0003b30 msgr2=0x7f38d0006630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:45.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.903+0000 7f38f4a28640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38d0003b30 0x7f38d0006630 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f38f00d0680 tx=0x7f38e402c100 comp rx=0 tx=0).stop 2023-12-01T01:35:45.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.903+0000 7f38f4a28640 1 -- 172.21.15.115:0/3963191837 shutdown_connections 2023-12-01T01:35:45.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.903+0000 7f38f4a28640 1 --2- 172.21.15.115:0/3963191837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38d0003b30 0x7f38d0006630 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:45.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.903+0000 7f38f4a28640 1 -- 172.21.15.115:0/3963191837 >> 172.21.15.115:0/3963191837 conn(0x7f38f006a0e0 msgr2=0x7f38f006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:45.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.904+0000 7f38f4a28640 1 -- 172.21.15.115:0/3963191837 shutdown_connections 2023-12-01T01:35:45.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.904+0000 7f38f4a28640 1 -- 172.21.15.115:0/3963191837 wait complete. 2023-12-01T01:35:45.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.907+0000 7f38f4a28640 1 Processor -- start 2023-12-01T01:35:45.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.907+0000 7f38f4a28640 1 -- start start 2023-12-01T01:35:45.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.908+0000 7f38f4a28640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0003b30 0x7f38f00d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:45.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.908+0000 7f38f4a28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38d0006b70 0x7f38f0158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:45.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.908+0000 7f38f4a28640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f015cf70 0x7f38f01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:45.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.908+0000 7f38f4a28640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0155dc0 con 0x7f38d0006b70 2023-12-01T01:35:45.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.908+0000 7f38f4a28640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f00d1d50 con 0x7f38f015cf70 2023-12-01T01:35:45.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.908+0000 7f38f4a28640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f00d1140 con 0x7f38d0003b30 2023-12-01T01:35:45.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.909+0000 7f38edd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38d0006b70 0x7f38f0158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:45.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.909+0000 7f38edd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38d0006b70 0x7f38f0158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60484/0 (socket says 172.21.15.115:60484) 2023-12-01T01:35:45.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.909+0000 7f38edd9b640 1 -- 172.21.15.115:0/433536396 learned_addr learned my addr 172.21.15.115:0/433536396 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:35:45.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.909+0000 7f38eed9d640 1 --2- 172.21.15.115:0/433536396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f015cf70 0x7f38f01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:45.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.909+0000 7f38ee59c640 1 --2- 172.21.15.115:0/433536396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0003b30 0x7f38f00d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:45.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.909+0000 7f38edd9b640 1 -- 172.21.15.115:0/433536396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0003b30 msgr2=0x7f38f00d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:45.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.909+0000 7f38edd9b640 1 --2- 172.21.15.115:0/433536396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0003b30 0x7f38f00d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:45.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.909+0000 7f38edd9b640 1 -- 172.21.15.115:0/433536396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f015cf70 msgr2=0x7f38f01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:45.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.909+0000 7f38edd9b640 1 --2- 172.21.15.115:0/433536396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f015cf70 0x7f38f01594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:45.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.909+0000 7f38edd9b640 1 -- 172.21.15.115:0/433536396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e400f040 con 0x7f38d0006b70 2023-12-01T01:35:45.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.910+0000 7f38edd9b640 1 --2- 172.21.15.115:0/433536396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38d0006b70 0x7f38f0158f60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38e4005060 tx=0x7f38e4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:45.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.910+0000 7f38deffd640 1 -- 172.21.15.115:0/433536396 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e40069d0 con 0x7f38d0006b70 2023-12-01T01:35:45.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.910+0000 7f38deffd640 1 -- 172.21.15.115:0/433536396 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e4006b30 con 0x7f38d0006b70 2023-12-01T01:35:45.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.910+0000 7f38f4a28640 1 -- 172.21.15.115:0/433536396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f00d1420 con 0x7f38d0006b70 2023-12-01T01:35:45.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.910+0000 7f38deffd640 1 -- 172.21.15.115:0/433536396 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e40110c0 con 0x7f38d0006b70 2023-12-01T01:35:45.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.910+0000 7f38f4a28640 1 -- 172.21.15.115:0/433536396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f00d19e0 con 0x7f38d0006b70 2023-12-01T01:35:45.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.910+0000 7f38deffd640 1 -- 172.21.15.115:0/433536396 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f38e4036410 con 0x7f38d0006b70 2023-12-01T01:35:45.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.910+0000 7f38deffd640 1 -- 172.21.15.115:0/433536396 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f38e4007b10 con 0x7f38d0006b70 2023-12-01T01:35:45.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.911+0000 7f38f4a28640 1 -- 172.21.15.115:0/433536396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f004e890 con 0x7f38d0006b70 2023-12-01T01:35:45.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:45.914+0000 7f38deffd640 1 -- 172.21.15.115:0/433536396 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f38e4043020 con 0x7f38d0006b70 2023-12-01T01:35:46.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.005+0000 7f38f4a28640 1 -- 172.21.15.115:0/433536396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f38f00b16d0 con 0x7f38d0006b70 2023-12-01T01:35:46.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.012+0000 7f38deffd640 1 -- 172.21.15.115:0/433536396 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f38e4056090 con 0x7f38d0006b70 2023-12-01T01:35:46.013 INFO:teuthology.orchestra.run.smithi115.stderr:adjusted tunables profile to default 2023-12-01T01:35:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.013+0000 7f38f4a28640 1 -- 172.21.15.115:0/433536396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38d0006b70 msgr2=0x7f38f0158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.013+0000 7f38f4a28640 1 --2- 172.21.15.115:0/433536396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38d0006b70 0x7f38f0158f60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38e4005060 tx=0x7f38e4001120 comp rx=0 tx=0).stop 2023-12-01T01:35:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.013+0000 7f38f4a28640 1 -- 172.21.15.115:0/433536396 shutdown_connections 2023-12-01T01:35:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.013+0000 7f38f4a28640 1 --2- 172.21.15.115:0/433536396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f015cf70 0x7f38f01594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.013+0000 7f38f4a28640 1 --2- 172.21.15.115:0/433536396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38d0006b70 0x7f38f0158f60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.013+0000 7f38f4a28640 1 --2- 172.21.15.115:0/433536396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38d0003b30 0x7f38f00d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.013+0000 7f38f4a28640 1 -- 172.21.15.115:0/433536396 >> 172.21.15.115:0/433536396 conn(0x7f38f006a0e0 msgr2=0x7f38f0156170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.014+0000 7f38f4a28640 1 -- 172.21.15.115:0/433536396 shutdown_connections 2023-12-01T01:35:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:46.014+0000 7f38f4a28640 1 -- 172.21.15.115:0/433536396 wait complete. 2023-12-01T01:35:46.032 INFO:tasks.ceph:check_enable_crimson: False 2023-12-01T01:35:46.032 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-01T01:35:46.033 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:46.033 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-01T01:35:46.121 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:46.121 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-01T01:35:46.253 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:46.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-01T01:35:46.380 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:35:46.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-01T01:35:46.462 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:46.462 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-01T01:35:46.478 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:46.476+0000 7f7afba45040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:35:46.528 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:46.527+0000 7f1908018040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:35:46.543 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:46.541+0000 7f7afba45040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:35:46.558 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:46.558 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-01T01:35:46.593 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:46.591+0000 7f1908018040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:35:46.648 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:46.646+0000 7f1908018040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:35:46.681 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:46.681 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-01T01:35:46.717 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:46.716+0000 7f1908018040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:35:46.771 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:46.770+0000 7f1908018040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:35:46.809 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:35:46.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-01T01:35:46.893 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:46.891+0000 7f1908018040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:35:46.942 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new aaf68732-34d8-41ee-9104-1789636cd3ad 0 2023-12-01T01:35:46.972 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:46.970+0000 7f1908018040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:35:47.025 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:47.023+0000 7f1908018040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:35:47.084 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:47.082+0000 7f1908018040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:35:47.123 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.121+0000 7fc88e1f0640 1 Processor -- start 2023-12-01T01:35:47.123 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.122+0000 7fc88e1f0640 1 -- start start 2023-12-01T01:35:47.125 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.123+0000 7fc88e1f0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc88815cf70 0x7fc8880b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.123+0000 7fc88e1f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8880d0260 0x7fc8880b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.123+0000 7fc88e1f0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8880b7760 con 0x7fc8880b18f0 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.123+0000 7fc88e1f0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8880b2170 con 0x7fc88815cf70 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.123+0000 7fc88e1f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8880b8b70 con 0x7fc8880d0260 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.123+0000 7fc8877fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc88815cf70 0x7fc8880b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.123+0000 7fc8877fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc88815cf70 0x7fc8880b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32894/0 (socket says 172.21.15.142:32894) 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.123+0000 7fc8877fe640 1 -- 172.21.15.142:0/220351684 learned_addr learned my addr 172.21.15.142:0/220351684 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.123+0000 7fc87ffff640 1 --1- 172.21.15.142:0/220351684 >> v1:172.21.15.142:6789/0 conn(0x7fc8880b18f0 0x7fc8880cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:58882/0 (socket says 172.21.15.142:58882) 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.124+0000 7fc887fff640 1 --2- 172.21.15.142:0/220351684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8880d0260 0x7fc8880b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.124+0000 7fc8877fe640 1 -- 172.21.15.142:0/220351684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8880d0260 msgr2=0x7fc8880b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.124+0000 7fc8877fe640 1 --2- 172.21.15.142:0/220351684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8880d0260 0x7fc8880b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.124+0000 7fc8877fe640 1 -- 172.21.15.142:0/220351684 >> v1:172.21.15.142:6789/0 conn(0x7fc8880b18f0 legacy=0x7fc8880cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.126 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.124+0000 7fc8877fe640 1 -- 172.21.15.142:0/220351684 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8880b6e30 con 0x7fc88815cf70 2023-12-01T01:35:47.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.125+0000 7fc8877fe640 1 --2- 172.21.15.142:0/220351684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88815cf70 0x7fc8880b8470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc878009870 tx=0x7fc87802ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e11a9035a995ea9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:47.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.125+0000 7fc886ffd640 1 -- 172.21.15.142:0/220351684 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87802f830 con 0x7fc88815cf70 2023-12-01T01:35:47.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.125+0000 7fc886ffd640 1 -- 172.21.15.142:0/220351684 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87802f990 con 0x7fc88815cf70 2023-12-01T01:35:47.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.125+0000 7fc886ffd640 1 -- 172.21.15.142:0/220351684 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87802fd20 con 0x7fc88815cf70 2023-12-01T01:35:47.127 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.126+0000 7fc88e1f0640 1 -- 172.21.15.142:0/220351684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88815cf70 msgr2=0x7fc8880b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.126+0000 7fc88e1f0640 1 --2- 172.21.15.142:0/220351684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88815cf70 0x7fc8880b8470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc878009870 tx=0x7fc87802ba10 comp rx=0 tx=0).stop 2023-12-01T01:35:47.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.126+0000 7fc88e1f0640 1 -- 172.21.15.142:0/220351684 shutdown_connections 2023-12-01T01:35:47.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.126+0000 7fc88e1f0640 1 --2- 172.21.15.142:0/220351684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8880d0260 0x7fc8880b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.126+0000 7fc88e1f0640 1 --2- 172.21.15.142:0/220351684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88815cf70 0x7fc8880b8470 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.126+0000 7fc88e1f0640 1 -- 172.21.15.142:0/220351684 >> 172.21.15.142:0/220351684 conn(0x7fc88806a0e0 msgr2=0x7fc88806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:47.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.126+0000 7fc88e1f0640 1 -- 172.21.15.142:0/220351684 shutdown_connections 2023-12-01T01:35:47.128 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.126+0000 7fc88e1f0640 1 -- 172.21.15.142:0/220351684 wait complete. 2023-12-01T01:35:47.129 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.128+0000 7fc88e1f0640 1 Processor -- start 2023-12-01T01:35:47.129 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.128+0000 7fc88e1f0640 1 -- start start 2023-12-01T01:35:47.131 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc88e1f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8880b18f0 0x7fc888153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.131 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc88e1f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8880d0260 0x7fc8881558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.131 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc88e1f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88815cf70 0x7fc88815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.131 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc88e1f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc888155490 con 0x7fc8880b18f0 2023-12-01T01:35:47.131 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc88e1f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8881536d0 con 0x7fc8880d0260 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc88e1f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc888154af0 con 0x7fc88815cf70 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc87ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8880d0260 0x7fc8881558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc887fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88815cf70 0x7fc88815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc887fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88815cf70 0x7fc88815d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:41990/0 (socket says 172.21.15.142:41990) 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc887fff640 1 -- 172.21.15.142:0/3069683633 learned_addr learned my addr 172.21.15.142:0/3069683633 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc87ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8880d0260 0x7fc8881558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32902/0 (socket says 172.21.15.142:32902) 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc8877fe640 1 --2- 172.21.15.142:0/3069683633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8880b18f0 0x7fc888153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.129+0000 7fc887fff640 1 -- 172.21.15.142:0/3069683633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8880d0260 msgr2=0x7fc8881558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc887fff640 1 --2- 172.21.15.142:0/3069683633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8880d0260 0x7fc8881558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc887fff640 1 -- 172.21.15.142:0/3069683633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8880b18f0 msgr2=0x7fc888153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc887fff640 1 --2- 172.21.15.142:0/3069683633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8880b18f0 0x7fc888153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc887fff640 1 -- 172.21.15.142:0/3069683633 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780094d0 con 0x7fc88815cf70 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc8877fe640 1 --2- 172.21.15.142:0/3069683633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8880b18f0 0x7fc888153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:35:47.132 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc887fff640 1 --2- 172.21.15.142:0/3069683633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88815cf70 0x7fc88815d350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc880009dd0 tx=0x7fc88000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:47.133 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc884ff9640 1 -- 172.21.15.142:0/3069683633 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc880014480 con 0x7fc88815cf70 2023-12-01T01:35:47.134 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc884ff9640 1 -- 172.21.15.142:0/3069683633 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88000f040 con 0x7fc88815cf70 2023-12-01T01:35:47.134 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc884ff9640 1 -- 172.21.15.142:0/3069683633 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc880014840 con 0x7fc88815cf70 2023-12-01T01:35:47.134 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc88e1f0640 1 -- 172.21.15.142:0/3069683633 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc888154dd0 con 0x7fc88815cf70 2023-12-01T01:35:47.134 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc88e1f0640 1 -- 172.21.15.142:0/3069683633 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8880cac10 con 0x7fc88815cf70 2023-12-01T01:35:47.134 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.130+0000 7fc884ff9640 1 -- 172.21.15.142:0/3069683633 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc880014ca0 con 0x7fc88815cf70 2023-12-01T01:35:47.134 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.131+0000 7fc884ff9640 1 -- 172.21.15.142:0/3069683633 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc88001de20 con 0x7fc88815cf70 2023-12-01T01:35:47.134 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.131+0000 7fc88e1f0640 1 -- 172.21.15.142:0/3069683633 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8881555d0 con 0x7fc88815cf70 2023-12-01T01:35:47.135 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.134+0000 7fc884ff9640 1 -- 172.21.15.142:0/3069683633 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fc8881555d0 con 0x7fc88815cf70 2023-12-01T01:35:47.139 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:47.137+0000 7f1908018040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:35:47.197 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:47.195+0000 7f7afba45040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:35:47.214 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.212+0000 7fc88e1f0640 1 -- 172.21.15.142:0/3069683633 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "aaf68732-34d8-41ee-9104-1789636cd3ad", "id": 0} v 0) v1 -- 0x7fc8880b70e0 con 0x7fc88815cf70 2023-12-01T01:35:47.223 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.221+0000 7fc884ff9640 1 -- 172.21.15.142:0/3069683633 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aaf68732-34d8-41ee-9104-1789636cd3ad", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc880018020 con 0x7fc88815cf70 2023-12-01T01:35:47.223 INFO:teuthology.orchestra.run.smithi142.stdout:0 2023-12-01T01:35:47.224 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.223+0000 7fc88e1f0640 1 -- 172.21.15.142:0/3069683633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88815cf70 msgr2=0x7fc88815d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.224 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.223+0000 7fc88e1f0640 1 --2- 172.21.15.142:0/3069683633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88815cf70 0x7fc88815d350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc880009dd0 tx=0x7fc88000c490 comp rx=0 tx=0).stop 2023-12-01T01:35:47.224 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.223+0000 7fc88e1f0640 1 -- 172.21.15.142:0/3069683633 shutdown_connections 2023-12-01T01:35:47.224 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.223+0000 7fc88e1f0640 1 --2- 172.21.15.142:0/3069683633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88815cf70 0x7fc88815d350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.225 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.223+0000 7fc88e1f0640 1 --2- 172.21.15.142:0/3069683633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8880d0260 0x7fc8881558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.225 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.223+0000 7fc88e1f0640 1 --2- 172.21.15.142:0/3069683633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8880b18f0 0x7fc888153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.225 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.223+0000 7fc88e1f0640 1 -- 172.21.15.142:0/3069683633 >> 172.21.15.142:0/3069683633 conn(0x7fc88806a0e0 msgr2=0x7fc8880b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:47.225 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.223+0000 7fc88e1f0640 1 -- 172.21.15.142:0/3069683633 shutdown_connections 2023-12-01T01:35:47.225 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.223+0000 7fc88e1f0640 1 -- 172.21.15.142:0/3069683633 wait complete. 2023-12-01T01:35:47.241 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new ae0caaf6-7427-4951-85a2-68c07f744762 1 2023-12-01T01:35:47.263 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:47.261+0000 7f7afba45040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:35:47.319 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:47.317+0000 7f7afba45040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:35:47.385 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:47.382+0000 7f7afba45040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:35:47.430 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.428+0000 7f5f6d065640 1 Processor -- start 2023-12-01T01:35:47.430 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.428+0000 7f5f6d065640 1 -- start start 2023-12-01T01:35:47.431 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.430+0000 7f5f6d065640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68153c30 0x7f5f680ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.431 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.430+0000 7f5f6d065640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5f68154010 0x7f5f680cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.431 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.430+0000 7f5f6d065640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f680cdcd0 con 0x7f5f680cf9b0 2023-12-01T01:35:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.430+0000 7f5f6d065640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f680cfe20 con 0x7f5f68154010 2023-12-01T01:35:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.430+0000 7f5f6d065640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f68161830 con 0x7f5f68153c30 2023-12-01T01:35:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.430+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68153c30 0x7f5f680ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.432 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.430+0000 7f5f677fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5f68154010 0x7f5f680cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.433 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.430+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68153c30 0x7f5f680ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:42004/0 (socket says 172.21.15.142:42004) 2023-12-01T01:35:47.433 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.430+0000 7f5f67fff640 1 -- 172.21.15.142:0/4293919343 learned_addr learned my addr 172.21.15.142:0/4293919343 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:47.433 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.431+0000 7f5f67fff640 1 -- 172.21.15.142:0/4293919343 >> v2:172.21.15.142:3300/0 conn(0x7f5f68154010 msgr2=0x7f5f680cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:47.433 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.431+0000 7f5f67fff640 1 --2- 172.21.15.142:0/4293919343 >> v2:172.21.15.142:3300/0 conn(0x7f5f68154010 0x7f5f680cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.433 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.431+0000 7f5f67fff640 1 -- 172.21.15.142:0/4293919343 >> v1:172.21.15.142:6789/0 conn(0x7f5f680cf9b0 legacy=0x7f5f68160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.433 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.431+0000 7f5f67fff640 1 -- 172.21.15.142:0/4293919343 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f680cd3a0 con 0x7f5f68153c30 2023-12-01T01:35:47.433 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.431+0000 7f5f66ffd640 1 -- 172.21.15.142:0/4293919343 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025963102 0 0) 0x7f5f680cdcd0 con 0x7f5f680cf9b0 2023-12-01T01:35:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.432+0000 7f5f67fff640 1 --2- 172.21.15.142:0/4293919343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68153c30 0x7f5f680ce160 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009870 tx=0x7f5f5802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a6e3c990ce02bfd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.432+0000 7f5f66ffd640 1 -- 172.21.15.142:0/4293919343 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58030840 con 0x7f5f68153c30 2023-12-01T01:35:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.432+0000 7f5f66ffd640 1 -- 172.21.15.142:0/4293919343 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f580309a0 con 0x7f5f68153c30 2023-12-01T01:35:47.434 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.432+0000 7f5f66ffd640 1 -- 172.21.15.142:0/4293919343 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58030d30 con 0x7f5f68153c30 2023-12-01T01:35:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.433+0000 7f5f6d065640 1 -- 172.21.15.142:0/4293919343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68153c30 msgr2=0x7f5f680ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.433+0000 7f5f6d065640 1 --2- 172.21.15.142:0/4293919343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68153c30 0x7f5f680ce160 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009870 tx=0x7f5f5802b980 comp rx=0 tx=0).stop 2023-12-01T01:35:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.433+0000 7f5f6d065640 1 -- 172.21.15.142:0/4293919343 shutdown_connections 2023-12-01T01:35:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.433+0000 7f5f6d065640 1 --2- 172.21.15.142:0/4293919343 >> v2:172.21.15.142:3300/0 conn(0x7f5f68154010 0x7f5f680cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.433+0000 7f5f6d065640 1 --2- 172.21.15.142:0/4293919343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68153c30 0x7f5f680ce160 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.433+0000 7f5f6d065640 1 -- 172.21.15.142:0/4293919343 >> 172.21.15.142:0/4293919343 conn(0x7f5f6806a0b0 msgr2=0x7f5f6806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.433+0000 7f5f6d065640 1 -- 172.21.15.142:0/4293919343 shutdown_connections 2023-12-01T01:35:47.435 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.433+0000 7f5f6d065640 1 -- 172.21.15.142:0/4293919343 wait complete. 2023-12-01T01:35:47.436 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.435+0000 7f5f6d065640 1 Processor -- start 2023-12-01T01:35:47.436 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.435+0000 7f5f6d065640 1 -- start start 2023-12-01T01:35:47.437 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:47.435+0000 7f7afba45040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:35:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f6d065640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f680cf9b0 0x7f5f681f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f6d065640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68153c30 0x7f5f6815c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f6d065640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68154010 0x7f5f680cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f6d065640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681f3a10 con 0x7f5f68153c30 2023-12-01T01:35:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f6d065640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f680c9ff0 con 0x7f5f680cf9b0 2023-12-01T01:35:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f6d065640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f680ca2d0 con 0x7f5f68154010 2023-12-01T01:35:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f680cf9b0 0x7f5f681f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f67fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f680cf9b0 0x7f5f681f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32922/0 (socket says 172.21.15.142:32922) 2023-12-01T01:35:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f67fff640 1 -- 172.21.15.142:0/1293124075 learned_addr learned my addr 172.21.15.142:0/1293124075 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:47.438 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f6c864640 1 --2- 172.21.15.142:0/1293124075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68154010 0x7f5f680cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f67fff640 1 -- 172.21.15.142:0/1293124075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68154010 msgr2=0x7f5f680cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f677fe640 1 --2- 172.21.15.142:0/1293124075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68153c30 0x7f5f6815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f67fff640 1 --2- 172.21.15.142:0/1293124075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68154010 0x7f5f680cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f67fff640 1 -- 172.21.15.142:0/1293124075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68153c30 msgr2=0x7f5f6815c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f67fff640 1 --2- 172.21.15.142:0/1293124075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68153c30 0x7f5f6815c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.439 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.436+0000 7f5f67fff640 1 -- 172.21.15.142:0/1293124075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f580094d0 con 0x7f5f680cf9b0 2023-12-01T01:35:47.440 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.437+0000 7f5f67fff640 1 --2- 172.21.15.142:0/1293124075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f680cf9b0 0x7f5f681f34d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5f58000c00 tx=0x7f5f58004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:47.440 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.437+0000 7f5f64ff9640 1 -- 172.21.15.142:0/1293124075 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58039c90 con 0x7f5f680cf9b0 2023-12-01T01:35:47.440 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.437+0000 7f5f64ff9640 1 -- 172.21.15.142:0/1293124075 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f58039df0 con 0x7f5f680cf9b0 2023-12-01T01:35:47.440 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.437+0000 7f5f6d065640 1 -- 172.21.15.142:0/1293124075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f680ca5b0 con 0x7f5f680cf9b0 2023-12-01T01:35:47.440 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.437+0000 7f5f64ff9640 1 -- 172.21.15.142:0/1293124075 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58043640 con 0x7f5f680cf9b0 2023-12-01T01:35:47.440 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.437+0000 7f5f6d065640 1 -- 172.21.15.142:0/1293124075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f680cab70 con 0x7f5f680cf9b0 2023-12-01T01:35:47.440 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.437+0000 7f5f64ff9640 1 -- 172.21.15.142:0/1293124075 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5f58043aa0 con 0x7f5f680cf9b0 2023-12-01T01:35:47.440 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.437+0000 7f5f64ff9640 1 -- 172.21.15.142:0/1293124075 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f5f5803e070 con 0x7f5f680cf9b0 2023-12-01T01:35:47.440 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.438+0000 7f5f6d065640 1 -- 172.21.15.142:0/1293124075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f680cad90 con 0x7f5f680cf9b0 2023-12-01T01:35:47.445 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.443+0000 7f5f64ff9640 1 -- 172.21.15.142:0/1293124075 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f5f58034070 con 0x7f5f680cf9b0 2023-12-01T01:35:47.522 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.520+0000 7f5f6d065640 1 -- 172.21.15.142:0/1293124075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ae0caaf6-7427-4951-85a2-68c07f744762", "id": 1} v 0) v1 -- 0x7f5f680618d0 con 0x7f5f680cf9b0 2023-12-01T01:35:47.532 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.530+0000 7f5f64ff9640 1 -- 172.21.15.142:0/1293124075 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae0caaf6-7427-4951-85a2-68c07f744762", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f5804d460 con 0x7f5f680cf9b0 2023-12-01T01:35:47.532 INFO:teuthology.orchestra.run.smithi142.stdout:1 2023-12-01T01:35:47.533 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.531+0000 7f5f6d065640 1 -- 172.21.15.142:0/1293124075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f680cf9b0 msgr2=0x7f5f681f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.533 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.531+0000 7f5f6d065640 1 --2- 172.21.15.142:0/1293124075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f680cf9b0 0x7f5f681f34d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5f58000c00 tx=0x7f5f58004730 comp rx=0 tx=0).stop 2023-12-01T01:35:47.533 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.532+0000 7f5f6d065640 1 -- 172.21.15.142:0/1293124075 shutdown_connections 2023-12-01T01:35:47.534 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.532+0000 7f5f6d065640 1 --2- 172.21.15.142:0/1293124075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f68154010 0x7f5f680cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.534 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.532+0000 7f5f6d065640 1 --2- 172.21.15.142:0/1293124075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f68153c30 0x7f5f6815c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.534 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.532+0000 7f5f6d065640 1 --2- 172.21.15.142:0/1293124075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f680cf9b0 0x7f5f681f34d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.534 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.532+0000 7f5f6d065640 1 -- 172.21.15.142:0/1293124075 >> 172.21.15.142:0/1293124075 conn(0x7f5f6806a0b0 msgr2=0x7f5f68152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:47.534 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.532+0000 7f5f6d065640 1 -- 172.21.15.142:0/1293124075 shutdown_connections 2023-12-01T01:35:47.534 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.532+0000 7f5f6d065640 1 -- 172.21.15.142:0/1293124075 wait complete. 2023-12-01T01:35:47.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 5ba3cc79-ca5f-4da6-adcb-8a99c60652ff 2 2023-12-01T01:35:47.559 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:47.557+0000 7f7afba45040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:35:47.589 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:47.587+0000 7f1908018040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:35:47.644 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:47.642+0000 7f7afba45040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:35:47.700 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:47.698+0000 7f7afba45040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:35:47.749 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:47.747+0000 7f1908018040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:35:47.750 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.748+0000 7fa4325eb640 1 Processor -- start 2023-12-01T01:35:47.750 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.749+0000 7fa4325eb640 1 -- start start 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.750+0000 7fa4325eb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa42c15ccf0 0x7fa42c0b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.750+0000 7fa4325eb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa42c0b1e90 0x7fa42c065960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.750+0000 7fa4325eb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa42c065ea0 0x7fa42c0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.750+0000 7fa4325eb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42c066310 con 0x7fa42c0b1e90 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.750+0000 7fa4325eb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42c0b4af0 con 0x7fa42c15ccf0 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.750+0000 7fa4325eb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c06d970 con 0x7fa42c065ea0 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.751+0000 7fa4315e9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa42c15ccf0 0x7fa42c0b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.751+0000 7fa4315e9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa42c15ccf0 0x7fa42c0b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32932/0 (socket says 172.21.15.142:32932) 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.751+0000 7fa431dea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa42c065ea0 0x7fa42c0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.751+0000 7fa4315e9640 1 -- 172.21.15.142:0/4264247179 learned_addr learned my addr 172.21.15.142:0/4264247179 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.751+0000 7fa430de8640 1 --2- 172.21.15.142:0/4264247179 >> v2:172.21.15.115:3300/0 conn(0x7fa42c0b1e90 0x7fa42c065960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.751+0000 7fa4315e9640 1 -- 172.21.15.142:0/4264247179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa42c065ea0 msgr2=0x7fa42c0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.751+0000 7fa4315e9640 1 --2- 172.21.15.142:0/4264247179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa42c065ea0 0x7fa42c0b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.753 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.751+0000 7fa4315e9640 1 -- 172.21.15.142:0/4264247179 >> v2:172.21.15.115:3300/0 conn(0x7fa42c0b1e90 msgr2=0x7fa42c065960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:47.754 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.751+0000 7fa4315e9640 1 --2- 172.21.15.142:0/4264247179 >> v2:172.21.15.115:3300/0 conn(0x7fa42c0b1e90 0x7fa42c065960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.754 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.751+0000 7fa4315e9640 1 -- 172.21.15.142:0/4264247179 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c06dbc0 con 0x7fa42c15ccf0 2023-12-01T01:35:47.754 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.752+0000 7fa4315e9640 1 --2- 172.21.15.142:0/4264247179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c15ccf0 0x7fa42c0b45b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa42c072240 tx=0x7fa42802b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff3d7792bbf8a171 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:47.754 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.752+0000 7fa423fff640 1 -- 172.21.15.142:0/4264247179 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4280317d0 con 0x7fa42c15ccf0 2023-12-01T01:35:47.754 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.752+0000 7fa423fff640 1 -- 172.21.15.142:0/4264247179 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa428031930 con 0x7fa42c15ccf0 2023-12-01T01:35:47.754 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.752+0000 7fa423fff640 1 -- 172.21.15.142:0/4264247179 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428031cc0 con 0x7fa42c15ccf0 2023-12-01T01:35:47.755 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.753+0000 7fa4325eb640 1 -- 172.21.15.142:0/4264247179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c15ccf0 msgr2=0x7fa42c0b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.755 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.753+0000 7fa4325eb640 1 --2- 172.21.15.142:0/4264247179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c15ccf0 0x7fa42c0b45b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa42c072240 tx=0x7fa42802b870 comp rx=0 tx=0).stop 2023-12-01T01:35:47.755 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.753+0000 7fa4325eb640 1 -- 172.21.15.142:0/4264247179 shutdown_connections 2023-12-01T01:35:47.755 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.753+0000 7fa4325eb640 1 --2- 172.21.15.142:0/4264247179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa42c065ea0 0x7fa42c0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.755 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.753+0000 7fa4325eb640 1 --2- 172.21.15.142:0/4264247179 >> v2:172.21.15.115:3300/0 conn(0x7fa42c0b1e90 0x7fa42c065960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.756 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.753+0000 7fa4325eb640 1 --2- 172.21.15.142:0/4264247179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c15ccf0 0x7fa42c0b45b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.756 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.753+0000 7fa4325eb640 1 -- 172.21.15.142:0/4264247179 >> 172.21.15.142:0/4264247179 conn(0x7fa42c06a760 msgr2=0x7fa42c06c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:47.756 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.753+0000 7fa4325eb640 1 -- 172.21.15.142:0/4264247179 shutdown_connections 2023-12-01T01:35:47.756 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.753+0000 7fa4325eb640 1 -- 172.21.15.142:0/4264247179 wait complete. 2023-12-01T01:35:47.756 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.755+0000 7fa4325eb640 1 Processor -- start 2023-12-01T01:35:47.756 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.755+0000 7fa4325eb640 1 -- start start 2023-12-01T01:35:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.756+0000 7fa4325eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa42c065ea0 0x7fa42c15edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.756+0000 7fa4325eb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c0b1e90 0x7fa42c14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.756+0000 7fa4325eb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa42c15ccf0 0x7fa42c0aa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.756+0000 7fa4325eb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0aae40 con 0x7fa42c065ea0 2023-12-01T01:35:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.756+0000 7fa4325eb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0ab0e0 con 0x7fa42c0b1e90 2023-12-01T01:35:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.756+0000 7fa4325eb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0ab3c0 con 0x7fa42c15ccf0 2023-12-01T01:35:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.756+0000 7fa430de8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c0b1e90 0x7fa42c14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.756+0000 7fa430de8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c0b1e90 0x7fa42c14f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32944/0 (socket says 172.21.15.142:32944) 2023-12-01T01:35:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.756+0000 7fa430de8640 1 -- 172.21.15.142:0/1160269663 learned_addr learned my addr 172.21.15.142:0/1160269663 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:47.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.756+0000 7fa431dea640 1 --2- 172.21.15.142:0/1160269663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa42c15ccf0 0x7fa42c0aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa4315e9640 1 --2- 172.21.15.142:0/1160269663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa42c065ea0 0x7fa42c15edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa430de8640 1 -- 172.21.15.142:0/1160269663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa42c15ccf0 msgr2=0x7fa42c0aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa430de8640 1 --2- 172.21.15.142:0/1160269663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa42c15ccf0 0x7fa42c0aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa430de8640 1 -- 172.21.15.142:0/1160269663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa42c065ea0 msgr2=0x7fa42c15edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa430de8640 1 --2- 172.21.15.142:0/1160269663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa42c065ea0 0x7fa42c15edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa430de8640 1 -- 172.21.15.142:0/1160269663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280094d0 con 0x7fa42c0b1e90 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa4315e9640 1 --2- 172.21.15.142:0/1160269663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa42c065ea0 0x7fa42c15edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa430de8640 1 --2- 172.21.15.142:0/1160269663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c0b1e90 0x7fa42c14f210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00d800 tx=0x7fa41c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa421ffb640 1 -- 172.21.15.142:0/1160269663 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c00b6c0 con 0x7fa42c0b1e90 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa421ffb640 1 -- 172.21.15.142:0/1160269663 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c00b820 con 0x7fa42c0b1e90 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa421ffb640 1 -- 172.21.15.142:0/1160269663 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c00bbb0 con 0x7fa42c0b1e90 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa4325eb640 1 -- 172.21.15.142:0/1160269663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42c0a89f0 con 0x7fa42c0b1e90 2023-12-01T01:35:47.760 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.757+0000 7fa4325eb640 1 -- 172.21.15.142:0/1160269663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42c0a8fb0 con 0x7fa42c0b1e90 2023-12-01T01:35:47.761 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.758+0000 7fa421ffb640 1 -- 172.21.15.142:0/1160269663 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa41c01e760 con 0x7fa42c0b1e90 2023-12-01T01:35:47.761 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.758+0000 7fa421ffb640 1 -- 172.21.15.142:0/1160269663 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fa41c014990 con 0x7fa42c0b1e90 2023-12-01T01:35:47.761 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.759+0000 7fa4325eb640 1 -- 172.21.15.142:0/1160269663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42c0618d0 con 0x7fa42c0b1e90 2023-12-01T01:35:47.763 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.761+0000 7fa421ffb640 1 -- 172.21.15.142:0/1160269663 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fa41c00bd10 con 0x7fa42c0b1e90 2023-12-01T01:35:47.764 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:47.763+0000 7f7afba45040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:35:47.821 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:47.820+0000 7f7afba45040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:35:47.841 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.839+0000 7fa4325eb640 1 -- 172.21.15.142:0/1160269663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5ba3cc79-ca5f-4da6-adcb-8a99c60652ff", "id": 2} v 0) v1 -- 0x7fa42c0ccd10 con 0x7fa42c0b1e90 2023-12-01T01:35:47.851 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.849+0000 7fa421ffb640 1 -- 172.21.15.142:0/1160269663 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5ba3cc79-ca5f-4da6-adcb-8a99c60652ff", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa41c014640 con 0x7fa42c0b1e90 2023-12-01T01:35:47.851 INFO:teuthology.orchestra.run.smithi142.stdout:2 2023-12-01T01:35:47.853 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.851+0000 7fa4325eb640 1 -- 172.21.15.142:0/1160269663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c0b1e90 msgr2=0x7fa42c14f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:47.853 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.851+0000 7fa4325eb640 1 --2- 172.21.15.142:0/1160269663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c0b1e90 0x7fa42c14f210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00d800 tx=0x7fa41c00dcc0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.853 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.851+0000 7fa4325eb640 1 -- 172.21.15.142:0/1160269663 shutdown_connections 2023-12-01T01:35:47.853 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.851+0000 7fa4325eb640 1 --2- 172.21.15.142:0/1160269663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa42c15ccf0 0x7fa42c0aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.853 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.851+0000 7fa4325eb640 1 --2- 172.21.15.142:0/1160269663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa42c0b1e90 0x7fa42c14f210 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.853 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.851+0000 7fa4325eb640 1 --2- 172.21.15.142:0/1160269663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa42c065ea0 0x7fa42c15edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:47.853 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.851+0000 7fa4325eb640 1 -- 172.21.15.142:0/1160269663 >> 172.21.15.142:0/1160269663 conn(0x7fa42c06a760 msgr2=0x7fa42c0cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:47.853 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.851+0000 7fa4325eb640 1 -- 172.21.15.142:0/1160269663 shutdown_connections 2023-12-01T01:35:47.853 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:47.851+0000 7fa4325eb640 1 -- 172.21.15.142:0/1160269663 wait complete. 2023-12-01T01:35:47.869 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new abcdd200-4c30-4703-815f-8f8adbe1dd08 3 2023-12-01T01:35:47.892 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:47.891+0000 7f1908018040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:35:47.945 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:47.943+0000 7f1908018040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:35:48.053 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:48.051+0000 7f1908018040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:35:48.062 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.061+0000 7fd6a65bf640 1 Processor -- start 2023-12-01T01:35:48.062 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.061+0000 7fd6a65bf640 1 -- start start 2023-12-01T01:35:48.064 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.062+0000 7fd6a65bf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd6a00ce160 0x7fd6a0154010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.064 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.062+0000 7fd6a65bf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a00cdcd0 con 0x7fd6a00cfb80 2023-12-01T01:35:48.064 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.062+0000 7fd6a65bf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a0161830 con 0x7fd6a0153c30 2023-12-01T01:35:48.064 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.062+0000 7fd6a65bf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a00cd3a0 con 0x7fd6a00ce160 2023-12-01T01:35:48.064 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.062+0000 7fd6a55bd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fd6a0153c30 0x7fd6a00cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:58900/0 (socket says 172.21.15.142:58900) 2023-12-01T01:35:48.064 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.062+0000 7fd6a55bd640 1 -- 172.21.15.142:0/1749068813 learned_addr learned my addr 172.21.15.142:0/1749068813 (peer_addr_for_me v1:172.21.15.142:0/0) 2023-12-01T01:35:48.064 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.062+0000 7fd6a5dbe640 1 --2- 172.21.15.142:0/1749068813 >> v2:172.21.15.142:3300/0 conn(0x7fd6a00ce160 0x7fd6a0154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.065 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.063+0000 7fd697fff640 1 -- 172.21.15.142:0/1749068813 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936219897 0 0) 0x7fd6a0161830 con 0x7fd6a0153c30 2023-12-01T01:35:48.065 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.063+0000 7fd697fff640 1 -- 172.21.15.142:0/1749068813 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd68c003520 con 0x7fd6a0153c30 2023-12-01T01:35:48.065 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.063+0000 7fd6a5dbe640 1 -- 172.21.15.142:0/1749068813 >> v1:172.21.15.142:6789/0 conn(0x7fd6a0153c30 legacy=0x7fd6a00cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.065 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.063+0000 7fd6a5dbe640 1 -- 172.21.15.142:0/1749068813 >> v1:172.21.15.115:6789/0 conn(0x7fd6a00cfb80 legacy=0x7fd6a0160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.065 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.063+0000 7fd6a5dbe640 1 -- 172.21.15.142:0/1749068813 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a00cd680 con 0x7fd6a00ce160 2023-12-01T01:35:48.066 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.064+0000 7fd6a5dbe640 1 --2- 172.21.15.142:0/1749068813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 0x7fd6a0154010 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd698009d90 tx=0x7fd69802bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72dbed8ebe249666 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:48.066 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.064+0000 7fd697fff640 1 -- 172.21.15.142:0/1749068813 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69802ccf0 con 0x7fd6a00ce160 2023-12-01T01:35:48.066 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.064+0000 7fd697fff640 1 -- 172.21.15.142:0/1749068813 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69802ce50 con 0x7fd6a00ce160 2023-12-01T01:35:48.066 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.064+0000 7fd697fff640 1 -- 172.21.15.142:0/1749068813 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698034660 con 0x7fd6a00ce160 2023-12-01T01:35:48.067 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.065+0000 7fd6a65bf640 1 -- 172.21.15.142:0/1749068813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 msgr2=0x7fd6a0154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.067 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.065+0000 7fd6a65bf640 1 --2- 172.21.15.142:0/1749068813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 0x7fd6a0154010 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd698009d90 tx=0x7fd69802bf40 comp rx=0 tx=0).stop 2023-12-01T01:35:48.067 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.065+0000 7fd6a65bf640 1 -- 172.21.15.142:0/1749068813 shutdown_connections 2023-12-01T01:35:48.067 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.065+0000 7fd6a65bf640 1 --2- 172.21.15.142:0/1749068813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 0x7fd6a0154010 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.067 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.065+0000 7fd6a65bf640 1 -- 172.21.15.142:0/1749068813 >> 172.21.15.142:0/1749068813 conn(0x7fd6a006a0b0 msgr2=0x7fd6a006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:48.067 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.065+0000 7fd6a65bf640 1 -- 172.21.15.142:0/1749068813 shutdown_connections 2023-12-01T01:35:48.067 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.065+0000 7fd6a65bf640 1 -- 172.21.15.142:0/1749068813 wait complete. 2023-12-01T01:35:48.068 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.067+0000 7fd6a65bf640 1 Processor -- start 2023-12-01T01:35:48.068 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.067+0000 7fd6a65bf640 1 -- start start 2023-12-01T01:35:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a65bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 0x7fd6a0142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a65bf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6a00cfb80 0x7fd6a013fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a65bf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6a0153c30 0x7fd6a0141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a65bf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a0140070 con 0x7fd6a00cfb80 2023-12-01T01:35:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a65bf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a01402b0 con 0x7fd6a00ce160 2023-12-01T01:35:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a65bf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a0140590 con 0x7fd6a0153c30 2023-12-01T01:35:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a55bd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 0x7fd6a0142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a55bd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 0x7fd6a0142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32962/0 (socket says 172.21.15.142:32962) 2023-12-01T01:35:48.071 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a55bd640 1 -- 172.21.15.142:0/4182203579 learned_addr learned my addr 172.21.15.142:0/4182203579 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a5dbe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6a0153c30 0x7fd6a0141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.068+0000 7fd6a4dbc640 1 --2- 172.21.15.142:0/4182203579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6a00cfb80 0x7fd6a013fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd6a55bd640 1 -- 172.21.15.142:0/4182203579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6a0153c30 msgr2=0x7fd6a0141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd6a55bd640 1 --2- 172.21.15.142:0/4182203579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6a0153c30 0x7fd6a0141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.072 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd6a55bd640 1 -- 172.21.15.142:0/4182203579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6a00cfb80 msgr2=0x7fd6a013fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd6a55bd640 1 --2- 172.21.15.142:0/4182203579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6a00cfb80 0x7fd6a013fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd6a55bd640 1 -- 172.21.15.142:0/4182203579 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698009a70 con 0x7fd6a00ce160 2023-12-01T01:35:48.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd6a5dbe640 1 --2- 172.21.15.142:0/4182203579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6a0153c30 0x7fd6a0141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:35:48.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd6a4dbc640 1 --2- 172.21.15.142:0/4182203579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6a00cfb80 0x7fd6a013fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:35:48.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd6a55bd640 1 --2- 172.21.15.142:0/4182203579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 0x7fd6a0142720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00c410 tx=0x7fd69c00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:48.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd695ffb640 1 -- 172.21.15.142:0/4182203579 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c005ce0 con 0x7fd6a00ce160 2023-12-01T01:35:48.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd695ffb640 1 -- 172.21.15.142:0/4182203579 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c005e40 con 0x7fd6a00ce160 2023-12-01T01:35:48.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd695ffb640 1 -- 172.21.15.142:0/4182203579 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0072f0 con 0x7fd6a00ce160 2023-12-01T01:35:48.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd6a65bf640 1 -- 172.21.15.142:0/4182203579 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a0140870 con 0x7fd6a00ce160 2023-12-01T01:35:48.074 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd6a65bf640 1 -- 172.21.15.142:0/4182203579 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a0140e30 con 0x7fd6a00ce160 2023-12-01T01:35:48.074 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.069+0000 7fd695ffb640 1 -- 172.21.15.142:0/4182203579 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd69c01c440 con 0x7fd6a00ce160 2023-12-01T01:35:48.074 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.070+0000 7fd695ffb640 1 -- 172.21.15.142:0/4182203579 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd69c012980 con 0x7fd6a00ce160 2023-12-01T01:35:48.074 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.070+0000 7fd6a65bf640 1 -- 172.21.15.142:0/4182203579 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a00618d0 con 0x7fd6a00ce160 2023-12-01T01:35:48.074 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.073+0000 7fd695ffb640 1 -- 172.21.15.142:0/4182203579 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fd69c017070 con 0x7fd6a00ce160 2023-12-01T01:35:48.152 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.151+0000 7fd6a65bf640 1 -- 172.21.15.142:0/4182203579 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "abcdd200-4c30-4703-815f-8f8adbe1dd08", "id": 3} v 0) v1 -- 0x7fd6a00d3110 con 0x7fd6a00ce160 2023-12-01T01:35:48.162 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.160+0000 7fd695ffb640 1 -- 172.21.15.142:0/4182203579 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "abcdd200-4c30-4703-815f-8f8adbe1dd08", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd69c012df0 con 0x7fd6a00ce160 2023-12-01T01:35:48.162 INFO:teuthology.orchestra.run.smithi142.stdout:3 2023-12-01T01:35:48.164 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.162+0000 7fd6a65bf640 1 -- 172.21.15.142:0/4182203579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 msgr2=0x7fd6a0142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.164 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.162+0000 7fd6a65bf640 1 --2- 172.21.15.142:0/4182203579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 0x7fd6a0142720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00c410 tx=0x7fd69c00c8d0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.164 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.162+0000 7fd6a65bf640 1 -- 172.21.15.142:0/4182203579 shutdown_connections 2023-12-01T01:35:48.164 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.162+0000 7fd6a65bf640 1 --2- 172.21.15.142:0/4182203579 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6a0153c30 0x7fd6a0141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.164 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.162+0000 7fd6a65bf640 1 --2- 172.21.15.142:0/4182203579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6a00cfb80 0x7fd6a013fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.164 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.162+0000 7fd6a65bf640 1 --2- 172.21.15.142:0/4182203579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6a00ce160 0x7fd6a0142720 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.164 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.162+0000 7fd6a65bf640 1 -- 172.21.15.142:0/4182203579 >> 172.21.15.142:0/4182203579 conn(0x7fd6a006a0b0 msgr2=0x7fd6a00b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:48.164 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.162+0000 7fd6a65bf640 1 -- 172.21.15.142:0/4182203579 shutdown_connections 2023-12-01T01:35:48.165 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.162+0000 7fd6a65bf640 1 -- 172.21.15.142:0/4182203579 wait complete. 2023-12-01T01:35:48.165 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:48.163+0000 7f1908018040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:35:48.181 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new ce369f10-36d4-4573-bd78-b67c31ceb9bb 4 2023-12-01T01:35:48.303 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:48.301+0000 7f7afba45040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:35:48.323 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:48.322+0000 7f1908018040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:35:48.368 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.366+0000 7f2dd9c49640 1 Processor -- start 2023-12-01T01:35:48.368 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.367+0000 7f2dd9c49640 1 -- start start 2023-12-01T01:35:48.370 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.368+0000 7f2dd9c49640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2dd4153c30 0x7f2dd40ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.370 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.368+0000 7f2dd9c49640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd40cfb80 0x7f2dd4160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.370 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.368+0000 7f2dd9c49640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd40cdcd0 con 0x7f2dd4154010 2023-12-01T01:35:48.370 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.368+0000 7f2dd9c49640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd4161830 con 0x7f2dd4153c30 2023-12-01T01:35:48.371 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.368+0000 7f2dd9c49640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40cd3a0 con 0x7f2dd40cfb80 2023-12-01T01:35:48.371 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.368+0000 7f2dd9448640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd40cfb80 0x7f2dd4160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.371 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.368+0000 7f2dd3fff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f2dd4154010 0x7f2dd40cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:58904/0 (socket says 172.21.15.142:58904) 2023-12-01T01:35:48.371 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.368+0000 7f2dd3fff640 1 -- 172.21.15.142:0/4215850516 learned_addr learned my addr 172.21.15.142:0/4215850516 (peer_addr_for_me v1:172.21.15.142:0/0) 2023-12-01T01:35:48.371 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.369+0000 7f2dd8c47640 1 --2- 172.21.15.142:0/4215850516 >> v2:172.21.15.115:3300/0 conn(0x7f2dd4153c30 0x7f2dd40ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.371 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.369+0000 7f2dd9448640 1 -- 172.21.15.142:0/4215850516 >> v2:172.21.15.115:3300/0 conn(0x7f2dd4153c30 msgr2=0x7f2dd40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:48.371 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.369+0000 7f2dd9448640 1 --2- 172.21.15.142:0/4215850516 >> v2:172.21.15.115:3300/0 conn(0x7f2dd4153c30 0x7f2dd40ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.371 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.369+0000 7f2dd37fe640 1 -- 172.21.15.142:0/4215850516 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750210349 0 0) 0x7f2dd40cdcd0 con 0x7f2dd4154010 2023-12-01T01:35:48.371 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.369+0000 7f2dd9448640 1 -- 172.21.15.142:0/4215850516 >> v1:172.21.15.142:6789/0 conn(0x7f2dd4154010 legacy=0x7f2dd40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.371 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.369+0000 7f2dd9448640 1 -- 172.21.15.142:0/4215850516 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd40cd680 con 0x7f2dd40cfb80 2023-12-01T01:35:48.372 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.370+0000 7f2dd9448640 1 --2- 172.21.15.142:0/4215850516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd40cfb80 0x7f2dd4160ac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009d90 tx=0x7f2dcc02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=779603795525f088 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.370+0000 7f2dd37fe640 1 -- 172.21.15.142:0/4215850516 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc02cc50 con 0x7f2dd40cfb80 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.370+0000 7f2dd37fe640 1 -- 172.21.15.142:0/4215850516 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc02cdb0 con 0x7f2dd40cfb80 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.370+0000 7f2dd37fe640 1 -- 172.21.15.142:0/4215850516 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc034660 con 0x7f2dd40cfb80 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.371+0000 7f2dd9c49640 1 -- 172.21.15.142:0/4215850516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd40cfb80 msgr2=0x7f2dd4160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.371+0000 7f2dd9c49640 1 --2- 172.21.15.142:0/4215850516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd40cfb80 0x7f2dd4160ac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009d90 tx=0x7f2dcc02bea0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.371+0000 7f2dd9c49640 1 -- 172.21.15.142:0/4215850516 shutdown_connections 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.371+0000 7f2dd9c49640 1 --2- 172.21.15.142:0/4215850516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd40cfb80 0x7f2dd4160ac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.371+0000 7f2dd9c49640 1 --2- 172.21.15.142:0/4215850516 >> v2:172.21.15.115:3300/0 conn(0x7f2dd4153c30 0x7f2dd40ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.371+0000 7f2dd9c49640 1 -- 172.21.15.142:0/4215850516 >> 172.21.15.142:0/4215850516 conn(0x7f2dd406a0b0 msgr2=0x7f2dd406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.371+0000 7f2dd9c49640 1 -- 172.21.15.142:0/4215850516 shutdown_connections 2023-12-01T01:35:48.373 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.371+0000 7f2dd9c49640 1 -- 172.21.15.142:0/4215850516 wait complete. 2023-12-01T01:35:48.374 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:48.371+0000 7f1908018040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:35:48.374 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.372+0000 7f2dd9c49640 1 Processor -- start 2023-12-01T01:35:48.374 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.373+0000 7f2dd9c49640 1 -- start start 2023-12-01T01:35:48.376 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.373+0000 7f2dd9c49640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd40cfb80 0x7f2dd40c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.376 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.373+0000 7f2dd9c49640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd4153c30 0x7f2dd40c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.376 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.373+0000 7f2dd9c49640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4154010 0x7f2dd40c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.376 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9c49640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40cb730 con 0x7f2dd40cfb80 2023-12-01T01:35:48.376 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9c49640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40c7b10 con 0x7f2dd4154010 2023-12-01T01:35:48.376 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9c49640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40cad90 con 0x7f2dd4153c30 2023-12-01T01:35:48.376 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9448640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4154010 0x7f2dd40c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.376 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd3fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd4153c30 0x7f2dd40c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.376 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9448640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4154010 0x7f2dd40c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32968/0 (socket says 172.21.15.142:32968) 2023-12-01T01:35:48.377 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9448640 1 -- 172.21.15.142:0/2435670059 learned_addr learned my addr 172.21.15.142:0/2435670059 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:48.377 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd8c47640 1 --2- 172.21.15.142:0/2435670059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd40cfb80 0x7f2dd40c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.377 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9448640 1 -- 172.21.15.142:0/2435670059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd4153c30 msgr2=0x7f2dd40c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.377 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9448640 1 --2- 172.21.15.142:0/2435670059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd4153c30 0x7f2dd40c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.377 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9448640 1 -- 172.21.15.142:0/2435670059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd40cfb80 msgr2=0x7f2dd40c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.377 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9448640 1 --2- 172.21.15.142:0/2435670059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd40cfb80 0x7f2dd40c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.377 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd9448640 1 -- 172.21.15.142:0/2435670059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc009a70 con 0x7f2dd4154010 2023-12-01T01:35:48.377 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.374+0000 7f2dd3fff640 1 --2- 172.21.15.142:0/2435670059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd4153c30 0x7f2dd40c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:35:48.377 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.375+0000 7f2dd9448640 1 --2- 172.21.15.142:0/2435670059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4154010 0x7f2dd40c89b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009eb0 tx=0x7f2dcc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:48.378 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.375+0000 7f2dd17fa640 1 -- 172.21.15.142:0/2435670059 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc040650 con 0x7f2dd4154010 2023-12-01T01:35:48.378 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.375+0000 7f2dd17fa640 1 -- 172.21.15.142:0/2435670059 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc0407d0 con 0x7f2dd4154010 2023-12-01T01:35:48.378 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.375+0000 7f2dd17fa640 1 -- 172.21.15.142:0/2435670059 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc040b80 con 0x7f2dd4154010 2023-12-01T01:35:48.378 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.375+0000 7f2dd9c49640 1 -- 172.21.15.142:0/2435670059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd40cb070 con 0x7f2dd4154010 2023-12-01T01:35:48.378 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.375+0000 7f2dd9c49640 1 -- 172.21.15.142:0/2435670059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd40c91d0 con 0x7f2dd4154010 2023-12-01T01:35:48.379 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.375+0000 7f2dd17fa640 1 -- 172.21.15.142:0/2435670059 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2dcc049400 con 0x7f2dd4154010 2023-12-01T01:35:48.379 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.375+0000 7f2dd17fa640 1 -- 172.21.15.142:0/2435670059 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2dcc049bc0 con 0x7f2dd4154010 2023-12-01T01:35:48.379 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.376+0000 7f2dd9c49640 1 -- 172.21.15.142:0/2435670059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd40cb870 con 0x7f2dd4154010 2023-12-01T01:35:48.380 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.379+0000 7f2dd17fa640 1 -- 172.21.15.142:0/2435670059 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f2dcc067b10 con 0x7f2dd4154010 2023-12-01T01:35:48.458 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.457+0000 7f2dd9c49640 1 -- 172.21.15.142:0/2435670059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ce369f10-36d4-4573-bd78-b67c31ceb9bb", "id": 4} v 0) v1 -- 0x7f2dd40618d0 con 0x7f2dd4154010 2023-12-01T01:35:48.468 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.466+0000 7f2dd17fa640 1 -- 172.21.15.142:0/2435670059 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce369f10-36d4-4573-bd78-b67c31ceb9bb", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2dcc056030 con 0x7f2dd4154010 2023-12-01T01:35:48.468 INFO:teuthology.orchestra.run.smithi142.stdout:4 2023-12-01T01:35:48.469 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:48.467+0000 7f7afba45040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:35:48.470 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.468+0000 7f2dd9c49640 1 -- 172.21.15.142:0/2435670059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4154010 msgr2=0x7f2dd40c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.468+0000 7f2dd9c49640 1 --2- 172.21.15.142:0/2435670059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4154010 0x7f2dd40c89b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009eb0 tx=0x7f2dcc004a40 comp rx=0 tx=0).stop 2023-12-01T01:35:48.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.468+0000 7f2dd9c49640 1 -- 172.21.15.142:0/2435670059 shutdown_connections 2023-12-01T01:35:48.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.468+0000 7f2dd9c49640 1 --2- 172.21.15.142:0/2435670059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2dd4153c30 0x7f2dd40c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.468+0000 7f2dd9c49640 1 --2- 172.21.15.142:0/2435670059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2dd4154010 0x7f2dd40c89b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.468+0000 7f2dd9c49640 1 --2- 172.21.15.142:0/2435670059 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dd40cfb80 0x7f2dd40c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.468+0000 7f2dd9c49640 1 -- 172.21.15.142:0/2435670059 >> 172.21.15.142:0/2435670059 conn(0x7f2dd406a0b0 msgr2=0x7f2dd415c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:48.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.468+0000 7f2dd9c49640 1 -- 172.21.15.142:0/2435670059 shutdown_connections 2023-12-01T01:35:48.471 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.468+0000 7f2dd9c49640 1 -- 172.21.15.142:0/2435670059 wait complete. 2023-12-01T01:35:48.486 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new ec117800-c03d-4c2a-bdb2-b5c96726712a 5 2023-12-01T01:35:48.529 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:48.527+0000 7f1908018040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:35:48.602 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:48.600+0000 7f7afba45040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:35:48.658 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:48.656+0000 7f7afba45040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:35:48.668 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.667+0000 7ff98e799640 1 Processor -- start 2023-12-01T01:35:48.669 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.667+0000 7ff98e799640 1 -- start start 2023-12-01T01:35:48.671 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff98e799640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff9880cfd60 0x7ff9880cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.671 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff98e799640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff9880b5fa0 0x7ff988161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.671 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff98e799640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9880ce5e0 con 0x7ff98815cd90 2023-12-01T01:35:48.671 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff98e799640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9880d0ec0 con 0x7ff9880b5fa0 2023-12-01T01:35:48.671 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff98e799640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff988161950 con 0x7ff9880cfd60 2023-12-01T01:35:48.671 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff97f7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff9880cfd60 0x7ff9880cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.671 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff97f7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff9880cfd60 0x7ff9880cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32976/0 (socket says 172.21.15.142:32976) 2023-12-01T01:35:48.671 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff97f7fe640 1 -- 172.21.15.142:0/609662204 learned_addr learned my addr 172.21.15.142:0/609662204 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:48.671 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff98cd36640 1 --2- 172.21.15.142:0/609662204 >> v2:172.21.15.115:3300/0 conn(0x7ff9880b5fa0 0x7ff988161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.671 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff97f7fe640 1 -- 172.21.15.142:0/609662204 >> v2:172.21.15.115:3300/0 conn(0x7ff9880b5fa0 msgr2=0x7ff988161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:48.672 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff97f7fe640 1 --2- 172.21.15.142:0/609662204 >> v2:172.21.15.115:3300/0 conn(0x7ff9880b5fa0 0x7ff988161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.672 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff97f7fe640 1 -- 172.21.15.142:0/609662204 >> v1:172.21.15.115:6789/0 conn(0x7ff98815cd90 legacy=0x7ff9880cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:48.672 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.669+0000 7ff97f7fe640 1 -- 172.21.15.142:0/609662204 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9880ce300 con 0x7ff9880cfd60 2023-12-01T01:35:48.672 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.670+0000 7ff97f7fe640 1 --2- 172.21.15.142:0/609662204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880cfd60 0x7ff9880cdd20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff9880cdcb0 tx=0x7ff97002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8d907c5bddd63b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:48.672 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.671+0000 7ff9877fe640 1 -- 172.21.15.142:0/609662204 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970031870 con 0x7ff9880cfd60 2023-12-01T01:35:48.672 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.671+0000 7ff9877fe640 1 -- 172.21.15.142:0/609662204 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9700319d0 con 0x7ff9880cfd60 2023-12-01T01:35:48.673 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.671+0000 7ff9877fe640 1 -- 172.21.15.142:0/609662204 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970031d60 con 0x7ff9880cfd60 2023-12-01T01:35:48.673 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.671+0000 7ff98e799640 1 -- 172.21.15.142:0/609662204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880cfd60 msgr2=0x7ff9880cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.673 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.671+0000 7ff98e799640 1 --2- 172.21.15.142:0/609662204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880cfd60 0x7ff9880cdd20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff9880cdcb0 tx=0x7ff97002b910 comp rx=0 tx=0).stop 2023-12-01T01:35:48.673 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.671+0000 7ff98e799640 1 -- 172.21.15.142:0/609662204 shutdown_connections 2023-12-01T01:35:48.674 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.671+0000 7ff98e799640 1 --2- 172.21.15.142:0/609662204 >> v2:172.21.15.115:3300/0 conn(0x7ff9880b5fa0 0x7ff988161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.674 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.671+0000 7ff98e799640 1 --2- 172.21.15.142:0/609662204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880cfd60 0x7ff9880cdd20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.674 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.671+0000 7ff98e799640 1 -- 172.21.15.142:0/609662204 >> 172.21.15.142:0/609662204 conn(0x7ff988065960 msgr2=0x7ff988065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:48.674 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.672+0000 7ff98e799640 1 -- 172.21.15.142:0/609662204 shutdown_connections 2023-12-01T01:35:48.674 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.672+0000 7ff98e799640 1 -- 172.21.15.142:0/609662204 wait complete. 2023-12-01T01:35:48.674 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.673+0000 7ff98e799640 1 Processor -- start 2023-12-01T01:35:48.675 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.673+0000 7ff98e799640 1 -- start start 2023-12-01T01:35:48.677 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.674+0000 7ff98e799640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880b5fa0 0x7ff9880cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.677 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.674+0000 7ff98e799640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9880cfd60 0x7ff9880c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.677 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.674+0000 7ff98e799640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98815cd90 0x7ff9881583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.677 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.674+0000 7ff98e799640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880ca660 con 0x7ff9880cfd60 2023-12-01T01:35:48.677 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.674+0000 7ff98e799640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880c9d90 con 0x7ff9880b5fa0 2023-12-01T01:35:48.677 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.674+0000 7ff98e799640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880ccf50 con 0x7ff98815cd90 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.674+0000 7ff987fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880b5fa0 0x7ff9880cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.674+0000 7ff98cd36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98815cd90 0x7ff9881583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.674+0000 7ff987fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880b5fa0 0x7ff9880cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:32982/0 (socket says 172.21.15.142:32982) 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff987fff640 1 -- 172.21.15.142:0/1180948808 learned_addr learned my addr 172.21.15.142:0/1180948808 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff98cd36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98815cd90 0x7ff9881583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:42102/0 (socket says 172.21.15.142:42102) 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff97f7fe640 1 --2- 172.21.15.142:0/1180948808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9880cfd60 0x7ff9880c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff98cd36640 1 -- 172.21.15.142:0/1180948808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880b5fa0 msgr2=0x7ff9880cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff98cd36640 1 --2- 172.21.15.142:0/1180948808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880b5fa0 0x7ff9880cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff98cd36640 1 -- 172.21.15.142:0/1180948808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9880cfd60 msgr2=0x7ff9880c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff98cd36640 1 --2- 172.21.15.142:0/1180948808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9880cfd60 0x7ff9880c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff98cd36640 1 -- 172.21.15.142:0/1180948808 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9700094d0 con 0x7ff98815cd90 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff987fff640 1 --2- 172.21.15.142:0/1180948808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880b5fa0 0x7ff9880cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff98cd36640 1 --2- 172.21.15.142:0/1180948808 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98815cd90 0x7ff9881583f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff98000d530 tx=0x7ff98000d9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:48.678 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.675+0000 7ff9857fa640 1 -- 172.21.15.142:0/1180948808 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9800125f0 con 0x7ff98815cd90 2023-12-01T01:35:48.679 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.676+0000 7ff9857fa640 1 -- 172.21.15.142:0/1180948808 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff980012750 con 0x7ff98815cd90 2023-12-01T01:35:48.679 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.676+0000 7ff9857fa640 1 -- 172.21.15.142:0/1180948808 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980012ae0 con 0x7ff98815cd90 2023-12-01T01:35:48.679 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.676+0000 7ff98e799640 1 -- 172.21.15.142:0/1180948808 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98815d4c0 con 0x7ff98815cd90 2023-12-01T01:35:48.679 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.676+0000 7ff98e799640 1 -- 172.21.15.142:0/1180948808 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9880ca280 con 0x7ff98815cd90 2023-12-01T01:35:48.679 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.676+0000 7ff9857fa640 1 -- 172.21.15.142:0/1180948808 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff98001e460 con 0x7ff98815cd90 2023-12-01T01:35:48.679 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.676+0000 7ff9857fa640 1 -- 172.21.15.142:0/1180948808 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7ff980014b40 con 0x7ff98815cd90 2023-12-01T01:35:48.679 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.677+0000 7ff98e799640 1 -- 172.21.15.142:0/1180948808 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9880ca7a0 con 0x7ff98815cd90 2023-12-01T01:35:48.681 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.679+0000 7ff9857fa640 1 -- 172.21.15.142:0/1180948808 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7ff980014de0 con 0x7ff98815cd90 2023-12-01T01:35:48.759 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.757+0000 7ff98e799640 1 -- 172.21.15.142:0/1180948808 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ec117800-c03d-4c2a-bdb2-b5c96726712a", "id": 5} v 0) v1 -- 0x7ff98815c5f0 con 0x7ff98815cd90 2023-12-01T01:35:48.769 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.767+0000 7ff9857fa640 1 -- 172.21.15.142:0/1180948808 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ec117800-c03d-4c2a-bdb2-b5c96726712a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff980014690 con 0x7ff98815cd90 2023-12-01T01:35:48.769 INFO:teuthology.orchestra.run.smithi142.stdout:5 2023-12-01T01:35:48.770 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.768+0000 7ff98e799640 1 -- 172.21.15.142:0/1180948808 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98815cd90 msgr2=0x7ff9881583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.770 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.768+0000 7ff98e799640 1 --2- 172.21.15.142:0/1180948808 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98815cd90 0x7ff9881583f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff98000d530 tx=0x7ff98000d9f0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.770 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.769+0000 7ff98e799640 1 -- 172.21.15.142:0/1180948808 shutdown_connections 2023-12-01T01:35:48.770 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.769+0000 7ff98e799640 1 --2- 172.21.15.142:0/1180948808 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98815cd90 0x7ff9881583f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.770 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.769+0000 7ff98e799640 1 --2- 172.21.15.142:0/1180948808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9880cfd60 0x7ff9880c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.771 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.769+0000 7ff98e799640 1 --2- 172.21.15.142:0/1180948808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9880b5fa0 0x7ff9880cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.771 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.769+0000 7ff98e799640 1 -- 172.21.15.142:0/1180948808 >> 172.21.15.142:0/1180948808 conn(0x7ff988065960 msgr2=0x7ff98815bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:48.771 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.769+0000 7ff98e799640 1 -- 172.21.15.142:0/1180948808 shutdown_connections 2023-12-01T01:35:48.771 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.769+0000 7ff98e799640 1 -- 172.21.15.142:0/1180948808 wait complete. 2023-12-01T01:35:48.774 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:48.772+0000 7f7afba45040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:35:48.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new c9536043-c132-4a02-a76a-2cbef11be908 6 2023-12-01T01:35:48.892 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:48.890+0000 7f7afba45040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:35:48.973 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.971+0000 7fa645e3e640 1 Processor -- start 2023-12-01T01:35:48.973 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.971+0000 7fa645e3e640 1 -- start start 2023-12-01T01:35:48.975 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.973+0000 7fa645e3e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400ce160 0x7fa640154010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.975 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.973+0000 7fa645e3e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6400cdcd0 con 0x7fa640153c30 2023-12-01T01:35:48.975 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.973+0000 7fa645e3e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa640161830 con 0x7fa6400cfb80 2023-12-01T01:35:48.975 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.973+0000 7fa645e3e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6400cd3a0 con 0x7fa6400ce160 2023-12-01T01:35:48.975 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.973+0000 7fa64563d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400ce160 0x7fa640154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.975 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.973+0000 7fa63ffff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fa6400cfb80 0x7fa640160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:58910/0 (socket says 172.21.15.142:58910) 2023-12-01T01:35:48.975 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.973+0000 7fa63ffff640 1 -- 172.21.15.142:0/3291983792 learned_addr learned my addr 172.21.15.142:0/3291983792 (peer_addr_for_me v1:172.21.15.142:0/0) 2023-12-01T01:35:48.976 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.974+0000 7fa64563d640 1 -- 172.21.15.142:0/3291983792 >> v1:172.21.15.142:6789/0 conn(0x7fa6400cfb80 legacy=0x7fa640160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.976 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.974+0000 7fa64563d640 1 -- 172.21.15.142:0/3291983792 >> v1:172.21.15.115:6789/0 conn(0x7fa640153c30 legacy=0x7fa6400cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:48.976 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.974+0000 7fa64563d640 1 -- 172.21.15.142:0/3291983792 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6400cd680 con 0x7fa6400ce160 2023-12-01T01:35:48.976 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.974+0000 7fa63f7fe640 1 -- 172.21.15.142:0/3291983792 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721688077 0 0) 0x7fa640161830 con 0x7fa6400cfb80 2023-12-01T01:35:48.977 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.975+0000 7fa64563d640 1 --2- 172.21.15.142:0/3291983792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400ce160 0x7fa640154010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa638009dc0 tx=0x7fa63802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da4413bbedca58b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:48.977 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.975+0000 7fa63f7fe640 1 -- 172.21.15.142:0/3291983792 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63802cc40 con 0x7fa6400ce160 2023-12-01T01:35:48.977 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.975+0000 7fa63f7fe640 1 -- 172.21.15.142:0/3291983792 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63802cda0 con 0x7fa6400ce160 2023-12-01T01:35:48.977 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.975+0000 7fa63f7fe640 1 -- 172.21.15.142:0/3291983792 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638034660 con 0x7fa6400ce160 2023-12-01T01:35:48.978 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.976+0000 7fa645e3e640 1 -- 172.21.15.142:0/3291983792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400ce160 msgr2=0x7fa640154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.978 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.976+0000 7fa645e3e640 1 --2- 172.21.15.142:0/3291983792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400ce160 0x7fa640154010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa638009dc0 tx=0x7fa63802be90 comp rx=0 tx=0).stop 2023-12-01T01:35:48.978 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.976+0000 7fa645e3e640 1 -- 172.21.15.142:0/3291983792 shutdown_connections 2023-12-01T01:35:48.978 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.976+0000 7fa645e3e640 1 --2- 172.21.15.142:0/3291983792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400ce160 0x7fa640154010 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.978 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.976+0000 7fa645e3e640 1 -- 172.21.15.142:0/3291983792 >> 172.21.15.142:0/3291983792 conn(0x7fa64006a0b0 msgr2=0x7fa64006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:48.978 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.976+0000 7fa645e3e640 1 -- 172.21.15.142:0/3291983792 shutdown_connections 2023-12-01T01:35:48.978 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.976+0000 7fa645e3e640 1 -- 172.21.15.142:0/3291983792 wait complete. 2023-12-01T01:35:48.979 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.978+0000 7fa645e3e640 1 Processor -- start 2023-12-01T01:35:48.979 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.978+0000 7fa645e3e640 1 -- start start 2023-12-01T01:35:48.981 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa645e3e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6400ce160 0x7fa6400b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.981 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa645e3e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400cfb80 0x7fa6400b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.981 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa645e3e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa640153c30 0x7fa64015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:48.981 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa645e3e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6401f6760 con 0x7fa6400ce160 2023-12-01T01:35:48.981 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa645e3e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64015ae10 con 0x7fa640153c30 2023-12-01T01:35:48.981 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa645e3e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6400ccf00 con 0x7fa6400cfb80 2023-12-01T01:35:48.981 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa63ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400cfb80 0x7fa6400b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.981 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa63ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400cfb80 0x7fa6400b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:42112/0 (socket says 172.21.15.142:42112) 2023-12-01T01:35:48.982 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa64563d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa640153c30 0x7fa64015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.982 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa63ffff640 1 -- 172.21.15.142:0/3180224466 learned_addr learned my addr 172.21.15.142:0/3180224466 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:48.982 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa644e3c640 1 --2- 172.21.15.142:0/3180224466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6400ce160 0x7fa6400b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:48.982 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa63ffff640 1 -- 172.21.15.142:0/3180224466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa640153c30 msgr2=0x7fa64015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.982 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa63ffff640 1 --2- 172.21.15.142:0/3180224466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa640153c30 0x7fa64015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.982 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa63ffff640 1 -- 172.21.15.142:0/3180224466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6400ce160 msgr2=0x7fa6400b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:48.982 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa63ffff640 1 --2- 172.21.15.142:0/3180224466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6400ce160 0x7fa6400b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:48.982 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.979+0000 7fa63ffff640 1 -- 172.21.15.142:0/3180224466 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa638009a70 con 0x7fa6400cfb80 2023-12-01T01:35:48.982 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.980+0000 7fa63ffff640 1 --2- 172.21.15.142:0/3180224466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400cfb80 0x7fa6400b3b30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa63000c4b0 tx=0x7fa63000c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:48.983 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.980+0000 7fa63d7fa640 1 -- 172.21.15.142:0/3180224466 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300074e0 con 0x7fa6400cfb80 2023-12-01T01:35:48.983 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.980+0000 7fa63d7fa640 1 -- 172.21.15.142:0/3180224466 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa630007660 con 0x7fa6400cfb80 2023-12-01T01:35:48.983 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.980+0000 7fa645e3e640 1 -- 172.21.15.142:0/3180224466 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6400b41f0 con 0x7fa6400cfb80 2023-12-01T01:35:48.983 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.980+0000 7fa63d7fa640 1 -- 172.21.15.142:0/3180224466 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630004a20 con 0x7fa6400cfb80 2023-12-01T01:35:48.983 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.980+0000 7fa645e3e640 1 -- 172.21.15.142:0/3180224466 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6400b47b0 con 0x7fa6400cfb80 2023-12-01T01:35:48.983 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.980+0000 7fa63d7fa640 1 -- 172.21.15.142:0/3180224466 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa63001d440 con 0x7fa6400cfb80 2023-12-01T01:35:48.983 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.980+0000 7fa63d7fa640 1 -- 172.21.15.142:0/3180224466 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fa630012ad0 con 0x7fa6400cfb80 2023-12-01T01:35:48.984 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.981+0000 7fa645e3e640 1 -- 172.21.15.142:0/3180224466 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6400618d0 con 0x7fa6400cfb80 2023-12-01T01:35:48.985 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:48.983+0000 7fa63d7fa640 1 -- 172.21.15.142:0/3180224466 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fa630018070 con 0x7fa6400cfb80 2023-12-01T01:35:49.059 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:49.057+0000 7f7afba45040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:35:49.062 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.061+0000 7fa645e3e640 1 -- 172.21.15.142:0/3180224466 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c9536043-c132-4a02-a76a-2cbef11be908", "id": 6} v 0) v1 -- 0x7fa640070750 con 0x7fa6400cfb80 2023-12-01T01:35:49.071 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.070+0000 7fa63d7fa640 1 -- 172.21.15.142:0/3180224466 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c9536043-c132-4a02-a76a-2cbef11be908", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa6300125d0 con 0x7fa6400cfb80 2023-12-01T01:35:49.072 INFO:teuthology.orchestra.run.smithi142.stdout:6 2023-12-01T01:35:49.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.071+0000 7fa645e3e640 1 -- 172.21.15.142:0/3180224466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400cfb80 msgr2=0x7fa6400b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.071+0000 7fa645e3e640 1 --2- 172.21.15.142:0/3180224466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400cfb80 0x7fa6400b3b30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa63000c4b0 tx=0x7fa63000c970 comp rx=0 tx=0).stop 2023-12-01T01:35:49.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.071+0000 7fa645e3e640 1 -- 172.21.15.142:0/3180224466 shutdown_connections 2023-12-01T01:35:49.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.071+0000 7fa645e3e640 1 --2- 172.21.15.142:0/3180224466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6400cfb80 0x7fa6400b3b30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.073 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.071+0000 7fa645e3e640 1 --2- 172.21.15.142:0/3180224466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa640153c30 0x7fa64015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.074 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.071+0000 7fa645e3e640 1 --2- 172.21.15.142:0/3180224466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6400ce160 0x7fa6400b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.074 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.071+0000 7fa645e3e640 1 -- 172.21.15.142:0/3180224466 >> 172.21.15.142:0/3180224466 conn(0x7fa64006a0b0 msgr2=0x7fa64015c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:49.074 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.071+0000 7fa645e3e640 1 -- 172.21.15.142:0/3180224466 shutdown_connections 2023-12-01T01:35:49.074 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.071+0000 7fa645e3e640 1 -- 172.21.15.142:0/3180224466 wait complete. 2023-12-01T01:35:49.089 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new f333b559-012d-4988-a29f-21609f533af8 7 2023-12-01T01:35:49.114 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:49.113+0000 7f7afba45040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:35:49.279 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:49.277+0000 7f7afba45040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:35:49.283 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.282+0000 7f9a10316640 1 Processor -- start 2023-12-01T01:35:49.283 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.282+0000 7f9a10316640 1 -- start start 2023-12-01T01:35:49.285 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.283+0000 7f9a10316640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9a0815cf70 0x7f9a080b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:49.285 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.283+0000 7f9a10316640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080d0260 0x7f9a080b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:49.285 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.283+0000 7f9a10316640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a080b7760 con 0x7f9a080b18f0 2023-12-01T01:35:49.286 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.283+0000 7f9a10316640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a080b2170 con 0x7f9a0815cf70 2023-12-01T01:35:49.286 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.283+0000 7f9a10316640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080b8b70 con 0x7f9a080d0260 2023-12-01T01:35:49.286 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.283+0000 7f9a0e0b2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9a0815cf70 0x7f9a080b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:49.286 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.283+0000 7f9a0e8b3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080d0260 0x7f9a080b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:49.286 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.284+0000 7f9a0e0b2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9a0815cf70 0x7f9a080b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33004/0 (socket says 172.21.15.142:33004) 2023-12-01T01:35:49.286 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.284+0000 7f9a0e0b2640 1 -- 172.21.15.142:0/1813857203 learned_addr learned my addr 172.21.15.142:0/1813857203 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:49.286 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.284+0000 7f9a0e8b3640 1 -- 172.21.15.142:0/1813857203 >> v2:172.21.15.142:3300/0 conn(0x7f9a0815cf70 msgr2=0x7f9a080b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:49.286 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.284+0000 7f9a0e8b3640 1 --2- 172.21.15.142:0/1813857203 >> v2:172.21.15.142:3300/0 conn(0x7f9a0815cf70 0x7f9a080b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.286 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.284+0000 7f9a0e8b3640 1 -- 172.21.15.142:0/1813857203 >> v1:172.21.15.115:6789/0 conn(0x7f9a080b18f0 legacy=0x7f9a080cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.286 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.284+0000 7f9a0e8b3640 1 -- 172.21.15.142:0/1813857203 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a080b6e30 con 0x7f9a080d0260 2023-12-01T01:35:49.287 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.285+0000 7f9a0e8b3640 1 --2- 172.21.15.142:0/1813857203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080d0260 0x7f9a080b7bf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009d90 tx=0x7f9a0002bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb76790db06e9ea server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:49.287 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.285+0000 7f9a0d0b0640 1 -- 172.21.15.142:0/1813857203 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0002cce0 con 0x7f9a080d0260 2023-12-01T01:35:49.287 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.285+0000 7f9a0d0b0640 1 -- 172.21.15.142:0/1813857203 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0002ce40 con 0x7f9a080d0260 2023-12-01T01:35:49.287 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.285+0000 7f9a0d0b0640 1 -- 172.21.15.142:0/1813857203 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00034660 con 0x7f9a080d0260 2023-12-01T01:35:49.288 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.286+0000 7f9a10316640 1 -- 172.21.15.142:0/1813857203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080d0260 msgr2=0x7f9a080b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.288 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.286+0000 7f9a10316640 1 --2- 172.21.15.142:0/1813857203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080d0260 0x7f9a080b7bf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009d90 tx=0x7f9a0002bf30 comp rx=0 tx=0).stop 2023-12-01T01:35:49.288 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.286+0000 7f9a10316640 1 -- 172.21.15.142:0/1813857203 shutdown_connections 2023-12-01T01:35:49.288 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.286+0000 7f9a10316640 1 --2- 172.21.15.142:0/1813857203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080d0260 0x7f9a080b7bf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.288 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.286+0000 7f9a10316640 1 --2- 172.21.15.142:0/1813857203 >> v2:172.21.15.142:3300/0 conn(0x7f9a0815cf70 0x7f9a080b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.288 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.286+0000 7f9a10316640 1 -- 172.21.15.142:0/1813857203 >> 172.21.15.142:0/1813857203 conn(0x7f9a0806a0e0 msgr2=0x7f9a0806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:49.288 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.286+0000 7f9a10316640 1 -- 172.21.15.142:0/1813857203 shutdown_connections 2023-12-01T01:35:49.288 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.286+0000 7f9a10316640 1 -- 172.21.15.142:0/1813857203 wait complete. 2023-12-01T01:35:49.290 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.289+0000 7f9a10316640 1 Processor -- start 2023-12-01T01:35:49.290 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.289+0000 7f9a10316640 1 -- start start 2023-12-01T01:35:49.293 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a10316640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080b18f0 0x7f9a081584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:49.293 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a10316640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a080d0260 0x7f9a080cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:49.293 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a10316640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a0815cf70 0x7f9a080cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:49.293 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a10316640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0815fa00 con 0x7f9a080d0260 2023-12-01T01:35:49.293 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a10316640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080aad40 con 0x7f9a0815cf70 2023-12-01T01:35:49.293 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a10316640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08158a20 con 0x7f9a080b18f0 2023-12-01T01:35:49.293 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0e0b2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080b18f0 0x7f9a081584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0e8b3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a0815cf70 0x7f9a080cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0e8b3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a0815cf70 0x7f9a080cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:33016/0 (socket says 172.21.15.142:33016) 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0e8b3640 1 -- 172.21.15.142:0/3089451661 learned_addr learned my addr 172.21.15.142:0/3089451661 (peer_addr_for_me v2:172.21.15.142:0/0) 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0d8b1640 1 --2- 172.21.15.142:0/3089451661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a080d0260 0x7f9a080cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0e8b3640 1 -- 172.21.15.142:0/3089451661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080b18f0 msgr2=0x7f9a081584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0e8b3640 1 --2- 172.21.15.142:0/3089451661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080b18f0 0x7f9a081584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0e8b3640 1 -- 172.21.15.142:0/3089451661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a080d0260 msgr2=0x7f9a080cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0e8b3640 1 --2- 172.21.15.142:0/3089451661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a080d0260 0x7f9a080cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0e8b3640 1 -- 172.21.15.142:0/3089451661 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00009a70 con 0x7f9a0815cf70 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0e0b2640 1 --2- 172.21.15.142:0/3089451661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080b18f0 0x7f9a081584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.291+0000 7f9a0d8b1640 1 --2- 172.21.15.142:0/3089451661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a080d0260 0x7f9a080cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:35:49.294 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.292+0000 7f9a0e8b3640 1 --2- 172.21.15.142:0/3089451661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a0815cf70 0x7f9a080cdbd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009d90 tx=0x7f9a00004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:49.295 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.292+0000 7f99feffd640 1 -- 172.21.15.142:0/3089451661 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0003d600 con 0x7f9a0815cf70 2023-12-01T01:35:49.295 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.292+0000 7f99feffd640 1 -- 172.21.15.142:0/3089451661 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0003d780 con 0x7f9a0815cf70 2023-12-01T01:35:49.295 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.292+0000 7f9a10316640 1 -- 172.21.15.142:0/3089451661 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0815e2d0 con 0x7f9a0815cf70 2023-12-01T01:35:49.296 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.292+0000 7f99feffd640 1 -- 172.21.15.142:0/3089451661 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0003db30 con 0x7f9a0815cf70 2023-12-01T01:35:49.296 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.292+0000 7f9a10316640 1 -- 172.21.15.142:0/3089451661 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0815e890 con 0x7f9a0815cf70 2023-12-01T01:35:49.296 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.293+0000 7f99feffd640 1 -- 172.21.15.142:0/3089451661 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9a0003b040 con 0x7f9a0815cf70 2023-12-01T01:35:49.296 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.293+0000 7f99feffd640 1 -- 172.21.15.142:0/3089451661 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f9a00049c10 con 0x7f9a0815cf70 2023-12-01T01:35:49.296 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.293+0000 7f9a10316640 1 -- 172.21.15.142:0/3089451661 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a080b70c0 con 0x7f9a0815cf70 2023-12-01T01:35:49.297 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.296+0000 7f99feffd640 1 -- 172.21.15.142:0/3089451661 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f9a0004e060 con 0x7f9a0815cf70 2023-12-01T01:35:49.376 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.374+0000 7f9a10316640 1 -- 172.21.15.142:0/3089451661 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f333b559-012d-4988-a29f-21609f533af8", "id": 7} v 0) v1 -- 0x7f9a080aaf00 con 0x7f9a0815cf70 2023-12-01T01:35:49.384 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.382+0000 7f99feffd640 1 -- 172.21.15.142:0/3089451661 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f333b559-012d-4988-a29f-21609f533af8", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9a00044070 con 0x7f9a0815cf70 2023-12-01T01:35:49.384 INFO:teuthology.orchestra.run.smithi142.stdout:7 2023-12-01T01:35:49.385 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.383+0000 7f9a10316640 1 -- 172.21.15.142:0/3089451661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a0815cf70 msgr2=0x7f9a080cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.385 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.383+0000 7f9a10316640 1 --2- 172.21.15.142:0/3089451661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a0815cf70 0x7f9a080cdbd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009d90 tx=0x7f9a00004a40 comp rx=0 tx=0).stop 2023-12-01T01:35:49.385 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.383+0000 7f9a10316640 1 -- 172.21.15.142:0/3089451661 shutdown_connections 2023-12-01T01:35:49.385 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.383+0000 7f9a10316640 1 --2- 172.21.15.142:0/3089451661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a0815cf70 0x7f9a080cdbd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.385 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.383+0000 7f9a10316640 1 --2- 172.21.15.142:0/3089451661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a080d0260 0x7f9a080cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.385 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.383+0000 7f9a10316640 1 --2- 172.21.15.142:0/3089451661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a080b18f0 0x7f9a081584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.385 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.383+0000 7f9a10316640 1 -- 172.21.15.142:0/3089451661 >> 172.21.15.142:0/3089451661 conn(0x7f9a0806a0e0 msgr2=0x7f9a0806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:49.386 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.384+0000 7f9a10316640 1 -- 172.21.15.142:0/3089451661 shutdown_connections 2023-12-01T01:35:49.386 INFO:teuthology.orchestra.run.smithi142.stderr:2023-12-01T01:35:49.384+0000 7f9a10316640 1 -- 172.21.15.142:0/3089451661 wait complete. 2023-12-01T01:35:49.402 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-01T01:35:49.403 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-12-01T01:35:49.411 INFO:tasks.ceph.osd.0:Started 2023-12-01T01:35:49.411 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-01T01:35:49.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-12-01T01:35:49.419 INFO:tasks.ceph.osd.1:Started 2023-12-01T01:35:49.419 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-01T01:35:49.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-12-01T01:35:49.427 INFO:tasks.ceph.osd.2:Started 2023-12-01T01:35:49.427 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-01T01:35:49.427 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-12-01T01:35:49.431 INFO:tasks.ceph.osd.3:Started 2023-12-01T01:35:49.431 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-01T01:35:49.432 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-12-01T01:35:49.451 INFO:tasks.ceph.osd.4:Started 2023-12-01T01:35:49.451 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-01T01:35:49.451 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-12-01T01:35:49.457 INFO:tasks.ceph.osd.5:Started 2023-12-01T01:35:49.458 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-01T01:35:49.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-12-01T01:35:49.463 INFO:tasks.ceph.osd.6:Started 2023-12-01T01:35:49.463 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-01T01:35:49.463 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-12-01T01:35:49.469 INFO:tasks.ceph.osd.7:Started 2023-12-01T01:35:49.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:35:49.650 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:49.648+0000 7f1908018040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:35:49.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.682+0000 7f1fa9277640 1 Processor -- start 2023-12-01T01:35:49.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.682+0000 7f1fa9277640 1 -- start start 2023-12-01T01:35:49.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.683+0000 7f1fa9277640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1fa415cd20 0x7f1fa40d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:49.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.683+0000 7f1fa9277640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1fa40b5d80 0x7f1fa40b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.683+0000 7f1fa9277640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa40ce2e0 con 0x7f1fa40d1230 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.683+0000 7f1fa9277640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa40b2160 con 0x7f1fa415cd20 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.683+0000 7f1fa9277640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa40d2a10 con 0x7f1fa40b5d80 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.683+0000 7f1fa2ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1fa415cd20 0x7f1fa40d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.684+0000 7f1fa2ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1fa415cd20 0x7f1fa40d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38022/0 (socket says 172.21.15.115:38022) 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.684+0000 7f1fa2ffd640 1 -- 172.21.15.115:0/1436207068 learned_addr learned my addr 172.21.15.115:0/1436207068 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.684+0000 7f1fa27fc640 1 --2- 172.21.15.115:0/1436207068 >> v2:172.21.15.142:3300/0 conn(0x7f1fa40b5d80 0x7f1fa40b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.684+0000 7f1fa1ffb640 1 -- 172.21.15.115:0/1436207068 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450837781 0 0) 0x7f1fa40ce2e0 con 0x7f1fa40d1230 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.684+0000 7f1fa1ffb640 1 -- 172.21.15.115:0/1436207068 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f8c003520 con 0x7f1fa40d1230 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.684+0000 7f1fa27fc640 1 -- 172.21.15.115:0/1436207068 >> v2:172.21.15.115:3300/0 conn(0x7f1fa415cd20 msgr2=0x7f1fa40d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.684+0000 7f1fa27fc640 1 --2- 172.21.15.115:0/1436207068 >> v2:172.21.15.115:3300/0 conn(0x7f1fa415cd20 0x7f1fa40d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.684+0000 7f1fa27fc640 1 -- 172.21.15.115:0/1436207068 >> v1:172.21.15.115:6789/0 conn(0x7f1fa40d1230 legacy=0x7f1fa40d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.684+0000 7f1fa27fc640 1 -- 172.21.15.115:0/1436207068 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa40ce000 con 0x7f1fa40b5d80 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.685+0000 7f1fa27fc640 1 --2- 172.21.15.115:0/1436207068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40b5d80 0x7f1fa40b8570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40cf630 tx=0x7f1f9802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e09f3bb669a1939 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.685+0000 7f1fa1ffb640 1 -- 172.21.15.115:0/1436207068 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9802e070 con 0x7f1fa40b5d80 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.685+0000 7f1fa1ffb640 1 -- 172.21.15.115:0/1436207068 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f980315a0 con 0x7f1fa40b5d80 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.686+0000 7f1fa1ffb640 1 -- 172.21.15.115:0/1436207068 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98031910 con 0x7f1fa40b5d80 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.686+0000 7f1fa9277640 1 -- 172.21.15.115:0/1436207068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40b5d80 msgr2=0x7f1fa40b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.686+0000 7f1fa9277640 1 --2- 172.21.15.115:0/1436207068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40b5d80 0x7f1fa40b8570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40cf630 tx=0x7f1f9802b840 comp rx=0 tx=0).stop 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.686+0000 7f1fa9277640 1 -- 172.21.15.115:0/1436207068 shutdown_connections 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.686+0000 7f1fa9277640 1 --2- 172.21.15.115:0/1436207068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40b5d80 0x7f1fa40b8570 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.686+0000 7f1fa9277640 1 --2- 172.21.15.115:0/1436207068 >> v2:172.21.15.115:3300/0 conn(0x7f1fa415cd20 0x7f1fa40d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.686+0000 7f1fa9277640 1 -- 172.21.15.115:0/1436207068 >> 172.21.15.115:0/1436207068 conn(0x7f1fa406a0e0 msgr2=0x7f1fa406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.686+0000 7f1fa9277640 1 -- 172.21.15.115:0/1436207068 shutdown_connections 2023-12-01T01:35:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.686+0000 7f1fa9277640 1 -- 172.21.15.115:0/1436207068 wait complete. 2023-12-01T01:35:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.688+0000 7f1fa9277640 1 Processor -- start 2023-12-01T01:35:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.688+0000 7f1fa9277640 1 -- start start 2023-12-01T01:35:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa9277640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa40b5d80 0x7f1fa40cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa9277640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40d1230 0x7f1fa41f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa9277640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fa415cd20 0x7f1fa40d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa9277640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41f6690 con 0x7f1fa40b5d80 2023-12-01T01:35:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa9277640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa40ccb50 con 0x7f1fa40d1230 2023-12-01T01:35:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa9277640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa415ef40 con 0x7f1fa415cd20 2023-12-01T01:35:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa40b5d80 0x7f1fa40cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa40b5d80 0x7f1fa40cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38026/0 (socket says 172.21.15.115:38026) 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa2ffd640 1 -- 172.21.15.115:0/1763711275 learned_addr learned my addr 172.21.15.115:0/1763711275 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa27fc640 1 --2- 172.21.15.115:0/1763711275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40d1230 0x7f1fa41f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.689+0000 7f1fa37fe640 1 --2- 172.21.15.115:0/1763711275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fa415cd20 0x7f1fa40d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1fa2ffd640 1 -- 172.21.15.115:0/1763711275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fa415cd20 msgr2=0x7f1fa40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1fa2ffd640 1 --2- 172.21.15.115:0/1763711275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fa415cd20 0x7f1fa40d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1fa2ffd640 1 -- 172.21.15.115:0/1763711275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40d1230 msgr2=0x7f1fa41f6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1fa2ffd640 1 --2- 172.21.15.115:0/1763711275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40d1230 0x7f1fa41f6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1fa2ffd640 1 -- 172.21.15.115:0/1763711275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98009400 con 0x7f1fa40b5d80 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1fa2ffd640 1 --2- 172.21.15.115:0/1763711275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa40b5d80 0x7f1fa40cc570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400d7d0 tx=0x7f1f9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1f8b7fe640 1 -- 172.21.15.115:0/1763711275 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f940096a0 con 0x7f1fa40b5d80 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1f8b7fe640 1 -- 172.21.15.115:0/1763711275 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f94009800 con 0x7f1fa40b5d80 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1f8b7fe640 1 -- 172.21.15.115:0/1763711275 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94009bb0 con 0x7f1fa40b5d80 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1fa9277640 1 -- 172.21.15.115:0/1763711275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa415f220 con 0x7f1fa40b5d80 2023-12-01T01:35:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1fa9277640 1 -- 172.21.15.115:0/1763711275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa415f7e0 con 0x7f1fa40b5d80 2023-12-01T01:35:49.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.690+0000 7f1f8b7fe640 1 -- 172.21.15.115:0/1763711275 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1f94014710 con 0x7f1fa40b5d80 2023-12-01T01:35:49.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.691+0000 7f1f8b7fe640 1 -- 172.21.15.115:0/1763711275 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f1f94009d10 con 0x7f1fa40b5d80 2023-12-01T01:35:49.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.691+0000 7f1fa9277640 1 -- 172.21.15.115:0/1763711275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa404e890 con 0x7f1fa40b5d80 2023-12-01T01:35:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.694+0000 7f1f8b7fe640 1 -- 172.21.15.115:0/1763711275 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f1f9403cb10 con 0x7f1fa40b5d80 2023-12-01T01:35:49.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.760+0000 7f1fa9277640 1 -- 172.21.15.115:0/1763711275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fa415c0a0 con 0x7f1fa40b5d80 2023-12-01T01:35:49.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.761+0000 7f1f8b7fe640 1 -- 172.21.15.115:0/1763711275 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1f9402b070 con 0x7f1fa40b5d80 2023-12-01T01:35:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:35:49.763 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":10,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:35:49.376922+0000","last_up_change":"0.000000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:35:49.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.762+0000 7f1fa9277640 1 -- 172.21.15.115:0/1763711275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa40b5d80 msgr2=0x7f1fa40cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:49.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.762+0000 7f1fa9277640 1 --2- 172.21.15.115:0/1763711275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa40b5d80 0x7f1fa40cc570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400d7d0 tx=0x7f1f9400dc90 comp rx=0 tx=0).stop 2023-12-01T01:35:49.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.762+0000 7f1fa9277640 1 -- 172.21.15.115:0/1763711275 shutdown_connections 2023-12-01T01:35:49.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.762+0000 7f1fa9277640 1 --2- 172.21.15.115:0/1763711275 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fa415cd20 0x7f1fa40d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.762+0000 7f1fa9277640 1 --2- 172.21.15.115:0/1763711275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fa40d1230 0x7f1fa41f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.762+0000 7f1fa9277640 1 --2- 172.21.15.115:0/1763711275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fa40b5d80 0x7f1fa40cc570 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:49.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.762+0000 7f1fa9277640 1 -- 172.21.15.115:0/1763711275 >> 172.21.15.115:0/1763711275 conn(0x7f1fa406a0e0 msgr2=0x7f1fa4158650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:49.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.762+0000 7f1fa9277640 1 -- 172.21.15.115:0/1763711275 shutdown_connections 2023-12-01T01:35:49.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:35:49.762+0000 7f1fa9277640 1 -- 172.21.15.115:0/1763711275 wait complete. 2023-12-01T01:35:49.779 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-01T01:35:49.780 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-01T01:35:49.964 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:49.963+0000 7f1908018040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:35:50.023 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:50.021+0000 7f1908018040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:35:50.130 INFO:tasks.ceph.osd.2.smithi115.stderr:2023-12-01T01:35:50.128+0000 7f75a5794500 -1 Falling back to public interface 2023-12-01T01:35:50.135 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:50.134+0000 7f1908018040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:35:50.137 INFO:tasks.ceph.osd.1.smithi115.stderr:2023-12-01T01:35:50.136+0000 7f2ff820d500 -1 Falling back to public interface 2023-12-01T01:35:50.141 INFO:tasks.ceph.osd.0.smithi115.stderr:2023-12-01T01:35:50.140+0000 7f3ccacd6500 -1 Falling back to public interface 2023-12-01T01:35:50.146 INFO:tasks.ceph.osd.4.smithi142.stderr:2023-12-01T01:35:50.144+0000 7fdf14339500 -1 Falling back to public interface 2023-12-01T01:35:50.152 INFO:tasks.ceph.osd.3.smithi115.stderr:2023-12-01T01:35:50.151+0000 7f72163e2500 -1 Falling back to public interface 2023-12-01T01:35:50.153 INFO:tasks.ceph.osd.6.smithi142.stderr:2023-12-01T01:35:50.152+0000 7fb0967ad500 -1 Falling back to public interface 2023-12-01T01:35:50.183 INFO:tasks.ceph.osd.7.smithi142.stderr:2023-12-01T01:35:50.181+0000 7fc367a0e500 -1 Falling back to public interface 2023-12-01T01:35:50.183 INFO:tasks.ceph.osd.5.smithi142.stderr:2023-12-01T01:35:50.181+0000 7fad26417500 -1 Falling back to public interface 2023-12-01T01:35:50.472 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:50.470+0000 7f7afba45040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:35:50.584 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-01T01:35:50.751 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.749+0000 7fb297977640 1 Processor -- start 2023-12-01T01:35:50.751 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.749+0000 7fb297977640 1 -- start start 2023-12-01T01:35:50.752 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.750+0000 7fb297977640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb2900b5d80 0x7fb2900d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.750+0000 7fb297977640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb2900d1230 0x7fb2900d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.750+0000 7fb297977640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2900d1b90 con 0x7fb29015cd20 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.750+0000 7fb297977640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2900b2160 con 0x7fb2900b5d80 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.750+0000 7fb297977640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2900b8c80 con 0x7fb2900d1230 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.751+0000 7fb295713640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb29015cd20 0x7fb2900b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52722/0 (socket says 172.21.15.115:52722) 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.751+0000 7fb295713640 1 -- 172.21.15.115:0/3859534659 learned_addr learned my addr 172.21.15.115:0/3859534659 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.751+0000 7fb294f12640 1 --2- 172.21.15.115:0/3859534659 >> v2:172.21.15.115:3300/0 conn(0x7fb2900b5d80 0x7fb2900d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.751+0000 7fb295f14640 1 --2- 172.21.15.115:0/3859534659 >> v2:172.21.15.142:3300/0 conn(0x7fb2900d1230 0x7fb2900d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.751+0000 7fb287fff640 1 -- 172.21.15.115:0/3859534659 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090790080 0 0) 0x7fb28c001110 con 0x7fb29015cd20 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.751+0000 7fb287fff640 1 -- 172.21.15.115:0/3859534659 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb27c003520 con 0x7fb29015cd20 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.751+0000 7fb294f12640 1 -- 172.21.15.115:0/3859534659 >> v2:172.21.15.142:3300/0 conn(0x7fb2900d1230 msgr2=0x7fb2900d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.751+0000 7fb294f12640 1 --2- 172.21.15.115:0/3859534659 >> v2:172.21.15.142:3300/0 conn(0x7fb2900d1230 0x7fb2900d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:50.753 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.751+0000 7fb294f12640 1 -- 172.21.15.115:0/3859534659 >> v1:172.21.15.115:6789/0 conn(0x7fb29015cd20 legacy=0x7fb2900b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:50.754 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.751+0000 7fb294f12640 1 -- 172.21.15.115:0/3859534659 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2900ce2e0 con 0x7fb2900b5d80 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.752+0000 7fb294f12640 1 --2- 172.21.15.115:0/3859534659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 0x7fb2900d24d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb2900c6720 tx=0x7fb28002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e618b1b22615319c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.752+0000 7fb287fff640 1 -- 172.21.15.115:0/3859534659 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28002e070 con 0x7fb2900b5d80 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.753+0000 7fb287fff640 1 -- 172.21.15.115:0/3859534659 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2800315a0 con 0x7fb2900b5d80 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.753+0000 7fb287fff640 1 -- 172.21.15.115:0/3859534659 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280031930 con 0x7fb2900b5d80 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.753+0000 7fb297977640 1 -- 172.21.15.115:0/3859534659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 msgr2=0x7fb2900d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.753+0000 7fb297977640 1 --2- 172.21.15.115:0/3859534659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 0x7fb2900d24d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb2900c6720 tx=0x7fb28002b840 comp rx=0 tx=0).stop 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.753+0000 7fb297977640 1 -- 172.21.15.115:0/3859534659 shutdown_connections 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.753+0000 7fb297977640 1 --2- 172.21.15.115:0/3859534659 >> v2:172.21.15.142:3300/0 conn(0x7fb2900d1230 0x7fb2900d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.753+0000 7fb297977640 1 --2- 172.21.15.115:0/3859534659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 0x7fb2900d24d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.753+0000 7fb297977640 1 -- 172.21.15.115:0/3859534659 >> 172.21.15.115:0/3859534659 conn(0x7fb29006a0e0 msgr2=0x7fb29006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.753+0000 7fb297977640 1 -- 172.21.15.115:0/3859534659 shutdown_connections 2023-12-01T01:35:50.755 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.753+0000 7fb297977640 1 -- 172.21.15.115:0/3859534659 wait complete. 2023-12-01T01:35:50.756 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.755+0000 7fb297977640 1 Processor -- start 2023-12-01T01:35:50.756 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.755+0000 7fb297977640 1 -- start start 2023-12-01T01:35:50.758 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.756+0000 7fb297977640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 0x7fb2900b7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.756+0000 7fb297977640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2900d1230 0x7fb290156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.756+0000 7fb297977640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb29015cd20 0x7fb290156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.756+0000 7fb297977640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901f2f80 con 0x7fb2900b5d80 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.756+0000 7fb297977640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901f5560 con 0x7fb2900d1230 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.756+0000 7fb297977640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901f57b0 con 0x7fb29015cd20 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.756+0000 7fb295713640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 0x7fb2900b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.756+0000 7fb295713640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 0x7fb2900b7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38132/0 (socket says 172.21.15.115:38132) 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.756+0000 7fb295713640 1 -- 172.21.15.115:0/4205026802 learned_addr learned my addr 172.21.15.115:0/4205026802 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.756+0000 7fb295f14640 1 --2- 172.21.15.115:0/4205026802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb29015cd20 0x7fb290156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb295713640 1 -- 172.21.15.115:0/4205026802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb29015cd20 msgr2=0x7fb290156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb295713640 1 --2- 172.21.15.115:0/4205026802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb29015cd20 0x7fb290156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb295713640 1 -- 172.21.15.115:0/4205026802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2900d1230 msgr2=0x7fb290156500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb295713640 1 --2- 172.21.15.115:0/4205026802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2900d1230 0x7fb290156500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb295713640 1 -- 172.21.15.115:0/4205026802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb280009400 con 0x7fb2900b5d80 2023-12-01T01:35:50.759 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb295713640 1 --2- 172.21.15.115:0/4205026802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 0x7fb2900b7550 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00c420 tx=0x7fb28c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:50.760 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb285ffb640 1 -- 172.21.15.115:0/4205026802 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c005ce0 con 0x7fb2900b5d80 2023-12-01T01:35:50.760 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb285ffb640 1 -- 172.21.15.115:0/4205026802 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c005e40 con 0x7fb2900b5d80 2023-12-01T01:35:50.760 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb285ffb640 1 -- 172.21.15.115:0/4205026802 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0072f0 con 0x7fb2900b5d80 2023-12-01T01:35:50.760 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb297977640 1 -- 172.21.15.115:0/4205026802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2901f5a90 con 0x7fb2900b5d80 2023-12-01T01:35:50.760 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.757+0000 7fb297977640 1 -- 172.21.15.115:0/4205026802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2901f6050 con 0x7fb2900b5d80 2023-12-01T01:35:50.760 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.758+0000 7fb285ffb640 1 -- 172.21.15.115:0/4205026802 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 2) v1 ==== 44255+0+0 (secure 0 0 0) 0x7fb28c007510 con 0x7fb2900b5d80 2023-12-01T01:35:50.760 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.758+0000 7fb297977640 1 -- 172.21.15.115:0/4205026802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29004e890 con 0x7fb2900b5d80 2023-12-01T01:35:50.760 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.759+0000 7fb285ffb640 1 -- 172.21.15.115:0/4205026802 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb28c0483d0 con 0x7fb2900b5d80 2023-12-01T01:35:50.762 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.761+0000 7fb285ffb640 1 -- 172.21.15.115:0/4205026802 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fb28c007da0 con 0x7fb2900b5d80 2023-12-01T01:35:50.804 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:50.802+0000 7f7afba45040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:35:50.824 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.823+0000 7fb297977640 1 -- 172.21.15.115:0/4205026802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb290157630 con 0x7fb2900b5d80 2023-12-01T01:35:50.826 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.824+0000 7fb285ffb640 1 -- 172.21.15.115:0/4205026802 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb28c017070 con 0x7fb2900b5d80 2023-12-01T01:35:50.826 INFO:teuthology.misc.health.smithi115.stdout: 2023-12-01T01:35:50.826 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":10,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:35:49.376922+0000","last_up_change":"0.000000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:35:50.827 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.825+0000 7fb297977640 1 -- 172.21.15.115:0/4205026802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 msgr2=0x7fb2900b7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:50.827 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.825+0000 7fb297977640 1 --2- 172.21.15.115:0/4205026802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 0x7fb2900b7550 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00c420 tx=0x7fb28c00c8e0 comp rx=0 tx=0).stop 2023-12-01T01:35:50.827 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.825+0000 7fb297977640 1 -- 172.21.15.115:0/4205026802 shutdown_connections 2023-12-01T01:35:50.827 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.825+0000 7fb297977640 1 --2- 172.21.15.115:0/4205026802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb29015cd20 0x7fb290156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:50.827 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.825+0000 7fb297977640 1 --2- 172.21.15.115:0/4205026802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2900d1230 0x7fb290156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:50.827 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.825+0000 7fb297977640 1 --2- 172.21.15.115:0/4205026802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2900b5d80 0x7fb2900b7550 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:50.827 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.825+0000 7fb297977640 1 -- 172.21.15.115:0/4205026802 >> 172.21.15.115:0/4205026802 conn(0x7fb29006a0e0 msgr2=0x7fb290158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:50.827 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.825+0000 7fb297977640 1 -- 172.21.15.115:0/4205026802 shutdown_connections 2023-12-01T01:35:50.828 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:50.826+0000 7fb297977640 1 -- 172.21.15.115:0/4205026802 wait complete. 2023-12-01T01:35:50.840 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-01T01:35:50.868 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:50.866+0000 7f7afba45040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:35:50.983 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:35:50.981+0000 7f7afba45040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:35:51.174 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:51.172+0000 7f18a9115640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-01T01:35:51.174 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:35:51.173+0000 7f18a9115640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-01T01:35:53.700 INFO:tasks.ceph.osd.4.smithi142.stderr:2023-12-01T01:35:53.698+0000 7fdf14339500 -1 osd.4 0 log_to_monitors true 2023-12-01T01:35:53.700 INFO:tasks.ceph.osd.6.smithi142.stderr:2023-12-01T01:35:53.699+0000 7fb0967ad500 -1 osd.6 0 log_to_monitors true 2023-12-01T01:35:53.703 INFO:tasks.ceph.osd.1.smithi115.stderr:2023-12-01T01:35:53.702+0000 7f2ff820d500 -1 osd.1 0 log_to_monitors true 2023-12-01T01:35:53.704 INFO:tasks.ceph.osd.2.smithi115.stderr:2023-12-01T01:35:53.703+0000 7f75a5794500 -1 osd.2 0 log_to_monitors true 2023-12-01T01:35:53.729 INFO:tasks.ceph.osd.7.smithi142.stderr:2023-12-01T01:35:53.727+0000 7fc367a0e500 -1 osd.7 0 log_to_monitors true 2023-12-01T01:35:53.743 INFO:tasks.ceph.osd.3.smithi115.stderr:2023-12-01T01:35:53.742+0000 7f72163e2500 -1 osd.3 0 log_to_monitors true 2023-12-01T01:35:53.762 INFO:tasks.ceph.osd.5.smithi142.stderr:2023-12-01T01:35:53.761+0000 7fad26417500 -1 osd.5 0 log_to_monitors true 2023-12-01T01:35:53.854 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:35:53.861 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:35:53.969 INFO:tasks.ceph.osd.0.smithi115.stderr:2023-12-01T01:35:53.968+0000 7f3ccacd6500 -1 osd.0 0 log_to_monitors true 2023-12-01T01:35:57.636 INFO:tasks.ceph.osd.4.smithi142.stderr:2023-12-01T01:35:57.634+0000 7fdf0faa1640 -1 osd.4 0 waiting for initial osdmap 2023-12-01T01:35:57.645 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-01T01:35:57.652 INFO:tasks.ceph.osd.7.smithi142.stderr:2023-12-01T01:35:57.650+0000 7fc36431e640 -1 osd.7 0 waiting for initial osdmap 2023-12-01T01:35:57.653 INFO:tasks.ceph.osd.6.smithi142.stderr:2023-12-01T01:35:57.652+0000 7fb091f17640 -1 osd.6 0 waiting for initial osdmap 2023-12-01T01:35:57.659 INFO:tasks.ceph.osd.3.smithi115.stderr:2023-12-01T01:35:57.658+0000 7f7211b4c640 -1 osd.3 0 waiting for initial osdmap 2023-12-01T01:35:57.660 INFO:tasks.ceph.osd.4.smithi142.stderr:2023-12-01T01:35:57.659+0000 7fdf080d3640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:35:57.671 INFO:tasks.ceph.osd.7.smithi142.stderr:2023-12-01T01:35:57.670+0000 7fc35b115640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:35:57.673 INFO:tasks.ceph.osd.3.smithi115.stderr:2023-12-01T01:35:57.672+0000 7f720a17e640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:35:57.674 INFO:tasks.ceph.osd.6.smithi142.stderr:2023-12-01T01:35:57.673+0000 7fb08a549640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:35:57.681 INFO:tasks.ceph.osd.0.smithi115.stderr:2023-12-01T01:35:57.679+0000 7f3cc643e640 -1 osd.0 0 waiting for initial osdmap 2023-12-01T01:35:57.682 INFO:tasks.ceph.osd.1.smithi115.stderr:2023-12-01T01:35:57.680+0000 7f2ff33a5640 -1 osd.1 0 waiting for initial osdmap 2023-12-01T01:35:57.685 INFO:tasks.ceph.osd.2.smithi115.stderr:2023-12-01T01:35:57.683+0000 7f75a0efc640 -1 osd.2 0 waiting for initial osdmap 2023-12-01T01:35:57.694 INFO:tasks.ceph.osd.1.smithi115.stderr:2023-12-01T01:35:57.693+0000 7f2feb9d7640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:35:57.698 INFO:tasks.ceph.osd.0.smithi115.stderr:2023-12-01T01:35:57.697+0000 7f3cbea70640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:35:57.702 INFO:tasks.ceph.osd.5.smithi142.stderr:2023-12-01T01:35:57.700+0000 7fad215bf640 -1 osd.5 0 waiting for initial osdmap 2023-12-01T01:35:57.704 INFO:tasks.ceph.osd.2.smithi115.stderr:2023-12-01T01:35:57.703+0000 7f759952e640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:35:57.715 INFO:tasks.ceph.osd.5.smithi142.stderr:2023-12-01T01:35:57.713+0000 7fad19bf1640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:35:57.803 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.801+0000 7fb98342c640 1 Processor -- start 2023-12-01T01:35:57.803 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.801+0000 7fb98342c640 1 -- start start 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb98342c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb97c15e2f0 0x7fb97c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb98342c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb97c06f2f0 0x7fb97c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb98342c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c0add10 con 0x7fb97c0adf10 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb98342c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c0c5760 con 0x7fb97c15e2f0 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb98342c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c15e6d0 con 0x7fb97c06f2f0 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb9811c8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb97c15e2f0 0x7fb97c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb9811c8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb97c15e2f0 0x7fb97c0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55552/0 (socket says 172.21.15.115:55552) 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb9811c8640 1 -- 172.21.15.115:0/612096752 learned_addr learned my addr 172.21.15.115:0/612096752 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb9819c9640 1 --1- 172.21.15.115:0/612096752 >> v1:172.21.15.115:6789/0 conn(0x7fb97c0adf10 0x7fb97c0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36144/0 (socket says 172.21.15.115:36144) 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb9809c7640 1 --2- 172.21.15.115:0/612096752 >> v2:172.21.15.142:3300/0 conn(0x7fb97c06f2f0 0x7fb97c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:57.805 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb9811c8640 1 -- 172.21.15.115:0/612096752 >> v2:172.21.15.142:3300/0 conn(0x7fb97c06f2f0 msgr2=0x7fb97c0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:35:57.806 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb9811c8640 1 --2- 172.21.15.115:0/612096752 >> v2:172.21.15.142:3300/0 conn(0x7fb97c06f2f0 0x7fb97c0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:57.806 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb9811c8640 1 -- 172.21.15.115:0/612096752 >> v1:172.21.15.115:6789/0 conn(0x7fb97c0adf10 legacy=0x7fb97c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:57.806 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb9811c8640 1 -- 172.21.15.115:0/612096752 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c0cf7b0 con 0x7fb97c15e2f0 2023-12-01T01:35:57.806 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb9809c7640 1 --2- 172.21.15.115:0/612096752 >> v2:172.21.15.142:3300/0 conn(0x7fb97c06f2f0 0x7fb97c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:35:57.806 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.803+0000 7fb96ffff640 1 -- 172.21.15.115:0/612096752 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638244845 0 0) 0x7fb97c0add10 con 0x7fb97c0adf10 2023-12-01T01:35:57.807 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.804+0000 7fb9811c8640 1 --2- 172.21.15.115:0/612096752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c15e2f0 0x7fb97c0ca660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb97c06de80 tx=0x7fb97002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d25ad8e14b6315f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:57.807 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb96ffff640 1 -- 172.21.15.115:0/612096752 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9700317e0 con 0x7fb97c15e2f0 2023-12-01T01:35:57.807 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb96ffff640 1 -- 172.21.15.115:0/612096752 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb970031940 con 0x7fb97c15e2f0 2023-12-01T01:35:57.807 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb96ffff640 1 -- 172.21.15.115:0/612096752 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970031cd0 con 0x7fb97c15e2f0 2023-12-01T01:35:57.807 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb98342c640 1 -- 172.21.15.115:0/612096752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c15e2f0 msgr2=0x7fb97c0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:57.807 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb98342c640 1 --2- 172.21.15.115:0/612096752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c15e2f0 0x7fb97c0ca660 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb97c06de80 tx=0x7fb97002b880 comp rx=0 tx=0).stop 2023-12-01T01:35:57.807 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb98342c640 1 -- 172.21.15.115:0/612096752 shutdown_connections 2023-12-01T01:35:57.807 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb98342c640 1 --2- 172.21.15.115:0/612096752 >> v2:172.21.15.142:3300/0 conn(0x7fb97c06f2f0 0x7fb97c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:57.808 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb98342c640 1 --2- 172.21.15.115:0/612096752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c15e2f0 0x7fb97c0ca660 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:57.808 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb98342c640 1 -- 172.21.15.115:0/612096752 >> 172.21.15.115:0/612096752 conn(0x7fb97c0ce880 msgr2=0x7fb97c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:57.808 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb98342c640 1 -- 172.21.15.115:0/612096752 shutdown_connections 2023-12-01T01:35:57.808 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.805+0000 7fb98342c640 1 -- 172.21.15.115:0/612096752 wait complete. 2023-12-01T01:35:57.808 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.807+0000 7fb98342c640 1 Processor -- start 2023-12-01T01:35:57.808 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.807+0000 7fb98342c640 1 -- start start 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb98342c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c06f2f0 0x7fb97c0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb98342c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb97c0adf10 0x7fb97c0af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb98342c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb97c15e2f0 0x7fb97c0c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb98342c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c0c8020 con 0x7fb97c06f2f0 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb98342c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c0c7450 con 0x7fb97c15e2f0 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb98342c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c0c76f0 con 0x7fb97c0adf10 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb9811c8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c06f2f0 0x7fb97c0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb9811c8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c06f2f0 0x7fb97c0aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55562/0 (socket says 172.21.15.115:55562) 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb9811c8640 1 -- 172.21.15.115:0/4281438050 learned_addr learned my addr 172.21.15.115:0/4281438050 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb9811c8640 1 -- 172.21.15.115:0/4281438050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb97c0adf10 msgr2=0x7fb97c0af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb9811c8640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb97c0adf10 0x7fb97c0af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:57.810 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb9811c8640 1 -- 172.21.15.115:0/4281438050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb97c15e2f0 msgr2=0x7fb97c0c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:57.811 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb9811c8640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb97c15e2f0 0x7fb97c0c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:57.811 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.808+0000 7fb9811c8640 1 -- 172.21.15.115:0/4281438050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9700094d0 con 0x7fb97c06f2f0 2023-12-01T01:35:57.811 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.809+0000 7fb9811c8640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c06f2f0 0x7fb97c0aeb60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb970034040 tx=0x7fb9700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:57.811 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.809+0000 7fb96dffb640 1 -- 172.21.15.115:0/4281438050 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970004230 con 0x7fb97c06f2f0 2023-12-01T01:35:57.811 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.809+0000 7fb96dffb640 1 -- 172.21.15.115:0/4281438050 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9700043b0 con 0x7fb97c06f2f0 2023-12-01T01:35:57.812 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.809+0000 7fb98342c640 1 -- 172.21.15.115:0/4281438050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c0c79d0 con 0x7fb97c06f2f0 2023-12-01T01:35:57.812 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.809+0000 7fb96dffb640 1 -- 172.21.15.115:0/4281438050 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970004520 con 0x7fb97c06f2f0 2023-12-01T01:35:57.812 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.809+0000 7fb98342c640 1 -- 172.21.15.115:0/4281438050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c0c4040 con 0x7fb97c06f2f0 2023-12-01T01:35:57.812 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.810+0000 7fb98342c640 1 -- 172.21.15.115:0/4281438050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c06fda0 con 0x7fb97c06f2f0 2023-12-01T01:35:57.812 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.811+0000 7fb96dffb640 1 -- 172.21.15.115:0/4281438050 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 4) v1 ==== 88551+0+0 (secure 0 0 0) 0x7fb970043070 con 0x7fb97c06f2f0 2023-12-01T01:35:57.814 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.811+0000 7fb96dffb640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb96006a3a0 0x7fb96006c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:35:57.814 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.812+0000 7fb96dffb640 1 -- 172.21.15.115:0/4281438050 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fb9700a9c00 con 0x7fb97c06f2f0 2023-12-01T01:35:57.814 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.812+0000 7fb9809c7640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb96006a3a0 0x7fb96006c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:35:57.814 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.812+0000 7fb9809c7640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb96006a3a0 0x7fb96006c860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb978006fd0 tx=0x7fb978008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:35:57.818 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.817+0000 7fb96dffb640 1 -- 172.21.15.115:0/4281438050 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb970076170 con 0x7fb97c06f2f0 2023-12-01T01:35:57.998 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.997+0000 7fb98342c640 1 -- 172.21.15.115:0/4281438050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb97c04e890 con 0x7fb97c06f2f0 2023-12-01T01:35:57.999 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:57.998+0000 7fb96dffb640 1 -- 172.21.15.115:0/4281438050 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb970079790 con 0x7fb97c06f2f0 2023-12-01T01:35:57.999 INFO:teuthology.misc.health.smithi115.stdout: 2023-12-01T01:35:57.999 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":12,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:35:55.296255+0000","last_up_change":"0.000000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:35:58.002 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 -- 172.21.15.115:0/4281438050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb96006a3a0 msgr2=0x7fb96006c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:58.002 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb96006a3a0 0x7fb96006c860 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb978006fd0 tx=0x7fb978008040 comp rx=0 tx=0).stop 2023-12-01T01:35:58.002 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 -- 172.21.15.115:0/4281438050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c06f2f0 msgr2=0x7fb97c0aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:35:58.002 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c06f2f0 0x7fb97c0aeb60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb970034040 tx=0x7fb9700046c0 comp rx=0 tx=0).stop 2023-12-01T01:35:58.002 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 -- 172.21.15.115:0/4281438050 shutdown_connections 2023-12-01T01:35:58.002 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb96006a3a0 0x7fb96006c860 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:58.002 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb97c0adf10 0x7fb97c0af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:58.002 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb97c15e2f0 0x7fb97c0c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:58.003 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 --2- 172.21.15.115:0/4281438050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb97c06f2f0 0x7fb97c0aeb60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:35:58.003 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 -- 172.21.15.115:0/4281438050 >> 172.21.15.115:0/4281438050 conn(0x7fb97c0ce880 msgr2=0x7fb97c06b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:35:58.003 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 -- 172.21.15.115:0/4281438050 shutdown_connections 2023-12-01T01:35:58.003 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:35:58.000+0000 7fb98342c640 1 -- 172.21.15.115:0/4281438050 wait complete. 2023-12-01T01:35:58.016 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-01T01:36:04.822 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:05.008 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.007+0000 7f684a284640 1 Processor -- start 2023-12-01T01:36:05.008 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.007+0000 7f684a284640 1 -- start start 2023-12-01T01:36:05.009 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.008+0000 7f684a284640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f68440b8610 0x7f68440cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.010 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.008+0000 7f684a284640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f68440cdf00 0x7f68440b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.010 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.008+0000 7f684a284640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68440d1740 con 0x7f684415cf90 2023-12-01T01:36:05.010 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.008+0000 7f684a284640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68440ce330 con 0x7f68440b8610 2023-12-01T01:36:05.010 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.008+0000 7f684a284640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684415d370 con 0x7f68440cdf00 2023-12-01T01:36:05.011 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.008+0000 7f683bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f68440b8610 0x7f68440cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.011 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.009+0000 7f6843fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f684415cf90 0x7f68440d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36154/0 (socket says 172.21.15.115:36154) 2023-12-01T01:36:05.011 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.009+0000 7f6843fff640 1 -- 172.21.15.115:0/1445670234 learned_addr learned my addr 172.21.15.115:0/1445670234 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:05.011 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.009+0000 7f6848821640 1 --2- 172.21.15.115:0/1445670234 >> v2:172.21.15.142:3300/0 conn(0x7f68440cdf00 0x7f68440b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.011 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.009+0000 7f683bfff640 1 -- 172.21.15.115:0/1445670234 >> v2:172.21.15.142:3300/0 conn(0x7f68440cdf00 msgr2=0x7f68440b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:05.011 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.009+0000 7f683bfff640 1 --2- 172.21.15.115:0/1445670234 >> v2:172.21.15.142:3300/0 conn(0x7f68440cdf00 0x7f68440b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.011 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.009+0000 7f683bfff640 1 -- 172.21.15.115:0/1445670234 >> v1:172.21.15.115:6789/0 conn(0x7f684415cf90 legacy=0x7f68440d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.011 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.009+0000 7f683bfff640 1 -- 172.21.15.115:0/1445670234 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68440d19a0 con 0x7f68440b8610 2023-12-01T01:36:05.012 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.010+0000 7f683bfff640 1 --2- 172.21.15.115:0/1445670234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68440b8610 0x7f68440cd9c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f68440d0af0 tx=0x7f682c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461a032193205ab0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:05.100 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.099+0000 7f68437fe640 1 -- 172.21.15.115:0/1445670234 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c031830 con 0x7f68440b8610 2023-12-01T01:36:05.101 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.099+0000 7f68437fe640 1 -- 172.21.15.115:0/1445670234 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c031990 con 0x7f68440b8610 2023-12-01T01:36:05.101 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.099+0000 7f68437fe640 1 -- 172.21.15.115:0/1445670234 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c031d20 con 0x7f68440b8610 2023-12-01T01:36:05.102 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.100+0000 7f684a284640 1 -- 172.21.15.115:0/1445670234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68440b8610 msgr2=0x7f68440cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.102 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.100+0000 7f684a284640 1 --2- 172.21.15.115:0/1445670234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68440b8610 0x7f68440cd9c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f68440d0af0 tx=0x7f682c02b8d0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.102 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.101+0000 7f684a284640 1 -- 172.21.15.115:0/1445670234 shutdown_connections 2023-12-01T01:36:05.102 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.101+0000 7f684a284640 1 --2- 172.21.15.115:0/1445670234 >> v2:172.21.15.142:3300/0 conn(0x7f68440cdf00 0x7f68440b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.102 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.101+0000 7f684a284640 1 --2- 172.21.15.115:0/1445670234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68440b8610 0x7f68440cd9c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.102 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.101+0000 7f684a284640 1 -- 172.21.15.115:0/1445670234 >> 172.21.15.115:0/1445670234 conn(0x7f6844065960 msgr2=0x7f6844065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:05.103 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.101+0000 7f684a284640 1 -- 172.21.15.115:0/1445670234 shutdown_connections 2023-12-01T01:36:05.103 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.101+0000 7f684a284640 1 -- 172.21.15.115:0/1445670234 wait complete. 2023-12-01T01:36:05.105 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.103+0000 7f684a284640 1 Processor -- start 2023-12-01T01:36:05.105 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.104+0000 7f684a284640 1 -- start start 2023-12-01T01:36:05.107 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.105+0000 7f684a284640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68440b8610 0x7f684415feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.107 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.105+0000 7f684a284640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68440cdf00 0x7f6844151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.107 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.105+0000 7f684a284640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f684415cf90 0x7f6844143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.107 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.105+0000 7f684a284640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6844154210 con 0x7f684415cf90 2023-12-01T01:36:05.108 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.105+0000 7f684a284640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6844154d90 con 0x7f68440cdf00 2023-12-01T01:36:05.108 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.105+0000 7f684a284640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6844154460 con 0x7f68440b8610 2023-12-01T01:36:05.108 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.105+0000 7f6848821640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f684415cf90 0x7f6844143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.108 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.105+0000 7f6848821640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f684415cf90 0x7f6844143b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55618/0 (socket says 172.21.15.115:55618) 2023-12-01T01:36:05.108 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.105+0000 7f6848821640 1 -- 172.21.15.115:0/3613190886 learned_addr learned my addr 172.21.15.115:0/3613190886 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:05.108 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.106+0000 7f6843fff640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68440b8610 0x7f684415feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.108 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.106+0000 7f683bfff640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68440cdf00 0x7f6844151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.109 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.106+0000 7f6848821640 1 -- 172.21.15.115:0/3613190886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68440b8610 msgr2=0x7f684415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.109 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.106+0000 7f6848821640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68440b8610 0x7f684415feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.109 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.106+0000 7f6848821640 1 -- 172.21.15.115:0/3613190886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68440cdf00 msgr2=0x7f6844151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.109 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.106+0000 7f6848821640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68440cdf00 0x7f6844151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.109 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.106+0000 7f6848821640 1 -- 172.21.15.115:0/3613190886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c009400 con 0x7f684415cf90 2023-12-01T01:36:05.109 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.106+0000 7f683bfff640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68440cdf00 0x7f6844151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:05.110 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.106+0000 7f6848821640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f684415cf90 0x7f6844143b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f683c00efc0 tx=0x7f683c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:05.110 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.107+0000 7f68417fa640 1 -- 172.21.15.115:0/3613190886 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c014480 con 0x7f684415cf90 2023-12-01T01:36:05.110 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.107+0000 7f684a284640 1 -- 172.21.15.115:0/3613190886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6844154740 con 0x7f684415cf90 2023-12-01T01:36:05.110 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.107+0000 7f68417fa640 1 -- 172.21.15.115:0/3613190886 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c00f040 con 0x7f684415cf90 2023-12-01T01:36:05.110 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.107+0000 7f68417fa640 1 -- 172.21.15.115:0/3613190886 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c014820 con 0x7f684415cf90 2023-12-01T01:36:05.110 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.107+0000 7f684a284640 1 -- 172.21.15.115:0/3613190886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68441603f0 con 0x7f684415cf90 2023-12-01T01:36:05.110 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.108+0000 7f684a284640 1 -- 172.21.15.115:0/3613190886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68440d2220 con 0x7f684415cf90 2023-12-01T01:36:05.111 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.110+0000 7f68417fa640 1 -- 172.21.15.115:0/3613190886 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 4) v1 ==== 88551+0+0 (secure 0 0 0) 0x7f683c014980 con 0x7f684415cf90 2023-12-01T01:36:05.112 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.111+0000 7f68417fa640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a3a0 0x7f682406c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.112 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.111+0000 7f68417fa640 1 -- 172.21.15.115:0/3613190886 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f683c089060 con 0x7f684415cf90 2023-12-01T01:36:05.113 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.111+0000 7f6843fff640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a3a0 0x7f682406c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.113 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.112+0000 7f6843fff640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a3a0 0x7f682406c860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6834000f60 tx=0x7f6834009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:05.118 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.117+0000 7f68417fa640 1 -- 172.21.15.115:0/3613190886 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f683c054e60 con 0x7f684415cf90 2023-12-01T01:36:05.127 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.125+0000 7f68417fa640 1 -- 172.21.15.115:0/3613190886 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f683c081d60 con 0x7f684415cf90 2023-12-01T01:36:05.318 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.317+0000 7f684a284640 1 -- 172.21.15.115:0/3613190886 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68440d21f0 con 0x7f684415cf90 2023-12-01T01:36:05.320 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.319+0000 7f68417fa640 1 -- 172.21.15.115:0/3613190886 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12430 (secure 0 0 0) 0x7f683c058290 con 0x7f684415cf90 2023-12-01T01:36:05.321 INFO:teuthology.misc.health.smithi115.stdout: 2023-12-01T01:36:05.321 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":17,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:02.902562+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:05.323 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.321+0000 7f684a284640 1 -- 172.21.15.115:0/3613190886 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a3a0 msgr2=0x7f682406c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.323 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.321+0000 7f684a284640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a3a0 0x7f682406c860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6834000f60 tx=0x7f6834009040 comp rx=0 tx=0).stop 2023-12-01T01:36:05.323 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.321+0000 7f684a284640 1 -- 172.21.15.115:0/3613190886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f684415cf90 msgr2=0x7f6844143b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.323 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.321+0000 7f684a284640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f684415cf90 0x7f6844143b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f683c00efc0 tx=0x7f683c00c490 comp rx=0 tx=0).stop 2023-12-01T01:36:05.324 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.322+0000 7f684a284640 1 -- 172.21.15.115:0/3613190886 shutdown_connections 2023-12-01T01:36:05.324 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.322+0000 7f684a284640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a3a0 0x7f682406c860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.324 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.322+0000 7f684a284640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f684415cf90 0x7f6844143b70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.324 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.322+0000 7f684a284640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68440cdf00 0x7f6844151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.324 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.322+0000 7f684a284640 1 --2- 172.21.15.115:0/3613190886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68440b8610 0x7f684415feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.324 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.322+0000 7f684a284640 1 -- 172.21.15.115:0/3613190886 >> 172.21.15.115:0/3613190886 conn(0x7f6844065960 msgr2=0x7f68440b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:05.324 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.322+0000 7f684a284640 1 -- 172.21.15.115:0/3613190886 shutdown_connections 2023-12-01T01:36:05.324 INFO:teuthology.misc.health.smithi115.stderr:2023-12-01T01:36:05.322+0000 7f684a284640 1 -- 172.21.15.115:0/3613190886 wait complete. 2023-12-01T01:36:05.338 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-01T01:36:05.338 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-01T01:36:05.339 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T01:36:05.339 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T01:36:05.347 INFO:tasks.ceph.mds.a:Started 2023-12-01T01:36:05.347 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T01:36:05.347 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T01:36:05.355 INFO:tasks.ceph.mds.c:Started 2023-12-01T01:36:05.355 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T01:36:05.355 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T01:36:05.360 INFO:tasks.ceph.mds.b:Started 2023-12-01T01:36:05.360 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T01:36:05.360 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T01:36:05.366 INFO:tasks.ceph.mds.d:Started 2023-12-01T01:36:05.366 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-12-01T01:36:05.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:05.505 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:36:05.506 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T01:36:05.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.553+0000 7f5e6898d640 1 Processor -- start 2023-12-01T01:36:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.553+0000 7f5e6898d640 1 -- start start 2023-12-01T01:36:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.554+0000 7f5e6898d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5e64160130 0x7f5e640d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.554+0000 7f5e6898d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e640d26e0 0x7f5e640b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.554+0000 7f5e6898d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e640b8b60 con 0x7f5e640cfb20 2023-12-01T01:36:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.554+0000 7f5e6898d640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e64160510 con 0x7f5e64160130 2023-12-01T01:36:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.554+0000 7f5e6898d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e640b6d20 con 0x7f5e640d26e0 2023-12-01T01:36:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.554+0000 7f5e62d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5e64160130 0x7f5e640d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.554+0000 7f5e62d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5e64160130 0x7f5e640d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55660/0 (socket says 172.21.15.115:55660) 2023-12-01T01:36:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.554+0000 7f5e62d9d640 1 -- 172.21.15.115:0/1515211550 learned_addr learned my addr 172.21.15.115:0/1515211550 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.555+0000 7f5e62d9d640 1 -- 172.21.15.115:0/1515211550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e640d26e0 msgr2=0x7f5e640b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.555+0000 7f5e62d9d640 1 --2- 172.21.15.115:0/1515211550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e640d26e0 0x7f5e640b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.555+0000 7f5e62d9d640 1 -- 172.21.15.115:0/1515211550 >> v1:172.21.15.142:6789/0 conn(0x7f5e640cfb20 legacy=0x7f5e640d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.555+0000 7f5e62d9d640 1 -- 172.21.15.115:0/1515211550 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e640d1640 con 0x7f5e64160130 2023-12-01T01:36:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.556+0000 7f5e62d9d640 1 --2- 172.21.15.115:0/1515211550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 0x7f5e640d0fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009850 tx=0x7f5e5402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a704fd86552f933e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.556+0000 7f5e61d9b640 1 -- 172.21.15.115:0/1515211550 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54004230 con 0x7f5e64160130 2023-12-01T01:36:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.556+0000 7f5e61d9b640 1 -- 172.21.15.115:0/1515211550 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54004390 con 0x7f5e64160130 2023-12-01T01:36:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.556+0000 7f5e61d9b640 1 -- 172.21.15.115:0/1515211550 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54031830 con 0x7f5e64160130 2023-12-01T01:36:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.557+0000 7f5e6898d640 1 -- 172.21.15.115:0/1515211550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 msgr2=0x7f5e640d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.557+0000 7f5e6898d640 1 --2- 172.21.15.115:0/1515211550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 0x7f5e640d0fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009850 tx=0x7f5e5402b8d0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.557+0000 7f5e6898d640 1 -- 172.21.15.115:0/1515211550 shutdown_connections 2023-12-01T01:36:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.557+0000 7f5e6898d640 1 --2- 172.21.15.115:0/1515211550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e640d26e0 0x7f5e640b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.557+0000 7f5e6898d640 1 --2- 172.21.15.115:0/1515211550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 0x7f5e640d0fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.557+0000 7f5e6898d640 1 -- 172.21.15.115:0/1515211550 >> 172.21.15.115:0/1515211550 conn(0x7f5e6406a0b0 msgr2=0x7f5e6406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.557+0000 7f5e6898d640 1 -- 172.21.15.115:0/1515211550 shutdown_connections 2023-12-01T01:36:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.557+0000 7f5e6898d640 1 -- 172.21.15.115:0/1515211550 wait complete. 2023-12-01T01:36:05.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.559+0000 7f5e6898d640 1 Processor -- start 2023-12-01T01:36:05.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.559+0000 7f5e6898d640 1 -- start start 2023-12-01T01:36:05.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6898d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e640cfb20 0x7f5e640ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6898d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e640d26e0 0x7f5e641f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6898d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 0x7f5e640c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6898d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641f2c20 con 0x7f5e64160130 2023-12-01T01:36:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6898d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641f2f00 con 0x7f5e640cfb20 2023-12-01T01:36:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6898d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e641f31e0 con 0x7f5e640d26e0 2023-12-01T01:36:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 0x7f5e640c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 0x7f5e640c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55664/0 (socket says 172.21.15.115:55664) 2023-12-01T01:36:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6359e640 1 -- 172.21.15.115:0/1354928611 learned_addr learned my addr 172.21.15.115:0/1354928611 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e62d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e640cfb20 0x7f5e640ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6259c640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e640d26e0 0x7f5e641f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6359e640 1 -- 172.21.15.115:0/1354928611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e640d26e0 msgr2=0x7f5e641f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6359e640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e640d26e0 0x7f5e641f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6359e640 1 -- 172.21.15.115:0/1354928611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e640cfb20 msgr2=0x7f5e640ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6359e640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e640cfb20 0x7f5e640ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.560+0000 7f5e6359e640 1 -- 172.21.15.115:0/1354928611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e54009460 con 0x7f5e64160130 2023-12-01T01:36:05.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.561+0000 7f5e6359e640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 0x7f5e640c8b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c780 tx=0x7f5e5c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:05.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.561+0000 7f5e4b7fe640 1 -- 172.21.15.115:0/1354928611 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c003b70 con 0x7f5e64160130 2023-12-01T01:36:05.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.561+0000 7f5e4b7fe640 1 -- 172.21.15.115:0/1354928611 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c003cd0 con 0x7f5e64160130 2023-12-01T01:36:05.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.561+0000 7f5e6898d640 1 -- 172.21.15.115:0/1354928611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e641f5910 con 0x7f5e64160130 2023-12-01T01:36:05.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.561+0000 7f5e4b7fe640 1 -- 172.21.15.115:0/1354928611 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0052b0 con 0x7f5e64160130 2023-12-01T01:36:05.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.561+0000 7f5e6898d640 1 -- 172.21.15.115:0/1354928611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e641f28a0 con 0x7f5e64160130 2023-12-01T01:36:05.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.562+0000 7f5e6898d640 1 -- 172.21.15.115:0/1354928611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e641f5e30 con 0x7f5e64160130 2023-12-01T01:36:05.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.563+0000 7f5e4b7fe640 1 -- 172.21.15.115:0/1354928611 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5e5c005410 con 0x7f5e64160130 2023-12-01T01:36:05.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.564+0000 7f5e4b7fe640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 0x7f5e5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.564+0000 7f5e4b7fe640 1 -- 172.21.15.115:0/1354928611 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e5c087cd0 con 0x7f5e64160130 2023-12-01T01:36:05.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.565+0000 7f5e62d9d640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 0x7f5e5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.565+0000 7f5e62d9d640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 0x7f5e5006c8f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e54035000 tx=0x7f5e54008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:05.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.569+0000 7f5e4b7fe640 1 -- 172.21.15.115:0/1354928611 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5e5c0536c0 con 0x7f5e64160130 2023-12-01T01:36:05.584 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T01:36:05.585 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T01:36:05.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.765+0000 7f5e6898d640 1 -- 172.21.15.115:0/1354928611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e640673a0 con 0x7f5e64160130 2023-12-01T01:36:05.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.767+0000 7f5e4b7fe640 1 -- 172.21.15.115:0/1354928611 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12430 (secure 0 0 0) 0x7f5e5c017020 con 0x7f5e64160130 2023-12-01T01:36:05.769 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:05.769 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":17,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:02.902562+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:05.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 -- 172.21.15.115:0/1354928611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 msgr2=0x7f5e5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 0x7f5e5006c8f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e54035000 tx=0x7f5e54008030 comp rx=0 tx=0).stop 2023-12-01T01:36:05.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 -- 172.21.15.115:0/1354928611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 msgr2=0x7f5e640c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 0x7f5e640c8b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c780 tx=0x7f5e5c00cb80 comp rx=0 tx=0).stop 2023-12-01T01:36:05.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 -- 172.21.15.115:0/1354928611 shutdown_connections 2023-12-01T01:36:05.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 0x7f5e5006c8f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e640d26e0 0x7f5e641f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e64160130 0x7f5e640c8b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 --2- 172.21.15.115:0/1354928611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e640cfb20 0x7f5e640ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 -- 172.21.15.115:0/1354928611 >> 172.21.15.115:0/1354928611 conn(0x7f5e6406a0b0 msgr2=0x7f5e640cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:05.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 -- 172.21.15.115:0/1354928611 shutdown_connections 2023-12-01T01:36:05.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.770+0000 7f5e6898d640 1 -- 172.21.15.115:0/1354928611 wait complete. 2023-12-01T01:36:05.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:36:05.791 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:36:05.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.991+0000 7fb5dd19f640 1 Processor -- start 2023-12-01T01:36:05.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.991+0000 7fb5dd19f640 1 -- start start 2023-12-01T01:36:05.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.993+0000 7fb5dd19f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d80ce1b0 0x7fb5d80cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.993+0000 7fb5dd19f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d80cc9a0 con 0x7fb5d80b0e60 2023-12-01T01:36:05.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.993+0000 7fb5dd19f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d80cbd90 con 0x7fb5d815ac80 2023-12-01T01:36:05.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.993+0000 7fb5dd19f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815b060 con 0x7fb5d80ce1b0 2023-12-01T01:36:05.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.993+0000 7fb5d759e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb5d80b0e60 0x7fb5d806b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36176/0 (socket says 172.21.15.115:36176) 2023-12-01T01:36:05.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.993+0000 7fb5d759e640 1 -- 172.21.15.115:0/476257761 learned_addr learned my addr 172.21.15.115:0/476257761 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:05.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.993+0000 7fb5cffff640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d80ce1b0 0x7fb5d80cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241053547 0 0) 0x7fb5d80cc9a0 con 0x7fb5d80b0e60 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c4003520 con 0x7fb5d80b0e60 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2736988055 0 0) 0x7fb5c4003520 con 0x7fb5d80b0e60 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d80ce1b0 msgr2=0x7fb5d80cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d80ce1b0 0x7fb5d80cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 >> v1:172.21.15.142:6789/0 conn(0x7fb5d815ac80 legacy=0x7fb5d80cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d80cc070 con 0x7fb5d80b0e60 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb5d00019f0 con 0x7fb5d80b0e60 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 >> v1:172.21.15.115:6789/0 conn(0x7fb5d80b0e60 legacy=0x7fb5d806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c40039a0 0x7fb5c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d80ce1b0 0x7fb5c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c4004d00 0x7fb5c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815b060 con 0x7fb5c4004d00 2023-12-01T01:36:05.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c400ba30 con 0x7fb5c40039a0 2023-12-01T01:36:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c400bbf0 con 0x7fb5d80ce1b0 2023-12-01T01:36:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.994+0000 7fb5d759e640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c4004d00 0x7fb5c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.995+0000 7fb5cffff640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d80ce1b0 0x7fb5c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.995+0000 7fb5d6d9d640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c40039a0 0x7fb5c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.995+0000 7fb5d759e640 1 -- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d80ce1b0 msgr2=0x7fb5c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.995+0000 7fb5d759e640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d80ce1b0 0x7fb5c40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.995+0000 7fb5d759e640 1 -- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c40039a0 msgr2=0x7fb5c4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.995+0000 7fb5d759e640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c40039a0 0x7fb5c4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.995+0000 7fb5d759e640 1 -- 172.21.15.115:0/476257761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c4003520 con 0x7fb5c4004d00 2023-12-01T01:36:05.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.995+0000 7fb5d6d9d640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5c40039a0 0x7fb5c4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.996+0000 7fb5d759e640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c4004d00 0x7fb5c4005180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000a7f0 tx=0x7fb5c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248cefc02f6d364a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.996+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00072d0 con 0x7fb5c4004d00 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.997+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d002f070 con 0x7fb5c4004d00 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.997+0000 7fb5d659c640 1 -- 172.21.15.115:0/476257761 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00076b0 con 0x7fb5c4004d00 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.997+0000 7fb5dd19f640 1 -- 172.21.15.115:0/476257761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c4004d00 msgr2=0x7fb5c4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.997+0000 7fb5dd19f640 1 --2- 172.21.15.115:0/476257761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5c4004d00 0x7fb5c4005180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000a7f0 tx=0x7fb5c40057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.997+0000 7fb5d759e640 1 -- 172.21.15.115:0/476257761 reap_dead start 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.997+0000 7fb5dd19f640 1 -- 172.21.15.115:0/476257761 shutdown_connections 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.997+0000 7fb5dd19f640 1 -- 172.21.15.115:0/476257761 >> 172.21.15.115:0/476257761 conn(0x7fb5d8065960 msgr2=0x7fb5d8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.997+0000 7fb5dd19f640 1 -- 172.21.15.115:0/476257761 shutdown_connections 2023-12-01T01:36:06.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.997+0000 7fb5dd19f640 1 -- 172.21.15.115:0/476257761 wait complete. 2023-12-01T01:36:06.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.999+0000 7fb5dd19f640 1 Processor -- start 2023-12-01T01:36:06.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:05.999+0000 7fb5dd19f640 1 -- start start 2023-12-01T01:36:06.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.000+0000 7fb5dd19f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5c4004d00 0x7fb5d815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.000+0000 7fb5dd19f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d815c000 0x7fb5d814c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.000+0000 7fb5dd19f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d8149690 0x7fb5d8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.000+0000 7fb5dd19f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815cb40 con 0x7fb5d815c000 2023-12-01T01:36:06.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.000+0000 7fb5dd19f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815b240 con 0x7fb5d8149690 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.000+0000 7fb5dd19f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815b520 con 0x7fb5c4004d00 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.000+0000 7fb5cffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d815c000 0x7fb5d814c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.000+0000 7fb5cffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d815c000 0x7fb5d814c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55686/0 (socket says 172.21.15.115:55686) 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.000+0000 7fb5cffff640 1 -- 172.21.15.115:0/890471712 learned_addr learned my addr 172.21.15.115:0/890471712 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5d759e640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d8149690 0x7fb5d8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5d6d9d640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5c4004d00 0x7fb5d815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5cffff640 1 -- 172.21.15.115:0/890471712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5c4004d00 msgr2=0x7fb5d815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5cffff640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5c4004d00 0x7fb5d815c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5cffff640 1 -- 172.21.15.115:0/890471712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d8149690 msgr2=0x7fb5d8152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5cffff640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d8149690 0x7fb5d8152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5cffff640 1 -- 172.21.15.115:0/890471712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d815b800 con 0x7fb5d815c000 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5cffff640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d815c000 0x7fb5d814c9e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0011600 tx=0x7fb5c0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5cf7fe640 1 -- 172.21.15.115:0/890471712 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000f920 con 0x7fb5d815c000 2023-12-01T01:36:06.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5cf7fe640 1 -- 172.21.15.115:0/890471712 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c000fa80 con 0x7fb5d815c000 2023-12-01T01:36:06.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5dd19f640 1 -- 172.21.15.115:0/890471712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d815bae0 con 0x7fb5d815c000 2023-12-01T01:36:06.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5cf7fe640 1 -- 172.21.15.115:0/890471712 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000fdf0 con 0x7fb5d815c000 2023-12-01T01:36:06.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.001+0000 7fb5dd19f640 1 -- 172.21.15.115:0/890471712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d8149b10 con 0x7fb5d815c000 2023-12-01T01:36:06.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.002+0000 7fb5dd19f640 1 -- 172.21.15.115:0/890471712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d815bc70 con 0x7fb5d815c000 2023-12-01T01:36:06.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.003+0000 7fb5cf7fe640 1 -- 172.21.15.115:0/890471712 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb5c0015460 con 0x7fb5d815c000 2023-12-01T01:36:06.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.004+0000 7fb5cf7fe640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5b806a460 0x7fb5b806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.004+0000 7fb5cf7fe640 1 -- 172.21.15.115:0/890471712 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb5c0088a90 con 0x7fb5d815c000 2023-12-01T01:36:06.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.004+0000 7fb5d6d9d640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5b806a460 0x7fb5b806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.005+0000 7fb5d6d9d640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5b806a460 0x7fb5b806c920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8007fd0 tx=0x7fb5c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:06.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.009+0000 7fb5cf7fe640 1 -- 172.21.15.115:0/890471712 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb5c00543d0 con 0x7fb5d815c000 2023-12-01T01:36:06.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.199+0000 7fb5dd19f640 1 -- 172.21.15.115:0/890471712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5d804e890 con 0x7fb5d815c000 2023-12-01T01:36:06.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.200+0000 7fb5cf7fe640 1 -- 172.21.15.115:0/890471712 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5c00579f0 con 0x7fb5d815c000 2023-12-01T01:36:06.202 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 -- 172.21.15.115:0/890471712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5b806a460 msgr2=0x7fb5b806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5b806a460 0x7fb5b806c920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8007fd0 tx=0x7fb5c8009040 comp rx=0 tx=0).stop 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 -- 172.21.15.115:0/890471712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d815c000 msgr2=0x7fb5d814c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d815c000 0x7fb5d814c9e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0011600 tx=0x7fb5c0011ac0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 -- 172.21.15.115:0/890471712 shutdown_connections 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5b806a460 0x7fb5b806c920 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d8149690 0x7fb5d8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d815c000 0x7fb5d814c9e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 --2- 172.21.15.115:0/890471712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5c4004d00 0x7fb5d815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 -- 172.21.15.115:0/890471712 >> 172.21.15.115:0/890471712 conn(0x7fb5d8065960 msgr2=0x7fb5d80b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:06.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 -- 172.21.15.115:0/890471712 shutdown_connections 2023-12-01T01:36:06.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.203+0000 7fb5dd19f640 1 -- 172.21.15.115:0/890471712 wait complete. 2023-12-01T01:36:06.222 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:36:06.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.417+0000 7fdc8cc70640 1 Processor -- start 2023-12-01T01:36:06.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.418+0000 7fdc8cc70640 1 -- start start 2023-12-01T01:36:06.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.419+0000 7fdc8cc70640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc8815cd20 0x7fdc880d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.419+0000 7fdc8cc70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc880b5d80 0x7fdc880b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.419+0000 7fdc8cc70640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdc880d1230 0x7fdc880d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.419+0000 7fdc8cc70640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc880b2160 con 0x7fdc8815cd20 2023-12-01T01:36:06.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.419+0000 7fdc8cc70640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc880d1be0 con 0x7fdc880d1230 2023-12-01T01:36:06.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.419+0000 7fdc8cc70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc880d2a10 con 0x7fdc880b5d80 2023-12-01T01:36:06.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.419+0000 7fdc8659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc8815cd20 0x7fdc880d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.419+0000 7fdc8659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc8815cd20 0x7fdc880d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55694/0 (socket says 172.21.15.115:55694) 2023-12-01T01:36:06.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.419+0000 7fdc8659c640 1 -- 172.21.15.115:0/1934001865 learned_addr learned my addr 172.21.15.115:0/1934001865 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:06.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.420+0000 7fdc85d9b640 1 --2- 172.21.15.115:0/1934001865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc880b5d80 0x7fdc880b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.420+0000 7fdc8659c640 1 -- 172.21.15.115:0/1934001865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc880b5d80 msgr2=0x7fdc880b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:06.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.420+0000 7fdc8659c640 1 --2- 172.21.15.115:0/1934001865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc880b5d80 0x7fdc880b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.420+0000 7fdc8659c640 1 -- 172.21.15.115:0/1934001865 >> v2:172.21.15.142:3300/0 conn(0x7fdc880d1230 msgr2=0x7fdc880d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:06.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.420+0000 7fdc8659c640 1 --2- 172.21.15.115:0/1934001865 >> v2:172.21.15.142:3300/0 conn(0x7fdc880d1230 0x7fdc880d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.420+0000 7fdc8659c640 1 -- 172.21.15.115:0/1934001865 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc880ce2e0 con 0x7fdc8815cd20 2023-12-01T01:36:06.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.421+0000 7fdc8659c640 1 --2- 172.21.15.115:0/1934001865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 0x7fdc880d24d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc880b5b50 tx=0x7fdc7802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ad81a04b7fb695 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:06.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.421+0000 7fdc8559a640 1 -- 172.21.15.115:0/1934001865 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7802c9e0 con 0x7fdc8815cd20 2023-12-01T01:36:06.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.421+0000 7fdc8559a640 1 -- 172.21.15.115:0/1934001865 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7802cb40 con 0x7fdc8815cd20 2023-12-01T01:36:06.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.421+0000 7fdc8559a640 1 -- 172.21.15.115:0/1934001865 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780343f0 con 0x7fdc8815cd20 2023-12-01T01:36:06.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.422+0000 7fdc8cc70640 1 -- 172.21.15.115:0/1934001865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 msgr2=0x7fdc880d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.422+0000 7fdc8cc70640 1 --2- 172.21.15.115:0/1934001865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 0x7fdc880d24d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc880b5b50 tx=0x7fdc7802be40 comp rx=0 tx=0).stop 2023-12-01T01:36:06.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.422+0000 7fdc8cc70640 1 -- 172.21.15.115:0/1934001865 shutdown_connections 2023-12-01T01:36:06.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.422+0000 7fdc8cc70640 1 --2- 172.21.15.115:0/1934001865 >> v2:172.21.15.142:3300/0 conn(0x7fdc880d1230 0x7fdc880d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.422+0000 7fdc8cc70640 1 --2- 172.21.15.115:0/1934001865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc880b5d80 0x7fdc880b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.422+0000 7fdc8cc70640 1 --2- 172.21.15.115:0/1934001865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 0x7fdc880d24d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.422+0000 7fdc8cc70640 1 -- 172.21.15.115:0/1934001865 >> 172.21.15.115:0/1934001865 conn(0x7fdc8806a0e0 msgr2=0x7fdc8806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:06.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.422+0000 7fdc8cc70640 1 -- 172.21.15.115:0/1934001865 shutdown_connections 2023-12-01T01:36:06.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.422+0000 7fdc8cc70640 1 -- 172.21.15.115:0/1934001865 wait complete. 2023-12-01T01:36:06.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.423+0000 7fdc8cc70640 1 Processor -- start 2023-12-01T01:36:06.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.424+0000 7fdc8cc70640 1 -- start start 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.424+0000 7fdc8cc70640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc880b5d80 0x7fdc8815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc8cc70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc880d1230 0x7fdc8815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc8cc70640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 0x7fdc880c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc8cc70640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8814d450 con 0x7fdc8815cd20 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc8cc70640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc880b2fd0 con 0x7fdc880b5d80 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc8cc70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8815f1f0 con 0x7fdc880d1230 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc86d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 0x7fdc880c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc86d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 0x7fdc880c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55698/0 (socket says 172.21.15.115:55698) 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc86d9d640 1 -- 172.21.15.115:0/2103534613 learned_addr learned my addr 172.21.15.115:0/2103534613 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc8659c640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc880b5d80 0x7fdc8815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc85d9b640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc880d1230 0x7fdc8815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc86d9d640 1 -- 172.21.15.115:0/2103534613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc880d1230 msgr2=0x7fdc8815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc86d9d640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc880d1230 0x7fdc8815ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc86d9d640 1 -- 172.21.15.115:0/2103534613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc880b5d80 msgr2=0x7fdc8815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc86d9d640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc880b5d80 0x7fdc8815dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc86d9d640 1 -- 172.21.15.115:0/2103534613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78009b30 con 0x7fdc8815cd20 2023-12-01T01:36:06.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.425+0000 7fdc8659c640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc880b5d80 0x7fdc8815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:06.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.426+0000 7fdc86d9d640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 0x7fdc880c5f90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009fd0 tx=0x7fdc8000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:06.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.426+0000 7fdc7effd640 1 -- 172.21.15.115:0/2103534613 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc80011830 con 0x7fdc8815cd20 2023-12-01T01:36:06.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.426+0000 7fdc7effd640 1 -- 172.21.15.115:0/2103534613 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc80011990 con 0x7fdc8815cd20 2023-12-01T01:36:06.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.426+0000 7fdc8cc70640 1 -- 172.21.15.115:0/2103534613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8815f4d0 con 0x7fdc8815cd20 2023-12-01T01:36:06.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.426+0000 7fdc7effd640 1 -- 172.21.15.115:0/2103534613 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc80011d20 con 0x7fdc8815cd20 2023-12-01T01:36:06.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.426+0000 7fdc8cc70640 1 -- 172.21.15.115:0/2103534613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc880b3500 con 0x7fdc8815cd20 2023-12-01T01:36:06.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.427+0000 7fdc8cc70640 1 -- 172.21.15.115:0/2103534613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc880b3720 con 0x7fdc8815cd20 2023-12-01T01:36:06.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.428+0000 7fdc7effd640 1 -- 172.21.15.115:0/2103534613 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdc8001b460 con 0x7fdc8815cd20 2023-12-01T01:36:06.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.429+0000 7fdc7effd640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc6806a430 0x7fdc6806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.429+0000 7fdc7effd640 1 -- 172.21.15.115:0/2103534613 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdc80088550 con 0x7fdc8815cd20 2023-12-01T01:36:06.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.429+0000 7fdc8659c640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc6806a430 0x7fdc6806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.430+0000 7fdc8659c640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc6806a430 0x7fdc6806c8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc78035000 tx=0x7fdc78030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:06.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.433+0000 7fdc7effd640 1 -- 172.21.15.115:0/2103534613 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdc800577f0 con 0x7fdc8815cd20 2023-12-01T01:36:06.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.663+0000 7fdc8cc70640 1 -- 172.21.15.115:0/2103534613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc880b4ba0 con 0x7fdc8815cd20 2023-12-01T01:36:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.664+0000 7fdc7effd640 1 -- 172.21.15.115:0/2103534613 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7fdc80057380 con 0x7fdc8815cd20 2023-12-01T01:36:06.666 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:06.666 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3656795862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3656795862},{"type":"v1","addr":"172.21.15.142:6834","nonce":3656795862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1125002206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1125002206},{"type":"v1","addr":"172.21.15.115:6836","nonce":1125002206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3350782192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3350782192},{"type":"v1","addr":"172.21.15.115:6837","nonce":3350782192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2808777047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2808777047},{"type":"v1","addr":"172.21.15.142:6835","nonce":2808777047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-12-01T01:36:06.666 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 3 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 -- 172.21.15.115:0/2103534613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc6806a430 msgr2=0x7fdc6806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc6806a430 0x7fdc6806c8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc78035000 tx=0x7fdc78030000 comp rx=0 tx=0).stop 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 -- 172.21.15.115:0/2103534613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 msgr2=0x7fdc880c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 0x7fdc880c5f90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009fd0 tx=0x7fdc8000b8e0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 -- 172.21.15.115:0/2103534613 shutdown_connections 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc6806a430 0x7fdc6806c8f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc880d1230 0x7fdc8815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc8815cd20 0x7fdc880c5f90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 --2- 172.21.15.115:0/2103534613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc880b5d80 0x7fdc8815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 -- 172.21.15.115:0/2103534613 >> 172.21.15.115:0/2103534613 conn(0x7fdc8806a0e0 msgr2=0x7fdc880cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 -- 172.21.15.115:0/2103534613 shutdown_connections 2023-12-01T01:36:06.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.667+0000 7fdc8cc70640 1 -- 172.21.15.115:0/2103534613 wait complete. 2023-12-01T01:36:06.687 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:06.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.883+0000 7fe5bce5f640 1 Processor -- start 2023-12-01T01:36:06.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.884+0000 7fe5bce5f640 1 -- start start 2023-12-01T01:36:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.885+0000 7fe5bce5f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe5b80b5d80 0x7fe5b80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.885+0000 7fe5bce5f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b80ce2e0 con 0x7fe5b80d1230 2023-12-01T01:36:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.885+0000 7fe5bce5f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b80ce000 con 0x7fe5b815cd20 2023-12-01T01:36:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.885+0000 7fe5bce5f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b80b2160 con 0x7fe5b80b5d80 2023-12-01T01:36:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.885+0000 7fe5b6d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe5b80d1230 0x7fe5b80d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36188/0 (socket says 172.21.15.115:36188) 2023-12-01T01:36:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.885+0000 7fe5b6d9d640 1 -- 172.21.15.115:0/3272789761 learned_addr learned my addr 172.21.15.115:0/3272789761 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630823040 0 0) 0x7fe5b80ce2e0 con 0x7fe5b80d1230 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b5d9b640 1 --2- 172.21.15.115:0/3272789761 >> v2:172.21.15.142:3300/0 conn(0x7fe5b80b5d80 0x7fe5b80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5a4003520 con 0x7fe5b80d1230 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1510205804 0 0) 0x7fe5a4003520 con 0x7fe5b80d1230 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 >> v2:172.21.15.142:3300/0 conn(0x7fe5b80b5d80 msgr2=0x7fe5b80d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 --2- 172.21.15.115:0/3272789761 >> v2:172.21.15.142:3300/0 conn(0x7fe5b80b5d80 0x7fe5b80d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 >> v1:172.21.15.142:6789/0 conn(0x7fe5b815cd20 legacy=0x7fe5b80b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b80cd3d0 con 0x7fe5b80d1230 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fe5b0001670 con 0x7fe5b80d1230 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 >> v1:172.21.15.115:6789/0 conn(0x7fe5b80d1230 legacy=0x7fe5b80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5a4003a30 0x7fe5a4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.886+0000 7fe5b559a640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5a40047c0 0x7fe5a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b559a640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a400c140 con 0x7fe5a40052a0 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a400c2b0 con 0x7fe5a4003a30 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a400c470 con 0x7fe5a40047c0 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b6d9d640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b5d9b640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5a40047c0 0x7fe5a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b659c640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5a4003a30 0x7fe5a4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b6d9d640 1 -- 172.21.15.115:0/3272789761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5a40047c0 msgr2=0x7fe5a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b6d9d640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5a40047c0 0x7fe5a4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b6d9d640 1 -- 172.21.15.115:0/3272789761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5a4003a30 msgr2=0x7fe5a4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b6d9d640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5a4003a30 0x7fe5a4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.887+0000 7fe5b6d9d640 1 -- 172.21.15.115:0/3272789761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a4003520 con 0x7fe5a40052a0 2023-12-01T01:36:06.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.888+0000 7fe5b6d9d640 1 -- 172.21.15.115:0/3272789761 reap_dead start 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.888+0000 7fe5b6d9d640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5a400b980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0001050 tx=0x7fe5a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a1b1ca66b8c273 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.889+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0005ad0 con 0x7fe5a40052a0 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.889+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b0005c70 con 0x7fe5a40052a0 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.889+0000 7fe5b559a640 1 -- 172.21.15.115:0/3272789761 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b002c7c0 con 0x7fe5a40052a0 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.889+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3272789761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 msgr2=0x7fe5a400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.889+0000 7fe5bce5f640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5a400b980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0001050 tx=0x7fe5a40057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.889+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3272789761 shutdown_connections 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.889+0000 7fe5bce5f640 1 --2- 172.21.15.115:0/3272789761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5a400b980 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.889+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3272789761 >> 172.21.15.115:0/3272789761 conn(0x7fe5b806a0e0 msgr2=0x7fe5b806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.889+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3272789761 shutdown_connections 2023-12-01T01:36:06.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.889+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3272789761 wait complete. 2023-12-01T01:36:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.891+0000 7fe5bce5f640 1 Processor -- start 2023-12-01T01:36:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.891+0000 7fe5bce5f640 1 -- start start 2023-12-01T01:36:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.892+0000 7fe5bce5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5b814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.892+0000 7fe5bce5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5b8156150 0x7fe5b81565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.892+0000 7fe5bce5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b8152b70 0x7fe5b8152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.892+0000 7fe5bce5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b81f6b20 con 0x7fe5a40052a0 2023-12-01T01:36:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.892+0000 7fe5bce5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b8156b10 con 0x7fe5b8152b70 2023-12-01T01:36:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.892+0000 7fe5bce5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b8151420 con 0x7fe5b8156150 2023-12-01T01:36:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.892+0000 7fe5b659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5b814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.892+0000 7fe5b659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5b814ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55728/0 (socket says 172.21.15.115:55728) 2023-12-01T01:36:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.892+0000 7fe5b659c640 1 -- 172.21.15.115:0/3800502592 learned_addr learned my addr 172.21.15.115:0/3800502592 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5b5d9b640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5b8156150 0x7fe5b81565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5b659c640 1 -- 172.21.15.115:0/3800502592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5b8156150 msgr2=0x7fe5b81565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5b659c640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5b8156150 0x7fe5b81565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5b659c640 1 -- 172.21.15.115:0/3800502592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b8152b70 msgr2=0x7fe5b8152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5b659c640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b8152b70 0x7fe5b8152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5b659c640 1 -- 172.21.15.115:0/3800502592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b81f5ee0 con 0x7fe5a40052a0 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5b659c640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5b814ea30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800e820 tx=0x7fe5a800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5aeffd640 1 -- 172.21.15.115:0/3800502592 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8003b80 con 0x7fe5a40052a0 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5aeffd640 1 -- 172.21.15.115:0/3800502592 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a8003ce0 con 0x7fe5a40052a0 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5aeffd640 1 -- 172.21.15.115:0/3800502592 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8005310 con 0x7fe5a40052a0 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.893+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3800502592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b8153530 con 0x7fe5a40052a0 2023-12-01T01:36:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.894+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3800502592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b81f6460 con 0x7fe5a40052a0 2023-12-01T01:36:06.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.895+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3800502592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b804e890 con 0x7fe5a40052a0 2023-12-01T01:36:06.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.895+0000 7fe5aeffd640 1 -- 172.21.15.115:0/3800502592 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe5a8005470 con 0x7fe5a40052a0 2023-12-01T01:36:06.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.896+0000 7fe5aeffd640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe59806a460 0x7fe59806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:06.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.896+0000 7fe5aeffd640 1 -- 172.21.15.115:0/3800502592 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe5a80885e0 con 0x7fe5a40052a0 2023-12-01T01:36:06.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.896+0000 7fe5b5d9b640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe59806a460 0x7fe59806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:06.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.897+0000 7fe5b5d9b640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe59806a460 0x7fe59806c920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80b1ae0 tx=0x7fe5a0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:06.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:06.901+0000 7fe5aeffd640 1 -- 172.21.15.115:0/3800502592 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe5a8053f30 con 0x7fe5a40052a0 2023-12-01T01:36:07.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.086+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3800502592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5b80b1b30 con 0x7fe5a40052a0 2023-12-01T01:36:07.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.088+0000 7fe5aeffd640 1 -- 172.21.15.115:0/3800502592 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12430 (secure 0 0 0) 0x7fe5a80018a0 con 0x7fe5a40052a0 2023-12-01T01:36:07.090 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:07.090 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":17,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:02.902562+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:07.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.090+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3800502592 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe59806a460 msgr2=0x7fe59806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.090+0000 7fe5bce5f640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe59806a460 0x7fe59806c920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80b1ae0 tx=0x7fe5a0008040 comp rx=0 tx=0).stop 2023-12-01T01:36:07.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.090+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3800502592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 msgr2=0x7fe5b814ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.090+0000 7fe5bce5f640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5b814ea30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800e820 tx=0x7fe5a800ece0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.091+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3800502592 shutdown_connections 2023-12-01T01:36:07.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.091+0000 7fe5bce5f640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe59806a460 0x7fe59806c920 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.091+0000 7fe5bce5f640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5b8156150 0x7fe5b81565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.091+0000 7fe5bce5f640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b8152b70 0x7fe5b8152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.091+0000 7fe5bce5f640 1 --2- 172.21.15.115:0/3800502592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5a40052a0 0x7fe5b814ea30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.091+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3800502592 >> 172.21.15.115:0/3800502592 conn(0x7fe5b806a0e0 msgr2=0x7fe5b80c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:07.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.091+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3800502592 shutdown_connections 2023-12-01T01:36:07.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.091+0000 7fe5bce5f640 1 -- 172.21.15.115:0/3800502592 wait complete. 2023-12-01T01:36:07.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:36:07.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:36:07.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.314+0000 7f870db8e640 1 Processor -- start 2023-12-01T01:36:07.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.314+0000 7f870db8e640 1 -- start start 2023-12-01T01:36:07.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.315+0000 7f870db8e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f87080af370 0x7f87080c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.315+0000 7f870db8e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f87080c95a0 0x7f8708070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.315+0000 7f870db8e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870815e440 con 0x7f87081599e0 2023-12-01T01:36:07.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.315+0000 7f870db8e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870815d550 con 0x7f87080af370 2023-12-01T01:36:07.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.315+0000 7f870db8e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870815d830 con 0x7f87080c95a0 2023-12-01T01:36:07.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.315+0000 7f8707fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f87080af370 0x7f87080c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.316+0000 7f8707fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f87080af370 0x7f87080c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55732/0 (socket says 172.21.15.115:55732) 2023-12-01T01:36:07.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.316+0000 7f8707fff640 1 -- 172.21.15.115:0/2160242629 learned_addr learned my addr 172.21.15.115:0/2160242629 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:07.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.316+0000 7f8707fff640 1 -- 172.21.15.115:0/2160242629 >> v2:172.21.15.142:3300/0 conn(0x7f87080c95a0 msgr2=0x7f8708070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:07.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.316+0000 7f8707fff640 1 --2- 172.21.15.115:0/2160242629 >> v2:172.21.15.142:3300/0 conn(0x7f87080c95a0 0x7f8708070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.316+0000 7f8707fff640 1 -- 172.21.15.115:0/2160242629 >> v1:172.21.15.142:6789/0 conn(0x7f87081599e0 legacy=0x7f8708159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:07.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.316+0000 7f8707fff640 1 -- 172.21.15.115:0/2160242629 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870815db10 con 0x7f87080af370 2023-12-01T01:36:07.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.317+0000 7f8707fff640 1 --2- 172.21.15.115:0/2160242629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080af370 0x7f87080c9060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f870806c6b0 tx=0x7f86f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28e5fbdf828ab44 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:07.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.317+0000 7f87077fe640 1 -- 172.21.15.115:0/2160242629 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4031870 con 0x7f87080af370 2023-12-01T01:36:07.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.317+0000 7f87077fe640 1 -- 172.21.15.115:0/2160242629 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f40319d0 con 0x7f87080af370 2023-12-01T01:36:07.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.317+0000 7f87077fe640 1 -- 172.21.15.115:0/2160242629 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4031d60 con 0x7f87080af370 2023-12-01T01:36:07.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.317+0000 7f870db8e640 1 -- 172.21.15.115:0/2160242629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080af370 msgr2=0x7f87080c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.317+0000 7f870db8e640 1 --2- 172.21.15.115:0/2160242629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080af370 0x7f87080c9060 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f870806c6b0 tx=0x7f86f402b910 comp rx=0 tx=0).stop 2023-12-01T01:36:07.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.318+0000 7f870db8e640 1 -- 172.21.15.115:0/2160242629 shutdown_connections 2023-12-01T01:36:07.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.318+0000 7f870db8e640 1 --2- 172.21.15.115:0/2160242629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080af370 0x7f87080c9060 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.318+0000 7f870db8e640 1 --2- 172.21.15.115:0/2160242629 >> v2:172.21.15.142:3300/0 conn(0x7f87080c95a0 0x7f8708070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.318+0000 7f870db8e640 1 -- 172.21.15.115:0/2160242629 >> 172.21.15.115:0/2160242629 conn(0x7f8708065960 msgr2=0x7f8708065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:07.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.318+0000 7f870db8e640 1 -- 172.21.15.115:0/2160242629 shutdown_connections 2023-12-01T01:36:07.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.318+0000 7f870db8e640 1 -- 172.21.15.115:0/2160242629 wait complete. 2023-12-01T01:36:07.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.319+0000 7f870db8e640 1 Processor -- start 2023-12-01T01:36:07.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.319+0000 7f870db8e640 1 -- start start 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.320+0000 7f870db8e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87080af370 0x7f87080b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.320+0000 7f870db8e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080c95a0 0x7f870806f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.320+0000 7f870db8e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87081599e0 0x7f87080acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.320+0000 7f870db8e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87080c6e40 con 0x7f87080c95a0 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.320+0000 7f870db8e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87080c6850 con 0x7f87081599e0 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.320+0000 7f870db8e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870806ff30 con 0x7f87080af370 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f8707fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080c95a0 0x7f870806f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f8707fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080c95a0 0x7f870806f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55738/0 (socket says 172.21.15.115:55738) 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f8707fff640 1 -- 172.21.15.115:0/367990012 learned_addr learned my addr 172.21.15.115:0/367990012 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f8707fff640 1 -- 172.21.15.115:0/367990012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87080af370 msgr2=0x7f87080b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f8707fff640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87080af370 0x7f87080b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f8707fff640 1 -- 172.21.15.115:0/367990012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87081599e0 msgr2=0x7f87080acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f8707fff640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87081599e0 0x7f87080acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f8707fff640 1 -- 172.21.15.115:0/367990012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f40094d0 con 0x7f87080c95a0 2023-12-01T01:36:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f8707fff640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080c95a0 0x7f870806f9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f86f4000c00 tx=0x7f86f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f87057fa640 1 -- 172.21.15.115:0/367990012 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f403ad10 con 0x7f87080c95a0 2023-12-01T01:36:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f87057fa640 1 -- 172.21.15.115:0/367990012 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f403ae70 con 0x7f87080c95a0 2023-12-01T01:36:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f87057fa640 1 -- 172.21.15.115:0/367990012 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4044610 con 0x7f87080c95a0 2023-12-01T01:36:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.321+0000 7f870db8e640 1 -- 172.21.15.115:0/367990012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8708070b40 con 0x7f87080c95a0 2023-12-01T01:36:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.322+0000 7f870db8e640 1 -- 172.21.15.115:0/367990012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87080c6a60 con 0x7f87080c95a0 2023-12-01T01:36:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.323+0000 7f870db8e640 1 -- 172.21.15.115:0/367990012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87080c6f80 con 0x7f87080c95a0 2023-12-01T01:36:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.323+0000 7f87057fa640 1 -- 172.21.15.115:0/367990012 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f86f404d450 con 0x7f87080c95a0 2023-12-01T01:36:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.324+0000 7f87057fa640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86ec06a430 0x7f86ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.324+0000 7f87057fa640 1 -- 172.21.15.115:0/367990012 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f86f402d040 con 0x7f87080c95a0 2023-12-01T01:36:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.324+0000 7f870cb8c640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86ec06a430 0x7f86ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.325+0000 7f870cb8c640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86ec06a430 0x7f86ec06c8f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87080ad3f0 tx=0x7f86fc006ba0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.329+0000 7f87057fa640 1 -- 172.21.15.115:0/367990012 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f87080c6f80 con 0x7f87080c95a0 2023-12-01T01:36:07.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.518+0000 7f870db8e640 1 -- 172.21.15.115:0/367990012 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87080c6f80 con 0x7f87080c95a0 2023-12-01T01:36:07.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.519+0000 7f87057fa640 1 -- 172.21.15.115:0/367990012 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87080c6f80 con 0x7f87080c95a0 2023-12-01T01:36:07.521 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 -- 172.21.15.115:0/367990012 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86ec06a430 msgr2=0x7f86ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86ec06a430 0x7f86ec06c8f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87080ad3f0 tx=0x7f86fc006ba0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 -- 172.21.15.115:0/367990012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080c95a0 msgr2=0x7f870806f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080c95a0 0x7f870806f9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f86f4000c00 tx=0x7f86f40046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 -- 172.21.15.115:0/367990012 shutdown_connections 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86ec06a430 0x7f86ec06c8f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87081599e0 0x7f87080acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87080c95a0 0x7f870806f9f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 --2- 172.21.15.115:0/367990012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87080af370 0x7f87080b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 -- 172.21.15.115:0/367990012 >> 172.21.15.115:0/367990012 conn(0x7f8708065960 msgr2=0x7f87080afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 -- 172.21.15.115:0/367990012 shutdown_connections 2023-12-01T01:36:07.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.522+0000 7f870db8e640 1 -- 172.21.15.115:0/367990012 wait complete. 2023-12-01T01:36:07.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T01:36:07.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.740+0000 7ff7581f0640 1 Processor -- start 2023-12-01T01:36:07.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.740+0000 7ff7581f0640 1 -- start start 2023-12-01T01:36:07.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.742+0000 7ff7581f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500adac0 0x7ff7500cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.742+0000 7ff7581f0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7500ce150 con 0x7ff7500c8ed0 2023-12-01T01:36:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.742+0000 7ff7581f0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75006d620 con 0x7ff75006b350 2023-12-01T01:36:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.742+0000 7ff7581f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500cf3a0 con 0x7ff7500adac0 2023-12-01T01:36:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.742+0000 7ff75578b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff7500c8ed0 0x7ff7500cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41342/0 (socket says 172.21.15.115:41342) 2023-12-01T01:36:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.742+0000 7ff75578b640 1 -- 172.21.15.115:0/3991825190 learned_addr learned my addr 172.21.15.115:0/3991825190 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff755f8c640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500adac0 0x7ff7500cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930175528 0 0) 0x7ff7500ce150 con 0x7ff7500c8ed0 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff73c003520 con 0x7ff7500c8ed0 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4092653435 0 0) 0x7ff73c003520 con 0x7ff7500c8ed0 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500adac0 msgr2=0x7ff7500cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500adac0 0x7ff7500cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 >> v1:172.21.15.142:6789/0 conn(0x7ff75006b350 legacy=0x7ff75006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500c1d90 con 0x7ff7500c8ed0 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7ff740001670 con 0x7ff7500c8ed0 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 >> v1:172.21.15.115:6789/0 conn(0x7ff7500c8ed0 legacy=0x7ff7500cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff73c003a10 0x7ff73c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73c0047c0 0x7ff73c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500adac0 0x7ff73c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500cf3a0 con 0x7ff73c003a10 2023-12-01T01:36:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c00ba80 con 0x7ff73c0047c0 2023-12-01T01:36:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c00bc40 con 0x7ff7500adac0 2023-12-01T01:36:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.743+0000 7ff755f8c640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff73c003a10 0x7ff73c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.744+0000 7ff75678d640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500adac0 0x7ff73c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.744+0000 7ff75578b640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73c0047c0 0x7ff73c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.744+0000 7ff755f8c640 1 -- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500adac0 msgr2=0x7ff73c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.744+0000 7ff755f8c640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500adac0 0x7ff73c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.744+0000 7ff755f8c640 1 -- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73c0047c0 msgr2=0x7ff73c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.744+0000 7ff755f8c640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73c0047c0 0x7ff73c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.744+0000 7ff755f8c640 1 -- 172.21.15.115:0/3991825190 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c003520 con 0x7ff73c003a10 2023-12-01T01:36:07.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.745+0000 7ff755f8c640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff73c003a10 0x7ff73c003e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009ac0 tx=0x7ff73c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5df85a33429709 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:07.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.745+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c02c6b0 con 0x7ff73c003a10 2023-12-01T01:36:07.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.745+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c02c810 con 0x7ff73c003a10 2023-12-01T01:36:07.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.745+0000 7ff754f8a640 1 -- 172.21.15.115:0/3991825190 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c02cba0 con 0x7ff73c003a10 2023-12-01T01:36:07.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.746+0000 7ff7581f0640 1 -- 172.21.15.115:0/3991825190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff73c003a10 msgr2=0x7ff73c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.746+0000 7ff7581f0640 1 --2- 172.21.15.115:0/3991825190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff73c003a10 0x7ff73c003e30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009ac0 tx=0x7ff73c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.746+0000 7ff75678d640 1 -- 172.21.15.115:0/3991825190 reap_dead start 2023-12-01T01:36:07.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.746+0000 7ff7581f0640 1 -- 172.21.15.115:0/3991825190 shutdown_connections 2023-12-01T01:36:07.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.746+0000 7ff7581f0640 1 -- 172.21.15.115:0/3991825190 >> 172.21.15.115:0/3991825190 conn(0x7ff7500ce880 msgr2=0x7ff7500cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:07.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.746+0000 7ff7581f0640 1 -- 172.21.15.115:0/3991825190 shutdown_connections 2023-12-01T01:36:07.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.746+0000 7ff7581f0640 1 -- 172.21.15.115:0/3991825190 wait complete. 2023-12-01T01:36:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.748+0000 7ff7581f0640 1 Processor -- start 2023-12-01T01:36:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.748+0000 7ff7581f0640 1 -- start start 2023-12-01T01:36:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.748+0000 7ff7581f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73c003a10 0x7ff75015fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.748+0000 7ff7581f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500c2390 0x7ff7500c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff7581f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7500c3930 0x7ff7500ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff7581f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500c2890 con 0x7ff7500c3930 2023-12-01T01:36:07.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff7581f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500c2b70 con 0x7ff73c003a10 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff7581f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500c2e50 con 0x7ff7500c2390 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff75678d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7500c3930 0x7ff7500ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff75678d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7500c3930 0x7ff7500ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42608/0 (socket says 172.21.15.115:42608) 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff75678d640 1 -- 172.21.15.115:0/3370225970 learned_addr learned my addr 172.21.15.115:0/3370225970 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff755f8c640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73c003a10 0x7ff75015fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff75578b640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500c2390 0x7ff7500c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff75678d640 1 -- 172.21.15.115:0/3370225970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500c2390 msgr2=0x7ff7500c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff75678d640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500c2390 0x7ff7500c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff75678d640 1 -- 172.21.15.115:0/3370225970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73c003a10 msgr2=0x7ff75015fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff75678d640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73c003a10 0x7ff75015fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff75678d640 1 -- 172.21.15.115:0/3370225970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500c3130 con 0x7ff7500c3930 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.749+0000 7ff75678d640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7500c3930 0x7ff7500ae2a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff75006ed00 tx=0x7ff74800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.750+0000 7ff7467fc640 1 -- 172.21.15.115:0/3370225970 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748003b70 con 0x7ff7500c3930 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.750+0000 7ff7467fc640 1 -- 172.21.15.115:0/3370225970 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff748003cd0 con 0x7ff7500c3930 2023-12-01T01:36:07.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.750+0000 7ff7467fc640 1 -- 172.21.15.115:0/3370225970 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7480052b0 con 0x7ff7500c3930 2023-12-01T01:36:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.750+0000 7ff7581f0640 1 -- 172.21.15.115:0/3370225970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7500c3410 con 0x7ff7500c3930 2023-12-01T01:36:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.750+0000 7ff7581f0640 1 -- 172.21.15.115:0/3370225970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7500c8880 con 0x7ff7500c3930 2023-12-01T01:36:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.751+0000 7ff7581f0640 1 -- 172.21.15.115:0/3370225970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7500c3600 con 0x7ff7500c3930 2023-12-01T01:36:07.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.752+0000 7ff7467fc640 1 -- 172.21.15.115:0/3370225970 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff748005410 con 0x7ff7500c3930 2023-12-01T01:36:07.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.753+0000 7ff7467fc640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff73406a460 0x7ff73406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:07.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.753+0000 7ff7467fc640 1 -- 172.21.15.115:0/3370225970 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff748087fd0 con 0x7ff7500c3930 2023-12-01T01:36:07.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.753+0000 7ff755f8c640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff73406a460 0x7ff73406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:07.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.753+0000 7ff755f8c640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff73406a460 0x7ff73406c920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff74c036000 tx=0x7ff74c030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:07.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.757+0000 7ff7467fc640 1 -- 172.21.15.115:0/3370225970 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff7480539a0 con 0x7ff7500c3930 2023-12-01T01:36:07.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.935+0000 7ff7581f0640 1 -- 172.21.15.115:0/3370225970 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff75004e890 con 0x7ff7500c3930 2023-12-01T01:36:07.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.936+0000 7ff7467fc640 1 -- 172.21.15.115:0/3370225970 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff748018020 con 0x7ff7500c3930 2023-12-01T01:36:07.937 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:07.937 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:36:07.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:36:07.937 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T01:36:07.937 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T01:36:07.937 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:36:07.937 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:36:07.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.938+0000 7ff7581f0640 1 -- 172.21.15.115:0/3370225970 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff73406a460 msgr2=0x7ff73406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.938+0000 7ff7581f0640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff73406a460 0x7ff73406c920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff74c036000 tx=0x7ff74c030000 comp rx=0 tx=0).stop 2023-12-01T01:36:07.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.938+0000 7ff7581f0640 1 -- 172.21.15.115:0/3370225970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7500c3930 msgr2=0x7ff7500ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:07.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.938+0000 7ff7581f0640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7500c3930 0x7ff7500ae2a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff75006ed00 tx=0x7ff74800dc00 comp rx=0 tx=0).stop 2023-12-01T01:36:07.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.939+0000 7ff7581f0640 1 -- 172.21.15.115:0/3370225970 shutdown_connections 2023-12-01T01:36:07.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.939+0000 7ff7581f0640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff73406a460 0x7ff73406c920 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.939+0000 7ff7581f0640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7500c2390 0x7ff7500c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.939+0000 7ff7581f0640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7500c3930 0x7ff7500ae2a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.939+0000 7ff7581f0640 1 --2- 172.21.15.115:0/3370225970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff73c003a10 0x7ff75015fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:07.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.939+0000 7ff7581f0640 1 -- 172.21.15.115:0/3370225970 >> 172.21.15.115:0/3370225970 conn(0x7ff7500ce880 msgr2=0x7ff7500cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:07.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.939+0000 7ff7581f0640 1 -- 172.21.15.115:0/3370225970 shutdown_connections 2023-12-01T01:36:07.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:07.939+0000 7ff7581f0640 1 -- 172.21.15.115:0/3370225970 wait complete. 2023-12-01T01:36:07.957 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T01:36:07.957 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T01:36:08.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.168+0000 7fecf2c4d640 1 Processor -- start 2023-12-01T01:36:08.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.169+0000 7fecf2c4d640 1 -- start start 2023-12-01T01:36:08.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.170+0000 7fecf2c4d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fecec161620 0x7fecec0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:08.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.170+0000 7fecf2c4d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fecec0add00 0x7fecec0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:08.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.170+0000 7fecf2c4d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec161a00 con 0x7fecec0ce4a0 2023-12-01T01:36:08.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.170+0000 7fecf2c4d640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec0ae250 con 0x7fecec0add00 2023-12-01T01:36:08.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.170+0000 7fecf2c4d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec0ad900 con 0x7fecec161620 2023-12-01T01:36:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.170+0000 7fecf144a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fecec0add00 0x7fecec0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.170+0000 7fecf244c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fecec0ce4a0 0x7fecec0c9490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41352/0 (socket says 172.21.15.115:41352) 2023-12-01T01:36:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.170+0000 7fecf244c640 1 -- 172.21.15.115:0/1753953794 learned_addr learned my addr 172.21.15.115:0/1753953794 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.171+0000 7fecf1c4b640 1 --2- 172.21.15.115:0/1753953794 >> v2:172.21.15.142:3300/0 conn(0x7fecec161620 0x7fecec0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.171+0000 7fecf144a640 1 -- 172.21.15.115:0/1753953794 >> v2:172.21.15.142:3300/0 conn(0x7fecec161620 msgr2=0x7fecec0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.171+0000 7fecf144a640 1 --2- 172.21.15.115:0/1753953794 >> v2:172.21.15.142:3300/0 conn(0x7fecec161620 0x7fecec0cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.171+0000 7fecf144a640 1 -- 172.21.15.115:0/1753953794 >> v1:172.21.15.115:6789/0 conn(0x7fecec0ce4a0 legacy=0x7fecec0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.171+0000 7fecf0c49640 1 -- 172.21.15.115:0/1753953794 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687342908 0 0) 0x7fecec161a00 con 0x7fecec0ce4a0 2023-12-01T01:36:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.171+0000 7fecf144a640 1 -- 172.21.15.115:0/1753953794 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0cb1e0 con 0x7fecec0add00 2023-12-01T01:36:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.171+0000 7fecf1c4b640 1 --2- 172.21.15.115:0/1753953794 >> v2:172.21.15.142:3300/0 conn(0x7fecec161620 0x7fecec0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.172+0000 7fecf144a640 1 --2- 172.21.15.115:0/1753953794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0add00 0x7fecec0cdf60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fecec067010 tx=0x7fece002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69778a6aa88cbb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.172+0000 7fecf0c49640 1 -- 172.21.15.115:0/1753953794 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0031710 con 0x7fecec0add00 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.172+0000 7fecf0c49640 1 -- 172.21.15.115:0/1753953794 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece002f050 con 0x7fecec0add00 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.172+0000 7fecf0c49640 1 -- 172.21.15.115:0/1753953794 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0031a90 con 0x7fecec0add00 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.173+0000 7fecf2c4d640 1 -- 172.21.15.115:0/1753953794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0add00 msgr2=0x7fecec0cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.173+0000 7fecf2c4d640 1 --2- 172.21.15.115:0/1753953794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0add00 0x7fecec0cdf60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fecec067010 tx=0x7fece002b7b0 comp rx=0 tx=0).stop 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.173+0000 7fecf2c4d640 1 -- 172.21.15.115:0/1753953794 shutdown_connections 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.173+0000 7fecf2c4d640 1 --2- 172.21.15.115:0/1753953794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0add00 0x7fecec0cdf60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.173+0000 7fecf2c4d640 1 --2- 172.21.15.115:0/1753953794 >> v2:172.21.15.142:3300/0 conn(0x7fecec161620 0x7fecec0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.173+0000 7fecf2c4d640 1 -- 172.21.15.115:0/1753953794 >> 172.21.15.115:0/1753953794 conn(0x7fecec0cb8c0 msgr2=0x7fecec0cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.173+0000 7fecf2c4d640 1 -- 172.21.15.115:0/1753953794 shutdown_connections 2023-12-01T01:36:08.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.173+0000 7fecf2c4d640 1 -- 172.21.15.115:0/1753953794 wait complete. 2023-12-01T01:36:08.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.175+0000 7fecf2c4d640 1 Processor -- start 2023-12-01T01:36:08.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.175+0000 7fecf2c4d640 1 -- start start 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf2c4d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec0add00 0x7fecec15bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf2c4d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0ce4a0 0x7fecec0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf2c4d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec161620 0x7fecec06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf2c4d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec1f5bb0 con 0x7fecec0ce4a0 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf2c4d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b46a0 con 0x7fecec0add00 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf2c4d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec1f31d0 con 0x7fecec161620 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf144a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0ce4a0 0x7fecec0c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf144a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0ce4a0 0x7fecec0c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42626/0 (socket says 172.21.15.115:42626) 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf144a640 1 -- 172.21.15.115:0/2784193447 learned_addr learned my addr 172.21.15.115:0/2784193447 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf1c4b640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec0add00 0x7fecec15bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:08.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf144a640 1 -- 172.21.15.115:0/2784193447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec161620 msgr2=0x7fecec06c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf144a640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec161620 0x7fecec06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf144a640 1 -- 172.21.15.115:0/2784193447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec0add00 msgr2=0x7fecec15bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf144a640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec0add00 0x7fecec15bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf144a640 1 -- 172.21.15.115:0/2784193447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0009400 con 0x7fecec0ce4a0 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.176+0000 7fecf1c4b640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec0add00 0x7fecec15bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.177+0000 7fecf144a640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0ce4a0 0x7fecec0c7510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fece002bdc0 tx=0x7fece00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.177+0000 7fecde7fc640 1 -- 172.21.15.115:0/2784193447 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece00339d0 con 0x7fecec0ce4a0 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.177+0000 7fecde7fc640 1 -- 172.21.15.115:0/2784193447 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece0033b30 con 0x7fecec0ce4a0 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.177+0000 7fecde7fc640 1 -- 172.21.15.115:0/2784193447 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece003c400 con 0x7fecec0ce4a0 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.177+0000 7fecf2c4d640 1 -- 172.21.15.115:0/2784193447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec1f2ef0 con 0x7fecec0ce4a0 2023-12-01T01:36:08.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.177+0000 7fecf2c4d640 1 -- 172.21.15.115:0/2784193447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec0c31e0 con 0x7fecec0ce4a0 2023-12-01T01:36:08.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.178+0000 7fecf2c4d640 1 -- 172.21.15.115:0/2784193447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec0618d0 con 0x7fecec0ce4a0 2023-12-01T01:36:08.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.179+0000 7fecde7fc640 1 -- 172.21.15.115:0/2784193447 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fece003a040 con 0x7fecec0ce4a0 2023-12-01T01:36:08.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.180+0000 7fecde7fc640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fecd006a460 0x7fecd006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:08.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.180+0000 7fecde7fc640 1 -- 172.21.15.115:0/2784193447 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fece00a9be0 con 0x7fecec0ce4a0 2023-12-01T01:36:08.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.180+0000 7fecf1c4b640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fecd006a460 0x7fecd006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:08.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.181+0000 7fecf1c4b640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fecd006a460 0x7fecd006c920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fece8006fd0 tx=0x7fece8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:08.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.184+0000 7fecde7fc640 1 -- 172.21.15.115:0/2784193447 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fece0075480 con 0x7fecec0ce4a0 2023-12-01T01:36:08.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:08.375+0000 7fecf2c4d640 1 -- 172.21.15.115:0/2784193447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fecec0cc5b0 con 0x7fecec0ce4a0 2023-12-01T01:36:09.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.146+0000 7fecde7fc640 1 -- 172.21.15.115:0/2784193447 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fece0078aa0 con 0x7fecec0ce4a0 2023-12-01T01:36:09.148 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T01:36:09.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.151+0000 7fecf2c4d640 1 -- 172.21.15.115:0/2784193447 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fecd006a460 msgr2=0x7fecd006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:09.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.151+0000 7fecf2c4d640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fecd006a460 0x7fecd006c920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fece8006fd0 tx=0x7fece8008040 comp rx=0 tx=0).stop 2023-12-01T01:36:09.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.151+0000 7fecf2c4d640 1 -- 172.21.15.115:0/2784193447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0ce4a0 msgr2=0x7fecec0c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:09.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.151+0000 7fecf2c4d640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0ce4a0 0x7fecec0c7510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fece002bdc0 tx=0x7fece00046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:09.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.151+0000 7fecf2c4d640 1 -- 172.21.15.115:0/2784193447 shutdown_connections 2023-12-01T01:36:09.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.151+0000 7fecf2c4d640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fecd006a460 0x7fecd006c920 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fece8006fd0 tx=0x7fece8008040 comp rx=0 tx=0).stop 2023-12-01T01:36:09.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.151+0000 7fecf2c4d640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fecec161620 0x7fecec06c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:09.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.151+0000 7fecf2c4d640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fecec0ce4a0 0x7fecec0c7510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:09.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.151+0000 7fecf2c4d640 1 --2- 172.21.15.115:0/2784193447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecec0add00 0x7fecec15bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:09.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.151+0000 7fecf2c4d640 1 -- 172.21.15.115:0/2784193447 >> 172.21.15.115:0/2784193447 conn(0x7fecec0cb8c0 msgr2=0x7fecec0af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:09.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.152+0000 7fecf2c4d640 1 -- 172.21.15.115:0/2784193447 shutdown_connections 2023-12-01T01:36:09.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.152+0000 7fecf2c4d640 1 -- 172.21.15.115:0/2784193447 wait complete. 2023-12-01T01:36:09.176 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T01:36:09.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.374+0000 7fd8e5b5c640 1 Processor -- start 2023-12-01T01:36:09.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.374+0000 7fd8e5b5c640 1 -- start start 2023-12-01T01:36:09.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.375+0000 7fd8e5b5c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd8e015ced0 0x7fd8e00b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.376+0000 7fd8e5b5c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd8e00d2750 0x7fd8e00d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.376+0000 7fd8e5b5c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e00d1480 con 0x7fd8e00b4660 2023-12-01T01:36:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.376+0000 7fd8e5b5c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e00d16b0 con 0x7fd8e00d2750 2023-12-01T01:36:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.376+0000 7fd8e5b5c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e00d1990 con 0x7fd8e015ced0 2023-12-01T01:36:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.376+0000 7fd8d7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd8e00d2750 0x7fd8e00d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.376+0000 7fd8d7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd8e00d2750 0x7fd8e00d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42628/0 (socket says 172.21.15.115:42628) 2023-12-01T01:36:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.376+0000 7fd8d7fff640 1 -- 172.21.15.115:0/1831626329 learned_addr learned my addr 172.21.15.115:0/1831626329 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:09.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.376+0000 7fd8df7fe640 1 --2- 172.21.15.115:0/1831626329 >> v2:172.21.15.142:3300/0 conn(0x7fd8e015ced0 0x7fd8e00b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:09.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.377+0000 7fd8d7fff640 1 -- 172.21.15.115:0/1831626329 >> v2:172.21.15.142:3300/0 conn(0x7fd8e015ced0 msgr2=0x7fd8e00b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:09.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.377+0000 7fd8d7fff640 1 --2- 172.21.15.115:0/1831626329 >> v2:172.21.15.142:3300/0 conn(0x7fd8e015ced0 0x7fd8e00b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:09.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.377+0000 7fd8d7fff640 1 -- 172.21.15.115:0/1831626329 >> v1:172.21.15.142:6789/0 conn(0x7fd8e00b4660 legacy=0x7fd8e0069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:09.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.377+0000 7fd8d7fff640 1 -- 172.21.15.115:0/1831626329 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e00d1c70 con 0x7fd8e00d2750 2023-12-01T01:36:09.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.378+0000 7fd8d7fff640 1 --2- 172.21.15.115:0/1831626329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00d2750 0x7fd8e00d0f40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00cf5c0 tx=0x7fd8c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8cca774a5174e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:09.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.378+0000 7fd8deffd640 1 -- 172.21.15.115:0/1831626329 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8031870 con 0x7fd8e00d2750 2023-12-01T01:36:09.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.378+0000 7fd8deffd640 1 -- 172.21.15.115:0/1831626329 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c80319d0 con 0x7fd8e00d2750 2023-12-01T01:36:09.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.378+0000 7fd8deffd640 1 -- 172.21.15.115:0/1831626329 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8031d60 con 0x7fd8e00d2750 2023-12-01T01:36:09.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.378+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/1831626329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00d2750 msgr2=0x7fd8e00d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:09.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.378+0000 7fd8e5b5c640 1 --2- 172.21.15.115:0/1831626329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00d2750 0x7fd8e00d0f40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00cf5c0 tx=0x7fd8c802b910 comp rx=0 tx=0).stop 2023-12-01T01:36:09.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.379+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/1831626329 shutdown_connections 2023-12-01T01:36:09.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.379+0000 7fd8e5b5c640 1 --2- 172.21.15.115:0/1831626329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00d2750 0x7fd8e00d0f40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:09.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.379+0000 7fd8e5b5c640 1 --2- 172.21.15.115:0/1831626329 >> v2:172.21.15.142:3300/0 conn(0x7fd8e015ced0 0x7fd8e00b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:09.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.379+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/1831626329 >> 172.21.15.115:0/1831626329 conn(0x7fd8e0065f90 msgr2=0x7fd8e00cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:09.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.379+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/1831626329 shutdown_connections 2023-12-01T01:36:09.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.379+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/1831626329 wait complete. 2023-12-01T01:36:09.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.380+0000 7fd8e5b5c640 1 Processor -- start 2023-12-01T01:36:09.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.381+0000 7fd8e5b5c640 1 -- start start 2023-12-01T01:36:09.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.381+0000 7fd8e5b5c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00b4660 0x7fd8e00c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:09.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.381+0000 7fd8e5b5c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2750 0x7fd8e015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:09.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.381+0000 7fd8e5b5c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015ced0 0x7fd8e015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:09.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.381+0000 7fd8e5b5c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00b63b0 con 0x7fd8e00b4660 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8e5b5c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00c8140 con 0x7fd8e015ced0 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8e5b5c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00c83c0 con 0x7fd8e00d2750 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8df7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00b4660 0x7fd8e00c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8df7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00b4660 0x7fd8e00c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42638/0 (socket says 172.21.15.115:42638) 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8df7fe640 1 -- 172.21.15.115:0/329192901 learned_addr learned my addr 172.21.15.115:0/329192901 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8dffff640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015ced0 0x7fd8e015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8d7fff640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2750 0x7fd8e015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8df7fe640 1 -- 172.21.15.115:0/329192901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2750 msgr2=0x7fd8e015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8df7fe640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2750 0x7fd8e015aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8df7fe640 1 -- 172.21.15.115:0/329192901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015ced0 msgr2=0x7fd8e015ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8df7fe640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015ced0 0x7fd8e015ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8df7fe640 1 -- 172.21.15.115:0/329192901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c80094d0 con 0x7fd8e00b4660 2023-12-01T01:36:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.382+0000 7fd8dffff640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015ced0 0x7fd8e015ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:09.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.383+0000 7fd8df7fe640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00b4660 0x7fd8e00c5c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000d5f0 tx=0x7fd8d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:09.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.383+0000 7fd8dcff9640 1 -- 172.21.15.115:0/329192901 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00125f0 con 0x7fd8e00b4660 2023-12-01T01:36:09.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.383+0000 7fd8dcff9640 1 -- 172.21.15.115:0/329192901 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d0012750 con 0x7fd8e00b4660 2023-12-01T01:36:09.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.383+0000 7fd8dcff9640 1 -- 172.21.15.115:0/329192901 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0012b00 con 0x7fd8e00b4660 2023-12-01T01:36:09.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.383+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/329192901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e00c86a0 con 0x7fd8e00b4660 2023-12-01T01:36:09.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.383+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/329192901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e00c8c60 con 0x7fd8e00b4660 2023-12-01T01:36:09.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.384+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/329192901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e00d0c50 con 0x7fd8e00b4660 2023-12-01T01:36:09.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.385+0000 7fd8dcff9640 1 -- 172.21.15.115:0/329192901 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd8d0012c60 con 0x7fd8e00b4660 2023-12-01T01:36:09.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.386+0000 7fd8dcff9640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8c006a430 0x7fd8c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:09.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.386+0000 7fd8dcff9640 1 -- 172.21.15.115:0/329192901 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd8d0088ba0 con 0x7fd8e00b4660 2023-12-01T01:36:09.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.386+0000 7fd8d7fff640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8c006a430 0x7fd8c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:09.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.387+0000 7fd8d7fff640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8c006a430 0x7fd8c006c8f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8036000 tx=0x7fd8c803c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:09.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.391+0000 7fd8dcff9640 1 -- 172.21.15.115:0/329192901 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd8d0054370 con 0x7fd8e00b4660 2023-12-01T01:36:09.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:09.582+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/329192901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd8e004e890 con 0x7fd8e00b4660 2023-12-01T01:36:10.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.157+0000 7fd8dcff9640 1 -- 172.21.15.115:0/329192901 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd8d0057990 con 0x7fd8e00b4660 2023-12-01T01:36:10.161 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T01:36:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.167+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/329192901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8c006a430 msgr2=0x7fd8c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.167+0000 7fd8e5b5c640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8c006a430 0x7fd8c006c8f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8036000 tx=0x7fd8c803c040 comp rx=0 tx=0).stop 2023-12-01T01:36:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.167+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/329192901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00b4660 msgr2=0x7fd8e00c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.167+0000 7fd8e5b5c640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00b4660 0x7fd8e00c5c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000d5f0 tx=0x7fd8d000dab0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.168+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/329192901 shutdown_connections 2023-12-01T01:36:10.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.168+0000 7fd8e5b5c640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8c006a430 0x7fd8c006c8f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.168+0000 7fd8e5b5c640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2750 0x7fd8e015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.168+0000 7fd8e5b5c640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015ced0 0x7fd8e015ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.168+0000 7fd8e5b5c640 1 --2- 172.21.15.115:0/329192901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e00b4660 0x7fd8e00c5c40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.168+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/329192901 >> 172.21.15.115:0/329192901 conn(0x7fd8e0065f90 msgr2=0x7fd8e00d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.170+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/329192901 shutdown_connections 2023-12-01T01:36:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.170+0000 7fd8e5b5c640 1 -- 172.21.15.115:0/329192901 wait complete. 2023-12-01T01:36:10.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T01:36:10.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.388+0000 7f6c14ed5640 1 Processor -- start 2023-12-01T01:36:10.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.388+0000 7f6c14ed5640 1 -- start start 2023-12-01T01:36:10.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.389+0000 7f6c14ed5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c10142870 0x7f6c100ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.389+0000 7f6c14ed5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6c100cd010 0x7f6c100b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.389+0000 7f6c14ed5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c100cd470 con 0x7f6c100b1c50 2023-12-01T01:36:10.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.389+0000 7f6c14ed5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c100b3ae0 con 0x7f6c100cd010 2023-12-01T01:36:10.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.389+0000 7f6c14ed5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10142c50 con 0x7f6c10142870 2023-12-01T01:36:10.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.390+0000 7f6c0e59c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c10142870 0x7f6c100ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:10.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.390+0000 7f6c0e59c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c10142870 0x7f6c100ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:40850/0 (socket says 172.21.15.115:40850) 2023-12-01T01:36:10.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.390+0000 7f6c0e59c640 1 -- 172.21.15.115:0/3684199574 learned_addr learned my addr 172.21.15.115:0/3684199574 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:10.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.390+0000 7f6c05d9b640 1 --2- 172.21.15.115:0/3684199574 >> v2:172.21.15.142:3300/0 conn(0x7f6c100cd010 0x7f6c100b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:10.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.391+0000 7f6c0e59c640 1 -- 172.21.15.115:0/3684199574 >> v2:172.21.15.142:3300/0 conn(0x7f6c100cd010 msgr2=0x7f6c100b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:10.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.391+0000 7f6c0e59c640 1 --2- 172.21.15.115:0/3684199574 >> v2:172.21.15.142:3300/0 conn(0x7f6c100cd010 0x7f6c100b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.391+0000 7f6c0e59c640 1 -- 172.21.15.115:0/3684199574 >> v1:172.21.15.142:6789/0 conn(0x7f6c100b1c50 legacy=0x7f6c1015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.391+0000 7f6c0e59c640 1 -- 172.21.15.115:0/3684199574 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c100b20e0 con 0x7f6c10142870 2023-12-01T01:36:10.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.392+0000 7f6c0e59c640 1 --2- 172.21.15.115:0/3684199574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c10142870 0x7f6c100ccad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009870 tx=0x7f6c0002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e085f54f7723b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:10.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.393+0000 7f6c0dd9b640 1 -- 172.21.15.115:0/3684199574 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00030840 con 0x7f6c10142870 2023-12-01T01:36:10.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.393+0000 7f6c0dd9b640 1 -- 172.21.15.115:0/3684199574 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c000309a0 con 0x7f6c10142870 2023-12-01T01:36:10.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.393+0000 7f6c0dd9b640 1 -- 172.21.15.115:0/3684199574 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00030d30 con 0x7f6c10142870 2023-12-01T01:36:10.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.393+0000 7f6c14ed5640 1 -- 172.21.15.115:0/3684199574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c10142870 msgr2=0x7f6c100ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.393+0000 7f6c14ed5640 1 --2- 172.21.15.115:0/3684199574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c10142870 0x7f6c100ccad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009870 tx=0x7f6c0002b980 comp rx=0 tx=0).stop 2023-12-01T01:36:10.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.394+0000 7f6c14ed5640 1 -- 172.21.15.115:0/3684199574 shutdown_connections 2023-12-01T01:36:10.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.394+0000 7f6c14ed5640 1 --2- 172.21.15.115:0/3684199574 >> v2:172.21.15.142:3300/0 conn(0x7f6c100cd010 0x7f6c100b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.394+0000 7f6c14ed5640 1 --2- 172.21.15.115:0/3684199574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c10142870 0x7f6c100ccad0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.394+0000 7f6c14ed5640 1 -- 172.21.15.115:0/3684199574 >> 172.21.15.115:0/3684199574 conn(0x7f6c10065960 msgr2=0x7f6c10065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:10.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.394+0000 7f6c14ed5640 1 -- 172.21.15.115:0/3684199574 shutdown_connections 2023-12-01T01:36:10.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.394+0000 7f6c14ed5640 1 -- 172.21.15.115:0/3684199574 wait complete. 2023-12-01T01:36:10.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.395+0000 7f6c14ed5640 1 Processor -- start 2023-12-01T01:36:10.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.395+0000 7f6c14ed5640 1 -- start start 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.396+0000 7f6c14ed5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c100b1c50 0x7f6c100aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.396+0000 7f6c14ed5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cd010 0x7f6c100c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.396+0000 7f6c14ed5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c10142870 0x7f6c100c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.396+0000 7f6c14ed5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100c40f0 con 0x7f6c100cd010 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.396+0000 7f6c14ed5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100c43d0 con 0x7f6c10142870 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.396+0000 7f6c14ed5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100c46b0 con 0x7f6c100b1c50 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c05d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cd010 0x7f6c100c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c05d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cd010 0x7f6c100c5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42652/0 (socket says 172.21.15.115:42652) 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c05d9b640 1 -- 172.21.15.115:0/2310285017 learned_addr learned my addr 172.21.15.115:0/2310285017 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c0e59c640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c100b1c50 0x7f6c100aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:10.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c0ed9d640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c10142870 0x7f6c100c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c05d9b640 1 -- 172.21.15.115:0/2310285017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c100b1c50 msgr2=0x7f6c100aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c05d9b640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c100b1c50 0x7f6c100aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c05d9b640 1 -- 172.21.15.115:0/2310285017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c10142870 msgr2=0x7f6c100c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c05d9b640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c10142870 0x7f6c100c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c05d9b640 1 -- 172.21.15.115:0/2310285017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c000094d0 con 0x7f6c100cd010 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.397+0000 7f6c0ed9d640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c10142870 0x7f6c100c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.398+0000 7f6c05d9b640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cd010 0x7f6c100c5190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800e9b0 tx=0x7f6bf800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.398+0000 7f6c077fe640 1 -- 172.21.15.115:0/2310285017 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8009670 con 0x7f6c100cd010 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.398+0000 7f6c077fe640 1 -- 172.21.15.115:0/2310285017 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf800c9a0 con 0x7f6c100cd010 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.398+0000 7f6c14ed5640 1 -- 172.21.15.115:0/2310285017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c100c4990 con 0x7f6c100cd010 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.398+0000 7f6c077fe640 1 -- 172.21.15.115:0/2310285017 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800cd50 con 0x7f6c100cd010 2023-12-01T01:36:10.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.398+0000 7f6c14ed5640 1 -- 172.21.15.115:0/2310285017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c100af4c0 con 0x7f6c100cd010 2023-12-01T01:36:10.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.399+0000 7f6c14ed5640 1 -- 172.21.15.115:0/2310285017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1004e890 con 0x7f6c100cd010 2023-12-01T01:36:10.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.401+0000 7f6c077fe640 1 -- 172.21.15.115:0/2310285017 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6bf8013470 con 0x7f6c100cd010 2023-12-01T01:36:10.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.401+0000 7f6c077fe640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a430 0x7f6bf006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.402+0000 7f6c077fe640 1 -- 172.21.15.115:0/2310285017 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6bf8088b50 con 0x7f6c100cd010 2023-12-01T01:36:10.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.402+0000 7f6c0e59c640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a430 0x7f6bf006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:10.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.402+0000 7f6c0e59c640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a430 0x7f6bf006c8f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009870 tx=0x7f6c0003b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:10.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.406+0000 7f6c077fe640 1 -- 172.21.15.115:0/2310285017 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6bf8054190 con 0x7f6c100cd010 2023-12-01T01:36:10.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.635+0000 7f6c14ed5640 1 -- 172.21.15.115:0/2310285017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6c1006b6d0 con 0x7f6c100cd010 2023-12-01T01:36:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.643+0000 7f6c077fe640 1 -- 172.21.15.115:0/2310285017 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6bf80170c0 con 0x7f6c100cd010 2023-12-01T01:36:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 2 and data pool 3 2023-12-01T01:36:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.649+0000 7f6c14ed5640 1 -- 172.21.15.115:0/2310285017 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a430 msgr2=0x7f6bf006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.649+0000 7f6c14ed5640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a430 0x7f6bf006c8f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009870 tx=0x7f6c0003b040 comp rx=0 tx=0).stop 2023-12-01T01:36:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.649+0000 7f6c14ed5640 1 -- 172.21.15.115:0/2310285017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cd010 msgr2=0x7f6c100c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.649+0000 7f6c14ed5640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cd010 0x7f6c100c5190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800e9b0 tx=0x7f6bf800ee70 comp rx=0 tx=0).stop 2023-12-01T01:36:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.651+0000 7f6c14ed5640 1 -- 172.21.15.115:0/2310285017 shutdown_connections 2023-12-01T01:36:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.651+0000 7f6c14ed5640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a430 0x7f6bf006c8f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.651+0000 7f6c14ed5640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c10142870 0x7f6c100c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.651+0000 7f6c14ed5640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cd010 0x7f6c100c5190 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800e9b0 tx=0x7f6bf800ee70 comp rx=0 tx=0).stop 2023-12-01T01:36:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.651+0000 7f6c14ed5640 1 --2- 172.21.15.115:0/2310285017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c100b1c50 0x7f6c100aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.651+0000 7f6c14ed5640 1 -- 172.21.15.115:0/2310285017 >> 172.21.15.115:0/2310285017 conn(0x7f6c10065960 msgr2=0x7f6c100afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.651+0000 7f6c14ed5640 1 -- 172.21.15.115:0/2310285017 shutdown_connections 2023-12-01T01:36:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.651+0000 7f6c14ed5640 1 -- 172.21.15.115:0/2310285017 wait complete. 2023-12-01T01:36:10.686 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.860+0000 7ff391c51640 1 Processor -- start 2023-12-01T01:36:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.860+0000 7ff391c51640 1 -- start start 2023-12-01T01:36:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.862+0000 7ff391c51640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff38c15cd20 0x7ff38c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.862+0000 7ff391c51640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff38c0d1230 0x7ff38c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.862+0000 7ff391c51640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c0d1b90 con 0x7ff38c0b5d80 2023-12-01T01:36:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.862+0000 7ff391c51640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38c0b2160 con 0x7ff38c15cd20 2023-12-01T01:36:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.862+0000 7ff391c51640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38c0b8c80 con 0x7ff38c0d1230 2023-12-01T01:36:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.862+0000 7ff38b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff38c15cd20 0x7ff38c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.862+0000 7ff38b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff38c15cd20 0x7ff38c0d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42656/0 (socket says 172.21.15.115:42656) 2023-12-01T01:36:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.862+0000 7ff38b7fe640 1 -- 172.21.15.115:0/4272814200 learned_addr learned my addr 172.21.15.115:0/4272814200 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.863+0000 7ff38b7fe640 1 -- 172.21.15.115:0/4272814200 >> v2:172.21.15.142:3300/0 conn(0x7ff38c0d1230 msgr2=0x7ff38c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:10.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.863+0000 7ff38b7fe640 1 --2- 172.21.15.115:0/4272814200 >> v2:172.21.15.142:3300/0 conn(0x7ff38c0d1230 0x7ff38c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.863+0000 7ff38b7fe640 1 -- 172.21.15.115:0/4272814200 >> v1:172.21.15.142:6789/0 conn(0x7ff38c0b5d80 legacy=0x7ff38c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:10.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.863+0000 7ff38b7fe640 1 -- 172.21.15.115:0/4272814200 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0ce2e0 con 0x7ff38c15cd20 2023-12-01T01:36:10.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.864+0000 7ff38b7fe640 1 --2- 172.21.15.115:0/4272814200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c15cd20 0x7ff38c0d24d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0c6720 tx=0x7ff37c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ee99ad119c4814 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:10.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.864+0000 7ff38a7fc640 1 -- 172.21.15.115:0/4272814200 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c02c9f0 con 0x7ff38c15cd20 2023-12-01T01:36:10.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.864+0000 7ff38a7fc640 1 -- 172.21.15.115:0/4272814200 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c02cb50 con 0x7ff38c15cd20 2023-12-01T01:36:10.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.864+0000 7ff38a7fc640 1 -- 172.21.15.115:0/4272814200 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c0343f0 con 0x7ff38c15cd20 2023-12-01T01:36:10.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.865+0000 7ff391c51640 1 -- 172.21.15.115:0/4272814200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c15cd20 msgr2=0x7ff38c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.865+0000 7ff391c51640 1 --2- 172.21.15.115:0/4272814200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c15cd20 0x7ff38c0d24d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0c6720 tx=0x7ff37c02be50 comp rx=0 tx=0).stop 2023-12-01T01:36:10.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.865+0000 7ff391c51640 1 -- 172.21.15.115:0/4272814200 shutdown_connections 2023-12-01T01:36:10.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.865+0000 7ff391c51640 1 --2- 172.21.15.115:0/4272814200 >> v2:172.21.15.142:3300/0 conn(0x7ff38c0d1230 0x7ff38c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.865+0000 7ff391c51640 1 --2- 172.21.15.115:0/4272814200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c15cd20 0x7ff38c0d24d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.865+0000 7ff391c51640 1 -- 172.21.15.115:0/4272814200 >> 172.21.15.115:0/4272814200 conn(0x7ff38c06a0e0 msgr2=0x7ff38c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:10.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.865+0000 7ff391c51640 1 -- 172.21.15.115:0/4272814200 shutdown_connections 2023-12-01T01:36:10.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.865+0000 7ff391c51640 1 -- 172.21.15.115:0/4272814200 wait complete. 2023-12-01T01:36:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.867+0000 7ff391c51640 1 Processor -- start 2023-12-01T01:36:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.867+0000 7ff391c51640 1 -- start start 2023-12-01T01:36:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.868+0000 7ff391c51640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c0b5d80 0x7ff38c152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.868+0000 7ff391c51640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff38c0d1230 0x7ff38c0cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.868+0000 7ff391c51640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff38c15cd20 0x7ff38c153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.868+0000 7ff391c51640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0ccb30 con 0x7ff38c0b5d80 2023-12-01T01:36:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.868+0000 7ff391c51640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0cb550 con 0x7ff38c0d1230 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.868+0000 7ff391c51640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0cb830 con 0x7ff38c15cd20 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.868+0000 7ff38b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c0b5d80 0x7ff38c152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.868+0000 7ff38b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c0b5d80 0x7ff38c152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42660/0 (socket says 172.21.15.115:42660) 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.868+0000 7ff38b7fe640 1 -- 172.21.15.115:0/2010975221 learned_addr learned my addr 172.21.15.115:0/2010975221 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff38b7fe640 1 -- 172.21.15.115:0/2010975221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff38c15cd20 msgr2=0x7ff38c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff38b7fe640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff38c15cd20 0x7ff38c153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff38b7fe640 1 -- 172.21.15.115:0/2010975221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff38c0d1230 msgr2=0x7ff38c0cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff38b7fe640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff38c0d1230 0x7ff38c0cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff38b7fe640 1 -- 172.21.15.115:0/2010975221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c009b30 con 0x7ff38c0b5d80 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff38b7fe640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c0b5d80 0x7ff38c152270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0093e0 tx=0x7ff37c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff383fff640 1 -- 172.21.15.115:0/2010975221 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c03e450 con 0x7ff38c0b5d80 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff383fff640 1 -- 172.21.15.115:0/2010975221 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c03e5b0 con 0x7ff38c0b5d80 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff383fff640 1 -- 172.21.15.115:0/2010975221 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c03e960 con 0x7ff38c0b5d80 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff391c51640 1 -- 172.21.15.115:0/2010975221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c0cbb10 con 0x7ff38c0b5d80 2023-12-01T01:36:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.869+0000 7ff391c51640 1 -- 172.21.15.115:0/2010975221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c0cc0d0 con 0x7ff38c0b5d80 2023-12-01T01:36:10.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.871+0000 7ff391c51640 1 -- 172.21.15.115:0/2010975221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c04e890 con 0x7ff38c0b5d80 2023-12-01T01:36:10.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.871+0000 7ff383fff640 1 -- 172.21.15.115:0/2010975221 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff37c03eac0 con 0x7ff38c0b5d80 2023-12-01T01:36:10.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.872+0000 7ff383fff640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff36c06a460 0x7ff36c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:10.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.872+0000 7ff383fff640 1 -- 172.21.15.115:0/2010975221 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff37c0ab3a0 con 0x7ff38c0b5d80 2023-12-01T01:36:10.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.873+0000 7ff38affd640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff36c06a460 0x7ff36c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:10.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.873+0000 7ff38affd640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff36c06a460 0x7ff36c06c920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff38c06abd0 tx=0x7ff374007990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:10.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:10.877+0000 7ff383fff640 1 -- 172.21.15.115:0/2010975221 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff37c076b40 con 0x7ff38c0b5d80 2023-12-01T01:36:11.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.059+0000 7ff391c51640 1 -- 172.21.15.115:0/2010975221 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff38c0d1b90 con 0x7ff38c0b5d80 2023-12-01T01:36:11.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.061+0000 7ff383fff640 1 -- 172.21.15.115:0/2010975221 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16102 (secure 0 0 0) 0x7ff38c0d1b90 con 0x7ff38c0b5d80 2023-12-01T01:36:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:11.063 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":20,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:10.637069+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:11.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 -- 172.21.15.115:0/2010975221 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff36c06a460 msgr2=0x7ff36c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff36c06a460 0x7ff36c06c920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff38c06abd0 tx=0x7ff374007990 comp rx=0 tx=0).stop 2023-12-01T01:36:11.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 -- 172.21.15.115:0/2010975221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c0b5d80 msgr2=0x7ff38c152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c0b5d80 0x7ff38c152270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0093e0 tx=0x7ff37c004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:11.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 -- 172.21.15.115:0/2010975221 shutdown_connections 2023-12-01T01:36:11.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff36c06a460 0x7ff36c06c920 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff38c15cd20 0x7ff38c153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff38c0d1230 0x7ff38c0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 --2- 172.21.15.115:0/2010975221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff38c0b5d80 0x7ff38c152270 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 -- 172.21.15.115:0/2010975221 >> 172.21.15.115:0/2010975221 conn(0x7ff38c06a0e0 msgr2=0x7ff38c0b7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:11.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 -- 172.21.15.115:0/2010975221 shutdown_connections 2023-12-01T01:36:11.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.064+0000 7ff391c51640 1 -- 172.21.15.115:0/2010975221 wait complete. 2023-12-01T01:36:11.082 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:11.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.286+0000 7f84b95e0640 1 Processor -- start 2023-12-01T01:36:11.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.286+0000 7f84b95e0640 1 -- start start 2023-12-01T01:36:11.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.288+0000 7f84b95e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c9ef0 0x7f84b406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.288+0000 7f84b95e0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f84b406b6f0 0x7f84b4070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.288+0000 7f84b95e0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b406f8b0 con 0x7f84b4159dd0 2023-12-01T01:36:11.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.288+0000 7f84b95e0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b40adcb0 con 0x7f84b406b6f0 2023-12-01T01:36:11.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.288+0000 7f84b95e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40ad9d0 con 0x7f84b40c9ef0 2023-12-01T01:36:11.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.288+0000 7f84b2ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f84b4159dd0 0x7f84b40c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41354/0 (socket says 172.21.15.115:41354) 2023-12-01T01:36:11.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.288+0000 7f84b2ffd640 1 -- 172.21.15.115:0/325066187 learned_addr learned my addr 172.21.15.115:0/325066187 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:11.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610639781 0 0) 0x7f84b406f8b0 con 0x7f84b4159dd0 2023-12-01T01:36:11.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84a0003520 con 0x7f84b4159dd0 2023-12-01T01:36:11.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84aa7fc640 1 --2- 172.21.15.115:0/325066187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c9ef0 0x7f84b406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:11.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b37fe640 1 --2- 172.21.15.115:0/325066187 >> v2:172.21.15.142:3300/0 conn(0x7f84b406b6f0 0x7f84b4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:11.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3830514363 0 0) 0x7f84a0003520 con 0x7f84b4159dd0 2023-12-01T01:36:11.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c9ef0 msgr2=0x7f84b406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 --2- 172.21.15.115:0/325066187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c9ef0 0x7f84b406fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 >> v2:172.21.15.142:3300/0 conn(0x7f84b406b6f0 msgr2=0x7f84b4070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 --2- 172.21.15.115:0/325066187 >> v2:172.21.15.142:3300/0 conn(0x7f84b406b6f0 0x7f84b4070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40c9000 con 0x7f84b4159dd0 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f84a40019f0 con 0x7f84b4159dd0 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b37fe640 1 --2- 172.21.15.115:0/325066187 >> v2:172.21.15.142:3300/0 conn(0x7f84b406b6f0 0x7f84b4070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 >> v1:172.21.15.115:6789/0 conn(0x7f84b4159dd0 legacy=0x7f84b40c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 --2- 172.21.15.115:0/325066187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 0x7f84a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 --2- 172.21.15.115:0/325066187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a00047c0 0x7f84a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 --2- 172.21.15.115:0/325066187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c9ef0 0x7f84a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40adcb0 con 0x7f84a00039e0 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40ad9d0 con 0x7f84a00047c0 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a0005530 con 0x7f84b40c9ef0 2023-12-01T01:36:11.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.289+0000 7f84b2ffd640 1 --2- 172.21.15.115:0/325066187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 0x7f84a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:11.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.290+0000 7f84b2ffd640 1 -- 172.21.15.115:0/325066187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c9ef0 msgr2=0x7f84a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:11.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.290+0000 7f84b2ffd640 1 --2- 172.21.15.115:0/325066187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c9ef0 0x7f84a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.290+0000 7f84b2ffd640 1 -- 172.21.15.115:0/325066187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a00047c0 msgr2=0x7f84a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:11.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.290+0000 7f84b2ffd640 1 --2- 172.21.15.115:0/325066187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84a00047c0 0x7f84a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.290+0000 7f84b2ffd640 1 -- 172.21.15.115:0/325066187 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a0003520 con 0x7f84a00039e0 2023-12-01T01:36:11.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.291+0000 7f84b2ffd640 1 --2- 172.21.15.115:0/325066187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 0x7f84a0003e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f84a400a780 tx=0x7f84a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e199033002e0356b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:11.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.291+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4034630 con 0x7f84a00039e0 2023-12-01T01:36:11.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.291+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a402f070 con 0x7f84a00039e0 2023-12-01T01:36:11.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.291+0000 7f84b27fc640 1 -- 172.21.15.115:0/325066187 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40349d0 con 0x7f84a00039e0 2023-12-01T01:36:11.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.291+0000 7f84b95e0640 1 -- 172.21.15.115:0/325066187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 msgr2=0x7f84a0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.291+0000 7f84b95e0640 1 --2- 172.21.15.115:0/325066187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 0x7f84a0003e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f84a400a780 tx=0x7f84a00057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.291+0000 7f84b37fe640 1 -- 172.21.15.115:0/325066187 reap_dead start 2023-12-01T01:36:11.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.292+0000 7f84b95e0640 1 -- 172.21.15.115:0/325066187 shutdown_connections 2023-12-01T01:36:11.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.292+0000 7f84b95e0640 1 -- 172.21.15.115:0/325066187 >> 172.21.15.115:0/325066187 conn(0x7f84b40ce880 msgr2=0x7f84b40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:11.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.292+0000 7f84b95e0640 1 -- 172.21.15.115:0/325066187 shutdown_connections 2023-12-01T01:36:11.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.292+0000 7f84b95e0640 1 -- 172.21.15.115:0/325066187 wait complete. 2023-12-01T01:36:11.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.293+0000 7f84b95e0640 1 Processor -- start 2023-12-01T01:36:11.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.293+0000 7f84b95e0640 1 -- start start 2023-12-01T01:36:11.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.294+0000 7f84b95e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 0x7f84b4147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.294+0000 7f84b95e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b41448c0 0x7f84b406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.294+0000 7f84b95e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c5950 0x7f84b40c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.294+0000 7f84b95e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40c6310 con 0x7f84a00039e0 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.294+0000 7f84b95e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4144d40 con 0x7f84b41448c0 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.294+0000 7f84b95e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4070100 con 0x7f84b40c5950 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.294+0000 7f84b2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 0x7f84b4147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84b2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 0x7f84b4147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42688/0 (socket says 172.21.15.115:42688) 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84b2ffd640 1 -- 172.21.15.115:0/2544380108 learned_addr learned my addr 172.21.15.115:0/2544380108 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84b2ffd640 1 -- 172.21.15.115:0/2544380108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c5950 msgr2=0x7f84b40c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84b2ffd640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c5950 0x7f84b40c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84b2ffd640 1 -- 172.21.15.115:0/2544380108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b41448c0 msgr2=0x7f84b406fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84b2ffd640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b41448c0 0x7f84b406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84b2ffd640 1 -- 172.21.15.115:0/2544380108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40703d0 con 0x7f84a00039e0 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84b2ffd640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 0x7f84b4147c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84a402cad0 tx=0x7f84a4001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84abfff640 1 -- 172.21.15.115:0/2544380108 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4034980 con 0x7f84a00039e0 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84abfff640 1 -- 172.21.15.115:0/2544380108 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a4034ae0 con 0x7f84a00039e0 2023-12-01T01:36:11.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84abfff640 1 -- 172.21.15.115:0/2544380108 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4034e70 con 0x7f84a00039e0 2023-12-01T01:36:11.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.295+0000 7f84b95e0640 1 -- 172.21.15.115:0/2544380108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b40706b0 con 0x7f84a00039e0 2023-12-01T01:36:11.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.296+0000 7f84b95e0640 1 -- 172.21.15.115:0/2544380108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b4070c70 con 0x7f84a00039e0 2023-12-01T01:36:11.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.297+0000 7f84b95e0640 1 -- 172.21.15.115:0/2544380108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b404e890 con 0x7f84a00039e0 2023-12-01T01:36:11.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.297+0000 7f84abfff640 1 -- 172.21.15.115:0/2544380108 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f84a4031030 con 0x7f84a00039e0 2023-12-01T01:36:11.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.298+0000 7f84abfff640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a460 0x7f849406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.298+0000 7f84abfff640 1 -- 172.21.15.115:0/2544380108 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84a40aa1e0 con 0x7f84a00039e0 2023-12-01T01:36:11.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.299+0000 7f84aa7fc640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a460 0x7f849406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:11.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.299+0000 7f84aa7fc640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a460 0x7f849406c920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f849c006f00 tx=0x7f849c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:11.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.302+0000 7f84abfff640 1 -- 172.21.15.115:0/2544380108 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f84a40757d0 con 0x7f84a00039e0 2023-12-01T01:36:11.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.479+0000 7f84b95e0640 1 -- 172.21.15.115:0/2544380108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84b40ca650 con 0x7f84a00039e0 2023-12-01T01:36:11.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.482+0000 7f84abfff640 1 -- 172.21.15.115:0/2544380108 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16102 (secure 0 0 0) 0x7f84a4078df0 con 0x7f84a00039e0 2023-12-01T01:36:11.486 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:11.486 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":20,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:10.637069+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:11.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 -- 172.21.15.115:0/2544380108 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a460 msgr2=0x7f849406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a460 0x7f849406c920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f849c006f00 tx=0x7f849c008040 comp rx=0 tx=0).stop 2023-12-01T01:36:11.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 -- 172.21.15.115:0/2544380108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 msgr2=0x7f84b4147c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 0x7f84b4147c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84a402cad0 tx=0x7f84a4001360 comp rx=0 tx=0).stop 2023-12-01T01:36:11.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 -- 172.21.15.115:0/2544380108 shutdown_connections 2023-12-01T01:36:11.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a460 0x7f849406c920 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40c5950 0x7f84b40c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b41448c0 0x7f84b406fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 --2- 172.21.15.115:0/2544380108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a00039e0 0x7f84b4147c80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 -- 172.21.15.115:0/2544380108 >> 172.21.15.115:0/2544380108 conn(0x7f84b40ce880 msgr2=0x7f84b406bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:11.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 -- 172.21.15.115:0/2544380108 shutdown_connections 2023-12-01T01:36:11.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.485+0000 7f84b95e0640 1 -- 172.21.15.115:0/2544380108 wait complete. 2023-12-01T01:36:11.505 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T01:36:11.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.705+0000 7fe62a45f640 1 Processor -- start 2023-12-01T01:36:11.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.705+0000 7fe62a45f640 1 -- start start 2023-12-01T01:36:11.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.707+0000 7fe62a45f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe624142870 0x7fe6240ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.707+0000 7fe62a45f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe6240cd010 0x7fe6240b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.707+0000 7fe62a45f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6240cd470 con 0x7fe6240b1c50 2023-12-01T01:36:11.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.707+0000 7fe62a45f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6240b3ae0 con 0x7fe6240cd010 2023-12-01T01:36:11.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.707+0000 7fe62a45f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe624142c50 con 0x7fe624142870 2023-12-01T01:36:11.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.707+0000 7fe6237fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe6240cd010 0x7fe6240b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:11.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.707+0000 7fe6237fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe6240cd010 0x7fe6240b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42702/0 (socket says 172.21.15.115:42702) 2023-12-01T01:36:11.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.707+0000 7fe6237fe640 1 -- 172.21.15.115:0/4040079239 learned_addr learned my addr 172.21.15.115:0/4040079239 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:11.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.707+0000 7fe623fff640 1 --2- 172.21.15.115:0/4040079239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe624142870 0x7fe6240ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:11.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.708+0000 7fe6237fe640 1 -- 172.21.15.115:0/4040079239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe624142870 msgr2=0x7fe6240ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:11.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.708+0000 7fe6237fe640 1 --2- 172.21.15.115:0/4040079239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe624142870 0x7fe6240ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.708+0000 7fe6237fe640 1 -- 172.21.15.115:0/4040079239 >> v1:172.21.15.142:6789/0 conn(0x7fe6240b1c50 legacy=0x7fe62415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:11.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.708+0000 7fe6237fe640 1 -- 172.21.15.115:0/4040079239 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6240b20e0 con 0x7fe6240cd010 2023-12-01T01:36:11.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.708+0000 7fe623fff640 1 --2- 172.21.15.115:0/4040079239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe624142870 0x7fe6240ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:11.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.709+0000 7fe6237fe640 1 --2- 172.21.15.115:0/4040079239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 0x7fe6240b1710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe61800bb40 tx=0x7fe61802e0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeb90eb65442ecf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:11.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.709+0000 7fe622ffd640 1 -- 172.21.15.115:0/4040079239 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61802ee60 con 0x7fe6240cd010 2023-12-01T01:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.709+0000 7fe622ffd640 1 -- 172.21.15.115:0/4040079239 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe618036440 con 0x7fe6240cd010 2023-12-01T01:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.709+0000 7fe622ffd640 1 -- 172.21.15.115:0/4040079239 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6180367d0 con 0x7fe6240cd010 2023-12-01T01:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.709+0000 7fe62a45f640 1 -- 172.21.15.115:0/4040079239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 msgr2=0x7fe6240b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.709+0000 7fe62a45f640 1 --2- 172.21.15.115:0/4040079239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 0x7fe6240b1710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe61800bb40 tx=0x7fe61802e0b0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.710+0000 7fe62a45f640 1 -- 172.21.15.115:0/4040079239 shutdown_connections 2023-12-01T01:36:11.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.710+0000 7fe62a45f640 1 --2- 172.21.15.115:0/4040079239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 0x7fe6240b1710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.710+0000 7fe62a45f640 1 --2- 172.21.15.115:0/4040079239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe624142870 0x7fe6240ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.710+0000 7fe62a45f640 1 -- 172.21.15.115:0/4040079239 >> 172.21.15.115:0/4040079239 conn(0x7fe624065960 msgr2=0x7fe624065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:11.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.710+0000 7fe62a45f640 1 -- 172.21.15.115:0/4040079239 shutdown_connections 2023-12-01T01:36:11.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.710+0000 7fe62a45f640 1 -- 172.21.15.115:0/4040079239 wait complete. 2023-12-01T01:36:11.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.711+0000 7fe62a45f640 1 Processor -- start 2023-12-01T01:36:11.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.712+0000 7fe62a45f640 1 -- start start 2023-12-01T01:36:11.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.712+0000 7fe62a45f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6240b1c50 0x7fe6241333d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.712+0000 7fe62a45f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 0x7fe62412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.712+0000 7fe62a45f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe624142870 0x7fe6240ac660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.712+0000 7fe62a45f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624130500 con 0x7fe6240cd010 2023-12-01T01:36:11.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.712+0000 7fe62a45f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624130770 con 0x7fe6240b1c50 2023-12-01T01:36:11.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.712+0000 7fe62a45f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240acba0 con 0x7fe624142870 2023-12-01T01:36:11.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.713+0000 7fe6237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 0x7fe62412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:11.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.713+0000 7fe6237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 0x7fe62412ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42704/0 (socket says 172.21.15.115:42704) 2023-12-01T01:36:11.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.713+0000 7fe6237fe640 1 -- 172.21.15.115:0/489302748 learned_addr learned my addr 172.21.15.115:0/489302748 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:11.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.713+0000 7fe6237fe640 1 -- 172.21.15.115:0/489302748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe624142870 msgr2=0x7fe6240ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.713+0000 7fe6237fe640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe624142870 0x7fe6240ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.713+0000 7fe6237fe640 1 -- 172.21.15.115:0/489302748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6240b1c50 msgr2=0x7fe6241333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:11.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.713+0000 7fe6237fe640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6240b1c50 0x7fe6241333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:11.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.713+0000 7fe6237fe640 1 -- 172.21.15.115:0/489302748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61800b770 con 0x7fe6240cd010 2023-12-01T01:36:11.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.713+0000 7fe6237fe640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 0x7fe62412ffc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe61800b2c0 tx=0x7fe618004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:11.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.714+0000 7fe620ff9640 1 -- 172.21.15.115:0/489302748 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe618034b80 con 0x7fe6240cd010 2023-12-01T01:36:11.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.714+0000 7fe62a45f640 1 -- 172.21.15.115:0/489302748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6240ace40 con 0x7fe6240cd010 2023-12-01T01:36:11.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.714+0000 7fe620ff9640 1 -- 172.21.15.115:0/489302748 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe618034d00 con 0x7fe6240cd010 2023-12-01T01:36:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.714+0000 7fe620ff9640 1 -- 172.21.15.115:0/489302748 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6180416c0 con 0x7fe6240cd010 2023-12-01T01:36:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.714+0000 7fe62a45f640 1 -- 172.21.15.115:0/489302748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6240ad400 con 0x7fe6240cd010 2023-12-01T01:36:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.715+0000 7fe62a45f640 1 -- 172.21.15.115:0/489302748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62404e890 con 0x7fe6240cd010 2023-12-01T01:36:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.716+0000 7fe620ff9640 1 -- 172.21.15.115:0/489302748 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe618041820 con 0x7fe6240cd010 2023-12-01T01:36:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.717+0000 7fe620ff9640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe61006a430 0x7fe61006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.717+0000 7fe620ff9640 1 -- 172.21.15.115:0/489302748 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6180abb20 con 0x7fe6240cd010 2023-12-01T01:36:11.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.717+0000 7fe623fff640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe61006a430 0x7fe61006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:11.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.717+0000 7fe623fff640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe61006a430 0x7fe61006c8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe614006fd0 tx=0x7fe614008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:11.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.721+0000 7fe620ff9640 1 -- 172.21.15.115:0/489302748 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe618077080 con 0x7fe6240cd010 2023-12-01T01:36:11.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:11.936+0000 7fe62a45f640 1 -- 172.21.15.115:0/489302748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe6240b20e0 con 0x7fe6240cd010 2023-12-01T01:36:12.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.658+0000 7fe620ff9640 1 -- 172.21.15.115:0/489302748 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe6240b20e0 con 0x7fe6240cd010 2023-12-01T01:36:12.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 -- 172.21.15.115:0/489302748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe61006a430 msgr2=0x7fe61006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:12.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe61006a430 0x7fe61006c8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe614006fd0 tx=0x7fe614008040 comp rx=0 tx=0).stop 2023-12-01T01:36:12.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 -- 172.21.15.115:0/489302748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 msgr2=0x7fe62412ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:12.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 0x7fe62412ffc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe61800b2c0 tx=0x7fe618004730 comp rx=0 tx=0).stop 2023-12-01T01:36:12.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 -- 172.21.15.115:0/489302748 shutdown_connections 2023-12-01T01:36:12.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe61006a430 0x7fe61006c8f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:12.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe624142870 0x7fe6240ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:12.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6240cd010 0x7fe62412ffc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:12.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 --2- 172.21.15.115:0/489302748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6240b1c50 0x7fe6241333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:12.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 -- 172.21.15.115:0/489302748 >> 172.21.15.115:0/489302748 conn(0x7fe624065960 msgr2=0x7fe62413f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:12.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 -- 172.21.15.115:0/489302748 shutdown_connections 2023-12-01T01:36:12.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.662+0000 7fe62a45f640 1 -- 172.21.15.115:0/489302748 wait complete. 2023-12-01T01:36:12.683 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T01:36:12.683 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T01:36:12.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.898+0000 7f2758dca640 1 Processor -- start 2023-12-01T01:36:12.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.898+0000 7f2758dca640 1 -- start start 2023-12-01T01:36:12.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.899+0000 7f2758dca640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f27540ca2d0 0x7f2754167970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:12.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.899+0000 7f2758dca640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27540ca700 con 0x7f275415cb60 2023-12-01T01:36:12.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.900+0000 7f2758dca640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2754070000 con 0x7f27540cb600 2023-12-01T01:36:12.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.900+0000 7f2758dca640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27540702e0 con 0x7f27540ca2d0 2023-12-01T01:36:12.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.900+0000 7f27537fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f275415cb60 0x7f27540caef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41358/0 (socket says 172.21.15.115:41358) 2023-12-01T01:36:12.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.900+0000 7f27537fe640 1 -- 172.21.15.115:0/3604033189 learned_addr learned my addr 172.21.15.115:0/3604033189 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:12.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.900+0000 7f2753fff640 1 --2- 172.21.15.115:0/3604033189 >> v2:172.21.15.115:3300/0 conn(0x7f27540ca2d0 0x7f2754167970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:12.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.900+0000 7f2752ffd640 1 -- 172.21.15.115:0/3604033189 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877487352 0 0) 0x7f27540ca700 con 0x7f275415cb60 2023-12-01T01:36:12.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.900+0000 7f2752ffd640 1 -- 172.21.15.115:0/3604033189 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2740003520 con 0x7f275415cb60 2023-12-01T01:36:12.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.900+0000 7f2753fff640 1 -- 172.21.15.115:0/3604033189 >> v1:172.21.15.142:6789/0 conn(0x7f27540cb600 legacy=0x7f27540c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:12.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.900+0000 7f2753fff640 1 -- 172.21.15.115:0/3604033189 >> v1:172.21.15.115:6789/0 conn(0x7f275415cb60 legacy=0x7f27540caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:12.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.900+0000 7f2753fff640 1 -- 172.21.15.115:0/3604033189 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27540705c0 con 0x7f27540ca2d0 2023-12-01T01:36:12.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.902+0000 7f2753fff640 1 --2- 172.21.15.115:0/3604033189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27540ca2d0 0x7f2754167970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f274c009d90 tx=0x7f274c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e336d322bfa15f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:12.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.902+0000 7f2752ffd640 1 -- 172.21.15.115:0/3604033189 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c02cc60 con 0x7f27540ca2d0 2023-12-01T01:36:12.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.902+0000 7f2752ffd640 1 -- 172.21.15.115:0/3604033189 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274c02cdc0 con 0x7f27540ca2d0 2023-12-01T01:36:12.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.902+0000 7f2752ffd640 1 -- 172.21.15.115:0/3604033189 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c034660 con 0x7f27540ca2d0 2023-12-01T01:36:12.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.902+0000 7f2758dca640 1 -- 172.21.15.115:0/3604033189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27540ca2d0 msgr2=0x7f2754167970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:12.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.902+0000 7f2758dca640 1 --2- 172.21.15.115:0/3604033189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27540ca2d0 0x7f2754167970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f274c009d90 tx=0x7f274c02beb0 comp rx=0 tx=0).stop 2023-12-01T01:36:12.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.902+0000 7f2758dca640 1 -- 172.21.15.115:0/3604033189 shutdown_connections 2023-12-01T01:36:12.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.902+0000 7f2758dca640 1 --2- 172.21.15.115:0/3604033189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27540ca2d0 0x7f2754167970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:12.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.903+0000 7f2758dca640 1 -- 172.21.15.115:0/3604033189 >> 172.21.15.115:0/3604033189 conn(0x7f2754065960 msgr2=0x7f2754065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.903+0000 7f2758dca640 1 -- 172.21.15.115:0/3604033189 shutdown_connections 2023-12-01T01:36:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.903+0000 7f2758dca640 1 -- 172.21.15.115:0/3604033189 wait complete. 2023-12-01T01:36:12.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.904+0000 7f2758dca640 1 Processor -- start 2023-12-01T01:36:12.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.905+0000 7f2758dca640 1 -- start start 2023-12-01T01:36:12.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.905+0000 7f2758dca640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27540ca2d0 0x7f2754156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:12.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.905+0000 7f2758dca640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27540cb600 0x7f27541f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:12.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2758dca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f275415cb60 0x7f27541455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:12.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2758dca640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275415f330 con 0x7f275415cb60 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2758dca640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275415ea40 con 0x7f27540cb600 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2758dca640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275415ece0 con 0x7f27540ca2d0 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2753fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f275415cb60 0x7f27541455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2753fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f275415cb60 0x7f27541455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42730/0 (socket says 172.21.15.115:42730) 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2753fff640 1 -- 172.21.15.115:0/3574013098 learned_addr learned my addr 172.21.15.115:0/3574013098 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f274bfff640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27540cb600 0x7f27541f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2753fff640 1 -- 172.21.15.115:0/3574013098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27540ca2d0 msgr2=0x7f2754156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2753fff640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27540ca2d0 0x7f2754156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2753fff640 1 -- 172.21.15.115:0/3574013098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27540cb600 msgr2=0x7f27541f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2753fff640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27540cb600 0x7f27541f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2753fff640 1 -- 172.21.15.115:0/3574013098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c009a70 con 0x7f275415cb60 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f274bfff640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27540cb600 0x7f27541f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.906+0000 7f2753fff640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f275415cb60 0x7f27541455f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f274c009d90 tx=0x7f274c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.907+0000 7f2750ff9640 1 -- 172.21.15.115:0/3574013098 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c040650 con 0x7f275415cb60 2023-12-01T01:36:12.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.907+0000 7f2750ff9640 1 -- 172.21.15.115:0/3574013098 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274c0407b0 con 0x7f275415cb60 2023-12-01T01:36:12.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.907+0000 7f2750ff9640 1 -- 172.21.15.115:0/3574013098 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c040b60 con 0x7f275415cb60 2023-12-01T01:36:12.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.907+0000 7f2758dca640 1 -- 172.21.15.115:0/3574013098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275415efc0 con 0x7f275415cb60 2023-12-01T01:36:12.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.907+0000 7f2758dca640 1 -- 172.21.15.115:0/3574013098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2754157180 con 0x7f275415cb60 2023-12-01T01:36:12.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.908+0000 7f2758dca640 1 -- 172.21.15.115:0/3574013098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27540618d0 con 0x7f275415cb60 2023-12-01T01:36:12.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.909+0000 7f2750ff9640 1 -- 172.21.15.115:0/3574013098 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f274c040cc0 con 0x7f275415cb60 2023-12-01T01:36:12.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.910+0000 7f2750ff9640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f273406a430 0x7f273406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:12.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.910+0000 7f2750ff9640 1 -- 172.21.15.115:0/3574013098 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f274c0ab640 con 0x7f275415cb60 2023-12-01T01:36:12.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.910+0000 7f27537fe640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f273406a430 0x7f273406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:12.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.911+0000 7f27537fe640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f273406a430 0x7f273406c8f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2744001570 tx=0x7f2744007930 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:12.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:12.915+0000 7f2750ff9640 1 -- 172.21.15.115:0/3574013098 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f274c076d80 con 0x7f275415cb60 2023-12-01T01:36:13.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.144+0000 7f2758dca640 1 -- 172.21.15.115:0/3574013098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f27540b46d0 con 0x7f275415cb60 2023-12-01T01:36:13.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.669+0000 7f2750ff9640 1 -- 172.21.15.115:0/3574013098 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f274c009450 con 0x7f275415cb60 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.672+0000 7f2758dca640 1 -- 172.21.15.115:0/3574013098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f273406a430 msgr2=0x7f273406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.672+0000 7f2758dca640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f273406a430 0x7f273406c8f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2744001570 tx=0x7f2744007930 comp rx=0 tx=0).stop 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.673+0000 7f2758dca640 1 -- 172.21.15.115:0/3574013098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f275415cb60 msgr2=0x7f27541455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.673+0000 7f2758dca640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f275415cb60 0x7f27541455f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f274c009d90 tx=0x7f274c004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.673+0000 7f2758dca640 1 -- 172.21.15.115:0/3574013098 shutdown_connections 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.673+0000 7f2758dca640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f273406a430 0x7f273406c8f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.673+0000 7f2758dca640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f275415cb60 0x7f27541455f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.673+0000 7f2758dca640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27540cb600 0x7f27541f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.673+0000 7f2758dca640 1 --2- 172.21.15.115:0/3574013098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27540ca2d0 0x7f2754156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.673+0000 7f2758dca640 1 -- 172.21.15.115:0/3574013098 >> 172.21.15.115:0/3574013098 conn(0x7f2754065960 msgr2=0x7f2754155660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.673+0000 7f2758dca640 1 -- 172.21.15.115:0/3574013098 shutdown_connections 2023-12-01T01:36:13.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.673+0000 7f2758dca640 1 -- 172.21.15.115:0/3574013098 wait complete. 2023-12-01T01:36:13.695 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T01:36:13.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:36:13.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.906+0000 7f34c55ab640 1 Processor -- start 2023-12-01T01:36:13.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.907+0000 7f34c55ab640 1 -- start start 2023-12-01T01:36:13.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.908+0000 7f34c55ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c00b5d80 0x7f34c00d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:13.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.908+0000 7f34c55ab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f34c00d1230 0x7f34c00d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:13.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.908+0000 7f34c55ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c00ce2e0 con 0x7f34c015cd20 2023-12-01T01:36:13.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.908+0000 7f34c55ab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c00b2160 con 0x7f34c00d1230 2023-12-01T01:36:13.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.908+0000 7f34c55ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c00d1be0 con 0x7f34c00b5d80 2023-12-01T01:36:13.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.908+0000 7f34bf7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f34c00d1230 0x7f34c00d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:13.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.908+0000 7f34bf7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f34c00d1230 0x7f34c00d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42742/0 (socket says 172.21.15.115:42742) 2023-12-01T01:36:13.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.908+0000 7f34bf7fe640 1 -- 172.21.15.115:0/1854487995 learned_addr learned my addr 172.21.15.115:0/1854487995 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:13.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.909+0000 7f34bf7fe640 1 -- 172.21.15.115:0/1854487995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c00b5d80 msgr2=0x7f34c00d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:13.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.909+0000 7f34bf7fe640 1 --2- 172.21.15.115:0/1854487995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c00b5d80 0x7f34c00d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:13.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.909+0000 7f34bf7fe640 1 -- 172.21.15.115:0/1854487995 >> v1:172.21.15.142:6789/0 conn(0x7f34c015cd20 legacy=0x7f34c00b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:13.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.909+0000 7f34bf7fe640 1 -- 172.21.15.115:0/1854487995 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c00ce000 con 0x7f34c00d1230 2023-12-01T01:36:13.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.910+0000 7f34bf7fe640 1 --2- 172.21.15.115:0/1854487995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 0x7f34c00d16a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009870 tx=0x7f34b802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71264b134dc325 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:13.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.910+0000 7f34bdffb640 1 -- 172.21.15.115:0/1854487995 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b80317c0 con 0x7f34c00d1230 2023-12-01T01:36:13.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.910+0000 7f34bdffb640 1 -- 172.21.15.115:0/1854487995 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b8031920 con 0x7f34c00d1230 2023-12-01T01:36:13.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.910+0000 7f34bdffb640 1 -- 172.21.15.115:0/1854487995 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8031cb0 con 0x7f34c00d1230 2023-12-01T01:36:13.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.911+0000 7f34c55ab640 1 -- 172.21.15.115:0/1854487995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 msgr2=0x7f34c00d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:13.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.911+0000 7f34c55ab640 1 --2- 172.21.15.115:0/1854487995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 0x7f34c00d16a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009870 tx=0x7f34b802b860 comp rx=0 tx=0).stop 2023-12-01T01:36:13.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.911+0000 7f34c55ab640 1 -- 172.21.15.115:0/1854487995 shutdown_connections 2023-12-01T01:36:13.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.911+0000 7f34c55ab640 1 --2- 172.21.15.115:0/1854487995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 0x7f34c00d16a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:13.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.911+0000 7f34c55ab640 1 --2- 172.21.15.115:0/1854487995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c00b5d80 0x7f34c00d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:13.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.911+0000 7f34c55ab640 1 -- 172.21.15.115:0/1854487995 >> 172.21.15.115:0/1854487995 conn(0x7f34c006a0e0 msgr2=0x7f34c006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:13.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.911+0000 7f34c55ab640 1 -- 172.21.15.115:0/1854487995 shutdown_connections 2023-12-01T01:36:13.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.911+0000 7f34c55ab640 1 -- 172.21.15.115:0/1854487995 wait complete. 2023-12-01T01:36:13.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.912+0000 7f34c55ab640 1 Processor -- start 2023-12-01T01:36:13.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.913+0000 7f34c55ab640 1 -- start start 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34c55ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c00b5d80 0x7f34c0156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34c55ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 0x7f34c015f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34c55ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c015cd20 0x7f34c015fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34c55ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c00b4250 con 0x7f34c00d1230 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34c55ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c01f3600 con 0x7f34c015cd20 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34c55ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c01f38e0 con 0x7f34c00b5d80 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34be7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 0x7f34c015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34be7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 0x7f34c015f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42758/0 (socket says 172.21.15.115:42758) 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34be7fc640 1 -- 172.21.15.115:0/1507491015 learned_addr learned my addr 172.21.15.115:0/1507491015 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34beffd640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c00b5d80 0x7f34c0156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34bf7fe640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c015cd20 0x7f34c015fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34be7fc640 1 -- 172.21.15.115:0/1507491015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c00b5d80 msgr2=0x7f34c0156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:13.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34be7fc640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c00b5d80 0x7f34c0156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:13.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34be7fc640 1 -- 172.21.15.115:0/1507491015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c015cd20 msgr2=0x7f34c015fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:13.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34be7fc640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c015cd20 0x7f34c015fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:13.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34be7fc640 1 -- 172.21.15.115:0/1507491015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80094d0 con 0x7f34c00d1230 2023-12-01T01:36:13.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.914+0000 7f34bf7fe640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c015cd20 0x7f34c015fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:13.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.915+0000 7f34be7fc640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 0x7f34c015f980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f34a800b450 tx=0x7f34a800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:13.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.915+0000 7f34b77fe640 1 -- 172.21.15.115:0/1507491015 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8011890 con 0x7f34c00d1230 2023-12-01T01:36:13.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.915+0000 7f34b77fe640 1 -- 172.21.15.115:0/1507491015 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a80119f0 con 0x7f34c00d1230 2023-12-01T01:36:13.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.915+0000 7f34b77fe640 1 -- 172.21.15.115:0/1507491015 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8011da0 con 0x7f34c00d1230 2023-12-01T01:36:13.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.915+0000 7f34c55ab640 1 -- 172.21.15.115:0/1507491015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c01f3bc0 con 0x7f34c00d1230 2023-12-01T01:36:13.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.915+0000 7f34c55ab640 1 -- 172.21.15.115:0/1507491015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c00b34f0 con 0x7f34c00d1230 2023-12-01T01:36:13.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.916+0000 7f34c55ab640 1 -- 172.21.15.115:0/1507491015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c01f40e0 con 0x7f34c00d1230 2023-12-01T01:36:13.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.917+0000 7f34b77fe640 1 -- 172.21.15.115:0/1507491015 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f34a801a400 con 0x7f34c00d1230 2023-12-01T01:36:13.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.918+0000 7f34b77fe640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f34a006a430 0x7f34a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:13.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.918+0000 7f34b77fe640 1 -- 172.21.15.115:0/1507491015 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34a8088d20 con 0x7f34c00d1230 2023-12-01T01:36:13.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.918+0000 7f34beffd640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f34a006a430 0x7f34a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:13.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.919+0000 7f34beffd640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f34a006a430 0x7f34a006c8f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f34c00c90b0 tx=0x7f34b00094f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:13.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:13.922+0000 7f34b77fe640 1 -- 172.21.15.115:0/1507491015 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f34a8054300 con 0x7f34c00d1230 2023-12-01T01:36:14.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.149+0000 7f34c55ab640 1 -- 172.21.15.115:0/1507491015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34c00d2990 con 0x7f34c00d1230 2023-12-01T01:36:14.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.150+0000 7f34b77fe640 1 -- 172.21.15.115:0/1507491015 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4654 (secure 0 0 0) 0x7f34a8015090 con 0x7f34c00d1230 2023-12-01T01:36:14.153 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:14.153 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3656795862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3656795862},{"type":"v1","addr":"172.21.15.142:6834","nonce":3656795862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1125002206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1125002206},{"type":"v1","addr":"172.21.15.115:6836","nonce":1125002206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3350782192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3350782192},{"type":"v1","addr":"172.21.15.115:6837","nonce":3350782192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:36:10.637450+0000","modified":"2023-12-01T01:36:13.663004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2808777047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2808777047},{"type":"v1","addr":"172.21.15.142:6835","nonce":2808777047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:36:14.154 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 8 2023-12-01T01:36:14.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 -- 172.21.15.115:0/1507491015 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f34a006a430 msgr2=0x7f34a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f34a006a430 0x7f34a006c8f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f34c00c90b0 tx=0x7f34b00094f0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 -- 172.21.15.115:0/1507491015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 msgr2=0x7f34c015f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 0x7f34c015f980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f34a800b450 tx=0x7f34a800b910 comp rx=0 tx=0).stop 2023-12-01T01:36:14.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 -- 172.21.15.115:0/1507491015 shutdown_connections 2023-12-01T01:36:14.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f34a006a430 0x7f34a006c8f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c015cd20 0x7f34c015fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c00d1230 0x7f34c015f980 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 --2- 172.21.15.115:0/1507491015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34c00b5d80 0x7f34c0156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 -- 172.21.15.115:0/1507491015 >> 172.21.15.115:0/1507491015 conn(0x7f34c006a0e0 msgr2=0x7f34c0158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:14.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.153+0000 7f34c55ab640 1 -- 172.21.15.115:0/1507491015 shutdown_connections 2023-12-01T01:36:14.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.154+0000 7f34c55ab640 1 -- 172.21.15.115:0/1507491015 wait complete. 2023-12-01T01:36:14.174 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:14.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.382+0000 7fa8774c4640 1 Processor -- start 2023-12-01T01:36:14.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.382+0000 7fa8774c4640 1 -- start start 2023-12-01T01:36:14.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa8774c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa87015cf90 0x7fa8700d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa8774c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700ce0d0 0x7fa8700b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa8774c4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87015d370 con 0x7fa8700b8610 2023-12-01T01:36:14.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa8774c4640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700d19a0 con 0x7fa87015cf90 2023-12-01T01:36:14.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa8774c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700d1c80 con 0x7fa8700ce0d0 2023-12-01T01:36:14.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa875260640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa87015cf90 0x7fa8700d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa875260640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa87015cf90 0x7fa8700d1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42760/0 (socket says 172.21.15.115:42760) 2023-12-01T01:36:14.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa875260640 1 -- 172.21.15.115:0/1283669800 learned_addr learned my addr 172.21.15.115:0/1283669800 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:14.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa875a61640 1 --2- 172.21.15.115:0/1283669800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700ce0d0 0x7fa8700b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa875260640 1 -- 172.21.15.115:0/1283669800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700ce0d0 msgr2=0x7fa8700b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:14.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa875260640 1 --2- 172.21.15.115:0/1283669800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700ce0d0 0x7fa8700b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa875260640 1 -- 172.21.15.115:0/1283669800 >> v1:172.21.15.115:6789/0 conn(0x7fa8700b8610 legacy=0x7fa8700cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa875260640 1 -- 172.21.15.115:0/1283669800 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700d1f60 con 0x7fa87015cf90 2023-12-01T01:36:14.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa867fff640 1 -- 172.21.15.115:0/1283669800 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970719481 0 0) 0x7fa87015d370 con 0x7fa8700b8610 2023-12-01T01:36:14.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.384+0000 7fa875a61640 1 --2- 172.21.15.115:0/1283669800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700ce0d0 0x7fa8700b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:14.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.386+0000 7fa875260640 1 --2- 172.21.15.115:0/1283669800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa87015cf90 0x7fa8700d1030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009870 tx=0x7fa86c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5714597395a955ce server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:14.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.386+0000 7fa867fff640 1 -- 172.21.15.115:0/1283669800 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c030840 con 0x7fa87015cf90 2023-12-01T01:36:14.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.386+0000 7fa867fff640 1 -- 172.21.15.115:0/1283669800 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c0309a0 con 0x7fa87015cf90 2023-12-01T01:36:14.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.386+0000 7fa867fff640 1 -- 172.21.15.115:0/1283669800 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c030d30 con 0x7fa87015cf90 2023-12-01T01:36:14.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.386+0000 7fa8774c4640 1 -- 172.21.15.115:0/1283669800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa87015cf90 msgr2=0x7fa8700d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.386+0000 7fa8774c4640 1 --2- 172.21.15.115:0/1283669800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa87015cf90 0x7fa8700d1030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009870 tx=0x7fa86c02b980 comp rx=0 tx=0).stop 2023-12-01T01:36:14.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.386+0000 7fa8774c4640 1 -- 172.21.15.115:0/1283669800 shutdown_connections 2023-12-01T01:36:14.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.386+0000 7fa8774c4640 1 --2- 172.21.15.115:0/1283669800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700ce0d0 0x7fa8700b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.386+0000 7fa8774c4640 1 --2- 172.21.15.115:0/1283669800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa87015cf90 0x7fa8700d1030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.386+0000 7fa8774c4640 1 -- 172.21.15.115:0/1283669800 >> 172.21.15.115:0/1283669800 conn(0x7fa870065960 msgr2=0x7fa870065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.387+0000 7fa8774c4640 1 -- 172.21.15.115:0/1283669800 shutdown_connections 2023-12-01T01:36:14.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.387+0000 7fa8774c4640 1 -- 172.21.15.115:0/1283669800 wait complete. 2023-12-01T01:36:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.388+0000 7fa8774c4640 1 Processor -- start 2023-12-01T01:36:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.388+0000 7fa8774c4640 1 -- start start 2023-12-01T01:36:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.389+0000 7fa8774c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700b8610 0x7fa8700b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.389+0000 7fa8774c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8700ce0d0 0x7fa8701f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.389+0000 7fa8774c4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa87015cf90 0x7fa8701f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.389+0000 7fa8774c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa870158770 con 0x7fa8700ce0d0 2023-12-01T01:36:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.389+0000 7fa8774c4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700b5d50 con 0x7fa87015cf90 2023-12-01T01:36:14.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.389+0000 7fa8774c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700b3470 con 0x7fa8700b8610 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.389+0000 7fa874a5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8700ce0d0 0x7fa8701f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa874a5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8700ce0d0 0x7fa8701f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42772/0 (socket says 172.21.15.115:42772) 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa874a5f640 1 -- 172.21.15.115:0/1093875182 learned_addr learned my addr 172.21.15.115:0/1093875182 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa875260640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700b8610 0x7fa8700b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa874a5f640 1 -- 172.21.15.115:0/1093875182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700b8610 msgr2=0x7fa8700b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa874a5f640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700b8610 0x7fa8700b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa874a5f640 1 -- 172.21.15.115:0/1093875182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa87015cf90 msgr2=0x7fa8701f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa874a5f640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa87015cf90 0x7fa8701f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa874a5f640 1 -- 172.21.15.115:0/1093875182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0094d0 con 0x7fa8700ce0d0 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa875260640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700b8610 0x7fa8700b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa874a5f640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8700ce0d0 0x7fa8701f27e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa86000c520 tx=0x7fa86000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa865ffb640 1 -- 172.21.15.115:0/1093875182 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600035f0 con 0x7fa8700ce0d0 2023-12-01T01:36:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa865ffb640 1 -- 172.21.15.115:0/1093875182 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860003770 con 0x7fa8700ce0d0 2023-12-01T01:36:14.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa865ffb640 1 -- 172.21.15.115:0/1093875182 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860003b20 con 0x7fa8700ce0d0 2023-12-01T01:36:14.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.390+0000 7fa8774c4640 1 -- 172.21.15.115:0/1093875182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8700b3750 con 0x7fa8700ce0d0 2023-12-01T01:36:14.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.391+0000 7fa8774c4640 1 -- 172.21.15.115:0/1093875182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8700b3d10 con 0x7fa8700ce0d0 2023-12-01T01:36:14.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.392+0000 7fa8774c4640 1 -- 172.21.15.115:0/1093875182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8700d2220 con 0x7fa8700ce0d0 2023-12-01T01:36:14.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.393+0000 7fa865ffb640 1 -- 172.21.15.115:0/1093875182 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa860003c80 con 0x7fa8700ce0d0 2023-12-01T01:36:14.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.393+0000 7fa865ffb640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa85406a430 0x7fa85406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.393+0000 7fa865ffb640 1 -- 172.21.15.115:0/1093875182 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa860087f00 con 0x7fa8700ce0d0 2023-12-01T01:36:14.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.394+0000 7fa875260640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa85406a430 0x7fa85406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.394+0000 7fa875260640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa85406a430 0x7fa85406c8f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa86c036000 tx=0x7fa86c03b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:14.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.398+0000 7fa865ffb640 1 -- 172.21.15.115:0/1093875182 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa8700d2220 con 0x7fa8700ce0d0 2023-12-01T01:36:14.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.574+0000 7fa8774c4640 1 -- 172.21.15.115:0/1093875182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8700b5f00 con 0x7fa8700ce0d0 2023-12-01T01:36:14.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.576+0000 7fa865ffb640 1 -- 172.21.15.115:0/1093875182 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16089 (secure 0 0 0) 0x7fa8700b5f00 con 0x7fa8700ce0d0 2023-12-01T01:36:14.578 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:14.578 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:11.641476+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:14.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.578+0000 7fa8774c4640 1 -- 172.21.15.115:0/1093875182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa85406a430 msgr2=0x7fa85406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.578+0000 7fa8774c4640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa85406a430 0x7fa85406c8f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa86c036000 tx=0x7fa86c03b040 comp rx=0 tx=0).stop 2023-12-01T01:36:14.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.579+0000 7fa8774c4640 1 -- 172.21.15.115:0/1093875182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8700ce0d0 msgr2=0x7fa8701f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.579+0000 7fa8774c4640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8700ce0d0 0x7fa8701f27e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa86000c520 tx=0x7fa86000c9e0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.579+0000 7fa8774c4640 1 -- 172.21.15.115:0/1093875182 shutdown_connections 2023-12-01T01:36:14.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.579+0000 7fa8774c4640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa85406a430 0x7fa85406c8f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.579+0000 7fa8774c4640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa87015cf90 0x7fa8701f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.579+0000 7fa8774c4640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8700ce0d0 0x7fa8701f27e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.579+0000 7fa8774c4640 1 --2- 172.21.15.115:0/1093875182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8700b8610 0x7fa8700b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.579+0000 7fa8774c4640 1 -- 172.21.15.115:0/1093875182 >> 172.21.15.115:0/1093875182 conn(0x7fa870065960 msgr2=0x7fa8700c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:14.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.579+0000 7fa8774c4640 1 -- 172.21.15.115:0/1093875182 shutdown_connections 2023-12-01T01:36:14.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.579+0000 7fa8774c4640 1 -- 172.21.15.115:0/1093875182 wait complete. 2023-12-01T01:36:14.599 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T01:36:14.599 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:36:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.796+0000 7f6a32e70640 1 Processor -- start 2023-12-01T01:36:14.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.796+0000 7f6a32e70640 1 -- start start 2023-12-01T01:36:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.797+0000 7f6a32e70640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6a2c159dd0 0x7f6a2c0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.797+0000 7f6a32e70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c070d30 0x7f6a2c0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.798+0000 7f6a32e70640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a2c06f8b0 con 0x7f6a2c06b6f0 2023-12-01T01:36:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.798+0000 7f6a32e70640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2c0adcb0 con 0x7f6a2c159dd0 2023-12-01T01:36:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.798+0000 7f6a32e70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c0ad9d0 con 0x7f6a2c070d30 2023-12-01T01:36:14.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.798+0000 7f6a3140d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c070d30 0x7f6a2c0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.798+0000 7f6a3140d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c070d30 0x7f6a2c0c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:40964/0 (socket says 172.21.15.115:40964) 2023-12-01T01:36:14.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.798+0000 7f6a3140d640 1 -- 172.21.15.115:0/669327698 learned_addr learned my addr 172.21.15.115:0/669327698 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:14.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.798+0000 7f6a30c0c640 1 --2- 172.21.15.115:0/669327698 >> v2:172.21.15.142:3300/0 conn(0x7f6a2c159dd0 0x7f6a2c0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.799+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/669327698 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14990586 0 0) 0x7f6a2c06f8b0 con 0x7f6a2c06b6f0 2023-12-01T01:36:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.799+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/669327698 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a1c005730 con 0x7f6a2c06b6f0 2023-12-01T01:36:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.799+0000 7f6a3140d640 1 -- 172.21.15.115:0/669327698 >> v2:172.21.15.142:3300/0 conn(0x7f6a2c159dd0 msgr2=0x7f6a2c0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.799+0000 7f6a3140d640 1 --2- 172.21.15.115:0/669327698 >> v2:172.21.15.142:3300/0 conn(0x7f6a2c159dd0 0x7f6a2c0c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.799+0000 7f6a3140d640 1 -- 172.21.15.115:0/669327698 >> v1:172.21.15.142:6789/0 conn(0x7f6a2c06b6f0 legacy=0x7f6a2c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.799+0000 7f6a3140d640 1 -- 172.21.15.115:0/669327698 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c0c9000 con 0x7f6a2c070d30 2023-12-01T01:36:14.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.800+0000 7f6a3140d640 1 --2- 172.21.15.115:0/669327698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c070d30 0x7f6a2c0c9d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009e50 tx=0x7f6a2402be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=639b27462ff3daa6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:14.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.801+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/669327698 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2402c9e0 con 0x7f6a2c070d30 2023-12-01T01:36:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.801+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/669327698 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2402cb40 con 0x7f6a2c070d30 2023-12-01T01:36:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.801+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/669327698 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a240343f0 con 0x7f6a2c070d30 2023-12-01T01:36:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.802+0000 7f6a32e70640 1 -- 172.21.15.115:0/669327698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c070d30 msgr2=0x7f6a2c0c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.802+0000 7f6a32e70640 1 --2- 172.21.15.115:0/669327698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c070d30 0x7f6a2c0c9d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009e50 tx=0x7f6a2402be40 comp rx=0 tx=0).stop 2023-12-01T01:36:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.802+0000 7f6a32e70640 1 -- 172.21.15.115:0/669327698 shutdown_connections 2023-12-01T01:36:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.802+0000 7f6a32e70640 1 --2- 172.21.15.115:0/669327698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c070d30 0x7f6a2c0c9d20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.802+0000 7f6a32e70640 1 --2- 172.21.15.115:0/669327698 >> v2:172.21.15.142:3300/0 conn(0x7f6a2c159dd0 0x7f6a2c0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.802+0000 7f6a32e70640 1 -- 172.21.15.115:0/669327698 >> 172.21.15.115:0/669327698 conn(0x7f6a2c0ce880 msgr2=0x7f6a2c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.802+0000 7f6a32e70640 1 -- 172.21.15.115:0/669327698 shutdown_connections 2023-12-01T01:36:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.802+0000 7f6a32e70640 1 -- 172.21.15.115:0/669327698 wait complete. 2023-12-01T01:36:14.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.804+0000 7f6a32e70640 1 Processor -- start 2023-12-01T01:36:14.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.805+0000 7f6a32e70640 1 -- start start 2023-12-01T01:36:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a32e70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c06b6f0 0x7f6a2c14ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a32e70640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c070d30 0x7f6a2c14b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a32e70640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c159dd0 0x7f6a2c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a32e70640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c15bb70 con 0x7f6a2c070d30 2023-12-01T01:36:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a32e70640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c0c23b0 con 0x7f6a2c159dd0 2023-12-01T01:36:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a32e70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c15bd20 con 0x7f6a2c06b6f0 2023-12-01T01:36:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a2bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c070d30 0x7f6a2c14b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a2bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c070d30 0x7f6a2c14b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42778/0 (socket says 172.21.15.115:42778) 2023-12-01T01:36:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a2bfff640 1 -- 172.21.15.115:0/741518973 learned_addr learned my addr 172.21.15.115:0/741518973 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a30c0c640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c06b6f0 0x7f6a2c14ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.806+0000 7f6a3140d640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c159dd0 0x7f6a2c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a2bfff640 1 -- 172.21.15.115:0/741518973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c06b6f0 msgr2=0x7f6a2c14ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a2bfff640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c06b6f0 0x7f6a2c14ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a2bfff640 1 -- 172.21.15.115:0/741518973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c159dd0 msgr2=0x7f6a2c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a2bfff640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c159dd0 0x7f6a2c0ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a2bfff640 1 -- 172.21.15.115:0/741518973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a24009b30 con 0x7f6a2c070d30 2023-12-01T01:36:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a2bfff640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c070d30 0x7f6a2c14b500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c490 tx=0x7f6a1800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a297fa640 1 -- 172.21.15.115:0/741518973 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a180052d0 con 0x7f6a2c070d30 2023-12-01T01:36:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a297fa640 1 -- 172.21.15.115:0/741518973 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a18005430 con 0x7f6a2c070d30 2023-12-01T01:36:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a32e70640 1 -- 172.21.15.115:0/741518973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2c0c2600 con 0x7f6a2c070d30 2023-12-01T01:36:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a297fa640 1 -- 172.21.15.115:0/741518973 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18012870 con 0x7f6a2c070d30 2023-12-01T01:36:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.807+0000 7f6a32e70640 1 -- 172.21.15.115:0/741518973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2c15bfe0 con 0x7f6a2c070d30 2023-12-01T01:36:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.808+0000 7f6a32e70640 1 -- 172.21.15.115:0/741518973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2c04e890 con 0x7f6a2c070d30 2023-12-01T01:36:14.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.809+0000 7f6a297fa640 1 -- 172.21.15.115:0/741518973 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6a180129d0 con 0x7f6a2c070d30 2023-12-01T01:36:14.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.810+0000 7f6a297fa640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a1006a430 0x7f6a1006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:14.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.810+0000 7f6a297fa640 1 -- 172.21.15.115:0/741518973 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a18088430 con 0x7f6a2c070d30 2023-12-01T01:36:14.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.810+0000 7f6a30c0c640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a1006a430 0x7f6a1006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:14.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.811+0000 7f6a30c0c640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a1006a430 0x7f6a1006c8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009a30 tx=0x7f6a20008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:14.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.814+0000 7f6a297fa640 1 -- 172.21.15.115:0/741518973 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6a18057540 con 0x7f6a2c070d30 2023-12-01T01:36:15.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:14.999+0000 7f6a32e70640 1 -- 172.21.15.115:0/741518973 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a2c06d0d0 con 0x7f6a1006a430 2023-12-01T01:36:15.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.014+0000 7f6a297fa640 1 -- 172.21.15.115:0/741518973 <== mgr.4098 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253689 (secure 0 0 0) 0x7f6a20009680 con 0x7f6a1006a430 2023-12-01T01:36:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:15.022 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:36:15.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 -- 172.21.15.115:0/741518973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a1006a430 msgr2=0x7f6a1006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a1006a430 0x7f6a1006c8f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009a30 tx=0x7f6a20008040 comp rx=0 tx=0).stop 2023-12-01T01:36:15.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 -- 172.21.15.115:0/741518973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c070d30 msgr2=0x7f6a2c14b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c070d30 0x7f6a2c14b500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c490 tx=0x7f6a1800c950 comp rx=0 tx=0).stop 2023-12-01T01:36:15.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 -- 172.21.15.115:0/741518973 shutdown_connections 2023-12-01T01:36:15.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a1006a430 0x7f6a1006c8f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c159dd0 0x7f6a2c0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c070d30 0x7f6a2c14b500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 --2- 172.21.15.115:0/741518973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c06b6f0 0x7f6a2c14ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 -- 172.21.15.115:0/741518973 >> 172.21.15.115:0/741518973 conn(0x7f6a2c0ce880 msgr2=0x7f6a2c158080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:15.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 -- 172.21.15.115:0/741518973 shutdown_connections 2023-12-01T01:36:15.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.020+0000 7f6a32e70640 1 -- 172.21.15.115:0/741518973 wait complete. 2023-12-01T01:36:15.040 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-01T01:36:14.156842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218404,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756316,"statfs":{"total":773094113280,"available":772870467584,"internally_reserved":0,"allocated":2531328,"data_stored":1583952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.015779"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.161563+0000","last_change":"2023-12-01T01:36:09.161563+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.161563+0000","last_undegraded":"2023-12-01T01:36:09.161563+0000","last_fullsized":"2023-12-01T01:36:09.161563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.154512+0000","last_change":"2023-12-01T01:36:09.154512+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.154512+0000","last_undegraded":"2023-12-01T01:36:09.154512+0000","last_fullsized":"2023-12-01T01:36:09.154512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:52.363022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.165453+0000","last_change":"2023-12-01T01:36:09.165453+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.165453+0000","last_undegraded":"2023-12-01T01:36:09.165453+0000","last_fullsized":"2023-12-01T01:36:09.165453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.153669+0000","last_change":"2023-12-01T01:36:09.153669+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.153669+0000","last_undegraded":"2023-12-01T01:36:09.153669+0000","last_fullsized":"2023-12-01T01:36:09.153669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:09.863515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.161422+0000","last_change":"2023-12-01T01:36:09.161422+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.161422+0000","last_undegraded":"2023-12-01T01:36:09.161422+0000","last_fullsized":"2023-12-01T01:36:09.161422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157570+0000","last_change":"2023-12-01T01:36:09.157570+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157570+0000","last_undegraded":"2023-12-01T01:36:09.157570+0000","last_fullsized":"2023-12-01T01:36:09.157570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157396+0000","last_change":"2023-12-01T01:36:09.157396+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157396+0000","last_undegraded":"2023-12-01T01:36:09.157396+0000","last_fullsized":"2023-12-01T01:36:09.157396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.147258+0000","last_change":"2023-12-01T01:36:09.147258+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.147258+0000","last_undegraded":"2023-12-01T01:36:09.147258+0000","last_fullsized":"2023-12-01T01:36:09.147258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:06:15.188480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156929+0000","last_change":"2023-12-01T01:36:09.156929+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156929+0000","last_undegraded":"2023-12-01T01:36:09.156929+0000","last_fullsized":"2023-12-01T01:36:09.156929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.153757+0000","last_change":"2023-12-01T01:36:09.153757+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.153757+0000","last_undegraded":"2023-12-01T01:36:09.153757+0000","last_fullsized":"2023-12-01T01:36:09.153757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:14.154709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156811+0000","last_change":"2023-12-01T01:36:09.156811+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156811+0000","last_undegraded":"2023-12-01T01:36:09.156811+0000","last_fullsized":"2023-12-01T01:36:09.156811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148460+0000","last_change":"2023-12-01T01:36:09.148460+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148460+0000","last_undegraded":"2023-12-01T01:36:09.148460+0000","last_fullsized":"2023-12-01T01:36:09.148460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156334+0000","last_change":"2023-12-01T01:36:09.156334+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156334+0000","last_undegraded":"2023-12-01T01:36:09.156334+0000","last_fullsized":"2023-12-01T01:36:09.156334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.151058+0000","last_change":"2023-12-01T01:36:09.151058+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.151058+0000","last_undegraded":"2023-12-01T01:36:09.151058+0000","last_fullsized":"2023-12-01T01:36:09.151058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.162373+0000","last_change":"2023-12-01T01:36:09.162373+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.162373+0000","last_undegraded":"2023-12-01T01:36:09.162373+0000","last_fullsized":"2023-12-01T01:36:09.162373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.165332+0000","last_change":"2023-12-01T01:36:09.165332+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.165332+0000","last_undegraded":"2023-12-01T01:36:09.165332+0000","last_fullsized":"2023-12-01T01:36:09.165332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-12-01T01:36:05.602943+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:36:05.602943+0000","last_peered":"2023-12-01T01:36:05.602943+0000","last_clean":"2023-12-01T01:36:05.602943+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:36:05.602943+0000","last_undegraded":"2023-12-01T01:36:05.602943+0000","last_fullsized":"2023-12-01T01:36:05.602943+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.152296+0000","last_change":"2023-12-01T01:36:09.152296+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.152296+0000","last_undegraded":"2023-12-01T01:36:09.152296+0000","last_fullsized":"2023-12-01T01:36:09.152296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.150327+0000","last_change":"2023-12-01T01:36:09.150327+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.150327+0000","last_undegraded":"2023-12-01T01:36:09.150327+0000","last_fullsized":"2023-12-01T01:36:09.150327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:35.765727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157369+0000","last_change":"2023-12-01T01:36:09.157369+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157369+0000","last_undegraded":"2023-12-01T01:36:09.157369+0000","last_fullsized":"2023-12-01T01:36:09.157369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.150068+0000","last_change":"2023-12-01T01:36:09.150068+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.150068+0000","last_undegraded":"2023-12-01T01:36:09.150068+0000","last_fullsized":"2023-12-01T01:36:09.150068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:40.636957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.161575+0000","last_change":"2023-12-01T01:36:09.161575+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.161575+0000","last_undegraded":"2023-12-01T01:36:09.161575+0000","last_fullsized":"2023-12-01T01:36:09.161575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.152093+0000","last_change":"2023-12-01T01:36:09.152093+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.152093+0000","last_undegraded":"2023-12-01T01:36:09.152093+0000","last_fullsized":"2023-12-01T01:36:09.152093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148727+0000","last_change":"2023-12-01T01:36:09.148727+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148727+0000","last_undegraded":"2023-12-01T01:36:09.148727+0000","last_fullsized":"2023-12-01T01:36:09.148727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.151521+0000","last_change":"2023-12-01T01:36:09.151521+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.151521+0000","last_undegraded":"2023-12-01T01:36:09.151521+0000","last_fullsized":"2023-12-01T01:36:09.151521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:51.741532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156425+0000","last_change":"2023-12-01T01:36:09.156425+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156425+0000","last_undegraded":"2023-12-01T01:36:09.156425+0000","last_fullsized":"2023-12-01T01:36:09.156425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148525+0000","last_change":"2023-12-01T01:36:09.148525+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148525+0000","last_undegraded":"2023-12-01T01:36:09.148525+0000","last_fullsized":"2023-12-01T01:36:09.148525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.166197+0000","last_change":"2023-12-01T01:36:09.166197+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.166197+0000","last_undegraded":"2023-12-01T01:36:09.166197+0000","last_fullsized":"2023-12-01T01:36:09.166197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.154290+0000","last_change":"2023-12-01T01:36:09.154290+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.154290+0000","last_undegraded":"2023-12-01T01:36:09.154290+0000","last_fullsized":"2023-12-01T01:36:09.154290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157890+0000","last_change":"2023-12-01T01:36:09.157890+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157890+0000","last_undegraded":"2023-12-01T01:36:09.157890+0000","last_fullsized":"2023-12-01T01:36:09.157890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156923+0000","last_change":"2023-12-01T01:36:09.156923+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156923+0000","last_undegraded":"2023-12-01T01:36:09.156923+0000","last_fullsized":"2023-12-01T01:36:09.156923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.162550+0000","last_change":"2023-12-01T01:36:09.162550+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.162550+0000","last_undegraded":"2023-12-01T01:36:09.162550+0000","last_fullsized":"2023-12-01T01:36:09.162550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148658+0000","last_change":"2023-12-01T01:36:09.148658+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148658+0000","last_undegraded":"2023-12-01T01:36:09.148658+0000","last_fullsized":"2023-12-01T01:36:09.148658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.925}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8420000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.921}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.891}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1030000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.766}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.476}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.532}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0270000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.671}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.804}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.29}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.831}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.756}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.919}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.472}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9890000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:36:15.044 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:36:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.295+0000 7f8237b4e640 1 Processor -- start 2023-12-01T01:36:15.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.296+0000 7f8237b4e640 1 -- start start 2023-12-01T01:36:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.297+0000 7f8237b4e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82300cad70 0x7f823006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.297+0000 7f8237b4e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82300add10 con 0x7f823006f2f0 2023-12-01T01:36:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.297+0000 7f8237b4e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82300c5760 con 0x7f823015e2f0 2023-12-01T01:36:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.297+0000 7f8237b4e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823015e6d0 con 0x7f82300cad70 2023-12-01T01:36:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.297+0000 7f82350e9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82300cad70 0x7f823006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.297+0000 7f82350e9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82300cad70 0x7f823006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42794/0 (socket says 172.21.15.115:42794) 2023-12-01T01:36:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.297+0000 7f82350e9640 1 -- 172.21.15.115:0/3936070875 learned_addr learned my addr 172.21.15.115:0/3936070875 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:15.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.298+0000 7f82350e9640 1 -- 172.21.15.115:0/3936070875 >> v1:172.21.15.142:6789/0 conn(0x7f823015e2f0 legacy=0x7f82300ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.298+0000 7f82350e9640 1 -- 172.21.15.115:0/3936070875 >> v1:172.21.15.115:6789/0 conn(0x7f823006f2f0 legacy=0x7f823006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.298+0000 7f82350e9640 1 -- 172.21.15.115:0/3936070875 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82300cf7b0 con 0x7f82300cad70 2023-12-01T01:36:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.298+0000 7f82348e8640 1 -- 172.21.15.115:0/3936070875 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056845 0 0) 0x7f82300add10 con 0x7f823006f2f0 2023-12-01T01:36:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.299+0000 7f82350e9640 1 --2- 172.21.15.115:0/3936070875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82300cad70 0x7f823006b5c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f82200098a0 tx=0x7f822002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b365ae0ae376b563 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.299+0000 7f82348e8640 1 -- 172.21.15.115:0/3936070875 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220031870 con 0x7f82300cad70 2023-12-01T01:36:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.299+0000 7f82348e8640 1 -- 172.21.15.115:0/3936070875 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82200319d0 con 0x7f82300cad70 2023-12-01T01:36:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.299+0000 7f82348e8640 1 -- 172.21.15.115:0/3936070875 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220031d60 con 0x7f82300cad70 2023-12-01T01:36:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.300+0000 7f8237b4e640 1 -- 172.21.15.115:0/3936070875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82300cad70 msgr2=0x7f823006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.300+0000 7f8237b4e640 1 --2- 172.21.15.115:0/3936070875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82300cad70 0x7f823006b5c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f82200098a0 tx=0x7f822002b910 comp rx=0 tx=0).stop 2023-12-01T01:36:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.300+0000 7f8237b4e640 1 -- 172.21.15.115:0/3936070875 shutdown_connections 2023-12-01T01:36:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.300+0000 7f8237b4e640 1 --2- 172.21.15.115:0/3936070875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82300cad70 0x7f823006b5c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.300+0000 7f8237b4e640 1 -- 172.21.15.115:0/3936070875 >> 172.21.15.115:0/3936070875 conn(0x7f82300ce880 msgr2=0x7f82300cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.300+0000 7f8237b4e640 1 -- 172.21.15.115:0/3936070875 shutdown_connections 2023-12-01T01:36:15.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.300+0000 7f8237b4e640 1 -- 172.21.15.115:0/3936070875 wait complete. 2023-12-01T01:36:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.302+0000 7f8237b4e640 1 Processor -- start 2023-12-01T01:36:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.302+0000 7f8237b4e640 1 -- start start 2023-12-01T01:36:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f8237b4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823006f2f0 0x7f82300c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f8237b4e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82300cad70 0x7f82300acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f8237b4e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823015e2f0 0x7f8230152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f8237b4e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300aa290 con 0x7f823006f2f0 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f8237b4e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300aa570 con 0x7f823015e2f0 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f8237b4e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82300a85c0 con 0x7f82300cad70 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f82358ea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823006f2f0 0x7f82300c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f82358ea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823006f2f0 0x7f82300c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42802/0 (socket says 172.21.15.115:42802) 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f82358ea640 1 -- 172.21.15.115:0/1282426134 learned_addr learned my addr 172.21.15.115:0/1282426134 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f82358ea640 1 -- 172.21.15.115:0/1282426134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82300cad70 msgr2=0x7f82300acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f82358ea640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82300cad70 0x7f82300acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f82358ea640 1 -- 172.21.15.115:0/1282426134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823015e2f0 msgr2=0x7f8230152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f82358ea640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823015e2f0 0x7f8230152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f82358ea640 1 -- 172.21.15.115:0/1282426134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82200094d0 con 0x7f823006f2f0 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.303+0000 7f82358ea640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823006f2f0 0x7f82300c3400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c400 tx=0x7f822c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:15.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.304+0000 7f82267fc640 1 -- 172.21.15.115:0/1282426134 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c0052d0 con 0x7f823006f2f0 2023-12-01T01:36:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.304+0000 7f82267fc640 1 -- 172.21.15.115:0/1282426134 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c005430 con 0x7f823006f2f0 2023-12-01T01:36:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.304+0000 7f8237b4e640 1 -- 172.21.15.115:0/1282426134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82300a88a0 con 0x7f823006f2f0 2023-12-01T01:36:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.304+0000 7f82267fc640 1 -- 172.21.15.115:0/1282426134 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c003830 con 0x7f823006f2f0 2023-12-01T01:36:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.304+0000 7f8237b4e640 1 -- 172.21.15.115:0/1282426134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82300ad1b0 con 0x7f823006f2f0 2023-12-01T01:36:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.305+0000 7f8237b4e640 1 -- 172.21.15.115:0/1282426134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823006fda0 con 0x7f823006f2f0 2023-12-01T01:36:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.306+0000 7f82267fc640 1 -- 172.21.15.115:0/1282426134 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f822c003990 con 0x7f823006f2f0 2023-12-01T01:36:15.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.307+0000 7f82267fc640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f821406a460 0x7f821406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.307+0000 7f82267fc640 1 -- 172.21.15.115:0/1282426134 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f822c087e20 con 0x7f823006f2f0 2023-12-01T01:36:15.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.307+0000 7f82350e9640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f821406a460 0x7f821406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:15.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.308+0000 7f82350e9640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f821406a460 0x7f821406c920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8220036000 tx=0x7f8220030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:15.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.311+0000 7f82267fc640 1 -- 172.21.15.115:0/1282426134 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f822c0534a0 con 0x7f823006f2f0 2023-12-01T01:36:15.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.499+0000 7f8237b4e640 1 -- 172.21.15.115:0/1282426134 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82300b43d0 con 0x7f821406a460 2023-12-01T01:36:15.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.520+0000 7f82267fc640 1 -- 172.21.15.115:0/1282426134 <== mgr.4098 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253689 (secure 0 0 0) 0x7f8220031c10 con 0x7f821406a460 2023-12-01T01:36:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:15.527 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:36:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.526+0000 7f8237b4e640 1 -- 172.21.15.115:0/1282426134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f821406a460 msgr2=0x7f821406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.526+0000 7f8237b4e640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f821406a460 0x7f821406c920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8220036000 tx=0x7f8220030000 comp rx=0 tx=0).stop 2023-12-01T01:36:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.526+0000 7f8237b4e640 1 -- 172.21.15.115:0/1282426134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823006f2f0 msgr2=0x7f82300c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.526+0000 7f8237b4e640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823006f2f0 0x7f82300c3400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c400 tx=0x7f822c00c8c0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.526+0000 7f8237b4e640 1 -- 172.21.15.115:0/1282426134 shutdown_connections 2023-12-01T01:36:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.526+0000 7f8237b4e640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f821406a460 0x7f821406c920 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.526+0000 7f8237b4e640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82300cad70 0x7f82300acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.526+0000 7f8237b4e640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823015e2f0 0x7f8230152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.526+0000 7f8237b4e640 1 --2- 172.21.15.115:0/1282426134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f823006f2f0 0x7f82300c3400 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.526+0000 7f8237b4e640 1 -- 172.21.15.115:0/1282426134 >> 172.21.15.115:0/1282426134 conn(0x7f82300ce880 msgr2=0x7f82300cc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.527+0000 7f8237b4e640 1 -- 172.21.15.115:0/1282426134 shutdown_connections 2023-12-01T01:36:15.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.527+0000 7f8237b4e640 1 -- 172.21.15.115:0/1282426134 wait complete. 2023-12-01T01:36:15.546 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-01T01:36:14.156842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218404,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756316,"statfs":{"total":773094113280,"available":772870467584,"internally_reserved":0,"allocated":2531328,"data_stored":1583952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.015779"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.161563+0000","last_change":"2023-12-01T01:36:09.161563+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.161563+0000","last_undegraded":"2023-12-01T01:36:09.161563+0000","last_fullsized":"2023-12-01T01:36:09.161563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.154512+0000","last_change":"2023-12-01T01:36:09.154512+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.154512+0000","last_undegraded":"2023-12-01T01:36:09.154512+0000","last_fullsized":"2023-12-01T01:36:09.154512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:52.363022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.165453+0000","last_change":"2023-12-01T01:36:09.165453+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.165453+0000","last_undegraded":"2023-12-01T01:36:09.165453+0000","last_fullsized":"2023-12-01T01:36:09.165453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.153669+0000","last_change":"2023-12-01T01:36:09.153669+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.153669+0000","last_undegraded":"2023-12-01T01:36:09.153669+0000","last_fullsized":"2023-12-01T01:36:09.153669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:09.863515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.161422+0000","last_change":"2023-12-01T01:36:09.161422+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.161422+0000","last_undegraded":"2023-12-01T01:36:09.161422+0000","last_fullsized":"2023-12-01T01:36:09.161422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157570+0000","last_change":"2023-12-01T01:36:09.157570+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157570+0000","last_undegraded":"2023-12-01T01:36:09.157570+0000","last_fullsized":"2023-12-01T01:36:09.157570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157396+0000","last_change":"2023-12-01T01:36:09.157396+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157396+0000","last_undegraded":"2023-12-01T01:36:09.157396+0000","last_fullsized":"2023-12-01T01:36:09.157396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.147258+0000","last_change":"2023-12-01T01:36:09.147258+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.147258+0000","last_undegraded":"2023-12-01T01:36:09.147258+0000","last_fullsized":"2023-12-01T01:36:09.147258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:06:15.188480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156929+0000","last_change":"2023-12-01T01:36:09.156929+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156929+0000","last_undegraded":"2023-12-01T01:36:09.156929+0000","last_fullsized":"2023-12-01T01:36:09.156929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.153757+0000","last_change":"2023-12-01T01:36:09.153757+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.153757+0000","last_undegraded":"2023-12-01T01:36:09.153757+0000","last_fullsized":"2023-12-01T01:36:09.153757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:14.154709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156811+0000","last_change":"2023-12-01T01:36:09.156811+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156811+0000","last_undegraded":"2023-12-01T01:36:09.156811+0000","last_fullsized":"2023-12-01T01:36:09.156811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148460+0000","last_change":"2023-12-01T01:36:09.148460+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148460+0000","last_undegraded":"2023-12-01T01:36:09.148460+0000","last_fullsized":"2023-12-01T01:36:09.148460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156334+0000","last_change":"2023-12-01T01:36:09.156334+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156334+0000","last_undegraded":"2023-12-01T01:36:09.156334+0000","last_fullsized":"2023-12-01T01:36:09.156334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.151058+0000","last_change":"2023-12-01T01:36:09.151058+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.151058+0000","last_undegraded":"2023-12-01T01:36:09.151058+0000","last_fullsized":"2023-12-01T01:36:09.151058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.162373+0000","last_change":"2023-12-01T01:36:09.162373+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.162373+0000","last_undegraded":"2023-12-01T01:36:09.162373+0000","last_fullsized":"2023-12-01T01:36:09.162373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.165332+0000","last_change":"2023-12-01T01:36:09.165332+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.165332+0000","last_undegraded":"2023-12-01T01:36:09.165332+0000","last_fullsized":"2023-12-01T01:36:09.165332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-12-01T01:36:05.602943+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:36:05.602943+0000","last_peered":"2023-12-01T01:36:05.602943+0000","last_clean":"2023-12-01T01:36:05.602943+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:36:05.602943+0000","last_undegraded":"2023-12-01T01:36:05.602943+0000","last_fullsized":"2023-12-01T01:36:05.602943+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.152296+0000","last_change":"2023-12-01T01:36:09.152296+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.152296+0000","last_undegraded":"2023-12-01T01:36:09.152296+0000","last_fullsized":"2023-12-01T01:36:09.152296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.150327+0000","last_change":"2023-12-01T01:36:09.150327+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.150327+0000","last_undegraded":"2023-12-01T01:36:09.150327+0000","last_fullsized":"2023-12-01T01:36:09.150327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:35.765727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157369+0000","last_change":"2023-12-01T01:36:09.157369+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157369+0000","last_undegraded":"2023-12-01T01:36:09.157369+0000","last_fullsized":"2023-12-01T01:36:09.157369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.150068+0000","last_change":"2023-12-01T01:36:09.150068+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.150068+0000","last_undegraded":"2023-12-01T01:36:09.150068+0000","last_fullsized":"2023-12-01T01:36:09.150068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:40.636957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.161575+0000","last_change":"2023-12-01T01:36:09.161575+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.161575+0000","last_undegraded":"2023-12-01T01:36:09.161575+0000","last_fullsized":"2023-12-01T01:36:09.161575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.152093+0000","last_change":"2023-12-01T01:36:09.152093+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.152093+0000","last_undegraded":"2023-12-01T01:36:09.152093+0000","last_fullsized":"2023-12-01T01:36:09.152093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148727+0000","last_change":"2023-12-01T01:36:09.148727+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148727+0000","last_undegraded":"2023-12-01T01:36:09.148727+0000","last_fullsized":"2023-12-01T01:36:09.148727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.151521+0000","last_change":"2023-12-01T01:36:09.151521+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.151521+0000","last_undegraded":"2023-12-01T01:36:09.151521+0000","last_fullsized":"2023-12-01T01:36:09.151521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:51.741532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156425+0000","last_change":"2023-12-01T01:36:09.156425+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156425+0000","last_undegraded":"2023-12-01T01:36:09.156425+0000","last_fullsized":"2023-12-01T01:36:09.156425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148525+0000","last_change":"2023-12-01T01:36:09.148525+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148525+0000","last_undegraded":"2023-12-01T01:36:09.148525+0000","last_fullsized":"2023-12-01T01:36:09.148525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.166197+0000","last_change":"2023-12-01T01:36:09.166197+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.166197+0000","last_undegraded":"2023-12-01T01:36:09.166197+0000","last_fullsized":"2023-12-01T01:36:09.166197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.154290+0000","last_change":"2023-12-01T01:36:09.154290+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.154290+0000","last_undegraded":"2023-12-01T01:36:09.154290+0000","last_fullsized":"2023-12-01T01:36:09.154290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157890+0000","last_change":"2023-12-01T01:36:09.157890+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157890+0000","last_undegraded":"2023-12-01T01:36:09.157890+0000","last_fullsized":"2023-12-01T01:36:09.157890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156923+0000","last_change":"2023-12-01T01:36:09.156923+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156923+0000","last_undegraded":"2023-12-01T01:36:09.156923+0000","last_fullsized":"2023-12-01T01:36:09.156923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.162550+0000","last_change":"2023-12-01T01:36:09.162550+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.162550+0000","last_undegraded":"2023-12-01T01:36:09.162550+0000","last_fullsized":"2023-12-01T01:36:09.162550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148658+0000","last_change":"2023-12-01T01:36:09.148658+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148658+0000","last_undegraded":"2023-12-01T01:36:09.148658+0000","last_fullsized":"2023-12-01T01:36:09.148658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.925}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8420000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.921}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.891}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1030000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.766}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.476}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.532}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0270000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.671}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.804}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.29}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.831}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.756}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.919}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.472}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9890000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:36:15.550 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:36:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.806+0000 7faf3fa2c640 1 Processor -- start 2023-12-01T01:36:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.806+0000 7faf3fa2c640 1 -- start start 2023-12-01T01:36:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.807+0000 7faf3fa2c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faf380c9ef0 0x7faf3806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.807+0000 7faf3fa2c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faf3806b6f0 0x7faf38070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.807+0000 7faf3fa2c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3806f8b0 con 0x7faf38159dd0 2023-12-01T01:36:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.808+0000 7faf3fa2c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf380adcb0 con 0x7faf380c9ef0 2023-12-01T01:36:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.808+0000 7faf3fa2c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf380ad9d0 con 0x7faf3806b6f0 2023-12-01T01:36:15.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.808+0000 7faf3cfc7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faf380c9ef0 0x7faf3806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.808+0000 7faf3cfc7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faf380c9ef0 0x7faf3806fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42804/0 (socket says 172.21.15.115:42804) 2023-12-01T01:36:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.808+0000 7faf3cfc7640 1 -- 172.21.15.115:0/956589091 learned_addr learned my addr 172.21.15.115:0/956589091 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.808+0000 7faf3dfc9640 1 --2- 172.21.15.115:0/956589091 >> v2:172.21.15.142:3300/0 conn(0x7faf3806b6f0 0x7faf38070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.808+0000 7faf3cfc7640 1 -- 172.21.15.115:0/956589091 >> v2:172.21.15.142:3300/0 conn(0x7faf3806b6f0 msgr2=0x7faf38070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.808+0000 7faf3cfc7640 1 --2- 172.21.15.115:0/956589091 >> v2:172.21.15.142:3300/0 conn(0x7faf3806b6f0 0x7faf38070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.808+0000 7faf3cfc7640 1 -- 172.21.15.115:0/956589091 >> v1:172.21.15.142:6789/0 conn(0x7faf38159dd0 legacy=0x7faf380c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.808+0000 7faf3cfc7640 1 -- 172.21.15.115:0/956589091 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380c9000 con 0x7faf380c9ef0 2023-12-01T01:36:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.809+0000 7faf3cfc7640 1 --2- 172.21.15.115:0/956589091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 0x7faf3806fd20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faf3806d470 tx=0x7faf2802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a305cb9f96e22f04 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf2ffff640 1 -- 172.21.15.115:0/956589091 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28031750 con 0x7faf380c9ef0 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf2ffff640 1 -- 172.21.15.115:0/956589091 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf280318b0 con 0x7faf380c9ef0 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf2ffff640 1 -- 172.21.15.115:0/956589091 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28031c40 con 0x7faf380c9ef0 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf3fa2c640 1 -- 172.21.15.115:0/956589091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 msgr2=0x7faf3806fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf3fa2c640 1 --2- 172.21.15.115:0/956589091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 0x7faf3806fd20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faf3806d470 tx=0x7faf2802b7f0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf3fa2c640 1 -- 172.21.15.115:0/956589091 shutdown_connections 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf3fa2c640 1 --2- 172.21.15.115:0/956589091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 0x7faf3806fd20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf3fa2c640 1 --2- 172.21.15.115:0/956589091 >> v2:172.21.15.142:3300/0 conn(0x7faf3806b6f0 0x7faf38070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf3fa2c640 1 -- 172.21.15.115:0/956589091 >> 172.21.15.115:0/956589091 conn(0x7faf380ce880 msgr2=0x7faf380cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf3fa2c640 1 -- 172.21.15.115:0/956589091 shutdown_connections 2023-12-01T01:36:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.810+0000 7faf3fa2c640 1 -- 172.21.15.115:0/956589091 wait complete. 2023-12-01T01:36:15.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.812+0000 7faf3fa2c640 1 Processor -- start 2023-12-01T01:36:15.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.812+0000 7faf3fa2c640 1 -- start start 2023-12-01T01:36:15.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.813+0000 7faf3fa2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3806b6f0 0x7faf3814cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.813+0000 7faf3fa2c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 0x7faf38152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.813+0000 7faf3fa2c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf38159dd0 0x7faf38152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.813+0000 7faf3fa2c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38151c50 con 0x7faf380c9ef0 2023-12-01T01:36:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.813+0000 7faf3fa2c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815a960 con 0x7faf38159dd0 2023-12-01T01:36:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.813+0000 7faf3fa2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf381520b0 con 0x7faf3806b6f0 2023-12-01T01:36:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.813+0000 7faf3cfc7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 0x7faf38152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.813+0000 7faf3cfc7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 0x7faf38152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42810/0 (socket says 172.21.15.115:42810) 2023-12-01T01:36:15.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.813+0000 7faf3cfc7640 1 -- 172.21.15.115:0/2199177253 learned_addr learned my addr 172.21.15.115:0/2199177253 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf3d7c8640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3806b6f0 0x7faf3814cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf3cfc7640 1 -- 172.21.15.115:0/2199177253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3806b6f0 msgr2=0x7faf3814cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf3cfc7640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3806b6f0 0x7faf3814cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf3cfc7640 1 -- 172.21.15.115:0/2199177253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf38159dd0 msgr2=0x7faf38152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf3dfc9640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf38159dd0 0x7faf38152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:15.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf3cfc7640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf38159dd0 0x7faf38152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf3cfc7640 1 -- 172.21.15.115:0/2199177253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf280094d0 con 0x7faf380c9ef0 2023-12-01T01:36:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf3cfc7640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 0x7faf38152810 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf2802b830 tx=0x7faf280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf2dffb640 1 -- 172.21.15.115:0/2199177253 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28033c80 con 0x7faf380c9ef0 2023-12-01T01:36:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf2dffb640 1 -- 172.21.15.115:0/2199177253 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf28033de0 con 0x7faf380c9ef0 2023-12-01T01:36:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf2dffb640 1 -- 172.21.15.115:0/2199177253 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2803c640 con 0x7faf380c9ef0 2023-12-01T01:36:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.814+0000 7faf3fa2c640 1 -- 172.21.15.115:0/2199177253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf38151690 con 0x7faf380c9ef0 2023-12-01T01:36:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.815+0000 7faf3fa2c640 1 -- 172.21.15.115:0/2199177253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3815a5f0 con 0x7faf380c9ef0 2023-12-01T01:36:15.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.816+0000 7faf3fa2c640 1 -- 172.21.15.115:0/2199177253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3815aaf0 con 0x7faf380c9ef0 2023-12-01T01:36:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.816+0000 7faf2dffb640 1 -- 172.21.15.115:0/2199177253 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faf2803a040 con 0x7faf380c9ef0 2023-12-01T01:36:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.817+0000 7faf2dffb640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf1c06a460 0x7faf1c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.817+0000 7faf2dffb640 1 -- 172.21.15.115:0/2199177253 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf280aa5f0 con 0x7faf380c9ef0 2023-12-01T01:36:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.818+0000 7faf3d7c8640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf1c06a460 0x7faf1c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:15.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.818+0000 7faf3d7c8640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf1c06a460 0x7faf1c06c920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faf38153430 tx=0x7faf34009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:15.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:15.822+0000 7faf2dffb640 1 -- 172.21.15.115:0/2199177253 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faf28075cf0 con 0x7faf380c9ef0 2023-12-01T01:36:16.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.007+0000 7faf3fa2c640 1 -- 172.21.15.115:0/2199177253 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf3806d0d0 con 0x7faf1c06a460 2023-12-01T01:36:16.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.015+0000 7faf2dffb640 1 -- 172.21.15.115:0/2199177253 <== mgr.4098 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253689 (secure 0 0 0) 0x7faf3400d4f0 con 0x7faf1c06a460 2023-12-01T01:36:16.022 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:16.023 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:36:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 -- 172.21.15.115:0/2199177253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf1c06a460 msgr2=0x7faf1c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf1c06a460 0x7faf1c06c920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faf38153430 tx=0x7faf34009040 comp rx=0 tx=0).stop 2023-12-01T01:36:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 -- 172.21.15.115:0/2199177253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 msgr2=0x7faf38152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 0x7faf38152810 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf2802b830 tx=0x7faf280046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 -- 172.21.15.115:0/2199177253 shutdown_connections 2023-12-01T01:36:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf1c06a460 0x7faf1c06c920 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf38159dd0 0x7faf38152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380c9ef0 0x7faf38152810 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 --2- 172.21.15.115:0/2199177253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3806b6f0 0x7faf3814cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 -- 172.21.15.115:0/2199177253 >> 172.21.15.115:0/2199177253 conn(0x7faf380ce880 msgr2=0x7faf3806c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 -- 172.21.15.115:0/2199177253 shutdown_connections 2023-12-01T01:36:16.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.021+0000 7faf3fa2c640 1 -- 172.21.15.115:0/2199177253 wait complete. 2023-12-01T01:36:16.042 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-01T01:36:14.156842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218404,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756316,"statfs":{"total":773094113280,"available":772870467584,"internally_reserved":0,"allocated":2531328,"data_stored":1583952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.015779"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.161563+0000","last_change":"2023-12-01T01:36:09.161563+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.161563+0000","last_undegraded":"2023-12-01T01:36:09.161563+0000","last_fullsized":"2023-12-01T01:36:09.161563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.154512+0000","last_change":"2023-12-01T01:36:09.154512+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.154512+0000","last_undegraded":"2023-12-01T01:36:09.154512+0000","last_fullsized":"2023-12-01T01:36:09.154512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:52.363022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.165453+0000","last_change":"2023-12-01T01:36:09.165453+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.165453+0000","last_undegraded":"2023-12-01T01:36:09.165453+0000","last_fullsized":"2023-12-01T01:36:09.165453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.153669+0000","last_change":"2023-12-01T01:36:09.153669+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.153669+0000","last_undegraded":"2023-12-01T01:36:09.153669+0000","last_fullsized":"2023-12-01T01:36:09.153669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:09.863515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.161422+0000","last_change":"2023-12-01T01:36:09.161422+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.161422+0000","last_undegraded":"2023-12-01T01:36:09.161422+0000","last_fullsized":"2023-12-01T01:36:09.161422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157570+0000","last_change":"2023-12-01T01:36:09.157570+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157570+0000","last_undegraded":"2023-12-01T01:36:09.157570+0000","last_fullsized":"2023-12-01T01:36:09.157570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157396+0000","last_change":"2023-12-01T01:36:09.157396+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157396+0000","last_undegraded":"2023-12-01T01:36:09.157396+0000","last_fullsized":"2023-12-01T01:36:09.157396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.147258+0000","last_change":"2023-12-01T01:36:09.147258+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.147258+0000","last_undegraded":"2023-12-01T01:36:09.147258+0000","last_fullsized":"2023-12-01T01:36:09.147258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:06:15.188480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156929+0000","last_change":"2023-12-01T01:36:09.156929+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156929+0000","last_undegraded":"2023-12-01T01:36:09.156929+0000","last_fullsized":"2023-12-01T01:36:09.156929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.153757+0000","last_change":"2023-12-01T01:36:09.153757+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.153757+0000","last_undegraded":"2023-12-01T01:36:09.153757+0000","last_fullsized":"2023-12-01T01:36:09.153757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:14.154709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156811+0000","last_change":"2023-12-01T01:36:09.156811+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156811+0000","last_undegraded":"2023-12-01T01:36:09.156811+0000","last_fullsized":"2023-12-01T01:36:09.156811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148460+0000","last_change":"2023-12-01T01:36:09.148460+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148460+0000","last_undegraded":"2023-12-01T01:36:09.148460+0000","last_fullsized":"2023-12-01T01:36:09.148460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156334+0000","last_change":"2023-12-01T01:36:09.156334+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156334+0000","last_undegraded":"2023-12-01T01:36:09.156334+0000","last_fullsized":"2023-12-01T01:36:09.156334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.151058+0000","last_change":"2023-12-01T01:36:09.151058+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.151058+0000","last_undegraded":"2023-12-01T01:36:09.151058+0000","last_fullsized":"2023-12-01T01:36:09.151058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.162373+0000","last_change":"2023-12-01T01:36:09.162373+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.162373+0000","last_undegraded":"2023-12-01T01:36:09.162373+0000","last_fullsized":"2023-12-01T01:36:09.162373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.165332+0000","last_change":"2023-12-01T01:36:09.165332+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.165332+0000","last_undegraded":"2023-12-01T01:36:09.165332+0000","last_fullsized":"2023-12-01T01:36:09.165332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-12-01T01:36:05.602943+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:36:05.602943+0000","last_peered":"2023-12-01T01:36:05.602943+0000","last_clean":"2023-12-01T01:36:05.602943+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:36:05.602943+0000","last_undegraded":"2023-12-01T01:36:05.602943+0000","last_fullsized":"2023-12-01T01:36:05.602943+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.152296+0000","last_change":"2023-12-01T01:36:09.152296+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.152296+0000","last_undegraded":"2023-12-01T01:36:09.152296+0000","last_fullsized":"2023-12-01T01:36:09.152296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.150327+0000","last_change":"2023-12-01T01:36:09.150327+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.150327+0000","last_undegraded":"2023-12-01T01:36:09.150327+0000","last_fullsized":"2023-12-01T01:36:09.150327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:35.765727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157369+0000","last_change":"2023-12-01T01:36:09.157369+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157369+0000","last_undegraded":"2023-12-01T01:36:09.157369+0000","last_fullsized":"2023-12-01T01:36:09.157369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.150068+0000","last_change":"2023-12-01T01:36:09.150068+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.150068+0000","last_undegraded":"2023-12-01T01:36:09.150068+0000","last_fullsized":"2023-12-01T01:36:09.150068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:40.636957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.161575+0000","last_change":"2023-12-01T01:36:09.161575+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.161575+0000","last_undegraded":"2023-12-01T01:36:09.161575+0000","last_fullsized":"2023-12-01T01:36:09.161575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.152093+0000","last_change":"2023-12-01T01:36:09.152093+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.152093+0000","last_undegraded":"2023-12-01T01:36:09.152093+0000","last_fullsized":"2023-12-01T01:36:09.152093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148727+0000","last_change":"2023-12-01T01:36:09.148727+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148727+0000","last_undegraded":"2023-12-01T01:36:09.148727+0000","last_fullsized":"2023-12-01T01:36:09.148727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.151521+0000","last_change":"2023-12-01T01:36:09.151521+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.151521+0000","last_undegraded":"2023-12-01T01:36:09.151521+0000","last_fullsized":"2023-12-01T01:36:09.151521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:51.741532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156425+0000","last_change":"2023-12-01T01:36:09.156425+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156425+0000","last_undegraded":"2023-12-01T01:36:09.156425+0000","last_fullsized":"2023-12-01T01:36:09.156425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148525+0000","last_change":"2023-12-01T01:36:09.148525+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148525+0000","last_undegraded":"2023-12-01T01:36:09.148525+0000","last_fullsized":"2023-12-01T01:36:09.148525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.166197+0000","last_change":"2023-12-01T01:36:09.166197+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.166197+0000","last_undegraded":"2023-12-01T01:36:09.166197+0000","last_fullsized":"2023-12-01T01:36:09.166197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.154290+0000","last_change":"2023-12-01T01:36:09.154290+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.154290+0000","last_undegraded":"2023-12-01T01:36:09.154290+0000","last_fullsized":"2023-12-01T01:36:09.154290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.157890+0000","last_change":"2023-12-01T01:36:09.157890+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.157890+0000","last_undegraded":"2023-12-01T01:36:09.157890+0000","last_fullsized":"2023-12-01T01:36:09.157890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.156923+0000","last_change":"2023-12-01T01:36:09.156923+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.156923+0000","last_undegraded":"2023-12-01T01:36:09.156923+0000","last_fullsized":"2023-12-01T01:36:09.156923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.162550+0000","last_change":"2023-12-01T01:36:09.162550+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.162550+0000","last_undegraded":"2023-12-01T01:36:09.162550+0000","last_fullsized":"2023-12-01T01:36:09.162550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:36:10.140343+0000","last_change":"2023-12-01T01:36:10.140343+0000","last_active":"2023-12-01T01:36:10.140343+0000","last_peered":"2023-12-01T01:36:10.140343+0000","last_clean":"2023-12-01T01:36:10.140343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:10.140343+0000","last_undegraded":"2023-12-01T01:36:10.140343+0000","last_fullsized":"2023-12-01T01:36:10.140343+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-01T01:36:09.148658+0000","last_change":"2023-12-01T01:36:09.148658+0000","last_active":"2023-12-01T01:36:09.128286+0000","last_peered":"2023-12-01T01:36:09.128286+0000","last_clean":"2023-12-01T01:36:09.128286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:36:09.148658+0000","last_undegraded":"2023-12-01T01:36:09.148658+0000","last_fullsized":"2023-12-01T01:36:09.148658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.925}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8420000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7010000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.921}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.891}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1030000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.766}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.476}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.532}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0270000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.671}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.804}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.29}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":761856,"data_stored":640770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.831}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.756}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.919}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":167936,"data_stored":50402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.472}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9890000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:36:16.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2023-12-01T01:36:16.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2023-12-01T01:36:16.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2023-12-01T01:36:16.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2023-12-01T01:36:16.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2023-12-01T01:36:16.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2023-12-01T01:36:16.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2023-12-01T01:36:16.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2023-12-01T01:36:16.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2023-12-01T01:36:16.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2023-12-01T01:36:16.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2023-12-01T01:36:16.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2023-12-01T01:36:16.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2023-12-01T01:36:16.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2023-12-01T01:36:16.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2023-12-01T01:36:16.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2023-12-01T01:36:16.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-12-01T01:36:16.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2023-12-01T01:36:16.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.161563+0000', 'last_change': '2023-12-01T01:36:09.161563+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.161563+0000', 'last_undegraded': '2023-12-01T01:36:09.161563+0000', 'last_fullsized': '2023-12-01T01:36:09.161563+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:36:16.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-12-01T01:36:16.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2023-12-01T01:36:16.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.154512+0000', 'last_change': '2023-12-01T01:36:09.154512+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.154512+0000', 'last_undegraded': '2023-12-01T01:36:09.154512+0000', 'last_fullsized': '2023-12-01T01:36:09.154512+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T01:45:52.363022+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:36:16.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2023-12-01T01:36:16.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.165453+0000', 'last_change': '2023-12-01T01:36:09.165453+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.165453+0000', 'last_undegraded': '2023-12-01T01:36:09.165453+0000', 'last_fullsized': '2023-12-01T01:36:09.165453+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:36:16.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-12-01T01:36:16.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2023-12-01T01:36:16.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.153669+0000', 'last_change': '2023-12-01T01:36:09.153669+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.153669+0000', 'last_undegraded': '2023-12-01T01:36:09.153669+0000', 'last_fullsized': '2023-12-01T01:36:09.153669+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:01:09.863515+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:36:16.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-12-01T01:36:16.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2023-12-01T01:36:16.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.161422+0000', 'last_change': '2023-12-01T01:36:09.161422+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.161422+0000', 'last_undegraded': '2023-12-01T01:36:09.161422+0000', 'last_fullsized': '2023-12-01T01:36:09.161422+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:36:16.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-12-01T01:36:16.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2023-12-01T01:36:16.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.157570+0000', 'last_change': '2023-12-01T01:36:09.157570+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.157570+0000', 'last_undegraded': '2023-12-01T01:36:09.157570+0000', 'last_fullsized': '2023-12-01T01:36:09.157570+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:36:16.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-12-01T01:36:16.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2023-12-01T01:36:16.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.157396+0000', 'last_change': '2023-12-01T01:36:09.157396+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.157396+0000', 'last_undegraded': '2023-12-01T01:36:09.157396+0000', 'last_fullsized': '2023-12-01T01:36:09.157396+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:36:16.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-12-01T01:36:16.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2023-12-01T01:36:16.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.147258+0000', 'last_change': '2023-12-01T01:36:09.147258+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.147258+0000', 'last_undegraded': '2023-12-01T01:36:09.147258+0000', 'last_fullsized': '2023-12-01T01:36:09.147258+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:06:15.188480+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:36:16.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-12-01T01:36:16.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2023-12-01T01:36:16.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.156929+0000', 'last_change': '2023-12-01T01:36:09.156929+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.156929+0000', 'last_undegraded': '2023-12-01T01:36:09.156929+0000', 'last_fullsized': '2023-12-01T01:36:09.156929+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:36:16.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-12-01T01:36:16.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2023-12-01T01:36:16.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.153757+0000', 'last_change': '2023-12-01T01:36:09.153757+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.153757+0000', 'last_undegraded': '2023-12-01T01:36:09.153757+0000', 'last_fullsized': '2023-12-01T01:36:09.153757+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:01:14.154709+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:36:16.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-12-01T01:36:16.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.156811+0000', 'last_change': '2023-12-01T01:36:09.156811+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.156811+0000', 'last_undegraded': '2023-12-01T01:36:09.156811+0000', 'last_fullsized': '2023-12-01T01:36:09.156811+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.148460+0000', 'last_change': '2023-12-01T01:36:09.148460+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.148460+0000', 'last_undegraded': '2023-12-01T01:36:09.148460+0000', 'last_fullsized': '2023-12-01T01:36:09.148460+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.156334+0000', 'last_change': '2023-12-01T01:36:09.156334+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.156334+0000', 'last_undegraded': '2023-12-01T01:36:09.156334+0000', 'last_fullsized': '2023-12-01T01:36:09.156334+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-12-01T01:36:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.151058+0000', 'last_change': '2023-12-01T01:36:09.151058+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.151058+0000', 'last_undegraded': '2023-12-01T01:36:09.151058+0000', 'last_fullsized': '2023-12-01T01:36:09.151058+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.162373+0000', 'last_change': '2023-12-01T01:36:09.162373+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.162373+0000', 'last_undegraded': '2023-12-01T01:36:09.162373+0000', 'last_fullsized': '2023-12-01T01:36:09.162373+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.165332+0000', 'last_change': '2023-12-01T01:36:09.165332+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.165332+0000', 'last_undegraded': '2023-12-01T01:36:09.165332+0000', 'last_fullsized': '2023-12-01T01:36:09.165332+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-12-01T01:36:16.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.152296+0000', 'last_change': '2023-12-01T01:36:09.152296+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.152296+0000', 'last_undegraded': '2023-12-01T01:36:09.152296+0000', 'last_fullsized': '2023-12-01T01:36:09.152296+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.150327+0000', 'last_change': '2023-12-01T01:36:09.150327+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.150327+0000', 'last_undegraded': '2023-12-01T01:36:09.150327+0000', 'last_fullsized': '2023-12-01T01:36:09.150327+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:07:35.765727+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.157369+0000', 'last_change': '2023-12-01T01:36:09.157369+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.157369+0000', 'last_undegraded': '2023-12-01T01:36:09.157369+0000', 'last_fullsized': '2023-12-01T01:36:09.157369+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-12-01T01:36:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.150068+0000', 'last_change': '2023-12-01T01:36:09.150068+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.150068+0000', 'last_undegraded': '2023-12-01T01:36:09.150068+0000', 'last_fullsized': '2023-12-01T01:36:09.150068+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:47:40.636957+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.161575+0000', 'last_change': '2023-12-01T01:36:09.161575+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.161575+0000', 'last_undegraded': '2023-12-01T01:36:09.161575+0000', 'last_fullsized': '2023-12-01T01:36:09.161575+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.152093+0000', 'last_change': '2023-12-01T01:36:09.152093+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.152093+0000', 'last_undegraded': '2023-12-01T01:36:09.152093+0000', 'last_fullsized': '2023-12-01T01:36:09.152093+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-12-01T01:36:16.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.148727+0000', 'last_change': '2023-12-01T01:36:09.148727+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.148727+0000', 'last_undegraded': '2023-12-01T01:36:09.148727+0000', 'last_fullsized': '2023-12-01T01:36:09.148727+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.151521+0000', 'last_change': '2023-12-01T01:36:09.151521+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.151521+0000', 'last_undegraded': '2023-12-01T01:36:09.151521+0000', 'last_fullsized': '2023-12-01T01:36:09.151521+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:12:51.741532+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.156425+0000', 'last_change': '2023-12-01T01:36:09.156425+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.156425+0000', 'last_undegraded': '2023-12-01T01:36:09.156425+0000', 'last_fullsized': '2023-12-01T01:36:09.156425+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-12-01T01:36:16.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2023-12-01T01:36:16.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.148525+0000', 'last_change': '2023-12-01T01:36:09.148525+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.148525+0000', 'last_undegraded': '2023-12-01T01:36:09.148525+0000', 'last_fullsized': '2023-12-01T01:36:09.148525+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:36:16.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-12-01T01:36:16.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2023-12-01T01:36:16.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.166197+0000', 'last_change': '2023-12-01T01:36:09.166197+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.166197+0000', 'last_undegraded': '2023-12-01T01:36:09.166197+0000', 'last_fullsized': '2023-12-01T01:36:09.166197+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:36:16.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-12-01T01:36:16.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2023-12-01T01:36:16.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.154290+0000', 'last_change': '2023-12-01T01:36:09.154290+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.154290+0000', 'last_undegraded': '2023-12-01T01:36:09.154290+0000', 'last_fullsized': '2023-12-01T01:36:09.154290+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:36:16.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-12-01T01:36:16.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-12-01T01:36:16.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2023-12-01T01:36:16.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.157890+0000', 'last_change': '2023-12-01T01:36:09.157890+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.157890+0000', 'last_undegraded': '2023-12-01T01:36:09.157890+0000', 'last_fullsized': '2023-12-01T01:36:09.157890+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:36:16.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-12-01T01:36:16.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2023-12-01T01:36:16.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.156923+0000', 'last_change': '2023-12-01T01:36:09.156923+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.156923+0000', 'last_undegraded': '2023-12-01T01:36:09.156923+0000', 'last_fullsized': '2023-12-01T01:36:09.156923+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:36:16.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-12-01T01:36:16.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2023-12-01T01:36:16.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.162550+0000', 'last_change': '2023-12-01T01:36:09.162550+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.162550+0000', 'last_undegraded': '2023-12-01T01:36:09.162550+0000', 'last_fullsized': '2023-12-01T01:36:09.162550+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:36:16.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-12-01T01:36:16.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T01:36:10.140343+0000', 'last_change': '2023-12-01T01:36:10.140343+0000', 'last_active': '2023-12-01T01:36:10.140343+0000', 'last_peered': '2023-12-01T01:36:10.140343+0000', 'last_clean': '2023-12-01T01:36:10.140343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:10.140343+0000', 'last_undegraded': '2023-12-01T01:36:10.140343+0000', 'last_fullsized': '2023-12-01T01:36:10.140343+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:10.140343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T01:36:16.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2023-12-01T01:36:16.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:36:09.148658+0000', 'last_change': '2023-12-01T01:36:09.148658+0000', 'last_active': '2023-12-01T01:36:09.128286+0000', 'last_peered': '2023-12-01T01:36:09.128286+0000', 'last_clean': '2023-12-01T01:36:09.128286+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:36:09.148658+0000', 'last_undegraded': '2023-12-01T01:36:09.148658+0000', 'last_fullsized': '2023-12-01T01:36:09.148658+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'last_clean_scrub_stamp': '2023-12-01T01:36:09.128286+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:36:16.071 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-01T01:36:16.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.304+0000 7f2beb2fb640 1 Processor -- start 2023-12-01T01:36:16.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.305+0000 7f2beb2fb640 1 -- start start 2023-12-01T01:36:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.306+0000 7f2beb2fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be406d820 0x7f2be40707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.306+0000 7f2beb2fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2be4070cf0 0x7f2be40b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.306+0000 7f2beb2fb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be40cc9f0 con 0x7f2be415ad20 2023-12-01T01:36:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.306+0000 7f2beb2fb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be4071400 con 0x7f2be4070cf0 2023-12-01T01:36:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.306+0000 7f2beb2fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4066650 con 0x7f2be406d820 2023-12-01T01:36:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.306+0000 7f2be9898640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2be4070cf0 0x7f2be40b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.306+0000 7f2be9898640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2be4070cf0 0x7f2be40b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42826/0 (socket says 172.21.15.115:42826) 2023-12-01T01:36:16.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.306+0000 7f2be9898640 1 -- 172.21.15.115:0/1161159157 learned_addr learned my addr 172.21.15.115:0/1161159157 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.307+0000 7f2be8896640 1 --2- 172.21.15.115:0/1161159157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be406d820 0x7f2be40707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.307+0000 7f2be9898640 1 -- 172.21.15.115:0/1161159157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be406d820 msgr2=0x7f2be40707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.307+0000 7f2be9898640 1 --2- 172.21.15.115:0/1161159157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be406d820 0x7f2be40707b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.307+0000 7f2be9898640 1 -- 172.21.15.115:0/1161159157 >> v1:172.21.15.142:6789/0 conn(0x7f2be415ad20 legacy=0x7f2be406d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.307+0000 7f2be9898640 1 -- 172.21.15.115:0/1161159157 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be4071180 con 0x7f2be4070cf0 2023-12-01T01:36:16.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.308+0000 7f2be9898640 1 --2- 172.21.15.115:0/1161159157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 0x7f2be40b2890 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0098a0 tx=0x7f2bdc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a373b5c577a796 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.308+0000 7f2be3fff640 1 -- 172.21.15.115:0/1161159157 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc030840 con 0x7f2be4070cf0 2023-12-01T01:36:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.309+0000 7f2be3fff640 1 -- 172.21.15.115:0/1161159157 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc0309a0 con 0x7f2be4070cf0 2023-12-01T01:36:16.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.309+0000 7f2be3fff640 1 -- 172.21.15.115:0/1161159157 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc030d30 con 0x7f2be4070cf0 2023-12-01T01:36:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.309+0000 7f2beb2fb640 1 -- 172.21.15.115:0/1161159157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 msgr2=0x7f2be40b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.309+0000 7f2beb2fb640 1 --2- 172.21.15.115:0/1161159157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 0x7f2be40b2890 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0098a0 tx=0x7f2bdc02b980 comp rx=0 tx=0).stop 2023-12-01T01:36:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.309+0000 7f2beb2fb640 1 -- 172.21.15.115:0/1161159157 shutdown_connections 2023-12-01T01:36:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.309+0000 7f2beb2fb640 1 --2- 172.21.15.115:0/1161159157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 0x7f2be40b2890 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.309+0000 7f2beb2fb640 1 --2- 172.21.15.115:0/1161159157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be406d820 0x7f2be40707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.309+0000 7f2beb2fb640 1 -- 172.21.15.115:0/1161159157 >> 172.21.15.115:0/1161159157 conn(0x7f2be4069cc0 msgr2=0x7f2be406a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.309+0000 7f2beb2fb640 1 -- 172.21.15.115:0/1161159157 shutdown_connections 2023-12-01T01:36:16.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.309+0000 7f2beb2fb640 1 -- 172.21.15.115:0/1161159157 wait complete. 2023-12-01T01:36:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.311+0000 7f2beb2fb640 1 Processor -- start 2023-12-01T01:36:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.311+0000 7f2beb2fb640 1 -- start start 2023-12-01T01:36:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.312+0000 7f2beb2fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be406d820 0x7f2be41443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.312+0000 7f2beb2fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 0x7f2be40b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.312+0000 7f2beb2fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be415ad20 0x7f2be40b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.312+0000 7f2beb2fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4144900 con 0x7f2be4070cf0 2023-12-01T01:36:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.312+0000 7f2beb2fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40b3580 con 0x7f2be406d820 2023-12-01T01:36:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.312+0000 7f2beb2fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40b2970 con 0x7f2be415ad20 2023-12-01T01:36:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.312+0000 7f2be8896640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 0x7f2be40b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.313+0000 7f2be8896640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 0x7f2be40b4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42834/0 (socket says 172.21.15.115:42834) 2023-12-01T01:36:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.313+0000 7f2be8896640 1 -- 172.21.15.115:0/832172395 learned_addr learned my addr 172.21.15.115:0/832172395 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:16.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.313+0000 7f2be9097640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be406d820 0x7f2be41443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.313+0000 7f2be9898640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be415ad20 0x7f2be40b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.313+0000 7f2be8896640 1 -- 172.21.15.115:0/832172395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be415ad20 msgr2=0x7f2be40b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.313+0000 7f2be8896640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be415ad20 0x7f2be40b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.313+0000 7f2be8896640 1 -- 172.21.15.115:0/832172395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be406d820 msgr2=0x7f2be41443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.313+0000 7f2be8896640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be406d820 0x7f2be41443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.313+0000 7f2be8896640 1 -- 172.21.15.115:0/832172395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc0094d0 con 0x7f2be4070cf0 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.313+0000 7f2be8896640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 0x7f2be40b4380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c7f0 tx=0x7f2bd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.314+0000 7f2be1ffb640 1 -- 172.21.15.115:0/832172395 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0004220 con 0x7f2be4070cf0 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.314+0000 7f2be1ffb640 1 -- 172.21.15.115:0/832172395 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd00043a0 con 0x7f2be4070cf0 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.314+0000 7f2be1ffb640 1 -- 172.21.15.115:0/832172395 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0007860 con 0x7f2be4070cf0 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.314+0000 7f2beb2fb640 1 -- 172.21.15.115:0/832172395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be40b2c50 con 0x7f2be4070cf0 2023-12-01T01:36:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.314+0000 7f2beb2fb640 1 -- 172.21.15.115:0/832172395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be40b3210 con 0x7f2be4070cf0 2023-12-01T01:36:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.315+0000 7f2beb2fb640 1 -- 172.21.15.115:0/832172395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be404e890 con 0x7f2be4070cf0 2023-12-01T01:36:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.316+0000 7f2be1ffb640 1 -- 172.21.15.115:0/832172395 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2bd0007bd0 con 0x7f2be4070cf0 2023-12-01T01:36:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.317+0000 7f2be1ffb640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bc806a480 0x7f2bc806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.317+0000 7f2be1ffb640 1 -- 172.21.15.115:0/832172395 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bd0088830 con 0x7f2be4070cf0 2023-12-01T01:36:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.317+0000 7f2be9097640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bc806a480 0x7f2bc806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.318+0000 7f2be9097640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bc806a480 0x7f2bc806c940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2be41f59e0 tx=0x7f2bd8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:16.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.321+0000 7f2be1ffb640 1 -- 172.21.15.115:0/832172395 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2bd0053f00 con 0x7f2be4070cf0 2023-12-01T01:36:16.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.584+0000 7f2beb2fb640 1 -- 172.21.15.115:0/832172395 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2be414c8a0 con 0x7f2be4070cf0 2023-12-01T01:36:16.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.585+0000 7f2be1ffb640 1 -- 172.21.15.115:0/832172395 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1193 (secure 0 0 0) 0x7f2bd001e4c0 con 0x7f2be4070cf0 2023-12-01T01:36:16.586 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:16.586 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1701394558,"num_in_osds":8,"osd_in_since":1701394549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":223645696,"bytes_avail":772870467584,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042,"read_bytes_sec":1837,"write_bytes_sec":53488,"read_op_per_sec":1,"write_op_per_sec":2},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4294}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-12-01T01:35:45.136858+0000","services":{}},"progress_events":{}} 2023-12-01T01:36:16.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.587+0000 7f2beb2fb640 1 -- 172.21.15.115:0/832172395 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bc806a480 msgr2=0x7f2bc806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.587+0000 7f2beb2fb640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bc806a480 0x7f2bc806c940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2be41f59e0 tx=0x7f2bd8009040 comp rx=0 tx=0).stop 2023-12-01T01:36:16.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.587+0000 7f2beb2fb640 1 -- 172.21.15.115:0/832172395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 msgr2=0x7f2be40b4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.587+0000 7f2beb2fb640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 0x7f2be40b4380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c7f0 tx=0x7f2bd000ccb0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.587+0000 7f2beb2fb640 1 -- 172.21.15.115:0/832172395 shutdown_connections 2023-12-01T01:36:16.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.587+0000 7f2beb2fb640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bc806a480 0x7f2bc806c940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.587+0000 7f2beb2fb640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2be415ad20 0x7f2be40b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.587+0000 7f2beb2fb640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2be4070cf0 0x7f2be40b4380 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.587+0000 7f2beb2fb640 1 --2- 172.21.15.115:0/832172395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2be406d820 0x7f2be41443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.587+0000 7f2beb2fb640 1 -- 172.21.15.115:0/832172395 >> 172.21.15.115:0/832172395 conn(0x7f2be4069cc0 msgr2=0x7f2be4159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.588+0000 7f2beb2fb640 1 -- 172.21.15.115:0/832172395 shutdown_connections 2023-12-01T01:36:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.588+0000 7f2beb2fb640 1 -- 172.21.15.115:0/832172395 wait complete. 2023-12-01T01:36:16.601 INFO:tasks.ceph.mds.d.smithi142.stderr:2023-12-01T01:36:16.600+0000 7f73e741c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:36:16.610 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-01T01:36:16.610 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:36:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.817+0000 7f3e8d511640 1 Processor -- start 2023-12-01T01:36:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.817+0000 7f3e8d511640 1 -- start start 2023-12-01T01:36:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.818+0000 7f3e8d511640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3e8815cd20 0x7f3e880d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.818+0000 7f3e8d511640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e880ce2e0 con 0x7f3e880b5d80 2023-12-01T01:36:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.818+0000 7f3e8d511640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e880ce000 con 0x7f3e880d1230 2023-12-01T01:36:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.818+0000 7f3e8d511640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e880b2160 con 0x7f3e8815cd20 2023-12-01T01:36:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.818+0000 7f3e867fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3e880b5d80 0x7f3e880b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41380/0 (socket says 172.21.15.115:41380) 2023-12-01T01:36:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.818+0000 7f3e867fc640 1 -- 172.21.15.115:0/3504523249 learned_addr learned my addr 172.21.15.115:0/3504523249 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e86ffd640 1 --2- 172.21.15.115:0/3504523249 >> v2:172.21.15.142:3300/0 conn(0x7f3e8815cd20 0x7f3e880d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580129757 0 0) 0x7f3e880ce2e0 con 0x7f3e880b5d80 2023-12-01T01:36:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e74003520 con 0x7f3e880b5d80 2023-12-01T01:36:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 535551282 0 0) 0x7f3e74003520 con 0x7f3e880b5d80 2023-12-01T01:36:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 >> v2:172.21.15.142:3300/0 conn(0x7f3e8815cd20 msgr2=0x7f3e880d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 --2- 172.21.15.115:0/3504523249 >> v2:172.21.15.142:3300/0 conn(0x7f3e8815cd20 0x7f3e880d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 >> v1:172.21.15.142:6789/0 conn(0x7f3e880d1230 legacy=0x7f3e880d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e880cd3d0 con 0x7f3e880b5d80 2023-12-01T01:36:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3e70001670 con 0x7f3e880b5d80 2023-12-01T01:36:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 >> v1:172.21.15.115:6789/0 conn(0x7f3e880b5d80 legacy=0x7f3e880b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740039e0 0x7f3e74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e740047c0 0x7f3e74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e740051f0 0x7f3e7400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7400c120 con 0x7f3e740051f0 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7400c290 con 0x7f3e740039e0 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.819+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7400c450 con 0x7f3e740047c0 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.820+0000 7f3e877fe640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e740051f0 0x7f3e7400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.820+0000 7f3e867fc640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e740047c0 0x7f3e74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.820+0000 7f3e86ffd640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740039e0 0x7f3e74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.820+0000 7f3e877fe640 1 -- 172.21.15.115:0/3504523249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e740047c0 msgr2=0x7f3e74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.820+0000 7f3e877fe640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e740047c0 0x7f3e74004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.820+0000 7f3e877fe640 1 -- 172.21.15.115:0/3504523249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740039e0 msgr2=0x7f3e74003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:16.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.820+0000 7f3e877fe640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740039e0 0x7f3e74003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.820+0000 7f3e877fe640 1 -- 172.21.15.115:0/3504523249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74003520 con 0x7f3e740051f0 2023-12-01T01:36:16.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.821+0000 7f3e877fe640 1 -- 172.21.15.115:0/3504523249 reap_dead start 2023-12-01T01:36:16.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.822+0000 7f3e877fe640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e740051f0 0x7f3e7400b980 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3e80001750 tx=0x7f3e740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183fcd83289c0d0a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:16.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.822+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e800052d0 con 0x7f3e740051f0 2023-12-01T01:36:16.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.822+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8002e050 con 0x7f3e740051f0 2023-12-01T01:36:16.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.822+0000 7f3e85ffb640 1 -- 172.21.15.115:0/3504523249 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80003830 con 0x7f3e740051f0 2023-12-01T01:36:16.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.822+0000 7f3e8d511640 1 -- 172.21.15.115:0/3504523249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e740051f0 msgr2=0x7f3e7400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.822+0000 7f3e8d511640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e740051f0 0x7f3e7400b980 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3e80001750 tx=0x7f3e740057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.822+0000 7f3e8d511640 1 -- 172.21.15.115:0/3504523249 shutdown_connections 2023-12-01T01:36:16.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.822+0000 7f3e8d511640 1 --2- 172.21.15.115:0/3504523249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e740051f0 0x7f3e7400b980 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.822+0000 7f3e8d511640 1 -- 172.21.15.115:0/3504523249 >> 172.21.15.115:0/3504523249 conn(0x7f3e8806a0e0 msgr2=0x7f3e8806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:16.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.823+0000 7f3e8d511640 1 -- 172.21.15.115:0/3504523249 shutdown_connections 2023-12-01T01:36:16.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.823+0000 7f3e8d511640 1 -- 172.21.15.115:0/3504523249 wait complete. 2023-12-01T01:36:16.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.824+0000 7f3e8d511640 1 Processor -- start 2023-12-01T01:36:16.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.824+0000 7f3e8d511640 1 -- start start 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.825+0000 7f3e8d511640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740051f0 0x7f3e8814d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.825+0000 7f3e8d511640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e881e6dc0 0x7f3e880b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.825+0000 7f3e8d511640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e8815a620 0x7f3e8815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.825+0000 7f3e8d511640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8815afe0 con 0x7f3e881e6dc0 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.825+0000 7f3e8d511640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8815e4b0 con 0x7f3e740051f0 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.825+0000 7f3e8d511640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880b32b0 con 0x7f3e8815a620 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e867fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e881e6dc0 0x7f3e880b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e867fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e881e6dc0 0x7f3e880b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42862/0 (socket says 172.21.15.115:42862) 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e867fc640 1 -- 172.21.15.115:0/787159755 learned_addr learned my addr 172.21.15.115:0/787159755 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e877fe640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e8815a620 0x7f3e8815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e86ffd640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740051f0 0x7f3e8814d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e867fc640 1 -- 172.21.15.115:0/787159755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e8815a620 msgr2=0x7f3e8815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e867fc640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e8815a620 0x7f3e8815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e867fc640 1 -- 172.21.15.115:0/787159755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740051f0 msgr2=0x7f3e8814d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:16.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e867fc640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740051f0 0x7f3e8814d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:16.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e867fc640 1 -- 172.21.15.115:0/787159755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e880b3590 con 0x7f3e881e6dc0 2023-12-01T01:36:16.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e86ffd640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740051f0 0x7f3e8814d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:16.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.826+0000 7f3e867fc640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e881e6dc0 0x7f3e880b2a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000e760 tx=0x7f3e7000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:16.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.827+0000 7f3e7f7fe640 1 -- 172.21.15.115:0/787159755 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000ee70 con 0x7f3e881e6dc0 2023-12-01T01:36:16.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.827+0000 7f3e7f7fe640 1 -- 172.21.15.115:0/787159755 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7000c870 con 0x7f3e881e6dc0 2023-12-01T01:36:16.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.827+0000 7f3e8d511640 1 -- 172.21.15.115:0/787159755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e880b3870 con 0x7f3e881e6dc0 2023-12-01T01:36:16.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.827+0000 7f3e7f7fe640 1 -- 172.21.15.115:0/787159755 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000cc20 con 0x7f3e881e6dc0 2023-12-01T01:36:16.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.827+0000 7f3e8d511640 1 -- 172.21.15.115:0/787159755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e880b3b50 con 0x7f3e881e6dc0 2023-12-01T01:36:16.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.828+0000 7f3e8d511640 1 -- 172.21.15.115:0/787159755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8804e890 con 0x7f3e881e6dc0 2023-12-01T01:36:16.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.829+0000 7f3e7f7fe640 1 -- 172.21.15.115:0/787159755 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e7001d460 con 0x7f3e881e6dc0 2023-12-01T01:36:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.830+0000 7f3e7f7fe640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e6806a460 0x7f3e6806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.830+0000 7f3e7f7fe640 1 -- 172.21.15.115:0/787159755 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e70088840 con 0x7f3e881e6dc0 2023-12-01T01:36:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.830+0000 7f3e86ffd640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e6806a460 0x7f3e6806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:16.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.831+0000 7f3e86ffd640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e6806a460 0x7f3e6806c920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e78007440 tx=0x7f3e7800b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:16.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:16.834+0000 7f3e7f7fe640 1 -- 172.21.15.115:0/787159755 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3e700578c0 con 0x7f3e881e6dc0 2023-12-01T01:36:17.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.022+0000 7f3e8d511640 1 -- 172.21.15.115:0/787159755 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e880cd990 con 0x7f3e6806a460 2023-12-01T01:36:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.033+0000 7f3e7f7fe640 1 -- 172.21.15.115:0/787159755 <== mgr.4098 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264434 (secure 0 0 0) 0x7f3e78016420 con 0x7f3e6806a460 2023-12-01T01:36:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:36:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.038+0000 7f3e8d511640 1 -- 172.21.15.115:0/787159755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e6806a460 msgr2=0x7f3e6806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.038+0000 7f3e8d511640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e6806a460 0x7f3e6806c920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e78007440 tx=0x7f3e7800b040 comp rx=0 tx=0).stop 2023-12-01T01:36:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.038+0000 7f3e8d511640 1 -- 172.21.15.115:0/787159755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e881e6dc0 msgr2=0x7f3e880b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.038+0000 7f3e8d511640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e881e6dc0 0x7f3e880b2a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000e760 tx=0x7f3e7000ec20 comp rx=0 tx=0).stop 2023-12-01T01:36:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.039+0000 7f3e8d511640 1 -- 172.21.15.115:0/787159755 shutdown_connections 2023-12-01T01:36:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.039+0000 7f3e8d511640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e6806a460 0x7f3e6806c920 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.039+0000 7f3e8d511640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e8815a620 0x7f3e8815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.039+0000 7f3e8d511640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e881e6dc0 0x7f3e880b2a80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.039+0000 7f3e8d511640 1 --2- 172.21.15.115:0/787159755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740051f0 0x7f3e8814d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.039+0000 7f3e8d511640 1 -- 172.21.15.115:0/787159755 >> 172.21.15.115:0/787159755 conn(0x7f3e8806a0e0 msgr2=0x7f3e8814f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.039+0000 7f3e8d511640 1 -- 172.21.15.115:0/787159755 shutdown_connections 2023-12-01T01:36:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:17.039+0000 7f3e8d511640 1 -- 172.21.15.115:0/787159755 wait complete. 2023-12-01T01:36:17.059 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-01T01:36:16.157256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218536,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756184,"statfs":{"total":773094113280,"available":772870332416,"internally_reserved":0,"allocated":2990080,"data_stored":1790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.003209"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847556+0000","last_change":"2023-12-01T01:36:10.681289+0000","last_active":"2023-12-01T01:36:11.847556+0000","last_peered":"2023-12-01T01:36:11.847556+0000","last_clean":"2023-12-01T01:36:11.847556+0000","last_became_active":"2023-12-01T01:36:10.680689+0000","last_became_peered":"2023-12-01T01:36:10.680689+0000","last_unstale":"2023-12-01T01:36:11.847556+0000","last_undegraded":"2023-12-01T01:36:11.847556+0000","last_fullsized":"2023-12-01T01:36:11.847556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:12:29.670579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844746+0000","last_change":"2023-12-01T01:36:10.668026+0000","last_active":"2023-12-01T01:36:11.844746+0000","last_peered":"2023-12-01T01:36:11.844746+0000","last_clean":"2023-12-01T01:36:11.844746+0000","last_became_active":"2023-12-01T01:36:10.667717+0000","last_became_peered":"2023-12-01T01:36:10.667717+0000","last_unstale":"2023-12-01T01:36:11.844746+0000","last_undegraded":"2023-12-01T01:36:11.844746+0000","last_fullsized":"2023-12-01T01:36:11.844746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:12:29.670579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844579+0000","last_change":"2023-12-01T01:36:10.665172+0000","last_active":"2023-12-01T01:36:11.844579+0000","last_peered":"2023-12-01T01:36:11.844579+0000","last_clean":"2023-12-01T01:36:11.844579+0000","last_became_active":"2023-12-01T01:36:10.664880+0000","last_became_peered":"2023-12-01T01:36:10.664880+0000","last_unstale":"2023-12-01T01:36:11.844579+0000","last_undegraded":"2023-12-01T01:36:11.844579+0000","last_fullsized":"2023-12-01T01:36:11.844579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:41:51.516524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843255+0000","last_change":"2023-12-01T01:36:10.666933+0000","last_active":"2023-12-01T01:36:11.843255+0000","last_peered":"2023-12-01T01:36:11.843255+0000","last_clean":"2023-12-01T01:36:11.843255+0000","last_became_active":"2023-12-01T01:36:10.666669+0000","last_became_peered":"2023-12-01T01:36:10.666669+0000","last_unstale":"2023-12-01T01:36:11.843255+0000","last_undegraded":"2023-12-01T01:36:11.843255+0000","last_fullsized":"2023-12-01T01:36:11.843255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846486+0000","last_change":"2023-12-01T01:36:10.660927+0000","last_active":"2023-12-01T01:36:11.846486+0000","last_peered":"2023-12-01T01:36:11.846486+0000","last_clean":"2023-12-01T01:36:11.846486+0000","last_became_active":"2023-12-01T01:36:10.660693+0000","last_became_peered":"2023-12-01T01:36:10.660693+0000","last_unstale":"2023-12-01T01:36:11.846486+0000","last_undegraded":"2023-12-01T01:36:11.846486+0000","last_fullsized":"2023-12-01T01:36:11.846486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:02.974458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842420+0000","last_change":"2023-12-01T01:36:10.670715+0000","last_active":"2023-12-01T01:36:11.842420+0000","last_peered":"2023-12-01T01:36:11.842420+0000","last_clean":"2023-12-01T01:36:11.842420+0000","last_became_active":"2023-12-01T01:36:10.669975+0000","last_became_peered":"2023-12-01T01:36:10.669975+0000","last_unstale":"2023-12-01T01:36:11.842420+0000","last_undegraded":"2023-12-01T01:36:11.842420+0000","last_fullsized":"2023-12-01T01:36:11.842420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845913+0000","last_change":"2023-12-01T01:36:10.670567+0000","last_active":"2023-12-01T01:36:11.845913+0000","last_peered":"2023-12-01T01:36:11.845913+0000","last_clean":"2023-12-01T01:36:11.845913+0000","last_became_active":"2023-12-01T01:36:10.670325+0000","last_became_peered":"2023-12-01T01:36:10.670325+0000","last_unstale":"2023-12-01T01:36:11.845913+0000","last_undegraded":"2023-12-01T01:36:11.845913+0000","last_fullsized":"2023-12-01T01:36:11.845913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:49.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843702+0000","last_change":"2023-12-01T01:36:10.672309+0000","last_active":"2023-12-01T01:36:11.843702+0000","last_peered":"2023-12-01T01:36:11.843702+0000","last_clean":"2023-12-01T01:36:11.843702+0000","last_became_active":"2023-12-01T01:36:10.671949+0000","last_became_peered":"2023-12-01T01:36:10.671949+0000","last_unstale":"2023-12-01T01:36:11.843702+0000","last_undegraded":"2023-12-01T01:36:11.843702+0000","last_fullsized":"2023-12-01T01:36:11.843702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:44:20.605895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844629+0000","last_change":"2023-12-01T01:36:10.664390+0000","last_active":"2023-12-01T01:36:11.844629+0000","last_peered":"2023-12-01T01:36:11.844629+0000","last_clean":"2023-12-01T01:36:11.844629+0000","last_became_active":"2023-12-01T01:36:10.663998+0000","last_became_peered":"2023-12-01T01:36:10.663998+0000","last_unstale":"2023-12-01T01:36:11.844629+0000","last_undegraded":"2023-12-01T01:36:11.844629+0000","last_fullsized":"2023-12-01T01:36:11.844629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847720+0000","last_change":"2023-12-01T01:36:10.662705+0000","last_active":"2023-12-01T01:36:11.847720+0000","last_peered":"2023-12-01T01:36:11.847720+0000","last_clean":"2023-12-01T01:36:11.847720+0000","last_became_active":"2023-12-01T01:36:10.662424+0000","last_became_peered":"2023-12-01T01:36:10.662424+0000","last_unstale":"2023-12-01T01:36:11.847720+0000","last_undegraded":"2023-12-01T01:36:11.847720+0000","last_fullsized":"2023-12-01T01:36:11.847720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:44:55.685435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846761+0000","last_change":"2023-12-01T01:36:10.669520+0000","last_active":"2023-12-01T01:36:11.846761+0000","last_peered":"2023-12-01T01:36:11.846761+0000","last_clean":"2023-12-01T01:36:11.846761+0000","last_became_active":"2023-12-01T01:36:10.665644+0000","last_became_peered":"2023-12-01T01:36:10.665644+0000","last_unstale":"2023-12-01T01:36:11.846761+0000","last_undegraded":"2023-12-01T01:36:11.846761+0000","last_fullsized":"2023-12-01T01:36:11.846761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:38.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846407+0000","last_change":"2023-12-01T01:36:10.668672+0000","last_active":"2023-12-01T01:36:11.846407+0000","last_peered":"2023-12-01T01:36:11.846407+0000","last_clean":"2023-12-01T01:36:11.846407+0000","last_became_active":"2023-12-01T01:36:10.668180+0000","last_became_peered":"2023-12-01T01:36:10.668180+0000","last_unstale":"2023-12-01T01:36:11.846407+0000","last_undegraded":"2023-12-01T01:36:11.846407+0000","last_fullsized":"2023-12-01T01:36:11.846407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:33.765782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843462+0000","last_change":"2023-12-01T01:36:10.665068+0000","last_active":"2023-12-01T01:36:11.843462+0000","last_peered":"2023-12-01T01:36:11.843462+0000","last_clean":"2023-12-01T01:36:11.843462+0000","last_became_active":"2023-12-01T01:36:10.664845+0000","last_became_peered":"2023-12-01T01:36:10.664845+0000","last_unstale":"2023-12-01T01:36:11.843462+0000","last_undegraded":"2023-12-01T01:36:11.843462+0000","last_fullsized":"2023-12-01T01:36:11.843462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:27.561648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843064+0000","last_change":"2023-12-01T01:36:10.682708+0000","last_active":"2023-12-01T01:36:11.843064+0000","last_peered":"2023-12-01T01:36:11.843064+0000","last_clean":"2023-12-01T01:36:11.843064+0000","last_became_active":"2023-12-01T01:36:10.679209+0000","last_became_peered":"2023-12-01T01:36:10.679209+0000","last_unstale":"2023-12-01T01:36:11.843064+0000","last_undegraded":"2023-12-01T01:36:11.843064+0000","last_fullsized":"2023-12-01T01:36:11.843064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:25.357183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843449+0000","last_change":"2023-12-01T01:36:10.673333+0000","last_active":"2023-12-01T01:36:11.843449+0000","last_peered":"2023-12-01T01:36:11.843449+0000","last_clean":"2023-12-01T01:36:11.843449+0000","last_became_active":"2023-12-01T01:36:10.673021+0000","last_became_peered":"2023-12-01T01:36:10.673021+0000","last_unstale":"2023-12-01T01:36:11.843449+0000","last_undegraded":"2023-12-01T01:36:11.843449+0000","last_fullsized":"2023-12-01T01:36:11.843449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:24.066059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659010+0000","last_change":"2023-12-01T01:36:10.668827+0000","last_active":"2023-12-01T01:36:11.659010+0000","last_peered":"2023-12-01T01:36:11.659010+0000","last_clean":"2023-12-01T01:36:11.659010+0000","last_became_active":"2023-12-01T01:36:10.668507+0000","last_became_peered":"2023-12-01T01:36:10.668507+0000","last_unstale":"2023-12-01T01:36:11.659010+0000","last_undegraded":"2023-12-01T01:36:11.659010+0000","last_fullsized":"2023-12-01T01:36:11.659010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:29:32.707369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660612+0000","last_change":"2023-12-01T01:36:10.681970+0000","last_active":"2023-12-01T01:36:11.660612+0000","last_peered":"2023-12-01T01:36:11.660612+0000","last_clean":"2023-12-01T01:36:11.660612+0000","last_became_active":"2023-12-01T01:36:10.681726+0000","last_became_peered":"2023-12-01T01:36:10.681726+0000","last_unstale":"2023-12-01T01:36:11.660612+0000","last_undegraded":"2023-12-01T01:36:11.660612+0000","last_fullsized":"2023-12-01T01:36:11.660612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:58.572550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659519+0000","last_change":"2023-12-01T01:36:10.685524+0000","last_active":"2023-12-01T01:36:11.659519+0000","last_peered":"2023-12-01T01:36:11.659519+0000","last_clean":"2023-12-01T01:36:11.659519+0000","last_became_active":"2023-12-01T01:36:10.685227+0000","last_became_peered":"2023-12-01T01:36:10.685227+0000","last_unstale":"2023-12-01T01:36:11.659519+0000","last_undegraded":"2023-12-01T01:36:11.659519+0000","last_fullsized":"2023-12-01T01:36:11.659519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660039+0000","last_change":"2023-12-01T01:36:10.673805+0000","last_active":"2023-12-01T01:36:11.660039+0000","last_peered":"2023-12-01T01:36:11.660039+0000","last_clean":"2023-12-01T01:36:11.660039+0000","last_became_active":"2023-12-01T01:36:10.672297+0000","last_became_peered":"2023-12-01T01:36:10.672297+0000","last_unstale":"2023-12-01T01:36:11.660039+0000","last_undegraded":"2023-12-01T01:36:11.660039+0000","last_fullsized":"2023-12-01T01:36:11.660039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846363+0000","last_change":"2023-12-01T01:36:10.668469+0000","last_active":"2023-12-01T01:36:11.846363+0000","last_peered":"2023-12-01T01:36:11.846363+0000","last_clean":"2023-12-01T01:36:11.846363+0000","last_became_active":"2023-12-01T01:36:10.667900+0000","last_became_peered":"2023-12-01T01:36:10.667900+0000","last_unstale":"2023-12-01T01:36:11.846363+0000","last_undegraded":"2023-12-01T01:36:11.846363+0000","last_fullsized":"2023-12-01T01:36:11.846363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660756+0000","last_change":"2023-12-01T01:36:10.666824+0000","last_active":"2023-12-01T01:36:11.660756+0000","last_peered":"2023-12-01T01:36:11.660756+0000","last_clean":"2023-12-01T01:36:11.660756+0000","last_became_active":"2023-12-01T01:36:10.666614+0000","last_became_peered":"2023-12-01T01:36:10.666614+0000","last_unstale":"2023-12-01T01:36:11.660756+0000","last_undegraded":"2023-12-01T01:36:11.660756+0000","last_fullsized":"2023-12-01T01:36:11.660756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:02.974458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843911+0000","last_change":"2023-12-01T01:36:10.676262+0000","last_active":"2023-12-01T01:36:11.843911+0000","last_peered":"2023-12-01T01:36:11.843911+0000","last_clean":"2023-12-01T01:36:11.843911+0000","last_became_active":"2023-12-01T01:36:10.674999+0000","last_became_peered":"2023-12-01T01:36:10.674999+0000","last_unstale":"2023-12-01T01:36:11.843911+0000","last_undegraded":"2023-12-01T01:36:11.843911+0000","last_fullsized":"2023-12-01T01:36:11.843911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843925+0000","last_change":"2023-12-01T01:36:10.660058+0000","last_active":"2023-12-01T01:36:11.843925+0000","last_peered":"2023-12-01T01:36:11.843925+0000","last_clean":"2023-12-01T01:36:11.843925+0000","last_became_active":"2023-12-01T01:36:10.659667+0000","last_became_peered":"2023-12-01T01:36:10.659667+0000","last_unstale":"2023-12-01T01:36:11.843925+0000","last_undegraded":"2023-12-01T01:36:11.843925+0000","last_fullsized":"2023-12-01T01:36:11.843925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659022+0000","last_change":"2023-12-01T01:36:10.681619+0000","last_active":"2023-12-01T01:36:11.659022+0000","last_peered":"2023-12-01T01:36:11.659022+0000","last_clean":"2023-12-01T01:36:11.659022+0000","last_became_active":"2023-12-01T01:36:10.681370+0000","last_became_peered":"2023-12-01T01:36:10.681370+0000","last_unstale":"2023-12-01T01:36:11.659022+0000","last_undegraded":"2023-12-01T01:36:11.659022+0000","last_fullsized":"2023-12-01T01:36:11.659022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:05:04.661902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844932+0000","last_change":"2023-12-01T01:36:10.663979+0000","last_active":"2023-12-01T01:36:11.844932+0000","last_peered":"2023-12-01T01:36:11.844932+0000","last_clean":"2023-12-01T01:36:11.844932+0000","last_became_active":"2023-12-01T01:36:10.663730+0000","last_became_peered":"2023-12-01T01:36:10.663730+0000","last_unstale":"2023-12-01T01:36:11.844932+0000","last_undegraded":"2023-12-01T01:36:11.844932+0000","last_fullsized":"2023-12-01T01:36:11.844932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:44:55.685435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843066+0000","last_change":"2023-12-01T01:36:10.681889+0000","last_active":"2023-12-01T01:36:11.843066+0000","last_peered":"2023-12-01T01:36:11.843066+0000","last_clean":"2023-12-01T01:36:11.843066+0000","last_became_active":"2023-12-01T01:36:10.681607+0000","last_became_peered":"2023-12-01T01:36:10.681607+0000","last_unstale":"2023-12-01T01:36:11.843066+0000","last_undegraded":"2023-12-01T01:36:11.843066+0000","last_fullsized":"2023-12-01T01:36:11.843066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:10.875571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844651+0000","last_change":"2023-12-01T01:36:10.681120+0000","last_active":"2023-12-01T01:36:11.844651+0000","last_peered":"2023-12-01T01:36:11.844651+0000","last_clean":"2023-12-01T01:36:11.844651+0000","last_became_active":"2023-12-01T01:36:10.680861+0000","last_became_peered":"2023-12-01T01:36:10.680861+0000","last_unstale":"2023-12-01T01:36:11.844651+0000","last_undegraded":"2023-12-01T01:36:11.844651+0000","last_fullsized":"2023-12-01T01:36:11.844651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:14:40.275119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843591+0000","last_change":"2023-12-01T01:36:10.671802+0000","last_active":"2023-12-01T01:36:11.843591+0000","last_peered":"2023-12-01T01:36:11.843591+0000","last_clean":"2023-12-01T01:36:11.843591+0000","last_became_active":"2023-12-01T01:36:10.671492+0000","last_became_peered":"2023-12-01T01:36:10.671492+0000","last_unstale":"2023-12-01T01:36:11.843591+0000","last_undegraded":"2023-12-01T01:36:11.843591+0000","last_fullsized":"2023-12-01T01:36:11.843591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:13:49.007903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844344+0000","last_change":"2023-12-01T01:36:10.679590+0000","last_active":"2023-12-01T01:36:11.844344+0000","last_peered":"2023-12-01T01:36:11.844344+0000","last_clean":"2023-12-01T01:36:11.844344+0000","last_became_active":"2023-12-01T01:36:10.679306+0000","last_became_peered":"2023-12-01T01:36:10.679306+0000","last_unstale":"2023-12-01T01:36:11.844344+0000","last_undegraded":"2023-12-01T01:36:11.844344+0000","last_fullsized":"2023-12-01T01:36:11.844344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659556+0000","last_change":"2023-12-01T01:36:10.671990+0000","last_active":"2023-12-01T01:36:11.659556+0000","last_peered":"2023-12-01T01:36:11.659556+0000","last_clean":"2023-12-01T01:36:11.659556+0000","last_became_active":"2023-12-01T01:36:10.666648+0000","last_became_peered":"2023-12-01T01:36:10.666648+0000","last_unstale":"2023-12-01T01:36:11.659556+0000","last_undegraded":"2023-12-01T01:36:11.659556+0000","last_fullsized":"2023-12-01T01:36:11.659556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:51:53.546128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.658760+0000","last_change":"2023-12-01T01:36:10.677811+0000","last_active":"2023-12-01T01:36:11.658760+0000","last_peered":"2023-12-01T01:36:11.658760+0000","last_clean":"2023-12-01T01:36:11.658760+0000","last_became_active":"2023-12-01T01:36:10.677266+0000","last_became_peered":"2023-12-01T01:36:10.677266+0000","last_unstale":"2023-12-01T01:36:11.658760+0000","last_undegraded":"2023-12-01T01:36:11.658760+0000","last_fullsized":"2023-12-01T01:36:11.658760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842525+0000","last_change":"2023-12-01T01:36:10.664264+0000","last_active":"2023-12-01T01:36:11.842525+0000","last_peered":"2023-12-01T01:36:11.842525+0000","last_clean":"2023-12-01T01:36:11.842525+0000","last_became_active":"2023-12-01T01:36:10.663947+0000","last_became_peered":"2023-12-01T01:36:10.663947+0000","last_unstale":"2023-12-01T01:36:11.842525+0000","last_undegraded":"2023-12-01T01:36:11.842525+0000","last_fullsized":"2023-12-01T01:36:11.842525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844143+0000","last_change":"2023-12-01T01:36:10.660556+0000","last_active":"2023-12-01T01:36:11.844143+0000","last_peered":"2023-12-01T01:36:11.844143+0000","last_clean":"2023-12-01T01:36:11.844143+0000","last_became_active":"2023-12-01T01:36:10.658792+0000","last_became_peered":"2023-12-01T01:36:10.658792+0000","last_unstale":"2023-12-01T01:36:11.844143+0000","last_undegraded":"2023-12-01T01:36:11.844143+0000","last_fullsized":"2023-12-01T01:36:11.844143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:10.875571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.678963+0000","last_change":"2023-12-01T01:36:10.186029+0000","last_active":"2023-12-01T01:36:10.678963+0000","last_peered":"2023-12-01T01:36:10.678963+0000","last_clean":"2023-12-01T01:36:10.678963+0000","last_became_active":"2023-12-01T01:36:10.185703+0000","last_became_peered":"2023-12-01T01:36:10.185703+0000","last_unstale":"2023-12-01T01:36:10.678963+0000","last_undegraded":"2023-12-01T01:36:10.678963+0000","last_fullsized":"2023-12-01T01:36:10.678963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660789+0000","last_change":"2023-12-01T01:36:10.672533+0000","last_active":"2023-12-01T01:36:11.660789+0000","last_peered":"2023-12-01T01:36:11.660789+0000","last_clean":"2023-12-01T01:36:11.660789+0000","last_became_active":"2023-12-01T01:36:10.672202+0000","last_became_peered":"2023-12-01T01:36:10.672202+0000","last_unstale":"2023-12-01T01:36:11.660789+0000","last_undegraded":"2023-12-01T01:36:11.660789+0000","last_fullsized":"2023-12-01T01:36:11.660789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:49.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685718+0000","last_change":"2023-12-01T01:36:10.186695+0000","last_active":"2023-12-01T01:36:10.685718+0000","last_peered":"2023-12-01T01:36:10.685718+0000","last_clean":"2023-12-01T01:36:10.685718+0000","last_became_active":"2023-12-01T01:36:10.186437+0000","last_became_peered":"2023-12-01T01:36:10.186437+0000","last_unstale":"2023-12-01T01:36:10.685718+0000","last_undegraded":"2023-12-01T01:36:10.685718+0000","last_fullsized":"2023-12-01T01:36:10.685718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:52.363022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.649986+0000","last_change":"2023-12-01T01:36:10.192965+0000","last_active":"2023-12-01T01:36:10.649986+0000","last_peered":"2023-12-01T01:36:10.649986+0000","last_clean":"2023-12-01T01:36:10.649986+0000","last_became_active":"2023-12-01T01:36:10.192659+0000","last_became_peered":"2023-12-01T01:36:10.192659+0000","last_unstale":"2023-12-01T01:36:10.649986+0000","last_undegraded":"2023-12-01T01:36:10.649986+0000","last_fullsized":"2023-12-01T01:36:10.649986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844146+0000","last_change":"2023-12-01T01:36:10.665664+0000","last_active":"2023-12-01T01:36:11.844146+0000","last_peered":"2023-12-01T01:36:11.844146+0000","last_clean":"2023-12-01T01:36:11.844146+0000","last_became_active":"2023-12-01T01:36:10.662774+0000","last_became_peered":"2023-12-01T01:36:10.662774+0000","last_unstale":"2023-12-01T01:36:11.844146+0000","last_undegraded":"2023-12-01T01:36:11.844146+0000","last_fullsized":"2023-12-01T01:36:11.844146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.667386+0000","last_change":"2023-12-01T01:36:10.177490+0000","last_active":"2023-12-01T01:36:10.667386+0000","last_peered":"2023-12-01T01:36:10.667386+0000","last_clean":"2023-12-01T01:36:10.667386+0000","last_became_active":"2023-12-01T01:36:10.176976+0000","last_became_peered":"2023-12-01T01:36:10.176976+0000","last_unstale":"2023-12-01T01:36:10.667386+0000","last_undegraded":"2023-12-01T01:36:10.667386+0000","last_fullsized":"2023-12-01T01:36:10.667386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:09.863515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660588+0000","last_change":"2023-12-01T01:36:10.663142+0000","last_active":"2023-12-01T01:36:11.660588+0000","last_peered":"2023-12-01T01:36:11.660588+0000","last_clean":"2023-12-01T01:36:11.660588+0000","last_became_active":"2023-12-01T01:36:10.662866+0000","last_became_peered":"2023-12-01T01:36:10.662866+0000","last_unstale":"2023-12-01T01:36:11.660588+0000","last_undegraded":"2023-12-01T01:36:11.660588+0000","last_fullsized":"2023-12-01T01:36:11.660588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:25.357183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.647474+0000","last_change":"2023-12-01T01:36:10.189645+0000","last_active":"2023-12-01T01:36:10.647474+0000","last_peered":"2023-12-01T01:36:10.647474+0000","last_clean":"2023-12-01T01:36:10.647474+0000","last_became_active":"2023-12-01T01:36:10.189381+0000","last_became_peered":"2023-12-01T01:36:10.189381+0000","last_unstale":"2023-12-01T01:36:10.647474+0000","last_undegraded":"2023-12-01T01:36:10.647474+0000","last_fullsized":"2023-12-01T01:36:10.647474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843160+0000","last_change":"2023-12-01T01:36:10.679049+0000","last_active":"2023-12-01T01:36:11.843160+0000","last_peered":"2023-12-01T01:36:11.843160+0000","last_clean":"2023-12-01T01:36:11.843160+0000","last_became_active":"2023-12-01T01:36:10.678600+0000","last_became_peered":"2023-12-01T01:36:10.678600+0000","last_unstale":"2023-12-01T01:36:11.843160+0000","last_undegraded":"2023-12-01T01:36:11.843160+0000","last_fullsized":"2023-12-01T01:36:11.843160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650463+0000","last_change":"2023-12-01T01:36:10.190157+0000","last_active":"2023-12-01T01:36:10.650463+0000","last_peered":"2023-12-01T01:36:10.650463+0000","last_clean":"2023-12-01T01:36:10.650463+0000","last_became_active":"2023-12-01T01:36:10.189519+0000","last_became_peered":"2023-12-01T01:36:10.189519+0000","last_unstale":"2023-12-01T01:36:10.650463+0000","last_undegraded":"2023-12-01T01:36:10.650463+0000","last_fullsized":"2023-12-01T01:36:10.650463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845357+0000","last_change":"2023-12-01T01:36:10.670783+0000","last_active":"2023-12-01T01:36:11.845357+0000","last_peered":"2023-12-01T01:36:11.845357+0000","last_clean":"2023-12-01T01:36:11.845357+0000","last_became_active":"2023-12-01T01:36:10.669960+0000","last_became_peered":"2023-12-01T01:36:10.669960+0000","last_unstale":"2023-12-01T01:36:11.845357+0000","last_undegraded":"2023-12-01T01:36:11.845357+0000","last_fullsized":"2023-12-01T01:36:11.845357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:33.765782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.686400+0000","last_change":"2023-12-01T01:36:10.191392+0000","last_active":"2023-12-01T01:36:10.686400+0000","last_peered":"2023-12-01T01:36:10.686400+0000","last_clean":"2023-12-01T01:36:10.686400+0000","last_became_active":"2023-12-01T01:36:10.191010+0000","last_became_peered":"2023-12-01T01:36:10.191010+0000","last_unstale":"2023-12-01T01:36:10.686400+0000","last_undegraded":"2023-12-01T01:36:10.686400+0000","last_fullsized":"2023-12-01T01:36:10.686400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659710+0000","last_change":"2023-12-01T01:36:10.661986+0000","last_active":"2023-12-01T01:36:11.659710+0000","last_peered":"2023-12-01T01:36:11.659710+0000","last_clean":"2023-12-01T01:36:11.659710+0000","last_became_active":"2023-12-01T01:36:10.661711+0000","last_became_peered":"2023-12-01T01:36:10.661711+0000","last_unstale":"2023-12-01T01:36:11.659710+0000","last_undegraded":"2023-12-01T01:36:11.659710+0000","last_fullsized":"2023-12-01T01:36:11.659710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:12.241846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.688331+0000","last_change":"2023-12-01T01:36:10.190140+0000","last_active":"2023-12-01T01:36:10.688331+0000","last_peered":"2023-12-01T01:36:10.688331+0000","last_clean":"2023-12-01T01:36:10.688331+0000","last_became_active":"2023-12-01T01:36:10.188743+0000","last_became_peered":"2023-12-01T01:36:10.188743+0000","last_unstale":"2023-12-01T01:36:10.688331+0000","last_undegraded":"2023-12-01T01:36:10.688331+0000","last_fullsized":"2023-12-01T01:36:10.688331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:06:15.188480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846990+0000","last_change":"2023-12-01T01:36:10.670314+0000","last_active":"2023-12-01T01:36:11.846990+0000","last_peered":"2023-12-01T01:36:11.846990+0000","last_clean":"2023-12-01T01:36:11.846990+0000","last_became_active":"2023-12-01T01:36:10.669184+0000","last_became_peered":"2023-12-01T01:36:10.669184+0000","last_unstale":"2023-12-01T01:36:11.846990+0000","last_undegraded":"2023-12-01T01:36:11.846990+0000","last_fullsized":"2023-12-01T01:36:11.846990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.684491+0000","last_change":"2023-12-01T01:36:10.195876+0000","last_active":"2023-12-01T01:36:10.684491+0000","last_peered":"2023-12-01T01:36:10.684491+0000","last_clean":"2023-12-01T01:36:10.684491+0000","last_became_active":"2023-12-01T01:36:10.195018+0000","last_became_peered":"2023-12-01T01:36:10.195018+0000","last_unstale":"2023-12-01T01:36:10.684491+0000","last_undegraded":"2023-12-01T01:36:10.684491+0000","last_fullsized":"2023-12-01T01:36:10.684491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843370+0000","last_change":"2023-12-01T01:36:10.677075+0000","last_active":"2023-12-01T01:36:11.843370+0000","last_peered":"2023-12-01T01:36:11.843370+0000","last_clean":"2023-12-01T01:36:11.843370+0000","last_became_active":"2023-12-01T01:36:10.676560+0000","last_became_peered":"2023-12-01T01:36:10.676560+0000","last_unstale":"2023-12-01T01:36:11.843370+0000","last_undegraded":"2023-12-01T01:36:11.843370+0000","last_fullsized":"2023-12-01T01:36:11.843370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:41.649014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.660745+0000","last_change":"2023-12-01T01:36:10.179441+0000","last_active":"2023-12-01T01:36:10.660745+0000","last_peered":"2023-12-01T01:36:10.660745+0000","last_clean":"2023-12-01T01:36:10.660745+0000","last_became_active":"2023-12-01T01:36:10.178179+0000","last_became_peered":"2023-12-01T01:36:10.178179+0000","last_unstale":"2023-12-01T01:36:10.660745+0000","last_undegraded":"2023-12-01T01:36:10.660745+0000","last_fullsized":"2023-12-01T01:36:10.660745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:14.154709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659624+0000","last_change":"2023-12-01T01:36:10.665880+0000","last_active":"2023-12-01T01:36:11.659624+0000","last_peered":"2023-12-01T01:36:11.659624+0000","last_clean":"2023-12-01T01:36:11.659624+0000","last_became_active":"2023-12-01T01:36:10.665639+0000","last_became_peered":"2023-12-01T01:36:10.665639+0000","last_unstale":"2023-12-01T01:36:11.659624+0000","last_undegraded":"2023-12-01T01:36:11.659624+0000","last_fullsized":"2023-12-01T01:36:11.659624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:47.015657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650017+0000","last_change":"2023-12-01T01:36:10.192976+0000","last_active":"2023-12-01T01:36:10.650017+0000","last_peered":"2023-12-01T01:36:10.650017+0000","last_clean":"2023-12-01T01:36:10.650017+0000","last_became_active":"2023-12-01T01:36:10.192692+0000","last_became_peered":"2023-12-01T01:36:10.192692+0000","last_unstale":"2023-12-01T01:36:10.650017+0000","last_undegraded":"2023-12-01T01:36:10.650017+0000","last_fullsized":"2023-12-01T01:36:10.650017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843415+0000","last_change":"2023-12-01T01:36:10.659027+0000","last_active":"2023-12-01T01:36:11.843415+0000","last_peered":"2023-12-01T01:36:11.843415+0000","last_clean":"2023-12-01T01:36:11.843415+0000","last_became_active":"2023-12-01T01:36:10.658683+0000","last_became_peered":"2023-12-01T01:36:10.658683+0000","last_unstale":"2023-12-01T01:36:11.843415+0000","last_undegraded":"2023-12-01T01:36:11.843415+0000","last_fullsized":"2023-12-01T01:36:11.843415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.687400+0000","last_change":"2023-12-01T01:36:10.180654+0000","last_active":"2023-12-01T01:36:10.687400+0000","last_peered":"2023-12-01T01:36:10.687400+0000","last_clean":"2023-12-01T01:36:10.687400+0000","last_became_active":"2023-12-01T01:36:10.179301+0000","last_became_peered":"2023-12-01T01:36:10.179301+0000","last_unstale":"2023-12-01T01:36:10.687400+0000","last_undegraded":"2023-12-01T01:36:10.687400+0000","last_fullsized":"2023-12-01T01:36:10.687400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659085+0000","last_change":"2023-12-01T01:36:10.664772+0000","last_active":"2023-12-01T01:36:11.659085+0000","last_peered":"2023-12-01T01:36:11.659085+0000","last_clean":"2023-12-01T01:36:11.659085+0000","last_became_active":"2023-12-01T01:36:10.664504+0000","last_became_peered":"2023-12-01T01:36:10.664504+0000","last_unstale":"2023-12-01T01:36:11.659085+0000","last_undegraded":"2023-12-01T01:36:11.659085+0000","last_fullsized":"2023-12-01T01:36:11.659085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:44:20.605895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.686349+0000","last_change":"2023-12-01T01:36:10.189157+0000","last_active":"2023-12-01T01:36:10.686349+0000","last_peered":"2023-12-01T01:36:10.686349+0000","last_clean":"2023-12-01T01:36:10.686349+0000","last_became_active":"2023-12-01T01:36:10.188580+0000","last_became_peered":"2023-12-01T01:36:10.188580+0000","last_unstale":"2023-12-01T01:36:10.686349+0000","last_undegraded":"2023-12-01T01:36:10.686349+0000","last_fullsized":"2023-12-01T01:36:10.686349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843263+0000","last_change":"2023-12-01T01:36:10.666504+0000","last_active":"2023-12-01T01:36:11.843263+0000","last_peered":"2023-12-01T01:36:11.843263+0000","last_clean":"2023-12-01T01:36:11.843263+0000","last_became_active":"2023-12-01T01:36:10.666252+0000","last_became_peered":"2023-12-01T01:36:10.666252+0000","last_unstale":"2023-12-01T01:36:11.843263+0000","last_undegraded":"2023-12-01T01:36:11.843263+0000","last_fullsized":"2023-12-01T01:36:11.843263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:41.649014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683819+0000","last_change":"2023-12-01T01:36:10.186429+0000","last_active":"2023-12-01T01:36:10.683819+0000","last_peered":"2023-12-01T01:36:10.683819+0000","last_clean":"2023-12-01T01:36:10.683819+0000","last_became_active":"2023-12-01T01:36:10.185294+0000","last_became_peered":"2023-12-01T01:36:10.185294+0000","last_unstale":"2023-12-01T01:36:10.683819+0000","last_undegraded":"2023-12-01T01:36:10.683819+0000","last_fullsized":"2023-12-01T01:36:10.683819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847638+0000","last_change":"2023-12-01T01:36:10.667398+0000","last_active":"2023-12-01T01:36:11.847638+0000","last_peered":"2023-12-01T01:36:11.847638+0000","last_clean":"2023-12-01T01:36:11.847638+0000","last_became_active":"2023-12-01T01:36:10.667108+0000","last_became_peered":"2023-12-01T01:36:10.667108+0000","last_unstale":"2023-12-01T01:36:11.847638+0000","last_undegraded":"2023-12-01T01:36:11.847638+0000","last_fullsized":"2023-12-01T01:36:11.847638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:41:51.516524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.680767+0000","last_change":"2023-12-01T01:36:10.198699+0000","last_active":"2023-12-01T01:36:10.680767+0000","last_peered":"2023-12-01T01:36:10.680767+0000","last_clean":"2023-12-01T01:36:10.680767+0000","last_became_active":"2023-12-01T01:36:10.198340+0000","last_became_peered":"2023-12-01T01:36:10.198340+0000","last_unstale":"2023-12-01T01:36:10.680767+0000","last_undegraded":"2023-12-01T01:36:10.680767+0000","last_fullsized":"2023-12-01T01:36:10.680767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846471+0000","last_change":"2023-12-01T01:36:10.680027+0000","last_active":"2023-12-01T01:36:11.846471+0000","last_peered":"2023-12-01T01:36:11.846471+0000","last_clean":"2023-12-01T01:36:11.846471+0000","last_became_active":"2023-12-01T01:36:10.679670+0000","last_became_peered":"2023-12-01T01:36:10.679670+0000","last_unstale":"2023-12-01T01:36:11.846471+0000","last_undegraded":"2023-12-01T01:36:11.846471+0000","last_fullsized":"2023-12-01T01:36:11.846471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.657130+0000","last_change":"2023-12-01T01:36:10.194952+0000","last_active":"2023-12-01T01:36:10.657130+0000","last_peered":"2023-12-01T01:36:10.657130+0000","last_clean":"2023-12-01T01:36:10.657130+0000","last_became_active":"2023-12-01T01:36:10.194708+0000","last_became_peered":"2023-12-01T01:36:10.194708+0000","last_unstale":"2023-12-01T01:36:10.657130+0000","last_undegraded":"2023-12-01T01:36:10.657130+0000","last_fullsized":"2023-12-01T01:36:10.657130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659619+0000","last_change":"2023-12-01T01:36:10.663313+0000","last_active":"2023-12-01T01:36:11.659619+0000","last_peered":"2023-12-01T01:36:11.659619+0000","last_clean":"2023-12-01T01:36:11.659619+0000","last_became_active":"2023-12-01T01:36:10.662987+0000","last_became_peered":"2023-12-01T01:36:10.662987+0000","last_unstale":"2023-12-01T01:36:11.659619+0000","last_undegraded":"2023-12-01T01:36:11.659619+0000","last_fullsized":"2023-12-01T01:36:11.659619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.646527+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:36:10.646527+0000","last_peered":"2023-12-01T01:36:10.646527+0000","last_clean":"2023-12-01T01:36:10.646527+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:36:10.646527+0000","last_undegraded":"2023-12-01T01:36:10.646527+0000","last_fullsized":"2023-12-01T01:36:10.646527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.664313+0000","last_change":"2023-12-01T01:36:10.194117+0000","last_active":"2023-12-01T01:36:10.664313+0000","last_peered":"2023-12-01T01:36:10.664313+0000","last_clean":"2023-12-01T01:36:10.664313+0000","last_became_active":"2023-12-01T01:36:10.193801+0000","last_became_peered":"2023-12-01T01:36:10.193801+0000","last_unstale":"2023-12-01T01:36:10.664313+0000","last_undegraded":"2023-12-01T01:36:10.664313+0000","last_fullsized":"2023-12-01T01:36:10.664313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660654+0000","last_change":"2023-12-01T01:36:10.669848+0000","last_active":"2023-12-01T01:36:11.660654+0000","last_peered":"2023-12-01T01:36:11.660654+0000","last_clean":"2023-12-01T01:36:11.660654+0000","last_became_active":"2023-12-01T01:36:10.669557+0000","last_became_peered":"2023-12-01T01:36:10.669557+0000","last_unstale":"2023-12-01T01:36:11.660654+0000","last_undegraded":"2023-12-01T01:36:11.660654+0000","last_fullsized":"2023-12-01T01:36:11.660654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.667473+0000","last_change":"2023-12-01T01:36:10.183096+0000","last_active":"2023-12-01T01:36:10.667473+0000","last_peered":"2023-12-01T01:36:10.667473+0000","last_clean":"2023-12-01T01:36:10.667473+0000","last_became_active":"2023-12-01T01:36:10.182810+0000","last_became_peered":"2023-12-01T01:36:10.182810+0000","last_unstale":"2023-12-01T01:36:10.667473+0000","last_undegraded":"2023-12-01T01:36:10.667473+0000","last_fullsized":"2023-12-01T01:36:10.667473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:35.765727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844076+0000","last_change":"2023-12-01T01:36:10.661869+0000","last_active":"2023-12-01T01:36:11.844076+0000","last_peered":"2023-12-01T01:36:11.844076+0000","last_clean":"2023-12-01T01:36:11.844076+0000","last_became_active":"2023-12-01T01:36:10.661404+0000","last_became_peered":"2023-12-01T01:36:10.661404+0000","last_unstale":"2023-12-01T01:36:11.844076+0000","last_undegraded":"2023-12-01T01:36:11.844076+0000","last_fullsized":"2023-12-01T01:36:11.844076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.669601+0000","last_change":"2023-12-01T01:36:10.188465+0000","last_active":"2023-12-01T01:36:10.669601+0000","last_peered":"2023-12-01T01:36:10.669601+0000","last_clean":"2023-12-01T01:36:10.669601+0000","last_became_active":"2023-12-01T01:36:10.188157+0000","last_became_peered":"2023-12-01T01:36:10.188157+0000","last_unstale":"2023-12-01T01:36:10.669601+0000","last_undegraded":"2023-12-01T01:36:10.669601+0000","last_fullsized":"2023-12-01T01:36:10.669601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843928+0000","last_change":"2023-12-01T01:36:10.679247+0000","last_active":"2023-12-01T01:36:11.843928+0000","last_peered":"2023-12-01T01:36:11.843928+0000","last_clean":"2023-12-01T01:36:11.843928+0000","last_became_active":"2023-12-01T01:36:10.678097+0000","last_became_peered":"2023-12-01T01:36:10.678097+0000","last_unstale":"2023-12-01T01:36:11.843928+0000","last_undegraded":"2023-12-01T01:36:11.843928+0000","last_fullsized":"2023-12-01T01:36:11.843928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.661362+0000","last_change":"2023-12-01T01:36:10.182533+0000","last_active":"2023-12-01T01:36:10.661362+0000","last_peered":"2023-12-01T01:36:10.661362+0000","last_clean":"2023-12-01T01:36:10.661362+0000","last_became_active":"2023-12-01T01:36:10.182071+0000","last_became_peered":"2023-12-01T01:36:10.182071+0000","last_unstale":"2023-12-01T01:36:10.661362+0000","last_undegraded":"2023-12-01T01:36:10.661362+0000","last_fullsized":"2023-12-01T01:36:10.661362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:40.636957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846563+0000","last_change":"2023-12-01T01:36:10.666142+0000","last_active":"2023-12-01T01:36:11.846563+0000","last_peered":"2023-12-01T01:36:11.846563+0000","last_clean":"2023-12-01T01:36:11.846563+0000","last_became_active":"2023-12-01T01:36:10.665873+0000","last_became_peered":"2023-12-01T01:36:10.665873+0000","last_unstale":"2023-12-01T01:36:11.846563+0000","last_undegraded":"2023-12-01T01:36:11.846563+0000","last_fullsized":"2023-12-01T01:36:11.846563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.660940+0000","last_change":"2023-12-01T01:36:10.190758+0000","last_active":"2023-12-01T01:36:10.660940+0000","last_peered":"2023-12-01T01:36:10.660940+0000","last_clean":"2023-12-01T01:36:10.660940+0000","last_became_active":"2023-12-01T01:36:10.190280+0000","last_became_peered":"2023-12-01T01:36:10.190280+0000","last_unstale":"2023-12-01T01:36:10.660940+0000","last_undegraded":"2023-12-01T01:36:10.660940+0000","last_fullsized":"2023-12-01T01:36:10.660940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842977+0000","last_change":"2023-12-01T01:36:10.665004+0000","last_active":"2023-12-01T01:36:11.842977+0000","last_peered":"2023-12-01T01:36:11.842977+0000","last_clean":"2023-12-01T01:36:11.842977+0000","last_became_active":"2023-12-01T01:36:10.664692+0000","last_became_peered":"2023-12-01T01:36:10.664692+0000","last_unstale":"2023-12-01T01:36:11.842977+0000","last_undegraded":"2023-12-01T01:36:11.842977+0000","last_fullsized":"2023-12-01T01:36:11.842977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:53.375079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.666452+0000","last_change":"2023-12-01T01:36:10.186489+0000","last_active":"2023-12-01T01:36:10.666452+0000","last_peered":"2023-12-01T01:36:10.666452+0000","last_clean":"2023-12-01T01:36:10.666452+0000","last_became_active":"2023-12-01T01:36:10.186161+0000","last_became_peered":"2023-12-01T01:36:10.186161+0000","last_unstale":"2023-12-01T01:36:10.666452+0000","last_undegraded":"2023-12-01T01:36:10.666452+0000","last_fullsized":"2023-12-01T01:36:10.666452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843867+0000","last_change":"2023-12-01T01:36:10.663707+0000","last_active":"2023-12-01T01:36:11.843867+0000","last_peered":"2023-12-01T01:36:11.843867+0000","last_clean":"2023-12-01T01:36:11.843867+0000","last_became_active":"2023-12-01T01:36:10.663158+0000","last_became_peered":"2023-12-01T01:36:10.663158+0000","last_unstale":"2023-12-01T01:36:11.843867+0000","last_undegraded":"2023-12-01T01:36:11.843867+0000","last_fullsized":"2023-12-01T01:36:11.843867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.655254+0000","last_change":"2023-12-01T01:36:10.188465+0000","last_active":"2023-12-01T01:36:10.655254+0000","last_peered":"2023-12-01T01:36:10.655254+0000","last_clean":"2023-12-01T01:36:10.655254+0000","last_became_active":"2023-12-01T01:36:10.187631+0000","last_became_peered":"2023-12-01T01:36:10.187631+0000","last_unstale":"2023-12-01T01:36:10.655254+0000","last_undegraded":"2023-12-01T01:36:10.655254+0000","last_fullsized":"2023-12-01T01:36:10.655254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661611+0000","last_change":"2023-12-01T01:36:10.665647+0000","last_active":"2023-12-01T01:36:11.661611+0000","last_peered":"2023-12-01T01:36:11.661611+0000","last_clean":"2023-12-01T01:36:11.661611+0000","last_became_active":"2023-12-01T01:36:10.665371+0000","last_became_peered":"2023-12-01T01:36:10.665371+0000","last_unstale":"2023-12-01T01:36:11.661611+0000","last_undegraded":"2023-12-01T01:36:11.661611+0000","last_fullsized":"2023-12-01T01:36:11.661611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685898+0000","last_change":"2023-12-01T01:36:10.194331+0000","last_active":"2023-12-01T01:36:10.685898+0000","last_peered":"2023-12-01T01:36:10.685898+0000","last_clean":"2023-12-01T01:36:10.685898+0000","last_became_active":"2023-12-01T01:36:10.194058+0000","last_became_peered":"2023-12-01T01:36:10.194058+0000","last_unstale":"2023-12-01T01:36:10.685898+0000","last_undegraded":"2023-12-01T01:36:10.685898+0000","last_fullsized":"2023-12-01T01:36:10.685898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:51.741532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661128+0000","last_change":"2023-12-01T01:36:10.680546+0000","last_active":"2023-12-01T01:36:11.661128+0000","last_peered":"2023-12-01T01:36:11.661128+0000","last_clean":"2023-12-01T01:36:11.661128+0000","last_became_active":"2023-12-01T01:36:10.680099+0000","last_became_peered":"2023-12-01T01:36:10.680099+0000","last_unstale":"2023-12-01T01:36:11.661128+0000","last_undegraded":"2023-12-01T01:36:11.661128+0000","last_fullsized":"2023-12-01T01:36:11.661128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:22.177011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650417+0000","last_change":"2023-12-01T01:36:10.193127+0000","last_active":"2023-12-01T01:36:10.650417+0000","last_peered":"2023-12-01T01:36:10.650417+0000","last_clean":"2023-12-01T01:36:10.650417+0000","last_became_active":"2023-12-01T01:36:10.192884+0000","last_became_peered":"2023-12-01T01:36:10.192884+0000","last_unstale":"2023-12-01T01:36:10.650417+0000","last_undegraded":"2023-12-01T01:36:10.650417+0000","last_fullsized":"2023-12-01T01:36:10.650417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659185+0000","last_change":"2023-12-01T01:36:10.677457+0000","last_active":"2023-12-01T01:36:11.659185+0000","last_peered":"2023-12-01T01:36:11.659185+0000","last_clean":"2023-12-01T01:36:11.659185+0000","last_became_active":"2023-12-01T01:36:10.677156+0000","last_became_peered":"2023-12-01T01:36:10.677156+0000","last_unstale":"2023-12-01T01:36:11.659185+0000","last_undegraded":"2023-12-01T01:36:11.659185+0000","last_fullsized":"2023-12-01T01:36:11.659185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:41.113753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.654051+0000","last_change":"2023-12-01T01:36:10.189035+0000","last_active":"2023-12-01T01:36:10.654051+0000","last_peered":"2023-12-01T01:36:10.654051+0000","last_clean":"2023-12-01T01:36:10.654051+0000","last_became_active":"2023-12-01T01:36:10.188370+0000","last_became_peered":"2023-12-01T01:36:10.188370+0000","last_unstale":"2023-12-01T01:36:10.654051+0000","last_undegraded":"2023-12-01T01:36:10.654051+0000","last_fullsized":"2023-12-01T01:36:10.654051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660520+0000","last_change":"2023-12-01T01:36:10.682836+0000","last_active":"2023-12-01T01:36:11.660520+0000","last_peered":"2023-12-01T01:36:11.660520+0000","last_clean":"2023-12-01T01:36:11.660520+0000","last_became_active":"2023-12-01T01:36:10.682361+0000","last_became_peered":"2023-12-01T01:36:10.682361+0000","last_unstale":"2023-12-01T01:36:11.660520+0000","last_undegraded":"2023-12-01T01:36:11.660520+0000","last_fullsized":"2023-12-01T01:36:11.660520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:27.561648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.651403+0000","last_change":"2023-12-01T01:36:10.190808+0000","last_active":"2023-12-01T01:36:10.651403+0000","last_peered":"2023-12-01T01:36:10.651403+0000","last_clean":"2023-12-01T01:36:10.651403+0000","last_became_active":"2023-12-01T01:36:10.190355+0000","last_became_peered":"2023-12-01T01:36:10.190355+0000","last_unstale":"2023-12-01T01:36:10.651403+0000","last_undegraded":"2023-12-01T01:36:10.651403+0000","last_fullsized":"2023-12-01T01:36:10.651403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659213+0000","last_change":"2023-12-01T01:36:10.685203+0000","last_active":"2023-12-01T01:36:11.659213+0000","last_peered":"2023-12-01T01:36:11.659213+0000","last_clean":"2023-12-01T01:36:11.659213+0000","last_became_active":"2023-12-01T01:36:10.684118+0000","last_became_peered":"2023-12-01T01:36:10.684118+0000","last_unstale":"2023-12-01T01:36:11.659213+0000","last_undegraded":"2023-12-01T01:36:11.659213+0000","last_fullsized":"2023-12-01T01:36:11.659213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683327+0000","last_change":"2023-12-01T01:36:10.194065+0000","last_active":"2023-12-01T01:36:10.683327+0000","last_peered":"2023-12-01T01:36:10.683327+0000","last_clean":"2023-12-01T01:36:10.683327+0000","last_became_active":"2023-12-01T01:36:10.193785+0000","last_became_peered":"2023-12-01T01:36:10.193785+0000","last_unstale":"2023-12-01T01:36:10.683327+0000","last_undegraded":"2023-12-01T01:36:10.683327+0000","last_fullsized":"2023-12-01T01:36:10.683327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659511+0000","last_change":"2023-12-01T01:36:10.654568+0000","last_active":"2023-12-01T01:36:11.659511+0000","last_peered":"2023-12-01T01:36:11.659511+0000","last_clean":"2023-12-01T01:36:11.659511+0000","last_became_active":"2023-12-01T01:36:10.654312+0000","last_became_peered":"2023-12-01T01:36:10.654312+0000","last_unstale":"2023-12-01T01:36:11.659511+0000","last_undegraded":"2023-12-01T01:36:11.659511+0000","last_fullsized":"2023-12-01T01:36:11.659511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:24.066059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659518+0000","last_change":"2023-12-01T01:36:10.672526+0000","last_active":"2023-12-01T01:36:11.659518+0000","last_peered":"2023-12-01T01:36:11.659518+0000","last_clean":"2023-12-01T01:36:11.659518+0000","last_became_active":"2023-12-01T01:36:10.671950+0000","last_became_peered":"2023-12-01T01:36:10.671950+0000","last_unstale":"2023-12-01T01:36:11.659518+0000","last_undegraded":"2023-12-01T01:36:11.659518+0000","last_fullsized":"2023-12-01T01:36:11.659518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:07.041952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.681098+0000","last_change":"2023-12-01T01:36:10.189066+0000","last_active":"2023-12-01T01:36:10.681098+0000","last_peered":"2023-12-01T01:36:10.681098+0000","last_clean":"2023-12-01T01:36:10.681098+0000","last_became_active":"2023-12-01T01:36:10.188395+0000","last_became_peered":"2023-12-01T01:36:10.188395+0000","last_unstale":"2023-12-01T01:36:10.681098+0000","last_undegraded":"2023-12-01T01:36:10.681098+0000","last_fullsized":"2023-12-01T01:36:10.681098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844042+0000","last_change":"2023-12-01T01:36:10.677457+0000","last_active":"2023-12-01T01:36:11.844042+0000","last_peered":"2023-12-01T01:36:11.844042+0000","last_clean":"2023-12-01T01:36:11.844042+0000","last_became_active":"2023-12-01T01:36:10.677200+0000","last_became_peered":"2023-12-01T01:36:10.677200+0000","last_unstale":"2023-12-01T01:36:11.844042+0000","last_undegraded":"2023-12-01T01:36:11.844042+0000","last_fullsized":"2023-12-01T01:36:11.844042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:53.375079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.681045+0000","last_change":"2023-12-01T01:36:10.193800+0000","last_active":"2023-12-01T01:36:10.681045+0000","last_peered":"2023-12-01T01:36:10.681045+0000","last_clean":"2023-12-01T01:36:10.681045+0000","last_became_active":"2023-12-01T01:36:10.193469+0000","last_became_peered":"2023-12-01T01:36:10.193469+0000","last_unstale":"2023-12-01T01:36:10.681045+0000","last_undegraded":"2023-12-01T01:36:10.681045+0000","last_fullsized":"2023-12-01T01:36:10.681045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845128+0000","last_change":"2023-12-01T01:36:10.665085+0000","last_active":"2023-12-01T01:36:11.845128+0000","last_peered":"2023-12-01T01:36:11.845128+0000","last_clean":"2023-12-01T01:36:11.845128+0000","last_became_active":"2023-12-01T01:36:10.663166+0000","last_became_peered":"2023-12-01T01:36:10.663166+0000","last_unstale":"2023-12-01T01:36:11.845128+0000","last_undegraded":"2023-12-01T01:36:11.845128+0000","last_fullsized":"2023-12-01T01:36:11.845128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:38.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683819+0000","last_change":"2023-12-01T01:36:10.188667+0000","last_active":"2023-12-01T01:36:10.683819+0000","last_peered":"2023-12-01T01:36:10.683819+0000","last_clean":"2023-12-01T01:36:10.683819+0000","last_became_active":"2023-12-01T01:36:10.188354+0000","last_became_peered":"2023-12-01T01:36:10.188354+0000","last_unstale":"2023-12-01T01:36:10.683819+0000","last_undegraded":"2023-12-01T01:36:10.683819+0000","last_fullsized":"2023-12-01T01:36:10.683819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845005+0000","last_change":"2023-12-01T01:36:10.660331+0000","last_active":"2023-12-01T01:36:11.845005+0000","last_peered":"2023-12-01T01:36:11.845005+0000","last_clean":"2023-12-01T01:36:11.845005+0000","last_became_active":"2023-12-01T01:36:10.660037+0000","last_became_peered":"2023-12-01T01:36:10.660037+0000","last_unstale":"2023-12-01T01:36:11.845005+0000","last_undegraded":"2023-12-01T01:36:11.845005+0000","last_fullsized":"2023-12-01T01:36:11.845005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:12.241846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685486+0000","last_change":"2023-12-01T01:36:10.189468+0000","last_active":"2023-12-01T01:36:10.685486+0000","last_peered":"2023-12-01T01:36:10.685486+0000","last_clean":"2023-12-01T01:36:10.685486+0000","last_became_active":"2023-12-01T01:36:10.187185+0000","last_became_peered":"2023-12-01T01:36:10.187185+0000","last_unstale":"2023-12-01T01:36:10.685486+0000","last_undegraded":"2023-12-01T01:36:10.685486+0000","last_fullsized":"2023-12-01T01:36:10.685486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":75693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.756}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.952}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5680000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":819200,"data_stored":666029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":77805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.744}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1920000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":76151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.788}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":815104,"data_stored":666007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7649999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":75625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.546}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.54}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:36:20.066 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:36:20.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.285+0000 7f1eacbd4640 1 Processor -- start 2023-12-01T01:36:20.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.285+0000 7f1eacbd4640 1 -- start start 2023-12-01T01:36:20.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.287+0000 7f1eacbd4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1ea806b270 0x7f1ea80ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.287+0000 7f1eacbd4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea80ce120 con 0x7f1ea81597d0 2023-12-01T01:36:20.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.287+0000 7f1eacbd4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea806b700 con 0x7f1ea80cdcc0 2023-12-01T01:36:20.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.287+0000 7f1eacbd4640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea806b9c0 con 0x7f1ea806b270 2023-12-01T01:36:20.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.287+0000 7f1ea77fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1ea81597d0 0x7f1ea815db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51314/0 (socket says 172.21.15.115:51314) 2023-12-01T01:36:20.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.287+0000 7f1ea77fe640 1 -- 172.21.15.115:0/3983269080 learned_addr learned my addr 172.21.15.115:0/3983269080 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:20.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.287+0000 7f1ea7fff640 1 --2- 172.21.15.115:0/3983269080 >> v2:172.21.15.115:3300/0 conn(0x7f1ea806b270 0x7f1ea80ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.288+0000 7f1ea67fc640 1 -- 172.21.15.115:0/3983269080 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403627583 0 0) 0x7f1ea80ce120 con 0x7f1ea81597d0 2023-12-01T01:36:20.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.288+0000 7f1ea67fc640 1 -- 172.21.15.115:0/3983269080 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e90005730 con 0x7f1ea81597d0 2023-12-01T01:36:20.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.288+0000 7f1ea7fff640 1 -- 172.21.15.115:0/3983269080 >> v1:172.21.15.142:6789/0 conn(0x7f1ea80cdcc0 legacy=0x7f1ea80adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.288+0000 7f1ea7fff640 1 -- 172.21.15.115:0/3983269080 >> v1:172.21.15.115:6789/0 conn(0x7f1ea81597d0 legacy=0x7f1ea815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.288+0000 7f1ea7fff640 1 -- 172.21.15.115:0/3983269080 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea806bca0 con 0x7f1ea806b270 2023-12-01T01:36:20.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.289+0000 7f1ea7fff640 1 --2- 172.21.15.115:0/3983269080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea806b270 0x7f1ea80ca350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009d90 tx=0x7f1ea002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8043d0fbc7c551b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:20.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.289+0000 7f1ea67fc640 1 -- 172.21.15.115:0/3983269080 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea002cbd0 con 0x7f1ea806b270 2023-12-01T01:36:20.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.289+0000 7f1ea67fc640 1 -- 172.21.15.115:0/3983269080 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea002cd30 con 0x7f1ea806b270 2023-12-01T01:36:20.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.289+0000 7f1ea67fc640 1 -- 172.21.15.115:0/3983269080 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea002cea0 con 0x7f1ea806b270 2023-12-01T01:36:20.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.289+0000 7f1eacbd4640 1 -- 172.21.15.115:0/3983269080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea806b270 msgr2=0x7f1ea80ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.289+0000 7f1eacbd4640 1 --2- 172.21.15.115:0/3983269080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea806b270 0x7f1ea80ca350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009d90 tx=0x7f1ea002be20 comp rx=0 tx=0).stop 2023-12-01T01:36:20.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.290+0000 7f1eacbd4640 1 -- 172.21.15.115:0/3983269080 shutdown_connections 2023-12-01T01:36:20.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.290+0000 7f1eacbd4640 1 --2- 172.21.15.115:0/3983269080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea806b270 0x7f1ea80ca350 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.290+0000 7f1eacbd4640 1 -- 172.21.15.115:0/3983269080 >> 172.21.15.115:0/3983269080 conn(0x7f1ea80ce640 msgr2=0x7f1ea80cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:20.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.290+0000 7f1eacbd4640 1 -- 172.21.15.115:0/3983269080 shutdown_connections 2023-12-01T01:36:20.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.290+0000 7f1eacbd4640 1 -- 172.21.15.115:0/3983269080 wait complete. 2023-12-01T01:36:20.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.291+0000 7f1eacbd4640 1 Processor -- start 2023-12-01T01:36:20.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.292+0000 7f1eacbd4640 1 -- start start 2023-12-01T01:36:20.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.292+0000 7f1eacbd4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea806b270 0x7f1ea814af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.292+0000 7f1eacbd4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ea80cdcc0 0x7f1ea8153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1eacbd4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea81597d0 0x7f1ea80adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1eacbd4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8154f40 con 0x7f1ea81597d0 2023-12-01T01:36:20.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1eacbd4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea81541e0 con 0x7f1ea806b270 2023-12-01T01:36:20.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1eacbd4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8154610 con 0x7f1ea80cdcc0 2023-12-01T01:36:20.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea81597d0 0x7f1ea80adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea81597d0 0x7f1ea80adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55720/0 (socket says 172.21.15.115:55720) 2023-12-01T01:36:20.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1ea7fff640 1 -- 172.21.15.115:0/630665117 learned_addr learned my addr 172.21.15.115:0/630665117 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:20.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1ea77fe640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea806b270 0x7f1ea814af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1ea7fff640 1 -- 172.21.15.115:0/630665117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ea80cdcc0 msgr2=0x7f1ea8153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1ea7fff640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ea80cdcc0 0x7f1ea8153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1ea7fff640 1 -- 172.21.15.115:0/630665117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea806b270 msgr2=0x7f1ea814af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1ea7fff640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea806b270 0x7f1ea814af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.293+0000 7f1ea7fff640 1 -- 172.21.15.115:0/630665117 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea0009a70 con 0x7f1ea81597d0 2023-12-01T01:36:20.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.294+0000 7f1ea7fff640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea81597d0 0x7f1ea80adfc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009d90 tx=0x7f1ea0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.294+0000 7f1e8ffff640 1 -- 172.21.15.115:0/630665117 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea002cea0 con 0x7f1ea81597d0 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.294+0000 7f1e8ffff640 1 -- 172.21.15.115:0/630665117 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea0040670 con 0x7f1ea81597d0 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.294+0000 7f1eacbd4640 1 -- 172.21.15.115:0/630665117 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea81548f0 con 0x7f1ea81597d0 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.294+0000 7f1e8ffff640 1 -- 172.21.15.115:0/630665117 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0040a20 con 0x7f1ea81597d0 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.294+0000 7f1eacbd4640 1 -- 172.21.15.115:0/630665117 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea8151090 con 0x7f1ea81597d0 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.295+0000 7f1eacbd4640 1 -- 172.21.15.115:0/630665117 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea80673a0 con 0x7f1ea81597d0 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.296+0000 7f1e8ffff640 1 -- 172.21.15.115:0/630665117 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1ea0040b80 con 0x7f1ea81597d0 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.297+0000 7f1e8ffff640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e9406a460 0x7f1e9406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.297+0000 7f1e8ffff640 1 -- 172.21.15.115:0/630665117 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ea00ab350 con 0x7f1ea81597d0 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.297+0000 7f1ea77fe640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e9406a460 0x7f1e9406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.298+0000 7f1ea77fe640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e9406a460 0x7f1e9406c920 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1e980011e0 tx=0x7f1e98009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:20.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.301+0000 7f1e8ffff640 1 -- 172.21.15.115:0/630665117 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1ea0076930 con 0x7f1ea81597d0 2023-12-01T01:36:20.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.487+0000 7f1eacbd4640 1 -- 172.21.15.115:0/630665117 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ea815b080 con 0x7f1e9406a460 2023-12-01T01:36:20.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.509+0000 7f1e8ffff640 1 -- 172.21.15.115:0/630665117 <== mgr.4098 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264244 (secure 0 0 0) 0x7f1ea815b080 con 0x7f1e9406a460 2023-12-01T01:36:20.514 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:20.515 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 -- 172.21.15.115:0/630665117 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e9406a460 msgr2=0x7f1e9406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e9406a460 0x7f1e9406c920 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1e980011e0 tx=0x7f1e98009040 comp rx=0 tx=0).stop 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 -- 172.21.15.115:0/630665117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea81597d0 msgr2=0x7f1ea80adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea81597d0 0x7f1ea80adfc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009d90 tx=0x7f1ea0004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 -- 172.21.15.115:0/630665117 shutdown_connections 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e9406a460 0x7f1e9406c920 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ea80cdcc0 0x7f1ea8153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ea81597d0 0x7f1ea80adfc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 --2- 172.21.15.115:0/630665117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ea806b270 0x7f1ea814af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 -- 172.21.15.115:0/630665117 >> 172.21.15.115:0/630665117 conn(0x7f1ea80ce640 msgr2=0x7f1ea80ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:20.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 -- 172.21.15.115:0/630665117 shutdown_connections 2023-12-01T01:36:20.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.514+0000 7f1eacbd4640 1 -- 172.21.15.115:0/630665117 wait complete. 2023-12-01T01:36:20.535 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T01:36:20.158768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218536,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756184,"statfs":{"total":773094113280,"available":772870332416,"internally_reserved":0,"allocated":2990080,"data_stored":1790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.517117"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847556+0000","last_change":"2023-12-01T01:36:10.681289+0000","last_active":"2023-12-01T01:36:11.847556+0000","last_peered":"2023-12-01T01:36:11.847556+0000","last_clean":"2023-12-01T01:36:11.847556+0000","last_became_active":"2023-12-01T01:36:10.680689+0000","last_became_peered":"2023-12-01T01:36:10.680689+0000","last_unstale":"2023-12-01T01:36:11.847556+0000","last_undegraded":"2023-12-01T01:36:11.847556+0000","last_fullsized":"2023-12-01T01:36:11.847556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:12:29.670579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844746+0000","last_change":"2023-12-01T01:36:10.668026+0000","last_active":"2023-12-01T01:36:11.844746+0000","last_peered":"2023-12-01T01:36:11.844746+0000","last_clean":"2023-12-01T01:36:11.844746+0000","last_became_active":"2023-12-01T01:36:10.667717+0000","last_became_peered":"2023-12-01T01:36:10.667717+0000","last_unstale":"2023-12-01T01:36:11.844746+0000","last_undegraded":"2023-12-01T01:36:11.844746+0000","last_fullsized":"2023-12-01T01:36:11.844746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:12:29.670579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844579+0000","last_change":"2023-12-01T01:36:10.665172+0000","last_active":"2023-12-01T01:36:11.844579+0000","last_peered":"2023-12-01T01:36:11.844579+0000","last_clean":"2023-12-01T01:36:11.844579+0000","last_became_active":"2023-12-01T01:36:10.664880+0000","last_became_peered":"2023-12-01T01:36:10.664880+0000","last_unstale":"2023-12-01T01:36:11.844579+0000","last_undegraded":"2023-12-01T01:36:11.844579+0000","last_fullsized":"2023-12-01T01:36:11.844579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:41:51.516524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843255+0000","last_change":"2023-12-01T01:36:10.666933+0000","last_active":"2023-12-01T01:36:11.843255+0000","last_peered":"2023-12-01T01:36:11.843255+0000","last_clean":"2023-12-01T01:36:11.843255+0000","last_became_active":"2023-12-01T01:36:10.666669+0000","last_became_peered":"2023-12-01T01:36:10.666669+0000","last_unstale":"2023-12-01T01:36:11.843255+0000","last_undegraded":"2023-12-01T01:36:11.843255+0000","last_fullsized":"2023-12-01T01:36:11.843255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846486+0000","last_change":"2023-12-01T01:36:10.660927+0000","last_active":"2023-12-01T01:36:11.846486+0000","last_peered":"2023-12-01T01:36:11.846486+0000","last_clean":"2023-12-01T01:36:11.846486+0000","last_became_active":"2023-12-01T01:36:10.660693+0000","last_became_peered":"2023-12-01T01:36:10.660693+0000","last_unstale":"2023-12-01T01:36:11.846486+0000","last_undegraded":"2023-12-01T01:36:11.846486+0000","last_fullsized":"2023-12-01T01:36:11.846486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:02.974458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842420+0000","last_change":"2023-12-01T01:36:10.670715+0000","last_active":"2023-12-01T01:36:11.842420+0000","last_peered":"2023-12-01T01:36:11.842420+0000","last_clean":"2023-12-01T01:36:11.842420+0000","last_became_active":"2023-12-01T01:36:10.669975+0000","last_became_peered":"2023-12-01T01:36:10.669975+0000","last_unstale":"2023-12-01T01:36:11.842420+0000","last_undegraded":"2023-12-01T01:36:11.842420+0000","last_fullsized":"2023-12-01T01:36:11.842420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845913+0000","last_change":"2023-12-01T01:36:10.670567+0000","last_active":"2023-12-01T01:36:11.845913+0000","last_peered":"2023-12-01T01:36:11.845913+0000","last_clean":"2023-12-01T01:36:11.845913+0000","last_became_active":"2023-12-01T01:36:10.670325+0000","last_became_peered":"2023-12-01T01:36:10.670325+0000","last_unstale":"2023-12-01T01:36:11.845913+0000","last_undegraded":"2023-12-01T01:36:11.845913+0000","last_fullsized":"2023-12-01T01:36:11.845913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:49.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843702+0000","last_change":"2023-12-01T01:36:10.672309+0000","last_active":"2023-12-01T01:36:11.843702+0000","last_peered":"2023-12-01T01:36:11.843702+0000","last_clean":"2023-12-01T01:36:11.843702+0000","last_became_active":"2023-12-01T01:36:10.671949+0000","last_became_peered":"2023-12-01T01:36:10.671949+0000","last_unstale":"2023-12-01T01:36:11.843702+0000","last_undegraded":"2023-12-01T01:36:11.843702+0000","last_fullsized":"2023-12-01T01:36:11.843702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:44:20.605895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844629+0000","last_change":"2023-12-01T01:36:10.664390+0000","last_active":"2023-12-01T01:36:11.844629+0000","last_peered":"2023-12-01T01:36:11.844629+0000","last_clean":"2023-12-01T01:36:11.844629+0000","last_became_active":"2023-12-01T01:36:10.663998+0000","last_became_peered":"2023-12-01T01:36:10.663998+0000","last_unstale":"2023-12-01T01:36:11.844629+0000","last_undegraded":"2023-12-01T01:36:11.844629+0000","last_fullsized":"2023-12-01T01:36:11.844629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847720+0000","last_change":"2023-12-01T01:36:10.662705+0000","last_active":"2023-12-01T01:36:11.847720+0000","last_peered":"2023-12-01T01:36:11.847720+0000","last_clean":"2023-12-01T01:36:11.847720+0000","last_became_active":"2023-12-01T01:36:10.662424+0000","last_became_peered":"2023-12-01T01:36:10.662424+0000","last_unstale":"2023-12-01T01:36:11.847720+0000","last_undegraded":"2023-12-01T01:36:11.847720+0000","last_fullsized":"2023-12-01T01:36:11.847720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:44:55.685435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846761+0000","last_change":"2023-12-01T01:36:10.669520+0000","last_active":"2023-12-01T01:36:11.846761+0000","last_peered":"2023-12-01T01:36:11.846761+0000","last_clean":"2023-12-01T01:36:11.846761+0000","last_became_active":"2023-12-01T01:36:10.665644+0000","last_became_peered":"2023-12-01T01:36:10.665644+0000","last_unstale":"2023-12-01T01:36:11.846761+0000","last_undegraded":"2023-12-01T01:36:11.846761+0000","last_fullsized":"2023-12-01T01:36:11.846761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:38.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846407+0000","last_change":"2023-12-01T01:36:10.668672+0000","last_active":"2023-12-01T01:36:11.846407+0000","last_peered":"2023-12-01T01:36:11.846407+0000","last_clean":"2023-12-01T01:36:11.846407+0000","last_became_active":"2023-12-01T01:36:10.668180+0000","last_became_peered":"2023-12-01T01:36:10.668180+0000","last_unstale":"2023-12-01T01:36:11.846407+0000","last_undegraded":"2023-12-01T01:36:11.846407+0000","last_fullsized":"2023-12-01T01:36:11.846407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:33.765782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843462+0000","last_change":"2023-12-01T01:36:10.665068+0000","last_active":"2023-12-01T01:36:11.843462+0000","last_peered":"2023-12-01T01:36:11.843462+0000","last_clean":"2023-12-01T01:36:11.843462+0000","last_became_active":"2023-12-01T01:36:10.664845+0000","last_became_peered":"2023-12-01T01:36:10.664845+0000","last_unstale":"2023-12-01T01:36:11.843462+0000","last_undegraded":"2023-12-01T01:36:11.843462+0000","last_fullsized":"2023-12-01T01:36:11.843462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:27.561648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843064+0000","last_change":"2023-12-01T01:36:10.682708+0000","last_active":"2023-12-01T01:36:11.843064+0000","last_peered":"2023-12-01T01:36:11.843064+0000","last_clean":"2023-12-01T01:36:11.843064+0000","last_became_active":"2023-12-01T01:36:10.679209+0000","last_became_peered":"2023-12-01T01:36:10.679209+0000","last_unstale":"2023-12-01T01:36:11.843064+0000","last_undegraded":"2023-12-01T01:36:11.843064+0000","last_fullsized":"2023-12-01T01:36:11.843064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:25.357183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843449+0000","last_change":"2023-12-01T01:36:10.673333+0000","last_active":"2023-12-01T01:36:11.843449+0000","last_peered":"2023-12-01T01:36:11.843449+0000","last_clean":"2023-12-01T01:36:11.843449+0000","last_became_active":"2023-12-01T01:36:10.673021+0000","last_became_peered":"2023-12-01T01:36:10.673021+0000","last_unstale":"2023-12-01T01:36:11.843449+0000","last_undegraded":"2023-12-01T01:36:11.843449+0000","last_fullsized":"2023-12-01T01:36:11.843449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:24.066059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659010+0000","last_change":"2023-12-01T01:36:10.668827+0000","last_active":"2023-12-01T01:36:11.659010+0000","last_peered":"2023-12-01T01:36:11.659010+0000","last_clean":"2023-12-01T01:36:11.659010+0000","last_became_active":"2023-12-01T01:36:10.668507+0000","last_became_peered":"2023-12-01T01:36:10.668507+0000","last_unstale":"2023-12-01T01:36:11.659010+0000","last_undegraded":"2023-12-01T01:36:11.659010+0000","last_fullsized":"2023-12-01T01:36:11.659010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:29:32.707369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660612+0000","last_change":"2023-12-01T01:36:10.681970+0000","last_active":"2023-12-01T01:36:11.660612+0000","last_peered":"2023-12-01T01:36:11.660612+0000","last_clean":"2023-12-01T01:36:11.660612+0000","last_became_active":"2023-12-01T01:36:10.681726+0000","last_became_peered":"2023-12-01T01:36:10.681726+0000","last_unstale":"2023-12-01T01:36:11.660612+0000","last_undegraded":"2023-12-01T01:36:11.660612+0000","last_fullsized":"2023-12-01T01:36:11.660612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:58.572550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659519+0000","last_change":"2023-12-01T01:36:10.685524+0000","last_active":"2023-12-01T01:36:11.659519+0000","last_peered":"2023-12-01T01:36:11.659519+0000","last_clean":"2023-12-01T01:36:11.659519+0000","last_became_active":"2023-12-01T01:36:10.685227+0000","last_became_peered":"2023-12-01T01:36:10.685227+0000","last_unstale":"2023-12-01T01:36:11.659519+0000","last_undegraded":"2023-12-01T01:36:11.659519+0000","last_fullsized":"2023-12-01T01:36:11.659519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660039+0000","last_change":"2023-12-01T01:36:10.673805+0000","last_active":"2023-12-01T01:36:11.660039+0000","last_peered":"2023-12-01T01:36:11.660039+0000","last_clean":"2023-12-01T01:36:11.660039+0000","last_became_active":"2023-12-01T01:36:10.672297+0000","last_became_peered":"2023-12-01T01:36:10.672297+0000","last_unstale":"2023-12-01T01:36:11.660039+0000","last_undegraded":"2023-12-01T01:36:11.660039+0000","last_fullsized":"2023-12-01T01:36:11.660039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846363+0000","last_change":"2023-12-01T01:36:10.668469+0000","last_active":"2023-12-01T01:36:11.846363+0000","last_peered":"2023-12-01T01:36:11.846363+0000","last_clean":"2023-12-01T01:36:11.846363+0000","last_became_active":"2023-12-01T01:36:10.667900+0000","last_became_peered":"2023-12-01T01:36:10.667900+0000","last_unstale":"2023-12-01T01:36:11.846363+0000","last_undegraded":"2023-12-01T01:36:11.846363+0000","last_fullsized":"2023-12-01T01:36:11.846363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660756+0000","last_change":"2023-12-01T01:36:10.666824+0000","last_active":"2023-12-01T01:36:11.660756+0000","last_peered":"2023-12-01T01:36:11.660756+0000","last_clean":"2023-12-01T01:36:11.660756+0000","last_became_active":"2023-12-01T01:36:10.666614+0000","last_became_peered":"2023-12-01T01:36:10.666614+0000","last_unstale":"2023-12-01T01:36:11.660756+0000","last_undegraded":"2023-12-01T01:36:11.660756+0000","last_fullsized":"2023-12-01T01:36:11.660756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:02.974458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843911+0000","last_change":"2023-12-01T01:36:10.676262+0000","last_active":"2023-12-01T01:36:11.843911+0000","last_peered":"2023-12-01T01:36:11.843911+0000","last_clean":"2023-12-01T01:36:11.843911+0000","last_became_active":"2023-12-01T01:36:10.674999+0000","last_became_peered":"2023-12-01T01:36:10.674999+0000","last_unstale":"2023-12-01T01:36:11.843911+0000","last_undegraded":"2023-12-01T01:36:11.843911+0000","last_fullsized":"2023-12-01T01:36:11.843911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843925+0000","last_change":"2023-12-01T01:36:10.660058+0000","last_active":"2023-12-01T01:36:11.843925+0000","last_peered":"2023-12-01T01:36:11.843925+0000","last_clean":"2023-12-01T01:36:11.843925+0000","last_became_active":"2023-12-01T01:36:10.659667+0000","last_became_peered":"2023-12-01T01:36:10.659667+0000","last_unstale":"2023-12-01T01:36:11.843925+0000","last_undegraded":"2023-12-01T01:36:11.843925+0000","last_fullsized":"2023-12-01T01:36:11.843925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659022+0000","last_change":"2023-12-01T01:36:10.681619+0000","last_active":"2023-12-01T01:36:11.659022+0000","last_peered":"2023-12-01T01:36:11.659022+0000","last_clean":"2023-12-01T01:36:11.659022+0000","last_became_active":"2023-12-01T01:36:10.681370+0000","last_became_peered":"2023-12-01T01:36:10.681370+0000","last_unstale":"2023-12-01T01:36:11.659022+0000","last_undegraded":"2023-12-01T01:36:11.659022+0000","last_fullsized":"2023-12-01T01:36:11.659022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:05:04.661902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844932+0000","last_change":"2023-12-01T01:36:10.663979+0000","last_active":"2023-12-01T01:36:11.844932+0000","last_peered":"2023-12-01T01:36:11.844932+0000","last_clean":"2023-12-01T01:36:11.844932+0000","last_became_active":"2023-12-01T01:36:10.663730+0000","last_became_peered":"2023-12-01T01:36:10.663730+0000","last_unstale":"2023-12-01T01:36:11.844932+0000","last_undegraded":"2023-12-01T01:36:11.844932+0000","last_fullsized":"2023-12-01T01:36:11.844932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:44:55.685435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843066+0000","last_change":"2023-12-01T01:36:10.681889+0000","last_active":"2023-12-01T01:36:11.843066+0000","last_peered":"2023-12-01T01:36:11.843066+0000","last_clean":"2023-12-01T01:36:11.843066+0000","last_became_active":"2023-12-01T01:36:10.681607+0000","last_became_peered":"2023-12-01T01:36:10.681607+0000","last_unstale":"2023-12-01T01:36:11.843066+0000","last_undegraded":"2023-12-01T01:36:11.843066+0000","last_fullsized":"2023-12-01T01:36:11.843066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:10.875571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844651+0000","last_change":"2023-12-01T01:36:10.681120+0000","last_active":"2023-12-01T01:36:11.844651+0000","last_peered":"2023-12-01T01:36:11.844651+0000","last_clean":"2023-12-01T01:36:11.844651+0000","last_became_active":"2023-12-01T01:36:10.680861+0000","last_became_peered":"2023-12-01T01:36:10.680861+0000","last_unstale":"2023-12-01T01:36:11.844651+0000","last_undegraded":"2023-12-01T01:36:11.844651+0000","last_fullsized":"2023-12-01T01:36:11.844651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:14:40.275119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843591+0000","last_change":"2023-12-01T01:36:10.671802+0000","last_active":"2023-12-01T01:36:11.843591+0000","last_peered":"2023-12-01T01:36:11.843591+0000","last_clean":"2023-12-01T01:36:11.843591+0000","last_became_active":"2023-12-01T01:36:10.671492+0000","last_became_peered":"2023-12-01T01:36:10.671492+0000","last_unstale":"2023-12-01T01:36:11.843591+0000","last_undegraded":"2023-12-01T01:36:11.843591+0000","last_fullsized":"2023-12-01T01:36:11.843591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:13:49.007903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844344+0000","last_change":"2023-12-01T01:36:10.679590+0000","last_active":"2023-12-01T01:36:11.844344+0000","last_peered":"2023-12-01T01:36:11.844344+0000","last_clean":"2023-12-01T01:36:11.844344+0000","last_became_active":"2023-12-01T01:36:10.679306+0000","last_became_peered":"2023-12-01T01:36:10.679306+0000","last_unstale":"2023-12-01T01:36:11.844344+0000","last_undegraded":"2023-12-01T01:36:11.844344+0000","last_fullsized":"2023-12-01T01:36:11.844344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659556+0000","last_change":"2023-12-01T01:36:10.671990+0000","last_active":"2023-12-01T01:36:11.659556+0000","last_peered":"2023-12-01T01:36:11.659556+0000","last_clean":"2023-12-01T01:36:11.659556+0000","last_became_active":"2023-12-01T01:36:10.666648+0000","last_became_peered":"2023-12-01T01:36:10.666648+0000","last_unstale":"2023-12-01T01:36:11.659556+0000","last_undegraded":"2023-12-01T01:36:11.659556+0000","last_fullsized":"2023-12-01T01:36:11.659556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:51:53.546128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.658760+0000","last_change":"2023-12-01T01:36:10.677811+0000","last_active":"2023-12-01T01:36:11.658760+0000","last_peered":"2023-12-01T01:36:11.658760+0000","last_clean":"2023-12-01T01:36:11.658760+0000","last_became_active":"2023-12-01T01:36:10.677266+0000","last_became_peered":"2023-12-01T01:36:10.677266+0000","last_unstale":"2023-12-01T01:36:11.658760+0000","last_undegraded":"2023-12-01T01:36:11.658760+0000","last_fullsized":"2023-12-01T01:36:11.658760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842525+0000","last_change":"2023-12-01T01:36:10.664264+0000","last_active":"2023-12-01T01:36:11.842525+0000","last_peered":"2023-12-01T01:36:11.842525+0000","last_clean":"2023-12-01T01:36:11.842525+0000","last_became_active":"2023-12-01T01:36:10.663947+0000","last_became_peered":"2023-12-01T01:36:10.663947+0000","last_unstale":"2023-12-01T01:36:11.842525+0000","last_undegraded":"2023-12-01T01:36:11.842525+0000","last_fullsized":"2023-12-01T01:36:11.842525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844143+0000","last_change":"2023-12-01T01:36:10.660556+0000","last_active":"2023-12-01T01:36:11.844143+0000","last_peered":"2023-12-01T01:36:11.844143+0000","last_clean":"2023-12-01T01:36:11.844143+0000","last_became_active":"2023-12-01T01:36:10.658792+0000","last_became_peered":"2023-12-01T01:36:10.658792+0000","last_unstale":"2023-12-01T01:36:11.844143+0000","last_undegraded":"2023-12-01T01:36:11.844143+0000","last_fullsized":"2023-12-01T01:36:11.844143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:10.875571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.678963+0000","last_change":"2023-12-01T01:36:10.186029+0000","last_active":"2023-12-01T01:36:10.678963+0000","last_peered":"2023-12-01T01:36:10.678963+0000","last_clean":"2023-12-01T01:36:10.678963+0000","last_became_active":"2023-12-01T01:36:10.185703+0000","last_became_peered":"2023-12-01T01:36:10.185703+0000","last_unstale":"2023-12-01T01:36:10.678963+0000","last_undegraded":"2023-12-01T01:36:10.678963+0000","last_fullsized":"2023-12-01T01:36:10.678963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660789+0000","last_change":"2023-12-01T01:36:10.672533+0000","last_active":"2023-12-01T01:36:11.660789+0000","last_peered":"2023-12-01T01:36:11.660789+0000","last_clean":"2023-12-01T01:36:11.660789+0000","last_became_active":"2023-12-01T01:36:10.672202+0000","last_became_peered":"2023-12-01T01:36:10.672202+0000","last_unstale":"2023-12-01T01:36:11.660789+0000","last_undegraded":"2023-12-01T01:36:11.660789+0000","last_fullsized":"2023-12-01T01:36:11.660789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:49.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685718+0000","last_change":"2023-12-01T01:36:10.186695+0000","last_active":"2023-12-01T01:36:10.685718+0000","last_peered":"2023-12-01T01:36:10.685718+0000","last_clean":"2023-12-01T01:36:10.685718+0000","last_became_active":"2023-12-01T01:36:10.186437+0000","last_became_peered":"2023-12-01T01:36:10.186437+0000","last_unstale":"2023-12-01T01:36:10.685718+0000","last_undegraded":"2023-12-01T01:36:10.685718+0000","last_fullsized":"2023-12-01T01:36:10.685718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:52.363022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.649986+0000","last_change":"2023-12-01T01:36:10.192965+0000","last_active":"2023-12-01T01:36:10.649986+0000","last_peered":"2023-12-01T01:36:10.649986+0000","last_clean":"2023-12-01T01:36:10.649986+0000","last_became_active":"2023-12-01T01:36:10.192659+0000","last_became_peered":"2023-12-01T01:36:10.192659+0000","last_unstale":"2023-12-01T01:36:10.649986+0000","last_undegraded":"2023-12-01T01:36:10.649986+0000","last_fullsized":"2023-12-01T01:36:10.649986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844146+0000","last_change":"2023-12-01T01:36:10.665664+0000","last_active":"2023-12-01T01:36:11.844146+0000","last_peered":"2023-12-01T01:36:11.844146+0000","last_clean":"2023-12-01T01:36:11.844146+0000","last_became_active":"2023-12-01T01:36:10.662774+0000","last_became_peered":"2023-12-01T01:36:10.662774+0000","last_unstale":"2023-12-01T01:36:11.844146+0000","last_undegraded":"2023-12-01T01:36:11.844146+0000","last_fullsized":"2023-12-01T01:36:11.844146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.667386+0000","last_change":"2023-12-01T01:36:10.177490+0000","last_active":"2023-12-01T01:36:10.667386+0000","last_peered":"2023-12-01T01:36:10.667386+0000","last_clean":"2023-12-01T01:36:10.667386+0000","last_became_active":"2023-12-01T01:36:10.176976+0000","last_became_peered":"2023-12-01T01:36:10.176976+0000","last_unstale":"2023-12-01T01:36:10.667386+0000","last_undegraded":"2023-12-01T01:36:10.667386+0000","last_fullsized":"2023-12-01T01:36:10.667386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:09.863515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660588+0000","last_change":"2023-12-01T01:36:10.663142+0000","last_active":"2023-12-01T01:36:11.660588+0000","last_peered":"2023-12-01T01:36:11.660588+0000","last_clean":"2023-12-01T01:36:11.660588+0000","last_became_active":"2023-12-01T01:36:10.662866+0000","last_became_peered":"2023-12-01T01:36:10.662866+0000","last_unstale":"2023-12-01T01:36:11.660588+0000","last_undegraded":"2023-12-01T01:36:11.660588+0000","last_fullsized":"2023-12-01T01:36:11.660588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:25.357183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.647474+0000","last_change":"2023-12-01T01:36:10.189645+0000","last_active":"2023-12-01T01:36:10.647474+0000","last_peered":"2023-12-01T01:36:10.647474+0000","last_clean":"2023-12-01T01:36:10.647474+0000","last_became_active":"2023-12-01T01:36:10.189381+0000","last_became_peered":"2023-12-01T01:36:10.189381+0000","last_unstale":"2023-12-01T01:36:10.647474+0000","last_undegraded":"2023-12-01T01:36:10.647474+0000","last_fullsized":"2023-12-01T01:36:10.647474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843160+0000","last_change":"2023-12-01T01:36:10.679049+0000","last_active":"2023-12-01T01:36:11.843160+0000","last_peered":"2023-12-01T01:36:11.843160+0000","last_clean":"2023-12-01T01:36:11.843160+0000","last_became_active":"2023-12-01T01:36:10.678600+0000","last_became_peered":"2023-12-01T01:36:10.678600+0000","last_unstale":"2023-12-01T01:36:11.843160+0000","last_undegraded":"2023-12-01T01:36:11.843160+0000","last_fullsized":"2023-12-01T01:36:11.843160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650463+0000","last_change":"2023-12-01T01:36:10.190157+0000","last_active":"2023-12-01T01:36:10.650463+0000","last_peered":"2023-12-01T01:36:10.650463+0000","last_clean":"2023-12-01T01:36:10.650463+0000","last_became_active":"2023-12-01T01:36:10.189519+0000","last_became_peered":"2023-12-01T01:36:10.189519+0000","last_unstale":"2023-12-01T01:36:10.650463+0000","last_undegraded":"2023-12-01T01:36:10.650463+0000","last_fullsized":"2023-12-01T01:36:10.650463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845357+0000","last_change":"2023-12-01T01:36:10.670783+0000","last_active":"2023-12-01T01:36:11.845357+0000","last_peered":"2023-12-01T01:36:11.845357+0000","last_clean":"2023-12-01T01:36:11.845357+0000","last_became_active":"2023-12-01T01:36:10.669960+0000","last_became_peered":"2023-12-01T01:36:10.669960+0000","last_unstale":"2023-12-01T01:36:11.845357+0000","last_undegraded":"2023-12-01T01:36:11.845357+0000","last_fullsized":"2023-12-01T01:36:11.845357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:33.765782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.686400+0000","last_change":"2023-12-01T01:36:10.191392+0000","last_active":"2023-12-01T01:36:10.686400+0000","last_peered":"2023-12-01T01:36:10.686400+0000","last_clean":"2023-12-01T01:36:10.686400+0000","last_became_active":"2023-12-01T01:36:10.191010+0000","last_became_peered":"2023-12-01T01:36:10.191010+0000","last_unstale":"2023-12-01T01:36:10.686400+0000","last_undegraded":"2023-12-01T01:36:10.686400+0000","last_fullsized":"2023-12-01T01:36:10.686400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659710+0000","last_change":"2023-12-01T01:36:10.661986+0000","last_active":"2023-12-01T01:36:11.659710+0000","last_peered":"2023-12-01T01:36:11.659710+0000","last_clean":"2023-12-01T01:36:11.659710+0000","last_became_active":"2023-12-01T01:36:10.661711+0000","last_became_peered":"2023-12-01T01:36:10.661711+0000","last_unstale":"2023-12-01T01:36:11.659710+0000","last_undegraded":"2023-12-01T01:36:11.659710+0000","last_fullsized":"2023-12-01T01:36:11.659710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:12.241846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.688331+0000","last_change":"2023-12-01T01:36:10.190140+0000","last_active":"2023-12-01T01:36:10.688331+0000","last_peered":"2023-12-01T01:36:10.688331+0000","last_clean":"2023-12-01T01:36:10.688331+0000","last_became_active":"2023-12-01T01:36:10.188743+0000","last_became_peered":"2023-12-01T01:36:10.188743+0000","last_unstale":"2023-12-01T01:36:10.688331+0000","last_undegraded":"2023-12-01T01:36:10.688331+0000","last_fullsized":"2023-12-01T01:36:10.688331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:06:15.188480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846990+0000","last_change":"2023-12-01T01:36:10.670314+0000","last_active":"2023-12-01T01:36:11.846990+0000","last_peered":"2023-12-01T01:36:11.846990+0000","last_clean":"2023-12-01T01:36:11.846990+0000","last_became_active":"2023-12-01T01:36:10.669184+0000","last_became_peered":"2023-12-01T01:36:10.669184+0000","last_unstale":"2023-12-01T01:36:11.846990+0000","last_undegraded":"2023-12-01T01:36:11.846990+0000","last_fullsized":"2023-12-01T01:36:11.846990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.684491+0000","last_change":"2023-12-01T01:36:10.195876+0000","last_active":"2023-12-01T01:36:10.684491+0000","last_peered":"2023-12-01T01:36:10.684491+0000","last_clean":"2023-12-01T01:36:10.684491+0000","last_became_active":"2023-12-01T01:36:10.195018+0000","last_became_peered":"2023-12-01T01:36:10.195018+0000","last_unstale":"2023-12-01T01:36:10.684491+0000","last_undegraded":"2023-12-01T01:36:10.684491+0000","last_fullsized":"2023-12-01T01:36:10.684491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843370+0000","last_change":"2023-12-01T01:36:10.677075+0000","last_active":"2023-12-01T01:36:11.843370+0000","last_peered":"2023-12-01T01:36:11.843370+0000","last_clean":"2023-12-01T01:36:11.843370+0000","last_became_active":"2023-12-01T01:36:10.676560+0000","last_became_peered":"2023-12-01T01:36:10.676560+0000","last_unstale":"2023-12-01T01:36:11.843370+0000","last_undegraded":"2023-12-01T01:36:11.843370+0000","last_fullsized":"2023-12-01T01:36:11.843370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:41.649014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.660745+0000","last_change":"2023-12-01T01:36:10.179441+0000","last_active":"2023-12-01T01:36:10.660745+0000","last_peered":"2023-12-01T01:36:10.660745+0000","last_clean":"2023-12-01T01:36:10.660745+0000","last_became_active":"2023-12-01T01:36:10.178179+0000","last_became_peered":"2023-12-01T01:36:10.178179+0000","last_unstale":"2023-12-01T01:36:10.660745+0000","last_undegraded":"2023-12-01T01:36:10.660745+0000","last_fullsized":"2023-12-01T01:36:10.660745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:14.154709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659624+0000","last_change":"2023-12-01T01:36:10.665880+0000","last_active":"2023-12-01T01:36:11.659624+0000","last_peered":"2023-12-01T01:36:11.659624+0000","last_clean":"2023-12-01T01:36:11.659624+0000","last_became_active":"2023-12-01T01:36:10.665639+0000","last_became_peered":"2023-12-01T01:36:10.665639+0000","last_unstale":"2023-12-01T01:36:11.659624+0000","last_undegraded":"2023-12-01T01:36:11.659624+0000","last_fullsized":"2023-12-01T01:36:11.659624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:47.015657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650017+0000","last_change":"2023-12-01T01:36:10.192976+0000","last_active":"2023-12-01T01:36:10.650017+0000","last_peered":"2023-12-01T01:36:10.650017+0000","last_clean":"2023-12-01T01:36:10.650017+0000","last_became_active":"2023-12-01T01:36:10.192692+0000","last_became_peered":"2023-12-01T01:36:10.192692+0000","last_unstale":"2023-12-01T01:36:10.650017+0000","last_undegraded":"2023-12-01T01:36:10.650017+0000","last_fullsized":"2023-12-01T01:36:10.650017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843415+0000","last_change":"2023-12-01T01:36:10.659027+0000","last_active":"2023-12-01T01:36:11.843415+0000","last_peered":"2023-12-01T01:36:11.843415+0000","last_clean":"2023-12-01T01:36:11.843415+0000","last_became_active":"2023-12-01T01:36:10.658683+0000","last_became_peered":"2023-12-01T01:36:10.658683+0000","last_unstale":"2023-12-01T01:36:11.843415+0000","last_undegraded":"2023-12-01T01:36:11.843415+0000","last_fullsized":"2023-12-01T01:36:11.843415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.687400+0000","last_change":"2023-12-01T01:36:10.180654+0000","last_active":"2023-12-01T01:36:10.687400+0000","last_peered":"2023-12-01T01:36:10.687400+0000","last_clean":"2023-12-01T01:36:10.687400+0000","last_became_active":"2023-12-01T01:36:10.179301+0000","last_became_peered":"2023-12-01T01:36:10.179301+0000","last_unstale":"2023-12-01T01:36:10.687400+0000","last_undegraded":"2023-12-01T01:36:10.687400+0000","last_fullsized":"2023-12-01T01:36:10.687400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659085+0000","last_change":"2023-12-01T01:36:10.664772+0000","last_active":"2023-12-01T01:36:11.659085+0000","last_peered":"2023-12-01T01:36:11.659085+0000","last_clean":"2023-12-01T01:36:11.659085+0000","last_became_active":"2023-12-01T01:36:10.664504+0000","last_became_peered":"2023-12-01T01:36:10.664504+0000","last_unstale":"2023-12-01T01:36:11.659085+0000","last_undegraded":"2023-12-01T01:36:11.659085+0000","last_fullsized":"2023-12-01T01:36:11.659085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:44:20.605895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.686349+0000","last_change":"2023-12-01T01:36:10.189157+0000","last_active":"2023-12-01T01:36:10.686349+0000","last_peered":"2023-12-01T01:36:10.686349+0000","last_clean":"2023-12-01T01:36:10.686349+0000","last_became_active":"2023-12-01T01:36:10.188580+0000","last_became_peered":"2023-12-01T01:36:10.188580+0000","last_unstale":"2023-12-01T01:36:10.686349+0000","last_undegraded":"2023-12-01T01:36:10.686349+0000","last_fullsized":"2023-12-01T01:36:10.686349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843263+0000","last_change":"2023-12-01T01:36:10.666504+0000","last_active":"2023-12-01T01:36:11.843263+0000","last_peered":"2023-12-01T01:36:11.843263+0000","last_clean":"2023-12-01T01:36:11.843263+0000","last_became_active":"2023-12-01T01:36:10.666252+0000","last_became_peered":"2023-12-01T01:36:10.666252+0000","last_unstale":"2023-12-01T01:36:11.843263+0000","last_undegraded":"2023-12-01T01:36:11.843263+0000","last_fullsized":"2023-12-01T01:36:11.843263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:41.649014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683819+0000","last_change":"2023-12-01T01:36:10.186429+0000","last_active":"2023-12-01T01:36:10.683819+0000","last_peered":"2023-12-01T01:36:10.683819+0000","last_clean":"2023-12-01T01:36:10.683819+0000","last_became_active":"2023-12-01T01:36:10.185294+0000","last_became_peered":"2023-12-01T01:36:10.185294+0000","last_unstale":"2023-12-01T01:36:10.683819+0000","last_undegraded":"2023-12-01T01:36:10.683819+0000","last_fullsized":"2023-12-01T01:36:10.683819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847638+0000","last_change":"2023-12-01T01:36:10.667398+0000","last_active":"2023-12-01T01:36:11.847638+0000","last_peered":"2023-12-01T01:36:11.847638+0000","last_clean":"2023-12-01T01:36:11.847638+0000","last_became_active":"2023-12-01T01:36:10.667108+0000","last_became_peered":"2023-12-01T01:36:10.667108+0000","last_unstale":"2023-12-01T01:36:11.847638+0000","last_undegraded":"2023-12-01T01:36:11.847638+0000","last_fullsized":"2023-12-01T01:36:11.847638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:41:51.516524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.680767+0000","last_change":"2023-12-01T01:36:10.198699+0000","last_active":"2023-12-01T01:36:10.680767+0000","last_peered":"2023-12-01T01:36:10.680767+0000","last_clean":"2023-12-01T01:36:10.680767+0000","last_became_active":"2023-12-01T01:36:10.198340+0000","last_became_peered":"2023-12-01T01:36:10.198340+0000","last_unstale":"2023-12-01T01:36:10.680767+0000","last_undegraded":"2023-12-01T01:36:10.680767+0000","last_fullsized":"2023-12-01T01:36:10.680767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846471+0000","last_change":"2023-12-01T01:36:10.680027+0000","last_active":"2023-12-01T01:36:11.846471+0000","last_peered":"2023-12-01T01:36:11.846471+0000","last_clean":"2023-12-01T01:36:11.846471+0000","last_became_active":"2023-12-01T01:36:10.679670+0000","last_became_peered":"2023-12-01T01:36:10.679670+0000","last_unstale":"2023-12-01T01:36:11.846471+0000","last_undegraded":"2023-12-01T01:36:11.846471+0000","last_fullsized":"2023-12-01T01:36:11.846471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.657130+0000","last_change":"2023-12-01T01:36:10.194952+0000","last_active":"2023-12-01T01:36:10.657130+0000","last_peered":"2023-12-01T01:36:10.657130+0000","last_clean":"2023-12-01T01:36:10.657130+0000","last_became_active":"2023-12-01T01:36:10.194708+0000","last_became_peered":"2023-12-01T01:36:10.194708+0000","last_unstale":"2023-12-01T01:36:10.657130+0000","last_undegraded":"2023-12-01T01:36:10.657130+0000","last_fullsized":"2023-12-01T01:36:10.657130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659619+0000","last_change":"2023-12-01T01:36:10.663313+0000","last_active":"2023-12-01T01:36:11.659619+0000","last_peered":"2023-12-01T01:36:11.659619+0000","last_clean":"2023-12-01T01:36:11.659619+0000","last_became_active":"2023-12-01T01:36:10.662987+0000","last_became_peered":"2023-12-01T01:36:10.662987+0000","last_unstale":"2023-12-01T01:36:11.659619+0000","last_undegraded":"2023-12-01T01:36:11.659619+0000","last_fullsized":"2023-12-01T01:36:11.659619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.646527+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:36:10.646527+0000","last_peered":"2023-12-01T01:36:10.646527+0000","last_clean":"2023-12-01T01:36:10.646527+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:36:10.646527+0000","last_undegraded":"2023-12-01T01:36:10.646527+0000","last_fullsized":"2023-12-01T01:36:10.646527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.664313+0000","last_change":"2023-12-01T01:36:10.194117+0000","last_active":"2023-12-01T01:36:10.664313+0000","last_peered":"2023-12-01T01:36:10.664313+0000","last_clean":"2023-12-01T01:36:10.664313+0000","last_became_active":"2023-12-01T01:36:10.193801+0000","last_became_peered":"2023-12-01T01:36:10.193801+0000","last_unstale":"2023-12-01T01:36:10.664313+0000","last_undegraded":"2023-12-01T01:36:10.664313+0000","last_fullsized":"2023-12-01T01:36:10.664313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660654+0000","last_change":"2023-12-01T01:36:10.669848+0000","last_active":"2023-12-01T01:36:11.660654+0000","last_peered":"2023-12-01T01:36:11.660654+0000","last_clean":"2023-12-01T01:36:11.660654+0000","last_became_active":"2023-12-01T01:36:10.669557+0000","last_became_peered":"2023-12-01T01:36:10.669557+0000","last_unstale":"2023-12-01T01:36:11.660654+0000","last_undegraded":"2023-12-01T01:36:11.660654+0000","last_fullsized":"2023-12-01T01:36:11.660654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.667473+0000","last_change":"2023-12-01T01:36:10.183096+0000","last_active":"2023-12-01T01:36:10.667473+0000","last_peered":"2023-12-01T01:36:10.667473+0000","last_clean":"2023-12-01T01:36:10.667473+0000","last_became_active":"2023-12-01T01:36:10.182810+0000","last_became_peered":"2023-12-01T01:36:10.182810+0000","last_unstale":"2023-12-01T01:36:10.667473+0000","last_undegraded":"2023-12-01T01:36:10.667473+0000","last_fullsized":"2023-12-01T01:36:10.667473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:35.765727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844076+0000","last_change":"2023-12-01T01:36:10.661869+0000","last_active":"2023-12-01T01:36:11.844076+0000","last_peered":"2023-12-01T01:36:11.844076+0000","last_clean":"2023-12-01T01:36:11.844076+0000","last_became_active":"2023-12-01T01:36:10.661404+0000","last_became_peered":"2023-12-01T01:36:10.661404+0000","last_unstale":"2023-12-01T01:36:11.844076+0000","last_undegraded":"2023-12-01T01:36:11.844076+0000","last_fullsized":"2023-12-01T01:36:11.844076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.669601+0000","last_change":"2023-12-01T01:36:10.188465+0000","last_active":"2023-12-01T01:36:10.669601+0000","last_peered":"2023-12-01T01:36:10.669601+0000","last_clean":"2023-12-01T01:36:10.669601+0000","last_became_active":"2023-12-01T01:36:10.188157+0000","last_became_peered":"2023-12-01T01:36:10.188157+0000","last_unstale":"2023-12-01T01:36:10.669601+0000","last_undegraded":"2023-12-01T01:36:10.669601+0000","last_fullsized":"2023-12-01T01:36:10.669601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843928+0000","last_change":"2023-12-01T01:36:10.679247+0000","last_active":"2023-12-01T01:36:11.843928+0000","last_peered":"2023-12-01T01:36:11.843928+0000","last_clean":"2023-12-01T01:36:11.843928+0000","last_became_active":"2023-12-01T01:36:10.678097+0000","last_became_peered":"2023-12-01T01:36:10.678097+0000","last_unstale":"2023-12-01T01:36:11.843928+0000","last_undegraded":"2023-12-01T01:36:11.843928+0000","last_fullsized":"2023-12-01T01:36:11.843928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.661362+0000","last_change":"2023-12-01T01:36:10.182533+0000","last_active":"2023-12-01T01:36:10.661362+0000","last_peered":"2023-12-01T01:36:10.661362+0000","last_clean":"2023-12-01T01:36:10.661362+0000","last_became_active":"2023-12-01T01:36:10.182071+0000","last_became_peered":"2023-12-01T01:36:10.182071+0000","last_unstale":"2023-12-01T01:36:10.661362+0000","last_undegraded":"2023-12-01T01:36:10.661362+0000","last_fullsized":"2023-12-01T01:36:10.661362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:40.636957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846563+0000","last_change":"2023-12-01T01:36:10.666142+0000","last_active":"2023-12-01T01:36:11.846563+0000","last_peered":"2023-12-01T01:36:11.846563+0000","last_clean":"2023-12-01T01:36:11.846563+0000","last_became_active":"2023-12-01T01:36:10.665873+0000","last_became_peered":"2023-12-01T01:36:10.665873+0000","last_unstale":"2023-12-01T01:36:11.846563+0000","last_undegraded":"2023-12-01T01:36:11.846563+0000","last_fullsized":"2023-12-01T01:36:11.846563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.660940+0000","last_change":"2023-12-01T01:36:10.190758+0000","last_active":"2023-12-01T01:36:10.660940+0000","last_peered":"2023-12-01T01:36:10.660940+0000","last_clean":"2023-12-01T01:36:10.660940+0000","last_became_active":"2023-12-01T01:36:10.190280+0000","last_became_peered":"2023-12-01T01:36:10.190280+0000","last_unstale":"2023-12-01T01:36:10.660940+0000","last_undegraded":"2023-12-01T01:36:10.660940+0000","last_fullsized":"2023-12-01T01:36:10.660940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842977+0000","last_change":"2023-12-01T01:36:10.665004+0000","last_active":"2023-12-01T01:36:11.842977+0000","last_peered":"2023-12-01T01:36:11.842977+0000","last_clean":"2023-12-01T01:36:11.842977+0000","last_became_active":"2023-12-01T01:36:10.664692+0000","last_became_peered":"2023-12-01T01:36:10.664692+0000","last_unstale":"2023-12-01T01:36:11.842977+0000","last_undegraded":"2023-12-01T01:36:11.842977+0000","last_fullsized":"2023-12-01T01:36:11.842977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:53.375079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.666452+0000","last_change":"2023-12-01T01:36:10.186489+0000","last_active":"2023-12-01T01:36:10.666452+0000","last_peered":"2023-12-01T01:36:10.666452+0000","last_clean":"2023-12-01T01:36:10.666452+0000","last_became_active":"2023-12-01T01:36:10.186161+0000","last_became_peered":"2023-12-01T01:36:10.186161+0000","last_unstale":"2023-12-01T01:36:10.666452+0000","last_undegraded":"2023-12-01T01:36:10.666452+0000","last_fullsized":"2023-12-01T01:36:10.666452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843867+0000","last_change":"2023-12-01T01:36:10.663707+0000","last_active":"2023-12-01T01:36:11.843867+0000","last_peered":"2023-12-01T01:36:11.843867+0000","last_clean":"2023-12-01T01:36:11.843867+0000","last_became_active":"2023-12-01T01:36:10.663158+0000","last_became_peered":"2023-12-01T01:36:10.663158+0000","last_unstale":"2023-12-01T01:36:11.843867+0000","last_undegraded":"2023-12-01T01:36:11.843867+0000","last_fullsized":"2023-12-01T01:36:11.843867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661244+0000","last_change":"2023-12-01T01:36:10.188465+0000","last_active":"2023-12-01T01:36:11.661244+0000","last_peered":"2023-12-01T01:36:11.661244+0000","last_clean":"2023-12-01T01:36:11.661244+0000","last_became_active":"2023-12-01T01:36:10.187631+0000","last_became_peered":"2023-12-01T01:36:10.187631+0000","last_unstale":"2023-12-01T01:36:11.661244+0000","last_undegraded":"2023-12-01T01:36:11.661244+0000","last_fullsized":"2023-12-01T01:36:11.661244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661611+0000","last_change":"2023-12-01T01:36:10.665647+0000","last_active":"2023-12-01T01:36:11.661611+0000","last_peered":"2023-12-01T01:36:11.661611+0000","last_clean":"2023-12-01T01:36:11.661611+0000","last_became_active":"2023-12-01T01:36:10.665371+0000","last_became_peered":"2023-12-01T01:36:10.665371+0000","last_unstale":"2023-12-01T01:36:11.661611+0000","last_undegraded":"2023-12-01T01:36:11.661611+0000","last_fullsized":"2023-12-01T01:36:11.661611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685898+0000","last_change":"2023-12-01T01:36:10.194331+0000","last_active":"2023-12-01T01:36:10.685898+0000","last_peered":"2023-12-01T01:36:10.685898+0000","last_clean":"2023-12-01T01:36:10.685898+0000","last_became_active":"2023-12-01T01:36:10.194058+0000","last_became_peered":"2023-12-01T01:36:10.194058+0000","last_unstale":"2023-12-01T01:36:10.685898+0000","last_undegraded":"2023-12-01T01:36:10.685898+0000","last_fullsized":"2023-12-01T01:36:10.685898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:51.741532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661128+0000","last_change":"2023-12-01T01:36:10.680546+0000","last_active":"2023-12-01T01:36:11.661128+0000","last_peered":"2023-12-01T01:36:11.661128+0000","last_clean":"2023-12-01T01:36:11.661128+0000","last_became_active":"2023-12-01T01:36:10.680099+0000","last_became_peered":"2023-12-01T01:36:10.680099+0000","last_unstale":"2023-12-01T01:36:11.661128+0000","last_undegraded":"2023-12-01T01:36:11.661128+0000","last_fullsized":"2023-12-01T01:36:11.661128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:22.177011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650417+0000","last_change":"2023-12-01T01:36:10.193127+0000","last_active":"2023-12-01T01:36:10.650417+0000","last_peered":"2023-12-01T01:36:10.650417+0000","last_clean":"2023-12-01T01:36:10.650417+0000","last_became_active":"2023-12-01T01:36:10.192884+0000","last_became_peered":"2023-12-01T01:36:10.192884+0000","last_unstale":"2023-12-01T01:36:10.650417+0000","last_undegraded":"2023-12-01T01:36:10.650417+0000","last_fullsized":"2023-12-01T01:36:10.650417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659185+0000","last_change":"2023-12-01T01:36:10.677457+0000","last_active":"2023-12-01T01:36:11.659185+0000","last_peered":"2023-12-01T01:36:11.659185+0000","last_clean":"2023-12-01T01:36:11.659185+0000","last_became_active":"2023-12-01T01:36:10.677156+0000","last_became_peered":"2023-12-01T01:36:10.677156+0000","last_unstale":"2023-12-01T01:36:11.659185+0000","last_undegraded":"2023-12-01T01:36:11.659185+0000","last_fullsized":"2023-12-01T01:36:11.659185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:41.113753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.654051+0000","last_change":"2023-12-01T01:36:10.189035+0000","last_active":"2023-12-01T01:36:10.654051+0000","last_peered":"2023-12-01T01:36:10.654051+0000","last_clean":"2023-12-01T01:36:10.654051+0000","last_became_active":"2023-12-01T01:36:10.188370+0000","last_became_peered":"2023-12-01T01:36:10.188370+0000","last_unstale":"2023-12-01T01:36:10.654051+0000","last_undegraded":"2023-12-01T01:36:10.654051+0000","last_fullsized":"2023-12-01T01:36:10.654051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660520+0000","last_change":"2023-12-01T01:36:10.682836+0000","last_active":"2023-12-01T01:36:11.660520+0000","last_peered":"2023-12-01T01:36:11.660520+0000","last_clean":"2023-12-01T01:36:11.660520+0000","last_became_active":"2023-12-01T01:36:10.682361+0000","last_became_peered":"2023-12-01T01:36:10.682361+0000","last_unstale":"2023-12-01T01:36:11.660520+0000","last_undegraded":"2023-12-01T01:36:11.660520+0000","last_fullsized":"2023-12-01T01:36:11.660520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:27.561648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.651403+0000","last_change":"2023-12-01T01:36:10.190808+0000","last_active":"2023-12-01T01:36:10.651403+0000","last_peered":"2023-12-01T01:36:10.651403+0000","last_clean":"2023-12-01T01:36:10.651403+0000","last_became_active":"2023-12-01T01:36:10.190355+0000","last_became_peered":"2023-12-01T01:36:10.190355+0000","last_unstale":"2023-12-01T01:36:10.651403+0000","last_undegraded":"2023-12-01T01:36:10.651403+0000","last_fullsized":"2023-12-01T01:36:10.651403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659213+0000","last_change":"2023-12-01T01:36:10.685203+0000","last_active":"2023-12-01T01:36:11.659213+0000","last_peered":"2023-12-01T01:36:11.659213+0000","last_clean":"2023-12-01T01:36:11.659213+0000","last_became_active":"2023-12-01T01:36:10.684118+0000","last_became_peered":"2023-12-01T01:36:10.684118+0000","last_unstale":"2023-12-01T01:36:11.659213+0000","last_undegraded":"2023-12-01T01:36:11.659213+0000","last_fullsized":"2023-12-01T01:36:11.659213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683327+0000","last_change":"2023-12-01T01:36:10.194065+0000","last_active":"2023-12-01T01:36:10.683327+0000","last_peered":"2023-12-01T01:36:10.683327+0000","last_clean":"2023-12-01T01:36:10.683327+0000","last_became_active":"2023-12-01T01:36:10.193785+0000","last_became_peered":"2023-12-01T01:36:10.193785+0000","last_unstale":"2023-12-01T01:36:10.683327+0000","last_undegraded":"2023-12-01T01:36:10.683327+0000","last_fullsized":"2023-12-01T01:36:10.683327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659511+0000","last_change":"2023-12-01T01:36:10.654568+0000","last_active":"2023-12-01T01:36:11.659511+0000","last_peered":"2023-12-01T01:36:11.659511+0000","last_clean":"2023-12-01T01:36:11.659511+0000","last_became_active":"2023-12-01T01:36:10.654312+0000","last_became_peered":"2023-12-01T01:36:10.654312+0000","last_unstale":"2023-12-01T01:36:11.659511+0000","last_undegraded":"2023-12-01T01:36:11.659511+0000","last_fullsized":"2023-12-01T01:36:11.659511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:24.066059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659518+0000","last_change":"2023-12-01T01:36:10.672526+0000","last_active":"2023-12-01T01:36:11.659518+0000","last_peered":"2023-12-01T01:36:11.659518+0000","last_clean":"2023-12-01T01:36:11.659518+0000","last_became_active":"2023-12-01T01:36:10.671950+0000","last_became_peered":"2023-12-01T01:36:10.671950+0000","last_unstale":"2023-12-01T01:36:11.659518+0000","last_undegraded":"2023-12-01T01:36:11.659518+0000","last_fullsized":"2023-12-01T01:36:11.659518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:07.041952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.681098+0000","last_change":"2023-12-01T01:36:10.189066+0000","last_active":"2023-12-01T01:36:10.681098+0000","last_peered":"2023-12-01T01:36:10.681098+0000","last_clean":"2023-12-01T01:36:10.681098+0000","last_became_active":"2023-12-01T01:36:10.188395+0000","last_became_peered":"2023-12-01T01:36:10.188395+0000","last_unstale":"2023-12-01T01:36:10.681098+0000","last_undegraded":"2023-12-01T01:36:10.681098+0000","last_fullsized":"2023-12-01T01:36:10.681098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844042+0000","last_change":"2023-12-01T01:36:10.677457+0000","last_active":"2023-12-01T01:36:11.844042+0000","last_peered":"2023-12-01T01:36:11.844042+0000","last_clean":"2023-12-01T01:36:11.844042+0000","last_became_active":"2023-12-01T01:36:10.677200+0000","last_became_peered":"2023-12-01T01:36:10.677200+0000","last_unstale":"2023-12-01T01:36:11.844042+0000","last_undegraded":"2023-12-01T01:36:11.844042+0000","last_fullsized":"2023-12-01T01:36:11.844042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:53.375079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.681045+0000","last_change":"2023-12-01T01:36:10.193800+0000","last_active":"2023-12-01T01:36:10.681045+0000","last_peered":"2023-12-01T01:36:10.681045+0000","last_clean":"2023-12-01T01:36:10.681045+0000","last_became_active":"2023-12-01T01:36:10.193469+0000","last_became_peered":"2023-12-01T01:36:10.193469+0000","last_unstale":"2023-12-01T01:36:10.681045+0000","last_undegraded":"2023-12-01T01:36:10.681045+0000","last_fullsized":"2023-12-01T01:36:10.681045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845128+0000","last_change":"2023-12-01T01:36:10.665085+0000","last_active":"2023-12-01T01:36:11.845128+0000","last_peered":"2023-12-01T01:36:11.845128+0000","last_clean":"2023-12-01T01:36:11.845128+0000","last_became_active":"2023-12-01T01:36:10.663166+0000","last_became_peered":"2023-12-01T01:36:10.663166+0000","last_unstale":"2023-12-01T01:36:11.845128+0000","last_undegraded":"2023-12-01T01:36:11.845128+0000","last_fullsized":"2023-12-01T01:36:11.845128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:38.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683819+0000","last_change":"2023-12-01T01:36:10.188667+0000","last_active":"2023-12-01T01:36:10.683819+0000","last_peered":"2023-12-01T01:36:10.683819+0000","last_clean":"2023-12-01T01:36:10.683819+0000","last_became_active":"2023-12-01T01:36:10.188354+0000","last_became_peered":"2023-12-01T01:36:10.188354+0000","last_unstale":"2023-12-01T01:36:10.683819+0000","last_undegraded":"2023-12-01T01:36:10.683819+0000","last_fullsized":"2023-12-01T01:36:10.683819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845005+0000","last_change":"2023-12-01T01:36:10.660331+0000","last_active":"2023-12-01T01:36:11.845005+0000","last_peered":"2023-12-01T01:36:11.845005+0000","last_clean":"2023-12-01T01:36:11.845005+0000","last_became_active":"2023-12-01T01:36:10.660037+0000","last_became_peered":"2023-12-01T01:36:10.660037+0000","last_unstale":"2023-12-01T01:36:11.845005+0000","last_undegraded":"2023-12-01T01:36:11.845005+0000","last_fullsized":"2023-12-01T01:36:11.845005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:12.241846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685486+0000","last_change":"2023-12-01T01:36:10.189468+0000","last_active":"2023-12-01T01:36:10.685486+0000","last_peered":"2023-12-01T01:36:10.685486+0000","last_clean":"2023-12-01T01:36:10.685486+0000","last_became_active":"2023-12-01T01:36:10.187185+0000","last_became_peered":"2023-12-01T01:36:10.187185+0000","last_unstale":"2023-12-01T01:36:10.685486+0000","last_undegraded":"2023-12-01T01:36:10.685486+0000","last_fullsized":"2023-12-01T01:36:10.685486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":75693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.883}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.968}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":819200,"data_stored":666029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":77805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5229999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.885}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7769999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.849}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":76151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.665}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.76}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":815104,"data_stored":666007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.778}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.637}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":75625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.639}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.798}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:36:20.540 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T01:36:20.540 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-01T01:36:20.540 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-01T01:36:20.540 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-01T01:36:20.541 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-01T01:36:20.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.794+0000 7efd414bd640 1 Processor -- start 2023-12-01T01:36:20.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.794+0000 7efd414bd640 1 -- start start 2023-12-01T01:36:20.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.796+0000 7efd414bd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efd3c15d220 0x7efd3c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.796+0000 7efd414bd640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c15d600 con 0x7efd3c070ac0 2023-12-01T01:36:20.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.796+0000 7efd414bd640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c06c170 con 0x7efd3c0ad2b0 2023-12-01T01:36:20.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.796+0000 7efd414bd640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c06cc10 con 0x7efd3c15d220 2023-12-01T01:36:20.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.797+0000 7efd3b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efd3c070ac0 0x7efd3c06ba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51326/0 (socket says 172.21.15.115:51326) 2023-12-01T01:36:20.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.797+0000 7efd3b7fe640 1 -- 172.21.15.115:0/1669760975 learned_addr learned my addr 172.21.15.115:0/1669760975 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:20.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.797+0000 7efd3affd640 1 --2- 172.21.15.115:0/1669760975 >> v2:172.21.15.142:3300/0 conn(0x7efd3c15d220 0x7efd3c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.797+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251227140 0 0) 0x7efd34001110 con 0x7efd3c070ac0 2023-12-01T01:36:20.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.797+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd28003520 con 0x7efd3c070ac0 2023-12-01T01:36:20.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.797+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681846063 0 0) 0x7efd28003520 con 0x7efd3c070ac0 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.797+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 >> v2:172.21.15.142:3300/0 conn(0x7efd3c15d220 msgr2=0x7efd3c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.797+0000 7efd3a7fc640 1 --2- 172.21.15.115:0/1669760975 >> v2:172.21.15.142:3300/0 conn(0x7efd3c15d220 0x7efd3c0cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.797+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 >> v1:172.21.15.142:6789/0 conn(0x7efd3c0ad2b0 legacy=0x7efd3c06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c06c4b0 con 0x7efd3c070ac0 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7efd34001800 con 0x7efd3c070ac0 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 >> v1:172.21.15.115:6789/0 conn(0x7efd3c070ac0 legacy=0x7efd3c06ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3a7fc640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280039e0 0x7efd28003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3a7fc640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280047c0 0x7efd28004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3a7fc640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd28005180 0x7efd2800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c06cc10 con 0x7efd280047c0 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2800c0b0 con 0x7efd280039e0 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2800c270 con 0x7efd28005180 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd327fc640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280047c0 0x7efd28004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3affd640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280039e0 0x7efd28003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd327fc640 1 -- 172.21.15.115:0/1669760975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd28005180 msgr2=0x7efd2800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:20.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3b7fe640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd28005180 0x7efd2800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd327fc640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd28005180 0x7efd2800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd327fc640 1 -- 172.21.15.115:0/1669760975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280039e0 msgr2=0x7efd28003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:20.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd327fc640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280039e0 0x7efd28003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd327fc640 1 -- 172.21.15.115:0/1669760975 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd28003520 con 0x7efd280047c0 2023-12-01T01:36:20.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3affd640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280039e0 0x7efd28003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:20.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3b7fe640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd28005180 0x7efd2800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:20.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.798+0000 7efd3b7fe640 1 -- 172.21.15.115:0/1669760975 reap_dead start 2023-12-01T01:36:20.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.799+0000 7efd327fc640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280047c0 0x7efd28004c40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efd24001860 tx=0x7efd280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4cf869055ee044 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.800+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2402f050 con 0x7efd280047c0 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.800+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd24033610 con 0x7efd280047c0 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.800+0000 7efd3a7fc640 1 -- 172.21.15.115:0/1669760975 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd240339c0 con 0x7efd280047c0 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.800+0000 7efd414bd640 1 -- 172.21.15.115:0/1669760975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280047c0 msgr2=0x7efd28004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.800+0000 7efd414bd640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280047c0 0x7efd28004c40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efd24001860 tx=0x7efd280057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.800+0000 7efd414bd640 1 -- 172.21.15.115:0/1669760975 shutdown_connections 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.800+0000 7efd414bd640 1 --2- 172.21.15.115:0/1669760975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280047c0 0x7efd28004c40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.800+0000 7efd414bd640 1 -- 172.21.15.115:0/1669760975 >> 172.21.15.115:0/1669760975 conn(0x7efd3c0ce880 msgr2=0x7efd3c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.801+0000 7efd414bd640 1 -- 172.21.15.115:0/1669760975 shutdown_connections 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.801+0000 7efd414bd640 1 -- 172.21.15.115:0/1669760975 wait complete. 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.802+0000 7efd414bd640 1 Processor -- start 2023-12-01T01:36:20.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.802+0000 7efd414bd640 1 -- start start 2023-12-01T01:36:20.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.803+0000 7efd414bd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280047c0 0x7efd3c152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.803+0000 7efd414bd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd3c15aac0 0x7efd3c15af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.803+0000 7efd414bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3c157520 0x7efd3c1579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.803+0000 7efd414bd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0b2ef0 con 0x7efd3c157520 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.803+0000 7efd414bd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c15b480 con 0x7efd280047c0 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.803+0000 7efd414bd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c158230 con 0x7efd3c15aac0 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3c157520 0x7efd3c1579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3c157520 0x7efd3c1579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55750/0 (socket says 172.21.15.115:55750) 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3b7fe640 1 -- 172.21.15.115:0/4278554472 learned_addr learned my addr 172.21.15.115:0/4278554472 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd327fc640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd3c15aac0 0x7efd3c15af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3affd640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280047c0 0x7efd3c152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3b7fe640 1 -- 172.21.15.115:0/4278554472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd3c15aac0 msgr2=0x7efd3c15af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3b7fe640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd3c15aac0 0x7efd3c15af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3b7fe640 1 -- 172.21.15.115:0/4278554472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280047c0 msgr2=0x7efd3c152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3b7fe640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280047c0 0x7efd3c152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3b7fe640 1 -- 172.21.15.115:0/4278554472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c158510 con 0x7efd3c157520 2023-12-01T01:36:20.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3affd640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280047c0 0x7efd3c152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.804+0000 7efd3b7fe640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3c157520 0x7efd3c1579b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd3c06ed00 tx=0x7efd3400d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.805+0000 7efd33fff640 1 -- 172.21.15.115:0/4278554472 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340074e0 con 0x7efd3c157520 2023-12-01T01:36:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.805+0000 7efd33fff640 1 -- 172.21.15.115:0/4278554472 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34007640 con 0x7efd3c157520 2023-12-01T01:36:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.805+0000 7efd33fff640 1 -- 172.21.15.115:0/4278554472 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34004a00 con 0x7efd3c157520 2023-12-01T01:36:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.805+0000 7efd414bd640 1 -- 172.21.15.115:0/4278554472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c0b2b10 con 0x7efd3c157520 2023-12-01T01:36:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.805+0000 7efd414bd640 1 -- 172.21.15.115:0/4278554472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c1538b0 con 0x7efd3c157520 2023-12-01T01:36:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.806+0000 7efd414bd640 1 -- 172.21.15.115:0/4278554472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c0b3030 con 0x7efd3c157520 2023-12-01T01:36:20.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.807+0000 7efd33fff640 1 -- 172.21.15.115:0/4278554472 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd34004b60 con 0x7efd3c157520 2023-12-01T01:36:20.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.809+0000 7efd33fff640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd1c06a430 0x7efd1c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:20.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.809+0000 7efd33fff640 1 -- 172.21.15.115:0/4278554472 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd34087770 con 0x7efd3c157520 2023-12-01T01:36:20.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.809+0000 7efd3affd640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd1c06a430 0x7efd1c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:20.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.809+0000 7efd3affd640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd1c06a430 0x7efd1c06c8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd2c007520 tx=0x7efd2c00d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:20.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:20.813+0000 7efd33fff640 1 -- 172.21.15.115:0/4278554472 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efd3c0b3030 con 0x7efd3c157520 2023-12-01T01:36:21.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.053+0000 7efd414bd640 1 -- 172.21.15.115:0/4278554472 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efd3c15d600 con 0x7efd3c157520 2023-12-01T01:36:21.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.062+0000 7efd33fff640 1 -- 172.21.15.115:0/4278554472 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169969 (secure 0 0 0) 0x7efd3c15d600 con 0x7efd3c157520 2023-12-01T01:36:21.066 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:21.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 -- 172.21.15.115:0/4278554472 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd1c06a430 msgr2=0x7efd1c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd1c06a430 0x7efd1c06c8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd2c007520 tx=0x7efd2c00d040 comp rx=0 tx=0).stop 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 -- 172.21.15.115:0/4278554472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3c157520 msgr2=0x7efd3c1579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3c157520 0x7efd3c1579b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd3c06ed00 tx=0x7efd3400d880 comp rx=0 tx=0).stop 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 -- 172.21.15.115:0/4278554472 shutdown_connections 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd1c06a430 0x7efd1c06c8f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd3c15aac0 0x7efd3c15af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd3c157520 0x7efd3c1579b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 --2- 172.21.15.115:0/4278554472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd280047c0 0x7efd3c152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 -- 172.21.15.115:0/4278554472 >> 172.21.15.115:0/4278554472 conn(0x7efd3c0ce880 msgr2=0x7efd3c0c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 -- 172.21.15.115:0/4278554472 shutdown_connections 2023-12-01T01:36:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.066+0000 7efd414bd640 1 -- 172.21.15.115:0/4278554472 wait complete. 2023-12-01T01:36:21.087 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":5,"active_gid":4098,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":88317},{"type":"v1","addr":"172.21.15.115:6824","nonce":88317}]},"active_addr":"172.21.15.115:6824/88317","active_change":"2023-12-01T01:35:50.137631+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":297799080}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":2515539287}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":4134228660}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":1314334167}]}]} 2023-12-01T01:36:21.091 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-01T01:36:21.091 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-01T01:36:21.091 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:21.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.303+0000 7f6639e34640 1 Processor -- start 2023-12-01T01:36:21.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.303+0000 7f6639e34640 1 -- start start 2023-12-01T01:36:21.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.304+0000 7f6639e34640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f663406b980 0x7f6634167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.304+0000 7f6639e34640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66340cc8e0 con 0x7f66340cdd80 2023-12-01T01:36:21.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.304+0000 7f6639e34640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66340ce260 con 0x7f6634145620 2023-12-01T01:36:21.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.304+0000 7f6639e34640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66340cc320 con 0x7f663406b980 2023-12-01T01:36:21.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.305+0000 7f6633fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f66340cdd80 0x7f663406b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51332/0 (socket says 172.21.15.115:51332) 2023-12-01T01:36:21.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.305+0000 7f6633fff640 1 -- 172.21.15.115:0/3900605511 learned_addr learned my addr 172.21.15.115:0/3900605511 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:21.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.305+0000 7f6639633640 1 --2- 172.21.15.115:0/3900605511 >> v2:172.21.15.115:3300/0 conn(0x7f663406b980 0x7f6634167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.305+0000 7f66337fe640 1 -- 172.21.15.115:0/3900605511 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902256083 0 0) 0x7f66340cc8e0 con 0x7f66340cdd80 2023-12-01T01:36:21.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.305+0000 7f6639633640 1 -- 172.21.15.115:0/3900605511 >> v1:172.21.15.142:6789/0 conn(0x7f6634145620 legacy=0x7f66340ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.305+0000 7f6639633640 1 -- 172.21.15.115:0/3900605511 >> v1:172.21.15.115:6789/0 conn(0x7f66340cdd80 legacy=0x7f663406b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.305+0000 7f6639633640 1 -- 172.21.15.115:0/3900605511 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66340cc600 con 0x7f663406b980 2023-12-01T01:36:21.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.307+0000 7f6639633640 1 --2- 172.21.15.115:0/3900605511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f663406b980 0x7f6634167ae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f662c009d90 tx=0x7f662c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a930e1856b8b3bd4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:21.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.307+0000 7f66337fe640 1 -- 172.21.15.115:0/3900605511 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c02cd00 con 0x7f663406b980 2023-12-01T01:36:21.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.307+0000 7f66337fe640 1 -- 172.21.15.115:0/3900605511 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c02ce60 con 0x7f663406b980 2023-12-01T01:36:21.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.307+0000 7f66337fe640 1 -- 172.21.15.115:0/3900605511 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c034660 con 0x7f663406b980 2023-12-01T01:36:21.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.307+0000 7f6639e34640 1 -- 172.21.15.115:0/3900605511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f663406b980 msgr2=0x7f6634167ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.307+0000 7f6639e34640 1 --2- 172.21.15.115:0/3900605511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f663406b980 0x7f6634167ae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f662c009d90 tx=0x7f662c02bf50 comp rx=0 tx=0).stop 2023-12-01T01:36:21.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.308+0000 7f6639e34640 1 -- 172.21.15.115:0/3900605511 shutdown_connections 2023-12-01T01:36:21.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.308+0000 7f6639e34640 1 --2- 172.21.15.115:0/3900605511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f663406b980 0x7f6634167ae0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.308+0000 7f6639e34640 1 -- 172.21.15.115:0/3900605511 >> 172.21.15.115:0/3900605511 conn(0x7f66340ce640 msgr2=0x7f66340cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:21.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.308+0000 7f6639e34640 1 -- 172.21.15.115:0/3900605511 shutdown_connections 2023-12-01T01:36:21.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.308+0000 7f6639e34640 1 -- 172.21.15.115:0/3900605511 wait complete. 2023-12-01T01:36:21.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.309+0000 7f6639e34640 1 Processor -- start 2023-12-01T01:36:21.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.309+0000 7f6639e34640 1 -- start start 2023-12-01T01:36:21.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.310+0000 7f6639e34640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f663406b980 0x7f663415d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.310+0000 7f6639e34640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66340cdd80 0x7f663415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.310+0000 7f6639e34640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6634145620 0x7f6634071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.310+0000 7f6639e34640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663414f430 con 0x7f6634145620 2023-12-01T01:36:21.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.310+0000 7f6639e34640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663413e6f0 con 0x7f663406b980 2023-12-01T01:36:21.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.310+0000 7f6639e34640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663415d5a0 con 0x7f66340cdd80 2023-12-01T01:36:21.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.311+0000 7f6639633640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6634145620 0x7f6634071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.311+0000 7f6639633640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6634145620 0x7f6634071260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55760/0 (socket says 172.21.15.115:55760) 2023-12-01T01:36:21.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.311+0000 7f6639633640 1 -- 172.21.15.115:0/1348751288 learned_addr learned my addr 172.21.15.115:0/1348751288 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:21.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.311+0000 7f6638e32640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f663406b980 0x7f663415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.311+0000 7f6633fff640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66340cdd80 0x7f663415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.311+0000 7f6639633640 1 -- 172.21.15.115:0/1348751288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66340cdd80 msgr2=0x7f663415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.311+0000 7f6639633640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66340cdd80 0x7f663415ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.311+0000 7f6639633640 1 -- 172.21.15.115:0/1348751288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f663406b980 msgr2=0x7f663415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.311+0000 7f6639633640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f663406b980 0x7f663415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.311+0000 7f6639633640 1 -- 172.21.15.115:0/1348751288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c009a70 con 0x7f6634145620 2023-12-01T01:36:21.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.312+0000 7f6639633640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6634145620 0x7f6634071260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f662c037040 tx=0x7f662c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:21.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.312+0000 7f66317fa640 1 -- 172.21.15.115:0/1348751288 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c03e670 con 0x7f6634145620 2023-12-01T01:36:21.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.312+0000 7f66317fa640 1 -- 172.21.15.115:0/1348751288 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c03e7f0 con 0x7f6634145620 2023-12-01T01:36:21.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.312+0000 7f66317fa640 1 -- 172.21.15.115:0/1348751288 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c03eba0 con 0x7f6634145620 2023-12-01T01:36:21.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.312+0000 7f6639e34640 1 -- 172.21.15.115:0/1348751288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663413e130 con 0x7f6634145620 2023-12-01T01:36:21.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.312+0000 7f6639e34640 1 -- 172.21.15.115:0/1348751288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663414f050 con 0x7f6634145620 2023-12-01T01:36:21.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.313+0000 7f6639e34640 1 -- 172.21.15.115:0/1348751288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6634144e60 con 0x7f6634145620 2023-12-01T01:36:21.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.314+0000 7f66317fa640 1 -- 172.21.15.115:0/1348751288 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f662c03b040 con 0x7f6634145620 2023-12-01T01:36:21.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.315+0000 7f66317fa640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f661806a430 0x7f661806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.315+0000 7f66317fa640 1 -- 172.21.15.115:0/1348751288 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f662c0aad10 con 0x7f6634145620 2023-12-01T01:36:21.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.315+0000 7f6638e32640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f661806a430 0x7f661806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.316+0000 7f6638e32640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f661806a430 0x7f661806c8f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6628000f60 tx=0x7f6628009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:21.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.319+0000 7f66317fa640 1 -- 172.21.15.115:0/1348751288 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f662c076390 con 0x7f6634145620 2023-12-01T01:36:21.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.510+0000 7f6639e34640 1 -- 172.21.15.115:0/1348751288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f663414f570 con 0x7f6634145620 2023-12-01T01:36:21.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.513+0000 7f66317fa640 1 -- 172.21.15.115:0/1348751288 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f663414f570 con 0x7f6634145620 2023-12-01T01:36:21.515 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:21.515 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:11.641476+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 -- 172.21.15.115:0/1348751288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f661806a430 msgr2=0x7f661806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f661806a430 0x7f661806c8f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6628000f60 tx=0x7f6628009040 comp rx=0 tx=0).stop 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 -- 172.21.15.115:0/1348751288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6634145620 msgr2=0x7f6634071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6634145620 0x7f6634071260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f662c037040 tx=0x7f662c004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 -- 172.21.15.115:0/1348751288 shutdown_connections 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f661806a430 0x7f661806c8f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66340cdd80 0x7f663415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6634145620 0x7f6634071260 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 --2- 172.21.15.115:0/1348751288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f663406b980 0x7f663415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 -- 172.21.15.115:0/1348751288 >> 172.21.15.115:0/1348751288 conn(0x7f66340ce640 msgr2=0x7f66340c4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 -- 172.21.15.115:0/1348751288 shutdown_connections 2023-12-01T01:36:21.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.515+0000 7f6639e34640 1 -- 172.21.15.115:0/1348751288 wait complete. 2023-12-01T01:36:21.536 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-01T01:36:21.537 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:21.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.745+0000 7f624359e640 1 Processor -- start 2023-12-01T01:36:21.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.745+0000 7f624359e640 1 -- start start 2023-12-01T01:36:21.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f624359e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f62440cdcc0 0x7f624406b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f624359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f62440adf00 0x7f62440ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f624359e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440ce120 con 0x7f62441597d0 2023-12-01T01:36:21.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f624359e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624406b7b0 con 0x7f62440adf00 2023-12-01T01:36:21.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f624359e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624406b9c0 con 0x7f62440cdcc0 2023-12-01T01:36:21.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f6242d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f62440adf00 0x7f62440ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f6242d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f62440adf00 0x7f62440ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55768/0 (socket says 172.21.15.115:55768) 2023-12-01T01:36:21.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f6242d9d640 1 -- 172.21.15.115:0/3417212277 learned_addr learned my addr 172.21.15.115:0/3417212277 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:21.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f6241d9b640 1 --2- 172.21.15.115:0/3417212277 >> v2:172.21.15.142:3300/0 conn(0x7f62440cdcc0 0x7f624406b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f6242d9d640 1 -- 172.21.15.115:0/3417212277 >> v2:172.21.15.142:3300/0 conn(0x7f62440cdcc0 msgr2=0x7f624406b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:21.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.747+0000 7f6242d9d640 1 --2- 172.21.15.115:0/3417212277 >> v2:172.21.15.142:3300/0 conn(0x7f62440cdcc0 0x7f624406b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.748+0000 7f6242d9d640 1 -- 172.21.15.115:0/3417212277 >> v1:172.21.15.142:6789/0 conn(0x7f62441597d0 legacy=0x7f624415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.748+0000 7f6242d9d640 1 -- 172.21.15.115:0/3417212277 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624406bca0 con 0x7f62440adf00 2023-12-01T01:36:21.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.748+0000 7f6241d9b640 1 --2- 172.21.15.115:0/3417212277 >> v2:172.21.15.142:3300/0 conn(0x7f62440cdcc0 0x7f624406b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:21.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.749+0000 7f6242d9d640 1 --2- 172.21.15.115:0/3417212277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 0x7f62440ae390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f624406d9b0 tx=0x7f623c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10679f27c465cc0b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:21.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.749+0000 7f624159a640 1 -- 172.21.15.115:0/3417212277 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c02ca80 con 0x7f62440adf00 2023-12-01T01:36:21.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.749+0000 7f624159a640 1 -- 172.21.15.115:0/3417212277 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c02cbe0 con 0x7f62440adf00 2023-12-01T01:36:21.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.749+0000 7f624159a640 1 -- 172.21.15.115:0/3417212277 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c0343f0 con 0x7f62440adf00 2023-12-01T01:36:21.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.749+0000 7f624359e640 1 -- 172.21.15.115:0/3417212277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 msgr2=0x7f62440ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.749+0000 7f624359e640 1 --2- 172.21.15.115:0/3417212277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 0x7f62440ae390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f624406d9b0 tx=0x7f623c02bee0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.750+0000 7f624359e640 1 -- 172.21.15.115:0/3417212277 shutdown_connections 2023-12-01T01:36:21.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.750+0000 7f624359e640 1 --2- 172.21.15.115:0/3417212277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 0x7f62440ae390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.750+0000 7f624359e640 1 --2- 172.21.15.115:0/3417212277 >> v2:172.21.15.142:3300/0 conn(0x7f62440cdcc0 0x7f624406b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.750+0000 7f624359e640 1 -- 172.21.15.115:0/3417212277 >> 172.21.15.115:0/3417212277 conn(0x7f62440ce640 msgr2=0x7f62440cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:21.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.750+0000 7f624359e640 1 -- 172.21.15.115:0/3417212277 shutdown_connections 2023-12-01T01:36:21.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.750+0000 7f624359e640 1 -- 172.21.15.115:0/3417212277 wait complete. 2023-12-01T01:36:21.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.751+0000 7f624359e640 1 Processor -- start 2023-12-01T01:36:21.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.752+0000 7f624359e640 1 -- start start 2023-12-01T01:36:21.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.752+0000 7f624359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 0x7f62441422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.752+0000 7f624359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62440cdcc0 0x7f62440c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62441597d0 0x7f62440b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624359e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440b35a0 con 0x7f62440adf00 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244142800 con 0x7f62440cdcc0 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62440b2920 con 0x7f62441597d0 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 0x7f62441422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 0x7f62441422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55780/0 (socket says 172.21.15.115:55780) 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624259c640 1 -- 172.21.15.115:0/3181975869 learned_addr learned my addr 172.21.15.115:0/3181975869 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f6241d9b640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62440cdcc0 0x7f62440c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f6242d9d640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62441597d0 0x7f62440b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624259c640 1 -- 172.21.15.115:0/3181975869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62441597d0 msgr2=0x7f62440b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624259c640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62441597d0 0x7f62440b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624259c640 1 -- 172.21.15.115:0/3181975869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62440cdcc0 msgr2=0x7f62440c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624259c640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62440cdcc0 0x7f62440c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.753+0000 7f624259c640 1 -- 172.21.15.115:0/3181975869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c009b30 con 0x7f62440adf00 2023-12-01T01:36:21.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.754+0000 7f624259c640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 0x7f62441422c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f623400c450 tx=0x7f623400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:21.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.754+0000 7f6232ffd640 1 -- 172.21.15.115:0/3181975869 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62340052d0 con 0x7f62440adf00 2023-12-01T01:36:21.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.754+0000 7f6232ffd640 1 -- 172.21.15.115:0/3181975869 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6234005430 con 0x7f62440adf00 2023-12-01T01:36:21.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.754+0000 7f624359e640 1 -- 172.21.15.115:0/3181975869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62440b2c00 con 0x7f62440adf00 2023-12-01T01:36:21.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.754+0000 7f6232ffd640 1 -- 172.21.15.115:0/3181975869 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234012870 con 0x7f62440adf00 2023-12-01T01:36:21.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.754+0000 7f624359e640 1 -- 172.21.15.115:0/3181975869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62440b31c0 con 0x7f62440adf00 2023-12-01T01:36:21.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.755+0000 7f624359e640 1 -- 172.21.15.115:0/3181975869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62440673a0 con 0x7f62440adf00 2023-12-01T01:36:21.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.756+0000 7f6232ffd640 1 -- 172.21.15.115:0/3181975869 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f62340129d0 con 0x7f62440adf00 2023-12-01T01:36:21.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.757+0000 7f6232ffd640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f622406a430 0x7f622406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:21.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.758+0000 7f6232ffd640 1 -- 172.21.15.115:0/3181975869 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6234088460 con 0x7f62440adf00 2023-12-01T01:36:21.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.758+0000 7f6241d9b640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f622406a430 0x7f622406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:21.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.758+0000 7f6241d9b640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f622406a430 0x7f622406c8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6244152990 tx=0x7f6238008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:21.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.762+0000 7f6232ffd640 1 -- 172.21.15.115:0/3181975869 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f62340539b0 con 0x7f62440adf00 2023-12-01T01:36:21.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.952+0000 7f624359e640 1 -- 172.21.15.115:0/3181975869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62440af590 con 0x7f62440adf00 2023-12-01T01:36:21.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.954+0000 7f6232ffd640 1 -- 172.21.15.115:0/3181975869 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f6234056fd0 con 0x7f62440adf00 2023-12-01T01:36:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:21.956 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:11.641476+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:21.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 -- 172.21.15.115:0/3181975869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f622406a430 msgr2=0x7f622406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f622406a430 0x7f622406c8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6244152990 tx=0x7f6238008040 comp rx=0 tx=0).stop 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 -- 172.21.15.115:0/3181975869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 msgr2=0x7f62441422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 0x7f62441422c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f623400c450 tx=0x7f623400c910 comp rx=0 tx=0).stop 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 -- 172.21.15.115:0/3181975869 shutdown_connections 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f622406a430 0x7f622406c8f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62441597d0 0x7f62440b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62440cdcc0 0x7f62440c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 --2- 172.21.15.115:0/3181975869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62440adf00 0x7f62441422c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 -- 172.21.15.115:0/3181975869 >> 172.21.15.115:0/3181975869 conn(0x7f62440ce640 msgr2=0x7f62440ca420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 -- 172.21.15.115:0/3181975869 shutdown_connections 2023-12-01T01:36:21.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:21.957+0000 7f624359e640 1 -- 172.21.15.115:0/3181975869 wait complete. 2023-12-01T01:36:21.979 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-01T01:36:21.979 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-01T01:36:21.980 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-01T01:36:21.980 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-01T01:36:21.981 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-01T01:36:21.981 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-01T01:36:21.981 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-01T01:36:21.982 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-01T01:36:22.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.280+0000 7fb459429640 1 Processor -- start 2023-12-01T01:36:22.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.281+0000 7fb459429640 1 -- start start 2023-12-01T01:36:22.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.282+0000 7fb459429640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb44c0df3f0 0x7fb44c007470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.283+0000 7fb453fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb44c0df3f0 0x7fb44c007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.283+0000 7fb453fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb44c0df3f0 0x7fb44c007470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55784/0 (socket says 172.21.15.115:55784) 2023-12-01T01:36:22.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.284+0000 7fb453fff640 1 -- 172.21.15.115:0/1268639084 learned_addr learned my addr 172.21.15.115:0/1268639084 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.284+0000 7fb459429640 1 -- 172.21.15.115:0/1268639084 --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c0063d0 con 0x7fb44c050270 2023-12-01T01:36:22.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.284+0000 7fb459429640 1 -- 172.21.15.115:0/1268639084 --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c0066b0 con 0x7fb44c055620 2023-12-01T01:36:22.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.285+0000 7fb459429640 1 -- 172.21.15.115:0/1268639084 --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44c0df7d0 con 0x7fb44c0df3f0 2023-12-01T01:36:22.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.285+0000 7fb452ffd640 1 -- 172.21.15.115:0/1268639084 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137565891 0 0) 0x7fb44c0066b0 con 0x7fb44c055620 2023-12-01T01:36:22.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.286+0000 7fb452ffd640 1 -- 172.21.15.115:0/1268639084 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb444003520 con 0x7fb44c055620 2023-12-01T01:36:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.286+0000 7fb453fff640 1 -- 172.21.15.115:0/1268639084 >> v1:172.21.15.142:6789/0 conn(0x7fb44c055620 legacy=0x7fb44c04fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.286+0000 7fb453fff640 1 -- 172.21.15.115:0/1268639084 >> v1:172.21.15.115:6789/0 conn(0x7fb44c050270 legacy=0x7fb44c0380c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:36:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.287+0000 7fb453fff640 1 -- 172.21.15.115:0/1268639084 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c006990 con 0x7fb44c0df3f0 2023-12-01T01:36:22.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.289+0000 7fb453fff640 1 --2- 172.21.15.115:0/1268639084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb44c0df3f0 0x7fb44c007470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb44800b440 tx=0x7fb44802c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f10c1bdce19344 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.289+0000 7fb452ffd640 1 -- 172.21.15.115:0/1268639084 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448007ad0 con 0x7fb44c0df3f0 2023-12-01T01:36:22.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.289+0000 7fb452ffd640 1 -- 172.21.15.115:0/1268639084 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44802f050 con 0x7fb44c0df3f0 2023-12-01T01:36:22.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.290+0000 7fb452ffd640 1 -- 172.21.15.115:0/1268639084 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448007e50 con 0x7fb44c0df3f0 2023-12-01T01:36:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.291+0000 7fb459429640 1 -- 172.21.15.115:0/1268639084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb44c0df3f0 msgr2=0x7fb44c007470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.291+0000 7fb459429640 1 --2- 172.21.15.115:0/1268639084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb44c0df3f0 0x7fb44c007470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb44800b440 tx=0x7fb44802c510 comp rx=0 tx=0).stop 2023-12-01T01:36:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.292+0000 7fb459429640 1 -- 172.21.15.115:0/1268639084 shutdown_connections 2023-12-01T01:36:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.292+0000 7fb459429640 1 --2- 172.21.15.115:0/1268639084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb44c0df3f0 0x7fb44c007470 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.292+0000 7fd35f59e640 1 Processor -- start 2023-12-01T01:36:22.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.292+0000 7fb459429640 1 -- 172.21.15.115:0/1268639084 >> 172.21.15.115:0/1268639084 conn(0x7fb44c054b00 msgr2=0x7fb44c054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.292+0000 7fb459429640 1 -- 172.21.15.115:0/1268639084 shutdown_connections 2023-12-01T01:36:22.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.292+0000 7fb459429640 1 -- 172.21.15.115:0/1268639084 wait complete. 2023-12-01T01:36:22.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.292+0000 7fd35f59e640 1 -- start start 2023-12-01T01:36:22.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.294+0000 7fd35f59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd358055400 0x7fd3580557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.294+0000 7fd35f59e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd358049ee0 0x7fd358051930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.294+0000 7fd35f59e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd358055d20 con 0x7fd3580543c0 2023-12-01T01:36:22.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.294+0000 7fd35f59e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd358039850 con 0x7fd358055400 2023-12-01T01:36:22.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.294+0000 7fd35f59e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3580e8920 con 0x7fd358049ee0 2023-12-01T01:36:22.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.295+0000 7fd35dd9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd3580543c0 0x7fd358039140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51352/0 (socket says 172.21.15.115:51352) 2023-12-01T01:36:22.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.295+0000 7fd35dd9b640 1 -- 172.21.15.115:0/2891176911 learned_addr learned my addr 172.21.15.115:0/2891176911 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:22.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.296+0000 7fd35e59c640 1 --2- 172.21.15.115:0/2891176911 >> v2:172.21.15.115:3300/0 conn(0x7fd358055400 0x7fd3580557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.296+0000 7fd35ed9d640 1 --2- 172.21.15.115:0/2891176911 >> v2:172.21.15.142:3300/0 conn(0x7fd358049ee0 0x7fd358051930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.296+0000 7fd35d59a640 1 -- 172.21.15.115:0/2891176911 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726396354 0 0) 0x7fd358055d20 con 0x7fd3580543c0 2023-12-01T01:36:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.297+0000 7fd35d59a640 1 -- 172.21.15.115:0/2891176911 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd348003520 con 0x7fd3580543c0 2023-12-01T01:36:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.297+0000 7fd35ed9d640 1 -- 172.21.15.115:0/2891176911 >> v2:172.21.15.115:3300/0 conn(0x7fd358055400 msgr2=0x7fd3580557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.297+0000 7fd35ed9d640 1 --2- 172.21.15.115:0/2891176911 >> v2:172.21.15.115:3300/0 conn(0x7fd358055400 0x7fd3580557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.297+0000 7fd35ed9d640 1 -- 172.21.15.115:0/2891176911 >> v1:172.21.15.115:6789/0 conn(0x7fd3580543c0 legacy=0x7fd358039140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.297+0000 7fd35ed9d640 1 -- 172.21.15.115:0/2891176911 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580532c0 con 0x7fd358049ee0 2023-12-01T01:36:22.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.298+0000 7fb459429640 1 Processor -- start 2023-12-01T01:36:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.297+0000 7fd35e59c640 1 --2- 172.21.15.115:0/2891176911 >> v2:172.21.15.115:3300/0 conn(0x7fd358055400 0x7fd3580557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.298+0000 7fb459429640 1 -- start start 2023-12-01T01:36:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.299+0000 7fb511773640 1 Processor -- start 2023-12-01T01:36:22.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.299+0000 7fd35ed9d640 1 --2- 172.21.15.115:0/2891176911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd358049ee0 0x7fd358051930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd360068e10 tx=0x7fd3600a5c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe11fd9ca5fe7d22 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.300+0000 7fd35d59a640 1 -- 172.21.15.115:0/2891176911 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3600a68e0 con 0x7fd358049ee0 2023-12-01T01:36:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.300+0000 7fd35d59a640 1 -- 172.21.15.115:0/2891176911 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3600a6a40 con 0x7fd358049ee0 2023-12-01T01:36:22.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.300+0000 7fd35d59a640 1 -- 172.21.15.115:0/2891176911 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3600a68e0 con 0x7fd358049ee0 2023-12-01T01:36:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fb459429640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb44c050270 0x7fb44c0daf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fb459429640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb44c055620 0x7fb44c047510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fd35f59e640 1 -- 172.21.15.115:0/2891176911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd358049ee0 msgr2=0x7fd358051930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fd35f59e640 1 --2- 172.21.15.115:0/2891176911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd358049ee0 0x7fd358051930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd360068e10 tx=0x7fd3600a5c80 comp rx=0 tx=0).stop 2023-12-01T01:36:22.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fd35f59e640 1 -- 172.21.15.115:0/2891176911 shutdown_connections 2023-12-01T01:36:22.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fb459429640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb44c0df3f0 0x7fb44c0d9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fd35f59e640 1 --2- 172.21.15.115:0/2891176911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd358049ee0 0x7fd358051930 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fd35f59e640 1 --2- 172.21.15.115:0/2891176911 >> v2:172.21.15.115:3300/0 conn(0x7fd358055400 0x7fd3580557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fd35f59e640 1 -- 172.21.15.115:0/2891176911 >> 172.21.15.115:0/2891176911 conn(0x7fd3580538a0 msgr2=0x7fd358053cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fd35f59e640 1 -- 172.21.15.115:0/2891176911 shutdown_connections 2023-12-01T01:36:22.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb459429640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c048fb0 con 0x7fb44c0df3f0 2023-12-01T01:36:22.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fd35f59e640 1 -- 172.21.15.115:0/2891176911 wait complete. 2023-12-01T01:36:22.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb459429640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c0db4c0 con 0x7fb44c055620 2023-12-01T01:36:22.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb459429640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c0da200 con 0x7fb44c050270 2023-12-01T01:36:22.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb4537fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb44c055620 0x7fb44c047510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb4537fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb44c055620 0x7fb44c047510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53174/0 (socket says 172.21.15.115:53174) 2023-12-01T01:36:22.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb4537fe640 1 -- 172.21.15.115:0/3992561809 learned_addr learned my addr 172.21.15.115:0/3992561809 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb453fff640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb44c050270 0x7fb44c0daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.299+0000 7fb511773640 1 -- start start 2023-12-01T01:36:22.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb458c28640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb44c0df3f0 0x7fb44c0d9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fb511773640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb50c15cc20 0x7fb50c1609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fb511773640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb50c15d000 0x7fb50c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.301+0000 7fb511773640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb50c0b84e0 0x7fb50c0cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb511773640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50c161940 con 0x7fb50c15d000 2023-12-01T01:36:22.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb511773640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50c066710 con 0x7fb50c15cc20 2023-12-01T01:36:22.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.302+0000 7fb511773640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0d1150 con 0x7fb50c0b84e0 2023-12-01T01:36:22.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.303+0000 7fb453fff640 1 -- 172.21.15.115:0/3992561809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb44c055620 msgr2=0x7fb44c047510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.303+0000 7fb453fff640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb44c055620 0x7fb44c047510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.303+0000 7fb453fff640 1 -- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb44c0df3f0 msgr2=0x7fb44c0d9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.303+0000 7fb453fff640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb44c0df3f0 0x7fb44c0d9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.303+0000 7fb453fff640 1 -- 172.21.15.115:0/3992561809 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb448009df0 con 0x7fb44c050270 2023-12-01T01:36:22.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.303+0000 7fb50affd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb50c15cc20 0x7fb50c1609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.303+0000 7fb50affd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb50c15cc20 0x7fb50c1609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53180/0 (socket says 172.21.15.115:53180) 2023-12-01T01:36:22.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.304+0000 7fb453fff640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb44c050270 0x7fb44c0daf80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb448032000 tx=0x7fb4480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.303+0000 7fb50affd640 1 -- 172.21.15.115:0/377167677 learned_addr learned my addr 172.21.15.115:0/377167677 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.303+0000 7fb5027fc640 1 --2- 172.21.15.115:0/377167677 >> v2:172.21.15.115:3300/0 conn(0x7fb50c15d000 0x7fb50c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.304+0000 7fb5027fc640 1 -- 172.21.15.115:0/377167677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb50c0b84e0 msgr2=0x7fb50c0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.304+0000 7fb50b7fe640 1 --2- 172.21.15.115:0/377167677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb50c0b84e0 0x7fb50c0cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.304+0000 7fb5027fc640 1 --2- 172.21.15.115:0/377167677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb50c0b84e0 0x7fb50c0cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.304+0000 7fb5027fc640 1 -- 172.21.15.115:0/377167677 >> v2:172.21.15.142:3300/0 conn(0x7fb50c15cc20 msgr2=0x7fb50c1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.304+0000 7fb5027fc640 1 --2- 172.21.15.115:0/377167677 >> v2:172.21.15.142:3300/0 conn(0x7fb50c15cc20 0x7fb50c1609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.304+0000 7fb5027fc640 1 -- 172.21.15.115:0/377167677 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c1610a0 con 0x7fb50c15d000 2023-12-01T01:36:22.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.305+0000 7fb450ff9640 1 -- 172.21.15.115:0/3992561809 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448003610 con 0x7fb44c050270 2023-12-01T01:36:22.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.305+0000 7fb450ff9640 1 -- 172.21.15.115:0/3992561809 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb448003770 con 0x7fb44c050270 2023-12-01T01:36:22.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.305+0000 7fb450ff9640 1 -- 172.21.15.115:0/3992561809 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448003ae0 con 0x7fb44c050270 2023-12-01T01:36:22.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.305+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c048610 con 0x7fb44c050270 2023-12-01T01:36:22.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.305+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c048bd0 con 0x7fb44c050270 2023-12-01T01:36:22.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.305+0000 7fd35f59e640 1 Processor -- start 2023-12-01T01:36:22.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.305+0000 7fd35f59e640 1 -- start start 2023-12-01T01:36:22.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.306+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb44c006f50 con 0x7fb44c050270 2023-12-01T01:36:22.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.307+0000 7fd35f59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd358049ee0 0x7fd3580d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.307+0000 7fb5027fc640 1 --2- 172.21.15.115:0/377167677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50c15d000 0x7fb50c0cf3d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb50c08bd90 tx=0x7fb4fc034030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d582c898a4777114 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.307+0000 7fd35f59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3580543c0 0x7fd3581894a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.307+0000 7fd35f59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd358055400 0x7fd3581899e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.307+0000 7fd35f59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35803b170 con 0x7fd3580543c0 2023-12-01T01:36:22.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.307+0000 7fd35f59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35803a5f0 con 0x7fd358049ee0 2023-12-01T01:36:22.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.307+0000 7fd35f59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35803a8d0 con 0x7fd358055400 2023-12-01T01:36:22.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.307+0000 7fd35dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3580543c0 0x7fd3581894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.307+0000 7fd35dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3580543c0 0x7fd3581894a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55818/0 (socket says 172.21.15.115:55818) 2023-12-01T01:36:22.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.307+0000 7fd35dd9b640 1 -- 172.21.15.115:0/197954921 learned_addr learned my addr 172.21.15.115:0/197954921 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7fb50a7fc640 1 -- 172.21.15.115:0/377167677 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc034de0 con 0x7fb50c15d000 2023-12-01T01:36:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7fd35ed9d640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd358055400 0x7fd3581899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7fd35e59c640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd358049ee0 0x7fd3580d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7fb50a7fc640 1 -- 172.21.15.115:0/377167677 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4fc03c440 con 0x7fb50c15d000 2023-12-01T01:36:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7fb50a7fc640 1 -- 172.21.15.115:0/377167677 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc03c7d0 con 0x7fb50c15d000 2023-12-01T01:36:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7fd35dd9b640 1 -- 172.21.15.115:0/197954921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd358055400 msgr2=0x7fd3581899e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7fd35dd9b640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd358055400 0x7fd3581899e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7fd35dd9b640 1 -- 172.21.15.115:0/197954921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd358049ee0 msgr2=0x7fd3580d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7fd35dd9b640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd358049ee0 0x7fd3580d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7fd35dd9b640 1 -- 172.21.15.115:0/197954921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd350009400 con 0x7fd3580543c0 2023-12-01T01:36:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.308+0000 7f5e74f27640 1 Processor -- start 2023-12-01T01:36:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.309+0000 7fb511773640 1 -- 172.21.15.115:0/377167677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50c15d000 msgr2=0x7fb50c0cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.309+0000 7fb511773640 1 --2- 172.21.15.115:0/377167677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50c15d000 0x7fb50c0cf3d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb50c08bd90 tx=0x7fb4fc034030 comp rx=0 tx=0).stop 2023-12-01T01:36:22.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.309+0000 7fb511773640 1 -- 172.21.15.115:0/377167677 shutdown_connections 2023-12-01T01:36:22.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.309+0000 7fb511773640 1 --2- 172.21.15.115:0/377167677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb50c0b84e0 0x7fb50c0cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.309+0000 7fb511773640 1 --2- 172.21.15.115:0/377167677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50c15d000 0x7fb50c0cf3d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.309+0000 7fb511773640 1 --2- 172.21.15.115:0/377167677 >> v2:172.21.15.142:3300/0 conn(0x7fb50c15cc20 0x7fb50c1609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.309+0000 7fb511773640 1 -- 172.21.15.115:0/377167677 >> 172.21.15.115:0/377167677 conn(0x7fb50c065960 msgr2=0x7fb50c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.309+0000 7f5e74f27640 1 -- start start 2023-12-01T01:36:22.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.309+0000 7fd35dd9b640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3580543c0 0x7fd3581894a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd35400c410 tx=0x7fd35400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.310+0000 7fb511773640 1 -- 172.21.15.115:0/377167677 shutdown_connections 2023-12-01T01:36:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.309+0000 7fd34effd640 1 -- 172.21.15.115:0/197954921 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3540074e0 con 0x7fd3580543c0 2023-12-01T01:36:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.310+0000 7fd34effd640 1 -- 172.21.15.115:0/197954921 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd354007640 con 0x7fd3580543c0 2023-12-01T01:36:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.310+0000 7fd34effd640 1 -- 172.21.15.115:0/197954921 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3540074e0 con 0x7fd3580543c0 2023-12-01T01:36:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.310+0000 7f5e74f27640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5e7015d0b0 0x7f5e700b8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.310+0000 7f5e74f27640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5e700d21c0 0x7f5e700d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.310+0000 7f5e74f27640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e700d2620 0x7f5e700d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7f5e74f27640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e700b2330 con 0x7f5e700d21c0 2023-12-01T01:36:22.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7f5e74f27640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e700d1120 con 0x7f5e7015d0b0 2023-12-01T01:36:22.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7f5e74f27640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e700d1400 con 0x7f5e700d2620 2023-12-01T01:36:22.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7f5e67fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5e700d21c0 0x7f5e700d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7fb450ff9640 1 -- 172.21.15.115:0/3992561809 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb448003c40 con 0x7fb44c050270 2023-12-01T01:36:22.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd360072570 con 0x7fd3580543c0 2023-12-01T01:36:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7f5e6e59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5e7015d0b0 0x7f5e700b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7f5e6ed9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e700d2620 0x7f5e700d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7f5e6e59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5e7015d0b0 0x7f5e700b8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53190/0 (socket says 172.21.15.115:53190) 2023-12-01T01:36:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7f5e6e59c640 1 -- 172.21.15.115:0/999466794 learned_addr learned my addr 172.21.15.115:0/999466794 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.311+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35803ae90 con 0x7fd3580543c0 2023-12-01T01:36:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.310+0000 7fb511773640 1 -- 172.21.15.115:0/377167677 wait complete. 2023-12-01T01:36:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.312+0000 7fb511773640 1 Processor -- start 2023-12-01T01:36:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.313+0000 7fb511773640 1 -- start start 2023-12-01T01:36:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.313+0000 7fb450ff9640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb43806a430 0x7fb43806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.313+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd3580e8b30 con 0x7fd3580543c0 2023-12-01T01:36:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.312+0000 7f5e6ed9d640 1 -- 172.21.15.115:0/999466794 >> v2:172.21.15.142:3300/0 conn(0x7f5e7015d0b0 msgr2=0x7f5e700b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.312+0000 7f5e6ed9d640 1 --2- 172.21.15.115:0/999466794 >> v2:172.21.15.142:3300/0 conn(0x7f5e7015d0b0 0x7f5e700b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.312+0000 7f5e6ed9d640 1 -- 172.21.15.115:0/999466794 >> v2:172.21.15.115:3300/0 conn(0x7f5e700d21c0 msgr2=0x7f5e700d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.312+0000 7f5e6ed9d640 1 --2- 172.21.15.115:0/999466794 >> v2:172.21.15.115:3300/0 conn(0x7f5e700d21c0 0x7f5e700d2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.312+0000 7f5e6ed9d640 1 -- 172.21.15.115:0/999466794 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e700d16e0 con 0x7f5e700d2620 2023-12-01T01:36:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.313+0000 7fb450ff9640 1 -- 172.21.15.115:0/3992561809 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4480ab6b0 con 0x7fb44c050270 2023-12-01T01:36:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.314+0000 7fb450ff9640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] conn(0x7fb4380725d0 0x7fb4380749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.314+0000 7fb450ff9640 1 -- 172.21.15.115:0/3992561809 --> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4380750a0 con 0x7fb4380725d0 2023-12-01T01:36:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.314+0000 7fb450ff9640 1 -- 172.21.15.115:0/3992561809 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4480abad0 con 0x7fb44c050270 2023-12-01T01:36:22.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.314+0000 7fb4537fe640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb43806a430 0x7fb43806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.314+0000 7f0d60899640 1 Processor -- start 2023-12-01T01:36:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.314+0000 7fb458c28640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] conn(0x7fb4380725d0 0x7fb4380749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.314+0000 7f5e6ed9d640 1 --2- 172.21.15.115:0/999466794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e700d2620 0x7f5e700d30b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5e700b5190 tx=0x7f5e6802bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bb0846c54ca2aeb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.314+0000 7f0d60899640 1 -- start start 2023-12-01T01:36:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.315+0000 7fb511773640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb50c0b84e0 0x7fb50c150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.315+0000 7fb511773640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb50c15cc20 0x7fb50c0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.315+0000 7fb458c28640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] conn(0x7fb4380725d0 0x7fb4380749f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.315+0000 7fb511773640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50c0b7f90 0x7fb50c15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.315+0000 7f5e6dd9b640 1 -- 172.21.15.115:0/999466794 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6802ccb0 con 0x7f5e700d2620 2023-12-01T01:36:22.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7f5e6dd9b640 1 -- 172.21.15.115:0/999466794 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6802ce10 con 0x7f5e700d2620 2023-12-01T01:36:22.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7f5e6dd9b640 1 -- 172.21.15.115:0/999466794 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68034660 con 0x7f5e700d2620 2023-12-01T01:36:22.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7fb50b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50c0b7f90 0x7fb50c15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7fb50b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50c0b7f90 0x7fb50c15b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55824/0 (socket says 172.21.15.115:55824) 2023-12-01T01:36:22.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7fb5027fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb50c15cc20 0x7fb50c0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7fb50b7fe640 1 -- 172.21.15.115:0/1122409629 learned_addr learned my addr 172.21.15.115:0/1122409629 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0c7370 con 0x7fb50c0b7f90 2023-12-01T01:36:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0c66f0 con 0x7fb50c0b84e0 2023-12-01T01:36:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c0c69d0 con 0x7fb50c15cc20 2023-12-01T01:36:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7f0d60899640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 0x7f0d5c0b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7f0d60899640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0d5c06b2a0 0x7f0d5c152e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.316+0000 7f0d60899640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0d5c153360 0x7f0d5c0b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7f865f38e640 1 Processor -- start 2023-12-01T01:36:22.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7fb50affd640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb50c0b84e0 0x7fb50c150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7f0d5bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0d5c153360 0x7f0d5c0b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7fd34effd640 1 -- 172.21.15.115:0/197954921 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd3540047e0 con 0x7fd3580543c0 2023-12-01T01:36:22.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7fb450ff9640 1 -- 172.21.15.115:0/3992561809 <== osd.0 v2:172.21.15.115:6815/88579 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb4380750a0 con 0x7fb4380725d0 2023-12-01T01:36:22.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7fb4537fe640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb43806a430 0x7fb43806c8f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0e2940 tx=0x7fb440009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7f0d5bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0d5c153360 0x7f0d5c0b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55840/0 (socket says 172.21.15.115:55840) 2023-12-01T01:36:22.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7f0d5bfff640 1 -- 172.21.15.115:0/2134437305 learned_addr learned my addr 172.21.15.115:0/2134437305 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7f0d60899640 1 -- 172.21.15.115:0/2134437305 --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5c0cb2c0 con 0x7f0d5c153360 2023-12-01T01:36:22.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7f0d60899640 1 -- 172.21.15.115:0/2134437305 --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5c0cb5a0 con 0x7f0d5c06b2a0 2023-12-01T01:36:22.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7f0d60899640 1 -- 172.21.15.115:0/2134437305 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c0cb880 con 0x7f0d5c066450 2023-12-01T01:36:22.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7f0d5b7fe640 1 --2- 172.21.15.115:0/2134437305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 0x7f0d5c0b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7f0d5affd640 1 --2- 172.21.15.115:0/2134437305 >> v2:172.21.15.142:3300/0 conn(0x7f0d5c06b2a0 0x7f0d5c152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.317+0000 7f5e74f27640 1 -- 172.21.15.115:0/999466794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e700d2620 msgr2=0x7f5e700d30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7f5e74f27640 1 --2- 172.21.15.115:0/999466794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e700d2620 0x7f5e700d30b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5e700b5190 tx=0x7f5e6802bf00 comp rx=0 tx=0).stop 2023-12-01T01:36:22.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7f0d5bfff640 1 -- 172.21.15.115:0/2134437305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 msgr2=0x7f0d5c0b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7f0d5bfff640 1 --2- 172.21.15.115:0/2134437305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 0x7f0d5c0b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7f0d5bfff640 1 -- 172.21.15.115:0/2134437305 >> v2:172.21.15.142:3300/0 conn(0x7f0d5c06b2a0 msgr2=0x7f0d5c152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7f0d5bfff640 1 --2- 172.21.15.115:0/2134437305 >> v2:172.21.15.142:3300/0 conn(0x7f0d5c06b2a0 0x7f0d5c152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7f0d5bfff640 1 -- 172.21.15.115:0/2134437305 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5c0b1be0 con 0x7f0d5c153360 2023-12-01T01:36:22.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7fd34effd640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd34006a490 0x7fd34006c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7fb50b7fe640 1 -- 172.21.15.115:0/1122409629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb50c15cc20 msgr2=0x7fb50c0b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7fb50b7fe640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb50c15cc20 0x7fb50c0b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7fb50b7fe640 1 -- 172.21.15.115:0/1122409629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb50c0b84e0 msgr2=0x7fb50c150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7fb50b7fe640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb50c0b84e0 0x7fb50c150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7fd34effd640 1 -- 172.21.15.115:0/197954921 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd354088ca0 con 0x7fd3580543c0 2023-12-01T01:36:22.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7fb50b7fe640 1 -- 172.21.15.115:0/1122409629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0093a0 con 0x7fb50c0b7f90 2023-12-01T01:36:22.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7fd34effd640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] conn(0x7fd3400726b0 0x7fd340074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.319+0000 7fd34effd640 1 -- 172.21.15.115:0/197954921 --> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd340075180 con 0x7fd3400726b0 2023-12-01T01:36:22.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.319+0000 7fd34effd640 1 -- 172.21.15.115:0/197954921 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd354089050 con 0x7fd3580543c0 2023-12-01T01:36:22.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.319+0000 7fd35e59c640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd34006a490 0x7fd34006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.319+0000 7fb50b7fe640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50c0b7f90 0x7fb50c15b8c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb50400cd60 tx=0x7fb50400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.319+0000 7f0d5bfff640 1 --2- 172.21.15.115:0/2134437305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d5c153360 0x7f0d5c0b0830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c06b000 tx=0x7f0d5402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f60deb8c40e85f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.319+0000 7fb503fff640 1 -- 172.21.15.115:0/1122409629 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504003b90 con 0x7fb50c0b7f90 2023-12-01T01:36:22.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.319+0000 7fd35e59c640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd34006a490 0x7fd34006c950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd350009c10 tx=0x7fd350008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7fb503fff640 1 -- 172.21.15.115:0/1122409629 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb504003d10 con 0x7fb50c0b7f90 2023-12-01T01:36:22.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c0c6cb0 con 0x7fb50c0b7f90 2023-12-01T01:36:22.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c0c93b0 con 0x7fb50c0b7f90 2023-12-01T01:36:22.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7fb503fff640 1 -- 172.21.15.115:0/1122409629 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504004260 con 0x7fb50c0b7f90 2023-12-01T01:36:22.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7f0d5a7fc640 1 -- 172.21.15.115:0/2134437305 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5402cb20 con 0x7f0d5c153360 2023-12-01T01:36:22.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7f0d5a7fc640 1 -- 172.21.15.115:0/2134437305 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5402cc80 con 0x7f0d5c153360 2023-12-01T01:36:22.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7f0d5a7fc640 1 -- 172.21.15.115:0/2134437305 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54034610 con 0x7f0d5c153360 2023-12-01T01:36:22.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f5e74f27640 1 -- 172.21.15.115:0/999466794 shutdown_connections 2023-12-01T01:36:22.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f5e74f27640 1 --2- 172.21.15.115:0/999466794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e700d2620 0x7f5e700d30b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f5e74f27640 1 --2- 172.21.15.115:0/999466794 >> v2:172.21.15.115:3300/0 conn(0x7f5e700d21c0 0x7f5e700d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f5e74f27640 1 --2- 172.21.15.115:0/999466794 >> v2:172.21.15.142:3300/0 conn(0x7f5e7015d0b0 0x7f5e700b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f5e74f27640 1 -- 172.21.15.115:0/999466794 >> 172.21.15.115:0/999466794 conn(0x7f5e7006a0b0 msgr2=0x7f5e7006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f0d60899640 1 -- 172.21.15.115:0/2134437305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d5c153360 msgr2=0x7f0d5c0b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f0d60899640 1 --2- 172.21.15.115:0/2134437305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d5c153360 0x7f0d5c0b0830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c06b000 tx=0x7f0d5402bf80 comp rx=0 tx=0).stop 2023-12-01T01:36:22.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.318+0000 7f865f38e640 1 -- start start 2023-12-01T01:36:22.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f5e74f27640 1 -- 172.21.15.115:0/999466794 shutdown_connections 2023-12-01T01:36:22.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7f865f38e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86500e1180 0x7f8650054ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7f865f38e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86500e1560 con 0x7f8650052300 2023-12-01T01:36:22.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7f865f38e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8650055860 con 0x7f8650056900 2023-12-01T01:36:22.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7f865f38e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8650055b40 con 0x7f86500e1180 2023-12-01T01:36:22.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7f865db8b640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f8650056900 0x7f8650051bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:56814/0 (socket says 172.21.15.115:56814) 2023-12-01T01:36:22.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.320+0000 7f865db8b640 1 -- 172.21.15.115:0/4282086138 learned_addr learned my addr 172.21.15.115:0/4282086138 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:22.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f865e38c640 1 --2- 172.21.15.115:0/4282086138 >> v2:172.21.15.115:3300/0 conn(0x7f86500e1180 0x7f8650054ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb4f0000fc0 con 0x7fb50c0b7f90 2023-12-01T01:36:22.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.322+0000 7fd35ed9d640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] conn(0x7fd3400726b0 0x7fd340074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f5e74f27640 1 -- 172.21.15.115:0/999466794 wait complete. 2023-12-01T01:36:22.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.322+0000 7f0d60899640 1 -- 172.21.15.115:0/2134437305 shutdown_connections 2023-12-01T01:36:22.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.322+0000 7f0d60899640 1 --2- 172.21.15.115:0/2134437305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d5c153360 0x7f0d5c0b0830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.322+0000 7f0d60899640 1 --2- 172.21.15.115:0/2134437305 >> v2:172.21.15.142:3300/0 conn(0x7f0d5c06b2a0 0x7f0d5c152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.322+0000 7f0d60899640 1 --2- 172.21.15.115:0/2134437305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 0x7f0d5c0b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.322+0000 7f0d60899640 1 -- 172.21.15.115:0/2134437305 >> 172.21.15.115:0/2134437305 conn(0x7f0d5c065960 msgr2=0x7f0d5c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.323+0000 7fd35ed9d640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] conn(0x7fd3400726b0 0x7fd340074ad0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.323+0000 7f0d60899640 1 -- 172.21.15.115:0/2134437305 shutdown_connections 2023-12-01T01:36:22.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.323+0000 7f0d60899640 1 -- 172.21.15.115:0/2134437305 wait complete. 2023-12-01T01:36:22.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f865d38a640 1 -- 172.21.15.115:0/4282086138 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386607428 0 0) 0x7f86500e1560 con 0x7f8650052300 2023-12-01T01:36:22.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f865d38a640 1 -- 172.21.15.115:0/4282086138 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8644003520 con 0x7f8650052300 2023-12-01T01:36:22.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f865d38a640 1 -- 172.21.15.115:0/4282086138 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904621626 0 0) 0x7f8650055860 con 0x7f8650056900 2023-12-01T01:36:22.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.321+0000 7f865d38a640 1 -- 172.21.15.115:0/4282086138 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86500e1560 con 0x7f8650056900 2023-12-01T01:36:22.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.322+0000 7f865d38a640 1 -- 172.21.15.115:0/4282086138 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1024015758 0 0) 0x7f8644003520 con 0x7f8650052300 2023-12-01T01:36:22.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.322+0000 7f865e38c640 1 -- 172.21.15.115:0/4282086138 >> v1:172.21.15.142:6789/0 conn(0x7f8650056900 legacy=0x7f8650051bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.322+0000 7f865e38c640 1 -- 172.21.15.115:0/4282086138 >> v1:172.21.15.115:6789/0 conn(0x7f8650052300 legacy=0x7f8650039fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.322+0000 7f865e38c640 1 -- 172.21.15.115:0/4282086138 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8650055e20 con 0x7f86500e1180 2023-12-01T01:36:22.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.324+0000 7f865e38c640 1 --2- 172.21.15.115:0/4282086138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86500e1180 0x7f8650054ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f86540097a0 tx=0x7f8654033fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5885bc5a0634e93 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.324+0000 7f5e74f27640 1 Processor -- start 2023-12-01T01:36:22.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.324+0000 7fb503fff640 1 -- 172.21.15.115:0/1122409629 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb5040043c0 con 0x7fb50c0b7f90 2023-12-01T01:36:22.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.325+0000 7f865d38a640 1 -- 172.21.15.115:0/4282086138 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865804e890 con 0x7f86500e1180 2023-12-01T01:36:22.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.325+0000 7f134deaf640 1 Processor -- start 2023-12-01T01:36:22.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.325+0000 7f865d38a640 1 -- 172.21.15.115:0/4282086138 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654034ad0 con 0x7f86500e1180 2023-12-01T01:36:22.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.325+0000 7f865d38a640 1 -- 172.21.15.115:0/4282086138 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654034e40 con 0x7f86500e1180 2023-12-01T01:36:22.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.325+0000 7f134deaf640 1 -- start start 2023-12-01T01:36:22.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.325+0000 7f5e74f27640 1 -- start start 2023-12-01T01:36:22.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.325+0000 7f0d60899640 1 Processor -- start 2023-12-01T01:36:22.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f865f38e640 1 -- 172.21.15.115:0/4282086138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86500e1180 msgr2=0x7f8650054ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7fb503fff640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4ec06a460 0x7fb4ec06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f865f38e640 1 --2- 172.21.15.115:0/4282086138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86500e1180 0x7f8650054ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f86540097a0 tx=0x7f8654033fa0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f5e74f27640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e700d21c0 0x7f5e700c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f5e74f27640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e700d2620 0x7f5e7015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f5e74f27640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e7015d0b0 0x7f5e700c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f5e74f27640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7015f010 con 0x7f5e7015d0b0 2023-12-01T01:36:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f5e74f27640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7015eb80 con 0x7f5e700d2620 2023-12-01T01:36:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f5e74f27640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7015df70 con 0x7f5e700d21c0 2023-12-01T01:36:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f0d60899640 1 -- start start 2023-12-01T01:36:22.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f865f38e640 1 -- 172.21.15.115:0/4282086138 shutdown_connections 2023-12-01T01:36:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f865f38e640 1 --2- 172.21.15.115:0/4282086138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86500e1180 0x7f8650054ef0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7f865f38e640 1 -- 172.21.15.115:0/4282086138 >> 172.21.15.115:0/4282086138 conn(0x7f86500538a0 msgr2=0x7f8650053cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f0d60899640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 0x7f0d5c15c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f0d60899640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d5c06b2a0 0x7f0d5c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f134deaf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f13480d21c0 0x7f13480d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f134deaf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13480d2600 0x7f13480d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f134deaf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13480d1120 con 0x7f134815d0b0 2023-12-01T01:36:22.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f134deaf640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13480b2330 con 0x7f13480d21c0 2023-12-01T01:36:22.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f134deaf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13480b8c10 con 0x7f13480d2600 2023-12-01T01:36:22.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.325+0000 7fd34effd640 1 -- 172.21.15.115:0/197954921 <== osd.2 v2:172.21.15.115:6800/88578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd340075180 con 0x7fd3400726b0 2023-12-01T01:36:22.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f5e67fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e700d2620 0x7f5e7015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f0d60899640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d5c153360 0x7f0d5c14a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f133ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f13480d21c0 0x7f13480d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f133ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f13480d21c0 0x7f13480d2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55874/0 (socket says 172.21.15.115:55874) 2023-12-01T01:36:22.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f133ffff640 1 -- 172.21.15.115:0/2654663327 learned_addr learned my addr 172.21.15.115:0/2654663327 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f5e67fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e700d2620 0x7f5e7015d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53208/0 (socket says 172.21.15.115:53208) 2023-12-01T01:36:22.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f5e67fff640 1 -- 172.21.15.115:0/842664554 learned_addr learned my addr 172.21.15.115:0/842664554 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f5e6e59c640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e700d21c0 0x7f5e700c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7fb503fff640 1 -- 172.21.15.115:0/1122409629 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb504012070 con 0x7fb50c0b7f90 2023-12-01T01:36:22.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7fb503fff640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] conn(0x7fb4ec072680 0x7fb4ec074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7fb503fff640 1 -- 172.21.15.115:0/1122409629 --> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4ec075150 con 0x7fb4ec072680 2023-12-01T01:36:22.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f0d60899640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c14dac0 con 0x7f0d5c153360 2023-12-01T01:36:22.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f0d60899640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c14dda0 con 0x7f0d5c06b2a0 2023-12-01T01:36:22.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f0d60899640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c14e080 con 0x7f0d5c066450 2023-12-01T01:36:22.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.326+0000 7fb503fff640 1 -- 172.21.15.115:0/1122409629 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb50408b520 con 0x7fb50c0b7f90 2023-12-01T01:36:22.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f0d5b7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 0x7f0d5c15c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f0d5affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d5c06b2a0 0x7f0d5c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f0d5b7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 0x7f0d5c15c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:33822/0 (socket says 172.21.15.115:33822) 2023-12-01T01:36:22.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.327+0000 7f0d5b7fe640 1 -- 172.21.15.115:0/4281159384 learned_addr learned my addr 172.21.15.115:0/4281159384 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f5e6ed9d640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e7015d0b0 0x7f5e700c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f1347fff640 1 --2- 172.21.15.115:0/2654663327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13480d2600 0x7f13480d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f133ffff640 1 -- 172.21.15.115:0/2654663327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13480d2600 msgr2=0x7f13480d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f865f38e640 1 -- 172.21.15.115:0/4282086138 shutdown_connections 2023-12-01T01:36:22.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f133ffff640 1 --2- 172.21.15.115:0/2654663327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13480d2600 0x7f13480d30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f133ffff640 1 -- 172.21.15.115:0/2654663327 >> v1:172.21.15.115:6789/0 conn(0x7f134815d0b0 legacy=0x7f13480b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f133ffff640 1 -- 172.21.15.115:0/2654663327 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13480d1400 con 0x7f13480d21c0 2023-12-01T01:36:22.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f1347fff640 1 --2- 172.21.15.115:0/2654663327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13480d2600 0x7f13480d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:22.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f0d5b7fe640 1 -- 172.21.15.115:0/4281159384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d5c06b2a0 msgr2=0x7f0d5c15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f0d5b7fe640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d5c06b2a0 0x7f0d5c15cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f5e6ed9d640 1 -- 172.21.15.115:0/842664554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e700d21c0 msgr2=0x7f5e700c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f5e6ed9d640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e700d21c0 0x7f5e700c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f5e6ed9d640 1 -- 172.21.15.115:0/842664554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e700d2620 msgr2=0x7f5e7015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f5e6ed9d640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e700d2620 0x7f5e7015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f0d5b7fe640 1 -- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d5c153360 msgr2=0x7f0d5c14a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f0d5bfff640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d5c153360 0x7f0d5c14a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f5e6ed9d640 1 -- 172.21.15.115:0/842664554 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e68009a70 con 0x7f5e7015d0b0 2023-12-01T01:36:22.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.329+0000 7f0d5b7fe640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d5c153360 0x7f0d5c14a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.329+0000 7f0d5b7fe640 1 -- 172.21.15.115:0/4281159384 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d54009b30 con 0x7f0d5c066450 2023-12-01T01:36:22.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.329+0000 7f0d5bfff640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d5c153360 0x7f0d5c14a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:22.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.329+0000 7f5e6ed9d640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e7015d0b0 0x7f5e700c66d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5e68033000 tx=0x7f5e68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.329+0000 7f0d5b7fe640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 0x7f0d5c15c640 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000e9b0 tx=0x7f0d5000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.330+0000 7f5e66ffd640 1 -- 172.21.15.115:0/842664554 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6803e610 con 0x7f5e7015d0b0 2023-12-01T01:36:22.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.330+0000 7f5e74f27640 1 -- 172.21.15.115:0/842664554 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7015e250 con 0x7f5e7015d0b0 2023-12-01T01:36:22.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.330+0000 7fb5027fc640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] conn(0x7fb4ec072680 0x7fb4ec074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.330+0000 7fb50affd640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4ec06a460 0x7fb4ec06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.330+0000 7f5e74f27640 1 -- 172.21.15.115:0/842664554 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7015e810 con 0x7f5e7015d0b0 2023-12-01T01:36:22.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.330+0000 7f0d43fff640 1 -- 172.21.15.115:0/4281159384 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50009670 con 0x7f0d5c066450 2023-12-01T01:36:22.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.330+0000 7f0d43fff640 1 -- 172.21.15.115:0/4281159384 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5000c9a0 con 0x7f0d5c066450 2023-12-01T01:36:22.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.330+0000 7f0d43fff640 1 -- 172.21.15.115:0/4281159384 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000cd50 con 0x7f0d5c066450 2023-12-01T01:36:22.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.330+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c14e360 con 0x7f0d5c066450 2023-12-01T01:36:22.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.331+0000 7fb50affd640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4ec06a460 0x7fb4ec06c920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb50c15fbd0 tx=0x7fb4f4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.331+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c14eb60 con 0x7f0d5c066450 2023-12-01T01:36:22.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.331+0000 7f5e66ffd640 1 -- 172.21.15.115:0/842664554 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6803e770 con 0x7f5e7015d0b0 2023-12-01T01:36:22.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.331+0000 7f5e66ffd640 1 -- 172.21.15.115:0/842664554 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6803eae0 con 0x7f5e7015d0b0 2023-12-01T01:36:22.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.331+0000 7f7a5e718640 1 Processor -- start 2023-12-01T01:36:22.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.331+0000 7f5e64ff9640 1 -- 172.21.15.115:0/842664554 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e700d1ca0 con 0x7f5e7015d0b0 2023-12-01T01:36:22.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.328+0000 7f865f38e640 1 -- 172.21.15.115:0/4282086138 wait complete. 2023-12-01T01:36:22.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.331+0000 7f865f38e640 1 Processor -- start 2023-12-01T01:36:22.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.332+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0d5c0b1ef0 con 0x7f0d5c066450 2023-12-01T01:36:22.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.332+0000 7f133ffff640 1 --2- 172.21.15.115:0/2654663327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13480d21c0 0x7f13480d2b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1330009870 tx=0x7f133002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4299550736d673ef server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.332+0000 7f865f38e640 1 -- start start 2023-12-01T01:36:22.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.331+0000 7f7a5e718640 1 -- start start 2023-12-01T01:36:22.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.333+0000 7f7a5e718640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a58161430 0x7f7a580c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.333+0000 7fb5027fc640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] conn(0x7fb4ec072680 0x7fb4ec074aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.333+0000 7f1346ffd640 1 -- 172.21.15.115:0/2654663327 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133002f840 con 0x7f13480d21c0 2023-12-01T01:36:22.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.333+0000 7f1346ffd640 1 -- 172.21.15.115:0/2654663327 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133002f9a0 con 0x7f13480d21c0 2023-12-01T01:36:22.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.334+0000 7f5e66ffd640 1 -- 172.21.15.115:0/842664554 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5e6803b050 con 0x7f5e7015d0b0 2023-12-01T01:36:22.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.333+0000 7f1346ffd640 1 -- 172.21.15.115:0/2654663327 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133002fd30 con 0x7f13480d21c0 2023-12-01T01:36:22.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.333+0000 7f7a5e718640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7a580ce5a0 0x7f7a580cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.333+0000 7f7a5e718640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a580c1db0 con 0x7f7a5806f5c0 2023-12-01T01:36:22.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.333+0000 7f7a5e718640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5806b800 con 0x7f7a580ce5a0 2023-12-01T01:36:22.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.333+0000 7f7a5e718640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5806bae0 con 0x7f7a58161430 2023-12-01T01:36:22.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.334+0000 7f134deaf640 1 -- 172.21.15.115:0/2654663327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13480d21c0 msgr2=0x7f13480d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.334+0000 7f134deaf640 1 --2- 172.21.15.115:0/2654663327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13480d21c0 0x7f13480d2b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1330009870 tx=0x7f133002ba20 comp rx=0 tx=0).stop 2023-12-01T01:36:22.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.334+0000 7f134deaf640 1 -- 172.21.15.115:0/2654663327 shutdown_connections 2023-12-01T01:36:22.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.334+0000 7f134deaf640 1 --2- 172.21.15.115:0/2654663327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13480d2600 0x7f13480d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.334+0000 7f134deaf640 1 --2- 172.21.15.115:0/2654663327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13480d21c0 0x7f13480d2b70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.334+0000 7f134deaf640 1 -- 172.21.15.115:0/2654663327 >> 172.21.15.115:0/2654663327 conn(0x7f134806a0b0 msgr2=0x7f134806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.335+0000 7f134deaf640 1 -- 172.21.15.115:0/2654663327 shutdown_connections 2023-12-01T01:36:22.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.335+0000 7f134deaf640 1 -- 172.21.15.115:0/2654663327 wait complete. 2023-12-01T01:36:22.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.335+0000 7f0d43fff640 1 -- 172.21.15.115:0/4281159384 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0d50013470 con 0x7f0d5c066450 2023-12-01T01:36:22.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.335+0000 7f7a57fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a58161430 0x7f7a580c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.335+0000 7f7a577fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7a5806f5c0 0x7f7a580cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51384/0 (socket says 172.21.15.115:51384) 2023-12-01T01:36:22.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.335+0000 7f7a577fe640 1 -- 172.21.15.115:0/1940530081 learned_addr learned my addr 172.21.15.115:0/1940530081 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:22.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.335+0000 7f7a5ccb5640 1 --2- 172.21.15.115:0/1940530081 >> v2:172.21.15.115:3300/0 conn(0x7f7a580ce5a0 0x7f7a580cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.335+0000 7fb503fff640 1 -- 172.21.15.115:0/1122409629 <== osd.1 v2:172.21.15.115:6808/88580 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb4ec075150 con 0x7fb4ec072680 2023-12-01T01:36:22.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.336+0000 7f5e66ffd640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 0x7f5e5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.336+0000 7f7a5ccb5640 1 -- 172.21.15.115:0/1940530081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a58161430 msgr2=0x7f7a580c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.336+0000 7f7a5ccb5640 1 --2- 172.21.15.115:0/1940530081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a58161430 0x7f7a580c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.336+0000 7f7a5ccb5640 1 -- 172.21.15.115:0/1940530081 >> v1:172.21.15.115:6789/0 conn(0x7f7a5806f5c0 legacy=0x7f7a580cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.336+0000 7f7a5ccb5640 1 -- 172.21.15.115:0/1940530081 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5806bdc0 con 0x7f7a580ce5a0 2023-12-01T01:36:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.336+0000 7f0d43fff640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d4806a480 0x7f0d4806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.336+0000 7f0d43fff640 1 -- 172.21.15.115:0/4281159384 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d50088e30 con 0x7f0d5c066450 2023-12-01T01:36:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.336+0000 7f0d43fff640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] conn(0x7f0d480726a0 0x7f0d48074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.336+0000 7f0d43fff640 1 -- 172.21.15.115:0/4281159384 --> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d48075170 con 0x7f0d480726a0 2023-12-01T01:36:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.336+0000 7f0d43fff640 1 -- 172.21.15.115:0/4281159384 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d500891a0 con 0x7f0d5c066450 2023-12-01T01:36:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.337+0000 7f5e66ffd640 1 -- 172.21.15.115:0/842664554 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e680aadb0 con 0x7f5e7015d0b0 2023-12-01T01:36:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.337+0000 7f5e66ffd640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] conn(0x7f5e50072650 0x7f5e50074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.337+0000 7f5e66ffd640 1 -- 172.21.15.115:0/842664554 --> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e50075120 con 0x7f5e50072650 2023-12-01T01:36:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.337+0000 7f5e66ffd640 1 -- 172.21.15.115:0/842664554 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e680b0380 con 0x7f5e7015d0b0 2023-12-01T01:36:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.337+0000 7f134deaf640 1 Processor -- start 2023-12-01T01:36:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.337+0000 7f0d5affd640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d4806a480 0x7f0d4806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.337+0000 7f134deaf640 1 -- start start 2023-12-01T01:36:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.337+0000 7f865f38e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8650052300 0x7f86500508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.338+0000 7f5e6e59c640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 0x7f5e5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.338+0000 7f0d5affd640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d4806a480 0x7f0d4806c940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c1519d0 tx=0x7f0d4c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.338+0000 7f0d5bfff640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] conn(0x7f0d480726a0 0x7f0d48074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.338+0000 7f865e38c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8650052300 0x7f86500508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.338+0000 7f5e6e59c640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 0x7f5e5006c8f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009ae0 tx=0x7f5e60009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f134deaf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13480d21c0 0x7f134814c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f134deaf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13480d2600 0x7f13480cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f134deaf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f134815d0b0 0x7f134814f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f134deaf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1348160090 con 0x7f134815d0b0 2023-12-01T01:36:22.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f0d5bfff640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] conn(0x7f0d480726a0 0x7f0d48074ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f5e67fff640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] conn(0x7f5e50072650 0x7f5e50074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f134deaf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134815f480 con 0x7f13480d21c0 2023-12-01T01:36:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f134deaf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134815f760 con 0x7f13480d2600 2023-12-01T01:36:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f13477fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13480d21c0 0x7f134814c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f13477fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13480d21c0 0x7f134814c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53244/0 (socket says 172.21.15.115:53244) 2023-12-01T01:36:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f13477fe640 1 -- 172.21.15.115:0/4259623833 learned_addr learned my addr 172.21.15.115:0/4259623833 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.339+0000 7f1347fff640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f134815d0b0 0x7f134814f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.338+0000 7f865e38c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8650052300 0x7f86500508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53228/0 (socket says 172.21.15.115:53228) 2023-12-01T01:36:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f865e38c640 1 -- 172.21.15.115:0/3562067576 learned_addr learned my addr 172.21.15.115:0/3562067576 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f133ffff640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13480d2600 0x7f13480cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f865f38e640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8650056900 0x7f86500d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f7a5ccb5640 1 --2- 172.21.15.115:0/1940530081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580ce5a0 0x7f7a580cc820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009d90 tx=0x7f7a4802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a6682687699fd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f865f38e640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86500e1180 0x7f865004c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865004e460 con 0x7f8650056900 2023-12-01T01:36:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865004bf00 con 0x7f8650052300 2023-12-01T01:36:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865004c1e0 con 0x7f86500e1180 2023-12-01T01:36:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f7a56ffd640 1 -- 172.21.15.115:0/1940530081 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4802cb40 con 0x7f7a580ce5a0 2023-12-01T01:36:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f7a56ffd640 1 -- 172.21.15.115:0/1940530081 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4802cca0 con 0x7f7a580ce5a0 2023-12-01T01:36:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f7a56ffd640 1 -- 172.21.15.115:0/1940530081 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48034660 con 0x7f7a580ce5a0 2023-12-01T01:36:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f0d43fff640 1 -- 172.21.15.115:0/4281159384 <== osd.3 v2:172.21.15.115:6826/88584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0d48075170 con 0x7f0d480726a0 2023-12-01T01:36:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.341+0000 7f13477fe640 1 -- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13480d2600 msgr2=0x7f13480cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.341+0000 7f13477fe640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13480d2600 0x7f13480cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.341+0000 7f13477fe640 1 -- 172.21.15.115:0/4259623833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f134815d0b0 msgr2=0x7f134814f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.341+0000 7f13477fe640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f134815d0b0 0x7f134814f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.341+0000 7f13477fe640 1 -- 172.21.15.115:0/4259623833 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13300094d0 con 0x7f13480d21c0 2023-12-01T01:36:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.341+0000 7f1347fff640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f134815d0b0 0x7f134814f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.341+0000 7f7a5e718640 1 -- 172.21.15.115:0/1940530081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580ce5a0 msgr2=0x7f7a580cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.341+0000 7f7a5e718640 1 --2- 172.21.15.115:0/1940530081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580ce5a0 0x7f7a580cc820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009d90 tx=0x7f7a4802bd90 comp rx=0 tx=0).stop 2023-12-01T01:36:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.341+0000 7f13477fe640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13480d21c0 0x7f134814c730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f133800c3c0 tx=0x7f133800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f7a5e718640 1 -- 172.21.15.115:0/1940530081 shutdown_connections 2023-12-01T01:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f7a5e718640 1 --2- 172.21.15.115:0/1940530081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580ce5a0 0x7f7a580cc820 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f7a5e718640 1 --2- 172.21.15.115:0/1940530081 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a58161430 0x7f7a580c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f7a5e718640 1 -- 172.21.15.115:0/1940530081 >> 172.21.15.115:0/1940530081 conn(0x7f7a580cece0 msgr2=0x7f7a580cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f5e67fff640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] conn(0x7f5e50072650 0x7f5e50074a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f7a5e718640 1 -- 172.21.15.115:0/1940530081 shutdown_connections 2023-12-01T01:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f1344ff9640 1 -- 172.21.15.115:0/4259623833 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13380052d0 con 0x7f13480d21c0 2023-12-01T01:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f1344ff9640 1 -- 172.21.15.115:0/4259623833 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1338005430 con 0x7f13480d21c0 2023-12-01T01:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f1344ff9640 1 -- 172.21.15.115:0/4259623833 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338003830 con 0x7f13480d21c0 2023-12-01T01:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134815fa40 con 0x7f13480d21c0 2023-12-01T01:36:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.342+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134814cc70 con 0x7f13480d21c0 2023-12-01T01:36:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.343+0000 7f7a5e718640 1 -- 172.21.15.115:0/1940530081 wait complete. 2023-12-01T01:36:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.343+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13480d19c0 con 0x7f13480d21c0 2023-12-01T01:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.340+0000 7f865eb8d640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86500e1180 0x7f865004c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.343+0000 7f865e38c640 1 -- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86500e1180 msgr2=0x7f865004c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.343+0000 7f865e38c640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86500e1180 0x7f865004c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.343+0000 7f865e38c640 1 -- 172.21.15.115:0/3562067576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8650056900 msgr2=0x7f86500d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.341+0000 7f865db8b640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8650056900 0x7f86500d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.344+0000 7f865e38c640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8650056900 0x7f86500d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.344+0000 7f865e38c640 1 -- 172.21.15.115:0/3562067576 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654009400 con 0x7f8650052300 2023-12-01T01:36:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.344+0000 7f865eb8d640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86500e1180 0x7f865004c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.344+0000 7f865db8b640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8650056900 0x7f86500d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.345+0000 7f865e38c640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8650052300 0x7f86500508e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f86540345b0 tx=0x7f86540046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.345+0000 7f864effd640 1 -- 172.21.15.115:0/3562067576 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654003820 con 0x7f8650052300 2023-12-01T01:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.345+0000 7f5e66ffd640 1 -- 172.21.15.115:0/842664554 <== osd.5 v2:172.21.15.142:6816/88938 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5e50075120 con 0x7f5e50072650 2023-12-01T01:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.345+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865004c4c0 con 0x7f8650052300 2023-12-01T01:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.345+0000 7f864effd640 1 -- 172.21.15.115:0/3562067576 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654003980 con 0x7f8650052300 2023-12-01T01:36:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.345+0000 7f864effd640 1 -- 172.21.15.115:0/3562067576 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654003d10 con 0x7f8650052300 2023-12-01T01:36:22.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.348+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 --> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb44c006c70 con 0x7fb4380725d0 2023-12-01T01:36:22.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.346+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8650050e20 con 0x7f8650052300 2023-12-01T01:36:22.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.347+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86500563e0 con 0x7f8650052300 2023-12-01T01:36:22.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.347+0000 7f7a5e718640 1 Processor -- start 2023-12-01T01:36:22.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.347+0000 7f7a5e718640 1 -- start start 2023-12-01T01:36:22.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.347+0000 7f1344ff9640 1 -- 172.21.15.115:0/4259623833 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1338003990 con 0x7f13480d21c0 2023-12-01T01:36:22.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f1344ff9640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f132806a460 0x7f132806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f7a5e718640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5806f5c0 0x7f7a5815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f7a5e718640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580ce5a0 0x7f7a581ea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f7a5e718640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a58161430 0x7f7a580cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f7a5e718640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a581fa5c0 con 0x7f7a580ce5a0 2023-12-01T01:36:22.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f7a5e718640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a581fa8a0 con 0x7f7a58161430 2023-12-01T01:36:22.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f7a5e718640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a581fab80 con 0x7f7a5806f5c0 2023-12-01T01:36:22.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f1344ff9640 1 -- 172.21.15.115:0/4259623833 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1338088440 con 0x7f13480d21c0 2023-12-01T01:36:22.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f1344ff9640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] conn(0x7f1328072600 0x7f1328074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f1344ff9640 1 -- 172.21.15.115:0/4259623833 --> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13280750d0 con 0x7f1328072600 2023-12-01T01:36:22.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f1344ff9640 1 -- 172.21.15.115:0/4259623833 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1338088890 con 0x7f13480d21c0 2023-12-01T01:36:22.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7f133ffff640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f132806a460 0x7f132806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.348+0000 7fb450ff9640 1 -- 172.21.15.115:0/3992561809 <== osd.0 v2:172.21.15.115:6815/88579 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb45404ef30 con 0x7fb4380725d0 2023-12-01T01:36:22.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] conn(0x7fb4380725d0 msgr2=0x7fb4380749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7fb459429640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] conn(0x7fb4380725d0 0x7fb4380749f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb43806a430 msgr2=0x7fb43806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7fb459429640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb43806a430 0x7fb43806c8f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0e2940 tx=0x7fb440009040 comp rx=0 tx=0).stop 2023-12-01T01:36:22.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb44c050270 msgr2=0x7fb44c0daf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7fb459429640 1 --2- 172.21.15.115:0/3992561809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb44c050270 0x7fb44c0daf80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb448032000 tx=0x7fb4480046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.349+0000 7fb458c28640 1 -- 172.21.15.115:0/3992561809 reap_dead start 2023-12-01T01:36:22.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.350+0000 7f7a5ccb5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a58161430 0x7f7a580cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.350+0000 7f7a5ccb5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a58161430 0x7f7a580cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53256/0 (socket says 172.21.15.115:53256) 2023-12-01T01:36:22.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.350+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 shutdown_connections 2023-12-01T01:36:22.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.350+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 >> 172.21.15.115:0/3992561809 conn(0x7fb44c054b00 msgr2=0x7fb44c0dfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.350+0000 7f7a5ccb5640 1 -- 172.21.15.115:0/90002329 learned_addr learned my addr 172.21.15.115:0/90002329 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.350+0000 7f7a57fff640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5806f5c0 0x7f7a5815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.350+0000 7f7a577fe640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580ce5a0 0x7f7a581ea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.350+0000 7f133ffff640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f132806a460 0x7f132806c920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1330036000 tx=0x7f1330008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.351+0000 7f1347fff640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] conn(0x7f1328072600 0x7f1328074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.351+0000 7f7a5ccb5640 1 -- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5806f5c0 msgr2=0x7f7a5815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.351+0000 7f7a5ccb5640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5806f5c0 0x7f7a5815d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.351+0000 7f7a5ccb5640 1 -- 172.21.15.115:0/90002329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580ce5a0 msgr2=0x7f7a581ea8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.351+0000 7f7a5ccb5640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580ce5a0 0x7f7a581ea8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.351+0000 7f7a5ccb5640 1 -- 172.21.15.115:0/90002329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48009a70 con 0x7f7a58161430 2023-12-01T01:36:22.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.352+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 shutdown_connections 2023-12-01T01:36:22.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.352+0000 7fb459429640 1 -- 172.21.15.115:0/3992561809 wait complete. 2023-12-01T01:36:22.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.352+0000 7f7a5ccb5640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a58161430 0x7f7a580cd3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009d90 tx=0x7f7a48004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.352+0000 7f864effd640 1 -- 172.21.15.115:0/3562067576 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8654043070 con 0x7f8650052300 2023-12-01T01:36:22.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.352+0000 7f7a54ff9640 1 -- 172.21.15.115:0/90002329 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48040650 con 0x7f7a58161430 2023-12-01T01:36:22.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.352+0000 7f7a54ff9640 1 -- 172.21.15.115:0/90002329 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a480407b0 con 0x7f7a58161430 2023-12-01T01:36:22.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.352+0000 7f7a54ff9640 1 -- 172.21.15.115:0/90002329 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48040b60 con 0x7f7a58161430 2023-12-01T01:36:22.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.352+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a581fae60 con 0x7f7a58161430 2023-12-01T01:36:22.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.353+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a581fa200 con 0x7f7a58161430 2023-12-01T01:36:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.352+0000 7f1347fff640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] conn(0x7f1328072600 0x7f1328074a20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.353+0000 7f864effd640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f863c06a430 0x7f863c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.354+0000 7f864effd640 1 -- 172.21.15.115:0/3562067576 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86540b2fc0 con 0x7f8650052300 2023-12-01T01:36:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.354+0000 7f864effd640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] conn(0x7f863c0725d0 0x7f863c0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.354+0000 7f864effd640 1 -- 172.21.15.115:0/3562067576 --> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f863c0750a0 con 0x7f863c0725d0 2023-12-01T01:36:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.354+0000 7f864effd640 1 -- 172.21.15.115:0/3562067576 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86540b3410 con 0x7f8650052300 2023-12-01T01:36:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.354+0000 7f865db8b640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f863c06a430 0x7f863c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.354+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a580c5d20 con 0x7f7a58161430 2023-12-01T01:36:22.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.355+0000 7f865eb8d640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] conn(0x7f863c0725d0 0x7f863c0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.355+0000 7f1344ff9640 1 -- 172.21.15.115:0/4259623833 <== osd.7 v2:172.21.15.142:6817/88947 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f13280750d0 con 0x7f1328072600 2023-12-01T01:36:22.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.356+0000 7f865eb8d640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] conn(0x7f863c0725d0 0x7f863c0749f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.357+0000 7f865db8b640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f863c06a430 0x7f863c06c8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f86500d1e80 tx=0x7f8648009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.358+0000 7f864effd640 1 -- 172.21.15.115:0/3562067576 <== osd.4 v2:172.21.15.142:6800/88936 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f863c0750a0 con 0x7f863c0725d0 2023-12-01T01:36:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.360+0000 7f7a54ff9640 1 -- 172.21.15.115:0/90002329 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7a48040cc0 con 0x7f7a58161430 2023-12-01T01:36:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.361+0000 7f7a54ff9640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a3806a430 0x7f7a3806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.361+0000 7f7a54ff9640 1 -- 172.21.15.115:0/90002329 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a4807b070 con 0x7f7a58161430 2023-12-01T01:36:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.362+0000 7f7a54ff9640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] conn(0x7f7a380732b0 0x7f7a38075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.362+0000 7f7a54ff9640 1 -- 172.21.15.115:0/90002329 --> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a38075e20 con 0x7f7a380732b0 2023-12-01T01:36:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.362+0000 7f7a54ff9640 1 -- 172.21.15.115:0/90002329 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a4807ae40 con 0x7f7a58161430 2023-12-01T01:36:22.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.363+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 --> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd3580e8e10 con 0x7fd3400726b0 2023-12-01T01:36:22.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.364+0000 7fd34effd640 1 -- 172.21.15.115:0/197954921 <== osd.2 v2:172.21.15.115:6800/88578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd3600a6800 con 0x7fd3400726b0 2023-12-01T01:36:22.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.364+0000 7f7a57fff640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a3806a430 0x7f7a3806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.365+0000 7f7a577fe640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] conn(0x7f7a380732b0 0x7f7a38075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.365+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] conn(0x7fd3400726b0 msgr2=0x7fd340074ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.365+0000 7fd35f59e640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] conn(0x7fd3400726b0 0x7fd340074ad0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd34006a490 msgr2=0x7fd34006c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7fd35f59e640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd34006a490 0x7fd34006c950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd350009c10 tx=0x7fd350008040 comp rx=0 tx=0).stop 2023-12-01T01:36:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3580543c0 msgr2=0x7fd3581894a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7fd35f59e640 1 --2- 172.21.15.115:0/197954921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3580543c0 0x7fd3581894a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd35400c410 tx=0x7fd35400c8d0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7fd35ed9d640 1 -- 172.21.15.115:0/197954921 reap_dead start 2023-12-01T01:36:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 shutdown_connections 2023-12-01T01:36:22.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 >> 172.21.15.115:0/197954921 conn(0x7fd3580538a0 msgr2=0x7fd3580e7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 shutdown_connections 2023-12-01T01:36:22.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7fd35f59e640 1 -- 172.21.15.115:0/197954921 wait complete. 2023-12-01T01:36:22.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7f5e64ff9640 1 -- 172.21.15.115:0/842664554 --> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5e700d19c0 con 0x7f5e50072650 2023-12-01T01:36:22.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7f7a577fe640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] conn(0x7f7a380732b0 0x7f7a38075750 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.366+0000 7f7a57fff640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a3806a430 0x7f7a3806c8f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009aa0 tx=0x7f7a50009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.367+0000 7f5e66ffd640 1 -- 172.21.15.115:0/842664554 <== osd.5 v2:172.21.15.142:6816/88938 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5e5800c040 con 0x7f5e50072650 2023-12-01T01:36:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.368+0000 7f5e64ff9640 1 -- 172.21.15.115:0/842664554 >> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] conn(0x7f5e50072650 msgr2=0x7f5e50074a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.368+0000 7f5e64ff9640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] conn(0x7f5e50072650 0x7f5e50074a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.368+0000 7f5e64ff9640 1 -- 172.21.15.115:0/842664554 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 msgr2=0x7f5e5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.368+0000 7f5e64ff9640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e5006a430 0x7f5e5006c8f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e60009ae0 tx=0x7f5e60009210 comp rx=0 tx=0).stop 2023-12-01T01:36:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.368+0000 7f5e64ff9640 1 -- 172.21.15.115:0/842664554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e7015d0b0 msgr2=0x7f5e700c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.368+0000 7f5e64ff9640 1 --2- 172.21.15.115:0/842664554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e7015d0b0 0x7f5e700c66d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5e68033000 tx=0x7f5e68004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.368+0000 7f5e6ed9d640 1 -- 172.21.15.115:0/842664554 reap_dead start 2023-12-01T01:36:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.367+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 --> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0d5c14e1c0 con 0x7f0d480726a0 2023-12-01T01:36:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.368+0000 7f0d43fff640 1 -- 172.21.15.115:0/4281159384 <== osd.3 v2:172.21.15.115:6826/88584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0d5c14e1c0 con 0x7f0d480726a0 2023-12-01T01:36:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.369+0000 7f7a54ff9640 1 -- 172.21.15.115:0/90002329 <== osd.6 v2:172.21.15.142:6808/88937 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7a38075e20 con 0x7f7a380732b0 2023-12-01T01:36:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.370+0000 7f5e64ff9640 1 -- 172.21.15.115:0/842664554 shutdown_connections 2023-12-01T01:36:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.370+0000 7f5e64ff9640 1 -- 172.21.15.115:0/842664554 >> 172.21.15.115:0/842664554 conn(0x7f5e7006a0b0 msgr2=0x7f5e700cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.370+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] conn(0x7f0d480726a0 msgr2=0x7f0d48074ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.370+0000 7f0d60899640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] conn(0x7f0d480726a0 0x7f0d48074ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.370+0000 7f5e64ff9640 1 -- 172.21.15.115:0/842664554 shutdown_connections 2023-12-01T01:36:22.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.370+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d4806a480 msgr2=0x7f0d4806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.370+0000 7f0d60899640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d4806a480 0x7f0d4806c940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c1519d0 tx=0x7f0d4c009190 comp rx=0 tx=0).stop 2023-12-01T01:36:22.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.370+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 msgr2=0x7f0d5c15c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.370+0000 7f0d60899640 1 --2- 172.21.15.115:0/4281159384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0d5c066450 0x7f0d5c15c640 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000e9b0 tx=0x7f0d5000ee70 comp rx=0 tx=0).stop 2023-12-01T01:36:22.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.370+0000 7f0d5bfff640 1 -- 172.21.15.115:0/4281159384 reap_dead start 2023-12-01T01:36:22.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.371+0000 7f5e64ff9640 1 -- 172.21.15.115:0/842664554 wait complete. 2023-12-01T01:36:22.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.371+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 shutdown_connections 2023-12-01T01:36:22.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.371+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 >> 172.21.15.115:0/4281159384 conn(0x7f0d5c065960 msgr2=0x7f0d5c06b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.371+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 shutdown_connections 2023-12-01T01:36:22.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.371+0000 7f0d60899640 1 -- 172.21.15.115:0/4281159384 wait complete. 2023-12-01T01:36:22.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.373+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 --> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb4f0002cb0 con 0x7fb4ec072680 2023-12-01T01:36:22.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.374+0000 7fb503fff640 1 -- 172.21.15.115:0/1122409629 <== osd.1 v2:172.21.15.115:6808/88580 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb4f0002cb0 con 0x7fb4ec072680 2023-12-01T01:36:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.375+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] conn(0x7fb4ec072680 msgr2=0x7fb4ec074aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.375+0000 7fb511773640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] conn(0x7fb4ec072680 0x7fb4ec074aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.375+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4ec06a460 msgr2=0x7fb4ec06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.375+0000 7fb511773640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4ec06a460 0x7fb4ec06c920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb50c15fbd0 tx=0x7fb4f4008040 comp rx=0 tx=0).stop 2023-12-01T01:36:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.375+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50c0b7f90 msgr2=0x7fb50c15b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.375+0000 7fb511773640 1 --2- 172.21.15.115:0/1122409629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb50c0b7f90 0x7fb50c15b8c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb50400cd60 tx=0x7fb50400a660 comp rx=0 tx=0).stop 2023-12-01T01:36:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.378+0000 7fb50b7fe640 1 -- 172.21.15.115:0/1122409629 reap_dead start 2023-12-01T01:36:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.378+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 shutdown_connections 2023-12-01T01:36:22.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.378+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 >> 172.21.15.115:0/1122409629 conn(0x7fb50c065960 msgr2=0x7fb50c0cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.407 INFO:teuthology.orchestra.run.smithi115.stdout:55834574856 2023-12-01T01:36:22.408 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-01T01:36:22.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.380+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 shutdown_connections 2023-12-01T01:36:22.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.380+0000 7fb511773640 1 -- 172.21.15.115:0/1122409629 wait complete. 2023-12-01T01:36:22.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.383+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 --> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f134815f8d0 con 0x7f1328072600 2023-12-01T01:36:22.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.384+0000 7f1344ff9640 1 -- 172.21.15.115:0/4259623833 <== osd.7 v2:172.21.15.142:6817/88947 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f134815f8d0 con 0x7f1328072600 2023-12-01T01:36:22.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.384+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] conn(0x7f1328072600 msgr2=0x7f1328074a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.384+0000 7f134deaf640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] conn(0x7f1328072600 0x7f1328074a20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.384+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f132806a460 msgr2=0x7f132806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.384+0000 7f134deaf640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f132806a460 0x7f132806c920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1330036000 tx=0x7f1330008030 comp rx=0 tx=0).stop 2023-12-01T01:36:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.384+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13480d21c0 msgr2=0x7f134814c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.384+0000 7f134deaf640 1 --2- 172.21.15.115:0/4259623833 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13480d21c0 0x7f134814c730 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f133800c3c0 tx=0x7f133800c880 comp rx=0 tx=0).stop 2023-12-01T01:36:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.385+0000 7f1347fff640 1 -- 172.21.15.115:0/4259623833 reap_dead start 2023-12-01T01:36:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.385+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 shutdown_connections 2023-12-01T01:36:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.385+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 >> 172.21.15.115:0/4259623833 conn(0x7f134806a0b0 msgr2=0x7f13480b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.385+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 shutdown_connections 2023-12-01T01:36:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.385+0000 7f134deaf640 1 -- 172.21.15.115:0/4259623833 wait complete. 2023-12-01T01:36:22.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.386+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 --> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8650056100 con 0x7f863c0725d0 2023-12-01T01:36:22.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.388+0000 7f864effd640 1 -- 172.21.15.115:0/3562067576 <== osd.4 v2:172.21.15.142:6800/88936 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8658072570 con 0x7f863c0725d0 2023-12-01T01:36:22.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.388+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] conn(0x7f863c0725d0 msgr2=0x7f863c0749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.388+0000 7f865f38e640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] conn(0x7f863c0725d0 0x7f863c0749f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.389+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f863c06a430 msgr2=0x7f863c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.389+0000 7f865f38e640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f863c06a430 0x7f863c06c8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f86500d1e80 tx=0x7f8648009040 comp rx=0 tx=0).stop 2023-12-01T01:36:22.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.389+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8650052300 msgr2=0x7f86500508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.389+0000 7f865f38e640 1 --2- 172.21.15.115:0/3562067576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8650052300 0x7f86500508e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f86540345b0 tx=0x7f86540046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.389+0000 7f865eb8d640 1 -- 172.21.15.115:0/3562067576 reap_dead start 2023-12-01T01:36:22.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.389+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 shutdown_connections 2023-12-01T01:36:22.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.389+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 >> 172.21.15.115:0/3562067576 conn(0x7f86500538a0 msgr2=0x7f86500dbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.389+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 shutdown_connections 2023-12-01T01:36:22.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.389+0000 7f865f38e640 1 -- 172.21.15.115:0/3562067576 wait complete. 2023-12-01T01:36:22.414 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:22.415 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-01T01:36:22.415 INFO:teuthology.orchestra.run.smithi115.stdout:55834574856 2023-12-01T01:36:22.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-01T01:36:22.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.398+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 --> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7a58161810 con 0x7f7a380732b0 2023-12-01T01:36:22.417 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:22.417 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-01T01:36:22.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.399+0000 7f7a54ff9640 1 -- 172.21.15.115:0/90002329 <== osd.6 v2:172.21.15.142:6808/88937 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7a40001580 con 0x7f7a380732b0 2023-12-01T01:36:22.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.400+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] conn(0x7f7a380732b0 msgr2=0x7f7a38075750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.400+0000 7f7a5e718640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] conn(0x7f7a380732b0 0x7f7a38075750 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.400+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a3806a430 msgr2=0x7f7a3806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.400+0000 7f7a5e718640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a3806a430 0x7f7a3806c8f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009aa0 tx=0x7f7a50009190 comp rx=0 tx=0).stop 2023-12-01T01:36:22.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.400+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a58161430 msgr2=0x7f7a580cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.400+0000 7f7a5e718640 1 --2- 172.21.15.115:0/90002329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a58161430 0x7f7a580cd3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009d90 tx=0x7f7a48004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:22.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.400+0000 7f7a5ccb5640 1 -- 172.21.15.115:0/90002329 reap_dead start 2023-12-01T01:36:22.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.400+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 shutdown_connections 2023-12-01T01:36:22.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.400+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 >> 172.21.15.115:0/90002329 conn(0x7f7a580cece0 msgr2=0x7f7a5815f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.401+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 shutdown_connections 2023-12-01T01:36:22.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.401+0000 7f7a5e718640 1 -- 172.21.15.115:0/90002329 wait complete. 2023-12-01T01:36:22.420 INFO:teuthology.orchestra.run.smithi115.stdout:55834574854 2023-12-01T01:36:22.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-01T01:36:22.421 INFO:teuthology.orchestra.run.smithi115.stdout:55834574856 2023-12-01T01:36:22.421 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-01T01:36:22.422 INFO:teuthology.orchestra.run.smithi115.stdout:55834574856 2023-12-01T01:36:22.422 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-01T01:36:22.423 INFO:teuthology.orchestra.run.smithi115.stdout:55834574857 2023-12-01T01:36:22.423 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-01T01:36:22.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.745+0000 7f258f9bf640 1 Processor -- start 2023-12-01T01:36:22.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.746+0000 7f258f9bf640 1 -- start start 2023-12-01T01:36:22.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.748+0000 7f258f9bf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f25800e11e0 0x7f2580038790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.749+0000 7f258f9bf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25800524b0 con 0x7f25800e15c0 2023-12-01T01:36:22.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.749+0000 7f258f9bf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2580055490 con 0x7f25800542b0 2023-12-01T01:36:22.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.750+0000 7f258f9bf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2580056930 con 0x7f25800e11e0 2023-12-01T01:36:22.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.749+0000 7f258cf5a640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f25800542b0 0x7f2580051690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:56816/0 (socket says 172.21.15.115:56816) 2023-12-01T01:36:22.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.750+0000 7f258cf5a640 1 -- 172.21.15.115:0/404659269 learned_addr learned my addr 172.21.15.115:0/404659269 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:22.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.750+0000 7f258d75b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f25800e11e0 0x7f2580038790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.751+0000 7f257ffff640 1 -- 172.21.15.115:0/404659269 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610801564 0 0) 0x7f2580055490 con 0x7f25800542b0 2023-12-01T01:36:22.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.751+0000 7f257ffff640 1 -- 172.21.15.115:0/404659269 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2574003520 con 0x7f25800542b0 2023-12-01T01:36:22.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.751+0000 7f257ffff640 1 -- 172.21.15.115:0/404659269 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830842752 0 0) 0x7f25800524b0 con 0x7f25800e15c0 2023-12-01T01:36:22.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.751+0000 7f258d75b640 1 -- 172.21.15.115:0/404659269 >> v1:172.21.15.142:6789/0 conn(0x7f25800542b0 legacy=0x7f2580051690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.751+0000 7f258d75b640 1 -- 172.21.15.115:0/404659269 >> v1:172.21.15.115:6789/0 conn(0x7f25800e15c0 legacy=0x7f2580051da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.752+0000 7f258d75b640 1 -- 172.21.15.115:0/404659269 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580056550 con 0x7f25800e11e0 2023-12-01T01:36:22.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.753+0000 7f3fa1a0e640 1 Processor -- start 2023-12-01T01:36:22.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.754+0000 7f258d75b640 1 --2- 172.21.15.115:0/404659269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25800e11e0 0x7f2580038790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25780097a0 tx=0x7f257802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d900b7f11858edbe server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.754+0000 7f3fa1a0e640 1 -- start start 2023-12-01T01:36:22.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.755+0000 7f257ffff640 1 -- 172.21.15.115:0/404659269 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578030860 con 0x7f25800e11e0 2023-12-01T01:36:22.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.755+0000 7f257ffff640 1 -- 172.21.15.115:0/404659269 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25780309c0 con 0x7f25800e11e0 2023-12-01T01:36:22.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.756+0000 7f257ffff640 1 -- 172.21.15.115:0/404659269 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578030860 con 0x7f25800e11e0 2023-12-01T01:36:22.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.756+0000 7f258f9bf640 1 -- 172.21.15.115:0/404659269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25800e11e0 msgr2=0x7f2580038790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.756+0000 7f258f9bf640 1 --2- 172.21.15.115:0/404659269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25800e11e0 0x7f2580038790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25780097a0 tx=0x7f257802b9a0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.756+0000 7f3fa1a0e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3f940e34a0 0x7f3f940543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.756+0000 7f3fa1a0e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3f94057400 0x7f3f9403c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.756+0000 7f258f9bf640 1 -- 172.21.15.115:0/404659269 shutdown_connections 2023-12-01T01:36:22.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f258f9bf640 1 --2- 172.21.15.115:0/404659269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25800e11e0 0x7f2580038790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f258f9bf640 1 -- 172.21.15.115:0/404659269 >> 172.21.15.115:0/404659269 conn(0x7f2580053790 msgr2=0x7f2580053ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f258f9bf640 1 -- 172.21.15.115:0/404659269 shutdown_connections 2023-12-01T01:36:22.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f258f9bf640 1 -- 172.21.15.115:0/404659269 wait complete. 2023-12-01T01:36:22.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.756+0000 7f3fa0a0c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3f940e34a0 0x7f3f940543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.756+0000 7f3fa0a0c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3f940e34a0 0x7f3f940543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53284/0 (socket says 172.21.15.115:53284) 2023-12-01T01:36:22.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f3f9bfff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f3f940582c0 0x7f3f94056cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:56820/0 (socket says 172.21.15.115:56820) 2023-12-01T01:36:22.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f3fa120d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3f94057400 0x7f3f9403c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f3fa120d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3f94057400 0x7f3f9403c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55936/0 (socket says 172.21.15.115:55936) 2023-12-01T01:36:22.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f3fa1a0e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f94058dd0 con 0x7f3f940582c0 2023-12-01T01:36:22.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f3fa120d640 1 -- 172.21.15.115:0/3988982187 learned_addr learned my addr 172.21.15.115:0/3988982187 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/3988982187 --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f94057b90 con 0x7f3f94057400 2023-12-01T01:36:22.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.757+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/3988982187 --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f94057e70 con 0x7f3f940e34a0 2023-12-01T01:36:22.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.758+0000 7f3fa120d640 1 -- 172.21.15.115:0/3988982187 >> v2:172.21.15.142:3300/0 conn(0x7f3f940e34a0 msgr2=0x7f3f940543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.758+0000 7f3fa120d640 1 --2- 172.21.15.115:0/3988982187 >> v2:172.21.15.142:3300/0 conn(0x7f3f940e34a0 0x7f3f940543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.758+0000 7f3fa120d640 1 -- 172.21.15.115:0/3988982187 >> v1:172.21.15.142:6789/0 conn(0x7f3f940582c0 legacy=0x7f3f94056cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.758+0000 7f3fa120d640 1 -- 172.21.15.115:0/3988982187 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f94053810 con 0x7f3f94057400 2023-12-01T01:36:22.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.760+0000 7f3fa120d640 1 --2- 172.21.15.115:0/3988982187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f94057400 0x7f3f9403c2c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009ad0 tx=0x7f3f88034340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdaef43afb902d86 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.760+0000 7f258f9bf640 1 Processor -- start 2023-12-01T01:36:22.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.760+0000 7f3f9b7fe640 1 -- 172.21.15.115:0/3988982187 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8803c440 con 0x7f3f94057400 2023-12-01T01:36:22.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.760+0000 7f258f9bf640 1 -- start start 2023-12-01T01:36:22.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.760+0000 7f3f9b7fe640 1 -- 172.21.15.115:0/3988982187 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8803c5a0 con 0x7f3f94057400 2023-12-01T01:36:22.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.760+0000 7f3f9b7fe640 1 -- 172.21.15.115:0/3988982187 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8803c930 con 0x7f3f94057400 2023-12-01T01:36:22.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/3988982187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f94057400 msgr2=0x7f3f9403c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f3fa1a0e640 1 --2- 172.21.15.115:0/3988982187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f94057400 0x7f3f9403c2c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009ad0 tx=0x7f3f88034340 comp rx=0 tx=0).stop 2023-12-01T01:36:22.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f258f9bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25800e11e0 0x7f25800d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f258f9bf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25800e15c0 0x7f25800e3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f258f9bf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f258003b7b0 0x7f258003bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f258f9bf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25800d6ac0 con 0x7f25800e15c0 2023-12-01T01:36:22.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f258f9bf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25800cc6f0 con 0x7f25800e11e0 2023-12-01T01:36:22.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f258f9bf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258003c320 con 0x7f258003b7b0 2023-12-01T01:36:22.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f258d75b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25800e11e0 0x7f25800d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f258cf5a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25800e15c0 0x7f25800e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f258cf5a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25800e15c0 0x7f25800e3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55946/0 (socket says 172.21.15.115:55946) 2023-12-01T01:36:22.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.762+0000 7f258cf5a640 1 -- 172.21.15.115:0/2803734494 learned_addr learned my addr 172.21.15.115:0/2803734494 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.763+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/3988982187 shutdown_connections 2023-12-01T01:36:22.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.763+0000 7f258df5c640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f258003b7b0 0x7f258003bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.763+0000 7f3fa1a0e640 1 --2- 172.21.15.115:0/3988982187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f94057400 0x7f3f9403c2c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.763+0000 7f3fa1a0e640 1 --2- 172.21.15.115:0/3988982187 >> v2:172.21.15.142:3300/0 conn(0x7f3f940e34a0 0x7f3f940543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.763+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/3988982187 >> 172.21.15.115:0/3988982187 conn(0x7f3f94059120 msgr2=0x7f3f94059530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.764+0000 7f258cf5a640 1 -- 172.21.15.115:0/2803734494 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f258003b7b0 msgr2=0x7f258003bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.764+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/3988982187 shutdown_connections 2023-12-01T01:36:22.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.764+0000 7f258cf5a640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f258003b7b0 0x7f258003bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.764+0000 7f258cf5a640 1 -- 172.21.15.115:0/2803734494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25800e11e0 msgr2=0x7f25800d77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.764+0000 7f258cf5a640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25800e11e0 0x7f25800d77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.764+0000 7f258cf5a640 1 -- 172.21.15.115:0/2803734494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578009400 con 0x7f25800e15c0 2023-12-01T01:36:22.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.764+0000 7f258d75b640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25800e11e0 0x7f25800d77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:22.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.764+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/3988982187 wait complete. 2023-12-01T01:36:22.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.765+0000 7f258cf5a640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25800e15c0 0x7f25800e3840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f258400c430 tx=0x7f258400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.766+0000 7f257dffb640 1 -- 172.21.15.115:0/2803734494 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584005ce0 con 0x7f25800e15c0 2023-12-01T01:36:22.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.766+0000 7f257dffb640 1 -- 172.21.15.115:0/2803734494 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2584005e40 con 0x7f25800e15c0 2023-12-01T01:36:22.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.766+0000 7f277b869640 1 Processor -- start 2023-12-01T01:36:22.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.766+0000 7f277b869640 1 -- start start 2023-12-01T01:36:22.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.767+0000 7f3fa1a0e640 1 Processor -- start 2023-12-01T01:36:22.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.766+0000 7f257dffb640 1 -- 172.21.15.115:0/2803734494 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840072f0 con 0x7f25800e15c0 2023-12-01T01:36:22.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.767+0000 7f258f9bf640 1 -- 172.21.15.115:0/2803734494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25800d6d10 con 0x7f25800e15c0 2023-12-01T01:36:22.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.767+0000 7f258f9bf640 1 -- 172.21.15.115:0/2803734494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25800d72d0 con 0x7f25800e15c0 2023-12-01T01:36:22.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.769+0000 7f3fa1a0e640 1 -- start start 2023-12-01T01:36:22.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.767+0000 7fc6e2907640 1 Processor -- start 2023-12-01T01:36:22.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.767+0000 7fc6e2907640 1 -- start start 2023-12-01T01:36:22.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.768+0000 7f277b869640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f277406d3d0 0x7f2774158c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.768+0000 7f277b869640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27740ccdf0 con 0x7f27740c8e60 2023-12-01T01:36:22.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.768+0000 7f277b869640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27740afb50 con 0x7f27740cb9d0 2023-12-01T01:36:22.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.768+0000 7f277b869640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277415fa00 con 0x7f277406d3d0 2023-12-01T01:36:22.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.769+0000 7f258f9bf640 1 -- 172.21.15.115:0/2803734494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25800d74f0 con 0x7f25800e15c0 2023-12-01T01:36:22.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.769+0000 7f2778e04640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f27740c8e60 0x7f277406f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51396/0 (socket says 172.21.15.115:51396) 2023-12-01T01:36:22.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.769+0000 7f2778e04640 1 -- 172.21.15.115:0/1922635360 learned_addr learned my addr 172.21.15.115:0/1922635360 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:22.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.769+0000 7fc6e2907640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 0x7fc6dc162060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.769+0000 7fc6e2907640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc6dc0b16a0 0x7fc6dc0cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.769+0000 7fc6e2907640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc071220 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.769+0000 7f2779605640 1 --2- 172.21.15.115:0/1922635360 >> v2:172.21.15.142:3300/0 conn(0x7f277406d3d0 0x7f2774158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.769+0000 7fc6e2907640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc1625a0 con 0x7fc6dc0b16a0 2023-12-01T01:36:22.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.769+0000 7fc6e2907640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0b1b60 con 0x7fc6dc070710 2023-12-01T01:36:22.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.771+0000 7f3fa1a0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f94057400 0x7f3f94178840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.771+0000 7f3fa1a0e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f940582c0 0x7f3f94174090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.771+0000 7f3fa1a0e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f940e34a0 0x7f3f9403dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.771+0000 7f3fa1a0e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f940db300 con 0x7f3f94057400 2023-12-01T01:36:22.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.771+0000 7f3fa1a0e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94178d80 con 0x7f3f940582c0 2023-12-01T01:36:22.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.771+0000 7f3fa1a0e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f940da450 con 0x7f3f940e34a0 2023-12-01T01:36:22.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.770+0000 7fc6e1905640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 0x7fc6dc162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.770+0000 7fc6e1905640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 0x7fc6dc162060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:33874/0 (socket says 172.21.15.115:33874) 2023-12-01T01:36:22.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.770+0000 7fc6e1905640 1 -- 172.21.15.115:0/930478806 learned_addr learned my addr 172.21.15.115:0/930478806 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.770+0000 7fc6e1104640 1 --2- 172.21.15.115:0/930478806 >> v2:172.21.15.142:3300/0 conn(0x7fc6dc0b16a0 0x7fc6dc0cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.771+0000 7f2779605640 1 -- 172.21.15.115:0/1922635360 >> v1:172.21.15.142:6789/0 conn(0x7f27740cb9d0 legacy=0x7f27740cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.771+0000 7f2773fff640 1 -- 172.21.15.115:0/1922635360 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164487143 0 0) 0x7f27740afb50 con 0x7f27740cb9d0 2023-12-01T01:36:22.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.772+0000 7f3f9bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f940582c0 0x7f3f94174090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.772+0000 7fc6e1104640 1 -- 172.21.15.115:0/930478806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 msgr2=0x7fc6dc162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.772+0000 7fc6e1104640 1 --2- 172.21.15.115:0/930478806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 0x7fc6dc162060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.772+0000 7fc6e1104640 1 -- 172.21.15.115:0/930478806 >> v1:172.21.15.142:6789/0 conn(0x7fc6dc06b2a0 legacy=0x7fc6dc0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.772+0000 7f3f9bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f940582c0 0x7f3f94174090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53328/0 (socket says 172.21.15.115:53328) 2023-12-01T01:36:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.772+0000 7f3f9bfff640 1 -- 172.21.15.115:0/2945418393 learned_addr learned my addr 172.21.15.115:0/2945418393 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.772+0000 7f3fa0a0c640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f94057400 0x7f3f94178840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.772+0000 7f3fa120d640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f940e34a0 0x7f3f9403dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.773+0000 7f3fa0a0c640 1 -- 172.21.15.115:0/2945418393 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f940e34a0 msgr2=0x7f3f9403dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.773+0000 7f3fa0a0c640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f940e34a0 0x7f3f9403dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.773+0000 7f3fa0a0c640 1 -- 172.21.15.115:0/2945418393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f940582c0 msgr2=0x7f3f94174090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.773+0000 7f3fa0a0c640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f940582c0 0x7f3f94174090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.773+0000 7f3fa0a0c640 1 -- 172.21.15.115:0/2945418393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f880094d0 con 0x7f3f94057400 2023-12-01T01:36:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.773+0000 7f3fa0a0c640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f94057400 0x7f3f94178840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0683c0 tx=0x7f3f9c067060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.772+0000 7f2779605640 1 -- 172.21.15.115:0/1922635360 >> v1:172.21.15.115:6789/0 conn(0x7f27740c8e60 legacy=0x7f277406f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.772+0000 7fc6e1104640 1 -- 172.21.15.115:0/930478806 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc0b1e40 con 0x7fc6dc0b16a0 2023-12-01T01:36:22.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.773+0000 7f2779605640 1 -- 172.21.15.115:0/1922635360 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27740afe30 con 0x7f277406d3d0 2023-12-01T01:36:22.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.774+0000 7f257dffb640 1 -- 172.21.15.115:0/2803734494 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2584007450 con 0x7f25800e15c0 2023-12-01T01:36:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.775+0000 7fc6e1104640 1 --2- 172.21.15.115:0/930478806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc0b16a0 0x7fc6dc0cb510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009d90 tx=0x7fc6d802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ccee230119682a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.775+0000 7f2779605640 1 --2- 172.21.15.115:0/1922635360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f277406d3d0 0x7f2774158c00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2768009d90 tx=0x7f276802bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60c6f884cab14f50 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.775+0000 7fc6e0903640 1 -- 172.21.15.115:0/930478806 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d802cbc0 con 0x7fc6dc0b16a0 2023-12-01T01:36:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.775+0000 7fc6e0903640 1 -- 172.21.15.115:0/930478806 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d802cd20 con 0x7fc6dc0b16a0 2023-12-01T01:36:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.775+0000 7fc6e0903640 1 -- 172.21.15.115:0/930478806 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8034660 con 0x7fc6dc0b16a0 2023-12-01T01:36:22.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.775+0000 7f2773fff640 1 -- 172.21.15.115:0/1922635360 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276802cd80 con 0x7f277406d3d0 2023-12-01T01:36:22.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.775+0000 7f2773fff640 1 -- 172.21.15.115:0/1922635360 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2768034440 con 0x7f277406d3d0 2023-12-01T01:36:22.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.775+0000 7f2773fff640 1 -- 172.21.15.115:0/1922635360 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680347d0 con 0x7f277406d3d0 2023-12-01T01:36:22.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.776+0000 7f257dffb640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f256c06a430 0x7f256c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.776+0000 7f257dffb640 1 -- 172.21.15.115:0/2803734494 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25840882d0 con 0x7f25800e15c0 2023-12-01T01:36:22.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.776+0000 7f277b869640 1 -- 172.21.15.115:0/1922635360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f277406d3d0 msgr2=0x7f2774158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.776+0000 7fc6e2907640 1 -- 172.21.15.115:0/930478806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc0b16a0 msgr2=0x7fc6dc0cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.776+0000 7f277b869640 1 --2- 172.21.15.115:0/1922635360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f277406d3d0 0x7f2774158c00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2768009d90 tx=0x7f276802bfd0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.776+0000 7fc6e2907640 1 --2- 172.21.15.115:0/930478806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc0b16a0 0x7fc6dc0cb510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009d90 tx=0x7fc6d802be10 comp rx=0 tx=0).stop 2023-12-01T01:36:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.776+0000 7f277b869640 1 -- 172.21.15.115:0/1922635360 shutdown_connections 2023-12-01T01:36:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.777+0000 7f277b869640 1 --2- 172.21.15.115:0/1922635360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f277406d3d0 0x7f2774158c00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.777+0000 7f277b869640 1 -- 172.21.15.115:0/1922635360 >> 172.21.15.115:0/1922635360 conn(0x7f2774065b40 msgr2=0x7f2774065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.777+0000 7fc6e2907640 1 -- 172.21.15.115:0/930478806 shutdown_connections 2023-12-01T01:36:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.777+0000 7fc6e2907640 1 --2- 172.21.15.115:0/930478806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc0b16a0 0x7fc6dc0cb510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.777+0000 7f9ad0a65640 1 Processor -- start 2023-12-01T01:36:22.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.777+0000 7fc6e2907640 1 --2- 172.21.15.115:0/930478806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 0x7fc6dc162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.777+0000 7fc6e2907640 1 -- 172.21.15.115:0/930478806 >> 172.21.15.115:0/930478806 conn(0x7fc6dc065960 msgr2=0x7fc6dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.777+0000 7fc6e2907640 1 -- 172.21.15.115:0/930478806 shutdown_connections 2023-12-01T01:36:22.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.777+0000 7f9ad0a65640 1 -- start start 2023-12-01T01:36:22.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.778+0000 7f277b869640 1 -- 172.21.15.115:0/1922635360 shutdown_connections 2023-12-01T01:36:22.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.777+0000 7f258d75b640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f256c06a430 0x7f256c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.778+0000 7fc6e2907640 1 -- 172.21.15.115:0/930478806 wait complete. 2023-12-01T01:36:22.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.778+0000 7f277b869640 1 -- 172.21.15.115:0/1922635360 wait complete. 2023-12-01T01:36:22.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.778+0000 7f9ad0a65640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9acc15ccc0 0x7f9acc0d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.778+0000 7f9ad0a65640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc0b8420 0x7f9acc0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.778+0000 7f9ad0a65640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc0d1fe0 con 0x7f9acc0cf430 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.778+0000 7f9ad0a65640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc0d13d0 con 0x7f9acc15ccc0 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.778+0000 7f9ad0a65640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0d16b0 con 0x7f9acc0b8420 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.779+0000 7f9acbfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9acc0cf430 0x7f9acc0b44a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51400/0 (socket says 172.21.15.115:51400) 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.779+0000 7f9acbfff640 1 -- 172.21.15.115:0/3058126492 learned_addr learned my addr 172.21.15.115:0/3058126492 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.779+0000 7f9acb7fe640 1 --2- 172.21.15.115:0/3058126492 >> v2:172.21.15.142:3300/0 conn(0x7f9acc15ccc0 0x7f9acc0d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.780+0000 7f9acb7fe640 1 -- 172.21.15.115:0/3058126492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc0b8420 msgr2=0x7f9acc0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.780+0000 7f9acb7fe640 1 --2- 172.21.15.115:0/3058126492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc0b8420 0x7f9acc0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.780+0000 7f9acb7fe640 1 -- 172.21.15.115:0/3058126492 >> v1:172.21.15.115:6789/0 conn(0x7f9acc0cf430 legacy=0x7f9acc0b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.780+0000 7f9acb7fe640 1 -- 172.21.15.115:0/3058126492 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0d1990 con 0x7f9acc15ccc0 2023-12-01T01:36:22.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.780+0000 7f258d75b640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f256c06a430 0x7f256c06c8f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2578036000 tx=0x7f2578008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.780+0000 7fc6e2907640 1 Processor -- start 2023-12-01T01:36:22.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.781+0000 7fb1240c7640 1 Processor -- start 2023-12-01T01:36:22.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.781+0000 7fc6e2907640 1 -- start start 2023-12-01T01:36:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.781+0000 7f277b869640 1 Processor -- start 2023-12-01T01:36:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.781+0000 7f277b869640 1 -- start start 2023-12-01T01:36:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.781+0000 7fb1240c7640 1 -- start start 2023-12-01T01:36:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f257dffb640 1 -- 172.21.15.115:0/2803734494 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2584053a10 con 0x7f25800e15c0 2023-12-01T01:36:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f9acb7fe640 1 --2- 172.21.15.115:0/3058126492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9acc15ccc0 0x7f9acc0d2470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0097a0 tx=0x7f9abc02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75da01f220052715 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f3f997fa640 1 -- 172.21.15.115:0/2945418393 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c092430 con 0x7f3f94057400 2023-12-01T01:36:22.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f9aca7fc640 1 -- 172.21.15.115:0/3058126492 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc02d040 con 0x7f9acc15ccc0 2023-12-01T01:36:22.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f277b869640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f277406d3d0 0x7f277414b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f9aca7fc640 1 -- 172.21.15.115:0/3058126492 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc031810 con 0x7f9acc15ccc0 2023-12-01T01:36:22.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f3f997fa640 1 -- 172.21.15.115:0/2945418393 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9c092590 con 0x7f3f94057400 2023-12-01T01:36:22.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f3f997fa640 1 -- 172.21.15.115:0/2945418393 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c092940 con 0x7f3f94057400 2023-12-01T01:36:22.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7fc6e2907640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc06b2a0 0x7fc6dc15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f277b869640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27740c8e60 0x7f27740c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f277b869640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27740cb9d0 0x7f27740c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f277b869640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27740c7e80 con 0x7f27740c8e60 2023-12-01T01:36:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f277b869640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27740c8160 con 0x7f27740cb9d0 2023-12-01T01:36:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f277b869640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27740c8440 con 0x7f277406d3d0 2023-12-01T01:36:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.782+0000 7f9aca7fc640 1 -- 172.21.15.115:0/3058126492 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc031ba0 con 0x7f9acc15ccc0 2023-12-01T01:36:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7fb1240c7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb11c0d21c0 0x7fb11c0d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7fc6e2907640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 0x7fc6dc0cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7fc6e2907640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6dc0b16a0 0x7fc6dc1f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7fb1240c7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0d1120 con 0x7fb11c15d0b0 2023-12-01T01:36:22.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7fb1240c7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c0d1400 con 0x7fb11c0d2600 2023-12-01T01:36:22.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7fb1240c7640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11c0b2330 con 0x7fb11c0d21c0 2023-12-01T01:36:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7fc6e2907640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc15d380 con 0x7fc6dc0b16a0 2023-12-01T01:36:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7fc6e2907640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc1f7420 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7fc6e2907640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc1f7140 con 0x7fc6dc070710 2023-12-01T01:36:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7fc6e1905640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc06b2a0 0x7fc6dc15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7fc6e1905640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc06b2a0 0x7fc6dc15c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53350/0 (socket says 172.21.15.115:53350) 2023-12-01T01:36:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7fc6e1905640 1 -- 172.21.15.115:0/3680878385 learned_addr learned my addr 172.21.15.115:0/3680878385 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7fc6e2106640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6dc0b16a0 0x7fc6dc1f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7fc6e1104640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 0x7fc6dc0cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/2945418393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f940da6f0 con 0x7f3f94057400 2023-12-01T01:36:22.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7fb121e63640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb11c15d0b0 0x7fb11c0b8500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51410/0 (socket says 172.21.15.115:51410) 2023-12-01T01:36:22.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7fb121e63640 1 -- 172.21.15.115:0/2871553015 learned_addr learned my addr 172.21.15.115:0/2871553015 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:22.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/2945418393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f940dacb0 con 0x7f3f94057400 2023-12-01T01:36:22.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7fb121662640 1 --2- 172.21.15.115:0/2871553015 >> v2:172.21.15.115:3300/0 conn(0x7fb11c0d21c0 0x7fb11c0d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7f2779605640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f277406d3d0 0x7f277414b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7f2778e04640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27740c8e60 0x7f27740c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7f2779605640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f277406d3d0 0x7f277414b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:33892/0 (socket says 172.21.15.115:33892) 2023-12-01T01:36:22.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7f2779605640 1 -- 172.21.15.115:0/2935881692 learned_addr learned my addr 172.21.15.115:0/2935881692 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.783+0000 7f2779e06640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27740cb9d0 0x7f27740c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7f2779605640 1 -- 172.21.15.115:0/2935881692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27740cb9d0 msgr2=0x7f27740c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7f2779605640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27740cb9d0 0x7f27740c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.784+0000 7f2779605640 1 -- 172.21.15.115:0/2935881692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27740c8e60 msgr2=0x7f27740c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7fc6e1905640 1 -- 172.21.15.115:0/3680878385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 msgr2=0x7fc6dc0cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7fc6e1905640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 0x7fc6dc0cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7fc6e1905640 1 -- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6dc0b16a0 msgr2=0x7fc6dc1f64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7f2ab2cb9640 1 Processor -- start 2023-12-01T01:36:22.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494119182 0 0) 0x7fb11c0d1120 con 0x7fb11c15d0b0 2023-12-01T01:36:22.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb10c003520 con 0x7fb11c15d0b0 2023-12-01T01:36:22.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7f9ad0a65640 1 -- 172.21.15.115:0/3058126492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9acc15ccc0 msgr2=0x7f9acc0d2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7f9ad0a65640 1 --2- 172.21.15.115:0/3058126492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9acc15ccc0 0x7f9acc0d2470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0097a0 tx=0x7f9abc02b8b0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7f2ab2cb9640 1 -- start start 2023-12-01T01:36:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7f2779605640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27740c8e60 0x7f27740c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7f2779605640 1 -- 172.21.15.115:0/2935881692 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2768009a70 con 0x7f277406d3d0 2023-12-01T01:36:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7fc6e1905640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6dc0b16a0 0x7fc6dc1f64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7fc6e1905640 1 -- 172.21.15.115:0/3680878385 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d8009a70 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7fc6e2106640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6dc0b16a0 0x7fc6dc1f64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:22.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7f9ad0a65640 1 -- 172.21.15.115:0/3058126492 shutdown_connections 2023-12-01T01:36:22.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7f9ad0a65640 1 --2- 172.21.15.115:0/3058126492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc0b8420 0x7f9acc0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7f9ad0a65640 1 --2- 172.21.15.115:0/3058126492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9acc15ccc0 0x7f9acc0d2470 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7f9ad0a65640 1 -- 172.21.15.115:0/3058126492 >> 172.21.15.115:0/3058126492 conn(0x7f9acc065960 msgr2=0x7f9acc065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7f2778e04640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27740c8e60 0x7f27740c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:22.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954691257 0 0) 0x7fb11c0d1400 con 0x7fb11c0d2600 2023-12-01T01:36:22.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.785+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb11c0d1120 con 0x7fb11c0d2600 2023-12-01T01:36:22.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7f2779605640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f277406d3d0 0x7f277414b520 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2768000f80 tx=0x7f2768004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2140772741 0 0) 0x7fb10c003520 con 0x7fb11c15d0b0 2023-12-01T01:36:22.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 >> v2:172.21.15.115:3300/0 conn(0x7fb11c0d21c0 msgr2=0x7fb11c0d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 --2- 172.21.15.115:0/2871553015 >> v2:172.21.15.115:3300/0 conn(0x7fb11c0d21c0 0x7fb11c0d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 >> v1:172.21.15.142:6789/0 conn(0x7fb11c0d2600 legacy=0x7fb11c0b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7f9ad0a65640 1 -- 172.21.15.115:0/3058126492 shutdown_connections 2023-12-01T01:36:22.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7f9ad0a65640 1 -- 172.21.15.115:0/3058126492 wait complete. 2023-12-01T01:36:22.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fc6e1905640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc06b2a0 0x7fc6dc15c8b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009fd0 tx=0x7fc6cc00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7f2771ffb640 1 -- 172.21.15.115:0/2935881692 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768003b90 con 0x7f277406d3d0 2023-12-01T01:36:22.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7f2771ffb640 1 -- 172.21.15.115:0/2935881692 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2768003cf0 con 0x7f277406d3d0 2023-12-01T01:36:22.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7f2771ffb640 1 -- 172.21.15.115:0/2935881692 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768004270 con 0x7f277406d3d0 2023-12-01T01:36:22.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7f277b869640 1 -- 172.21.15.115:0/2935881692 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27740c8720 con 0x7f277406d3d0 2023-12-01T01:36:22.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7f277b869640 1 -- 172.21.15.115:0/2935881692 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277414ba60 con 0x7f277406d3d0 2023-12-01T01:36:22.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/2945418393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f940058e0 con 0x7f3f94057400 2023-12-01T01:36:22.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7f2ab2cb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0b6090 0x7f2aac0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7f2ab2cb9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac160750 con 0x7f2aac0cf740 2023-12-01T01:36:22.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7f2ab2cb9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac0ce380 con 0x7f2aac160370 2023-12-01T01:36:22.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7f2ab2cb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0ce660 con 0x7f2aac0b6090 2023-12-01T01:36:22.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c0d16e0 con 0x7fb11c15d0b0 2023-12-01T01:36:22.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb110001a40 con 0x7fb11c15d0b0 2023-12-01T01:36:22.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 >> v1:172.21.15.115:6789/0 conn(0x7fb11c15d0b0 legacy=0x7fb11c0b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7fc6d27fc640 1 -- 172.21.15.115:0/3680878385 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00c940 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7fc6d27fc640 1 -- 172.21.15.115:0/3680878385 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc00caa0 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7fc6d27fc640 1 -- 172.21.15.115:0/3680878385 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00ce30 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.788+0000 7fc6e2907640 1 -- 172.21.15.115:0/3680878385 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc15bdd0 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.788+0000 7f2ab0a55640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f2aac160370 0x7f2aac1613b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:56872/0 (socket says 172.21.15.115:56872) 2023-12-01T01:36:22.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.788+0000 7f2ab0a55640 1 -- 172.21.15.115:0/1890360544 learned_addr learned my addr 172.21.15.115:0/1890360544 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:22.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.788+0000 7f2aabfff640 1 --2- 172.21.15.115:0/1890360544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0b6090 0x7f2aac0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10c004b60 0x7fb10c004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c003760 0x7fb10c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb10c004120 0x7fb10c014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c014780 con 0x7fb10c003760 2023-12-01T01:36:22.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0148c0 con 0x7fb10c004120 2023-12-01T01:36:22.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.786+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c014a80 con 0x7fb10c004b60 2023-12-01T01:36:22.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.788+0000 7fc6e2907640 1 -- 172.21.15.115:0/3680878385 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc15c390 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7fb121662640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c003760 0x7fb10c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.787+0000 7fb121e63640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10c004b60 0x7fb10c004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.788+0000 7fb121e63640 1 -- 172.21.15.115:0/2871553015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb10c004120 msgr2=0x7fb10c014190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-01T01:36:22.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.788+0000 7fb121e63640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb10c004120 0x7fb10c014190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.788+0000 7fb121e63640 1 -- 172.21.15.115:0/2871553015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c003760 msgr2=0x7fb10c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.789+0000 7f2aabfff640 1 -- 172.21.15.115:0/1890360544 >> v1:172.21.15.142:6789/0 conn(0x7f2aac160370 legacy=0x7f2aac1613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.789+0000 7f2aabfff640 1 -- 172.21.15.115:0/1890360544 >> v1:172.21.15.115:6789/0 conn(0x7f2aac0cf740 legacy=0x7f2aac0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.789+0000 7f9ad0a65640 1 Processor -- start 2023-12-01T01:36:22.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.790+0000 7f9ad0a65640 1 -- start start 2023-12-01T01:36:22.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.789+0000 7f2aabfff640 1 -- 172.21.15.115:0/1890360544 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac0d1120 con 0x7f2aac0b6090 2023-12-01T01:36:22.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.791+0000 7fb121e63640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c003760 0x7fb10c003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.791+0000 7fb121e63640 1 -- 172.21.15.115:0/2871553015 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c0d16e0 con 0x7fb10c004b60 2023-12-01T01:36:22.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.791+0000 7fb122664640 1 -- 172.21.15.115:0/2871553015 reap_dead start 2023-12-01T01:36:22.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.791+0000 7fb121662640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c003760 0x7fb10c003be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:22.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.791+0000 7fc6e2907640 1 -- 172.21.15.115:0/3680878385 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c4005250 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.789+0000 7f277b869640 1 -- 172.21.15.115:0/2935881692 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2754005250 con 0x7f277406d3d0 2023-12-01T01:36:22.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.791+0000 7f2aabfff640 1 --2- 172.21.15.115:0/1890360544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0b6090 0x7f2aac0cdcf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2a980097d0 tx=0x7f2a9802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2580aa73ae6cd54 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.792+0000 7f3f997fa640 1 -- 172.21.15.115:0/2945418393 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3f9c092aa0 con 0x7f3f94057400 2023-12-01T01:36:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.792+0000 7f9ad0a65640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9acc0b8420 0x7f9acc1f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.792+0000 7f9ad0a65640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9acc0cf430 0x7f9acc0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.792+0000 7f9ad0a65640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc15ccc0 0x7f9acc0b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.792+0000 7f9ad0a65640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b16a0 con 0x7f9acc0b8420 2023-12-01T01:36:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.792+0000 7f9ad0a65640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1f31f0 con 0x7f9acc0cf430 2023-12-01T01:36:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.792+0000 7f9ad0a65640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b18f0 con 0x7f9acc15ccc0 2023-12-01T01:36:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.793+0000 7fb121e63640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10c004b60 0x7fb10c004f60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb11000a880 tx=0x7fb10c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d06e2b349ff6fc1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.793+0000 7f9acb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9acc0b8420 0x7f9acc1f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.793+0000 7f9acbfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc15ccc0 0x7f9acc0b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.793+0000 7f9acbfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc15ccc0 0x7f9acc0b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:33916/0 (socket says 172.21.15.115:33916) 2023-12-01T01:36:22.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.793+0000 7f9acbfff640 1 -- 172.21.15.115:0/4223556753 learned_addr learned my addr 172.21.15.115:0/4223556753 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.794+0000 7f3f997fa640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3f8006a530 0x7f3f8006c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.794+0000 7f9acaffd640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9acc0cf430 0x7f9acc0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.794+0000 7f3f997fa640 1 -- 172.21.15.115:0/2945418393 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f9c102a90 con 0x7f3f94057400 2023-12-01T01:36:22.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.794+0000 7f83cab8b640 1 Processor -- start 2023-12-01T01:36:22.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.794+0000 7f3f9bfff640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3f8006a530 0x7f3f8006c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.794+0000 7f83cab8b640 1 -- start start 2023-12-01T01:36:22.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.793+0000 7fc6d27fc640 1 -- 172.21.15.115:0/3680878385 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc6cc013470 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.795+0000 7fc6d27fc640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c006a500 0x7fc6c006c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.795+0000 7f9acbfff640 1 -- 172.21.15.115:0/4223556753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9acc0cf430 msgr2=0x7f9acc0b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.795+0000 7f9acbfff640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9acc0cf430 0x7f9acc0b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.795+0000 7f9acbfff640 1 -- 172.21.15.115:0/4223556753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9acc0b8420 msgr2=0x7f9acc1f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.795+0000 7f9acbfff640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9acc0b8420 0x7f9acc1f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.795+0000 7f9acbfff640 1 -- 172.21.15.115:0/4223556753 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc009400 con 0x7f9acc15ccc0 2023-12-01T01:36:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.795+0000 7fc6d27fc640 1 -- 172.21.15.115:0/3680878385 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6cc0889e0 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.794+0000 7f2771ffb640 1 -- 172.21.15.115:0/2935881692 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f276803b050 con 0x7f277406d3d0 2023-12-01T01:36:22.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.795+0000 7fc6e1104640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c006a500 0x7fc6c006c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.795+0000 7f9acbfff640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc15ccc0 0x7f9acc0b33b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400cb40 tx=0x7f9ac400cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.795+0000 7f3f9bfff640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3f8006a530 0x7f3f8006c9f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f90007fd0 tx=0x7f3f90007d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.796+0000 7f2771ffb640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f275806a530 0x7f275806c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.796+0000 7f2771ffb640 1 -- 172.21.15.115:0/2935881692 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27680ab680 con 0x7f277406d3d0 2023-12-01T01:36:22.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.796+0000 7f2778e04640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f275806a530 0x7f275806c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.796+0000 7f9acb7fe640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9acc0b8420 0x7f9acc1f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:22.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.796+0000 7fc6e1104640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c006a500 0x7fc6c006c9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8036000 tx=0x7fc6d8030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.797+0000 7f83cab8b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f83c415d0b0 0x7f83c40b8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.797+0000 7f83cab8b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f83c40d21c0 0x7f83c40d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.797+0000 7f3f997fa640 1 -- 172.21.15.115:0/2945418393 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3f9c0ce060 con 0x7f3f94057400 2023-12-01T01:36:22.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.797+0000 7f2778e04640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f275806a530 0x7f275806c9f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f277414f310 tx=0x7f2760009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.797+0000 7f83cab8b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c40d1120 con 0x7f83c40d2620 2023-12-01T01:36:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.797+0000 7f83cab8b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c40b2330 con 0x7f83c415d0b0 2023-12-01T01:36:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.797+0000 7f83cab8b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c40d1400 con 0x7f83c40d21c0 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.797+0000 7f83c8927640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f83c415d0b0 0x7f83c40b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.797+0000 7f83c8927640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f83c415d0b0 0x7f83c40b8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56002/0 (socket says 172.21.15.115:56002) 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.797+0000 7f83c8927640 1 -- 172.21.15.115:0/3366424842 learned_addr learned my addr 172.21.15.115:0/3366424842 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.798+0000 7f83c8927640 1 -- 172.21.15.115:0/3366424842 >> v2:172.21.15.142:3300/0 conn(0x7f83c40d21c0 msgr2=0x7f83c40d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.798+0000 7f83c8927640 1 --2- 172.21.15.115:0/3366424842 >> v2:172.21.15.142:3300/0 conn(0x7f83c40d21c0 0x7f83c40d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.798+0000 7f83c8927640 1 -- 172.21.15.115:0/3366424842 >> v1:172.21.15.142:6789/0 conn(0x7f83c40d2620 legacy=0x7f83c40b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.798+0000 7f83c8927640 1 -- 172.21.15.115:0/3366424842 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c40d16e0 con 0x7f83c415d0b0 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.800+0000 7f83c8927640 1 --2- 172.21.15.115:0/3366424842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 0x7f83c40b8500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83c40d0760 tx=0x7f83b802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8001d54570ac43ca server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.800+0000 7f83c37fe640 1 -- 172.21.15.115:0/3366424842 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8004230 con 0x7f83c415d0b0 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.800+0000 7f83c37fe640 1 -- 172.21.15.115:0/3366424842 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b8004390 con 0x7f83c415d0b0 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.800+0000 7f83c37fe640 1 -- 172.21.15.115:0/3366424842 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8030840 con 0x7f83c415d0b0 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.801+0000 7f83cab8b640 1 -- 172.21.15.115:0/3366424842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 msgr2=0x7f83c40b8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.801+0000 7f83cab8b640 1 --2- 172.21.15.115:0/3366424842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 0x7f83c40b8500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83c40d0760 tx=0x7f83b802b980 comp rx=0 tx=0).stop 2023-12-01T01:36:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.801+0000 7f83cab8b640 1 -- 172.21.15.115:0/3366424842 shutdown_connections 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.801+0000 7f83cab8b640 1 --2- 172.21.15.115:0/3366424842 >> v2:172.21.15.142:3300/0 conn(0x7f83c40d21c0 0x7f83c40d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.801+0000 7f83cab8b640 1 --2- 172.21.15.115:0/3366424842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 0x7f83c40b8500 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.801+0000 7f83cab8b640 1 -- 172.21.15.115:0/3366424842 >> 172.21.15.115:0/3366424842 conn(0x7f83c406a0b0 msgr2=0x7f83c406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.801+0000 7f83cab8b640 1 -- 172.21.15.115:0/3366424842 shutdown_connections 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.801+0000 7f83cab8b640 1 -- 172.21.15.115:0/3366424842 wait complete. 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.803+0000 7f83cab8b640 1 Processor -- start 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.803+0000 7f83cab8b640 1 -- start start 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.804+0000 7f83cab8b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c40d21c0 0x7f83c41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.804+0000 7f83cab8b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c40d2620 0x7f83c41f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.804+0000 7f83cab8b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 0x7f83c40cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.804+0000 7f83cab8b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c41566f0 con 0x7f83c415d0b0 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.804+0000 7f83cab8b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40cb100 con 0x7f83c40d21c0 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.804+0000 7f83cab8b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40ca4f0 con 0x7f83c40d2620 2023-12-01T01:36:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.804+0000 7f83c9128640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 0x7f83c40cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.804+0000 7f83c9128640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 0x7f83c40cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56012/0 (socket says 172.21.15.115:56012) 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.804+0000 7f83c9128640 1 -- 172.21.15.115:0/2925629800 learned_addr learned my addr 172.21.15.115:0/2925629800 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c8927640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c40d21c0 0x7f83c41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c3fff640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c40d2620 0x7f83c41f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c9128640 1 -- 172.21.15.115:0/2925629800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c40d2620 msgr2=0x7f83c41f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c9128640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c40d2620 0x7f83c41f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c9128640 1 -- 172.21.15.115:0/2925629800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c40d21c0 msgr2=0x7f83c41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c9128640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c40d21c0 0x7f83c41561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c9128640 1 -- 172.21.15.115:0/2925629800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b8009460 con 0x7f83c415d0b0 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c9128640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 0x7f83c40cb590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00c7b0 tx=0x7f83bc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c17fa640 1 -- 172.21.15.115:0/2925629800 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc003d80 con 0x7f83c415d0b0 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c17fa640 1 -- 172.21.15.115:0/2925629800 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83bc005070 con 0x7f83c415d0b0 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83cab8b640 1 -- 172.21.15.115:0/2925629800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c40ca7d0 con 0x7f83c415d0b0 2023-12-01T01:36:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.805+0000 7f83c17fa640 1 -- 172.21.15.115:0/2925629800 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc005420 con 0x7f83c415d0b0 2023-12-01T01:36:22.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.806+0000 7f83cab8b640 1 -- 172.21.15.115:0/2925629800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c40cad90 con 0x7f83c415d0b0 2023-12-01T01:36:22.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.807+0000 7f2aab7fe640 1 -- 172.21.15.115:0/1890360544 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98031890 con 0x7f2aac0b6090 2023-12-01T01:36:22.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.807+0000 7f83cab8b640 1 -- 172.21.15.115:0/2925629800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c404e890 con 0x7f83c415d0b0 2023-12-01T01:36:22.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.807+0000 7f2aab7fe640 1 -- 172.21.15.115:0/1890360544 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a980319f0 con 0x7f2aac0b6090 2023-12-01T01:36:22.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.807+0000 7f2aab7fe640 1 -- 172.21.15.115:0/1890360544 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98031d80 con 0x7f2aac0b6090 2023-12-01T01:36:22.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100072d0 con 0x7fb10c004b60 2023-12-01T01:36:22.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11002f070 con 0x7fb10c004b60 2023-12-01T01:36:22.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7fb120e61640 1 -- 172.21.15.115:0/2871553015 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100076b0 con 0x7fb10c004b60 2023-12-01T01:36:22.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7f2771ffb640 1 -- 172.21.15.115:0/2935881692 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2768076ce0 con 0x7f277406d3d0 2023-12-01T01:36:22.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7f9ab3fff640 1 -- 172.21.15.115:0/4223556753 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400ac30 con 0x7f9acc15ccc0 2023-12-01T01:36:22.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7f2ab2cb9640 1 -- 172.21.15.115:0/1890360544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0b6090 msgr2=0x7f2aac0cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7f9ad0a65640 1 -- 172.21.15.115:0/4223556753 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc0b1bd0 con 0x7f9acc15ccc0 2023-12-01T01:36:22.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7f2ab2cb9640 1 --2- 172.21.15.115:0/1890360544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0b6090 0x7f2aac0cdcf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2a980097d0 tx=0x7f2a9802b930 comp rx=0 tx=0).stop 2023-12-01T01:36:22.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7fb1240c7640 1 -- 172.21.15.115:0/2871553015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10c004b60 msgr2=0x7fb10c004f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7fb1240c7640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10c004b60 0x7fb10c004f60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb11000a880 tx=0x7fb10c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.808+0000 7f9ad0a65640 1 -- 172.21.15.115:0/4223556753 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc0b2190 con 0x7f9acc15ccc0 2023-12-01T01:36:22.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.809+0000 7f9ab3fff640 1 -- 172.21.15.115:0/4223556753 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac400ad90 con 0x7f9acc15ccc0 2023-12-01T01:36:22.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.809+0000 7fb1240c7640 1 -- 172.21.15.115:0/2871553015 shutdown_connections 2023-12-01T01:36:22.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.809+0000 7fb1240c7640 1 --2- 172.21.15.115:0/2871553015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10c004b60 0x7fb10c004f60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.809+0000 7f9ab3fff640 1 -- 172.21.15.115:0/4223556753 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400ac30 con 0x7f9acc15ccc0 2023-12-01T01:36:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.809+0000 7fb1240c7640 1 -- 172.21.15.115:0/2871553015 >> 172.21.15.115:0/2871553015 conn(0x7fb11c06a0b0 msgr2=0x7fb11c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.809+0000 7f2ab2cb9640 1 -- 172.21.15.115:0/1890360544 shutdown_connections 2023-12-01T01:36:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.809+0000 7f2ab2cb9640 1 --2- 172.21.15.115:0/1890360544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0b6090 0x7f2aac0cdcf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.809+0000 7f2ab2cb9640 1 -- 172.21.15.115:0/1890360544 >> 172.21.15.115:0/1890360544 conn(0x7f2aac065960 msgr2=0x7f2aac065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:22.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.809+0000 7f83c17fa640 1 -- 172.21.15.115:0/2925629800 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f83bc005580 con 0x7f83c415d0b0 2023-12-01T01:36:22.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.810+0000 7fb1240c7640 1 -- 172.21.15.115:0/2871553015 shutdown_connections 2023-12-01T01:36:22.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.810+0000 7fb1240c7640 1 -- 172.21.15.115:0/2871553015 wait complete. 2023-12-01T01:36:22.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.810+0000 7f83c17fa640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f83a806a460 0x7f83a806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.810+0000 7f83c17fa640 1 -- 172.21.15.115:0/2925629800 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83bc089d60 con 0x7f83c415d0b0 2023-12-01T01:36:22.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.811+0000 7f83c8927640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f83a806a460 0x7f83a806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.809+0000 7f2ab2cb9640 1 -- 172.21.15.115:0/1890360544 shutdown_connections 2023-12-01T01:36:22.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.810+0000 7f9ad0a65640 1 -- 172.21.15.115:0/4223556753 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aac005250 con 0x7f9acc15ccc0 2023-12-01T01:36:22.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.811+0000 7f2ab2cb9640 1 -- 172.21.15.115:0/1890360544 wait complete. 2023-12-01T01:36:22.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.812+0000 7fc6d27fc640 1 -- 172.21.15.115:0/3680878385 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc6cc0540f0 con 0x7fc6dc06b2a0 2023-12-01T01:36:22.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.813+0000 7fb1240c7640 1 Processor -- start 2023-12-01T01:36:22.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.813+0000 7fb1240c7640 1 -- start start 2023-12-01T01:36:22.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.813+0000 7f2ab2cb9640 1 Processor -- start 2023-12-01T01:36:22.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.813+0000 7f9ab3fff640 1 -- 172.21.15.115:0/4223556753 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9ac4012400 con 0x7f9acc15ccc0 2023-12-01T01:36:22.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.814+0000 7f9ab3fff640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ab806a490 0x7f9ab806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.814+0000 7f9ab3fff640 1 -- 172.21.15.115:0/4223556753 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ac4088280 con 0x7f9acc15ccc0 2023-12-01T01:36:22.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.815+0000 7f83c8927640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f83a806a460 0x7f83a806c920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f83b8035000 tx=0x7f83b8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.814+0000 7f2ab2cb9640 1 -- start start 2023-12-01T01:36:22.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.815+0000 7fb1240c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c004b60 0x7fb11c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.815+0000 7f2ab2cb9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aac0b6090 0x7f2aac0d3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.815+0000 7f2ab2cb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0cf740 0x7f2aac14aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.815+0000 7f2ab2cb9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac160370 0x7f2aac0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.815+0000 7f2ab2cb9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0cc300 con 0x7f2aac160370 2023-12-01T01:36:22.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.815+0000 7f2ab2cb9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0cc790 con 0x7f2aac0b6090 2023-12-01T01:36:22.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.815+0000 7f2ab2cb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0cb6f0 con 0x7f2aac0cf740 2023-12-01T01:36:22.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.815+0000 7fb1240c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c0c8ee0 0x7fb11c1edf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.816+0000 7fb121e63640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c004b60 0x7fb11c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.816+0000 7f2ab1256640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac160370 0x7f2aac0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.816+0000 7fb121e63640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c004b60 0x7fb11c0b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56020/0 (socket says 172.21.15.115:56020) 2023-12-01T01:36:22.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.816+0000 7f2ab1256640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac160370 0x7f2aac0c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56026/0 (socket says 172.21.15.115:56026) 2023-12-01T01:36:22.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.816+0000 7f2ab1256640 1 -- 172.21.15.115:0/885134756 learned_addr learned my addr 172.21.15.115:0/885134756 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.816+0000 7f2aabfff640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0cf740 0x7f2aac14aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.816+0000 7f2ab1256640 1 -- 172.21.15.115:0/885134756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0cf740 msgr2=0x7f2aac14aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.816+0000 7f2ab1256640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0cf740 0x7f2aac14aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7f2ab1256640 1 -- 172.21.15.115:0/885134756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aac0b6090 msgr2=0x7f2aac0d3080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:36:22.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7f2ab1256640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aac0b6090 0x7f2aac0d3080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.816+0000 7fb121e63640 1 -- 172.21.15.115:0/1643070343 learned_addr learned my addr 172.21.15.115:0/1643070343 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:22.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7f2ab1256640 1 -- 172.21.15.115:0/885134756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a98009400 con 0x7f2aac160370 2023-12-01T01:36:22.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7fb1240c7640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb11c0c9340 0x7fb11c15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7f2ab1256640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac160370 0x7f2aac0c8a90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4001eb0 tx=0x7f2aa400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7f9acb7fe640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ab806a490 0x7f9ab806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7fb121662640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c0c8ee0 0x7fb11c1edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c15e910 con 0x7fb10c004b60 2023-12-01T01:36:22.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c15ebf0 con 0x7fb11c0c8ee0 2023-12-01T01:36:22.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c15eed0 con 0x7fb11c0c9340 2023-12-01T01:36:22.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.817+0000 7f2aa97fa640 1 -- 172.21.15.115:0/885134756 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa40052d0 con 0x7f2aac160370 2023-12-01T01:36:22.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.818+0000 7f2ab2cb9640 1 -- 172.21.15.115:0/885134756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac0cb9d0 con 0x7f2aac160370 2023-12-01T01:36:22.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.818+0000 7fb122664640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb11c0c9340 0x7fb11c15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.818+0000 7f2ab2cb9640 1 -- 172.21.15.115:0/885134756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac0cbf90 con 0x7f2aac160370 2023-12-01T01:36:22.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.818+0000 7f2aa97fa640 1 -- 172.21.15.115:0/885134756 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa4005430 con 0x7f2aac160370 2023-12-01T01:36:22.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.818+0000 7f2aa97fa640 1 -- 172.21.15.115:0/885134756 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4003810 con 0x7f2aac160370 2023-12-01T01:36:22.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.818+0000 7f83c17fa640 1 -- 172.21.15.115:0/2925629800 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f83bc08d060 con 0x7f83c415d0b0 2023-12-01T01:36:22.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.819+0000 7fb121e63640 1 -- 172.21.15.115:0/1643070343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb11c0c9340 msgr2=0x7fb11c15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.819+0000 7fb121e63640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb11c0c9340 0x7fb11c15f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.819+0000 7fb121e63640 1 -- 172.21.15.115:0/1643070343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c0c8ee0 msgr2=0x7fb11c1edf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:22.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.819+0000 7fb121e63640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c0c8ee0 0x7fb11c1edf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:22.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.819+0000 7fb121e63640 1 -- 172.21.15.115:0/1643070343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c15f1b0 con 0x7fb10c004b60 2023-12-01T01:36:22.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.819+0000 7f9acb7fe640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ab806a490 0x7f9ab806c950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9abc036000 tx=0x7f9abc03c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.819+0000 7fb121662640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c0c8ee0 0x7fb11c1edf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:22.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.820+0000 7fb121e63640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c004b60 0x7fb11c0b1ac0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb11000a880 tx=0x7fb110001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.820+0000 7fb11a7fc640 1 -- 172.21.15.115:0/1643070343 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110007390 con 0x7fb10c004b60 2023-12-01T01:36:22.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.820+0000 7fb11a7fc640 1 -- 172.21.15.115:0/1643070343 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1100074f0 con 0x7fb10c004b60 2023-12-01T01:36:22.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.820+0000 7fb11a7fc640 1 -- 172.21.15.115:0/1643070343 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb110007860 con 0x7fb10c004b60 2023-12-01T01:36:22.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.820+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c15f490 con 0x7fb10c004b60 2023-12-01T01:36:22.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.820+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c0b21e0 con 0x7fb10c004b60 2023-12-01T01:36:22.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.822+0000 7f2aa97fa640 1 -- 172.21.15.115:0/885134756 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2aa4003970 con 0x7f2aac160370 2023-12-01T01:36:22.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.823+0000 7f2a96ffd640 1 -- 172.21.15.115:0/885134756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac04e890 con 0x7f2aac160370 2023-12-01T01:36:22.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.824+0000 7f2aa97fa640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a9006a430 0x7f2a9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.824+0000 7f2aa97fa640 1 -- 172.21.15.115:0/885134756 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2aa4087e40 con 0x7f2aac160370 2023-12-01T01:36:22.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.825+0000 7f2ab0a55640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a9006a430 0x7f2a9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.823+0000 7fb11a7fc640 1 -- 172.21.15.115:0/1643070343 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb110031030 con 0x7fb10c004b60 2023-12-01T01:36:22.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.825+0000 7fb11a7fc640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb10006a4b0 0x7fb10006c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:22.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.825+0000 7fb11a7fc640 1 -- 172.21.15.115:0/1643070343 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1100aa500 con 0x7fb10c004b60 2023-12-01T01:36:22.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.826+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c0b21e0 con 0x7fb10c004b60 2023-12-01T01:36:22.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.826+0000 7f2ab0a55640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a9006a430 0x7f2a9006c8f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0001f60 tx=0x7f2aa0001550 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.826+0000 7fb121662640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb10006a4b0 0x7fb10006c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:22.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.827+0000 7fb121662640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb10006a4b0 0x7fb10006c970 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0c6cc0 tx=0x7fb108008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:22.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.828+0000 7f9ab3fff640 1 -- 172.21.15.115:0/4223556753 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9ac4053920 con 0x7f9acc15ccc0 2023-12-01T01:36:22.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.838+0000 7f2aa97fa640 1 -- 172.21.15.115:0/885134756 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2aa40534d0 con 0x7f2aac160370 2023-12-01T01:36:22.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:22.847+0000 7fb11a7fc640 1 -- 172.21.15.115:0/1643070343 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb11c0b21e0 con 0x7fb10c004b60 2023-12-01T01:36:23.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.168+0000 7f258f9bf640 1 -- 172.21.15.115:0/2803734494 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2580005790 con 0x7f25800e15c0 2023-12-01T01:36:23.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.169+0000 7f257dffb640 1 -- 172.21.15.115:0/2803734494 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2584017090 con 0x7f25800e15c0 2023-12-01T01:36:23.174 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:23.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.173+0000 7f258f9bf640 1 -- 172.21.15.115:0/2803734494 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f256c06a430 msgr2=0x7f256c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.173+0000 7f258f9bf640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f256c06a430 0x7f256c06c8f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2578036000 tx=0x7f2578008030 comp rx=0 tx=0).stop 2023-12-01T01:36:23.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.173+0000 7f258f9bf640 1 -- 172.21.15.115:0/2803734494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25800e15c0 msgr2=0x7f25800e3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.173+0000 7f258f9bf640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25800e15c0 0x7f25800e3840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f258400c430 tx=0x7f258400c830 comp rx=0 tx=0).stop 2023-12-01T01:36:23.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.174+0000 7f258f9bf640 1 -- 172.21.15.115:0/2803734494 shutdown_connections 2023-12-01T01:36:23.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.174+0000 7f258f9bf640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f256c06a430 0x7f256c06c8f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.174+0000 7f258f9bf640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f258003b7b0 0x7f258003bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.174+0000 7f258f9bf640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25800e15c0 0x7f25800e3840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.174+0000 7f258f9bf640 1 --2- 172.21.15.115:0/2803734494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25800e11e0 0x7f25800d77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.174+0000 7f258f9bf640 1 -- 172.21.15.115:0/2803734494 >> 172.21.15.115:0/2803734494 conn(0x7f2580053790 msgr2=0x7f2580053b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:23.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.175+0000 7f258f9bf640 1 -- 172.21.15.115:0/2803734494 shutdown_connections 2023-12-01T01:36:23.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.175+0000 7f258f9bf640 1 -- 172.21.15.115:0/2803734494 wait complete. 2023-12-01T01:36:23.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-12-01T01:36:23.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.206+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/2945418393 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3f940595a0 con 0x7f3f94057400 2023-12-01T01:36:23.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.207+0000 7f3f997fa640 1 -- 172.21.15.115:0/2945418393 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f9c097090 con 0x7f3f94057400 2023-12-01T01:36:23.208 INFO:teuthology.orchestra.run.smithi115.stdout:55834574854 2023-12-01T01:36:23.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.210+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3f8006a530 msgr2=0x7f3f8006c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.210+0000 7f3fa1a0e640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3f8006a530 0x7f3f8006c9f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f90007fd0 tx=0x7f3f90007d40 comp rx=0 tx=0).stop 2023-12-01T01:36:23.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.210+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f94057400 msgr2=0x7f3f94178840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.210+0000 7f3fa1a0e640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f94057400 0x7f3f94178840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0683c0 tx=0x7f3f9c067060 comp rx=0 tx=0).stop 2023-12-01T01:36:23.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.211+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/2945418393 shutdown_connections 2023-12-01T01:36:23.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.211+0000 7f3fa1a0e640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3f8006a530 0x7f3f8006c9f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.211+0000 7f3fa1a0e640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f940e34a0 0x7f3f9403dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.211+0000 7f3fa1a0e640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f940582c0 0x7f3f94174090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.211+0000 7f3fa1a0e640 1 --2- 172.21.15.115:0/2945418393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f94057400 0x7f3f94178840 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.211+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/2945418393 >> 172.21.15.115:0/2945418393 conn(0x7f3f94059120 msgr2=0x7f3f94039c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:23.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.211+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/2945418393 shutdown_connections 2023-12-01T01:36:23.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.211+0000 7f3fa1a0e640 1 -- 172.21.15.115:0/2945418393 wait complete. 2023-12-01T01:36:23.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.224+0000 7f83cab8b640 1 -- 172.21.15.115:0/2925629800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f83c40c5c60 con 0x7f83c415d0b0 2023-12-01T01:36:23.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.225+0000 7f83c17fa640 1 -- 172.21.15.115:0/2925629800 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f83bc053150 con 0x7f83c415d0b0 2023-12-01T01:36:23.226 INFO:teuthology.orchestra.run.smithi115.stdout:55834574856 2023-12-01T01:36:23.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f277b869640 1 -- 172.21.15.115:0/2935881692 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f27540057d0 con 0x7f277406d3d0 2023-12-01T01:36:23.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f83cab8b640 1 -- 172.21.15.115:0/2925629800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f83a806a460 msgr2=0x7f83a806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f83cab8b640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f83a806a460 0x7f83a806c920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f83b8035000 tx=0x7f83b8008030 comp rx=0 tx=0).stop 2023-12-01T01:36:23.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f83cab8b640 1 -- 172.21.15.115:0/2925629800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 msgr2=0x7f83c40cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f83cab8b640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 0x7f83c40cb590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00c7b0 tx=0x7f83bc00cc70 comp rx=0 tx=0).stop 2023-12-01T01:36:23.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f83cab8b640 1 -- 172.21.15.115:0/2925629800 shutdown_connections 2023-12-01T01:36:23.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f83cab8b640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f83a806a460 0x7f83a806c920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f83cab8b640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83c40d2620 0x7f83c41f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f83cab8b640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83c415d0b0 0x7f83c40cb590 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f83cab8b640 1 --2- 172.21.15.115:0/2925629800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c40d21c0 0x7f83c41561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.227+0000 7f83cab8b640 1 -- 172.21.15.115:0/2925629800 >> 172.21.15.115:0/2925629800 conn(0x7f83c406a0b0 msgr2=0x7f83c415c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:23.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.228+0000 7f83cab8b640 1 -- 172.21.15.115:0/2925629800 shutdown_connections 2023-12-01T01:36:23.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.228+0000 7f2771ffb640 1 -- 172.21.15.115:0/2935881692 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f276807a300 con 0x7f277406d3d0 2023-12-01T01:36:23.230 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:23.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.228+0000 7f83cab8b640 1 -- 172.21.15.115:0/2925629800 wait complete. 2023-12-01T01:36:23.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.231+0000 7f277b869640 1 -- 172.21.15.115:0/2935881692 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f275806a530 msgr2=0x7f275806c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.231+0000 7f277b869640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f275806a530 0x7f275806c9f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f277414f310 tx=0x7f2760009040 comp rx=0 tx=0).stop 2023-12-01T01:36:23.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.231+0000 7f277b869640 1 -- 172.21.15.115:0/2935881692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f277406d3d0 msgr2=0x7f277414b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.231+0000 7f277b869640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f277406d3d0 0x7f277414b520 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2768000f80 tx=0x7f2768004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:23.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.231+0000 7f2a96ffd640 1 -- 172.21.15.115:0/885134756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2aac0d0b30 con 0x7f2aac160370 2023-12-01T01:36:23.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.231+0000 7f277b869640 1 -- 172.21.15.115:0/2935881692 shutdown_connections 2023-12-01T01:36:23.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.231+0000 7f277b869640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f275806a530 0x7f275806c9f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.232+0000 7f277b869640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27740cb9d0 0x7f27740c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.232+0000 7f277b869640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27740c8e60 0x7f27740c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.232+0000 7f277b869640 1 --2- 172.21.15.115:0/2935881692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f277406d3d0 0x7f277414b520 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.232+0000 7f277b869640 1 -- 172.21.15.115:0/2935881692 >> 172.21.15.115:0/2935881692 conn(0x7f2774065b40 msgr2=0x7f277406b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:23.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.232+0000 7f2aa97fa640 1 -- 172.21.15.115:0/885134756 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2aa4017020 con 0x7f2aac160370 2023-12-01T01:36:23.234 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:23.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.232+0000 7f277b869640 1 -- 172.21.15.115:0/2935881692 shutdown_connections 2023-12-01T01:36:23.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.232+0000 7fc6e2907640 1 -- 172.21.15.115:0/3680878385 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc6c4005500 con 0x7fc6dc06b2a0 2023-12-01T01:36:23.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.232+0000 7f277b869640 1 -- 172.21.15.115:0/2935881692 wait complete. 2023-12-01T01:36:23.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.233+0000 7fc6d27fc640 1 -- 172.21.15.115:0/3680878385 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc6cc057710 con 0x7fc6dc06b2a0 2023-12-01T01:36:23.235 INFO:teuthology.orchestra.run.smithi115.stdout:55834574854 2023-12-01T01:36:23.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.235+0000 7f2a96ffd640 1 -- 172.21.15.115:0/885134756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a9006a430 msgr2=0x7f2a9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.235+0000 7f2a96ffd640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a9006a430 0x7f2a9006c8f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0001f60 tx=0x7f2aa0001550 comp rx=0 tx=0).stop 2023-12-01T01:36:23.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.235+0000 7f2a96ffd640 1 -- 172.21.15.115:0/885134756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac160370 msgr2=0x7f2aac0c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.235+0000 7f2a96ffd640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac160370 0x7f2aac0c8a90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4001eb0 tx=0x7f2aa400c890 comp rx=0 tx=0).stop 2023-12-01T01:36:23.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.235+0000 7f2a96ffd640 1 -- 172.21.15.115:0/885134756 shutdown_connections 2023-12-01T01:36:23.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.235+0000 7f2a96ffd640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a9006a430 0x7f2a9006c8f0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0001f60 tx=0x7f2aa0001550 comp rx=0 tx=0).stop 2023-12-01T01:36:23.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.235+0000 7f2a96ffd640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac0cf740 0x7f2aac14aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.235+0000 7f2a96ffd640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac160370 0x7f2aac0c8a90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.235+0000 7f2a96ffd640 1 --2- 172.21.15.115:0/885134756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aac0b6090 0x7f2aac0d3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.235+0000 7f2a96ffd640 1 -- 172.21.15.115:0/885134756 >> 172.21.15.115:0/885134756 conn(0x7f2aac065960 msgr2=0x7f2aac0d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:23.238 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-12-01T01:36:23.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7f2a96ffd640 1 -- 172.21.15.115:0/885134756 shutdown_connections 2023-12-01T01:36:23.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7f2a96ffd640 1 -- 172.21.15.115:0/885134756 wait complete. 2023-12-01T01:36:23.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7fc6e2907640 1 -- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c006a500 msgr2=0x7fc6c006c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7fc6e2907640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c006a500 0x7fc6c006c9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8036000 tx=0x7fc6d8030000 comp rx=0 tx=0).stop 2023-12-01T01:36:23.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7fc6e2907640 1 -- 172.21.15.115:0/3680878385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc06b2a0 msgr2=0x7fc6dc15c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7fc6e2907640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc06b2a0 0x7fc6dc15c8b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009fd0 tx=0x7fc6cc00ed40 comp rx=0 tx=0).stop 2023-12-01T01:36:23.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7fc6e2907640 1 -- 172.21.15.115:0/3680878385 shutdown_connections 2023-12-01T01:36:23.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7fc6e2907640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c006a500 0x7fc6c006c9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7fc6e2907640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6dc070710 0x7fc6dc0cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7fc6e2907640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6dc0b16a0 0x7fc6dc1f64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7fc6e2907640 1 --2- 172.21.15.115:0/3680878385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6dc06b2a0 0x7fc6dc15c8b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.237+0000 7fc6e2907640 1 -- 172.21.15.115:0/3680878385 >> 172.21.15.115:0/3680878385 conn(0x7fc6dc065960 msgr2=0x7fc6dc161900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:23.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.238+0000 7fc6e2907640 1 -- 172.21.15.115:0/3680878385 shutdown_connections 2023-12-01T01:36:23.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.238+0000 7fc6e2907640 1 -- 172.21.15.115:0/3680878385 wait complete. 2023-12-01T01:36:23.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.251+0000 7f9ad0a65640 1 -- 172.21.15.115:0/4223556753 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9aac0050c0 con 0x7f9acc15ccc0 2023-12-01T01:36:23.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.252+0000 7f9ab3fff640 1 -- 172.21.15.115:0/4223556753 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ac4056f40 con 0x7f9acc15ccc0 2023-12-01T01:36:23.254 INFO:teuthology.orchestra.run.smithi115.stdout:55834574853 2023-12-01T01:36:23.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2023-12-01T01:36:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 -- 172.21.15.115:0/4223556753 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ab806a490 msgr2=0x7f9ab806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ab806a490 0x7f9ab806c950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9abc036000 tx=0x7f9abc03c040 comp rx=0 tx=0).stop 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 -- 172.21.15.115:0/4223556753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc15ccc0 msgr2=0x7f9acc0b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc15ccc0 0x7f9acc0b33b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400cb40 tx=0x7f9ac400cf40 comp rx=0 tx=0).stop 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 -- 172.21.15.115:0/4223556753 shutdown_connections 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ab806a490 0x7f9ab806c950 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9acc15ccc0 0x7f9acc0b33b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9acc0cf430 0x7f9acc0b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 --2- 172.21.15.115:0/4223556753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9acc0b8420 0x7f9acc1f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 -- 172.21.15.115:0/4223556753 >> 172.21.15.115:0/4223556753 conn(0x7f9acc065960 msgr2=0x7f9acc0b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 -- 172.21.15.115:0/4223556753 shutdown_connections 2023-12-01T01:36:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.256+0000 7f9ad0a65640 1 -- 172.21.15.115:0/4223556753 wait complete. 2023-12-01T01:36:23.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2023-12-01T01:36:23.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-12-01T01:36:23.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.261+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb11c0b21e0 con 0x7fb10c004b60 2023-12-01T01:36:23.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.262+0000 7fb11a7fc640 1 -- 172.21.15.115:0/1643070343 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb11c0b21e0 con 0x7fb10c004b60 2023-12-01T01:36:23.263 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:23.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-12-01T01:36:23.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb10006a4b0 msgr2=0x7fb10006c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb10006a4b0 0x7fb10006c970 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0c6cc0 tx=0x7fb108008040 comp rx=0 tx=0).stop 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c004b60 msgr2=0x7fb11c0b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c004b60 0x7fb11c0b1ac0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb11000a880 tx=0x7fb110001d00 comp rx=0 tx=0).stop 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 shutdown_connections 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb10006a4b0 0x7fb10006c970 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb11c0c9340 0x7fb11c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb11c0c8ee0 0x7fb11c1edf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 --2- 172.21.15.115:0/1643070343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10c004b60 0x7fb11c0b1ac0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 >> 172.21.15.115:0/1643070343 conn(0x7fb11c06a0b0 msgr2=0x7fb11c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 shutdown_connections 2023-12-01T01:36:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:23.264+0000 7fb1240c7640 1 -- 172.21.15.115:0/1643070343 wait complete. 2023-12-01T01:36:23.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-12-01T01:36:23.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2023-12-01T01:36:24.088 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:36:24.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:36:24.204 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-01T01:36:24.238 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-01T01:36:24.255 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-01T01:36:24.260 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-01T01:36:24.262 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-01T01:36:24.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-01T01:36:24.276 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-01T01:36:24.282 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-01T01:36:24.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.465+0000 7f683d2ab640 1 Processor -- start 2023-12-01T01:36:24.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.465+0000 7f683d2ab640 1 -- start start 2023-12-01T01:36:24.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.467+0000 7f683d2ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68380ca520 0x7f68380ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.468+0000 7f683d2ab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f68380af4f0 0x7f68380af980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.468+0000 7f683d2ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683806c9a0 con 0x7f683816df40 2023-12-01T01:36:24.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.468+0000 7f683d2ab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68380cee50 con 0x7f68380af4f0 2023-12-01T01:36:24.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.468+0000 7f683d2ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380ce240 con 0x7f68380ca520 2023-12-01T01:36:24.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.468+0000 7f6836ffd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f683816df40 0x7f683808a790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:56898/0 (socket says 172.21.15.115:56898) 2023-12-01T01:36:24.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.468+0000 7f6836ffd640 1 -- 172.21.15.115:0/4288962197 learned_addr learned my addr 172.21.15.115:0/4288962197 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:24.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.468+0000 7f68367fc640 1 --2- 172.21.15.115:0/4288962197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68380ca520 0x7f68380ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.468+0000 7f68377fe640 1 --2- 172.21.15.115:0/4288962197 >> v2:172.21.15.142:3300/0 conn(0x7f68380af4f0 0x7f68380af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.469+0000 7f68367fc640 1 -- 172.21.15.115:0/4288962197 >> v2:172.21.15.142:3300/0 conn(0x7f68380af4f0 msgr2=0x7f68380af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:24.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.469+0000 7f68367fc640 1 --2- 172.21.15.115:0/4288962197 >> v2:172.21.15.142:3300/0 conn(0x7f68380af4f0 0x7f68380af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.469+0000 7f6835ffb640 1 -- 172.21.15.115:0/4288962197 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29759480 0 0) 0x7f6828001110 con 0x7f683816df40 2023-12-01T01:36:24.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.469+0000 7f68367fc640 1 -- 172.21.15.115:0/4288962197 >> v1:172.21.15.142:6789/0 conn(0x7f683816df40 legacy=0x7f683808a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.469+0000 7f68367fc640 1 -- 172.21.15.115:0/4288962197 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68380ce520 con 0x7f68380ca520 2023-12-01T01:36:24.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.471+0000 7f68367fc640 1 --2- 172.21.15.115:0/4288962197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68380ca520 0x7f68380ca980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f682c0097a0 tx=0x7f682c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894068f1225400fe server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.472+0000 7f6835ffb640 1 -- 172.21.15.115:0/4288962197 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c031640 con 0x7f68380ca520 2023-12-01T01:36:24.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.472+0000 7f6835ffb640 1 -- 172.21.15.115:0/4288962197 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c0317a0 con 0x7f68380ca520 2023-12-01T01:36:24.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.473+0000 7f6835ffb640 1 -- 172.21.15.115:0/4288962197 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c031b30 con 0x7f68380ca520 2023-12-01T01:36:24.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.474+0000 7f683d2ab640 1 -- 172.21.15.115:0/4288962197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68380ca520 msgr2=0x7f68380ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.474+0000 7f683d2ab640 1 --2- 172.21.15.115:0/4288962197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68380ca520 0x7f68380ca980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f682c0097a0 tx=0x7f682c02b8f0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.474+0000 7f683d2ab640 1 -- 172.21.15.115:0/4288962197 shutdown_connections 2023-12-01T01:36:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.475+0000 7f683d2ab640 1 --2- 172.21.15.115:0/4288962197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68380ca520 0x7f68380ca980 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.475+0000 7f683d2ab640 1 --2- 172.21.15.115:0/4288962197 >> v2:172.21.15.142:3300/0 conn(0x7f68380af4f0 0x7f68380af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.475+0000 7f683d2ab640 1 -- 172.21.15.115:0/4288962197 >> 172.21.15.115:0/4288962197 conn(0x7f6838065960 msgr2=0x7f6838065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.475+0000 7f683d2ab640 1 -- 172.21.15.115:0/4288962197 shutdown_connections 2023-12-01T01:36:24.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.475+0000 7f683d2ab640 1 -- 172.21.15.115:0/4288962197 wait complete. 2023-12-01T01:36:24.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.479+0000 7f683d2ab640 1 Processor -- start 2023-12-01T01:36:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.480+0000 7f683d2ab640 1 -- start start 2023-12-01T01:36:24.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.482+0000 7f683d2ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68380af4f0 0x7f683815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.483+0000 7f683d2ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68380ca520 0x7f68380c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.483+0000 7f683d2ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f683816df40 0x7f68380b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.483+0000 7f683d2ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c69c0 con 0x7f68380af4f0 2023-12-01T01:36:24.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.483+0000 7f683d2ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c7530 con 0x7f68380ca520 2023-12-01T01:36:24.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.483+0000 7f683d2ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c6c00 con 0x7f683816df40 2023-12-01T01:36:24.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.484+0000 7f6836ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68380af4f0 0x7f683815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.484+0000 7f6836ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68380af4f0 0x7f683815b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56036/0 (socket says 172.21.15.115:56036) 2023-12-01T01:36:24.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.484+0000 7f6836ffd640 1 -- 172.21.15.115:0/1030119147 learned_addr learned my addr 172.21.15.115:0/1030119147 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.485+0000 7f68367fc640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68380ca520 0x7f68380c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.485+0000 7f68377fe640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f683816df40 0x7f68380b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.486+0000 7f6836ffd640 1 -- 172.21.15.115:0/1030119147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f683816df40 msgr2=0x7f68380b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.486+0000 7f6836ffd640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f683816df40 0x7f68380b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.486+0000 7f6836ffd640 1 -- 172.21.15.115:0/1030119147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68380ca520 msgr2=0x7f68380c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.486+0000 7f6836ffd640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68380ca520 0x7f68380c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.486+0000 7f6836ffd640 1 -- 172.21.15.115:0/1030119147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c009400 con 0x7f68380af4f0 2023-12-01T01:36:24.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.487+0000 7f6836ffd640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68380af4f0 0x7f683815b180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f682800c430 tx=0x7f682800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.488+0000 7f681f7fe640 1 -- 172.21.15.115:0/1030119147 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280074e0 con 0x7f68380af4f0 2023-12-01T01:36:24.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.488+0000 7f681f7fe640 1 -- 172.21.15.115:0/1030119147 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6828007660 con 0x7f68380af4f0 2023-12-01T01:36:24.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.488+0000 7f681f7fe640 1 -- 172.21.15.115:0/1030119147 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828004a20 con 0x7f68380af4f0 2023-12-01T01:36:24.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.489+0000 7f683d2ab640 1 -- 172.21.15.115:0/1030119147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68380c6ee0 con 0x7f68380af4f0 2023-12-01T01:36:24.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.489+0000 7f683d2ab640 1 -- 172.21.15.115:0/1030119147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683815b6c0 con 0x7f68380af4f0 2023-12-01T01:36:24.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.491+0000 7f683d2ab640 1 -- 172.21.15.115:0/1030119147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683804e890 con 0x7f68380af4f0 2023-12-01T01:36:24.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.500+0000 7f681f7fe640 1 -- 172.21.15.115:0/1030119147 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6828004b80 con 0x7f68380af4f0 2023-12-01T01:36:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.505+0000 7f681f7fe640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a430 0x7f682406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.507+0000 7f68367fc640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a430 0x7f682406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.507+0000 7f681f7fe640 1 -- 172.21.15.115:0/1030119147 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68280884c0 con 0x7f68380af4f0 2023-12-01T01:36:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.508+0000 7f681f7fe640 1 -- 172.21.15.115:0/1030119147 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f68280887b0 con 0x7f68380af4f0 2023-12-01T01:36:24.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.509+0000 7f68367fc640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a430 0x7f682406c8f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f682c036000 tx=0x7f682c03d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.535+0000 7fdd7be2c640 1 Processor -- start 2023-12-01T01:36:24.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.535+0000 7fdd7be2c640 1 -- start start 2023-12-01T01:36:24.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.537+0000 7fdd7be2c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd6c0de3d0 0x7fdd6c006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.537+0000 7fdd7be2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c0066a0 0x7fdd6c0380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.538+0000 7fdd79bc8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd6c0de3d0 0x7fdd6c006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.538+0000 7fdd7be2c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6c057020 con 0x7fdd6c038620 2023-12-01T01:36:24.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.538+0000 7fdd7be2c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c058e90 con 0x7fdd6c0de3d0 2023-12-01T01:36:24.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.538+0000 7fdd7be2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c056cf0 con 0x7fdd6c0066a0 2023-12-01T01:36:24.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.538+0000 7fdd79bc8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd6c0de3d0 0x7fdd6c006160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56046/0 (socket says 172.21.15.115:56046) 2023-12-01T01:36:24.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.538+0000 7fdd79bc8640 1 -- 172.21.15.115:0/1617605137 learned_addr learned my addr 172.21.15.115:0/1617605137 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.539+0000 7fdd793c7640 1 --2- 172.21.15.115:0/1617605137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c0066a0 0x7fdd6c0380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.539+0000 7fdd79bc8640 1 -- 172.21.15.115:0/1617605137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c0066a0 msgr2=0x7fdd6c0380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:24.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.539+0000 7fdd79bc8640 1 --2- 172.21.15.115:0/1617605137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c0066a0 0x7fdd6c0380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.539+0000 7fdd79bc8640 1 -- 172.21.15.115:0/1617605137 >> v1:172.21.15.142:6789/0 conn(0x7fdd6c038620 legacy=0x7fdd6c0eb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.539+0000 7fdd79bc8640 1 -- 172.21.15.115:0/1617605137 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0588d0 con 0x7fdd6c0de3d0 2023-12-01T01:36:24.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.541+0000 7fdd79bc8640 1 --2- 172.21.15.115:0/1617605137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd6c0de3d0 0x7fdd6c006160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009870 tx=0x7fdd6402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2bb956427cf342 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.542+0000 7f2c36a12640 1 Processor -- start 2023-12-01T01:36:24.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.542+0000 7fdd78bc6640 1 -- 172.21.15.115:0/1617605137 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64030860 con 0x7fdd6c0de3d0 2023-12-01T01:36:24.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.542+0000 7f2c36a12640 1 -- start start 2023-12-01T01:36:24.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.544+0000 7fdd78bc6640 1 -- 172.21.15.115:0/1617605137 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd640309c0 con 0x7fdd6c0de3d0 2023-12-01T01:36:24.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.544+0000 7fdd78bc6640 1 -- 172.21.15.115:0/1617605137 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64030d90 con 0x7fdd6c0de3d0 2023-12-01T01:36:24.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.545+0000 7fdd7be2c640 1 -- 172.21.15.115:0/1617605137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd6c0de3d0 msgr2=0x7fdd6c006160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.545+0000 7fdd7be2c640 1 --2- 172.21.15.115:0/1617605137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd6c0de3d0 0x7fdd6c006160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009870 tx=0x7fdd6402b9a0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.545+0000 7fdd7be2c640 1 -- 172.21.15.115:0/1617605137 shutdown_connections 2023-12-01T01:36:24.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.545+0000 7fdd7be2c640 1 --2- 172.21.15.115:0/1617605137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c0066a0 0x7fdd6c0380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.545+0000 7fdd7be2c640 1 --2- 172.21.15.115:0/1617605137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd6c0de3d0 0x7fdd6c006160 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.545+0000 7fdd7be2c640 1 -- 172.21.15.115:0/1617605137 >> 172.21.15.115:0/1617605137 conn(0x7fdd6c055a90 msgr2=0x7fdd6c055e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:24.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.544+0000 7f2c36a12640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2c300c9160 0x7f2c300b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.545+0000 7f2c36a12640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2c300b4220 0x7f2c300ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.545+0000 7f2c36a12640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c300c96d0 con 0x7f2c3015abc0 2023-12-01T01:36:24.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.545+0000 7f2c36a12640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c300ca350 con 0x7f2c300c9160 2023-12-01T01:36:24.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.545+0000 7f2c36a12640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3015afa0 con 0x7f2c300b4220 2023-12-01T01:36:24.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.546+0000 7f2c34faf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2c300b4220 0x7f2c300ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.546+0000 7fdd7be2c640 1 -- 172.21.15.115:0/1617605137 shutdown_connections 2023-12-01T01:36:24.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.546+0000 7fc4606af640 1 Processor -- start 2023-12-01T01:36:24.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.546+0000 7fdd7be2c640 1 -- 172.21.15.115:0/1617605137 wait complete. 2023-12-01T01:36:24.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.547+0000 7fc4606af640 1 -- start start 2023-12-01T01:36:24.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.546+0000 7f2c34faf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2c300b4220 0x7f2c300ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53430/0 (socket says 172.21.15.115:53430) 2023-12-01T01:36:24.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.546+0000 7f2c34faf640 1 -- 172.21.15.115:0/2896534431 learned_addr learned my addr 172.21.15.115:0/2896534431 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.548+0000 7f2c2f7fe640 1 --2- 172.21.15.115:0/2896534431 >> v2:172.21.15.115:3300/0 conn(0x7f2c300c9160 0x7f2c300b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.549+0000 7fc4606af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580cf160 0x7fc4580c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.549+0000 7fc4606af640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc4580b0050 0x7fc45815d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.549+0000 7fc4606af640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4580c9070 con 0x7fc4580afc70 2023-12-01T01:36:24.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.549+0000 7fc4606af640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4580c8e60 con 0x7fc4580b0050 2023-12-01T01:36:24.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.549+0000 7fc4606af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580c9350 con 0x7fc4580cf160 2023-12-01T01:36:24.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.549+0000 7fc45e44b640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fc4580afc70 0x7fc4580cea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:56920/0 (socket says 172.21.15.115:56920) 2023-12-01T01:36:24.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.549+0000 7fc45e44b640 1 -- 172.21.15.115:0/2317166008 learned_addr learned my addr 172.21.15.115:0/2317166008 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:24.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.549+0000 7fc45dc4a640 1 --2- 172.21.15.115:0/2317166008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580cf160 0x7fc4580c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.549+0000 7fdd7be2c640 1 Processor -- start 2023-12-01T01:36:24.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.551+0000 7f2c2f7fe640 1 -- 172.21.15.115:0/2896534431 >> v2:172.21.15.142:3300/0 conn(0x7f2c300b4220 msgr2=0x7f2c300ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:24.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.551+0000 7f2c2f7fe640 1 --2- 172.21.15.115:0/2896534431 >> v2:172.21.15.142:3300/0 conn(0x7f2c300b4220 0x7f2c300ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.551+0000 7f2c2f7fe640 1 -- 172.21.15.115:0/2896534431 >> v1:172.21.15.115:6789/0 conn(0x7f2c3015abc0 legacy=0x7f2c300b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.551+0000 7f2c2f7fe640 1 -- 172.21.15.115:0/2896534431 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c300c99b0 con 0x7f2c300c9160 2023-12-01T01:36:24.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.551+0000 7fdd7be2c640 1 -- start start 2023-12-01T01:36:24.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.553+0000 7fdd7be2c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd6c0066a0 0x7fdd6c0db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.553+0000 7fdd7be2c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd6c038620 0x7fdd6c180c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.550+0000 7fc45d449640 1 -- 172.21.15.115:0/2317166008 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023669109 0 0) 0x7fc4580c9070 con 0x7fc4580afc70 2023-12-01T01:36:24.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.550+0000 7fc45d449640 1 -- 172.21.15.115:0/2317166008 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc444003520 con 0x7fc4580afc70 2023-12-01T01:36:24.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.550+0000 7fc45dc4a640 1 -- 172.21.15.115:0/2317166008 >> v2:172.21.15.142:3300/0 conn(0x7fc4580b0050 msgr2=0x7fc45815d7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-01T01:36:24.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.550+0000 7fc45dc4a640 1 --2- 172.21.15.115:0/2317166008 >> v2:172.21.15.142:3300/0 conn(0x7fc4580b0050 0x7fc45815d7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.550+0000 7fc45dc4a640 1 -- 172.21.15.115:0/2317166008 >> v1:172.21.15.142:6789/0 conn(0x7fc4580afc70 legacy=0x7fc4580cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.550+0000 7fc45dc4a640 1 -- 172.21.15.115:0/2317166008 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4580c9630 con 0x7fc4580cf160 2023-12-01T01:36:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.552+0000 7fc45dc4a640 1 --2- 172.21.15.115:0/2317166008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580cf160 0x7fc4580c9e30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc4480097a0 tx=0x7fc44802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7758f05b82b60f9d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.553+0000 7fc45d449640 1 -- 172.21.15.115:0/2317166008 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448031640 con 0x7fc4580cf160 2023-12-01T01:36:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.553+0000 7fc45d449640 1 -- 172.21.15.115:0/2317166008 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4480317a0 con 0x7fc4580cf160 2023-12-01T01:36:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.553+0000 7fc45d449640 1 -- 172.21.15.115:0/2317166008 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448031b70 con 0x7fc4580cf160 2023-12-01T01:36:24.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.553+0000 7fdd7be2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c180560 0x7fdd6c03a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.553+0000 7fdd7be2c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0e1640 con 0x7fdd6c0066a0 2023-12-01T01:36:24.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.553+0000 7fdd7be2c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0e09c0 con 0x7fdd6c038620 2023-12-01T01:36:24.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.553+0000 7fdd7be2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0e0ca0 con 0x7fdd6c180560 2023-12-01T01:36:24.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.554+0000 7fdd7a3c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c180560 0x7fdd6c03a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.553+0000 7f2c2f7fe640 1 --2- 172.21.15.115:0/2896534431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 0x7f2c300b3ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2c300ce530 tx=0x7f2c2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79adff1035d2f154 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.554+0000 7fdd7a3c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c180560 0x7fdd6c03a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:33984/0 (socket says 172.21.15.115:33984) 2023-12-01T01:36:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.554+0000 7fdd7a3c9640 1 -- 172.21.15.115:0/2466924760 learned_addr learned my addr 172.21.15.115:0/2466924760 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.554+0000 7f2c2effd640 1 -- 172.21.15.115:0/2896534431 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24031870 con 0x7f2c300c9160 2023-12-01T01:36:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.554+0000 7fdd793c7640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd6c038620 0x7fdd6c180c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.554+0000 7f2c2effd640 1 -- 172.21.15.115:0/2896534431 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c240319d0 con 0x7f2c300c9160 2023-12-01T01:36:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.554+0000 7f2c2effd640 1 -- 172.21.15.115:0/2896534431 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24031d80 con 0x7f2c300c9160 2023-12-01T01:36:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7fdd7a3c9640 1 -- 172.21.15.115:0/2466924760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd6c038620 msgr2=0x7fdd6c180c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7fdd7a3c9640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd6c038620 0x7fdd6c180c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7fdd7a3c9640 1 -- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd6c0066a0 msgr2=0x7fdd6c0db7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:36:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7fb5a74f8640 1 Processor -- start 2023-12-01T01:36:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7fdd79bc8640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd6c0066a0 0x7fdd6c0db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7fdd7a3c9640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd6c0066a0 0x7fdd6c0db7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7fdd7a3c9640 1 -- 172.21.15.115:0/2466924760 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640094d0 con 0x7fdd6c180560 2023-12-01T01:36:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7fdd793c7640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd6c038620 0x7fdd6c180c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7fdd79bc8640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd6c0066a0 0x7fdd6c0db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7f2c36a12640 1 -- 172.21.15.115:0/2896534431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 msgr2=0x7f2c300b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7f2c36a12640 1 --2- 172.21.15.115:0/2896534431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 0x7f2c300b3ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2c300ce530 tx=0x7f2c2402b910 comp rx=0 tx=0).stop 2023-12-01T01:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.555+0000 7fb5a74f8640 1 -- start start 2023-12-01T01:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7fdd7a3c9640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c180560 0x7fdd6c03a980 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c3c0 tx=0x7fdd7000c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7fdd6a7fc640 1 -- 172.21.15.115:0/2466924760 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd700052d0 con 0x7fdd6c180560 2023-12-01T01:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7fdd6a7fc640 1 -- 172.21.15.115:0/2466924760 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70005430 con 0x7fdd6c180560 2023-12-01T01:36:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7fdd6a7fc640 1 -- 172.21.15.115:0/2466924760 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70007260 con 0x7fdd6c180560 2023-12-01T01:36:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7fdd7be2c640 1 -- 172.21.15.115:0/2466924760 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c0e0f80 con 0x7fdd6c180560 2023-12-01T01:36:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.557+0000 7fdd7be2c640 1 -- 172.21.15.115:0/2466924760 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c0e1a60 con 0x7fdd6c180560 2023-12-01T01:36:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.557+0000 7fc4606af640 1 -- 172.21.15.115:0/2317166008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580cf160 msgr2=0x7fc4580c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.557+0000 7fc4606af640 1 --2- 172.21.15.115:0/2317166008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580cf160 0x7fc4580c9e30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc4480097a0 tx=0x7fc44802b8f0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.558+0000 7fc4606af640 1 -- 172.21.15.115:0/2317166008 shutdown_connections 2023-12-01T01:36:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.558+0000 7fc4606af640 1 --2- 172.21.15.115:0/2317166008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580cf160 0x7fc4580c9e30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.558+0000 7fc4606af640 1 --2- 172.21.15.115:0/2317166008 >> v2:172.21.15.142:3300/0 conn(0x7fc4580b0050 0x7fc45815d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.558+0000 7fc4606af640 1 -- 172.21.15.115:0/2317166008 >> 172.21.15.115:0/2317166008 conn(0x7fc458065b40 msgr2=0x7fc458065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7f2c36a12640 1 -- 172.21.15.115:0/2896534431 shutdown_connections 2023-12-01T01:36:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7f2c36a12640 1 --2- 172.21.15.115:0/2896534431 >> v2:172.21.15.142:3300/0 conn(0x7f2c300b4220 0x7f2c300ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7f2c36a12640 1 --2- 172.21.15.115:0/2896534431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 0x7f2c300b3ce0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7f2c36a12640 1 -- 172.21.15.115:0/2896534431 >> 172.21.15.115:0/2896534431 conn(0x7f2c30065d50 msgr2=0x7f2c30066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7f2c36a12640 1 -- 172.21.15.115:0/2896534431 shutdown_connections 2023-12-01T01:36:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.556+0000 7f2c36a12640 1 -- 172.21.15.115:0/2896534431 wait complete. 2023-12-01T01:36:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.559+0000 7f2c36a12640 1 Processor -- start 2023-12-01T01:36:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.559+0000 7f2c36a12640 1 -- start start 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.557+0000 7fb5a74f8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb5980e77a0 0x7fb59804f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.557+0000 7fb5a74f8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb5980eb1b0 0x7fb5980eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.557+0000 7fb5a74f8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb598052ed0 0x7fb598053360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.557+0000 7fb5a74f8640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb598050230 con 0x7fb5980eb1b0 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.557+0000 7fb5a74f8640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb598006370 con 0x7fb5980e77a0 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.557+0000 7fb5a74f8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5980ebb30 con 0x7fb598052ed0 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.559+0000 7fb5a5294640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb5980e77a0 0x7fb59804f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.559+0000 7fb5a5294640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb5980e77a0 0x7fb59804f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53454/0 (socket says 172.21.15.115:53454) 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.559+0000 7fb5a5294640 1 -- 172.21.15.115:0/3660113408 learned_addr learned my addr 172.21.15.115:0/3660113408 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.560+0000 7fb5a4a93640 1 --2- 172.21.15.115:0/3660113408 >> v2:172.21.15.115:3300/0 conn(0x7fb5980eb1b0 0x7fb5980eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.560+0000 7fb5a5294640 1 -- 172.21.15.115:0/3660113408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb598052ed0 msgr2=0x7fb598053360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.560+0000 7fb5a5294640 1 --2- 172.21.15.115:0/3660113408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb598052ed0 0x7fb598053360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.560+0000 7fb5a5294640 1 -- 172.21.15.115:0/3660113408 >> v2:172.21.15.115:3300/0 conn(0x7fb5980eb1b0 msgr2=0x7fb5980eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:24.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.560+0000 7fb5a5294640 1 --2- 172.21.15.115:0/3660113408 >> v2:172.21.15.115:3300/0 conn(0x7fb5980eb1b0 0x7fb5980eb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.560+0000 7fb5a5294640 1 -- 172.21.15.115:0/3660113408 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59804fc70 con 0x7fb5980e77a0 2023-12-01T01:36:24.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.560+0000 7fb5a4a93640 1 --2- 172.21.15.115:0/3660113408 >> v2:172.21.15.115:3300/0 conn(0x7fb5980eb1b0 0x7fb5980eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:24.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.558+0000 7fc4606af640 1 -- 172.21.15.115:0/2317166008 shutdown_connections 2023-12-01T01:36:24.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.559+0000 7fdd7be2c640 1 -- 172.21.15.115:0/2466924760 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c0e1780 con 0x7fdd6c180560 2023-12-01T01:36:24.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.558+0000 7fc4606af640 1 -- 172.21.15.115:0/2317166008 wait complete. 2023-12-01T01:36:24.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.561+0000 7f2c36a12640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c300b4220 0x7f2c3006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.561+0000 7f2c36a12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 0x7f2c30070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.562+0000 7fc4606af640 1 Processor -- start 2023-12-01T01:36:24.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.562+0000 7fc4606af640 1 -- start start 2023-12-01T01:36:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.562+0000 7fb5a5294640 1 --2- 172.21.15.115:0/3660113408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5980e77a0 0x7fb59804f150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb59800bc60 tx=0x7fb59c02b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f93d2ed85dd88d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.562+0000 7f2c36a12640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c3015abc0 0x7f2c300cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.562+0000 7f2c2ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c300b4220 0x7f2c3006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.562+0000 7f2c2ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c300b4220 0x7f2c3006ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:33990/0 (socket says 172.21.15.115:33990) 2023-12-01T01:36:24.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.562+0000 7fb597fff640 1 -- 172.21.15.115:0/3660113408 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0317f0 con 0x7fb5980e77a0 2023-12-01T01:36:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.562+0000 7f2c36a12640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3014f570 con 0x7f2c300c9160 2023-12-01T01:36:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.562+0000 7f2c36a12640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3015df30 con 0x7f2c3015abc0 2023-12-01T01:36:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.562+0000 7f2c36a12640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3015d690 con 0x7f2c300b4220 2023-12-01T01:36:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.563+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 0x7f2c30070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.563+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 0x7f2c30070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56094/0 (socket says 172.21.15.115:56094) 2023-12-01T01:36:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.563+0000 7f2c2f7fe640 1 -- 172.21.15.115:0/2066611466 learned_addr learned my addr 172.21.15.115:0/2066611466 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.563+0000 7f2c34faf640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c3015abc0 0x7f2c300cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.563+0000 7fb597fff640 1 -- 172.21.15.115:0/3660113408 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59c031950 con 0x7fb5980e77a0 2023-12-01T01:36:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.563+0000 7fb597fff640 1 -- 172.21.15.115:0/3660113408 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c031ce0 con 0x7fb5980e77a0 2023-12-01T01:36:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.564+0000 7f2c2f7fe640 1 -- 172.21.15.115:0/2066611466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c300b4220 msgr2=0x7f2c3006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.564+0000 7f2c2f7fe640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c300b4220 0x7f2c3006ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.564+0000 7fc4606af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580afc70 0x7fc4580b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.564+0000 7f2c2f7fe640 1 -- 172.21.15.115:0/2066611466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c3015abc0 msgr2=0x7f2c300cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.564+0000 7f2c2f7fe640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c3015abc0 0x7f2c300cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.564+0000 7fc4606af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4580b0050 0x7fc458151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.564+0000 7f2c2f7fe640 1 -- 172.21.15.115:0/2066611466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c24009400 con 0x7f2c300c9160 2023-12-01T01:36:24.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.564+0000 7fb5a74f8640 1 -- 172.21.15.115:0/3660113408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5980e77a0 msgr2=0x7fb59804f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.564+0000 7fc4606af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4580cf160 0x7fc4581503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fb5a74f8640 1 --2- 172.21.15.115:0/3660113408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5980e77a0 0x7fb59804f150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb59800bc60 tx=0x7fb59c02b890 comp rx=0 tx=0).stop 2023-12-01T01:36:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fc45e44b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580afc70 0x7fc4580b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fc45e44b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580afc70 0x7fc4580b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:33992/0 (socket says 172.21.15.115:33992) 2023-12-01T01:36:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.564+0000 7fdd6a7fc640 1 -- 172.21.15.115:0/2466924760 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdd700073c0 con 0x7fdd6c180560 2023-12-01T01:36:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fc45ec4c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4580cf160 0x7fc4581503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fc45e44b640 1 -- 172.21.15.115:0/1642785669 learned_addr learned my addr 172.21.15.115:0/1642785669 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc458151dc0 con 0x7fc4580cf160 2023-12-01T01:36:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fc45dc4a640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4580b0050 0x7fc458151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fb5a74f8640 1 -- 172.21.15.115:0/3660113408 shutdown_connections 2023-12-01T01:36:24.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fb5a74f8640 1 --2- 172.21.15.115:0/3660113408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb598052ed0 0x7fb598053360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fb5a74f8640 1 --2- 172.21.15.115:0/3660113408 >> v2:172.21.15.115:3300/0 conn(0x7fb5980eb1b0 0x7fb5980eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fb5a74f8640 1 --2- 172.21.15.115:0/3660113408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5980e77a0 0x7fb59804f150 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fb5a74f8640 1 -- 172.21.15.115:0/3660113408 >> 172.21.15.115:0/3660113408 conn(0x7fb598054b00 msgr2=0x7fb598054ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc458150920 con 0x7fc4580b0050 2023-12-01T01:36:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45815c1b0 con 0x7fc4580afc70 2023-12-01T01:36:24.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.565+0000 7f2c2f7fe640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 0x7f2c30070c10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009890 tx=0x7f2c240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.566+0000 7fb5a74f8640 1 -- 172.21.15.115:0/3660113408 shutdown_connections 2023-12-01T01:36:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.566+0000 7fdd6a7fc640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4b0 0x7fdd5806c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.566+0000 7f2c2cff9640 1 -- 172.21.15.115:0/2066611466 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24003810 con 0x7f2c300c9160 2023-12-01T01:36:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.566+0000 7fc45ec4c640 1 -- 172.21.15.115:0/1642785669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580afc70 msgr2=0x7fc4580b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.566+0000 7fc45ec4c640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580afc70 0x7fc4580b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.566+0000 7fc45ec4c640 1 -- 172.21.15.115:0/1642785669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4580b0050 msgr2=0x7fc458151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.566+0000 7fc45ec4c640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4580b0050 0x7fc458151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.566+0000 7fc45ec4c640 1 -- 172.21.15.115:0/1642785669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc448009400 con 0x7fc4580cf160 2023-12-01T01:36:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.567+0000 7fdd6a7fc640 1 -- 172.21.15.115:0/2466924760 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd70087bf0 con 0x7fdd6c180560 2023-12-01T01:36:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.567+0000 7f2c36a12640 1 -- 172.21.15.115:0/2066611466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3015d970 con 0x7f2c300c9160 2023-12-01T01:36:24.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.567+0000 7f2c36a12640 1 -- 172.21.15.115:0/2066611466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c30148fb0 con 0x7f2c300c9160 2023-12-01T01:36:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.567+0000 7f2c2cff9640 1 -- 172.21.15.115:0/2066611466 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c24003970 con 0x7f2c300c9160 2023-12-01T01:36:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.567+0000 7f2c2cff9640 1 -- 172.21.15.115:0/2066611466 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24003d00 con 0x7f2c300c9160 2023-12-01T01:36:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.567+0000 7fdd79bc8640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4b0 0x7fdd5806c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.566+0000 7fb5a74f8640 1 -- 172.21.15.115:0/3660113408 wait complete. 2023-12-01T01:36:24.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.569+0000 7fb5a74f8640 1 Processor -- start 2023-12-01T01:36:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.569+0000 7fdd79bc8640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4b0 0x7fdd5806c970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd64036000 tx=0x7fdd6403b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.569+0000 7fc45ec4c640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4580cf160 0x7fc4581503e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc45000bdf0 tx=0x7fc450009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.569+0000 7fb5a74f8640 1 -- start start 2023-12-01T01:36:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.569+0000 7fc44effd640 1 -- 172.21.15.115:0/1642785669 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000ce70 con 0x7fc4580cf160 2023-12-01T01:36:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.569+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45815c450 con 0x7fc4580cf160 2023-12-01T01:36:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.570+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45815ca10 con 0x7fc4580cf160 2023-12-01T01:36:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.570+0000 7fc44effd640 1 -- 172.21.15.115:0/1642785669 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc450014440 con 0x7fc4580cf160 2023-12-01T01:36:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.570+0000 7fc44effd640 1 -- 172.21.15.115:0/1642785669 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000ce70 con 0x7fc4580cf160 2023-12-01T01:36:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.570+0000 7f2c167fc640 1 -- 172.21.15.115:0/2066611466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c08005250 con 0x7f2c300c9160 2023-12-01T01:36:24.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.572+0000 7fb5a74f8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb598052ed0 0x7fb59817a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.572+0000 7f2c2cff9640 1 -- 172.21.15.115:0/2066611466 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2c2403a420 con 0x7f2c300c9160 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.572+0000 7fb5a74f8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5980e77a0 0x7fb59804abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.572+0000 7fb5a74f8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5980eb1b0 0x7fb5980e5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.573+0000 7fb5a5294640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb598052ed0 0x7fb59817a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.573+0000 7fb5a5294640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb598052ed0 0x7fb59817a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56100/0 (socket says 172.21.15.115:56100) 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.573+0000 7fb5a4a93640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5980e77a0 0x7fb59804abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.573+0000 7fb5a5294640 1 -- 172.21.15.115:0/289824778 learned_addr learned my addr 172.21.15.115:0/289824778 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.573+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59817ad50 con 0x7fb598052ed0 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.573+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59804b100 con 0x7fb5980eb1b0 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.573+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb598178a60 con 0x7fb5980e77a0 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.573+0000 7fb5a5a95640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5980eb1b0 0x7fb5980e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.574+0000 7fb5a5a95640 1 -- 172.21.15.115:0/289824778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5980e77a0 msgr2=0x7fb59804abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.574+0000 7fb5a5a95640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5980e77a0 0x7fb59804abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.574+0000 7fb5a5a95640 1 -- 172.21.15.115:0/289824778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb598052ed0 msgr2=0x7fb59817a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.574+0000 7f2c2cff9640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2c1c06a430 0x7f2c1c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.575+0000 7f2c2cff9640 1 -- 172.21.15.115:0/2066611466 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c2402d040 con 0x7f2c300c9160 2023-12-01T01:36:24.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.575+0000 7fb5a5a95640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb598052ed0 0x7fb59817a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.575+0000 7fb5a5a95640 1 -- 172.21.15.115:0/289824778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c0094d0 con 0x7fb5980eb1b0 2023-12-01T01:36:24.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.575+0000 7fb5a4a93640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5980e77a0 0x7fb59804abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:24.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.576+0000 7fb5a5294640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb598052ed0 0x7fb59817a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:24.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.575+0000 7fc44effd640 1 -- 172.21.15.115:0/1642785669 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc45001d3b0 con 0x7fc4580cf160 2023-12-01T01:36:24.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.576+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4580b3040 con 0x7fc4580cf160 2023-12-01T01:36:24.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.576+0000 7f2c2ffff640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2c1c06a430 0x7f2c1c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.577+0000 7fb5a5a95640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5980eb1b0 0x7fb5980e5ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0068bb0 tx=0x7fb5a0067eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.577+0000 7fb595ffb640 1 -- 172.21.15.115:0/289824778 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0067210 con 0x7fb5980eb1b0 2023-12-01T01:36:24.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.577+0000 7f2c2ffff640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2c1c06a430 0x7f2c1c06c8f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c300cab60 tx=0x7f2c20009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.577+0000 7fc44effd640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc43c06a490 0x7fc43c06c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.577+0000 7fc44effd640 1 -- 172.21.15.115:0/1642785669 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc450088700 con 0x7fc4580cf160 2023-12-01T01:36:24.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.577+0000 7fb595ffb640 1 -- 172.21.15.115:0/289824778 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a0072320 con 0x7fb5980eb1b0 2023-12-01T01:36:24.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.577+0000 7fb595ffb640 1 -- 172.21.15.115:0/289824778 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a006f870 con 0x7fb5980eb1b0 2023-12-01T01:36:24.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.578+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59804b330 con 0x7fb5980eb1b0 2023-12-01T01:36:24.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.578+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb598178f90 con 0x7fb5980eb1b0 2023-12-01T01:36:24.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.579+0000 7fc45e44b640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc43c06a490 0x7fc43c06c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.580+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59804e960 con 0x7fb5980eb1b0 2023-12-01T01:36:24.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.580+0000 7fc45e44b640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc43c06a490 0x7fc43c06c950 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4540011e0 tx=0x7fc454009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.580+0000 7fdd6a7fc640 1 -- 172.21.15.115:0/2466924760 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdd6c0e1780 con 0x7fdd6c180560 2023-12-01T01:36:24.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.582+0000 7fb595ffb640 1 -- 172.21.15.115:0/289824778 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb5a006f9d0 con 0x7fb5980eb1b0 2023-12-01T01:36:24.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.584+0000 7fb595ffb640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb58406a500 0x7fb58406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.584+0000 7fb595ffb640 1 -- 172.21.15.115:0/289824778 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5a0101860 con 0x7fb5980eb1b0 2023-12-01T01:36:24.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.584+0000 7fb5a5294640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb58406a500 0x7fb58406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.585+0000 7f2c2cff9640 1 -- 172.21.15.115:0/2066611466 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2c24076530 con 0x7f2c300c9160 2023-12-01T01:36:24.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.585+0000 7fb5a5294640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb58406a500 0x7fb58406c9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb59c034040 tx=0x7fb59c03d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.592+0000 7fc44effd640 1 -- 172.21.15.115:0/1642785669 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc4580b3040 con 0x7fc4580cf160 2023-12-01T01:36:24.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.596+0000 7fb595ffb640 1 -- 172.21.15.115:0/289824778 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb5a00cce30 con 0x7fb5980eb1b0 2023-12-01T01:36:24.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.612+0000 7f0dabb4e640 1 Processor -- start 2023-12-01T01:36:24.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.613+0000 7f8581669640 1 Processor -- start 2023-12-01T01:36:24.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.613+0000 7f8581669640 1 -- start start 2023-12-01T01:36:24.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.613+0000 7f0dabb4e640 1 -- start start 2023-12-01T01:36:24.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.615+0000 7f0dabb4e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0da40d3280 0x7f0da40b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.615+0000 7f0dabb4e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da40d1120 con 0x7f0da415d0b0 2023-12-01T01:36:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.615+0000 7f0dabb4e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da40d1400 con 0x7f0da40d21c0 2023-12-01T01:36:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.615+0000 7f0dabb4e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da40b2330 con 0x7f0da40d3280 2023-12-01T01:36:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.615+0000 7f0da98ea640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0da415d0b0 0x7f0da40b8500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51444/0 (socket says 172.21.15.115:51444) 2023-12-01T01:36:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.615+0000 7f0da98ea640 1 -- 172.21.15.115:0/3128745341 learned_addr learned my addr 172.21.15.115:0/3128745341 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:24.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.616+0000 7f0daa0eb640 1 --2- 172.21.15.115:0/3128745341 >> v2:172.21.15.115:3300/0 conn(0x7f0da40d3280 0x7f0da40b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.617+0000 7f8581669640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f857c0b6090 0x7f857c0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.617+0000 7f0da88e8640 1 -- 172.21.15.115:0/3128745341 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890513779 0 0) 0x7f0da40d1400 con 0x7f0da40d21c0 2023-12-01T01:36:24.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.617+0000 7f8581669640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f857c0cf740 0x7f857c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.617+0000 7f8581669640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c0d00f0 con 0x7f857c160370 2023-12-01T01:36:24.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.618+0000 7f8581669640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857c0ce660 con 0x7f857c0b6090 2023-12-01T01:36:24.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.618+0000 7f8581669640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857c0d2c10 con 0x7f857c0cf740 2023-12-01T01:36:24.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.618+0000 7f857b7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f857c0cf740 0x7f857c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.618+0000 7f857a7fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f857c0b6090 0x7f857c0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.618+0000 7f857b7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f857c0cf740 0x7f857c0cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53488/0 (socket says 172.21.15.115:53488) 2023-12-01T01:36:24.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.618+0000 7f857b7fe640 1 -- 172.21.15.115:0/3296010460 learned_addr learned my addr 172.21.15.115:0/3296010460 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.617+0000 7f0daa0eb640 1 -- 172.21.15.115:0/3128745341 >> v1:172.21.15.142:6789/0 conn(0x7f0da40d21c0 legacy=0x7f0da40d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.617+0000 7f0daa0eb640 1 -- 172.21.15.115:0/3128745341 >> v1:172.21.15.115:6789/0 conn(0x7f0da415d0b0 legacy=0x7f0da40b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.617+0000 7f0daa0eb640 1 -- 172.21.15.115:0/3128745341 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da40d16e0 con 0x7f0da40d3280 2023-12-01T01:36:24.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.619+0000 7f0daa0eb640 1 --2- 172.21.15.115:0/3128745341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d3280 0x7f0da40b5d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0da00097a0 tx=0x7f0da002ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10f62336db634bb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.619+0000 7f857a7fc640 1 -- 172.21.15.115:0/3296010460 >> v2:172.21.15.142:3300/0 conn(0x7f857c0cf740 msgr2=0x7f857c0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:24.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.619+0000 7f857a7fc640 1 --2- 172.21.15.115:0/3296010460 >> v2:172.21.15.142:3300/0 conn(0x7f857c0cf740 0x7f857c0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.619+0000 7f857a7fc640 1 -- 172.21.15.115:0/3296010460 >> v1:172.21.15.142:6789/0 conn(0x7f857c160370 legacy=0x7f857c1613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:24.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.619+0000 7f857a7fc640 1 -- 172.21.15.115:0/3296010460 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c160750 con 0x7f857c0b6090 2023-12-01T01:36:24.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.620+0000 7f0da88e8640 1 -- 172.21.15.115:0/3128745341 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da002f830 con 0x7f0da40d3280 2023-12-01T01:36:24.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.620+0000 7f0da88e8640 1 -- 172.21.15.115:0/3128745341 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da002f990 con 0x7f0da40d3280 2023-12-01T01:36:24.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.621+0000 7f0da88e8640 1 -- 172.21.15.115:0/3128745341 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da002f830 con 0x7f0da40d3280 2023-12-01T01:36:24.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.621+0000 7f0dabb4e640 1 -- 172.21.15.115:0/3128745341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d3280 msgr2=0x7f0da40b5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.621+0000 7f0dabb4e640 1 --2- 172.21.15.115:0/3128745341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d3280 0x7f0da40b5d10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0da00097a0 tx=0x7f0da002ba10 comp rx=0 tx=0).stop 2023-12-01T01:36:24.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.621+0000 7f0dabb4e640 1 -- 172.21.15.115:0/3128745341 shutdown_connections 2023-12-01T01:36:24.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.622+0000 7f0dabb4e640 1 --2- 172.21.15.115:0/3128745341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d3280 0x7f0da40b5d10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.622+0000 7f0dabb4e640 1 -- 172.21.15.115:0/3128745341 >> 172.21.15.115:0/3128745341 conn(0x7f0da406a0b0 msgr2=0x7f0da406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:24.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.622+0000 7f857a7fc640 1 --2- 172.21.15.115:0/3296010460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f857c0b6090 0x7f857c0cdcf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f857c0c9e10 tx=0x7f857002bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc45cd22242fae5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.623+0000 7f8579ffb640 1 -- 172.21.15.115:0/3296010460 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857002d660 con 0x7f857c0b6090 2023-12-01T01:36:24.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.623+0000 7f8579ffb640 1 -- 172.21.15.115:0/3296010460 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857002d7c0 con 0x7f857c0b6090 2023-12-01T01:36:24.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.623+0000 7f8579ffb640 1 -- 172.21.15.115:0/3296010460 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857002db50 con 0x7f857c0b6090 2023-12-01T01:36:24.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.622+0000 7f0dabb4e640 1 -- 172.21.15.115:0/3128745341 shutdown_connections 2023-12-01T01:36:24.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.622+0000 7f0dabb4e640 1 -- 172.21.15.115:0/3128745341 wait complete. 2023-12-01T01:36:24.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.624+0000 7f8581669640 1 -- 172.21.15.115:0/3296010460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f857c0b6090 msgr2=0x7f857c0cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.624+0000 7f8581669640 1 --2- 172.21.15.115:0/3296010460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f857c0b6090 0x7f857c0cdcf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f857c0c9e10 tx=0x7f857002bab0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.625+0000 7f8581669640 1 -- 172.21.15.115:0/3296010460 shutdown_connections 2023-12-01T01:36:24.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.625+0000 7f8581669640 1 --2- 172.21.15.115:0/3296010460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f857c0b6090 0x7f857c0cdcf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.625+0000 7f8581669640 1 --2- 172.21.15.115:0/3296010460 >> v2:172.21.15.142:3300/0 conn(0x7f857c0cf740 0x7f857c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.625+0000 7f8581669640 1 -- 172.21.15.115:0/3296010460 >> 172.21.15.115:0/3296010460 conn(0x7f857c065960 msgr2=0x7f857c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:24.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.625+0000 7f8581669640 1 -- 172.21.15.115:0/3296010460 shutdown_connections 2023-12-01T01:36:24.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.626+0000 7f0dabb4e640 1 Processor -- start 2023-12-01T01:36:24.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.626+0000 7f8581669640 1 -- 172.21.15.115:0/3296010460 wait complete. 2023-12-01T01:36:24.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.626+0000 7f0dabb4e640 1 -- start start 2023-12-01T01:36:24.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.627+0000 7f0dabb4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d21c0 0x7f0da41f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.628+0000 7f0dabb4e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0da40d3280 0x7f0da4065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.628+0000 7f0dabb4e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0da415d0b0 0x7f0da41fa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.628+0000 7f0dabb4e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da41f6f10 con 0x7f0da40d21c0 2023-12-01T01:36:24.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.628+0000 7f0dabb4e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da41facf0 con 0x7f0da40d3280 2023-12-01T01:36:24.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.628+0000 7f0dabb4e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da41fafd0 con 0x7f0da415d0b0 2023-12-01T01:36:24.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.628+0000 7f0da98ea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d21c0 0x7f0da41f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.628+0000 7f0da98ea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d21c0 0x7f0da41f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56136/0 (socket says 172.21.15.115:56136) 2023-12-01T01:36:24.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.628+0000 7f0da98ea640 1 -- 172.21.15.115:0/1931139254 learned_addr learned my addr 172.21.15.115:0/1931139254 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.629+0000 7f8581669640 1 Processor -- start 2023-12-01T01:36:24.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.629+0000 7f0da98ea640 1 -- 172.21.15.115:0/1931139254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0da415d0b0 msgr2=0x7f0da41fa180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:36:24.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.629+0000 7f8581669640 1 -- start start 2023-12-01T01:36:24.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.629+0000 7f0da98ea640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0da415d0b0 0x7f0da41fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.629+0000 7f0da98ea640 1 -- 172.21.15.115:0/1931139254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0da40d3280 msgr2=0x7f0da4065bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:36:24.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.629+0000 7f0da98ea640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0da40d3280 0x7f0da4065bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.629+0000 7f0da98ea640 1 -- 172.21.15.115:0/1931139254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0009400 con 0x7f0da40d21c0 2023-12-01T01:36:24.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.630+0000 7f0da98ea640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d21c0 0x7f0da41f7c40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c730 tx=0x7f0d9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.630+0000 7f0d9a7fc640 1 -- 172.21.15.115:0/1931139254 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00ce40 con 0x7f0da40d21c0 2023-12-01T01:36:24.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.631+0000 7f0d9a7fc640 1 -- 172.21.15.115:0/1931139254 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c00a640 con 0x7f0da40d21c0 2023-12-01T01:36:24.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.631+0000 7f0d9a7fc640 1 -- 172.21.15.115:0/1931139254 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00aa70 con 0x7f0da40d21c0 2023-12-01T01:36:24.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.631+0000 7f0dabb4e640 1 -- 172.21.15.115:0/1931139254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da415edb0 con 0x7f0da40d21c0 2023-12-01T01:36:24.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.631+0000 7f0dabb4e640 1 -- 172.21.15.115:0/1931139254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da415f370 con 0x7f0da40d21c0 2023-12-01T01:36:24.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.632+0000 7f0dabb4e640 1 -- 172.21.15.115:0/1931139254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da415f560 con 0x7f0da40d21c0 2023-12-01T01:36:24.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.635+0000 7f8581669640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f857c0b6090 0x7f857c14b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.635+0000 7f8581669640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f857c0cf740 0x7f857c0b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.635+0000 7f8581669640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f857c160370 0x7f857c0b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.635+0000 7f8581669640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c0b4f80 con 0x7f857c160370 2023-12-01T01:36:24.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.636+0000 7f8581669640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c0c65e0 con 0x7f857c0b6090 2023-12-01T01:36:24.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.636+0000 7f8581669640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857c0c68c0 con 0x7f857c0cf740 2023-12-01T01:36:24.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.636+0000 7f857a7fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f857c0cf740 0x7f857c0b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.636+0000 7f857affd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f857c0b6090 0x7f857c14b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.636+0000 7f857a7fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f857c0cf740 0x7f857c0b4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:34028/0 (socket says 172.21.15.115:34028) 2023-12-01T01:36:24.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.636+0000 7f857a7fc640 1 -- 172.21.15.115:0/542067492 learned_addr learned my addr 172.21.15.115:0/542067492 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.637+0000 7f857a7fc640 1 -- 172.21.15.115:0/542067492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f857c0b6090 msgr2=0x7f857c14b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.637+0000 7f857a7fc640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f857c0b6090 0x7f857c14b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.637+0000 7f857a7fc640 1 -- 172.21.15.115:0/542067492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f857c160370 msgr2=0x7f857c0b4a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:36:24.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.637+0000 7f857a7fc640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f857c160370 0x7f857c0b4a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.637+0000 7f857a7fc640 1 -- 172.21.15.115:0/542067492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8570009400 con 0x7f857c0cf740 2023-12-01T01:36:24.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.638+0000 7f857a7fc640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f857c0cf740 0x7f857c0b4500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8570009af0 tx=0x7f85700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.636+0000 7f0d9a7fc640 1 -- 172.21.15.115:0/1931139254 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0d9c00abd0 con 0x7f0da40d21c0 2023-12-01T01:36:24.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.638+0000 7f0d9a7fc640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d8806a430 0x7f0d8806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.639+0000 7f0da90e9640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d8806a430 0x7f0d8806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.639+0000 7f85637fe640 1 -- 172.21.15.115:0/542067492 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857003b040 con 0x7f857c0cf740 2023-12-01T01:36:24.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.639+0000 7f85637fe640 1 -- 172.21.15.115:0/542067492 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857002ec00 con 0x7f857c0cf740 2023-12-01T01:36:24.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.639+0000 7f85637fe640 1 -- 172.21.15.115:0/542067492 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570037400 con 0x7f857c0cf740 2023-12-01T01:36:24.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.639+0000 7f8581669640 1 -- 172.21.15.115:0/542067492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857c0c6ba0 con 0x7f857c0cf740 2023-12-01T01:36:24.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.640+0000 7f0d9a7fc640 1 -- 172.21.15.115:0/1931139254 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d9c057ef0 con 0x7f0da40d21c0 2023-12-01T01:36:24.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.639+0000 7f8581669640 1 -- 172.21.15.115:0/542067492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c0c7160 con 0x7f857c0cf740 2023-12-01T01:36:24.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.641+0000 7f8581669640 1 -- 172.21.15.115:0/542067492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857c0c7380 con 0x7f857c0cf740 2023-12-01T01:36:24.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.641+0000 7f0da90e9640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d8806a430 0x7f0d8806c8f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d94000f60 tx=0x7f0d94007880 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.644+0000 7f0d9a7fc640 1 -- 172.21.15.115:0/1931139254 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0d9c0542c0 con 0x7f0da40d21c0 2023-12-01T01:36:24.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.646+0000 7f85637fe640 1 -- 172.21.15.115:0/542067492 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8570037560 con 0x7f857c0cf740 2023-12-01T01:36:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.647+0000 7f85637fe640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f856806a430 0x7f856806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.647+0000 7f85637fe640 1 -- 172.21.15.115:0/542067492 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85700aa860 con 0x7f857c0cf740 2023-12-01T01:36:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.648+0000 7f857affd640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f856806a430 0x7f856806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.650+0000 7f857affd640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f856806a430 0x7f856806c8f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f856c004ea0 tx=0x7f856c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.654+0000 7f85637fe640 1 -- 172.21.15.115:0/542067492 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8570075fe0 con 0x7f857c0cf740 2023-12-01T01:36:24.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.675+0000 7f2fc4a79640 1 Processor -- start 2023-12-01T01:36:24.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.675+0000 7f2fc4a79640 1 -- start start 2023-12-01T01:36:24.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.677+0000 7f2fc4a79640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2fc01601a0 0x7f2fc0160840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.677+0000 7f2fc4a79640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2fc00b8600 0x7f2fc00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.677+0000 7f2fc4a79640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc00cddf0 0x7f2fc00b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.677+0000 7f2fc4a79640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00ce280 con 0x7f2fc00b8600 2023-12-01T01:36:24.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.677+0000 7f2fc4a79640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00b5d80 con 0x7f2fc01601a0 2023-12-01T01:36:24.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.677+0000 7f2fc4a79640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0161770 con 0x7f2fc00cddf0 2023-12-01T01:36:24.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.678+0000 7f2fbf7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2fc01601a0 0x7f2fc0160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.678+0000 7f2fbf7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2fc01601a0 0x7f2fc0160840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53508/0 (socket says 172.21.15.115:53508) 2023-12-01T01:36:24.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.678+0000 7f2fbf7fe640 1 -- 172.21.15.115:0/801224643 learned_addr learned my addr 172.21.15.115:0/801224643 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.678+0000 7f2fbffff640 1 --2- 172.21.15.115:0/801224643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc00cddf0 0x7f2fc00b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.679+0000 7f2fbf7fe640 1 -- 172.21.15.115:0/801224643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc00cddf0 msgr2=0x7f2fc00b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:24.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.679+0000 7f2fbf7fe640 1 --2- 172.21.15.115:0/801224643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc00cddf0 0x7f2fc00b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.679+0000 7f2fbf7fe640 1 -- 172.21.15.115:0/801224643 >> v2:172.21.15.115:3300/0 conn(0x7f2fc00b8600 msgr2=0x7f2fc00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:24.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.679+0000 7f2fbf7fe640 1 --2- 172.21.15.115:0/801224643 >> v2:172.21.15.115:3300/0 conn(0x7f2fc00b8600 0x7f2fc00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.679+0000 7f2fbf7fe640 1 -- 172.21.15.115:0/801224643 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc01611b0 con 0x7f2fc01601a0 2023-12-01T01:36:24.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.681+0000 7f2fbf7fe640 1 --2- 172.21.15.115:0/801224643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc01601a0 0x7f2fc0160840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0066e90 tx=0x7f2fb402b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=309e6dbf9ff56d06 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.682+0000 7f2fbe7fc640 1 -- 172.21.15.115:0/801224643 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4031740 con 0x7f2fc01601a0 2023-12-01T01:36:24.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.683+0000 7f2fbe7fc640 1 -- 172.21.15.115:0/801224643 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb40318a0 con 0x7f2fc01601a0 2023-12-01T01:36:24.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.683+0000 7f2fbe7fc640 1 -- 172.21.15.115:0/801224643 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4031c30 con 0x7f2fc01601a0 2023-12-01T01:36:24.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.684+0000 7f2fc4a79640 1 -- 172.21.15.115:0/801224643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc01601a0 msgr2=0x7f2fc0160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.684+0000 7f2fc4a79640 1 --2- 172.21.15.115:0/801224643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc01601a0 0x7f2fc0160840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0066e90 tx=0x7f2fb402b7e0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.685+0000 7f2fc4a79640 1 -- 172.21.15.115:0/801224643 shutdown_connections 2023-12-01T01:36:24.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.685+0000 7f2fc4a79640 1 --2- 172.21.15.115:0/801224643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc00cddf0 0x7f2fc00b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.685+0000 7f2fc4a79640 1 --2- 172.21.15.115:0/801224643 >> v2:172.21.15.115:3300/0 conn(0x7f2fc00b8600 0x7f2fc00cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.685+0000 7f2fc4a79640 1 --2- 172.21.15.115:0/801224643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc01601a0 0x7f2fc0160840 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.685+0000 7f2fc4a79640 1 -- 172.21.15.115:0/801224643 >> 172.21.15.115:0/801224643 conn(0x7f2fc0065960 msgr2=0x7f2fc0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:24.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.685+0000 7f2fc4a79640 1 -- 172.21.15.115:0/801224643 shutdown_connections 2023-12-01T01:36:24.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.686+0000 7f2fc4a79640 1 -- 172.21.15.115:0/801224643 wait complete. 2023-12-01T01:36:24.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.688+0000 7f2fc4a79640 1 Processor -- start 2023-12-01T01:36:24.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.689+0000 7f2fc4a79640 1 -- start start 2023-12-01T01:36:24.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.691+0000 7f2fc4a79640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc00b8600 0x7f2fc014aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.691+0000 7f2fc4a79640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc00cddf0 0x7f2fc015e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.692+0000 7f2fc4a79640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc01601a0 0x7f2fc00d1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.693+0000 7f2fbf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc00b8600 0x7f2fc014aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.693+0000 7f2fbf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc00b8600 0x7f2fc014aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53518/0 (socket says 172.21.15.115:53518) 2023-12-01T01:36:24.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.693+0000 7f2fbf7fe640 1 -- 172.21.15.115:0/1595728336 learned_addr learned my addr 172.21.15.115:0/1595728336 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:24.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.693+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc015f5b0 con 0x7f2fc00cddf0 2023-12-01T01:36:24.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.693+0000 7f2fbffff640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc01601a0 0x7f2fc00d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.693+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc015d170 con 0x7f2fc00b8600 2023-12-01T01:36:24.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.693+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc015d450 con 0x7f2fc01601a0 2023-12-01T01:36:24.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.694+0000 7f2fbeffd640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc00cddf0 0x7f2fc015e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.695+0000 7f2fbeffd640 1 -- 172.21.15.115:0/1595728336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc01601a0 msgr2=0x7f2fc00d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.695+0000 7f2fbeffd640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc01601a0 0x7f2fc00d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.695+0000 7f2fbeffd640 1 -- 172.21.15.115:0/1595728336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc00b8600 msgr2=0x7f2fc014aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.695+0000 7f2fbeffd640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc00b8600 0x7f2fc014aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.695+0000 7f2fbeffd640 1 -- 172.21.15.115:0/1595728336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb40094d0 con 0x7f2fc00cddf0 2023-12-01T01:36:24.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.696+0000 7f2fbeffd640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc00cddf0 0x7f2fc015e210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000b3e0 tx=0x7f2fb000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.697+0000 7f2fbf7fe640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc00b8600 0x7f2fc014aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:24.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.697+0000 7f2fa7fff640 1 -- 172.21.15.115:0/1595728336 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0011830 con 0x7f2fc00cddf0 2023-12-01T01:36:24.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.697+0000 7f2fa7fff640 1 -- 172.21.15.115:0/1595728336 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb0011990 con 0x7f2fc00cddf0 2023-12-01T01:36:24.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.697+0000 7f2fa7fff640 1 -- 172.21.15.115:0/1595728336 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0011d40 con 0x7f2fc00cddf0 2023-12-01T01:36:24.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.697+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc015d730 con 0x7f2fc00cddf0 2023-12-01T01:36:24.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.697+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc015dcf0 con 0x7f2fc00cddf0 2023-12-01T01:36:24.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.700+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa0005250 con 0x7f2fc00cddf0 2023-12-01T01:36:24.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.701+0000 7f2fa7fff640 1 -- 172.21.15.115:0/1595728336 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2fb001a400 con 0x7f2fc00cddf0 2023-12-01T01:36:24.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.702+0000 7f2fa7fff640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fac06a430 0x7f2fac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:24.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.707+0000 7f2fa7fff640 1 -- 172.21.15.115:0/1595728336 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fb0058240 con 0x7f2fc00cddf0 2023-12-01T01:36:24.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.711+0000 7f2fbf7fe640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fac06a430 0x7f2fac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:24.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.712+0000 7f2fbf7fe640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fac06a430 0x7f2fac06c8f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40090b0 tx=0x7f2fb4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:24.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.713+0000 7f2fa7fff640 1 -- 172.21.15.115:0/1595728336 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2fb0053e80 con 0x7f2fc00cddf0 2023-12-01T01:36:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.891+0000 7f683d2ab640 1 -- 172.21.15.115:0/1030119147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f683806c9a0 con 0x7f68380af4f0 2023-12-01T01:36:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.892+0000 7f681f7fe640 1 -- 172.21.15.115:0/1030119147 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f683806c9a0 con 0x7f68380af4f0 2023-12-01T01:36:24.893 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:24.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.896+0000 7f683d2ab640 1 -- 172.21.15.115:0/1030119147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a430 msgr2=0x7f682406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.896+0000 7f683d2ab640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a430 0x7f682406c8f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f682c036000 tx=0x7f682c03d040 comp rx=0 tx=0).stop 2023-12-01T01:36:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.896+0000 7f683d2ab640 1 -- 172.21.15.115:0/1030119147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68380af4f0 msgr2=0x7f683815b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.896+0000 7f683d2ab640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68380af4f0 0x7f683815b180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f682800c430 tx=0x7f682800c8f0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.896+0000 7f683d2ab640 1 -- 172.21.15.115:0/1030119147 shutdown_connections 2023-12-01T01:36:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.896+0000 7f683d2ab640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f682406a430 0x7f682406c8f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.896+0000 7f683d2ab640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f683816df40 0x7f68380b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.896+0000 7f683d2ab640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68380ca520 0x7f68380c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.896+0000 7f683d2ab640 1 --2- 172.21.15.115:0/1030119147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68380af4f0 0x7f683815b180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.896+0000 7f683d2ab640 1 -- 172.21.15.115:0/1030119147 >> 172.21.15.115:0/1030119147 conn(0x7f6838065960 msgr2=0x7f68380cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.897+0000 7f683d2ab640 1 -- 172.21.15.115:0/1030119147 shutdown_connections 2023-12-01T01:36:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.897+0000 7f683d2ab640 1 -- 172.21.15.115:0/1030119147 wait complete. 2023-12-01T01:36:24.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-12-01T01:36:24.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.977+0000 7f0dabb4e640 1 -- 172.21.15.115:0/1931139254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0da404e890 con 0x7f0da40d21c0 2023-12-01T01:36:24.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.977+0000 7f0d9a7fc640 1 -- 172.21.15.115:0/1931139254 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d9c017070 con 0x7f0da40d21c0 2023-12-01T01:36:24.979 INFO:teuthology.orchestra.run.smithi115.stdout:55834574853 2023-12-01T01:36:24.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 -- 172.21.15.115:0/1931139254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d8806a430 msgr2=0x7f0d8806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d8806a430 0x7f0d8806c8f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d94000f60 tx=0x7f0d94007880 comp rx=0 tx=0).stop 2023-12-01T01:36:24.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 -- 172.21.15.115:0/1931139254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d21c0 msgr2=0x7f0da41f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:24.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d21c0 0x7f0da41f7c40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c730 tx=0x7f0d9c00cbf0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 -- 172.21.15.115:0/1931139254 shutdown_connections 2023-12-01T01:36:24.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0d8806a430 0x7f0d8806c8f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0da415d0b0 0x7f0da41fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0da40d3280 0x7f0da4065bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 --2- 172.21.15.115:0/1931139254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0da40d21c0 0x7f0da41f7c40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:24.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 -- 172.21.15.115:0/1931139254 >> 172.21.15.115:0/1931139254 conn(0x7f0da406a0b0 msgr2=0x7f0da406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:24.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.980+0000 7f0dabb4e640 1 -- 172.21.15.115:0/1931139254 shutdown_connections 2023-12-01T01:36:24.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.981+0000 7f0dabb4e640 1 -- 172.21.15.115:0/1931139254 wait complete. 2023-12-01T01:36:24.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.994+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb5980527a0 con 0x7fb5980eb1b0 2023-12-01T01:36:24.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.995+0000 7f2c167fc640 1 -- 172.21.15.115:0/2066611466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2c080057d0 con 0x7f2c300c9160 2023-12-01T01:36:24.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.995+0000 7fb595ffb640 1 -- 172.21.15.115:0/289824778 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb5a00d0450 con 0x7fb5980eb1b0 2023-12-01T01:36:24.998 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:24.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.996+0000 7f2c2cff9640 1 -- 172.21.15.115:0/2066611466 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c24003e60 con 0x7f2c300c9160 2023-12-01T01:36:24.999 INFO:teuthology.orchestra.run.smithi115.stdout:55834574856 2023-12-01T01:36:25.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-12-01T01:36:25.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.998+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb58406a500 msgr2=0x7fb58406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.998+0000 7fb5a74f8640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb58406a500 0x7fb58406c9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb59c034040 tx=0x7fb59c03d040 comp rx=0 tx=0).stop 2023-12-01T01:36:25.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.998+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5980eb1b0 msgr2=0x7fb5980e5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.998+0000 7fb5a74f8640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5980eb1b0 0x7fb5980e5ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0068bb0 tx=0x7fb5a0067eb0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.998+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 shutdown_connections 2023-12-01T01:36:25.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.998+0000 7fb5a74f8640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb58406a500 0x7fb58406c9c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.998+0000 7fb5a74f8640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5980e77a0 0x7fb59804abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.998+0000 7fb5a74f8640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5980eb1b0 0x7fb5980e5ad0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.998+0000 7fb5a74f8640 1 --2- 172.21.15.115:0/289824778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb598052ed0 0x7fb59817a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.998+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 >> 172.21.15.115:0/289824778 conn(0x7fb598054b00 msgr2=0x7fb598053320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:25.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 -- 172.21.15.115:0/2066611466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2c1c06a430 msgr2=0x7f2c1c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2c1c06a430 0x7f2c1c06c8f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c300cab60 tx=0x7f2c20009040 comp rx=0 tx=0).stop 2023-12-01T01:36:25.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 -- 172.21.15.115:0/2066611466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 msgr2=0x7f2c30070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 0x7f2c30070c10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009890 tx=0x7f2c240046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 -- 172.21.15.115:0/2066611466 shutdown_connections 2023-12-01T01:36:25.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2c1c06a430 0x7f2c1c06c8f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c3015abc0 0x7f2c300cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c300c9160 0x7f2c30070c10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 --2- 172.21.15.115:0/2066611466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c300b4220 0x7f2c3006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 -- 172.21.15.115:0/2066611466 >> 172.21.15.115:0/2066611466 conn(0x7f2c30065d50 msgr2=0x7f2c30159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 shutdown_connections 2023-12-01T01:36:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7f2c167fc640 1 -- 172.21.15.115:0/2066611466 shutdown_connections 2023-12-01T01:36:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc4580b3040 con 0x7fc4580cf160 2023-12-01T01:36:25.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:24.999+0000 7fb5a74f8640 1 -- 172.21.15.115:0/289824778 wait complete. 2023-12-01T01:36:25.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.000+0000 7f2c167fc640 1 -- 172.21.15.115:0/2066611466 wait complete. 2023-12-01T01:36:25.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.000+0000 7fc44effd640 1 -- 172.21.15.115:0/1642785669 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc4580b3040 con 0x7fc4580cf160 2023-12-01T01:36:25.005 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:25.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.003+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc43c06a490 msgr2=0x7fc43c06c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.003+0000 7fc4606af640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc43c06a490 0x7fc43c06c950 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4540011e0 tx=0x7fc454009040 comp rx=0 tx=0).stop 2023-12-01T01:36:25.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.003+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4580cf160 msgr2=0x7fc4581503e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.003+0000 7fc4606af640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4580cf160 0x7fc4581503e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc45000bdf0 tx=0x7fc450009ef0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.004+0000 7fdd7be2c640 1 -- 172.21.15.115:0/2466924760 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdd6c0e1780 con 0x7fdd6c180560 2023-12-01T01:36:25.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.004+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 shutdown_connections 2023-12-01T01:36:25.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.004+0000 7fc4606af640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc43c06a490 0x7fc43c06c950 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.004+0000 7fc4606af640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4580cf160 0x7fc4581503e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.004+0000 7fc4606af640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4580b0050 0x7fc458151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.004+0000 7fc4606af640 1 --2- 172.21.15.115:0/1642785669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4580afc70 0x7fc4580b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.004+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 >> 172.21.15.115:0/1642785669 conn(0x7fc458065b40 msgr2=0x7fc4580cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:25.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.004+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 shutdown_connections 2023-12-01T01:36:25.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.004+0000 7fc4606af640 1 -- 172.21.15.115:0/1642785669 wait complete. 2023-12-01T01:36:25.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.005+0000 7fdd6a7fc640 1 -- 172.21.15.115:0/2466924760 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdd6c0e1780 con 0x7fdd6c180560 2023-12-01T01:36:25.007 INFO:teuthology.orchestra.run.smithi115.stdout:55834574854 2023-12-01T01:36:25.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.007+0000 7fdd7be2c640 1 -- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4b0 msgr2=0x7fdd5806c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.007+0000 7fdd7be2c640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4b0 0x7fdd5806c970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd64036000 tx=0x7fdd6403b040 comp rx=0 tx=0).stop 2023-12-01T01:36:25.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.008+0000 7fdd7be2c640 1 -- 172.21.15.115:0/2466924760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c180560 msgr2=0x7fdd6c03a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.008+0000 7fdd7be2c640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c180560 0x7fdd6c03a980 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c3c0 tx=0x7fdd7000c7c0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.008+0000 7fdd7be2c640 1 -- 172.21.15.115:0/2466924760 shutdown_connections 2023-12-01T01:36:25.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.008+0000 7fdd7be2c640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4b0 0x7fdd5806c970 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.008+0000 7fdd7be2c640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd6c180560 0x7fdd6c03a980 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.008+0000 7fdd7be2c640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd6c038620 0x7fdd6c180c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.008+0000 7fdd7be2c640 1 --2- 172.21.15.115:0/2466924760 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd6c0066a0 0x7fdd6c0db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.008+0000 7fdd7be2c640 1 -- 172.21.15.115:0/2466924760 >> 172.21.15.115:0/2466924760 conn(0x7fdd6c055a90 msgr2=0x7fdd6c038a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:25.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.008+0000 7fdd7be2c640 1 -- 172.21.15.115:0/2466924760 shutdown_connections 2023-12-01T01:36:25.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.008+0000 7fdd7be2c640 1 -- 172.21.15.115:0/2466924760 wait complete. 2023-12-01T01:36:25.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-12-01T01:36:25.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2023-12-01T01:36:25.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-12-01T01:36:25.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2023-12-01T01:36:25.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.042+0000 7f8581669640 1 -- 172.21.15.115:0/542067492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f857c0d00f0 con 0x7f857c0cf740 2023-12-01T01:36:25.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.043+0000 7f85637fe640 1 -- 172.21.15.115:0/542067492 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f857c0d00f0 con 0x7f857c0cf740 2023-12-01T01:36:25.044 INFO:teuthology.orchestra.run.smithi115.stdout:55834574854 2023-12-01T01:36:25.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 -- 172.21.15.115:0/542067492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f856806a430 msgr2=0x7f856806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f856806a430 0x7f856806c8f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f856c004ea0 tx=0x7f856c009040 comp rx=0 tx=0).stop 2023-12-01T01:36:25.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 -- 172.21.15.115:0/542067492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f857c0cf740 msgr2=0x7f857c0b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f857c0cf740 0x7f857c0b4500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8570009af0 tx=0x7f85700046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 -- 172.21.15.115:0/542067492 shutdown_connections 2023-12-01T01:36:25.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f856806a430 0x7f856806c8f0 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f856c004ea0 tx=0x7f856c009040 comp rx=0 tx=0).stop 2023-12-01T01:36:25.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f857c0cf740 0x7f857c0b4500 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f857c160370 0x7f857c0b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 --2- 172.21.15.115:0/542067492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f857c0b6090 0x7f857c14b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 -- 172.21.15.115:0/542067492 >> 172.21.15.115:0/542067492 conn(0x7f857c065960 msgr2=0x7f857c0cd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:25.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 -- 172.21.15.115:0/542067492 shutdown_connections 2023-12-01T01:36:25.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.045+0000 7f8581669640 1 -- 172.21.15.115:0/542067492 wait complete. 2023-12-01T01:36:25.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-12-01T01:36:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.067+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2fa0005500 con 0x7f2fc00cddf0 2023-12-01T01:36:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.068+0000 7f2fa7fff640 1 -- 172.21.15.115:0/1595728336 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2fb0015070 con 0x7f2fc00cddf0 2023-12-01T01:36:25.070 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.070+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fac06a430 msgr2=0x7f2fac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.070+0000 7f2fc4a79640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fac06a430 0x7f2fac06c8f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40090b0 tx=0x7f2fb4008030 comp rx=0 tx=0).stop 2023-12-01T01:36:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.070+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc00cddf0 msgr2=0x7f2fc015e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.070+0000 7f2fc4a79640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc00cddf0 0x7f2fc015e210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000b3e0 tx=0x7f2fb000b8a0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.070+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 shutdown_connections 2023-12-01T01:36:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.070+0000 7f2fc4a79640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fac06a430 0x7f2fac06c8f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.070+0000 7f2fc4a79640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc01601a0 0x7f2fc00d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.070+0000 7f2fc4a79640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc00cddf0 0x7f2fc015e210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.070+0000 7f2fc4a79640 1 --2- 172.21.15.115:0/1595728336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc00b8600 0x7f2fc014aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.070+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 >> 172.21.15.115:0/1595728336 conn(0x7f2fc0065960 msgr2=0x7f2fc0155230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:25.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.071+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 shutdown_connections 2023-12-01T01:36:25.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:25.071+0000 7f2fc4a79640 1 -- 172.21.15.115:0/1595728336 wait complete. 2023-12-01T01:36:25.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2023-12-01T01:36:25.925 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-01T01:36:26.000 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-01T01:36:26.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-01T01:36:26.031 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-01T01:36:26.031 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-01T01:36:26.031 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-01T01:36:26.065 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-01T01:36:26.089 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-01T01:36:26.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.186+0000 7fce7ffff640 1 Processor -- start 2023-12-01T01:36:26.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.186+0000 7fce7ffff640 1 -- start start 2023-12-01T01:36:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.188+0000 7fce7ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce780552f0 0x7fce780520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.188+0000 7fce7ffff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce78056300 con 0x7fce780e7910 2023-12-01T01:36:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.188+0000 7fce7ffff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce78055a60 con 0x7fce780e6e20 2023-12-01T01:36:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.188+0000 7fce7ffff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce78057af0 con 0x7fce780552f0 2023-12-01T01:36:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.189+0000 7fce7effd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fce780e6e20 0x7fce780e7200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:56950/0 (socket says 172.21.15.115:56950) 2023-12-01T01:36:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.189+0000 7fce7effd640 1 -- 172.21.15.115:0/2810563910 learned_addr learned my addr 172.21.15.115:0/2810563910 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.189+0000 7fce7f7fe640 1 --2- 172.21.15.115:0/2810563910 >> v2:172.21.15.142:3300/0 conn(0x7fce780552f0 0x7fce780520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.190+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718754453 0 0) 0x7fce78055a60 con 0x7fce780e6e20 2023-12-01T01:36:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.190+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce6800bd30 con 0x7fce780e6e20 2023-12-01T01:36:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.191+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1916688860 0 0) 0x7fce6800bd30 con 0x7fce780e6e20 2023-12-01T01:36:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.191+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 >> v2:172.21.15.142:3300/0 conn(0x7fce780552f0 msgr2=0x7fce780520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.191+0000 7fce7dffb640 1 --2- 172.21.15.115:0/2810563910 >> v2:172.21.15.142:3300/0 conn(0x7fce780552f0 0x7fce780520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.191+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 >> v1:172.21.15.115:6789/0 conn(0x7fce780e7910 legacy=0x7fce780519b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:36:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.192+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78055d40 con 0x7fce780e6e20 2023-12-01T01:36:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.192+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fce80072570 con 0x7fce780e6e20 2023-12-01T01:36:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.192+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 >> v1:172.21.15.142:6789/0 conn(0x7fce780e6e20 legacy=0x7fce780e7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.192+0000 7fce7dffb640 1 --2- 172.21.15.115:0/2810563910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6800c180 0x7fce6800c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.192+0000 7fce7dffb640 1 --2- 172.21.15.115:0/2810563910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6800cfd0 0x7fce6800d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.192+0000 7fce7dffb640 1 --2- 172.21.15.115:0/2810563910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6800d990 0x7fce68014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.192+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce78057af0 con 0x7fce6800c180 2023-12-01T01:36:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.192+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce68014810 con 0x7fce6800d990 2023-12-01T01:36:26.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.192+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce680149d0 con 0x7fce6800cfd0 2023-12-01T01:36:26.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.195+0000 7fce7effd640 1 --2- 172.21.15.115:0/2810563910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6800c180 0x7fce6800c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.196+0000 7fce7effd640 1 -- 172.21.15.115:0/2810563910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6800cfd0 msgr2=0x7fce6800d450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-01T01:36:26.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.196+0000 7fce7effd640 1 --2- 172.21.15.115:0/2810563910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6800cfd0 0x7fce6800d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.196+0000 7fce7effd640 1 -- 172.21.15.115:0/2810563910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6800d990 msgr2=0x7fce68014190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-01T01:36:26.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.196+0000 7fce7f7fe640 1 --2- 172.21.15.115:0/2810563910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6800d990 0x7fce68014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.196+0000 7fce7effd640 1 --2- 172.21.15.115:0/2810563910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6800d990 0x7fce68014190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.196+0000 7fce7effd640 1 -- 172.21.15.115:0/2810563910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78055d40 con 0x7fce6800c180 2023-12-01T01:36:26.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.197+0000 7fce7f7fe640 1 -- 172.21.15.115:0/2810563910 reap_dead start 2023-12-01T01:36:26.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.198+0000 7fce7effd640 1 --2- 172.21.15.115:0/2810563910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6800c180 0x7fce6800c580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce800683c0 tx=0x7fce6800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4031f5221fd5a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.199+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce800ad400 con 0x7fce6800c180 2023-12-01T01:36:26.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.199+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce800a9050 con 0x7fce6800c180 2023-12-01T01:36:26.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.199+0000 7fce7dffb640 1 -- 172.21.15.115:0/2810563910 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce800ad880 con 0x7fce6800c180 2023-12-01T01:36:26.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.200+0000 7fce7ffff640 1 -- 172.21.15.115:0/2810563910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6800c180 msgr2=0x7fce6800c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.200+0000 7fce7ffff640 1 --2- 172.21.15.115:0/2810563910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6800c180 0x7fce6800c580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce800683c0 tx=0x7fce6800e000 comp rx=0 tx=0).stop 2023-12-01T01:36:26.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.201+0000 7fce7ffff640 1 -- 172.21.15.115:0/2810563910 shutdown_connections 2023-12-01T01:36:26.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.201+0000 7fce7ffff640 1 --2- 172.21.15.115:0/2810563910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6800c180 0x7fce6800c580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.201+0000 7fce7ffff640 1 -- 172.21.15.115:0/2810563910 >> 172.21.15.115:0/2810563910 conn(0x7fce78056d10 msgr2=0x7fce78057120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.202+0000 7fce7ffff640 1 -- 172.21.15.115:0/2810563910 shutdown_connections 2023-12-01T01:36:26.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.202+0000 7fce7ffff640 1 -- 172.21.15.115:0/2810563910 wait complete. 2023-12-01T01:36:26.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.205+0000 7fce7ffff640 1 Processor -- start 2023-12-01T01:36:26.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.205+0000 7fce7ffff640 1 -- start start 2023-12-01T01:36:26.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.208+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6800c180 0x7fce781898c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.208+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce78185250 0x7fce780da050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.208+0000 7fce7ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce780da590 0x7fce78008d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.209+0000 7fce7ffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780daa10 con 0x7fce78185250 2023-12-01T01:36:26.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.209+0000 7fce7ffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce78189e00 con 0x7fce6800c180 2023-12-01T01:36:26.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.209+0000 7fce7e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce78185250 0x7fce780da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.209+0000 7fce7e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce78185250 0x7fce780da050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56176/0 (socket says 172.21.15.115:56176) 2023-12-01T01:36:26.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.209+0000 7fce7effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6800c180 0x7fce781898c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.210+0000 7fce7e7fc640 1 -- 172.21.15.115:0/3598317632 learned_addr learned my addr 172.21.15.115:0/3598317632 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.209+0000 7fce7f7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce780da590 0x7fce78008d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.210+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7803b880 con 0x7fce780da590 2023-12-01T01:36:26.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.211+0000 7fce7e7fc640 1 -- 172.21.15.115:0/3598317632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce780da590 msgr2=0x7fce78008d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.211+0000 7fce7e7fc640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce780da590 0x7fce78008d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.211+0000 7fce7e7fc640 1 -- 172.21.15.115:0/3598317632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6800c180 msgr2=0x7fce781898c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.211+0000 7fce7e7fc640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6800c180 0x7fce781898c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.211+0000 7fce7e7fc640 1 -- 172.21.15.115:0/3598317632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7800a450 con 0x7fce78185250 2023-12-01T01:36:26.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.212+0000 7fce7f7fe640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce780da590 0x7fce78008d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:26.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.214+0000 7fce7e7fc640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce78185250 0x7fce780da050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c410 tx=0x7fce7000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.217+0000 7fce67fff640 1 -- 172.21.15.115:0/3598317632 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70011790 con 0x7fce78185250 2023-12-01T01:36:26.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.218+0000 7fce67fff640 1 -- 172.21.15.115:0/3598317632 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce700118f0 con 0x7fce78185250 2023-12-01T01:36:26.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.219+0000 7fce67fff640 1 -- 172.21.15.115:0/3598317632 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70011ce0 con 0x7fce78185250 2023-12-01T01:36:26.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.219+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7800a730 con 0x7fce78185250 2023-12-01T01:36:26.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.219+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7803bb60 con 0x7fce78185250 2023-12-01T01:36:26.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.224+0000 7fce67fff640 1 -- 172.21.15.115:0/3598317632 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fce7001c460 con 0x7fce78185250 2023-12-01T01:36:26.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.227+0000 7fce67fff640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fce6c06a530 0x7fce6c06c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.227+0000 7fce67fff640 1 -- 172.21.15.115:0/3598317632 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce70088630 con 0x7fce78185250 2023-12-01T01:36:26.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.227+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce78005790 con 0x7fce78185250 2023-12-01T01:36:26.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.231+0000 7fce7effd640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fce6c06a530 0x7fce6c06c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.233+0000 7fce7effd640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fce6c06a530 0x7fce6c06c9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce680147e0 tx=0x7fce8006c030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.238+0000 7fce67fff640 1 -- 172.21.15.115:0/3598317632 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fce70053d40 con 0x7fce78185250 2023-12-01T01:36:26.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.266+0000 7fad80696640 1 Processor -- start 2023-12-01T01:36:26.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.267+0000 7fad80696640 1 -- start start 2023-12-01T01:36:26.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.268+0000 7fad80696640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fad7815ccc0 0x7fad780d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.268+0000 7fad80696640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fad780cf430 0x7fad780cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.268+0000 7fad80696640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad780d1fe0 con 0x7fad780b8420 2023-12-01T01:36:26.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.268+0000 7fad80696640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad780d11b0 con 0x7fad780cf430 2023-12-01T01:36:26.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.268+0000 7fad80696640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad780cfd80 con 0x7fad7815ccc0 2023-12-01T01:36:26.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.269+0000 7fad7dc31640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fad780b8420 0x7fad780d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:56956/0 (socket says 172.21.15.115:56956) 2023-12-01T01:36:26.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.269+0000 7fad7dc31640 1 -- 172.21.15.115:0/110343659 learned_addr learned my addr 172.21.15.115:0/110343659 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:26.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.270+0000 7fad7e432640 1 --2- 172.21.15.115:0/110343659 >> v2:172.21.15.142:3300/0 conn(0x7fad7815ccc0 0x7fad780d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.270+0000 7fad7ec33640 1 --2- 172.21.15.115:0/110343659 >> v2:172.21.15.115:3300/0 conn(0x7fad780cf430 0x7fad780cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.270+0000 7fad7d430640 1 -- 172.21.15.115:0/110343659 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837029626 0 0) 0x7fad780d1fe0 con 0x7fad780b8420 2023-12-01T01:36:26.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.270+0000 7fad7d430640 1 -- 172.21.15.115:0/110343659 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad64003520 con 0x7fad780b8420 2023-12-01T01:36:26.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.271+0000 7fad7e432640 1 -- 172.21.15.115:0/110343659 >> v2:172.21.15.115:3300/0 conn(0x7fad780cf430 msgr2=0x7fad780cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:26.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.271+0000 7fad7e432640 1 --2- 172.21.15.115:0/110343659 >> v2:172.21.15.115:3300/0 conn(0x7fad780cf430 0x7fad780cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.271+0000 7fad7e432640 1 -- 172.21.15.115:0/110343659 >> v1:172.21.15.142:6789/0 conn(0x7fad780b8420 legacy=0x7fad780d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.271+0000 7fad7e432640 1 -- 172.21.15.115:0/110343659 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad780d13d0 con 0x7fad7815ccc0 2023-12-01T01:36:26.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.272+0000 7fad7e432640 1 --2- 172.21.15.115:0/110343659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7815ccc0 0x7fad780d2470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad780718f0 tx=0x7fad7402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab45b4c2ada88498 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.273+0000 7fad7d430640 1 -- 172.21.15.115:0/110343659 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74031640 con 0x7fad7815ccc0 2023-12-01T01:36:26.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.273+0000 7fad7d430640 1 -- 172.21.15.115:0/110343659 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad740317a0 con 0x7fad7815ccc0 2023-12-01T01:36:26.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.273+0000 7fad7d430640 1 -- 172.21.15.115:0/110343659 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74031b70 con 0x7fad7815ccc0 2023-12-01T01:36:26.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.274+0000 7fad80696640 1 -- 172.21.15.115:0/110343659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7815ccc0 msgr2=0x7fad780d2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.274+0000 7fad80696640 1 --2- 172.21.15.115:0/110343659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7815ccc0 0x7fad780d2470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad780718f0 tx=0x7fad7402b8f0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.274+0000 7fad80696640 1 -- 172.21.15.115:0/110343659 shutdown_connections 2023-12-01T01:36:26.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.274+0000 7fad80696640 1 --2- 172.21.15.115:0/110343659 >> v2:172.21.15.115:3300/0 conn(0x7fad780cf430 0x7fad780cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.275+0000 7fad80696640 1 --2- 172.21.15.115:0/110343659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7815ccc0 0x7fad780d2470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.275+0000 7fad80696640 1 -- 172.21.15.115:0/110343659 >> 172.21.15.115:0/110343659 conn(0x7fad78065960 msgr2=0x7fad78065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.275+0000 7fad80696640 1 -- 172.21.15.115:0/110343659 shutdown_connections 2023-12-01T01:36:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.277+0000 7fad80696640 1 -- 172.21.15.115:0/110343659 wait complete. 2023-12-01T01:36:26.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.279+0000 7fad80696640 1 Processor -- start 2023-12-01T01:36:26.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.280+0000 7fad80696640 1 -- start start 2023-12-01T01:36:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.281+0000 7fad80696640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad780b8420 0x7fad7814c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.281+0000 7fad80696640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad780cf430 0x7fad781fa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.281+0000 7fad80696640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7815ccc0 0x7fad781e6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.281+0000 7fad80696640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad781facd0 con 0x7fad780b8420 2023-12-01T01:36:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.281+0000 7fad80696640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7814d250 con 0x7fad7815ccc0 2023-12-01T01:36:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.281+0000 7fad80696640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7814c920 con 0x7fad780cf430 2023-12-01T01:36:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.282+0000 7fad7dc31640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad780cf430 0x7fad781fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.282+0000 7fad7e432640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad780b8420 0x7fad7814c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.282+0000 7fad7e432640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad780b8420 0x7fad7814c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56196/0 (socket says 172.21.15.115:56196) 2023-12-01T01:36:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.282+0000 7fad7e432640 1 -- 172.21.15.115:0/1024869273 learned_addr learned my addr 172.21.15.115:0/1024869273 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.283+0000 7fad7e432640 1 -- 172.21.15.115:0/1024869273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad780cf430 msgr2=0x7fad781fa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.283+0000 7fad7e432640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad780cf430 0x7fad781fa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.283+0000 7fad7e432640 1 -- 172.21.15.115:0/1024869273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7815ccc0 msgr2=0x7fad781e6c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:36:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.283+0000 7fad7e432640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7815ccc0 0x7fad781e6c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.283+0000 7fad7e432640 1 -- 172.21.15.115:0/1024869273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad74009400 con 0x7fad780b8420 2023-12-01T01:36:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.283+0000 7fad7dc31640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad780cf430 0x7fad781fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.283+0000 7fad7e432640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad780b8420 0x7fad7814c070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad74000c00 tx=0x7fad740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.284+0000 7fad6effd640 1 -- 172.21.15.115:0/1024869273 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7403ac60 con 0x7fad780b8420 2023-12-01T01:36:26.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.284+0000 7fad6effd640 1 -- 172.21.15.115:0/1024869273 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7403adc0 con 0x7fad780b8420 2023-12-01T01:36:26.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.284+0000 7fad6effd640 1 -- 172.21.15.115:0/1024869273 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74044630 con 0x7fad780b8420 2023-12-01T01:36:26.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.285+0000 7fad80696640 1 -- 172.21.15.115:0/1024869273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7814cc00 con 0x7fad780b8420 2023-12-01T01:36:26.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.285+0000 7fad80696640 1 -- 172.21.15.115:0/1024869273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad781fa9c0 con 0x7fad780b8420 2023-12-01T01:36:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.288+0000 7fad80696640 1 -- 172.21.15.115:0/1024869273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad780673a0 con 0x7fad780b8420 2023-12-01T01:36:26.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.291+0000 7fad6effd640 1 -- 172.21.15.115:0/1024869273 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fad7402f050 con 0x7fad780b8420 2023-12-01T01:36:26.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.292+0000 7fad6effd640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad5c06a480 0x7fad5c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.297+0000 7fad6effd640 1 -- 172.21.15.115:0/1024869273 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad74031e40 con 0x7fad780b8420 2023-12-01T01:36:26.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.297+0000 7fad7dc31640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad5c06a480 0x7fad5c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.302+0000 7fad7dc31640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad5c06a480 0x7fad5c06c940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fad780cfb00 tx=0x7fad68009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.304+0000 7fad6effd640 1 -- 172.21.15.115:0/1024869273 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fad74076540 con 0x7fad780b8420 2023-12-01T01:36:26.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.337+0000 7f77e48c4640 1 Processor -- start 2023-12-01T01:36:26.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.337+0000 7f77e48c4640 1 -- start start 2023-12-01T01:36:26.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.339+0000 7f77e48c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00cdb10 0x7f77e0164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.339+0000 7f77e48c4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e01653f0 con 0x7f77e0161330 2023-12-01T01:36:26.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.339+0000 7f77e48c4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e00c7010 con 0x7f77e00b2150 2023-12-01T01:36:26.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.339+0000 7f77e48c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e0161710 con 0x7f77e00cdb10 2023-12-01T01:36:26.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.340+0000 7f77ded9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00cdb10 0x7f77e0164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.340+0000 7f77de59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f77e0161330 0x7f77e00c6900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51470/0 (socket says 172.21.15.115:51470) 2023-12-01T01:36:26.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.340+0000 7f77de59c640 1 -- 172.21.15.115:0/3114652354 learned_addr learned my addr 172.21.15.115:0/3114652354 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:26.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.341+0000 7f48fbc21640 1 Processor -- start 2023-12-01T01:36:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.341+0000 7f77ddd9b640 1 -- 172.21.15.115:0/3114652354 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507417281 0 0) 0x7f77e00c7010 con 0x7f77e00b2150 2023-12-01T01:36:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.341+0000 7f77ddd9b640 1 -- 172.21.15.115:0/3114652354 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77cc003520 con 0x7f77e00b2150 2023-12-01T01:36:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.341+0000 7f77ddd9b640 1 -- 172.21.15.115:0/3114652354 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860596524 0 0) 0x7f77e01653f0 con 0x7f77e0161330 2023-12-01T01:36:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.341+0000 7f77ddd9b640 1 -- 172.21.15.115:0/3114652354 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77e00c7010 con 0x7f77e0161330 2023-12-01T01:36:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.342+0000 7f77ded9d640 1 -- 172.21.15.115:0/3114652354 >> v1:172.21.15.142:6789/0 conn(0x7f77e00b2150 legacy=0x7f77e00cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.342+0000 7f77ded9d640 1 -- 172.21.15.115:0/3114652354 >> v1:172.21.15.115:6789/0 conn(0x7f77e0161330 legacy=0x7f77e00c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.342+0000 7f77ded9d640 1 -- 172.21.15.115:0/3114652354 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e0165620 con 0x7f77e00cdb10 2023-12-01T01:36:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.341+0000 7f48fbc21640 1 -- start start 2023-12-01T01:36:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.343+0000 7f48fbc21640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f48f40ceb30 0x7f48f40ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.343+0000 7f48fbc21640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f40c8d30 con 0x7f48f40ad6e0 2023-12-01T01:36:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.343+0000 7f48fbc21640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f40cb240 con 0x7f48f4158c10 2023-12-01T01:36:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.343+0000 7f48fbc21640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f40ae060 con 0x7f48f40ceb30 2023-12-01T01:36:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.344+0000 7f77ded9d640 1 --2- 172.21.15.115:0/3114652354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00cdb10 0x7f77e0164eb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009dc0 tx=0x7f77d802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=118402d81d9dae0e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.344+0000 7f48f91bc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f48f40ceb30 0x7f48f40ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.345+0000 7f48f99bd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f48f4158c10 0x7f48f40ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:56970/0 (socket says 172.21.15.115:56970) 2023-12-01T01:36:26.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.345+0000 7f48f99bd640 1 -- 172.21.15.115:0/1596766623 learned_addr learned my addr 172.21.15.115:0/1596766623 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:26.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.345+0000 7f77ddd9b640 1 -- 172.21.15.115:0/3114652354 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d802cbc0 con 0x7f77e00cdb10 2023-12-01T01:36:26.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.345+0000 7f77ddd9b640 1 -- 172.21.15.115:0/3114652354 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d802cd20 con 0x7f77e00cdb10 2023-12-01T01:36:26.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.345+0000 7f77ddd9b640 1 -- 172.21.15.115:0/3114652354 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8034640 con 0x7f77e00cdb10 2023-12-01T01:36:26.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.346+0000 7f48f91bc640 1 -- 172.21.15.115:0/1596766623 >> v1:172.21.15.142:6789/0 conn(0x7f48f4158c10 legacy=0x7f48f40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.346+0000 7f48f91bc640 1 -- 172.21.15.115:0/1596766623 >> v1:172.21.15.115:6789/0 conn(0x7f48f40ad6e0 legacy=0x7f48f40cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.346+0000 7f48f91bc640 1 -- 172.21.15.115:0/1596766623 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40cb520 con 0x7f48f40ceb30 2023-12-01T01:36:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.347+0000 7f77e48c4640 1 -- 172.21.15.115:0/3114652354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00cdb10 msgr2=0x7f77e0164eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.347+0000 7f77e48c4640 1 --2- 172.21.15.115:0/3114652354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00cdb10 0x7f77e0164eb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009dc0 tx=0x7f77d802be10 comp rx=0 tx=0).stop 2023-12-01T01:36:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.347+0000 7f77e48c4640 1 -- 172.21.15.115:0/3114652354 shutdown_connections 2023-12-01T01:36:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.347+0000 7f77e48c4640 1 --2- 172.21.15.115:0/3114652354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00cdb10 0x7f77e0164eb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.347+0000 7f77e48c4640 1 -- 172.21.15.115:0/3114652354 >> 172.21.15.115:0/3114652354 conn(0x7f77e0065960 msgr2=0x7f77e0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.348+0000 7f48f91bc640 1 --2- 172.21.15.115:0/1596766623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f40ceb30 0x7f48f40ccb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48e00071d0 tx=0x7f48e002c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e3ac4416f6bf8bf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.348+0000 7f77e48c4640 1 -- 172.21.15.115:0/3114652354 shutdown_connections 2023-12-01T01:36:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.348+0000 7f77e48c4640 1 -- 172.21.15.115:0/3114652354 wait complete. 2023-12-01T01:36:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.349+0000 7f48f89bb640 1 -- 172.21.15.115:0/1596766623 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0008df0 con 0x7f48f40ceb30 2023-12-01T01:36:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.349+0000 7f48f89bb640 1 -- 172.21.15.115:0/1596766623 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e000a0d0 con 0x7f48f40ceb30 2023-12-01T01:36:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.349+0000 7f48f89bb640 1 -- 172.21.15.115:0/1596766623 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e000a460 con 0x7f48f40ceb30 2023-12-01T01:36:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.350+0000 7f48fbc21640 1 -- 172.21.15.115:0/1596766623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f40ceb30 msgr2=0x7f48f40ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.350+0000 7f48fbc21640 1 --2- 172.21.15.115:0/1596766623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f40ceb30 0x7f48f40ccb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48e00071d0 tx=0x7f48e002c9e0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.351+0000 7fc7aab4e640 1 Processor -- start 2023-12-01T01:36:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.351+0000 7f48fbc21640 1 -- 172.21.15.115:0/1596766623 shutdown_connections 2023-12-01T01:36:26.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.351+0000 7f77e48c4640 1 Processor -- start 2023-12-01T01:36:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.351+0000 7f48fbc21640 1 --2- 172.21.15.115:0/1596766623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f40ceb30 0x7f48f40ccb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.351+0000 7f48fbc21640 1 -- 172.21.15.115:0/1596766623 >> 172.21.15.115:0/1596766623 conn(0x7f48f4065d50 msgr2=0x7f48f4066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.351+0000 7f77e48c4640 1 -- start start 2023-12-01T01:36:26.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.351+0000 7f48fbc21640 1 -- 172.21.15.115:0/1596766623 shutdown_connections 2023-12-01T01:36:26.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.352+0000 7f48fbc21640 1 -- 172.21.15.115:0/1596766623 wait complete. 2023-12-01T01:36:26.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.353+0000 7fe289573640 1 Processor -- start 2023-12-01T01:36:26.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.351+0000 7fc7aab4e640 1 -- start start 2023-12-01T01:36:26.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.353+0000 7fc7aab4e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a4160370 0x7fc7a40cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.353+0000 7fc7aab4e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc7a40cf740 0x7fc7a40cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.353+0000 7fc7aab4e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a4160750 con 0x7fc7a40b6090 2023-12-01T01:36:26.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.353+0000 7fc7aab4e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a40ce660 con 0x7fc7a40cf740 2023-12-01T01:36:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.354+0000 7f77e48c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00b2150 0x7f77e015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.353+0000 7fc7aab4e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40d0130 con 0x7fc7a4160370 2023-12-01T01:36:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.354+0000 7f77e48c4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e00cdb10 0x7f77e00b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.353+0000 7fc7a88ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a4160370 0x7fc7a40cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.354+0000 7fc7a88ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a4160370 0x7fc7a40cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:34112/0 (socket says 172.21.15.115:34112) 2023-12-01T01:36:26.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.354+0000 7f77e48c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e0161330 0x7f77e00d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.354+0000 7fc7a88ea640 1 -- 172.21.15.115:0/4095811464 learned_addr learned my addr 172.21.15.115:0/4095811464 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.353+0000 7fe289573640 1 -- start start 2023-12-01T01:36:26.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7f77ded9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e0161330 0x7f77e00d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7f77ded9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e0161330 0x7f77e00d2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56212/0 (socket says 172.21.15.115:56212) 2023-12-01T01:36:26.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7f77ded9d640 1 -- 172.21.15.115:0/2010166585 learned_addr learned my addr 172.21.15.115:0/2010166585 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00b2c30 con 0x7f77e0161330 2023-12-01T01:36:26.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00b40c0 con 0x7f77e00cdb10 2023-12-01T01:36:26.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7f77d5d9b640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e00cdb10 0x7f77e00b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7fc7a88ea640 1 -- 172.21.15.115:0/4095811464 >> v2:172.21.15.142:3300/0 conn(0x7fc7a40cf740 msgr2=0x7fc7a40cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:26.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7fc7a88ea640 1 --2- 172.21.15.115:0/4095811464 >> v2:172.21.15.142:3300/0 conn(0x7fc7a40cf740 0x7fc7a40cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7fc7a88ea640 1 -- 172.21.15.115:0/4095811464 >> v1:172.21.15.142:6789/0 conn(0x7fc7a40b6090 legacy=0x7fc7a41613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:26.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00b3020 con 0x7f77e00b2150 2023-12-01T01:36:26.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7fe289573640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe284158c10 0x7fe2840ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.355+0000 7fc7a88ea640 1 -- 172.21.15.115:0/4095811464 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a40d2c10 con 0x7fc7a4160370 2023-12-01T01:36:26.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.356+0000 7f77de59c640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00b2150 0x7f77e015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.356+0000 7fe289573640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe28415c790 0x7fe2840ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.356+0000 7fe289573640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2840c8d30 con 0x7fe2840ad6e0 2023-12-01T01:36:26.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.356+0000 7f77ded9d640 1 -- 172.21.15.115:0/2010166585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00b2150 msgr2=0x7f77e015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.356+0000 7f77ded9d640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00b2150 0x7f77e015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.356+0000 7f77ded9d640 1 -- 172.21.15.115:0/2010166585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e00cdb10 msgr2=0x7f77e00b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.356+0000 7f77ded9d640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e00cdb10 0x7f77e00b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.356+0000 7f77ded9d640 1 -- 172.21.15.115:0/2010166585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8009a70 con 0x7f77e0161330 2023-12-01T01:36:26.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.356+0000 7fe282ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe284158c10 0x7fe2840ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.357+0000 7fe289573640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2840ae060 con 0x7fe28415c790 2023-12-01T01:36:26.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.357+0000 7fe282ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe284158c10 0x7fe2840ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:34136/0 (socket says 172.21.15.115:34136) 2023-12-01T01:36:26.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.357+0000 7fe282ffd640 1 -- 172.21.15.115:0/48733038 learned_addr learned my addr 172.21.15.115:0/48733038 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.357+0000 7fe289573640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840cb240 con 0x7fe284158c10 2023-12-01T01:36:26.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.357+0000 7fe2837fe640 1 --2- 172.21.15.115:0/48733038 >> v2:172.21.15.142:3300/0 conn(0x7fe28415c790 0x7fe2840ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.357+0000 7f77ded9d640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e0161330 0x7f77e00d2db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f77d8037040 tx=0x7f77d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.357+0000 7f77d77fe640 1 -- 172.21.15.115:0/2010166585 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d80045f0 con 0x7f77e0161330 2023-12-01T01:36:26.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.357+0000 7f77d77fe640 1 -- 172.21.15.115:0/2010166585 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d802ce90 con 0x7f77e0161330 2023-12-01T01:36:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.357+0000 7f77d77fe640 1 -- 172.21.15.115:0/2010166585 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8009450 con 0x7f77e0161330 2023-12-01T01:36:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.358+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e00b3300 con 0x7f77e0161330 2023-12-01T01:36:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.358+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e00b38c0 con 0x7f77e0161330 2023-12-01T01:36:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.358+0000 7fe282ffd640 1 -- 172.21.15.115:0/48733038 >> v2:172.21.15.142:3300/0 conn(0x7fe28415c790 msgr2=0x7fe2840ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.358+0000 7fe282ffd640 1 --2- 172.21.15.115:0/48733038 >> v2:172.21.15.142:3300/0 conn(0x7fe28415c790 0x7fe2840ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.358+0000 7fe282ffd640 1 -- 172.21.15.115:0/48733038 >> v1:172.21.15.142:6789/0 conn(0x7fe2840ad6e0 legacy=0x7fe284158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.358+0000 7fe282ffd640 1 -- 172.21.15.115:0/48733038 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2840cb520 con 0x7fe284158c10 2023-12-01T01:36:26.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.360+0000 7f48fbc21640 1 Processor -- start 2023-12-01T01:36:26.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.360+0000 7fc7a88ea640 1 --2- 172.21.15.115:0/4095811464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a4160370 0x7fc7a40cdcf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009e50 tx=0x7fc79c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=194e25684dbba551 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.360+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e00b2d70 con 0x7f77e0161330 2023-12-01T01:36:26.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.360+0000 7f48fbc21640 1 -- start start 2023-12-01T01:36:26.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.360+0000 7fe282ffd640 1 --2- 172.21.15.115:0/48733038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe284158c10 0x7fe2840ce420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00aa10 tx=0x7fe27c02cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6210efd70405c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.360+0000 7fc7a37fe640 1 -- 172.21.15.115:0/4095811464 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c02ca70 con 0x7fc7a4160370 2023-12-01T01:36:26.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.360+0000 7fe281ffb640 1 -- 172.21.15.115:0/48733038 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c02db80 con 0x7fe284158c10 2023-12-01T01:36:26.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.361+0000 7fe281ffb640 1 -- 172.21.15.115:0/48733038 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c02dce0 con 0x7fe284158c10 2023-12-01T01:36:26.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.361+0000 7fe281ffb640 1 -- 172.21.15.115:0/48733038 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c035660 con 0x7fe284158c10 2023-12-01T01:36:26.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.361+0000 7fc7a37fe640 1 -- 172.21.15.115:0/4095811464 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c02cbd0 con 0x7fc7a4160370 2023-12-01T01:36:26.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.361+0000 7fc7a37fe640 1 -- 172.21.15.115:0/4095811464 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c02ca70 con 0x7fc7a4160370 2023-12-01T01:36:26.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.362+0000 7f48fbc21640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f40ad6e0 0x7f48f406fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.362+0000 7f48fbc21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40ceb30 0x7f48f4149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.362+0000 7f48fbc21640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4158c10 0x7f48f40af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.362+0000 7f48fbc21640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f4155840 con 0x7f48f40ad6e0 2023-12-01T01:36:26.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.362+0000 7f48fbc21640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f4155b20 con 0x7f48f4158c10 2023-12-01T01:36:26.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.362+0000 7f48fbc21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f414a310 con 0x7f48f40ceb30 2023-12-01T01:36:26.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.362+0000 7fe289573640 1 -- 172.21.15.115:0/48733038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe284158c10 msgr2=0x7fe2840ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.362+0000 7fe289573640 1 --2- 172.21.15.115:0/48733038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe284158c10 0x7fe2840ce420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00aa10 tx=0x7fe27c02cdd0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fc7aab4e640 1 -- 172.21.15.115:0/4095811464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a4160370 msgr2=0x7fc7a40cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fc7aab4e640 1 --2- 172.21.15.115:0/4095811464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a4160370 0x7fc7a40cdcf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009e50 tx=0x7fc79c02bed0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fe289573640 1 -- 172.21.15.115:0/48733038 shutdown_connections 2023-12-01T01:36:26.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7f48f99bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f40ad6e0 0x7f48f406fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fe289573640 1 --2- 172.21.15.115:0/48733038 >> v2:172.21.15.142:3300/0 conn(0x7fe28415c790 0x7fe2840ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fe289573640 1 --2- 172.21.15.115:0/48733038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe284158c10 0x7fe2840ce420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fe289573640 1 -- 172.21.15.115:0/48733038 >> 172.21.15.115:0/48733038 conn(0x7fe284065d50 msgr2=0x7fe284066160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fc7aab4e640 1 -- 172.21.15.115:0/4095811464 shutdown_connections 2023-12-01T01:36:26.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fc7aab4e640 1 --2- 172.21.15.115:0/4095811464 >> v2:172.21.15.142:3300/0 conn(0x7fc7a40cf740 0x7fc7a40cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fc7aab4e640 1 --2- 172.21.15.115:0/4095811464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a4160370 0x7fc7a40cdcf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fc7aab4e640 1 -- 172.21.15.115:0/4095811464 >> 172.21.15.115:0/4095811464 conn(0x7fc7a4065960 msgr2=0x7fc7a4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7f48f91bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40ceb30 0x7f48f4149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7f48f91bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40ceb30 0x7f48f4149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:34138/0 (socket says 172.21.15.115:34138) 2023-12-01T01:36:26.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7f48f91bc640 1 -- 172.21.15.115:0/429488864 learned_addr learned my addr 172.21.15.115:0/429488864 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7f48fa1be640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4158c10 0x7f48f40af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fc7aab4e640 1 -- 172.21.15.115:0/4095811464 shutdown_connections 2023-12-01T01:36:26.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.364+0000 7f48f99bd640 1 -- 172.21.15.115:0/429488864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40ceb30 msgr2=0x7f48f4149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fc7aab4e640 1 -- 172.21.15.115:0/4095811464 wait complete. 2023-12-01T01:36:26.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.364+0000 7f48f99bd640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40ceb30 0x7f48f4149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.364+0000 7f48f99bd640 1 -- 172.21.15.115:0/429488864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4158c10 msgr2=0x7f48f40af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.364+0000 7f48f99bd640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4158c10 0x7f48f40af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.364+0000 7f48f99bd640 1 -- 172.21.15.115:0/429488864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e000ae30 con 0x7f48f40ad6e0 2023-12-01T01:36:26.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.364+0000 7f48fa1be640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4158c10 0x7f48f40af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:26.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.365+0000 7f48f99bd640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f40ad6e0 0x7f48f406fac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48e800caa0 tx=0x7f48e800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fe289573640 1 -- 172.21.15.115:0/48733038 shutdown_connections 2023-12-01T01:36:26.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.363+0000 7fe289573640 1 -- 172.21.15.115:0/48733038 wait complete. 2023-12-01T01:36:26.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.365+0000 7f77d77fe640 1 -- 172.21.15.115:0/2010166585 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f77d8034440 con 0x7f77e0161330 2023-12-01T01:36:26.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.366+0000 7fe289573640 1 Processor -- start 2023-12-01T01:36:26.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.366+0000 7fc7aab4e640 1 Processor -- start 2023-12-01T01:36:26.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.366+0000 7fc7aab4e640 1 -- start start 2023-12-01T01:36:26.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.366+0000 7fe289573640 1 -- start start 2023-12-01T01:36:26.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.367+0000 7f77d77fe640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77c006a430 0x7f77c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.367+0000 7f77d77fe640 1 -- 172.21.15.115:0/2010166585 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77d80ab3d0 con 0x7f77e0161330 2023-12-01T01:36:26.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.367+0000 7f77de59c640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77c006a430 0x7f77c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.368+0000 7fc7aab4e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a40b6090 0x7fc7a4153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.368+0000 7fc7aab4e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7a40cf740 0x7fc7a4153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.368+0000 7fc7aab4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a4160370 0x7fc7a4154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.368+0000 7fc7aab4e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a4143cd0 con 0x7fc7a4160370 2023-12-01T01:36:26.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.368+0000 7fc7aab4e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a4143f70 con 0x7fc7a40cf740 2023-12-01T01:36:26.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.368+0000 7fc7aab4e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a4144250 con 0x7fc7a40b6090 2023-12-01T01:36:26.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.368+0000 7fc7a90eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a4160370 0x7fc7a4154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7fc7a90eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a4160370 0x7fc7a4154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56224/0 (socket says 172.21.15.115:56224) 2023-12-01T01:36:26.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7f77de59c640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77c006a430 0x7f77c006c8f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f77c80011e0 tx=0x7f77c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7fc7a90eb640 1 -- 172.21.15.115:0/3471608070 learned_addr learned my addr 172.21.15.115:0/3471608070 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7fc7a88ea640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a40b6090 0x7fc7a4153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7fe289573640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2840ad6e0 0x7fe284147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7fe289573640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe284158c10 0x7fe2841f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7fe289573640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe28415c790 0x7fe2841f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7fe289573640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe284147c30 con 0x7fe284158c10 2023-12-01T01:36:26.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7fe289573640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe284147a60 con 0x7fe2840ad6e0 2023-12-01T01:36:26.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7fe289573640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2841f3260 con 0x7fe28415c790 2023-12-01T01:36:26.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.369+0000 7fc7a3fff640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7a40cf740 0x7fc7a4153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fc7a90eb640 1 -- 172.21.15.115:0/3471608070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a40b6090 msgr2=0x7fc7a4153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fe2827fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe284158c10 0x7fe2841f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fe2837fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe28415c790 0x7fe2841f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fe2827fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe284158c10 0x7fe2841f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56240/0 (socket says 172.21.15.115:56240) 2023-12-01T01:36:26.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fe2827fc640 1 -- 172.21.15.115:0/2413615888 learned_addr learned my addr 172.21.15.115:0/2413615888 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fc7a90eb640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a40b6090 0x7fc7a4153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fc7a90eb640 1 -- 172.21.15.115:0/3471608070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7a40cf740 msgr2=0x7fc7a4153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fc7a90eb640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7a40cf740 0x7fc7a4153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fc7a90eb640 1 -- 172.21.15.115:0/3471608070 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c009b30 con 0x7fc7a4160370 2023-12-01T01:36:26.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fe282ffd640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2840ad6e0 0x7fe284147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fe2827fc640 1 -- 172.21.15.115:0/2413615888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe28415c790 msgr2=0x7fe2841f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fe2827fc640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe28415c790 0x7fe2841f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fc7a90eb640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a4160370 0x7fc7a4154530 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc798009fd0 tx=0x7fc79800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fe2827fc640 1 -- 172.21.15.115:0/2413615888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2840ad6e0 msgr2=0x7fe284147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fe2827fc640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2840ad6e0 0x7fe284147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.371+0000 7fe2827fc640 1 -- 172.21.15.115:0/2413615888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c00a670 con 0x7fe284158c10 2023-12-01T01:36:26.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.372+0000 7fe2827fc640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe284158c10 0x7fe2841f2d20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe27400c7a0 tx=0x7fe27400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.374+0000 7f48f27fc640 1 -- 172.21.15.115:0/429488864 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e800aba0 con 0x7f48f40ad6e0 2023-12-01T01:36:26.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.374+0000 7fc7a17fa640 1 -- 172.21.15.115:0/3471608070 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798011830 con 0x7fc7a4160370 2023-12-01T01:36:26.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.375+0000 7fc7a17fa640 1 -- 172.21.15.115:0/3471608070 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc798011990 con 0x7fc7a4160370 2023-12-01T01:36:26.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.375+0000 7f48f27fc640 1 -- 172.21.15.115:0/429488864 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e800ad00 con 0x7f48f40ad6e0 2023-12-01T01:36:26.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.375+0000 7f48fbc21640 1 -- 172.21.15.115:0/429488864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f414a5f0 con 0x7f48f40ad6e0 2023-12-01T01:36:26.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.375+0000 7f48f27fc640 1 -- 172.21.15.115:0/429488864 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8012600 con 0x7f48f40ad6e0 2023-12-01T01:36:26.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.375+0000 7fe27b7fe640 1 -- 172.21.15.115:0/2413615888 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274003b70 con 0x7fe284158c10 2023-12-01T01:36:26.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.375+0000 7fe27b7fe640 1 -- 172.21.15.115:0/2413615888 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe274003cd0 con 0x7fe284158c10 2023-12-01T01:36:26.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.375+0000 7fe27b7fe640 1 -- 172.21.15.115:0/2413615888 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27400aa90 con 0x7fe284158c10 2023-12-01T01:36:26.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.376+0000 7f77d77fe640 1 -- 172.21.15.115:0/2010166585 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f77d8076b80 con 0x7f77e0161330 2023-12-01T01:36:26.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.375+0000 7fc7aab4e640 1 -- 172.21.15.115:0/3471608070 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a4144530 con 0x7fc7a4160370 2023-12-01T01:36:26.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.376+0000 7fc7a17fa640 1 -- 172.21.15.115:0/3471608070 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798011830 con 0x7fc7a4160370 2023-12-01T01:36:26.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.376+0000 7fc7aab4e640 1 -- 172.21.15.115:0/3471608070 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a4154a70 con 0x7fc7a4160370 2023-12-01T01:36:26.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.376+0000 7f48fbc21640 1 -- 172.21.15.115:0/429488864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f41497c0 con 0x7f48f40ad6e0 2023-12-01T01:36:26.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.378+0000 7f48fbc21640 1 -- 172.21.15.115:0/429488864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f40673a0 con 0x7f48f40ad6e0 2023-12-01T01:36:26.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.378+0000 7fe289573640 1 -- 172.21.15.115:0/2413615888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe284071c40 con 0x7fe284158c10 2023-12-01T01:36:26.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.378+0000 7fe289573640 1 -- 172.21.15.115:0/2413615888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2841f6050 con 0x7fe284158c10 2023-12-01T01:36:26.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.378+0000 7fc7aab4e640 1 -- 172.21.15.115:0/3471608070 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a4144a50 con 0x7fc7a4160370 2023-12-01T01:36:26.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.380+0000 7fc7a17fa640 1 -- 172.21.15.115:0/3471608070 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc798011b00 con 0x7fc7a4160370 2023-12-01T01:36:26.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.380+0000 7fe289573640 1 -- 172.21.15.115:0/2413615888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2840673a0 con 0x7fe284158c10 2023-12-01T01:36:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.382+0000 7fc7a17fa640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79006a520 0x7fc79006c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.382+0000 7fe27b7fe640 1 -- 172.21.15.115:0/2413615888 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe27400abf0 con 0x7fe284158c10 2023-12-01T01:36:26.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.382+0000 7fc7a17fa640 1 -- 172.21.15.115:0/3471608070 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc798088370 con 0x7fc7a4160370 2023-12-01T01:36:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.381+0000 7f48f27fc640 1 -- 172.21.15.115:0/429488864 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f48e8012760 con 0x7f48f40ad6e0 2023-12-01T01:36:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.382+0000 7f48f27fc640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f48d806a430 0x7f48d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.383+0000 7fc7a88ea640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79006a520 0x7fc79006c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.383+0000 7f48f91bc640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f48d806a430 0x7f48d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.384+0000 7fc7a88ea640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79006a520 0x7fc79006c9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc79c035000 tx=0x7fc79c030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.384+0000 7fe27b7fe640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe26406a480 0x7fe26406c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.385+0000 7fe282ffd640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe26406a480 0x7fe26406c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.385+0000 7f48f91bc640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f48d806a430 0x7f48d806c8f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f48f41514f0 tx=0x7f48e000a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.385+0000 7fe27b7fe640 1 -- 172.21.15.115:0/2413615888 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2740580c0 con 0x7fe284158c10 2023-12-01T01:36:26.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.386+0000 7fe282ffd640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe26406a480 0x7fe26406c940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe27c037000 tx=0x7fe27c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.392+0000 7fc7a17fa640 1 -- 172.21.15.115:0/3471608070 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc7a4144a50 con 0x7fc7a4160370 2023-12-01T01:36:26.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.401+0000 7fe27b7fe640 1 -- 172.21.15.115:0/2413615888 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe274053d00 con 0x7fe284158c10 2023-12-01T01:36:26.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.401+0000 7f48f27fc640 1 -- 172.21.15.115:0/429488864 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48e8057800 con 0x7f48f40ad6e0 2023-12-01T01:36:26.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.408+0000 7f48f27fc640 1 -- 172.21.15.115:0/429488864 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f48e80542d0 con 0x7f48f40ad6e0 2023-12-01T01:36:26.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.412+0000 7f264e82c640 1 Processor -- start 2023-12-01T01:36:26.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.413+0000 7f264e82c640 1 -- start start 2023-12-01T01:36:26.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.415+0000 7f264e82c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2648161330 0x7f2648164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.415+0000 7f264e82c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f26480cd400 0x7f26480cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.415+0000 7f264e82c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26481653f0 con 0x7f26480b2150 2023-12-01T01:36:26.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.415+0000 7f264e82c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26480cddb0 con 0x7f26480cd400 2023-12-01T01:36:26.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.415+0000 7f264e82c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26480c7010 con 0x7f2648161330 2023-12-01T01:36:26.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.416+0000 7f264cdc9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f26480cd400 0x7f26480cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.416+0000 7f264cdc9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f26480cd400 0x7f26480cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56250/0 (socket says 172.21.15.115:56250) 2023-12-01T01:36:26.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.416+0000 7f264cdc9640 1 -- 172.21.15.115:0/766467469 learned_addr learned my addr 172.21.15.115:0/766467469 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.416+0000 7f2647fff640 1 --2- 172.21.15.115:0/766467469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2648161330 0x7f2648164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.417+0000 7f264cdc9640 1 -- 172.21.15.115:0/766467469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2648161330 msgr2=0x7f2648164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:26.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.417+0000 7f264cdc9640 1 --2- 172.21.15.115:0/766467469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2648161330 0x7f2648164eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.417+0000 7f264cdc9640 1 -- 172.21.15.115:0/766467469 >> v1:172.21.15.142:6789/0 conn(0x7f26480b2150 legacy=0x7f26480c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.417+0000 7f264cdc9640 1 -- 172.21.15.115:0/766467469 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2648161710 con 0x7f26480cd400 2023-12-01T01:36:26.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.419+0000 7f264cdc9640 1 --2- 172.21.15.115:0/766467469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26480cd400 0x7f26480cd870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2640009e80 tx=0x7f264002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d656ea40729874 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.419+0000 7f26477fe640 1 -- 172.21.15.115:0/766467469 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264002ca70 con 0x7f26480cd400 2023-12-01T01:36:26.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.419+0000 7f26477fe640 1 -- 172.21.15.115:0/766467469 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264002cbd0 con 0x7f26480cd400 2023-12-01T01:36:26.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.419+0000 7f26477fe640 1 -- 172.21.15.115:0/766467469 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400343f0 con 0x7f26480cd400 2023-12-01T01:36:26.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.420+0000 7f264e82c640 1 -- 172.21.15.115:0/766467469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26480cd400 msgr2=0x7f26480cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.420+0000 7f264e82c640 1 --2- 172.21.15.115:0/766467469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26480cd400 0x7f26480cd870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2640009e80 tx=0x7f264002bed0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.420+0000 7f264e82c640 1 -- 172.21.15.115:0/766467469 shutdown_connections 2023-12-01T01:36:26.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.420+0000 7f264e82c640 1 --2- 172.21.15.115:0/766467469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26480cd400 0x7f26480cd870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.421+0000 7f9648082640 1 Processor -- start 2023-12-01T01:36:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.420+0000 7f264e82c640 1 --2- 172.21.15.115:0/766467469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2648161330 0x7f2648164eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.421+0000 7f264e82c640 1 -- 172.21.15.115:0/766467469 >> 172.21.15.115:0/766467469 conn(0x7f2648065960 msgr2=0x7f2648065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.421+0000 7f264e82c640 1 -- 172.21.15.115:0/766467469 shutdown_connections 2023-12-01T01:36:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.421+0000 7f9648082640 1 -- start start 2023-12-01T01:36:26.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.421+0000 7f264e82c640 1 -- 172.21.15.115:0/766467469 wait complete. 2023-12-01T01:36:26.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.423+0000 7f9648082640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f96400d21c0 0x7f96400d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.423+0000 7f9648082640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f96400d2600 0x7f96400d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.423+0000 7f9648082640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96400d1120 con 0x7f964015d0b0 2023-12-01T01:36:26.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.423+0000 7f9648082640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400b2330 con 0x7f96400d21c0 2023-12-01T01:36:26.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.423+0000 7f9648082640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400b8c10 con 0x7f96400d2600 2023-12-01T01:36:26.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.424+0000 7f9645e1e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f964015d0b0 0x7f96400b8500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51498/0 (socket says 172.21.15.115:51498) 2023-12-01T01:36:26.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.424+0000 7f9645e1e640 1 -- 172.21.15.115:0/746390247 learned_addr learned my addr 172.21.15.115:0/746390247 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.424+0000 7f964661f640 1 --2- 172.21.15.115:0/746390247 >> v2:172.21.15.142:3300/0 conn(0x7f96400d2600 0x7f96400d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.424+0000 7f964561d640 1 --2- 172.21.15.115:0/746390247 >> v2:172.21.15.115:3300/0 conn(0x7f96400d21c0 0x7f96400d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.425+0000 7f964661f640 1 -- 172.21.15.115:0/746390247 >> v2:172.21.15.115:3300/0 conn(0x7f96400d21c0 msgr2=0x7f96400d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.425+0000 7f9644e1c640 1 -- 172.21.15.115:0/746390247 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069954949 0 0) 0x7f96400d1120 con 0x7f964015d0b0 2023-12-01T01:36:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.425+0000 7f964661f640 1 --2- 172.21.15.115:0/746390247 >> v2:172.21.15.115:3300/0 conn(0x7f96400d21c0 0x7f96400d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.425+0000 7f964661f640 1 -- 172.21.15.115:0/746390247 >> v1:172.21.15.115:6789/0 conn(0x7f964015d0b0 legacy=0x7f96400b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.425+0000 7f964661f640 1 -- 172.21.15.115:0/746390247 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400d1400 con 0x7f96400d2600 2023-12-01T01:36:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.425+0000 7f964561d640 1 --2- 172.21.15.115:0/746390247 >> v2:172.21.15.115:3300/0 conn(0x7f96400d21c0 0x7f96400d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.426+0000 7f264e82c640 1 Processor -- start 2023-12-01T01:36:26.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.426+0000 7f264e82c640 1 -- start start 2023-12-01T01:36:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.427+0000 7f264e82c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26480b2150 0x7f26480c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.428+0000 7f964661f640 1 --2- 172.21.15.115:0/746390247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96400d2600 0x7f96400d30b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f96400cd3e0 tx=0x7f963802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=244c709c9bb94b45 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.427+0000 7f264e82c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26480cd400 0x7f26480c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.427+0000 7f264e82c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2648161330 0x7f26480b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.428+0000 7f264e82c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26480ca600 con 0x7f26480cd400 2023-12-01T01:36:26.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.428+0000 7f264e82c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26480b7900 con 0x7f2648161330 2023-12-01T01:36:26.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.428+0000 7f264e82c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26480b2850 con 0x7f26480b2150 2023-12-01T01:36:26.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.428+0000 7f264cdc9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2648161330 0x7f26480b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.428+0000 7f264cdc9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2648161330 0x7f26480b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53618/0 (socket says 172.21.15.115:53618) 2023-12-01T01:36:26.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.428+0000 7f264cdc9640 1 -- 172.21.15.115:0/1141767256 learned_addr learned my addr 172.21.15.115:0/1141767256 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.429+0000 7f263f7fe640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26480cd400 0x7f26480c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.429+0000 7f263f7fe640 1 -- 172.21.15.115:0/1141767256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26480b2150 msgr2=0x7f26480c6320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:36:26.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.429+0000 7f263f7fe640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26480b2150 0x7f26480c6320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.430+0000 7f263f7fe640 1 -- 172.21.15.115:0/1141767256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2648161330 msgr2=0x7f26480b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.430+0000 7f263f7fe640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2648161330 0x7f26480b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.430+0000 7f263f7fe640 1 -- 172.21.15.115:0/1141767256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2640009b30 con 0x7f26480cd400 2023-12-01T01:36:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.428+0000 7f9644e1c640 1 -- 172.21.15.115:0/746390247 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963802cd10 con 0x7f96400d2600 2023-12-01T01:36:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.429+0000 7f9644e1c640 1 -- 172.21.15.115:0/746390247 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963802ce70 con 0x7f96400d2600 2023-12-01T01:36:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.429+0000 7f9644e1c640 1 -- 172.21.15.115:0/746390247 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638034640 con 0x7f96400d2600 2023-12-01T01:36:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.430+0000 7f9648082640 1 -- 172.21.15.115:0/746390247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96400d2600 msgr2=0x7f96400d30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.430+0000 7f9648082640 1 --2- 172.21.15.115:0/746390247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96400d2600 0x7f96400d30b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f96400cd3e0 tx=0x7f963802bf60 comp rx=0 tx=0).stop 2023-12-01T01:36:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.430+0000 7f9648082640 1 -- 172.21.15.115:0/746390247 shutdown_connections 2023-12-01T01:36:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.430+0000 7f9648082640 1 --2- 172.21.15.115:0/746390247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96400d2600 0x7f96400d30b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.430+0000 7f9648082640 1 --2- 172.21.15.115:0/746390247 >> v2:172.21.15.115:3300/0 conn(0x7f96400d21c0 0x7f96400d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.430+0000 7f9648082640 1 -- 172.21.15.115:0/746390247 >> 172.21.15.115:0/746390247 conn(0x7f964006a0b0 msgr2=0x7f964006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.431+0000 7f263f7fe640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26480cd400 0x7f26480c8440 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f263800c3c0 tx=0x7f263800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.431+0000 7f9648082640 1 -- 172.21.15.115:0/746390247 shutdown_connections 2023-12-01T01:36:26.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.431+0000 7f26457fa640 1 -- 172.21.15.115:0/1141767256 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26380052d0 con 0x7f26480cd400 2023-12-01T01:36:26.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.431+0000 7f9648082640 1 -- 172.21.15.115:0/746390247 wait complete. 2023-12-01T01:36:26.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.433+0000 7f9648082640 1 Processor -- start 2023-12-01T01:36:26.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.434+0000 7f9648082640 1 -- start start 2023-12-01T01:36:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.431+0000 7f26457fa640 1 -- 172.21.15.115:0/1141767256 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2638005430 con 0x7f26480cd400 2023-12-01T01:36:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.431+0000 7f26457fa640 1 -- 172.21.15.115:0/1141767256 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26380052d0 con 0x7f26480cd400 2023-12-01T01:36:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.431+0000 7f264e82c640 1 -- 172.21.15.115:0/1141767256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26480b7340 con 0x7f26480cd400 2023-12-01T01:36:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.431+0000 7f264e82c640 1 -- 172.21.15.115:0/1141767256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26480c9fb0 con 0x7f26480cd400 2023-12-01T01:36:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.433+0000 7f264e82c640 1 -- 172.21.15.115:0/1141767256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26480ca7b0 con 0x7f26480cd400 2023-12-01T01:36:26.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.436+0000 7f9648082640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96400d21c0 0x7f96400cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.436+0000 7f9648082640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96400d2600 0x7f9640159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.436+0000 7f9648082640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f964015d0b0 0x7f96400c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.437+0000 7f9648082640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96401f3610 con 0x7f964015d0b0 2023-12-01T01:36:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.437+0000 7f9648082640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400cae40 con 0x7f96400d21c0 2023-12-01T01:36:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.437+0000 7f964561d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96400d2600 0x7f9640159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.437+0000 7f9648082640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96400cb120 con 0x7f96400d2600 2023-12-01T01:36:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.437+0000 7f964561d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96400d2600 0x7f9640159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:34184/0 (socket says 172.21.15.115:34184) 2023-12-01T01:36:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.437+0000 7f964561d640 1 -- 172.21.15.115:0/1719811135 learned_addr learned my addr 172.21.15.115:0/1719811135 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.437+0000 7f9645e1e640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96400d21c0 0x7f96400cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.437+0000 7f964661f640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f964015d0b0 0x7f96400c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.440+0000 7f964661f640 1 -- 172.21.15.115:0/1719811135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96400d2600 msgr2=0x7f9640159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.441+0000 7f964661f640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96400d2600 0x7f9640159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.441+0000 7f964661f640 1 -- 172.21.15.115:0/1719811135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96400d21c0 msgr2=0x7f96400cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.441+0000 7f964661f640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96400d21c0 0x7f96400cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.441+0000 7f964661f640 1 -- 172.21.15.115:0/1719811135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630009400 con 0x7f964015d0b0 2023-12-01T01:36:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.441+0000 7f9645e1e640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96400d21c0 0x7f96400cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.441+0000 7f964561d640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96400d2600 0x7f9640159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:36:26.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.438+0000 7f26457fa640 1 -- 172.21.15.115:0/1141767256 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f26380035f0 con 0x7f26480cd400 2023-12-01T01:36:26.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.440+0000 7f26457fa640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f262806a490 0x7f262806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.440+0000 7f26457fa640 1 -- 172.21.15.115:0/1141767256 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2638087bb0 con 0x7f26480cd400 2023-12-01T01:36:26.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.442+0000 7f2647fff640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f262806a490 0x7f262806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.443+0000 7f2647fff640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f262806a490 0x7f262806c950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2630009930 tx=0x7f2630009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.444+0000 7f964661f640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f964015d0b0 0x7f96400c8610 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9638033000 tx=0x7f9638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.445+0000 7f26457fa640 1 -- 172.21.15.115:0/1141767256 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f26380532c0 con 0x7f26480cd400 2023-12-01T01:36:26.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.445+0000 7f96367fc640 1 -- 172.21.15.115:0/1719811135 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963803e610 con 0x7f964015d0b0 2023-12-01T01:36:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.446+0000 7f96367fc640 1 -- 172.21.15.115:0/1719811135 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963803e770 con 0x7f964015d0b0 2023-12-01T01:36:26.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.446+0000 7f96367fc640 1 -- 172.21.15.115:0/1719811135 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963803eb20 con 0x7f964015d0b0 2023-12-01T01:36:26.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.446+0000 7f9648082640 1 -- 172.21.15.115:0/1719811135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9638009a70 con 0x7f964015d0b0 2023-12-01T01:36:26.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.446+0000 7f9648082640 1 -- 172.21.15.115:0/1719811135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96400cbe60 con 0x7f964015d0b0 2023-12-01T01:36:26.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.449+0000 7f9648082640 1 -- 172.21.15.115:0/1719811135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9628005250 con 0x7f964015d0b0 2023-12-01T01:36:26.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.458+0000 7f96367fc640 1 -- 172.21.15.115:0/1719811135 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f963803b050 con 0x7f964015d0b0 2023-12-01T01:36:26.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.459+0000 7f96367fc640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f962406a500 0x7f962406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:26.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.459+0000 7f96367fc640 1 -- 172.21.15.115:0/1719811135 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96380aa930 con 0x7f964015d0b0 2023-12-01T01:36:26.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.460+0000 7f9645e1e640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f962406a500 0x7f962406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:26.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.462+0000 7f9645e1e640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f962406a500 0x7f962406c9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96400b45a0 tx=0x7f963c009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:26.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.463+0000 7f96367fc640 1 -- 172.21.15.115:0/1719811135 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9638076040 con 0x7f964015d0b0 2023-12-01T01:36:26.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.601+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fce780395f0 con 0x7fce78185250 2023-12-01T01:36:26.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.604+0000 7fce67fff640 1 -- 172.21.15.115:0/3598317632 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fce70057360 con 0x7fce78185250 2023-12-01T01:36:26.608 INFO:teuthology.orchestra.run.smithi115.stdout:55834574857 2023-12-01T01:36:26.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.607+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fce6c06a530 msgr2=0x7fce6c06c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.607+0000 7fce7ffff640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fce6c06a530 0x7fce6c06c9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce680147e0 tx=0x7fce8006c030 comp rx=0 tx=0).stop 2023-12-01T01:36:26.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.607+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce78185250 msgr2=0x7fce780da050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.607+0000 7fce7ffff640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce78185250 0x7fce780da050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c410 tx=0x7fce7000c8d0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.607+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 shutdown_connections 2023-12-01T01:36:26.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.607+0000 7fce7ffff640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fce6c06a530 0x7fce6c06c9f0 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce680147e0 tx=0x7fce8006c030 comp rx=0 tx=0).stop 2023-12-01T01:36:26.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.607+0000 7fce7ffff640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce780da590 0x7fce78008d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.607+0000 7fce7ffff640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce78185250 0x7fce780da050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.607+0000 7fce7ffff640 1 --2- 172.21.15.115:0/3598317632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6800c180 0x7fce781898c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.608+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 >> 172.21.15.115:0/3598317632 conn(0x7fce78056d10 msgr2=0x7fce780db450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.608+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 shutdown_connections 2023-12-01T01:36:26.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.609+0000 7fce7ffff640 1 -- 172.21.15.115:0/3598317632 wait complete. 2023-12-01T01:36:26.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.0 2023-12-01T01:36:26.638 DEBUG:teuthology.parallel:result is None 2023-12-01T01:36:26.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.695+0000 7fad80696640 1 -- 172.21.15.115:0/1024869273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fad780ccdc0 con 0x7fad780b8420 2023-12-01T01:36:26.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.696+0000 7fad6effd640 1 -- 172.21.15.115:0/1024869273 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fad74035070 con 0x7fad780b8420 2023-12-01T01:36:26.698 INFO:teuthology.orchestra.run.smithi115.stdout:55834574855 2023-12-01T01:36:26.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.699+0000 7fad80696640 1 -- 172.21.15.115:0/1024869273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad5c06a480 msgr2=0x7fad5c06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.699+0000 7fad80696640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad5c06a480 0x7fad5c06c940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fad780cfb00 tx=0x7fad68009040 comp rx=0 tx=0).stop 2023-12-01T01:36:26.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.699+0000 7fad80696640 1 -- 172.21.15.115:0/1024869273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad780b8420 msgr2=0x7fad7814c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.699+0000 7fad80696640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad780b8420 0x7fad7814c070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad74000c00 tx=0x7fad740046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.699+0000 7fad80696640 1 -- 172.21.15.115:0/1024869273 shutdown_connections 2023-12-01T01:36:26.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.699+0000 7fad80696640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad5c06a480 0x7fad5c06c940 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fad780cfb00 tx=0x7fad68009040 comp rx=0 tx=0).stop 2023-12-01T01:36:26.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.699+0000 7fad80696640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad780cf430 0x7fad781fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.699+0000 7fad80696640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad7815ccc0 0x7fad781e6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.699+0000 7fad80696640 1 --2- 172.21.15.115:0/1024869273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad780b8420 0x7fad7814c070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.699+0000 7fad80696640 1 -- 172.21.15.115:0/1024869273 >> 172.21.15.115:0/1024869273 conn(0x7fad78065960 msgr2=0x7fad780d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.703+0000 7fad80696640 1 -- 172.21.15.115:0/1024869273 shutdown_connections 2023-12-01T01:36:26.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.703+0000 7fad80696640 1 -- 172.21.15.115:0/1024869273 wait complete. 2023-12-01T01:36:26.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.1 2023-12-01T01:36:26.731 DEBUG:teuthology.parallel:result is None 2023-12-01T01:36:26.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.745+0000 7fc7aab4e640 1 -- 172.21.15.115:0/3471608070 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc7a4144a50 con 0x7fc7a4160370 2023-12-01T01:36:26.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.746+0000 7fc7a17fa640 1 -- 172.21.15.115:0/3471608070 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc7a4144a50 con 0x7fc7a4160370 2023-12-01T01:36:26.747 INFO:teuthology.orchestra.run.smithi115.stdout:55834574858 2023-12-01T01:36:26.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.748+0000 7fc7aab4e640 1 -- 172.21.15.115:0/3471608070 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79006a520 msgr2=0x7fc79006c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.748+0000 7fc7aab4e640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79006a520 0x7fc79006c9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc79c035000 tx=0x7fc79c030000 comp rx=0 tx=0).stop 2023-12-01T01:36:26.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.748+0000 7fc7aab4e640 1 -- 172.21.15.115:0/3471608070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a4160370 msgr2=0x7fc7a4154530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.748+0000 7fc7aab4e640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a4160370 0x7fc7a4154530 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc798009fd0 tx=0x7fc79800b8e0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.748+0000 7fc7aab4e640 1 -- 172.21.15.115:0/3471608070 shutdown_connections 2023-12-01T01:36:26.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.748+0000 7fc7aab4e640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79006a520 0x7fc79006c9e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.748+0000 7fc7aab4e640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a4160370 0x7fc7a4154530 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.748+0000 7fc7aab4e640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7a40cf740 0x7fc7a4153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.748+0000 7fc7aab4e640 1 --2- 172.21.15.115:0/3471608070 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a40b6090 0x7fc7a4153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.748+0000 7fc7aab4e640 1 -- 172.21.15.115:0/3471608070 >> 172.21.15.115:0/3471608070 conn(0x7fc7a4065960 msgr2=0x7fc7a415f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.749+0000 7fc7aab4e640 1 -- 172.21.15.115:0/3471608070 shutdown_connections 2023-12-01T01:36:26.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.749+0000 7fc7aab4e640 1 -- 172.21.15.115:0/3471608070 wait complete. 2023-12-01T01:36:26.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.764+0000 7fe289573640 1 -- 172.21.15.115:0/2413615888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe2840cc830 con 0x7fe284158c10 2023-12-01T01:36:26.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.6 2023-12-01T01:36:26.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.765+0000 7fe27b7fe640 1 -- 172.21.15.115:0/2413615888 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe274057320 con 0x7fe284158c10 2023-12-01T01:36:26.767 INFO:teuthology.orchestra.run.smithi115.stdout:55834574856 2023-12-01T01:36:26.767 DEBUG:teuthology.parallel:result is None 2023-12-01T01:36:26.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 -- 172.21.15.115:0/2413615888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe26406a480 msgr2=0x7fe26406c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe26406a480 0x7fe26406c940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe27c037000 tx=0x7fe27c008040 comp rx=0 tx=0).stop 2023-12-01T01:36:26.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 -- 172.21.15.115:0/2413615888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe284158c10 msgr2=0x7fe2841f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe284158c10 0x7fe2841f2d20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe27400c7a0 tx=0x7fe27400cc60 comp rx=0 tx=0).stop 2023-12-01T01:36:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 -- 172.21.15.115:0/2413615888 shutdown_connections 2023-12-01T01:36:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe26406a480 0x7fe26406c940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe28415c790 0x7fe2841f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe284158c10 0x7fe2841f2d20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 --2- 172.21.15.115:0/2413615888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2840ad6e0 0x7fe284147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 -- 172.21.15.115:0/2413615888 >> 172.21.15.115:0/2413615888 conn(0x7fe284065d50 msgr2=0x7fe2840af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 -- 172.21.15.115:0/2413615888 shutdown_connections 2023-12-01T01:36:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.767+0000 7fe289573640 1 -- 172.21.15.115:0/2413615888 wait complete. 2023-12-01T01:36:26.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.2 2023-12-01T01:36:26.784 DEBUG:teuthology.parallel:result is None 2023-12-01T01:36:26.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.784+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f77e004e890 con 0x7f77e0161330 2023-12-01T01:36:26.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.785+0000 7f77d77fe640 1 -- 172.21.15.115:0/2010166585 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77d807a1a0 con 0x7f77e0161330 2023-12-01T01:36:26.786 INFO:teuthology.orchestra.run.smithi115.stdout:55834574857 2023-12-01T01:36:26.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77c006a430 msgr2=0x7f77c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77c006a430 0x7f77c006c8f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f77c80011e0 tx=0x7f77c8009040 comp rx=0 tx=0).stop 2023-12-01T01:36:26.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e0161330 msgr2=0x7f77e00d2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e0161330 0x7f77e00d2db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f77d8037040 tx=0x7f77d8004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:26.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 shutdown_connections 2023-12-01T01:36:26.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77c006a430 0x7f77c006c8f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e0161330 0x7f77e00d2db0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77e00cdb10 0x7f77e00b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 --2- 172.21.15.115:0/2010166585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77e00b2150 0x7f77e015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 >> 172.21.15.115:0/2010166585 conn(0x7f77e0065960 msgr2=0x7f77e00cd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 shutdown_connections 2023-12-01T01:36:26.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.787+0000 7f77e48c4640 1 -- 172.21.15.115:0/2010166585 wait complete. 2023-12-01T01:36:26.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.791+0000 7f48fbc21640 1 -- 172.21.15.115:0/429488864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f48f40c8b10 con 0x7f48f40ad6e0 2023-12-01T01:36:26.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.792+0000 7f48f27fc640 1 -- 172.21.15.115:0/429488864 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f48e8054470 con 0x7f48f40ad6e0 2023-12-01T01:36:26.793 INFO:teuthology.orchestra.run.smithi115.stdout:55834574857 2023-12-01T01:36:26.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.794+0000 7f48fbc21640 1 -- 172.21.15.115:0/429488864 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f48d806a430 msgr2=0x7f48d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.794+0000 7f48fbc21640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f48d806a430 0x7f48d806c8f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f48f41514f0 tx=0x7f48e000a000 comp rx=0 tx=0).stop 2023-12-01T01:36:26.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.794+0000 7f48fbc21640 1 -- 172.21.15.115:0/429488864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f40ad6e0 msgr2=0x7f48f406fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.794+0000 7f48fbc21640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f40ad6e0 0x7f48f406fac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48e800caa0 tx=0x7f48e800cf60 comp rx=0 tx=0).stop 2023-12-01T01:36:26.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.794+0000 7f48fbc21640 1 -- 172.21.15.115:0/429488864 shutdown_connections 2023-12-01T01:36:26.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.794+0000 7f48fbc21640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f48d806a430 0x7f48d806c8f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.794+0000 7f48fbc21640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40ceb30 0x7f48f4149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.794+0000 7f48fbc21640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f4158c10 0x7f48f40af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.794+0000 7f48fbc21640 1 --2- 172.21.15.115:0/429488864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48f40ad6e0 0x7f48f406fac0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.794+0000 7f48fbc21640 1 -- 172.21.15.115:0/429488864 >> 172.21.15.115:0/429488864 conn(0x7f48f4065d50 msgr2=0x7f48f40ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.795+0000 7f48fbc21640 1 -- 172.21.15.115:0/429488864 shutdown_connections 2023-12-01T01:36:26.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.795+0000 7f48fbc21640 1 -- 172.21.15.115:0/429488864 wait complete. 2023-12-01T01:36:26.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.5 2023-12-01T01:36:26.805 DEBUG:teuthology.parallel:result is None 2023-12-01T01:36:26.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.7 2023-12-01T01:36:26.812 DEBUG:teuthology.parallel:result is None 2023-12-01T01:36:26.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.839+0000 7f264e82c640 1 -- 172.21.15.115:0/1141767256 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f26480ca7b0 con 0x7f26480cd400 2023-12-01T01:36:26.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.840+0000 7f26457fa640 1 -- 172.21.15.115:0/1141767256 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f26480ca7b0 con 0x7f26480cd400 2023-12-01T01:36:26.841 INFO:teuthology.orchestra.run.smithi115.stdout:55834574856 2023-12-01T01:36:26.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.842+0000 7f264e82c640 1 -- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f262806a490 msgr2=0x7f262806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.842+0000 7f264e82c640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f262806a490 0x7f262806c950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2630009930 tx=0x7f2630009190 comp rx=0 tx=0).stop 2023-12-01T01:36:26.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.842+0000 7f264e82c640 1 -- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26480cd400 msgr2=0x7f26480c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.842+0000 7f264e82c640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26480cd400 0x7f26480c8440 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f263800c3c0 tx=0x7f263800c880 comp rx=0 tx=0).stop 2023-12-01T01:36:26.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.843+0000 7f264e82c640 1 -- 172.21.15.115:0/1141767256 shutdown_connections 2023-12-01T01:36:26.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.843+0000 7f264e82c640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f262806a490 0x7f262806c950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.843+0000 7f264e82c640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2648161330 0x7f26480b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.843+0000 7f264e82c640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26480cd400 0x7f26480c8440 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.843+0000 7f264e82c640 1 --2- 172.21.15.115:0/1141767256 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26480b2150 0x7f26480c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.843+0000 7f264e82c640 1 -- 172.21.15.115:0/1141767256 >> 172.21.15.115:0/1141767256 conn(0x7f2648065960 msgr2=0x7f2648161930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.843+0000 7f264e82c640 1 -- 172.21.15.115:0/1141767256 shutdown_connections 2023-12-01T01:36:26.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.843+0000 7f264e82c640 1 -- 172.21.15.115:0/1141767256 wait complete. 2023-12-01T01:36:26.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.855+0000 7f9648082640 1 -- 172.21.15.115:0/1719811135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9628005500 con 0x7f964015d0b0 2023-12-01T01:36:26.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.855+0000 7f96367fc640 1 -- 172.21.15.115:0/1719811135 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f963804d020 con 0x7f964015d0b0 2023-12-01T01:36:26.857 INFO:teuthology.orchestra.run.smithi115.stdout:55834574857 2023-12-01T01:36:26.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 -- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f962406a500 msgr2=0x7f962406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f962406a500 0x7f962406c9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96400b45a0 tx=0x7f963c009060 comp rx=0 tx=0).stop 2023-12-01T01:36:26.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 -- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f964015d0b0 msgr2=0x7f96400c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:26.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f964015d0b0 0x7f96400c8610 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9638033000 tx=0x7f9638004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:26.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 -- 172.21.15.115:0/1719811135 shutdown_connections 2023-12-01T01:36:26.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f962406a500 0x7f962406c9c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96400d2600 0x7f9640159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f964015d0b0 0x7f96400c8610 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 --2- 172.21.15.115:0/1719811135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96400d21c0 0x7f96400cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:26.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 -- 172.21.15.115:0/1719811135 >> 172.21.15.115:0/1719811135 conn(0x7f964006a0b0 msgr2=0x7f964015aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:26.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.857+0000 7f9648082640 1 -- 172.21.15.115:0/1719811135 shutdown_connections 2023-12-01T01:36:26.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:26.858+0000 7f9648082640 1 -- 172.21.15.115:0/1719811135 wait complete. 2023-12-01T01:36:26.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.3 2023-12-01T01:36:26.861 DEBUG:teuthology.parallel:result is None 2023-12-01T01:36:26.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.4 2023-12-01T01:36:26.876 DEBUG:teuthology.parallel:result is None 2023-12-01T01:36:26.876 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-01T01:36:26.876 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:36:27.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.124+0000 7f0507d27640 1 Processor -- start 2023-12-01T01:36:27.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.124+0000 7f0507d27640 1 -- start start 2023-12-01T01:36:27.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.125+0000 7f0507d27640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f050015e2f0 0x7f05000ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.125+0000 7f0507d27640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f05000adf10 0x7f05000caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.125+0000 7f0507d27640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05000add10 con 0x7f050006f2f0 2023-12-01T01:36:27.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.125+0000 7f0507d27640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05000c5760 con 0x7f05000adf10 2023-12-01T01:36:27.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.125+0000 7f0507d27640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050015e6d0 con 0x7f050015e2f0 2023-12-01T01:36:27.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.125+0000 7f0505ac3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f050015e2f0 0x7f05000ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:27.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.126+0000 7f0505ac3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f050015e2f0 0x7f05000ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:34188/0 (socket says 172.21.15.115:34188) 2023-12-01T01:36:27.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.126+0000 7f0505ac3640 1 -- 172.21.15.115:0/2959398805 learned_addr learned my addr 172.21.15.115:0/2959398805 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:27.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.126+0000 7f05062c4640 1 --2- 172.21.15.115:0/2959398805 >> v2:172.21.15.142:3300/0 conn(0x7f05000adf10 0x7f05000caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:27.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.126+0000 7f0505ac3640 1 -- 172.21.15.115:0/2959398805 >> v2:172.21.15.142:3300/0 conn(0x7f05000adf10 msgr2=0x7f05000caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:27.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.126+0000 7f0505ac3640 1 --2- 172.21.15.115:0/2959398805 >> v2:172.21.15.142:3300/0 conn(0x7f05000adf10 0x7f05000caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.126+0000 7f0505ac3640 1 -- 172.21.15.115:0/2959398805 >> v1:172.21.15.142:6789/0 conn(0x7f050006f2f0 legacy=0x7f05000c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.126+0000 7f0505ac3640 1 -- 172.21.15.115:0/2959398805 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05000cf7b0 con 0x7f050015e2f0 2023-12-01T01:36:27.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.128+0000 7f0505ac3640 1 --2- 172.21.15.115:0/2959398805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f050015e2f0 0x7f05000ca660 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009870 tx=0x7f04fc02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7bb6d28650d070b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:27.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0504ac1640 1 -- 172.21.15.115:0/2959398805 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0317c0 con 0x7f050015e2f0 2023-12-01T01:36:27.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0504ac1640 1 -- 172.21.15.115:0/2959398805 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc031920 con 0x7f050015e2f0 2023-12-01T01:36:27.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0504ac1640 1 -- 172.21.15.115:0/2959398805 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc031cb0 con 0x7f050015e2f0 2023-12-01T01:36:27.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0507d27640 1 -- 172.21.15.115:0/2959398805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f050015e2f0 msgr2=0x7f05000ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0507d27640 1 --2- 172.21.15.115:0/2959398805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f050015e2f0 0x7f05000ca660 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009870 tx=0x7f04fc02b860 comp rx=0 tx=0).stop 2023-12-01T01:36:27.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0507d27640 1 -- 172.21.15.115:0/2959398805 shutdown_connections 2023-12-01T01:36:27.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0507d27640 1 --2- 172.21.15.115:0/2959398805 >> v2:172.21.15.142:3300/0 conn(0x7f05000adf10 0x7f05000caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0507d27640 1 --2- 172.21.15.115:0/2959398805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f050015e2f0 0x7f05000ca660 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0507d27640 1 -- 172.21.15.115:0/2959398805 >> 172.21.15.115:0/2959398805 conn(0x7f05000ce880 msgr2=0x7f05000cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:27.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0507d27640 1 -- 172.21.15.115:0/2959398805 shutdown_connections 2023-12-01T01:36:27.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.129+0000 7f0507d27640 1 -- 172.21.15.115:0/2959398805 wait complete. 2023-12-01T01:36:27.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.131+0000 7f0507d27640 1 Processor -- start 2023-12-01T01:36:27.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.131+0000 7f0507d27640 1 -- start start 2023-12-01T01:36:27.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.132+0000 7f0507d27640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f050006f2f0 0x7f050015d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.132+0000 7f0507d27640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05000adf10 0x7f05000af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.132+0000 7f0507d27640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f050015e2f0 0x7f05000af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.132+0000 7f0507d27640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050015db00 con 0x7f050006f2f0 2023-12-01T01:36:27.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.132+0000 7f0507d27640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0500158740 con 0x7f050015e2f0 2023-12-01T01:36:27.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.132+0000 7f0507d27640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05001589e0 con 0x7f05000adf10 2023-12-01T01:36:27.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.132+0000 7f0505ac3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f050006f2f0 0x7f050015d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:27.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.132+0000 7f0505ac3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f050006f2f0 0x7f050015d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56292/0 (socket says 172.21.15.115:56292) 2023-12-01T01:36:27.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.132+0000 7f0505ac3640 1 -- 172.21.15.115:0/572055406 learned_addr learned my addr 172.21.15.115:0/572055406 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f05062c4640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f050015e2f0 0x7f05000af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f05052c2640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05000adf10 0x7f05000af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f0505ac3640 1 -- 172.21.15.115:0/572055406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05000adf10 msgr2=0x7f05000af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f0505ac3640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05000adf10 0x7f05000af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f0505ac3640 1 -- 172.21.15.115:0/572055406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f050015e2f0 msgr2=0x7f05000af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f0505ac3640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f050015e2f0 0x7f05000af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f0505ac3640 1 -- 172.21.15.115:0/572055406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc0094d0 con 0x7f050006f2f0 2023-12-01T01:36:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f05052c2640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05000adf10 0x7f05000af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f0505ac3640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f050006f2f0 0x7f050015d2d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04fc02b8a0 tx=0x7f04fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f04f67fc640 1 -- 172.21.15.115:0/572055406 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc032860 con 0x7f050006f2f0 2023-12-01T01:36:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f04f67fc640 1 -- 172.21.15.115:0/572055406 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc032a00 con 0x7f050006f2f0 2023-12-01T01:36:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f04f67fc640 1 -- 172.21.15.115:0/572055406 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc032db0 con 0x7f050006f2f0 2023-12-01T01:36:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.133+0000 7f0507d27640 1 -- 172.21.15.115:0/572055406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0500158cc0 con 0x7f050006f2f0 2023-12-01T01:36:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.134+0000 7f0507d27640 1 -- 172.21.15.115:0/572055406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050015d810 con 0x7f050006f2f0 2023-12-01T01:36:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.135+0000 7f0507d27640 1 -- 172.21.15.115:0/572055406 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050006fda0 con 0x7f050006f2f0 2023-12-01T01:36:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.135+0000 7f04f67fc640 1 -- 172.21.15.115:0/572055406 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f04fc04e450 con 0x7f050006f2f0 2023-12-01T01:36:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.136+0000 7f04f67fc640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f04e406a430 0x7f04e406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.136+0000 7f04f67fc640 1 -- 172.21.15.115:0/572055406 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04fc0aa9b0 con 0x7f050006f2f0 2023-12-01T01:36:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.136+0000 7f05052c2640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f04e406a430 0x7f04e406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.137+0000 7f05052c2640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f04e406a430 0x7f04e406c8f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f050015b810 tx=0x7f04f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:27.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.140+0000 7f04f67fc640 1 -- 172.21.15.115:0/572055406 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f04fc0760c0 con 0x7f050006f2f0 2023-12-01T01:36:27.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.316+0000 7f0507d27640 1 -- 172.21.15.115:0/572055406 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05000b43d0 con 0x7f04e406a430 2023-12-01T01:36:27.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.337+0000 7f04f67fc640 1 -- 172.21.15.115:0/572055406 <== mgr.4098 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264287 (secure 0 0 0) 0x7f04f0001c90 con 0x7f04e406a430 2023-12-01T01:36:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:27.342 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 -- 172.21.15.115:0/572055406 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f04e406a430 msgr2=0x7f04e406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f04e406a430 0x7f04e406c8f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f050015b810 tx=0x7f04f0009040 comp rx=0 tx=0).stop 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 -- 172.21.15.115:0/572055406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f050006f2f0 msgr2=0x7f050015d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f050006f2f0 0x7f050015d2d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04fc02b8a0 tx=0x7f04fc0046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 -- 172.21.15.115:0/572055406 shutdown_connections 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f04e406a430 0x7f04e406c8f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05000adf10 0x7f05000af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f050015e2f0 0x7f05000af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 --2- 172.21.15.115:0/572055406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f050006f2f0 0x7f050015d2d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 -- 172.21.15.115:0/572055406 >> 172.21.15.115:0/572055406 conn(0x7f05000ce880 msgr2=0x7f05000cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:27.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 -- 172.21.15.115:0/572055406 shutdown_connections 2023-12-01T01:36:27.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.343+0000 7f0507d27640 1 -- 172.21.15.115:0/572055406 wait complete. 2023-12-01T01:36:27.362 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-01T01:36:26.161041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218536,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756184,"statfs":{"total":773094113280,"available":772870332416,"internally_reserved":0,"allocated":2990080,"data_stored":1790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004199"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847556+0000","last_change":"2023-12-01T01:36:10.681289+0000","last_active":"2023-12-01T01:36:11.847556+0000","last_peered":"2023-12-01T01:36:11.847556+0000","last_clean":"2023-12-01T01:36:11.847556+0000","last_became_active":"2023-12-01T01:36:10.680689+0000","last_became_peered":"2023-12-01T01:36:10.680689+0000","last_unstale":"2023-12-01T01:36:11.847556+0000","last_undegraded":"2023-12-01T01:36:11.847556+0000","last_fullsized":"2023-12-01T01:36:11.847556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:12:29.670579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844746+0000","last_change":"2023-12-01T01:36:10.668026+0000","last_active":"2023-12-01T01:36:11.844746+0000","last_peered":"2023-12-01T01:36:11.844746+0000","last_clean":"2023-12-01T01:36:11.844746+0000","last_became_active":"2023-12-01T01:36:10.667717+0000","last_became_peered":"2023-12-01T01:36:10.667717+0000","last_unstale":"2023-12-01T01:36:11.844746+0000","last_undegraded":"2023-12-01T01:36:11.844746+0000","last_fullsized":"2023-12-01T01:36:11.844746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:12:29.670579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844579+0000","last_change":"2023-12-01T01:36:10.665172+0000","last_active":"2023-12-01T01:36:11.844579+0000","last_peered":"2023-12-01T01:36:11.844579+0000","last_clean":"2023-12-01T01:36:11.844579+0000","last_became_active":"2023-12-01T01:36:10.664880+0000","last_became_peered":"2023-12-01T01:36:10.664880+0000","last_unstale":"2023-12-01T01:36:11.844579+0000","last_undegraded":"2023-12-01T01:36:11.844579+0000","last_fullsized":"2023-12-01T01:36:11.844579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:41:51.516524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843255+0000","last_change":"2023-12-01T01:36:10.666933+0000","last_active":"2023-12-01T01:36:11.843255+0000","last_peered":"2023-12-01T01:36:11.843255+0000","last_clean":"2023-12-01T01:36:11.843255+0000","last_became_active":"2023-12-01T01:36:10.666669+0000","last_became_peered":"2023-12-01T01:36:10.666669+0000","last_unstale":"2023-12-01T01:36:11.843255+0000","last_undegraded":"2023-12-01T01:36:11.843255+0000","last_fullsized":"2023-12-01T01:36:11.843255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846486+0000","last_change":"2023-12-01T01:36:10.660927+0000","last_active":"2023-12-01T01:36:11.846486+0000","last_peered":"2023-12-01T01:36:11.846486+0000","last_clean":"2023-12-01T01:36:11.846486+0000","last_became_active":"2023-12-01T01:36:10.660693+0000","last_became_peered":"2023-12-01T01:36:10.660693+0000","last_unstale":"2023-12-01T01:36:11.846486+0000","last_undegraded":"2023-12-01T01:36:11.846486+0000","last_fullsized":"2023-12-01T01:36:11.846486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:02.974458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842420+0000","last_change":"2023-12-01T01:36:10.670715+0000","last_active":"2023-12-01T01:36:11.842420+0000","last_peered":"2023-12-01T01:36:11.842420+0000","last_clean":"2023-12-01T01:36:11.842420+0000","last_became_active":"2023-12-01T01:36:10.669975+0000","last_became_peered":"2023-12-01T01:36:10.669975+0000","last_unstale":"2023-12-01T01:36:11.842420+0000","last_undegraded":"2023-12-01T01:36:11.842420+0000","last_fullsized":"2023-12-01T01:36:11.842420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845913+0000","last_change":"2023-12-01T01:36:10.670567+0000","last_active":"2023-12-01T01:36:11.845913+0000","last_peered":"2023-12-01T01:36:11.845913+0000","last_clean":"2023-12-01T01:36:11.845913+0000","last_became_active":"2023-12-01T01:36:10.670325+0000","last_became_peered":"2023-12-01T01:36:10.670325+0000","last_unstale":"2023-12-01T01:36:11.845913+0000","last_undegraded":"2023-12-01T01:36:11.845913+0000","last_fullsized":"2023-12-01T01:36:11.845913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:49.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843702+0000","last_change":"2023-12-01T01:36:10.672309+0000","last_active":"2023-12-01T01:36:11.843702+0000","last_peered":"2023-12-01T01:36:11.843702+0000","last_clean":"2023-12-01T01:36:11.843702+0000","last_became_active":"2023-12-01T01:36:10.671949+0000","last_became_peered":"2023-12-01T01:36:10.671949+0000","last_unstale":"2023-12-01T01:36:11.843702+0000","last_undegraded":"2023-12-01T01:36:11.843702+0000","last_fullsized":"2023-12-01T01:36:11.843702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:44:20.605895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844629+0000","last_change":"2023-12-01T01:36:10.664390+0000","last_active":"2023-12-01T01:36:11.844629+0000","last_peered":"2023-12-01T01:36:11.844629+0000","last_clean":"2023-12-01T01:36:11.844629+0000","last_became_active":"2023-12-01T01:36:10.663998+0000","last_became_peered":"2023-12-01T01:36:10.663998+0000","last_unstale":"2023-12-01T01:36:11.844629+0000","last_undegraded":"2023-12-01T01:36:11.844629+0000","last_fullsized":"2023-12-01T01:36:11.844629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847720+0000","last_change":"2023-12-01T01:36:10.662705+0000","last_active":"2023-12-01T01:36:11.847720+0000","last_peered":"2023-12-01T01:36:11.847720+0000","last_clean":"2023-12-01T01:36:11.847720+0000","last_became_active":"2023-12-01T01:36:10.662424+0000","last_became_peered":"2023-12-01T01:36:10.662424+0000","last_unstale":"2023-12-01T01:36:11.847720+0000","last_undegraded":"2023-12-01T01:36:11.847720+0000","last_fullsized":"2023-12-01T01:36:11.847720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:44:55.685435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846761+0000","last_change":"2023-12-01T01:36:10.669520+0000","last_active":"2023-12-01T01:36:11.846761+0000","last_peered":"2023-12-01T01:36:11.846761+0000","last_clean":"2023-12-01T01:36:11.846761+0000","last_became_active":"2023-12-01T01:36:10.665644+0000","last_became_peered":"2023-12-01T01:36:10.665644+0000","last_unstale":"2023-12-01T01:36:11.846761+0000","last_undegraded":"2023-12-01T01:36:11.846761+0000","last_fullsized":"2023-12-01T01:36:11.846761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:38.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846407+0000","last_change":"2023-12-01T01:36:10.668672+0000","last_active":"2023-12-01T01:36:11.846407+0000","last_peered":"2023-12-01T01:36:11.846407+0000","last_clean":"2023-12-01T01:36:11.846407+0000","last_became_active":"2023-12-01T01:36:10.668180+0000","last_became_peered":"2023-12-01T01:36:10.668180+0000","last_unstale":"2023-12-01T01:36:11.846407+0000","last_undegraded":"2023-12-01T01:36:11.846407+0000","last_fullsized":"2023-12-01T01:36:11.846407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:33.765782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843462+0000","last_change":"2023-12-01T01:36:10.665068+0000","last_active":"2023-12-01T01:36:11.843462+0000","last_peered":"2023-12-01T01:36:11.843462+0000","last_clean":"2023-12-01T01:36:11.843462+0000","last_became_active":"2023-12-01T01:36:10.664845+0000","last_became_peered":"2023-12-01T01:36:10.664845+0000","last_unstale":"2023-12-01T01:36:11.843462+0000","last_undegraded":"2023-12-01T01:36:11.843462+0000","last_fullsized":"2023-12-01T01:36:11.843462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:27.561648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843064+0000","last_change":"2023-12-01T01:36:10.682708+0000","last_active":"2023-12-01T01:36:11.843064+0000","last_peered":"2023-12-01T01:36:11.843064+0000","last_clean":"2023-12-01T01:36:11.843064+0000","last_became_active":"2023-12-01T01:36:10.679209+0000","last_became_peered":"2023-12-01T01:36:10.679209+0000","last_unstale":"2023-12-01T01:36:11.843064+0000","last_undegraded":"2023-12-01T01:36:11.843064+0000","last_fullsized":"2023-12-01T01:36:11.843064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:25.357183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843449+0000","last_change":"2023-12-01T01:36:10.673333+0000","last_active":"2023-12-01T01:36:11.843449+0000","last_peered":"2023-12-01T01:36:11.843449+0000","last_clean":"2023-12-01T01:36:11.843449+0000","last_became_active":"2023-12-01T01:36:10.673021+0000","last_became_peered":"2023-12-01T01:36:10.673021+0000","last_unstale":"2023-12-01T01:36:11.843449+0000","last_undegraded":"2023-12-01T01:36:11.843449+0000","last_fullsized":"2023-12-01T01:36:11.843449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:24.066059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659010+0000","last_change":"2023-12-01T01:36:10.668827+0000","last_active":"2023-12-01T01:36:11.659010+0000","last_peered":"2023-12-01T01:36:11.659010+0000","last_clean":"2023-12-01T01:36:11.659010+0000","last_became_active":"2023-12-01T01:36:10.668507+0000","last_became_peered":"2023-12-01T01:36:10.668507+0000","last_unstale":"2023-12-01T01:36:11.659010+0000","last_undegraded":"2023-12-01T01:36:11.659010+0000","last_fullsized":"2023-12-01T01:36:11.659010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:29:32.707369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660612+0000","last_change":"2023-12-01T01:36:10.681970+0000","last_active":"2023-12-01T01:36:11.660612+0000","last_peered":"2023-12-01T01:36:11.660612+0000","last_clean":"2023-12-01T01:36:11.660612+0000","last_became_active":"2023-12-01T01:36:10.681726+0000","last_became_peered":"2023-12-01T01:36:10.681726+0000","last_unstale":"2023-12-01T01:36:11.660612+0000","last_undegraded":"2023-12-01T01:36:11.660612+0000","last_fullsized":"2023-12-01T01:36:11.660612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:58.572550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659519+0000","last_change":"2023-12-01T01:36:10.685524+0000","last_active":"2023-12-01T01:36:11.659519+0000","last_peered":"2023-12-01T01:36:11.659519+0000","last_clean":"2023-12-01T01:36:11.659519+0000","last_became_active":"2023-12-01T01:36:10.685227+0000","last_became_peered":"2023-12-01T01:36:10.685227+0000","last_unstale":"2023-12-01T01:36:11.659519+0000","last_undegraded":"2023-12-01T01:36:11.659519+0000","last_fullsized":"2023-12-01T01:36:11.659519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660039+0000","last_change":"2023-12-01T01:36:10.673805+0000","last_active":"2023-12-01T01:36:11.660039+0000","last_peered":"2023-12-01T01:36:11.660039+0000","last_clean":"2023-12-01T01:36:11.660039+0000","last_became_active":"2023-12-01T01:36:10.672297+0000","last_became_peered":"2023-12-01T01:36:10.672297+0000","last_unstale":"2023-12-01T01:36:11.660039+0000","last_undegraded":"2023-12-01T01:36:11.660039+0000","last_fullsized":"2023-12-01T01:36:11.660039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846363+0000","last_change":"2023-12-01T01:36:10.668469+0000","last_active":"2023-12-01T01:36:11.846363+0000","last_peered":"2023-12-01T01:36:11.846363+0000","last_clean":"2023-12-01T01:36:11.846363+0000","last_became_active":"2023-12-01T01:36:10.667900+0000","last_became_peered":"2023-12-01T01:36:10.667900+0000","last_unstale":"2023-12-01T01:36:11.846363+0000","last_undegraded":"2023-12-01T01:36:11.846363+0000","last_fullsized":"2023-12-01T01:36:11.846363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660756+0000","last_change":"2023-12-01T01:36:10.666824+0000","last_active":"2023-12-01T01:36:11.660756+0000","last_peered":"2023-12-01T01:36:11.660756+0000","last_clean":"2023-12-01T01:36:11.660756+0000","last_became_active":"2023-12-01T01:36:10.666614+0000","last_became_peered":"2023-12-01T01:36:10.666614+0000","last_unstale":"2023-12-01T01:36:11.660756+0000","last_undegraded":"2023-12-01T01:36:11.660756+0000","last_fullsized":"2023-12-01T01:36:11.660756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:02.974458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843911+0000","last_change":"2023-12-01T01:36:10.676262+0000","last_active":"2023-12-01T01:36:11.843911+0000","last_peered":"2023-12-01T01:36:11.843911+0000","last_clean":"2023-12-01T01:36:11.843911+0000","last_became_active":"2023-12-01T01:36:10.674999+0000","last_became_peered":"2023-12-01T01:36:10.674999+0000","last_unstale":"2023-12-01T01:36:11.843911+0000","last_undegraded":"2023-12-01T01:36:11.843911+0000","last_fullsized":"2023-12-01T01:36:11.843911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843925+0000","last_change":"2023-12-01T01:36:10.660058+0000","last_active":"2023-12-01T01:36:11.843925+0000","last_peered":"2023-12-01T01:36:11.843925+0000","last_clean":"2023-12-01T01:36:11.843925+0000","last_became_active":"2023-12-01T01:36:10.659667+0000","last_became_peered":"2023-12-01T01:36:10.659667+0000","last_unstale":"2023-12-01T01:36:11.843925+0000","last_undegraded":"2023-12-01T01:36:11.843925+0000","last_fullsized":"2023-12-01T01:36:11.843925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659022+0000","last_change":"2023-12-01T01:36:10.681619+0000","last_active":"2023-12-01T01:36:11.659022+0000","last_peered":"2023-12-01T01:36:11.659022+0000","last_clean":"2023-12-01T01:36:11.659022+0000","last_became_active":"2023-12-01T01:36:10.681370+0000","last_became_peered":"2023-12-01T01:36:10.681370+0000","last_unstale":"2023-12-01T01:36:11.659022+0000","last_undegraded":"2023-12-01T01:36:11.659022+0000","last_fullsized":"2023-12-01T01:36:11.659022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:05:04.661902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844932+0000","last_change":"2023-12-01T01:36:10.663979+0000","last_active":"2023-12-01T01:36:11.844932+0000","last_peered":"2023-12-01T01:36:11.844932+0000","last_clean":"2023-12-01T01:36:11.844932+0000","last_became_active":"2023-12-01T01:36:10.663730+0000","last_became_peered":"2023-12-01T01:36:10.663730+0000","last_unstale":"2023-12-01T01:36:11.844932+0000","last_undegraded":"2023-12-01T01:36:11.844932+0000","last_fullsized":"2023-12-01T01:36:11.844932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:44:55.685435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843066+0000","last_change":"2023-12-01T01:36:10.681889+0000","last_active":"2023-12-01T01:36:11.843066+0000","last_peered":"2023-12-01T01:36:11.843066+0000","last_clean":"2023-12-01T01:36:11.843066+0000","last_became_active":"2023-12-01T01:36:10.681607+0000","last_became_peered":"2023-12-01T01:36:10.681607+0000","last_unstale":"2023-12-01T01:36:11.843066+0000","last_undegraded":"2023-12-01T01:36:11.843066+0000","last_fullsized":"2023-12-01T01:36:11.843066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:10.875571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844651+0000","last_change":"2023-12-01T01:36:10.681120+0000","last_active":"2023-12-01T01:36:11.844651+0000","last_peered":"2023-12-01T01:36:11.844651+0000","last_clean":"2023-12-01T01:36:11.844651+0000","last_became_active":"2023-12-01T01:36:10.680861+0000","last_became_peered":"2023-12-01T01:36:10.680861+0000","last_unstale":"2023-12-01T01:36:11.844651+0000","last_undegraded":"2023-12-01T01:36:11.844651+0000","last_fullsized":"2023-12-01T01:36:11.844651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:14:40.275119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843591+0000","last_change":"2023-12-01T01:36:10.671802+0000","last_active":"2023-12-01T01:36:11.843591+0000","last_peered":"2023-12-01T01:36:11.843591+0000","last_clean":"2023-12-01T01:36:11.843591+0000","last_became_active":"2023-12-01T01:36:10.671492+0000","last_became_peered":"2023-12-01T01:36:10.671492+0000","last_unstale":"2023-12-01T01:36:11.843591+0000","last_undegraded":"2023-12-01T01:36:11.843591+0000","last_fullsized":"2023-12-01T01:36:11.843591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:13:49.007903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844344+0000","last_change":"2023-12-01T01:36:10.679590+0000","last_active":"2023-12-01T01:36:11.844344+0000","last_peered":"2023-12-01T01:36:11.844344+0000","last_clean":"2023-12-01T01:36:11.844344+0000","last_became_active":"2023-12-01T01:36:10.679306+0000","last_became_peered":"2023-12-01T01:36:10.679306+0000","last_unstale":"2023-12-01T01:36:11.844344+0000","last_undegraded":"2023-12-01T01:36:11.844344+0000","last_fullsized":"2023-12-01T01:36:11.844344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659556+0000","last_change":"2023-12-01T01:36:10.671990+0000","last_active":"2023-12-01T01:36:11.659556+0000","last_peered":"2023-12-01T01:36:11.659556+0000","last_clean":"2023-12-01T01:36:11.659556+0000","last_became_active":"2023-12-01T01:36:10.666648+0000","last_became_peered":"2023-12-01T01:36:10.666648+0000","last_unstale":"2023-12-01T01:36:11.659556+0000","last_undegraded":"2023-12-01T01:36:11.659556+0000","last_fullsized":"2023-12-01T01:36:11.659556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:51:53.546128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.658760+0000","last_change":"2023-12-01T01:36:10.677811+0000","last_active":"2023-12-01T01:36:11.658760+0000","last_peered":"2023-12-01T01:36:11.658760+0000","last_clean":"2023-12-01T01:36:11.658760+0000","last_became_active":"2023-12-01T01:36:10.677266+0000","last_became_peered":"2023-12-01T01:36:10.677266+0000","last_unstale":"2023-12-01T01:36:11.658760+0000","last_undegraded":"2023-12-01T01:36:11.658760+0000","last_fullsized":"2023-12-01T01:36:11.658760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842525+0000","last_change":"2023-12-01T01:36:10.664264+0000","last_active":"2023-12-01T01:36:11.842525+0000","last_peered":"2023-12-01T01:36:11.842525+0000","last_clean":"2023-12-01T01:36:11.842525+0000","last_became_active":"2023-12-01T01:36:10.663947+0000","last_became_peered":"2023-12-01T01:36:10.663947+0000","last_unstale":"2023-12-01T01:36:11.842525+0000","last_undegraded":"2023-12-01T01:36:11.842525+0000","last_fullsized":"2023-12-01T01:36:11.842525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844143+0000","last_change":"2023-12-01T01:36:10.660556+0000","last_active":"2023-12-01T01:36:11.844143+0000","last_peered":"2023-12-01T01:36:11.844143+0000","last_clean":"2023-12-01T01:36:11.844143+0000","last_became_active":"2023-12-01T01:36:10.658792+0000","last_became_peered":"2023-12-01T01:36:10.658792+0000","last_unstale":"2023-12-01T01:36:11.844143+0000","last_undegraded":"2023-12-01T01:36:11.844143+0000","last_fullsized":"2023-12-01T01:36:11.844143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:10.875571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.678963+0000","last_change":"2023-12-01T01:36:10.186029+0000","last_active":"2023-12-01T01:36:10.678963+0000","last_peered":"2023-12-01T01:36:10.678963+0000","last_clean":"2023-12-01T01:36:10.678963+0000","last_became_active":"2023-12-01T01:36:10.185703+0000","last_became_peered":"2023-12-01T01:36:10.185703+0000","last_unstale":"2023-12-01T01:36:10.678963+0000","last_undegraded":"2023-12-01T01:36:10.678963+0000","last_fullsized":"2023-12-01T01:36:10.678963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660789+0000","last_change":"2023-12-01T01:36:10.672533+0000","last_active":"2023-12-01T01:36:11.660789+0000","last_peered":"2023-12-01T01:36:11.660789+0000","last_clean":"2023-12-01T01:36:11.660789+0000","last_became_active":"2023-12-01T01:36:10.672202+0000","last_became_peered":"2023-12-01T01:36:10.672202+0000","last_unstale":"2023-12-01T01:36:11.660789+0000","last_undegraded":"2023-12-01T01:36:11.660789+0000","last_fullsized":"2023-12-01T01:36:11.660789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:49.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685718+0000","last_change":"2023-12-01T01:36:10.186695+0000","last_active":"2023-12-01T01:36:10.685718+0000","last_peered":"2023-12-01T01:36:10.685718+0000","last_clean":"2023-12-01T01:36:10.685718+0000","last_became_active":"2023-12-01T01:36:10.186437+0000","last_became_peered":"2023-12-01T01:36:10.186437+0000","last_unstale":"2023-12-01T01:36:10.685718+0000","last_undegraded":"2023-12-01T01:36:10.685718+0000","last_fullsized":"2023-12-01T01:36:10.685718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:52.363022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.649986+0000","last_change":"2023-12-01T01:36:10.192965+0000","last_active":"2023-12-01T01:36:10.649986+0000","last_peered":"2023-12-01T01:36:10.649986+0000","last_clean":"2023-12-01T01:36:10.649986+0000","last_became_active":"2023-12-01T01:36:10.192659+0000","last_became_peered":"2023-12-01T01:36:10.192659+0000","last_unstale":"2023-12-01T01:36:10.649986+0000","last_undegraded":"2023-12-01T01:36:10.649986+0000","last_fullsized":"2023-12-01T01:36:10.649986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844146+0000","last_change":"2023-12-01T01:36:10.665664+0000","last_active":"2023-12-01T01:36:11.844146+0000","last_peered":"2023-12-01T01:36:11.844146+0000","last_clean":"2023-12-01T01:36:11.844146+0000","last_became_active":"2023-12-01T01:36:10.662774+0000","last_became_peered":"2023-12-01T01:36:10.662774+0000","last_unstale":"2023-12-01T01:36:11.844146+0000","last_undegraded":"2023-12-01T01:36:11.844146+0000","last_fullsized":"2023-12-01T01:36:11.844146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.667386+0000","last_change":"2023-12-01T01:36:10.177490+0000","last_active":"2023-12-01T01:36:10.667386+0000","last_peered":"2023-12-01T01:36:10.667386+0000","last_clean":"2023-12-01T01:36:10.667386+0000","last_became_active":"2023-12-01T01:36:10.176976+0000","last_became_peered":"2023-12-01T01:36:10.176976+0000","last_unstale":"2023-12-01T01:36:10.667386+0000","last_undegraded":"2023-12-01T01:36:10.667386+0000","last_fullsized":"2023-12-01T01:36:10.667386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:09.863515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660588+0000","last_change":"2023-12-01T01:36:10.663142+0000","last_active":"2023-12-01T01:36:11.660588+0000","last_peered":"2023-12-01T01:36:11.660588+0000","last_clean":"2023-12-01T01:36:11.660588+0000","last_became_active":"2023-12-01T01:36:10.662866+0000","last_became_peered":"2023-12-01T01:36:10.662866+0000","last_unstale":"2023-12-01T01:36:11.660588+0000","last_undegraded":"2023-12-01T01:36:11.660588+0000","last_fullsized":"2023-12-01T01:36:11.660588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:25.357183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.647474+0000","last_change":"2023-12-01T01:36:10.189645+0000","last_active":"2023-12-01T01:36:10.647474+0000","last_peered":"2023-12-01T01:36:10.647474+0000","last_clean":"2023-12-01T01:36:10.647474+0000","last_became_active":"2023-12-01T01:36:10.189381+0000","last_became_peered":"2023-12-01T01:36:10.189381+0000","last_unstale":"2023-12-01T01:36:10.647474+0000","last_undegraded":"2023-12-01T01:36:10.647474+0000","last_fullsized":"2023-12-01T01:36:10.647474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843160+0000","last_change":"2023-12-01T01:36:10.679049+0000","last_active":"2023-12-01T01:36:11.843160+0000","last_peered":"2023-12-01T01:36:11.843160+0000","last_clean":"2023-12-01T01:36:11.843160+0000","last_became_active":"2023-12-01T01:36:10.678600+0000","last_became_peered":"2023-12-01T01:36:10.678600+0000","last_unstale":"2023-12-01T01:36:11.843160+0000","last_undegraded":"2023-12-01T01:36:11.843160+0000","last_fullsized":"2023-12-01T01:36:11.843160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650463+0000","last_change":"2023-12-01T01:36:10.190157+0000","last_active":"2023-12-01T01:36:10.650463+0000","last_peered":"2023-12-01T01:36:10.650463+0000","last_clean":"2023-12-01T01:36:10.650463+0000","last_became_active":"2023-12-01T01:36:10.189519+0000","last_became_peered":"2023-12-01T01:36:10.189519+0000","last_unstale":"2023-12-01T01:36:10.650463+0000","last_undegraded":"2023-12-01T01:36:10.650463+0000","last_fullsized":"2023-12-01T01:36:10.650463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845357+0000","last_change":"2023-12-01T01:36:10.670783+0000","last_active":"2023-12-01T01:36:11.845357+0000","last_peered":"2023-12-01T01:36:11.845357+0000","last_clean":"2023-12-01T01:36:11.845357+0000","last_became_active":"2023-12-01T01:36:10.669960+0000","last_became_peered":"2023-12-01T01:36:10.669960+0000","last_unstale":"2023-12-01T01:36:11.845357+0000","last_undegraded":"2023-12-01T01:36:11.845357+0000","last_fullsized":"2023-12-01T01:36:11.845357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:33.765782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.686400+0000","last_change":"2023-12-01T01:36:10.191392+0000","last_active":"2023-12-01T01:36:10.686400+0000","last_peered":"2023-12-01T01:36:10.686400+0000","last_clean":"2023-12-01T01:36:10.686400+0000","last_became_active":"2023-12-01T01:36:10.191010+0000","last_became_peered":"2023-12-01T01:36:10.191010+0000","last_unstale":"2023-12-01T01:36:10.686400+0000","last_undegraded":"2023-12-01T01:36:10.686400+0000","last_fullsized":"2023-12-01T01:36:10.686400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659710+0000","last_change":"2023-12-01T01:36:10.661986+0000","last_active":"2023-12-01T01:36:11.659710+0000","last_peered":"2023-12-01T01:36:11.659710+0000","last_clean":"2023-12-01T01:36:11.659710+0000","last_became_active":"2023-12-01T01:36:10.661711+0000","last_became_peered":"2023-12-01T01:36:10.661711+0000","last_unstale":"2023-12-01T01:36:11.659710+0000","last_undegraded":"2023-12-01T01:36:11.659710+0000","last_fullsized":"2023-12-01T01:36:11.659710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:12.241846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.688331+0000","last_change":"2023-12-01T01:36:10.190140+0000","last_active":"2023-12-01T01:36:10.688331+0000","last_peered":"2023-12-01T01:36:10.688331+0000","last_clean":"2023-12-01T01:36:10.688331+0000","last_became_active":"2023-12-01T01:36:10.188743+0000","last_became_peered":"2023-12-01T01:36:10.188743+0000","last_unstale":"2023-12-01T01:36:10.688331+0000","last_undegraded":"2023-12-01T01:36:10.688331+0000","last_fullsized":"2023-12-01T01:36:10.688331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:06:15.188480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846990+0000","last_change":"2023-12-01T01:36:10.670314+0000","last_active":"2023-12-01T01:36:11.846990+0000","last_peered":"2023-12-01T01:36:11.846990+0000","last_clean":"2023-12-01T01:36:11.846990+0000","last_became_active":"2023-12-01T01:36:10.669184+0000","last_became_peered":"2023-12-01T01:36:10.669184+0000","last_unstale":"2023-12-01T01:36:11.846990+0000","last_undegraded":"2023-12-01T01:36:11.846990+0000","last_fullsized":"2023-12-01T01:36:11.846990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.684491+0000","last_change":"2023-12-01T01:36:10.195876+0000","last_active":"2023-12-01T01:36:10.684491+0000","last_peered":"2023-12-01T01:36:10.684491+0000","last_clean":"2023-12-01T01:36:10.684491+0000","last_became_active":"2023-12-01T01:36:10.195018+0000","last_became_peered":"2023-12-01T01:36:10.195018+0000","last_unstale":"2023-12-01T01:36:10.684491+0000","last_undegraded":"2023-12-01T01:36:10.684491+0000","last_fullsized":"2023-12-01T01:36:10.684491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843370+0000","last_change":"2023-12-01T01:36:10.677075+0000","last_active":"2023-12-01T01:36:11.843370+0000","last_peered":"2023-12-01T01:36:11.843370+0000","last_clean":"2023-12-01T01:36:11.843370+0000","last_became_active":"2023-12-01T01:36:10.676560+0000","last_became_peered":"2023-12-01T01:36:10.676560+0000","last_unstale":"2023-12-01T01:36:11.843370+0000","last_undegraded":"2023-12-01T01:36:11.843370+0000","last_fullsized":"2023-12-01T01:36:11.843370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:41.649014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.660745+0000","last_change":"2023-12-01T01:36:10.179441+0000","last_active":"2023-12-01T01:36:10.660745+0000","last_peered":"2023-12-01T01:36:10.660745+0000","last_clean":"2023-12-01T01:36:10.660745+0000","last_became_active":"2023-12-01T01:36:10.178179+0000","last_became_peered":"2023-12-01T01:36:10.178179+0000","last_unstale":"2023-12-01T01:36:10.660745+0000","last_undegraded":"2023-12-01T01:36:10.660745+0000","last_fullsized":"2023-12-01T01:36:10.660745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:14.154709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659624+0000","last_change":"2023-12-01T01:36:10.665880+0000","last_active":"2023-12-01T01:36:11.659624+0000","last_peered":"2023-12-01T01:36:11.659624+0000","last_clean":"2023-12-01T01:36:11.659624+0000","last_became_active":"2023-12-01T01:36:10.665639+0000","last_became_peered":"2023-12-01T01:36:10.665639+0000","last_unstale":"2023-12-01T01:36:11.659624+0000","last_undegraded":"2023-12-01T01:36:11.659624+0000","last_fullsized":"2023-12-01T01:36:11.659624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:47.015657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650017+0000","last_change":"2023-12-01T01:36:10.192976+0000","last_active":"2023-12-01T01:36:10.650017+0000","last_peered":"2023-12-01T01:36:10.650017+0000","last_clean":"2023-12-01T01:36:10.650017+0000","last_became_active":"2023-12-01T01:36:10.192692+0000","last_became_peered":"2023-12-01T01:36:10.192692+0000","last_unstale":"2023-12-01T01:36:10.650017+0000","last_undegraded":"2023-12-01T01:36:10.650017+0000","last_fullsized":"2023-12-01T01:36:10.650017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843415+0000","last_change":"2023-12-01T01:36:10.659027+0000","last_active":"2023-12-01T01:36:11.843415+0000","last_peered":"2023-12-01T01:36:11.843415+0000","last_clean":"2023-12-01T01:36:11.843415+0000","last_became_active":"2023-12-01T01:36:10.658683+0000","last_became_peered":"2023-12-01T01:36:10.658683+0000","last_unstale":"2023-12-01T01:36:11.843415+0000","last_undegraded":"2023-12-01T01:36:11.843415+0000","last_fullsized":"2023-12-01T01:36:11.843415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.687400+0000","last_change":"2023-12-01T01:36:10.180654+0000","last_active":"2023-12-01T01:36:10.687400+0000","last_peered":"2023-12-01T01:36:10.687400+0000","last_clean":"2023-12-01T01:36:10.687400+0000","last_became_active":"2023-12-01T01:36:10.179301+0000","last_became_peered":"2023-12-01T01:36:10.179301+0000","last_unstale":"2023-12-01T01:36:10.687400+0000","last_undegraded":"2023-12-01T01:36:10.687400+0000","last_fullsized":"2023-12-01T01:36:10.687400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659085+0000","last_change":"2023-12-01T01:36:10.664772+0000","last_active":"2023-12-01T01:36:11.659085+0000","last_peered":"2023-12-01T01:36:11.659085+0000","last_clean":"2023-12-01T01:36:11.659085+0000","last_became_active":"2023-12-01T01:36:10.664504+0000","last_became_peered":"2023-12-01T01:36:10.664504+0000","last_unstale":"2023-12-01T01:36:11.659085+0000","last_undegraded":"2023-12-01T01:36:11.659085+0000","last_fullsized":"2023-12-01T01:36:11.659085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:44:20.605895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.686349+0000","last_change":"2023-12-01T01:36:10.189157+0000","last_active":"2023-12-01T01:36:10.686349+0000","last_peered":"2023-12-01T01:36:10.686349+0000","last_clean":"2023-12-01T01:36:10.686349+0000","last_became_active":"2023-12-01T01:36:10.188580+0000","last_became_peered":"2023-12-01T01:36:10.188580+0000","last_unstale":"2023-12-01T01:36:10.686349+0000","last_undegraded":"2023-12-01T01:36:10.686349+0000","last_fullsized":"2023-12-01T01:36:10.686349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843263+0000","last_change":"2023-12-01T01:36:10.666504+0000","last_active":"2023-12-01T01:36:11.843263+0000","last_peered":"2023-12-01T01:36:11.843263+0000","last_clean":"2023-12-01T01:36:11.843263+0000","last_became_active":"2023-12-01T01:36:10.666252+0000","last_became_peered":"2023-12-01T01:36:10.666252+0000","last_unstale":"2023-12-01T01:36:11.843263+0000","last_undegraded":"2023-12-01T01:36:11.843263+0000","last_fullsized":"2023-12-01T01:36:11.843263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:41.649014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683819+0000","last_change":"2023-12-01T01:36:10.186429+0000","last_active":"2023-12-01T01:36:10.683819+0000","last_peered":"2023-12-01T01:36:10.683819+0000","last_clean":"2023-12-01T01:36:10.683819+0000","last_became_active":"2023-12-01T01:36:10.185294+0000","last_became_peered":"2023-12-01T01:36:10.185294+0000","last_unstale":"2023-12-01T01:36:10.683819+0000","last_undegraded":"2023-12-01T01:36:10.683819+0000","last_fullsized":"2023-12-01T01:36:10.683819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847638+0000","last_change":"2023-12-01T01:36:10.667398+0000","last_active":"2023-12-01T01:36:11.847638+0000","last_peered":"2023-12-01T01:36:11.847638+0000","last_clean":"2023-12-01T01:36:11.847638+0000","last_became_active":"2023-12-01T01:36:10.667108+0000","last_became_peered":"2023-12-01T01:36:10.667108+0000","last_unstale":"2023-12-01T01:36:11.847638+0000","last_undegraded":"2023-12-01T01:36:11.847638+0000","last_fullsized":"2023-12-01T01:36:11.847638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:41:51.516524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.680767+0000","last_change":"2023-12-01T01:36:10.198699+0000","last_active":"2023-12-01T01:36:10.680767+0000","last_peered":"2023-12-01T01:36:10.680767+0000","last_clean":"2023-12-01T01:36:10.680767+0000","last_became_active":"2023-12-01T01:36:10.198340+0000","last_became_peered":"2023-12-01T01:36:10.198340+0000","last_unstale":"2023-12-01T01:36:10.680767+0000","last_undegraded":"2023-12-01T01:36:10.680767+0000","last_fullsized":"2023-12-01T01:36:10.680767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846471+0000","last_change":"2023-12-01T01:36:10.680027+0000","last_active":"2023-12-01T01:36:11.846471+0000","last_peered":"2023-12-01T01:36:11.846471+0000","last_clean":"2023-12-01T01:36:11.846471+0000","last_became_active":"2023-12-01T01:36:10.679670+0000","last_became_peered":"2023-12-01T01:36:10.679670+0000","last_unstale":"2023-12-01T01:36:11.846471+0000","last_undegraded":"2023-12-01T01:36:11.846471+0000","last_fullsized":"2023-12-01T01:36:11.846471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.657130+0000","last_change":"2023-12-01T01:36:10.194952+0000","last_active":"2023-12-01T01:36:10.657130+0000","last_peered":"2023-12-01T01:36:10.657130+0000","last_clean":"2023-12-01T01:36:10.657130+0000","last_became_active":"2023-12-01T01:36:10.194708+0000","last_became_peered":"2023-12-01T01:36:10.194708+0000","last_unstale":"2023-12-01T01:36:10.657130+0000","last_undegraded":"2023-12-01T01:36:10.657130+0000","last_fullsized":"2023-12-01T01:36:10.657130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659619+0000","last_change":"2023-12-01T01:36:10.663313+0000","last_active":"2023-12-01T01:36:11.659619+0000","last_peered":"2023-12-01T01:36:11.659619+0000","last_clean":"2023-12-01T01:36:11.659619+0000","last_became_active":"2023-12-01T01:36:10.662987+0000","last_became_peered":"2023-12-01T01:36:10.662987+0000","last_unstale":"2023-12-01T01:36:11.659619+0000","last_undegraded":"2023-12-01T01:36:11.659619+0000","last_fullsized":"2023-12-01T01:36:11.659619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.646527+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:36:10.646527+0000","last_peered":"2023-12-01T01:36:10.646527+0000","last_clean":"2023-12-01T01:36:10.646527+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:36:10.646527+0000","last_undegraded":"2023-12-01T01:36:10.646527+0000","last_fullsized":"2023-12-01T01:36:10.646527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.664313+0000","last_change":"2023-12-01T01:36:10.194117+0000","last_active":"2023-12-01T01:36:10.664313+0000","last_peered":"2023-12-01T01:36:10.664313+0000","last_clean":"2023-12-01T01:36:10.664313+0000","last_became_active":"2023-12-01T01:36:10.193801+0000","last_became_peered":"2023-12-01T01:36:10.193801+0000","last_unstale":"2023-12-01T01:36:10.664313+0000","last_undegraded":"2023-12-01T01:36:10.664313+0000","last_fullsized":"2023-12-01T01:36:10.664313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660654+0000","last_change":"2023-12-01T01:36:10.669848+0000","last_active":"2023-12-01T01:36:11.660654+0000","last_peered":"2023-12-01T01:36:11.660654+0000","last_clean":"2023-12-01T01:36:11.660654+0000","last_became_active":"2023-12-01T01:36:10.669557+0000","last_became_peered":"2023-12-01T01:36:10.669557+0000","last_unstale":"2023-12-01T01:36:11.660654+0000","last_undegraded":"2023-12-01T01:36:11.660654+0000","last_fullsized":"2023-12-01T01:36:11.660654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.667473+0000","last_change":"2023-12-01T01:36:10.183096+0000","last_active":"2023-12-01T01:36:10.667473+0000","last_peered":"2023-12-01T01:36:10.667473+0000","last_clean":"2023-12-01T01:36:10.667473+0000","last_became_active":"2023-12-01T01:36:10.182810+0000","last_became_peered":"2023-12-01T01:36:10.182810+0000","last_unstale":"2023-12-01T01:36:10.667473+0000","last_undegraded":"2023-12-01T01:36:10.667473+0000","last_fullsized":"2023-12-01T01:36:10.667473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:35.765727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844076+0000","last_change":"2023-12-01T01:36:10.661869+0000","last_active":"2023-12-01T01:36:11.844076+0000","last_peered":"2023-12-01T01:36:11.844076+0000","last_clean":"2023-12-01T01:36:11.844076+0000","last_became_active":"2023-12-01T01:36:10.661404+0000","last_became_peered":"2023-12-01T01:36:10.661404+0000","last_unstale":"2023-12-01T01:36:11.844076+0000","last_undegraded":"2023-12-01T01:36:11.844076+0000","last_fullsized":"2023-12-01T01:36:11.844076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.669601+0000","last_change":"2023-12-01T01:36:10.188465+0000","last_active":"2023-12-01T01:36:10.669601+0000","last_peered":"2023-12-01T01:36:10.669601+0000","last_clean":"2023-12-01T01:36:10.669601+0000","last_became_active":"2023-12-01T01:36:10.188157+0000","last_became_peered":"2023-12-01T01:36:10.188157+0000","last_unstale":"2023-12-01T01:36:10.669601+0000","last_undegraded":"2023-12-01T01:36:10.669601+0000","last_fullsized":"2023-12-01T01:36:10.669601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843928+0000","last_change":"2023-12-01T01:36:10.679247+0000","last_active":"2023-12-01T01:36:11.843928+0000","last_peered":"2023-12-01T01:36:11.843928+0000","last_clean":"2023-12-01T01:36:11.843928+0000","last_became_active":"2023-12-01T01:36:10.678097+0000","last_became_peered":"2023-12-01T01:36:10.678097+0000","last_unstale":"2023-12-01T01:36:11.843928+0000","last_undegraded":"2023-12-01T01:36:11.843928+0000","last_fullsized":"2023-12-01T01:36:11.843928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.661362+0000","last_change":"2023-12-01T01:36:10.182533+0000","last_active":"2023-12-01T01:36:10.661362+0000","last_peered":"2023-12-01T01:36:10.661362+0000","last_clean":"2023-12-01T01:36:10.661362+0000","last_became_active":"2023-12-01T01:36:10.182071+0000","last_became_peered":"2023-12-01T01:36:10.182071+0000","last_unstale":"2023-12-01T01:36:10.661362+0000","last_undegraded":"2023-12-01T01:36:10.661362+0000","last_fullsized":"2023-12-01T01:36:10.661362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:40.636957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846563+0000","last_change":"2023-12-01T01:36:10.666142+0000","last_active":"2023-12-01T01:36:11.846563+0000","last_peered":"2023-12-01T01:36:11.846563+0000","last_clean":"2023-12-01T01:36:11.846563+0000","last_became_active":"2023-12-01T01:36:10.665873+0000","last_became_peered":"2023-12-01T01:36:10.665873+0000","last_unstale":"2023-12-01T01:36:11.846563+0000","last_undegraded":"2023-12-01T01:36:11.846563+0000","last_fullsized":"2023-12-01T01:36:11.846563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.660940+0000","last_change":"2023-12-01T01:36:10.190758+0000","last_active":"2023-12-01T01:36:10.660940+0000","last_peered":"2023-12-01T01:36:10.660940+0000","last_clean":"2023-12-01T01:36:10.660940+0000","last_became_active":"2023-12-01T01:36:10.190280+0000","last_became_peered":"2023-12-01T01:36:10.190280+0000","last_unstale":"2023-12-01T01:36:10.660940+0000","last_undegraded":"2023-12-01T01:36:10.660940+0000","last_fullsized":"2023-12-01T01:36:10.660940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842977+0000","last_change":"2023-12-01T01:36:10.665004+0000","last_active":"2023-12-01T01:36:11.842977+0000","last_peered":"2023-12-01T01:36:11.842977+0000","last_clean":"2023-12-01T01:36:11.842977+0000","last_became_active":"2023-12-01T01:36:10.664692+0000","last_became_peered":"2023-12-01T01:36:10.664692+0000","last_unstale":"2023-12-01T01:36:11.842977+0000","last_undegraded":"2023-12-01T01:36:11.842977+0000","last_fullsized":"2023-12-01T01:36:11.842977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:53.375079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.666452+0000","last_change":"2023-12-01T01:36:10.186489+0000","last_active":"2023-12-01T01:36:10.666452+0000","last_peered":"2023-12-01T01:36:10.666452+0000","last_clean":"2023-12-01T01:36:10.666452+0000","last_became_active":"2023-12-01T01:36:10.186161+0000","last_became_peered":"2023-12-01T01:36:10.186161+0000","last_unstale":"2023-12-01T01:36:10.666452+0000","last_undegraded":"2023-12-01T01:36:10.666452+0000","last_fullsized":"2023-12-01T01:36:10.666452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843867+0000","last_change":"2023-12-01T01:36:10.663707+0000","last_active":"2023-12-01T01:36:11.843867+0000","last_peered":"2023-12-01T01:36:11.843867+0000","last_clean":"2023-12-01T01:36:11.843867+0000","last_became_active":"2023-12-01T01:36:10.663158+0000","last_became_peered":"2023-12-01T01:36:10.663158+0000","last_unstale":"2023-12-01T01:36:11.843867+0000","last_undegraded":"2023-12-01T01:36:11.843867+0000","last_fullsized":"2023-12-01T01:36:11.843867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661244+0000","last_change":"2023-12-01T01:36:10.188465+0000","last_active":"2023-12-01T01:36:11.661244+0000","last_peered":"2023-12-01T01:36:11.661244+0000","last_clean":"2023-12-01T01:36:11.661244+0000","last_became_active":"2023-12-01T01:36:10.187631+0000","last_became_peered":"2023-12-01T01:36:10.187631+0000","last_unstale":"2023-12-01T01:36:11.661244+0000","last_undegraded":"2023-12-01T01:36:11.661244+0000","last_fullsized":"2023-12-01T01:36:11.661244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661611+0000","last_change":"2023-12-01T01:36:10.665647+0000","last_active":"2023-12-01T01:36:11.661611+0000","last_peered":"2023-12-01T01:36:11.661611+0000","last_clean":"2023-12-01T01:36:11.661611+0000","last_became_active":"2023-12-01T01:36:10.665371+0000","last_became_peered":"2023-12-01T01:36:10.665371+0000","last_unstale":"2023-12-01T01:36:11.661611+0000","last_undegraded":"2023-12-01T01:36:11.661611+0000","last_fullsized":"2023-12-01T01:36:11.661611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685898+0000","last_change":"2023-12-01T01:36:10.194331+0000","last_active":"2023-12-01T01:36:10.685898+0000","last_peered":"2023-12-01T01:36:10.685898+0000","last_clean":"2023-12-01T01:36:10.685898+0000","last_became_active":"2023-12-01T01:36:10.194058+0000","last_became_peered":"2023-12-01T01:36:10.194058+0000","last_unstale":"2023-12-01T01:36:10.685898+0000","last_undegraded":"2023-12-01T01:36:10.685898+0000","last_fullsized":"2023-12-01T01:36:10.685898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:51.741532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661128+0000","last_change":"2023-12-01T01:36:10.680546+0000","last_active":"2023-12-01T01:36:11.661128+0000","last_peered":"2023-12-01T01:36:11.661128+0000","last_clean":"2023-12-01T01:36:11.661128+0000","last_became_active":"2023-12-01T01:36:10.680099+0000","last_became_peered":"2023-12-01T01:36:10.680099+0000","last_unstale":"2023-12-01T01:36:11.661128+0000","last_undegraded":"2023-12-01T01:36:11.661128+0000","last_fullsized":"2023-12-01T01:36:11.661128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:22.177011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650417+0000","last_change":"2023-12-01T01:36:10.193127+0000","last_active":"2023-12-01T01:36:10.650417+0000","last_peered":"2023-12-01T01:36:10.650417+0000","last_clean":"2023-12-01T01:36:10.650417+0000","last_became_active":"2023-12-01T01:36:10.192884+0000","last_became_peered":"2023-12-01T01:36:10.192884+0000","last_unstale":"2023-12-01T01:36:10.650417+0000","last_undegraded":"2023-12-01T01:36:10.650417+0000","last_fullsized":"2023-12-01T01:36:10.650417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659185+0000","last_change":"2023-12-01T01:36:10.677457+0000","last_active":"2023-12-01T01:36:11.659185+0000","last_peered":"2023-12-01T01:36:11.659185+0000","last_clean":"2023-12-01T01:36:11.659185+0000","last_became_active":"2023-12-01T01:36:10.677156+0000","last_became_peered":"2023-12-01T01:36:10.677156+0000","last_unstale":"2023-12-01T01:36:11.659185+0000","last_undegraded":"2023-12-01T01:36:11.659185+0000","last_fullsized":"2023-12-01T01:36:11.659185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:41.113753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.654051+0000","last_change":"2023-12-01T01:36:10.189035+0000","last_active":"2023-12-01T01:36:10.654051+0000","last_peered":"2023-12-01T01:36:10.654051+0000","last_clean":"2023-12-01T01:36:10.654051+0000","last_became_active":"2023-12-01T01:36:10.188370+0000","last_became_peered":"2023-12-01T01:36:10.188370+0000","last_unstale":"2023-12-01T01:36:10.654051+0000","last_undegraded":"2023-12-01T01:36:10.654051+0000","last_fullsized":"2023-12-01T01:36:10.654051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660520+0000","last_change":"2023-12-01T01:36:10.682836+0000","last_active":"2023-12-01T01:36:11.660520+0000","last_peered":"2023-12-01T01:36:11.660520+0000","last_clean":"2023-12-01T01:36:11.660520+0000","last_became_active":"2023-12-01T01:36:10.682361+0000","last_became_peered":"2023-12-01T01:36:10.682361+0000","last_unstale":"2023-12-01T01:36:11.660520+0000","last_undegraded":"2023-12-01T01:36:11.660520+0000","last_fullsized":"2023-12-01T01:36:11.660520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:27.561648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.651403+0000","last_change":"2023-12-01T01:36:10.190808+0000","last_active":"2023-12-01T01:36:10.651403+0000","last_peered":"2023-12-01T01:36:10.651403+0000","last_clean":"2023-12-01T01:36:10.651403+0000","last_became_active":"2023-12-01T01:36:10.190355+0000","last_became_peered":"2023-12-01T01:36:10.190355+0000","last_unstale":"2023-12-01T01:36:10.651403+0000","last_undegraded":"2023-12-01T01:36:10.651403+0000","last_fullsized":"2023-12-01T01:36:10.651403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659213+0000","last_change":"2023-12-01T01:36:10.685203+0000","last_active":"2023-12-01T01:36:11.659213+0000","last_peered":"2023-12-01T01:36:11.659213+0000","last_clean":"2023-12-01T01:36:11.659213+0000","last_became_active":"2023-12-01T01:36:10.684118+0000","last_became_peered":"2023-12-01T01:36:10.684118+0000","last_unstale":"2023-12-01T01:36:11.659213+0000","last_undegraded":"2023-12-01T01:36:11.659213+0000","last_fullsized":"2023-12-01T01:36:11.659213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683327+0000","last_change":"2023-12-01T01:36:10.194065+0000","last_active":"2023-12-01T01:36:10.683327+0000","last_peered":"2023-12-01T01:36:10.683327+0000","last_clean":"2023-12-01T01:36:10.683327+0000","last_became_active":"2023-12-01T01:36:10.193785+0000","last_became_peered":"2023-12-01T01:36:10.193785+0000","last_unstale":"2023-12-01T01:36:10.683327+0000","last_undegraded":"2023-12-01T01:36:10.683327+0000","last_fullsized":"2023-12-01T01:36:10.683327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659511+0000","last_change":"2023-12-01T01:36:10.654568+0000","last_active":"2023-12-01T01:36:11.659511+0000","last_peered":"2023-12-01T01:36:11.659511+0000","last_clean":"2023-12-01T01:36:11.659511+0000","last_became_active":"2023-12-01T01:36:10.654312+0000","last_became_peered":"2023-12-01T01:36:10.654312+0000","last_unstale":"2023-12-01T01:36:11.659511+0000","last_undegraded":"2023-12-01T01:36:11.659511+0000","last_fullsized":"2023-12-01T01:36:11.659511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:24.066059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659518+0000","last_change":"2023-12-01T01:36:10.672526+0000","last_active":"2023-12-01T01:36:11.659518+0000","last_peered":"2023-12-01T01:36:11.659518+0000","last_clean":"2023-12-01T01:36:11.659518+0000","last_became_active":"2023-12-01T01:36:10.671950+0000","last_became_peered":"2023-12-01T01:36:10.671950+0000","last_unstale":"2023-12-01T01:36:11.659518+0000","last_undegraded":"2023-12-01T01:36:11.659518+0000","last_fullsized":"2023-12-01T01:36:11.659518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:07.041952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.681098+0000","last_change":"2023-12-01T01:36:10.189066+0000","last_active":"2023-12-01T01:36:10.681098+0000","last_peered":"2023-12-01T01:36:10.681098+0000","last_clean":"2023-12-01T01:36:10.681098+0000","last_became_active":"2023-12-01T01:36:10.188395+0000","last_became_peered":"2023-12-01T01:36:10.188395+0000","last_unstale":"2023-12-01T01:36:10.681098+0000","last_undegraded":"2023-12-01T01:36:10.681098+0000","last_fullsized":"2023-12-01T01:36:10.681098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844042+0000","last_change":"2023-12-01T01:36:10.677457+0000","last_active":"2023-12-01T01:36:11.844042+0000","last_peered":"2023-12-01T01:36:11.844042+0000","last_clean":"2023-12-01T01:36:11.844042+0000","last_became_active":"2023-12-01T01:36:10.677200+0000","last_became_peered":"2023-12-01T01:36:10.677200+0000","last_unstale":"2023-12-01T01:36:11.844042+0000","last_undegraded":"2023-12-01T01:36:11.844042+0000","last_fullsized":"2023-12-01T01:36:11.844042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:53.375079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.681045+0000","last_change":"2023-12-01T01:36:10.193800+0000","last_active":"2023-12-01T01:36:10.681045+0000","last_peered":"2023-12-01T01:36:10.681045+0000","last_clean":"2023-12-01T01:36:10.681045+0000","last_became_active":"2023-12-01T01:36:10.193469+0000","last_became_peered":"2023-12-01T01:36:10.193469+0000","last_unstale":"2023-12-01T01:36:10.681045+0000","last_undegraded":"2023-12-01T01:36:10.681045+0000","last_fullsized":"2023-12-01T01:36:10.681045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845128+0000","last_change":"2023-12-01T01:36:10.665085+0000","last_active":"2023-12-01T01:36:11.845128+0000","last_peered":"2023-12-01T01:36:11.845128+0000","last_clean":"2023-12-01T01:36:11.845128+0000","last_became_active":"2023-12-01T01:36:10.663166+0000","last_became_peered":"2023-12-01T01:36:10.663166+0000","last_unstale":"2023-12-01T01:36:11.845128+0000","last_undegraded":"2023-12-01T01:36:11.845128+0000","last_fullsized":"2023-12-01T01:36:11.845128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:38.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683819+0000","last_change":"2023-12-01T01:36:10.188667+0000","last_active":"2023-12-01T01:36:10.683819+0000","last_peered":"2023-12-01T01:36:10.683819+0000","last_clean":"2023-12-01T01:36:10.683819+0000","last_became_active":"2023-12-01T01:36:10.188354+0000","last_became_peered":"2023-12-01T01:36:10.188354+0000","last_unstale":"2023-12-01T01:36:10.683819+0000","last_undegraded":"2023-12-01T01:36:10.683819+0000","last_fullsized":"2023-12-01T01:36:10.683819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845005+0000","last_change":"2023-12-01T01:36:10.660331+0000","last_active":"2023-12-01T01:36:11.845005+0000","last_peered":"2023-12-01T01:36:11.845005+0000","last_clean":"2023-12-01T01:36:11.845005+0000","last_became_active":"2023-12-01T01:36:10.660037+0000","last_became_peered":"2023-12-01T01:36:10.660037+0000","last_unstale":"2023-12-01T01:36:11.845005+0000","last_undegraded":"2023-12-01T01:36:11.845005+0000","last_fullsized":"2023-12-01T01:36:11.845005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:12.241846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685486+0000","last_change":"2023-12-01T01:36:10.189468+0000","last_active":"2023-12-01T01:36:10.685486+0000","last_peered":"2023-12-01T01:36:10.685486+0000","last_clean":"2023-12-01T01:36:10.685486+0000","last_became_active":"2023-12-01T01:36:10.187185+0000","last_became_peered":"2023-12-01T01:36:10.187185+0000","last_unstale":"2023-12-01T01:36:10.685486+0000","last_undegraded":"2023-12-01T01:36:10.685486+0000","last_fullsized":"2023-12-01T01:36:10.685486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":75693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":819200,"data_stored":666029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":77805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":76151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.883}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7729999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.599}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.845}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.738}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":815104,"data_stored":666007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":75625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.25}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:36:27.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:36:27.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.613+0000 7f6e7792e640 1 Processor -- start 2023-12-01T01:36:27.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.613+0000 7f6e7792e640 1 -- start start 2023-12-01T01:36:27.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.614+0000 7f6e7792e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e7015e2f0 0x7f6e700ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.614+0000 7f6e7792e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e700add10 con 0x7f6e7006f2f0 2023-12-01T01:36:27.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.615+0000 7f6e7792e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e700c5760 con 0x7f6e700adf10 2023-12-01T01:36:27.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.615+0000 7f6e7792e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e700c9cf0 con 0x7f6e7015e2f0 2023-12-01T01:36:27.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.615+0000 7f6e756ca640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e7015e2f0 0x7f6e700ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:27.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.615+0000 7f6e756ca640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e7015e2f0 0x7f6e700ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60786/0 (socket says 172.21.15.115:60786) 2023-12-01T01:36:27.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.615+0000 7f6e756ca640 1 -- 172.21.15.115:0/180357390 learned_addr learned my addr 172.21.15.115:0/180357390 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:27.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.615+0000 7f6e74ec9640 1 --1- 172.21.15.115:0/180357390 >> v1:172.21.15.115:6789/0 conn(0x7f6e7006f2f0 0x7f6e700c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38032/0 (socket says 172.21.15.115:38032) 2023-12-01T01:36:27.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.615+0000 7f6e756ca640 1 -- 172.21.15.115:0/180357390 >> v1:172.21.15.142:6789/0 conn(0x7f6e700adf10 legacy=0x7f6e700c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:27.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.615+0000 7f6e756ca640 1 -- 172.21.15.115:0/180357390 >> v1:172.21.15.115:6789/0 conn(0x7f6e7006f2f0 legacy=0x7f6e700c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.615+0000 7f6e756ca640 1 -- 172.21.15.115:0/180357390 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7015e6d0 con 0x7f6e7015e2f0 2023-12-01T01:36:27.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.615+0000 7f6e67fff640 1 -- 172.21.15.115:0/180357390 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271961225 0 0) 0x7f6e700add10 con 0x7f6e7006f2f0 2023-12-01T01:36:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.616+0000 7f6e756ca640 1 --2- 172.21.15.115:0/180357390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7015e2f0 0x7f6e700ca660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009870 tx=0x7f6e6c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3e3ba44c84d7e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.616+0000 7f6e67fff640 1 -- 172.21.15.115:0/180357390 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0317e0 con 0x7f6e7015e2f0 2023-12-01T01:36:27.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.616+0000 7f6e67fff640 1 -- 172.21.15.115:0/180357390 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c031940 con 0x7f6e7015e2f0 2023-12-01T01:36:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.616+0000 7f6e67fff640 1 -- 172.21.15.115:0/180357390 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c031cd0 con 0x7f6e7015e2f0 2023-12-01T01:36:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.617+0000 7f6e7792e640 1 -- 172.21.15.115:0/180357390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7015e2f0 msgr2=0x7f6e700ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.617+0000 7f6e7792e640 1 --2- 172.21.15.115:0/180357390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7015e2f0 0x7f6e700ca660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009870 tx=0x7f6e6c02b880 comp rx=0 tx=0).stop 2023-12-01T01:36:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.617+0000 7f6e7792e640 1 -- 172.21.15.115:0/180357390 shutdown_connections 2023-12-01T01:36:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.617+0000 7f6e7792e640 1 --2- 172.21.15.115:0/180357390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7015e2f0 0x7f6e700ca660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.617+0000 7f6e7792e640 1 -- 172.21.15.115:0/180357390 >> 172.21.15.115:0/180357390 conn(0x7f6e700ce880 msgr2=0x7f6e700cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.617+0000 7f6e7792e640 1 -- 172.21.15.115:0/180357390 shutdown_connections 2023-12-01T01:36:27.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.617+0000 7f6e7792e640 1 -- 172.21.15.115:0/180357390 wait complete. 2023-12-01T01:36:27.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.619+0000 7f6e7792e640 1 Processor -- start 2023-12-01T01:36:27.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.619+0000 7f6e7792e640 1 -- start start 2023-12-01T01:36:27.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e7792e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7006f2f0 0x7f6e70159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e7792e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e700adf10 0x7f6e700c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e7792e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e7015e2f0 0x7f6e700c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e7792e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700c2790 con 0x7f6e7006f2f0 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e7792e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700c6780 con 0x7f6e7015e2f0 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e7792e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700c6a60 con 0x7f6e700adf10 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e756ca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7006f2f0 0x7f6e70159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e756ca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7006f2f0 0x7f6e70159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60796/0 (socket says 172.21.15.115:60796) 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e756ca640 1 -- 172.21.15.115:0/435360979 learned_addr learned my addr 172.21.15.115:0/435360979 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e756ca640 1 -- 172.21.15.115:0/435360979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e700adf10 msgr2=0x7f6e700c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e756ca640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e700adf10 0x7f6e700c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e756ca640 1 -- 172.21.15.115:0/435360979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e7015e2f0 msgr2=0x7f6e700c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e756ca640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e7015e2f0 0x7f6e700c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.620+0000 7f6e756ca640 1 -- 172.21.15.115:0/435360979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c0094d0 con 0x7f6e7006f2f0 2023-12-01T01:36:27.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.621+0000 7f6e756ca640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7006f2f0 0x7f6e70159050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c000c00 tx=0x7f6e6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:27.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.621+0000 7f6e65ffb640 1 -- 172.21.15.115:0/435360979 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c032880 con 0x7f6e7006f2f0 2023-12-01T01:36:27.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.621+0000 7f6e65ffb640 1 -- 172.21.15.115:0/435360979 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c032a00 con 0x7f6e7006f2f0 2023-12-01T01:36:27.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.621+0000 7f6e7792e640 1 -- 172.21.15.115:0/435360979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e700c6d40 con 0x7f6e7006f2f0 2023-12-01T01:36:27.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.621+0000 7f6e65ffb640 1 -- 172.21.15.115:0/435360979 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c032db0 con 0x7f6e7006f2f0 2023-12-01T01:36:27.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.621+0000 7f6e7792e640 1 -- 172.21.15.115:0/435360979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e700c59e0 con 0x7f6e7006f2f0 2023-12-01T01:36:27.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.622+0000 7f6e7792e640 1 -- 172.21.15.115:0/435360979 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7006fda0 con 0x7f6e7006f2f0 2023-12-01T01:36:27.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.623+0000 7f6e65ffb640 1 -- 172.21.15.115:0/435360979 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6e6c04e450 con 0x7f6e7006f2f0 2023-12-01T01:36:27.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.624+0000 7f6e65ffb640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e5406a430 0x7f6e5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:27.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.624+0000 7f6e65ffb640 1 -- 172.21.15.115:0/435360979 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e6c0aab80 con 0x7f6e7006f2f0 2023-12-01T01:36:27.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.624+0000 7f6e74ec9640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e5406a430 0x7f6e5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:27.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.624+0000 7f6e74ec9640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e5406a430 0x7f6e5406c8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6e60007fd0 tx=0x7f6e60007960 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:27.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.628+0000 7f6e65ffb640 1 -- 172.21.15.115:0/435360979 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6e6c076210 con 0x7f6e7006f2f0 2023-12-01T01:36:27.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.798+0000 7f6e7792e640 1 -- 172.21.15.115:0/435360979 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e700c6bd0 con 0x7f6e5406a430 2023-12-01T01:36:27.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.819+0000 7f6e65ffb640 1 -- 172.21.15.115:0/435360979 <== mgr.4098 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264287 (secure 0 0 0) 0x7f6e700c6bd0 con 0x7f6e5406a430 2023-12-01T01:36:27.823 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:27.824 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:36:27.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 -- 172.21.15.115:0/435360979 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e5406a430 msgr2=0x7f6e5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e5406a430 0x7f6e5406c8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6e60007fd0 tx=0x7f6e60007960 comp rx=0 tx=0).stop 2023-12-01T01:36:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 -- 172.21.15.115:0/435360979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7006f2f0 msgr2=0x7f6e70159050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7006f2f0 0x7f6e70159050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c000c00 tx=0x7f6e6c0046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 -- 172.21.15.115:0/435360979 shutdown_connections 2023-12-01T01:36:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e5406a430 0x7f6e5406c8f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e700adf10 0x7f6e700c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e7015e2f0 0x7f6e700c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 --2- 172.21.15.115:0/435360979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e7006f2f0 0x7f6e70159050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 -- 172.21.15.115:0/435360979 >> 172.21.15.115:0/435360979 conn(0x7f6e700ce880 msgr2=0x7f6e700cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 -- 172.21.15.115:0/435360979 shutdown_connections 2023-12-01T01:36:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:27.824+0000 7f6e7792e640 1 -- 172.21.15.115:0/435360979 wait complete. 2023-12-01T01:36:27.842 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-01T01:36:26.161041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218536,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756184,"statfs":{"total":773094113280,"available":772870332416,"internally_reserved":0,"allocated":2990080,"data_stored":1790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004199"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847556+0000","last_change":"2023-12-01T01:36:10.681289+0000","last_active":"2023-12-01T01:36:11.847556+0000","last_peered":"2023-12-01T01:36:11.847556+0000","last_clean":"2023-12-01T01:36:11.847556+0000","last_became_active":"2023-12-01T01:36:10.680689+0000","last_became_peered":"2023-12-01T01:36:10.680689+0000","last_unstale":"2023-12-01T01:36:11.847556+0000","last_undegraded":"2023-12-01T01:36:11.847556+0000","last_fullsized":"2023-12-01T01:36:11.847556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:12:29.670579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844746+0000","last_change":"2023-12-01T01:36:10.668026+0000","last_active":"2023-12-01T01:36:11.844746+0000","last_peered":"2023-12-01T01:36:11.844746+0000","last_clean":"2023-12-01T01:36:11.844746+0000","last_became_active":"2023-12-01T01:36:10.667717+0000","last_became_peered":"2023-12-01T01:36:10.667717+0000","last_unstale":"2023-12-01T01:36:11.844746+0000","last_undegraded":"2023-12-01T01:36:11.844746+0000","last_fullsized":"2023-12-01T01:36:11.844746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:12:29.670579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844579+0000","last_change":"2023-12-01T01:36:10.665172+0000","last_active":"2023-12-01T01:36:11.844579+0000","last_peered":"2023-12-01T01:36:11.844579+0000","last_clean":"2023-12-01T01:36:11.844579+0000","last_became_active":"2023-12-01T01:36:10.664880+0000","last_became_peered":"2023-12-01T01:36:10.664880+0000","last_unstale":"2023-12-01T01:36:11.844579+0000","last_undegraded":"2023-12-01T01:36:11.844579+0000","last_fullsized":"2023-12-01T01:36:11.844579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:41:51.516524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843255+0000","last_change":"2023-12-01T01:36:10.666933+0000","last_active":"2023-12-01T01:36:11.843255+0000","last_peered":"2023-12-01T01:36:11.843255+0000","last_clean":"2023-12-01T01:36:11.843255+0000","last_became_active":"2023-12-01T01:36:10.666669+0000","last_became_peered":"2023-12-01T01:36:10.666669+0000","last_unstale":"2023-12-01T01:36:11.843255+0000","last_undegraded":"2023-12-01T01:36:11.843255+0000","last_fullsized":"2023-12-01T01:36:11.843255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846486+0000","last_change":"2023-12-01T01:36:10.660927+0000","last_active":"2023-12-01T01:36:11.846486+0000","last_peered":"2023-12-01T01:36:11.846486+0000","last_clean":"2023-12-01T01:36:11.846486+0000","last_became_active":"2023-12-01T01:36:10.660693+0000","last_became_peered":"2023-12-01T01:36:10.660693+0000","last_unstale":"2023-12-01T01:36:11.846486+0000","last_undegraded":"2023-12-01T01:36:11.846486+0000","last_fullsized":"2023-12-01T01:36:11.846486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:02.974458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842420+0000","last_change":"2023-12-01T01:36:10.670715+0000","last_active":"2023-12-01T01:36:11.842420+0000","last_peered":"2023-12-01T01:36:11.842420+0000","last_clean":"2023-12-01T01:36:11.842420+0000","last_became_active":"2023-12-01T01:36:10.669975+0000","last_became_peered":"2023-12-01T01:36:10.669975+0000","last_unstale":"2023-12-01T01:36:11.842420+0000","last_undegraded":"2023-12-01T01:36:11.842420+0000","last_fullsized":"2023-12-01T01:36:11.842420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845913+0000","last_change":"2023-12-01T01:36:10.670567+0000","last_active":"2023-12-01T01:36:11.845913+0000","last_peered":"2023-12-01T01:36:11.845913+0000","last_clean":"2023-12-01T01:36:11.845913+0000","last_became_active":"2023-12-01T01:36:10.670325+0000","last_became_peered":"2023-12-01T01:36:10.670325+0000","last_unstale":"2023-12-01T01:36:11.845913+0000","last_undegraded":"2023-12-01T01:36:11.845913+0000","last_fullsized":"2023-12-01T01:36:11.845913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:49.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843702+0000","last_change":"2023-12-01T01:36:10.672309+0000","last_active":"2023-12-01T01:36:11.843702+0000","last_peered":"2023-12-01T01:36:11.843702+0000","last_clean":"2023-12-01T01:36:11.843702+0000","last_became_active":"2023-12-01T01:36:10.671949+0000","last_became_peered":"2023-12-01T01:36:10.671949+0000","last_unstale":"2023-12-01T01:36:11.843702+0000","last_undegraded":"2023-12-01T01:36:11.843702+0000","last_fullsized":"2023-12-01T01:36:11.843702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:44:20.605895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844629+0000","last_change":"2023-12-01T01:36:10.664390+0000","last_active":"2023-12-01T01:36:11.844629+0000","last_peered":"2023-12-01T01:36:11.844629+0000","last_clean":"2023-12-01T01:36:11.844629+0000","last_became_active":"2023-12-01T01:36:10.663998+0000","last_became_peered":"2023-12-01T01:36:10.663998+0000","last_unstale":"2023-12-01T01:36:11.844629+0000","last_undegraded":"2023-12-01T01:36:11.844629+0000","last_fullsized":"2023-12-01T01:36:11.844629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847720+0000","last_change":"2023-12-01T01:36:10.662705+0000","last_active":"2023-12-01T01:36:11.847720+0000","last_peered":"2023-12-01T01:36:11.847720+0000","last_clean":"2023-12-01T01:36:11.847720+0000","last_became_active":"2023-12-01T01:36:10.662424+0000","last_became_peered":"2023-12-01T01:36:10.662424+0000","last_unstale":"2023-12-01T01:36:11.847720+0000","last_undegraded":"2023-12-01T01:36:11.847720+0000","last_fullsized":"2023-12-01T01:36:11.847720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:44:55.685435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846761+0000","last_change":"2023-12-01T01:36:10.669520+0000","last_active":"2023-12-01T01:36:11.846761+0000","last_peered":"2023-12-01T01:36:11.846761+0000","last_clean":"2023-12-01T01:36:11.846761+0000","last_became_active":"2023-12-01T01:36:10.665644+0000","last_became_peered":"2023-12-01T01:36:10.665644+0000","last_unstale":"2023-12-01T01:36:11.846761+0000","last_undegraded":"2023-12-01T01:36:11.846761+0000","last_fullsized":"2023-12-01T01:36:11.846761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:38.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846407+0000","last_change":"2023-12-01T01:36:10.668672+0000","last_active":"2023-12-01T01:36:11.846407+0000","last_peered":"2023-12-01T01:36:11.846407+0000","last_clean":"2023-12-01T01:36:11.846407+0000","last_became_active":"2023-12-01T01:36:10.668180+0000","last_became_peered":"2023-12-01T01:36:10.668180+0000","last_unstale":"2023-12-01T01:36:11.846407+0000","last_undegraded":"2023-12-01T01:36:11.846407+0000","last_fullsized":"2023-12-01T01:36:11.846407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:33.765782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843462+0000","last_change":"2023-12-01T01:36:10.665068+0000","last_active":"2023-12-01T01:36:11.843462+0000","last_peered":"2023-12-01T01:36:11.843462+0000","last_clean":"2023-12-01T01:36:11.843462+0000","last_became_active":"2023-12-01T01:36:10.664845+0000","last_became_peered":"2023-12-01T01:36:10.664845+0000","last_unstale":"2023-12-01T01:36:11.843462+0000","last_undegraded":"2023-12-01T01:36:11.843462+0000","last_fullsized":"2023-12-01T01:36:11.843462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:27.561648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843064+0000","last_change":"2023-12-01T01:36:10.682708+0000","last_active":"2023-12-01T01:36:11.843064+0000","last_peered":"2023-12-01T01:36:11.843064+0000","last_clean":"2023-12-01T01:36:11.843064+0000","last_became_active":"2023-12-01T01:36:10.679209+0000","last_became_peered":"2023-12-01T01:36:10.679209+0000","last_unstale":"2023-12-01T01:36:11.843064+0000","last_undegraded":"2023-12-01T01:36:11.843064+0000","last_fullsized":"2023-12-01T01:36:11.843064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:25.357183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843449+0000","last_change":"2023-12-01T01:36:10.673333+0000","last_active":"2023-12-01T01:36:11.843449+0000","last_peered":"2023-12-01T01:36:11.843449+0000","last_clean":"2023-12-01T01:36:11.843449+0000","last_became_active":"2023-12-01T01:36:10.673021+0000","last_became_peered":"2023-12-01T01:36:10.673021+0000","last_unstale":"2023-12-01T01:36:11.843449+0000","last_undegraded":"2023-12-01T01:36:11.843449+0000","last_fullsized":"2023-12-01T01:36:11.843449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:24.066059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659010+0000","last_change":"2023-12-01T01:36:10.668827+0000","last_active":"2023-12-01T01:36:11.659010+0000","last_peered":"2023-12-01T01:36:11.659010+0000","last_clean":"2023-12-01T01:36:11.659010+0000","last_became_active":"2023-12-01T01:36:10.668507+0000","last_became_peered":"2023-12-01T01:36:10.668507+0000","last_unstale":"2023-12-01T01:36:11.659010+0000","last_undegraded":"2023-12-01T01:36:11.659010+0000","last_fullsized":"2023-12-01T01:36:11.659010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:29:32.707369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660612+0000","last_change":"2023-12-01T01:36:10.681970+0000","last_active":"2023-12-01T01:36:11.660612+0000","last_peered":"2023-12-01T01:36:11.660612+0000","last_clean":"2023-12-01T01:36:11.660612+0000","last_became_active":"2023-12-01T01:36:10.681726+0000","last_became_peered":"2023-12-01T01:36:10.681726+0000","last_unstale":"2023-12-01T01:36:11.660612+0000","last_undegraded":"2023-12-01T01:36:11.660612+0000","last_fullsized":"2023-12-01T01:36:11.660612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:58.572550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659519+0000","last_change":"2023-12-01T01:36:10.685524+0000","last_active":"2023-12-01T01:36:11.659519+0000","last_peered":"2023-12-01T01:36:11.659519+0000","last_clean":"2023-12-01T01:36:11.659519+0000","last_became_active":"2023-12-01T01:36:10.685227+0000","last_became_peered":"2023-12-01T01:36:10.685227+0000","last_unstale":"2023-12-01T01:36:11.659519+0000","last_undegraded":"2023-12-01T01:36:11.659519+0000","last_fullsized":"2023-12-01T01:36:11.659519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660039+0000","last_change":"2023-12-01T01:36:10.673805+0000","last_active":"2023-12-01T01:36:11.660039+0000","last_peered":"2023-12-01T01:36:11.660039+0000","last_clean":"2023-12-01T01:36:11.660039+0000","last_became_active":"2023-12-01T01:36:10.672297+0000","last_became_peered":"2023-12-01T01:36:10.672297+0000","last_unstale":"2023-12-01T01:36:11.660039+0000","last_undegraded":"2023-12-01T01:36:11.660039+0000","last_fullsized":"2023-12-01T01:36:11.660039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846363+0000","last_change":"2023-12-01T01:36:10.668469+0000","last_active":"2023-12-01T01:36:11.846363+0000","last_peered":"2023-12-01T01:36:11.846363+0000","last_clean":"2023-12-01T01:36:11.846363+0000","last_became_active":"2023-12-01T01:36:10.667900+0000","last_became_peered":"2023-12-01T01:36:10.667900+0000","last_unstale":"2023-12-01T01:36:11.846363+0000","last_undegraded":"2023-12-01T01:36:11.846363+0000","last_fullsized":"2023-12-01T01:36:11.846363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660756+0000","last_change":"2023-12-01T01:36:10.666824+0000","last_active":"2023-12-01T01:36:11.660756+0000","last_peered":"2023-12-01T01:36:11.660756+0000","last_clean":"2023-12-01T01:36:11.660756+0000","last_became_active":"2023-12-01T01:36:10.666614+0000","last_became_peered":"2023-12-01T01:36:10.666614+0000","last_unstale":"2023-12-01T01:36:11.660756+0000","last_undegraded":"2023-12-01T01:36:11.660756+0000","last_fullsized":"2023-12-01T01:36:11.660756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:02.974458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843911+0000","last_change":"2023-12-01T01:36:10.676262+0000","last_active":"2023-12-01T01:36:11.843911+0000","last_peered":"2023-12-01T01:36:11.843911+0000","last_clean":"2023-12-01T01:36:11.843911+0000","last_became_active":"2023-12-01T01:36:10.674999+0000","last_became_peered":"2023-12-01T01:36:10.674999+0000","last_unstale":"2023-12-01T01:36:11.843911+0000","last_undegraded":"2023-12-01T01:36:11.843911+0000","last_fullsized":"2023-12-01T01:36:11.843911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843925+0000","last_change":"2023-12-01T01:36:10.660058+0000","last_active":"2023-12-01T01:36:11.843925+0000","last_peered":"2023-12-01T01:36:11.843925+0000","last_clean":"2023-12-01T01:36:11.843925+0000","last_became_active":"2023-12-01T01:36:10.659667+0000","last_became_peered":"2023-12-01T01:36:10.659667+0000","last_unstale":"2023-12-01T01:36:11.843925+0000","last_undegraded":"2023-12-01T01:36:11.843925+0000","last_fullsized":"2023-12-01T01:36:11.843925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659022+0000","last_change":"2023-12-01T01:36:10.681619+0000","last_active":"2023-12-01T01:36:11.659022+0000","last_peered":"2023-12-01T01:36:11.659022+0000","last_clean":"2023-12-01T01:36:11.659022+0000","last_became_active":"2023-12-01T01:36:10.681370+0000","last_became_peered":"2023-12-01T01:36:10.681370+0000","last_unstale":"2023-12-01T01:36:11.659022+0000","last_undegraded":"2023-12-01T01:36:11.659022+0000","last_fullsized":"2023-12-01T01:36:11.659022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:05:04.661902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844932+0000","last_change":"2023-12-01T01:36:10.663979+0000","last_active":"2023-12-01T01:36:11.844932+0000","last_peered":"2023-12-01T01:36:11.844932+0000","last_clean":"2023-12-01T01:36:11.844932+0000","last_became_active":"2023-12-01T01:36:10.663730+0000","last_became_peered":"2023-12-01T01:36:10.663730+0000","last_unstale":"2023-12-01T01:36:11.844932+0000","last_undegraded":"2023-12-01T01:36:11.844932+0000","last_fullsized":"2023-12-01T01:36:11.844932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:44:55.685435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843066+0000","last_change":"2023-12-01T01:36:10.681889+0000","last_active":"2023-12-01T01:36:11.843066+0000","last_peered":"2023-12-01T01:36:11.843066+0000","last_clean":"2023-12-01T01:36:11.843066+0000","last_became_active":"2023-12-01T01:36:10.681607+0000","last_became_peered":"2023-12-01T01:36:10.681607+0000","last_unstale":"2023-12-01T01:36:11.843066+0000","last_undegraded":"2023-12-01T01:36:11.843066+0000","last_fullsized":"2023-12-01T01:36:11.843066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:10.875571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844651+0000","last_change":"2023-12-01T01:36:10.681120+0000","last_active":"2023-12-01T01:36:11.844651+0000","last_peered":"2023-12-01T01:36:11.844651+0000","last_clean":"2023-12-01T01:36:11.844651+0000","last_became_active":"2023-12-01T01:36:10.680861+0000","last_became_peered":"2023-12-01T01:36:10.680861+0000","last_unstale":"2023-12-01T01:36:11.844651+0000","last_undegraded":"2023-12-01T01:36:11.844651+0000","last_fullsized":"2023-12-01T01:36:11.844651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:14:40.275119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843591+0000","last_change":"2023-12-01T01:36:10.671802+0000","last_active":"2023-12-01T01:36:11.843591+0000","last_peered":"2023-12-01T01:36:11.843591+0000","last_clean":"2023-12-01T01:36:11.843591+0000","last_became_active":"2023-12-01T01:36:10.671492+0000","last_became_peered":"2023-12-01T01:36:10.671492+0000","last_unstale":"2023-12-01T01:36:11.843591+0000","last_undegraded":"2023-12-01T01:36:11.843591+0000","last_fullsized":"2023-12-01T01:36:11.843591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:13:49.007903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844344+0000","last_change":"2023-12-01T01:36:10.679590+0000","last_active":"2023-12-01T01:36:11.844344+0000","last_peered":"2023-12-01T01:36:11.844344+0000","last_clean":"2023-12-01T01:36:11.844344+0000","last_became_active":"2023-12-01T01:36:10.679306+0000","last_became_peered":"2023-12-01T01:36:10.679306+0000","last_unstale":"2023-12-01T01:36:11.844344+0000","last_undegraded":"2023-12-01T01:36:11.844344+0000","last_fullsized":"2023-12-01T01:36:11.844344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659556+0000","last_change":"2023-12-01T01:36:10.671990+0000","last_active":"2023-12-01T01:36:11.659556+0000","last_peered":"2023-12-01T01:36:11.659556+0000","last_clean":"2023-12-01T01:36:11.659556+0000","last_became_active":"2023-12-01T01:36:10.666648+0000","last_became_peered":"2023-12-01T01:36:10.666648+0000","last_unstale":"2023-12-01T01:36:11.659556+0000","last_undegraded":"2023-12-01T01:36:11.659556+0000","last_fullsized":"2023-12-01T01:36:11.659556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:51:53.546128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.658760+0000","last_change":"2023-12-01T01:36:10.677811+0000","last_active":"2023-12-01T01:36:11.658760+0000","last_peered":"2023-12-01T01:36:11.658760+0000","last_clean":"2023-12-01T01:36:11.658760+0000","last_became_active":"2023-12-01T01:36:10.677266+0000","last_became_peered":"2023-12-01T01:36:10.677266+0000","last_unstale":"2023-12-01T01:36:11.658760+0000","last_undegraded":"2023-12-01T01:36:11.658760+0000","last_fullsized":"2023-12-01T01:36:11.658760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:15.166766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842525+0000","last_change":"2023-12-01T01:36:10.664264+0000","last_active":"2023-12-01T01:36:11.842525+0000","last_peered":"2023-12-01T01:36:11.842525+0000","last_clean":"2023-12-01T01:36:11.842525+0000","last_became_active":"2023-12-01T01:36:10.663947+0000","last_became_peered":"2023-12-01T01:36:10.663947+0000","last_unstale":"2023-12-01T01:36:11.842525+0000","last_undegraded":"2023-12-01T01:36:11.842525+0000","last_fullsized":"2023-12-01T01:36:11.842525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:52.753589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844143+0000","last_change":"2023-12-01T01:36:10.660556+0000","last_active":"2023-12-01T01:36:11.844143+0000","last_peered":"2023-12-01T01:36:11.844143+0000","last_clean":"2023-12-01T01:36:11.844143+0000","last_became_active":"2023-12-01T01:36:10.658792+0000","last_became_peered":"2023-12-01T01:36:10.658792+0000","last_unstale":"2023-12-01T01:36:11.844143+0000","last_undegraded":"2023-12-01T01:36:11.844143+0000","last_fullsized":"2023-12-01T01:36:11.844143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:10.875571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.678963+0000","last_change":"2023-12-01T01:36:10.186029+0000","last_active":"2023-12-01T01:36:10.678963+0000","last_peered":"2023-12-01T01:36:10.678963+0000","last_clean":"2023-12-01T01:36:10.678963+0000","last_became_active":"2023-12-01T01:36:10.185703+0000","last_became_peered":"2023-12-01T01:36:10.185703+0000","last_unstale":"2023-12-01T01:36:10.678963+0000","last_undegraded":"2023-12-01T01:36:10.678963+0000","last_fullsized":"2023-12-01T01:36:10.678963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660789+0000","last_change":"2023-12-01T01:36:10.672533+0000","last_active":"2023-12-01T01:36:11.660789+0000","last_peered":"2023-12-01T01:36:11.660789+0000","last_clean":"2023-12-01T01:36:11.660789+0000","last_became_active":"2023-12-01T01:36:10.672202+0000","last_became_peered":"2023-12-01T01:36:10.672202+0000","last_unstale":"2023-12-01T01:36:11.660789+0000","last_undegraded":"2023-12-01T01:36:11.660789+0000","last_fullsized":"2023-12-01T01:36:11.660789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:49.318542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685718+0000","last_change":"2023-12-01T01:36:10.186695+0000","last_active":"2023-12-01T01:36:10.685718+0000","last_peered":"2023-12-01T01:36:10.685718+0000","last_clean":"2023-12-01T01:36:10.685718+0000","last_became_active":"2023-12-01T01:36:10.186437+0000","last_became_peered":"2023-12-01T01:36:10.186437+0000","last_unstale":"2023-12-01T01:36:10.685718+0000","last_undegraded":"2023-12-01T01:36:10.685718+0000","last_fullsized":"2023-12-01T01:36:10.685718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:52.363022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.649986+0000","last_change":"2023-12-01T01:36:10.192965+0000","last_active":"2023-12-01T01:36:10.649986+0000","last_peered":"2023-12-01T01:36:10.649986+0000","last_clean":"2023-12-01T01:36:10.649986+0000","last_became_active":"2023-12-01T01:36:10.192659+0000","last_became_peered":"2023-12-01T01:36:10.192659+0000","last_unstale":"2023-12-01T01:36:10.649986+0000","last_undegraded":"2023-12-01T01:36:10.649986+0000","last_fullsized":"2023-12-01T01:36:10.649986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844146+0000","last_change":"2023-12-01T01:36:10.665664+0000","last_active":"2023-12-01T01:36:11.844146+0000","last_peered":"2023-12-01T01:36:11.844146+0000","last_clean":"2023-12-01T01:36:11.844146+0000","last_became_active":"2023-12-01T01:36:10.662774+0000","last_became_peered":"2023-12-01T01:36:10.662774+0000","last_unstale":"2023-12-01T01:36:11.844146+0000","last_undegraded":"2023-12-01T01:36:11.844146+0000","last_fullsized":"2023-12-01T01:36:11.844146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.667386+0000","last_change":"2023-12-01T01:36:10.177490+0000","last_active":"2023-12-01T01:36:10.667386+0000","last_peered":"2023-12-01T01:36:10.667386+0000","last_clean":"2023-12-01T01:36:10.667386+0000","last_became_active":"2023-12-01T01:36:10.176976+0000","last_became_peered":"2023-12-01T01:36:10.176976+0000","last_unstale":"2023-12-01T01:36:10.667386+0000","last_undegraded":"2023-12-01T01:36:10.667386+0000","last_fullsized":"2023-12-01T01:36:10.667386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:09.863515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660588+0000","last_change":"2023-12-01T01:36:10.663142+0000","last_active":"2023-12-01T01:36:11.660588+0000","last_peered":"2023-12-01T01:36:11.660588+0000","last_clean":"2023-12-01T01:36:11.660588+0000","last_became_active":"2023-12-01T01:36:10.662866+0000","last_became_peered":"2023-12-01T01:36:10.662866+0000","last_unstale":"2023-12-01T01:36:11.660588+0000","last_undegraded":"2023-12-01T01:36:11.660588+0000","last_fullsized":"2023-12-01T01:36:11.660588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:25.357183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.647474+0000","last_change":"2023-12-01T01:36:10.189645+0000","last_active":"2023-12-01T01:36:10.647474+0000","last_peered":"2023-12-01T01:36:10.647474+0000","last_clean":"2023-12-01T01:36:10.647474+0000","last_became_active":"2023-12-01T01:36:10.189381+0000","last_became_peered":"2023-12-01T01:36:10.189381+0000","last_unstale":"2023-12-01T01:36:10.647474+0000","last_undegraded":"2023-12-01T01:36:10.647474+0000","last_fullsized":"2023-12-01T01:36:10.647474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843160+0000","last_change":"2023-12-01T01:36:10.679049+0000","last_active":"2023-12-01T01:36:11.843160+0000","last_peered":"2023-12-01T01:36:11.843160+0000","last_clean":"2023-12-01T01:36:11.843160+0000","last_became_active":"2023-12-01T01:36:10.678600+0000","last_became_peered":"2023-12-01T01:36:10.678600+0000","last_unstale":"2023-12-01T01:36:11.843160+0000","last_undegraded":"2023-12-01T01:36:11.843160+0000","last_fullsized":"2023-12-01T01:36:11.843160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650463+0000","last_change":"2023-12-01T01:36:10.190157+0000","last_active":"2023-12-01T01:36:10.650463+0000","last_peered":"2023-12-01T01:36:10.650463+0000","last_clean":"2023-12-01T01:36:10.650463+0000","last_became_active":"2023-12-01T01:36:10.189519+0000","last_became_peered":"2023-12-01T01:36:10.189519+0000","last_unstale":"2023-12-01T01:36:10.650463+0000","last_undegraded":"2023-12-01T01:36:10.650463+0000","last_fullsized":"2023-12-01T01:36:10.650463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845357+0000","last_change":"2023-12-01T01:36:10.670783+0000","last_active":"2023-12-01T01:36:11.845357+0000","last_peered":"2023-12-01T01:36:11.845357+0000","last_clean":"2023-12-01T01:36:11.845357+0000","last_became_active":"2023-12-01T01:36:10.669960+0000","last_became_peered":"2023-12-01T01:36:10.669960+0000","last_unstale":"2023-12-01T01:36:11.845357+0000","last_undegraded":"2023-12-01T01:36:11.845357+0000","last_fullsized":"2023-12-01T01:36:11.845357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:33.765782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.686400+0000","last_change":"2023-12-01T01:36:10.191392+0000","last_active":"2023-12-01T01:36:10.686400+0000","last_peered":"2023-12-01T01:36:10.686400+0000","last_clean":"2023-12-01T01:36:10.686400+0000","last_became_active":"2023-12-01T01:36:10.191010+0000","last_became_peered":"2023-12-01T01:36:10.191010+0000","last_unstale":"2023-12-01T01:36:10.686400+0000","last_undegraded":"2023-12-01T01:36:10.686400+0000","last_fullsized":"2023-12-01T01:36:10.686400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:54.003428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659710+0000","last_change":"2023-12-01T01:36:10.661986+0000","last_active":"2023-12-01T01:36:11.659710+0000","last_peered":"2023-12-01T01:36:11.659710+0000","last_clean":"2023-12-01T01:36:11.659710+0000","last_became_active":"2023-12-01T01:36:10.661711+0000","last_became_peered":"2023-12-01T01:36:10.661711+0000","last_unstale":"2023-12-01T01:36:11.659710+0000","last_undegraded":"2023-12-01T01:36:11.659710+0000","last_fullsized":"2023-12-01T01:36:11.659710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:12.241846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.688331+0000","last_change":"2023-12-01T01:36:10.190140+0000","last_active":"2023-12-01T01:36:10.688331+0000","last_peered":"2023-12-01T01:36:10.688331+0000","last_clean":"2023-12-01T01:36:10.688331+0000","last_became_active":"2023-12-01T01:36:10.188743+0000","last_became_peered":"2023-12-01T01:36:10.188743+0000","last_unstale":"2023-12-01T01:36:10.688331+0000","last_undegraded":"2023-12-01T01:36:10.688331+0000","last_fullsized":"2023-12-01T01:36:10.688331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:06:15.188480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846990+0000","last_change":"2023-12-01T01:36:10.670314+0000","last_active":"2023-12-01T01:36:11.846990+0000","last_peered":"2023-12-01T01:36:11.846990+0000","last_clean":"2023-12-01T01:36:11.846990+0000","last_became_active":"2023-12-01T01:36:10.669184+0000","last_became_peered":"2023-12-01T01:36:10.669184+0000","last_unstale":"2023-12-01T01:36:11.846990+0000","last_undegraded":"2023-12-01T01:36:11.846990+0000","last_fullsized":"2023-12-01T01:36:11.846990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.684491+0000","last_change":"2023-12-01T01:36:10.195876+0000","last_active":"2023-12-01T01:36:10.684491+0000","last_peered":"2023-12-01T01:36:10.684491+0000","last_clean":"2023-12-01T01:36:10.684491+0000","last_became_active":"2023-12-01T01:36:10.195018+0000","last_became_peered":"2023-12-01T01:36:10.195018+0000","last_unstale":"2023-12-01T01:36:10.684491+0000","last_undegraded":"2023-12-01T01:36:10.684491+0000","last_fullsized":"2023-12-01T01:36:10.684491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843370+0000","last_change":"2023-12-01T01:36:10.677075+0000","last_active":"2023-12-01T01:36:11.843370+0000","last_peered":"2023-12-01T01:36:11.843370+0000","last_clean":"2023-12-01T01:36:11.843370+0000","last_became_active":"2023-12-01T01:36:10.676560+0000","last_became_peered":"2023-12-01T01:36:10.676560+0000","last_unstale":"2023-12-01T01:36:11.843370+0000","last_undegraded":"2023-12-01T01:36:11.843370+0000","last_fullsized":"2023-12-01T01:36:11.843370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:41.649014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.660745+0000","last_change":"2023-12-01T01:36:10.179441+0000","last_active":"2023-12-01T01:36:10.660745+0000","last_peered":"2023-12-01T01:36:10.660745+0000","last_clean":"2023-12-01T01:36:10.660745+0000","last_became_active":"2023-12-01T01:36:10.178179+0000","last_became_peered":"2023-12-01T01:36:10.178179+0000","last_unstale":"2023-12-01T01:36:10.660745+0000","last_undegraded":"2023-12-01T01:36:10.660745+0000","last_fullsized":"2023-12-01T01:36:10.660745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:14.154709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659624+0000","last_change":"2023-12-01T01:36:10.665880+0000","last_active":"2023-12-01T01:36:11.659624+0000","last_peered":"2023-12-01T01:36:11.659624+0000","last_clean":"2023-12-01T01:36:11.659624+0000","last_became_active":"2023-12-01T01:36:10.665639+0000","last_became_peered":"2023-12-01T01:36:10.665639+0000","last_unstale":"2023-12-01T01:36:11.659624+0000","last_undegraded":"2023-12-01T01:36:11.659624+0000","last_fullsized":"2023-12-01T01:36:11.659624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:47.015657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650017+0000","last_change":"2023-12-01T01:36:10.192976+0000","last_active":"2023-12-01T01:36:10.650017+0000","last_peered":"2023-12-01T01:36:10.650017+0000","last_clean":"2023-12-01T01:36:10.650017+0000","last_became_active":"2023-12-01T01:36:10.192692+0000","last_became_peered":"2023-12-01T01:36:10.192692+0000","last_unstale":"2023-12-01T01:36:10.650017+0000","last_undegraded":"2023-12-01T01:36:10.650017+0000","last_fullsized":"2023-12-01T01:36:10.650017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843415+0000","last_change":"2023-12-01T01:36:10.659027+0000","last_active":"2023-12-01T01:36:11.843415+0000","last_peered":"2023-12-01T01:36:11.843415+0000","last_clean":"2023-12-01T01:36:11.843415+0000","last_became_active":"2023-12-01T01:36:10.658683+0000","last_became_peered":"2023-12-01T01:36:10.658683+0000","last_unstale":"2023-12-01T01:36:11.843415+0000","last_undegraded":"2023-12-01T01:36:11.843415+0000","last_fullsized":"2023-12-01T01:36:11.843415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.687400+0000","last_change":"2023-12-01T01:36:10.180654+0000","last_active":"2023-12-01T01:36:10.687400+0000","last_peered":"2023-12-01T01:36:10.687400+0000","last_clean":"2023-12-01T01:36:10.687400+0000","last_became_active":"2023-12-01T01:36:10.179301+0000","last_became_peered":"2023-12-01T01:36:10.179301+0000","last_unstale":"2023-12-01T01:36:10.687400+0000","last_undegraded":"2023-12-01T01:36:10.687400+0000","last_fullsized":"2023-12-01T01:36:10.687400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659085+0000","last_change":"2023-12-01T01:36:10.664772+0000","last_active":"2023-12-01T01:36:11.659085+0000","last_peered":"2023-12-01T01:36:11.659085+0000","last_clean":"2023-12-01T01:36:11.659085+0000","last_became_active":"2023-12-01T01:36:10.664504+0000","last_became_peered":"2023-12-01T01:36:10.664504+0000","last_unstale":"2023-12-01T01:36:11.659085+0000","last_undegraded":"2023-12-01T01:36:11.659085+0000","last_fullsized":"2023-12-01T01:36:11.659085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:44:20.605895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.686349+0000","last_change":"2023-12-01T01:36:10.189157+0000","last_active":"2023-12-01T01:36:10.686349+0000","last_peered":"2023-12-01T01:36:10.686349+0000","last_clean":"2023-12-01T01:36:10.686349+0000","last_became_active":"2023-12-01T01:36:10.188580+0000","last_became_peered":"2023-12-01T01:36:10.188580+0000","last_unstale":"2023-12-01T01:36:10.686349+0000","last_undegraded":"2023-12-01T01:36:10.686349+0000","last_fullsized":"2023-12-01T01:36:10.686349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843263+0000","last_change":"2023-12-01T01:36:10.666504+0000","last_active":"2023-12-01T01:36:11.843263+0000","last_peered":"2023-12-01T01:36:11.843263+0000","last_clean":"2023-12-01T01:36:11.843263+0000","last_became_active":"2023-12-01T01:36:10.666252+0000","last_became_peered":"2023-12-01T01:36:10.666252+0000","last_unstale":"2023-12-01T01:36:11.843263+0000","last_undegraded":"2023-12-01T01:36:11.843263+0000","last_fullsized":"2023-12-01T01:36:11.843263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:41.649014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683819+0000","last_change":"2023-12-01T01:36:10.186429+0000","last_active":"2023-12-01T01:36:10.683819+0000","last_peered":"2023-12-01T01:36:10.683819+0000","last_clean":"2023-12-01T01:36:10.683819+0000","last_became_active":"2023-12-01T01:36:10.185294+0000","last_became_peered":"2023-12-01T01:36:10.185294+0000","last_unstale":"2023-12-01T01:36:10.683819+0000","last_undegraded":"2023-12-01T01:36:10.683819+0000","last_fullsized":"2023-12-01T01:36:10.683819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.847638+0000","last_change":"2023-12-01T01:36:10.667398+0000","last_active":"2023-12-01T01:36:11.847638+0000","last_peered":"2023-12-01T01:36:11.847638+0000","last_clean":"2023-12-01T01:36:11.847638+0000","last_became_active":"2023-12-01T01:36:10.667108+0000","last_became_peered":"2023-12-01T01:36:10.667108+0000","last_unstale":"2023-12-01T01:36:11.847638+0000","last_undegraded":"2023-12-01T01:36:11.847638+0000","last_fullsized":"2023-12-01T01:36:11.847638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:41:51.516524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.680767+0000","last_change":"2023-12-01T01:36:10.198699+0000","last_active":"2023-12-01T01:36:10.680767+0000","last_peered":"2023-12-01T01:36:10.680767+0000","last_clean":"2023-12-01T01:36:10.680767+0000","last_became_active":"2023-12-01T01:36:10.198340+0000","last_became_peered":"2023-12-01T01:36:10.198340+0000","last_unstale":"2023-12-01T01:36:10.680767+0000","last_undegraded":"2023-12-01T01:36:10.680767+0000","last_fullsized":"2023-12-01T01:36:10.680767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846471+0000","last_change":"2023-12-01T01:36:10.680027+0000","last_active":"2023-12-01T01:36:11.846471+0000","last_peered":"2023-12-01T01:36:11.846471+0000","last_clean":"2023-12-01T01:36:11.846471+0000","last_became_active":"2023-12-01T01:36:10.679670+0000","last_became_peered":"2023-12-01T01:36:10.679670+0000","last_unstale":"2023-12-01T01:36:11.846471+0000","last_undegraded":"2023-12-01T01:36:11.846471+0000","last_fullsized":"2023-12-01T01:36:11.846471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:29:32.786541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.657130+0000","last_change":"2023-12-01T01:36:10.194952+0000","last_active":"2023-12-01T01:36:10.657130+0000","last_peered":"2023-12-01T01:36:10.657130+0000","last_clean":"2023-12-01T01:36:10.657130+0000","last_became_active":"2023-12-01T01:36:10.194708+0000","last_became_peered":"2023-12-01T01:36:10.194708+0000","last_unstale":"2023-12-01T01:36:10.657130+0000","last_undegraded":"2023-12-01T01:36:10.657130+0000","last_fullsized":"2023-12-01T01:36:10.657130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659619+0000","last_change":"2023-12-01T01:36:10.663313+0000","last_active":"2023-12-01T01:36:11.659619+0000","last_peered":"2023-12-01T01:36:11.659619+0000","last_clean":"2023-12-01T01:36:11.659619+0000","last_became_active":"2023-12-01T01:36:10.662987+0000","last_became_peered":"2023-12-01T01:36:10.662987+0000","last_unstale":"2023-12-01T01:36:11.659619+0000","last_undegraded":"2023-12-01T01:36:11.659619+0000","last_fullsized":"2023-12-01T01:36:11.659619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.646527+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:36:10.646527+0000","last_peered":"2023-12-01T01:36:10.646527+0000","last_clean":"2023-12-01T01:36:10.646527+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:36:10.646527+0000","last_undegraded":"2023-12-01T01:36:10.646527+0000","last_fullsized":"2023-12-01T01:36:10.646527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.664313+0000","last_change":"2023-12-01T01:36:10.194117+0000","last_active":"2023-12-01T01:36:10.664313+0000","last_peered":"2023-12-01T01:36:10.664313+0000","last_clean":"2023-12-01T01:36:10.664313+0000","last_became_active":"2023-12-01T01:36:10.193801+0000","last_became_peered":"2023-12-01T01:36:10.193801+0000","last_unstale":"2023-12-01T01:36:10.664313+0000","last_undegraded":"2023-12-01T01:36:10.664313+0000","last_fullsized":"2023-12-01T01:36:10.664313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660654+0000","last_change":"2023-12-01T01:36:10.669848+0000","last_active":"2023-12-01T01:36:11.660654+0000","last_peered":"2023-12-01T01:36:11.660654+0000","last_clean":"2023-12-01T01:36:11.660654+0000","last_became_active":"2023-12-01T01:36:10.669557+0000","last_became_peered":"2023-12-01T01:36:10.669557+0000","last_unstale":"2023-12-01T01:36:11.660654+0000","last_undegraded":"2023-12-01T01:36:11.660654+0000","last_fullsized":"2023-12-01T01:36:11.660654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:19.318402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.667473+0000","last_change":"2023-12-01T01:36:10.183096+0000","last_active":"2023-12-01T01:36:10.667473+0000","last_peered":"2023-12-01T01:36:10.667473+0000","last_clean":"2023-12-01T01:36:10.667473+0000","last_became_active":"2023-12-01T01:36:10.182810+0000","last_became_peered":"2023-12-01T01:36:10.182810+0000","last_unstale":"2023-12-01T01:36:10.667473+0000","last_undegraded":"2023-12-01T01:36:10.667473+0000","last_fullsized":"2023-12-01T01:36:10.667473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:35.765727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844076+0000","last_change":"2023-12-01T01:36:10.661869+0000","last_active":"2023-12-01T01:36:11.844076+0000","last_peered":"2023-12-01T01:36:11.844076+0000","last_clean":"2023-12-01T01:36:11.844076+0000","last_became_active":"2023-12-01T01:36:10.661404+0000","last_became_peered":"2023-12-01T01:36:10.661404+0000","last_unstale":"2023-12-01T01:36:11.844076+0000","last_undegraded":"2023-12-01T01:36:11.844076+0000","last_fullsized":"2023-12-01T01:36:11.844076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.669601+0000","last_change":"2023-12-01T01:36:10.188465+0000","last_active":"2023-12-01T01:36:10.669601+0000","last_peered":"2023-12-01T01:36:10.669601+0000","last_clean":"2023-12-01T01:36:10.669601+0000","last_became_active":"2023-12-01T01:36:10.188157+0000","last_became_peered":"2023-12-01T01:36:10.188157+0000","last_unstale":"2023-12-01T01:36:10.669601+0000","last_undegraded":"2023-12-01T01:36:10.669601+0000","last_fullsized":"2023-12-01T01:36:10.669601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:26.175979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843928+0000","last_change":"2023-12-01T01:36:10.679247+0000","last_active":"2023-12-01T01:36:11.843928+0000","last_peered":"2023-12-01T01:36:11.843928+0000","last_clean":"2023-12-01T01:36:11.843928+0000","last_became_active":"2023-12-01T01:36:10.678097+0000","last_became_peered":"2023-12-01T01:36:10.678097+0000","last_unstale":"2023-12-01T01:36:11.843928+0000","last_undegraded":"2023-12-01T01:36:11.843928+0000","last_fullsized":"2023-12-01T01:36:11.843928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:35.730773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.661362+0000","last_change":"2023-12-01T01:36:10.182533+0000","last_active":"2023-12-01T01:36:10.661362+0000","last_peered":"2023-12-01T01:36:10.661362+0000","last_clean":"2023-12-01T01:36:10.661362+0000","last_became_active":"2023-12-01T01:36:10.182071+0000","last_became_peered":"2023-12-01T01:36:10.182071+0000","last_unstale":"2023-12-01T01:36:10.661362+0000","last_undegraded":"2023-12-01T01:36:10.661362+0000","last_fullsized":"2023-12-01T01:36:10.661362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:40.636957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.846563+0000","last_change":"2023-12-01T01:36:10.666142+0000","last_active":"2023-12-01T01:36:11.846563+0000","last_peered":"2023-12-01T01:36:11.846563+0000","last_clean":"2023-12-01T01:36:11.846563+0000","last_became_active":"2023-12-01T01:36:10.665873+0000","last_became_peered":"2023-12-01T01:36:10.665873+0000","last_unstale":"2023-12-01T01:36:11.846563+0000","last_undegraded":"2023-12-01T01:36:11.846563+0000","last_fullsized":"2023-12-01T01:36:11.846563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.660940+0000","last_change":"2023-12-01T01:36:10.190758+0000","last_active":"2023-12-01T01:36:10.660940+0000","last_peered":"2023-12-01T01:36:10.660940+0000","last_clean":"2023-12-01T01:36:10.660940+0000","last_became_active":"2023-12-01T01:36:10.190280+0000","last_became_peered":"2023-12-01T01:36:10.190280+0000","last_unstale":"2023-12-01T01:36:10.660940+0000","last_undegraded":"2023-12-01T01:36:10.660940+0000","last_fullsized":"2023-12-01T01:36:10.660940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.842977+0000","last_change":"2023-12-01T01:36:10.665004+0000","last_active":"2023-12-01T01:36:11.842977+0000","last_peered":"2023-12-01T01:36:11.842977+0000","last_clean":"2023-12-01T01:36:11.842977+0000","last_became_active":"2023-12-01T01:36:10.664692+0000","last_became_peered":"2023-12-01T01:36:10.664692+0000","last_unstale":"2023-12-01T01:36:11.842977+0000","last_undegraded":"2023-12-01T01:36:11.842977+0000","last_fullsized":"2023-12-01T01:36:11.842977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:53.375079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.666452+0000","last_change":"2023-12-01T01:36:10.186489+0000","last_active":"2023-12-01T01:36:10.666452+0000","last_peered":"2023-12-01T01:36:10.666452+0000","last_clean":"2023-12-01T01:36:10.666452+0000","last_became_active":"2023-12-01T01:36:10.186161+0000","last_became_peered":"2023-12-01T01:36:10.186161+0000","last_unstale":"2023-12-01T01:36:10.666452+0000","last_undegraded":"2023-12-01T01:36:10.666452+0000","last_fullsized":"2023-12-01T01:36:10.666452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.843867+0000","last_change":"2023-12-01T01:36:10.663707+0000","last_active":"2023-12-01T01:36:11.843867+0000","last_peered":"2023-12-01T01:36:11.843867+0000","last_clean":"2023-12-01T01:36:11.843867+0000","last_became_active":"2023-12-01T01:36:10.663158+0000","last_became_peered":"2023-12-01T01:36:10.663158+0000","last_unstale":"2023-12-01T01:36:11.843867+0000","last_undegraded":"2023-12-01T01:36:11.843867+0000","last_fullsized":"2023-12-01T01:36:11.843867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:07:36.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661244+0000","last_change":"2023-12-01T01:36:10.188465+0000","last_active":"2023-12-01T01:36:11.661244+0000","last_peered":"2023-12-01T01:36:11.661244+0000","last_clean":"2023-12-01T01:36:11.661244+0000","last_became_active":"2023-12-01T01:36:10.187631+0000","last_became_peered":"2023-12-01T01:36:10.187631+0000","last_unstale":"2023-12-01T01:36:11.661244+0000","last_undegraded":"2023-12-01T01:36:11.661244+0000","last_fullsized":"2023-12-01T01:36:11.661244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661611+0000","last_change":"2023-12-01T01:36:10.665647+0000","last_active":"2023-12-01T01:36:11.661611+0000","last_peered":"2023-12-01T01:36:11.661611+0000","last_clean":"2023-12-01T01:36:11.661611+0000","last_became_active":"2023-12-01T01:36:10.665371+0000","last_became_peered":"2023-12-01T01:36:10.665371+0000","last_unstale":"2023-12-01T01:36:11.661611+0000","last_undegraded":"2023-12-01T01:36:11.661611+0000","last_fullsized":"2023-12-01T01:36:11.661611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:01:37.815228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685898+0000","last_change":"2023-12-01T01:36:10.194331+0000","last_active":"2023-12-01T01:36:10.685898+0000","last_peered":"2023-12-01T01:36:10.685898+0000","last_clean":"2023-12-01T01:36:10.685898+0000","last_became_active":"2023-12-01T01:36:10.194058+0000","last_became_peered":"2023-12-01T01:36:10.194058+0000","last_unstale":"2023-12-01T01:36:10.685898+0000","last_undegraded":"2023-12-01T01:36:10.685898+0000","last_fullsized":"2023-12-01T01:36:10.685898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:12:51.741532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.661128+0000","last_change":"2023-12-01T01:36:10.680546+0000","last_active":"2023-12-01T01:36:11.661128+0000","last_peered":"2023-12-01T01:36:11.661128+0000","last_clean":"2023-12-01T01:36:11.661128+0000","last_became_active":"2023-12-01T01:36:10.680099+0000","last_became_peered":"2023-12-01T01:36:10.680099+0000","last_unstale":"2023-12-01T01:36:11.661128+0000","last_undegraded":"2023-12-01T01:36:11.661128+0000","last_fullsized":"2023-12-01T01:36:11.661128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:22.177011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.650417+0000","last_change":"2023-12-01T01:36:10.193127+0000","last_active":"2023-12-01T01:36:10.650417+0000","last_peered":"2023-12-01T01:36:10.650417+0000","last_clean":"2023-12-01T01:36:10.650417+0000","last_became_active":"2023-12-01T01:36:10.192884+0000","last_became_peered":"2023-12-01T01:36:10.192884+0000","last_unstale":"2023-12-01T01:36:10.650417+0000","last_undegraded":"2023-12-01T01:36:10.650417+0000","last_fullsized":"2023-12-01T01:36:10.650417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659185+0000","last_change":"2023-12-01T01:36:10.677457+0000","last_active":"2023-12-01T01:36:11.659185+0000","last_peered":"2023-12-01T01:36:11.659185+0000","last_clean":"2023-12-01T01:36:11.659185+0000","last_became_active":"2023-12-01T01:36:10.677156+0000","last_became_peered":"2023-12-01T01:36:10.677156+0000","last_unstale":"2023-12-01T01:36:11.659185+0000","last_undegraded":"2023-12-01T01:36:11.659185+0000","last_fullsized":"2023-12-01T01:36:11.659185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:41.113753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.654051+0000","last_change":"2023-12-01T01:36:10.189035+0000","last_active":"2023-12-01T01:36:10.654051+0000","last_peered":"2023-12-01T01:36:10.654051+0000","last_clean":"2023-12-01T01:36:10.654051+0000","last_became_active":"2023-12-01T01:36:10.188370+0000","last_became_peered":"2023-12-01T01:36:10.188370+0000","last_unstale":"2023-12-01T01:36:10.654051+0000","last_undegraded":"2023-12-01T01:36:10.654051+0000","last_fullsized":"2023-12-01T01:36:10.654051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:41.082838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.660520+0000","last_change":"2023-12-01T01:36:10.682836+0000","last_active":"2023-12-01T01:36:11.660520+0000","last_peered":"2023-12-01T01:36:11.660520+0000","last_clean":"2023-12-01T01:36:11.660520+0000","last_became_active":"2023-12-01T01:36:10.682361+0000","last_became_peered":"2023-12-01T01:36:10.682361+0000","last_unstale":"2023-12-01T01:36:11.660520+0000","last_undegraded":"2023-12-01T01:36:11.660520+0000","last_fullsized":"2023-12-01T01:36:11.660520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:27.561648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.651403+0000","last_change":"2023-12-01T01:36:10.190808+0000","last_active":"2023-12-01T01:36:10.651403+0000","last_peered":"2023-12-01T01:36:10.651403+0000","last_clean":"2023-12-01T01:36:10.651403+0000","last_became_active":"2023-12-01T01:36:10.190355+0000","last_became_peered":"2023-12-01T01:36:10.190355+0000","last_unstale":"2023-12-01T01:36:10.651403+0000","last_undegraded":"2023-12-01T01:36:10.651403+0000","last_fullsized":"2023-12-01T01:36:10.651403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:52:48.583591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659213+0000","last_change":"2023-12-01T01:36:10.685203+0000","last_active":"2023-12-01T01:36:11.659213+0000","last_peered":"2023-12-01T01:36:11.659213+0000","last_clean":"2023-12-01T01:36:11.659213+0000","last_became_active":"2023-12-01T01:36:10.684118+0000","last_became_peered":"2023-12-01T01:36:10.684118+0000","last_unstale":"2023-12-01T01:36:11.659213+0000","last_undegraded":"2023-12-01T01:36:11.659213+0000","last_fullsized":"2023-12-01T01:36:11.659213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:30.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683327+0000","last_change":"2023-12-01T01:36:10.194065+0000","last_active":"2023-12-01T01:36:10.683327+0000","last_peered":"2023-12-01T01:36:10.683327+0000","last_clean":"2023-12-01T01:36:10.683327+0000","last_became_active":"2023-12-01T01:36:10.193785+0000","last_became_peered":"2023-12-01T01:36:10.193785+0000","last_unstale":"2023-12-01T01:36:10.683327+0000","last_undegraded":"2023-12-01T01:36:10.683327+0000","last_fullsized":"2023-12-01T01:36:10.683327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:13:29.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659511+0000","last_change":"2023-12-01T01:36:10.654568+0000","last_active":"2023-12-01T01:36:11.659511+0000","last_peered":"2023-12-01T01:36:11.659511+0000","last_clean":"2023-12-01T01:36:11.659511+0000","last_became_active":"2023-12-01T01:36:10.654312+0000","last_became_peered":"2023-12-01T01:36:10.654312+0000","last_unstale":"2023-12-01T01:36:11.659511+0000","last_undegraded":"2023-12-01T01:36:11.659511+0000","last_fullsized":"2023-12-01T01:36:11.659511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:24.066059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.659518+0000","last_change":"2023-12-01T01:36:10.672526+0000","last_active":"2023-12-01T01:36:11.659518+0000","last_peered":"2023-12-01T01:36:11.659518+0000","last_clean":"2023-12-01T01:36:11.659518+0000","last_became_active":"2023-12-01T01:36:10.671950+0000","last_became_peered":"2023-12-01T01:36:10.671950+0000","last_unstale":"2023-12-01T01:36:11.659518+0000","last_undegraded":"2023-12-01T01:36:11.659518+0000","last_fullsized":"2023-12-01T01:36:11.659518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:07.041952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.681098+0000","last_change":"2023-12-01T01:36:10.189066+0000","last_active":"2023-12-01T01:36:10.681098+0000","last_peered":"2023-12-01T01:36:10.681098+0000","last_clean":"2023-12-01T01:36:10.681098+0000","last_became_active":"2023-12-01T01:36:10.188395+0000","last_became_peered":"2023-12-01T01:36:10.188395+0000","last_unstale":"2023-12-01T01:36:10.681098+0000","last_undegraded":"2023-12-01T01:36:10.681098+0000","last_fullsized":"2023-12-01T01:36:10.681098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.844042+0000","last_change":"2023-12-01T01:36:10.677457+0000","last_active":"2023-12-01T01:36:11.844042+0000","last_peered":"2023-12-01T01:36:11.844042+0000","last_clean":"2023-12-01T01:36:11.844042+0000","last_became_active":"2023-12-01T01:36:10.677200+0000","last_became_peered":"2023-12-01T01:36:10.677200+0000","last_unstale":"2023-12-01T01:36:11.844042+0000","last_undegraded":"2023-12-01T01:36:11.844042+0000","last_fullsized":"2023-12-01T01:36:11.844042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:45:53.375079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.681045+0000","last_change":"2023-12-01T01:36:10.193800+0000","last_active":"2023-12-01T01:36:10.681045+0000","last_peered":"2023-12-01T01:36:10.681045+0000","last_clean":"2023-12-01T01:36:10.681045+0000","last_became_active":"2023-12-01T01:36:10.193469+0000","last_became_peered":"2023-12-01T01:36:10.193469+0000","last_unstale":"2023-12-01T01:36:10.681045+0000","last_undegraded":"2023-12-01T01:36:10.681045+0000","last_fullsized":"2023-12-01T01:36:10.681045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:21.560661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845128+0000","last_change":"2023-12-01T01:36:10.665085+0000","last_active":"2023-12-01T01:36:11.845128+0000","last_peered":"2023-12-01T01:36:11.845128+0000","last_clean":"2023-12-01T01:36:11.845128+0000","last_became_active":"2023-12-01T01:36:10.663166+0000","last_became_peered":"2023-12-01T01:36:10.663166+0000","last_unstale":"2023-12-01T01:36:11.845128+0000","last_undegraded":"2023-12-01T01:36:11.845128+0000","last_fullsized":"2023-12-01T01:36:11.845128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:37:38.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.683819+0000","last_change":"2023-12-01T01:36:10.188667+0000","last_active":"2023-12-01T01:36:10.683819+0000","last_peered":"2023-12-01T01:36:10.683819+0000","last_clean":"2023-12-01T01:36:10.683819+0000","last_became_active":"2023-12-01T01:36:10.188354+0000","last_became_peered":"2023-12-01T01:36:10.188354+0000","last_unstale":"2023-12-01T01:36:10.683819+0000","last_undegraded":"2023-12-01T01:36:10.683819+0000","last_fullsized":"2023-12-01T01:36:10.683819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:43.902076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-01T01:36:11.845005+0000","last_change":"2023-12-01T01:36:10.660331+0000","last_active":"2023-12-01T01:36:11.845005+0000","last_peered":"2023-12-01T01:36:11.845005+0000","last_clean":"2023-12-01T01:36:11.845005+0000","last_became_active":"2023-12-01T01:36:10.660037+0000","last_became_peered":"2023-12-01T01:36:10.660037+0000","last_unstale":"2023-12-01T01:36:11.845005+0000","last_undegraded":"2023-12-01T01:36:11.845005+0000","last_fullsized":"2023-12-01T01:36:11.845005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:10.140343+0000","last_clean_scrub_stamp":"2023-12-01T01:36:10.140343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:12.241846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-01T01:36:10.685486+0000","last_change":"2023-12-01T01:36:10.189468+0000","last_active":"2023-12-01T01:36:10.685486+0000","last_peered":"2023-12-01T01:36:10.685486+0000","last_clean":"2023-12-01T01:36:10.685486+0000","last_became_active":"2023-12-01T01:36:10.187185+0000","last_became_peered":"2023-12-01T01:36:10.187185+0000","last_unstale":"2023-12-01T01:36:10.685486+0000","last_undegraded":"2023-12-01T01:36:10.685486+0000","last_fullsized":"2023-12-01T01:36:10.685486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:09.128286+0000","last_clean_scrub_stamp":"2023-12-01T01:36:09.128286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:21:21.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":75693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":819200,"data_stored":666029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":77805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":76151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.883}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7729999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.599}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.845}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.738}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":815104,"data_stored":666007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":75625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.25}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:36:27.846 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-01T01:36:27.846 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-01T01:36:27.846 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-01T01:36:27.847 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-01T01:36:28.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.095+0000 7f3c60641640 1 Processor -- start 2023-12-01T01:36:28.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.096+0000 7f3c60641640 1 -- start start 2023-12-01T01:36:28.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.097+0000 7f3c60641640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c5815cd20 0x7f3c580d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.097+0000 7f3c60641640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3c580d1230 0x7f3c580d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.097+0000 7f3c60641640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c580ce2e0 con 0x7f3c580b5d80 2023-12-01T01:36:28.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.097+0000 7f3c60641640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c580b2160 con 0x7f3c580d1230 2023-12-01T01:36:28.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.097+0000 7f3c60641640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580d1be0 con 0x7f3c5815cd20 2023-12-01T01:36:28.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.097+0000 7f3c5ebde640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3c580d1230 0x7f3c580d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.097+0000 7f3c5ebde640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3c580d1230 0x7f3c580d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60806/0 (socket says 172.21.15.115:60806) 2023-12-01T01:36:28.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.097+0000 7f3c5ebde640 1 -- 172.21.15.115:0/4256394268 learned_addr learned my addr 172.21.15.115:0/4256394268 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:28.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.098+0000 7f3c5e3dd640 1 --2- 172.21.15.115:0/4256394268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c5815cd20 0x7f3c580d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.098+0000 7f3c5ebde640 1 -- 172.21.15.115:0/4256394268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c5815cd20 msgr2=0x7f3c580d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:28.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.098+0000 7f3c5ebde640 1 --2- 172.21.15.115:0/4256394268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c5815cd20 0x7f3c580d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.098+0000 7f3c5ebde640 1 -- 172.21.15.115:0/4256394268 >> v1:172.21.15.142:6789/0 conn(0x7f3c580b5d80 legacy=0x7f3c580b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.098+0000 7f3c5ebde640 1 -- 172.21.15.115:0/4256394268 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c580ce000 con 0x7f3c580d1230 2023-12-01T01:36:28.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.099+0000 7f3c5ebde640 1 --2- 172.21.15.115:0/4256394268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c580d1230 0x7f3c580d16a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009e50 tx=0x7f3c5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eebdc502210f48 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:28.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.099+0000 7f3c5d3db640 1 -- 172.21.15.115:0/4256394268 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5002ca70 con 0x7f3c580d1230 2023-12-01T01:36:28.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.099+0000 7f3c5d3db640 1 -- 172.21.15.115:0/4256394268 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5002cbd0 con 0x7f3c580d1230 2023-12-01T01:36:28.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.099+0000 7f3c5d3db640 1 -- 172.21.15.115:0/4256394268 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500343f0 con 0x7f3c580d1230 2023-12-01T01:36:28.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.100+0000 7f3c60641640 1 -- 172.21.15.115:0/4256394268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c580d1230 msgr2=0x7f3c580d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.100+0000 7f3c60641640 1 --2- 172.21.15.115:0/4256394268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c580d1230 0x7f3c580d16a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009e50 tx=0x7f3c5002bed0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.100+0000 7f3c60641640 1 -- 172.21.15.115:0/4256394268 shutdown_connections 2023-12-01T01:36:28.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.100+0000 7f3c60641640 1 --2- 172.21.15.115:0/4256394268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c580d1230 0x7f3c580d16a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.100+0000 7f3c60641640 1 --2- 172.21.15.115:0/4256394268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c5815cd20 0x7f3c580d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.100+0000 7f3c60641640 1 -- 172.21.15.115:0/4256394268 >> 172.21.15.115:0/4256394268 conn(0x7f3c5806a0e0 msgr2=0x7f3c5806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:28.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.100+0000 7f3c60641640 1 -- 172.21.15.115:0/4256394268 shutdown_connections 2023-12-01T01:36:28.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.100+0000 7f3c60641640 1 -- 172.21.15.115:0/4256394268 wait complete. 2023-12-01T01:36:28.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.102+0000 7f3c60641640 1 Processor -- start 2023-12-01T01:36:28.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.102+0000 7f3c60641640 1 -- start start 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.103+0000 7f3c60641640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c580b5d80 0x7f3c580b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.103+0000 7f3c60641640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c580d1230 0x7f3c580b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.103+0000 7f3c60641640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5815cd20 0x7f3c580b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.103+0000 7f3c60641640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580c6440 con 0x7f3c5815cd20 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.103+0000 7f3c60641640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580b2de0 con 0x7f3c580b5d80 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.103+0000 7f3c60641640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580cc9a0 con 0x7f3c580d1230 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.103+0000 7f3c5ebde640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5815cd20 0x7f3c580b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.103+0000 7f3c5ebde640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5815cd20 0x7f3c580b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60812/0 (socket says 172.21.15.115:60812) 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.103+0000 7f3c5ebde640 1 -- 172.21.15.115:0/2087344134 learned_addr learned my addr 172.21.15.115:0/2087344134 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c5dbdc640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c580d1230 0x7f3c580b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c5e3dd640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c580b5d80 0x7f3c580b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c5ebde640 1 -- 172.21.15.115:0/2087344134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c580d1230 msgr2=0x7f3c580b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c5ebde640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c580d1230 0x7f3c580b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c5ebde640 1 -- 172.21.15.115:0/2087344134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c580b5d80 msgr2=0x7f3c580b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c5ebde640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c580b5d80 0x7f3c580b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c5ebde640 1 -- 172.21.15.115:0/2087344134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c50009b30 con 0x7f3c5815cd20 2023-12-01T01:36:28.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c5ebde640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5815cd20 0x7f3c580b4720 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3c50036040 tx=0x7f3c50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:28.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c4effd640 1 -- 172.21.15.115:0/2087344134 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50009510 con 0x7f3c5815cd20 2023-12-01T01:36:28.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c4effd640 1 -- 172.21.15.115:0/2087344134 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5003e450 con 0x7f3c5815cd20 2023-12-01T01:36:28.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c60641640 1 -- 172.21.15.115:0/2087344134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c580cbd90 con 0x7f3c5815cd20 2023-12-01T01:36:28.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.104+0000 7f3c4effd640 1 -- 172.21.15.115:0/2087344134 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5003e800 con 0x7f3c5815cd20 2023-12-01T01:36:28.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.105+0000 7f3c60641640 1 -- 172.21.15.115:0/2087344134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c580cc350 con 0x7f3c5815cd20 2023-12-01T01:36:28.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.106+0000 7f3c60641640 1 -- 172.21.15.115:0/2087344134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c580ccb10 con 0x7f3c5815cd20 2023-12-01T01:36:28.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.107+0000 7f3c4effd640 1 -- 172.21.15.115:0/2087344134 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3c5003e960 con 0x7f3c5815cd20 2023-12-01T01:36:28.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.107+0000 7f3c4effd640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3c3c06a430 0x7f3c3c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.108+0000 7f3c4effd640 1 -- 172.21.15.115:0/2087344134 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c500aba40 con 0x7f3c5815cd20 2023-12-01T01:36:28.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.108+0000 7f3c5e3dd640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3c3c06a430 0x7f3c3c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.108+0000 7f3c5e3dd640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3c3c06a430 0x7f3c3c06c8f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c54006fd0 tx=0x7f3c54008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:28.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.112+0000 7f3c4effd640 1 -- 172.21.15.115:0/2087344134 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3c50076f90 con 0x7f3c5815cd20 2023-12-01T01:36:28.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.340+0000 7f3c60641640 1 -- 172.21.15.115:0/2087344134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3c5804e890 con 0x7f3c5815cd20 2023-12-01T01:36:28.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.341+0000 7f3c4effd640 1 -- 172.21.15.115:0/2087344134 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3c50043040 con 0x7f3c5815cd20 2023-12-01T01:36:28.341 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:28.341 INFO:teuthology.orchestra.run.smithi115.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-01T01:36:28.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.343+0000 7f3c60641640 1 -- 172.21.15.115:0/2087344134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3c3c06a430 msgr2=0x7f3c3c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.343+0000 7f3c60641640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3c3c06a430 0x7f3c3c06c8f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3c54006fd0 tx=0x7f3c54008040 comp rx=0 tx=0).stop 2023-12-01T01:36:28.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.343+0000 7f3c60641640 1 -- 172.21.15.115:0/2087344134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5815cd20 msgr2=0x7f3c580b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.343+0000 7f3c60641640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5815cd20 0x7f3c580b4720 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3c50036040 tx=0x7f3c50004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:28.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.344+0000 7f3c60641640 1 -- 172.21.15.115:0/2087344134 shutdown_connections 2023-12-01T01:36:28.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.344+0000 7f3c60641640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3c3c06a430 0x7f3c3c06c8f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.344+0000 7f3c60641640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c580d1230 0x7f3c580b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.344+0000 7f3c60641640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c5815cd20 0x7f3c580b4720 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.344+0000 7f3c60641640 1 --2- 172.21.15.115:0/2087344134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c580b5d80 0x7f3c580b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.344+0000 7f3c60641640 1 -- 172.21.15.115:0/2087344134 >> 172.21.15.115:0/2087344134 conn(0x7f3c5806a0e0 msgr2=0x7f3c58157500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:28.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.344+0000 7f3c60641640 1 -- 172.21.15.115:0/2087344134 shutdown_connections 2023-12-01T01:36:28.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.344+0000 7f3c60641640 1 -- 172.21.15.115:0/2087344134 wait complete. 2023-12-01T01:36:28.362 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-01T01:36:28.362 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:28.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.570+0000 7f41849cb640 1 Processor -- start 2023-12-01T01:36:28.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.571+0000 7f41849cb640 1 -- start start 2023-12-01T01:36:28.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.572+0000 7f41849cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 0x7f41800cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.572+0000 7f41849cb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f41800cdf30 0x7f4180160a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.572+0000 7f41849cb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41800b51e0 con 0x7f418015d030 2023-12-01T01:36:28.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.572+0000 7f41849cb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41800ce3c0 con 0x7f41800cdf30 2023-12-01T01:36:28.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.572+0000 7f41849cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41800b48b0 con 0x7f41800b5e70 2023-12-01T01:36:28.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.572+0000 7f417e59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f418015d030 0x7f41800cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38034/0 (socket says 172.21.15.115:38034) 2023-12-01T01:36:28.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.572+0000 7f417e59c640 1 -- 172.21.15.115:0/2492470145 learned_addr learned my addr 172.21.15.115:0/2492470145 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:28.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.572+0000 7f417ed9d640 1 --2- 172.21.15.115:0/2492470145 >> v2:172.21.15.115:3300/0 conn(0x7f41800cdf30 0x7f4180160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.573+0000 7f417dd9b640 1 --2- 172.21.15.115:0/2492470145 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 0x7f41800cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.573+0000 7f417d59a640 1 -- 172.21.15.115:0/2492470145 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551128433 0 0) 0x7f41800b51e0 con 0x7f418015d030 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.573+0000 7f417d59a640 1 -- 172.21.15.115:0/2492470145 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f416c003520 con 0x7f418015d030 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.573+0000 7f417ed9d640 1 -- 172.21.15.115:0/2492470145 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 msgr2=0x7f41800cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.573+0000 7f417ed9d640 1 --2- 172.21.15.115:0/2492470145 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 0x7f41800cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.573+0000 7f417ed9d640 1 -- 172.21.15.115:0/2492470145 >> v1:172.21.15.115:6789/0 conn(0x7f418015d030 legacy=0x7f41800cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.573+0000 7f417ed9d640 1 -- 172.21.15.115:0/2492470145 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800b4b90 con 0x7f41800cdf30 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.573+0000 7f417dd9b640 1 --2- 172.21.15.115:0/2492470145 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 0x7f41800cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.574+0000 7f417ed9d640 1 --2- 172.21.15.115:0/2492470145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 0x7f4180160a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f41780097a0 tx=0x7f417802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a357ac1314682b3b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.574+0000 7f417d59a640 1 -- 172.21.15.115:0/2492470145 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417802d040 con 0x7f41800cdf30 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.574+0000 7f417d59a640 1 -- 172.21.15.115:0/2492470145 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4178031810 con 0x7f41800cdf30 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.574+0000 7f417d59a640 1 -- 172.21.15.115:0/2492470145 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178031ba0 con 0x7f41800cdf30 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.575+0000 7f41849cb640 1 -- 172.21.15.115:0/2492470145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 msgr2=0x7f4180160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.575+0000 7f41849cb640 1 --2- 172.21.15.115:0/2492470145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 0x7f4180160a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f41780097a0 tx=0x7f417802b8b0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.575+0000 7f41849cb640 1 -- 172.21.15.115:0/2492470145 shutdown_connections 2023-12-01T01:36:28.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.575+0000 7f41849cb640 1 --2- 172.21.15.115:0/2492470145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 0x7f4180160a20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.575+0000 7f41849cb640 1 --2- 172.21.15.115:0/2492470145 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 0x7f41800cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.575+0000 7f41849cb640 1 -- 172.21.15.115:0/2492470145 >> 172.21.15.115:0/2492470145 conn(0x7f4180065960 msgr2=0x7f4180065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:28.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.575+0000 7f41849cb640 1 -- 172.21.15.115:0/2492470145 shutdown_connections 2023-12-01T01:36:28.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.575+0000 7f41849cb640 1 -- 172.21.15.115:0/2492470145 wait complete. 2023-12-01T01:36:28.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.577+0000 7f41849cb640 1 Processor -- start 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.577+0000 7f41849cb640 1 -- start start 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f41849cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 0x7f41801f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f41849cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 0x7f4180153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f41849cb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f418015d030 0x7f4180153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f41849cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800cc820 con 0x7f41800cdf30 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f41849cb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800cb930 con 0x7f418015d030 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f41849cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41800cbc10 con 0x7f41800b5e70 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f417dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 0x7f4180153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f417dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 0x7f4180153880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60838/0 (socket says 172.21.15.115:60838) 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f417dd9b640 1 -- 172.21.15.115:0/3093652615 learned_addr learned my addr 172.21.15.115:0/3093652615 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f417e59c640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 0x7f41801f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.578+0000 7f417ed9d640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f418015d030 0x7f4180153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f417dd9b640 1 -- 172.21.15.115:0/3093652615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 msgr2=0x7f41801f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f417dd9b640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 0x7f41801f1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f417dd9b640 1 -- 172.21.15.115:0/3093652615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f418015d030 msgr2=0x7f4180153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f417dd9b640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f418015d030 0x7f4180153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f417dd9b640 1 -- 172.21.15.115:0/3093652615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4178009400 con 0x7f41800cdf30 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f417dd9b640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 0x7f4180153880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f416800c7c0 tx=0x7f416800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f4176ffd640 1 -- 172.21.15.115:0/3093652615 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168007820 con 0x7f41800cdf30 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f4176ffd640 1 -- 172.21.15.115:0/3093652615 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4168007980 con 0x7f41800cdf30 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f4176ffd640 1 -- 172.21.15.115:0/3093652615 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168007d10 con 0x7f41800cdf30 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f41849cb640 1 -- 172.21.15.115:0/3093652615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41800cbef0 con 0x7f41800cdf30 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.579+0000 7f41849cb640 1 -- 172.21.15.115:0/3093652615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41800cc4b0 con 0x7f41800cdf30 2023-12-01T01:36:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.580+0000 7f41849cb640 1 -- 172.21.15.115:0/3093652615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4180074210 con 0x7f41800cdf30 2023-12-01T01:36:28.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.581+0000 7f4176ffd640 1 -- 172.21.15.115:0/3093652615 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f41680113f0 con 0x7f41800cdf30 2023-12-01T01:36:28.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.582+0000 7f4176ffd640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a430 0x7f416006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:28.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.582+0000 7f4176ffd640 1 -- 172.21.15.115:0/3093652615 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4168088a40 con 0x7f41800cdf30 2023-12-01T01:36:28.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.582+0000 7f417e59c640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a430 0x7f416006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:28.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.583+0000 7f417e59c640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a430 0x7f416006c8f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4170005c00 tx=0x7f4170009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:28.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.586+0000 7f4176ffd640 1 -- 172.21.15.115:0/3093652615 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4168053f90 con 0x7f41800cdf30 2023-12-01T01:36:28.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.776+0000 7f41849cb640 1 -- 172.21.15.115:0/3093652615 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41800cc640 con 0x7f41800cdf30 2023-12-01T01:36:28.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.778+0000 7f4176ffd640 1 -- 172.21.15.115:0/3093652615 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f416801f090 con 0x7f41800cdf30 2023-12-01T01:36:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:28.780 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:11.641476+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.781+0000 7f41849cb640 1 -- 172.21.15.115:0/3093652615 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a430 msgr2=0x7f416006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.781+0000 7f41849cb640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a430 0x7f416006c8f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4170005c00 tx=0x7f4170009040 comp rx=0 tx=0).stop 2023-12-01T01:36:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.781+0000 7f41849cb640 1 -- 172.21.15.115:0/3093652615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 msgr2=0x7f4180153880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.781+0000 7f41849cb640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 0x7f4180153880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f416800c7c0 tx=0x7f416800cc80 comp rx=0 tx=0).stop 2023-12-01T01:36:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.781+0000 7f41849cb640 1 -- 172.21.15.115:0/3093652615 shutdown_connections 2023-12-01T01:36:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.781+0000 7f41849cb640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a430 0x7f416006c8f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.781+0000 7f41849cb640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f418015d030 0x7f4180153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.781+0000 7f41849cb640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41800cdf30 0x7f4180153880 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.781+0000 7f41849cb640 1 --2- 172.21.15.115:0/3093652615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41800b5e70 0x7f41801f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.781+0000 7f41849cb640 1 -- 172.21.15.115:0/3093652615 >> 172.21.15.115:0/3093652615 conn(0x7f4180065960 msgr2=0x7f41800b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.782+0000 7f41849cb640 1 -- 172.21.15.115:0/3093652615 shutdown_connections 2023-12-01T01:36:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:28.782+0000 7f41849cb640 1 -- 172.21.15.115:0/3093652615 wait complete. 2023-12-01T01:36:28.799 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:36:28.800 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:36:29.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.014+0000 7fd8e12b6640 1 Processor -- start 2023-12-01T01:36:29.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.015+0000 7fd8e12b6640 1 -- start start 2023-12-01T01:36:29.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.016+0000 7fd8e12b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc160200 0x7fd8dc0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.016+0000 7fd8e12b6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd8dc0b0f80 0x7fd8dc06b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.017+0000 7fd8e12b6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8dc0cb290 con 0x7fd8dc06b7a0 2023-12-01T01:36:29.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.017+0000 7fd8e12b6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8dc0b35f0 con 0x7fd8dc0b0f80 2023-12-01T01:36:29.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.017+0000 7fd8e12b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc0b0ae0 con 0x7fd8dc160200 2023-12-01T01:36:29.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.017+0000 7fd8daffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc160200 0x7fd8dc0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.017+0000 7fd8daffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc160200 0x7fd8dc0b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:58842/0 (socket says 172.21.15.115:58842) 2023-12-01T01:36:29.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.017+0000 7fd8daffd640 1 -- 172.21.15.115:0/21551083 learned_addr learned my addr 172.21.15.115:0/21551083 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:29.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.018+0000 7fd8d27fc640 1 --2- 172.21.15.115:0/21551083 >> v2:172.21.15.142:3300/0 conn(0x7fd8dc0b0f80 0x7fd8dc06b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.018+0000 7fd8daffd640 1 -- 172.21.15.115:0/21551083 >> v2:172.21.15.142:3300/0 conn(0x7fd8dc0b0f80 msgr2=0x7fd8dc06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:29.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.018+0000 7fd8daffd640 1 --2- 172.21.15.115:0/21551083 >> v2:172.21.15.142:3300/0 conn(0x7fd8dc0b0f80 0x7fd8dc06b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.018+0000 7fd8daffd640 1 -- 172.21.15.115:0/21551083 >> v1:172.21.15.142:6789/0 conn(0x7fd8dc06b7a0 legacy=0x7fd8dc0cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.018+0000 7fd8daffd640 1 -- 172.21.15.115:0/21551083 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0b3310 con 0x7fd8dc160200 2023-12-01T01:36:29.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.019+0000 7fd8daffd640 1 --2- 172.21.15.115:0/21551083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc160200 0x7fd8dc0b2c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009870 tx=0x7fd8cc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22049f2a32c92655 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:29.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.020+0000 7fd8da7fc640 1 -- 172.21.15.115:0/21551083 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc030840 con 0x7fd8dc160200 2023-12-01T01:36:29.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.020+0000 7fd8da7fc640 1 -- 172.21.15.115:0/21551083 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc0309a0 con 0x7fd8dc160200 2023-12-01T01:36:29.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.020+0000 7fd8da7fc640 1 -- 172.21.15.115:0/21551083 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc030d30 con 0x7fd8dc160200 2023-12-01T01:36:29.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.020+0000 7fd8e12b6640 1 -- 172.21.15.115:0/21551083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc160200 msgr2=0x7fd8dc0b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.020+0000 7fd8e12b6640 1 --2- 172.21.15.115:0/21551083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc160200 0x7fd8dc0b2c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc009870 tx=0x7fd8cc02b980 comp rx=0 tx=0).stop 2023-12-01T01:36:29.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.020+0000 7fd8e12b6640 1 -- 172.21.15.115:0/21551083 shutdown_connections 2023-12-01T01:36:29.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.020+0000 7fd8e12b6640 1 --2- 172.21.15.115:0/21551083 >> v2:172.21.15.142:3300/0 conn(0x7fd8dc0b0f80 0x7fd8dc06b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.020+0000 7fd8e12b6640 1 --2- 172.21.15.115:0/21551083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc160200 0x7fd8dc0b2c80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.020+0000 7fd8e12b6640 1 -- 172.21.15.115:0/21551083 >> 172.21.15.115:0/21551083 conn(0x7fd8dc065960 msgr2=0x7fd8dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:29.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.021+0000 7fd8e12b6640 1 -- 172.21.15.115:0/21551083 shutdown_connections 2023-12-01T01:36:29.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.021+0000 7fd8e12b6640 1 -- 172.21.15.115:0/21551083 wait complete. 2023-12-01T01:36:29.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.022+0000 7fd8e12b6640 1 Processor -- start 2023-12-01T01:36:29.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.022+0000 7fd8e12b6640 1 -- start start 2023-12-01T01:36:29.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.023+0000 7fd8e12b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc06b7a0 0x7fd8dc146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.023+0000 7fd8e12b6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8dc0b0f80 0x7fd8dc070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.023+0000 7fd8e12b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8dc160200 0x7fd8dc070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.023+0000 7fd8e12b6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc146cd0 con 0x7fd8dc160200 2023-12-01T01:36:29.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.023+0000 7fd8e12b6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc1f6bc0 con 0x7fd8dc0b0f80 2023-12-01T01:36:29.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.023+0000 7fd8e12b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8dc150da0 con 0x7fd8dc06b7a0 2023-12-01T01:36:29.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.024+0000 7fd8db7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8dc160200 0x7fd8dc070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.024+0000 7fd8db7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8dc160200 0x7fd8dc070ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60852/0 (socket says 172.21.15.115:60852) 2023-12-01T01:36:29.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.024+0000 7fd8db7fe640 1 -- 172.21.15.115:0/4053624265 learned_addr learned my addr 172.21.15.115:0/4053624265 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:29.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.024+0000 7fd8daffd640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc06b7a0 0x7fd8dc146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.024+0000 7fd8d27fc640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8dc0b0f80 0x7fd8dc070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.024+0000 7fd8db7fe640 1 -- 172.21.15.115:0/4053624265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc06b7a0 msgr2=0x7fd8dc146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.024+0000 7fd8db7fe640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc06b7a0 0x7fd8dc146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.024+0000 7fd8db7fe640 1 -- 172.21.15.115:0/4053624265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8dc0b0f80 msgr2=0x7fd8dc070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.024+0000 7fd8db7fe640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8dc0b0f80 0x7fd8dc070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.024+0000 7fd8db7fe640 1 -- 172.21.15.115:0/4053624265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc0094d0 con 0x7fd8dc160200 2023-12-01T01:36:29.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.025+0000 7fd8db7fe640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8dc160200 0x7fd8dc070ba0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c680 tx=0x7fd8d400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:29.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.025+0000 7fd8d3fff640 1 -- 172.21.15.115:0/4053624265 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4003b70 con 0x7fd8dc160200 2023-12-01T01:36:29.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.025+0000 7fd8d3fff640 1 -- 172.21.15.115:0/4053624265 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4003cd0 con 0x7fd8dc160200 2023-12-01T01:36:29.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.025+0000 7fd8d3fff640 1 -- 172.21.15.115:0/4053624265 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4005310 con 0x7fd8dc160200 2023-12-01T01:36:29.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.025+0000 7fd8e12b6640 1 -- 172.21.15.115:0/4053624265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8dc151080 con 0x7fd8dc160200 2023-12-01T01:36:29.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.025+0000 7fd8e12b6640 1 -- 172.21.15.115:0/4053624265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8dc151640 con 0x7fd8dc160200 2023-12-01T01:36:29.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.026+0000 7fd8e12b6640 1 -- 172.21.15.115:0/4053624265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8dc04e890 con 0x7fd8dc160200 2023-12-01T01:36:29.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.027+0000 7fd8d3fff640 1 -- 172.21.15.115:0/4053624265 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd8d4005470 con 0x7fd8dc160200 2023-12-01T01:36:29.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.028+0000 7fd8d3fff640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8bc06a430 0x7fd8bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.028+0000 7fd8d3fff640 1 -- 172.21.15.115:0/4053624265 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8d40886b0 con 0x7fd8dc160200 2023-12-01T01:36:29.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.028+0000 7fd8daffd640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8bc06a430 0x7fd8bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.029+0000 7fd8daffd640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8bc06a430 0x7fd8bc06c8f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc036000 tx=0x7fd8cc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:29.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.033+0000 7fd8d3fff640 1 -- 172.21.15.115:0/4053624265 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd8d4053cb0 con 0x7fd8dc160200 2023-12-01T01:36:29.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.224+0000 7fd8e12b6640 1 -- 172.21.15.115:0/4053624265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8dc0b1e20 con 0x7fd8dc160200 2023-12-01T01:36:29.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.225+0000 7fd8d3fff640 1 -- 172.21.15.115:0/4053624265 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8d40572d0 con 0x7fd8dc160200 2023-12-01T01:36:29.225 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:36:29.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.227+0000 7fd8e12b6640 1 -- 172.21.15.115:0/4053624265 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8bc06a430 msgr2=0x7fd8bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.227+0000 7fd8e12b6640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8bc06a430 0x7fd8bc06c8f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc036000 tx=0x7fd8cc008030 comp rx=0 tx=0).stop 2023-12-01T01:36:29.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.227+0000 7fd8e12b6640 1 -- 172.21.15.115:0/4053624265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8dc160200 msgr2=0x7fd8dc070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.227+0000 7fd8e12b6640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8dc160200 0x7fd8dc070ba0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c680 tx=0x7fd8d400cb40 comp rx=0 tx=0).stop 2023-12-01T01:36:29.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.227+0000 7fd8e12b6640 1 -- 172.21.15.115:0/4053624265 shutdown_connections 2023-12-01T01:36:29.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.227+0000 7fd8e12b6640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd8bc06a430 0x7fd8bc06c8f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.228+0000 7fd8e12b6640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8dc160200 0x7fd8dc070ba0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.228+0000 7fd8e12b6640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8dc0b0f80 0x7fd8dc070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.228+0000 7fd8e12b6640 1 --2- 172.21.15.115:0/4053624265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8dc06b7a0 0x7fd8dc146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.228+0000 7fd8e12b6640 1 -- 172.21.15.115:0/4053624265 >> 172.21.15.115:0/4053624265 conn(0x7fd8dc065960 msgr2=0x7fd8dc0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:29.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.228+0000 7fd8e12b6640 1 -- 172.21.15.115:0/4053624265 shutdown_connections 2023-12-01T01:36:29.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.228+0000 7fd8e12b6640 1 -- 172.21.15.115:0/4053624265 wait complete. 2023-12-01T01:36:29.246 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:36:29.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.461+0000 7f0363fff640 1 Processor -- start 2023-12-01T01:36:29.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.461+0000 7f0363fff640 1 -- start start 2023-12-01T01:36:29.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.463+0000 7f0363fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03640cfb80 0x7f03640ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.463+0000 7f0363fff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03640cdcd0 con 0x7f0364154010 2023-12-01T01:36:29.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.463+0000 7f0363fff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0364161830 con 0x7f0364153c30 2023-12-01T01:36:29.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.463+0000 7f0363fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03640cd3a0 con 0x7f03640cfb80 2023-12-01T01:36:29.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.463+0000 7f035bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03640cfb80 0x7f03640ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.463+0000 7f035bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03640cfb80 0x7f03640ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60866/0 (socket says 172.21.15.115:60866) 2023-12-01T01:36:29.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.463+0000 7f035bfff640 1 -- 172.21.15.115:0/3046433990 learned_addr learned my addr 172.21.15.115:0/3046433990 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:29.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.463+0000 7f03637fe640 1 --1- 172.21.15.115:0/3046433990 >> v1:172.21.15.115:6789/0 conn(0x7f0364154010 0x7f0364160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38042/0 (socket says 172.21.15.115:38042) 2023-12-01T01:36:29.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.464+0000 7f035bfff640 1 -- 172.21.15.115:0/3046433990 >> v1:172.21.15.142:6789/0 conn(0x7f0364153c30 legacy=0x7f03640cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:29.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.464+0000 7f035bfff640 1 -- 172.21.15.115:0/3046433990 >> v1:172.21.15.115:6789/0 conn(0x7f0364154010 legacy=0x7f0364160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.464+0000 7f035bfff640 1 -- 172.21.15.115:0/3046433990 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03640cd680 con 0x7f03640cfb80 2023-12-01T01:36:29.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.465+0000 7f035bfff640 1 --2- 172.21.15.115:0/3046433990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 0x7f03640ce160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f034c009870 tx=0x7f034c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9fab3c3ac83679 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:29.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.465+0000 7f03627fc640 1 -- 172.21.15.115:0/3046433990 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c030850 con 0x7f03640cfb80 2023-12-01T01:36:29.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.465+0000 7f03627fc640 1 -- 172.21.15.115:0/3046433990 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c0309b0 con 0x7f03640cfb80 2023-12-01T01:36:29.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.465+0000 7f03627fc640 1 -- 172.21.15.115:0/3046433990 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c030d40 con 0x7f03640cfb80 2023-12-01T01:36:29.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.465+0000 7f0363fff640 1 -- 172.21.15.115:0/3046433990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 msgr2=0x7f03640ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.465+0000 7f0363fff640 1 --2- 172.21.15.115:0/3046433990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 0x7f03640ce160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f034c009870 tx=0x7f034c02b990 comp rx=0 tx=0).stop 2023-12-01T01:36:29.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.466+0000 7f0363fff640 1 -- 172.21.15.115:0/3046433990 shutdown_connections 2023-12-01T01:36:29.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.466+0000 7f0363fff640 1 --2- 172.21.15.115:0/3046433990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 0x7f03640ce160 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.466+0000 7f0363fff640 1 -- 172.21.15.115:0/3046433990 >> 172.21.15.115:0/3046433990 conn(0x7f036406a0b0 msgr2=0x7f036406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:29.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.466+0000 7f0363fff640 1 -- 172.21.15.115:0/3046433990 shutdown_connections 2023-12-01T01:36:29.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.466+0000 7f0363fff640 1 -- 172.21.15.115:0/3046433990 wait complete. 2023-12-01T01:36:29.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.467+0000 7f0363fff640 1 Processor -- start 2023-12-01T01:36:29.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.468+0000 7f0363fff640 1 -- start start 2023-12-01T01:36:29.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.468+0000 7f0363fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 0x7f0364155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0363fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0364153c30 0x7f03641f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0363fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0364154010 0x7f0364154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0363fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0364156570 con 0x7f03640cfb80 2023-12-01T01:36:29.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0363fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0364155780 con 0x7f0364153c30 2023-12-01T01:36:29.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0363fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03641f2ce0 con 0x7f0364154010 2023-12-01T01:36:29.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0362ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 0x7f0364155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0362ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 0x7f0364155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60880/0 (socket says 172.21.15.115:60880) 2023-12-01T01:36:29.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0362ffd640 1 -- 172.21.15.115:0/104453588 learned_addr learned my addr 172.21.15.115:0/104453588 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:29.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0362ffd640 1 -- 172.21.15.115:0/104453588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0364154010 msgr2=0x7f0364154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0362ffd640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0364154010 0x7f0364154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0362ffd640 1 -- 172.21.15.115:0/104453588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0364153c30 msgr2=0x7f03641f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0362ffd640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0364153c30 0x7f03641f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0362ffd640 1 -- 172.21.15.115:0/104453588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c0094d0 con 0x7f03640cfb80 2023-12-01T01:36:29.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.469+0000 7f0362ffd640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 0x7f0364155240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f035400c430 tx=0x7f035400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:29.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.470+0000 7f035b7fe640 1 -- 172.21.15.115:0/104453588 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0354011790 con 0x7f03640cfb80 2023-12-01T01:36:29.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.470+0000 7f035b7fe640 1 -- 172.21.15.115:0/104453588 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03540118f0 con 0x7f03640cfb80 2023-12-01T01:36:29.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.470+0000 7f035b7fe640 1 -- 172.21.15.115:0/104453588 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0354011ce0 con 0x7f03640cfb80 2023-12-01T01:36:29.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.470+0000 7f0363fff640 1 -- 172.21.15.115:0/104453588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03641f2fc0 con 0x7f03640cfb80 2023-12-01T01:36:29.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.470+0000 7f0363fff640 1 -- 172.21.15.115:0/104453588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03641f32a0 con 0x7f03640cfb80 2023-12-01T01:36:29.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.471+0000 7f0363fff640 1 -- 172.21.15.115:0/104453588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03641566b0 con 0x7f03640cfb80 2023-12-01T01:36:29.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.472+0000 7f035b7fe640 1 -- 172.21.15.115:0/104453588 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f035401c460 con 0x7f03640cfb80 2023-12-01T01:36:29.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.473+0000 7f035b7fe640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034406a430 0x7f034406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.473+0000 7f035b7fe640 1 -- 172.21.15.115:0/104453588 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03540888f0 con 0x7f03640cfb80 2023-12-01T01:36:29.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.473+0000 7f035bfff640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034406a430 0x7f034406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.474+0000 7f035bfff640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034406a430 0x7f034406c8f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f034c036000 tx=0x7f034c03b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:29.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.477+0000 7f035b7fe640 1 -- 172.21.15.115:0/104453588 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f03641566b0 con 0x7f03640cfb80 2023-12-01T01:36:29.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.668+0000 7f0363fff640 1 -- 172.21.15.115:0/104453588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03641566b0 con 0x7f03640cfb80 2023-12-01T01:36:29.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.669+0000 7f035b7fe640 1 -- 172.21.15.115:0/104453588 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03641566b0 con 0x7f03640cfb80 2023-12-01T01:36:29.669 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:36:29.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.671+0000 7f0363fff640 1 -- 172.21.15.115:0/104453588 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034406a430 msgr2=0x7f034406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.671+0000 7f0363fff640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034406a430 0x7f034406c8f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f034c036000 tx=0x7f034c03b040 comp rx=0 tx=0).stop 2023-12-01T01:36:29.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.671+0000 7f0363fff640 1 -- 172.21.15.115:0/104453588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 msgr2=0x7f0364155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.671+0000 7f0363fff640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 0x7f0364155240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f035400c430 tx=0x7f035400c8f0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.671+0000 7f0363fff640 1 -- 172.21.15.115:0/104453588 shutdown_connections 2023-12-01T01:36:29.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.671+0000 7f0363fff640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034406a430 0x7f034406c8f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.671+0000 7f0363fff640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0364154010 0x7f0364154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.671+0000 7f0363fff640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0364153c30 0x7f03641f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.671+0000 7f0363fff640 1 --2- 172.21.15.115:0/104453588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03640cfb80 0x7f0364155240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.671+0000 7f0363fff640 1 -- 172.21.15.115:0/104453588 >> 172.21.15.115:0/104453588 conn(0x7f036406a0b0 msgr2=0x7f03640ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:29.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.672+0000 7f0363fff640 1 -- 172.21.15.115:0/104453588 shutdown_connections 2023-12-01T01:36:29.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.672+0000 7f0363fff640 1 -- 172.21.15.115:0/104453588 wait complete. 2023-12-01T01:36:29.691 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:36:29.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.886+0000 7f25280ab640 1 Processor -- start 2023-12-01T01:36:29.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.886+0000 7f25280ab640 1 -- start start 2023-12-01T01:36:29.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.888+0000 7f25280ab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2520142870 0x7f25200ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.888+0000 7f25280ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 0x7f25200b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.888+0000 7f25280ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25200cd470 con 0x7f25200b1c50 2023-12-01T01:36:29.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.888+0000 7f25280ab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25200b3ae0 con 0x7f2520142870 2023-12-01T01:36:29.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.888+0000 7f25280ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2520142c50 con 0x7f25200cd010 2023-12-01T01:36:29.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.888+0000 7f2525646640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 0x7f25200b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.889+0000 7f2525646640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 0x7f25200b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:58872/0 (socket says 172.21.15.115:58872) 2023-12-01T01:36:29.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.889+0000 7f2525646640 1 -- 172.21.15.115:0/1599235255 learned_addr learned my addr 172.21.15.115:0/1599235255 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:29.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.889+0000 7f2525e47640 1 --2- 172.21.15.115:0/1599235255 >> v2:172.21.15.142:3300/0 conn(0x7f2520142870 0x7f25200ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.889+0000 7f2525646640 1 -- 172.21.15.115:0/1599235255 >> v2:172.21.15.142:3300/0 conn(0x7f2520142870 msgr2=0x7f25200ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:29.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.889+0000 7f2525646640 1 --2- 172.21.15.115:0/1599235255 >> v2:172.21.15.142:3300/0 conn(0x7f2520142870 0x7f25200ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.889+0000 7f2525646640 1 -- 172.21.15.115:0/1599235255 >> v1:172.21.15.142:6789/0 conn(0x7f25200b1c50 legacy=0x7f252015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.889+0000 7f2525646640 1 -- 172.21.15.115:0/1599235255 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25200b20e0 con 0x7f25200cd010 2023-12-01T01:36:29.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.890+0000 7f2525646640 1 --2- 172.21.15.115:0/1599235255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 0x7f25200b1710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2510009870 tx=0x7f251002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48b50641a6bc300 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:29.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.891+0000 7f2524e45640 1 -- 172.21.15.115:0/1599235255 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510030840 con 0x7f25200cd010 2023-12-01T01:36:29.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.891+0000 7f2524e45640 1 -- 172.21.15.115:0/1599235255 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25100309a0 con 0x7f25200cd010 2023-12-01T01:36:29.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.891+0000 7f2524e45640 1 -- 172.21.15.115:0/1599235255 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510030d30 con 0x7f25200cd010 2023-12-01T01:36:29.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.891+0000 7f25280ab640 1 -- 172.21.15.115:0/1599235255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 msgr2=0x7f25200b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.891+0000 7f25280ab640 1 --2- 172.21.15.115:0/1599235255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 0x7f25200b1710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2510009870 tx=0x7f251002b980 comp rx=0 tx=0).stop 2023-12-01T01:36:29.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.892+0000 7f25280ab640 1 -- 172.21.15.115:0/1599235255 shutdown_connections 2023-12-01T01:36:29.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.892+0000 7f25280ab640 1 --2- 172.21.15.115:0/1599235255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 0x7f25200b1710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.892+0000 7f25280ab640 1 --2- 172.21.15.115:0/1599235255 >> v2:172.21.15.142:3300/0 conn(0x7f2520142870 0x7f25200ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.892+0000 7f25280ab640 1 -- 172.21.15.115:0/1599235255 >> 172.21.15.115:0/1599235255 conn(0x7f2520065960 msgr2=0x7f2520065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:29.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.892+0000 7f25280ab640 1 -- 172.21.15.115:0/1599235255 shutdown_connections 2023-12-01T01:36:29.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.892+0000 7f25280ab640 1 -- 172.21.15.115:0/1599235255 wait complete. 2023-12-01T01:36:29.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.894+0000 7f25280ab640 1 Processor -- start 2023-12-01T01:36:29.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.894+0000 7f25280ab640 1 -- start start 2023-12-01T01:36:29.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.895+0000 7f25280ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25200b1c50 0x7f25200c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.895+0000 7f25280ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 0x7f25200ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.895+0000 7f25280ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2520142870 0x7f252013a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.895+0000 7f25280ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25200cbab0 con 0x7f2520142870 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.895+0000 7f25280ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25200caea0 con 0x7f25200b1c50 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.895+0000 7f25280ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25200cb180 con 0x7f25200cd010 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.896+0000 7f2526648640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2520142870 0x7f252013a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.896+0000 7f2526648640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2520142870 0x7f252013a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60890/0 (socket says 172.21.15.115:60890) 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.896+0000 7f2526648640 1 -- 172.21.15.115:0/3931385159 learned_addr learned my addr 172.21.15.115:0/3931385159 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.896+0000 7f2525646640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 0x7f25200ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.896+0000 7f2525e47640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25200b1c50 0x7f25200c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.896+0000 7f2526648640 1 -- 172.21.15.115:0/3931385159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 msgr2=0x7f25200ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.896+0000 7f2526648640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 0x7f25200ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.896+0000 7f2526648640 1 -- 172.21.15.115:0/3931385159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25200b1c50 msgr2=0x7f25200c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:29.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.896+0000 7f2526648640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25200b1c50 0x7f25200c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:29.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.896+0000 7f2526648640 1 -- 172.21.15.115:0/3931385159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25100094d0 con 0x7f2520142870 2023-12-01T01:36:29.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.897+0000 7f2526648640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2520142870 0x7f252013a2a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2518001eb0 tx=0x7f251800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:29.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.897+0000 7f25167fc640 1 -- 172.21.15.115:0/3931385159 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180052d0 con 0x7f2520142870 2023-12-01T01:36:29.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.897+0000 7f25167fc640 1 -- 172.21.15.115:0/3931385159 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2518005430 con 0x7f2520142870 2023-12-01T01:36:29.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.897+0000 7f25280ab640 1 -- 172.21.15.115:0/3931385159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25200cb460 con 0x7f2520142870 2023-12-01T01:36:29.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.897+0000 7f25167fc640 1 -- 172.21.15.115:0/3931385159 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518003810 con 0x7f2520142870 2023-12-01T01:36:29.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.897+0000 7f25280ab640 1 -- 172.21.15.115:0/3931385159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25200c8ba0 con 0x7f2520142870 2023-12-01T01:36:29.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.898+0000 7f25280ab640 1 -- 172.21.15.115:0/3931385159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25200c8d30 con 0x7f2520142870 2023-12-01T01:36:29.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.899+0000 7f25167fc640 1 -- 172.21.15.115:0/3931385159 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2518003970 con 0x7f2520142870 2023-12-01T01:36:29.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.900+0000 7f25167fc640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250406a430 0x7f250406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:29.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.900+0000 7f25167fc640 1 -- 172.21.15.115:0/3931385159 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2518087ed0 con 0x7f2520142870 2023-12-01T01:36:29.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.900+0000 7f2525e47640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250406a430 0x7f250406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:29.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.901+0000 7f2525e47640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250406a430 0x7f250406c8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f251c0073f0 tx=0x7f251c00b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:29.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:29.904+0000 7f25167fc640 1 -- 172.21.15.115:0/3931385159 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f25180534b0 con 0x7f2520142870 2023-12-01T01:36:30.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.093+0000 7f25280ab640 1 -- 172.21.15.115:0/3931385159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f252004e890 con 0x7f2520142870 2023-12-01T01:36:30.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.094+0000 7f25167fc640 1 -- 172.21.15.115:0/3931385159 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2518056ad0 con 0x7f2520142870 2023-12-01T01:36:30.094 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:36:30.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.096+0000 7f25280ab640 1 -- 172.21.15.115:0/3931385159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250406a430 msgr2=0x7f250406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.096+0000 7f25280ab640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250406a430 0x7f250406c8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f251c0073f0 tx=0x7f251c00b040 comp rx=0 tx=0).stop 2023-12-01T01:36:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.096+0000 7f25280ab640 1 -- 172.21.15.115:0/3931385159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2520142870 msgr2=0x7f252013a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.096+0000 7f25280ab640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2520142870 0x7f252013a2a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2518001eb0 tx=0x7f251800c890 comp rx=0 tx=0).stop 2023-12-01T01:36:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.097+0000 7f25280ab640 1 -- 172.21.15.115:0/3931385159 shutdown_connections 2023-12-01T01:36:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.097+0000 7f25280ab640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250406a430 0x7f250406c8f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.097+0000 7f25280ab640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25200cd010 0x7f25200ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.097+0000 7f25280ab640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2520142870 0x7f252013a2a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.097+0000 7f25280ab640 1 --2- 172.21.15.115:0/3931385159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25200b1c50 0x7f25200c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.097+0000 7f25280ab640 1 -- 172.21.15.115:0/3931385159 >> 172.21.15.115:0/3931385159 conn(0x7f2520065960 msgr2=0x7f252013e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.097+0000 7f25280ab640 1 -- 172.21.15.115:0/3931385159 shutdown_connections 2023-12-01T01:36:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.097+0000 7f25280ab640 1 -- 172.21.15.115:0/3931385159 wait complete. 2023-12-01T01:36:30.116 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:36:30.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.326+0000 7f1956118640 1 Processor -- start 2023-12-01T01:36:30.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.326+0000 7f1956118640 1 -- start start 2023-12-01T01:36:30.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.327+0000 7f1956118640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f19500cad70 0x7f195006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.327+0000 7f1956118640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f195006f2f0 0x7f195006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.327+0000 7f1956118640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19500add10 con 0x7f195015e2f0 2023-12-01T01:36:30.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.327+0000 7f1956118640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19500c5760 con 0x7f19500cad70 2023-12-01T01:36:30.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.327+0000 7f1956118640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195015e6d0 con 0x7f195006f2f0 2023-12-01T01:36:30.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.328+0000 7f194effd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f19500cad70 0x7f195006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.328+0000 7f194effd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f19500cad70 0x7f195006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60902/0 (socket says 172.21.15.115:60902) 2023-12-01T01:36:30.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.328+0000 7f194effd640 1 -- 172.21.15.115:0/2479923081 learned_addr learned my addr 172.21.15.115:0/2479923081 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:30.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.328+0000 7f194ffff640 1 --2- 172.21.15.115:0/2479923081 >> v2:172.21.15.142:3300/0 conn(0x7f195006f2f0 0x7f195006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.328+0000 7f194effd640 1 -- 172.21.15.115:0/2479923081 >> v2:172.21.15.142:3300/0 conn(0x7f195006f2f0 msgr2=0x7f195006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:30.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.328+0000 7f194effd640 1 --2- 172.21.15.115:0/2479923081 >> v2:172.21.15.142:3300/0 conn(0x7f195006f2f0 0x7f195006bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.328+0000 7f194effd640 1 -- 172.21.15.115:0/2479923081 >> v1:172.21.15.142:6789/0 conn(0x7f195015e2f0 legacy=0x7f19500ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:30.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.328+0000 7f194effd640 1 -- 172.21.15.115:0/2479923081 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19500cf7b0 con 0x7f19500cad70 2023-12-01T01:36:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.330+0000 7f194effd640 1 --2- 172.21.15.115:0/2479923081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19500cad70 0x7f195006b5c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f195006de80 tx=0x7f194402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4cda9c0c191cb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.330+0000 7f194e7fc640 1 -- 172.21.15.115:0/2479923081 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944031710 con 0x7f19500cad70 2023-12-01T01:36:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.330+0000 7f194e7fc640 1 -- 172.21.15.115:0/2479923081 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194402f050 con 0x7f19500cad70 2023-12-01T01:36:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.330+0000 7f194e7fc640 1 -- 172.21.15.115:0/2479923081 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944031a90 con 0x7f19500cad70 2023-12-01T01:36:30.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.331+0000 7f1956118640 1 -- 172.21.15.115:0/2479923081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19500cad70 msgr2=0x7f195006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.331+0000 7f1956118640 1 --2- 172.21.15.115:0/2479923081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19500cad70 0x7f195006b5c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f195006de80 tx=0x7f194402b7b0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.331+0000 7f1956118640 1 -- 172.21.15.115:0/2479923081 shutdown_connections 2023-12-01T01:36:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.331+0000 7f1956118640 1 --2- 172.21.15.115:0/2479923081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19500cad70 0x7f195006b5c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.331+0000 7f1956118640 1 --2- 172.21.15.115:0/2479923081 >> v2:172.21.15.142:3300/0 conn(0x7f195006f2f0 0x7f195006bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.331+0000 7f1956118640 1 -- 172.21.15.115:0/2479923081 >> 172.21.15.115:0/2479923081 conn(0x7f19500ce880 msgr2=0x7f19500cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.331+0000 7f1956118640 1 -- 172.21.15.115:0/2479923081 shutdown_connections 2023-12-01T01:36:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.331+0000 7f1956118640 1 -- 172.21.15.115:0/2479923081 wait complete. 2023-12-01T01:36:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.333+0000 7f1956118640 1 Processor -- start 2023-12-01T01:36:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.333+0000 7f1956118640 1 -- start start 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f1956118640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195006f2f0 0x7f19501579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f1956118640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19500cad70 0x7f19501f0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f1956118640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195015e2f0 0x7f19500c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f1956118640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19500c5270 con 0x7f195015e2f0 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f1956118640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19500c23b0 con 0x7f19500cad70 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f1956118640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19500c2650 con 0x7f195006f2f0 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195015e2f0 0x7f19500c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195015e2f0 0x7f19500c4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60914/0 (socket says 172.21.15.115:60914) 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194ffff640 1 -- 172.21.15.115:0/2799942127 learned_addr learned my addr 172.21.15.115:0/2799942127 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194f7fe640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195006f2f0 0x7f19501579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194ffff640 1 -- 172.21.15.115:0/2799942127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195006f2f0 msgr2=0x7f19501579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194effd640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19500cad70 0x7f19501f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194ffff640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195006f2f0 0x7f19501579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194ffff640 1 -- 172.21.15.115:0/2799942127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19500cad70 msgr2=0x7f19501f0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194ffff640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19500cad70 0x7f19501f0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194ffff640 1 -- 172.21.15.115:0/2799942127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19440094d0 con 0x7f195015e2f0 2023-12-01T01:36:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.334+0000 7f194effd640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19500cad70 0x7f19501f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.335+0000 7f194ffff640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195015e2f0 0x7f19500c4d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1948007cf0 tx=0x7f194800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.335+0000 7f1937fff640 1 -- 172.21.15.115:0/2799942127 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948014420 con 0x7f195015e2f0 2023-12-01T01:36:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.335+0000 7f1937fff640 1 -- 172.21.15.115:0/2799942127 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19480145a0 con 0x7f195015e2f0 2023-12-01T01:36:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.335+0000 7f1956118640 1 -- 172.21.15.115:0/2799942127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19500c2930 con 0x7f195015e2f0 2023-12-01T01:36:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.335+0000 7f1937fff640 1 -- 172.21.15.115:0/2799942127 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948014950 con 0x7f195015e2f0 2023-12-01T01:36:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.335+0000 7f1956118640 1 -- 172.21.15.115:0/2799942127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19500c2ef0 con 0x7f195015e2f0 2023-12-01T01:36:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.336+0000 7f1956118640 1 -- 172.21.15.115:0/2799942127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195006fda0 con 0x7f195015e2f0 2023-12-01T01:36:30.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.337+0000 7f1937fff640 1 -- 172.21.15.115:0/2799942127 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1948014ab0 con 0x7f195015e2f0 2023-12-01T01:36:30.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.338+0000 7f1937fff640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f193c06a430 0x7f193c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.338+0000 7f1937fff640 1 -- 172.21.15.115:0/2799942127 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1948089140 con 0x7f195015e2f0 2023-12-01T01:36:30.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.338+0000 7f194f7fe640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f193c06a430 0x7f193c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.339+0000 7f194f7fe640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f193c06a430 0x7f193c06c8f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f194000a750 tx=0x7f1940008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:30.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.343+0000 7f1937fff640 1 -- 172.21.15.115:0/2799942127 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f19480547c0 con 0x7f195015e2f0 2023-12-01T01:36:30.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.575+0000 7f1956118640 1 -- 172.21.15.115:0/2799942127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f195004e890 con 0x7f195015e2f0 2023-12-01T01:36:30.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.576+0000 7f1937fff640 1 -- 172.21.15.115:0/2799942127 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4654 (secure 0 0 0) 0x7f1948057de0 con 0x7f195015e2f0 2023-12-01T01:36:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:30.577 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3656795862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3656795862},{"type":"v1","addr":"172.21.15.142:6834","nonce":3656795862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1125002206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1125002206},{"type":"v1","addr":"172.21.15.115:6836","nonce":1125002206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3350782192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3350782192},{"type":"v1","addr":"172.21.15.115:6837","nonce":3350782192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:36:10.637450+0000","modified":"2023-12-01T01:36:13.663004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2808777047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2808777047},{"type":"v1","addr":"172.21.15.142:6835","nonce":2808777047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:36:30.577 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 8 2023-12-01T01:36:30.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.578+0000 7f1956118640 1 -- 172.21.15.115:0/2799942127 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f193c06a430 msgr2=0x7f193c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.578+0000 7f1956118640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f193c06a430 0x7f193c06c8f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f194000a750 tx=0x7f1940008040 comp rx=0 tx=0).stop 2023-12-01T01:36:30.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.578+0000 7f1956118640 1 -- 172.21.15.115:0/2799942127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195015e2f0 msgr2=0x7f19500c4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.578+0000 7f1956118640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195015e2f0 0x7f19500c4d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1948007cf0 tx=0x7f194800e4b0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.578+0000 7f1956118640 1 -- 172.21.15.115:0/2799942127 shutdown_connections 2023-12-01T01:36:30.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.578+0000 7f1956118640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f193c06a430 0x7f193c06c8f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.578+0000 7f1956118640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195015e2f0 0x7f19500c4d30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.578+0000 7f1956118640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19500cad70 0x7f19501f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.578+0000 7f1956118640 1 --2- 172.21.15.115:0/2799942127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195006f2f0 0x7f19501579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.578+0000 7f1956118640 1 -- 172.21.15.115:0/2799942127 >> 172.21.15.115:0/2799942127 conn(0x7f19500ce880 msgr2=0x7f195015d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:30.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.579+0000 7f1956118640 1 -- 172.21.15.115:0/2799942127 shutdown_connections 2023-12-01T01:36:30.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.579+0000 7f1956118640 1 -- 172.21.15.115:0/2799942127 wait complete. 2023-12-01T01:36:30.596 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:30.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.798+0000 7f76474dc640 1 Processor -- start 2023-12-01T01:36:30.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.798+0000 7f76474dc640 1 -- start start 2023-12-01T01:36:30.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.800+0000 7f76474dc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015afe0 0x7f764006b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.800+0000 7f76474dc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76400ce4a0 con 0x7f764006b7b0 2023-12-01T01:36:30.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.800+0000 7f76474dc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76400cc180 con 0x7f76401584b0 2023-12-01T01:36:30.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.800+0000 7f76474dc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764006bc40 con 0x7f764015afe0 2023-12-01T01:36:30.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.800+0000 7f7646cdb640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f764006b7b0 0x7f76400cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38048/0 (socket says 172.21.15.115:38048) 2023-12-01T01:36:30.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.800+0000 7f7646cdb640 1 -- 172.21.15.115:0/1971260927 learned_addr learned my addr 172.21.15.115:0/1971260927 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:30.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.800+0000 7f7645cd9640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015afe0 0x7f764006b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481503950 0 0) 0x7f7638001490 con 0x7f764006b7b0 2023-12-01T01:36:30.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f762c003520 con 0x7f764006b7b0 2023-12-01T01:36:30.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 817299264 0 0) 0x7f762c003520 con 0x7f764006b7b0 2023-12-01T01:36:30.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015afe0 msgr2=0x7f764006b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:30.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015afe0 0x7f764006b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 >> v1:172.21.15.142:6789/0 conn(0x7f76401584b0 legacy=0x7f76400cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764015b550 con 0x7f764006b7b0 2023-12-01T01:36:30.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f7638001b80 con 0x7f764006b7b0 2023-12-01T01:36:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 >> v1:172.21.15.115:6789/0 conn(0x7f764006b7b0 legacy=0x7f76400cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015afe0 0x7f762c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f762c003f90 0x7f762c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f762c004d00 0x7f762c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764006bc40 con 0x7f762c003f90 2023-12-01T01:36:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762c00bae0 con 0x7f762c004d00 2023-12-01T01:36:30.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762c00bca0 con 0x7f764015afe0 2023-12-01T01:36:30.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.801+0000 7f7645cd9640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f762c003f90 0x7f762c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.802+0000 7f76464da640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015afe0 0x7f762c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.802+0000 7f7645cd9640 1 -- 172.21.15.115:0/1971260927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015afe0 msgr2=0x7f762c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:30.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.802+0000 7f7645cd9640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015afe0 0x7f762c003a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.802+0000 7f7645cd9640 1 -- 172.21.15.115:0/1971260927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f762c004d00 msgr2=0x7f762c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:30.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.802+0000 7f7645cd9640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f762c004d00 0x7f762c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.802+0000 7f7645cd9640 1 -- 172.21.15.115:0/1971260927 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c003520 con 0x7f762c003f90 2023-12-01T01:36:30.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.803+0000 7f7645cd9640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f762c003f90 0x7f762c0047c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7630009ac0 tx=0x7f762c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648f211a4f535a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:30.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.803+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763002c600 con 0x7f762c003f90 2023-12-01T01:36:30.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.803+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763002c760 con 0x7f762c003f90 2023-12-01T01:36:30.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.803+0000 7f76454d8640 1 -- 172.21.15.115:0/1971260927 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763002caf0 con 0x7f762c003f90 2023-12-01T01:36:30.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.804+0000 7f76474dc640 1 -- 172.21.15.115:0/1971260927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f762c003f90 msgr2=0x7f762c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.804+0000 7f76474dc640 1 --2- 172.21.15.115:0/1971260927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f762c003f90 0x7f762c0047c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7630009ac0 tx=0x7f762c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.804+0000 7f7646cdb640 1 -- 172.21.15.115:0/1971260927 reap_dead start 2023-12-01T01:36:30.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.804+0000 7f76474dc640 1 -- 172.21.15.115:0/1971260927 shutdown_connections 2023-12-01T01:36:30.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.804+0000 7f76474dc640 1 -- 172.21.15.115:0/1971260927 >> 172.21.15.115:0/1971260927 conn(0x7f76400ce640 msgr2=0x7f76400cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:30.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.804+0000 7f76474dc640 1 -- 172.21.15.115:0/1971260927 shutdown_connections 2023-12-01T01:36:30.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.804+0000 7f76474dc640 1 -- 172.21.15.115:0/1971260927 wait complete. 2023-12-01T01:36:30.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.806+0000 7f76474dc640 1 Processor -- start 2023-12-01T01:36:30.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.806+0000 7f76474dc640 1 -- start start 2023-12-01T01:36:30.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f76474dc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f762c003f90 0x7f7640070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f76474dc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015a9c0 0x7f76401f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f76474dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f764014c1b0 0x7f76400c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f76474dc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76401f6c00 con 0x7f764014c1b0 2023-12-01T01:36:30.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f76474dc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76400c5920 con 0x7f762c003f90 2023-12-01T01:36:30.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f76474dc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76400c72d0 con 0x7f764015a9c0 2023-12-01T01:36:30.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f7646cdb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f764014c1b0 0x7f76400c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f7646cdb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f764014c1b0 0x7f76400c7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60918/0 (socket says 172.21.15.115:60918) 2023-12-01T01:36:30.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f7646cdb640 1 -- 172.21.15.115:0/1164440890 learned_addr learned my addr 172.21.15.115:0/1164440890 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:30.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f7645cd9640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015a9c0 0x7f76401f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.807+0000 7f76464da640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f762c003f90 0x7f7640070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f7646cdb640 1 -- 172.21.15.115:0/1164440890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015a9c0 msgr2=0x7f76401f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f7646cdb640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015a9c0 0x7f76401f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f7646cdb640 1 -- 172.21.15.115:0/1164440890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f762c003f90 msgr2=0x7f7640070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f7646cdb640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f762c003f90 0x7f7640070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f7646cdb640 1 -- 172.21.15.115:0/1164440890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76401f6360 con 0x7f764014c1b0 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f76464da640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f762c003f90 0x7f7640070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f7646cdb640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f764014c1b0 0x7f76400c7a90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f76400cb020 tx=0x7f763800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f7636ffd640 1 -- 172.21.15.115:0/1164440890 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800a5b0 con 0x7f764014c1b0 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f7636ffd640 1 -- 172.21.15.115:0/1164440890 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763800a710 con 0x7f764014c1b0 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f7636ffd640 1 -- 172.21.15.115:0/1164440890 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800aac0 con 0x7f764014c1b0 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f76474dc640 1 -- 172.21.15.115:0/1164440890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76401f6640 con 0x7f764014c1b0 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.808+0000 7f76474dc640 1 -- 172.21.15.115:0/1164440890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764014c630 con 0x7f764014c1b0 2023-12-01T01:36:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.810+0000 7f76474dc640 1 -- 172.21.15.115:0/1164440890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76400c5680 con 0x7f764014c1b0 2023-12-01T01:36:30.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.810+0000 7f7636ffd640 1 -- 172.21.15.115:0/1164440890 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f763800ac20 con 0x7f764014c1b0 2023-12-01T01:36:30.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.811+0000 7f7636ffd640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f762406a460 0x7f762406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:30.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.811+0000 7f7636ffd640 1 -- 172.21.15.115:0/1164440890 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7638087ba0 con 0x7f764014c1b0 2023-12-01T01:36:30.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.811+0000 7f76464da640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f762406a460 0x7f762406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:30.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.812+0000 7f76464da640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f762406a460 0x7f762406c920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f763c0016f0 tx=0x7f763c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:30.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:30.816+0000 7f7636ffd640 1 -- 172.21.15.115:0/1164440890 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f76400c5680 con 0x7f764014c1b0 2023-12-01T01:36:31.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.007+0000 7f76474dc640 1 -- 172.21.15.115:0/1164440890 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76400c5a60 con 0x7f764014c1b0 2023-12-01T01:36:31.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.008+0000 7f7636ffd640 1 -- 172.21.15.115:0/1164440890 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f76400c5a60 con 0x7f764014c1b0 2023-12-01T01:36:31.009 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:31.009 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:11.641476+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 -- 172.21.15.115:0/1164440890 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f762406a460 msgr2=0x7f762406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f762406a460 0x7f762406c920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f763c0016f0 tx=0x7f763c009040 comp rx=0 tx=0).stop 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 -- 172.21.15.115:0/1164440890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f764014c1b0 msgr2=0x7f76400c7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f764014c1b0 0x7f76400c7a90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f76400cb020 tx=0x7f763800ccd0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 -- 172.21.15.115:0/1164440890 shutdown_connections 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f762406a460 0x7f762406c920 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f764015a9c0 0x7f76401f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f764014c1b0 0x7f76400c7a90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 --2- 172.21.15.115:0/1164440890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f762c003f90 0x7f7640070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 -- 172.21.15.115:0/1164440890 >> 172.21.15.115:0/1164440890 conn(0x7f76400ce640 msgr2=0x7f76400ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:31.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 -- 172.21.15.115:0/1164440890 shutdown_connections 2023-12-01T01:36:31.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.011+0000 7f76474dc640 1 -- 172.21.15.115:0/1164440890 wait complete. 2023-12-01T01:36:31.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:36:31.032 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:36:31.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.246+0000 7fea2a6ec640 1 Processor -- start 2023-12-01T01:36:31.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.246+0000 7fea2a6ec640 1 -- start start 2023-12-01T01:36:31.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea2a6ec640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea24142870 0x7fea240ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea2a6ec640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fea240b1e20 0x7fea2415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea2a6ec640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea240cd470 con 0x7fea240cd010 2023-12-01T01:36:31.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea2a6ec640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea240b3ae0 con 0x7fea240b1e20 2023-12-01T01:36:31.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea2a6ec640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea24142c50 con 0x7fea24142870 2023-12-01T01:36:31.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea1bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fea240cd010 0x7fea240b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38056/0 (socket says 172.21.15.115:38056) 2023-12-01T01:36:31.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea1bfff640 1 -- 172.21.15.115:0/121534341 learned_addr learned my addr 172.21.15.115:0/121534341 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:31.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea28c89640 1 --2- 172.21.15.115:0/121534341 >> v2:172.21.15.115:3300/0 conn(0x7fea240b1e20 0x7fea2415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea23fff640 1 --2- 172.21.15.115:0/121534341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea24142870 0x7fea240ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea28c89640 1 -- 172.21.15.115:0/121534341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea24142870 msgr2=0x7fea240ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:31.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea28c89640 1 --2- 172.21.15.115:0/121534341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea24142870 0x7fea240ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea28c89640 1 -- 172.21.15.115:0/121534341 >> v1:172.21.15.115:6789/0 conn(0x7fea240cd010 legacy=0x7fea240b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.248+0000 7fea28c89640 1 -- 172.21.15.115:0/121534341 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea240b2f30 con 0x7fea240b1e20 2023-12-01T01:36:31.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.250+0000 7fea28c89640 1 --2- 172.21.15.115:0/121534341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240b1e20 0x7fea2415a0d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009dc0 tx=0x7fea1c02be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c537553852a5567 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:31.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.250+0000 7fea237fe640 1 -- 172.21.15.115:0/121534341 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c02cc30 con 0x7fea240b1e20 2023-12-01T01:36:31.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.250+0000 7fea237fe640 1 -- 172.21.15.115:0/121534341 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c02cd90 con 0x7fea240b1e20 2023-12-01T01:36:31.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.250+0000 7fea237fe640 1 -- 172.21.15.115:0/121534341 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c034660 con 0x7fea240b1e20 2023-12-01T01:36:31.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.250+0000 7fea2a6ec640 1 -- 172.21.15.115:0/121534341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240b1e20 msgr2=0x7fea2415a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.250+0000 7fea2a6ec640 1 --2- 172.21.15.115:0/121534341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240b1e20 0x7fea2415a0d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009dc0 tx=0x7fea1c02be80 comp rx=0 tx=0).stop 2023-12-01T01:36:31.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.251+0000 7fea2a6ec640 1 -- 172.21.15.115:0/121534341 shutdown_connections 2023-12-01T01:36:31.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.251+0000 7fea2a6ec640 1 --2- 172.21.15.115:0/121534341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240b1e20 0x7fea2415a0d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.251+0000 7fea2a6ec640 1 --2- 172.21.15.115:0/121534341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea24142870 0x7fea240ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.251+0000 7fea2a6ec640 1 -- 172.21.15.115:0/121534341 >> 172.21.15.115:0/121534341 conn(0x7fea24065960 msgr2=0x7fea24065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:31.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.251+0000 7fea2a6ec640 1 -- 172.21.15.115:0/121534341 shutdown_connections 2023-12-01T01:36:31.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.251+0000 7fea2a6ec640 1 -- 172.21.15.115:0/121534341 wait complete. 2023-12-01T01:36:31.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.252+0000 7fea2a6ec640 1 Processor -- start 2023-12-01T01:36:31.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.252+0000 7fea2a6ec640 1 -- start start 2023-12-01T01:36:31.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.253+0000 7fea2a6ec640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea240b1e20 0x7fea24136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.253+0000 7fea2a6ec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240cd010 0x7fea2413c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.253+0000 7fea2a6ec640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea24142870 0x7fea2413dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.253+0000 7fea2a6ec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2413cd50 con 0x7fea240cd010 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.253+0000 7fea2a6ec640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2413e510 con 0x7fea24142870 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.253+0000 7fea2a6ec640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2413ba50 con 0x7fea240b1e20 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea1bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240cd010 0x7fea2413c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea1bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240cd010 0x7fea2413c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60950/0 (socket says 172.21.15.115:60950) 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea1bfff640 1 -- 172.21.15.115:0/2434088364 learned_addr learned my addr 172.21.15.115:0/2434088364 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea1bfff640 1 -- 172.21.15.115:0/2434088364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea240b1e20 msgr2=0x7fea24136d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea1bfff640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea240b1e20 0x7fea24136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea1bfff640 1 -- 172.21.15.115:0/2434088364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea24142870 msgr2=0x7fea2413dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea1bfff640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea24142870 0x7fea2413dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea1bfff640 1 -- 172.21.15.115:0/2434088364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c009a70 con 0x7fea240cd010 2023-12-01T01:36:31.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea1bfff640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240cd010 0x7fea2413c810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00c4a0 tx=0x7fea0c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:31.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea217fa640 1 -- 172.21.15.115:0/2434088364 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c005ce0 con 0x7fea240cd010 2023-12-01T01:36:31.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea217fa640 1 -- 172.21.15.115:0/2434088364 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c005e40 con 0x7fea240cd010 2023-12-01T01:36:31.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea2a6ec640 1 -- 172.21.15.115:0/2434088364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2413bd30 con 0x7fea240cd010 2023-12-01T01:36:31.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.254+0000 7fea217fa640 1 -- 172.21.15.115:0/2434088364 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c007270 con 0x7fea240cd010 2023-12-01T01:36:31.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.255+0000 7fea2a6ec640 1 -- 172.21.15.115:0/2434088364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2413c2f0 con 0x7fea240cd010 2023-12-01T01:36:31.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.256+0000 7fea2a6ec640 1 -- 172.21.15.115:0/2434088364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2415e5c0 con 0x7fea240cd010 2023-12-01T01:36:31.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.257+0000 7fea217fa640 1 -- 172.21.15.115:0/2434088364 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fea0c0073d0 con 0x7fea240cd010 2023-12-01T01:36:31.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.257+0000 7fea217fa640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea0406a430 0x7fea0406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.258+0000 7fea217fa640 1 -- 172.21.15.115:0/2434088364 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea0c087d80 con 0x7fea240cd010 2023-12-01T01:36:31.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.258+0000 7fea23fff640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea0406a430 0x7fea0406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.258+0000 7fea23fff640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea0406a430 0x7fea0406c8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea14006fd0 tx=0x7fea14008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:31.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.262+0000 7fea217fa640 1 -- 172.21.15.115:0/2434088364 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fea0c053410 con 0x7fea240cd010 2023-12-01T01:36:31.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.446+0000 7fea2a6ec640 1 -- 172.21.15.115:0/2434088364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea2404e890 con 0x7fea240cd010 2023-12-01T01:36:31.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.447+0000 7fea217fa640 1 -- 172.21.15.115:0/2434088364 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea0c0170a0 con 0x7fea240cd010 2023-12-01T01:36:31.447 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:36:31.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 -- 172.21.15.115:0/2434088364 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea0406a430 msgr2=0x7fea0406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea0406a430 0x7fea0406c8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea14006fd0 tx=0x7fea14008040 comp rx=0 tx=0).stop 2023-12-01T01:36:31.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 -- 172.21.15.115:0/2434088364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240cd010 msgr2=0x7fea2413c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240cd010 0x7fea2413c810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00c4a0 tx=0x7fea0c00c960 comp rx=0 tx=0).stop 2023-12-01T01:36:31.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 -- 172.21.15.115:0/2434088364 shutdown_connections 2023-12-01T01:36:31.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea0406a430 0x7fea0406c8f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea24142870 0x7fea2413dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea240cd010 0x7fea2413c810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 --2- 172.21.15.115:0/2434088364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea240b1e20 0x7fea24136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 -- 172.21.15.115:0/2434088364 >> 172.21.15.115:0/2434088364 conn(0x7fea24065960 msgr2=0x7fea24065fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:31.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 -- 172.21.15.115:0/2434088364 shutdown_connections 2023-12-01T01:36:31.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.450+0000 7fea2a6ec640 1 -- 172.21.15.115:0/2434088364 wait complete. 2023-12-01T01:36:31.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:36:31.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.680+0000 7f60f1204640 1 Processor -- start 2023-12-01T01:36:31.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.681+0000 7f60f1204640 1 -- start start 2023-12-01T01:36:31.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.682+0000 7f60f1204640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f60ec15cd90 0x7f60ec0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.682+0000 7f60f1204640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ec0ce5e0 con 0x7f60ec0b5fa0 2023-12-01T01:36:31.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.682+0000 7f60f1204640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ec0ce300 con 0x7f60ec0cfd60 2023-12-01T01:36:31.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.682+0000 7f60f1204640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec0d0ec0 con 0x7f60ec15cd90 2023-12-01T01:36:31.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.682+0000 7f60e3fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f60ec0b5fa0 0x7f60ec0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38068/0 (socket says 172.21.15.115:38068) 2023-12-01T01:36:31.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.682+0000 7f60e3fff640 1 -- 172.21.15.115:0/1507103909 learned_addr learned my addr 172.21.15.115:0/1507103909 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:31.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237341607 0 0) 0x7f60d4001110 con 0x7f60ec0b5fa0 2023-12-01T01:36:31.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60d8003520 con 0x7f60ec0b5fa0 2023-12-01T01:36:31.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ead9d640 1 --2- 172.21.15.115:0/1507103909 >> v2:172.21.15.142:3300/0 conn(0x7f60ec15cd90 0x7f60ec0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2359778532 0 0) 0x7f60d8003520 con 0x7f60ec0b5fa0 2023-12-01T01:36:31.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 >> v2:172.21.15.142:3300/0 conn(0x7f60ec15cd90 msgr2=0x7f60ec0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:31.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 --2- 172.21.15.115:0/1507103909 >> v2:172.21.15.142:3300/0 conn(0x7f60ec15cd90 0x7f60ec0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 >> v1:172.21.15.142:6789/0 conn(0x7f60ec0cfd60 legacy=0x7f60ec0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:31.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec0d10a0 con 0x7f60ec0b5fa0 2023-12-01T01:36:31.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f60d4001800 con 0x7f60ec0b5fa0 2023-12-01T01:36:31.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 >> v1:172.21.15.115:6789/0 conn(0x7f60ec0b5fa0 legacy=0x7f60ec0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d80039c0 0x7f60d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80047c0 0x7f60d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60d80051d0 0x7f60d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.683+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec0d0ec0 con 0x7f60d80047c0 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d800c120 con 0x7f60d80039c0 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d800c2e0 con 0x7f60d80051d0 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60e3fff640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80047c0 0x7f60d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60ead9d640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d80039c0 0x7f60d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60eb59e640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60d80051d0 0x7f60d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60e3fff640 1 -- 172.21.15.115:0/1507103909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60d80051d0 msgr2=0x7f60d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60e3fff640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60d80051d0 0x7f60d800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60e3fff640 1 -- 172.21.15.115:0/1507103909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d80039c0 msgr2=0x7f60d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60e3fff640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d80039c0 0x7f60d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60e3fff640 1 -- 172.21.15.115:0/1507103909 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d8003520 con 0x7f60d80047c0 2023-12-01T01:36:31.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60eb59e640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60d80051d0 0x7f60d800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:31.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.684+0000 7f60eb59e640 1 -- 172.21.15.115:0/1507103909 reap_dead start 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.685+0000 7f60e3fff640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80047c0 0x7f60d8004c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0b6410 tx=0x7f60d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b7ee49321c8ec1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.686+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4005ad0 con 0x7f60d80047c0 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.686+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d4005c50 con 0x7f60d80047c0 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.686+0000 7f60ea59c640 1 -- 172.21.15.115:0/1507103909 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d402c7c0 con 0x7f60d80047c0 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.686+0000 7f60f1204640 1 -- 172.21.15.115:0/1507103909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80047c0 msgr2=0x7f60d8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.686+0000 7f60f1204640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80047c0 0x7f60d8004c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0b6410 tx=0x7f60d80057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.686+0000 7f60f1204640 1 -- 172.21.15.115:0/1507103909 shutdown_connections 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.686+0000 7f60f1204640 1 --2- 172.21.15.115:0/1507103909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80047c0 0x7f60d8004c40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.686+0000 7f60f1204640 1 -- 172.21.15.115:0/1507103909 >> 172.21.15.115:0/1507103909 conn(0x7f60ec065960 msgr2=0x7f60ec065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.686+0000 7f60f1204640 1 -- 172.21.15.115:0/1507103909 shutdown_connections 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.686+0000 7f60f1204640 1 -- 172.21.15.115:0/1507103909 wait complete. 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.688+0000 7f60f1204640 1 Processor -- start 2023-12-01T01:36:31.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.688+0000 7f60f1204640 1 -- start start 2023-12-01T01:36:31.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.689+0000 7f60f1204640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d80047c0 0x7f60ec0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.689+0000 7f60f1204640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ec14a160 0x7f60ec14a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.689+0000 7f60f1204640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec1480d0 0x7f60ec148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.689+0000 7f60f1204640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec149930 con 0x7f60ec14a160 2023-12-01T01:36:31.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.689+0000 7f60f1204640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec148a90 con 0x7f60d80047c0 2023-12-01T01:36:31.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.689+0000 7f60f1204640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec14afe0 con 0x7f60ec1480d0 2023-12-01T01:36:31.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.689+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ec14a160 0x7f60ec14a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.689+0000 7f60e3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ec14a160 0x7f60ec14a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60976/0 (socket says 172.21.15.115:60976) 2023-12-01T01:36:31.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.689+0000 7f60e3fff640 1 -- 172.21.15.115:0/1019030543 learned_addr learned my addr 172.21.15.115:0/1019030543 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:31.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60ead9d640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d80047c0 0x7f60ec0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60eb59e640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec1480d0 0x7f60ec148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60e3fff640 1 -- 172.21.15.115:0/1019030543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec1480d0 msgr2=0x7f60ec148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60e3fff640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec1480d0 0x7f60ec148550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60e3fff640 1 -- 172.21.15.115:0/1019030543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d80047c0 msgr2=0x7f60ec0b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60e3fff640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d80047c0 0x7f60ec0b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60e3fff640 1 -- 172.21.15.115:0/1019030543 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec14ab00 con 0x7f60ec14a160 2023-12-01T01:36:31.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60ead9d640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d80047c0 0x7f60ec0b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:31.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60e3fff640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ec14a160 0x7f60ec14a5c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f60d402c430 tx=0x7f60d402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:31.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60e37fe640 1 -- 172.21.15.115:0/1019030543 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d403c040 con 0x7f60ec14a160 2023-12-01T01:36:31.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60e37fe640 1 -- 172.21.15.115:0/1019030543 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d402cbd0 con 0x7f60ec14a160 2023-12-01T01:36:31.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.690+0000 7f60e37fe640 1 -- 172.21.15.115:0/1019030543 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4007070 con 0x7f60ec14a160 2023-12-01T01:36:31.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.691+0000 7f60f1204640 1 -- 172.21.15.115:0/1019030543 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec148cb0 con 0x7f60ec14a160 2023-12-01T01:36:31.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.691+0000 7f60f1204640 1 -- 172.21.15.115:0/1019030543 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec149270 con 0x7f60ec14a160 2023-12-01T01:36:31.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.692+0000 7f60f1204640 1 -- 172.21.15.115:0/1019030543 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec149a70 con 0x7f60ec14a160 2023-12-01T01:36:31.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.693+0000 7f60e37fe640 1 -- 172.21.15.115:0/1019030543 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f60d4032070 con 0x7f60ec14a160 2023-12-01T01:36:31.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.693+0000 7f60e37fe640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f60cc06a430 0x7f60cc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:31.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.694+0000 7f60e37fe640 1 -- 172.21.15.115:0/1019030543 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60d40a9870 con 0x7f60ec14a160 2023-12-01T01:36:31.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.694+0000 7f60ead9d640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f60cc06a430 0x7f60cc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:31.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.694+0000 7f60ead9d640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f60cc06a430 0x7f60cc06c8f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60dc006fd0 tx=0x7f60dc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:31.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.698+0000 7f60e37fe640 1 -- 172.21.15.115:0/1019030543 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f60ec149a70 con 0x7f60ec14a160 2023-12-01T01:36:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.873+0000 7f60f1204640 1 -- 172.21.15.115:0/1019030543 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60ec0ce5e0 con 0x7f60ec14a160 2023-12-01T01:36:31.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.874+0000 7f60e37fe640 1 -- 172.21.15.115:0/1019030543 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f60ec0ce5e0 con 0x7f60ec14a160 2023-12-01T01:36:31.874 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:36:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.876+0000 7f60f1204640 1 -- 172.21.15.115:0/1019030543 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f60cc06a430 msgr2=0x7f60cc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.876+0000 7f60f1204640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f60cc06a430 0x7f60cc06c8f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f60dc006fd0 tx=0x7f60dc008040 comp rx=0 tx=0).stop 2023-12-01T01:36:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.876+0000 7f60f1204640 1 -- 172.21.15.115:0/1019030543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ec14a160 msgr2=0x7f60ec14a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.876+0000 7f60f1204640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ec14a160 0x7f60ec14a5c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f60d402c430 tx=0x7f60d402e070 comp rx=0 tx=0).stop 2023-12-01T01:36:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.876+0000 7f60f1204640 1 -- 172.21.15.115:0/1019030543 shutdown_connections 2023-12-01T01:36:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.876+0000 7f60f1204640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f60cc06a430 0x7f60cc06c8f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.876+0000 7f60f1204640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ec1480d0 0x7f60ec148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.876+0000 7f60f1204640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ec14a160 0x7f60ec14a5c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.876+0000 7f60f1204640 1 --2- 172.21.15.115:0/1019030543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60d80047c0 0x7f60ec0b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:31.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.876+0000 7f60f1204640 1 -- 172.21.15.115:0/1019030543 >> 172.21.15.115:0/1019030543 conn(0x7f60ec065960 msgr2=0x7f60ec0b7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:31.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.877+0000 7f60f1204640 1 -- 172.21.15.115:0/1019030543 shutdown_connections 2023-12-01T01:36:31.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:31.877+0000 7f60f1204640 1 -- 172.21.15.115:0/1019030543 wait complete. 2023-12-01T01:36:31.895 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:36:32.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.110+0000 7f727c516640 1 Processor -- start 2023-12-01T01:36:32.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.110+0000 7f727c516640 1 -- start start 2023-12-01T01:36:32.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.112+0000 7f727c516640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7274159970 0x7f72740ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.112+0000 7f727c516640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 0x7f72740cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.112+0000 7f727c516640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727415a760 con 0x7f7274159d50 2023-12-01T01:36:32.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.112+0000 7f727c516640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727415a1d0 con 0x7f7274159970 2023-12-01T01:36:32.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.112+0000 7f727c516640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7274070ee0 con 0x7f727415abf0 2023-12-01T01:36:32.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.112+0000 7f727aab3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 0x7f72740cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.113+0000 7f727aab3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 0x7f72740cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:58938/0 (socket says 172.21.15.115:58938) 2023-12-01T01:36:32.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.113+0000 7f727aab3640 1 -- 172.21.15.115:0/60599608 learned_addr learned my addr 172.21.15.115:0/60599608 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:32.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.113+0000 7f727a2b2640 1 --2- 172.21.15.115:0/60599608 >> v2:172.21.15.142:3300/0 conn(0x7f7274159970 0x7f72740ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.113+0000 7f727aab3640 1 -- 172.21.15.115:0/60599608 >> v2:172.21.15.142:3300/0 conn(0x7f7274159970 msgr2=0x7f72740ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:32.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.113+0000 7f727aab3640 1 --2- 172.21.15.115:0/60599608 >> v2:172.21.15.142:3300/0 conn(0x7f7274159970 0x7f72740ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.113+0000 7f727aab3640 1 -- 172.21.15.115:0/60599608 >> v1:172.21.15.142:6789/0 conn(0x7f7274159d50 legacy=0x7f72740707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.113+0000 7f727aab3640 1 -- 172.21.15.115:0/60599608 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727415b060 con 0x7f727415abf0 2023-12-01T01:36:32.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.114+0000 7f727aab3640 1 --2- 172.21.15.115:0/60599608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 0x7f72740cb510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f726c009e50 tx=0x7f726c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=addfd922ced46ca9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:32.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.115+0000 7f72792b0640 1 -- 172.21.15.115:0/60599608 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c02cb00 con 0x7f727415abf0 2023-12-01T01:36:32.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.115+0000 7f72792b0640 1 -- 172.21.15.115:0/60599608 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c02cc60 con 0x7f727415abf0 2023-12-01T01:36:32.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.115+0000 7f72792b0640 1 -- 172.21.15.115:0/60599608 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c0343f0 con 0x7f727415abf0 2023-12-01T01:36:32.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.116+0000 7f727c516640 1 -- 172.21.15.115:0/60599608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 msgr2=0x7f72740cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.116+0000 7f727c516640 1 --2- 172.21.15.115:0/60599608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 0x7f72740cb510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f726c009e50 tx=0x7f726c02bf60 comp rx=0 tx=0).stop 2023-12-01T01:36:32.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.116+0000 7f727c516640 1 -- 172.21.15.115:0/60599608 shutdown_connections 2023-12-01T01:36:32.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.116+0000 7f727c516640 1 --2- 172.21.15.115:0/60599608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 0x7f72740cb510 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.116+0000 7f727c516640 1 --2- 172.21.15.115:0/60599608 >> v2:172.21.15.142:3300/0 conn(0x7f7274159970 0x7f72740ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.116+0000 7f727c516640 1 -- 172.21.15.115:0/60599608 >> 172.21.15.115:0/60599608 conn(0x7f7274065b40 msgr2=0x7f7274065f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:32.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.116+0000 7f727c516640 1 -- 172.21.15.115:0/60599608 shutdown_connections 2023-12-01T01:36:32.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.116+0000 7f727c516640 1 -- 172.21.15.115:0/60599608 wait complete. 2023-12-01T01:36:32.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.118+0000 7f727c516640 1 Processor -- start 2023-12-01T01:36:32.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.118+0000 7f727c516640 1 -- start start 2023-12-01T01:36:32.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.119+0000 7f727c516640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7274159970 0x7f7274152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.119+0000 7f727c516640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7274159d50 0x7f72740cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.119+0000 7f727c516640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 0x7f72741549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.119+0000 7f727c516640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72741524b0 con 0x7f7274159970 2023-12-01T01:36:32.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.119+0000 7f727c516640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72741555b0 con 0x7f7274159d50 2023-12-01T01:36:32.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.119+0000 7f727c516640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7274155890 con 0x7f727415abf0 2023-12-01T01:36:32.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.119+0000 7f727a2b2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7274159970 0x7f7274152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.119+0000 7f727a2b2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7274159970 0x7f7274152ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60978/0 (socket says 172.21.15.115:60978) 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.119+0000 7f727a2b2640 1 -- 172.21.15.115:0/1657169102 learned_addr learned my addr 172.21.15.115:0/1657169102 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f727aab3640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 0x7f72741549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f727a2b2640 1 -- 172.21.15.115:0/1657169102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 msgr2=0x7f72741549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f7279ab1640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7274159d50 0x7f72740cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f727a2b2640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 0x7f72741549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f727a2b2640 1 -- 172.21.15.115:0/1657169102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7274159d50 msgr2=0x7f72740cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f727a2b2640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7274159d50 0x7f72740cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f727a2b2640 1 -- 172.21.15.115:0/1657169102 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c009b30 con 0x7f7274159970 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f7279ab1640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7274159d50 0x7f72740cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:32.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f727a2b2640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7274159970 0x7f7274152ec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f727000e9b0 tx=0x7f727000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:32.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f726affd640 1 -- 172.21.15.115:0/1657169102 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270009670 con 0x7f7274159970 2023-12-01T01:36:32.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f726affd640 1 -- 172.21.15.115:0/1657169102 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727000c9a0 con 0x7f7274159970 2023-12-01T01:36:32.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f726affd640 1 -- 172.21.15.115:0/1657169102 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727000cd50 con 0x7f7274159970 2023-12-01T01:36:32.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f727c516640 1 -- 172.21.15.115:0/1657169102 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72741526c0 con 0x7f7274159970 2023-12-01T01:36:32.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.120+0000 7f727c516640 1 -- 172.21.15.115:0/1657169102 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7274151c30 con 0x7f7274159970 2023-12-01T01:36:32.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.122+0000 7f727c516640 1 -- 172.21.15.115:0/1657169102 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7274155ab0 con 0x7f7274159970 2023-12-01T01:36:32.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.122+0000 7f726affd640 1 -- 172.21.15.115:0/1657169102 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7270013470 con 0x7f7274159970 2023-12-01T01:36:32.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.123+0000 7f726affd640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f725806a430 0x7f725806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.123+0000 7f726affd640 1 -- 172.21.15.115:0/1657169102 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7270088c10 con 0x7f7274159970 2023-12-01T01:36:32.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.124+0000 7f7279ab1640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f725806a430 0x7f725806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.124+0000 7f7279ab1640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f725806a430 0x7f725806c8f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f72741f5b20 tx=0x7f7264009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:32.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.127+0000 7f726affd640 1 -- 172.21.15.115:0/1657169102 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7270054170 con 0x7f7274159970 2023-12-01T01:36:32.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.317+0000 7f727c516640 1 -- 172.21.15.115:0/1657169102 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f727404e890 con 0x7f7274159970 2023-12-01T01:36:32.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.318+0000 7f726affd640 1 -- 172.21.15.115:0/1657169102 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7270057790 con 0x7f7274159970 2023-12-01T01:36:32.318 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:36:32.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 -- 172.21.15.115:0/1657169102 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f725806a430 msgr2=0x7f725806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f725806a430 0x7f725806c8f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f72741f5b20 tx=0x7f7264009040 comp rx=0 tx=0).stop 2023-12-01T01:36:32.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 -- 172.21.15.115:0/1657169102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7274159970 msgr2=0x7f7274152ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7274159970 0x7f7274152ec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f727000e9b0 tx=0x7f727000ee70 comp rx=0 tx=0).stop 2023-12-01T01:36:32.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 -- 172.21.15.115:0/1657169102 shutdown_connections 2023-12-01T01:36:32.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f725806a430 0x7f725806c8f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f727415abf0 0x7f72741549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7274159d50 0x7f72740cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 --2- 172.21.15.115:0/1657169102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7274159970 0x7f7274152ec0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 -- 172.21.15.115:0/1657169102 >> 172.21.15.115:0/1657169102 conn(0x7f7274065b40 msgr2=0x7f72740c3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:32.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 -- 172.21.15.115:0/1657169102 shutdown_connections 2023-12-01T01:36:32.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.320+0000 7f727c516640 1 -- 172.21.15.115:0/1657169102 wait complete. 2023-12-01T01:36:32.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:36:32.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.542+0000 7fe58a5cb640 1 Processor -- start 2023-12-01T01:36:32.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.543+0000 7fe58a5cb640 1 -- start start 2023-12-01T01:36:32.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.544+0000 7fe58a5cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 0x7fe58406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.544+0000 7fe58a5cb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe58406f2f0 0x7fe58406bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.544+0000 7fe58a5cb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5840add10 con 0x7fe58415e2f0 2023-12-01T01:36:32.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.544+0000 7fe58a5cb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5840c5760 con 0x7fe58406f2f0 2023-12-01T01:36:32.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.544+0000 7fe58a5cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58415e6d0 con 0x7fe5840cad70 2023-12-01T01:36:32.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.544+0000 7fe583fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe58415e2f0 0x7fe5840ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38084/0 (socket says 172.21.15.115:38084) 2023-12-01T01:36:32.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.544+0000 7fe583fff640 1 -- 172.21.15.115:0/1674042362 learned_addr learned my addr 172.21.15.115:0/1674042362 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:32.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.544+0000 7fe588b68640 1 --2- 172.21.15.115:0/1674042362 >> v2:172.21.15.115:3300/0 conn(0x7fe58406f2f0 0x7fe58406bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.545+0000 7fe57bfff640 1 --2- 172.21.15.115:0/1674042362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 0x7fe58406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.545+0000 7fe5837fe640 1 -- 172.21.15.115:0/1674042362 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506188338 0 0) 0x7fe5840add10 con 0x7fe58415e2f0 2023-12-01T01:36:32.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.545+0000 7fe5837fe640 1 -- 172.21.15.115:0/1674042362 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe570003520 con 0x7fe58415e2f0 2023-12-01T01:36:32.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.545+0000 7fe588b68640 1 -- 172.21.15.115:0/1674042362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 msgr2=0x7fe58406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:32.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.545+0000 7fe588b68640 1 --2- 172.21.15.115:0/1674042362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 0x7fe58406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.545+0000 7fe588b68640 1 -- 172.21.15.115:0/1674042362 >> v1:172.21.15.115:6789/0 conn(0x7fe58415e2f0 legacy=0x7fe5840ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.545+0000 7fe588b68640 1 -- 172.21.15.115:0/1674042362 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5840cf7b0 con 0x7fe58406f2f0 2023-12-01T01:36:32.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.545+0000 7fe57bfff640 1 --2- 172.21.15.115:0/1674042362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 0x7fe58406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:32.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.546+0000 7fe588b68640 1 --2- 172.21.15.115:0/1674042362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 0x7fe58406bb00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009d90 tx=0x7fe57c02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a18772e703ba68 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.546+0000 7fe5837fe640 1 -- 172.21.15.115:0/1674042362 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c02cb30 con 0x7fe58406f2f0 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.546+0000 7fe5837fe640 1 -- 172.21.15.115:0/1674042362 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c02cc90 con 0x7fe58406f2f0 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.546+0000 7fe5837fe640 1 -- 172.21.15.115:0/1674042362 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c034440 con 0x7fe58406f2f0 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.547+0000 7fe58a5cb640 1 -- 172.21.15.115:0/1674042362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 msgr2=0x7fe58406bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.547+0000 7fe58a5cb640 1 --2- 172.21.15.115:0/1674042362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 0x7fe58406bb00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009d90 tx=0x7fe57c02bd80 comp rx=0 tx=0).stop 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.547+0000 7fe58a5cb640 1 -- 172.21.15.115:0/1674042362 shutdown_connections 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.547+0000 7fe58a5cb640 1 --2- 172.21.15.115:0/1674042362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 0x7fe58406bb00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.547+0000 7fe58a5cb640 1 --2- 172.21.15.115:0/1674042362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 0x7fe58406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.547+0000 7fe58a5cb640 1 -- 172.21.15.115:0/1674042362 >> 172.21.15.115:0/1674042362 conn(0x7fe5840ce880 msgr2=0x7fe5840cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.547+0000 7fe58a5cb640 1 -- 172.21.15.115:0/1674042362 shutdown_connections 2023-12-01T01:36:32.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.547+0000 7fe58a5cb640 1 -- 172.21.15.115:0/1674042362 wait complete. 2023-12-01T01:36:32.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.549+0000 7fe58a5cb640 1 Processor -- start 2023-12-01T01:36:32.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.549+0000 7fe58a5cb640 1 -- start start 2023-12-01T01:36:32.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe58a5cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 0x7fe5841588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe58a5cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 0x7fe5841f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe58a5cb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe58415e2f0 0x7fe5841f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe58a5cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840c6da0 con 0x7fe58406f2f0 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe58a5cb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5841f31d0 con 0x7fe58415e2f0 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe58a5cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840c6400 con 0x7fe5840cad70 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe583fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 0x7fe5841588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe583fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 0x7fe5841588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60996/0 (socket says 172.21.15.115:60996) 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe583fff640 1 -- 172.21.15.115:0/38455417 learned_addr learned my addr 172.21.15.115:0/38455417 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe588b68640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe58415e2f0 0x7fe5841f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe57bfff640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 0x7fe5841f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.550+0000 7fe583fff640 1 -- 172.21.15.115:0/38455417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 msgr2=0x7fe5841f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.551+0000 7fe583fff640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 0x7fe5841f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.551+0000 7fe583fff640 1 -- 172.21.15.115:0/38455417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe58415e2f0 msgr2=0x7fe5841f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.551+0000 7fe583fff640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe58415e2f0 0x7fe5841f3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.551+0000 7fe583fff640 1 -- 172.21.15.115:0/38455417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c009a70 con 0x7fe58406f2f0 2023-12-01T01:36:32.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.551+0000 7fe583fff640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 0x7fe5841588e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c440 tx=0x7fe57400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:32.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.551+0000 7fe5817fa640 1 -- 172.21.15.115:0/38455417 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574005ce0 con 0x7fe58406f2f0 2023-12-01T01:36:32.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.551+0000 7fe5817fa640 1 -- 172.21.15.115:0/38455417 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe574005e40 con 0x7fe58406f2f0 2023-12-01T01:36:32.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.551+0000 7fe5817fa640 1 -- 172.21.15.115:0/38455417 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5740072b0 con 0x7fe58406f2f0 2023-12-01T01:36:32.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.551+0000 7fe58a5cb640 1 -- 172.21.15.115:0/38455417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5840c66e0 con 0x7fe58406f2f0 2023-12-01T01:36:32.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.551+0000 7fe58a5cb640 1 -- 172.21.15.115:0/38455417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe584159400 con 0x7fe58406f2f0 2023-12-01T01:36:32.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.552+0000 7fe58a5cb640 1 -- 172.21.15.115:0/38455417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58406fda0 con 0x7fe58406f2f0 2023-12-01T01:36:32.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.553+0000 7fe5817fa640 1 -- 172.21.15.115:0/38455417 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe574007410 con 0x7fe58406f2f0 2023-12-01T01:36:32.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.554+0000 7fe5817fa640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a460 0x7fe56406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:32.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.554+0000 7fe5817fa640 1 -- 172.21.15.115:0/38455417 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe574087e40 con 0x7fe58406f2f0 2023-12-01T01:36:32.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.554+0000 7fe57bfff640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a460 0x7fe56406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:32.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.555+0000 7fe57bfff640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a460 0x7fe56406c920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe58415ab80 tx=0x7fe56c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:32.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.558+0000 7fe5817fa640 1 -- 172.21.15.115:0/38455417 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe5740534e0 con 0x7fe58406f2f0 2023-12-01T01:36:32.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.784+0000 7fe58a5cb640 1 -- 172.21.15.115:0/38455417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5840c6ee0 con 0x7fe58406f2f0 2023-12-01T01:36:32.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.785+0000 7fe5817fa640 1 -- 172.21.15.115:0/38455417 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4654 (secure 0 0 0) 0x7fe5840c6ee0 con 0x7fe58406f2f0 2023-12-01T01:36:32.786 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:32.786 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3656795862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3656795862},{"type":"v1","addr":"172.21.15.142:6834","nonce":3656795862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1125002206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1125002206},{"type":"v1","addr":"172.21.15.115:6836","nonce":1125002206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3350782192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3350782192},{"type":"v1","addr":"172.21.15.115:6837","nonce":3350782192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:36:10.637450+0000","modified":"2023-12-01T01:36:13.663004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2808777047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2808777047},{"type":"v1","addr":"172.21.15.142:6835","nonce":2808777047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:36:32.790 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 8 2023-12-01T01:36:32.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 -- 172.21.15.115:0/38455417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a460 msgr2=0x7fe56406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a460 0x7fe56406c920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe58415ab80 tx=0x7fe56c008040 comp rx=0 tx=0).stop 2023-12-01T01:36:32.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 -- 172.21.15.115:0/38455417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 msgr2=0x7fe5841588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:32.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 0x7fe5841588e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c440 tx=0x7fe57400c900 comp rx=0 tx=0).stop 2023-12-01T01:36:32.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 -- 172.21.15.115:0/38455417 shutdown_connections 2023-12-01T01:36:32.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a460 0x7fe56406c920 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5840cad70 0x7fe5841f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe58415e2f0 0x7fe5841f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 --2- 172.21.15.115:0/38455417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe58406f2f0 0x7fe5841588e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:32.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 -- 172.21.15.115:0/38455417 >> 172.21.15.115:0/38455417 conn(0x7fe5840ce880 msgr2=0x7fe5840cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:32.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 -- 172.21.15.115:0/38455417 shutdown_connections 2023-12-01T01:36:32.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:32.788+0000 7fe58a5cb640 1 -- 172.21.15.115:0/38455417 wait complete. 2023-12-01T01:36:32.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:33.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.021+0000 7fb22d6b6640 1 Processor -- start 2023-12-01T01:36:33.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.022+0000 7fb22d6b6640 1 -- start start 2023-12-01T01:36:33.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.023+0000 7fb22d6b6640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb22815e2f0 0x7fb2280ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.023+0000 7fb22d6b6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb22806f2f0 0x7fb2280caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.023+0000 7fb22d6b6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2280add10 con 0x7fb2280adf10 2023-12-01T01:36:33.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.023+0000 7fb22d6b6640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2280c5760 con 0x7fb22815e2f0 2023-12-01T01:36:33.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.023+0000 7fb22d6b6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22815e6d0 con 0x7fb22806f2f0 2023-12-01T01:36:33.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.023+0000 7fb226ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb22815e2f0 0x7fb2280ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:33.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.024+0000 7fb226ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb22815e2f0 0x7fb2280ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60998/0 (socket says 172.21.15.115:60998) 2023-12-01T01:36:33.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.024+0000 7fb226ffd640 1 -- 172.21.15.115:0/1042425248 learned_addr learned my addr 172.21.15.115:0/1042425248 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:33.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.024+0000 7fb2277fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb2280adf10 0x7fb2280c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38098/0 (socket says 172.21.15.115:38098) 2023-12-01T01:36:33.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.024+0000 7fb226ffd640 1 -- 172.21.15.115:0/1042425248 >> v2:172.21.15.142:3300/0 conn(0x7fb22806f2f0 msgr2=0x7fb2280caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:33.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.024+0000 7fb226ffd640 1 --2- 172.21.15.115:0/1042425248 >> v2:172.21.15.142:3300/0 conn(0x7fb22806f2f0 0x7fb2280caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.024+0000 7fb226ffd640 1 -- 172.21.15.115:0/1042425248 >> v1:172.21.15.115:6789/0 conn(0x7fb2280adf10 legacy=0x7fb2280c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.024+0000 7fb226ffd640 1 -- 172.21.15.115:0/1042425248 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280cf7b0 con 0x7fb22815e2f0 2023-12-01T01:36:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.025+0000 7fb226ffd640 1 --2- 172.21.15.115:0/1042425248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22815e2f0 0x7fb2280ca660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb22806de80 tx=0x7fb21802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9ca16117df5de3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.025+0000 7fb2267fc640 1 -- 172.21.15.115:0/1042425248 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2180317e0 con 0x7fb22815e2f0 2023-12-01T01:36:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.025+0000 7fb2267fc640 1 -- 172.21.15.115:0/1042425248 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218031940 con 0x7fb22815e2f0 2023-12-01T01:36:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.025+0000 7fb2267fc640 1 -- 172.21.15.115:0/1042425248 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218031cd0 con 0x7fb22815e2f0 2023-12-01T01:36:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.026+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1042425248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22815e2f0 msgr2=0x7fb2280ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.026+0000 7fb22d6b6640 1 --2- 172.21.15.115:0/1042425248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22815e2f0 0x7fb2280ca660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb22806de80 tx=0x7fb21802b880 comp rx=0 tx=0).stop 2023-12-01T01:36:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.026+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1042425248 shutdown_connections 2023-12-01T01:36:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.026+0000 7fb22d6b6640 1 --2- 172.21.15.115:0/1042425248 >> v2:172.21.15.142:3300/0 conn(0x7fb22806f2f0 0x7fb2280caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.026+0000 7fb22d6b6640 1 --2- 172.21.15.115:0/1042425248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22815e2f0 0x7fb2280ca660 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.026+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1042425248 >> 172.21.15.115:0/1042425248 conn(0x7fb2280ce880 msgr2=0x7fb2280cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.026+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1042425248 shutdown_connections 2023-12-01T01:36:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.026+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1042425248 wait complete. 2023-12-01T01:36:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.028+0000 7fb22d6b6640 1 Processor -- start 2023-12-01T01:36:33.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.028+0000 7fb22d6b6640 1 -- start start 2023-12-01T01:36:33.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.029+0000 7fb22d6b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22806f2f0 0x7fb228151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.029+0000 7fb22d6b6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2280adf10 0x7fb2280c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.029+0000 7fb22d6b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb22815e2f0 0x7fb2280c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.029+0000 7fb22d6b6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb228152300 con 0x7fb22806f2f0 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.029+0000 7fb22d6b6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280c3260 con 0x7fb2280adf10 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.029+0000 7fb22d6b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280c23b0 con 0x7fb22815e2f0 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.029+0000 7fb226ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22806f2f0 0x7fb228151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.029+0000 7fb226ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22806f2f0 0x7fb228151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32770/0 (socket says 172.21.15.115:32770) 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.029+0000 7fb226ffd640 1 -- 172.21.15.115:0/1339325632 learned_addr learned my addr 172.21.15.115:0/1339325632 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb226ffd640 1 -- 172.21.15.115:0/1339325632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb22815e2f0 msgr2=0x7fb2280c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb226ffd640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb22815e2f0 0x7fb2280c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb226ffd640 1 -- 172.21.15.115:0/1339325632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2280adf10 msgr2=0x7fb2280c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb21ffff640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2280adf10 0x7fb2280c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb226ffd640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2280adf10 0x7fb2280c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb226ffd640 1 -- 172.21.15.115:0/1339325632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180094d0 con 0x7fb22806f2f0 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb226ffd640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22806f2f0 0x7fb228151dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb21802b8c0 tx=0x7fb2180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:33.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb21f7fe640 1 -- 172.21.15.115:0/1339325632 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2180329c0 con 0x7fb22806f2f0 2023-12-01T01:36:33.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb21f7fe640 1 -- 172.21.15.115:0/1339325632 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218032b40 con 0x7fb22806f2f0 2023-12-01T01:36:33.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1339325632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2280c2650 con 0x7fb22806f2f0 2023-12-01T01:36:33.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb21f7fe640 1 -- 172.21.15.115:0/1339325632 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2180453f0 con 0x7fb22806f2f0 2023-12-01T01:36:33.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.030+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1339325632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2280c2c10 con 0x7fb22806f2f0 2023-12-01T01:36:33.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.031+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1339325632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22806fda0 con 0x7fb22806f2f0 2023-12-01T01:36:33.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.032+0000 7fb21f7fe640 1 -- 172.21.15.115:0/1339325632 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb21804d3f0 con 0x7fb22806f2f0 2023-12-01T01:36:33.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.033+0000 7fb21f7fe640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb20806a430 0x7fb20806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.033+0000 7fb21f7fe640 1 -- 172.21.15.115:0/1339325632 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2180aa820 con 0x7fb22806f2f0 2023-12-01T01:36:33.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.033+0000 7fb21ffff640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb20806a430 0x7fb20806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:33.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.034+0000 7fb21ffff640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb20806a430 0x7fb20806c8f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb2281f5670 tx=0x7fb210008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:33.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.037+0000 7fb21f7fe640 1 -- 172.21.15.115:0/1339325632 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb218075ec0 con 0x7fb22806f2f0 2023-12-01T01:36:33.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.215+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1339325632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb22804e890 con 0x7fb22806f2f0 2023-12-01T01:36:33.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.218+0000 7fb21f7fe640 1 -- 172.21.15.115:0/1339325632 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16089 (secure 0 0 0) 0x7fb2180aa070 con 0x7fb22806f2f0 2023-12-01T01:36:33.219 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:33.219 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:11.641476+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:33.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.221+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1339325632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb20806a430 msgr2=0x7fb20806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.221+0000 7fb22d6b6640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb20806a430 0x7fb20806c8f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb2281f5670 tx=0x7fb210008040 comp rx=0 tx=0).stop 2023-12-01T01:36:33.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.221+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1339325632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22806f2f0 msgr2=0x7fb228151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.221+0000 7fb22d6b6640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22806f2f0 0x7fb228151dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb21802b8c0 tx=0x7fb2180046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.221+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1339325632 shutdown_connections 2023-12-01T01:36:33.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.221+0000 7fb22d6b6640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb20806a430 0x7fb20806c8f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.222+0000 7fb22d6b6640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb22815e2f0 0x7fb2280c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.222+0000 7fb22d6b6640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2280adf10 0x7fb2280c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.222+0000 7fb22d6b6640 1 --2- 172.21.15.115:0/1339325632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb22806f2f0 0x7fb228151dc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.222+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1339325632 >> 172.21.15.115:0/1339325632 conn(0x7fb2280ce880 msgr2=0x7fb2280ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:33.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.222+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1339325632 shutdown_connections 2023-12-01T01:36:33.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.222+0000 7fb22d6b6640 1 -- 172.21.15.115:0/1339325632 wait complete. 2023-12-01T01:36:33.241 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:36:33.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.450+0000 7f31199d0640 1 Processor -- start 2023-12-01T01:36:33.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.451+0000 7f31199d0640 1 -- start start 2023-12-01T01:36:33.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.452+0000 7f31199d0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f311415d7b0 0x7f311415db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.452+0000 7f31199d0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31140d05d0 0x7f31140d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.452+0000 7f31199d0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31140cd3f0 con 0x7f3114160670 2023-12-01T01:36:33.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.452+0000 7f31199d0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31140cd0f0 con 0x7f311415d7b0 2023-12-01T01:36:33.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.452+0000 7f31199d0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3114160ec0 con 0x7f31140d05d0 2023-12-01T01:36:33.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.453+0000 7f31191cf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31140d05d0 0x7f31140d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:33.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.453+0000 7f31191cf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31140d05d0 0x7f31140d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:58972/0 (socket says 172.21.15.115:58972) 2023-12-01T01:36:33.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.453+0000 7f31191cf640 1 -- 172.21.15.115:0/3327749463 learned_addr learned my addr 172.21.15.115:0/3327749463 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:33.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.453+0000 7f31189ce640 1 --2- 172.21.15.115:0/3327749463 >> v2:172.21.15.142:3300/0 conn(0x7f311415d7b0 0x7f311415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:33.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.454+0000 7f31191cf640 1 -- 172.21.15.115:0/3327749463 >> v2:172.21.15.142:3300/0 conn(0x7f311415d7b0 msgr2=0x7f311415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:33.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.454+0000 7f31191cf640 1 --2- 172.21.15.115:0/3327749463 >> v2:172.21.15.142:3300/0 conn(0x7f311415d7b0 0x7f311415db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.454+0000 7f31191cf640 1 -- 172.21.15.115:0/3327749463 >> v1:172.21.15.142:6789/0 conn(0x7f3114160670 legacy=0x7f31140cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.454+0000 7f31191cf640 1 -- 172.21.15.115:0/3327749463 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31141611a0 con 0x7f31140d05d0 2023-12-01T01:36:33.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.455+0000 7f31191cf640 1 --2- 172.21.15.115:0/3327749463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31140d05d0 0x7f31140d0a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f310c009870 tx=0x7f310c02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=565d062b110d289d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:33.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.455+0000 7f31137fe640 1 -- 172.21.15.115:0/3327749463 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c02f840 con 0x7f31140d05d0 2023-12-01T01:36:33.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.455+0000 7f31137fe640 1 -- 172.21.15.115:0/3327749463 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c02f9a0 con 0x7f31140d05d0 2023-12-01T01:36:33.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.455+0000 7f31137fe640 1 -- 172.21.15.115:0/3327749463 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c02fd30 con 0x7f31140d05d0 2023-12-01T01:36:33.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.456+0000 7f31199d0640 1 -- 172.21.15.115:0/3327749463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31140d05d0 msgr2=0x7f31140d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.456+0000 7f31199d0640 1 --2- 172.21.15.115:0/3327749463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31140d05d0 0x7f31140d0a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f310c009870 tx=0x7f310c02ba20 comp rx=0 tx=0).stop 2023-12-01T01:36:33.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.456+0000 7f31199d0640 1 -- 172.21.15.115:0/3327749463 shutdown_connections 2023-12-01T01:36:33.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.456+0000 7f31199d0640 1 --2- 172.21.15.115:0/3327749463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31140d05d0 0x7f31140d0a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.456+0000 7f31199d0640 1 --2- 172.21.15.115:0/3327749463 >> v2:172.21.15.142:3300/0 conn(0x7f311415d7b0 0x7f311415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.456+0000 7f31199d0640 1 -- 172.21.15.115:0/3327749463 >> 172.21.15.115:0/3327749463 conn(0x7f311406a0e0 msgr2=0x7f311406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:33.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.456+0000 7f31199d0640 1 -- 172.21.15.115:0/3327749463 shutdown_connections 2023-12-01T01:36:33.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.456+0000 7f31199d0640 1 -- 172.21.15.115:0/3327749463 wait complete. 2023-12-01T01:36:33.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.458+0000 7f31199d0640 1 Processor -- start 2023-12-01T01:36:33.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.458+0000 7f31199d0640 1 -- start start 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f31199d0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31140d05d0 0x7f311414e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f31199d0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f311415d7b0 0x7f31140b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f31199d0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3114160670 0x7f311415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f31199d0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31140b4810 con 0x7f3114160670 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f31199d0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31140c9d70 con 0x7f31140d05d0 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f31199d0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31140b3c00 con 0x7f311415d7b0 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f31191cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3114160670 0x7f311415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f31191cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3114160670 0x7f311415ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32778/0 (socket says 172.21.15.115:32778) 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f31191cf640 1 -- 172.21.15.115:0/1446830713 learned_addr learned my addr 172.21.15.115:0/1446830713 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f31189ce640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31140d05d0 0x7f311414e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:33.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.459+0000 7f3113fff640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f311415d7b0 0x7f31140b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:33.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.460+0000 7f31191cf640 1 -- 172.21.15.115:0/1446830713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f311415d7b0 msgr2=0x7f31140b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.460+0000 7f31191cf640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f311415d7b0 0x7f31140b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.460+0000 7f31191cf640 1 -- 172.21.15.115:0/1446830713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31140d05d0 msgr2=0x7f311414e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.460+0000 7f31191cf640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31140d05d0 0x7f311414e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.460+0000 7f31191cf640 1 -- 172.21.15.115:0/1446830713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c0094d0 con 0x7f3114160670 2023-12-01T01:36:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.460+0000 7f31191cf640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3114160670 0x7f311415ed10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f310c009870 tx=0x7f310c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.460+0000 7f31117fa640 1 -- 172.21.15.115:0/1446830713 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c037c40 con 0x7f3114160670 2023-12-01T01:36:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.460+0000 7f31117fa640 1 -- 172.21.15.115:0/1446830713 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c037dc0 con 0x7f3114160670 2023-12-01T01:36:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.460+0000 7f31199d0640 1 -- 172.21.15.115:0/1446830713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31140b3ee0 con 0x7f3114160670 2023-12-01T01:36:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.460+0000 7f31117fa640 1 -- 172.21.15.115:0/1446830713 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c041640 con 0x7f3114160670 2023-12-01T01:36:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.461+0000 7f31199d0640 1 -- 172.21.15.115:0/1446830713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31140b44a0 con 0x7f3114160670 2023-12-01T01:36:33.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.462+0000 7f31199d0640 1 -- 172.21.15.115:0/1446830713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31140673a0 con 0x7f3114160670 2023-12-01T01:36:33.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.462+0000 7f31117fa640 1 -- 172.21.15.115:0/1446830713 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f310c0417a0 con 0x7f3114160670 2023-12-01T01:36:33.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.463+0000 7f31117fa640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30f406a460 0x7f30f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:33.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.463+0000 7f31117fa640 1 -- 172.21.15.115:0/1446830713 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f310c03c070 con 0x7f3114160670 2023-12-01T01:36:33.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.463+0000 7f31189ce640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30f406a460 0x7f30f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:33.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.464+0000 7f31189ce640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30f406a460 0x7f30f406c920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31040074a0 tx=0x7f310400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:33.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.467+0000 7f31117fa640 1 -- 172.21.15.115:0/1446830713 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f310c076bc0 con 0x7f3114160670 2023-12-01T01:36:33.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.695+0000 7f31199d0640 1 -- 172.21.15.115:0/1446830713 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31140d3100 con 0x7f3114160670 2023-12-01T01:36:33.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.696+0000 7f31117fa640 1 -- 172.21.15.115:0/1446830713 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4654 (secure 0 0 0) 0x7f310c07a1e0 con 0x7f3114160670 2023-12-01T01:36:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:33.696 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3656795862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3656795862},{"type":"v1","addr":"172.21.15.142:6834","nonce":3656795862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1125002206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1125002206},{"type":"v1","addr":"172.21.15.115:6836","nonce":1125002206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3350782192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3350782192},{"type":"v1","addr":"172.21.15.115:6837","nonce":3350782192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:36:10.637450+0000","modified":"2023-12-01T01:36:13.663004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2808777047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2808777047},{"type":"v1","addr":"172.21.15.142:6835","nonce":2808777047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:36:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 8 2023-12-01T01:36:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.698+0000 7f31199d0640 1 -- 172.21.15.115:0/1446830713 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30f406a460 msgr2=0x7f30f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.698+0000 7f31199d0640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30f406a460 0x7f30f406c920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31040074a0 tx=0x7f310400b040 comp rx=0 tx=0).stop 2023-12-01T01:36:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.698+0000 7f31199d0640 1 -- 172.21.15.115:0/1446830713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3114160670 msgr2=0x7f311415ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.698+0000 7f31199d0640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3114160670 0x7f311415ed10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f310c009870 tx=0x7f310c004730 comp rx=0 tx=0).stop 2023-12-01T01:36:33.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.699+0000 7f31199d0640 1 -- 172.21.15.115:0/1446830713 shutdown_connections 2023-12-01T01:36:33.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.699+0000 7f31199d0640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30f406a460 0x7f30f406c920 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.699+0000 7f31199d0640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f311415d7b0 0x7f31140b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.699+0000 7f31199d0640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3114160670 0x7f311415ed10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.699+0000 7f31199d0640 1 --2- 172.21.15.115:0/1446830713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31140d05d0 0x7f311414e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:33.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.699+0000 7f31199d0640 1 -- 172.21.15.115:0/1446830713 >> 172.21.15.115:0/1446830713 conn(0x7f311406a0e0 msgr2=0x7f311406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:33.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.699+0000 7f31199d0640 1 -- 172.21.15.115:0/1446830713 shutdown_connections 2023-12-01T01:36:33.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:33.699+0000 7f31199d0640 1 -- 172.21.15.115:0/1446830713 wait complete. 2023-12-01T01:36:34.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T01:36:10.637450+0000', 'modified': '2023-12-01T01:36:13.663004+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6835/2808777047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6833', 'nonce': 2808777047}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2808777047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T01:36:34.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T01:36:34.121 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-01T01:36:34.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.345+0000 7f689d60e640 1 Processor -- start 2023-12-01T01:36:34.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.346+0000 7f689d60e640 1 -- start start 2023-12-01T01:36:34.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.347+0000 7f689d60e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6898161300 0x7f689806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.347+0000 7f689d60e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f68980ca060 0x7f689806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.347+0000 7f689d60e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68980ca560 con 0x7f68980c1a90 2023-12-01T01:36:34.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.347+0000 7f689d60e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68980ca840 con 0x7f68980ca060 2023-12-01T01:36:34.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.347+0000 7f689d60e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980cab20 con 0x7f6898161300 2023-12-01T01:36:34.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.348+0000 7f6897fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6898161300 0x7f689806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.348+0000 7f6897fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6898161300 0x7f689806f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:59000/0 (socket says 172.21.15.115:59000) 2023-12-01T01:36:34.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.348+0000 7f6897fff640 1 -- 172.21.15.115:0/2328769969 learned_addr learned my addr 172.21.15.115:0/2328769969 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:34.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.348+0000 7f688ffff640 1 --2- 172.21.15.115:0/2328769969 >> v2:172.21.15.142:3300/0 conn(0x7f68980ca060 0x7f689806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.349+0000 7f6897fff640 1 -- 172.21.15.115:0/2328769969 >> v2:172.21.15.142:3300/0 conn(0x7f68980ca060 msgr2=0x7f689806feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:34.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.349+0000 7f6897fff640 1 --2- 172.21.15.115:0/2328769969 >> v2:172.21.15.142:3300/0 conn(0x7f68980ca060 0x7f689806feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.349+0000 7f68977fe640 1 -- 172.21.15.115:0/2328769969 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752106325 0 0) 0x7f68980ca560 con 0x7f68980c1a90 2023-12-01T01:36:34.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.349+0000 7f6897fff640 1 -- 172.21.15.115:0/2328769969 >> v1:172.21.15.142:6789/0 conn(0x7f68980c1a90 legacy=0x7f6898164f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:34.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.349+0000 7f6897fff640 1 -- 172.21.15.115:0/2328769969 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689806b2c0 con 0x7f6898161300 2023-12-01T01:36:34.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.350+0000 7f6897fff640 1 --2- 172.21.15.115:0/2328769969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6898161300 0x7f689806f970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6888009870 tx=0x7f688802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3228fcc97937b7b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:34.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.351+0000 7f68977fe640 1 -- 172.21.15.115:0/2328769969 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888030840 con 0x7f6898161300 2023-12-01T01:36:34.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.351+0000 7f68977fe640 1 -- 172.21.15.115:0/2328769969 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68880309a0 con 0x7f6898161300 2023-12-01T01:36:34.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.351+0000 7f68977fe640 1 -- 172.21.15.115:0/2328769969 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888030d30 con 0x7f6898161300 2023-12-01T01:36:34.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.352+0000 7f689d60e640 1 -- 172.21.15.115:0/2328769969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6898161300 msgr2=0x7f689806f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:34.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.352+0000 7f689d60e640 1 --2- 172.21.15.115:0/2328769969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6898161300 0x7f689806f970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6888009870 tx=0x7f688802b980 comp rx=0 tx=0).stop 2023-12-01T01:36:34.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.352+0000 7f689d60e640 1 -- 172.21.15.115:0/2328769969 shutdown_connections 2023-12-01T01:36:34.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.352+0000 7f689d60e640 1 --2- 172.21.15.115:0/2328769969 >> v2:172.21.15.142:3300/0 conn(0x7f68980ca060 0x7f689806feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.352+0000 7f689d60e640 1 --2- 172.21.15.115:0/2328769969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6898161300 0x7f689806f970 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.352+0000 7f689d60e640 1 -- 172.21.15.115:0/2328769969 >> 172.21.15.115:0/2328769969 conn(0x7f68980ce620 msgr2=0x7f68980cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:34.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.352+0000 7f689d60e640 1 -- 172.21.15.115:0/2328769969 shutdown_connections 2023-12-01T01:36:34.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.352+0000 7f689d60e640 1 -- 172.21.15.115:0/2328769969 wait complete. 2023-12-01T01:36:34.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.354+0000 7f689d60e640 1 Processor -- start 2023-12-01T01:36:34.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.354+0000 7f689d60e640 1 -- start start 2023-12-01T01:36:34.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f689d60e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980c1a90 0x7f689815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f689d60e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68980ca060 0x7f68980c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f689d60e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898161300 0x7f68980c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f689d60e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980c0050 con 0x7f68980ca060 2023-12-01T01:36:34.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f689d60e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980c4cb0 con 0x7f6898161300 2023-12-01T01:36:34.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f689d60e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980bd380 con 0x7f68980c1a90 2023-12-01T01:36:34.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f688ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68980ca060 0x7f68980c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f688ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68980ca060 0x7f68980c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32788/0 (socket says 172.21.15.115:32788) 2023-12-01T01:36:34.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f688ffff640 1 -- 172.21.15.115:0/881075419 learned_addr learned my addr 172.21.15.115:0/881075419 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:34.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f6897fff640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980c1a90 0x7f689815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f689ce0d640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898161300 0x7f68980c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f688ffff640 1 -- 172.21.15.115:0/881075419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980c1a90 msgr2=0x7f689815f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:34.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f688ffff640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980c1a90 0x7f689815f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f688ffff640 1 -- 172.21.15.115:0/881075419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898161300 msgr2=0x7f68980c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:34.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f688ffff640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898161300 0x7f68980c3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f688ffff640 1 -- 172.21.15.115:0/881075419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68880094d0 con 0x7f68980ca060 2023-12-01T01:36:34.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.355+0000 7f6897fff640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980c1a90 0x7f689815f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:34.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.356+0000 7f688ffff640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68980ca060 0x7f68980c38a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f688000e9b0 tx=0x7f688000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:34.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.356+0000 7f68957fa640 1 -- 172.21.15.115:0/881075419 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880009670 con 0x7f68980ca060 2023-12-01T01:36:34.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.356+0000 7f68957fa640 1 -- 172.21.15.115:0/881075419 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688000c9a0 con 0x7f68980ca060 2023-12-01T01:36:34.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.356+0000 7f689d60e640 1 -- 172.21.15.115:0/881075419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68980c4410 con 0x7f68980ca060 2023-12-01T01:36:34.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.356+0000 7f68957fa640 1 -- 172.21.15.115:0/881075419 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688000cd50 con 0x7f68980ca060 2023-12-01T01:36:34.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.356+0000 7f689d60e640 1 -- 172.21.15.115:0/881075419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68980c49d0 con 0x7f68980ca060 2023-12-01T01:36:34.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.358+0000 7f68957fa640 1 -- 172.21.15.115:0/881075419 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6880013470 con 0x7f68980ca060 2023-12-01T01:36:34.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.359+0000 7f68957fa640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f687806a430 0x7f687806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.359+0000 7f68957fa640 1 -- 172.21.15.115:0/881075419 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6880088bb0 con 0x7f68980ca060 2023-12-01T01:36:34.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.360+0000 7f6897fff640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f687806a430 0x7f687806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.360+0000 7f68a13e8740 1 Processor -- start 2023-12-01T01:36:34.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.360+0000 7f68a13e8740 1 -- start start 2023-12-01T01:36:34.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.360+0000 7f6897fff640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f687806a430 0x7f687806c8f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6888036000 tx=0x7f688803b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:34.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.361+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x555602245ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.361+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602215a80 0x555602242d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.361+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x5556022432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.361+0000 7f68a13e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556021f12f0 con 0x555602245b00 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.361+0000 7f68a13e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555602185f80 con 0x555602215a80 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.361+0000 7f68a13e8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x555602183710 con 0x555602219680 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.361+0000 7f6897fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x555602245ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.361+0000 7f6897fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x555602245ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32798/0 (socket says 172.21.15.115:32798) 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.361+0000 7f6897fff640 1 -- 172.21.15.115:0/3818648880 learned_addr learned my addr 172.21.15.115:0/3818648880 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.362+0000 7f6897fff640 1 -- 172.21.15.115:0/3818648880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 msgr2=0x5556022432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.362+0000 7f6897fff640 1 --2- 172.21.15.115:0/3818648880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x5556022432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.362+0000 7f6897fff640 1 -- 172.21.15.115:0/3818648880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602215a80 msgr2=0x555602242d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.362+0000 7f6897fff640 1 --2- 172.21.15.115:0/3818648880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602215a80 0x555602242d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.362+0000 7f6897fff640 1 -- 172.21.15.115:0/3818648880 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555602146690 con 0x555602245b00 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.362+0000 7f6897fff640 1 --2- 172.21.15.115:0/3818648880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x555602245ee0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f688802bf90 tx=0x7f6888030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf2a496285d06c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.362+0000 7f688dffb640 1 -- 172.21.15.115:0/3818648880 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888044d40 con 0x555602245b00 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.362+0000 7f688dffb640 1 -- 172.21.15.115:0/3818648880 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688804a040 con 0x555602245b00 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.362+0000 7f688dffb640 1 -- 172.21.15.115:0/3818648880 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888044ea0 con 0x555602245b00 2023-12-01T01:36:34.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.363+0000 7f68a13e8740 1 -- 172.21.15.115:0/3818648880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 msgr2=0x555602245ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.363+0000 7f68a13e8740 1 --2- 172.21.15.115:0/3818648880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x555602245ee0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f688802bf90 tx=0x7f6888030000 comp rx=0 tx=0).stop 2023-12-01T01:36:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.363+0000 7f68a13e8740 1 -- 172.21.15.115:0/3818648880 shutdown_connections 2023-12-01T01:36:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.363+0000 7f68a13e8740 1 --2- 172.21.15.115:0/3818648880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x5556022432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.363+0000 7f68a13e8740 1 --2- 172.21.15.115:0/3818648880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602215a80 0x555602242d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.363+0000 7f68a13e8740 1 --2- 172.21.15.115:0/3818648880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x555602245ee0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.363+0000 7f68a13e8740 1 -- 172.21.15.115:0/3818648880 >> 172.21.15.115:0/3818648880 conn(0x555602217d90 msgr2=0x555602260910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.363+0000 7f68a13e8740 1 -- 172.21.15.115:0/3818648880 shutdown_connections 2023-12-01T01:36:34.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.363+0000 7f68a13e8740 1 -- 172.21.15.115:0/3818648880 wait complete. 2023-12-01T01:36:34.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.364+0000 7f68a13e8740 1 -- start start 2023-12-01T01:36:34.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.364+0000 7f68a13e8740 1 Processor -- start 2023-12-01T01:36:34.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x555602240470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602245b00 0x555602249930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560222c640 0x555602248450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f68a13e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560217f000 con 0x55560222c640 2023-12-01T01:36:34.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f68a13e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556021207d0 con 0x555602245b00 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f68a13e8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x555602120470 con 0x555602219680 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f68a13e8740 10 client.0 _ll_register_callbacks cb 0x55560216f1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f689ce0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560222c640 0x555602248450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f689ce0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560222c640 0x555602248450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32810/0 (socket says 172.21.15.115:32810) 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f689ce0d640 1 -- 172.21.15.115:0/4269246761 learned_addr learned my addr 172.21.15.115:0/4269246761 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f6897fff640 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x555602240470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.365+0000 7f688ffff640 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602245b00 0x555602249930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f689ce0d640 1 -- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 msgr2=0x555602240470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f689ce0d640 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x555602240470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f689ce0d640 1 -- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602245b00 msgr2=0x555602249930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f689ce0d640 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602245b00 0x555602249930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f689ce0d640 1 -- 172.21.15.115:0/4269246761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689806b2c0 con 0x55560222c640 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f688ffff640 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602245b00 0x555602249930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f689ce0d640 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560222c640 0x555602248450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f689000d280 tx=0x7f689000d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64da1048bd620eb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:34.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f6876ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f6876ffd640 1 -- 172.21.15.115:0/4269246761 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689000b070 con 0x55560222c640 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f6876ffd640 1 -- 172.21.15.115:0/4269246761 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6890003d80 con 0x55560222c640 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f68a13e8740 1 -- 172.21.15.115:0/4269246761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556021a9550 con 0x55560222c640 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f6876ffd640 1 -- 172.21.15.115:0/4269246761 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6890001650 con 0x55560222c640 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f68a13e8740 10 client.4695 fetch_fsmap learned FSMap version 8 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.366+0000 7f68a13e8740 1 -- 172.21.15.115:0/4269246761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55560217d680 con 0x55560222c640 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.367+0000 7f6876ffd640 1 -- 172.21.15.115:0/4269246761 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5556021a9550 con 0x55560222c640 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.367+0000 7f6876ffd640 1 -- 172.21.15.115:0/4269246761 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f68900050d0 con 0x55560222c640 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.367+0000 7f68a13e8740 10 client.4695 fetch_fsmap finished waiting for FSMap version 8 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.367+0000 7f68a13e8740 10 client.4695 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.367+0000 7f68a13e8740 20 client.4695 populate_metadata read hostname 'smithi115' 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.367+0000 7f68a13e8740 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:6833/2808777047,v1:172.21.15.142:6835/2808777047] conn(0x555602273700 0x555602273ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.367+0000 7f68a13e8740 4 client.4695 mds_command: new command op to 4294 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:36:34.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.367+0000 7f68a13e8740 1 -- 172.21.15.115:0/4269246761 --> [v2:172.21.15.142:6833/2808777047,v1:172.21.15.142:6835/2808777047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555602171b50 con 0x555602273700 2023-12-01T01:36:34.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.367+0000 7f6897fff640 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:6833/2808777047,v1:172.21.15.142:6835/2808777047] conn(0x555602273700 0x555602273ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.368+0000 7f6876ffd640 10 client.4695 ms_handle_connect on v2:172.21.15.142:6833/2808777047 2023-12-01T01:36:34.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.368+0000 7f6897fff640 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:6833/2808777047,v1:172.21.15.142:6835/2808777047] conn(0x555602273700 0x555602273ae0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a2a7307a9236957 server_cookie=2cfb48f516f92742 in_seq=0 out_seq=0 2023-12-01T01:36:34.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f6876ffd640 1 -- 172.21.15.115:0/4269246761 <== mds.0 v2:172.21.15.142:6833/2808777047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555602171b50 con 0x555602273700 2023-12-01T01:36:34.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f6876ffd640 10 client.4695 handle_command_reply: tid=0 2023-12-01T01:36:34.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f68a13e8740 1 client.4695 shutdown 2023-12-01T01:36:34.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f68a13e8740 1 -- 172.21.15.115:0/4269246761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560222c640 msgr2=0x555602248450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f68a13e8740 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560222c640 0x555602248450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f689000d280 tx=0x7f689000d740 comp rx=0 tx=0).stop 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f68a13e8740 1 -- 172.21.15.115:0/4269246761 shutdown_connections 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f68a13e8740 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:6833/2808777047,v1:172.21.15.142:6835/2808777047] conn(0x555602273700 0x555602273ae0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f68a13e8740 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55560222c640 0x555602248450 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f68a13e8740 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602245b00 0x555602249930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f68a13e8740 1 --2- 172.21.15.115:0/4269246761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x555602240470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f68a13e8740 1 -- 172.21.15.115:0/4269246761 >> 172.21.15.115:0/4269246761 conn(0x555602215a80 msgr2=0x555602263e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f6876ffd640 0 client.4695 ms_handle_reset on v2:172.21.15.142:6833/2808777047 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.370+0000 7f68a13e8740 1 -- 172.21.15.115:0/4269246761 shutdown_connections 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.371+0000 7f68a13e8740 1 -- 172.21.15.115:0/4269246761 wait complete. 2023-12-01T01:36:34.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.371+0000 7f68a13e8740 20 client.4695 trim_cache size 0 max 16384 2023-12-01T01:36:34.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.387+0000 7f68a13e8740 1 Processor -- start 2023-12-01T01:36:34.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.387+0000 7f68a13e8740 1 -- start start 2023-12-01T01:36:34.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602219680 0x555602219b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55560222c640 0x55560223d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x55560223df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f68a13e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555601ee6f20 con 0x555602245b00 2023-12-01T01:36:34.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f68a13e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556022619b0 con 0x555602219680 2023-12-01T01:36:34.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f68a13e8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55560223e5f0 con 0x55560222c640 2023-12-01T01:36:34.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f689ce0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x55560223df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f689ce0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x55560223df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32820/0 (socket says 172.21.15.115:32820) 2023-12-01T01:36:34.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f689ce0d640 1 -- 172.21.15.115:0/2905729482 learned_addr learned my addr 172.21.15.115:0/2905729482 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:34.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f689ce0d640 1 -- 172.21.15.115:0/2905729482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55560222c640 msgr2=0x55560223d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:34.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f689ce0d640 1 --2- 172.21.15.115:0/2905729482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55560222c640 0x55560223d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f689ce0d640 1 -- 172.21.15.115:0/2905729482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602219680 msgr2=0x555602219b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:34.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f689ce0d640 1 --2- 172.21.15.115:0/2905729482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602219680 0x555602219b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.388+0000 7f689ce0d640 1 -- 172.21.15.115:0/2905729482 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55560211f9e0 con 0x555602245b00 2023-12-01T01:36:34.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f689ce0d640 1 --2- 172.21.15.115:0/2905729482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x55560223df30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6890001f20 tx=0x7f689000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b435f85db5d998a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:34.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f688e7fc640 1 -- 172.21.15.115:0/2905729482 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890005820 con 0x555602245b00 2023-12-01T01:36:34.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f688e7fc640 1 -- 172.21.15.115:0/2905729482 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689000dc40 con 0x555602245b00 2023-12-01T01:36:34.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f688e7fc640 1 -- 172.21.15.115:0/2905729482 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890003dc0 con 0x555602245b00 2023-12-01T01:36:34.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f68a13e8740 1 -- 172.21.15.115:0/2905729482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 msgr2=0x55560223df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:34.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f68a13e8740 1 --2- 172.21.15.115:0/2905729482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x55560223df30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6890001f20 tx=0x7f689000b000 comp rx=0 tx=0).stop 2023-12-01T01:36:34.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f68a13e8740 1 -- 172.21.15.115:0/2905729482 shutdown_connections 2023-12-01T01:36:34.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f68a13e8740 1 --2- 172.21.15.115:0/2905729482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55560222c640 0x55560223d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f68a13e8740 1 --2- 172.21.15.115:0/2905729482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x55560223df30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f68a13e8740 1 --2- 172.21.15.115:0/2905729482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555602219680 0x555602219b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.389+0000 7f68a13e8740 1 -- 172.21.15.115:0/2905729482 >> 172.21.15.115:0/2905729482 conn(0x555602215a80 msgr2=0x555602215f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:34.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.390+0000 7f68a13e8740 1 -- 172.21.15.115:0/2905729482 shutdown_connections 2023-12-01T01:36:34.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.390+0000 7f68a13e8740 1 -- 172.21.15.115:0/2905729482 wait complete. 2023-12-01T01:36:34.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.390+0000 7f68a13e8740 1 -- start start 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.390+0000 7f68a13e8740 1 Processor -- start 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.391+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x55560223d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.391+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55560222c640 0x55560223df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.391+0000 7f68a13e8740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x5556022c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.391+0000 7f68a13e8740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556022c67d0 con 0x555602245b00 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.391+0000 7f68a13e8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560223b250 con 0x55560222c640 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.391+0000 7f68a13e8740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x555602219cf0 con 0x555602219680 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.391+0000 7f68a13e8740 10 client.0 _ll_register_callbacks cb 0x7f6880009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f689ce0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x5556022c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f689ce0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x5556022c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32822/0 (socket says 172.21.15.115:32822) 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f689ce0d640 1 -- 172.21.15.115:0/314731894 learned_addr learned my addr 172.21.15.115:0/314731894 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f6897fff640 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x55560223d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f689ce0d640 1 -- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 msgr2=0x55560223d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f689ce0d640 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x55560223d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f689ce0d640 1 -- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55560222c640 msgr2=0x55560223df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f688ffff640 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55560222c640 0x55560223df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f689ce0d640 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55560222c640 0x55560223df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f689ce0d640 1 -- 172.21.15.115:0/314731894 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55560217d680 con 0x555602245b00 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f689ce0d640 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x5556022c9b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6890001bf0 tx=0x7f6890007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307778b7254f3ee9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.392+0000 7f6876ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f6876ffd640 1 -- 172.21.15.115:0/314731894 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890005820 con 0x555602245b00 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f6876ffd640 1 -- 172.21.15.115:0/314731894 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6890036af0 con 0x555602245b00 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f68a13e8740 1 -- 172.21.15.115:0/314731894 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f688804a040 con 0x555602245b00 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f6876ffd640 1 -- 172.21.15.115:0/314731894 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6890036cb0 con 0x555602245b00 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f68a13e8740 10 client.4701 fetch_fsmap learned FSMap version 8 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f68a13e8740 1 -- 172.21.15.115:0/314731894 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x555602144ef0 con 0x555602245b00 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f6876ffd640 1 -- 172.21.15.115:0/314731894 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688804a040 con 0x555602245b00 2023-12-01T01:36:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f6876ffd640 1 -- 172.21.15.115:0/314731894 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6890036020 con 0x555602245b00 2023-12-01T01:36:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f68a13e8740 10 client.4701 fetch_fsmap finished waiting for FSMap version 8 2023-12-01T01:36:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f68a13e8740 10 client.4701 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-01T01:36:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f68a13e8740 20 client.4701 populate_metadata read hostname 'smithi115' 2023-12-01T01:36:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f68a13e8740 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:6833/2808777047,v1:172.21.15.142:6835/2808777047] conn(0x5556022c0440 0x5556022ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f68a13e8740 4 client.4701 mds_command: new command op to 4294 tid=0[{"prefix": "status"}] 2023-12-01T01:36:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.393+0000 7f68a13e8740 1 -- 172.21.15.115:0/314731894 --> [v2:172.21.15.142:6833/2808777047,v1:172.21.15.142:6835/2808777047] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555602178080 con 0x5556022c0440 2023-12-01T01:36:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.394+0000 7f6897fff640 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:6833/2808777047,v1:172.21.15.142:6835/2808777047] conn(0x5556022c0440 0x5556022ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f6897fff640 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:6833/2808777047,v1:172.21.15.142:6835/2808777047] conn(0x5556022c0440 0x5556022ca0c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=341f1102f6b02ca1 server_cookie=ffdd23371ef56d22 in_seq=0 out_seq=0 2023-12-01T01:36:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f6876ffd640 10 client.4701 ms_handle_connect on v2:172.21.15.142:6833/2808777047 2023-12-01T01:36:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f6876ffd640 1 -- 172.21.15.115:0/314731894 <== mds.0 v2:172.21.15.142:6833/2808777047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x555602178080 con 0x5556022c0440 2023-12-01T01:36:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f6876ffd640 10 client.4701 handle_command_reply: tid=0 2023-12-01T01:36:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 client.4701 shutdown 2023-12-01T01:36:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 -- 172.21.15.115:0/314731894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 msgr2=0x5556022c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x5556022c9b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6890001bf0 tx=0x7f6890007b60 comp rx=0 tx=0).stop 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 -- 172.21.15.115:0/314731894 shutdown_connections 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:6833/2808777047,v1:172.21.15.142:6835/2808777047] conn(0x5556022c0440 0x5556022ca0c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555602245b00 0x5556022c9b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55560222c640 0x55560223df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 --2- 172.21.15.115:0/314731894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555602219680 0x55560223d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 -- 172.21.15.115:0/314731894 >> 172.21.15.115:0/314731894 conn(0x555602215a80 msgr2=0x555602215f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f6876ffd640 0 client.4701 ms_handle_reset on v2:172.21.15.142:6833/2808777047 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 -- 172.21.15.115:0/314731894 shutdown_connections 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.395+0000 7f68a13e8740 1 -- 172.21.15.115:0/314731894 wait complete. 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f68a13e8740 20 client.4701 trim_cache size 0 max 16384 2023-12-01T01:36:34.398 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 4294, 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 23.751135523999999, 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 8, 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 21, 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 21, 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 28.805172243000001 2023-12-01T01:36:34.399 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:36:34.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 -- 172.21.15.115:0/881075419 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f687806a430 msgr2=0x7f687806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:34.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f687806a430 0x7f687806c8f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6888036000 tx=0x7f688803b040 comp rx=0 tx=0).stop 2023-12-01T01:36:34.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 -- 172.21.15.115:0/881075419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68980ca060 msgr2=0x7f68980c38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:34.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68980ca060 0x7f68980c38a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f688000e9b0 tx=0x7f688000ee70 comp rx=0 tx=0).stop 2023-12-01T01:36:34.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 -- 172.21.15.115:0/881075419 shutdown_connections 2023-12-01T01:36:34.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f687806a430 0x7f687806c8f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6898161300 0x7f68980c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68980ca060 0x7f68980c38a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 --2- 172.21.15.115:0/881075419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980c1a90 0x7f689815f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:34.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 -- 172.21.15.115:0/881075419 >> 172.21.15.115:0/881075419 conn(0x7f68980ce620 msgr2=0x7f68981608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:34.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 -- 172.21.15.115:0/881075419 shutdown_connections 2023-12-01T01:36:34.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:34.396+0000 7f689d60e640 1 -- 172.21.15.115:0/881075419 wait complete. 2023-12-01T01:36:34.413 INFO:teuthology.run_tasks:Running task kclient... 2023-12-01T01:36:34.436 INFO:tasks.kclient:Mounting kernel clients... 2023-12-01T01:36:34.437 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-12-01T01:36:34.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:34.437 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:36:34.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:34.492 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link delete ceph-brx 2023-12-01T01:36:34.592 INFO:teuthology.orchestra.run.smithi115.stderr:Cannot find device "ceph-brx" 2023-12-01T01:36:34.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:36:34.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:34.597 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:36:34.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:34.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link delete ceph-brx 2023-12-01T01:36:34.736 INFO:teuthology.orchestra.run.smithi115.stderr:Cannot find device "ceph-brx" 2023-12-01T01:36:34.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:36:34.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:34.739 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2023-12-01T01:36:34.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:34.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2023-12-01T01:36:34.904 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2023-12-01T01:36:34.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:36:34.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:34.909 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2023-12-01T01:36:34.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:34.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2023-12-01T01:36:35.058 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2023-12-01T01:36:35.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:36:35.063 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-12-01T01:36:35.063 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-01T01:36:35.063 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:36:35.063 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:36:35.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.284+0000 7f934c036640 1 Processor -- start 2023-12-01T01:36:35.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.284+0000 7f934c036640 1 -- start start 2023-12-01T01:36:35.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.285+0000 7f934c036640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9344159dd0 0x7f93440c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:35.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.286+0000 7f934c036640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934406f8b0 con 0x7f9344070d30 2023-12-01T01:36:35.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.286+0000 7f934c036640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93440adcb0 con 0x7f934406b6f0 2023-12-01T01:36:35.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.286+0000 7f934c036640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93440cd270 con 0x7f9344159dd0 2023-12-01T01:36:35.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.286+0000 7f934a5d3640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9344070d30 0x7f93440ccb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38106/0 (socket says 172.21.15.115:38106) 2023-12-01T01:36:35.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.286+0000 7f934a5d3640 1 -- 172.21.15.115:0/4281318139 learned_addr learned my addr 172.21.15.115:0/4281318139 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:35.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.286+0000 7f9349dd2640 1 --2- 172.21.15.115:0/4281318139 >> v2:172.21.15.142:3300/0 conn(0x7f9344159dd0 0x7f93440c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:35.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.286+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540045335 0 0) 0x7f934406f8b0 con 0x7f9344070d30 2023-12-01T01:36:35.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.286+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9330003520 con 0x7f9344070d30 2023-12-01T01:36:35.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2579887105 0 0) 0x7f9330003520 con 0x7f9344070d30 2023-12-01T01:36:35.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 >> v2:172.21.15.142:3300/0 conn(0x7f9344159dd0 msgr2=0x7f93440c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 --2- 172.21.15.115:0/4281318139 >> v2:172.21.15.142:3300/0 conn(0x7f9344159dd0 0x7f93440c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 >> v1:172.21.15.142:6789/0 conn(0x7f934406b6f0 legacy=0x7f93440cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440ad9d0 con 0x7f9344070d30 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f933c0019f0 con 0x7f9344070d30 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 >> v1:172.21.15.115:6789/0 conn(0x7f9344070d30 legacy=0x7f93440ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 --2- 172.21.15.115:0/4281318139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9330003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 --2- 172.21.15.115:0/4281318139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93300047c0 0x7f9330004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 --2- 172.21.15.115:0/4281318139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9330005280 0x7f933000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440cd270 con 0x7f93300039e0 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933000e3d0 con 0x7f9330005280 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933000e590 con 0x7f93300047c0 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9349dd2640 1 --2- 172.21.15.115:0/4281318139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9330003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9349dd2640 1 -- 172.21.15.115:0/4281318139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93300047c0 msgr2=0x7f9330004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9349dd2640 1 --2- 172.21.15.115:0/4281318139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93300047c0 0x7f9330004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9349dd2640 1 -- 172.21.15.115:0/4281318139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9330005280 msgr2=0x7f933000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:35.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9349dd2640 1 --2- 172.21.15.115:0/4281318139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9330005280 0x7f933000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f9349dd2640 1 -- 172.21.15.115:0/4281318139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9330003520 con 0x7f93300039e0 2023-12-01T01:36:35.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.287+0000 7f934a5d3640 1 -- 172.21.15.115:0/4281318139 reap_dead start 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.288+0000 7f9349dd2640 1 --2- 172.21.15.115:0/4281318139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9330003e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9340009ac0 tx=0x7f93300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd00b4be7ecf351 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.289+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934002c710 con 0x7f93300039e0 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.289+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934002c870 con 0x7f93300039e0 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.289+0000 7f9348dd0640 1 -- 172.21.15.115:0/4281318139 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934002cc00 con 0x7f93300039e0 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.289+0000 7f934c036640 1 -- 172.21.15.115:0/4281318139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 msgr2=0x7f9330003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.289+0000 7f934c036640 1 --2- 172.21.15.115:0/4281318139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9330003e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9340009ac0 tx=0x7f93300057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.289+0000 7f934c036640 1 -- 172.21.15.115:0/4281318139 shutdown_connections 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.289+0000 7f934c036640 1 --2- 172.21.15.115:0/4281318139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9330003e00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.289+0000 7f934c036640 1 -- 172.21.15.115:0/4281318139 >> 172.21.15.115:0/4281318139 conn(0x7f93440ce880 msgr2=0x7f93440cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.289+0000 7f934c036640 1 -- 172.21.15.115:0/4281318139 shutdown_connections 2023-12-01T01:36:35.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.289+0000 7f934c036640 1 -- 172.21.15.115:0/4281318139 wait complete. 2023-12-01T01:36:35.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.291+0000 7f934c036640 1 Processor -- start 2023-12-01T01:36:35.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.291+0000 7f934c036640 1 -- start start 2023-12-01T01:36:35.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.292+0000 7f934c036640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9344151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:35.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.292+0000 7f934c036640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f934414e170 0x7f934414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:35.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.292+0000 7f934c036640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93441f18d0 0x7f93441f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:35.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.292+0000 7f934c036640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93441518d0 con 0x7f93300039e0 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.292+0000 7f934c036640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9344150940 con 0x7f93441f18d0 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.292+0000 7f934c036640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934415cee0 con 0x7f934414e170 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.292+0000 7f9349dd2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9344151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.292+0000 7f9349dd2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9344151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32842/0 (socket says 172.21.15.115:32842) 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.292+0000 7f9349dd2640 1 -- 172.21.15.115:0/3230792987 learned_addr learned my addr 172.21.15.115:0/3230792987 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f93495d1640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f934414e170 0x7f934414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f9349dd2640 1 -- 172.21.15.115:0/3230792987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f934414e170 msgr2=0x7f934414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f9349dd2640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f934414e170 0x7f934414a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f9349dd2640 1 -- 172.21.15.115:0/3230792987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93441f18d0 msgr2=0x7f93441f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f9349dd2640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93441f18d0 0x7f93441f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f9349dd2640 1 -- 172.21.15.115:0/3230792987 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934415d1c0 con 0x7f93300039e0 2023-12-01T01:36:35.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f9349dd2640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9344151390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9340009ac0 tx=0x7f9340004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:35.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f933a7fc640 1 -- 172.21.15.115:0/3230792987 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340003810 con 0x7f93300039e0 2023-12-01T01:36:35.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f933a7fc640 1 -- 172.21.15.115:0/3230792987 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934002de90 con 0x7f93300039e0 2023-12-01T01:36:35.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f933a7fc640 1 -- 172.21.15.115:0/3230792987 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340003b90 con 0x7f93300039e0 2023-12-01T01:36:35.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.293+0000 7f934c036640 1 -- 172.21.15.115:0/3230792987 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9344150b90 con 0x7f93300039e0 2023-12-01T01:36:35.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.294+0000 7f934c036640 1 -- 172.21.15.115:0/3230792987 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934414a930 con 0x7f93300039e0 2023-12-01T01:36:35.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.295+0000 7f934c036640 1 -- 172.21.15.115:0/3230792987 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934404e890 con 0x7f93300039e0 2023-12-01T01:36:35.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.296+0000 7f933a7fc640 1 -- 172.21.15.115:0/3230792987 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9340003cf0 con 0x7f93300039e0 2023-12-01T01:36:35.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.296+0000 7f933a7fc640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f932806a460 0x7f932806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:35.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.297+0000 7f933a7fc640 1 -- 172.21.15.115:0/3230792987 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93400aa750 con 0x7f93300039e0 2023-12-01T01:36:35.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.297+0000 7f93495d1640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f932806a460 0x7f932806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:35.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.297+0000 7f93495d1640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f932806a460 0x7f932806c920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f93440c4b50 tx=0x7f9334009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:35.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.301+0000 7f933a7fc640 1 -- 172.21.15.115:0/3230792987 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9340075dd0 con 0x7f93300039e0 2023-12-01T01:36:35.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.527+0000 7f934c036640 1 -- 172.21.15.115:0/3230792987 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f93440c8cb0 con 0x7f93300039e0 2023-12-01T01:36:35.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.528+0000 7f933a7fc640 1 -- 172.21.15.115:0/3230792987 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f93400793f0 con 0x7f93300039e0 2023-12-01T01:36:35.529 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:36:35.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 -- 172.21.15.115:0/3230792987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f932806a460 msgr2=0x7f932806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:35.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f932806a460 0x7f932806c920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f93440c4b50 tx=0x7f9334009040 comp rx=0 tx=0).stop 2023-12-01T01:36:35.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 -- 172.21.15.115:0/3230792987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 msgr2=0x7f9344151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:35.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9344151390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9340009ac0 tx=0x7f9340004730 comp rx=0 tx=0).stop 2023-12-01T01:36:35.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 -- 172.21.15.115:0/3230792987 shutdown_connections 2023-12-01T01:36:35.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f932806a460 0x7f932806c920 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f934414e170 0x7f934414a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93441f18d0 0x7f93441f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 --2- 172.21.15.115:0/3230792987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93300039e0 0x7f9344151390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:35.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 -- 172.21.15.115:0/3230792987 >> 172.21.15.115:0/3230792987 conn(0x7f93440ce880 msgr2=0x7f934406c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:35.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 -- 172.21.15.115:0/3230792987 shutdown_connections 2023-12-01T01:36:35.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:35.531+0000 7f934c036640 1 -- 172.21.15.115:0/3230792987 wait complete. 2023-12-01T01:36:35.552 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-01T01:36:35.552 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:36:35.552 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T01:36:35.552 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T01:36:35.552 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:36:35.552 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-01T01:36:35.553 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-01T01:36:35.553 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:36:35.553 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:36:35.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:35.553 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T01:36:35.599 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:36:35.599 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:36:35.599 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:36:35.599 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:35.600 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:36:35.600 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:35.600 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:35.600 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:35.600 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T01:36:35.600 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:35.600 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:35.600 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T01:36:35.600 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:35.600 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:35.601 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:36:35.601 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:35.601 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:36:35.601 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 42361sec preferred_lft 42361sec 2023-12-01T01:36:35.601 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T01:36:35.601 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:35.601 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-01T01:36:35.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:35.602 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:36:35.602 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:36:35.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add name ceph-brx type bridge 2023-12-01T01:36:35.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip addr flush dev ceph-brx 2023-12-01T01:36:35.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set ceph-brx up 2023-12-01T01:36:35.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-01T01:36:35.602 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:36:36.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:36.118 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:36:36.205 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T01:36:36.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:36.209 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T01:36:36.273 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T01:36:36.273 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:36:36.273 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:36:36.273 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:36:36.273 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:36:36.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:36.274 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:36:36.274 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:36:36.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:36:36.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:36:36.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:36:36.274 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:36:37.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:37.057 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:36:37.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:37.114 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list-id 2023-12-01T01:36:37.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:37.170 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:36:37.170 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:36:37.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:36:37.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-01T01:36:37.170 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:36:37.314 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-01T01:36:37.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:37.314 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:36:37.314 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:36:37.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-01T01:36:37.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-01T01:36:37.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-01T01:36:37.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-01T01:36:37.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-01T01:36:37.315 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:36:37.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:37.593 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:36:37.593 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:36:37.593 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T01:36:37.593 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set dev brx.0 master ceph-brx 2023-12-01T01:36:37.593 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:36:37.749 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:36:37.749 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T01:36:37.798 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:36:37.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:36:37.798 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:36:37.896 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:36:37.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:37.896 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:36:38.053 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:36:38.053 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:36:38.054 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:36:38.527 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:36:38.527 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:36:38.528 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:36:38.528 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:36:38.528 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:36:38.561 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:36:38.561 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:36:38.561 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:36:38.562 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:36:38.562 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:36:38.562 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:36:38.562 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:36:38.562 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:36:38.564 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T01:36:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:36:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:36:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:36:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:36:38.570 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:36:38.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:38.570 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:36:38.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:38.666 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:38.742 INFO:teuthology.orchestra.run.smithi115.stdout:client4436 2023-12-01T01:36:38.743 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:36:38.743 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:36:38.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:38.800 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:38.871 INFO:teuthology.orchestra.run.smithi115.stdout:client4436 2023-12-01T01:36:38.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:38.872 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:38.949 INFO:teuthology.orchestra.run.smithi115.stdout:client4436 2023-12-01T01:36:38.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:38.949 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/5ba3726b-7abf-4a38-892b-ec2cd5ba09fc.client4436/status') 2023-12-01T01:36:39.060 INFO:teuthology.orchestra.run.smithi115.stdout:instance: client.4436 (3)172.21.15.115:0/691698653 2023-12-01T01:36:39.060 INFO:teuthology.orchestra.run.smithi115.stdout:blocklisted: false 2023-12-01T01:36:39.060 INFO:teuthology.orchestra.run.smithi115.stderr:0+1 records in 2023-12-01T01:36:39.061 INFO:teuthology.orchestra.run.smithi115.stderr:0+1 records out 2023-12-01T01:36:39.061 INFO:teuthology.orchestra.run.smithi115.stderr:70 bytes copied, 6.5983e-05 s, 1.1 MB/s 2023-12-01T01:36:39.065 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-12-01T01:36:39.065 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-01T01:36:39.065 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-01T01:36:39.065 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:36:39.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.278+0000 7fa72824d640 1 Processor -- start 2023-12-01T01:36:39.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.279+0000 7fa72824d640 1 -- start start 2023-12-01T01:36:39.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.280+0000 7fa72824d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa7200b5d80 0x7fa7200d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:39.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.280+0000 7fa72824d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7200ce2e0 con 0x7fa7200d1230 2023-12-01T01:36:39.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.280+0000 7fa72824d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7200ce000 con 0x7fa72015cd20 2023-12-01T01:36:39.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.280+0000 7fa72824d640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7200b2160 con 0x7fa7200b5d80 2023-12-01T01:36:39.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.280+0000 7fa7257e8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa7200b5d80 0x7fa7200d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.281+0000 7fa7257e8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa7200b5d80 0x7fa7200d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52312/0 (socket says 172.21.15.115:52312) 2023-12-01T01:36:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.281+0000 7fa7257e8640 1 -- 172.21.15.115:0/3343614691 learned_addr learned my addr 172.21.15.115:0/3343614691 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.281+0000 7fa7267ea640 1 --1- 172.21.15.115:0/3343614691 >> v1:172.21.15.115:6789/0 conn(0x7fa7200d1230 0x7fa7200d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:53522/0 (socket says 172.21.15.115:53522) 2023-12-01T01:36:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.281+0000 7fa7257e8640 1 -- 172.21.15.115:0/3343614691 >> v1:172.21.15.142:6789/0 conn(0x7fa72015cd20 legacy=0x7fa7200b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.281+0000 7fa7257e8640 1 -- 172.21.15.115:0/3343614691 >> v1:172.21.15.115:6789/0 conn(0x7fa7200d1230 legacy=0x7fa7200d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.281+0000 7fa7257e8640 1 -- 172.21.15.115:0/3343614691 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7200cd3d0 con 0x7fa7200b5d80 2023-12-01T01:36:39.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.282+0000 7fa7257e8640 1 --2- 172.21.15.115:0/3343614691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 0x7fa7200d24d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa710009870 tx=0x7fa71002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc8ac7a6c48510 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:39.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.282+0000 7fa724fe7640 1 -- 172.21.15.115:0/3343614691 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710031870 con 0x7fa7200b5d80 2023-12-01T01:36:39.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.282+0000 7fa724fe7640 1 -- 172.21.15.115:0/3343614691 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7100319d0 con 0x7fa7200b5d80 2023-12-01T01:36:39.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.282+0000 7fa724fe7640 1 -- 172.21.15.115:0/3343614691 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710031d60 con 0x7fa7200b5d80 2023-12-01T01:36:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.283+0000 7fa72824d640 1 -- 172.21.15.115:0/3343614691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 msgr2=0x7fa7200d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.283+0000 7fa72824d640 1 --2- 172.21.15.115:0/3343614691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 0x7fa7200d24d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa710009870 tx=0x7fa71002b910 comp rx=0 tx=0).stop 2023-12-01T01:36:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.283+0000 7fa72824d640 1 -- 172.21.15.115:0/3343614691 shutdown_connections 2023-12-01T01:36:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.283+0000 7fa72824d640 1 --2- 172.21.15.115:0/3343614691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 0x7fa7200d24d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.283+0000 7fa72824d640 1 -- 172.21.15.115:0/3343614691 >> 172.21.15.115:0/3343614691 conn(0x7fa72006a0e0 msgr2=0x7fa72006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.283+0000 7fa72824d640 1 -- 172.21.15.115:0/3343614691 shutdown_connections 2023-12-01T01:36:39.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.283+0000 7fa72824d640 1 -- 172.21.15.115:0/3343614691 wait complete. 2023-12-01T01:36:39.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.285+0000 7fa72824d640 1 Processor -- start 2023-12-01T01:36:39.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.286+0000 7fa72824d640 1 -- start start 2023-12-01T01:36:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.287+0000 7fa72824d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 0x7fa7200b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.287+0000 7fa72824d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7200d1230 0x7fa7201f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.287+0000 7fa72824d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72015cd20 0x7fa7200c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.287+0000 7fa72824d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200c7eb0 con 0x7fa7200b5d80 2023-12-01T01:36:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.287+0000 7fa72824d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200c8190 con 0x7fa7200d1230 2023-12-01T01:36:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.287+0000 7fa72824d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200c8470 con 0x7fa72015cd20 2023-12-01T01:36:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.287+0000 7fa725fe9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 0x7fa7200b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.287+0000 7fa725fe9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 0x7fa7200b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52314/0 (socket says 172.21.15.115:52314) 2023-12-01T01:36:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.287+0000 7fa725fe9640 1 -- 172.21.15.115:0/73818074 learned_addr learned my addr 172.21.15.115:0/73818074 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa7257e8640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7200d1230 0x7fa7201f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa7267ea640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72015cd20 0x7fa7200c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa725fe9640 1 -- 172.21.15.115:0/73818074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72015cd20 msgr2=0x7fa7200c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa725fe9640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72015cd20 0x7fa7200c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa725fe9640 1 -- 172.21.15.115:0/73818074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7200d1230 msgr2=0x7fa7201f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa725fe9640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7200d1230 0x7fa7201f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa725fe9640 1 -- 172.21.15.115:0/73818074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7100094d0 con 0x7fa7200b5d80 2023-12-01T01:36:39.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa7267ea640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72015cd20 0x7fa7200c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:39.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa725fe9640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 0x7fa7200b5130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00b3f0 tx=0x7fa71c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:39.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa7167fc640 1 -- 172.21.15.115:0/73818074 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c011830 con 0x7fa7200b5d80 2023-12-01T01:36:39.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa7167fc640 1 -- 172.21.15.115:0/73818074 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c011990 con 0x7fa7200b5d80 2023-12-01T01:36:39.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa7167fc640 1 -- 172.21.15.115:0/73818074 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c011d40 con 0x7fa7200b5d80 2023-12-01T01:36:39.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa72824d640 1 -- 172.21.15.115:0/73818074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7200c8750 con 0x7fa7200b5d80 2023-12-01T01:36:39.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.288+0000 7fa72824d640 1 -- 172.21.15.115:0/73818074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7201f5530 con 0x7fa7200b5d80 2023-12-01T01:36:39.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.290+0000 7fa72824d640 1 -- 172.21.15.115:0/73818074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7200c8c70 con 0x7fa7200b5d80 2023-12-01T01:36:39.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.291+0000 7fa7167fc640 1 -- 172.21.15.115:0/73818074 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa71c01a400 con 0x7fa7200b5d80 2023-12-01T01:36:39.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.292+0000 7fa7167fc640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa70406a430 0x7fa70406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:39.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.292+0000 7fa7167fc640 1 -- 172.21.15.115:0/73818074 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa71c088710 con 0x7fa7200b5d80 2023-12-01T01:36:39.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.292+0000 7fa7257e8640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa70406a430 0x7fa70406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:39.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.293+0000 7fa7257e8640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa70406a430 0x7fa70406c8f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa710036000 tx=0x7fa71003c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:39.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.297+0000 7fa7167fc640 1 -- 172.21.15.115:0/73818074 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa7200c8c70 con 0x7fa7200b5d80 2023-12-01T01:36:39.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.528+0000 7fa72824d640 1 -- 172.21.15.115:0/73818074 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa72004e890 con 0x7fa7200b5d80 2023-12-01T01:36:39.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.529+0000 7fa7167fc640 1 -- 172.21.15.115:0/73818074 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa71c053d90 con 0x7fa7200b5d80 2023-12-01T01:36:39.530 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:36:39.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 -- 172.21.15.115:0/73818074 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa70406a430 msgr2=0x7fa70406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:39.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa70406a430 0x7fa70406c8f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa710036000 tx=0x7fa71003c040 comp rx=0 tx=0).stop 2023-12-01T01:36:39.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 -- 172.21.15.115:0/73818074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 msgr2=0x7fa7200b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:39.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 0x7fa7200b5130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00b3f0 tx=0x7fa71c00b8b0 comp rx=0 tx=0).stop 2023-12-01T01:36:39.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 -- 172.21.15.115:0/73818074 shutdown_connections 2023-12-01T01:36:39.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa70406a430 0x7fa70406c8f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:39.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72015cd20 0x7fa7200c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:39.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7200d1230 0x7fa7201f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:39.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 --2- 172.21.15.115:0/73818074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7200b5d80 0x7fa7200b5130 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:39.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 -- 172.21.15.115:0/73818074 >> 172.21.15.115:0/73818074 conn(0x7fa72006a0e0 msgr2=0x7fa72006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:39.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.531+0000 7fa72824d640 1 -- 172.21.15.115:0/73818074 shutdown_connections 2023-12-01T01:36:39.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:39.532+0000 7fa72824d640 1 -- 172.21.15.115:0/73818074 wait complete. 2023-12-01T01:36:39.551 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-01T01:36:39.552 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:36:39.552 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T01:36:39.552 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-12-01T01:36:39.552 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-01T01:36:39.552 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-01T01:36:39.552 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-01T01:36:39.552 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:36:39.552 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-12-01T01:36:39.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:39.552 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T01:36:39.608 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 42357sec preferred_lft 42357sec 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:39.609 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:36:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:36:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link tentative 2023-12-01T01:36:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:39.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:39.610 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:36:39.696 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T01:36:39.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:39.700 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T01:36:39.750 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T01:36:39.750 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:36:39.751 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:36:39.751 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:36:39.751 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:36:39.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:39.753 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:36:39.754 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:36:39.754 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:36:39.754 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:36:39.754 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:36:39.754 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:36:39.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:39.944 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:36:39.995 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T01:36:39.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:39.996 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list-id 2023-12-01T01:36:40.044 INFO:teuthology.orchestra.run.smithi115.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-12-01T01:36:40.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:40.140 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:36:40.140 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:36:40.140 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T01:36:40.140 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-12-01T01:36:40.140 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:36:40.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:40.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-01T01:36:40.369 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:36:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:36:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:36:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:36:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:40.370 INFO:teuthology.orchestra.run.smithi115.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:36:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether fe:0a:28:75:87:a7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-01T01:36:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-01T01:36:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::fc0a:28ff:fe75:87a7/64 scope link 2023-12-01T01:36:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:40.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:40.373 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-01T01:36:40.464 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:36:40.464 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:36:40.464 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:36:40.465 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:40.465 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:36:40.465 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:40.465 INFO:teuthology.orchestra.run.smithi115.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:36:40.465 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether fe:0a:28:75:87:a7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-01T01:36:40.465 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-01T01:36:40.465 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:40.465 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::fc0a:28ff:fe75:87a7/64 scope link 2023-12-01T01:36:40.465 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:40.469 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-12-01T01:36:40.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:40.470 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:36:40.470 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:36:40.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-12-01T01:36:40.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-01T01:36:40.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-12-01T01:36:40.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-12-01T01:36:40.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-12-01T01:36:40.470 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:36:40.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:40.740 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:36:40.741 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:36:40.741 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.1 up 2023-12-01T01:36:40.741 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set dev brx.1 master ceph-brx 2023-12-01T01:36:40.741 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:36:40.912 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:36:40.912 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-12-01T01:36:40.962 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-12-01T01:36:40.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:36:40.962 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-12-01T01:36:41.010 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:36:41.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:41.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-12-01T01:36:41.150 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-12-01T01:36:41.151 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-12-01T01:36:41.151 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:36:41.172 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-12-01T01:36:41.172 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-12-01T01:36:41.172 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:36:41.172 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:36:41.172 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:36:41.202 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-12-01T01:36:41.202 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-12-01T01:36:41.202 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:36:41.202 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:36:41.202 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:36:41.202 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:36:41.202 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:36:41.203 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:36:41.205 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2023-12-01T01:36:41.205 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:36:41.205 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:36:41.205 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:36:41.205 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:36:41.211 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:36:41.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:41.211 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-12-01T01:36:41.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:41.301 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:41.357 INFO:teuthology.orchestra.run.smithi115.stdout:client4439 2023-12-01T01:36:41.357 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:36:41.358 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:36:41.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:41.411 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:41.517 INFO:teuthology.orchestra.run.smithi115.stdout:client4439 2023-12-01T01:36:41.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:41.518 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:41.578 INFO:teuthology.orchestra.run.smithi115.stdout:client4439 2023-12-01T01:36:41.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:41.578 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/5ba3726b-7abf-4a38-892b-ec2cd5ba09fc.client4439/status') 2023-12-01T01:36:41.664 INFO:teuthology.orchestra.run.smithi115.stdout:instance: client.4439 (3)172.21.15.115:0/4038682981 2023-12-01T01:36:41.664 INFO:teuthology.orchestra.run.smithi115.stdout:blocklisted: false 2023-12-01T01:36:41.665 INFO:teuthology.orchestra.run.smithi115.stderr:0+1 records in 2023-12-01T01:36:41.665 INFO:teuthology.orchestra.run.smithi115.stderr:0+1 records out 2023-12-01T01:36:41.665 INFO:teuthology.orchestra.run.smithi115.stderr:71 bytes copied, 7.4322e-05 s, 955 kB/s 2023-12-01T01:36:41.668 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-12-01T01:36:41.668 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-01T01:36:41.668 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-01T01:36:41.668 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:36:41.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.875+0000 7fced1e81640 1 Processor -- start 2023-12-01T01:36:41.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.875+0000 7fced1e81640 1 -- start start 2023-12-01T01:36:41.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.877+0000 7fced1e81640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcecc0c9ef0 0x7fcecc06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:41.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.877+0000 7fced1e81640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcecc06f8b0 con 0x7fcecc06b6f0 2023-12-01T01:36:41.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.877+0000 7fced1e81640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcecc0adcb0 con 0x7fcecc159dd0 2023-12-01T01:36:41.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.877+0000 7fced1e81640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcecc0ad9d0 con 0x7fcecc0c9ef0 2023-12-01T01:36:41.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.877+0000 7fcec3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcecc0c9ef0 0x7fcecc06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:41.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.877+0000 7fcec3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcecc0c9ef0 0x7fcecc06fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52320/0 (socket says 172.21.15.115:52320) 2023-12-01T01:36:41.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.877+0000 7fcec3fff640 1 -- 172.21.15.115:0/414824107 learned_addr learned my addr 172.21.15.115:0/414824107 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:41.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.877+0000 7fcec3fff640 1 -- 172.21.15.115:0/414824107 >> v1:172.21.15.142:6789/0 conn(0x7fcecc159dd0 legacy=0x7fcecc0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:41.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.878+0000 7fcec3fff640 1 -- 172.21.15.115:0/414824107 >> v1:172.21.15.115:6789/0 conn(0x7fcecc06b6f0 legacy=0x7fcecc070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:41.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.878+0000 7fcec3fff640 1 -- 172.21.15.115:0/414824107 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0c9000 con 0x7fcecc0c9ef0 2023-12-01T01:36:41.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.878+0000 7fcecaffd640 1 -- 172.21.15.115:0/414824107 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472908155 0 0) 0x7fcecc06f8b0 con 0x7fcecc06b6f0 2023-12-01T01:36:41.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.879+0000 7fcec3fff640 1 --2- 172.21.15.115:0/414824107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc0c9ef0 0x7fcecc06fd20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009870 tx=0x7fceb402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5961387a97b92a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:41.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.879+0000 7fcecaffd640 1 -- 172.21.15.115:0/414824107 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4031870 con 0x7fcecc0c9ef0 2023-12-01T01:36:41.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.879+0000 7fcecaffd640 1 -- 172.21.15.115:0/414824107 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb40319d0 con 0x7fcecc0c9ef0 2023-12-01T01:36:41.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.879+0000 7fcecaffd640 1 -- 172.21.15.115:0/414824107 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4031d60 con 0x7fcecc0c9ef0 2023-12-01T01:36:41.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.879+0000 7fced1e81640 1 -- 172.21.15.115:0/414824107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc0c9ef0 msgr2=0x7fcecc06fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.879+0000 7fced1e81640 1 --2- 172.21.15.115:0/414824107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc0c9ef0 0x7fcecc06fd20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009870 tx=0x7fceb402b910 comp rx=0 tx=0).stop 2023-12-01T01:36:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.880+0000 7fced1e81640 1 -- 172.21.15.115:0/414824107 shutdown_connections 2023-12-01T01:36:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.880+0000 7fced1e81640 1 --2- 172.21.15.115:0/414824107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc0c9ef0 0x7fcecc06fd20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.880+0000 7fced1e81640 1 -- 172.21.15.115:0/414824107 >> 172.21.15.115:0/414824107 conn(0x7fcecc0ce880 msgr2=0x7fcecc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.880+0000 7fced1e81640 1 -- 172.21.15.115:0/414824107 shutdown_connections 2023-12-01T01:36:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.880+0000 7fced1e81640 1 -- 172.21.15.115:0/414824107 wait complete. 2023-12-01T01:36:41.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.881+0000 7fced1e81640 1 Processor -- start 2023-12-01T01:36:41.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.882+0000 7fced1e81640 1 -- start start 2023-12-01T01:36:41.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.882+0000 7fced1e81640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcecc06b6f0 0x7fcecc1e6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:41.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.882+0000 7fced1e81640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcecc0c9ef0 0x7fcecc1f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:41.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.882+0000 7fced1e81640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc159dd0 0x7fcecc0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:41.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.882+0000 7fced1e81640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0af5d0 con 0x7fcecc159dd0 2023-12-01T01:36:41.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.882+0000 7fced1e81640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0af8b0 con 0x7fcecc0c9ef0 2023-12-01T01:36:41.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.882+0000 7fced1e81640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0afb90 con 0x7fcecc06b6f0 2023-12-01T01:36:41.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc159dd0 0x7fcecc0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:41.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc159dd0 0x7fcecc0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52324/0 (socket says 172.21.15.115:52324) 2023-12-01T01:36:41.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecbfff640 1 -- 172.21.15.115:0/929485941 learned_addr learned my addr 172.21.15.115:0/929485941 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:41.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcec3fff640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcecc0c9ef0 0x7fcecc1f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:41.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecb7fe640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcecc06b6f0 0x7fcecc1e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:41.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecbfff640 1 -- 172.21.15.115:0/929485941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcecc06b6f0 msgr2=0x7fcecc1e6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:41.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecbfff640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcecc06b6f0 0x7fcecc1e6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:41.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecbfff640 1 -- 172.21.15.115:0/929485941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcecc0c9ef0 msgr2=0x7fcecc1f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:41.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecbfff640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcecc0c9ef0 0x7fcecc1f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:41.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecbfff640 1 -- 172.21.15.115:0/929485941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb40094d0 con 0x7fcecc159dd0 2023-12-01T01:36:41.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcec3fff640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcecc0c9ef0 0x7fcecc1f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:41.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecb7fe640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcecc06b6f0 0x7fcecc1e6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:41.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.883+0000 7fcecbfff640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc159dd0 0x7fcecc0aeed0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcec400c760 tx=0x7fcec400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:41.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.884+0000 7fcec8ff9640 1 -- 172.21.15.115:0/929485941 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4003da0 con 0x7fcecc159dd0 2023-12-01T01:36:41.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.884+0000 7fcec8ff9640 1 -- 172.21.15.115:0/929485941 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec400ce70 con 0x7fcecc159dd0 2023-12-01T01:36:41.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.884+0000 7fcec8ff9640 1 -- 172.21.15.115:0/929485941 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4005290 con 0x7fcecc159dd0 2023-12-01T01:36:41.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.884+0000 7fced1e81640 1 -- 172.21.15.115:0/929485941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcecc0afe70 con 0x7fcecc159dd0 2023-12-01T01:36:41.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.884+0000 7fced1e81640 1 -- 172.21.15.115:0/929485941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcecc154a50 con 0x7fcecc159dd0 2023-12-01T01:36:41.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.885+0000 7fced1e81640 1 -- 172.21.15.115:0/929485941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcecc1e4920 con 0x7fcecc159dd0 2023-12-01T01:36:41.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.886+0000 7fcec8ff9640 1 -- 172.21.15.115:0/929485941 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcec40053f0 con 0x7fcecc159dd0 2023-12-01T01:36:41.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.887+0000 7fcec8ff9640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fceac06a460 0x7fceac06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:41.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.887+0000 7fcec8ff9640 1 -- 172.21.15.115:0/929485941 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcec40887a0 con 0x7fcecc159dd0 2023-12-01T01:36:41.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.887+0000 7fcecb7fe640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fceac06a460 0x7fceac06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:41.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.888+0000 7fcecb7fe640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fceac06a460 0x7fceac06c920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcebc000f60 tx=0x7fcebc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:41.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:41.891+0000 7fcec8ff9640 1 -- 172.21.15.115:0/929485941 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcecc1e4920 con 0x7fcecc159dd0 2023-12-01T01:36:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.117+0000 7fced1e81640 1 -- 172.21.15.115:0/929485941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcecc1e4920 con 0x7fcecc159dd0 2023-12-01T01:36:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.117+0000 7fcec8ff9640 1 -- 172.21.15.115:0/929485941 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcecc1e4920 con 0x7fcecc159dd0 2023-12-01T01:36:42.119 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:36:42.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 -- 172.21.15.115:0/929485941 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fceac06a460 msgr2=0x7fceac06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:42.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fceac06a460 0x7fceac06c920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcebc000f60 tx=0x7fcebc009040 comp rx=0 tx=0).stop 2023-12-01T01:36:42.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 -- 172.21.15.115:0/929485941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc159dd0 msgr2=0x7fcecc0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:42.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc159dd0 0x7fcecc0aeed0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcec400c760 tx=0x7fcec400cc20 comp rx=0 tx=0).stop 2023-12-01T01:36:42.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 -- 172.21.15.115:0/929485941 shutdown_connections 2023-12-01T01:36:42.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fceac06a460 0x7fceac06c920 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:42.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcecc159dd0 0x7fcecc0aeed0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:42.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcecc0c9ef0 0x7fcecc1f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:42.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 --2- 172.21.15.115:0/929485941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcecc06b6f0 0x7fcecc1e6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:42.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 -- 172.21.15.115:0/929485941 >> 172.21.15.115:0/929485941 conn(0x7fcecc0ce880 msgr2=0x7fcecc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:42.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 -- 172.21.15.115:0/929485941 shutdown_connections 2023-12-01T01:36:42.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:42.120+0000 7fced1e81640 1 -- 172.21.15.115:0/929485941 wait complete. 2023-12-01T01:36:42.142 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-01T01:36:42.142 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:36:42.143 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2023-12-01T01:36:42.143 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-01T01:36:42.143 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-01T01:36:42.143 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-01T01:36:42.143 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-01T01:36:42.143 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:36:42.143 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-01T01:36:42.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:42.143 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2023-12-01T01:36:42.200 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2023-12-01T01:36:42.201 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:42.202 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:42.202 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:36:42.202 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:42.202 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:36:42.202 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42351sec preferred_lft 42351sec 2023-12-01T01:36:42.202 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2023-12-01T01:36:42.202 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:42.203 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-01T01:36:42.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:42.203 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2023-12-01T01:36:42.203 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2023-12-01T01:36:42.203 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add name ceph-brx type bridge 2023-12-01T01:36:42.203 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr flush dev ceph-brx 2023-12-01T01:36:42.203 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set ceph-brx up 2023-12-01T01:36:42.204 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-01T01:36:42.204 DEBUG:teuthology.orchestra.run.smithi142:> ') 2023-12-01T01:36:42.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:42.659 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:36:42.741 INFO:teuthology.orchestra.run.smithi142.stdout:1 2023-12-01T01:36:42.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:42.745 DEBUG:teuthology.orchestra.run.smithi142:> route 2023-12-01T01:36:42.799 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2023-12-01T01:36:42.799 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:36:42.799 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:36:42.799 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:36:42.799 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:36:42.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:42.801 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2023-12-01T01:36:42.801 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2023-12-01T01:36:42.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:36:42.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:36:42.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:36:42.801 DEBUG:teuthology.orchestra.run.smithi142:> ') 2023-12-01T01:36:43.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:43.548 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2023-12-01T01:36:43.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:43.604 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list-id 2023-12-01T01:36:43.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:43.659 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2023-12-01T01:36:43.659 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2023-12-01T01:36:43.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-01T01:36:43.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-12-01T01:36:43.659 DEBUG:teuthology.orchestra.run.smithi142:> ') 2023-12-01T01:36:43.796 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-12-01T01:36:43.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:43.796 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2023-12-01T01:36:43.797 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2023-12-01T01:36:43.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-12-01T01:36:43.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-01T01:36:43.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-12-01T01:36:43.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-12-01T01:36:43.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-12-01T01:36:43.797 DEBUG:teuthology.orchestra.run.smithi142:> ') 2023-12-01T01:36:44.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:44.100 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2023-12-01T01:36:44.100 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2023-12-01T01:36:44.100 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2023-12-01T01:36:44.100 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set dev brx.0 master ceph-brx 2023-12-01T01:36:44.100 DEBUG:teuthology.orchestra.run.smithi142:> ') 2023-12-01T01:36:44.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:36:44.264 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-01T01:36:44.311 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-01T01:36:44.312 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:36:44.312 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-01T01:36:44.406 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:36:44.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:44.406 DEBUG:teuthology.orchestra.run.smithi142:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-12-01T01:36:44.541 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-12-01T01:36:44.541 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-12-01T01:36:44.541 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2023-12-01T01:36:44.983 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-12-01T01:36:44.983 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-12-01T01:36:44.983 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2023-12-01T01:36:44.983 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:36:44.984 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:36:45.015 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-12-01T01:36:45.015 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-12-01T01:36:45.015 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2023-12-01T01:36:45.015 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:36:45.015 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:36:45.016 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:36:45.016 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:36:45.016 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:36:45.020 INFO:teuthology.orchestra.run.smithi142.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-12-01T01:36:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:36:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:36:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:36:45.021 INFO:teuthology.orchestra.run.smithi142.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:36:45.024 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:36:45.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:45.025 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-01T01:36:45.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:45.117 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:45.172 INFO:teuthology.orchestra.run.smithi142.stdout:client4442 2023-12-01T01:36:45.172 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:36:45.173 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:36:45.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:45.222 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:45.329 INFO:teuthology.orchestra.run.smithi142.stdout:client4442 2023-12-01T01:36:45.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:45.330 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:45.391 INFO:teuthology.orchestra.run.smithi142.stdout:client4442 2023-12-01T01:36:45.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:45.391 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/5ba3726b-7abf-4a38-892b-ec2cd5ba09fc.client4442/status') 2023-12-01T01:36:45.489 INFO:teuthology.orchestra.run.smithi142.stdout:instance: client.4442 (3)192.168.0.1:0/3122507551 2023-12-01T01:36:45.490 INFO:teuthology.orchestra.run.smithi142.stdout:blocklisted: false 2023-12-01T01:36:45.490 INFO:teuthology.orchestra.run.smithi142.stderr:0+1 records in 2023-12-01T01:36:45.490 INFO:teuthology.orchestra.run.smithi142.stderr:0+1 records out 2023-12-01T01:36:45.490 INFO:teuthology.orchestra.run.smithi142.stderr:69 bytes copied, 6.9574e-05 s, 992 kB/s 2023-12-01T01:36:45.491 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-12-01T01:36:45.491 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-01T01:36:45.491 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-01T01:36:45.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:36:45.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.712+0000 7f3db3e2c640 1 Processor -- start 2023-12-01T01:36:45.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.712+0000 7f3db3e2c640 1 -- start start 2023-12-01T01:36:45.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.714+0000 7f3db3e2c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3dac15e2f0 0x7f3dac0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:45.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.714+0000 7f3db3e2c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dac0add10 con 0x7f3dac06f2f0 2023-12-01T01:36:45.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.714+0000 7f3db3e2c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dac0c5760 con 0x7f3dac0adf10 2023-12-01T01:36:45.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.714+0000 7f3db3e2c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dac0c9cf0 con 0x7f3dac15e2f0 2023-12-01T01:36:45.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.714+0000 7f3db13c7640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3dac06f2f0 0x7f3dac0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:53546/0 (socket says 172.21.15.115:53546) 2023-12-01T01:36:45.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.714+0000 7f3db13c7640 1 -- 172.21.15.115:0/2199639803 learned_addr learned my addr 172.21.15.115:0/2199639803 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:45.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db1bc8640 1 --2- 172.21.15.115:0/2199639803 >> v2:172.21.15.142:3300/0 conn(0x7f3dac15e2f0 0x7f3dac0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:45.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043212986 0 0) 0x7f3dac0add10 con 0x7f3dac06f2f0 2023-12-01T01:36:45.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d98003520 con 0x7f3dac06f2f0 2023-12-01T01:36:45.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 104122806 0 0) 0x7f3d98003520 con 0x7f3dac06f2f0 2023-12-01T01:36:45.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 >> v2:172.21.15.142:3300/0 conn(0x7f3dac15e2f0 msgr2=0x7f3dac0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:45.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 --2- 172.21.15.115:0/2199639803 >> v2:172.21.15.142:3300/0 conn(0x7f3dac15e2f0 0x7f3dac0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 >> v1:172.21.15.142:6789/0 conn(0x7f3dac0adf10 legacy=0x7f3dac0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:45.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac15e6d0 con 0x7f3dac06f2f0 2023-12-01T01:36:45.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3d9c001670 con 0x7f3dac06f2f0 2023-12-01T01:36:45.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 >> v1:172.21.15.115:6789/0 conn(0x7f3dac06f2f0 legacy=0x7f3dac0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:45.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 --2- 172.21.15.115:0/2199639803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d980039e0 0x7f3d98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:45.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 --2- 172.21.15.115:0/2199639803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3d98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:45.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 --2- 172.21.15.115:0/2199639803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d98005330 0x7f3d9800b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:45.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0c9cf0 con 0x7f3d980047c0 2023-12-01T01:36:45.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9800c280 con 0x7f3d980039e0 2023-12-01T01:36:45.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.715+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9800c440 con 0x7f3d98005330 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.716+0000 7f3db13c7640 1 --2- 172.21.15.115:0/2199639803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3d98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.716+0000 7f3db1bc8640 1 --2- 172.21.15.115:0/2199639803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d980039e0 0x7f3d98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.716+0000 7f3db13c7640 1 -- 172.21.15.115:0/2199639803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d98005330 msgr2=0x7f3d9800b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.716+0000 7f3db13c7640 1 --2- 172.21.15.115:0/2199639803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d98005330 0x7f3d9800b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.716+0000 7f3db13c7640 1 -- 172.21.15.115:0/2199639803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d980039e0 msgr2=0x7f3d98003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.716+0000 7f3db13c7640 1 --2- 172.21.15.115:0/2199639803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d980039e0 0x7f3d98003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.716+0000 7f3db13c7640 1 -- 172.21.15.115:0/2199639803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d98003520 con 0x7f3d980047c0 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.716+0000 7f3db23c9640 1 -- 172.21.15.115:0/2199639803 reap_dead start 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.717+0000 7f3db13c7640 1 --2- 172.21.15.115:0/2199639803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3d98004c40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00c240 tx=0x7f3d980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5085f6be53fe27c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.717+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c005ad0 con 0x7f3d980047c0 2023-12-01T01:36:45.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.717+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c005c30 con 0x7f3d980047c0 2023-12-01T01:36:45.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.717+0000 7f3db0bc6640 1 -- 172.21.15.115:0/2199639803 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c02caf0 con 0x7f3d980047c0 2023-12-01T01:36:45.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.718+0000 7f3db3e2c640 1 -- 172.21.15.115:0/2199639803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 msgr2=0x7f3d98004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:45.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.718+0000 7f3db3e2c640 1 --2- 172.21.15.115:0/2199639803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3d98004c40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00c240 tx=0x7f3d980057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.718+0000 7f3db3e2c640 1 -- 172.21.15.115:0/2199639803 shutdown_connections 2023-12-01T01:36:45.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.718+0000 7f3db3e2c640 1 --2- 172.21.15.115:0/2199639803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3d98004c40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.718+0000 7f3db3e2c640 1 -- 172.21.15.115:0/2199639803 >> 172.21.15.115:0/2199639803 conn(0x7f3dac0ce880 msgr2=0x7f3dac0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:45.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.718+0000 7f3db3e2c640 1 -- 172.21.15.115:0/2199639803 shutdown_connections 2023-12-01T01:36:45.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.718+0000 7f3db3e2c640 1 -- 172.21.15.115:0/2199639803 wait complete. 2023-12-01T01:36:45.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.720+0000 7f3db3e2c640 1 Processor -- start 2023-12-01T01:36:45.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.720+0000 7f3db3e2c640 1 -- start start 2023-12-01T01:36:45.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.721+0000 7f3db3e2c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3dac15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:45.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.721+0000 7f3db3e2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3dac0cde30 0x7f3dac0b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:45.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.721+0000 7f3db3e2c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dac147af0 0x7f3dac0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:45.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.721+0000 7f3db3e2c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0b1520 con 0x7f3d980047c0 2023-12-01T01:36:45.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.721+0000 7f3db3e2c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0b1770 con 0x7f3dac147af0 2023-12-01T01:36:45.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.721+0000 7f3db3e2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0b1a50 con 0x7f3dac0cde30 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.721+0000 7f3db1bc8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3dac15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.721+0000 7f3db1bc8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3dac15d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52338/0 (socket says 172.21.15.115:52338) 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.721+0000 7f3db1bc8640 1 -- 172.21.15.115:0/3042527932 learned_addr learned my addr 172.21.15.115:0/3042527932 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3db1bc8640 1 -- 172.21.15.115:0/3042527932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3dac0cde30 msgr2=0x7f3dac0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3db1bc8640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3dac0cde30 0x7f3dac0b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3db1bc8640 1 -- 172.21.15.115:0/3042527932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dac147af0 msgr2=0x7f3dac0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3db1bc8640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dac147af0 0x7f3dac0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3db1bc8640 1 -- 172.21.15.115:0/3042527932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac0b1d30 con 0x7f3d980047c0 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3db1bc8640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3dac15d880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3da800f600 tx=0x7f3da800fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3da27fc640 1 -- 172.21.15.115:0/3042527932 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8014870 con 0x7f3d980047c0 2023-12-01T01:36:45.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3da27fc640 1 -- 172.21.15.115:0/3042527932 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da80149d0 con 0x7f3d980047c0 2023-12-01T01:36:45.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3db3e2c640 1 -- 172.21.15.115:0/3042527932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac0b2010 con 0x7f3d980047c0 2023-12-01T01:36:45.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3da27fc640 1 -- 172.21.15.115:0/3042527932 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8014d40 con 0x7f3d980047c0 2023-12-01T01:36:45.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.722+0000 7f3db3e2c640 1 -- 172.21.15.115:0/3042527932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac0ce290 con 0x7f3d980047c0 2023-12-01T01:36:45.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.723+0000 7f3db3e2c640 1 -- 172.21.15.115:0/3042527932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dac06fda0 con 0x7f3d980047c0 2023-12-01T01:36:45.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.724+0000 7f3da27fc640 1 -- 172.21.15.115:0/3042527932 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3da800d490 con 0x7f3d980047c0 2023-12-01T01:36:45.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.725+0000 7f3da27fc640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d9006a430 0x7f3d9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:45.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.725+0000 7f3da27fc640 1 -- 172.21.15.115:0/3042527932 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3da8089420 con 0x7f3d980047c0 2023-12-01T01:36:45.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.725+0000 7f3db13c7640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d9006a430 0x7f3d9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:45.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.726+0000 7f3db13c7640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d9006a430 0x7f3d9006c8f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c034000 tx=0x7f3d9c039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:45.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.730+0000 7f3da27fc640 1 -- 172.21.15.115:0/3042527932 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3da8054970 con 0x7f3d980047c0 2023-12-01T01:36:45.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.953+0000 7f3db3e2c640 1 -- 172.21.15.115:0/3042527932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3dac06d5a0 con 0x7f3d980047c0 2023-12-01T01:36:45.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.954+0000 7f3da27fc640 1 -- 172.21.15.115:0/3042527932 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3da8057f90 con 0x7f3d980047c0 2023-12-01T01:36:45.956 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:36:45.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.956+0000 7f3db3e2c640 1 -- 172.21.15.115:0/3042527932 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d9006a430 msgr2=0x7f3d9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:45.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.956+0000 7f3db3e2c640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d9006a430 0x7f3d9006c8f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c034000 tx=0x7f3d9c039040 comp rx=0 tx=0).stop 2023-12-01T01:36:45.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.956+0000 7f3db3e2c640 1 -- 172.21.15.115:0/3042527932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 msgr2=0x7f3dac15d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:45.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.956+0000 7f3db3e2c640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3dac15d880 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3da800f600 tx=0x7f3da800fac0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.957+0000 7f3db3e2c640 1 -- 172.21.15.115:0/3042527932 shutdown_connections 2023-12-01T01:36:45.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.957+0000 7f3db3e2c640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d9006a430 0x7f3d9006c8f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.957+0000 7f3db3e2c640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3dac0cde30 0x7f3dac0b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.957+0000 7f3db3e2c640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3dac147af0 0x7f3dac0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.957+0000 7f3db3e2c640 1 --2- 172.21.15.115:0/3042527932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d980047c0 0x7f3dac15d880 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:45.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.957+0000 7f3db3e2c640 1 -- 172.21.15.115:0/3042527932 >> 172.21.15.115:0/3042527932 conn(0x7f3dac0ce880 msgr2=0x7f3dac06c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:45.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.957+0000 7f3db3e2c640 1 -- 172.21.15.115:0/3042527932 shutdown_connections 2023-12-01T01:36:45.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:45.957+0000 7f3db3e2c640 1 -- 172.21.15.115:0/3042527932 wait complete. 2023-12-01T01:36:45.976 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-01T01:36:45.977 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:36:45.977 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2023-12-01T01:36:45.977 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-12-01T01:36:45.977 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-01T01:36:45.977 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-01T01:36:45.977 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-01T01:36:45.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:36:45.977 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-12-01T01:36:45.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:45.977 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2023-12-01T01:36:46.039 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:36:46.039 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:36:46.040 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42347sec preferred_lft 42347sec 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether d6:60:2a:55:83:7c brd ff:ff:ff:ff:ff:ff 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::848:16ff:fe51:fe77/64 scope link 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether d6:60:2a:55:83:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::d460:2aff:fe55:837c/64 scope link 2023-12-01T01:36:46.041 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:46.043 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:36:46.135 INFO:teuthology.orchestra.run.smithi142.stdout:1 2023-12-01T01:36:46.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:46.140 DEBUG:teuthology.orchestra.run.smithi142:> route 2023-12-01T01:36:46.193 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2023-12-01T01:36:46.193 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:36:46.193 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:36:46.193 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:36:46.193 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:36:46.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:46.194 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2023-12-01T01:36:46.194 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2023-12-01T01:36:46.194 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:36:46.194 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:36:46.194 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:36:46.194 DEBUG:teuthology.orchestra.run.smithi142:> ') 2023-12-01T01:36:46.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:46.374 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2023-12-01T01:36:46.425 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-01T01:36:46.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:46.427 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list-id 2023-12-01T01:36:46.478 INFO:teuthology.orchestra.run.smithi142.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-12-01T01:36:46.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:46.479 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2023-12-01T01:36:46.479 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2023-12-01T01:36:46.479 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-01T01:36:46.479 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-12-01T01:36:46.479 DEBUG:teuthology.orchestra.run.smithi142:> ') 2023-12-01T01:36:46.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:46.623 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-01T01:36:46.711 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:36:46.711 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:36:46.711 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:36:46.711 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.711 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2023-12-01T01:36:46.711 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.712 INFO:teuthology.orchestra.run.smithi142.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:36:46.712 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether de:ab:1d:7e:23:d3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-01T01:36:46.712 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-01T01:36:46.712 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.712 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::dcab:1dff:fe7e:23d3/64 scope link 2023-12-01T01:36:46.712 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:46.715 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-01T01:36:46.811 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:36:46.811 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:36:46.811 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:36:46.811 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.811 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2023-12-01T01:36:46.811 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.811 INFO:teuthology.orchestra.run.smithi142.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:36:46.811 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether de:ab:1d:7e:23:d3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-01T01:36:46.812 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-01T01:36:46.812 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.812 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::dcab:1dff:fe7e:23d3/64 scope link 2023-12-01T01:36:46.812 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:36:46.815 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-12-01T01:36:46.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:46.815 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2023-12-01T01:36:46.815 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2023-12-01T01:36:46.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-12-01T01:36:46.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-01T01:36:46.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-12-01T01:36:46.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-12-01T01:36:46.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-12-01T01:36:46.816 DEBUG:teuthology.orchestra.run.smithi142:> ') 2023-12-01T01:36:47.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:47.085 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2023-12-01T01:36:47.085 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2023-12-01T01:36:47.085 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.1 up 2023-12-01T01:36:47.085 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set dev brx.1 master ceph-brx 2023-12-01T01:36:47.085 DEBUG:teuthology.orchestra.run.smithi142:> ') 2023-12-01T01:36:47.255 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:36:47.255 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-12-01T01:36:47.301 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-12-01T01:36:47.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:36:47.302 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-12-01T01:36:47.399 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:36:47.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:47.399 DEBUG:teuthology.orchestra.run.smithi142:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-12-01T01:36:47.536 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-12-01T01:36:47.537 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-12-01T01:36:47.537 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2023-12-01T01:36:47.557 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-12-01T01:36:47.558 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-12-01T01:36:47.558 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2023-12-01T01:36:47.558 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:36:47.558 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:36:47.583 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-12-01T01:36:47.583 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-12-01T01:36:47.583 INFO:teuthology.orchestra.run.smithi142.stdout:invalid new device string format 2023-12-01T01:36:47.583 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:36:47.583 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:36:47.583 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:36:47.583 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:36:47.583 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:36:47.585 INFO:teuthology.orchestra.run.smithi142.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2023-12-01T01:36:47.585 INFO:teuthology.orchestra.run.smithi142.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:36:47.585 INFO:teuthology.orchestra.run.smithi142.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:36:47.585 INFO:teuthology.orchestra.run.smithi142.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:36:47.585 INFO:teuthology.orchestra.run.smithi142.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:36:47.590 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:36:47.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:47.591 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-12-01T01:36:47.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:47.676 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:47.734 INFO:teuthology.orchestra.run.smithi142.stdout:client4445 2023-12-01T01:36:47.735 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:36:47.735 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:36:47.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:47.789 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:47.845 INFO:teuthology.orchestra.run.smithi142.stdout:client4445 2023-12-01T01:36:47.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:47.846 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:36:47.905 INFO:teuthology.orchestra.run.smithi142.stdout:client4445 2023-12-01T01:36:47.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:47.905 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/5ba3726b-7abf-4a38-892b-ec2cd5ba09fc.client4445/status') 2023-12-01T01:36:47.994 INFO:teuthology.orchestra.run.smithi142.stdout:instance: client.4445 (3)192.168.0.2:0/96854855 2023-12-01T01:36:47.994 INFO:teuthology.orchestra.run.smithi142.stdout:blocklisted: false 2023-12-01T01:36:47.994 INFO:teuthology.orchestra.run.smithi142.stderr:0+1 records in 2023-12-01T01:36:47.994 INFO:teuthology.orchestra.run.smithi142.stderr:0+1 records out 2023-12-01T01:36:47.994 INFO:teuthology.orchestra.run.smithi142.stderr:67 bytes copied, 7.2211e-05 s, 928 kB/s 2023-12-01T01:36:47.997 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-12-01T01:36:48.052 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:48.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.275+0000 7f09c7e51640 1 Processor -- start 2023-12-01T01:36:48.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.276+0000 7f09c7e51640 1 -- start start 2023-12-01T01:36:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.277+0000 7f09c7e51640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00b5d80 0x7f09c00d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.277+0000 7f09c7e51640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f09c00d1230 0x7f09c00d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.277+0000 7f09c7e51640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c00ce2e0 con 0x7f09c015cd20 2023-12-01T01:36:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.277+0000 7f09c7e51640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c00b2160 con 0x7f09c00d1230 2023-12-01T01:36:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.277+0000 7f09c7e51640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00d1be0 con 0x7f09c00b5d80 2023-12-01T01:36:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.277+0000 7f09c63ee640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f09c00d1230 0x7f09c00d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.277+0000 7f09c63ee640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f09c00d1230 0x7f09c00d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45868/0 (socket says 172.21.15.115:45868) 2023-12-01T01:36:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.277+0000 7f09c63ee640 1 -- 172.21.15.115:0/3466854129 learned_addr learned my addr 172.21.15.115:0/3466854129 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:48.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.278+0000 7f09c63ee640 1 -- 172.21.15.115:0/3466854129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00b5d80 msgr2=0x7f09c00d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:48.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.278+0000 7f09c53ec640 1 --2- 172.21.15.115:0/3466854129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00b5d80 0x7f09c00d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.278+0000 7f09c63ee640 1 --2- 172.21.15.115:0/3466854129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00b5d80 0x7f09c00d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.278+0000 7f09c63ee640 1 -- 172.21.15.115:0/3466854129 >> v1:172.21.15.142:6789/0 conn(0x7f09c015cd20 legacy=0x7f09c00b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.278+0000 7f09c63ee640 1 -- 172.21.15.115:0/3466854129 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00ce000 con 0x7f09c00d1230 2023-12-01T01:36:48.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.279+0000 7f09c63ee640 1 --2- 172.21.15.115:0/3466854129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c00d1230 0x7f09c00d16a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009870 tx=0x7f09b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cc7720a4116e79 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:48.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.280+0000 7f09c4beb640 1 -- 172.21.15.115:0/3466854129 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8031640 con 0x7f09c00d1230 2023-12-01T01:36:48.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.280+0000 7f09c4beb640 1 -- 172.21.15.115:0/3466854129 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b80317a0 con 0x7f09c00d1230 2023-12-01T01:36:48.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.280+0000 7f09c4beb640 1 -- 172.21.15.115:0/3466854129 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8031b30 con 0x7f09c00d1230 2023-12-01T01:36:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.280+0000 7f09c7e51640 1 -- 172.21.15.115:0/3466854129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c00d1230 msgr2=0x7f09c00d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.280+0000 7f09c7e51640 1 --2- 172.21.15.115:0/3466854129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c00d1230 0x7f09c00d16a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009870 tx=0x7f09b802b8f0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.280+0000 7f09c7e51640 1 -- 172.21.15.115:0/3466854129 shutdown_connections 2023-12-01T01:36:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.280+0000 7f09c7e51640 1 --2- 172.21.15.115:0/3466854129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c00d1230 0x7f09c00d16a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.280+0000 7f09c7e51640 1 --2- 172.21.15.115:0/3466854129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00b5d80 0x7f09c00d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.280+0000 7f09c7e51640 1 -- 172.21.15.115:0/3466854129 >> 172.21.15.115:0/3466854129 conn(0x7f09c006a0e0 msgr2=0x7f09c006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.280+0000 7f09c7e51640 1 -- 172.21.15.115:0/3466854129 shutdown_connections 2023-12-01T01:36:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.281+0000 7f09c7e51640 1 -- 172.21.15.115:0/3466854129 wait complete. 2023-12-01T01:36:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.282+0000 7f09c7e51640 1 Processor -- start 2023-12-01T01:36:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.282+0000 7f09c7e51640 1 -- start start 2023-12-01T01:36:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.283+0000 7f09c7e51640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c00b5d80 0x7f09c015de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.283+0000 7f09c7e51640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00d1230 0x7f09c00b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.283+0000 7f09c7e51640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c015cd20 0x7f09c00c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.283+0000 7f09c7e51640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00b2be0 con 0x7f09c015cd20 2023-12-01T01:36:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.283+0000 7f09c7e51640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00b5160 con 0x7f09c00b5d80 2023-12-01T01:36:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.283+0000 7f09c7e51640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00b22b0 con 0x7f09c00d1230 2023-12-01T01:36:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c63ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c015cd20 0x7f09c00c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c63ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c015cd20 0x7f09c00c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45876/0 (socket says 172.21.15.115:45876) 2023-12-01T01:36:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c63ee640 1 -- 172.21.15.115:0/1840369938 learned_addr learned my addr 172.21.15.115:0/1840369938 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c63ee640 1 -- 172.21.15.115:0/1840369938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00d1230 msgr2=0x7f09c00b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c53ec640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00d1230 0x7f09c00b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c5bed640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c00b5d80 0x7f09c015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c63ee640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00d1230 0x7f09c00b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c63ee640 1 -- 172.21.15.115:0/1840369938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c00b5d80 msgr2=0x7f09c015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c63ee640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c00b5d80 0x7f09c015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c63ee640 1 -- 172.21.15.115:0/1840369938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b80094d0 con 0x7f09c015cd20 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c53ec640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00d1230 0x7f09c00b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.284+0000 7f09c63ee640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c015cd20 0x7f09c00c8d50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009870 tx=0x7f09b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.285+0000 7f09be7fc640 1 -- 172.21.15.115:0/1840369938 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b803ac60 con 0x7f09c015cd20 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.285+0000 7f09be7fc640 1 -- 172.21.15.115:0/1840369938 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b803ade0 con 0x7f09c015cd20 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.285+0000 7f09be7fc640 1 -- 172.21.15.115:0/1840369938 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8044630 con 0x7f09c015cd20 2023-12-01T01:36:48.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.285+0000 7f09c7e51640 1 -- 172.21.15.115:0/1840369938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c00b2590 con 0x7f09c015cd20 2023-12-01T01:36:48.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.285+0000 7f09c7e51640 1 -- 172.21.15.115:0/1840369938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c01f1cd0 con 0x7f09c015cd20 2023-12-01T01:36:48.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.286+0000 7f09c7e51640 1 -- 172.21.15.115:0/1840369938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c00b2d50 con 0x7f09c015cd20 2023-12-01T01:36:48.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.287+0000 7f09be7fc640 1 -- 172.21.15.115:0/1840369938 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f09b802f050 con 0x7f09c015cd20 2023-12-01T01:36:48.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.288+0000 7f09be7fc640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09a406a430 0x7f09a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.288+0000 7f09be7fc640 1 -- 172.21.15.115:0/1840369938 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09b80aa9c0 con 0x7f09c015cd20 2023-12-01T01:36:48.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.288+0000 7f09c5bed640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09a406a430 0x7f09a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.289+0000 7f09c5bed640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09a406a430 0x7f09a406c8f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09b4007fd0 tx=0x7f09b4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:48.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.292+0000 7f09be7fc640 1 -- 172.21.15.115:0/1840369938 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f09b8075fb0 con 0x7f09c015cd20 2023-12-01T01:36:48.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.483+0000 7f09c7e51640 1 -- 172.21.15.115:0/1840369938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09c004e890 con 0x7f09c015cd20 2023-12-01T01:36:48.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.486+0000 7f09be7fc640 1 -- 172.21.15.115:0/1840369938 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f09b80795d0 con 0x7f09c015cd20 2023-12-01T01:36:48.488 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:48.488 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:11.641476+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:48.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.488+0000 7f09c7e51640 1 -- 172.21.15.115:0/1840369938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09a406a430 msgr2=0x7f09a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.488+0000 7f09c7e51640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09a406a430 0x7f09a406c8f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09b4007fd0 tx=0x7f09b4009040 comp rx=0 tx=0).stop 2023-12-01T01:36:48.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.488+0000 7f09c7e51640 1 -- 172.21.15.115:0/1840369938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c015cd20 msgr2=0x7f09c00c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.488+0000 7f09c7e51640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c015cd20 0x7f09c00c8d50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009870 tx=0x7f09b80046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.488+0000 7f09c7e51640 1 -- 172.21.15.115:0/1840369938 shutdown_connections 2023-12-01T01:36:48.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.488+0000 7f09c7e51640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09a406a430 0x7f09a406c8f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.488+0000 7f09c7e51640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09c00d1230 0x7f09c00b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.488+0000 7f09c7e51640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09c015cd20 0x7f09c00c8d50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.488+0000 7f09c7e51640 1 --2- 172.21.15.115:0/1840369938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09c00b5d80 0x7f09c015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.488+0000 7f09c7e51640 1 -- 172.21.15.115:0/1840369938 >> 172.21.15.115:0/1840369938 conn(0x7f09c006a0e0 msgr2=0x7f09c006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.489+0000 7f09c7e51640 1 -- 172.21.15.115:0/1840369938 shutdown_connections 2023-12-01T01:36:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.489+0000 7f09c7e51640 1 -- 172.21.15.115:0/1840369938 wait complete. 2023-12-01T01:36:48.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:36:48.511 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:36:48.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.721+0000 7f61f0dab640 1 Processor -- start 2023-12-01T01:36:48.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.722+0000 7f61f0dab640 1 -- start start 2023-12-01T01:36:48.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.723+0000 7f61f0dab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61ec15cd90 0x7f61ec0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.723+0000 7f61f0dab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f61ec0b5fa0 0x7f61ec161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.723+0000 7f61f0dab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f61ec0cf650 0x7f61ec0cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.723+0000 7f61f0dab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ec0d0ec0 con 0x7f61ec0cf650 2023-12-01T01:36:48.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.723+0000 7f61f0dab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ec0ce5e0 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.723+0000 7f61f0dab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec161950 con 0x7f61ec15cd90 2023-12-01T01:36:48.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.723+0000 7f61ead9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f61ec0cf650 0x7f61ec0cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.723+0000 7f61ead9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f61ec0cf650 0x7f61ec0cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45878/0 (socket says 172.21.15.115:45878) 2023-12-01T01:36:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.723+0000 7f61ead9d640 1 -- 172.21.15.115:0/3291494986 learned_addr learned my addr 172.21.15.115:0/3291494986 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.724+0000 7f61e9d9b640 1 --2- 172.21.15.115:0/3291494986 >> v2:172.21.15.142:3300/0 conn(0x7f61ec0b5fa0 0x7f61ec161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.724+0000 7f61ea59c640 1 --2- 172.21.15.115:0/3291494986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61ec15cd90 0x7f61ec0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.724+0000 7f61ead9d640 1 -- 172.21.15.115:0/3291494986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61ec15cd90 msgr2=0x7f61ec0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.724+0000 7f61ead9d640 1 --2- 172.21.15.115:0/3291494986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61ec15cd90 0x7f61ec0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.724+0000 7f61ead9d640 1 -- 172.21.15.115:0/3291494986 >> v2:172.21.15.142:3300/0 conn(0x7f61ec0b5fa0 msgr2=0x7f61ec161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.724+0000 7f61ead9d640 1 --2- 172.21.15.115:0/3291494986 >> v2:172.21.15.142:3300/0 conn(0x7f61ec0b5fa0 0x7f61ec161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.724+0000 7f61ead9d640 1 -- 172.21.15.115:0/3291494986 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec0d0000 con 0x7f61ec0cf650 2023-12-01T01:36:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.725+0000 7f61ead9d640 1 --2- 172.21.15.115:0/3291494986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0cf650 0x7f61ec0cfac0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0b5880 tx=0x7f61e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c912e7f94c2ed7b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.725+0000 7f61e959a640 1 -- 172.21.15.115:0/3291494986 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4031870 con 0x7f61ec0cf650 2023-12-01T01:36:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.725+0000 7f61e959a640 1 -- 172.21.15.115:0/3291494986 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e40319d0 con 0x7f61ec0cf650 2023-12-01T01:36:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.725+0000 7f61e959a640 1 -- 172.21.15.115:0/3291494986 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4031d60 con 0x7f61ec0cf650 2023-12-01T01:36:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.726+0000 7f61f0dab640 1 -- 172.21.15.115:0/3291494986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0cf650 msgr2=0x7f61ec0cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.726+0000 7f61f0dab640 1 --2- 172.21.15.115:0/3291494986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0cf650 0x7f61ec0cfac0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f61ec0b5880 tx=0x7f61e402b910 comp rx=0 tx=0).stop 2023-12-01T01:36:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.726+0000 7f61f0dab640 1 -- 172.21.15.115:0/3291494986 shutdown_connections 2023-12-01T01:36:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.726+0000 7f61f0dab640 1 --2- 172.21.15.115:0/3291494986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0cf650 0x7f61ec0cfac0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.726+0000 7f61f0dab640 1 --2- 172.21.15.115:0/3291494986 >> v2:172.21.15.142:3300/0 conn(0x7f61ec0b5fa0 0x7f61ec161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.726+0000 7f61f0dab640 1 --2- 172.21.15.115:0/3291494986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61ec15cd90 0x7f61ec0cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.726+0000 7f61f0dab640 1 -- 172.21.15.115:0/3291494986 >> 172.21.15.115:0/3291494986 conn(0x7f61ec065960 msgr2=0x7f61ec065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.726+0000 7f61f0dab640 1 -- 172.21.15.115:0/3291494986 shutdown_connections 2023-12-01T01:36:48.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.726+0000 7f61f0dab640 1 -- 172.21.15.115:0/3291494986 wait complete. 2023-12-01T01:36:48.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.728+0000 7f61f0dab640 1 Processor -- start 2023-12-01T01:36:48.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.728+0000 7f61f0dab640 1 -- start start 2023-12-01T01:36:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61f0dab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0b5fa0 0x7f61ec15c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61f0dab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61ec0cf650 0x7f61ec15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61f0dab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ec15cd90 0x7f61ec1f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61f0dab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec15d8f0 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61f0dab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec1f3170 con 0x7f61ec15cd90 2023-12-01T01:36:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61f0dab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec15dae0 con 0x7f61ec0cf650 2023-12-01T01:36:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61ea59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0b5fa0 0x7f61ec15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61ea59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0b5fa0 0x7f61ec15c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45892/0 (socket says 172.21.15.115:45892) 2023-12-01T01:36:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61ea59c640 1 -- 172.21.15.115:0/2909891467 learned_addr learned my addr 172.21.15.115:0/2909891467 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61ea59c640 1 -- 172.21.15.115:0/2909891467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61ec0cf650 msgr2=0x7f61ec15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61ea59c640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61ec0cf650 0x7f61ec15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61ea59c640 1 -- 172.21.15.115:0/2909891467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ec15cd90 msgr2=0x7f61ec1f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.729+0000 7f61ead9d640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ec15cd90 0x7f61ec1f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.730+0000 7f61ea59c640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ec15cd90 0x7f61ec1f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.730+0000 7f61ea59c640 1 -- 172.21.15.115:0/2909891467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e40094d0 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.730+0000 7f61ead9d640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ec15cd90 0x7f61ec1f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.730+0000 7f61ea59c640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0b5fa0 0x7f61ec15c160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f61dc007e40 tx=0x7f61dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.730+0000 7f61e2ffd640 1 -- 172.21.15.115:0/2909891467 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc014620 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.730+0000 7f61e2ffd640 1 -- 172.21.15.115:0/2909891467 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc00ee90 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.730+0000 7f61e2ffd640 1 -- 172.21.15.115:0/2909891467 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0149c0 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.730+0000 7f61f0dab640 1 -- 172.21.15.115:0/2909891467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ec15ddc0 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.730+0000 7f61f0dab640 1 -- 172.21.15.115:0/2909891467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec15e380 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.731+0000 7f61f0dab640 1 -- 172.21.15.115:0/2909891467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ec04e890 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.732+0000 7f61e2ffd640 1 -- 172.21.15.115:0/2909891467 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f61dc014b20 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.733+0000 7f61e2ffd640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61cc06a430 0x7f61cc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:48.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.733+0000 7f61e2ffd640 1 -- 172.21.15.115:0/2909891467 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61dc089320 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.733+0000 7f61e9d9b640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61cc06a430 0x7f61cc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:48.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.734+0000 7f61e9d9b640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61cc06a430 0x7f61cc06c8f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f61ec160370 tx=0x7f61d4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.738+0000 7f61e2ffd640 1 -- 172.21.15.115:0/2909891467 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f61dc054870 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.923+0000 7f61f0dab640 1 -- 172.21.15.115:0/2909891467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61ec0d0a70 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.924+0000 7f61e2ffd640 1 -- 172.21.15.115:0/2909891467 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61dc057e90 con 0x7f61ec0b5fa0 2023-12-01T01:36:48.926 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 -- 172.21.15.115:0/2909891467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61cc06a430 msgr2=0x7f61cc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61cc06a430 0x7f61cc06c8f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f61ec160370 tx=0x7f61d4008040 comp rx=0 tx=0).stop 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 -- 172.21.15.115:0/2909891467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0b5fa0 msgr2=0x7f61ec15c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0b5fa0 0x7f61ec15c160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f61dc007e40 tx=0x7f61dc00e4b0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 -- 172.21.15.115:0/2909891467 shutdown_connections 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61cc06a430 0x7f61cc06c8f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61ec0cf650 0x7f61ec15d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ec15cd90 0x7f61ec1f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 --2- 172.21.15.115:0/2909891467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61ec0b5fa0 0x7f61ec15c160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 -- 172.21.15.115:0/2909891467 >> 172.21.15.115:0/2909891467 conn(0x7f61ec065960 msgr2=0x7f61ec156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 -- 172.21.15.115:0/2909891467 shutdown_connections 2023-12-01T01:36:48.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:48.927+0000 7f61f0dab640 1 -- 172.21.15.115:0/2909891467 wait complete. 2023-12-01T01:36:48.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:36:49.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.162+0000 7f85f5834640 1 Processor -- start 2023-12-01T01:36:49.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.162+0000 7f85f5834640 1 -- start start 2023-12-01T01:36:49.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.164+0000 7f85f5834640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f00b04c0 0x7f85f00cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.164+0000 7f85f5834640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f00cbac0 con 0x7f85f015ac00 2023-12-01T01:36:49.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.164+0000 7f85f5834640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f015afe0 con 0x7f85f00cb630 2023-12-01T01:36:49.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.164+0000 7f85f5834640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f00b12b0 con 0x7f85f00b04c0 2023-12-01T01:36:49.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.164+0000 7f85f4832640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f85f015ac00 0x7f85f00ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39712/0 (socket says 172.21.15.115:39712) 2023-12-01T01:36:49.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.164+0000 7f85f4832640 1 -- 172.21.15.115:0/2178036906 learned_addr learned my addr 172.21.15.115:0/2178036906 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:49.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85effff640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f00b04c0 0x7f85f00cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939974563 0 0) 0x7f85f00cbac0 con 0x7f85f015ac00 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e0003520 con 0x7f85f015ac00 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1193867339 0 0) 0x7f85e0003520 con 0x7f85f015ac00 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f00b04c0 msgr2=0x7f85f00cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f00b04c0 0x7f85f00cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 >> v1:172.21.15.142:6789/0 conn(0x7f85f00cb630 legacy=0x7f85f006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00b0a10 con 0x7f85f015ac00 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f85e4001670 con 0x7f85f015ac00 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 >> v1:172.21.15.115:6789/0 conn(0x7f85f015ac00 legacy=0x7f85f00ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 0x7f85e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f00b04c0 0x7f85e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e0004d00 0x7f85e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f00b12b0 con 0x7f85e00039a0 2023-12-01T01:36:49.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e000bb10 con 0x7f85e0004d00 2023-12-01T01:36:49.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.165+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e000bcd0 con 0x7f85f00b04c0 2023-12-01T01:36:49.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.166+0000 7f85f4832640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 0x7f85e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.166+0000 7f85effff640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f00b04c0 0x7f85e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.166+0000 7f85f5033640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e0004d00 0x7f85e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.166+0000 7f85f4832640 1 -- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f00b04c0 msgr2=0x7f85e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:49.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.166+0000 7f85f4832640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f00b04c0 0x7f85e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.166+0000 7f85f4832640 1 -- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e0004d00 msgr2=0x7f85e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:49.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.166+0000 7f85f4832640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85e0004d00 0x7f85e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.166+0000 7f85f4832640 1 -- 172.21.15.115:0/2178036906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0003520 con 0x7f85e00039a0 2023-12-01T01:36:49.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.167+0000 7f85f4832640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 0x7f85e0003dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f85e400c0e0 tx=0x7f85e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d711b6343be9b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:49.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.168+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4005ad0 con 0x7f85e00039a0 2023-12-01T01:36:49.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.168+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e4005c30 con 0x7f85e00039a0 2023-12-01T01:36:49.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.168+0000 7f85ef7fe640 1 -- 172.21.15.115:0/2178036906 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e402c810 con 0x7f85e00039a0 2023-12-01T01:36:49.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.168+0000 7f85f5834640 1 -- 172.21.15.115:0/2178036906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 msgr2=0x7f85e0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.168+0000 7f85f5834640 1 --2- 172.21.15.115:0/2178036906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 0x7f85e0003dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f85e400c0e0 tx=0x7f85e00057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.168+0000 7f85f5033640 1 -- 172.21.15.115:0/2178036906 reap_dead start 2023-12-01T01:36:49.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.168+0000 7f85f5834640 1 -- 172.21.15.115:0/2178036906 shutdown_connections 2023-12-01T01:36:49.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.168+0000 7f85f5834640 1 -- 172.21.15.115:0/2178036906 >> 172.21.15.115:0/2178036906 conn(0x7f85f0065960 msgr2=0x7f85f0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:49.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.168+0000 7f85f5834640 1 -- 172.21.15.115:0/2178036906 shutdown_connections 2023-12-01T01:36:49.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.169+0000 7f85f5834640 1 -- 172.21.15.115:0/2178036906 wait complete. 2023-12-01T01:36:49.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.170+0000 7f85f5834640 1 Processor -- start 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.170+0000 7f85f5834640 1 -- start start 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.171+0000 7f85f5834640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 0x7f85f00c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.171+0000 7f85f5834640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f01f30f0 0x7f85f00ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.171+0000 7f85f5834640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f015c760 0x7f85f015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.171+0000 7f85f5834640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f00c2e10 con 0x7f85e00039a0 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.171+0000 7f85f5834640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f00c2260 con 0x7f85f01f30f0 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.171+0000 7f85f5834640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f015d150 con 0x7f85f015c760 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.172+0000 7f85f4832640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 0x7f85f00c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.172+0000 7f85f4832640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 0x7f85f00c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45906/0 (socket says 172.21.15.115:45906) 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.172+0000 7f85f4832640 1 -- 172.21.15.115:0/858692681 learned_addr learned my addr 172.21.15.115:0/858692681 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.172+0000 7f85f4832640 1 -- 172.21.15.115:0/858692681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f015c760 msgr2=0x7f85f015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.172+0000 7f85f4832640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f015c760 0x7f85f015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.172+0000 7f85f4832640 1 -- 172.21.15.115:0/858692681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f01f30f0 msgr2=0x7f85f00ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.172+0000 7f85f4832640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f01f30f0 0x7f85f00ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.172+0000 7f85f4832640 1 -- 172.21.15.115:0/858692681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00c2470 con 0x7f85e00039a0 2023-12-01T01:36:49.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.172+0000 7f85f4832640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 0x7f85f00c3cd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f85e402c3b0 tx=0x7f85e4001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.173+0000 7f85ed7fa640 1 -- 172.21.15.115:0/858692681 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e403c040 con 0x7f85e00039a0 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.173+0000 7f85ed7fa640 1 -- 172.21.15.115:0/858692681 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e402cc20 con 0x7f85e00039a0 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.173+0000 7f85f5834640 1 -- 172.21.15.115:0/858692681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f00c2750 con 0x7f85e00039a0 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.173+0000 7f85ed7fa640 1 -- 172.21.15.115:0/858692681 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4007070 con 0x7f85e00039a0 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.173+0000 7f85f5834640 1 -- 172.21.15.115:0/858692681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f00c4210 con 0x7f85e00039a0 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.174+0000 7f85f5834640 1 -- 172.21.15.115:0/858692681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f00ad8a0 con 0x7f85e00039a0 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.175+0000 7f85ed7fa640 1 -- 172.21.15.115:0/858692681 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f85e4032070 con 0x7f85e00039a0 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.175+0000 7f85ed7fa640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85d406a460 0x7f85d406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.176+0000 7f85ed7fa640 1 -- 172.21.15.115:0/858692681 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85e40a9e90 con 0x7f85e00039a0 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.176+0000 7f85effff640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85d406a460 0x7f85d406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.176+0000 7f85effff640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85d406a460 0x7f85d406c920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f85f00c1bb0 tx=0x7f85dc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:49.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.180+0000 7f85ed7fa640 1 -- 172.21.15.115:0/858692681 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f85e4075480 con 0x7f85e00039a0 2023-12-01T01:36:49.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.356+0000 7f85f5834640 1 -- 172.21.15.115:0/858692681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85f00618d0 con 0x7f85e00039a0 2023-12-01T01:36:49.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.357+0000 7f85ed7fa640 1 -- 172.21.15.115:0/858692681 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85e4046020 con 0x7f85e00039a0 2023-12-01T01:36:49.359 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:36:49.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.359+0000 7f85f5834640 1 -- 172.21.15.115:0/858692681 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85d406a460 msgr2=0x7f85d406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.359+0000 7f85f5834640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85d406a460 0x7f85d406c920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f85f00c1bb0 tx=0x7f85dc008040 comp rx=0 tx=0).stop 2023-12-01T01:36:49.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.359+0000 7f85f5834640 1 -- 172.21.15.115:0/858692681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 msgr2=0x7f85f00c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.359+0000 7f85f5834640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 0x7f85f00c3cd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f85e402c3b0 tx=0x7f85e4001930 comp rx=0 tx=0).stop 2023-12-01T01:36:49.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.359+0000 7f85f5834640 1 -- 172.21.15.115:0/858692681 shutdown_connections 2023-12-01T01:36:49.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.359+0000 7f85f5834640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85d406a460 0x7f85d406c920 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.359+0000 7f85f5834640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85f015c760 0x7f85f015cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.359+0000 7f85f5834640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85f01f30f0 0x7f85f00ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.359+0000 7f85f5834640 1 --2- 172.21.15.115:0/858692681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85e00039a0 0x7f85f00c3cd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.359+0000 7f85f5834640 1 -- 172.21.15.115:0/858692681 >> 172.21.15.115:0/858692681 conn(0x7f85f0065960 msgr2=0x7f85f00cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:49.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.360+0000 7f85f5834640 1 -- 172.21.15.115:0/858692681 shutdown_connections 2023-12-01T01:36:49.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.360+0000 7f85f5834640 1 -- 172.21.15.115:0/858692681 wait complete. 2023-12-01T01:36:49.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:36:49.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.591+0000 7fef97f99640 1 Processor -- start 2023-12-01T01:36:49.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.591+0000 7fef97f99640 1 -- start start 2023-12-01T01:36:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.593+0000 7fef97f99640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef9015ac80 0x7fef900cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.593+0000 7fef97f99640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fef900b0e60 0x7fef900b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.593+0000 7fef97f99640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fef900cdaa0 0x7fef900cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.593+0000 7fef97f99640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9015b060 con 0x7fef900b0e60 2023-12-01T01:36:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.593+0000 7fef97f99640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef900b0ce0 con 0x7fef900cdaa0 2023-12-01T01:36:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.593+0000 7fef97f99640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900ce470 con 0x7fef9015ac80 2023-12-01T01:36:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.593+0000 7fef95534640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fef900b0e60 0x7fef900b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.594+0000 7fef95534640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fef900b0e60 0x7fef900b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45914/0 (socket says 172.21.15.115:45914) 2023-12-01T01:36:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.594+0000 7fef95534640 1 -- 172.21.15.115:0/1488966123 learned_addr learned my addr 172.21.15.115:0/1488966123 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.594+0000 7fef96536640 1 --2- 172.21.15.115:0/1488966123 >> v2:172.21.15.142:3300/0 conn(0x7fef900cdaa0 0x7fef900cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.594+0000 7fef95d35640 1 --2- 172.21.15.115:0/1488966123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef9015ac80 0x7fef900cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.594+0000 7fef95534640 1 -- 172.21.15.115:0/1488966123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef9015ac80 msgr2=0x7fef900cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.594+0000 7fef95534640 1 --2- 172.21.15.115:0/1488966123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef9015ac80 0x7fef900cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.594+0000 7fef95534640 1 -- 172.21.15.115:0/1488966123 >> v2:172.21.15.142:3300/0 conn(0x7fef900cdaa0 msgr2=0x7fef900cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:49.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.594+0000 7fef95534640 1 --2- 172.21.15.115:0/1488966123 >> v2:172.21.15.142:3300/0 conn(0x7fef900cdaa0 0x7fef900cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.594+0000 7fef95534640 1 -- 172.21.15.115:0/1488966123 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900cc9a0 con 0x7fef900b0e60 2023-12-01T01:36:49.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.594+0000 7fef95d35640 1 --2- 172.21.15.115:0/1488966123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef9015ac80 0x7fef900cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:49.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.595+0000 7fef95534640 1 --2- 172.21.15.115:0/1488966123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 0x7fef900b07a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fef9006d710 tx=0x7fef8402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599ee579375ce426 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.595+0000 7fef94d33640 1 -- 172.21.15.115:0/1488966123 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84031860 con 0x7fef900b0e60 2023-12-01T01:36:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef94d33640 1 -- 172.21.15.115:0/1488966123 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef840319c0 con 0x7fef900b0e60 2023-12-01T01:36:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef94d33640 1 -- 172.21.15.115:0/1488966123 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84031d50 con 0x7fef900b0e60 2023-12-01T01:36:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef97f99640 1 -- 172.21.15.115:0/1488966123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 msgr2=0x7fef900b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef97f99640 1 --2- 172.21.15.115:0/1488966123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 0x7fef900b07a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fef9006d710 tx=0x7fef8402b900 comp rx=0 tx=0).stop 2023-12-01T01:36:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef97f99640 1 -- 172.21.15.115:0/1488966123 shutdown_connections 2023-12-01T01:36:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef97f99640 1 --2- 172.21.15.115:0/1488966123 >> v2:172.21.15.142:3300/0 conn(0x7fef900cdaa0 0x7fef900cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef97f99640 1 --2- 172.21.15.115:0/1488966123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 0x7fef900b07a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef97f99640 1 --2- 172.21.15.115:0/1488966123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef9015ac80 0x7fef900cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef97f99640 1 -- 172.21.15.115:0/1488966123 >> 172.21.15.115:0/1488966123 conn(0x7fef90065960 msgr2=0x7fef90065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:49.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef97f99640 1 -- 172.21.15.115:0/1488966123 shutdown_connections 2023-12-01T01:36:49.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.596+0000 7fef97f99640 1 -- 172.21.15.115:0/1488966123 wait complete. 2023-12-01T01:36:49.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.598+0000 7fef97f99640 1 Processor -- start 2023-12-01T01:36:49.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.598+0000 7fef97f99640 1 -- start start 2023-12-01T01:36:49.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.599+0000 7fef97f99640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 0x7fef90150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.599+0000 7fef97f99640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef900cdaa0 0x7fef900b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.599+0000 7fef97f99640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef9015ac80 0x7fef9006f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.599+0000 7fef97f99640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901f2f80 con 0x7fef900b0e60 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.599+0000 7fef97f99640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901f2070 con 0x7fef9015ac80 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.599+0000 7fef97f99640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef901f2300 con 0x7fef900cdaa0 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.599+0000 7fef95d35640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 0x7fef90150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.599+0000 7fef95d35640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 0x7fef90150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45918/0 (socket says 172.21.15.115:45918) 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.600+0000 7fef95d35640 1 -- 172.21.15.115:0/3204640112 learned_addr learned my addr 172.21.15.115:0/3204640112 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.600+0000 7fef95534640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef900cdaa0 0x7fef900b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.600+0000 7fef95d35640 1 -- 172.21.15.115:0/3204640112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef900cdaa0 msgr2=0x7fef900b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.600+0000 7fef95d35640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef900cdaa0 0x7fef900b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.600+0000 7fef95d35640 1 -- 172.21.15.115:0/3204640112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef9015ac80 msgr2=0x7fef9006f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.600+0000 7fef96536640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef9015ac80 0x7fef9006f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.600+0000 7fef95d35640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef9015ac80 0x7fef9006f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.600+0000 7fef95d35640 1 -- 172.21.15.115:0/3204640112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840094d0 con 0x7fef900b0e60 2023-12-01T01:36:49.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.600+0000 7fef95534640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef900cdaa0 0x7fef900b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:49.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.600+0000 7fef95d35640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 0x7fef90150350 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00c7f0 tx=0x7fef8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:49.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.601+0000 7fef827fc640 1 -- 172.21.15.115:0/3204640112 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c007820 con 0x7fef900b0e60 2023-12-01T01:36:49.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.601+0000 7fef827fc640 1 -- 172.21.15.115:0/3204640112 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c007980 con 0x7fef900b0e60 2023-12-01T01:36:49.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.601+0000 7fef827fc640 1 -- 172.21.15.115:0/3204640112 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c007cf0 con 0x7fef900b0e60 2023-12-01T01:36:49.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.601+0000 7fef97f99640 1 -- 172.21.15.115:0/3204640112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef901f25e0 con 0x7fef900b0e60 2023-12-01T01:36:49.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.601+0000 7fef97f99640 1 -- 172.21.15.115:0/3204640112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef901f2ba0 con 0x7fef900b0e60 2023-12-01T01:36:49.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.602+0000 7fef97f99640 1 -- 172.21.15.115:0/3204640112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9004e890 con 0x7fef900b0e60 2023-12-01T01:36:49.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.603+0000 7fef827fc640 1 -- 172.21.15.115:0/3204640112 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fef8c0113f0 con 0x7fef900b0e60 2023-12-01T01:36:49.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.604+0000 7fef827fc640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef7406a430 0x7fef7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:49.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.604+0000 7fef827fc640 1 -- 172.21.15.115:0/3204640112 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef8c088be0 con 0x7fef900b0e60 2023-12-01T01:36:49.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.604+0000 7fef95534640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef7406a430 0x7fef7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:49.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.605+0000 7fef95534640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef7406a430 0x7fef7406c8f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef84036000 tx=0x7fef8403c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:49.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.608+0000 7fef827fc640 1 -- 172.21.15.115:0/3204640112 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fef8c054260 con 0x7fef900b0e60 2023-12-01T01:36:49.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.798+0000 7fef97f99640 1 -- 172.21.15.115:0/3204640112 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fef9006fdf0 con 0x7fef900b0e60 2023-12-01T01:36:49.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.799+0000 7fef827fc640 1 -- 172.21.15.115:0/3204640112 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fef8c01f0a0 con 0x7fef900b0e60 2023-12-01T01:36:49.801 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:36:49.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.801+0000 7fef97f99640 1 -- 172.21.15.115:0/3204640112 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef7406a430 msgr2=0x7fef7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.801+0000 7fef97f99640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef7406a430 0x7fef7406c8f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef84036000 tx=0x7fef8403c040 comp rx=0 tx=0).stop 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.801+0000 7fef97f99640 1 -- 172.21.15.115:0/3204640112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 msgr2=0x7fef90150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.801+0000 7fef97f99640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 0x7fef90150350 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00c7f0 tx=0x7fef8c00ccb0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.802+0000 7fef97f99640 1 -- 172.21.15.115:0/3204640112 shutdown_connections 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.802+0000 7fef97f99640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef7406a430 0x7fef7406c8f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.802+0000 7fef97f99640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef900cdaa0 0x7fef900b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.802+0000 7fef97f99640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef9015ac80 0x7fef9006f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.802+0000 7fef97f99640 1 --2- 172.21.15.115:0/3204640112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef900b0e60 0x7fef90150350 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.802+0000 7fef97f99640 1 -- 172.21.15.115:0/3204640112 >> 172.21.15.115:0/3204640112 conn(0x7fef90065960 msgr2=0x7fef9015dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.802+0000 7fef97f99640 1 -- 172.21.15.115:0/3204640112 shutdown_connections 2023-12-01T01:36:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:49.802+0000 7fef97f99640 1 -- 172.21.15.115:0/3204640112 wait complete. 2023-12-01T01:36:49.823 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:50.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.032+0000 7fac46dcb640 1 Processor -- start 2023-12-01T01:36:50.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.032+0000 7fac46dcb640 1 -- start start 2023-12-01T01:36:50.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.033+0000 7fac46dcb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 0x7fac400cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.033+0000 7fac46dcb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fac400cf9a0 0x7fac400b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.033+0000 7fac46dcb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac40160b20 con 0x7fac4015dbf0 2023-12-01T01:36:50.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.033+0000 7fac46dcb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac40160dc0 con 0x7fac400cf9a0 2023-12-01T01:36:50.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.033+0000 7fac46dcb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac401610a0 con 0x7fac4015dfd0 2023-12-01T01:36:50.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.034+0000 7fac44b67640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fac4015dbf0 0x7fac400cdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39728/0 (socket says 172.21.15.115:39728) 2023-12-01T01:36:50.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.034+0000 7fac44b67640 1 -- 172.21.15.115:0/2327332686 learned_addr learned my addr 172.21.15.115:0/2327332686 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:50.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.034+0000 7fac45368640 1 --2- 172.21.15.115:0/2327332686 >> v2:172.21.15.115:3300/0 conn(0x7fac400cf9a0 0x7fac400b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.034+0000 7fac3ffff640 1 --2- 172.21.15.115:0/2327332686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 0x7fac400cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.034+0000 7fac45368640 1 -- 172.21.15.115:0/2327332686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 msgr2=0x7fac400cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:50.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.034+0000 7fac45368640 1 --2- 172.21.15.115:0/2327332686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 0x7fac400cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.034+0000 7fac45368640 1 -- 172.21.15.115:0/2327332686 >> v1:172.21.15.115:6789/0 conn(0x7fac4015dbf0 legacy=0x7fac400cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.034+0000 7fac45368640 1 -- 172.21.15.115:0/2327332686 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac40161380 con 0x7fac400cf9a0 2023-12-01T01:36:50.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.034+0000 7fac3ffff640 1 --2- 172.21.15.115:0/2327332686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 0x7fac400cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:50.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.034+0000 7fac3f7fe640 1 -- 172.21.15.115:0/2327332686 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208422922 0 0) 0x7fac40160b20 con 0x7fac4015dbf0 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.035+0000 7fac45368640 1 --2- 172.21.15.115:0/2327332686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac400cf9a0 0x7fac400b7ef0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac38009d90 tx=0x7fac3802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabc525a38969856 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.036+0000 7fac3f7fe640 1 -- 172.21.15.115:0/2327332686 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3802cc50 con 0x7fac400cf9a0 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.036+0000 7fac3f7fe640 1 -- 172.21.15.115:0/2327332686 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3802cdb0 con 0x7fac400cf9a0 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.036+0000 7fac3f7fe640 1 -- 172.21.15.115:0/2327332686 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38034660 con 0x7fac400cf9a0 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.036+0000 7fac46dcb640 1 -- 172.21.15.115:0/2327332686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac400cf9a0 msgr2=0x7fac400b7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.036+0000 7fac46dcb640 1 --2- 172.21.15.115:0/2327332686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac400cf9a0 0x7fac400b7ef0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac38009d90 tx=0x7fac3802bea0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.036+0000 7fac46dcb640 1 -- 172.21.15.115:0/2327332686 shutdown_connections 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.036+0000 7fac46dcb640 1 --2- 172.21.15.115:0/2327332686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac400cf9a0 0x7fac400b7ef0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.036+0000 7fac46dcb640 1 --2- 172.21.15.115:0/2327332686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 0x7fac400cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.036+0000 7fac46dcb640 1 -- 172.21.15.115:0/2327332686 >> 172.21.15.115:0/2327332686 conn(0x7fac40065960 msgr2=0x7fac40065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.036+0000 7fac46dcb640 1 -- 172.21.15.115:0/2327332686 shutdown_connections 2023-12-01T01:36:50.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.037+0000 7fac46dcb640 1 -- 172.21.15.115:0/2327332686 wait complete. 2023-12-01T01:36:50.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.039+0000 7fac46dcb640 1 Processor -- start 2023-12-01T01:36:50.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.039+0000 7fac46dcb640 1 -- start start 2023-12-01T01:36:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.040+0000 7fac46dcb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac400cf9a0 0x7fac401f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.040+0000 7fac46dcb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac4015dbf0 0x7fac40158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.040+0000 7fac46dcb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 0x7fac40158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.040+0000 7fac46dcb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac401f6b70 con 0x7fac4015dbf0 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.040+0000 7fac46dcb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac400cd6c0 con 0x7fac400cf9a0 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.040+0000 7fac46dcb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac400cdb50 con 0x7fac4015dfd0 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.040+0000 7fac3ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac4015dbf0 0x7fac40158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.040+0000 7fac3ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac4015dbf0 0x7fac40158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45950/0 (socket says 172.21.15.115:45950) 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.040+0000 7fac3ffff640 1 -- 172.21.15.115:0/1559035050 learned_addr learned my addr 172.21.15.115:0/1559035050 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac44b67640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac400cf9a0 0x7fac401f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac45368640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 0x7fac40158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac3ffff640 1 -- 172.21.15.115:0/1559035050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 msgr2=0x7fac40158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac3ffff640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 0x7fac40158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac3ffff640 1 -- 172.21.15.115:0/1559035050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac400cf9a0 msgr2=0x7fac401f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac3ffff640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac400cf9a0 0x7fac401f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac3ffff640 1 -- 172.21.15.115:0/1559035050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac38009a70 con 0x7fac4015dbf0 2023-12-01T01:36:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac3ffff640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac4015dbf0 0x7fac40158400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fac2800c7f0 tx=0x7fac2800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac3d7fa640 1 -- 172.21.15.115:0/1559035050 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac28007a30 con 0x7fac4015dbf0 2023-12-01T01:36:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac3d7fa640 1 -- 172.21.15.115:0/1559035050 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac28007b90 con 0x7fac4015dbf0 2023-12-01T01:36:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac3d7fa640 1 -- 172.21.15.115:0/1559035050 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2800ce40 con 0x7fac4015dbf0 2023-12-01T01:36:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.041+0000 7fac46dcb640 1 -- 172.21.15.115:0/1559035050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac400ccab0 con 0x7fac4015dbf0 2023-12-01T01:36:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.042+0000 7fac46dcb640 1 -- 172.21.15.115:0/1559035050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac400cd070 con 0x7fac4015dbf0 2023-12-01T01:36:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.043+0000 7fac46dcb640 1 -- 172.21.15.115:0/1559035050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4004e890 con 0x7fac4015dbf0 2023-12-01T01:36:50.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.044+0000 7fac3d7fa640 1 -- 172.21.15.115:0/1559035050 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fac280113f0 con 0x7fac4015dbf0 2023-12-01T01:36:50.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.044+0000 7fac3d7fa640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac2006a430 0x7fac2006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.045+0000 7fac3d7fa640 1 -- 172.21.15.115:0/1559035050 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac28088b10 con 0x7fac4015dbf0 2023-12-01T01:36:50.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.045+0000 7fac44b67640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac2006a430 0x7fac2006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.045+0000 7fac44b67640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac2006a430 0x7fac2006c8f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac30000f60 tx=0x7fac30007c00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.049+0000 7fac3d7fa640 1 -- 172.21.15.115:0/1559035050 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fac28054100 con 0x7fac4015dbf0 2023-12-01T01:36:50.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.253+0000 7fac46dcb640 1 -- 172.21.15.115:0/1559035050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac400d0960 con 0x7fac4015dbf0 2023-12-01T01:36:50.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.255+0000 7fac3d7fa640 1 -- 172.21.15.115:0/1559035050 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16089 (secure 0 0 0) 0x7fac2801f090 con 0x7fac4015dbf0 2023-12-01T01:36:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:50.257 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:11.641476+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 -- 172.21.15.115:0/1559035050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac2006a430 msgr2=0x7fac2006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac2006a430 0x7fac2006c8f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac30000f60 tx=0x7fac30007c00 comp rx=0 tx=0).stop 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 -- 172.21.15.115:0/1559035050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac4015dbf0 msgr2=0x7fac40158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac4015dbf0 0x7fac40158400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fac2800c7f0 tx=0x7fac2800cbf0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 -- 172.21.15.115:0/1559035050 shutdown_connections 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac2006a430 0x7fac2006c8f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac4015dfd0 0x7fac40158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac4015dbf0 0x7fac40158400 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 --2- 172.21.15.115:0/1559035050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac400cf9a0 0x7fac401f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 -- 172.21.15.115:0/1559035050 >> 172.21.15.115:0/1559035050 conn(0x7fac40065960 msgr2=0x7fac400d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 -- 172.21.15.115:0/1559035050 shutdown_connections 2023-12-01T01:36:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.258+0000 7fac46dcb640 1 -- 172.21.15.115:0/1559035050 wait complete. 2023-12-01T01:36:50.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:36:50.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:36:50.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.473+0000 7f14a32c7640 1 Processor -- start 2023-12-01T01:36:50.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.473+0000 7f14a32c7640 1 -- start start 2023-12-01T01:36:50.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.474+0000 7f14a32c7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f149c15cd90 0x7f149c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.475+0000 7f14a32c7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149c0ce5e0 con 0x7f149c0b5fa0 2023-12-01T01:36:50.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.475+0000 7f14a32c7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149c0ce300 con 0x7f149c0cfd60 2023-12-01T01:36:50.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.475+0000 7f14a32c7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149c0d0ec0 con 0x7f149c15cd90 2023-12-01T01:36:50.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.475+0000 7f14a0862640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f149c0b5fa0 0x7f149c0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39730/0 (socket says 172.21.15.115:39730) 2023-12-01T01:36:50.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.475+0000 7f14a0862640 1 -- 172.21.15.115:0/3343458071 learned_addr learned my addr 172.21.15.115:0/3343458071 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:50.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.475+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013330210 0 0) 0x7f1490001110 con 0x7f149c0b5fa0 2023-12-01T01:36:50.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.475+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1484003520 con 0x7f149c0b5fa0 2023-12-01T01:36:50.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.475+0000 7f14a1063640 1 --2- 172.21.15.115:0/3343458071 >> v2:172.21.15.142:3300/0 conn(0x7f149c15cd90 0x7f149c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2698074334 0 0) 0x7f1484003520 con 0x7f149c0b5fa0 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 >> v2:172.21.15.142:3300/0 conn(0x7f149c15cd90 msgr2=0x7f149c0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 --2- 172.21.15.115:0/3343458071 >> v2:172.21.15.142:3300/0 conn(0x7f149c15cd90 0x7f149c0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 >> v1:172.21.15.142:6789/0 conn(0x7f149c0cfd60 legacy=0x7f149c0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c0d10a0 con 0x7f149c0b5fa0 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f1490001800 con 0x7f149c0b5fa0 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 >> v1:172.21.15.115:6789/0 conn(0x7f149c0b5fa0 legacy=0x7f149c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840039c0 0x7f1484003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14840047c0 0x7f1484004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14840051d0 0x7f148400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0d0ec0 con 0x7f14840039c0 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148400c120 con 0x7f14840051d0 2023-12-01T01:36:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148400c2e0 con 0x7f14840047c0 2023-12-01T01:36:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.476+0000 7f14a1063640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840039c0 0x7f1484003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.477+0000 7f14a1063640 1 -- 172.21.15.115:0/3343458071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14840047c0 msgr2=0x7f1484004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.477+0000 7f14a0862640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14840047c0 0x7f1484004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.477+0000 7f14a1063640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14840047c0 0x7f1484004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.477+0000 7f14a1063640 1 -- 172.21.15.115:0/3343458071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14840051d0 msgr2=0x7f148400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.477+0000 7f14a1063640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14840051d0 0x7f148400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.477+0000 7f14a1063640 1 -- 172.21.15.115:0/3343458071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484003520 con 0x7f14840039c0 2023-12-01T01:36:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.477+0000 7f14a1864640 1 -- 172.21.15.115:0/3343458071 reap_dead start 2023-12-01T01:36:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.477+0000 7f14a0862640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14840047c0 0x7f1484004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.478+0000 7f14a1063640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840039c0 0x7f1484003de0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f148c007360 tx=0x7f14840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e5619c8d44b487 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.478+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c02ccf0 con 0x7f14840039c0 2023-12-01T01:36:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.478+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c02ce50 con 0x7f14840039c0 2023-12-01T01:36:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.478+0000 7f149bfff640 1 -- 172.21.15.115:0/3343458071 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c034660 con 0x7f14840039c0 2023-12-01T01:36:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.479+0000 7f14a32c7640 1 -- 172.21.15.115:0/3343458071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840039c0 msgr2=0x7f1484003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.479+0000 7f14a32c7640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840039c0 0x7f1484003de0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f148c007360 tx=0x7f14840057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.479+0000 7f14a32c7640 1 -- 172.21.15.115:0/3343458071 shutdown_connections 2023-12-01T01:36:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.479+0000 7f14a32c7640 1 --2- 172.21.15.115:0/3343458071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840039c0 0x7f1484003de0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.479+0000 7f14a32c7640 1 -- 172.21.15.115:0/3343458071 >> 172.21.15.115:0/3343458071 conn(0x7f149c065960 msgr2=0x7f149c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.479+0000 7f14a32c7640 1 -- 172.21.15.115:0/3343458071 shutdown_connections 2023-12-01T01:36:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.479+0000 7f14a32c7640 1 -- 172.21.15.115:0/3343458071 wait complete. 2023-12-01T01:36:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.481+0000 7f14a32c7640 1 Processor -- start 2023-12-01T01:36:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.481+0000 7f14a32c7640 1 -- start start 2023-12-01T01:36:50.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a32c7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14840039c0 0x7f149c1f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a32c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f149c1f5230 0x7f149c1f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a32c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f149c0ca9f0 0x7f149c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a32c7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0cba50 con 0x7f149c1f5230 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a32c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c1f5bf0 con 0x7f149c0ca9f0 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a32c7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0cb770 con 0x7f14840039c0 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a0862640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f149c1f5230 0x7f149c1f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a0862640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f149c1f5230 0x7f149c1f56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45964/0 (socket says 172.21.15.115:45964) 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a0862640 1 -- 172.21.15.115:0/626611307 learned_addr learned my addr 172.21.15.115:0/626611307 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a1063640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14840039c0 0x7f149c1f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a0862640 1 -- 172.21.15.115:0/626611307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14840039c0 msgr2=0x7f149c1f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a0862640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14840039c0 0x7f149c1f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a0862640 1 -- 172.21.15.115:0/626611307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f149c0ca9f0 msgr2=0x7f149c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a0862640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f149c0ca9f0 0x7f149c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.482+0000 7f14a0862640 1 -- 172.21.15.115:0/626611307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c153d40 con 0x7f149c1f5230 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.483+0000 7f14a0862640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f149c1f5230 0x7f149c1f56b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f149000d770 tx=0x7f149000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.483+0000 7f1499ffb640 1 -- 172.21.15.115:0/626611307 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14900074e0 con 0x7f149c1f5230 2023-12-01T01:36:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.483+0000 7f1499ffb640 1 -- 172.21.15.115:0/626611307 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1490007640 con 0x7f149c1f5230 2023-12-01T01:36:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.483+0000 7f1499ffb640 1 -- 172.21.15.115:0/626611307 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14900049e0 con 0x7f149c1f5230 2023-12-01T01:36:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.483+0000 7f14a32c7640 1 -- 172.21.15.115:0/626611307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149c154020 con 0x7f149c1f5230 2023-12-01T01:36:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.483+0000 7f14a32c7640 1 -- 172.21.15.115:0/626611307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c0cb3b0 con 0x7f149c1f5230 2023-12-01T01:36:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.484+0000 7f14a32c7640 1 -- 172.21.15.115:0/626611307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149c153ba0 con 0x7f149c1f5230 2023-12-01T01:36:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.485+0000 7f1499ffb640 1 -- 172.21.15.115:0/626611307 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1490004b40 con 0x7f149c1f5230 2023-12-01T01:36:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.486+0000 7f1499ffb640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a460 0x7f148806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.486+0000 7f1499ffb640 1 -- 172.21.15.115:0/626611307 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14900880f0 con 0x7f149c1f5230 2023-12-01T01:36:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.486+0000 7f14a1063640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a460 0x7f148806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.487+0000 7f14a1063640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a460 0x7f148806c920 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f148c036000 tx=0x7f148c030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.490+0000 7f1499ffb640 1 -- 172.21.15.115:0/626611307 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f149c153ba0 con 0x7f149c1f5230 2023-12-01T01:36:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.685+0000 7f14a32c7640 1 -- 172.21.15.115:0/626611307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f149c0ce5e0 con 0x7f149c1f5230 2023-12-01T01:36:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.686+0000 7f1499ffb640 1 -- 172.21.15.115:0/626611307 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f149c0ce5e0 con 0x7f149c1f5230 2023-12-01T01:36:50.688 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 -- 172.21.15.115:0/626611307 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a460 msgr2=0x7f148806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a460 0x7f148806c920 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f148c036000 tx=0x7f148c030000 comp rx=0 tx=0).stop 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 -- 172.21.15.115:0/626611307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f149c1f5230 msgr2=0x7f149c1f56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f149c1f5230 0x7f149c1f56b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f149000d770 tx=0x7f149000dc30 comp rx=0 tx=0).stop 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 -- 172.21.15.115:0/626611307 shutdown_connections 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a460 0x7f148806c920 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f149c0ca9f0 0x7f149c0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f149c1f5230 0x7f149c1f56b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 --2- 172.21.15.115:0/626611307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14840039c0 0x7f149c1f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 -- 172.21.15.115:0/626611307 >> 172.21.15.115:0/626611307 conn(0x7f149c065960 msgr2=0x7f149c0b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 -- 172.21.15.115:0/626611307 shutdown_connections 2023-12-01T01:36:50.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.689+0000 7f14a32c7640 1 -- 172.21.15.115:0/626611307 wait complete. 2023-12-01T01:36:50.707 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:36:50.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.915+0000 7f3af62af640 1 Processor -- start 2023-12-01T01:36:50.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.915+0000 7f3af62af640 1 -- start start 2023-12-01T01:36:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af62af640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3af0142870 0x7f3af00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af62af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af00cd010 0x7f3af00b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af62af640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3af00b1c50 0x7f3af015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af62af640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af00b3ae0 con 0x7f3af00b1c50 2023-12-01T01:36:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af62af640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af0142c50 con 0x7f3af0142870 2023-12-01T01:36:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af62af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af00cd470 con 0x7f3af00cd010 2023-12-01T01:36:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af484c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3af00b1c50 0x7f3af015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af484c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3af00b1c50 0x7f3af015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45974/0 (socket says 172.21.15.115:45974) 2023-12-01T01:36:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af484c640 1 -- 172.21.15.115:0/3732032180 learned_addr learned my addr 172.21.15.115:0/3732032180 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3aeffff640 1 --2- 172.21.15.115:0/3732032180 >> v2:172.21.15.142:3300/0 conn(0x7f3af0142870 0x7f3af00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3aef7fe640 1 --2- 172.21.15.115:0/3732032180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af00cd010 0x7f3af00b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af484c640 1 -- 172.21.15.115:0/3732032180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af00cd010 msgr2=0x7f3af00b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.917+0000 7f3af484c640 1 --2- 172.21.15.115:0/3732032180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af00cd010 0x7f3af00b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.918+0000 7f3af484c640 1 -- 172.21.15.115:0/3732032180 >> v2:172.21.15.142:3300/0 conn(0x7f3af0142870 msgr2=0x7f3af00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.918+0000 7f3af484c640 1 --2- 172.21.15.115:0/3732032180 >> v2:172.21.15.142:3300/0 conn(0x7f3af0142870 0x7f3af00ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.918+0000 7f3af484c640 1 -- 172.21.15.115:0/3732032180 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af00b20e0 con 0x7f3af00b1c50 2023-12-01T01:36:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.919+0000 7f3af484c640 1 --2- 172.21.15.115:0/3732032180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00b1c50 0x7f3af015d690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3af00b14d0 tx=0x7f3ae802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718c2165079a12e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.919+0000 7f3aeeffd640 1 -- 172.21.15.115:0/3732032180 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae802ca70 con 0x7f3af00b1c50 2023-12-01T01:36:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.919+0000 7f3aeeffd640 1 -- 172.21.15.115:0/3732032180 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae802cbd0 con 0x7f3af00b1c50 2023-12-01T01:36:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.919+0000 7f3aeeffd640 1 -- 172.21.15.115:0/3732032180 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80343f0 con 0x7f3af00b1c50 2023-12-01T01:36:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.920+0000 7f3af62af640 1 -- 172.21.15.115:0/3732032180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00b1c50 msgr2=0x7f3af015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.920+0000 7f3af62af640 1 --2- 172.21.15.115:0/3732032180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00b1c50 0x7f3af015d690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3af00b14d0 tx=0x7f3ae802bed0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.920+0000 7f3af62af640 1 -- 172.21.15.115:0/3732032180 shutdown_connections 2023-12-01T01:36:50.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.920+0000 7f3af62af640 1 --2- 172.21.15.115:0/3732032180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00b1c50 0x7f3af015d690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.920+0000 7f3af62af640 1 --2- 172.21.15.115:0/3732032180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af00cd010 0x7f3af00b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.920+0000 7f3af62af640 1 --2- 172.21.15.115:0/3732032180 >> v2:172.21.15.142:3300/0 conn(0x7f3af0142870 0x7f3af00ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.920+0000 7f3af62af640 1 -- 172.21.15.115:0/3732032180 >> 172.21.15.115:0/3732032180 conn(0x7f3af0065960 msgr2=0x7f3af0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:50.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.920+0000 7f3af62af640 1 -- 172.21.15.115:0/3732032180 shutdown_connections 2023-12-01T01:36:50.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.920+0000 7f3af62af640 1 -- 172.21.15.115:0/3732032180 wait complete. 2023-12-01T01:36:50.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.921+0000 7f3af62af640 1 Processor -- start 2023-12-01T01:36:50.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.922+0000 7f3af62af640 1 -- start start 2023-12-01T01:36:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.922+0000 7f3af62af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3af00b1c50 0x7f3af006bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.922+0000 7f3af62af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00cd010 0x7f3af0155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.922+0000 7f3af62af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af0142870 0x7f3af0158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.922+0000 7f3af62af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0153640 con 0x7f3af00cd010 2023-12-01T01:36:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.922+0000 7f3af62af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0153920 con 0x7f3af00b1c50 2023-12-01T01:36:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.922+0000 7f3af62af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0153c00 con 0x7f3af0142870 2023-12-01T01:36:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.923+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00cd010 0x7f3af0155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.923+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00cd010 0x7f3af0155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45980/0 (socket says 172.21.15.115:45980) 2023-12-01T01:36:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.923+0000 7f3aef7fe640 1 -- 172.21.15.115:0/3220095253 learned_addr learned my addr 172.21.15.115:0/3220095253 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.923+0000 7f3af484c640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af0142870 0x7f3af0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.923+0000 7f3aef7fe640 1 -- 172.21.15.115:0/3220095253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af0142870 msgr2=0x7f3af0158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.923+0000 7f3aef7fe640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af0142870 0x7f3af0158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.923+0000 7f3aef7fe640 1 -- 172.21.15.115:0/3220095253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3af00b1c50 msgr2=0x7f3af006bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.923+0000 7f3aef7fe640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3af00b1c50 0x7f3af006bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.923+0000 7f3aef7fe640 1 -- 172.21.15.115:0/3220095253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae8009b30 con 0x7f3af00cd010 2023-12-01T01:36:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.923+0000 7f3af484c640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af0142870 0x7f3af0158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:50.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.924+0000 7f3aef7fe640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00cd010 0x7f3af0155c70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400c7f0 tx=0x7f3ae400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.924+0000 7f3aecff9640 1 -- 172.21.15.115:0/3220095253 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4004220 con 0x7f3af00cd010 2023-12-01T01:36:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.924+0000 7f3aecff9640 1 -- 172.21.15.115:0/3220095253 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae4004380 con 0x7f3af00cd010 2023-12-01T01:36:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.924+0000 7f3aecff9640 1 -- 172.21.15.115:0/3220095253 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4007860 con 0x7f3af00cd010 2023-12-01T01:36:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.924+0000 7f3af62af640 1 -- 172.21.15.115:0/3220095253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af0153ee0 con 0x7f3af00cd010 2023-12-01T01:36:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.924+0000 7f3af62af640 1 -- 172.21.15.115:0/3220095253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af0153260 con 0x7f3af00cd010 2023-12-01T01:36:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.925+0000 7f3af62af640 1 -- 172.21.15.115:0/3220095253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af004e890 con 0x7f3af00cd010 2023-12-01T01:36:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.926+0000 7f3aecff9640 1 -- 172.21.15.115:0/3220095253 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3ae4007bd0 con 0x7f3af00cd010 2023-12-01T01:36:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.927+0000 7f3aecff9640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3adc06a430 0x7f3adc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.927+0000 7f3aecff9640 1 -- 172.21.15.115:0/3220095253 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ae40888a0 con 0x7f3af00cd010 2023-12-01T01:36:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.927+0000 7f3aeffff640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3adc06a430 0x7f3adc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.928+0000 7f3aeffff640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3adc06a430 0x7f3adc06c8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0006fd0 tx=0x7f3ae0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:50.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:50.931+0000 7f3aecff9640 1 -- 172.21.15.115:0/3220095253 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3ae4053e00 con 0x7f3af00cd010 2023-12-01T01:36:51.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.121+0000 7f3af62af640 1 -- 172.21.15.115:0/3220095253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3af006b6d0 con 0x7f3af00cd010 2023-12-01T01:36:51.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.123+0000 7f3aecff9640 1 -- 172.21.15.115:0/3220095253 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ae401e020 con 0x7f3af00cd010 2023-12-01T01:36:51.125 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 -- 172.21.15.115:0/3220095253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3adc06a430 msgr2=0x7f3adc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3adc06a430 0x7f3adc06c8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0006fd0 tx=0x7f3ae0008040 comp rx=0 tx=0).stop 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 -- 172.21.15.115:0/3220095253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00cd010 msgr2=0x7f3af0155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00cd010 0x7f3af0155c70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400c7f0 tx=0x7f3ae400ccb0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 -- 172.21.15.115:0/3220095253 shutdown_connections 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3adc06a430 0x7f3adc06c8f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3af0142870 0x7f3af0158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3af00cd010 0x7f3af0155c70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 --2- 172.21.15.115:0/3220095253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3af00b1c50 0x7f3af006bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 -- 172.21.15.115:0/3220095253 >> 172.21.15.115:0/3220095253 conn(0x7f3af0065960 msgr2=0x7f3af0140e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 -- 172.21.15.115:0/3220095253 shutdown_connections 2023-12-01T01:36:51.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.126+0000 7f3af62af640 1 -- 172.21.15.115:0/3220095253 wait complete. 2023-12-01T01:36:51.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:36:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.354+0000 7feb1185f640 1 Processor -- start 2023-12-01T01:36:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.354+0000 7feb1185f640 1 -- start start 2023-12-01T01:36:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.356+0000 7feb1185f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c143eb0 0x7feb0c06b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.356+0000 7feb1185f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c0ce1b0 con 0x7feb0c06b7a0 2023-12-01T01:36:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.356+0000 7feb1185f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c0ce490 con 0x7feb0c0cbc80 2023-12-01T01:36:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.356+0000 7feb1185f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0caa90 con 0x7feb0c143eb0 2023-12-01T01:36:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.356+0000 7feb027fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7feb0c06b7a0 0x7feb0c0cb570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39740/0 (socket says 172.21.15.115:39740) 2023-12-01T01:36:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.356+0000 7feb027fc640 1 -- 172.21.15.115:0/4140759118 learned_addr learned my addr 172.21.15.115:0/4140759118 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0affd640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c143eb0 0x7feb0c06b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191184806 0 0) 0x7feb0c0ce1b0 con 0x7feb0c06b7a0 2023-12-01T01:36:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaec003520 con 0x7feb0c06b7a0 2023-12-01T01:36:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 518488774 0 0) 0x7feaec003520 con 0x7feb0c06b7a0 2023-12-01T01:36:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c143eb0 msgr2=0x7feb0c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c143eb0 0x7feb0c06b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 >> v1:172.21.15.142:6789/0 conn(0x7feb0c0cbc80 legacy=0x7feb0c164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0cad70 con 0x7feb0c06b7a0 2023-12-01T01:36:51.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7feaf4001670 con 0x7feb0c06b7a0 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0affd640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c143eb0 0x7feb0c06b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 >> v1:172.21.15.115:6789/0 conn(0x7feb0c06b7a0 legacy=0x7feb0c0cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c143eb0 0x7feaec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 0x7feaec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaec004d00 0x7feaec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0caa90 con 0x7feaec003f90 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec00bae0 con 0x7feaec004d00 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.357+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaec00bca0 con 0x7feb0c143eb0 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.358+0000 7feb027fc640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 0x7feaec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.358+0000 7feb0affd640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c143eb0 0x7feaec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.358+0000 7feb0b7fe640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaec004d00 0x7feaec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.358+0000 7feb027fc640 1 -- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c143eb0 msgr2=0x7feaec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.358+0000 7feb027fc640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c143eb0 0x7feaec0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.358+0000 7feb027fc640 1 -- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaec004d00 msgr2=0x7feaec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.358+0000 7feb027fc640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaec004d00 0x7feaec005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.358+0000 7feb027fc640 1 -- 172.21.15.115:0/4140759118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec003520 con 0x7feaec003f90 2023-12-01T01:36:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.358+0000 7feb0b7fe640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaec004d00 0x7feaec005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.359+0000 7feb027fc640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 0x7feaec0047c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c240 tx=0x7feaec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dd6706d3b9baac server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.359+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4005ad0 con 0x7feaec003f90 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.359+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4005c30 con 0x7feaec003f90 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.359+0000 7feb0a7fc640 1 -- 172.21.15.115:0/4140759118 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf402ca50 con 0x7feaec003f90 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.360+0000 7feb1185f640 1 -- 172.21.15.115:0/4140759118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 msgr2=0x7feaec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.360+0000 7feb1185f640 1 --2- 172.21.15.115:0/4140759118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 0x7feaec0047c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c240 tx=0x7feaec0057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.360+0000 7feb0b7fe640 1 -- 172.21.15.115:0/4140759118 reap_dead start 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.360+0000 7feb1185f640 1 -- 172.21.15.115:0/4140759118 shutdown_connections 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.360+0000 7feb1185f640 1 -- 172.21.15.115:0/4140759118 >> 172.21.15.115:0/4140759118 conn(0x7feb0c065960 msgr2=0x7feb0c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.360+0000 7feb1185f640 1 -- 172.21.15.115:0/4140759118 shutdown_connections 2023-12-01T01:36:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.360+0000 7feb1185f640 1 -- 172.21.15.115:0/4140759118 wait complete. 2023-12-01T01:36:51.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.362+0000 7feb1185f640 1 Processor -- start 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.362+0000 7feb1185f640 1 -- start start 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb1185f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 0x7feb0c15b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb1185f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c1580b0 0x7feb0c156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb1185f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c13f980 0x7feb0c1f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb1185f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c158550 con 0x7feaec003f90 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb1185f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c15be40 con 0x7feb0c13f980 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb1185f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1f8510 con 0x7feb0c1580b0 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 0x7feb0c15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb0affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 0x7feb0c15b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46002/0 (socket says 172.21.15.115:46002) 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb0affd640 1 -- 172.21.15.115:0/1000706608 learned_addr learned my addr 172.21.15.115:0/1000706608 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb027fc640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c1580b0 0x7feb0c156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.363+0000 7feb0b7fe640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c13f980 0x7feb0c1f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb0affd640 1 -- 172.21.15.115:0/1000706608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c1580b0 msgr2=0x7feb0c156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb0affd640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c1580b0 0x7feb0c156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb0affd640 1 -- 172.21.15.115:0/1000706608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c13f980 msgr2=0x7feb0c1f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb0affd640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c13f980 0x7feb0c1f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb0affd640 1 -- 172.21.15.115:0/1000706608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c1f87f0 con 0x7feaec003f90 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb027fc640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c1580b0 0x7feb0c156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb0b7fe640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c13f980 0x7feb0c1f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb0affd640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 0x7feb0c15b250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feafc010860 tx=0x7feafc010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb08ff9640 1 -- 172.21.15.115:0/1000706608 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00cb40 con 0x7feaec003f90 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb08ff9640 1 -- 172.21.15.115:0/1000706608 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00cca0 con 0x7feaec003f90 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb08ff9640 1 -- 172.21.15.115:0/1000706608 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc015690 con 0x7feaec003f90 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb1185f640 1 -- 172.21.15.115:0/1000706608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c157450 con 0x7feaec003f90 2023-12-01T01:36:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.364+0000 7feb1185f640 1 -- 172.21.15.115:0/1000706608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c15c120 con 0x7feaec003f90 2023-12-01T01:36:51.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.365+0000 7feb1185f640 1 -- 172.21.15.115:0/1000706608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c04e890 con 0x7feaec003f90 2023-12-01T01:36:51.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.366+0000 7feb08ff9640 1 -- 172.21.15.115:0/1000706608 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7feafc0157f0 con 0x7feaec003f90 2023-12-01T01:36:51.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.367+0000 7feb08ff9640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feaf006a460 0x7feaf006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.367+0000 7feb08ff9640 1 -- 172.21.15.115:0/1000706608 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7feafc088db0 con 0x7feaec003f90 2023-12-01T01:36:51.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.367+0000 7feb027fc640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feaf006a460 0x7feaf006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.368+0000 7feb027fc640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feaf006a460 0x7feaf006c920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feaf4034000 tx=0x7feaf4009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:51.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.372+0000 7feb08ff9640 1 -- 172.21.15.115:0/1000706608 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7feafc054410 con 0x7feaec003f90 2023-12-01T01:36:51.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.559+0000 7feb1185f640 1 -- 172.21.15.115:0/1000706608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb0c0b12a0 con 0x7feaec003f90 2023-12-01T01:36:51.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.560+0000 7feb08ff9640 1 -- 172.21.15.115:0/1000706608 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7feafc057a30 con 0x7feaec003f90 2023-12-01T01:36:51.562 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:36:51.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 -- 172.21.15.115:0/1000706608 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feaf006a460 msgr2=0x7feaf006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feaf006a460 0x7feaf006c920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feaf4034000 tx=0x7feaf4009030 comp rx=0 tx=0).stop 2023-12-01T01:36:51.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 -- 172.21.15.115:0/1000706608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 msgr2=0x7feb0c15b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 0x7feb0c15b250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feafc010860 tx=0x7feafc010d20 comp rx=0 tx=0).stop 2023-12-01T01:36:51.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 -- 172.21.15.115:0/1000706608 shutdown_connections 2023-12-01T01:36:51.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feaf006a460 0x7feaf006c920 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb0c1580b0 0x7feb0c156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb0c13f980 0x7feb0c1f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 --2- 172.21.15.115:0/1000706608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaec003f90 0x7feb0c15b250 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 -- 172.21.15.115:0/1000706608 >> 172.21.15.115:0/1000706608 conn(0x7feb0c065960 msgr2=0x7feb0c164bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:51.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 -- 172.21.15.115:0/1000706608 shutdown_connections 2023-12-01T01:36:51.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.563+0000 7feb1185f640 1 -- 172.21.15.115:0/1000706608 wait complete. 2023-12-01T01:36:51.584 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:51.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.787+0000 7f2bd7170640 1 Processor -- start 2023-12-01T01:36:51.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.788+0000 7f2bd7170640 1 -- start start 2023-12-01T01:36:51.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.789+0000 7f2bd7170640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd01584b0 0x7f2bd00cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.789+0000 7f2bd7170640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2bd015afe0 0x7f2bd006b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.789+0000 7f2bd7170640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2bd006b7b0 0x7f2bd015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.789+0000 7f2bd7170640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd006bc40 con 0x7f2bd015afe0 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.789+0000 7f2bd7170640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd00cc180 con 0x7f2bd006b7b0 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.789+0000 7f2bd7170640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd015b550 con 0x7f2bd01584b0 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.789+0000 7f2bd596d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2bd015afe0 0x7f2bd006b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.790+0000 7f2bd596d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2bd015afe0 0x7f2bd006b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46014/0 (socket says 172.21.15.115:46014) 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.790+0000 7f2bd596d640 1 -- 172.21.15.115:0/2615184010 learned_addr learned my addr 172.21.15.115:0/2615184010 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.790+0000 7f2bd616e640 1 --2- 172.21.15.115:0/2615184010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd01584b0 0x7f2bd00cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.790+0000 7f2bd696f640 1 --2- 172.21.15.115:0/2615184010 >> v2:172.21.15.142:3300/0 conn(0x7f2bd006b7b0 0x7f2bd015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.790+0000 7f2bd596d640 1 -- 172.21.15.115:0/2615184010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd01584b0 msgr2=0x7f2bd00cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.790+0000 7f2bd596d640 1 --2- 172.21.15.115:0/2615184010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd01584b0 0x7f2bd00cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.790+0000 7f2bd596d640 1 -- 172.21.15.115:0/2615184010 >> v2:172.21.15.142:3300/0 conn(0x7f2bd006b7b0 msgr2=0x7f2bd015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.790+0000 7f2bd596d640 1 --2- 172.21.15.115:0/2615184010 >> v2:172.21.15.142:3300/0 conn(0x7f2bd006b7b0 0x7f2bd015e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.790+0000 7f2bd596d640 1 -- 172.21.15.115:0/2615184010 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd015b830 con 0x7f2bd015afe0 2023-12-01T01:36:51.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.791+0000 7f2bd596d640 1 --2- 172.21.15.115:0/2615184010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd015afe0 0x7f2bd006b270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00b1330 tx=0x7f2bc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b87152ff50f125 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:51.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.792+0000 7f2bd516c640 1 -- 172.21.15.115:0/2615184010 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0031870 con 0x7f2bd015afe0 2023-12-01T01:36:51.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.792+0000 7f2bd516c640 1 -- 172.21.15.115:0/2615184010 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc00319d0 con 0x7f2bd015afe0 2023-12-01T01:36:51.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.792+0000 7f2bd516c640 1 -- 172.21.15.115:0/2615184010 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0031d60 con 0x7f2bd015afe0 2023-12-01T01:36:51.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.792+0000 7f2bd7170640 1 -- 172.21.15.115:0/2615184010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd015afe0 msgr2=0x7f2bd006b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.792+0000 7f2bd7170640 1 --2- 172.21.15.115:0/2615184010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd015afe0 0x7f2bd006b270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00b1330 tx=0x7f2bc002b910 comp rx=0 tx=0).stop 2023-12-01T01:36:51.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.793+0000 7f2bd7170640 1 -- 172.21.15.115:0/2615184010 shutdown_connections 2023-12-01T01:36:51.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.793+0000 7f2bd7170640 1 --2- 172.21.15.115:0/2615184010 >> v2:172.21.15.142:3300/0 conn(0x7f2bd006b7b0 0x7f2bd015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.793+0000 7f2bd7170640 1 --2- 172.21.15.115:0/2615184010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd015afe0 0x7f2bd006b270 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.793+0000 7f2bd7170640 1 --2- 172.21.15.115:0/2615184010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd01584b0 0x7f2bd00cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.793+0000 7f2bd7170640 1 -- 172.21.15.115:0/2615184010 >> 172.21.15.115:0/2615184010 conn(0x7f2bd00ce640 msgr2=0x7f2bd00cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:51.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.793+0000 7f2bd7170640 1 -- 172.21.15.115:0/2615184010 shutdown_connections 2023-12-01T01:36:51.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.793+0000 7f2bd7170640 1 -- 172.21.15.115:0/2615184010 wait complete. 2023-12-01T01:36:51.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.794+0000 7f2bd7170640 1 Processor -- start 2023-12-01T01:36:51.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.795+0000 7f2bd7170640 1 -- start start 2023-12-01T01:36:51.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.795+0000 7f2bd7170640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd006b7b0 0x7f2bd014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.795+0000 7f2bd7170640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd01584b0 0x7f2bd00c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.795+0000 7f2bd7170640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd015afe0 0x7f2bd00ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.795+0000 7f2bd7170640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd014c7f0 con 0x7f2bd01584b0 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.795+0000 7f2bd7170640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00b3820 con 0x7f2bd006b7b0 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.795+0000 7f2bd7170640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00b3b00 con 0x7f2bd015afe0 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd596d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd01584b0 0x7f2bd00c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd596d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd01584b0 0x7f2bd00c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46028/0 (socket says 172.21.15.115:46028) 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd596d640 1 -- 172.21.15.115:0/351522584 learned_addr learned my addr 172.21.15.115:0/351522584 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd616e640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd006b7b0 0x7f2bd014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd596d640 1 -- 172.21.15.115:0/351522584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd015afe0 msgr2=0x7f2bd00ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd596d640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd015afe0 0x7f2bd00ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd596d640 1 -- 172.21.15.115:0/351522584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd006b7b0 msgr2=0x7f2bd014c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd596d640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd006b7b0 0x7f2bd014c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:51.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd596d640 1 -- 172.21.15.115:0/351522584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00094d0 con 0x7f2bd01584b0 2023-12-01T01:36:51.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd616e640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd006b7b0 0x7f2bd014c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:51.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.796+0000 7f2bd596d640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd01584b0 0x7f2bd00c95f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0000c00 tx=0x7f2bc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:51.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.797+0000 7f2bc6ffd640 1 -- 172.21.15.115:0/351522584 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc003abe0 con 0x7f2bd01584b0 2023-12-01T01:36:51.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.797+0000 7f2bc6ffd640 1 -- 172.21.15.115:0/351522584 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc003ad40 con 0x7f2bd01584b0 2023-12-01T01:36:51.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.797+0000 7f2bc6ffd640 1 -- 172.21.15.115:0/351522584 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0044610 con 0x7f2bd01584b0 2023-12-01T01:36:51.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.797+0000 7f2bd7170640 1 -- 172.21.15.115:0/351522584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd00b3de0 con 0x7f2bd01584b0 2023-12-01T01:36:51.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.797+0000 7f2bd7170640 1 -- 172.21.15.115:0/351522584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd00b43a0 con 0x7f2bd01584b0 2023-12-01T01:36:51.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.798+0000 7f2bd7170640 1 -- 172.21.15.115:0/351522584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd00673a0 con 0x7f2bd01584b0 2023-12-01T01:36:51.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.799+0000 7f2bc6ffd640 1 -- 172.21.15.115:0/351522584 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2bc004d450 con 0x7f2bd01584b0 2023-12-01T01:36:51.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.800+0000 7f2bc6ffd640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bb406a460 0x7f2bb406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:51.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.800+0000 7f2bc6ffd640 1 -- 172.21.15.115:0/351522584 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2bc002d040 con 0x7f2bd01584b0 2023-12-01T01:36:51.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.800+0000 7f2bd616e640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bb406a460 0x7f2bb406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:51.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.801+0000 7f2bd616e640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bb406a460 0x7f2bb406c920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc006fd0 tx=0x7f2bcc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:51.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.804+0000 7f2bc6ffd640 1 -- 172.21.15.115:0/351522584 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2bc0076560 con 0x7f2bd01584b0 2023-12-01T01:36:51.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.995+0000 7f2bd7170640 1 -- 172.21.15.115:0/351522584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bd00af1f0 con 0x7f2bd01584b0 2023-12-01T01:36:51.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:51.997+0000 7f2bc6ffd640 1 -- 172.21.15.115:0/351522584 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16138 (secure 0 0 0) 0x7f2bc0079b80 con 0x7f2bd01584b0 2023-12-01T01:36:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:51.999 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":22,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:51.119274+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:52.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 -- 172.21.15.115:0/351522584 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bb406a460 msgr2=0x7f2bb406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bb406a460 0x7f2bb406c920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc006fd0 tx=0x7f2bcc008040 comp rx=0 tx=0).stop 2023-12-01T01:36:52.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 -- 172.21.15.115:0/351522584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd01584b0 msgr2=0x7f2bd00c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd01584b0 0x7f2bd00c95f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0000c00 tx=0x7f2bc00046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 -- 172.21.15.115:0/351522584 shutdown_connections 2023-12-01T01:36:52.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2bb406a460 0x7f2bb406c920 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd015afe0 0x7f2bd00ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bd01584b0 0x7f2bd00c95f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 --2- 172.21.15.115:0/351522584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd006b7b0 0x7f2bd014c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 -- 172.21.15.115:0/351522584 >> 172.21.15.115:0/351522584 conn(0x7f2bd00ce640 msgr2=0x7f2bd00cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:52.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.000+0000 7f2bd7170640 1 -- 172.21.15.115:0/351522584 shutdown_connections 2023-12-01T01:36:52.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.001+0000 7f2bd7170640 1 -- 172.21.15.115:0/351522584 wait complete. 2023-12-01T01:36:52.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:36:52.021 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:36:52.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.243+0000 7fbed9849640 1 Processor -- start 2023-12-01T01:36:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.243+0000 7fbed9849640 1 -- start start 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.245+0000 7fbed9849640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4153c30 0x7fbed40ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.245+0000 7fbed9849640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbed40cfb80 0x7fbed4160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.245+0000 7fbed9849640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed40cdcd0 con 0x7fbed4154010 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.245+0000 7fbed9849640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed4161830 con 0x7fbed40cfb80 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.245+0000 7fbed9849640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed40cd3a0 con 0x7fbed4153c30 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.245+0000 7fbed3fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fbed4154010 0x7fbed40cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39742/0 (socket says 172.21.15.115:39742) 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.245+0000 7fbed3fff640 1 -- 172.21.15.115:0/3694500356 learned_addr learned my addr 172.21.15.115:0/3694500356 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.245+0000 7fbed8847640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4153c30 0x7fbed40ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed9048640 1 --2- 172.21.15.115:0/3694500356 >> v2:172.21.15.142:3300/0 conn(0x7fbed40cfb80 0x7fbed4160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903452706 0 0) 0x7fbed40cdcd0 con 0x7fbed4154010 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbec4003520 con 0x7fbed4154010 2023-12-01T01:36:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1878225721 0 0) 0x7fbec4003520 con 0x7fbed4154010 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4153c30 msgr2=0x7fbed40ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4153c30 0x7fbed40ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 >> v2:172.21.15.142:3300/0 conn(0x7fbed40cfb80 msgr2=0x7fbed4160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 --2- 172.21.15.115:0/3694500356 >> v2:172.21.15.142:3300/0 conn(0x7fbed40cfb80 0x7fbed4160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed40cd680 con 0x7fbed4154010 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fbec0001670 con 0x7fbed4154010 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 >> v1:172.21.15.115:6789/0 conn(0x7fbed4154010 legacy=0x7fbed40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4003970 0x7fbec4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4153c30 0x7fbec40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec4004d00 0x7fbec4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed4161830 con 0x7fbec4004d00 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed40cd3a0 con 0x7fbec4003970 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.246+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec400b980 con 0x7fbed4153c30 2023-12-01T01:36:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.247+0000 7fbed9048640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec4004d00 0x7fbec4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.247+0000 7fbed8847640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4003970 0x7fbec4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.247+0000 7fbed3fff640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4153c30 0x7fbec40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.247+0000 7fbed9048640 1 -- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4153c30 msgr2=0x7fbec40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.247+0000 7fbed9048640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4153c30 0x7fbec40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.247+0000 7fbed9048640 1 -- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4003970 msgr2=0x7fbec4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.247+0000 7fbed9048640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4003970 0x7fbec4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.247+0000 7fbed9048640 1 -- 172.21.15.115:0/3694500356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec4003520 con 0x7fbec4004d00 2023-12-01T01:36:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.247+0000 7fbed8847640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4003970 0x7fbec4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.247+0000 7fbed3fff640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4153c30 0x7fbec40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.248+0000 7fbed9048640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec4004d00 0x7fbec4005180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009fc0 tx=0x7fbec40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85b8dd3a40d4594 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.248+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc02cc30 con 0x7fbec4004d00 2023-12-01T01:36:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.248+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc00f040 con 0x7fbec4004d00 2023-12-01T01:36:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.248+0000 7fbed37fe640 1 -- 172.21.15.115:0/3694500356 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc034440 con 0x7fbec4004d00 2023-12-01T01:36:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.249+0000 7fbed9849640 1 -- 172.21.15.115:0/3694500356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec4004d00 msgr2=0x7fbec4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.249+0000 7fbed9849640 1 --2- 172.21.15.115:0/3694500356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec4004d00 0x7fbec4005180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009fc0 tx=0x7fbec40057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.249+0000 7fbed9048640 1 -- 172.21.15.115:0/3694500356 reap_dead start 2023-12-01T01:36:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.249+0000 7fbed9849640 1 -- 172.21.15.115:0/3694500356 shutdown_connections 2023-12-01T01:36:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.249+0000 7fbed9849640 1 -- 172.21.15.115:0/3694500356 >> 172.21.15.115:0/3694500356 conn(0x7fbed406a0b0 msgr2=0x7fbed406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.249+0000 7fbed9849640 1 -- 172.21.15.115:0/3694500356 shutdown_connections 2023-12-01T01:36:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.249+0000 7fbed9849640 1 -- 172.21.15.115:0/3694500356 wait complete. 2023-12-01T01:36:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.251+0000 7fbed9849640 1 Processor -- start 2023-12-01T01:36:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.251+0000 7fbed9849640 1 -- start start 2023-12-01T01:36:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.252+0000 7fbed9849640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4004d00 0x7fbed414fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.252+0000 7fbed9849640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4155a60 0x7fbed40cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.252+0000 7fbed9849640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed4154470 0x7fbed41561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.252+0000 7fbed9849640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed41506f0 con 0x7fbed4154470 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.252+0000 7fbed9849640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed41555d0 con 0x7fbec4004d00 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.252+0000 7fbed9849640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed41549c0 con 0x7fbed4155a60 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.252+0000 7fbed9048640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed4154470 0x7fbed41561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.252+0000 7fbed9048640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed4154470 0x7fbed41561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46036/0 (socket says 172.21.15.115:46036) 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.252+0000 7fbed9048640 1 -- 172.21.15.115:0/3341071612 learned_addr learned my addr 172.21.15.115:0/3341071612 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed3fff640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4155a60 0x7fbed40cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed8847640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4004d00 0x7fbed414fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed9048640 1 -- 172.21.15.115:0/3341071612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4155a60 msgr2=0x7fbed40cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed9048640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4155a60 0x7fbed40cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed9048640 1 -- 172.21.15.115:0/3341071612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4004d00 msgr2=0x7fbed414fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed9048640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4004d00 0x7fbed414fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed9048640 1 -- 172.21.15.115:0/3341071612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed4154ca0 con 0x7fbed4154470 2023-12-01T01:36:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed3fff640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4155a60 0x7fbed40cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed8847640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4004d00 0x7fbed414fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed9048640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed4154470 0x7fbed41561e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbecc037040 tx=0x7fbecc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed17fa640 1 -- 172.21.15.115:0/3341071612 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc02ce20 con 0x7fbed4154470 2023-12-01T01:36:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed17fa640 1 -- 172.21.15.115:0/3341071612 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc034b50 con 0x7fbed4154470 2023-12-01T01:36:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed17fa640 1 -- 172.21.15.115:0/3341071612 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc003990 con 0x7fbed4154470 2023-12-01T01:36:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed9849640 1 -- 172.21.15.115:0/3341071612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed4154f80 con 0x7fbed4154470 2023-12-01T01:36:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.253+0000 7fbed9849640 1 -- 172.21.15.115:0/3341071612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed4156720 con 0x7fbed4154470 2023-12-01T01:36:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.255+0000 7fbed9849640 1 -- 172.21.15.115:0/3341071612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed40618d0 con 0x7fbed4154470 2023-12-01T01:36:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.255+0000 7fbed17fa640 1 -- 172.21.15.115:0/3341071612 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbecc047460 con 0x7fbed4154470 2023-12-01T01:36:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.256+0000 7fbed17fa640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbeb806a430 0x7fbeb806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.256+0000 7fbed17fa640 1 -- 172.21.15.115:0/3341071612 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbecc0aa710 con 0x7fbed4154470 2023-12-01T01:36:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.257+0000 7fbed8847640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbeb806a430 0x7fbeb806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.257+0000 7fbed8847640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbeb806a430 0x7fbeb806c8f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbed40d1e80 tx=0x7fbec8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:52.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.261+0000 7fbed17fa640 1 -- 172.21.15.115:0/3341071612 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbecc075cd0 con 0x7fbed4154470 2023-12-01T01:36:52.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.451+0000 7fbed9849640 1 -- 172.21.15.115:0/3341071612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbed40cc9e0 con 0x7fbed4154470 2023-12-01T01:36:52.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.452+0000 7fbed17fa640 1 -- 172.21.15.115:0/3341071612 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbecc0792f0 con 0x7fbed4154470 2023-12-01T01:36:52.453 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:36:52.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.454+0000 7fbed9849640 1 -- 172.21.15.115:0/3341071612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbeb806a430 msgr2=0x7fbeb806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.454+0000 7fbed9849640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbeb806a430 0x7fbeb806c8f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbed40d1e80 tx=0x7fbec8008040 comp rx=0 tx=0).stop 2023-12-01T01:36:52.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.454+0000 7fbed9849640 1 -- 172.21.15.115:0/3341071612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed4154470 msgr2=0x7fbed41561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.454+0000 7fbed9849640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed4154470 0x7fbed41561e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbecc037040 tx=0x7fbecc009750 comp rx=0 tx=0).stop 2023-12-01T01:36:52.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.454+0000 7fbed9849640 1 -- 172.21.15.115:0/3341071612 shutdown_connections 2023-12-01T01:36:52.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.454+0000 7fbed9849640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbeb806a430 0x7fbeb806c8f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.454+0000 7fbed9849640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbed4155a60 0x7fbed40cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.455+0000 7fbed9849640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbed4154470 0x7fbed41561e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.455+0000 7fbed9849640 1 --2- 172.21.15.115:0/3341071612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbec4004d00 0x7fbed414fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.455+0000 7fbed9849640 1 -- 172.21.15.115:0/3341071612 >> 172.21.15.115:0/3341071612 conn(0x7fbed406a0b0 msgr2=0x7fbed4150fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.455+0000 7fbed9849640 1 -- 172.21.15.115:0/3341071612 shutdown_connections 2023-12-01T01:36:52.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.455+0000 7fbed9849640 1 -- 172.21.15.115:0/3341071612 wait complete. 2023-12-01T01:36:52.475 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:36:52.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.688+0000 7f8f75789640 1 Processor -- start 2023-12-01T01:36:52.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.688+0000 7f8f75789640 1 -- start start 2023-12-01T01:36:52.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.690+0000 7f8f75789640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8f700ce1b0 0x7f8f700cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.690+0000 7f8f75789640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f700cc9a0 con 0x7f8f700b0e60 2023-12-01T01:36:52.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.690+0000 7f8f75789640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f700cbd90 con 0x7f8f7015ac80 2023-12-01T01:36:52.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.690+0000 7f8f75789640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7015b060 con 0x7f8f700ce1b0 2023-12-01T01:36:52.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.690+0000 7f8f667fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8f700ce1b0 0x7f8f700cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.690+0000 7f8f667fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8f700ce1b0 0x7f8f700cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46038/0 (socket says 172.21.15.115:46038) 2023-12-01T01:36:52.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.690+0000 7f8f6f7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8f700b0e60 0x7f8f7006b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39744/0 (socket says 172.21.15.115:39744) 2023-12-01T01:36:52.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.690+0000 7f8f667fc640 1 -- 172.21.15.115:0/1601305837 learned_addr learned my addr 172.21.15.115:0/1601305837 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:52.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.691+0000 7f8f667fc640 1 -- 172.21.15.115:0/1601305837 >> v1:172.21.15.142:6789/0 conn(0x7f8f7015ac80 legacy=0x7f8f700cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:52.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.691+0000 7f8f667fc640 1 -- 172.21.15.115:0/1601305837 >> v1:172.21.15.115:6789/0 conn(0x7f8f700b0e60 legacy=0x7f8f7006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.691+0000 7f8f667fc640 1 -- 172.21.15.115:0/1601305837 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f700cc070 con 0x7f8f700ce1b0 2023-12-01T01:36:52.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.691+0000 7f8f6e7fc640 1 -- 172.21.15.115:0/1601305837 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563225236 0 0) 0x7f8f700cc9a0 con 0x7f8f700b0e60 2023-12-01T01:36:52.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.692+0000 7f8f667fc640 1 --2- 172.21.15.115:0/1601305837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700ce1b0 0x7f8f700cce30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009870 tx=0x7f8f5802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f099af8731429af4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:52.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.692+0000 7f8f6e7fc640 1 -- 172.21.15.115:0/1601305837 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58030860 con 0x7f8f700ce1b0 2023-12-01T01:36:52.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.692+0000 7f8f6e7fc640 1 -- 172.21.15.115:0/1601305837 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f580309c0 con 0x7f8f700ce1b0 2023-12-01T01:36:52.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.692+0000 7f8f6e7fc640 1 -- 172.21.15.115:0/1601305837 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58030d50 con 0x7f8f700ce1b0 2023-12-01T01:36:52.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.693+0000 7f8f75789640 1 -- 172.21.15.115:0/1601305837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700ce1b0 msgr2=0x7f8f700cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.693+0000 7f8f75789640 1 --2- 172.21.15.115:0/1601305837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700ce1b0 0x7f8f700cce30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009870 tx=0x7f8f5802b9a0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.693+0000 7f8f75789640 1 -- 172.21.15.115:0/1601305837 shutdown_connections 2023-12-01T01:36:52.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.693+0000 7f8f75789640 1 --2- 172.21.15.115:0/1601305837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700ce1b0 0x7f8f700cce30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.693+0000 7f8f75789640 1 -- 172.21.15.115:0/1601305837 >> 172.21.15.115:0/1601305837 conn(0x7f8f70065960 msgr2=0x7f8f70065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:52.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.693+0000 7f8f75789640 1 -- 172.21.15.115:0/1601305837 shutdown_connections 2023-12-01T01:36:52.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.693+0000 7f8f75789640 1 -- 172.21.15.115:0/1601305837 wait complete. 2023-12-01T01:36:52.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.695+0000 7f8f75789640 1 Processor -- start 2023-12-01T01:36:52.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.695+0000 7f8f75789640 1 -- start start 2023-12-01T01:36:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f75789640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700b0e60 0x7f8f70153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f75789640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f700ce1b0 0x7f8f7015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f75789640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f7015ac80 0x7f8f7015c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f75789640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f70153fd0 con 0x7f8f700b0e60 2023-12-01T01:36:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f75789640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f70159050 con 0x7f8f7015ac80 2023-12-01T01:36:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f75789640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f70156fa0 con 0x7f8f700ce1b0 2023-12-01T01:36:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f6effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700b0e60 0x7f8f70153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f6effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700b0e60 0x7f8f70153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46050/0 (socket says 172.21.15.115:46050) 2023-12-01T01:36:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f6effd640 1 -- 172.21.15.115:0/1309825675 learned_addr learned my addr 172.21.15.115:0/1309825675 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f6effd640 1 -- 172.21.15.115:0/1309825675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f700ce1b0 msgr2=0x7f8f7015c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f667fc640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f700ce1b0 0x7f8f7015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f6f7fe640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f7015ac80 0x7f8f7015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f6effd640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f700ce1b0 0x7f8f7015c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f6effd640 1 -- 172.21.15.115:0/1309825675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f7015ac80 msgr2=0x7f8f7015c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f6effd640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f7015ac80 0x7f8f7015c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f6effd640 1 -- 172.21.15.115:0/1309825675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f580094d0 con 0x7f8f700b0e60 2023-12-01T01:36:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.696+0000 7f8f667fc640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f700ce1b0 0x7f8f7015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.697+0000 7f8f6effd640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700b0e60 0x7f8f70153a90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000b3f0 tx=0x7f8f6000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.697+0000 7f8f67fff640 1 -- 172.21.15.115:0/1309825675 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60011830 con 0x7f8f700b0e60 2023-12-01T01:36:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.697+0000 7f8f67fff640 1 -- 172.21.15.115:0/1309825675 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f60011990 con 0x7f8f700b0e60 2023-12-01T01:36:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.697+0000 7f8f67fff640 1 -- 172.21.15.115:0/1309825675 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60011d40 con 0x7f8f700b0e60 2023-12-01T01:36:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.697+0000 7f8f75789640 1 -- 172.21.15.115:0/1309825675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f70158a00 con 0x7f8f700b0e60 2023-12-01T01:36:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.697+0000 7f8f75789640 1 -- 172.21.15.115:0/1309825675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7013ccf0 con 0x7f8f700b0e60 2023-12-01T01:36:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.698+0000 7f8f75789640 1 -- 172.21.15.115:0/1309825675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f70158860 con 0x7f8f700b0e60 2023-12-01T01:36:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.699+0000 7f8f67fff640 1 -- 172.21.15.115:0/1309825675 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8f6001a400 con 0x7f8f700b0e60 2023-12-01T01:36:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.700+0000 7f8f67fff640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f5006a430 0x7f8f5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:52.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.700+0000 7f8f67fff640 1 -- 172.21.15.115:0/1309825675 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8f60088800 con 0x7f8f700b0e60 2023-12-01T01:36:52.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.700+0000 7f8f667fc640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f5006a430 0x7f8f5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:52.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.701+0000 7f8f667fc640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f5006a430 0x7f8f5006c8f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f58036000 tx=0x7f8f5803b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.705+0000 7f8f67fff640 1 -- 172.21.15.115:0/1309825675 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8f70158860 con 0x7f8f700b0e60 2023-12-01T01:36:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.896+0000 7f8f75789640 1 -- 172.21.15.115:0/1309825675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f70158860 con 0x7f8f700b0e60 2023-12-01T01:36:52.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.897+0000 7f8f67fff640 1 -- 172.21.15.115:0/1309825675 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f70158860 con 0x7f8f700b0e60 2023-12-01T01:36:52.898 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:36:52.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.899+0000 7f8f75789640 1 -- 172.21.15.115:0/1309825675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f5006a430 msgr2=0x7f8f5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.899+0000 7f8f75789640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f5006a430 0x7f8f5006c8f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f58036000 tx=0x7f8f5803b040 comp rx=0 tx=0).stop 2023-12-01T01:36:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.899+0000 7f8f75789640 1 -- 172.21.15.115:0/1309825675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700b0e60 msgr2=0x7f8f70153a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.899+0000 7f8f75789640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700b0e60 0x7f8f70153a90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000b3f0 tx=0x7f8f6000b8b0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.900+0000 7f8f75789640 1 -- 172.21.15.115:0/1309825675 shutdown_connections 2023-12-01T01:36:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.900+0000 7f8f75789640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f5006a430 0x7f8f5006c8f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.900+0000 7f8f75789640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f700ce1b0 0x7f8f7015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.900+0000 7f8f75789640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f7015ac80 0x7f8f7015c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.900+0000 7f8f75789640 1 --2- 172.21.15.115:0/1309825675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f700b0e60 0x7f8f70153a90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.900+0000 7f8f75789640 1 -- 172.21.15.115:0/1309825675 >> 172.21.15.115:0/1309825675 conn(0x7f8f70065960 msgr2=0x7f8f700b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.900+0000 7f8f75789640 1 -- 172.21.15.115:0/1309825675 shutdown_connections 2023-12-01T01:36:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:52.900+0000 7f8f75789640 1 -- 172.21.15.115:0/1309825675 wait complete. 2023-12-01T01:36:52.919 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:36:53.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.129+0000 7f5dc2920640 1 Processor -- start 2023-12-01T01:36:53.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.129+0000 7f5dc2920640 1 -- start start 2023-12-01T01:36:53.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.131+0000 7f5dc2920640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc0ce1b0 0x7f5dbc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.131+0000 7f5dc2920640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dbc0cc9a0 con 0x7f5dbc0b0e60 2023-12-01T01:36:53.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.131+0000 7f5dc2920640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dbc0cbd90 con 0x7f5dbc15ac80 2023-12-01T01:36:53.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.131+0000 7f5dc2920640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc15b060 con 0x7f5dbc0ce1b0 2023-12-01T01:36:53.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.131+0000 7f5dc0ebd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5dbc0b0e60 0x7f5dbc06b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39754/0 (socket says 172.21.15.115:39754) 2023-12-01T01:36:53.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.131+0000 7f5dc0ebd640 1 -- 172.21.15.115:0/3476661050 learned_addr learned my addr 172.21.15.115:0/3476661050 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:53.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.131+0000 7f5dbb7fe640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc0ce1b0 0x7f5dbc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.131+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253573139 0 0) 0x7f5dbc0cc9a0 con 0x7f5dbc0b0e60 2023-12-01T01:36:53.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5da4003520 con 0x7f5dbc0b0e60 2023-12-01T01:36:53.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3766546952 0 0) 0x7f5da4003520 con 0x7f5dbc0b0e60 2023-12-01T01:36:53.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc0ce1b0 msgr2=0x7f5dbc0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc0ce1b0 0x7f5dbc0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 >> v1:172.21.15.142:6789/0 conn(0x7f5dbc15ac80 legacy=0x7f5dbc0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0cc070 con 0x7f5dbc0b0e60 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5db40019f0 con 0x7f5dbc0b0e60 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 >> v1:172.21.15.115:6789/0 conn(0x7f5dbc0b0e60 legacy=0x7f5dbc06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da40039a0 0x7f5da4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 0x7f5da4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc0ce1b0 0x7f5da4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc15b060 con 0x7f5da40047c0 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da400ba30 con 0x7f5da40039a0 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da400bbf0 con 0x7f5dbc0ce1b0 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.132+0000 7f5dbb7fe640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 0x7f5da4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.133+0000 7f5dbbfff640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da40039a0 0x7f5da4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.133+0000 7f5dbb7fe640 1 -- 172.21.15.115:0/3476661050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc0ce1b0 msgr2=0x7f5da4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:53.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.133+0000 7f5dbb7fe640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc0ce1b0 0x7f5da4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.133+0000 7f5dbb7fe640 1 -- 172.21.15.115:0/3476661050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da40039a0 msgr2=0x7f5da4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:53.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.133+0000 7f5dbb7fe640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da40039a0 0x7f5da4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.133+0000 7f5dbb7fe640 1 -- 172.21.15.115:0/3476661050 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da4003520 con 0x7f5da40047c0 2023-12-01T01:36:53.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.134+0000 7f5dbb7fe640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 0x7f5da4004c40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009a90 tx=0x7f5da40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbc03eb045ac20e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:53.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.134+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db002c6b0 con 0x7f5da40047c0 2023-12-01T01:36:53.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.134+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db002c810 con 0x7f5da40047c0 2023-12-01T01:36:53.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.134+0000 7f5dbaffd640 1 -- 172.21.15.115:0/3476661050 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db002cba0 con 0x7f5da40047c0 2023-12-01T01:36:53.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.135+0000 7f5dc2920640 1 -- 172.21.15.115:0/3476661050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 msgr2=0x7f5da4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.135+0000 7f5dc2920640 1 --2- 172.21.15.115:0/3476661050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 0x7f5da4004c40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009a90 tx=0x7f5da40057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.135+0000 7f5dc0ebd640 1 -- 172.21.15.115:0/3476661050 reap_dead start 2023-12-01T01:36:53.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.135+0000 7f5dc2920640 1 -- 172.21.15.115:0/3476661050 shutdown_connections 2023-12-01T01:36:53.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.135+0000 7f5dc2920640 1 -- 172.21.15.115:0/3476661050 >> 172.21.15.115:0/3476661050 conn(0x7f5dbc065960 msgr2=0x7f5dbc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:53.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.135+0000 7f5dc2920640 1 -- 172.21.15.115:0/3476661050 shutdown_connections 2023-12-01T01:36:53.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.135+0000 7f5dc2920640 1 -- 172.21.15.115:0/3476661050 wait complete. 2023-12-01T01:36:53.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.137+0000 7f5dc2920640 1 Processor -- start 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.137+0000 7f5dc2920640 1 -- start start 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.138+0000 7f5dc2920640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 0x7f5dbc0c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.138+0000 7f5dc2920640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc070110 0x7f5dbc14be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.138+0000 7f5dc2920640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dbc070590 0x7f5dbc14c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.138+0000 7f5dc2920640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0c8210 con 0x7f5da40047c0 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.138+0000 7f5dc2920640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc14c8f0 con 0x7f5dbc070590 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.138+0000 7f5dc2920640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0c7590 con 0x7f5dbc070110 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.138+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 0x7f5dbc0c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.138+0000 7f5dbbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 0x7f5dbc0c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46070/0 (socket says 172.21.15.115:46070) 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.138+0000 7f5dbbfff640 1 -- 172.21.15.115:0/3920770930 learned_addr learned my addr 172.21.15.115:0/3920770930 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5dbb7fe640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc070110 0x7f5dbc14be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5dbbfff640 1 -- 172.21.15.115:0/3920770930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc070110 msgr2=0x7f5dbc14be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5dbbfff640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc070110 0x7f5dbc14be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5dbbfff640 1 -- 172.21.15.115:0/3920770930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dbc070590 msgr2=0x7f5dbc14c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5dbbfff640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dbc070590 0x7f5dbc14c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5dbbfff640 1 -- 172.21.15.115:0/3920770930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0c7870 con 0x7f5da40047c0 2023-12-01T01:36:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5dbbfff640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 0x7f5dbc0c8630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00e580 tx=0x7f5dac00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5db8ff9640 1 -- 172.21.15.115:0/3920770930 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0052e0 con 0x7f5da40047c0 2023-12-01T01:36:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5db8ff9640 1 -- 172.21.15.115:0/3920770930 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac005440 con 0x7f5da40047c0 2023-12-01T01:36:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5db8ff9640 1 -- 172.21.15.115:0/3920770930 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac003810 con 0x7f5da40047c0 2023-12-01T01:36:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5dc2920640 1 -- 172.21.15.115:0/3920770930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc0c7b50 con 0x7f5da40047c0 2023-12-01T01:36:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.139+0000 7f5dc2920640 1 -- 172.21.15.115:0/3920770930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc0c6860 con 0x7f5da40047c0 2023-12-01T01:36:53.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.141+0000 7f5dc2920640 1 -- 172.21.15.115:0/3920770930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dbc06cca0 con 0x7f5da40047c0 2023-12-01T01:36:53.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.141+0000 7f5db8ff9640 1 -- 172.21.15.115:0/3920770930 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5dac003970 con 0x7f5da40047c0 2023-12-01T01:36:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.142+0000 7f5db8ff9640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5da806a460 0x7f5da806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.143+0000 7f5db8ff9640 1 -- 172.21.15.115:0/3920770930 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5dac088700 con 0x7f5da40047c0 2023-12-01T01:36:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.143+0000 7f5dbb7fe640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5da806a460 0x7f5da806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.143+0000 7f5dbb7fe640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5da806a460 0x7f5da806c920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5db0036000 tx=0x7f5db0030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:53.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.146+0000 7f5db8ff9640 1 -- 172.21.15.115:0/3920770930 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5dac053c40 con 0x7f5da40047c0 2023-12-01T01:36:53.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.337+0000 7f5dc2920640 1 -- 172.21.15.115:0/3920770930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5dbc04e890 con 0x7f5da40047c0 2023-12-01T01:36:53.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.338+0000 7f5db8ff9640 1 -- 172.21.15.115:0/3920770930 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5dac057260 con 0x7f5da40047c0 2023-12-01T01:36:53.339 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:36:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.340+0000 7f5dc2920640 1 -- 172.21.15.115:0/3920770930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5da806a460 msgr2=0x7f5da806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.340+0000 7f5dc2920640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5da806a460 0x7f5da806c920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5db0036000 tx=0x7f5db0030000 comp rx=0 tx=0).stop 2023-12-01T01:36:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.340+0000 7f5dc2920640 1 -- 172.21.15.115:0/3920770930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 msgr2=0x7f5dbc0c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.340+0000 7f5dc2920640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 0x7f5dbc0c8630 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00e580 tx=0x7f5dac00ea40 comp rx=0 tx=0).stop 2023-12-01T01:36:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.340+0000 7f5dc2920640 1 -- 172.21.15.115:0/3920770930 shutdown_connections 2023-12-01T01:36:53.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.340+0000 7f5dc2920640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5da806a460 0x7f5da806c920 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.340+0000 7f5dc2920640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5dbc070110 0x7f5dbc14be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.340+0000 7f5dc2920640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5dbc070590 0x7f5dbc14c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.340+0000 7f5dc2920640 1 --2- 172.21.15.115:0/3920770930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5da40047c0 0x7f5dbc0c8630 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.340+0000 7f5dc2920640 1 -- 172.21.15.115:0/3920770930 >> 172.21.15.115:0/3920770930 conn(0x7f5dbc065960 msgr2=0x7f5dbc0ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:53.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.341+0000 7f5dc2920640 1 -- 172.21.15.115:0/3920770930 shutdown_connections 2023-12-01T01:36:53.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.341+0000 7f5dc2920640 1 -- 172.21.15.115:0/3920770930 wait complete. 2023-12-01T01:36:53.361 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:53.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.569+0000 7fd24a9f0640 1 Processor -- start 2023-12-01T01:36:53.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.569+0000 7fd24a9f0640 1 -- start start 2023-12-01T01:36:53.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.570+0000 7fd24a9f0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd24406f2f0 0x7fd24406bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.571+0000 7fd24a9f0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2440add10 con 0x7fd24415e2f0 2023-12-01T01:36:53.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.571+0000 7fd24a9f0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2440c5760 con 0x7fd2440cad70 2023-12-01T01:36:53.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.571+0000 7fd24a9f0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24415e6d0 con 0x7fd24406f2f0 2023-12-01T01:36:53.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.571+0000 7fd243fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd24415e2f0 0x7fd2440ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39768/0 (socket says 172.21.15.115:39768) 2023-12-01T01:36:53.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.571+0000 7fd243fff640 1 -- 172.21.15.115:0/1790955704 learned_addr learned my addr 172.21.15.115:0/1790955704 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.571+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983772068 0 0) 0x7fd2440add10 con 0x7fd24415e2f0 2023-12-01T01:36:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.571+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c003520 con 0x7fd24415e2f0 2023-12-01T01:36:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.571+0000 7fd248f8d640 1 --2- 172.21.15.115:0/1790955704 >> v2:172.21.15.142:3300/0 conn(0x7fd24406f2f0 0x7fd24406bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2161100260 0 0) 0x7fd22c003520 con 0x7fd24415e2f0 2023-12-01T01:36:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 >> v2:172.21.15.142:3300/0 conn(0x7fd24406f2f0 msgr2=0x7fd24406bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:53.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 --2- 172.21.15.115:0/1790955704 >> v2:172.21.15.142:3300/0 conn(0x7fd24406f2f0 0x7fd24406bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 >> v1:172.21.15.142:6789/0 conn(0x7fd2440cad70 legacy=0x7fd24406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440cf7b0 con 0x7fd24415e2f0 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd234001690 con 0x7fd24415e2f0 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 >> v1:172.21.15.115:6789/0 conn(0x7fd24415e2f0 legacy=0x7fd2440ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd22c003a60 0x7fd22c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd22c0047c0 0x7fd22c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd22c0053b0 0x7fd22c00ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24415e6d0 con 0x7fd22c003a60 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c00c300 con 0x7fd22c0047c0 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c00c4c0 con 0x7fd22c0053b0 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd243fff640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd22c003a60 0x7fd22c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd243fff640 1 -- 172.21.15.115:0/1790955704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd22c0053b0 msgr2=0x7fd22c00ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd243fff640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd22c0053b0 0x7fd22c00ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.572+0000 7fd2437fe640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd22c0047c0 0x7fd22c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.573+0000 7fd243fff640 1 -- 172.21.15.115:0/1790955704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd22c0047c0 msgr2=0x7fd22c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:53.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.573+0000 7fd243fff640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd22c0047c0 0x7fd22c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.573+0000 7fd243fff640 1 -- 172.21.15.115:0/1790955704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c003520 con 0x7fd22c003a60 2023-12-01T01:36:53.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.573+0000 7fd248f8d640 1 -- 172.21.15.115:0/1790955704 reap_dead start 2023-12-01T01:36:53.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.573+0000 7fd2437fe640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd22c0047c0 0x7fd22c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.574+0000 7fd243fff640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd22c003a60 0x7fd22c003e80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c160 tx=0x7fd22c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0cc5b810e0b8bf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.574+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234005ad0 con 0x7fd22c003a60 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.574+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd234005c30 con 0x7fd22c003a60 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.574+0000 7fd242ffd640 1 -- 172.21.15.115:0/1790955704 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23402c7c0 con 0x7fd22c003a60 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.574+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1790955704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd22c003a60 msgr2=0x7fd22c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.574+0000 7fd24a9f0640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd22c003a60 0x7fd22c003e80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c160 tx=0x7fd22c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.574+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1790955704 shutdown_connections 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.575+0000 7fd24a9f0640 1 --2- 172.21.15.115:0/1790955704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd22c003a60 0x7fd22c003e80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.575+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1790955704 >> 172.21.15.115:0/1790955704 conn(0x7fd2440ce880 msgr2=0x7fd2440cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.575+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1790955704 shutdown_connections 2023-12-01T01:36:53.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.575+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1790955704 wait complete. 2023-12-01T01:36:53.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.576+0000 7fd24a9f0640 1 Processor -- start 2023-12-01T01:36:53.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.577+0000 7fd24a9f0640 1 -- start start 2023-12-01T01:36:53.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.577+0000 7fd24a9f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd22c003a60 0x7fd2440c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.577+0000 7fd24a9f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd24414fc10 0x7fd2440cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd24a9f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd244150070 0x7fd2440cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd24a9f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440c6470 con 0x7fd24414fc10 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd24a9f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440c6750 con 0x7fd244150070 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd24a9f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440c3610 con 0x7fd22c003a60 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd24414fc10 0x7fd2440cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd24414fc10 0x7fd2440cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46088/0 (socket says 172.21.15.115:46088) 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd2437fe640 1 -- 172.21.15.115:0/1054707805 learned_addr learned my addr 172.21.15.115:0/1054707805 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd243fff640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd22c003a60 0x7fd2440c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd2437fe640 1 -- 172.21.15.115:0/1054707805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd22c003a60 msgr2=0x7fd2440c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd2437fe640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd22c003a60 0x7fd2440c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd2437fe640 1 -- 172.21.15.115:0/1054707805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd244150070 msgr2=0x7fd2440cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd248f8d640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd244150070 0x7fd2440cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd2437fe640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd244150070 0x7fd2440cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd2437fe640 1 -- 172.21.15.115:0/1054707805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440c38f0 con 0x7fd24414fc10 2023-12-01T01:36:53.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd243fff640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd22c003a60 0x7fd2440c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:53.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.578+0000 7fd248f8d640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd244150070 0x7fd2440cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:53.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.579+0000 7fd2437fe640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd24414fc10 0x7fd2440cd4a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd23800d480 tx=0x7fd23800d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:53.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.579+0000 7fd240ff9640 1 -- 172.21.15.115:0/1054707805 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238005480 con 0x7fd24414fc10 2023-12-01T01:36:53.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.579+0000 7fd240ff9640 1 -- 172.21.15.115:0/1054707805 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd238003610 con 0x7fd24414fc10 2023-12-01T01:36:53.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.579+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1054707805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2440c5e90 con 0x7fd24414fc10 2023-12-01T01:36:53.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.579+0000 7fd240ff9640 1 -- 172.21.15.115:0/1054707805 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238003a00 con 0x7fd24414fc10 2023-12-01T01:36:53.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.579+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1054707805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24415acb0 con 0x7fd24414fc10 2023-12-01T01:36:53.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.580+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1054707805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24406fda0 con 0x7fd24414fc10 2023-12-01T01:36:53.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.581+0000 7fd240ff9640 1 -- 172.21.15.115:0/1054707805 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd238003b60 con 0x7fd24414fc10 2023-12-01T01:36:53.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.582+0000 7fd240ff9640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd23006a430 0x7fd23006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:53.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.582+0000 7fd240ff9640 1 -- 172.21.15.115:0/1054707805 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd238088810 con 0x7fd24414fc10 2023-12-01T01:36:53.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.582+0000 7fd243fff640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd23006a430 0x7fd23006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:53.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.583+0000 7fd243fff640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd23006a430 0x7fd23006c8f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd234034000 tx=0x7fd234009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:53.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.586+0000 7fd240ff9640 1 -- 172.21.15.115:0/1054707805 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd238053e90 con 0x7fd24414fc10 2023-12-01T01:36:53.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.783+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1054707805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2440c3a80 con 0x7fd24414fc10 2023-12-01T01:36:53.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.785+0000 7fd240ff9640 1 -- 172.21.15.115:0/1054707805 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16138 (secure 0 0 0) 0x7fd2380574b0 con 0x7fd24414fc10 2023-12-01T01:36:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:53.787 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":23,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:52.262397+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:53.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.787+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1054707805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd23006a430 msgr2=0x7fd23006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.787+0000 7fd24a9f0640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd23006a430 0x7fd23006c8f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd234034000 tx=0x7fd234009030 comp rx=0 tx=0).stop 2023-12-01T01:36:53.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.787+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1054707805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd24414fc10 msgr2=0x7fd2440cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:53.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.787+0000 7fd24a9f0640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd24414fc10 0x7fd2440cd4a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd23800d480 tx=0x7fd23800d940 comp rx=0 tx=0).stop 2023-12-01T01:36:53.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.788+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1054707805 shutdown_connections 2023-12-01T01:36:53.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.788+0000 7fd24a9f0640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd23006a430 0x7fd23006c8f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.788+0000 7fd24a9f0640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd244150070 0x7fd2440cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.788+0000 7fd24a9f0640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd24414fc10 0x7fd2440cd4a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.788+0000 7fd24a9f0640 1 --2- 172.21.15.115:0/1054707805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd22c003a60 0x7fd2440c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:53.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.788+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1054707805 >> 172.21.15.115:0/1054707805 conn(0x7fd2440ce880 msgr2=0x7fd24415b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:53.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.788+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1054707805 shutdown_connections 2023-12-01T01:36:53.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:53.788+0000 7fd24a9f0640 1 -- 172.21.15.115:0/1054707805 wait complete. 2023-12-01T01:36:53.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:36:53.808 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:36:54.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.012+0000 7f3231dfb640 1 Processor -- start 2023-12-01T01:36:54.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.012+0000 7f3231dfb640 1 -- start start 2023-12-01T01:36:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.013+0000 7f3231dfb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f322c15b0e0 0x7f322c164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.013+0000 7f3231dfb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c0b2f30 con 0x7f322c142870 2023-12-01T01:36:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.013+0000 7f3231dfb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c0b2320 con 0x7f322c0cd1e0 2023-12-01T01:36:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.013+0000 7f3231dfb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0b3ae0 con 0x7f322c15b0e0 2023-12-01T01:36:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.014+0000 7f322b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f322c142870 0x7f322c0ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39782/0 (socket says 172.21.15.115:39782) 2023-12-01T01:36:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.014+0000 7f322b7fe640 1 -- 172.21.15.115:0/2181446083 learned_addr learned my addr 172.21.15.115:0/2181446083 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:54.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.014+0000 7f322bfff640 1 --2- 172.21.15.115:0/2181446083 >> v2:172.21.15.115:3300/0 conn(0x7f322c15b0e0 0x7f322c164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.014+0000 7f322bfff640 1 -- 172.21.15.115:0/2181446083 >> v1:172.21.15.142:6789/0 conn(0x7f322c0cd1e0 legacy=0x7f322c0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:54.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.014+0000 7f322affd640 1 -- 172.21.15.115:0/2181446083 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384488208 0 0) 0x7f322c0b2f30 con 0x7f322c142870 2023-12-01T01:36:54.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.014+0000 7f322bfff640 1 -- 172.21.15.115:0/2181446083 >> v1:172.21.15.115:6789/0 conn(0x7f322c142870 legacy=0x7f322c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.014+0000 7f322bfff640 1 -- 172.21.15.115:0/2181446083 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0b2600 con 0x7f322c15b0e0 2023-12-01T01:36:54.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.015+0000 7f322bfff640 1 --2- 172.21.15.115:0/2181446083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c15b0e0 0x7f322c164a20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3224009d90 tx=0x7f322402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6502ed7db2618eb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:54.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.016+0000 7f322affd640 1 -- 172.21.15.115:0/2181446083 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322402ee70 con 0x7f322c15b0e0 2023-12-01T01:36:54.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.016+0000 7f322affd640 1 -- 172.21.15.115:0/2181446083 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3224036440 con 0x7f322c15b0e0 2023-12-01T01:36:54.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.016+0000 7f322affd640 1 -- 172.21.15.115:0/2181446083 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32240367d0 con 0x7f322c15b0e0 2023-12-01T01:36:54.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.016+0000 7f3231dfb640 1 -- 172.21.15.115:0/2181446083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c15b0e0 msgr2=0x7f322c164a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.016+0000 7f3231dfb640 1 --2- 172.21.15.115:0/2181446083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c15b0e0 0x7f322c164a20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3224009d90 tx=0x7f322402e0c0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.016+0000 7f3231dfb640 1 -- 172.21.15.115:0/2181446083 shutdown_connections 2023-12-01T01:36:54.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.016+0000 7f3231dfb640 1 --2- 172.21.15.115:0/2181446083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c15b0e0 0x7f322c164a20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.016+0000 7f3231dfb640 1 -- 172.21.15.115:0/2181446083 >> 172.21.15.115:0/2181446083 conn(0x7f322c065960 msgr2=0x7f322c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:54.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.016+0000 7f3231dfb640 1 -- 172.21.15.115:0/2181446083 shutdown_connections 2023-12-01T01:36:54.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.016+0000 7f3231dfb640 1 -- 172.21.15.115:0/2181446083 wait complete. 2023-12-01T01:36:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.018+0000 7f3231dfb640 1 Processor -- start 2023-12-01T01:36:54.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.018+0000 7f3231dfb640 1 -- start start 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.019+0000 7f3231dfb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c0cd1e0 0x7f322c0a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.019+0000 7f3231dfb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f322c142870 0x7f322c0aa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.019+0000 7f3231dfb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f322c1f2760 0x7f322c0b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.019+0000 7f3231dfb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0ca100 con 0x7f322c0cd1e0 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.019+0000 7f3231dfb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0aa810 con 0x7f322c1f2760 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.019+0000 7f3231dfb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1f2cd0 con 0x7f322c142870 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.019+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c0cd1e0 0x7f322c0a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.019+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c0cd1e0 0x7f322c0a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46114/0 (socket says 172.21.15.115:46114) 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.019+0000 7f322b7fe640 1 -- 172.21.15.115:0/181783756 learned_addr learned my addr 172.21.15.115:0/181783756 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f322bfff640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f322c1f2760 0x7f322c0b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f3223fff640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f322c142870 0x7f322c0aa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f322b7fe640 1 -- 172.21.15.115:0/181783756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f322c142870 msgr2=0x7f322c0aa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f322b7fe640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f322c142870 0x7f322c0aa2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f322b7fe640 1 -- 172.21.15.115:0/181783756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f322c1f2760 msgr2=0x7f322c0b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f322b7fe640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f322c1f2760 0x7f322c0b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f322b7fe640 1 -- 172.21.15.115:0/181783756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3224009a70 con 0x7f322c0cd1e0 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f322bfff640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f322c1f2760 0x7f322c0b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f3223fff640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f322c142870 0x7f322c0aa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f322b7fe640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c0cd1e0 0x7f322c0a8480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f321c00c4d0 tx=0x7f321c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f3228ff9640 1 -- 172.21.15.115:0/181783756 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0074e0 con 0x7f322c0cd1e0 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f3228ff9640 1 -- 172.21.15.115:0/181783756 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c007660 con 0x7f322c0cd1e0 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f3228ff9640 1 -- 172.21.15.115:0/181783756 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c004a20 con 0x7f322c0cd1e0 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f3231dfb640 1 -- 172.21.15.115:0/181783756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c1f2fb0 con 0x7f322c0cd1e0 2023-12-01T01:36:54.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.020+0000 7f3231dfb640 1 -- 172.21.15.115:0/181783756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c1f3570 con 0x7f322c0cd1e0 2023-12-01T01:36:54.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.022+0000 7f3231dfb640 1 -- 172.21.15.115:0/181783756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c04e890 con 0x7f322c0cd1e0 2023-12-01T01:36:54.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.022+0000 7f3228ff9640 1 -- 172.21.15.115:0/181783756 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f321c004b80 con 0x7f322c0cd1e0 2023-12-01T01:36:54.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.023+0000 7f3228ff9640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f320c06a430 0x7f320c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.023+0000 7f3228ff9640 1 -- 172.21.15.115:0/181783756 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f321c087b10 con 0x7f322c0cd1e0 2023-12-01T01:36:54.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.024+0000 7f3223fff640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f320c06a430 0x7f320c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.024+0000 7f3223fff640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f320c06a430 0x7f320c06c8f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3214007fd0 tx=0x7f3214009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:54.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.027+0000 7f3228ff9640 1 -- 172.21.15.115:0/181783756 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f321c0530e0 con 0x7f322c0cd1e0 2023-12-01T01:36:54.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.205+0000 7f3231dfb640 1 -- 172.21.15.115:0/181783756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f322c06c0e0 con 0x7f322c0cd1e0 2023-12-01T01:36:54.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.206+0000 7f3228ff9640 1 -- 172.21.15.115:0/181783756 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f321c018020 con 0x7f322c0cd1e0 2023-12-01T01:36:54.208 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:36:54.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 -- 172.21.15.115:0/181783756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f320c06a430 msgr2=0x7f320c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f320c06a430 0x7f320c06c8f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3214007fd0 tx=0x7f3214009040 comp rx=0 tx=0).stop 2023-12-01T01:36:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 -- 172.21.15.115:0/181783756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c0cd1e0 msgr2=0x7f322c0a8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c0cd1e0 0x7f322c0a8480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f321c00c4d0 tx=0x7f321c00c990 comp rx=0 tx=0).stop 2023-12-01T01:36:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 -- 172.21.15.115:0/181783756 shutdown_connections 2023-12-01T01:36:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f320c06a430 0x7f320c06c8f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f322c142870 0x7f322c0aa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f322c1f2760 0x7f322c0b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 --2- 172.21.15.115:0/181783756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f322c0cd1e0 0x7f322c0a8480 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 -- 172.21.15.115:0/181783756 >> 172.21.15.115:0/181783756 conn(0x7f322c065960 msgr2=0x7f322c141a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 -- 172.21.15.115:0/181783756 shutdown_connections 2023-12-01T01:36:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.209+0000 7f3231dfb640 1 -- 172.21.15.115:0/181783756 wait complete. 2023-12-01T01:36:54.227 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:36:54.257 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:36:54.279 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:36:54.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.427+0000 7f0f22a8c640 1 Processor -- start 2023-12-01T01:36:54.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.427+0000 7f0f22a8c640 1 -- start start 2023-12-01T01:36:54.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.428+0000 7f0f22a8c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0f1c15ac00 0x7f0f1c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.428+0000 7f0f22a8c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c15afe0 con 0x7f0f1c0aede0 2023-12-01T01:36:54.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.428+0000 7f0f22a8c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1c0b0a10 con 0x7f0f1c0cb800 2023-12-01T01:36:54.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.428+0000 7f0f22a8c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1c0b12b0 con 0x7f0f1c15ac00 2023-12-01T01:36:54.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.429+0000 7f0f21289640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0f1c0aede0 0x7f0f1c0cb0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39796/0 (socket says 172.21.15.115:39796) 2023-12-01T01:36:54.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.429+0000 7f0f21289640 1 -- 172.21.15.115:0/3007704094 learned_addr learned my addr 172.21.15.115:0/3007704094 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:54.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.429+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056935748 0 0) 0x7f0f1c15afe0 con 0x7f0f1c0aede0 2023-12-01T01:36:54.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.429+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f08003520 con 0x7f0f1c0aede0 2023-12-01T01:36:54.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.429+0000 7f0f21a8a640 1 --2- 172.21.15.115:0/3007704094 >> v2:172.21.15.142:3300/0 conn(0x7f0f1c15ac00 0x7f0f1c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2236817989 0 0) 0x7f0f08003520 con 0x7f0f1c0aede0 2023-12-01T01:36:54.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 >> v2:172.21.15.142:3300/0 conn(0x7f0f1c15ac00 msgr2=0x7f0f1c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:54.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 --2- 172.21.15.115:0/3007704094 >> v2:172.21.15.142:3300/0 conn(0x7f0f1c15ac00 0x7f0f1c0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 >> v1:172.21.15.142:6789/0 conn(0x7f0f1c0cb800 legacy=0x7f0f1c06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c0b0cf0 con 0x7f0f1c0aede0 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0f10001690 con 0x7f0f1c0aede0 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 >> v1:172.21.15.115:6789/0 conn(0x7f0f1c0aede0 legacy=0x7f0f1c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 --2- 172.21.15.115:0/3007704094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f08003970 0x7f0f08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 --2- 172.21.15.115:0/3007704094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 --2- 172.21.15.115:0/3007704094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f080052a0 0x7f0f0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c0b12b0 con 0x7f0f080047c0 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0800c170 con 0x7f0f08003970 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0800c330 con 0x7f0f080052a0 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f21289640 1 --2- 172.21.15.115:0/3007704094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f21289640 1 -- 172.21.15.115:0/3007704094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f080052a0 msgr2=0x7f0f0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:54.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f21289640 1 --2- 172.21.15.115:0/3007704094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f080052a0 0x7f0f0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f21289640 1 -- 172.21.15.115:0/3007704094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f08003970 msgr2=0x7f0f08003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:54.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.430+0000 7f0f21289640 1 --2- 172.21.15.115:0/3007704094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f08003970 0x7f0f08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.431+0000 7f0f21289640 1 -- 172.21.15.115:0/3007704094 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f08003520 con 0x7f0f080047c0 2023-12-01T01:36:54.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.431+0000 7f0f2228b640 1 -- 172.21.15.115:0/3007704094 reap_dead start 2023-12-01T01:36:54.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.432+0000 7f0f21289640 1 --2- 172.21.15.115:0/3007704094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f08004c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0f10001eb0 tx=0x7f0f080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3bf2ff22b01aa4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:54.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.432+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10005ad0 con 0x7f0f080047c0 2023-12-01T01:36:54.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.432+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f10005c30 con 0x7f0f080047c0 2023-12-01T01:36:54.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.432+0000 7f0f20a88640 1 -- 172.21.15.115:0/3007704094 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1002c870 con 0x7f0f080047c0 2023-12-01T01:36:54.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.432+0000 7f0f22a8c640 1 -- 172.21.15.115:0/3007704094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 msgr2=0x7f0f08004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.432+0000 7f0f22a8c640 1 --2- 172.21.15.115:0/3007704094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f08004c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0f10001eb0 tx=0x7f0f080057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.433+0000 7f0f22a8c640 1 -- 172.21.15.115:0/3007704094 shutdown_connections 2023-12-01T01:36:54.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.433+0000 7f0f22a8c640 1 --2- 172.21.15.115:0/3007704094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f08004c40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.433+0000 7f0f22a8c640 1 -- 172.21.15.115:0/3007704094 >> 172.21.15.115:0/3007704094 conn(0x7f0f1c065960 msgr2=0x7f0f1c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:54.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.433+0000 7f0f22a8c640 1 -- 172.21.15.115:0/3007704094 shutdown_connections 2023-12-01T01:36:54.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.433+0000 7f0f22a8c640 1 -- 172.21.15.115:0/3007704094 wait complete. 2023-12-01T01:36:54.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.434+0000 7f0f22a8c640 1 Processor -- start 2023-12-01T01:36:54.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.435+0000 7f0f22a8c640 1 -- start start 2023-12-01T01:36:54.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.435+0000 7f0f22a8c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f1c147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.435+0000 7f0f22a8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f1c0c4c60 0x7f0f1c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.435+0000 7f0f22a8c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f1c155a80 0x7f0f1c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.435+0000 7f0f22a8c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c15ca20 con 0x7f0f080047c0 2023-12-01T01:36:54.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.435+0000 7f0f22a8c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c156440 con 0x7f0f1c155a80 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.435+0000 7f0f22a8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c15c590 con 0x7f0f1c0c4c60 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.436+0000 7f0f21a8a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f1c147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.436+0000 7f0f21a8a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f1c147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46118/0 (socket says 172.21.15.115:46118) 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.436+0000 7f0f21a8a640 1 -- 172.21.15.115:0/1687876969 learned_addr learned my addr 172.21.15.115:0/1687876969 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.436+0000 7f0f21a8a640 1 -- 172.21.15.115:0/1687876969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f1c0c4c60 msgr2=0x7f0f1c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.436+0000 7f0f21a8a640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f1c0c4c60 0x7f0f1c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.436+0000 7f0f21a8a640 1 -- 172.21.15.115:0/1687876969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f1c155a80 msgr2=0x7f0f1c155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.436+0000 7f0f21a8a640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f1c155a80 0x7f0f1c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.436+0000 7f0f21a8a640 1 -- 172.21.15.115:0/1687876969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c0c50e0 con 0x7f0f080047c0 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.436+0000 7f0f21a8a640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f1c147c80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800d830 tx=0x7f0f1800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.436+0000 7f0f0e7fc640 1 -- 172.21.15.115:0/1687876969 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1800b6c0 con 0x7f0f080047c0 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.437+0000 7f0f0e7fc640 1 -- 172.21.15.115:0/1687876969 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1800b820 con 0x7f0f080047c0 2023-12-01T01:36:54.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.437+0000 7f0f0e7fc640 1 -- 172.21.15.115:0/1687876969 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1800bbd0 con 0x7f0f080047c0 2023-12-01T01:36:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.437+0000 7f0f22a8c640 1 -- 172.21.15.115:0/1687876969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1c15b6e0 con 0x7f0f080047c0 2023-12-01T01:36:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.437+0000 7f0f22a8c640 1 -- 172.21.15.115:0/1687876969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1c15bc60 con 0x7f0f080047c0 2023-12-01T01:36:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.438+0000 7f0f22a8c640 1 -- 172.21.15.115:0/1687876969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1c0618d0 con 0x7f0f080047c0 2023-12-01T01:36:54.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.439+0000 7f0f0e7fc640 1 -- 172.21.15.115:0/1687876969 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0f1801e460 con 0x7f0f080047c0 2023-12-01T01:36:54.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.439+0000 7f0f0e7fc640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0f0006a430 0x7f0f0006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.440+0000 7f0f0e7fc640 1 -- 172.21.15.115:0/1687876969 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0f180890f0 con 0x7f0f080047c0 2023-12-01T01:36:54.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.440+0000 7f0f21289640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0f0006a430 0x7f0f0006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.440+0000 7f0f21289640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0f0006a430 0x7f0f0006c8f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f10034000 tx=0x7f0f10039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:54.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.444+0000 7f0f0e7fc640 1 -- 172.21.15.115:0/1687876969 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0f180546b0 con 0x7f0f080047c0 2023-12-01T01:36:54.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.631+0000 7f0f22a8c640 1 -- 172.21.15.115:0/1687876969 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f1c0afae0 con 0x7f0f080047c0 2023-12-01T01:36:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.632+0000 7f0f0e7fc640 1 -- 172.21.15.115:0/1687876969 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f18057cd0 con 0x7f0f080047c0 2023-12-01T01:36:54.634 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:36:54.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 -- 172.21.15.115:0/1687876969 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0f0006a430 msgr2=0x7f0f0006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0f0006a430 0x7f0f0006c8f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f10034000 tx=0x7f0f10039040 comp rx=0 tx=0).stop 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 -- 172.21.15.115:0/1687876969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 msgr2=0x7f0f1c147c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f1c147c80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800d830 tx=0x7f0f1800dcf0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 -- 172.21.15.115:0/1687876969 shutdown_connections 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0f0006a430 0x7f0f0006c8f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f1c0c4c60 0x7f0f1c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f1c155a80 0x7f0f1c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 --2- 172.21.15.115:0/1687876969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f080047c0 0x7f0f1c147c80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 -- 172.21.15.115:0/1687876969 >> 172.21.15.115:0/1687876969 conn(0x7f0f1c065960 msgr2=0x7f0f1c0cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 -- 172.21.15.115:0/1687876969 shutdown_connections 2023-12-01T01:36:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.635+0000 7f0f22a8c640 1 -- 172.21.15.115:0/1687876969 wait complete. 2023-12-01T01:36:54.654 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:36:54.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.869+0000 7ff4f4f47640 1 Processor -- start 2023-12-01T01:36:54.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.869+0000 7ff4f4f47640 1 -- start start 2023-12-01T01:36:54.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4f4f47640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff4f00ce1b0 0x7ff4f00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4f4f47640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff4f00b0e60 0x7ff4f00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4f4f47640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f00cc9a0 con 0x7ff4f015ac80 2023-12-01T01:36:54.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4f4f47640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f015b060 con 0x7ff4f00ce1b0 2023-12-01T01:36:54.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4f4f47640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f00b0ce0 con 0x7ff4f00b0e60 2023-12-01T01:36:54.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4edd9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff4f00ce1b0 0x7ff4f00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4edd9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff4f00ce1b0 0x7ff4f00cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46126/0 (socket says 172.21.15.115:46126) 2023-12-01T01:36:54.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4edd9b640 1 -- 172.21.15.115:0/1099992109 learned_addr learned my addr 172.21.15.115:0/1099992109 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:54.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4eed9d640 1 --2- 172.21.15.115:0/1099992109 >> v2:172.21.15.142:3300/0 conn(0x7ff4f00b0e60 0x7ff4f00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4edd9b640 1 -- 172.21.15.115:0/1099992109 >> v2:172.21.15.142:3300/0 conn(0x7ff4f00b0e60 msgr2=0x7ff4f00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:54.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4edd9b640 1 --2- 172.21.15.115:0/1099992109 >> v2:172.21.15.142:3300/0 conn(0x7ff4f00b0e60 0x7ff4f00b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4edd9b640 1 -- 172.21.15.115:0/1099992109 >> v1:172.21.15.142:6789/0 conn(0x7ff4f015ac80 legacy=0x7ff4f00cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:54.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4edd9b640 1 -- 172.21.15.115:0/1099992109 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00cbd90 con 0x7ff4f00ce1b0 2023-12-01T01:36:54.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.871+0000 7ff4eed9d640 1 --2- 172.21.15.115:0/1099992109 >> v2:172.21.15.142:3300/0 conn(0x7ff4f00b0e60 0x7ff4f00b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:54.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.872+0000 7ff4edd9b640 1 --2- 172.21.15.115:0/1099992109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f00ce1b0 0x7ff4f00cce30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00cd6e0 tx=0x7ff4e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a2bc12e74922df server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:54.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.873+0000 7ff4ed59a640 1 -- 172.21.15.115:0/1099992109 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4031870 con 0x7ff4f00ce1b0 2023-12-01T01:36:54.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.873+0000 7ff4ed59a640 1 -- 172.21.15.115:0/1099992109 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e40319d0 con 0x7ff4f00ce1b0 2023-12-01T01:36:54.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.873+0000 7ff4ed59a640 1 -- 172.21.15.115:0/1099992109 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4031d60 con 0x7ff4f00ce1b0 2023-12-01T01:36:54.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.873+0000 7ff4f4f47640 1 -- 172.21.15.115:0/1099992109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f00ce1b0 msgr2=0x7ff4f00cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.873+0000 7ff4f4f47640 1 --2- 172.21.15.115:0/1099992109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f00ce1b0 0x7ff4f00cce30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00cd6e0 tx=0x7ff4e402b910 comp rx=0 tx=0).stop 2023-12-01T01:36:54.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.874+0000 7ff4f4f47640 1 -- 172.21.15.115:0/1099992109 shutdown_connections 2023-12-01T01:36:54.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.874+0000 7ff4f4f47640 1 --2- 172.21.15.115:0/1099992109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f00ce1b0 0x7ff4f00cce30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.874+0000 7ff4f4f47640 1 --2- 172.21.15.115:0/1099992109 >> v2:172.21.15.142:3300/0 conn(0x7ff4f00b0e60 0x7ff4f00b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.874+0000 7ff4f4f47640 1 -- 172.21.15.115:0/1099992109 >> 172.21.15.115:0/1099992109 conn(0x7ff4f0065960 msgr2=0x7ff4f0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:54.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.874+0000 7ff4f4f47640 1 -- 172.21.15.115:0/1099992109 shutdown_connections 2023-12-01T01:36:54.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.874+0000 7ff4f4f47640 1 -- 172.21.15.115:0/1099992109 wait complete. 2023-12-01T01:36:54.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.875+0000 7ff4f4f47640 1 Processor -- start 2023-12-01T01:36:54.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.876+0000 7ff4f4f47640 1 -- start start 2023-12-01T01:36:54.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.876+0000 7ff4f4f47640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4f00b0e60 0x7ff4f00c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4f4f47640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4f00ce1b0 0x7ff4f0144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4f4f47640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f015ac80 0x7ff4f01453d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4f4f47640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00c9ee0 con 0x7ff4f015ac80 2023-12-01T01:36:54.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4f4f47640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00c6440 con 0x7ff4f00b0e60 2023-12-01T01:36:54.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4f4f47640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f0142b40 con 0x7ff4f00ce1b0 2023-12-01T01:36:54.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4eed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f015ac80 0x7ff4f01453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4eed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f015ac80 0x7ff4f01453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46128/0 (socket says 172.21.15.115:46128) 2023-12-01T01:36:54.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4eed9d640 1 -- 172.21.15.115:0/2432006358 learned_addr learned my addr 172.21.15.115:0/2432006358 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:54.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4ee59c640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4f00b0e60 0x7ff4f00c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4edd9b640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4f00ce1b0 0x7ff4f0144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4eed9d640 1 -- 172.21.15.115:0/2432006358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4f00ce1b0 msgr2=0x7ff4f0144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4eed9d640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4f00ce1b0 0x7ff4f0144e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4eed9d640 1 -- 172.21.15.115:0/2432006358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4f00b0e60 msgr2=0x7ff4f00c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:54.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4eed9d640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4f00b0e60 0x7ff4f00c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:54.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4eed9d640 1 -- 172.21.15.115:0/2432006358 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e40094d0 con 0x7ff4f015ac80 2023-12-01T01:36:54.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.877+0000 7ff4edd9b640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4f00ce1b0 0x7ff4f0144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:54.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.878+0000 7ff4eed9d640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f015ac80 0x7ff4f01453d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800cef0 tx=0x7ff4e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:54.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.878+0000 7ff4deffd640 1 -- 172.21.15.115:0/2432006358 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e800d840 con 0x7ff4f015ac80 2023-12-01T01:36:54.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.878+0000 7ff4deffd640 1 -- 172.21.15.115:0/2432006358 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e800d9c0 con 0x7ff4f015ac80 2023-12-01T01:36:54.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.878+0000 7ff4f4f47640 1 -- 172.21.15.115:0/2432006358 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f0156750 con 0x7ff4f015ac80 2023-12-01T01:36:54.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.878+0000 7ff4deffd640 1 -- 172.21.15.115:0/2432006358 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e800dd70 con 0x7ff4f015ac80 2023-12-01T01:36:54.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.878+0000 7ff4f4f47640 1 -- 172.21.15.115:0/2432006358 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f00c5b10 con 0x7ff4f015ac80 2023-12-01T01:36:54.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.879+0000 7ff4f4f47640 1 -- 172.21.15.115:0/2432006358 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f004e890 con 0x7ff4f015ac80 2023-12-01T01:36:54.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.880+0000 7ff4deffd640 1 -- 172.21.15.115:0/2432006358 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff4e801e3b0 con 0x7ff4f015ac80 2023-12-01T01:36:54.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.881+0000 7ff4deffd640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff4d006a430 0x7ff4d006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:54.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.881+0000 7ff4deffd640 1 -- 172.21.15.115:0/2432006358 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff4e80894a0 con 0x7ff4f015ac80 2023-12-01T01:36:54.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.881+0000 7ff4ee59c640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff4d006a430 0x7ff4d006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:54.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.882+0000 7ff4ee59c640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff4d006a430 0x7ff4d006c8f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0006fd0 tx=0x7ff4e0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:54.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:54.885+0000 7ff4deffd640 1 -- 172.21.15.115:0/2432006358 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff4e8054a90 con 0x7ff4f015ac80 2023-12-01T01:36:55.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.072+0000 7ff4f4f47640 1 -- 172.21.15.115:0/2432006358 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4f00c6580 con 0x7ff4f015ac80 2023-12-01T01:36:55.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.073+0000 7ff4deffd640 1 -- 172.21.15.115:0/2432006358 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4e801a020 con 0x7ff4f015ac80 2023-12-01T01:36:55.074 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:36:55.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.075+0000 7ff4f4f47640 1 -- 172.21.15.115:0/2432006358 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff4d006a430 msgr2=0x7ff4d006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:55.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.075+0000 7ff4f4f47640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff4d006a430 0x7ff4d006c8f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0006fd0 tx=0x7ff4e0008040 comp rx=0 tx=0).stop 2023-12-01T01:36:55.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.075+0000 7ff4f4f47640 1 -- 172.21.15.115:0/2432006358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f015ac80 msgr2=0x7ff4f01453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:55.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.075+0000 7ff4f4f47640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f015ac80 0x7ff4f01453d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800cef0 tx=0x7ff4e800a710 comp rx=0 tx=0).stop 2023-12-01T01:36:55.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.075+0000 7ff4f4f47640 1 -- 172.21.15.115:0/2432006358 shutdown_connections 2023-12-01T01:36:55.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.075+0000 7ff4f4f47640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff4d006a430 0x7ff4d006c8f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.075+0000 7ff4f4f47640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4f00ce1b0 0x7ff4f0144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.075+0000 7ff4f4f47640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4f015ac80 0x7ff4f01453d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.075+0000 7ff4f4f47640 1 --2- 172.21.15.115:0/2432006358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4f00b0e60 0x7ff4f00c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.075+0000 7ff4f4f47640 1 -- 172.21.15.115:0/2432006358 >> 172.21.15.115:0/2432006358 conn(0x7ff4f0065960 msgr2=0x7ff4f006b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:55.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.076+0000 7ff4f4f47640 1 -- 172.21.15.115:0/2432006358 shutdown_connections 2023-12-01T01:36:55.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.076+0000 7ff4f4f47640 1 -- 172.21.15.115:0/2432006358 wait complete. 2023-12-01T01:36:55.133 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-12-01T01:36:55.133 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-01T01:36:55.134 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-12-01T01:36:55.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.348+0000 7fc020339640 1 Processor -- start 2023-12-01T01:36:55.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.348+0000 7fc020339640 1 -- start start 2023-12-01T01:36:55.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.350+0000 7fc020339640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc0180cad70 0x7fc01806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:55.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.350+0000 7fc020339640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0180add10 con 0x7fc01806f2f0 2023-12-01T01:36:55.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.350+0000 7fc020339640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0180c5760 con 0x7fc01815e2f0 2023-12-01T01:36:55.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.350+0000 7fc020339640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01815e6d0 con 0x7fc0180cad70 2023-12-01T01:36:55.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.350+0000 7fc01e8d6640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc01806f2f0 0x7fc01806bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39812/0 (socket says 172.21.15.115:39812) 2023-12-01T01:36:55.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.350+0000 7fc01e8d6640 1 -- 172.21.15.115:0/3380081569 learned_addr learned my addr 172.21.15.115:0/3380081569 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:55.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d8d4640 1 --2- 172.21.15.115:0/3380081569 >> v2:172.21.15.142:3300/0 conn(0x7fc0180cad70 0x7fc01806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:55.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850273283 0 0) 0x7fc0180add10 con 0x7fc01806f2f0 2023-12-01T01:36:55.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc004003520 con 0x7fc01806f2f0 2023-12-01T01:36:55.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3927347157 0 0) 0x7fc004003520 con 0x7fc01806f2f0 2023-12-01T01:36:55.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 >> v2:172.21.15.142:3300/0 conn(0x7fc0180cad70 msgr2=0x7fc01806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:55.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 --2- 172.21.15.115:0/3380081569 >> v2:172.21.15.142:3300/0 conn(0x7fc0180cad70 0x7fc01806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 >> v1:172.21.15.142:6789/0 conn(0x7fc01815e2f0 legacy=0x7fc0180ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0180cf7b0 con 0x7fc01806f2f0 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc010001a10 con 0x7fc01806f2f0 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 >> v1:172.21.15.115:6789/0 conn(0x7fc01806f2f0 legacy=0x7fc01806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc004003a30 0x7fc004003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.351+0000 7fc01d0d3640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0040047c0 0x7fc004004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01d0d3640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040052d0 0x7fc00400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01815e6d0 con 0x7fc0040052d0 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00400c210 con 0x7fc004003a30 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00400c3d0 con 0x7fc0040047c0 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01e8d6640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040052d0 0x7fc00400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01e0d5640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc004003a30 0x7fc004003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01d8d4640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0040047c0 0x7fc004004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01e8d6640 1 -- 172.21.15.115:0/3380081569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0040047c0 msgr2=0x7fc004004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01e8d6640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0040047c0 0x7fc004004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01e8d6640 1 -- 172.21.15.115:0/3380081569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc004003a30 msgr2=0x7fc004003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:55.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01e8d6640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc004003a30 0x7fc004003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01e8d6640 1 -- 172.21.15.115:0/3380081569 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004003520 con 0x7fc0040052d0 2023-12-01T01:36:55.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.352+0000 7fc01e0d5640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc004003a30 0x7fc004003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.353+0000 7fc01e8d6640 1 -- 172.21.15.115:0/3380081569 reap_dead start 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.353+0000 7fc01e8d6640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040052d0 0x7fc00400b980 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc01806f700 tx=0x7fc0040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ada1976fe14f73f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.354+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010034630 con 0x7fc0040052d0 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.354+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01002f070 con 0x7fc0040052d0 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.354+0000 7fc01d0d3640 1 -- 172.21.15.115:0/3380081569 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0100349f0 con 0x7fc0040052d0 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.354+0000 7fc020339640 1 -- 172.21.15.115:0/3380081569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040052d0 msgr2=0x7fc00400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.354+0000 7fc020339640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040052d0 0x7fc00400b980 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc01806f700 tx=0x7fc0040057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.354+0000 7fc020339640 1 -- 172.21.15.115:0/3380081569 shutdown_connections 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.354+0000 7fc020339640 1 --2- 172.21.15.115:0/3380081569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040052d0 0x7fc00400b980 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.354+0000 7fc020339640 1 -- 172.21.15.115:0/3380081569 >> 172.21.15.115:0/3380081569 conn(0x7fc0180ce880 msgr2=0x7fc0180cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:55.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.354+0000 7fc020339640 1 -- 172.21.15.115:0/3380081569 shutdown_connections 2023-12-01T01:36:55.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.354+0000 7fc020339640 1 -- 172.21.15.115:0/3380081569 wait complete. 2023-12-01T01:36:55.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.356+0000 7fc020339640 1 Processor -- start 2023-12-01T01:36:55.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.356+0000 7fc020339640 1 -- start start 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.357+0000 7fc020339640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0040052d0 0x7fc0180ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.357+0000 7fc020339640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0181f6850 0x7fc0181e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.357+0000 7fc020339640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0181f1d20 0x7fc0181f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.357+0000 7fc020339640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0181f26e0 con 0x7fc0181f1d20 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.357+0000 7fc020339640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0180ce470 con 0x7fc0040052d0 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.357+0000 7fc020339640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0180c24e0 con 0x7fc0181f6850 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.357+0000 7fc01e8d6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0181f1d20 0x7fc0181f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.357+0000 7fc01e8d6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0181f1d20 0x7fc0181f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46152/0 (socket says 172.21.15.115:46152) 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc01e8d6640 1 -- 172.21.15.115:0/3175114263 learned_addr learned my addr 172.21.15.115:0/3175114263 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc01e8d6640 1 -- 172.21.15.115:0/3175114263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0181f6850 msgr2=0x7fc0181e43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc01e8d6640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0181f6850 0x7fc0181e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc01e8d6640 1 -- 172.21.15.115:0/3175114263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0040052d0 msgr2=0x7fc0180ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:55.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc01e0d5640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0040052d0 0x7fc0180ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:55.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc01e8d6640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0040052d0 0x7fc0180ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:55.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc01e8d6640 1 -- 172.21.15.115:0/3175114263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0180ce190 con 0x7fc0181f1d20 2023-12-01T01:36:55.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc01e8d6640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0181f1d20 0x7fc0181f21a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc01002cae0 tx=0x7fc010001cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:55.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc00effd640 1 -- 172.21.15.115:0/3175114263 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010004d60 con 0x7fc0181f1d20 2023-12-01T01:36:55.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc00effd640 1 -- 172.21.15.115:0/3175114263 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc010034430 con 0x7fc0181f1d20 2023-12-01T01:36:55.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc00effd640 1 -- 172.21.15.115:0/3175114263 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0100347e0 con 0x7fc0181f1d20 2023-12-01T01:36:55.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.358+0000 7fc020339640 1 -- 172.21.15.115:0/3175114263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0180c2780 con 0x7fc0181f1d20 2023-12-01T01:36:55.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.359+0000 7fc020339640 1 -- 172.21.15.115:0/3175114263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0180cde50 con 0x7fc0181f1d20 2023-12-01T01:36:55.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.360+0000 7fc020339640 1 -- 172.21.15.115:0/3175114263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01806fda0 con 0x7fc0181f1d20 2023-12-01T01:36:55.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.361+0000 7fc00effd640 1 -- 172.21.15.115:0/3175114263 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc010031030 con 0x7fc0181f1d20 2023-12-01T01:36:55.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.362+0000 7fc00effd640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbffc06a430 0x7fbffc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:55.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.362+0000 7fc00effd640 1 -- 172.21.15.115:0/3175114263 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc0100a9d40 con 0x7fc0181f1d20 2023-12-01T01:36:55.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.362+0000 7fc01e0d5640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbffc06a430 0x7fbffc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:55.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.363+0000 7fc01e0d5640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbffc06a430 0x7fbffc06c8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc014007fd0 tx=0x7fc014009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:55.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.367+0000 7fc00effd640 1 -- 172.21.15.115:0/3175114263 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc010075300 con 0x7fc0181f1d20 2023-12-01T01:36:55.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:55.616+0000 7fc020339640 1 -- 172.21.15.115:0/3175114263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fc0181e57e0 con 0x7fc0181f1d20 2023-12-01T01:36:56.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.292+0000 7fc00effd640 1 -- 172.21.15.115:0/3175114263 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc010078920 con 0x7fc0181f1d20 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 -- 172.21.15.115:0/3175114263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbffc06a430 msgr2=0x7fbffc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbffc06a430 0x7fbffc06c8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc014007fd0 tx=0x7fc014009040 comp rx=0 tx=0).stop 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 -- 172.21.15.115:0/3175114263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0181f1d20 msgr2=0x7fc0181f21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0181f1d20 0x7fc0181f21a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc01002cae0 tx=0x7fc010001cd0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 -- 172.21.15.115:0/3175114263 shutdown_connections 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbffc06a430 0x7fbffc06c8f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0181f6850 0x7fc0181e43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0181f1d20 0x7fc0181f21a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 --2- 172.21.15.115:0/3175114263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0040052d0 0x7fc0180ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 -- 172.21.15.115:0/3175114263 >> 172.21.15.115:0/3175114263 conn(0x7fc0180ce880 msgr2=0x7fc01815d150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 -- 172.21.15.115:0/3175114263 shutdown_connections 2023-12-01T01:36:56.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.296+0000 7fc020339640 1 -- 172.21.15.115:0/3175114263 wait complete. 2023-12-01T01:36:56.317 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T01:36:56.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.535+0000 7f84879ae640 1 Processor -- start 2023-12-01T01:36:56.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.535+0000 7f84879ae640 1 -- start start 2023-12-01T01:36:56.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.536+0000 7f84879ae640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f848015ac80 0x7f84800cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.536+0000 7f84879ae640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f84800b0e60 0x7f84800b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.536+0000 7f84879ae640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84800cc9a0 con 0x7f84800cdaa0 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.536+0000 7f84879ae640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848015b060 con 0x7f84800b0e60 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.536+0000 7f84879ae640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84800b0ce0 con 0x7f848015ac80 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.537+0000 7f8484f49640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f84800b0e60 0x7f84800b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.537+0000 7f8484f49640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f84800b0e60 0x7f84800b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46166/0 (socket says 172.21.15.115:46166) 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.537+0000 7f8484f49640 1 -- 172.21.15.115:0/2104912048 learned_addr learned my addr 172.21.15.115:0/2104912048 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.537+0000 7f8484f49640 1 -- 172.21.15.115:0/2104912048 >> v2:172.21.15.142:3300/0 conn(0x7f848015ac80 msgr2=0x7f84800cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.537+0000 7f8484f49640 1 --2- 172.21.15.115:0/2104912048 >> v2:172.21.15.142:3300/0 conn(0x7f848015ac80 0x7f84800cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.537+0000 7f8484f49640 1 -- 172.21.15.115:0/2104912048 >> v1:172.21.15.142:6789/0 conn(0x7f84800cdaa0 legacy=0x7f848006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:56.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.537+0000 7f8484f49640 1 -- 172.21.15.115:0/2104912048 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800cbd90 con 0x7f84800b0e60 2023-12-01T01:36:56.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.539+0000 7f8484f49640 1 --2- 172.21.15.115:0/2104912048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84800b0e60 0x7f84800b07a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f84800b3cc0 tx=0x7f847002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c677810285ea61 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:56.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.539+0000 7f8477fff640 1 -- 172.21.15.115:0/2104912048 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470031870 con 0x7f84800b0e60 2023-12-01T01:36:56.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.539+0000 7f8477fff640 1 -- 172.21.15.115:0/2104912048 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84700319d0 con 0x7f84800b0e60 2023-12-01T01:36:56.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.539+0000 7f8477fff640 1 -- 172.21.15.115:0/2104912048 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470031d60 con 0x7f84800b0e60 2023-12-01T01:36:56.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.539+0000 7f84879ae640 1 -- 172.21.15.115:0/2104912048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84800b0e60 msgr2=0x7f84800b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:56.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.539+0000 7f84879ae640 1 --2- 172.21.15.115:0/2104912048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84800b0e60 0x7f84800b07a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f84800b3cc0 tx=0x7f847002b910 comp rx=0 tx=0).stop 2023-12-01T01:36:56.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.540+0000 7f84879ae640 1 -- 172.21.15.115:0/2104912048 shutdown_connections 2023-12-01T01:36:56.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.540+0000 7f84879ae640 1 --2- 172.21.15.115:0/2104912048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84800b0e60 0x7f84800b07a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.540+0000 7f84879ae640 1 --2- 172.21.15.115:0/2104912048 >> v2:172.21.15.142:3300/0 conn(0x7f848015ac80 0x7f84800cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.540+0000 7f84879ae640 1 -- 172.21.15.115:0/2104912048 >> 172.21.15.115:0/2104912048 conn(0x7f8480065960 msgr2=0x7f8480065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:56.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.540+0000 7f84879ae640 1 -- 172.21.15.115:0/2104912048 shutdown_connections 2023-12-01T01:36:56.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.540+0000 7f84879ae640 1 -- 172.21.15.115:0/2104912048 wait complete. 2023-12-01T01:36:56.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.541+0000 7f84879ae640 1 Processor -- start 2023-12-01T01:36:56.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.542+0000 7f84879ae640 1 -- start start 2023-12-01T01:36:56.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.542+0000 7f84879ae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84800b0e60 0x7f8480152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:56.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.542+0000 7f84879ae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84800cdaa0 0x7f8480153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:56.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f84879ae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f848015ac80 0x7f8480154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:56.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f84879ae640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848014fde0 con 0x7f848015ac80 2023-12-01T01:36:56.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f84879ae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8480157530 con 0x7f84800b0e60 2023-12-01T01:36:56.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f84879ae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8480153ac0 con 0x7f84800cdaa0 2023-12-01T01:36:56.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f8485f4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f848015ac80 0x7f8480154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:56.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f8485f4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f848015ac80 0x7f8480154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46180/0 (socket says 172.21.15.115:46180) 2023-12-01T01:36:56.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f8485f4b640 1 -- 172.21.15.115:0/2049625574 learned_addr learned my addr 172.21.15.115:0/2049625574 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:56.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f8484f49640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84800cdaa0 0x7f8480153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:56.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f848574a640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84800b0e60 0x7f8480152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:56.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f8485f4b640 1 -- 172.21.15.115:0/2049625574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84800cdaa0 msgr2=0x7f8480153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:56.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f8485f4b640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84800cdaa0 0x7f8480153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f8485f4b640 1 -- 172.21.15.115:0/2049625574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84800b0e60 msgr2=0x7f8480152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:56.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f8485f4b640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84800b0e60 0x7f8480152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.543+0000 7f8485f4b640 1 -- 172.21.15.115:0/2049625574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84700094d0 con 0x7f848015ac80 2023-12-01T01:36:56.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.544+0000 7f8485f4b640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f848015ac80 0x7f8480154a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f847800c780 tx=0x7f847800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:56.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.544+0000 7f8475ffb640 1 -- 172.21.15.115:0/2049625574 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478004b60 con 0x7f848015ac80 2023-12-01T01:36:56.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.544+0000 7f8475ffb640 1 -- 172.21.15.115:0/2049625574 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847800ce90 con 0x7f848015ac80 2023-12-01T01:36:56.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.544+0000 7f84879ae640 1 -- 172.21.15.115:0/2049625574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8480153da0 con 0x7f848015ac80 2023-12-01T01:36:56.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.544+0000 7f8475ffb640 1 -- 172.21.15.115:0/2049625574 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847800a850 con 0x7f848015ac80 2023-12-01T01:36:56.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.544+0000 7f84879ae640 1 -- 172.21.15.115:0/2049625574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848015a050 con 0x7f848015ac80 2023-12-01T01:36:56.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.545+0000 7f84879ae640 1 -- 172.21.15.115:0/2049625574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848006feb0 con 0x7f848015ac80 2023-12-01T01:36:56.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.546+0000 7f8475ffb640 1 -- 172.21.15.115:0/2049625574 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f847800a9b0 con 0x7f848015ac80 2023-12-01T01:36:56.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.547+0000 7f8475ffb640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f846406a430 0x7f846406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:56.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.547+0000 7f8475ffb640 1 -- 172.21.15.115:0/2049625574 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8478088940 con 0x7f848015ac80 2023-12-01T01:36:56.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.547+0000 7f848574a640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f846406a430 0x7f846406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:56.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.548+0000 7f848574a640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f846406a430 0x7f846406c8f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f84800ca060 tx=0x7f847c009060 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:56.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.552+0000 7f8475ffb640 1 -- 172.21.15.115:0/2049625574 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8478054050 con 0x7f848015ac80 2023-12-01T01:36:56.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.741+0000 7f84879ae640 1 -- 172.21.15.115:0/2049625574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f848004e890 con 0x7f848015ac80 2023-12-01T01:36:56.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.746+0000 7f8475ffb640 1 -- 172.21.15.115:0/2049625574 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8478057670 con 0x7f848015ac80 2023-12-01T01:36:56.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 -- 172.21.15.115:0/2049625574 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f846406a430 msgr2=0x7f846406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:56.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f846406a430 0x7f846406c8f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f84800ca060 tx=0x7f847c009060 comp rx=0 tx=0).stop 2023-12-01T01:36:56.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 -- 172.21.15.115:0/2049625574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f848015ac80 msgr2=0x7f8480154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:56.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f848015ac80 0x7f8480154a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f847800c780 tx=0x7f847800cc40 comp rx=0 tx=0).stop 2023-12-01T01:36:56.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 -- 172.21.15.115:0/2049625574 shutdown_connections 2023-12-01T01:36:56.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f846406a430 0x7f846406c8f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84800cdaa0 0x7f8480153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f848015ac80 0x7f8480154a70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 --2- 172.21.15.115:0/2049625574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84800b0e60 0x7f8480152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:56.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 -- 172.21.15.115:0/2049625574 >> 172.21.15.115:0/2049625574 conn(0x7f8480065960 msgr2=0x7f848006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:56.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 -- 172.21.15.115:0/2049625574 shutdown_connections 2023-12-01T01:36:56.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:56.749+0000 7f84879ae640 1 -- 172.21.15.115:0/2049625574 wait complete. 2023-12-01T01:36:56.770 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:36:56.770 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:36:56.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:36:57.000 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:36:57.000 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:36:57.001 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:36:57.058 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:36:57.058 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:36:57.111 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:36:57.111 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:36:57.169 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:57.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.382+0000 7f6fbd051640 1 Processor -- start 2023-12-01T01:36:57.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.382+0000 7f6fbd051640 1 -- start start 2023-12-01T01:36:57.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.383+0000 7f6fbd051640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6fb815e2f0 0x7f6fb80ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.383+0000 7f6fbd051640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6fb80adf10 0x7f6fb80caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fbd051640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb80add10 con 0x7f6fb806f2f0 2023-12-01T01:36:57.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fbd051640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb80c5760 con 0x7f6fb80adf10 2023-12-01T01:36:57.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fbd051640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb815e6d0 con 0x7f6fb815e2f0 2023-12-01T01:36:57.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fb759e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6fb80adf10 0x7f6fb80caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fb759e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6fb80adf10 0x7f6fb80caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46188/0 (socket says 172.21.15.115:46188) 2023-12-01T01:36:57.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fb759e640 1 -- 172.21.15.115:0/1397566154 learned_addr learned my addr 172.21.15.115:0/1397566154 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:57.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fb6d9d640 1 --2- 172.21.15.115:0/1397566154 >> v2:172.21.15.142:3300/0 conn(0x7f6fb815e2f0 0x7f6fb80ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fb759e640 1 -- 172.21.15.115:0/1397566154 >> v2:172.21.15.142:3300/0 conn(0x7f6fb815e2f0 msgr2=0x7f6fb80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:57.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fb759e640 1 --2- 172.21.15.115:0/1397566154 >> v2:172.21.15.142:3300/0 conn(0x7f6fb815e2f0 0x7f6fb80ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fb759e640 1 -- 172.21.15.115:0/1397566154 >> v1:172.21.15.142:6789/0 conn(0x7f6fb806f2f0 legacy=0x7f6fb80c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:57.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fb759e640 1 -- 172.21.15.115:0/1397566154 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb80cf7b0 con 0x7f6fb80adf10 2023-12-01T01:36:57.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.384+0000 7f6fb6d9d640 1 --2- 172.21.15.115:0/1397566154 >> v2:172.21.15.142:3300/0 conn(0x7f6fb815e2f0 0x7f6fb80ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:57.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.385+0000 7f6fb759e640 1 --2- 172.21.15.115:0/1397566154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb80adf10 0x7f6fb80caba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6fb806ef60 tx=0x7f6fb002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96700cc2d3159ec server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:57.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.386+0000 7f6fb659c640 1 -- 172.21.15.115:0/1397566154 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb002c960 con 0x7f6fb80adf10 2023-12-01T01:36:57.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.386+0000 7f6fb659c640 1 -- 172.21.15.115:0/1397566154 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb002cac0 con 0x7f6fb80adf10 2023-12-01T01:36:57.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.386+0000 7f6fb659c640 1 -- 172.21.15.115:0/1397566154 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb002ce50 con 0x7f6fb80adf10 2023-12-01T01:36:57.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.387+0000 7f6fbd051640 1 -- 172.21.15.115:0/1397566154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb80adf10 msgr2=0x7f6fb80caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:57.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.387+0000 7f6fbd051640 1 --2- 172.21.15.115:0/1397566154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb80adf10 0x7f6fb80caba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6fb806ef60 tx=0x7f6fb002bdc0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.387+0000 7f6fbd051640 1 -- 172.21.15.115:0/1397566154 shutdown_connections 2023-12-01T01:36:57.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.387+0000 7f6fbd051640 1 --2- 172.21.15.115:0/1397566154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb80adf10 0x7f6fb80caba0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.387+0000 7f6fbd051640 1 --2- 172.21.15.115:0/1397566154 >> v2:172.21.15.142:3300/0 conn(0x7f6fb815e2f0 0x7f6fb80ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.387+0000 7f6fbd051640 1 -- 172.21.15.115:0/1397566154 >> 172.21.15.115:0/1397566154 conn(0x7f6fb80ce880 msgr2=0x7f6fb80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:57.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.387+0000 7f6fbd051640 1 -- 172.21.15.115:0/1397566154 shutdown_connections 2023-12-01T01:36:57.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.387+0000 7f6fbd051640 1 -- 172.21.15.115:0/1397566154 wait complete. 2023-12-01T01:36:57.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.388+0000 7f6fbd051640 1 Processor -- start 2023-12-01T01:36:57.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.389+0000 7f6fbd051640 1 -- start start 2023-12-01T01:36:57.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fbd051640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb806f2f0 0x7f6fb81f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fbd051640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80adf10 0x7f6fb80cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fbd051640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb815e2f0 0x7f6fb80ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fbd051640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80c5950 con 0x7f6fb815e2f0 2023-12-01T01:36:57.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fbd051640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80c61f0 con 0x7f6fb80adf10 2023-12-01T01:36:57.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fbd051640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80c2870 con 0x7f6fb806f2f0 2023-12-01T01:36:57.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fb759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb815e2f0 0x7f6fb80ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fb759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb815e2f0 0x7f6fb80ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46200/0 (socket says 172.21.15.115:46200) 2023-12-01T01:36:57.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fb759e640 1 -- 172.21.15.115:0/1382155207 learned_addr learned my addr 172.21.15.115:0/1382155207 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:57.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6faffff640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80adf10 0x7f6fb80cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fb6d9d640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb806f2f0 0x7f6fb81f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fb759e640 1 -- 172.21.15.115:0/1382155207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb806f2f0 msgr2=0x7f6fb81f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:57.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fb759e640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb806f2f0 0x7f6fb81f1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fb759e640 1 -- 172.21.15.115:0/1382155207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80adf10 msgr2=0x7f6fb80cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:57.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fb759e640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80adf10 0x7f6fb80cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6fb759e640 1 -- 172.21.15.115:0/1382155207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb0009b30 con 0x7f6fb815e2f0 2023-12-01T01:36:57.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.390+0000 7f6faffff640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80adf10 0x7f6fb80cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:57.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.391+0000 7f6fb759e640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb815e2f0 0x7f6fb80ac8a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009f40 tx=0x7f6fb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:57.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.391+0000 7f6faf7fe640 1 -- 172.21.15.115:0/1382155207 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb003e450 con 0x7f6fb815e2f0 2023-12-01T01:36:57.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.391+0000 7f6faf7fe640 1 -- 172.21.15.115:0/1382155207 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb003e5d0 con 0x7f6fb815e2f0 2023-12-01T01:36:57.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.391+0000 7f6faf7fe640 1 -- 172.21.15.115:0/1382155207 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb003e980 con 0x7f6fb815e2f0 2023-12-01T01:36:57.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.391+0000 7f6fbd051640 1 -- 172.21.15.115:0/1382155207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb80cd760 con 0x7f6fb815e2f0 2023-12-01T01:36:57.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.391+0000 7f6fbd051640 1 -- 172.21.15.115:0/1382155207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb80c5e80 con 0x7f6fb815e2f0 2023-12-01T01:36:57.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.392+0000 7f6fbd051640 1 -- 172.21.15.115:0/1382155207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb806fda0 con 0x7f6fb815e2f0 2023-12-01T01:36:57.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.393+0000 7f6faf7fe640 1 -- 172.21.15.115:0/1382155207 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6fb003eae0 con 0x7f6fb815e2f0 2023-12-01T01:36:57.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.394+0000 7f6faf7fe640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f9806a460 0x7f6f9806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.394+0000 7f6faf7fe640 1 -- 172.21.15.115:0/1382155207 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6fb00ab5b0 con 0x7f6fb815e2f0 2023-12-01T01:36:57.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.394+0000 7f6fb6d9d640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f9806a460 0x7f6f9806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.395+0000 7f6fb6d9d640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f9806a460 0x7f6f9806c920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b840 tx=0x7f6fa8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:57.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.398+0000 7f6faf7fe640 1 -- 172.21.15.115:0/1382155207 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6fb0076ae0 con 0x7f6fb815e2f0 2023-12-01T01:36:57.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.589+0000 7f6fbd051640 1 -- 172.21.15.115:0/1382155207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fb80c6360 con 0x7f6fb815e2f0 2023-12-01T01:36:57.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.592+0000 7f6faf7fe640 1 -- 172.21.15.115:0/1382155207 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16138 (secure 0 0 0) 0x7f6fb007a100 con 0x7f6fb815e2f0 2023-12-01T01:36:57.594 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:36:57.594 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":23,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:52.262397+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 -- 172.21.15.115:0/1382155207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f9806a460 msgr2=0x7f6f9806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f9806a460 0x7f6f9806c920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b840 tx=0x7f6fa8008040 comp rx=0 tx=0).stop 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 -- 172.21.15.115:0/1382155207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb815e2f0 msgr2=0x7f6fb80ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb815e2f0 0x7f6fb80ac8a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009f40 tx=0x7f6fb0004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 -- 172.21.15.115:0/1382155207 shutdown_connections 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f9806a460 0x7f6f9806c920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb815e2f0 0x7f6fb80ac8a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80adf10 0x7f6fb80cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 --2- 172.21.15.115:0/1382155207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb806f2f0 0x7f6fb81f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 -- 172.21.15.115:0/1382155207 >> 172.21.15.115:0/1382155207 conn(0x7f6fb80ce880 msgr2=0x7f6fb815c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 -- 172.21.15.115:0/1382155207 shutdown_connections 2023-12-01T01:36:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.595+0000 7f6fbd051640 1 -- 172.21.15.115:0/1382155207 wait complete. 2023-12-01T01:36:57.614 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:36:57.615 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:36:57.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.819+0000 7f5b51c5f640 1 Processor -- start 2023-12-01T01:36:57.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.819+0000 7f5b51c5f640 1 -- start start 2023-12-01T01:36:57.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.821+0000 7f5b51c5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 0x7f5b4c160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.821+0000 7f5b51c5f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5b4c0cdb20 0x7f5b4c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.821+0000 7f5b51c5f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4c0b0cd0 con 0x7f5b4c15cd10 2023-12-01T01:36:57.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.821+0000 7f5b51c5f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c1618d0 con 0x7f5b4c0cdb20 2023-12-01T01:36:57.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.821+0000 7f5b51c5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0b2460 con 0x7f5b4c0b38f0 2023-12-01T01:36:57.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.821+0000 7f5b4bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5b4c0cdb20 0x7f5b4c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.821+0000 7f5b4bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5b4c0cdb20 0x7f5b4c0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48082/0 (socket says 172.21.15.115:48082) 2023-12-01T01:36:57.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.821+0000 7f5b4bfff640 1 -- 172.21.15.115:0/725904105 learned_addr learned my addr 172.21.15.115:0/725904105 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:57.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.822+0000 7f5b43fff640 1 --2- 172.21.15.115:0/725904105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 0x7f5b4c160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.822+0000 7f5b4bfff640 1 -- 172.21.15.115:0/725904105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 msgr2=0x7f5b4c160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:57.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.822+0000 7f5b4bfff640 1 --2- 172.21.15.115:0/725904105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 0x7f5b4c160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.822+0000 7f5b4bfff640 1 -- 172.21.15.115:0/725904105 >> v1:172.21.15.142:6789/0 conn(0x7f5b4c15cd10 legacy=0x7f5b4c0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:57.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.822+0000 7f5b4bfff640 1 -- 172.21.15.115:0/725904105 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c161030 con 0x7f5b4c0cdb20 2023-12-01T01:36:57.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.822+0000 7f5b43fff640 1 --2- 172.21.15.115:0/725904105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 0x7f5b4c160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:57.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.823+0000 7f5b4bfff640 1 --2- 172.21.15.115:0/725904105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c0cdb20 0x7f5b4c0cdfb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009e50 tx=0x7f5b4402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7912d445a212f2ac server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:57.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.823+0000 7f5b4affd640 1 -- 172.21.15.115:0/725904105 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4402ca70 con 0x7f5b4c0cdb20 2023-12-01T01:36:57.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.823+0000 7f5b4affd640 1 -- 172.21.15.115:0/725904105 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4402cbd0 con 0x7f5b4c0cdb20 2023-12-01T01:36:57.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.823+0000 7f5b4affd640 1 -- 172.21.15.115:0/725904105 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b440343f0 con 0x7f5b4c0cdb20 2023-12-01T01:36:57.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.824+0000 7f5b51c5f640 1 -- 172.21.15.115:0/725904105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c0cdb20 msgr2=0x7f5b4c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:57.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.824+0000 7f5b51c5f640 1 --2- 172.21.15.115:0/725904105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c0cdb20 0x7f5b4c0cdfb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009e50 tx=0x7f5b4402bed0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.824+0000 7f5b51c5f640 1 -- 172.21.15.115:0/725904105 shutdown_connections 2023-12-01T01:36:57.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.824+0000 7f5b51c5f640 1 --2- 172.21.15.115:0/725904105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c0cdb20 0x7f5b4c0cdfb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.824+0000 7f5b51c5f640 1 --2- 172.21.15.115:0/725904105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 0x7f5b4c160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.824+0000 7f5b51c5f640 1 -- 172.21.15.115:0/725904105 >> 172.21.15.115:0/725904105 conn(0x7f5b4c072d00 msgr2=0x7f5b4c0730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:57.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.824+0000 7f5b51c5f640 1 -- 172.21.15.115:0/725904105 shutdown_connections 2023-12-01T01:36:57.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.824+0000 7f5b51c5f640 1 -- 172.21.15.115:0/725904105 wait complete. 2023-12-01T01:36:57.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.826+0000 7f5b51c5f640 1 Processor -- start 2023-12-01T01:36:57.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.826+0000 7f5b51c5f640 1 -- start start 2023-12-01T01:36:57.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b51c5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 0x7f5b4c15e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b51c5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b4c0cdb20 0x7f5b4c1f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b51c5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c15cd10 0x7f5b4c15d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b51c5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0ca7e0 con 0x7f5b4c15cd10 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b51c5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0cad80 con 0x7f5b4c0cdb20 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b51c5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c15e050 con 0x7f5b4c0b38f0 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b4bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c15cd10 0x7f5b4c15d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b4bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c15cd10 0x7f5b4c15d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48088/0 (socket says 172.21.15.115:48088) 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b4bfff640 1 -- 172.21.15.115:0/1352939585 learned_addr learned my addr 172.21.15.115:0/1352939585 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b43fff640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b4c0cdb20 0x7f5b4c1f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b4b7fe640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 0x7f5b4c15e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b4bfff640 1 -- 172.21.15.115:0/1352939585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 msgr2=0x7f5b4c15e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:57.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b4bfff640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 0x7f5b4c15e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b4bfff640 1 -- 172.21.15.115:0/1352939585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b4c0cdb20 msgr2=0x7f5b4c1f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b4bfff640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b4c0cdb20 0x7f5b4c1f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b4bfff640 1 -- 172.21.15.115:0/1352939585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b44009b30 con 0x7f5b4c15cd10 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b43fff640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b4c0cdb20 0x7f5b4c1f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.827+0000 7f5b4b7fe640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 0x7f5b4c15e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.828+0000 7f5b4bfff640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c15cd10 0x7f5b4c15d0f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009e50 tx=0x7f5b44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.828+0000 7f5b48ff9640 1 -- 172.21.15.115:0/1352939585 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44009510 con 0x7f5b4c15cd10 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.828+0000 7f5b48ff9640 1 -- 172.21.15.115:0/1352939585 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4403e450 con 0x7f5b4c15cd10 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.828+0000 7f5b48ff9640 1 -- 172.21.15.115:0/1352939585 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4403e800 con 0x7f5b4c15cd10 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.828+0000 7f5b51c5f640 1 -- 172.21.15.115:0/1352939585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c15da90 con 0x7f5b4c15cd10 2023-12-01T01:36:57.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.828+0000 7f5b51c5f640 1 -- 172.21.15.115:0/1352939585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c0caa10 con 0x7f5b4c15cd10 2023-12-01T01:36:57.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.829+0000 7f5b51c5f640 1 -- 172.21.15.115:0/1352939585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c0cabd0 con 0x7f5b4c15cd10 2023-12-01T01:36:57.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.830+0000 7f5b48ff9640 1 -- 172.21.15.115:0/1352939585 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5b4403e960 con 0x7f5b4c15cd10 2023-12-01T01:36:57.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.831+0000 7f5b48ff9640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5b2c06a430 0x7f5b2c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:57.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.831+0000 7f5b48ff9640 1 -- 172.21.15.115:0/1352939585 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5b440ab7d0 con 0x7f5b4c15cd10 2023-12-01T01:36:57.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.831+0000 7f5b4b7fe640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5b2c06a430 0x7f5b2c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:57.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.832+0000 7f5b4b7fe640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5b2c06a430 0x7f5b2c06c8f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000f60 tx=0x7f5b3c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:57.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:57.835+0000 7f5b48ff9640 1 -- 172.21.15.115:0/1352939585 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5b44076db0 con 0x7f5b4c15cd10 2023-12-01T01:36:58.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.028+0000 7f5b51c5f640 1 -- 172.21.15.115:0/1352939585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b4c0caef0 con 0x7f5b4c15cd10 2023-12-01T01:36:58.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.029+0000 7f5b48ff9640 1 -- 172.21.15.115:0/1352939585 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b4407a3d0 con 0x7f5b4c15cd10 2023-12-01T01:36:58.030 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:36:58.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.031+0000 7f5b51c5f640 1 -- 172.21.15.115:0/1352939585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5b2c06a430 msgr2=0x7f5b2c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5b2c06a430 0x7f5b2c06c8f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000f60 tx=0x7f5b3c009040 comp rx=0 tx=0).stop 2023-12-01T01:36:58.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 -- 172.21.15.115:0/1352939585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c15cd10 msgr2=0x7f5b4c15d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c15cd10 0x7f5b4c15d0f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009e50 tx=0x7f5b44004a40 comp rx=0 tx=0).stop 2023-12-01T01:36:58.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 -- 172.21.15.115:0/1352939585 shutdown_connections 2023-12-01T01:36:58.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5b2c06a430 0x7f5b2c06c8f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b4c15cd10 0x7f5b4c15d0f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b4c0cdb20 0x7f5b4c1f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 --2- 172.21.15.115:0/1352939585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b4c0b38f0 0x7f5b4c15e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 -- 172.21.15.115:0/1352939585 >> 172.21.15.115:0/1352939585 conn(0x7f5b4c072d00 msgr2=0x7f5b4c070990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:58.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 -- 172.21.15.115:0/1352939585 shutdown_connections 2023-12-01T01:36:58.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.032+0000 7f5b51c5f640 1 -- 172.21.15.115:0/1352939585 wait complete. 2023-12-01T01:36:58.052 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:36:58.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.259+0000 7f7887ae7640 1 Processor -- start 2023-12-01T01:36:58.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.259+0000 7f7887ae7640 1 -- start start 2023-12-01T01:36:58.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.261+0000 7f7887ae7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f788015b0e0 0x7f7880164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.261+0000 7f7887ae7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78800b2f30 con 0x7f7880142870 2023-12-01T01:36:58.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.261+0000 7f7887ae7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78800b2320 con 0x7f78800cd1e0 2023-12-01T01:36:58.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.261+0000 7f7887ae7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78800b3ae0 con 0x7f788015b0e0 2023-12-01T01:36:58.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.261+0000 7f7885883640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7880142870 0x7f78800ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34170/0 (socket says 172.21.15.115:34170) 2023-12-01T01:36:58.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.261+0000 7f7885883640 1 -- 172.21.15.115:0/3286308956 learned_addr learned my addr 172.21.15.115:0/3286308956 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:58.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.261+0000 7f7886084640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f788015b0e0 0x7f7880164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.261+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157226313 0 0) 0x7f78800b2f30 con 0x7f7880142870 2023-12-01T01:36:58.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.261+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f786c003520 con 0x7f7880142870 2023-12-01T01:36:58.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2413114587 0 0) 0x7f786c003520 con 0x7f7880142870 2023-12-01T01:36:58.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f788015b0e0 msgr2=0x7f7880164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f788015b0e0 0x7f7880164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 >> v1:172.21.15.142:6789/0 conn(0x7f78800cd1e0 legacy=0x7f78800b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78800b2600 con 0x7f7880142870 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f787c001670 con 0x7f7880142870 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 >> v1:172.21.15.115:6789/0 conn(0x7f7880142870 legacy=0x7f78800ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f786c0039a0 0x7f786c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f786c0047c0 0x7f786c004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 0x7f786c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800b3ae0 con 0x7f786c005140 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c00e0d0 con 0x7f786c0039a0 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c00e210 con 0x7f786c0047c0 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7886084640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 0x7f786c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.262+0000 7f7885883640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f786c0039a0 0x7f786c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.263+0000 7f7886084640 1 -- 172.21.15.115:0/3286308956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f786c0047c0 msgr2=0x7f786c004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:58.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.263+0000 7f7886084640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f786c0047c0 0x7f786c004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.263+0000 7f7886084640 1 -- 172.21.15.115:0/3286308956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f786c0039a0 msgr2=0x7f786c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:58.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.263+0000 7f7886084640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f786c0039a0 0x7f786c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.263+0000 7f7886084640 1 -- 172.21.15.115:0/3286308956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c003520 con 0x7f786c005140 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.264+0000 7f7886084640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 0x7f786c00db90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7878009f90 tx=0x7f786c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a04e9f949270d6f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.264+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787802cc00 con 0x7f786c005140 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.264+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787800f040 con 0x7f786c005140 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.264+0000 7f7884881640 1 -- 172.21.15.115:0/3286308956 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878034440 con 0x7f786c005140 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.264+0000 7f7887ae7640 1 -- 172.21.15.115:0/3286308956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 msgr2=0x7f786c00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.264+0000 7f7887ae7640 1 --2- 172.21.15.115:0/3286308956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 0x7f786c00db90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7878009f90 tx=0x7f786c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.265+0000 7f7886084640 1 -- 172.21.15.115:0/3286308956 reap_dead start 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.265+0000 7f7887ae7640 1 -- 172.21.15.115:0/3286308956 shutdown_connections 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.265+0000 7f7887ae7640 1 -- 172.21.15.115:0/3286308956 >> 172.21.15.115:0/3286308956 conn(0x7f7880065960 msgr2=0x7f7880065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.265+0000 7f7887ae7640 1 -- 172.21.15.115:0/3286308956 shutdown_connections 2023-12-01T01:36:58.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.265+0000 7f7887ae7640 1 -- 172.21.15.115:0/3286308956 wait complete. 2023-12-01T01:36:58.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.266+0000 7f7887ae7640 1 Processor -- start 2023-12-01T01:36:58.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.267+0000 7f7887ae7640 1 -- start start 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.267+0000 7f7887ae7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 0x7f7880150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.267+0000 7f7887ae7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f788013f9a0 0x7f78800cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.267+0000 7f7887ae7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f788013fe00 0x7f788013bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.267+0000 7f7887ae7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78801500b0 con 0x7f786c005140 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.267+0000 7f7887ae7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788014f4a0 con 0x7f788013fe00 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.267+0000 7f7887ae7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788014f780 con 0x7f788013f9a0 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885883640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 0x7f7880150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885883640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 0x7f7880150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48098/0 (socket says 172.21.15.115:48098) 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885883640 1 -- 172.21.15.115:0/2666242726 learned_addr learned my addr 172.21.15.115:0/2666242726 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885883640 1 -- 172.21.15.115:0/2666242726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f788013f9a0 msgr2=0x7f78800cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885082640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f788013f9a0 0x7f78800cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7886084640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f788013fe00 0x7f788013bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885883640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f788013f9a0 0x7f78800cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885883640 1 -- 172.21.15.115:0/2666242726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f788013fe00 msgr2=0x7f788013bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885883640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f788013fe00 0x7f788013bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885883640 1 -- 172.21.15.115:0/2666242726 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788014fa60 con 0x7f786c005140 2023-12-01T01:36:58.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885082640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f788013f9a0 0x7f78800cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:58.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.268+0000 7f7885883640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 0x7f7880150540 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e800 tx=0x7f787c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:58.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.269+0000 7f78767fc640 1 -- 172.21.15.115:0/2666242726 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00c870 con 0x7f786c005140 2023-12-01T01:36:58.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.269+0000 7f78767fc640 1 -- 172.21.15.115:0/2666242726 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c00c9d0 con 0x7f786c005140 2023-12-01T01:36:58.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.269+0000 7f7887ae7640 1 -- 172.21.15.115:0/2666242726 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788014fd40 con 0x7f786c005140 2023-12-01T01:36:58.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.269+0000 7f78767fc640 1 -- 172.21.15.115:0/2666242726 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00cd80 con 0x7f786c005140 2023-12-01T01:36:58.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.269+0000 7f7887ae7640 1 -- 172.21.15.115:0/2666242726 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7880150a80 con 0x7f786c005140 2023-12-01T01:36:58.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.270+0000 7f7887ae7640 1 -- 172.21.15.115:0/2666242726 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788004e890 con 0x7f786c005140 2023-12-01T01:36:58.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.271+0000 7f78767fc640 1 -- 172.21.15.115:0/2666242726 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f787c01d460 con 0x7f786c005140 2023-12-01T01:36:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.272+0000 7f78767fc640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f786406a460 0x7f786406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.272+0000 7f78767fc640 1 -- 172.21.15.115:0/2666242726 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f787c0889b0 con 0x7f786c005140 2023-12-01T01:36:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.273+0000 7f7885082640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f786406a460 0x7f786406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.273+0000 7f7885082640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f786406a460 0x7f786406c920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f78800c9070 tx=0x7f7870009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:58.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.277+0000 7f78767fc640 1 -- 172.21.15.115:0/2666242726 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f787c053ed0 con 0x7f786c005140 2023-12-01T01:36:58.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.469+0000 7f7887ae7640 1 -- 172.21.15.115:0/2666242726 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78800b28b0 con 0x7f786c005140 2023-12-01T01:36:58.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.470+0000 7f78767fc640 1 -- 172.21.15.115:0/2666242726 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f787c001560 con 0x7f786c005140 2023-12-01T01:36:58.471 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:36:58.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 -- 172.21.15.115:0/2666242726 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f786406a460 msgr2=0x7f786406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f786406a460 0x7f786406c920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f78800c9070 tx=0x7f7870009040 comp rx=0 tx=0).stop 2023-12-01T01:36:58.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 -- 172.21.15.115:0/2666242726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 msgr2=0x7f7880150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 0x7f7880150540 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e800 tx=0x7f787c00ecc0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 -- 172.21.15.115:0/2666242726 shutdown_connections 2023-12-01T01:36:58.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f786406a460 0x7f786406c920 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f788013f9a0 0x7f78800cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f788013fe00 0x7f788013bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 --2- 172.21.15.115:0/2666242726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f786c005140 0x7f7880150540 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 -- 172.21.15.115:0/2666242726 >> 172.21.15.115:0/2666242726 conn(0x7f7880065960 msgr2=0x7f788013cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:58.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 -- 172.21.15.115:0/2666242726 shutdown_connections 2023-12-01T01:36:58.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.472+0000 7f7887ae7640 1 -- 172.21.15.115:0/2666242726 wait complete. 2023-12-01T01:36:58.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:36:58.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.698+0000 7fc6900af640 1 Processor -- start 2023-12-01T01:36:58.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.698+0000 7fc6900af640 1 -- start start 2023-12-01T01:36:58.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.700+0000 7fc6900af640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc68815cd90 0x7fc6880cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.700+0000 7fc6900af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6880b5fa0 0x7fc688161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.700+0000 7fc6900af640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6880ce5e0 con 0x7fc6880cf650 2023-12-01T01:36:58.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.700+0000 7fc6900af640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6880d0ec0 con 0x7fc68815cd90 2023-12-01T01:36:58.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.700+0000 7fc6900af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc688161950 con 0x7fc6880b5fa0 2023-12-01T01:36:58.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.700+0000 7fc68de4b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc68815cd90 0x7fc6880cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.700+0000 7fc68de4b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc68815cd90 0x7fc6880cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48110/0 (socket says 172.21.15.115:48110) 2023-12-01T01:36:58.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.700+0000 7fc68de4b640 1 -- 172.21.15.115:0/4116147382 learned_addr learned my addr 172.21.15.115:0/4116147382 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:58.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.701+0000 7fc68de4b640 1 -- 172.21.15.115:0/4116147382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6880b5fa0 msgr2=0x7fc688161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:58.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.701+0000 7fc68de4b640 1 --2- 172.21.15.115:0/4116147382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6880b5fa0 0x7fc688161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.701+0000 7fc68de4b640 1 -- 172.21.15.115:0/4116147382 >> v1:172.21.15.142:6789/0 conn(0x7fc6880cf650 legacy=0x7fc6880b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:58.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.701+0000 7fc68de4b640 1 -- 172.21.15.115:0/4116147382 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6880ce300 con 0x7fc68815cd90 2023-12-01T01:36:58.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.702+0000 7fc68de4b640 1 --2- 172.21.15.115:0/4116147382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc68815cd90 0x7fc6880cdd20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc684009870 tx=0x7fc68402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d74ee6259f2b7a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:58.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.702+0000 7fc68ce49640 1 -- 172.21.15.115:0/4116147382 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684031640 con 0x7fc68815cd90 2023-12-01T01:36:58.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.702+0000 7fc68ce49640 1 -- 172.21.15.115:0/4116147382 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6840317a0 con 0x7fc68815cd90 2023-12-01T01:36:58.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.702+0000 7fc68ce49640 1 -- 172.21.15.115:0/4116147382 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684031b30 con 0x7fc68815cd90 2023-12-01T01:36:58.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.702+0000 7fc6900af640 1 -- 172.21.15.115:0/4116147382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc68815cd90 msgr2=0x7fc6880cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.702+0000 7fc6900af640 1 --2- 172.21.15.115:0/4116147382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc68815cd90 0x7fc6880cdd20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc684009870 tx=0x7fc68402b8f0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.703+0000 7fc6900af640 1 -- 172.21.15.115:0/4116147382 shutdown_connections 2023-12-01T01:36:58.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.703+0000 7fc6900af640 1 --2- 172.21.15.115:0/4116147382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6880b5fa0 0x7fc688161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.703+0000 7fc6900af640 1 --2- 172.21.15.115:0/4116147382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc68815cd90 0x7fc6880cdd20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.703+0000 7fc6900af640 1 -- 172.21.15.115:0/4116147382 >> 172.21.15.115:0/4116147382 conn(0x7fc688065960 msgr2=0x7fc688065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:58.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.703+0000 7fc6900af640 1 -- 172.21.15.115:0/4116147382 shutdown_connections 2023-12-01T01:36:58.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.703+0000 7fc6900af640 1 -- 172.21.15.115:0/4116147382 wait complete. 2023-12-01T01:36:58.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.704+0000 7fc6900af640 1 Processor -- start 2023-12-01T01:36:58.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.704+0000 7fc6900af640 1 -- start start 2023-12-01T01:36:58.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.705+0000 7fc6900af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6880b5fa0 0x7fc68815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.705+0000 7fc6900af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6880cf650 0x7fc688160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.705+0000 7fc6900af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc68815cd90 0x7fc6881f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.705+0000 7fc6900af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6880a96a0 con 0x7fc6880cf650 2023-12-01T01:36:58.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.705+0000 7fc6900af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc688160780 con 0x7fc6880b5fa0 2023-12-01T01:36:58.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.705+0000 7fc6900af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6881f2ee0 con 0x7fc68815cd90 2023-12-01T01:36:58.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68d64a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6880cf650 0x7fc688160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68d64a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6880cf650 0x7fc688160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48112/0 (socket says 172.21.15.115:48112) 2023-12-01T01:36:58.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68d64a640 1 -- 172.21.15.115:0/3770833601 learned_addr learned my addr 172.21.15.115:0/3770833601 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68de4b640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6880b5fa0 0x7fc68815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68d64a640 1 -- 172.21.15.115:0/3770833601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc68815cd90 msgr2=0x7fc6881f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68e64c640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc68815cd90 0x7fc6881f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68d64a640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc68815cd90 0x7fc6881f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68d64a640 1 -- 172.21.15.115:0/3770833601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6880b5fa0 msgr2=0x7fc68815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68d64a640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6880b5fa0 0x7fc68815d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68d64a640 1 -- 172.21.15.115:0/3770833601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840094d0 con 0x7fc6880cf650 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68de4b640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6880b5fa0 0x7fc68815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.706+0000 7fc68d64a640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6880cf650 0x7fc688160240 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc67800b4a0 tx=0x7fc67800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.707+0000 7fc67e7fc640 1 -- 172.21.15.115:0/3770833601 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678010610 con 0x7fc6880cf650 2023-12-01T01:36:58.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.707+0000 7fc67e7fc640 1 -- 172.21.15.115:0/3770833601 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678010770 con 0x7fc6880cf650 2023-12-01T01:36:58.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.707+0000 7fc67e7fc640 1 -- 172.21.15.115:0/3770833601 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678010b20 con 0x7fc6880cf650 2023-12-01T01:36:58.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.707+0000 7fc6900af640 1 -- 172.21.15.115:0/3770833601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6881f31c0 con 0x7fc6880cf650 2023-12-01T01:36:58.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.707+0000 7fc6900af640 1 -- 172.21.15.115:0/3770833601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6881f3780 con 0x7fc6880cf650 2023-12-01T01:36:58.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.708+0000 7fc6900af640 1 -- 172.21.15.115:0/3770833601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6880d3bc0 con 0x7fc6880cf650 2023-12-01T01:36:58.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.709+0000 7fc67e7fc640 1 -- 172.21.15.115:0/3770833601 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc678010c80 con 0x7fc6880cf650 2023-12-01T01:36:58.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.710+0000 7fc67e7fc640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc66c06a430 0x7fc66c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:58.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.710+0000 7fc67e7fc640 1 -- 172.21.15.115:0/3770833601 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc678088460 con 0x7fc6880cf650 2023-12-01T01:36:58.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.710+0000 7fc68de4b640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc66c06a430 0x7fc66c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:58.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.711+0000 7fc68de4b640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc66c06a430 0x7fc66c06c8f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc684036000 tx=0x7fc68403c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:58.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.714+0000 7fc67e7fc640 1 -- 172.21.15.115:0/3770833601 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc678053ac0 con 0x7fc6880cf650 2023-12-01T01:36:58.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.901+0000 7fc6900af640 1 -- 172.21.15.115:0/3770833601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6880a97e0 con 0x7fc6880cf650 2023-12-01T01:36:58.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.902+0000 7fc67e7fc640 1 -- 172.21.15.115:0/3770833601 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6880a97e0 con 0x7fc6880cf650 2023-12-01T01:36:58.903 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:36:58.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.904+0000 7fc6900af640 1 -- 172.21.15.115:0/3770833601 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc66c06a430 msgr2=0x7fc66c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.904+0000 7fc6900af640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc66c06a430 0x7fc66c06c8f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc684036000 tx=0x7fc68403c040 comp rx=0 tx=0).stop 2023-12-01T01:36:58.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.904+0000 7fc6900af640 1 -- 172.21.15.115:0/3770833601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6880cf650 msgr2=0x7fc688160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:58.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.904+0000 7fc6900af640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6880cf650 0x7fc688160240 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc67800b4a0 tx=0x7fc67800b960 comp rx=0 tx=0).stop 2023-12-01T01:36:58.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.905+0000 7fc6900af640 1 -- 172.21.15.115:0/3770833601 shutdown_connections 2023-12-01T01:36:58.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.905+0000 7fc6900af640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc66c06a430 0x7fc66c06c8f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.905+0000 7fc6900af640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc68815cd90 0x7fc6881f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.905+0000 7fc6900af640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6880cf650 0x7fc688160240 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.905+0000 7fc6900af640 1 --2- 172.21.15.115:0/3770833601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6880b5fa0 0x7fc68815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:58.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.905+0000 7fc6900af640 1 -- 172.21.15.115:0/3770833601 >> 172.21.15.115:0/3770833601 conn(0x7fc688065960 msgr2=0x7fc6880b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:58.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.905+0000 7fc6900af640 1 -- 172.21.15.115:0/3770833601 shutdown_connections 2023-12-01T01:36:58.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:58.905+0000 7fc6900af640 1 -- 172.21.15.115:0/3770833601 wait complete. 2023-12-01T01:36:58.926 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:36:59.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.138+0000 7f9a375ab640 1 Processor -- start 2023-12-01T01:36:59.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.139+0000 7f9a375ab640 1 -- start start 2023-12-01T01:36:59.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.140+0000 7f9a375ab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9a300c9ef0 0x7f9a3006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.140+0000 7f9a375ab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3006f8b0 con 0x7f9a3006b6f0 2023-12-01T01:36:59.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.140+0000 7f9a375ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a300adcb0 con 0x7f9a30159dd0 2023-12-01T01:36:59.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.140+0000 7f9a375ab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a300ad9d0 con 0x7f9a300c9ef0 2023-12-01T01:36:59.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.140+0000 7f9a34b46640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9a300c9ef0 0x7f9a3006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:59.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.141+0000 7f9a34b46640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9a300c9ef0 0x7f9a3006fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48128/0 (socket says 172.21.15.115:48128) 2023-12-01T01:36:59.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.141+0000 7f9a35b48640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9a3006b6f0 0x7f9a30070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34172/0 (socket says 172.21.15.115:34172) 2023-12-01T01:36:59.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.141+0000 7f9a34b46640 1 -- 172.21.15.115:0/3148725247 learned_addr learned my addr 172.21.15.115:0/3148725247 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:59.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.141+0000 7f9a34b46640 1 -- 172.21.15.115:0/3148725247 >> v1:172.21.15.142:6789/0 conn(0x7f9a30159dd0 legacy=0x7f9a300c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:59.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.141+0000 7f9a34b46640 1 -- 172.21.15.115:0/3148725247 >> v1:172.21.15.115:6789/0 conn(0x7f9a3006b6f0 legacy=0x7f9a30070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:59.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.141+0000 7f9a34b46640 1 -- 172.21.15.115:0/3148725247 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a300c9000 con 0x7f9a300c9ef0 2023-12-01T01:36:59.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.141+0000 7f9a27fff640 1 -- 172.21.15.115:0/3148725247 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578033595 0 0) 0x7f9a3006f8b0 con 0x7f9a3006b6f0 2023-12-01T01:36:59.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.142+0000 7f9a34b46640 1 --2- 172.21.15.115:0/3148725247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 0x7f9a3006fd20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009870 tx=0x7f9a2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6758c42c4c45140 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:59.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.142+0000 7f9a27fff640 1 -- 172.21.15.115:0/3148725247 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20031870 con 0x7f9a300c9ef0 2023-12-01T01:36:59.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.142+0000 7f9a27fff640 1 -- 172.21.15.115:0/3148725247 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a200319d0 con 0x7f9a300c9ef0 2023-12-01T01:36:59.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.142+0000 7f9a27fff640 1 -- 172.21.15.115:0/3148725247 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20031d60 con 0x7f9a300c9ef0 2023-12-01T01:36:59.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.143+0000 7f9a375ab640 1 -- 172.21.15.115:0/3148725247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 msgr2=0x7f9a3006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:59.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.143+0000 7f9a375ab640 1 --2- 172.21.15.115:0/3148725247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 0x7f9a3006fd20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009870 tx=0x7f9a2002b910 comp rx=0 tx=0).stop 2023-12-01T01:36:59.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.143+0000 7f9a375ab640 1 -- 172.21.15.115:0/3148725247 shutdown_connections 2023-12-01T01:36:59.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.143+0000 7f9a375ab640 1 --2- 172.21.15.115:0/3148725247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 0x7f9a3006fd20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.143+0000 7f9a375ab640 1 -- 172.21.15.115:0/3148725247 >> 172.21.15.115:0/3148725247 conn(0x7f9a300ce880 msgr2=0x7f9a300cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:59.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.143+0000 7f9a375ab640 1 -- 172.21.15.115:0/3148725247 shutdown_connections 2023-12-01T01:36:59.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.143+0000 7f9a375ab640 1 -- 172.21.15.115:0/3148725247 wait complete. 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.145+0000 7f9a375ab640 1 Processor -- start 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.145+0000 7f9a375ab640 1 -- start start 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.146+0000 7f9a375ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a3006b6f0 0x7f9a30147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.146+0000 7f9a375ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 0x7f9a301e5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.146+0000 7f9a375ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a30159dd0 0x7f9a301484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.146+0000 7f9a375ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a301e5730 con 0x7f9a300c9ef0 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.146+0000 7f9a375ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a301e5a10 con 0x7f9a3006b6f0 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.146+0000 7f9a375ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3015b610 con 0x7f9a30159dd0 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.146+0000 7f9a34b46640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 0x7f9a301e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.146+0000 7f9a34b46640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 0x7f9a301e5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48134/0 (socket says 172.21.15.115:48134) 2023-12-01T01:36:59.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.146+0000 7f9a34b46640 1 -- 172.21.15.115:0/3068734270 learned_addr learned my addr 172.21.15.115:0/3068734270 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a35b48640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a30159dd0 0x7f9a301484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a35347640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a3006b6f0 0x7f9a30147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a34b46640 1 -- 172.21.15.115:0/3068734270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a30159dd0 msgr2=0x7f9a301484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a34b46640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a30159dd0 0x7f9a301484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a34b46640 1 -- 172.21.15.115:0/3068734270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a3006b6f0 msgr2=0x7f9a30147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a34b46640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a3006b6f0 0x7f9a30147db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a34b46640 1 -- 172.21.15.115:0/3068734270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a200094d0 con 0x7f9a300c9ef0 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a34b46640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 0x7f9a301e5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9a20000c00 tx=0x7f9a200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a25ffb640 1 -- 172.21.15.115:0/3068734270 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20031840 con 0x7f9a300c9ef0 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a25ffb640 1 -- 172.21.15.115:0/3068734270 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a200443f0 con 0x7f9a300c9ef0 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a25ffb640 1 -- 172.21.15.115:0/3068734270 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a200447c0 con 0x7f9a300c9ef0 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.147+0000 7f9a375ab640 1 -- 172.21.15.115:0/3068734270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3015b8f0 con 0x7f9a300c9ef0 2023-12-01T01:36:59.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.148+0000 7f9a375ab640 1 -- 172.21.15.115:0/3068734270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a301e7040 con 0x7f9a300c9ef0 2023-12-01T01:36:59.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.149+0000 7f9a375ab640 1 -- 172.21.15.115:0/3068734270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3015be10 con 0x7f9a300c9ef0 2023-12-01T01:36:59.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.150+0000 7f9a25ffb640 1 -- 172.21.15.115:0/3068734270 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9a2004d450 con 0x7f9a300c9ef0 2023-12-01T01:36:59.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.150+0000 7f9a25ffb640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9a1406a430 0x7f9a1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.151+0000 7f9a25ffb640 1 -- 172.21.15.115:0/3068734270 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9a2002d040 con 0x7f9a300c9ef0 2023-12-01T01:36:59.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.151+0000 7f9a35347640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9a1406a430 0x7f9a1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:59.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.151+0000 7f9a35347640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9a1406a430 0x7f9a1406c8f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c000f60 tx=0x7f9a2c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:59.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.155+0000 7f9a25ffb640 1 -- 172.21.15.115:0/3068734270 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9a3015be10 con 0x7f9a300c9ef0 2023-12-01T01:36:59.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.358+0000 7f9a375ab640 1 -- 172.21.15.115:0/3068734270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9a3015be10 con 0x7f9a300c9ef0 2023-12-01T01:36:59.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.359+0000 7f9a25ffb640 1 -- 172.21.15.115:0/3068734270 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9a3015be10 con 0x7f9a300c9ef0 2023-12-01T01:36:59.360 INFO:teuthology.orchestra.run.smithi115.stderr:listed 0 entries 2023-12-01T01:36:59.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 -- 172.21.15.115:0/3068734270 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9a1406a430 msgr2=0x7f9a1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:59.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9a1406a430 0x7f9a1406c8f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c000f60 tx=0x7f9a2c009040 comp rx=0 tx=0).stop 2023-12-01T01:36:59.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 -- 172.21.15.115:0/3068734270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 msgr2=0x7f9a301e5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:59.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 0x7f9a301e5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9a20000c00 tx=0x7f9a200046c0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 -- 172.21.15.115:0/3068734270 shutdown_connections 2023-12-01T01:36:59.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9a1406a430 0x7f9a1406c8f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a30159dd0 0x7f9a301484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a300c9ef0 0x7f9a301e5050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 --2- 172.21.15.115:0/3068734270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a3006b6f0 0x7f9a30147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 -- 172.21.15.115:0/3068734270 >> 172.21.15.115:0/3068734270 conn(0x7f9a300ce880 msgr2=0x7f9a30071010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:59.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 -- 172.21.15.115:0/3068734270 shutdown_connections 2023-12-01T01:36:59.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.361+0000 7f9a375ab640 1 -- 172.21.15.115:0/3068734270 wait complete. 2023-12-01T01:36:59.381 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T01:36:59.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:59.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-12-01T01:36:59.484 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:36:59.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:36:59.484 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T01:36:59.545 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:36:59.545 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:36:59.605 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:36:59.605 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:36:59.658 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:36:59.658 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:36:59.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:36:59.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.928+0000 7ff972c5f640 1 Processor -- start 2023-12-01T01:36:59.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.928+0000 7ff972c5f640 1 -- start start 2023-12-01T01:36:59.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.930+0000 7ff972c5f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff96c15cd20 0x7ff96c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.930+0000 7ff972c5f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff96c0d1230 0x7ff96c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.930+0000 7ff972c5f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96c0d1b90 con 0x7ff96c0b5d80 2023-12-01T01:36:59.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.930+0000 7ff972c5f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96c0b2160 con 0x7ff96c15cd20 2023-12-01T01:36:59.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.930+0000 7ff972c5f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96c0b8c80 con 0x7ff96c0d1230 2023-12-01T01:36:59.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.930+0000 7ff9709fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff96c15cd20 0x7ff96c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:59.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.930+0000 7ff96bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff96c0b5d80 0x7ff96c0b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34180/0 (socket says 172.21.15.115:34180) 2023-12-01T01:36:59.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.930+0000 7ff96bfff640 1 -- 172.21.15.115:0/1340788620 learned_addr learned my addr 172.21.15.115:0/1340788620 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:36:59.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.930+0000 7ff9709fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff96c15cd20 0x7ff96c0d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48138/0 (socket says 172.21.15.115:48138) 2023-12-01T01:36:59.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.931+0000 7ff9709fb640 1 -- 172.21.15.115:0/1340788620 >> v2:172.21.15.142:3300/0 conn(0x7ff96c0d1230 msgr2=0x7ff96c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:36:59.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.931+0000 7ff9709fb640 1 --2- 172.21.15.115:0/1340788620 >> v2:172.21.15.142:3300/0 conn(0x7ff96c0d1230 0x7ff96c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.931+0000 7ff9709fb640 1 -- 172.21.15.115:0/1340788620 >> v1:172.21.15.115:6789/0 conn(0x7ff96c0b5d80 legacy=0x7ff96c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:59.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.931+0000 7ff9709fb640 1 -- 172.21.15.115:0/1340788620 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c0ce2e0 con 0x7ff96c15cd20 2023-12-01T01:36:59.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.931+0000 7ff96b7fe640 1 -- 172.21.15.115:0/1340788620 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124840019 0 0) 0x7ff958001110 con 0x7ff96c0b5d80 2023-12-01T01:36:59.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.932+0000 7ff9709fb640 1 --2- 172.21.15.115:0/1340788620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c15cd20 0x7ff96c0d24d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0c6720 tx=0x7ff96002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6c2360d95e2ffa server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:59.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.932+0000 7ff96b7fe640 1 -- 172.21.15.115:0/1340788620 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96002e070 con 0x7ff96c15cd20 2023-12-01T01:36:59.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.932+0000 7ff96b7fe640 1 -- 172.21.15.115:0/1340788620 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9600315a0 con 0x7ff96c15cd20 2023-12-01T01:36:59.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.932+0000 7ff96b7fe640 1 -- 172.21.15.115:0/1340788620 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960031930 con 0x7ff96c15cd20 2023-12-01T01:36:59.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.933+0000 7ff972c5f640 1 -- 172.21.15.115:0/1340788620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c15cd20 msgr2=0x7ff96c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:59.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.933+0000 7ff972c5f640 1 --2- 172.21.15.115:0/1340788620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c15cd20 0x7ff96c0d24d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0c6720 tx=0x7ff96002b840 comp rx=0 tx=0).stop 2023-12-01T01:36:59.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.933+0000 7ff972c5f640 1 -- 172.21.15.115:0/1340788620 shutdown_connections 2023-12-01T01:36:59.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.933+0000 7ff972c5f640 1 --2- 172.21.15.115:0/1340788620 >> v2:172.21.15.142:3300/0 conn(0x7ff96c0d1230 0x7ff96c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.933+0000 7ff972c5f640 1 --2- 172.21.15.115:0/1340788620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c15cd20 0x7ff96c0d24d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.933+0000 7ff972c5f640 1 -- 172.21.15.115:0/1340788620 >> 172.21.15.115:0/1340788620 conn(0x7ff96c06a0e0 msgr2=0x7ff96c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:36:59.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.933+0000 7ff972c5f640 1 -- 172.21.15.115:0/1340788620 shutdown_connections 2023-12-01T01:36:59.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.933+0000 7ff972c5f640 1 -- 172.21.15.115:0/1340788620 wait complete. 2023-12-01T01:36:59.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.935+0000 7ff972c5f640 1 Processor -- start 2023-12-01T01:36:59.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.935+0000 7ff972c5f640 1 -- start start 2023-12-01T01:36:59.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.936+0000 7ff972c5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c0b5d80 0x7ff96c0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.936+0000 7ff972c5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0d1230 0x7ff96c1fa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.936+0000 7ff972c5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff96c15cd20 0x7ff96c06f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.936+0000 7ff972c5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c06fa80 con 0x7ff96c0d1230 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.936+0000 7ff972c5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c06fd60 con 0x7ff96c0b5d80 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.936+0000 7ff972c5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c1f9b80 con 0x7ff96c15cd20 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.936+0000 7ff96bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0d1230 0x7ff96c1fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.936+0000 7ff96bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0d1230 0x7ff96c1fa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48152/0 (socket says 172.21.15.115:48152) 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.936+0000 7ff96bfff640 1 -- 172.21.15.115:0/3552970369 learned_addr learned my addr 172.21.15.115:0/3552970369 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff9709fb640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c0b5d80 0x7ff96c0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff96bfff640 1 -- 172.21.15.115:0/3552970369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff96c15cd20 msgr2=0x7ff96c06f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff9711fc640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff96c15cd20 0x7ff96c06f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff96bfff640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff96c15cd20 0x7ff96c06f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff96bfff640 1 -- 172.21.15.115:0/3552970369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c0b5d80 msgr2=0x7ff96c0c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff96bfff640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c0b5d80 0x7ff96c0c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff96bfff640 1 -- 172.21.15.115:0/3552970369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960009400 con 0x7ff96c0d1230 2023-12-01T01:36:59.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff9709fb640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c0b5d80 0x7ff96c0c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:36:59.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff96bfff640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0d1230 0x7ff96c1fa380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c780 tx=0x7ff95800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:59.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff9697fa640 1 -- 172.21.15.115:0/3552970369 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580072d0 con 0x7ff96c0d1230 2023-12-01T01:36:59.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff9697fa640 1 -- 172.21.15.115:0/3552970369 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff958007450 con 0x7ff96c0d1230 2023-12-01T01:36:59.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff9697fa640 1 -- 172.21.15.115:0/3552970369 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580047e0 con 0x7ff96c0d1230 2023-12-01T01:36:59.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.937+0000 7ff972c5f640 1 -- 172.21.15.115:0/3552970369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c1f9e60 con 0x7ff96c0d1230 2023-12-01T01:36:59.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.938+0000 7ff972c5f640 1 -- 172.21.15.115:0/3552970369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c1fa8c0 con 0x7ff96c0d1230 2023-12-01T01:36:59.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.939+0000 7ff972c5f640 1 -- 172.21.15.115:0/3552970369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c1fa050 con 0x7ff96c0d1230 2023-12-01T01:36:59.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.940+0000 7ff9697fa640 1 -- 172.21.15.115:0/3552970369 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff958004960 con 0x7ff96c0d1230 2023-12-01T01:36:59.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.941+0000 7ff9697fa640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff95006a430 0x7ff95006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:36:59.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.941+0000 7ff9697fa640 1 -- 172.21.15.115:0/3552970369 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff958087af0 con 0x7ff96c0d1230 2023-12-01T01:36:59.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.941+0000 7ff9709fb640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff95006a430 0x7ff95006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:36:59.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.941+0000 7ff9709fb640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff95006a430 0x7ff95006c8f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff960036000 tx=0x7ff96003d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:36:59.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:36:59.945+0000 7ff9697fa640 1 -- 172.21.15.115:0/3552970369 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff9580530c0 con 0x7ff96c0d1230 2023-12-01T01:37:00.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.137+0000 7ff972c5f640 1 -- 172.21.15.115:0/3552970369 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff96c0d1b90 con 0x7ff96c0d1230 2023-12-01T01:37:00.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.140+0000 7ff9697fa640 1 -- 172.21.15.115:0/3552970369 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16138 (secure 0 0 0) 0x7ff96c0d1b90 con 0x7ff96c0d1230 2023-12-01T01:37:00.141 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:00.142 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":23,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:52.262397+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:37:00.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.142+0000 7ff972c5f640 1 -- 172.21.15.115:0/3552970369 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff95006a430 msgr2=0x7ff95006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.142+0000 7ff972c5f640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff95006a430 0x7ff95006c8f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff960036000 tx=0x7ff96003d040 comp rx=0 tx=0).stop 2023-12-01T01:37:00.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.142+0000 7ff972c5f640 1 -- 172.21.15.115:0/3552970369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0d1230 msgr2=0x7ff96c1fa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.142+0000 7ff972c5f640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0d1230 0x7ff96c1fa380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c780 tx=0x7ff95800cc40 comp rx=0 tx=0).stop 2023-12-01T01:37:00.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.143+0000 7ff972c5f640 1 -- 172.21.15.115:0/3552970369 shutdown_connections 2023-12-01T01:37:00.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.143+0000 7ff972c5f640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff95006a430 0x7ff95006c8f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.143+0000 7ff972c5f640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff96c15cd20 0x7ff96c06f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.143+0000 7ff972c5f640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0d1230 0x7ff96c1fa380 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.143+0000 7ff972c5f640 1 --2- 172.21.15.115:0/3552970369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c0b5d80 0x7ff96c0c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.143+0000 7ff972c5f640 1 -- 172.21.15.115:0/3552970369 >> 172.21.15.115:0/3552970369 conn(0x7ff96c06a0e0 msgr2=0x7ff96c15bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:00.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.143+0000 7ff972c5f640 1 -- 172.21.15.115:0/3552970369 shutdown_connections 2023-12-01T01:37:00.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.143+0000 7ff972c5f640 1 -- 172.21.15.115:0/3552970369 wait complete. 2023-12-01T01:37:00.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:37:00.163 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:37:00.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.369+0000 7f2e527cb640 1 Processor -- start 2023-12-01T01:37:00.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.369+0000 7f2e527cb640 1 -- start start 2023-12-01T01:37:00.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.371+0000 7f2e527cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 0x7f2e4c06b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.371+0000 7f2e527cb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2e4c06b7a0 0x7f2e4c0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.371+0000 7f2e527cb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c0ce1b0 con 0x7f2e4c0cbab0 2023-12-01T01:37:00.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.371+0000 7f2e527cb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c0cbf20 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.371+0000 7f2e527cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0ce490 con 0x7f2e4c143eb0 2023-12-01T01:37:00.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.371+0000 7f2e43fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2e4c06b7a0 0x7f2e4c0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:00.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.371+0000 7f2e43fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2e4c06b7a0 0x7f2e4c0cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48154/0 (socket says 172.21.15.115:48154) 2023-12-01T01:37:00.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.371+0000 7f2e43fff640 1 -- 172.21.15.115:0/2395471187 learned_addr learned my addr 172.21.15.115:0/2395471187 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:00.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.371+0000 7f2e4bfff640 1 --2- 172.21.15.115:0/2395471187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 0x7f2e4c06b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:00.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.372+0000 7f2e43fff640 1 -- 172.21.15.115:0/2395471187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 msgr2=0x7f2e4c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:00.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.372+0000 7f2e43fff640 1 --2- 172.21.15.115:0/2395471187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 0x7f2e4c06b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.372+0000 7f2e43fff640 1 -- 172.21.15.115:0/2395471187 >> v1:172.21.15.142:6789/0 conn(0x7f2e4c0cbab0 legacy=0x7f2e4c164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.372+0000 7f2e43fff640 1 -- 172.21.15.115:0/2395471187 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c0caa90 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.372+0000 7f2e4bfff640 1 --2- 172.21.15.115:0/2395471187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 0x7f2e4c06b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:37:00.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.373+0000 7f2e43fff640 1 --2- 172.21.15.115:0/2395471187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 0x7f2e4c0cb570 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009870 tx=0x7f2e3402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf1dded827d0b45 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:00.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.373+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/2395471187 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34030840 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.373+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/2395471187 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e340309a0 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.373+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/2395471187 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34030d30 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.374+0000 7f2e527cb640 1 -- 172.21.15.115:0/2395471187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 msgr2=0x7f2e4c0cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.374+0000 7f2e527cb640 1 --2- 172.21.15.115:0/2395471187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 0x7f2e4c0cb570 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009870 tx=0x7f2e3402b980 comp rx=0 tx=0).stop 2023-12-01T01:37:00.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.374+0000 7f2e527cb640 1 -- 172.21.15.115:0/2395471187 shutdown_connections 2023-12-01T01:37:00.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.374+0000 7f2e527cb640 1 --2- 172.21.15.115:0/2395471187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 0x7f2e4c0cb570 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.374+0000 7f2e527cb640 1 --2- 172.21.15.115:0/2395471187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 0x7f2e4c06b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.374+0000 7f2e527cb640 1 -- 172.21.15.115:0/2395471187 >> 172.21.15.115:0/2395471187 conn(0x7f2e4c065960 msgr2=0x7f2e4c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:00.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.374+0000 7f2e527cb640 1 -- 172.21.15.115:0/2395471187 shutdown_connections 2023-12-01T01:37:00.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.374+0000 7f2e527cb640 1 -- 172.21.15.115:0/2395471187 wait complete. 2023-12-01T01:37:00.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.376+0000 7f2e527cb640 1 Processor -- start 2023-12-01T01:37:00.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.376+0000 7f2e527cb640 1 -- start start 2023-12-01T01:37:00.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.377+0000 7f2e527cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 0x7f2e4c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.377+0000 7f2e527cb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e4c0cbab0 0x7f2e4c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.377+0000 7f2e527cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 0x7f2e4c1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.377+0000 7f2e527cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c13feb0 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.377+0000 7f2e527cb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c13a1b0 con 0x7f2e4c0cbab0 2023-12-01T01:37:00.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.377+0000 7f2e527cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c139910 con 0x7f2e4c143eb0 2023-12-01T01:37:00.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.377+0000 7f2e4bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 0x7f2e4c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:00.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.377+0000 7f2e4bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 0x7f2e4c0b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48160/0 (socket says 172.21.15.115:48160) 2023-12-01T01:37:00.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.377+0000 7f2e4bfff640 1 -- 172.21.15.115:0/1563311967 learned_addr learned my addr 172.21.15.115:0/1563311967 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:00.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e50d68640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 0x7f2e4c1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:00.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e43fff640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e4c0cbab0 0x7f2e4c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:00.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e4bfff640 1 -- 172.21.15.115:0/1563311967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 msgr2=0x7f2e4c1586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e4bfff640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 0x7f2e4c1586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e4bfff640 1 -- 172.21.15.115:0/1563311967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e4c0cbab0 msgr2=0x7f2e4c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e4bfff640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e4c0cbab0 0x7f2e4c156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e4bfff640 1 -- 172.21.15.115:0/1563311967 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340094d0 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e4bfff640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 0x7f2e4c0b2810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00d7d0 tx=0x7f2e3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:00.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e497fa640 1 -- 172.21.15.115:0/1563311967 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c004030 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e497fa640 1 -- 172.21.15.115:0/1563311967 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c004190 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e497fa640 1 -- 172.21.15.115:0/1563311967 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c004520 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e527cb640 1 -- 172.21.15.115:0/1563311967 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c139bf0 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.378+0000 7f2e527cb640 1 -- 172.21.15.115:0/1563311967 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c13e050 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.380+0000 7f2e527cb640 1 -- 172.21.15.115:0/1563311967 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c04e890 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.381+0000 7f2e497fa640 1 -- 172.21.15.115:0/1563311967 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2e3c00b6c0 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.381+0000 7f2e497fa640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e2c06a430 0x7f2e2c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.381+0000 7f2e497fa640 1 -- 172.21.15.115:0/1563311967 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2e3c088a60 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.382+0000 7f2e43fff640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e2c06a430 0x7f2e2c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:00.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.382+0000 7f2e43fff640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e2c06a430 0x7f2e2c06c8f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2e34036000 tx=0x7f2e3403b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:00.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.386+0000 7f2e497fa640 1 -- 172.21.15.115:0/1563311967 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2e3c0540c0 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.575+0000 7f2e527cb640 1 -- 172.21.15.115:0/1563311967 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e4c0b12a0 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.575+0000 7f2e497fa640 1 -- 172.21.15.115:0/1563311967 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e3c0576e0 con 0x7f2e4c06b7a0 2023-12-01T01:37:00.577 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:37:00.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 -- 172.21.15.115:0/1563311967 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e2c06a430 msgr2=0x7f2e2c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e2c06a430 0x7f2e2c06c8f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2e34036000 tx=0x7f2e3403b040 comp rx=0 tx=0).stop 2023-12-01T01:37:00.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 -- 172.21.15.115:0/1563311967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 msgr2=0x7f2e4c0b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 0x7f2e4c0b2810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00d7d0 tx=0x7f2e3c00dc90 comp rx=0 tx=0).stop 2023-12-01T01:37:00.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 -- 172.21.15.115:0/1563311967 shutdown_connections 2023-12-01T01:37:00.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e2c06a430 0x7f2e2c06c8f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c143eb0 0x7f2e4c1586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e4c0cbab0 0x7f2e4c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 --2- 172.21.15.115:0/1563311967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c06b7a0 0x7f2e4c0b2810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 -- 172.21.15.115:0/1563311967 >> 172.21.15.115:0/1563311967 conn(0x7f2e4c065960 msgr2=0x7f2e4c070620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:00.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 -- 172.21.15.115:0/1563311967 shutdown_connections 2023-12-01T01:37:00.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.578+0000 7f2e527cb640 1 -- 172.21.15.115:0/1563311967 wait complete. 2023-12-01T01:37:00.599 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:37:00.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.810+0000 7f547e02c640 1 Processor -- start 2023-12-01T01:37:00.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.810+0000 7f547e02c640 1 -- start start 2023-12-01T01:37:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.812+0000 7f547e02c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f547815cd90 0x7f54780cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.812+0000 7f547e02c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54780cfd60 0x7f5478161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.812+0000 7f547e02c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54780ce5e0 con 0x7f54780b5fa0 2023-12-01T01:37:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.812+0000 7f547e02c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54780d0ec0 con 0x7f547815cd90 2023-12-01T01:37:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.812+0000 7f547e02c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5478161950 con 0x7f54780cfd60 2023-12-01T01:37:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.812+0000 7f54777fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f547815cd90 0x7f54780cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.812+0000 7f54777fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f547815cd90 0x7f54780cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48166/0 (socket says 172.21.15.115:48166) 2023-12-01T01:37:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.812+0000 7f54777fe640 1 -- 172.21.15.115:0/3542735252 learned_addr learned my addr 172.21.15.115:0/3542735252 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:00.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.812+0000 7f546ffff640 1 --1- 172.21.15.115:0/3542735252 >> v1:172.21.15.115:6789/0 conn(0x7f54780b5fa0 0x7f54780cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34182/0 (socket says 172.21.15.115:34182) 2023-12-01T01:37:00.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.813+0000 7f54777fe640 1 -- 172.21.15.115:0/3542735252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54780cfd60 msgr2=0x7f5478161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:00.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.813+0000 7f54777fe640 1 --2- 172.21.15.115:0/3542735252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54780cfd60 0x7f5478161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.813+0000 7f54777fe640 1 -- 172.21.15.115:0/3542735252 >> v1:172.21.15.115:6789/0 conn(0x7f54780b5fa0 legacy=0x7f54780cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.813+0000 7f54777fe640 1 -- 172.21.15.115:0/3542735252 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54780ce300 con 0x7f547815cd90 2023-12-01T01:37:00.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.814+0000 7f54777fe640 1 --2- 172.21.15.115:0/3542735252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f547815cd90 0x7f54780cdd20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5468009870 tx=0x7f546802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacd4a044c3b23ba server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:00.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.814+0000 7f5476ffd640 1 -- 172.21.15.115:0/3542735252 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468030840 con 0x7f547815cd90 2023-12-01T01:37:00.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.814+0000 7f5476ffd640 1 -- 172.21.15.115:0/3542735252 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54680309a0 con 0x7f547815cd90 2023-12-01T01:37:00.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.814+0000 7f5476ffd640 1 -- 172.21.15.115:0/3542735252 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468030d30 con 0x7f547815cd90 2023-12-01T01:37:00.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.815+0000 7f547e02c640 1 -- 172.21.15.115:0/3542735252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f547815cd90 msgr2=0x7f54780cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.815+0000 7f547e02c640 1 --2- 172.21.15.115:0/3542735252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f547815cd90 0x7f54780cdd20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5468009870 tx=0x7f546802b980 comp rx=0 tx=0).stop 2023-12-01T01:37:00.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.815+0000 7f547e02c640 1 -- 172.21.15.115:0/3542735252 shutdown_connections 2023-12-01T01:37:00.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.815+0000 7f547e02c640 1 --2- 172.21.15.115:0/3542735252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54780cfd60 0x7f5478161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.815+0000 7f547e02c640 1 --2- 172.21.15.115:0/3542735252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f547815cd90 0x7f54780cdd20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.815+0000 7f547e02c640 1 -- 172.21.15.115:0/3542735252 >> 172.21.15.115:0/3542735252 conn(0x7f5478065960 msgr2=0x7f5478065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:00.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.815+0000 7f547e02c640 1 -- 172.21.15.115:0/3542735252 shutdown_connections 2023-12-01T01:37:00.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.815+0000 7f547e02c640 1 -- 172.21.15.115:0/3542735252 wait complete. 2023-12-01T01:37:00.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.817+0000 7f547e02c640 1 Processor -- start 2023-12-01T01:37:00.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.817+0000 7f547e02c640 1 -- start start 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f547e02c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780b5fa0 0x7f547815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f547e02c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54780cfd60 0x7f54781f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f547e02c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f547815cd90 0x7f54781f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f547e02c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54780c9590 con 0x7f54780b5fa0 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f547e02c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547815d9a0 con 0x7f54780cfd60 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f547e02c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547815dc80 con 0x7f547815cd90 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f54777fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780b5fa0 0x7f547815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f54777fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780b5fa0 0x7f547815d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48174/0 (socket says 172.21.15.115:48174) 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f54777fe640 1 -- 172.21.15.115:0/1536171077 learned_addr learned my addr 172.21.15.115:0/1536171077 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f54777fe640 1 -- 172.21.15.115:0/1536171077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f547815cd90 msgr2=0x7f54781f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f54777fe640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f547815cd90 0x7f54781f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f54777fe640 1 -- 172.21.15.115:0/1536171077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54780cfd60 msgr2=0x7f54781f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:00.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f54777fe640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54780cfd60 0x7f54781f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:00.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.818+0000 7f54777fe640 1 -- 172.21.15.115:0/1536171077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54680094d0 con 0x7f54780b5fa0 2023-12-01T01:37:00.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.819+0000 7f54777fe640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780b5fa0 0x7f547815d170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5468009870 tx=0x7f5468004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:00.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.819+0000 7f5474ff9640 1 -- 172.21.15.115:0/1536171077 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468039b60 con 0x7f54780b5fa0 2023-12-01T01:37:00.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.819+0000 7f5474ff9640 1 -- 172.21.15.115:0/1536171077 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5468039cc0 con 0x7f54780b5fa0 2023-12-01T01:37:00.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.819+0000 7f5474ff9640 1 -- 172.21.15.115:0/1536171077 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468043630 con 0x7f54780b5fa0 2023-12-01T01:37:00.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.819+0000 7f547e02c640 1 -- 172.21.15.115:0/1536171077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547815df60 con 0x7f54780b5fa0 2023-12-01T01:37:00.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.819+0000 7f547e02c640 1 -- 172.21.15.115:0/1536171077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547815e520 con 0x7f54780b5fa0 2023-12-01T01:37:00.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.820+0000 7f547e02c640 1 -- 172.21.15.115:0/1536171077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547804e890 con 0x7f54780b5fa0 2023-12-01T01:37:00.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.821+0000 7f5474ff9640 1 -- 172.21.15.115:0/1536171077 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5468042390 con 0x7f54780b5fa0 2023-12-01T01:37:00.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.822+0000 7f5474ff9640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f545806a430 0x7f545806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:00.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.822+0000 7f5474ff9640 1 -- 172.21.15.115:0/1536171077 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f54680aacd0 con 0x7f54780b5fa0 2023-12-01T01:37:00.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.822+0000 7f546ffff640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f545806a430 0x7f545806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:00.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.823+0000 7f546ffff640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f545806a430 0x7f545806c8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f547815f3c0 tx=0x7f54600079c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:00.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:00.826+0000 7f5474ff9640 1 -- 172.21.15.115:0/1536171077 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5468076200 con 0x7f54780b5fa0 2023-12-01T01:37:01.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.013+0000 7f547e02c640 1 -- 172.21.15.115:0/1536171077 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f54780d0a70 con 0x7f54780b5fa0 2023-12-01T01:37:01.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.014+0000 7f5474ff9640 1 -- 172.21.15.115:0/1536171077 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5468079820 con 0x7f54780b5fa0 2023-12-01T01:37:01.015 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:37:01.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.016+0000 7f547e02c640 1 -- 172.21.15.115:0/1536171077 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f545806a430 msgr2=0x7f545806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.016+0000 7f547e02c640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f545806a430 0x7f545806c8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f547815f3c0 tx=0x7f54600079c0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.016+0000 7f547e02c640 1 -- 172.21.15.115:0/1536171077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780b5fa0 msgr2=0x7f547815d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.016+0000 7f547e02c640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780b5fa0 0x7f547815d170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5468009870 tx=0x7f5468004730 comp rx=0 tx=0).stop 2023-12-01T01:37:01.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.016+0000 7f547e02c640 1 -- 172.21.15.115:0/1536171077 shutdown_connections 2023-12-01T01:37:01.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.016+0000 7f547e02c640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f545806a430 0x7f545806c8f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.016+0000 7f547e02c640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f547815cd90 0x7f54781f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.016+0000 7f547e02c640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54780cfd60 0x7f54781f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.016+0000 7f547e02c640 1 --2- 172.21.15.115:0/1536171077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780b5fa0 0x7f547815d170 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.016+0000 7f547e02c640 1 -- 172.21.15.115:0/1536171077 >> 172.21.15.115:0/1536171077 conn(0x7f5478065960 msgr2=0x7f54780cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.017+0000 7f547e02c640 1 -- 172.21.15.115:0/1536171077 shutdown_connections 2023-12-01T01:37:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.017+0000 7f547e02c640 1 -- 172.21.15.115:0/1536171077 wait complete. 2023-12-01T01:37:01.038 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:37:01.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.251+0000 7f7bf95a7640 1 Processor -- start 2023-12-01T01:37:01.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.251+0000 7f7bf95a7640 1 -- start start 2023-12-01T01:37:01.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.253+0000 7f7bf95a7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7bf415ac80 0x7f7bf40cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.253+0000 7f7bf95a7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bf40b0e60 0x7f7bf40b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.253+0000 7f7bf95a7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf40cc9a0 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.253+0000 7f7bf95a7640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf415b060 con 0x7f7bf415ac80 2023-12-01T01:37:01.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.253+0000 7f7bf95a7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf40b0ce0 con 0x7f7bf40b0e60 2023-12-01T01:37:01.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.254+0000 7f7bf2ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7bf415ac80 0x7f7bf40cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:01.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.254+0000 7f7bf2ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7bf415ac80 0x7f7bf40cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48178/0 (socket says 172.21.15.115:48178) 2023-12-01T01:37:01.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.254+0000 7f7bf2ffd640 1 -- 172.21.15.115:0/2032988221 learned_addr learned my addr 172.21.15.115:0/2032988221 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:01.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.254+0000 7f7bea7fc640 1 --2- 172.21.15.115:0/2032988221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bf40b0e60 0x7f7bf40b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:01.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.254+0000 7f7bf2ffd640 1 -- 172.21.15.115:0/2032988221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bf40b0e60 msgr2=0x7f7bf40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:01.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.254+0000 7f7bf2ffd640 1 --2- 172.21.15.115:0/2032988221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bf40b0e60 0x7f7bf40b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.254+0000 7f7bf2ffd640 1 -- 172.21.15.115:0/2032988221 >> v1:172.21.15.142:6789/0 conn(0x7f7bf40cdaa0 legacy=0x7f7bf406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:01.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.254+0000 7f7bf2ffd640 1 -- 172.21.15.115:0/2032988221 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf40cbd90 con 0x7f7bf415ac80 2023-12-01T01:37:01.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.255+0000 7f7bf2ffd640 1 --2- 172.21.15.115:0/2032988221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf415ac80 0x7f7bf40cce30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009870 tx=0x7f7be402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68dc400d84732035 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:01.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.256+0000 7f7bf27fc640 1 -- 172.21.15.115:0/2032988221 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4031640 con 0x7f7bf415ac80 2023-12-01T01:37:01.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.256+0000 7f7bf27fc640 1 -- 172.21.15.115:0/2032988221 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be40317a0 con 0x7f7bf415ac80 2023-12-01T01:37:01.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.256+0000 7f7bf27fc640 1 -- 172.21.15.115:0/2032988221 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4031b30 con 0x7f7bf415ac80 2023-12-01T01:37:01.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.256+0000 7f7bf95a7640 1 -- 172.21.15.115:0/2032988221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf415ac80 msgr2=0x7f7bf40cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.256+0000 7f7bf95a7640 1 --2- 172.21.15.115:0/2032988221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf415ac80 0x7f7bf40cce30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009870 tx=0x7f7be402b8f0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.256+0000 7f7bf95a7640 1 -- 172.21.15.115:0/2032988221 shutdown_connections 2023-12-01T01:37:01.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.256+0000 7f7bf95a7640 1 --2- 172.21.15.115:0/2032988221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bf40b0e60 0x7f7bf40b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.256+0000 7f7bf95a7640 1 --2- 172.21.15.115:0/2032988221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf415ac80 0x7f7bf40cce30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.256+0000 7f7bf95a7640 1 -- 172.21.15.115:0/2032988221 >> 172.21.15.115:0/2032988221 conn(0x7f7bf4065960 msgr2=0x7f7bf4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:01.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.257+0000 7f7bf95a7640 1 -- 172.21.15.115:0/2032988221 shutdown_connections 2023-12-01T01:37:01.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.257+0000 7f7bf95a7640 1 -- 172.21.15.115:0/2032988221 wait complete. 2023-12-01T01:37:01.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.258+0000 7f7bf95a7640 1 Processor -- start 2023-12-01T01:37:01.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.258+0000 7f7bf95a7640 1 -- start start 2023-12-01T01:37:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.259+0000 7f7bf95a7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bf40b0e60 0x7f7bf4154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.259+0000 7f7bf95a7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf40cdaa0 0x7f7bf4147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.259+0000 7f7bf95a7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bf415ac80 0x7f7bf41530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.259+0000 7f7bf95a7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf4147790 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.259+0000 7f7bf95a7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf4153a10 con 0x7f7bf40b0e60 2023-12-01T01:37:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.259+0000 7f7bf95a7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf4153cf0 con 0x7f7bf415ac80 2023-12-01T01:37:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.260+0000 7f7bea7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf40cdaa0 0x7f7bf4147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.260+0000 7f7bea7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf40cdaa0 0x7f7bf4147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48188/0 (socket says 172.21.15.115:48188) 2023-12-01T01:37:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.260+0000 7f7bea7fc640 1 -- 172.21.15.115:0/935774315 learned_addr learned my addr 172.21.15.115:0/935774315 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:01.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.260+0000 7f7bea7fc640 1 -- 172.21.15.115:0/935774315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bf415ac80 msgr2=0x7f7bf41530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.260+0000 7f7bea7fc640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bf415ac80 0x7f7bf41530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.260+0000 7f7bea7fc640 1 -- 172.21.15.115:0/935774315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bf40b0e60 msgr2=0x7f7bf4154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.260+0000 7f7bea7fc640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bf40b0e60 0x7f7bf4154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.260+0000 7f7bea7fc640 1 -- 172.21.15.115:0/935774315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40094d0 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.260+0000 7f7bea7fc640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf40cdaa0 0x7f7bf4147250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00d800 tx=0x7f7bdc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.261+0000 7f7bebfff640 1 -- 172.21.15.115:0/935774315 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00b6c0 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.261+0000 7f7bebfff640 1 -- 172.21.15.115:0/935774315 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bdc00b820 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.261+0000 7f7bebfff640 1 -- 172.21.15.115:0/935774315 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00bbb0 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.261+0000 7f7bf95a7640 1 -- 172.21.15.115:0/935774315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf4153fd0 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.261+0000 7f7bf95a7640 1 -- 172.21.15.115:0/935774315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf4154590 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.262+0000 7f7bf95a7640 1 -- 172.21.15.115:0/935774315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf404e890 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.263+0000 7f7bebfff640 1 -- 172.21.15.115:0/935774315 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7bdc00bd10 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.264+0000 7f7bebfff640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bd406a430 0x7f7bd406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.264+0000 7f7bebfff640 1 -- 172.21.15.115:0/935774315 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7bdc088df0 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.264+0000 7f7bf2ffd640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bd406a430 0x7f7bd406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:01.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.265+0000 7f7bf2ffd640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bd406a430 0x7f7bd406c8f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7be4036000 tx=0x7f7be4030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:01.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.268+0000 7f7bebfff640 1 -- 172.21.15.115:0/935774315 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7bdc054450 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.459+0000 7f7bf95a7640 1 -- 172.21.15.115:0/935774315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7bf40cf1b0 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.460+0000 7f7bebfff640 1 -- 172.21.15.115:0/935774315 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7bdc057a70 con 0x7f7bf40cdaa0 2023-12-01T01:37:01.462 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:37:01.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 -- 172.21.15.115:0/935774315 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bd406a430 msgr2=0x7f7bd406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bd406a430 0x7f7bd406c8f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7be4036000 tx=0x7f7be4030000 comp rx=0 tx=0).stop 2023-12-01T01:37:01.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 -- 172.21.15.115:0/935774315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf40cdaa0 msgr2=0x7f7bf4147250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf40cdaa0 0x7f7bf4147250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00d800 tx=0x7f7bdc00dcc0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 -- 172.21.15.115:0/935774315 shutdown_connections 2023-12-01T01:37:01.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bd406a430 0x7f7bd406c8f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bf415ac80 0x7f7bf41530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bf40cdaa0 0x7f7bf4147250 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 --2- 172.21.15.115:0/935774315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bf40b0e60 0x7f7bf4154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 -- 172.21.15.115:0/935774315 >> 172.21.15.115:0/935774315 conn(0x7f7bf4065960 msgr2=0x7f7bf414edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:01.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 -- 172.21.15.115:0/935774315 shutdown_connections 2023-12-01T01:37:01.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.463+0000 7f7bf95a7640 1 -- 172.21.15.115:0/935774315 wait complete. 2023-12-01T01:37:01.483 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:37:01.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.693+0000 7fa2b405f640 1 Processor -- start 2023-12-01T01:37:01.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.694+0000 7fa2b405f640 1 -- start start 2023-12-01T01:37:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.695+0000 7fa2b405f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa2ac15dfd0 0x7fa2ac0cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.695+0000 7fa2b405f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac160b20 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.695+0000 7fa2b405f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac160dc0 con 0x7fa2ac15dbf0 2023-12-01T01:37:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.695+0000 7fa2b405f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac1610a0 con 0x7fa2ac15dfd0 2023-12-01T01:37:01.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.696+0000 7fa2b25fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa2ac0cf9a0 0x7fa2ac0b7ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34190/0 (socket says 172.21.15.115:34190) 2023-12-01T01:37:01.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.696+0000 7fa2b25fc640 1 -- 172.21.15.115:0/765054750 learned_addr learned my addr 172.21.15.115:0/765054750 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:01.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.696+0000 7fa2b15fa640 1 --2- 172.21.15.115:0/765054750 >> v2:172.21.15.115:3300/0 conn(0x7fa2ac15dfd0 0x7fa2ac0cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:01.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.696+0000 7fa2b15fa640 1 -- 172.21.15.115:0/765054750 >> v1:172.21.15.142:6789/0 conn(0x7fa2ac15dbf0 legacy=0x7fa2ac0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.696+0000 7fa2b15fa640 1 -- 172.21.15.115:0/765054750 >> v1:172.21.15.115:6789/0 conn(0x7fa2ac0cf9a0 legacy=0x7fa2ac0b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.696+0000 7fa2b15fa640 1 -- 172.21.15.115:0/765054750 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac161380 con 0x7fa2ac15dfd0 2023-12-01T01:37:01.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.696+0000 7fa2b0df9640 1 -- 172.21.15.115:0/765054750 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392422361 0 0) 0x7fa2ac160b20 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.697+0000 7fa2b15fa640 1 --2- 172.21.15.115:0/765054750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac15dfd0 0x7fa2ac0cf460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00097a0 tx=0x7fa2a002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49888b99a0e938e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:01.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.697+0000 7fa2b0df9640 1 -- 172.21.15.115:0/765054750 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0030610 con 0x7fa2ac15dfd0 2023-12-01T01:37:01.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.698+0000 7fa2b0df9640 1 -- 172.21.15.115:0/765054750 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a0030770 con 0x7fa2ac15dfd0 2023-12-01T01:37:01.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.698+0000 7fa2b0df9640 1 -- 172.21.15.115:0/765054750 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0030b00 con 0x7fa2ac15dfd0 2023-12-01T01:37:01.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.698+0000 7fa2b405f640 1 -- 172.21.15.115:0/765054750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac15dfd0 msgr2=0x7fa2ac0cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.698+0000 7fa2b405f640 1 --2- 172.21.15.115:0/765054750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac15dfd0 0x7fa2ac0cf460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00097a0 tx=0x7fa2a002b960 comp rx=0 tx=0).stop 2023-12-01T01:37:01.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.698+0000 7fa2b405f640 1 -- 172.21.15.115:0/765054750 shutdown_connections 2023-12-01T01:37:01.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.698+0000 7fa2b405f640 1 --2- 172.21.15.115:0/765054750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac15dfd0 0x7fa2ac0cf460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.698+0000 7fa2b405f640 1 -- 172.21.15.115:0/765054750 >> 172.21.15.115:0/765054750 conn(0x7fa2ac065960 msgr2=0x7fa2ac065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:01.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.698+0000 7fa2b405f640 1 -- 172.21.15.115:0/765054750 shutdown_connections 2023-12-01T01:37:01.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.698+0000 7fa2b405f640 1 -- 172.21.15.115:0/765054750 wait complete. 2023-12-01T01:37:01.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.700+0000 7fa2b405f640 1 Processor -- start 2023-12-01T01:37:01.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.700+0000 7fa2b405f640 1 -- start start 2023-12-01T01:37:01.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.701+0000 7fa2b405f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac0cf9a0 0x7fa2ac157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.701+0000 7fa2b405f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2ac15dbf0 0x7fa2ac154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.701+0000 7fa2b405f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ac15dfd0 0x7fa2ac1f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.701+0000 7fa2b405f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac157da0 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.701+0000 7fa2b405f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1550e0 con 0x7fa2ac15dfd0 2023-12-01T01:37:01.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.701+0000 7fa2b405f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac1f2c70 con 0x7fa2ac15dbf0 2023-12-01T01:37:01.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.701+0000 7fa2b1dfb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac0cf9a0 0x7fa2ac157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:01.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.701+0000 7fa2b1dfb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac0cf9a0 0x7fa2ac157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48206/0 (socket says 172.21.15.115:48206) 2023-12-01T01:37:01.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.701+0000 7fa2b1dfb640 1 -- 172.21.15.115:0/1248446797 learned_addr learned my addr 172.21.15.115:0/1248446797 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:01.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa2b15fa640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2ac15dbf0 0x7fa2ac154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:01.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa2b1dfb640 1 -- 172.21.15.115:0/1248446797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2ac15dbf0 msgr2=0x7fa2ac154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa2b1dfb640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2ac15dbf0 0x7fa2ac154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa2b25fc640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ac15dfd0 0x7fa2ac1f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:01.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa2b1dfb640 1 -- 172.21.15.115:0/1248446797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ac15dfd0 msgr2=0x7fa2ac1f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa2b1dfb640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ac15dfd0 0x7fa2ac1f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa2b1dfb640 1 -- 172.21.15.115:0/1248446797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a0009400 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa2b1dfb640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac0cf9a0 0x7fa2ac157860 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800c450 tx=0x7fa2a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:01.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa29e7fc640 1 -- 172.21.15.115:0/1248446797 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80052d0 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa29e7fc640 1 -- 172.21.15.115:0/1248446797 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a8005430 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.702+0000 7fa29e7fc640 1 -- 172.21.15.115:0/1248446797 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8012870 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.703+0000 7fa2b405f640 1 -- 172.21.15.115:0/1248446797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ac1596a0 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.703+0000 7fa2b405f640 1 -- 172.21.15.115:0/1248446797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac159c60 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.704+0000 7fa2b405f640 1 -- 172.21.15.115:0/1248446797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac04e890 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.705+0000 7fa29e7fc640 1 -- 172.21.15.115:0/1248446797 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa2a80129d0 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.705+0000 7fa29e7fc640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa29006a430 0x7fa29006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:01.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.706+0000 7fa29e7fc640 1 -- 172.21.15.115:0/1248446797 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa2a8088410 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.706+0000 7fa2b15fa640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa29006a430 0x7fa29006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:01.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.706+0000 7fa2b15fa640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa29006a430 0x7fa29006c8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0036000 tx=0x7fa2a003b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:01.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.710+0000 7fa29e7fc640 1 -- 172.21.15.115:0/1248446797 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa2a8053a70 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.914+0000 7fa2b405f640 1 -- 172.21.15.115:0/1248446797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa2ac0d0960 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.915+0000 7fa29e7fc640 1 -- 172.21.15.115:0/1248446797 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa2a8057090 con 0x7fa2ac0cf9a0 2023-12-01T01:37:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:listed 0 entries 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 -- 172.21.15.115:0/1248446797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa29006a430 msgr2=0x7fa29006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa29006a430 0x7fa29006c8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0036000 tx=0x7fa2a003b040 comp rx=0 tx=0).stop 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 -- 172.21.15.115:0/1248446797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac0cf9a0 msgr2=0x7fa2ac157860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac0cf9a0 0x7fa2ac157860 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800c450 tx=0x7fa2a800c910 comp rx=0 tx=0).stop 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 -- 172.21.15.115:0/1248446797 shutdown_connections 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa29006a430 0x7fa29006c8f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2ac15dbf0 0x7fa2ac154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2ac15dfd0 0x7fa2ac1f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 --2- 172.21.15.115:0/1248446797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2ac0cf9a0 0x7fa2ac157860 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 -- 172.21.15.115:0/1248446797 >> 172.21.15.115:0/1248446797 conn(0x7fa2ac065960 msgr2=0x7fa2ac0d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 -- 172.21.15.115:0/1248446797 shutdown_connections 2023-12-01T01:37:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:01.918+0000 7fa2b405f640 1 -- 172.21.15.115:0/1248446797 wait complete. 2023-12-01T01:37:01.938 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-12-01T01:37:01.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:01.939 DEBUG:teuthology.orchestra.run.smithi142:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-12-01T01:37:02.046 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T01:37:02.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:02.046 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-01T01:37:02.112 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:37:02.112 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:37:02.172 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:37:02.172 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:37:02.223 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:37:02.224 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:37:02.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:37:02.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.487+0000 7f0bc3e70640 1 Processor -- start 2023-12-01T01:37:02.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.487+0000 7f0bc3e70640 1 -- start start 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc3e70640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0bbc15e2f0 0x7f0bbc0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc3e70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bbc0adf10 0x7f0bbc0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc3e70640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bbc0add10 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc3e70640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bbc0c5760 con 0x7f0bbc15e2f0 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc3e70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc15e6d0 con 0x7f0bbc0adf10 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc1c0c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0bbc15e2f0 0x7f0bbc0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc1c0c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0bbc15e2f0 0x7f0bbc0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48208/0 (socket says 172.21.15.115:48208) 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc1c0c640 1 -- 172.21.15.115:0/915389257 learned_addr learned my addr 172.21.15.115:0/915389257 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc1c0c640 1 -- 172.21.15.115:0/915389257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bbc0adf10 msgr2=0x7f0bbc0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc1c0c640 1 --2- 172.21.15.115:0/915389257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bbc0adf10 0x7f0bbc0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc1c0c640 1 -- 172.21.15.115:0/915389257 >> v1:172.21.15.142:6789/0 conn(0x7f0bbc06f2f0 legacy=0x7f0bbc0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.489+0000 7f0bc1c0c640 1 -- 172.21.15.115:0/915389257 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc0cf7b0 con 0x7f0bbc15e2f0 2023-12-01T01:37:02.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.490+0000 7f0bc1c0c640 1 --2- 172.21.15.115:0/915389257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc15e2f0 0x7f0bbc0ca660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc06ef60 tx=0x7f0bb802b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe189d380bb70c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:02.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.491+0000 7f0bc0c0a640 1 -- 172.21.15.115:0/915389257 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8031520 con 0x7f0bbc15e2f0 2023-12-01T01:37:02.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.491+0000 7f0bc0c0a640 1 -- 172.21.15.115:0/915389257 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb8031680 con 0x7f0bbc15e2f0 2023-12-01T01:37:02.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.491+0000 7f0bc0c0a640 1 -- 172.21.15.115:0/915389257 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8031a10 con 0x7f0bbc15e2f0 2023-12-01T01:37:02.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.491+0000 7f0bc3e70640 1 -- 172.21.15.115:0/915389257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc15e2f0 msgr2=0x7f0bbc0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:02.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.491+0000 7f0bc3e70640 1 --2- 172.21.15.115:0/915389257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc15e2f0 0x7f0bbc0ca660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc06ef60 tx=0x7f0bb802b7c0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.492+0000 7f0bc3e70640 1 -- 172.21.15.115:0/915389257 shutdown_connections 2023-12-01T01:37:02.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.492+0000 7f0bc3e70640 1 --2- 172.21.15.115:0/915389257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bbc0adf10 0x7f0bbc0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.492+0000 7f0bc3e70640 1 --2- 172.21.15.115:0/915389257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc15e2f0 0x7f0bbc0ca660 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.492+0000 7f0bc3e70640 1 -- 172.21.15.115:0/915389257 >> 172.21.15.115:0/915389257 conn(0x7f0bbc0ce880 msgr2=0x7f0bbc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:02.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.492+0000 7f0bc3e70640 1 -- 172.21.15.115:0/915389257 shutdown_connections 2023-12-01T01:37:02.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.492+0000 7f0bc3e70640 1 -- 172.21.15.115:0/915389257 wait complete. 2023-12-01T01:37:02.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.494+0000 7f0bc3e70640 1 Processor -- start 2023-12-01T01:37:02.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.494+0000 7f0bc3e70640 1 -- start start 2023-12-01T01:37:02.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.495+0000 7f0bc3e70640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc06f2f0 0x7f0bbc0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.495+0000 7f0bc3e70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bbc0adf10 0x7f0bbc1f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.495+0000 7f0bc3e70640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bbc15e2f0 0x7f0bbc06be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.495+0000 7f0bc3e70640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0c84c0 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.495+0000 7f0bc3e70640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0c6140 con 0x7f0bbc15e2f0 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.495+0000 7f0bc3e70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0c89d0 con 0x7f0bbc0adf10 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.495+0000 7f0bc1c0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc06f2f0 0x7f0bbc0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.495+0000 7f0bc1c0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc06f2f0 0x7f0bbc0c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48212/0 (socket says 172.21.15.115:48212) 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.496+0000 7f0bc1c0c640 1 -- 172.21.15.115:0/338655031 learned_addr learned my addr 172.21.15.115:0/338655031 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.496+0000 7f0bc1c0c640 1 -- 172.21.15.115:0/338655031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bbc0adf10 msgr2=0x7f0bbc1f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.496+0000 7f0bc1c0c640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bbc0adf10 0x7f0bbc1f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.496+0000 7f0bc1c0c640 1 -- 172.21.15.115:0/338655031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bbc15e2f0 msgr2=0x7f0bbc06be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.496+0000 7f0bc1c0c640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bbc15e2f0 0x7f0bbc06be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.496+0000 7f0bc1c0c640 1 -- 172.21.15.115:0/338655031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb80094d0 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.496+0000 7f0bc1c0c640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc06f2f0 0x7f0bbc0c7f80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8000c00 tx=0x7f0bb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:02.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.496+0000 7f0bb27fc640 1 -- 172.21.15.115:0/338655031 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8031520 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.497+0000 7f0bb27fc640 1 -- 172.21.15.115:0/338655031 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb8033c70 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.497+0000 7f0bc3e70640 1 -- 172.21.15.115:0/338655031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bbc0c86f0 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.497+0000 7f0bb27fc640 1 -- 172.21.15.115:0/338655031 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb803d470 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.497+0000 7f0bc3e70640 1 -- 172.21.15.115:0/338655031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bbc0c5dd0 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.498+0000 7f0bc3e70640 1 -- 172.21.15.115:0/338655031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bbc06fda0 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.499+0000 7f0bb27fc640 1 -- 172.21.15.115:0/338655031 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0bb803a040 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.500+0000 7f0bb27fc640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ba006a430 0x7f0ba006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.500+0000 7f0bb27fc640 1 -- 172.21.15.115:0/338655031 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0bb80aa650 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.500+0000 7f0bc140b640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ba006a430 0x7f0ba006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:02.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.501+0000 7f0bc140b640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ba006a430 0x7f0ba006c8f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc15db50 tx=0x7f0bac006ba0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:02.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.504+0000 7f0bb27fc640 1 -- 172.21.15.115:0/338655031 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0bb8075ca0 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.694+0000 7f0bc3e70640 1 -- 172.21.15.115:0/338655031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bbc0c5930 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.696+0000 7f0bb27fc640 1 -- 172.21.15.115:0/338655031 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16138 (secure 0 0 0) 0x7f0bb80792c0 con 0x7f0bbc06f2f0 2023-12-01T01:37:02.698 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:02.698 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":23,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:52.262397+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:37:02.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 -- 172.21.15.115:0/338655031 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ba006a430 msgr2=0x7f0ba006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ba006a430 0x7f0ba006c8f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc15db50 tx=0x7f0bac006ba0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 -- 172.21.15.115:0/338655031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc06f2f0 msgr2=0x7f0bbc0c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc06f2f0 0x7f0bbc0c7f80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8000c00 tx=0x7f0bb80046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 -- 172.21.15.115:0/338655031 shutdown_connections 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ba006a430 0x7f0ba006c8f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0bbc0adf10 0x7f0bbc1f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bbc15e2f0 0x7f0bbc06be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 --2- 172.21.15.115:0/338655031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bbc06f2f0 0x7f0bbc0c7f80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 -- 172.21.15.115:0/338655031 >> 172.21.15.115:0/338655031 conn(0x7f0bbc0ce880 msgr2=0x7f0bbc0cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 -- 172.21.15.115:0/338655031 shutdown_connections 2023-12-01T01:37:02.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.699+0000 7f0bc3e70640 1 -- 172.21.15.115:0/338655031 wait complete. 2023-12-01T01:37:02.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:37:02.721 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:37:02.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.939+0000 7f8df42ce640 1 Processor -- start 2023-12-01T01:37:02.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.939+0000 7f8df42ce640 1 -- start start 2023-12-01T01:37:02.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.940+0000 7f8df42ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0cfd60 0x7f8dec0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.940+0000 7f8df42ce640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8dec0b5fa0 0x7f8dec161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.940+0000 7f8df42ce640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dec0ce5e0 con 0x7f8dec15cd90 2023-12-01T01:37:02.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.940+0000 7f8df42ce640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dec0d0ec0 con 0x7f8dec0b5fa0 2023-12-01T01:37:02.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.940+0000 7f8df42ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec161950 con 0x7f8dec0cfd60 2023-12-01T01:37:02.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.941+0000 7f8df1869640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0cfd60 0x7f8dec0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:02.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.941+0000 7f8df286b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8dec0b5fa0 0x7f8dec161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:02.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.941+0000 7f8df206a640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f8dec15cd90 0x7f8dec0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:55172/0 (socket says 172.21.15.115:55172) 2023-12-01T01:37:02.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.941+0000 7f8df1869640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0cfd60 0x7f8dec0cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:46862/0 (socket says 172.21.15.115:46862) 2023-12-01T01:37:02.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.941+0000 7f8df206a640 1 -- 172.21.15.115:0/2009582979 learned_addr learned my addr 172.21.15.115:0/2009582979 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:02.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.942+0000 7f8df286b640 1 -- 172.21.15.115:0/2009582979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0cfd60 msgr2=0x7f8dec0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:02.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.942+0000 7f8df286b640 1 --2- 172.21.15.115:0/2009582979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0cfd60 0x7f8dec0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.942+0000 7f8df286b640 1 -- 172.21.15.115:0/2009582979 >> v1:172.21.15.142:6789/0 conn(0x7f8dec15cd90 legacy=0x7f8dec0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:02.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.942+0000 7f8df286b640 1 -- 172.21.15.115:0/2009582979 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec0ce300 con 0x7f8dec0b5fa0 2023-12-01T01:37:02.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.943+0000 7f8df286b640 1 --2- 172.21.15.115:0/2009582979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dec0b5fa0 0x7f8dec161410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8de40097a0 tx=0x7f8de402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9824158c717787b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:02.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.943+0000 7f8df1068640 1 -- 172.21.15.115:0/2009582979 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4031780 con 0x7f8dec0b5fa0 2023-12-01T01:37:02.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.943+0000 7f8df1068640 1 -- 172.21.15.115:0/2009582979 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de40318e0 con 0x7f8dec0b5fa0 2023-12-01T01:37:02.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.943+0000 7f8df1068640 1 -- 172.21.15.115:0/2009582979 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4031c70 con 0x7f8dec0b5fa0 2023-12-01T01:37:02.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.944+0000 7f8df42ce640 1 -- 172.21.15.115:0/2009582979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dec0b5fa0 msgr2=0x7f8dec161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:02.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.944+0000 7f8df42ce640 1 --2- 172.21.15.115:0/2009582979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dec0b5fa0 0x7f8dec161410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8de40097a0 tx=0x7f8de402b820 comp rx=0 tx=0).stop 2023-12-01T01:37:02.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.944+0000 7f8df42ce640 1 -- 172.21.15.115:0/2009582979 shutdown_connections 2023-12-01T01:37:02.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.944+0000 7f8df42ce640 1 --2- 172.21.15.115:0/2009582979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0cfd60 0x7f8dec0cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.944+0000 7f8df42ce640 1 --2- 172.21.15.115:0/2009582979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dec0b5fa0 0x7f8dec161410 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.944+0000 7f8df42ce640 1 -- 172.21.15.115:0/2009582979 >> 172.21.15.115:0/2009582979 conn(0x7f8dec065960 msgr2=0x7f8dec065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:02.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.944+0000 7f8df42ce640 1 -- 172.21.15.115:0/2009582979 shutdown_connections 2023-12-01T01:37:02.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.944+0000 7f8df42ce640 1 -- 172.21.15.115:0/2009582979 wait complete. 2023-12-01T01:37:02.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.946+0000 7f8df42ce640 1 Processor -- start 2023-12-01T01:37:02.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.946+0000 7f8df42ce640 1 -- start start 2023-12-01T01:37:02.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.947+0000 7f8df42ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0b5fa0 0x7f8dec0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.947+0000 7f8df42ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec0cfd60 0x7f8dec153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.947+0000 7f8df42ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dec15cd90 0x7f8dec0b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.947+0000 7f8df42ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec15aa70 con 0x7f8dec0cfd60 2023-12-01T01:37:02.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.947+0000 7f8df42ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec0ce070 con 0x7f8dec15cd90 2023-12-01T01:37:02.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.947+0000 7f8df42ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec1f21e0 con 0x7f8dec0b5fa0 2023-12-01T01:37:02.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.947+0000 7f8df1869640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec0cfd60 0x7f8dec153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:02.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.947+0000 7f8df1869640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec0cfd60 0x7f8dec153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48228/0 (socket says 172.21.15.115:48228) 2023-12-01T01:37:02.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.947+0000 7f8df1869640 1 -- 172.21.15.115:0/3798676254 learned_addr learned my addr 172.21.15.115:0/3798676254 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:02.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8df206a640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0b5fa0 0x7f8dec0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:02.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8df1869640 1 -- 172.21.15.115:0/3798676254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0b5fa0 msgr2=0x7f8dec0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:02.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8df286b640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dec15cd90 0x7f8dec0b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8df1869640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0b5fa0 0x7f8dec0d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8df1869640 1 -- 172.21.15.115:0/3798676254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dec15cd90 msgr2=0x7f8dec0b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8df1869640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dec15cd90 0x7f8dec0b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8df1869640 1 -- 172.21.15.115:0/3798676254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de4009400 con 0x7f8dec0cfd60 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8df1869640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec0cfd60 0x7f8dec153720 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e9b0 tx=0x7f8de000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8deaffd640 1 -- 172.21.15.115:0/3798676254 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0009670 con 0x7f8dec0cfd60 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8deaffd640 1 -- 172.21.15.115:0/3798676254 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de000c9a0 con 0x7f8dec0cfd60 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8df42ce640 1 -- 172.21.15.115:0/3798676254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dec159180 con 0x7f8dec0cfd60 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8deaffd640 1 -- 172.21.15.115:0/3798676254 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de000cd50 con 0x7f8dec0cfd60 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.948+0000 7f8df42ce640 1 -- 172.21.15.115:0/3798676254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dec15ed50 con 0x7f8dec0cfd60 2023-12-01T01:37:02.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.950+0000 7f8df42ce640 1 -- 172.21.15.115:0/3798676254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dec0cf610 con 0x7f8dec0cfd60 2023-12-01T01:37:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.951+0000 7f8deaffd640 1 -- 172.21.15.115:0/3798676254 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8de0013470 con 0x7f8dec0cfd60 2023-12-01T01:37:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.951+0000 7f8deaffd640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8dd806a430 0x7f8dd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.952+0000 7f8deaffd640 1 -- 172.21.15.115:0/3798676254 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8de0088b70 con 0x7f8dec0cfd60 2023-12-01T01:37:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.952+0000 7f8df206a640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8dd806a430 0x7f8dd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.952+0000 7f8df206a640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8dd806a430 0x7f8dd806c8f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0012e0 tx=0x7f8ddc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:02.956+0000 7f8deaffd640 1 -- 172.21.15.115:0/3798676254 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8de00541d0 con 0x7f8dec0cfd60 2023-12-01T01:37:03.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.147+0000 7f8df42ce640 1 -- 172.21.15.115:0/3798676254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8dec04e890 con 0x7f8dec0cfd60 2023-12-01T01:37:03.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.148+0000 7f8deaffd640 1 -- 172.21.15.115:0/3798676254 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8de00170a0 con 0x7f8dec0cfd60 2023-12-01T01:37:03.150 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:37:03.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 -- 172.21.15.115:0/3798676254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8dd806a430 msgr2=0x7f8dd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8dd806a430 0x7f8dd806c8f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0012e0 tx=0x7f8ddc009040 comp rx=0 tx=0).stop 2023-12-01T01:37:03.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 -- 172.21.15.115:0/3798676254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec0cfd60 msgr2=0x7f8dec153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec0cfd60 0x7f8dec153720 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e9b0 tx=0x7f8de000ee70 comp rx=0 tx=0).stop 2023-12-01T01:37:03.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 -- 172.21.15.115:0/3798676254 shutdown_connections 2023-12-01T01:37:03.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8dd806a430 0x7f8dd806c8f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dec15cd90 0x7f8dec0b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dec0cfd60 0x7f8dec153720 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 --2- 172.21.15.115:0/3798676254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dec0b5fa0 0x7f8dec0d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 -- 172.21.15.115:0/3798676254 >> 172.21.15.115:0/3798676254 conn(0x7f8dec065960 msgr2=0x7f8dec0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:03.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 -- 172.21.15.115:0/3798676254 shutdown_connections 2023-12-01T01:37:03.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.151+0000 7f8df42ce640 1 -- 172.21.15.115:0/3798676254 wait complete. 2023-12-01T01:37:03.172 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:37:03.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.383+0000 7fb5f2451640 1 Processor -- start 2023-12-01T01:37:03.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.384+0000 7fb5f2451640 1 -- start start 2023-12-01T01:37:03.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.385+0000 7fb5f2451640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb5ec0cb100 0x7fb5ec0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.385+0000 7fb5f2451640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec0d07b0 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.385+0000 7fb5f2451640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec0cb570 con 0x7fb5ec06b900 2023-12-01T01:37:03.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.385+0000 7fb5f2451640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec0d1890 con 0x7fb5ec0cb100 2023-12-01T01:37:03.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.385+0000 7fb5ebfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb5ec0ccca0 0x7fb5ec06a6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34194/0 (socket says 172.21.15.115:34194) 2023-12-01T01:37:03.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.385+0000 7fb5ebfff640 1 -- 172.21.15.115:0/2957210575 learned_addr learned my addr 172.21.15.115:0/2957210575 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:03.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.385+0000 7fb5f09ee640 1 --2- 172.21.15.115:0/2957210575 >> v2:172.21.15.115:3300/0 conn(0x7fb5ec0cb100 0x7fb5ec0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:03.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.386+0000 7fb5eb7fe640 1 -- 172.21.15.115:0/2957210575 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943079585 0 0) 0x7fb5ec0d07b0 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.386+0000 7fb5eb7fe640 1 -- 172.21.15.115:0/2957210575 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d8003520 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.386+0000 7fb5f09ee640 1 -- 172.21.15.115:0/2957210575 >> v1:172.21.15.142:6789/0 conn(0x7fb5ec06b900 legacy=0x7fb5ec0d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:03.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.386+0000 7fb5f09ee640 1 -- 172.21.15.115:0/2957210575 >> v1:172.21.15.115:6789/0 conn(0x7fb5ec0ccca0 legacy=0x7fb5ec06a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.386+0000 7fb5f09ee640 1 -- 172.21.15.115:0/2957210575 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec06af00 con 0x7fb5ec0cb100 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.387+0000 7fb5f09ee640 1 --2- 172.21.15.115:0/2957210575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0cb100 0x7fb5ec0d1350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009d90 tx=0x7fb5e402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110e51e04209df8b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.387+0000 7fb5eb7fe640 1 -- 172.21.15.115:0/2957210575 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e402cc60 con 0x7fb5ec0cb100 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.387+0000 7fb5eb7fe640 1 -- 172.21.15.115:0/2957210575 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e402cdc0 con 0x7fb5ec0cb100 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.387+0000 7fb5eb7fe640 1 -- 172.21.15.115:0/2957210575 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4034660 con 0x7fb5ec0cb100 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.388+0000 7fb5f2451640 1 -- 172.21.15.115:0/2957210575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0cb100 msgr2=0x7fb5ec0d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.388+0000 7fb5f2451640 1 --2- 172.21.15.115:0/2957210575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0cb100 0x7fb5ec0d1350 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009d90 tx=0x7fb5e402beb0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.388+0000 7fb5f2451640 1 -- 172.21.15.115:0/2957210575 shutdown_connections 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.388+0000 7fb5f2451640 1 --2- 172.21.15.115:0/2957210575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0cb100 0x7fb5ec0d1350 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.388+0000 7fb5f2451640 1 -- 172.21.15.115:0/2957210575 >> 172.21.15.115:0/2957210575 conn(0x7fb5ec0713d0 msgr2=0x7fb5ec0717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.388+0000 7fb5f2451640 1 -- 172.21.15.115:0/2957210575 shutdown_connections 2023-12-01T01:37:03.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.388+0000 7fb5f2451640 1 -- 172.21.15.115:0/2957210575 wait complete. 2023-12-01T01:37:03.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.390+0000 7fb5f2451640 1 Processor -- start 2023-12-01T01:37:03.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.390+0000 7fb5f2451640 1 -- start start 2023-12-01T01:37:03.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.391+0000 7fb5f2451640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5ec06b900 0x7fb5ec0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.391+0000 7fb5f2451640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5ec0cb100 0x7fb5ec1f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.391+0000 7fb5f2451640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0ccca0 0x7fb5ec0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.391+0000 7fb5f2451640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec1476d0 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.391+0000 7fb5f2451640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec147f70 con 0x7fb5ec06b900 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.391+0000 7fb5f2451640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0b34a0 con 0x7fb5ec0cb100 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.391+0000 7fb5f09ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0ccca0 0x7fb5ec0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.391+0000 7fb5f09ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0ccca0 0x7fb5ec0d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48236/0 (socket says 172.21.15.115:48236) 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.391+0000 7fb5f09ee640 1 -- 172.21.15.115:0/2216626646 learned_addr learned my addr 172.21.15.115:0/2216626646 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.392+0000 7fb5ebfff640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5ec06b900 0x7fb5ec0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.392+0000 7fb5e3fff640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5ec0cb100 0x7fb5ec1f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.392+0000 7fb5f09ee640 1 -- 172.21.15.115:0/2216626646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5ec0cb100 msgr2=0x7fb5ec1f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.392+0000 7fb5f09ee640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5ec0cb100 0x7fb5ec1f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.392+0000 7fb5f09ee640 1 -- 172.21.15.115:0/2216626646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5ec06b900 msgr2=0x7fb5ec0c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.392+0000 7fb5f09ee640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5ec06b900 0x7fb5ec0c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.392+0000 7fb5f09ee640 1 -- 172.21.15.115:0/2216626646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e4009a70 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.392+0000 7fb5f09ee640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0ccca0 0x7fb5ec0d0c40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4037040 tx=0x7fb5e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:03.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.392+0000 7fb5e97fa640 1 -- 172.21.15.115:0/2216626646 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4040650 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.393+0000 7fb5e97fa640 1 -- 172.21.15.115:0/2216626646 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e40407b0 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.393+0000 7fb5e97fa640 1 -- 172.21.15.115:0/2216626646 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4040b60 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.393+0000 7fb5f2451640 1 -- 172.21.15.115:0/2216626646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec0ca810 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.393+0000 7fb5f2451640 1 -- 172.21.15.115:0/2216626646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec147c00 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.394+0000 7fb5f2451640 1 -- 172.21.15.115:0/2216626646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ec06be50 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.395+0000 7fb5e97fa640 1 -- 172.21.15.115:0/2216626646 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb5e4040cc0 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.396+0000 7fb5e97fa640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5cc06a430 0x7fb5cc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.396+0000 7fb5e97fa640 1 -- 172.21.15.115:0/2216626646 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb5e40ab6d0 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.396+0000 7fb5ebfff640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5cc06a430 0x7fb5cc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:03.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.397+0000 7fb5ebfff640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5cc06a430 0x7fb5cc06c8f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0011e0 tx=0x7fb5dc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:03.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.400+0000 7fb5e97fa640 1 -- 172.21.15.115:0/2216626646 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb5e4076d50 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.586+0000 7fb5f2451640 1 -- 172.21.15.115:0/2216626646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5ec04e890 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.587+0000 7fb5e97fa640 1 -- 172.21.15.115:0/2216626646 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5e4009450 con 0x7fb5ec0ccca0 2023-12-01T01:37:03.589 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:37:03.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.589+0000 7fb5f2451640 1 -- 172.21.15.115:0/2216626646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5cc06a430 msgr2=0x7fb5cc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5cc06a430 0x7fb5cc06c8f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0011e0 tx=0x7fb5dc009040 comp rx=0 tx=0).stop 2023-12-01T01:37:03.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 -- 172.21.15.115:0/2216626646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0ccca0 msgr2=0x7fb5ec0d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0ccca0 0x7fb5ec0d0c40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4037040 tx=0x7fb5e4004a40 comp rx=0 tx=0).stop 2023-12-01T01:37:03.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 -- 172.21.15.115:0/2216626646 shutdown_connections 2023-12-01T01:37:03.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb5cc06a430 0x7fb5cc06c8f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5ec0cb100 0x7fb5ec1f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5ec0ccca0 0x7fb5ec0d0c40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 --2- 172.21.15.115:0/2216626646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5ec06b900 0x7fb5ec0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 -- 172.21.15.115:0/2216626646 >> 172.21.15.115:0/2216626646 conn(0x7fb5ec0713d0 msgr2=0x7fb5ec069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:03.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 -- 172.21.15.115:0/2216626646 shutdown_connections 2023-12-01T01:37:03.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.590+0000 7fb5f2451640 1 -- 172.21.15.115:0/2216626646 wait complete. 2023-12-01T01:37:03.607 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:37:03.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.800+0000 7f28cf2c4640 1 Processor -- start 2023-12-01T01:37:03.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.800+0000 7f28cf2c4640 1 -- start start 2023-12-01T01:37:03.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.802+0000 7f28cf2c4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f28c815cd90 0x7f28c80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.802+0000 7f28cf2c4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80ce5e0 con 0x7f28c80b5fa0 2023-12-01T01:37:03.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.802+0000 7f28cf2c4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80ce300 con 0x7f28c80cfd60 2023-12-01T01:37:03.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.802+0000 7f28cf2c4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c80d0ec0 con 0x7f28c815cd90 2023-12-01T01:37:03.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.802+0000 7f28cc85f640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f28c80b5fa0 0x7f28c80cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34208/0 (socket says 172.21.15.115:34208) 2023-12-01T01:37:03.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.802+0000 7f28cc85f640 1 -- 172.21.15.115:0/1164726088 learned_addr learned my addr 172.21.15.115:0/1164726088 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:03.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28cd060640 1 --2- 172.21.15.115:0/1164726088 >> v2:172.21.15.142:3300/0 conn(0x7f28c815cd90 0x7f28c80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:03.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469094356 0 0) 0x7f28b8001110 con 0x7f28c80b5fa0 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b4003520 con 0x7f28c80b5fa0 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2406081951 0 0) 0x7f28b4003520 con 0x7f28c80b5fa0 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 >> v2:172.21.15.142:3300/0 conn(0x7f28c815cd90 msgr2=0x7f28c80cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 --2- 172.21.15.115:0/1164726088 >> v2:172.21.15.142:3300/0 conn(0x7f28c815cd90 0x7f28c80cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 >> v1:172.21.15.142:6789/0 conn(0x7f28c80cfd60 legacy=0x7f28c80d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80d10a0 con 0x7f28c80b5fa0 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f28b8001800 con 0x7f28c80b5fa0 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 >> v1:172.21.15.115:6789/0 conn(0x7f28c80b5fa0 legacy=0x7f28c80cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 --2- 172.21.15.115:0/1164726088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28b40039c0 0x7f28b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 --2- 172.21.15.115:0/1164726088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b40047c0 0x7f28b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 --2- 172.21.15.115:0/1164726088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b40051d0 0x7f28b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80d0ec0 con 0x7f28b40039c0 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b400c120 con 0x7f28b40051d0 2023-12-01T01:37:03.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b400c2e0 con 0x7f28b40047c0 2023-12-01T01:37:03.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.803+0000 7f28cd060640 1 --2- 172.21.15.115:0/1164726088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28b40039c0 0x7f28b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:03.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.804+0000 7f28cd060640 1 -- 172.21.15.115:0/1164726088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b40047c0 msgr2=0x7f28b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:03.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.804+0000 7f28cd060640 1 --2- 172.21.15.115:0/1164726088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b40047c0 0x7f28b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.804+0000 7f28cd060640 1 -- 172.21.15.115:0/1164726088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b40051d0 msgr2=0x7f28b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:03.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.804+0000 7f28cd060640 1 --2- 172.21.15.115:0/1164726088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b40051d0 0x7f28b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.804+0000 7f28cd060640 1 -- 172.21.15.115:0/1164726088 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b4003520 con 0x7f28b40039c0 2023-12-01T01:37:03.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.804+0000 7f28cd861640 1 -- 172.21.15.115:0/1164726088 reap_dead start 2023-12-01T01:37:03.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.805+0000 7f28cd060640 1 --2- 172.21.15.115:0/1164726088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28b40039c0 0x7f28b4003de0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f28c400ba30 tx=0x7f28b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7c58fb517dcf7e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:03.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.805+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4032680 con 0x7f28b40039c0 2023-12-01T01:37:03.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.805+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c40327e0 con 0x7f28b40039c0 2023-12-01T01:37:03.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.805+0000 7f28bffff640 1 -- 172.21.15.115:0/1164726088 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4032b70 con 0x7f28b40039c0 2023-12-01T01:37:03.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.806+0000 7f28cf2c4640 1 -- 172.21.15.115:0/1164726088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28b40039c0 msgr2=0x7f28b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.806+0000 7f28cf2c4640 1 --2- 172.21.15.115:0/1164726088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28b40039c0 0x7f28b4003de0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f28c400ba30 tx=0x7f28b40057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.806+0000 7f28cf2c4640 1 -- 172.21.15.115:0/1164726088 shutdown_connections 2023-12-01T01:37:03.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.806+0000 7f28cf2c4640 1 --2- 172.21.15.115:0/1164726088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28b40039c0 0x7f28b4003de0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.806+0000 7f28cf2c4640 1 -- 172.21.15.115:0/1164726088 >> 172.21.15.115:0/1164726088 conn(0x7f28c8065960 msgr2=0x7f28c8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:03.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.806+0000 7f28cf2c4640 1 -- 172.21.15.115:0/1164726088 shutdown_connections 2023-12-01T01:37:03.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.806+0000 7f28cf2c4640 1 -- 172.21.15.115:0/1164726088 wait complete. 2023-12-01T01:37:03.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.807+0000 7f28cf2c4640 1 Processor -- start 2023-12-01T01:37:03.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.808+0000 7f28cf2c4640 1 -- start start 2023-12-01T01:37:03.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.808+0000 7f28cf2c4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b40039c0 0x7f28c8152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.808+0000 7f28cf2c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c815f940 0x7f28c815fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.808+0000 7f28cf2c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28c815b480 0x7f28c815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.808+0000 7f28cf2c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8160300 con 0x7f28c815f940 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.808+0000 7f28cf2c4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c81f38a0 con 0x7f28b40039c0 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.808+0000 7f28cf2c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c815c160 con 0x7f28c815b480 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.809+0000 7f28cc85f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c815f940 0x7f28c815fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.809+0000 7f28cc85f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c815f940 0x7f28c815fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48264/0 (socket says 172.21.15.115:48264) 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.809+0000 7f28cc85f640 1 -- 172.21.15.115:0/3370983938 learned_addr learned my addr 172.21.15.115:0/3370983938 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.809+0000 7f28cd861640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28c815b480 0x7f28c815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.809+0000 7f28cc85f640 1 -- 172.21.15.115:0/3370983938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28c815b480 msgr2=0x7f28c815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.809+0000 7f28cc85f640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28c815b480 0x7f28c815b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.809+0000 7f28cc85f640 1 -- 172.21.15.115:0/3370983938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b40039c0 msgr2=0x7f28c8152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.809+0000 7f28cc85f640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b40039c0 0x7f28c8152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.809+0000 7f28cc85f640 1 -- 172.21.15.115:0/3370983938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c81f5f30 con 0x7f28c815f940 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.809+0000 7f28cc85f640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c815f940 0x7f28c815fdc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f28b800c700 tx=0x7f28b800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.810+0000 7f28bdffb640 1 -- 172.21.15.115:0/3370983938 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80072d0 con 0x7f28c815f940 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.810+0000 7f28bdffb640 1 -- 172.21.15.115:0/3370983938 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b8007450 con 0x7f28c815f940 2023-12-01T01:37:03.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.810+0000 7f28bdffb640 1 -- 172.21.15.115:0/3370983938 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80047e0 con 0x7f28c815f940 2023-12-01T01:37:03.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.810+0000 7f28cf2c4640 1 -- 172.21.15.115:0/3370983938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c81f6210 con 0x7f28c815f940 2023-12-01T01:37:03.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.810+0000 7f28cf2c4640 1 -- 172.21.15.115:0/3370983938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c81f67d0 con 0x7f28c815f940 2023-12-01T01:37:03.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.811+0000 7f28cf2c4640 1 -- 172.21.15.115:0/3370983938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c804e890 con 0x7f28c815f940 2023-12-01T01:37:03.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.812+0000 7f28bdffb640 1 -- 172.21.15.115:0/3370983938 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f28b8004960 con 0x7f28c815f940 2023-12-01T01:37:03.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.812+0000 7f28bdffb640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ac06a460 0x7f28ac06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:03.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.813+0000 7f28bdffb640 1 -- 172.21.15.115:0/3370983938 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f28b8087bd0 con 0x7f28c815f940 2023-12-01T01:37:03.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.813+0000 7f28cd060640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ac06a460 0x7f28ac06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:03.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.813+0000 7f28cd060640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ac06a460 0x7f28ac06c920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28c400ba00 tx=0x7f28c4032000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:03.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:03.817+0000 7f28bdffb640 1 -- 172.21.15.115:0/3370983938 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f28b8056b90 con 0x7f28c815f940 2023-12-01T01:37:04.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.010+0000 7f28cf2c4640 1 -- 172.21.15.115:0/3370983938 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28c80ce5e0 con 0x7f28c815f940 2023-12-01T01:37:04.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.011+0000 7f28bdffb640 1 -- 172.21.15.115:0/3370983938 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28c80ce5e0 con 0x7f28c815f940 2023-12-01T01:37:04.013 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:37:04.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 -- 172.21.15.115:0/3370983938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ac06a460 msgr2=0x7f28ac06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:04.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ac06a460 0x7f28ac06c920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f28c400ba00 tx=0x7f28c4032000 comp rx=0 tx=0).stop 2023-12-01T01:37:04.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 -- 172.21.15.115:0/3370983938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c815f940 msgr2=0x7f28c815fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:04.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c815f940 0x7f28c815fdc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f28b800c700 tx=0x7f28b800cbc0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 -- 172.21.15.115:0/3370983938 shutdown_connections 2023-12-01T01:37:04.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ac06a460 0x7f28ac06c920 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28c815b480 0x7f28c815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c815f940 0x7f28c815fdc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 --2- 172.21.15.115:0/3370983938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b40039c0 0x7f28c8152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 -- 172.21.15.115:0/3370983938 >> 172.21.15.115:0/3370983938 conn(0x7f28c8065960 msgr2=0x7f28c80d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:04.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 -- 172.21.15.115:0/3370983938 shutdown_connections 2023-12-01T01:37:04.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.015+0000 7f28cf2c4640 1 -- 172.21.15.115:0/3370983938 wait complete. 2023-12-01T01:37:04.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:37:04.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.237+0000 7f156279f640 1 Processor -- start 2023-12-01T01:37:04.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.237+0000 7f156279f640 1 -- start start 2023-12-01T01:37:04.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.238+0000 7f156279f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c15e2f0 0x7f155c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:04.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.238+0000 7f156279f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f155c06f2f0 0x7f155c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:04.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.238+0000 7f156279f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f155c0add10 con 0x7f155c0adf10 2023-12-01T01:37:04.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.238+0000 7f156279f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155c0c5760 con 0x7f155c06f2f0 2023-12-01T01:37:04.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.238+0000 7f156279f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155c15e6d0 con 0x7f155c15e2f0 2023-12-01T01:37:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.239+0000 7f155bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c15e2f0 0x7f155c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.239+0000 7f155bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c15e2f0 0x7f155c0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:46902/0 (socket says 172.21.15.115:46902) 2023-12-01T01:37:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.239+0000 7f155bfff640 1 -- 172.21.15.115:0/1876003324 learned_addr learned my addr 172.21.15.115:0/1876003324 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.239+0000 7f155b7fe640 1 --2- 172.21.15.115:0/1876003324 >> v2:172.21.15.142:3300/0 conn(0x7f155c06f2f0 0x7f155c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.240+0000 7f155bfff640 1 -- 172.21.15.115:0/1876003324 >> v2:172.21.15.142:3300/0 conn(0x7f155c06f2f0 msgr2=0x7f155c0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.240+0000 7f155bfff640 1 --2- 172.21.15.115:0/1876003324 >> v2:172.21.15.142:3300/0 conn(0x7f155c06f2f0 0x7f155c0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.240+0000 7f155bfff640 1 -- 172.21.15.115:0/1876003324 >> v1:172.21.15.142:6789/0 conn(0x7f155c0adf10 legacy=0x7f155c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:04.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.240+0000 7f155affd640 1 -- 172.21.15.115:0/1876003324 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629348011 0 0) 0x7f155c0add10 con 0x7f155c0adf10 2023-12-01T01:37:04.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.240+0000 7f155bfff640 1 -- 172.21.15.115:0/1876003324 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c0cf7b0 con 0x7f155c15e2f0 2023-12-01T01:37:04.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.241+0000 7f155bfff640 1 --2- 172.21.15.115:0/1876003324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c15e2f0 0x7f155c0ca660 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f154c009870 tx=0x7f154c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=826257b303504407 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.242+0000 7f155affd640 1 -- 172.21.15.115:0/1876003324 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c0317c0 con 0x7f155c15e2f0 2023-12-01T01:37:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.242+0000 7f155affd640 1 -- 172.21.15.115:0/1876003324 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c031920 con 0x7f155c15e2f0 2023-12-01T01:37:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.242+0000 7f155affd640 1 -- 172.21.15.115:0/1876003324 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c031cb0 con 0x7f155c15e2f0 2023-12-01T01:37:04.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.242+0000 7f156279f640 1 -- 172.21.15.115:0/1876003324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c15e2f0 msgr2=0x7f155c0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:04.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.242+0000 7f156279f640 1 --2- 172.21.15.115:0/1876003324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c15e2f0 0x7f155c0ca660 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f154c009870 tx=0x7f154c02b860 comp rx=0 tx=0).stop 2023-12-01T01:37:04.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.243+0000 7f156279f640 1 -- 172.21.15.115:0/1876003324 shutdown_connections 2023-12-01T01:37:04.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.243+0000 7f156279f640 1 --2- 172.21.15.115:0/1876003324 >> v2:172.21.15.142:3300/0 conn(0x7f155c06f2f0 0x7f155c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.243+0000 7f156279f640 1 --2- 172.21.15.115:0/1876003324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c15e2f0 0x7f155c0ca660 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.243+0000 7f156279f640 1 -- 172.21.15.115:0/1876003324 >> 172.21.15.115:0/1876003324 conn(0x7f155c0ce880 msgr2=0x7f155c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:04.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.243+0000 7f156279f640 1 -- 172.21.15.115:0/1876003324 shutdown_connections 2023-12-01T01:37:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.243+0000 7f156279f640 1 -- 172.21.15.115:0/1876003324 wait complete. 2023-12-01T01:37:04.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.244+0000 7f156279f640 1 Processor -- start 2023-12-01T01:37:04.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.245+0000 7f156279f640 1 -- start start 2023-12-01T01:37:04.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.245+0000 7f156279f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c06f2f0 0x7f155c0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:04.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f156279f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f155c0adf10 0x7f155c15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:04.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f156279f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f155c15e2f0 0x7f155c0a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:04.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f156279f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c0c77d0 con 0x7f155c0adf10 2023-12-01T01:37:04.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f156279f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c0c6bc0 con 0x7f155c15e2f0 2023-12-01T01:37:04.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f156279f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155c0c6ea0 con 0x7f155c06f2f0 2023-12-01T01:37:04.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f155b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f155c0adf10 0x7f155c15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f155b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f155c0adf10 0x7f155c15a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48280/0 (socket says 172.21.15.115:48280) 2023-12-01T01:37:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f155b7fe640 1 -- 172.21.15.115:0/3727802860 learned_addr learned my addr 172.21.15.115:0/3727802860 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f155bfff640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c06f2f0 0x7f155c0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f1560d3c640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f155c15e2f0 0x7f155c0a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f155b7fe640 1 -- 172.21.15.115:0/3727802860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c06f2f0 msgr2=0x7f155c0c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f155b7fe640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c06f2f0 0x7f155c0c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f155b7fe640 1 -- 172.21.15.115:0/3727802860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f155c15e2f0 msgr2=0x7f155c0a9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f155b7fe640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f155c15e2f0 0x7f155c0a9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f155b7fe640 1 -- 172.21.15.115:0/3727802860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0094d0 con 0x7f155c0adf10 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.246+0000 7f1560d3c640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f155c15e2f0 0x7f155c0a9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.247+0000 7f155b7fe640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f155c0adf10 0x7f155c15a290 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f154400efc0 tx=0x7f154400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.247+0000 7f1558ff9640 1 -- 172.21.15.115:0/3727802860 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544009b80 con 0x7f155c0adf10 2023-12-01T01:37:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.247+0000 7f1558ff9640 1 -- 172.21.15.115:0/3727802860 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154400f040 con 0x7f155c0adf10 2023-12-01T01:37:04.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.247+0000 7f156279f640 1 -- 172.21.15.115:0/3727802860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155c0c7180 con 0x7f155c0adf10 2023-12-01T01:37:04.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.247+0000 7f1558ff9640 1 -- 172.21.15.115:0/3727802860 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15440146a0 con 0x7f155c0adf10 2023-12-01T01:37:04.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.247+0000 7f156279f640 1 -- 172.21.15.115:0/3727802860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155c15ad50 con 0x7f155c0adf10 2023-12-01T01:37:04.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.248+0000 7f156279f640 1 -- 172.21.15.115:0/3727802860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155c06fda0 con 0x7f155c0adf10 2023-12-01T01:37:04.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.249+0000 7f1558ff9640 1 -- 172.21.15.115:0/3727802860 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1544014800 con 0x7f155c0adf10 2023-12-01T01:37:04.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.250+0000 7f1558ff9640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f153c06a430 0x7f153c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:04.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.250+0000 7f1558ff9640 1 -- 172.21.15.115:0/3727802860 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1544089800 con 0x7f155c0adf10 2023-12-01T01:37:04.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.250+0000 7f155bfff640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f153c06a430 0x7f153c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:04.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.251+0000 7f155bfff640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f153c06a430 0x7f153c06c8f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f154c036000 tx=0x7f154c03d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:04.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.254+0000 7f1558ff9640 1 -- 172.21.15.115:0/3727802860 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1544054d30 con 0x7f155c0adf10 2023-12-01T01:37:04.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.459+0000 7f156279f640 1 -- 172.21.15.115:0/3727802860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f155c04e890 con 0x7f155c0adf10 2023-12-01T01:37:04.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.460+0000 7f1558ff9640 1 -- 172.21.15.115:0/3727802860 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1544058350 con 0x7f155c0adf10 2023-12-01T01:37:04.462 INFO:teuthology.orchestra.run.smithi115.stderr:listed 0 entries 2023-12-01T01:37:04.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.462+0000 7f156279f640 1 -- 172.21.15.115:0/3727802860 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f153c06a430 msgr2=0x7f153c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:04.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.462+0000 7f156279f640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f153c06a430 0x7f153c06c8f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f154c036000 tx=0x7f154c03d040 comp rx=0 tx=0).stop 2023-12-01T01:37:04.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.462+0000 7f156279f640 1 -- 172.21.15.115:0/3727802860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f155c0adf10 msgr2=0x7f155c15a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:04.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.462+0000 7f156279f640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f155c0adf10 0x7f155c15a290 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f154400efc0 tx=0x7f154400c490 comp rx=0 tx=0).stop 2023-12-01T01:37:04.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.463+0000 7f156279f640 1 -- 172.21.15.115:0/3727802860 shutdown_connections 2023-12-01T01:37:04.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.463+0000 7f156279f640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f153c06a430 0x7f153c06c8f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.463+0000 7f156279f640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f155c15e2f0 0x7f155c0a9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.463+0000 7f156279f640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f155c0adf10 0x7f155c15a290 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.463+0000 7f156279f640 1 --2- 172.21.15.115:0/3727802860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f155c06f2f0 0x7f155c0c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:04.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.463+0000 7f156279f640 1 -- 172.21.15.115:0/3727802860 >> 172.21.15.115:0/3727802860 conn(0x7f155c0ce880 msgr2=0x7f155c15d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:04.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.463+0000 7f156279f640 1 -- 172.21.15.115:0/3727802860 shutdown_connections 2023-12-01T01:37:04.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:04.463+0000 7f156279f640 1 -- 172.21.15.115:0/3727802860 wait complete. 2023-12-01T01:37:04.485 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-12-01T01:37:04.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:04.485 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-12-01T01:37:04.584 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:37:04.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:04.585 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-01T01:37:04.636 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:37:04.636 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:37:04.696 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:37:04.696 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:37:04.752 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:37:04.752 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:37:04.804 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:37:05.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.020+0000 7fdfc82f3640 1 Processor -- start 2023-12-01T01:37:05.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.020+0000 7fdfc82f3640 1 -- start start 2023-12-01T01:37:05.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.021+0000 7fdfc82f3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdfc00c9ef0 0x7fdfc006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.021+0000 7fdfc82f3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfc006b6f0 0x7fdfc0070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.021+0000 7fdfc82f3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc006f8b0 con 0x7fdfc0159dd0 2023-12-01T01:37:05.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.021+0000 7fdfc82f3640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc00adcb0 con 0x7fdfc00c9ef0 2023-12-01T01:37:05.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.021+0000 7fdfc82f3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc00ad9d0 con 0x7fdfc006b6f0 2023-12-01T01:37:05.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.022+0000 7fdfc588e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdfc00c9ef0 0x7fdfc006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.022+0000 7fdfc588e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdfc00c9ef0 0x7fdfc006fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48292/0 (socket says 172.21.15.115:48292) 2023-12-01T01:37:05.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.022+0000 7fdfc588e640 1 -- 172.21.15.115:0/2644961635 learned_addr learned my addr 172.21.15.115:0/2644961635 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:05.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.022+0000 7fdfc588e640 1 -- 172.21.15.115:0/2644961635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfc006b6f0 msgr2=0x7fdfc0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:05.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.022+0000 7fdfc588e640 1 --2- 172.21.15.115:0/2644961635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfc006b6f0 0x7fdfc0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.022+0000 7fdfc588e640 1 -- 172.21.15.115:0/2644961635 >> v1:172.21.15.142:6789/0 conn(0x7fdfc0159dd0 legacy=0x7fdfc00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:05.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.022+0000 7fdfc588e640 1 -- 172.21.15.115:0/2644961635 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc00c9000 con 0x7fdfc00c9ef0 2023-12-01T01:37:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.023+0000 7fdfc588e640 1 --2- 172.21.15.115:0/2644961635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc00c9ef0 0x7fdfc006fd20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009870 tx=0x7fdfb002d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2522a860782eb6c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.023+0000 7fdfc508d640 1 -- 172.21.15.115:0/2644961635 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0031810 con 0x7fdfc00c9ef0 2023-12-01T01:37:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.023+0000 7fdfc508d640 1 -- 172.21.15.115:0/2644961635 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb0031970 con 0x7fdfc00c9ef0 2023-12-01T01:37:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.024+0000 7fdfc508d640 1 -- 172.21.15.115:0/2644961635 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0031d00 con 0x7fdfc00c9ef0 2023-12-01T01:37:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.024+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2644961635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc00c9ef0 msgr2=0x7fdfc006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.024+0000 7fdfc82f3640 1 --2- 172.21.15.115:0/2644961635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc00c9ef0 0x7fdfc006fd20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009870 tx=0x7fdfb002d9f0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.024+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2644961635 shutdown_connections 2023-12-01T01:37:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.024+0000 7fdfc82f3640 1 --2- 172.21.15.115:0/2644961635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfc006b6f0 0x7fdfc0070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.024+0000 7fdfc82f3640 1 --2- 172.21.15.115:0/2644961635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc00c9ef0 0x7fdfc006fd20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.024+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2644961635 >> 172.21.15.115:0/2644961635 conn(0x7fdfc00ce880 msgr2=0x7fdfc00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.024+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2644961635 shutdown_connections 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.024+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2644961635 wait complete. 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.026+0000 7fdfc82f3640 1 Processor -- start 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.026+0000 7fdfc82f3640 1 -- start start 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.027+0000 7fdfc82f3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc006b6f0 0x7fdfc015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.027+0000 7fdfc82f3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc00c9ef0 0x7fdfc015a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.027+0000 7fdfc82f3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfc0159dd0 0x7fdfc0145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.027+0000 7fdfc82f3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0147fa0 con 0x7fdfc006b6f0 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.027+0000 7fdfc82f3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc015d350 con 0x7fdfc00c9ef0 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.027+0000 7fdfc82f3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0147670 con 0x7fdfc0159dd0 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.027+0000 7fdfc608f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc006b6f0 0x7fdfc015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.027+0000 7fdfc608f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc006b6f0 0x7fdfc015cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48300/0 (socket says 172.21.15.115:48300) 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.027+0000 7fdfc608f640 1 -- 172.21.15.115:0/2401830792 learned_addr learned my addr 172.21.15.115:0/2401830792 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:05.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfc588e640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc00c9ef0 0x7fdfc015a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfc608f640 1 -- 172.21.15.115:0/2401830792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfc0159dd0 msgr2=0x7fdfc0145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfc6890640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfc0159dd0 0x7fdfc0145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfc608f640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfc0159dd0 0x7fdfc0145b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfc608f640 1 -- 172.21.15.115:0/2401830792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc00c9ef0 msgr2=0x7fdfc015a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfc608f640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc00c9ef0 0x7fdfc015a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfc608f640 1 -- 172.21.15.115:0/2401830792 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb00094d0 con 0x7fdfc006b6f0 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfc588e640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc00c9ef0 0x7fdfc015a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfc608f640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc006b6f0 0x7fdfc015cd50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c430 tx=0x7fdfbc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfb6ffd640 1 -- 172.21.15.115:0/2401830792 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc011790 con 0x7fdfc006b6f0 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfb6ffd640 1 -- 172.21.15.115:0/2401830792 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc0118f0 con 0x7fdfc006b6f0 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfb6ffd640 1 -- 172.21.15.115:0/2401830792 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc011ce0 con 0x7fdfc006b6f0 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.028+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2401830792 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc0147950 con 0x7fdfc006b6f0 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.029+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2401830792 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc0154380 con 0x7fdfc006b6f0 2023-12-01T01:37:05.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.030+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2401830792 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc004e890 con 0x7fdfc006b6f0 2023-12-01T01:37:05.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.030+0000 7fdfb6ffd640 1 -- 172.21.15.115:0/2401830792 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdfbc01c460 con 0x7fdfc006b6f0 2023-12-01T01:37:05.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.031+0000 7fdfb6ffd640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdfa406a430 0x7fdfa406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.031+0000 7fdfb6ffd640 1 -- 172.21.15.115:0/2401830792 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fdfbc0888a0 con 0x7fdfc006b6f0 2023-12-01T01:37:05.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.031+0000 7fdfc588e640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdfa406a430 0x7fdfa406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.032+0000 7fdfc588e640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdfa406a430 0x7fdfa406c8f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0038000 tx=0x7fdfb003b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:05.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.035+0000 7fdfb6ffd640 1 -- 172.21.15.115:0/2401830792 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdfbc053e70 con 0x7fdfc006b6f0 2023-12-01T01:37:05.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.227+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2401830792 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfc0070200 con 0x7fdfc006b6f0 2023-12-01T01:37:05.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.231+0000 7fdfb6ffd640 1 -- 172.21.15.115:0/2401830792 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16138 (secure 0 0 0) 0x7fdfbc057490 con 0x7fdfc006b6f0 2023-12-01T01:37:05.233 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:05.233 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":23,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:52.262397+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:37:05.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.233+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2401830792 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdfa406a430 msgr2=0x7fdfa406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.233+0000 7fdfc82f3640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdfa406a430 0x7fdfa406c8f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0038000 tx=0x7fdfb003b040 comp rx=0 tx=0).stop 2023-12-01T01:37:05.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.233+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2401830792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc006b6f0 msgr2=0x7fdfc015cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.233+0000 7fdfc82f3640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc006b6f0 0x7fdfc015cd50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00c430 tx=0x7fdfbc00c8f0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.233+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2401830792 shutdown_connections 2023-12-01T01:37:05.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.233+0000 7fdfc82f3640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdfa406a430 0x7fdfa406c8f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.233+0000 7fdfc82f3640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfc0159dd0 0x7fdfc0145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.233+0000 7fdfc82f3640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc00c9ef0 0x7fdfc015a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.233+0000 7fdfc82f3640 1 --2- 172.21.15.115:0/2401830792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfc006b6f0 0x7fdfc015cd50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.233+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2401830792 >> 172.21.15.115:0/2401830792 conn(0x7fdfc00ce880 msgr2=0x7fdfc00ca2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:05.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.234+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2401830792 shutdown_connections 2023-12-01T01:37:05.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.234+0000 7fdfc82f3640 1 -- 172.21.15.115:0/2401830792 wait complete. 2023-12-01T01:37:05.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:37:05.256 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:37:05.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.459+0000 7fbbb2fc6640 1 Processor -- start 2023-12-01T01:37:05.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.460+0000 7fbbb2fc6640 1 -- start start 2023-12-01T01:37:05.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.461+0000 7fbbb2fc6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 0x7fbbac0d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.461+0000 7fbbb2fc6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbbac0d0f70 0x7fbbac0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.461+0000 7fbbb2fc6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbac0b6ab0 con 0x7fbbac0cf900 2023-12-01T01:37:05.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.461+0000 7fbbb2fc6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbac0cfd90 con 0x7fbbac0d0f70 2023-12-01T01:37:05.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.461+0000 7fbbb2fc6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac15d070 con 0x7fbbac15cc90 2023-12-01T01:37:05.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.462+0000 7fbbb1fc4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 0x7fbbac0d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.462+0000 7fbbb1fc4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 0x7fbbac0d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:46938/0 (socket says 172.21.15.115:46938) 2023-12-01T01:37:05.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.462+0000 7fbbb1fc4640 1 -- 172.21.15.115:0/2970692835 learned_addr learned my addr 172.21.15.115:0/2970692835 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:05.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.462+0000 7fbbb17c3640 1 --2- 172.21.15.115:0/2970692835 >> v2:172.21.15.142:3300/0 conn(0x7fbbac0d0f70 0x7fbbac0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.462+0000 7fbbb1fc4640 1 -- 172.21.15.115:0/2970692835 >> v2:172.21.15.142:3300/0 conn(0x7fbbac0d0f70 msgr2=0x7fbbac0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:05.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.462+0000 7fbbb1fc4640 1 --2- 172.21.15.115:0/2970692835 >> v2:172.21.15.142:3300/0 conn(0x7fbbac0d0f70 0x7fbbac0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.462+0000 7fbbb1fc4640 1 -- 172.21.15.115:0/2970692835 >> v1:172.21.15.142:6789/0 conn(0x7fbbac0cf900 legacy=0x7fbbac0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.462+0000 7fbbb1fc4640 1 -- 172.21.15.115:0/2970692835 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac0d15a0 con 0x7fbbac15cc90 2023-12-01T01:37:05.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.464+0000 7fbbb1fc4640 1 --2- 172.21.15.115:0/2970692835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 0x7fbbac0d0a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009870 tx=0x7fbba802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47e939b72230a60 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:05.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.464+0000 7fbbb0fc2640 1 -- 172.21.15.115:0/2970692835 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8030840 con 0x7fbbac15cc90 2023-12-01T01:37:05.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.464+0000 7fbbb0fc2640 1 -- 172.21.15.115:0/2970692835 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba80309a0 con 0x7fbbac15cc90 2023-12-01T01:37:05.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.464+0000 7fbbb0fc2640 1 -- 172.21.15.115:0/2970692835 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8030d30 con 0x7fbbac15cc90 2023-12-01T01:37:05.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.465+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/2970692835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 msgr2=0x7fbbac0d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.465+0000 7fbbb2fc6640 1 --2- 172.21.15.115:0/2970692835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 0x7fbbac0d0a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009870 tx=0x7fbba802b980 comp rx=0 tx=0).stop 2023-12-01T01:37:05.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.465+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/2970692835 shutdown_connections 2023-12-01T01:37:05.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.465+0000 7fbbb2fc6640 1 --2- 172.21.15.115:0/2970692835 >> v2:172.21.15.142:3300/0 conn(0x7fbbac0d0f70 0x7fbbac0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.465+0000 7fbbb2fc6640 1 --2- 172.21.15.115:0/2970692835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 0x7fbbac0d0a30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.465+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/2970692835 >> 172.21.15.115:0/2970692835 conn(0x7fbbac06a0e0 msgr2=0x7fbbac06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:05.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.465+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/2970692835 shutdown_connections 2023-12-01T01:37:05.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.465+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/2970692835 wait complete. 2023-12-01T01:37:05.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.467+0000 7fbbb2fc6640 1 Processor -- start 2023-12-01T01:37:05.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.467+0000 7fbbb2fc6640 1 -- start start 2023-12-01T01:37:05.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.468+0000 7fbbb2fc6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbac0cf900 0x7fbbac0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.468+0000 7fbbb2fc6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0d0f70 0x7fbbac145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.468+0000 7fbbb2fc6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 0x7fbbac15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.468+0000 7fbbb2fc6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac1431b0 con 0x7fbbac0d0f70 2023-12-01T01:37:05.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.468+0000 7fbbb2fc6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac0ccba0 con 0x7fbbac0cf900 2023-12-01T01:37:05.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.468+0000 7fbbb2fc6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac0cc8c0 con 0x7fbbac15cc90 2023-12-01T01:37:05.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.468+0000 7fbbb17c3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0d0f70 0x7fbbac145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.468+0000 7fbbb17c3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0d0f70 0x7fbbac145320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48316/0 (socket says 172.21.15.115:48316) 2023-12-01T01:37:05.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.468+0000 7fbbb17c3640 1 -- 172.21.15.115:0/1380760109 learned_addr learned my addr 172.21.15.115:0/1380760109 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.469+0000 7fbbb27c5640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 0x7fbbac15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.469+0000 7fbbb1fc4640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbac0cf900 0x7fbbac0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.469+0000 7fbbb17c3640 1 -- 172.21.15.115:0/1380760109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 msgr2=0x7fbbac15b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.469+0000 7fbbb17c3640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 0x7fbbac15b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.469+0000 7fbbb17c3640 1 -- 172.21.15.115:0/1380760109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbac0cf900 msgr2=0x7fbbac0c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.469+0000 7fbbb17c3640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbac0cf900 0x7fbbac0c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.469+0000 7fbbb17c3640 1 -- 172.21.15.115:0/1380760109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80094d0 con 0x7fbbac0d0f70 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.469+0000 7fbbb1fc4640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbac0cf900 0x7fbbac0c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.469+0000 7fbbb17c3640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0d0f70 0x7fbbac145320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbba000e9b0 tx=0x7fbba000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.470+0000 7fbb9e7fc640 1 -- 172.21.15.115:0/1380760109 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0009670 con 0x7fbbac0d0f70 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.470+0000 7fbb9e7fc640 1 -- 172.21.15.115:0/1380760109 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba000c9a0 con 0x7fbbac0d0f70 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.470+0000 7fbb9e7fc640 1 -- 172.21.15.115:0/1380760109 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba000cd50 con 0x7fbbac0d0f70 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.470+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/1380760109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac142b60 con 0x7fbbac0d0f70 2023-12-01T01:37:05.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.470+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/1380760109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac1522d0 con 0x7fbbac0d0f70 2023-12-01T01:37:05.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.471+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/1380760109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac143320 con 0x7fbbac0d0f70 2023-12-01T01:37:05.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.472+0000 7fbb9e7fc640 1 -- 172.21.15.115:0/1380760109 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbba0013470 con 0x7fbbac0d0f70 2023-12-01T01:37:05.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.473+0000 7fbb9e7fc640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb9006a460 0x7fbb9006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.473+0000 7fbb9e7fc640 1 -- 172.21.15.115:0/1380760109 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbba0088c10 con 0x7fbbac0d0f70 2023-12-01T01:37:05.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.473+0000 7fbbb1fc4640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb9006a460 0x7fbb9006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.474+0000 7fbbb1fc4640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb9006a460 0x7fbb9006c920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbba8036000 tx=0x7fbba803b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:05.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.481+0000 7fbb9e7fc640 1 -- 172.21.15.115:0/1380760109 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbba00541d0 con 0x7fbbac0d0f70 2023-12-01T01:37:05.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.675+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/1380760109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbac0618d0 con 0x7fbbac0d0f70 2023-12-01T01:37:05.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.676+0000 7fbb9e7fc640 1 -- 172.21.15.115:0/1380760109 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbba00170a0 con 0x7fbbac0d0f70 2023-12-01T01:37:05.677 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:37:05.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/1380760109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb9006a460 msgr2=0x7fbb9006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb9006a460 0x7fbb9006c920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbba8036000 tx=0x7fbba803b040 comp rx=0 tx=0).stop 2023-12-01T01:37:05.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/1380760109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0d0f70 msgr2=0x7fbbac145320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0d0f70 0x7fbbac145320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbba000e9b0 tx=0x7fbba000ee70 comp rx=0 tx=0).stop 2023-12-01T01:37:05.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/1380760109 shutdown_connections 2023-12-01T01:37:05.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb9006a460 0x7fbb9006c920 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbac15cc90 0x7fbbac15b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbac0d0f70 0x7fbbac145320 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 --2- 172.21.15.115:0/1380760109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbac0cf900 0x7fbbac0c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/1380760109 >> 172.21.15.115:0/1380760109 conn(0x7fbbac06a0e0 msgr2=0x7fbbac15ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:05.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/1380760109 shutdown_connections 2023-12-01T01:37:05.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.678+0000 7fbbb2fc6640 1 -- 172.21.15.115:0/1380760109 wait complete. 2023-12-01T01:37:05.697 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:37:05.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.884+0000 7f5ed2ebd640 1 Processor -- start 2023-12-01T01:37:05.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.884+0000 7f5ed2ebd640 1 -- start start 2023-12-01T01:37:05.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.886+0000 7f5ed2ebd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.886+0000 7f5ed2ebd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5ecc0b5fa0 0x7f5ecc161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.886+0000 7f5ed2ebd640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ecc0ce5e0 con 0x7f5ecc0cf650 2023-12-01T01:37:05.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.886+0000 7f5ed2ebd640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ecc0d0ec0 con 0x7f5ecc0b5fa0 2023-12-01T01:37:05.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.886+0000 7f5ed2ebd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc161950 con 0x7f5ecc15cd90 2023-12-01T01:37:05.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.886+0000 7f5ed0c59640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.886+0000 7f5ed0c59640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:46960/0 (socket says 172.21.15.115:46960) 2023-12-01T01:37:05.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.886+0000 7f5ed0c59640 1 -- 172.21.15.115:0/1371911587 learned_addr learned my addr 172.21.15.115:0/1371911587 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:05.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.886+0000 7f5ecbfff640 1 --2- 172.21.15.115:0/1371911587 >> v2:172.21.15.142:3300/0 conn(0x7f5ecc0b5fa0 0x7f5ecc161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.887+0000 7f5ed0c59640 1 -- 172.21.15.115:0/1371911587 >> v2:172.21.15.142:3300/0 conn(0x7f5ecc0b5fa0 msgr2=0x7f5ecc161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:05.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.887+0000 7f5ed0c59640 1 --2- 172.21.15.115:0/1371911587 >> v2:172.21.15.142:3300/0 conn(0x7f5ecc0b5fa0 0x7f5ecc161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.887+0000 7f5ed0c59640 1 -- 172.21.15.115:0/1371911587 >> v1:172.21.15.142:6789/0 conn(0x7f5ecc0cf650 legacy=0x7f5ecc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.887+0000 7f5ed0c59640 1 -- 172.21.15.115:0/1371911587 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc0ce300 con 0x7f5ecc15cd90 2023-12-01T01:37:05.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.889+0000 7f5ed0c59640 1 --2- 172.21.15.115:0/1371911587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0cdd20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009870 tx=0x7f5ebc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10515eeb5b1844f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:05.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.889+0000 7f5ecb7fe640 1 -- 172.21.15.115:0/1371911587 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc030840 con 0x7f5ecc15cd90 2023-12-01T01:37:05.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.889+0000 7f5ecb7fe640 1 -- 172.21.15.115:0/1371911587 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc0309a0 con 0x7f5ecc15cd90 2023-12-01T01:37:05.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.889+0000 7f5ecb7fe640 1 -- 172.21.15.115:0/1371911587 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc030d30 con 0x7f5ecc15cd90 2023-12-01T01:37:05.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.890+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/1371911587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 msgr2=0x7f5ecc0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.890+0000 7f5ed2ebd640 1 --2- 172.21.15.115:0/1371911587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0cdd20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009870 tx=0x7f5ebc02b980 comp rx=0 tx=0).stop 2023-12-01T01:37:05.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.890+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/1371911587 shutdown_connections 2023-12-01T01:37:05.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.890+0000 7f5ed2ebd640 1 --2- 172.21.15.115:0/1371911587 >> v2:172.21.15.142:3300/0 conn(0x7f5ecc0b5fa0 0x7f5ecc161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.890+0000 7f5ed2ebd640 1 --2- 172.21.15.115:0/1371911587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0cdd20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.890+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/1371911587 >> 172.21.15.115:0/1371911587 conn(0x7f5ecc065960 msgr2=0x7f5ecc065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:05.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.890+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/1371911587 shutdown_connections 2023-12-01T01:37:05.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.890+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/1371911587 wait complete. 2023-12-01T01:37:05.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.891+0000 7f5ed2ebd640 1 Processor -- start 2023-12-01T01:37:05.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.892+0000 7f5ed2ebd640 1 -- start start 2023-12-01T01:37:05.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ed2ebd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ecc0b5fa0 0x7f5ecc157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ed2ebd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ecc0cf650 0x7f5ecc15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ed2ebd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ed2ebd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0b5380 con 0x7f5ecc0cf650 2023-12-01T01:37:05.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ed2ebd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc15ec30 con 0x7f5ecc0b5fa0 2023-12-01T01:37:05.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ed2ebd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc15ef10 con 0x7f5ecc15cd90 2023-12-01T01:37:05.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ecc0cf650 0x7f5ecc15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ecc0cf650 0x7f5ecc15fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48320/0 (socket says 172.21.15.115:48320) 2023-12-01T01:37:05.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ecbfff640 1 -- 172.21.15.115:0/873114914 learned_addr learned my addr 172.21.15.115:0/873114914 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:05.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ed145a640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.893+0000 7f5ed0c59640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ecc0b5fa0 0x7f5ecc157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ecbfff640 1 -- 172.21.15.115:0/873114914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 msgr2=0x7f5ecc0a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ecbfff640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ecbfff640 1 -- 172.21.15.115:0/873114914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ecc0b5fa0 msgr2=0x7f5ecc157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:05.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ecbfff640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ecc0b5fa0 0x7f5ecc157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:05.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ecbfff640 1 -- 172.21.15.115:0/873114914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc0094d0 con 0x7f5ecc0cf650 2023-12-01T01:37:05.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ed145a640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:37:05.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ecbfff640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ecc0cf650 0x7f5ecc15fcd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400efc0 tx=0x7f5eb400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:05.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ec97fa640 1 -- 172.21.15.115:0/873114914 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4009b80 con 0x7f5ecc0cf650 2023-12-01T01:37:05.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ec97fa640 1 -- 172.21.15.115:0/873114914 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb400f040 con 0x7f5ecc0cf650 2023-12-01T01:37:05.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ec97fa640 1 -- 172.21.15.115:0/873114914 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40146a0 con 0x7f5ecc0cf650 2023-12-01T01:37:05.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/873114914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc15f1f0 con 0x7f5ecc0cf650 2023-12-01T01:37:05.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.894+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/873114914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc15f7b0 con 0x7f5ecc0cf650 2023-12-01T01:37:05.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.895+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/873114914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc15f970 con 0x7f5ecc0cf650 2023-12-01T01:37:05.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.896+0000 7f5ec97fa640 1 -- 172.21.15.115:0/873114914 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5eb4014800 con 0x7f5ecc0cf650 2023-12-01T01:37:05.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.897+0000 7f5ec97fa640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5eb006a430 0x7f5eb006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:05.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.897+0000 7f5ec97fa640 1 -- 172.21.15.115:0/873114914 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5eb4089800 con 0x7f5ecc0cf650 2023-12-01T01:37:05.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.898+0000 7f5ed0c59640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5eb006a430 0x7f5eb006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:05.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.898+0000 7f5ed0c59640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5eb006a430 0x7f5eb006c8f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc036000 tx=0x7f5ebc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:05.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:05.902+0000 7f5ec97fa640 1 -- 172.21.15.115:0/873114914 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5eb4054d30 con 0x7f5ecc0cf650 2023-12-01T01:37:06.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.084+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/873114914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ecc0d1070 con 0x7f5ecc0cf650 2023-12-01T01:37:06.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.085+0000 7f5ec97fa640 1 -- 172.21.15.115:0/873114914 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5eb4058350 con 0x7f5ecc0cf650 2023-12-01T01:37:06.086 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:37:06.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.087+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/873114914 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5eb006a430 msgr2=0x7f5eb006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.087+0000 7f5ed2ebd640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5eb006a430 0x7f5eb006c8f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc036000 tx=0x7f5ebc008030 comp rx=0 tx=0).stop 2023-12-01T01:37:06.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.087+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/873114914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ecc0cf650 msgr2=0x7f5ecc15fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.087+0000 7f5ed2ebd640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ecc0cf650 0x7f5ecc15fcd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400efc0 tx=0x7f5eb400c490 comp rx=0 tx=0).stop 2023-12-01T01:37:06.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.088+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/873114914 shutdown_connections 2023-12-01T01:37:06.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.088+0000 7f5ed2ebd640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5eb006a430 0x7f5eb006c8f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.088+0000 7f5ed2ebd640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5ecc15cd90 0x7f5ecc0a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.088+0000 7f5ed2ebd640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ecc0cf650 0x7f5ecc15fcd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.088+0000 7f5ed2ebd640 1 --2- 172.21.15.115:0/873114914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ecc0b5fa0 0x7f5ecc157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.088+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/873114914 >> 172.21.15.115:0/873114914 conn(0x7f5ecc065960 msgr2=0x7f5ecc0b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:06.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.088+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/873114914 shutdown_connections 2023-12-01T01:37:06.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.088+0000 7f5ed2ebd640 1 -- 172.21.15.115:0/873114914 wait complete. 2023-12-01T01:37:06.106 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:37:06.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.303+0000 7fcd6af16640 1 Processor -- start 2023-12-01T01:37:06.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.303+0000 7fcd6af16640 1 -- start start 2023-12-01T01:37:06.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.304+0000 7fcd6af16640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd640cd470 0x7fcd640b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.304+0000 7fcd6af16640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd640b2040 0x7fcd6415e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.304+0000 7fcd6af16640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd640cc2b0 con 0x7fcd640cd850 2023-12-01T01:37:06.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.304+0000 7fcd6af16640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd640ce0b0 con 0x7fcd640cd470 2023-12-01T01:37:06.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.304+0000 7fcd6af16640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd640cddd0 con 0x7fcd640b2040 2023-12-01T01:37:06.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.304+0000 7fcd68cb2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd640cd470 0x7fcd640b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:06.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.304+0000 7fcd68cb2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd640cd470 0x7fcd640b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48332/0 (socket says 172.21.15.115:48332) 2023-12-01T01:37:06.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.304+0000 7fcd68cb2640 1 -- 172.21.15.115:0/1437349169 learned_addr learned my addr 172.21.15.115:0/1437349169 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:06.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.305+0000 7fcd68cb2640 1 -- 172.21.15.115:0/1437349169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd640b2040 msgr2=0x7fcd6415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:06.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.305+0000 7fcd63fff640 1 --2- 172.21.15.115:0/1437349169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd640b2040 0x7fcd6415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:06.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.305+0000 7fcd68cb2640 1 --2- 172.21.15.115:0/1437349169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd640b2040 0x7fcd6415e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.305+0000 7fcd68cb2640 1 -- 172.21.15.115:0/1437349169 >> v1:172.21.15.142:6789/0 conn(0x7fcd640cd850 legacy=0x7fcd640c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:06.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.305+0000 7fcd68cb2640 1 -- 172.21.15.115:0/1437349169 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd640d1780 con 0x7fcd640cd470 2023-12-01T01:37:06.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.305+0000 7fcd63fff640 1 --2- 172.21.15.115:0/1437349169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd640b2040 0x7fcd6415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:06.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.306+0000 7fcd68cb2640 1 --2- 172.21.15.115:0/1437349169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640cd470 0x7fcd640b1a70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009870 tx=0x7fcd5802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab1b5ecc77d729 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:06.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.306+0000 7fcd637fe640 1 -- 172.21.15.115:0/1437349169 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58031640 con 0x7fcd640cd470 2023-12-01T01:37:06.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.306+0000 7fcd637fe640 1 -- 172.21.15.115:0/1437349169 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd580317a0 con 0x7fcd640cd470 2023-12-01T01:37:06.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.306+0000 7fcd637fe640 1 -- 172.21.15.115:0/1437349169 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58031b30 con 0x7fcd640cd470 2023-12-01T01:37:06.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.307+0000 7fcd6af16640 1 -- 172.21.15.115:0/1437349169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640cd470 msgr2=0x7fcd640b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.307+0000 7fcd6af16640 1 --2- 172.21.15.115:0/1437349169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640cd470 0x7fcd640b1a70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009870 tx=0x7fcd5802b8f0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.307+0000 7fcd6af16640 1 -- 172.21.15.115:0/1437349169 shutdown_connections 2023-12-01T01:37:06.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.307+0000 7fcd6af16640 1 --2- 172.21.15.115:0/1437349169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd640b2040 0x7fcd6415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.307+0000 7fcd6af16640 1 --2- 172.21.15.115:0/1437349169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640cd470 0x7fcd640b1a70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.307+0000 7fcd6af16640 1 -- 172.21.15.115:0/1437349169 >> 172.21.15.115:0/1437349169 conn(0x7fcd640d0ba0 msgr2=0x7fcd640d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:06.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.307+0000 7fcd6af16640 1 -- 172.21.15.115:0/1437349169 shutdown_connections 2023-12-01T01:37:06.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.307+0000 7fcd6af16640 1 -- 172.21.15.115:0/1437349169 wait complete. 2023-12-01T01:37:06.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.309+0000 7fcd6af16640 1 Processor -- start 2023-12-01T01:37:06.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.309+0000 7fcd6af16640 1 -- start start 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.310+0000 7fcd6af16640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640b2040 0x7fcd641e5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.310+0000 7fcd6af16640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd640cd470 0x7fcd641e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.310+0000 7fcd6af16640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd640cd850 0x7fcd64154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd6af16640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd641e5020 con 0x7fcd640b2040 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd6af16640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd641e4100 con 0x7fcd640cd470 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd6af16640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd641e43a0 con 0x7fcd640cd850 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd68cb2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640b2040 0x7fcd641e5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd68cb2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640b2040 0x7fcd641e5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48344/0 (socket says 172.21.15.115:48344) 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd68cb2640 1 -- 172.21.15.115:0/787064427 learned_addr learned my addr 172.21.15.115:0/787064427 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd68cb2640 1 -- 172.21.15.115:0/787064427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd640cd850 msgr2=0x7fcd64154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd68cb2640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd640cd850 0x7fcd64154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd68cb2640 1 -- 172.21.15.115:0/787064427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd640cd470 msgr2=0x7fcd641e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd68cb2640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd640cd470 0x7fcd641e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd68cb2640 1 -- 172.21.15.115:0/787064427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580094d0 con 0x7fcd640b2040 2023-12-01T01:37:06.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.311+0000 7fcd68cb2640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640b2040 0x7fcd641e5780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009870 tx=0x7fcd580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:06.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.312+0000 7fcd617fa640 1 -- 172.21.15.115:0/787064427 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5803ac60 con 0x7fcd640b2040 2023-12-01T01:37:06.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.312+0000 7fcd617fa640 1 -- 172.21.15.115:0/787064427 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5803ade0 con 0x7fcd640b2040 2023-12-01T01:37:06.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.312+0000 7fcd617fa640 1 -- 172.21.15.115:0/787064427 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58044630 con 0x7fcd640b2040 2023-12-01T01:37:06.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.312+0000 7fcd6af16640 1 -- 172.21.15.115:0/787064427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd641e4680 con 0x7fcd640b2040 2023-12-01T01:37:06.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.312+0000 7fcd6af16640 1 -- 172.21.15.115:0/787064427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd641e4c40 con 0x7fcd640b2040 2023-12-01T01:37:06.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.313+0000 7fcd6af16640 1 -- 172.21.15.115:0/787064427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd64067850 con 0x7fcd640b2040 2023-12-01T01:37:06.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.314+0000 7fcd617fa640 1 -- 172.21.15.115:0/787064427 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcd5802f050 con 0x7fcd640b2040 2023-12-01T01:37:06.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.315+0000 7fcd617fa640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4806a430 0x7fcd4806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.315+0000 7fcd617fa640 1 -- 172.21.15.115:0/787064427 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcd580aab10 con 0x7fcd640b2040 2023-12-01T01:37:06.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.315+0000 7fcd63fff640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4806a430 0x7fcd4806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:06.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.316+0000 7fcd63fff640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4806a430 0x7fcd4806c8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcd50006fd0 tx=0x7fcd50008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:06.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.320+0000 7fcd617fa640 1 -- 172.21.15.115:0/787064427 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcd580760d0 con 0x7fcd640b2040 2023-12-01T01:37:06.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.501+0000 7fcd6af16640 1 -- 172.21.15.115:0/787064427 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd64067850 con 0x7fcd640b2040 2023-12-01T01:37:06.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.502+0000 7fcd617fa640 1 -- 172.21.15.115:0/787064427 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd580796f0 con 0x7fcd640b2040 2023-12-01T01:37:06.504 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 -- 172.21.15.115:0/787064427 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4806a430 msgr2=0x7fcd4806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4806a430 0x7fcd4806c8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcd50006fd0 tx=0x7fcd50008040 comp rx=0 tx=0).stop 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 -- 172.21.15.115:0/787064427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640b2040 msgr2=0x7fcd641e5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640b2040 0x7fcd641e5780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009870 tx=0x7fcd580046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 -- 172.21.15.115:0/787064427 shutdown_connections 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4806a430 0x7fcd4806c8f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd640cd850 0x7fcd64154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd640cd470 0x7fcd641e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 --2- 172.21.15.115:0/787064427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640b2040 0x7fcd641e5780 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 -- 172.21.15.115:0/787064427 >> 172.21.15.115:0/787064427 conn(0x7fcd640d0ba0 msgr2=0x7fcd6406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 -- 172.21.15.115:0/787064427 shutdown_connections 2023-12-01T01:37:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.505+0000 7fcd6af16640 1 -- 172.21.15.115:0/787064427 wait complete. 2023-12-01T01:37:06.523 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:37:06.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.717+0000 7fe2c3c21640 1 Processor -- start 2023-12-01T01:37:06.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.717+0000 7fe2c3c21640 1 -- start start 2023-12-01T01:37:06.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.718+0000 7fe2c3c21640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe2bc159dd0 0x7fe2bc0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.718+0000 7fe2c3c21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2bc070d30 0x7fe2bc0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.718+0000 7fe2c3c21640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc06f8b0 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.718+0000 7fe2c3c21640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2bc0adcb0 con 0x7fe2bc159dd0 2023-12-01T01:37:06.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.718+0000 7fe2c3c21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0ad9d0 con 0x7fe2bc070d30 2023-12-01T01:37:06.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.719+0000 7fe2c21be640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2bc070d30 0x7fe2bc0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:06.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.719+0000 7fe2c19bd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe2bc159dd0 0x7fe2bc0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:06.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.719+0000 7fe2c11bc640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fe2bc06b6f0 0x7fe2bc0cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:55234/0 (socket says 172.21.15.115:55234) 2023-12-01T01:37:06.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.719+0000 7fe2c11bc640 1 -- 172.21.15.115:0/1015183084 learned_addr learned my addr 172.21.15.115:0/1015183084 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.720+0000 7fe2c21be640 1 -- 172.21.15.115:0/1015183084 >> v2:172.21.15.142:3300/0 conn(0x7fe2bc159dd0 msgr2=0x7fe2bc0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.720+0000 7fe2c21be640 1 --2- 172.21.15.115:0/1015183084 >> v2:172.21.15.142:3300/0 conn(0x7fe2bc159dd0 0x7fe2bc0c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.720+0000 7fe2c21be640 1 -- 172.21.15.115:0/1015183084 >> v1:172.21.15.142:6789/0 conn(0x7fe2bc06b6f0 legacy=0x7fe2bc0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.720+0000 7fe2c21be640 1 -- 172.21.15.115:0/1015183084 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0c9000 con 0x7fe2bc070d30 2023-12-01T01:37:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.720+0000 7fe2c19bd640 1 --2- 172.21.15.115:0/1015183084 >> v2:172.21.15.142:3300/0 conn(0x7fe2bc159dd0 0x7fe2bc0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:37:06.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.722+0000 7fe2c21be640 1 --2- 172.21.15.115:0/1015183084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2bc070d30 0x7fe2bc0c9d20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009d90 tx=0x7fe2b402bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d589846518835fb8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.722+0000 7fe2c09bb640 1 -- 172.21.15.115:0/1015183084 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b402cb30 con 0x7fe2bc070d30 2023-12-01T01:37:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.722+0000 7fe2c09bb640 1 -- 172.21.15.115:0/1015183084 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b402cc90 con 0x7fe2bc070d30 2023-12-01T01:37:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.722+0000 7fe2c09bb640 1 -- 172.21.15.115:0/1015183084 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4034440 con 0x7fe2bc070d30 2023-12-01T01:37:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.723+0000 7fe2c3c21640 1 -- 172.21.15.115:0/1015183084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2bc070d30 msgr2=0x7fe2bc0c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.723+0000 7fe2c3c21640 1 --2- 172.21.15.115:0/1015183084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2bc070d30 0x7fe2bc0c9d20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009d90 tx=0x7fe2b402bd80 comp rx=0 tx=0).stop 2023-12-01T01:37:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.723+0000 7fe2c3c21640 1 -- 172.21.15.115:0/1015183084 shutdown_connections 2023-12-01T01:37:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.723+0000 7fe2c3c21640 1 --2- 172.21.15.115:0/1015183084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2bc070d30 0x7fe2bc0c9d20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.723+0000 7fe2c3c21640 1 --2- 172.21.15.115:0/1015183084 >> v2:172.21.15.142:3300/0 conn(0x7fe2bc159dd0 0x7fe2bc0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.723+0000 7fe2c3c21640 1 -- 172.21.15.115:0/1015183084 >> 172.21.15.115:0/1015183084 conn(0x7fe2bc0ce880 msgr2=0x7fe2bc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:06.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.723+0000 7fe2c3c21640 1 -- 172.21.15.115:0/1015183084 shutdown_connections 2023-12-01T01:37:06.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.723+0000 7fe2c3c21640 1 -- 172.21.15.115:0/1015183084 wait complete. 2023-12-01T01:37:06.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.724+0000 7fe2c3c21640 1 Processor -- start 2023-12-01T01:37:06.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.725+0000 7fe2c3c21640 1 -- start start 2023-12-01T01:37:06.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.725+0000 7fe2c3c21640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2bc06b6f0 0x7fe2bc154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.725+0000 7fe2c3c21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2bc070d30 0x7fe2bc156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c3c21640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2bc159dd0 0x7fe2bc15c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c3c21640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc1e5680 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c3c21640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc155450 con 0x7fe2bc159dd0 2023-12-01T01:37:06.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c3c21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc157680 con 0x7fe2bc070d30 2023-12-01T01:37:06.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c19bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2bc06b6f0 0x7fe2bc154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:06.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c19bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2bc06b6f0 0x7fe2bc154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48348/0 (socket says 172.21.15.115:48348) 2023-12-01T01:37:06.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c19bd640 1 -- 172.21.15.115:0/3686615800 learned_addr learned my addr 172.21.15.115:0/3686615800 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:06.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c19bd640 1 -- 172.21.15.115:0/3686615800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2bc070d30 msgr2=0x7fe2bc156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c19bd640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2bc070d30 0x7fe2bc156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c19bd640 1 -- 172.21.15.115:0/3686615800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2bc159dd0 msgr2=0x7fe2bc15c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c19bd640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2bc159dd0 0x7fe2bc15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c19bd640 1 -- 172.21.15.115:0/3686615800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b8009400 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.726+0000 7fe2c19bd640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2bc06b6f0 0x7fe2bc154f10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800e960 tx=0x7fe2b800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:06.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.727+0000 7fe2b27fc640 1 -- 172.21.15.115:0/3686615800 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b800ca00 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.727+0000 7fe2b27fc640 1 -- 172.21.15.115:0/3686615800 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b800cb60 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.727+0000 7fe2b27fc640 1 -- 172.21.15.115:0/3686615800 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8013470 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.727+0000 7fe2c3c21640 1 -- 172.21.15.115:0/3686615800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b4009a70 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.727+0000 7fe2c3c21640 1 -- 172.21.15.115:0/3686615800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc156de0 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.728+0000 7fe29ffff640 1 -- 172.21.15.115:0/3686615800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc04e890 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.729+0000 7fe2b27fc640 1 -- 172.21.15.115:0/3686615800 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe2b80135d0 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.731+0000 7fe2b27fc640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe2a006a430 0x7fe2a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:06.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.731+0000 7fe2b27fc640 1 -- 172.21.15.115:0/3686615800 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe2b8088de0 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.731+0000 7fe2c11bc640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe2a006a430 0x7fe2a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:06.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.731+0000 7fe2c11bc640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe2a006a430 0x7fe2a006c8f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc1f5330 tx=0x7fe2ac007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:06.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.735+0000 7fe2b27fc640 1 -- 172.21.15.115:0/3686615800 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe2b8054200 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.926+0000 7fe29ffff640 1 -- 172.21.15.115:0/3686615800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe2bc06a2f0 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.927+0000 7fe2b27fc640 1 -- 172.21.15.115:0/3686615800 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe2b8057820 con 0x7fe2bc06b6f0 2023-12-01T01:37:06.929 INFO:teuthology.orchestra.run.smithi115.stderr:listed 0 entries 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 -- 172.21.15.115:0/3686615800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe2a006a430 msgr2=0x7fe2a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe2a006a430 0x7fe2a006c8f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc1f5330 tx=0x7fe2ac007a40 comp rx=0 tx=0).stop 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 -- 172.21.15.115:0/3686615800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2bc06b6f0 msgr2=0x7fe2bc154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2bc06b6f0 0x7fe2bc154f10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800e960 tx=0x7fe2b800ee20 comp rx=0 tx=0).stop 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 -- 172.21.15.115:0/3686615800 shutdown_connections 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe2a006a430 0x7fe2a006c8f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe2bc070d30 0x7fe2bc156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2bc159dd0 0x7fe2bc15c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 --2- 172.21.15.115:0/3686615800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2bc06b6f0 0x7fe2bc154f10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 -- 172.21.15.115:0/3686615800 >> 172.21.15.115:0/3686615800 conn(0x7fe2bc0ce880 msgr2=0x7fe2bc159100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 -- 172.21.15.115:0/3686615800 shutdown_connections 2023-12-01T01:37:06.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:06.929+0000 7fe29ffff640 1 -- 172.21.15.115:0/3686615800 wait complete. 2023-12-01T01:37:06.949 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-12-01T01:37:06.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:06.949 DEBUG:teuthology.orchestra.run.smithi142:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-12-01T01:37:07.059 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T01:37:07.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:07.060 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-01T01:37:07.110 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T01:37:07.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:37:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.324+0000 7fdbf14b6640 1 Processor -- start 2023-12-01T01:37:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.324+0000 7fdbf14b6640 1 -- start start 2023-12-01T01:37:07.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.326+0000 7fdbf14b6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdbec15db90 0x7fdbec0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.326+0000 7fdbf14b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbec0cddc0 0x7fdbec0d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.326+0000 7fdbf14b6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec15e020 con 0x7fdbec15d7b0 2023-12-01T01:37:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.326+0000 7fdbf14b6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbec0cd3f0 con 0x7fdbec15db90 2023-12-01T01:37:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.326+0000 7fdbf14b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0cd0f0 con 0x7fdbec0cddc0 2023-12-01T01:37:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.327+0000 7fdbe3fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdbec15db90 0x7fdbec0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.327+0000 7fdbebfff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fdbec15d7b0 0x7fdbec160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:55244/0 (socket says 172.21.15.115:55244) 2023-12-01T01:37:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.327+0000 7fdbf0cb5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbec0cddc0 0x7fdbec0d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.327+0000 7fdbe3fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdbec15db90 0x7fdbec0cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:45220/0 (socket says 172.21.15.115:45220) 2023-12-01T01:37:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.327+0000 7fdbebfff640 1 -- 172.21.15.115:0/818127204 learned_addr learned my addr 172.21.15.115:0/818127204 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.327+0000 7fdbe3fff640 1 -- 172.21.15.115:0/818127204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbec0cddc0 msgr2=0x7fdbec0d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.327+0000 7fdbe3fff640 1 --2- 172.21.15.115:0/818127204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbec0cddc0 0x7fdbec0d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.327+0000 7fdbe3fff640 1 -- 172.21.15.115:0/818127204 >> v1:172.21.15.142:6789/0 conn(0x7fdbec15d7b0 legacy=0x7fdbec160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.327+0000 7fdbe3fff640 1 -- 172.21.15.115:0/818127204 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec160ec0 con 0x7fdbec15db90 2023-12-01T01:37:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.329+0000 7fdbe3fff640 1 --2- 172.21.15.115:0/818127204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbec15db90 0x7fdbec0cd880 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40097a0 tx=0x7fdbd402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa9e8a9a72fafed server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.329+0000 7fdbeb7fe640 1 -- 172.21.15.115:0/818127204 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4030610 con 0x7fdbec15db90 2023-12-01T01:37:07.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.329+0000 7fdbeb7fe640 1 -- 172.21.15.115:0/818127204 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd4030770 con 0x7fdbec15db90 2023-12-01T01:37:07.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.329+0000 7fdbeb7fe640 1 -- 172.21.15.115:0/818127204 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4030b00 con 0x7fdbec15db90 2023-12-01T01:37:07.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.330+0000 7fdbf14b6640 1 -- 172.21.15.115:0/818127204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbec15db90 msgr2=0x7fdbec0cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.330+0000 7fdbf14b6640 1 --2- 172.21.15.115:0/818127204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbec15db90 0x7fdbec0cd880 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40097a0 tx=0x7fdbd402b960 comp rx=0 tx=0).stop 2023-12-01T01:37:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.330+0000 7fdbf14b6640 1 -- 172.21.15.115:0/818127204 shutdown_connections 2023-12-01T01:37:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.330+0000 7fdbf14b6640 1 --2- 172.21.15.115:0/818127204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbec0cddc0 0x7fdbec0d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.330+0000 7fdbf14b6640 1 --2- 172.21.15.115:0/818127204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbec15db90 0x7fdbec0cd880 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.330+0000 7fdbf14b6640 1 -- 172.21.15.115:0/818127204 >> 172.21.15.115:0/818127204 conn(0x7fdbec06a0e0 msgr2=0x7fdbec06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.330+0000 7fdbf14b6640 1 -- 172.21.15.115:0/818127204 shutdown_connections 2023-12-01T01:37:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.330+0000 7fdbf14b6640 1 -- 172.21.15.115:0/818127204 wait complete. 2023-12-01T01:37:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.332+0000 7fdbf14b6640 1 Processor -- start 2023-12-01T01:37:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.332+0000 7fdbf14b6640 1 -- start start 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.333+0000 7fdbf14b6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbec0cddc0 0x7fdbec0c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.333+0000 7fdbf14b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbec15d7b0 0x7fdbec0b8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbf14b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbec15db90 0x7fdbec0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbf14b6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0c8b50 con 0x7fdbec15d7b0 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbf14b6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0cb600 con 0x7fdbec0cddc0 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbf14b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0cb8e0 con 0x7fdbec15db90 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbe3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbec15d7b0 0x7fdbec0b8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbe3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbec15d7b0 0x7fdbec0b8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48364/0 (socket says 172.21.15.115:48364) 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbe3fff640 1 -- 172.21.15.115:0/2427126348 learned_addr learned my addr 172.21.15.115:0/2427126348 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbebfff640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbec0cddc0 0x7fdbec0c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbf0cb5640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbec15db90 0x7fdbec0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbe3fff640 1 -- 172.21.15.115:0/2427126348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbec15db90 msgr2=0x7fdbec0cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbe3fff640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbec15db90 0x7fdbec0cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbe3fff640 1 -- 172.21.15.115:0/2427126348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbec0cddc0 msgr2=0x7fdbec0c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbe3fff640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbec0cddc0 0x7fdbec0c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbe3fff640 1 -- 172.21.15.115:0/2427126348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd4009400 con 0x7fdbec15d7b0 2023-12-01T01:37:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.334+0000 7fdbebfff640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbec0cddc0 0x7fdbec0c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:37:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.335+0000 7fdbe3fff640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbec15d7b0 0x7fdbec0b8330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40098c0 tx=0x7fdbd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:07.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.335+0000 7fdbe97fa640 1 -- 172.21.15.115:0/2427126348 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd403e070 con 0x7fdbec15d7b0 2023-12-01T01:37:07.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.335+0000 7fdbe97fa640 1 -- 172.21.15.115:0/2427126348 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd4030e10 con 0x7fdbec15d7b0 2023-12-01T01:37:07.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.335+0000 7fdbf14b6640 1 -- 172.21.15.115:0/2427126348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec0cbbc0 con 0x7fdbec15d7b0 2023-12-01T01:37:07.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.335+0000 7fdbe97fa640 1 -- 172.21.15.115:0/2427126348 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4039c50 con 0x7fdbec15d7b0 2023-12-01T01:37:07.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.335+0000 7fdbf14b6640 1 -- 172.21.15.115:0/2427126348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec0cc180 con 0x7fdbec15d7b0 2023-12-01T01:37:07.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.336+0000 7fdbf14b6640 1 -- 172.21.15.115:0/2427126348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbec0673a0 con 0x7fdbec15d7b0 2023-12-01T01:37:07.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.337+0000 7fdbe97fa640 1 -- 172.21.15.115:0/2427126348 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdbd40423f0 con 0x7fdbec15d7b0 2023-12-01T01:37:07.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.338+0000 7fdbe97fa640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbcc06a430 0x7fdbcc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.338+0000 7fdbe97fa640 1 -- 172.21.15.115:0/2427126348 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fdbd40aa530 con 0x7fdbec15d7b0 2023-12-01T01:37:07.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.338+0000 7fdbebfff640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbcc06a430 0x7fdbcc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.339+0000 7fdbebfff640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbcc06a430 0x7fdbcc06c8f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc005880 tx=0x7fdbdc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:07.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.343+0000 7fdbe97fa640 1 -- 172.21.15.115:0/2427126348 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdbd4075b80 con 0x7fdbec15d7b0 2023-12-01T01:37:07.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.569+0000 7fdbf14b6640 1 -- 172.21.15.115:0/2427126348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbec0d3100 con 0x7fdbec15d7b0 2023-12-01T01:37:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.571+0000 7fdbe97fa640 1 -- 172.21.15.115:0/2427126348 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4654 (secure 0 0 0) 0x7fdbd40791a0 con 0x7fdbec15d7b0 2023-12-01T01:37:07.572 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:07.572 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3656795862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3656795862},{"type":"v1","addr":"172.21.15.142:6834","nonce":3656795862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1125002206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1125002206},{"type":"v1","addr":"172.21.15.115:6836","nonce":1125002206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3350782192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3350782192},{"type":"v1","addr":"172.21.15.115:6837","nonce":3350782192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:36:10.637450+0000","modified":"2023-12-01T01:36:13.663004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2808777047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2808777047},{"type":"v1","addr":"172.21.15.142:6835","nonce":2808777047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:37:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 8 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 -- 172.21.15.115:0/2427126348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbcc06a430 msgr2=0x7fdbcc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbcc06a430 0x7fdbcc06c8f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc005880 tx=0x7fdbdc009040 comp rx=0 tx=0).stop 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 -- 172.21.15.115:0/2427126348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbec15d7b0 msgr2=0x7fdbec0b8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbec15d7b0 0x7fdbec0b8330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40098c0 tx=0x7fdbd40046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 -- 172.21.15.115:0/2427126348 shutdown_connections 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbcc06a430 0x7fdbcc06c8f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbec15db90 0x7fdbec0cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbec15d7b0 0x7fdbec0b8330 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 --2- 172.21.15.115:0/2427126348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbec0cddc0 0x7fdbec0c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 -- 172.21.15.115:0/2427126348 >> 172.21.15.115:0/2427126348 conn(0x7fdbec06a0e0 msgr2=0x7fdbec0b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 -- 172.21.15.115:0/2427126348 shutdown_connections 2023-12-01T01:37:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.573+0000 7fdbf14b6640 1 -- 172.21.15.115:0/2427126348 wait complete. 2023-12-01T01:37:07.592 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:37:07.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.801+0000 7f5f3c0c4640 1 Processor -- start 2023-12-01T01:37:07.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.802+0000 7f5f3c0c4640 1 -- start start 2023-12-01T01:37:07.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f3c0c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f3415cd20 0x7f5f340d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f3c0c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5f340d1230 0x7f5f340d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f3c0c4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f340ce2e0 con 0x7f5f340b5d80 2023-12-01T01:37:07.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f3c0c4640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f340b2160 con 0x7f5f340d1230 2023-12-01T01:37:07.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f3c0c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340d1be0 con 0x7f5f3415cd20 2023-12-01T01:37:07.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f3a661640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5f340d1230 0x7f5f340d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f3a661640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5f340d1230 0x7f5f340d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36776/0 (socket says 172.21.15.115:36776) 2023-12-01T01:37:07.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f3a661640 1 -- 172.21.15.115:0/2935153216 learned_addr learned my addr 172.21.15.115:0/2935153216 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:07.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f39e60640 1 --2- 172.21.15.115:0/2935153216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f3415cd20 0x7f5f340d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f3a661640 1 -- 172.21.15.115:0/2935153216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f3415cd20 msgr2=0x7f5f340d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:07.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f3a661640 1 --2- 172.21.15.115:0/2935153216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f3415cd20 0x7f5f340d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.804+0000 7f5f3a661640 1 -- 172.21.15.115:0/2935153216 >> v1:172.21.15.115:6789/0 conn(0x7f5f340b5d80 legacy=0x7f5f340b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.803+0000 7f5f38e5e640 1 -- 172.21.15.115:0/2935153216 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980447302 0 0) 0x7f5f340ce2e0 con 0x7f5f340b5d80 2023-12-01T01:37:07.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.804+0000 7f5f3a661640 1 -- 172.21.15.115:0/2935153216 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f340ce000 con 0x7f5f340d1230 2023-12-01T01:37:07.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.804+0000 7f5f39e60640 1 --2- 172.21.15.115:0/2935153216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f3415cd20 0x7f5f340d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:07.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.805+0000 7f5f3a661640 1 --2- 172.21.15.115:0/2935153216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 0x7f5f340d16a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009870 tx=0x7f5f2c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0ec0179491f6dc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:07.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.805+0000 7f5f38e5e640 1 -- 172.21.15.115:0/2935153216 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c031640 con 0x7f5f340d1230 2023-12-01T01:37:07.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.805+0000 7f5f38e5e640 1 -- 172.21.15.115:0/2935153216 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c0317a0 con 0x7f5f340d1230 2023-12-01T01:37:07.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.805+0000 7f5f38e5e640 1 -- 172.21.15.115:0/2935153216 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c031b30 con 0x7f5f340d1230 2023-12-01T01:37:07.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.806+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/2935153216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 msgr2=0x7f5f340d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.806+0000 7f5f3c0c4640 1 --2- 172.21.15.115:0/2935153216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 0x7f5f340d16a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009870 tx=0x7f5f2c02b8f0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.806+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/2935153216 shutdown_connections 2023-12-01T01:37:07.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.806+0000 7f5f3c0c4640 1 --2- 172.21.15.115:0/2935153216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 0x7f5f340d16a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.806+0000 7f5f3c0c4640 1 --2- 172.21.15.115:0/2935153216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f3415cd20 0x7f5f340d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.806+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/2935153216 >> 172.21.15.115:0/2935153216 conn(0x7f5f3406a0e0 msgr2=0x7f5f3406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:07.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.806+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/2935153216 shutdown_connections 2023-12-01T01:37:07.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.806+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/2935153216 wait complete. 2023-12-01T01:37:07.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.808+0000 7f5f3c0c4640 1 Processor -- start 2023-12-01T01:37:07.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.808+0000 7f5f3c0c4640 1 -- start start 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3c0c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f340b5d80 0x7f5f340b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3c0c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 0x7f5f34154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3c0c4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f3415cd20 0x7f5f340b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3c0c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340c8610 con 0x7f5f340d1230 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3c0c4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340b2c50 con 0x7f5f3415cd20 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3c0c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f341f63c0 con 0x7f5f340b5d80 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3965f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 0x7f5f34154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3965f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 0x7f5f34154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36786/0 (socket says 172.21.15.115:36786) 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3965f640 1 -- 172.21.15.115:0/3103214090 learned_addr learned my addr 172.21.15.115:0/3103214090 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f39e60640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f340b5d80 0x7f5f340b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3a661640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f3415cd20 0x7f5f340b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.809+0000 7f5f3965f640 1 -- 172.21.15.115:0/3103214090 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f340b5d80 msgr2=0x7f5f340b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.810+0000 7f5f3965f640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f340b5d80 0x7f5f340b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.810+0000 7f5f3965f640 1 -- 172.21.15.115:0/3103214090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f3415cd20 msgr2=0x7f5f340b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.810+0000 7f5f3965f640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f3415cd20 0x7f5f340b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:07.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.810+0000 7f5f3965f640 1 -- 172.21.15.115:0/3103214090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0094d0 con 0x7f5f340d1230 2023-12-01T01:37:07.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.810+0000 7f5f3965f640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 0x7f5f34154d50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000c410 tx=0x7f5f2000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:07.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.810+0000 7f5f327fc640 1 -- 172.21.15.115:0/3103214090 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200035f0 con 0x7f5f340d1230 2023-12-01T01:37:07.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.810+0000 7f5f327fc640 1 -- 172.21.15.115:0/3103214090 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20003750 con 0x7f5f340d1230 2023-12-01T01:37:07.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.810+0000 7f5f327fc640 1 -- 172.21.15.115:0/3103214090 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20003b20 con 0x7f5f340d1230 2023-12-01T01:37:07.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.810+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/3103214090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f340c8270 con 0x7f5f340d1230 2023-12-01T01:37:07.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.810+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/3103214090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f340b2970 con 0x7f5f340d1230 2023-12-01T01:37:07.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.812+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/3103214090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f340c8750 con 0x7f5f340d1230 2023-12-01T01:37:07.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.812+0000 7f5f327fc640 1 -- 172.21.15.115:0/3103214090 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5f20003c80 con 0x7f5f340d1230 2023-12-01T01:37:07.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.813+0000 7f5f327fc640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f1806a430 0x7f5f1806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:07.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.813+0000 7f5f327fc640 1 -- 172.21.15.115:0/3103214090 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5f20087e80 con 0x7f5f340d1230 2023-12-01T01:37:07.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.813+0000 7f5f39e60640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f1806a430 0x7f5f1806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:07.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.814+0000 7f5f39e60640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f1806a430 0x7f5f1806c8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5f280075b0 tx=0x7f5f28007540 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:07.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.818+0000 7f5f327fc640 1 -- 172.21.15.115:0/3103214090 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5f340c8750 con 0x7f5f340d1230 2023-12-01T01:37:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.993+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/3103214090 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f340c8750 con 0x7f5f340d1230 2023-12-01T01:37:07.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.995+0000 7f5f327fc640 1 -- 172.21.15.115:0/3103214090 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16138 (secure 0 0 0) 0x7f5f340c8750 con 0x7f5f340d1230 2023-12-01T01:37:07.997 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:07.997 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":23,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:52.262397+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:37:07.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.997+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/3103214090 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f1806a430 msgr2=0x7f5f1806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:07.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.997+0000 7f5f3c0c4640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f1806a430 0x7f5f1806c8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5f280075b0 tx=0x7f5f28007540 comp rx=0 tx=0).stop 2023-12-01T01:37:07.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.997+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/3103214090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 msgr2=0x7f5f34154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.997+0000 7f5f3c0c4640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 0x7f5f34154d50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000c410 tx=0x7f5f2000c8d0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.998+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/3103214090 shutdown_connections 2023-12-01T01:37:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.998+0000 7f5f3c0c4640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f1806a430 0x7f5f1806c8f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.998+0000 7f5f3c0c4640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f3415cd20 0x7f5f340b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.998+0000 7f5f3c0c4640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f340d1230 0x7f5f34154d50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.998+0000 7f5f3c0c4640 1 --2- 172.21.15.115:0/3103214090 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f340b5d80 0x7f5f340b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.998+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/3103214090 >> 172.21.15.115:0/3103214090 conn(0x7f5f3406a0e0 msgr2=0x7f5f3406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.998+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/3103214090 shutdown_connections 2023-12-01T01:37:08.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:07.998+0000 7f5f3c0c4640 1 -- 172.21.15.115:0/3103214090 wait complete. 2023-12-01T01:37:08.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:36:08.429168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:36:09.629775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:37:08.018 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:37:08.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.229+0000 7fd6b12d8640 1 Processor -- start 2023-12-01T01:37:08.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.229+0000 7fd6b12d8640 1 -- start start 2023-12-01T01:37:08.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.231+0000 7fd6b12d8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6ac160200 0x7fd6ac0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.231+0000 7fd6b12d8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd6ac0b0f80 0x7fd6ac06b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.231+0000 7fd6b12d8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ac0cb290 con 0x7fd6ac06b7a0 2023-12-01T01:37:08.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.231+0000 7fd6b12d8640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ac0b35f0 con 0x7fd6ac0b0f80 2023-12-01T01:37:08.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.231+0000 7fd6b12d8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0b0ae0 con 0x7fd6ac160200 2023-12-01T01:37:08.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.231+0000 7fd6a3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd6ac0b0f80 0x7fd6ac06b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.231+0000 7fd6a3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd6ac0b0f80 0x7fd6ac06b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36794/0 (socket says 172.21.15.115:36794) 2023-12-01T01:37:08.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.231+0000 7fd6a3fff640 1 -- 172.21.15.115:0/1102288570 learned_addr learned my addr 172.21.15.115:0/1102288570 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:08.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.232+0000 7fd6a3fff640 1 -- 172.21.15.115:0/1102288570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6ac160200 msgr2=0x7fd6ac0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:08.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.232+0000 7fd6aaffd640 1 --2- 172.21.15.115:0/1102288570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6ac160200 0x7fd6ac0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.232+0000 7fd6a3fff640 1 --2- 172.21.15.115:0/1102288570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6ac160200 0x7fd6ac0b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.232+0000 7fd6a3fff640 1 -- 172.21.15.115:0/1102288570 >> v1:172.21.15.142:6789/0 conn(0x7fd6ac06b7a0 legacy=0x7fd6ac0cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:08.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.232+0000 7fd6a3fff640 1 -- 172.21.15.115:0/1102288570 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac0b3310 con 0x7fd6ac0b0f80 2023-12-01T01:37:08.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.233+0000 7fd6a3fff640 1 --2- 172.21.15.115:0/1102288570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac0b0f80 0x7fd6ac06b260 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd694009870 tx=0x7fd69402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fea8c993feb2b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:08.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.233+0000 7fd6aa7fc640 1 -- 172.21.15.115:0/1102288570 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694031640 con 0x7fd6ac0b0f80 2023-12-01T01:37:08.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.233+0000 7fd6aa7fc640 1 -- 172.21.15.115:0/1102288570 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6940317a0 con 0x7fd6ac0b0f80 2023-12-01T01:37:08.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.233+0000 7fd6aa7fc640 1 -- 172.21.15.115:0/1102288570 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694031b30 con 0x7fd6ac0b0f80 2023-12-01T01:37:08.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.234+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1102288570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac0b0f80 msgr2=0x7fd6ac06b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.234+0000 7fd6b12d8640 1 --2- 172.21.15.115:0/1102288570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac0b0f80 0x7fd6ac06b260 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd694009870 tx=0x7fd69402b8f0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.234+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1102288570 shutdown_connections 2023-12-01T01:37:08.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.234+0000 7fd6b12d8640 1 --2- 172.21.15.115:0/1102288570 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac0b0f80 0x7fd6ac06b260 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.234+0000 7fd6b12d8640 1 --2- 172.21.15.115:0/1102288570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6ac160200 0x7fd6ac0b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.234+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1102288570 >> 172.21.15.115:0/1102288570 conn(0x7fd6ac065960 msgr2=0x7fd6ac065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:08.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.234+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1102288570 shutdown_connections 2023-12-01T01:37:08.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.234+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1102288570 wait complete. 2023-12-01T01:37:08.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.236+0000 7fd6b12d8640 1 Processor -- start 2023-12-01T01:37:08.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.236+0000 7fd6b12d8640 1 -- start start 2023-12-01T01:37:08.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.237+0000 7fd6b12d8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6ac06b7a0 0x7fd6ac13cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.237+0000 7fd6b12d8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6ac0b0f80 0x7fd6ac1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.237+0000 7fd6b12d8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac160200 0x7fd6ac15ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.237+0000 7fd6b12d8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac14c5c0 con 0x7fd6ac160200 2023-12-01T01:37:08.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.237+0000 7fd6b12d8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac159700 con 0x7fd6ac06b7a0 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.237+0000 7fd6b12d8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac158120 con 0x7fd6ac0b0f80 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.237+0000 7fd6ab7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac160200 0x7fd6ac15ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.237+0000 7fd6ab7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac160200 0x7fd6ac15ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36796/0 (socket says 172.21.15.115:36796) 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.237+0000 7fd6ab7fe640 1 -- 172.21.15.115:0/1139667900 learned_addr learned my addr 172.21.15.115:0/1139667900 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6a3fff640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6ac0b0f80 0x7fd6ac1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6aaffd640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6ac06b7a0 0x7fd6ac13cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6ab7fe640 1 -- 172.21.15.115:0/1139667900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6ac0b0f80 msgr2=0x7fd6ac1591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6ab7fe640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6ac0b0f80 0x7fd6ac1591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6ab7fe640 1 -- 172.21.15.115:0/1139667900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6ac06b7a0 msgr2=0x7fd6ac13cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6ab7fe640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6ac06b7a0 0x7fd6ac13cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6ab7fe640 1 -- 172.21.15.115:0/1139667900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940094d0 con 0x7fd6ac160200 2023-12-01T01:37:08.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6ab7fe640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac160200 0x7fd6ac15ce70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400c700 tx=0x7fd6a400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:08.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6a37fe640 1 -- 172.21.15.115:0/1139667900 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a400aa50 con 0x7fd6ac160200 2023-12-01T01:37:08.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6a37fe640 1 -- 172.21.15.115:0/1139667900 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a400abb0 con 0x7fd6ac160200 2023-12-01T01:37:08.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1139667900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac158400 con 0x7fd6ac160200 2023-12-01T01:37:08.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.238+0000 7fd6a37fe640 1 -- 172.21.15.115:0/1139667900 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a400ce10 con 0x7fd6ac160200 2023-12-01T01:37:08.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.239+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1139667900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac1589c0 con 0x7fd6ac160200 2023-12-01T01:37:08.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.240+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1139667900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac04e890 con 0x7fd6ac160200 2023-12-01T01:37:08.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.241+0000 7fd6a37fe640 1 -- 172.21.15.115:0/1139667900 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd6a4012420 con 0x7fd6ac160200 2023-12-01T01:37:08.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.241+0000 7fd6a37fe640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd68c06a460 0x7fd68c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.242+0000 7fd6a37fe640 1 -- 172.21.15.115:0/1139667900 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd6a4088d80 con 0x7fd6ac160200 2023-12-01T01:37:08.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.242+0000 7fd6aaffd640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd68c06a460 0x7fd68c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.243+0000 7fd6aaffd640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd68c06a460 0x7fd68c06c920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00b7c0 tx=0x7fd69c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:08.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.246+0000 7fd6a37fe640 1 -- 172.21.15.115:0/1139667900 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd6a4054350 con 0x7fd6ac160200 2023-12-01T01:37:08.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.435+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1139667900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6ac0b1e20 con 0x7fd6ac160200 2023-12-01T01:37:08.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.436+0000 7fd6a37fe640 1 -- 172.21.15.115:0/1139667900 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6a40160a0 con 0x7fd6ac160200 2023-12-01T01:37:08.437 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:37:08.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.438+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1139667900 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd68c06a460 msgr2=0x7fd68c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.438+0000 7fd6b12d8640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd68c06a460 0x7fd68c06c920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00b7c0 tx=0x7fd69c008040 comp rx=0 tx=0).stop 2023-12-01T01:37:08.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.438+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1139667900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac160200 msgr2=0x7fd6ac15ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.438+0000 7fd6b12d8640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac160200 0x7fd6ac15ce70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400c700 tx=0x7fd6a400cbc0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.438+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1139667900 shutdown_connections 2023-12-01T01:37:08.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.438+0000 7fd6b12d8640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd68c06a460 0x7fd68c06c920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.438+0000 7fd6b12d8640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6ac0b0f80 0x7fd6ac1591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.438+0000 7fd6b12d8640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6ac160200 0x7fd6ac15ce70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.439+0000 7fd6b12d8640 1 --2- 172.21.15.115:0/1139667900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6ac06b7a0 0x7fd6ac13cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.439+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1139667900 >> 172.21.15.115:0/1139667900 conn(0x7fd6ac065960 msgr2=0x7fd6ac070090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:08.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.439+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1139667900 shutdown_connections 2023-12-01T01:37:08.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.439+0000 7fd6b12d8640 1 -- 172.21.15.115:0/1139667900 wait complete. 2023-12-01T01:37:08.459 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:37:08.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.674+0000 7f91f57d9640 1 Processor -- start 2023-12-01T01:37:08.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.674+0000 7f91f57d9640 1 -- start start 2023-12-01T01:37:08.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.676+0000 7f91f57d9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f91f0142870 0x7f91f00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.677+0000 7f91f57d9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f00b2f30 con 0x7f91f00cd010 2023-12-01T01:37:08.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.677+0000 7f91f57d9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f00b2320 con 0x7f91f00b1e20 2023-12-01T01:37:08.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.677+0000 7f91f57d9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00b3ae0 con 0x7f91f0142870 2023-12-01T01:37:08.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.677+0000 7f91e7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f91f00cd010 0x7f91f00b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46624/0 (socket says 172.21.15.115:46624) 2023-12-01T01:37:08.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.677+0000 7f91e7fff640 1 -- 172.21.15.115:0/818185315 learned_addr learned my addr 172.21.15.115:0/818185315 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:08.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.677+0000 7f91eeffd640 1 --2- 172.21.15.115:0/818185315 >> v2:172.21.15.142:3300/0 conn(0x7f91f0142870 0x7f91f00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.677+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784770023 0 0) 0x7f91f00b2f30 con 0x7f91f00cd010 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.677+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91dc003520 con 0x7f91f00cd010 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1788971744 0 0) 0x7f91dc003520 con 0x7f91f00cd010 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 >> v2:172.21.15.142:3300/0 conn(0x7f91f0142870 msgr2=0x7f91f00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 --2- 172.21.15.115:0/818185315 >> v2:172.21.15.142:3300/0 conn(0x7f91f0142870 0x7f91f00ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 >> v1:172.21.15.142:6789/0 conn(0x7f91f00b1e20 legacy=0x7f91f015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00b2600 con 0x7f91f00cd010 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f91d8001670 con 0x7f91f00cd010 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 >> v1:172.21.15.115:6789/0 conn(0x7f91f00cd010 legacy=0x7f91f00b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91dc003970 0x7f91dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91dc0047c0 0x7f91dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91dc005180 0x7f91dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f00b3ae0 con 0x7f91dc005180 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc00c020 con 0x7f91dc0047c0 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc00c1e0 con 0x7f91dc003970 2023-12-01T01:37:08.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91ef7fe640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91dc005180 0x7f91dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.678+0000 7f91eeffd640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91dc003970 0x7f91dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.679+0000 7f91e7fff640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91dc0047c0 0x7f91dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.679+0000 7f91ef7fe640 1 -- 172.21.15.115:0/818185315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91dc003970 msgr2=0x7f91dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:08.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.679+0000 7f91ef7fe640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91dc003970 0x7f91dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.679+0000 7f91ef7fe640 1 -- 172.21.15.115:0/818185315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91dc0047c0 msgr2=0x7f91dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:08.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.679+0000 7f91ef7fe640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91dc0047c0 0x7f91dc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.679+0000 7f91ef7fe640 1 -- 172.21.15.115:0/818185315 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc003520 con 0x7f91dc005180 2023-12-01T01:37:08.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.680+0000 7f91ef7fe640 1 -- 172.21.15.115:0/818185315 reap_dead start 2023-12-01T01:37:08.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.680+0000 7f91ef7fe640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91dc005180 0x7f91dc00b980 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f91f006d580 tx=0x7f91dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad432cfaf5723078 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:08.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.680+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8003b70 con 0x7f91dc005180 2023-12-01T01:37:08.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.680+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8003cd0 con 0x7f91dc005180 2023-12-01T01:37:08.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.680+0000 7f91ee7fc640 1 -- 172.21.15.115:0/818185315 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8005310 con 0x7f91dc005180 2023-12-01T01:37:08.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.681+0000 7f91f57d9640 1 -- 172.21.15.115:0/818185315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91dc005180 msgr2=0x7f91dc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.681+0000 7f91f57d9640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91dc005180 0x7f91dc00b980 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f91f006d580 tx=0x7f91dc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.681+0000 7f91f57d9640 1 -- 172.21.15.115:0/818185315 shutdown_connections 2023-12-01T01:37:08.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.681+0000 7f91f57d9640 1 --2- 172.21.15.115:0/818185315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91dc005180 0x7f91dc00b980 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.681+0000 7f91f57d9640 1 -- 172.21.15.115:0/818185315 >> 172.21.15.115:0/818185315 conn(0x7f91f0065960 msgr2=0x7f91f0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:08.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.681+0000 7f91f57d9640 1 -- 172.21.15.115:0/818185315 shutdown_connections 2023-12-01T01:37:08.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.681+0000 7f91f57d9640 1 -- 172.21.15.115:0/818185315 wait complete. 2023-12-01T01:37:08.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.682+0000 7f91f57d9640 1 Processor -- start 2023-12-01T01:37:08.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.683+0000 7f91f57d9640 1 -- start start 2023-12-01T01:37:08.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.683+0000 7f91f57d9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91dc005180 0x7f91f014e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91f57d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91f013d5d0 0x7f91f014b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91f57d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91f00ad3d0 0x7f91f0141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91f57d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f00b0430 con 0x7f91f013d5d0 2023-12-01T01:37:08.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91f57d9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f0151590 con 0x7f91dc005180 2023-12-01T01:37:08.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91f57d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f00afe90 con 0x7f91f00ad3d0 2023-12-01T01:37:08.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91f013d5d0 0x7f91f014b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91f013d5d0 0x7f91f014b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36814/0 (socket says 172.21.15.115:36814) 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91e7fff640 1 -- 172.21.15.115:0/2146982277 learned_addr learned my addr 172.21.15.115:0/2146982277 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91eeffd640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91dc005180 0x7f91f014e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91e7fff640 1 -- 172.21.15.115:0/2146982277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91f00ad3d0 msgr2=0x7f91f0141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91ef7fe640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91f00ad3d0 0x7f91f0141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91e7fff640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91f00ad3d0 0x7f91f0141590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91e7fff640 1 -- 172.21.15.115:0/2146982277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91dc005180 msgr2=0x7f91f014e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91e7fff640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91dc005180 0x7f91f014e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91e7fff640 1 -- 172.21.15.115:0/2146982277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00b0150 con 0x7f91f013d5d0 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.684+0000 7f91eeffd640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91dc005180 0x7f91f014e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.685+0000 7f91e7fff640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91f013d5d0 0x7f91f014b530 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f91d800d770 tx=0x7f91d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.685+0000 7f91e77fe640 1 -- 172.21.15.115:0/2146982277 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d80074e0 con 0x7f91f013d5d0 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.685+0000 7f91e77fe640 1 -- 172.21.15.115:0/2146982277 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8007640 con 0x7f91f013d5d0 2023-12-01T01:37:08.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.685+0000 7f91e77fe640 1 -- 172.21.15.115:0/2146982277 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8004a20 con 0x7f91f013d5d0 2023-12-01T01:37:08.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.685+0000 7f91f57d9640 1 -- 172.21.15.115:0/2146982277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f0151220 con 0x7f91f013d5d0 2023-12-01T01:37:08.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.685+0000 7f91f57d9640 1 -- 172.21.15.115:0/2146982277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f0141ad0 con 0x7f91f013d5d0 2023-12-01T01:37:08.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.686+0000 7f91f57d9640 1 -- 172.21.15.115:0/2146982277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f0151080 con 0x7f91f013d5d0 2023-12-01T01:37:08.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.687+0000 7f91e77fe640 1 -- 172.21.15.115:0/2146982277 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f91d8004b80 con 0x7f91f013d5d0 2023-12-01T01:37:08.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.688+0000 7f91e77fe640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f91d006a430 0x7f91d006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:08.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.688+0000 7f91e77fe640 1 -- 172.21.15.115:0/2146982277 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f91d80882a0 con 0x7f91f013d5d0 2023-12-01T01:37:08.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.688+0000 7f91eeffd640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f91d006a430 0x7f91d006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:08.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.689+0000 7f91eeffd640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f91d006a430 0x7f91d006c8f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009980 tx=0x7f91e0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:08.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.693+0000 7f91e77fe640 1 -- 172.21.15.115:0/2146982277 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f91f0151080 con 0x7f91f013d5d0 2023-12-01T01:37:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.882+0000 7f91f57d9640 1 -- 172.21.15.115:0/2146982277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91f0151080 con 0x7f91f013d5d0 2023-12-01T01:37:08.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.883+0000 7f91e77fe640 1 -- 172.21.15.115:0/2146982277 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f91f0151080 con 0x7f91f013d5d0 2023-12-01T01:37:08.884 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:37:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.885+0000 7f91f57d9640 1 -- 172.21.15.115:0/2146982277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f91d006a430 msgr2=0x7f91d006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.885+0000 7f91f57d9640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f91d006a430 0x7f91d006c8f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009980 tx=0x7f91e0008040 comp rx=0 tx=0).stop 2023-12-01T01:37:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.885+0000 7f91f57d9640 1 -- 172.21.15.115:0/2146982277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91f013d5d0 msgr2=0x7f91f014b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.885+0000 7f91f57d9640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91f013d5d0 0x7f91f014b530 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f91d800d770 tx=0x7f91d800dc30 comp rx=0 tx=0).stop 2023-12-01T01:37:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.886+0000 7f91f57d9640 1 -- 172.21.15.115:0/2146982277 shutdown_connections 2023-12-01T01:37:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.886+0000 7f91f57d9640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f91d006a430 0x7f91d006c8f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.886+0000 7f91f57d9640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91f00ad3d0 0x7f91f0141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.886+0000 7f91f57d9640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91f013d5d0 0x7f91f014b530 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.886+0000 7f91f57d9640 1 --2- 172.21.15.115:0/2146982277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91dc005180 0x7f91f014e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:08.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.886+0000 7f91f57d9640 1 -- 172.21.15.115:0/2146982277 >> 172.21.15.115:0/2146982277 conn(0x7f91f0065960 msgr2=0x7f91f00ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:08.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.886+0000 7f91f57d9640 1 -- 172.21.15.115:0/2146982277 shutdown_connections 2023-12-01T01:37:08.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:08.886+0000 7f91f57d9640 1 -- 172.21.15.115:0/2146982277 wait complete. 2023-12-01T01:37:08.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:37:09.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.121+0000 7faa3d7c9640 1 Processor -- start 2023-12-01T01:37:09.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.121+0000 7faa3d7c9640 1 -- start start 2023-12-01T01:37:09.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.123+0000 7faa3d7c9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faa3815ac80 0x7faa380cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.123+0000 7faa3d7c9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa380cc9a0 con 0x7faa380b0e60 2023-12-01T01:37:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.123+0000 7faa3d7c9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa380cbd90 con 0x7faa380ce1b0 2023-12-01T01:37:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.123+0000 7faa3d7c9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3815b060 con 0x7faa3815ac80 2023-12-01T01:37:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.123+0000 7faa36ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faa3815ac80 0x7faa380cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.123+0000 7faa36ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faa3815ac80 0x7faa380cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36820/0 (socket says 172.21.15.115:36820) 2023-12-01T01:37:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.123+0000 7faa367fc640 1 --1- 172.21.15.115:0/3464326768 >> v1:172.21.15.115:6789/0 conn(0x7faa380b0e60 0x7faa380cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46636/0 (socket says 172.21.15.115:46636) 2023-12-01T01:37:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.123+0000 7faa36ffd640 1 -- 172.21.15.115:0/3464326768 learned_addr learned my addr 172.21.15.115:0/3464326768 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:09.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.124+0000 7faa36ffd640 1 -- 172.21.15.115:0/3464326768 >> v1:172.21.15.142:6789/0 conn(0x7faa380ce1b0 legacy=0x7faa3806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:09.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.124+0000 7faa36ffd640 1 -- 172.21.15.115:0/3464326768 >> v1:172.21.15.115:6789/0 conn(0x7faa380b0e60 legacy=0x7faa380cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.124+0000 7faa36ffd640 1 -- 172.21.15.115:0/3464326768 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380cc070 con 0x7faa3815ac80 2023-12-01T01:37:09.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.124+0000 7faa35ffb640 1 -- 172.21.15.115:0/3464326768 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737454328 0 0) 0x7faa380cc9a0 con 0x7faa380b0e60 2023-12-01T01:37:09.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.126+0000 7faa36ffd640 1 --2- 172.21.15.115:0/3464326768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 0x7faa380cce30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faa28009870 tx=0x7faa2802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9360a1768c90b278 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:09.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.126+0000 7faa35ffb640 1 -- 172.21.15.115:0/3464326768 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28030860 con 0x7faa3815ac80 2023-12-01T01:37:09.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.126+0000 7faa35ffb640 1 -- 172.21.15.115:0/3464326768 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa280309c0 con 0x7faa3815ac80 2023-12-01T01:37:09.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.126+0000 7faa35ffb640 1 -- 172.21.15.115:0/3464326768 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28030d50 con 0x7faa3815ac80 2023-12-01T01:37:09.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.127+0000 7faa3d7c9640 1 -- 172.21.15.115:0/3464326768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 msgr2=0x7faa380cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.127+0000 7faa3d7c9640 1 --2- 172.21.15.115:0/3464326768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 0x7faa380cce30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faa28009870 tx=0x7faa2802b9a0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.127+0000 7faa3d7c9640 1 -- 172.21.15.115:0/3464326768 shutdown_connections 2023-12-01T01:37:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.127+0000 7faa3d7c9640 1 --2- 172.21.15.115:0/3464326768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 0x7faa380cce30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.127+0000 7faa3d7c9640 1 -- 172.21.15.115:0/3464326768 >> 172.21.15.115:0/3464326768 conn(0x7faa38065960 msgr2=0x7faa38065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.127+0000 7faa3d7c9640 1 -- 172.21.15.115:0/3464326768 shutdown_connections 2023-12-01T01:37:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.127+0000 7faa3d7c9640 1 -- 172.21.15.115:0/3464326768 wait complete. 2023-12-01T01:37:09.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.128+0000 7faa3d7c9640 1 Processor -- start 2023-12-01T01:37:09.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.129+0000 7faa3d7c9640 1 -- start start 2023-12-01T01:37:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.129+0000 7faa3d7c9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa380b0e60 0x7faa38150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.129+0000 7faa3d7c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa380ce1b0 0x7faa381f0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.129+0000 7faa3d7c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 0x7faa38158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.129+0000 7faa3d7c9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa38157a90 con 0x7faa3815ac80 2023-12-01T01:37:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.129+0000 7faa3d7c9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa381f0d80 con 0x7faa380b0e60 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.129+0000 7faa3d7c9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa381f1060 con 0x7faa380ce1b0 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.130+0000 7faa377fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 0x7faa38158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.130+0000 7faa377fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 0x7faa38158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36824/0 (socket says 172.21.15.115:36824) 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.130+0000 7faa377fe640 1 -- 172.21.15.115:0/980760429 learned_addr learned my addr 172.21.15.115:0/980760429 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.130+0000 7faa36ffd640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa380b0e60 0x7faa38150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.130+0000 7faa377fe640 1 -- 172.21.15.115:0/980760429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa380ce1b0 msgr2=0x7faa381f0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.130+0000 7faa377fe640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa380ce1b0 0x7faa381f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.130+0000 7faa377fe640 1 -- 172.21.15.115:0/980760429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa380b0e60 msgr2=0x7faa38150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.130+0000 7faa377fe640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa380b0e60 0x7faa38150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.130+0000 7faa377fe640 1 -- 172.21.15.115:0/980760429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa280094d0 con 0x7faa3815ac80 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.130+0000 7faa36ffd640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa380b0e60 0x7faa38150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.131+0000 7faa377fe640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 0x7faa38158200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faa3000c700 tx=0x7faa3000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.131+0000 7faa1f7fe640 1 -- 172.21.15.115:0/980760429 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3000aa50 con 0x7faa3815ac80 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.131+0000 7faa1f7fe640 1 -- 172.21.15.115:0/980760429 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3000abb0 con 0x7faa3815ac80 2023-12-01T01:37:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.131+0000 7faa1f7fe640 1 -- 172.21.15.115:0/980760429 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3000ce10 con 0x7faa3815ac80 2023-12-01T01:37:09.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.131+0000 7faa3d7c9640 1 -- 172.21.15.115:0/980760429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa381f1340 con 0x7faa3815ac80 2023-12-01T01:37:09.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.131+0000 7faa3d7c9640 1 -- 172.21.15.115:0/980760429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa381f0a40 con 0x7faa3815ac80 2023-12-01T01:37:09.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.132+0000 7faa3d7c9640 1 -- 172.21.15.115:0/980760429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa38157f20 con 0x7faa3815ac80 2023-12-01T01:37:09.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.133+0000 7faa1f7fe640 1 -- 172.21.15.115:0/980760429 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faa30012420 con 0x7faa3815ac80 2023-12-01T01:37:09.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.134+0000 7faa1f7fe640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa2406a430 0x7faa2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.135+0000 7faa1f7fe640 1 -- 172.21.15.115:0/980760429 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7faa30088ec0 con 0x7faa3815ac80 2023-12-01T01:37:09.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.135+0000 7faa36ffd640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa2406a430 0x7faa2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.135+0000 7faa36ffd640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa2406a430 0x7faa2406c8f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faa28036000 tx=0x7faa2803b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:09.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.139+0000 7faa1f7fe640 1 -- 172.21.15.115:0/980760429 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faa38157f20 con 0x7faa3815ac80 2023-12-01T01:37:09.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.330+0000 7faa3d7c9640 1 -- 172.21.15.115:0/980760429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa38157f20 con 0x7faa3815ac80 2023-12-01T01:37:09.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.330+0000 7faa1f7fe640 1 -- 172.21.15.115:0/980760429 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa38157f20 con 0x7faa3815ac80 2023-12-01T01:37:09.332 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 -- 172.21.15.115:0/980760429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa2406a430 msgr2=0x7faa2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa2406a430 0x7faa2406c8f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faa28036000 tx=0x7faa2803b040 comp rx=0 tx=0).stop 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 -- 172.21.15.115:0/980760429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 msgr2=0x7faa38158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 0x7faa38158200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faa3000c700 tx=0x7faa3000cbc0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 -- 172.21.15.115:0/980760429 shutdown_connections 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa2406a430 0x7faa2406c8f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa380ce1b0 0x7faa381f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa3815ac80 0x7faa38158200 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 --2- 172.21.15.115:0/980760429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa380b0e60 0x7faa38150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 -- 172.21.15.115:0/980760429 >> 172.21.15.115:0/980760429 conn(0x7faa38065960 msgr2=0x7faa380ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 -- 172.21.15.115:0/980760429 shutdown_connections 2023-12-01T01:37:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.334+0000 7faa3d7c9640 1 -- 172.21.15.115:0/980760429 wait complete. 2023-12-01T01:37:09.356 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:37:09.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.561+0000 7f3bf759e640 1 Processor -- start 2023-12-01T01:37:09.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.561+0000 7f3bf759e640 1 -- start start 2023-12-01T01:37:09.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.563+0000 7f3bf759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3bf80cdcc0 0x7f3bf806b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.563+0000 7f3bf759e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf80ce120 con 0x7f3bf81597d0 2023-12-01T01:37:09.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.563+0000 7f3bf759e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf806b9c0 con 0x7f3bf80adf00 2023-12-01T01:37:09.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.563+0000 7f3bf759e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf806b7b0 con 0x7f3bf80cdcc0 2023-12-01T01:37:09.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.563+0000 7f3bf659c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3bf81597d0 0x7f3bf815db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46638/0 (socket says 172.21.15.115:46638) 2023-12-01T01:37:09.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.563+0000 7f3bf659c640 1 -- 172.21.15.115:0/657780793 learned_addr learned my addr 172.21.15.115:0/657780793 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755614666 0 0) 0x7f3bf80ce120 con 0x7f3bf81597d0 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be4005730 con 0x7f3bf81597d0 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3beffff640 1 --2- 172.21.15.115:0/657780793 >> v2:172.21.15.142:3300/0 conn(0x7f3bf80cdcc0 0x7f3bf806b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1032262606 0 0) 0x7f3be4005730 con 0x7f3bf81597d0 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 >> v2:172.21.15.142:3300/0 conn(0x7f3bf80cdcc0 msgr2=0x7f3bf806b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 --2- 172.21.15.115:0/657780793 >> v2:172.21.15.142:3300/0 conn(0x7f3bf80cdcc0 0x7f3bf806b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 >> v1:172.21.15.142:6789/0 conn(0x7f3bf80adf00 legacy=0x7f3bf80ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf806bca0 con 0x7f3bf81597d0 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3be8001670 con 0x7f3bf81597d0 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3beffff640 1 --2- 172.21.15.115:0/657780793 >> v2:172.21.15.142:3300/0 conn(0x7f3bf80cdcc0 0x7f3bf806b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 >> v1:172.21.15.115:6789/0 conn(0x7f3bf81597d0 legacy=0x7f3bf815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be4005bf0 0x7f3be4005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be40069d0 0x7f3be4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be4007590 0x7f3be400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf806b7b0 con 0x7f3be40069d0 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be400e4b0 con 0x7f3be4005bf0 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.564+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be400e670 con 0x7f3be4007590 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.565+0000 7f3beffff640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be40069d0 0x7f3be4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.565+0000 7f3bf659c640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be4005bf0 0x7f3be4005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.565+0000 7f3bf6d9d640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be4007590 0x7f3be400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.565+0000 7f3beffff640 1 -- 172.21.15.115:0/657780793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be4007590 msgr2=0x7f3be400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.565+0000 7f3beffff640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be4007590 0x7f3be400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.565+0000 7f3beffff640 1 -- 172.21.15.115:0/657780793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be4005bf0 msgr2=0x7f3be4005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.565+0000 7f3beffff640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be4005bf0 0x7f3be4005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.565+0000 7f3bf6d9d640 1 -- 172.21.15.115:0/657780793 reap_dead start 2023-12-01T01:37:09.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.565+0000 7f3beffff640 1 -- 172.21.15.115:0/657780793 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be4005730 con 0x7f3be40069d0 2023-12-01T01:37:09.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.566+0000 7f3beffff640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be40069d0 0x7f3be4006e50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3be000aa80 tx=0x7f3be4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4a3af6dfa20fc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:09.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.567+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0037600 con 0x7f3be40069d0 2023-12-01T01:37:09.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.567+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be0037760 con 0x7f3be40069d0 2023-12-01T01:37:09.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.567+0000 7f3bf5d9b640 1 -- 172.21.15.115:0/657780793 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0037b30 con 0x7f3be40069d0 2023-12-01T01:37:09.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.567+0000 7f3bf759e640 1 -- 172.21.15.115:0/657780793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be40069d0 msgr2=0x7f3be4006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.567+0000 7f3bf759e640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be40069d0 0x7f3be4006e50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3be000aa80 tx=0x7f3be4007a00 comp rx=0 tx=0).stop 2023-12-01T01:37:09.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.568+0000 7f3bf759e640 1 -- 172.21.15.115:0/657780793 shutdown_connections 2023-12-01T01:37:09.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.568+0000 7f3bf759e640 1 --2- 172.21.15.115:0/657780793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3be40069d0 0x7f3be4006e50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.568+0000 7f3bf759e640 1 -- 172.21.15.115:0/657780793 >> 172.21.15.115:0/657780793 conn(0x7f3bf80ce640 msgr2=0x7f3bf80cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:09.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.568+0000 7f3bf759e640 1 -- 172.21.15.115:0/657780793 shutdown_connections 2023-12-01T01:37:09.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.568+0000 7f3bf759e640 1 -- 172.21.15.115:0/657780793 wait complete. 2023-12-01T01:37:09.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.569+0000 7f3bf759e640 1 Processor -- start 2023-12-01T01:37:09.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.570+0000 7f3bf759e640 1 -- start start 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.570+0000 7f3bf759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be40069d0 0x7f3bf806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3bf759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bf80c4af0 0x7f3bf814afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3bf759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bf80c4f70 0x7f3bf814b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3bf759e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80b2660 con 0x7f3bf80c4af0 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3bf759e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf806f450 con 0x7f3be40069d0 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3bf759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80b1d00 con 0x7f3bf80c4f70 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3beffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bf80c4af0 0x7f3bf814afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3beffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bf80c4af0 0x7f3bf814afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36838/0 (socket says 172.21.15.115:36838) 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3beffff640 1 -- 172.21.15.115:0/1014506141 learned_addr learned my addr 172.21.15.115:0/1014506141 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3bf6d9d640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bf80c4f70 0x7f3bf814b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3beffff640 1 -- 172.21.15.115:0/1014506141 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bf80c4f70 msgr2=0x7f3bf814b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3beffff640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bf80c4f70 0x7f3bf814b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3beffff640 1 -- 172.21.15.115:0/1014506141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be40069d0 msgr2=0x7f3bf806ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3bf659c640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be40069d0 0x7f3bf806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3beffff640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be40069d0 0x7f3bf806ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3beffff640 1 -- 172.21.15.115:0/1014506141 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf815ab70 con 0x7f3bf80c4af0 2023-12-01T01:37:09.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.571+0000 7f3bf6d9d640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bf80c4f70 0x7f3bf814b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:09.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.572+0000 7f3beffff640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bf80c4af0 0x7f3bf814afa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3be000a1a0 tx=0x7f3be0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:09.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.572+0000 7f3beeffd640 1 -- 172.21.15.115:0/1014506141 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be003d070 con 0x7f3bf80c4af0 2023-12-01T01:37:09.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.572+0000 7f3beeffd640 1 -- 172.21.15.115:0/1014506141 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be00375f0 con 0x7f3bf80c4af0 2023-12-01T01:37:09.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.572+0000 7f3bf759e640 1 -- 172.21.15.115:0/1014506141 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf80b1fa0 con 0x7f3bf80c4af0 2023-12-01T01:37:09.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.572+0000 7f3beeffd640 1 -- 172.21.15.115:0/1014506141 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be002f810 con 0x7f3bf80c4af0 2023-12-01T01:37:09.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.572+0000 7f3bf759e640 1 -- 172.21.15.115:0/1014506141 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf8159eb0 con 0x7f3bf80c4af0 2023-12-01T01:37:09.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.573+0000 7f3bf759e640 1 -- 172.21.15.115:0/1014506141 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf80b28d0 con 0x7f3bf80c4af0 2023-12-01T01:37:09.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.574+0000 7f3beeffd640 1 -- 172.21.15.115:0/1014506141 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3be002f970 con 0x7f3bf80c4af0 2023-12-01T01:37:09.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.575+0000 7f3beeffd640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3bd806a430 0x7f3bd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:09.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.575+0000 7f3beeffd640 1 -- 172.21.15.115:0/1014506141 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3be00aa650 con 0x7f3bf80c4af0 2023-12-01T01:37:09.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.576+0000 7f3bf659c640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3bd806a430 0x7f3bd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:09.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.576+0000 7f3bf659c640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3bd806a430 0x7f3bd806c8f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3be8001050 tx=0x7f3be8000fe0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:09.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.580+0000 7f3beeffd640 1 -- 172.21.15.115:0/1014506141 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3be0075c10 con 0x7f3bf80c4af0 2023-12-01T01:37:09.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.806+0000 7f3bf759e640 1 -- 172.21.15.115:0/1014506141 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bf806f450 con 0x7f3bf80c4af0 2023-12-01T01:37:09.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.807+0000 7f3beeffd640 1 -- 172.21.15.115:0/1014506141 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4654 (secure 0 0 0) 0x7f3be0079230 con 0x7f3bf80c4af0 2023-12-01T01:37:09.809 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:09.810 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3656795862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3656795862},{"type":"v1","addr":"172.21.15.142:6834","nonce":3656795862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1125002206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1125002206},{"type":"v1","addr":"172.21.15.115:6836","nonce":1125002206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3350782192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3350782192},{"type":"v1","addr":"172.21.15.115:6837","nonce":3350782192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:36:10.637450+0000","modified":"2023-12-01T01:36:13.663004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2808777047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2808777047},{"type":"v1","addr":"172.21.15.142:6835","nonce":2808777047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:37:09.810 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 8 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 -- 172.21.15.115:0/1014506141 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3bd806a430 msgr2=0x7f3bd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3bd806a430 0x7f3bd806c8f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3be8001050 tx=0x7f3be8000fe0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 -- 172.21.15.115:0/1014506141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bf80c4af0 msgr2=0x7f3bf814afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bf80c4af0 0x7f3bf814afa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3be000a1a0 tx=0x7f3be0004730 comp rx=0 tx=0).stop 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 -- 172.21.15.115:0/1014506141 shutdown_connections 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3bd806a430 0x7f3bd806c8f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bf80c4f70 0x7f3bf814b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bf80c4af0 0x7f3bf814afa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 --2- 172.21.15.115:0/1014506141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be40069d0 0x7f3bf806ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 -- 172.21.15.115:0/1014506141 >> 172.21.15.115:0/1014506141 conn(0x7f3bf80ce640 msgr2=0x7f3bf80c9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:09.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 -- 172.21.15.115:0/1014506141 shutdown_connections 2023-12-01T01:37:09.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:09.810+0000 7f3bf759e640 1 -- 172.21.15.115:0/1014506141 wait complete. 2023-12-01T01:37:09.830 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:37:10.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.035+0000 7f10eed3e640 1 Processor -- start 2023-12-01T01:37:10.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.036+0000 7f10eed3e640 1 -- start start 2023-12-01T01:37:10.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.037+0000 7f10eed3e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f10e815e2f0 0x7f10e80ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.037+0000 7f10eed3e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e806f2f0 0x7f10e80caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.037+0000 7f10eed3e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e80add10 con 0x7f10e80adf10 2023-12-01T01:37:10.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.037+0000 7f10eed3e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e80c5760 con 0x7f10e815e2f0 2023-12-01T01:37:10.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.037+0000 7f10eed3e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e815e6d0 con 0x7f10e806f2f0 2023-12-01T01:37:10.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.037+0000 7f10ed2db640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f10e80adf10 0x7f10e80c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46646/0 (socket says 172.21.15.115:46646) 2023-12-01T01:37:10.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.037+0000 7f10ed2db640 1 -- 172.21.15.115:0/2298371169 learned_addr learned my addr 172.21.15.115:0/2298371169 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:10.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.037+0000 7f10ecada640 1 --2- 172.21.15.115:0/2298371169 >> v2:172.21.15.142:3300/0 conn(0x7f10e815e2f0 0x7f10e80ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.037+0000 7f10e7fff640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e806f2f0 0x7f10e80caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.037+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080537419 0 0) 0x7f10e80add10 con 0x7f10e80adf10 2023-12-01T01:37:10.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10d4003520 con 0x7f10e80adf10 2023-12-01T01:37:10.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565142505 0 0) 0x7f10d4003520 con 0x7f10e80adf10 2023-12-01T01:37:10.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e806f2f0 msgr2=0x7f10e80caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:10.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e806f2f0 0x7f10e80caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 >> v2:172.21.15.142:3300/0 conn(0x7f10e815e2f0 msgr2=0x7f10e80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:10.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 --2- 172.21.15.115:0/2298371169 >> v2:172.21.15.142:3300/0 conn(0x7f10e815e2f0 0x7f10e80ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e80cf7b0 con 0x7f10e80adf10 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f10e0001670 con 0x7f10e80adf10 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10ecada640 1 --2- 172.21.15.115:0/2298371169 >> v2:172.21.15.142:3300/0 conn(0x7f10e815e2f0 0x7f10e80ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 >> v1:172.21.15.115:6789/0 conn(0x7f10e80adf10 legacy=0x7f10e80c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e806f2f0 0x7f10d40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10d4003f60 0x7f10d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d4004d00 0x7f10d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e80c5760 con 0x7f10d4003f60 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e815e6d0 con 0x7f10d4004d00 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d400bb90 con 0x7f10e806f2f0 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.038+0000 7f10e7fff640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10d4003f60 0x7f10d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.039+0000 7f10ecada640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e806f2f0 0x7f10d40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.039+0000 7f10ed2db640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d4004d00 0x7f10d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.039+0000 7f10e7fff640 1 -- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e806f2f0 msgr2=0x7f10d40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.039+0000 7f10e7fff640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e806f2f0 0x7f10d40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.039+0000 7f10e7fff640 1 -- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d4004d00 msgr2=0x7f10d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:10.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.039+0000 7f10e7fff640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d4004d00 0x7f10d4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.039+0000 7f10e7fff640 1 -- 172.21.15.115:0/2298371169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d4003520 con 0x7f10d4003f60 2023-12-01T01:37:10.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.039+0000 7f10ecada640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e806f2f0 0x7f10d40039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:37:10.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.040+0000 7f10e7fff640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10d4003f60 0x7f10d40047c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009ac0 tx=0x7f10d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd8aa8a73e78409 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:10.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.040+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d002c6b0 con 0x7f10d4003f60 2023-12-01T01:37:10.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.040+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d002c810 con 0x7f10d4003f60 2023-12-01T01:37:10.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.040+0000 7f10e77fe640 1 -- 172.21.15.115:0/2298371169 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d002cba0 con 0x7f10d4003f60 2023-12-01T01:37:10.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.041+0000 7f10eed3e640 1 -- 172.21.15.115:0/2298371169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10d4003f60 msgr2=0x7f10d40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.041+0000 7f10eed3e640 1 --2- 172.21.15.115:0/2298371169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10d4003f60 0x7f10d40047c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009ac0 tx=0x7f10d40057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.041+0000 7f10ed2db640 1 -- 172.21.15.115:0/2298371169 reap_dead start 2023-12-01T01:37:10.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.041+0000 7f10eed3e640 1 -- 172.21.15.115:0/2298371169 shutdown_connections 2023-12-01T01:37:10.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.041+0000 7f10eed3e640 1 -- 172.21.15.115:0/2298371169 >> 172.21.15.115:0/2298371169 conn(0x7f10e80ce880 msgr2=0x7f10e80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:10.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.041+0000 7f10eed3e640 1 -- 172.21.15.115:0/2298371169 shutdown_connections 2023-12-01T01:37:10.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.041+0000 7f10eed3e640 1 -- 172.21.15.115:0/2298371169 wait complete. 2023-12-01T01:37:10.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.043+0000 7f10eed3e640 1 Processor -- start 2023-12-01T01:37:10.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.043+0000 7f10eed3e640 1 -- start start 2023-12-01T01:37:10.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10eed3e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d4003f60 0x7f10e81483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10eed3e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e80c4530 0x7f10e80c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10eed3e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e806b4c0 0x7f10e806b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10eed3e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e806be80 con 0x7f10e806b4c0 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10eed3e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e814b5a0 con 0x7f10d4003f60 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10eed3e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e814acd0 con 0x7f10e80c4530 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10ed2db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e806b4c0 0x7f10e806b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10ed2db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e806b4c0 0x7f10e806b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36858/0 (socket says 172.21.15.115:36858) 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10ed2db640 1 -- 172.21.15.115:0/2978093622 learned_addr learned my addr 172.21.15.115:0/2978093622 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10e7fff640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e80c4530 0x7f10e80c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10ed2db640 1 -- 172.21.15.115:0/2978093622 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e80c4530 msgr2=0x7f10e80c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.044+0000 7f10ed2db640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e80c4530 0x7f10e80c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.045+0000 7f10ed2db640 1 -- 172.21.15.115:0/2978093622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d4003f60 msgr2=0x7f10e81483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.045+0000 7f10ed2db640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d4003f60 0x7f10e81483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.045+0000 7f10ed2db640 1 -- 172.21.15.115:0/2978093622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e80c50b0 con 0x7f10e806b4c0 2023-12-01T01:37:10.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.045+0000 7f10ed2db640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e806b4c0 0x7f10e806b940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f10e0000c00 tx=0x7f10e000e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:10.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.045+0000 7f10e57fa640 1 -- 172.21.15.115:0/2978093622 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0010510 con 0x7f10e806b4c0 2023-12-01T01:37:10.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.045+0000 7f10e57fa640 1 -- 172.21.15.115:0/2978093622 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e0010670 con 0x7f10e806b4c0 2023-12-01T01:37:10.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.045+0000 7f10e57fa640 1 -- 172.21.15.115:0/2978093622 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0010a20 con 0x7f10e806b4c0 2023-12-01T01:37:10.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.045+0000 7f10eed3e640 1 -- 172.21.15.115:0/2978093622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e814af50 con 0x7f10e806b4c0 2023-12-01T01:37:10.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.045+0000 7f10eed3e640 1 -- 172.21.15.115:0/2978093622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e8148910 con 0x7f10e806b4c0 2023-12-01T01:37:10.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.046+0000 7f10eed3e640 1 -- 172.21.15.115:0/2978093622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e806fda0 con 0x7f10e806b4c0 2023-12-01T01:37:10.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.047+0000 7f10e57fa640 1 -- 172.21.15.115:0/2978093622 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f10e0010b80 con 0x7f10e806b4c0 2023-12-01T01:37:10.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.048+0000 7f10e57fa640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f10c806a430 0x7f10c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.048+0000 7f10e57fa640 1 -- 172.21.15.115:0/2978093622 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f10e0087ce0 con 0x7f10e806b4c0 2023-12-01T01:37:10.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.049+0000 7f10ecada640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f10c806a430 0x7f10c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.049+0000 7f10ecada640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f10c806a430 0x7f10c806c8f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009a30 tx=0x7f10d8003990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:10.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.053+0000 7f10e57fa640 1 -- 172.21.15.115:0/2978093622 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f10e0053210 con 0x7f10e806b4c0 2023-12-01T01:37:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.241+0000 7f10eed3e640 1 -- 172.21.15.115:0/2978093622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10e814b6e0 con 0x7f10e806b4c0 2023-12-01T01:37:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.244+0000 7f10e57fa640 1 -- 172.21.15.115:0/2978093622 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16138 (secure 0 0 0) 0x7f10e814b6e0 con 0x7f10e806b4c0 2023-12-01T01:37:10.246 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:10.246 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":23,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:52.262397+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 -- 172.21.15.115:0/2978093622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f10c806a430 msgr2=0x7f10c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f10c806a430 0x7f10c806c8f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009a30 tx=0x7f10d8003990 comp rx=0 tx=0).stop 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 -- 172.21.15.115:0/2978093622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e806b4c0 msgr2=0x7f10e806b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e806b4c0 0x7f10e806b940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f10e0000c00 tx=0x7f10e000e890 comp rx=0 tx=0).stop 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 -- 172.21.15.115:0/2978093622 shutdown_connections 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f10c806a430 0x7f10c806c8f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10e80c4530 0x7f10e80c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e806b4c0 0x7f10e806b940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 --2- 172.21.15.115:0/2978093622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d4003f60 0x7f10e81483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 -- 172.21.15.115:0/2978093622 >> 172.21.15.115:0/2978093622 conn(0x7f10e80ce880 msgr2=0x7f10e80af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 -- 172.21.15.115:0/2978093622 shutdown_connections 2023-12-01T01:37:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.247+0000 7f10eed3e640 1 -- 172.21.15.115:0/2978093622 wait complete. 2023-12-01T01:37:10.268 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T01:37:10.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:37:10.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.478+0000 7efd9e9cb640 1 Processor -- start 2023-12-01T01:37:10.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.479+0000 7efd9e9cb640 1 -- start start 2023-12-01T01:37:10.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.480+0000 7efd9e9cb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efd9815cd20 0x7efd980d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.480+0000 7efd9e9cb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd980ce2e0 con 0x7efd980d1230 2023-12-01T01:37:10.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.480+0000 7efd9e9cb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd980ce000 con 0x7efd980b5d80 2023-12-01T01:37:10.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.480+0000 7efd9e9cb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd980b2160 con 0x7efd9815cd20 2023-12-01T01:37:10.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.480+0000 7efd97fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efd9815cd20 0x7efd980d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.480+0000 7efd97fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efd9815cd20 0x7efd980d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36862/0 (socket says 172.21.15.115:36862) 2023-12-01T01:37:10.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.480+0000 7efd97fff640 1 -- 172.21.15.115:0/2945998063 learned_addr learned my addr 172.21.15.115:0/2945998063 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:10.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.481+0000 7efd97fff640 1 -- 172.21.15.115:0/2945998063 >> v1:172.21.15.142:6789/0 conn(0x7efd980b5d80 legacy=0x7efd980b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:10.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.481+0000 7efd97fff640 1 -- 172.21.15.115:0/2945998063 >> v1:172.21.15.115:6789/0 conn(0x7efd980d1230 legacy=0x7efd980d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.481+0000 7efd97fff640 1 -- 172.21.15.115:0/2945998063 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd980cd3d0 con 0x7efd9815cd20 2023-12-01T01:37:10.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.482+0000 7efd97fff640 1 --2- 172.21.15.115:0/2945998063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd9815cd20 0x7efd980d24d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efd88009e50 tx=0x7efd8802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c6f485e4a5a6f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:10.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.482+0000 7efd96ffd640 1 -- 172.21.15.115:0/2945998063 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8802ca80 con 0x7efd9815cd20 2023-12-01T01:37:10.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.482+0000 7efd96ffd640 1 -- 172.21.15.115:0/2945998063 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8802cbe0 con 0x7efd9815cd20 2023-12-01T01:37:10.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.482+0000 7efd96ffd640 1 -- 172.21.15.115:0/2945998063 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd880343f0 con 0x7efd9815cd20 2023-12-01T01:37:10.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.483+0000 7efd9e9cb640 1 -- 172.21.15.115:0/2945998063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd9815cd20 msgr2=0x7efd980d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.483+0000 7efd9e9cb640 1 --2- 172.21.15.115:0/2945998063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd9815cd20 0x7efd980d24d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efd88009e50 tx=0x7efd8802bee0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.483+0000 7efd9e9cb640 1 -- 172.21.15.115:0/2945998063 shutdown_connections 2023-12-01T01:37:10.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.483+0000 7efd9e9cb640 1 --2- 172.21.15.115:0/2945998063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd9815cd20 0x7efd980d24d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.483+0000 7efd9e9cb640 1 -- 172.21.15.115:0/2945998063 >> 172.21.15.115:0/2945998063 conn(0x7efd9806a0e0 msgr2=0x7efd9806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:10.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.483+0000 7efd9e9cb640 1 -- 172.21.15.115:0/2945998063 shutdown_connections 2023-12-01T01:37:10.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.483+0000 7efd9e9cb640 1 -- 172.21.15.115:0/2945998063 wait complete. 2023-12-01T01:37:10.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.485+0000 7efd9e9cb640 1 Processor -- start 2023-12-01T01:37:10.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.485+0000 7efd9e9cb640 1 -- start start 2023-12-01T01:37:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.486+0000 7efd9e9cb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd980b5d80 0x7efd980b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.486+0000 7efd9e9cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd980d1230 0x7efd9815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.486+0000 7efd9e9cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd9815cd20 0x7efd981f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.486+0000 7efd9e9cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd980b54b0 con 0x7efd980d1230 2023-12-01T01:37:10.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.486+0000 7efd9e9cb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9815d640 con 0x7efd980b5d80 2023-12-01T01:37:10.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.486+0000 7efd9e9cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd981f22b0 con 0x7efd9815cd20 2023-12-01T01:37:10.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.486+0000 7efd977fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd980d1230 0x7efd9815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.486+0000 7efd977fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd980d1230 0x7efd9815d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36874/0 (socket says 172.21.15.115:36874) 2023-12-01T01:37:10.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.486+0000 7efd977fe640 1 -- 172.21.15.115:0/4127111080 learned_addr learned my addr 172.21.15.115:0/4127111080 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:10.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd977fe640 1 -- 172.21.15.115:0/4127111080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd9815cd20 msgr2=0x7efd981f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd977fe640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd9815cd20 0x7efd981f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd977fe640 1 -- 172.21.15.115:0/4127111080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd980b5d80 msgr2=0x7efd980b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd977fe640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd980b5d80 0x7efd980b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd977fe640 1 -- 172.21.15.115:0/4127111080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd88009b30 con 0x7efd980d1230 2023-12-01T01:37:10.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd977fe640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd980d1230 0x7efd9815d100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c430 tx=0x7efd8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:10.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd94ff9640 1 -- 172.21.15.115:0/4127111080 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80011790 con 0x7efd980d1230 2023-12-01T01:37:10.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd94ff9640 1 -- 172.21.15.115:0/4127111080 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd800118f0 con 0x7efd980d1230 2023-12-01T01:37:10.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd9e9cb640 1 -- 172.21.15.115:0/4127111080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd98160300 con 0x7efd980d1230 2023-12-01T01:37:10.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd94ff9640 1 -- 172.21.15.115:0/4127111080 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80011ce0 con 0x7efd980d1230 2023-12-01T01:37:10.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.487+0000 7efd9e9cb640 1 -- 172.21.15.115:0/4127111080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9815fd40 con 0x7efd980d1230 2023-12-01T01:37:10.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.488+0000 7efd9e9cb640 1 -- 172.21.15.115:0/4127111080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd980c7440 con 0x7efd980d1230 2023-12-01T01:37:10.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.489+0000 7efd94ff9640 1 -- 172.21.15.115:0/4127111080 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd8001c460 con 0x7efd980d1230 2023-12-01T01:37:10.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.490+0000 7efd94ff9640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd7806a460 0x7efd7806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.490+0000 7efd94ff9640 1 -- 172.21.15.115:0/4127111080 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7efd80088910 con 0x7efd980d1230 2023-12-01T01:37:10.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.490+0000 7efd97fff640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd7806a460 0x7efd7806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.491+0000 7efd97fff640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd7806a460 0x7efd7806c920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd88035000 tx=0x7efd8802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:10.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.495+0000 7efd94ff9640 1 -- 172.21.15.115:0/4127111080 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efd980c7440 con 0x7efd980d1230 2023-12-01T01:37:10.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.717+0000 7efd9e9cb640 1 -- 172.21.15.115:0/4127111080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd980c7440 con 0x7efd980d1230 2023-12-01T01:37:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.718+0000 7efd94ff9640 1 -- 172.21.15.115:0/4127111080 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4654 (secure 0 0 0) 0x7efd980c7440 con 0x7efd980d1230 2023-12-01T01:37:10.719 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:10.719 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3656795862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3656795862},{"type":"v1","addr":"172.21.15.142:6834","nonce":3656795862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1125002206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1125002206},{"type":"v1","addr":"172.21.15.115:6836","nonce":1125002206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3350782192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3350782192},{"type":"v1","addr":"172.21.15.115:6837","nonce":3350782192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:36:10.637450+0000","modified":"2023-12-01T01:36:13.663004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2808777047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2808777047},{"type":"v1","addr":"172.21.15.142:6835","nonce":2808777047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:37:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 8 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 -- 172.21.15.115:0/4127111080 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd7806a460 msgr2=0x7efd7806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd7806a460 0x7efd7806c920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd88035000 tx=0x7efd8802e000 comp rx=0 tx=0).stop 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 -- 172.21.15.115:0/4127111080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd980d1230 msgr2=0x7efd9815d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd980d1230 0x7efd9815d100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c430 tx=0x7efd8000c8f0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 -- 172.21.15.115:0/4127111080 shutdown_connections 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd7806a460 0x7efd7806c920 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd9815cd20 0x7efd981f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd980d1230 0x7efd9815d100 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 --2- 172.21.15.115:0/4127111080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd980b5d80 0x7efd980b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 -- 172.21.15.115:0/4127111080 >> 172.21.15.115:0/4127111080 conn(0x7efd9806a0e0 msgr2=0x7efd980d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 -- 172.21.15.115:0/4127111080 shutdown_connections 2023-12-01T01:37:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.720+0000 7efd9e9cb640 1 -- 172.21.15.115:0/4127111080 wait complete. 2023-12-01T01:37:10.740 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:37:10.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.944+0000 7fa60ba96640 1 Processor -- start 2023-12-01T01:37:10.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.944+0000 7fa60ba96640 1 -- start start 2023-12-01T01:37:10.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.945+0000 7fa60ba96640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa60415cd20 0x7fa6040d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.946+0000 7fa60ba96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6040d1230 0x7fa6040d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.946+0000 7fa60ba96640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6040d1b90 con 0x7fa6040b5d80 2023-12-01T01:37:10.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.946+0000 7fa60ba96640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6040b2160 con 0x7fa60415cd20 2023-12-01T01:37:10.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.946+0000 7fa60ba96640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040b8c80 con 0x7fa6040d1230 2023-12-01T01:37:10.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.946+0000 7fa60a033640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6040d1230 0x7fa6040d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.946+0000 7fa60a033640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6040d1230 0x7fa6040d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:39894/0 (socket says 172.21.15.115:39894) 2023-12-01T01:37:10.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.946+0000 7fa60a033640 1 -- 172.21.15.115:0/3081310147 learned_addr learned my addr 172.21.15.115:0/3081310147 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:10.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.946+0000 7fa609031640 1 --1- 172.21.15.115:0/3081310147 >> v1:172.21.15.142:6789/0 conn(0x7fa6040b5d80 0x7fa6040b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:57992/0 (socket says 172.21.15.115:57992) 2023-12-01T01:37:10.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.946+0000 7fa609832640 1 --2- 172.21.15.115:0/3081310147 >> v2:172.21.15.142:3300/0 conn(0x7fa60415cd20 0x7fa6040d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.947+0000 7fa60a033640 1 -- 172.21.15.115:0/3081310147 >> v2:172.21.15.142:3300/0 conn(0x7fa60415cd20 msgr2=0x7fa6040d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:10.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.947+0000 7fa60a033640 1 --2- 172.21.15.115:0/3081310147 >> v2:172.21.15.142:3300/0 conn(0x7fa60415cd20 0x7fa6040d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.947+0000 7fa60a033640 1 -- 172.21.15.115:0/3081310147 >> v1:172.21.15.142:6789/0 conn(0x7fa6040b5d80 legacy=0x7fa6040b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.947+0000 7fa60a033640 1 -- 172.21.15.115:0/3081310147 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040ce2e0 con 0x7fa6040d1230 2023-12-01T01:37:10.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.948+0000 7fa60a033640 1 --2- 172.21.15.115:0/3081310147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6040d1230 0x7fa6040d1650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009e50 tx=0x7fa5fc02bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=569fbfc0494e4e99 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:10.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.949+0000 7fa608830640 1 -- 172.21.15.115:0/3081310147 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc02ca80 con 0x7fa6040d1230 2023-12-01T01:37:10.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.949+0000 7fa608830640 1 -- 172.21.15.115:0/3081310147 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc02cbe0 con 0x7fa6040d1230 2023-12-01T01:37:10.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.949+0000 7fa608830640 1 -- 172.21.15.115:0/3081310147 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc0343f0 con 0x7fa6040d1230 2023-12-01T01:37:10.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.950+0000 7fa60ba96640 1 -- 172.21.15.115:0/3081310147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6040d1230 msgr2=0x7fa6040d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.950+0000 7fa60ba96640 1 --2- 172.21.15.115:0/3081310147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6040d1230 0x7fa6040d1650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009e50 tx=0x7fa5fc02bee0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.950+0000 7fa60ba96640 1 -- 172.21.15.115:0/3081310147 shutdown_connections 2023-12-01T01:37:10.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.950+0000 7fa60ba96640 1 --2- 172.21.15.115:0/3081310147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6040d1230 0x7fa6040d1650 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.950+0000 7fa60ba96640 1 --2- 172.21.15.115:0/3081310147 >> v2:172.21.15.142:3300/0 conn(0x7fa60415cd20 0x7fa6040d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.950+0000 7fa60ba96640 1 -- 172.21.15.115:0/3081310147 >> 172.21.15.115:0/3081310147 conn(0x7fa60406a0e0 msgr2=0x7fa60406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:10.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.950+0000 7fa60ba96640 1 -- 172.21.15.115:0/3081310147 shutdown_connections 2023-12-01T01:37:10.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.950+0000 7fa60ba96640 1 -- 172.21.15.115:0/3081310147 wait complete. 2023-12-01T01:37:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.951+0000 7fa60ba96640 1 Processor -- start 2023-12-01T01:37:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.952+0000 7fa60ba96640 1 -- start start 2023-12-01T01:37:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.952+0000 7fa60ba96640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6040b5d80 0x7fa60415d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.952+0000 7fa60ba96640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040d1230 0x7fa60415d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa60ba96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa60415cd20 0x7fa60415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa60ba96640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040cb720 con 0x7fa6040d1230 2023-12-01T01:37:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa60ba96640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040cc4a0 con 0x7fa6040b5d80 2023-12-01T01:37:10.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa60ba96640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6040caaa0 con 0x7fa60415cd20 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609031640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040d1230 0x7fa60415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609031640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040d1230 0x7fa60415d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36890/0 (socket says 172.21.15.115:36890) 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609031640 1 -- 172.21.15.115:0/2351712891 learned_addr learned my addr 172.21.15.115:0/2351712891 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa60a033640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa60415cd20 0x7fa60415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609832640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6040b5d80 0x7fa60415d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609031640 1 -- 172.21.15.115:0/2351712891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa60415cd20 msgr2=0x7fa60415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609031640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa60415cd20 0x7fa60415de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609031640 1 -- 172.21.15.115:0/2351712891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6040b5d80 msgr2=0x7fa60415d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609031640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6040b5d80 0x7fa60415d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609031640 1 -- 172.21.15.115:0/2351712891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc009b30 con 0x7fa6040d1230 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609832640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6040b5d80 0x7fa60415d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.953+0000 7fa609031640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040d1230 0x7fa60415d920 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400c3e0 tx=0x7fa5f400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.954+0000 7fa5fa7fc640 1 -- 172.21.15.115:0/2351712891 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40074e0 con 0x7fa6040d1230 2023-12-01T01:37:10.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.954+0000 7fa5fa7fc640 1 -- 172.21.15.115:0/2351712891 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f4007640 con 0x7fa6040d1230 2023-12-01T01:37:10.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.954+0000 7fa60ba96640 1 -- 172.21.15.115:0/2351712891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6040cad80 con 0x7fa6040d1230 2023-12-01T01:37:10.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.954+0000 7fa5fa7fc640 1 -- 172.21.15.115:0/2351712891 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40049e0 con 0x7fa6040d1230 2023-12-01T01:37:10.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.954+0000 7fa60ba96640 1 -- 172.21.15.115:0/2351712891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6040cb340 con 0x7fa6040d1230 2023-12-01T01:37:10.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.955+0000 7fa60ba96640 1 -- 172.21.15.115:0/2351712891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60415d100 con 0x7fa6040d1230 2023-12-01T01:37:10.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.956+0000 7fa5fa7fc640 1 -- 172.21.15.115:0/2351712891 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa5f4004b40 con 0x7fa6040d1230 2023-12-01T01:37:10.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.957+0000 7fa5fa7fc640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e806a430 0x7fa5e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:10.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.957+0000 7fa5fa7fc640 1 -- 172.21.15.115:0/2351712891 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa5f4088100 con 0x7fa6040d1230 2023-12-01T01:37:10.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.957+0000 7fa609832640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e806a430 0x7fa5e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:10.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.958+0000 7fa609832640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e806a430 0x7fa5e806c8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa600009ac0 tx=0x7fa600008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:10.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:10.961+0000 7fa5fa7fc640 1 -- 172.21.15.115:0/2351712891 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa60415d100 con 0x7fa6040d1230 2023-12-01T01:37:11.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.172+0000 7fa60ba96640 1 -- 172.21.15.115:0/2351712891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa60415d100 con 0x7fa6040d1230 2023-12-01T01:37:11.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.173+0000 7fa5fa7fc640 1 -- 172.21.15.115:0/2351712891 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4654 (secure 0 0 0) 0x7fa60415d100 con 0x7fa6040d1230 2023-12-01T01:37:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:11.175 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3656795862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3656795862},{"type":"v1","addr":"172.21.15.142:6834","nonce":3656795862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1125002206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1125002206},{"type":"v1","addr":"172.21.15.115:6836","nonce":1125002206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3350782192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3350782192},{"type":"v1","addr":"172.21.15.115:6837","nonce":3350782192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:36:10.637450+0000","modified":"2023-12-01T01:36:13.663004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/2808777047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2808777047},{"type":"v1","addr":"172.21.15.142:6835","nonce":2808777047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:37:11.175 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 8 2023-12-01T01:37:11.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.175+0000 7fa60ba96640 1 -- 172.21.15.115:0/2351712891 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e806a430 msgr2=0x7fa5e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.175+0000 7fa60ba96640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e806a430 0x7fa5e806c8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa600009ac0 tx=0x7fa600008040 comp rx=0 tx=0).stop 2023-12-01T01:37:11.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.175+0000 7fa60ba96640 1 -- 172.21.15.115:0/2351712891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040d1230 msgr2=0x7fa60415d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.175+0000 7fa60ba96640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040d1230 0x7fa60415d920 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400c3e0 tx=0x7fa5f400c8a0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.176+0000 7fa60ba96640 1 -- 172.21.15.115:0/2351712891 shutdown_connections 2023-12-01T01:37:11.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.176+0000 7fa60ba96640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e806a430 0x7fa5e806c8f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.176+0000 7fa60ba96640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa60415cd20 0x7fa60415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.176+0000 7fa60ba96640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6040d1230 0x7fa60415d920 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.176+0000 7fa60ba96640 1 --2- 172.21.15.115:0/2351712891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6040b5d80 0x7fa60415d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.176+0000 7fa60ba96640 1 -- 172.21.15.115:0/2351712891 >> 172.21.15.115:0/2351712891 conn(0x7fa60406a0e0 msgr2=0x7fa6040d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:11.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.176+0000 7fa60ba96640 1 -- 172.21.15.115:0/2351712891 shutdown_connections 2023-12-01T01:37:11.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.176+0000 7fa60ba96640 1 -- 172.21.15.115:0/2351712891 wait complete. 2023-12-01T01:37:11.193 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:37:11.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.394+0000 7f1c53243640 1 Processor -- start 2023-12-01T01:37:11.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.394+0000 7f1c53243640 1 -- start start 2023-12-01T01:37:11.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.395+0000 7f1c53243640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1c4c0b8610 0x7f1c4c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c53243640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c4c0cdf00 0x7f1c4c0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c53243640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4c0d1740 con 0x7f1c4c15cf90 2023-12-01T01:37:11.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c53243640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c0ce370 con 0x7f1c4c0b8610 2023-12-01T01:37:11.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c53243640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c15d370 con 0x7f1c4c0cdf00 2023-12-01T01:37:11.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c50fdf640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1c4c15cf90 0x7f1c4c0d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46670/0 (socket says 172.21.15.115:46670) 2023-12-01T01:37:11.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c50fdf640 1 -- 172.21.15.115:0/1409894235 learned_addr learned my addr 172.21.15.115:0/1409894235 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:11.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c4bfff640 1 --2- 172.21.15.115:0/1409894235 >> v2:172.21.15.115:3300/0 conn(0x7f1c4c0b8610 0x7f1c4c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c517e0640 1 --2- 172.21.15.115:0/1409894235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c4c0cdf00 0x7f1c4c0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c4b7fe640 1 -- 172.21.15.115:0/1409894235 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3235838057 0 0) 0x7f1c4c0d1740 con 0x7f1c4c15cf90 2023-12-01T01:37:11.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c4b7fe640 1 -- 172.21.15.115:0/1409894235 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c3c003520 con 0x7f1c4c15cf90 2023-12-01T01:37:11.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c4bfff640 1 -- 172.21.15.115:0/1409894235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c4c0cdf00 msgr2=0x7f1c4c0b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:11.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c4bfff640 1 --2- 172.21.15.115:0/1409894235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c4c0cdf00 0x7f1c4c0b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c4bfff640 1 -- 172.21.15.115:0/1409894235 >> v1:172.21.15.115:6789/0 conn(0x7f1c4c15cf90 legacy=0x7f1c4c0d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.396+0000 7f1c4bfff640 1 -- 172.21.15.115:0/1409894235 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c0d19a0 con 0x7f1c4c0b8610 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.398+0000 7f1c4bfff640 1 --2- 172.21.15.115:0/1409894235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c0b8610 0x7f1c4c0cd9c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1c380097a0 tx=0x7f1c3802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d265aa44d7a6b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.398+0000 7f1c4b7fe640 1 -- 172.21.15.115:0/1409894235 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3802d040 con 0x7f1c4c0b8610 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.398+0000 7f1c4b7fe640 1 -- 172.21.15.115:0/1409894235 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c38031810 con 0x7f1c4c0b8610 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.398+0000 7f1c4b7fe640 1 -- 172.21.15.115:0/1409894235 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38031ba0 con 0x7f1c4c0b8610 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.398+0000 7f1c53243640 1 -- 172.21.15.115:0/1409894235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c0b8610 msgr2=0x7f1c4c0cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.398+0000 7f1c53243640 1 --2- 172.21.15.115:0/1409894235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c0b8610 0x7f1c4c0cd9c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1c380097a0 tx=0x7f1c3802b8b0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.398+0000 7f1c53243640 1 -- 172.21.15.115:0/1409894235 shutdown_connections 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.398+0000 7f1c53243640 1 --2- 172.21.15.115:0/1409894235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c4c0cdf00 0x7f1c4c0b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.398+0000 7f1c53243640 1 --2- 172.21.15.115:0/1409894235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c0b8610 0x7f1c4c0cd9c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.398+0000 7f1c53243640 1 -- 172.21.15.115:0/1409894235 >> 172.21.15.115:0/1409894235 conn(0x7f1c4c065960 msgr2=0x7f1c4c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.399+0000 7f1c53243640 1 -- 172.21.15.115:0/1409894235 shutdown_connections 2023-12-01T01:37:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.399+0000 7f1c53243640 1 -- 172.21.15.115:0/1409894235 wait complete. 2023-12-01T01:37:11.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.400+0000 7f1c53243640 1 Processor -- start 2023-12-01T01:37:11.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.400+0000 7f1c53243640 1 -- start start 2023-12-01T01:37:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.401+0000 7f1c53243640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c4c0b8610 0x7f1c4c0b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.401+0000 7f1c53243640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c4c0cdf00 0x7f1c4c15fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.401+0000 7f1c53243640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c15cf90 0x7f1c4c140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.401+0000 7f1c53243640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b6d90 con 0x7f1c4c15cf90 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.401+0000 7f1c53243640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b24e0 con 0x7f1c4c0b8610 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.401+0000 7f1c53243640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b27c0 con 0x7f1c4c0cdf00 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c517e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c15cf90 0x7f1c4c140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c517e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c15cf90 0x7f1c4c140a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36902/0 (socket says 172.21.15.115:36902) 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c517e0640 1 -- 172.21.15.115:0/2500665166 learned_addr learned my addr 172.21.15.115:0/2500665166 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c50fdf640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c4c0b8610 0x7f1c4c0b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c4bfff640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c4c0cdf00 0x7f1c4c15fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c517e0640 1 -- 172.21.15.115:0/2500665166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c4c0cdf00 msgr2=0x7f1c4c15fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c517e0640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c4c0cdf00 0x7f1c4c15fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c517e0640 1 -- 172.21.15.115:0/2500665166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c4c0b8610 msgr2=0x7f1c4c0b7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c517e0640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c4c0b8610 0x7f1c4c0b7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c517e0640 1 -- 172.21.15.115:0/2500665166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c38009400 con 0x7f1c4c15cf90 2023-12-01T01:37:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c50fdf640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c4c0b8610 0x7f1c4c0b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.402+0000 7f1c517e0640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c15cf90 0x7f1c4c140a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1c44007e40 tx=0x7f1c4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.403+0000 7f1c497fa640 1 -- 172.21.15.115:0/2500665166 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44014620 con 0x7f1c4c15cf90 2023-12-01T01:37:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.403+0000 7f1c497fa640 1 -- 172.21.15.115:0/2500665166 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4400ee90 con 0x7f1c4c15cf90 2023-12-01T01:37:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.403+0000 7f1c53243640 1 -- 172.21.15.115:0/2500665166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c0b6a20 con 0x7f1c4c15cf90 2023-12-01T01:37:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.403+0000 7f1c497fa640 1 -- 172.21.15.115:0/2500665166 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c440149e0 con 0x7f1c4c15cf90 2023-12-01T01:37:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.403+0000 7f1c53243640 1 -- 172.21.15.115:0/2500665166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c0b2170 con 0x7f1c4c15cf90 2023-12-01T01:37:11.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.404+0000 7f1c53243640 1 -- 172.21.15.115:0/2500665166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c0b2980 con 0x7f1c4c15cf90 2023-12-01T01:37:11.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.405+0000 7f1c497fa640 1 -- 172.21.15.115:0/2500665166 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1c44014b40 con 0x7f1c4c15cf90 2023-12-01T01:37:11.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.406+0000 7f1c497fa640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1c3006a430 0x7f1c3006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.406+0000 7f1c497fa640 1 -- 172.21.15.115:0/2500665166 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1c44089380 con 0x7f1c4c15cf90 2023-12-01T01:37:11.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.406+0000 7f1c50fdf640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1c3006a430 0x7f1c3006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.407+0000 7f1c50fdf640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1c3006a430 0x7f1c3006c8f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1c400011e0 tx=0x7f1c40009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:11.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.410+0000 7f1c497fa640 1 -- 172.21.15.115:0/2500665166 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1c440548b0 con 0x7f1c4c15cf90 2023-12-01T01:37:11.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.602+0000 7f1c53243640 1 -- 172.21.15.115:0/2500665166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c4c04e890 con 0x7f1c4c15cf90 2023-12-01T01:37:11.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.605+0000 7f1c497fa640 1 -- 172.21.15.115:0/2500665166 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16138 (secure 0 0 0) 0x7f1c44057ed0 con 0x7f1c4c15cf90 2023-12-01T01:37:11.607 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:11.607 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":23,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:36:52.262397+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:36:08.429168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:36:09.629775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 -- 172.21.15.115:0/2500665166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1c3006a430 msgr2=0x7f1c3006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1c3006a430 0x7f1c3006c8f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1c400011e0 tx=0x7f1c40009040 comp rx=0 tx=0).stop 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 -- 172.21.15.115:0/2500665166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c15cf90 msgr2=0x7f1c4c140a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c15cf90 0x7f1c4c140a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1c44007e40 tx=0x7f1c4400e4b0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 -- 172.21.15.115:0/2500665166 shutdown_connections 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1c3006a430 0x7f1c3006c8f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c4c0cdf00 0x7f1c4c15fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c4c15cf90 0x7f1c4c140a70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 --2- 172.21.15.115:0/2500665166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c4c0b8610 0x7f1c4c0b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 -- 172.21.15.115:0/2500665166 >> 172.21.15.115:0/2500665166 conn(0x7f1c4c065960 msgr2=0x7f1c4c0c6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 -- 172.21.15.115:0/2500665166 shutdown_connections 2023-12-01T01:37:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.608+0000 7f1c53243640 1 -- 172.21.15.115:0/2500665166 wait complete. 2023-12-01T01:37:11.628 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T01:37:11.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.821+0000 7ff5f37cb640 1 Processor -- start 2023-12-01T01:37:11.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.822+0000 7ff5f37cb640 1 -- start start 2023-12-01T01:37:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.823+0000 7ff5f37cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ec15cf90 0x7ff5ec0d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.823+0000 7ff5f37cb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5ec0b8610 0x7ff5ec0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.823+0000 7ff5f37cb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff5ec0cdf00 0x7ff5ec0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.823+0000 7ff5f37cb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ec0ce390 con 0x7ff5ec0b8610 2023-12-01T01:37:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.823+0000 7ff5f37cb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ec15d370 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.823+0000 7ff5f37cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec0d19a0 con 0x7ff5ec15cf90 2023-12-01T01:37:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.823+0000 7ff5f0d66640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5ec0b8610 0x7ff5ec0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.823+0000 7ff5f0d66640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5ec0b8610 0x7ff5ec0cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36904/0 (socket says 172.21.15.115:36904) 2023-12-01T01:37:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.823+0000 7ff5f0d66640 1 -- 172.21.15.115:0/1146218694 learned_addr learned my addr 172.21.15.115:0/1146218694 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.824+0000 7ff5f1d68640 1 --2- 172.21.15.115:0/1146218694 >> v2:172.21.15.142:3300/0 conn(0x7ff5ec0cdf00 0x7ff5ec0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.824+0000 7ff5f1567640 1 --2- 172.21.15.115:0/1146218694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ec15cf90 0x7ff5ec0d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.824+0000 7ff5f0d66640 1 -- 172.21.15.115:0/1146218694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ec15cf90 msgr2=0x7ff5ec0d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.824+0000 7ff5f0d66640 1 --2- 172.21.15.115:0/1146218694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ec15cf90 0x7ff5ec0d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.824+0000 7ff5f0d66640 1 -- 172.21.15.115:0/1146218694 >> v2:172.21.15.142:3300/0 conn(0x7ff5ec0cdf00 msgr2=0x7ff5ec0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.824+0000 7ff5f0d66640 1 --2- 172.21.15.115:0/1146218694 >> v2:172.21.15.142:3300/0 conn(0x7ff5ec0cdf00 0x7ff5ec0b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.824+0000 7ff5f0d66640 1 -- 172.21.15.115:0/1146218694 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec0d1c80 con 0x7ff5ec0b8610 2023-12-01T01:37:11.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.825+0000 7ff5f0d66640 1 --2- 172.21.15.115:0/1146218694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0b8610 0x7ff5ec0cd9c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec066930 tx=0x7ff5e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5a76ad5ae27f0f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:11.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.825+0000 7ff5dffff640 1 -- 172.21.15.115:0/1146218694 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e802ca70 con 0x7ff5ec0b8610 2023-12-01T01:37:11.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.825+0000 7ff5dffff640 1 -- 172.21.15.115:0/1146218694 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e802cbd0 con 0x7ff5ec0b8610 2023-12-01T01:37:11.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.825+0000 7ff5dffff640 1 -- 172.21.15.115:0/1146218694 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80343f0 con 0x7ff5ec0b8610 2023-12-01T01:37:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.826+0000 7ff5f37cb640 1 -- 172.21.15.115:0/1146218694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0b8610 msgr2=0x7ff5ec0cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.826+0000 7ff5f37cb640 1 --2- 172.21.15.115:0/1146218694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0b8610 0x7ff5ec0cd9c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec066930 tx=0x7ff5e802bed0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.826+0000 7ff5f37cb640 1 -- 172.21.15.115:0/1146218694 shutdown_connections 2023-12-01T01:37:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.826+0000 7ff5f37cb640 1 --2- 172.21.15.115:0/1146218694 >> v2:172.21.15.142:3300/0 conn(0x7ff5ec0cdf00 0x7ff5ec0b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.826+0000 7ff5f37cb640 1 --2- 172.21.15.115:0/1146218694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0b8610 0x7ff5ec0cd9c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.826+0000 7ff5f37cb640 1 --2- 172.21.15.115:0/1146218694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ec15cf90 0x7ff5ec0d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.826+0000 7ff5f37cb640 1 -- 172.21.15.115:0/1146218694 >> 172.21.15.115:0/1146218694 conn(0x7ff5ec065960 msgr2=0x7ff5ec065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:11.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.826+0000 7ff5f37cb640 1 -- 172.21.15.115:0/1146218694 shutdown_connections 2023-12-01T01:37:11.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.826+0000 7ff5f37cb640 1 -- 172.21.15.115:0/1146218694 wait complete. 2023-12-01T01:37:11.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.828+0000 7ff5f37cb640 1 Processor -- start 2023-12-01T01:37:11.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.828+0000 7ff5f37cb640 1 -- start start 2023-12-01T01:37:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.829+0000 7ff5f37cb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ec0b8610 0x7ff5ec15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.829+0000 7ff5f37cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0cdf00 0x7ff5ec0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.829+0000 7ff5f37cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ec15cf90 0x7ff5ec0cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.829+0000 7ff5f37cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec15ab00 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.829+0000 7ff5f37cb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec0ca030 con 0x7ff5ec0b8610 2023-12-01T01:37:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.829+0000 7ff5f37cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec0ca310 con 0x7ff5ec15cf90 2023-12-01T01:37:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.829+0000 7ff5f0d66640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0cdf00 0x7ff5ec0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.829+0000 7ff5f0d66640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0cdf00 0x7ff5ec0cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36912/0 (socket says 172.21.15.115:36912) 2023-12-01T01:37:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.829+0000 7ff5f0d66640 1 -- 172.21.15.115:0/2736284277 learned_addr learned my addr 172.21.15.115:0/2736284277 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:11.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.830+0000 7ff5f0d66640 1 -- 172.21.15.115:0/2736284277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ec15cf90 msgr2=0x7ff5ec0cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.830+0000 7ff5f0d66640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ec15cf90 0x7ff5ec0cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.830+0000 7ff5f0d66640 1 -- 172.21.15.115:0/2736284277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ec0b8610 msgr2=0x7ff5ec15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:11.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.830+0000 7ff5f0d66640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ec0b8610 0x7ff5ec15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:11.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.830+0000 7ff5f0d66640 1 -- 172.21.15.115:0/2736284277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8009b30 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.830+0000 7ff5f0d66640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0cdf00 0x7ff5ec0cb820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8000f80 tx=0x7ff5e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.830+0000 7ff5ddffb640 1 -- 172.21.15.115:0/2736284277 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8009510 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.830+0000 7ff5ddffb640 1 -- 172.21.15.115:0/2736284277 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e803e450 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.830+0000 7ff5f37cb640 1 -- 172.21.15.115:0/2736284277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ec0ca5f0 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.831+0000 7ff5ddffb640 1 -- 172.21.15.115:0/2736284277 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e803e800 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.831+0000 7ff5f37cb640 1 -- 172.21.15.115:0/2736284277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ec0cabb0 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.832+0000 7ff5f37cb640 1 -- 172.21.15.115:0/2736284277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ec0cada0 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.833+0000 7ff5ddffb640 1 -- 172.21.15.115:0/2736284277 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff5e803e960 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.833+0000 7ff5ddffb640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5d006a460 0x7ff5d006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:11.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.834+0000 7ff5ddffb640 1 -- 172.21.15.115:0/2736284277 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff5e80ab780 con 0x7ff5ec0cdf00 2023-12-01T01:37:11.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.834+0000 7ff5f1567640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5d006a460 0x7ff5d006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:11.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.834+0000 7ff5f1567640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5d006a460 0x7ff5d006c920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0006fd0 tx=0x7ff5e0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:11.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:11.838+0000 7ff5ddffb640 1 -- 172.21.15.115:0/2736284277 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff5e8076d40 con 0x7ff5ec0cdf00 2023-12-01T01:37:12.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.051+0000 7ff5f37cb640 1 -- 172.21.15.115:0/2736284277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff5ec04e890 con 0x7ff5ec0cdf00 2023-12-01T01:37:12.054 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T01:37:12.053+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:37:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.067+0000 7ff5ddffb640 1 -- 172.21.15.115:0/2736284277 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff5e807a360 con 0x7ff5ec0cdf00 2023-12-01T01:37:12.069 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T01:37:12.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 -- 172.21.15.115:0/2736284277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5d006a460 msgr2=0x7ff5d006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5d006a460 0x7ff5d006c920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0006fd0 tx=0x7ff5e0008040 comp rx=0 tx=0).stop 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 -- 172.21.15.115:0/2736284277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0cdf00 msgr2=0x7ff5ec0cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0cdf00 0x7ff5ec0cb820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8000f80 tx=0x7ff5e8004a40 comp rx=0 tx=0).stop 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 -- 172.21.15.115:0/2736284277 shutdown_connections 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5d006a460 0x7ff5d006c920 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ec15cf90 0x7ff5ec0cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ec0cdf00 0x7ff5ec0cb820 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 --2- 172.21.15.115:0/2736284277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ec0b8610 0x7ff5ec15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 -- 172.21.15.115:0/2736284277 >> 172.21.15.115:0/2736284277 conn(0x7ff5ec065960 msgr2=0x7ff5ec0b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 -- 172.21.15.115:0/2736284277 shutdown_connections 2023-12-01T01:37:12.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.070+0000 7ff5f37cb640 1 -- 172.21.15.115:0/2736284277 wait complete. 2023-12-01T01:37:12.088 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T01:37:12.105 INFO:tasks.ceph.mds.d.smithi142.stderr: -7358> 2023-12-01T01:36:16.600+0000 7f73e741c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:37:12.195 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T01:37:12.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.287+0000 7f26dee99640 1 Processor -- start 2023-12-01T01:37:12.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.288+0000 7f26dee99640 1 -- start start 2023-12-01T01:37:12.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.289+0000 7f26dee99640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f26d815cd90 0x7f26d80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:12.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.289+0000 7f26dee99640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d80ce5e0 con 0x7f26d80b5fa0 2023-12-01T01:37:12.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.289+0000 7f26dee99640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d80ce300 con 0x7f26d80cfd60 2023-12-01T01:37:12.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.289+0000 7f26dee99640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d80d0ec0 con 0x7f26d815cd90 2023-12-01T01:37:12.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.289+0000 7f26d7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f26d80b5fa0 0x7f26d80cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46684/0 (socket says 172.21.15.115:46684) 2023-12-01T01:37:12.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.289+0000 7f26d7fff640 1 -- 172.21.15.115:0/596030923 learned_addr learned my addr 172.21.15.115:0/596030923 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:12.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.290+0000 7f26dcc35640 1 --2- 172.21.15.115:0/596030923 >> v2:172.21.15.115:3300/0 conn(0x7f26d815cd90 0x7f26d80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:12.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.290+0000 7f26d77fe640 1 -- 172.21.15.115:0/596030923 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294692601 0 0) 0x7f26c0001110 con 0x7f26d80b5fa0 2023-12-01T01:37:12.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.290+0000 7f26d77fe640 1 -- 172.21.15.115:0/596030923 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26c4003520 con 0x7f26d80b5fa0 2023-12-01T01:37:12.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.290+0000 7f26dcc35640 1 -- 172.21.15.115:0/596030923 >> v1:172.21.15.142:6789/0 conn(0x7f26d80cfd60 legacy=0x7f26d80d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:12.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.290+0000 7f26dcc35640 1 -- 172.21.15.115:0/596030923 >> v1:172.21.15.115:6789/0 conn(0x7f26d80b5fa0 legacy=0x7f26d80cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:12.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.290+0000 7f26dcc35640 1 -- 172.21.15.115:0/596030923 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d80d10a0 con 0x7f26d815cd90 2023-12-01T01:37:12.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.292+0000 7f26dcc35640 1 --2- 172.21.15.115:0/596030923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 0x7f26d80cdd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009d90 tx=0x7f26c802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934c1808044d6cf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:12.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.292+0000 7f26d77fe640 1 -- 172.21.15.115:0/596030923 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c802cd00 con 0x7f26d815cd90 2023-12-01T01:37:12.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.292+0000 7f26d77fe640 1 -- 172.21.15.115:0/596030923 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c802ce60 con 0x7f26d815cd90 2023-12-01T01:37:12.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.292+0000 7f26d77fe640 1 -- 172.21.15.115:0/596030923 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8034660 con 0x7f26d815cd90 2023-12-01T01:37:12.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.292+0000 7f26dee99640 1 -- 172.21.15.115:0/596030923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 msgr2=0x7f26d80cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:12.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.292+0000 7f26dee99640 1 --2- 172.21.15.115:0/596030923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 0x7f26d80cdd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009d90 tx=0x7f26c802bf50 comp rx=0 tx=0).stop 2023-12-01T01:37:12.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.292+0000 7f26dee99640 1 -- 172.21.15.115:0/596030923 shutdown_connections 2023-12-01T01:37:12.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.293+0000 7f26dee99640 1 --2- 172.21.15.115:0/596030923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 0x7f26d80cdd20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.293+0000 7f26dee99640 1 -- 172.21.15.115:0/596030923 >> 172.21.15.115:0/596030923 conn(0x7f26d8065960 msgr2=0x7f26d8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.293+0000 7f26dee99640 1 -- 172.21.15.115:0/596030923 shutdown_connections 2023-12-01T01:37:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.293+0000 7f26dee99640 1 -- 172.21.15.115:0/596030923 wait complete. 2023-12-01T01:37:12.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.294+0000 7f26dee99640 1 Processor -- start 2023-12-01T01:37:12.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.295+0000 7f26dee99640 1 -- start start 2023-12-01T01:37:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.295+0000 7f26dee99640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80b5fa0 0x7f26d815b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dee99640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26d80cfd60 0x7f26d8156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dee99640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 0x7f26d80cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dee99640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d81e81d0 con 0x7f26d815cd90 2023-12-01T01:37:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dee99640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d81e7550 con 0x7f26d80cfd60 2023-12-01T01:37:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dee99640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d81e7830 con 0x7f26d80b5fa0 2023-12-01T01:37:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dd436640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 0x7f26d80cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:12.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dd436640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 0x7f26d80cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36926/0 (socket says 172.21.15.115:36926) 2023-12-01T01:37:12.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dd436640 1 -- 172.21.15.115:0/2363106903 learned_addr learned my addr 172.21.15.115:0/2363106903 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:12.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dd436640 1 -- 172.21.15.115:0/2363106903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80b5fa0 msgr2=0x7f26d815b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:12.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dd436640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80b5fa0 0x7f26d815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:12.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dd436640 1 -- 172.21.15.115:0/2363106903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26d80cfd60 msgr2=0x7f26d8156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:12.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dd436640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26d80cfd60 0x7f26d8156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:12.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.296+0000 7f26dd436640 1 -- 172.21.15.115:0/2363106903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8009a70 con 0x7f26d815cd90 2023-12-01T01:37:12.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.297+0000 7f26dd436640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 0x7f26d80cbda0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f26d0001eb0 tx=0x7f26d000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:12.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.297+0000 7f26d57fa640 1 -- 172.21.15.115:0/2363106903 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d00052d0 con 0x7f26d815cd90 2023-12-01T01:37:12.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.297+0000 7f26d57fa640 1 -- 172.21.15.115:0/2363106903 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d0005430 con 0x7f26d815cd90 2023-12-01T01:37:12.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.297+0000 7f26d57fa640 1 -- 172.21.15.115:0/2363106903 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0003810 con 0x7f26d815cd90 2023-12-01T01:37:12.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.297+0000 7f26dee99640 1 -- 172.21.15.115:0/2363106903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d81e7b10 con 0x7f26d815cd90 2023-12-01T01:37:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.297+0000 7f26dee99640 1 -- 172.21.15.115:0/2363106903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d815b770 con 0x7f26d815cd90 2023-12-01T01:37:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.298+0000 7f26dee99640 1 -- 172.21.15.115:0/2363106903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d804e890 con 0x7f26d815cd90 2023-12-01T01:37:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.299+0000 7f26d57fa640 1 -- 172.21.15.115:0/2363106903 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f26d0003970 con 0x7f26d815cd90 2023-12-01T01:37:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.300+0000 7f26d57fa640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f26b806a430 0x7f26b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.300+0000 7f26d57fa640 1 -- 172.21.15.115:0/2363106903 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f26d0087fb0 con 0x7f26d815cd90 2023-12-01T01:37:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.300+0000 7f26dcc35640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f26b806a430 0x7f26b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.301+0000 7f26dcc35640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f26b806a430 0x7f26b806c8f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f26c8036000 tx=0x7f26c802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:12.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.304+0000 7f26d57fa640 1 -- 172.21.15.115:0/2363106903 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f26d00534b0 con 0x7f26d815cd90 2023-12-01T01:37:12.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:12.537+0000 7f26dee99640 1 -- 172.21.15.115:0/2363106903 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f26d80d0a70 con 0x7f26d815cd90 2023-12-01T01:37:13.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.079+0000 7f26d57fa640 1 -- 172.21.15.115:0/2363106903 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f26d0056ad0 con 0x7f26d815cd90 2023-12-01T01:37:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.082+0000 7f26dee99640 1 -- 172.21.15.115:0/2363106903 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f26b806a430 msgr2=0x7f26b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.082+0000 7f26dee99640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f26b806a430 0x7f26b806c8f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f26c8036000 tx=0x7f26c802e000 comp rx=0 tx=0).stop 2023-12-01T01:37:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.082+0000 7f26dee99640 1 -- 172.21.15.115:0/2363106903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 msgr2=0x7f26d80cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.082+0000 7f26dee99640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 0x7f26d80cbda0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f26d0001eb0 tx=0x7f26d000c890 comp rx=0 tx=0).stop 2023-12-01T01:37:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.082+0000 7f26dee99640 1 -- 172.21.15.115:0/2363106903 shutdown_connections 2023-12-01T01:37:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.082+0000 7f26dee99640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f26b806a430 0x7f26b806c8f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.082+0000 7f26dee99640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26d815cd90 0x7f26d80cbda0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:13.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.082+0000 7f26dee99640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26d80cfd60 0x7f26d8156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:13.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.082+0000 7f26dee99640 1 --2- 172.21.15.115:0/2363106903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80b5fa0 0x7f26d815b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:13.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.082+0000 7f26dee99640 1 -- 172.21.15.115:0/2363106903 >> 172.21.15.115:0/2363106903 conn(0x7f26d8065960 msgr2=0x7f26d80b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:13.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.083+0000 7f26dee99640 1 -- 172.21.15.115:0/2363106903 shutdown_connections 2023-12-01T01:37:13.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.083+0000 7f26dee99640 1 -- 172.21.15.115:0/2363106903 wait complete. 2023-12-01T01:37:13.104 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T01:37:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.316+0000 7f1de0cdb640 1 Processor -- start 2023-12-01T01:37:13.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.317+0000 7f1de0cdb640 1 -- start start 2023-12-01T01:37:13.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.318+0000 7f1de0cdb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15ce40 0x7f1ddc0b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:13.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.318+0000 7f1de0cdb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1ddc0d2730 0x7f1ddc0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:13.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.318+0000 7f1de0cdb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ddc0d1460 con 0x7f1ddc0b4640 2023-12-01T01:37:13.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.318+0000 7f1de0cdb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ddc15d220 con 0x7f1ddc0d2730 2023-12-01T01:37:13.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.318+0000 7f1de0cdb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0d1690 con 0x7f1ddc15ce40 2023-12-01T01:37:13.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.319+0000 7f1dda59c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15ce40 0x7f1ddc0b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.319+0000 7f1dda59c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15ce40 0x7f1ddc0b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:39976/0 (socket says 172.21.15.115:39976) 2023-12-01T01:37:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.319+0000 7f1dda59c640 1 -- 172.21.15.115:0/1199761570 learned_addr learned my addr 172.21.15.115:0/1199761570 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.319+0000 7f1dd9d9b640 1 --2- 172.21.15.115:0/1199761570 >> v2:172.21.15.142:3300/0 conn(0x7f1ddc0d2730 0x7f1ddc0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.320+0000 7f1dda59c640 1 -- 172.21.15.115:0/1199761570 >> v2:172.21.15.142:3300/0 conn(0x7f1ddc0d2730 msgr2=0x7f1ddc0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.320+0000 7f1dda59c640 1 --2- 172.21.15.115:0/1199761570 >> v2:172.21.15.142:3300/0 conn(0x7f1ddc0d2730 0x7f1ddc0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.320+0000 7f1dda59c640 1 -- 172.21.15.115:0/1199761570 >> v1:172.21.15.142:6789/0 conn(0x7f1ddc0b4640 legacy=0x7f1ddc069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:13.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.320+0000 7f1dda59c640 1 -- 172.21.15.115:0/1199761570 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc0d1970 con 0x7f1ddc15ce40 2023-12-01T01:37:13.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.321+0000 7f1dda59c640 1 --2- 172.21.15.115:0/1199761570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15ce40 0x7f1ddc0b8710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009e50 tx=0x7f1dcc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d8210a836208122 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.321+0000 7f1dd959a640 1 -- 172.21.15.115:0/1199761570 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc02ca70 con 0x7f1ddc15ce40 2023-12-01T01:37:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.322+0000 7f1dd959a640 1 -- 172.21.15.115:0/1199761570 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dcc02cbd0 con 0x7f1ddc15ce40 2023-12-01T01:37:13.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.322+0000 7f1dd959a640 1 -- 172.21.15.115:0/1199761570 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0343f0 con 0x7f1ddc15ce40 2023-12-01T01:37:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.322+0000 7f1de0cdb640 1 -- 172.21.15.115:0/1199761570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15ce40 msgr2=0x7f1ddc0b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.322+0000 7f1de0cdb640 1 --2- 172.21.15.115:0/1199761570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15ce40 0x7f1ddc0b8710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009e50 tx=0x7f1dcc02bed0 comp rx=0 tx=0).stop 2023-12-01T01:37:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.322+0000 7f1de0cdb640 1 -- 172.21.15.115:0/1199761570 shutdown_connections 2023-12-01T01:37:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.322+0000 7f1de0cdb640 1 --2- 172.21.15.115:0/1199761570 >> v2:172.21.15.142:3300/0 conn(0x7f1ddc0d2730 0x7f1ddc0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:13.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.322+0000 7f1de0cdb640 1 --2- 172.21.15.115:0/1199761570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15ce40 0x7f1ddc0b8710 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.322+0000 7f1de0cdb640 1 -- 172.21.15.115:0/1199761570 >> 172.21.15.115:0/1199761570 conn(0x7f1ddc065f90 msgr2=0x7f1ddc0cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.322+0000 7f1de0cdb640 1 -- 172.21.15.115:0/1199761570 shutdown_connections 2023-12-01T01:37:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.322+0000 7f1de0cdb640 1 -- 172.21.15.115:0/1199761570 wait complete. 2023-12-01T01:37:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.324+0000 7f1de0cdb640 1 Processor -- start 2023-12-01T01:37:13.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.324+0000 7f1de0cdb640 1 -- start start 2023-12-01T01:37:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.325+0000 7f1de0cdb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b4640 0x7f1ddc14c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.325+0000 7f1de0cdb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc0d2730 0x7f1ddc15f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.325+0000 7f1de0cdb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ddc15ce40 0x7f1ddc14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.325+0000 7f1de0cdb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc14c030 con 0x7f1ddc15ce40 2023-12-01T01:37:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.325+0000 7f1de0cdb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc14b420 con 0x7f1ddc0d2730 2023-12-01T01:37:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.325+0000 7f1de0cdb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc14b700 con 0x7f1ddc0b4640 2023-12-01T01:37:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.325+0000 7f1ddad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ddc15ce40 0x7f1ddc14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1ddad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ddc15ce40 0x7f1ddc14d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36930/0 (socket says 172.21.15.115:36930) 2023-12-01T01:37:13.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1ddad9d640 1 -- 172.21.15.115:0/3162848756 learned_addr learned my addr 172.21.15.115:0/3162848756 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:13.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1dda59c640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b4640 0x7f1ddc14c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:13.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1dd9d9b640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc0d2730 0x7f1ddc15f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1ddad9d640 1 -- 172.21.15.115:0/3162848756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b4640 msgr2=0x7f1ddc14c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1ddad9d640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b4640 0x7f1ddc14c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1ddad9d640 1 -- 172.21.15.115:0/3162848756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc0d2730 msgr2=0x7f1ddc15f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1ddad9d640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc0d2730 0x7f1ddc15f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1ddad9d640 1 -- 172.21.15.115:0/3162848756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc009b30 con 0x7f1ddc15ce40 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1dd9d9b640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc0d2730 0x7f1ddc15f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1ddad9d640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ddc15ce40 0x7f1ddc14d660 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400c390 tx=0x7f1dd400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.326+0000 7f1dd2ffd640 1 -- 172.21.15.115:0/3162848756 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd40052d0 con 0x7f1ddc15ce40 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.327+0000 7f1dd2ffd640 1 -- 172.21.15.115:0/3162848756 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd4005430 con 0x7f1ddc15ce40 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.327+0000 7f1dd2ffd640 1 -- 172.21.15.115:0/3162848756 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4003810 con 0x7f1ddc15ce40 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.327+0000 7f1de0cdb640 1 -- 172.21.15.115:0/3162848756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ddc14b9e0 con 0x7f1ddc15ce40 2023-12-01T01:37:13.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.327+0000 7f1de0cdb640 1 -- 172.21.15.115:0/3162848756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ddc14b0c0 con 0x7f1ddc15ce40 2023-12-01T01:37:13.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.328+0000 7f1de0cdb640 1 -- 172.21.15.115:0/3162848756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ddc073950 con 0x7f1ddc15ce40 2023-12-01T01:37:13.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.329+0000 7f1dd2ffd640 1 -- 172.21.15.115:0/3162848756 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1dd4003970 con 0x7f1ddc15ce40 2023-12-01T01:37:13.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.329+0000 7f1dd2ffd640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1dbc06a430 0x7f1dbc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:13.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.330+0000 7f1dd2ffd640 1 -- 172.21.15.115:0/3162848756 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f1dd4087e90 con 0x7f1ddc15ce40 2023-12-01T01:37:13.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.330+0000 7f1dda59c640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1dbc06a430 0x7f1dbc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:13.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.330+0000 7f1dda59c640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1dbc06a430 0x7f1dbc06c8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc035000 tx=0x7f1dcc030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:13.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.334+0000 7f1dd2ffd640 1 -- 172.21.15.115:0/3162848756 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1dd4053420 con 0x7f1ddc15ce40 2023-12-01T01:37:13.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:13.523+0000 7f1de0cdb640 1 -- 172.21.15.115:0/3162848756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ddc074210 con 0x7f1ddc15ce40 2023-12-01T01:37:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.391+0000 7f1dd2ffd640 1 -- 172.21.15.115:0/3162848756 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1dd4056a40 con 0x7f1ddc15ce40 2023-12-01T01:37:14.393 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 -- 172.21.15.115:0/3162848756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1dbc06a430 msgr2=0x7f1dbc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1dbc06a430 0x7f1dbc06c8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc035000 tx=0x7f1dcc030000 comp rx=0 tx=0).stop 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 -- 172.21.15.115:0/3162848756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ddc15ce40 msgr2=0x7f1ddc14d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ddc15ce40 0x7f1ddc14d660 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400c390 tx=0x7f1dd400c850 comp rx=0 tx=0).stop 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 -- 172.21.15.115:0/3162848756 shutdown_connections 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1dbc06a430 0x7f1dbc06c8f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ddc15ce40 0x7f1ddc14d660 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc0d2730 0x7f1ddc15f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 --2- 172.21.15.115:0/3162848756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b4640 0x7f1ddc14c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 -- 172.21.15.115:0/3162848756 >> 172.21.15.115:0/3162848756 conn(0x7f1ddc065f90 msgr2=0x7f1ddc0b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 -- 172.21.15.115:0/3162848756 shutdown_connections 2023-12-01T01:37:14.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.395+0000 7f1de0cdb640 1 -- 172.21.15.115:0/3162848756 wait complete. 2023-12-01T01:37:14.417 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T01:37:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.641+0000 7f343f051640 1 Processor -- start 2023-12-01T01:37:14.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.641+0000 7f343f051640 1 -- start start 2023-12-01T01:37:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.643+0000 7f343f051640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f34380ada10 0x7f343806b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:14.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.643+0000 7f343f051640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3438164ff0 con 0x7f3438165480 2023-12-01T01:37:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.643+0000 7f343f051640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34380c9c30 con 0x7f34380c8df0 2023-12-01T01:37:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.643+0000 7f343f051640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34380c1bb0 con 0x7f34380ada10 2023-12-01T01:37:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.644+0000 7f343cded640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f34380ada10 0x7f343806b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.644+0000 7f343d5ee640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3438165480 0x7f34380cd950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46686/0 (socket says 172.21.15.115:46686) 2023-12-01T01:37:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.644+0000 7f343d5ee640 1 -- 172.21.15.115:0/3789739428 learned_addr learned my addr 172.21.15.115:0/3789739428 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:14.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.644+0000 7f343cded640 1 -- 172.21.15.115:0/3789739428 >> v1:172.21.15.142:6789/0 conn(0x7f34380c8df0 legacy=0x7f34380cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:14.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.644+0000 7f343cded640 1 -- 172.21.15.115:0/3789739428 >> v1:172.21.15.115:6789/0 conn(0x7f3438165480 legacy=0x7f34380cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:14.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.644+0000 7f343cded640 1 -- 172.21.15.115:0/3789739428 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34380c9390 con 0x7f34380ada10 2023-12-01T01:37:14.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.646+0000 7f343cded640 1 --2- 172.21.15.115:0/3789739428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 0x7f343806b290 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f34340097a0 tx=0x7f343402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da13229cd441c890 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:14.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.646+0000 7f342f7fe640 1 -- 172.21.15.115:0/3789739428 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434031830 con 0x7f34380ada10 2023-12-01T01:37:14.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.646+0000 7f342f7fe640 1 -- 172.21.15.115:0/3789739428 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434031990 con 0x7f34380ada10 2023-12-01T01:37:14.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.646+0000 7f342f7fe640 1 -- 172.21.15.115:0/3789739428 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434031d20 con 0x7f34380ada10 2023-12-01T01:37:14.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.647+0000 7f343f051640 1 -- 172.21.15.115:0/3789739428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 msgr2=0x7f343806b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:14.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.647+0000 7f343f051640 1 --2- 172.21.15.115:0/3789739428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 0x7f343806b290 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f34340097a0 tx=0x7f343402b8d0 comp rx=0 tx=0).stop 2023-12-01T01:37:14.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.647+0000 7f343f051640 1 -- 172.21.15.115:0/3789739428 shutdown_connections 2023-12-01T01:37:14.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.647+0000 7f343f051640 1 --2- 172.21.15.115:0/3789739428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 0x7f343806b290 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:14.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.647+0000 7f343f051640 1 -- 172.21.15.115:0/3789739428 >> 172.21.15.115:0/3789739428 conn(0x7f34380662c0 msgr2=0x7f34380b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:14.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.647+0000 7f343f051640 1 -- 172.21.15.115:0/3789739428 shutdown_connections 2023-12-01T01:37:14.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.647+0000 7f343f051640 1 -- 172.21.15.115:0/3789739428 wait complete. 2023-12-01T01:37:14.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.649+0000 7f343f051640 1 Processor -- start 2023-12-01T01:37:14.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.649+0000 7f343f051640 1 -- start start 2023-12-01T01:37:14.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.650+0000 7f343f051640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 0x7f34380c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:14.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.650+0000 7f343f051640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34380c8df0 0x7f343815e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:14.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.650+0000 7f343f051640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3438165480 0x7f34380af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.650+0000 7f343f051640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34381fb000 con 0x7f34380ada10 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.650+0000 7f343f051640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34381fa3f0 con 0x7f34380c8df0 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.650+0000 7f343f051640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34381fa6d0 con 0x7f3438165480 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f343cded640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 0x7f34380c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f343cded640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 0x7f34380c2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36948/0 (socket says 172.21.15.115:36948) 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f343cded640 1 -- 172.21.15.115:0/2566417120 learned_addr learned my addr 172.21.15.115:0/2566417120 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f342ffff640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34380c8df0 0x7f343815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f343cded640 1 -- 172.21.15.115:0/2566417120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3438165480 msgr2=0x7f34380af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f343cded640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3438165480 0x7f34380af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f343cded640 1 -- 172.21.15.115:0/2566417120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34380c8df0 msgr2=0x7f343815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f343cded640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34380c8df0 0x7f343815e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f343cded640 1 -- 172.21.15.115:0/2566417120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434009400 con 0x7f34380ada10 2023-12-01T01:37:14.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f342ffff640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34380c8df0 0x7f343815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:14.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.651+0000 7f343cded640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 0x7f34380c2c40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f34340098c0 tx=0x7f34340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.652+0000 7f342d7fa640 1 -- 172.21.15.115:0/2566417120 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343403ba70 con 0x7f34380ada10 2023-12-01T01:37:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.652+0000 7f342d7fa640 1 -- 172.21.15.115:0/2566417120 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343403bbf0 con 0x7f34380ada10 2023-12-01T01:37:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.652+0000 7f342d7fa640 1 -- 172.21.15.115:0/2566417120 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340453f0 con 0x7f34380ada10 2023-12-01T01:37:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.652+0000 7f343f051640 1 -- 172.21.15.115:0/2566417120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34381fa9b0 con 0x7f34380ada10 2023-12-01T01:37:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.652+0000 7f343f051640 1 -- 172.21.15.115:0/2566417120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34380c2470 con 0x7f34380ada10 2023-12-01T01:37:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.653+0000 7f343f051640 1 -- 172.21.15.115:0/2566417120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3438074210 con 0x7f34380ada10 2023-12-01T01:37:14.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.654+0000 7f342d7fa640 1 -- 172.21.15.115:0/2566417120 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f343404e450 con 0x7f34380ada10 2023-12-01T01:37:14.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.655+0000 7f342d7fa640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f342406a430 0x7f342406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:14.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.655+0000 7f342d7fa640 1 -- 172.21.15.115:0/2566417120 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f34340aaa20 con 0x7f34380ada10 2023-12-01T01:37:14.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.655+0000 7f342ffff640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f342406a430 0x7f342406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:14.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.656+0000 7f342ffff640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f342406a430 0x7f342406c8f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3428007fd0 tx=0x7f3428009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:14.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.659+0000 7f342d7fa640 1 -- 172.21.15.115:0/2566417120 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3434076150 con 0x7f34380ada10 2023-12-01T01:37:14.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:14.854+0000 7f343f051640 1 -- 172.21.15.115:0/2566417120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34380c2090 con 0x7f34380ada10 2023-12-01T01:37:15.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.407+0000 7f342d7fa640 1 -- 172.21.15.115:0/2566417120 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3434079770 con 0x7f34380ada10 2023-12-01T01:37:15.409 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T01:37:15.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.410+0000 7f343f051640 1 -- 172.21.15.115:0/2566417120 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f342406a430 msgr2=0x7f342406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:15.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.410+0000 7f343f051640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f342406a430 0x7f342406c8f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3428007fd0 tx=0x7f3428009040 comp rx=0 tx=0).stop 2023-12-01T01:37:15.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.410+0000 7f343f051640 1 -- 172.21.15.115:0/2566417120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 msgr2=0x7f34380c2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:15.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.410+0000 7f343f051640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 0x7f34380c2c40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f34340098c0 tx=0x7f34340046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.410+0000 7f343f051640 1 -- 172.21.15.115:0/2566417120 shutdown_connections 2023-12-01T01:37:15.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.410+0000 7f343f051640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f342406a430 0x7f342406c8f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.410+0000 7f343f051640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3438165480 0x7f34380af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.410+0000 7f343f051640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34380c8df0 0x7f343815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.410+0000 7f343f051640 1 --2- 172.21.15.115:0/2566417120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34380ada10 0x7f34380c2c40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.410+0000 7f343f051640 1 -- 172.21.15.115:0/2566417120 >> 172.21.15.115:0/2566417120 conn(0x7f34380662c0 msgr2=0x7f343815f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:15.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.411+0000 7f343f051640 1 -- 172.21.15.115:0/2566417120 shutdown_connections 2023-12-01T01:37:15.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.411+0000 7f343f051640 1 -- 172.21.15.115:0/2566417120 wait complete. 2023-12-01T01:37:15.435 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T01:37:15.435 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T01:37:15.435 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T01:37:15.435 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:37:15.435 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T01:37:15.435 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T01:37:15.436 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T01:37:15.436 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:37:15.436 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T01:37:15.436 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T01:37:15.436 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T01:37:15.436 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:37:15.436 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T01:37:15.436 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T01:37:15.437 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T01:37:15.437 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:37:15.453 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T01:37:15.453 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T01:37:15.454 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T01:37:15.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T01:37:15.455 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T01:37:15.456 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T01:37:15.457 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T01:37:15.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T01:37:15.502 INFO:tasks.ceph.mds.a:Started 2023-12-01T01:37:15.502 DEBUG:teuthology.parallel:result is None 2023-12-01T01:37:15.505 INFO:tasks.ceph.mds.d:Started 2023-12-01T01:37:15.505 DEBUG:teuthology.parallel:result is None 2023-12-01T01:37:15.509 INFO:tasks.ceph.mds.c:Started 2023-12-01T01:37:15.509 DEBUG:teuthology.parallel:result is None 2023-12-01T01:37:15.511 INFO:tasks.ceph.mds.b:Started 2023-12-01T01:37:15.511 DEBUG:teuthology.parallel:result is None 2023-12-01T01:37:15.512 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T01:37:15.659 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-12-01T01:37:15.659 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-12-01T01:37:15.659 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-12-01T01:37:15.664 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T01:37:15.665 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T01:37:15.686 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T01:37:15.694 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:37:15.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.728+0000 7fc383842640 1 Processor -- start 2023-12-01T01:37:15.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.728+0000 7fc383842640 1 -- start start 2023-12-01T01:37:15.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.729+0000 7fc383842640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c06f2f0 0x7fc37c06bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:15.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.729+0000 7fc383842640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37c0add10 con 0x7fc37c0cad70 2023-12-01T01:37:15.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.729+0000 7fc383842640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37c0c5760 con 0x7fc37c15e2f0 2023-12-01T01:37:15.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.729+0000 7fc383842640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c15e6d0 con 0x7fc37c06f2f0 2023-12-01T01:37:15.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.729+0000 7fc380ddd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc37c0cad70 0x7fc37c06b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46698/0 (socket says 172.21.15.115:46698) 2023-12-01T01:37:15.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.729+0000 7fc380ddd640 1 -- 172.21.15.115:0/1759047009 learned_addr learned my addr 172.21.15.115:0/1759047009 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:15.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc381ddf640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c06f2f0 0x7fc37c06bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:15.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306941676 0 0) 0x7fc37c0add10 con 0x7fc37c0cad70 2023-12-01T01:37:15.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc368003520 con 0x7fc37c0cad70 2023-12-01T01:37:15.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525485774 0 0) 0x7fc37c0c5760 con 0x7fc37c15e2f0 2023-12-01T01:37:15.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc37c0add10 con 0x7fc37c15e2f0 2023-12-01T01:37:15.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1772640049 0 0) 0x7fc368003520 con 0x7fc37c0cad70 2023-12-01T01:37:15.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c06f2f0 msgr2=0x7fc37c06bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:15.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c06f2f0 0x7fc37c06bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 >> v1:172.21.15.142:6789/0 conn(0x7fc37c15e2f0 legacy=0x7fc37c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c0cf7b0 con 0x7fc37c0cad70 2023-12-01T01:37:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc36c001670 con 0x7fc37c0cad70 2023-12-01T01:37:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 >> v1:172.21.15.115:6789/0 conn(0x7fc37c0cad70 legacy=0x7fc37c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368003ba0 0x7fc368003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3680047c0 0x7fc368004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:15.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c06f2f0 0x7fc36800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c15e6d0 con 0x7fc368003ba0 2023-12-01T01:37:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3680054f0 con 0x7fc3680047c0 2023-12-01T01:37:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.730+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36800bec0 con 0x7fc37c06f2f0 2023-12-01T01:37:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.731+0000 7fc3815de640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368003ba0 0x7fc368003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.731+0000 7fc380ddd640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3680047c0 0x7fc368004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:15.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.731+0000 7fc381ddf640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c06f2f0 0x7fc36800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:15.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.731+0000 7fc3815de640 1 -- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c06f2f0 msgr2=0x7fc36800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:15.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.731+0000 7fc3815de640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c06f2f0 0x7fc36800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.731+0000 7fc3815de640 1 -- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3680047c0 msgr2=0x7fc368004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:15.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.731+0000 7fc3815de640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3680047c0 0x7fc368004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.731+0000 7fc3815de640 1 -- 172.21.15.115:0/1759047009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368003520 con 0x7fc368003ba0 2023-12-01T01:37:15.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.731+0000 7fc381ddf640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c06f2f0 0x7fc36800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.732+0000 7fc3815de640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368003ba0 0x7fc368003fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc3780018d0 tx=0x7fc3680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a338e4154d9eb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.732+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3780052d0 con 0x7fc368003ba0 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.732+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378005430 con 0x7fc368003ba0 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.732+0000 7fc373fff640 1 -- 172.21.15.115:0/1759047009 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378003810 con 0x7fc368003ba0 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.733+0000 7fc383842640 1 -- 172.21.15.115:0/1759047009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368003ba0 msgr2=0x7fc368003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.733+0000 7fc383842640 1 --2- 172.21.15.115:0/1759047009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc368003ba0 0x7fc368003fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc3780018d0 tx=0x7fc3680057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.733+0000 7fc381ddf640 1 -- 172.21.15.115:0/1759047009 reap_dead start 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.733+0000 7fc383842640 1 -- 172.21.15.115:0/1759047009 shutdown_connections 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.733+0000 7fc383842640 1 -- 172.21.15.115:0/1759047009 >> 172.21.15.115:0/1759047009 conn(0x7fc37c0ce880 msgr2=0x7fc37c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.733+0000 7fc383842640 1 -- 172.21.15.115:0/1759047009 shutdown_connections 2023-12-01T01:37:15.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.733+0000 7fc383842640 1 -- 172.21.15.115:0/1759047009 wait complete. 2023-12-01T01:37:15.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.735+0000 7fc383842640 1 Processor -- start 2023-12-01T01:37:15.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.735+0000 7fc383842640 1 -- start start 2023-12-01T01:37:15.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc383842640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37c14a340 0x7fc37c0c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:15.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc383842640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c14a720 0x7fc37c0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc383842640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c14d980 0x7fc37c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc383842640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c14cca0 con 0x7fc37c14d980 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc383842640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0c88e0 con 0x7fc37c14a340 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc383842640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c14d4f0 con 0x7fc37c14a720 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc381ddf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c14d980 0x7fc37c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc381ddf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c14d980 0x7fc37c0ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37004/0 (socket says 172.21.15.115:37004) 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc381ddf640 1 -- 172.21.15.115:0/887297304 learned_addr learned my addr 172.21.15.115:0/887297304 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc3815de640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37c14a340 0x7fc37c0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.736+0000 7fc380ddd640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c14a720 0x7fc37c0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.737+0000 7fc381ddf640 1 -- 172.21.15.115:0/887297304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c14a720 msgr2=0x7fc37c0c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.737+0000 7fc381ddf640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c14a720 0x7fc37c0c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.737+0000 7fc381ddf640 1 -- 172.21.15.115:0/887297304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37c14a340 msgr2=0x7fc37c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.737+0000 7fc381ddf640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37c14a340 0x7fc37c0c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.737+0000 7fc381ddf640 1 -- 172.21.15.115:0/887297304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c0c8600 con 0x7fc37c14d980 2023-12-01T01:37:15.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.737+0000 7fc381ddf640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c14d980 0x7fc37c0ac8a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc37c06f700 tx=0x7fc374009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.737+0000 7fc371ffb640 1 -- 172.21.15.115:0/887297304 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37401b070 con 0x7fc37c14d980 2023-12-01T01:37:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.737+0000 7fc371ffb640 1 -- 172.21.15.115:0/887297304 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37400f040 con 0x7fc37c14d980 2023-12-01T01:37:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.737+0000 7fc371ffb640 1 -- 172.21.15.115:0/887297304 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37400e050 con 0x7fc37c14d980 2023-12-01T01:37:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.738+0000 7fc383842640 1 -- 172.21.15.115:0/887297304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c14cea0 con 0x7fc37c14d980 2023-12-01T01:37:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.738+0000 7fc383842640 1 -- 172.21.15.115:0/887297304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37c15d3c0 con 0x7fc37c14d980 2023-12-01T01:37:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.739+0000 7fc383842640 1 -- 172.21.15.115:0/887297304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37c06fda0 con 0x7fc37c14d980 2023-12-01T01:37:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.741+0000 7fc371ffb640 1 -- 172.21.15.115:0/887297304 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc374003990 con 0x7fc37c14d980 2023-12-01T01:37:15.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.741+0000 7fc371ffb640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc36006a460 0x7fc36006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.741+0000 7fc371ffb640 1 -- 172.21.15.115:0/887297304 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc374089130 con 0x7fc37c14d980 2023-12-01T01:37:15.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.742+0000 7fc3815de640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc36006a460 0x7fc36006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:15.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.743+0000 7fc3815de640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc36006a460 0x7fc36006c920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc378034000 tx=0x7fc378031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.745+0000 7fc371ffb640 1 -- 172.21.15.115:0/887297304 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc374054ab0 con 0x7fc37c14d980 2023-12-01T01:37:15.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:15.946+0000 7fc383842640 1 -- 172.21.15.115:0/887297304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc37c071b00 con 0x7fc37c14d980 2023-12-01T01:37:16.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.724+0000 7fc371ffb640 1 -- 172.21.15.115:0/887297304 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc3740580d0 con 0x7fc37c14d980 2023-12-01T01:37:16.726 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T01:37:16.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.728+0000 7fc383842640 1 -- 172.21.15.115:0/887297304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc36006a460 msgr2=0x7fc36006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:16.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.728+0000 7fc383842640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc36006a460 0x7fc36006c920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc378034000 tx=0x7fc378031000 comp rx=0 tx=0).stop 2023-12-01T01:37:16.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.728+0000 7fc383842640 1 -- 172.21.15.115:0/887297304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c14d980 msgr2=0x7fc37c0ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:16.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.728+0000 7fc383842640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c14d980 0x7fc37c0ac8a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc37c06f700 tx=0x7fc374009f90 comp rx=0 tx=0).stop 2023-12-01T01:37:16.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.728+0000 7fc383842640 1 -- 172.21.15.115:0/887297304 shutdown_connections 2023-12-01T01:37:16.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.728+0000 7fc383842640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc36006a460 0x7fc36006c920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:16.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.728+0000 7fc383842640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37c14a720 0x7fc37c0c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:16.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.728+0000 7fc383842640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc37c14d980 0x7fc37c0ac8a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:16.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.728+0000 7fc383842640 1 --2- 172.21.15.115:0/887297304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37c14a340 0x7fc37c0c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:16.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.728+0000 7fc383842640 1 -- 172.21.15.115:0/887297304 >> 172.21.15.115:0/887297304 conn(0x7fc37c0ce880 msgr2=0x7fc37c070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:16.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.729+0000 7fc383842640 1 -- 172.21.15.115:0/887297304 shutdown_connections 2023-12-01T01:37:16.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.729+0000 7fc383842640 1 -- 172.21.15.115:0/887297304 wait complete. 2023-12-01T01:37:16.751 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T01:37:16.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.957+0000 7fa2d20e5640 1 Processor -- start 2023-12-01T01:37:16.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.958+0000 7fa2d20e5640 1 -- start start 2023-12-01T01:37:16.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.959+0000 7fa2d20e5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa2cc0cad70 0x7fa2cc06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:16.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.959+0000 7fa2d20e5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc06f2f0 0x7fa2cc06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:16.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.959+0000 7fa2d20e5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc0add10 con 0x7fa2cc15e2f0 2023-12-01T01:37:16.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.959+0000 7fa2d20e5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2cc0c5760 con 0x7fa2cc0cad70 2023-12-01T01:37:16.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.959+0000 7fa2d20e5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc15e6d0 con 0x7fa2cc06f2f0 2023-12-01T01:37:16.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.959+0000 7fa2c3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa2cc0cad70 0x7fa2cc06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:16.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.959+0000 7fa2c3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa2cc0cad70 0x7fa2cc06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37008/0 (socket says 172.21.15.115:37008) 2023-12-01T01:37:16.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.959+0000 7fa2c3fff640 1 -- 172.21.15.115:0/2678690720 learned_addr learned my addr 172.21.15.115:0/2678690720 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:16.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.960+0000 7fa2cbfff640 1 --2- 172.21.15.115:0/2678690720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc06f2f0 0x7fa2cc06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:16.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.960+0000 7fa2c3fff640 1 -- 172.21.15.115:0/2678690720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc06f2f0 msgr2=0x7fa2cc06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:16.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.960+0000 7fa2c3fff640 1 --2- 172.21.15.115:0/2678690720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc06f2f0 0x7fa2cc06bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:16.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.960+0000 7fa2c3fff640 1 -- 172.21.15.115:0/2678690720 >> v1:172.21.15.115:6789/0 conn(0x7fa2cc15e2f0 legacy=0x7fa2cc0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:16.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.960+0000 7fa2c3fff640 1 -- 172.21.15.115:0/2678690720 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0cf7b0 con 0x7fa2cc0cad70 2023-12-01T01:37:16.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.961+0000 7fa2c3fff640 1 --2- 172.21.15.115:0/2678690720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 0x7fa2cc06b5c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009870 tx=0x7fa2b402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e362506d09e78a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.961+0000 7fa2caffd640 1 -- 172.21.15.115:0/2678690720 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40317c0 con 0x7fa2cc0cad70 2023-12-01T01:37:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.961+0000 7fa2caffd640 1 -- 172.21.15.115:0/2678690720 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b4031920 con 0x7fa2cc0cad70 2023-12-01T01:37:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.961+0000 7fa2caffd640 1 -- 172.21.15.115:0/2678690720 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4031cb0 con 0x7fa2cc0cad70 2023-12-01T01:37:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.962+0000 7fa2d20e5640 1 -- 172.21.15.115:0/2678690720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 msgr2=0x7fa2cc06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.962+0000 7fa2d20e5640 1 --2- 172.21.15.115:0/2678690720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 0x7fa2cc06b5c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009870 tx=0x7fa2b402b860 comp rx=0 tx=0).stop 2023-12-01T01:37:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.962+0000 7fa2d20e5640 1 -- 172.21.15.115:0/2678690720 shutdown_connections 2023-12-01T01:37:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.962+0000 7fa2d20e5640 1 --2- 172.21.15.115:0/2678690720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc06f2f0 0x7fa2cc06bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.962+0000 7fa2d20e5640 1 --2- 172.21.15.115:0/2678690720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 0x7fa2cc06b5c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.962+0000 7fa2d20e5640 1 -- 172.21.15.115:0/2678690720 >> 172.21.15.115:0/2678690720 conn(0x7fa2cc0ce880 msgr2=0x7fa2cc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:16.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.962+0000 7fa2d20e5640 1 -- 172.21.15.115:0/2678690720 shutdown_connections 2023-12-01T01:37:16.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.962+0000 7fa2d20e5640 1 -- 172.21.15.115:0/2678690720 wait complete. 2023-12-01T01:37:16.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.964+0000 7fa2d20e5640 1 Processor -- start 2023-12-01T01:37:16.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.964+0000 7fa2d20e5640 1 -- start start 2023-12-01T01:37:16.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2d20e5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc06f2f0 0x7fa2cc0c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:16.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2d20e5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 0x7fa2cc0c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2d20e5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc15e2f0 0x7fa2cc0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2d20e5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc0c3f50 con 0x7fa2cc0cad70 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2d20e5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc0c8870 con 0x7fa2cc06f2f0 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2d20e5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc0c2970 con 0x7fa2cc15e2f0 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2c3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 0x7fa2cc0c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2c3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 0x7fa2cc0c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37022/0 (socket says 172.21.15.115:37022) 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2c3fff640 1 -- 172.21.15.115:0/878877777 learned_addr learned my addr 172.21.15.115:0/878877777 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2c3fff640 1 -- 172.21.15.115:0/878877777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc15e2f0 msgr2=0x7fa2cc0c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2c3fff640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc15e2f0 0x7fa2cc0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2c3fff640 1 -- 172.21.15.115:0/878877777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc06f2f0 msgr2=0x7fa2cc0c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:16.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2c3fff640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc06f2f0 0x7fa2cc0c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:16.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.965+0000 7fa2c3fff640 1 -- 172.21.15.115:0/878877777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b40094d0 con 0x7fa2cc0cad70 2023-12-01T01:37:16.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.966+0000 7fa2c3fff640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 0x7fa2cc0c8330 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa2b402b8a0 tx=0x7fa2b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:16.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.966+0000 7fa2c8ff9640 1 -- 172.21.15.115:0/878877777 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40329a0 con 0x7fa2cc0cad70 2023-12-01T01:37:16.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.966+0000 7fa2c8ff9640 1 -- 172.21.15.115:0/878877777 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b4032b40 con 0x7fa2cc0cad70 2023-12-01T01:37:16.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.966+0000 7fa2c8ff9640 1 -- 172.21.15.115:0/878877777 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40453f0 con 0x7fa2cc0cad70 2023-12-01T01:37:16.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.966+0000 7fa2d20e5640 1 -- 172.21.15.115:0/878877777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc0c2c50 con 0x7fa2cc0cad70 2023-12-01T01:37:16.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.966+0000 7fa2d20e5640 1 -- 172.21.15.115:0/878877777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc0c3210 con 0x7fa2cc0cad70 2023-12-01T01:37:16.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.967+0000 7fa2d20e5640 1 -- 172.21.15.115:0/878877777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc06fda0 con 0x7fa2cc0cad70 2023-12-01T01:37:16.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.968+0000 7fa2c8ff9640 1 -- 172.21.15.115:0/878877777 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa2b404e450 con 0x7fa2cc0cad70 2023-12-01T01:37:16.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.969+0000 7fa2c8ff9640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2ac06a430 0x7fa2ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:16.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.969+0000 7fa2c8ff9640 1 -- 172.21.15.115:0/878877777 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa2b40aa9d0 con 0x7fa2cc0cad70 2023-12-01T01:37:16.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.969+0000 7fa2cb7fe640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2ac06a430 0x7fa2ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:16.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.970+0000 7fa2cb7fe640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2ac06a430 0x7fa2ac06c8f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000f60 tx=0x7fa2bc0079b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:16.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:16.973+0000 7fa2c8ff9640 1 -- 172.21.15.115:0/878877777 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa2b4076270 con 0x7fa2cc0cad70 2023-12-01T01:37:17.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.214+0000 7fa2d20e5640 1 -- 172.21.15.115:0/878877777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa2cc04e890 con 0x7fa2cc0cad70 2023-12-01T01:37:17.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.216+0000 7fa2c8ff9640 1 -- 172.21.15.115:0/878877777 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fa2b4079890 con 0x7fa2cc0cad70 2023-12-01T01:37:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:17.218 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:37:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T01:37:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T01:37:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T01:37:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:37:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:37:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:37:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T01:37:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T01:37:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:37:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:37:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:37:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T01:37:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T01:37:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:37:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:37:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:37:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T01:37:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T01:37:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:37:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:37:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:37:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T01:37:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T01:37:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:37:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:37:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:37:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:37:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T01:37:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T01:37:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:37:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:37:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:37:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:37:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T01:37:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T01:37:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:37:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:37:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:37:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:37:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T01:37:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T01:37:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:37:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:37:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:37:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:37:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T01:37:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T01:37:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:37:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:37:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:37:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:37:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T01:37:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T01:37:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:37:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:37:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:37:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.bootstrap-mds", 2023-12-01T01:37:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBxOGllXIQJCBAA7JdzjNN2lKwFpg0Jk9TGsg==", 2023-12-01T01:37:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile bootstrap-mds" 2023-12-01T01:37:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.bootstrap-mgr", 2023-12-01T01:37:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBxOGllwqAJCBAAawKSk7xK2QSy5noY9Ue+Fw==", 2023-12-01T01:37:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile bootstrap-mgr" 2023-12-01T01:37:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.bootstrap-osd", 2023-12-01T01:37:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBxOGllrrkJCBAAUlYyZmiU1R7NgaReFfr5sA==", 2023-12-01T01:37:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile bootstrap-osd" 2023-12-01T01:37:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.bootstrap-rbd", 2023-12-01T01:37:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBxOGllV9MJCBAAlUeggkCXcAcgt1o+fwUJ9w==", 2023-12-01T01:37:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile bootstrap-rbd" 2023-12-01T01:37:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-12-01T01:37:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBxOGllW+0JCBAAabFfk88kqB79kbjNeT6k4Q==", 2023-12-01T01:37:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-12-01T01:37:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.bootstrap-rgw", 2023-12-01T01:37:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBxOGlllAcKCBAAVLNI+oePIMeYV0phb/3+Tg==", 2023-12-01T01:37:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile bootstrap-rgw" 2023-12-01T01:37:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T01:37:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T01:37:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:37:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:37:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:37:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T01:37:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T01:37:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:37:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:37:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:37:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:37:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T01:37:17.250 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:37:17.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.218+0000 7fa2d20e5640 1 -- 172.21.15.115:0/878877777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2ac06a430 msgr2=0x7fa2ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.218+0000 7fa2d20e5640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2ac06a430 0x7fa2ac06c8f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000f60 tx=0x7fa2bc0079b0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.218+0000 7fa2d20e5640 1 -- 172.21.15.115:0/878877777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 msgr2=0x7fa2cc0c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.218+0000 7fa2d20e5640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 0x7fa2cc0c8330 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa2b402b8a0 tx=0x7fa2b40046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.219+0000 7fa2d20e5640 1 -- 172.21.15.115:0/878877777 shutdown_connections 2023-12-01T01:37:17.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.219+0000 7fa2d20e5640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2ac06a430 0x7fa2ac06c8f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.219+0000 7fa2d20e5640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc15e2f0 0x7fa2cc0c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.219+0000 7fa2d20e5640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2cc0cad70 0x7fa2cc0c8330 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.219+0000 7fa2d20e5640 1 --2- 172.21.15.115:0/878877777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc06f2f0 0x7fa2cc0c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.219+0000 7fa2d20e5640 1 -- 172.21.15.115:0/878877777 >> 172.21.15.115:0/878877777 conn(0x7fa2cc0ce880 msgr2=0x7fa2cc0cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:17.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.219+0000 7fa2d20e5640 1 -- 172.21.15.115:0/878877777 shutdown_connections 2023-12-01T01:37:17.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.219+0000 7fa2d20e5640 1 -- 172.21.15.115:0/878877777 wait complete. 2023-12-01T01:37:17.255 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-12-01T01:37:17.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.466+0000 7fef3b573640 1 Processor -- start 2023-12-01T01:37:17.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.466+0000 7fef3b573640 1 -- start start 2023-12-01T01:37:17.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.468+0000 7fef3b573640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fef34070820 0x7fef340af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.468+0000 7fef3b573640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef340cea50 0x7fef340ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.468+0000 7fef3b573640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef34070e90 con 0x7fef3415acf0 2023-12-01T01:37:17.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.468+0000 7fef3b573640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef340b01a0 con 0x7fef34070820 2023-12-01T01:37:17.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.468+0000 7fef3b573640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3415f670 con 0x7fef340cea50 2023-12-01T01:37:17.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.468+0000 7fef3930f640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fef3415acf0 0x7fef3415ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46714/0 (socket says 172.21.15.115:46714) 2023-12-01T01:37:17.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.468+0000 7fef3930f640 1 -- 172.21.15.115:0/228644106 learned_addr learned my addr 172.21.15.115:0/228644106 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:17.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.468+0000 7fef39b10640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef340cea50 0x7fef340ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.468+0000 7fef38b0e640 1 --2- 172.21.15.115:0/228644106 >> v2:172.21.15.142:3300/0 conn(0x7fef34070820 0x7fef340af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.468+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122558459 0 0) 0x7fef34070e90 con 0x7fef3415acf0 2023-12-01T01:37:17.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef20003520 con 0x7fef3415acf0 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 81635585 0 0) 0x7fef20003520 con 0x7fef3415acf0 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef340cea50 msgr2=0x7fef340ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef340cea50 0x7fef340ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 >> v2:172.21.15.142:3300/0 conn(0x7fef34070820 msgr2=0x7fef340af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 --2- 172.21.15.115:0/228644106 >> v2:172.21.15.142:3300/0 conn(0x7fef34070820 0x7fef340af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef34071170 con 0x7fef3415acf0 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fef30001670 con 0x7fef3415acf0 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef39b10640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef340cea50 0x7fef340ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 >> v1:172.21.15.115:6789/0 conn(0x7fef3415acf0 legacy=0x7fef3415ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef340cea50 0x7fef20003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef20003fb0 0x7fef200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 0x7fef20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340b01a0 con 0x7fef20004d00 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3415f670 con 0x7fef20003fb0 2023-12-01T01:37:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2000bab0 con 0x7fef340cea50 2023-12-01T01:37:17.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.469+0000 7fef39b10640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 0x7fef20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.470+0000 7fef3930f640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef340cea50 0x7fef20003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.470+0000 7fef39b10640 1 -- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef340cea50 msgr2=0x7fef20003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:17.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.470+0000 7fef39b10640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef340cea50 0x7fef20003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.470+0000 7fef38b0e640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef20003fb0 0x7fef200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.470+0000 7fef39b10640 1 -- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef20003fb0 msgr2=0x7fef200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:17.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.470+0000 7fef39b10640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef20003fb0 0x7fef200047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.470+0000 7fef39b10640 1 -- 172.21.15.115:0/228644106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef20003520 con 0x7fef20004d00 2023-12-01T01:37:17.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.471+0000 7fef39b10640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 0x7fef20005180 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0073e0 tx=0x7fef200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda7b577c459fe7b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:17.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.471+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c02cd30 con 0x7fef20004d00 2023-12-01T01:37:17.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.471+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c02ce90 con 0x7fef20004d00 2023-12-01T01:37:17.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.471+0000 7fef2bfff640 1 -- 172.21.15.115:0/228644106 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c034640 con 0x7fef20004d00 2023-12-01T01:37:17.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.471+0000 7fef3b573640 1 -- 172.21.15.115:0/228644106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 msgr2=0x7fef20005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.471+0000 7fef3b573640 1 --2- 172.21.15.115:0/228644106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 0x7fef20005180 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0073e0 tx=0x7fef200057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.472+0000 7fef39b10640 1 -- 172.21.15.115:0/228644106 reap_dead start 2023-12-01T01:37:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.472+0000 7fef3b573640 1 -- 172.21.15.115:0/228644106 shutdown_connections 2023-12-01T01:37:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.472+0000 7fef3b573640 1 -- 172.21.15.115:0/228644106 >> 172.21.15.115:0/228644106 conn(0x7fef34065b40 msgr2=0x7fef34065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.472+0000 7fef3b573640 1 -- 172.21.15.115:0/228644106 shutdown_connections 2023-12-01T01:37:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.472+0000 7fef3b573640 1 -- 172.21.15.115:0/228644106 wait complete. 2023-12-01T01:37:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.473+0000 7fef3b573640 1 Processor -- start 2023-12-01T01:37:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.474+0000 7fef3b573640 1 -- start start 2023-12-01T01:37:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3b573640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 0x7fef3414f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3b573640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef34150fe0 0x7fef34151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3b573640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef340c7490 0x7fef340c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3b573640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef341519a0 con 0x7fef20004d00 2023-12-01T01:37:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3b573640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340cce40 con 0x7fef340c7490 2023-12-01T01:37:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3b573640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef340cc510 con 0x7fef34150fe0 2023-12-01T01:37:17.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3930f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 0x7fef3414f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3930f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 0x7fef3414f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37036/0 (socket says 172.21.15.115:37036) 2023-12-01T01:37:17.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3930f640 1 -- 172.21.15.115:0/1345441482 learned_addr learned my addr 172.21.15.115:0/1345441482 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef38b0e640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef34150fe0 0x7fef34151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef39b10640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef340c7490 0x7fef340c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3930f640 1 -- 172.21.15.115:0/1345441482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef34150fe0 msgr2=0x7fef34151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3930f640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef34150fe0 0x7fef34151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3930f640 1 -- 172.21.15.115:0/1345441482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef340c7490 msgr2=0x7fef340c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3930f640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef340c7490 0x7fef340c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.475+0000 7fef3930f640 1 -- 172.21.15.115:0/1345441482 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef340cc7f0 con 0x7fef20004d00 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.476+0000 7fef3930f640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 0x7fef3414f920 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fef3000d810 tx=0x7fef3000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.476+0000 7fef29ffb640 1 -- 172.21.15.115:0/1345441482 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300074e0 con 0x7fef20004d00 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.476+0000 7fef29ffb640 1 -- 172.21.15.115:0/1345441482 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef30007640 con 0x7fef20004d00 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.476+0000 7fef29ffb640 1 -- 172.21.15.115:0/1345441482 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef300049e0 con 0x7fef20004d00 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.476+0000 7fef3b573640 1 -- 172.21.15.115:0/1345441482 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef340ccad0 con 0x7fef20004d00 2023-12-01T01:37:17.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.476+0000 7fef3b573640 1 -- 172.21.15.115:0/1345441482 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef340cd2d0 con 0x7fef20004d00 2023-12-01T01:37:17.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.477+0000 7fef3b573640 1 -- 172.21.15.115:0/1345441482 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3415cf70 con 0x7fef20004d00 2023-12-01T01:37:17.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.478+0000 7fef29ffb640 1 -- 172.21.15.115:0/1345441482 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fef30004b40 con 0x7fef20004d00 2023-12-01T01:37:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.479+0000 7fef29ffb640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef1806a460 0x7fef1806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.479+0000 7fef29ffb640 1 -- 172.21.15.115:0/1345441482 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fef30087f50 con 0x7fef20004d00 2023-12-01T01:37:17.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.479+0000 7fef38b0e640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef1806a460 0x7fef1806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.480+0000 7fef38b0e640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef1806a460 0x7fef1806c920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef3406b670 tx=0x7fef24008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:17.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.483+0000 7fef29ffb640 1 -- 172.21.15.115:0/1345441482 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fef30053920 con 0x7fef20004d00 2023-12-01T01:37:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.743+0000 7fef3b573640 1 -- 172.21.15.115:0/1345441482 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fef3404e890 con 0x7fef20004d00 2023-12-01T01:37:17.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.750+0000 7fef29ffb640 1 -- 172.21.15.115:0/1345441482 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fef30019020 con 0x7fef20004d00 2023-12-01T01:37:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 -- 172.21.15.115:0/1345441482 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef1806a460 msgr2=0x7fef1806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef1806a460 0x7fef1806c920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef3406b670 tx=0x7fef24008040 comp rx=0 tx=0).stop 2023-12-01T01:37:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 -- 172.21.15.115:0/1345441482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 msgr2=0x7fef3414f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 0x7fef3414f920 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fef3000d810 tx=0x7fef3000dcd0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 -- 172.21.15.115:0/1345441482 shutdown_connections 2023-12-01T01:37:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef1806a460 0x7fef1806c920 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef34150fe0 0x7fef34151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef340c7490 0x7fef340c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 --2- 172.21.15.115:0/1345441482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef20004d00 0x7fef3414f920 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 -- 172.21.15.115:0/1345441482 >> 172.21.15.115:0/1345441482 conn(0x7fef34065b40 msgr2=0x7fef3406be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:17.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 -- 172.21.15.115:0/1345441482 shutdown_connections 2023-12-01T01:37:17.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.753+0000 7fef3b573640 1 -- 172.21.15.115:0/1345441482 wait complete. 2023-12-01T01:37:17.774 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-12-01T01:37:17.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.983+0000 7f7810243640 1 Processor -- start 2023-12-01T01:37:17.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.984+0000 7f7810243640 1 -- start start 2023-12-01T01:37:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.985+0000 7f7810243640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f780815acf0 0x7f7808070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.985+0000 7f7810243640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f78080af790 0x7f780815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.985+0000 7f7810243640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7808070e90 con 0x7f78080cea50 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.985+0000 7f7810243640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78080b01a0 con 0x7f780815acf0 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.985+0000 7f7810243640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7808071170 con 0x7f78080af790 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.985+0000 7f780dfdf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f780815acf0 0x7f7808070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.985+0000 7f780dfdf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f780815acf0 0x7f7808070820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49380/0 (socket says 172.21.15.115:49380) 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.985+0000 7f780dfdf640 1 -- 172.21.15.115:0/1669347167 learned_addr learned my addr 172.21.15.115:0/1669347167 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.986+0000 7f780d7de640 1 --2- 172.21.15.115:0/1669347167 >> v2:172.21.15.142:3300/0 conn(0x7f78080af790 0x7f780815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.986+0000 7f780dfdf640 1 -- 172.21.15.115:0/1669347167 >> v2:172.21.15.142:3300/0 conn(0x7f78080af790 msgr2=0x7f780815ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.986+0000 7f780dfdf640 1 --2- 172.21.15.115:0/1669347167 >> v2:172.21.15.142:3300/0 conn(0x7f78080af790 0x7f780815ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.986+0000 7f780dfdf640 1 -- 172.21.15.115:0/1669347167 >> v1:172.21.15.142:6789/0 conn(0x7f78080cea50 legacy=0x7f78080c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:17.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.986+0000 7f780dfdf640 1 -- 172.21.15.115:0/1669347167 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780815e760 con 0x7f780815acf0 2023-12-01T01:37:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.987+0000 7f780dfdf640 1 --2- 172.21.15.115:0/1669347167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f780815acf0 0x7f7808070820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f78080cc390 tx=0x7f77fc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fe819801e5fd04 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.987+0000 7f780cfdd640 1 -- 172.21.15.115:0/1669347167 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc02c9f0 con 0x7f780815acf0 2023-12-01T01:37:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.987+0000 7f780cfdd640 1 -- 172.21.15.115:0/1669347167 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc02cb50 con 0x7f780815acf0 2023-12-01T01:37:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.987+0000 7f780cfdd640 1 -- 172.21.15.115:0/1669347167 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0343f0 con 0x7f780815acf0 2023-12-01T01:37:17.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.988+0000 7f7810243640 1 -- 172.21.15.115:0/1669347167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f780815acf0 msgr2=0x7f7808070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.988+0000 7f7810243640 1 --2- 172.21.15.115:0/1669347167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f780815acf0 0x7f7808070820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f78080cc390 tx=0x7f77fc02be50 comp rx=0 tx=0).stop 2023-12-01T01:37:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.988+0000 7f7810243640 1 -- 172.21.15.115:0/1669347167 shutdown_connections 2023-12-01T01:37:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.988+0000 7f7810243640 1 --2- 172.21.15.115:0/1669347167 >> v2:172.21.15.142:3300/0 conn(0x7f78080af790 0x7f780815ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.988+0000 7f7810243640 1 --2- 172.21.15.115:0/1669347167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f780815acf0 0x7f7808070820 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.988+0000 7f7810243640 1 -- 172.21.15.115:0/1669347167 >> 172.21.15.115:0/1669347167 conn(0x7f7808065b40 msgr2=0x7f7808065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.988+0000 7f7810243640 1 -- 172.21.15.115:0/1669347167 shutdown_connections 2023-12-01T01:37:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.988+0000 7f7810243640 1 -- 172.21.15.115:0/1669347167 wait complete. 2023-12-01T01:37:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.990+0000 7f7810243640 1 Processor -- start 2023-12-01T01:37:17.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.990+0000 7f7810243640 1 -- start start 2023-12-01T01:37:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.991+0000 7f7810243640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78080af790 0x7f78080af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.991+0000 7f7810243640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78080cea50 0x7f780815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.991+0000 7f7810243640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f780815acf0 0x7f78080b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.991+0000 7f7810243640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780815cf70 con 0x7f78080af790 2023-12-01T01:37:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.991+0000 7f7810243640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780815d250 con 0x7f780815acf0 2023-12-01T01:37:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.991+0000 7f7810243640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780815d530 con 0x7f78080cea50 2023-12-01T01:37:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.991+0000 7f780dfdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78080af790 0x7f78080af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.991+0000 7f780dfdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78080af790 0x7f78080af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49382/0 (socket says 172.21.15.115:49382) 2023-12-01T01:37:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.991+0000 7f780dfdf640 1 -- 172.21.15.115:0/3526857734 learned_addr learned my addr 172.21.15.115:0/3526857734 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f780dfdf640 1 -- 172.21.15.115:0/3526857734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78080cea50 msgr2=0x7f780815b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f780dfdf640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78080cea50 0x7f780815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f780dfdf640 1 -- 172.21.15.115:0/3526857734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f780815acf0 msgr2=0x7f78080b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f780dfdf640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f780815acf0 0x7f78080b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f780dfdf640 1 -- 172.21.15.115:0/3526857734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc009b30 con 0x7f78080af790 2023-12-01T01:37:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f780dfdf640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78080af790 0x7f78080af0b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f77fc036040 tx=0x7f77fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f78067fc640 1 -- 172.21.15.115:0/3526857734 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc009510 con 0x7f78080af790 2023-12-01T01:37:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f78067fc640 1 -- 172.21.15.115:0/3526857734 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc03e450 con 0x7f78080af790 2023-12-01T01:37:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f78067fc640 1 -- 172.21.15.115:0/3526857734 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc03e7e0 con 0x7f78080af790 2023-12-01T01:37:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f7810243640 1 -- 172.21.15.115:0/3526857734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780815d810 con 0x7f78080af790 2023-12-01T01:37:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.992+0000 7f7810243640 1 -- 172.21.15.115:0/3526857734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780815e010 con 0x7f78080af790 2023-12-01T01:37:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.993+0000 7f7810243640 1 -- 172.21.15.115:0/3526857734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780815dd30 con 0x7f78080af790 2023-12-01T01:37:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.994+0000 7f78067fc640 1 -- 172.21.15.115:0/3526857734 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f77fc03e940 con 0x7f78080af790 2023-12-01T01:37:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.995+0000 7f78067fc640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77ec06a460 0x7f77ec06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.995+0000 7f78067fc640 1 -- 172.21.15.115:0/3526857734 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f77fc0ab150 con 0x7f78080af790 2023-12-01T01:37:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.995+0000 7f780d7de640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77ec06a460 0x7f77ec06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:17.996+0000 7f780d7de640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77ec06a460 0x7f77ec06c920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f77f4006e80 tx=0x7f77f4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:18.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.000+0000 7f78067fc640 1 -- 172.21.15.115:0/3526857734 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f780815dd30 con 0x7f78080af790 2023-12-01T01:37:18.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.249+0000 7f7810243640 1 -- 172.21.15.115:0/3526857734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f780815dd30 con 0x7f78080af790 2023-12-01T01:37:18.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.256+0000 7f78067fc640 1 -- 172.21.15.115:0/3526857734 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f780815dd30 con 0x7f78080af790 2023-12-01T01:37:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.258+0000 7f7810243640 1 -- 172.21.15.115:0/3526857734 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77ec06a460 msgr2=0x7f77ec06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.258+0000 7f7810243640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77ec06a460 0x7f77ec06c920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f77f4006e80 tx=0x7f77f4008040 comp rx=0 tx=0).stop 2023-12-01T01:37:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.258+0000 7f7810243640 1 -- 172.21.15.115:0/3526857734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78080af790 msgr2=0x7f78080af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.258+0000 7f7810243640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78080af790 0x7f78080af0b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f77fc036040 tx=0x7f77fc004a40 comp rx=0 tx=0).stop 2023-12-01T01:37:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.258+0000 7f7810243640 1 -- 172.21.15.115:0/3526857734 shutdown_connections 2023-12-01T01:37:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.258+0000 7f7810243640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f77ec06a460 0x7f77ec06c920 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.258+0000 7f7810243640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78080cea50 0x7f780815b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.258+0000 7f7810243640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f780815acf0 0x7f78080b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.258+0000 7f7810243640 1 --2- 172.21.15.115:0/3526857734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78080af790 0x7f78080af0b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.258+0000 7f7810243640 1 -- 172.21.15.115:0/3526857734 >> 172.21.15.115:0/3526857734 conn(0x7f7808065b40 msgr2=0x7f78080c36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.259+0000 7f7810243640 1 -- 172.21.15.115:0/3526857734 shutdown_connections 2023-12-01T01:37:18.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.259+0000 7f7810243640 1 -- 172.21.15.115:0/3526857734 wait complete. 2023-12-01T01:37:18.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-12-01T01:37:18.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.475+0000 7f1f7da8f640 1 Processor -- start 2023-12-01T01:37:18.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.475+0000 7f1f7da8f640 1 -- start start 2023-12-01T01:37:18.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.476+0000 7f1f7da8f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1f780b6090 0x7f1f780cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.476+0000 7f1f7da8f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f780cf740 0x7f1f780cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.476+0000 7f1f7da8f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f78160750 con 0x7f1f78160370 2023-12-01T01:37:18.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.476+0000 7f1f7da8f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f780ce660 con 0x7f1f780b6090 2023-12-01T01:37:18.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.476+0000 7f1f7da8f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780d0130 con 0x7f1f780cf740 2023-12-01T01:37:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.477+0000 7f1f76ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1f780b6090 0x7f1f780cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.477+0000 7f1f76ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1f780b6090 0x7f1f780cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49392/0 (socket says 172.21.15.115:49392) 2023-12-01T01:37:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.477+0000 7f1f76ffd640 1 -- 172.21.15.115:0/3160089384 learned_addr learned my addr 172.21.15.115:0/3160089384 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.477+0000 7f1f77fff640 1 --2- 172.21.15.115:0/3160089384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f780cf740 0x7f1f780cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.477+0000 7f1f76ffd640 1 -- 172.21.15.115:0/3160089384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f780cf740 msgr2=0x7f1f780cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.477+0000 7f1f76ffd640 1 --2- 172.21.15.115:0/3160089384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f780cf740 0x7f1f780cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.477+0000 7f1f76ffd640 1 -- 172.21.15.115:0/3160089384 >> v1:172.21.15.142:6789/0 conn(0x7f1f78160370 legacy=0x7f1f781613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.477+0000 7f1f76ffd640 1 -- 172.21.15.115:0/3160089384 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f780d2c10 con 0x7f1f780b6090 2023-12-01T01:37:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.477+0000 7f1f77fff640 1 --2- 172.21.15.115:0/3160089384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f780cf740 0x7f1f780cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.478+0000 7f1f76ffd640 1 --2- 172.21.15.115:0/3160089384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 0x7f1f780cdcf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009e50 tx=0x7f1f6802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fdfcb9bdbfac02 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.479+0000 7f1f767fc640 1 -- 172.21.15.115:0/3160089384 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6802ca70 con 0x7f1f780b6090 2023-12-01T01:37:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.479+0000 7f1f767fc640 1 -- 172.21.15.115:0/3160089384 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6802cbd0 con 0x7f1f780b6090 2023-12-01T01:37:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.479+0000 7f1f767fc640 1 -- 172.21.15.115:0/3160089384 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f680343f0 con 0x7f1f780b6090 2023-12-01T01:37:18.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.479+0000 7f1f7da8f640 1 -- 172.21.15.115:0/3160089384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 msgr2=0x7f1f780cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:18.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.479+0000 7f1f7da8f640 1 --2- 172.21.15.115:0/3160089384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 0x7f1f780cdcf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009e50 tx=0x7f1f6802bed0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.479+0000 7f1f7da8f640 1 -- 172.21.15.115:0/3160089384 shutdown_connections 2023-12-01T01:37:18.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.479+0000 7f1f7da8f640 1 --2- 172.21.15.115:0/3160089384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f780cf740 0x7f1f780cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.479+0000 7f1f7da8f640 1 --2- 172.21.15.115:0/3160089384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 0x7f1f780cdcf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.479+0000 7f1f7da8f640 1 -- 172.21.15.115:0/3160089384 >> 172.21.15.115:0/3160089384 conn(0x7f1f78065960 msgr2=0x7f1f78065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.480+0000 7f1f7da8f640 1 -- 172.21.15.115:0/3160089384 shutdown_connections 2023-12-01T01:37:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.480+0000 7f1f7da8f640 1 -- 172.21.15.115:0/3160089384 wait complete. 2023-12-01T01:37:18.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.481+0000 7f1f7da8f640 1 Processor -- start 2023-12-01T01:37:18.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.481+0000 7f1f7da8f640 1 -- start start 2023-12-01T01:37:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.482+0000 7f1f7da8f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 0x7f1f780b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.482+0000 7f1f7da8f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f780cf740 0x7f1f7815ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.482+0000 7f1f7da8f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f78160370 0x7f1f7815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.482+0000 7f1f7da8f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780c6500 con 0x7f1f780b6090 2023-12-01T01:37:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.482+0000 7f1f7da8f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780c67a0 con 0x7f1f780cf740 2023-12-01T01:37:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.482+0000 7f1f7da8f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780c6a80 con 0x7f1f78160370 2023-12-01T01:37:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f777fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 0x7f1f780b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f777fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 0x7f1f780b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49400/0 (socket says 172.21.15.115:49400) 2023-12-01T01:37:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f777fe640 1 -- 172.21.15.115:0/1875154426 learned_addr learned my addr 172.21.15.115:0/1875154426 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f76ffd640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f780cf740 0x7f1f7815ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f777fe640 1 -- 172.21.15.115:0/1875154426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f78160370 msgr2=0x7f1f7815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f77fff640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f78160370 0x7f1f7815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f777fe640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f78160370 0x7f1f7815efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f777fe640 1 -- 172.21.15.115:0/1875154426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f780cf740 msgr2=0x7f1f7815ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f777fe640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f780cf740 0x7f1f7815ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f777fe640 1 -- 172.21.15.115:0/1875154426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f68009b30 con 0x7f1f780b6090 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f76ffd640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f780cf740 0x7f1f7815ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.483+0000 7f1f777fe640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 0x7f1f780b4590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000b3f0 tx=0x7f1f6000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.484+0000 7f1f6ffff640 1 -- 172.21.15.115:0/1875154426 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60011830 con 0x7f1f780b6090 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.484+0000 7f1f6ffff640 1 -- 172.21.15.115:0/1875154426 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f60011990 con 0x7f1f780b6090 2023-12-01T01:37:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.484+0000 7f1f7da8f640 1 -- 172.21.15.115:0/1875154426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f780c6d60 con 0x7f1f780b6090 2023-12-01T01:37:18.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.484+0000 7f1f6ffff640 1 -- 172.21.15.115:0/1875154426 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60011d40 con 0x7f1f780b6090 2023-12-01T01:37:18.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.484+0000 7f1f7da8f640 1 -- 172.21.15.115:0/1875154426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f780b4ad0 con 0x7f1f780b6090 2023-12-01T01:37:18.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.485+0000 7f1f7da8f640 1 -- 172.21.15.115:0/1875154426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f780c7280 con 0x7f1f780b6090 2023-12-01T01:37:18.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.486+0000 7f1f6ffff640 1 -- 172.21.15.115:0/1875154426 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1f6001a400 con 0x7f1f780b6090 2023-12-01T01:37:18.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.487+0000 7f1f6ffff640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f5806a430 0x7f1f5806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.487+0000 7f1f6ffff640 1 -- 172.21.15.115:0/1875154426 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1f60088460 con 0x7f1f780b6090 2023-12-01T01:37:18.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.487+0000 7f1f76ffd640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f5806a430 0x7f1f5806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.488+0000 7f1f76ffd640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f5806a430 0x7f1f5806c8f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f68035000 tx=0x7f1f68030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:18.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.491+0000 7f1f6ffff640 1 -- 172.21.15.115:0/1875154426 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1f780c7280 con 0x7f1f780b6090 2023-12-01T01:37:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.750+0000 7f1f7da8f640 1 -- 172.21.15.115:0/1875154426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f1f780c7280 con 0x7f1f780b6090 2023-12-01T01:37:18.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.755+0000 7f1f6ffff640 1 -- 172.21.15.115:0/1875154426 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1f780c7280 con 0x7f1f780b6090 2023-12-01T01:37:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 -- 172.21.15.115:0/1875154426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f5806a430 msgr2=0x7f1f5806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f5806a430 0x7f1f5806c8f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f68035000 tx=0x7f1f68030000 comp rx=0 tx=0).stop 2023-12-01T01:37:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 -- 172.21.15.115:0/1875154426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 msgr2=0x7f1f780b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 0x7f1f780b4590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000b3f0 tx=0x7f1f6000b8b0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 -- 172.21.15.115:0/1875154426 shutdown_connections 2023-12-01T01:37:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f5806a430 0x7f1f5806c8f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f78160370 0x7f1f7815efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f780cf740 0x7f1f7815ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 --2- 172.21.15.115:0/1875154426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f780b6090 0x7f1f780b4590 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 -- 172.21.15.115:0/1875154426 >> 172.21.15.115:0/1875154426 conn(0x7f1f78065960 msgr2=0x7f1f78065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:18.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 -- 172.21.15.115:0/1875154426 shutdown_connections 2023-12-01T01:37:18.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.758+0000 7f1f7da8f640 1 -- 172.21.15.115:0/1875154426 wait complete. 2023-12-01T01:37:18.777 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-12-01T01:37:18.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.980+0000 7fe4a9eb9640 1 Processor -- start 2023-12-01T01:37:18.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.981+0000 7fe4a9eb9640 1 -- start start 2023-12-01T01:37:18.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.982+0000 7fe4a9eb9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe4a415cd10 0x7fe4a40cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a9eb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4a40b4280 0x7fe4a40700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a9eb9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a40cb660 con 0x7fe4a40b0930 2023-12-01T01:37:18.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a9eb9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a40cbc20 con 0x7fe4a415cd10 2023-12-01T01:37:18.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a9eb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40cb940 con 0x7fe4a40b4280 2023-12-01T01:37:18.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a8eb7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe4a415cd10 0x7fe4a40cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a8eb7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe4a415cd10 0x7fe4a40cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49412/0 (socket says 172.21.15.115:49412) 2023-12-01T01:37:18.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a8eb7640 1 -- 172.21.15.115:0/4025356490 learned_addr learned my addr 172.21.15.115:0/4025356490 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:18.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a8eb7640 1 -- 172.21.15.115:0/4025356490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4a40b4280 msgr2=0x7fe4a40700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:18.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a8eb7640 1 --2- 172.21.15.115:0/4025356490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4a40b4280 0x7fe4a40700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a8eb7640 1 -- 172.21.15.115:0/4025356490 >> v1:172.21.15.142:6789/0 conn(0x7fe4a40b0930 legacy=0x7fe4a40b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:18.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.983+0000 7fe4a8eb7640 1 -- 172.21.15.115:0/4025356490 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a406f8c0 con 0x7fe4a415cd10 2023-12-01T01:37:18.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.984+0000 7fe4a8eb7640 1 --2- 172.21.15.115:0/4025356490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 0x7fe4a40cb080 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe498009870 tx=0x7fe49802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec604a619a0a7fa server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:18.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.985+0000 7fe4a37fe640 1 -- 172.21.15.115:0/4025356490 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4980317c0 con 0x7fe4a415cd10 2023-12-01T01:37:18.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.985+0000 7fe4a37fe640 1 -- 172.21.15.115:0/4025356490 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe498031920 con 0x7fe4a415cd10 2023-12-01T01:37:18.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.985+0000 7fe4a37fe640 1 -- 172.21.15.115:0/4025356490 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498031cb0 con 0x7fe4a415cd10 2023-12-01T01:37:18.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.985+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/4025356490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 msgr2=0x7fe4a40cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:18.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.985+0000 7fe4a9eb9640 1 --2- 172.21.15.115:0/4025356490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 0x7fe4a40cb080 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe498009870 tx=0x7fe49802b860 comp rx=0 tx=0).stop 2023-12-01T01:37:18.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.986+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/4025356490 shutdown_connections 2023-12-01T01:37:18.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.986+0000 7fe4a9eb9640 1 --2- 172.21.15.115:0/4025356490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4a40b4280 0x7fe4a40700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.986+0000 7fe4a9eb9640 1 --2- 172.21.15.115:0/4025356490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 0x7fe4a40cb080 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.986+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/4025356490 >> 172.21.15.115:0/4025356490 conn(0x7fe4a4065960 msgr2=0x7fe4a4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:18.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.986+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/4025356490 shutdown_connections 2023-12-01T01:37:18.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.986+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/4025356490 wait complete. 2023-12-01T01:37:18.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.987+0000 7fe4a9eb9640 1 Processor -- start 2023-12-01T01:37:18.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.987+0000 7fe4a9eb9640 1 -- start start 2023-12-01T01:37:18.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.988+0000 7fe4a9eb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4a40b0930 0x7fe4a415efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.988+0000 7fe4a9eb9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4a40b4280 0x7fe4a414e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.988+0000 7fe4a9eb9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 0x7fe4a40a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.988+0000 7fe4a9eb9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40ab290 con 0x7fe4a415cd10 2023-12-01T01:37:18.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.988+0000 7fe4a9eb9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a41f2320 con 0x7fe4a40b4280 2023-12-01T01:37:18.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.988+0000 7fe4a9eb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a415f600 con 0x7fe4a40b0930 2023-12-01T01:37:18.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a96b8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 0x7fe4a40a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a96b8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 0x7fe4a40a92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49414/0 (socket says 172.21.15.115:49414) 2023-12-01T01:37:18.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a96b8640 1 -- 172.21.15.115:0/616501306 learned_addr learned my addr 172.21.15.115:0/616501306 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:18.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a8eb7640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4a40b0930 0x7fe4a415efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a96b8640 1 -- 172.21.15.115:0/616501306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4a40b0930 msgr2=0x7fe4a415efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a3fff640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4a40b4280 0x7fe4a414e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a96b8640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4a40b0930 0x7fe4a415efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a96b8640 1 -- 172.21.15.115:0/616501306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4a40b4280 msgr2=0x7fe4a414e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a96b8640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4a40b4280 0x7fe4a414e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a96b8640 1 -- 172.21.15.115:0/616501306 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4980094d0 con 0x7fe4a415cd10 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.989+0000 7fe4a96b8640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 0x7fe4a40a92f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00b970 tx=0x7fe49c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.990+0000 7fe4a17fa640 1 -- 172.21.15.115:0/616501306 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c00cb40 con 0x7fe4a415cd10 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.990+0000 7fe4a17fa640 1 -- 172.21.15.115:0/616501306 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c00ccc0 con 0x7fe4a415cd10 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.990+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/616501306 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a415f8e0 con 0x7fe4a415cd10 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.990+0000 7fe4a17fa640 1 -- 172.21.15.115:0/616501306 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c014680 con 0x7fe4a415cd10 2023-12-01T01:37:18.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.990+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/616501306 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a415fea0 con 0x7fe4a415cd10 2023-12-01T01:37:18.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.991+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/616501306 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a41f24e0 con 0x7fe4a415cd10 2023-12-01T01:37:18.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.992+0000 7fe4a17fa640 1 -- 172.21.15.115:0/616501306 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe49c01d3b0 con 0x7fe4a415cd10 2023-12-01T01:37:18.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.993+0000 7fe4a17fa640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe48806a430 0x7fe48806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:18.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.993+0000 7fe4a17fa640 1 -- 172.21.15.115:0/616501306 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe49c0885b0 con 0x7fe4a415cd10 2023-12-01T01:37:18.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.993+0000 7fe4a8eb7640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe48806a430 0x7fe48806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:18.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.994+0000 7fe4a8eb7640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe48806a430 0x7fe48806c8f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe498036000 tx=0x7fe49803d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:18.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:18.997+0000 7fe4a17fa640 1 -- 172.21.15.115:0/616501306 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe49c053e60 con 0x7fe4a415cd10 2023-12-01T01:37:19.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.238+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/616501306 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fe4a40673a0 con 0x7fe4a415cd10 2023-12-01T01:37:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.243+0000 7fe4a17fa640 1 -- 172.21.15.115:0/616501306 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe49c019020 con 0x7fe4a415cd10 2023-12-01T01:37:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/616501306 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe48806a430 msgr2=0x7fe48806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe48806a430 0x7fe48806c8f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe498036000 tx=0x7fe49803d040 comp rx=0 tx=0).stop 2023-12-01T01:37:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/616501306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 msgr2=0x7fe4a40a92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 0x7fe4a40a92f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00b970 tx=0x7fe49c00be30 comp rx=0 tx=0).stop 2023-12-01T01:37:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/616501306 shutdown_connections 2023-12-01T01:37:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe48806a430 0x7fe48806c8f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4a415cd10 0x7fe4a40a92f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4a40b4280 0x7fe4a414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 --2- 172.21.15.115:0/616501306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4a40b0930 0x7fe4a415efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/616501306 >> 172.21.15.115:0/616501306 conn(0x7fe4a4065960 msgr2=0x7fe4a40c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/616501306 shutdown_connections 2023-12-01T01:37:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.246+0000 7fe4a9eb9640 1 -- 172.21.15.115:0/616501306 wait complete. 2023-12-01T01:37:19.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-12-01T01:37:19.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.468+0000 7fb015dae640 1 Processor -- start 2023-12-01T01:37:19.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.468+0000 7fb015dae640 1 -- start start 2023-12-01T01:37:19.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.469+0000 7fb015dae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb01015acf0 0x7fb010070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.469+0000 7fb015dae640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb010070e90 con 0x7fb0100cea50 2023-12-01T01:37:19.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.469+0000 7fb015dae640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb010071170 con 0x7fb0100af790 2023-12-01T01:37:19.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.469+0000 7fb015dae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100b01a0 con 0x7fb01015acf0 2023-12-01T01:37:19.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.469+0000 7fb00ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb0100cea50 0x7fb0100c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36064/0 (socket says 172.21.15.115:36064) 2023-12-01T01:37:19.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00ffff640 1 -- 172.21.15.115:0/2821753106 learned_addr learned my addr 172.21.15.115:0/2821753106 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:19.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00f7fe640 1 --2- 172.21.15.115:0/2821753106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb01015acf0 0x7fb010070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159663394 0 0) 0x7fb010070e90 con 0x7fb0100cea50 2023-12-01T01:37:19.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faffc003520 con 0x7fb0100cea50 2023-12-01T01:37:19.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187108534 0 0) 0x7faffc003520 con 0x7fb0100cea50 2023-12-01T01:37:19.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb01015acf0 msgr2=0x7fb010070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:19.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 --2- 172.21.15.115:0/2821753106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb01015acf0 0x7fb010070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 >> v1:172.21.15.142:6789/0 conn(0x7fb0100af790 legacy=0x7fb01015ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01015e760 con 0x7fb0100cea50 2023-12-01T01:37:19.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb0080019f0 con 0x7fb0100cea50 2023-12-01T01:37:19.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 >> v1:172.21.15.115:6789/0 conn(0x7fb0100cea50 legacy=0x7fb0100c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 --2- 172.21.15.115:0/2821753106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faffc0039a0 0x7faffc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 --2- 172.21.15.115:0/2821753106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb01015acf0 0x7faffc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.470+0000 7fb00effd640 1 --2- 172.21.15.115:0/2821753106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faffc004d00 0x7faffc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.471+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100b01a0 con 0x7faffc0039a0 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.471+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc00bab0 con 0x7faffc004d00 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.471+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc00bc70 con 0x7fb01015acf0 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.471+0000 7fb00f7fe640 1 --2- 172.21.15.115:0/2821753106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faffc0039a0 0x7faffc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.471+0000 7fb00f7fe640 1 -- 172.21.15.115:0/2821753106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb01015acf0 msgr2=0x7faffc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.471+0000 7fb00f7fe640 1 --2- 172.21.15.115:0/2821753106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb01015acf0 0x7faffc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.471+0000 7fb00f7fe640 1 -- 172.21.15.115:0/2821753106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faffc004d00 msgr2=0x7faffc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.471+0000 7fb00f7fe640 1 --2- 172.21.15.115:0/2821753106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faffc004d00 0x7faffc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.471+0000 7fb00f7fe640 1 -- 172.21.15.115:0/2821753106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc003520 con 0x7faffc0039a0 2023-12-01T01:37:19.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.472+0000 7fb00f7fe640 1 --2- 172.21.15.115:0/2821753106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faffc0039a0 0x7faffc003dc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb000009ac0 tx=0x7faffc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edaf6106f35c66f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:19.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.472+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00002c630 con 0x7faffc0039a0 2023-12-01T01:37:19.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.472+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00002c790 con 0x7faffc0039a0 2023-12-01T01:37:19.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.472+0000 7fb00effd640 1 -- 172.21.15.115:0/2821753106 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00002cb20 con 0x7faffc0039a0 2023-12-01T01:37:19.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.473+0000 7fb015dae640 1 -- 172.21.15.115:0/2821753106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faffc0039a0 msgr2=0x7faffc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.473+0000 7fb015dae640 1 --2- 172.21.15.115:0/2821753106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faffc0039a0 0x7faffc003dc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb000009ac0 tx=0x7faffc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.473+0000 7fb00ffff640 1 -- 172.21.15.115:0/2821753106 reap_dead start 2023-12-01T01:37:19.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.473+0000 7fb015dae640 1 -- 172.21.15.115:0/2821753106 shutdown_connections 2023-12-01T01:37:19.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.473+0000 7fb015dae640 1 -- 172.21.15.115:0/2821753106 >> 172.21.15.115:0/2821753106 conn(0x7fb010065b40 msgr2=0x7fb010065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:19.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.473+0000 7fb015dae640 1 -- 172.21.15.115:0/2821753106 shutdown_connections 2023-12-01T01:37:19.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.473+0000 7fb015dae640 1 -- 172.21.15.115:0/2821753106 wait complete. 2023-12-01T01:37:19.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.475+0000 7fb015dae640 1 Processor -- start 2023-12-01T01:37:19.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.475+0000 7fb015dae640 1 -- start start 2023-12-01T01:37:19.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.476+0000 7fb015dae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faffc0039a0 0x7fb0100c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.476+0000 7fb015dae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0100c8e40 0x7fb01014a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.476+0000 7fb015dae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100b2e30 0x7fb0100b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.476+0000 7fb015dae640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100b3800 con 0x7fb0100c8e40 2023-12-01T01:37:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.476+0000 7fb015dae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100ca6a0 con 0x7fb0100b2e30 2023-12-01T01:37:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.476+0000 7fb015dae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100c9600 con 0x7faffc0039a0 2023-12-01T01:37:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.476+0000 7fb007fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0100c8e40 0x7fb01014a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.476+0000 7fb007fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0100c8e40 0x7fb01014a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49426/0 (socket says 172.21.15.115:49426) 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.476+0000 7fb007fff640 1 -- 172.21.15.115:0/2653197233 learned_addr learned my addr 172.21.15.115:0/2653197233 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb00f7fe640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faffc0039a0 0x7fb0100c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb007fff640 1 -- 172.21.15.115:0/2653197233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faffc0039a0 msgr2=0x7fb0100c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb007fff640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faffc0039a0 0x7fb0100c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb00ffff640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100b2e30 0x7fb0100b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb007fff640 1 -- 172.21.15.115:0/2653197233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100b2e30 msgr2=0x7fb0100b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb007fff640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100b2e30 0x7fb0100b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb007fff640 1 -- 172.21.15.115:0/2653197233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100c98e0 con 0x7fb0100c8e40 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb00f7fe640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faffc0039a0 0x7fb0100c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb007fff640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0100c8e40 0x7fb01014a280 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faff800c530 tx=0x7faff800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb00cff9640 1 -- 172.21.15.115:0/2653197233 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff80052d0 con 0x7fb0100c8e40 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb00cff9640 1 -- 172.21.15.115:0/2653197233 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff8005430 con 0x7fb0100c8e40 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb00cff9640 1 -- 172.21.15.115:0/2653197233 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8012870 con 0x7fb0100c8e40 2023-12-01T01:37:19.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb015dae640 1 -- 172.21.15.115:0/2653197233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0100c9bc0 con 0x7fb0100c8e40 2023-12-01T01:37:19.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.477+0000 7fb015dae640 1 -- 172.21.15.115:0/2653197233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0100ca180 con 0x7fb0100c8e40 2023-12-01T01:37:19.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.479+0000 7fb015dae640 1 -- 172.21.15.115:0/2653197233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0100ca340 con 0x7fb0100c8e40 2023-12-01T01:37:19.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.479+0000 7fb00cff9640 1 -- 172.21.15.115:0/2653197233 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faff80129d0 con 0x7fb0100c8e40 2023-12-01T01:37:19.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.480+0000 7fb00cff9640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faff006a430 0x7faff006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.480+0000 7fb00cff9640 1 -- 172.21.15.115:0/2653197233 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faff8088190 con 0x7fb0100c8e40 2023-12-01T01:37:19.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.481+0000 7fb00f7fe640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faff006a430 0x7faff006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.481+0000 7fb00f7fe640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faff006a430 0x7faff006c8f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb000036000 tx=0x7fb000030000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:19.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.485+0000 7fb00cff9640 1 -- 172.21.15.115:0/2653197233 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faff8053ac0 con 0x7fb0100c8e40 2023-12-01T01:37:19.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.744+0000 7fb015dae640 1 -- 172.21.15.115:0/2653197233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fb01004e890 con 0x7fb0100c8e40 2023-12-01T01:37:19.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.752+0000 7fb00cff9640 1 -- 172.21.15.115:0/2653197233 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7faff80570e0 con 0x7fb0100c8e40 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 -- 172.21.15.115:0/2653197233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faff006a430 msgr2=0x7faff006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faff006a430 0x7faff006c8f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb000036000 tx=0x7fb000030000 comp rx=0 tx=0).stop 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 -- 172.21.15.115:0/2653197233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0100c8e40 msgr2=0x7fb01014a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0100c8e40 0x7fb01014a280 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faff800c530 tx=0x7faff800c9f0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 -- 172.21.15.115:0/2653197233 shutdown_connections 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faff006a430 0x7faff006c8f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0100b2e30 0x7fb0100b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0100c8e40 0x7fb01014a280 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 --2- 172.21.15.115:0/2653197233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faffc0039a0 0x7fb0100c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 -- 172.21.15.115:0/2653197233 >> 172.21.15.115:0/2653197233 conn(0x7fb010065b40 msgr2=0x7fb010155100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 -- 172.21.15.115:0/2653197233 shutdown_connections 2023-12-01T01:37:19.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.755+0000 7fb015dae640 1 -- 172.21.15.115:0/2653197233 wait complete. 2023-12-01T01:37:19.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-12-01T01:37:19.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.978+0000 7f21241a9640 1 Processor -- start 2023-12-01T01:37:19.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.979+0000 7f21241a9640 1 -- start start 2023-12-01T01:37:19.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.980+0000 7f21241a9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f211c160370 0x7f211c0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.980+0000 7f21241a9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f211c0b6090 0x7f211c1613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.980+0000 7f21241a9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211c0d2c10 con 0x7f211c0cf740 2023-12-01T01:37:19.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.980+0000 7f21241a9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211c0ce660 con 0x7f211c0b6090 2023-12-01T01:37:19.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.980+0000 7f21241a9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211c1618f0 con 0x7f211c160370 2023-12-01T01:37:19.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.981+0000 7f2121744640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f211c0b6090 0x7f211c1613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.981+0000 7f2121744640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f211c0b6090 0x7f211c1613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49442/0 (socket says 172.21.15.115:49442) 2023-12-01T01:37:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.981+0000 7f2121744640 1 -- 172.21.15.115:0/1745555889 learned_addr learned my addr 172.21.15.115:0/1745555889 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.981+0000 7f2121f45640 1 --2- 172.21.15.115:0/1745555889 >> v2:172.21.15.142:3300/0 conn(0x7f211c160370 0x7f211c0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.981+0000 7f2121744640 1 -- 172.21.15.115:0/1745555889 >> v2:172.21.15.142:3300/0 conn(0x7f211c160370 msgr2=0x7f211c0cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.981+0000 7f2121744640 1 --2- 172.21.15.115:0/1745555889 >> v2:172.21.15.142:3300/0 conn(0x7f211c160370 0x7f211c0cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.981+0000 7f2121744640 1 -- 172.21.15.115:0/1745555889 >> v1:172.21.15.142:6789/0 conn(0x7f211c0cf740 legacy=0x7f211c0b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:19.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.981+0000 7f2121744640 1 -- 172.21.15.115:0/1745555889 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c160750 con 0x7f211c0b6090 2023-12-01T01:37:19.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.982+0000 7f2121744640 1 --2- 172.21.15.115:0/1745555889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0b6090 0x7f211c1613b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f211c0d3de0 tx=0x7f211002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d061dc254ff43994 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:19.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.983+0000 7f2120f43640 1 -- 172.21.15.115:0/1745555889 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110031870 con 0x7f211c0b6090 2023-12-01T01:37:19.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.983+0000 7f2120f43640 1 -- 172.21.15.115:0/1745555889 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21100319d0 con 0x7f211c0b6090 2023-12-01T01:37:19.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.983+0000 7f2120f43640 1 -- 172.21.15.115:0/1745555889 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110031d60 con 0x7f211c0b6090 2023-12-01T01:37:19.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.983+0000 7f21241a9640 1 -- 172.21.15.115:0/1745555889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0b6090 msgr2=0x7f211c1613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.983+0000 7f21241a9640 1 --2- 172.21.15.115:0/1745555889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0b6090 0x7f211c1613b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f211c0d3de0 tx=0x7f211002b910 comp rx=0 tx=0).stop 2023-12-01T01:37:19.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.983+0000 7f21241a9640 1 -- 172.21.15.115:0/1745555889 shutdown_connections 2023-12-01T01:37:19.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.983+0000 7f21241a9640 1 --2- 172.21.15.115:0/1745555889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0b6090 0x7f211c1613b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.983+0000 7f21241a9640 1 --2- 172.21.15.115:0/1745555889 >> v2:172.21.15.142:3300/0 conn(0x7f211c160370 0x7f211c0cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.983+0000 7f21241a9640 1 -- 172.21.15.115:0/1745555889 >> 172.21.15.115:0/1745555889 conn(0x7f211c065960 msgr2=0x7f211c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:19.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.984+0000 7f21241a9640 1 -- 172.21.15.115:0/1745555889 shutdown_connections 2023-12-01T01:37:19.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.984+0000 7f21241a9640 1 -- 172.21.15.115:0/1745555889 wait complete. 2023-12-01T01:37:19.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.985+0000 7f21241a9640 1 Processor -- start 2023-12-01T01:37:19.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.985+0000 7f21241a9640 1 -- start start 2023-12-01T01:37:19.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f21241a9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f211c0b6090 0x7f211c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f21241a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0cf740 0x7f211c151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f21241a9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c160370 0x7f211c152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f21241a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c157ab0 con 0x7f211c0cf740 2023-12-01T01:37:19.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f21241a9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c1525c0 con 0x7f211c160370 2023-12-01T01:37:19.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f21241a9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c156e30 con 0x7f211c0b6090 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f2121744640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0cf740 0x7f211c151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f2121744640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0cf740 0x7f211c151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49448/0 (socket says 172.21.15.115:49448) 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f2121744640 1 -- 172.21.15.115:0/285124401 learned_addr learned my addr 172.21.15.115:0/285124401 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f2121744640 1 -- 172.21.15.115:0/285124401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f211c0b6090 msgr2=0x7f211c0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f2121744640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f211c0b6090 0x7f211c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f2121744640 1 -- 172.21.15.115:0/285124401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c160370 msgr2=0x7f211c152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f2122746640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c160370 0x7f211c152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f2121744640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c160370 0x7f211c152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f2121744640 1 -- 172.21.15.115:0/285124401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21100094d0 con 0x7f211c0cf740 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.987+0000 7f2121744640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0cf740 0x7f211c151b40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2110000c00 tx=0x7f21100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.988+0000 7f211a7fc640 1 -- 172.21.15.115:0/285124401 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211003abe0 con 0x7f211c0cf740 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.988+0000 7f211a7fc640 1 -- 172.21.15.115:0/285124401 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211003ad40 con 0x7f211c0cf740 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.988+0000 7f211a7fc640 1 -- 172.21.15.115:0/285124401 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110044610 con 0x7f211c0cf740 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.988+0000 7f21241a9640 1 -- 172.21.15.115:0/285124401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c157110 con 0x7f211c0cf740 2023-12-01T01:37:19.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.988+0000 7f21241a9640 1 -- 172.21.15.115:0/285124401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c1576d0 con 0x7f211c0cf740 2023-12-01T01:37:19.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.989+0000 7f21241a9640 1 -- 172.21.15.115:0/285124401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c157bf0 con 0x7f211c0cf740 2023-12-01T01:37:19.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.990+0000 7f211a7fc640 1 -- 172.21.15.115:0/285124401 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f211004d450 con 0x7f211c0cf740 2023-12-01T01:37:19.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.990+0000 7f211a7fc640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f210806a430 0x7f210806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:19.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.991+0000 7f211a7fc640 1 -- 172.21.15.115:0/285124401 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f211002d040 con 0x7f211c0cf740 2023-12-01T01:37:19.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.991+0000 7f2121f45640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f210806a430 0x7f210806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:19.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.991+0000 7f2121f45640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f210806a430 0x7f210806c8f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f210c006fd0 tx=0x7f210c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:19.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:19.995+0000 7f211a7fc640 1 -- 172.21.15.115:0/285124401 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f211c157bf0 con 0x7f211c0cf740 2023-12-01T01:37:20.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.249+0000 7f21241a9640 1 -- 172.21.15.115:0/285124401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f211c0d2c10 con 0x7f211c0cf740 2023-12-01T01:37:20.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.256+0000 7f211a7fc640 1 -- 172.21.15.115:0/285124401 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f211c0d2c10 con 0x7f211c0cf740 2023-12-01T01:37:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 -- 172.21.15.115:0/285124401 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f210806a430 msgr2=0x7f210806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f210806a430 0x7f210806c8f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f210c006fd0 tx=0x7f210c008040 comp rx=0 tx=0).stop 2023-12-01T01:37:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 -- 172.21.15.115:0/285124401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0cf740 msgr2=0x7f211c151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0cf740 0x7f211c151b40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2110000c00 tx=0x7f21100046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 -- 172.21.15.115:0/285124401 shutdown_connections 2023-12-01T01:37:20.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f210806a430 0x7f210806c8f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c160370 0x7f211c152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f211c0cf740 0x7f211c151b40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 --2- 172.21.15.115:0/285124401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f211c0b6090 0x7f211c0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 -- 172.21.15.115:0/285124401 >> 172.21.15.115:0/285124401 conn(0x7f211c065960 msgr2=0x7f211c0d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:20.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 -- 172.21.15.115:0/285124401 shutdown_connections 2023-12-01T01:37:20.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.259+0000 7f21241a9640 1 -- 172.21.15.115:0/285124401 wait complete. 2023-12-01T01:37:20.279 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:37:20.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.481+0000 7f19c0f29640 1 Processor -- start 2023-12-01T01:37:20.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.481+0000 7f19c0f29640 1 -- start start 2023-12-01T01:37:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.482+0000 7f19c0f29640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc145620 0x7f19bc0ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.482+0000 7f19c0f29640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f19bc0cdd80 0x7f19bc06b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.482+0000 7f19c0f29640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f19bc06b7b0 0x7f19bc160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.482+0000 7f19c0f29640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19bc06bc20 con 0x7f19bc0cdd80 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.482+0000 7f19c0f29640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19bc0ad7e0 con 0x7f19bc06b7b0 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.482+0000 7f19c0f29640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc0cc8e0 con 0x7f19bc145620 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.482+0000 7f19b2ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f19bc0cdd80 0x7f19bc06b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.482+0000 7f19b2ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f19bc0cdd80 0x7f19bc06b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49456/0 (socket says 172.21.15.115:49456) 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.482+0000 7f19b2ffd640 1 -- 172.21.15.115:0/453009959 learned_addr learned my addr 172.21.15.115:0/453009959 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.483+0000 7f19bb7fe640 1 --2- 172.21.15.115:0/453009959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc145620 0x7f19bc0ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.483+0000 7f19b2ffd640 1 -- 172.21.15.115:0/453009959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc145620 msgr2=0x7f19bc0ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.483+0000 7f19bbfff640 1 --2- 172.21.15.115:0/453009959 >> v2:172.21.15.142:3300/0 conn(0x7f19bc06b7b0 0x7f19bc160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.483+0000 7f19b2ffd640 1 --2- 172.21.15.115:0/453009959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc145620 0x7f19bc0ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.483+0000 7f19b2ffd640 1 -- 172.21.15.115:0/453009959 >> v2:172.21.15.142:3300/0 conn(0x7f19bc06b7b0 msgr2=0x7f19bc160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.483+0000 7f19b2ffd640 1 --2- 172.21.15.115:0/453009959 >> v2:172.21.15.142:3300/0 conn(0x7f19bc06b7b0 0x7f19bc160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.483+0000 7f19b2ffd640 1 -- 172.21.15.115:0/453009959 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc0ce260 con 0x7f19bc0cdd80 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.483+0000 7f19bbfff640 1 --2- 172.21.15.115:0/453009959 >> v2:172.21.15.142:3300/0 conn(0x7f19bc06b7b0 0x7f19bc160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:20.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.483+0000 7f19bb7fe640 1 --2- 172.21.15.115:0/453009959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc145620 0x7f19bc0ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.484+0000 7f19b2ffd640 1 --2- 172.21.15.115:0/453009959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc0cdd80 0x7f19bc06b270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f19bc06bf60 tx=0x7f19a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7b9505f8fe9862 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.484+0000 7f19baffd640 1 -- 172.21.15.115:0/453009959 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4031870 con 0x7f19bc0cdd80 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.484+0000 7f19baffd640 1 -- 172.21.15.115:0/453009959 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a40319d0 con 0x7f19bc0cdd80 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.484+0000 7f19baffd640 1 -- 172.21.15.115:0/453009959 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4031d60 con 0x7f19bc0cdd80 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.485+0000 7f19c0f29640 1 -- 172.21.15.115:0/453009959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc0cdd80 msgr2=0x7f19bc06b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.485+0000 7f19c0f29640 1 --2- 172.21.15.115:0/453009959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc0cdd80 0x7f19bc06b270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f19bc06bf60 tx=0x7f19a402b910 comp rx=0 tx=0).stop 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.485+0000 7f19c0f29640 1 -- 172.21.15.115:0/453009959 shutdown_connections 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.485+0000 7f19c0f29640 1 --2- 172.21.15.115:0/453009959 >> v2:172.21.15.142:3300/0 conn(0x7f19bc06b7b0 0x7f19bc160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.485+0000 7f19c0f29640 1 --2- 172.21.15.115:0/453009959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc0cdd80 0x7f19bc06b270 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.485+0000 7f19c0f29640 1 --2- 172.21.15.115:0/453009959 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc145620 0x7f19bc0ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.485+0000 7f19c0f29640 1 -- 172.21.15.115:0/453009959 >> 172.21.15.115:0/453009959 conn(0x7f19bc0ce640 msgr2=0x7f19bc0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:20.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.485+0000 7f19c0f29640 1 -- 172.21.15.115:0/453009959 shutdown_connections 2023-12-01T01:37:20.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.485+0000 7f19c0f29640 1 -- 172.21.15.115:0/453009959 wait complete. 2023-12-01T01:37:20.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.487+0000 7f19c0f29640 1 Processor -- start 2023-12-01T01:37:20.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.487+0000 7f19c0f29640 1 -- start start 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19c0f29640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc06b7b0 0x7f19bc1f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19c0f29640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc0cdd80 0x7f19bc1f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19c0f29640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc145620 0x7f19bc1f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19c0f29640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc1f36f0 con 0x7f19bc06b7b0 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19c0f29640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc144730 con 0x7f19bc145620 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19c0f29640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc143e00 con 0x7f19bc0cdd80 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc06b7b0 0x7f19bc1f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc06b7b0 0x7f19bc1f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49466/0 (socket says 172.21.15.115:49466) 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19bb7fe640 1 -- 172.21.15.115:0/1061118627 learned_addr learned my addr 172.21.15.115:0/1061118627 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19b2ffd640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc0cdd80 0x7f19bc1f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19bbfff640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc145620 0x7f19bc1f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19bb7fe640 1 -- 172.21.15.115:0/1061118627 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc0cdd80 msgr2=0x7f19bc1f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19bb7fe640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc0cdd80 0x7f19bc1f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19bb7fe640 1 -- 172.21.15.115:0/1061118627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc145620 msgr2=0x7f19bc1f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19bb7fe640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc145620 0x7f19bc1f6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.488+0000 7f19bb7fe640 1 -- 172.21.15.115:0/1061118627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a40094d0 con 0x7f19bc06b7b0 2023-12-01T01:37:20.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.489+0000 7f19bb7fe640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc06b7b0 0x7f19bc1f2c50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00c820 tx=0x7f19ac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:20.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.489+0000 7f19b8ff9640 1 -- 172.21.15.115:0/1061118627 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac007a30 con 0x7f19bc06b7b0 2023-12-01T01:37:20.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.489+0000 7f19b8ff9640 1 -- 172.21.15.115:0/1061118627 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac007b90 con 0x7f19bc06b7b0 2023-12-01T01:37:20.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.489+0000 7f19b8ff9640 1 -- 172.21.15.115:0/1061118627 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac00ce70 con 0x7f19bc06b7b0 2023-12-01T01:37:20.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.489+0000 7f19c0f29640 1 -- 172.21.15.115:0/1061118627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc1440e0 con 0x7f19bc06b7b0 2023-12-01T01:37:20.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.489+0000 7f19c0f29640 1 -- 172.21.15.115:0/1061118627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19bc143700 con 0x7f19bc06b7b0 2023-12-01T01:37:20.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.490+0000 7f19c0f29640 1 -- 172.21.15.115:0/1061118627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19bc0742c0 con 0x7f19bc06b7b0 2023-12-01T01:37:20.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.491+0000 7f19b8ff9640 1 -- 172.21.15.115:0/1061118627 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f19ac0113f0 con 0x7f19bc06b7b0 2023-12-01T01:37:20.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.492+0000 7f19b8ff9640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f199c06a430 0x7f199c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.492+0000 7f19b8ff9640 1 -- 172.21.15.115:0/1061118627 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f19ac0887e0 con 0x7f19bc06b7b0 2023-12-01T01:37:20.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.492+0000 7f19b2ffd640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f199c06a430 0x7f199c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.494+0000 7f19b2ffd640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f199c06a430 0x7f199c06c8f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f19a4036000 tx=0x7f19a403c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:20.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.496+0000 7f19b8ff9640 1 -- 172.21.15.115:0/1061118627 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f19ac054080 con 0x7f19bc06b7b0 2023-12-01T01:37:20.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.688+0000 7f19c0f29640 1 -- 172.21.15.115:0/1061118627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19bc0673a0 con 0x7f19bc06b7b0 2023-12-01T01:37:20.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.690+0000 7f19b8ff9640 1 -- 172.21.15.115:0/1061118627 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12437 (secure 0 0 0) 0x7f19ac0576a0 con 0x7f19bc06b7b0 2023-12-01T01:37:20.690 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:20.691 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":31,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:37:16.717170+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:37:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 -- 172.21.15.115:0/1061118627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f199c06a430 msgr2=0x7f199c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f199c06a430 0x7f199c06c8f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f19a4036000 tx=0x7f19a403c040 comp rx=0 tx=0).stop 2023-12-01T01:37:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 -- 172.21.15.115:0/1061118627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc06b7b0 msgr2=0x7f19bc1f2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc06b7b0 0x7f19bc1f2c50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00c820 tx=0x7f19ac00cc20 comp rx=0 tx=0).stop 2023-12-01T01:37:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 -- 172.21.15.115:0/1061118627 shutdown_connections 2023-12-01T01:37:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f199c06a430 0x7f199c06c8f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19bc0cdd80 0x7f19bc1f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19bc145620 0x7f19bc1f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 --2- 172.21.15.115:0/1061118627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f19bc06b7b0 0x7f19bc1f2c50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 -- 172.21.15.115:0/1061118627 >> 172.21.15.115:0/1061118627 conn(0x7f19bc0ce640 msgr2=0x7f19bc13fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:20.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 -- 172.21.15.115:0/1061118627 shutdown_connections 2023-12-01T01:37:20.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.692+0000 7f19c0f29640 1 -- 172.21.15.115:0/1061118627 wait complete. 2023-12-01T01:37:20.713 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:37:20.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:37:20.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.924+0000 7fc202438640 1 Processor -- start 2023-12-01T01:37:20.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.924+0000 7fc202438640 1 -- start start 2023-12-01T01:37:20.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc202438640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1fc0b0f80 0x7fc1fc0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc202438640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc0cb290 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc202438640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc0b0ae0 con 0x7fc1fc160200 2023-12-01T01:37:20.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc202438640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b35f0 con 0x7fc1fc0b0f80 2023-12-01T01:37:20.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc1f3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1fc0b0f80 0x7fc1fc0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc1f3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1fc0b0f80 0x7fc1fc0b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49476/0 (socket says 172.21.15.115:49476) 2023-12-01T01:37:20.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc1f3fff640 1 -- 172.21.15.115:0/493526883 learned_addr learned my addr 172.21.15.115:0/493526883 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:20.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc1f3fff640 1 -- 172.21.15.115:0/493526883 >> v1:172.21.15.142:6789/0 conn(0x7fc1fc160200 legacy=0x7fc1fc06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:20.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc1f3fff640 1 -- 172.21.15.115:0/493526883 >> v1:172.21.15.115:6789/0 conn(0x7fc1fc0cbd30 legacy=0x7fc1fc0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc1f3fff640 1 -- 172.21.15.115:0/493526883 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc0b3310 con 0x7fc1fc0b0f80 2023-12-01T01:37:20.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.926+0000 7fc1fb7fe640 1 -- 172.21.15.115:0/493526883 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425003006 0 0) 0x7fc1fc0cb290 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.928+0000 7fc1f3fff640 1 --2- 172.21.15.115:0/493526883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0b0f80 0x7fc1fc0b2c80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009870 tx=0x7fc1e402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88edda6bcb887d92 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:20.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.928+0000 7fc1fb7fe640 1 -- 172.21.15.115:0/493526883 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4030860 con 0x7fc1fc0b0f80 2023-12-01T01:37:20.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.928+0000 7fc1fb7fe640 1 -- 172.21.15.115:0/493526883 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e40309c0 con 0x7fc1fc0b0f80 2023-12-01T01:37:20.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.928+0000 7fc1fb7fe640 1 -- 172.21.15.115:0/493526883 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4030d50 con 0x7fc1fc0b0f80 2023-12-01T01:37:20.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.929+0000 7fc202438640 1 -- 172.21.15.115:0/493526883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0b0f80 msgr2=0x7fc1fc0b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.929+0000 7fc202438640 1 --2- 172.21.15.115:0/493526883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0b0f80 0x7fc1fc0b2c80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009870 tx=0x7fc1e402b9a0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.929+0000 7fc202438640 1 -- 172.21.15.115:0/493526883 shutdown_connections 2023-12-01T01:37:20.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.929+0000 7fc202438640 1 --2- 172.21.15.115:0/493526883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0b0f80 0x7fc1fc0b2c80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.929+0000 7fc202438640 1 -- 172.21.15.115:0/493526883 >> 172.21.15.115:0/493526883 conn(0x7fc1fc065960 msgr2=0x7fc1fc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:20.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.929+0000 7fc202438640 1 -- 172.21.15.115:0/493526883 shutdown_connections 2023-12-01T01:37:20.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.929+0000 7fc202438640 1 -- 172.21.15.115:0/493526883 wait complete. 2023-12-01T01:37:20.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.931+0000 7fc202438640 1 Processor -- start 2023-12-01T01:37:20.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.931+0000 7fc202438640 1 -- start start 2023-12-01T01:37:20.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.932+0000 7fc202438640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b0f80 0x7fc1fc0c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.932+0000 7fc202438640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cbd30 0x7fc1fc156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.932+0000 7fc202438640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1fc160200 0x7fc1fc157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.932+0000 7fc202438640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc139c50 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.932+0000 7fc202438640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0c90c0 con 0x7fc1fc160200 2023-12-01T01:37:20.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.932+0000 7fc202438640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0c93a0 con 0x7fc1fc0b0f80 2023-12-01T01:37:20.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.933+0000 7fc1f3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cbd30 0x7fc1fc156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.933+0000 7fc1f3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cbd30 0x7fc1fc156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49482/0 (socket says 172.21.15.115:49482) 2023-12-01T01:37:20.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.933+0000 7fc1f3fff640 1 -- 172.21.15.115:0/817574253 learned_addr learned my addr 172.21.15.115:0/817574253 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:20.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.933+0000 7fc1f3fff640 1 -- 172.21.15.115:0/817574253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b0f80 msgr2=0x7fc1fc0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.933+0000 7fc1fbfff640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b0f80 0x7fc1fc0c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.933+0000 7fc1f3fff640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b0f80 0x7fc1fc0c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.933+0000 7fc1f3fff640 1 -- 172.21.15.115:0/817574253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1fc160200 msgr2=0x7fc1fc157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:20.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.933+0000 7fc1f3fff640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1fc160200 0x7fc1fc157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:20.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.933+0000 7fc1f3fff640 1 -- 172.21.15.115:0/817574253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40094d0 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.934+0000 7fc1f3fff640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cbd30 0x7fc1fc156cf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009870 tx=0x7fc1e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:20.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.934+0000 7fc1f97fa640 1 -- 172.21.15.115:0/817574253 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4039b60 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.934+0000 7fc1f97fa640 1 -- 172.21.15.115:0/817574253 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4039cc0 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.934+0000 7fc1f97fa640 1 -- 172.21.15.115:0/817574253 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4043630 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.934+0000 7fc202438640 1 -- 172.21.15.115:0/817574253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc0c9680 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.934+0000 7fc202438640 1 -- 172.21.15.115:0/817574253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc0c9c40 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.935+0000 7fc202438640 1 -- 172.21.15.115:0/817574253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1fc04e890 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.936+0000 7fc1f97fa640 1 -- 172.21.15.115:0/817574253 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc1e402f3b0 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.937+0000 7fc1f97fa640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1dc06a430 0x7fc1dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:20.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.937+0000 7fc1f97fa640 1 -- 172.21.15.115:0/817574253 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc1e40aaa70 con 0x7fc1fc0cbd30 2023-12-01T01:37:20.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.937+0000 7fc1fbfff640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1dc06a430 0x7fc1dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:20.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.938+0000 7fc1fbfff640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1dc06a430 0x7fc1dc06c8f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec003d90 tx=0x7fc1ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:20.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:20.942+0000 7fc1f97fa640 1 -- 172.21.15.115:0/817574253 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc1e4076280 con 0x7fc1fc0cbd30 2023-12-01T01:37:21.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.130+0000 7fc202438640 1 -- 172.21.15.115:0/817574253 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1fc0b2d10 con 0x7fc1fc0cbd30 2023-12-01T01:37:21.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.135+0000 7fc1f97fa640 1 -- 172.21.15.115:0/817574253 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1e40480a0 con 0x7fc1fc0cbd30 2023-12-01T01:37:21.136 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:37:21.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 -- 172.21.15.115:0/817574253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1dc06a430 msgr2=0x7fc1dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1dc06a430 0x7fc1dc06c8f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec003d90 tx=0x7fc1ec009040 comp rx=0 tx=0).stop 2023-12-01T01:37:21.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 -- 172.21.15.115:0/817574253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cbd30 msgr2=0x7fc1fc156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cbd30 0x7fc1fc156cf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009870 tx=0x7fc1e4004730 comp rx=0 tx=0).stop 2023-12-01T01:37:21.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 -- 172.21.15.115:0/817574253 shutdown_connections 2023-12-01T01:37:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1dc06a430 0x7fc1dc06c8f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1fc160200 0x7fc1fc157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cbd30 0x7fc1fc156cf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 --2- 172.21.15.115:0/817574253 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b0f80 0x7fc1fc0c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 -- 172.21.15.115:0/817574253 >> 172.21.15.115:0/817574253 conn(0x7fc1fc065960 msgr2=0x7fc1fc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 -- 172.21.15.115:0/817574253 shutdown_connections 2023-12-01T01:37:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.138+0000 7fc202438640 1 -- 172.21.15.115:0/817574253 wait complete. 2023-12-01T01:37:21.159 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T01:37:21.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.371+0000 7fe5dd763640 1 Processor -- start 2023-12-01T01:37:21.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.372+0000 7fe5dd763640 1 -- start start 2023-12-01T01:37:21.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.373+0000 7fe5dd763640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe5d815cd20 0x7fe5d80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.373+0000 7fe5dd763640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe5d80b5d80 0x7fe5d80b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.373+0000 7fe5dd763640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d80ce2e0 con 0x7fe5d80d1230 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.373+0000 7fe5dd763640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d80b2160 con 0x7fe5d80b5d80 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.373+0000 7fe5dd763640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d80d2a10 con 0x7fe5d815cd20 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.373+0000 7fe5d67fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe5d80b5d80 0x7fe5d80b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.373+0000 7fe5d77fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe5d80d1230 0x7fe5d80d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36076/0 (socket says 172.21.15.115:36076) 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.374+0000 7fe5d77fe640 1 -- 172.21.15.115:0/3654432694 learned_addr learned my addr 172.21.15.115:0/3654432694 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.374+0000 7fe5d6ffd640 1 --2- 172.21.15.115:0/3654432694 >> v2:172.21.15.142:3300/0 conn(0x7fe5d815cd20 0x7fe5d80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.374+0000 7fe5d67fc640 1 -- 172.21.15.115:0/3654432694 >> v2:172.21.15.142:3300/0 conn(0x7fe5d815cd20 msgr2=0x7fe5d80d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.374+0000 7fe5d67fc640 1 --2- 172.21.15.115:0/3654432694 >> v2:172.21.15.142:3300/0 conn(0x7fe5d815cd20 0x7fe5d80d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.374+0000 7fe5d67fc640 1 -- 172.21.15.115:0/3654432694 >> v1:172.21.15.115:6789/0 conn(0x7fe5d80d1230 legacy=0x7fe5d80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.374+0000 7fe5d5ffb640 1 -- 172.21.15.115:0/3654432694 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635090836 0 0) 0x7fe5d80ce2e0 con 0x7fe5d80d1230 2023-12-01T01:37:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.374+0000 7fe5d67fc640 1 -- 172.21.15.115:0/3654432694 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d80ce000 con 0x7fe5d80b5d80 2023-12-01T01:37:21.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.375+0000 7fe5d67fc640 1 --2- 172.21.15.115:0/3654432694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 0x7fe5d80b8570 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe5d808bdb0 tx=0x7fe5c802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8a1e20f75b6ebe server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:21.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.375+0000 7fe5d5ffb640 1 -- 172.21.15.115:0/3654432694 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c802cbd0 con 0x7fe5d80b5d80 2023-12-01T01:37:21.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.376+0000 7fe5d5ffb640 1 -- 172.21.15.115:0/3654432694 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c802cd30 con 0x7fe5d80b5d80 2023-12-01T01:37:21.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.376+0000 7fe5d5ffb640 1 -- 172.21.15.115:0/3654432694 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c802cea0 con 0x7fe5d80b5d80 2023-12-01T01:37:21.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.376+0000 7fe5dd763640 1 -- 172.21.15.115:0/3654432694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 msgr2=0x7fe5d80b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.376+0000 7fe5dd763640 1 --2- 172.21.15.115:0/3654432694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 0x7fe5d80b8570 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe5d808bdb0 tx=0x7fe5c802be20 comp rx=0 tx=0).stop 2023-12-01T01:37:21.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.376+0000 7fe5dd763640 1 -- 172.21.15.115:0/3654432694 shutdown_connections 2023-12-01T01:37:21.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.376+0000 7fe5dd763640 1 --2- 172.21.15.115:0/3654432694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 0x7fe5d80b8570 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.376+0000 7fe5dd763640 1 --2- 172.21.15.115:0/3654432694 >> v2:172.21.15.142:3300/0 conn(0x7fe5d815cd20 0x7fe5d80d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.376+0000 7fe5dd763640 1 -- 172.21.15.115:0/3654432694 >> 172.21.15.115:0/3654432694 conn(0x7fe5d806a0e0 msgr2=0x7fe5d806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:21.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.376+0000 7fe5dd763640 1 -- 172.21.15.115:0/3654432694 shutdown_connections 2023-12-01T01:37:21.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.376+0000 7fe5dd763640 1 -- 172.21.15.115:0/3654432694 wait complete. 2023-12-01T01:37:21.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.378+0000 7fe5dd763640 1 Processor -- start 2023-12-01T01:37:21.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.378+0000 7fe5dd763640 1 -- start start 2023-12-01T01:37:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.379+0000 7fe5dd763640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 0x7fe5d80c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.379+0000 7fe5dd763640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d80d1230 0x7fe5d80b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.379+0000 7fe5dd763640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5d815cd20 0x7fe5d80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.379+0000 7fe5dd763640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d815eb50 con 0x7fe5d80b5d80 2023-12-01T01:37:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.379+0000 7fe5dd763640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d815e260 con 0x7fe5d80d1230 2023-12-01T01:37:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.379+0000 7fe5dd763640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80c6750 con 0x7fe5d815cd20 2023-12-01T01:37:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.379+0000 7fe5d6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 0x7fe5d80c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5d6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 0x7fe5d80c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49498/0 (socket says 172.21.15.115:49498) 2023-12-01T01:37:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5d6ffd640 1 -- 172.21.15.115:0/1409396399 learned_addr learned my addr 172.21.15.115:0/1409396399 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5d6ffd640 1 -- 172.21.15.115:0/1409396399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5d815cd20 msgr2=0x7fe5d80d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5d6ffd640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5d815cd20 0x7fe5d80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5d6ffd640 1 -- 172.21.15.115:0/1409396399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d80d1230 msgr2=0x7fe5d80b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5d6ffd640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d80d1230 0x7fe5d80b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5d6ffd640 1 -- 172.21.15.115:0/1409396399 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c8009a70 con 0x7fe5d80b5d80 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5d6ffd640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 0x7fe5d80c9060 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000d5f0 tx=0x7fe5c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5cf7fe640 1 -- 172.21.15.115:0/1409396399 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00125f0 con 0x7fe5d80b5d80 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5cf7fe640 1 -- 172.21.15.115:0/1409396399 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c0012750 con 0x7fe5d80b5d80 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.380+0000 7fe5cf7fe640 1 -- 172.21.15.115:0/1409396399 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0012ae0 con 0x7fe5d80b5d80 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.381+0000 7fe5dd763640 1 -- 172.21.15.115:0/1409396399 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d815e500 con 0x7fe5d80b5d80 2023-12-01T01:37:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.381+0000 7fe5dd763640 1 -- 172.21.15.115:0/1409396399 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d81f6660 con 0x7fe5d80b5d80 2023-12-01T01:37:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.382+0000 7fe5dd763640 1 -- 172.21.15.115:0/1409396399 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d804e890 con 0x7fe5d80b5d80 2023-12-01T01:37:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.383+0000 7fe5cf7fe640 1 -- 172.21.15.115:0/1409396399 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe5c0012c40 con 0x7fe5d80b5d80 2023-12-01T01:37:21.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.384+0000 7fe5cf7fe640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5b806a430 0x7fe5b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.384+0000 7fe5cf7fe640 1 -- 172.21.15.115:0/1409396399 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe5c0088ae0 con 0x7fe5d80b5d80 2023-12-01T01:37:21.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.384+0000 7fe5d67fc640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5b806a430 0x7fe5b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:21.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.385+0000 7fe5d67fc640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5b806a430 0x7fe5b806c8f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8036000 tx=0x7fe5c802e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:21.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.389+0000 7fe5cf7fe640 1 -- 172.21.15.115:0/1409396399 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe5c0054410 con 0x7fe5d80b5d80 2023-12-01T01:37:21.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.580+0000 7fe5dd763640 1 -- 172.21.15.115:0/1409396399 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe5d80cf110 con 0x7fe5d80b5d80 2023-12-01T01:37:21.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.581+0000 7fe5cf7fe640 1 -- 172.21.15.115:0/1409396399 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe5c0057a30 con 0x7fe5d80b5d80 2023-12-01T01:37:21.582 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:21.582 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:37:21.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:21.582 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T01:37:21.583 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T01:37:21.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:21.583 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:37:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 -- 172.21.15.115:0/1409396399 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5b806a430 msgr2=0x7fe5b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5b806a430 0x7fe5b806c8f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8036000 tx=0x7fe5c802e000 comp rx=0 tx=0).stop 2023-12-01T01:37:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 -- 172.21.15.115:0/1409396399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 msgr2=0x7fe5d80c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 0x7fe5d80c9060 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000d5f0 tx=0x7fe5c000dab0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 -- 172.21.15.115:0/1409396399 shutdown_connections 2023-12-01T01:37:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5b806a430 0x7fe5b806c8f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5d815cd20 0x7fe5d80d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d80d1230 0x7fe5d80b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 --2- 172.21.15.115:0/1409396399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5d80b5d80 0x7fe5d80c9060 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 -- 172.21.15.115:0/1409396399 >> 172.21.15.115:0/1409396399 conn(0x7fe5d806a0e0 msgr2=0x7fe5d8158540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:21.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 -- 172.21.15.115:0/1409396399 shutdown_connections 2023-12-01T01:37:21.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.584+0000 7fe5dd763640 1 -- 172.21.15.115:0/1409396399 wait complete. 2023-12-01T01:37:21.603 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T01:37:21.604 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T01:37:21.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.815+0000 7f0318599640 1 Processor -- start 2023-12-01T01:37:21.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.815+0000 7f0318599640 1 -- start start 2023-12-01T01:37:21.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.816+0000 7f0318599640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f03100cb8c0 0x7f03100cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.816+0000 7f0318599640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03100b4070 0x7f03100cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.816+0000 7f0318599640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031006c860 con 0x7f031015ac70 2023-12-01T01:37:21.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.816+0000 7f0318599640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031015b050 con 0x7f03100cb8c0 2023-12-01T01:37:21.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.816+0000 7f0318599640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03100ce0b0 con 0x7f03100b4070 2023-12-01T01:37:21.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.817+0000 7f0316335640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f031015ac70 0x7f03100cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36086/0 (socket says 172.21.15.115:36086) 2023-12-01T01:37:21.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.817+0000 7f0316335640 1 -- 172.21.15.115:0/3526473883 learned_addr learned my addr 172.21.15.115:0/3526473883 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.817+0000 7f0316b36640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03100b4070 0x7f03100cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.817+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082927994 0 0) 0x7f031006c860 con 0x7f031015ac70 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.817+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02fc003520 con 0x7f031015ac70 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.817+0000 7f0315b34640 1 --2- 172.21.15.115:0/3526473883 >> v2:172.21.15.142:3300/0 conn(0x7f03100cb8c0 0x7f03100cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.817+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 734935635 0 0) 0x7f02fc003520 con 0x7f031015ac70 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03100b4070 msgr2=0x7f03100cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03100b4070 0x7f03100cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 >> v2:172.21.15.142:3300/0 conn(0x7f03100cb8c0 msgr2=0x7f03100cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 --2- 172.21.15.115:0/3526473883 >> v2:172.21.15.142:3300/0 conn(0x7f03100cb8c0 0x7f03100cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03100ce390 con 0x7f031015ac70 2023-12-01T01:37:21.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f030c001670 con 0x7f031015ac70 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0316b36640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03100b4070 0x7f03100cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 >> v1:172.21.15.115:6789/0 conn(0x7f031015ac70 legacy=0x7f03100cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02fc003970 0x7f02fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02fc0047c0 0x7f02fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03100b4070 0x7f02fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031015b050 con 0x7f02fc0047c0 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03100ce0b0 con 0x7f02fc003970 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc00b980 con 0x7f03100b4070 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315b34640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02fc0047c0 0x7f02fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0316335640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02fc003970 0x7f02fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315b34640 1 -- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03100b4070 msgr2=0x7f02fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315b34640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03100b4070 0x7f02fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315b34640 1 -- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02fc003970 msgr2=0x7f02fc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:21.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315b34640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02fc003970 0x7f02fc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.818+0000 7f0315b34640 1 -- 172.21.15.115:0/3526473883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc003520 con 0x7f02fc0047c0 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.819+0000 7f0315b34640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02fc0047c0 0x7f02fc004c40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0300009af0 tx=0x7f02fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f490a9f6bc05d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.820+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030002c710 con 0x7f02fc0047c0 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.820+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030002c870 con 0x7f02fc0047c0 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.820+0000 7f0315333640 1 -- 172.21.15.115:0/3526473883 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030002cc00 con 0x7f02fc0047c0 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.820+0000 7f0318599640 1 -- 172.21.15.115:0/3526473883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02fc0047c0 msgr2=0x7f02fc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.820+0000 7f0318599640 1 --2- 172.21.15.115:0/3526473883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02fc0047c0 0x7f02fc004c40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0300009af0 tx=0x7f02fc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.820+0000 7f0316b36640 1 -- 172.21.15.115:0/3526473883 reap_dead start 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.820+0000 7f0318599640 1 -- 172.21.15.115:0/3526473883 shutdown_connections 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.820+0000 7f0318599640 1 -- 172.21.15.115:0/3526473883 >> 172.21.15.115:0/3526473883 conn(0x7f03100662c0 msgr2=0x7f03100b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.820+0000 7f0318599640 1 -- 172.21.15.115:0/3526473883 shutdown_connections 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.820+0000 7f0318599640 1 -- 172.21.15.115:0/3526473883 wait complete. 2023-12-01T01:37:21.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.822+0000 7f0318599640 1 Processor -- start 2023-12-01T01:37:21.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.822+0000 7f0318599640 1 -- start start 2023-12-01T01:37:21.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.823+0000 7f0318599640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02fc0047c0 0x7f03101599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.823+0000 7f0318599640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03101f6940 0x7f03101f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.823+0000 7f0318599640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03100c44a0 0x7f03100c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.823+0000 7f0318599640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03100c4e60 con 0x7f03100c44a0 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.823+0000 7f0318599640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03101f79f0 con 0x7f03101f6940 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.823+0000 7f0318599640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03101f7cb0 con 0x7f02fc0047c0 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.824+0000 7f0316b36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03100c44a0 0x7f03100c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.824+0000 7f0316b36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03100c44a0 0x7f03100c4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49512/0 (socket says 172.21.15.115:49512) 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.824+0000 7f0316b36640 1 -- 172.21.15.115:0/3337540964 learned_addr learned my addr 172.21.15.115:0/3337540964 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.824+0000 7f0316b36640 1 -- 172.21.15.115:0/3337540964 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02fc0047c0 msgr2=0x7f03101599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.824+0000 7f0316b36640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02fc0047c0 0x7f03101599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.824+0000 7f0316b36640 1 -- 172.21.15.115:0/3337540964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03101f6940 msgr2=0x7f03101f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.824+0000 7f0316b36640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03101f6940 0x7f03101f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.824+0000 7f0316b36640 1 -- 172.21.15.115:0/3337540964 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03101f7430 con 0x7f03100c44a0 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.824+0000 7f0316b36640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03100c44a0 0x7f03100c4920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f031006f9a0 tx=0x7f030800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.825+0000 7f0306ffd640 1 -- 172.21.15.115:0/3337540964 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030800de90 con 0x7f03100c44a0 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.825+0000 7f0306ffd640 1 -- 172.21.15.115:0/3337540964 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030800f040 con 0x7f03100c44a0 2023-12-01T01:37:21.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.825+0000 7f0306ffd640 1 -- 172.21.15.115:0/3337540964 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308003bd0 con 0x7f03100c44a0 2023-12-01T01:37:21.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.825+0000 7f0318599640 1 -- 172.21.15.115:0/3337540964 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03101f7710 con 0x7f03100c44a0 2023-12-01T01:37:21.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.825+0000 7f0318599640 1 -- 172.21.15.115:0/3337540964 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0310159f00 con 0x7f03100c44a0 2023-12-01T01:37:21.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.826+0000 7f0318599640 1 -- 172.21.15.115:0/3337540964 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0310074210 con 0x7f03100c44a0 2023-12-01T01:37:21.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.827+0000 7f0306ffd640 1 -- 172.21.15.115:0/3337540964 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0308003d30 con 0x7f03100c44a0 2023-12-01T01:37:21.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.828+0000 7f0306ffd640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f02f406a460 0x7f02f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:21.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.828+0000 7f0306ffd640 1 -- 172.21.15.115:0/3337540964 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f03080883d0 con 0x7f03100c44a0 2023-12-01T01:37:21.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.828+0000 7f0316335640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f02f406a460 0x7f02f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:21.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.828+0000 7f0316335640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f02f406a460 0x7f02f406c920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f030c001eb0 tx=0x7f030c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:21.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:21.832+0000 7f0306ffd640 1 -- 172.21.15.115:0/3337540964 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0308053d00 con 0x7f03100c44a0 2023-12-01T01:37:22.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.022+0000 7f0318599640 1 -- 172.21.15.115:0/3337540964 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f031006cdb0 con 0x7f03100c44a0 2023-12-01T01:37:22.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.151+0000 7f0306ffd640 1 -- 172.21.15.115:0/3337540964 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0308057320 con 0x7f03100c44a0 2023-12-01T01:37:22.152 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T01:37:22.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.155+0000 7f0318599640 1 -- 172.21.15.115:0/3337540964 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f02f406a460 msgr2=0x7f02f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:22.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.155+0000 7f0318599640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f02f406a460 0x7f02f406c920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f030c001eb0 tx=0x7f030c009040 comp rx=0 tx=0).stop 2023-12-01T01:37:22.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.155+0000 7f0318599640 1 -- 172.21.15.115:0/3337540964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03100c44a0 msgr2=0x7f03100c4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:22.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.155+0000 7f0318599640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03100c44a0 0x7f03100c4920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f031006f9a0 tx=0x7f030800d4a0 comp rx=0 tx=0).stop 2023-12-01T01:37:22.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.156+0000 7f0318599640 1 -- 172.21.15.115:0/3337540964 shutdown_connections 2023-12-01T01:37:22.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.156+0000 7f0318599640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f02f406a460 0x7f02f406c920 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:22.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.156+0000 7f0318599640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03100c44a0 0x7f03100c4920 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:22.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.156+0000 7f0318599640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03101f6940 0x7f03101f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:22.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.156+0000 7f0318599640 1 --2- 172.21.15.115:0/3337540964 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02fc0047c0 0x7f03101599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:22.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.156+0000 7f0318599640 1 -- 172.21.15.115:0/3337540964 >> 172.21.15.115:0/3337540964 conn(0x7f03100662c0 msgr2=0x7f03100ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:22.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.157+0000 7f0318599640 1 -- 172.21.15.115:0/3337540964 shutdown_connections 2023-12-01T01:37:22.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.157+0000 7f0318599640 1 -- 172.21.15.115:0/3337540964 wait complete. 2023-12-01T01:37:22.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T01:37:22.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-12-01T01:37:22.371 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-12-01T01:37:22.371 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-12-01T01:37:22.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.391+0000 7f25298ee640 1 Processor -- start 2023-12-01T01:37:22.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.391+0000 7f25298ee640 1 -- start start 2023-12-01T01:37:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.392+0000 7f25298ee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 0x7f25240b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.392+0000 7f25298ee640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f25240b07f0 0x7f25240c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.393+0000 7f25298ee640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25240b2560 con 0x7f25240b1d00 2023-12-01T01:37:22.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.393+0000 7f25298ee640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25240c99d0 con 0x7f25240b07f0 2023-12-01T01:37:22.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.393+0000 7f25298ee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252415b010 con 0x7f252415ac30 2023-12-01T01:37:22.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.393+0000 7f2523fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f25240b1d00 0x7f25240b00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36098/0 (socket says 172.21.15.115:36098) 2023-12-01T01:37:22.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.393+0000 7f2523fff640 1 -- 172.21.15.115:0/1205055895 learned_addr learned my addr 172.21.15.115:0/1205055895 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:22.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.393+0000 7f25290ed640 1 --2- 172.21.15.115:0/1205055895 >> v2:172.21.15.115:3300/0 conn(0x7f25240b07f0 0x7f25240c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:22.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.393+0000 7f25288ec640 1 --2- 172.21.15.115:0/1205055895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 0x7f25240b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:22.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.393+0000 7f25237fe640 1 -- 172.21.15.115:0/1205055895 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390303522 0 0) 0x7f25240b2560 con 0x7f25240b1d00 2023-12-01T01:37:22.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.393+0000 7f25237fe640 1 -- 172.21.15.115:0/1205055895 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2514003520 con 0x7f25240b1d00 2023-12-01T01:37:22.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.394+0000 7f25290ed640 1 -- 172.21.15.115:0/1205055895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 msgr2=0x7f25240b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:22.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.394+0000 7f25290ed640 1 --2- 172.21.15.115:0/1205055895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 0x7f25240b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:22.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.394+0000 7f25290ed640 1 -- 172.21.15.115:0/1205055895 >> v1:172.21.15.115:6789/0 conn(0x7f25240b1d00 legacy=0x7f25240b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:22.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.394+0000 7f25290ed640 1 -- 172.21.15.115:0/1205055895 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25240b2280 con 0x7f25240b07f0 2023-12-01T01:37:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.395+0000 7f25290ed640 1 --2- 172.21.15.115:0/1205055895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b07f0 0x7f25240c9e60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f251c009d90 tx=0x7f251c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfde5163b9ec84e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.395+0000 7f25237fe640 1 -- 172.21.15.115:0/1205055895 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c02cbc0 con 0x7f25240b07f0 2023-12-01T01:37:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.395+0000 7f25237fe640 1 -- 172.21.15.115:0/1205055895 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c02cd20 con 0x7f25240b07f0 2023-12-01T01:37:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.395+0000 7f25237fe640 1 -- 172.21.15.115:0/1205055895 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c034660 con 0x7f25240b07f0 2023-12-01T01:37:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.396+0000 7f25298ee640 1 -- 172.21.15.115:0/1205055895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b07f0 msgr2=0x7f25240c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.396+0000 7f25298ee640 1 --2- 172.21.15.115:0/1205055895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b07f0 0x7f25240c9e60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f251c009d90 tx=0x7f251c02be10 comp rx=0 tx=0).stop 2023-12-01T01:37:22.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.396+0000 7f25298ee640 1 -- 172.21.15.115:0/1205055895 shutdown_connections 2023-12-01T01:37:22.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.396+0000 7f25298ee640 1 --2- 172.21.15.115:0/1205055895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b07f0 0x7f25240c9e60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:22.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.396+0000 7f25298ee640 1 --2- 172.21.15.115:0/1205055895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 0x7f25240b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:22.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.396+0000 7f25298ee640 1 -- 172.21.15.115:0/1205055895 >> 172.21.15.115:0/1205055895 conn(0x7f2524066400 msgr2=0x7f25240cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:22.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.396+0000 7f25298ee640 1 -- 172.21.15.115:0/1205055895 shutdown_connections 2023-12-01T01:37:22.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.396+0000 7f25298ee640 1 -- 172.21.15.115:0/1205055895 wait complete. 2023-12-01T01:37:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.397+0000 7f25298ee640 1 Processor -- start 2023-12-01T01:37:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.398+0000 7f25298ee640 1 -- start start 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.398+0000 7f25298ee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25240b07f0 0x7f25240ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f25298ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b1d00 0x7f25240cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f25298ee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 0x7f25240ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f25298ee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524148180 con 0x7f25240b1d00 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f25298ee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25240c6520 con 0x7f25240b07f0 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f25298ee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25240c6800 con 0x7f252415ac30 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f2523fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b1d00 0x7f25240cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f2523fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b1d00 0x7f25240cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49536/0 (socket says 172.21.15.115:49536) 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f2523fff640 1 -- 172.21.15.115:0/2729166533 learned_addr learned my addr 172.21.15.115:0/2729166533 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f25290ed640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 0x7f25240ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f2523fff640 1 -- 172.21.15.115:0/2729166533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 msgr2=0x7f25240ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f25288ec640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25240b07f0 0x7f25240ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f2523fff640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 0x7f25240ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:22.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f2523fff640 1 -- 172.21.15.115:0/2729166533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25240b07f0 msgr2=0x7f25240ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f2523fff640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25240b07f0 0x7f25240ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f2523fff640 1 -- 172.21.15.115:0/2729166533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c009a70 con 0x7f25240b1d00 2023-12-01T01:37:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.399+0000 7f25290ed640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 0x7f25240ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:22.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.400+0000 7f2523fff640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b1d00 0x7f25240cbcc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f251000c410 tx=0x7f251000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.400+0000 7f25217fa640 1 -- 172.21.15.115:0/2729166533 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510005ce0 con 0x7f25240b1d00 2023-12-01T01:37:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.400+0000 7f25217fa640 1 -- 172.21.15.115:0/2729166533 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2510005e40 con 0x7f25240b1d00 2023-12-01T01:37:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.400+0000 7f25217fa640 1 -- 172.21.15.115:0/2729166533 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25100072f0 con 0x7f25240b1d00 2023-12-01T01:37:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.400+0000 7f25298ee640 1 -- 172.21.15.115:0/2729166533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25240c6ae0 con 0x7f25240b1d00 2023-12-01T01:37:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.400+0000 7f25298ee640 1 -- 172.21.15.115:0/2729166533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25240c70a0 con 0x7f25240b1d00 2023-12-01T01:37:22.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.401+0000 7f25298ee640 1 -- 172.21.15.115:0/2729166533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25240c7290 con 0x7f25240b1d00 2023-12-01T01:37:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.402+0000 7f25217fa640 1 -- 172.21.15.115:0/2729166533 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2510007510 con 0x7f25240b1d00 2023-12-01T01:37:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.403+0000 7f25217fa640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250806a430 0x7f250806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.404+0000 7f25217fa640 1 -- 172.21.15.115:0/2729166533 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2510087e40 con 0x7f25240b1d00 2023-12-01T01:37:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.404+0000 7f25288ec640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250806a430 0x7f250806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:22.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.404+0000 7f25288ec640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250806a430 0x7f250806c8f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2518009970 tx=0x7f2518008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:22.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.413+0000 7f25217fa640 1 -- 172.21.15.115:0/2729166533 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2510053630 con 0x7f25240b1d00 2023-12-01T01:37:22.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:22.606+0000 7f25298ee640 1 -- 172.21.15.115:0/2729166533 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f25240618d0 con 0x7f25240b1d00 2023-12-01T01:37:23.176 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T01:37:23.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.169+0000 7f25217fa640 1 -- 172.21.15.115:0/2729166533 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7f251005be60 con 0x7f25240b1d00 2023-12-01T01:37:23.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.178+0000 7f25298ee640 1 -- 172.21.15.115:0/2729166533 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250806a430 msgr2=0x7f250806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.178+0000 7f25298ee640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250806a430 0x7f250806c8f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2518009970 tx=0x7f2518008040 comp rx=0 tx=0).stop 2023-12-01T01:37:23.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.178+0000 7f25298ee640 1 -- 172.21.15.115:0/2729166533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b1d00 msgr2=0x7f25240cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.178+0000 7f25298ee640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b1d00 0x7f25240cbcc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f251000c410 tx=0x7f251000c8d0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.179+0000 7f25298ee640 1 -- 172.21.15.115:0/2729166533 shutdown_connections 2023-12-01T01:37:23.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.179+0000 7f25298ee640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f250806a430 0x7f250806c8f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.179+0000 7f25298ee640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f252415ac30 0x7f25240ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.179+0000 7f25298ee640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25240b1d00 0x7f25240cbcc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.179+0000 7f25298ee640 1 --2- 172.21.15.115:0/2729166533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25240b07f0 0x7f25240ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.179+0000 7f25298ee640 1 -- 172.21.15.115:0/2729166533 >> 172.21.15.115:0/2729166533 conn(0x7f2524066400 msgr2=0x7f2524159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:23.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.180+0000 7f25298ee640 1 -- 172.21.15.115:0/2729166533 shutdown_connections 2023-12-01T01:37:23.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.180+0000 7f25298ee640 1 -- 172.21.15.115:0/2729166533 wait complete. 2023-12-01T01:37:23.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T01:37:23.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.399+0000 7f656f8af640 1 Processor -- start 2023-12-01T01:37:23.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.400+0000 7f656f8af640 1 -- start start 2023-12-01T01:37:23.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.401+0000 7f656f8af640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6568071740 0x7f6568070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.401+0000 7f656f8af640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f656815b200 0x7f65680ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.401+0000 7f656f8af640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656815c1f0 con 0x7f656815b6b0 2023-12-01T01:37:23.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.401+0000 7f656f8af640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65680cba40 con 0x7f656815b200 2023-12-01T01:37:23.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.401+0000 7f656f8af640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65681556f0 con 0x7f6568071740 2023-12-01T01:37:23.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.402+0000 7f656ce4a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f656815b200 0x7f65680ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:23.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.402+0000 7f656ce4a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f656815b200 0x7f65680ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49546/0 (socket says 172.21.15.115:49546) 2023-12-01T01:37:23.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.402+0000 7f656ce4a640 1 -- 172.21.15.115:0/2752966902 learned_addr learned my addr 172.21.15.115:0/2752966902 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:23.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.402+0000 7f656ce4a640 1 -- 172.21.15.115:0/2752966902 >> v2:172.21.15.142:3300/0 conn(0x7f6568071740 msgr2=0x7f6568070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:23.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.402+0000 7f656ce4a640 1 --2- 172.21.15.115:0/2752966902 >> v2:172.21.15.142:3300/0 conn(0x7f6568071740 0x7f6568070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.402+0000 7f656ce4a640 1 -- 172.21.15.115:0/2752966902 >> v1:172.21.15.142:6789/0 conn(0x7f656815b6b0 legacy=0x7f65680cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:23.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.402+0000 7f656ce4a640 1 -- 172.21.15.115:0/2752966902 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656815bc30 con 0x7f656815b200 2023-12-01T01:37:23.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.403+0000 7f656ce4a640 1 --2- 172.21.15.115:0/2752966902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b200 0x7f65680ceed0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f65680d0110 tx=0x7f655c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1f1b48dff8984 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:23.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.403+0000 7f6567fff640 1 -- 172.21.15.115:0/2752966902 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c0317e0 con 0x7f656815b200 2023-12-01T01:37:23.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.403+0000 7f6567fff640 1 -- 172.21.15.115:0/2752966902 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c031940 con 0x7f656815b200 2023-12-01T01:37:23.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.403+0000 7f6567fff640 1 -- 172.21.15.115:0/2752966902 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c031cd0 con 0x7f656815b200 2023-12-01T01:37:23.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.404+0000 7f656f8af640 1 -- 172.21.15.115:0/2752966902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b200 msgr2=0x7f65680ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.404+0000 7f656f8af640 1 --2- 172.21.15.115:0/2752966902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b200 0x7f65680ceed0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f65680d0110 tx=0x7f655c02b880 comp rx=0 tx=0).stop 2023-12-01T01:37:23.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.404+0000 7f656f8af640 1 -- 172.21.15.115:0/2752966902 shutdown_connections 2023-12-01T01:37:23.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.404+0000 7f656f8af640 1 --2- 172.21.15.115:0/2752966902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b200 0x7f65680ceed0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.404+0000 7f656f8af640 1 --2- 172.21.15.115:0/2752966902 >> v2:172.21.15.142:3300/0 conn(0x7f6568071740 0x7f6568070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.404+0000 7f656f8af640 1 -- 172.21.15.115:0/2752966902 >> 172.21.15.115:0/2752966902 conn(0x7f656806bd00 msgr2=0x7f656806c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.404+0000 7f656f8af640 1 -- 172.21.15.115:0/2752966902 shutdown_connections 2023-12-01T01:37:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.404+0000 7f656f8af640 1 -- 172.21.15.115:0/2752966902 wait complete. 2023-12-01T01:37:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.406+0000 7f656f8af640 1 Processor -- start 2023-12-01T01:37:23.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.406+0000 7f656f8af640 1 -- start start 2023-12-01T01:37:23.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.407+0000 7f656f8af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6568071740 0x7f65681e38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.407+0000 7f656f8af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656815b200 0x7f656806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.407+0000 7f656f8af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b6b0 0x7f656814a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.407+0000 7f656f8af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65681e34b0 con 0x7f656815b6b0 2023-12-01T01:37:23.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.407+0000 7f656f8af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65681e2830 con 0x7f656815b200 2023-12-01T01:37:23.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.407+0000 7f656f8af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65681e2b10 con 0x7f6568071740 2023-12-01T01:37:23.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.407+0000 7f656de4c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b6b0 0x7f656814a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:23.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.407+0000 7f656de4c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b6b0 0x7f656814a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49558/0 (socket says 172.21.15.115:49558) 2023-12-01T01:37:23.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.407+0000 7f656de4c640 1 -- 172.21.15.115:0/906064684 learned_addr learned my addr 172.21.15.115:0/906064684 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:23.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.408+0000 7f656d64b640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6568071740 0x7f65681e38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:23.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.408+0000 7f656de4c640 1 -- 172.21.15.115:0/906064684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6568071740 msgr2=0x7f65681e38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.408+0000 7f656de4c640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6568071740 0x7f65681e38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.408+0000 7f656ce4a640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656815b200 0x7f656806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.408+0000 7f656de4c640 1 -- 172.21.15.115:0/906064684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656815b200 msgr2=0x7f656806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.408+0000 7f656de4c640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656815b200 0x7f656806e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.408+0000 7f656de4c640 1 -- 172.21.15.115:0/906064684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0094d0 con 0x7f656815b6b0 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.408+0000 7f656d64b640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6568071740 0x7f65681e38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.408+0000 7f656de4c640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b6b0 0x7f656814a280 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f656000c390 tx=0x7f656000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.408+0000 7f6565ffb640 1 -- 172.21.15.115:0/906064684 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65600035f0 con 0x7f656815b6b0 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.409+0000 7f6565ffb640 1 -- 172.21.15.115:0/906064684 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6560003750 con 0x7f656815b6b0 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.409+0000 7f6565ffb640 1 -- 172.21.15.115:0/906064684 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560003b00 con 0x7f656815b6b0 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.409+0000 7f656f8af640 1 -- 172.21.15.115:0/906064684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65681e2df0 con 0x7f656815b6b0 2023-12-01T01:37:23.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.409+0000 7f656f8af640 1 -- 172.21.15.115:0/906064684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656814a7c0 con 0x7f656815b6b0 2023-12-01T01:37:23.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.410+0000 7f656f8af640 1 -- 172.21.15.115:0/906064684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65681e35f0 con 0x7f656815b6b0 2023-12-01T01:37:23.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.411+0000 7f6565ffb640 1 -- 172.21.15.115:0/906064684 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6560003c60 con 0x7f656815b6b0 2023-12-01T01:37:23.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.412+0000 7f6565ffb640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f655406a430 0x7f655406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.412+0000 7f6565ffb640 1 -- 172.21.15.115:0/906064684 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6560087a90 con 0x7f656815b6b0 2023-12-01T01:37:23.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.412+0000 7f656d64b640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f655406a430 0x7f655406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:23.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.413+0000 7f656d64b640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f655406a430 0x7f655406c8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6558007eb0 tx=0x7f65580073c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:23.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.417+0000 7f6565ffb640 1 -- 172.21.15.115:0/906064684 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f65681e35f0 con 0x7f656815b6b0 2023-12-01T01:37:23.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.643+0000 7f656f8af640 1 -- 172.21.15.115:0/906064684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f65681e35f0 con 0x7f656815b6b0 2023-12-01T01:37:23.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.653+0000 7f6565ffb640 1 -- 172.21.15.115:0/906064684 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f65681e35f0 con 0x7f656815b6b0 2023-12-01T01:37:23.654 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 4 and data pool 5 2023-12-01T01:37:23.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.659+0000 7f656f8af640 1 -- 172.21.15.115:0/906064684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f655406a430 msgr2=0x7f655406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.659+0000 7f656f8af640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f655406a430 0x7f655406c8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6558007eb0 tx=0x7f65580073c0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.659+0000 7f656f8af640 1 -- 172.21.15.115:0/906064684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b6b0 msgr2=0x7f656814a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.659+0000 7f656f8af640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b6b0 0x7f656814a280 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f656000c390 tx=0x7f656000c850 comp rx=0 tx=0).stop 2023-12-01T01:37:23.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.660+0000 7f656f8af640 1 -- 172.21.15.115:0/906064684 shutdown_connections 2023-12-01T01:37:23.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.660+0000 7f656f8af640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f655406a430 0x7f655406c8f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.660+0000 7f656f8af640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f656815b6b0 0x7f656814a280 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.660+0000 7f656f8af640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656815b200 0x7f656806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.660+0000 7f656f8af640 1 --2- 172.21.15.115:0/906064684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6568071740 0x7f65681e38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.660+0000 7f656f8af640 1 -- 172.21.15.115:0/906064684 >> 172.21.15.115:0/906064684 conn(0x7f656806bd00 msgr2=0x7f6568155da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:23.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.661+0000 7f656f8af640 1 -- 172.21.15.115:0/906064684 shutdown_connections 2023-12-01T01:37:23.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.661+0000 7f656f8af640 1 -- 172.21.15.115:0/906064684 wait complete. 2023-12-01T01:37:23.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:37:23.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.867+0000 7f025d93e640 1 Processor -- start 2023-12-01T01:37:23.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.867+0000 7f025d93e640 1 -- start start 2023-12-01T01:37:23.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.868+0000 7f025d93e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f02580b5d80 0x7f02580d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.869+0000 7f025d93e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02580ce2e0 con 0x7f025815cd20 2023-12-01T01:37:23.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.869+0000 7f025d93e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02580ce000 con 0x7f02580d1230 2023-12-01T01:37:23.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.869+0000 7f025d93e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02580b2160 con 0x7f02580b5d80 2023-12-01T01:37:23.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.869+0000 7f02567fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f02580b5d80 0x7f02580d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:23.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.869+0000 7f02567fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f02580b5d80 0x7f02580d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49572/0 (socket says 172.21.15.115:49572) 2023-12-01T01:37:23.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.869+0000 7f02567fc640 1 -- 172.21.15.115:0/2952687188 learned_addr learned my addr 172.21.15.115:0/2952687188 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:23.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.869+0000 7f02567fc640 1 -- 172.21.15.115:0/2952687188 >> v1:172.21.15.142:6789/0 conn(0x7f02580d1230 legacy=0x7f02580d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:23.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.869+0000 7f02567fc640 1 -- 172.21.15.115:0/2952687188 >> v1:172.21.15.115:6789/0 conn(0x7f025815cd20 legacy=0x7f02580b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.869+0000 7f02567fc640 1 -- 172.21.15.115:0/2952687188 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02580cd3d0 con 0x7f02580b5d80 2023-12-01T01:37:23.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.870+0000 7f02567fc640 1 --2- 172.21.15.115:0/2952687188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02580b5d80 0x7f02580d24d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f024c009870 tx=0x7f024c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f447596521e017 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:23.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.870+0000 7f0255ffb640 1 -- 172.21.15.115:0/2952687188 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c031870 con 0x7f02580b5d80 2023-12-01T01:37:23.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.871+0000 7f0255ffb640 1 -- 172.21.15.115:0/2952687188 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c0319d0 con 0x7f02580b5d80 2023-12-01T01:37:23.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.871+0000 7f0255ffb640 1 -- 172.21.15.115:0/2952687188 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c031d60 con 0x7f02580b5d80 2023-12-01T01:37:23.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.871+0000 7f025d93e640 1 -- 172.21.15.115:0/2952687188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02580b5d80 msgr2=0x7f02580d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.871+0000 7f025d93e640 1 --2- 172.21.15.115:0/2952687188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02580b5d80 0x7f02580d24d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f024c009870 tx=0x7f024c02b910 comp rx=0 tx=0).stop 2023-12-01T01:37:23.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.871+0000 7f025d93e640 1 -- 172.21.15.115:0/2952687188 shutdown_connections 2023-12-01T01:37:23.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.871+0000 7f025d93e640 1 --2- 172.21.15.115:0/2952687188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02580b5d80 0x7f02580d24d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.871+0000 7f025d93e640 1 -- 172.21.15.115:0/2952687188 >> 172.21.15.115:0/2952687188 conn(0x7f025806a0e0 msgr2=0x7f025806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:23.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.871+0000 7f025d93e640 1 -- 172.21.15.115:0/2952687188 shutdown_connections 2023-12-01T01:37:23.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.871+0000 7f025d93e640 1 -- 172.21.15.115:0/2952687188 wait complete. 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.873+0000 7f025d93e640 1 Processor -- start 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.873+0000 7f025d93e640 1 -- start start 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.874+0000 7f025d93e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02580b5d80 0x7f0258147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.874+0000 7f025d93e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02580d1230 0x7f0258149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.874+0000 7f025d93e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025815cd20 0x7f025815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.874+0000 7f025d93e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02580c9420 con 0x7f025815cd20 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.874+0000 7f025d93e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025815ec20 con 0x7f02580b5d80 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.874+0000 7f025d93e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025815e940 con 0x7f02580d1230 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.874+0000 7f02577fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025815cd20 0x7f025815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.874+0000 7f02577fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025815cd20 0x7f025815dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49580/0 (socket says 172.21.15.115:49580) 2023-12-01T01:37:23.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.874+0000 7f02577fe640 1 -- 172.21.15.115:0/1494233769 learned_addr learned my addr 172.21.15.115:0/1494233769 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f02577fe640 1 -- 172.21.15.115:0/1494233769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02580d1230 msgr2=0x7f0258149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f02567fc640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02580d1230 0x7f0258149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f02577fe640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02580d1230 0x7f0258149930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f02577fe640 1 -- 172.21.15.115:0/1494233769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02580b5d80 msgr2=0x7f0258147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f0256ffd640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02580b5d80 0x7f0258147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f02577fe640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02580b5d80 0x7f0258147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f02577fe640 1 -- 172.21.15.115:0/1494233769 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0094d0 con 0x7f025815cd20 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f02577fe640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025815cd20 0x7f025815dfd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f025000c780 tx=0x7f025000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f023ffff640 1 -- 172.21.15.115:0/1494233769 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250003d80 con 0x7f025815cd20 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f023ffff640 1 -- 172.21.15.115:0/1494233769 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025000ce90 con 0x7f025815cd20 2023-12-01T01:37:23.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f025d93e640 1 -- 172.21.15.115:0/1494233769 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02580b39c0 con 0x7f025815cd20 2023-12-01T01:37:23.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f023ffff640 1 -- 172.21.15.115:0/1494233769 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02500052b0 con 0x7f025815cd20 2023-12-01T01:37:23.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.875+0000 7f025d93e640 1 -- 172.21.15.115:0/1494233769 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02580b3f80 con 0x7f025815cd20 2023-12-01T01:37:23.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.876+0000 7f025d93e640 1 -- 172.21.15.115:0/1494233769 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02580b4170 con 0x7f025815cd20 2023-12-01T01:37:23.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.877+0000 7f023ffff640 1 -- 172.21.15.115:0/1494233769 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0250005410 con 0x7f025815cd20 2023-12-01T01:37:23.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.878+0000 7f023ffff640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f024406a460 0x7f024406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:23.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.878+0000 7f023ffff640 1 -- 172.21.15.115:0/1494233769 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0250088190 con 0x7f025815cd20 2023-12-01T01:37:23.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.878+0000 7f0256ffd640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f024406a460 0x7f024406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:23.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.879+0000 7f0256ffd640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f024406a460 0x7f024406c920 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0248000f60 tx=0x7f0248009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:23.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:23.883+0000 7f023ffff640 1 -- 172.21.15.115:0/1494233769 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0250053770 con 0x7f025815cd20 2023-12-01T01:37:24.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.072+0000 7f025d93e640 1 -- 172.21.15.115:0/1494233769 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f025804e890 con 0x7f025815cd20 2023-12-01T01:37:24.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.075+0000 7f023ffff640 1 -- 172.21.15.115:0/1494233769 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16102 (secure 0 0 0) 0x7f0250056d90 con 0x7f025815cd20 2023-12-01T01:37:24.076 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:24.076 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":34,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:37:23.644795+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:37:22.073677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-01T01:37:22.660650+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:37:24.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 -- 172.21.15.115:0/1494233769 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f024406a460 msgr2=0x7f024406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f024406a460 0x7f024406c920 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0248000f60 tx=0x7f0248009040 comp rx=0 tx=0).stop 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 -- 172.21.15.115:0/1494233769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025815cd20 msgr2=0x7f025815dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025815cd20 0x7f025815dfd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f025000c780 tx=0x7f025000cc40 comp rx=0 tx=0).stop 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 -- 172.21.15.115:0/1494233769 shutdown_connections 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f024406a460 0x7f024406c920 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02580d1230 0x7f0258149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f025815cd20 0x7f025815dfd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 --2- 172.21.15.115:0/1494233769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02580b5d80 0x7f0258147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 -- 172.21.15.115:0/1494233769 >> 172.21.15.115:0/1494233769 conn(0x7f025806a0e0 msgr2=0x7f0258158670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 -- 172.21.15.115:0/1494233769 shutdown_connections 2023-12-01T01:37:24.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.078+0000 7f025d93e640 1 -- 172.21.15.115:0/1494233769 wait complete. 2023-12-01T01:37:24.098 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:37:24.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.299+0000 7f612c9fd640 1 Processor -- start 2023-12-01T01:37:24.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.300+0000 7f612c9fd640 1 -- start start 2023-12-01T01:37:24.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.301+0000 7f612c9fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f612815e2f0 0x7f61280ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.301+0000 7f612c9fd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f61280adf10 0x7f61280caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.301+0000 7f612c9fd640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61280add10 con 0x7f612806f2f0 2023-12-01T01:37:24.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.301+0000 7f612c9fd640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61280c5760 con 0x7f61280adf10 2023-12-01T01:37:24.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.301+0000 7f612c9fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612815e6d0 con 0x7f612815e2f0 2023-12-01T01:37:24.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.302+0000 7f6126d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f61280adf10 0x7f61280caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.302+0000 7f612659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f612815e2f0 0x7f61280ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.302+0000 7f6126d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f61280adf10 0x7f61280caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53108/0 (socket says 172.21.15.115:53108) 2023-12-01T01:37:24.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.302+0000 7f612659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f612815e2f0 0x7f61280ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:54112/0 (socket says 172.21.15.115:54112) 2023-12-01T01:37:24.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.302+0000 7f6126d9d640 1 -- 172.21.15.115:0/3927712763 learned_addr learned my addr 172.21.15.115:0/3927712763 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:24.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.303+0000 7f6126d9d640 1 -- 172.21.15.115:0/3927712763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f612815e2f0 msgr2=0x7f61280ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:24.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.303+0000 7f6126d9d640 1 --2- 172.21.15.115:0/3927712763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f612815e2f0 0x7f61280ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.303+0000 7f6126d9d640 1 -- 172.21.15.115:0/3927712763 >> v1:172.21.15.142:6789/0 conn(0x7f612806f2f0 legacy=0x7f61280c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.303+0000 7f6126d9d640 1 -- 172.21.15.115:0/3927712763 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61280cf7b0 con 0x7f61280adf10 2023-12-01T01:37:24.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.303+0000 7f6125d9b640 1 -- 172.21.15.115:0/3927712763 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329688810 0 0) 0x7f61280add10 con 0x7f612806f2f0 2023-12-01T01:37:24.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.304+0000 7f6126d9d640 1 --2- 172.21.15.115:0/3927712763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61280adf10 0x7f61280caba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6120009e50 tx=0x7f612002be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f275aaaf4c5ef012 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:24.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.305+0000 7f6125d9b640 1 -- 172.21.15.115:0/3927712763 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612002c9e0 con 0x7f61280adf10 2023-12-01T01:37:24.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.305+0000 7f6125d9b640 1 -- 172.21.15.115:0/3927712763 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612002cb40 con 0x7f61280adf10 2023-12-01T01:37:24.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.305+0000 7f6125d9b640 1 -- 172.21.15.115:0/3927712763 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61200343f0 con 0x7f61280adf10 2023-12-01T01:37:24.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.305+0000 7f612c9fd640 1 -- 172.21.15.115:0/3927712763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61280adf10 msgr2=0x7f61280caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.305+0000 7f612c9fd640 1 --2- 172.21.15.115:0/3927712763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61280adf10 0x7f61280caba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6120009e50 tx=0x7f612002be40 comp rx=0 tx=0).stop 2023-12-01T01:37:24.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.305+0000 7f612c9fd640 1 -- 172.21.15.115:0/3927712763 shutdown_connections 2023-12-01T01:37:24.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.305+0000 7f612c9fd640 1 --2- 172.21.15.115:0/3927712763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61280adf10 0x7f61280caba0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.305+0000 7f612c9fd640 1 --2- 172.21.15.115:0/3927712763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f612815e2f0 0x7f61280ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.305+0000 7f612c9fd640 1 -- 172.21.15.115:0/3927712763 >> 172.21.15.115:0/3927712763 conn(0x7f61280ce880 msgr2=0x7f61280cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:24.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.306+0000 7f612c9fd640 1 -- 172.21.15.115:0/3927712763 shutdown_connections 2023-12-01T01:37:24.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.306+0000 7f612c9fd640 1 -- 172.21.15.115:0/3927712763 wait complete. 2023-12-01T01:37:24.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.308+0000 7f612c9fd640 1 Processor -- start 2023-12-01T01:37:24.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.308+0000 7f612c9fd640 1 -- start start 2023-12-01T01:37:24.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.309+0000 7f612c9fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f612806f2f0 0x7f61281581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.309+0000 7f612c9fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61280adf10 0x7f61280c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.309+0000 7f612c9fd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f612815e2f0 0x7f61280ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.309+0000 7f612c9fd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61280ccaf0 con 0x7f612806f2f0 2023-12-01T01:37:24.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.309+0000 7f612c9fd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61280c5240 con 0x7f612815e2f0 2023-12-01T01:37:24.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.309+0000 7f612c9fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61280c4c80 con 0x7f61280adf10 2023-12-01T01:37:24.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.309+0000 7f612659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f612806f2f0 0x7f61281581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.309+0000 7f612659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f612806f2f0 0x7f61281581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49584/0 (socket says 172.21.15.115:49584) 2023-12-01T01:37:24.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.309+0000 7f612659c640 1 -- 172.21.15.115:0/3144991182 learned_addr learned my addr 172.21.15.115:0/3144991182 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:24.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.309+0000 7f611ffff640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61280adf10 0x7f61280c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f6126d9d640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f612815e2f0 0x7f61280ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f612659c640 1 -- 172.21.15.115:0/3144991182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61280adf10 msgr2=0x7f61280c4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f612659c640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61280adf10 0x7f61280c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f612659c640 1 -- 172.21.15.115:0/3144991182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f612815e2f0 msgr2=0x7f61280ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f612659c640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f612815e2f0 0x7f61280ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f612659c640 1 -- 172.21.15.115:0/3144991182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6120009b30 con 0x7f612806f2f0 2023-12-01T01:37:24.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f612659c640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f612806f2f0 0x7f61281581d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f611800e870 tx=0x7f611800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:24.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f611effd640 1 -- 172.21.15.115:0/3144991182 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180094f0 con 0x7f612806f2f0 2023-12-01T01:37:24.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f611effd640 1 -- 172.21.15.115:0/3144991182 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611800c940 con 0x7f612806f2f0 2023-12-01T01:37:24.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f612c9fd640 1 -- 172.21.15.115:0/3144991182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61280c4f60 con 0x7f612806f2f0 2023-12-01T01:37:24.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f611effd640 1 -- 172.21.15.115:0/3144991182 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611800ccf0 con 0x7f612806f2f0 2023-12-01T01:37:24.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.310+0000 7f612c9fd640 1 -- 172.21.15.115:0/3144991182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61280c4870 con 0x7f612806f2f0 2023-12-01T01:37:24.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.311+0000 7f612c9fd640 1 -- 172.21.15.115:0/3144991182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612806fda0 con 0x7f612806f2f0 2023-12-01T01:37:24.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.312+0000 7f611effd640 1 -- 172.21.15.115:0/3144991182 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6118013470 con 0x7f612806f2f0 2023-12-01T01:37:24.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.313+0000 7f611effd640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f610806a430 0x7f610806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.313+0000 7f611effd640 1 -- 172.21.15.115:0/3144991182 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6118088a30 con 0x7f612806f2f0 2023-12-01T01:37:24.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.313+0000 7f611ffff640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f610806a430 0x7f610806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.314+0000 7f611ffff640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f610806a430 0x7f610806c8f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f612806c910 tx=0x7f6110009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:24.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.317+0000 7f611effd640 1 -- 172.21.15.115:0/3144991182 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6118054010 con 0x7f612806f2f0 2023-12-01T01:37:24.414 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:37:24.423 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:37:24.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.501+0000 7f612c9fd640 1 -- 172.21.15.115:0/3144991182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f612815d970 con 0x7f612806f2f0 2023-12-01T01:37:24.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.503+0000 7f611effd640 1 -- 172.21.15.115:0/3144991182 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16102 (secure 0 0 0) 0x7f6118017090 con 0x7f612806f2f0 2023-12-01T01:37:24.503 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:24.503 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":34,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:37:23.644795+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:37:22.073677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-01T01:37:22.660650+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 -- 172.21.15.115:0/3144991182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f610806a430 msgr2=0x7f610806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f610806a430 0x7f610806c8f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f612806c910 tx=0x7f6110009040 comp rx=0 tx=0).stop 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 -- 172.21.15.115:0/3144991182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f612806f2f0 msgr2=0x7f61281581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f612806f2f0 0x7f61281581d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f611800e870 tx=0x7f611800ed30 comp rx=0 tx=0).stop 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 -- 172.21.15.115:0/3144991182 shutdown_connections 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f610806a430 0x7f610806c8f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61280adf10 0x7f61280c4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f612815e2f0 0x7f61280ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 --2- 172.21.15.115:0/3144991182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f612806f2f0 0x7f61281581d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 -- 172.21.15.115:0/3144991182 >> 172.21.15.115:0/3144991182 conn(0x7f61280ce880 msgr2=0x7f61280cb680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:24.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 -- 172.21.15.115:0/3144991182 shutdown_connections 2023-12-01T01:37:24.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.505+0000 7f612c9fd640 1 -- 172.21.15.115:0/3144991182 wait complete. 2023-12-01T01:37:24.524 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T01:37:24.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.737+0000 7fe57e9c3640 1 Processor -- start 2023-12-01T01:37:24.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.737+0000 7fe57e9c3640 1 -- start start 2023-12-01T01:37:24.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.739+0000 7fe57e9c3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5780ccf80 0x7fe578159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.739+0000 7fe57e9c3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe57815abf0 0x7fe5780cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.739+0000 7fe57e9c3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57815b080 con 0x7fe578159970 2023-12-01T01:37:24.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.739+0000 7fe57e9c3640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe578070ee0 con 0x7fe57815abf0 2023-12-01T01:37:24.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.739+0000 7fe57e9c3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57815a760 con 0x7fe5780ccf80 2023-12-01T01:37:24.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.739+0000 7fe577fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe578159970 0x7fe5780707d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36118/0 (socket says 172.21.15.115:36118) 2023-12-01T01:37:24.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.739+0000 7fe577fff640 1 -- 172.21.15.115:0/1876450410 learned_addr learned my addr 172.21.15.115:0/1876450410 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:24.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.739+0000 7fe57cf60640 1 --2- 172.21.15.115:0/1876450410 >> v2:172.21.15.115:3300/0 conn(0x7fe57815abf0 0x7fe5780cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.740+0000 7fe576ffd640 1 -- 172.21.15.115:0/1876450410 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912310141 0 0) 0x7fe57815b080 con 0x7fe578159970 2023-12-01T01:37:24.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.740+0000 7fe576ffd640 1 -- 172.21.15.115:0/1876450410 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe560003520 con 0x7fe578159970 2023-12-01T01:37:24.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.740+0000 7fe5777fe640 1 --2- 172.21.15.115:0/1876450410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5780ccf80 0x7fe578159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.740+0000 7fe57cf60640 1 -- 172.21.15.115:0/1876450410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5780ccf80 msgr2=0x7fe578159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:24.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.740+0000 7fe57cf60640 1 --2- 172.21.15.115:0/1876450410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5780ccf80 0x7fe578159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.740+0000 7fe576ffd640 1 -- 172.21.15.115:0/1876450410 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473563911 0 0) 0x7fe560003520 con 0x7fe578159970 2023-12-01T01:37:24.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.740+0000 7fe57cf60640 1 -- 172.21.15.115:0/1876450410 >> v1:172.21.15.115:6789/0 conn(0x7fe578159970 legacy=0x7fe5780707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.740+0000 7fe57cf60640 1 -- 172.21.15.115:0/1876450410 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe578071100 con 0x7fe57815abf0 2023-12-01T01:37:24.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.741+0000 7fe57cf60640 1 --2- 172.21.15.115:0/1876450410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe57815abf0 0x7fe5780cb510 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe570009d90 tx=0x7fe57002bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7162da8a5db59e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:24.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.741+0000 7fe576ffd640 1 -- 172.21.15.115:0/1876450410 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568001670 con 0x7fe57815abf0 2023-12-01T01:37:24.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.741+0000 7fe576ffd640 1 -- 172.21.15.115:0/1876450410 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57002c8a0 con 0x7fe57815abf0 2023-12-01T01:37:24.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.741+0000 7fe576ffd640 1 -- 172.21.15.115:0/1876450410 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57002cc10 con 0x7fe57815abf0 2023-12-01T01:37:24.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.742+0000 7fe57e9c3640 1 -- 172.21.15.115:0/1876450410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe57815abf0 msgr2=0x7fe5780cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.742+0000 7fe57e9c3640 1 --2- 172.21.15.115:0/1876450410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe57815abf0 0x7fe5780cb510 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe570009d90 tx=0x7fe57002bd70 comp rx=0 tx=0).stop 2023-12-01T01:37:24.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.742+0000 7fe57e9c3640 1 -- 172.21.15.115:0/1876450410 shutdown_connections 2023-12-01T01:37:24.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.742+0000 7fe57e9c3640 1 --2- 172.21.15.115:0/1876450410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe57815abf0 0x7fe5780cb510 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.742+0000 7fe57e9c3640 1 --2- 172.21.15.115:0/1876450410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5780ccf80 0x7fe578159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.742+0000 7fe57e9c3640 1 -- 172.21.15.115:0/1876450410 >> 172.21.15.115:0/1876450410 conn(0x7fe578065b40 msgr2=0x7fe578065f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:24.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.742+0000 7fe57e9c3640 1 -- 172.21.15.115:0/1876450410 shutdown_connections 2023-12-01T01:37:24.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.742+0000 7fe57e9c3640 1 -- 172.21.15.115:0/1876450410 wait complete. 2023-12-01T01:37:24.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.744+0000 7fe57e9c3640 1 Processor -- start 2023-12-01T01:37:24.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.744+0000 7fe57e9c3640 1 -- start start 2023-12-01T01:37:24.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.745+0000 7fe57e9c3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5780ccf80 0x7fe5781e4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.745+0000 7fe57e9c3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe578159970 0x7fe578156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.745+0000 7fe57e9c3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe57815abf0 0x7fe5780c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.745+0000 7fe57e9c3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5781e4bc0 con 0x7fe5780ccf80 2023-12-01T01:37:24.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.745+0000 7fe57e9c3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe578156f50 con 0x7fe57815abf0 2023-12-01T01:37:24.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.745+0000 7fe57e9c3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe578157230 con 0x7fe578159970 2023-12-01T01:37:24.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.745+0000 7fe577fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5780ccf80 0x7fe5781e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.745+0000 7fe577fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5780ccf80 0x7fe5781e4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49600/0 (socket says 172.21.15.115:49600) 2023-12-01T01:37:24.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.745+0000 7fe577fff640 1 -- 172.21.15.115:0/782191354 learned_addr learned my addr 172.21.15.115:0/782191354 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:24.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe577fff640 1 -- 172.21.15.115:0/782191354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe578159970 msgr2=0x7fe578156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe5777fe640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe578159970 0x7fe578156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe577fff640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe578159970 0x7fe578156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe577fff640 1 -- 172.21.15.115:0/782191354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe57815abf0 msgr2=0x7fe5780c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:24.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe577fff640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe57815abf0 0x7fe5780c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:24.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe577fff640 1 -- 172.21.15.115:0/782191354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe570009a70 con 0x7fe5780ccf80 2023-12-01T01:37:24.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe577fff640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5780ccf80 0x7fe5781e4680 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe56800c750 tx=0x7fe56800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:24.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe574ff9640 1 -- 172.21.15.115:0/782191354 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568005ad0 con 0x7fe5780ccf80 2023-12-01T01:37:24.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe574ff9640 1 -- 172.21.15.115:0/782191354 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe568005c50 con 0x7fe5780ccf80 2023-12-01T01:37:24.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe574ff9640 1 -- 172.21.15.115:0/782191354 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568007070 con 0x7fe5780ccf80 2023-12-01T01:37:24.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe57e9c3640 1 -- 172.21.15.115:0/782191354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe578157510 con 0x7fe5780ccf80 2023-12-01T01:37:24.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.746+0000 7fe57e9c3640 1 -- 172.21.15.115:0/782191354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe578157ad0 con 0x7fe5780ccf80 2023-12-01T01:37:24.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.747+0000 7fe57e9c3640 1 -- 172.21.15.115:0/782191354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57804e890 con 0x7fe5780ccf80 2023-12-01T01:37:24.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.748+0000 7fe574ff9640 1 -- 172.21.15.115:0/782191354 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe5680071d0 con 0x7fe5780ccf80 2023-12-01T01:37:24.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.749+0000 7fe574ff9640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a430 0x7fe56406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:24.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.749+0000 7fe574ff9640 1 -- 172.21.15.115:0/782191354 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe568088690 con 0x7fe5780ccf80 2023-12-01T01:37:24.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.750+0000 7fe5777fe640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a430 0x7fe56406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:24.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.750+0000 7fe5777fe640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a430 0x7fe56406c8f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe578153b20 tx=0x7fe56c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:24.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.754+0000 7fe574ff9640 1 -- 172.21.15.115:0/782191354 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe568053c80 con 0x7fe5780ccf80 2023-12-01T01:37:24.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:24.974+0000 7fe57e9c3640 1 -- 172.21.15.115:0/782191354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe5780cd910 con 0x7fe5780ccf80 2023-12-01T01:37:25.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.667+0000 7fe574ff9640 1 -- 172.21.15.115:0/782191354 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe5680572a0 con 0x7fe5780ccf80 2023-12-01T01:37:25.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.670+0000 7fe57e9c3640 1 -- 172.21.15.115:0/782191354 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a430 msgr2=0x7fe56406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:25.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.670+0000 7fe57e9c3640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a430 0x7fe56406c8f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe578153b20 tx=0x7fe56c008040 comp rx=0 tx=0).stop 2023-12-01T01:37:25.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.670+0000 7fe57e9c3640 1 -- 172.21.15.115:0/782191354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5780ccf80 msgr2=0x7fe5781e4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:25.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.670+0000 7fe57e9c3640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5780ccf80 0x7fe5781e4680 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe56800c750 tx=0x7fe56800cc10 comp rx=0 tx=0).stop 2023-12-01T01:37:25.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.671+0000 7fe57e9c3640 1 -- 172.21.15.115:0/782191354 shutdown_connections 2023-12-01T01:37:25.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.671+0000 7fe57e9c3640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe56406a430 0x7fe56406c8f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:25.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.671+0000 7fe57e9c3640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe578159970 0x7fe578156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:25.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.671+0000 7fe57e9c3640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe57815abf0 0x7fe5780c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:25.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.671+0000 7fe57e9c3640 1 --2- 172.21.15.115:0/782191354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5780ccf80 0x7fe5781e4680 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:25.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.671+0000 7fe57e9c3640 1 -- 172.21.15.115:0/782191354 >> 172.21.15.115:0/782191354 conn(0x7fe578065b40 msgr2=0x7fe578153e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:25.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.671+0000 7fe57e9c3640 1 -- 172.21.15.115:0/782191354 shutdown_connections 2023-12-01T01:37:25.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.671+0000 7fe57e9c3640 1 -- 172.21.15.115:0/782191354 wait complete. 2023-12-01T01:37:25.692 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T01:37:25.692 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T01:37:25.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.901+0000 7f9317cce640 1 Processor -- start 2023-12-01T01:37:25.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.901+0000 7f9317cce640 1 -- start start 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.902+0000 7f9317cce640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9310142870 0x7f93100ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.902+0000 7f9317cce640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f93100cd010 0x7f93100b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9317cce640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93100b2f30 con 0x7f93100b1c50 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9317cce640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93100b3ae0 con 0x7f93100cd010 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9317cce640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93100cd4a0 con 0x7f9310142870 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9315269640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f93100cd010 0x7f93100b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9315269640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f93100cd010 0x7f93100b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49606/0 (socket says 172.21.15.115:49606) 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9315269640 1 -- 172.21.15.115:0/638945100 learned_addr learned my addr 172.21.15.115:0/638945100 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9315a6a640 1 --2- 172.21.15.115:0/638945100 >> v2:172.21.15.142:3300/0 conn(0x7f9310142870 0x7f93100ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9315269640 1 -- 172.21.15.115:0/638945100 >> v2:172.21.15.142:3300/0 conn(0x7f9310142870 msgr2=0x7f93100ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9315269640 1 --2- 172.21.15.115:0/638945100 >> v2:172.21.15.142:3300/0 conn(0x7f9310142870 0x7f93100ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9315269640 1 -- 172.21.15.115:0/638945100 >> v1:172.21.15.142:6789/0 conn(0x7f93100b1c50 legacy=0x7f931015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:25.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.903+0000 7f9315269640 1 -- 172.21.15.115:0/638945100 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93100b20c0 con 0x7f93100cd010 2023-12-01T01:37:25.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.904+0000 7f9315269640 1 --2- 172.21.15.115:0/638945100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 0x7f93100b1710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f93100b14d0 tx=0x7f930002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62b9974d823b0d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:25.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.905+0000 7f9314a68640 1 -- 172.21.15.115:0/638945100 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930002ee90 con 0x7f93100cd010 2023-12-01T01:37:25.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.905+0000 7f9314a68640 1 -- 172.21.15.115:0/638945100 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9300036440 con 0x7f93100cd010 2023-12-01T01:37:25.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.905+0000 7f9314a68640 1 -- 172.21.15.115:0/638945100 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93000367d0 con 0x7f93100cd010 2023-12-01T01:37:25.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.905+0000 7f9317cce640 1 -- 172.21.15.115:0/638945100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 msgr2=0x7f93100b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:25.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.905+0000 7f9317cce640 1 --2- 172.21.15.115:0/638945100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 0x7f93100b1710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f93100b14d0 tx=0x7f930002e0e0 comp rx=0 tx=0).stop 2023-12-01T01:37:25.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.906+0000 7f9317cce640 1 -- 172.21.15.115:0/638945100 shutdown_connections 2023-12-01T01:37:25.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.906+0000 7f9317cce640 1 --2- 172.21.15.115:0/638945100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 0x7f93100b1710 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:25.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.906+0000 7f9317cce640 1 --2- 172.21.15.115:0/638945100 >> v2:172.21.15.142:3300/0 conn(0x7f9310142870 0x7f93100ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:25.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.906+0000 7f9317cce640 1 -- 172.21.15.115:0/638945100 >> 172.21.15.115:0/638945100 conn(0x7f9310065960 msgr2=0x7f9310065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:25.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.906+0000 7f9317cce640 1 -- 172.21.15.115:0/638945100 shutdown_connections 2023-12-01T01:37:25.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.906+0000 7f9317cce640 1 -- 172.21.15.115:0/638945100 wait complete. 2023-12-01T01:37:25.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.907+0000 7f9317cce640 1 Processor -- start 2023-12-01T01:37:25.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.907+0000 7f9317cce640 1 -- start start 2023-12-01T01:37:25.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.908+0000 7f9317cce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93100b1c50 0x7f931015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:25.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.908+0000 7f9317cce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 0x7f9310159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:25.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.908+0000 7f9317cce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9310142870 0x7f931013e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:25.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.908+0000 7f9317cce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931015e080 con 0x7f93100cd010 2023-12-01T01:37:25.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.908+0000 7f9317cce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931013f510 con 0x7f9310142870 2023-12-01T01:37:25.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.908+0000 7f9317cce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931013ef50 con 0x7f93100b1c50 2023-12-01T01:37:25.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.908+0000 7f9315269640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 0x7f9310159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f9315269640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 0x7f9310159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49618/0 (socket says 172.21.15.115:49618) 2023-12-01T01:37:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f9315269640 1 -- 172.21.15.115:0/3375765299 learned_addr learned my addr 172.21.15.115:0/3375765299 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f9315269640 1 -- 172.21.15.115:0/3375765299 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93100b1c50 msgr2=0x7f931015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f9315269640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93100b1c50 0x7f931015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f9315269640 1 -- 172.21.15.115:0/3375765299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9310142870 msgr2=0x7f931013e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f931626b640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9310142870 0x7f931013e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f9315269640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9310142870 0x7f931013e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f9315269640 1 -- 172.21.15.115:0/3375765299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930000b770 con 0x7f93100cd010 2023-12-01T01:37:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f931626b640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9310142870 0x7f931013e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:25.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f9315269640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 0x7f9310159ed0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f930002e120 tx=0x7f93000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:25.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.909+0000 7f93067fc640 1 -- 172.21.15.115:0/3375765299 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300034b80 con 0x7f93100cd010 2023-12-01T01:37:25.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.910+0000 7f93067fc640 1 -- 172.21.15.115:0/3375765299 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9300036e90 con 0x7f93100cd010 2023-12-01T01:37:25.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.910+0000 7f93067fc640 1 -- 172.21.15.115:0/3375765299 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300041440 con 0x7f93100cd010 2023-12-01T01:37:25.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.910+0000 7f9317cce640 1 -- 172.21.15.115:0/3375765299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931013f230 con 0x7f93100cd010 2023-12-01T01:37:25.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.910+0000 7f9317cce640 1 -- 172.21.15.115:0/3375765299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931015c130 con 0x7f93100cd010 2023-12-01T01:37:25.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.911+0000 7f9317cce640 1 -- 172.21.15.115:0/3375765299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931015c320 con 0x7f93100cd010 2023-12-01T01:37:25.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.912+0000 7f93067fc640 1 -- 172.21.15.115:0/3375765299 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f930003d040 con 0x7f93100cd010 2023-12-01T01:37:25.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.912+0000 7f93067fc640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f92f406a430 0x7f92f406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:25.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.913+0000 7f93067fc640 1 -- 172.21.15.115:0/3375765299 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93000ad7d0 con 0x7f93100cd010 2023-12-01T01:37:25.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.913+0000 7f9315a6a640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f92f406a430 0x7f92f406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:25.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.913+0000 7f9315a6a640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f92f406a430 0x7f92f406c8f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f93100b2a80 tx=0x7f930c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:25.917+0000 7f93067fc640 1 -- 172.21.15.115:0/3375765299 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9300078db0 con 0x7f93100cd010 2023-12-01T01:37:26.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.143+0000 7f9317cce640 1 -- 172.21.15.115:0/3375765299 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f931004e890 con 0x7f93100cd010 2023-12-01T01:37:26.424 INFO:tasks.ceph.mds.a.smithi115.stderr:2023-12-01T01:37:26.424+0000 7f114e03c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:37:26.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.670+0000 7f93067fc640 1 -- 172.21.15.115:0/3375765299 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f930007c3d0 con 0x7f93100cd010 2023-12-01T01:37:26.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.674+0000 7f9317cce640 1 -- 172.21.15.115:0/3375765299 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f92f406a430 msgr2=0x7f92f406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:26.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.674+0000 7f9317cce640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f92f406a430 0x7f92f406c8f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f93100b2a80 tx=0x7f930c008040 comp rx=0 tx=0).stop 2023-12-01T01:37:26.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.674+0000 7f9317cce640 1 -- 172.21.15.115:0/3375765299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 msgr2=0x7f9310159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:26.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.674+0000 7f9317cce640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 0x7f9310159ed0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f930002e120 tx=0x7f93000046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.674+0000 7f9317cce640 1 -- 172.21.15.115:0/3375765299 shutdown_connections 2023-12-01T01:37:26.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.675+0000 7f9317cce640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f92f406a430 0x7f92f406c8f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.675+0000 7f9317cce640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9310142870 0x7f931013e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.675+0000 7f9317cce640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93100cd010 0x7f9310159ed0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.675+0000 7f9317cce640 1 --2- 172.21.15.115:0/3375765299 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93100b1c50 0x7f931015ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.675+0000 7f9317cce640 1 -- 172.21.15.115:0/3375765299 >> 172.21.15.115:0/3375765299 conn(0x7f9310065960 msgr2=0x7f9310140400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:26.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.675+0000 7f9317cce640 1 -- 172.21.15.115:0/3375765299 shutdown_connections 2023-12-01T01:37:26.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.675+0000 7f9317cce640 1 -- 172.21.15.115:0/3375765299 wait complete. 2023-12-01T01:37:26.694 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T01:37:26.694 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:37:26.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.899+0000 7f8015b00640 1 Processor -- start 2023-12-01T01:37:26.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.899+0000 7f8015b00640 1 -- start start 2023-12-01T01:37:26.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.901+0000 7f8015b00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801015e2f0 0x7f80100ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:26.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.901+0000 7f8015b00640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f801006f2f0 0x7f80100caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:26.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.901+0000 7f8015b00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f80100adf10 0x7f80100c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:26.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.901+0000 7f8015b00640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80100add10 con 0x7f801006f2f0 2023-12-01T01:37:26.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.901+0000 7f8015b00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80100c5760 con 0x7f80100adf10 2023-12-01T01:37:26.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.901+0000 7f8015b00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801015e6d0 con 0x7f801015e2f0 2023-12-01T01:37:26.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.901+0000 7f8007fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f801006f2f0 0x7f80100caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:26.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.901+0000 7f8007fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f801006f2f0 0x7f80100caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49634/0 (socket says 172.21.15.115:49634) 2023-12-01T01:37:26.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.901+0000 7f8007fff640 1 -- 172.21.15.115:0/3067916129 learned_addr learned my addr 172.21.15.115:0/3067916129 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:26.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.902+0000 7f800ffff640 1 --2- 172.21.15.115:0/3067916129 >> v2:172.21.15.142:3300/0 conn(0x7f80100adf10 0x7f80100c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:26.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.902+0000 7f800f7fe640 1 --2- 172.21.15.115:0/3067916129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801015e2f0 0x7f80100ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:26.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.902+0000 7f8007fff640 1 -- 172.21.15.115:0/3067916129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801015e2f0 msgr2=0x7f80100ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:26.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.902+0000 7f8007fff640 1 --2- 172.21.15.115:0/3067916129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801015e2f0 0x7f80100ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.902+0000 7f8007fff640 1 -- 172.21.15.115:0/3067916129 >> v2:172.21.15.142:3300/0 conn(0x7f80100adf10 msgr2=0x7f80100c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:26.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.902+0000 7f8007fff640 1 --2- 172.21.15.115:0/3067916129 >> v2:172.21.15.142:3300/0 conn(0x7f80100adf10 0x7f80100c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.902+0000 7f8007fff640 1 -- 172.21.15.115:0/3067916129 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80100cf7b0 con 0x7f801006f2f0 2023-12-01T01:37:26.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.903+0000 7f8007fff640 1 --2- 172.21.15.115:0/3067916129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801006f2f0 0x7f80100caba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f80100cb2b0 tx=0x7f7ff802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba00ae521d7574d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:26.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.903+0000 7f800effd640 1 -- 172.21.15.115:0/3067916129 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8031740 con 0x7f801006f2f0 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.903+0000 7f800effd640 1 -- 172.21.15.115:0/3067916129 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff80318a0 con 0x7f801006f2f0 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.903+0000 7f800effd640 1 -- 172.21.15.115:0/3067916129 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8031c30 con 0x7f801006f2f0 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.904+0000 7f8015b00640 1 -- 172.21.15.115:0/3067916129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801006f2f0 msgr2=0x7f80100caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.904+0000 7f8015b00640 1 --2- 172.21.15.115:0/3067916129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801006f2f0 0x7f80100caba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f80100cb2b0 tx=0x7f7ff802b7e0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.904+0000 7f8015b00640 1 -- 172.21.15.115:0/3067916129 shutdown_connections 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.904+0000 7f8015b00640 1 --2- 172.21.15.115:0/3067916129 >> v2:172.21.15.142:3300/0 conn(0x7f80100adf10 0x7f80100c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.904+0000 7f8015b00640 1 --2- 172.21.15.115:0/3067916129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801006f2f0 0x7f80100caba0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.904+0000 7f8015b00640 1 --2- 172.21.15.115:0/3067916129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801015e2f0 0x7f80100ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.904+0000 7f8015b00640 1 -- 172.21.15.115:0/3067916129 >> 172.21.15.115:0/3067916129 conn(0x7f80100ce880 msgr2=0x7f80100cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.904+0000 7f8015b00640 1 -- 172.21.15.115:0/3067916129 shutdown_connections 2023-12-01T01:37:26.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.904+0000 7f8015b00640 1 -- 172.21.15.115:0/3067916129 wait complete. 2023-12-01T01:37:26.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.906+0000 7f8015b00640 1 Processor -- start 2023-12-01T01:37:26.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.906+0000 7f8015b00640 1 -- start start 2023-12-01T01:37:26.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.907+0000 7f8015b00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f801006f2f0 0x7f8010158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:26.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.907+0000 7f8015b00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100adf10 0x7f80100c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:26.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.907+0000 7f8015b00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801015e2f0 0x7f80100c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:26.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.907+0000 7f8015b00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80101f1110 con 0x7f801015e2f0 2023-12-01T01:37:26.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.907+0000 7f8015b00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80100c60d0 con 0x7f801006f2f0 2023-12-01T01:37:26.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.907+0000 7f8015b00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80100c8860 con 0x7f80100adf10 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.907+0000 7f800ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801015e2f0 0x7f80100c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.907+0000 7f800ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801015e2f0 0x7f80100c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49636/0 (socket says 172.21.15.115:49636) 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.907+0000 7f800ffff640 1 -- 172.21.15.115:0/3836059263 learned_addr learned my addr 172.21.15.115:0/3836059263 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f800f7fe640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f801006f2f0 0x7f8010158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f8007fff640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100adf10 0x7f80100c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f800ffff640 1 -- 172.21.15.115:0/3836059263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100adf10 msgr2=0x7f80100c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f800ffff640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100adf10 0x7f80100c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f800ffff640 1 -- 172.21.15.115:0/3836059263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f801006f2f0 msgr2=0x7f8010158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f800ffff640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f801006f2f0 0x7f8010158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f800ffff640 1 -- 172.21.15.115:0/3836059263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80094d0 con 0x7f801015e2f0 2023-12-01T01:37:26.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f800ffff640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801015e2f0 0x7f80100c8320 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8008007d40 tx=0x7f800800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:26.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f800cff9640 1 -- 172.21.15.115:0/3836059263 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008014420 con 0x7f801015e2f0 2023-12-01T01:37:26.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f800cff9640 1 -- 172.21.15.115:0/3836059263 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8008014580 con 0x7f801015e2f0 2023-12-01T01:37:26.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f800cff9640 1 -- 172.21.15.115:0/3836059263 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008014930 con 0x7f801015e2f0 2023-12-01T01:37:26.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f8015b00640 1 -- 172.21.15.115:0/3836059263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80100c3890 con 0x7f801015e2f0 2023-12-01T01:37:26.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.908+0000 7f8015b00640 1 -- 172.21.15.115:0/3836059263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80101f0490 con 0x7f801015e2f0 2023-12-01T01:37:26.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.909+0000 7f8015b00640 1 -- 172.21.15.115:0/3836059263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801006fda0 con 0x7f801015e2f0 2023-12-01T01:37:26.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.911+0000 7f800cff9640 1 -- 172.21.15.115:0/3836059263 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8008014a90 con 0x7f801015e2f0 2023-12-01T01:37:26.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.911+0000 7f800cff9640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff006a430 0x7f7ff006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:26.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.912+0000 7f800cff9640 1 -- 172.21.15.115:0/3836059263 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80080891a0 con 0x7f801015e2f0 2023-12-01T01:37:26.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.912+0000 7f800f7fe640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff006a430 0x7f7ff006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:26.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.912+0000 7f800f7fe640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff006a430 0x7f7ff006c8f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8000006fd0 tx=0x7f8000008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:26.916+0000 7f800cff9640 1 -- 172.21.15.115:0/3836059263 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8008054790 con 0x7f801015e2f0 2023-12-01T01:37:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.136+0000 7f8015b00640 1 -- 172.21.15.115:0/3836059263 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80100cf4b0 con 0x7f801015e2f0 2023-12-01T01:37:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.137+0000 7f800cff9640 1 -- 172.21.15.115:0/3836059263 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f80100cf4b0 con 0x7f801015e2f0 2023-12-01T01:37:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:27.138 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1226717603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1226717603},{"type":"v1","addr":"172.21.15.142:6834","nonce":1226717603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2864659456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2864659456},{"type":"v1","addr":"172.21.15.142:6835","nonce":2864659456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4774,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/253656345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":253656345},{"type":"v1","addr":"172.21.15.115:6837","nonce":253656345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:37:23.645235+0000","modified":"2023-12-01T01:37:26.663085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5091},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5091":{"gid":5091,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6835/4242588071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4242588071},{"type":"v1","addr":"172.21.15.115:6835","nonce":4242588071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-01T01:37:27.138 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2023-12-01T01:37:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.141+0000 7f8015b00640 1 -- 172.21.15.115:0/3836059263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff006a430 msgr2=0x7f7ff006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.141+0000 7f8015b00640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff006a430 0x7f7ff006c8f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8000006fd0 tx=0x7f8000008040 comp rx=0 tx=0).stop 2023-12-01T01:37:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.141+0000 7f8015b00640 1 -- 172.21.15.115:0/3836059263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801015e2f0 msgr2=0x7f80100c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.141+0000 7f8015b00640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801015e2f0 0x7f80100c8320 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8008007d40 tx=0x7f800800e4b0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.141+0000 7f8015b00640 1 -- 172.21.15.115:0/3836059263 shutdown_connections 2023-12-01T01:37:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.141+0000 7f8015b00640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff006a430 0x7f7ff006c8f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.141+0000 7f8015b00640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80100adf10 0x7f80100c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.141+0000 7f8015b00640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f801015e2f0 0x7f80100c8320 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.141+0000 7f8015b00640 1 --2- 172.21.15.115:0/3836059263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f801006f2f0 0x7f8010158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.141+0000 7f8015b00640 1 -- 172.21.15.115:0/3836059263 >> 172.21.15.115:0/3836059263 conn(0x7f80100ce880 msgr2=0x7f801015bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.142+0000 7f8015b00640 1 -- 172.21.15.115:0/3836059263 shutdown_connections 2023-12-01T01:37:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.142+0000 7f8015b00640 1 -- 172.21.15.115:0/3836059263 wait complete. 2023-12-01T01:37:27.161 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:37:27.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.362+0000 7f6715028640 1 Processor -- start 2023-12-01T01:37:27.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.362+0000 7f6715028640 1 -- start start 2023-12-01T01:37:27.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.364+0000 7f6715028640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b5d80 0x7f67100d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.364+0000 7f6715028640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100ce2e0 con 0x7f671015cd20 2023-12-01T01:37:27.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.364+0000 7f6715028640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100ce000 con 0x7f67100d1230 2023-12-01T01:37:27.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.364+0000 7f6715028640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67100b2160 con 0x7f67100b5d80 2023-12-01T01:37:27.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.364+0000 7f670ed9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f671015cd20 0x7f67100b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36124/0 (socket says 172.21.15.115:36124) 2023-12-01T01:37:27.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.364+0000 7f670ed9d640 1 -- 172.21.15.115:0/3659656023 learned_addr learned my addr 172.21.15.115:0/3659656023 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854507356 0 0) 0x7f67100ce2e0 con 0x7f671015cd20 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66fc003520 con 0x7f671015cd20 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f6707fff640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b5d80 0x7f67100d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2131859965 0 0) 0x7f66fc003520 con 0x7f671015cd20 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b5d80 msgr2=0x7f67100d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b5d80 0x7f67100d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 >> v1:172.21.15.142:6789/0 conn(0x7f67100d1230 legacy=0x7f67100d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67100cd3d0 con 0x7f671015cd20 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6700001670 con 0x7f671015cd20 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 >> v1:172.21.15.115:6789/0 conn(0x7f671015cd20 legacy=0x7f67100b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b5d80 0x7f66fc003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66fc004070 0x7f66fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66fc004d50 0x7f66fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc00bc30 con 0x7f66fc004070 2023-12-01T01:37:27.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc00bda0 con 0x7f66fc004d50 2023-12-01T01:37:27.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.365+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc00bf60 con 0x7f67100b5d80 2023-12-01T01:37:27.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.366+0000 7f6707fff640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66fc004070 0x7f66fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.366+0000 7f670ed9d640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b5d80 0x7f66fc003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.366+0000 7f6707fff640 1 -- 172.21.15.115:0/3659656023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b5d80 msgr2=0x7f66fc003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:27.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.366+0000 7f6707fff640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b5d80 0x7f66fc003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.366+0000 7f6707fff640 1 -- 172.21.15.115:0/3659656023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66fc004d50 msgr2=0x7f66fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:27.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.366+0000 7f6707fff640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66fc004d50 0x7f66fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.366+0000 7f6707fff640 1 -- 172.21.15.115:0/3659656023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc003520 con 0x7f66fc004070 2023-12-01T01:37:27.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.367+0000 7f6707fff640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66fc004070 0x7f66fc0047c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f67100b6160 tx=0x7f66fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32e31aecec95c83 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:27.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.368+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8033680 con 0x7f66fc004070 2023-12-01T01:37:27.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.368+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f80337e0 con 0x7f66fc004070 2023-12-01T01:37:27.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.368+0000 7f670e59c640 1 -- 172.21.15.115:0/3659656023 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8033b70 con 0x7f66fc004070 2023-12-01T01:37:27.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.368+0000 7f6715028640 1 -- 172.21.15.115:0/3659656023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66fc004070 msgr2=0x7f66fc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.368+0000 7f6715028640 1 --2- 172.21.15.115:0/3659656023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66fc004070 0x7f66fc0047c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f67100b6160 tx=0x7f66fc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.368+0000 7f670f59e640 1 -- 172.21.15.115:0/3659656023 reap_dead start 2023-12-01T01:37:27.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.368+0000 7f6715028640 1 -- 172.21.15.115:0/3659656023 shutdown_connections 2023-12-01T01:37:27.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.368+0000 7f6715028640 1 -- 172.21.15.115:0/3659656023 >> 172.21.15.115:0/3659656023 conn(0x7f671006a0e0 msgr2=0x7f671006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:27.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.368+0000 7f6715028640 1 -- 172.21.15.115:0/3659656023 shutdown_connections 2023-12-01T01:37:27.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.369+0000 7f6715028640 1 -- 172.21.15.115:0/3659656023 wait complete. 2023-12-01T01:37:27.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.370+0000 7f6715028640 1 Processor -- start 2023-12-01T01:37:27.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.370+0000 7f6715028640 1 -- start start 2023-12-01T01:37:27.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.371+0000 7f6715028640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66fc004070 0x7f67100c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.371+0000 7f6715028640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b21d0 0x7f67100ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.371+0000 7f6715028640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67100b2650 0x7f67100cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.371+0000 7f6715028640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67100c9b10 con 0x7f67100b2650 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.371+0000 7f6715028640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67101f2700 con 0x7f66fc004070 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.371+0000 7f6715028640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67101f29e0 con 0x7f67100b21d0 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.371+0000 7f670f59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67100b2650 0x7f67100cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f670f59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67100b2650 0x7f67100cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49644/0 (socket says 172.21.15.115:49644) 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f670f59e640 1 -- 172.21.15.115:0/260805611 learned_addr learned my addr 172.21.15.115:0/260805611 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f670f59e640 1 -- 172.21.15.115:0/260805611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b21d0 msgr2=0x7f67100ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f670f59e640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b21d0 0x7f67100ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f670f59e640 1 -- 172.21.15.115:0/260805611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66fc004070 msgr2=0x7f67100c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f670f59e640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66fc004070 0x7f67100c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f670f59e640 1 -- 172.21.15.115:0/260805611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67101f2cc0 con 0x7f67100b2650 2023-12-01T01:37:27.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f670f59e640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67100b2650 0x7f67100cab50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f67100b1d50 tx=0x7f670800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:27.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f67077fe640 1 -- 172.21.15.115:0/260805611 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6708007070 con 0x7f67100b2650 2023-12-01T01:37:27.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f67077fe640 1 -- 172.21.15.115:0/260805611 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67080071f0 con 0x7f67100b2650 2023-12-01T01:37:27.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f67077fe640 1 -- 172.21.15.115:0/260805611 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67080075a0 con 0x7f67100b2650 2023-12-01T01:37:27.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.372+0000 7f6715028640 1 -- 172.21.15.115:0/260805611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67101f2fa0 con 0x7f67100b2650 2023-12-01T01:37:27.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.373+0000 7f6715028640 1 -- 172.21.15.115:0/260805611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67101f5c40 con 0x7f67100b2650 2023-12-01T01:37:27.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.374+0000 7f6715028640 1 -- 172.21.15.115:0/260805611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67101f5960 con 0x7f67100b2650 2023-12-01T01:37:27.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.375+0000 7f67077fe640 1 -- 172.21.15.115:0/260805611 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6708007700 con 0x7f67100b2650 2023-12-01T01:37:27.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.376+0000 7f67077fe640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66f006a460 0x7f66f006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.376+0000 7f67077fe640 1 -- 172.21.15.115:0/260805611 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6708087d90 con 0x7f67100b2650 2023-12-01T01:37:27.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.376+0000 7f670ed9d640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66f006a460 0x7f66f006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.376+0000 7f670ed9d640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66f006a460 0x7f66f006c920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6700001050 tx=0x7f6700001e30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:27.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.380+0000 7f67077fe640 1 -- 172.21.15.115:0/260805611 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f67101f5960 con 0x7f67100b2650 2023-12-01T01:37:27.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.570+0000 7f6715028640 1 -- 172.21.15.115:0/260805611 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67101f5960 con 0x7f67100b2650 2023-12-01T01:37:27.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.572+0000 7f67077fe640 1 -- 172.21.15.115:0/260805611 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f67101f5960 con 0x7f67100b2650 2023-12-01T01:37:27.573 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:27.573 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":35,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:37:24.649549+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:37:22.073677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-01T01:37:22.660650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:37:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 -- 172.21.15.115:0/260805611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66f006a460 msgr2=0x7f66f006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66f006a460 0x7f66f006c920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6700001050 tx=0x7f6700001e30 comp rx=0 tx=0).stop 2023-12-01T01:37:27.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 -- 172.21.15.115:0/260805611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67100b2650 msgr2=0x7f67100cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67100b2650 0x7f67100cab50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f67100b1d50 tx=0x7f670800c980 comp rx=0 tx=0).stop 2023-12-01T01:37:27.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 -- 172.21.15.115:0/260805611 shutdown_connections 2023-12-01T01:37:27.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66f006a460 0x7f66f006c920 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67100b21d0 0x7f67100ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67100b2650 0x7f67100cab50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 --2- 172.21.15.115:0/260805611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66fc004070 0x7f67100c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 -- 172.21.15.115:0/260805611 >> 172.21.15.115:0/260805611 conn(0x7f671006a0e0 msgr2=0x7f671006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:27.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 -- 172.21.15.115:0/260805611 shutdown_connections 2023-12-01T01:37:27.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.575+0000 7f6715028640 1 -- 172.21.15.115:0/260805611 wait complete. 2023-12-01T01:37:27.594 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T01:37:27.594 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:37:27.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.799+0000 7f7a63b16640 1 Processor -- start 2023-12-01T01:37:27.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.799+0000 7f7a63b16640 1 -- start start 2023-12-01T01:37:27.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.800+0000 7f7a63b16640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7a5c15d220 0x7f7a5c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.800+0000 7f7a63b16640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5c070ac0 0x7f7a5c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.800+0000 7f7a63b16640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c15d600 con 0x7f7a5c0ad2b0 2023-12-01T01:37:27.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.800+0000 7f7a63b16640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5c06cc10 con 0x7f7a5c15d220 2023-12-01T01:37:27.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.800+0000 7f7a63b16640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c06c720 con 0x7f7a5c070ac0 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.801+0000 7f7a610b1640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7a5c0ad2b0 0x7f7a5c06b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51914/0 (socket says 172.21.15.115:51914) 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.801+0000 7f7a610b1640 1 -- 172.21.15.115:0/1923549608 learned_addr learned my addr 172.21.15.115:0/1923549608 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.801+0000 7f7a618b2640 1 --2- 172.21.15.115:0/1923549608 >> v2:172.21.15.142:3300/0 conn(0x7f7a5c15d220 0x7f7a5c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.801+0000 7f7a620b3640 1 --2- 172.21.15.115:0/1923549608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5c070ac0 0x7f7a5c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.801+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459146397 0 0) 0x7f7a50001110 con 0x7f7a5c0ad2b0 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.801+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a48003520 con 0x7f7a5c0ad2b0 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.801+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 575200611 0 0) 0x7f7a48003520 con 0x7f7a5c0ad2b0 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5c070ac0 msgr2=0x7f7a5c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 --2- 172.21.15.115:0/1923549608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5c070ac0 0x7f7a5c0ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 >> v2:172.21.15.142:3300/0 conn(0x7f7a5c15d220 msgr2=0x7f7a5c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 --2- 172.21.15.115:0/1923549608 >> v2:172.21.15.142:3300/0 conn(0x7f7a5c15d220 0x7f7a5c0cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c06c4b0 con 0x7f7a5c0ad2b0 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f7a50001800 con 0x7f7a5c0ad2b0 2023-12-01T01:37:27.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 >> v1:172.21.15.115:6789/0 conn(0x7f7a5c0ad2b0 legacy=0x7f7a5c06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 --2- 172.21.15.115:0/1923549608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a480039e0 0x7f7a48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 --2- 172.21.15.115:0/1923549608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5c070ac0 0x7f7a480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 --2- 172.21.15.115:0/1923549608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a48004d00 0x7f7a48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c06cc10 con 0x7f7a48004d00 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c06c720 con 0x7f7a480039e0 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4800ba80 con 0x7f7a5c070ac0 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a620b3640 1 --2- 172.21.15.115:0/1923549608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a48004d00 0x7f7a48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a620b3640 1 -- 172.21.15.115:0/1923549608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5c070ac0 msgr2=0x7f7a480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a620b3640 1 --2- 172.21.15.115:0/1923549608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a5c070ac0 0x7f7a480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a620b3640 1 -- 172.21.15.115:0/1923549608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a480039e0 msgr2=0x7f7a48003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a620b3640 1 --2- 172.21.15.115:0/1923549608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a480039e0 0x7f7a48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.802+0000 7f7a620b3640 1 -- 172.21.15.115:0/1923549608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48003520 con 0x7f7a48004d00 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.803+0000 7f7a620b3640 1 --2- 172.21.15.115:0/1923549608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a48004d00 0x7f7a48005180 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009fc0 tx=0x7f7a480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7928a3dcd19b3968 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.804+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5402cc60 con 0x7f7a48004d00 2023-12-01T01:37:27.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.804+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5400f040 con 0x7f7a48004d00 2023-12-01T01:37:27.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.804+0000 7f7a608b0640 1 -- 172.21.15.115:0/1923549608 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54034440 con 0x7f7a48004d00 2023-12-01T01:37:27.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.804+0000 7f7a63b16640 1 -- 172.21.15.115:0/1923549608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a48004d00 msgr2=0x7f7a48005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.804+0000 7f7a63b16640 1 --2- 172.21.15.115:0/1923549608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a48004d00 0x7f7a48005180 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009fc0 tx=0x7f7a480057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.804+0000 7f7a620b3640 1 -- 172.21.15.115:0/1923549608 reap_dead start 2023-12-01T01:37:27.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.804+0000 7f7a63b16640 1 -- 172.21.15.115:0/1923549608 shutdown_connections 2023-12-01T01:37:27.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.804+0000 7f7a63b16640 1 -- 172.21.15.115:0/1923549608 >> 172.21.15.115:0/1923549608 conn(0x7f7a5c0ce880 msgr2=0x7f7a5c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:27.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.804+0000 7f7a63b16640 1 -- 172.21.15.115:0/1923549608 shutdown_connections 2023-12-01T01:37:27.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.804+0000 7f7a63b16640 1 -- 172.21.15.115:0/1923549608 wait complete. 2023-12-01T01:37:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.806+0000 7f7a63b16640 1 Processor -- start 2023-12-01T01:37:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.806+0000 7f7a63b16640 1 -- start start 2023-12-01T01:37:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.807+0000 7f7a63b16640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48004d00 0x7f7a5c0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.807+0000 7f7a63b16640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a5c1510f0 0x7f7a5c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.807+0000 7f7a63b16640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c151580 0x7f7a5c0c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.807+0000 7f7a63b16640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0c2dc0 con 0x7f7a5c1510f0 2023-12-01T01:37:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.807+0000 7f7a63b16640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0c9c80 con 0x7f7a5c151580 2023-12-01T01:37:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.807+0000 7f7a63b16640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0c2fd0 con 0x7f7a48004d00 2023-12-01T01:37:27.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.807+0000 7f7a610b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a5c1510f0 0x7f7a5c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.807+0000 7f7a610b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a5c1510f0 0x7f7a5c0c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56636/0 (socket says 172.21.15.115:56636) 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.807+0000 7f7a610b1640 1 -- 172.21.15.115:0/2146004728 learned_addr learned my addr 172.21.15.115:0/2146004728 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a620b3640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c151580 0x7f7a5c0c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a618b2640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48004d00 0x7f7a5c0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a610b1640 1 -- 172.21.15.115:0/2146004728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48004d00 msgr2=0x7f7a5c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a610b1640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48004d00 0x7f7a5c0c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a610b1640 1 -- 172.21.15.115:0/2146004728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c151580 msgr2=0x7f7a5c0c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a610b1640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c151580 0x7f7a5c0c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a610b1640 1 -- 172.21.15.115:0/2146004728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c0c32b0 con 0x7f7a5c1510f0 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a620b3640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c151580 0x7f7a5c0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a618b2640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48004d00 0x7f7a5c0c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a610b1640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a5c1510f0 0x7f7a5c0c8eb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000c700 tx=0x7f7a5000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a4e7fc640 1 -- 172.21.15.115:0/2146004728 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500072d0 con 0x7f7a5c1510f0 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a4e7fc640 1 -- 172.21.15.115:0/2146004728 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a50007430 con 0x7f7a5c1510f0 2023-12-01T01:37:27.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a63b16640 1 -- 172.21.15.115:0/2146004728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c0c3590 con 0x7f7a5c1510f0 2023-12-01T01:37:27.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a4e7fc640 1 -- 172.21.15.115:0/2146004728 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500047e0 con 0x7f7a5c1510f0 2023-12-01T01:37:27.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.808+0000 7f7a63b16640 1 -- 172.21.15.115:0/2146004728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c0c3d90 con 0x7f7a5c1510f0 2023-12-01T01:37:27.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.810+0000 7f7a63b16640 1 -- 172.21.15.115:0/2146004728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5c04e890 con 0x7f7a5c1510f0 2023-12-01T01:37:27.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.811+0000 7f7a4e7fc640 1 -- 172.21.15.115:0/2146004728 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7a50004940 con 0x7f7a5c1510f0 2023-12-01T01:37:27.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.811+0000 7f7a4e7fc640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a4006a460 0x7f7a4006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:27.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.811+0000 7f7a4e7fc640 1 -- 172.21.15.115:0/2146004728 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a50087e80 con 0x7f7a5c1510f0 2023-12-01T01:37:27.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.812+0000 7f7a618b2640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a4006a460 0x7f7a4006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:27.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.812+0000 7f7a618b2640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a4006a460 0x7f7a4006c920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a58006a90 tx=0x7f7a58008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:27.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.817+0000 7f7a4e7fc640 1 -- 172.21.15.115:0/2146004728 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7a500533d0 con 0x7f7a5c1510f0 2023-12-01T01:37:27.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:27.987+0000 7f7a63b16640 1 -- 172.21.15.115:0/2146004728 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a5c06ce10 con 0x7f7a4006a460 2023-12-01T01:37:28.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.000+0000 7f7a4e7fc640 1 -- 172.21.15.115:0/2146004728 <== mgr.4098 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286376 (secure 0 0 0) 0x7f7a58009db0 con 0x7f7a4006a460 2023-12-01T01:37:28.005 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:28.006 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.006+0000 7f7a63b16640 1 -- 172.21.15.115:0/2146004728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a4006a460 msgr2=0x7f7a4006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.006+0000 7f7a63b16640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a4006a460 0x7f7a4006c920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a58006a90 tx=0x7f7a58008040 comp rx=0 tx=0).stop 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.006+0000 7f7a63b16640 1 -- 172.21.15.115:0/2146004728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a5c1510f0 msgr2=0x7f7a5c0c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.006+0000 7f7a63b16640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a5c1510f0 0x7f7a5c0c8eb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000c700 tx=0x7f7a5000cbc0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.007+0000 7f7a63b16640 1 -- 172.21.15.115:0/2146004728 shutdown_connections 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.007+0000 7f7a63b16640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7a4006a460 0x7f7a4006c920 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.007+0000 7f7a63b16640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5c151580 0x7f7a5c0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.007+0000 7f7a63b16640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a5c1510f0 0x7f7a5c0c8eb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.007+0000 7f7a63b16640 1 --2- 172.21.15.115:0/2146004728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a48004d00 0x7f7a5c0c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.007+0000 7f7a63b16640 1 -- 172.21.15.115:0/2146004728 >> 172.21.15.115:0/2146004728 conn(0x7f7a5c0ce880 msgr2=0x7f7a5c153230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.007+0000 7f7a63b16640 1 -- 172.21.15.115:0/2146004728 shutdown_connections 2023-12-01T01:37:28.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.007+0000 7f7a63b16640 1 -- 172.21.15.115:0/2146004728 wait complete. 2023-12-01T01:37:28.026 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-12-01T01:37:26.179645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219832,"kb_used_data":4216,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754888,"statfs":{"total":773094113280,"available":772869005312,"internally_reserved":0,"allocated":4317184,"data_stored":2376839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031467"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.687197+0000","last_change":"2023-12-01T01:37:23.681998+0000","last_active":"2023-12-01T01:37:23.687197+0000","last_peered":"2023-12-01T01:37:23.687197+0000","last_clean":"2023-12-01T01:37:23.687197+0000","last_became_active":"2023-12-01T01:37:23.681685+0000","last_became_peered":"2023-12-01T01:37:23.681685+0000","last_unstale":"2023-12-01T01:37:23.687197+0000","last_undegraded":"2023-12-01T01:37:23.687197+0000","last_fullsized":"2023-12-01T01:37:23.687197+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:11:34.426076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.693125+0000","last_change":"2023-12-01T01:37:23.690847+0000","last_active":"2023-12-01T01:37:23.693125+0000","last_peered":"2023-12-01T01:37:23.693125+0000","last_clean":"2023-12-01T01:37:23.693125+0000","last_became_active":"2023-12-01T01:37:23.690589+0000","last_became_peered":"2023-12-01T01:37:23.690589+0000","last_unstale":"2023-12-01T01:37:23.693125+0000","last_undegraded":"2023-12-01T01:37:23.693125+0000","last_fullsized":"2023-12-01T01:37:23.693125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:23:06.622152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.680609+0000","last_change":"2023-12-01T01:37:23.679534+0000","last_active":"2023-12-01T01:37:23.680609+0000","last_peered":"2023-12-01T01:37:23.680609+0000","last_clean":"2023-12-01T01:37:23.680609+0000","last_became_active":"2023-12-01T01:37:23.678352+0000","last_became_peered":"2023-12-01T01:37:23.678352+0000","last_unstale":"2023-12-01T01:37:23.680609+0000","last_undegraded":"2023-12-01T01:37:23.680609+0000","last_fullsized":"2023-12-01T01:37:23.680609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:48.141395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.693831+0000","last_change":"2023-12-01T01:37:23.679726+0000","last_active":"2023-12-01T01:37:23.693831+0000","last_peered":"2023-12-01T01:37:23.693831+0000","last_clean":"2023-12-01T01:37:23.693831+0000","last_became_active":"2023-12-01T01:37:23.679422+0000","last_became_peered":"2023-12-01T01:37:23.679422+0000","last_unstale":"2023-12-01T01:37:23.693831+0000","last_undegraded":"2023-12-01T01:37:23.693831+0000","last_fullsized":"2023-12-01T01:37:23.693831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:12.379313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.685465+0000","last_change":"2023-12-01T01:37:23.679919+0000","last_active":"2023-12-01T01:37:23.685465+0000","last_peered":"2023-12-01T01:37:23.685465+0000","last_clean":"2023-12-01T01:37:23.685465+0000","last_became_active":"2023-12-01T01:37:23.679653+0000","last_became_peered":"2023-12-01T01:37:23.679653+0000","last_unstale":"2023-12-01T01:37:23.685465+0000","last_undegraded":"2023-12-01T01:37:23.685465+0000","last_fullsized":"2023-12-01T01:37:23.685465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:05:29.927110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.687559+0000","last_change":"2023-12-01T01:37:23.684151+0000","last_active":"2023-12-01T01:37:23.687559+0000","last_peered":"2023-12-01T01:37:23.687559+0000","last_clean":"2023-12-01T01:37:23.687559+0000","last_became_active":"2023-12-01T01:37:23.683868+0000","last_became_peered":"2023-12-01T01:37:23.683868+0000","last_unstale":"2023-12-01T01:37:23.687559+0000","last_undegraded":"2023-12-01T01:37:23.687559+0000","last_fullsized":"2023-12-01T01:37:23.687559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:31:10.954677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.683693+0000","last_change":"2023-12-01T01:37:23.678567+0000","last_active":"2023-12-01T01:37:23.683693+0000","last_peered":"2023-12-01T01:37:23.683693+0000","last_clean":"2023-12-01T01:37:23.683693+0000","last_became_active":"2023-12-01T01:37:23.678191+0000","last_became_peered":"2023-12-01T01:37:23.678191+0000","last_unstale":"2023-12-01T01:37:23.683693+0000","last_undegraded":"2023-12-01T01:37:23.683693+0000","last_fullsized":"2023-12-01T01:37:23.683693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:36:28.074355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690596+0000","last_change":"2023-12-01T01:37:23.687321+0000","last_active":"2023-12-01T01:37:23.690596+0000","last_peered":"2023-12-01T01:37:23.690596+0000","last_clean":"2023-12-01T01:37:23.690596+0000","last_became_active":"2023-12-01T01:37:23.687008+0000","last_became_peered":"2023-12-01T01:37:23.687008+0000","last_unstale":"2023-12-01T01:37:23.690596+0000","last_undegraded":"2023-12-01T01:37:23.690596+0000","last_fullsized":"2023-12-01T01:37:23.690596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:14.848033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686173+0000","last_change":"2023-12-01T01:37:23.684066+0000","last_active":"2023-12-01T01:37:23.686173+0000","last_peered":"2023-12-01T01:37:23.686173+0000","last_clean":"2023-12-01T01:37:23.686173+0000","last_became_active":"2023-12-01T01:37:23.683836+0000","last_became_peered":"2023-12-01T01:37:23.683836+0000","last_unstale":"2023-12-01T01:37:23.686173+0000","last_undegraded":"2023-12-01T01:37:23.686173+0000","last_fullsized":"2023-12-01T01:37:23.686173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:39.378914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694325+0000","last_change":"2023-12-01T01:37:23.684965+0000","last_active":"2023-12-01T01:37:23.694325+0000","last_peered":"2023-12-01T01:37:23.694325+0000","last_clean":"2023-12-01T01:37:23.694325+0000","last_became_active":"2023-12-01T01:37:23.681896+0000","last_became_peered":"2023-12-01T01:37:23.681896+0000","last_unstale":"2023-12-01T01:37:23.694325+0000","last_undegraded":"2023-12-01T01:37:23.694325+0000","last_fullsized":"2023-12-01T01:37:23.694325+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:31.630266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695741+0000","last_change":"2023-12-01T01:37:23.692712+0000","last_active":"2023-12-01T01:37:23.695741+0000","last_peered":"2023-12-01T01:37:23.695741+0000","last_clean":"2023-12-01T01:37:23.695741+0000","last_became_active":"2023-12-01T01:37:23.691965+0000","last_became_peered":"2023-12-01T01:37:23.691965+0000","last_unstale":"2023-12-01T01:37:23.695741+0000","last_undegraded":"2023-12-01T01:37:23.695741+0000","last_fullsized":"2023-12-01T01:37:23.695741+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:12.379313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.687326+0000","last_change":"2023-12-01T01:37:23.685697+0000","last_active":"2023-12-01T01:37:23.687326+0000","last_peered":"2023-12-01T01:37:23.687326+0000","last_clean":"2023-12-01T01:37:23.687326+0000","last_became_active":"2023-12-01T01:37:23.684823+0000","last_became_peered":"2023-12-01T01:37:23.684823+0000","last_unstale":"2023-12-01T01:37:23.687326+0000","last_undegraded":"2023-12-01T01:37:23.687326+0000","last_fullsized":"2023-12-01T01:37:23.687326+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:19.801685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.692944+0000","last_change":"2023-12-01T01:37:23.691511+0000","last_active":"2023-12-01T01:37:23.692944+0000","last_peered":"2023-12-01T01:37:23.692944+0000","last_clean":"2023-12-01T01:37:23.692944+0000","last_became_active":"2023-12-01T01:37:23.690500+0000","last_became_peered":"2023-12-01T01:37:23.690500+0000","last_unstale":"2023-12-01T01:37:23.692944+0000","last_undegraded":"2023-12-01T01:37:23.692944+0000","last_fullsized":"2023-12-01T01:37:23.692944+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:15.049560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694491+0000","last_change":"2023-12-01T01:37:23.687497+0000","last_active":"2023-12-01T01:37:23.694491+0000","last_peered":"2023-12-01T01:37:23.694491+0000","last_clean":"2023-12-01T01:37:23.694491+0000","last_became_active":"2023-12-01T01:37:23.687175+0000","last_became_peered":"2023-12-01T01:37:23.687175+0000","last_unstale":"2023-12-01T01:37:23.694491+0000","last_undegraded":"2023-12-01T01:37:23.694491+0000","last_fullsized":"2023-12-01T01:37:23.694491+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:36:28.074355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.683132+0000","last_change":"2023-12-01T01:37:23.679775+0000","last_active":"2023-12-01T01:37:23.683132+0000","last_peered":"2023-12-01T01:37:23.683132+0000","last_clean":"2023-12-01T01:37:23.683132+0000","last_became_active":"2023-12-01T01:37:23.679477+0000","last_became_peered":"2023-12-01T01:37:23.679477+0000","last_unstale":"2023-12-01T01:37:23.683132+0000","last_undegraded":"2023-12-01T01:37:23.683132+0000","last_fullsized":"2023-12-01T01:37:23.683132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:15.049560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686885+0000","last_change":"2023-12-01T01:37:23.684855+0000","last_active":"2023-12-01T01:37:23.686885+0000","last_peered":"2023-12-01T01:37:23.686885+0000","last_clean":"2023-12-01T01:37:23.686885+0000","last_became_active":"2023-12-01T01:37:23.684483+0000","last_became_peered":"2023-12-01T01:37:23.684483+0000","last_unstale":"2023-12-01T01:37:23.686885+0000","last_undegraded":"2023-12-01T01:37:23.686885+0000","last_fullsized":"2023-12-01T01:37:23.686885+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:52.714705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696231+0000","last_change":"2023-12-01T01:37:23.688379+0000","last_active":"2023-12-01T01:37:23.696231+0000","last_peered":"2023-12-01T01:37:23.696231+0000","last_clean":"2023-12-01T01:37:23.696231+0000","last_became_active":"2023-12-01T01:37:23.688122+0000","last_became_peered":"2023-12-01T01:37:23.688122+0000","last_unstale":"2023-12-01T01:37:23.696231+0000","last_undegraded":"2023-12-01T01:37:23.696231+0000","last_fullsized":"2023-12-01T01:37:23.696231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:33.319519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694560+0000","last_change":"2023-12-01T01:37:23.685576+0000","last_active":"2023-12-01T01:37:23.694560+0000","last_peered":"2023-12-01T01:37:23.694560+0000","last_clean":"2023-12-01T01:37:23.694560+0000","last_became_active":"2023-12-01T01:37:23.685311+0000","last_became_peered":"2023-12-01T01:37:23.685311+0000","last_unstale":"2023-12-01T01:37:23.694560+0000","last_undegraded":"2023-12-01T01:37:23.694560+0000","last_fullsized":"2023-12-01T01:37:23.694560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:29:16.659485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686317+0000","last_change":"2023-12-01T01:37:23.682949+0000","last_active":"2023-12-01T01:37:23.686317+0000","last_peered":"2023-12-01T01:37:23.686317+0000","last_clean":"2023-12-01T01:37:23.686317+0000","last_became_active":"2023-12-01T01:37:23.680770+0000","last_became_peered":"2023-12-01T01:37:23.680770+0000","last_unstale":"2023-12-01T01:37:23.686317+0000","last_undegraded":"2023-12-01T01:37:23.686317+0000","last_fullsized":"2023-12-01T01:37:23.686317+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:34:42.074507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.688125+0000","last_change":"2023-12-01T01:37:23.683476+0000","last_active":"2023-12-01T01:37:23.688125+0000","last_peered":"2023-12-01T01:37:23.688125+0000","last_clean":"2023-12-01T01:37:23.688125+0000","last_became_active":"2023-12-01T01:37:23.683195+0000","last_became_peered":"2023-12-01T01:37:23.683195+0000","last_unstale":"2023-12-01T01:37:23.688125+0000","last_undegraded":"2023-12-01T01:37:23.688125+0000","last_fullsized":"2023-12-01T01:37:23.688125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:36.901860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.682829+0000","last_change":"2023-12-01T01:37:23.678480+0000","last_active":"2023-12-01T01:37:23.682829+0000","last_peered":"2023-12-01T01:37:23.682829+0000","last_clean":"2023-12-01T01:37:23.682829+0000","last_became_active":"2023-12-01T01:37:23.675283+0000","last_became_peered":"2023-12-01T01:37:23.675283+0000","last_unstale":"2023-12-01T01:37:23.682829+0000","last_undegraded":"2023-12-01T01:37:23.682829+0000","last_fullsized":"2023-12-01T01:37:23.682829+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:31.630266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695095+0000","last_change":"2023-12-01T01:37:23.693003+0000","last_active":"2023-12-01T01:37:23.695095+0000","last_peered":"2023-12-01T01:37:23.695095+0000","last_clean":"2023-12-01T01:37:23.695095+0000","last_became_active":"2023-12-01T01:37:23.691440+0000","last_became_peered":"2023-12-01T01:37:23.691440+0000","last_unstale":"2023-12-01T01:37:23.695095+0000","last_undegraded":"2023-12-01T01:37:23.695095+0000","last_fullsized":"2023-12-01T01:37:23.695095+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:31.630266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686551+0000","last_change":"2023-12-01T01:37:23.675906+0000","last_active":"2023-12-01T01:37:23.686551+0000","last_peered":"2023-12-01T01:37:23.686551+0000","last_clean":"2023-12-01T01:37:23.686551+0000","last_became_active":"2023-12-01T01:37:23.675559+0000","last_became_peered":"2023-12-01T01:37:23.675559+0000","last_unstale":"2023-12-01T01:37:23.686551+0000","last_undegraded":"2023-12-01T01:37:23.686551+0000","last_fullsized":"2023-12-01T01:37:23.686551+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:14.848033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695139+0000","last_change":"2023-12-01T01:37:23.693336+0000","last_active":"2023-12-01T01:37:23.695139+0000","last_peered":"2023-12-01T01:37:23.695139+0000","last_clean":"2023-12-01T01:37:23.695139+0000","last_became_active":"2023-12-01T01:37:23.693101+0000","last_became_peered":"2023-12-01T01:37:23.693101+0000","last_unstale":"2023-12-01T01:37:23.695139+0000","last_undegraded":"2023-12-01T01:37:23.695139+0000","last_fullsized":"2023-12-01T01:37:23.695139+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:21.718145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694195+0000","last_change":"2023-12-01T01:37:23.681716+0000","last_active":"2023-12-01T01:37:23.694195+0000","last_peered":"2023-12-01T01:37:23.694195+0000","last_clean":"2023-12-01T01:37:23.694195+0000","last_became_active":"2023-12-01T01:37:23.681433+0000","last_became_peered":"2023-12-01T01:37:23.681433+0000","last_unstale":"2023-12-01T01:37:23.694195+0000","last_undegraded":"2023-12-01T01:37:23.694195+0000","last_fullsized":"2023-12-01T01:37:23.694195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:05:29.927110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696989+0000","last_change":"2023-12-01T01:37:23.688550+0000","last_active":"2023-12-01T01:37:23.696989+0000","last_peered":"2023-12-01T01:37:23.696989+0000","last_clean":"2023-12-01T01:37:23.696989+0000","last_became_active":"2023-12-01T01:37:23.679710+0000","last_became_peered":"2023-12-01T01:37:23.679710+0000","last_unstale":"2023-12-01T01:37:23.696989+0000","last_undegraded":"2023-12-01T01:37:23.696989+0000","last_fullsized":"2023-12-01T01:37:23.696989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:42.155049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.684087+0000","last_change":"2023-12-01T01:37:23.678830+0000","last_active":"2023-12-01T01:37:23.684087+0000","last_peered":"2023-12-01T01:37:23.684087+0000","last_clean":"2023-12-01T01:37:23.684087+0000","last_became_active":"2023-12-01T01:37:23.677197+0000","last_became_peered":"2023-12-01T01:37:23.677197+0000","last_unstale":"2023-12-01T01:37:23.684087+0000","last_undegraded":"2023-12-01T01:37:23.684087+0000","last_fullsized":"2023-12-01T01:37:23.684087+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:21.433110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690805+0000","last_change":"2023-12-01T01:37:23.689087+0000","last_active":"2023-12-01T01:37:23.690805+0000","last_peered":"2023-12-01T01:37:23.690805+0000","last_clean":"2023-12-01T01:37:23.690805+0000","last_became_active":"2023-12-01T01:37:23.688934+0000","last_became_peered":"2023-12-01T01:37:23.688934+0000","last_unstale":"2023-12-01T01:37:23.690805+0000","last_undegraded":"2023-12-01T01:37:23.690805+0000","last_fullsized":"2023-12-01T01:37:23.690805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:42.155049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.691722+0000","last_change":"2023-12-01T01:37:23.687797+0000","last_active":"2023-12-01T01:37:23.691722+0000","last_peered":"2023-12-01T01:37:23.691722+0000","last_clean":"2023-12-01T01:37:23.691722+0000","last_became_active":"2023-12-01T01:37:23.687493+0000","last_became_peered":"2023-12-01T01:37:23.687493+0000","last_unstale":"2023-12-01T01:37:23.691722+0000","last_undegraded":"2023-12-01T01:37:23.691722+0000","last_fullsized":"2023-12-01T01:37:23.691722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:58:10.711651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.692557+0000","last_change":"2023-12-01T01:37:23.688405+0000","last_active":"2023-12-01T01:37:23.692557+0000","last_peered":"2023-12-01T01:37:23.692557+0000","last_clean":"2023-12-01T01:37:23.692557+0000","last_became_active":"2023-12-01T01:37:23.688124+0000","last_became_peered":"2023-12-01T01:37:23.688124+0000","last_unstale":"2023-12-01T01:37:23.692557+0000","last_undegraded":"2023-12-01T01:37:23.692557+0000","last_fullsized":"2023-12-01T01:37:23.692557+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:14.848033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690569+0000","last_change":"2023-12-01T01:37:23.685538+0000","last_active":"2023-12-01T01:37:23.690569+0000","last_peered":"2023-12-01T01:37:23.690569+0000","last_clean":"2023-12-01T01:37:23.690569+0000","last_became_active":"2023-12-01T01:37:23.685250+0000","last_became_peered":"2023-12-01T01:37:23.685250+0000","last_unstale":"2023-12-01T01:37:23.690569+0000","last_undegraded":"2023-12-01T01:37:23.690569+0000","last_fullsized":"2023-12-01T01:37:23.690569+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:52.714705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694658+0000","last_change":"2023-12-01T01:37:23.683959+0000","last_active":"2023-12-01T01:37:23.694658+0000","last_peered":"2023-12-01T01:37:23.694658+0000","last_clean":"2023-12-01T01:37:23.694658+0000","last_became_active":"2023-12-01T01:37:23.683723+0000","last_became_peered":"2023-12-01T01:37:23.683723+0000","last_unstale":"2023-12-01T01:37:23.694658+0000","last_undegraded":"2023-12-01T01:37:23.694658+0000","last_fullsized":"2023-12-01T01:37:23.694658+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:26:37.036519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.697258+0000","last_change":"2023-12-01T01:37:23.695796+0000","last_active":"2023-12-01T01:37:23.697258+0000","last_peered":"2023-12-01T01:37:23.697258+0000","last_clean":"2023-12-01T01:37:23.697258+0000","last_became_active":"2023-12-01T01:37:23.695419+0000","last_became_peered":"2023-12-01T01:37:23.695419+0000","last_unstale":"2023-12-01T01:37:23.697258+0000","last_undegraded":"2023-12-01T01:37:23.697258+0000","last_fullsized":"2023-12-01T01:37:23.697258+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:11:34.426076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.691709+0000","last_change":"2023-12-01T01:37:23.197332+0000","last_active":"2023-12-01T01:37:23.691709+0000","last_peered":"2023-12-01T01:37:23.691709+0000","last_clean":"2023-12-01T01:37:23.691709+0000","last_became_active":"2023-12-01T01:37:23.197041+0000","last_became_peered":"2023-12-01T01:37:23.197041+0000","last_unstale":"2023-12-01T01:37:23.691709+0000","last_undegraded":"2023-12-01T01:37:23.691709+0000","last_fullsized":"2023-12-01T01:37:23.691709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:41.139273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690576+0000","last_change":"2023-12-01T01:37:23.687315+0000","last_active":"2023-12-01T01:37:23.690576+0000","last_peered":"2023-12-01T01:37:23.690576+0000","last_clean":"2023-12-01T01:37:23.690576+0000","last_became_active":"2023-12-01T01:37:23.687040+0000","last_became_peered":"2023-12-01T01:37:23.687040+0000","last_unstale":"2023-12-01T01:37:23.690576+0000","last_undegraded":"2023-12-01T01:37:23.690576+0000","last_fullsized":"2023-12-01T01:37:23.690576+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:47.357095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.693966+0000","last_change":"2023-12-01T01:37:23.193859+0000","last_active":"2023-12-01T01:37:23.693966+0000","last_peered":"2023-12-01T01:37:23.693966+0000","last_clean":"2023-12-01T01:37:23.693966+0000","last_became_active":"2023-12-01T01:37:23.193204+0000","last_became_peered":"2023-12-01T01:37:23.193204+0000","last_unstale":"2023-12-01T01:37:23.693966+0000","last_undegraded":"2023-12-01T01:37:23.693966+0000","last_fullsized":"2023-12-01T01:37:23.693966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:38.363139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695223+0000","last_change":"2023-12-01T01:37:23.693532+0000","last_active":"2023-12-01T01:37:23.695223+0000","last_peered":"2023-12-01T01:37:23.695223+0000","last_clean":"2023-12-01T01:37:23.695223+0000","last_became_active":"2023-12-01T01:37:23.693308+0000","last_became_peered":"2023-12-01T01:37:23.693308+0000","last_unstale":"2023-12-01T01:37:23.695223+0000","last_undegraded":"2023-12-01T01:37:23.695223+0000","last_fullsized":"2023-12-01T01:37:23.695223+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:21.433110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.688636+0000","last_change":"2023-12-01T01:37:23.203553+0000","last_active":"2023-12-01T01:37:23.688636+0000","last_peered":"2023-12-01T01:37:23.688636+0000","last_clean":"2023-12-01T01:37:23.688636+0000","last_became_active":"2023-12-01T01:37:23.202873+0000","last_became_peered":"2023-12-01T01:37:23.202873+0000","last_unstale":"2023-12-01T01:37:23.688636+0000","last_undegraded":"2023-12-01T01:37:23.688636+0000","last_fullsized":"2023-12-01T01:37:23.688636+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:53:05.658395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694284+0000","last_change":"2023-12-01T01:37:23.680504+0000","last_active":"2023-12-01T01:37:23.694284+0000","last_peered":"2023-12-01T01:37:23.694284+0000","last_clean":"2023-12-01T01:37:23.694284+0000","last_became_active":"2023-12-01T01:37:23.680241+0000","last_became_peered":"2023-12-01T01:37:23.680241+0000","last_unstale":"2023-12-01T01:37:23.694284+0000","last_undegraded":"2023-12-01T01:37:23.694284+0000","last_fullsized":"2023-12-01T01:37:23.694284+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:31.615342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690261+0000","last_change":"2023-12-01T01:37:23.198906+0000","last_active":"2023-12-01T01:37:23.690261+0000","last_peered":"2023-12-01T01:37:23.690261+0000","last_clean":"2023-12-01T01:37:23.690261+0000","last_became_active":"2023-12-01T01:37:23.198493+0000","last_became_peered":"2023-12-01T01:37:23.198493+0000","last_unstale":"2023-12-01T01:37:23.690261+0000","last_undegraded":"2023-12-01T01:37:23.690261+0000","last_fullsized":"2023-12-01T01:37:23.690261+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:42:03.471777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690176+0000","last_change":"2023-12-01T01:37:23.688239+0000","last_active":"2023-12-01T01:37:23.690176+0000","last_peered":"2023-12-01T01:37:23.690176+0000","last_clean":"2023-12-01T01:37:23.690176+0000","last_became_active":"2023-12-01T01:37:23.687947+0000","last_became_peered":"2023-12-01T01:37:23.687947+0000","last_unstale":"2023-12-01T01:37:23.690176+0000","last_undegraded":"2023-12-01T01:37:23.690176+0000","last_fullsized":"2023-12-01T01:37:23.690176+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:31:10.954677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.684827+0000","last_change":"2023-12-01T01:37:23.191336+0000","last_active":"2023-12-01T01:37:23.684827+0000","last_peered":"2023-12-01T01:37:23.684827+0000","last_clean":"2023-12-01T01:37:23.684827+0000","last_became_active":"2023-12-01T01:37:23.190957+0000","last_became_peered":"2023-12-01T01:37:23.190957+0000","last_unstale":"2023-12-01T01:37:23.684827+0000","last_undegraded":"2023-12-01T01:37:23.684827+0000","last_fullsized":"2023-12-01T01:37:23.684827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:53:05.658395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.691021+0000","last_change":"2023-12-01T01:37:23.689589+0000","last_active":"2023-12-01T01:37:23.691021+0000","last_peered":"2023-12-01T01:37:23.691021+0000","last_clean":"2023-12-01T01:37:23.691021+0000","last_became_active":"2023-12-01T01:37:23.689229+0000","last_became_peered":"2023-12-01T01:37:23.689229+0000","last_unstale":"2023-12-01T01:37:23.691021+0000","last_undegraded":"2023-12-01T01:37:23.691021+0000","last_fullsized":"2023-12-01T01:37:23.691021+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:15.049560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686895+0000","last_change":"2023-12-01T01:37:23.199499+0000","last_active":"2023-12-01T01:37:23.686895+0000","last_peered":"2023-12-01T01:37:23.686895+0000","last_clean":"2023-12-01T01:37:23.686895+0000","last_became_active":"2023-12-01T01:37:23.199243+0000","last_became_peered":"2023-12-01T01:37:23.199243+0000","last_unstale":"2023-12-01T01:37:23.686895+0000","last_undegraded":"2023-12-01T01:37:23.686895+0000","last_fullsized":"2023-12-01T01:37:23.686895+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:42.327013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.667003+0000","last_change":"2023-12-01T01:37:23.663674+0000","last_active":"2023-12-01T01:37:23.667003+0000","last_peered":"2023-12-01T01:37:23.667003+0000","last_clean":"2023-12-01T01:37:23.667003+0000","last_became_active":"2023-12-01T01:37:23.663490+0000","last_became_peered":"2023-12-01T01:37:23.663490+0000","last_unstale":"2023-12-01T01:37:23.667003+0000","last_undegraded":"2023-12-01T01:37:23.667003+0000","last_fullsized":"2023-12-01T01:37:23.667003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:43.342788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.665588+0000","last_change":"2023-12-01T01:37:23.202531+0000","last_active":"2023-12-01T01:37:23.665588+0000","last_peered":"2023-12-01T01:37:23.665588+0000","last_clean":"2023-12-01T01:37:23.665588+0000","last_became_active":"2023-12-01T01:37:23.202242+0000","last_became_peered":"2023-12-01T01:37:23.202242+0000","last_unstale":"2023-12-01T01:37:23.665588+0000","last_undegraded":"2023-12-01T01:37:23.665588+0000","last_fullsized":"2023-12-01T01:37:23.665588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:07.036395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694363+0000","last_change":"2023-12-01T01:37:23.685915+0000","last_active":"2023-12-01T01:37:23.694363+0000","last_peered":"2023-12-01T01:37:23.694363+0000","last_clean":"2023-12-01T01:37:23.694363+0000","last_became_active":"2023-12-01T01:37:23.682054+0000","last_became_peered":"2023-12-01T01:37:23.682054+0000","last_unstale":"2023-12-01T01:37:23.694363+0000","last_undegraded":"2023-12-01T01:37:23.694363+0000","last_fullsized":"2023-12-01T01:37:23.694363+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:21.433110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672282+0000","last_change":"2023-12-01T01:37:23.193655+0000","last_active":"2023-12-01T01:37:23.672282+0000","last_peered":"2023-12-01T01:37:23.672282+0000","last_clean":"2023-12-01T01:37:23.672282+0000","last_became_active":"2023-12-01T01:37:23.193348+0000","last_became_peered":"2023-12-01T01:37:23.193348+0000","last_unstale":"2023-12-01T01:37:23.672282+0000","last_undegraded":"2023-12-01T01:37:23.672282+0000","last_fullsized":"2023-12-01T01:37:23.672282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:05:30.626410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695671+0000","last_change":"2023-12-01T01:37:23.692414+0000","last_active":"2023-12-01T01:37:23.695671+0000","last_peered":"2023-12-01T01:37:23.695671+0000","last_clean":"2023-12-01T01:37:23.695671+0000","last_became_active":"2023-12-01T01:37:23.688590+0000","last_became_peered":"2023-12-01T01:37:23.688590+0000","last_unstale":"2023-12-01T01:37:23.695671+0000","last_undegraded":"2023-12-01T01:37:23.695671+0000","last_fullsized":"2023-12-01T01:37:23.695671+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:11:26.830819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.667074+0000","last_change":"2023-12-01T01:37:23.201058+0000","last_active":"2023-12-01T01:37:23.667074+0000","last_peered":"2023-12-01T01:37:23.667074+0000","last_clean":"2023-12-01T01:37:23.667074+0000","last_became_active":"2023-12-01T01:37:23.200513+0000","last_became_peered":"2023-12-01T01:37:23.200513+0000","last_unstale":"2023-12-01T01:37:23.667074+0000","last_undegraded":"2023-12-01T01:37:23.667074+0000","last_fullsized":"2023-12-01T01:37:23.667074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:00:19.556191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694180+0000","last_change":"2023-12-01T01:37:23.681615+0000","last_active":"2023-12-01T01:37:23.694180+0000","last_peered":"2023-12-01T01:37:23.694180+0000","last_clean":"2023-12-01T01:37:23.694180+0000","last_became_active":"2023-12-01T01:37:23.681319+0000","last_became_peered":"2023-12-01T01:37:23.681319+0000","last_unstale":"2023-12-01T01:37:23.694180+0000","last_undegraded":"2023-12-01T01:37:23.694180+0000","last_fullsized":"2023-12-01T01:37:23.694180+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:34.711426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672324+0000","last_change":"2023-12-01T01:37:23.201065+0000","last_active":"2023-12-01T01:37:23.672324+0000","last_peered":"2023-12-01T01:37:23.672324+0000","last_clean":"2023-12-01T01:37:23.672324+0000","last_became_active":"2023-12-01T01:37:23.200778+0000","last_became_peered":"2023-12-01T01:37:23.200778+0000","last_unstale":"2023-12-01T01:37:23.672324+0000","last_undegraded":"2023-12-01T01:37:23.672324+0000","last_fullsized":"2023-12-01T01:37:23.672324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:46.341319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694439+0000","last_change":"2023-12-01T01:37:23.685634+0000","last_active":"2023-12-01T01:37:23.694439+0000","last_peered":"2023-12-01T01:37:23.694439+0000","last_clean":"2023-12-01T01:37:23.694439+0000","last_became_active":"2023-12-01T01:37:23.682383+0000","last_became_peered":"2023-12-01T01:37:23.682383+0000","last_unstale":"2023-12-01T01:37:23.694439+0000","last_undegraded":"2023-12-01T01:37:23.694439+0000","last_fullsized":"2023-12-01T01:37:23.694439+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:27:09.123090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.692818+0000","last_change":"2023-12-01T01:37:23.192448+0000","last_active":"2023-12-01T01:37:23.692818+0000","last_peered":"2023-12-01T01:37:23.692818+0000","last_clean":"2023-12-01T01:37:23.692818+0000","last_became_active":"2023-12-01T01:37:23.192188+0000","last_became_peered":"2023-12-01T01:37:23.192188+0000","last_unstale":"2023-12-01T01:37:23.692818+0000","last_undegraded":"2023-12-01T01:37:23.692818+0000","last_fullsized":"2023-12-01T01:37:23.692818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:36:27.058579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695725+0000","last_change":"2023-12-01T01:37:23.692604+0000","last_active":"2023-12-01T01:37:23.695725+0000","last_peered":"2023-12-01T01:37:23.695725+0000","last_clean":"2023-12-01T01:37:23.695725+0000","last_became_active":"2023-12-01T01:37:23.692321+0000","last_became_peered":"2023-12-01T01:37:23.692321+0000","last_unstale":"2023-12-01T01:37:23.695725+0000","last_undegraded":"2023-12-01T01:37:23.695725+0000","last_fullsized":"2023-12-01T01:37:23.695725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:02:54.947127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.699611+0000","last_change":"2023-12-01T01:37:23.199582+0000","last_active":"2023-12-01T01:37:23.699611+0000","last_peered":"2023-12-01T01:37:23.699611+0000","last_clean":"2023-12-01T01:37:23.699611+0000","last_became_active":"2023-12-01T01:37:23.199029+0000","last_became_peered":"2023-12-01T01:37:23.199029+0000","last_unstale":"2023-12-01T01:37:23.699611+0000","last_undegraded":"2023-12-01T01:37:23.699611+0000","last_fullsized":"2023-12-01T01:37:23.699611+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:31:09.938902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686575+0000","last_change":"2023-12-01T01:37:23.677735+0000","last_active":"2023-12-01T01:37:23.686575+0000","last_peered":"2023-12-01T01:37:23.686575+0000","last_clean":"2023-12-01T01:37:23.686575+0000","last_became_active":"2023-12-01T01:37:23.677447+0000","last_became_peered":"2023-12-01T01:37:23.677447+0000","last_unstale":"2023-12-01T01:37:23.686575+0000","last_undegraded":"2023-12-01T01:37:23.686575+0000","last_fullsized":"2023-12-01T01:37:23.686575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:52.714705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.693853+0000","last_change":"2023-12-01T01:37:23.192003+0000","last_active":"2023-12-01T01:37:23.693853+0000","last_peered":"2023-12-01T01:37:23.693853+0000","last_clean":"2023-12-01T01:37:23.693853+0000","last_became_active":"2023-12-01T01:37:23.191287+0000","last_became_peered":"2023-12-01T01:37:23.191287+0000","last_unstale":"2023-12-01T01:37:23.693853+0000","last_undegraded":"2023-12-01T01:37:23.693853+0000","last_fullsized":"2023-12-01T01:37:23.693853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:44.390008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696982+0000","last_change":"2023-12-01T01:37:23.689085+0000","last_active":"2023-12-01T01:37:23.696982+0000","last_peered":"2023-12-01T01:37:23.696982+0000","last_clean":"2023-12-01T01:37:23.696982+0000","last_became_active":"2023-12-01T01:37:23.688533+0000","last_became_peered":"2023-12-01T01:37:23.688533+0000","last_unstale":"2023-12-01T01:37:23.696982+0000","last_undegraded":"2023-12-01T01:37:23.696982+0000","last_fullsized":"2023-12-01T01:37:23.696982+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:26:37.036519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672237+0000","last_change":"2023-12-01T01:37:23.192431+0000","last_active":"2023-12-01T01:37:23.672237+0000","last_peered":"2023-12-01T01:37:23.672237+0000","last_clean":"2023-12-01T01:37:23.672237+0000","last_became_active":"2023-12-01T01:37:23.192084+0000","last_became_peered":"2023-12-01T01:37:23.192084+0000","last_unstale":"2023-12-01T01:37:23.672237+0000","last_undegraded":"2023-12-01T01:37:23.672237+0000","last_fullsized":"2023-12-01T01:37:23.672237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:31:09.938902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.683669+0000","last_change":"2023-12-01T01:37:23.679748+0000","last_active":"2023-12-01T01:37:23.683669+0000","last_peered":"2023-12-01T01:37:23.683669+0000","last_clean":"2023-12-01T01:37:23.683669+0000","last_became_active":"2023-12-01T01:37:23.678911+0000","last_became_peered":"2023-12-01T01:37:23.678911+0000","last_unstale":"2023-12-01T01:37:23.683669+0000","last_undegraded":"2023-12-01T01:37:23.683669+0000","last_fullsized":"2023-12-01T01:37:23.683669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:58:10.711651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.668590+0000","last_change":"2023-12-01T01:37:23.187434+0000","last_active":"2023-12-01T01:37:23.668590+0000","last_peered":"2023-12-01T01:37:23.668590+0000","last_clean":"2023-12-01T01:37:23.668590+0000","last_became_active":"2023-12-01T01:37:23.187025+0000","last_became_peered":"2023-12-01T01:37:23.187025+0000","last_unstale":"2023-12-01T01:37:23.668590+0000","last_undegraded":"2023-12-01T01:37:23.668590+0000","last_fullsized":"2023-12-01T01:37:23.668590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:58.522365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694647+0000","last_change":"2023-12-01T01:37:23.691320+0000","last_active":"2023-12-01T01:37:23.694647+0000","last_peered":"2023-12-01T01:37:23.694647+0000","last_clean":"2023-12-01T01:37:23.694647+0000","last_became_active":"2023-12-01T01:37:23.679036+0000","last_became_peered":"2023-12-01T01:37:23.679036+0000","last_unstale":"2023-12-01T01:37:23.694647+0000","last_undegraded":"2023-12-01T01:37:23.694647+0000","last_fullsized":"2023-12-01T01:37:23.694647+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:33.690000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696930+0000","last_change":"2023-12-01T01:37:23.203110+0000","last_active":"2023-12-01T01:37:23.696930+0000","last_peered":"2023-12-01T01:37:23.696930+0000","last_clean":"2023-12-01T01:37:23.696930+0000","last_became_active":"2023-12-01T01:37:23.202804+0000","last_became_peered":"2023-12-01T01:37:23.202804+0000","last_unstale":"2023-12-01T01:37:23.696930+0000","last_undegraded":"2023-12-01T01:37:23.696930+0000","last_fullsized":"2023-12-01T01:37:23.696930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.213589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696185+0000","last_change":"2023-12-01T01:37:23.695194+0000","last_active":"2023-12-01T01:37:23.696185+0000","last_peered":"2023-12-01T01:37:23.696185+0000","last_clean":"2023-12-01T01:37:23.696185+0000","last_became_active":"2023-12-01T01:37:23.694894+0000","last_became_peered":"2023-12-01T01:37:23.694894+0000","last_unstale":"2023-12-01T01:37:23.696185+0000","last_undegraded":"2023-12-01T01:37:23.696185+0000","last_fullsized":"2023-12-01T01:37:23.696185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:50:28.875734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694776+0000","last_change":"2023-12-01T01:37:23.200211+0000","last_active":"2023-12-01T01:37:23.694776+0000","last_peered":"2023-12-01T01:37:23.694776+0000","last_clean":"2023-12-01T01:37:23.694776+0000","last_became_active":"2023-12-01T01:37:23.199862+0000","last_became_peered":"2023-12-01T01:37:23.199862+0000","last_unstale":"2023-12-01T01:37:23.694776+0000","last_undegraded":"2023-12-01T01:37:23.694776+0000","last_fullsized":"2023-12-01T01:37:23.694776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:05:30.626410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.668725+0000","last_change":"2023-12-01T01:37:23.665736+0000","last_active":"2023-12-01T01:37:23.668725+0000","last_peered":"2023-12-01T01:37:23.668725+0000","last_clean":"2023-12-01T01:37:23.668725+0000","last_became_active":"2023-12-01T01:37:23.665479+0000","last_became_peered":"2023-12-01T01:37:23.665479+0000","last_unstale":"2023-12-01T01:37:23.668725+0000","last_undegraded":"2023-12-01T01:37:23.668725+0000","last_fullsized":"2023-12-01T01:37:23.668725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:28.569362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.693948+0000","last_change":"2023-12-01T01:37:23.193240+0000","last_active":"2023-12-01T01:37:23.693948+0000","last_peered":"2023-12-01T01:37:23.693948+0000","last_clean":"2023-12-01T01:37:23.693948+0000","last_became_active":"2023-12-01T01:37:23.192518+0000","last_became_peered":"2023-12-01T01:37:23.192518+0000","last_unstale":"2023-12-01T01:37:23.693948+0000","last_undegraded":"2023-12-01T01:37:23.693948+0000","last_fullsized":"2023-12-01T01:37:23.693948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:20:13.018107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.684092+0000","last_change":"2023-12-01T01:37:23.675898+0000","last_active":"2023-12-01T01:37:23.684092+0000","last_peered":"2023-12-01T01:37:23.684092+0000","last_clean":"2023-12-01T01:37:23.684092+0000","last_became_active":"2023-12-01T01:37:23.675553+0000","last_became_peered":"2023-12-01T01:37:23.675553+0000","last_unstale":"2023-12-01T01:37:23.684092+0000","last_undegraded":"2023-12-01T01:37:23.684092+0000","last_fullsized":"2023-12-01T01:37:23.684092+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:34.711426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.691251+0000","last_change":"2023-12-01T01:37:23.196668+0000","last_active":"2023-12-01T01:37:23.691251+0000","last_peered":"2023-12-01T01:37:23.691251+0000","last_clean":"2023-12-01T01:37:23.691251+0000","last_became_active":"2023-12-01T01:37:23.196227+0000","last_became_peered":"2023-12-01T01:37:23.196227+0000","last_unstale":"2023-12-01T01:37:23.691251+0000","last_undegraded":"2023-12-01T01:37:23.691251+0000","last_fullsized":"2023-12-01T01:37:23.691251+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:58:09.695875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696489+0000","last_change":"2023-12-01T01:37:23.686495+0000","last_active":"2023-12-01T01:37:23.696489+0000","last_peered":"2023-12-01T01:37:23.696489+0000","last_clean":"2023-12-01T01:37:23.696489+0000","last_became_active":"2023-12-01T01:37:23.686099+0000","last_became_peered":"2023-12-01T01:37:23.686099+0000","last_unstale":"2023-12-01T01:37:23.696489+0000","last_undegraded":"2023-12-01T01:37:23.696489+0000","last_fullsized":"2023-12-01T01:37:23.696489+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:26:44.405332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.660029+0000","last_change":"2023-12-01T01:37:23.198532+0000","last_active":"2023-12-01T01:37:23.660029+0000","last_peered":"2023-12-01T01:37:23.660029+0000","last_clean":"2023-12-01T01:37:23.660029+0000","last_became_active":"2023-12-01T01:37:23.197995+0000","last_became_peered":"2023-12-01T01:37:23.197995+0000","last_unstale":"2023-12-01T01:37:23.660029+0000","last_undegraded":"2023-12-01T01:37:23.660029+0000","last_fullsized":"2023-12-01T01:37:23.660029+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:58.522365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.687007+0000","last_change":"2023-12-01T01:37:23.680888+0000","last_active":"2023-12-01T01:37:23.687007+0000","last_peered":"2023-12-01T01:37:23.687007+0000","last_clean":"2023-12-01T01:37:23.687007+0000","last_became_active":"2023-12-01T01:37:23.680652+0000","last_became_peered":"2023-12-01T01:37:23.680652+0000","last_unstale":"2023-12-01T01:37:23.687007+0000","last_undegraded":"2023-12-01T01:37:23.687007+0000","last_fullsized":"2023-12-01T01:37:23.687007+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:58:10.711651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695081+0000","last_change":"2023-12-01T01:37:23.201283+0000","last_active":"2023-12-01T01:37:23.695081+0000","last_peered":"2023-12-01T01:37:23.695081+0000","last_clean":"2023-12-01T01:37:23.695081+0000","last_became_active":"2023-12-01T01:37:23.198361+0000","last_became_peered":"2023-12-01T01:37:23.198361+0000","last_unstale":"2023-12-01T01:37:23.695081+0000","last_undegraded":"2023-12-01T01:37:23.695081+0000","last_fullsized":"2023-12-01T01:37:23.695081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:14.033784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.679039+0000","last_change":"2023-12-01T01:37:23.671853+0000","last_active":"2023-12-01T01:37:23.679039+0000","last_peered":"2023-12-01T01:37:23.679039+0000","last_clean":"2023-12-01T01:37:23.679039+0000","last_became_active":"2023-12-01T01:37:23.671579+0000","last_became_peered":"2023-12-01T01:37:23.671579+0000","last_unstale":"2023-12-01T01:37:23.679039+0000","last_undegraded":"2023-12-01T01:37:23.679039+0000","last_fullsized":"2023-12-01T01:37:23.679039+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:21.718145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.673041+0000","last_change":"2023-12-01T01:37:23.191984+0000","last_active":"2023-12-01T01:37:23.673041+0000","last_peered":"2023-12-01T01:37:23.673041+0000","last_clean":"2023-12-01T01:37:23.673041+0000","last_became_active":"2023-12-01T01:37:23.191288+0000","last_became_peered":"2023-12-01T01:37:23.191288+0000","last_unstale":"2023-12-01T01:37:23.673041+0000","last_undegraded":"2023-12-01T01:37:23.673041+0000","last_fullsized":"2023-12-01T01:37:23.673041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:07.036395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696313+0000","last_change":"2023-12-01T01:37:23.692851+0000","last_active":"2023-12-01T01:37:23.696313+0000","last_peered":"2023-12-01T01:37:23.696313+0000","last_clean":"2023-12-01T01:37:23.696313+0000","last_became_active":"2023-12-01T01:37:23.692252+0000","last_became_peered":"2023-12-01T01:37:23.692252+0000","last_unstale":"2023-12-01T01:37:23.696313+0000","last_undegraded":"2023-12-01T01:37:23.696313+0000","last_fullsized":"2023-12-01T01:37:23.696313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:47.357095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.657263+0000","last_change":"2023-12-01T01:37:23.197829+0000","last_active":"2023-12-01T01:37:23.657263+0000","last_peered":"2023-12-01T01:37:23.657263+0000","last_clean":"2023-12-01T01:37:23.657263+0000","last_became_active":"2023-12-01T01:37:23.197476+0000","last_became_peered":"2023-12-01T01:37:23.197476+0000","last_unstale":"2023-12-01T01:37:23.657263+0000","last_undegraded":"2023-12-01T01:37:23.657263+0000","last_fullsized":"2023-12-01T01:37:23.657263+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:12:11.572753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.668772+0000","last_change":"2023-12-01T01:37:23.665478+0000","last_active":"2023-12-01T01:37:23.668772+0000","last_peered":"2023-12-01T01:37:23.668772+0000","last_clean":"2023-12-01T01:37:23.668772+0000","last_became_active":"2023-12-01T01:37:23.664626+0000","last_became_peered":"2023-12-01T01:37:23.664626+0000","last_unstale":"2023-12-01T01:37:23.668772+0000","last_undegraded":"2023-12-01T01:37:23.668772+0000","last_fullsized":"2023-12-01T01:37:23.668772+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:20:14.033882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.667585+0000","last_change":"2023-12-01T01:37:23.197617+0000","last_active":"2023-12-01T01:37:23.667585+0000","last_peered":"2023-12-01T01:37:23.667585+0000","last_clean":"2023-12-01T01:37:23.667585+0000","last_became_active":"2023-12-01T01:37:23.197355+0000","last_became_peered":"2023-12-01T01:37:23.197355+0000","last_unstale":"2023-12-01T01:37:23.667585+0000","last_undegraded":"2023-12-01T01:37:23.667585+0000","last_fullsized":"2023-12-01T01:37:23.667585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:51.698929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694578+0000","last_change":"2023-12-01T01:37:23.688106+0000","last_active":"2023-12-01T01:37:23.694578+0000","last_peered":"2023-12-01T01:37:23.694578+0000","last_clean":"2023-12-01T01:37:23.694578+0000","last_became_active":"2023-12-01T01:37:23.687761+0000","last_became_peered":"2023-12-01T01:37:23.687761+0000","last_unstale":"2023-12-01T01:37:23.694578+0000","last_undegraded":"2023-12-01T01:37:23.694578+0000","last_fullsized":"2023-12-01T01:37:23.694578+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:59.538140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.685676+0000","last_change":"2023-12-01T01:37:23.200459+0000","last_active":"2023-12-01T01:37:23.685676+0000","last_peered":"2023-12-01T01:37:23.685676+0000","last_clean":"2023-12-01T01:37:23.685676+0000","last_became_active":"2023-12-01T01:37:23.200172+0000","last_became_peered":"2023-12-01T01:37:23.200172+0000","last_unstale":"2023-12-01T01:37:23.685676+0000","last_undegraded":"2023-12-01T01:37:23.685676+0000","last_fullsized":"2023-12-01T01:37:23.685676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:44.390008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696865+0000","last_change":"2023-12-01T01:37:23.696298+0000","last_active":"2023-12-01T01:37:23.696865+0000","last_peered":"2023-12-01T01:37:23.696865+0000","last_clean":"2023-12-01T01:37:23.696865+0000","last_became_active":"2023-12-01T01:37:23.694835+0000","last_became_peered":"2023-12-01T01:37:23.694835+0000","last_unstale":"2023-12-01T01:37:23.696865+0000","last_undegraded":"2023-12-01T01:37:23.696865+0000","last_fullsized":"2023-12-01T01:37:23.696865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:48:28.610023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672438+0000","last_change":"2023-12-01T01:37:23.200481+0000","last_active":"2023-12-01T01:37:23.672438+0000","last_peered":"2023-12-01T01:37:23.672438+0000","last_clean":"2023-12-01T01:37:23.672438+0000","last_became_active":"2023-12-01T01:37:23.199663+0000","last_became_peered":"2023-12-01T01:37:23.199663+0000","last_unstale":"2023-12-01T01:37:23.672438+0000","last_undegraded":"2023-12-01T01:37:23.672438+0000","last_fullsized":"2023-12-01T01:37:23.672438+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:11:33.410300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696507+0000","last_change":"2023-12-01T01:37:23.691487+0000","last_active":"2023-12-01T01:37:23.696507+0000","last_peered":"2023-12-01T01:37:23.696507+0000","last_clean":"2023-12-01T01:37:23.696507+0000","last_became_active":"2023-12-01T01:37:23.691149+0000","last_became_peered":"2023-12-01T01:37:23.691149+0000","last_unstale":"2023-12-01T01:37:23.696507+0000","last_undegraded":"2023-12-01T01:37:23.696507+0000","last_fullsized":"2023-12-01T01:37:23.696507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:42:04.487553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.671026+0000","last_change":"2023-12-01T01:37:23.199357+0000","last_active":"2023-12-01T01:37:23.671026+0000","last_peered":"2023-12-01T01:37:23.671026+0000","last_clean":"2023-12-01T01:37:23.671026+0000","last_became_active":"2023-12-01T01:37:23.197994+0000","last_became_peered":"2023-12-01T01:37:23.197994+0000","last_unstale":"2023-12-01T01:37:23.671026+0000","last_undegraded":"2023-12-01T01:37:23.671026+0000","last_fullsized":"2023-12-01T01:37:23.671026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:27.553586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686625+0000","last_change":"2023-12-01T01:37:23.684220+0000","last_active":"2023-12-01T01:37:23.686625+0000","last_peered":"2023-12-01T01:37:23.686625+0000","last_clean":"2023-12-01T01:37:23.686625+0000","last_became_active":"2023-12-01T01:37:23.683970+0000","last_became_peered":"2023-12-01T01:37:23.683970+0000","last_unstale":"2023-12-01T01:37:23.686625+0000","last_undegraded":"2023-12-01T01:37:23.686625+0000","last_fullsized":"2023-12-01T01:37:23.686625+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:36:28.074355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.664160+0000","last_change":"2023-12-01T01:37:23.198686+0000","last_active":"2023-12-01T01:37:23.664160+0000","last_peered":"2023-12-01T01:37:23.664160+0000","last_clean":"2023-12-01T01:37:23.664160+0000","last_became_active":"2023-12-01T01:37:23.197993+0000","last_became_peered":"2023-12-01T01:37:23.197993+0000","last_unstale":"2023-12-01T01:37:23.664160+0000","last_undegraded":"2023-12-01T01:37:23.664160+0000","last_fullsized":"2023-12-01T01:37:23.664160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:42.327013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694399+0000","last_change":"2023-12-01T01:37:23.686092+0000","last_active":"2023-12-01T01:37:23.694399+0000","last_peered":"2023-12-01T01:37:23.694399+0000","last_clean":"2023-12-01T01:37:23.694399+0000","last_became_active":"2023-12-01T01:37:23.685733+0000","last_became_peered":"2023-12-01T01:37:23.685733+0000","last_unstale":"2023-12-01T01:37:23.694399+0000","last_undegraded":"2023-12-01T01:37:23.694399+0000","last_fullsized":"2023-12-01T01:37:23.694399+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:03:52.184710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.665612+0000","last_change":"2023-12-01T01:37:23.203314+0000","last_active":"2023-12-01T01:37:23.665612+0000","last_peered":"2023-12-01T01:37:23.665612+0000","last_clean":"2023-12-01T01:37:23.665612+0000","last_became_active":"2023-12-01T01:37:23.203060+0000","last_became_peered":"2023-12-01T01:37:23.203060+0000","last_unstale":"2023-12-01T01:37:23.665612+0000","last_undegraded":"2023-12-01T01:37:23.665612+0000","last_fullsized":"2023-12-01T01:37:23.665612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:20:13.018107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694521+0000","last_change":"2023-12-01T01:37:23.687528+0000","last_active":"2023-12-01T01:37:23.694521+0000","last_peered":"2023-12-01T01:37:23.694521+0000","last_clean":"2023-12-01T01:37:23.694521+0000","last_became_active":"2023-12-01T01:37:23.682496+0000","last_became_peered":"2023-12-01T01:37:23.682496+0000","last_unstale":"2023-12-01T01:37:23.694521+0000","last_undegraded":"2023-12-01T01:37:23.694521+0000","last_fullsized":"2023-12-01T01:37:23.694521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:18.237392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672574+0000","last_change":"2023-12-01T01:37:23.194677+0000","last_active":"2023-12-01T01:37:23.672574+0000","last_peered":"2023-12-01T01:37:23.672574+0000","last_clean":"2023-12-01T01:37:23.672574+0000","last_became_active":"2023-12-01T01:37:23.194349+0000","last_became_peered":"2023-12-01T01:37:23.194349+0000","last_unstale":"2023-12-01T01:37:23.672574+0000","last_undegraded":"2023-12-01T01:37:23.672574+0000","last_fullsized":"2023-12-01T01:37:23.672574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:47.125619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.692408+0000","last_change":"2023-12-01T01:37:23.686716+0000","last_active":"2023-12-01T01:37:23.692408+0000","last_peered":"2023-12-01T01:37:23.692408+0000","last_clean":"2023-12-01T01:37:23.692408+0000","last_became_active":"2023-12-01T01:37:23.686380+0000","last_became_peered":"2023-12-01T01:37:23.686380+0000","last_unstale":"2023-12-01T01:37:23.692408+0000","last_undegraded":"2023-12-01T01:37:23.692408+0000","last_fullsized":"2023-12-01T01:37:23.692408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:53:09.949391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.668609+0000","last_change":"2023-12-01T01:37:23.193514+0000","last_active":"2023-12-01T01:37:23.668609+0000","last_peered":"2023-12-01T01:37:23.668609+0000","last_clean":"2023-12-01T01:37:23.668609+0000","last_became_active":"2023-12-01T01:37:23.192427+0000","last_became_peered":"2023-12-01T01:37:23.192427+0000","last_unstale":"2023-12-01T01:37:23.668609+0000","last_undegraded":"2023-12-01T01:37:23.668609+0000","last_fullsized":"2023-12-01T01:37:23.668609+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:18.785910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.682398+0000","last_change":"2023-12-01T01:37:23.674887+0000","last_active":"2023-12-01T01:37:23.682398+0000","last_peered":"2023-12-01T01:37:23.682398+0000","last_clean":"2023-12-01T01:37:23.682398+0000","last_became_active":"2023-12-01T01:37:23.674710+0000","last_became_peered":"2023-12-01T01:37:23.674710+0000","last_unstale":"2023-12-01T01:37:23.682398+0000","last_undegraded":"2023-12-01T01:37:23.682398+0000","last_fullsized":"2023-12-01T01:37:23.682398+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:42.155049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672739+0000","last_change":"2023-12-01T01:37:23.196555+0000","last_active":"2023-12-01T01:37:23.672739+0000","last_peered":"2023-12-01T01:37:23.672739+0000","last_clean":"2023-12-01T01:37:23.672739+0000","last_became_active":"2023-12-01T01:37:23.196227+0000","last_became_peered":"2023-12-01T01:37:23.196227+0000","last_unstale":"2023-12-01T01:37:23.672739+0000","last_undegraded":"2023-12-01T01:37:23.672739+0000","last_fullsized":"2023-12-01T01:37:23.672739+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:13.832258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.188459+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:37:23.188459+0000","last_peered":"2023-12-01T01:37:23.188459+0000","last_clean":"2023-12-01T01:37:23.188459+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:37:23.188459+0000","last_undegraded":"2023-12-01T01:37:23.188459+0000","last_fullsized":"2023-12-01T01:37:23.188459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":142997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.212},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.78100000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.873},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.077}]},{"osd":3,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.3180000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.258},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.419}]},{"osd":5,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.3680000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.016},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":1.139,"5min":1.139,"15min":1.139},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":1.1220000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":151336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.0369999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.113},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.96699999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.5620000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.304}]},{"osd":5,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.399},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.4399999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.613,"15min":1.613},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.651,"5min":1.651,"15min":1.651},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.6419999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":740722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":3.3980000000000001,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.0129999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.206},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.96899999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.0669999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.349},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.466}]},{"osd":6,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.161},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":1.25}]},{"osd":7,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":3.012,"5min":3.012,"15min":3.012},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.51}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":151358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":3.8069999999999999,"5min":3.8069999999999999,"15min":3.8069999999999999},"last":1.298}]},{"osd":1,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1180000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.0800000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.518}]},{"osd":5,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.357}]},{"osd":6,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.401}]},{"osd":7,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.575,"15min":1.575},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":3.2789999999999999,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.256},{"interface":"front","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.663}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":150548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.93500000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1819999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.9350000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.046},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.3999999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":3.476,"5min":3.476,"15min":3.476},"last":1.272},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":1.1080000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.534},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.621}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":149806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.179}]},{"osd":1,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.2749999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.6080000000000001,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.4510000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.708},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.8440000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":3.0099999999999998,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":1.635}]},{"osd":6,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.7609999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.3859999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.796},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.9470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":740210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.056,"5min":3.056,"15min":3.056},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.95499999999999996}]},{"osd":2,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.3660000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":1.0629999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.464,"5min":3.464,"15min":3.464},"last":1.302},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.0960000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.0269999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":3.7810000000000001,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":1.2190000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":3.2839999999999998,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":1.147},{"interface":"front","average":{"1min":1.609,"5min":1.609,"15min":1.609},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":3.6669999999999998,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.575,"15min":1.575},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":3.7400000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":1.008}]},{"osd":7,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":4.6749999999999998,"5min":4.6749999999999998,"15min":4.6749999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":3.5379999999999998,"5min":3.5379999999999998,"15min":3.5379999999999998},"last":1.339}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":149862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.0600000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.95399999999999996}]},{"osd":3,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.105}]},{"osd":4,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.077}]},{"osd":5,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.173}]},{"osd":6,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.3939999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.8999999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.05,"5min":1.05,"15min":1.05},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.298}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:37:28.031 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:37:28.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.280+0000 7ff86a100640 1 Processor -- start 2023-12-01T01:37:28.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.280+0000 7ff86a100640 1 -- start start 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff86a100640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff86415e2f0 0x7ff8640ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff86a100640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff86406f2f0 0x7ff8640caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff86a100640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8640add10 con 0x7ff8640adf10 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff86a100640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8640c5760 con 0x7ff86415e2f0 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff86a100640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86415e6d0 con 0x7ff86406f2f0 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff8637fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff86415e2f0 0x7ff8640ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff8637fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff86415e2f0 0x7ff8640ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56642/0 (socket says 172.21.15.115:56642) 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff8637fe640 1 -- 172.21.15.115:0/1138479412 learned_addr learned my addr 172.21.15.115:0/1138479412 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff8637fe640 1 -- 172.21.15.115:0/1138479412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff86406f2f0 msgr2=0x7ff8640caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff8637fe640 1 --2- 172.21.15.115:0/1138479412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff86406f2f0 0x7ff8640caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff8637fe640 1 -- 172.21.15.115:0/1138479412 >> v1:172.21.15.115:6789/0 conn(0x7ff8640adf10 legacy=0x7ff8640c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.282+0000 7ff8637fe640 1 -- 172.21.15.115:0/1138479412 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8640cf7b0 con 0x7ff86415e2f0 2023-12-01T01:37:28.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.283+0000 7ff8637fe640 1 --2- 172.21.15.115:0/1138479412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86415e2f0 0x7ff8640ca660 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff854009870 tx=0x7ff85402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed4a7059af27624 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:28.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.284+0000 7ff8627fc640 1 -- 172.21.15.115:0/1138479412 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8540317d0 con 0x7ff86415e2f0 2023-12-01T01:37:28.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.284+0000 7ff8627fc640 1 -- 172.21.15.115:0/1138479412 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff854031930 con 0x7ff86415e2f0 2023-12-01T01:37:28.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.284+0000 7ff8627fc640 1 -- 172.21.15.115:0/1138479412 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854031cc0 con 0x7ff86415e2f0 2023-12-01T01:37:28.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.284+0000 7ff86a100640 1 -- 172.21.15.115:0/1138479412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86415e2f0 msgr2=0x7ff8640ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.284+0000 7ff86a100640 1 --2- 172.21.15.115:0/1138479412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86415e2f0 0x7ff8640ca660 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff854009870 tx=0x7ff85402b870 comp rx=0 tx=0).stop 2023-12-01T01:37:28.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.284+0000 7ff86a100640 1 -- 172.21.15.115:0/1138479412 shutdown_connections 2023-12-01T01:37:28.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.284+0000 7ff86a100640 1 --2- 172.21.15.115:0/1138479412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff86406f2f0 0x7ff8640caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.284+0000 7ff86a100640 1 --2- 172.21.15.115:0/1138479412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86415e2f0 0x7ff8640ca660 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.284+0000 7ff86a100640 1 -- 172.21.15.115:0/1138479412 >> 172.21.15.115:0/1138479412 conn(0x7ff8640ce880 msgr2=0x7ff8640cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:28.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.285+0000 7ff86a100640 1 -- 172.21.15.115:0/1138479412 shutdown_connections 2023-12-01T01:37:28.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.285+0000 7ff86a100640 1 -- 172.21.15.115:0/1138479412 wait complete. 2023-12-01T01:37:28.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.286+0000 7ff86a100640 1 Processor -- start 2023-12-01T01:37:28.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.286+0000 7ff86a100640 1 -- start start 2023-12-01T01:37:28.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.287+0000 7ff86a100640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86406f2f0 0x7ff8640c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.287+0000 7ff86a100640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8640adf10 0x7ff86415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.287+0000 7ff86a100640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff86415e2f0 0x7ff8641f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.287+0000 7ff86a100640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640c8030 con 0x7ff86406f2f0 2023-12-01T01:37:28.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.287+0000 7ff86a100640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640c73b0 con 0x7ff8640adf10 2023-12-01T01:37:28.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.287+0000 7ff86a100640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640c7690 con 0x7ff86415e2f0 2023-12-01T01:37:28.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.288+0000 7ff8637fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86406f2f0 0x7ff8640c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:28.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.288+0000 7ff8637fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86406f2f0 0x7ff8640c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56650/0 (socket says 172.21.15.115:56650) 2023-12-01T01:37:28.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.288+0000 7ff8637fe640 1 -- 172.21.15.115:0/249830400 learned_addr learned my addr 172.21.15.115:0/249830400 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.288+0000 7ff8637fe640 1 -- 172.21.15.115:0/249830400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff86415e2f0 msgr2=0x7ff8641f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.288+0000 7ff8637fe640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff86415e2f0 0x7ff8641f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.288+0000 7ff8637fe640 1 -- 172.21.15.115:0/249830400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8640adf10 msgr2=0x7ff86415c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.288+0000 7ff8637fe640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8640adf10 0x7ff86415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.288+0000 7ff8637fe640 1 -- 172.21.15.115:0/249830400 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8540094d0 con 0x7ff86406f2f0 2023-12-01T01:37:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.288+0000 7ff8637fe640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86406f2f0 0x7ff8640c8450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff854000c00 tx=0x7ff8540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.289+0000 7ff84bfff640 1 -- 172.21.15.115:0/249830400 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8540329b0 con 0x7ff86406f2f0 2023-12-01T01:37:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.289+0000 7ff84bfff640 1 -- 172.21.15.115:0/249830400 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff854032b30 con 0x7ff86406f2f0 2023-12-01T01:37:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.289+0000 7ff84bfff640 1 -- 172.21.15.115:0/249830400 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8540453f0 con 0x7ff86406f2f0 2023-12-01T01:37:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.289+0000 7ff86a100640 1 -- 172.21.15.115:0/249830400 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8640c7970 con 0x7ff86406f2f0 2023-12-01T01:37:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.289+0000 7ff86a100640 1 -- 172.21.15.115:0/249830400 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8640c4e80 con 0x7ff86406f2f0 2023-12-01T01:37:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.290+0000 7ff86a100640 1 -- 172.21.15.115:0/249830400 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86406fda0 con 0x7ff86406f2f0 2023-12-01T01:37:28.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.291+0000 7ff84bfff640 1 -- 172.21.15.115:0/249830400 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff854030070 con 0x7ff86406f2f0 2023-12-01T01:37:28.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.291+0000 7ff84bfff640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff85006a430 0x7ff85006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.291+0000 7ff84bfff640 1 -- 172.21.15.115:0/249830400 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8540aa570 con 0x7ff86406f2f0 2023-12-01T01:37:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.292+0000 7ff862ffd640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff85006a430 0x7ff85006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.292+0000 7ff862ffd640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff85006a430 0x7ff85006c8f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff8640ce360 tx=0x7ff858006bf0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.296+0000 7ff84bfff640 1 -- 172.21.15.115:0/249830400 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff854075b50 con 0x7ff86406f2f0 2023-12-01T01:37:28.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.483+0000 7ff86a100640 1 -- 172.21.15.115:0/249830400 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8640b43d0 con 0x7ff85006a430 2023-12-01T01:37:28.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.497+0000 7ff84bfff640 1 -- 172.21.15.115:0/249830400 <== mgr.4098 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286376 (secure 0 0 0) 0x7ff858012390 con 0x7ff85006a430 2023-12-01T01:37:28.501 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:28.503 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:37:28.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 -- 172.21.15.115:0/249830400 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff85006a430 msgr2=0x7ff85006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff85006a430 0x7ff85006c8f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff8640ce360 tx=0x7ff858006bf0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 -- 172.21.15.115:0/249830400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86406f2f0 msgr2=0x7ff8640c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86406f2f0 0x7ff8640c8450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff854000c00 tx=0x7ff8540046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 -- 172.21.15.115:0/249830400 shutdown_connections 2023-12-01T01:37:28.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff85006a430 0x7ff85006c8f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff86415e2f0 0x7ff8641f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8640adf10 0x7ff86415c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 --2- 172.21.15.115:0/249830400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff86406f2f0 0x7ff8640c8450 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 -- 172.21.15.115:0/249830400 >> 172.21.15.115:0/249830400 conn(0x7ff8640ce880 msgr2=0x7ff8640ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:28.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 -- 172.21.15.115:0/249830400 shutdown_connections 2023-12-01T01:37:28.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.503+0000 7ff86a100640 1 -- 172.21.15.115:0/249830400 wait complete. 2023-12-01T01:37:28.521 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-12-01T01:37:28.180276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219832,"kb_used_data":4216,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754888,"statfs":{"total":773094113280,"available":772869005312,"internally_reserved":0,"allocated":4317184,"data_stored":2376839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001680"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.687197+0000","last_change":"2023-12-01T01:37:23.681998+0000","last_active":"2023-12-01T01:37:23.687197+0000","last_peered":"2023-12-01T01:37:23.687197+0000","last_clean":"2023-12-01T01:37:23.687197+0000","last_became_active":"2023-12-01T01:37:23.681685+0000","last_became_peered":"2023-12-01T01:37:23.681685+0000","last_unstale":"2023-12-01T01:37:23.687197+0000","last_undegraded":"2023-12-01T01:37:23.687197+0000","last_fullsized":"2023-12-01T01:37:23.687197+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:11:34.426076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.693125+0000","last_change":"2023-12-01T01:37:23.690847+0000","last_active":"2023-12-01T01:37:23.693125+0000","last_peered":"2023-12-01T01:37:23.693125+0000","last_clean":"2023-12-01T01:37:23.693125+0000","last_became_active":"2023-12-01T01:37:23.690589+0000","last_became_peered":"2023-12-01T01:37:23.690589+0000","last_unstale":"2023-12-01T01:37:23.693125+0000","last_undegraded":"2023-12-01T01:37:23.693125+0000","last_fullsized":"2023-12-01T01:37:23.693125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:23:06.622152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.680609+0000","last_change":"2023-12-01T01:37:23.679534+0000","last_active":"2023-12-01T01:37:23.680609+0000","last_peered":"2023-12-01T01:37:23.680609+0000","last_clean":"2023-12-01T01:37:23.680609+0000","last_became_active":"2023-12-01T01:37:23.678352+0000","last_became_peered":"2023-12-01T01:37:23.678352+0000","last_unstale":"2023-12-01T01:37:23.680609+0000","last_undegraded":"2023-12-01T01:37:23.680609+0000","last_fullsized":"2023-12-01T01:37:23.680609+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:48.141395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.693831+0000","last_change":"2023-12-01T01:37:23.679726+0000","last_active":"2023-12-01T01:37:23.693831+0000","last_peered":"2023-12-01T01:37:23.693831+0000","last_clean":"2023-12-01T01:37:23.693831+0000","last_became_active":"2023-12-01T01:37:23.679422+0000","last_became_peered":"2023-12-01T01:37:23.679422+0000","last_unstale":"2023-12-01T01:37:23.693831+0000","last_undegraded":"2023-12-01T01:37:23.693831+0000","last_fullsized":"2023-12-01T01:37:23.693831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:12.379313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.685465+0000","last_change":"2023-12-01T01:37:23.679919+0000","last_active":"2023-12-01T01:37:23.685465+0000","last_peered":"2023-12-01T01:37:23.685465+0000","last_clean":"2023-12-01T01:37:23.685465+0000","last_became_active":"2023-12-01T01:37:23.679653+0000","last_became_peered":"2023-12-01T01:37:23.679653+0000","last_unstale":"2023-12-01T01:37:23.685465+0000","last_undegraded":"2023-12-01T01:37:23.685465+0000","last_fullsized":"2023-12-01T01:37:23.685465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:05:29.927110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.687559+0000","last_change":"2023-12-01T01:37:23.684151+0000","last_active":"2023-12-01T01:37:23.687559+0000","last_peered":"2023-12-01T01:37:23.687559+0000","last_clean":"2023-12-01T01:37:23.687559+0000","last_became_active":"2023-12-01T01:37:23.683868+0000","last_became_peered":"2023-12-01T01:37:23.683868+0000","last_unstale":"2023-12-01T01:37:23.687559+0000","last_undegraded":"2023-12-01T01:37:23.687559+0000","last_fullsized":"2023-12-01T01:37:23.687559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:31:10.954677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.683693+0000","last_change":"2023-12-01T01:37:23.678567+0000","last_active":"2023-12-01T01:37:23.683693+0000","last_peered":"2023-12-01T01:37:23.683693+0000","last_clean":"2023-12-01T01:37:23.683693+0000","last_became_active":"2023-12-01T01:37:23.678191+0000","last_became_peered":"2023-12-01T01:37:23.678191+0000","last_unstale":"2023-12-01T01:37:23.683693+0000","last_undegraded":"2023-12-01T01:37:23.683693+0000","last_fullsized":"2023-12-01T01:37:23.683693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:36:28.074355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690596+0000","last_change":"2023-12-01T01:37:23.687321+0000","last_active":"2023-12-01T01:37:23.690596+0000","last_peered":"2023-12-01T01:37:23.690596+0000","last_clean":"2023-12-01T01:37:23.690596+0000","last_became_active":"2023-12-01T01:37:23.687008+0000","last_became_peered":"2023-12-01T01:37:23.687008+0000","last_unstale":"2023-12-01T01:37:23.690596+0000","last_undegraded":"2023-12-01T01:37:23.690596+0000","last_fullsized":"2023-12-01T01:37:23.690596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:14.848033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686173+0000","last_change":"2023-12-01T01:37:23.684066+0000","last_active":"2023-12-01T01:37:23.686173+0000","last_peered":"2023-12-01T01:37:23.686173+0000","last_clean":"2023-12-01T01:37:23.686173+0000","last_became_active":"2023-12-01T01:37:23.683836+0000","last_became_peered":"2023-12-01T01:37:23.683836+0000","last_unstale":"2023-12-01T01:37:23.686173+0000","last_undegraded":"2023-12-01T01:37:23.686173+0000","last_fullsized":"2023-12-01T01:37:23.686173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:39.378914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694325+0000","last_change":"2023-12-01T01:37:23.684965+0000","last_active":"2023-12-01T01:37:23.694325+0000","last_peered":"2023-12-01T01:37:23.694325+0000","last_clean":"2023-12-01T01:37:23.694325+0000","last_became_active":"2023-12-01T01:37:23.681896+0000","last_became_peered":"2023-12-01T01:37:23.681896+0000","last_unstale":"2023-12-01T01:37:23.694325+0000","last_undegraded":"2023-12-01T01:37:23.694325+0000","last_fullsized":"2023-12-01T01:37:23.694325+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:31.630266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695741+0000","last_change":"2023-12-01T01:37:23.692712+0000","last_active":"2023-12-01T01:37:23.695741+0000","last_peered":"2023-12-01T01:37:23.695741+0000","last_clean":"2023-12-01T01:37:23.695741+0000","last_became_active":"2023-12-01T01:37:23.691965+0000","last_became_peered":"2023-12-01T01:37:23.691965+0000","last_unstale":"2023-12-01T01:37:23.695741+0000","last_undegraded":"2023-12-01T01:37:23.695741+0000","last_fullsized":"2023-12-01T01:37:23.695741+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:12.379313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.687326+0000","last_change":"2023-12-01T01:37:23.685697+0000","last_active":"2023-12-01T01:37:23.687326+0000","last_peered":"2023-12-01T01:37:23.687326+0000","last_clean":"2023-12-01T01:37:23.687326+0000","last_became_active":"2023-12-01T01:37:23.684823+0000","last_became_peered":"2023-12-01T01:37:23.684823+0000","last_unstale":"2023-12-01T01:37:23.687326+0000","last_undegraded":"2023-12-01T01:37:23.687326+0000","last_fullsized":"2023-12-01T01:37:23.687326+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:19.801685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.692944+0000","last_change":"2023-12-01T01:37:23.691511+0000","last_active":"2023-12-01T01:37:23.692944+0000","last_peered":"2023-12-01T01:37:23.692944+0000","last_clean":"2023-12-01T01:37:23.692944+0000","last_became_active":"2023-12-01T01:37:23.690500+0000","last_became_peered":"2023-12-01T01:37:23.690500+0000","last_unstale":"2023-12-01T01:37:23.692944+0000","last_undegraded":"2023-12-01T01:37:23.692944+0000","last_fullsized":"2023-12-01T01:37:23.692944+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:15.049560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694491+0000","last_change":"2023-12-01T01:37:23.687497+0000","last_active":"2023-12-01T01:37:23.694491+0000","last_peered":"2023-12-01T01:37:23.694491+0000","last_clean":"2023-12-01T01:37:23.694491+0000","last_became_active":"2023-12-01T01:37:23.687175+0000","last_became_peered":"2023-12-01T01:37:23.687175+0000","last_unstale":"2023-12-01T01:37:23.694491+0000","last_undegraded":"2023-12-01T01:37:23.694491+0000","last_fullsized":"2023-12-01T01:37:23.694491+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:36:28.074355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.683132+0000","last_change":"2023-12-01T01:37:23.679775+0000","last_active":"2023-12-01T01:37:23.683132+0000","last_peered":"2023-12-01T01:37:23.683132+0000","last_clean":"2023-12-01T01:37:23.683132+0000","last_became_active":"2023-12-01T01:37:23.679477+0000","last_became_peered":"2023-12-01T01:37:23.679477+0000","last_unstale":"2023-12-01T01:37:23.683132+0000","last_undegraded":"2023-12-01T01:37:23.683132+0000","last_fullsized":"2023-12-01T01:37:23.683132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:15.049560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686885+0000","last_change":"2023-12-01T01:37:23.684855+0000","last_active":"2023-12-01T01:37:23.686885+0000","last_peered":"2023-12-01T01:37:23.686885+0000","last_clean":"2023-12-01T01:37:23.686885+0000","last_became_active":"2023-12-01T01:37:23.684483+0000","last_became_peered":"2023-12-01T01:37:23.684483+0000","last_unstale":"2023-12-01T01:37:23.686885+0000","last_undegraded":"2023-12-01T01:37:23.686885+0000","last_fullsized":"2023-12-01T01:37:23.686885+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:52.714705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696231+0000","last_change":"2023-12-01T01:37:23.688379+0000","last_active":"2023-12-01T01:37:23.696231+0000","last_peered":"2023-12-01T01:37:23.696231+0000","last_clean":"2023-12-01T01:37:23.696231+0000","last_became_active":"2023-12-01T01:37:23.688122+0000","last_became_peered":"2023-12-01T01:37:23.688122+0000","last_unstale":"2023-12-01T01:37:23.696231+0000","last_undegraded":"2023-12-01T01:37:23.696231+0000","last_fullsized":"2023-12-01T01:37:23.696231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:33.319519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694560+0000","last_change":"2023-12-01T01:37:23.685576+0000","last_active":"2023-12-01T01:37:23.694560+0000","last_peered":"2023-12-01T01:37:23.694560+0000","last_clean":"2023-12-01T01:37:23.694560+0000","last_became_active":"2023-12-01T01:37:23.685311+0000","last_became_peered":"2023-12-01T01:37:23.685311+0000","last_unstale":"2023-12-01T01:37:23.694560+0000","last_undegraded":"2023-12-01T01:37:23.694560+0000","last_fullsized":"2023-12-01T01:37:23.694560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:29:16.659485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686317+0000","last_change":"2023-12-01T01:37:23.682949+0000","last_active":"2023-12-01T01:37:23.686317+0000","last_peered":"2023-12-01T01:37:23.686317+0000","last_clean":"2023-12-01T01:37:23.686317+0000","last_became_active":"2023-12-01T01:37:23.680770+0000","last_became_peered":"2023-12-01T01:37:23.680770+0000","last_unstale":"2023-12-01T01:37:23.686317+0000","last_undegraded":"2023-12-01T01:37:23.686317+0000","last_fullsized":"2023-12-01T01:37:23.686317+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:34:42.074507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.688125+0000","last_change":"2023-12-01T01:37:23.683476+0000","last_active":"2023-12-01T01:37:23.688125+0000","last_peered":"2023-12-01T01:37:23.688125+0000","last_clean":"2023-12-01T01:37:23.688125+0000","last_became_active":"2023-12-01T01:37:23.683195+0000","last_became_peered":"2023-12-01T01:37:23.683195+0000","last_unstale":"2023-12-01T01:37:23.688125+0000","last_undegraded":"2023-12-01T01:37:23.688125+0000","last_fullsized":"2023-12-01T01:37:23.688125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:36.901860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.682829+0000","last_change":"2023-12-01T01:37:23.678480+0000","last_active":"2023-12-01T01:37:23.682829+0000","last_peered":"2023-12-01T01:37:23.682829+0000","last_clean":"2023-12-01T01:37:23.682829+0000","last_became_active":"2023-12-01T01:37:23.675283+0000","last_became_peered":"2023-12-01T01:37:23.675283+0000","last_unstale":"2023-12-01T01:37:23.682829+0000","last_undegraded":"2023-12-01T01:37:23.682829+0000","last_fullsized":"2023-12-01T01:37:23.682829+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:31.630266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695095+0000","last_change":"2023-12-01T01:37:23.693003+0000","last_active":"2023-12-01T01:37:23.695095+0000","last_peered":"2023-12-01T01:37:23.695095+0000","last_clean":"2023-12-01T01:37:23.695095+0000","last_became_active":"2023-12-01T01:37:23.691440+0000","last_became_peered":"2023-12-01T01:37:23.691440+0000","last_unstale":"2023-12-01T01:37:23.695095+0000","last_undegraded":"2023-12-01T01:37:23.695095+0000","last_fullsized":"2023-12-01T01:37:23.695095+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:31.630266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686551+0000","last_change":"2023-12-01T01:37:23.675906+0000","last_active":"2023-12-01T01:37:23.686551+0000","last_peered":"2023-12-01T01:37:23.686551+0000","last_clean":"2023-12-01T01:37:23.686551+0000","last_became_active":"2023-12-01T01:37:23.675559+0000","last_became_peered":"2023-12-01T01:37:23.675559+0000","last_unstale":"2023-12-01T01:37:23.686551+0000","last_undegraded":"2023-12-01T01:37:23.686551+0000","last_fullsized":"2023-12-01T01:37:23.686551+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:14.848033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695139+0000","last_change":"2023-12-01T01:37:23.693336+0000","last_active":"2023-12-01T01:37:23.695139+0000","last_peered":"2023-12-01T01:37:23.695139+0000","last_clean":"2023-12-01T01:37:23.695139+0000","last_became_active":"2023-12-01T01:37:23.693101+0000","last_became_peered":"2023-12-01T01:37:23.693101+0000","last_unstale":"2023-12-01T01:37:23.695139+0000","last_undegraded":"2023-12-01T01:37:23.695139+0000","last_fullsized":"2023-12-01T01:37:23.695139+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:21.718145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694195+0000","last_change":"2023-12-01T01:37:23.681716+0000","last_active":"2023-12-01T01:37:23.694195+0000","last_peered":"2023-12-01T01:37:23.694195+0000","last_clean":"2023-12-01T01:37:23.694195+0000","last_became_active":"2023-12-01T01:37:23.681433+0000","last_became_peered":"2023-12-01T01:37:23.681433+0000","last_unstale":"2023-12-01T01:37:23.694195+0000","last_undegraded":"2023-12-01T01:37:23.694195+0000","last_fullsized":"2023-12-01T01:37:23.694195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:05:29.927110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696989+0000","last_change":"2023-12-01T01:37:23.688550+0000","last_active":"2023-12-01T01:37:23.696989+0000","last_peered":"2023-12-01T01:37:23.696989+0000","last_clean":"2023-12-01T01:37:23.696989+0000","last_became_active":"2023-12-01T01:37:23.679710+0000","last_became_peered":"2023-12-01T01:37:23.679710+0000","last_unstale":"2023-12-01T01:37:23.696989+0000","last_undegraded":"2023-12-01T01:37:23.696989+0000","last_fullsized":"2023-12-01T01:37:23.696989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:42.155049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.684087+0000","last_change":"2023-12-01T01:37:23.678830+0000","last_active":"2023-12-01T01:37:23.684087+0000","last_peered":"2023-12-01T01:37:23.684087+0000","last_clean":"2023-12-01T01:37:23.684087+0000","last_became_active":"2023-12-01T01:37:23.677197+0000","last_became_peered":"2023-12-01T01:37:23.677197+0000","last_unstale":"2023-12-01T01:37:23.684087+0000","last_undegraded":"2023-12-01T01:37:23.684087+0000","last_fullsized":"2023-12-01T01:37:23.684087+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:21.433110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690805+0000","last_change":"2023-12-01T01:37:23.689087+0000","last_active":"2023-12-01T01:37:23.690805+0000","last_peered":"2023-12-01T01:37:23.690805+0000","last_clean":"2023-12-01T01:37:23.690805+0000","last_became_active":"2023-12-01T01:37:23.688934+0000","last_became_peered":"2023-12-01T01:37:23.688934+0000","last_unstale":"2023-12-01T01:37:23.690805+0000","last_undegraded":"2023-12-01T01:37:23.690805+0000","last_fullsized":"2023-12-01T01:37:23.690805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:42.155049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.691722+0000","last_change":"2023-12-01T01:37:23.687797+0000","last_active":"2023-12-01T01:37:23.691722+0000","last_peered":"2023-12-01T01:37:23.691722+0000","last_clean":"2023-12-01T01:37:23.691722+0000","last_became_active":"2023-12-01T01:37:23.687493+0000","last_became_peered":"2023-12-01T01:37:23.687493+0000","last_unstale":"2023-12-01T01:37:23.691722+0000","last_undegraded":"2023-12-01T01:37:23.691722+0000","last_fullsized":"2023-12-01T01:37:23.691722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:58:10.711651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.692557+0000","last_change":"2023-12-01T01:37:23.688405+0000","last_active":"2023-12-01T01:37:23.692557+0000","last_peered":"2023-12-01T01:37:23.692557+0000","last_clean":"2023-12-01T01:37:23.692557+0000","last_became_active":"2023-12-01T01:37:23.688124+0000","last_became_peered":"2023-12-01T01:37:23.688124+0000","last_unstale":"2023-12-01T01:37:23.692557+0000","last_undegraded":"2023-12-01T01:37:23.692557+0000","last_fullsized":"2023-12-01T01:37:23.692557+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:14.848033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690569+0000","last_change":"2023-12-01T01:37:23.685538+0000","last_active":"2023-12-01T01:37:23.690569+0000","last_peered":"2023-12-01T01:37:23.690569+0000","last_clean":"2023-12-01T01:37:23.690569+0000","last_became_active":"2023-12-01T01:37:23.685250+0000","last_became_peered":"2023-12-01T01:37:23.685250+0000","last_unstale":"2023-12-01T01:37:23.690569+0000","last_undegraded":"2023-12-01T01:37:23.690569+0000","last_fullsized":"2023-12-01T01:37:23.690569+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:52.714705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694658+0000","last_change":"2023-12-01T01:37:23.683959+0000","last_active":"2023-12-01T01:37:23.694658+0000","last_peered":"2023-12-01T01:37:23.694658+0000","last_clean":"2023-12-01T01:37:23.694658+0000","last_became_active":"2023-12-01T01:37:23.683723+0000","last_became_peered":"2023-12-01T01:37:23.683723+0000","last_unstale":"2023-12-01T01:37:23.694658+0000","last_undegraded":"2023-12-01T01:37:23.694658+0000","last_fullsized":"2023-12-01T01:37:23.694658+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:26:37.036519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.697258+0000","last_change":"2023-12-01T01:37:23.695796+0000","last_active":"2023-12-01T01:37:23.697258+0000","last_peered":"2023-12-01T01:37:23.697258+0000","last_clean":"2023-12-01T01:37:23.697258+0000","last_became_active":"2023-12-01T01:37:23.695419+0000","last_became_peered":"2023-12-01T01:37:23.695419+0000","last_unstale":"2023-12-01T01:37:23.697258+0000","last_undegraded":"2023-12-01T01:37:23.697258+0000","last_fullsized":"2023-12-01T01:37:23.697258+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:11:34.426076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.691709+0000","last_change":"2023-12-01T01:37:23.197332+0000","last_active":"2023-12-01T01:37:23.691709+0000","last_peered":"2023-12-01T01:37:23.691709+0000","last_clean":"2023-12-01T01:37:23.691709+0000","last_became_active":"2023-12-01T01:37:23.197041+0000","last_became_peered":"2023-12-01T01:37:23.197041+0000","last_unstale":"2023-12-01T01:37:23.691709+0000","last_undegraded":"2023-12-01T01:37:23.691709+0000","last_fullsized":"2023-12-01T01:37:23.691709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:41.139273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690576+0000","last_change":"2023-12-01T01:37:23.687315+0000","last_active":"2023-12-01T01:37:23.690576+0000","last_peered":"2023-12-01T01:37:23.690576+0000","last_clean":"2023-12-01T01:37:23.690576+0000","last_became_active":"2023-12-01T01:37:23.687040+0000","last_became_peered":"2023-12-01T01:37:23.687040+0000","last_unstale":"2023-12-01T01:37:23.690576+0000","last_undegraded":"2023-12-01T01:37:23.690576+0000","last_fullsized":"2023-12-01T01:37:23.690576+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:47.357095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.693966+0000","last_change":"2023-12-01T01:37:23.193859+0000","last_active":"2023-12-01T01:37:23.693966+0000","last_peered":"2023-12-01T01:37:23.693966+0000","last_clean":"2023-12-01T01:37:23.693966+0000","last_became_active":"2023-12-01T01:37:23.193204+0000","last_became_peered":"2023-12-01T01:37:23.193204+0000","last_unstale":"2023-12-01T01:37:23.693966+0000","last_undegraded":"2023-12-01T01:37:23.693966+0000","last_fullsized":"2023-12-01T01:37:23.693966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:38.363139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695223+0000","last_change":"2023-12-01T01:37:23.693532+0000","last_active":"2023-12-01T01:37:23.695223+0000","last_peered":"2023-12-01T01:37:23.695223+0000","last_clean":"2023-12-01T01:37:23.695223+0000","last_became_active":"2023-12-01T01:37:23.693308+0000","last_became_peered":"2023-12-01T01:37:23.693308+0000","last_unstale":"2023-12-01T01:37:23.695223+0000","last_undegraded":"2023-12-01T01:37:23.695223+0000","last_fullsized":"2023-12-01T01:37:23.695223+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:21.433110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.688636+0000","last_change":"2023-12-01T01:37:23.203553+0000","last_active":"2023-12-01T01:37:23.688636+0000","last_peered":"2023-12-01T01:37:23.688636+0000","last_clean":"2023-12-01T01:37:23.688636+0000","last_became_active":"2023-12-01T01:37:23.202873+0000","last_became_peered":"2023-12-01T01:37:23.202873+0000","last_unstale":"2023-12-01T01:37:23.688636+0000","last_undegraded":"2023-12-01T01:37:23.688636+0000","last_fullsized":"2023-12-01T01:37:23.688636+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:53:05.658395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694284+0000","last_change":"2023-12-01T01:37:23.680504+0000","last_active":"2023-12-01T01:37:23.694284+0000","last_peered":"2023-12-01T01:37:23.694284+0000","last_clean":"2023-12-01T01:37:23.694284+0000","last_became_active":"2023-12-01T01:37:23.680241+0000","last_became_peered":"2023-12-01T01:37:23.680241+0000","last_unstale":"2023-12-01T01:37:23.694284+0000","last_undegraded":"2023-12-01T01:37:23.694284+0000","last_fullsized":"2023-12-01T01:37:23.694284+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:31.615342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690261+0000","last_change":"2023-12-01T01:37:23.198906+0000","last_active":"2023-12-01T01:37:23.690261+0000","last_peered":"2023-12-01T01:37:23.690261+0000","last_clean":"2023-12-01T01:37:23.690261+0000","last_became_active":"2023-12-01T01:37:23.198493+0000","last_became_peered":"2023-12-01T01:37:23.198493+0000","last_unstale":"2023-12-01T01:37:23.690261+0000","last_undegraded":"2023-12-01T01:37:23.690261+0000","last_fullsized":"2023-12-01T01:37:23.690261+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:42:03.471777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.690176+0000","last_change":"2023-12-01T01:37:23.688239+0000","last_active":"2023-12-01T01:37:23.690176+0000","last_peered":"2023-12-01T01:37:23.690176+0000","last_clean":"2023-12-01T01:37:23.690176+0000","last_became_active":"2023-12-01T01:37:23.687947+0000","last_became_peered":"2023-12-01T01:37:23.687947+0000","last_unstale":"2023-12-01T01:37:23.690176+0000","last_undegraded":"2023-12-01T01:37:23.690176+0000","last_fullsized":"2023-12-01T01:37:23.690176+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:31:10.954677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.684827+0000","last_change":"2023-12-01T01:37:23.191336+0000","last_active":"2023-12-01T01:37:23.684827+0000","last_peered":"2023-12-01T01:37:23.684827+0000","last_clean":"2023-12-01T01:37:23.684827+0000","last_became_active":"2023-12-01T01:37:23.190957+0000","last_became_peered":"2023-12-01T01:37:23.190957+0000","last_unstale":"2023-12-01T01:37:23.684827+0000","last_undegraded":"2023-12-01T01:37:23.684827+0000","last_fullsized":"2023-12-01T01:37:23.684827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:53:05.658395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.691021+0000","last_change":"2023-12-01T01:37:23.689589+0000","last_active":"2023-12-01T01:37:23.691021+0000","last_peered":"2023-12-01T01:37:23.691021+0000","last_clean":"2023-12-01T01:37:23.691021+0000","last_became_active":"2023-12-01T01:37:23.689229+0000","last_became_peered":"2023-12-01T01:37:23.689229+0000","last_unstale":"2023-12-01T01:37:23.691021+0000","last_undegraded":"2023-12-01T01:37:23.691021+0000","last_fullsized":"2023-12-01T01:37:23.691021+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:15.049560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686895+0000","last_change":"2023-12-01T01:37:23.199499+0000","last_active":"2023-12-01T01:37:23.686895+0000","last_peered":"2023-12-01T01:37:23.686895+0000","last_clean":"2023-12-01T01:37:23.686895+0000","last_became_active":"2023-12-01T01:37:23.199243+0000","last_became_peered":"2023-12-01T01:37:23.199243+0000","last_unstale":"2023-12-01T01:37:23.686895+0000","last_undegraded":"2023-12-01T01:37:23.686895+0000","last_fullsized":"2023-12-01T01:37:23.686895+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:42.327013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.667003+0000","last_change":"2023-12-01T01:37:23.663674+0000","last_active":"2023-12-01T01:37:23.667003+0000","last_peered":"2023-12-01T01:37:23.667003+0000","last_clean":"2023-12-01T01:37:23.667003+0000","last_became_active":"2023-12-01T01:37:23.663490+0000","last_became_peered":"2023-12-01T01:37:23.663490+0000","last_unstale":"2023-12-01T01:37:23.667003+0000","last_undegraded":"2023-12-01T01:37:23.667003+0000","last_fullsized":"2023-12-01T01:37:23.667003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:43.342788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.665588+0000","last_change":"2023-12-01T01:37:23.202531+0000","last_active":"2023-12-01T01:37:23.665588+0000","last_peered":"2023-12-01T01:37:23.665588+0000","last_clean":"2023-12-01T01:37:23.665588+0000","last_became_active":"2023-12-01T01:37:23.202242+0000","last_became_peered":"2023-12-01T01:37:23.202242+0000","last_unstale":"2023-12-01T01:37:23.665588+0000","last_undegraded":"2023-12-01T01:37:23.665588+0000","last_fullsized":"2023-12-01T01:37:23.665588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:07.036395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694363+0000","last_change":"2023-12-01T01:37:23.685915+0000","last_active":"2023-12-01T01:37:23.694363+0000","last_peered":"2023-12-01T01:37:23.694363+0000","last_clean":"2023-12-01T01:37:23.694363+0000","last_became_active":"2023-12-01T01:37:23.682054+0000","last_became_peered":"2023-12-01T01:37:23.682054+0000","last_unstale":"2023-12-01T01:37:23.694363+0000","last_undegraded":"2023-12-01T01:37:23.694363+0000","last_fullsized":"2023-12-01T01:37:23.694363+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:21.433110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672282+0000","last_change":"2023-12-01T01:37:23.193655+0000","last_active":"2023-12-01T01:37:23.672282+0000","last_peered":"2023-12-01T01:37:23.672282+0000","last_clean":"2023-12-01T01:37:23.672282+0000","last_became_active":"2023-12-01T01:37:23.193348+0000","last_became_peered":"2023-12-01T01:37:23.193348+0000","last_unstale":"2023-12-01T01:37:23.672282+0000","last_undegraded":"2023-12-01T01:37:23.672282+0000","last_fullsized":"2023-12-01T01:37:23.672282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:05:30.626410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695671+0000","last_change":"2023-12-01T01:37:23.692414+0000","last_active":"2023-12-01T01:37:23.695671+0000","last_peered":"2023-12-01T01:37:23.695671+0000","last_clean":"2023-12-01T01:37:23.695671+0000","last_became_active":"2023-12-01T01:37:23.688590+0000","last_became_peered":"2023-12-01T01:37:23.688590+0000","last_unstale":"2023-12-01T01:37:23.695671+0000","last_undegraded":"2023-12-01T01:37:23.695671+0000","last_fullsized":"2023-12-01T01:37:23.695671+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:11:26.830819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.667074+0000","last_change":"2023-12-01T01:37:23.201058+0000","last_active":"2023-12-01T01:37:23.667074+0000","last_peered":"2023-12-01T01:37:23.667074+0000","last_clean":"2023-12-01T01:37:23.667074+0000","last_became_active":"2023-12-01T01:37:23.200513+0000","last_became_peered":"2023-12-01T01:37:23.200513+0000","last_unstale":"2023-12-01T01:37:23.667074+0000","last_undegraded":"2023-12-01T01:37:23.667074+0000","last_fullsized":"2023-12-01T01:37:23.667074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:00:19.556191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694180+0000","last_change":"2023-12-01T01:37:23.681615+0000","last_active":"2023-12-01T01:37:23.694180+0000","last_peered":"2023-12-01T01:37:23.694180+0000","last_clean":"2023-12-01T01:37:23.694180+0000","last_became_active":"2023-12-01T01:37:23.681319+0000","last_became_peered":"2023-12-01T01:37:23.681319+0000","last_unstale":"2023-12-01T01:37:23.694180+0000","last_undegraded":"2023-12-01T01:37:23.694180+0000","last_fullsized":"2023-12-01T01:37:23.694180+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:34.711426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672324+0000","last_change":"2023-12-01T01:37:23.201065+0000","last_active":"2023-12-01T01:37:23.672324+0000","last_peered":"2023-12-01T01:37:23.672324+0000","last_clean":"2023-12-01T01:37:23.672324+0000","last_became_active":"2023-12-01T01:37:23.200778+0000","last_became_peered":"2023-12-01T01:37:23.200778+0000","last_unstale":"2023-12-01T01:37:23.672324+0000","last_undegraded":"2023-12-01T01:37:23.672324+0000","last_fullsized":"2023-12-01T01:37:23.672324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:46.341319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694439+0000","last_change":"2023-12-01T01:37:23.685634+0000","last_active":"2023-12-01T01:37:23.694439+0000","last_peered":"2023-12-01T01:37:23.694439+0000","last_clean":"2023-12-01T01:37:23.694439+0000","last_became_active":"2023-12-01T01:37:23.682383+0000","last_became_peered":"2023-12-01T01:37:23.682383+0000","last_unstale":"2023-12-01T01:37:23.694439+0000","last_undegraded":"2023-12-01T01:37:23.694439+0000","last_fullsized":"2023-12-01T01:37:23.694439+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:27:09.123090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.692818+0000","last_change":"2023-12-01T01:37:23.192448+0000","last_active":"2023-12-01T01:37:23.692818+0000","last_peered":"2023-12-01T01:37:23.692818+0000","last_clean":"2023-12-01T01:37:23.692818+0000","last_became_active":"2023-12-01T01:37:23.192188+0000","last_became_peered":"2023-12-01T01:37:23.192188+0000","last_unstale":"2023-12-01T01:37:23.692818+0000","last_undegraded":"2023-12-01T01:37:23.692818+0000","last_fullsized":"2023-12-01T01:37:23.692818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:36:27.058579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695725+0000","last_change":"2023-12-01T01:37:23.692604+0000","last_active":"2023-12-01T01:37:23.695725+0000","last_peered":"2023-12-01T01:37:23.695725+0000","last_clean":"2023-12-01T01:37:23.695725+0000","last_became_active":"2023-12-01T01:37:23.692321+0000","last_became_peered":"2023-12-01T01:37:23.692321+0000","last_unstale":"2023-12-01T01:37:23.695725+0000","last_undegraded":"2023-12-01T01:37:23.695725+0000","last_fullsized":"2023-12-01T01:37:23.695725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:02:54.947127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.699611+0000","last_change":"2023-12-01T01:37:23.199582+0000","last_active":"2023-12-01T01:37:23.699611+0000","last_peered":"2023-12-01T01:37:23.699611+0000","last_clean":"2023-12-01T01:37:23.699611+0000","last_became_active":"2023-12-01T01:37:23.199029+0000","last_became_peered":"2023-12-01T01:37:23.199029+0000","last_unstale":"2023-12-01T01:37:23.699611+0000","last_undegraded":"2023-12-01T01:37:23.699611+0000","last_fullsized":"2023-12-01T01:37:23.699611+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:31:09.938902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686575+0000","last_change":"2023-12-01T01:37:23.677735+0000","last_active":"2023-12-01T01:37:23.686575+0000","last_peered":"2023-12-01T01:37:23.686575+0000","last_clean":"2023-12-01T01:37:23.686575+0000","last_became_active":"2023-12-01T01:37:23.677447+0000","last_became_peered":"2023-12-01T01:37:23.677447+0000","last_unstale":"2023-12-01T01:37:23.686575+0000","last_undegraded":"2023-12-01T01:37:23.686575+0000","last_fullsized":"2023-12-01T01:37:23.686575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:52.714705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.693853+0000","last_change":"2023-12-01T01:37:23.192003+0000","last_active":"2023-12-01T01:37:23.693853+0000","last_peered":"2023-12-01T01:37:23.693853+0000","last_clean":"2023-12-01T01:37:23.693853+0000","last_became_active":"2023-12-01T01:37:23.191287+0000","last_became_peered":"2023-12-01T01:37:23.191287+0000","last_unstale":"2023-12-01T01:37:23.693853+0000","last_undegraded":"2023-12-01T01:37:23.693853+0000","last_fullsized":"2023-12-01T01:37:23.693853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:44.390008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696982+0000","last_change":"2023-12-01T01:37:23.689085+0000","last_active":"2023-12-01T01:37:23.696982+0000","last_peered":"2023-12-01T01:37:23.696982+0000","last_clean":"2023-12-01T01:37:23.696982+0000","last_became_active":"2023-12-01T01:37:23.688533+0000","last_became_peered":"2023-12-01T01:37:23.688533+0000","last_unstale":"2023-12-01T01:37:23.696982+0000","last_undegraded":"2023-12-01T01:37:23.696982+0000","last_fullsized":"2023-12-01T01:37:23.696982+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:26:37.036519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672237+0000","last_change":"2023-12-01T01:37:23.192431+0000","last_active":"2023-12-01T01:37:23.672237+0000","last_peered":"2023-12-01T01:37:23.672237+0000","last_clean":"2023-12-01T01:37:23.672237+0000","last_became_active":"2023-12-01T01:37:23.192084+0000","last_became_peered":"2023-12-01T01:37:23.192084+0000","last_unstale":"2023-12-01T01:37:23.672237+0000","last_undegraded":"2023-12-01T01:37:23.672237+0000","last_fullsized":"2023-12-01T01:37:23.672237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:31:09.938902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.683669+0000","last_change":"2023-12-01T01:37:23.679748+0000","last_active":"2023-12-01T01:37:23.683669+0000","last_peered":"2023-12-01T01:37:23.683669+0000","last_clean":"2023-12-01T01:37:23.683669+0000","last_became_active":"2023-12-01T01:37:23.678911+0000","last_became_peered":"2023-12-01T01:37:23.678911+0000","last_unstale":"2023-12-01T01:37:23.683669+0000","last_undegraded":"2023-12-01T01:37:23.683669+0000","last_fullsized":"2023-12-01T01:37:23.683669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:58:10.711651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.668590+0000","last_change":"2023-12-01T01:37:23.187434+0000","last_active":"2023-12-01T01:37:23.668590+0000","last_peered":"2023-12-01T01:37:23.668590+0000","last_clean":"2023-12-01T01:37:23.668590+0000","last_became_active":"2023-12-01T01:37:23.187025+0000","last_became_peered":"2023-12-01T01:37:23.187025+0000","last_unstale":"2023-12-01T01:37:23.668590+0000","last_undegraded":"2023-12-01T01:37:23.668590+0000","last_fullsized":"2023-12-01T01:37:23.668590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:58.522365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694647+0000","last_change":"2023-12-01T01:37:23.691320+0000","last_active":"2023-12-01T01:37:23.694647+0000","last_peered":"2023-12-01T01:37:23.694647+0000","last_clean":"2023-12-01T01:37:23.694647+0000","last_became_active":"2023-12-01T01:37:23.679036+0000","last_became_peered":"2023-12-01T01:37:23.679036+0000","last_unstale":"2023-12-01T01:37:23.694647+0000","last_undegraded":"2023-12-01T01:37:23.694647+0000","last_fullsized":"2023-12-01T01:37:23.694647+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:33.690000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696930+0000","last_change":"2023-12-01T01:37:23.203110+0000","last_active":"2023-12-01T01:37:23.696930+0000","last_peered":"2023-12-01T01:37:23.696930+0000","last_clean":"2023-12-01T01:37:23.696930+0000","last_became_active":"2023-12-01T01:37:23.202804+0000","last_became_peered":"2023-12-01T01:37:23.202804+0000","last_unstale":"2023-12-01T01:37:23.696930+0000","last_undegraded":"2023-12-01T01:37:23.696930+0000","last_fullsized":"2023-12-01T01:37:23.696930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.213589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696185+0000","last_change":"2023-12-01T01:37:23.695194+0000","last_active":"2023-12-01T01:37:23.696185+0000","last_peered":"2023-12-01T01:37:23.696185+0000","last_clean":"2023-12-01T01:37:23.696185+0000","last_became_active":"2023-12-01T01:37:23.694894+0000","last_became_peered":"2023-12-01T01:37:23.694894+0000","last_unstale":"2023-12-01T01:37:23.696185+0000","last_undegraded":"2023-12-01T01:37:23.696185+0000","last_fullsized":"2023-12-01T01:37:23.696185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:50:28.875734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694776+0000","last_change":"2023-12-01T01:37:23.200211+0000","last_active":"2023-12-01T01:37:23.694776+0000","last_peered":"2023-12-01T01:37:23.694776+0000","last_clean":"2023-12-01T01:37:23.694776+0000","last_became_active":"2023-12-01T01:37:23.199862+0000","last_became_peered":"2023-12-01T01:37:23.199862+0000","last_unstale":"2023-12-01T01:37:23.694776+0000","last_undegraded":"2023-12-01T01:37:23.694776+0000","last_fullsized":"2023-12-01T01:37:23.694776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:05:30.626410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.668725+0000","last_change":"2023-12-01T01:37:23.665736+0000","last_active":"2023-12-01T01:37:23.668725+0000","last_peered":"2023-12-01T01:37:23.668725+0000","last_clean":"2023-12-01T01:37:23.668725+0000","last_became_active":"2023-12-01T01:37:23.665479+0000","last_became_peered":"2023-12-01T01:37:23.665479+0000","last_unstale":"2023-12-01T01:37:23.668725+0000","last_undegraded":"2023-12-01T01:37:23.668725+0000","last_fullsized":"2023-12-01T01:37:23.668725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:28.569362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.693948+0000","last_change":"2023-12-01T01:37:23.193240+0000","last_active":"2023-12-01T01:37:23.693948+0000","last_peered":"2023-12-01T01:37:23.693948+0000","last_clean":"2023-12-01T01:37:23.693948+0000","last_became_active":"2023-12-01T01:37:23.192518+0000","last_became_peered":"2023-12-01T01:37:23.192518+0000","last_unstale":"2023-12-01T01:37:23.693948+0000","last_undegraded":"2023-12-01T01:37:23.693948+0000","last_fullsized":"2023-12-01T01:37:23.693948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:20:13.018107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.684092+0000","last_change":"2023-12-01T01:37:23.675898+0000","last_active":"2023-12-01T01:37:23.684092+0000","last_peered":"2023-12-01T01:37:23.684092+0000","last_clean":"2023-12-01T01:37:23.684092+0000","last_became_active":"2023-12-01T01:37:23.675553+0000","last_became_peered":"2023-12-01T01:37:23.675553+0000","last_unstale":"2023-12-01T01:37:23.684092+0000","last_undegraded":"2023-12-01T01:37:23.684092+0000","last_fullsized":"2023-12-01T01:37:23.684092+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:34.711426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.691251+0000","last_change":"2023-12-01T01:37:23.196668+0000","last_active":"2023-12-01T01:37:23.691251+0000","last_peered":"2023-12-01T01:37:23.691251+0000","last_clean":"2023-12-01T01:37:23.691251+0000","last_became_active":"2023-12-01T01:37:23.196227+0000","last_became_peered":"2023-12-01T01:37:23.196227+0000","last_unstale":"2023-12-01T01:37:23.691251+0000","last_undegraded":"2023-12-01T01:37:23.691251+0000","last_fullsized":"2023-12-01T01:37:23.691251+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:58:09.695875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696489+0000","last_change":"2023-12-01T01:37:23.686495+0000","last_active":"2023-12-01T01:37:23.696489+0000","last_peered":"2023-12-01T01:37:23.696489+0000","last_clean":"2023-12-01T01:37:23.696489+0000","last_became_active":"2023-12-01T01:37:23.686099+0000","last_became_peered":"2023-12-01T01:37:23.686099+0000","last_unstale":"2023-12-01T01:37:23.696489+0000","last_undegraded":"2023-12-01T01:37:23.696489+0000","last_fullsized":"2023-12-01T01:37:23.696489+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:26:44.405332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.660029+0000","last_change":"2023-12-01T01:37:23.198532+0000","last_active":"2023-12-01T01:37:23.660029+0000","last_peered":"2023-12-01T01:37:23.660029+0000","last_clean":"2023-12-01T01:37:23.660029+0000","last_became_active":"2023-12-01T01:37:23.197995+0000","last_became_peered":"2023-12-01T01:37:23.197995+0000","last_unstale":"2023-12-01T01:37:23.660029+0000","last_undegraded":"2023-12-01T01:37:23.660029+0000","last_fullsized":"2023-12-01T01:37:23.660029+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:58.522365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.687007+0000","last_change":"2023-12-01T01:37:23.680888+0000","last_active":"2023-12-01T01:37:23.687007+0000","last_peered":"2023-12-01T01:37:23.687007+0000","last_clean":"2023-12-01T01:37:23.687007+0000","last_became_active":"2023-12-01T01:37:23.680652+0000","last_became_peered":"2023-12-01T01:37:23.680652+0000","last_unstale":"2023-12-01T01:37:23.687007+0000","last_undegraded":"2023-12-01T01:37:23.687007+0000","last_fullsized":"2023-12-01T01:37:23.687007+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:58:10.711651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.695081+0000","last_change":"2023-12-01T01:37:23.201283+0000","last_active":"2023-12-01T01:37:23.695081+0000","last_peered":"2023-12-01T01:37:23.695081+0000","last_clean":"2023-12-01T01:37:23.695081+0000","last_became_active":"2023-12-01T01:37:23.198361+0000","last_became_peered":"2023-12-01T01:37:23.198361+0000","last_unstale":"2023-12-01T01:37:23.695081+0000","last_undegraded":"2023-12-01T01:37:23.695081+0000","last_fullsized":"2023-12-01T01:37:23.695081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:14.033784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.679039+0000","last_change":"2023-12-01T01:37:23.671853+0000","last_active":"2023-12-01T01:37:23.679039+0000","last_peered":"2023-12-01T01:37:23.679039+0000","last_clean":"2023-12-01T01:37:23.679039+0000","last_became_active":"2023-12-01T01:37:23.671579+0000","last_became_peered":"2023-12-01T01:37:23.671579+0000","last_unstale":"2023-12-01T01:37:23.679039+0000","last_undegraded":"2023-12-01T01:37:23.679039+0000","last_fullsized":"2023-12-01T01:37:23.679039+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:21.718145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.673041+0000","last_change":"2023-12-01T01:37:23.191984+0000","last_active":"2023-12-01T01:37:23.673041+0000","last_peered":"2023-12-01T01:37:23.673041+0000","last_clean":"2023-12-01T01:37:23.673041+0000","last_became_active":"2023-12-01T01:37:23.191288+0000","last_became_peered":"2023-12-01T01:37:23.191288+0000","last_unstale":"2023-12-01T01:37:23.673041+0000","last_undegraded":"2023-12-01T01:37:23.673041+0000","last_fullsized":"2023-12-01T01:37:23.673041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:07.036395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696313+0000","last_change":"2023-12-01T01:37:23.692851+0000","last_active":"2023-12-01T01:37:23.696313+0000","last_peered":"2023-12-01T01:37:23.696313+0000","last_clean":"2023-12-01T01:37:23.696313+0000","last_became_active":"2023-12-01T01:37:23.692252+0000","last_became_peered":"2023-12-01T01:37:23.692252+0000","last_unstale":"2023-12-01T01:37:23.696313+0000","last_undegraded":"2023-12-01T01:37:23.696313+0000","last_fullsized":"2023-12-01T01:37:23.696313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:47.357095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.657263+0000","last_change":"2023-12-01T01:37:23.197829+0000","last_active":"2023-12-01T01:37:23.657263+0000","last_peered":"2023-12-01T01:37:23.657263+0000","last_clean":"2023-12-01T01:37:23.657263+0000","last_became_active":"2023-12-01T01:37:23.197476+0000","last_became_peered":"2023-12-01T01:37:23.197476+0000","last_unstale":"2023-12-01T01:37:23.657263+0000","last_undegraded":"2023-12-01T01:37:23.657263+0000","last_fullsized":"2023-12-01T01:37:23.657263+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:12:11.572753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.668772+0000","last_change":"2023-12-01T01:37:23.665478+0000","last_active":"2023-12-01T01:37:23.668772+0000","last_peered":"2023-12-01T01:37:23.668772+0000","last_clean":"2023-12-01T01:37:23.668772+0000","last_became_active":"2023-12-01T01:37:23.664626+0000","last_became_peered":"2023-12-01T01:37:23.664626+0000","last_unstale":"2023-12-01T01:37:23.668772+0000","last_undegraded":"2023-12-01T01:37:23.668772+0000","last_fullsized":"2023-12-01T01:37:23.668772+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:20:14.033882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.667585+0000","last_change":"2023-12-01T01:37:23.197617+0000","last_active":"2023-12-01T01:37:23.667585+0000","last_peered":"2023-12-01T01:37:23.667585+0000","last_clean":"2023-12-01T01:37:23.667585+0000","last_became_active":"2023-12-01T01:37:23.197355+0000","last_became_peered":"2023-12-01T01:37:23.197355+0000","last_unstale":"2023-12-01T01:37:23.667585+0000","last_undegraded":"2023-12-01T01:37:23.667585+0000","last_fullsized":"2023-12-01T01:37:23.667585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:51.698929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694578+0000","last_change":"2023-12-01T01:37:23.688106+0000","last_active":"2023-12-01T01:37:23.694578+0000","last_peered":"2023-12-01T01:37:23.694578+0000","last_clean":"2023-12-01T01:37:23.694578+0000","last_became_active":"2023-12-01T01:37:23.687761+0000","last_became_peered":"2023-12-01T01:37:23.687761+0000","last_unstale":"2023-12-01T01:37:23.694578+0000","last_undegraded":"2023-12-01T01:37:23.694578+0000","last_fullsized":"2023-12-01T01:37:23.694578+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:59.538140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.685676+0000","last_change":"2023-12-01T01:37:23.200459+0000","last_active":"2023-12-01T01:37:23.685676+0000","last_peered":"2023-12-01T01:37:23.685676+0000","last_clean":"2023-12-01T01:37:23.685676+0000","last_became_active":"2023-12-01T01:37:23.200172+0000","last_became_peered":"2023-12-01T01:37:23.200172+0000","last_unstale":"2023-12-01T01:37:23.685676+0000","last_undegraded":"2023-12-01T01:37:23.685676+0000","last_fullsized":"2023-12-01T01:37:23.685676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:44.390008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696865+0000","last_change":"2023-12-01T01:37:23.696298+0000","last_active":"2023-12-01T01:37:23.696865+0000","last_peered":"2023-12-01T01:37:23.696865+0000","last_clean":"2023-12-01T01:37:23.696865+0000","last_became_active":"2023-12-01T01:37:23.694835+0000","last_became_peered":"2023-12-01T01:37:23.694835+0000","last_unstale":"2023-12-01T01:37:23.696865+0000","last_undegraded":"2023-12-01T01:37:23.696865+0000","last_fullsized":"2023-12-01T01:37:23.696865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:48:28.610023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672438+0000","last_change":"2023-12-01T01:37:23.200481+0000","last_active":"2023-12-01T01:37:23.672438+0000","last_peered":"2023-12-01T01:37:23.672438+0000","last_clean":"2023-12-01T01:37:23.672438+0000","last_became_active":"2023-12-01T01:37:23.199663+0000","last_became_peered":"2023-12-01T01:37:23.199663+0000","last_unstale":"2023-12-01T01:37:23.672438+0000","last_undegraded":"2023-12-01T01:37:23.672438+0000","last_fullsized":"2023-12-01T01:37:23.672438+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:11:33.410300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.696507+0000","last_change":"2023-12-01T01:37:23.691487+0000","last_active":"2023-12-01T01:37:23.696507+0000","last_peered":"2023-12-01T01:37:23.696507+0000","last_clean":"2023-12-01T01:37:23.696507+0000","last_became_active":"2023-12-01T01:37:23.691149+0000","last_became_peered":"2023-12-01T01:37:23.691149+0000","last_unstale":"2023-12-01T01:37:23.696507+0000","last_undegraded":"2023-12-01T01:37:23.696507+0000","last_fullsized":"2023-12-01T01:37:23.696507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:42:04.487553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.671026+0000","last_change":"2023-12-01T01:37:23.199357+0000","last_active":"2023-12-01T01:37:23.671026+0000","last_peered":"2023-12-01T01:37:23.671026+0000","last_clean":"2023-12-01T01:37:23.671026+0000","last_became_active":"2023-12-01T01:37:23.197994+0000","last_became_peered":"2023-12-01T01:37:23.197994+0000","last_unstale":"2023-12-01T01:37:23.671026+0000","last_undegraded":"2023-12-01T01:37:23.671026+0000","last_fullsized":"2023-12-01T01:37:23.671026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:27.553586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.686625+0000","last_change":"2023-12-01T01:37:23.684220+0000","last_active":"2023-12-01T01:37:23.686625+0000","last_peered":"2023-12-01T01:37:23.686625+0000","last_clean":"2023-12-01T01:37:23.686625+0000","last_became_active":"2023-12-01T01:37:23.683970+0000","last_became_peered":"2023-12-01T01:37:23.683970+0000","last_unstale":"2023-12-01T01:37:23.686625+0000","last_undegraded":"2023-12-01T01:37:23.686625+0000","last_fullsized":"2023-12-01T01:37:23.686625+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:36:28.074355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.664160+0000","last_change":"2023-12-01T01:37:23.198686+0000","last_active":"2023-12-01T01:37:23.664160+0000","last_peered":"2023-12-01T01:37:23.664160+0000","last_clean":"2023-12-01T01:37:23.664160+0000","last_became_active":"2023-12-01T01:37:23.197993+0000","last_became_peered":"2023-12-01T01:37:23.197993+0000","last_unstale":"2023-12-01T01:37:23.664160+0000","last_undegraded":"2023-12-01T01:37:23.664160+0000","last_fullsized":"2023-12-01T01:37:23.664160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:42.327013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694399+0000","last_change":"2023-12-01T01:37:23.686092+0000","last_active":"2023-12-01T01:37:23.694399+0000","last_peered":"2023-12-01T01:37:23.694399+0000","last_clean":"2023-12-01T01:37:23.694399+0000","last_became_active":"2023-12-01T01:37:23.685733+0000","last_became_peered":"2023-12-01T01:37:23.685733+0000","last_unstale":"2023-12-01T01:37:23.694399+0000","last_undegraded":"2023-12-01T01:37:23.694399+0000","last_fullsized":"2023-12-01T01:37:23.694399+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:03:52.184710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.665612+0000","last_change":"2023-12-01T01:37:23.203314+0000","last_active":"2023-12-01T01:37:23.665612+0000","last_peered":"2023-12-01T01:37:23.665612+0000","last_clean":"2023-12-01T01:37:23.665612+0000","last_became_active":"2023-12-01T01:37:23.203060+0000","last_became_peered":"2023-12-01T01:37:23.203060+0000","last_unstale":"2023-12-01T01:37:23.665612+0000","last_undegraded":"2023-12-01T01:37:23.665612+0000","last_fullsized":"2023-12-01T01:37:23.665612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:20:13.018107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.694521+0000","last_change":"2023-12-01T01:37:23.687528+0000","last_active":"2023-12-01T01:37:23.694521+0000","last_peered":"2023-12-01T01:37:23.694521+0000","last_clean":"2023-12-01T01:37:23.694521+0000","last_became_active":"2023-12-01T01:37:23.682496+0000","last_became_peered":"2023-12-01T01:37:23.682496+0000","last_unstale":"2023-12-01T01:37:23.694521+0000","last_undegraded":"2023-12-01T01:37:23.694521+0000","last_fullsized":"2023-12-01T01:37:23.694521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:18.237392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672574+0000","last_change":"2023-12-01T01:37:23.194677+0000","last_active":"2023-12-01T01:37:23.672574+0000","last_peered":"2023-12-01T01:37:23.672574+0000","last_clean":"2023-12-01T01:37:23.672574+0000","last_became_active":"2023-12-01T01:37:23.194349+0000","last_became_peered":"2023-12-01T01:37:23.194349+0000","last_unstale":"2023-12-01T01:37:23.672574+0000","last_undegraded":"2023-12-01T01:37:23.672574+0000","last_fullsized":"2023-12-01T01:37:23.672574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:47.125619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.692408+0000","last_change":"2023-12-01T01:37:23.686716+0000","last_active":"2023-12-01T01:37:23.692408+0000","last_peered":"2023-12-01T01:37:23.692408+0000","last_clean":"2023-12-01T01:37:23.692408+0000","last_became_active":"2023-12-01T01:37:23.686380+0000","last_became_peered":"2023-12-01T01:37:23.686380+0000","last_unstale":"2023-12-01T01:37:23.692408+0000","last_undegraded":"2023-12-01T01:37:23.692408+0000","last_fullsized":"2023-12-01T01:37:23.692408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:53:09.949391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.668609+0000","last_change":"2023-12-01T01:37:23.193514+0000","last_active":"2023-12-01T01:37:23.668609+0000","last_peered":"2023-12-01T01:37:23.668609+0000","last_clean":"2023-12-01T01:37:23.668609+0000","last_became_active":"2023-12-01T01:37:23.192427+0000","last_became_peered":"2023-12-01T01:37:23.192427+0000","last_unstale":"2023-12-01T01:37:23.668609+0000","last_undegraded":"2023-12-01T01:37:23.668609+0000","last_fullsized":"2023-12-01T01:37:23.668609+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:18.785910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.682398+0000","last_change":"2023-12-01T01:37:23.674887+0000","last_active":"2023-12-01T01:37:23.682398+0000","last_peered":"2023-12-01T01:37:23.682398+0000","last_clean":"2023-12-01T01:37:23.682398+0000","last_became_active":"2023-12-01T01:37:23.674710+0000","last_became_peered":"2023-12-01T01:37:23.674710+0000","last_unstale":"2023-12-01T01:37:23.682398+0000","last_undegraded":"2023-12-01T01:37:23.682398+0000","last_fullsized":"2023-12-01T01:37:23.682398+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:23.148185+0000","last_clean_scrub_stamp":"2023-12-01T01:37:23.148185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:42.155049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.672739+0000","last_change":"2023-12-01T01:37:23.196555+0000","last_active":"2023-12-01T01:37:23.672739+0000","last_peered":"2023-12-01T01:37:23.672739+0000","last_clean":"2023-12-01T01:37:23.672739+0000","last_became_active":"2023-12-01T01:37:23.196227+0000","last_became_peered":"2023-12-01T01:37:23.196227+0000","last_unstale":"2023-12-01T01:37:23.672739+0000","last_undegraded":"2023-12-01T01:37:23.672739+0000","last_fullsized":"2023-12-01T01:37:23.672739+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:37:22.132409+0000","last_clean_scrub_stamp":"2023-12-01T01:37:22.132409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:13.832258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-01T01:37:23.188459+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:37:23.188459+0000","last_peered":"2023-12-01T01:37:23.188459+0000","last_clean":"2023-12-01T01:37:23.188459+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:37:23.188459+0000","last_undegraded":"2023-12-01T01:37:23.188459+0000","last_fullsized":"2023-12-01T01:37:23.188459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":372736,"data_stored":142997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.212},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.78100000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.873},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.077}]},{"osd":3,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.3180000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.258},{"interface":"front","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.419}]},{"osd":5,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.3680000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:37:04 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.016},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":1.139,"5min":1.139,"15min":1.139},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":1.1220000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":151336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.0369999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.113},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.96699999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.5620000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":1.304}]},{"osd":5,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.399},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.4399999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.613,"15min":1.613},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.651,"5min":1.651,"15min":1.651},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.6419999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":740722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":3.3980000000000001,"5min":3.3980000000000001,"15min":3.3980000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.0129999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.206},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.96899999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.0669999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":1.349},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.466}]},{"osd":6,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.161},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":1.25}]},{"osd":7,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":3.012,"5min":3.012,"15min":3.012},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.51}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":151358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":3.8069999999999999,"5min":3.8069999999999999,"15min":3.8069999999999999},"last":1.298}]},{"osd":1,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1180000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.0800000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.518}]},{"osd":5,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.357}]},{"osd":6,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.401}]},{"osd":7,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.575,"15min":1.575},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":3.2789999999999999,"5min":3.2789999999999999,"15min":3.2789999999999999},"last":1.256},{"interface":"front","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.663}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":150548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.93500000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1819999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.9350000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":1.046},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.3999999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":3.476,"5min":3.476,"15min":3.476},"last":1.272},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":1.1080000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.534},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.621}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":149806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.179}]},{"osd":1,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.2749999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":2.6080000000000001,"5min":2.6080000000000001,"15min":2.6080000000000001},"last":1.4510000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.708},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.8440000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":3.0099999999999998,"5min":3.0099999999999998,"15min":3.0099999999999998},"last":1.635}]},{"osd":6,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.7609999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.3859999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.796},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.9470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":740210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.056,"5min":3.056,"15min":3.056},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.95499999999999996}]},{"osd":2,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.3660000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":1.0629999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.464,"5min":3.464,"15min":3.464},"last":1.302},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.0960000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.0269999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":3.7810000000000001,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":1.2190000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":3.2839999999999998,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":1.147},{"interface":"front","average":{"1min":1.609,"5min":1.609,"15min":1.609},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":3.6669999999999998,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.575,"15min":1.575},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":3.7400000000000002,"5min":3.7400000000000002,"15min":3.7400000000000002},"last":1.008}]},{"osd":7,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":4.6749999999999998,"5min":4.6749999999999998,"15min":4.6749999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":3.5379999999999998,"5min":3.5379999999999998,"15min":3.5379999999999998},"last":1.339}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":149862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.0600000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":2.8159999999999998},"last":0.95399999999999996}]},{"osd":3,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.105}]},{"osd":4,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.077}]},{"osd":5,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.173}]},{"osd":6,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.3939999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.8999999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.504,"15min":1.504},"min":{"1min":1.05,"5min":1.05,"15min":1.05},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.298}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:37:28.526 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T01:37:28.527 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T01:37:28.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.786+0000 7fe79d58e640 1 Processor -- start 2023-12-01T01:37:28.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.786+0000 7fe79d58e640 1 -- start start 2023-12-01T01:37:28.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.787+0000 7fe79d58e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7981552b0 0x7fe7980ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.787+0000 7fe79d58e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe79806c1c0 0x7fe7980ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.787+0000 7fe79d58e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79806b400 con 0x7fe7980af700 2023-12-01T01:37:28.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.787+0000 7fe79d58e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe798155690 con 0x7fe79806c1c0 2023-12-01T01:37:28.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.787+0000 7fe79d58e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79806bda0 con 0x7fe7981552b0 2023-12-01T01:37:28.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.788+0000 7fe796ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7981552b0 0x7fe7980ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:28.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.788+0000 7fe796ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7981552b0 0x7fe7980ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:60876/0 (socket says 172.21.15.115:60876) 2023-12-01T01:37:28.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.788+0000 7fe796ffd640 1 -- 172.21.15.115:0/3953027592 learned_addr learned my addr 172.21.15.115:0/3953027592 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:28.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.788+0000 7fe78ffff640 1 --2- 172.21.15.115:0/3953027592 >> v2:172.21.15.142:3300/0 conn(0x7fe79806c1c0 0x7fe7980ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:28.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.789+0000 7fe796ffd640 1 -- 172.21.15.115:0/3953027592 >> v2:172.21.15.142:3300/0 conn(0x7fe79806c1c0 msgr2=0x7fe7980ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:28.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.789+0000 7fe796ffd640 1 --2- 172.21.15.115:0/3953027592 >> v2:172.21.15.142:3300/0 conn(0x7fe79806c1c0 0x7fe7980ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.789+0000 7fe796ffd640 1 -- 172.21.15.115:0/3953027592 >> v1:172.21.15.142:6789/0 conn(0x7fe7980af700 legacy=0x7fe7980afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.789+0000 7fe796ffd640 1 -- 172.21.15.115:0/3953027592 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79806b6e0 con 0x7fe7981552b0 2023-12-01T01:37:28.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.790+0000 7fe796ffd640 1 --2- 172.21.15.115:0/3953027592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7981552b0 0x7fe7980ce760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe788009870 tx=0x7fe78802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d8dc6f217fecd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:28.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.790+0000 7fe7967fc640 1 -- 172.21.15.115:0/3953027592 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788030840 con 0x7fe7981552b0 2023-12-01T01:37:28.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.790+0000 7fe7967fc640 1 -- 172.21.15.115:0/3953027592 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7880309a0 con 0x7fe7981552b0 2023-12-01T01:37:28.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.790+0000 7fe7967fc640 1 -- 172.21.15.115:0/3953027592 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788030d30 con 0x7fe7981552b0 2023-12-01T01:37:28.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.791+0000 7fe79d58e640 1 -- 172.21.15.115:0/3953027592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7981552b0 msgr2=0x7fe7980ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.791+0000 7fe79d58e640 1 --2- 172.21.15.115:0/3953027592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7981552b0 0x7fe7980ce760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe788009870 tx=0x7fe78802b980 comp rx=0 tx=0).stop 2023-12-01T01:37:28.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.791+0000 7fe79d58e640 1 -- 172.21.15.115:0/3953027592 shutdown_connections 2023-12-01T01:37:28.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.791+0000 7fe79d58e640 1 --2- 172.21.15.115:0/3953027592 >> v2:172.21.15.142:3300/0 conn(0x7fe79806c1c0 0x7fe7980ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.791+0000 7fe79d58e640 1 --2- 172.21.15.115:0/3953027592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7981552b0 0x7fe7980ce760 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.791+0000 7fe79d58e640 1 -- 172.21.15.115:0/3953027592 >> 172.21.15.115:0/3953027592 conn(0x7fe798065960 msgr2=0x7fe798065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:28.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.791+0000 7fe79d58e640 1 -- 172.21.15.115:0/3953027592 shutdown_connections 2023-12-01T01:37:28.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.791+0000 7fe79d58e640 1 -- 172.21.15.115:0/3953027592 wait complete. 2023-12-01T01:37:28.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.793+0000 7fe79d58e640 1 Processor -- start 2023-12-01T01:37:28.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.793+0000 7fe79d58e640 1 -- start start 2023-12-01T01:37:28.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.794+0000 7fe79d58e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe79806c1c0 0x7fe7980a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.794+0000 7fe79d58e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7980af700 0x7fe79814c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.794+0000 7fe79d58e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79814e330 0x7fe7980aa720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.794+0000 7fe79d58e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe798155bb0 con 0x7fe7980af700 2023-12-01T01:37:28.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.794+0000 7fe79d58e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980a9450 con 0x7fe79814e330 2023-12-01T01:37:28.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.794+0000 7fe79d58e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980a9730 con 0x7fe79806c1c0 2023-12-01T01:37:28.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.794+0000 7fe78ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7980af700 0x7fe79814c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:28.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.794+0000 7fe78ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7980af700 0x7fe79814c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56652/0 (socket says 172.21.15.115:56652) 2023-12-01T01:37:28.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.794+0000 7fe78ffff640 1 -- 172.21.15.115:0/1643650538 learned_addr learned my addr 172.21.15.115:0/1643650538 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:28.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe7977fe640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79814e330 0x7fe7980aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:28.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe796ffd640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe79806c1c0 0x7fe7980a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:28.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe78ffff640 1 -- 172.21.15.115:0/1643650538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe79806c1c0 msgr2=0x7fe7980a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe78ffff640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe79806c1c0 0x7fe7980a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe78ffff640 1 -- 172.21.15.115:0/1643650538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79814e330 msgr2=0x7fe7980aa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:28.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe78ffff640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79814e330 0x7fe7980aa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:28.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe78ffff640 1 -- 172.21.15.115:0/1643650538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7880094d0 con 0x7fe7980af700 2023-12-01T01:37:28.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe78ffff640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7980af700 0x7fe79814c820 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe78000e9b0 tx=0x7fe78000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:28.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe78f7fe640 1 -- 172.21.15.115:0/1643650538 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780009670 con 0x7fe7980af700 2023-12-01T01:37:28.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe78f7fe640 1 -- 172.21.15.115:0/1643650538 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78000c9a0 con 0x7fe7980af700 2023-12-01T01:37:28.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe78f7fe640 1 -- 172.21.15.115:0/1643650538 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78000cd50 con 0x7fe7980af700 2023-12-01T01:37:28.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.795+0000 7fe79d58e640 1 -- 172.21.15.115:0/1643650538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7980a9a10 con 0x7fe7980af700 2023-12-01T01:37:28.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.796+0000 7fe79d58e640 1 -- 172.21.15.115:0/1643650538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79814e070 con 0x7fe7980af700 2023-12-01T01:37:28.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.797+0000 7fe79d58e640 1 -- 172.21.15.115:0/1643650538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79804e890 con 0x7fe7980af700 2023-12-01T01:37:28.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.798+0000 7fe78f7fe640 1 -- 172.21.15.115:0/1643650538 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe780013470 con 0x7fe7980af700 2023-12-01T01:37:28.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.798+0000 7fe78f7fe640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe77806a460 0x7fe77806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:28.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.798+0000 7fe78f7fe640 1 -- 172.21.15.115:0/1643650538 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe780088be0 con 0x7fe7980af700 2023-12-01T01:37:28.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.799+0000 7fe796ffd640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe77806a460 0x7fe77806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:28.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.799+0000 7fe796ffd640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe77806a460 0x7fe77806c920 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe788036000 tx=0x7fe788008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:28.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:28.803+0000 7fe78f7fe640 1 -- 172.21.15.115:0/1643650538 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe7800541d0 con 0x7fe7980af700 2023-12-01T01:37:29.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.041+0000 7fe79d58e640 1 -- 172.21.15.115:0/1643650538 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe79806b990 con 0x7fe7980af700 2023-12-01T01:37:29.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.050+0000 7fe78f7fe640 1 -- 172.21.15.115:0/1643650538 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe7800170e0 con 0x7fe7980af700 2023-12-01T01:37:29.050 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T01:37:29.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.052+0000 7fe79d58e640 1 -- 172.21.15.115:0/1643650538 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe77806a460 msgr2=0x7fe77806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:29.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.052+0000 7fe79d58e640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe77806a460 0x7fe77806c920 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe788036000 tx=0x7fe788008030 comp rx=0 tx=0).stop 2023-12-01T01:37:29.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.052+0000 7fe79d58e640 1 -- 172.21.15.115:0/1643650538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7980af700 msgr2=0x7fe79814c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:29.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.052+0000 7fe79d58e640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7980af700 0x7fe79814c820 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe78000e9b0 tx=0x7fe78000ee70 comp rx=0 tx=0).stop 2023-12-01T01:37:29.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.053+0000 7fe79d58e640 1 -- 172.21.15.115:0/1643650538 shutdown_connections 2023-12-01T01:37:29.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.053+0000 7fe79d58e640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe77806a460 0x7fe77806c920 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.053+0000 7fe79d58e640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe79814e330 0x7fe7980aa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.053+0000 7fe79d58e640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7980af700 0x7fe79814c820 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.053+0000 7fe79d58e640 1 --2- 172.21.15.115:0/1643650538 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe79806c1c0 0x7fe7980a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.053+0000 7fe79d58e640 1 -- 172.21.15.115:0/1643650538 >> 172.21.15.115:0/1643650538 conn(0x7fe798065960 msgr2=0x7fe798159490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:29.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.053+0000 7fe79d58e640 1 -- 172.21.15.115:0/1643650538 shutdown_connections 2023-12-01T01:37:29.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.053+0000 7fe79d58e640 1 -- 172.21.15.115:0/1643650538 wait complete. 2023-12-01T01:37:29.072 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:37:29.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.281+0000 7ff3d9a28640 1 Processor -- start 2023-12-01T01:37:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.281+0000 7ff3d9a28640 1 -- start start 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.282+0000 7ff3d9a28640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3d40c9ef0 0x7ff3d406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.282+0000 7ff3d9a28640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3d406b6f0 0x7ff3d4070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.282+0000 7ff3d9a28640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d406f8b0 con 0x7ff3d4159dd0 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.282+0000 7ff3d9a28640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d40adcb0 con 0x7ff3d406b6f0 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.282+0000 7ff3d9a28640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40ad9d0 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.283+0000 7ff3d37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3d406b6f0 0x7ff3d4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.283+0000 7ff3d37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3d406b6f0 0x7ff3d4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56666/0 (socket says 172.21.15.115:56666) 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.283+0000 7ff3d37fe640 1 -- 172.21.15.115:0/2567801935 learned_addr learned my addr 172.21.15.115:0/2567801935 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.283+0000 7ff3d27fc640 1 --2- 172.21.15.115:0/2567801935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3d40c9ef0 0x7ff3d406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.283+0000 7ff3d37fe640 1 -- 172.21.15.115:0/2567801935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3d40c9ef0 msgr2=0x7ff3d406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.283+0000 7ff3d37fe640 1 --2- 172.21.15.115:0/2567801935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3d40c9ef0 0x7ff3d406fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.283+0000 7ff3d37fe640 1 -- 172.21.15.115:0/2567801935 >> v1:172.21.15.142:6789/0 conn(0x7ff3d4159dd0 legacy=0x7ff3d40c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.283+0000 7ff3d37fe640 1 -- 172.21.15.115:0/2567801935 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40c9000 con 0x7ff3d406b6f0 2023-12-01T01:37:29.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.284+0000 7ff3d37fe640 1 --2- 172.21.15.115:0/2567801935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d406b6f0 0x7ff3d4070260 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009e50 tx=0x7ff3cc02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b1fbd72d52cf39 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:29.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.285+0000 7ff3d1ffb640 1 -- 172.21.15.115:0/2567801935 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc02c950 con 0x7ff3d406b6f0 2023-12-01T01:37:29.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.285+0000 7ff3d1ffb640 1 -- 172.21.15.115:0/2567801935 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3cc02cab0 con 0x7ff3d406b6f0 2023-12-01T01:37:29.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.285+0000 7ff3d1ffb640 1 -- 172.21.15.115:0/2567801935 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc02ce40 con 0x7ff3d406b6f0 2023-12-01T01:37:29.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.285+0000 7ff3d9a28640 1 -- 172.21.15.115:0/2567801935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d406b6f0 msgr2=0x7ff3d4070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:29.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.285+0000 7ff3d9a28640 1 --2- 172.21.15.115:0/2567801935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d406b6f0 0x7ff3d4070260 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009e50 tx=0x7ff3cc02bdb0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.285+0000 7ff3d9a28640 1 -- 172.21.15.115:0/2567801935 shutdown_connections 2023-12-01T01:37:29.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.285+0000 7ff3d9a28640 1 --2- 172.21.15.115:0/2567801935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d406b6f0 0x7ff3d4070260 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.285+0000 7ff3d9a28640 1 --2- 172.21.15.115:0/2567801935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3d40c9ef0 0x7ff3d406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.286+0000 7ff3d9a28640 1 -- 172.21.15.115:0/2567801935 >> 172.21.15.115:0/2567801935 conn(0x7ff3d40ce880 msgr2=0x7ff3d40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:29.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.286+0000 7ff3d9a28640 1 -- 172.21.15.115:0/2567801935 shutdown_connections 2023-12-01T01:37:29.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.286+0000 7ff3d9a28640 1 -- 172.21.15.115:0/2567801935 wait complete. 2023-12-01T01:37:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.287+0000 7ff3d9a28640 1 Processor -- start 2023-12-01T01:37:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.287+0000 7ff3d9a28640 1 -- start start 2023-12-01T01:37:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.288+0000 7ff3d9a28640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3d406b6f0 0x7ff3d40c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.288+0000 7ff3d9a28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d40c9ef0 0x7ff3d414ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.288+0000 7ff3d9a28640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d4159dd0 0x7ff3d41e24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.288+0000 7ff3d9a28640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d414b290 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.288+0000 7ff3d9a28640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d41e5640 con 0x7ff3d4159dd0 2023-12-01T01:37:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.288+0000 7ff3d9a28640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d41e29f0 con 0x7ff3d406b6f0 2023-12-01T01:37:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d27fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d40c9ef0 0x7ff3d414ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d27fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d40c9ef0 0x7ff3d414ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56680/0 (socket says 172.21.15.115:56680) 2023-12-01T01:37:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d27fc640 1 -- 172.21.15.115:0/568271755 learned_addr learned my addr 172.21.15.115:0/568271755 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d37fe640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d4159dd0 0x7ff3d41e24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d27fc640 1 -- 172.21.15.115:0/568271755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3d406b6f0 msgr2=0x7ff3d40c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d2ffd640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3d406b6f0 0x7ff3d40c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d27fc640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3d406b6f0 0x7ff3d40c26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d27fc640 1 -- 172.21.15.115:0/568271755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d4159dd0 msgr2=0x7ff3d41e24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d27fc640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d4159dd0 0x7ff3d41e24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d27fc640 1 -- 172.21.15.115:0/568271755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc009b30 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d37fe640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d4159dd0 0x7ff3d41e24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.289+0000 7ff3d27fc640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d40c9ef0 0x7ff3d414ad50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800d800 tx=0x7ff3c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.290+0000 7ff3bbfff640 1 -- 172.21.15.115:0/568271755 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800b660 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.290+0000 7ff3bbfff640 1 -- 172.21.15.115:0/568271755 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c800b7c0 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.290+0000 7ff3bbfff640 1 -- 172.21.15.115:0/568271755 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800bb50 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.290+0000 7ff3d9a28640 1 -- 172.21.15.115:0/568271755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d40c51f0 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.290+0000 7ff3d9a28640 1 -- 172.21.15.115:0/568271755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d41e4ff0 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.291+0000 7ff3d9a28640 1 -- 172.21.15.115:0/568271755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d404e890 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.292+0000 7ff3bbfff640 1 -- 172.21.15.115:0/568271755 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff3c800bcb0 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.293+0000 7ff3bbfff640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff3c006a430 0x7ff3c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:29.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.293+0000 7ff3bbfff640 1 -- 172.21.15.115:0/568271755 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3c8088e50 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.293+0000 7ff3d2ffd640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff3c006a430 0x7ff3c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:29.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.294+0000 7ff3d2ffd640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff3c006a430 0x7ff3c006c8f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400a750 tx=0x7ff3c4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.297+0000 7ff3bbfff640 1 -- 172.21.15.115:0/568271755 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff3c8054430 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.525+0000 7ff3d9a28640 1 -- 172.21.15.115:0/568271755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3d40cd140 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.527+0000 7ff3bbfff640 1 -- 172.21.15.115:0/568271755 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7ff3c8057a50 con 0x7ff3d40c9ef0 2023-12-01T01:37:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:37:29.527 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1226717603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1226717603},{"type":"v1","addr":"172.21.15.142:6834","nonce":1226717603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2864659456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2864659456},{"type":"v1","addr":"172.21.15.142:6835","nonce":2864659456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4774,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/253656345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":253656345},{"type":"v1","addr":"172.21.15.115:6837","nonce":253656345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:37:23.645235+0000","modified":"2023-12-01T01:37:26.663085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5091},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5091":{"gid":5091,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6835/4242588071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4242588071},{"type":"v1","addr":"172.21.15.115:6835","nonce":4242588071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-01T01:37:29.528 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2023-12-01T01:37:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 -- 172.21.15.115:0/568271755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff3c006a430 msgr2=0x7ff3c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff3c006a430 0x7ff3c006c8f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400a750 tx=0x7ff3c4008040 comp rx=0 tx=0).stop 2023-12-01T01:37:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 -- 172.21.15.115:0/568271755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d40c9ef0 msgr2=0x7ff3d414ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d40c9ef0 0x7ff3d414ad50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800d800 tx=0x7ff3c800dcc0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 -- 172.21.15.115:0/568271755 shutdown_connections 2023-12-01T01:37:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff3c006a430 0x7ff3c006c8f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d4159dd0 0x7ff3d41e24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3d40c9ef0 0x7ff3d414ad50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 --2- 172.21.15.115:0/568271755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3d406b6f0 0x7ff3d40c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 -- 172.21.15.115:0/568271755 >> 172.21.15.115:0/568271755 conn(0x7ff3d40ce880 msgr2=0x7ff3d4158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 -- 172.21.15.115:0/568271755 shutdown_connections 2023-12-01T01:37:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:29.529+0000 7ff3d9a28640 1 -- 172.21.15.115:0/568271755 wait complete. 2023-12-01T01:37:29.949 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T01:37:23.645235+0000', 'modified': '2023-12-01T01:37:26.663085+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5091}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5091': {'gid': 5091, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.115:6835/4242588071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6834', 'nonce': 4242588071}, {'type': 'v1', 'addr': '172.21.15.115:6835', 'nonce': 4242588071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T01:37:29.950 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T01:37:29.950 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-01T01:37:30.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.168+0000 7f384975c640 1 Processor -- start 2023-12-01T01:37:30.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.169+0000 7f384975c640 1 -- start start 2023-12-01T01:37:30.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.170+0000 7f384975c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f384415cd20 0x7f38440d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.170+0000 7f384975c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f38440d1230 0x7f38440d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.170+0000 7f384975c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38440d1b90 con 0x7f38440b5d80 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.170+0000 7f384975c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38440b2160 con 0x7f384415cd20 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.170+0000 7f384975c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38440b8c80 con 0x7f38440d1230 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.171+0000 7f3842ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f384415cd20 0x7f38440d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.171+0000 7f3842ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f384415cd20 0x7f38440d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56694/0 (socket says 172.21.15.115:56694) 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.171+0000 7f38427fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f38440b5d80 0x7f38440b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51942/0 (socket says 172.21.15.115:51942) 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.171+0000 7f3842ffd640 1 -- 172.21.15.115:0/3293646999 learned_addr learned my addr 172.21.15.115:0/3293646999 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.171+0000 7f38437fe640 1 --2- 172.21.15.115:0/3293646999 >> v2:172.21.15.142:3300/0 conn(0x7f38440d1230 0x7f38440d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.171+0000 7f3842ffd640 1 -- 172.21.15.115:0/3293646999 >> v2:172.21.15.142:3300/0 conn(0x7f38440d1230 msgr2=0x7f38440d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.171+0000 7f3842ffd640 1 --2- 172.21.15.115:0/3293646999 >> v2:172.21.15.142:3300/0 conn(0x7f38440d1230 0x7f38440d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.171+0000 7f3842ffd640 1 -- 172.21.15.115:0/3293646999 >> v1:172.21.15.115:6789/0 conn(0x7f38440b5d80 legacy=0x7f38440b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.171+0000 7f3842ffd640 1 -- 172.21.15.115:0/3293646999 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38440ce2e0 con 0x7f384415cd20 2023-12-01T01:37:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.171+0000 7f3841ffb640 1 -- 172.21.15.115:0/3293646999 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409128470 0 0) 0x7f382c001110 con 0x7f38440b5d80 2023-12-01T01:37:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.172+0000 7f3842ffd640 1 --2- 172.21.15.115:0/3293646999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f384415cd20 0x7f38440d24d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f38440c6720 tx=0x7f383402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3f1c497fb99276 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.173+0000 7f3841ffb640 1 -- 172.21.15.115:0/3293646999 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383402ca80 con 0x7f384415cd20 2023-12-01T01:37:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.173+0000 7f3841ffb640 1 -- 172.21.15.115:0/3293646999 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383402cbe0 con 0x7f384415cd20 2023-12-01T01:37:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.173+0000 7f3841ffb640 1 -- 172.21.15.115:0/3293646999 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38340343f0 con 0x7f384415cd20 2023-12-01T01:37:30.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.173+0000 7f384975c640 1 -- 172.21.15.115:0/3293646999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f384415cd20 msgr2=0x7f38440d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.173+0000 7f384975c640 1 --2- 172.21.15.115:0/3293646999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f384415cd20 0x7f38440d24d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f38440c6720 tx=0x7f383402bee0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.173+0000 7f384975c640 1 -- 172.21.15.115:0/3293646999 shutdown_connections 2023-12-01T01:37:30.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.173+0000 7f384975c640 1 --2- 172.21.15.115:0/3293646999 >> v2:172.21.15.142:3300/0 conn(0x7f38440d1230 0x7f38440d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.173+0000 7f384975c640 1 --2- 172.21.15.115:0/3293646999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f384415cd20 0x7f38440d24d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.173+0000 7f384975c640 1 -- 172.21.15.115:0/3293646999 >> 172.21.15.115:0/3293646999 conn(0x7f384406a0e0 msgr2=0x7f384406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:30.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.174+0000 7f384975c640 1 -- 172.21.15.115:0/3293646999 shutdown_connections 2023-12-01T01:37:30.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.174+0000 7f384975c640 1 -- 172.21.15.115:0/3293646999 wait complete. 2023-12-01T01:37:30.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.175+0000 7f384975c640 1 Processor -- start 2023-12-01T01:37:30.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.176+0000 7f384975c640 1 -- start start 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.176+0000 7f384975c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38440b5d80 0x7f38441f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f384975c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38440d1230 0x7f384414c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f384975c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f384415cd20 0x7f384414cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f384975c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384415eb20 con 0x7f38440d1230 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f384975c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384415e840 con 0x7f384415cd20 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f384975c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38440a9770 con 0x7f38440b5d80 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f38427fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38440d1230 0x7f384414c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f38427fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38440d1230 0x7f384414c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56698/0 (socket says 172.21.15.115:56698) 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f38427fc640 1 -- 172.21.15.115:0/1308425966 learned_addr learned my addr 172.21.15.115:0/1308425966 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f38427fc640 1 -- 172.21.15.115:0/1308425966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38440b5d80 msgr2=0x7f38441f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f38427fc640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38440b5d80 0x7f38441f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f38427fc640 1 -- 172.21.15.115:0/1308425966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f384415cd20 msgr2=0x7f384414cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f38437fe640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f384415cd20 0x7f384414cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f38427fc640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f384415cd20 0x7f384414cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.177+0000 7f38427fc640 1 -- 172.21.15.115:0/1308425966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3834009b30 con 0x7f38440d1230 2023-12-01T01:37:30.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.178+0000 7f38427fc640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38440d1230 0x7f384414c810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c780 tx=0x7f382c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:30.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.178+0000 7f383b7fe640 1 -- 172.21.15.115:0/1308425966 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c0072d0 con 0x7f38440d1230 2023-12-01T01:37:30.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.178+0000 7f383b7fe640 1 -- 172.21.15.115:0/1308425966 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c007450 con 0x7f38440d1230 2023-12-01T01:37:30.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.178+0000 7f383b7fe640 1 -- 172.21.15.115:0/1308425966 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c0047e0 con 0x7f38440d1230 2023-12-01T01:37:30.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.178+0000 7f384975c640 1 -- 172.21.15.115:0/1308425966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38440a9a50 con 0x7f38440d1230 2023-12-01T01:37:30.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.178+0000 7f384975c640 1 -- 172.21.15.115:0/1308425966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384414dc70 con 0x7f38440d1230 2023-12-01T01:37:30.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.180+0000 7f383b7fe640 1 -- 172.21.15.115:0/1308425966 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f382c004960 con 0x7f38440d1230 2023-12-01T01:37:30.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.181+0000 7f383b7fe640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f382406a430 0x7f382406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.181+0000 7f383b7fe640 1 -- 172.21.15.115:0/1308425966 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f382c087a90 con 0x7f38440d1230 2023-12-01T01:37:30.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.181+0000 7f3842ffd640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f382406a430 0x7f382406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.182+0000 7f3842ffd640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f382406a430 0x7f382406c8f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3834035000 tx=0x7f383402e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:30.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.182+0000 7f384c3ae740 1 Processor -- start 2023-12-01T01:37:30.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.182+0000 7f384c3ae740 1 -- start start 2023-12-01T01:37:30.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.183+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c78d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.183+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c488a0 0x562f58c75b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.183+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 0x562f58c760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.183+0000 7f384c3ae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f58c242f0 con 0x562f58c78920 2023-12-01T01:37:30.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.183+0000 7f384c3ae740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f58bb8f80 con 0x562f58c4c4a0 2023-12-01T01:37:30.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.183+0000 7f384c3ae740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f58bb6710 con 0x562f58c488a0 2023-12-01T01:37:30.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.184+0000 7f3842ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c78d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.184+0000 7f3842ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c78d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56700/0 (socket says 172.21.15.115:56700) 2023-12-01T01:37:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.184+0000 7f3842ffd640 1 -- 172.21.15.115:0/2418348529 learned_addr learned my addr 172.21.15.115:0/2418348529 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:30.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.184+0000 7f38437fe640 1 --2- 172.21.15.115:0/2418348529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 0x562f58c760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.184+0000 7f38427fc640 1 --2- 172.21.15.115:0/2418348529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c488a0 0x562f58c75b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.184+0000 7f3842ffd640 1 -- 172.21.15.115:0/2418348529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c488a0 msgr2=0x562f58c75b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.184+0000 7f3842ffd640 1 --2- 172.21.15.115:0/2418348529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c488a0 0x562f58c75b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.184+0000 7f3842ffd640 1 -- 172.21.15.115:0/2418348529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 msgr2=0x562f58c760d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.184+0000 7f3842ffd640 1 --2- 172.21.15.115:0/2418348529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 0x562f58c760d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.184+0000 7f3842ffd640 1 -- 172.21.15.115:0/2418348529 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f58b79690 con 0x562f58c78920 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.185+0000 7f3842ffd640 1 --2- 172.21.15.115:0/2418348529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c78d00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3834036040 tx=0x7f3834009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9b5695e2378102 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.185+0000 7f3838ff9640 1 -- 172.21.15.115:0/2418348529 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383404e070 con 0x562f58c78920 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.185+0000 7f3838ff9640 1 -- 172.21.15.115:0/2418348529 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834046040 con 0x562f58c78920 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.185+0000 7f3838ff9640 1 -- 172.21.15.115:0/2418348529 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834040b90 con 0x562f58c78920 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.185+0000 7f384c3ae740 1 -- 172.21.15.115:0/2418348529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 msgr2=0x562f58c78d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.185+0000 7f384c3ae740 1 --2- 172.21.15.115:0/2418348529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c78d00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3834036040 tx=0x7f3834009000 comp rx=0 tx=0).stop 2023-12-01T01:37:30.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.186+0000 7f384c3ae740 1 -- 172.21.15.115:0/2418348529 shutdown_connections 2023-12-01T01:37:30.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.186+0000 7f384c3ae740 1 --2- 172.21.15.115:0/2418348529 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c488a0 0x562f58c75b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.186+0000 7f384c3ae740 1 --2- 172.21.15.115:0/2418348529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 0x562f58c760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.186+0000 7f384c3ae740 1 --2- 172.21.15.115:0/2418348529 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c78d00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.186+0000 7f384c3ae740 1 -- 172.21.15.115:0/2418348529 >> 172.21.15.115:0/2418348529 conn(0x562f58c4abb0 msgr2=0x562f58c93730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:30.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.186+0000 7f384c3ae740 1 -- 172.21.15.115:0/2418348529 shutdown_connections 2023-12-01T01:37:30.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.186+0000 7f384c3ae740 1 -- 172.21.15.115:0/2418348529 wait complete. 2023-12-01T01:37:30.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.186+0000 7f384c3ae740 1 -- start start 2023-12-01T01:37:30.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.187+0000 7f384c3ae740 1 Processor -- start 2023-12-01T01:37:30.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 0x562f58c7c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c78920 0x562f58c6e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58c6ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f384c3ae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f58bb2000 con 0x562f58c5ca20 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f384c3ae740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f58b537d0 con 0x562f58c4c4a0 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f384c3ae740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f58b53470 con 0x562f58c78920 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f384c3ae740 10 client.0 _ll_register_callbacks cb 0x562f58ba21b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f38437fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58c6ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f38437fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58c6ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56708/0 (socket says 172.21.15.115:56708) 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f38437fe640 1 -- 172.21.15.115:0/2508505061 learned_addr learned my addr 172.21.15.115:0/2508505061 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f38427fc640 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c78920 0x562f58c6e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f3842ffd640 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 0x562f58c7c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f38437fe640 1 -- 172.21.15.115:0/2508505061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c78920 msgr2=0x562f58c6e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f38437fe640 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c78920 0x562f58c6e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f38437fe640 1 -- 172.21.15.115:0/2508505061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 msgr2=0x562f58c7c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.188+0000 7f38437fe640 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 0x562f58c7c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f38437fe640 1 -- 172.21.15.115:0/2508505061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38440ce2e0 con 0x562f58c5ca20 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f38427fc640 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c78920 0x562f58c6e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f3842ffd640 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 0x562f58c7c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f38437fe640 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58c6ed60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f383c00f5a0 tx=0x7f383c00fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05edefa878a6c9b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f3821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f3821ffb640 1 -- 172.21.15.115:0/2508505061 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c004020 con 0x562f58c5ca20 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f3821ffb640 1 -- 172.21.15.115:0/2508505061 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f383c00b050 con 0x562f58c5ca20 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f384c3ae740 1 -- 172.21.15.115:0/2508505061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f58bdc550 con 0x562f58c5ca20 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f3821ffb640 1 -- 172.21.15.115:0/2508505061 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f383c004180 con 0x562f58c5ca20 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f384c3ae740 10 client.5268 fetch_fsmap learned FSMap version 16 2023-12-01T01:37:30.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.189+0000 7f384c3ae740 1 -- 172.21.15.115:0/2508505061 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x562f58bb0680 con 0x562f58c5ca20 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.190+0000 7f3821ffb640 1 -- 172.21.15.115:0/2508505061 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f58bdc550 con 0x562f58c5ca20 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.190+0000 7f3821ffb640 1 -- 172.21.15.115:0/2508505061 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f383c011040 con 0x562f58c5ca20 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.190+0000 7f384c3ae740 10 client.5268 fetch_fsmap finished waiting for FSMap version 16 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.190+0000 7f384c3ae740 10 client.5268 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.190+0000 7f384c3ae740 20 client.5268 populate_metadata read hostname 'smithi115' 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.190+0000 7f384c3ae740 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.115:6834/4242588071,v1:172.21.15.115:6835/4242588071] conn(0x562f58c5eab0 0x562f58c5ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.190+0000 7f384c3ae740 4 client.5268 mds_command: new command op to 5091 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.190+0000 7f384c3ae740 1 -- 172.21.15.115:0/2508505061 --> [v2:172.21.15.115:6834/4242588071,v1:172.21.15.115:6835/4242588071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f58ba4b50 con 0x562f58c5eab0 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.190+0000 7f3842ffd640 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.115:6834/4242588071,v1:172.21.15.115:6835/4242588071] conn(0x562f58c5eab0 0x562f58c5ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.191+0000 7f3842ffd640 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.115:6834/4242588071,v1:172.21.15.115:6835/4242588071] conn(0x562f58c5eab0 0x562f58c5ee90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6eeb6ef48a0dcb0 server_cookie=c62f93aa225408eb in_seq=0 out_seq=0 2023-12-01T01:37:30.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.191+0000 7f3821ffb640 10 client.5268 ms_handle_connect on v2:172.21.15.115:6834/4242588071 2023-12-01T01:37:30.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.191+0000 7f3821ffb640 1 -- 172.21.15.115:0/2508505061 <== mds.0 v2:172.21.15.115:6834/4242588071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562f58ba4b50 con 0x562f58c5eab0 2023-12-01T01:37:30.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f3821ffb640 10 client.5268 handle_command_reply: tid=0 2023-12-01T01:37:30.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 client.5268 shutdown 2023-12-01T01:37:30.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 -- 172.21.15.115:0/2508505061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 msgr2=0x562f58c6ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58c6ed60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f383c00f5a0 tx=0x7f383c00fa60 comp rx=0 tx=0).stop 2023-12-01T01:37:30.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 -- 172.21.15.115:0/2508505061 shutdown_connections 2023-12-01T01:37:30.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.115:6834/4242588071,v1:172.21.15.115:6835/4242588071] conn(0x562f58c5eab0 0x562f58c5ee90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c78920 0x562f58c6e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58c6ed60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f3821ffb640 0 client.5268 ms_handle_reset on v2:172.21.15.115:6834/4242588071 2023-12-01T01:37:30.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 --2- 172.21.15.115:0/2508505061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c4c4a0 0x562f58c7c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 -- 172.21.15.115:0/2508505061 >> 172.21.15.115:0/2508505061 conn(0x562f58c488a0 msgr2=0x562f58c4ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:30.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 -- 172.21.15.115:0/2508505061 shutdown_connections 2023-12-01T01:37:30.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 1 -- 172.21.15.115:0/2508505061 wait complete. 2023-12-01T01:37:30.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.192+0000 7f384c3ae740 20 client.5268 trim_cache size 0 max 16384 2023-12-01T01:37:30.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.209+0000 7f384c3ae740 1 Processor -- start 2023-12-01T01:37:30.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.209+0000 7f384c3ae740 1 -- start start 2023-12-01T01:37:30.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.210+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c4c4a0 0x562f58c70f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.210+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c5ca20 0x562f58c91480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.210+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c47920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.210+0000 7f384c3ae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f58919f20 con 0x562f58c78920 2023-12-01T01:37:30.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.210+0000 7f384c3ae740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f58c7cb20 con 0x562f58c5ca20 2023-12-01T01:37:30.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.210+0000 7f384c3ae740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f58b72a10 con 0x562f58c4c4a0 2023-12-01T01:37:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.210+0000 7f38437fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c47920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.210+0000 7f38437fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c47920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56718/0 (socket says 172.21.15.115:56718) 2023-12-01T01:37:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.210+0000 7f38437fe640 1 -- 172.21.15.115:0/3244196350 learned_addr learned my addr 172.21.15.115:0/3244196350 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f3842ffd640 1 --2- 172.21.15.115:0/3244196350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c4c4a0 0x562f58c70f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38427fc640 1 --2- 172.21.15.115:0/3244196350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c5ca20 0x562f58c91480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38437fe640 1 -- 172.21.15.115:0/3244196350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c4c4a0 msgr2=0x562f58c70f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38437fe640 1 --2- 172.21.15.115:0/3244196350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c4c4a0 0x562f58c70f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38437fe640 1 -- 172.21.15.115:0/3244196350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c5ca20 msgr2=0x562f58c91480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38437fe640 1 --2- 172.21.15.115:0/3244196350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c5ca20 0x562f58c91480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38437fe640 1 -- 172.21.15.115:0/3244196350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f58b529e0 con 0x562f58c78920 2023-12-01T01:37:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38427fc640 1 --2- 172.21.15.115:0/3244196350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c5ca20 0x562f58c91480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38437fe640 1 --2- 172.21.15.115:0/3244196350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c47920 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x562f58ca64f0 tx=0x7f383c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a5d811a182b485 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38397fa640 1 -- 172.21.15.115:0/3244196350 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c004420 con 0x562f58c78920 2023-12-01T01:37:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38397fa640 1 -- 172.21.15.115:0/3244196350 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c00c040 con 0x562f58c78920 2023-12-01T01:37:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.211+0000 7f38397fa640 1 -- 172.21.15.115:0/3244196350 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c004220 con 0x562f58c78920 2023-12-01T01:37:30.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.212+0000 7f384c3ae740 1 -- 172.21.15.115:0/3244196350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 msgr2=0x562f58c47920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.212+0000 7f384c3ae740 1 --2- 172.21.15.115:0/3244196350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c47920 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x562f58ca64f0 tx=0x7f383c015000 comp rx=0 tx=0).stop 2023-12-01T01:37:30.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.212+0000 7f384c3ae740 1 -- 172.21.15.115:0/3244196350 shutdown_connections 2023-12-01T01:37:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.212+0000 7f384c3ae740 1 --2- 172.21.15.115:0/3244196350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c78920 0x562f58c47920 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.212+0000 7f384c3ae740 1 --2- 172.21.15.115:0/3244196350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c5ca20 0x562f58c91480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.212+0000 7f384c3ae740 1 --2- 172.21.15.115:0/3244196350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c4c4a0 0x562f58c70f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.212+0000 7f384c3ae740 1 -- 172.21.15.115:0/3244196350 >> 172.21.15.115:0/3244196350 conn(0x562f58c488a0 msgr2=0x562f58c44040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.212+0000 7f384c3ae740 1 -- 172.21.15.115:0/3244196350 shutdown_connections 2023-12-01T01:37:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.212+0000 7f384c3ae740 1 -- 172.21.15.115:0/3244196350 wait complete. 2023-12-01T01:37:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.213+0000 7f384c3ae740 1 -- start start 2023-12-01T01:37:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.213+0000 7f384c3ae740 1 Processor -- start 2023-12-01T01:37:30.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.214+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c4c4a0 0x562f58c48330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.214+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58cef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.214+0000 7f384c3ae740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c78920 0x562f58c5de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.214+0000 7f384c3ae740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f58c4ea30 con 0x562f58c5ca20 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.214+0000 7f384c3ae740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f58cfcbb0 con 0x562f58c78920 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.214+0000 7f384c3ae740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f58c7b520 con 0x562f58c4c4a0 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.214+0000 7f384c3ae740 10 client.0 _ll_register_callbacks cb 0x7f3834009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f38427fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58cef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f38427fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58cef330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56724/0 (socket says 172.21.15.115:56724) 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f38427fc640 1 -- 172.21.15.115:0/1658005241 learned_addr learned my addr 172.21.15.115:0/1658005241 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f38427fc640 1 -- 172.21.15.115:0/1658005241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c4c4a0 msgr2=0x562f58c48330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f38427fc640 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c4c4a0 0x562f58c48330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f38427fc640 1 -- 172.21.15.115:0/1658005241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c78920 msgr2=0x562f58c5de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f38427fc640 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c78920 0x562f58c5de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f38427fc640 1 -- 172.21.15.115:0/1658005241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f58bb0680 con 0x562f58c5ca20 2023-12-01T01:37:30.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f38427fc640 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58cef330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f382c05e7f0 tx=0x7f382c057580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d20dc777c6d137 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f3821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f3821ffb640 1 -- 172.21.15.115:0/1658005241 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c056430 con 0x562f58c5ca20 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f3821ffb640 1 -- 172.21.15.115:0/1658005241 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f382c06be20 con 0x562f58c5ca20 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.215+0000 7f384c3ae740 1 -- 172.21.15.115:0/1658005241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3834046040 con 0x562f58c5ca20 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f3821ffb640 1 -- 172.21.15.115:0/1658005241 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f382c06e1b0 con 0x562f58c5ca20 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f384c3ae740 10 client.5274 fetch_fsmap learned FSMap version 16 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f384c3ae740 1 -- 172.21.15.115:0/1658005241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x562f58b77ef0 con 0x562f58c5ca20 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f3821ffb640 1 -- 172.21.15.115:0/1658005241 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834046040 con 0x562f58c5ca20 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f3821ffb640 1 -- 172.21.15.115:0/1658005241 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f382c05e890 con 0x562f58c5ca20 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f384c3ae740 10 client.5274 fetch_fsmap finished waiting for FSMap version 16 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f384c3ae740 10 client.5274 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-01T01:37:30.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f384c3ae740 20 client.5274 populate_metadata read hostname 'smithi115' 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f384c3ae740 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.115:6834/4242588071,v1:172.21.15.115:6835/4242588071] conn(0x562f58c6d0b0 0x562f58c6d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f384c3ae740 4 client.5274 mds_command: new command op to 5091 tid=0[{"prefix": "status"}] 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f384c3ae740 1 -- 172.21.15.115:0/1658005241 --> [v2:172.21.15.115:6834/4242588071,v1:172.21.15.115:6835/4242588071] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562f58bab080 con 0x562f58c6d0b0 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.216+0000 7f3842ffd640 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.115:6834/4242588071,v1:172.21.15.115:6835/4242588071] conn(0x562f58c6d0b0 0x562f58c6d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f3842ffd640 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.115:6834/4242588071,v1:172.21.15.115:6835/4242588071] conn(0x562f58c6d0b0 0x562f58c6d4b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e744ffe728f0e04 server_cookie=7d66d5bcdf1b4882 in_seq=0 out_seq=0 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f3821ffb640 10 client.5274 ms_handle_connect on v2:172.21.15.115:6834/4242588071 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f3821ffb640 1 -- 172.21.15.115:0/1658005241 <== mds.0 v2:172.21.15.115:6834/4242588071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x562f58bab080 con 0x562f58c6d0b0 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f3821ffb640 10 client.5274 handle_command_reply: tid=0 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f384c3ae740 1 client.5274 shutdown 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f384c3ae740 1 -- 172.21.15.115:0/1658005241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 msgr2=0x562f58cef330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f384c3ae740 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58cef330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f382c05e7f0 tx=0x7f382c057580 comp rx=0 tx=0).stop 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f384c3ae740 1 -- 172.21.15.115:0/1658005241 shutdown_connections 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f384c3ae740 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.115:6834/4242588071,v1:172.21.15.115:6835/4242588071] conn(0x562f58c6d0b0 0x562f58c6d4b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f384c3ae740 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562f58c78920 0x562f58c5de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f384c3ae740 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x562f58c5ca20 0x562f58cef330 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f384c3ae740 1 --2- 172.21.15.115:0/1658005241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562f58c4c4a0 0x562f58c48330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f3821ffb640 0 client.5274 ms_handle_reset on v2:172.21.15.115:6834/4242588071 2023-12-01T01:37:30.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f384c3ae740 1 -- 172.21.15.115:0/1658005241 >> 172.21.15.115:0/1658005241 conn(0x562f58c488a0 msgr2=0x562f58c44040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:30.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.217+0000 7f384c3ae740 1 -- 172.21.15.115:0/1658005241 shutdown_connections 2023-12-01T01:37:30.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384c3ae740 1 -- 172.21.15.115:0/1658005241 wait complete. 2023-12-01T01:37:30.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384c3ae740 20 client.5274 trim_cache size 0 max 16384 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 5091, 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 6.5668868729999996, 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 16, 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 35, 2023-12-01T01:37:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 35, 2023-12-01T01:37:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 14.524695809000001 2023-12-01T01:37:30.224 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:37:30.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384975c640 1 -- 172.21.15.115:0/1308425966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f382406a430 msgr2=0x7f382406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384975c640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f382406a430 0x7f382406c8f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3834035000 tx=0x7f383402e000 comp rx=0 tx=0).stop 2023-12-01T01:37:30.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384975c640 1 -- 172.21.15.115:0/1308425966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38440d1230 msgr2=0x7f384414c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384975c640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38440d1230 0x7f384414c810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c780 tx=0x7f382c00cc40 comp rx=0 tx=0).stop 2023-12-01T01:37:30.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384975c640 1 -- 172.21.15.115:0/1308425966 shutdown_connections 2023-12-01T01:37:30.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384975c640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f382406a430 0x7f382406c8f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384975c640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f384415cd20 0x7f384414cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384975c640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38440d1230 0x7f384414c810 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384975c640 1 --2- 172.21.15.115:0/1308425966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38440b5d80 0x7f38441f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.218+0000 7f384975c640 1 -- 172.21.15.115:0/1308425966 >> 172.21.15.115:0/1308425966 conn(0x7f384406a0e0 msgr2=0x7f38440b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:30.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.219+0000 7f384975c640 1 -- 172.21.15.115:0/1308425966 shutdown_connections 2023-12-01T01:37:30.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.219+0000 7f384975c640 1 -- 172.21.15.115:0/1308425966 wait complete. 2023-12-01T01:37:30.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:37:30.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.447+0000 7fb20f773640 1 Processor -- start 2023-12-01T01:37:30.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.448+0000 7fb20f773640 1 -- start start 2023-12-01T01:37:30.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.449+0000 7fb20f773640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb20815d220 0x7fb2080cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.449+0000 7fb20f773640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb2080ad2b0 0x7fb2080ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.449+0000 7fb20f773640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20815d600 con 0x7fb2080ade80 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.449+0000 7fb20f773640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20806cc10 con 0x7fb2080ad2b0 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.449+0000 7fb20f773640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20806c720 con 0x7fb20815d220 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.449+0000 7fb20cd0e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb2080ad2b0 0x7fb2080ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.449+0000 7fb20dd10640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb2080ade80 0x7fb20806b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51944/0 (socket says 172.21.15.115:51944) 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.450+0000 7fb20dd10640 1 -- 172.21.15.115:0/4021198968 learned_addr learned my addr 172.21.15.115:0/4021198968 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.450+0000 7fb20d50f640 1 --2- 172.21.15.115:0/4021198968 >> v2:172.21.15.142:3300/0 conn(0x7fb20815d220 0x7fb2080cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.450+0000 7fb20cd0e640 1 -- 172.21.15.115:0/4021198968 >> v2:172.21.15.142:3300/0 conn(0x7fb20815d220 msgr2=0x7fb2080cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.450+0000 7fb20cd0e640 1 --2- 172.21.15.115:0/4021198968 >> v2:172.21.15.142:3300/0 conn(0x7fb20815d220 0x7fb2080cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.450+0000 7fb20cd0e640 1 -- 172.21.15.115:0/4021198968 >> v1:172.21.15.115:6789/0 conn(0x7fb2080ade80 legacy=0x7fb20806b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.450+0000 7fb20cd0e640 1 -- 172.21.15.115:0/4021198968 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20806c4b0 con 0x7fb2080ad2b0 2023-12-01T01:37:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.450+0000 7fb1fffff640 1 -- 172.21.15.115:0/4021198968 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479575795 0 0) 0x7fb200001490 con 0x7fb2080ade80 2023-12-01T01:37:30.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.452+0000 7fb20cd0e640 1 --2- 172.21.15.115:0/4021198968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2080ad2b0 0x7fb2080ccab0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb2080724a0 tx=0x7fb1f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea2783cbe81cff6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:30.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.452+0000 7fb1fffff640 1 -- 172.21.15.115:0/4021198968 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f802e070 con 0x7fb2080ad2b0 2023-12-01T01:37:30.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.452+0000 7fb1fffff640 1 -- 172.21.15.115:0/4021198968 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f80315a0 con 0x7fb2080ad2b0 2023-12-01T01:37:30.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.452+0000 7fb1fffff640 1 -- 172.21.15.115:0/4021198968 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8031930 con 0x7fb2080ad2b0 2023-12-01T01:37:30.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.452+0000 7fb20f773640 1 -- 172.21.15.115:0/4021198968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2080ad2b0 msgr2=0x7fb2080ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.453+0000 7fb20f773640 1 --2- 172.21.15.115:0/4021198968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2080ad2b0 0x7fb2080ccab0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb2080724a0 tx=0x7fb1f802b840 comp rx=0 tx=0).stop 2023-12-01T01:37:30.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.453+0000 7fb20f773640 1 -- 172.21.15.115:0/4021198968 shutdown_connections 2023-12-01T01:37:30.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.453+0000 7fb20f773640 1 --2- 172.21.15.115:0/4021198968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2080ad2b0 0x7fb2080ccab0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.453+0000 7fb20f773640 1 --2- 172.21.15.115:0/4021198968 >> v2:172.21.15.142:3300/0 conn(0x7fb20815d220 0x7fb2080cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.453+0000 7fb20f773640 1 -- 172.21.15.115:0/4021198968 >> 172.21.15.115:0/4021198968 conn(0x7fb2080ce880 msgr2=0x7fb2080cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:30.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.453+0000 7fb20f773640 1 -- 172.21.15.115:0/4021198968 shutdown_connections 2023-12-01T01:37:30.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.453+0000 7fb20f773640 1 -- 172.21.15.115:0/4021198968 wait complete. 2023-12-01T01:37:30.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.455+0000 7fb20f773640 1 Processor -- start 2023-12-01T01:37:30.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.455+0000 7fb20f773640 1 -- start start 2023-12-01T01:37:30.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20f773640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2080ad2b0 0x7fb208157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20f773640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2080ade80 0x7fb2080cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20f773640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb20815d220 0x7fb2080ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20f773640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2081f2b80 con 0x7fb20815d220 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20f773640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2081583f0 con 0x7fb2080ade80 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20f773640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2081f65d0 con 0x7fb2080ad2b0 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20dd10640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb20815d220 0x7fb2080ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20dd10640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb20815d220 0x7fb2080ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56734/0 (socket says 172.21.15.115:56734) 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20dd10640 1 -- 172.21.15.115:0/3348268773 learned_addr learned my addr 172.21.15.115:0/3348268773 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20cd0e640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2080ade80 0x7fb2080cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20d50f640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2080ad2b0 0x7fb208157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20dd10640 1 -- 172.21.15.115:0/3348268773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2080ad2b0 msgr2=0x7fb208157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20dd10640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2080ad2b0 0x7fb208157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20dd10640 1 -- 172.21.15.115:0/3348268773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2080ade80 msgr2=0x7fb2080cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.456+0000 7fb20dd10640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2080ade80 0x7fb2080cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.457+0000 7fb20dd10640 1 -- 172.21.15.115:0/3348268773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f8009400 con 0x7fb20815d220 2023-12-01T01:37:30.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.457+0000 7fb20d50f640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2080ad2b0 0x7fb208157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:30.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.457+0000 7fb20dd10640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb20815d220 0x7fb2080ccab0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb20000c700 tx=0x7fb20000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:30.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.457+0000 7fb1fdffb640 1 -- 172.21.15.115:0/3348268773 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000ce10 con 0x7fb20815d220 2023-12-01T01:37:30.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.457+0000 7fb1fdffb640 1 -- 172.21.15.115:0/3348268773 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20000a640 con 0x7fb20815d220 2023-12-01T01:37:30.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.457+0000 7fb1fdffb640 1 -- 172.21.15.115:0/3348268773 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20000aa70 con 0x7fb20815d220 2023-12-01T01:37:30.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.457+0000 7fb20f773640 1 -- 172.21.15.115:0/3348268773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2081f24c0 con 0x7fb20815d220 2023-12-01T01:37:30.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.457+0000 7fb20f773640 1 -- 172.21.15.115:0/3348268773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2081f21b0 con 0x7fb20815d220 2023-12-01T01:37:30.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.458+0000 7fb20f773640 1 -- 172.21.15.115:0/3348268773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2081f2cc0 con 0x7fb20815d220 2023-12-01T01:37:30.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.459+0000 7fb1fdffb640 1 -- 172.21.15.115:0/3348268773 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb20000abd0 con 0x7fb20815d220 2023-12-01T01:37:30.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.460+0000 7fb1fdffb640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb1ec06a430 0x7fb1ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:30.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.460+0000 7fb1fdffb640 1 -- 172.21.15.115:0/3348268773 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb200087cf0 con 0x7fb20815d220 2023-12-01T01:37:30.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.461+0000 7fb20d50f640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb1ec06a430 0x7fb1ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:30.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.461+0000 7fb20d50f640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb1ec06a430 0x7fb1ec06c8f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb204006fd0 tx=0x7fb204008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:30.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.465+0000 7fb1fdffb640 1 -- 172.21.15.115:0/3348268773 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb2081f2cc0 con 0x7fb20815d220 2023-12-01T01:37:30.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.697+0000 7fb20f773640 1 -- 172.21.15.115:0/3348268773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb20815d600 con 0x7fb20815d220 2023-12-01T01:37:30.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.698+0000 7fb1fdffb640 1 -- 172.21.15.115:0/3348268773 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb20815d600 con 0x7fb20815d220 2023-12-01T01:37:30.698 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:37:30.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 -- 172.21.15.115:0/3348268773 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb1ec06a430 msgr2=0x7fb1ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb1ec06a430 0x7fb1ec06c8f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb204006fd0 tx=0x7fb204008040 comp rx=0 tx=0).stop 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 -- 172.21.15.115:0/3348268773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb20815d220 msgr2=0x7fb2080ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb20815d220 0x7fb2080ccab0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb20000c700 tx=0x7fb20000cbc0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 -- 172.21.15.115:0/3348268773 shutdown_connections 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb1ec06a430 0x7fb1ec06c8f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb20815d220 0x7fb2080ccab0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2080ade80 0x7fb2080cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 --2- 172.21.15.115:0/3348268773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2080ad2b0 0x7fb208157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 -- 172.21.15.115:0/3348268773 >> 172.21.15.115:0/3348268773 conn(0x7fb2080ce880 msgr2=0x7fb2080cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 -- 172.21.15.115:0/3348268773 shutdown_connections 2023-12-01T01:37:30.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:30.701+0000 7fb20f773640 1 -- 172.21.15.115:0/3348268773 wait complete. 2023-12-01T01:37:30.719 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T01:37:30.720 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:37:30.720 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T01:37:30.720 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T01:37:30.720 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:37:30.720 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T01:37:30.720 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T01:37:30.720 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:37:30.720 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:37:30.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:30.720 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T01:37:30.773 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:37:30.774 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:37:30.774 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:37:30.774 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:37:30.774 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:37:30.774 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:37:30.774 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:37:30.774 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:37:30.774 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T01:37:30.774 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 42306sec preferred_lft 42306sec 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T01:37:30.775 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:37:30.776 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T01:37:30.776 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:37:30.776 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:37:30.776 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:37:30.776 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T01:37:30.776 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:37:30.776 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:37:30.776 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T01:37:30.776 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T01:37:30.776 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:37:30.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:30.778 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:37:30.865 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T01:37:30.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:30.870 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T01:37:30.924 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T01:37:30.924 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:37:30.924 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:37:30.924 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:37:30.924 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:37:30.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:30.925 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:37:30.925 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:37:30.925 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:37:30.925 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:37:30.925 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:37:30.925 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:37:31.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:31.112 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:37:31.164 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T01:37:31.164 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T01:37:31.165 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:37:31.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:31.166 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T01:37:31.259 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:37:31.259 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T01:37:31.310 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:37:31.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:37:31.311 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:37:31.361 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:37:31.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:31.361 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:37:31.502 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:37:31.502 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:37:31.503 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:37:31.524 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:37:31.524 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:37:31.524 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:37:31.524 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:37:31.524 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:37:31.551 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:37:31.551 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:37:31.552 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:37:31.552 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:37:31.552 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:37:31.552 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:37:31.552 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:37:31.552 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:37:31.553 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T01:37:31.553 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:37:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:37:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:37:31.554 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:37:31.558 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:37:31.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:31.558 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:37:31.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:31.644 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:37:31.709 INFO:teuthology.orchestra.run.smithi115.stdout:client5286 2023-12-01T01:37:31.710 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:37:31.710 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:37:31.759 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T01:37:32.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.018+0000 7fe464cd5640 1 Processor -- start 2023-12-01T01:37:32.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.018+0000 7fe464cd5640 1 -- start start 2023-12-01T01:37:32.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.020+0000 7fe464cd5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe46015e2f0 0x7fe4600ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.020+0000 7fe464cd5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4600adf10 0x7fe4600caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.020+0000 7fe464cd5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4600add10 con 0x7fe46006f2f0 2023-12-01T01:37:32.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.020+0000 7fe464cd5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4600c5760 con 0x7fe46015e2f0 2023-12-01T01:37:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.020+0000 7fe464cd5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46015e6d0 con 0x7fe4600adf10 2023-12-01T01:37:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.020+0000 7fe45e59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe46015e2f0 0x7fe4600ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.020+0000 7fe45e59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe46015e2f0 0x7fe4600ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:41884/0 (socket says 172.21.15.115:41884) 2023-12-01T01:37:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.020+0000 7fe45e59c640 1 -- 172.21.15.115:0/3208205774 learned_addr learned my addr 172.21.15.115:0/3208205774 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:32.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.020+0000 7fe45ed9d640 1 --2- 172.21.15.115:0/3208205774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4600adf10 0x7fe4600caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:32.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.021+0000 7fe45e59c640 1 -- 172.21.15.115:0/3208205774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4600adf10 msgr2=0x7fe4600caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:32.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.021+0000 7fe45e59c640 1 --2- 172.21.15.115:0/3208205774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4600adf10 0x7fe4600caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.021+0000 7fe45e59c640 1 -- 172.21.15.115:0/3208205774 >> v1:172.21.15.142:6789/0 conn(0x7fe46006f2f0 legacy=0x7fe4600c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:32.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.021+0000 7fe45e59c640 1 -- 172.21.15.115:0/3208205774 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4600cf7b0 con 0x7fe46015e2f0 2023-12-01T01:37:32.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.023+0000 7fe45e59c640 1 --2- 172.21.15.115:0/3208205774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46015e2f0 0x7fe4600ca660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe450009870 tx=0x7fe45002b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b47d10132b6324d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:32.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.023+0000 7fe45dd9b640 1 -- 172.21.15.115:0/3208205774 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4500317c0 con 0x7fe46015e2f0 2023-12-01T01:37:32.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.023+0000 7fe45dd9b640 1 -- 172.21.15.115:0/3208205774 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe450031920 con 0x7fe46015e2f0 2023-12-01T01:37:32.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.023+0000 7fe45dd9b640 1 -- 172.21.15.115:0/3208205774 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450031cb0 con 0x7fe46015e2f0 2023-12-01T01:37:32.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.024+0000 7fe464cd5640 1 -- 172.21.15.115:0/3208205774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46015e2f0 msgr2=0x7fe4600ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:32.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.024+0000 7fe464cd5640 1 --2- 172.21.15.115:0/3208205774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46015e2f0 0x7fe4600ca660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe450009870 tx=0x7fe45002b860 comp rx=0 tx=0).stop 2023-12-01T01:37:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.024+0000 7fe464cd5640 1 -- 172.21.15.115:0/3208205774 shutdown_connections 2023-12-01T01:37:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.024+0000 7fe464cd5640 1 --2- 172.21.15.115:0/3208205774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4600adf10 0x7fe4600caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.024+0000 7fe464cd5640 1 --2- 172.21.15.115:0/3208205774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46015e2f0 0x7fe4600ca660 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.024+0000 7fe464cd5640 1 -- 172.21.15.115:0/3208205774 >> 172.21.15.115:0/3208205774 conn(0x7fe4600ce880 msgr2=0x7fe4600cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.024+0000 7fe464cd5640 1 -- 172.21.15.115:0/3208205774 shutdown_connections 2023-12-01T01:37:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.024+0000 7fe464cd5640 1 -- 172.21.15.115:0/3208205774 wait complete. 2023-12-01T01:37:32.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.026+0000 7fe464cd5640 1 Processor -- start 2023-12-01T01:37:32.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.026+0000 7fe464cd5640 1 -- start start 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.027+0000 7fe464cd5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe46006f2f0 0x7fe4600cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.027+0000 7fe464cd5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600adf10 0x7fe4600c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.027+0000 7fe464cd5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46015e2f0 0x7fe4600ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.027+0000 7fe464cd5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600c5c30 con 0x7fe46006f2f0 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.027+0000 7fe464cd5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600c7650 con 0x7fe4600adf10 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.027+0000 7fe464cd5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600c7930 con 0x7fe46015e2f0 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.027+0000 7fe45e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe46006f2f0 0x7fe4600cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe45e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe46006f2f0 0x7fe4600cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56738/0 (socket says 172.21.15.115:56738) 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe45e59c640 1 -- 172.21.15.115:0/2046039621 learned_addr learned my addr 172.21.15.115:0/2046039621 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe457fff640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600adf10 0x7fe4600c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe45ed9d640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46015e2f0 0x7fe4600ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe45e59c640 1 -- 172.21.15.115:0/2046039621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46015e2f0 msgr2=0x7fe4600ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe45e59c640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46015e2f0 0x7fe4600ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe45e59c640 1 -- 172.21.15.115:0/2046039621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600adf10 msgr2=0x7fe4600c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe45e59c640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600adf10 0x7fe4600c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe45e59c640 1 -- 172.21.15.115:0/2046039621 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4500094d0 con 0x7fe46006f2f0 2023-12-01T01:37:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe45ed9d640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46015e2f0 0x7fe4600ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe457fff640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600adf10 0x7fe4600c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.028+0000 7fe45e59c640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe46006f2f0 0x7fe4600cce60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe45002b8a0 tx=0x7fe4500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:32.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.029+0000 7fe456ffd640 1 -- 172.21.15.115:0/2046039621 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450032860 con 0x7fe46006f2f0 2023-12-01T01:37:32.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.029+0000 7fe456ffd640 1 -- 172.21.15.115:0/2046039621 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe450032a00 con 0x7fe46006f2f0 2023-12-01T01:37:32.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.029+0000 7fe464cd5640 1 -- 172.21.15.115:0/2046039621 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4600c7c10 con 0x7fe46006f2f0 2023-12-01T01:37:32.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.029+0000 7fe456ffd640 1 -- 172.21.15.115:0/2046039621 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450032db0 con 0x7fe46006f2f0 2023-12-01T01:37:32.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.029+0000 7fe464cd5640 1 -- 172.21.15.115:0/2046039621 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4600c81d0 con 0x7fe46006f2f0 2023-12-01T01:37:32.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.031+0000 7fe464cd5640 1 -- 172.21.15.115:0/2046039621 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46006fda0 con 0x7fe46006f2f0 2023-12-01T01:37:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.031+0000 7fe456ffd640 1 -- 172.21.15.115:0/2046039621 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe45004e450 con 0x7fe46006f2f0 2023-12-01T01:37:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.033+0000 7fe456ffd640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe44006a480 0x7fe44006c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.033+0000 7fe456ffd640 1 -- 172.21.15.115:0/2046039621 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4500aaab0 con 0x7fe46006f2f0 2023-12-01T01:37:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.033+0000 7fe457fff640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe44006a480 0x7fe44006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.034+0000 7fe457fff640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe44006a480 0x7fe44006c940 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe4600c9df0 tx=0x7fe448009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:32.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.037+0000 7fe456ffd640 1 -- 172.21.15.115:0/2046039621 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe450076230 con 0x7fe46006f2f0 2023-12-01T01:37:32.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.252+0000 7fe464cd5640 1 -- 172.21.15.115:0/2046039621 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4600b43d0 con 0x7fe44006a480 2023-12-01T01:37:32.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.254+0000 7fe456ffd640 1 -- 172.21.15.115:0/2046039621 <== mgr.4098 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe44800d4f0 con 0x7fe44006a480 2023-12-01T01:37:32.257 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:37:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:37:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T01:37:32.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:37:32.257 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 -- 172.21.15.115:0/2046039621 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe44006a480 msgr2=0x7fe44006c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe44006a480 0x7fe44006c940 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe4600c9df0 tx=0x7fe448009040 comp rx=0 tx=0).stop 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 -- 172.21.15.115:0/2046039621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe46006f2f0 msgr2=0x7fe4600cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe46006f2f0 0x7fe4600cce60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe45002b8a0 tx=0x7fe4500046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 -- 172.21.15.115:0/2046039621 shutdown_connections 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe44006a480 0x7fe44006c940 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe46015e2f0 0x7fe4600ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4600adf10 0x7fe4600c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 --2- 172.21.15.115:0/2046039621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe46006f2f0 0x7fe4600cce60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 -- 172.21.15.115:0/2046039621 >> 172.21.15.115:0/2046039621 conn(0x7fe4600ce880 msgr2=0x7fe46015d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 -- 172.21.15.115:0/2046039621 shutdown_connections 2023-12-01T01:37:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.257+0000 7fe464cd5640 1 -- 172.21.15.115:0/2046039621 wait complete. 2023-12-01T01:37:32.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T01:37:32.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.487+0000 7f6afd981640 1 Processor -- start 2023-12-01T01:37:32.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.487+0000 7f6afd981640 1 -- start start 2023-12-01T01:37:32.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.488+0000 7f6afd981640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af815cf70 0x7f6af80b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.488+0000 7f6afd981640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6af80b18f0 0x7f6af80b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.488+0000 7f6afd981640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af80b7760 con 0x7f6af80cfb50 2023-12-01T01:37:32.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.488+0000 7f6afd981640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af80b2170 con 0x7f6af80b18f0 2023-12-01T01:37:32.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.488+0000 7f6afd981640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80b8b70 con 0x7f6af815cf70 2023-12-01T01:37:32.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.489+0000 7f6af67fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6af80b18f0 0x7f6af80b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.489+0000 7f6af77fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6af80cfb50 0x7f6af80d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51946/0 (socket says 172.21.15.115:51946) 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.489+0000 7f6af67fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6af80b18f0 0x7f6af80b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56754/0 (socket says 172.21.15.115:56754) 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.489+0000 7f6af77fe640 1 -- 172.21.15.115:0/3559302280 learned_addr learned my addr 172.21.15.115:0/3559302280 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.489+0000 7f6af6ffd640 1 --2- 172.21.15.115:0/3559302280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af815cf70 0x7f6af80b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.489+0000 7f6af67fc640 1 -- 172.21.15.115:0/3559302280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af815cf70 msgr2=0x7f6af80b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.489+0000 7f6af67fc640 1 --2- 172.21.15.115:0/3559302280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af815cf70 0x7f6af80b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.489+0000 7f6af67fc640 1 -- 172.21.15.115:0/3559302280 >> v1:172.21.15.115:6789/0 conn(0x7f6af80cfb50 legacy=0x7f6af80d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.489+0000 7f6af67fc640 1 -- 172.21.15.115:0/3559302280 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af80b6e30 con 0x7f6af80b18f0 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.490+0000 7f6af67fc640 1 --2- 172.21.15.115:0/3559302280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 0x7f6af80b7bf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0097a0 tx=0x7f6aec02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd58c09fa6501ca server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.490+0000 7f6af5ffb640 1 -- 172.21.15.115:0/3559302280 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec030800 con 0x7f6af80b18f0 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.490+0000 7f6af5ffb640 1 -- 172.21.15.115:0/3559302280 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec030960 con 0x7f6af80b18f0 2023-12-01T01:37:32.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.491+0000 7f6af5ffb640 1 -- 172.21.15.115:0/3559302280 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec030cf0 con 0x7f6af80b18f0 2023-12-01T01:37:32.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.491+0000 7f6afd981640 1 -- 172.21.15.115:0/3559302280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 msgr2=0x7f6af80b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:32.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.491+0000 7f6afd981640 1 --2- 172.21.15.115:0/3559302280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 0x7f6af80b7bf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0097a0 tx=0x7f6aec02b940 comp rx=0 tx=0).stop 2023-12-01T01:37:32.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.491+0000 7f6afd981640 1 -- 172.21.15.115:0/3559302280 shutdown_connections 2023-12-01T01:37:32.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.491+0000 7f6afd981640 1 --2- 172.21.15.115:0/3559302280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 0x7f6af80b7bf0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.491+0000 7f6afd981640 1 --2- 172.21.15.115:0/3559302280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af815cf70 0x7f6af80b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.491+0000 7f6afd981640 1 -- 172.21.15.115:0/3559302280 >> 172.21.15.115:0/3559302280 conn(0x7f6af806a0e0 msgr2=0x7f6af806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:32.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.491+0000 7f6afd981640 1 -- 172.21.15.115:0/3559302280 shutdown_connections 2023-12-01T01:37:32.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.491+0000 7f6afd981640 1 -- 172.21.15.115:0/3559302280 wait complete. 2023-12-01T01:37:32.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.493+0000 7f6afd981640 1 Processor -- start 2023-12-01T01:37:32.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.493+0000 7f6afd981640 1 -- start start 2023-12-01T01:37:32.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.494+0000 7f6afd981640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 0x7f6af81f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.494+0000 7f6afd981640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af80cfb50 0x7f6af8153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.494+0000 7f6afd981640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af815cf70 0x7f6af80c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.494+0000 7f6afd981640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af8153660 con 0x7f6af80b18f0 2023-12-01T01:37:32.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.494+0000 7f6afd981640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af81f2e70 con 0x7f6af815cf70 2023-12-01T01:37:32.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.494+0000 7f6afd981640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af81f3150 con 0x7f6af80cfb50 2023-12-01T01:37:32.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.494+0000 7f6af6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 0x7f6af81f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:32.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6af6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 0x7f6af81f2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56770/0 (socket says 172.21.15.115:56770) 2023-12-01T01:37:32.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6af6ffd640 1 -- 172.21.15.115:0/2102883035 learned_addr learned my addr 172.21.15.115:0/2102883035 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:32.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6af6ffd640 1 -- 172.21.15.115:0/2102883035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af80cfb50 msgr2=0x7f6af8153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:32.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6af6ffd640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af80cfb50 0x7f6af8153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6af6ffd640 1 -- 172.21.15.115:0/2102883035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af815cf70 msgr2=0x7f6af80c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:32.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6af6ffd640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af815cf70 0x7f6af80c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:32.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6af6ffd640 1 -- 172.21.15.115:0/2102883035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec009400 con 0x7f6af80b18f0 2023-12-01T01:37:32.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6af6ffd640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 0x7f6af81f2880 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800d5f0 tx=0x7f6ae800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:32.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6adffff640 1 -- 172.21.15.115:0/2102883035 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80125f0 con 0x7f6af80b18f0 2023-12-01T01:37:32.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6adffff640 1 -- 172.21.15.115:0/2102883035 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae8012750 con 0x7f6af80b18f0 2023-12-01T01:37:32.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.495+0000 7f6adffff640 1 -- 172.21.15.115:0/2102883035 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8012b00 con 0x7f6af80b18f0 2023-12-01T01:37:32.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.496+0000 7f6afd981640 1 -- 172.21.15.115:0/2102883035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af81f3430 con 0x7f6af80b18f0 2023-12-01T01:37:32.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.496+0000 7f6afd981640 1 -- 172.21.15.115:0/2102883035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af80cbca0 con 0x7f6af80b18f0 2023-12-01T01:37:32.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.497+0000 7f6afd981640 1 -- 172.21.15.115:0/2102883035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af80cbe30 con 0x7f6af80b18f0 2023-12-01T01:37:32.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.498+0000 7f6adffff640 1 -- 172.21.15.115:0/2102883035 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6ae8012c60 con 0x7f6af80b18f0 2023-12-01T01:37:32.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.499+0000 7f6adffff640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ae406a430 0x7f6ae406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:32.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.499+0000 7f6adffff640 1 -- 172.21.15.115:0/2102883035 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ae8088de0 con 0x7f6af80b18f0 2023-12-01T01:37:32.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.499+0000 7f6af67fc640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ae406a430 0x7f6ae406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:32.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.500+0000 7f6af67fc640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ae406a430 0x7f6ae406c8f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6aec036000 tx=0x7f6aec03b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:32.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.504+0000 7f6adffff640 1 -- 172.21.15.115:0/2102883035 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6ae80543d0 con 0x7f6af80b18f0 2023-12-01T01:37:32.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:32.739+0000 7f6afd981640 1 -- 172.21.15.115:0/2102883035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f6af80d03c0 con 0x7f6af80b18f0 2023-12-01T01:37:33.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.241+0000 7f6adffff640 1 -- 172.21.15.115:0/2102883035 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 6) v1 ==== 88624+0+0 (secure 0 0 0) 0x7f6ae80547a0 con 0x7f6af80b18f0 2023-12-01T01:37:33.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.242+0000 7f6adffff640 1 -- 172.21.15.115:0/2102883035 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f6ae806ffc0 con 0x7f6af80b18f0 2023-12-01T01:37:33.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 -- 172.21.15.115:0/2102883035 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ae406a430 msgr2=0x7f6ae406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ae406a430 0x7f6ae406c8f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6aec036000 tx=0x7f6aec03b040 comp rx=0 tx=0).stop 2023-12-01T01:37:33.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 -- 172.21.15.115:0/2102883035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 msgr2=0x7f6af81f2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 0x7f6af81f2880 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800d5f0 tx=0x7f6ae800dab0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 -- 172.21.15.115:0/2102883035 shutdown_connections 2023-12-01T01:37:33.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ae406a430 0x7f6ae406c8f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6af80cfb50 0x7f6af8153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6af815cf70 0x7f6af80c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 --2- 172.21.15.115:0/2102883035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6af80b18f0 0x7f6af81f2880 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 -- 172.21.15.115:0/2102883035 >> 172.21.15.115:0/2102883035 conn(0x7f6af806a0e0 msgr2=0x7f6af80cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:33.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 -- 172.21.15.115:0/2102883035 shutdown_connections 2023-12-01T01:37:33.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.245+0000 7f6afd981640 1 -- 172.21.15.115:0/2102883035 wait complete. 2023-12-01T01:37:33.271 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T01:37:33.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.464+0000 7f50c8a15640 1 Processor -- start 2023-12-01T01:37:33.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.464+0000 7f50c8a15640 1 -- start start 2023-12-01T01:37:33.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.465+0000 7f50c8a15640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f50c40b18f0 0x7f50c40b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.465+0000 7f50c8a15640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c40b7760 con 0x7f50c40d0260 2023-12-01T01:37:33.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.465+0000 7f50c8a15640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c40b8b70 con 0x7f50c415cf70 2023-12-01T01:37:33.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.465+0000 7f50c8a15640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c40b2170 con 0x7f50c40b18f0 2023-12-01T01:37:33.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.466+0000 7f50c1d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f50c40d0260 0x7f50c40d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51960/0 (socket says 172.21.15.115:51960) 2023-12-01T01:37:33.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.466+0000 7f50c1d9b640 1 -- 172.21.15.115:0/3208753986 learned_addr learned my addr 172.21.15.115:0/3208753986 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:33.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.466+0000 7f50c2d9d640 1 --2- 172.21.15.115:0/3208753986 >> v2:172.21.15.142:3300/0 conn(0x7f50c40b18f0 0x7f50c40b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:33.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.466+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031628852 0 0) 0x7f50c40b7760 con 0x7f50c40d0260 2023-12-01T01:37:33.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.466+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50ac003520 con 0x7f50c40d0260 2023-12-01T01:37:33.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1786474721 0 0) 0x7f50ac003520 con 0x7f50c40d0260 2023-12-01T01:37:33.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 >> v2:172.21.15.142:3300/0 conn(0x7f50c40b18f0 msgr2=0x7f50c40b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:33.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 --2- 172.21.15.115:0/3208753986 >> v2:172.21.15.142:3300/0 conn(0x7f50c40b18f0 0x7f50c40b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 >> v1:172.21.15.142:6789/0 conn(0x7f50c415cf70 legacy=0x7f50c40cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c40b6e30 con 0x7f50c40d0260 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f50b8001670 con 0x7f50c40d0260 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 >> v1:172.21.15.115:6789/0 conn(0x7f50c40d0260 legacy=0x7f50c40d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac003970 0x7f50ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50ac005180 0x7f50ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40b2170 con 0x7f50ac0047c0 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac00c020 con 0x7f50ac003970 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac00c1e0 con 0x7f50ac005180 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c1d9b640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.467+0000 7f50c259c640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac003970 0x7f50ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.468+0000 7f50c1d9b640 1 -- 172.21.15.115:0/3208753986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50ac005180 msgr2=0x7f50ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.468+0000 7f50c2d9d640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50ac005180 0x7f50ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.468+0000 7f50c1d9b640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50ac005180 0x7f50ac00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.468+0000 7f50c1d9b640 1 -- 172.21.15.115:0/3208753986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac003970 msgr2=0x7f50ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.468+0000 7f50c1d9b640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac003970 0x7f50ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.468+0000 7f50c1d9b640 1 -- 172.21.15.115:0/3208753986 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac003520 con 0x7f50ac0047c0 2023-12-01T01:37:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.468+0000 7f50c259c640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50ac003970 0x7f50ac003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.468+0000 7f50c2d9d640 1 -- 172.21.15.115:0/3208753986 reap_dead start 2023-12-01T01:37:33.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.469+0000 7f50c1d9b640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50ac004c40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c210 tx=0x7f50ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1ea520cd5c522c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:33.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.469+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8005ad0 con 0x7f50ac0047c0 2023-12-01T01:37:33.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.469+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b8005c30 con 0x7f50ac0047c0 2023-12-01T01:37:33.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.469+0000 7f50c159a640 1 -- 172.21.15.115:0/3208753986 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b802c9d0 con 0x7f50ac0047c0 2023-12-01T01:37:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.469+0000 7f50c8a15640 1 -- 172.21.15.115:0/3208753986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 msgr2=0x7f50ac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.469+0000 7f50c8a15640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50ac004c40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c210 tx=0x7f50ac0057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.470+0000 7f50c8a15640 1 -- 172.21.15.115:0/3208753986 shutdown_connections 2023-12-01T01:37:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.470+0000 7f50c8a15640 1 --2- 172.21.15.115:0/3208753986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50ac004c40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.470+0000 7f50c8a15640 1 -- 172.21.15.115:0/3208753986 >> 172.21.15.115:0/3208753986 conn(0x7f50c406a0e0 msgr2=0x7f50c406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.470+0000 7f50c8a15640 1 -- 172.21.15.115:0/3208753986 shutdown_connections 2023-12-01T01:37:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.470+0000 7f50c8a15640 1 -- 172.21.15.115:0/3208753986 wait complete. 2023-12-01T01:37:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.472+0000 7f50c8a15640 1 Processor -- start 2023-12-01T01:37:33.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.472+0000 7f50c8a15640 1 -- start start 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.473+0000 7f50c8a15640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50c4156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.473+0000 7f50c8a15640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50c41f22e0 0x7f50c40cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.473+0000 7f50c8a15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50c415a720 0x7f50c415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.473+0000 7f50c8a15640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c415b0e0 con 0x7f50ac0047c0 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.473+0000 7f50c8a15640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c4156c80 con 0x7f50c415a720 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.473+0000 7f50c8a15640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40ca030 con 0x7f50c41f22e0 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.473+0000 7f50c259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50c4156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.473+0000 7f50c259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50c4156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56806/0 (socket says 172.21.15.115:56806) 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.473+0000 7f50c259c640 1 -- 172.21.15.115:0/3558248849 learned_addr learned my addr 172.21.15.115:0/3558248849 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50c259c640 1 -- 172.21.15.115:0/3558248849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50c41f22e0 msgr2=0x7f50c40cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50c259c640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50c41f22e0 0x7f50c40cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50c259c640 1 -- 172.21.15.115:0/3558248849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50c415a720 msgr2=0x7f50c415aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50c259c640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50c415a720 0x7f50c415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50c259c640 1 -- 172.21.15.115:0/3558248849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c40ca310 con 0x7f50ac0047c0 2023-12-01T01:37:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50c259c640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50c4156740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f50b4000f60 tx=0x7f50b400d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50b2ffd640 1 -- 172.21.15.115:0/3558248849 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4005480 con 0x7f50ac0047c0 2023-12-01T01:37:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50b2ffd640 1 -- 172.21.15.115:0/3558248849 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b4003610 con 0x7f50ac0047c0 2023-12-01T01:37:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50c8a15640 1 -- 172.21.15.115:0/3558248849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c40ca5f0 con 0x7f50ac0047c0 2023-12-01T01:37:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50b2ffd640 1 -- 172.21.15.115:0/3558248849 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b40039c0 con 0x7f50ac0047c0 2023-12-01T01:37:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.474+0000 7f50c8a15640 1 -- 172.21.15.115:0/3558248849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c40cabb0 con 0x7f50ac0047c0 2023-12-01T01:37:33.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.475+0000 7f50c8a15640 1 -- 172.21.15.115:0/3558248849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c404e890 con 0x7f50ac0047c0 2023-12-01T01:37:33.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.477+0000 7f50b2ffd640 1 -- 172.21.15.115:0/3558248849 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 6) v1 ==== 88624+0+0 (secure 0 0 0) 0x7f50b4003b20 con 0x7f50ac0047c0 2023-12-01T01:37:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.478+0000 7f50b2ffd640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f50a406a480 0x7f50a406c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.478+0000 7f50c1d9b640 1 -- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f50a406a480 msgr2=0x7f50a406c940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:37:33.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.478+0000 7f50b2ffd640 1 -- 172.21.15.115:0/3558248849 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50b40888f0 con 0x7f50ac0047c0 2023-12-01T01:37:33.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.478+0000 7f50c1d9b640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f50a406a480 0x7f50a406c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:37:33.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.482+0000 7f50b2ffd640 1 -- 172.21.15.115:0/3558248849 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f50b4053e70 con 0x7f50ac0047c0 2023-12-01T01:37:33.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.669+0000 7f50c8a15640 1 -- 172.21.15.115:0/3558248849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f50c415c1a0 con 0x7f50ac0047c0 2023-12-01T01:37:33.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.678+0000 7f50b2ffd640 1 -- 172.21.15.115:0/3558248849 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f50b4001230 con 0x7f50ac0047c0 2023-12-01T01:37:33.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.679+0000 7f50c1d9b640 1 -- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f50a406a480 msgr2=0x7f50a406c940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:37:33.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.679+0000 7f50c1d9b640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f50a406a480 0x7f50a406c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:37:33.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.680+0000 7f50c8a15640 1 -- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f50a406a480 msgr2=0x7f50a406c940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.680+0000 7f50c8a15640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f50a406a480 0x7f50a406c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.680+0000 7f50c8a15640 1 -- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 msgr2=0x7f50c4156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.680+0000 7f50c8a15640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50c4156740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f50b4000f60 tx=0x7f50b400d9f0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.681+0000 7f50c8a15640 1 -- 172.21.15.115:0/3558248849 shutdown_connections 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.681+0000 7f50c8a15640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f50a406a480 0x7f50a406c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.681+0000 7f50c8a15640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50c41f22e0 0x7f50c40cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.681+0000 7f50c8a15640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50c415a720 0x7f50c415aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.681+0000 7f50c8a15640 1 --2- 172.21.15.115:0/3558248849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50ac0047c0 0x7f50c4156740 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.681+0000 7f50c8a15640 1 -- 172.21.15.115:0/3558248849 >> 172.21.15.115:0/3558248849 conn(0x7f50c406a0e0 msgr2=0x7f50c406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.681+0000 7f50c8a15640 1 -- 172.21.15.115:0/3558248849 shutdown_connections 2023-12-01T01:37:33.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.681+0000 7f50c8a15640 1 -- 172.21.15.115:0/3558248849 wait complete. 2023-12-01T01:37:33.698 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T01:37:33.751 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:33.749+0000 7f046b82b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:37:33.760 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:33.759+0000 7f9d35641040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:37:33.803 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:33.801+0000 7f046b82b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:37:33.817 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:33.815+0000 7f9d35641040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:37:33.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.890+0000 7faa32ec5640 1 Processor -- start 2023-12-01T01:37:33.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.890+0000 7faa32ec5640 1 -- start start 2023-12-01T01:37:33.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.892+0000 7faa32ec5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa2c070b10 0x7faa2c070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.892+0000 7faa32ec5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faa2c065960 0x7faa2c065df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.892+0000 7faa32ec5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c15bf00 con 0x7faa2c144120 2023-12-01T01:37:33.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.892+0000 7faa32ec5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c066330 con 0x7faa2c065960 2023-12-01T01:37:33.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.892+0000 7faa32ec5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c15b2f0 con 0x7faa2c070b10 2023-12-01T01:37:33.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.892+0000 7faa326c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faa2c065960 0x7faa2c065df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:33.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.892+0000 7faa326c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faa2c065960 0x7faa2c065df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56814/0 (socket says 172.21.15.115:56814) 2023-12-01T01:37:33.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.892+0000 7faa326c4640 1 -- 172.21.15.115:0/2339222799 learned_addr learned my addr 172.21.15.115:0/2339222799 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:33.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.893+0000 7faa326c4640 1 -- 172.21.15.115:0/2339222799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa2c070b10 msgr2=0x7faa2c070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:33.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.893+0000 7faa316c2640 1 --2- 172.21.15.115:0/2339222799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa2c070b10 0x7faa2c070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:33.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.893+0000 7faa326c4640 1 --2- 172.21.15.115:0/2339222799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa2c070b10 0x7faa2c070f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.893+0000 7faa326c4640 1 -- 172.21.15.115:0/2339222799 >> v1:172.21.15.142:6789/0 conn(0x7faa2c144120 legacy=0x7faa2c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:33.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.893+0000 7faa326c4640 1 -- 172.21.15.115:0/2339222799 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c15b5d0 con 0x7faa2c065960 2023-12-01T01:37:33.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.894+0000 7faa326c4640 1 --2- 172.21.15.115:0/2339222799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c065960 0x7faa2c065df0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7faa24009e50 tx=0x7faa2402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff69fa800fd602c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:33.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.894+0000 7faa30ec1640 1 -- 172.21.15.115:0/2339222799 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2402ca70 con 0x7faa2c065960 2023-12-01T01:37:33.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.894+0000 7faa30ec1640 1 -- 172.21.15.115:0/2339222799 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2402cbd0 con 0x7faa2c065960 2023-12-01T01:37:33.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.894+0000 7faa30ec1640 1 -- 172.21.15.115:0/2339222799 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa240343f0 con 0x7faa2c065960 2023-12-01T01:37:33.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.895+0000 7faa32ec5640 1 -- 172.21.15.115:0/2339222799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c065960 msgr2=0x7faa2c065df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.895+0000 7faa32ec5640 1 --2- 172.21.15.115:0/2339222799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c065960 0x7faa2c065df0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7faa24009e50 tx=0x7faa2402bed0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.895+0000 7faa32ec5640 1 -- 172.21.15.115:0/2339222799 shutdown_connections 2023-12-01T01:37:33.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.895+0000 7faa32ec5640 1 --2- 172.21.15.115:0/2339222799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c065960 0x7faa2c065df0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.895+0000 7faa32ec5640 1 --2- 172.21.15.115:0/2339222799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa2c070b10 0x7faa2c070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.895+0000 7faa32ec5640 1 -- 172.21.15.115:0/2339222799 >> 172.21.15.115:0/2339222799 conn(0x7faa2c073330 msgr2=0x7faa2c0cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:33.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.895+0000 7faa32ec5640 1 -- 172.21.15.115:0/2339222799 shutdown_connections 2023-12-01T01:37:33.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.895+0000 7faa32ec5640 1 -- 172.21.15.115:0/2339222799 wait complete. 2023-12-01T01:37:33.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.897+0000 7faa32ec5640 1 Processor -- start 2023-12-01T01:37:33.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.897+0000 7faa32ec5640 1 -- start start 2023-12-01T01:37:33.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.898+0000 7faa32ec5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa2c065960 0x7faa2c0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.898+0000 7faa32ec5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c070b10 0x7faa2c0cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.898+0000 7faa32ec5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa2c144120 0x7faa2c13e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.898+0000 7faa32ec5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0cc6b0 con 0x7faa2c070b10 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.898+0000 7faa32ec5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c13e720 con 0x7faa2c065960 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.898+0000 7faa32ec5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0cc930 con 0x7faa2c144120 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.898+0000 7faa316c2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c070b10 0x7faa2c0cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.898+0000 7faa316c2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c070b10 0x7faa2c0cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56822/0 (socket says 172.21.15.115:56822) 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.898+0000 7faa316c2640 1 -- 172.21.15.115:0/2605223666 learned_addr learned my addr 172.21.15.115:0/2605223666 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa31ec3640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa2c065960 0x7faa2c0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa326c4640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa2c144120 0x7faa2c13e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa316c2640 1 -- 172.21.15.115:0/2605223666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa2c144120 msgr2=0x7faa2c13e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa316c2640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa2c144120 0x7faa2c13e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa316c2640 1 -- 172.21.15.115:0/2605223666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa2c065960 msgr2=0x7faa2c0cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa316c2640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa2c065960 0x7faa2c0cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:33.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa316c2640 1 -- 172.21.15.115:0/2605223666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24009b30 con 0x7faa2c070b10 2023-12-01T01:37:33.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa316c2640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c070b10 0x7faa2c0cc170 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c7f0 tx=0x7faa1c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:33.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa227fc640 1 -- 172.21.15.115:0/2605223666 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c004220 con 0x7faa2c070b10 2023-12-01T01:37:33.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa227fc640 1 -- 172.21.15.115:0/2605223666 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c0043a0 con 0x7faa2c070b10 2023-12-01T01:37:33.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa227fc640 1 -- 172.21.15.115:0/2605223666 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c007860 con 0x7faa2c070b10 2023-12-01T01:37:33.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa32ec5640 1 -- 172.21.15.115:0/2605223666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c0ccc10 con 0x7faa2c070b10 2023-12-01T01:37:33.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.899+0000 7faa32ec5640 1 -- 172.21.15.115:0/2605223666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c0ca040 con 0x7faa2c070b10 2023-12-01T01:37:33.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.900+0000 7faa32ec5640 1 -- 172.21.15.115:0/2605223666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c0cd130 con 0x7faa2c070b10 2023-12-01T01:37:33.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.901+0000 7faa227fc640 1 -- 172.21.15.115:0/2605223666 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 6) v1 ==== 88624+0+0 (secure 0 0 0) 0x7faa1c007bd0 con 0x7faa2c070b10 2023-12-01T01:37:33.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.902+0000 7faa227fc640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa1006a480 0x7faa1006c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:33.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.902+0000 7faa31ec3640 1 -- 172.21.15.115:0/2605223666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa1006a480 msgr2=0x7faa1006c940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:37:33.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.902+0000 7faa227fc640 1 -- 172.21.15.115:0/2605223666 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa1c088a50 con 0x7faa2c070b10 2023-12-01T01:37:33.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.902+0000 7faa31ec3640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa1006a480 0x7faa1006c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:37:33.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:33.911+0000 7faa227fc640 1 -- 172.21.15.115:0/2605223666 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faa2c0cd130 con 0x7faa2c070b10 2023-12-01T01:37:34.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.093+0000 7faa32ec5640 1 -- 172.21.15.115:0/2605223666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7faa2c0cd130 con 0x7faa2c070b10 2023-12-01T01:37:34.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.100+0000 7faa227fc640 1 -- 172.21.15.115:0/2605223666 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7faa2c0cd130 con 0x7faa2c070b10 2023-12-01T01:37:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.102+0000 7faa32ec5640 1 -- 172.21.15.115:0/2605223666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa1006a480 msgr2=0x7faa1006c940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:37:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.102+0000 7faa32ec5640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa1006a480 0x7faa1006c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.102+0000 7faa32ec5640 1 -- 172.21.15.115:0/2605223666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c070b10 msgr2=0x7faa2c0cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.102+0000 7faa32ec5640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c070b10 0x7faa2c0cc170 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c7f0 tx=0x7faa1c00ccb0 comp rx=0 tx=0).stop 2023-12-01T01:37:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.102+0000 7faa32ec5640 1 -- 172.21.15.115:0/2605223666 shutdown_connections 2023-12-01T01:37:34.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.102+0000 7faa32ec5640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa1006a480 0x7faa1006c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.102+0000 7faa32ec5640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa2c144120 0x7faa2c13e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.102+0000 7faa32ec5640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa2c070b10 0x7faa2c0cc170 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.102+0000 7faa32ec5640 1 --2- 172.21.15.115:0/2605223666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa2c065960 0x7faa2c0cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.102+0000 7faa32ec5640 1 -- 172.21.15.115:0/2605223666 >> 172.21.15.115:0/2605223666 conn(0x7faa2c073330 msgr2=0x7faa2c142420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.103+0000 7faa32ec5640 1 -- 172.21.15.115:0/2605223666 shutdown_connections 2023-12-01T01:37:34.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.103+0000 7faa32ec5640 1 -- 172.21.15.115:0/2605223666 wait complete. 2023-12-01T01:37:34.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:34.122 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-01T01:37:34.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:34.176 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-12-01T01:37:34.224 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir0 1M 2023-12-01T01:37:34.254 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:34.252+0000 7f046b82b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:37:34.292 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:34.290+0000 7f9d35641040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:37:34.305 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:34.303+0000 7f046b82b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:37:34.346 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:34.344+0000 7f9d35641040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:37:34.355 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:34.353+0000 7f046b82b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:37:34.398 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:34.397+0000 7f9d35641040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:37:34.416 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:34.414+0000 7f046b82b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:37:34.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.423+0000 7fa46a634640 1 Processor -- start 2023-12-01T01:37:34.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.423+0000 7fa46a634640 1 -- start start 2023-12-01T01:37:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.424+0000 7fa46a634640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa4640cfb80 0x7fa4640ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.424+0000 7fa46a634640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4640cdcd0 con 0x7fa464154010 2023-12-01T01:37:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.424+0000 7fa46a634640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa464161830 con 0x7fa464153c30 2023-12-01T01:37:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.424+0000 7fa46a634640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4640cd3a0 con 0x7fa4640cfb80 2023-12-01T01:37:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.425+0000 7fa468e31640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa4640cfb80 0x7fa4640ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.425+0000 7fa468e31640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa4640cfb80 0x7fa4640ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56826/0 (socket says 172.21.15.115:56826) 2023-12-01T01:37:34.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.425+0000 7fa468e31640 1 -- 172.21.15.115:0/346071452 learned_addr learned my addr 172.21.15.115:0/346071452 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.425+0000 7fa468e31640 1 -- 172.21.15.115:0/346071452 >> v1:172.21.15.142:6789/0 conn(0x7fa464153c30 legacy=0x7fa4640cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.425+0000 7fa468e31640 1 -- 172.21.15.115:0/346071452 >> v1:172.21.15.115:6789/0 conn(0x7fa464154010 legacy=0x7fa464160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:34.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.425+0000 7fa468e31640 1 -- 172.21.15.115:0/346071452 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4640cd680 con 0x7fa4640cfb80 2023-12-01T01:37:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.426+0000 7fa468e31640 1 --2- 172.21.15.115:0/346071452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 0x7fa4640ce160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa454009870 tx=0x7fa45402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c004da41fd8d99 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.426+0000 7fa45bfff640 1 -- 172.21.15.115:0/346071452 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454030850 con 0x7fa4640cfb80 2023-12-01T01:37:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.426+0000 7fa45bfff640 1 -- 172.21.15.115:0/346071452 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4540309b0 con 0x7fa4640cfb80 2023-12-01T01:37:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.426+0000 7fa45bfff640 1 -- 172.21.15.115:0/346071452 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454030d40 con 0x7fa4640cfb80 2023-12-01T01:37:34.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.427+0000 7fa46a634640 1 -- 172.21.15.115:0/346071452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 msgr2=0x7fa4640ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.427+0000 7fa46a634640 1 --2- 172.21.15.115:0/346071452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 0x7fa4640ce160 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa454009870 tx=0x7fa45402b990 comp rx=0 tx=0).stop 2023-12-01T01:37:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.427+0000 7fa46a634640 1 -- 172.21.15.115:0/346071452 shutdown_connections 2023-12-01T01:37:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.427+0000 7fa46a634640 1 --2- 172.21.15.115:0/346071452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 0x7fa4640ce160 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.427+0000 7fa46a634640 1 -- 172.21.15.115:0/346071452 >> 172.21.15.115:0/346071452 conn(0x7fa46406a0b0 msgr2=0x7fa46406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.427+0000 7fa46a634640 1 -- 172.21.15.115:0/346071452 shutdown_connections 2023-12-01T01:37:34.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.427+0000 7fa46a634640 1 -- 172.21.15.115:0/346071452 wait complete. 2023-12-01T01:37:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.429+0000 7fa46a634640 1 Processor -- start 2023-12-01T01:37:34.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.429+0000 7fa46a634640 1 -- start start 2023-12-01T01:37:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.430+0000 7fa46a634640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 0x7fa464155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.430+0000 7fa46a634640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa464153c30 0x7fa4641f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.430+0000 7fa46a634640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa464154010 0x7fa464154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.430+0000 7fa46a634640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464156570 con 0x7fa4640cfb80 2023-12-01T01:37:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.430+0000 7fa46a634640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464155780 con 0x7fa464154010 2023-12-01T01:37:34.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.430+0000 7fa46a634640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4641f2ce0 con 0x7fa464153c30 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.430+0000 7fa469632640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 0x7fa464155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.430+0000 7fa469632640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 0x7fa464155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56838/0 (socket says 172.21.15.115:56838) 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.430+0000 7fa469632640 1 -- 172.21.15.115:0/4072683174 learned_addr learned my addr 172.21.15.115:0/4072683174 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.430+0000 7fa468e31640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa464153c30 0x7fa4641f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa469e33640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa464154010 0x7fa464154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa469632640 1 -- 172.21.15.115:0/4072683174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa464153c30 msgr2=0x7fa4641f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa469632640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa464153c30 0x7fa4641f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa469632640 1 -- 172.21.15.115:0/4072683174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa464154010 msgr2=0x7fa464154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa469632640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa464154010 0x7fa464154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa469632640 1 -- 172.21.15.115:0/4072683174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540094d0 con 0x7fa4640cfb80 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa469e33640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa464154010 0x7fa464154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa469632640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 0x7fa464155240 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c410 tx=0x7fa46000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4600035f0 con 0x7fa4640cfb80 2023-12-01T01:37:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa460003750 con 0x7fa4640cfb80 2023-12-01T01:37:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460003b40 con 0x7fa4640cfb80 2023-12-01T01:37:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa46a634640 1 -- 172.21.15.115:0/4072683174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4641f2fc0 con 0x7fa4640cfb80 2023-12-01T01:37:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.431+0000 7fa46a634640 1 -- 172.21.15.115:0/4072683174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4641f32a0 con 0x7fa4640cfb80 2023-12-01T01:37:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.433+0000 7fa46a634640 1 -- 172.21.15.115:0/4072683174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4640618d0 con 0x7fa4640cfb80 2023-12-01T01:37:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.433+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 6) v1 ==== 88624+0+0 (secure 0 0 0) 0x7fa460003ca0 con 0x7fa4640cfb80 2023-12-01T01:37:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.434+0000 7fa459ffb640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.434+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa460087c50 con 0x7fa4640cfb80 2023-12-01T01:37:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.434+0000 7fa468e31640 1 -- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 msgr2=0x7fa44806c940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:37:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.434+0000 7fa468e31640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:37:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.438+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa460053230 con 0x7fa4640cfb80 2023-12-01T01:37:34.466 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:34.465+0000 7f046b82b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:37:34.479 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:34.477+0000 7f9d35641040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:37:34.541 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:34.539+0000 7f9d35641040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:37:34.577 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:34.575+0000 7f046b82b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:37:34.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.635+0000 7fa468e31640 1 -- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 msgr2=0x7fa44806c940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:37:34.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.635+0000 7fa468e31640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:37:34.639 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:34.637+0000 7f046b82b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:37:34.655 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:34.654+0000 7f9d35641040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:37:34.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:34.683+0000 7fa46a634640 1 -- 172.21.15.115:0/4072683174 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4640cd960 con 0x7fa44806a480 2023-12-01T01:37:34.688 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:34.686+0000 7f046b82b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:37:34.719 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:34.718+0000 7f9d35641040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:37:34.745 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:34.743+0000 7f046b82b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:37:34.769 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:34.767+0000 7f9d35641040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:37:34.795 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:34.794+0000 7f046b82b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:37:34.825 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:34.823+0000 7f9d35641040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:37:34.876 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:34.874+0000 7f9d35641040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:37:35.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:35.036+0000 7fa468e31640 1 -- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 msgr2=0x7fa44806c940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:37:35.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:35.036+0000 7fa468e31640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:37:35.179 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:35.178+0000 7f046b82b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:37:35.257 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:35.255+0000 7f9d35641040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:37:35.337 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:35.335+0000 7f046b82b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:37:35.415 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:35.413+0000 7f9d35641040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:37:35.444 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:35.442+0000 7f046b82b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:37:35.494 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:35.492+0000 7f046b82b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:37:35.523 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:35.521+0000 7f9d35641040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:37:35.573 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:35.572+0000 7f9d35641040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:37:35.603 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:35.602+0000 7f046b82b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:37:35.681 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:35.680+0000 7f9d35641040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:37:35.712 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:35.710+0000 7f046b82b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:37:35.789 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:35.788+0000 7f9d35641040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:37:35.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:35.837+0000 7fa468e31640 1 -- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 msgr2=0x7fa44806c940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:37:35.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:35.837+0000 7fa468e31640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:37:35.867 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:35.865+0000 7f046b82b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:37:35.916 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:35.914+0000 7f046b82b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:37:35.944 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:35.942+0000 7f9d35641040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:37:35.993 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:35.992+0000 7f9d35641040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:37:36.044 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:36.042+0000 7f046b82b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:37:36.111 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:36.109+0000 7f9d35641040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:37:36.854 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:36.852+0000 7f046b82b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:37:36.934 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:36.932+0000 7f9d35641040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:37:37.147 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:37.145+0000 7f046b82b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:37:37.204 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:37.202+0000 7f046b82b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:37:37.227 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.225+0000 7f9d35641040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:37:37.283 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.281+0000 7f9d35641040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:37:37.315 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:37:37.313+0000 7f046b82b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:37:37.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:37.384+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 7) v1 ==== 88624+0+0 (secure 0 0 0) 0x7fa460053600 con 0x7fa4640cfb80 2023-12-01T01:37:37.401 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.399+0000 7f9d35641040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:37:37.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:37.424+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 8) v1 ==== 88335+0+0 (secure 0 0 0) 0x7fa46005ea40 con 0x7fa4640cfb80 2023-12-01T01:37:37.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:37.424+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 msgr2=0x7fa44806c940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:37:37.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:37.424+0000 7fa459ffb640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:37.619 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.617+0000 7f9cb491c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:37:37.619 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.617+0000 7f9cb491c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:37:37.619 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.617+0000 7f9cb491c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:37:37.619 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.617+0000 7f9cb491c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:37:37.619 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.617+0000 7f9cb491c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:37:37.620 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.618+0000 7f9cb2117640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:37:37.620 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.618+0000 7f9cb2117640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:37:37.620 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.618+0000 7f9cb2117640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:37:37.620 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.618+0000 7f9cb2117640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:37:37.620 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:37:37.618+0000 7f9cb2117640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:37:38.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.433+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 <== mon.0 v2:172.21.15.115:3300/0 9 ==== mgrmap(e 9) v1 ==== 88462+0+0 (secure 0 0 0) 0x7fa46006e3c0 con 0x7fa4640cfb80 2023-12-01T01:37:38.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.434+0000 7fa459ffb640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:38.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.434+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa44801c450 con 0x7fa44806a480 2023-12-01T01:37:38.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.440+0000 7fa468e31640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:38.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.447+0000 7fa468e31640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c8c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa454036000 tx=0x7fa454008030 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:38.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.632+0000 7fa459ffb640 1 -- 172.21.15.115:0/4072683174 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa44801c450 con 0x7fa44806a480 2023-12-01T01:37:38.633 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/dir0 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 -- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 msgr2=0x7fa44806c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c8c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa454036000 tx=0x7fa454008030 comp rx=0 tx=0).stop 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 -- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 msgr2=0x7fa464155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 0x7fa464155240 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c410 tx=0x7fa46000c8d0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 -- 172.21.15.115:0/4072683174 shutdown_connections 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa44806a480 0x7fa44806c8c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa464153c30 0x7fa4641f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa464154010 0x7fa464154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 --2- 172.21.15.115:0/4072683174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4640cfb80 0x7fa464155240 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 -- 172.21.15.115:0/4072683174 >> 172.21.15.115:0/4072683174 conn(0x7fa46406a0b0 msgr2=0x7fa4640ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 -- 172.21.15.115:0/4072683174 shutdown_connections 2023-12-01T01:37:38.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.635+0000 7fa46a634640 1 -- 172.21.15.115:0/4072683174 wait complete. 2023-12-01T01:37:38.656 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-12-01T01:37:38.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:38.656 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-12-01T01:37:38.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir1 1M 2023-12-01T01:37:38.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.925+0000 7f51c6aee640 1 Processor -- start 2023-12-01T01:37:38.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.925+0000 7f51c6aee640 1 -- start start 2023-12-01T01:37:38.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.926+0000 7f51c6aee640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f51c015ac80 0x7f51c00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:38.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.927+0000 7f51c6aee640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c00cc9a0 con 0x7f51c00ce1b0 2023-12-01T01:37:38.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.927+0000 7f51c6aee640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c00cbd90 con 0x7f51c00b0e60 2023-12-01T01:37:38.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.927+0000 7f51c6aee640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c015b060 con 0x7f51c015ac80 2023-12-01T01:37:38.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.927+0000 7f51c508b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f51c00ce1b0 0x7f51c006b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42670/0 (socket says 172.21.15.115:42670) 2023-12-01T01:37:38.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.927+0000 7f51c508b640 1 -- 172.21.15.115:0/166470212 learned_addr learned my addr 172.21.15.115:0/166470212 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:38.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.927+0000 7f51c488a640 1 --2- 172.21.15.115:0/166470212 >> v2:172.21.15.142:3300/0 conn(0x7f51c015ac80 0x7f51c00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:38.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.927+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988303380 0 0) 0x7f51c00cc9a0 con 0x7f51c00ce1b0 2023-12-01T01:37:38.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.927+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b0003520 con 0x7f51c00ce1b0 2023-12-01T01:37:38.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2540329599 0 0) 0x7f51b0003520 con 0x7f51c00ce1b0 2023-12-01T01:37:38.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 >> v2:172.21.15.142:3300/0 conn(0x7f51c015ac80 msgr2=0x7f51c00cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 --2- 172.21.15.115:0/166470212 >> v2:172.21.15.142:3300/0 conn(0x7f51c015ac80 0x7f51c00cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 >> v1:172.21.15.142:6789/0 conn(0x7f51c00b0e60 legacy=0x7f51c00cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00cc070 con 0x7f51c00ce1b0 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f51b80019f0 con 0x7f51c00ce1b0 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 >> v1:172.21.15.115:6789/0 conn(0x7f51c00ce1b0 legacy=0x7f51c006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0003970 0x7f51b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51b00047c0 0x7f51b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51b0005180 0x7f51b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c015b060 con 0x7f51b0005180 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b000c020 con 0x7f51b0003970 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b000c1e0 con 0x7f51b00047c0 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.928+0000 7f51c508b640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51b0005180 0x7f51b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:38.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.929+0000 7f51c488a640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0003970 0x7f51b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:38.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.929+0000 7f51bffff640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51b00047c0 0x7f51b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:38.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.929+0000 7f51c508b640 1 -- 172.21.15.115:0/166470212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51b00047c0 msgr2=0x7f51b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:38.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.929+0000 7f51c508b640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51b00047c0 0x7f51b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.929+0000 7f51c508b640 1 -- 172.21.15.115:0/166470212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0003970 msgr2=0x7f51b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:38.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.929+0000 7f51c508b640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0003970 0x7f51b0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.929+0000 7f51c508b640 1 -- 172.21.15.115:0/166470212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0003520 con 0x7f51b0005180 2023-12-01T01:37:38.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.930+0000 7f51c508b640 1 -- 172.21.15.115:0/166470212 reap_dead start 2023-12-01T01:37:38.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.930+0000 7f51c508b640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51b0005180 0x7f51b000b980 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f51b800a800 tx=0x7f51b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40202f4b818296b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:38.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.931+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b80072d0 con 0x7f51b0005180 2023-12-01T01:37:38.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.931+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b802f070 con 0x7f51b0005180 2023-12-01T01:37:38.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.931+0000 7f51bf7fe640 1 -- 172.21.15.115:0/166470212 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8007670 con 0x7f51b0005180 2023-12-01T01:37:38.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.932+0000 7f51c6aee640 1 -- 172.21.15.115:0/166470212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51b0005180 msgr2=0x7f51b000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:38.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.932+0000 7f51c6aee640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51b0005180 0x7f51b000b980 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f51b800a800 tx=0x7f51b00057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.932+0000 7f51c6aee640 1 -- 172.21.15.115:0/166470212 shutdown_connections 2023-12-01T01:37:38.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.932+0000 7f51c6aee640 1 --2- 172.21.15.115:0/166470212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51b0005180 0x7f51b000b980 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.932+0000 7f51c6aee640 1 -- 172.21.15.115:0/166470212 >> 172.21.15.115:0/166470212 conn(0x7f51c0065960 msgr2=0x7f51c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:38.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.932+0000 7f51c6aee640 1 -- 172.21.15.115:0/166470212 shutdown_connections 2023-12-01T01:37:38.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.932+0000 7f51c6aee640 1 -- 172.21.15.115:0/166470212 wait complete. 2023-12-01T01:37:38.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.934+0000 7f51c6aee640 1 Processor -- start 2023-12-01T01:37:38.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.934+0000 7f51c6aee640 1 -- start start 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c6aee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0005180 0x7f51c01e62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c6aee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51c01e4cf0 0x7f51c01430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c6aee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c01e30f0 0x7f51c01f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c6aee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c01e5e50 con 0x7f51c01e30f0 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c6aee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c01e5240 con 0x7f51b0005180 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c6aee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c01e5520 con 0x7f51c01e4cf0 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c508b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c01e30f0 0x7f51c01f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c508b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c01e30f0 0x7f51c01f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40874/0 (socket says 172.21.15.115:40874) 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c508b640 1 -- 172.21.15.115:0/850637336 learned_addr learned my addr 172.21.15.115:0/850637336 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51bffff640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51c01e4cf0 0x7f51c01430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c488a640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0005180 0x7f51c01e62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:38.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c508b640 1 -- 172.21.15.115:0/850637336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51c01e4cf0 msgr2=0x7f51c01430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:38.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c508b640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51c01e4cf0 0x7f51c01430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c508b640 1 -- 172.21.15.115:0/850637336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0005180 msgr2=0x7f51c01e62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:38.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.935+0000 7f51c508b640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0005180 0x7f51c01e62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:38.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.936+0000 7f51c508b640 1 -- 172.21.15.115:0/850637336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c01e5800 con 0x7f51c01e30f0 2023-12-01T01:37:38.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.936+0000 7f51c488a640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0005180 0x7f51c01e62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:38.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.936+0000 7f51c508b640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c01e30f0 0x7f51c01f3fc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f51b802cc20 tx=0x7f51b8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:38.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.936+0000 7f51bd7fa640 1 -- 172.21.15.115:0/850637336 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8004d60 con 0x7f51c01e30f0 2023-12-01T01:37:38.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.936+0000 7f51bd7fa640 1 -- 172.21.15.115:0/850637336 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b802ce90 con 0x7f51c01e30f0 2023-12-01T01:37:38.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.936+0000 7f51c6aee640 1 -- 172.21.15.115:0/850637336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c01e5ae0 con 0x7f51c01e30f0 2023-12-01T01:37:38.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.936+0000 7f51bd7fa640 1 -- 172.21.15.115:0/850637336 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8007310 con 0x7f51c01e30f0 2023-12-01T01:37:38.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.936+0000 7f51c6aee640 1 -- 172.21.15.115:0/850637336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c0143620 con 0x7f51c01e30f0 2023-12-01T01:37:38.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.938+0000 7f51c6aee640 1 -- 172.21.15.115:0/850637336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c01e5fc0 con 0x7f51c01e30f0 2023-12-01T01:37:38.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.938+0000 7f51bd7fa640 1 -- 172.21.15.115:0/850637336 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 9) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f51b8031030 con 0x7f51c01e30f0 2023-12-01T01:37:38.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.940+0000 7f51bd7fa640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a406a2d0 0x7f51a406c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:38.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.940+0000 7f51bd7fa640 1 -- 172.21.15.115:0/850637336 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f51b80aa1e0 con 0x7f51c01e30f0 2023-12-01T01:37:38.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.940+0000 7f51c488a640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a406a2d0 0x7f51a406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:38.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.941+0000 7f51c488a640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a406a2d0 0x7f51a406c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f51c01f5010 tx=0x7f51b4008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:38.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:38.944+0000 7f51bd7fa640 1 -- 172.21.15.115:0/850637336 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f51b8075940 con 0x7f51c01e30f0 2023-12-01T01:37:39.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.156+0000 7f51c6aee640 1 -- 172.21.15.115:0/850637336 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f51c00cc630 con 0x7f51a406a2d0 2023-12-01T01:37:39.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.314+0000 7f51bd7fa640 1 -- 172.21.15.115:0/850637336 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f51b4012950 con 0x7f51a406a2d0 2023-12-01T01:37:39.317 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/dir1 2023-12-01T01:37:39.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.317+0000 7f51c6aee640 1 -- 172.21.15.115:0/850637336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a406a2d0 msgr2=0x7f51a406c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.317+0000 7f51c6aee640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a406a2d0 0x7f51a406c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f51c01f5010 tx=0x7f51b4008040 comp rx=0 tx=0).stop 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.317+0000 7f51c6aee640 1 -- 172.21.15.115:0/850637336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c01e30f0 msgr2=0x7f51c01f3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.317+0000 7f51c6aee640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c01e30f0 0x7f51c01f3fc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f51b802cc20 tx=0x7f51b8001d80 comp rx=0 tx=0).stop 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.318+0000 7f51c6aee640 1 -- 172.21.15.115:0/850637336 shutdown_connections 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.318+0000 7f51c6aee640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a406a2d0 0x7f51a406c790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.318+0000 7f51c6aee640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51c01e4cf0 0x7f51c01430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.318+0000 7f51c6aee640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c01e30f0 0x7f51c01f3fc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.318+0000 7f51c6aee640 1 --2- 172.21.15.115:0/850637336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0005180 0x7f51c01e62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.318+0000 7f51c6aee640 1 -- 172.21.15.115:0/850637336 >> 172.21.15.115:0/850637336 conn(0x7f51c0065960 msgr2=0x7f51c006f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.318+0000 7f51c6aee640 1 -- 172.21.15.115:0/850637336 shutdown_connections 2023-12-01T01:37:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.318+0000 7f51c6aee640 1 -- 172.21.15.115:0/850637336 wait complete. 2023-12-01T01:37:39.338 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-12-01T01:37:39.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:39.339 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-12-01T01:37:39.391 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir2 1M 2023-12-01T01:37:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.617+0000 7f1086457640 1 Processor -- start 2023-12-01T01:37:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.617+0000 7f1086457640 1 -- start start 2023-12-01T01:37:39.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.618+0000 7f1086457640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1080157680 0x7f10800caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:39.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1086457640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f108015d510 0x7f1080065960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:39.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1086457640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1080065ea0 0x7f10800cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:39.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1086457640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10800ccd20 con 0x7f108015d510 2023-12-01T01:37:39.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1086457640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10800b29c0 con 0x7f1080157680 2023-12-01T01:37:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1086457640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1080157a60 con 0x7f1080065ea0 2023-12-01T01:37:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1084c54640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f108015d510 0x7f1080065960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1084c54640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f108015d510 0x7f1080065960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40898/0 (socket says 172.21.15.115:40898) 2023-12-01T01:37:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1084c54640 1 -- 172.21.15.115:0/2465512737 learned_addr learned my addr 172.21.15.115:0/2465512737 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1085455640 1 --2- 172.21.15.115:0/2465512737 >> v2:172.21.15.142:3300/0 conn(0x7f1080157680 0x7f10800caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1085c56640 1 --2- 172.21.15.115:0/2465512737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1080065ea0 0x7f10800cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1084c54640 1 -- 172.21.15.115:0/2465512737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1080065ea0 msgr2=0x7f10800cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:39.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1084c54640 1 --2- 172.21.15.115:0/2465512737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1080065ea0 0x7f10800cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1084c54640 1 -- 172.21.15.115:0/2465512737 >> v2:172.21.15.142:3300/0 conn(0x7f1080157680 msgr2=0x7f10800caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:39.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1084c54640 1 --2- 172.21.15.115:0/2465512737 >> v2:172.21.15.142:3300/0 conn(0x7f1080157680 0x7f10800caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1084c54640 1 -- 172.21.15.115:0/2465512737 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10800662f0 con 0x7f108015d510 2023-12-01T01:37:39.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.619+0000 7f1085455640 1 --2- 172.21.15.115:0/2465512737 >> v2:172.21.15.142:3300/0 conn(0x7f1080157680 0x7f10800caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:39.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.621+0000 7f1084c54640 1 --2- 172.21.15.115:0/2465512737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f108015d510 0x7f1080065960 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f108006f510 tx=0x7f107002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9e9cc704169b5d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:39.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.621+0000 7f1077fff640 1 -- 172.21.15.115:0/2465512737 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070030850 con 0x7f108015d510 2023-12-01T01:37:39.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.621+0000 7f1077fff640 1 -- 172.21.15.115:0/2465512737 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10700309b0 con 0x7f108015d510 2023-12-01T01:37:39.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.621+0000 7f1077fff640 1 -- 172.21.15.115:0/2465512737 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070030d40 con 0x7f108015d510 2023-12-01T01:37:39.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.622+0000 7f1086457640 1 -- 172.21.15.115:0/2465512737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f108015d510 msgr2=0x7f1080065960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:39.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.622+0000 7f1086457640 1 --2- 172.21.15.115:0/2465512737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f108015d510 0x7f1080065960 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f108006f510 tx=0x7f107002b990 comp rx=0 tx=0).stop 2023-12-01T01:37:39.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.622+0000 7f1086457640 1 -- 172.21.15.115:0/2465512737 shutdown_connections 2023-12-01T01:37:39.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.622+0000 7f1086457640 1 --2- 172.21.15.115:0/2465512737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1080065ea0 0x7f10800cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.622+0000 7f1086457640 1 --2- 172.21.15.115:0/2465512737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f108015d510 0x7f1080065960 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.622+0000 7f1086457640 1 --2- 172.21.15.115:0/2465512737 >> v2:172.21.15.142:3300/0 conn(0x7f1080157680 0x7f10800caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.622+0000 7f1086457640 1 -- 172.21.15.115:0/2465512737 >> 172.21.15.115:0/2465512737 conn(0x7f108006c510 msgr2=0x7f108006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.622+0000 7f1086457640 1 -- 172.21.15.115:0/2465512737 shutdown_connections 2023-12-01T01:37:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.622+0000 7f1086457640 1 -- 172.21.15.115:0/2465512737 wait complete. 2023-12-01T01:37:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.624+0000 7f1086457640 1 Processor -- start 2023-12-01T01:37:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.624+0000 7f1086457640 1 -- start start 2023-12-01T01:37:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.625+0000 7f1086457640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1080065ea0 0x7f10800c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.625+0000 7f1086457640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1080157680 0x7f10800c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.625+0000 7f1086457640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f108015d510 0x7f1080153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.625+0000 7f1086457640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10800c8cf0 con 0x7f1080065ea0 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.625+0000 7f1086457640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1080154320 con 0x7f108015d510 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.625+0000 7f1086457640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10800c80e0 con 0x7f1080157680 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.625+0000 7f1085455640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1080065ea0 0x7f10800c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.625+0000 7f1085455640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1080065ea0 0x7f10800c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40912/0 (socket says 172.21.15.115:40912) 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.625+0000 7f1085455640 1 -- 172.21.15.115:0/1512897023 learned_addr learned my addr 172.21.15.115:0/1512897023 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.626+0000 7f1085455640 1 -- 172.21.15.115:0/1512897023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1080157680 msgr2=0x7f10800c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.626+0000 7f1085455640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1080157680 0x7f10800c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.626+0000 7f1085455640 1 -- 172.21.15.115:0/1512897023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f108015d510 msgr2=0x7f1080153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.626+0000 7f1085455640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f108015d510 0x7f1080153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.626+0000 7f1085455640 1 -- 172.21.15.115:0/1512897023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700094d0 con 0x7f1080065ea0 2023-12-01T01:37:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.626+0000 7f1085455640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1080065ea0 0x7f10800c79e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c7f0 tx=0x7f107c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:39.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.626+0000 7f1075ffb640 1 -- 172.21.15.115:0/1512897023 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c007a30 con 0x7f1080065ea0 2023-12-01T01:37:39.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.626+0000 7f1075ffb640 1 -- 172.21.15.115:0/1512897023 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c007b90 con 0x7f1080065ea0 2023-12-01T01:37:39.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.626+0000 7f1075ffb640 1 -- 172.21.15.115:0/1512897023 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c00ce40 con 0x7f1080065ea0 2023-12-01T01:37:39.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.626+0000 7f1086457640 1 -- 172.21.15.115:0/1512897023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10800c83c0 con 0x7f1080065ea0 2023-12-01T01:37:39.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.627+0000 7f1086457640 1 -- 172.21.15.115:0/1512897023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10800c8980 con 0x7f1080065ea0 2023-12-01T01:37:39.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.628+0000 7f1086457640 1 -- 172.21.15.115:0/1512897023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10800618d0 con 0x7f1080065ea0 2023-12-01T01:37:39.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.628+0000 7f1075ffb640 1 -- 172.21.15.115:0/1512897023 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 10) v1 ==== 88727+0+0 (secure 0 0 0) 0x7f107c0113f0 con 0x7f1080065ea0 2023-12-01T01:37:39.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.629+0000 7f1075ffb640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f106406a5f0 0x7f106406cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:39.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.629+0000 7f1075ffb640 1 -- 172.21.15.115:0/1512897023 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f107c088f40 con 0x7f1080065ea0 2023-12-01T01:37:39.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.630+0000 7f1084c54640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f106406a5f0 0x7f106406cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:39.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.630+0000 7f1084c54640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f106406a5f0 0x7f106406cab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1070036000 tx=0x7f107003b040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:39.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.634+0000 7f1075ffb640 1 -- 172.21.15.115:0/1512897023 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f107c0541f0 con 0x7f1080065ea0 2023-12-01T01:37:39.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.851+0000 7f1086457640 1 -- 172.21.15.115:0/1512897023 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f108015dd90 con 0x7f106406a5f0 2023-12-01T01:37:40.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:39.998+0000 7f1075ffb640 1 -- 172.21.15.115:0/1512897023 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1070030c20 con 0x7f106406a5f0 2023-12-01T01:37:40.000 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/dir2 2023-12-01T01:37:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.001+0000 7f1086457640 1 -- 172.21.15.115:0/1512897023 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f106406a5f0 msgr2=0x7f106406cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.001+0000 7f1086457640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f106406a5f0 0x7f106406cab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1070036000 tx=0x7f107003b040 comp rx=0 tx=0).stop 2023-12-01T01:37:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.001+0000 7f1086457640 1 -- 172.21.15.115:0/1512897023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1080065ea0 msgr2=0x7f10800c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.001+0000 7f1086457640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1080065ea0 0x7f10800c79e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c7f0 tx=0x7f107c00cbf0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.001+0000 7f1086457640 1 -- 172.21.15.115:0/1512897023 shutdown_connections 2023-12-01T01:37:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.002+0000 7f1086457640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f106406a5f0 0x7f106406cab0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.002+0000 7f1086457640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1080157680 0x7f10800c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.002+0000 7f1086457640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f108015d510 0x7f1080153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.002+0000 7f1086457640 1 --2- 172.21.15.115:0/1512897023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1080065ea0 0x7f10800c79e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.002+0000 7f1086457640 1 -- 172.21.15.115:0/1512897023 >> 172.21.15.115:0/1512897023 conn(0x7f108006c510 msgr2=0x7f10800ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.002+0000 7f1086457640 1 -- 172.21.15.115:0/1512897023 shutdown_connections 2023-12-01T01:37:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.002+0000 7f1086457640 1 -- 172.21.15.115:0/1512897023 wait complete. 2023-12-01T01:37:40.024 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-12-01T01:37:40.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:40.024 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-12-01T01:37:40.081 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir3 1M 2023-12-01T01:37:40.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.294+0000 7f81d6b4e640 1 Processor -- start 2023-12-01T01:37:40.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.295+0000 7f81d6b4e640 1 -- start start 2023-12-01T01:37:40.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.296+0000 7f81d6b4e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81d00ccca0 0x7f81d006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.296+0000 7f81d6b4e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d00cb100 0x7f81d006a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.296+0000 7f81d6b4e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d00d07b0 con 0x7f81d00d0c40 2023-12-01T01:37:40.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.296+0000 7f81d6b4e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d00cb560 con 0x7f81d00ccca0 2023-12-01T01:37:40.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.296+0000 7f81d6b4e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d006af00 con 0x7f81d00cb100 2023-12-01T01:37:40.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.296+0000 7f81d48ea640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81d00ccca0 0x7f81d006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.296+0000 7f81d48ea640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81d00ccca0 0x7f81d006b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40928/0 (socket says 172.21.15.115:40928) 2023-12-01T01:37:40.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.296+0000 7f81d48ea640 1 -- 172.21.15.115:0/905381014 learned_addr learned my addr 172.21.15.115:0/905381014 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:40.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.297+0000 7f81d48ea640 1 -- 172.21.15.115:0/905381014 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d00cb100 msgr2=0x7f81d006a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:40.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.297+0000 7f81d48ea640 1 --2- 172.21.15.115:0/905381014 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d00cb100 0x7f81d006a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.297+0000 7f81d48ea640 1 -- 172.21.15.115:0/905381014 >> v1:172.21.15.142:6789/0 conn(0x7f81d00d0c40 legacy=0x7f81d00d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:40.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.297+0000 7f81d48ea640 1 -- 172.21.15.115:0/905381014 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d00d0160 con 0x7f81d00ccca0 2023-12-01T01:37:40.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.298+0000 7f81d48ea640 1 --2- 172.21.15.115:0/905381014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00ccca0 0x7f81d006b900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009870 tx=0x7f81cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5966f0af862b02 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:40.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.298+0000 7f81c77fe640 1 -- 172.21.15.115:0/905381014 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc031640 con 0x7f81d00ccca0 2023-12-01T01:37:40.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.298+0000 7f81c77fe640 1 -- 172.21.15.115:0/905381014 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc0317a0 con 0x7f81d00ccca0 2023-12-01T01:37:40.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.298+0000 7f81c77fe640 1 -- 172.21.15.115:0/905381014 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc031b30 con 0x7f81d00ccca0 2023-12-01T01:37:40.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.299+0000 7f81d6b4e640 1 -- 172.21.15.115:0/905381014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00ccca0 msgr2=0x7f81d006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:40.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.299+0000 7f81d6b4e640 1 --2- 172.21.15.115:0/905381014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00ccca0 0x7f81d006b900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009870 tx=0x7f81cc02b8f0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.299+0000 7f81d6b4e640 1 -- 172.21.15.115:0/905381014 shutdown_connections 2023-12-01T01:37:40.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.299+0000 7f81d6b4e640 1 --2- 172.21.15.115:0/905381014 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d00cb100 0x7f81d006a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.299+0000 7f81d6b4e640 1 --2- 172.21.15.115:0/905381014 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00ccca0 0x7f81d006b900 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.299+0000 7f81d6b4e640 1 -- 172.21.15.115:0/905381014 >> 172.21.15.115:0/905381014 conn(0x7f81d00713d0 msgr2=0x7f81d00717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:40.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.299+0000 7f81d6b4e640 1 -- 172.21.15.115:0/905381014 shutdown_connections 2023-12-01T01:37:40.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.299+0000 7f81d6b4e640 1 -- 172.21.15.115:0/905381014 wait complete. 2023-12-01T01:37:40.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.301+0000 7f81d6b4e640 1 Processor -- start 2023-12-01T01:37:40.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.301+0000 7f81d6b4e640 1 -- start start 2023-12-01T01:37:40.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.302+0000 7f81d6b4e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d00cb100 0x7f81d015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.302+0000 7f81d6b4e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d00ccca0 0x7f81d01f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.302+0000 7f81d6b4e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00d0c40 0x7f81d01f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.302+0000 7f81d6b4e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d015bdb0 con 0x7f81d00d0c40 2023-12-01T01:37:40.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.302+0000 7f81d6b4e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00c8d00 con 0x7f81d00ccca0 2023-12-01T01:37:40.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.302+0000 7f81d6b4e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d0157780 con 0x7f81d00cb100 2023-12-01T01:37:40.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.302+0000 7f81d50eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00d0c40 0x7f81d01f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.302+0000 7f81d50eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00d0c40 0x7f81d01f2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40940/0 (socket says 172.21.15.115:40940) 2023-12-01T01:37:40.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.302+0000 7f81d50eb640 1 -- 172.21.15.115:0/2703905165 learned_addr learned my addr 172.21.15.115:0/2703905165 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:40.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.302+0000 7f81d48ea640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d00cb100 0x7f81d015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81c7fff640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d00ccca0 0x7f81d01f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81d50eb640 1 -- 172.21.15.115:0/2703905165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d00cb100 msgr2=0x7f81d015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:40.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81d50eb640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d00cb100 0x7f81d015b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81d50eb640 1 -- 172.21.15.115:0/2703905165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d00ccca0 msgr2=0x7f81d01f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:40.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81d50eb640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d00ccca0 0x7f81d01f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81d50eb640 1 -- 172.21.15.115:0/2703905165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0094d0 con 0x7f81d00d0c40 2023-12-01T01:37:40.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81d50eb640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00d0c40 0x7f81d01f2070 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81c800c780 tx=0x7f81c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:40.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81c57fa640 1 -- 172.21.15.115:0/2703905165 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8003d80 con 0x7f81d00d0c40 2023-12-01T01:37:40.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81c57fa640 1 -- 172.21.15.115:0/2703905165 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c800ce90 con 0x7f81d00d0c40 2023-12-01T01:37:40.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81c57fa640 1 -- 172.21.15.115:0/2703905165 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c80052b0 con 0x7f81d00d0c40 2023-12-01T01:37:40.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.303+0000 7f81d6b4e640 1 -- 172.21.15.115:0/2703905165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d0157a60 con 0x7f81d00d0c40 2023-12-01T01:37:40.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.304+0000 7f81d6b4e640 1 -- 172.21.15.115:0/2703905165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d0158020 con 0x7f81d00d0c40 2023-12-01T01:37:40.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.305+0000 7f81d6b4e640 1 -- 172.21.15.115:0/2703905165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d00c8e70 con 0x7f81d00d0c40 2023-12-01T01:37:40.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.306+0000 7f81c57fa640 1 -- 172.21.15.115:0/2703905165 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 10) v1 ==== 88727+0+0 (secure 0 0 0) 0x7f81c8005410 con 0x7f81d00d0c40 2023-12-01T01:37:40.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.307+0000 7f81c57fa640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81bc06a5d0 0x7f81bc06ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.307+0000 7f81c57fa640 1 -- 172.21.15.115:0/2703905165 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f81c8087ff0 con 0x7f81d00d0c40 2023-12-01T01:37:40.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.307+0000 7f81d48ea640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81bc06a5d0 0x7f81bc06ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.308+0000 7f81d48ea640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81bc06a5d0 0x7f81bc06ca90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f81cc036000 tx=0x7f81cc03c040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:40.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.312+0000 7f81c57fa640 1 -- 172.21.15.115:0/2703905165 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f81c80bac80 con 0x7f81d00d0c40 2023-12-01T01:37:40.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.525+0000 7f81d6b4e640 1 -- 172.21.15.115:0/2703905165 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f81d00d0440 con 0x7f81bc06a5d0 2023-12-01T01:37:40.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.680+0000 7f81c57fa640 1 -- 172.21.15.115:0/2703905165 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f81cc031a90 con 0x7f81bc06a5d0 2023-12-01T01:37:40.682 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/dir3 2023-12-01T01:37:40.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.683+0000 7f81d6b4e640 1 -- 172.21.15.115:0/2703905165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81bc06a5d0 msgr2=0x7f81bc06ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:40.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.683+0000 7f81d6b4e640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81bc06a5d0 0x7f81bc06ca90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f81cc036000 tx=0x7f81cc03c040 comp rx=0 tx=0).stop 2023-12-01T01:37:40.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.683+0000 7f81d6b4e640 1 -- 172.21.15.115:0/2703905165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00d0c40 msgr2=0x7f81d01f2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:40.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.683+0000 7f81d6b4e640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00d0c40 0x7f81d01f2070 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81c800c780 tx=0x7f81c800cc40 comp rx=0 tx=0).stop 2023-12-01T01:37:40.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.684+0000 7f81d6b4e640 1 -- 172.21.15.115:0/2703905165 shutdown_connections 2023-12-01T01:37:40.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.684+0000 7f81d6b4e640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81bc06a5d0 0x7f81bc06ca90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.684+0000 7f81d6b4e640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81d00d0c40 0x7f81d01f2070 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.684+0000 7f81d6b4e640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81d00ccca0 0x7f81d01f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.684+0000 7f81d6b4e640 1 --2- 172.21.15.115:0/2703905165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81d00cb100 0x7f81d015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.684+0000 7f81d6b4e640 1 -- 172.21.15.115:0/2703905165 >> 172.21.15.115:0/2703905165 conn(0x7f81d00713d0 msgr2=0x7f81d00d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:40.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.684+0000 7f81d6b4e640 1 -- 172.21.15.115:0/2703905165 shutdown_connections 2023-12-01T01:37:40.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.684+0000 7f81d6b4e640 1 -- 172.21.15.115:0/2703905165 wait complete. 2023-12-01T01:37:40.706 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-12-01T01:37:40.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:40.707 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-12-01T01:37:40.760 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir4 1M 2023-12-01T01:37:40.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.972+0000 7fe13dcee640 1 Processor -- start 2023-12-01T01:37:40.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.972+0000 7fe13dcee640 1 -- start start 2023-12-01T01:37:40.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.974+0000 7fe13dcee640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe138142870 0x7fe1380ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.974+0000 7fe13dcee640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1380b1e20 0x7fe13815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.974+0000 7fe13dcee640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1380b2f30 con 0x7fe1380cd010 2023-12-01T01:37:40.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.974+0000 7fe13dcee640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1380b3ae0 con 0x7fe138142870 2023-12-01T01:37:40.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.974+0000 7fe13dcee640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1380cd4a0 con 0x7fe1380b1e20 2023-12-01T01:37:40.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.974+0000 7fe1377fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe138142870 0x7fe1380ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.974+0000 7fe1377fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe138142870 0x7fe1380ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40954/0 (socket says 172.21.15.115:40954) 2023-12-01T01:37:40.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.974+0000 7fe1377fe640 1 -- 172.21.15.115:0/299899342 learned_addr learned my addr 172.21.15.115:0/299899342 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:40.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.974+0000 7fe137fff640 1 --2- 172.21.15.115:0/299899342 >> v2:172.21.15.142:3300/0 conn(0x7fe1380b1e20 0x7fe13815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.975+0000 7fe1377fe640 1 -- 172.21.15.115:0/299899342 >> v2:172.21.15.142:3300/0 conn(0x7fe1380b1e20 msgr2=0x7fe13815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:40.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.975+0000 7fe1377fe640 1 --2- 172.21.15.115:0/299899342 >> v2:172.21.15.142:3300/0 conn(0x7fe1380b1e20 0x7fe13815a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.975+0000 7fe1377fe640 1 -- 172.21.15.115:0/299899342 >> v1:172.21.15.142:6789/0 conn(0x7fe1380cd010 legacy=0x7fe1380b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:40.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.975+0000 7fe1377fe640 1 -- 172.21.15.115:0/299899342 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1380b2320 con 0x7fe138142870 2023-12-01T01:37:40.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.976+0000 7fe1377fe640 1 --2- 172.21.15.115:0/299899342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe138142870 0x7fe1380ccad0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe13806efe0 tx=0x7fe12802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897a3365c5a22466 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:40.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.976+0000 7fe136ffd640 1 -- 172.21.15.115:0/299899342 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128031870 con 0x7fe138142870 2023-12-01T01:37:40.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.976+0000 7fe136ffd640 1 -- 172.21.15.115:0/299899342 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1280319d0 con 0x7fe138142870 2023-12-01T01:37:40.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.976+0000 7fe136ffd640 1 -- 172.21.15.115:0/299899342 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128031d60 con 0x7fe138142870 2023-12-01T01:37:40.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.977+0000 7fe13dcee640 1 -- 172.21.15.115:0/299899342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe138142870 msgr2=0x7fe1380ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:40.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.977+0000 7fe13dcee640 1 --2- 172.21.15.115:0/299899342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe138142870 0x7fe1380ccad0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe13806efe0 tx=0x7fe12802b910 comp rx=0 tx=0).stop 2023-12-01T01:37:40.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.977+0000 7fe13dcee640 1 -- 172.21.15.115:0/299899342 shutdown_connections 2023-12-01T01:37:40.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.977+0000 7fe13dcee640 1 --2- 172.21.15.115:0/299899342 >> v2:172.21.15.142:3300/0 conn(0x7fe1380b1e20 0x7fe13815a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.977+0000 7fe13dcee640 1 --2- 172.21.15.115:0/299899342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe138142870 0x7fe1380ccad0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.977+0000 7fe13dcee640 1 -- 172.21.15.115:0/299899342 >> 172.21.15.115:0/299899342 conn(0x7fe138065960 msgr2=0x7fe138065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:40.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.977+0000 7fe13dcee640 1 -- 172.21.15.115:0/299899342 shutdown_connections 2023-12-01T01:37:40.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.977+0000 7fe13dcee640 1 -- 172.21.15.115:0/299899342 wait complete. 2023-12-01T01:37:40.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.979+0000 7fe13dcee640 1 Processor -- start 2023-12-01T01:37:40.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.979+0000 7fe13dcee640 1 -- start start 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.980+0000 7fe13dcee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1380b1e20 0x7fe138155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.980+0000 7fe13dcee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1380cd010 0x7fe13813b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.980+0000 7fe13dcee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe138142870 0x7fe138136cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.980+0000 7fe13dcee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13813c0e0 con 0x7fe1380b1e20 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.980+0000 7fe13dcee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13813a930 con 0x7fe1380cd010 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.980+0000 7fe13dcee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13813ac10 con 0x7fe138142870 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.980+0000 7fe1377fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1380b1e20 0x7fe138155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.980+0000 7fe1377fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1380b1e20 0x7fe138155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40966/0 (socket says 172.21.15.115:40966) 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.980+0000 7fe1377fe640 1 -- 172.21.15.115:0/2490999118 learned_addr learned my addr 172.21.15.115:0/2490999118 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe12ffff640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1380cd010 0x7fe13813b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe137fff640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe138142870 0x7fe138136cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe1377fe640 1 -- 172.21.15.115:0/2490999118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe138142870 msgr2=0x7fe138136cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe1377fe640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe138142870 0x7fe138136cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe1377fe640 1 -- 172.21.15.115:0/2490999118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1380cd010 msgr2=0x7fe13813b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:40.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe1377fe640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1380cd010 0x7fe13813b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:40.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe1377fe640 1 -- 172.21.15.115:0/2490999118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280094d0 con 0x7fe1380b1e20 2023-12-01T01:37:40.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe12ffff640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1380cd010 0x7fe13813b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:40.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe1377fe640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1380b1e20 0x7fe138155fe0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe1280090b0 tx=0x7fe1280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:40.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe134ff9640 1 -- 172.21.15.115:0/2490999118 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12803aca0 con 0x7fe1380b1e20 2023-12-01T01:37:40.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe134ff9640 1 -- 172.21.15.115:0/2490999118 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12803ae00 con 0x7fe1380b1e20 2023-12-01T01:37:40.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe134ff9640 1 -- 172.21.15.115:0/2490999118 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128044630 con 0x7fe1380b1e20 2023-12-01T01:37:40.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.981+0000 7fe13dcee640 1 -- 172.21.15.115:0/2490999118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13813aef0 con 0x7fe1380b1e20 2023-12-01T01:37:40.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.982+0000 7fe13dcee640 1 -- 172.21.15.115:0/2490999118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1381678d0 con 0x7fe1380b1e20 2023-12-01T01:37:40.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.983+0000 7fe13dcee640 1 -- 172.21.15.115:0/2490999118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13813b410 con 0x7fe1380b1e20 2023-12-01T01:37:40.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.984+0000 7fe134ff9640 1 -- 172.21.15.115:0/2490999118 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 10) v1 ==== 88727+0+0 (secure 0 0 0) 0x7fe12804d450 con 0x7fe1380b1e20 2023-12-01T01:37:40.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.984+0000 7fe134ff9640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe11806a5a0 0x7fe11806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:40.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.985+0000 7fe134ff9640 1 -- 172.21.15.115:0/2490999118 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe12802d040 con 0x7fe1380b1e20 2023-12-01T01:37:40.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.985+0000 7fe12ffff640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe11806a5a0 0x7fe11806ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:40.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.985+0000 7fe12ffff640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe11806a5a0 0x7fe11806ca60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe1381f6270 tx=0x7fe120009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:40.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:40.989+0000 7fe134ff9640 1 -- 172.21.15.115:0/2490999118 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe13813b410 con 0x7fe1380b1e20 2023-12-01T01:37:41.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.200+0000 7fe13dcee640 1 -- 172.21.15.115:0/2490999118 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe138167ac0 con 0x7fe11806a5a0 2023-12-01T01:37:41.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.352+0000 7fe134ff9640 1 -- 172.21.15.115:0/2490999118 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe120005030 con 0x7fe11806a5a0 2023-12-01T01:37:41.354 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/dir4 2023-12-01T01:37:41.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.355+0000 7fe13dcee640 1 -- 172.21.15.115:0/2490999118 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe11806a5a0 msgr2=0x7fe11806ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:41.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.355+0000 7fe13dcee640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe11806a5a0 0x7fe11806ca60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe1381f6270 tx=0x7fe120009040 comp rx=0 tx=0).stop 2023-12-01T01:37:41.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.355+0000 7fe13dcee640 1 -- 172.21.15.115:0/2490999118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1380b1e20 msgr2=0x7fe138155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:41.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.355+0000 7fe13dcee640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1380b1e20 0x7fe138155fe0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe1280090b0 tx=0x7fe1280046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:41.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.355+0000 7fe13dcee640 1 -- 172.21.15.115:0/2490999118 shutdown_connections 2023-12-01T01:37:41.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.355+0000 7fe13dcee640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe11806a5a0 0x7fe11806ca60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:41.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.355+0000 7fe13dcee640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe138142870 0x7fe138136cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:41.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.355+0000 7fe13dcee640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1380cd010 0x7fe13813b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:41.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.355+0000 7fe13dcee640 1 --2- 172.21.15.115:0/2490999118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1380b1e20 0x7fe138155fe0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:41.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.356+0000 7fe13dcee640 1 -- 172.21.15.115:0/2490999118 >> 172.21.15.115:0/2490999118 conn(0x7fe138065960 msgr2=0x7fe13813fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:41.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.356+0000 7fe13dcee640 1 -- 172.21.15.115:0/2490999118 shutdown_connections 2023-12-01T01:37:41.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.356+0000 7fe13dcee640 1 -- 172.21.15.115:0/2490999118 wait complete. 2023-12-01T01:37:41.377 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-12-01T01:37:41.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:41.377 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-12-01T01:37:41.433 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir5 1M 2023-12-01T01:37:41.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.631+0000 7fc54af8a640 1 Processor -- start 2023-12-01T01:37:41.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.631+0000 7fc54af8a640 1 -- start start 2023-12-01T01:37:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.633+0000 7fc54af8a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc54415cdf0 0x7fc544065960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.633+0000 7fc54af8a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc5440ae6f0 0x7fc544065ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.633+0000 7fc54af8a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5440cdf40 con 0x7fc5440aedf0 2023-12-01T01:37:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.633+0000 7fc54af8a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440cd4f0 con 0x7fc54415cdf0 2023-12-01T01:37:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.633+0000 7fc54af8a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440cdc60 con 0x7fc5440ae6f0 2023-12-01T01:37:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.633+0000 7fc549f88640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc54415cdf0 0x7fc544065960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.633+0000 7fc549f88640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc54415cdf0 0x7fc544065960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40980/0 (socket says 172.21.15.115:40980) 2023-12-01T01:37:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.633+0000 7fc549f88640 1 -- 172.21.15.115:0/3798223839 learned_addr learned my addr 172.21.15.115:0/3798223839 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.634+0000 7fc549787640 1 --2- 172.21.15.115:0/3798223839 >> v2:172.21.15.142:3300/0 conn(0x7fc5440ae6f0 0x7fc544065ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.634+0000 7fc549f88640 1 -- 172.21.15.115:0/3798223839 >> v2:172.21.15.142:3300/0 conn(0x7fc5440ae6f0 msgr2=0x7fc544065ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.634+0000 7fc549f88640 1 --2- 172.21.15.115:0/3798223839 >> v2:172.21.15.142:3300/0 conn(0x7fc5440ae6f0 0x7fc544065ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.634+0000 7fc549f88640 1 -- 172.21.15.115:0/3798223839 >> v1:172.21.15.142:6789/0 conn(0x7fc5440aedf0 legacy=0x7fc544070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.634+0000 7fc549f88640 1 -- 172.21.15.115:0/3798223839 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440c8aa0 con 0x7fc54415cdf0 2023-12-01T01:37:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.634+0000 7fc549787640 1 --2- 172.21.15.115:0/3798223839 >> v2:172.21.15.142:3300/0 conn(0x7fc5440ae6f0 0x7fc544065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:37:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.635+0000 7fc549f88640 1 --2- 172.21.15.115:0/3798223839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 0x7fc544065960 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc54406f660 tx=0x7fc54002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0354c09b5549fc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:41.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.636+0000 7fc548f86640 1 -- 172.21.15.115:0/3798223839 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540031870 con 0x7fc54415cdf0 2023-12-01T01:37:41.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.636+0000 7fc548f86640 1 -- 172.21.15.115:0/3798223839 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5400319d0 con 0x7fc54415cdf0 2023-12-01T01:37:41.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.636+0000 7fc548f86640 1 -- 172.21.15.115:0/3798223839 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540031d60 con 0x7fc54415cdf0 2023-12-01T01:37:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.636+0000 7fc54af8a640 1 -- 172.21.15.115:0/3798223839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 msgr2=0x7fc544065960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.636+0000 7fc54af8a640 1 --2- 172.21.15.115:0/3798223839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 0x7fc544065960 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc54406f660 tx=0x7fc54002b910 comp rx=0 tx=0).stop 2023-12-01T01:37:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.636+0000 7fc54af8a640 1 -- 172.21.15.115:0/3798223839 shutdown_connections 2023-12-01T01:37:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.636+0000 7fc54af8a640 1 --2- 172.21.15.115:0/3798223839 >> v2:172.21.15.142:3300/0 conn(0x7fc5440ae6f0 0x7fc544065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.636+0000 7fc54af8a640 1 --2- 172.21.15.115:0/3798223839 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 0x7fc544065960 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.636+0000 7fc54af8a640 1 -- 172.21.15.115:0/3798223839 >> 172.21.15.115:0/3798223839 conn(0x7fc5440cf6f0 msgr2=0x7fc5440cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.637+0000 7fc54af8a640 1 -- 172.21.15.115:0/3798223839 shutdown_connections 2023-12-01T01:37:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.637+0000 7fc54af8a640 1 -- 172.21.15.115:0/3798223839 wait complete. 2023-12-01T01:37:41.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.638+0000 7fc54af8a640 1 Processor -- start 2023-12-01T01:37:41.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.639+0000 7fc54af8a640 1 -- start start 2023-12-01T01:37:41.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.640+0000 7fc54af8a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5440ae6f0 0x7fc544146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:41.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.640+0000 7fc54af8a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5440aedf0 0x7fc5440b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:41.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.640+0000 7fc54af8a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 0x7fc5440b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:41.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.640+0000 7fc54af8a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5440b52b0 con 0x7fc54415cdf0 2023-12-01T01:37:41.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.640+0000 7fc54af8a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc544153b30 con 0x7fc5440ae6f0 2023-12-01T01:37:41.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.640+0000 7fc54af8a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc544153e10 con 0x7fc5440aedf0 2023-12-01T01:37:41.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.640+0000 7fc54a789640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 0x7fc5440b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:41.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.640+0000 7fc54a789640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 0x7fc5440b4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40988/0 (socket says 172.21.15.115:40988) 2023-12-01T01:37:41.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.640+0000 7fc54a789640 1 -- 172.21.15.115:0/3466102441 learned_addr learned my addr 172.21.15.115:0/3466102441 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:41.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.640+0000 7fc549f88640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5440ae6f0 0x7fc544146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:41.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.641+0000 7fc549787640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5440aedf0 0x7fc5440b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:41.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.641+0000 7fc54a789640 1 -- 172.21.15.115:0/3466102441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5440aedf0 msgr2=0x7fc5440b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.641+0000 7fc54a789640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5440aedf0 0x7fc5440b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.641+0000 7fc54a789640 1 -- 172.21.15.115:0/3466102441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5440ae6f0 msgr2=0x7fc544146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.641+0000 7fc54a789640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5440ae6f0 0x7fc544146030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.641+0000 7fc54a789640 1 -- 172.21.15.115:0/3466102441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5400094d0 con 0x7fc54415cdf0 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.641+0000 7fc54a789640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 0x7fc5440b4d70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00b970 tx=0x7fc53c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.642+0000 7fc53a7fc640 1 -- 172.21.15.115:0/3466102441 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00cb40 con 0x7fc54415cdf0 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.642+0000 7fc54af8a640 1 -- 172.21.15.115:0/3466102441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5441540f0 con 0x7fc54415cdf0 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.642+0000 7fc53a7fc640 1 -- 172.21.15.115:0/3466102441 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c00cca0 con 0x7fc54415cdf0 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.642+0000 7fc53a7fc640 1 -- 172.21.15.115:0/3466102441 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c014680 con 0x7fc54415cdf0 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.642+0000 7fc54af8a640 1 -- 172.21.15.115:0/3466102441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc544146570 con 0x7fc54415cdf0 2023-12-01T01:37:41.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.643+0000 7fc54af8a640 1 -- 172.21.15.115:0/3466102441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5440c9040 con 0x7fc54415cdf0 2023-12-01T01:37:41.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.644+0000 7fc53a7fc640 1 -- 172.21.15.115:0/3466102441 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7fc53c01d3b0 con 0x7fc54415cdf0 2023-12-01T01:37:41.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.645+0000 7fc53a7fc640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc52806a630 0x7fc52806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:41.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.646+0000 7fc549f88640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc52806a630 0x7fc52806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:41.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.646+0000 7fc53a7fc640 1 -- 172.21.15.115:0/3466102441 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc53c088c60 con 0x7fc54415cdf0 2023-12-01T01:37:41.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.646+0000 7fc549f88640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc52806a630 0x7fc52806caf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc540036000 tx=0x7fc54003c040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:41.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.650+0000 7fc53a7fc640 1 -- 172.21.15.115:0/3466102441 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc5440c9040 con 0x7fc54415cdf0 2023-12-01T01:37:41.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:41.859+0000 7fc54af8a640 1 -- 172.21.15.115:0/3466102441 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5440c8d80 con 0x7fc52806a630 2023-12-01T01:37:42.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.008+0000 7fc53a7fc640 1 -- 172.21.15.115:0/3466102441 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc540031bc0 con 0x7fc52806a630 2023-12-01T01:37:42.010 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/dir5 2023-12-01T01:37:42.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.011+0000 7fc54af8a640 1 -- 172.21.15.115:0/3466102441 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc52806a630 msgr2=0x7fc52806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.011+0000 7fc54af8a640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc52806a630 0x7fc52806caf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc540036000 tx=0x7fc54003c040 comp rx=0 tx=0).stop 2023-12-01T01:37:42.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.011+0000 7fc54af8a640 1 -- 172.21.15.115:0/3466102441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 msgr2=0x7fc5440b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.011+0000 7fc54af8a640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 0x7fc5440b4d70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00b970 tx=0x7fc53c00be30 comp rx=0 tx=0).stop 2023-12-01T01:37:42.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.011+0000 7fc54af8a640 1 -- 172.21.15.115:0/3466102441 shutdown_connections 2023-12-01T01:37:42.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.011+0000 7fc54af8a640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc52806a630 0x7fc52806caf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.011+0000 7fc54af8a640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5440aedf0 0x7fc5440b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.011+0000 7fc54af8a640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc54415cdf0 0x7fc5440b4d70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.012+0000 7fc54af8a640 1 --2- 172.21.15.115:0/3466102441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5440ae6f0 0x7fc544146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.012+0000 7fc54af8a640 1 -- 172.21.15.115:0/3466102441 >> 172.21.15.115:0/3466102441 conn(0x7fc5440cf6f0 msgr2=0x7fc5440cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.012+0000 7fc54af8a640 1 -- 172.21.15.115:0/3466102441 shutdown_connections 2023-12-01T01:37:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.012+0000 7fc54af8a640 1 -- 172.21.15.115:0/3466102441 wait complete. 2023-12-01T01:37:42.032 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-12-01T01:37:42.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:42.032 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-12-01T01:37:42.085 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir6 1M 2023-12-01T01:37:42.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.297+0000 7faca759e640 1 Processor -- start 2023-12-01T01:37:42.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.297+0000 7faca759e640 1 -- start start 2023-12-01T01:37:42.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.298+0000 7faca759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faca815a290 0x7faca806fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.298+0000 7faca759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca80701f0 0x7faca8156360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.299+0000 7faca759e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca8070640 con 0x7faca80ade60 2023-12-01T01:37:42.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.299+0000 7faca759e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca815a6e0 con 0x7faca815a290 2023-12-01T01:37:42.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.299+0000 7faca759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca806be70 con 0x7faca80701f0 2023-12-01T01:37:42.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.299+0000 7faca5d9b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faca815a290 0x7faca806fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.299+0000 7faca5d9b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faca815a290 0x7faca806fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:46310/0 (socket says 172.21.15.115:46310) 2023-12-01T01:37:42.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.299+0000 7faca5d9b640 1 -- 172.21.15.115:0/2913547991 learned_addr learned my addr 172.21.15.115:0/2913547991 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:42.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.300+0000 7faca6d9d640 1 --2- 172.21.15.115:0/2913547991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca80701f0 0x7faca8156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.300+0000 7faca5d9b640 1 -- 172.21.15.115:0/2913547991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca80701f0 msgr2=0x7faca8156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:42.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.300+0000 7faca5d9b640 1 --2- 172.21.15.115:0/2913547991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca80701f0 0x7faca8156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.300+0000 7faca5d9b640 1 -- 172.21.15.115:0/2913547991 >> v1:172.21.15.142:6789/0 conn(0x7faca80ade60 legacy=0x7faca806b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:42.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.300+0000 7faca5d9b640 1 -- 172.21.15.115:0/2913547991 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca80c9290 con 0x7faca815a290 2023-12-01T01:37:42.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.302+0000 7faca5d9b640 1 --2- 172.21.15.115:0/2913547991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca815a290 0x7faca806fcb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faca806f5c0 tx=0x7fac9c02cfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de27ee4fa681477e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:42.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.302+0000 7faca559a640 1 -- 172.21.15.115:0/2913547991 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c02d950 con 0x7faca815a290 2023-12-01T01:37:42.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.302+0000 7faca559a640 1 -- 172.21.15.115:0/2913547991 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c02dab0 con 0x7faca815a290 2023-12-01T01:37:42.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.302+0000 7faca559a640 1 -- 172.21.15.115:0/2913547991 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c02de40 con 0x7faca815a290 2023-12-01T01:37:42.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.303+0000 7faca759e640 1 -- 172.21.15.115:0/2913547991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca815a290 msgr2=0x7faca806fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.303+0000 7faca759e640 1 --2- 172.21.15.115:0/2913547991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca815a290 0x7faca806fcb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faca806f5c0 tx=0x7fac9c02cfb0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.303+0000 7faca759e640 1 -- 172.21.15.115:0/2913547991 shutdown_connections 2023-12-01T01:37:42.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.303+0000 7faca759e640 1 --2- 172.21.15.115:0/2913547991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca80701f0 0x7faca8156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.303+0000 7faca759e640 1 --2- 172.21.15.115:0/2913547991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca815a290 0x7faca806fcb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.303+0000 7faca759e640 1 -- 172.21.15.115:0/2913547991 >> 172.21.15.115:0/2913547991 conn(0x7faca8065960 msgr2=0x7faca8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:42.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.303+0000 7faca759e640 1 -- 172.21.15.115:0/2913547991 shutdown_connections 2023-12-01T01:37:42.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.303+0000 7faca759e640 1 -- 172.21.15.115:0/2913547991 wait complete. 2023-12-01T01:37:42.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.306+0000 7faca759e640 1 Processor -- start 2023-12-01T01:37:42.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.306+0000 7faca759e640 1 -- start start 2023-12-01T01:37:42.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.307+0000 7faca759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca80701f0 0x7faca8157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.307+0000 7faca759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faca80ade60 0x7faca80c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.307+0000 7faca759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca815a290 0x7faca80c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.307+0000 7faca759e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca81f0540 con 0x7faca80ade60 2023-12-01T01:37:42.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.307+0000 7faca759e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca81f0820 con 0x7faca815a290 2023-12-01T01:37:42.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.307+0000 7faca759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca81f0b00 con 0x7faca80701f0 2023-12-01T01:37:42.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.307+0000 7faca5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faca80ade60 0x7faca80c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.307+0000 7faca5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faca80ade60 0x7faca80c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40996/0 (socket says 172.21.15.115:40996) 2023-12-01T01:37:42.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.307+0000 7faca5d9b640 1 -- 172.21.15.115:0/2488049277 learned_addr learned my addr 172.21.15.115:0/2488049277 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:42.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.307+0000 7faca659c640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca80701f0 0x7faca8157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7faca6d9d640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca815a290 0x7faca80c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7faca5d9b640 1 -- 172.21.15.115:0/2488049277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca80701f0 msgr2=0x7faca8157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7faca5d9b640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca80701f0 0x7faca8157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7faca5d9b640 1 -- 172.21.15.115:0/2488049277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca815a290 msgr2=0x7faca80c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7faca5d9b640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca815a290 0x7faca80c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7faca5d9b640 1 -- 172.21.15.115:0/2488049277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c00ac40 con 0x7faca80ade60 2023-12-01T01:37:42.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7faca5d9b640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faca80ade60 0x7faca80c7a10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fac9c000f80 tx=0x7fac9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:42.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7fac96ffd640 1 -- 172.21.15.115:0/2488049277 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c00a5e0 con 0x7faca80ade60 2023-12-01T01:37:42.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7fac96ffd640 1 -- 172.21.15.115:0/2488049277 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c03f450 con 0x7faca80ade60 2023-12-01T01:37:42.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7fac96ffd640 1 -- 172.21.15.115:0/2488049277 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c03f800 con 0x7faca80ade60 2023-12-01T01:37:42.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.308+0000 7faca759e640 1 -- 172.21.15.115:0/2488049277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca81f0de0 con 0x7faca80ade60 2023-12-01T01:37:42.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.309+0000 7faca759e640 1 -- 172.21.15.115:0/2488049277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca81f00e0 con 0x7faca80ade60 2023-12-01T01:37:42.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.310+0000 7faca759e640 1 -- 172.21.15.115:0/2488049277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca8157ab0 con 0x7faca80ade60 2023-12-01T01:37:42.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.311+0000 7fac96ffd640 1 -- 172.21.15.115:0/2488049277 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7fac9c03f960 con 0x7faca80ade60 2023-12-01T01:37:42.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.311+0000 7fac96ffd640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac8806a630 0x7fac8806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.312+0000 7fac96ffd640 1 -- 172.21.15.115:0/2488049277 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fac9c0abdc0 con 0x7faca80ade60 2023-12-01T01:37:42.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.312+0000 7faca659c640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac8806a630 0x7fac8806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.312+0000 7faca659c640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac8806a630 0x7fac8806caf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fac98007fd0 tx=0x7fac98009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:42.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.316+0000 7fac96ffd640 1 -- 172.21.15.115:0/2488049277 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fac9c0771e0 con 0x7faca80ade60 2023-12-01T01:37:42.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.528+0000 7faca759e640 1 -- 172.21.15.115:0/2488049277 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7faca81f0990 con 0x7fac8806a630 2023-12-01T01:37:42.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.682+0000 7fac96ffd640 1 -- 172.21.15.115:0/2488049277 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7faca81f0990 con 0x7fac8806a630 2023-12-01T01:37:42.684 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/dir6 2023-12-01T01:37:42.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.686+0000 7faca759e640 1 -- 172.21.15.115:0/2488049277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac8806a630 msgr2=0x7fac8806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.686+0000 7faca759e640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac8806a630 0x7fac8806caf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fac98007fd0 tx=0x7fac98009040 comp rx=0 tx=0).stop 2023-12-01T01:37:42.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.686+0000 7faca759e640 1 -- 172.21.15.115:0/2488049277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faca80ade60 msgr2=0x7faca80c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.686+0000 7faca759e640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faca80ade60 0x7faca80c7a10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fac9c000f80 tx=0x7fac9c004a40 comp rx=0 tx=0).stop 2023-12-01T01:37:42.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.687+0000 7faca759e640 1 -- 172.21.15.115:0/2488049277 shutdown_connections 2023-12-01T01:37:42.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.687+0000 7faca759e640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fac8806a630 0x7fac8806caf0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.687+0000 7faca759e640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca815a290 0x7faca80c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.687+0000 7faca759e640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faca80ade60 0x7faca80c7a10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.687+0000 7faca759e640 1 --2- 172.21.15.115:0/2488049277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca80701f0 0x7faca8157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.687+0000 7faca759e640 1 -- 172.21.15.115:0/2488049277 >> 172.21.15.115:0/2488049277 conn(0x7faca8065960 msgr2=0x7faca8152c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:42.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.687+0000 7faca759e640 1 -- 172.21.15.115:0/2488049277 shutdown_connections 2023-12-01T01:37:42.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.687+0000 7faca759e640 1 -- 172.21.15.115:0/2488049277 wait complete. 2023-12-01T01:37:42.710 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-12-01T01:37:42.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:42.711 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-12-01T01:37:42.761 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir7 1M 2023-12-01T01:37:42.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.958+0000 7f4f6760b640 1 Processor -- start 2023-12-01T01:37:42.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.958+0000 7f4f6760b640 1 -- start start 2023-12-01T01:37:42.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.959+0000 7f4f6760b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4f600ce1b0 0x7f4f600cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.959+0000 7f4f6760b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f600cc9a0 con 0x7f4f6015ac80 2023-12-01T01:37:42.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.959+0000 7f4f6760b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f600cbd90 con 0x7f4f600b0e60 2023-12-01T01:37:42.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.959+0000 7f4f6760b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6015b060 con 0x7f4f600ce1b0 2023-12-01T01:37:42.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.960+0000 7f4f653a7640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4f6015ac80 0x7f4f600cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42682/0 (socket says 172.21.15.115:42682) 2023-12-01T01:37:42.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.960+0000 7f4f653a7640 1 -- 172.21.15.115:0/876179817 learned_addr learned my addr 172.21.15.115:0/876179817 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:42.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.960+0000 7f4f64ba6640 1 --2- 172.21.15.115:0/876179817 >> v2:172.21.15.142:3300/0 conn(0x7f4f600ce1b0 0x7f4f600cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.960+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462202295 0 0) 0x7f4f600cc9a0 con 0x7f4f6015ac80 2023-12-01T01:37:42.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.960+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f4c003520 con 0x7f4f6015ac80 2023-12-01T01:37:42.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3270423938 0 0) 0x7f4f4c003520 con 0x7f4f6015ac80 2023-12-01T01:37:42.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 >> v2:172.21.15.142:3300/0 conn(0x7f4f600ce1b0 msgr2=0x7f4f600cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 --2- 172.21.15.115:0/876179817 >> v2:172.21.15.142:3300/0 conn(0x7f4f600ce1b0 0x7f4f600cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 >> v1:172.21.15.142:6789/0 conn(0x7f4f600b0e60 legacy=0x7f4f6006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f600cc070 con 0x7f4f6015ac80 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4f5c001670 con 0x7f4f6015ac80 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 >> v1:172.21.15.115:6789/0 conn(0x7f4f6015ac80 legacy=0x7f4f600cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 --2- 172.21.15.115:0/876179817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f4c0039c0 0x7f4f4c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 --2- 172.21.15.115:0/876179817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f4c0047c0 0x7f4f4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 --2- 172.21.15.115:0/876179817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f4c005180 0x7f4f4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015b060 con 0x7f4f4c005180 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c00c080 con 0x7f4f4c0039c0 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c00c240 con 0x7f4f4c0047c0 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f65ba8640 1 --2- 172.21.15.115:0/876179817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f4c005180 0x7f4f4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.961+0000 7f4f653a7640 1 --2- 172.21.15.115:0/876179817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f4c0039c0 0x7f4f4c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.962+0000 7f4f65ba8640 1 -- 172.21.15.115:0/876179817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f4c0047c0 msgr2=0x7f4f4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:42.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.962+0000 7f4f65ba8640 1 --2- 172.21.15.115:0/876179817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f4c0047c0 0x7f4f4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.962+0000 7f4f65ba8640 1 -- 172.21.15.115:0/876179817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f4c0039c0 msgr2=0x7f4f4c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:42.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.962+0000 7f4f65ba8640 1 --2- 172.21.15.115:0/876179817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f4c0039c0 0x7f4f4c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.962+0000 7f4f65ba8640 1 -- 172.21.15.115:0/876179817 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c003520 con 0x7f4f4c005180 2023-12-01T01:37:42.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.963+0000 7f4f65ba8640 1 -- 172.21.15.115:0/876179817 reap_dead start 2023-12-01T01:37:42.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.963+0000 7f4f65ba8640 1 --2- 172.21.15.115:0/876179817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f4c005180 0x7f4f4c00b980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4f58001a30 tx=0x7f4f4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf5b87e1270d0df server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:42.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.963+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58003b70 con 0x7f4f4c005180 2023-12-01T01:37:42.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.963+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f58003d10 con 0x7f4f4c005180 2023-12-01T01:37:42.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.963+0000 7f4f53fff640 1 -- 172.21.15.115:0/876179817 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58030a40 con 0x7f4f4c005180 2023-12-01T01:37:42.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.963+0000 7f4f6760b640 1 -- 172.21.15.115:0/876179817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f4c005180 msgr2=0x7f4f4c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.963+0000 7f4f6760b640 1 --2- 172.21.15.115:0/876179817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f4c005180 0x7f4f4c00b980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4f58001a30 tx=0x7f4f4c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.964+0000 7f4f6760b640 1 -- 172.21.15.115:0/876179817 shutdown_connections 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.964+0000 7f4f6760b640 1 --2- 172.21.15.115:0/876179817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f4c005180 0x7f4f4c00b980 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.964+0000 7f4f6760b640 1 -- 172.21.15.115:0/876179817 >> 172.21.15.115:0/876179817 conn(0x7f4f60065960 msgr2=0x7f4f60065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.964+0000 7f4f6760b640 1 -- 172.21.15.115:0/876179817 shutdown_connections 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.964+0000 7f4f6760b640 1 -- 172.21.15.115:0/876179817 wait complete. 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.965+0000 7f4f6760b640 1 Processor -- start 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.966+0000 7f4f6760b640 1 -- start start 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.966+0000 7f4f6760b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f4c005180 0x7f4f601567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.966+0000 7f4f6760b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f60157980 0x7f4f6006b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.966+0000 7f4f6760b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f60157e00 0x7f4f6006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.966+0000 7f4f6760b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015bf50 con 0x7f4f60157980 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.966+0000 7f4f6760b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015b340 con 0x7f4f60157e00 2023-12-01T01:37:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.966+0000 7f4f6760b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015b620 con 0x7f4f4c005180 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.967+0000 7f4f64ba6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f60157980 0x7f4f6006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.967+0000 7f4f64ba6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f60157980 0x7f4f6006b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41022/0 (socket says 172.21.15.115:41022) 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.967+0000 7f4f64ba6640 1 -- 172.21.15.115:0/315773815 learned_addr learned my addr 172.21.15.115:0/315773815 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.967+0000 7f4f64ba6640 1 -- 172.21.15.115:0/315773815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f4c005180 msgr2=0x7f4f601567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.967+0000 7f4f64ba6640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f4c005180 0x7f4f601567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.967+0000 7f4f64ba6640 1 -- 172.21.15.115:0/315773815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f60157e00 msgr2=0x7f4f6006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.967+0000 7f4f64ba6640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f60157e00 0x7f4f6006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.967+0000 7f4f64ba6640 1 -- 172.21.15.115:0/315773815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6015b900 con 0x7f4f60157980 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.967+0000 7f4f64ba6640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f60157980 0x7f4f6006b500 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400e7a0 tx=0x7f4f5400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.968+0000 7f4f51ffb640 1 -- 172.21.15.115:0/315773815 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400eeb0 con 0x7f4f60157980 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.968+0000 7f4f51ffb640 1 -- 172.21.15.115:0/315773815 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5400c940 con 0x7f4f60157980 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.968+0000 7f4f51ffb640 1 -- 172.21.15.115:0/315773815 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400cd10 con 0x7f4f60157980 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.968+0000 7f4f6760b640 1 -- 172.21.15.115:0/315773815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6015bbe0 con 0x7f4f60157980 2023-12-01T01:37:42.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.968+0000 7f4f6760b640 1 -- 172.21.15.115:0/315773815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f60145720 con 0x7f4f60157980 2023-12-01T01:37:42.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.969+0000 7f4f6760b640 1 -- 172.21.15.115:0/315773815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6004e890 con 0x7f4f60157980 2023-12-01T01:37:42.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.970+0000 7f4f51ffb640 1 -- 172.21.15.115:0/315773815 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f4f54013470 con 0x7f4f60157980 2023-12-01T01:37:42.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.971+0000 7f4f51ffb640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4f4406a630 0x7f4f4406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:42.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.971+0000 7f4f51ffb640 1 -- 172.21.15.115:0/315773815 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4f54088fa0 con 0x7f4f60157980 2023-12-01T01:37:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.971+0000 7f4f653a7640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4f4406a630 0x7f4f4406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:42.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.972+0000 7f4f653a7640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4f4406a630 0x7f4f4406caf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4f60151a50 tx=0x7f4f5c000ef0 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:42.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:42.975+0000 7f4f51ffb640 1 -- 172.21.15.115:0/315773815 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4f54054170 con 0x7f4f60157980 2023-12-01T01:37:43.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.187+0000 7f4f6760b640 1 -- 172.21.15.115:0/315773815 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f600cc630 con 0x7f4f4406a630 2023-12-01T01:37:43.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.341+0000 7f4f51ffb640 1 -- 172.21.15.115:0/315773815 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4f5c001bd0 con 0x7f4f4406a630 2023-12-01T01:37:43.343 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/dir7 2023-12-01T01:37:43.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.344+0000 7f4f6760b640 1 -- 172.21.15.115:0/315773815 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4f4406a630 msgr2=0x7f4f4406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:43.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.344+0000 7f4f6760b640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4f4406a630 0x7f4f4406caf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4f60151a50 tx=0x7f4f5c000ef0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.344+0000 7f4f6760b640 1 -- 172.21.15.115:0/315773815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f60157980 msgr2=0x7f4f6006b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:43.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.344+0000 7f4f6760b640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f60157980 0x7f4f6006b500 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400e7a0 tx=0x7f4f5400ec60 comp rx=0 tx=0).stop 2023-12-01T01:37:43.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.344+0000 7f4f6760b640 1 -- 172.21.15.115:0/315773815 shutdown_connections 2023-12-01T01:37:43.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.344+0000 7f4f6760b640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4f4406a630 0x7f4f4406caf0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.344+0000 7f4f6760b640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f60157e00 0x7f4f6006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.344+0000 7f4f6760b640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f60157980 0x7f4f6006b500 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.344+0000 7f4f6760b640 1 --2- 172.21.15.115:0/315773815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f4c005180 0x7f4f601567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.344+0000 7f4f6760b640 1 -- 172.21.15.115:0/315773815 >> 172.21.15.115:0/315773815 conn(0x7f4f60065960 msgr2=0x7f4f600b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:43.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.345+0000 7f4f6760b640 1 -- 172.21.15.115:0/315773815 shutdown_connections 2023-12-01T01:37:43.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.345+0000 7f4f6760b640 1 -- 172.21.15.115:0/315773815 wait complete. 2023-12-01T01:37:43.365 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-12-01T01:37:43.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:43.366 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-12-01T01:37:43.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir8 1M 2023-12-01T01:37:43.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.619+0000 7f6f56a21640 1 Processor -- start 2023-12-01T01:37:43.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.619+0000 7f6f56a21640 1 -- start start 2023-12-01T01:37:43.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f56a21640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6f500cd1e0 0x7f6f500b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:43.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f56a21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 0x7f6f5015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:43.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f56a21640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f500b2f30 con 0x7f6f50142870 2023-12-01T01:37:43.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f56a21640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f500b3ae0 con 0x7f6f500cd1e0 2023-12-01T01:37:43.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f56a21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50142c50 con 0x7f6f500b1c50 2023-12-01T01:37:43.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f4f7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6f500cd1e0 0x7f6f500b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:43.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f4f7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6f500cd1e0 0x7f6f500b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41036/0 (socket says 172.21.15.115:41036) 2023-12-01T01:37:43.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f4f7fe640 1 -- 172.21.15.115:0/2711112169 learned_addr learned my addr 172.21.15.115:0/2711112169 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:43.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f4f7fe640 1 -- 172.21.15.115:0/2711112169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 msgr2=0x7f6f5015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:43.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f54fbe640 1 --2- 172.21.15.115:0/2711112169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 0x7f6f5015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:43.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.621+0000 7f6f4f7fe640 1 --2- 172.21.15.115:0/2711112169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 0x7f6f5015a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.622+0000 7f6f4f7fe640 1 -- 172.21.15.115:0/2711112169 >> v1:172.21.15.142:6789/0 conn(0x7f6f50142870 legacy=0x7f6f500ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:43.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.622+0000 7f6f4f7fe640 1 -- 172.21.15.115:0/2711112169 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f500b20c0 con 0x7f6f500cd1e0 2023-12-01T01:37:43.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.622+0000 7f6f54fbe640 1 --2- 172.21.15.115:0/2711112169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 0x7f6f5015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:43.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.623+0000 7f6f4f7fe640 1 --2- 172.21.15.115:0/2711112169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f500cd1e0 0x7f6f500b1710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009e80 tx=0x7f6f4002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cdb7c99a869c3c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:43.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.623+0000 7f6f4effd640 1 -- 172.21.15.115:0/2711112169 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4002ca50 con 0x7f6f500cd1e0 2023-12-01T01:37:43.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.623+0000 7f6f4effd640 1 -- 172.21.15.115:0/2711112169 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4002cbb0 con 0x7f6f500cd1e0 2023-12-01T01:37:43.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.623+0000 7f6f4effd640 1 -- 172.21.15.115:0/2711112169 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f400343f0 con 0x7f6f500cd1e0 2023-12-01T01:37:43.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.624+0000 7f6f56a21640 1 -- 172.21.15.115:0/2711112169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f500cd1e0 msgr2=0x7f6f500b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:43.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.624+0000 7f6f56a21640 1 --2- 172.21.15.115:0/2711112169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f500cd1e0 0x7f6f500b1710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009e80 tx=0x7f6f4002beb0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.624+0000 7f6f56a21640 1 -- 172.21.15.115:0/2711112169 shutdown_connections 2023-12-01T01:37:43.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.624+0000 7f6f56a21640 1 --2- 172.21.15.115:0/2711112169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 0x7f6f5015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.624+0000 7f6f56a21640 1 --2- 172.21.15.115:0/2711112169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f500cd1e0 0x7f6f500b1710 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.624+0000 7f6f56a21640 1 -- 172.21.15.115:0/2711112169 >> 172.21.15.115:0/2711112169 conn(0x7f6f50065960 msgr2=0x7f6f50065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:43.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.624+0000 7f6f56a21640 1 -- 172.21.15.115:0/2711112169 shutdown_connections 2023-12-01T01:37:43.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.624+0000 7f6f56a21640 1 -- 172.21.15.115:0/2711112169 wait complete. 2023-12-01T01:37:43.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.626+0000 7f6f56a21640 1 Processor -- start 2023-12-01T01:37:43.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.626+0000 7f6f56a21640 1 -- start start 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.627+0000 7f6f56a21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 0x7f6f500ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.627+0000 7f6f56a21640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f500cd1e0 0x7f6f501f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.627+0000 7f6f56a21640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f50142870 0x7f6f501e0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.627+0000 7f6f56a21640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501f2eb0 con 0x7f6f50142870 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.627+0000 7f6f56a21640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501f28f0 con 0x7f6f500cd1e0 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.627+0000 7f6f56a21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501f2bd0 con 0x7f6f500b1c50 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.627+0000 7f6f54fbe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f50142870 0x7f6f501e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.627+0000 7f6f54fbe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f50142870 0x7f6f501e0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41046/0 (socket says 172.21.15.115:41046) 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.627+0000 7f6f54fbe640 1 -- 172.21.15.115:0/1497449814 learned_addr learned my addr 172.21.15.115:0/1497449814 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.627+0000 7f6f4ffff640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 0x7f6f500ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f4f7fe640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f500cd1e0 0x7f6f501f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f54fbe640 1 -- 172.21.15.115:0/1497449814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 msgr2=0x7f6f500ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:43.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f54fbe640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 0x7f6f500ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f54fbe640 1 -- 172.21.15.115:0/1497449814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f500cd1e0 msgr2=0x7f6f501f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:43.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f54fbe640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f500cd1e0 0x7f6f501f1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:43.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f54fbe640 1 -- 172.21.15.115:0/1497449814 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40009b30 con 0x7f6f50142870 2023-12-01T01:37:43.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f54fbe640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f50142870 0x7f6f501e0980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800c7f0 tx=0x7f6f4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:43.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f4cff9640 1 -- 172.21.15.115:0/1497449814 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48007820 con 0x7f6f50142870 2023-12-01T01:37:43.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f4cff9640 1 -- 172.21.15.115:0/1497449814 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f48007980 con 0x7f6f50142870 2023-12-01T01:37:43.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f56a21640 1 -- 172.21.15.115:0/1497449814 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f500ac930 con 0x7f6f50142870 2023-12-01T01:37:43.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f4cff9640 1 -- 172.21.15.115:0/1497449814 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48007d70 con 0x7f6f50142870 2023-12-01T01:37:43.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.628+0000 7f6f56a21640 1 -- 172.21.15.115:0/1497449814 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f500b0100 con 0x7f6f50142870 2023-12-01T01:37:43.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.629+0000 7f6f56a21640 1 -- 172.21.15.115:0/1497449814 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f500ace50 con 0x7f6f50142870 2023-12-01T01:37:43.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.630+0000 7f6f4cff9640 1 -- 172.21.15.115:0/1497449814 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f6f480113f0 con 0x7f6f50142870 2023-12-01T01:37:43.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.631+0000 7f6f4cff9640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f3006a630 0x7f6f3006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:43.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.631+0000 7f6f4cff9640 1 -- 172.21.15.115:0/1497449814 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f6f480892a0 con 0x7f6f50142870 2023-12-01T01:37:43.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.632+0000 7f6f4ffff640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f3006a630 0x7f6f3006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:43.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.632+0000 7f6f4ffff640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f3006a630 0x7f6f3006caf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6f38006fd0 tx=0x7f6f38008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:43.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.636+0000 7f6f4cff9640 1 -- 172.21.15.115:0/1497449814 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6f500ace50 con 0x7f6f50142870 2023-12-01T01:37:43.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.851+0000 7f6f56a21640 1 -- 172.21.15.115:0/1497449814 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f500b2600 con 0x7f6f3006a630 2023-12-01T01:37:43.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:43.997+0000 7f6f4cff9640 1 -- 172.21.15.115:0/1497449814 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6f3800bd80 con 0x7f6f3006a630 2023-12-01T01:37:43.999 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/dir8 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 -- 172.21.15.115:0/1497449814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f3006a630 msgr2=0x7f6f3006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f3006a630 0x7f6f3006caf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6f38006fd0 tx=0x7f6f38008040 comp rx=0 tx=0).stop 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 -- 172.21.15.115:0/1497449814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f50142870 msgr2=0x7f6f501e0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f50142870 0x7f6f501e0980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800c7f0 tx=0x7f6f4800ccb0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 -- 172.21.15.115:0/1497449814 shutdown_connections 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6f3006a630 0x7f6f3006caf0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6f50142870 0x7f6f501e0980 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f500cd1e0 0x7f6f501f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 --2- 172.21.15.115:0/1497449814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f500b1c50 0x7f6f500ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 -- 172.21.15.115:0/1497449814 >> 172.21.15.115:0/1497449814 conn(0x7f6f50065960 msgr2=0x7f6f501413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:44.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 -- 172.21.15.115:0/1497449814 shutdown_connections 2023-12-01T01:37:44.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.001+0000 7f6f56a21640 1 -- 172.21.15.115:0/1497449814 wait complete. 2023-12-01T01:37:44.024 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-12-01T01:37:44.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:37:44.024 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-12-01T01:37:44.077 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir9 1M 2023-12-01T01:37:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.289+0000 7f02705e7640 1 Processor -- start 2023-12-01T01:37:44.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.289+0000 7f02705e7640 1 -- start start 2023-12-01T01:37:44.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.290+0000 7f02705e7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f026815cd90 0x7f02680cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.290+0000 7f02705e7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02680ce5e0 con 0x7f02680cfd60 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.290+0000 7f02705e7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02680ce300 con 0x7f02680b5fa0 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.290+0000 7f02705e7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680d0ec0 con 0x7f026815cd90 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.291+0000 7f026eb84640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f02680cfd60 0x7f02680d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42684/0 (socket says 172.21.15.115:42684) 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.291+0000 7f026eb84640 1 -- 172.21.15.115:0/4276086901 learned_addr learned my addr 172.21.15.115:0/4276086901 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.291+0000 7f026e383640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f026815cd90 0x7f02680cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.291+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596761175 0 0) 0x7f0260001110 con 0x7f02680cfd60 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.291+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0250003520 con 0x7f02680cfd60 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613599517 0 0) 0x7f0250003520 con 0x7f02680cfd60 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f026815cd90 msgr2=0x7f02680cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f026815cd90 0x7f02680cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 >> v1:172.21.15.142:6789/0 conn(0x7f02680b5fa0 legacy=0x7f02680cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680d10a0 con 0x7f02680cfd60 2023-12-01T01:37:44.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0260001800 con 0x7f02680cfd60 2023-12-01T01:37:44.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 >> v1:172.21.15.115:6789/0 conn(0x7f02680cfd60 legacy=0x7f02680d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02500039a0 0x7f0250003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f026815cd90 0x7f02500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0250004d00 0x7f0250005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680d0ec0 con 0x7f02500039a0 2023-12-01T01:37:44.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025000bb60 con 0x7f0250004d00 2023-12-01T01:37:44.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025000bd20 con 0x7f026815cd90 2023-12-01T01:37:44.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026e383640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02500039a0 0x7f0250003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026eb84640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0250004d00 0x7f0250005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026db82640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f026815cd90 0x7f02500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026e383640 1 -- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f026815cd90 msgr2=0x7f02500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:44.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026e383640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f026815cd90 0x7f02500047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026e383640 1 -- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0250004d00 msgr2=0x7f0250005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:44.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026e383640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0250004d00 0x7f0250005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.292+0000 7f026e383640 1 -- 172.21.15.115:0/4276086901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0250003520 con 0x7f02500039a0 2023-12-01T01:37:44.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.294+0000 7f026e383640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02500039a0 0x7f0250003dc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0258007360 tx=0x7f02500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1b2d2667f7154b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:44.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.294+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025802ccd0 con 0x7f02500039a0 2023-12-01T01:37:44.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.294+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025802ce30 con 0x7f02500039a0 2023-12-01T01:37:44.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.294+0000 7f026d381640 1 -- 172.21.15.115:0/4276086901 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258034660 con 0x7f02500039a0 2023-12-01T01:37:44.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.294+0000 7f02705e7640 1 -- 172.21.15.115:0/4276086901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02500039a0 msgr2=0x7f0250003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.294+0000 7f02705e7640 1 --2- 172.21.15.115:0/4276086901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02500039a0 0x7f0250003dc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0258007360 tx=0x7f02500057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.294+0000 7f026eb84640 1 -- 172.21.15.115:0/4276086901 reap_dead start 2023-12-01T01:37:44.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.295+0000 7f02705e7640 1 -- 172.21.15.115:0/4276086901 shutdown_connections 2023-12-01T01:37:44.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.295+0000 7f02705e7640 1 -- 172.21.15.115:0/4276086901 >> 172.21.15.115:0/4276086901 conn(0x7f0268065960 msgr2=0x7f0268065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:44.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.295+0000 7f02705e7640 1 -- 172.21.15.115:0/4276086901 shutdown_connections 2023-12-01T01:37:44.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.295+0000 7f02705e7640 1 -- 172.21.15.115:0/4276086901 wait complete. 2023-12-01T01:37:44.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.296+0000 7f02705e7640 1 Processor -- start 2023-12-01T01:37:44.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.296+0000 7f02705e7640 1 -- start start 2023-12-01T01:37:44.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.297+0000 7f02705e7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02500039a0 0x7f02681f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.297+0000 7f02705e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f026815d4d0 0x7f026815d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.297+0000 7f02705e7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f026815fa70 0x7f026815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.297+0000 7f02705e7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026815e550 con 0x7f026815d4d0 2023-12-01T01:37:44.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.297+0000 7f02705e7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02681e6070 con 0x7f026815fa70 2023-12-01T01:37:44.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.297+0000 7f02705e7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02681f3b40 con 0x7f02500039a0 2023-12-01T01:37:44.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.298+0000 7f026db82640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f026815d4d0 0x7f026815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.298+0000 7f026db82640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f026815d4d0 0x7f026815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41064/0 (socket says 172.21.15.115:41064) 2023-12-01T01:37:44.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.298+0000 7f026db82640 1 -- 172.21.15.115:0/2737674216 learned_addr learned my addr 172.21.15.115:0/2737674216 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:44.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.298+0000 7f026db82640 1 -- 172.21.15.115:0/2737674216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02500039a0 msgr2=0x7f02681f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.298+0000 7f026db82640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02500039a0 0x7f02681f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.298+0000 7f026db82640 1 -- 172.21.15.115:0/2737674216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f026815fa70 msgr2=0x7f026815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.298+0000 7f026eb84640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f026815fa70 0x7f026815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.298+0000 7f026db82640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f026815fa70 0x7f026815fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.298+0000 7f026db82640 1 -- 172.21.15.115:0/2737674216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026815df90 con 0x7f026815d4d0 2023-12-01T01:37:44.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.299+0000 7f026db82640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f026815d4d0 0x7f026815d950 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f025c00d5a0 tx=0x7f025c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:44.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.299+0000 7f0266ffd640 1 -- 172.21.15.115:0/2737674216 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c005480 con 0x7f026815d4d0 2023-12-01T01:37:44.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.299+0000 7f0266ffd640 1 -- 172.21.15.115:0/2737674216 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c003610 con 0x7f026815d4d0 2023-12-01T01:37:44.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.299+0000 7f0266ffd640 1 -- 172.21.15.115:0/2737674216 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c003980 con 0x7f026815d4d0 2023-12-01T01:37:44.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.299+0000 7f02705e7640 1 -- 172.21.15.115:0/2737674216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026815e270 con 0x7f026815d4d0 2023-12-01T01:37:44.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.299+0000 7f02705e7640 1 -- 172.21.15.115:0/2737674216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0268160430 con 0x7f026815d4d0 2023-12-01T01:37:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.300+0000 7f02705e7640 1 -- 172.21.15.115:0/2737674216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026804e890 con 0x7f026815d4d0 2023-12-01T01:37:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.301+0000 7f0266ffd640 1 -- 172.21.15.115:0/2737674216 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f025c003ae0 con 0x7f026815d4d0 2023-12-01T01:37:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.302+0000 7f0266ffd640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025406a660 0x7f025406cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.302+0000 7f0266ffd640 1 -- 172.21.15.115:0/2737674216 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f025c088d50 con 0x7f026815d4d0 2023-12-01T01:37:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.302+0000 7f026e383640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025406a660 0x7f025406cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.303+0000 7f026e383640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025406a660 0x7f025406cb20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0258036000 tx=0x7f025802e000 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:44.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.306+0000 7f0266ffd640 1 -- 172.21.15.115:0/2737674216 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f025c053ea0 con 0x7f026815d4d0 2023-12-01T01:37:44.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.519+0000 7f02705e7640 1 -- 172.21.15.115:0/2737674216 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f02680d1660 con 0x7f025406a660 2023-12-01T01:37:44.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.671+0000 7f0266ffd640 1 -- 172.21.15.115:0/2737674216 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f025802ccb0 con 0x7f025406a660 2023-12-01T01:37:44.673 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/dir9 2023-12-01T01:37:44.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 -- 172.21.15.115:0/2737674216 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025406a660 msgr2=0x7f025406cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025406a660 0x7f025406cb20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0258036000 tx=0x7f025802e000 comp rx=0 tx=0).stop 2023-12-01T01:37:44.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 -- 172.21.15.115:0/2737674216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f026815d4d0 msgr2=0x7f026815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f026815d4d0 0x7f026815d950 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f025c00d5a0 tx=0x7f025c00da60 comp rx=0 tx=0).stop 2023-12-01T01:37:44.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 -- 172.21.15.115:0/2737674216 shutdown_connections 2023-12-01T01:37:44.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025406a660 0x7f025406cb20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f026815fa70 0x7f026815fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f026815d4d0 0x7f026815d950 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 --2- 172.21.15.115:0/2737674216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02500039a0 0x7f02681f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 -- 172.21.15.115:0/2737674216 >> 172.21.15.115:0/2737674216 conn(0x7f0268065960 msgr2=0x7f02680d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:44.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 -- 172.21.15.115:0/2737674216 shutdown_connections 2023-12-01T01:37:44.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.675+0000 7f02705e7640 1 -- 172.21.15.115:0/2737674216 wait complete. 2023-12-01T01:37:44.697 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-12-01T01:37:44.698 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir0 1M 2023-12-01T01:37:44.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.909+0000 7f40ce1ec640 1 Processor -- start 2023-12-01T01:37:44.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.910+0000 7f40ce1ec640 1 -- start start 2023-12-01T01:37:44.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.911+0000 7f40ce1ec640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 0x7f40c80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.911+0000 7f40ce1ec640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f40c80b5fa0 0x7f40c8161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.911+0000 7f40ce1ec640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c80ce5e0 con 0x7f40c80cf650 2023-12-01T01:37:44.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.911+0000 7f40ce1ec640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c80d0ec0 con 0x7f40c80b5fa0 2023-12-01T01:37:44.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.911+0000 7f40ce1ec640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8161950 con 0x7f40c815cd90 2023-12-01T01:37:44.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.912+0000 7f40c77fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 0x7f40c80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.912+0000 7f40c6ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f40c80b5fa0 0x7f40c8161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.912+0000 7f40c6ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f40c80b5fa0 0x7f40c8161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:46408/0 (socket says 172.21.15.115:46408) 2023-12-01T01:37:44.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.912+0000 7f40c6ffd640 1 -- 172.21.15.115:0/218479647 learned_addr learned my addr 172.21.15.115:0/218479647 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:44.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.912+0000 7f40c77fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 0x7f40c80cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:41858/0 (socket says 172.21.15.115:41858) 2023-12-01T01:37:44.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.913+0000 7f40c6ffd640 1 -- 172.21.15.115:0/218479647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 msgr2=0x7f40c80cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:44.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.913+0000 7f40c6ffd640 1 --2- 172.21.15.115:0/218479647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 0x7f40c80cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.913+0000 7f40c6ffd640 1 -- 172.21.15.115:0/218479647 >> v1:172.21.15.142:6789/0 conn(0x7f40c80cf650 legacy=0x7f40c80b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.913+0000 7f40c6ffd640 1 -- 172.21.15.115:0/218479647 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c80ce300 con 0x7f40c80b5fa0 2023-12-01T01:37:44.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.914+0000 7f40c6ffd640 1 --2- 172.21.15.115:0/218479647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c80b5fa0 0x7f40c8161410 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009870 tx=0x7f40b002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1c26005b99146f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:44.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.915+0000 7f40c67fc640 1 -- 172.21.15.115:0/218479647 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0030840 con 0x7f40c80b5fa0 2023-12-01T01:37:44.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.915+0000 7f40c67fc640 1 -- 172.21.15.115:0/218479647 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b00309a0 con 0x7f40c80b5fa0 2023-12-01T01:37:44.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.915+0000 7f40c67fc640 1 -- 172.21.15.115:0/218479647 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0030d30 con 0x7f40c80b5fa0 2023-12-01T01:37:44.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.915+0000 7f40ce1ec640 1 -- 172.21.15.115:0/218479647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c80b5fa0 msgr2=0x7f40c8161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.915+0000 7f40ce1ec640 1 --2- 172.21.15.115:0/218479647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c80b5fa0 0x7f40c8161410 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009870 tx=0x7f40b002b980 comp rx=0 tx=0).stop 2023-12-01T01:37:44.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.915+0000 7f40ce1ec640 1 -- 172.21.15.115:0/218479647 shutdown_connections 2023-12-01T01:37:44.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.915+0000 7f40ce1ec640 1 --2- 172.21.15.115:0/218479647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c80b5fa0 0x7f40c8161410 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.915+0000 7f40ce1ec640 1 --2- 172.21.15.115:0/218479647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 0x7f40c80cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.915+0000 7f40ce1ec640 1 -- 172.21.15.115:0/218479647 >> 172.21.15.115:0/218479647 conn(0x7f40c8065960 msgr2=0x7f40c8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:44.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.915+0000 7f40ce1ec640 1 -- 172.21.15.115:0/218479647 shutdown_connections 2023-12-01T01:37:44.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.916+0000 7f40ce1ec640 1 -- 172.21.15.115:0/218479647 wait complete. 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.917+0000 7f40ce1ec640 1 Processor -- start 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.917+0000 7f40ce1ec640 1 -- start start 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.918+0000 7f40ce1ec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c80b5fa0 0x7f40c815dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.918+0000 7f40ce1ec640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c80cf650 0x7f40c81f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.918+0000 7f40ce1ec640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 0x7f40c81530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.918+0000 7f40ce1ec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c81f33a0 con 0x7f40c80b5fa0 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.918+0000 7f40ce1ec640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c81f2410 con 0x7f40c80cf650 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.918+0000 7f40ce1ec640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c815d400 con 0x7f40c815cd90 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c80b5fa0 0x7f40c815dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c80b5fa0 0x7f40c815dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41068/0 (socket says 172.21.15.115:41068) 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c77fe640 1 -- 172.21.15.115:0/2714514317 learned_addr learned my addr 172.21.15.115:0/2714514317 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c7fff640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 0x7f40c81530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c6ffd640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c80cf650 0x7f40c81f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c77fe640 1 -- 172.21.15.115:0/2714514317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 msgr2=0x7f40c81530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c77fe640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 0x7f40c81530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c77fe640 1 -- 172.21.15.115:0/2714514317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c80cf650 msgr2=0x7f40c81f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c77fe640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c80cf650 0x7f40c81f2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c77fe640 1 -- 172.21.15.115:0/2714514317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00094d0 con 0x7f40c80b5fa0 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c7fff640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 0x7f40c81530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.919+0000 7f40c77fe640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c80b5fa0 0x7f40c815dad0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f40b800efc0 tx=0x7f40b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.920+0000 7f40bffff640 1 -- 172.21.15.115:0/2714514317 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8014480 con 0x7f40c80b5fa0 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.920+0000 7f40bffff640 1 -- 172.21.15.115:0/2714514317 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b800f040 con 0x7f40c80b5fa0 2023-12-01T01:37:44.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.920+0000 7f40bffff640 1 -- 172.21.15.115:0/2714514317 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8014800 con 0x7f40c80b5fa0 2023-12-01T01:37:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.920+0000 7f40ce1ec640 1 -- 172.21.15.115:0/2714514317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c815e160 con 0x7f40c80b5fa0 2023-12-01T01:37:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.920+0000 7f40ce1ec640 1 -- 172.21.15.115:0/2714514317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c81f2660 con 0x7f40c80b5fa0 2023-12-01T01:37:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.921+0000 7f40ce1ec640 1 -- 172.21.15.115:0/2714514317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c815e680 con 0x7f40c80b5fa0 2023-12-01T01:37:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.922+0000 7f40bffff640 1 -- 172.21.15.115:0/2714514317 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f40b8014960 con 0x7f40c80b5fa0 2023-12-01T01:37:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.923+0000 7f40bffff640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40a806a630 0x7f40a806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.923+0000 7f40bffff640 1 -- 172.21.15.115:0/2714514317 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f40b8089bd0 con 0x7f40c80b5fa0 2023-12-01T01:37:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.924+0000 7f40c6ffd640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40a806a630 0x7f40a806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.924+0000 7f40c6ffd640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40a806a630 0x7f40a806caf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40b0036000 tx=0x7f40b003b040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:44.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:44.928+0000 7f40bffff640 1 -- 172.21.15.115:0/2714514317 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f40c815e680 con 0x7f40c80b5fa0 2023-12-01T01:37:45.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.140+0000 7f40ce1ec640 1 -- 172.21.15.115:0/2714514317 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f40c80d10a0 con 0x7f40a806a630 2023-12-01T01:37:45.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.263+0000 7f40bffff640 1 -- 172.21.15.115:0/2714514317 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f40b0030b90 con 0x7f40a806a630 2023-12-01T01:37:45.265 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-12-01T01:37:45.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.266+0000 7f40ce1ec640 1 -- 172.21.15.115:0/2714514317 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40a806a630 msgr2=0x7f40a806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:45.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.266+0000 7f40ce1ec640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40a806a630 0x7f40a806caf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40b0036000 tx=0x7f40b003b040 comp rx=0 tx=0).stop 2023-12-01T01:37:45.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.266+0000 7f40ce1ec640 1 -- 172.21.15.115:0/2714514317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c80b5fa0 msgr2=0x7f40c815dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:45.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.266+0000 7f40ce1ec640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c80b5fa0 0x7f40c815dad0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f40b800efc0 tx=0x7f40b800c490 comp rx=0 tx=0).stop 2023-12-01T01:37:45.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.266+0000 7f40ce1ec640 1 -- 172.21.15.115:0/2714514317 shutdown_connections 2023-12-01T01:37:45.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.266+0000 7f40ce1ec640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40a806a630 0x7f40a806caf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.266+0000 7f40ce1ec640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40c815cd90 0x7f40c81530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.266+0000 7f40ce1ec640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40c80cf650 0x7f40c81f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.266+0000 7f40ce1ec640 1 --2- 172.21.15.115:0/2714514317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40c80b5fa0 0x7f40c815dad0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.266+0000 7f40ce1ec640 1 -- 172.21.15.115:0/2714514317 >> 172.21.15.115:0/2714514317 conn(0x7f40c8065960 msgr2=0x7f40c80b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:45.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.267+0000 7f40ce1ec640 1 -- 172.21.15.115:0/2714514317 shutdown_connections 2023-12-01T01:37:45.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.267+0000 7f40ce1ec640 1 -- 172.21.15.115:0/2714514317 wait complete. 2023-12-01T01:37:45.288 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-12-01T01:37:45.289 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir1 1M 2023-12-01T01:37:45.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.508+0000 7f460f5ce640 1 Processor -- start 2023-12-01T01:37:45.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.508+0000 7f460f5ce640 1 -- start start 2023-12-01T01:37:45.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.510+0000 7f460f5ce640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f46080b5cd0 0x7f46080b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:45.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.510+0000 7f460f5ce640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f46080cfc10 0x7f46080d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:45.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.510+0000 7f460f5ce640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46080d05a0 con 0x7f460815cfa0 2023-12-01T01:37:45.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.510+0000 7f460f5ce640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46080b66a0 con 0x7f46080b5cd0 2023-12-01T01:37:45.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.510+0000 7f460f5ce640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46080b6890 con 0x7f46080cfc10 2023-12-01T01:37:45.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.510+0000 7f460cb69640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f46080b5cd0 0x7f46080b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:45.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.510+0000 7f460cb69640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f46080b5cd0 0x7f46080b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41076/0 (socket says 172.21.15.115:41076) 2023-12-01T01:37:45.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.510+0000 7f460cb69640 1 -- 172.21.15.115:0/3892904561 learned_addr learned my addr 172.21.15.115:0/3892904561 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:45.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.510+0000 7f460db6b640 1 --2- 172.21.15.115:0/3892904561 >> v2:172.21.15.142:3300/0 conn(0x7f46080cfc10 0x7f46080d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:45.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.511+0000 7f460cb69640 1 -- 172.21.15.115:0/3892904561 >> v2:172.21.15.142:3300/0 conn(0x7f46080cfc10 msgr2=0x7f46080d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:45.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.511+0000 7f460cb69640 1 --2- 172.21.15.115:0/3892904561 >> v2:172.21.15.142:3300/0 conn(0x7f46080cfc10 0x7f46080d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.511+0000 7f460cb69640 1 -- 172.21.15.115:0/3892904561 >> v1:172.21.15.142:6789/0 conn(0x7f460815cfa0 legacy=0x7f46080b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:45.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.511+0000 7f460cb69640 1 -- 172.21.15.115:0/3892904561 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080b6b70 con 0x7f46080b5cd0 2023-12-01T01:37:45.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.512+0000 7f460cb69640 1 --2- 172.21.15.115:0/3892904561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080b5cd0 0x7f46080b6160 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f46080c9a60 tx=0x7f45f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9d1bca7ea36b8f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:45.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.512+0000 7f45fffff640 1 -- 172.21.15.115:0/3892904561 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80317e0 con 0x7f46080b5cd0 2023-12-01T01:37:45.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.512+0000 7f45fffff640 1 -- 172.21.15.115:0/3892904561 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f8031940 con 0x7f46080b5cd0 2023-12-01T01:37:45.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.512+0000 7f45fffff640 1 -- 172.21.15.115:0/3892904561 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8031cd0 con 0x7f46080b5cd0 2023-12-01T01:37:45.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.513+0000 7f460f5ce640 1 -- 172.21.15.115:0/3892904561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080b5cd0 msgr2=0x7f46080b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:45.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.513+0000 7f460f5ce640 1 --2- 172.21.15.115:0/3892904561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080b5cd0 0x7f46080b6160 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f46080c9a60 tx=0x7f45f802b880 comp rx=0 tx=0).stop 2023-12-01T01:37:45.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.513+0000 7f460f5ce640 1 -- 172.21.15.115:0/3892904561 shutdown_connections 2023-12-01T01:37:45.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.513+0000 7f460f5ce640 1 --2- 172.21.15.115:0/3892904561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080b5cd0 0x7f46080b6160 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.513+0000 7f460f5ce640 1 --2- 172.21.15.115:0/3892904561 >> v2:172.21.15.142:3300/0 conn(0x7f46080cfc10 0x7f46080d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.513+0000 7f460f5ce640 1 -- 172.21.15.115:0/3892904561 >> 172.21.15.115:0/3892904561 conn(0x7f460806a0b0 msgr2=0x7f460806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:45.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.513+0000 7f460f5ce640 1 -- 172.21.15.115:0/3892904561 shutdown_connections 2023-12-01T01:37:45.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.513+0000 7f460f5ce640 1 -- 172.21.15.115:0/3892904561 wait complete. 2023-12-01T01:37:45.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.515+0000 7f460f5ce640 1 Processor -- start 2023-12-01T01:37:45.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.515+0000 7f460f5ce640 1 -- start start 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460f5ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46080b5cd0 0x7f46080d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460f5ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080cfc10 0x7f460815de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460f5ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f460815cfa0 0x7f4608150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460f5ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080cc1b0 con 0x7f46080cfc10 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460f5ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46081571f0 con 0x7f460815cfa0 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460f5ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460815f110 con 0x7f46080b5cd0 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460cb69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080cfc10 0x7f460815de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460cb69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080cfc10 0x7f460815de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41078/0 (socket says 172.21.15.115:41078) 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460cb69640 1 -- 172.21.15.115:0/3816638593 learned_addr learned my addr 172.21.15.115:0/3816638593 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460d36a640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46080b5cd0 0x7f46080d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:45.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.516+0000 7f460cb69640 1 -- 172.21.15.115:0/3816638593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46080b5cd0 msgr2=0x7f46080d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f460cb69640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46080b5cd0 0x7f46080d21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f460db6b640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f460815cfa0 0x7f4608150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f460cb69640 1 -- 172.21.15.115:0/3816638593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f460815cfa0 msgr2=0x7f4608150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f460cb69640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f460815cfa0 0x7f4608150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f460cb69640 1 -- 172.21.15.115:0/3816638593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f80094d0 con 0x7f46080cfc10 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f460cb69640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080cfc10 0x7f460815de20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f45f802b8c0 tx=0x7f45f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f45fdffb640 1 -- 172.21.15.115:0/3816638593 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8032880 con 0x7f46080cfc10 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f45fdffb640 1 -- 172.21.15.115:0/3816638593 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f80329e0 con 0x7f46080cfc10 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f460f5ce640 1 -- 172.21.15.115:0/3816638593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4608150f50 con 0x7f46080cfc10 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f45fdffb640 1 -- 172.21.15.115:0/3816638593 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8032d90 con 0x7f46080cfc10 2023-12-01T01:37:45.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.517+0000 7f460f5ce640 1 -- 172.21.15.115:0/3816638593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460815e7e0 con 0x7f46080cfc10 2023-12-01T01:37:45.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.519+0000 7f460f5ce640 1 -- 172.21.15.115:0/3816638593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460804e890 con 0x7f46080cfc10 2023-12-01T01:37:45.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.519+0000 7f45fdffb640 1 -- 172.21.15.115:0/3816638593 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f45f804e450 con 0x7f46080cfc10 2023-12-01T01:37:45.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.520+0000 7f45fdffb640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f45ec06a630 0x7f45ec06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:45.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.520+0000 7f45fdffb640 1 -- 172.21.15.115:0/3816638593 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f45f80aae20 con 0x7f46080cfc10 2023-12-01T01:37:45.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.520+0000 7f460d36a640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f45ec06a630 0x7f45ec06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:45.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.521+0000 7f460d36a640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f45ec06a630 0x7f45ec06caf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4604003d90 tx=0x7f4604009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:45.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.525+0000 7f45fdffb640 1 -- 172.21.15.115:0/3816638593 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f45f8075f60 con 0x7f46080cfc10 2023-12-01T01:37:45.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.739+0000 7f460f5ce640 1 -- 172.21.15.115:0/3816638593 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f46080b7130 con 0x7f45ec06a630 2023-12-01T01:37:45.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.864+0000 7f45fdffb640 1 -- 172.21.15.115:0/3816638593 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4604003980 con 0x7f45ec06a630 2023-12-01T01:37:45.866 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 -- 172.21.15.115:0/3816638593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f45ec06a630 msgr2=0x7f45ec06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f45ec06a630 0x7f45ec06caf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4604003d90 tx=0x7f4604009040 comp rx=0 tx=0).stop 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 -- 172.21.15.115:0/3816638593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080cfc10 msgr2=0x7f460815de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080cfc10 0x7f460815de20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f45f802b8c0 tx=0x7f45f80046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 -- 172.21.15.115:0/3816638593 shutdown_connections 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f45ec06a630 0x7f45ec06caf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f460815cfa0 0x7f4608150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46080cfc10 0x7f460815de20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 --2- 172.21.15.115:0/3816638593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46080b5cd0 0x7f46080d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 -- 172.21.15.115:0/3816638593 >> 172.21.15.115:0/3816638593 conn(0x7f460806a0b0 msgr2=0x7f4608153220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 -- 172.21.15.115:0/3816638593 shutdown_connections 2023-12-01T01:37:45.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:45.867+0000 7f460f5ce640 1 -- 172.21.15.115:0/3816638593 wait complete. 2023-12-01T01:37:45.889 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-12-01T01:37:45.889 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir2 1M 2023-12-01T01:37:46.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.105+0000 7f42ecf89640 1 Processor -- start 2023-12-01T01:37:46.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.106+0000 7f42ecf89640 1 -- start start 2023-12-01T01:37:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.107+0000 7f42ecf89640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f42e80cfd60 0x7f42e80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.107+0000 7f42ecf89640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f42e80b5fa0 0x7f42e8161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.107+0000 7f42ecf89640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e80ce5e0 con 0x7f42e815cd90 2023-12-01T01:37:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.107+0000 7f42ecf89640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e80d0ec0 con 0x7f42e80b5fa0 2023-12-01T01:37:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.107+0000 7f42ecf89640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e8161950 con 0x7f42e80cfd60 2023-12-01T01:37:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.108+0000 7f42e659c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f42e815cd90 0x7f42e80cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42692/0 (socket says 172.21.15.115:42692) 2023-12-01T01:37:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.108+0000 7f42e659c640 1 -- 172.21.15.115:0/785885288 learned_addr learned my addr 172.21.15.115:0/785885288 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.108+0000 7f42e6d9d640 1 --2- 172.21.15.115:0/785885288 >> v2:172.21.15.115:3300/0 conn(0x7f42e80b5fa0 0x7f42e8161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.108+0000 7f42e5d9b640 1 --2- 172.21.15.115:0/785885288 >> v2:172.21.15.142:3300/0 conn(0x7f42e80cfd60 0x7f42e80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.108+0000 7f42e559a640 1 -- 172.21.15.115:0/785885288 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156017041 0 0) 0x7f42d8001490 con 0x7f42e815cd90 2023-12-01T01:37:46.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.108+0000 7f42e559a640 1 -- 172.21.15.115:0/785885288 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d4003520 con 0x7f42e815cd90 2023-12-01T01:37:46.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.108+0000 7f42e6d9d640 1 -- 172.21.15.115:0/785885288 >> v2:172.21.15.142:3300/0 conn(0x7f42e80cfd60 msgr2=0x7f42e80cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:46.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.108+0000 7f42e6d9d640 1 --2- 172.21.15.115:0/785885288 >> v2:172.21.15.142:3300/0 conn(0x7f42e80cfd60 0x7f42e80cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.108+0000 7f42e6d9d640 1 -- 172.21.15.115:0/785885288 >> v1:172.21.15.115:6789/0 conn(0x7f42e815cd90 legacy=0x7f42e80cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.108+0000 7f42e6d9d640 1 -- 172.21.15.115:0/785885288 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e80ce300 con 0x7f42e80b5fa0 2023-12-01T01:37:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.109+0000 7f42e6d9d640 1 --2- 172.21.15.115:0/785885288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80b5fa0 0x7f42e8161410 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f42e80cdcb0 tx=0x7f42e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91a60bc81243da4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.110+0000 7f42e559a640 1 -- 172.21.15.115:0/785885288 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0031830 con 0x7f42e80b5fa0 2023-12-01T01:37:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.110+0000 7f42e559a640 1 -- 172.21.15.115:0/785885288 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e0031990 con 0x7f42e80b5fa0 2023-12-01T01:37:46.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.110+0000 7f42e559a640 1 -- 172.21.15.115:0/785885288 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0031d20 con 0x7f42e80b5fa0 2023-12-01T01:37:46.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.110+0000 7f42ecf89640 1 -- 172.21.15.115:0/785885288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80b5fa0 msgr2=0x7f42e8161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:46.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.110+0000 7f42ecf89640 1 --2- 172.21.15.115:0/785885288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80b5fa0 0x7f42e8161410 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f42e80cdcb0 tx=0x7f42e002b8d0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.110+0000 7f42ecf89640 1 -- 172.21.15.115:0/785885288 shutdown_connections 2023-12-01T01:37:46.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.110+0000 7f42ecf89640 1 --2- 172.21.15.115:0/785885288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80b5fa0 0x7f42e8161410 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.110+0000 7f42ecf89640 1 --2- 172.21.15.115:0/785885288 >> v2:172.21.15.142:3300/0 conn(0x7f42e80cfd60 0x7f42e80cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.110+0000 7f42ecf89640 1 -- 172.21.15.115:0/785885288 >> 172.21.15.115:0/785885288 conn(0x7f42e8065960 msgr2=0x7f42e8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.110+0000 7f42ecf89640 1 -- 172.21.15.115:0/785885288 shutdown_connections 2023-12-01T01:37:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.111+0000 7f42ecf89640 1 -- 172.21.15.115:0/785885288 wait complete. 2023-12-01T01:37:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.112+0000 7f42ecf89640 1 Processor -- start 2023-12-01T01:37:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.112+0000 7f42ecf89640 1 -- start start 2023-12-01T01:37:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.113+0000 7f42ecf89640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e80b5fa0 0x7f42e8153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.113+0000 7f42ecf89640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80cfd60 0x7f42e815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.113+0000 7f42ecf89640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e815cd90 0x7f42e8154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.113+0000 7f42ecf89640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e815ebd0 con 0x7f42e80cfd60 2023-12-01T01:37:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.113+0000 7f42ecf89640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81f26f0 con 0x7f42e80b5fa0 2023-12-01T01:37:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.113+0000 7f42ecf89640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e815dae0 con 0x7f42e815cd90 2023-12-01T01:37:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42e5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80cfd60 0x7f42e815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42e5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80cfd60 0x7f42e815d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41100/0 (socket says 172.21.15.115:41100) 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42e5d9b640 1 -- 172.21.15.115:0/4205501474 learned_addr learned my addr 172.21.15.115:0/4205501474 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42e659c640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e80b5fa0 0x7f42e8153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42e5d9b640 1 -- 172.21.15.115:0/4205501474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e815cd90 msgr2=0x7f42e8154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42e5d9b640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e815cd90 0x7f42e8154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42e5d9b640 1 -- 172.21.15.115:0/4205501474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e80b5fa0 msgr2=0x7f42e8153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42e5d9b640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e80b5fa0 0x7f42e8153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42e5d9b640 1 -- 172.21.15.115:0/4205501474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e0009400 con 0x7f42e80cfd60 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42e5d9b640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80cfd60 0x7f42e815d170 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f42d000d800 tx=0x7f42d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.114+0000 7f42deffd640 1 -- 172.21.15.115:0/4205501474 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000b6c0 con 0x7f42e80cfd60 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.115+0000 7f42deffd640 1 -- 172.21.15.115:0/4205501474 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d000b820 con 0x7f42e80cfd60 2023-12-01T01:37:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.115+0000 7f42deffd640 1 -- 172.21.15.115:0/4205501474 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000bbd0 con 0x7f42e80cfd60 2023-12-01T01:37:46.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.115+0000 7f42ecf89640 1 -- 172.21.15.115:0/4205501474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e815ddc0 con 0x7f42e80cfd60 2023-12-01T01:37:46.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.115+0000 7f42ecf89640 1 -- 172.21.15.115:0/4205501474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e815e860 con 0x7f42e80cfd60 2023-12-01T01:37:46.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.116+0000 7f42ecf89640 1 -- 172.21.15.115:0/4205501474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e815ed40 con 0x7f42e80cfd60 2023-12-01T01:37:46.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.117+0000 7f42deffd640 1 -- 172.21.15.115:0/4205501474 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f42d000bd30 con 0x7f42e80cfd60 2023-12-01T01:37:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.117+0000 7f42deffd640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f42c806a630 0x7f42c806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.118+0000 7f42deffd640 1 -- 172.21.15.115:0/4205501474 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f42d00892e0 con 0x7f42e80cfd60 2023-12-01T01:37:46.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.118+0000 7f42e659c640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f42c806a630 0x7f42c806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.118+0000 7f42e659c640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f42c806a630 0x7f42c806caf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f42d80016f0 tx=0x7f42d8009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:46.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.122+0000 7f42deffd640 1 -- 172.21.15.115:0/4205501474 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f42d00544f0 con 0x7f42e80cfd60 2023-12-01T01:37:46.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.339+0000 7f42ecf89640 1 -- 172.21.15.115:0/4205501474 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f42e80d1380 con 0x7f42c806a630 2023-12-01T01:37:46.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.465+0000 7f42deffd640 1 -- 172.21.15.115:0/4205501474 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f42d8001cc0 con 0x7f42c806a630 2023-12-01T01:37:46.467 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 -- 172.21.15.115:0/4205501474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f42c806a630 msgr2=0x7f42c806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f42c806a630 0x7f42c806caf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f42d80016f0 tx=0x7f42d8009040 comp rx=0 tx=0).stop 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 -- 172.21.15.115:0/4205501474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80cfd60 msgr2=0x7f42e815d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80cfd60 0x7f42e815d170 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f42d000d800 tx=0x7f42d000dcc0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 -- 172.21.15.115:0/4205501474 shutdown_connections 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f42c806a630 0x7f42c806caf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e815cd90 0x7f42e8154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42e80cfd60 0x7f42e815d170 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 --2- 172.21.15.115:0/4205501474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e80b5fa0 0x7f42e8153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 -- 172.21.15.115:0/4205501474 >> 172.21.15.115:0/4205501474 conn(0x7f42e8065960 msgr2=0x7f42e80d1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 -- 172.21.15.115:0/4205501474 shutdown_connections 2023-12-01T01:37:46.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.469+0000 7f42ecf89640 1 -- 172.21.15.115:0/4205501474 wait complete. 2023-12-01T01:37:46.492 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-12-01T01:37:46.493 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir3 1M 2023-12-01T01:37:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.705+0000 7f507f238640 1 Processor -- start 2023-12-01T01:37:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.705+0000 7f507f238640 1 -- start start 2023-12-01T01:37:46.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.706+0000 7f507f238640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f50780ce1b0 0x7f50780cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.706+0000 7f507f238640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50780b0e60 0x7f50780b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.706+0000 7f507f238640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50780cc9a0 con 0x7f507815ac80 2023-12-01T01:37:46.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.706+0000 7f507f238640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507815b060 con 0x7f50780ce1b0 2023-12-01T01:37:46.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.706+0000 7f507f238640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780b0ce0 con 0x7f50780b0e60 2023-12-01T01:37:46.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.707+0000 7f507cfd4640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f507815ac80 0x7f50780cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42700/0 (socket says 172.21.15.115:42700) 2023-12-01T01:37:46.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.707+0000 7f507cfd4640 1 -- 172.21.15.115:0/1979975488 learned_addr learned my addr 172.21.15.115:0/1979975488 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:46.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.707+0000 7f5077fff640 1 --2- 172.21.15.115:0/1979975488 >> v2:172.21.15.115:3300/0 conn(0x7f50780ce1b0 0x7f50780cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.707+0000 7f50777fe640 1 -- 172.21.15.115:0/1979975488 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614757389 0 0) 0x7f50780cc9a0 con 0x7f507815ac80 2023-12-01T01:37:46.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.707+0000 7f507d7d5640 1 --2- 172.21.15.115:0/1979975488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50780b0e60 0x7f50780b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.707+0000 7f50777fe640 1 -- 172.21.15.115:0/1979975488 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5068003520 con 0x7f507815ac80 2023-12-01T01:37:46.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.707+0000 7f5077fff640 1 -- 172.21.15.115:0/1979975488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50780b0e60 msgr2=0x7f50780b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:46.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.707+0000 7f5077fff640 1 --2- 172.21.15.115:0/1979975488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50780b0e60 0x7f50780b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.707+0000 7f5077fff640 1 -- 172.21.15.115:0/1979975488 >> v1:172.21.15.115:6789/0 conn(0x7f507815ac80 legacy=0x7f50780cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:46.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.707+0000 7f5077fff640 1 -- 172.21.15.115:0/1979975488 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780cbd90 con 0x7f50780ce1b0 2023-12-01T01:37:46.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.709+0000 7f5077fff640 1 --2- 172.21.15.115:0/1979975488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50780ce1b0 0x7f50780cce30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f50640097a0 tx=0x7f506402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c476a6490166c61d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:46.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.709+0000 7f50777fe640 1 -- 172.21.15.115:0/1979975488 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064031830 con 0x7f50780ce1b0 2023-12-01T01:37:46.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.709+0000 7f50777fe640 1 -- 172.21.15.115:0/1979975488 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5064031990 con 0x7f50780ce1b0 2023-12-01T01:37:46.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.709+0000 7f50777fe640 1 -- 172.21.15.115:0/1979975488 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064031d20 con 0x7f50780ce1b0 2023-12-01T01:37:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.709+0000 7f507f238640 1 -- 172.21.15.115:0/1979975488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50780ce1b0 msgr2=0x7f50780cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:46.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.709+0000 7f507f238640 1 --2- 172.21.15.115:0/1979975488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50780ce1b0 0x7f50780cce30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f50640097a0 tx=0x7f506402b8d0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.710+0000 7f507f238640 1 -- 172.21.15.115:0/1979975488 shutdown_connections 2023-12-01T01:37:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.710+0000 7f507f238640 1 --2- 172.21.15.115:0/1979975488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50780b0e60 0x7f50780b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.710+0000 7f507f238640 1 --2- 172.21.15.115:0/1979975488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50780ce1b0 0x7f50780cce30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.710+0000 7f507f238640 1 -- 172.21.15.115:0/1979975488 >> 172.21.15.115:0/1979975488 conn(0x7f5078065960 msgr2=0x7f5078065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.710+0000 7f507f238640 1 -- 172.21.15.115:0/1979975488 shutdown_connections 2023-12-01T01:37:46.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.710+0000 7f507f238640 1 -- 172.21.15.115:0/1979975488 wait complete. 2023-12-01T01:37:46.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.711+0000 7f507f238640 1 Processor -- start 2023-12-01T01:37:46.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.711+0000 7f507f238640 1 -- start start 2023-12-01T01:37:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.712+0000 7f507f238640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50780b0e60 0x7f50781589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.712+0000 7f507f238640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50780ce1b0 0x7f50781491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.712+0000 7f507f238640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f507815ac80 0x7f50781531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.712+0000 7f507f238640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078149720 con 0x7f507815ac80 2023-12-01T01:37:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.712+0000 7f507f238640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078158fb0 con 0x7f50780ce1b0 2023-12-01T01:37:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.712+0000 7f507f238640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5078159290 con 0x7f50780b0e60 2023-12-01T01:37:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f507d7d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f507815ac80 0x7f50781531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f507d7d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f507815ac80 0x7f50781531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41108/0 (socket says 172.21.15.115:41108) 2023-12-01T01:37:46.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f507d7d5640 1 -- 172.21.15.115:0/80095782 learned_addr learned my addr 172.21.15.115:0/80095782 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f507cfd4640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50780b0e60 0x7f50781589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f507d7d5640 1 -- 172.21.15.115:0/80095782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50780b0e60 msgr2=0x7f50781589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f507d7d5640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50780b0e60 0x7f50781589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f5077fff640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50780ce1b0 0x7f50781491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f507d7d5640 1 -- 172.21.15.115:0/80095782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50780ce1b0 msgr2=0x7f50781491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f507d7d5640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50780ce1b0 0x7f50781491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f507d7d5640 1 -- 172.21.15.115:0/80095782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5064009400 con 0x7f507815ac80 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.713+0000 7f507d7d5640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f507815ac80 0x7f50781531c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5070007e40 tx=0x7f507000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.714+0000 7f50757fa640 1 -- 172.21.15.115:0/80095782 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070014620 con 0x7f507815ac80 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.714+0000 7f50757fa640 1 -- 172.21.15.115:0/80095782 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507000ee90 con 0x7f507815ac80 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.714+0000 7f50757fa640 1 -- 172.21.15.115:0/80095782 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50700149e0 con 0x7f507815ac80 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.714+0000 7f507f238640 1 -- 172.21.15.115:0/80095782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5078159570 con 0x7f507815ac80 2023-12-01T01:37:46.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.714+0000 7f507f238640 1 -- 172.21.15.115:0/80095782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5078159b30 con 0x7f507815ac80 2023-12-01T01:37:46.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.715+0000 7f507f238640 1 -- 172.21.15.115:0/80095782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5078159d20 con 0x7f507815ac80 2023-12-01T01:37:46.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.716+0000 7f50757fa640 1 -- 172.21.15.115:0/80095782 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f5070014b40 con 0x7f507815ac80 2023-12-01T01:37:46.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.717+0000 7f50757fa640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f505c06a630 0x7f505c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:46.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.717+0000 7f50757fa640 1 -- 172.21.15.115:0/80095782 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f5070089760 con 0x7f507815ac80 2023-12-01T01:37:46.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.717+0000 7f507cfd4640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f505c06a630 0x7f505c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:46.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.718+0000 7f507cfd4640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f505c06a630 0x7f505c06caf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f506c0011e0 tx=0x7f506c009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:46.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.721+0000 7f50757fa640 1 -- 172.21.15.115:0/80095782 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f50700549b0 con 0x7f507815ac80 2023-12-01T01:37:46.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:46.938+0000 7f507f238640 1 -- 172.21.15.115:0/80095782 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f50780cc350 con 0x7f505c06a630 2023-12-01T01:37:47.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.063+0000 7f50757fa640 1 -- 172.21.15.115:0/80095782 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f506c0012d0 con 0x7f505c06a630 2023-12-01T01:37:47.065 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-12-01T01:37:47.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 -- 172.21.15.115:0/80095782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f505c06a630 msgr2=0x7f505c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f505c06a630 0x7f505c06caf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f506c0011e0 tx=0x7f506c009040 comp rx=0 tx=0).stop 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 -- 172.21.15.115:0/80095782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f507815ac80 msgr2=0x7f50781531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f507815ac80 0x7f50781531c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5070007e40 tx=0x7f507000e4b0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 -- 172.21.15.115:0/80095782 shutdown_connections 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f505c06a630 0x7f505c06caf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f507815ac80 0x7f50781531c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50780ce1b0 0x7f50781491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 --2- 172.21.15.115:0/80095782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f50780b0e60 0x7f50781589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 -- 172.21.15.115:0/80095782 >> 172.21.15.115:0/80095782 conn(0x7f5078065960 msgr2=0x7f50780ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 -- 172.21.15.115:0/80095782 shutdown_connections 2023-12-01T01:37:47.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.066+0000 7f507f238640 1 -- 172.21.15.115:0/80095782 wait complete. 2023-12-01T01:37:47.090 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-12-01T01:37:47.090 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir4 1M 2023-12-01T01:37:47.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.308+0000 7fc066e2c640 1 Processor -- start 2023-12-01T01:37:47.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.309+0000 7fc066e2c640 1 -- start start 2023-12-01T01:37:47.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.310+0000 7fc066e2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0600b5fa0 0x7fc0600cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.310+0000 7fc066e2c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0600ce5e0 con 0x7fc0600cfd60 2023-12-01T01:37:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.310+0000 7fc066e2c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0600ce300 con 0x7fc06015cd90 2023-12-01T01:37:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.310+0000 7fc066e2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0600d0ec0 con 0x7fc0600b5fa0 2023-12-01T01:37:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.310+0000 7fc05ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc0600cfd60 0x7fc0600d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42712/0 (socket says 172.21.15.115:42712) 2023-12-01T01:37:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.310+0000 7fc05ffff640 1 -- 172.21.15.115:0/1173572370 learned_addr learned my addr 172.21.15.115:0/1173572370 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc0653c9640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0600b5fa0 0x7fc0600cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157636251 0 0) 0x7fc048001110 con 0x7fc0600cfd60 2023-12-01T01:37:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc04c003520 con 0x7fc0600cfd60 2023-12-01T01:37:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 650474582 0 0) 0x7fc04c003520 con 0x7fc0600cfd60 2023-12-01T01:37:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0600b5fa0 msgr2=0x7fc0600cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0600b5fa0 0x7fc0600cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 >> v1:172.21.15.142:6789/0 conn(0x7fc06015cd90 legacy=0x7fc0600cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0600d10a0 con 0x7fc0600cfd60 2023-12-01T01:37:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc048001800 con 0x7fc0600cfd60 2023-12-01T01:37:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 >> v1:172.21.15.115:6789/0 conn(0x7fc0600cfd60 legacy=0x7fc0600d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc04c0039a0 0x7fc04c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.311+0000 7fc05f7fe640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0600b5fa0 0x7fc04c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc05f7fe640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc04c004d00 0x7fc04c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0600d0ec0 con 0x7fc04c004d00 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c00ba30 con 0x7fc04c0039a0 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c00bbf0 con 0x7fc0600b5fa0 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc0653c9640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc04c004d00 0x7fc04c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc05ffff640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0600b5fa0 0x7fc04c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc064bc8640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc04c0039a0 0x7fc04c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc0653c9640 1 -- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0600b5fa0 msgr2=0x7fc04c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc0653c9640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0600b5fa0 0x7fc04c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc0653c9640 1 -- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc04c0039a0 msgr2=0x7fc04c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc0653c9640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc04c0039a0 0x7fc04c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.312+0000 7fc0653c9640 1 -- 172.21.15.115:0/1173572370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c003520 con 0x7fc04c004d00 2023-12-01T01:37:47.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.313+0000 7fc0653c9640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc04c004d00 0x7fc04c005180 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc058009a90 tx=0x7fc04c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cb82521086dc5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:47.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.314+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05802c680 con 0x7fc04c004d00 2023-12-01T01:37:47.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.314+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05802c7e0 con 0x7fc04c004d00 2023-12-01T01:37:47.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.314+0000 7fc05f7fe640 1 -- 172.21.15.115:0/1173572370 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05802cb70 con 0x7fc04c004d00 2023-12-01T01:37:47.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.314+0000 7fc066e2c640 1 -- 172.21.15.115:0/1173572370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc04c004d00 msgr2=0x7fc04c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.314+0000 7fc066e2c640 1 --2- 172.21.15.115:0/1173572370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc04c004d00 0x7fc04c005180 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc058009a90 tx=0x7fc04c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.314+0000 7fc0653c9640 1 -- 172.21.15.115:0/1173572370 reap_dead start 2023-12-01T01:37:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.314+0000 7fc066e2c640 1 -- 172.21.15.115:0/1173572370 shutdown_connections 2023-12-01T01:37:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.314+0000 7fc066e2c640 1 -- 172.21.15.115:0/1173572370 >> 172.21.15.115:0/1173572370 conn(0x7fc060065960 msgr2=0x7fc060065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.314+0000 7fc066e2c640 1 -- 172.21.15.115:0/1173572370 shutdown_connections 2023-12-01T01:37:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.314+0000 7fc066e2c640 1 -- 172.21.15.115:0/1173572370 wait complete. 2023-12-01T01:37:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.316+0000 7fc066e2c640 1 Processor -- start 2023-12-01T01:37:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.316+0000 7fc066e2c640 1 -- start start 2023-12-01T01:37:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.317+0000 7fc066e2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc04c004d00 0x7fc0600ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.317+0000 7fc066e2c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc06014dfe0 0x7fc06015acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.317+0000 7fc066e2c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06014e440 0x7fc06015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.317+0000 7fc066e2c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0600ac7a0 con 0x7fc06014dfe0 2023-12-01T01:37:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.317+0000 7fc066e2c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0600cb1c0 con 0x7fc06014e440 2023-12-01T01:37:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.317+0000 7fc066e2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0600cb4a0 con 0x7fc04c004d00 2023-12-01T01:37:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.317+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc06014dfe0 0x7fc06015acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.318+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc06014dfe0 0x7fc06015acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41122/0 (socket says 172.21.15.115:41122) 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.318+0000 7fc05ffff640 1 -- 172.21.15.115:0/2666805558 learned_addr learned my addr 172.21.15.115:0/2666805558 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.318+0000 7fc05ffff640 1 -- 172.21.15.115:0/2666805558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc04c004d00 msgr2=0x7fc0600ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.318+0000 7fc05ffff640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc04c004d00 0x7fc0600ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.318+0000 7fc05ffff640 1 -- 172.21.15.115:0/2666805558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06014e440 msgr2=0x7fc06015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.318+0000 7fc05ffff640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06014e440 0x7fc06015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.318+0000 7fc05ffff640 1 -- 172.21.15.115:0/2666805558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0600cb780 con 0x7fc06014dfe0 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.318+0000 7fc05ffff640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc06014dfe0 0x7fc06015acc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e830 tx=0x7fc04800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.318+0000 7fc05d7fa640 1 -- 172.21.15.115:0/2666805558 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800c8d0 con 0x7fc06014dfe0 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.319+0000 7fc05d7fa640 1 -- 172.21.15.115:0/2666805558 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04800ca30 con 0x7fc06014dfe0 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.319+0000 7fc05d7fa640 1 -- 172.21.15.115:0/2666805558 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800cde0 con 0x7fc06014dfe0 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.319+0000 7fc066e2c640 1 -- 172.21.15.115:0/2666805558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0600cba60 con 0x7fc06014dfe0 2023-12-01T01:37:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.319+0000 7fc066e2c640 1 -- 172.21.15.115:0/2666805558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0600abde0 con 0x7fc06014dfe0 2023-12-01T01:37:47.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.320+0000 7fc066e2c640 1 -- 172.21.15.115:0/2666805558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06004e890 con 0x7fc06014dfe0 2023-12-01T01:37:47.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.321+0000 7fc05d7fa640 1 -- 172.21.15.115:0/2666805558 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7fc04801d460 con 0x7fc06014dfe0 2023-12-01T01:37:47.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.321+0000 7fc05d7fa640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc04006a630 0x7fc04006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.322+0000 7fc05d7fa640 1 -- 172.21.15.115:0/2666805558 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc048088bf0 con 0x7fc06014dfe0 2023-12-01T01:37:47.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.322+0000 7fc064bc8640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc04006a630 0x7fc04006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.323+0000 7fc064bc8640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc04006a630 0x7fc04006caf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc050007fd0 tx=0x7fc050009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:47.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.326+0000 7fc05d7fa640 1 -- 172.21.15.115:0/2666805558 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc048053e60 con 0x7fc06014dfe0 2023-12-01T01:37:47.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.541+0000 7fc066e2c640 1 -- 172.21.15.115:0/2666805558 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0600d1660 con 0x7fc04006a630 2023-12-01T01:37:47.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.663+0000 7fc05d7fa640 1 -- 172.21.15.115:0/2666805558 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc050003970 con 0x7fc04006a630 2023-12-01T01:37:47.665 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-12-01T01:37:47.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.666+0000 7fc066e2c640 1 -- 172.21.15.115:0/2666805558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc04006a630 msgr2=0x7fc04006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.666+0000 7fc066e2c640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc04006a630 0x7fc04006caf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc050007fd0 tx=0x7fc050009040 comp rx=0 tx=0).stop 2023-12-01T01:37:47.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.666+0000 7fc066e2c640 1 -- 172.21.15.115:0/2666805558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc06014dfe0 msgr2=0x7fc06015acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.666+0000 7fc066e2c640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc06014dfe0 0x7fc06015acc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc04800e830 tx=0x7fc04800ecf0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.666+0000 7fc066e2c640 1 -- 172.21.15.115:0/2666805558 shutdown_connections 2023-12-01T01:37:47.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.666+0000 7fc066e2c640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc04006a630 0x7fc04006caf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.667+0000 7fc066e2c640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc06014e440 0x7fc06015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.667+0000 7fc066e2c640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc06014dfe0 0x7fc06015acc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.667+0000 7fc066e2c640 1 --2- 172.21.15.115:0/2666805558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc04c004d00 0x7fc0600ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.667+0000 7fc066e2c640 1 -- 172.21.15.115:0/2666805558 >> 172.21.15.115:0/2666805558 conn(0x7fc060065960 msgr2=0x7fc0600cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:47.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.667+0000 7fc066e2c640 1 -- 172.21.15.115:0/2666805558 shutdown_connections 2023-12-01T01:37:47.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.667+0000 7fc066e2c640 1 -- 172.21.15.115:0/2666805558 wait complete. 2023-12-01T01:37:47.690 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-12-01T01:37:47.690 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir5 1M 2023-12-01T01:37:47.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.887+0000 7fba0e241640 1 Processor -- start 2023-12-01T01:37:47.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.887+0000 7fba0e241640 1 -- start start 2023-12-01T01:37:47.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.888+0000 7fba0e241640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 0x7fba080cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.889+0000 7fba0e241640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fba080b29d0 0x7fba080cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.889+0000 7fba0e241640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba080b33e0 con 0x7fba080afae0 2023-12-01T01:37:47.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.889+0000 7fba0e241640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0815d490 con 0x7fba080b29d0 2023-12-01T01:37:47.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.889+0000 7fba0e241640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba08069df0 con 0x7fba0815d0b0 2023-12-01T01:37:47.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.889+0000 7fba077fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 0x7fba080cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.889+0000 7fba077fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 0x7fba080cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:45510/0 (socket says 172.21.15.115:45510) 2023-12-01T01:37:47.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.889+0000 7fba077fe640 1 -- 172.21.15.115:0/1411073334 learned_addr learned my addr 172.21.15.115:0/1411073334 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:47.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.889+0000 7fba06ffd640 1 --2- 172.21.15.115:0/1411073334 >> v2:172.21.15.142:3300/0 conn(0x7fba080b29d0 0x7fba080cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.890+0000 7fba077fe640 1 -- 172.21.15.115:0/1411073334 >> v2:172.21.15.142:3300/0 conn(0x7fba080b29d0 msgr2=0x7fba080cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:47.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.890+0000 7fba077fe640 1 --2- 172.21.15.115:0/1411073334 >> v2:172.21.15.142:3300/0 conn(0x7fba080b29d0 0x7fba080cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.890+0000 7fba077fe640 1 -- 172.21.15.115:0/1411073334 >> v1:172.21.15.142:6789/0 conn(0x7fba080afae0 legacy=0x7fba0806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.890+0000 7fba077fe640 1 -- 172.21.15.115:0/1411073334 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba080cce90 con 0x7fba0815d0b0 2023-12-01T01:37:47.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.892+0000 7fba077fe640 1 --2- 172.21.15.115:0/1411073334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 0x7fba080cb010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009870 tx=0x7fb9f802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb84eb7038e96d8f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:47.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.892+0000 7fba067fc640 1 -- 172.21.15.115:0/1411073334 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8030840 con 0x7fba0815d0b0 2023-12-01T01:37:47.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.892+0000 7fba067fc640 1 -- 172.21.15.115:0/1411073334 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f80309a0 con 0x7fba0815d0b0 2023-12-01T01:37:47.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.892+0000 7fba067fc640 1 -- 172.21.15.115:0/1411073334 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8030d30 con 0x7fba0815d0b0 2023-12-01T01:37:47.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.893+0000 7fba0e241640 1 -- 172.21.15.115:0/1411073334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 msgr2=0x7fba080cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.893+0000 7fba0e241640 1 --2- 172.21.15.115:0/1411073334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 0x7fba080cb010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009870 tx=0x7fb9f802b980 comp rx=0 tx=0).stop 2023-12-01T01:37:47.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.893+0000 7fba0e241640 1 -- 172.21.15.115:0/1411073334 shutdown_connections 2023-12-01T01:37:47.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.893+0000 7fba0e241640 1 --2- 172.21.15.115:0/1411073334 >> v2:172.21.15.142:3300/0 conn(0x7fba080b29d0 0x7fba080cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.893+0000 7fba0e241640 1 --2- 172.21.15.115:0/1411073334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 0x7fba080cb010 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.893+0000 7fba0e241640 1 -- 172.21.15.115:0/1411073334 >> 172.21.15.115:0/1411073334 conn(0x7fba08071430 msgr2=0x7fba08071840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:47.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.893+0000 7fba0e241640 1 -- 172.21.15.115:0/1411073334 shutdown_connections 2023-12-01T01:37:47.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.893+0000 7fba0e241640 1 -- 172.21.15.115:0/1411073334 wait complete. 2023-12-01T01:37:47.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.896+0000 7fba0e241640 1 Processor -- start 2023-12-01T01:37:47.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.896+0000 7fba0e241640 1 -- start start 2023-12-01T01:37:47.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.897+0000 7fba0e241640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba080afae0 0x7fba080b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.897+0000 7fba0e241640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba080b29d0 0x7fba080c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.897+0000 7fba0e241640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 0x7fba080b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.897+0000 7fba0e241640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba080c44f0 con 0x7fba080b29d0 2023-12-01T01:37:47.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.897+0000 7fba0e241640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba080c71b0 con 0x7fba080afae0 2023-12-01T01:37:47.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.897+0000 7fba0e241640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba080b5e70 con 0x7fba0815d0b0 2023-12-01T01:37:47.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.897+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba080b29d0 0x7fba080c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.897+0000 7fba06ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba080b29d0 0x7fba080c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34862/0 (socket says 172.21.15.115:34862) 2023-12-01T01:37:47.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.897+0000 7fba06ffd640 1 -- 172.21.15.115:0/314078116 learned_addr learned my addr 172.21.15.115:0/314078116 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:47.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fba077fe640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba080afae0 0x7fba080b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fba07fff640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 0x7fba080b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fba06ffd640 1 -- 172.21.15.115:0/314078116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 msgr2=0x7fba080b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fba06ffd640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 0x7fba080b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fba06ffd640 1 -- 172.21.15.115:0/314078116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba080afae0 msgr2=0x7fba080b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:47.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fba06ffd640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba080afae0 0x7fba080b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:47.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fba06ffd640 1 -- 172.21.15.115:0/314078116 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f80094d0 con 0x7fba080b29d0 2023-12-01T01:37:47.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fba06ffd640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba080b29d0 0x7fba080c6c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00e9e0 tx=0x7fb9fc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:47.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fb9effff640 1 -- 172.21.15.115:0/314078116 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc009670 con 0x7fba080b29d0 2023-12-01T01:37:47.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fb9effff640 1 -- 172.21.15.115:0/314078116 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00c9d0 con 0x7fba080b29d0 2023-12-01T01:37:47.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fb9effff640 1 -- 172.21.15.115:0/314078116 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00cd80 con 0x7fba080b29d0 2023-12-01T01:37:47.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.898+0000 7fba0e241640 1 -- 172.21.15.115:0/314078116 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba080b6150 con 0x7fba080b29d0 2023-12-01T01:37:47.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.899+0000 7fba0e241640 1 -- 172.21.15.115:0/314078116 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba080b58a0 con 0x7fba080b29d0 2023-12-01T01:37:47.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.900+0000 7fba0e241640 1 -- 172.21.15.115:0/314078116 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0804e890 con 0x7fba080b29d0 2023-12-01T01:37:47.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.901+0000 7fb9effff640 1 -- 172.21.15.115:0/314078116 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7fb9fc013470 con 0x7fba080b29d0 2023-12-01T01:37:47.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.902+0000 7fb9effff640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9f406a630 0x7fb9f406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:47.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.902+0000 7fb9effff640 1 -- 172.21.15.115:0/314078116 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb9fc0890c0 con 0x7fba080b29d0 2023-12-01T01:37:47.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.902+0000 7fba077fe640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9f406a630 0x7fb9f406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:47.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.903+0000 7fba077fe640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9f406a630 0x7fb9f406caf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8036000 tx=0x7fb9f803b040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:47.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:47.906+0000 7fb9effff640 1 -- 172.21.15.115:0/314078116 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb9fc0542b0 con 0x7fba080b29d0 2023-12-01T01:37:48.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.119+0000 7fba0e241640 1 -- 172.21.15.115:0/314078116 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fba080b5fe0 con 0x7fb9f406a630 2023-12-01T01:37:48.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.247+0000 7fb9effff640 1 -- 172.21.15.115:0/314078116 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fba080b5fe0 con 0x7fb9f406a630 2023-12-01T01:37:48.249 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 -- 172.21.15.115:0/314078116 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9f406a630 msgr2=0x7fb9f406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9f406a630 0x7fb9f406caf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8036000 tx=0x7fb9f803b040 comp rx=0 tx=0).stop 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 -- 172.21.15.115:0/314078116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba080b29d0 msgr2=0x7fba080c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba080b29d0 0x7fba080c6c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00e9e0 tx=0x7fb9fc00eea0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 -- 172.21.15.115:0/314078116 shutdown_connections 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9f406a630 0x7fb9f406caf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba0815d0b0 0x7fba080b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba080b29d0 0x7fba080c6c70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 --2- 172.21.15.115:0/314078116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba080afae0 0x7fba080b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 -- 172.21.15.115:0/314078116 >> 172.21.15.115:0/314078116 conn(0x7fba08071430 msgr2=0x7fba080caff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.250+0000 7fba0e241640 1 -- 172.21.15.115:0/314078116 shutdown_connections 2023-12-01T01:37:48.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.251+0000 7fba0e241640 1 -- 172.21.15.115:0/314078116 wait complete. 2023-12-01T01:37:48.272 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-12-01T01:37:48.272 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir6 1M 2023-12-01T01:37:48.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.477+0000 7ffab5fd9640 1 Processor -- start 2023-12-01T01:37:48.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.478+0000 7ffab5fd9640 1 -- start start 2023-12-01T01:37:48.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.479+0000 7ffab5fd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015ac80 0x7ffab00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:48.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.479+0000 7ffab5fd9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffab00ce1b0 0x7ffab00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:48.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.479+0000 7ffab5fd9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab00cc9a0 con 0x7ffab00b0e60 2023-12-01T01:37:48.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.479+0000 7ffab5fd9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab015b060 con 0x7ffab00ce1b0 2023-12-01T01:37:48.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.479+0000 7ffab5fd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00b0ce0 con 0x7ffab015ac80 2023-12-01T01:37:48.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.479+0000 7ffaa7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ffab00b0e60 0x7ffab00cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:33382/0 (socket says 172.21.15.115:33382) 2023-12-01T01:37:48.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.479+0000 7ffaa7fff640 1 -- 172.21.15.115:0/1583123876 learned_addr learned my addr 172.21.15.115:0/1583123876 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:37:48.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaf7fe640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015ac80 0x7ffab00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:48.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaffff640 1 --2- 172.21.15.115:0/1583123876 >> v2:172.21.15.142:3300/0 conn(0x7ffab00ce1b0 0x7ffab00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:48.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364286304 0 0) 0x7ffab00cc9a0 con 0x7ffab00b0e60 2023-12-01T01:37:48.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa9c003520 con 0x7ffab00b0e60 2023-12-01T01:37:48.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 324521477 0 0) 0x7ffa9c003520 con 0x7ffab00b0e60 2023-12-01T01:37:48.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015ac80 msgr2=0x7ffab00cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:48.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015ac80 0x7ffab00cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 >> v2:172.21.15.142:3300/0 conn(0x7ffab00ce1b0 msgr2=0x7ffab00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:48.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 --2- 172.21.15.115:0/1583123876 >> v2:172.21.15.142:3300/0 conn(0x7ffab00ce1b0 0x7ffab00b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab00cbd90 con 0x7ffab00b0e60 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7ffa98001670 con 0x7ffab00b0e60 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 >> v1:172.21.15.115:6789/0 conn(0x7ffab00b0e60 legacy=0x7ffab00cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.480+0000 7ffaaeffd640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015ac80 0x7ffa9c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaaeffd640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa9c003ef0 0x7ffa9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaaeffd640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa9c004d00 0x7ffa9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab015b060 con 0x7ffa9c003ef0 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00b0ce0 con 0x7ffa9c004d00 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c00b980 con 0x7ffab015ac80 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaa7fff640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa9c003ef0 0x7ffa9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaaf7fe640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015ac80 0x7ffa9c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaaffff640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa9c004d00 0x7ffa9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaa7fff640 1 -- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015ac80 msgr2=0x7ffa9c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaa7fff640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffab015ac80 0x7ffa9c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaa7fff640 1 -- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa9c004d00 msgr2=0x7ffa9c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:48.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaa7fff640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa9c004d00 0x7ffa9c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.481+0000 7ffaa7fff640 1 -- 172.21.15.115:0/1583123876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c003520 con 0x7ffa9c003ef0 2023-12-01T01:37:48.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.483+0000 7ffaa7fff640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa9c003ef0 0x7ffa9c0047c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800c240 tx=0x7ffa9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00f91885bfcb356 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:48.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.483+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9802cae0 con 0x7ffa9c003ef0 2023-12-01T01:37:48.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.483+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9802cc40 con 0x7ffa9c003ef0 2023-12-01T01:37:48.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.483+0000 7ffaaeffd640 1 -- 172.21.15.115:0/1583123876 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98005af0 con 0x7ffa9c003ef0 2023-12-01T01:37:48.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.483+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1583123876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa9c003ef0 msgr2=0x7ffa9c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:48.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.483+0000 7ffab5fd9640 1 --2- 172.21.15.115:0/1583123876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa9c003ef0 0x7ffa9c0047c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800c240 tx=0x7ffa9c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.483+0000 7ffaaffff640 1 -- 172.21.15.115:0/1583123876 reap_dead start 2023-12-01T01:37:48.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.484+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1583123876 shutdown_connections 2023-12-01T01:37:48.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.484+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1583123876 >> 172.21.15.115:0/1583123876 conn(0x7ffab0065960 msgr2=0x7ffab0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:48.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.484+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1583123876 shutdown_connections 2023-12-01T01:37:48.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.484+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1583123876 wait complete. 2023-12-01T01:37:48.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.485+0000 7ffab5fd9640 1 Processor -- start 2023-12-01T01:37:48.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.486+0000 7ffab5fd9640 1 -- start start 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.486+0000 7ffab5fd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa9c003ef0 0x7ffab0158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffab5fd9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab0150350 0x7ffab006f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffab5fd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffab006f760 0x7ffab015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffab5fd9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab006fbe0 con 0x7ffab006f760 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffab5fd9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0158a20 con 0x7ffab0150350 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffab5fd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab01593f0 con 0x7ffa9c003ef0 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffaaffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffab006f760 0x7ffab015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffaaffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffab006f760 0x7ffab015c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34876/0 (socket says 172.21.15.115:34876) 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffaaffff640 1 -- 172.21.15.115:0/1286733412 learned_addr learned my addr 172.21.15.115:0/1286733412 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffaaf7fe640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa9c003ef0 0x7ffab0158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffaa7fff640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab0150350 0x7ffab006f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffaaffff640 1 -- 172.21.15.115:0/1286733412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa9c003ef0 msgr2=0x7ffab0158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:48.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffaaffff640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa9c003ef0 0x7ffab0158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffaaffff640 1 -- 172.21.15.115:0/1286733412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab0150350 msgr2=0x7ffab006f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:48.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffaaffff640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab0150350 0x7ffab006f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.487+0000 7ffaaffff640 1 -- 172.21.15.115:0/1286733412 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab015d150 con 0x7ffab006f760 2023-12-01T01:37:48.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.488+0000 7ffaaffff640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffab006f760 0x7ffab015c490 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ffab00b3c90 tx=0x7ffaa80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:48.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.488+0000 7ffaacff9640 1 -- 172.21.15.115:0/1286733412 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800e070 con 0x7ffab006f760 2023-12-01T01:37:48.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.488+0000 7ffaacff9640 1 -- 172.21.15.115:0/1286733412 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa8011e40 con 0x7ffab006f760 2023-12-01T01:37:48.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.488+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1286733412 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab01583d0 con 0x7ffab006f760 2023-12-01T01:37:48.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.488+0000 7ffaacff9640 1 -- 172.21.15.115:0/1286733412 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8003990 con 0x7ffab006f760 2023-12-01T01:37:48.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.488+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1286733412 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab01507d0 con 0x7ffab006f760 2023-12-01T01:37:48.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.489+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1286733412 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab004e890 con 0x7ffab006f760 2023-12-01T01:37:48.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.490+0000 7ffaacff9640 1 -- 172.21.15.115:0/1286733412 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7ffaa8003b10 con 0x7ffab006f760 2023-12-01T01:37:48.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.491+0000 7ffaacff9640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa9006a630 0x7ffa9006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:48.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.491+0000 7ffaacff9640 1 -- 172.21.15.115:0/1286733412 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ffaa80895e0 con 0x7ffab006f760 2023-12-01T01:37:48.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.491+0000 7ffaaf7fe640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa9006a630 0x7ffa9006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:48.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.492+0000 7ffaaf7fe640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa9006a630 0x7ffa9006caf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009870 tx=0x7ffaa0008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:48.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.495+0000 7ffaacff9640 1 -- 172.21.15.115:0/1286733412 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ffaa80546a0 con 0x7ffab006f760 2023-12-01T01:37:48.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.706+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1286733412 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ffab0159e30 con 0x7ffa9006a630 2023-12-01T01:37:48.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.834+0000 7ffaacff9640 1 -- 172.21.15.115:0/1286733412 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ffab0159e30 con 0x7ffa9006a630 2023-12-01T01:37:48.836 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-12-01T01:37:48.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.837+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1286733412 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa9006a630 msgr2=0x7ffa9006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:48.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.837+0000 7ffab5fd9640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa9006a630 0x7ffa9006caf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009870 tx=0x7ffaa0008040 comp rx=0 tx=0).stop 2023-12-01T01:37:48.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.837+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1286733412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffab006f760 msgr2=0x7ffab015c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:48.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.837+0000 7ffab5fd9640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffab006f760 0x7ffab015c490 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ffab00b3c90 tx=0x7ffaa80115a0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.838+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1286733412 shutdown_connections 2023-12-01T01:37:48.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.838+0000 7ffab5fd9640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa9006a630 0x7ffa9006caf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.838+0000 7ffab5fd9640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffab006f760 0x7ffab015c490 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.838+0000 7ffab5fd9640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffab0150350 0x7ffab006f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.838+0000 7ffab5fd9640 1 --2- 172.21.15.115:0/1286733412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa9c003ef0 0x7ffab0158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:48.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.838+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1286733412 >> 172.21.15.115:0/1286733412 conn(0x7ffab0065960 msgr2=0x7ffab0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:48.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.838+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1286733412 shutdown_connections 2023-12-01T01:37:48.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:48.838+0000 7ffab5fd9640 1 -- 172.21.15.115:0/1286733412 wait complete. 2023-12-01T01:37:48.861 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-12-01T01:37:48.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir7 1M 2023-12-01T01:37:49.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.080+0000 7f240ccfb640 1 Processor -- start 2023-12-01T01:37:49.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.081+0000 7f240ccfb640 1 -- start start 2023-12-01T01:37:49.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.082+0000 7f240ccfb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f240815cfa0 0x7f24080b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.082+0000 7f240ccfb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 0x7f24080b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.082+0000 7f240ccfb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24080b6890 con 0x7f24080b6210 2023-12-01T01:37:49.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.082+0000 7f240ccfb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24080b6b70 con 0x7f240815cfa0 2023-12-01T01:37:49.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.082+0000 7f240ccfb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24080b6e50 con 0x7f24080b7e70 2023-12-01T01:37:49.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.083+0000 7f2405d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 0x7f24080b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.083+0000 7f2405d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 0x7f24080b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:45548/0 (socket says 172.21.15.115:45548) 2023-12-01T01:37:49.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.083+0000 7f2405d9b640 1 -- 172.21.15.115:0/1494821183 learned_addr learned my addr 172.21.15.115:0/1494821183 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:49.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.083+0000 7f240659c640 1 --2- 172.21.15.115:0/1494821183 >> v2:172.21.15.142:3300/0 conn(0x7f240815cfa0 0x7f24080b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.084+0000 7f2405d9b640 1 -- 172.21.15.115:0/1494821183 >> v2:172.21.15.142:3300/0 conn(0x7f240815cfa0 msgr2=0x7f24080b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:49.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.084+0000 7f2405d9b640 1 --2- 172.21.15.115:0/1494821183 >> v2:172.21.15.142:3300/0 conn(0x7f240815cfa0 0x7f24080b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.084+0000 7f2405d9b640 1 -- 172.21.15.115:0/1494821183 >> v1:172.21.15.142:6789/0 conn(0x7f24080b6210 legacy=0x7f24080cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:49.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.084+0000 7f2405d9b640 1 -- 172.21.15.115:0/1494821183 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24080b7130 con 0x7f24080b7e70 2023-12-01T01:37:49.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.084+0000 7f240659c640 1 --2- 172.21.15.115:0/1494821183 >> v2:172.21.15.142:3300/0 conn(0x7f240815cfa0 0x7f24080b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:37:49.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.085+0000 7f2405d9b640 1 --2- 172.21.15.115:0/1494821183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 0x7f24080b5cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009e50 tx=0x7f23f802be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7406c2cc7fa4b3df server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:49.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.085+0000 7f240559a640 1 -- 172.21.15.115:0/1494821183 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f802c9f0 con 0x7f24080b7e70 2023-12-01T01:37:49.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.085+0000 7f240559a640 1 -- 172.21.15.115:0/1494821183 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f802cb50 con 0x7f24080b7e70 2023-12-01T01:37:49.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.085+0000 7f240559a640 1 -- 172.21.15.115:0/1494821183 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80343f0 con 0x7f24080b7e70 2023-12-01T01:37:49.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.086+0000 7f240ccfb640 1 -- 172.21.15.115:0/1494821183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 msgr2=0x7f24080b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:49.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.086+0000 7f240ccfb640 1 --2- 172.21.15.115:0/1494821183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 0x7f24080b5cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009e50 tx=0x7f23f802be50 comp rx=0 tx=0).stop 2023-12-01T01:37:49.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.086+0000 7f240ccfb640 1 -- 172.21.15.115:0/1494821183 shutdown_connections 2023-12-01T01:37:49.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.086+0000 7f240ccfb640 1 --2- 172.21.15.115:0/1494821183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 0x7f24080b5cd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.086+0000 7f240ccfb640 1 --2- 172.21.15.115:0/1494821183 >> v2:172.21.15.142:3300/0 conn(0x7f240815cfa0 0x7f24080b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.086+0000 7f240ccfb640 1 -- 172.21.15.115:0/1494821183 >> 172.21.15.115:0/1494821183 conn(0x7f240806a0b0 msgr2=0x7f240806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:49.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.086+0000 7f240ccfb640 1 -- 172.21.15.115:0/1494821183 shutdown_connections 2023-12-01T01:37:49.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.086+0000 7f240ccfb640 1 -- 172.21.15.115:0/1494821183 wait complete. 2023-12-01T01:37:49.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.088+0000 7f240ccfb640 1 Processor -- start 2023-12-01T01:37:49.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.089+0000 7f240ccfb640 1 -- start start 2023-12-01T01:37:49.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.090+0000 7f240ccfb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24080b6210 0x7f240814b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.090+0000 7f240ccfb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 0x7f240815f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.090+0000 7f240ccfb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f240815cfa0 0x7f24080d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.090+0000 7f240ccfb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240815eaa0 con 0x7f240815cfa0 2023-12-01T01:37:49.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.090+0000 7f240ccfb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240815ed80 con 0x7f24080b6210 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.090+0000 7f240ccfb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240815fae0 con 0x7f24080b7e70 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.090+0000 7f2406d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f240815cfa0 0x7f24080d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.090+0000 7f2406d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f240815cfa0 0x7f24080d21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34882/0 (socket says 172.21.15.115:34882) 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.090+0000 7f2406d9d640 1 -- 172.21.15.115:0/3826400411 learned_addr learned my addr 172.21.15.115:0/3826400411 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.090+0000 7f240659c640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24080b6210 0x7f240814b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f2405d9b640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 0x7f240815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f2406d9d640 1 -- 172.21.15.115:0/3826400411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 msgr2=0x7f240815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f2406d9d640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 0x7f240815f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f2406d9d640 1 -- 172.21.15.115:0/3826400411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24080b6210 msgr2=0x7f240814b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f2406d9d640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24080b6210 0x7f240814b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f2406d9d640 1 -- 172.21.15.115:0/3826400411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f0009400 con 0x7f240815cfa0 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f240659c640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24080b6210 0x7f240814b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:37:49.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f2406d9d640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f240815cfa0 0x7f24080d21b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f240000b3e0 tx=0x7f240000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:49.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f23feffd640 1 -- 172.21.15.115:0/3826400411 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400011830 con 0x7f240815cfa0 2023-12-01T01:37:49.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f23feffd640 1 -- 172.21.15.115:0/3826400411 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2400011990 con 0x7f240815cfa0 2023-12-01T01:37:49.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f240ccfb640 1 -- 172.21.15.115:0/3826400411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f8009b30 con 0x7f240815cfa0 2023-12-01T01:37:49.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.091+0000 7f23feffd640 1 -- 172.21.15.115:0/3826400411 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400011d80 con 0x7f240815cfa0 2023-12-01T01:37:49.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.092+0000 7f240ccfb640 1 -- 172.21.15.115:0/3826400411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24080b8c00 con 0x7f240815cfa0 2023-12-01T01:37:49.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.093+0000 7f240ccfb640 1 -- 172.21.15.115:0/3826400411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240815eef0 con 0x7f240815cfa0 2023-12-01T01:37:49.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.094+0000 7f23feffd640 1 -- 172.21.15.115:0/3826400411 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f240001a400 con 0x7f240815cfa0 2023-12-01T01:37:49.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.095+0000 7f23feffd640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f23e806a660 0x7f23e806cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.095+0000 7f23feffd640 1 -- 172.21.15.115:0/3826400411 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f2400088980 con 0x7f240815cfa0 2023-12-01T01:37:49.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.095+0000 7f240659c640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f23e806a660 0x7f23e806cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.096+0000 7f240659c640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f23e806a660 0x7f23e806cb20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009d20 tx=0x7f23f0008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:49.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.099+0000 7f23feffd640 1 -- 172.21.15.115:0/3826400411 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2400053c80 con 0x7f240815cfa0 2023-12-01T01:37:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.314+0000 7f240ccfb640 1 -- 172.21.15.115:0/3826400411 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f24080b8a80 con 0x7f23e806a660 2023-12-01T01:37:49.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.446+0000 7f23feffd640 1 -- 172.21.15.115:0/3826400411 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f23f0011e90 con 0x7f23e806a660 2023-12-01T01:37:49.448 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-12-01T01:37:49.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.449+0000 7f240ccfb640 1 -- 172.21.15.115:0/3826400411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f23e806a660 msgr2=0x7f23e806cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:49.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.449+0000 7f240ccfb640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f23e806a660 0x7f23e806cb20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009d20 tx=0x7f23f0008040 comp rx=0 tx=0).stop 2023-12-01T01:37:49.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.450+0000 7f240ccfb640 1 -- 172.21.15.115:0/3826400411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f240815cfa0 msgr2=0x7f24080d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:49.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.450+0000 7f240ccfb640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f240815cfa0 0x7f24080d21b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f240000b3e0 tx=0x7f240000b8a0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.450+0000 7f240ccfb640 1 -- 172.21.15.115:0/3826400411 shutdown_connections 2023-12-01T01:37:49.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.450+0000 7f240ccfb640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f23e806a660 0x7f23e806cb20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.450+0000 7f240ccfb640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24080b7e70 0x7f240815f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.450+0000 7f240ccfb640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f240815cfa0 0x7f24080d21b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.450+0000 7f240ccfb640 1 --2- 172.21.15.115:0/3826400411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24080b6210 0x7f240814b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.450+0000 7f240ccfb640 1 -- 172.21.15.115:0/3826400411 >> 172.21.15.115:0/3826400411 conn(0x7f240806a0b0 msgr2=0x7f24080cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:49.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.450+0000 7f240ccfb640 1 -- 172.21.15.115:0/3826400411 shutdown_connections 2023-12-01T01:37:49.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.450+0000 7f240ccfb640 1 -- 172.21.15.115:0/3826400411 wait complete. 2023-12-01T01:37:49.471 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-12-01T01:37:49.471 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir8 1M 2023-12-01T01:37:49.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.680+0000 7f41256e9640 1 Processor -- start 2023-12-01T01:37:49.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.680+0000 7f41256e9640 1 -- start start 2023-12-01T01:37:49.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.682+0000 7f41256e9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f412015ac80 0x7f41200cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.682+0000 7f41256e9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f41200ce1b0 0x7f41200b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.682+0000 7f41256e9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41200cc9a0 con 0x7f41200b0e60 2023-12-01T01:37:49.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.682+0000 7f41256e9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412015b060 con 0x7f41200ce1b0 2023-12-01T01:37:49.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.682+0000 7f41256e9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41200b0ce0 con 0x7f412015ac80 2023-12-01T01:37:49.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.683+0000 7f411f7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f41200ce1b0 0x7f41200b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.683+0000 7f411f7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f41200ce1b0 0x7f41200b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:52776/0 (socket says 172.21.15.115:52776) 2023-12-01T01:37:49.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.683+0000 7f411f7fe640 1 -- 172.21.15.115:0/2204934425 learned_addr learned my addr 172.21.15.115:0/2204934425 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:49.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.683+0000 7f411effd640 1 --2- 172.21.15.115:0/2204934425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f412015ac80 0x7f41200cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.683+0000 7f411effd640 1 -- 172.21.15.115:0/2204934425 >> v2:172.21.15.142:3300/0 conn(0x7f41200ce1b0 msgr2=0x7f41200b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:49.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.683+0000 7f411effd640 1 --2- 172.21.15.115:0/2204934425 >> v2:172.21.15.142:3300/0 conn(0x7f41200ce1b0 0x7f41200b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.683+0000 7f411effd640 1 -- 172.21.15.115:0/2204934425 >> v1:172.21.15.142:6789/0 conn(0x7f41200b0e60 legacy=0x7f41200cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:49.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.683+0000 7f411effd640 1 -- 172.21.15.115:0/2204934425 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41200cbd90 con 0x7f412015ac80 2023-12-01T01:37:49.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.685+0000 7f411effd640 1 --2- 172.21.15.115:0/2204934425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f412015ac80 0x7f41200cce30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4110009d90 tx=0x7f411002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523a8aec3a0488cf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:49.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.686+0000 7f411e7fc640 1 -- 172.21.15.115:0/2204934425 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411002cc50 con 0x7f412015ac80 2023-12-01T01:37:49.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.686+0000 7f411e7fc640 1 -- 172.21.15.115:0/2204934425 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411002cdb0 con 0x7f412015ac80 2023-12-01T01:37:49.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.686+0000 7f411e7fc640 1 -- 172.21.15.115:0/2204934425 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110034660 con 0x7f412015ac80 2023-12-01T01:37:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.686+0000 7f41256e9640 1 -- 172.21.15.115:0/2204934425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f412015ac80 msgr2=0x7f41200cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.686+0000 7f41256e9640 1 --2- 172.21.15.115:0/2204934425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f412015ac80 0x7f41200cce30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4110009d90 tx=0x7f411002bea0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.686+0000 7f41256e9640 1 -- 172.21.15.115:0/2204934425 shutdown_connections 2023-12-01T01:37:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.686+0000 7f41256e9640 1 --2- 172.21.15.115:0/2204934425 >> v2:172.21.15.142:3300/0 conn(0x7f41200ce1b0 0x7f41200b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.686+0000 7f41256e9640 1 --2- 172.21.15.115:0/2204934425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f412015ac80 0x7f41200cce30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.686+0000 7f41256e9640 1 -- 172.21.15.115:0/2204934425 >> 172.21.15.115:0/2204934425 conn(0x7f4120065960 msgr2=0x7f4120065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.686+0000 7f41256e9640 1 -- 172.21.15.115:0/2204934425 shutdown_connections 2023-12-01T01:37:49.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.687+0000 7f41256e9640 1 -- 172.21.15.115:0/2204934425 wait complete. 2023-12-01T01:37:49.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.689+0000 7f41256e9640 1 Processor -- start 2023-12-01T01:37:49.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.689+0000 7f41256e9640 1 -- start start 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.690+0000 7f41256e9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41200b0e60 0x7f41201f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.690+0000 7f41256e9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41200ce1b0 0x7f4120159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.690+0000 7f41256e9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f412015ac80 0x7f412015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.690+0000 7f41256e9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412015ccf0 con 0x7f412015ac80 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.690+0000 7f41256e9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41201f2bc0 con 0x7f41200b0e60 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.690+0000 7f41256e9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412015b500 con 0x7f41200ce1b0 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.691+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f412015ac80 0x7f412015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.691+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f412015ac80 0x7f412015c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34888/0 (socket says 172.21.15.115:34888) 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.691+0000 7f411f7fe640 1 -- 172.21.15.115:0/616253719 learned_addr learned my addr 172.21.15.115:0/616253719 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.691+0000 7f411effd640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41200b0e60 0x7f41201f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.691+0000 7f41167fc640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41200ce1b0 0x7f4120159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.691+0000 7f411f7fe640 1 -- 172.21.15.115:0/616253719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41200ce1b0 msgr2=0x7f4120159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:49.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.691+0000 7f411f7fe640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41200ce1b0 0x7f4120159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.691+0000 7f411f7fe640 1 -- 172.21.15.115:0/616253719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41200b0e60 msgr2=0x7f41201f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:49.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.691+0000 7f411f7fe640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41200b0e60 0x7f41201f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:49.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.691+0000 7f411f7fe640 1 -- 172.21.15.115:0/616253719 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4110009a70 con 0x7f412015ac80 2023-12-01T01:37:49.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.692+0000 7f411f7fe640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f412015ac80 0x7f412015c0e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f411800e840 tx=0x7f411800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.692+0000 7f4117fff640 1 -- 172.21.15.115:0/616253719 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118009660 con 0x7f412015ac80 2023-12-01T01:37:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.692+0000 7f4117fff640 1 -- 172.21.15.115:0/616253719 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411800c940 con 0x7f412015ac80 2023-12-01T01:37:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.692+0000 7f4117fff640 1 -- 172.21.15.115:0/616253719 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411800cd10 con 0x7f412015ac80 2023-12-01T01:37:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.692+0000 7f41256e9640 1 -- 172.21.15.115:0/616253719 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4120157450 con 0x7f412015ac80 2023-12-01T01:37:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.692+0000 7f41256e9640 1 -- 172.21.15.115:0/616253719 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4120157a10 con 0x7f412015ac80 2023-12-01T01:37:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.693+0000 7f41256e9640 1 -- 172.21.15.115:0/616253719 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412004e890 con 0x7f412015ac80 2023-12-01T01:37:49.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.694+0000 7f4117fff640 1 -- 172.21.15.115:0/616253719 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f4118013470 con 0x7f412015ac80 2023-12-01T01:37:49.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.695+0000 7f4117fff640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f410006a630 0x7f410006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:49.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.695+0000 7f4117fff640 1 -- 172.21.15.115:0/616253719 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4118088f40 con 0x7f412015ac80 2023-12-01T01:37:49.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.696+0000 7f411effd640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f410006a630 0x7f410006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:49.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.696+0000 7f411effd640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f410006a630 0x7f410006caf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4110036000 tx=0x7f4110030000 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:49.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.700+0000 7f4117fff640 1 -- 172.21.15.115:0/616253719 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4118054240 con 0x7f412015ac80 2023-12-01T01:37:49.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:49.902+0000 7f41256e9640 1 -- 172.21.15.115:0/616253719 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f41200cc350 con 0x7f410006a630 2023-12-01T01:37:50.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.022+0000 7f4117fff640 1 -- 172.21.15.115:0/616253719 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4110009450 con 0x7f410006a630 2023-12-01T01:37:50.024 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-12-01T01:37:50.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 -- 172.21.15.115:0/616253719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f410006a630 msgr2=0x7f410006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:50.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f410006a630 0x7f410006caf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4110036000 tx=0x7f4110030000 comp rx=0 tx=0).stop 2023-12-01T01:37:50.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 -- 172.21.15.115:0/616253719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f412015ac80 msgr2=0x7f412015c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:50.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f412015ac80 0x7f412015c0e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f411800e840 tx=0x7f411800ed00 comp rx=0 tx=0).stop 2023-12-01T01:37:50.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 -- 172.21.15.115:0/616253719 shutdown_connections 2023-12-01T01:37:50.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f410006a630 0x7f410006caf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41200ce1b0 0x7f4120159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f412015ac80 0x7f412015c0e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 --2- 172.21.15.115:0/616253719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41200b0e60 0x7f41201f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 -- 172.21.15.115:0/616253719 >> 172.21.15.115:0/616253719 conn(0x7f4120065960 msgr2=0x7f41200ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:50.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 -- 172.21.15.115:0/616253719 shutdown_connections 2023-12-01T01:37:50.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.025+0000 7f41256e9640 1 -- 172.21.15.115:0/616253719 wait complete. 2023-12-01T01:37:50.045 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-12-01T01:37:50.045 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir9 1M 2023-12-01T01:37:50.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.248+0000 7f5831834640 1 Processor -- start 2023-12-01T01:37:50.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.249+0000 7f5831834640 1 -- start start 2023-12-01T01:37:50.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.250+0000 7f5831834640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f582c153c30 0x7f582c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:50.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.250+0000 7f5831834640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f582c154010 0x7f582c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:50.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.250+0000 7f5831834640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582c0cdcd0 con 0x7f582c0cf9b0 2023-12-01T01:37:50.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.250+0000 7f5831834640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c0cfe20 con 0x7f582c154010 2023-12-01T01:37:50.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.250+0000 7f5831834640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c161830 con 0x7f582c153c30 2023-12-01T01:37:50.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.250+0000 7f582bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f582c154010 0x7f582c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:50.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.250+0000 7f582bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f582c154010 0x7f582c0cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34894/0 (socket says 172.21.15.115:34894) 2023-12-01T01:37:50.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.250+0000 7f582bfff640 1 -- 172.21.15.115:0/4048396581 learned_addr learned my addr 172.21.15.115:0/4048396581 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.251+0000 7f5830832640 1 --2- 172.21.15.115:0/4048396581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f582c153c30 0x7f582c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.251+0000 7f582bfff640 1 -- 172.21.15.115:0/4048396581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f582c153c30 msgr2=0x7f582c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.251+0000 7f582bfff640 1 --2- 172.21.15.115:0/4048396581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f582c153c30 0x7f582c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.251+0000 7f582bfff640 1 -- 172.21.15.115:0/4048396581 >> v1:172.21.15.142:6789/0 conn(0x7f582c0cf9b0 legacy=0x7f582c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:37:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.251+0000 7f582bfff640 1 -- 172.21.15.115:0/4048396581 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0cd3a0 con 0x7f582c154010 2023-12-01T01:37:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.252+0000 7f582bfff640 1 --2- 172.21.15.115:0/4048396581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c154010 0x7f582c0cf470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5814009870 tx=0x7f581402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6517829a40e29d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:50.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.252+0000 7f582b7fe640 1 -- 172.21.15.115:0/4048396581 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814031640 con 0x7f582c154010 2023-12-01T01:37:50.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.252+0000 7f582b7fe640 1 -- 172.21.15.115:0/4048396581 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58140317a0 con 0x7f582c154010 2023-12-01T01:37:50.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.252+0000 7f582b7fe640 1 -- 172.21.15.115:0/4048396581 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814031b30 con 0x7f582c154010 2023-12-01T01:37:50.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.253+0000 7f5831834640 1 -- 172.21.15.115:0/4048396581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c154010 msgr2=0x7f582c0cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:50.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.253+0000 7f5831834640 1 --2- 172.21.15.115:0/4048396581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c154010 0x7f582c0cf470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5814009870 tx=0x7f581402b8f0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.253+0000 7f5831834640 1 -- 172.21.15.115:0/4048396581 shutdown_connections 2023-12-01T01:37:50.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.253+0000 7f5831834640 1 --2- 172.21.15.115:0/4048396581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c154010 0x7f582c0cf470 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.253+0000 7f5831834640 1 --2- 172.21.15.115:0/4048396581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f582c153c30 0x7f582c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.253+0000 7f5831834640 1 -- 172.21.15.115:0/4048396581 >> 172.21.15.115:0/4048396581 conn(0x7f582c06a0b0 msgr2=0x7f582c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:50.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.253+0000 7f5831834640 1 -- 172.21.15.115:0/4048396581 shutdown_connections 2023-12-01T01:37:50.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.253+0000 7f5831834640 1 -- 172.21.15.115:0/4048396581 wait complete. 2023-12-01T01:37:50.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.255+0000 7f5831834640 1 Processor -- start 2023-12-01T01:37:50.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.255+0000 7f5831834640 1 -- start start 2023-12-01T01:37:50.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f5831834640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f582c0cf9b0 0x7f582c1500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:50.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f5831834640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c153c30 0x7f582c0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:50.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f5831834640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f582c154010 0x7f582c0ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:50.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f5831834640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c15a090 con 0x7f582c153c30 2023-12-01T01:37:50.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f5831834640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c159480 con 0x7f582c154010 2023-12-01T01:37:50.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f5831834640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c159760 con 0x7f582c0cf9b0 2023-12-01T01:37:50.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f582bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c153c30 0x7f582c0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:50.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f582bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c153c30 0x7f582c0ca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34902/0 (socket says 172.21.15.115:34902) 2023-12-01T01:37:50.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f582bfff640 1 -- 172.21.15.115:0/3635603857 learned_addr learned my addr 172.21.15.115:0/3635603857 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:37:50.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f5830832640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f582c0cf9b0 0x7f582c1500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f5831033640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f582c154010 0x7f582c0ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.256+0000 7f582bfff640 1 -- 172.21.15.115:0/3635603857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f582c0cf9b0 msgr2=0x7f582c1500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.257+0000 7f582bfff640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f582c0cf9b0 0x7f582c1500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.257+0000 7f582bfff640 1 -- 172.21.15.115:0/3635603857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f582c154010 msgr2=0x7f582c0ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.257+0000 7f582bfff640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f582c154010 0x7f582c0ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.257+0000 7f582bfff640 1 -- 172.21.15.115:0/3635603857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140094d0 con 0x7f582c153c30 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.257+0000 7f582bfff640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c153c30 0x7f582c0ca400 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f58140090b0 tx=0x7f58140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.257+0000 7f58297fa640 1 -- 172.21.15.115:0/3635603857 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581403ac60 con 0x7f582c153c30 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.257+0000 7f58297fa640 1 -- 172.21.15.115:0/3635603857 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581403ade0 con 0x7f582c153c30 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.257+0000 7f58297fa640 1 -- 172.21.15.115:0/3635603857 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814044630 con 0x7f582c153c30 2023-12-01T01:37:50.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.257+0000 7f5831834640 1 -- 172.21.15.115:0/3635603857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c159a40 con 0x7f582c153c30 2023-12-01T01:37:50.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.257+0000 7f5831834640 1 -- 172.21.15.115:0/3635603857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c1505f0 con 0x7f582c153c30 2023-12-01T01:37:50.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.258+0000 7f5831834640 1 -- 172.21.15.115:0/3635603857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c1507e0 con 0x7f582c153c30 2023-12-01T01:37:50.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.259+0000 7f58297fa640 1 -- 172.21.15.115:0/3635603857 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 11) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f581402f050 con 0x7f582c153c30 2023-12-01T01:37:50.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.260+0000 7f58297fa640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f580c06a630 0x7f580c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:37:50.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.260+0000 7f58297fa640 1 -- 172.21.15.115:0/3635603857 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f58140aace0 con 0x7f582c153c30 2023-12-01T01:37:50.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.260+0000 7f5830832640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f580c06a630 0x7f580c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:37:50.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.261+0000 7f5830832640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f580c06a630 0x7f580c06caf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f581c009a60 tx=0x7f581c003990 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:37:50.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.265+0000 7f58297fa640 1 -- 172.21.15.115:0/3635603857 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5814075f60 con 0x7f582c153c30 2023-12-01T01:37:50.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.477+0000 7f5831834640 1 -- 172.21.15.115:0/3635603857 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f582c0cd680 con 0x7f580c06a630 2023-12-01T01:37:50.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.605+0000 7f58297fa640 1 -- 172.21.15.115:0/3635603857 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f581c0096f0 con 0x7f580c06a630 2023-12-01T01:37:50.607 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-12-01T01:37:50.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 -- 172.21.15.115:0/3635603857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f580c06a630 msgr2=0x7f580c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f580c06a630 0x7f580c06caf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f581c009a60 tx=0x7f581c003990 comp rx=0 tx=0).stop 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 -- 172.21.15.115:0/3635603857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c153c30 msgr2=0x7f582c0ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c153c30 0x7f582c0ca400 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f58140090b0 tx=0x7f58140046c0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 -- 172.21.15.115:0/3635603857 shutdown_connections 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f580c06a630 0x7f580c06caf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f582c154010 0x7f582c0ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f582c153c30 0x7f582c0ca400 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 --2- 172.21.15.115:0/3635603857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f582c0cf9b0 0x7f582c1500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 -- 172.21.15.115:0/3635603857 >> 172.21.15.115:0/3635603857 conn(0x7f582c06a0b0 msgr2=0x7f582c067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 -- 172.21.15.115:0/3635603857 shutdown_connections 2023-12-01T01:37:50.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:37:50.608+0000 7f5831834640 1 -- 172.21.15.115:0/3635603857 wait complete. 2023-12-01T01:37:50.629 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-12-01T01:37:54.562 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:37:54.572 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:38:00.050 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:38:00.048+0000 7f9c9d06d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:38:00.050 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:38:00.048+0000 7f9c9d06d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:38:00.051 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:38:00.049+0000 7f9c9d06d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:38:00.051 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:38:00.049+0000 7f9c9d06d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:38:00.051 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:38:00.049+0000 7f9c9d06d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:38:24.727 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:38:24.737 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:38:54.888 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:38:54.898 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:39:25.058 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:39:25.068 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:39:55.201 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:39:55.211 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:40:25.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:40:25.365 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:40:55.509 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:40:55.520 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:41:25.664 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:41:25.674 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:41:55.825 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:41:55.835 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:42:26.007 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:42:26.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:42:56.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:42:56.365 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:43:01.242 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:43:01.240+0000 7f9ca48fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:43:01.242 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:43:01.240+0000 7f9ca48fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:43:01.242 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:43:01.240+0000 7f9ca48fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:43:01.242 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:43:01.240+0000 7f9ca48fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:43:01.242 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:43:01.240+0000 7f9ca48fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:43:26.524 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:43:26.534 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:43:56.678 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:43:56.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:44:26.826 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:44:26.836 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:44:56.973 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:44:56.983 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:45:27.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:45:27.158 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:45:57.316 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:45:57.326 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:46:27.480 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:46:27.489 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:46:57.634 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:46:57.644 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:47:27.780 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:47:27.790 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:47:57.920 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:47:57.930 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:48:21.630 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir0 1M 2023-12-01T01:48:21.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.861+0000 7fd66fda9640 1 Processor -- start 2023-12-01T01:48:21.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.862+0000 7fd66fda9640 1 -- start start 2023-12-01T01:48:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.863+0000 7fd66fda9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd668142870 0x7fd6680ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.863+0000 7fd66fda9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6680cd010 0x7fd6680b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.863+0000 7fd66fda9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6680cd470 con 0x7fd6680b1c50 2023-12-01T01:48:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.863+0000 7fd66fda9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6680b3ae0 con 0x7fd668142870 2023-12-01T01:48:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.863+0000 7fd66fda9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd668142c50 con 0x7fd6680cd010 2023-12-01T01:48:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66e346640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd6680b1c50 0x7fd66815d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56136/0 (socket says 172.21.15.115:56136) 2023-12-01T01:48:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66e346640 1 -- 172.21.15.115:0/1593384823 learned_addr learned my addr 172.21.15.115:0/1593384823 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66db45640 1 --2- 172.21.15.115:0/1593384823 >> v2:172.21.15.142:3300/0 conn(0x7fd668142870 0x7fd6680ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66d344640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6680cd010 0x7fd6680b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035034258 0 0) 0x7fd6680cd470 con 0x7fd6680b1c50 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd650003520 con 0x7fd6680b1c50 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246042804 0 0) 0x7fd650003520 con 0x7fd6680b1c50 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6680cd010 msgr2=0x7fd6680b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66cb43640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6680cd010 0x7fd6680b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 >> v2:172.21.15.142:3300/0 conn(0x7fd668142870 msgr2=0x7fd6680ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.864+0000 7fd66cb43640 1 --2- 172.21.15.115:0/1593384823 >> v2:172.21.15.142:3300/0 conn(0x7fd668142870 0x7fd6680ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680b20e0 con 0x7fd6680b1c50 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd6600019f0 con 0x7fd6680b1c50 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 >> v1:172.21.15.115:6789/0 conn(0x7fd6680b1c50 legacy=0x7fd66815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66cb43640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6500039a0 0x7fd650003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66cb43640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500047c0 0x7fd650004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66cb43640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6680cd010 0x7fd650005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680b3ae0 con 0x7fd6500047c0 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668142c50 con 0x7fd6500039a0 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65000db90 con 0x7fd6680cd010 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66d344640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500047c0 0x7fd650004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66db45640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6500039a0 0x7fd650003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66e346640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6680cd010 0x7fd650005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.865+0000 7fd66d344640 1 -- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6680cd010 msgr2=0x7fd650005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.866+0000 7fd66d344640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6680cd010 0x7fd650005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.866+0000 7fd66d344640 1 -- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6500039a0 msgr2=0x7fd650003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.866+0000 7fd66d344640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6500039a0 0x7fd650003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:21.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.866+0000 7fd66d344640 1 -- 172.21.15.115:0/1593384823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650003520 con 0x7fd6500047c0 2023-12-01T01:48:21.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.867+0000 7fd66d344640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500047c0 0x7fd650004c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009ac0 tx=0x7fd6500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61050aa01fe69e03 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:21.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.867+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c02c6b0 con 0x7fd6500047c0 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.867+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c02c810 con 0x7fd6500047c0 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.867+0000 7fd66cb43640 1 -- 172.21.15.115:0/1593384823 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c02cba0 con 0x7fd6500047c0 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.867+0000 7fd66fda9640 1 -- 172.21.15.115:0/1593384823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500047c0 msgr2=0x7fd650004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.867+0000 7fd66fda9640 1 --2- 172.21.15.115:0/1593384823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500047c0 0x7fd650004c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009ac0 tx=0x7fd6500057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.867+0000 7fd66e346640 1 -- 172.21.15.115:0/1593384823 reap_dead start 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.868+0000 7fd66fda9640 1 -- 172.21.15.115:0/1593384823 shutdown_connections 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.868+0000 7fd66fda9640 1 -- 172.21.15.115:0/1593384823 >> 172.21.15.115:0/1593384823 conn(0x7fd668065960 msgr2=0x7fd668065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.868+0000 7fd66fda9640 1 -- 172.21.15.115:0/1593384823 shutdown_connections 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.868+0000 7fd66fda9640 1 -- 172.21.15.115:0/1593384823 wait complete. 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.869+0000 7fd66fda9640 1 Processor -- start 2023-12-01T01:48:21.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.870+0000 7fd66fda9640 1 -- start start 2023-12-01T01:48:21.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66fda9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6500047c0 0x7fd6681f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:21.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66fda9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6681df950 0x7fd6680adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:21.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66fda9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6681dfdd0 0x7fd66813d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:21.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66fda9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680cc5a0 con 0x7fd6681df950 2023-12-01T01:48:21.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66fda9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681f3620 con 0x7fd6681dfdd0 2023-12-01T01:48:21.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66fda9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680acba0 con 0x7fd6500047c0 2023-12-01T01:48:21.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66d344640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6681df950 0x7fd6680adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:21.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66d344640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6681df950 0x7fd6680adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47176/0 (socket says 172.21.15.115:47176) 2023-12-01T01:48:21.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66d344640 1 -- 172.21.15.115:0/2163670888 learned_addr learned my addr 172.21.15.115:0/2163670888 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66db45640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6500047c0 0x7fd6681f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66e346640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6681dfdd0 0x7fd66813d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66d344640 1 -- 172.21.15.115:0/2163670888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6500047c0 msgr2=0x7fd6681f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66d344640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6500047c0 0x7fd6681f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66d344640 1 -- 172.21.15.115:0/2163670888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6681dfdd0 msgr2=0x7fd66813d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66d344640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6681dfdd0 0x7fd66813d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66d344640 1 -- 172.21.15.115:0/2163670888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680ace80 con 0x7fd6681df950 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.871+0000 7fd66e346640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6681dfdd0 0x7fd66813d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.872+0000 7fd66d344640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6681df950 0x7fd6680adc40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd65c000c00 tx=0x7fd65c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.872+0000 7fd6667fc640 1 -- 172.21.15.115:0/2163670888 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c003810 con 0x7fd6681df950 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.872+0000 7fd6667fc640 1 -- 172.21.15.115:0/2163670888 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c003970 con 0x7fd6681df950 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.872+0000 7fd66fda9640 1 -- 172.21.15.115:0/2163670888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6680ad160 con 0x7fd6681df950 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.872+0000 7fd6667fc640 1 -- 172.21.15.115:0/2163670888 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c003ce0 con 0x7fd6681df950 2023-12-01T01:48:21.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.872+0000 7fd66fda9640 1 -- 172.21.15.115:0/2163670888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6680ad720 con 0x7fd6681df950 2023-12-01T01:48:21.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.873+0000 7fd66fda9640 1 -- 172.21.15.115:0/2163670888 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66804e890 con 0x7fd6681df950 2023-12-01T01:48:21.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.874+0000 7fd6667fc640 1 -- 172.21.15.115:0/2163670888 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fd65c040450 con 0x7fd6681df950 2023-12-01T01:48:21.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.876+0000 7fd6667fc640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd65406a630 0x7fd65406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:21.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.876+0000 7fd6667fc640 1 -- 172.21.15.115:0/2163670888 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fd65c0aad90 con 0x7fd6681df950 2023-12-01T01:48:21.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.876+0000 7fd66db45640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd65406a630 0x7fd65406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:21.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.877+0000 7fd66db45640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd65406a630 0x7fd65406caf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6681e6f50 tx=0x7fd658008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:21.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:21.881+0000 7fd6667fc640 1 -- 172.21.15.115:0/2163670888 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd65c075ee0 con 0x7fd6681df950 2023-12-01T01:48:22.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.092+0000 7fd66fda9640 1 -- 172.21.15.115:0/2163670888 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6680b2320 con 0x7fd65406a630 2023-12-01T01:48:22.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.260+0000 7fd6667fc640 1 -- 172.21.15.115:0/2163670888 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd658009700 con 0x7fd65406a630 2023-12-01T01:48:22.262 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-12-01T01:48:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.263+0000 7fd66fda9640 1 -- 172.21.15.115:0/2163670888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd65406a630 msgr2=0x7fd65406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.263+0000 7fd66fda9640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd65406a630 0x7fd65406caf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6681e6f50 tx=0x7fd658008040 comp rx=0 tx=0).stop 2023-12-01T01:48:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.264+0000 7fd66fda9640 1 -- 172.21.15.115:0/2163670888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6681df950 msgr2=0x7fd6680adc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.264+0000 7fd66fda9640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6681df950 0x7fd6680adc40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd65c000c00 tx=0x7fd65c004730 comp rx=0 tx=0).stop 2023-12-01T01:48:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.264+0000 7fd66fda9640 1 -- 172.21.15.115:0/2163670888 shutdown_connections 2023-12-01T01:48:22.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.264+0000 7fd66fda9640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd65406a630 0x7fd65406caf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.264+0000 7fd66fda9640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6681dfdd0 0x7fd66813d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.264+0000 7fd66fda9640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6681df950 0x7fd6680adc40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.264+0000 7fd66fda9640 1 --2- 172.21.15.115:0/2163670888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6500047c0 0x7fd6681f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.264+0000 7fd66fda9640 1 -- 172.21.15.115:0/2163670888 >> 172.21.15.115:0/2163670888 conn(0x7fd668065960 msgr2=0x7fd66813eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.264+0000 7fd66fda9640 1 -- 172.21.15.115:0/2163670888 shutdown_connections 2023-12-01T01:48:22.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.264+0000 7fd66fda9640 1 -- 172.21.15.115:0/2163670888 wait complete. 2023-12-01T01:48:22.286 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-12-01T01:48:22.286 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir1 1M 2023-12-01T01:48:22.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.512+0000 7f448359e640 1 Processor -- start 2023-12-01T01:48:22.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.512+0000 7f448359e640 1 -- start start 2023-12-01T01:48:22.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.514+0000 7f448359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44841599e0 0x7f4484159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:22.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.514+0000 7f448359e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f44840af370 0x7f44840c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:22.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.514+0000 7f448359e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448415e440 con 0x7f44840c95a0 2023-12-01T01:48:22.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.514+0000 7f448359e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448415a300 con 0x7f44840af370 2023-12-01T01:48:22.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.514+0000 7f448359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448415d550 con 0x7f44841599e0 2023-12-01T01:48:22.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.514+0000 7f4482d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f44840c95a0 0x7f448406f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56146/0 (socket says 172.21.15.115:56146) 2023-12-01T01:48:22.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.514+0000 7f4482d9d640 1 -- 172.21.15.115:0/1658591763 learned_addr learned my addr 172.21.15.115:0/1658591763 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f447bfff640 1 --2- 172.21.15.115:0/1658591763 >> v2:172.21.15.142:3300/0 conn(0x7f44840af370 0x7f44840c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f448259c640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44841599e0 0x7f4484159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782295262 0 0) 0x7f448415e440 con 0x7f44840c95a0 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4470003520 con 0x7f44840c95a0 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4137177490 0 0) 0x7f4470003520 con 0x7f44840c95a0 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44841599e0 msgr2=0x7f4484159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44841599e0 0x7f4484159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 >> v2:172.21.15.142:3300/0 conn(0x7f44840af370 msgr2=0x7f44840c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 --2- 172.21.15.115:0/1658591763 >> v2:172.21.15.142:3300/0 conn(0x7f44840af370 0x7f44840c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448415d830 con 0x7f44840c95a0 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f447c0019f0 con 0x7f44840c95a0 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 >> v1:172.21.15.115:6789/0 conn(0x7f44840c95a0 legacy=0x7f448406f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:22.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44841599e0 0x7f4470003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4470003f40 0x7f44700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4470004d00 0x7f4470005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448415a300 con 0x7f4470004d00 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448415d550 con 0x7f4470003f40 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.515+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447000db90 con 0x7f44841599e0 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.516+0000 7f4482d9d640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4470004d00 0x7f4470005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.516+0000 7f447bfff640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4470003f40 0x7f44700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.516+0000 7f448259c640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44841599e0 0x7f4470003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.516+0000 7f4482d9d640 1 -- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44841599e0 msgr2=0x7f4470003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.516+0000 7f4482d9d640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44841599e0 0x7f4470003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.516+0000 7f4482d9d640 1 -- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4470003f40 msgr2=0x7f44700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:22.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.516+0000 7f4482d9d640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4470003f40 0x7f44700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.516+0000 7f4482d9d640 1 -- 172.21.15.115:0/1658591763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4470003520 con 0x7f4470004d00 2023-12-01T01:48:22.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.516+0000 7f447bfff640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4470003f40 0x7f44700047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:22.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.517+0000 7f4482d9d640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4470004d00 0x7f4470005180 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f447c00a7b0 tx=0x7f44700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bdd6e76fddb94 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:22.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.517+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c034630 con 0x7f4470004d00 2023-12-01T01:48:22.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.518+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c02f070 con 0x7f4470004d00 2023-12-01T01:48:22.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.518+0000 7f4481d9b640 1 -- 172.21.15.115:0/1658591763 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c034a10 con 0x7f4470004d00 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.518+0000 7f448359e640 1 -- 172.21.15.115:0/1658591763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4470004d00 msgr2=0x7f4470005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.518+0000 7f448359e640 1 --2- 172.21.15.115:0/1658591763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4470004d00 0x7f4470005180 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f447c00a7b0 tx=0x7f44700057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.518+0000 7f4482d9d640 1 -- 172.21.15.115:0/1658591763 reap_dead start 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.518+0000 7f448359e640 1 -- 172.21.15.115:0/1658591763 shutdown_connections 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.518+0000 7f448359e640 1 -- 172.21.15.115:0/1658591763 >> 172.21.15.115:0/1658591763 conn(0x7f4484065960 msgr2=0x7f4484065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.518+0000 7f448359e640 1 -- 172.21.15.115:0/1658591763 shutdown_connections 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.518+0000 7f448359e640 1 -- 172.21.15.115:0/1658591763 wait complete. 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.520+0000 7f448359e640 1 Processor -- start 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.520+0000 7f448359e640 1 -- start start 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.521+0000 7f448359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4470004d00 0x7f44840af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:22.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.521+0000 7f448359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44840acfe0 0x7f44840aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.521+0000 7f448359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44840c7980 0x7f44840c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.521+0000 7f448359e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44841f1cf0 con 0x7f44840c7980 2023-12-01T01:48:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.521+0000 7f448359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44840c65c0 con 0x7f44840acfe0 2023-12-01T01:48:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.521+0000 7f448359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44840c47b0 con 0x7f4470004d00 2023-12-01T01:48:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.521+0000 7f4482d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44840c7980 0x7f44840c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.522+0000 7f4482d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44840c7980 0x7f44840c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47190/0 (socket says 172.21.15.115:47190) 2023-12-01T01:48:22.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.522+0000 7f4482d9d640 1 -- 172.21.15.115:0/1023054273 learned_addr learned my addr 172.21.15.115:0/1023054273 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:22.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.522+0000 7f448259c640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4470004d00 0x7f44840af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:22.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.522+0000 7f447bfff640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44840acfe0 0x7f44840aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:22.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.522+0000 7f4482d9d640 1 -- 172.21.15.115:0/1023054273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4470004d00 msgr2=0x7f44840af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.522+0000 7f4482d9d640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4470004d00 0x7f44840af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.522+0000 7f4482d9d640 1 -- 172.21.15.115:0/1023054273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44840acfe0 msgr2=0x7f44840aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.522+0000 7f4482d9d640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44840acfe0 0x7f44840aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.522+0000 7f4482d9d640 1 -- 172.21.15.115:0/1023054273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44841f2110 con 0x7f44840c7980 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.522+0000 7f4482d9d640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44840c7980 0x7f44840c7e00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f447c02cab0 tx=0x7f447c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.523+0000 7f447affd640 1 -- 172.21.15.115:0/1023054273 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c034430 con 0x7f44840c7980 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.523+0000 7f447affd640 1 -- 172.21.15.115:0/1023054273 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c034590 con 0x7f44840c7980 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.523+0000 7f447affd640 1 -- 172.21.15.115:0/1023054273 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c034900 con 0x7f44840c7980 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.523+0000 7f448359e640 1 -- 172.21.15.115:0/1023054273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44841f1350 con 0x7f44840c7980 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.523+0000 7f448359e640 1 -- 172.21.15.115:0/1023054273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44841f1910 con 0x7f44840c7980 2023-12-01T01:48:22.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.524+0000 7f448359e640 1 -- 172.21.15.115:0/1023054273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44841f1e30 con 0x7f44840c7980 2023-12-01T01:48:22.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.525+0000 7f447affd640 1 -- 172.21.15.115:0/1023054273 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f447c031030 con 0x7f44840c7980 2023-12-01T01:48:22.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.526+0000 7f447affd640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446406a660 0x7f446406cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:22.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.526+0000 7f447affd640 1 -- 172.21.15.115:0/1023054273 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f447c0aafa0 con 0x7f44840c7980 2023-12-01T01:48:22.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.527+0000 7f448259c640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446406a660 0x7f446406cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:22.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.527+0000 7f448259c640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446406a660 0x7f446406cb20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4474009a10 tx=0x7f4474008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:22.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.531+0000 7f447affd640 1 -- 172.21.15.115:0/1023054273 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f44841f1e30 con 0x7f44840c7980 2023-12-01T01:48:22.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.745+0000 7f448359e640 1 -- 172.21.15.115:0/1023054273 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f448415ddf0 con 0x7f446406a660 2023-12-01T01:48:22.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.910+0000 7f447affd640 1 -- 172.21.15.115:0/1023054273 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f44740128f0 con 0x7f446406a660 2023-12-01T01:48:22.912 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-12-01T01:48:22.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.913+0000 7f448359e640 1 -- 172.21.15.115:0/1023054273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446406a660 msgr2=0x7f446406cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:22.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446406a660 0x7f446406cb20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4474009a10 tx=0x7f4474008040 comp rx=0 tx=0).stop 2023-12-01T01:48:22.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 -- 172.21.15.115:0/1023054273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44840c7980 msgr2=0x7f44840c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:22.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44840c7980 0x7f44840c7e00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f447c02cab0 tx=0x7f447c001360 comp rx=0 tx=0).stop 2023-12-01T01:48:22.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 -- 172.21.15.115:0/1023054273 shutdown_connections 2023-12-01T01:48:22.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446406a660 0x7f446406cb20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44840c7980 0x7f44840c7e00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44840acfe0 0x7f44840aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 --2- 172.21.15.115:0/1023054273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4470004d00 0x7f44840af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:22.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 -- 172.21.15.115:0/1023054273 >> 172.21.15.115:0/1023054273 conn(0x7f4484065960 msgr2=0x7f4484065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:22.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 -- 172.21.15.115:0/1023054273 shutdown_connections 2023-12-01T01:48:22.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:22.914+0000 7f448359e640 1 -- 172.21.15.115:0/1023054273 wait complete. 2023-12-01T01:48:22.937 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-12-01T01:48:22.937 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir2 1M 2023-12-01T01:48:23.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.137+0000 7f677aadb640 1 Processor -- start 2023-12-01T01:48:23.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.137+0000 7f677aadb640 1 -- start start 2023-12-01T01:48:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.139+0000 7f677aadb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f67740cc440 0x7f67740b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.139+0000 7f677aadb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67740cb290 con 0x7f6774160200 2023-12-01T01:48:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.139+0000 7f677aadb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67740b0ae0 con 0x7f67740b0f80 2023-12-01T01:48:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.139+0000 7f677aadb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67740b35f0 con 0x7f67740cc440 2023-12-01T01:48:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.139+0000 7f6778877640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6774160200 0x7f677406b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56156/0 (socket says 172.21.15.115:56156) 2023-12-01T01:48:23.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.139+0000 7f6778877640 1 -- 172.21.15.115:0/930086953 learned_addr learned my addr 172.21.15.115:0/930086953 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:23.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.139+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921308929 0 0) 0x7f67740cb290 con 0x7f6774160200 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.139+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6764003520 con 0x7f6774160200 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.139+0000 7f6779078640 1 --2- 172.21.15.115:0/930086953 >> v2:172.21.15.142:3300/0 conn(0x7f67740cc440 0x7f67740b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1849957937 0 0) 0x7f6764003520 con 0x7f6774160200 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 >> v2:172.21.15.142:3300/0 conn(0x7f67740cc440 msgr2=0x7f67740b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 --2- 172.21.15.115:0/930086953 >> v2:172.21.15.142:3300/0 conn(0x7f67740cc440 0x7f67740b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 >> v1:172.21.15.142:6789/0 conn(0x7f67740b0f80 legacy=0x7f67740cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740b3310 con 0x7f6774160200 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6768001670 con 0x7f6774160200 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 >> v1:172.21.15.115:6789/0 conn(0x7f6774160200 legacy=0x7f677406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67640039c0 0x7f6764003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67640047c0 0x7f6764004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67640051d0 0x7f676400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67740b35f0 con 0x7f67640047c0 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676400c100 con 0x7f67640051d0 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676400c2c0 con 0x7f67640039c0 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f6779078640 1 --2- 172.21.15.115:0/930086953 >> v2:172.21.15.142:3300/0 conn(0x7f67740cc440 0x7f67740b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.140+0000 7f6773fff640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67640047c0 0x7f6764004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.141+0000 7f6778877640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67640039c0 0x7f6764003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.141+0000 7f6779078640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67640051d0 0x7f676400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.141+0000 7f6773fff640 1 -- 172.21.15.115:0/930086953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67640039c0 msgr2=0x7f6764003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.141+0000 7f6773fff640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67640039c0 0x7f6764003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.141+0000 7f6773fff640 1 -- 172.21.15.115:0/930086953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67640051d0 msgr2=0x7f676400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.141+0000 7f6773fff640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67640051d0 0x7f676400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.141+0000 7f6773fff640 1 -- 172.21.15.115:0/930086953 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764003520 con 0x7f67640047c0 2023-12-01T01:48:23.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.141+0000 7f6779078640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67640051d0 0x7f676400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:23.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.141+0000 7f6779078640 1 -- 172.21.15.115:0/930086953 reap_dead start 2023-12-01T01:48:23.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.141+0000 7f6778877640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67640039c0 0x7f6764003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:23.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.142+0000 7f6773fff640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67640047c0 0x7f6764004c40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6760003d80 tx=0x7f67640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197592f986328463 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:23.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.142+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760032600 con 0x7f67640047c0 2023-12-01T01:48:23.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.142+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676002e050 con 0x7f67640047c0 2023-12-01T01:48:23.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.142+0000 7f67737fe640 1 -- 172.21.15.115:0/930086953 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600329e0 con 0x7f67640047c0 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.143+0000 7f677aadb640 1 -- 172.21.15.115:0/930086953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67640047c0 msgr2=0x7f6764004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.143+0000 7f677aadb640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67640047c0 0x7f6764004c40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6760003d80 tx=0x7f67640057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.143+0000 7f677aadb640 1 -- 172.21.15.115:0/930086953 shutdown_connections 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.143+0000 7f677aadb640 1 --2- 172.21.15.115:0/930086953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67640047c0 0x7f6764004c40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.143+0000 7f677aadb640 1 -- 172.21.15.115:0/930086953 >> 172.21.15.115:0/930086953 conn(0x7f6774065960 msgr2=0x7f6774065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.143+0000 7f677aadb640 1 -- 172.21.15.115:0/930086953 shutdown_connections 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.143+0000 7f677aadb640 1 -- 172.21.15.115:0/930086953 wait complete. 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.145+0000 7f677aadb640 1 Processor -- start 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.145+0000 7f677aadb640 1 -- start start 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.146+0000 7f677aadb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67640047c0 0x7f677414d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.146+0000 7f677aadb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f677414fb10 0x7f677414db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.146+0000 7f677aadb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67740ca170 0x7f677414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.146+0000 7f677aadb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677414fff0 con 0x7f677414fb10 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.146+0000 7f677aadb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67740c9690 con 0x7f67640047c0 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.146+0000 7f677aadb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67740c9970 con 0x7f67740ca170 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.146+0000 7f6773fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f677414fb10 0x7f677414db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.146+0000 7f6773fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f677414fb10 0x7f677414db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47198/0 (socket says 172.21.15.115:47198) 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.146+0000 7f6773fff640 1 -- 172.21.15.115:0/2078962904 learned_addr learned my addr 172.21.15.115:0/2078962904 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f6778877640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67640047c0 0x7f677414d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f6773fff640 1 -- 172.21.15.115:0/2078962904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67740ca170 msgr2=0x7f677414e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f6773fff640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67740ca170 0x7f677414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f6773fff640 1 -- 172.21.15.115:0/2078962904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67640047c0 msgr2=0x7f677414d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f6773fff640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67640047c0 0x7f677414d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f6773fff640 1 -- 172.21.15.115:0/2078962904 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740c9c50 con 0x7f677414fb10 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f6773fff640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f677414fb10 0x7f677414db20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6760035040 tx=0x7f67600036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:23.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f67717fa640 1 -- 172.21.15.115:0/2078962904 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676003c610 con 0x7f677414fb10 2023-12-01T01:48:23.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f67717fa640 1 -- 172.21.15.115:0/2078962904 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676003c770 con 0x7f677414fb10 2023-12-01T01:48:23.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f67717fa640 1 -- 172.21.15.115:0/2078962904 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676003cb00 con 0x7f677414fb10 2023-12-01T01:48:23.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f677aadb640 1 -- 172.21.15.115:0/2078962904 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677414edd0 con 0x7f677414fb10 2023-12-01T01:48:23.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.147+0000 7f677aadb640 1 -- 172.21.15.115:0/2078962904 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677414e5a0 con 0x7f677414fb10 2023-12-01T01:48:23.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.148+0000 7f677aadb640 1 -- 172.21.15.115:0/2078962904 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67740681f0 con 0x7f677414fb10 2023-12-01T01:48:23.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.149+0000 7f67717fa640 1 -- 172.21.15.115:0/2078962904 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f6760039050 con 0x7f677414fb10 2023-12-01T01:48:23.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.150+0000 7f67717fa640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f675806a630 0x7f675806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.150+0000 7f67717fa640 1 -- 172.21.15.115:0/2078962904 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f67600aa780 con 0x7f677414fb10 2023-12-01T01:48:23.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.151+0000 7f6778877640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f675806a630 0x7f675806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.151+0000 7f6778877640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f675806a630 0x7f675806caf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6768000fd0 tx=0x7f6768009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:23.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.154+0000 7f67717fa640 1 -- 172.21.15.115:0/2078962904 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f67600757c0 con 0x7f677414fb10 2023-12-01T01:48:23.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.367+0000 7f677aadb640 1 -- 172.21.15.115:0/2078962904 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f67740cb810 con 0x7f675806a630 2023-12-01T01:48:23.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.537+0000 7f67717fa640 1 -- 172.21.15.115:0/2078962904 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f676800d040 con 0x7f675806a630 2023-12-01T01:48:23.539 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-12-01T01:48:23.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.540+0000 7f677aadb640 1 -- 172.21.15.115:0/2078962904 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f675806a630 msgr2=0x7f675806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:23.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.540+0000 7f677aadb640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f675806a630 0x7f675806caf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6768000fd0 tx=0x7f6768009040 comp rx=0 tx=0).stop 2023-12-01T01:48:23.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.540+0000 7f677aadb640 1 -- 172.21.15.115:0/2078962904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f677414fb10 msgr2=0x7f677414db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.540+0000 7f677aadb640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f677414fb10 0x7f677414db20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6760035040 tx=0x7f67600036f0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.541+0000 7f677aadb640 1 -- 172.21.15.115:0/2078962904 shutdown_connections 2023-12-01T01:48:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.541+0000 7f677aadb640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f675806a630 0x7f675806caf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.541+0000 7f677aadb640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67740ca170 0x7f677414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.541+0000 7f677aadb640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f677414fb10 0x7f677414db20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.541+0000 7f677aadb640 1 --2- 172.21.15.115:0/2078962904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67640047c0 0x7f677414d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.541+0000 7f677aadb640 1 -- 172.21.15.115:0/2078962904 >> 172.21.15.115:0/2078962904 conn(0x7f6774065960 msgr2=0x7f6774065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.541+0000 7f677aadb640 1 -- 172.21.15.115:0/2078962904 shutdown_connections 2023-12-01T01:48:23.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.541+0000 7f677aadb640 1 -- 172.21.15.115:0/2078962904 wait complete. 2023-12-01T01:48:23.562 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-12-01T01:48:23.563 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir3 1M 2023-12-01T01:48:23.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.777+0000 7f2e64111640 1 Processor -- start 2023-12-01T01:48:23.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.777+0000 7f2e64111640 1 -- start start 2023-12-01T01:48:23.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.779+0000 7f2e64111640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2e5c0cbc20 0x7f2e5c15e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.779+0000 7f2e64111640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2e5c06ff00 0x7f2e5c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.779+0000 7f2e64111640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5c15ec60 con 0x7f2e5c15ab10 2023-12-01T01:48:23.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.779+0000 7f2e64111640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5c15f730 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.779+0000 7f2e64111640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5c0ae6d0 con 0x7f2e5c06ff00 2023-12-01T01:48:23.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.779+0000 7f2e616ac640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2e5c0cbc20 0x7f2e5c15e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.779+0000 7f2e616ac640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2e5c0cbc20 0x7f2e5c15e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47208/0 (socket says 172.21.15.115:47208) 2023-12-01T01:48:23.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.779+0000 7f2e616ac640 1 -- 172.21.15.115:0/2806063709 learned_addr learned my addr 172.21.15.115:0/2806063709 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:23.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.780+0000 7f2e616ac640 1 -- 172.21.15.115:0/2806063709 >> v2:172.21.15.142:3300/0 conn(0x7f2e5c06ff00 msgr2=0x7f2e5c0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:23.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.780+0000 7f2e616ac640 1 --2- 172.21.15.115:0/2806063709 >> v2:172.21.15.142:3300/0 conn(0x7f2e5c06ff00 0x7f2e5c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.780+0000 7f2e616ac640 1 -- 172.21.15.115:0/2806063709 >> v1:172.21.15.115:6789/0 conn(0x7f2e5c15ab10 legacy=0x7f2e5c0cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:23.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.780+0000 7f2e616ac640 1 -- 172.21.15.115:0/2806063709 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c15ee90 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.781+0000 7f2e616ac640 1 --2- 172.21.15.115:0/2806063709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 0x7f2e5c15e720 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c06f3c0 tx=0x7f2e4c02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6990dfb5265bfdd9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:23.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.781+0000 7f2e60eab640 1 -- 172.21.15.115:0/2806063709 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c030870 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.781+0000 7f2e60eab640 1 -- 172.21.15.115:0/2806063709 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c0309d0 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.781+0000 7f2e60eab640 1 -- 172.21.15.115:0/2806063709 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c030d60 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.782+0000 7f2e64111640 1 -- 172.21.15.115:0/2806063709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 msgr2=0x7f2e5c15e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:23.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.782+0000 7f2e64111640 1 --2- 172.21.15.115:0/2806063709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 0x7f2e5c15e720 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c06f3c0 tx=0x7f2e4c02b9b0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.782+0000 7f2e64111640 1 -- 172.21.15.115:0/2806063709 shutdown_connections 2023-12-01T01:48:23.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.782+0000 7f2e64111640 1 --2- 172.21.15.115:0/2806063709 >> v2:172.21.15.142:3300/0 conn(0x7f2e5c06ff00 0x7f2e5c0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.782+0000 7f2e64111640 1 --2- 172.21.15.115:0/2806063709 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 0x7f2e5c15e720 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.782+0000 7f2e64111640 1 -- 172.21.15.115:0/2806063709 >> 172.21.15.115:0/2806063709 conn(0x7f2e5c065b40 msgr2=0x7f2e5c065f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:23.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.782+0000 7f2e64111640 1 -- 172.21.15.115:0/2806063709 shutdown_connections 2023-12-01T01:48:23.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.782+0000 7f2e64111640 1 -- 172.21.15.115:0/2806063709 wait complete. 2023-12-01T01:48:23.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.784+0000 7f2e64111640 1 Processor -- start 2023-12-01T01:48:23.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.784+0000 7f2e64111640 1 -- start start 2023-12-01T01:48:23.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e64111640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e5c06ff00 0x7f2e5c1f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e64111640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 0x7f2e5c0c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e64111640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c15ab10 0x7f2e5c0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e64111640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0c5dd0 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e64111640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0c6c10 con 0x7f2e5c15ab10 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e64111640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0c6000 con 0x7f2e5c06ff00 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e616ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 0x7f2e5c0c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e616ac640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 0x7f2e5c0c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47218/0 (socket says 172.21.15.115:47218) 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e616ac640 1 -- 172.21.15.115:0/1442935062 learned_addr learned my addr 172.21.15.115:0/1442935062 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e61ead640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e5c06ff00 0x7f2e5c1f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e616ac640 1 -- 172.21.15.115:0/1442935062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e5c06ff00 msgr2=0x7f2e5c1f02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e626ae640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c15ab10 0x7f2e5c0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e616ac640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e5c06ff00 0x7f2e5c1f02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e616ac640 1 -- 172.21.15.115:0/1442935062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c15ab10 msgr2=0x7f2e5c0c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:23.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e616ac640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c15ab10 0x7f2e5c0c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:23.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e616ac640 1 -- 172.21.15.115:0/1442935062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c0094d0 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.785+0000 7f2e626ae640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c15ab10 0x7f2e5c0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:23.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.786+0000 7f2e616ac640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 0x7f2e5c0c5890 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c000c00 tx=0x7f2e4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:23.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.786+0000 7f2e527fc640 1 -- 172.21.15.115:0/1442935062 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c030840 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.786+0000 7f2e527fc640 1 -- 172.21.15.115:0/1442935062 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c043400 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.786+0000 7f2e64111640 1 -- 172.21.15.115:0/1442935062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5c0c62e0 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.786+0000 7f2e527fc640 1 -- 172.21.15.115:0/1442935062 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0437b0 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.786+0000 7f2e64111640 1 -- 172.21.15.115:0/1442935062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c0c6670 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.787+0000 7f2e64111640 1 -- 172.21.15.115:0/1442935062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5c0c6da0 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.788+0000 7f2e527fc640 1 -- 172.21.15.115:0/1442935062 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f2e4c043910 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.790+0000 7f2e527fc640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e4006a630 0x7f2e4006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:23.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.790+0000 7f2e527fc640 1 -- 172.21.15.115:0/1442935062 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f2e4c03e070 con 0x7f2e5c0cbc20 2023-12-01T01:48:23.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.790+0000 7f2e61ead640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e4006a630 0x7f2e4006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:23.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.791+0000 7f2e61ead640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e4006a630 0x7f2e4006caf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2e58000f60 tx=0x7f2e58009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:23.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:23.794+0000 7f2e527fc640 1 -- 172.21.15.115:0/1442935062 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2e4c077460 con 0x7f2e5c0cbc20 2023-12-01T01:48:24.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.006+0000 7f2e64111640 1 -- 172.21.15.115:0/1442935062 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e5c15f450 con 0x7f2e4006a630 2023-12-01T01:48:24.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.172+0000 7f2e527fc640 1 -- 172.21.15.115:0/1442935062 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f2e58001240 con 0x7f2e4006a630 2023-12-01T01:48:24.174 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-12-01T01:48:24.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.175+0000 7f2e64111640 1 -- 172.21.15.115:0/1442935062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e4006a630 msgr2=0x7f2e4006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:24.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.175+0000 7f2e64111640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e4006a630 0x7f2e4006caf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2e58000f60 tx=0x7f2e58009040 comp rx=0 tx=0).stop 2023-12-01T01:48:24.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.175+0000 7f2e64111640 1 -- 172.21.15.115:0/1442935062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 msgr2=0x7f2e5c0c5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:24.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.175+0000 7f2e64111640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 0x7f2e5c0c5890 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c000c00 tx=0x7f2e4c004730 comp rx=0 tx=0).stop 2023-12-01T01:48:24.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.175+0000 7f2e64111640 1 -- 172.21.15.115:0/1442935062 shutdown_connections 2023-12-01T01:48:24.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.175+0000 7f2e64111640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2e4006a630 0x7f2e4006caf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.175+0000 7f2e64111640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e5c15ab10 0x7f2e5c0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.175+0000 7f2e64111640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e5c0cbc20 0x7f2e5c0c5890 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.175+0000 7f2e64111640 1 --2- 172.21.15.115:0/1442935062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e5c06ff00 0x7f2e5c1f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.175+0000 7f2e64111640 1 -- 172.21.15.115:0/1442935062 >> 172.21.15.115:0/1442935062 conn(0x7f2e5c065b40 msgr2=0x7f2e5c06bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:24.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.176+0000 7f2e64111640 1 -- 172.21.15.115:0/1442935062 shutdown_connections 2023-12-01T01:48:24.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.176+0000 7f2e64111640 1 -- 172.21.15.115:0/1442935062 wait complete. 2023-12-01T01:48:24.198 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-12-01T01:48:24.198 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir4 1M 2023-12-01T01:48:24.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.412+0000 7f20621e3640 1 Processor -- start 2023-12-01T01:48:24.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.412+0000 7f20621e3640 1 -- start start 2023-12-01T01:48:24.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.414+0000 7f20621e3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 0x7f205c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:24.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.414+0000 7f20621e3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c0cd470 con 0x7f205c0cd010 2023-12-01T01:48:24.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.414+0000 7f20621e3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c0b2f30 con 0x7f205c0b1e20 2023-12-01T01:48:24.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.414+0000 7f20621e3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0b3ae0 con 0x7f205c142870 2023-12-01T01:48:24.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.414+0000 7f2053fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f205c0cd010 0x7f205c0b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56174/0 (socket says 172.21.15.115:56174) 2023-12-01T01:48:24.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.414+0000 7f2053fff640 1 -- 172.21.15.115:0/383443016 learned_addr learned my addr 172.21.15.115:0/383443016 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:24.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.414+0000 7f205b7fe640 1 --2- 172.21.15.115:0/383443016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 0x7f205c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:24.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.415+0000 7f205b7fe640 1 -- 172.21.15.115:0/383443016 >> v1:172.21.15.142:6789/0 conn(0x7f205c0b1e20 legacy=0x7f205c15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:24.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.415+0000 7f205b7fe640 1 -- 172.21.15.115:0/383443016 >> v1:172.21.15.115:6789/0 conn(0x7f205c0cd010 legacy=0x7f205c0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:24.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.415+0000 7f205b7fe640 1 -- 172.21.15.115:0/383443016 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0b2320 con 0x7f205c142870 2023-12-01T01:48:24.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.417+0000 7f205b7fe640 1 --2- 172.21.15.115:0/383443016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 0x7f205c0ccad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f204c0097d0 tx=0x7f204c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a0acf89af684af server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:24.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.423+0000 7f205affd640 1 -- 172.21.15.115:0/383443016 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c030800 con 0x7f205c142870 2023-12-01T01:48:24.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.423+0000 7f205affd640 1 -- 172.21.15.115:0/383443016 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c030960 con 0x7f205c142870 2023-12-01T01:48:24.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.423+0000 7f205affd640 1 -- 172.21.15.115:0/383443016 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c030cf0 con 0x7f205c142870 2023-12-01T01:48:24.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.424+0000 7f20621e3640 1 -- 172.21.15.115:0/383443016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 msgr2=0x7f205c0ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:24.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.424+0000 7f20621e3640 1 --2- 172.21.15.115:0/383443016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 0x7f205c0ccad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f204c0097d0 tx=0x7f204c02b940 comp rx=0 tx=0).stop 2023-12-01T01:48:24.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.424+0000 7f20621e3640 1 -- 172.21.15.115:0/383443016 shutdown_connections 2023-12-01T01:48:24.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.424+0000 7f20621e3640 1 --2- 172.21.15.115:0/383443016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 0x7f205c0ccad0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.424+0000 7f20621e3640 1 -- 172.21.15.115:0/383443016 >> 172.21.15.115:0/383443016 conn(0x7f205c065960 msgr2=0x7f205c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:24.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.424+0000 7f20621e3640 1 -- 172.21.15.115:0/383443016 shutdown_connections 2023-12-01T01:48:24.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.424+0000 7f20621e3640 1 -- 172.21.15.115:0/383443016 wait complete. 2023-12-01T01:48:24.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.426+0000 7f20621e3640 1 Processor -- start 2023-12-01T01:48:24.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.426+0000 7f20621e3640 1 -- start start 2023-12-01T01:48:24.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.427+0000 7f20621e3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f205c0b1e20 0x7f205c14cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:24.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.427+0000 7f20621e3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f205c0cd010 0x7f205c0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:24.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.427+0000 7f20621e3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 0x7f205c1f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:24.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.427+0000 7f20621e3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0acbe0 con 0x7f205c0cd010 2023-12-01T01:48:24.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.427+0000 7f20621e3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c14d320 con 0x7f205c0b1e20 2023-12-01T01:48:24.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.427+0000 7f20621e3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0aed90 con 0x7f205c142870 2023-12-01T01:48:24.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.427+0000 7f2053fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f205c0cd010 0x7f205c0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.427+0000 7f2053fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f205c0cd010 0x7f205c0afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47228/0 (socket says 172.21.15.115:47228) 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.427+0000 7f2053fff640 1 -- 172.21.15.115:0/1916650548 learned_addr learned my addr 172.21.15.115:0/1916650548 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.427+0000 7f205b7fe640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f205c0b1e20 0x7f205c14cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f205bfff640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 0x7f205c1f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f2053fff640 1 -- 172.21.15.115:0/1916650548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 msgr2=0x7f205c1f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f2053fff640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 0x7f205c1f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f2053fff640 1 -- 172.21.15.115:0/1916650548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f205c0b1e20 msgr2=0x7f205c14cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f2053fff640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f205c0b1e20 0x7f205c14cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f2053fff640 1 -- 172.21.15.115:0/1916650548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c009400 con 0x7f205c0cd010 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f2053fff640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f205c0cd010 0x7f205c0afe30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f204400c4a0 tx=0x7f204400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f2058ff9640 1 -- 172.21.15.115:0/1916650548 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044005ce0 con 0x7f205c0cd010 2023-12-01T01:48:24.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f2058ff9640 1 -- 172.21.15.115:0/1916650548 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2044005e40 con 0x7f205c0cd010 2023-12-01T01:48:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f2058ff9640 1 -- 172.21.15.115:0/1916650548 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044007270 con 0x7f205c0cd010 2023-12-01T01:48:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.428+0000 7f20621e3640 1 -- 172.21.15.115:0/1916650548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c0af070 con 0x7f205c0cd010 2023-12-01T01:48:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.429+0000 7f20621e3640 1 -- 172.21.15.115:0/1916650548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c0af910 con 0x7f205c0cd010 2023-12-01T01:48:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.430+0000 7f20621e3640 1 -- 172.21.15.115:0/1916650548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c0af260 con 0x7f205c0cd010 2023-12-01T01:48:24.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.431+0000 7f2058ff9640 1 -- 172.21.15.115:0/1916650548 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f2044007490 con 0x7f205c0cd010 2023-12-01T01:48:24.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.432+0000 7f2058ff9640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f203c06a630 0x7f203c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:24.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.432+0000 7f2058ff9640 1 -- 172.21.15.115:0/1916650548 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f2044088230 con 0x7f205c0cd010 2023-12-01T01:48:24.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.432+0000 7f205b7fe640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f203c06a630 0x7f203c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:24.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.433+0000 7f205b7fe640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f203c06a630 0x7f203c06caf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f204c036000 tx=0x7f204c03b040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:24.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.436+0000 7f2058ff9640 1 -- 172.21.15.115:0/1916650548 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2044053560 con 0x7f205c0cd010 2023-12-01T01:48:24.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.652+0000 7f20621e3640 1 -- 172.21.15.115:0/1916650548 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f205c0b28e0 con 0x7f203c06a630 2023-12-01T01:48:24.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.815+0000 7f2058ff9640 1 -- 172.21.15.115:0/1916650548 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f204c030bd0 con 0x7f203c06a630 2023-12-01T01:48:24.817 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-12-01T01:48:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 -- 172.21.15.115:0/1916650548 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f203c06a630 msgr2=0x7f203c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f203c06a630 0x7f203c06caf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f204c036000 tx=0x7f204c03b040 comp rx=0 tx=0).stop 2023-12-01T01:48:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 -- 172.21.15.115:0/1916650548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f205c0cd010 msgr2=0x7f205c0afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f205c0cd010 0x7f205c0afe30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f204400c4a0 tx=0x7f204400c960 comp rx=0 tx=0).stop 2023-12-01T01:48:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 -- 172.21.15.115:0/1916650548 shutdown_connections 2023-12-01T01:48:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f203c06a630 0x7f203c06caf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f205c142870 0x7f205c1f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f205c0cd010 0x7f205c0afe30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 --2- 172.21.15.115:0/1916650548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f205c0b1e20 0x7f205c14cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 -- 172.21.15.115:0/1916650548 >> 172.21.15.115:0/1916650548 conn(0x7f205c065960 msgr2=0x7f205c141a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 -- 172.21.15.115:0/1916650548 shutdown_connections 2023-12-01T01:48:24.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:24.819+0000 7f20621e3640 1 -- 172.21.15.115:0/1916650548 wait complete. 2023-12-01T01:48:24.843 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-12-01T01:48:24.843 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir5 1M 2023-12-01T01:48:25.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.051+0000 7fa0cc1d9640 1 Processor -- start 2023-12-01T01:48:25.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.051+0000 7fa0cc1d9640 1 -- start start 2023-12-01T01:48:25.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.053+0000 7fa0cc1d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40cd1e0 0x7fa0c40b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.053+0000 7fa0cc1d9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40b20e0 con 0x7fa0c4142870 2023-12-01T01:48:25.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.053+0000 7fa0cc1d9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40b2f30 con 0x7fa0c40b1c50 2023-12-01T01:48:25.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.053+0000 7fa0cc1d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b3ae0 con 0x7fa0c40cd1e0 2023-12-01T01:48:25.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.053+0000 7fa0c9f75640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa0c4142870 0x7fa0c40ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56184/0 (socket says 172.21.15.115:56184) 2023-12-01T01:48:25.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.053+0000 7fa0c9f75640 1 -- 172.21.15.115:0/4226312847 learned_addr learned my addr 172.21.15.115:0/4226312847 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c9774640 1 --2- 172.21.15.115:0/4226312847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40cd1e0 0x7fa0c40b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871602336 0 0) 0x7fa0c0001110 con 0x7fa0c4142870 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b0003520 con 0x7fa0c4142870 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 453639667 0 0) 0x7fa0b0003520 con 0x7fa0c4142870 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40cd1e0 msgr2=0x7fa0c40b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 --2- 172.21.15.115:0/4226312847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40cd1e0 0x7fa0c40b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 >> v1:172.21.15.142:6789/0 conn(0x7fa0c40b1c50 legacy=0x7fa0c415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40b2320 con 0x7fa0c4142870 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa0c0001800 con 0x7fa0c4142870 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 >> v1:172.21.15.115:6789/0 conn(0x7fa0c4142870 legacy=0x7fa0c40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 --2- 172.21.15.115:0/4226312847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 0x7fa0b0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 --2- 172.21.15.115:0/4226312847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b00047c0 0x7fa0b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 --2- 172.21.15.115:0/4226312847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40cd1e0 0x7fa0b0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b3ae0 con 0x7fa0b00039a0 2023-12-01T01:48:25.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b000dd20 con 0x7fa0b00047c0 2023-12-01T01:48:25.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.054+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b000dee0 con 0x7fa0c40cd1e0 2023-12-01T01:48:25.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.055+0000 7fa0c9f75640 1 --2- 172.21.15.115:0/4226312847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 0x7fa0b0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.055+0000 7fa0c9f75640 1 -- 172.21.15.115:0/4226312847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40cd1e0 msgr2=0x7fa0b0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:25.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.055+0000 7fa0c9f75640 1 --2- 172.21.15.115:0/4226312847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40cd1e0 0x7fa0b0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.055+0000 7fa0c9f75640 1 -- 172.21.15.115:0/4226312847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b00047c0 msgr2=0x7fa0b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:25.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.055+0000 7fa0c9f75640 1 --2- 172.21.15.115:0/4226312847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0b00047c0 0x7fa0b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.055+0000 7fa0c9f75640 1 -- 172.21.15.115:0/4226312847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b0003520 con 0x7fa0b00039a0 2023-12-01T01:48:25.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.056+0000 7fa0c9f75640 1 --2- 172.21.15.115:0/4226312847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 0x7fa0b0003dc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0007fc0 tx=0x7fa0b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fcafcfb7035a26 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:25.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.056+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0005ad0 con 0x7fa0b00039a0 2023-12-01T01:48:25.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.056+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c0005c50 con 0x7fa0b00039a0 2023-12-01T01:48:25.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.056+0000 7fa0c8f73640 1 -- 172.21.15.115:0/4226312847 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c002c7e0 con 0x7fa0b00039a0 2023-12-01T01:48:25.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.057+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/4226312847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 msgr2=0x7fa0b0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.057+0000 7fa0cc1d9640 1 --2- 172.21.15.115:0/4226312847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 0x7fa0b0003dc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0007fc0 tx=0x7fa0b00057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.057+0000 7fa0ca776640 1 -- 172.21.15.115:0/4226312847 reap_dead start 2023-12-01T01:48:25.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.057+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/4226312847 shutdown_connections 2023-12-01T01:48:25.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.057+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/4226312847 >> 172.21.15.115:0/4226312847 conn(0x7fa0c4065960 msgr2=0x7fa0c4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:25.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.057+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/4226312847 shutdown_connections 2023-12-01T01:48:25.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.057+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/4226312847 wait complete. 2023-12-01T01:48:25.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.059+0000 7fa0cc1d9640 1 Processor -- start 2023-12-01T01:48:25.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.059+0000 7fa0cc1d9640 1 -- start start 2023-12-01T01:48:25.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0cc1d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 0x7fa0c413d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0cc1d9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0c40aff70 0x7fa0c40b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0cc1d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40c5e50 0x7fa0c40c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0cc1d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b0930 con 0x7fa0b00039a0 2023-12-01T01:48:25.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0cc1d9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40c6b80 con 0x7fa0c40aff70 2023-12-01T01:48:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0cc1d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40c6e60 con 0x7fa0c40c5e50 2023-12-01T01:48:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0c9f75640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 0x7fa0c413d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0c9f75640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 0x7fa0c413d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47248/0 (socket says 172.21.15.115:47248) 2023-12-01T01:48:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0c9f75640 1 -- 172.21.15.115:0/927759922 learned_addr learned my addr 172.21.15.115:0/927759922 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0c9774640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0c40aff70 0x7fa0c40b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0ca776640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40c5e50 0x7fa0c40c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0c9f75640 1 -- 172.21.15.115:0/927759922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40c5e50 msgr2=0x7fa0c40c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0c9f75640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40c5e50 0x7fa0c40c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0c9f75640 1 -- 172.21.15.115:0/927759922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0c40aff70 msgr2=0x7fa0c40b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0c9f75640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0c40aff70 0x7fa0c40b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.060+0000 7fa0c9f75640 1 -- 172.21.15.115:0/927759922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40c7140 con 0x7fa0b00039a0 2023-12-01T01:48:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.061+0000 7fa0c9f75640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 0x7fa0c413d4d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c1a0 tx=0x7fa0c0001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.061+0000 7fa0ba7fc640 1 -- 172.21.15.115:0/927759922 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0039070 con 0x7fa0b00039a0 2023-12-01T01:48:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.061+0000 7fa0ba7fc640 1 -- 172.21.15.115:0/927759922 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c002cbf0 con 0x7fa0b00039a0 2023-12-01T01:48:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.061+0000 7fa0ba7fc640 1 -- 172.21.15.115:0/927759922 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0000ed0 con 0x7fa0b00039a0 2023-12-01T01:48:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.061+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/927759922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c40c7420 con 0x7fa0b00039a0 2023-12-01T01:48:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.061+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/927759922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40c6810 con 0x7fa0b00039a0 2023-12-01T01:48:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.062+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/927759922 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c404e890 con 0x7fa0b00039a0 2023-12-01T01:48:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.063+0000 7fa0ba7fc640 1 -- 172.21.15.115:0/927759922 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fa0c0032070 con 0x7fa0b00039a0 2023-12-01T01:48:25.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.064+0000 7fa0ba7fc640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa0a806a630 0x7fa0a806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.064+0000 7fa0ba7fc640 1 -- 172.21.15.115:0/927759922 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fa0c003c040 con 0x7fa0b00039a0 2023-12-01T01:48:25.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.065+0000 7fa0c9774640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa0a806a630 0x7fa0a806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.065+0000 7fa0c9774640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa0a806a630 0x7fa0a806caf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006a90 tx=0x7fa0b4008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:25.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.069+0000 7fa0ba7fc640 1 -- 172.21.15.115:0/927759922 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa0c00753c0 con 0x7fa0b00039a0 2023-12-01T01:48:25.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.287+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/927759922 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0c40ca710 con 0x7fa0a806a630 2023-12-01T01:48:25.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.453+0000 7fa0ba7fc640 1 -- 172.21.15.115:0/927759922 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fa0c40ca710 con 0x7fa0a806a630 2023-12-01T01:48:25.455 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-12-01T01:48:25.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/927759922 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa0a806a630 msgr2=0x7fa0a806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa0a806a630 0x7fa0a806caf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006a90 tx=0x7fa0b4008040 comp rx=0 tx=0).stop 2023-12-01T01:48:25.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/927759922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 msgr2=0x7fa0c413d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 0x7fa0c413d4d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c1a0 tx=0x7fa0c0001ac0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/927759922 shutdown_connections 2023-12-01T01:48:25.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa0a806a630 0x7fa0a806caf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0c40c5e50 0x7fa0c40c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0c40aff70 0x7fa0c40b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 --2- 172.21.15.115:0/927759922 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0b00039a0 0x7fa0c413d4d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/927759922 >> 172.21.15.115:0/927759922 conn(0x7fa0c4065960 msgr2=0x7fa0c413e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:25.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/927759922 shutdown_connections 2023-12-01T01:48:25.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.457+0000 7fa0cc1d9640 1 -- 172.21.15.115:0/927759922 wait complete. 2023-12-01T01:48:25.477 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-12-01T01:48:25.478 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir6 1M 2023-12-01T01:48:25.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.690+0000 7fb8f15d5640 1 Processor -- start 2023-12-01T01:48:25.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.691+0000 7fb8f15d5640 1 -- start start 2023-12-01T01:48:25.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.692+0000 7fb8f15d5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb8ec0b5fa0 0x7fb8ec0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.692+0000 7fb8f15d5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec0ce5e0 con 0x7fb8ec15cd90 2023-12-01T01:48:25.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.692+0000 7fb8f15d5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec0ce300 con 0x7fb8ec0cfd60 2023-12-01T01:48:25.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.692+0000 7fb8f15d5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec0d0ec0 con 0x7fb8ec0b5fa0 2023-12-01T01:48:25.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.692+0000 7fb8eaffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb8ec15cd90 0x7fb8ec0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56198/0 (socket says 172.21.15.115:56198) 2023-12-01T01:48:25.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.692+0000 7fb8eaffd640 1 -- 172.21.15.115:0/1491136390 learned_addr learned my addr 172.21.15.115:0/1491136390 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:25.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8eb7fe640 1 --2- 172.21.15.115:0/1491136390 >> v2:172.21.15.142:3300/0 conn(0x7fb8ec0b5fa0 0x7fb8ec0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423584389 0 0) 0x7fb8dc001110 con 0x7fb8ec15cd90 2023-12-01T01:48:25.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8cc003520 con 0x7fb8ec15cd90 2023-12-01T01:48:25.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4184026956 0 0) 0x7fb8cc003520 con 0x7fb8ec15cd90 2023-12-01T01:48:25.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 >> v2:172.21.15.142:3300/0 conn(0x7fb8ec0b5fa0 msgr2=0x7fb8ec0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 --2- 172.21.15.115:0/1491136390 >> v2:172.21.15.142:3300/0 conn(0x7fb8ec0b5fa0 0x7fb8ec0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 >> v1:172.21.15.142:6789/0 conn(0x7fb8ec0cfd60 legacy=0x7fb8ec0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec0d10a0 con 0x7fb8ec15cd90 2023-12-01T01:48:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb8dc001800 con 0x7fb8ec15cd90 2023-12-01T01:48:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 >> v1:172.21.15.115:6789/0 conn(0x7fb8ec15cd90 legacy=0x7fb8ec0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8cc003970 0x7fb8cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8cc005180 0x7fb8cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0d0ec0 con 0x7fb8cc0047c0 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc00c020 con 0x7fb8cc003970 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.693+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc00c1e0 con 0x7fb8cc005180 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.694+0000 7fb8ea7fc640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.694+0000 7fb8eaffd640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8cc003970 0x7fb8cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.694+0000 7fb8eb7fe640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8cc005180 0x7fb8cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.694+0000 7fb8ea7fc640 1 -- 172.21.15.115:0/1491136390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8cc005180 msgr2=0x7fb8cc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.694+0000 7fb8ea7fc640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8cc005180 0x7fb8cc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.694+0000 7fb8ea7fc640 1 -- 172.21.15.115:0/1491136390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8cc003970 msgr2=0x7fb8cc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.694+0000 7fb8ea7fc640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8cc003970 0x7fb8cc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.694+0000 7fb8ea7fc640 1 -- 172.21.15.115:0/1491136390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc003520 con 0x7fb8cc0047c0 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.694+0000 7fb8eb7fe640 1 -- 172.21.15.115:0/1491136390 reap_dead start 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.695+0000 7fb8ea7fc640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8cc004c40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0d0170 tx=0x7fb8cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e875de3782eec01 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.696+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4003b70 con 0x7fb8cc0047c0 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.696+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d4003cd0 con 0x7fb8cc0047c0 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.696+0000 7fb8e9ffb640 1 -- 172.21.15.115:0/1491136390 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4005310 con 0x7fb8cc0047c0 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.696+0000 7fb8f15d5640 1 -- 172.21.15.115:0/1491136390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 msgr2=0x7fb8cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.696+0000 7fb8f15d5640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8cc004c40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0d0170 tx=0x7fb8cc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.696+0000 7fb8f15d5640 1 -- 172.21.15.115:0/1491136390 shutdown_connections 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.696+0000 7fb8f15d5640 1 --2- 172.21.15.115:0/1491136390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8cc004c40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.696+0000 7fb8f15d5640 1 -- 172.21.15.115:0/1491136390 >> 172.21.15.115:0/1491136390 conn(0x7fb8ec065960 msgr2=0x7fb8ec065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.696+0000 7fb8f15d5640 1 -- 172.21.15.115:0/1491136390 shutdown_connections 2023-12-01T01:48:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.697+0000 7fb8f15d5640 1 -- 172.21.15.115:0/1491136390 wait complete. 2023-12-01T01:48:25.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.698+0000 7fb8f15d5640 1 Processor -- start 2023-12-01T01:48:25.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.698+0000 7fb8f15d5640 1 -- start start 2023-12-01T01:48:25.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.699+0000 7fb8f15d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8ec1e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.699+0000 7fb8f15d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8ec1565b0 0x7fb8ec15f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.699+0000 7fb8f15d5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8ec15f6a0 0x7fb8ec1f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.699+0000 7fb8f15d5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec1e7830 con 0x7fb8cc0047c0 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.699+0000 7fb8f15d5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec1e6c60 con 0x7fb8ec15f6a0 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.699+0000 7fb8f15d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec1e8200 con 0x7fb8ec1565b0 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.700+0000 7fb8eaffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8ec1e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.700+0000 7fb8eaffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8ec1e7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47272/0 (socket says 172.21.15.115:47272) 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.700+0000 7fb8eaffd640 1 -- 172.21.15.115:0/2294820121 learned_addr learned my addr 172.21.15.115:0/2294820121 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.700+0000 7fb8eb7fe640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8ec15f6a0 0x7fb8ec1f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.700+0000 7fb8ea7fc640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8ec1565b0 0x7fb8ec15f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.700+0000 7fb8eaffd640 1 -- 172.21.15.115:0/2294820121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8ec1565b0 msgr2=0x7fb8ec15f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.700+0000 7fb8eaffd640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8ec1565b0 0x7fb8ec15f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.700+0000 7fb8eaffd640 1 -- 172.21.15.115:0/2294820121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8ec15f6a0 msgr2=0x7fb8ec1f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.700+0000 7fb8eaffd640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8ec15f6a0 0x7fb8ec1f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.700+0000 7fb8eaffd640 1 -- 172.21.15.115:0/2294820121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec1e6f00 con 0x7fb8cc0047c0 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.701+0000 7fb8eaffd640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8ec1e7cc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc001370 tx=0x7fb8dc0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.701+0000 7fb8e37fe640 1 -- 172.21.15.115:0/2294820121 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0011a0 con 0x7fb8cc0047c0 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.701+0000 7fb8e37fe640 1 -- 172.21.15.115:0/2294820121 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc00f780 con 0x7fb8cc0047c0 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.701+0000 7fb8e37fe640 1 -- 172.21.15.115:0/2294820121 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00fb30 con 0x7fb8cc0047c0 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.701+0000 7fb8f15d5640 1 -- 172.21.15.115:0/2294820121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec1e71e0 con 0x7fb8cc0047c0 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.701+0000 7fb8f15d5640 1 -- 172.21.15.115:0/2294820121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec156a30 con 0x7fb8cc0047c0 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.702+0000 7fb8f15d5640 1 -- 172.21.15.115:0/2294820121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec04e890 con 0x7fb8cc0047c0 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.703+0000 7fb8e37fe640 1 -- 172.21.15.115:0/2294820121 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fb8dc00fc90 con 0x7fb8cc0047c0 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.704+0000 7fb8e37fe640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d006a660 0x7fb8d006cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.704+0000 7fb8e37fe640 1 -- 172.21.15.115:0/2294820121 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fb8dc088d00 con 0x7fb8cc0047c0 2023-12-01T01:48:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.704+0000 7fb8ea7fc640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d006a660 0x7fb8d006cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.705+0000 7fb8ea7fc640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d006a660 0x7fb8d006cb20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4034000 tx=0x7fb8d4009030 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.708+0000 7fb8e37fe640 1 -- 172.21.15.115:0/2294820121 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb8dc053ee0 con 0x7fb8cc0047c0 2023-12-01T01:48:25.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:25.923+0000 7fb8f15d5640 1 -- 172.21.15.115:0/2294820121 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8ec0d1660 con 0x7fb8d006a660 2023-12-01T01:48:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.092+0000 7fb8e37fe640 1 -- 172.21.15.115:0/2294820121 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb8d403f040 con 0x7fb8d006a660 2023-12-01T01:48:26.094 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-12-01T01:48:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.095+0000 7fb8f15d5640 1 -- 172.21.15.115:0/2294820121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d006a660 msgr2=0x7fb8d006cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.095+0000 7fb8f15d5640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d006a660 0x7fb8d006cb20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4034000 tx=0x7fb8d4009030 comp rx=0 tx=0).stop 2023-12-01T01:48:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.095+0000 7fb8f15d5640 1 -- 172.21.15.115:0/2294820121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 msgr2=0x7fb8ec1e7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.095+0000 7fb8f15d5640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8ec1e7cc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc001370 tx=0x7fb8dc0074f0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.095+0000 7fb8f15d5640 1 -- 172.21.15.115:0/2294820121 shutdown_connections 2023-12-01T01:48:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.095+0000 7fb8f15d5640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d006a660 0x7fb8d006cb20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.095+0000 7fb8f15d5640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8ec1565b0 0x7fb8ec15f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.095+0000 7fb8f15d5640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8ec15f6a0 0x7fb8ec1f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.095+0000 7fb8f15d5640 1 --2- 172.21.15.115:0/2294820121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8cc0047c0 0x7fb8ec1e7cc0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.095+0000 7fb8f15d5640 1 -- 172.21.15.115:0/2294820121 >> 172.21.15.115:0/2294820121 conn(0x7fb8ec065960 msgr2=0x7fb8ec0b7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:26.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.096+0000 7fb8f15d5640 1 -- 172.21.15.115:0/2294820121 shutdown_connections 2023-12-01T01:48:26.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.096+0000 7fb8f15d5640 1 -- 172.21.15.115:0/2294820121 wait complete. 2023-12-01T01:48:26.116 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-12-01T01:48:26.116 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir7 1M 2023-12-01T01:48:26.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.321+0000 7f30e998b640 1 Processor -- start 2023-12-01T01:48:26.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.321+0000 7f30e998b640 1 -- start start 2023-12-01T01:48:26.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.323+0000 7f30e998b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f30e415b0e0 0x7f30e4164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.323+0000 7f30e998b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e40b2f30 con 0x7f30e40cd1e0 2023-12-01T01:48:26.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.323+0000 7f30e998b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e40b2320 con 0x7f30e4142870 2023-12-01T01:48:26.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.323+0000 7f30e998b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e40b3ae0 con 0x7f30e415b0e0 2023-12-01T01:48:26.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.323+0000 7f30dbfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f30e40cd1e0 0x7f30e40b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56200/0 (socket says 172.21.15.115:56200) 2023-12-01T01:48:26.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.323+0000 7f30dbfff640 1 -- 172.21.15.115:0/2300387191 learned_addr learned my addr 172.21.15.115:0/2300387191 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:26.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e37fe640 1 --2- 172.21.15.115:0/2300387191 >> v2:172.21.15.142:3300/0 conn(0x7f30e415b0e0 0x7f30e4164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214426465 0 0) 0x7f30e40b2f30 con 0x7f30e40cd1e0 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30d0005730 con 0x7f30e40cd1e0 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2995224898 0 0) 0x7f30d0005730 con 0x7f30e40cd1e0 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 >> v2:172.21.15.142:3300/0 conn(0x7f30e415b0e0 msgr2=0x7f30e4164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 --2- 172.21.15.115:0/2300387191 >> v2:172.21.15.142:3300/0 conn(0x7f30e415b0e0 0x7f30e4164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 >> v1:172.21.15.142:6789/0 conn(0x7f30e4142870 legacy=0x7f30e40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e40b2600 con 0x7f30e40cd1e0 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f30cc001670 con 0x7f30e40cd1e0 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 >> v1:172.21.15.115:6789/0 conn(0x7f30e40cd1e0 legacy=0x7f30e40b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 --2- 172.21.15.115:0/2300387191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d0005b80 0x7f30d0005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 --2- 172.21.15.115:0/2300387191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d00069d0 0x7f30d0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 --2- 172.21.15.115:0/2300387191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30d000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40b3ae0 con 0x7f30d0007390 2023-12-01T01:48:26.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d000e230 con 0x7f30d00069d0 2023-12-01T01:48:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.324+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d000e3f0 con 0x7f30d0005b80 2023-12-01T01:48:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.325+0000 7f30e37fe640 1 --2- 172.21.15.115:0/2300387191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30d000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.325+0000 7f30dbfff640 1 --2- 172.21.15.115:0/2300387191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d00069d0 0x7f30d0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.325+0000 7f30e37fe640 1 -- 172.21.15.115:0/2300387191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d0005b80 msgr2=0x7f30d0005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.325+0000 7f30e37fe640 1 --2- 172.21.15.115:0/2300387191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d0005b80 0x7f30d0005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.325+0000 7f30e37fe640 1 -- 172.21.15.115:0/2300387191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d00069d0 msgr2=0x7f30d0006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.325+0000 7f30e37fe640 1 --2- 172.21.15.115:0/2300387191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d00069d0 0x7f30d0006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.325+0000 7f30e37fe640 1 -- 172.21.15.115:0/2300387191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d0005730 con 0x7f30d0007390 2023-12-01T01:48:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.326+0000 7f30e37fe640 1 -- 172.21.15.115:0/2300387191 reap_dead start 2023-12-01T01:48:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.326+0000 7f30e37fe640 1 --2- 172.21.15.115:0/2300387191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30d000db90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009fc0 tx=0x7f30d0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c9a6e454b4933b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.326+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc02cc60 con 0x7f30d0007390 2023-12-01T01:48:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.327+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc00f040 con 0x7f30d0007390 2023-12-01T01:48:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.327+0000 7f30e27fc640 1 -- 172.21.15.115:0/2300387191 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc003990 con 0x7f30d0007390 2023-12-01T01:48:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.327+0000 7f30e998b640 1 -- 172.21.15.115:0/2300387191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 msgr2=0x7f30d000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.327+0000 7f30e998b640 1 --2- 172.21.15.115:0/2300387191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30d000db90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009fc0 tx=0x7f30d0007a00 comp rx=0 tx=0).stop 2023-12-01T01:48:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.327+0000 7f30e998b640 1 -- 172.21.15.115:0/2300387191 shutdown_connections 2023-12-01T01:48:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.327+0000 7f30e998b640 1 --2- 172.21.15.115:0/2300387191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30d000db90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.327+0000 7f30e998b640 1 -- 172.21.15.115:0/2300387191 >> 172.21.15.115:0/2300387191 conn(0x7f30e4065960 msgr2=0x7f30e4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.327+0000 7f30e998b640 1 -- 172.21.15.115:0/2300387191 shutdown_connections 2023-12-01T01:48:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.327+0000 7f30e998b640 1 -- 172.21.15.115:0/2300387191 wait complete. 2023-12-01T01:48:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.329+0000 7f30e998b640 1 Processor -- start 2023-12-01T01:48:26.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.329+0000 7f30e998b640 1 -- start start 2023-12-01T01:48:26.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.330+0000 7f30e998b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30e413cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.330+0000 7f30e998b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e413d050 0x7f30e40cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.330+0000 7f30e998b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30e40cb6d0 0x7f30e40cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.330+0000 7f30e998b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e412a400 con 0x7f30d0007390 2023-12-01T01:48:26.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.330+0000 7f30e998b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e413d4d0 con 0x7f30e413d050 2023-12-01T01:48:26.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.330+0000 7f30e998b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4129780 con 0x7f30e40cb6d0 2023-12-01T01:48:26.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.330+0000 7f30e2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30e413cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.330+0000 7f30e2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30e413cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47284/0 (socket says 172.21.15.115:47284) 2023-12-01T01:48:26.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.330+0000 7f30e2ffd640 1 -- 172.21.15.115:0/2424741728 learned_addr learned my addr 172.21.15.115:0/2424741728 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e37fe640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30e40cb6d0 0x7f30e40cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30dbfff640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e413d050 0x7f30e40cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e2ffd640 1 -- 172.21.15.115:0/2424741728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30e40cb6d0 msgr2=0x7f30e40cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e2ffd640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30e40cb6d0 0x7f30e40cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e2ffd640 1 -- 172.21.15.115:0/2424741728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e413d050 msgr2=0x7f30e40cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e2ffd640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e413d050 0x7f30e40cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e2ffd640 1 -- 172.21.15.115:0/2424741728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e4129a60 con 0x7f30d0007390 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e2ffd640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30e413cb10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f30d400c450 tx=0x7f30d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e0ff9640 1 -- 172.21.15.115:0/2424741728 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d40052d0 con 0x7f30d0007390 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e0ff9640 1 -- 172.21.15.115:0/2424741728 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d4005430 con 0x7f30d0007390 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e998b640 1 -- 172.21.15.115:0/2424741728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e4129d40 con 0x7f30d0007390 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.331+0000 7f30e0ff9640 1 -- 172.21.15.115:0/2424741728 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4003830 con 0x7f30d0007390 2023-12-01T01:48:26.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.332+0000 7f30e998b640 1 -- 172.21.15.115:0/2424741728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e4156bc0 con 0x7f30d0007390 2023-12-01T01:48:26.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.333+0000 7f30e998b640 1 -- 172.21.15.115:0/2424741728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e404e890 con 0x7f30d0007390 2023-12-01T01:48:26.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.334+0000 7f30e0ff9640 1 -- 172.21.15.115:0/2424741728 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f30d4003990 con 0x7f30d0007390 2023-12-01T01:48:26.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.334+0000 7f30e0ff9640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30c406a630 0x7f30c406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.335+0000 7f30e0ff9640 1 -- 172.21.15.115:0/2424741728 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f30d40883b0 con 0x7f30d0007390 2023-12-01T01:48:26.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.335+0000 7f30dbfff640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30c406a630 0x7f30c406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.335+0000 7f30dbfff640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30c406a630 0x7f30c406caf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30cc001eb0 tx=0x7f30cc000f30 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:26.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.339+0000 7f30e0ff9640 1 -- 172.21.15.115:0/2424741728 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f30d4053630 con 0x7f30d0007390 2023-12-01T01:48:26.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.547+0000 7f30e998b640 1 -- 172.21.15.115:0/2424741728 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f30e40b28e0 con 0x7f30c406a630 2023-12-01T01:48:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.710+0000 7f30e0ff9640 1 -- 172.21.15.115:0/2424741728 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f30cc001490 con 0x7f30c406a630 2023-12-01T01:48:26.712 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 -- 172.21.15.115:0/2424741728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30c406a630 msgr2=0x7f30c406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30c406a630 0x7f30c406caf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30cc001eb0 tx=0x7f30cc000f30 comp rx=0 tx=0).stop 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 -- 172.21.15.115:0/2424741728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 msgr2=0x7f30e413cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30e413cb10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f30d400c450 tx=0x7f30d400c910 comp rx=0 tx=0).stop 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 -- 172.21.15.115:0/2424741728 shutdown_connections 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f30c406a630 0x7f30c406caf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30e40cb6d0 0x7f30e40cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e413d050 0x7f30e40cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 --2- 172.21.15.115:0/2424741728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d0007390 0x7f30e413cb10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 -- 172.21.15.115:0/2424741728 >> 172.21.15.115:0/2424741728 conn(0x7f30e4065960 msgr2=0x7f30e406f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 -- 172.21.15.115:0/2424741728 shutdown_connections 2023-12-01T01:48:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.713+0000 7f30e998b640 1 -- 172.21.15.115:0/2424741728 wait complete. 2023-12-01T01:48:26.733 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-12-01T01:48:26.733 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir8 1M 2023-12-01T01:48:26.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.938+0000 7fe10b869640 1 Processor -- start 2023-12-01T01:48:26.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.938+0000 7fe10b869640 1 -- start start 2023-12-01T01:48:26.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.939+0000 7fe10b869640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe104142870 0x7fe1040ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.939+0000 7fe10b869640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 0x7fe1040b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.940+0000 7fe10b869640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1040cd470 con 0x7fe1040b1c50 2023-12-01T01:48:26.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.940+0000 7fe10b869640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1040b3ae0 con 0x7fe104142870 2023-12-01T01:48:26.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.940+0000 7fe10b869640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe104142c50 con 0x7fe1040cd010 2023-12-01T01:48:26.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.940+0000 7fe109605640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe104142870 0x7fe1040ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.940+0000 7fe108e04640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 0x7fe1040b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.940+0000 7fe109605640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe104142870 0x7fe1040ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:36264/0 (socket says 172.21.15.115:36264) 2023-12-01T01:48:26.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.940+0000 7fe109605640 1 -- 172.21.15.115:0/4151857011 learned_addr learned my addr 172.21.15.115:0/4151857011 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:26.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.941+0000 7fe109605640 1 -- 172.21.15.115:0/4151857011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 msgr2=0x7fe1040b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:26.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.941+0000 7fe109605640 1 --2- 172.21.15.115:0/4151857011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 0x7fe1040b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.941+0000 7fe109605640 1 -- 172.21.15.115:0/4151857011 >> v1:172.21.15.142:6789/0 conn(0x7fe1040b1c50 legacy=0x7fe10415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.941+0000 7fe109605640 1 -- 172.21.15.115:0/4151857011 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040b20e0 con 0x7fe104142870 2023-12-01T01:48:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.943+0000 7fe109605640 1 --2- 172.21.15.115:0/4151857011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe104142870 0x7fe1040ccad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80098a0 tx=0x7fe0f802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10a387be7df913a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.943+0000 7fe0f7fff640 1 -- 172.21.15.115:0/4151857011 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8030840 con 0x7fe104142870 2023-12-01T01:48:26.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.943+0000 7fe0f7fff640 1 -- 172.21.15.115:0/4151857011 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80309a0 con 0x7fe104142870 2023-12-01T01:48:26.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.943+0000 7fe0f7fff640 1 -- 172.21.15.115:0/4151857011 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8030d30 con 0x7fe104142870 2023-12-01T01:48:26.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.944+0000 7fe10b869640 1 -- 172.21.15.115:0/4151857011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe104142870 msgr2=0x7fe1040ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.944+0000 7fe10b869640 1 --2- 172.21.15.115:0/4151857011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe104142870 0x7fe1040ccad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80098a0 tx=0x7fe0f802b980 comp rx=0 tx=0).stop 2023-12-01T01:48:26.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.944+0000 7fe10b869640 1 -- 172.21.15.115:0/4151857011 shutdown_connections 2023-12-01T01:48:26.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.944+0000 7fe10b869640 1 --2- 172.21.15.115:0/4151857011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 0x7fe1040b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.944+0000 7fe10b869640 1 --2- 172.21.15.115:0/4151857011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe104142870 0x7fe1040ccad0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.944+0000 7fe10b869640 1 -- 172.21.15.115:0/4151857011 >> 172.21.15.115:0/4151857011 conn(0x7fe104065960 msgr2=0x7fe104065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:26.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.944+0000 7fe10b869640 1 -- 172.21.15.115:0/4151857011 shutdown_connections 2023-12-01T01:48:26.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.944+0000 7fe10b869640 1 -- 172.21.15.115:0/4151857011 wait complete. 2023-12-01T01:48:26.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.946+0000 7fe10b869640 1 Processor -- start 2023-12-01T01:48:26.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.947+0000 7fe10b869640 1 -- start start 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe10b869640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1040b1c50 0x7fe1040ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe10b869640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 0x7fe1040abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe10b869640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe104142870 0x7fe1041582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe10b869640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040ad700 con 0x7fe1040b1c50 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe10b869640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040ac660 con 0x7fe104142870 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe10b869640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040ac940 con 0x7fe1040cd010 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe109605640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1040b1c50 0x7fe1040ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe109605640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1040b1c50 0x7fe1040ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47288/0 (socket says 172.21.15.115:47288) 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe109605640 1 -- 172.21.15.115:0/3086146069 learned_addr learned my addr 172.21.15.115:0/3086146069 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe108e04640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 0x7fe1040abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe109605640 1 -- 172.21.15.115:0/3086146069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 msgr2=0x7fe1040abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe109605640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 0x7fe1040abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe109605640 1 -- 172.21.15.115:0/3086146069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe104142870 msgr2=0x7fe1041582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.948+0000 7fe109e06640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe104142870 0x7fe1041582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.949+0000 7fe109605640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe104142870 0x7fe1041582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.949+0000 7fe109605640 1 -- 172.21.15.115:0/3086146069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80094d0 con 0x7fe1040b1c50 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.949+0000 7fe108e04640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 0x7fe1040abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.949+0000 7fe109605640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1040b1c50 0x7fe1040ae830 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80097f0 tx=0x7fe0f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.949+0000 7fe0f5ffb640 1 -- 172.21.15.115:0/3086146069 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8039a90 con 0x7fe1040b1c50 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.949+0000 7fe0f5ffb640 1 -- 172.21.15.115:0/3086146069 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8039bf0 con 0x7fe1040b1c50 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.949+0000 7fe0f5ffb640 1 -- 172.21.15.115:0/3086146069 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80433f0 con 0x7fe1040b1c50 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.949+0000 7fe10b869640 1 -- 172.21.15.115:0/3086146069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1040acc20 con 0x7fe1040b1c50 2023-12-01T01:48:26.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.949+0000 7fe10b869640 1 -- 172.21.15.115:0/3086146069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1040ad1e0 con 0x7fe1040b1c50 2023-12-01T01:48:26.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.950+0000 7fe10b869640 1 -- 172.21.15.115:0/3086146069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10404e890 con 0x7fe1040b1c50 2023-12-01T01:48:26.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.951+0000 7fe0f5ffb640 1 -- 172.21.15.115:0/3086146069 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fe0f802f390 con 0x7fe1040b1c50 2023-12-01T01:48:26.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.952+0000 7fe0f5ffb640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe0e806a630 0x7fe0e806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:26.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.952+0000 7fe0f5ffb640 1 -- 172.21.15.115:0/3086146069 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fe0f80aae20 con 0x7fe1040b1c50 2023-12-01T01:48:26.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.952+0000 7fe108e04640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe0e806a630 0x7fe0e806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:26.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.953+0000 7fe108e04640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe0e806a630 0x7fe0e806caf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe1041f6c40 tx=0x7fe100008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:26.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:26.956+0000 7fe0f5ffb640 1 -- 172.21.15.115:0/3086146069 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe0f8075ef0 con 0x7fe1040b1c50 2023-12-01T01:48:27.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.169+0000 7fe10b869640 1 -- 172.21.15.115:0/3086146069 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1040b2320 con 0x7fe0e806a630 2023-12-01T01:48:27.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.343+0000 7fe0f5ffb640 1 -- 172.21.15.115:0/3086146069 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe100009750 con 0x7fe0e806a630 2023-12-01T01:48:27.345 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-12-01T01:48:27.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 -- 172.21.15.115:0/3086146069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe0e806a630 msgr2=0x7fe0e806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:27.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe0e806a630 0x7fe0e806caf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe1041f6c40 tx=0x7fe100008040 comp rx=0 tx=0).stop 2023-12-01T01:48:27.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 -- 172.21.15.115:0/3086146069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1040b1c50 msgr2=0x7fe1040ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:27.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1040b1c50 0x7fe1040ae830 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80097f0 tx=0x7fe0f8004730 comp rx=0 tx=0).stop 2023-12-01T01:48:27.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 -- 172.21.15.115:0/3086146069 shutdown_connections 2023-12-01T01:48:27.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe0e806a630 0x7fe0e806caf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1040cd010 0x7fe1040abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe104142870 0x7fe1041582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 --2- 172.21.15.115:0/3086146069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1040b1c50 0x7fe1040ae830 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 -- 172.21.15.115:0/3086146069 >> 172.21.15.115:0/3086146069 conn(0x7fe104065960 msgr2=0x7fe104140e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:27.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 -- 172.21.15.115:0/3086146069 shutdown_connections 2023-12-01T01:48:27.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.347+0000 7fe10b869640 1 -- 172.21.15.115:0/3086146069 wait complete. 2023-12-01T01:48:27.370 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-12-01T01:48:27.370 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir9 1M 2023-12-01T01:48:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.577+0000 7fef14921640 1 Processor -- start 2023-12-01T01:48:27.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.577+0000 7fef14921640 1 -- start start 2023-12-01T01:48:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.579+0000 7fef14921640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015ac00 0x7fef100ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:27.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.579+0000 7fef14921640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1015afe0 con 0x7fef100aede0 2023-12-01T01:48:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.579+0000 7fef14921640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef100b0a10 con 0x7fef100cb800 2023-12-01T01:48:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.579+0000 7fef14921640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef100b12b0 con 0x7fef1015ac00 2023-12-01T01:48:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.579+0000 7fef0e59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fef100aede0 0x7fef100cb0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56202/0 (socket says 172.21.15.115:56202) 2023-12-01T01:48:27.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.579+0000 7fef0e59c640 1 -- 172.21.15.115:0/3809114784 learned_addr learned my addr 172.21.15.115:0/3809114784 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083426019 0 0) 0x7fef1015afe0 con 0x7fef100aede0 2023-12-01T01:48:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feef8003520 con 0x7fef100aede0 2023-12-01T01:48:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0ed9d640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015ac00 0x7fef100ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2411064145 0 0) 0x7feef8003520 con 0x7fef100aede0 2023-12-01T01:48:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015ac00 msgr2=0x7fef100ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015ac00 0x7fef100ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 >> v1:172.21.15.142:6789/0 conn(0x7fef100cb800 legacy=0x7fef1006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef100b0cf0 con 0x7fef100aede0 2023-12-01T01:48:27.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fef04001670 con 0x7fef100aede0 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 >> v1:172.21.15.115:6789/0 conn(0x7fef100aede0 legacy=0x7fef100cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feef80039a0 0x7feef8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 0x7feef8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015ac00 0x7feef800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef100b12b0 con 0x7feef80047c0 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef80054e0 con 0x7feef80039a0 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef800bec0 con 0x7fef1015ac00 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.580+0000 7fef0e59c640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 0x7feef8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.581+0000 7fef0ed9d640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feef80039a0 0x7feef8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.581+0000 7fef0f59e640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015ac00 0x7feef800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:27.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.581+0000 7fef0e59c640 1 -- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015ac00 msgr2=0x7feef800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.581+0000 7fef0e59c640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015ac00 0x7feef800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.581+0000 7fef0e59c640 1 -- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feef80039a0 msgr2=0x7feef8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.581+0000 7fef0e59c640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feef80039a0 0x7feef8003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.581+0000 7fef0e59c640 1 -- 172.21.15.115:0/3809114784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef8003520 con 0x7feef80047c0 2023-12-01T01:48:27.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.582+0000 7fef0e59c640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 0x7feef8004c40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fef0400c240 tx=0x7feef80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333c16728edbcc73 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:27.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.582+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04005ad0 con 0x7feef80047c0 2023-12-01T01:48:27.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.582+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef04005c30 con 0x7feef80047c0 2023-12-01T01:48:27.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.582+0000 7fef0dd9b640 1 -- 172.21.15.115:0/3809114784 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0402c8b0 con 0x7feef80047c0 2023-12-01T01:48:27.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.583+0000 7fef14921640 1 -- 172.21.15.115:0/3809114784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 msgr2=0x7feef8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:27.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.583+0000 7fef14921640 1 --2- 172.21.15.115:0/3809114784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 0x7feef8004c40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fef0400c240 tx=0x7feef80057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.583+0000 7fef0f59e640 1 -- 172.21.15.115:0/3809114784 reap_dead start 2023-12-01T01:48:27.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.583+0000 7fef14921640 1 -- 172.21.15.115:0/3809114784 shutdown_connections 2023-12-01T01:48:27.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.583+0000 7fef14921640 1 -- 172.21.15.115:0/3809114784 >> 172.21.15.115:0/3809114784 conn(0x7fef10065960 msgr2=0x7fef10065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:27.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.583+0000 7fef14921640 1 -- 172.21.15.115:0/3809114784 shutdown_connections 2023-12-01T01:48:27.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.583+0000 7fef14921640 1 -- 172.21.15.115:0/3809114784 wait complete. 2023-12-01T01:48:27.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.585+0000 7fef14921640 1 Processor -- start 2023-12-01T01:48:27.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.585+0000 7fef14921640 1 -- start start 2023-12-01T01:48:27.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef14921640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 0x7fef1014da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:27.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef14921640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1015b1f0 0x7fef1015b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:27.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef14921640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015bbb0 0x7fef101f1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef14921640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1015c060 con 0x7feef80047c0 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef14921640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef100b1520 con 0x7fef1015b1f0 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef14921640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1014df60 con 0x7fef1015bbb0 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef0ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 0x7fef1014da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef0ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 0x7fef1014da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47310/0 (socket says 172.21.15.115:47310) 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef0ed9d640 1 -- 172.21.15.115:0/2905431865 learned_addr learned my addr 172.21.15.115:0/2905431865 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef0f59e640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015bbb0 0x7fef101f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef0e59c640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1015b1f0 0x7fef1015b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef0ed9d640 1 -- 172.21.15.115:0/2905431865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015bbb0 msgr2=0x7fef101f1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef0ed9d640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015bbb0 0x7fef101f1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef0ed9d640 1 -- 172.21.15.115:0/2905431865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1015b1f0 msgr2=0x7fef1015b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef0ed9d640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1015b1f0 0x7fef1015b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.586+0000 7fef0ed9d640 1 -- 172.21.15.115:0/2905431865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef101564c0 con 0x7feef80047c0 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.587+0000 7fef0ed9d640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 0x7fef1014da20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fef0000e7e0 tx=0x7fef0000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:27.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.587+0000 7feef77fe640 1 -- 172.21.15.115:0/2905431865 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0000c670 con 0x7feef80047c0 2023-12-01T01:48:27.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.587+0000 7feef77fe640 1 -- 172.21.15.115:0/2905431865 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0000c7d0 con 0x7feef80047c0 2023-12-01T01:48:27.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.587+0000 7feef77fe640 1 -- 172.21.15.115:0/2905431865 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0000cb40 con 0x7feef80047c0 2023-12-01T01:48:27.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.587+0000 7fef14921640 1 -- 172.21.15.115:0/2905431865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef101567a0 con 0x7feef80047c0 2023-12-01T01:48:27.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.587+0000 7fef14921640 1 -- 172.21.15.115:0/2905431865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef10156d60 con 0x7feef80047c0 2023-12-01T01:48:27.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.588+0000 7fef14921640 1 -- 172.21.15.115:0/2905431865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef100618d0 con 0x7feef80047c0 2023-12-01T01:48:27.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.589+0000 7feef77fe640 1 -- 172.21.15.115:0/2905431865 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fef0000cca0 con 0x7feef80047c0 2023-12-01T01:48:27.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.590+0000 7feef77fe640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feefc06a660 0x7feefc06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:27.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.590+0000 7feef77fe640 1 -- 172.21.15.115:0/2905431865 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fef00088dc0 con 0x7feef80047c0 2023-12-01T01:48:27.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.590+0000 7fef0e59c640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feefc06a660 0x7feefc06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:27.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.591+0000 7fef0e59c640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feefc06a660 0x7feefc06cb20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef04034000 tx=0x7fef04009030 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:27.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.594+0000 7feef77fe640 1 -- 172.21.15.115:0/2905431865 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fef00053e00 con 0x7feef80047c0 2023-12-01T01:48:27.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.820+0000 7fef14921640 1 -- 172.21.15.115:0/2905431865 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fef100b0fd0 con 0x7feefc06a660 2023-12-01T01:48:27.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.991+0000 7feef77fe640 1 -- 172.21.15.115:0/2905431865 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fef04005da0 con 0x7feefc06a660 2023-12-01T01:48:27.993 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-12-01T01:48:27.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.995+0000 7fef14921640 1 -- 172.21.15.115:0/2905431865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feefc06a660 msgr2=0x7feefc06cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:27.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.995+0000 7fef14921640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feefc06a660 0x7feefc06cb20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef04034000 tx=0x7fef04009030 comp rx=0 tx=0).stop 2023-12-01T01:48:27.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.995+0000 7fef14921640 1 -- 172.21.15.115:0/2905431865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 msgr2=0x7fef1014da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:27.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.995+0000 7fef14921640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 0x7fef1014da20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fef0000e7e0 tx=0x7fef0000eca0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.995+0000 7fef14921640 1 -- 172.21.15.115:0/2905431865 shutdown_connections 2023-12-01T01:48:27.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.995+0000 7fef14921640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feefc06a660 0x7feefc06cb20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.995+0000 7fef14921640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1015bbb0 0x7fef101f1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.995+0000 7fef14921640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef1015b1f0 0x7fef1015b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.995+0000 7fef14921640 1 --2- 172.21.15.115:0/2905431865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feef80047c0 0x7fef1014da20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:27.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.995+0000 7fef14921640 1 -- 172.21.15.115:0/2905431865 >> 172.21.15.115:0/2905431865 conn(0x7fef10065960 msgr2=0x7fef10157710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:27.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.996+0000 7fef14921640 1 -- 172.21.15.115:0/2905431865 shutdown_connections 2023-12-01T01:48:27.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:27.996+0000 7fef14921640 1 -- 172.21.15.115:0/2905431865 wait complete. 2023-12-01T01:48:28.018 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-12-01T01:48:28.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:28.018 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-12-01T01:48:28.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:48:28.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:48:28.081 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:28.081 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_01_UTC 2023-12-01T01:48:28.081 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_01_UTC 2023-12-01T01:48:28.081 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_01_UTC 2023-12-01T01:48:28.081 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_01_UTC 2023-12-01T01:48:28.081 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_01_UTC 2023-12-01T01:48:28.081 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:28.081 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:28.081 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:28.082 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:28.082 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:28.082 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_38_00_UTC', 'scheduled-2023-12-01-01_39_01_UTC', 'scheduled-2023-12-01-01_40_01_UTC', 'scheduled-2023-12-01-01_41_01_UTC', 'scheduled-2023-12-01-01_42_01_UTC', 'scheduled-2023-12-01-01_43_01_UTC', 'scheduled-2023-12-01-01_44_01_UTC', 'scheduled-2023-12-01-01_45_01_UTC', 'scheduled-2023-12-01-01_46_01_UTC', 'scheduled-2023-12-01-01_47_00_UTC', 'scheduled-2023-12-01-01_48_00_UTC'] 2023-12-01T01:48:28.082 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir0 2023-12-01T01:48:28.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.315+0000 7ff5b545f640 1 Processor -- start 2023-12-01T01:48:28.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.315+0000 7ff5b545f640 1 -- start start 2023-12-01T01:48:28.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.317+0000 7ff5b545f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5b01590b0 0x7ff5b0159490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.317+0000 7ff5b545f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff5b006baf0 0x7ff5b006bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.317+0000 7ff5b545f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b00cac30 con 0x7ff5b01599d0 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.317+0000 7ff5b545f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b006c4e0 con 0x7ff5b01590b0 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.317+0000 7ff5b545f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b00b2140 con 0x7ff5b006baf0 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.317+0000 7ff5aeffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5b01590b0 0x7ff5b0159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.317+0000 7ff5aeffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5b01590b0 0x7ff5b0159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36464/0 (socket says 172.21.15.115:36464) 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.317+0000 7ff5aeffd640 1 -- 172.21.15.115:0/1677945183 learned_addr learned my addr 172.21.15.115:0/1677945183 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.317+0000 7ff5ae7fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff5b01599d0 0x7ff5b00b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59140/0 (socket says 172.21.15.115:59140) 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.318+0000 7ff5af7fe640 1 --2- 172.21.15.115:0/1677945183 >> v2:172.21.15.142:3300/0 conn(0x7ff5b006baf0 0x7ff5b006bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.318+0000 7ff5aeffd640 1 -- 172.21.15.115:0/1677945183 >> v2:172.21.15.142:3300/0 conn(0x7ff5b006baf0 msgr2=0x7ff5b006bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.318+0000 7ff5aeffd640 1 --2- 172.21.15.115:0/1677945183 >> v2:172.21.15.142:3300/0 conn(0x7ff5b006baf0 0x7ff5b006bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.318+0000 7ff5aeffd640 1 -- 172.21.15.115:0/1677945183 >> v1:172.21.15.115:6789/0 conn(0x7ff5b01599d0 legacy=0x7ff5b00b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:28.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.318+0000 7ff5aeffd640 1 -- 172.21.15.115:0/1677945183 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00b2420 con 0x7ff5b01590b0 2023-12-01T01:48:28.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.318+0000 7ff5adffb640 1 -- 172.21.15.115:0/1677945183 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218033519 0 0) 0x7ff5b00cac30 con 0x7ff5b01599d0 2023-12-01T01:48:28.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.318+0000 7ff5af7fe640 1 --2- 172.21.15.115:0/1677945183 >> v2:172.21.15.142:3300/0 conn(0x7ff5b006baf0 0x7ff5b006bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:28.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.319+0000 7ff5aeffd640 1 --2- 172.21.15.115:0/1677945183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 0x7ff5b0159490 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00ce770 tx=0x7ff5a002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4507480cadfa8684 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:28.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.319+0000 7ff5adffb640 1 -- 172.21.15.115:0/1677945183 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0030860 con 0x7ff5b01590b0 2023-12-01T01:48:28.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.319+0000 7ff5adffb640 1 -- 172.21.15.115:0/1677945183 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a00309c0 con 0x7ff5b01590b0 2023-12-01T01:48:28.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.320+0000 7ff5adffb640 1 -- 172.21.15.115:0/1677945183 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0030d50 con 0x7ff5b01590b0 2023-12-01T01:48:28.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.320+0000 7ff5b545f640 1 -- 172.21.15.115:0/1677945183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 msgr2=0x7ff5b0159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:28.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.320+0000 7ff5b545f640 1 --2- 172.21.15.115:0/1677945183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 0x7ff5b0159490 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00ce770 tx=0x7ff5a002b9a0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.320+0000 7ff5b545f640 1 -- 172.21.15.115:0/1677945183 shutdown_connections 2023-12-01T01:48:28.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.320+0000 7ff5b545f640 1 --2- 172.21.15.115:0/1677945183 >> v2:172.21.15.142:3300/0 conn(0x7ff5b006baf0 0x7ff5b006bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.320+0000 7ff5b545f640 1 --2- 172.21.15.115:0/1677945183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 0x7ff5b0159490 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.320+0000 7ff5b545f640 1 -- 172.21.15.115:0/1677945183 >> 172.21.15.115:0/1677945183 conn(0x7ff5b0065960 msgr2=0x7ff5b0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:28.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.320+0000 7ff5b545f640 1 -- 172.21.15.115:0/1677945183 shutdown_connections 2023-12-01T01:48:28.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.320+0000 7ff5b545f640 1 -- 172.21.15.115:0/1677945183 wait complete. 2023-12-01T01:48:28.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.322+0000 7ff5b545f640 1 Processor -- start 2023-12-01T01:48:28.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.322+0000 7ff5b545f640 1 -- start start 2023-12-01T01:48:28.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.323+0000 7ff5b545f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5b006baf0 0x7ff5b013f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.323+0000 7ff5b545f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 0x7ff5b014dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.323+0000 7ff5b545f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5b01599d0 0x7ff5b014bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.323+0000 7ff5b545f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b014c620 con 0x7ff5b01590b0 2023-12-01T01:48:28.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.323+0000 7ff5b545f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b014c900 con 0x7ff5b006baf0 2023-12-01T01:48:28.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.323+0000 7ff5b545f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0148d90 con 0x7ff5b01599d0 2023-12-01T01:48:28.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.323+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 0x7ff5b014dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:28.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5ae7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 0x7ff5b014dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36466/0 (socket says 172.21.15.115:36466) 2023-12-01T01:48:28.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5ae7fc640 1 -- 172.21.15.115:0/1383330962 learned_addr learned my addr 172.21.15.115:0/1383330962 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:28.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5aeffd640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5b006baf0 0x7ff5b013f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:28.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5ae7fc640 1 -- 172.21.15.115:0/1383330962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5b01599d0 msgr2=0x7ff5b014bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:28.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5ae7fc640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5b01599d0 0x7ff5b014bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5ae7fc640 1 -- 172.21.15.115:0/1383330962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5b006baf0 msgr2=0x7ff5b013f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:28.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5ae7fc640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5b006baf0 0x7ff5b013f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5ae7fc640 1 -- 172.21.15.115:0/1383330962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a00094d0 con 0x7ff5b01590b0 2023-12-01T01:48:28.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5ae7fc640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 0x7ff5b014dc70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c4b0 tx=0x7ff59800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:28.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5a77fe640 1 -- 172.21.15.115:0/1383330962 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5980074e0 con 0x7ff5b01590b0 2023-12-01T01:48:28.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5a77fe640 1 -- 172.21.15.115:0/1383330962 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff598007660 con 0x7ff5b01590b0 2023-12-01T01:48:28.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.324+0000 7ff5b545f640 1 -- 172.21.15.115:0/1383330962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b0149070 con 0x7ff5b01590b0 2023-12-01T01:48:28.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.325+0000 7ff5a77fe640 1 -- 172.21.15.115:0/1383330962 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598004a20 con 0x7ff5b01590b0 2023-12-01T01:48:28.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.325+0000 7ff5b545f640 1 -- 172.21.15.115:0/1383330962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b014e1b0 con 0x7ff5b01590b0 2023-12-01T01:48:28.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.326+0000 7ff5b545f640 1 -- 172.21.15.115:0/1383330962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b0148bf0 con 0x7ff5b01590b0 2023-12-01T01:48:28.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.327+0000 7ff5a77fe640 1 -- 172.21.15.115:0/1383330962 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7ff598004b80 con 0x7ff5b01590b0 2023-12-01T01:48:28.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.327+0000 7ff5a77fe640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a630 0x7ff59006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.328+0000 7ff5a77fe640 1 -- 172.21.15.115:0/1383330962 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7ff598088400 con 0x7ff5b01590b0 2023-12-01T01:48:28.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.328+0000 7ff5aeffd640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a630 0x7ff59006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:28.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.329+0000 7ff5aeffd640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a630 0x7ff59006caf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0036000 tx=0x7ff5a003b040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:28.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.332+0000 7ff5a77fe640 1 -- 172.21.15.115:0/1383330962 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff5b0148bf0 con 0x7ff5b01590b0 2023-12-01T01:48:28.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.542+0000 7ff5b545f640 1 -- 172.21.15.115:0/1383330962 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5b00caed0 con 0x7ff59006a630 2023-12-01T01:48:28.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.560+0000 7ff5a77fe640 1 -- 172.21.15.115:0/1383330962 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7ff5a0030c30 con 0x7ff59006a630 2023-12-01T01:48:28.562 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:28.562 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:38", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.562+0000 7ff5b545f640 1 -- 172.21.15.115:0/1383330962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a630 msgr2=0x7ff59006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.562+0000 7ff5b545f640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a630 0x7ff59006caf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0036000 tx=0x7ff5a003b040 comp rx=0 tx=0).stop 2023-12-01T01:48:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.562+0000 7ff5b545f640 1 -- 172.21.15.115:0/1383330962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 msgr2=0x7ff5b014dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.562+0000 7ff5b545f640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 0x7ff5b014dc70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c4b0 tx=0x7ff59800c970 comp rx=0 tx=0).stop 2023-12-01T01:48:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.563+0000 7ff5b545f640 1 -- 172.21.15.115:0/1383330962 shutdown_connections 2023-12-01T01:48:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.563+0000 7ff5b545f640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a630 0x7ff59006caf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.563+0000 7ff5b545f640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5b01599d0 0x7ff5b014bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.563+0000 7ff5b545f640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5b01590b0 0x7ff5b014dc70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.563+0000 7ff5b545f640 1 --2- 172.21.15.115:0/1383330962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5b006baf0 0x7ff5b013f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.563+0000 7ff5b545f640 1 -- 172.21.15.115:0/1383330962 >> 172.21.15.115:0/1383330962 conn(0x7ff5b0065960 msgr2=0x7ff5b00cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:28.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.563+0000 7ff5b545f640 1 -- 172.21.15.115:0/1383330962 shutdown_connections 2023-12-01T01:48:28.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.563+0000 7ff5b545f640 1 -- 172.21.15.115:0/1383330962 wait complete. 2023-12-01T01:48:28.582 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:38", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:28.582 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:37:38', 'first': '2023-12-01T01:38:00', 'last': '2023-12-01T01:48:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-01T01:48:28.582 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T01:48:28.582 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T01:48:28.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:28.582 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-12-01T01:48:28.634 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_01_UTC 2023-12-01T01:48:28.634 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_01_UTC 2023-12-01T01:48:28.634 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:28.635 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:28.635 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_01_UTC 2023-12-01T01:48:28.635 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_01_UTC 2023-12-01T01:48:28.635 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:28.635 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:28.635 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:28.635 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:28.635 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:28.636 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_38_01_UTC', 'scheduled-2023-12-01-01_39_01_UTC', 'scheduled-2023-12-01-01_40_00_UTC', 'scheduled-2023-12-01-01_41_00_UTC', 'scheduled-2023-12-01-01_42_01_UTC', 'scheduled-2023-12-01-01_43_01_UTC', 'scheduled-2023-12-01-01_44_01_UTC', 'scheduled-2023-12-01-01_45_01_UTC', 'scheduled-2023-12-01-01_46_01_UTC', 'scheduled-2023-12-01-01_47_00_UTC', 'scheduled-2023-12-01-01_48_00_UTC'] 2023-12-01T01:48:28.636 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir1 2023-12-01T01:48:28.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.847+0000 7f485318e640 1 Processor -- start 2023-12-01T01:48:28.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.848+0000 7f485318e640 1 -- start start 2023-12-01T01:48:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.849+0000 7f485318e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f484c143eb0 0x7f484c06b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.849+0000 7f485318e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f484c06b7a0 0x7f484c0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.849+0000 7f485318e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484c0cbeb0 con 0x7f484c0cbab0 2023-12-01T01:48:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.849+0000 7f485318e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484c0ce1b0 con 0x7f484c06b7a0 2023-12-01T01:48:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.849+0000 7f485318e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484c0ce490 con 0x7f484c143eb0 2023-12-01T01:48:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.850+0000 7f484bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f484c06b7a0 0x7f484c0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.850+0000 7f485172b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f484c0cbab0 0x7f484c164c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59154/0 (socket says 172.21.15.115:59154) 2023-12-01T01:48:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.850+0000 7f485172b640 1 -- 172.21.15.115:0/2148106453 learned_addr learned my addr 172.21.15.115:0/2148106453 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:28.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.850+0000 7f4850f2a640 1 --2- 172.21.15.115:0/2148106453 >> v2:172.21.15.142:3300/0 conn(0x7f484c143eb0 0x7f484c06b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:28.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.850+0000 7f484bfff640 1 -- 172.21.15.115:0/2148106453 >> v2:172.21.15.142:3300/0 conn(0x7f484c143eb0 msgr2=0x7f484c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:28.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.850+0000 7f484bfff640 1 --2- 172.21.15.115:0/2148106453 >> v2:172.21.15.142:3300/0 conn(0x7f484c143eb0 0x7f484c06b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.850+0000 7f484bfff640 1 -- 172.21.15.115:0/2148106453 >> v1:172.21.15.115:6789/0 conn(0x7f484c0cbab0 legacy=0x7f484c164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:28.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.850+0000 7f484bfff640 1 -- 172.21.15.115:0/2148106453 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c0caa90 con 0x7f484c06b7a0 2023-12-01T01:48:28.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.850+0000 7f484b7fe640 1 -- 172.21.15.115:0/2148106453 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148929553 0 0) 0x7f484c0cbeb0 con 0x7f484c0cbab0 2023-12-01T01:48:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.851+0000 7f484bfff640 1 --2- 172.21.15.115:0/2148106453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c06b7a0 0x7f484c0cb570 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f484c0b0e20 tx=0x7f483802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc08d25a1e348fda server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f484b7fe640 1 -- 172.21.15.115:0/2148106453 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838031830 con 0x7f484c06b7a0 2023-12-01T01:48:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f484b7fe640 1 -- 172.21.15.115:0/2148106453 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4838031990 con 0x7f484c06b7a0 2023-12-01T01:48:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f484b7fe640 1 -- 172.21.15.115:0/2148106453 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838031d20 con 0x7f484c06b7a0 2023-12-01T01:48:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f485318e640 1 -- 172.21.15.115:0/2148106453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c06b7a0 msgr2=0x7f484c0cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f485318e640 1 --2- 172.21.15.115:0/2148106453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c06b7a0 0x7f484c0cb570 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f484c0b0e20 tx=0x7f483802b8d0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f485318e640 1 -- 172.21.15.115:0/2148106453 shutdown_connections 2023-12-01T01:48:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f485318e640 1 --2- 172.21.15.115:0/2148106453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c06b7a0 0x7f484c0cb570 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f485318e640 1 --2- 172.21.15.115:0/2148106453 >> v2:172.21.15.142:3300/0 conn(0x7f484c143eb0 0x7f484c06b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f485318e640 1 -- 172.21.15.115:0/2148106453 >> 172.21.15.115:0/2148106453 conn(0x7f484c065960 msgr2=0x7f484c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:28.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f485318e640 1 -- 172.21.15.115:0/2148106453 shutdown_connections 2023-12-01T01:48:28.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.852+0000 7f485318e640 1 -- 172.21.15.115:0/2148106453 wait complete. 2023-12-01T01:48:28.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.854+0000 7f485318e640 1 Processor -- start 2023-12-01T01:48:28.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.854+0000 7f485318e640 1 -- start start 2023-12-01T01:48:28.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.855+0000 7f485318e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f484c06b7a0 0x7f484c13be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.855+0000 7f485318e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c0cbab0 0x7f484c0b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.855+0000 7f485318e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c143eb0 0x7f484c139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.855+0000 7f485318e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c13d270 con 0x7f484c0cbab0 2023-12-01T01:48:28.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.855+0000 7f485318e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c13d550 con 0x7f484c06b7a0 2023-12-01T01:48:28.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.855+0000 7f485318e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484c1414a0 con 0x7f484c143eb0 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.855+0000 7f484bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c0cbab0 0x7f484c0b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.855+0000 7f484bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c0cbab0 0x7f484c0b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36486/0 (socket says 172.21.15.115:36486) 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.855+0000 7f484bfff640 1 -- 172.21.15.115:0/465223248 learned_addr learned my addr 172.21.15.115:0/465223248 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f4850f2a640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f484c06b7a0 0x7f484c13be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f484bfff640 1 -- 172.21.15.115:0/465223248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c143eb0 msgr2=0x7f484c139dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f484bfff640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c143eb0 0x7f484c139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f484bfff640 1 -- 172.21.15.115:0/465223248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f484c06b7a0 msgr2=0x7f484c13be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f484bfff640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f484c06b7a0 0x7f484c13be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f484bfff640 1 -- 172.21.15.115:0/465223248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4838009400 con 0x7f484c0cbab0 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f4850f2a640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f484c06b7a0 0x7f484c13be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f484bfff640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c0cbab0 0x7f484c0b2c20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4838009890 tx=0x7f48380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f48497fa640 1 -- 172.21.15.115:0/465223248 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380317d0 con 0x7f484c0cbab0 2023-12-01T01:48:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f48497fa640 1 -- 172.21.15.115:0/465223248 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483803ba70 con 0x7f484c0cbab0 2023-12-01T01:48:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f48497fa640 1 -- 172.21.15.115:0/465223248 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483803be20 con 0x7f484c0cbab0 2023-12-01T01:48:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.856+0000 7f485318e640 1 -- 172.21.15.115:0/465223248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484c141780 con 0x7f484c0cbab0 2023-12-01T01:48:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.857+0000 7f485318e640 1 -- 172.21.15.115:0/465223248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484c13ce60 con 0x7f484c0cbab0 2023-12-01T01:48:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.858+0000 7f485318e640 1 -- 172.21.15.115:0/465223248 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484c141ca0 con 0x7f484c0cbab0 2023-12-01T01:48:28.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.859+0000 7f48497fa640 1 -- 172.21.15.115:0/465223248 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f48380453f0 con 0x7f484c0cbab0 2023-12-01T01:48:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.859+0000 7f48497fa640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f483006a630 0x7f483006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.860+0000 7f48497fa640 1 -- 172.21.15.115:0/465223248 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f48380ab2d0 con 0x7f484c0cbab0 2023-12-01T01:48:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.860+0000 7f4850f2a640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f483006a630 0x7f483006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.860+0000 7f4850f2a640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f483006a630 0x7f483006caf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4840006fd0 tx=0x7f4840008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:28.865+0000 7f48497fa640 1 -- 172.21.15.115:0/465223248 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f484c141ca0 con 0x7f484c0cbab0 2023-12-01T01:48:29.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.078+0000 7f485318e640 1 -- 172.21.15.115:0/465223248 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f484c13d050 con 0x7f483006a630 2023-12-01T01:48:29.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.097+0000 7f48497fa640 1 -- 172.21.15.115:0/465223248 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f4840009530 con 0x7f483006a630 2023-12-01T01:48:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:29.098 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:39", "first": "2023-12-01T01:38:01", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.099+0000 7f485318e640 1 -- 172.21.15.115:0/465223248 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f483006a630 msgr2=0x7f483006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.099+0000 7f485318e640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f483006a630 0x7f483006caf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4840006fd0 tx=0x7f4840008040 comp rx=0 tx=0).stop 2023-12-01T01:48:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.099+0000 7f485318e640 1 -- 172.21.15.115:0/465223248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c0cbab0 msgr2=0x7f484c0b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.099+0000 7f485318e640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c0cbab0 0x7f484c0b2c20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4838009890 tx=0x7f48380046c0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.100+0000 7f485318e640 1 -- 172.21.15.115:0/465223248 shutdown_connections 2023-12-01T01:48:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.100+0000 7f485318e640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f483006a630 0x7f483006caf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.100+0000 7f485318e640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c143eb0 0x7f484c139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.100+0000 7f485318e640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f484c0cbab0 0x7f484c0b2c20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.100+0000 7f485318e640 1 --2- 172.21.15.115:0/465223248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f484c06b7a0 0x7f484c13be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.100+0000 7f485318e640 1 -- 172.21.15.115:0/465223248 >> 172.21.15.115:0/465223248 conn(0x7f484c065960 msgr2=0x7f484c0ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:29.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.100+0000 7f485318e640 1 -- 172.21.15.115:0/465223248 shutdown_connections 2023-12-01T01:48:29.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.100+0000 7f485318e640 1 -- 172.21.15.115:0/465223248 wait complete. 2023-12-01T01:48:29.118 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:39", "first": "2023-12-01T01:38:01", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:29.119 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:37:39', 'first': '2023-12-01T01:38:01', 'last': '2023-12-01T01:48:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-01T01:48:29.119 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T01:48:29.119 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T01:48:29.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:29.119 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-12-01T01:48:29.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:29.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:29.168 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:29.168 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:29.168 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:29.168 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:29.168 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:29.168 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:29.168 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:29.168 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_01_UTC 2023-12-01T01:48:29.168 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:29.169 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_38_00_UTC', 'scheduled-2023-12-01-01_39_00_UTC', 'scheduled-2023-12-01-01_40_00_UTC', 'scheduled-2023-12-01-01_41_00_UTC', 'scheduled-2023-12-01-01_42_00_UTC', 'scheduled-2023-12-01-01_43_00_UTC', 'scheduled-2023-12-01-01_44_00_UTC', 'scheduled-2023-12-01-01_45_01_UTC', 'scheduled-2023-12-01-01_46_01_UTC', 'scheduled-2023-12-01-01_47_01_UTC', 'scheduled-2023-12-01-01_48_01_UTC'] 2023-12-01T01:48:29.169 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir2 2023-12-01T01:48:29.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.383+0000 7fe3fec38640 1 Processor -- start 2023-12-01T01:48:29.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.383+0000 7fe3fec38640 1 -- start start 2023-12-01T01:48:29.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.384+0000 7fe3fec38640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe3f80cd1e0 0x7fe3f80b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.384+0000 7fe3fec38640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80b1c50 0x7fe3f815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.384+0000 7fe3fec38640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f80b2f30 con 0x7fe3f8142870 2023-12-01T01:48:29.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.384+0000 7fe3fec38640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80b3ae0 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.384+0000 7fe3fec38640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8142c50 con 0x7fe3f80b1c50 2023-12-01T01:48:29.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.385+0000 7fe3f7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe3f80cd1e0 0x7fe3f80b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:29.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.385+0000 7fe3f7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe3f80cd1e0 0x7fe3f80b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36492/0 (socket says 172.21.15.115:36492) 2023-12-01T01:48:29.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.385+0000 7fe3f7fff640 1 -- 172.21.15.115:0/2716837332 learned_addr learned my addr 172.21.15.115:0/2716837332 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:29.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.385+0000 7fe3f7fff640 1 -- 172.21.15.115:0/2716837332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80b1c50 msgr2=0x7fe3f815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:29.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.385+0000 7fe3f7fff640 1 --2- 172.21.15.115:0/2716837332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80b1c50 0x7fe3f815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.385+0000 7fe3f7fff640 1 -- 172.21.15.115:0/2716837332 >> v1:172.21.15.142:6789/0 conn(0x7fe3f8142870 legacy=0x7fe3f80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:29.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.385+0000 7fe3f7fff640 1 -- 172.21.15.115:0/2716837332 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f80b20c0 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.386+0000 7fe3f7fff640 1 --2- 172.21.15.115:0/2716837332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 0x7fe3f80b1710 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0098a0 tx=0x7fe3ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b34dfdcebfff09 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:29.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.386+0000 7fe3f77fe640 1 -- 172.21.15.115:0/2716837332 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec031640 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.386+0000 7fe3f77fe640 1 -- 172.21.15.115:0/2716837332 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec0317a0 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.386+0000 7fe3f77fe640 1 -- 172.21.15.115:0/2716837332 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec031b30 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.387+0000 7fe3fec38640 1 -- 172.21.15.115:0/2716837332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 msgr2=0x7fe3f80b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.387+0000 7fe3fec38640 1 --2- 172.21.15.115:0/2716837332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 0x7fe3f80b1710 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec0098a0 tx=0x7fe3ec02b8f0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.387+0000 7fe3fec38640 1 -- 172.21.15.115:0/2716837332 shutdown_connections 2023-12-01T01:48:29.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.387+0000 7fe3fec38640 1 --2- 172.21.15.115:0/2716837332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80b1c50 0x7fe3f815a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.387+0000 7fe3fec38640 1 --2- 172.21.15.115:0/2716837332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 0x7fe3f80b1710 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.387+0000 7fe3fec38640 1 -- 172.21.15.115:0/2716837332 >> 172.21.15.115:0/2716837332 conn(0x7fe3f8065960 msgr2=0x7fe3f8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:29.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.387+0000 7fe3fec38640 1 -- 172.21.15.115:0/2716837332 shutdown_connections 2023-12-01T01:48:29.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.387+0000 7fe3fec38640 1 -- 172.21.15.115:0/2716837332 wait complete. 2023-12-01T01:48:29.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.389+0000 7fe3fec38640 1 Processor -- start 2023-12-01T01:48:29.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.389+0000 7fe3fec38640 1 -- start start 2023-12-01T01:48:29.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.390+0000 7fe3fec38640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80b1c50 0x7fe3f813b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.390+0000 7fe3fec38640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 0x7fe3f80afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.390+0000 7fe3fec38640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f8142870 0x7fe3f80c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.390+0000 7fe3fec38640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80b0aa0 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.390+0000 7fe3fec38640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80ae200 con 0x7fe3f80b1c50 2023-12-01T01:48:29.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.390+0000 7fe3fec38640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80b07c0 con 0x7fe3f8142870 2023-12-01T01:48:29.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.390+0000 7fe3f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 0x7fe3f80afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:29.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.390+0000 7fe3f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 0x7fe3f80afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36498/0 (socket says 172.21.15.115:36498) 2023-12-01T01:48:29.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.390+0000 7fe3f7fff640 1 -- 172.21.15.115:0/3237695173 learned_addr learned my addr 172.21.15.115:0/3237695173 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:29.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.391+0000 7fe3f7fff640 1 -- 172.21.15.115:0/3237695173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f8142870 msgr2=0x7fe3f80c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.391+0000 7fe3f7fff640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f8142870 0x7fe3f80c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.391+0000 7fe3f7fff640 1 -- 172.21.15.115:0/3237695173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80b1c50 msgr2=0x7fe3f813b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.391+0000 7fe3f7fff640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80b1c50 0x7fe3f813b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.391+0000 7fe3f7fff640 1 -- 172.21.15.115:0/3237695173 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0094d0 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.391+0000 7fe3f7fff640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 0x7fe3f80afd70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec000c00 tx=0x7fe3ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:29.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.391+0000 7fe3f57fa640 1 -- 172.21.15.115:0/3237695173 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec03ac60 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.391+0000 7fe3f57fa640 1 -- 172.21.15.115:0/3237695173 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec03adc0 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.391+0000 7fe3f57fa640 1 -- 172.21.15.115:0/3237695173 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec044630 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.391+0000 7fe3fec38640 1 -- 172.21.15.115:0/3237695173 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f80adbb0 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.392+0000 7fe3fec38640 1 -- 172.21.15.115:0/3237695173 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f8156df0 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.393+0000 7fe3fec38640 1 -- 172.21.15.115:0/3237695173 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f804e890 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.393+0000 7fe3f57fa640 1 -- 172.21.15.115:0/3237695173 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fe3ec02f050 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.394+0000 7fe3f57fa640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e406a630 0x7fe3e406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.394+0000 7fe3f57fa640 1 -- 172.21.15.115:0/3237695173 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fe3ec0aafe0 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.395+0000 7fe3fc9d4640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e406a630 0x7fe3e406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:29.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.395+0000 7fe3fc9d4640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e406a630 0x7fe3e406caf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800abb0 tx=0x7fe3e8009250 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:29.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.398+0000 7fe3f57fa640 1 -- 172.21.15.115:0/3237695173 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe3ec076020 con 0x7fe3f80cd1e0 2023-12-01T01:48:29.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.609+0000 7fe3fec38640 1 -- 172.21.15.115:0/3237695173 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3f80b2600 con 0x7fe3e406a630 2023-12-01T01:48:29.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.627+0000 7fe3f57fa640 1 -- 172.21.15.115:0/3237695173 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fe3e800a7a0 con 0x7fe3e406a630 2023-12-01T01:48:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:29.628 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:39", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:29.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.629+0000 7fe3fec38640 1 -- 172.21.15.115:0/3237695173 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e406a630 msgr2=0x7fe3e406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.629+0000 7fe3fec38640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e406a630 0x7fe3e406caf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800abb0 tx=0x7fe3e8009250 comp rx=0 tx=0).stop 2023-12-01T01:48:29.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.629+0000 7fe3fec38640 1 -- 172.21.15.115:0/3237695173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 msgr2=0x7fe3f80afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.629+0000 7fe3fec38640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 0x7fe3f80afd70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec000c00 tx=0x7fe3ec0046c0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.629+0000 7fe3fec38640 1 -- 172.21.15.115:0/3237695173 shutdown_connections 2023-12-01T01:48:29.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.629+0000 7fe3fec38640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e406a630 0x7fe3e406caf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.629+0000 7fe3fec38640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f8142870 0x7fe3f80c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.629+0000 7fe3fec38640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3f80cd1e0 0x7fe3f80afd70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.629+0000 7fe3fec38640 1 --2- 172.21.15.115:0/3237695173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80b1c50 0x7fe3f813b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.629+0000 7fe3fec38640 1 -- 172.21.15.115:0/3237695173 >> 172.21.15.115:0/3237695173 conn(0x7fe3f8065960 msgr2=0x7fe3f81671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:29.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.630+0000 7fe3fec38640 1 -- 172.21.15.115:0/3237695173 shutdown_connections 2023-12-01T01:48:29.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.630+0000 7fe3fec38640 1 -- 172.21.15.115:0/3237695173 wait complete. 2023-12-01T01:48:29.651 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:39", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:29.651 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:37:39', 'first': '2023-12-01T01:38:00', 'last': '2023-12-01T01:48:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-01T01:48:29.651 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T01:48:29.651 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T01:48:29.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:29.651 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-12-01T01:48:29.706 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:29.706 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:29.706 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_01_UTC 2023-12-01T01:48:29.706 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_01_UTC 2023-12-01T01:48:29.706 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_01_UTC 2023-12-01T01:48:29.706 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_01_UTC 2023-12-01T01:48:29.706 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:29.706 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:29.707 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:29.707 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:29.707 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:29.707 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_38_00_UTC', 'scheduled-2023-12-01-01_39_00_UTC', 'scheduled-2023-12-01-01_40_01_UTC', 'scheduled-2023-12-01-01_41_01_UTC', 'scheduled-2023-12-01-01_42_01_UTC', 'scheduled-2023-12-01-01_43_01_UTC', 'scheduled-2023-12-01-01_44_01_UTC', 'scheduled-2023-12-01-01_45_00_UTC', 'scheduled-2023-12-01-01_46_00_UTC', 'scheduled-2023-12-01-01_47_00_UTC', 'scheduled-2023-12-01-01_48_01_UTC'] 2023-12-01T01:48:29.707 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir3 2023-12-01T01:48:29.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.915+0000 7f5f2359e640 1 Processor -- start 2023-12-01T01:48:29.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.916+0000 7f5f2359e640 1 -- start start 2023-12-01T01:48:29.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.917+0000 7f5f2359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f240cf460 0x7f5f240cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.917+0000 7f5f2359e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f240cfde0 con 0x7f5f240b5b80 2023-12-01T01:48:29.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.917+0000 7f5f2359e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f240b8610 con 0x7f5f2415ce60 2023-12-01T01:48:29.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.917+0000 7f5f2359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2415d240 con 0x7f5f240cf460 2023-12-01T01:48:29.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.918+0000 7f5f22d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5f240b5b80 0x7f5f240b6010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59164/0 (socket says 172.21.15.115:59164) 2023-12-01T01:48:29.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.918+0000 7f5f22d9d640 1 -- 172.21.15.115:0/1993774049 learned_addr learned my addr 172.21.15.115:0/1993774049 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:29.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.918+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176791961 0 0) 0x7f5f240cfde0 con 0x7f5f240b5b80 2023-12-01T01:48:29.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.918+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f10003520 con 0x7f5f240b5b80 2023-12-01T01:48:29.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.918+0000 7f5f21d9b640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f240cf460 0x7f5f240cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.918+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3131621412 0 0) 0x7f5f10003520 con 0x7f5f240b5b80 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.918+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f240cf460 msgr2=0x7f5f240cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.918+0000 7f5f2159a640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f240cf460 0x7f5f240cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.918+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 >> v1:172.21.15.142:6789/0 conn(0x7f5f2415ce60 legacy=0x7f5f240d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240d1630 con 0x7f5f240b5b80 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5f1c001670 con 0x7f5f240b5b80 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 >> v1:172.21.15.115:6789/0 conn(0x7f5f240b5b80 legacy=0x7f5f240b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f2159a640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f100039a0 0x7f5f10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f2159a640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 0x7f5f10004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f2159a640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f240cf460 0x7f5f100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2415d240 con 0x7f5f100047c0 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1000bab0 con 0x7f5f100039a0 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1000bc70 con 0x7f5f240cf460 2023-12-01T01:48:29.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f21d9b640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 0x7f5f10004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:29.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f21d9b640 1 -- 172.21.15.115:0/1993774049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f240cf460 msgr2=0x7f5f100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:29.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f21d9b640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f240cf460 0x7f5f100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f2259c640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f100039a0 0x7f5f10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:29.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f21d9b640 1 -- 172.21.15.115:0/1993774049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f100039a0 msgr2=0x7f5f10003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:29.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f21d9b640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f100039a0 0x7f5f10003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.919+0000 7f5f21d9b640 1 -- 172.21.15.115:0/1993774049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f10003520 con 0x7f5f100047c0 2023-12-01T01:48:29.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.920+0000 7f5f21d9b640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 0x7f5f10004c40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00ba30 tx=0x7f5f100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3659888486bdfed9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:29.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.921+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c032680 con 0x7f5f100047c0 2023-12-01T01:48:29.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.921+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c0327e0 con 0x7f5f100047c0 2023-12-01T01:48:29.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.921+0000 7f5f2159a640 1 -- 172.21.15.115:0/1993774049 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c032b70 con 0x7f5f100047c0 2023-12-01T01:48:29.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.921+0000 7f5f2359e640 1 -- 172.21.15.115:0/1993774049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 msgr2=0x7f5f10004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.921+0000 7f5f2359e640 1 --2- 172.21.15.115:0/1993774049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 0x7f5f10004c40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00ba30 tx=0x7f5f100057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.921+0000 7f5f22d9d640 1 -- 172.21.15.115:0/1993774049 reap_dead start 2023-12-01T01:48:29.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.921+0000 7f5f2359e640 1 -- 172.21.15.115:0/1993774049 shutdown_connections 2023-12-01T01:48:29.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.921+0000 7f5f2359e640 1 -- 172.21.15.115:0/1993774049 >> 172.21.15.115:0/1993774049 conn(0x7f5f24065960 msgr2=0x7f5f24065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:29.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.921+0000 7f5f2359e640 1 -- 172.21.15.115:0/1993774049 shutdown_connections 2023-12-01T01:48:29.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.922+0000 7f5f2359e640 1 -- 172.21.15.115:0/1993774049 wait complete. 2023-12-01T01:48:29.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.923+0000 7f5f2359e640 1 Processor -- start 2023-12-01T01:48:29.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.923+0000 7f5f2359e640 1 -- start start 2023-12-01T01:48:29.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.924+0000 7f5f2359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 0x7f5f24158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.924+0000 7f5f2359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f2415b620 0x7f5f240cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.924+0000 7f5f2359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f240b4470 0x7f5f240b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.924+0000 7f5f2359e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f240b4e60 con 0x7f5f100047c0 2023-12-01T01:48:29.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.924+0000 7f5f2359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f241f2400 con 0x7f5f240b4470 2023-12-01T01:48:29.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.924+0000 7f5f2359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f240b38d0 con 0x7f5f2415b620 2023-12-01T01:48:29.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.925+0000 7f5f2259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 0x7f5f24158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:29.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.925+0000 7f5f2259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 0x7f5f24158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36520/0 (socket says 172.21.15.115:36520) 2023-12-01T01:48:29.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.925+0000 7f5f2259c640 1 -- 172.21.15.115:0/359768631 learned_addr learned my addr 172.21.15.115:0/359768631 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.925+0000 7f5f2259c640 1 -- 172.21.15.115:0/359768631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f2415b620 msgr2=0x7f5f240cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.925+0000 7f5f2259c640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f2415b620 0x7f5f240cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.925+0000 7f5f2259c640 1 -- 172.21.15.115:0/359768631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f240b4470 msgr2=0x7f5f240b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.925+0000 7f5f2259c640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f240b4470 0x7f5f240b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.925+0000 7f5f2259c640 1 -- 172.21.15.115:0/359768631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f241f2600 con 0x7f5f100047c0 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.925+0000 7f5f2259c640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 0x7f5f24158b00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400d8e0 tx=0x7f5f1400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.926+0000 7f5f1affd640 1 -- 172.21.15.115:0/359768631 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14003da0 con 0x7f5f100047c0 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.926+0000 7f5f1affd640 1 -- 172.21.15.115:0/359768631 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1400b6b0 con 0x7f5f100047c0 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.926+0000 7f5f1affd640 1 -- 172.21.15.115:0/359768631 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400ba20 con 0x7f5f100047c0 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.926+0000 7f5f2359e640 1 -- 172.21.15.115:0/359768631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f241f28e0 con 0x7f5f100047c0 2023-12-01T01:48:29.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.926+0000 7f5f2359e640 1 -- 172.21.15.115:0/359768631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f241f2ea0 con 0x7f5f100047c0 2023-12-01T01:48:29.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.927+0000 7f5f2359e640 1 -- 172.21.15.115:0/359768631 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f24073ae0 con 0x7f5f100047c0 2023-12-01T01:48:29.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.928+0000 7f5f1affd640 1 -- 172.21.15.115:0/359768631 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f5f1400bb80 con 0x7f5f100047c0 2023-12-01T01:48:29.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.929+0000 7f5f1affd640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0406a660 0x7f5f0406cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:29.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.929+0000 7f5f1affd640 1 -- 172.21.15.115:0/359768631 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f5f14089280 con 0x7f5f100047c0 2023-12-01T01:48:29.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.929+0000 7f5f21d9b640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0406a660 0x7f5f0406cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:29.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.930+0000 7f5f21d9b640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0406a660 0x7f5f0406cb20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00ba00 tx=0x7f5f0c032000 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:29.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:29.933+0000 7f5f1affd640 1 -- 172.21.15.115:0/359768631 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5f14054350 con 0x7f5f100047c0 2023-12-01T01:48:30.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.147+0000 7f5f2359e640 1 -- 172.21.15.115:0/359768631 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f240d1910 con 0x7f5f0406a660 2023-12-01T01:48:30.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.165+0000 7f5f1affd640 1 -- 172.21.15.115:0/359768631 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f5f0c0329d0 con 0x7f5f0406a660 2023-12-01T01:48:30.167 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:30.167 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:40", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 -- 172.21.15.115:0/359768631 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0406a660 msgr2=0x7f5f0406cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0406a660 0x7f5f0406cb20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00ba00 tx=0x7f5f0c032000 comp rx=0 tx=0).stop 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 -- 172.21.15.115:0/359768631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 msgr2=0x7f5f24158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 0x7f5f24158b00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400d8e0 tx=0x7f5f1400dda0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 -- 172.21.15.115:0/359768631 shutdown_connections 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0406a660 0x7f5f0406cb20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f2415b620 0x7f5f240cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f240b4470 0x7f5f240b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 --2- 172.21.15.115:0/359768631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f100047c0 0x7f5f24158b00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 -- 172.21.15.115:0/359768631 >> 172.21.15.115:0/359768631 conn(0x7f5f24065960 msgr2=0x7f5f240d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 -- 172.21.15.115:0/359768631 shutdown_connections 2023-12-01T01:48:30.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.168+0000 7f5f2359e640 1 -- 172.21.15.115:0/359768631 wait complete. 2023-12-01T01:48:30.187 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:40", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:30.187 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:37:40', 'first': '2023-12-01T01:38:00', 'last': '2023-12-01T01:48:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-01T01:48:30.188 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T01:48:30.188 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T01:48:30.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:30.188 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-12-01T01:48:30.240 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_01_UTC 2023-12-01T01:48:30.240 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:30.240 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:30.240 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:30.240 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:30.240 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:30.240 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:30.240 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:30.240 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:30.241 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:30.241 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:30.242 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_38_01_UTC', 'scheduled-2023-12-01-01_39_00_UTC', 'scheduled-2023-12-01-01_40_00_UTC', 'scheduled-2023-12-01-01_41_00_UTC', 'scheduled-2023-12-01-01_42_00_UTC', 'scheduled-2023-12-01-01_43_00_UTC', 'scheduled-2023-12-01-01_44_00_UTC', 'scheduled-2023-12-01-01_45_00_UTC', 'scheduled-2023-12-01-01_46_01_UTC', 'scheduled-2023-12-01-01_47_00_UTC', 'scheduled-2023-12-01-01_48_00_UTC'] 2023-12-01T01:48:30.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir4 2023-12-01T01:48:30.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.450+0000 7fc6ee9d2640 1 Processor -- start 2023-12-01T01:48:30.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.450+0000 7fc6ee9d2640 1 -- start start 2023-12-01T01:48:30.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.452+0000 7fc6ee9d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e815cf70 0x7fc6e80b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.452+0000 7fc6ee9d2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc6e80b18f0 0x7fc6e80b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.452+0000 7fc6ee9d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e80b7760 con 0x7fc6e80cfb50 2023-12-01T01:48:30.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.452+0000 7fc6ee9d2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e80b2170 con 0x7fc6e80b18f0 2023-12-01T01:48:30.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.452+0000 7fc6ee9d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80b8b70 con 0x7fc6e815cf70 2023-12-01T01:48:30.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.452+0000 7fc6e7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e815cf70 0x7fc6e80b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:30.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.453+0000 7fc6e7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e815cf70 0x7fc6e80b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:51766/0 (socket says 172.21.15.115:51766) 2023-12-01T01:48:30.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.453+0000 7fc6e7fff640 1 -- 172.21.15.115:0/1245910825 learned_addr learned my addr 172.21.15.115:0/1245910825 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:30.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.453+0000 7fc6dffff640 1 --2- 172.21.15.115:0/1245910825 >> v2:172.21.15.142:3300/0 conn(0x7fc6e80b18f0 0x7fc6e80b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:30.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.454+0000 7fc6e7fff640 1 -- 172.21.15.115:0/1245910825 >> v2:172.21.15.142:3300/0 conn(0x7fc6e80b18f0 msgr2=0x7fc6e80b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:30.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.454+0000 7fc6e7fff640 1 --2- 172.21.15.115:0/1245910825 >> v2:172.21.15.142:3300/0 conn(0x7fc6e80b18f0 0x7fc6e80b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.454+0000 7fc6e7fff640 1 -- 172.21.15.115:0/1245910825 >> v1:172.21.15.142:6789/0 conn(0x7fc6e80cfb50 legacy=0x7fc6e80d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.454+0000 7fc6e7fff640 1 -- 172.21.15.115:0/1245910825 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80b6e30 con 0x7fc6e815cf70 2023-12-01T01:48:30.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.455+0000 7fc6e7fff640 1 --2- 172.21.15.115:0/1245910825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e815cf70 0x7fc6e80b8470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009870 tx=0x7fc6d802ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9a8492ea850051 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:30.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.455+0000 7fc6e77fe640 1 -- 172.21.15.115:0/1245910825 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d802f830 con 0x7fc6e815cf70 2023-12-01T01:48:30.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.461+0000 7fc6e77fe640 1 -- 172.21.15.115:0/1245910825 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d802f990 con 0x7fc6e815cf70 2023-12-01T01:48:30.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.461+0000 7fc6e77fe640 1 -- 172.21.15.115:0/1245910825 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d802fd20 con 0x7fc6e815cf70 2023-12-01T01:48:30.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.462+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1245910825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e815cf70 msgr2=0x7fc6e80b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.462+0000 7fc6ee9d2640 1 --2- 172.21.15.115:0/1245910825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e815cf70 0x7fc6e80b8470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009870 tx=0x7fc6d802ba10 comp rx=0 tx=0).stop 2023-12-01T01:48:30.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.462+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1245910825 shutdown_connections 2023-12-01T01:48:30.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.462+0000 7fc6ee9d2640 1 --2- 172.21.15.115:0/1245910825 >> v2:172.21.15.142:3300/0 conn(0x7fc6e80b18f0 0x7fc6e80b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.462+0000 7fc6ee9d2640 1 --2- 172.21.15.115:0/1245910825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e815cf70 0x7fc6e80b8470 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.462+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1245910825 >> 172.21.15.115:0/1245910825 conn(0x7fc6e806a0e0 msgr2=0x7fc6e806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:30.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.462+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1245910825 shutdown_connections 2023-12-01T01:48:30.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.462+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1245910825 wait complete. 2023-12-01T01:48:30.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.464+0000 7fc6ee9d2640 1 Processor -- start 2023-12-01T01:48:30.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.465+0000 7fc6ee9d2640 1 -- start start 2023-12-01T01:48:30.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.466+0000 7fc6ee9d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6e80b18f0 0x7fc6e8156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.466+0000 7fc6ee9d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e80cfb50 0x7fc6e8065960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.466+0000 7fc6ee9d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e815cf70 0x7fc6e8065ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.466+0000 7fc6ee9d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81544d0 con 0x7fc6e80b18f0 2023-12-01T01:48:30.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.466+0000 7fc6ee9d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80d1770 con 0x7fc6e815cf70 2023-12-01T01:48:30.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.466+0000 7fc6ee9d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80d1a50 con 0x7fc6e80cfb50 2023-12-01T01:48:30.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.466+0000 7fc6e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6e80b18f0 0x7fc6e8156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:30.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.466+0000 7fc6e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6e80b18f0 0x7fc6e8156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36522/0 (socket says 172.21.15.115:36522) 2023-12-01T01:48:30.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.466+0000 7fc6e7fff640 1 -- 172.21.15.115:0/1800504613 learned_addr learned my addr 172.21.15.115:0/1800504613 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6ecf6f640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e815cf70 0x7fc6e8065ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6dffff640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e80cfb50 0x7fc6e8065960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6e7fff640 1 -- 172.21.15.115:0/1800504613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e80cfb50 msgr2=0x7fc6e8065960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6e7fff640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e80cfb50 0x7fc6e8065960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6e7fff640 1 -- 172.21.15.115:0/1800504613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e815cf70 msgr2=0x7fc6e8065ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6e7fff640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e815cf70 0x7fc6e8065ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6e7fff640 1 -- 172.21.15.115:0/1800504613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80094d0 con 0x7fc6e80b18f0 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6e7fff640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6e80b18f0 0x7fc6e8156620 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8000c00 tx=0x7fc6d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6e57fa640 1 -- 172.21.15.115:0/1800504613 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d803b040 con 0x7fc6e80b18f0 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1800504613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e8154160 con 0x7fc6e80b18f0 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6e57fa640 1 -- 172.21.15.115:0/1800504613 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8037c40 con 0x7fc6e80b18f0 2023-12-01T01:48:30.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6e57fa640 1 -- 172.21.15.115:0/1800504613 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8042480 con 0x7fc6e80b18f0 2023-12-01T01:48:30.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.467+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1800504613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e80d0c20 con 0x7fc6e80b18f0 2023-12-01T01:48:30.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.468+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1800504613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e8153fc0 con 0x7fc6e80b18f0 2023-12-01T01:48:30.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.469+0000 7fc6e57fa640 1 -- 172.21.15.115:0/1800504613 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fc6d80425e0 con 0x7fc6e80b18f0 2023-12-01T01:48:30.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.470+0000 7fc6e57fa640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c806a660 0x7fc6c806cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.470+0000 7fc6e57fa640 1 -- 172.21.15.115:0/1800504613 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fc6d80abfa0 con 0x7fc6e80b18f0 2023-12-01T01:48:30.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.470+0000 7fc6dffff640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c806a660 0x7fc6c806cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:30.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.471+0000 7fc6dffff640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c806a660 0x7fc6c806cb20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80cc8c0 tx=0x7fc6d0008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.475+0000 7fc6e57fa640 1 -- 172.21.15.115:0/1800504613 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc6e8153fc0 con 0x7fc6e80b18f0 2023-12-01T01:48:30.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.669+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1800504613 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6e80b73f0 con 0x7fc6c806a660 2023-12-01T01:48:30.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.687+0000 7fc6e57fa640 1 -- 172.21.15.115:0/1800504613 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fc6d0010840 con 0x7fc6c806a660 2023-12-01T01:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:30.689 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:41", "first": "2023-12-01T01:38:01", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1800504613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c806a660 msgr2=0x7fc6c806cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c806a660 0x7fc6c806cb20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80cc8c0 tx=0x7fc6d0008040 comp rx=0 tx=0).stop 2023-12-01T01:48:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1800504613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6e80b18f0 msgr2=0x7fc6e8156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6e80b18f0 0x7fc6e8156620 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8000c00 tx=0x7fc6d80046c0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1800504613 shutdown_connections 2023-12-01T01:48:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6c806a660 0x7fc6c806cb20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e80cfb50 0x7fc6e8065960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6e815cf70 0x7fc6e8065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 --2- 172.21.15.115:0/1800504613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6e80b18f0 0x7fc6e8156620 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1800504613 >> 172.21.15.115:0/1800504613 conn(0x7fc6e806a0e0 msgr2=0x7fc6e81600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1800504613 shutdown_connections 2023-12-01T01:48:30.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.690+0000 7fc6ee9d2640 1 -- 172.21.15.115:0/1800504613 wait complete. 2023-12-01T01:48:30.710 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:41", "first": "2023-12-01T01:38:01", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:30.710 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:37:41', 'first': '2023-12-01T01:38:01', 'last': '2023-12-01T01:48:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-01T01:48:30.710 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T01:48:30.710 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T01:48:30.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:30.710 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_01_UTC 2023-12-01T01:48:30.762 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:30.763 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_38_00_UTC', 'scheduled-2023-12-01-01_39_00_UTC', 'scheduled-2023-12-01-01_40_00_UTC', 'scheduled-2023-12-01-01_41_00_UTC', 'scheduled-2023-12-01-01_42_00_UTC', 'scheduled-2023-12-01-01_43_00_UTC', 'scheduled-2023-12-01-01_44_00_UTC', 'scheduled-2023-12-01-01_45_00_UTC', 'scheduled-2023-12-01-01_46_01_UTC', 'scheduled-2023-12-01-01_47_01_UTC', 'scheduled-2023-12-01-01_48_01_UTC'] 2023-12-01T01:48:30.764 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir5 2023-12-01T01:48:30.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.967+0000 7fe47dcfb640 1 Processor -- start 2023-12-01T01:48:30.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.968+0000 7fe47dcfb640 1 -- start start 2023-12-01T01:48:30.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.969+0000 7fe47dcfb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe478142870 0x7fe4780ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.969+0000 7fe47dcfb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4780cd010 0x7fe4780b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.969+0000 7fe47dcfb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe4780b1c50 0x7fe47815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.969+0000 7fe47dcfb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4780b3ae0 con 0x7fe478142870 2023-12-01T01:48:30.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.969+0000 7fe47dcfb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe478142c50 con 0x7fe4780b1c50 2023-12-01T01:48:30.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.969+0000 7fe47dcfb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4780cd470 con 0x7fe4780cd010 2023-12-01T01:48:30.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.970+0000 7fe4777fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe478142870 0x7fe4780ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:30.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.970+0000 7fe4777fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe478142870 0x7fe4780ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36538/0 (socket says 172.21.15.115:36538) 2023-12-01T01:48:30.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.970+0000 7fe4777fe640 1 -- 172.21.15.115:0/3411431874 learned_addr learned my addr 172.21.15.115:0/3411431874 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:30.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.970+0000 7fe4777fe640 1 -- 172.21.15.115:0/3411431874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4780cd010 msgr2=0x7fe4780b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:30.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.970+0000 7fe4777fe640 1 --2- 172.21.15.115:0/3411431874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4780cd010 0x7fe4780b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.970+0000 7fe4777fe640 1 -- 172.21.15.115:0/3411431874 >> v2:172.21.15.142:3300/0 conn(0x7fe4780b1c50 msgr2=0x7fe47815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:30.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.970+0000 7fe4777fe640 1 --2- 172.21.15.115:0/3411431874 >> v2:172.21.15.142:3300/0 conn(0x7fe4780b1c50 0x7fe47815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.970+0000 7fe4777fe640 1 -- 172.21.15.115:0/3411431874 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4780b20e0 con 0x7fe478142870 2023-12-01T01:48:30.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.971+0000 7fe4777fe640 1 --2- 172.21.15.115:0/3411431874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe478142870 0x7fe4780ccad0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe4780b0f50 tx=0x7fe46802db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56adafd66f30f703 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:30.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.971+0000 7fe4767fc640 1 -- 172.21.15.115:0/3411431874 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46802f8c0 con 0x7fe478142870 2023-12-01T01:48:30.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.971+0000 7fe4767fc640 1 -- 172.21.15.115:0/3411431874 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46802fa20 con 0x7fe478142870 2023-12-01T01:48:30.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.971+0000 7fe4767fc640 1 -- 172.21.15.115:0/3411431874 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46802fdb0 con 0x7fe478142870 2023-12-01T01:48:30.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.972+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3411431874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe478142870 msgr2=0x7fe4780ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.972+0000 7fe47dcfb640 1 --2- 172.21.15.115:0/3411431874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe478142870 0x7fe4780ccad0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe4780b0f50 tx=0x7fe46802db00 comp rx=0 tx=0).stop 2023-12-01T01:48:30.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.972+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3411431874 shutdown_connections 2023-12-01T01:48:30.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.972+0000 7fe47dcfb640 1 --2- 172.21.15.115:0/3411431874 >> v2:172.21.15.142:3300/0 conn(0x7fe4780b1c50 0x7fe47815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.972+0000 7fe47dcfb640 1 --2- 172.21.15.115:0/3411431874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4780cd010 0x7fe4780b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.972+0000 7fe47dcfb640 1 --2- 172.21.15.115:0/3411431874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe478142870 0x7fe4780ccad0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.972+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3411431874 >> 172.21.15.115:0/3411431874 conn(0x7fe478065960 msgr2=0x7fe478065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:30.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.972+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3411431874 shutdown_connections 2023-12-01T01:48:30.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.972+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3411431874 wait complete. 2023-12-01T01:48:30.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.974+0000 7fe47dcfb640 1 Processor -- start 2023-12-01T01:48:30.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.975+0000 7fe47dcfb640 1 -- start start 2023-12-01T01:48:30.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe47dcfb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4780b1c50 0x7fe478158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe47dcfb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4780cd010 0x7fe478158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe47dcfb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe478142870 0x7fe4780c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe47dcfb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4780c4e90 con 0x7fe4780cd010 2023-12-01T01:48:30.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe47dcfb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4781e5080 con 0x7fe4780b1c50 2023-12-01T01:48:30.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe47dcfb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4781e5360 con 0x7fe478142870 2023-12-01T01:48:30.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe476ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4780cd010 0x7fe478158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:30.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe476ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4780cd010 0x7fe478158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36544/0 (socket says 172.21.15.115:36544) 2023-12-01T01:48:30.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe476ffd640 1 -- 172.21.15.115:0/3745475150 learned_addr learned my addr 172.21.15.115:0/3745475150 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe476ffd640 1 -- 172.21.15.115:0/3745475150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe478142870 msgr2=0x7fe4780c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe476ffd640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe478142870 0x7fe4780c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe4777fe640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4780b1c50 0x7fe478158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe476ffd640 1 -- 172.21.15.115:0/3745475150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4780b1c50 msgr2=0x7fe478158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe476ffd640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4780b1c50 0x7fe478158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.976+0000 7fe476ffd640 1 -- 172.21.15.115:0/3745475150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4680094d0 con 0x7fe4780cd010 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.977+0000 7fe476ffd640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4780cd010 0x7fe478158210 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00b4a0 tx=0x7fe46c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.977+0000 7fe45ffff640 1 -- 172.21.15.115:0/3745475150 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c010610 con 0x7fe4780cd010 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.977+0000 7fe45ffff640 1 -- 172.21.15.115:0/3745475150 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c010770 con 0x7fe4780cd010 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.977+0000 7fe45ffff640 1 -- 172.21.15.115:0/3745475150 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c010b20 con 0x7fe4780cd010 2023-12-01T01:48:30.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.977+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3745475150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4781e5640 con 0x7fe4780cd010 2023-12-01T01:48:30.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.977+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3745475150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4781e5c00 con 0x7fe4780cd010 2023-12-01T01:48:30.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.978+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3745475150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4781e5e20 con 0x7fe4780cd010 2023-12-01T01:48:30.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.979+0000 7fe45ffff640 1 -- 172.21.15.115:0/3745475150 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fe46c010c80 con 0x7fe4780cd010 2023-12-01T01:48:30.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.980+0000 7fe45ffff640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe46406a630 0x7fe46406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:30.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.980+0000 7fe45ffff640 1 -- 172.21.15.115:0/3745475150 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fe46c088a30 con 0x7fe4780cd010 2023-12-01T01:48:30.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.980+0000 7fe4777fe640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe46406a630 0x7fe46406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:30.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.981+0000 7fe4777fe640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe46406a630 0x7fe46406caf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe468038000 tx=0x7fe468008030 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:30.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:30.985+0000 7fe45ffff640 1 -- 172.21.15.115:0/3745475150 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe46c053c10 con 0x7fe4780cd010 2023-12-01T01:48:31.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.184+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3745475150 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4780b2320 con 0x7fe46406a630 2023-12-01T01:48:31.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.203+0000 7fe45ffff640 1 -- 172.21.15.115:0/3745475150 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fe46802f890 con 0x7fe46406a630 2023-12-01T01:48:31.204 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:31.205 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:41", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.205+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3745475150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe46406a630 msgr2=0x7fe46406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.205+0000 7fe47dcfb640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe46406a630 0x7fe46406caf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe468038000 tx=0x7fe468008030 comp rx=0 tx=0).stop 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.205+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3745475150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4780cd010 msgr2=0x7fe478158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.205+0000 7fe47dcfb640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4780cd010 0x7fe478158210 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00b4a0 tx=0x7fe46c00b960 comp rx=0 tx=0).stop 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.205+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3745475150 shutdown_connections 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.205+0000 7fe47dcfb640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe46406a630 0x7fe46406caf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.205+0000 7fe47dcfb640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe478142870 0x7fe4780c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.205+0000 7fe47dcfb640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4780cd010 0x7fe478158210 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.205+0000 7fe47dcfb640 1 --2- 172.21.15.115:0/3745475150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4780b1c50 0x7fe478158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.205+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3745475150 >> 172.21.15.115:0/3745475150 conn(0x7fe478065960 msgr2=0x7fe47806bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:31.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.206+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3745475150 shutdown_connections 2023-12-01T01:48:31.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.206+0000 7fe47dcfb640 1 -- 172.21.15.115:0/3745475150 wait complete. 2023-12-01T01:48:31.226 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:41", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:31.226 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:37:41', 'first': '2023-12-01T01:38:00', 'last': '2023-12-01T01:48:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-01T01:48:31.226 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T01:48:31.226 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T01:48:31.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:31.226 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_01_UTC 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:31.278 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:31.279 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_38_01_UTC', 'scheduled-2023-12-01-01_39_00_UTC', 'scheduled-2023-12-01-01_40_00_UTC', 'scheduled-2023-12-01-01_41_00_UTC', 'scheduled-2023-12-01-01_42_00_UTC', 'scheduled-2023-12-01-01_43_00_UTC', 'scheduled-2023-12-01-01_44_01_UTC', 'scheduled-2023-12-01-01_45_01_UTC', 'scheduled-2023-12-01-01_46_00_UTC', 'scheduled-2023-12-01-01_47_00_UTC', 'scheduled-2023-12-01-01_48_00_UTC'] 2023-12-01T01:48:31.280 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir6 2023-12-01T01:48:31.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.487+0000 7fc61213e640 1 Processor -- start 2023-12-01T01:48:31.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.487+0000 7fc61213e640 1 -- start start 2023-12-01T01:48:31.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.488+0000 7fc61213e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc60c15ac80 0x7fc60c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:31.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc61213e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc60c0ce1b0 0x7fc60c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc61213e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60c0cc9a0 con 0x7fc60c0b0e60 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc61213e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60c15b060 con 0x7fc60c15ac80 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc61213e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0b0ce0 con 0x7fc60c0ce1b0 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc60b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc60c15ac80 0x7fc60c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc603fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc60c0b0e60 0x7fc60c0cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59172/0 (socket says 172.21.15.115:59172) 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc603fff640 1 -- 172.21.15.115:0/391729174 learned_addr learned my addr 172.21.15.115:0/391729174 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc60b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc60c15ac80 0x7fc60c0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36554/0 (socket says 172.21.15.115:36554) 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc60b7fe640 1 -- 172.21.15.115:0/391729174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc60c0ce1b0 msgr2=0x7fc60c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc60b7fe640 1 --2- 172.21.15.115:0/391729174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc60c0ce1b0 0x7fc60c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc60b7fe640 1 -- 172.21.15.115:0/391729174 >> v1:172.21.15.115:6789/0 conn(0x7fc60c0b0e60 legacy=0x7fc60c0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:31.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.489+0000 7fc60b7fe640 1 -- 172.21.15.115:0/391729174 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c0cbd90 con 0x7fc60c15ac80 2023-12-01T01:48:31.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.490+0000 7fc60b7fe640 1 --2- 172.21.15.115:0/391729174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c15ac80 0x7fc60c0cce30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0097a0 tx=0x7fc5fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ebc1ae984512d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:31.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.491+0000 7fc60affd640 1 -- 172.21.15.115:0/391729174 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc031830 con 0x7fc60c15ac80 2023-12-01T01:48:31.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.491+0000 7fc60affd640 1 -- 172.21.15.115:0/391729174 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc031990 con 0x7fc60c15ac80 2023-12-01T01:48:31.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.491+0000 7fc60affd640 1 -- 172.21.15.115:0/391729174 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc031d20 con 0x7fc60c15ac80 2023-12-01T01:48:31.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.491+0000 7fc61213e640 1 -- 172.21.15.115:0/391729174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c15ac80 msgr2=0x7fc60c0cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:31.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.491+0000 7fc61213e640 1 --2- 172.21.15.115:0/391729174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c15ac80 0x7fc60c0cce30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0097a0 tx=0x7fc5fc02b8d0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.491+0000 7fc61213e640 1 -- 172.21.15.115:0/391729174 shutdown_connections 2023-12-01T01:48:31.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.491+0000 7fc61213e640 1 --2- 172.21.15.115:0/391729174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc60c0ce1b0 0x7fc60c0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.491+0000 7fc61213e640 1 --2- 172.21.15.115:0/391729174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c15ac80 0x7fc60c0cce30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.491+0000 7fc61213e640 1 -- 172.21.15.115:0/391729174 >> 172.21.15.115:0/391729174 conn(0x7fc60c065960 msgr2=0x7fc60c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:31.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.492+0000 7fc61213e640 1 -- 172.21.15.115:0/391729174 shutdown_connections 2023-12-01T01:48:31.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.492+0000 7fc61213e640 1 -- 172.21.15.115:0/391729174 wait complete. 2023-12-01T01:48:31.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.493+0000 7fc61213e640 1 Processor -- start 2023-12-01T01:48:31.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.493+0000 7fc61213e640 1 -- start start 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.494+0000 7fc61213e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c0b0e60 0x7fc60c1408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.494+0000 7fc61213e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc60c0ce1b0 0x7fc60c1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.494+0000 7fc61213e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc60c15ac80 0x7fc60c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.494+0000 7fc61213e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c141210 con 0x7fc60c0b0e60 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.494+0000 7fc61213e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c1414f0 con 0x7fc60c0ce1b0 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.494+0000 7fc61213e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c14e570 con 0x7fc60c15ac80 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc60b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c0b0e60 0x7fc60c1408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc60b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c0b0e60 0x7fc60c1408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36558/0 (socket says 172.21.15.115:36558) 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc60b7fe640 1 -- 172.21.15.115:0/4247419678 learned_addr learned my addr 172.21.15.115:0/4247419678 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc60b7fe640 1 -- 172.21.15.115:0/4247419678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc60c15ac80 msgr2=0x7fc60c151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc60bfff640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc60c15ac80 0x7fc60c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc603fff640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc60c0ce1b0 0x7fc60c1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc60b7fe640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc60c15ac80 0x7fc60c151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc60b7fe640 1 -- 172.21.15.115:0/4247419678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc60c0ce1b0 msgr2=0x7fc60c1516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:31.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc60b7fe640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc60c0ce1b0 0x7fc60c1516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc60b7fe640 1 -- 172.21.15.115:0/4247419678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc009400 con 0x7fc60c0b0e60 2023-12-01T01:48:31.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.495+0000 7fc60bfff640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc60c15ac80 0x7fc60c151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:31.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.496+0000 7fc60b7fe640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c0b0e60 0x7fc60c1408c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0098c0 tx=0x7fc5fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:31.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.496+0000 7fc608ff9640 1 -- 172.21.15.115:0/4247419678 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc03aa60 con 0x7fc60c0b0e60 2023-12-01T01:48:31.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.496+0000 7fc608ff9640 1 -- 172.21.15.115:0/4247419678 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc03abc0 con 0x7fc60c0b0e60 2023-12-01T01:48:31.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.496+0000 7fc608ff9640 1 -- 172.21.15.115:0/4247419678 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0423f0 con 0x7fc60c0b0e60 2023-12-01T01:48:31.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.496+0000 7fc61213e640 1 -- 172.21.15.115:0/4247419678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c152250 con 0x7fc60c0b0e60 2023-12-01T01:48:31.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.496+0000 7fc61213e640 1 -- 172.21.15.115:0/4247419678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c152810 con 0x7fc60c0b0e60 2023-12-01T01:48:31.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.497+0000 7fc61213e640 1 -- 172.21.15.115:0/4247419678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c04e890 con 0x7fc60c0b0e60 2023-12-01T01:48:31.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.498+0000 7fc608ff9640 1 -- 172.21.15.115:0/4247419678 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fc5fc030380 con 0x7fc60c0b0e60 2023-12-01T01:48:31.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.499+0000 7fc608ff9640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5ec06a630 0x7fc5ec06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:31.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.500+0000 7fc608ff9640 1 -- 172.21.15.115:0/4247419678 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fc5fc0aaae0 con 0x7fc60c0b0e60 2023-12-01T01:48:31.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.500+0000 7fc603fff640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5ec06a630 0x7fc5ec06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:31.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.500+0000 7fc603fff640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5ec06a630 0x7fc5ec06caf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc60c1f5fc0 tx=0x7fc5f4009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:31.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.504+0000 7fc608ff9640 1 -- 172.21.15.115:0/4247419678 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc5fc075ce0 con 0x7fc60c0b0e60 2023-12-01T01:48:31.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.716+0000 7fc61213e640 1 -- 172.21.15.115:0/4247419678 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc60c0cc350 con 0x7fc5ec06a630 2023-12-01T01:48:31.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.733+0000 7fc608ff9640 1 -- 172.21.15.115:0/4247419678 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fc5f4001340 con 0x7fc5ec06a630 2023-12-01T01:48:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:31.735 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:42", "first": "2023-12-01T01:38:01", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:31.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 -- 172.21.15.115:0/4247419678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5ec06a630 msgr2=0x7fc5ec06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:31.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5ec06a630 0x7fc5ec06caf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc60c1f5fc0 tx=0x7fc5f4009040 comp rx=0 tx=0).stop 2023-12-01T01:48:31.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 -- 172.21.15.115:0/4247419678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c0b0e60 msgr2=0x7fc60c1408c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:31.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c0b0e60 0x7fc60c1408c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0098c0 tx=0x7fc5fc0046c0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 -- 172.21.15.115:0/4247419678 shutdown_connections 2023-12-01T01:48:31.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5ec06a630 0x7fc5ec06caf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc60c15ac80 0x7fc60c151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc60c0ce1b0 0x7fc60c1516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 --2- 172.21.15.115:0/4247419678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc60c0b0e60 0x7fc60c1408c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:31.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 -- 172.21.15.115:0/4247419678 >> 172.21.15.115:0/4247419678 conn(0x7fc60c065960 msgr2=0x7fc60c0c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:31.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 -- 172.21.15.115:0/4247419678 shutdown_connections 2023-12-01T01:48:31.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:31.736+0000 7fc61213e640 1 -- 172.21.15.115:0/4247419678 wait complete. 2023-12-01T01:48:31.759 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:42", "first": "2023-12-01T01:38:01", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:31.759 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:37:42', 'first': '2023-12-01T01:38:01', 'last': '2023-12-01T01:48:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-01T01:48:31.759 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T01:48:31.759 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T01:48:31.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:31.759 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-12-01T01:48:31.813 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:31.813 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:31.814 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:31.814 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:31.814 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:31.814 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:31.814 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:31.814 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:31.814 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:31.814 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:31.814 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:31.815 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_38_00_UTC', 'scheduled-2023-12-01-01_39_00_UTC', 'scheduled-2023-12-01-01_40_00_UTC', 'scheduled-2023-12-01-01_41_00_UTC', 'scheduled-2023-12-01-01_42_00_UTC', 'scheduled-2023-12-01-01_43_00_UTC', 'scheduled-2023-12-01-01_44_00_UTC', 'scheduled-2023-12-01-01_45_00_UTC', 'scheduled-2023-12-01-01_46_00_UTC', 'scheduled-2023-12-01-01_47_00_UTC', 'scheduled-2023-12-01-01_48_00_UTC'] 2023-12-01T01:48:31.815 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir7 2023-12-01T01:48:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.032+0000 7f472ee43640 1 Processor -- start 2023-12-01T01:48:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.032+0000 7f472ee43640 1 -- start start 2023-12-01T01:48:32.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.034+0000 7f472ee43640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4728160200 0x7f47280b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.034+0000 7f472ee43640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47280cb290 con 0x7f47280b0f80 2023-12-01T01:48:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.034+0000 7f472ee43640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47280b0ae0 con 0x7f47280cbd30 2023-12-01T01:48:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.034+0000 7f472ee43640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280b35f0 con 0x7f4728160200 2023-12-01T01:48:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.034+0000 7f4727fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f47280b0f80 0x7f472806b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59184/0 (socket says 172.21.15.115:59184) 2023-12-01T01:48:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.034+0000 7f4727fff640 1 -- 172.21.15.115:0/4175997608 learned_addr learned my addr 172.21.15.115:0/4175997608 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:32.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f472cbdf640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4728160200 0x7f47280b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981235280 0 0) 0x7f47280cb290 con 0x7f47280b0f80 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4718003520 con 0x7f47280b0f80 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2721206623 0 0) 0x7f4718003520 con 0x7f47280b0f80 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4728160200 msgr2=0x7f47280b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4728160200 0x7f47280b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 >> v1:172.21.15.142:6789/0 conn(0x7f47280cbd30 legacy=0x7f47280cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280b3310 con 0x7f47280b0f80 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4714001670 con 0x7f47280b0f80 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 >> v1:172.21.15.115:6789/0 conn(0x7f47280b0f80 legacy=0x7f472806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47180039a0 0x7f4718003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47180047c0 0x7f4718004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4728160200 0x7f4718005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280b35f0 con 0x7f47180047c0 2023-12-01T01:48:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471800ba60 con 0x7f47180039a0 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.035+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471800bc20 con 0x7f4728160200 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.036+0000 7f4727fff640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47180047c0 0x7f4718004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.036+0000 7f472d3e0640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4728160200 0x7f4718005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.036+0000 7f4727fff640 1 -- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4728160200 msgr2=0x7f4718005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.036+0000 7f4727fff640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4728160200 0x7f4718005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.036+0000 7f472cbdf640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47180039a0 0x7f4718003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.036+0000 7f4727fff640 1 -- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47180039a0 msgr2=0x7f4718003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.036+0000 7f4727fff640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47180039a0 0x7f4718003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.036+0000 7f4727fff640 1 -- 172.21.15.115:0/4175997608 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718003520 con 0x7f47180047c0 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.036+0000 7f472d3e0640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4728160200 0x7f4718005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.037+0000 7f4727fff640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47180047c0 0x7f4718004c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4714001eb0 tx=0x7f47180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e8541e46375318 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.037+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714005ad0 con 0x7f47180047c0 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.037+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4714005c30 con 0x7f47180047c0 2023-12-01T01:48:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.037+0000 7f47277fe640 1 -- 172.21.15.115:0/4175997608 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471402c7c0 con 0x7f47180047c0 2023-12-01T01:48:32.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.038+0000 7f472ee43640 1 -- 172.21.15.115:0/4175997608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47180047c0 msgr2=0x7f4718004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.038+0000 7f472ee43640 1 --2- 172.21.15.115:0/4175997608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47180047c0 0x7f4718004c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4714001eb0 tx=0x7f47180057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.038+0000 7f472d3e0640 1 -- 172.21.15.115:0/4175997608 reap_dead start 2023-12-01T01:48:32.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.038+0000 7f472ee43640 1 -- 172.21.15.115:0/4175997608 shutdown_connections 2023-12-01T01:48:32.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.038+0000 7f472ee43640 1 -- 172.21.15.115:0/4175997608 >> 172.21.15.115:0/4175997608 conn(0x7f4728065960 msgr2=0x7f4728065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:32.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.038+0000 7f472ee43640 1 -- 172.21.15.115:0/4175997608 shutdown_connections 2023-12-01T01:48:32.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.038+0000 7f472ee43640 1 -- 172.21.15.115:0/4175997608 wait complete. 2023-12-01T01:48:32.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.040+0000 7f472ee43640 1 Processor -- start 2023-12-01T01:48:32.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.040+0000 7f472ee43640 1 -- start start 2023-12-01T01:48:32.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.041+0000 7f472ee43640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47180047c0 0x7f47281555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.041+0000 7f472ee43640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47280c7d40 0x7f47281457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.041+0000 7f472ee43640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f472815c150 0x7f472815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.041+0000 7f472ee43640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47281f5c00 con 0x7f472815c150 2023-12-01T01:48:32.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.041+0000 7f472ee43640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472815cb10 con 0x7f47180047c0 2023-12-01T01:48:32.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.041+0000 7f472ee43640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4728142ab0 con 0x7f47280c7d40 2023-12-01T01:48:32.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.041+0000 7f472d3e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f472815c150 0x7f472815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.041+0000 7f472d3e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f472815c150 0x7f472815c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36574/0 (socket says 172.21.15.115:36574) 2023-12-01T01:48:32.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.041+0000 7f472d3e0640 1 -- 172.21.15.115:0/2713305429 learned_addr learned my addr 172.21.15.115:0/2713305429 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:32.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f472cbdf640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47180047c0 0x7f47281555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f4727fff640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47280c7d40 0x7f47281457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f472d3e0640 1 -- 172.21.15.115:0/2713305429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47280c7d40 msgr2=0x7f47281457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f472d3e0640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47280c7d40 0x7f47281457f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f472d3e0640 1 -- 172.21.15.115:0/2713305429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47180047c0 msgr2=0x7f47281555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f472d3e0640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47180047c0 0x7f47281555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f472d3e0640 1 -- 172.21.15.115:0/2713305429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728145d30 con 0x7f472815c150 2023-12-01T01:48:32.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f472d3e0640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f472815c150 0x7f472815c5d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f47280b1b30 tx=0x7f472000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:32.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f47257fa640 1 -- 172.21.15.115:0/2713305429 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720006d50 con 0x7f472815c150 2023-12-01T01:48:32.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f47257fa640 1 -- 172.21.15.115:0/2713305429 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720003970 con 0x7f472815c150 2023-12-01T01:48:32.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f47257fa640 1 -- 172.21.15.115:0/2713305429 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720003d60 con 0x7f472815c150 2023-12-01T01:48:32.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.042+0000 7f472ee43640 1 -- 172.21.15.115:0/2713305429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47281f5e10 con 0x7f472815c150 2023-12-01T01:48:32.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.043+0000 7f472ee43640 1 -- 172.21.15.115:0/2713305429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47281f63d0 con 0x7f472815c150 2023-12-01T01:48:32.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.044+0000 7f472ee43640 1 -- 172.21.15.115:0/2713305429 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472804e890 con 0x7f472815c150 2023-12-01T01:48:32.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.045+0000 7f47257fa640 1 -- 172.21.15.115:0/2713305429 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f4720005070 con 0x7f472815c150 2023-12-01T01:48:32.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.046+0000 7f47257fa640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f470c06a660 0x7f470c06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.046+0000 7f47257fa640 1 -- 172.21.15.115:0/2713305429 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f4720088ea0 con 0x7f472815c150 2023-12-01T01:48:32.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.046+0000 7f472cbdf640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f470c06a660 0x7f470c06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.047+0000 7f472cbdf640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f470c06a660 0x7f470c06cb20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f471c009990 tx=0x7f471c008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:32.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.050+0000 7f47257fa640 1 -- 172.21.15.115:0/2713305429 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4720053ee0 con 0x7f472815c150 2023-12-01T01:48:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.259+0000 7f472ee43640 1 -- 172.21.15.115:0/2713305429 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47280c94a0 con 0x7f470c06a660 2023-12-01T01:48:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.277+0000 7f47257fa640 1 -- 172.21.15.115:0/2713305429 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f47280c94a0 con 0x7f470c06a660 2023-12-01T01:48:32.279 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:32.279 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:43", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 -- 172.21.15.115:0/2713305429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f470c06a660 msgr2=0x7f470c06cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f470c06a660 0x7f470c06cb20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f471c009990 tx=0x7f471c008040 comp rx=0 tx=0).stop 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 -- 172.21.15.115:0/2713305429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f472815c150 msgr2=0x7f472815c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f472815c150 0x7f472815c5d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f47280b1b30 tx=0x7f472000dc00 comp rx=0 tx=0).stop 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 -- 172.21.15.115:0/2713305429 shutdown_connections 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f470c06a660 0x7f470c06cb20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47280c7d40 0x7f47281457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f472815c150 0x7f472815c5d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 --2- 172.21.15.115:0/2713305429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47180047c0 0x7f47281555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 -- 172.21.15.115:0/2713305429 >> 172.21.15.115:0/2713305429 conn(0x7f4728065960 msgr2=0x7f47280cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 -- 172.21.15.115:0/2713305429 shutdown_connections 2023-12-01T01:48:32.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.280+0000 7f472ee43640 1 -- 172.21.15.115:0/2713305429 wait complete. 2023-12-01T01:48:32.299 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:43", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:32.299 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:37:43', 'first': '2023-12-01T01:38:00', 'last': '2023-12-01T01:48:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-01T01:48:32.299 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T01:48:32.299 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T01:48:32.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:32.300 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-12-01T01:48:32.350 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:32.350 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:32.350 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:32.350 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:32.350 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:32.351 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:32.351 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:32.351 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:32.351 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:32.351 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:32.351 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:32.352 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_38_00_UTC', 'scheduled-2023-12-01-01_39_00_UTC', 'scheduled-2023-12-01-01_40_00_UTC', 'scheduled-2023-12-01-01_41_00_UTC', 'scheduled-2023-12-01-01_42_00_UTC', 'scheduled-2023-12-01-01_43_00_UTC', 'scheduled-2023-12-01-01_44_00_UTC', 'scheduled-2023-12-01-01_45_00_UTC', 'scheduled-2023-12-01-01_46_00_UTC', 'scheduled-2023-12-01-01_47_00_UTC', 'scheduled-2023-12-01-01_48_00_UTC'] 2023-12-01T01:48:32.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir8 2023-12-01T01:48:32.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.566+0000 7f32092fb640 1 Processor -- start 2023-12-01T01:48:32.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.566+0000 7f32092fb640 1 -- start start 2023-12-01T01:48:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.567+0000 7f32092fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32040b0f80 0x7f32040b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.567+0000 7f32092fb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32040cb290 con 0x7f3204160200 2023-12-01T01:48:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.568+0000 7f32092fb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32040b0ae0 con 0x7f32040cbd30 2023-12-01T01:48:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.568+0000 7f32092fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32040b35f0 con 0x7f32040b0f80 2023-12-01T01:48:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.568+0000 7f3202ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3204160200 0x7f320406b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59186/0 (socket says 172.21.15.115:59186) 2023-12-01T01:48:32.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.568+0000 7f3202ffd640 1 -- 172.21.15.115:0/3925417592 learned_addr learned my addr 172.21.15.115:0/3925417592 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.568+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402929437 0 0) 0x7f32040cb290 con 0x7f3204160200 2023-12-01T01:48:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.568+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31f0003520 con 0x7f3204160200 2023-12-01T01:48:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.568+0000 7f31fa7fc640 1 --2- 172.21.15.115:0/3925417592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32040b0f80 0x7f32040b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2172104102 0 0) 0x7f31f0003520 con 0x7f3204160200 2023-12-01T01:48:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32040b0f80 msgr2=0x7f32040b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 --2- 172.21.15.115:0/3925417592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32040b0f80 0x7f32040b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 >> v1:172.21.15.142:6789/0 conn(0x7f32040cbd30 legacy=0x7f32040cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32040b3310 con 0x7f3204160200 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f31f4001670 con 0x7f3204160200 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 >> v1:172.21.15.115:6789/0 conn(0x7f3204160200 legacy=0x7f320406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 --2- 172.21.15.115:0/3925417592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31f00039a0 0x7f31f0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 --2- 172.21.15.115:0/3925417592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32040b0f80 0x7f31f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 --2- 172.21.15.115:0/3925417592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31f0004d00 0x7f31f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32040b35f0 con 0x7f31f00039a0 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f000bb10 con 0x7f31f0004d00 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f000bca0 con 0x7f32040b0f80 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f3202ffd640 1 --2- 172.21.15.115:0/3925417592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31f00039a0 0x7f31f0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f3202ffd640 1 -- 172.21.15.115:0/3925417592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32040b0f80 msgr2=0x7f31f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f3202ffd640 1 --2- 172.21.15.115:0/3925417592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32040b0f80 0x7f31f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f3202ffd640 1 -- 172.21.15.115:0/3925417592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31f0004d00 msgr2=0x7f31f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f3202ffd640 1 --2- 172.21.15.115:0/3925417592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31f0004d00 0x7f31f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.569+0000 7f3202ffd640 1 -- 172.21.15.115:0/3925417592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f0003520 con 0x7f31f00039a0 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.571+0000 7f3202ffd640 1 --2- 172.21.15.115:0/3925417592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31f00039a0 0x7f31f0003dc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c270 tx=0x7f31f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970ab103a1ae5bfc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:32.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.571+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4005ad0 con 0x7f31f00039a0 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.571+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f4005c30 con 0x7f31f00039a0 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.571+0000 7f32027fc640 1 -- 172.21.15.115:0/3925417592 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f402c8a0 con 0x7f31f00039a0 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.571+0000 7f32092fb640 1 -- 172.21.15.115:0/3925417592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31f00039a0 msgr2=0x7f31f0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.571+0000 7f32092fb640 1 --2- 172.21.15.115:0/3925417592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31f00039a0 0x7f31f0003dc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c270 tx=0x7f31f00057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.571+0000 7f32037fe640 1 -- 172.21.15.115:0/3925417592 reap_dead start 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.571+0000 7f32092fb640 1 -- 172.21.15.115:0/3925417592 shutdown_connections 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.571+0000 7f32092fb640 1 -- 172.21.15.115:0/3925417592 >> 172.21.15.115:0/3925417592 conn(0x7f3204065960 msgr2=0x7f3204065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.572+0000 7f32092fb640 1 -- 172.21.15.115:0/3925417592 shutdown_connections 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.572+0000 7f32092fb640 1 -- 172.21.15.115:0/3925417592 wait complete. 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.573+0000 7f32092fb640 1 Processor -- start 2023-12-01T01:48:32.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.573+0000 7f32092fb640 1 -- start start 2023-12-01T01:48:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.574+0000 7f32092fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31f00039a0 0x7f320414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.574+0000 7f32092fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32041560c0 0x7f32041507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.574+0000 7f32092fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f320406fa70 0x7f320406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.574+0000 7f32092fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3204070430 con 0x7f32041560c0 2023-12-01T01:48:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.574+0000 7f32092fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3204150320 con 0x7f320406fa70 2023-12-01T01:48:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.574+0000 7f32092fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320414d890 con 0x7f31f00039a0 2023-12-01T01:48:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.575+0000 7f31fa7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32041560c0 0x7f32041507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.575+0000 7f31fa7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32041560c0 0x7f32041507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36596/0 (socket says 172.21.15.115:36596) 2023-12-01T01:48:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.575+0000 7f31fa7fc640 1 -- 172.21.15.115:0/662217188 learned_addr learned my addr 172.21.15.115:0/662217188 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:32.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.575+0000 7f31fa7fc640 1 -- 172.21.15.115:0/662217188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31f00039a0 msgr2=0x7f320414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.575+0000 7f31fa7fc640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31f00039a0 0x7f320414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.575+0000 7f31fa7fc640 1 -- 172.21.15.115:0/662217188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f320406fa70 msgr2=0x7f320406fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.575+0000 7f31fa7fc640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f320406fa70 0x7f320406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.575+0000 7f31fa7fc640 1 -- 172.21.15.115:0/662217188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320414f710 con 0x7f32041560c0 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.575+0000 7f31fa7fc640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32041560c0 0x7f32041507b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00d620 tx=0x7f31ec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.575+0000 7f31fbfff640 1 -- 172.21.15.115:0/662217188 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00b620 con 0x7f32041560c0 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.576+0000 7f31fbfff640 1 -- 172.21.15.115:0/662217188 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec00b780 con 0x7f32041560c0 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.576+0000 7f32092fb640 1 -- 172.21.15.115:0/662217188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320414f9f0 con 0x7f32041560c0 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.576+0000 7f31fbfff640 1 -- 172.21.15.115:0/662217188 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00bb10 con 0x7f32041560c0 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.576+0000 7f32092fb640 1 -- 172.21.15.115:0/662217188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320414ffb0 con 0x7f32041560c0 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.577+0000 7f32092fb640 1 -- 172.21.15.115:0/662217188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3204150170 con 0x7f32041560c0 2023-12-01T01:48:32.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.578+0000 7f31fbfff640 1 -- 172.21.15.115:0/662217188 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f31ec00bc70 con 0x7f32041560c0 2023-12-01T01:48:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.579+0000 7f31fbfff640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f31e406a660 0x7f31e406cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.579+0000 7f31fbfff640 1 -- 172.21.15.115:0/662217188 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f31ec0891d0 con 0x7f32041560c0 2023-12-01T01:48:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.579+0000 7f3202ffd640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f31e406a660 0x7f31e406cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:32.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.580+0000 7f3202ffd640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f31e406a660 0x7f31e406cb20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f31f4034000 tx=0x7f31f4007000 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:32.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.583+0000 7f31fbfff640 1 -- 172.21.15.115:0/662217188 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f31ec0543c0 con 0x7f32041560c0 2023-12-01T01:48:32.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.793+0000 7f32092fb640 1 -- 172.21.15.115:0/662217188 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3204150490 con 0x7f31e406a660 2023-12-01T01:48:32.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.810+0000 7f31fbfff640 1 -- 172.21.15.115:0/662217188 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f3204150490 con 0x7f31e406a660 2023-12-01T01:48:32.817 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:32.817 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:43", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:32.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.812+0000 7f32092fb640 1 -- 172.21.15.115:0/662217188 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f31e406a660 msgr2=0x7f31e406cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.812+0000 7f32092fb640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f31e406a660 0x7f31e406cb20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f31f4034000 tx=0x7f31f4007000 comp rx=0 tx=0).stop 2023-12-01T01:48:32.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.812+0000 7f32092fb640 1 -- 172.21.15.115:0/662217188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32041560c0 msgr2=0x7f32041507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.812+0000 7f32092fb640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32041560c0 0x7f32041507b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00d620 tx=0x7f31ec00dae0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.813+0000 7f32092fb640 1 -- 172.21.15.115:0/662217188 shutdown_connections 2023-12-01T01:48:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.813+0000 7f32092fb640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f31e406a660 0x7f31e406cb20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.813+0000 7f32092fb640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f320406fa70 0x7f320406fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.813+0000 7f32092fb640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32041560c0 0x7f32041507b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.813+0000 7f32092fb640 1 --2- 172.21.15.115:0/662217188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31f00039a0 0x7f320414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.813+0000 7f32092fb640 1 -- 172.21.15.115:0/662217188 >> 172.21.15.115:0/662217188 conn(0x7f3204065960 msgr2=0x7f320406f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.813+0000 7f32092fb640 1 -- 172.21.15.115:0/662217188 shutdown_connections 2023-12-01T01:48:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:32.813+0000 7f32092fb640 1 -- 172.21.15.115:0/662217188 wait complete. 2023-12-01T01:48:32.834 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:43", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:32.834 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:37:43', 'first': '2023-12-01T01:38:00', 'last': '2023-12-01T01:48:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-01T01:48:32.834 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T01:48:32.834 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T01:48:32.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:32.835 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-12-01T01:48:32.888 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:32.889 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:32.889 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:32.889 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:32.889 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:32.889 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:32.889 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:32.889 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:32.889 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:32.889 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_01_UTC 2023-12-01T01:48:32.889 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:32.891 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_38_00_UTC', 'scheduled-2023-12-01-01_39_00_UTC', 'scheduled-2023-12-01-01_40_00_UTC', 'scheduled-2023-12-01-01_41_00_UTC', 'scheduled-2023-12-01-01_42_00_UTC', 'scheduled-2023-12-01-01_43_00_UTC', 'scheduled-2023-12-01-01_44_00_UTC', 'scheduled-2023-12-01-01_45_00_UTC', 'scheduled-2023-12-01-01_46_00_UTC', 'scheduled-2023-12-01-01_47_01_UTC', 'scheduled-2023-12-01-01_48_01_UTC'] 2023-12-01T01:48:32.891 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir9 2023-12-01T01:48:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.107+0000 7f705ef51640 1 Processor -- start 2023-12-01T01:48:33.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.107+0000 7f705ef51640 1 -- start start 2023-12-01T01:48:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.109+0000 7f705ef51640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 0x7f70580cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.109+0000 7f705ef51640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f70580b5fa0 0x7f7058161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.109+0000 7f705ef51640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70580ce5e0 con 0x7f705815cd90 2023-12-01T01:48:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.109+0000 7f705ef51640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70580d0ec0 con 0x7f70580b5fa0 2023-12-01T01:48:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.109+0000 7f705ef51640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7058161950 con 0x7f70580cfd60 2023-12-01T01:48:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.109+0000 7f705cced640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f705815cd90 0x7f70580cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59188/0 (socket says 172.21.15.115:59188) 2023-12-01T01:48:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.109+0000 7f705cced640 1 -- 172.21.15.115:0/3350094864 learned_addr learned my addr 172.21.15.115:0/3350094864 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:33.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.109+0000 7f705d4ee640 1 --2- 172.21.15.115:0/3350094864 >> v2:172.21.15.115:3300/0 conn(0x7f70580b5fa0 0x7f7058161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.109+0000 7f7057fff640 1 --2- 172.21.15.115:0/3350094864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 0x7f70580cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.110+0000 7f70577fe640 1 -- 172.21.15.115:0/3350094864 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253316235 0 0) 0x7f704c001110 con 0x7f705815cd90 2023-12-01T01:48:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.110+0000 7f705d4ee640 1 -- 172.21.15.115:0/3350094864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 msgr2=0x7f70580cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:33.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.110+0000 7f705d4ee640 1 --2- 172.21.15.115:0/3350094864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 0x7f70580cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.110+0000 7f705d4ee640 1 -- 172.21.15.115:0/3350094864 >> v1:172.21.15.115:6789/0 conn(0x7f705815cd90 legacy=0x7f70580cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.110+0000 7f705d4ee640 1 -- 172.21.15.115:0/3350094864 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70580ce300 con 0x7f70580b5fa0 2023-12-01T01:48:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.111+0000 7f705d4ee640 1 --2- 172.21.15.115:0/3350094864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70580b5fa0 0x7f7058161410 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7050009d90 tx=0x7f705002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da9d5c01b79bba3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.111+0000 7f70577fe640 1 -- 172.21.15.115:0/3350094864 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705002cbd0 con 0x7f70580b5fa0 2023-12-01T01:48:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.111+0000 7f70577fe640 1 -- 172.21.15.115:0/3350094864 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705002cd30 con 0x7f70580b5fa0 2023-12-01T01:48:33.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.111+0000 7f70577fe640 1 -- 172.21.15.115:0/3350094864 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705002cea0 con 0x7f70580b5fa0 2023-12-01T01:48:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.112+0000 7f705ef51640 1 -- 172.21.15.115:0/3350094864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70580b5fa0 msgr2=0x7f7058161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.112+0000 7f705ef51640 1 --2- 172.21.15.115:0/3350094864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70580b5fa0 0x7f7058161410 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7050009d90 tx=0x7f705002be20 comp rx=0 tx=0).stop 2023-12-01T01:48:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.112+0000 7f705ef51640 1 -- 172.21.15.115:0/3350094864 shutdown_connections 2023-12-01T01:48:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.112+0000 7f705ef51640 1 --2- 172.21.15.115:0/3350094864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70580b5fa0 0x7f7058161410 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.112+0000 7f705ef51640 1 --2- 172.21.15.115:0/3350094864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 0x7f70580cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.112+0000 7f705ef51640 1 -- 172.21.15.115:0/3350094864 >> 172.21.15.115:0/3350094864 conn(0x7f7058065960 msgr2=0x7f7058065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.112+0000 7f705ef51640 1 -- 172.21.15.115:0/3350094864 shutdown_connections 2023-12-01T01:48:33.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.112+0000 7f705ef51640 1 -- 172.21.15.115:0/3350094864 wait complete. 2023-12-01T01:48:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.114+0000 7f705ef51640 1 Processor -- start 2023-12-01T01:48:33.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.114+0000 7f705ef51640 1 -- start start 2023-12-01T01:48:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705ef51640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70580b5fa0 0x7f705815fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705ef51640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 0x7f70580b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705ef51640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705815cd90 0x7f705815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705ef51640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70581600d0 con 0x7f705815cd90 2023-12-01T01:48:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705ef51640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70580b5370 con 0x7f70580b5fa0 2023-12-01T01:48:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705ef51640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705815eaf0 con 0x7f70580cfd60 2023-12-01T01:48:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705d4ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705815cd90 0x7f705815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705d4ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705815cd90 0x7f705815e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36612/0 (socket says 172.21.15.115:36612) 2023-12-01T01:48:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705d4ee640 1 -- 172.21.15.115:0/2258730704 learned_addr learned my addr 172.21.15.115:0/2258730704 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:33.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f7057fff640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 0x7f70580b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705cced640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70580b5fa0 0x7f705815fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705d4ee640 1 -- 172.21.15.115:0/2258730704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 msgr2=0x7f70580b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705d4ee640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 0x7f70580b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705d4ee640 1 -- 172.21.15.115:0/2258730704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70580b5fa0 msgr2=0x7f705815fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705d4ee640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70580b5fa0 0x7f705815fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f705d4ee640 1 -- 172.21.15.115:0/2258730704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7050009a70 con 0x7f705815cd90 2023-12-01T01:48:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.115+0000 7f7057fff640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 0x7f70580b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:33.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.116+0000 7f705d4ee640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705815cd90 0x7f705815e2a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7050009d90 tx=0x7f7050004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:33.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.116+0000 7f70557fa640 1 -- 172.21.15.115:0/2258730704 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705002cea0 con 0x7f705815cd90 2023-12-01T01:48:33.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.116+0000 7f70557fa640 1 -- 172.21.15.115:0/2258730704 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7050040670 con 0x7f705815cd90 2023-12-01T01:48:33.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.116+0000 7f70557fa640 1 -- 172.21.15.115:0/2258730704 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050040a20 con 0x7f705815cd90 2023-12-01T01:48:33.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.116+0000 7f705ef51640 1 -- 172.21.15.115:0/2258730704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705815edd0 con 0x7f705815cd90 2023-12-01T01:48:33.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.116+0000 7f705ef51640 1 -- 172.21.15.115:0/2258730704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705815f390 con 0x7f705815cd90 2023-12-01T01:48:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.117+0000 7f705ef51640 1 -- 172.21.15.115:0/2258730704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705804e890 con 0x7f705815cd90 2023-12-01T01:48:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.118+0000 7f70557fa640 1 -- 172.21.15.115:0/2258730704 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f7050040b80 con 0x7f705815cd90 2023-12-01T01:48:33.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.119+0000 7f70557fa640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f703c06a660 0x7f703c06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.119+0000 7f70557fa640 1 -- 172.21.15.115:0/2258730704 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f70500abb60 con 0x7f705815cd90 2023-12-01T01:48:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.119+0000 7f705cced640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f703c06a660 0x7f703c06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.120+0000 7f705cced640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f703c06a660 0x7f703c06cb20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f704c000f60 tx=0x7f704c009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:33.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.123+0000 7f70557fa640 1 -- 172.21.15.115:0/2258730704 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7050076cb0 con 0x7f705815cd90 2023-12-01T01:48:33.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.332+0000 7f705ef51640 1 -- 172.21.15.115:0/2258730704 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70580d1380 con 0x7f703c06a660 2023-12-01T01:48:33.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.348+0000 7f70557fa640 1 -- 172.21.15.115:0/2258730704 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f704c001c00 con 0x7f703c06a660 2023-12-01T01:48:33.350 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:33.351 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:44", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:33.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.350+0000 7f705ef51640 1 -- 172.21.15.115:0/2258730704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f703c06a660 msgr2=0x7f703c06cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:33.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.350+0000 7f705ef51640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f703c06a660 0x7f703c06cb20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f704c000f60 tx=0x7f704c009040 comp rx=0 tx=0).stop 2023-12-01T01:48:33.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.350+0000 7f705ef51640 1 -- 172.21.15.115:0/2258730704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705815cd90 msgr2=0x7f705815e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:33.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.350+0000 7f705ef51640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705815cd90 0x7f705815e2a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7050009d90 tx=0x7f7050004a40 comp rx=0 tx=0).stop 2023-12-01T01:48:33.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.351+0000 7f705ef51640 1 -- 172.21.15.115:0/2258730704 shutdown_connections 2023-12-01T01:48:33.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.351+0000 7f705ef51640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f703c06a660 0x7f703c06cb20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.351+0000 7f705ef51640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70580cfd60 0x7f70580b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.351+0000 7f705ef51640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f705815cd90 0x7f705815e2a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.351+0000 7f705ef51640 1 --2- 172.21.15.115:0/2258730704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70580b5fa0 0x7f705815fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.351+0000 7f705ef51640 1 -- 172.21.15.115:0/2258730704 >> 172.21.15.115:0/2258730704 conn(0x7f7058065960 msgr2=0x7f705815b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:33.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.351+0000 7f705ef51640 1 -- 172.21.15.115:0/2258730704 shutdown_connections 2023-12-01T01:48:33.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.351+0000 7f705ef51640 1 -- 172.21.15.115:0/2258730704 wait complete. 2023-12-01T01:48:33.371 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:37:44", "first": "2023-12-01T01:38:00", "last": "2023-12-01T01:48:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-01T01:48:33.371 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:37:44', 'first': '2023-12-01T01:38:00', 'last': '2023-12-01T01:48:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-01T01:48:33.371 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T01:48:33.371 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T01:48:33.371 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir0 1M 2023-12-01T01:48:33.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.588+0000 7fad13cd5640 1 Processor -- start 2023-12-01T01:48:33.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.589+0000 7fad13cd5640 1 -- start start 2023-12-01T01:48:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.590+0000 7fad13cd5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c15ac80 0x7fad0c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.590+0000 7fad13cd5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fad0c0ce1b0 0x7fad0c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.590+0000 7fad13cd5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0cc9a0 con 0x7fad0c0b0e60 2023-12-01T01:48:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.590+0000 7fad13cd5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c15b060 con 0x7fad0c0ce1b0 2023-12-01T01:48:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.590+0000 7fad13cd5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0b0ce0 con 0x7fad0c15ac80 2023-12-01T01:48:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.591+0000 7fad11270640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fad0c0b0e60 0x7fad0c0cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59196/0 (socket says 172.21.15.115:59196) 2023-12-01T01:48:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.591+0000 7fad11270640 1 -- 172.21.15.115:0/2667279848 learned_addr learned my addr 172.21.15.115:0/2667279848 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.591+0000 7fad12272640 1 --2- 172.21.15.115:0/2667279848 >> v2:172.21.15.142:3300/0 conn(0x7fad0c0ce1b0 0x7fad0c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.591+0000 7fad11a71640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c15ac80 0x7fad0c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.591+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550931044 0 0) 0x7fad0c0cc9a0 con 0x7fad0c0b0e60 2023-12-01T01:48:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.591+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf8003520 con 0x7fad0c0b0e60 2023-12-01T01:48:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2415514334 0 0) 0x7facf8003520 con 0x7fad0c0b0e60 2023-12-01T01:48:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c15ac80 msgr2=0x7fad0c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c15ac80 0x7fad0c0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 >> v2:172.21.15.142:3300/0 conn(0x7fad0c0ce1b0 msgr2=0x7fad0c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 --2- 172.21.15.115:0/2667279848 >> v2:172.21.15.142:3300/0 conn(0x7fad0c0ce1b0 0x7fad0c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0cbd90 con 0x7fad0c0b0e60 2023-12-01T01:48:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7facfc001670 con 0x7fad0c0b0e60 2023-12-01T01:48:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 >> v1:172.21.15.115:6789/0 conn(0x7fad0c0b0e60 legacy=0x7fad0c0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facf8003970 0x7facf8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c15ac80 0x7facf80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facf8004d00 0x7facf8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c15b060 con 0x7facf8004d00 2023-12-01T01:48:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0b0ce0 con 0x7facf8003970 2023-12-01T01:48:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf800b980 con 0x7fad0c15ac80 2023-12-01T01:48:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad12272640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facf8004d00 0x7facf8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad11270640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c15ac80 0x7facf80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad12272640 1 -- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c15ac80 msgr2=0x7facf80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad12272640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c15ac80 0x7facf80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad11a71640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facf8003970 0x7facf8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad12272640 1 -- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facf8003970 msgr2=0x7facf8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad12272640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facf8003970 0x7facf8003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.592+0000 7fad12272640 1 -- 172.21.15.115:0/2667279848 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8003520 con 0x7facf8004d00 2023-12-01T01:48:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.593+0000 7fad12272640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facf8004d00 0x7facf8005180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fad04009f90 tx=0x7facf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847994c88249eee3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.594+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0402cc20 con 0x7facf8004d00 2023-12-01T01:48:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.594+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0400f040 con 0x7facf8004d00 2023-12-01T01:48:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.594+0000 7fad10a6f640 1 -- 172.21.15.115:0/2667279848 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04034440 con 0x7facf8004d00 2023-12-01T01:48:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.594+0000 7fad13cd5640 1 -- 172.21.15.115:0/2667279848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facf8004d00 msgr2=0x7facf8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.594+0000 7fad13cd5640 1 --2- 172.21.15.115:0/2667279848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7facf8004d00 0x7facf8005180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fad04009f90 tx=0x7facf80057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.594+0000 7fad12272640 1 -- 172.21.15.115:0/2667279848 reap_dead start 2023-12-01T01:48:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.594+0000 7fad13cd5640 1 -- 172.21.15.115:0/2667279848 shutdown_connections 2023-12-01T01:48:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.594+0000 7fad13cd5640 1 -- 172.21.15.115:0/2667279848 >> 172.21.15.115:0/2667279848 conn(0x7fad0c065960 msgr2=0x7fad0c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.595+0000 7fad13cd5640 1 -- 172.21.15.115:0/2667279848 shutdown_connections 2023-12-01T01:48:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.595+0000 7fad13cd5640 1 -- 172.21.15.115:0/2667279848 wait complete. 2023-12-01T01:48:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.596+0000 7fad13cd5640 1 Processor -- start 2023-12-01T01:48:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.596+0000 7fad13cd5640 1 -- start start 2023-12-01T01:48:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.597+0000 7fad13cd5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facf8004d00 0x7fad0c0c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.597+0000 7fad13cd5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c070370 0x7fad0c1f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.597+0000 7fad13cd5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad0c1f61d0 0x7fad0c1f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.597+0000 7fad13cd5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0a9b70 con 0x7fad0c1f61d0 2023-12-01T01:48:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.597+0000 7fad13cd5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0a8f30 con 0x7facf8004d00 2023-12-01T01:48:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.597+0000 7fad13cd5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0a91d0 con 0x7fad0c070370 2023-12-01T01:48:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad12272640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad0c1f61d0 0x7fad0c1f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad12272640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad0c1f61d0 0x7fad0c1f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36628/0 (socket says 172.21.15.115:36628) 2023-12-01T01:48:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad12272640 1 -- 172.21.15.115:0/632949267 learned_addr learned my addr 172.21.15.115:0/632949267 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad11270640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c070370 0x7fad0c1f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad11a71640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facf8004d00 0x7fad0c0c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad12272640 1 -- 172.21.15.115:0/632949267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c070370 msgr2=0x7fad0c1f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad12272640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c070370 0x7fad0c1f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad12272640 1 -- 172.21.15.115:0/632949267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facf8004d00 msgr2=0x7fad0c0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad12272640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facf8004d00 0x7fad0c0c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad12272640 1 -- 172.21.15.115:0/632949267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0a94b0 con 0x7fad0c1f61d0 2023-12-01T01:48:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.598+0000 7fad11a71640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facf8004d00 0x7fad0c0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.599+0000 7fad12272640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad0c1f61d0 0x7fad0c1f6650 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fad04037040 tx=0x7fad04009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.599+0000 7fad027fc640 1 -- 172.21.15.115:0/632949267 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04034d50 con 0x7fad0c1f61d0 2023-12-01T01:48:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.599+0000 7fad027fc640 1 -- 172.21.15.115:0/632949267 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad04003990 con 0x7fad0c1f61d0 2023-12-01T01:48:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.599+0000 7fad13cd5640 1 -- 172.21.15.115:0/632949267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c0a9790 con 0x7fad0c1f61d0 2023-12-01T01:48:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.599+0000 7fad027fc640 1 -- 172.21.15.115:0/632949267 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04003d20 con 0x7fad0c1f61d0 2023-12-01T01:48:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.599+0000 7fad13cd5640 1 -- 172.21.15.115:0/632949267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c0c9850 con 0x7fad0c1f61d0 2023-12-01T01:48:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.600+0000 7fad13cd5640 1 -- 172.21.15.115:0/632949267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c0a9cb0 con 0x7fad0c1f61d0 2023-12-01T01:48:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.601+0000 7fad027fc640 1 -- 172.21.15.115:0/632949267 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fad04004030 con 0x7fad0c1f61d0 2023-12-01T01:48:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.602+0000 7fad027fc640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7facf006a630 0x7facf006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.602+0000 7fad027fc640 1 -- 172.21.15.115:0/632949267 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fad040aa570 con 0x7fad0c1f61d0 2023-12-01T01:48:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.602+0000 7fad11a71640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7facf006a630 0x7facf006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.603+0000 7fad11a71640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7facf006a630 0x7facf006caf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad080099e0 tx=0x7fad08008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.606+0000 7fad027fc640 1 -- 172.21.15.115:0/632949267 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fad0c0a9cb0 con 0x7fad0c1f61d0 2023-12-01T01:48:33.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.803+0000 7fad13cd5640 1 -- 172.21.15.115:0/632949267 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fad0c0cc350 con 0x7facf006a630 2023-12-01T01:48:34.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:33.998+0000 7fad027fc640 1 -- 172.21.15.115:0/632949267 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fad08009650 con 0x7facf006a630 2023-12-01T01:48:34.000 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-12-01T01:48:34.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.001+0000 7fad13cd5640 1 -- 172.21.15.115:0/632949267 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7facf006a630 msgr2=0x7facf006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:34.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.001+0000 7fad13cd5640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7facf006a630 0x7facf006caf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad080099e0 tx=0x7fad08008040 comp rx=0 tx=0).stop 2023-12-01T01:48:34.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.001+0000 7fad13cd5640 1 -- 172.21.15.115:0/632949267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad0c1f61d0 msgr2=0x7fad0c1f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:34.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.001+0000 7fad13cd5640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad0c1f61d0 0x7fad0c1f6650 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fad04037040 tx=0x7fad04009750 comp rx=0 tx=0).stop 2023-12-01T01:48:34.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.002+0000 7fad13cd5640 1 -- 172.21.15.115:0/632949267 shutdown_connections 2023-12-01T01:48:34.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.002+0000 7fad13cd5640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7facf006a630 0x7facf006caf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:34.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.002+0000 7fad13cd5640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad0c070370 0x7fad0c1f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:34.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.002+0000 7fad13cd5640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad0c1f61d0 0x7fad0c1f6650 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:34.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.002+0000 7fad13cd5640 1 --2- 172.21.15.115:0/632949267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facf8004d00 0x7fad0c0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:34.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.002+0000 7fad13cd5640 1 -- 172.21.15.115:0/632949267 >> 172.21.15.115:0/632949267 conn(0x7fad0c065960 msgr2=0x7fad0c0ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:34.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.002+0000 7fad13cd5640 1 -- 172.21.15.115:0/632949267 shutdown_connections 2023-12-01T01:48:34.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:34.002+0000 7fad13cd5640 1 -- 172.21.15.115:0/632949267 wait complete. 2023-12-01T01:48:34.022 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-12-01T01:48:34.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.022 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-12-01T01:48:34.076 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_01_UTC 2023-12-01T01:48:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_01_UTC 2023-12-01T01:48:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_01_UTC 2023-12-01T01:48:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_01_UTC 2023-12-01T01:48:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_01_UTC 2023-12-01T01:48:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:34.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:34.078 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:34.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.078 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_38_00_UTC) 2023-12-01T01:48:34.145 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_39_01_UTC 2023-12-01T01:48:34.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.145 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_39_01_UTC) 2023-12-01T01:48:34.214 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_40_01_UTC 2023-12-01T01:48:34.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.215 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_40_01_UTC) 2023-12-01T01:48:34.283 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_41_01_UTC 2023-12-01T01:48:34.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.283 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_41_01_UTC) 2023-12-01T01:48:34.350 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_42_01_UTC 2023-12-01T01:48:34.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.351 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_42_01_UTC) 2023-12-01T01:48:34.415 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_43_01_UTC 2023-12-01T01:48:34.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.415 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_43_01_UTC) 2023-12-01T01:48:34.482 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:34.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.482 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_44_01_UTC) 2023-12-01T01:48:34.545 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:34.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.546 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_45_01_UTC) 2023-12-01T01:48:34.612 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:34.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.612 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_46_01_UTC) 2023-12-01T01:48:34.675 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:34.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.675 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_47_00_UTC) 2023-12-01T01:48:34.736 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:34.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.736 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-01-01_48_00_UTC) 2023-12-01T01:48:34.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:34.802 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-12-01T01:48:34.857 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir1 1M 2023-12-01T01:48:35.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.071+0000 7f0aa002c640 1 Processor -- start 2023-12-01T01:48:35.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.071+0000 7f0aa002c640 1 -- start start 2023-12-01T01:48:35.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.072+0000 7f0aa002c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9815cf70 0x7f0a980cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:35.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.072+0000 7f0aa002c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0a980b4d60 0x7f0a980b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:35.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.072+0000 7f0aa002c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a980d2a10 con 0x7f0a980d0db0 2023-12-01T01:48:35.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.073+0000 7f0aa002c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9815d350 con 0x7f0a980b4d60 2023-12-01T01:48:35.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.073+0000 7f0aa002c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980d1e50 con 0x7f0a9815cf70 2023-12-01T01:48:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.073+0000 7f0a9d5c7640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0a980d0db0 0x7f0a980b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59202/0 (socket says 172.21.15.115:59202) 2023-12-01T01:48:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.073+0000 7f0a9d5c7640 1 -- 172.21.15.115:0/3599708459 learned_addr learned my addr 172.21.15.115:0/3599708459 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.073+0000 7f0a9e5c9640 1 --2- 172.21.15.115:0/3599708459 >> v2:172.21.15.142:3300/0 conn(0x7f0a980b4d60 0x7f0a980b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.073+0000 7f0a9ddc8640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9815cf70 0x7f0a980cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.073+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947594330 0 0) 0x7f0a980d2a10 con 0x7f0a980d0db0 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.073+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a88003520 con 0x7f0a980d0db0 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 195754127 0 0) 0x7f0a88003520 con 0x7f0a980d0db0 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9815cf70 msgr2=0x7f0a980cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9815cf70 0x7f0a980cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 >> v2:172.21.15.142:3300/0 conn(0x7f0a980b4d60 msgr2=0x7f0a980b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 --2- 172.21.15.115:0/3599708459 >> v2:172.21.15.142:3300/0 conn(0x7f0a980b4d60 0x7f0a980b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a980d1240 con 0x7f0a980d0db0 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0a84001670 con 0x7f0a980d0db0 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 >> v1:172.21.15.115:6789/0 conn(0x7f0a980d0db0 legacy=0x7f0a980b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88003970 0x7f0a88003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9815cf70 0x7f0a880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a88004d00 0x7f0a88005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9815d350 con 0x7f0a88004d00 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980d1e50 con 0x7f0a88003970 2023-12-01T01:48:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8800b980 con 0x7f0a9815cf70 2023-12-01T01:48:35.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9e5c9640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a88004d00 0x7f0a88005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:35.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.074+0000 7f0a9ddc8640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88003970 0x7f0a88003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:35.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.075+0000 7f0a9d5c7640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9815cf70 0x7f0a880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:35.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.075+0000 7f0a9e5c9640 1 -- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9815cf70 msgr2=0x7f0a880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:35.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.075+0000 7f0a9e5c9640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9815cf70 0x7f0a880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.075+0000 7f0a9e5c9640 1 -- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88003970 msgr2=0x7f0a88003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:35.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.075+0000 7f0a9e5c9640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a88003970 0x7f0a88003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.075+0000 7f0a9e5c9640 1 -- 172.21.15.115:0/3599708459 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a88003520 con 0x7f0a88004d00 2023-12-01T01:48:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.076+0000 7f0a9e5c9640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a88004d00 0x7f0a88005180 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009a90 tx=0x7f0a880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3655bd9396cda8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.076+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9002c6b0 con 0x7f0a88004d00 2023-12-01T01:48:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.076+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9002c810 con 0x7f0a88004d00 2023-12-01T01:48:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.076+0000 7f0a9cdc6640 1 -- 172.21.15.115:0/3599708459 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9002cba0 con 0x7f0a88004d00 2023-12-01T01:48:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.077+0000 7f0aa002c640 1 -- 172.21.15.115:0/3599708459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a88004d00 msgr2=0x7f0a88005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.077+0000 7f0aa002c640 1 --2- 172.21.15.115:0/3599708459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a88004d00 0x7f0a88005180 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009a90 tx=0x7f0a880057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.077+0000 7f0a9e5c9640 1 -- 172.21.15.115:0/3599708459 reap_dead start 2023-12-01T01:48:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.077+0000 7f0aa002c640 1 -- 172.21.15.115:0/3599708459 shutdown_connections 2023-12-01T01:48:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.077+0000 7f0aa002c640 1 -- 172.21.15.115:0/3599708459 >> 172.21.15.115:0/3599708459 conn(0x7f0a98065d80 msgr2=0x7f0a98066190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.077+0000 7f0aa002c640 1 -- 172.21.15.115:0/3599708459 shutdown_connections 2023-12-01T01:48:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.077+0000 7f0aa002c640 1 -- 172.21.15.115:0/3599708459 wait complete. 2023-12-01T01:48:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.079+0000 7f0aa002c640 1 Processor -- start 2023-12-01T01:48:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.079+0000 7f0aa002c640 1 -- start start 2023-12-01T01:48:35.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.080+0000 7f0aa002c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a88004d00 0x7f0a980cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:35.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.080+0000 7f0aa002c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a981556b0 0x7f0a981e66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:35.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.080+0000 7f0aa002c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a980cb980 0x7f0a981e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:35.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.080+0000 7f0aa002c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980cc0e0 con 0x7f0a980cb980 2023-12-01T01:48:35.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.080+0000 7f0aa002c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9815dc80 con 0x7f0a981556b0 2023-12-01T01:48:35.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.080+0000 7f0aa002c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980cbe00 con 0x7f0a88004d00 2023-12-01T01:48:35.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.080+0000 7f0a9e5c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a980cb980 0x7f0a981e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:35.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.080+0000 7f0a9e5c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a980cb980 0x7f0a981e6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36660/0 (socket says 172.21.15.115:36660) 2023-12-01T01:48:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.080+0000 7f0a9e5c9640 1 -- 172.21.15.115:0/3563154287 learned_addr learned my addr 172.21.15.115:0/3563154287 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a9ddc8640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a88004d00 0x7f0a980cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a9d5c7640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a981556b0 0x7f0a981e66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a9e5c9640 1 -- 172.21.15.115:0/3563154287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a88004d00 msgr2=0x7f0a980cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a9e5c9640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a88004d00 0x7f0a980cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a9e5c9640 1 -- 172.21.15.115:0/3563154287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a981556b0 msgr2=0x7f0a981e66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a9e5c9640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a981556b0 0x7f0a981e66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a9e5c9640 1 -- 172.21.15.115:0/3563154287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9815d630 con 0x7f0a980cb980 2023-12-01T01:48:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a9ddc8640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a88004d00 0x7f0a980cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a9e5c9640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a980cb980 0x7f0a981e6be0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009a90 tx=0x7f0a90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:35.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a967fc640 1 -- 172.21.15.115:0/3563154287 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90003810 con 0x7f0a980cb980 2023-12-01T01:48:35.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a967fc640 1 -- 172.21.15.115:0/3563154287 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a90003970 con 0x7f0a980cb980 2023-12-01T01:48:35.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0aa002c640 1 -- 172.21.15.115:0/3563154287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9815d910 con 0x7f0a980cb980 2023-12-01T01:48:35.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.081+0000 7f0a967fc640 1 -- 172.21.15.115:0/3563154287 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90003d20 con 0x7f0a980cb980 2023-12-01T01:48:35.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.082+0000 7f0aa002c640 1 -- 172.21.15.115:0/3563154287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a98155e90 con 0x7f0a980cb980 2023-12-01T01:48:35.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.083+0000 7f0aa002c640 1 -- 172.21.15.115:0/3563154287 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9815d490 con 0x7f0a980cb980 2023-12-01T01:48:35.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.084+0000 7f0a967fc640 1 -- 172.21.15.115:0/3563154287 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f0a9003e440 con 0x7f0a980cb980 2023-12-01T01:48:35.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.085+0000 7f0a967fc640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a7c06a660 0x7f0a7c06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:35.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.085+0000 7f0a967fc640 1 -- 172.21.15.115:0/3563154287 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f0a90000e40 con 0x7f0a980cb980 2023-12-01T01:48:35.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.085+0000 7f0a9ddc8640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a7c06a660 0x7f0a7c06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:35.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.086+0000 7f0a9ddc8640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a7c06a660 0x7f0a7c06cb20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c007440 tx=0x7f0a8c00b040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:35.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.089+0000 7f0a967fc640 1 -- 172.21.15.115:0/3563154287 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0a9815d490 con 0x7f0a980cb980 2023-12-01T01:48:35.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.302+0000 7f0aa002c640 1 -- 172.21.15.115:0/3563154287 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a980d1800 con 0x7f0a7c06a660 2023-12-01T01:48:35.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.489+0000 7f0a967fc640 1 -- 172.21.15.115:0/3563154287 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f0a8c018470 con 0x7f0a7c06a660 2023-12-01T01:48:35.491 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-12-01T01:48:35.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.492+0000 7f0aa002c640 1 -- 172.21.15.115:0/3563154287 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a7c06a660 msgr2=0x7f0a7c06cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:35.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.492+0000 7f0aa002c640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a7c06a660 0x7f0a7c06cb20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c007440 tx=0x7f0a8c00b040 comp rx=0 tx=0).stop 2023-12-01T01:48:35.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.492+0000 7f0aa002c640 1 -- 172.21.15.115:0/3563154287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a980cb980 msgr2=0x7f0a981e6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:35.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.492+0000 7f0aa002c640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a980cb980 0x7f0a981e6be0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009a90 tx=0x7f0a90004730 comp rx=0 tx=0).stop 2023-12-01T01:48:35.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.493+0000 7f0aa002c640 1 -- 172.21.15.115:0/3563154287 shutdown_connections 2023-12-01T01:48:35.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.493+0000 7f0aa002c640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a7c06a660 0x7f0a7c06cb20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.493+0000 7f0aa002c640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a980cb980 0x7f0a981e6be0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.493+0000 7f0aa002c640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a981556b0 0x7f0a981e66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.493+0000 7f0aa002c640 1 --2- 172.21.15.115:0/3563154287 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a88004d00 0x7f0a980cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:35.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.493+0000 7f0aa002c640 1 -- 172.21.15.115:0/3563154287 >> 172.21.15.115:0/3563154287 conn(0x7f0a98065d80 msgr2=0x7f0a980b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:35.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.493+0000 7f0aa002c640 1 -- 172.21.15.115:0/3563154287 shutdown_connections 2023-12-01T01:48:35.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:35.493+0000 7f0aa002c640 1 -- 172.21.15.115:0/3563154287 wait complete. 2023-12-01T01:48:35.515 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-12-01T01:48:35.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:35.516 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-12-01T01:48:35.568 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_01_UTC 2023-12-01T01:48:35.569 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_01_UTC 2023-12-01T01:48:35.569 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:35.569 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:35.569 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_01_UTC 2023-12-01T01:48:35.569 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_01_UTC 2023-12-01T01:48:35.569 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:35.569 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:35.569 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:35.569 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:35.569 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:35.570 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_38_01_UTC 2023-12-01T01:48:35.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:35.570 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_38_01_UTC) 2023-12-01T01:48:35.637 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_39_01_UTC 2023-12-01T01:48:35.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:35.638 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_39_01_UTC) 2023-12-01T01:48:35.701 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:35.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:35.702 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_40_00_UTC) 2023-12-01T01:48:35.773 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:35.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:35.774 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_41_00_UTC) 2023-12-01T01:48:35.834 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_42_01_UTC 2023-12-01T01:48:35.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:35.834 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_42_01_UTC) 2023-12-01T01:48:35.897 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_43_01_UTC 2023-12-01T01:48:35.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:35.898 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_43_01_UTC) 2023-12-01T01:48:35.961 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:35.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:35.961 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_44_01_UTC) 2023-12-01T01:48:36.032 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:36.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:36.032 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_45_01_UTC) 2023-12-01T01:48:36.102 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:36.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:36.103 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_46_01_UTC) 2023-12-01T01:48:36.172 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:36.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:36.172 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_47_00_UTC) 2023-12-01T01:48:36.239 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:36.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:36.240 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-01-01_48_00_UTC) 2023-12-01T01:48:36.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:36.300 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-12-01T01:48:36.354 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir2 1M 2023-12-01T01:48:36.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.583+0000 7f8928865640 1 Processor -- start 2023-12-01T01:48:36.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.583+0000 7f8928865640 1 -- start start 2023-12-01T01:48:36.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.584+0000 7f8928865640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f89240caa30 0x7f8924161350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:36.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.584+0000 7f8928865640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f89240cae10 0x7f89240ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:36.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.584+0000 7f8928865640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89240ae2f0 con 0x7f89240ac8a0 2023-12-01T01:48:36.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.584+0000 7f8928865640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89240ce560 con 0x7f89240caa30 2023-12-01T01:48:36.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.584+0000 7f8928865640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89240ae590 con 0x7f89240cae10 2023-12-01T01:48:36.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.585+0000 7f89237fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f89240caa30 0x7f8924161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:36.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.585+0000 7f8923fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f89240ac8a0 0x7f89240cbe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59204/0 (socket says 172.21.15.115:59204) 2023-12-01T01:48:36.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.585+0000 7f8923fff640 1 -- 172.21.15.115:0/4126862459 learned_addr learned my addr 172.21.15.115:0/4126862459 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:36.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.585+0000 7f8922ffd640 1 --2- 172.21.15.115:0/4126862459 >> v2:172.21.15.142:3300/0 conn(0x7f89240cae10 0x7f89240ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:36.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.585+0000 7f89237fe640 1 -- 172.21.15.115:0/4126862459 >> v2:172.21.15.142:3300/0 conn(0x7f89240cae10 msgr2=0x7f89240ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:36.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.585+0000 7f89237fe640 1 --2- 172.21.15.115:0/4126862459 >> v2:172.21.15.142:3300/0 conn(0x7f89240cae10 0x7f89240ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:36.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.585+0000 7f89237fe640 1 -- 172.21.15.115:0/4126862459 >> v1:172.21.15.115:6789/0 conn(0x7f89240ac8a0 legacy=0x7f89240cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:36.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.585+0000 7f89237fe640 1 -- 172.21.15.115:0/4126862459 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89240ce820 con 0x7f89240caa30 2023-12-01T01:48:36.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.585+0000 7f8922ffd640 1 --2- 172.21.15.115:0/4126862459 >> v2:172.21.15.142:3300/0 conn(0x7f89240cae10 0x7f89240ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:48:36.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.586+0000 7f89237fe640 1 --2- 172.21.15.115:0/4126862459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240caa30 0x7f8924161350 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f89240cca10 tx=0x7f891402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2461cbb48f82e84b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:36.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.587+0000 7f89227fc640 1 -- 172.21.15.115:0/4126862459 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891402e070 con 0x7f89240caa30 2023-12-01T01:48:36.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.587+0000 7f89227fc640 1 -- 172.21.15.115:0/4126862459 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89140315a0 con 0x7f89240caa30 2023-12-01T01:48:36.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.587+0000 7f89227fc640 1 -- 172.21.15.115:0/4126862459 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914031930 con 0x7f89240caa30 2023-12-01T01:48:36.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.587+0000 7f8928865640 1 -- 172.21.15.115:0/4126862459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240caa30 msgr2=0x7f8924161350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:36.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.587+0000 7f8928865640 1 --2- 172.21.15.115:0/4126862459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240caa30 0x7f8924161350 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f89240cca10 tx=0x7f891402b840 comp rx=0 tx=0).stop 2023-12-01T01:48:36.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.588+0000 7f8928865640 1 -- 172.21.15.115:0/4126862459 shutdown_connections 2023-12-01T01:48:36.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.588+0000 7f8928865640 1 --2- 172.21.15.115:0/4126862459 >> v2:172.21.15.142:3300/0 conn(0x7f89240cae10 0x7f89240ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:36.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.588+0000 7f8928865640 1 --2- 172.21.15.115:0/4126862459 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240caa30 0x7f8924161350 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:36.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.588+0000 7f8928865640 1 -- 172.21.15.115:0/4126862459 >> 172.21.15.115:0/4126862459 conn(0x7f892406a0e0 msgr2=0x7f892406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:36.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.588+0000 7f8928865640 1 -- 172.21.15.115:0/4126862459 shutdown_connections 2023-12-01T01:48:36.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.588+0000 7f8928865640 1 -- 172.21.15.115:0/4126862459 wait complete. 2023-12-01T01:48:36.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.590+0000 7f8928865640 1 Processor -- start 2023-12-01T01:48:36.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.590+0000 7f8928865640 1 -- start start 2023-12-01T01:48:36.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f8928865640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240ac8a0 0x7f89240b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:36.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f8928865640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89240caa30 0x7f89240b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:36.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f8928865640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89240cae10 0x7f892415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:36.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f8928865640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89241e6100 con 0x7f89240ac8a0 2023-12-01T01:48:36.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f8928865640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89241e63e0 con 0x7f89240cae10 2023-12-01T01:48:36.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f8928865640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89241e66c0 con 0x7f89240caa30 2023-12-01T01:48:36.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f89237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240ac8a0 0x7f89240b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:36.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f89237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240ac8a0 0x7f89240b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36684/0 (socket says 172.21.15.115:36684) 2023-12-01T01:48:36.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f89237fe640 1 -- 172.21.15.115:0/1975724125 learned_addr learned my addr 172.21.15.115:0/1975724125 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:36.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f8923fff640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89240cae10 0x7f892415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:36.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f89237fe640 1 -- 172.21.15.115:0/1975724125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89240caa30 msgr2=0x7f89240b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:36.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f89237fe640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89240caa30 0x7f89240b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:36.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.591+0000 7f89237fe640 1 -- 172.21.15.115:0/1975724125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89240cae10 msgr2=0x7f892415b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:36.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.592+0000 7f89237fe640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89240cae10 0x7f892415b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:36.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.592+0000 7f89237fe640 1 -- 172.21.15.115:0/1975724125 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8914009400 con 0x7f89240ac8a0 2023-12-01T01:48:36.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.592+0000 7f89237fe640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240ac8a0 0x7f89240b4a30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8914000c00 tx=0x7f89140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.592+0000 7f890bfff640 1 -- 172.21.15.115:0/1975724125 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914032900 con 0x7f89240ac8a0 2023-12-01T01:48:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.592+0000 7f890bfff640 1 -- 172.21.15.115:0/1975724125 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8914032a80 con 0x7f89240ac8a0 2023-12-01T01:48:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.592+0000 7f890bfff640 1 -- 172.21.15.115:0/1975724125 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914032e30 con 0x7f89240ac8a0 2023-12-01T01:48:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.592+0000 7f8928865640 1 -- 172.21.15.115:0/1975724125 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89241e69a0 con 0x7f89240ac8a0 2023-12-01T01:48:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.592+0000 7f8928865640 1 -- 172.21.15.115:0/1975724125 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89240c4e80 con 0x7f89240ac8a0 2023-12-01T01:48:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.594+0000 7f8928865640 1 -- 172.21.15.115:0/1975724125 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89240618d0 con 0x7f89240ac8a0 2023-12-01T01:48:36.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.594+0000 7f890bfff640 1 -- 172.21.15.115:0/1975724125 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f891404d3f0 con 0x7f89240ac8a0 2023-12-01T01:48:36.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.595+0000 7f890bfff640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f891006a630 0x7f891006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:36.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.595+0000 7f890bfff640 1 -- 172.21.15.115:0/1975724125 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6523+0+0 (secure 0 0 0) 0x7f89140aad60 con 0x7f89240ac8a0 2023-12-01T01:48:36.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.596+0000 7f8922ffd640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f891006a630 0x7f891006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:36.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.596+0000 7f8922ffd640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f891006a630 0x7f891006caf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8918006fd0 tx=0x7f8918008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:36.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.600+0000 7f890bfff640 1 -- 172.21.15.115:0/1975724125 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8914075bf0 con 0x7f89240ac8a0 2023-12-01T01:48:36.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:36.842+0000 7f8928865640 1 -- 172.21.15.115:0/1975724125 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f89240cd040 con 0x7f891006a630 2023-12-01T01:48:37.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.028+0000 7f890bfff640 1 -- 172.21.15.115:0/1975724125 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f891800be60 con 0x7f891006a630 2023-12-01T01:48:37.030 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-12-01T01:48:37.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 -- 172.21.15.115:0/1975724125 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f891006a630 msgr2=0x7f891006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:37.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f891006a630 0x7f891006caf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8918006fd0 tx=0x7f8918008040 comp rx=0 tx=0).stop 2023-12-01T01:48:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 -- 172.21.15.115:0/1975724125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240ac8a0 msgr2=0x7f89240b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240ac8a0 0x7f89240b4a30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8914000c00 tx=0x7f89140046c0 comp rx=0 tx=0).stop 2023-12-01T01:48:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 -- 172.21.15.115:0/1975724125 shutdown_connections 2023-12-01T01:48:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f891006a630 0x7f891006caf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89240caa30 0x7f89240b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89240cae10 0x7f892415b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 --2- 172.21.15.115:0/1975724125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89240ac8a0 0x7f89240b4a30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 -- 172.21.15.115:0/1975724125 >> 172.21.15.115:0/1975724125 conn(0x7f892406a0e0 msgr2=0x7f892415e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 -- 172.21.15.115:0/1975724125 shutdown_connections 2023-12-01T01:48:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:37.032+0000 7f8928865640 1 -- 172.21.15.115:0/1975724125 wait complete. 2023-12-01T01:48:37.052 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-12-01T01:48:37.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.052 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-12-01T01:48:37.106 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:37.106 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:37.106 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:37.107 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:37.107 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:37.107 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:37.107 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:37.107 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:37.107 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:37.107 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_01_UTC 2023-12-01T01:48:37.107 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:37.108 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:37.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.108 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_38_00_UTC) 2023-12-01T01:48:37.168 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:37.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.169 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_39_00_UTC) 2023-12-01T01:48:37.219 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:37.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.219 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_40_00_UTC) 2023-12-01T01:48:37.271 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:37.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.272 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_41_00_UTC) 2023-12-01T01:48:37.335 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:37.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.335 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_42_00_UTC) 2023-12-01T01:48:37.398 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:37.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.398 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_43_00_UTC) 2023-12-01T01:48:37.466 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:37.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.466 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_44_00_UTC) 2023-12-01T01:48:37.556 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:37.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.556 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_45_01_UTC) 2023-12-01T01:48:37.627 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:37.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.627 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_46_01_UTC) 2023-12-01T01:48:37.688 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_47_01_UTC 2023-12-01T01:48:37.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.688 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_47_01_UTC) 2023-12-01T01:48:37.736 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:37.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.737 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-01-01_48_01_UTC) 2023-12-01T01:48:37.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:37.787 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-12-01T01:48:37.834 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir3 1M 2023-12-01T01:48:38.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.098+0000 7f1017f73640 1 Processor -- start 2023-12-01T01:48:38.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.098+0000 7f1017f73640 1 -- start start 2023-12-01T01:48:38.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.099+0000 7f1017f73640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1010160200 0x7f10100b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.100+0000 7f1017f73640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10100cb290 con 0x7f10100b0f80 2023-12-01T01:48:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.100+0000 7f1017f73640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10100b0ae0 con 0x7f10100cbd30 2023-12-01T01:48:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.100+0000 7f1017f73640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10100b35f0 con 0x7f1010160200 2023-12-01T01:48:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.100+0000 7f1015d0f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1010160200 0x7f10100b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.100+0000 7f1015d0f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1010160200 0x7f10100b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48954/0 (socket says 172.21.15.115:48954) 2023-12-01T01:48:38.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.100+0000 7f1015d0f640 1 -- 172.21.15.115:0/3718025656 learned_addr learned my addr 172.21.15.115:0/3718025656 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:38.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.100+0000 7f101550e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f10100b0f80 0x7f101006b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48878/0 (socket says 172.21.15.115:48878) 2023-12-01T01:48:38.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.100+0000 7f1015d0f640 1 -- 172.21.15.115:0/3718025656 >> v1:172.21.15.142:6789/0 conn(0x7f10100cbd30 legacy=0x7f10100cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:38.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.100+0000 7f1015d0f640 1 -- 172.21.15.115:0/3718025656 >> v1:172.21.15.115:6789/0 conn(0x7f10100b0f80 legacy=0x7f101006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:38.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.100+0000 7f1015d0f640 1 -- 172.21.15.115:0/3718025656 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10100b3310 con 0x7f1010160200 2023-12-01T01:48:38.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.101+0000 7f1015d0f640 1 --2- 172.21.15.115:0/3718025656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1010160200 0x7f10100b2c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f100c009870 tx=0x7f100c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306e5b8178ba6ce9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:38.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.102+0000 7f1014d0d640 1 -- 172.21.15.115:0/3718025656 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c030860 con 0x7f1010160200 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.102+0000 7f1014d0d640 1 -- 172.21.15.115:0/3718025656 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c0309c0 con 0x7f1010160200 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.102+0000 7f1014d0d640 1 -- 172.21.15.115:0/3718025656 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c030d50 con 0x7f1010160200 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.102+0000 7f1017f73640 1 -- 172.21.15.115:0/3718025656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1010160200 msgr2=0x7f10100b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.102+0000 7f1017f73640 1 --2- 172.21.15.115:0/3718025656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1010160200 0x7f10100b2c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f100c009870 tx=0x7f100c02b9a0 comp rx=0 tx=0).stop 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.103+0000 7f1017f73640 1 -- 172.21.15.115:0/3718025656 shutdown_connections 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.103+0000 7f1017f73640 1 --2- 172.21.15.115:0/3718025656 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1010160200 0x7f10100b2c80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.103+0000 7f1017f73640 1 -- 172.21.15.115:0/3718025656 >> 172.21.15.115:0/3718025656 conn(0x7f1010065960 msgr2=0x7f1010065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.103+0000 7f1017f73640 1 -- 172.21.15.115:0/3718025656 shutdown_connections 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.103+0000 7f1017f73640 1 -- 172.21.15.115:0/3718025656 wait complete. 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.105+0000 7f1017f73640 1 Processor -- start 2023-12-01T01:48:38.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.105+0000 7f1017f73640 1 -- start start 2023-12-01T01:48:38.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.106+0000 7f1017f73640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10100b0f80 0x7f1010145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:38.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.106+0000 7f1017f73640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10100cbd30 0x7f101014f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:38.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.106+0000 7f1017f73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010160200 0x7f10101514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:38.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.106+0000 7f1017f73640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10100c94e0 con 0x7f10100cbd30 2023-12-01T01:48:38.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.106+0000 7f1017f73640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101014ff60 con 0x7f1010160200 2023-12-01T01:48:38.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.106+0000 7f1017f73640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10100c8b40 con 0x7f10100b0f80 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.106+0000 7f101550e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10100cbd30 0x7f101014f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.106+0000 7f101550e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10100cbd30 0x7f101014f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48966/0 (socket says 172.21.15.115:48966) 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.106+0000 7f101550e640 1 -- 172.21.15.115:0/852980381 learned_addr learned my addr 172.21.15.115:0/852980381 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f1015d0f640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10100b0f80 0x7f1010145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f1016510640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010160200 0x7f10101514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f101550e640 1 -- 172.21.15.115:0/852980381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10100b0f80 msgr2=0x7f1010145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f101550e640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10100b0f80 0x7f1010145180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f101550e640 1 -- 172.21.15.115:0/852980381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010160200 msgr2=0x7f10101514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f101550e640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010160200 0x7f10101514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f101550e640 1 -- 172.21.15.115:0/852980381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c0094d0 con 0x7f10100cbd30 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f101550e640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10100cbd30 0x7f101014f510 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f100000c3e0 tx=0x7f100000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f10067fc640 1 -- 172.21.15.115:0/852980381 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10000074e0 con 0x7f10100cbd30 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f10067fc640 1 -- 172.21.15.115:0/852980381 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1000007640 con 0x7f10100cbd30 2023-12-01T01:48:38.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f10067fc640 1 -- 172.21.15.115:0/852980381 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10000049e0 con 0x7f10100cbd30 2023-12-01T01:48:38.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.107+0000 7f1017f73640 1 -- 172.21.15.115:0/852980381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10100c8e20 con 0x7f10100cbd30 2023-12-01T01:48:38.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.108+0000 7f1017f73640 1 -- 172.21.15.115:0/852980381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10101456c0 con 0x7f10100cbd30 2023-12-01T01:48:38.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.109+0000 7f1017f73640 1 -- 172.21.15.115:0/852980381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10100c9620 con 0x7f10100cbd30 2023-12-01T01:48:38.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.110+0000 7f10067fc640 1 -- 172.21.15.115:0/852980381 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f1000004b40 con 0x7f10100cbd30 2023-12-01T01:48:38.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.110+0000 7f10067fc640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ff406a630 0x7f0ff406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:38.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.111+0000 7f10067fc640 1 -- 172.21.15.115:0/852980381 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6459+0+0 (secure 0 0 0) 0x7f1000088ab0 con 0x7f10100cbd30 2023-12-01T01:48:38.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.111+0000 7f1015d0f640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ff406a630 0x7f0ff406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:38.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.111+0000 7f1015d0f640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ff406a630 0x7f0ff406caf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f100c036000 tx=0x7f100c03b040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:38.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.116+0000 7f10067fc640 1 -- 172.21.15.115:0/852980381 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f10100c9620 con 0x7f10100cbd30 2023-12-01T01:48:38.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.332+0000 7f1017f73640 1 -- 172.21.15.115:0/852980381 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f10100cb810 con 0x7f0ff406a630 2023-12-01T01:48:38.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.566+0000 7f10067fc640 1 -- 172.21.15.115:0/852980381 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f100c030bb0 con 0x7f0ff406a630 2023-12-01T01:48:38.568 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-12-01T01:48:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.571+0000 7f1017f73640 1 -- 172.21.15.115:0/852980381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ff406a630 msgr2=0x7f0ff406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.571+0000 7f1017f73640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ff406a630 0x7f0ff406caf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f100c036000 tx=0x7f100c03b040 comp rx=0 tx=0).stop 2023-12-01T01:48:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.571+0000 7f1017f73640 1 -- 172.21.15.115:0/852980381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10100cbd30 msgr2=0x7f101014f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.571+0000 7f1017f73640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10100cbd30 0x7f101014f510 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f100000c3e0 tx=0x7f100000c8a0 comp rx=0 tx=0).stop 2023-12-01T01:48:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.572+0000 7f1017f73640 1 -- 172.21.15.115:0/852980381 shutdown_connections 2023-12-01T01:48:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.572+0000 7f1017f73640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0ff406a630 0x7f0ff406caf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.572+0000 7f1017f73640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1010160200 0x7f10101514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.572+0000 7f1017f73640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10100cbd30 0x7f101014f510 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.572+0000 7f1017f73640 1 --2- 172.21.15.115:0/852980381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10100b0f80 0x7f1010145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.572+0000 7f1017f73640 1 -- 172.21.15.115:0/852980381 >> 172.21.15.115:0/852980381 conn(0x7f1010065960 msgr2=0x7f101015c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.572+0000 7f1017f73640 1 -- 172.21.15.115:0/852980381 shutdown_connections 2023-12-01T01:48:38.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:38.572+0000 7f1017f73640 1 -- 172.21.15.115:0/852980381 wait complete. 2023-12-01T01:48:38.601 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-12-01T01:48:38.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:38.601 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-12-01T01:48:38.642 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:38.642 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:38.642 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_01_UTC 2023-12-01T01:48:38.642 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_01_UTC 2023-12-01T01:48:38.642 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_01_UTC 2023-12-01T01:48:38.642 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_01_UTC 2023-12-01T01:48:38.643 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:38.643 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:38.643 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:38.643 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:38.643 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:38.644 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:38.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:38.644 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_38_00_UTC) 2023-12-01T01:48:38.754 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:38.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:38.755 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_39_00_UTC) 2023-12-01T01:48:38.825 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_40_01_UTC 2023-12-01T01:48:38.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:38.826 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_40_01_UTC) 2023-12-01T01:48:38.890 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_41_01_UTC 2023-12-01T01:48:38.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:38.890 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_41_01_UTC) 2023-12-01T01:48:38.954 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_42_01_UTC 2023-12-01T01:48:38.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:38.954 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_42_01_UTC) 2023-12-01T01:48:39.024 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_43_01_UTC 2023-12-01T01:48:39.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:39.024 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_43_01_UTC) 2023-12-01T01:48:39.092 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:39.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:39.092 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_44_01_UTC) 2023-12-01T01:48:39.163 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:39.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:39.163 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_45_00_UTC) 2023-12-01T01:48:39.232 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:39.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:39.232 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_46_00_UTC) 2023-12-01T01:48:39.301 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:39.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:39.301 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_47_00_UTC) 2023-12-01T01:48:39.362 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:39.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:39.362 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-01-01_48_01_UTC) 2023-12-01T01:48:39.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:39.415 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-12-01T01:48:39.468 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir4 1M 2023-12-01T01:48:39.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.669+0000 7f79fb696640 1 Processor -- start 2023-12-01T01:48:39.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.669+0000 7f79fb696640 1 -- start start 2023-12-01T01:48:39.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79fb696640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f79f4142870 0x7f79f40ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:39.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79fb696640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79f40cd010 0x7f79f40b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:39.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79fb696640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f40b1c50 0x7f79f415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:39.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79fb696640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f40b3ae0 con 0x7f79f4142870 2023-12-01T01:48:39.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79fb696640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f40cd4a0 con 0x7f79f40cd010 2023-12-01T01:48:39.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79fb696640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f4142c50 con 0x7f79f40b1c50 2023-12-01T01:48:39.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79f9432640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f79f4142870 0x7f79f40ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:39.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79f9432640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f79f4142870 0x7f79f40ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48976/0 (socket says 172.21.15.115:48976) 2023-12-01T01:48:39.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79f9432640 1 -- 172.21.15.115:0/2177142401 learned_addr learned my addr 172.21.15.115:0/2177142401 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:39.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79f9432640 1 -- 172.21.15.115:0/2177142401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f40b1c50 msgr2=0x7f79f415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:39.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79f8c31640 1 --2- 172.21.15.115:0/2177142401 >> v2:172.21.15.142:3300/0 conn(0x7f79f40cd010 0x7f79f40b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:39.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79f9c33640 1 --2- 172.21.15.115:0/2177142401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f40b1c50 0x7f79f415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:39.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79f9432640 1 --2- 172.21.15.115:0/2177142401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f40b1c50 0x7f79f415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:39.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79f9432640 1 -- 172.21.15.115:0/2177142401 >> v2:172.21.15.142:3300/0 conn(0x7f79f40cd010 msgr2=0x7f79f40b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:39.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79f9432640 1 --2- 172.21.15.115:0/2177142401 >> v2:172.21.15.142:3300/0 conn(0x7f79f40cd010 0x7f79f40b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:39.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.671+0000 7f79f9432640 1 -- 172.21.15.115:0/2177142401 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f40b20e0 con 0x7f79f4142870 2023-12-01T01:48:39.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.673+0000 7f79f9432640 1 --2- 172.21.15.115:0/2177142401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f4142870 0x7f79f40ccad0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f79f40cf7f0 tx=0x7f79e802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc4ca583a713900 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:39.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.673+0000 7f79e7fff640 1 -- 172.21.15.115:0/2177142401 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8031860 con 0x7f79f4142870 2023-12-01T01:48:39.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.673+0000 7f79e7fff640 1 -- 172.21.15.115:0/2177142401 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e80319c0 con 0x7f79f4142870 2023-12-01T01:48:39.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.673+0000 7f79e7fff640 1 -- 172.21.15.115:0/2177142401 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8031d50 con 0x7f79f4142870 2023-12-01T01:48:39.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.674+0000 7f79fb696640 1 -- 172.21.15.115:0/2177142401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f4142870 msgr2=0x7f79f40ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:39.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.674+0000 7f79fb696640 1 --2- 172.21.15.115:0/2177142401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f4142870 0x7f79f40ccad0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f79f40cf7f0 tx=0x7f79e802b900 comp rx=0 tx=0).stop 2023-12-01T01:48:39.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.674+0000 7f79fb696640 1 -- 172.21.15.115:0/2177142401 shutdown_connections 2023-12-01T01:48:39.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.674+0000 7f79fb696640 1 --2- 172.21.15.115:0/2177142401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f40b1c50 0x7f79f415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:39.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.674+0000 7f79fb696640 1 --2- 172.21.15.115:0/2177142401 >> v2:172.21.15.142:3300/0 conn(0x7f79f40cd010 0x7f79f40b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:39.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.674+0000 7f79fb696640 1 --2- 172.21.15.115:0/2177142401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f4142870 0x7f79f40ccad0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:39.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.674+0000 7f79fb696640 1 -- 172.21.15.115:0/2177142401 >> 172.21.15.115:0/2177142401 conn(0x7f79f4065960 msgr2=0x7f79f4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:39.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.674+0000 7f79fb696640 1 -- 172.21.15.115:0/2177142401 shutdown_connections 2023-12-01T01:48:39.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.674+0000 7f79fb696640 1 -- 172.21.15.115:0/2177142401 wait complete. 2023-12-01T01:48:39.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.676+0000 7f79fb696640 1 Processor -- start 2023-12-01T01:48:39.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.676+0000 7f79fb696640 1 -- start start 2023-12-01T01:48:39.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.677+0000 7f79fb696640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f40b1c50 0x7f79f40c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:39.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.677+0000 7f79fb696640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40cd010 0x7f79f40c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:39.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.677+0000 7f79fb696640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f4142870 0x7f79f4158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:39.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.677+0000 7f79fb696640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40c3f60 con 0x7f79f40cd010 2023-12-01T01:48:39.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.677+0000 7f79fb696640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40c4240 con 0x7f79f40b1c50 2023-12-01T01:48:39.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.677+0000 7f79fb696640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40c4520 con 0x7f79f4142870 2023-12-01T01:48:39.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.677+0000 7f79f8c31640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40cd010 0x7f79f40c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:39.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.677+0000 7f79f8c31640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40cd010 0x7f79f40c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48992/0 (socket says 172.21.15.115:48992) 2023-12-01T01:48:39.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.677+0000 7f79f8c31640 1 -- 172.21.15.115:0/750871829 learned_addr learned my addr 172.21.15.115:0/750871829 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:39.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.677+0000 7f79f9432640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f40b1c50 0x7f79f40c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:39.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79f9c33640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f4142870 0x7f79f4158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:39.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79f8c31640 1 -- 172.21.15.115:0/750871829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f4142870 msgr2=0x7f79f4158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:39.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79f8c31640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f4142870 0x7f79f4158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:39.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79f8c31640 1 -- 172.21.15.115:0/750871829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f40b1c50 msgr2=0x7f79f40c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:39.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79f8c31640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f40b1c50 0x7f79f40c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:39.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79f8c31640 1 -- 172.21.15.115:0/750871829 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e80094d0 con 0x7f79f40cd010 2023-12-01T01:48:39.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79f9432640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f40b1c50 0x7f79f40c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:39.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79f8c31640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40cd010 0x7f79f40c6990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f79f000c7f0 tx=0x7f79f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:39.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79e5ffb640 1 -- 172.21.15.115:0/750871829 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0004220 con 0x7f79f40cd010 2023-12-01T01:48:39.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79e5ffb640 1 -- 172.21.15.115:0/750871829 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f00043a0 con 0x7f79f40cd010 2023-12-01T01:48:39.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79fb696640 1 -- 172.21.15.115:0/750871829 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f40c4800 con 0x7f79f40cd010 2023-12-01T01:48:39.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79e5ffb640 1 -- 172.21.15.115:0/750871829 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0007860 con 0x7f79f40cd010 2023-12-01T01:48:39.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.678+0000 7f79fb696640 1 -- 172.21.15.115:0/750871829 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f40c6ed0 con 0x7f79f40cd010 2023-12-01T01:48:39.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.679+0000 7f79fb696640 1 -- 172.21.15.115:0/750871829 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f40c4d20 con 0x7f79f40cd010 2023-12-01T01:48:39.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.680+0000 7f79e5ffb640 1 -- 172.21.15.115:0/750871829 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f79f0007bd0 con 0x7f79f40cd010 2023-12-01T01:48:39.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.681+0000 7f79e5ffb640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f79d806a630 0x7f79d806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:39.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.681+0000 7f79e5ffb640 1 -- 172.21.15.115:0/750871829 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6539+0+0 (secure 0 0 0) 0x7f79f0089150 con 0x7f79f40cd010 2023-12-01T01:48:39.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.682+0000 7f79f9432640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f79d806a630 0x7f79d806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:39.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.682+0000 7f79f9432640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f79d806a630 0x7f79d806caf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79e8036000 tx=0x7f79e803c040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:39.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.686+0000 7f79e5ffb640 1 -- 172.21.15.115:0/750871829 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f79f40c4d20 con 0x7f79f40cd010 2023-12-01T01:48:39.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:39.916+0000 7f79fb696640 1 -- 172.21.15.115:0/750871829 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f79f40b2320 con 0x7f79d806a630 2023-12-01T01:48:40.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.070+0000 7f79e5ffb640 1 -- 172.21.15.115:0/750871829 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f79e8031cb0 con 0x7f79d806a630 2023-12-01T01:48:40.072 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.073+0000 7f79fb696640 1 -- 172.21.15.115:0/750871829 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f79d806a630 msgr2=0x7f79d806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.073+0000 7f79fb696640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f79d806a630 0x7f79d806caf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79e8036000 tx=0x7f79e803c040 comp rx=0 tx=0).stop 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.073+0000 7f79fb696640 1 -- 172.21.15.115:0/750871829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40cd010 msgr2=0x7f79f40c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.073+0000 7f79fb696640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40cd010 0x7f79f40c6990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f79f000c7f0 tx=0x7f79f000ccb0 comp rx=0 tx=0).stop 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.073+0000 7f79fb696640 1 -- 172.21.15.115:0/750871829 shutdown_connections 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.073+0000 7f79fb696640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f79d806a630 0x7f79d806caf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.073+0000 7f79fb696640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f4142870 0x7f79f4158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.073+0000 7f79fb696640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40cd010 0x7f79f40c6990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.073+0000 7f79fb696640 1 --2- 172.21.15.115:0/750871829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f40b1c50 0x7f79f40c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.073+0000 7f79fb696640 1 -- 172.21.15.115:0/750871829 >> 172.21.15.115:0/750871829 conn(0x7f79f4065960 msgr2=0x7f79f406fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:40.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.074+0000 7f79fb696640 1 -- 172.21.15.115:0/750871829 shutdown_connections 2023-12-01T01:48:40.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:40.074+0000 7f79fb696640 1 -- 172.21.15.115:0/750871829 wait complete. 2023-12-01T01:48:40.094 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-12-01T01:48:40.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.095 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-12-01T01:48:40.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_01_UTC 2023-12-01T01:48:40.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:40.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:40.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:40.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:40.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:40.149 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:40.149 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:40.149 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:40.149 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:40.149 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:40.150 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_38_01_UTC 2023-12-01T01:48:40.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.150 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_38_01_UTC) 2023-12-01T01:48:40.223 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:40.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.224 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_39_00_UTC) 2023-12-01T01:48:40.293 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:40.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.294 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_40_00_UTC) 2023-12-01T01:48:40.360 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:40.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.361 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_41_00_UTC) 2023-12-01T01:48:40.424 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:40.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.424 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_42_00_UTC) 2023-12-01T01:48:40.489 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:40.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.489 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_43_00_UTC) 2023-12-01T01:48:40.579 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:40.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.580 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_44_00_UTC) 2023-12-01T01:48:40.669 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:40.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.669 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_45_00_UTC) 2023-12-01T01:48:40.720 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:40.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.721 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_46_01_UTC) 2023-12-01T01:48:40.768 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:40.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.769 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_47_00_UTC) 2023-12-01T01:48:40.820 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:40.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.820 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-01-01_48_00_UTC) 2023-12-01T01:48:40.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:40.875 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-12-01T01:48:40.930 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir5 1M 2023-12-01T01:48:41.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.144+0000 7fb29ffab640 1 Processor -- start 2023-12-01T01:48:41.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.145+0000 7fb29ffab640 1 -- start start 2023-12-01T01:48:41.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.146+0000 7fb29ffab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb29815cd90 0x7fb2980cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:41.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.146+0000 7fb29ffab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980b5fa0 0x7fb298161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:41.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.146+0000 7fb29ffab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2980ce5e0 con 0x7fb2980cf650 2023-12-01T01:48:41.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.146+0000 7fb29ffab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2980d0ec0 con 0x7fb29815cd90 2023-12-01T01:48:41.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.146+0000 7fb29ffab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb298161950 con 0x7fb2980b5fa0 2023-12-01T01:48:41.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.147+0000 7fb29e548640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb2980cf650 0x7fb2980b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48892/0 (socket says 172.21.15.115:48892) 2023-12-01T01:48:41.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.147+0000 7fb29e548640 1 -- 172.21.15.115:0/1021640218 learned_addr learned my addr 172.21.15.115:0/1021640218 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:41.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.147+0000 7fb29d546640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980b5fa0 0x7fb298161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:41.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.147+0000 7fb29dd47640 1 --2- 172.21.15.115:0/1021640218 >> v2:172.21.15.142:3300/0 conn(0x7fb29815cd90 0x7fb2980cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:41.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.147+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001288303 0 0) 0x7fb290001110 con 0x7fb2980cf650 2023-12-01T01:48:41.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.147+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb288003520 con 0x7fb2980cf650 2023-12-01T01:48:41.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.147+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2197926605 0 0) 0x7fb288003520 con 0x7fb2980cf650 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.147+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980b5fa0 msgr2=0x7fb298161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.147+0000 7fb29cd45640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980b5fa0 0x7fb298161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 >> v2:172.21.15.142:3300/0 conn(0x7fb29815cd90 msgr2=0x7fb2980cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 --2- 172.21.15.115:0/1021640218 >> v2:172.21.15.142:3300/0 conn(0x7fb29815cd90 0x7fb2980cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2980ce300 con 0x7fb2980cf650 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb290001800 con 0x7fb2980cf650 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 >> v1:172.21.15.115:6789/0 conn(0x7fb2980cf650 legacy=0x7fb2980b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288003970 0x7fb288003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980b5fa0 0x7fb2880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288004d00 0x7fb288005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980d0ec0 con 0x7fb288004d00 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb298161950 con 0x7fb288003970 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28800b980 con 0x7fb2980b5fa0 2023-12-01T01:48:41.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29e548640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288004d00 0x7fb288005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:41.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29dd47640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288003970 0x7fb288003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:41.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29d546640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980b5fa0 0x7fb2880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:41.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29e548640 1 -- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980b5fa0 msgr2=0x7fb2880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:41.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29e548640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980b5fa0 0x7fb2880047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29e548640 1 -- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288003970 msgr2=0x7fb288003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:41.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29e548640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288003970 0x7fb288003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.148+0000 7fb29e548640 1 -- 172.21.15.115:0/1021640218 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb288003520 con 0x7fb288004d00 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.150+0000 7fb29e548640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288004d00 0x7fb288005180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb29000c240 tx=0x7fb2880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6f99051d927920 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.150+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290005ad0 con 0x7fb288004d00 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.150+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb290005c30 con 0x7fb288004d00 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.150+0000 7fb29cd45640 1 -- 172.21.15.115:0/1021640218 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29002c8b0 con 0x7fb288004d00 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.150+0000 7fb29ffab640 1 -- 172.21.15.115:0/1021640218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288004d00 msgr2=0x7fb288005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.150+0000 7fb29ffab640 1 --2- 172.21.15.115:0/1021640218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb288004d00 0x7fb288005180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb29000c240 tx=0x7fb2880057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.150+0000 7fb29e548640 1 -- 172.21.15.115:0/1021640218 reap_dead start 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.151+0000 7fb29ffab640 1 -- 172.21.15.115:0/1021640218 shutdown_connections 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.151+0000 7fb29ffab640 1 -- 172.21.15.115:0/1021640218 >> 172.21.15.115:0/1021640218 conn(0x7fb298065960 msgr2=0x7fb298065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.151+0000 7fb29ffab640 1 -- 172.21.15.115:0/1021640218 shutdown_connections 2023-12-01T01:48:41.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.151+0000 7fb29ffab640 1 -- 172.21.15.115:0/1021640218 wait complete. 2023-12-01T01:48:41.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.153+0000 7fb29ffab640 1 Processor -- start 2023-12-01T01:48:41.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.153+0000 7fb29ffab640 1 -- start start 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29ffab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288004d00 0x7fb2981f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29ffab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980cc130 0x7fb2980cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29ffab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2980c9d90 0x7fb2980ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29ffab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980ccaf0 con 0x7fb2980c9d90 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29ffab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980cb5f0 con 0x7fb288004d00 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29ffab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980ca750 con 0x7fb2980cc130 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29e548640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2980c9d90 0x7fb2980ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29e548640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2980c9d90 0x7fb2980ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49010/0 (socket says 172.21.15.115:49010) 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29e548640 1 -- 172.21.15.115:0/1669959394 learned_addr learned my addr 172.21.15.115:0/1669959394 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29d546640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980cc130 0x7fb2980cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29e548640 1 -- 172.21.15.115:0/1669959394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980cc130 msgr2=0x7fb2980cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:41.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29e548640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980cc130 0x7fb2980cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29e548640 1 -- 172.21.15.115:0/1669959394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288004d00 msgr2=0x7fb2981f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:41.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29e548640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288004d00 0x7fb2981f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29e548640 1 -- 172.21.15.115:0/1669959394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2980ca9e0 con 0x7fb2980c9d90 2023-12-01T01:48:41.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.154+0000 7fb29d546640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980cc130 0x7fb2980cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:41.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.155+0000 7fb29e548640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2980c9d90 0x7fb2980ca210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb29002c520 tx=0x7fb290001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:41.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.155+0000 7fb2967fc640 1 -- 172.21.15.115:0/1669959394 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29003c040 con 0x7fb2980c9d90 2023-12-01T01:48:41.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.155+0000 7fb2967fc640 1 -- 172.21.15.115:0/1669959394 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29002ccc0 con 0x7fb2980c9d90 2023-12-01T01:48:41.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.155+0000 7fb29ffab640 1 -- 172.21.15.115:0/1669959394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2980cacc0 con 0x7fb2980c9d90 2023-12-01T01:48:41.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.155+0000 7fb2967fc640 1 -- 172.21.15.115:0/1669959394 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290007290 con 0x7fb2980c9d90 2023-12-01T01:48:41.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.155+0000 7fb29ffab640 1 -- 172.21.15.115:0/1669959394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2980cb280 con 0x7fb2980c9d90 2023-12-01T01:48:41.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.156+0000 7fb29ffab640 1 -- 172.21.15.115:0/1669959394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29804e890 con 0x7fb2980c9d90 2023-12-01T01:48:41.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.157+0000 7fb2967fc640 1 -- 172.21.15.115:0/1669959394 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7fb290032070 con 0x7fb2980c9d90 2023-12-01T01:48:41.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.158+0000 7fb2967fc640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb27c06a660 0x7fb27c06cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:41.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.158+0000 7fb2967fc640 1 -- 172.21.15.115:0/1669959394 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6667+0+0 (secure 0 0 0) 0x7fb2900a9ed0 con 0x7fb2980c9d90 2023-12-01T01:48:41.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.158+0000 7fb29dd47640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb27c06a660 0x7fb27c06cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:41.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.159+0000 7fb29dd47640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb27c06a660 0x7fb27c06cb20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009fa0 tx=0x7fb28c009750 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:41.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.163+0000 7fb2967fc640 1 -- 172.21.15.115:0/1669959394 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb290074de0 con 0x7fb2980c9d90 2023-12-01T01:48:41.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.376+0000 7fb29ffab640 1 -- 172.21.15.115:0/1669959394 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2980d1380 con 0x7fb27c06a660 2023-12-01T01:48:41.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.612+0000 7fb2967fc640 1 -- 172.21.15.115:0/1669959394 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb28c00f050 con 0x7fb27c06a660 2023-12-01T01:48:41.613 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 -- 172.21.15.115:0/1669959394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb27c06a660 msgr2=0x7fb27c06cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb27c06a660 0x7fb27c06cb20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009fa0 tx=0x7fb28c009750 comp rx=0 tx=0).stop 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 -- 172.21.15.115:0/1669959394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2980c9d90 msgr2=0x7fb2980ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2980c9d90 0x7fb2980ca210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb29002c520 tx=0x7fb290001b90 comp rx=0 tx=0).stop 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 -- 172.21.15.115:0/1669959394 shutdown_connections 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb27c06a660 0x7fb27c06cb20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2980cc130 0x7fb2980cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2980c9d90 0x7fb2980ca210 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 --2- 172.21.15.115:0/1669959394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb288004d00 0x7fb2981f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 -- 172.21.15.115:0/1669959394 >> 172.21.15.115:0/1669959394 conn(0x7fb298065960 msgr2=0x7fb2980cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 -- 172.21.15.115:0/1669959394 shutdown_connections 2023-12-01T01:48:41.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:41.615+0000 7fb29ffab640 1 -- 172.21.15.115:0/1669959394 wait complete. 2023-12-01T01:48:41.634 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-12-01T01:48:41.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:41.635 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-12-01T01:48:41.676 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:41.676 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:41.676 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:41.676 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:41.677 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:41.677 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:41.677 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:41.677 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:41.677 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:41.677 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_01_UTC 2023-12-01T01:48:41.677 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:41.678 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:41.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:41.678 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_38_00_UTC) 2023-12-01T01:48:41.806 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:41.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:41.806 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_39_00_UTC) 2023-12-01T01:48:41.857 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:41.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:41.858 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_40_00_UTC) 2023-12-01T01:48:41.910 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:41.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:41.911 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_41_00_UTC) 2023-12-01T01:48:41.965 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:41.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:41.965 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_42_00_UTC) 2023-12-01T01:48:42.027 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:42.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:42.028 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_43_00_UTC) 2023-12-01T01:48:42.099 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:42.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:42.099 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_44_00_UTC) 2023-12-01T01:48:42.169 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:42.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:42.169 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_45_00_UTC) 2023-12-01T01:48:42.241 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_46_01_UTC 2023-12-01T01:48:42.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:42.241 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_46_01_UTC) 2023-12-01T01:48:42.314 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_47_01_UTC 2023-12-01T01:48:42.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:42.315 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_47_01_UTC) 2023-12-01T01:48:42.388 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:42.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:42.388 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-01-01_48_01_UTC) 2023-12-01T01:48:42.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:42.460 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-12-01T01:48:42.525 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir6 1M 2023-12-01T01:48:42.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.730+0000 7f407ca2d640 1 Processor -- start 2023-12-01T01:48:42.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.730+0000 7f407ca2d640 1 -- start start 2023-12-01T01:48:42.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.731+0000 7f407ca2d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40780cfb80 0x7f40780ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.731+0000 7f407ca2d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4078154010 0x7f4078160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.731+0000 7f407ca2d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40780cdcd0 con 0x7f4078153c30 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.731+0000 7f407ca2d640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4078161830 con 0x7f4078154010 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.731+0000 7f407ca2d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40780cd3a0 con 0x7f40780cfb80 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.732+0000 7f4077fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4078154010 0x7f4078160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.732+0000 7f4077fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4078154010 0x7f4078160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49022/0 (socket says 172.21.15.115:49022) 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.732+0000 7f4077fff640 1 -- 172.21.15.115:0/831942790 learned_addr learned my addr 172.21.15.115:0/831942790 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.732+0000 7f4076ffd640 1 --2- 172.21.15.115:0/831942790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40780cfb80 0x7f40780ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.732+0000 7f4077fff640 1 -- 172.21.15.115:0/831942790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40780cfb80 msgr2=0x7f40780ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.732+0000 7f4077fff640 1 --2- 172.21.15.115:0/831942790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40780cfb80 0x7f40780ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.732+0000 7f4077fff640 1 -- 172.21.15.115:0/831942790 >> v1:172.21.15.142:6789/0 conn(0x7f4078153c30 legacy=0x7f40780cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:42.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.732+0000 7f4077fff640 1 -- 172.21.15.115:0/831942790 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780cd680 con 0x7f4078154010 2023-12-01T01:48:42.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.732+0000 7f4076ffd640 1 --2- 172.21.15.115:0/831942790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40780cfb80 0x7f40780ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:42.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.733+0000 7f4077fff640 1 --2- 172.21.15.115:0/831942790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078154010 0x7f4078160ac0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4070009e50 tx=0x7f407002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c24d0e1b988c852 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:42.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.734+0000 7f40767fc640 1 -- 172.21.15.115:0/831942790 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407002ca70 con 0x7f4078154010 2023-12-01T01:48:42.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.734+0000 7f40767fc640 1 -- 172.21.15.115:0/831942790 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407002cbd0 con 0x7f4078154010 2023-12-01T01:48:42.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.734+0000 7f40767fc640 1 -- 172.21.15.115:0/831942790 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40700343f0 con 0x7f4078154010 2023-12-01T01:48:42.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.734+0000 7f407ca2d640 1 -- 172.21.15.115:0/831942790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078154010 msgr2=0x7f4078160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:42.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.734+0000 7f407ca2d640 1 --2- 172.21.15.115:0/831942790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078154010 0x7f4078160ac0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4070009e50 tx=0x7f407002bed0 comp rx=0 tx=0).stop 2023-12-01T01:48:42.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.735+0000 7f407ca2d640 1 -- 172.21.15.115:0/831942790 shutdown_connections 2023-12-01T01:48:42.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.735+0000 7f407ca2d640 1 --2- 172.21.15.115:0/831942790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078154010 0x7f4078160ac0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:42.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.735+0000 7f407ca2d640 1 --2- 172.21.15.115:0/831942790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40780cfb80 0x7f40780ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:42.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.735+0000 7f407ca2d640 1 -- 172.21.15.115:0/831942790 >> 172.21.15.115:0/831942790 conn(0x7f407806a0b0 msgr2=0x7f407806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:42.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.735+0000 7f407ca2d640 1 -- 172.21.15.115:0/831942790 shutdown_connections 2023-12-01T01:48:42.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.735+0000 7f407ca2d640 1 -- 172.21.15.115:0/831942790 wait complete. 2023-12-01T01:48:42.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.736+0000 7f407ca2d640 1 Processor -- start 2023-12-01T01:48:42.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.737+0000 7f407ca2d640 1 -- start start 2023-12-01T01:48:42.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.737+0000 7f407ca2d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40780cfb80 0x7f40780c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:42.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f407ca2d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078153c30 0x7f407815db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:42.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f407ca2d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4078154010 0x7f407815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:42.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f407ca2d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4078155470 con 0x7f4078153c30 2023-12-01T01:48:42.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f407ca2d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40781556c0 con 0x7f40780cfb80 2023-12-01T01:48:42.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f407ca2d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40781559a0 con 0x7f4078154010 2023-12-01T01:48:42.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f4076ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078153c30 0x7f407815db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:42.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f4076ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078153c30 0x7f407815db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49024/0 (socket says 172.21.15.115:49024) 2023-12-01T01:48:42.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f4076ffd640 1 -- 172.21.15.115:0/2855576973 learned_addr learned my addr 172.21.15.115:0/2855576973 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:42.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f40777fe640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40780cfb80 0x7f40780c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:42.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f4077fff640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4078154010 0x7f407815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:42.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f4076ffd640 1 -- 172.21.15.115:0/2855576973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4078154010 msgr2=0x7f407815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:42.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f4076ffd640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4078154010 0x7f407815c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:42.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f4076ffd640 1 -- 172.21.15.115:0/2855576973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40780cfb80 msgr2=0x7f40780c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:42.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f4076ffd640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40780cfb80 0x7f40780c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:42.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.738+0000 7f4076ffd640 1 -- 172.21.15.115:0/2855576973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4070009b30 con 0x7f4078153c30 2023-12-01T01:48:42.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.739+0000 7f4076ffd640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078153c30 0x7f407815db20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f406c00c7f0 tx=0x7f406c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:42.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.739+0000 7f405ffff640 1 -- 172.21.15.115:0/2855576973 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c007a30 con 0x7f4078153c30 2023-12-01T01:48:42.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.739+0000 7f405ffff640 1 -- 172.21.15.115:0/2855576973 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406c007b90 con 0x7f4078153c30 2023-12-01T01:48:42.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.739+0000 7f407ca2d640 1 -- 172.21.15.115:0/2855576973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4078155c80 con 0x7f4078153c30 2023-12-01T01:48:42.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.739+0000 7f405ffff640 1 -- 172.21.15.115:0/2855576973 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c00ce40 con 0x7f4078153c30 2023-12-01T01:48:42.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.739+0000 7f407ca2d640 1 -- 172.21.15.115:0/2855576973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4078156240 con 0x7f4078153c30 2023-12-01T01:48:42.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.740+0000 7f407ca2d640 1 -- 172.21.15.115:0/2855576973 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40780618d0 con 0x7f4078153c30 2023-12-01T01:48:42.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.741+0000 7f405ffff640 1 -- 172.21.15.115:0/2855576973 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f406c0113f0 con 0x7f4078153c30 2023-12-01T01:48:42.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.742+0000 7f405ffff640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f406406a660 0x7f406406cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:42.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.742+0000 7f405ffff640 1 -- 172.21.15.115:0/2855576973 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6635+0+0 (secure 0 0 0) 0x7f406c089250 con 0x7f4078153c30 2023-12-01T01:48:42.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.742+0000 7f40777fe640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f406406a660 0x7f406406cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:42.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.743+0000 7f40777fe640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f406406a660 0x7f406406cb20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4068003d90 tx=0x7f4068009040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:42.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.746+0000 7f405ffff640 1 -- 172.21.15.115:0/2855576973 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f406c054140 con 0x7f4078153c30 2023-12-01T01:48:42.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:42.968+0000 7f407ca2d640 1 -- 172.21.15.115:0/2855576973 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f40781604b0 con 0x7f406406a660 2023-12-01T01:48:43.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.162+0000 7f405ffff640 1 -- 172.21.15.115:0/2855576973 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f40781604b0 con 0x7f406406a660 2023-12-01T01:48:43.164 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-12-01T01:48:43.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 -- 172.21.15.115:0/2855576973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f406406a660 msgr2=0x7f406406cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:43.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f406406a660 0x7f406406cb20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4068003d90 tx=0x7f4068009040 comp rx=0 tx=0).stop 2023-12-01T01:48:43.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 -- 172.21.15.115:0/2855576973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078153c30 msgr2=0x7f407815db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:43.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078153c30 0x7f407815db20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f406c00c7f0 tx=0x7f406c00cbf0 comp rx=0 tx=0).stop 2023-12-01T01:48:43.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 -- 172.21.15.115:0/2855576973 shutdown_connections 2023-12-01T01:48:43.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f406406a660 0x7f406406cb20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:43.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4078154010 0x7f407815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:43.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4078153c30 0x7f407815db20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:43.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 --2- 172.21.15.115:0/2855576973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40780cfb80 0x7f40780c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:43.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 -- 172.21.15.115:0/2855576973 >> 172.21.15.115:0/2855576973 conn(0x7f407806a0b0 msgr2=0x7f40780d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:43.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 -- 172.21.15.115:0/2855576973 shutdown_connections 2023-12-01T01:48:43.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:43.166+0000 7f407ca2d640 1 -- 172.21.15.115:0/2855576973 wait complete. 2023-12-01T01:48:43.189 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-12-01T01:48:43.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.189 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_01_UTC 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:43.246 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:43.247 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_38_01_UTC 2023-12-01T01:48:43.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.248 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_38_01_UTC) 2023-12-01T01:48:43.320 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:43.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.320 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_39_00_UTC) 2023-12-01T01:48:43.382 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:43.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.383 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_40_00_UTC) 2023-12-01T01:48:43.447 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:43.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.448 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_41_00_UTC) 2023-12-01T01:48:43.520 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:43.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.520 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_42_00_UTC) 2023-12-01T01:48:43.617 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:43.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.617 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_43_00_UTC) 2023-12-01T01:48:43.669 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_44_01_UTC 2023-12-01T01:48:43.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.669 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_44_01_UTC) 2023-12-01T01:48:43.722 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_45_01_UTC 2023-12-01T01:48:43.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.722 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_45_01_UTC) 2023-12-01T01:48:43.783 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:43.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.783 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_46_00_UTC) 2023-12-01T01:48:43.848 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:43.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.848 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_47_00_UTC) 2023-12-01T01:48:43.915 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:43.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.915 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-01-01_48_00_UTC) 2023-12-01T01:48:43.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:43.984 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-12-01T01:48:44.047 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir7 1M 2023-12-01T01:48:44.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.271+0000 7ff88b59e640 1 Processor -- start 2023-12-01T01:48:44.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.272+0000 7ff88b59e640 1 -- start start 2023-12-01T01:48:44.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.273+0000 7ff88b59e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff88c15ac00 0x7ff88c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:44.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.273+0000 7ff88b59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0aede0 0x7ff88c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:44.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.274+0000 7ff88b59e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c15afe0 con 0x7ff88c0b04c0 2023-12-01T01:48:44.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.274+0000 7ff88b59e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88c0b12b0 con 0x7ff88c15ac00 2023-12-01T01:48:44.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.274+0000 7ff88b59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0b0a10 con 0x7ff88c0aede0 2023-12-01T01:48:44.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.274+0000 7ff88ad9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff88c0b04c0 0x7ff88c06b0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48904/0 (socket says 172.21.15.115:48904) 2023-12-01T01:48:44.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.274+0000 7ff88ad9d640 1 -- 172.21.15.115:0/2358906651 learned_addr learned my addr 172.21.15.115:0/2358906651 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:44.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.274+0000 7ff88a59c640 1 --2- 172.21.15.115:0/2358906651 >> v2:172.21.15.142:3300/0 conn(0x7ff88c15ac00 0x7ff88c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:44.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.274+0000 7ff889d9b640 1 --2- 172.21.15.115:0/2358906651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0aede0 0x7ff88c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:44.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.275+0000 7ff88959a640 1 -- 172.21.15.115:0/2358906651 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658469427 0 0) 0x7ff88c15afe0 con 0x7ff88c0b04c0 2023-12-01T01:48:44.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.276+0000 7ff88959a640 1 -- 172.21.15.115:0/2358906651 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff878003520 con 0x7ff88c0b04c0 2023-12-01T01:48:44.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.276+0000 7ff88a59c640 1 -- 172.21.15.115:0/2358906651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0aede0 msgr2=0x7ff88c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:44.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.276+0000 7ff88a59c640 1 --2- 172.21.15.115:0/2358906651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0aede0 0x7ff88c0cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:44.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.276+0000 7ff88a59c640 1 -- 172.21.15.115:0/2358906651 >> v1:172.21.15.115:6789/0 conn(0x7ff88c0b04c0 legacy=0x7ff88c06b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:44.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.276+0000 7ff88959a640 1 -- 172.21.15.115:0/2358906651 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1250934813 0 0) 0x7ff878003520 con 0x7ff88c0b04c0 2023-12-01T01:48:44.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.276+0000 7ff88a59c640 1 -- 172.21.15.115:0/2358906651 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c0b0cf0 con 0x7ff88c15ac00 2023-12-01T01:48:44.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.276+0000 7ff889d9b640 1 --2- 172.21.15.115:0/2358906651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0aede0 0x7ff88c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:48:44.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.279+0000 7ff88a59c640 1 --2- 172.21.15.115:0/2358906651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88c15ac00 0x7ff88c0ae8a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8840097a0 tx=0x7ff88402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2c1c48a33aeb32 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:44.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.280+0000 7ff88959a640 1 -- 172.21.15.115:0/2358906651 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0019f0 con 0x7ff88c15ac00 2023-12-01T01:48:44.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.280+0000 7ff88959a640 1 -- 172.21.15.115:0/2358906651 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88402e050 con 0x7ff88c15ac00 2023-12-01T01:48:44.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.280+0000 7ff88959a640 1 -- 172.21.15.115:0/2358906651 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff884030730 con 0x7ff88c15ac00 2023-12-01T01:48:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.281+0000 7ff88b59e640 1 -- 172.21.15.115:0/2358906651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88c15ac00 msgr2=0x7ff88c0ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:44.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.281+0000 7ff88b59e640 1 --2- 172.21.15.115:0/2358906651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88c15ac00 0x7ff88c0ae8a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8840097a0 tx=0x7ff88402b710 comp rx=0 tx=0).stop 2023-12-01T01:48:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.281+0000 7ff88b59e640 1 -- 172.21.15.115:0/2358906651 shutdown_connections 2023-12-01T01:48:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.281+0000 7ff88b59e640 1 --2- 172.21.15.115:0/2358906651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0aede0 0x7ff88c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.281+0000 7ff88b59e640 1 --2- 172.21.15.115:0/2358906651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88c15ac00 0x7ff88c0ae8a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.281+0000 7ff88b59e640 1 -- 172.21.15.115:0/2358906651 >> 172.21.15.115:0/2358906651 conn(0x7ff88c065960 msgr2=0x7ff88c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.282+0000 7ff88b59e640 1 -- 172.21.15.115:0/2358906651 shutdown_connections 2023-12-01T01:48:44.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.282+0000 7ff88b59e640 1 -- 172.21.15.115:0/2358906651 wait complete. 2023-12-01T01:48:44.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.284+0000 7ff88b59e640 1 Processor -- start 2023-12-01T01:48:44.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.285+0000 7ff88b59e640 1 -- start start 2023-12-01T01:48:44.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.286+0000 7ff88b59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88c0aede0 0x7ff88c14e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.286+0000 7ff88b59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0b04c0 0x7ff88c144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.286+0000 7ff88b59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c15ac00 0x7ff88c0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.286+0000 7ff88b59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c144360 con 0x7ff88c15ac00 2023-12-01T01:48:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.286+0000 7ff88b59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c150ff0 con 0x7ff88c0aede0 2023-12-01T01:48:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.286+0000 7ff88b59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c1512d0 con 0x7ff88c0b04c0 2023-12-01T01:48:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.287+0000 7ff88ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c15ac00 0x7ff88c0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.287+0000 7ff88ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c15ac00 0x7ff88c0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49026/0 (socket says 172.21.15.115:49026) 2023-12-01T01:48:44.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.287+0000 7ff88ad9d640 1 -- 172.21.15.115:0/1075478240 learned_addr learned my addr 172.21.15.115:0/1075478240 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.287+0000 7ff889d9b640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0b04c0 0x7ff88c144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.287+0000 7ff88ad9d640 1 -- 172.21.15.115:0/1075478240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0b04c0 msgr2=0x7ff88c144740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:44.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.287+0000 7ff88ad9d640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0b04c0 0x7ff88c144740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.287+0000 7ff88ad9d640 1 -- 172.21.15.115:0/1075478240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88c0aede0 msgr2=0x7ff88c14e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.287+0000 7ff88ad9d640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88c0aede0 0x7ff88c14e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.287+0000 7ff88ad9d640 1 -- 172.21.15.115:0/1075478240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff874009400 con 0x7ff88c15ac00 2023-12-01T01:48:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.288+0000 7ff88ad9d640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c15ac00 0x7ff88c0b26d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff87c001f70 tx=0x7ff87c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.288+0000 7ff882ffd640 1 -- 172.21.15.115:0/1075478240 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c012400 con 0x7ff88c15ac00 2023-12-01T01:48:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.288+0000 7ff882ffd640 1 -- 172.21.15.115:0/1075478240 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c012560 con 0x7ff88c15ac00 2023-12-01T01:48:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.288+0000 7ff882ffd640 1 -- 172.21.15.115:0/1075478240 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0128d0 con 0x7ff88c15ac00 2023-12-01T01:48:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.288+0000 7ff88b59e640 1 -- 172.21.15.115:0/1075478240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff884009400 con 0x7ff88c15ac00 2023-12-01T01:48:44.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.289+0000 7ff88b59e640 1 -- 172.21.15.115:0/1075478240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c151890 con 0x7ff88c15ac00 2023-12-01T01:48:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.291+0000 7ff882ffd640 1 -- 172.21.15.115:0/1075478240 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7ff87c012a30 con 0x7ff88c15ac00 2023-12-01T01:48:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.291+0000 7ff88b59e640 1 -- 172.21.15.115:0/1075478240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88c0618d0 con 0x7ff88c15ac00 2023-12-01T01:48:44.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.292+0000 7ff882ffd640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff86c06a630 0x7ff86c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:44.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.293+0000 7ff882ffd640 1 -- 172.21.15.115:0/1075478240 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6795+0+0 (secure 0 0 0) 0x7ff87c089830 con 0x7ff88c15ac00 2023-12-01T01:48:44.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.293+0000 7ff88a59c640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff86c06a630 0x7ff86c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:44.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.294+0000 7ff88a59c640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff86c06a630 0x7ff86c06caf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff884034000 tx=0x7ff88402f000 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:44.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.297+0000 7ff882ffd640 1 -- 172.21.15.115:0/1075478240 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff87c054760 con 0x7ff88c15ac00 2023-12-01T01:48:44.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.552+0000 7ff88b59e640 1 -- 172.21.15.115:0/1075478240 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff88c0b0fd0 con 0x7ff86c06a630 2023-12-01T01:48:44.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.686+0000 7ff882ffd640 1 -- 172.21.15.115:0/1075478240 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7ff884030760 con 0x7ff86c06a630 2023-12-01T01:48:44.687 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-12-01T01:48:44.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.689+0000 7ff88b59e640 1 -- 172.21.15.115:0/1075478240 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff86c06a630 msgr2=0x7ff86c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:44.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.689+0000 7ff88b59e640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff86c06a630 0x7ff86c06caf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff884034000 tx=0x7ff88402f000 comp rx=0 tx=0).stop 2023-12-01T01:48:44.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.689+0000 7ff88b59e640 1 -- 172.21.15.115:0/1075478240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c15ac00 msgr2=0x7ff88c0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:44.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.689+0000 7ff88b59e640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c15ac00 0x7ff88c0b26d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff87c001f70 tx=0x7ff87c00a430 comp rx=0 tx=0).stop 2023-12-01T01:48:44.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.689+0000 7ff88b59e640 1 -- 172.21.15.115:0/1075478240 shutdown_connections 2023-12-01T01:48:44.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.689+0000 7ff88b59e640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff86c06a630 0x7ff86c06caf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:44.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.689+0000 7ff88b59e640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88c0b04c0 0x7ff88c144740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:44.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.689+0000 7ff88b59e640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88c15ac00 0x7ff88c0b26d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:44.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.689+0000 7ff88b59e640 1 --2- 172.21.15.115:0/1075478240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88c0aede0 0x7ff88c14e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:44.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.689+0000 7ff88b59e640 1 -- 172.21.15.115:0/1075478240 >> 172.21.15.115:0/1075478240 conn(0x7ff88c065960 msgr2=0x7ff88c152b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.690+0000 7ff88b59e640 1 -- 172.21.15.115:0/1075478240 shutdown_connections 2023-12-01T01:48:44.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:44.690+0000 7ff88b59e640 1 -- 172.21.15.115:0/1075478240 wait complete. 2023-12-01T01:48:44.712 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-12-01T01:48:44.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:44.712 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-12-01T01:48:44.763 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:44.763 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:44.763 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:44.763 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:44.764 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:44.764 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:44.764 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:44.764 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:44.764 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:44.764 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:44.764 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:44.765 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:44.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:44.765 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_38_00_UTC) 2023-12-01T01:48:44.834 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:44.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:44.835 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_39_00_UTC) 2023-12-01T01:48:44.906 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:44.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:44.906 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_40_00_UTC) 2023-12-01T01:48:44.975 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:44.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:44.976 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_41_00_UTC) 2023-12-01T01:48:45.053 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:45.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:45.053 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_42_00_UTC) 2023-12-01T01:48:45.124 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:45.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:45.125 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_43_00_UTC) 2023-12-01T01:48:45.190 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:45.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:45.190 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_44_00_UTC) 2023-12-01T01:48:45.259 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:45.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:45.259 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_45_00_UTC) 2023-12-01T01:48:45.333 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:45.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:45.334 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_46_00_UTC) 2023-12-01T01:48:45.430 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:45.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:45.430 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_47_00_UTC) 2023-12-01T01:48:45.488 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:45.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:45.488 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-01-01_48_00_UTC) 2023-12-01T01:48:45.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:45.539 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-12-01T01:48:45.585 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir8 1M 2023-12-01T01:48:45.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.807+0000 7f06470af640 1 Processor -- start 2023-12-01T01:48:45.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.807+0000 7f06470af640 1 -- start start 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.808+0000 7f06470af640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0640142870 0x7f06400ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.808+0000 7f06470af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06400b1e20 0x7f064015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.808+0000 7f06470af640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06400b2f30 con 0x7f06400cd010 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.808+0000 7f06470af640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06400b3ae0 con 0x7f0640142870 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.808+0000 7f06470af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06400cd4a0 con 0x7f06400b1e20 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.809+0000 7f0644e4b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0640142870 0x7f06400ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.809+0000 7f063ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f06400cd010 0x7f06400b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48912/0 (socket says 172.21.15.115:48912) 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.809+0000 7f0644e4b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0640142870 0x7f06400ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49036/0 (socket says 172.21.15.115:49036) 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.809+0000 7f063ffff640 1 -- 172.21.15.115:0/2138419189 learned_addr learned my addr 172.21.15.115:0/2138419189 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.809+0000 7f0644e4b640 1 -- 172.21.15.115:0/2138419189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06400b1e20 msgr2=0x7f064015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.809+0000 7f0644e4b640 1 --2- 172.21.15.115:0/2138419189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06400b1e20 0x7f064015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:45.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.809+0000 7f0644e4b640 1 -- 172.21.15.115:0/2138419189 >> v1:172.21.15.115:6789/0 conn(0x7f06400cd010 legacy=0x7f06400b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:45.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.809+0000 7f0644e4b640 1 -- 172.21.15.115:0/2138419189 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400b2320 con 0x7f0640142870 2023-12-01T01:48:45.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.809+0000 7f063f7fe640 1 -- 172.21.15.115:0/2138419189 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733511358 0 0) 0x7f06400b2f30 con 0x7f06400cd010 2023-12-01T01:48:45.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.810+0000 7f0644e4b640 1 --2- 172.21.15.115:0/2138419189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0640142870 0x7f06400ccad0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f06340097a0 tx=0x7f063402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326c34e4f2593037 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:45.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.810+0000 7f063f7fe640 1 -- 172.21.15.115:0/2138419189 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634031830 con 0x7f0640142870 2023-12-01T01:48:45.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.811+0000 7f063f7fe640 1 -- 172.21.15.115:0/2138419189 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0634031990 con 0x7f0640142870 2023-12-01T01:48:45.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.811+0000 7f063f7fe640 1 -- 172.21.15.115:0/2138419189 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634031d20 con 0x7f0640142870 2023-12-01T01:48:45.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.811+0000 7f06470af640 1 -- 172.21.15.115:0/2138419189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0640142870 msgr2=0x7f06400ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:45.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.811+0000 7f06470af640 1 --2- 172.21.15.115:0/2138419189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0640142870 0x7f06400ccad0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f06340097a0 tx=0x7f063402b8d0 comp rx=0 tx=0).stop 2023-12-01T01:48:45.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.811+0000 7f06470af640 1 -- 172.21.15.115:0/2138419189 shutdown_connections 2023-12-01T01:48:45.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.811+0000 7f06470af640 1 --2- 172.21.15.115:0/2138419189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06400b1e20 0x7f064015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:45.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.811+0000 7f06470af640 1 --2- 172.21.15.115:0/2138419189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0640142870 0x7f06400ccad0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:45.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.811+0000 7f06470af640 1 -- 172.21.15.115:0/2138419189 >> 172.21.15.115:0/2138419189 conn(0x7f0640065960 msgr2=0x7f0640065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:45.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.811+0000 7f06470af640 1 -- 172.21.15.115:0/2138419189 shutdown_connections 2023-12-01T01:48:45.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.811+0000 7f06470af640 1 -- 172.21.15.115:0/2138419189 wait complete. 2023-12-01T01:48:45.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.813+0000 7f06470af640 1 Processor -- start 2023-12-01T01:48:45.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.813+0000 7f06470af640 1 -- start start 2023-12-01T01:48:45.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.814+0000 7f06470af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06400b1e20 0x7f0640156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:45.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.814+0000 7f06470af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06400cd010 0x7f06401404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:45.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.814+0000 7f06470af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0640142870 0x7f064013e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:45.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.814+0000 7f06470af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0640159ed0 con 0x7f06400b1e20 2023-12-01T01:48:45.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.814+0000 7f06470af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0640140e90 con 0x7f0640142870 2023-12-01T01:48:45.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.814+0000 7f06470af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064015a0e0 con 0x7f06400cd010 2023-12-01T01:48:45.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.814+0000 7f0644e4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06400b1e20 0x7f0640156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:45.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.814+0000 7f0644e4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06400b1e20 0x7f0640156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49048/0 (socket says 172.21.15.115:49048) 2023-12-01T01:48:45.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.814+0000 7f0644e4b640 1 -- 172.21.15.115:0/3825046355 learned_addr learned my addr 172.21.15.115:0/3825046355 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:45.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f0644e4b640 1 -- 172.21.15.115:0/3825046355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06400cd010 msgr2=0x7f06401404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:45.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f0644e4b640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06400cd010 0x7f06401404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:45.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f064564c640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0640142870 0x7f064013e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:45.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f0644e4b640 1 -- 172.21.15.115:0/3825046355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0640142870 msgr2=0x7f064013e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:45.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f0644e4b640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0640142870 0x7f064013e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:45.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f0644e4b640 1 -- 172.21.15.115:0/3825046355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0634009400 con 0x7f06400b1e20 2023-12-01T01:48:45.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f0644e4b640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06400b1e20 0x7f0640156010 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0634000c00 tx=0x7f06340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:45.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f063d7fa640 1 -- 172.21.15.115:0/3825046355 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063403aa60 con 0x7f06400b1e20 2023-12-01T01:48:45.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f063d7fa640 1 -- 172.21.15.115:0/3825046355 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063403abc0 con 0x7f06400b1e20 2023-12-01T01:48:45.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f06470af640 1 -- 172.21.15.115:0/3825046355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064015a3c0 con 0x7f06400b1e20 2023-12-01T01:48:45.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.815+0000 7f063d7fa640 1 -- 172.21.15.115:0/3825046355 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340443f0 con 0x7f06400b1e20 2023-12-01T01:48:45.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.816+0000 7f06470af640 1 -- 172.21.15.115:0/3825046355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06400cc120 con 0x7f06400b1e20 2023-12-01T01:48:45.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.817+0000 7f06470af640 1 -- 172.21.15.115:0/3825046355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064015a8e0 con 0x7f06400b1e20 2023-12-01T01:48:45.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.818+0000 7f063d7fa640 1 -- 172.21.15.115:0/3825046355 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f063404d450 con 0x7f06400b1e20 2023-12-01T01:48:45.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.819+0000 7f063d7fa640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f062406a630 0x7f062406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:45.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.819+0000 7f063d7fa640 1 -- 172.21.15.115:0/3825046355 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f06340ab4c0 con 0x7f06400b1e20 2023-12-01T01:48:45.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.819+0000 7f063ffff640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f062406a630 0x7f062406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:45.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.820+0000 7f063ffff640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f062406a630 0x7f062406caf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f062c007fd0 tx=0x7f062c007920 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:45.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:45.824+0000 7f063d7fa640 1 -- 172.21.15.115:0/3825046355 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f064015a8e0 con 0x7f06400b1e20 2023-12-01T01:48:46.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.048+0000 7f06470af640 1 -- 172.21.15.115:0/3825046355 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f064015d710 con 0x7f062406a630 2023-12-01T01:48:46.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.244+0000 7f063d7fa640 1 -- 172.21.15.115:0/3825046355 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f064015d710 con 0x7f062406a630 2023-12-01T01:48:46.246 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-12-01T01:48:46.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 -- 172.21.15.115:0/3825046355 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f062406a630 msgr2=0x7f062406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:46.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f062406a630 0x7f062406caf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f062c007fd0 tx=0x7f062c007920 comp rx=0 tx=0).stop 2023-12-01T01:48:46.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 -- 172.21.15.115:0/3825046355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06400b1e20 msgr2=0x7f0640156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:46.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06400b1e20 0x7f0640156010 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0634000c00 tx=0x7f06340046c0 comp rx=0 tx=0).stop 2023-12-01T01:48:46.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 -- 172.21.15.115:0/3825046355 shutdown_connections 2023-12-01T01:48:46.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f062406a630 0x7f062406caf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:46.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06400cd010 0x7f06401404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:46.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0640142870 0x7f064013e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:46.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 --2- 172.21.15.115:0/3825046355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06400b1e20 0x7f0640156010 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:46.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 -- 172.21.15.115:0/3825046355 >> 172.21.15.115:0/3825046355 conn(0x7f0640065960 msgr2=0x7f0640070550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:46.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 -- 172.21.15.115:0/3825046355 shutdown_connections 2023-12-01T01:48:46.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:46.248+0000 7f06470af640 1 -- 172.21.15.115:0/3825046355 wait complete. 2023-12-01T01:48:46.269 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-12-01T01:48:46.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.270 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-12-01T01:48:46.323 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:46.323 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:46.323 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:46.324 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:46.324 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:46.324 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:46.324 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:46.324 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:46.324 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:46.324 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:46.324 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:46.325 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:46.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.325 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_38_00_UTC) 2023-12-01T01:48:46.389 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:46.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.389 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_39_00_UTC) 2023-12-01T01:48:46.507 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:46.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.507 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_40_00_UTC) 2023-12-01T01:48:46.572 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:46.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.573 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_41_00_UTC) 2023-12-01T01:48:46.620 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:46.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.620 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_42_00_UTC) 2023-12-01T01:48:46.672 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:46.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.673 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_43_00_UTC) 2023-12-01T01:48:46.723 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:46.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.723 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_44_00_UTC) 2023-12-01T01:48:46.777 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:46.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.778 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_45_00_UTC) 2023-12-01T01:48:46.835 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:46.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.836 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_46_00_UTC) 2023-12-01T01:48:46.899 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_47_00_UTC 2023-12-01T01:48:46.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.899 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_47_00_UTC) 2023-12-01T01:48:46.963 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_48_00_UTC 2023-12-01T01:48:46.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:46.963 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-01-01_48_00_UTC) 2023-12-01T01:48:47.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:47.026 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-12-01T01:48:47.086 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir9 1M 2023-12-01T01:48:47.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.302+0000 7efd2a08c640 1 Processor -- start 2023-12-01T01:48:47.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.302+0000 7efd2a08c640 1 -- start start 2023-12-01T01:48:47.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.304+0000 7efd2a08c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efd241599e0 0x7efd24159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:47.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.304+0000 7efd2a08c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2415e440 con 0x7efd240af370 2023-12-01T01:48:47.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.304+0000 7efd2a08c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2415a300 con 0x7efd240c9770 2023-12-01T01:48:47.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.304+0000 7efd2a08c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd2415d550 con 0x7efd241599e0 2023-12-01T01:48:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.304+0000 7efd28889640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efd240af370 0x7efd240c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48916/0 (socket says 172.21.15.115:48916) 2023-12-01T01:48:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.304+0000 7efd28889640 1 -- 172.21.15.115:0/1452137075 learned_addr learned my addr 172.21.15.115:0/1452137075 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.304+0000 7efd2908a640 1 --2- 172.21.15.115:0/1452137075 >> v2:172.21.15.142:3300/0 conn(0x7efd241599e0 0x7efd24159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615033888 0 0) 0x7efd2415e440 con 0x7efd240af370 2023-12-01T01:48:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd10003520 con 0x7efd240af370 2023-12-01T01:48:47.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 364120296 0 0) 0x7efd10003520 con 0x7efd240af370 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 >> v2:172.21.15.142:3300/0 conn(0x7efd241599e0 msgr2=0x7efd24159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 --2- 172.21.15.115:0/1452137075 >> v2:172.21.15.142:3300/0 conn(0x7efd241599e0 0x7efd24159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 >> v1:172.21.15.142:6789/0 conn(0x7efd240c9770 legacy=0x7efd24070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2415d830 con 0x7efd240af370 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7efd14001670 con 0x7efd240af370 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 >> v1:172.21.15.115:6789/0 conn(0x7efd240af370 legacy=0x7efd240c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd100039c0 0x7efd10003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd100047c0 0x7efd10004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd10005180 0x7efd1000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2415d550 con 0x7efd10005180 2023-12-01T01:48:47.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1000c080 con 0x7efd100047c0 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1000c240 con 0x7efd100039c0 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.305+0000 7efd2988b640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd10005180 0x7efd1000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.306+0000 7efd28889640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd100047c0 0x7efd10004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.306+0000 7efd2908a640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd100039c0 0x7efd10003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.306+0000 7efd2988b640 1 -- 172.21.15.115:0/1452137075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd100039c0 msgr2=0x7efd10003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.306+0000 7efd2988b640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd100039c0 0x7efd10003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.306+0000 7efd2988b640 1 -- 172.21.15.115:0/1452137075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd100047c0 msgr2=0x7efd10004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.306+0000 7efd2988b640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd100047c0 0x7efd10004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.306+0000 7efd2988b640 1 -- 172.21.15.115:0/1452137075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd10003520 con 0x7efd10005180 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.307+0000 7efd2988b640 1 -- 172.21.15.115:0/1452137075 reap_dead start 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.307+0000 7efd2988b640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd10005180 0x7efd1000b980 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efd1c001e50 tx=0x7efd100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4134767531c9664d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.307+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c003b70 con 0x7efd10005180 2023-12-01T01:48:47.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.307+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c003d10 con 0x7efd10005180 2023-12-01T01:48:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.307+0000 7efd1bfff640 1 -- 172.21.15.115:0/1452137075 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c005310 con 0x7efd10005180 2023-12-01T01:48:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.308+0000 7efd2a08c640 1 -- 172.21.15.115:0/1452137075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd10005180 msgr2=0x7efd1000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.308+0000 7efd2a08c640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd10005180 0x7efd1000b980 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efd1c001e50 tx=0x7efd100057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.308+0000 7efd2a08c640 1 -- 172.21.15.115:0/1452137075 shutdown_connections 2023-12-01T01:48:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.308+0000 7efd2a08c640 1 --2- 172.21.15.115:0/1452137075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd10005180 0x7efd1000b980 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.308+0000 7efd2a08c640 1 -- 172.21.15.115:0/1452137075 >> 172.21.15.115:0/1452137075 conn(0x7efd24065960 msgr2=0x7efd24065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.308+0000 7efd2a08c640 1 -- 172.21.15.115:0/1452137075 shutdown_connections 2023-12-01T01:48:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.308+0000 7efd2a08c640 1 -- 172.21.15.115:0/1452137075 wait complete. 2023-12-01T01:48:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.310+0000 7efd2a08c640 1 Processor -- start 2023-12-01T01:48:47.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.310+0000 7efd2a08c640 1 -- start start 2023-12-01T01:48:47.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.311+0000 7efd2a08c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd10005180 0x7efd241f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:47.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.311+0000 7efd2a08c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240cd590 0x7efd240cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:47.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.311+0000 7efd2a08c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd240c3b20 0x7efd240c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:47.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.311+0000 7efd2a08c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd240cdf50 con 0x7efd240cd590 2023-12-01T01:48:47.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.311+0000 7efd2a08c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd240c6410 con 0x7efd10005180 2023-12-01T01:48:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.311+0000 7efd2a08c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd241f3360 con 0x7efd240c3b20 2023-12-01T01:48:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.311+0000 7efd28889640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240cd590 0x7efd240cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.311+0000 7efd28889640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240cd590 0x7efd240cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49068/0 (socket says 172.21.15.115:49068) 2023-12-01T01:48:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.311+0000 7efd28889640 1 -- 172.21.15.115:0/2346444357 learned_addr learned my addr 172.21.15.115:0/2346444357 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd2908a640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd10005180 0x7efd241f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd2988b640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd240c3b20 0x7efd240c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd28889640 1 -- 172.21.15.115:0/2346444357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd240c3b20 msgr2=0x7efd240c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd28889640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd240c3b20 0x7efd240c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd28889640 1 -- 172.21.15.115:0/2346444357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd10005180 msgr2=0x7efd241f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd28889640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd10005180 0x7efd241f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd28889640 1 -- 172.21.15.115:0/2346444357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd240c5800 con 0x7efd240cd590 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd2908a640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd10005180 0x7efd241f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd28889640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240cd590 0x7efd240cda10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efd1400e730 tx=0x7efd1400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd19ffb640 1 -- 172.21.15.115:0/2346444357 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1400c6c0 con 0x7efd240cd590 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd19ffb640 1 -- 172.21.15.115:0/2346444357 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1400c840 con 0x7efd240cd590 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd19ffb640 1 -- 172.21.15.115:0/2346444357 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1400cbf0 con 0x7efd240cd590 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd2a08c640 1 -- 172.21.15.115:0/2346444357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd240c5ae0 con 0x7efd240cd590 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.312+0000 7efd2a08c640 1 -- 172.21.15.115:0/2346444357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd240c60a0 con 0x7efd240cd590 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.313+0000 7efd2a08c640 1 -- 172.21.15.115:0/2346444357 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2415ddd0 con 0x7efd240cd590 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.314+0000 7efd19ffb640 1 -- 172.21.15.115:0/2346444357 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7efd1400cd50 con 0x7efd240cd590 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.316+0000 7efd19ffb640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd0806a630 0x7efd0806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:47.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.316+0000 7efd19ffb640 1 -- 172.21.15.115:0/2346444357 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6731+0+0 (secure 0 0 0) 0x7efd14088f70 con 0x7efd240cd590 2023-12-01T01:48:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.316+0000 7efd2908a640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd0806a630 0x7efd0806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:47.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.317+0000 7efd2908a640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd0806a630 0x7efd0806caf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd20009980 tx=0x7efd20008040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:47.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.320+0000 7efd19ffb640 1 -- 172.21.15.115:0/2346444357 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efd2415ddd0 con 0x7efd240cd590 2023-12-01T01:48:47.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.535+0000 7efd2a08c640 1 -- 172.21.15.115:0/2346444357 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efd2415e0d0 con 0x7efd0806a630 2023-12-01T01:48:47.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.737+0000 7efd19ffb640 1 -- 172.21.15.115:0/2346444357 <== mgr.5313 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7efd20012950 con 0x7efd0806a630 2023-12-01T01:48:47.739 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-12-01T01:48:47.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 -- 172.21.15.115:0/2346444357 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd0806a630 msgr2=0x7efd0806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd0806a630 0x7efd0806caf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd20009980 tx=0x7efd20008040 comp rx=0 tx=0).stop 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 -- 172.21.15.115:0/2346444357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240cd590 msgr2=0x7efd240cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240cd590 0x7efd240cda10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efd1400e730 tx=0x7efd1400ebf0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 -- 172.21.15.115:0/2346444357 shutdown_connections 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efd0806a630 0x7efd0806caf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd240c3b20 0x7efd240c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240cd590 0x7efd240cda10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 --2- 172.21.15.115:0/2346444357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd10005180 0x7efd241f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 -- 172.21.15.115:0/2346444357 >> 172.21.15.115:0/2346444357 conn(0x7efd24065960 msgr2=0x7efd240af770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 -- 172.21.15.115:0/2346444357 shutdown_connections 2023-12-01T01:48:47.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:47.741+0000 7efd2a08c640 1 -- 172.21.15.115:0/2346444357 wait complete. 2023-12-01T01:48:47.763 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-12-01T01:48:47.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:47.763 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-12-01T01:48:47.815 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:47.815 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:47.815 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:47.815 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:47.815 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:47.816 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:47.816 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:47.816 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:47.816 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:47.816 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_47_01_UTC 2023-12-01T01:48:47.816 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:47.817 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_38_00_UTC 2023-12-01T01:48:47.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:47.817 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_38_00_UTC) 2023-12-01T01:48:47.886 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_39_00_UTC 2023-12-01T01:48:47.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:47.886 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_39_00_UTC) 2023-12-01T01:48:47.957 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_40_00_UTC 2023-12-01T01:48:47.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:47.957 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_40_00_UTC) 2023-12-01T01:48:48.029 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_41_00_UTC 2023-12-01T01:48:48.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:48.029 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_41_00_UTC) 2023-12-01T01:48:48.097 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_42_00_UTC 2023-12-01T01:48:48.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:48.097 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_42_00_UTC) 2023-12-01T01:48:48.162 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_43_00_UTC 2023-12-01T01:48:48.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:48.163 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_43_00_UTC) 2023-12-01T01:48:48.234 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_44_00_UTC 2023-12-01T01:48:48.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:48.235 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_44_00_UTC) 2023-12-01T01:48:48.299 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_45_00_UTC 2023-12-01T01:48:48.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:48.299 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_45_00_UTC) 2023-12-01T01:48:48.355 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_46_00_UTC 2023-12-01T01:48:48.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:48.355 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_46_00_UTC) 2023-12-01T01:48:48.418 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_47_01_UTC 2023-12-01T01:48:48.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:48.419 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_47_01_UTC) 2023-12-01T01:48:48.491 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_48_01_UTC 2023-12-01T01:48:48.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:48.491 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-01-01_48_01_UTC) 2023-12-01T01:48:48.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:48.560 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-12-01T01:48:48.622 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T01:48:48.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.840+0000 7f02a6981640 1 Processor -- start 2023-12-01T01:48:48.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.840+0000 7f02a6981640 1 -- start start 2023-12-01T01:48:48.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.842+0000 7f02a6981640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f02a015ac80 0x7f02a00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:48.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.842+0000 7f02a6981640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a00cc9a0 con 0x7f02a00b0e60 2023-12-01T01:48:48.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.842+0000 7f02a6981640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a00cbd90 con 0x7f02a00ce1b0 2023-12-01T01:48:48.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.842+0000 7f02a6981640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a015b060 con 0x7f02a015ac80 2023-12-01T01:48:48.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.842+0000 7f029ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f02a015ac80 0x7f02a00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:48.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.842+0000 7f029ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f02a015ac80 0x7f02a00cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42564/0 (socket says 172.21.15.115:42564) 2023-12-01T01:48:48.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.842+0000 7f029ffff640 1 -- 172.21.15.115:0/1360654116 learned_addr learned my addr 172.21.15.115:0/1360654116 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:48.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.842+0000 7f02977fe640 1 --1- 172.21.15.115:0/1360654116 >> v1:172.21.15.115:6789/0 conn(0x7f02a00b0e60 0x7f02a00cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59624/0 (socket says 172.21.15.115:59624) 2023-12-01T01:48:48.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.843+0000 7f029ffff640 1 -- 172.21.15.115:0/1360654116 >> v1:172.21.15.142:6789/0 conn(0x7f02a00ce1b0 legacy=0x7f02a006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:48.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.843+0000 7f029ffff640 1 -- 172.21.15.115:0/1360654116 >> v1:172.21.15.115:6789/0 conn(0x7f02a00b0e60 legacy=0x7f02a00cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:48.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.843+0000 7f029f7fe640 1 -- 172.21.15.115:0/1360654116 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965753172 0 0) 0x7f02a00cc9a0 con 0x7f02a00b0e60 2023-12-01T01:48:48.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.843+0000 7f029ffff640 1 -- 172.21.15.115:0/1360654116 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a00cc070 con 0x7f02a015ac80 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.845+0000 7f029ffff640 1 --2- 172.21.15.115:0/1360654116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a015ac80 0x7f02a00cce30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0290009870 tx=0x7f029002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e8b953381d5dd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.845+0000 7f029f7fe640 1 -- 172.21.15.115:0/1360654116 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290030860 con 0x7f02a015ac80 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.845+0000 7f029f7fe640 1 -- 172.21.15.115:0/1360654116 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02900309c0 con 0x7f02a015ac80 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.845+0000 7f029f7fe640 1 -- 172.21.15.115:0/1360654116 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290030d50 con 0x7f02a015ac80 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.845+0000 7f02a6981640 1 -- 172.21.15.115:0/1360654116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a015ac80 msgr2=0x7f02a00cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.845+0000 7f02a6981640 1 --2- 172.21.15.115:0/1360654116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a015ac80 0x7f02a00cce30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0290009870 tx=0x7f029002b9a0 comp rx=0 tx=0).stop 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.846+0000 7f02a6981640 1 -- 172.21.15.115:0/1360654116 shutdown_connections 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.846+0000 7f02a6981640 1 --2- 172.21.15.115:0/1360654116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a015ac80 0x7f02a00cce30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.846+0000 7f02a6981640 1 -- 172.21.15.115:0/1360654116 >> 172.21.15.115:0/1360654116 conn(0x7f02a0065960 msgr2=0x7f02a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.846+0000 7f02a6981640 1 -- 172.21.15.115:0/1360654116 shutdown_connections 2023-12-01T01:48:48.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.846+0000 7f02a6981640 1 -- 172.21.15.115:0/1360654116 wait complete. 2023-12-01T01:48:48.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.848+0000 7f02a6981640 1 Processor -- start 2023-12-01T01:48:48.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.848+0000 7f02a6981640 1 -- start start 2023-12-01T01:48:48.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02a6981640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a00b0e60 0x7f02a014b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:48.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02a6981640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a00ce1b0 0x7f02a0156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:48.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02a6981640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02a015ac80 0x7f02a006b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:48.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02a6981640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00a8950 con 0x7f02a00ce1b0 2023-12-01T01:48:48.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02a6981640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a0157d70 con 0x7f02a00b0e60 2023-12-01T01:48:48.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02a6981640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00a85b0 con 0x7f02a015ac80 2023-12-01T01:48:48.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02977fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a00ce1b0 0x7f02a0156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:48.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02977fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a00ce1b0 0x7f02a0156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42572/0 (socket says 172.21.15.115:42572) 2023-12-01T01:48:48.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02977fe640 1 -- 172.21.15.115:0/916848516 learned_addr learned my addr 172.21.15.115:0/916848516 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:48.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f029ffff640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a00b0e60 0x7f02a014b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02977fe640 1 -- 172.21.15.115:0/916848516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02a015ac80 msgr2=0x7f02a006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02977fe640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02a015ac80 0x7f02a006b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02977fe640 1 -- 172.21.15.115:0/916848516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a00b0e60 msgr2=0x7f02a014b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02977fe640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a00b0e60 0x7f02a014b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.849+0000 7f02977fe640 1 -- 172.21.15.115:0/916848516 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900094d0 con 0x7f02a00ce1b0 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.850+0000 7f02977fe640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a00ce1b0 0x7f02a0156cb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f028800c4d0 tx=0x7f028800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.850+0000 7f029d7fa640 1 -- 172.21.15.115:0/916848516 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880074e0 con 0x7f02a00ce1b0 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.850+0000 7f029d7fa640 1 -- 172.21.15.115:0/916848516 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0288007660 con 0x7f02a00ce1b0 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.850+0000 7f02a6981640 1 -- 172.21.15.115:0/916848516 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a01577b0 con 0x7f02a00ce1b0 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.850+0000 7f029d7fa640 1 -- 172.21.15.115:0/916848516 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288004a20 con 0x7f02a00ce1b0 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.850+0000 7f02a6981640 1 -- 172.21.15.115:0/916848516 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a0155800 con 0x7f02a00ce1b0 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.851+0000 7f02a6981640 1 -- 172.21.15.115:0/916848516 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a00a8a90 con 0x7f02a00ce1b0 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.852+0000 7f029d7fa640 1 -- 172.21.15.115:0/916848516 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 14) v1 ==== 88795+0+0 (secure 0 0 0) 0x7f0288004b80 con 0x7f02a00ce1b0 2023-12-01T01:48:48.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.853+0000 7f029d7fa640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f028006a630 0x7f028006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:48.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.853+0000 7f029d7fa640 1 -- 172.21.15.115:0/916848516 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6731+0+0 (secure 0 0 0) 0x7f0288088600 con 0x7f02a00ce1b0 2023-12-01T01:48:48.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.854+0000 7f029ffff640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f028006a630 0x7f028006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:48.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.854+0000 7f029ffff640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f028006a630 0x7f028006caf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0290036000 tx=0x7f029003b040 comp rx=0 tx=0).ready entity=mgr.5313 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:48.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:48.858+0000 7f029d7fa640 1 -- 172.21.15.115:0/916848516 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f02a00a8a90 con 0x7f02a00ce1b0 2023-12-01T01:48:49.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.105+0000 7f02a6981640 1 -- 172.21.15.115:0/916848516 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f02a00a8a90 con 0x7f02a00ce1b0 2023-12-01T01:48:49.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.438+0000 7f029d7fa640 1 -- 172.21.15.115:0/916848516 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 15) v1 ==== 88778+0+0 (secure 0 0 0) 0x7f0288053930 con 0x7f02a00ce1b0 2023-12-01T01:48:49.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.439+0000 7f029d7fa640 1 -- 172.21.15.115:0/916848516 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v15) v1 ==== 93+0+0 (secure 0 0 0) 0x7f0288072190 con 0x7f02a00ce1b0 2023-12-01T01:48:49.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.442+0000 7f02a6981640 1 -- 172.21.15.115:0/916848516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f028006a630 msgr2=0x7f028006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:49.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.442+0000 7f02a6981640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f028006a630 0x7f028006caf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0290036000 tx=0x7f029003b040 comp rx=0 tx=0).stop 2023-12-01T01:48:49.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.442+0000 7f02a6981640 1 -- 172.21.15.115:0/916848516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a00ce1b0 msgr2=0x7f02a0156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:49.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.442+0000 7f02a6981640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a00ce1b0 0x7f02a0156cb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f028800c4d0 tx=0x7f028800c990 comp rx=0 tx=0).stop 2023-12-01T01:48:49.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.442+0000 7f02a6981640 1 -- 172.21.15.115:0/916848516 shutdown_connections 2023-12-01T01:48:49.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.442+0000 7f02a6981640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f028006a630 0x7f028006caf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:49.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.442+0000 7f02a6981640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02a015ac80 0x7f02a006b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:49.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.442+0000 7f02a6981640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02a00ce1b0 0x7f02a0156cb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:49.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.442+0000 7f02a6981640 1 --2- 172.21.15.115:0/916848516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02a00b0e60 0x7f02a014b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:49.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.442+0000 7f02a6981640 1 -- 172.21.15.115:0/916848516 >> 172.21.15.115:0/916848516 conn(0x7f02a0065960 msgr2=0x7f02a006fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:49.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.443+0000 7f02a6981640 1 -- 172.21.15.115:0/916848516 shutdown_connections 2023-12-01T01:48:49.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:49.443+0000 7f02a6981640 1 -- 172.21.15.115:0/916848516 wait complete. 2023-12-01T01:48:49.465 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:48:49.465 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:48:49.563 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:48:49.657 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:48:49.657 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:48:49.658 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:48:49.710 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:48:49.710 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:48:49.800 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:48:49.962 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:49.960+0000 7f2b7560f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:48:49.987 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:49.986+0000 7f7b93048040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:48:50.015 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:50.014+0000 7f2b7560f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:48:50.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.037+0000 7f6a2effd640 1 Processor -- start 2023-12-01T01:48:50.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.037+0000 7f6a2effd640 1 -- start start 2023-12-01T01:48:50.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.039+0000 7f6a2effd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6a2815d7b0 0x7f6a2815db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.039+0000 7f6a2effd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6a280d05d0 0x7f6a280d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.039+0000 7f6a2effd640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a280cd3f0 con 0x7f6a28160670 2023-12-01T01:48:50.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.039+0000 7f6a2effd640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a280d0f60 con 0x7f6a280d05d0 2023-12-01T01:48:50.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.039+0000 7f6a2effd640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a280cd0f0 con 0x7f6a2815d7b0 2023-12-01T01:48:50.041 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:50.039+0000 7f7b93048040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:48:50.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.039+0000 7f6a2e7fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6a280d05d0 0x7f6a280d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.039+0000 7f6a2e7fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6a280d05d0 0x7f6a280d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42588/0 (socket says 172.21.15.115:42588) 2023-12-01T01:48:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.039+0000 7f6a2e7fc640 1 -- 172.21.15.115:0/3009923216 learned_addr learned my addr 172.21.15.115:0/3009923216 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.039+0000 7f6a2dffb640 1 --2- 172.21.15.115:0/3009923216 >> v2:172.21.15.142:3300/0 conn(0x7f6a2815d7b0 0x7f6a2815db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.040+0000 7f6a2e7fc640 1 -- 172.21.15.115:0/3009923216 >> v2:172.21.15.142:3300/0 conn(0x7f6a2815d7b0 msgr2=0x7f6a2815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.040+0000 7f6a2e7fc640 1 --2- 172.21.15.115:0/3009923216 >> v2:172.21.15.142:3300/0 conn(0x7f6a2815d7b0 0x7f6a2815db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.040+0000 7f6a2e7fc640 1 -- 172.21.15.115:0/3009923216 >> v1:172.21.15.142:6789/0 conn(0x7f6a28160670 legacy=0x7f6a280cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.040+0000 7f6a2e7fc640 1 -- 172.21.15.115:0/3009923216 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a28160ec0 con 0x7f6a280d05d0 2023-12-01T01:48:50.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.041+0000 7f6a2e7fc640 1 --2- 172.21.15.115:0/3009923216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a280d05d0 0x7f6a280d0a20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6a280720e0 tx=0x7f6a2002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d4053425f47790 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.042+0000 7f6a2cff9640 1 -- 172.21.15.115:0/3009923216 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2002cbb0 con 0x7f6a280d05d0 2023-12-01T01:48:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.042+0000 7f6a2cff9640 1 -- 172.21.15.115:0/3009923216 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2002cd10 con 0x7f6a280d05d0 2023-12-01T01:48:50.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.042+0000 7f6a2cff9640 1 -- 172.21.15.115:0/3009923216 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20034610 con 0x7f6a280d05d0 2023-12-01T01:48:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.042+0000 7f6a2effd640 1 -- 172.21.15.115:0/3009923216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a280d05d0 msgr2=0x7f6a280d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.042+0000 7f6a2effd640 1 --2- 172.21.15.115:0/3009923216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a280d05d0 0x7f6a280d0a20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6a280720e0 tx=0x7f6a2002c010 comp rx=0 tx=0).stop 2023-12-01T01:48:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.042+0000 7f6a2effd640 1 -- 172.21.15.115:0/3009923216 shutdown_connections 2023-12-01T01:48:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.042+0000 7f6a2effd640 1 --2- 172.21.15.115:0/3009923216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a280d05d0 0x7f6a280d0a20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.042+0000 7f6a2effd640 1 --2- 172.21.15.115:0/3009923216 >> v2:172.21.15.142:3300/0 conn(0x7f6a2815d7b0 0x7f6a2815db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.042+0000 7f6a2effd640 1 -- 172.21.15.115:0/3009923216 >> 172.21.15.115:0/3009923216 conn(0x7f6a2806a0e0 msgr2=0x7f6a2806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.043+0000 7f6a2effd640 1 -- 172.21.15.115:0/3009923216 shutdown_connections 2023-12-01T01:48:50.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.043+0000 7f6a2effd640 1 -- 172.21.15.115:0/3009923216 wait complete. 2023-12-01T01:48:50.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.044+0000 7f6a2effd640 1 Processor -- start 2023-12-01T01:48:50.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.044+0000 7f6a2effd640 1 -- start start 2023-12-01T01:48:50.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.045+0000 7f6a2effd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a280d05d0 0x7f6a281568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.045+0000 7f6a2effd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2815d7b0 0x7f6a280c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.045+0000 7f6a2effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a28160670 0x7f6a281597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.045+0000 7f6a2effd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28156df0 con 0x7f6a28160670 2023-12-01T01:48:50.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.045+0000 7f6a2effd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a281f3360 con 0x7f6a2815d7b0 2023-12-01T01:48:50.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.045+0000 7f6a2effd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a280ca3b0 con 0x7f6a280d05d0 2023-12-01T01:48:50.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a28160670 0x7f6a281597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a28160670 0x7f6a281597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42592/0 (socket says 172.21.15.115:42592) 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2e7fc640 1 -- 172.21.15.115:0/2927338069 learned_addr learned my addr 172.21.15.115:0/2927338069 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2d7fa640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2815d7b0 0x7f6a280c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2dffb640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a280d05d0 0x7f6a281568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2e7fc640 1 -- 172.21.15.115:0/2927338069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a280d05d0 msgr2=0x7f6a281568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2e7fc640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a280d05d0 0x7f6a281568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2e7fc640 1 -- 172.21.15.115:0/2927338069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2815d7b0 msgr2=0x7f6a280c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2e7fc640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2815d7b0 0x7f6a280c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2e7fc640 1 -- 172.21.15.115:0/2927338069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a20009b30 con 0x7f6a28160670 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.046+0000 7f6a2d7fa640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2815d7b0 0x7f6a280c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.047+0000 7f6a2e7fc640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a28160670 0x7f6a281597a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009f40 tx=0x7f6a20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:50.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.047+0000 7f6a1e7fc640 1 -- 172.21.15.115:0/2927338069 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20009510 con 0x7f6a28160670 2023-12-01T01:48:50.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.047+0000 7f6a1e7fc640 1 -- 172.21.15.115:0/2927338069 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2003e450 con 0x7f6a28160670 2023-12-01T01:48:50.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.047+0000 7f6a1e7fc640 1 -- 172.21.15.115:0/2927338069 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2003e800 con 0x7f6a28160670 2023-12-01T01:48:50.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.047+0000 7f6a2effd640 1 -- 172.21.15.115:0/2927338069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a281589e0 con 0x7f6a28160670 2023-12-01T01:48:50.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.047+0000 7f6a2effd640 1 -- 172.21.15.115:0/2927338069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a28158fa0 con 0x7f6a28160670 2023-12-01T01:48:50.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.048+0000 7f6a2effd640 1 -- 172.21.15.115:0/2927338069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a280673a0 con 0x7f6a28160670 2023-12-01T01:48:50.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.050+0000 7f6a1e7fc640 1 -- 172.21.15.115:0/2927338069 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 88778+0+0 (secure 0 0 0) 0x7f6a2003e960 con 0x7f6a28160670 2023-12-01T01:48:50.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.051+0000 7f6a1e7fc640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a0c06a610 0x7f6a0c06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.051+0000 7f6a1e7fc640 1 -- 172.21.15.115:0/2927338069 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6731+0+0 (secure 0 0 0) 0x7f6a200abbd0 con 0x7f6a28160670 2023-12-01T01:48:50.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.051+0000 7f6a2dffb640 1 -- 172.21.15.115:0/2927338069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a0c06a610 msgr2=0x7f6a0c06cad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:50.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.051+0000 7f6a2dffb640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a0c06a610 0x7f6a0c06cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:48:50.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.056+0000 7f6a1e7fc640 1 -- 172.21.15.115:0/2927338069 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6a20076aa0 con 0x7f6a28160670 2023-12-01T01:48:50.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.247+0000 7f6a2effd640 1 -- 172.21.15.115:0/2927338069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a28070750 con 0x7f6a28160670 2023-12-01T01:48:50.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.249+0000 7f6a1e7fc640 1 -- 172.21.15.115:0/2927338069 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+17718 (secure 0 0 0) 0x7f6a2007a0c0 con 0x7f6a28160670 2023-12-01T01:48:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:50.251 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":49,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:48:46.392784+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:37:22.073677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-01T01:37:22.660650+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":199,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.23","mappings":[{"from":4,"to":5}]},{"pgid":"5.33","mappings":[{"from":4,"to":5}]},{"pgid":"5.3c","mappings":[{"from":4,"to":5}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:37:37.403693+0000","172.21.15.115:6822/88317":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/4134228660":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/297799080":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/1314334167":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/2515539287":"2023-12-02T01:37:37.403693+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":4,"length":1},{"begin":6,"length":1},{"begin":10,"length":1},{"begin":12,"length":1},{"begin":18,"length":2},{"begin":23,"length":1},{"begin":26,"length":2},{"begin":33,"length":1},{"begin":35,"length":2},{"begin":42,"length":1},{"begin":44,"length":1},{"begin":46,"length":1},{"begin":53,"length":2},{"begin":58,"length":1},{"begin":66,"length":3},{"begin":75,"length":2},{"begin":78,"length":1},{"begin":83,"length":1},{"begin":86,"length":1},{"begin":89,"length":1},{"begin":96,"length":2},{"begin":101,"length":2},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":111,"length":1},{"begin":166,"length":34}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":6,"length":1},{"begin":12,"length":1},{"begin":23,"length":1},{"begin":36,"length":1},{"begin":44,"length":1},{"begin":58,"length":1},{"begin":67,"length":1},{"begin":76,"length":1},{"begin":83,"length":1},{"begin":97,"length":1},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":188,"length":12}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 -- 172.21.15.115:0/2927338069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a0c06a610 msgr2=0x7f6a0c06cad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a0c06a610 0x7f6a0c06cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 -- 172.21.15.115:0/2927338069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a28160670 msgr2=0x7f6a281597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a28160670 0x7f6a281597a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009f40 tx=0x7f6a20004a40 comp rx=0 tx=0).stop 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 -- 172.21.15.115:0/2927338069 shutdown_connections 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6a0c06a610 0x7f6a0c06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a28160670 0x7f6a281597a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2815d7b0 0x7f6a280c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 --2- 172.21.15.115:0/2927338069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a280d05d0 0x7f6a281568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 -- 172.21.15.115:0/2927338069 >> 172.21.15.115:0/2927338069 conn(0x7f6a2806a0e0 msgr2=0x7f6a280b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.251+0000 7f6a2effd640 1 -- 172.21.15.115:0/2927338069 shutdown_connections 2023-12-01T01:48:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.252+0000 7f6a2effd640 1 -- 172.21.15.115:0/2927338069 wait complete. 2023-12-01T01:48:50.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:37:22.073677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:37:22.660650+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 199, 'snap_epoch': 49, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:48:50.272 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:48:50.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.474+0000 7f25e3aa0640 1 Processor -- start 2023-12-01T01:48:50.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.474+0000 7f25e3aa0640 1 -- start start 2023-12-01T01:48:50.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.475+0000 7f25e3aa0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc15cd90 0x7f25dc0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.476+0000 7f25e3aa0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25dc0ce5e0 con 0x7f25dc0b5fa0 2023-12-01T01:48:50.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.476+0000 7f25e3aa0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25dc0ce300 con 0x7f25dc0cfd60 2023-12-01T01:48:50.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.476+0000 7f25e3aa0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc0d0ec0 con 0x7f25dc15cd90 2023-12-01T01:48:50.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.476+0000 7f25e103b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f25dc0b5fa0 0x7f25dc0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59632/0 (socket says 172.21.15.115:59632) 2023-12-01T01:48:50.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.476+0000 7f25e103b640 1 -- 172.21.15.115:0/2187653786 learned_addr learned my addr 172.21.15.115:0/2187653786 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.476+0000 7f25e183c640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc15cd90 0x7f25dc0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.476+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791615186 0 0) 0x7f25cc001110 con 0x7f25dc0b5fa0 2023-12-01T01:48:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.476+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25c8003520 con 0x7f25dc0b5fa0 2023-12-01T01:48:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.476+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3668040818 0 0) 0x7f25c8003520 con 0x7f25dc0b5fa0 2023-12-01T01:48:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc15cd90 msgr2=0x7f25dc0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc15cd90 0x7f25dc0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 >> v1:172.21.15.142:6789/0 conn(0x7f25dc0cfd60 legacy=0x7f25dc0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc0d10a0 con 0x7f25dc0b5fa0 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f25cc001800 con 0x7f25dc0b5fa0 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 >> v1:172.21.15.115:6789/0 conn(0x7f25dc0b5fa0 legacy=0x7f25dc0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc15cd90 0x7f25c80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 0x7f25c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25c8004d00 0x7f25c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc0d0ec0 con 0x7f25c8003f20 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c800ba60 con 0x7f25c8004d00 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c800bc20 con 0x7f25dc15cd90 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e103b640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 0x7f25c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e183c640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc15cd90 0x7f25c80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.477+0000 7f25e203d640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25c8004d00 0x7f25c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.478+0000 7f25e103b640 1 -- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc15cd90 msgr2=0x7f25c80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.478+0000 7f25e103b640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc15cd90 0x7f25c80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.478+0000 7f25e103b640 1 -- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25c8004d00 msgr2=0x7f25c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.478+0000 7f25e103b640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25c8004d00 0x7f25c8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.478+0000 7f25e103b640 1 -- 172.21.15.115:0/2187653786 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c8003520 con 0x7f25c8003f20 2023-12-01T01:48:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.479+0000 7f25e103b640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 0x7f25c80047c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00c240 tx=0x7f25c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea22c0b976eb6cc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.479+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc005ad0 con 0x7f25c8003f20 2023-12-01T01:48:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.479+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc005c50 con 0x7f25c8003f20 2023-12-01T01:48:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.479+0000 7f25e083a640 1 -- 172.21.15.115:0/2187653786 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc02c8b0 con 0x7f25c8003f20 2023-12-01T01:48:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.479+0000 7f25e3aa0640 1 -- 172.21.15.115:0/2187653786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 msgr2=0x7f25c80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.479+0000 7f25e3aa0640 1 --2- 172.21.15.115:0/2187653786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 0x7f25c80047c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00c240 tx=0x7f25c80057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.479+0000 7f25e203d640 1 -- 172.21.15.115:0/2187653786 reap_dead start 2023-12-01T01:48:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.480+0000 7f25e3aa0640 1 -- 172.21.15.115:0/2187653786 shutdown_connections 2023-12-01T01:48:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.480+0000 7f25e3aa0640 1 -- 172.21.15.115:0/2187653786 >> 172.21.15.115:0/2187653786 conn(0x7f25dc065960 msgr2=0x7f25dc065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.480+0000 7f25e3aa0640 1 -- 172.21.15.115:0/2187653786 shutdown_connections 2023-12-01T01:48:50.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.480+0000 7f25e3aa0640 1 -- 172.21.15.115:0/2187653786 wait complete. 2023-12-01T01:48:50.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.481+0000 7f25e3aa0640 1 Processor -- start 2023-12-01T01:48:50.483 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:50.482+0000 7f2b7560f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:48:50.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.482+0000 7f25e3aa0640 1 -- start start 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e3aa0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 0x7f25dc159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e3aa0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25dc0b5030 0x7f25dc14a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e3aa0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc14a760 0x7f25dc14abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e3aa0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc14b120 con 0x7f25c8003f20 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e3aa0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc1481d0 con 0x7f25dc0b5030 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e3aa0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc1575a0 con 0x7f25dc14a760 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e183c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 0x7f25dc159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e183c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 0x7f25dc159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42614/0 (socket says 172.21.15.115:42614) 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e183c640 1 -- 172.21.15.115:0/570502761 learned_addr learned my addr 172.21.15.115:0/570502761 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e203d640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc14a760 0x7f25dc14abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e103b640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25dc0b5030 0x7f25dc14a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e183c640 1 -- 172.21.15.115:0/570502761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc14a760 msgr2=0x7f25dc14abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e183c640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc14a760 0x7f25dc14abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e183c640 1 -- 172.21.15.115:0/570502761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25dc0b5030 msgr2=0x7f25dc14a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e183c640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25dc0b5030 0x7f25dc14a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e183c640 1 -- 172.21.15.115:0/570502761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc159af0 con 0x7f25c8003f20 2023-12-01T01:48:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.483+0000 7f25e103b640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25dc0b5030 0x7f25dc14a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.484+0000 7f25e183c640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 0x7f25dc159490 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f25d8011600 tx=0x7f25d8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.484+0000 7f25d27fc640 1 -- 172.21.15.115:0/570502761 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800f920 con 0x7f25c8003f20 2023-12-01T01:48:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.484+0000 7f25d27fc640 1 -- 172.21.15.115:0/570502761 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d800fa80 con 0x7f25c8003f20 2023-12-01T01:48:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.484+0000 7f25d27fc640 1 -- 172.21.15.115:0/570502761 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800fdf0 con 0x7f25c8003f20 2023-12-01T01:48:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.484+0000 7f25e3aa0640 1 -- 172.21.15.115:0/570502761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc159dd0 con 0x7f25c8003f20 2023-12-01T01:48:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.484+0000 7f25e3aa0640 1 -- 172.21.15.115:0/570502761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc15a390 con 0x7f25c8003f20 2023-12-01T01:48:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.486+0000 7f25e3aa0640 1 -- 172.21.15.115:0/570502761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25dc04e890 con 0x7f25c8003f20 2023-12-01T01:48:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.486+0000 7f25d27fc640 1 -- 172.21.15.115:0/570502761 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 88778+0+0 (secure 0 0 0) 0x7f25d8015460 con 0x7f25c8003f20 2023-12-01T01:48:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.487+0000 7f25d27fc640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f25c006a610 0x7f25c006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.487+0000 7f25d27fc640 1 -- 172.21.15.115:0/570502761 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6731+0+0 (secure 0 0 0) 0x7f25d80895b0 con 0x7f25c8003f20 2023-12-01T01:48:50.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.487+0000 7f25e103b640 1 -- 172.21.15.115:0/570502761 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f25c006a610 msgr2=0x7f25c006cad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:50.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.487+0000 7f25e103b640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f25c006a610 0x7f25c006cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:48:50.489 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:50.489+0000 7f7b93048040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:48:50.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.492+0000 7f25d27fc640 1 -- 172.21.15.115:0/570502761 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f25d8054480 con 0x7f25c8003f20 2023-12-01T01:48:50.538 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:50.537+0000 7f2b7560f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:48:50.542 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:50.541+0000 7f7b93048040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:48:50.589 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:50.587+0000 7f2b7560f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:48:50.592 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:50.591+0000 7f7b93048040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:48:50.650 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:50.649+0000 7f2b7560f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:48:50.654 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:50.652+0000 7f7b93048040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:48:50.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.681+0000 7f25e3aa0640 1 -- 172.21.15.115:0/570502761 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25dc0d0a70 con 0x7f25c8003f20 2023-12-01T01:48:50.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.681+0000 7f25d27fc640 1 -- 172.21.15.115:0/570502761 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25d8057aa0 con 0x7f25c8003f20 2023-12-01T01:48:50.683 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:48:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 -- 172.21.15.115:0/570502761 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f25c006a610 msgr2=0x7f25c006cad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:48:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f25c006a610 0x7f25c006cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 -- 172.21.15.115:0/570502761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 msgr2=0x7f25dc159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 0x7f25dc159490 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f25d8011600 tx=0x7f25d8011ac0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 -- 172.21.15.115:0/570502761 shutdown_connections 2023-12-01T01:48:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f25c006a610 0x7f25c006cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25dc14a760 0x7f25dc14abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25dc0b5030 0x7f25dc14a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 --2- 172.21.15.115:0/570502761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25c8003f20 0x7f25dc159490 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 -- 172.21.15.115:0/570502761 >> 172.21.15.115:0/570502761 conn(0x7f25dc065960 msgr2=0x7f25dc15f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 -- 172.21.15.115:0/570502761 shutdown_connections 2023-12-01T01:48:50.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.685+0000 7f25e3aa0640 1 -- 172.21.15.115:0/570502761 wait complete. 2023-12-01T01:48:50.702 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:50.701+0000 7f7b93048040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:48:50.709 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:50.708+0000 7f2b7560f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:48:50.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:48:50.815 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:50.814+0000 7f7b93048040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:48:50.828 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:50.827+0000 7f2b7560f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:48:50.878 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:50.877+0000 7f7b93048040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:48:50.893 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:50.892+0000 7f2b7560f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:48:50.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.915+0000 7fa21b6a0640 1 Processor -- start 2023-12-01T01:48:50.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.915+0000 7fa21b6a0640 1 -- start start 2023-12-01T01:48:50.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.917+0000 7fa21b6a0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa21415cf10 0x7fa2140cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.917+0000 7fa21b6a0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa214070d30 0x7fa214065960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.917+0000 7fa21b6a0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21415d2f0 con 0x7fa2140705e0 2023-12-01T01:48:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.917+0000 7fa21b6a0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa214071640 con 0x7fa214070d30 2023-12-01T01:48:50.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.917+0000 7fa21b6a0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2140b4b30 con 0x7fa21415cf10 2023-12-01T01:48:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.918+0000 7fa219c3d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa2140705e0 0x7fa2140cdc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59638/0 (socket says 172.21.15.115:59638) 2023-12-01T01:48:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.918+0000 7fa219c3d640 1 -- 172.21.15.115:0/2928818807 learned_addr learned my addr 172.21.15.115:0/2928818807 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.918+0000 7fa21943c640 1 --2- 172.21.15.115:0/2928818807 >> v2:172.21.15.142:3300/0 conn(0x7fa21415cf10 0x7fa2140cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.918+0000 7fa218c3b640 1 --2- 172.21.15.115:0/2928818807 >> v2:172.21.15.115:3300/0 conn(0x7fa214070d30 0x7fa214065960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.918+0000 7fa20bfff640 1 -- 172.21.15.115:0/2928818807 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124035945 0 0) 0x7fa21415d2f0 con 0x7fa2140705e0 2023-12-01T01:48:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.918+0000 7fa20bfff640 1 -- 172.21.15.115:0/2928818807 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa200003520 con 0x7fa2140705e0 2023-12-01T01:48:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.918+0000 7fa218c3b640 1 -- 172.21.15.115:0/2928818807 >> v2:172.21.15.142:3300/0 conn(0x7fa21415cf10 msgr2=0x7fa2140cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.918+0000 7fa218c3b640 1 --2- 172.21.15.115:0/2928818807 >> v2:172.21.15.142:3300/0 conn(0x7fa21415cf10 0x7fa2140cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.918+0000 7fa218c3b640 1 -- 172.21.15.115:0/2928818807 >> v1:172.21.15.115:6789/0 conn(0x7fa2140705e0 legacy=0x7fa2140cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.918+0000 7fa218c3b640 1 -- 172.21.15.115:0/2928818807 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2140b3fb0 con 0x7fa214070d30 2023-12-01T01:48:50.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.920+0000 7fa218c3b640 1 --2- 172.21.15.115:0/2928818807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 0x7fa214065960 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa214070380 tx=0x7fa20402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79917195912c1aa8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:50.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.920+0000 7fa20bfff640 1 -- 172.21.15.115:0/2928818807 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20402e070 con 0x7fa214070d30 2023-12-01T01:48:50.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.920+0000 7fa20bfff640 1 -- 172.21.15.115:0/2928818807 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2040315a0 con 0x7fa214070d30 2023-12-01T01:48:50.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.920+0000 7fa20bfff640 1 -- 172.21.15.115:0/2928818807 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204031930 con 0x7fa214070d30 2023-12-01T01:48:50.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.921+0000 7fa21b6a0640 1 -- 172.21.15.115:0/2928818807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 msgr2=0x7fa214065960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.921+0000 7fa21b6a0640 1 --2- 172.21.15.115:0/2928818807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 0x7fa214065960 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa214070380 tx=0x7fa20402b840 comp rx=0 tx=0).stop 2023-12-01T01:48:50.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.921+0000 7fa21b6a0640 1 -- 172.21.15.115:0/2928818807 shutdown_connections 2023-12-01T01:48:50.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.921+0000 7fa21b6a0640 1 --2- 172.21.15.115:0/2928818807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 0x7fa214065960 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.921+0000 7fa21b6a0640 1 --2- 172.21.15.115:0/2928818807 >> v2:172.21.15.142:3300/0 conn(0x7fa21415cf10 0x7fa2140cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.921+0000 7fa21b6a0640 1 -- 172.21.15.115:0/2928818807 >> 172.21.15.115:0/2928818807 conn(0x7fa2140b1720 msgr2=0x7fa2140b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.921+0000 7fa21b6a0640 1 -- 172.21.15.115:0/2928818807 shutdown_connections 2023-12-01T01:48:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.921+0000 7fa21b6a0640 1 -- 172.21.15.115:0/2928818807 wait complete. 2023-12-01T01:48:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.923+0000 7fa21b6a0640 1 Processor -- start 2023-12-01T01:48:50.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.923+0000 7fa21b6a0640 1 -- start start 2023-12-01T01:48:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.924+0000 7fa21b6a0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2140705e0 0x7fa214152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.924+0000 7fa21b6a0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 0x7fa214155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.924+0000 7fa21b6a0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa21415cf10 0x7fa214155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.924+0000 7fa21b6a0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21406c870 con 0x7fa214070d30 2023-12-01T01:48:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.924+0000 7fa21b6a0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2141604b0 con 0x7fa21415cf10 2023-12-01T01:48:50.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.924+0000 7fa21b6a0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa214152b90 con 0x7fa2140705e0 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.924+0000 7fa218c3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 0x7fa214155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.924+0000 7fa218c3b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 0x7fa214155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42632/0 (socket says 172.21.15.115:42632) 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.924+0000 7fa218c3b640 1 -- 172.21.15.115:0/5259820 learned_addr learned my addr 172.21.15.115:0/5259820 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa21943c640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2140705e0 0x7fa214152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa219c3d640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa21415cf10 0x7fa214155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa218c3b640 1 -- 172.21.15.115:0/5259820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2140705e0 msgr2=0x7fa214152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa218c3b640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2140705e0 0x7fa214152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa218c3b640 1 -- 172.21.15.115:0/5259820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa21415cf10 msgr2=0x7fa214155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa218c3b640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa21415cf10 0x7fa214155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa218c3b640 1 -- 172.21.15.115:0/5259820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204009400 con 0x7fa214070d30 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa218c3b640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 0x7fa214155030 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa204009890 tx=0x7fa2040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:50.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa209ffb640 1 -- 172.21.15.115:0/5259820 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204032630 con 0x7fa214070d30 2023-12-01T01:48:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa209ffb640 1 -- 172.21.15.115:0/5259820 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa204032790 con 0x7fa214070d30 2023-12-01T01:48:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa21b6a0640 1 -- 172.21.15.115:0/5259820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21415f930 con 0x7fa214070d30 2023-12-01T01:48:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.925+0000 7fa209ffb640 1 -- 172.21.15.115:0/5259820 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204032b40 con 0x7fa214070d30 2023-12-01T01:48:50.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.926+0000 7fa21b6a0640 1 -- 172.21.15.115:0/5259820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21415fef0 con 0x7fa214070d30 2023-12-01T01:48:50.929 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:50.928+0000 7f7b93048040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:48:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.927+0000 7fa21b6a0640 1 -- 172.21.15.115:0/5259820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21406c9b0 con 0x7fa214070d30 2023-12-01T01:48:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.928+0000 7fa209ffb640 1 -- 172.21.15.115:0/5259820 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 88778+0+0 (secure 0 0 0) 0x7fa20403b3f0 con 0x7fa214070d30 2023-12-01T01:48:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.928+0000 7fa209ffb640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1f806a5e0 0x7fa1f806caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.929+0000 7fa209ffb640 1 -- 172.21.15.115:0/5259820 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6731+0+0 (secure 0 0 0) 0x7fa2040ab380 con 0x7fa214070d30 2023-12-01T01:48:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.929+0000 7fa21943c640 1 -- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1f806a5e0 msgr2=0x7fa1f806caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:50.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.929+0000 7fa21943c640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1f806a5e0 0x7fa1f806caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:48:50.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:50.937+0000 7fa209ffb640 1 -- 172.21.15.115:0/5259820 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa21406c9b0 con 0x7fa214070d30 2023-12-01T01:48:50.950 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:50.949+0000 7f2b7560f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:48:50.985 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:50.984+0000 7f7b93048040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:48:51.010 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:51.008+0000 7f2b7560f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:48:51.034 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:51.034+0000 7f7b93048040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:48:51.061 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:51.060+0000 7f2b7560f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:48:51.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.125+0000 7fa21b6a0640 1 -- 172.21.15.115:0/5259820 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa21406c9b0 con 0x7fa214070d30 2023-12-01T01:48:51.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.126+0000 7fa209ffb640 1 -- 172.21.15.115:0/5259820 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa21406c9b0 con 0x7fa214070d30 2023-12-01T01:48:51.128 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 -- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1f806a5e0 msgr2=0x7fa1f806caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21943c640 1 -- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1f806a5e0 msgr2=0x7fa1f806caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21943c640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1f806a5e0 0x7fa1f806caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1f806a5e0 0x7fa1f806caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 -- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 msgr2=0x7fa214155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 0x7fa214155030 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa204009890 tx=0x7fa2040046c0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 -- 172.21.15.115:0/5259820 shutdown_connections 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1f806a5e0 0x7fa1f806caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa21415cf10 0x7fa214155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa214070d30 0x7fa214155030 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 --2- 172.21.15.115:0/5259820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2140705e0 0x7fa214152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 -- 172.21.15.115:0/5259820 >> 172.21.15.115:0/5259820 conn(0x7fa2140b1720 msgr2=0x7fa2140b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:51.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 -- 172.21.15.115:0/5259820 shutdown_connections 2023-12-01T01:48:51.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.130+0000 7fa21b6a0640 1 -- 172.21.15.115:0/5259820 wait complete. 2023-12-01T01:48:51.156 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:48:51.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.350+0000 7f4bf7fff640 1 Processor -- start 2023-12-01T01:48:51.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.350+0000 7f4bf7fff640 1 -- start start 2023-12-01T01:48:51.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.352+0000 7f4bf7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bf815df70 0x7f4bf8159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.352+0000 7f4bf7fff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf806c300 con 0x7f4bf81431a0 2023-12-01T01:48:51.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.352+0000 7f4bf7fff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf80cdba0 con 0x7f4bf8070140 2023-12-01T01:48:51.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.352+0000 7f4bf7fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf80cde80 con 0x7f4bf815df70 2023-12-01T01:48:51.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.352+0000 7f4bf6ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4bf81431a0 0x7f4bf80cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59648/0 (socket says 172.21.15.115:59648) 2023-12-01T01:48:51.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.352+0000 7f4bf6ffd640 1 -- 172.21.15.115:0/3500620534 learned_addr learned my addr 172.21.15.115:0/3500620534 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf67fc640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bf815df70 0x7f4bf8159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365543078 0 0) 0x7f4bf806c300 con 0x7f4bf81431a0 2023-12-01T01:48:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be0003520 con 0x7f4bf81431a0 2023-12-01T01:48:51.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1243445610 0 0) 0x7f4be0003520 con 0x7f4bf81431a0 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bf815df70 msgr2=0x7f4bf8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bf815df70 0x7f4bf8159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 >> v1:172.21.15.142:6789/0 conn(0x7f4bf8070140 legacy=0x7f4bf806c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf80ce160 con 0x7f4bf81431a0 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4be8001670 con 0x7f4bf81431a0 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 >> v1:172.21.15.115:6789/0 conn(0x7f4bf81431a0 legacy=0x7f4bf80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be00039a0 0x7f4be0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bf815df70 0x7f4be00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4be0004d00 0x7f4be0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf80cde80 con 0x7f4be00039a0 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be000bb40 con 0x7f4be0004d00 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.353+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be000bd00 con 0x7f4bf815df70 2023-12-01T01:48:51.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.354+0000 7f4bf6ffd640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be00039a0 0x7f4be0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.354+0000 7f4bf67fc640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bf815df70 0x7f4be00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.354+0000 7f4bf77fe640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4be0004d00 0x7f4be0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.354+0000 7f4bf6ffd640 1 -- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bf815df70 msgr2=0x7f4be00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.354+0000 7f4bf6ffd640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bf815df70 0x7f4be00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.354+0000 7f4bf6ffd640 1 -- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4be0004d00 msgr2=0x7f4be0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.354+0000 7f4bf6ffd640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4be0004d00 0x7f4be0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.354+0000 7f4bf6ffd640 1 -- 172.21.15.115:0/3500620534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be0003520 con 0x7f4be00039a0 2023-12-01T01:48:51.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.354+0000 7f4bf77fe640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4be0004d00 0x7f4be0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.355+0000 7f4bf6ffd640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be00039a0 0x7f4be0003dc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4be800c0e0 tx=0x7f4be00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e1b193691c859a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.355+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8005ad0 con 0x7f4be00039a0 2023-12-01T01:48:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.355+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be8005c30 con 0x7f4be00039a0 2023-12-01T01:48:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.355+0000 7f4bf5ffb640 1 -- 172.21.15.115:0/3500620534 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be802c810 con 0x7f4be00039a0 2023-12-01T01:48:51.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.356+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3500620534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be00039a0 msgr2=0x7f4be0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.356+0000 7f4bf7fff640 1 --2- 172.21.15.115:0/3500620534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be00039a0 0x7f4be0003dc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4be800c0e0 tx=0x7f4be00057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.356+0000 7f4bf77fe640 1 -- 172.21.15.115:0/3500620534 reap_dead start 2023-12-01T01:48:51.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.356+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3500620534 shutdown_connections 2023-12-01T01:48:51.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.356+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3500620534 >> 172.21.15.115:0/3500620534 conn(0x7f4bf8073330 msgr2=0x7f4bf80cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:51.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.356+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3500620534 shutdown_connections 2023-12-01T01:48:51.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.356+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3500620534 wait complete. 2023-12-01T01:48:51.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.358+0000 7f4bf7fff640 1 Processor -- start 2023-12-01T01:48:51.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.358+0000 7f4bf7fff640 1 -- start start 2023-12-01T01:48:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.359+0000 7f4bf7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4be00039a0 0x7f4bf8154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.359+0000 7f4bf7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf81593e0 0x7f4bf81558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.359+0000 7f4bf7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf8159860 0x7f4bf80b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.359+0000 7f4bf7fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8158140 con 0x7f4bf8159860 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.359+0000 7f4bf7fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf81384c0 con 0x7f4bf81593e0 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.359+0000 7f4bf7fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8158340 con 0x7f4be00039a0 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.359+0000 7f4bf77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf8159860 0x7f4bf80b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.359+0000 7f4bf77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf8159860 0x7f4bf80b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42648/0 (socket says 172.21.15.115:42648) 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.359+0000 7f4bf77fe640 1 -- 172.21.15.115:0/3283912425 learned_addr learned my addr 172.21.15.115:0/3283912425 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.360+0000 7f4bf6ffd640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4be00039a0 0x7f4bf8154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.360+0000 7f4bf67fc640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf81593e0 0x7f4bf81558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.360+0000 7f4bf77fe640 1 -- 172.21.15.115:0/3283912425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4be00039a0 msgr2=0x7f4bf8154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.360+0000 7f4bf77fe640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4be00039a0 0x7f4bf8154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.360+0000 7f4bf77fe640 1 -- 172.21.15.115:0/3283912425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf81593e0 msgr2=0x7f4bf81558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.360+0000 7f4bf77fe640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf81593e0 0x7f4bf81558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.360+0000 7f4bf77fe640 1 -- 172.21.15.115:0/3283912425 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8158620 con 0x7f4bf8159860 2023-12-01T01:48:51.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.360+0000 7f4bf6ffd640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4be00039a0 0x7f4bf8154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:48:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.360+0000 7f4bf77fe640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf8159860 0x7f4bf80b3660 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0000f80 tx=0x7f4bf000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.361+0000 7f4bdf7fe640 1 -- 172.21.15.115:0/3283912425 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0006d50 con 0x7f4bf8159860 2023-12-01T01:48:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.361+0000 7f4bdf7fe640 1 -- 172.21.15.115:0/3283912425 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0003970 con 0x7f4bf8159860 2023-12-01T01:48:51.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.361+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3283912425 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf8158900 con 0x7f4bf8159860 2023-12-01T01:48:51.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.361+0000 7f4bdf7fe640 1 -- 172.21.15.115:0/3283912425 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0003d20 con 0x7f4bf8159860 2023-12-01T01:48:51.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.361+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3283912425 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf8158ec0 con 0x7f4bf8159860 2023-12-01T01:48:51.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.362+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3283912425 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf80618d0 con 0x7f4bf8159860 2023-12-01T01:48:51.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.364+0000 7f4bdf7fe640 1 -- 172.21.15.115:0/3283912425 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 88778+0+0 (secure 0 0 0) 0x7f4bf0005070 con 0x7f4bf8159860 2023-12-01T01:48:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.365+0000 7f4bdf7fe640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4be406a5e0 0x7f4be406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.365+0000 7f4bf6ffd640 1 -- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4be406a5e0 msgr2=0x7f4be406caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.365+0000 7f4bdf7fe640 1 -- 172.21.15.115:0/3283912425 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6731+0+0 (secure 0 0 0) 0x7f4bf0088f40 con 0x7f4bf8159860 2023-12-01T01:48:51.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.365+0000 7f4bf6ffd640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4be406a5e0 0x7f4be406caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:48:51.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.370+0000 7f4bdf7fe640 1 -- 172.21.15.115:0/3283912425 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4bf0053da0 con 0x7f4bf8159860 2023-12-01T01:48:51.419 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:51.418+0000 7f7b93048040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:48:51.517 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:51.516+0000 7f2b7560f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:48:51.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.566+0000 7f4bf6ffd640 1 -- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4be406a5e0 msgr2=0x7f4be406caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:51.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.566+0000 7f4bf6ffd640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4be406a5e0 0x7f4be406caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:48:51.644 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:51.643+0000 7f7b93048040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:48:51.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.642+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3283912425 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4bf80d0560 con 0x7f4bf8159860 2023-12-01T01:48:51.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.644+0000 7f4bdf7fe640 1 -- 172.21.15.115:0/3283912425 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4bf00573c0 con 0x7f4bf8159860 2023-12-01T01:48:51.646 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:48:51.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.652+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4be406a5e0 msgr2=0x7f4be406caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:48:51.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.652+0000 7f4bf7fff640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4be406a5e0 0x7f4be406caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.652+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf8159860 msgr2=0x7f4bf80b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.652+0000 7f4bf7fff640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf8159860 0x7f4bf80b3660 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0000f80 tx=0x7f4bf000dc00 comp rx=0 tx=0).stop 2023-12-01T01:48:51.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.652+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3283912425 shutdown_connections 2023-12-01T01:48:51.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.652+0000 7f4bf7fff640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4be406a5e0 0x7f4be406caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.652+0000 7f4bf7fff640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf8159860 0x7f4bf80b3660 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.652+0000 7f4bf7fff640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf81593e0 0x7f4bf81558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.652+0000 7f4bf7fff640 1 --2- 172.21.15.115:0/3283912425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4be00039a0 0x7f4bf8154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.652+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3283912425 >> 172.21.15.115:0/3283912425 conn(0x7f4bf8073330 msgr2=0x7f4bf80cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:51.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.653+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3283912425 shutdown_connections 2023-12-01T01:48:51.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.654+0000 7f4bf7fff640 1 -- 172.21.15.115:0/3283912425 wait complete. 2023-12-01T01:48:51.685 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:48:51.776 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:51.775+0000 7f2b7560f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:48:51.818 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:51.817+0000 7f7b93048040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:48:51.868 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:51.867+0000 7f7b93048040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:48:51.903 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:51.901+0000 7f2b7560f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:48:51.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.914+0000 7fda87484640 1 Processor -- start 2023-12-01T01:48:51.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.915+0000 7fda87484640 1 -- start start 2023-12-01T01:48:51.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda87484640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5d80 0x7fda800d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda87484640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fda800d1230 0x7fda800d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda87484640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda800ce2e0 con 0x7fda8015cd20 2023-12-01T01:48:51.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda87484640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda800b2160 con 0x7fda800d1230 2023-12-01T01:48:51.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda87484640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda800d1be0 con 0x7fda800b5d80 2023-12-01T01:48:51.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda85220640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fda8015cd20 0x7fda800b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59650/0 (socket says 172.21.15.115:59650) 2023-12-01T01:48:51.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda85220640 1 -- 172.21.15.115:0/2572295200 learned_addr learned my addr 172.21.15.115:0/2572295200 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda85a21640 1 --2- 172.21.15.115:0/2572295200 >> v2:172.21.15.142:3300/0 conn(0x7fda800d1230 0x7fda800d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda84a1f640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5d80 0x7fda800d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943911529 0 0) 0x7fda800ce2e0 con 0x7fda8015cd20 2023-12-01T01:48:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.916+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda6c003520 con 0x7fda8015cd20 2023-12-01T01:48:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2054233140 0 0) 0x7fda6c003520 con 0x7fda8015cd20 2023-12-01T01:48:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5d80 msgr2=0x7fda800d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5d80 0x7fda800d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 >> v2:172.21.15.142:3300/0 conn(0x7fda800d1230 msgr2=0x7fda800d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 --2- 172.21.15.115:0/2572295200 >> v2:172.21.15.142:3300/0 conn(0x7fda800d1230 0x7fda800d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda800ce000 con 0x7fda8015cd20 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fda7c001670 con 0x7fda8015cd20 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 >> v1:172.21.15.115:6789/0 conn(0x7fda8015cd20 legacy=0x7fda800b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5d80 0x7fda6c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda6c003f60 0x7fda6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda6c004d00 0x7fda6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c00b980 con 0x7fda6c004d00 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c00baf0 con 0x7fda6c003f60 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c00bcb0 con 0x7fda800b5d80 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda85a21640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda6c004d00 0x7fda6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda85220640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5d80 0x7fda6c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.917+0000 7fda84a1f640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda6c003f60 0x7fda6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.918+0000 7fda85a21640 1 -- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5d80 msgr2=0x7fda6c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:51.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.918+0000 7fda85a21640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5d80 0x7fda6c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.918+0000 7fda85a21640 1 -- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda6c003f60 msgr2=0x7fda6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:51.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.918+0000 7fda85a21640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda6c003f60 0x7fda6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.918+0000 7fda85a21640 1 -- 172.21.15.115:0/2572295200 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c003520 con 0x7fda6c004d00 2023-12-01T01:48:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.920+0000 7fda85a21640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda6c004d00 0x7fda6c005180 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fda78009fc0 tx=0x7fda6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79708373adf3dfe server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.920+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7802cc60 con 0x7fda6c004d00 2023-12-01T01:48:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.920+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7800f040 con 0x7fda6c004d00 2023-12-01T01:48:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.920+0000 7fda77fff640 1 -- 172.21.15.115:0/2572295200 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78034440 con 0x7fda6c004d00 2023-12-01T01:48:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.920+0000 7fda87484640 1 -- 172.21.15.115:0/2572295200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda6c004d00 msgr2=0x7fda6c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.920+0000 7fda87484640 1 --2- 172.21.15.115:0/2572295200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda6c004d00 0x7fda6c005180 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fda78009fc0 tx=0x7fda6c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.920+0000 7fda85a21640 1 -- 172.21.15.115:0/2572295200 reap_dead start 2023-12-01T01:48:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.920+0000 7fda87484640 1 -- 172.21.15.115:0/2572295200 shutdown_connections 2023-12-01T01:48:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.920+0000 7fda87484640 1 -- 172.21.15.115:0/2572295200 >> 172.21.15.115:0/2572295200 conn(0x7fda8006a0e0 msgr2=0x7fda8006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:51.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.921+0000 7fda87484640 1 -- 172.21.15.115:0/2572295200 shutdown_connections 2023-12-01T01:48:51.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.921+0000 7fda87484640 1 -- 172.21.15.115:0/2572295200 wait complete. 2023-12-01T01:48:51.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.922+0000 7fda87484640 1 Processor -- start 2023-12-01T01:48:51.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.922+0000 7fda87484640 1 -- start start 2023-12-01T01:48:51.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.923+0000 7fda87484640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda801564d0 0x7fda800b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.923+0000 7fda87484640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5350 0x7fda801f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.923+0000 7fda87484640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda801568b0 0x7fda801f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.923+0000 7fda87484640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda801f69f0 con 0x7fda801564d0 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.923+0000 7fda87484640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8015f6f0 con 0x7fda801568b0 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.923+0000 7fda87484640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8015f9a0 con 0x7fda800b5350 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda85220640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda801564d0 0x7fda800b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda85220640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda801564d0 0x7fda800b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42678/0 (socket says 172.21.15.115:42678) 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda85220640 1 -- 172.21.15.115:0/1810555291 learned_addr learned my addr 172.21.15.115:0/1810555291 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda84a1f640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5350 0x7fda801f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda85220640 1 -- 172.21.15.115:0/1810555291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5350 msgr2=0x7fda801f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda85220640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5350 0x7fda801f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda85220640 1 -- 172.21.15.115:0/1810555291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda801568b0 msgr2=0x7fda801f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda85220640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda801568b0 0x7fda801f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda85220640 1 -- 172.21.15.115:0/1810555291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8015fc80 con 0x7fda801564d0 2023-12-01T01:48:51.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda85220640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda801564d0 0x7fda800b4e10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fda7c007290 tx=0x7fda7c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:51.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.924+0000 7fda75ffb640 1 -- 172.21.15.115:0/1810555291 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c00f8e0 con 0x7fda801564d0 2023-12-01T01:48:51.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.925+0000 7fda75ffb640 1 -- 172.21.15.115:0/1810555291 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c00fa40 con 0x7fda801564d0 2023-12-01T01:48:51.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.925+0000 7fda75ffb640 1 -- 172.21.15.115:0/1810555291 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c00fdf0 con 0x7fda801564d0 2023-12-01T01:48:51.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.925+0000 7fda87484640 1 -- 172.21.15.115:0/1810555291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8015ff60 con 0x7fda801564d0 2023-12-01T01:48:51.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.925+0000 7fda87484640 1 -- 172.21.15.115:0/1810555291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda801f25a0 con 0x7fda801564d0 2023-12-01T01:48:51.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.926+0000 7fda87484640 1 -- 172.21.15.115:0/1810555291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda800b29a0 con 0x7fda801564d0 2023-12-01T01:48:51.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.927+0000 7fda75ffb640 1 -- 172.21.15.115:0/1810555291 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 88778+0+0 (secure 0 0 0) 0x7fda7c01d460 con 0x7fda801564d0 2023-12-01T01:48:51.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.928+0000 7fda75ffb640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda6406a5e0 0x7fda6406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:51.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.928+0000 7fda75ffb640 1 -- 172.21.15.115:0/1810555291 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6907+0+0 (secure 0 0 0) 0x7fda7c089240 con 0x7fda801564d0 2023-12-01T01:48:51.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.928+0000 7fda84a1f640 1 -- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda6406a5e0 msgr2=0x7fda6406caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:51.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.928+0000 7fda84a1f640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda6406a5e0 0x7fda6406caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:48:51.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:51.932+0000 7fda75ffb640 1 -- 172.21.15.115:0/1810555291 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fda800b29a0 con 0x7fda801564d0 2023-12-01T01:48:51.958 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:51.956+0000 7f2b7560f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:48:51.977 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:51.976+0000 7f7b93048040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:48:52.069 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:52.068+0000 7f2b7560f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:48:52.084 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:52.083+0000 7f7b93048040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:48:52.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.128+0000 7fda87484640 1 -- 172.21.15.115:0/1810555291 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fda800b29a0 con 0x7fda801564d0 2023-12-01T01:48:52.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.129+0000 7fda84a1f640 1 -- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda6406a5e0 msgr2=0x7fda6406caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:52.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.129+0000 7fda84a1f640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda6406a5e0 0x7fda6406caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:48:52.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.129+0000 7fda75ffb640 1 -- 172.21.15.115:0/1810555291 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v50) v1 ==== 80+0+349 (secure 0 0 0) 0x7fda800b29a0 con 0x7fda801564d0 2023-12-01T01:48:52.131 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/88317 2023-12-02T01:37:37.403693+0000 2023-12-01T01:48:52.131 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/88317 2023-12-02T01:37:37.403693+0000 2023-12-01T01:48:52.131 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/4134228660 2023-12-02T01:37:37.403693+0000 2023-12-01T01:48:52.131 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/297799080 2023-12-02T01:37:37.403693+0000 2023-12-01T01:48:52.131 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1314334167 2023-12-02T01:37:37.403693+0000 2023-12-01T01:48:52.131 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2515539287 2023-12-02T01:37:37.403693+0000 2023-12-01T01:48:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:listed 6 entries 2023-12-01T01:48:52.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.131+0000 7fda87484640 1 -- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda6406a5e0 msgr2=0x7fda6406caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:48:52.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.131+0000 7fda87484640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda6406a5e0 0x7fda6406caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.131+0000 7fda87484640 1 -- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda801564d0 msgr2=0x7fda800b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:52.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.131+0000 7fda87484640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda801564d0 0x7fda800b4e10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fda7c007290 tx=0x7fda7c007750 comp rx=0 tx=0).stop 2023-12-01T01:48:52.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.131+0000 7fda87484640 1 -- 172.21.15.115:0/1810555291 shutdown_connections 2023-12-01T01:48:52.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.131+0000 7fda87484640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda6406a5e0 0x7fda6406caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.131+0000 7fda87484640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda800b5350 0x7fda801f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.131+0000 7fda87484640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda801568b0 0x7fda801f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.131+0000 7fda87484640 1 --2- 172.21.15.115:0/1810555291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda801564d0 0x7fda800b4e10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.131+0000 7fda87484640 1 -- 172.21.15.115:0/1810555291 >> 172.21.15.115:0/1810555291 conn(0x7fda8006a0e0 msgr2=0x7fda801588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:52.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.132+0000 7fda87484640 1 -- 172.21.15.115:0/1810555291 shutdown_connections 2023-12-01T01:48:52.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.132+0000 7fda87484640 1 -- 172.21.15.115:0/1810555291 wait complete. 2023-12-01T01:48:52.151 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T01:48:52.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:52.151 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T01:48:52.183 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:52.181+0000 7f2b7560f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:48:52.241 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:52.240+0000 7f7b93048040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:48:52.244 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:48:52.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:48:52.244 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T01:48:52.291 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:52.290+0000 7f7b93048040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:48:52.334 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:48:52.334 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:48:52.351 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:52.349+0000 7f2b7560f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:48:52.388 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:48:52.388 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:48:52.388 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:48:52.403 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:52.401+0000 7f2b7560f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:48:52.414 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:52.413+0000 7f7b93048040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:48:52.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:48:52.442 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:48:52.442 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:48:52.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:48:52.497 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:48:52.536 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:52.535+0000 7f2b7560f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:48:52.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.694+0000 7f23a9e6c640 1 Processor -- start 2023-12-01T01:48:52.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.694+0000 7f23a9e6c640 1 -- start start 2023-12-01T01:48:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.696+0000 7f23a9e6c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f23a4145620 0x7f23a40ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.696+0000 7f23a9e6c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a40cdd80 0x7f23a406b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.696+0000 7f23a9e6c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a40ad7e0 con 0x7f23a406b7b0 2023-12-01T01:48:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.696+0000 7f23a9e6c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a406bc20 con 0x7f23a4145620 2023-12-01T01:48:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.696+0000 7f23a9e6c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a40cc8e0 con 0x7f23a40cdd80 2023-12-01T01:48:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.696+0000 7f23a966b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f23a406b7b0 0x7f23a4160750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59666/0 (socket says 172.21.15.115:59666) 2023-12-01T01:48:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.697+0000 7f23a966b640 1 -- 172.21.15.115:0/3364724609 learned_addr learned my addr 172.21.15.115:0/3364724609 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.697+0000 7f23a8e6a640 1 --2- 172.21.15.115:0/3364724609 >> v2:172.21.15.142:3300/0 conn(0x7f23a4145620 0x7f23a40ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.697+0000 7f23a3fff640 1 --2- 172.21.15.115:0/3364724609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a40cdd80 0x7f23a406b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.697+0000 7f23a37fe640 1 -- 172.21.15.115:0/3364724609 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843167181 0 0) 0x7f23a40ad7e0 con 0x7f23a406b7b0 2023-12-01T01:48:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.697+0000 7f23a37fe640 1 -- 172.21.15.115:0/3364724609 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2394003520 con 0x7f23a406b7b0 2023-12-01T01:48:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.697+0000 7f23a8e6a640 1 -- 172.21.15.115:0/3364724609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a40cdd80 msgr2=0x7f23a406b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.697+0000 7f23a8e6a640 1 --2- 172.21.15.115:0/3364724609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a40cdd80 0x7f23a406b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.697+0000 7f23a8e6a640 1 -- 172.21.15.115:0/3364724609 >> v1:172.21.15.115:6789/0 conn(0x7f23a406b7b0 legacy=0x7f23a4160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.697+0000 7f23a8e6a640 1 -- 172.21.15.115:0/3364724609 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a40ce260 con 0x7f23a4145620 2023-12-01T01:48:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.698+0000 7f23a8e6a640 1 --2- 172.21.15.115:0/3364724609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a4145620 0x7f23a40ad2a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f23980097a0 tx=0x7f239802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f673b93ff5431b6f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.699+0000 7f23a37fe640 1 -- 172.21.15.115:0/3364724609 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239802d040 con 0x7f23a4145620 2023-12-01T01:48:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.699+0000 7f23a37fe640 1 -- 172.21.15.115:0/3364724609 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398031810 con 0x7f23a4145620 2023-12-01T01:48:52.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.699+0000 7f23a37fe640 1 -- 172.21.15.115:0/3364724609 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398031b80 con 0x7f23a4145620 2023-12-01T01:48:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.699+0000 7f23a9e6c640 1 -- 172.21.15.115:0/3364724609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a4145620 msgr2=0x7f23a40ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.699+0000 7f23a9e6c640 1 --2- 172.21.15.115:0/3364724609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a4145620 0x7f23a40ad2a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f23980097a0 tx=0x7f239802b8b0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.700+0000 7f23a9e6c640 1 -- 172.21.15.115:0/3364724609 shutdown_connections 2023-12-01T01:48:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.700+0000 7f23a9e6c640 1 --2- 172.21.15.115:0/3364724609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a40cdd80 0x7f23a406b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.700+0000 7f23a9e6c640 1 --2- 172.21.15.115:0/3364724609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a4145620 0x7f23a40ad2a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.700+0000 7f23a9e6c640 1 -- 172.21.15.115:0/3364724609 >> 172.21.15.115:0/3364724609 conn(0x7f23a40ce640 msgr2=0x7f23a40cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:52.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.700+0000 7f23a9e6c640 1 -- 172.21.15.115:0/3364724609 shutdown_connections 2023-12-01T01:48:52.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.700+0000 7f23a9e6c640 1 -- 172.21.15.115:0/3364724609 wait complete. 2023-12-01T01:48:52.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.701+0000 7f23a9e6c640 1 Processor -- start 2023-12-01T01:48:52.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.701+0000 7f23a9e6c640 1 -- start start 2023-12-01T01:48:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.702+0000 7f23a9e6c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a406b7b0 0x7f23a4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.702+0000 7f23a9e6c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23a40cdd80 0x7f23a40705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.702+0000 7f23a9e6c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a4145620 0x7f23a40c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.702+0000 7f23a9e6c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a40c7c90 con 0x7f23a40cdd80 2023-12-01T01:48:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.702+0000 7f23a9e6c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4070b20 con 0x7f23a4145620 2023-12-01T01:48:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.702+0000 7f23a9e6c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a40c7f30 con 0x7f23a406b7b0 2023-12-01T01:48:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.703+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23a40cdd80 0x7f23a40705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.703+0000 7f23a3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23a40cdd80 0x7f23a40705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42694/0 (socket says 172.21.15.115:42694) 2023-12-01T01:48:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.703+0000 7f23a3fff640 1 -- 172.21.15.115:0/2129799000 learned_addr learned my addr 172.21.15.115:0/2129799000 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.703+0000 7f23a966b640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a4145620 0x7f23a40c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.703+0000 7f23a8e6a640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a406b7b0 0x7f23a4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.703+0000 7f23a3fff640 1 -- 172.21.15.115:0/2129799000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a406b7b0 msgr2=0x7f23a4071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.703+0000 7f23a3fff640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a406b7b0 0x7f23a4071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.703+0000 7f23a3fff640 1 -- 172.21.15.115:0/2129799000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a4145620 msgr2=0x7f23a40c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.703+0000 7f23a3fff640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a4145620 0x7f23a40c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.703+0000 7f23a3fff640 1 -- 172.21.15.115:0/2129799000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2398009400 con 0x7f23a40cdd80 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.704+0000 7f23a3fff640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23a40cdd80 0x7f23a40705e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f239000e9b0 tx=0x7f239000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.704+0000 7f23a17fa640 1 -- 172.21.15.115:0/2129799000 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390009670 con 0x7f23a40cdd80 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.704+0000 7f23a17fa640 1 -- 172.21.15.115:0/2129799000 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239000c9a0 con 0x7f23a40cdd80 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.704+0000 7f23a9e6c640 1 -- 172.21.15.115:0/2129799000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a40c8210 con 0x7f23a40cdd80 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.704+0000 7f23a17fa640 1 -- 172.21.15.115:0/2129799000 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000cd50 con 0x7f23a40cdd80 2023-12-01T01:48:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.704+0000 7f23a9e6c640 1 -- 172.21.15.115:0/2129799000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a40c5bc0 con 0x7f23a40cdd80 2023-12-01T01:48:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.705+0000 7f23a9e6c640 1 -- 172.21.15.115:0/2129799000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a40c8730 con 0x7f23a40cdd80 2023-12-01T01:48:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.706+0000 7f23a17fa640 1 -- 172.21.15.115:0/2129799000 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 88778+0+0 (secure 0 0 0) 0x7f2390013470 con 0x7f23a40cdd80 2023-12-01T01:48:52.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.707+0000 7f23a17fa640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f238806a5e0 0x7f238806caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.707+0000 7f23a17fa640 1 -- 172.21.15.115:0/2129799000 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6619+0+0 (secure 0 0 0) 0x7f2390089360 con 0x7f23a40cdd80 2023-12-01T01:48:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.707+0000 7f23a8e6a640 1 -- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f238806a5e0 msgr2=0x7f238806caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.707+0000 7f23a8e6a640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f238806a5e0 0x7f238806caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:48:52.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.712+0000 7f23a17fa640 1 -- 172.21.15.115:0/2129799000 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2390054220 con 0x7f23a40cdd80 2023-12-01T01:48:52.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.908+0000 7f23a8e6a640 1 -- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f238806a5e0 msgr2=0x7f238806caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:52.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.908+0000 7f23a8e6a640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f238806a5e0 0x7f238806caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:48:52.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.932+0000 7f23a9e6c640 1 -- 172.21.15.115:0/2129799000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23a40673a0 con 0x7f23a40cdd80 2023-12-01T01:48:52.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.932+0000 7f23a17fa640 1 -- 172.21.15.115:0/2129799000 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f2390057840 con 0x7f23a40cdd80 2023-12-01T01:48:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:52.934 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1226717603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1226717603},{"type":"v1","addr":"172.21.15.142:6834","nonce":1226717603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2864659456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2864659456},{"type":"v1","addr":"172.21.15.142:6835","nonce":2864659456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4774,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/253656345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":253656345},{"type":"v1","addr":"172.21.15.115:6837","nonce":253656345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:37:23.645235+0000","modified":"2023-12-01T01:37:26.663085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5091},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5091":{"gid":5091,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6835/4242588071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4242588071},{"type":"v1","addr":"172.21.15.115:6835","nonce":4242588071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-01T01:48:52.934 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2023-12-01T01:48:52.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 -- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f238806a5e0 msgr2=0x7f238806caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:48:52.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f238806a5e0 0x7f238806caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 -- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23a40cdd80 msgr2=0x7f23a40705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:52.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23a40cdd80 0x7f23a40705e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f239000e9b0 tx=0x7f239000ee70 comp rx=0 tx=0).stop 2023-12-01T01:48:52.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 -- 172.21.15.115:0/2129799000 shutdown_connections 2023-12-01T01:48:52.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f238806a5e0 0x7f238806caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23a4145620 0x7f23a40c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23a40cdd80 0x7f23a40705e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 --2- 172.21.15.115:0/2129799000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23a406b7b0 0x7f23a4071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:52.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 -- 172.21.15.115:0/2129799000 >> 172.21.15.115:0/2129799000 conn(0x7f23a40ce640 msgr2=0x7f23a4160660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:52.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 -- 172.21.15.115:0/2129799000 shutdown_connections 2023-12-01T01:48:52.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:52.935+0000 7f23a9e6c640 1 -- 172.21.15.115:0/2129799000 wait complete. 2023-12-01T01:48:52.955 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:48:53.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.188+0000 7f33bea96640 1 Processor -- start 2023-12-01T01:48:53.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.188+0000 7f33bea96640 1 -- start start 2023-12-01T01:48:53.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.190+0000 7f33bea96640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f33b80d1940 0x7f33b80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.190+0000 7f33bea96640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b80ce2e0 con 0x7f33b80b5d80 2023-12-01T01:48:53.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.190+0000 7f33bea96640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b80ce000 con 0x7f33b815cd20 2023-12-01T01:48:53.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.190+0000 7f33bea96640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b80b2160 con 0x7f33b80d1940 2023-12-01T01:48:53.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.190+0000 7f33b7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f33b80b5d80 0x7f33b80d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59680/0 (socket says 172.21.15.115:59680) 2023-12-01T01:48:53.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.190+0000 7f33b7fff640 1 -- 172.21.15.115:0/4185532488 learned_addr learned my addr 172.21.15.115:0/4185532488 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:53.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.190+0000 7f33bd033640 1 --2- 172.21.15.115:0/4185532488 >> v2:172.21.15.142:3300/0 conn(0x7f33b80d1940 0x7f33b80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.190+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461386718 0 0) 0x7f33b80ce2e0 con 0x7f33b80b5d80 2023-12-01T01:48:53.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.190+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a8003520 con 0x7f33b80b5d80 2023-12-01T01:48:53.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2878296550 0 0) 0x7f33a8003520 con 0x7f33b80b5d80 2023-12-01T01:48:53.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 >> v2:172.21.15.142:3300/0 conn(0x7f33b80d1940 msgr2=0x7f33b80d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 --2- 172.21.15.115:0/4185532488 >> v2:172.21.15.142:3300/0 conn(0x7f33b80d1940 0x7f33b80d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 >> v1:172.21.15.142:6789/0 conn(0x7f33b815cd20 legacy=0x7f33b80b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b80cd3d0 con 0x7f33b80b5d80 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f33a4001670 con 0x7f33b80b5d80 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 >> v1:172.21.15.115:6789/0 conn(0x7f33b80b5d80 legacy=0x7f33b80d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33a80039e0 0x7f33a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a80047c0 0x7f33a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a800c0f0 con 0x7f33a80051d0 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a800c260 con 0x7f33a80047c0 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a800c420 con 0x7f33a80039e0 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33bd033640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33b7fff640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a80047c0 0x7f33a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.191+0000 7f33bc832640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33a80039e0 0x7f33a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.192+0000 7f33bd033640 1 -- 172.21.15.115:0/4185532488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33a80039e0 msgr2=0x7f33a8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:53.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.192+0000 7f33bd033640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33a80039e0 0x7f33a8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.192+0000 7f33bd033640 1 -- 172.21.15.115:0/4185532488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a80047c0 msgr2=0x7f33a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:53.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.192+0000 7f33bd033640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33a80047c0 0x7f33a8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.192+0000 7f33bd033640 1 -- 172.21.15.115:0/4185532488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a8003520 con 0x7f33a80051d0 2023-12-01T01:48:53.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.192+0000 7f33bd033640 1 -- 172.21.15.115:0/4185532488 reap_dead start 2023-12-01T01:48:53.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.193+0000 7f33bd033640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33a800b980 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009a90 tx=0x7f33a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22e1f7d9b761d54 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:53.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.193+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0004020 con 0x7f33a80051d0 2023-12-01T01:48:53.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.193+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b0004180 con 0x7f33a80051d0 2023-12-01T01:48:53.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.193+0000 7f33b77fe640 1 -- 172.21.15.115:0/4185532488 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b002c6b0 con 0x7f33a80051d0 2023-12-01T01:48:53.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.193+0000 7f33bea96640 1 -- 172.21.15.115:0/4185532488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 msgr2=0x7f33a800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:53.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.193+0000 7f33bea96640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33a800b980 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009a90 tx=0x7f33a80057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.194+0000 7f33bea96640 1 -- 172.21.15.115:0/4185532488 shutdown_connections 2023-12-01T01:48:53.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.194+0000 7f33bea96640 1 --2- 172.21.15.115:0/4185532488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33a800b980 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.194+0000 7f33bea96640 1 -- 172.21.15.115:0/4185532488 >> 172.21.15.115:0/4185532488 conn(0x7f33b806a0e0 msgr2=0x7f33b806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:53.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.194+0000 7f33bea96640 1 -- 172.21.15.115:0/4185532488 shutdown_connections 2023-12-01T01:48:53.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.194+0000 7f33bea96640 1 -- 172.21.15.115:0/4185532488 wait complete. 2023-12-01T01:48:53.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.195+0000 7f33bea96640 1 Processor -- start 2023-12-01T01:48:53.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.195+0000 7f33bea96640 1 -- start start 2023-12-01T01:48:53.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.196+0000 7f33bea96640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33b81f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.196+0000 7f33bea96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33b815faf0 0x7f33b815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.196+0000 7f33bea96640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33b80d23b0 0x7f33b80d2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.196+0000 7f33bea96640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80d2d70 con 0x7f33a80051d0 2023-12-01T01:48:53.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.196+0000 7f33bea96640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b81f6350 con 0x7f33b80d23b0 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.196+0000 7f33bea96640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b815f480 con 0x7f33b815faf0 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bc832640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33b81f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bc832640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33b81f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42712/0 (socket says 172.21.15.115:42712) 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bc832640 1 -- 172.21.15.115:0/3906142219 learned_addr learned my addr 172.21.15.115:0/3906142219 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bd033640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33b80d23b0 0x7f33b80d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bc832640 1 -- 172.21.15.115:0/3906142219 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33b815faf0 msgr2=0x7f33b815ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33b7fff640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33b815faf0 0x7f33b815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bc832640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33b815faf0 0x7f33b815ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bc832640 1 -- 172.21.15.115:0/3906142219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33b80d23b0 msgr2=0x7f33b80d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bc832640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33b80d23b0 0x7f33b80d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bc832640 1 -- 172.21.15.115:0/3906142219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b815ff90 con 0x7f33a80051d0 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bd033640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33b80d23b0 0x7f33b80d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.197+0000 7f33bc832640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33b81f5e10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00e7d0 tx=0x7f33ac00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.198+0000 7f33b57fa640 1 -- 172.21.15.115:0/3906142219 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00eee0 con 0x7f33a80051d0 2023-12-01T01:48:53.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.198+0000 7f33b57fa640 1 -- 172.21.15.115:0/3906142219 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac003df0 con 0x7f33a80051d0 2023-12-01T01:48:53.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.198+0000 7f33b57fa640 1 -- 172.21.15.115:0/3906142219 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00c880 con 0x7f33a80051d0 2023-12-01T01:48:53.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.198+0000 7f33bea96640 1 -- 172.21.15.115:0/3906142219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b8160270 con 0x7f33a80051d0 2023-12-01T01:48:53.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.198+0000 7f33bea96640 1 -- 172.21.15.115:0/3906142219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b815e740 con 0x7f33a80051d0 2023-12-01T01:48:53.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.199+0000 7f33bea96640 1 -- 172.21.15.115:0/3906142219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b8158dd0 con 0x7f33a80051d0 2023-12-01T01:48:53.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.201+0000 7f33b57fa640 1 -- 172.21.15.115:0/3906142219 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 88778+0+0 (secure 0 0 0) 0x7f33ac00c9e0 con 0x7f33a80051d0 2023-12-01T01:48:53.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.201+0000 7f33b57fa640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f339c06a5e0 0x7f339c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.201+0000 7f33b57fa640 1 -- 172.21.15.115:0/3906142219 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6619+0+0 (secure 0 0 0) 0x7f33ac089360 con 0x7f33a80051d0 2023-12-01T01:48:53.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.201+0000 7f33b7fff640 1 -- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f339c06a5e0 msgr2=0x7f339c06caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:53.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.202+0000 7f33b7fff640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f339c06a5e0 0x7f339c06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:48:53.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.206+0000 7f33b57fa640 1 -- 172.21.15.115:0/3906142219 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f33ac054340 con 0x7f33a80051d0 2023-12-01T01:48:53.228 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:53.227+0000 7f7b93048040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:48:53.367 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.366+0000 7f2b7560f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:48:53.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.403+0000 7f33b7fff640 1 -- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f339c06a5e0 msgr2=0x7f339c06caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:53.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.403+0000 7f33b7fff640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f339c06a5e0 0x7f339c06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:48:53.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.421+0000 7f33bea96640 1 -- 172.21.15.115:0/3906142219 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33b804e890 con 0x7f33a80051d0 2023-12-01T01:48:53.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.423+0000 7f33b57fa640 1 -- 172.21.15.115:0/3906142219 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+17547 (secure 0 0 0) 0x7f33ac057960 con 0x7f33a80051d0 2023-12-01T01:48:53.425 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:53.425 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":51,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:48:52.438758+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:37:22.073677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-01T01:37:22.660650+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.23","mappings":[{"from":4,"to":5}]},{"pgid":"5.33","mappings":[{"from":4,"to":5}]},{"pgid":"5.3c","mappings":[{"from":4,"to":5}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:37:37.403693+0000","172.21.15.115:6822/88317":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/4134228660":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/297799080":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/1314334167":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/2515539287":"2023-12-02T01:37:37.403693+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1},{"begin":6,"length":1},{"begin":8,"length":1},{"begin":10,"length":1},{"begin":12,"length":1},{"begin":14,"length":1},{"begin":16,"length":1},{"begin":18,"length":2},{"begin":22,"length":2},{"begin":26,"length":3},{"begin":33,"length":5},{"begin":42,"length":1},{"begin":44,"length":3},{"begin":48,"length":1},{"begin":52,"length":3},{"begin":57,"length":2},{"begin":63,"length":1},{"begin":65,"length":4},{"begin":73,"length":1},{"begin":75,"length":4},{"begin":83,"length":4},{"begin":89,"length":1},{"begin":94,"length":1},{"begin":96,"length":2},{"begin":100,"length":3},{"begin":105,"length":3},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":166,"length":56}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":109,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:48:53.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.425+0000 7f33bea96640 1 -- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f339c06a5e0 msgr2=0x7f339c06caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:48:53.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.425+0000 7f33bea96640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f339c06a5e0 0x7f339c06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.425+0000 7f33bea96640 1 -- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 msgr2=0x7f33b81f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:53.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.425+0000 7f33bea96640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33b81f5e10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00e7d0 tx=0x7f33ac00ec90 comp rx=0 tx=0).stop 2023-12-01T01:48:53.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.425+0000 7f33bea96640 1 -- 172.21.15.115:0/3906142219 shutdown_connections 2023-12-01T01:48:53.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.425+0000 7f33bea96640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f339c06a5e0 0x7f339c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.425+0000 7f33bea96640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33b815faf0 0x7f33b815ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.425+0000 7f33bea96640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33b80d23b0 0x7f33b80d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.425+0000 7f33bea96640 1 --2- 172.21.15.115:0/3906142219 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33a80051d0 0x7f33b81f5e10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.425+0000 7f33bea96640 1 -- 172.21.15.115:0/3906142219 >> 172.21.15.115:0/3906142219 conn(0x7f33b806a0e0 msgr2=0x7f33b80cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:53.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.426+0000 7f33bea96640 1 -- 172.21.15.115:0/3906142219 shutdown_connections 2023-12-01T01:48:53.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.426+0000 7f33bea96640 1 -- 172.21.15.115:0/3906142219 wait complete. 2023-12-01T01:48:53.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:37:22.073677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:37:22.660650+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 221, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:48:53.445 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:48:53.524 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:53.524+0000 7f7b93048040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:48:53.581 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:53.580+0000 7f7b93048040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:48:53.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.632+0000 7efec7fff640 1 Processor -- start 2023-12-01T01:48:53.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.632+0000 7efec7fff640 1 -- start start 2023-12-01T01:48:53.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.633+0000 7efec7fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efec8153c30 0x7efec80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efec80cfb80 0x7efec8160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec7fff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec80cdcd0 con 0x7efec8154010 2023-12-01T01:48:53.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec7fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec8161830 con 0x7efec80cfb80 2023-12-01T01:48:53.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec7fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec80cd3a0 con 0x7efec8153c30 2023-12-01T01:48:53.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efec80cfb80 0x7efec8160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efec80cfb80 0x7efec8160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42714/0 (socket says 172.21.15.115:42714) 2023-12-01T01:48:53.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec77fe640 1 -- 172.21.15.115:0/642935002 learned_addr learned my addr 172.21.15.115:0/642935002 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:53.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec6ffd640 1 --2- 172.21.15.115:0/642935002 >> v2:172.21.15.142:3300/0 conn(0x7efec8153c30 0x7efec80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec77fe640 1 -- 172.21.15.115:0/642935002 >> v2:172.21.15.142:3300/0 conn(0x7efec8153c30 msgr2=0x7efec80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:53.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec77fe640 1 --2- 172.21.15.115:0/642935002 >> v2:172.21.15.142:3300/0 conn(0x7efec8153c30 0x7efec80ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec77fe640 1 -- 172.21.15.115:0/642935002 >> v1:172.21.15.142:6789/0 conn(0x7efec8154010 legacy=0x7efec80cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:53.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.635+0000 7efec77fe640 1 -- 172.21.15.115:0/642935002 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec80cd680 con 0x7efec80cfb80 2023-12-01T01:48:53.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.636+0000 7efec77fe640 1 --2- 172.21.15.115:0/642935002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 0x7efec8160ac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efec80d3800 tx=0x7efec002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17405b2ca742bb83 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.637+0000 7efec5ffb640 1 -- 172.21.15.115:0/642935002 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec002ca80 con 0x7efec80cfb80 2023-12-01T01:48:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.637+0000 7efec5ffb640 1 -- 172.21.15.115:0/642935002 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec002cbe0 con 0x7efec80cfb80 2023-12-01T01:48:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.637+0000 7efec5ffb640 1 -- 172.21.15.115:0/642935002 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00343f0 con 0x7efec80cfb80 2023-12-01T01:48:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.637+0000 7efec7fff640 1 -- 172.21.15.115:0/642935002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 msgr2=0x7efec8160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.637+0000 7efec7fff640 1 --2- 172.21.15.115:0/642935002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 0x7efec8160ac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efec80d3800 tx=0x7efec002bee0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.637+0000 7efec7fff640 1 -- 172.21.15.115:0/642935002 shutdown_connections 2023-12-01T01:48:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.637+0000 7efec7fff640 1 --2- 172.21.15.115:0/642935002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 0x7efec8160ac0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.637+0000 7efec7fff640 1 --2- 172.21.15.115:0/642935002 >> v2:172.21.15.142:3300/0 conn(0x7efec8153c30 0x7efec80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.638+0000 7efec7fff640 1 -- 172.21.15.115:0/642935002 >> 172.21.15.115:0/642935002 conn(0x7efec806a0b0 msgr2=0x7efec806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:53.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.638+0000 7efec7fff640 1 -- 172.21.15.115:0/642935002 shutdown_connections 2023-12-01T01:48:53.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.638+0000 7efec7fff640 1 -- 172.21.15.115:0/642935002 wait complete. 2023-12-01T01:48:53.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.640+0000 7efec7fff640 1 Processor -- start 2023-12-01T01:48:53.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.640+0000 7efec7fff640 1 -- start start 2023-12-01T01:48:53.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.641+0000 7efec7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 0x7efec8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.641+0000 7efec7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec8153c30 0x7efec80c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.641+0000 7efec7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efec8154010 0x7efec815a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.641+0000 7efec7fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec80c6a70 con 0x7efec80cfb80 2023-12-01T01:48:53.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.641+0000 7efec7fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec80c9530 con 0x7efec8153c30 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.641+0000 7efec7fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec81550b0 con 0x7efec8154010 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.641+0000 7efec6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 0x7efec8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.642+0000 7efec6ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 0x7efec8155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42730/0 (socket says 172.21.15.115:42730) 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.642+0000 7efec6ffd640 1 -- 172.21.15.115:0/683673453 learned_addr learned my addr 172.21.15.115:0/683673453 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.642+0000 7efec67fc640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec8153c30 0x7efec80c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.642+0000 7efec6ffd640 1 -- 172.21.15.115:0/683673453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efec8154010 msgr2=0x7efec815a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.642+0000 7efec6ffd640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efec8154010 0x7efec815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.642+0000 7efec6ffd640 1 -- 172.21.15.115:0/683673453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec8153c30 msgr2=0x7efec80c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.642+0000 7efec6ffd640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec8153c30 0x7efec80c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.642+0000 7efec6ffd640 1 -- 172.21.15.115:0/683673453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0009b30 con 0x7efec80cfb80 2023-12-01T01:48:53.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.642+0000 7efec6ffd640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 0x7efec8155d90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efeb800d800 tx=0x7efeb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:53.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.642+0000 7efeaf7fe640 1 -- 172.21.15.115:0/683673453 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb800b6c0 con 0x7efec80cfb80 2023-12-01T01:48:53.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.643+0000 7efeaf7fe640 1 -- 172.21.15.115:0/683673453 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb800b820 con 0x7efec80cfb80 2023-12-01T01:48:53.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.643+0000 7efeaf7fe640 1 -- 172.21.15.115:0/683673453 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb800bbb0 con 0x7efec80cfb80 2023-12-01T01:48:53.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.643+0000 7efec7fff640 1 -- 172.21.15.115:0/683673453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec8144950 con 0x7efec80cfb80 2023-12-01T01:48:53.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.643+0000 7efec7fff640 1 -- 172.21.15.115:0/683673453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec8155590 con 0x7efec80cfb80 2023-12-01T01:48:53.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.644+0000 7efec7fff640 1 -- 172.21.15.115:0/683673453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec80c92e0 con 0x7efec80cfb80 2023-12-01T01:48:53.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.645+0000 7efeaf7fe640 1 -- 172.21.15.115:0/683673453 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 15) v1 ==== 88778+0+0 (secure 0 0 0) 0x7efeb800bd10 con 0x7efec80cfb80 2023-12-01T01:48:53.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.646+0000 7efeaf7fe640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efeb406a5e0 0x7efeb406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:53.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.646+0000 7efeaf7fe640 1 -- 172.21.15.115:0/683673453 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6619+0+0 (secure 0 0 0) 0x7efeb80895c0 con 0x7efec80cfb80 2023-12-01T01:48:53.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.646+0000 7efec67fc640 1 -- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efeb406a5e0 msgr2=0x7efeb406caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:53.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.646+0000 7efec67fc640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efeb406a5e0 0x7efeb406caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:48:53.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.654+0000 7efeaf7fe640 1 -- 172.21.15.115:0/683673453 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efeb8054500 con 0x7efec80cfb80 2023-12-01T01:48:53.685 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.684+0000 7f2b7560f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:48:53.695 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:48:53.694+0000 7f7b93048040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:48:53.744 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.743+0000 7f2b7560f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:48:53.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.847+0000 7efec67fc640 1 -- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efeb406a5e0 msgr2=0x7efeb406caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:48:53.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.847+0000 7efec67fc640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efeb406a5e0 0x7efeb406caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:48:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.854+0000 7efec7fff640 1 -- 172.21.15.115:0/683673453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efec80618d0 con 0x7efec80cfb80 2023-12-01T01:48:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.855+0000 7efeaf7fe640 1 -- 172.21.15.115:0/683673453 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7efeb8057b20 con 0x7efec80cfb80 2023-12-01T01:48:53.855 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:48:53.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.857+0000 7efec7fff640 1 -- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efeb406a5e0 msgr2=0x7efeb406caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:48:53.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.857+0000 7efec7fff640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efeb406a5e0 0x7efeb406caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.857+0000 7efec7fff640 1 -- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 msgr2=0x7efec8155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.857+0000 7efec7fff640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 0x7efec8155d90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efeb800d800 tx=0x7efeb800dcc0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.857+0000 7efec7fff640 1 -- 172.21.15.115:0/683673453 shutdown_connections 2023-12-01T01:48:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.857+0000 7efec7fff640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efeb406a5e0 0x7efeb406caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.857+0000 7efec7fff640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efec8154010 0x7efec815a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.857+0000 7efec7fff640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efec8153c30 0x7efec80c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.858+0000 7efec7fff640 1 --2- 172.21.15.115:0/683673453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efec80cfb80 0x7efec8155d90 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.858+0000 7efec7fff640 1 -- 172.21.15.115:0/683673453 >> 172.21.15.115:0/683673453 conn(0x7efec806a0b0 msgr2=0x7efec806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.858+0000 7efec7fff640 1 -- 172.21.15.115:0/683673453 shutdown_connections 2023-12-01T01:48:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:53.858+0000 7efec7fff640 1 -- 172.21.15.115:0/683673453 wait complete. 2023-12-01T01:48:53.861 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.859+0000 7f2b7560f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:48:53.880 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:48:53.937 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.936+0000 7f2af9eb0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:48:53.937 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.936+0000 7f2af9eb0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:48:53.937 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.936+0000 7f2af9eb0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:48:53.937 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.936+0000 7f2af9eb0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:48:53.937 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.936+0000 7f2af9eb0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:48:53.938 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.938+0000 7f2af66a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:48:53.938 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.938+0000 7f2af66a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:48:53.938 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.938+0000 7f2af66a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:48:53.938 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.938+0000 7f2af66a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:48:53.938 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:48:53.938+0000 7f2af66a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:48:54.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.068+0000 7ff1d478e640 1 Processor -- start 2023-12-01T01:48:54.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.069+0000 7ff1d478e640 1 -- start start 2023-12-01T01:48:54.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.070+0000 7ff1d478e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff1cc0cd1e0 0x7ff1cc0b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.070+0000 7ff1d478e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1cc0b1c50 0x7ff1cc15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.070+0000 7ff1d478e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1cc0b2f30 con 0x7ff1cc142870 2023-12-01T01:48:54.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.070+0000 7ff1d478e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc0b3ae0 con 0x7ff1cc0cd1e0 2023-12-01T01:48:54.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.070+0000 7ff1d478e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc142c50 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.071+0000 7ff1d1d29640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff1cc0cd1e0 0x7ff1cc0b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.071+0000 7ff1d1d29640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff1cc0cd1e0 0x7ff1cc0b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:59424/0 (socket says 172.21.15.115:59424) 2023-12-01T01:48:54.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.071+0000 7ff1d1d29640 1 -- 172.21.15.115:0/1152768191 learned_addr learned my addr 172.21.15.115:0/1152768191 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:54.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.071+0000 7ff1d2d2b640 1 --2- 172.21.15.115:0/1152768191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1cc0b1c50 0x7ff1cc15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.072+0000 7ff1d1d29640 1 -- 172.21.15.115:0/1152768191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1cc0b1c50 msgr2=0x7ff1cc15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:54.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.072+0000 7ff1d1d29640 1 --2- 172.21.15.115:0/1152768191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1cc0b1c50 0x7ff1cc15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.072+0000 7ff1d1d29640 1 -- 172.21.15.115:0/1152768191 >> v1:172.21.15.142:6789/0 conn(0x7ff1cc142870 legacy=0x7ff1cc0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.072+0000 7ff1d1d29640 1 -- 172.21.15.115:0/1152768191 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0b20c0 con 0x7ff1cc0cd1e0 2023-12-01T01:48:54.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.073+0000 7ff1d1d29640 1 --2- 172.21.15.115:0/1152768191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc0cd1e0 0x7ff1cc0b1710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0098a0 tx=0x7ff1bc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ce8964e0063164 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:54.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.073+0000 7ff1d1528640 1 -- 172.21.15.115:0/1152768191 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc030840 con 0x7ff1cc0cd1e0 2023-12-01T01:48:54.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.073+0000 7ff1d1528640 1 -- 172.21.15.115:0/1152768191 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc0309a0 con 0x7ff1cc0cd1e0 2023-12-01T01:48:54.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.073+0000 7ff1d1528640 1 -- 172.21.15.115:0/1152768191 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc030d30 con 0x7ff1cc0cd1e0 2023-12-01T01:48:54.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.074+0000 7ff1d478e640 1 -- 172.21.15.115:0/1152768191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc0cd1e0 msgr2=0x7ff1cc0b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.074+0000 7ff1d478e640 1 --2- 172.21.15.115:0/1152768191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc0cd1e0 0x7ff1cc0b1710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0098a0 tx=0x7ff1bc02b980 comp rx=0 tx=0).stop 2023-12-01T01:48:54.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.074+0000 7ff1d478e640 1 -- 172.21.15.115:0/1152768191 shutdown_connections 2023-12-01T01:48:54.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.074+0000 7ff1d478e640 1 --2- 172.21.15.115:0/1152768191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1cc0b1c50 0x7ff1cc15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.074+0000 7ff1d478e640 1 --2- 172.21.15.115:0/1152768191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc0cd1e0 0x7ff1cc0b1710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.074+0000 7ff1d478e640 1 -- 172.21.15.115:0/1152768191 >> 172.21.15.115:0/1152768191 conn(0x7ff1cc065960 msgr2=0x7ff1cc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:54.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.074+0000 7ff1d478e640 1 -- 172.21.15.115:0/1152768191 shutdown_connections 2023-12-01T01:48:54.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.074+0000 7ff1d478e640 1 -- 172.21.15.115:0/1152768191 wait complete. 2023-12-01T01:48:54.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.077+0000 7ff1d478e640 1 Processor -- start 2023-12-01T01:48:54.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.077+0000 7ff1d478e640 1 -- start start 2023-12-01T01:48:54.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.078+0000 7ff1d478e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1cc0b1c50 0x7ff1cc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.078+0000 7ff1d478e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1cc0cd1e0 0x7ff1cc1e6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.078+0000 7ff1d478e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc142870 0x7ff1cc1e6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.078+0000 7ff1d478e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc158930 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.078+0000 7ff1d478e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1e1640 con 0x7ff1cc142870 2023-12-01T01:48:54.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.078+0000 7ff1d478e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1e1920 con 0x7ff1cc0cd1e0 2023-12-01T01:48:54.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.078+0000 7ff1d252a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1cc0b1c50 0x7ff1cc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.078+0000 7ff1d252a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1cc0b1c50 0x7ff1cc0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42828/0 (socket says 172.21.15.115:42828) 2023-12-01T01:48:54.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.078+0000 7ff1d252a640 1 -- 172.21.15.115:0/3462541789 learned_addr learned my addr 172.21.15.115:0/3462541789 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:54.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1d2d2b640 1 --2- 172.21.15.115:0/3462541789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc142870 0x7ff1cc1e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1d1d29640 1 --2- 172.21.15.115:0/3462541789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1cc0cd1e0 0x7ff1cc1e6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1d252a640 1 -- 172.21.15.115:0/3462541789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1cc0cd1e0 msgr2=0x7ff1cc1e6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1d252a640 1 --2- 172.21.15.115:0/3462541789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1cc0cd1e0 0x7ff1cc1e6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1d252a640 1 -- 172.21.15.115:0/3462541789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc142870 msgr2=0x7ff1cc1e6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1d252a640 1 --2- 172.21.15.115:0/3462541789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc142870 0x7ff1cc1e6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1d252a640 1 -- 172.21.15.115:0/3462541789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0094d0 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1d252a640 1 --2- 172.21.15.115:0/3462541789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1cc0b1c50 0x7ff1cc0b04b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800c400 tx=0x7ff1c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:54.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1c2ffd640 1 -- 172.21.15.115:0/3462541789 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c80052d0 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1c2ffd640 1 -- 172.21.15.115:0/3462541789 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8005430 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1d478e640 1 -- 172.21.15.115:0/3462541789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc1e1c00 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.079+0000 7ff1c2ffd640 1 -- 172.21.15.115:0/3462541789 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8003830 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.080+0000 7ff1d478e640 1 -- 172.21.15.115:0/3462541789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc1e21c0 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.081+0000 7ff1d478e640 1 -- 172.21.15.115:0/3462541789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc0cd5c0 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.081+0000 7ff1c2ffd640 1 -- 172.21.15.115:0/3462541789 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 88318+0+0 (secure 0 0 0) 0x7ff1c8003990 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.083+0000 7ff1c2ffd640 1 -- 172.21.15.115:0/3462541789 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff1c8087eb0 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.087+0000 7ff1c2ffd640 1 -- 172.21.15.115:0/3462541789 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff1c80533d0 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.276+0000 7ff1d478e640 1 -- 172.21.15.115:0/3462541789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1cc04e890 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.277+0000 7ff1c2ffd640 1 -- 172.21.15.115:0/3462541789 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1c80170a0 con 0x7ff1cc0b1c50 2023-12-01T01:48:54.278 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:48:54.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.279+0000 7ff1d478e640 1 -- 172.21.15.115:0/3462541789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1cc0b1c50 msgr2=0x7ff1cc0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.279+0000 7ff1d478e640 1 --2- 172.21.15.115:0/3462541789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1cc0b1c50 0x7ff1cc0b04b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800c400 tx=0x7ff1c800c8c0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.280+0000 7ff1d478e640 1 -- 172.21.15.115:0/3462541789 shutdown_connections 2023-12-01T01:48:54.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.280+0000 7ff1d478e640 1 --2- 172.21.15.115:0/3462541789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1cc0cd1e0 0x7ff1cc1e6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.280+0000 7ff1d478e640 1 --2- 172.21.15.115:0/3462541789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc142870 0x7ff1cc1e6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.280+0000 7ff1d478e640 1 --2- 172.21.15.115:0/3462541789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1cc0b1c50 0x7ff1cc0b04b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.280+0000 7ff1d478e640 1 -- 172.21.15.115:0/3462541789 >> 172.21.15.115:0/3462541789 conn(0x7ff1cc065960 msgr2=0x7ff1cc070a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:54.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.280+0000 7ff1d478e640 1 -- 172.21.15.115:0/3462541789 shutdown_connections 2023-12-01T01:48:54.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.280+0000 7ff1d478e640 1 -- 172.21.15.115:0/3462541789 wait complete. 2023-12-01T01:48:54.298 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:48:54.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.505+0000 7f73dc617640 1 Processor -- start 2023-12-01T01:48:54.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.505+0000 7f73dc617640 1 -- start start 2023-12-01T01:48:54.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73dc617640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f73d40cd1e0 0x7f73d40b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73dc617640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f73d40b1c50 0x7f73d415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73dc617640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d40b2f30 con 0x7f73d4142870 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73dc617640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d40b3ae0 con 0x7f73d40cd1e0 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73dc617640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d4142c50 con 0x7f73d40b1c50 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73d9bb2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f73d40cd1e0 0x7f73d40b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73d9bb2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f73d40cd1e0 0x7f73d40b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42834/0 (socket says 172.21.15.115:42834) 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73d9bb2640 1 -- 172.21.15.115:0/29546963 learned_addr learned my addr 172.21.15.115:0/29546963 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73dabb4640 1 --2- 172.21.15.115:0/29546963 >> v2:172.21.15.142:3300/0 conn(0x7f73d40b1c50 0x7f73d415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73d9bb2640 1 -- 172.21.15.115:0/29546963 >> v2:172.21.15.142:3300/0 conn(0x7f73d40b1c50 msgr2=0x7f73d415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73d9bb2640 1 --2- 172.21.15.115:0/29546963 >> v2:172.21.15.142:3300/0 conn(0x7f73d40b1c50 0x7f73d415a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73d9bb2640 1 -- 172.21.15.115:0/29546963 >> v1:172.21.15.142:6789/0 conn(0x7f73d4142870 legacy=0x7f73d40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:54.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.507+0000 7f73d9bb2640 1 -- 172.21.15.115:0/29546963 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d40b2080 con 0x7f73d40cd1e0 2023-12-01T01:48:54.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.508+0000 7f73d9bb2640 1 --2- 172.21.15.115:0/29546963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40cd1e0 0x7f73d40b1710 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f73d406f1b0 tx=0x7f73c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c09bf5c57c4d06 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:54.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.509+0000 7f73d93b1640 1 -- 172.21.15.115:0/29546963 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4031870 con 0x7f73d40cd1e0 2023-12-01T01:48:54.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.509+0000 7f73d93b1640 1 -- 172.21.15.115:0/29546963 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c40319d0 con 0x7f73d40cd1e0 2023-12-01T01:48:54.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.509+0000 7f73d93b1640 1 -- 172.21.15.115:0/29546963 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4031d60 con 0x7f73d40cd1e0 2023-12-01T01:48:54.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.509+0000 7f73dc617640 1 -- 172.21.15.115:0/29546963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40cd1e0 msgr2=0x7f73d40b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.509+0000 7f73dc617640 1 --2- 172.21.15.115:0/29546963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40cd1e0 0x7f73d40b1710 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f73d406f1b0 tx=0x7f73c402b910 comp rx=0 tx=0).stop 2023-12-01T01:48:54.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.509+0000 7f73dc617640 1 -- 172.21.15.115:0/29546963 shutdown_connections 2023-12-01T01:48:54.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.509+0000 7f73dc617640 1 --2- 172.21.15.115:0/29546963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40cd1e0 0x7f73d40b1710 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.509+0000 7f73dc617640 1 --2- 172.21.15.115:0/29546963 >> v2:172.21.15.142:3300/0 conn(0x7f73d40b1c50 0x7f73d415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.509+0000 7f73dc617640 1 -- 172.21.15.115:0/29546963 >> 172.21.15.115:0/29546963 conn(0x7f73d4065960 msgr2=0x7f73d4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:54.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.510+0000 7f73dc617640 1 -- 172.21.15.115:0/29546963 shutdown_connections 2023-12-01T01:48:54.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.510+0000 7f73dc617640 1 -- 172.21.15.115:0/29546963 wait complete. 2023-12-01T01:48:54.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.511+0000 7f73dc617640 1 Processor -- start 2023-12-01T01:48:54.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.511+0000 7f73dc617640 1 -- start start 2023-12-01T01:48:54.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.512+0000 7f73dc617640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40b1c50 0x7f73d40adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.512+0000 7f73dc617640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73d40cd1e0 0x7f73d40c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.512+0000 7f73dc617640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73d4142870 0x7f73d40c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.512+0000 7f73dc617640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d40af800 con 0x7f73d40b1c50 2023-12-01T01:48:54.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.512+0000 7f73dc617640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d40afae0 con 0x7f73d40cd1e0 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.512+0000 7f73dc617640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d40afdc0 con 0x7f73d4142870 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73da3b3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40b1c50 0x7f73d40adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73da3b3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40b1c50 0x7f73d40adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42844/0 (socket says 172.21.15.115:42844) 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73da3b3640 1 -- 172.21.15.115:0/3596241697 learned_addr learned my addr 172.21.15.115:0/3596241697 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73d9bb2640 1 --2- 172.21.15.115:0/3596241697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73d40cd1e0 0x7f73d40c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73dabb4640 1 --2- 172.21.15.115:0/3596241697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73d4142870 0x7f73d40c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73da3b3640 1 -- 172.21.15.115:0/3596241697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73d4142870 msgr2=0x7f73d40c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73da3b3640 1 --2- 172.21.15.115:0/3596241697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73d4142870 0x7f73d40c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73da3b3640 1 -- 172.21.15.115:0/3596241697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73d40cd1e0 msgr2=0x7f73d40c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73da3b3640 1 --2- 172.21.15.115:0/3596241697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73d40cd1e0 0x7f73d40c4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73da3b3640 1 -- 172.21.15.115:0/3596241697 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c40094d0 con 0x7f73d40b1c50 2023-12-01T01:48:54.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.513+0000 7f73da3b3640 1 --2- 172.21.15.115:0/3596241697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40b1c50 0x7f73d40adb70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f73d000b3e0 tx=0x7f73d000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:54.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.514+0000 7f73caffd640 1 -- 172.21.15.115:0/3596241697 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0011830 con 0x7f73d40b1c50 2023-12-01T01:48:54.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.514+0000 7f73caffd640 1 -- 172.21.15.115:0/3596241697 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d00119b0 con 0x7f73d40b1c50 2023-12-01T01:48:54.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.514+0000 7f73dc617640 1 -- 172.21.15.115:0/3596241697 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d40b00a0 con 0x7f73d40b1c50 2023-12-01T01:48:54.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.514+0000 7f73caffd640 1 -- 172.21.15.115:0/3596241697 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0011d60 con 0x7f73d40b1c50 2023-12-01T01:48:54.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.514+0000 7f73dc617640 1 -- 172.21.15.115:0/3596241697 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d40b08a0 con 0x7f73d40b1c50 2023-12-01T01:48:54.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.515+0000 7f73dc617640 1 -- 172.21.15.115:0/3596241697 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d404e890 con 0x7f73d40b1c50 2023-12-01T01:48:54.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.516+0000 7f73caffd640 1 -- 172.21.15.115:0/3596241697 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 16) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f73d001a400 con 0x7f73d40b1c50 2023-12-01T01:48:54.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.517+0000 7f73caffd640 1 -- 172.21.15.115:0/3596241697 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f73d00889a0 con 0x7f73d40b1c50 2023-12-01T01:48:54.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.521+0000 7f73caffd640 1 -- 172.21.15.115:0/3596241697 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f73d0053f50 con 0x7f73d40b1c50 2023-12-01T01:48:54.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.705+0000 7f73dc617640 1 -- 172.21.15.115:0/3596241697 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73d40cdc80 con 0x7f73d40b1c50 2023-12-01T01:48:54.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.706+0000 7f73caffd640 1 -- 172.21.15.115:0/3596241697 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f73d00150a0 con 0x7f73d40b1c50 2023-12-01T01:48:54.706 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:48:54.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.708+0000 7f73dc617640 1 -- 172.21.15.115:0/3596241697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40b1c50 msgr2=0x7f73d40adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.708+0000 7f73dc617640 1 --2- 172.21.15.115:0/3596241697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40b1c50 0x7f73d40adb70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f73d000b3e0 tx=0x7f73d000b8a0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.708+0000 7f73dc617640 1 -- 172.21.15.115:0/3596241697 shutdown_connections 2023-12-01T01:48:54.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.708+0000 7f73dc617640 1 --2- 172.21.15.115:0/3596241697 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73d4142870 0x7f73d40c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.708+0000 7f73dc617640 1 --2- 172.21.15.115:0/3596241697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73d40cd1e0 0x7f73d40c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.708+0000 7f73dc617640 1 --2- 172.21.15.115:0/3596241697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73d40b1c50 0x7f73d40adb70 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.708+0000 7f73dc617640 1 -- 172.21.15.115:0/3596241697 >> 172.21.15.115:0/3596241697 conn(0x7f73d4065960 msgr2=0x7f73d413cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:54.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.708+0000 7f73dc617640 1 -- 172.21.15.115:0/3596241697 shutdown_connections 2023-12-01T01:48:54.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.709+0000 7f73dc617640 1 -- 172.21.15.115:0/3596241697 wait complete. 2023-12-01T01:48:54.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:48:54.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.941+0000 7f94b9a21640 1 Processor -- start 2023-12-01T01:48:54.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.942+0000 7f94b9a21640 1 -- start start 2023-12-01T01:48:54.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.943+0000 7f94b9a21640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f94b415dfd0 0x7f94b40cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.943+0000 7f94b9a21640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f94b40cf9a0 0x7f94b40b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.943+0000 7f94b9a21640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b4160b20 con 0x7f94b415dbf0 2023-12-01T01:48:54.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.943+0000 7f94b9a21640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b4160dc0 con 0x7f94b40cf9a0 2023-12-01T01:48:54.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.943+0000 7f94b9a21640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b41610a0 con 0x7f94b415dfd0 2023-12-01T01:48:54.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.944+0000 7f94b37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f94b40cf9a0 0x7f94b40b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.944+0000 7f94b37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f94b40cf9a0 0x7f94b40b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42856/0 (socket says 172.21.15.115:42856) 2023-12-01T01:48:54.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.944+0000 7f94b37fe640 1 -- 172.21.15.115:0/360848616 learned_addr learned my addr 172.21.15.115:0/360848616 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:54.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.944+0000 7f94b37fe640 1 -- 172.21.15.115:0/360848616 >> v2:172.21.15.142:3300/0 conn(0x7f94b415dfd0 msgr2=0x7f94b40cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:54.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.944+0000 7f94b27fc640 1 --2- 172.21.15.115:0/360848616 >> v2:172.21.15.142:3300/0 conn(0x7f94b415dfd0 0x7f94b40cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.944+0000 7f94b37fe640 1 --2- 172.21.15.115:0/360848616 >> v2:172.21.15.142:3300/0 conn(0x7f94b415dfd0 0x7f94b40cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.944+0000 7f94b37fe640 1 -- 172.21.15.115:0/360848616 >> v1:172.21.15.142:6789/0 conn(0x7f94b415dbf0 legacy=0x7f94b40cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:54.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.944+0000 7f94b37fe640 1 -- 172.21.15.115:0/360848616 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4161380 con 0x7f94b40cf9a0 2023-12-01T01:48:54.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.945+0000 7f94b37fe640 1 --2- 172.21.15.115:0/360848616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b40cf9a0 0x7f94b40b7ef0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f94b40b21c0 tx=0x7f94ac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba16a92d6321fbb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:54.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.945+0000 7f94b1ffb640 1 -- 172.21.15.115:0/360848616 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac02ca80 con 0x7f94b40cf9a0 2023-12-01T01:48:54.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.945+0000 7f94b1ffb640 1 -- 172.21.15.115:0/360848616 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac02cbe0 con 0x7f94b40cf9a0 2023-12-01T01:48:54.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.945+0000 7f94b1ffb640 1 -- 172.21.15.115:0/360848616 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0343f0 con 0x7f94b40cf9a0 2023-12-01T01:48:54.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.946+0000 7f94b9a21640 1 -- 172.21.15.115:0/360848616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b40cf9a0 msgr2=0x7f94b40b7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.946+0000 7f94b9a21640 1 --2- 172.21.15.115:0/360848616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b40cf9a0 0x7f94b40b7ef0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f94b40b21c0 tx=0x7f94ac02bee0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.946+0000 7f94b9a21640 1 -- 172.21.15.115:0/360848616 shutdown_connections 2023-12-01T01:48:54.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.946+0000 7f94b9a21640 1 --2- 172.21.15.115:0/360848616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b40cf9a0 0x7f94b40b7ef0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.946+0000 7f94b9a21640 1 --2- 172.21.15.115:0/360848616 >> v2:172.21.15.142:3300/0 conn(0x7f94b415dfd0 0x7f94b40cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.946+0000 7f94b9a21640 1 -- 172.21.15.115:0/360848616 >> 172.21.15.115:0/360848616 conn(0x7f94b4065960 msgr2=0x7f94b4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:54.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.946+0000 7f94b9a21640 1 -- 172.21.15.115:0/360848616 shutdown_connections 2023-12-01T01:48:54.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.946+0000 7f94b9a21640 1 -- 172.21.15.115:0/360848616 wait complete. 2023-12-01T01:48:54.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.948+0000 7f94b9a21640 1 Processor -- start 2023-12-01T01:48:54.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.948+0000 7f94b9a21640 1 -- start start 2023-12-01T01:48:54.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.949+0000 7f94b9a21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94b40cf9a0 0x7f94b4152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.949+0000 7f94b9a21640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94b415dbf0 0x7f94b4153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.949+0000 7f94b9a21640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b415dfd0 0x7f94b4154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.949+0000 7f94b9a21640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b4152540 con 0x7f94b415dfd0 2023-12-01T01:48:54.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.949+0000 7f94b9a21640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b40cc960 con 0x7f94b415dbf0 2023-12-01T01:48:54.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.949+0000 7f94b9a21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b40ccc40 con 0x7f94b40cf9a0 2023-12-01T01:48:54.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.949+0000 7f94b37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b415dfd0 0x7f94b4154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.949+0000 7f94b37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b415dfd0 0x7f94b4154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42858/0 (socket says 172.21.15.115:42858) 2023-12-01T01:48:54.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.949+0000 7f94b37fe640 1 -- 172.21.15.115:0/2529386376 learned_addr learned my addr 172.21.15.115:0/2529386376 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:54.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f94b27fc640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94b415dbf0 0x7f94b4153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f94b2ffd640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94b40cf9a0 0x7f94b4152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f94b37fe640 1 -- 172.21.15.115:0/2529386376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94b40cf9a0 msgr2=0x7f94b4152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f94b37fe640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94b40cf9a0 0x7f94b4152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f94b37fe640 1 -- 172.21.15.115:0/2529386376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94b415dbf0 msgr2=0x7f94b4153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:54.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f94b37fe640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94b415dbf0 0x7f94b4153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:54.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f94b37fe640 1 -- 172.21.15.115:0/2529386376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac009b30 con 0x7f94b415dfd0 2023-12-01T01:48:54.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f94b2ffd640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94b40cf9a0 0x7f94b4152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:54.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f94b37fe640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b415dfd0 0x7f94b4154240 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f94ac036040 tx=0x7f94ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:54.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f949bfff640 1 -- 172.21.15.115:0/2529386376 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac03e450 con 0x7f94b415dfd0 2023-12-01T01:48:54.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f949bfff640 1 -- 172.21.15.115:0/2529386376 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac03e5d0 con 0x7f94b415dfd0 2023-12-01T01:48:54.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f94b9a21640 1 -- 172.21.15.115:0/2529386376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b40ccf20 con 0x7f94b415dfd0 2023-12-01T01:48:54.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.950+0000 7f949bfff640 1 -- 172.21.15.115:0/2529386376 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac03e980 con 0x7f94b415dfd0 2023-12-01T01:48:54.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.951+0000 7f94b9a21640 1 -- 172.21.15.115:0/2529386376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b40cd4e0 con 0x7f94b415dfd0 2023-12-01T01:48:54.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.952+0000 7f94b9a21640 1 -- 172.21.15.115:0/2529386376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b40cd700 con 0x7f94b415dfd0 2023-12-01T01:48:54.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.952+0000 7f949bfff640 1 -- 172.21.15.115:0/2529386376 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f94ac02cd50 con 0x7f94b415dfd0 2023-12-01T01:48:54.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.953+0000 7f949bfff640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f94a006a280 0x7f94a006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:54.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.953+0000 7f949bfff640 1 -- 172.21.15.115:0/2529386376 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f94ac0ab670 con 0x7f94b415dfd0 2023-12-01T01:48:54.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.954+0000 7f94b2ffd640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f94a006a280 0x7f94a006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:54.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.954+0000 7f94b2ffd640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f94a006a280 0x7f94a006c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f94a4007fd0 tx=0x7f94a4009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:54.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:54.958+0000 7f949bfff640 1 -- 172.21.15.115:0/2529386376 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f94ac076af0 con 0x7f94b415dfd0 2023-12-01T01:48:55.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.190+0000 7f94b9a21640 1 -- 172.21.15.115:0/2529386376 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94b404e890 con 0x7f94b415dfd0 2023-12-01T01:48:55.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.192+0000 7f949bfff640 1 -- 172.21.15.115:0/2529386376 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f94ac07a110 con 0x7f94b415dfd0 2023-12-01T01:48:55.192 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:55.192 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1226717603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1226717603},{"type":"v1","addr":"172.21.15.142:6834","nonce":1226717603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2864659456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2864659456},{"type":"v1","addr":"172.21.15.142:6835","nonce":2864659456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4774,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/253656345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":253656345},{"type":"v1","addr":"172.21.15.115:6837","nonce":253656345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:37:23.645235+0000","modified":"2023-12-01T01:37:26.663085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5091},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5091":{"gid":5091,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6835/4242588071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4242588071},{"type":"v1","addr":"172.21.15.115:6835","nonce":4242588071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-01T01:48:55.193 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2023-12-01T01:48:55.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.194+0000 7f94b9a21640 1 -- 172.21.15.115:0/2529386376 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f94a006a280 msgr2=0x7f94a006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.194+0000 7f94b9a21640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f94a006a280 0x7f94a006c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f94a4007fd0 tx=0x7f94a4009040 comp rx=0 tx=0).stop 2023-12-01T01:48:55.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.194+0000 7f94b9a21640 1 -- 172.21.15.115:0/2529386376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b415dfd0 msgr2=0x7f94b4154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.194+0000 7f94b9a21640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b415dfd0 0x7f94b4154240 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f94ac036040 tx=0x7f94ac004a40 comp rx=0 tx=0).stop 2023-12-01T01:48:55.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.194+0000 7f94b9a21640 1 -- 172.21.15.115:0/2529386376 shutdown_connections 2023-12-01T01:48:55.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.194+0000 7f94b9a21640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f94a006a280 0x7f94a006c740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.194+0000 7f94b9a21640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94b415dfd0 0x7f94b4154240 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.194+0000 7f94b9a21640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94b415dbf0 0x7f94b4153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.194+0000 7f94b9a21640 1 --2- 172.21.15.115:0/2529386376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94b40cf9a0 0x7f94b4152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.194+0000 7f94b9a21640 1 -- 172.21.15.115:0/2529386376 >> 172.21.15.115:0/2529386376 conn(0x7f94b4065960 msgr2=0x7f94b40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:55.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.195+0000 7f94b9a21640 1 -- 172.21.15.115:0/2529386376 shutdown_connections 2023-12-01T01:48:55.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.195+0000 7f94b9a21640 1 -- 172.21.15.115:0/2529386376 wait complete. 2023-12-01T01:48:55.214 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:48:55.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.407+0000 7f5a86ee5640 1 Processor -- start 2023-12-01T01:48:55.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.408+0000 7f5a86ee5640 1 -- start start 2023-12-01T01:48:55.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.409+0000 7f5a86ee5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015e2f0 0x7f5a800ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.409+0000 7f5a86ee5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a800add10 con 0x7f5a800adf10 2023-12-01T01:48:55.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.409+0000 7f5a86ee5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a800c5760 con 0x7f5a8006f2f0 2023-12-01T01:48:55.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.409+0000 7f5a86ee5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a800c9cf0 con 0x7f5a8015e2f0 2023-12-01T01:48:55.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.409+0000 7f5a85482640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5a800adf10 0x7f5a800c95e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59690/0 (socket says 172.21.15.115:59690) 2023-12-01T01:48:55.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.409+0000 7f5a85482640 1 -- 172.21.15.115:0/2430860177 learned_addr learned my addr 172.21.15.115:0/2430860177 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:55.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059666727 0 0) 0x7f5a800add10 con 0x7f5a800adf10 2023-12-01T01:48:55.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a68003520 con 0x7f5a800adf10 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a84c81640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015e2f0 0x7f5a800ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 422037144 0 0) 0x7f5a68003520 con 0x7f5a800adf10 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015e2f0 msgr2=0x7f5a800ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015e2f0 0x7f5a800ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 >> v1:172.21.15.142:6789/0 conn(0x7f5a8006f2f0 legacy=0x7f5a800c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8015e6d0 con 0x7f5a800adf10 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5a780019f0 con 0x7f5a800adf10 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a84c81640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015e2f0 0x7f5a800ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 >> v1:172.21.15.115:6789/0 conn(0x7f5a800adf10 legacy=0x7f5a800c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 0x7f5a68003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015e2f0 0x7f5a680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a68004d70 0x7f5a680051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a800c9cf0 con 0x7f5a68003a10 2023-12-01T01:48:55.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6800bc20 con 0x7f5a68004d70 2023-12-01T01:48:55.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.410+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6800bde0 con 0x7f5a8015e2f0 2023-12-01T01:48:55.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.411+0000 7f5a84c81640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 0x7f5a68003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.411+0000 7f5a84c81640 1 -- 172.21.15.115:0/2430860177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015e2f0 msgr2=0x7f5a680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:55.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.411+0000 7f5a84c81640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015e2f0 0x7f5a680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.411+0000 7f5a84c81640 1 -- 172.21.15.115:0/2430860177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a68004d70 msgr2=0x7f5a680051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:55.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.411+0000 7f5a84c81640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a68004d70 0x7f5a680051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.411+0000 7f5a84c81640 1 -- 172.21.15.115:0/2430860177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a68003520 con 0x7f5a68003a10 2023-12-01T01:48:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.412+0000 7f5a84c81640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 0x7f5a68003e30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5a740098a0 tx=0x7f5a680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bff15c59fe0fb9b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.412+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74004230 con 0x7f5a68003a10 2023-12-01T01:48:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.412+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a74004390 con 0x7f5a68003a10 2023-12-01T01:48:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.412+0000 7f5a7f7fe640 1 -- 172.21.15.115:0/2430860177 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74003830 con 0x7f5a68003a10 2023-12-01T01:48:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.413+0000 7f5a86ee5640 1 -- 172.21.15.115:0/2430860177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 msgr2=0x7f5a68003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.413+0000 7f5a86ee5640 1 --2- 172.21.15.115:0/2430860177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 0x7f5a68003e30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5a740098a0 tx=0x7f5a680057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.413+0000 7f5a85482640 1 -- 172.21.15.115:0/2430860177 reap_dead start 2023-12-01T01:48:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.413+0000 7f5a86ee5640 1 -- 172.21.15.115:0/2430860177 shutdown_connections 2023-12-01T01:48:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.413+0000 7f5a86ee5640 1 -- 172.21.15.115:0/2430860177 >> 172.21.15.115:0/2430860177 conn(0x7f5a800ce880 msgr2=0x7f5a800cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.413+0000 7f5a86ee5640 1 -- 172.21.15.115:0/2430860177 shutdown_connections 2023-12-01T01:48:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.413+0000 7f5a86ee5640 1 -- 172.21.15.115:0/2430860177 wait complete. 2023-12-01T01:48:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.415+0000 7f5a86ee5640 1 Processor -- start 2023-12-01T01:48:55.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.415+0000 7f5a86ee5640 1 -- start start 2023-12-01T01:48:55.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.415+0000 7f5a86ee5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 0x7f5a80154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.415+0000 7f5a86ee5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a8015c140 0x7f5a8015c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a86ee5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015a080 0x7f5a8015a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a86ee5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8015d2c0 con 0x7f5a68003a10 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a86ee5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a800c2930 con 0x7f5a8015c140 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a86ee5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8015cb00 con 0x7f5a8015a080 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a84c81640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 0x7f5a80154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a84c81640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 0x7f5a80154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42876/0 (socket says 172.21.15.115:42876) 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a84c81640 1 -- 172.21.15.115:0/1545175836 learned_addr learned my addr 172.21.15.115:0/1545175836 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a84c81640 1 -- 172.21.15.115:0/1545175836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015a080 msgr2=0x7f5a8015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a84c81640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015a080 0x7f5a8015a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a84c81640 1 -- 172.21.15.115:0/1545175836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a8015c140 msgr2=0x7f5a8015c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a84c81640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a8015c140 0x7f5a8015c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a84c81640 1 -- 172.21.15.115:0/1545175836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8015cd00 con 0x7f5a68003a10 2023-12-01T01:48:55.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.416+0000 7f5a84c81640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 0x7f5a80154090 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5a740099c0 tx=0x7f5a74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:55.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.417+0000 7f5a7d7fa640 1 -- 172.21.15.115:0/1545175836 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74030660 con 0x7f5a68003a10 2023-12-01T01:48:55.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.417+0000 7f5a7d7fa640 1 -- 172.21.15.115:0/1545175836 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a740307c0 con 0x7f5a68003a10 2023-12-01T01:48:55.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.417+0000 7f5a7d7fa640 1 -- 172.21.15.115:0/1545175836 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74030b50 con 0x7f5a68003a10 2023-12-01T01:48:55.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.417+0000 7f5a86ee5640 1 -- 172.21.15.115:0/1545175836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8015cfe0 con 0x7f5a68003a10 2023-12-01T01:48:55.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.417+0000 7f5a86ee5640 1 -- 172.21.15.115:0/1545175836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a800cd870 con 0x7f5a68003a10 2023-12-01T01:48:55.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.418+0000 7f5a86ee5640 1 -- 172.21.15.115:0/1545175836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8006fda0 con 0x7f5a68003a10 2023-12-01T01:48:55.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.419+0000 7f5a7d7fa640 1 -- 172.21.15.115:0/1545175836 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f5a74030cb0 con 0x7f5a68003a10 2023-12-01T01:48:55.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.420+0000 7f5a7d7fa640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a6c06a280 0x7f5a6c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.420+0000 7f5a7d7fa640 1 -- 172.21.15.115:0/1545175836 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5a740aa850 con 0x7f5a68003a10 2023-12-01T01:48:55.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.420+0000 7f5a7ffff640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a6c06a280 0x7f5a6c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.420+0000 7f5a7ffff640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a6c06a280 0x7f5a6c06c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5a800cd1e0 tx=0x7f5a70007990 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:55.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.424+0000 7f5a7d7fa640 1 -- 172.21.15.115:0/1545175836 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5a74075cf0 con 0x7f5a68003a10 2023-12-01T01:48:55.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.603+0000 7f5a86ee5640 1 -- 172.21.15.115:0/1545175836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a800c23b0 con 0x7f5a68003a10 2023-12-01T01:48:55.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.605+0000 7f5a7d7fa640 1 -- 172.21.15.115:0/1545175836 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+17917 (secure 0 0 0) 0x7f5a800c23b0 con 0x7f5a68003a10 2023-12-01T01:48:55.606 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:55.607 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":52,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:48:53.861318+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:37:22.073677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-01T01:37:22.660650+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.23","mappings":[{"from":4,"to":5}]},{"pgid":"5.33","mappings":[{"from":4,"to":5}]},{"pgid":"5.3c","mappings":[{"from":4,"to":5}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/1312252546":"2023-12-02T01:48:53.861286+0000","172.21.15.115:0/1127148126":"2023-12-02T01:48:53.861286+0000","172.21.15.115:0/892358745":"2023-12-02T01:48:53.861286+0000","172.21.15.115:6824/88317":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/3103964125":"2023-12-02T01:48:53.861286+0000","172.21.15.115:6822/88317":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/4134228660":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/3690374531":"2023-12-02T01:48:53.861286+0000","172.21.15.115:0/1343452792":"2023-12-02T01:48:53.861286+0000","172.21.15.115:0/297799080":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/2772845026":"2023-12-02T01:48:53.861286+0000","172.21.15.115:0/1314334167":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/2515539287":"2023-12-02T01:37:37.403693+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1},{"begin":6,"length":1},{"begin":8,"length":1},{"begin":10,"length":1},{"begin":12,"length":1},{"begin":14,"length":1},{"begin":16,"length":1},{"begin":18,"length":2},{"begin":22,"length":2},{"begin":26,"length":3},{"begin":33,"length":5},{"begin":42,"length":1},{"begin":44,"length":3},{"begin":48,"length":1},{"begin":52,"length":3},{"begin":57,"length":2},{"begin":63,"length":1},{"begin":65,"length":4},{"begin":73,"length":1},{"begin":75,"length":4},{"begin":83,"length":4},{"begin":89,"length":1},{"begin":94,"length":1},{"begin":96,"length":2},{"begin":100,"length":3},{"begin":105,"length":3},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":166,"length":56}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:48:55.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 -- 172.21.15.115:0/1545175836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a6c06a280 msgr2=0x7f5a6c06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a6c06a280 0x7f5a6c06c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5a800cd1e0 tx=0x7f5a70007990 comp rx=0 tx=0).stop 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 -- 172.21.15.115:0/1545175836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 msgr2=0x7f5a80154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 0x7f5a80154090 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5a740099c0 tx=0x7f5a74004730 comp rx=0 tx=0).stop 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 -- 172.21.15.115:0/1545175836 shutdown_connections 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a6c06a280 0x7f5a6c06c740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a8015a080 0x7f5a8015a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a8015c140 0x7f5a8015c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 --2- 172.21.15.115:0/1545175836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a68003a10 0x7f5a80154090 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 -- 172.21.15.115:0/1545175836 >> 172.21.15.115:0/1545175836 conn(0x7f5a800ce880 msgr2=0x7f5a80158770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 -- 172.21.15.115:0/1545175836 shutdown_connections 2023-12-01T01:48:55.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.608+0000 7f5a86ee5640 1 -- 172.21.15.115:0/1545175836 wait complete. 2023-12-01T01:48:55.626 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T01:48:55.627 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:48:55.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.833+0000 7f0ba4f27640 1 Processor -- start 2023-12-01T01:48:55.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.833+0000 7f0ba4f27640 1 -- start start 2023-12-01T01:48:55.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.834+0000 7f0ba4f27640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0ba015d220 0x7f0ba00cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.834+0000 7f0ba4f27640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba00ad2b0 0x7f0ba00ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.834+0000 7f0ba4f27640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba015d600 con 0x7f0ba00ade80 2023-12-01T01:48:55.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.834+0000 7f0ba4f27640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba006cc10 con 0x7f0ba015d220 2023-12-01T01:48:55.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.834+0000 7f0ba4f27640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba006c720 con 0x7f0ba00ad2b0 2023-12-01T01:48:55.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9ed9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0ba00ade80 0x7f0ba006b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59704/0 (socket says 172.21.15.115:59704) 2023-12-01T01:48:55.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9ed9d640 1 -- 172.21.15.115:0/2556144327 learned_addr learned my addr 172.21.15.115:0/2556144327 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:55.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b97fff640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba00ad2b0 0x7f0ba00ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9e59c640 1 --2- 172.21.15.115:0/2556144327 >> v2:172.21.15.142:3300/0 conn(0x7f0ba015d220 0x7f0ba00cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468903426 0 0) 0x7f0b98001490 con 0x7f0ba00ade80 2023-12-01T01:48:55.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b8c003520 con 0x7f0ba00ade80 2023-12-01T01:48:55.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 978344051 0 0) 0x7f0b8c003520 con 0x7f0ba00ade80 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba00ad2b0 msgr2=0x7f0ba00ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9dd9b640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba00ad2b0 0x7f0ba00ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 >> v2:172.21.15.142:3300/0 conn(0x7f0ba015d220 msgr2=0x7f0ba00cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9dd9b640 1 --2- 172.21.15.115:0/2556144327 >> v2:172.21.15.142:3300/0 conn(0x7f0ba015d220 0x7f0ba00cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.835+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba006c4b0 con 0x7f0ba00ade80 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0b98001b80 con 0x7f0ba00ade80 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9e59c640 1 --2- 172.21.15.115:0/2556144327 >> v2:172.21.15.142:3300/0 conn(0x7f0ba015d220 0x7f0ba00cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 >> v1:172.21.15.115:6789/0 conn(0x7f0ba00ade80 legacy=0x7f0ba006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9dd9b640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b8c0039e0 0x7f0b8c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9dd9b640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba00ad2b0 0x7f0b8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9dd9b640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c004d00 0x7f0b8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba006cc10 con 0x7f0b8c004d00 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba006c720 con 0x7f0b8c0039e0 2023-12-01T01:48:55.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c00bab0 con 0x7f0ba00ad2b0 2023-12-01T01:48:55.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9ed9d640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c004d00 0x7f0b8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9e59c640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b8c0039e0 0x7f0b8c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b97fff640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba00ad2b0 0x7f0b8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9ed9d640 1 -- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba00ad2b0 msgr2=0x7f0b8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:55.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9ed9d640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba00ad2b0 0x7f0b8c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9ed9d640 1 -- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b8c0039e0 msgr2=0x7f0b8c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:55.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9ed9d640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b8c0039e0 0x7f0b8c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.836+0000 7f0b9ed9d640 1 -- 172.21.15.115:0/2556144327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c003520 con 0x7f0b8c004d00 2023-12-01T01:48:55.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.837+0000 7f0b9ed9d640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c004d00 0x7f0b8c005180 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800a830 tx=0x7f0b8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b0423a1ed87702 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:55.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.838+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98034630 con 0x7f0b8c004d00 2023-12-01T01:48:55.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.838+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9802f070 con 0x7f0b8c004d00 2023-12-01T01:48:55.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.838+0000 7f0b9dd9b640 1 -- 172.21.15.115:0/2556144327 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b980349d0 con 0x7f0b8c004d00 2023-12-01T01:48:55.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.838+0000 7f0ba4f27640 1 -- 172.21.15.115:0/2556144327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c004d00 msgr2=0x7f0b8c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.838+0000 7f0ba4f27640 1 --2- 172.21.15.115:0/2556144327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c004d00 0x7f0b8c005180 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800a830 tx=0x7f0b8c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.838+0000 7f0b9ed9d640 1 -- 172.21.15.115:0/2556144327 reap_dead start 2023-12-01T01:48:55.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.838+0000 7f0ba4f27640 1 -- 172.21.15.115:0/2556144327 shutdown_connections 2023-12-01T01:48:55.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.838+0000 7f0ba4f27640 1 -- 172.21.15.115:0/2556144327 >> 172.21.15.115:0/2556144327 conn(0x7f0ba00ce880 msgr2=0x7f0ba00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:55.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.838+0000 7f0ba4f27640 1 -- 172.21.15.115:0/2556144327 shutdown_connections 2023-12-01T01:48:55.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.838+0000 7f0ba4f27640 1 -- 172.21.15.115:0/2556144327 wait complete. 2023-12-01T01:48:55.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.840+0000 7f0ba4f27640 1 Processor -- start 2023-12-01T01:48:55.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.840+0000 7f0ba4f27640 1 -- start start 2023-12-01T01:48:55.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.841+0000 7f0ba4f27640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b8c004d00 0x7f0ba0159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.841+0000 7f0ba4f27640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba0158e30 0x7f0ba00c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.841+0000 7f0ba4f27640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ba00c49f0 0x7f0ba00c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.841+0000 7f0ba4f27640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00c4e70 con 0x7f0ba00c49f0 2023-12-01T01:48:55.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.841+0000 7f0ba4f27640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00c38c0 con 0x7f0ba0158e30 2023-12-01T01:48:55.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.841+0000 7f0ba4f27640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00c7c70 con 0x7f0b8c004d00 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.841+0000 7f0b9ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ba00c49f0 0x7f0ba00c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.841+0000 7f0b9ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ba00c49f0 0x7f0ba00c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42900/0 (socket says 172.21.15.115:42900) 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.841+0000 7f0b9ed9d640 1 -- 172.21.15.115:0/1626547251 learned_addr learned my addr 172.21.15.115:0/1626547251 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b9e59c640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b8c004d00 0x7f0ba0159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b9ed9d640 1 -- 172.21.15.115:0/1626547251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b8c004d00 msgr2=0x7f0ba0159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b9ed9d640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b8c004d00 0x7f0ba0159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b9ed9d640 1 -- 172.21.15.115:0/1626547251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba0158e30 msgr2=0x7f0ba00c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b9ed9d640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba0158e30 0x7f0ba00c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b9ed9d640 1 -- 172.21.15.115:0/1626547251 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba00c7f50 con 0x7f0ba00c49f0 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b9e59c640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b8c004d00 0x7f0ba0159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b9ed9d640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ba00c49f0 0x7f0ba00c23b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0b98001db0 tx=0x7f0b98001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b96ffd640 1 -- 172.21.15.115:0/1626547251 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98042070 con 0x7f0ba00c49f0 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b96ffd640 1 -- 172.21.15.115:0/1626547251 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b98001490 con 0x7f0ba00c49f0 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0ba4f27640 1 -- 172.21.15.115:0/1626547251 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba00c8230 con 0x7f0ba00c49f0 2023-12-01T01:48:55.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.842+0000 7f0b96ffd640 1 -- 172.21.15.115:0/1626547251 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98034670 con 0x7f0ba00c49f0 2023-12-01T01:48:55.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.843+0000 7f0ba4f27640 1 -- 172.21.15.115:0/1626547251 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba01592b0 con 0x7f0ba00c49f0 2023-12-01T01:48:55.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.843+0000 7f0ba4f27640 1 -- 172.21.15.115:0/1626547251 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba00c8750 con 0x7f0ba00c49f0 2023-12-01T01:48:55.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.844+0000 7f0b96ffd640 1 -- 172.21.15.115:0/1626547251 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f0b98031030 con 0x7f0ba00c49f0 2023-12-01T01:48:55.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.845+0000 7f0b96ffd640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8006a2b0 0x7f0b8006c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:55.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.845+0000 7f0b96ffd640 1 -- 172.21.15.115:0/1626547251 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0b980a9910 con 0x7f0ba00c49f0 2023-12-01T01:48:55.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.846+0000 7f0b9e59c640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8006a2b0 0x7f0b8006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:55.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.846+0000 7f0b9e59c640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8006a2b0 0x7f0b8006c770 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0159da0 tx=0x7f0b90008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:55.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:55.850+0000 7f0b96ffd640 1 -- 172.21.15.115:0/1626547251 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0ba00c8750 con 0x7f0ba00c49f0 2023-12-01T01:48:56.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.087+0000 7f0ba4f27640 1 -- 172.21.15.115:0/1626547251 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ba015d600 con 0x7f0ba00c49f0 2023-12-01T01:48:56.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.088+0000 7f0b96ffd640 1 -- 172.21.15.115:0/1626547251 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f0ba015d600 con 0x7f0ba00c49f0 2023-12-01T01:48:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:56.088 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1226717603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1226717603},{"type":"v1","addr":"172.21.15.142:6834","nonce":1226717603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2864659456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2864659456},{"type":"v1","addr":"172.21.15.142:6835","nonce":2864659456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4774,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/253656345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":253656345},{"type":"v1","addr":"172.21.15.115:6837","nonce":253656345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:37:23.645235+0000","modified":"2023-12-01T01:37:26.663085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5091},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5091":{"gid":5091,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6835/4242588071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4242588071},{"type":"v1","addr":"172.21.15.115:6835","nonce":4242588071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-01T01:48:56.089 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2023-12-01T01:48:56.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 -- 172.21.15.115:0/1626547251 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8006a2b0 msgr2=0x7f0b8006c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8006a2b0 0x7f0b8006c770 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0159da0 tx=0x7f0b90008040 comp rx=0 tx=0).stop 2023-12-01T01:48:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 -- 172.21.15.115:0/1626547251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ba00c49f0 msgr2=0x7f0ba00c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ba00c49f0 0x7f0ba00c23b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0b98001db0 tx=0x7f0b98001360 comp rx=0 tx=0).stop 2023-12-01T01:48:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 -- 172.21.15.115:0/1626547251 shutdown_connections 2023-12-01T01:48:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8006a2b0 0x7f0b8006c770 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ba00c49f0 0x7f0ba00c23b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba0158e30 0x7f0ba00c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 --2- 172.21.15.115:0/1626547251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b8c004d00 0x7f0ba0159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 -- 172.21.15.115:0/1626547251 >> 172.21.15.115:0/1626547251 conn(0x7f0ba00ce880 msgr2=0x7f0ba015ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 -- 172.21.15.115:0/1626547251 shutdown_connections 2023-12-01T01:48:56.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.091+0000 7f0ba4f27640 1 -- 172.21.15.115:0/1626547251 wait complete. 2023-12-01T01:48:56.109 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:48:56.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.310+0000 7fd997d92640 1 Processor -- start 2023-12-01T01:48:56.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.311+0000 7fd997d92640 1 -- start start 2023-12-01T01:48:56.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.312+0000 7fd997d92640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd9900c4170 0x7fd99016bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.312+0000 7fd997d92640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9901618f0 con 0x7fd9900d17e0 2023-12-01T01:48:56.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.312+0000 7fd997d92640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9900b0d50 con 0x7fd990161510 2023-12-01T01:48:56.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.312+0000 7fd997d92640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9900b0a20 con 0x7fd9900c4170 2023-12-01T01:48:56.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.312+0000 7fd99532d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd9900d17e0 0x7fd9900c3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59716/0 (socket says 172.21.15.115:59716) 2023-12-01T01:48:56.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.313+0000 7fd99532d640 1 -- 172.21.15.115:0/2759477647 learned_addr learned my addr 172.21.15.115:0/2759477647 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:56.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.313+0000 7fd99632f640 1 --2- 172.21.15.115:0/2759477647 >> v2:172.21.15.115:3300/0 conn(0x7fd9900c4170 0x7fd99016bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.313+0000 7fd994b2c640 1 -- 172.21.15.115:0/2759477647 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321005671 0 0) 0x7fd9901618f0 con 0x7fd9900d17e0 2023-12-01T01:48:56.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.313+0000 7fd994b2c640 1 -- 172.21.15.115:0/2759477647 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd97c003520 con 0x7fd9900d17e0 2023-12-01T01:48:56.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.313+0000 7fd99632f640 1 -- 172.21.15.115:0/2759477647 >> v1:172.21.15.142:6789/0 conn(0x7fd990161510 legacy=0x7fd9900b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.313+0000 7fd99632f640 1 -- 172.21.15.115:0/2759477647 >> v1:172.21.15.115:6789/0 conn(0x7fd9900d17e0 legacy=0x7fd9900c3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.313+0000 7fd99632f640 1 -- 172.21.15.115:0/2759477647 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd990069e60 con 0x7fd9900c4170 2023-12-01T01:48:56.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.315+0000 7fd99632f640 1 --2- 172.21.15.115:0/2759477647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 0x7fd99016bfc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd988009d90 tx=0x7fd98802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c174ed6341499e8b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:56.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.315+0000 7fd994b2c640 1 -- 172.21.15.115:0/2759477647 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98802cd00 con 0x7fd9900c4170 2023-12-01T01:48:56.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.315+0000 7fd994b2c640 1 -- 172.21.15.115:0/2759477647 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98802ce60 con 0x7fd9900c4170 2023-12-01T01:48:56.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.315+0000 7fd994b2c640 1 -- 172.21.15.115:0/2759477647 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988034660 con 0x7fd9900c4170 2023-12-01T01:48:56.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.316+0000 7fd997d92640 1 -- 172.21.15.115:0/2759477647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 msgr2=0x7fd99016bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.316+0000 7fd997d92640 1 --2- 172.21.15.115:0/2759477647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 0x7fd99016bfc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd988009d90 tx=0x7fd98802bf50 comp rx=0 tx=0).stop 2023-12-01T01:48:56.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.316+0000 7fd997d92640 1 -- 172.21.15.115:0/2759477647 shutdown_connections 2023-12-01T01:48:56.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.316+0000 7fd997d92640 1 --2- 172.21.15.115:0/2759477647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 0x7fd99016bfc0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.316+0000 7fd997d92640 1 -- 172.21.15.115:0/2759477647 >> 172.21.15.115:0/2759477647 conn(0x7fd9900d0a90 msgr2=0x7fd9900d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:56.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.316+0000 7fd997d92640 1 -- 172.21.15.115:0/2759477647 shutdown_connections 2023-12-01T01:48:56.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.316+0000 7fd997d92640 1 -- 172.21.15.115:0/2759477647 wait complete. 2023-12-01T01:48:56.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.318+0000 7fd997d92640 1 Processor -- start 2023-12-01T01:48:56.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.318+0000 7fd997d92640 1 -- start start 2023-12-01T01:48:56.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd997d92640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 0x7fd99015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd997d92640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9900d17e0 0x7fd9900ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd997d92640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd990161510 0x7fd9900c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd997d92640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900c57d0 con 0x7fd9900c4170 2023-12-01T01:48:56.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd997d92640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900c6610 con 0x7fd9900d17e0 2023-12-01T01:48:56.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd997d92640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99006e070 con 0x7fd990161510 2023-12-01T01:48:56.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd995b2e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 0x7fd99015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd995b2e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 0x7fd99015d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42948/0 (socket says 172.21.15.115:42948) 2023-12-01T01:48:56.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd995b2e640 1 -- 172.21.15.115:0/1087882123 learned_addr learned my addr 172.21.15.115:0/1087882123 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:56.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd99532d640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9900d17e0 0x7fd9900ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.320+0000 7fd99632f640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd990161510 0x7fd9900c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd995b2e640 1 -- 172.21.15.115:0/1087882123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd990161510 msgr2=0x7fd9900c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd995b2e640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd990161510 0x7fd9900c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd995b2e640 1 -- 172.21.15.115:0/1087882123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9900d17e0 msgr2=0x7fd9900ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd995b2e640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9900d17e0 0x7fd9900ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd995b2e640 1 -- 172.21.15.115:0/1087882123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988009a70 con 0x7fd9900c4170 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd995b2e640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 0x7fd99015d640 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00c430 tx=0x7fd98c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd9827fc640 1 -- 172.21.15.115:0/1087882123 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0052d0 con 0x7fd9900c4170 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd9827fc640 1 -- 172.21.15.115:0/1087882123 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c005430 con 0x7fd9900c4170 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd9827fc640 1 -- 172.21.15.115:0/1087882123 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c003830 con 0x7fd9900c4170 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd997d92640 1 -- 172.21.15.115:0/1087882123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9900c5a00 con 0x7fd9900c4170 2023-12-01T01:48:56.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.321+0000 7fd997d92640 1 -- 172.21.15.115:0/1087882123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9900c5fc0 con 0x7fd9900c4170 2023-12-01T01:48:56.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.323+0000 7fd997d92640 1 -- 172.21.15.115:0/1087882123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99004e890 con 0x7fd9900c4170 2023-12-01T01:48:56.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.323+0000 7fd9827fc640 1 -- 172.21.15.115:0/1087882123 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 17) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fd98c003990 con 0x7fd9900c4170 2023-12-01T01:48:56.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.324+0000 7fd9827fc640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd97406a2b0 0x7fd97406c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.324+0000 7fd9827fc640 1 -- 172.21.15.115:0/1087882123 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd98c088040 con 0x7fd9900c4170 2023-12-01T01:48:56.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.324+0000 7fd99532d640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd97406a2b0 0x7fd97406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.325+0000 7fd99532d640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd97406a2b0 0x7fd97406c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd9900caab0 tx=0x7fd984009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:56.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.329+0000 7fd9827fc640 1 -- 172.21.15.115:0/1087882123 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd98c0bac80 con 0x7fd9900c4170 2023-12-01T01:48:56.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.475+0000 7fd9827fc640 1 -- 172.21.15.115:0/1087882123 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 18) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fd98c0537e0 con 0x7fd9900c4170 2023-12-01T01:48:56.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.562+0000 7fd997d92640 1 -- 172.21.15.115:0/1087882123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9900cc470 con 0x7fd9900c4170 2023-12-01T01:48:56.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.564+0000 7fd9827fc640 1 -- 172.21.15.115:0/1087882123 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fd98c072400 con 0x7fd9900c4170 2023-12-01T01:48:56.564 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:56.565 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1226717603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1226717603},{"type":"v1","addr":"172.21.15.142:6834","nonce":1226717603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2864659456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2864659456},{"type":"v1","addr":"172.21.15.142:6835","nonce":2864659456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4774,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/253656345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":253656345},{"type":"v1","addr":"172.21.15.115:6837","nonce":253656345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:37:23.645235+0000","modified":"2023-12-01T01:37:26.663085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5091},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5091":{"gid":5091,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.115:6835/4242588071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4242588071},{"type":"v1","addr":"172.21.15.115:6835","nonce":4242588071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-01T01:48:56.565 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 16 2023-12-01T01:48:56.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 -- 172.21.15.115:0/1087882123 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd97406a2b0 msgr2=0x7fd97406c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd97406a2b0 0x7fd97406c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd9900caab0 tx=0x7fd984009040 comp rx=0 tx=0).stop 2023-12-01T01:48:56.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 -- 172.21.15.115:0/1087882123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 msgr2=0x7fd99015d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 0x7fd99015d640 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00c430 tx=0x7fd98c00c8f0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 -- 172.21.15.115:0/1087882123 shutdown_connections 2023-12-01T01:48:56.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd97406a2b0 0x7fd97406c770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd990161510 0x7fd9900c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9900d17e0 0x7fd9900ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 --2- 172.21.15.115:0/1087882123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9900c4170 0x7fd99015d640 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 -- 172.21.15.115:0/1087882123 >> 172.21.15.115:0/1087882123 conn(0x7fd9900d0a90 msgr2=0x7fd9900cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:56.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 -- 172.21.15.115:0/1087882123 shutdown_connections 2023-12-01T01:48:56.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.567+0000 7fd997d92640 1 -- 172.21.15.115:0/1087882123 wait complete. 2023-12-01T01:48:56.585 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:48:56.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.786+0000 7fc3074d5640 1 Processor -- start 2023-12-01T01:48:56.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.786+0000 7fc3074d5640 1 -- start start 2023-12-01T01:48:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.788+0000 7fc3074d5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc30006f2f0 0x7fc30006bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.788+0000 7fc3074d5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3000add10 con 0x7fc30015e2f0 2023-12-01T01:48:56.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.788+0000 7fc3074d5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3000c5760 con 0x7fc3000cad70 2023-12-01T01:48:56.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.788+0000 7fc3074d5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30015e6d0 con 0x7fc30006f2f0 2023-12-01T01:48:56.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.788+0000 7fc305271640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc30015e2f0 0x7fc3000ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59730/0 (socket says 172.21.15.115:59730) 2023-12-01T01:48:56.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.788+0000 7fc305271640 1 -- 172.21.15.115:0/1335955662 learned_addr learned my addr 172.21.15.115:0/1335955662 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239620319 0 0) 0x7fc3000add10 con 0x7fc30015e2f0 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc305a72640 1 --2- 172.21.15.115:0/1335955662 >> v2:172.21.15.142:3300/0 conn(0x7fc30006f2f0 0x7fc30006bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2ec003520 con 0x7fc30015e2f0 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3975329721 0 0) 0x7fc2ec003520 con 0x7fc30015e2f0 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 >> v2:172.21.15.142:3300/0 conn(0x7fc30006f2f0 msgr2=0x7fc30006bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 --2- 172.21.15.115:0/1335955662 >> v2:172.21.15.142:3300/0 conn(0x7fc30006f2f0 0x7fc30006bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 >> v1:172.21.15.142:6789/0 conn(0x7fc3000cad70 legacy=0x7fc30006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3000cf7b0 con 0x7fc30015e2f0 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc2fc001690 con 0x7fc30015e2f0 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc305a72640 1 --2- 172.21.15.115:0/1335955662 >> v2:172.21.15.142:3300/0 conn(0x7fc30006f2f0 0x7fc30006bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 >> v1:172.21.15.115:6789/0 conn(0x7fc30015e2f0 legacy=0x7fc3000ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2ec003a10 0x7fc2ec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ec0047c0 0x7fc2ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc2ec00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30015e6d0 con 0x7fc2ec005380 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec00c280 con 0x7fc2ec0047c0 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.789+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec00c440 con 0x7fc2ec003a10 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.790+0000 7fc305a72640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc2ec00b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.790+0000 7fc305271640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2ec003a10 0x7fc2ec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.790+0000 7fc305a72640 1 -- 172.21.15.115:0/1335955662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2ec003a10 msgr2=0x7fc2ec003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.790+0000 7fc305a72640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2ec003a10 0x7fc2ec003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.790+0000 7fc305a72640 1 -- 172.21.15.115:0/1335955662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ec0047c0 msgr2=0x7fc2ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.790+0000 7fc304a70640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ec0047c0 0x7fc2ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.790+0000 7fc305a72640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ec0047c0 0x7fc2ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.790+0000 7fc305a72640 1 -- 172.21.15.115:0/1335955662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec003520 con 0x7fc2ec005380 2023-12-01T01:48:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.790+0000 7fc304a70640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2ec0047c0 0x7fc2ec004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.791+0000 7fc305a72640 1 -- 172.21.15.115:0/1335955662 reap_dead start 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.791+0000 7fc305a72640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc2ec00b9a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc30006f700 tx=0x7fc2ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8f73a735a6923a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.791+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f802cd30 con 0x7fc2ec005380 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.791+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f802ce90 con 0x7fc2ec005380 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.791+0000 7fc2f7fff640 1 -- 172.21.15.115:0/1335955662 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8003bd0 con 0x7fc2ec005380 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.792+0000 7fc3074d5640 1 -- 172.21.15.115:0/1335955662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 msgr2=0x7fc2ec00b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.792+0000 7fc3074d5640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc2ec00b9a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc30006f700 tx=0x7fc2ec0057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.792+0000 7fc3074d5640 1 -- 172.21.15.115:0/1335955662 shutdown_connections 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.792+0000 7fc3074d5640 1 --2- 172.21.15.115:0/1335955662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc2ec00b9a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.792+0000 7fc3074d5640 1 -- 172.21.15.115:0/1335955662 >> 172.21.15.115:0/1335955662 conn(0x7fc3000ce880 msgr2=0x7fc3000cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.792+0000 7fc3074d5640 1 -- 172.21.15.115:0/1335955662 shutdown_connections 2023-12-01T01:48:56.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.792+0000 7fc3074d5640 1 -- 172.21.15.115:0/1335955662 wait complete. 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.794+0000 7fc3074d5640 1 Processor -- start 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.794+0000 7fc3074d5640 1 -- start start 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc3074d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc300158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc3074d5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3000c3f60 0x7fc3000cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc3074d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3001f18c0 0x7fc3001f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc3074d5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000c44d0 con 0x7fc2ec005380 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc3074d5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000c47b0 con 0x7fc3000c3f60 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc3074d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000c4a90 con 0x7fc3001f18c0 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc305271640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc300158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc305271640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc300158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42962/0 (socket says 172.21.15.115:42962) 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc305271640 1 -- 172.21.15.115:0/767461560 learned_addr learned my addr 172.21.15.115:0/767461560 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc304a70640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3000c3f60 0x7fc3000cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc305271640 1 -- 172.21.15.115:0/767461560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3001f18c0 msgr2=0x7fc3001f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.795+0000 7fc305271640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3001f18c0 0x7fc3001f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.796+0000 7fc305271640 1 -- 172.21.15.115:0/767461560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3000c3f60 msgr2=0x7fc3000cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.796+0000 7fc305271640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3000c3f60 0x7fc3000cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.796+0000 7fc305271640 1 -- 172.21.15.115:0/767461560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3000c4d70 con 0x7fc2ec005380 2023-12-01T01:48:56.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.796+0000 7fc305271640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc300158740 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc001900 tx=0x7fc2fc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:56.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.796+0000 7fc2f5ffb640 1 -- 172.21.15.115:0/767461560 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00c6c0 con 0x7fc2ec005380 2023-12-01T01:48:56.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.796+0000 7fc2f5ffb640 1 -- 172.21.15.115:0/767461560 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc00c820 con 0x7fc2ec005380 2023-12-01T01:48:56.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.796+0000 7fc2f5ffb640 1 -- 172.21.15.115:0/767461560 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00cbd0 con 0x7fc2ec005380 2023-12-01T01:48:56.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.796+0000 7fc3074d5640 1 -- 172.21.15.115:0/767461560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3001f13a0 con 0x7fc2ec005380 2023-12-01T01:48:56.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.796+0000 7fc3074d5640 1 -- 172.21.15.115:0/767461560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3000cb150 con 0x7fc2ec005380 2023-12-01T01:48:56.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.797+0000 7fc3074d5640 1 -- 172.21.15.115:0/767461560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30006fda0 con 0x7fc2ec005380 2023-12-01T01:48:56.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.798+0000 7fc2f5ffb640 1 -- 172.21.15.115:0/767461560 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fc2fc00cd30 con 0x7fc2ec005380 2023-12-01T01:48:56.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.799+0000 7fc2f5ffb640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2e406a4f0 0x7fc2e406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:56.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.799+0000 7fc2f5ffb640 1 -- 172.21.15.115:0/767461560 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fc2fc088cd0 con 0x7fc2ec005380 2023-12-01T01:48:56.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.799+0000 7fc304a70640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2e406a4f0 0x7fc2e406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:56.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.800+0000 7fc304a70640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2e406a4f0 0x7fc2e406c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc3000b3610 tx=0x7fc2f0009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:56.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.803+0000 7fc2f5ffb640 1 -- 172.21.15.115:0/767461560 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc2fc053d40 con 0x7fc2ec005380 2023-12-01T01:48:56.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.982+0000 7fc3074d5640 1 -- 172.21.15.115:0/767461560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3001f1200 con 0x7fc2ec005380 2023-12-01T01:48:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.985+0000 7fc2f5ffb640 1 -- 172.21.15.115:0/767461560 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+17917 (secure 0 0 0) 0x7fc3001f1200 con 0x7fc2ec005380 2023-12-01T01:48:56.986 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:48:56.986 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":52,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:48:53.861318+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:37:22.073677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-01T01:37:22.660650+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.23","mappings":[{"from":4,"to":5}]},{"pgid":"5.33","mappings":[{"from":4,"to":5}]},{"pgid":"5.3c","mappings":[{"from":4,"to":5}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/1312252546":"2023-12-02T01:48:53.861286+0000","172.21.15.115:0/1127148126":"2023-12-02T01:48:53.861286+0000","172.21.15.115:0/892358745":"2023-12-02T01:48:53.861286+0000","172.21.15.115:6824/88317":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/3103964125":"2023-12-02T01:48:53.861286+0000","172.21.15.115:6822/88317":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/4134228660":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/3690374531":"2023-12-02T01:48:53.861286+0000","172.21.15.115:0/1343452792":"2023-12-02T01:48:53.861286+0000","172.21.15.115:0/297799080":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/2772845026":"2023-12-02T01:48:53.861286+0000","172.21.15.115:0/1314334167":"2023-12-02T01:37:37.403693+0000","172.21.15.115:0/2515539287":"2023-12-02T01:37:37.403693+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1},{"begin":6,"length":1},{"begin":8,"length":1},{"begin":10,"length":1},{"begin":12,"length":1},{"begin":14,"length":1},{"begin":16,"length":1},{"begin":18,"length":2},{"begin":22,"length":2},{"begin":26,"length":3},{"begin":33,"length":5},{"begin":42,"length":1},{"begin":44,"length":3},{"begin":48,"length":1},{"begin":52,"length":3},{"begin":57,"length":2},{"begin":63,"length":1},{"begin":65,"length":4},{"begin":73,"length":1},{"begin":75,"length":4},{"begin":83,"length":4},{"begin":89,"length":1},{"begin":94,"length":1},{"begin":96,"length":2},{"begin":100,"length":3},{"begin":105,"length":3},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":166,"length":56}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:48:56.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 -- 172.21.15.115:0/767461560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2e406a4f0 msgr2=0x7fc2e406c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2e406a4f0 0x7fc2e406c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc3000b3610 tx=0x7fc2f0009040 comp rx=0 tx=0).stop 2023-12-01T01:48:56.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 -- 172.21.15.115:0/767461560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 msgr2=0x7fc300158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc300158740 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc001900 tx=0x7fc2fc00eb90 comp rx=0 tx=0).stop 2023-12-01T01:48:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 -- 172.21.15.115:0/767461560 shutdown_connections 2023-12-01T01:48:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2e406a4f0 0x7fc2e406c9b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3001f18c0 0x7fc3001f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3000c3f60 0x7fc3000cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 --2- 172.21.15.115:0/767461560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ec005380 0x7fc300158740 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 -- 172.21.15.115:0/767461560 >> 172.21.15.115:0/767461560 conn(0x7fc3000ce880 msgr2=0x7fc3000cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 -- 172.21.15.115:0/767461560 shutdown_connections 2023-12-01T01:48:56.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:56.988+0000 7fc3074d5640 1 -- 172.21.15.115:0/767461560 wait complete. 2023-12-01T01:48:57.007 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T01:48:57.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.213+0000 7f08063ab640 1 Processor -- start 2023-12-01T01:48:57.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.214+0000 7f08063ab640 1 -- start start 2023-12-01T01:48:57.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.215+0000 7f08063ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 0x7f08000c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.215+0000 7f08063ab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0800070d30 0x7f08000c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.215+0000 7f08063ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f080006f8b0 con 0x7f080006b6f0 2023-12-01T01:48:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.215+0000 7f08063ab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08000adcb0 con 0x7f0800070d30 2023-12-01T01:48:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.215+0000 7f08063ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000ad9d0 con 0x7f0800159dd0 2023-12-01T01:48:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.216+0000 7f0804948640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0800070d30 0x7f08000c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.216+0000 7f0804948640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0800070d30 0x7f08000c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:59570/0 (socket says 172.21.15.115:59570) 2023-12-01T01:48:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.216+0000 7f0804948640 1 -- 172.21.15.115:0/727897532 learned_addr learned my addr 172.21.15.115:0/727897532 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.216+0000 7f07fffff640 1 --2- 172.21.15.115:0/727897532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 0x7f08000c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.217+0000 7f07fffff640 1 -- 172.21.15.115:0/727897532 >> v2:172.21.15.142:3300/0 conn(0x7f0800070d30 msgr2=0x7f08000c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.217+0000 7f07fffff640 1 --2- 172.21.15.115:0/727897532 >> v2:172.21.15.142:3300/0 conn(0x7f0800070d30 0x7f08000c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.217+0000 7f07fffff640 1 -- 172.21.15.115:0/727897532 >> v1:172.21.15.142:6789/0 conn(0x7f080006b6f0 legacy=0x7f08000cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.217+0000 7f07fffff640 1 -- 172.21.15.115:0/727897532 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000c9000 con 0x7f0800159dd0 2023-12-01T01:48:57.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.218+0000 7f07fffff640 1 --2- 172.21.15.115:0/727897532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 0x7f08000c97e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f07f00097a0 tx=0x7f07f002b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb56c6afaec49e4b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:57.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f07ff7fe640 1 -- 172.21.15.115:0/727897532 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f002f050 con 0x7f0800159dd0 2023-12-01T01:48:57.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f07ff7fe640 1 -- 172.21.15.115:0/727897532 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f0031700 con 0x7f0800159dd0 2023-12-01T01:48:57.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f07ff7fe640 1 -- 172.21.15.115:0/727897532 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0031a90 con 0x7f0800159dd0 2023-12-01T01:48:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f08063ab640 1 -- 172.21.15.115:0/727897532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 msgr2=0x7f08000c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f08063ab640 1 --2- 172.21.15.115:0/727897532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 0x7f08000c97e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f07f00097a0 tx=0x7f07f002b7a0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f08063ab640 1 -- 172.21.15.115:0/727897532 shutdown_connections 2023-12-01T01:48:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f08063ab640 1 --2- 172.21.15.115:0/727897532 >> v2:172.21.15.142:3300/0 conn(0x7f0800070d30 0x7f08000c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f08063ab640 1 --2- 172.21.15.115:0/727897532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 0x7f08000c97e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f08063ab640 1 -- 172.21.15.115:0/727897532 >> 172.21.15.115:0/727897532 conn(0x7f08000ce880 msgr2=0x7f08000cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f08063ab640 1 -- 172.21.15.115:0/727897532 shutdown_connections 2023-12-01T01:48:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.219+0000 7f08063ab640 1 -- 172.21.15.115:0/727897532 wait complete. 2023-12-01T01:48:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.221+0000 7f08063ab640 1 Processor -- start 2023-12-01T01:48:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.221+0000 7f08063ab640 1 -- start start 2023-12-01T01:48:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.222+0000 7f08063ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f080006b6f0 0x7f08001f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.222+0000 7f08063ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0800070d30 0x7f08000cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.222+0000 7f08063ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 0x7f08000cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.222+0000 7f08063ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001f66e0 con 0x7f080006b6f0 2023-12-01T01:48:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.222+0000 7f08063ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08001f69c0 con 0x7f0800070d30 2023-12-01T01:48:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.222+0000 7f08063ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08001f6ca0 con 0x7f0800159dd0 2023-12-01T01:48:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.222+0000 7f07fffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f080006b6f0 0x7f08001f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f07fffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f080006b6f0 0x7f08001f5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42974/0 (socket says 172.21.15.115:42974) 2023-12-01T01:48:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f07fffff640 1 -- 172.21.15.115:0/539592813 learned_addr learned my addr 172.21.15.115:0/539592813 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f0804948640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 0x7f08000cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f07f7fff640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0800070d30 0x7f08000cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f07fffff640 1 -- 172.21.15.115:0/539592813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 msgr2=0x7f08000cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f07fffff640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 0x7f08000cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f07fffff640 1 -- 172.21.15.115:0/539592813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0800070d30 msgr2=0x7f08000cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f07fffff640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0800070d30 0x7f08000cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f07fffff640 1 -- 172.21.15.115:0/539592813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0009400 con 0x7f080006b6f0 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f07f7fff640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0800070d30 0x7f08000cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.223+0000 7f07fffff640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f080006b6f0 0x7f08001f5de0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f07f00098c0 tx=0x7f07f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.224+0000 7f07fd7fa640 1 -- 172.21.15.115:0/539592813 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00339b0 con 0x7f080006b6f0 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.224+0000 7f07fd7fa640 1 -- 172.21.15.115:0/539592813 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f0033b30 con 0x7f080006b6f0 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.224+0000 7f07fd7fa640 1 -- 172.21.15.115:0/539592813 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f003d410 con 0x7f080006b6f0 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.224+0000 7f08063ab640 1 -- 172.21.15.115:0/539592813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08000c2660 con 0x7f080006b6f0 2023-12-01T01:48:57.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.224+0000 7f08063ab640 1 -- 172.21.15.115:0/539592813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08000c5950 con 0x7f080006b6f0 2023-12-01T01:48:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.225+0000 7f08063ab640 1 -- 172.21.15.115:0/539592813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080004e890 con 0x7f080006b6f0 2023-12-01T01:48:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.226+0000 7f07fd7fa640 1 -- 172.21.15.115:0/539592813 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f07f003a050 con 0x7f080006b6f0 2023-12-01T01:48:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.227+0000 7f07fd7fa640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f07e006a4c0 0x7f07e006c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.228+0000 7f07fd7fa640 1 -- 172.21.15.115:0/539592813 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f07f00aab80 con 0x7f080006b6f0 2023-12-01T01:48:57.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.228+0000 7f07f7fff640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f07e006a4c0 0x7f07e006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.229+0000 7f07f7fff640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f07e006a4c0 0x7f07e006c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0800157680 tx=0x7f07e8009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:57.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.232+0000 7f07fd7fa640 1 -- 172.21.15.115:0/539592813 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f07f0075b60 con 0x7f080006b6f0 2023-12-01T01:48:57.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.451+0000 7f08063ab640 1 -- 172.21.15.115:0/539592813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f080015c250 con 0x7f080006b6f0 2023-12-01T01:48:57.452 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T01:48:57.452+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:48:57.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.468+0000 7f07fd7fa640 1 -- 172.21.15.115:0/539592813 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f07f0079180 con 0x7f080006b6f0 2023-12-01T01:48:57.469 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T01:48:57.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.471+0000 7f08063ab640 1 -- 172.21.15.115:0/539592813 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f07e006a4c0 msgr2=0x7f07e006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:57.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.471+0000 7f08063ab640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f07e006a4c0 0x7f07e006c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0800157680 tx=0x7f07e8009040 comp rx=0 tx=0).stop 2023-12-01T01:48:57.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.471+0000 7f08063ab640 1 -- 172.21.15.115:0/539592813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f080006b6f0 msgr2=0x7f08001f5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:57.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.471+0000 7f08063ab640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f080006b6f0 0x7f08001f5de0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f07f00098c0 tx=0x7f07f00046c0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.471+0000 7f08063ab640 1 -- 172.21.15.115:0/539592813 shutdown_connections 2023-12-01T01:48:57.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.471+0000 7f08063ab640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f07e006a4c0 0x7f07e006c980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.471+0000 7f08063ab640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0800159dd0 0x7f08000cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.471+0000 7f08063ab640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0800070d30 0x7f08000cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.471+0000 7f08063ab640 1 --2- 172.21.15.115:0/539592813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f080006b6f0 0x7f08001f5de0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.471+0000 7f08063ab640 1 -- 172.21.15.115:0/539592813 >> 172.21.15.115:0/539592813 conn(0x7f08000ce880 msgr2=0x7f08000c1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:57.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.472+0000 7f08063ab640 1 -- 172.21.15.115:0/539592813 shutdown_connections 2023-12-01T01:48:57.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.472+0000 7f08063ab640 1 -- 172.21.15.115:0/539592813 wait complete. 2023-12-01T01:48:57.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T01:48:57.534 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T01:48:57.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.675+0000 7f34334ce640 1 Processor -- start 2023-12-01T01:48:57.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.675+0000 7f34334ce640 1 -- start start 2023-12-01T01:48:57.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.677+0000 7f34334ce640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f342c15ac80 0x7f342c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.677+0000 7f34334ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0b0e60 0x7f342c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.677+0000 7f34334ce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0cc9a0 con 0x7f342c0cdaa0 2023-12-01T01:48:57.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.677+0000 7f34334ce640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c15b060 con 0x7f342c15ac80 2023-12-01T01:48:57.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.677+0000 7f34334ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0b0ce0 con 0x7f342c0b0e60 2023-12-01T01:48:57.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.677+0000 7f3431a6b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f342c0cdaa0 0x7f342c06b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43382/0 (socket says 172.21.15.115:43382) 2023-12-01T01:48:57.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.677+0000 7f3431a6b640 1 -- 172.21.15.115:0/950915929 learned_addr learned my addr 172.21.15.115:0/950915929 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:48:57.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.677+0000 7f343126a640 1 --2- 172.21.15.115:0/950915929 >> v2:172.21.15.142:3300/0 conn(0x7f342c15ac80 0x7f342c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f3430a69640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0b0e60 0x7f342c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120691188 0 0) 0x7f342c0cc9a0 con 0x7f342c0cdaa0 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3418003520 con 0x7f342c0cdaa0 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1220714761 0 0) 0x7f3418003520 con 0x7f342c0cdaa0 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0b0e60 msgr2=0x7f342c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0b0e60 0x7f342c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 >> v2:172.21.15.142:3300/0 conn(0x7f342c15ac80 msgr2=0x7f342c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 --2- 172.21.15.115:0/950915929 >> v2:172.21.15.142:3300/0 conn(0x7f342c15ac80 0x7f342c0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0cbd90 con 0x7f342c0cdaa0 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f34240019f0 con 0x7f342c0cdaa0 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f343126a640 1 --2- 172.21.15.115:0/950915929 >> v2:172.21.15.142:3300/0 conn(0x7f342c15ac80 0x7f342c0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 >> v1:172.21.15.115:6789/0 conn(0x7f342c0cdaa0 legacy=0x7f342c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:57.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0b0e60 0x7f3418003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 0x7f34180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3418004d00 0x7f3418005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c15b060 con 0x7f3418003f40 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0b0ce0 con 0x7f3418004d00 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341800ba30 con 0x7f342c0b0e60 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.678+0000 7f3430a69640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 0x7f34180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.679+0000 7f343126a640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0b0e60 0x7f3418003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.679+0000 7f3430a69640 1 -- 172.21.15.115:0/950915929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0b0e60 msgr2=0x7f3418003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.679+0000 7f3430a69640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0b0e60 0x7f3418003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.679+0000 7f3430a69640 1 -- 172.21.15.115:0/950915929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3418004d00 msgr2=0x7f3418005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.679+0000 7f3430a69640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3418004d00 0x7f3418005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.679+0000 7f3430a69640 1 -- 172.21.15.115:0/950915929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3418003520 con 0x7f3418003f40 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.680+0000 7f3430a69640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 0x7f34180047c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3420009ac0 tx=0x7f34180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07300df5ab3614a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.680+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342002c6b0 con 0x7f3418003f40 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.680+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342002c810 con 0x7f3418003f40 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.680+0000 7f341ffff640 1 -- 172.21.15.115:0/950915929 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342002cba0 con 0x7f3418003f40 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.681+0000 7f34334ce640 1 -- 172.21.15.115:0/950915929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 msgr2=0x7f34180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.681+0000 7f34334ce640 1 --2- 172.21.15.115:0/950915929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 0x7f34180047c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3420009ac0 tx=0x7f34180057f0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.681+0000 7f3431a6b640 1 -- 172.21.15.115:0/950915929 reap_dead start 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.681+0000 7f34334ce640 1 -- 172.21.15.115:0/950915929 shutdown_connections 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.681+0000 7f34334ce640 1 -- 172.21.15.115:0/950915929 >> 172.21.15.115:0/950915929 conn(0x7f342c065960 msgr2=0x7f342c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.681+0000 7f34334ce640 1 -- 172.21.15.115:0/950915929 shutdown_connections 2023-12-01T01:48:57.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.681+0000 7f34334ce640 1 -- 172.21.15.115:0/950915929 wait complete. 2023-12-01T01:48:57.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.683+0000 7f34334ce640 1 Processor -- start 2023-12-01T01:48:57.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.683+0000 7f34334ce640 1 -- start start 2023-12-01T01:48:57.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f34334ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 0x7f342c147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f34334ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c15c670 0x7f342c15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f34334ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f342c156bb0 0x7f342c157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f34334ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c15d030 con 0x7f3418003f40 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f34334ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c157570 con 0x7f342c156bb0 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f34334ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1f2ac0 con 0x7f342c15c670 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f343126a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 0x7f342c147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f343126a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 0x7f342c147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55606/0 (socket says 172.21.15.115:55606) 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f343126a640 1 -- 172.21.15.115:0/379855671 learned_addr learned my addr 172.21.15.115:0/379855671 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f3430a69640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c15c670 0x7f342c15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f343126a640 1 -- 172.21.15.115:0/379855671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c15c670 msgr2=0x7f342c15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f343126a640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c15c670 0x7f342c15caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f343126a640 1 -- 172.21.15.115:0/379855671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f342c156bb0 msgr2=0x7f342c157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:57.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f343126a640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f342c156bb0 0x7f342c157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:57.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.684+0000 7f343126a640 1 -- 172.21.15.115:0/379855671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c1577e0 con 0x7f3418003f40 2023-12-01T01:48:57.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.685+0000 7f343126a640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 0x7f342c147350 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3428010810 tx=0x7f3428010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:57.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.685+0000 7f341dffb640 1 -- 172.21.15.115:0/379855671 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800eb60 con 0x7f3418003f40 2023-12-01T01:48:57.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.685+0000 7f341dffb640 1 -- 172.21.15.115:0/379855671 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342800c050 con 0x7f3418003f40 2023-12-01T01:48:57.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.685+0000 7f341dffb640 1 -- 172.21.15.115:0/379855671 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428015470 con 0x7f3418003f40 2023-12-01T01:48:57.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.685+0000 7f34334ce640 1 -- 172.21.15.115:0/379855671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c158050 con 0x7f3418003f40 2023-12-01T01:48:57.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.685+0000 7f34334ce640 1 -- 172.21.15.115:0/379855671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c158610 con 0x7f3418003f40 2023-12-01T01:48:57.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.686+0000 7f34334ce640 1 -- 172.21.15.115:0/379855671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c04e890 con 0x7f3418003f40 2023-12-01T01:48:57.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.687+0000 7f341dffb640 1 -- 172.21.15.115:0/379855671 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f34280155d0 con 0x7f3418003f40 2023-12-01T01:48:57.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.688+0000 7f341dffb640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f341006a4f0 0x7f341006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:57.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.689+0000 7f341dffb640 1 -- 172.21.15.115:0/379855671 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f34280899d0 con 0x7f3418003f40 2023-12-01T01:48:57.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.689+0000 7f3430a69640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f341006a4f0 0x7f341006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:57.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.690+0000 7f3430a69640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f341006a4f0 0x7f341006c9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3420036000 tx=0x7f3420030000 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:57.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.693+0000 7f341dffb640 1 -- 172.21.15.115:0/379855671 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f34280548d0 con 0x7f3418003f40 2023-12-01T01:48:57.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:57.914+0000 7f34334ce640 1 -- 172.21.15.115:0/379855671 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f342c06cb40 con 0x7f3418003f40 2023-12-01T01:48:58.220 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:48:58.229 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:48:58.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.491+0000 7f341dffb640 1 -- 172.21.15.115:0/379855671 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3428057ef0 con 0x7f3418003f40 2023-12-01T01:48:58.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.494+0000 7f34334ce640 1 -- 172.21.15.115:0/379855671 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f341006a4f0 msgr2=0x7f341006c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:58.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.494+0000 7f34334ce640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f341006a4f0 0x7f341006c9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3420036000 tx=0x7f3420030000 comp rx=0 tx=0).stop 2023-12-01T01:48:58.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.494+0000 7f34334ce640 1 -- 172.21.15.115:0/379855671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 msgr2=0x7f342c147350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:58.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.494+0000 7f34334ce640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 0x7f342c147350 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3428010810 tx=0x7f3428010cd0 comp rx=0 tx=0).stop 2023-12-01T01:48:58.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.494+0000 7f34334ce640 1 -- 172.21.15.115:0/379855671 shutdown_connections 2023-12-01T01:48:58.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.494+0000 7f34334ce640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f341006a4f0 0x7f341006c9b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:58.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.494+0000 7f34334ce640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c15c670 0x7f342c15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.494+0000 7f34334ce640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f342c156bb0 0x7f342c157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.494+0000 7f34334ce640 1 --2- 172.21.15.115:0/379855671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3418003f40 0x7f342c147350 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.494+0000 7f34334ce640 1 -- 172.21.15.115:0/379855671 >> 172.21.15.115:0/379855671 conn(0x7f342c065960 msgr2=0x7f342c0709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.495+0000 7f34334ce640 1 -- 172.21.15.115:0/379855671 shutdown_connections 2023-12-01T01:48:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.495+0000 7f34334ce640 1 -- 172.21.15.115:0/379855671 wait complete. 2023-12-01T01:48:58.519 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T01:48:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.726+0000 7faffd2af640 1 Processor -- start 2023-12-01T01:48:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.727+0000 7faffd2af640 1 -- start start 2023-12-01T01:48:58.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.728+0000 7faffd2af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815dcf0 0x7faff80d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:58.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.728+0000 7faffd2af640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faff80d2f10 0x7faff80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:58.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.728+0000 7faffd2af640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff80b2300 con 0x7faff815d910 2023-12-01T01:48:58.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.728+0000 7faffd2af640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff81612b0 con 0x7faff80d2f10 2023-12-01T01:48:58.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.728+0000 7faffd2af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff81616d0 con 0x7faff815dcf0 2023-12-01T01:48:58.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.729+0000 7faffcaae640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faff80d2f10 0x7faff80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:58.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.729+0000 7faff77fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815dcf0 0x7faff80d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:58.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.729+0000 7faffcaae640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faff80d2f10 0x7faff80d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:36754/0 (socket says 172.21.15.115:36754) 2023-12-01T01:48:58.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.729+0000 7faffcaae640 1 -- 172.21.15.115:0/565090214 learned_addr learned my addr 172.21.15.115:0/565090214 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:58.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.730+0000 7faff77fe640 1 -- 172.21.15.115:0/565090214 >> v2:172.21.15.142:3300/0 conn(0x7faff80d2f10 msgr2=0x7faff80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:58.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.730+0000 7faff77fe640 1 --2- 172.21.15.115:0/565090214 >> v2:172.21.15.142:3300/0 conn(0x7faff80d2f10 0x7faff80d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:58.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.730+0000 7faff77fe640 1 -- 172.21.15.115:0/565090214 >> v1:172.21.15.142:6789/0 conn(0x7faff815d910 legacy=0x7faff80cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:58.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.730+0000 7faff77fe640 1 -- 172.21.15.115:0/565090214 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff80ce260 con 0x7faff815dcf0 2023-12-01T01:48:58.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.730+0000 7faffcaae640 1 --2- 172.21.15.115:0/565090214 >> v2:172.21.15.142:3300/0 conn(0x7faff80d2f10 0x7faff80d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:48:58.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.731+0000 7faff77fe640 1 --2- 172.21.15.115:0/565090214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815dcf0 0x7faff80d29d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fafe00097a0 tx=0x7fafe002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=299b3d07771b28a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:58.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.731+0000 7faff6ffd640 1 -- 172.21.15.115:0/565090214 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0031780 con 0x7faff815dcf0 2023-12-01T01:48:58.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.731+0000 7faff6ffd640 1 -- 172.21.15.115:0/565090214 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe00318e0 con 0x7faff815dcf0 2023-12-01T01:48:58.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.731+0000 7faff6ffd640 1 -- 172.21.15.115:0/565090214 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0031c70 con 0x7faff815dcf0 2023-12-01T01:48:58.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.732+0000 7faffd2af640 1 -- 172.21.15.115:0/565090214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815dcf0 msgr2=0x7faff80d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:58.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.732+0000 7faffd2af640 1 --2- 172.21.15.115:0/565090214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815dcf0 0x7faff80d29d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fafe00097a0 tx=0x7fafe002b820 comp rx=0 tx=0).stop 2023-12-01T01:48:58.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.732+0000 7faffd2af640 1 -- 172.21.15.115:0/565090214 shutdown_connections 2023-12-01T01:48:58.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.732+0000 7faffd2af640 1 --2- 172.21.15.115:0/565090214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815dcf0 0x7faff80d29d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:58.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.732+0000 7faffd2af640 1 --2- 172.21.15.115:0/565090214 >> v2:172.21.15.142:3300/0 conn(0x7faff80d2f10 0x7faff80d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:58.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.732+0000 7faffd2af640 1 -- 172.21.15.115:0/565090214 >> 172.21.15.115:0/565090214 conn(0x7faff806a0b0 msgr2=0x7faff806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:58.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.732+0000 7faffd2af640 1 -- 172.21.15.115:0/565090214 shutdown_connections 2023-12-01T01:48:58.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.732+0000 7faffd2af640 1 -- 172.21.15.115:0/565090214 wait complete. 2023-12-01T01:48:58.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.734+0000 7faffd2af640 1 Processor -- start 2023-12-01T01:48:58.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.734+0000 7faffd2af640 1 -- start start 2023-12-01T01:48:58.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.735+0000 7faffd2af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faff80d2f10 0x7faff815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:58.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.735+0000 7faffd2af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815d910 0x7faff80c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:58.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.735+0000 7faffd2af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faff815dcf0 0x7faff80cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:58.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.735+0000 7faffd2af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff815b050 con 0x7faff815dcf0 2023-12-01T01:48:58.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.735+0000 7faffd2af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff80ca440 con 0x7faff80d2f10 2023-12-01T01:48:58.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.735+0000 7faffd2af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff80ca720 con 0x7faff815d910 2023-12-01T01:48:58.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faffcaae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faff815dcf0 0x7faff80cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:58.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faffcaae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faff815dcf0 0x7faff80cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55608/0 (socket says 172.21.15.115:55608) 2023-12-01T01:48:58.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faffcaae640 1 -- 172.21.15.115:0/1423088144 learned_addr learned my addr 172.21.15.115:0/1423088144 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:58.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faff7fff640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faff80d2f10 0x7faff815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:58.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faff77fe640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815d910 0x7faff80c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:58.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faffcaae640 1 -- 172.21.15.115:0/1423088144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815d910 msgr2=0x7faff80c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:58.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faffcaae640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815d910 0x7faff80c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:58.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faffcaae640 1 -- 172.21.15.115:0/1423088144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faff80d2f10 msgr2=0x7faff815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:58.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faffcaae640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faff80d2f10 0x7faff815ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:58.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faffcaae640 1 -- 172.21.15.115:0/1423088144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff00094d0 con 0x7faff815dcf0 2023-12-01T01:48:58.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.736+0000 7faffcaae640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faff815dcf0 0x7faff80cbd00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7faff000e890 tx=0x7faff000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:58.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.737+0000 7faff4ff9640 1 -- 172.21.15.115:0/1423088144 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0004020 con 0x7faff815dcf0 2023-12-01T01:48:58.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.737+0000 7faff4ff9640 1 -- 172.21.15.115:0/1423088144 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff0004180 con 0x7faff815dcf0 2023-12-01T01:48:58.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.737+0000 7faffd2af640 1 -- 172.21.15.115:0/1423088144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe0009400 con 0x7faff815dcf0 2023-12-01T01:48:58.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.737+0000 7faff4ff9640 1 -- 172.21.15.115:0/1423088144 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff000c940 con 0x7faff815dcf0 2023-12-01T01:48:58.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.737+0000 7faffd2af640 1 -- 172.21.15.115:0/1423088144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff80cace0 con 0x7faff815dcf0 2023-12-01T01:48:58.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.738+0000 7faffd2af640 1 -- 172.21.15.115:0/1423088144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff80618d0 con 0x7faff815dcf0 2023-12-01T01:48:58.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.739+0000 7faff4ff9640 1 -- 172.21.15.115:0/1423088144 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7faff000caa0 con 0x7faff815dcf0 2023-12-01T01:48:58.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.739+0000 7faff4ff9640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fafd806a550 0x7fafd806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:58.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.740+0000 7faff4ff9640 1 -- 172.21.15.115:0/1423088144 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6962+0+0 (secure 0 0 0) 0x7faff0088e00 con 0x7faff815dcf0 2023-12-01T01:48:58.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.740+0000 7faff7fff640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fafd806a550 0x7fafd806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:58.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.741+0000 7faff7fff640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fafd806a550 0x7fafd806ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fafe80012e0 tx=0x7fafe8009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:58.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.744+0000 7faff4ff9640 1 -- 172.21.15.115:0/1423088144 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faff0053df0 con 0x7faff815dcf0 2023-12-01T01:48:58.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:58.931+0000 7faffd2af640 1 -- 172.21.15.115:0/1423088144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faff80cd880 con 0x7faff815dcf0 2023-12-01T01:48:59.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.506+0000 7faff4ff9640 1 -- 172.21.15.115:0/1423088144 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) v1 ==== 181+0+0 (secure 0 0 0) 0x7faff00170d0 con 0x7faff815dcf0 2023-12-01T01:48:59.508 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.510+0000 7faffd2af640 1 -- 172.21.15.115:0/1423088144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fafd806a550 msgr2=0x7fafd806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.510+0000 7faffd2af640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fafd806a550 0x7fafd806ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fafe80012e0 tx=0x7fafe8009040 comp rx=0 tx=0).stop 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.510+0000 7faffd2af640 1 -- 172.21.15.115:0/1423088144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faff815dcf0 msgr2=0x7faff80cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.510+0000 7faffd2af640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faff815dcf0 0x7faff80cbd00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7faff000e890 tx=0x7faff000ed50 comp rx=0 tx=0).stop 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.510+0000 7faffd2af640 1 -- 172.21.15.115:0/1423088144 shutdown_connections 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.510+0000 7faffd2af640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fafd806a550 0x7fafd806ca10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.510+0000 7faffd2af640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faff815d910 0x7faff80c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.510+0000 7faffd2af640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faff815dcf0 0x7faff80cbd00 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.511+0000 7faffd2af640 1 --2- 172.21.15.115:0/1423088144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faff80d2f10 0x7faff815ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.511+0000 7faffd2af640 1 -- 172.21.15.115:0/1423088144 >> 172.21.15.115:0/1423088144 conn(0x7faff806a0b0 msgr2=0x7faff80d1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.511+0000 7faffd2af640 1 -- 172.21.15.115:0/1423088144 shutdown_connections 2023-12-01T01:48:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.511+0000 7faffd2af640 1 -- 172.21.15.115:0/1423088144 wait complete. 2023-12-01T01:48:59.533 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T01:48:59.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.752+0000 7f4183473640 1 Processor -- start 2023-12-01T01:48:59.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.752+0000 7f4183473640 1 -- start start 2023-12-01T01:48:59.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.754+0000 7f4183473640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0af370 0x7f417c0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:59.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.754+0000 7f4183473640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f417c0c95a0 0x7f417c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:59.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.754+0000 7f4183473640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417c15e440 con 0x7f417c1599e0 2023-12-01T01:48:59.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.754+0000 7f4183473640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c15d550 con 0x7f417c0c95a0 2023-12-01T01:48:59.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.754+0000 7f4183473640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c15d830 con 0x7f417c0af370 2023-12-01T01:48:59.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.754+0000 7f4182c72640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f417c0c95a0 0x7f417c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:59.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.754+0000 7f4182c72640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f417c0c95a0 0x7f417c070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55620/0 (socket says 172.21.15.115:55620) 2023-12-01T01:48:59.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.754+0000 7f4182c72640 1 -- 172.21.15.115:0/2803915651 learned_addr learned my addr 172.21.15.115:0/2803915651 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:59.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.755+0000 7f4181c70640 1 --2- 172.21.15.115:0/2803915651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0af370 0x7f417c0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:59.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.755+0000 7f4182c72640 1 -- 172.21.15.115:0/2803915651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0af370 msgr2=0x7f417c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:59.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.755+0000 7f4182c72640 1 --2- 172.21.15.115:0/2803915651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0af370 0x7f417c0c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:59.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.755+0000 7f4182c72640 1 -- 172.21.15.115:0/2803915651 >> v1:172.21.15.142:6789/0 conn(0x7f417c1599e0 legacy=0x7f417c159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:48:59.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.755+0000 7f4182c72640 1 -- 172.21.15.115:0/2803915651 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c15db10 con 0x7f417c0c95a0 2023-12-01T01:48:59.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.755+0000 7f4181c70640 1 --2- 172.21.15.115:0/2803915651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0af370 0x7f417c0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:48:59.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.756+0000 7f4182c72640 1 --2- 172.21.15.115:0/2803915651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c0c95a0 0x7f417c070300 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4174009e50 tx=0x7f417402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512495d4ad4066c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:59.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.756+0000 7f418146f640 1 -- 172.21.15.115:0/2803915651 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417402ca70 con 0x7f417c0c95a0 2023-12-01T01:48:59.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.756+0000 7f418146f640 1 -- 172.21.15.115:0/2803915651 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417402cbd0 con 0x7f417c0c95a0 2023-12-01T01:48:59.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.756+0000 7f418146f640 1 -- 172.21.15.115:0/2803915651 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41740343f0 con 0x7f417c0c95a0 2023-12-01T01:48:59.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.757+0000 7f4183473640 1 -- 172.21.15.115:0/2803915651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c0c95a0 msgr2=0x7f417c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:59.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.757+0000 7f4183473640 1 --2- 172.21.15.115:0/2803915651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c0c95a0 0x7f417c070300 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4174009e50 tx=0x7f417402bed0 comp rx=0 tx=0).stop 2023-12-01T01:48:59.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.757+0000 7f4183473640 1 -- 172.21.15.115:0/2803915651 shutdown_connections 2023-12-01T01:48:59.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.757+0000 7f4183473640 1 --2- 172.21.15.115:0/2803915651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c0c95a0 0x7f417c070300 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:59.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.757+0000 7f4183473640 1 --2- 172.21.15.115:0/2803915651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0af370 0x7f417c0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:59.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.757+0000 7f4183473640 1 -- 172.21.15.115:0/2803915651 >> 172.21.15.115:0/2803915651 conn(0x7f417c065960 msgr2=0x7f417c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:48:59.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.757+0000 7f4183473640 1 -- 172.21.15.115:0/2803915651 shutdown_connections 2023-12-01T01:48:59.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.757+0000 7f4183473640 1 -- 172.21.15.115:0/2803915651 wait complete. 2023-12-01T01:48:59.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.759+0000 7f4183473640 1 Processor -- start 2023-12-01T01:48:59.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.759+0000 7f4183473640 1 -- start start 2023-12-01T01:48:59.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4183473640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f417c0af370 0x7f417c148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:59.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4183473640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0c95a0 0x7f417c0ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:59.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4183473640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c1599e0 0x7f417c0ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:59.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4183473640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0c2aa0 con 0x7f417c1599e0 2023-12-01T01:48:59.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4183473640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c1499e0 con 0x7f417c0af370 2023-12-01T01:48:59.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4183473640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0c4eb0 con 0x7f417c0c95a0 2023-12-01T01:48:59.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4182c72640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c1599e0 0x7f417c0ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:59.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4182c72640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c1599e0 0x7f417c0ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55626/0 (socket says 172.21.15.115:55626) 2023-12-01T01:48:59.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4182c72640 1 -- 172.21.15.115:0/650852541 learned_addr learned my addr 172.21.15.115:0/650852541 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:48:59.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4181c70640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0c95a0 0x7f417c0ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:59.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.760+0000 7f4182471640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f417c0af370 0x7f417c148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:59.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4182c72640 1 -- 172.21.15.115:0/650852541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0c95a0 msgr2=0x7f417c0ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:59.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4182c72640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0c95a0 0x7f417c0ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:59.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4182c72640 1 -- 172.21.15.115:0/650852541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f417c0af370 msgr2=0x7f417c148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:48:59.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4182c72640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f417c0af370 0x7f417c148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:48:59.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4182c72640 1 -- 172.21.15.115:0/650852541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4174009b30 con 0x7f417c1599e0 2023-12-01T01:48:59.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4182c72640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c1599e0 0x7f417c0ca670 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4174009e50 tx=0x7f4174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:59.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4172ffd640 1 -- 172.21.15.115:0/650852541 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417403e450 con 0x7f417c1599e0 2023-12-01T01:48:59.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4172ffd640 1 -- 172.21.15.115:0/650852541 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417403e5d0 con 0x7f417c1599e0 2023-12-01T01:48:59.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4183473640 1 -- 172.21.15.115:0/650852541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c0c23e0 con 0x7f417c1599e0 2023-12-01T01:48:59.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4172ffd640 1 -- 172.21.15.115:0/650852541 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417403e980 con 0x7f417c1599e0 2023-12-01T01:48:59.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.761+0000 7f4183473640 1 -- 172.21.15.115:0/650852541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c1493d0 con 0x7f417c1599e0 2023-12-01T01:48:59.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.762+0000 7f4183473640 1 -- 172.21.15.115:0/650852541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417c0c2be0 con 0x7f417c1599e0 2023-12-01T01:48:59.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.763+0000 7f4172ffd640 1 -- 172.21.15.115:0/650852541 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f417403eae0 con 0x7f417c1599e0 2023-12-01T01:48:59.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.764+0000 7f4172ffd640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a550 0x7f416006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:48:59.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.764+0000 7f4172ffd640 1 -- 172.21.15.115:0/650852541 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6524+0+0 (secure 0 0 0) 0x7f41740abf70 con 0x7f417c1599e0 2023-12-01T01:48:59.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.765+0000 7f4182471640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a550 0x7f416006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:48:59.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.765+0000 7f4182471640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a550 0x7f416006ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f417c1535c0 tx=0x7f4178009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:48:59.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.769+0000 7f4172ffd640 1 -- 172.21.15.115:0/650852541 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f41740dec80 con 0x7f417c1599e0 2023-12-01T01:48:59.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:48:59.961+0000 7f4183473640 1 -- 172.21.15.115:0/650852541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f417c0618d0 con 0x7f417c1599e0 2023-12-01T01:49:00.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.407+0000 7f4172ffd640 1 -- 172.21.15.115:0/650852541 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) v1 ==== 169+0+0 (secure 0 0 0) 0x7f417c0c2be0 con 0x7f417c1599e0 2023-12-01T01:49:00.408 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T01:49:00.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.412+0000 7f4183473640 1 -- 172.21.15.115:0/650852541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a550 msgr2=0x7f416006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:00.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.412+0000 7f4183473640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a550 0x7f416006ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f417c1535c0 tx=0x7f4178009040 comp rx=0 tx=0).stop 2023-12-01T01:49:00.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.412+0000 7f4183473640 1 -- 172.21.15.115:0/650852541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c1599e0 msgr2=0x7f417c0ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:00.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.412+0000 7f4183473640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c1599e0 0x7f417c0ca670 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4174009e50 tx=0x7f4174004a40 comp rx=0 tx=0).stop 2023-12-01T01:49:00.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.412+0000 7f4183473640 1 -- 172.21.15.115:0/650852541 shutdown_connections 2023-12-01T01:49:00.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.412+0000 7f4183473640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f416006a550 0x7f416006ca10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.412+0000 7f4183473640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f417c0c95a0 0x7f417c0ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.412+0000 7f4183473640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f417c1599e0 0x7f417c0ca670 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.412+0000 7f4183473640 1 --2- 172.21.15.115:0/650852541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f417c0af370 0x7f417c148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.412+0000 7f4183473640 1 -- 172.21.15.115:0/650852541 >> 172.21.15.115:0/650852541 conn(0x7f417c065960 msgr2=0x7f417c06be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:00.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.413+0000 7f4183473640 1 -- 172.21.15.115:0/650852541 shutdown_connections 2023-12-01T01:49:00.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.413+0000 7f4183473640 1 -- 172.21.15.115:0/650852541 wait complete. 2023-12-01T01:49:00.435 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T01:49:00.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.638+0000 7f94476e0640 1 Processor -- start 2023-12-01T01:49:00.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.638+0000 7f94476e0640 1 -- start start 2023-12-01T01:49:00.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.640+0000 7f94476e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94400cd1e0 0x7f94400b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:00.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.640+0000 7f94476e0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400b20e0 con 0x7f9440142870 2023-12-01T01:49:00.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.640+0000 7f94476e0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400b2f30 con 0x7f94400b1c50 2023-12-01T01:49:00.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.640+0000 7f94476e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94400b3ae0 con 0x7f94400cd1e0 2023-12-01T01:49:00.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.640+0000 7f944547c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9440142870 0x7f94400ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43396/0 (socket says 172.21.15.115:43396) 2023-12-01T01:49:00.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.640+0000 7f944547c640 1 -- 172.21.15.115:0/928002644 learned_addr learned my addr 172.21.15.115:0/928002644 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:00.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.640+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328126222 0 0) 0x7f943c001110 con 0x7f9440142870 2023-12-01T01:49:00.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.640+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f942c003520 con 0x7f9440142870 2023-12-01T01:49:00.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.640+0000 7f9444c7b640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94400cd1e0 0x7f94400b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:00.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 671363803 0 0) 0x7f942c003520 con 0x7f9440142870 2023-12-01T01:49:00.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94400cd1e0 msgr2=0x7f94400b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94400cd1e0 0x7f94400b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 >> v1:172.21.15.142:6789/0 conn(0x7f94400b1c50 legacy=0x7f944015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400b2320 con 0x7f9440142870 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f943c001800 con 0x7f9440142870 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 >> v1:172.21.15.115:6789/0 conn(0x7f9440142870 legacy=0x7f94400ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94400cd1e0 0x7f942c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c004020 0x7f942c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f942c004d00 0x7f942c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400b3ae0 con 0x7f942c004d00 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c00bbc0 con 0x7f942c004020 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c00df90 con 0x7f94400cd1e0 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.641+0000 7f9445c7d640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f942c004d00 0x7f942c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.642+0000 7f944547c640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94400cd1e0 0x7f942c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.642+0000 7f9444c7b640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c004020 0x7f942c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:00.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.642+0000 7f9445c7d640 1 -- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94400cd1e0 msgr2=0x7f942c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:00.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.642+0000 7f9445c7d640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94400cd1e0 0x7f942c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.642+0000 7f9445c7d640 1 -- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c004020 msgr2=0x7f942c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:00.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.642+0000 7f9445c7d640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c004020 0x7f942c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.642+0000 7f9445c7d640 1 -- 172.21.15.115:0/928002644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c003520 con 0x7f942c004d00 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.643+0000 7f9445c7d640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f942c004d00 0x7f942c005180 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9438001e50 tx=0x7f942c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f0fbcaa0849eb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.643+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438003b70 con 0x7f942c004d00 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.643+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9438003cd0 con 0x7f942c004d00 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.643+0000 7f9437fff640 1 -- 172.21.15.115:0/928002644 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438005310 con 0x7f942c004d00 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.644+0000 7f94476e0640 1 -- 172.21.15.115:0/928002644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f942c004d00 msgr2=0x7f942c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.644+0000 7f94476e0640 1 --2- 172.21.15.115:0/928002644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f942c004d00 0x7f942c005180 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9438001e50 tx=0x7f942c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.644+0000 7f9445c7d640 1 -- 172.21.15.115:0/928002644 reap_dead start 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.644+0000 7f94476e0640 1 -- 172.21.15.115:0/928002644 shutdown_connections 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.644+0000 7f94476e0640 1 -- 172.21.15.115:0/928002644 >> 172.21.15.115:0/928002644 conn(0x7f9440065960 msgr2=0x7f9440065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.644+0000 7f94476e0640 1 -- 172.21.15.115:0/928002644 shutdown_connections 2023-12-01T01:49:00.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.644+0000 7f94476e0640 1 -- 172.21.15.115:0/928002644 wait complete. 2023-12-01T01:49:00.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.646+0000 7f94476e0640 1 Processor -- start 2023-12-01T01:49:00.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.646+0000 7f94476e0640 1 -- start start 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f94476e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c004d00 0x7f94401451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f94476e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94401e1410 0x7f9440146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f94476e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94401e4160 0x7f9440147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f94476e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94401e1890 con 0x7f94401e4160 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f94476e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94401458b0 con 0x7f942c004d00 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f94476e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9440145b90 con 0x7f94401e1410 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f9445c7d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94401e4160 0x7f9440147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f9445c7d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94401e4160 0x7f9440147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55640/0 (socket says 172.21.15.115:55640) 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f9445c7d640 1 -- 172.21.15.115:0/2757005632 learned_addr learned my addr 172.21.15.115:0/2757005632 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f944547c640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c004d00 0x7f94401451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:00.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f9444c7b640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94401e1410 0x7f9440146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:00.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f9445c7d640 1 -- 172.21.15.115:0/2757005632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94401e1410 msgr2=0x7f9440146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:00.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f9445c7d640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94401e1410 0x7f9440146950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f9445c7d640 1 -- 172.21.15.115:0/2757005632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c004d00 msgr2=0x7f94401451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:00.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f9445c7d640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c004d00 0x7f94401451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f9445c7d640 1 -- 172.21.15.115:0/2757005632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440145e70 con 0x7f94401e4160 2023-12-01T01:49:00.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.647+0000 7f944547c640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c004d00 0x7f94401451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:00.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.648+0000 7f9445c7d640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94401e4160 0x7f9440147210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9438001f70 tx=0x7f94380014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:00.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.648+0000 7f9435ffb640 1 -- 172.21.15.115:0/2757005632 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943803e070 con 0x7f94401e4160 2023-12-01T01:49:00.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.648+0000 7f9435ffb640 1 -- 172.21.15.115:0/2757005632 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9438003e40 con 0x7f94401e4160 2023-12-01T01:49:00.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.648+0000 7f94476e0640 1 -- 172.21.15.115:0/2757005632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9440146150 con 0x7f94401e4160 2023-12-01T01:49:00.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.648+0000 7f9435ffb640 1 -- 172.21.15.115:0/2757005632 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438006d70 con 0x7f94401e4160 2023-12-01T01:49:00.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.648+0000 7f94476e0640 1 -- 172.21.15.115:0/2757005632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9440146e90 con 0x7f94401e4160 2023-12-01T01:49:00.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.649+0000 7f94476e0640 1 -- 172.21.15.115:0/2757005632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9440146670 con 0x7f94401e4160 2023-12-01T01:49:00.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.650+0000 7f9435ffb640 1 -- 172.21.15.115:0/2757005632 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9438007030 con 0x7f94401e4160 2023-12-01T01:49:00.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.651+0000 7f9435ffb640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f942406a550 0x7f942406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:00.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.651+0000 7f9435ffb640 1 -- 172.21.15.115:0/2757005632 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5994+0+0 (secure 0 0 0) 0x7f94380a9fd0 con 0x7f94401e4160 2023-12-01T01:49:00.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.651+0000 7f944547c640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f942406a550 0x7f942406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:00.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.652+0000 7f944547c640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f942406a550 0x7f942406ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f943c001050 tx=0x7f943c001e40 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:00.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.656+0000 7f9435ffb640 1 -- 172.21.15.115:0/2757005632 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9440146670 con 0x7f94401e4160 2023-12-01T01:49:00.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.846+0000 7f94476e0640 1 -- 172.21.15.115:0/2757005632 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9440146670 con 0x7f94401e4160 2023-12-01T01:49:00.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.854+0000 7f9435ffb640 1 -- 172.21.15.115:0/2757005632 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9440146670 con 0x7f94401e4160 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.857+0000 7f94476e0640 1 -- 172.21.15.115:0/2757005632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f942406a550 msgr2=0x7f942406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.857+0000 7f94476e0640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f942406a550 0x7f942406ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f943c001050 tx=0x7f943c001e40 comp rx=0 tx=0).stop 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.857+0000 7f94476e0640 1 -- 172.21.15.115:0/2757005632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94401e4160 msgr2=0x7f9440147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.857+0000 7f94476e0640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94401e4160 0x7f9440147210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9438001f70 tx=0x7f94380014a0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.857+0000 7f94476e0640 1 -- 172.21.15.115:0/2757005632 shutdown_connections 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.857+0000 7f94476e0640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f942406a550 0x7f942406ca10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.857+0000 7f94476e0640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94401e1410 0x7f9440146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.857+0000 7f94476e0640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94401e4160 0x7f9440147210 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.857+0000 7f94476e0640 1 --2- 172.21.15.115:0/2757005632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c004d00 0x7f94401451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.857+0000 7f94476e0640 1 -- 172.21.15.115:0/2757005632 >> 172.21.15.115:0/2757005632 conn(0x7f9440065960 msgr2=0x7f94400ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:00.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.858+0000 7f94476e0640 1 -- 172.21.15.115:0/2757005632 shutdown_connections 2023-12-01T01:49:00.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:00.858+0000 7f94476e0640 1 -- 172.21.15.115:0/2757005632 wait complete. 2023-12-01T01:49:00.876 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T01:49:01.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.073+0000 7f21d4828640 1 Processor -- start 2023-12-01T01:49:01.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.074+0000 7f21d4828640 1 -- start start 2023-12-01T01:49:01.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.075+0000 7f21d4828640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f21d015cd90 0x7f21d00cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.075+0000 7f21d4828640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d00cfd60 0x7f21d0161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.075+0000 7f21d4828640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d00ce5e0 con 0x7f21d00b5fa0 2023-12-01T01:49:01.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.075+0000 7f21d4828640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d00d0ec0 con 0x7f21d015cd90 2023-12-01T01:49:01.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.075+0000 7f21d4828640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0161950 con 0x7f21d00cfd60 2023-12-01T01:49:01.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.075+0000 7f21cdd9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f21d00b5fa0 0x7f21d00cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43398/0 (socket says 172.21.15.115:43398) 2023-12-01T01:49:01.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.076+0000 7f21cdd9b640 1 -- 172.21.15.115:0/2086546710 learned_addr learned my addr 172.21.15.115:0/2086546710 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:01.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.076+0000 7f21ce59c640 1 --2- 172.21.15.115:0/2086546710 >> v2:172.21.15.115:3300/0 conn(0x7f21d015cd90 0x7f21d00cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.076+0000 7f21ced9d640 1 --2- 172.21.15.115:0/2086546710 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d00cfd60 0x7f21d0161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.076+0000 7f21cd59a640 1 -- 172.21.15.115:0/2086546710 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141574567 0 0) 0x7f21b8001110 con 0x7f21d00b5fa0 2023-12-01T01:49:01.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.076+0000 7f21cd59a640 1 -- 172.21.15.115:0/2086546710 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21bc003520 con 0x7f21d00b5fa0 2023-12-01T01:49:01.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.076+0000 7f21ce59c640 1 -- 172.21.15.115:0/2086546710 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d00cfd60 msgr2=0x7f21d0161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:01.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.076+0000 7f21ce59c640 1 --2- 172.21.15.115:0/2086546710 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d00cfd60 0x7f21d0161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.076+0000 7f21ce59c640 1 -- 172.21.15.115:0/2086546710 >> v1:172.21.15.115:6789/0 conn(0x7f21d00b5fa0 legacy=0x7f21d00cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.076+0000 7f21ce59c640 1 -- 172.21.15.115:0/2086546710 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d00ce300 con 0x7f21d015cd90 2023-12-01T01:49:01.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.077+0000 7f21ce59c640 1 --2- 172.21.15.115:0/2086546710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d015cd90 0x7f21d00cdd20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009d90 tx=0x7f21c002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52983aca4617f95b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:01.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.077+0000 7f21cd59a640 1 -- 172.21.15.115:0/2086546710 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c002cc50 con 0x7f21d015cd90 2023-12-01T01:49:01.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.078+0000 7f21cd59a640 1 -- 172.21.15.115:0/2086546710 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c002cdb0 con 0x7f21d015cd90 2023-12-01T01:49:01.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.078+0000 7f21cd59a640 1 -- 172.21.15.115:0/2086546710 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0034660 con 0x7f21d015cd90 2023-12-01T01:49:01.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.078+0000 7f21d4828640 1 -- 172.21.15.115:0/2086546710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d015cd90 msgr2=0x7f21d00cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.078+0000 7f21d4828640 1 --2- 172.21.15.115:0/2086546710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d015cd90 0x7f21d00cdd20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009d90 tx=0x7f21c002bea0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.078+0000 7f21d4828640 1 -- 172.21.15.115:0/2086546710 shutdown_connections 2023-12-01T01:49:01.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.078+0000 7f21d4828640 1 --2- 172.21.15.115:0/2086546710 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d00cfd60 0x7f21d0161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.078+0000 7f21d4828640 1 --2- 172.21.15.115:0/2086546710 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d015cd90 0x7f21d00cdd20 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.078+0000 7f21d4828640 1 -- 172.21.15.115:0/2086546710 >> 172.21.15.115:0/2086546710 conn(0x7f21d0065960 msgr2=0x7f21d0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:01.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.078+0000 7f21d4828640 1 -- 172.21.15.115:0/2086546710 shutdown_connections 2023-12-01T01:49:01.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.078+0000 7f21d4828640 1 -- 172.21.15.115:0/2086546710 wait complete. 2023-12-01T01:49:01.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.080+0000 7f21d4828640 1 Processor -- start 2023-12-01T01:49:01.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.080+0000 7f21d4828640 1 -- start start 2023-12-01T01:49:01.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.081+0000 7f21d4828640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d00b5fa0 0x7f21d0157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.081+0000 7f21d4828640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d00cfd60 0x7f21d00cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.081+0000 7f21d4828640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21d015cd90 0x7f21d015f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.081+0000 7f21d4828640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00ab060 con 0x7f21d00b5fa0 2023-12-01T01:49:01.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.081+0000 7f21d4828640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00abec0 con 0x7f21d015cd90 2023-12-01T01:49:01.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.081+0000 7f21d4828640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00ac350 con 0x7f21d00cfd60 2023-12-01T01:49:01.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.081+0000 7f21ce59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d00b5fa0 0x7f21d0157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.081+0000 7f21ce59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d00b5fa0 0x7f21d0157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55652/0 (socket says 172.21.15.115:55652) 2023-12-01T01:49:01.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.081+0000 7f21ce59c640 1 -- 172.21.15.115:0/1197675244 learned_addr learned my addr 172.21.15.115:0/1197675244 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:01.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.082+0000 7f21cdd9b640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d00cfd60 0x7f21d00cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.082+0000 7f21ce59c640 1 -- 172.21.15.115:0/1197675244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d00cfd60 msgr2=0x7f21d00cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.082+0000 7f21ce59c640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d00cfd60 0x7f21d00cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.082+0000 7f21ce59c640 1 -- 172.21.15.115:0/1197675244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21d015cd90 msgr2=0x7f21d015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.082+0000 7f21ced9d640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21d015cd90 0x7f21d015f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.082+0000 7f21ce59c640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21d015cd90 0x7f21d015f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.082+0000 7f21ce59c640 1 -- 172.21.15.115:0/1197675244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c0009a70 con 0x7f21d00b5fa0 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.082+0000 7f21ce59c640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d00b5fa0 0x7f21d0157d60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009eb0 tx=0x7f21c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.083+0000 7f21c6ffd640 1 -- 172.21.15.115:0/1197675244 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0040650 con 0x7f21d00b5fa0 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.083+0000 7f21c6ffd640 1 -- 172.21.15.115:0/1197675244 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c00407b0 con 0x7f21d00b5fa0 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.083+0000 7f21c6ffd640 1 -- 172.21.15.115:0/1197675244 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0040b60 con 0x7f21d00b5fa0 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.083+0000 7f21d4828640 1 -- 172.21.15.115:0/1197675244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d01f23e0 con 0x7f21d00b5fa0 2023-12-01T01:49:01.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.083+0000 7f21d4828640 1 -- 172.21.15.115:0/1197675244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d00ab590 con 0x7f21d00b5fa0 2023-12-01T01:49:01.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.084+0000 7f21d4828640 1 -- 172.21.15.115:0/1197675244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d004e890 con 0x7f21d00b5fa0 2023-12-01T01:49:01.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.085+0000 7f21c6ffd640 1 -- 172.21.15.115:0/1197675244 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f21c0040cc0 con 0x7f21d00b5fa0 2023-12-01T01:49:01.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.085+0000 7f21c6ffd640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21b006a580 0x7f21b006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.086+0000 7f21c6ffd640 1 -- 172.21.15.115:0/1197675244 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5994+0+0 (secure 0 0 0) 0x7f21c00ab6e0 con 0x7f21d00b5fa0 2023-12-01T01:49:01.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.086+0000 7f21cdd9b640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21b006a580 0x7f21b006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.086+0000 7f21cdd9b640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21b006a580 0x7f21b006ca40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f21d01601e0 tx=0x7f21b8009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:01.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.090+0000 7f21c6ffd640 1 -- 172.21.15.115:0/1197675244 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f21c0076b30 con 0x7f21d00b5fa0 2023-12-01T01:49:01.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.290+0000 7f21d4828640 1 -- 172.21.15.115:0/1197675244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f21d00680e0 con 0x7f21d00b5fa0 2023-12-01T01:49:01.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.298+0000 7f21c6ffd640 1 -- 172.21.15.115:0/1197675244 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v14) v1 ==== 120+0+0 (secure 0 0 0) 0x7f21c007a150 con 0x7f21d00b5fa0 2023-12-01T01:49:01.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.301+0000 7f21d4828640 1 -- 172.21.15.115:0/1197675244 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21b006a580 msgr2=0x7f21b006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.301+0000 7f21d4828640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21b006a580 0x7f21b006ca40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f21d01601e0 tx=0x7f21b8009040 comp rx=0 tx=0).stop 2023-12-01T01:49:01.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.301+0000 7f21d4828640 1 -- 172.21.15.115:0/1197675244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d00b5fa0 msgr2=0x7f21d0157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.301+0000 7f21d4828640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d00b5fa0 0x7f21d0157d60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009eb0 tx=0x7f21c0004a40 comp rx=0 tx=0).stop 2023-12-01T01:49:01.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.301+0000 7f21d4828640 1 -- 172.21.15.115:0/1197675244 shutdown_connections 2023-12-01T01:49:01.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.301+0000 7f21d4828640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21b006a580 0x7f21b006ca40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.301+0000 7f21d4828640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d00cfd60 0x7f21d00cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.301+0000 7f21d4828640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21d015cd90 0x7f21d015f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.301+0000 7f21d4828640 1 --2- 172.21.15.115:0/1197675244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21d00b5fa0 0x7f21d0157d60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.301+0000 7f21d4828640 1 -- 172.21.15.115:0/1197675244 >> 172.21.15.115:0/1197675244 conn(0x7f21d0065960 msgr2=0x7f21d0160960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:01.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.302+0000 7f21d4828640 1 -- 172.21.15.115:0/1197675244 shutdown_connections 2023-12-01T01:49:01.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.302+0000 7f21d4828640 1 -- 172.21.15.115:0/1197675244 wait complete. 2023-12-01T01:49:01.319 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T01:49:01.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.528+0000 7f41e514e640 1 Processor -- start 2023-12-01T01:49:01.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.529+0000 7f41e514e640 1 -- start start 2023-12-01T01:49:01.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.530+0000 7f41e514e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e00cfd60 0x7f41e00cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.530+0000 7f41e514e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f41e00b5fa0 0x7f41e0161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.530+0000 7f41e514e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e00ce5e0 con 0x7f41e015cd90 2023-12-01T01:49:01.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.530+0000 7f41e514e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e00d0ec0 con 0x7f41e00b5fa0 2023-12-01T01:49:01.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.530+0000 7f41e514e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0161950 con 0x7f41e00cfd60 2023-12-01T01:49:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.531+0000 7f41df59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f41e00b5fa0 0x7f41e0161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.531+0000 7f41df59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f41e00b5fa0 0x7f41e0161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55668/0 (socket says 172.21.15.115:55668) 2023-12-01T01:49:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.531+0000 7f41df59e640 1 -- 172.21.15.115:0/2170350158 learned_addr learned my addr 172.21.15.115:0/2170350158 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.531+0000 7f41d7fff640 1 --2- 172.21.15.115:0/2170350158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e00cfd60 0x7f41e00cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.531+0000 7f41df59e640 1 -- 172.21.15.115:0/2170350158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e00cfd60 msgr2=0x7f41e00cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.531+0000 7f41df59e640 1 --2- 172.21.15.115:0/2170350158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e00cfd60 0x7f41e00cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.531+0000 7f41df59e640 1 -- 172.21.15.115:0/2170350158 >> v1:172.21.15.142:6789/0 conn(0x7f41e015cd90 legacy=0x7f41e00cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.531+0000 7f41df59e640 1 -- 172.21.15.115:0/2170350158 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00ce300 con 0x7f41e00b5fa0 2023-12-01T01:49:01.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.532+0000 7f41df59e640 1 --2- 172.21.15.115:0/2170350158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00b5fa0 0x7f41e0161410 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009870 tx=0x7f41d802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a6d682f602db08 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:01.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.534+0000 7f41de59c640 1 -- 172.21.15.115:0/2170350158 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d80317c0 con 0x7f41e00b5fa0 2023-12-01T01:49:01.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.534+0000 7f41de59c640 1 -- 172.21.15.115:0/2170350158 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d8031920 con 0x7f41e00b5fa0 2023-12-01T01:49:01.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.534+0000 7f41de59c640 1 -- 172.21.15.115:0/2170350158 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8031cb0 con 0x7f41e00b5fa0 2023-12-01T01:49:01.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.535+0000 7f41e514e640 1 -- 172.21.15.115:0/2170350158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00b5fa0 msgr2=0x7f41e0161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.535+0000 7f41e514e640 1 --2- 172.21.15.115:0/2170350158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00b5fa0 0x7f41e0161410 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009870 tx=0x7f41d802b860 comp rx=0 tx=0).stop 2023-12-01T01:49:01.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.535+0000 7f41e514e640 1 -- 172.21.15.115:0/2170350158 shutdown_connections 2023-12-01T01:49:01.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.535+0000 7f41e514e640 1 --2- 172.21.15.115:0/2170350158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00b5fa0 0x7f41e0161410 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.535+0000 7f41e514e640 1 --2- 172.21.15.115:0/2170350158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e00cfd60 0x7f41e00cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.535+0000 7f41e514e640 1 -- 172.21.15.115:0/2170350158 >> 172.21.15.115:0/2170350158 conn(0x7f41e0065960 msgr2=0x7f41e0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:01.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.535+0000 7f41e514e640 1 -- 172.21.15.115:0/2170350158 shutdown_connections 2023-12-01T01:49:01.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.535+0000 7f41e514e640 1 -- 172.21.15.115:0/2170350158 wait complete. 2023-12-01T01:49:01.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.537+0000 7f41e514e640 1 Processor -- start 2023-12-01T01:49:01.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.537+0000 7f41e514e640 1 -- start start 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41e514e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e00b5fa0 0x7f41e0155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41e514e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00cfd60 0x7f41e015d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41e514e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015cd90 0x7f41e00b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41e514e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e01f23c0 con 0x7f41e00cfd60 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41e514e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e015e3f0 con 0x7f41e00b5fa0 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41e514e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e015e6d0 con 0x7f41e015cd90 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00cfd60 0x7f41e015d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00cfd60 0x7f41e015d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55678/0 (socket says 172.21.15.115:55678) 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41d7fff640 1 -- 172.21.15.115:0/310988502 learned_addr learned my addr 172.21.15.115:0/310988502 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41ded9d640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e00b5fa0 0x7f41e0155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.538+0000 7f41df59e640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015cd90 0x7f41e00b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41d7fff640 1 -- 172.21.15.115:0/310988502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015cd90 msgr2=0x7f41e00b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41d7fff640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015cd90 0x7f41e00b72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41d7fff640 1 -- 172.21.15.115:0/310988502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e00b5fa0 msgr2=0x7f41e0155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41d7fff640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e00b5fa0 0x7f41e0155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41d7fff640 1 -- 172.21.15.115:0/310988502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80094d0 con 0x7f41e00cfd60 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41df59e640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015cd90 0x7f41e00b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41ded9d640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e00b5fa0 0x7f41e0155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41d7fff640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00cfd60 0x7f41e015d9a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c7f0 tx=0x7f41c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41d77fe640 1 -- 172.21.15.115:0/310988502 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8004220 con 0x7f41e00cfd60 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41d77fe640 1 -- 172.21.15.115:0/310988502 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c80043a0 con 0x7f41e00cfd60 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41d77fe640 1 -- 172.21.15.115:0/310988502 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8007860 con 0x7f41e00cfd60 2023-12-01T01:49:01.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41e514e640 1 -- 172.21.15.115:0/310988502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e015e9b0 con 0x7f41e00cfd60 2023-12-01T01:49:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.539+0000 7f41e514e640 1 -- 172.21.15.115:0/310988502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e015d530 con 0x7f41e00cfd60 2023-12-01T01:49:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.540+0000 7f41e514e640 1 -- 172.21.15.115:0/310988502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e004e890 con 0x7f41e00cfd60 2023-12-01T01:49:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.541+0000 7f41d77fe640 1 -- 172.21.15.115:0/310988502 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f41c8007bd0 con 0x7f41e00cfd60 2023-12-01T01:49:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.543+0000 7f41d77fe640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f41c006a550 0x7f41c006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.543+0000 7f41d77fe640 1 -- 172.21.15.115:0/310988502 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5994+0+0 (secure 0 0 0) 0x7f41c8088d40 con 0x7f41e00cfd60 2023-12-01T01:49:01.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.543+0000 7f41ded9d640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f41c006a550 0x7f41c006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.544+0000 7f41ded9d640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f41c006a550 0x7f41c006ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f41d0004520 tx=0x7f41d0009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:01.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.547+0000 7f41d77fe640 1 -- 172.21.15.115:0/310988502 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f41c8053f50 con 0x7f41e00cfd60 2023-12-01T01:49:01.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.735+0000 7f41e514e640 1 -- 172.21.15.115:0/310988502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f41e00ccdc0 con 0x7f41e00cfd60 2023-12-01T01:49:01.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.740+0000 7f41d77fe640 1 -- 172.21.15.115:0/310988502 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v15) v1 ==== 115+0+0 (secure 0 0 0) 0x7f41c801e020 con 0x7f41e00cfd60 2023-12-01T01:49:01.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.743+0000 7f41e514e640 1 -- 172.21.15.115:0/310988502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f41c006a550 msgr2=0x7f41c006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f41c006a550 0x7f41c006ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f41d0004520 tx=0x7f41d0009040 comp rx=0 tx=0).stop 2023-12-01T01:49:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 -- 172.21.15.115:0/310988502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00cfd60 msgr2=0x7f41e015d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00cfd60 0x7f41e015d9a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c7f0 tx=0x7f41c800ccb0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 -- 172.21.15.115:0/310988502 shutdown_connections 2023-12-01T01:49:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f41c006a550 0x7f41c006ca10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41e015cd90 0x7f41e00b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41e00cfd60 0x7f41e015d9a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 --2- 172.21.15.115:0/310988502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41e00b5fa0 0x7f41e0155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 -- 172.21.15.115:0/310988502 >> 172.21.15.115:0/310988502 conn(0x7f41e0065960 msgr2=0x7f41e00b86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 -- 172.21.15.115:0/310988502 shutdown_connections 2023-12-01T01:49:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.744+0000 7f41e514e640 1 -- 172.21.15.115:0/310988502 wait complete. 2023-12-01T01:49:01.764 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-12-01T01:49:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.976+0000 7f61a922c640 1 Processor -- start 2023-12-01T01:49:01.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.976+0000 7f61a922c640 1 -- start start 2023-12-01T01:49:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.977+0000 7f61a922c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f61a415e2f0 0x7f61a40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.977+0000 7f61a922c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a40add10 con 0x7f61a40adf10 2023-12-01T01:49:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.977+0000 7f61a922c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a40c5760 con 0x7f61a406f2f0 2023-12-01T01:49:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.977+0000 7f61a922c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a40c9cf0 con 0x7f61a415e2f0 2023-12-01T01:49:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.978+0000 7f61a359e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f61a40adf10 0x7f61a40c95e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43414/0 (socket says 172.21.15.115:43414) 2023-12-01T01:49:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.978+0000 7f61a359e640 1 -- 172.21.15.115:0/1800844540 learned_addr learned my addr 172.21.15.115:0/1800844540 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:01.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.978+0000 7f61a2d9d640 1 --2- 172.21.15.115:0/1800844540 >> v2:172.21.15.115:3300/0 conn(0x7f61a415e2f0 0x7f61a40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.978+0000 7f61a2d9d640 1 -- 172.21.15.115:0/1800844540 >> v1:172.21.15.142:6789/0 conn(0x7f61a406f2f0 legacy=0x7f61a40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.978+0000 7f61a2d9d640 1 -- 172.21.15.115:0/1800844540 >> v1:172.21.15.115:6789/0 conn(0x7f61a40adf10 legacy=0x7f61a40c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.978+0000 7f61a2d9d640 1 -- 172.21.15.115:0/1800844540 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a415e6d0 con 0x7f61a415e2f0 2023-12-01T01:49:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.979+0000 7f61a2d9d640 1 --2- 172.21.15.115:0/1800844540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a415e2f0 0x7f61a40ca660 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f61940097a0 tx=0x7f619402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb058036e03601ed server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:01.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.979+0000 7f61a259c640 1 -- 172.21.15.115:0/1800844540 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619402f050 con 0x7f61a415e2f0 2023-12-01T01:49:01.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.979+0000 7f61a259c640 1 -- 172.21.15.115:0/1800844540 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6194031700 con 0x7f61a415e2f0 2023-12-01T01:49:01.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.979+0000 7f61a259c640 1 -- 172.21.15.115:0/1800844540 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194031a90 con 0x7f61a415e2f0 2023-12-01T01:49:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.980+0000 7f61a922c640 1 -- 172.21.15.115:0/1800844540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a415e2f0 msgr2=0x7f61a40ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.980+0000 7f61a922c640 1 --2- 172.21.15.115:0/1800844540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a415e2f0 0x7f61a40ca660 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f61940097a0 tx=0x7f619402b7a0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.980+0000 7f61a922c640 1 -- 172.21.15.115:0/1800844540 shutdown_connections 2023-12-01T01:49:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.980+0000 7f61a922c640 1 --2- 172.21.15.115:0/1800844540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a415e2f0 0x7f61a40ca660 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.980+0000 7f61a922c640 1 -- 172.21.15.115:0/1800844540 >> 172.21.15.115:0/1800844540 conn(0x7f61a40ce880 msgr2=0x7f61a40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.980+0000 7f61a922c640 1 -- 172.21.15.115:0/1800844540 shutdown_connections 2023-12-01T01:49:01.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.980+0000 7f61a922c640 1 -- 172.21.15.115:0/1800844540 wait complete. 2023-12-01T01:49:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.982+0000 7f61a922c640 1 Processor -- start 2023-12-01T01:49:01.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.982+0000 7f61a922c640 1 -- start start 2023-12-01T01:49:01.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.983+0000 7f61a922c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61a406f2f0 0x7f61a415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.983+0000 7f61a922c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a40adf10 0x7f61a4158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.983+0000 7f61a922c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61a415e2f0 0x7f61a41f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.983+0000 7f61a922c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a41f0200 con 0x7f61a40adf10 2023-12-01T01:49:01.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.983+0000 7f61a922c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c3b70 con 0x7f61a406f2f0 2023-12-01T01:49:01.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.983+0000 7f61a922c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c2f60 con 0x7f61a415e2f0 2023-12-01T01:49:01.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.983+0000 7f619bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a40adf10 0x7f61a4158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.983+0000 7f619bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a40adf10 0x7f61a4158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55704/0 (socket says 172.21.15.115:55704) 2023-12-01T01:49:01.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.983+0000 7f619bfff640 1 -- 172.21.15.115:0/2417738776 learned_addr learned my addr 172.21.15.115:0/2417738776 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:01.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.984+0000 7f619bfff640 1 -- 172.21.15.115:0/2417738776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61a415e2f0 msgr2=0x7f61a41f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.984+0000 7f619bfff640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61a415e2f0 0x7f61a41f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.984+0000 7f619bfff640 1 -- 172.21.15.115:0/2417738776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61a406f2f0 msgr2=0x7f61a415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:01.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.984+0000 7f619bfff640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61a406f2f0 0x7f61a415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:01.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.984+0000 7f619bfff640 1 -- 172.21.15.115:0/2417738776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6194009400 con 0x7f61a40adf10 2023-12-01T01:49:01.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.984+0000 7f619bfff640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a40adf10 0x7f61a4158a00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b420 tx=0x7f618c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:01.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.984+0000 7f619b7fe640 1 -- 172.21.15.115:0/2417738776 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c011830 con 0x7f61a40adf10 2023-12-01T01:49:01.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.984+0000 7f619b7fe640 1 -- 172.21.15.115:0/2417738776 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c011990 con 0x7f61a40adf10 2023-12-01T01:49:01.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.984+0000 7f619b7fe640 1 -- 172.21.15.115:0/2417738776 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c011d40 con 0x7f61a40adf10 2023-12-01T01:49:01.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.984+0000 7f61a922c640 1 -- 172.21.15.115:0/2417738776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a40c3240 con 0x7f61a40adf10 2023-12-01T01:49:01.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.985+0000 7f61a922c640 1 -- 172.21.15.115:0/2417738776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a40c3800 con 0x7f61a40adf10 2023-12-01T01:49:01.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.986+0000 7f61a922c640 1 -- 172.21.15.115:0/2417738776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a406fda0 con 0x7f61a40adf10 2023-12-01T01:49:01.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.987+0000 7f619b7fe640 1 -- 172.21.15.115:0/2417738776 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f618c01b460 con 0x7f61a40adf10 2023-12-01T01:49:01.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.987+0000 7f619b7fe640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f618406a550 0x7f618406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:01.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.988+0000 7f619b7fe640 1 -- 172.21.15.115:0/2417738776 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5994+0+0 (secure 0 0 0) 0x7f618c088770 con 0x7f61a40adf10 2023-12-01T01:49:01.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.988+0000 7f61a2d9d640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f618406a550 0x7f618406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:01.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.989+0000 7f61a2d9d640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f618406a550 0x7f618406ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6194035000 tx=0x7f6194030000 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:01.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:01.992+0000 7f619b7fe640 1 -- 172.21.15.115:0/2417738776 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f618c053b20 con 0x7f61a40adf10 2023-12-01T01:49:02.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.251+0000 7f61a922c640 1 -- 172.21.15.115:0/2417738776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f61a404e890 con 0x7f61a40adf10 2023-12-01T01:49:02.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.540+0000 7f619b7fe640 1 -- 172.21.15.115:0/2417738776 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v474) v1 ==== 158+0+0 (secure 0 0 0) 0x7f618c057140 con 0x7f61a40adf10 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 -- 172.21.15.115:0/2417738776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f618406a550 msgr2=0x7f618406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f618406a550 0x7f618406ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6194035000 tx=0x7f6194030000 comp rx=0 tx=0).stop 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 -- 172.21.15.115:0/2417738776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a40adf10 msgr2=0x7f61a4158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a40adf10 0x7f61a4158a00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b420 tx=0x7f618c00b8e0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 -- 172.21.15.115:0/2417738776 shutdown_connections 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f618406a550 0x7f618406ca10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61a415e2f0 0x7f61a41f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61a40adf10 0x7f61a4158a00 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 --2- 172.21.15.115:0/2417738776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61a406f2f0 0x7f61a415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 -- 172.21.15.115:0/2417738776 >> 172.21.15.115:0/2417738776 conn(0x7f61a40ce880 msgr2=0x7f61a40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 -- 172.21.15.115:0/2417738776 shutdown_connections 2023-12-01T01:49:02.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.544+0000 7f61a922c640 1 -- 172.21.15.115:0/2417738776 wait complete. 2023-12-01T01:49:02.564 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-12-01T01:49:02.565 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-12-01T01:49:02.565 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-01T01:49:02.565 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-12-01T01:49:02.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.775+0000 7fa8e2404640 1 Processor -- start 2023-12-01T01:49:02.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.775+0000 7fa8e2404640 1 -- start start 2023-12-01T01:49:02.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.777+0000 7fa8e2404640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa8dc15cd20 0x7fa8dc0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:02.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.777+0000 7fa8e2404640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc0b5d80 0x7fa8dc0b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:02.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.777+0000 7fa8e2404640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc0ce2e0 con 0x7fa8dc0d1230 2023-12-01T01:49:02.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.777+0000 7fa8e2404640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc0b2160 con 0x7fa8dc15cd20 2023-12-01T01:49:02.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.777+0000 7fa8e2404640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0d2a10 con 0x7fa8dc0b5d80 2023-12-01T01:49:02.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.777+0000 7fa8e09a1640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa8dc0d1230 0x7fa8dc0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43420/0 (socket says 172.21.15.115:43420) 2023-12-01T01:49:02.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.777+0000 7fa8e09a1640 1 -- 172.21.15.115:0/3096029515 learned_addr learned my addr 172.21.15.115:0/3096029515 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8dbfff640 1 --2- 172.21.15.115:0/3096029515 >> v2:172.21.15.142:3300/0 conn(0x7fa8dc15cd20 0x7fa8dc0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8d3fff640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc0b5d80 0x7fa8dc0b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789851732 0 0) 0x7fa8dc0ce2e0 con 0x7fa8dc0d1230 2023-12-01T01:49:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8c8003520 con 0x7fa8dc0d1230 2023-12-01T01:49:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799678625 0 0) 0x7fa8c8003520 con 0x7fa8dc0d1230 2023-12-01T01:49:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc0b5d80 msgr2=0x7fa8dc0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc0b5d80 0x7fa8dc0b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 >> v2:172.21.15.142:3300/0 conn(0x7fa8dc15cd20 msgr2=0x7fa8dc0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 --2- 172.21.15.115:0/3096029515 >> v2:172.21.15.142:3300/0 conn(0x7fa8dc15cd20 0x7fa8dc0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0ce000 con 0x7fa8dc0d1230 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa8d40019f0 con 0x7fa8dc0d1230 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 >> v1:172.21.15.115:6789/0 conn(0x7fa8dc0d1230 legacy=0x7fa8dc0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc0b5d80 0x7fa8c8003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c8003f90 0x7fa8c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8004d00 0x7fa8c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0d2a10 con 0x7fa8c8004d00 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c800b9e0 con 0x7fa8c8003f90 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.778+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c800bba0 con 0x7fa8dc0b5d80 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.779+0000 7fa8e09a1640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8004d00 0x7fa8c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.779+0000 7fa8dbfff640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc0b5d80 0x7fa8c8003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.779+0000 7fa8d3fff640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c8003f90 0x7fa8c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.779+0000 7fa8e09a1640 1 -- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc0b5d80 msgr2=0x7fa8c8003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.779+0000 7fa8e09a1640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc0b5d80 0x7fa8c8003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.779+0000 7fa8e09a1640 1 -- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c8003f90 msgr2=0x7fa8c80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:02.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.779+0000 7fa8e09a1640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c8003f90 0x7fa8c80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.779+0000 7fa8e09a1640 1 -- 172.21.15.115:0/3096029515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8003520 con 0x7fa8c8004d00 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.780+0000 7fa8e09a1640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8004d00 0x7fa8c8005180 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400a830 tx=0x7fa8c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32651123c8fd1766 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.781+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40072d0 con 0x7fa8c8004d00 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.781+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d402f070 con 0x7fa8c8004d00 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.781+0000 7fa8db7fe640 1 -- 172.21.15.115:0/3096029515 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40076f0 con 0x7fa8c8004d00 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.781+0000 7fa8e2404640 1 -- 172.21.15.115:0/3096029515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8004d00 msgr2=0x7fa8c8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.781+0000 7fa8e2404640 1 --2- 172.21.15.115:0/3096029515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c8004d00 0x7fa8c8005180 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400a830 tx=0x7fa8c80057f0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.781+0000 7fa8e09a1640 1 -- 172.21.15.115:0/3096029515 reap_dead start 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.781+0000 7fa8e2404640 1 -- 172.21.15.115:0/3096029515 shutdown_connections 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.781+0000 7fa8e2404640 1 -- 172.21.15.115:0/3096029515 >> 172.21.15.115:0/3096029515 conn(0x7fa8dc06a0e0 msgr2=0x7fa8dc06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.781+0000 7fa8e2404640 1 -- 172.21.15.115:0/3096029515 shutdown_connections 2023-12-01T01:49:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.781+0000 7fa8e2404640 1 -- 172.21.15.115:0/3096029515 wait complete. 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.783+0000 7fa8e2404640 1 Processor -- start 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.783+0000 7fa8e2404640 1 -- start start 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.784+0000 7fa8e2404640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c8004d00 0x7fa8dc15db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.784+0000 7fa8e2404640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc1585a0 0x7fa8dc158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.784+0000 7fa8e2404640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8dc1f2200 0x7fa8dc1f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.784+0000 7fa8e2404640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc158f60 con 0x7fa8dc1f2200 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.784+0000 7fa8e2404640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1f54d0 con 0x7fa8c8004d00 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.784+0000 7fa8e2404640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1f5740 con 0x7fa8dc1585a0 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8e09a1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8dc1f2200 0x7fa8dc1f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8e09a1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8dc1f2200 0x7fa8dc1f26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55728/0 (socket says 172.21.15.115:55728) 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8e09a1640 1 -- 172.21.15.115:0/396181036 learned_addr learned my addr 172.21.15.115:0/396181036 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8dbfff640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c8004d00 0x7fa8dc15db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8e09a1640 1 -- 172.21.15.115:0/396181036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc1585a0 msgr2=0x7fa8dc158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:02.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8d3fff640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc1585a0 0x7fa8dc158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:02.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8e09a1640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc1585a0 0x7fa8dc158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8e09a1640 1 -- 172.21.15.115:0/396181036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c8004d00 msgr2=0x7fa8dc15db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:02.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8e09a1640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c8004d00 0x7fa8dc15db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:02.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8e09a1640 1 -- 172.21.15.115:0/396181036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc1f5a20 con 0x7fa8dc1f2200 2023-12-01T01:49:02.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8dbfff640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c8004d00 0x7fa8dc15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:02.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8d3fff640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc1585a0 0x7fa8dc158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:02.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.785+0000 7fa8e09a1640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8dc1f2200 0x7fa8dc1f26b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa8d402cb70 tx=0x7fa8d4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:02.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.786+0000 7fa8d97fa640 1 -- 172.21.15.115:0/396181036 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4044070 con 0x7fa8dc1f2200 2023-12-01T01:49:02.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.786+0000 7fa8d97fa640 1 -- 172.21.15.115:0/396181036 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d4001960 con 0x7fa8dc1f2200 2023-12-01T01:49:02.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.786+0000 7fa8d97fa640 1 -- 172.21.15.115:0/396181036 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4007520 con 0x7fa8dc1f2200 2023-12-01T01:49:02.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.786+0000 7fa8e2404640 1 -- 172.21.15.115:0/396181036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc1f5d00 con 0x7fa8dc1f2200 2023-12-01T01:49:02.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.786+0000 7fa8e2404640 1 -- 172.21.15.115:0/396181036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc15e060 con 0x7fa8dc1f2200 2023-12-01T01:49:02.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.787+0000 7fa8e2404640 1 -- 172.21.15.115:0/396181036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc1f6220 con 0x7fa8dc1f2200 2023-12-01T01:49:02.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.788+0000 7fa8d97fa640 1 -- 172.21.15.115:0/396181036 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa8d4031030 con 0x7fa8dc1f2200 2023-12-01T01:49:02.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.789+0000 7fa8d97fa640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8bc06a550 0x7fa8bc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:02.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.789+0000 7fa8d97fa640 1 -- 172.21.15.115:0/396181036 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5994+0+0 (secure 0 0 0) 0x7fa8d40aa260 con 0x7fa8dc1f2200 2023-12-01T01:49:02.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.789+0000 7fa8dbfff640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8bc06a550 0x7fa8bc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:02.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.790+0000 7fa8dbfff640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8bc06a550 0x7fa8bc06ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc151910 tx=0x7fa8cc008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:02.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:02.793+0000 7fa8d97fa640 1 -- 172.21.15.115:0/396181036 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa8dc1f6220 con 0x7fa8dc1f2200 2023-12-01T01:49:03.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.051+0000 7fa8e2404640 1 -- 172.21.15.115:0/396181036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fa8dc1f6220 con 0x7fa8dc1f2200 2023-12-01T01:49:03.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.543+0000 7fa8d97fa640 1 -- 172.21.15.115:0/396181036 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v475) v1 ==== 157+0+0 (secure 0 0 0) 0x7fa8dc1f6220 con 0x7fa8dc1f2200 2023-12-01T01:49:03.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.547+0000 7fa8e2404640 1 -- 172.21.15.115:0/396181036 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8bc06a550 msgr2=0x7fa8bc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:03.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.547+0000 7fa8e2404640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8bc06a550 0x7fa8bc06ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc151910 tx=0x7fa8cc008040 comp rx=0 tx=0).stop 2023-12-01T01:49:03.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.547+0000 7fa8e2404640 1 -- 172.21.15.115:0/396181036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8dc1f2200 msgr2=0x7fa8dc1f26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:03.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.547+0000 7fa8e2404640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8dc1f2200 0x7fa8dc1f26b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa8d402cb70 tx=0x7fa8d4001d80 comp rx=0 tx=0).stop 2023-12-01T01:49:03.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.547+0000 7fa8e2404640 1 -- 172.21.15.115:0/396181036 shutdown_connections 2023-12-01T01:49:03.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.547+0000 7fa8e2404640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8bc06a550 0x7fa8bc06ca10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.547+0000 7fa8e2404640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8dc1585a0 0x7fa8dc158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.547+0000 7fa8e2404640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8dc1f2200 0x7fa8dc1f26b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.547+0000 7fa8e2404640 1 --2- 172.21.15.115:0/396181036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8c8004d00 0x7fa8dc15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.547+0000 7fa8e2404640 1 -- 172.21.15.115:0/396181036 >> 172.21.15.115:0/396181036 conn(0x7fa8dc06a0e0 msgr2=0x7fa8dc156d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:03.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.548+0000 7fa8e2404640 1 -- 172.21.15.115:0/396181036 shutdown_connections 2023-12-01T01:49:03.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.548+0000 7fa8e2404640 1 -- 172.21.15.115:0/396181036 wait complete. 2023-12-01T01:49:03.570 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T01:49:03.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.782+0000 7fc79f5fb640 1 Processor -- start 2023-12-01T01:49:03.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.782+0000 7fc79f5fb640 1 -- start start 2023-12-01T01:49:03.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79f5fb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc79815cfa0 0x7fc7980b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:03.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79f5fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7980b7e70 0x7fc7980b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:03.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79f5fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc7980b6210 0x7fc7980cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:03.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79f5fb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980b6890 con 0x7fc7980b6210 2023-12-01T01:49:03.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79f5fb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980b6b70 con 0x7fc79815cfa0 2023-12-01T01:49:03.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79f5fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980b6e50 con 0x7fc7980b7e70 2023-12-01T01:49:03.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79db98640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc7980b6210 0x7fc7980cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:03.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79d397640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc79815cfa0 0x7fc7980b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:03.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79d397640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc79815cfa0 0x7fc7980b7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:36838/0 (socket says 172.21.15.115:36838) 2023-12-01T01:49:03.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79d397640 1 -- 172.21.15.115:0/4054505766 learned_addr learned my addr 172.21.15.115:0/4054505766 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:03.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.784+0000 7fc79db98640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc7980b6210 0x7fc7980cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55732/0 (socket says 172.21.15.115:55732) 2023-12-01T01:49:03.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.785+0000 7fc79cb96640 1 --2- 172.21.15.115:0/4054505766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7980b7e70 0x7fc7980b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:03.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.785+0000 7fc79db98640 1 -- 172.21.15.115:0/4054505766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7980b7e70 msgr2=0x7fc7980b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:03.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.785+0000 7fc79db98640 1 --2- 172.21.15.115:0/4054505766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7980b7e70 0x7fc7980b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.785+0000 7fc79db98640 1 -- 172.21.15.115:0/4054505766 >> v2:172.21.15.142:3300/0 conn(0x7fc79815cfa0 msgr2=0x7fc7980b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:03.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.785+0000 7fc79db98640 1 --2- 172.21.15.115:0/4054505766 >> v2:172.21.15.142:3300/0 conn(0x7fc79815cfa0 0x7fc7980b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.785+0000 7fc79db98640 1 -- 172.21.15.115:0/4054505766 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980b7130 con 0x7fc7980b6210 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.786+0000 7fc79db98640 1 --2- 172.21.15.115:0/4054505766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b6210 0x7fc7980cfc10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc7980b55d0 tx=0x7fc79002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a3bf7f3658c13 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.786+0000 7fc78ffff640 1 -- 172.21.15.115:0/4054505766 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79002cb40 con 0x7fc7980b6210 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.786+0000 7fc78ffff640 1 -- 172.21.15.115:0/4054505766 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79002cca0 con 0x7fc7980b6210 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.786+0000 7fc78ffff640 1 -- 172.21.15.115:0/4054505766 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790034660 con 0x7fc7980b6210 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.787+0000 7fc79f5fb640 1 -- 172.21.15.115:0/4054505766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b6210 msgr2=0x7fc7980cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.787+0000 7fc79f5fb640 1 --2- 172.21.15.115:0/4054505766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b6210 0x7fc7980cfc10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc7980b55d0 tx=0x7fc79002bd90 comp rx=0 tx=0).stop 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.787+0000 7fc79f5fb640 1 -- 172.21.15.115:0/4054505766 shutdown_connections 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.787+0000 7fc79f5fb640 1 --2- 172.21.15.115:0/4054505766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b6210 0x7fc7980cfc10 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.787+0000 7fc79f5fb640 1 --2- 172.21.15.115:0/4054505766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7980b7e70 0x7fc7980b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.787+0000 7fc79f5fb640 1 --2- 172.21.15.115:0/4054505766 >> v2:172.21.15.142:3300/0 conn(0x7fc79815cfa0 0x7fc7980b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.787+0000 7fc79f5fb640 1 -- 172.21.15.115:0/4054505766 >> 172.21.15.115:0/4054505766 conn(0x7fc79806a0b0 msgr2=0x7fc79806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:03.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.787+0000 7fc79f5fb640 1 -- 172.21.15.115:0/4054505766 shutdown_connections 2023-12-01T01:49:03.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.787+0000 7fc79f5fb640 1 -- 172.21.15.115:0/4054505766 wait complete. 2023-12-01T01:49:03.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.789+0000 7fc79f5fb640 1 Processor -- start 2023-12-01T01:49:03.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.790+0000 7fc79f5fb640 1 -- start start 2023-12-01T01:49:03.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79f5fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7980b6210 0x7fc7980c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:03.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79f5fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b7e70 0x7fc798157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:03.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79f5fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79815cfa0 0x7fc79815d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:03.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79f5fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798144c00 con 0x7fc7980b7e70 2023-12-01T01:49:03.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79f5fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798143f80 con 0x7fc7980b6210 2023-12-01T01:49:03.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79f5fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc798144260 con 0x7fc79815cfa0 2023-12-01T01:49:03.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79cb96640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b7e70 0x7fc798157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:03.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79cb96640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b7e70 0x7fc798157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55736/0 (socket says 172.21.15.115:55736) 2023-12-01T01:49:03.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79cb96640 1 -- 172.21.15.115:0/1272998534 learned_addr learned my addr 172.21.15.115:0/1272998534 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:03.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79db98640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79815cfa0 0x7fc79815d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:03.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79d397640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7980b6210 0x7fc7980c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:03.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79cb96640 1 -- 172.21.15.115:0/1272998534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79815cfa0 msgr2=0x7fc79815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.791+0000 7fc79cb96640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79815cfa0 0x7fc79815d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc79cb96640 1 -- 172.21.15.115:0/1272998534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7980b6210 msgr2=0x7fc7980c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc79cb96640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7980b6210 0x7fc7980c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc79cb96640 1 -- 172.21.15.115:0/1272998534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc790009a70 con 0x7fc7980b7e70 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc79d397640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7980b6210 0x7fc7980c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc79db98640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79815cfa0 0x7fc79815d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc79cb96640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b7e70 0x7fc798157940 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc78800d800 tx=0x7fc78800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc78dffb640 1 -- 172.21.15.115:0/1272998534 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7880096a0 con 0x7fc7980b7e70 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc78dffb640 1 -- 172.21.15.115:0/1272998534 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788009800 con 0x7fc7980b7e70 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc79f5fb640 1 -- 172.21.15.115:0/1272998534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7980b8c00 con 0x7fc7980b7e70 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc78dffb640 1 -- 172.21.15.115:0/1272998534 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788009990 con 0x7fc7980b7e70 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.792+0000 7fc79f5fb640 1 -- 172.21.15.115:0/1272998534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc798144820 con 0x7fc7980b7e70 2023-12-01T01:49:03.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.793+0000 7fc79f5fb640 1 -- 172.21.15.115:0/1272998534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79804e890 con 0x7fc7980b7e70 2023-12-01T01:49:03.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.794+0000 7fc78dffb640 1 -- 172.21.15.115:0/1272998534 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc788009af0 con 0x7fc7980b7e70 2023-12-01T01:49:03.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.795+0000 7fc78dffb640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77c06a550 0x7fc77c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:03.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.795+0000 7fc78dffb640 1 -- 172.21.15.115:0/1272998534 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5994+0+0 (secure 0 0 0) 0x7fc7880890c0 con 0x7fc7980b7e70 2023-12-01T01:49:03.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.796+0000 7fc79d397640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77c06a550 0x7fc77c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:03.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.796+0000 7fc79d397640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77c06a550 0x7fc77c06ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc794009ac0 tx=0x7fc794008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:03.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.800+0000 7fc78dffb640 1 -- 172.21.15.115:0/1272998534 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc7880543e0 con 0x7fc7980b7e70 2023-12-01T01:49:03.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.991+0000 7fc79f5fb640 1 -- 172.21.15.115:0/1272998534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc798160c60 con 0x7fc7980b7e70 2023-12-01T01:49:03.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:03.997+0000 7fc78dffb640 1 -- 172.21.15.115:0/1272998534 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc78800b3a0 con 0x7fc7980b7e70 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 -- 172.21.15.115:0/1272998534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77c06a550 msgr2=0x7fc77c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77c06a550 0x7fc77c06ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc794009ac0 tx=0x7fc794008040 comp rx=0 tx=0).stop 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 -- 172.21.15.115:0/1272998534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b7e70 msgr2=0x7fc798157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b7e70 0x7fc798157940 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc78800d800 tx=0x7fc78800dcc0 comp rx=0 tx=0).stop 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 -- 172.21.15.115:0/1272998534 shutdown_connections 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77c06a550 0x7fc77c06ca10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79815cfa0 0x7fc79815d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7980b7e70 0x7fc798157940 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 --2- 172.21.15.115:0/1272998534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7980b6210 0x7fc7980c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 -- 172.21.15.115:0/1272998534 >> 172.21.15.115:0/1272998534 conn(0x7fc79806a0b0 msgr2=0x7fc7980b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 -- 172.21.15.115:0/1272998534 shutdown_connections 2023-12-01T01:49:04.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.000+0000 7fc79f5fb640 1 -- 172.21.15.115:0/1272998534 wait complete. 2023-12-01T01:49:04.019 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:49:04.019 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:49:04.129 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:49:04.250 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:49:04.344 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:49:04.344 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:49:04.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:49:04.405 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:49:04.405 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:49:04.464 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:49:04.465 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:49:04.465 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:49:04.524 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:49:04.525 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:49:04.525 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:49:04.584 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:49:04.584 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T01:49:04.584 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:49:04.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.804+0000 7f6b770f8640 1 Processor -- start 2023-12-01T01:49:04.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.804+0000 7f6b770f8640 1 -- start start 2023-12-01T01:49:04.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b770f8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6b700ccc80 0x7f6b7006fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:04.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b770f8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7015d600 con 0x7f6b7015d220 2023-12-01T01:49:04.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b770f8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7006c4b0 con 0x7f6b700ad2b0 2023-12-01T01:49:04.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b770f8640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7006cc10 con 0x7f6b700ccc80 2023-12-01T01:49:04.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b74e94640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6b7015d220 0x7f6b700cc570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43432/0 (socket says 172.21.15.115:43432) 2023-12-01T01:49:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b74e94640 1 -- 172.21.15.115:0/3143654411 learned_addr learned my addr 172.21.15.115:0/3143654411 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b6ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6b700ccc80 0x7f6b7006fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b6ffff640 1 -- 172.21.15.115:0/3143654411 >> v1:172.21.15.142:6789/0 conn(0x7f6b700ad2b0 legacy=0x7f6b7006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b6ffff640 1 -- 172.21.15.115:0/3143654411 >> v1:172.21.15.115:6789/0 conn(0x7f6b7015d220 legacy=0x7f6b700cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b6ffff640 1 -- 172.21.15.115:0/3143654411 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7006f2f0 con 0x7f6b700ccc80 2023-12-01T01:49:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.806+0000 7f6b6f7fe640 1 -- 172.21.15.115:0/3143654411 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568351920 0 0) 0x7f6b64001110 con 0x7f6b7015d220 2023-12-01T01:49:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.808+0000 7f6b6ffff640 1 --2- 172.21.15.115:0/3143654411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b700ccc80 0x7f6b7006fab0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0097a0 tx=0x7f6b5c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65535629ef301652 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.808+0000 7f6b6f7fe640 1 -- 172.21.15.115:0/3143654411 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c02e070 con 0x7f6b700ccc80 2023-12-01T01:49:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.808+0000 7f6b6f7fe640 1 -- 172.21.15.115:0/3143654411 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c0315a0 con 0x7f6b700ccc80 2023-12-01T01:49:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.808+0000 7f6b6f7fe640 1 -- 172.21.15.115:0/3143654411 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c031930 con 0x7f6b700ccc80 2023-12-01T01:49:04.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.808+0000 7f6b770f8640 1 -- 172.21.15.115:0/3143654411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b700ccc80 msgr2=0x7f6b7006fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:04.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.808+0000 7f6b770f8640 1 --2- 172.21.15.115:0/3143654411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b700ccc80 0x7f6b7006fab0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0097a0 tx=0x7f6b5c02b840 comp rx=0 tx=0).stop 2023-12-01T01:49:04.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.809+0000 7f6b770f8640 1 -- 172.21.15.115:0/3143654411 shutdown_connections 2023-12-01T01:49:04.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.809+0000 7f6b770f8640 1 --2- 172.21.15.115:0/3143654411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b700ccc80 0x7f6b7006fab0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:04.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.809+0000 7f6b770f8640 1 -- 172.21.15.115:0/3143654411 >> 172.21.15.115:0/3143654411 conn(0x7f6b700ce880 msgr2=0x7f6b700cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:04.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.809+0000 7f6b770f8640 1 -- 172.21.15.115:0/3143654411 shutdown_connections 2023-12-01T01:49:04.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.809+0000 7f6b770f8640 1 -- 172.21.15.115:0/3143654411 wait complete. 2023-12-01T01:49:04.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.810+0000 7f6b770f8640 1 Processor -- start 2023-12-01T01:49:04.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.810+0000 7f6b770f8640 1 -- start start 2023-12-01T01:49:04.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.811+0000 7f6b770f8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b700ad2b0 0x7f6b700ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:04.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.811+0000 7f6b770f8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b700ccc80 0x7f6b700ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:04.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.811+0000 7f6b770f8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b7015d220 0x7f6b700b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:04.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.811+0000 7f6b770f8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700ca7f0 con 0x7f6b7015d220 2023-12-01T01:49:04.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.811+0000 7f6b770f8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700c6ec0 con 0x7f6b700ccc80 2023-12-01T01:49:04.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.811+0000 7f6b770f8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700c71a0 con 0x7f6b700ad2b0 2023-12-01T01:49:04.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.812+0000 7f6b75695640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b7015d220 0x7f6b700b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:04.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.812+0000 7f6b75695640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b7015d220 0x7f6b700b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55760/0 (socket says 172.21.15.115:55760) 2023-12-01T01:49:04.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.812+0000 7f6b75695640 1 -- 172.21.15.115:0/1727124800 learned_addr learned my addr 172.21.15.115:0/1727124800 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:04.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.812+0000 7f6b75695640 1 -- 172.21.15.115:0/1727124800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b700ad2b0 msgr2=0x7f6b700ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:04.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.812+0000 7f6b75695640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b700ad2b0 0x7f6b700ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:04.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.812+0000 7f6b75695640 1 -- 172.21.15.115:0/1727124800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b700ccc80 msgr2=0x7f6b700ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:04.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.812+0000 7f6b75695640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b700ccc80 0x7f6b700ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:04.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.812+0000 7f6b75695640 1 -- 172.21.15.115:0/1727124800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c009400 con 0x7f6b7015d220 2023-12-01T01:49:04.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.812+0000 7f6b75695640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b7015d220 0x7f6b700b14e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800b990 tx=0x7f6b6800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:04.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.813+0000 7f6b6d7fa640 1 -- 172.21.15.115:0/1727124800 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6800cba0 con 0x7f6b7015d220 2023-12-01T01:49:04.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.813+0000 7f6b6d7fa640 1 -- 172.21.15.115:0/1727124800 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6800cd00 con 0x7f6b7015d220 2023-12-01T01:49:04.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.813+0000 7f6b6d7fa640 1 -- 172.21.15.115:0/1727124800 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68014680 con 0x7f6b7015d220 2023-12-01T01:49:04.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.813+0000 7f6b770f8640 1 -- 172.21.15.115:0/1727124800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b700c7480 con 0x7f6b7015d220 2023-12-01T01:49:04.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.813+0000 7f6b770f8640 1 -- 172.21.15.115:0/1727124800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b700c7a40 con 0x7f6b7015d220 2023-12-01T01:49:04.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.814+0000 7f6b770f8640 1 -- 172.21.15.115:0/1727124800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b700c7c30 con 0x7f6b7015d220 2023-12-01T01:49:04.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.815+0000 7f6b6d7fa640 1 -- 172.21.15.115:0/1727124800 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6b680147e0 con 0x7f6b7015d220 2023-12-01T01:49:04.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.816+0000 7f6b6d7fa640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6b5406a550 0x7f6b5406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:04.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.816+0000 7f6b6d7fa640 1 -- 172.21.15.115:0/1727124800 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5994+0+0 (secure 0 0 0) 0x7f6b68088ad0 con 0x7f6b7015d220 2023-12-01T01:49:04.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.816+0000 7f6b74e94640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6b5406a550 0x7f6b5406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:04.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.817+0000 7f6b74e94640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6b5406a550 0x7f6b5406ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b64007fd0 tx=0x7f6b64007960 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:04.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:04.820+0000 7f6b6d7fa640 1 -- 172.21.15.115:0/1727124800 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6b68053f20 con 0x7f6b7015d220 2023-12-01T01:49:05.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.049+0000 7f6b770f8640 1 -- 172.21.15.115:0/1727124800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b7004e890 con 0x7f6b7015d220 2023-12-01T01:49:05.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.050+0000 7f6b6d7fa640 1 -- 172.21.15.115:0/1727124800 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f6b68019090 con 0x7f6b7015d220 2023-12-01T01:49:05.051 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:05.051 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1226717603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1226717603},{"type":"v1","addr":"172.21.15.142:6834","nonce":1226717603}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2864659456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2864659456},{"type":"v1","addr":"172.21.15.142:6835","nonce":2864659456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4774,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/253656345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":253656345},{"type":"v1","addr":"172.21.15.115:6837","nonce":253656345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5853,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2810248749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2810248749},{"type":"v1","addr":"172.21.15.115:6835","nonce":2810248749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-12-01T01:49:05.052 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 18 2023-12-01T01:49:05.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 -- 172.21.15.115:0/1727124800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6b5406a550 msgr2=0x7f6b5406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:05.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6b5406a550 0x7f6b5406ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b64007fd0 tx=0x7f6b64007960 comp rx=0 tx=0).stop 2023-12-01T01:49:05.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 -- 172.21.15.115:0/1727124800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b7015d220 msgr2=0x7f6b700b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:05.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b7015d220 0x7f6b700b14e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800b990 tx=0x7f6b6800be50 comp rx=0 tx=0).stop 2023-12-01T01:49:05.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 -- 172.21.15.115:0/1727124800 shutdown_connections 2023-12-01T01:49:05.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6b5406a550 0x7f6b5406ca10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:05.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b7015d220 0x7f6b700b14e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:05.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b700ccc80 0x7f6b700ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:05.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 --2- 172.21.15.115:0/1727124800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b700ad2b0 0x7f6b700ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:05.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 -- 172.21.15.115:0/1727124800 >> 172.21.15.115:0/1727124800 conn(0x7f6b700ce880 msgr2=0x7f6b700c43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:05.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 -- 172.21.15.115:0/1727124800 shutdown_connections 2023-12-01T01:49:05.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.052+0000 7f6b770f8640 1 -- 172.21.15.115:0/1727124800 wait complete. 2023-12-01T01:49:05.073 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T01:49:05.073 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T01:49:05.074 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T01:49:05.074 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:49:05.074 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T01:49:05.074 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T01:49:05.074 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T01:49:05.075 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:49:05.075 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T01:49:05.075 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T01:49:05.075 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T01:49:05.075 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:49:05.075 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T01:49:05.075 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T01:49:05.076 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T01:49:05.076 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:49:05.093 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T01:49:05.093 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T01:49:05.094 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T01:49:05.094 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T01:49:05.096 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T01:49:05.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T01:49:05.106 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T01:49:05.106 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T01:49:05.139 INFO:tasks.ceph.mds.a:Started 2023-12-01T01:49:05.139 DEBUG:teuthology.parallel:result is None 2023-12-01T01:49:05.145 INFO:tasks.ceph.mds.c:Started 2023-12-01T01:49:05.145 DEBUG:teuthology.parallel:result is None 2023-12-01T01:49:05.153 INFO:tasks.ceph.mds.d:Started 2023-12-01T01:49:05.153 DEBUG:teuthology.parallel:result is None 2023-12-01T01:49:05.161 INFO:tasks.ceph.mds.b:Started 2023-12-01T01:49:05.161 DEBUG:teuthology.parallel:result is None 2023-12-01T01:49:05.161 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T01:49:05.298 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:49:05.298 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T01:49:05.320 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T01:49:05.321 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T01:49:05.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.349+0000 7fad85807640 1 Processor -- start 2023-12-01T01:49:05.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.349+0000 7fad85807640 1 -- start start 2023-12-01T01:49:05.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.350+0000 7fad85807640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006b9b0 0x7fad8006bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:05.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.350+0000 7fad85807640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fad800c52c0 0x7fad800ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:05.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.350+0000 7fad85807640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad800c9c80 con 0x7fad80159700 2023-12-01T01:49:05.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.350+0000 7fad85807640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad800c8b00 con 0x7fad800c52c0 2023-12-01T01:49:05.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.350+0000 7fad85807640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad800b2130 con 0x7fad8006b9b0 2023-12-01T01:49:05.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.351+0000 7fad84805640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006b9b0 0x7fad8006bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:05.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.351+0000 7fad77fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fad800c52c0 0x7fad800ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:05.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.351+0000 7fad84805640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006b9b0 0x7fad8006bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:58664/0 (socket says 172.21.15.115:58664) 2023-12-01T01:49:05.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.351+0000 7fad84805640 1 -- 172.21.15.115:0/3015372019 learned_addr learned my addr 172.21.15.115:0/3015372019 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:05.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.351+0000 7fad85006640 1 --1- 172.21.15.115:0/3015372019 >> v1:172.21.15.142:6789/0 conn(0x7fad80159700 0x7fad8015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:49340/0 (socket says 172.21.15.115:49340) 2023-12-01T01:49:05.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.351+0000 7fad84805640 1 -- 172.21.15.115:0/3015372019 >> v2:172.21.15.142:3300/0 conn(0x7fad800c52c0 msgr2=0x7fad800ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:05.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.351+0000 7fad84805640 1 --2- 172.21.15.115:0/3015372019 >> v2:172.21.15.142:3300/0 conn(0x7fad800c52c0 0x7fad800ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:05.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.351+0000 7fad84805640 1 -- 172.21.15.115:0/3015372019 >> v1:172.21.15.142:6789/0 conn(0x7fad80159700 legacy=0x7fad8015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:05.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.351+0000 7fad84805640 1 -- 172.21.15.115:0/3015372019 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8006c2d0 con 0x7fad8006b9b0 2023-12-01T01:49:05.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.353+0000 7fad84805640 1 --2- 172.21.15.115:0/3015372019 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006b9b0 0x7fad8006bd90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009870 tx=0x7fad7c02ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43a30446f850dd80 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:05.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.353+0000 7fad777fe640 1 -- 172.21.15.115:0/3015372019 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c02f830 con 0x7fad8006b9b0 2023-12-01T01:49:05.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.353+0000 7fad777fe640 1 -- 172.21.15.115:0/3015372019 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c02f990 con 0x7fad8006b9b0 2023-12-01T01:49:05.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.353+0000 7fad777fe640 1 -- 172.21.15.115:0/3015372019 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c02fd20 con 0x7fad8006b9b0 2023-12-01T01:49:05.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.353+0000 7fad85807640 1 -- 172.21.15.115:0/3015372019 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006b9b0 msgr2=0x7fad8006bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:05.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.353+0000 7fad85807640 1 --2- 172.21.15.115:0/3015372019 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006b9b0 0x7fad8006bd90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009870 tx=0x7fad7c02ba10 comp rx=0 tx=0).stop 2023-12-01T01:49:05.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.353+0000 7fad85807640 1 -- 172.21.15.115:0/3015372019 shutdown_connections 2023-12-01T01:49:05.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.353+0000 7fad85807640 1 --2- 172.21.15.115:0/3015372019 >> v2:172.21.15.142:3300/0 conn(0x7fad800c52c0 0x7fad800ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:05.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.353+0000 7fad85807640 1 --2- 172.21.15.115:0/3015372019 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006b9b0 0x7fad8006bd90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:05.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.353+0000 7fad85807640 1 -- 172.21.15.115:0/3015372019 >> 172.21.15.115:0/3015372019 conn(0x7fad800ce620 msgr2=0x7fad800cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:05.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.354+0000 7fad85807640 1 -- 172.21.15.115:0/3015372019 shutdown_connections 2023-12-01T01:49:05.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.354+0000 7fad85807640 1 -- 172.21.15.115:0/3015372019 wait complete. 2023-12-01T01:49:05.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.355+0000 7fad85807640 1 Processor -- start 2023-12-01T01:49:05.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.355+0000 7fad85807640 1 -- start start 2023-12-01T01:49:05.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.356+0000 7fad85807640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad8006b9b0 0x7fad80142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:05.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.356+0000 7fad85807640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad800c52c0 0x7fad800c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:05.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.356+0000 7fad85807640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80159700 0x7fad800b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:05.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.356+0000 7fad85807640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad800c2150 con 0x7fad8006b9b0 2023-12-01T01:49:05.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.356+0000 7fad85807640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad800c2430 con 0x7fad80159700 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.356+0000 7fad85807640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad800c2710 con 0x7fad800c52c0 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.356+0000 7fad84805640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad8006b9b0 0x7fad80142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.357+0000 7fad84805640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad8006b9b0 0x7fad80142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55788/0 (socket says 172.21.15.115:55788) 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.357+0000 7fad84805640 1 -- 172.21.15.115:0/3575872950 learned_addr learned my addr 172.21.15.115:0/3575872950 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.357+0000 7fad85006640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80159700 0x7fad800b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.357+0000 7fad77fff640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad800c52c0 0x7fad800c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.357+0000 7fad84805640 1 -- 172.21.15.115:0/3575872950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad800c52c0 msgr2=0x7fad800c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.357+0000 7fad84805640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad800c52c0 0x7fad800c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.357+0000 7fad84805640 1 -- 172.21.15.115:0/3575872950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80159700 msgr2=0x7fad800b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.357+0000 7fad84805640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80159700 0x7fad800b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:05.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.357+0000 7fad84805640 1 -- 172.21.15.115:0/3575872950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c0094d0 con 0x7fad8006b9b0 2023-12-01T01:49:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.357+0000 7fad84805640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad8006b9b0 0x7fad80142110 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009870 tx=0x7fad7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.358+0000 7fad757fa640 1 -- 172.21.15.115:0/3575872950 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c037c40 con 0x7fad8006b9b0 2023-12-01T01:49:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.358+0000 7fad757fa640 1 -- 172.21.15.115:0/3575872950 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c037dc0 con 0x7fad8006b9b0 2023-12-01T01:49:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.358+0000 7fad85807640 1 -- 172.21.15.115:0/3575872950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad800c29f0 con 0x7fad8006b9b0 2023-12-01T01:49:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.358+0000 7fad757fa640 1 -- 172.21.15.115:0/3575872950 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c041640 con 0x7fad8006b9b0 2023-12-01T01:49:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.358+0000 7fad85807640 1 -- 172.21.15.115:0/3575872950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad800cc600 con 0x7fad8006b9b0 2023-12-01T01:49:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.359+0000 7fad85807640 1 -- 172.21.15.115:0/3575872950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad80067850 con 0x7fad8006b9b0 2023-12-01T01:49:05.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.360+0000 7fad757fa640 1 -- 172.21.15.115:0/3575872950 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fad7c0417a0 con 0x7fad8006b9b0 2023-12-01T01:49:05.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.362+0000 7fad757fa640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad6c06a550 0x7fad6c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:05.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.362+0000 7fad757fa640 1 -- 172.21.15.115:0/3575872950 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fad7c03c070 con 0x7fad8006b9b0 2023-12-01T01:49:05.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.362+0000 7fad77fff640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad6c06a550 0x7fad6c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:05.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.363+0000 7fad77fff640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad6c06a550 0x7fad6c06ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fad8015c6c0 tx=0x7fad70009190 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:05.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.365+0000 7fad757fa640 1 -- 172.21.15.115:0/3575872950 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fad7c077750 con 0x7fad8006b9b0 2023-12-01T01:49:05.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:05.578+0000 7fad85807640 1 -- 172.21.15.115:0/3575872950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fad800673a0 con 0x7fad8006b9b0 2023-12-01T01:49:06.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.574+0000 7fad757fa640 1 -- 172.21.15.115:0/3575872950 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7fad7c07ad70 con 0x7fad8006b9b0 2023-12-01T01:49:06.576 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 -- 172.21.15.115:0/3575872950 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad6c06a550 msgr2=0x7fad6c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad6c06a550 0x7fad6c06ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fad8015c6c0 tx=0x7fad70009190 comp rx=0 tx=0).stop 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 -- 172.21.15.115:0/3575872950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad8006b9b0 msgr2=0x7fad80142110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad8006b9b0 0x7fad80142110 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009870 tx=0x7fad7c004730 comp rx=0 tx=0).stop 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 -- 172.21.15.115:0/3575872950 shutdown_connections 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad6c06a550 0x7fad6c06ca10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad800c52c0 0x7fad800c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80159700 0x7fad800b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 --2- 172.21.15.115:0/3575872950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad8006b9b0 0x7fad80142110 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 -- 172.21.15.115:0/3575872950 >> 172.21.15.115:0/3575872950 conn(0x7fad800ce620 msgr2=0x7fad8014e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 -- 172.21.15.115:0/3575872950 shutdown_connections 2023-12-01T01:49:06.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.578+0000 7fad85807640 1 -- 172.21.15.115:0/3575872950 wait complete. 2023-12-01T01:49:06.600 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T01:49:06.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.815+0000 7f78e1269640 1 Processor -- start 2023-12-01T01:49:06.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.815+0000 7f78e1269640 1 -- start start 2023-12-01T01:49:06.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.816+0000 7f78e1269640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 0x7f78dc06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:06.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.816+0000 7f78e1269640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f78dc06b6f0 0x7f78dc070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:06.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.816+0000 7f78e1269640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78dc06f8b0 con 0x7f78dc159dd0 2023-12-01T01:49:06.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.816+0000 7f78e1269640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78dc0adcb0 con 0x7f78dc06b6f0 2023-12-01T01:49:06.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.816+0000 7f78e1269640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0ad9d0 con 0x7f78dc0c9ef0 2023-12-01T01:49:06.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.817+0000 7f78db7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f78dc06b6f0 0x7f78dc070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:06.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.817+0000 7f78db7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f78dc06b6f0 0x7f78dc070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55804/0 (socket says 172.21.15.115:55804) 2023-12-01T01:49:06.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.817+0000 7f78db7fe640 1 -- 172.21.15.115:0/1272629683 learned_addr learned my addr 172.21.15.115:0/1272629683 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:06.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.817+0000 7f78da7fc640 1 --2- 172.21.15.115:0/1272629683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 0x7f78dc06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:06.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.817+0000 7f78db7fe640 1 -- 172.21.15.115:0/1272629683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 msgr2=0x7f78dc06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:06.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.817+0000 7f78db7fe640 1 --2- 172.21.15.115:0/1272629683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 0x7f78dc06fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:06.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.817+0000 7f78db7fe640 1 -- 172.21.15.115:0/1272629683 >> v1:172.21.15.142:6789/0 conn(0x7f78dc159dd0 legacy=0x7f78dc0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:06.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.817+0000 7f78db7fe640 1 -- 172.21.15.115:0/1272629683 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc0c9000 con 0x7f78dc06b6f0 2023-12-01T01:49:06.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.817+0000 7f78da7fc640 1 --2- 172.21.15.115:0/1272629683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 0x7f78dc06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:06.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.818+0000 7f78db7fe640 1 --2- 172.21.15.115:0/1272629683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 0x7f78dc070260 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f78d400d150 tx=0x7f78d402dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a303bcb80931d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:06.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78d9ffb640 1 -- 172.21.15.115:0/1272629683 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d402e950 con 0x7f78dc06b6f0 2023-12-01T01:49:06.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78d9ffb640 1 -- 172.21.15.115:0/1272629683 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d402eab0 con 0x7f78dc06b6f0 2023-12-01T01:49:06.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78d9ffb640 1 -- 172.21.15.115:0/1272629683 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d402ee40 con 0x7f78dc06b6f0 2023-12-01T01:49:06.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78e1269640 1 -- 172.21.15.115:0/1272629683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 msgr2=0x7f78dc070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:06.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78e1269640 1 --2- 172.21.15.115:0/1272629683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 0x7f78dc070260 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f78d400d150 tx=0x7f78d402dfb0 comp rx=0 tx=0).stop 2023-12-01T01:49:06.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78e1269640 1 -- 172.21.15.115:0/1272629683 shutdown_connections 2023-12-01T01:49:06.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78e1269640 1 --2- 172.21.15.115:0/1272629683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 0x7f78dc070260 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:06.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78e1269640 1 --2- 172.21.15.115:0/1272629683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 0x7f78dc06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:06.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78e1269640 1 -- 172.21.15.115:0/1272629683 >> 172.21.15.115:0/1272629683 conn(0x7f78dc0ce880 msgr2=0x7f78dc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:06.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78e1269640 1 -- 172.21.15.115:0/1272629683 shutdown_connections 2023-12-01T01:49:06.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.819+0000 7f78e1269640 1 -- 172.21.15.115:0/1272629683 wait complete. 2023-12-01T01:49:06.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.821+0000 7f78e1269640 1 Processor -- start 2023-12-01T01:49:06.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.821+0000 7f78e1269640 1 -- start start 2023-12-01T01:49:06.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.822+0000 7f78e1269640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 0x7f78dc157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:06.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.822+0000 7f78e1269640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 0x7f78dc15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:06.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.822+0000 7f78e1269640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78dc159dd0 0x7f78dc15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:06.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.822+0000 7f78e1269640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0acfa0 con 0x7f78dc06b6f0 2023-12-01T01:49:06.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.822+0000 7f78e1269640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc15d040 con 0x7f78dc159dd0 2023-12-01T01:49:06.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.822+0000 7f78e1269640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc15d280 con 0x7f78dc0c9ef0 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.822+0000 7f78daffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 0x7f78dc157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.822+0000 7f78daffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 0x7f78dc157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55808/0 (socket says 172.21.15.115:55808) 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.822+0000 7f78daffd640 1 -- 172.21.15.115:0/267460655 learned_addr learned my addr 172.21.15.115:0/267460655 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.823+0000 7f78da7fc640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 0x7f78dc15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.823+0000 7f78daffd640 1 -- 172.21.15.115:0/267460655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 msgr2=0x7f78dc15c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.823+0000 7f78db7fe640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78dc159dd0 0x7f78dc15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.823+0000 7f78daffd640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 0x7f78dc15c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.823+0000 7f78daffd640 1 -- 172.21.15.115:0/267460655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78dc159dd0 msgr2=0x7f78dc15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.823+0000 7f78daffd640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78dc159dd0 0x7f78dc15cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.823+0000 7f78daffd640 1 -- 172.21.15.115:0/267460655 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d400bd40 con 0x7f78dc06b6f0 2023-12-01T01:49:06.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.823+0000 7f78da7fc640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 0x7f78dc15c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:06.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.823+0000 7f78daffd640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 0x7f78dc157a50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00b3f0 tx=0x7f78cc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:06.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.824+0000 7f78c37fe640 1 -- 172.21.15.115:0/267460655 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc011830 con 0x7f78dc06b6f0 2023-12-01T01:49:06.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.824+0000 7f78c37fe640 1 -- 172.21.15.115:0/267460655 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc011990 con 0x7f78dc06b6f0 2023-12-01T01:49:06.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.824+0000 7f78c37fe640 1 -- 172.21.15.115:0/267460655 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc011d40 con 0x7f78dc06b6f0 2023-12-01T01:49:06.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.824+0000 7f78e1269640 1 -- 172.21.15.115:0/267460655 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78dc1f2530 con 0x7f78dc06b6f0 2023-12-01T01:49:06.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.824+0000 7f78e1269640 1 -- 172.21.15.115:0/267460655 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc1f2af0 con 0x7f78dc06b6f0 2023-12-01T01:49:06.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.825+0000 7f78e1269640 1 -- 172.21.15.115:0/267460655 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78dc0738e0 con 0x7f78dc06b6f0 2023-12-01T01:49:06.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.826+0000 7f78c37fe640 1 -- 172.21.15.115:0/267460655 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f78cc01a400 con 0x7f78dc06b6f0 2023-12-01T01:49:06.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.826+0000 7f78c37fe640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78c806a580 0x7f78c806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:06.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.827+0000 7f78c37fe640 1 -- 172.21.15.115:0/267460655 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f78cc0884b0 con 0x7f78dc06b6f0 2023-12-01T01:49:06.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.827+0000 7f78da7fc640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78c806a580 0x7f78c806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:06.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.827+0000 7f78da7fc640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78c806a580 0x7f78c806ca40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f78d0006fd0 tx=0x7f78d0008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:06.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:06.831+0000 7f78c37fe640 1 -- 172.21.15.115:0/267460655 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f78cc053d70 con 0x7f78dc06b6f0 2023-12-01T01:49:07.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.089+0000 7f78e1269640 1 -- 172.21.15.115:0/267460655 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f78dc0c92b0 con 0x7f78dc06b6f0 2023-12-01T01:49:07.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.090+0000 7f78c37fe640 1 -- 172.21.15.115:0/267460655 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f78cc057390 con 0x7f78dc06b6f0 2023-12-01T01:49:07.091 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:07.091 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T01:49:07.092 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.093 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.095 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:49:07.097 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T01:49:07.098 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.100 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T01:49:07.102 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T01:49:07.104 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:49:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.092+0000 7f78e1269640 1 -- 172.21.15.115:0/267460655 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78c806a580 msgr2=0x7f78c806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.092+0000 7f78e1269640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78c806a580 0x7f78c806ca40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f78d0006fd0 tx=0x7f78d0008040 comp rx=0 tx=0).stop 2023-12-01T01:49:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.092+0000 7f78e1269640 1 -- 172.21.15.115:0/267460655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 msgr2=0x7f78dc157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.092+0000 7f78e1269640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 0x7f78dc157a50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00b3f0 tx=0x7f78cc00b8b0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.093+0000 7f78e1269640 1 -- 172.21.15.115:0/267460655 shutdown_connections 2023-12-01T01:49:07.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.093+0000 7f78e1269640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78c806a580 0x7f78c806ca40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.093+0000 7f78e1269640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78dc0c9ef0 0x7f78dc15c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.093+0000 7f78e1269640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78dc159dd0 0x7f78dc15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.093+0000 7f78e1269640 1 --2- 172.21.15.115:0/267460655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78dc06b6f0 0x7f78dc157a50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.093+0000 7f78e1269640 1 -- 172.21.15.115:0/267460655 >> 172.21.15.115:0/267460655 conn(0x7f78dc0ce880 msgr2=0x7f78dc070680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:07.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.093+0000 7f78e1269640 1 -- 172.21.15.115:0/267460655 shutdown_connections 2023-12-01T01:49:07.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.093+0000 7f78e1269640 1 -- 172.21.15.115:0/267460655 wait complete. 2023-12-01T01:49:07.113 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:49:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.321+0000 7f5780f16640 1 Processor -- start 2023-12-01T01:49:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.321+0000 7f5780f16640 1 -- start start 2023-12-01T01:49:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.323+0000 7f5780f16640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f577c06f2f0 0x7f577c06bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.323+0000 7f5780f16640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577c0add10 con 0x7f577c15e2f0 2023-12-01T01:49:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.323+0000 7f5780f16640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577c0c5760 con 0x7f577c0cad70 2023-12-01T01:49:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.323+0000 7f5780f16640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c15e6d0 con 0x7f577c06f2f0 2023-12-01T01:49:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.323+0000 7f577a59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f577c15e2f0 0x7f577c0ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43456/0 (socket says 172.21.15.115:43456) 2023-12-01T01:49:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.323+0000 7f577a59c640 1 -- 172.21.15.115:0/356963203 learned_addr learned my addr 172.21.15.115:0/356963203 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577ad9d640 1 --2- 172.21.15.115:0/356963203 >> v2:172.21.15.142:3300/0 conn(0x7f577c06f2f0 0x7f577c06bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162532617 0 0) 0x7f577c0add10 con 0x7f577c15e2f0 2023-12-01T01:49:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5768003520 con 0x7f577c15e2f0 2023-12-01T01:49:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1134943162 0 0) 0x7f5768003520 con 0x7f577c15e2f0 2023-12-01T01:49:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 >> v2:172.21.15.142:3300/0 conn(0x7f577c06f2f0 msgr2=0x7f577c06bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 --2- 172.21.15.115:0/356963203 >> v2:172.21.15.142:3300/0 conn(0x7f577c06f2f0 0x7f577c06bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 >> v1:172.21.15.142:6789/0 conn(0x7f577c0cad70 legacy=0x7f577c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0cf7b0 con 0x7f577c15e2f0 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f576c001690 con 0x7f577c15e2f0 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 >> v1:172.21.15.115:6789/0 conn(0x7f577c15e2f0 legacy=0x7f577c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 --2- 172.21.15.115:0/356963203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5768003a60 0x7f5768003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 --2- 172.21.15.115:0/356963203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57680047c0 0x7f5768004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 --2- 172.21.15.115:0/356963203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5768005380 0x7f576800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c15e6d0 con 0x7f57680047c0 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576800c240 con 0x7f5768003a60 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576800c400 con 0x7f5768005380 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.324+0000 7f5779d9b640 1 --2- 172.21.15.115:0/356963203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57680047c0 0x7f5768004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.325+0000 7f577a59c640 1 --2- 172.21.15.115:0/356963203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5768003a60 0x7f5768003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.325+0000 7f5779d9b640 1 -- 172.21.15.115:0/356963203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5768005380 msgr2=0x7f576800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:07.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.325+0000 7f5779d9b640 1 --2- 172.21.15.115:0/356963203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5768005380 0x7f576800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.325+0000 7f5779d9b640 1 -- 172.21.15.115:0/356963203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5768003a60 msgr2=0x7f5768003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.325+0000 7f5779d9b640 1 --2- 172.21.15.115:0/356963203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5768003a60 0x7f5768003e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.325+0000 7f5779d9b640 1 -- 172.21.15.115:0/356963203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5768003520 con 0x7f57680047c0 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.325+0000 7f577ad9d640 1 -- 172.21.15.115:0/356963203 reap_dead start 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.326+0000 7f5779d9b640 1 --2- 172.21.15.115:0/356963203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57680047c0 0x7f5768004c40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5764001750 tx=0x7f57680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0b7f8e079a518e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.326+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57640052d0 con 0x7f57680047c0 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.326+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576402e050 con 0x7f57680047c0 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.326+0000 7f577959a640 1 -- 172.21.15.115:0/356963203 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764003830 con 0x7f57680047c0 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.327+0000 7f5780f16640 1 -- 172.21.15.115:0/356963203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57680047c0 msgr2=0x7f5768004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.327+0000 7f5780f16640 1 --2- 172.21.15.115:0/356963203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57680047c0 0x7f5768004c40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5764001750 tx=0x7f57680057f0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.327+0000 7f5780f16640 1 -- 172.21.15.115:0/356963203 shutdown_connections 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.327+0000 7f5780f16640 1 --2- 172.21.15.115:0/356963203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57680047c0 0x7f5768004c40 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.327+0000 7f5780f16640 1 -- 172.21.15.115:0/356963203 >> 172.21.15.115:0/356963203 conn(0x7f577c0ce880 msgr2=0x7f577c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.327+0000 7f5780f16640 1 -- 172.21.15.115:0/356963203 shutdown_connections 2023-12-01T01:49:07.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.327+0000 7f5780f16640 1 -- 172.21.15.115:0/356963203 wait complete. 2023-12-01T01:49:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.329+0000 7f5780f16640 1 Processor -- start 2023-12-01T01:49:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.329+0000 7f5780f16640 1 -- start start 2023-12-01T01:49:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f5780f16640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57680047c0 0x7f577c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f5780f16640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c0c48c0 0x7f577c0c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f5780f16640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f577c1f08f0 0x7f577c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f5780f16640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0c5bf0 con 0x7f577c1f08f0 2023-12-01T01:49:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f5780f16640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c1f1950 con 0x7f577c0c48c0 2023-12-01T01:49:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f5780f16640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c0c28f0 con 0x7f57680047c0 2023-12-01T01:49:07.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f577ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f577c1f08f0 0x7f577c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f577ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f577c1f08f0 0x7f577c0c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55820/0 (socket says 172.21.15.115:55820) 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f577ad9d640 1 -- 172.21.15.115:0/1080990469 learned_addr learned my addr 172.21.15.115:0/1080990469 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f577a59c640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57680047c0 0x7f577c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f5779d9b640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c0c48c0 0x7f577c0c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f577ad9d640 1 -- 172.21.15.115:0/1080990469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57680047c0 msgr2=0x7f577c0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f577ad9d640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57680047c0 0x7f577c0acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f577ad9d640 1 -- 172.21.15.115:0/1080990469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c0c48c0 msgr2=0x7f577c0c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f577ad9d640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c0c48c0 0x7f577c0c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.330+0000 7f577ad9d640 1 -- 172.21.15.115:0/1080990469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0c4d40 con 0x7f577c1f08f0 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.331+0000 7f5779d9b640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c0c48c0 0x7f577c0c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.331+0000 7f577ad9d640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f577c1f08f0 0x7f577c0c7650 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f577c06ef30 tx=0x7f577400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.331+0000 7f5772ffd640 1 -- 172.21.15.115:0/1080990469 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774010040 con 0x7f577c1f08f0 2023-12-01T01:49:07.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.331+0000 7f5772ffd640 1 -- 172.21.15.115:0/1080990469 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577400bd10 con 0x7f577c1f08f0 2023-12-01T01:49:07.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.331+0000 7f5772ffd640 1 -- 172.21.15.115:0/1080990469 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774014640 con 0x7f577c1f08f0 2023-12-01T01:49:07.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.331+0000 7f5780f16640 1 -- 172.21.15.115:0/1080990469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c1f10b0 con 0x7f577c1f08f0 2023-12-01T01:49:07.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.331+0000 7f5780f16640 1 -- 172.21.15.115:0/1080990469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c1f1670 con 0x7f577c1f08f0 2023-12-01T01:49:07.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.332+0000 7f5780f16640 1 -- 172.21.15.115:0/1080990469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c06fda0 con 0x7f577c1f08f0 2023-12-01T01:49:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.333+0000 7f5772ffd640 1 -- 172.21.15.115:0/1080990469 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f57740147a0 con 0x7f577c1f08f0 2023-12-01T01:49:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.334+0000 7f5772ffd640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f575c06a550 0x7f575c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.334+0000 7f5772ffd640 1 -- 172.21.15.115:0/1080990469 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5774088ca0 con 0x7f577c1f08f0 2023-12-01T01:49:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.334+0000 7f577a59c640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f575c06a550 0x7f575c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.335+0000 7f577a59c640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f575c06a550 0x7f575c06ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f576c000fd0 tx=0x7f576c009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:07.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.338+0000 7f5772ffd640 1 -- 172.21.15.115:0/1080990469 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f57740543b0 con 0x7f577c1f08f0 2023-12-01T01:49:07.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.530+0000 7f5780f16640 1 -- 172.21.15.115:0/1080990469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f577c0c5950 con 0x7f577c1f08f0 2023-12-01T01:49:07.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.531+0000 7f5772ffd640 1 -- 172.21.15.115:0/1080990469 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+12437 (secure 0 0 0) 0x7f577c0c5950 con 0x7f577c1f08f0 2023-12-01T01:49:07.533 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:07.533 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":60,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:49:06.559334+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.534+0000 7f5780f16640 1 -- 172.21.15.115:0/1080990469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f575c06a550 msgr2=0x7f575c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.534+0000 7f5780f16640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f575c06a550 0x7f575c06ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f576c000fd0 tx=0x7f576c009040 comp rx=0 tx=0).stop 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.534+0000 7f5780f16640 1 -- 172.21.15.115:0/1080990469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f577c1f08f0 msgr2=0x7f577c0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.534+0000 7f5780f16640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f577c1f08f0 0x7f577c0c7650 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f577c06ef30 tx=0x7f577400b480 comp rx=0 tx=0).stop 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.534+0000 7f5780f16640 1 -- 172.21.15.115:0/1080990469 shutdown_connections 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.534+0000 7f5780f16640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f575c06a550 0x7f575c06ca10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.534+0000 7f5780f16640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f577c1f08f0 0x7f577c0c7650 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.534+0000 7f5780f16640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f577c0c48c0 0x7f577c0c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.534+0000 7f5780f16640 1 --2- 172.21.15.115:0/1080990469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57680047c0 0x7f577c0acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.534+0000 7f5780f16640 1 -- 172.21.15.115:0/1080990469 >> 172.21.15.115:0/1080990469 conn(0x7f577c0ce880 msgr2=0x7f577c0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.535+0000 7f5780f16640 1 -- 172.21.15.115:0/1080990469 shutdown_connections 2023-12-01T01:49:07.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.535+0000 7f5780f16640 1 -- 172.21.15.115:0/1080990469 wait complete. 2023-12-01T01:49:07.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:49:07.554 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:49:07.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.767+0000 7fbdd7327640 1 Processor -- start 2023-12-01T01:49:07.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.767+0000 7fbdd7327640 1 -- start start 2023-12-01T01:49:07.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd7327640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbdd0160200 0x7fbdd00b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd7327640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdd00b0f80 0x7fbdd006b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd7327640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbdd006b7a0 0x7fbdd00cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd7327640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd00b35f0 con 0x7fbdd006b7a0 2023-12-01T01:49:07.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd7327640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd00cb290 con 0x7fbdd0160200 2023-12-01T01:49:07.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd7327640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd00b0ae0 con 0x7fbdd00b0f80 2023-12-01T01:49:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd58c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbdd006b7a0 0x7fbdd00cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd58c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbdd006b7a0 0x7fbdd00cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48384/0 (socket says 172.21.15.115:48384) 2023-12-01T01:49:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd58c4640 1 -- 172.21.15.115:0/3864104780 learned_addr learned my addr 172.21.15.115:0/3864104780 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd50c3640 1 --2- 172.21.15.115:0/3864104780 >> v2:172.21.15.142:3300/0 conn(0x7fbdd0160200 0x7fbdd00b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.769+0000 7fbdd48c2640 1 --2- 172.21.15.115:0/3864104780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdd00b0f80 0x7fbdd006b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.770+0000 7fbdd58c4640 1 -- 172.21.15.115:0/3864104780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdd00b0f80 msgr2=0x7fbdd006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.770+0000 7fbdd58c4640 1 --2- 172.21.15.115:0/3864104780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdd00b0f80 0x7fbdd006b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.770+0000 7fbdd58c4640 1 -- 172.21.15.115:0/3864104780 >> v2:172.21.15.142:3300/0 conn(0x7fbdd0160200 msgr2=0x7fbdd00b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.770+0000 7fbdd58c4640 1 --2- 172.21.15.115:0/3864104780 >> v2:172.21.15.142:3300/0 conn(0x7fbdd0160200 0x7fbdd00b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.770+0000 7fbdd58c4640 1 -- 172.21.15.115:0/3864104780 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00b3310 con 0x7fbdd006b7a0 2023-12-01T01:49:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.770+0000 7fbdd48c2640 1 --2- 172.21.15.115:0/3864104780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdd00b0f80 0x7fbdd006b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.771+0000 7fbdd58c4640 1 --2- 172.21.15.115:0/3864104780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd006b7a0 0x7fbdd00cbd30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbdd006ce00 tx=0x7fbdc802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9d55f3be7a88c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.771+0000 7fbdc7fff640 1 -- 172.21.15.115:0/3864104780 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc802ca70 con 0x7fbdd006b7a0 2023-12-01T01:49:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.771+0000 7fbdc7fff640 1 -- 172.21.15.115:0/3864104780 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc802cbd0 con 0x7fbdd006b7a0 2023-12-01T01:49:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.771+0000 7fbdc7fff640 1 -- 172.21.15.115:0/3864104780 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80343f0 con 0x7fbdd006b7a0 2023-12-01T01:49:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.772+0000 7fbdd7327640 1 -- 172.21.15.115:0/3864104780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd006b7a0 msgr2=0x7fbdd00cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.772+0000 7fbdd7327640 1 --2- 172.21.15.115:0/3864104780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd006b7a0 0x7fbdd00cbd30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbdd006ce00 tx=0x7fbdc802bed0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.772+0000 7fbdd7327640 1 -- 172.21.15.115:0/3864104780 shutdown_connections 2023-12-01T01:49:07.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.772+0000 7fbdd7327640 1 --2- 172.21.15.115:0/3864104780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd006b7a0 0x7fbdd00cbd30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.772+0000 7fbdd7327640 1 --2- 172.21.15.115:0/3864104780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdd00b0f80 0x7fbdd006b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.772+0000 7fbdd7327640 1 --2- 172.21.15.115:0/3864104780 >> v2:172.21.15.142:3300/0 conn(0x7fbdd0160200 0x7fbdd00b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.772+0000 7fbdd7327640 1 -- 172.21.15.115:0/3864104780 >> 172.21.15.115:0/3864104780 conn(0x7fbdd0065960 msgr2=0x7fbdd0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:07.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.772+0000 7fbdd7327640 1 -- 172.21.15.115:0/3864104780 shutdown_connections 2023-12-01T01:49:07.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.772+0000 7fbdd7327640 1 -- 172.21.15.115:0/3864104780 wait complete. 2023-12-01T01:49:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.774+0000 7fbdd7327640 1 Processor -- start 2023-12-01T01:49:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.774+0000 7fbdd7327640 1 -- start start 2023-12-01T01:49:07.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.775+0000 7fbdd7327640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdd006b7a0 0x7fbdd01492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.775+0000 7fbdd7327640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd00b0f80 0x7fbdd01f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.775+0000 7fbdd7327640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdd0160200 0x7fbdd00c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.775+0000 7fbdd7327640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd01f34b0 con 0x7fbdd00b0f80 2023-12-01T01:49:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.775+0000 7fbdd7327640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd01f2830 con 0x7fbdd0160200 2023-12-01T01:49:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.775+0000 7fbdd7327640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd01f2b10 con 0x7fbdd006b7a0 2023-12-01T01:49:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.775+0000 7fbdd48c2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd00b0f80 0x7fbdd01f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.775+0000 7fbdd48c2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd00b0f80 0x7fbdd01f1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48394/0 (socket says 172.21.15.115:48394) 2023-12-01T01:49:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.775+0000 7fbdd48c2640 1 -- 172.21.15.115:0/2191404038 learned_addr learned my addr 172.21.15.115:0/2191404038 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:07.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdd48c2640 1 -- 172.21.15.115:0/2191404038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdd006b7a0 msgr2=0x7fbdd01492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdd48c2640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdd006b7a0 0x7fbdd01492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdd48c2640 1 -- 172.21.15.115:0/2191404038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdd0160200 msgr2=0x7fbdd00c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdd58c4640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdd0160200 0x7fbdd00c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdd48c2640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdd0160200 0x7fbdd00c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdd48c2640 1 -- 172.21.15.115:0/2191404038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8009b30 con 0x7fbdd00b0f80 2023-12-01T01:49:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdd58c4640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdd0160200 0x7fbdd00c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdd48c2640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd00b0f80 0x7fbdd01f1d90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c7f0 tx=0x7fbdc000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdc5ffb640 1 -- 172.21.15.115:0/2191404038 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0007a30 con 0x7fbdd00b0f80 2023-12-01T01:49:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdc5ffb640 1 -- 172.21.15.115:0/2191404038 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc0007b90 con 0x7fbdd00b0f80 2023-12-01T01:49:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdc5ffb640 1 -- 172.21.15.115:0/2191404038 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000ce40 con 0x7fbdd00b0f80 2023-12-01T01:49:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdd7327640 1 -- 172.21.15.115:0/2191404038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd01f2df0 con 0x7fbdd00b0f80 2023-12-01T01:49:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.776+0000 7fbdd7327640 1 -- 172.21.15.115:0/2191404038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd0149820 con 0x7fbdd00b0f80 2023-12-01T01:49:07.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.777+0000 7fbdd7327640 1 -- 172.21.15.115:0/2191404038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd01f35f0 con 0x7fbdd00b0f80 2023-12-01T01:49:07.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.778+0000 7fbdc5ffb640 1 -- 172.21.15.115:0/2191404038 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fbdc00113f0 con 0x7fbdd00b0f80 2023-12-01T01:49:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.780+0000 7fbdc5ffb640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbdb406a550 0x7fbdb406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.780+0000 7fbdc5ffb640 1 -- 172.21.15.115:0/2191404038 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbdc0088a00 con 0x7fbdd00b0f80 2023-12-01T01:49:07.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.780+0000 7fbdd50c3640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbdb406a550 0x7fbdb406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:07.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.781+0000 7fbdd50c3640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbdb406a550 0x7fbdb406ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009870 tx=0x7fbdcc008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:07.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.784+0000 7fbdc5ffb640 1 -- 172.21.15.115:0/2191404038 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbdd01f35f0 con 0x7fbdd00b0f80 2023-12-01T01:49:07.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.977+0000 7fbdd7327640 1 -- 172.21.15.115:0/2191404038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbdd01f35f0 con 0x7fbdd00b0f80 2023-12-01T01:49:07.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.978+0000 7fbdc5ffb640 1 -- 172.21.15.115:0/2191404038 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbdd01f35f0 con 0x7fbdd00b0f80 2023-12-01T01:49:07.979 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:49:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.980+0000 7fbdd7327640 1 -- 172.21.15.115:0/2191404038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbdb406a550 msgr2=0x7fbdb406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.980+0000 7fbdd7327640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbdb406a550 0x7fbdb406ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009870 tx=0x7fbdcc008040 comp rx=0 tx=0).stop 2023-12-01T01:49:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.980+0000 7fbdd7327640 1 -- 172.21.15.115:0/2191404038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd00b0f80 msgr2=0x7fbdd01f1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.980+0000 7fbdd7327640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd00b0f80 0x7fbdd01f1d90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c7f0 tx=0x7fbdc000cbf0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.980+0000 7fbdd7327640 1 -- 172.21.15.115:0/2191404038 shutdown_connections 2023-12-01T01:49:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.980+0000 7fbdd7327640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbdb406a550 0x7fbdb406ca10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.980+0000 7fbdd7327640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdd0160200 0x7fbdd00c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.980+0000 7fbdd7327640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdd00b0f80 0x7fbdd01f1d90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.980+0000 7fbdd7327640 1 --2- 172.21.15.115:0/2191404038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdd006b7a0 0x7fbdd01492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:07.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.980+0000 7fbdd7327640 1 -- 172.21.15.115:0/2191404038 >> 172.21.15.115:0/2191404038 conn(0x7fbdd0065960 msgr2=0x7fbdd00ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:07.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.981+0000 7fbdd7327640 1 -- 172.21.15.115:0/2191404038 shutdown_connections 2023-12-01T01:49:07.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:07.981+0000 7fbdd7327640 1 -- 172.21.15.115:0/2191404038 wait complete. 2023-12-01T01:49:08.000 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T01:49:08.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.204+0000 7fd7d33f0640 1 Processor -- start 2023-12-01T01:49:08.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.204+0000 7fd7d33f0640 1 -- start start 2023-12-01T01:49:08.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.205+0000 7fd7d33f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7cc15e2f0 0x7fd7cc0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.205+0000 7fd7d33f0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc0add10 con 0x7fd7cc0adf10 2023-12-01T01:49:08.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.205+0000 7fd7d33f0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc0c5760 con 0x7fd7cc06f2f0 2023-12-01T01:49:08.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.205+0000 7fd7d33f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0c9cf0 con 0x7fd7cc15e2f0 2023-12-01T01:49:08.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.206+0000 7fd7d198d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd7cc0adf10 0x7fd7cc0c95e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58428/0 (socket says 172.21.15.115:58428) 2023-12-01T01:49:08.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.206+0000 7fd7d198d640 1 -- 172.21.15.115:0/1124551928 learned_addr learned my addr 172.21.15.115:0/1124551928 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.206+0000 7fd7d118c640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7cc15e2f0 0x7fd7cc0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.206+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692721552 0 0) 0x7fd7cc0add10 con 0x7fd7cc0adf10 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.206+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7bc003520 con 0x7fd7cc0adf10 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3472448360 0 0) 0x7fd7bc003520 con 0x7fd7cc0adf10 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7cc15e2f0 msgr2=0x7fd7cc0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7cc15e2f0 0x7fd7cc0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 >> v1:172.21.15.142:6789/0 conn(0x7fd7cc06f2f0 legacy=0x7fd7cc0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc15e6d0 con 0x7fd7cc0adf10 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd7c4001670 con 0x7fd7cc0adf10 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 >> v1:172.21.15.115:6789/0 conn(0x7fd7cc0adf10 legacy=0x7fd7cc0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7cc15e2f0 0x7fd7bc003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc003f90 0x7fd7bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7bc004d00 0x7fd7bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0c9cf0 con 0x7fd7bc004d00 2023-12-01T01:49:08.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc00bc10 con 0x7fd7bc003f90 2023-12-01T01:49:08.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc00bdd0 con 0x7fd7cc15e2f0 2023-12-01T01:49:08.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7d198d640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7bc004d00 0x7fd7bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7d098b640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc003f90 0x7fd7bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7d198d640 1 -- 172.21.15.115:0/1124551928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7cc15e2f0 msgr2=0x7fd7bc003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:08.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7d198d640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7cc15e2f0 0x7fd7bc003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7d198d640 1 -- 172.21.15.115:0/1124551928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc003f90 msgr2=0x7fd7bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:08.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7d198d640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7bc003f90 0x7fd7bc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.207+0000 7fd7d198d640 1 -- 172.21.15.115:0/1124551928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc003520 con 0x7fd7bc004d00 2023-12-01T01:49:08.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.209+0000 7fd7d198d640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7bc004d00 0x7fd7bc005180 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c210 tx=0x7fd7bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7606fcb3b4f0d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:08.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.209+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4005ad0 con 0x7fd7bc004d00 2023-12-01T01:49:08.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.209+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c4005c30 con 0x7fd7bc004d00 2023-12-01T01:49:08.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.209+0000 7fd7cbfff640 1 -- 172.21.15.115:0/1124551928 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c402cad0 con 0x7fd7bc004d00 2023-12-01T01:49:08.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.209+0000 7fd7d33f0640 1 -- 172.21.15.115:0/1124551928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7bc004d00 msgr2=0x7fd7bc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.209+0000 7fd7d33f0640 1 --2- 172.21.15.115:0/1124551928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7bc004d00 0x7fd7bc005180 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c210 tx=0x7fd7bc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.209+0000 7fd7d198d640 1 -- 172.21.15.115:0/1124551928 reap_dead start 2023-12-01T01:49:08.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.210+0000 7fd7d33f0640 1 -- 172.21.15.115:0/1124551928 shutdown_connections 2023-12-01T01:49:08.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.210+0000 7fd7d33f0640 1 -- 172.21.15.115:0/1124551928 >> 172.21.15.115:0/1124551928 conn(0x7fd7cc0ce880 msgr2=0x7fd7cc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:08.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.210+0000 7fd7d33f0640 1 -- 172.21.15.115:0/1124551928 shutdown_connections 2023-12-01T01:49:08.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.210+0000 7fd7d33f0640 1 -- 172.21.15.115:0/1124551928 wait complete. 2023-12-01T01:49:08.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.211+0000 7fd7d33f0640 1 Processor -- start 2023-12-01T01:49:08.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.212+0000 7fd7d33f0640 1 -- start start 2023-12-01T01:49:08.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.212+0000 7fd7d33f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7bc004d00 0x7fd7cc14e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.212+0000 7fd7d33f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7cc0ca8d0 0x7fd7cc1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.212+0000 7fd7d33f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7cc1589f0 0x7fd7cc0b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.212+0000 7fd7d33f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc14ebe0 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.212+0000 7fd7d33f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc158e70 con 0x7fd7cc1589f0 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.212+0000 7fd7d33f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0cc140 con 0x7fd7bc004d00 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.213+0000 7fd7d098b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7cc0ca8d0 0x7fd7cc1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.213+0000 7fd7d098b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7cc0ca8d0 0x7fd7cc1584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48426/0 (socket says 172.21.15.115:48426) 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.213+0000 7fd7d098b640 1 -- 172.21.15.115:0/2798195458 learned_addr learned my addr 172.21.15.115:0/2798195458 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.213+0000 7fd7d098b640 1 -- 172.21.15.115:0/2798195458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7bc004d00 msgr2=0x7fd7cc14e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.213+0000 7fd7d098b640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7bc004d00 0x7fd7cc14e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.213+0000 7fd7d098b640 1 -- 172.21.15.115:0/2798195458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7cc1589f0 msgr2=0x7fd7cc0b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.213+0000 7fd7d098b640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7cc1589f0 0x7fd7cc0b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.213+0000 7fd7d098b640 1 -- 172.21.15.115:0/2798195458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0cad50 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.213+0000 7fd7d098b640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7cc0ca8d0 0x7fd7cc1584b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800d570 tx=0x7fd7b800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:08.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.214+0000 7fd7c9ffb640 1 -- 172.21.15.115:0/2798195458 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8005480 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.214+0000 7fd7c9ffb640 1 -- 172.21.15.115:0/2798195458 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b8003610 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.214+0000 7fd7c9ffb640 1 -- 172.21.15.115:0/2798195458 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8003980 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.214+0000 7fd7d33f0640 1 -- 172.21.15.115:0/2798195458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc0cb530 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.214+0000 7fd7d33f0640 1 -- 172.21.15.115:0/2798195458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc0cbaf0 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.215+0000 7fd7d33f0640 1 -- 172.21.15.115:0/2798195458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc06fda0 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.216+0000 7fd7c9ffb640 1 -- 172.21.15.115:0/2798195458 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd7b8003ae0 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.217+0000 7fd7c9ffb640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7b006a580 0x7fd7b006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.217+0000 7fd7c9ffb640 1 -- 172.21.15.115:0/2798195458 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd7b8088670 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.218+0000 7fd7d118c640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7b006a580 0x7fd7b006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.218+0000 7fd7d118c640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7b006a580 0x7fd7b006ca40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0004b30 tx=0x7fd7c000d040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:08.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.222+0000 7fd7c9ffb640 1 -- 172.21.15.115:0/2798195458 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd7b8053da0 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.413+0000 7fd7d33f0640 1 -- 172.21.15.115:0/2798195458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd7cc0cec60 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.414+0000 7fd7c9ffb640 1 -- 172.21.15.115:0/2798195458 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd7b8001230 con 0x7fd7cc0ca8d0 2023-12-01T01:49:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:08.416 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:49:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T01:49:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T01:49:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:08.418 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 -- 172.21.15.115:0/2798195458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7b006a580 msgr2=0x7fd7b006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7b006a580 0x7fd7b006ca40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0004b30 tx=0x7fd7c000d040 comp rx=0 tx=0).stop 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 -- 172.21.15.115:0/2798195458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7cc0ca8d0 msgr2=0x7fd7cc1584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7cc0ca8d0 0x7fd7cc1584b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800d570 tx=0x7fd7b800da30 comp rx=0 tx=0).stop 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 -- 172.21.15.115:0/2798195458 shutdown_connections 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7b006a580 0x7fd7b006ca40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7cc1589f0 0x7fd7cc0b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7cc0ca8d0 0x7fd7cc1584b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 --2- 172.21.15.115:0/2798195458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7bc004d00 0x7fd7cc14e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 -- 172.21.15.115:0/2798195458 >> 172.21.15.115:0/2798195458 conn(0x7fd7cc0ce880 msgr2=0x7fd7cc0cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 -- 172.21.15.115:0/2798195458 shutdown_connections 2023-12-01T01:49:08.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.418+0000 7fd7d33f0640 1 -- 172.21.15.115:0/2798195458 wait complete. 2023-12-01T01:49:08.438 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T01:49:08.438 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T01:49:08.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.649+0000 7f8e97fff640 1 Processor -- start 2023-12-01T01:49:08.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.650+0000 7f8e97fff640 1 -- start start 2023-12-01T01:49:08.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.651+0000 7f8e97fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8e9815ac00 0x7f8e980ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.651+0000 7f8e97fff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9815afe0 con 0x7f8e980aede0 2023-12-01T01:49:08.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.652+0000 7f8e97fff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e980b0a10 con 0x7f8e980cb800 2023-12-01T01:49:08.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.652+0000 7f8e97fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e980b12b0 con 0x7f8e9815ac00 2023-12-01T01:49:08.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.652+0000 7f8e8ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8e980aede0 0x7f8e980cb0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58442/0 (socket says 172.21.15.115:58442) 2023-12-01T01:49:08.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.652+0000 7f8e8ffff640 1 -- 172.21.15.115:0/3364378925 learned_addr learned my addr 172.21.15.115:0/3364378925 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:08.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.652+0000 7f8e96ffd640 1 --2- 172.21.15.115:0/3364378925 >> v2:172.21.15.142:3300/0 conn(0x7f8e9815ac00 0x7f8e980ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.652+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421076175 0 0) 0x7f8e9815afe0 con 0x7f8e980aede0 2023-12-01T01:49:08.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.652+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e84003520 con 0x7f8e980aede0 2023-12-01T01:49:08.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 558003250 0 0) 0x7f8e84003520 con 0x7f8e980aede0 2023-12-01T01:49:08.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 >> v2:172.21.15.142:3300/0 conn(0x7f8e9815ac00 msgr2=0x7f8e980ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:08.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 --2- 172.21.15.115:0/3364378925 >> v2:172.21.15.142:3300/0 conn(0x7f8e9815ac00 0x7f8e980ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 >> v1:172.21.15.142:6789/0 conn(0x7f8e980cb800 legacy=0x7f8e9806fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e980b0cf0 con 0x7f8e980aede0 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f8e80001690 con 0x7f8e980aede0 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 >> v1:172.21.15.115:6789/0 conn(0x7f8e980aede0 legacy=0x7f8e980cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e84003970 0x7f8e84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e840047c0 0x7f8e84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e84005180 0x7f8e8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e980b12b0 con 0x7f8e84005180 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8400c120 con 0x7f8e840047c0 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8400c2e0 con 0x7f8e84003970 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.653+0000 7f8e977fe640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e84005180 0x7f8e8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.654+0000 7f8e8ffff640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e840047c0 0x7f8e84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.654+0000 7f8e96ffd640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e84003970 0x7f8e84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.654+0000 7f8e977fe640 1 -- 172.21.15.115:0/3364378925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e84003970 msgr2=0x7f8e84003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.654+0000 7f8e977fe640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e84003970 0x7f8e84003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.654+0000 7f8e977fe640 1 -- 172.21.15.115:0/3364378925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e840047c0 msgr2=0x7f8e84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:08.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.654+0000 7f8e977fe640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e840047c0 0x7f8e84004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.654+0000 7f8e977fe640 1 -- 172.21.15.115:0/3364378925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e84003520 con 0x7f8e84005180 2023-12-01T01:49:08.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.654+0000 7f8e96ffd640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e84003970 0x7f8e84003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.655+0000 7f8e977fe640 1 -- 172.21.15.115:0/3364378925 reap_dead start 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.655+0000 7f8e977fe640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e84005180 0x7f8e8400b980 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8e90001e20 tx=0x7f8e840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e056ce3546d563 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.655+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90003b70 con 0x7f8e84005180 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.655+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e90003cd0 con 0x7f8e84005180 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.655+0000 7f8e967fc640 1 -- 172.21.15.115:0/3364378925 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90005310 con 0x7f8e84005180 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.656+0000 7f8e97fff640 1 -- 172.21.15.115:0/3364378925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e84005180 msgr2=0x7f8e8400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.656+0000 7f8e97fff640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e84005180 0x7f8e8400b980 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8e90001e20 tx=0x7f8e840057f0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.656+0000 7f8e97fff640 1 -- 172.21.15.115:0/3364378925 shutdown_connections 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.656+0000 7f8e97fff640 1 --2- 172.21.15.115:0/3364378925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e84005180 0x7f8e8400b980 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.656+0000 7f8e97fff640 1 -- 172.21.15.115:0/3364378925 >> 172.21.15.115:0/3364378925 conn(0x7f8e98065960 msgr2=0x7f8e98065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:08.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.656+0000 7f8e97fff640 1 -- 172.21.15.115:0/3364378925 shutdown_connections 2023-12-01T01:49:08.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.656+0000 7f8e97fff640 1 -- 172.21.15.115:0/3364378925 wait complete. 2023-12-01T01:49:08.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.658+0000 7f8e97fff640 1 Processor -- start 2023-12-01T01:49:08.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.658+0000 7f8e97fff640 1 -- start start 2023-12-01T01:49:08.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e97fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e84005180 0x7f8e980c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e97fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e980b2930 0x7f8e980b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e97fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e980b1930 0x7f8e980b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e97fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e980b2db0 con 0x7f8e980b2930 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e97fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e980c8650 con 0x7f8e84005180 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e97fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e980c8930 con 0x7f8e980b1930 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e8ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e980b2930 0x7f8e980b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e8ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e980b2930 0x7f8e980b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48444/0 (socket says 172.21.15.115:48444) 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e8ffff640 1 -- 172.21.15.115:0/1215394038 learned_addr learned my addr 172.21.15.115:0/1215394038 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e8ffff640 1 -- 172.21.15.115:0/1215394038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e980b1930 msgr2=0x7f8e980b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e8ffff640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e980b1930 0x7f8e980b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e8ffff640 1 -- 172.21.15.115:0/1215394038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e84005180 msgr2=0x7f8e980c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e8ffff640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e84005180 0x7f8e980c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.659+0000 7f8e8ffff640 1 -- 172.21.15.115:0/1215394038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9815b3d0 con 0x7f8e980b2930 2023-12-01T01:49:08.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.660+0000 7f8e8ffff640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e980b2930 0x7f8e980b13f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8e80001220 tx=0x7f8e8000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:08.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.660+0000 7f8e94ff9640 1 -- 172.21.15.115:0/1215394038 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000c640 con 0x7f8e980b2930 2023-12-01T01:49:08.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.660+0000 7f8e94ff9640 1 -- 172.21.15.115:0/1215394038 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8000c7a0 con 0x7f8e980b2930 2023-12-01T01:49:08.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.660+0000 7f8e94ff9640 1 -- 172.21.15.115:0/1215394038 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000cb50 con 0x7f8e980b2930 2023-12-01T01:49:08.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.660+0000 7f8e97fff640 1 -- 172.21.15.115:0/1215394038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9815b6b0 con 0x7f8e980b2930 2023-12-01T01:49:08.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.660+0000 7f8e97fff640 1 -- 172.21.15.115:0/1215394038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e980c71d0 con 0x7f8e980b2930 2023-12-01T01:49:08.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.661+0000 7f8e97fff640 1 -- 172.21.15.115:0/1215394038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e980618d0 con 0x7f8e980b2930 2023-12-01T01:49:08.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.662+0000 7f8e94ff9640 1 -- 172.21.15.115:0/1215394038 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8e8001d460 con 0x7f8e980b2930 2023-12-01T01:49:08.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.663+0000 7f8e94ff9640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e7806a550 0x7f8e7806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:08.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.663+0000 7f8e94ff9640 1 -- 172.21.15.115:0/1215394038 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8e800888a0 con 0x7f8e980b2930 2023-12-01T01:49:08.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.663+0000 7f8e96ffd640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e7806a550 0x7f8e7806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:08.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.664+0000 7f8e96ffd640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e7806a550 0x7f8e7806ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8e880099e0 tx=0x7f8e88008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:08.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.667+0000 7f8e94ff9640 1 -- 172.21.15.115:0/1215394038 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8e80054030 con 0x7f8e980b2930 2023-12-01T01:49:08.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:08.857+0000 7f8e97fff640 1 -- 172.21.15.115:0/1215394038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8e980b0fa0 con 0x7f8e980b2930 2023-12-01T01:49:09.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.602+0000 7f8e94ff9640 1 -- 172.21.15.115:0/1215394038 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8e80001490 con 0x7f8e980b2930 2023-12-01T01:49:09.604 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T01:49:09.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 -- 172.21.15.115:0/1215394038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e7806a550 msgr2=0x7f8e7806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:09.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e7806a550 0x7f8e7806ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8e880099e0 tx=0x7f8e88008040 comp rx=0 tx=0).stop 2023-12-01T01:49:09.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 -- 172.21.15.115:0/1215394038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e980b2930 msgr2=0x7f8e980b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:09.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e980b2930 0x7f8e980b13f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8e80001220 tx=0x7f8e8000eb90 comp rx=0 tx=0).stop 2023-12-01T01:49:09.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 -- 172.21.15.115:0/1215394038 shutdown_connections 2023-12-01T01:49:09.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e7806a550 0x7f8e7806ca10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:09.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e980b1930 0x7f8e980b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:09.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e980b2930 0x7f8e980b13f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:09.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 --2- 172.21.15.115:0/1215394038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e84005180 0x7f8e980c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:09.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 -- 172.21.15.115:0/1215394038 >> 172.21.15.115:0/1215394038 conn(0x7f8e98065960 msgr2=0x7f8e980cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:09.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 -- 172.21.15.115:0/1215394038 shutdown_connections 2023-12-01T01:49:09.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.607+0000 7f8e97fff640 1 -- 172.21.15.115:0/1215394038 wait complete. 2023-12-01T01:49:09.632 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T01:49:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.841+0000 7f09d4773640 1 Processor -- start 2023-12-01T01:49:09.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.842+0000 7f09d4773640 1 -- start start 2023-12-01T01:49:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.843+0000 7f09d4773640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09cc15ced0 0x7f09cc0b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.843+0000 7f09d4773640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f09cc0b4d70 0x7f09cc0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.843+0000 7f09d4773640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09cc0d1480 con 0x7f09cc0d2750 2023-12-01T01:49:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.843+0000 7f09d4773640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc0d16b0 con 0x7f09cc0b4d70 2023-12-01T01:49:09.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.843+0000 7f09d4773640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0d1990 con 0x7f09cc15ced0 2023-12-01T01:49:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.843+0000 7f09d2d10640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f09cc0b4d70 0x7f09cc0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.844+0000 7f09d250f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09cc15ced0 0x7f09cc0b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.844+0000 7f09d2d10640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f09cc0b4d70 0x7f09cc0d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:54376/0 (socket says 172.21.15.115:54376) 2023-12-01T01:49:09.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.844+0000 7f09d2d10640 1 -- 172.21.15.115:0/3914457277 learned_addr learned my addr 172.21.15.115:0/3914457277 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.844+0000 7f09d2d10640 1 -- 172.21.15.115:0/3914457277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09cc15ced0 msgr2=0x7f09cc0b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.844+0000 7f09d2d10640 1 --2- 172.21.15.115:0/3914457277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09cc15ced0 0x7f09cc0b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.844+0000 7f09d2d10640 1 -- 172.21.15.115:0/3914457277 >> v1:172.21.15.142:6789/0 conn(0x7f09cc0d2750 legacy=0x7f09cc0b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:09.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.844+0000 7f09d2d10640 1 -- 172.21.15.115:0/3914457277 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc0d1c70 con 0x7f09cc0b4d70 2023-12-01T01:49:09.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.846+0000 7f09d2d10640 1 --2- 172.21.15.115:0/3914457277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09cc0b4d70 0x7f09cc0d0f40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009870 tx=0x7f09c402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7511854fce4a3974 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d150d640 1 -- 172.21.15.115:0/3914457277 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4030840 con 0x7f09cc0b4d70 2023-12-01T01:49:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d150d640 1 -- 172.21.15.115:0/3914457277 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c40309a0 con 0x7f09cc0b4d70 2023-12-01T01:49:09.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d150d640 1 -- 172.21.15.115:0/3914457277 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4030d30 con 0x7f09cc0b4d70 2023-12-01T01:49:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d4773640 1 -- 172.21.15.115:0/3914457277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09cc0b4d70 msgr2=0x7f09cc0d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d4773640 1 --2- 172.21.15.115:0/3914457277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09cc0b4d70 0x7f09cc0d0f40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009870 tx=0x7f09c402b980 comp rx=0 tx=0).stop 2023-12-01T01:49:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d4773640 1 -- 172.21.15.115:0/3914457277 shutdown_connections 2023-12-01T01:49:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d4773640 1 --2- 172.21.15.115:0/3914457277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09cc0b4d70 0x7f09cc0d0f40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d4773640 1 --2- 172.21.15.115:0/3914457277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09cc15ced0 0x7f09cc0b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d4773640 1 -- 172.21.15.115:0/3914457277 >> 172.21.15.115:0/3914457277 conn(0x7f09cc065f90 msgr2=0x7f09cc0cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d4773640 1 -- 172.21.15.115:0/3914457277 shutdown_connections 2023-12-01T01:49:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.847+0000 7f09d4773640 1 -- 172.21.15.115:0/3914457277 wait complete. 2023-12-01T01:49:09.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.849+0000 7f09d4773640 1 Processor -- start 2023-12-01T01:49:09.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.849+0000 7f09d4773640 1 -- start start 2023-12-01T01:49:09.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.850+0000 7f09d4773640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09cc0b4d70 0x7f09cc157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:09.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.850+0000 7f09d4773640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09cc0d2750 0x7f09cc0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.850+0000 7f09d4773640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09cc15ced0 0x7f09cc0ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.850+0000 7f09d4773640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1f6870 con 0x7f09cc0d2750 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.850+0000 7f09d4773640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc160330 con 0x7f09cc15ced0 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.850+0000 7f09d4773640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0cd0c0 con 0x7f09cc0b4d70 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.850+0000 7f09d1d0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09cc0d2750 0x7f09cc0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d1d0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09cc0d2750 0x7f09cc0cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48458/0 (socket says 172.21.15.115:48458) 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d1d0e640 1 -- 172.21.15.115:0/3639166348 learned_addr learned my addr 172.21.15.115:0/3639166348 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d250f640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09cc0b4d70 0x7f09cc157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d2d10640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09cc15ced0 0x7f09cc0ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d1d0e640 1 -- 172.21.15.115:0/3639166348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09cc0b4d70 msgr2=0x7f09cc157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d1d0e640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09cc0b4d70 0x7f09cc157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d1d0e640 1 -- 172.21.15.115:0/3639166348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09cc15ced0 msgr2=0x7f09cc0ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:09.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d1d0e640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09cc15ced0 0x7f09cc0ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d1d0e640 1 -- 172.21.15.115:0/3639166348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c40094d0 con 0x7f09cc0d2750 2023-12-01T01:49:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d2d10640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09cc15ced0 0x7f09cc0ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.851+0000 7f09d1d0e640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09cc0d2750 0x7f09cc0cc640 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f09c000c400 tx=0x7f09c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:09.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.852+0000 7f09caffd640 1 -- 172.21.15.115:0/3639166348 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00052d0 con 0x7f09cc0d2750 2023-12-01T01:49:09.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.852+0000 7f09caffd640 1 -- 172.21.15.115:0/3639166348 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c0005430 con 0x7f09cc0d2750 2023-12-01T01:49:09.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.852+0000 7f09caffd640 1 -- 172.21.15.115:0/3639166348 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0003830 con 0x7f09cc0d2750 2023-12-01T01:49:09.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.852+0000 7f09d4773640 1 -- 172.21.15.115:0/3639166348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc1f5bf0 con 0x7f09cc0d2750 2023-12-01T01:49:09.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.852+0000 7f09d4773640 1 -- 172.21.15.115:0/3639166348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc1f61b0 con 0x7f09cc0d2750 2023-12-01T01:49:09.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.853+0000 7f09d4773640 1 -- 172.21.15.115:0/3639166348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09cc1f69b0 con 0x7f09cc0d2750 2023-12-01T01:49:09.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.854+0000 7f09caffd640 1 -- 172.21.15.115:0/3639166348 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f09c0003990 con 0x7f09cc0d2750 2023-12-01T01:49:09.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.855+0000 7f09caffd640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09b806a550 0x7f09b806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:09.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.855+0000 7f09caffd640 1 -- 172.21.15.115:0/3639166348 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f09c00880b0 con 0x7f09cc0d2750 2023-12-01T01:49:09.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.856+0000 7f09d250f640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09b806a550 0x7f09b806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:09.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.856+0000 7f09d250f640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09b806a550 0x7f09b806ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0073f0 tx=0x7f09bc00b040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:09.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:09.859+0000 7f09caffd640 1 -- 172.21.15.115:0/3639166348 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f09c0053850 con 0x7f09cc0d2750 2023-12-01T01:49:10.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.051+0000 7f09d4773640 1 -- 172.21.15.115:0/3639166348 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f09cc1f69b0 con 0x7f09cc0d2750 2023-12-01T01:49:10.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.621+0000 7f09caffd640 1 -- 172.21.15.115:0/3639166348 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) v1 ==== 170+0+0 (secure 0 0 0) 0x7f09cc1f69b0 con 0x7f09cc0d2750 2023-12-01T01:49:10.623 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T01:49:10.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.626+0000 7f09d4773640 1 -- 172.21.15.115:0/3639166348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09b806a550 msgr2=0x7f09b806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:10.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.626+0000 7f09d4773640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09b806a550 0x7f09b806ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0073f0 tx=0x7f09bc00b040 comp rx=0 tx=0).stop 2023-12-01T01:49:10.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.626+0000 7f09d4773640 1 -- 172.21.15.115:0/3639166348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09cc0d2750 msgr2=0x7f09cc0cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:10.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.626+0000 7f09d4773640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09cc0d2750 0x7f09cc0cc640 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f09c000c400 tx=0x7f09c000c8c0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.628+0000 7f09d4773640 1 -- 172.21.15.115:0/3639166348 shutdown_connections 2023-12-01T01:49:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.628+0000 7f09d4773640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f09b806a550 0x7f09b806ca10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.628+0000 7f09d4773640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09cc15ced0 0x7f09cc0ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.628+0000 7f09d4773640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09cc0d2750 0x7f09cc0cc640 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.628+0000 7f09d4773640 1 --2- 172.21.15.115:0/3639166348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09cc0b4d70 0x7f09cc157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.628+0000 7f09d4773640 1 -- 172.21.15.115:0/3639166348 >> 172.21.15.115:0/3639166348 conn(0x7f09cc065f90 msgr2=0x7f09cc0b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.630+0000 7f09d4773640 1 -- 172.21.15.115:0/3639166348 shutdown_connections 2023-12-01T01:49:10.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.631+0000 7f09d4773640 1 -- 172.21.15.115:0/3639166348 wait complete. 2023-12-01T01:49:10.662 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T01:49:10.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.848+0000 7ff32bfff640 1 Processor -- start 2023-12-01T01:49:10.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.848+0000 7ff32bfff640 1 -- start start 2023-12-01T01:49:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.849+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff32c1599e0 0x7ff32c159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.850+0000 7ff32bfff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c15e440 con 0x7ff32c0c9770 2023-12-01T01:49:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.850+0000 7ff32bfff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c15a300 con 0x7ff32c0af370 2023-12-01T01:49:10.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.850+0000 7ff32bfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15d550 con 0x7ff32c1599e0 2023-12-01T01:49:10.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.850+0000 7ff32b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff32c0c9770 0x7ff32c070300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58450/0 (socket says 172.21.15.115:58450) 2023-12-01T01:49:10.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.850+0000 7ff32b7fe640 1 -- 172.21.15.115:0/1372728980 learned_addr learned my addr 172.21.15.115:0/1372728980 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:10.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.850+0000 7ff32affd640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff32c1599e0 0x7ff32c159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:10.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.850+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831021480 0 0) 0x7ff32c15e440 con 0x7ff32c0c9770 2023-12-01T01:49:10.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff318003520 con 0x7ff32c0c9770 2023-12-01T01:49:10.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4133638796 0 0) 0x7ff318003520 con 0x7ff32c0c9770 2023-12-01T01:49:10.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff32c1599e0 msgr2=0x7ff32c159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:10.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff32c1599e0 0x7ff32c159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 >> v1:172.21.15.142:6789/0 conn(0x7ff32c0af370 legacy=0x7ff32c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c15d830 con 0x7ff32c0c9770 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7ff3240019f0 con 0x7ff32c0c9770 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 >> v1:172.21.15.115:6789/0 conn(0x7ff32c0c9770 legacy=0x7ff32c070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3180039a0 0x7ff318003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3180047c0 0x7ff318004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff32c1599e0 0x7ff318005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15d550 con 0x7ff3180039a0 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31800ba30 con 0x7ff3180047c0 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31800bbf0 con 0x7ff32c1599e0 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.851+0000 7ff32affd640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3180039a0 0x7ff318003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.852+0000 7ff32b7fe640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff32c1599e0 0x7ff318005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.852+0000 7ff32affd640 1 -- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff32c1599e0 msgr2=0x7ff318005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.852+0000 7ff3227fc640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3180047c0 0x7ff318004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.852+0000 7ff32affd640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff32c1599e0 0x7ff318005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.852+0000 7ff32affd640 1 -- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3180047c0 msgr2=0x7ff318004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.852+0000 7ff32affd640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3180047c0 0x7ff318004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.852+0000 7ff32affd640 1 -- 172.21.15.115:0/1372728980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318003520 con 0x7ff3180039a0 2023-12-01T01:49:10.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.852+0000 7ff32b7fe640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff32c1599e0 0x7ff318005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.853+0000 7ff32affd640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3180039a0 0x7ff318003dc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009a90 tx=0x7ff3180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8aece4636182c73 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.853+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c02c660 con 0x7ff3180039a0 2023-12-01T01:49:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.853+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c02c7c0 con 0x7ff3180039a0 2023-12-01T01:49:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.853+0000 7ff32a7fc640 1 -- 172.21.15.115:0/1372728980 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c02cb50 con 0x7ff3180039a0 2023-12-01T01:49:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.854+0000 7ff32bfff640 1 -- 172.21.15.115:0/1372728980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3180039a0 msgr2=0x7ff318003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.854+0000 7ff32bfff640 1 --2- 172.21.15.115:0/1372728980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3180039a0 0x7ff318003dc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009a90 tx=0x7ff3180057f0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.854+0000 7ff32b7fe640 1 -- 172.21.15.115:0/1372728980 reap_dead start 2023-12-01T01:49:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.854+0000 7ff32bfff640 1 -- 172.21.15.115:0/1372728980 shutdown_connections 2023-12-01T01:49:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.854+0000 7ff32bfff640 1 -- 172.21.15.115:0/1372728980 >> 172.21.15.115:0/1372728980 conn(0x7ff32c065960 msgr2=0x7ff32c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.854+0000 7ff32bfff640 1 -- 172.21.15.115:0/1372728980 shutdown_connections 2023-12-01T01:49:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.854+0000 7ff32bfff640 1 -- 172.21.15.115:0/1372728980 wait complete. 2023-12-01T01:49:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.856+0000 7ff32bfff640 1 Processor -- start 2023-12-01T01:49:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.856+0000 7ff32bfff640 1 -- start start 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3180039a0 0x7ff32c1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff32c1f1a10 0x7ff32c0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff32c0c7de0 0x7ff32c06b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32bfff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0c8260 con 0x7ff32c0c7de0 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32bfff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c1f5dd0 con 0x7ff32c1f1a10 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32bfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0c8740 con 0x7ff3180039a0 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff32c0c7de0 0x7ff32c06b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff32c0c7de0 0x7ff32c06b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48488/0 (socket says 172.21.15.115:48488) 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32b7fe640 1 -- 172.21.15.115:0/2281814167 learned_addr learned my addr 172.21.15.115:0/2281814167 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff3227fc640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff32c1f1a10 0x7ff32c0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32affd640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3180039a0 0x7ff32c1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.857+0000 7ff32b7fe640 1 -- 172.21.15.115:0/2281814167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3180039a0 msgr2=0x7ff32c1f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.858+0000 7ff32b7fe640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3180039a0 0x7ff32c1f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.858+0000 7ff32b7fe640 1 -- 172.21.15.115:0/2281814167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff32c1f1a10 msgr2=0x7ff32c0c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.858+0000 7ff32b7fe640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff32c1f1a10 0x7ff32c0c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.858+0000 7ff32b7fe640 1 -- 172.21.15.115:0/2281814167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0c8460 con 0x7ff32c0c7de0 2023-12-01T01:49:10.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.858+0000 7ff32b7fe640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff32c0c7de0 0x7ff32c06b0e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff3240012e0 tx=0x7ff32400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:10.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.858+0000 7ff323fff640 1 -- 172.21.15.115:0/2281814167 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32400bb20 con 0x7ff32c0c7de0 2023-12-01T01:49:10.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.858+0000 7ff323fff640 1 -- 172.21.15.115:0/2281814167 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32400bca0 con 0x7ff32c0c7de0 2023-12-01T01:49:10.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.858+0000 7ff323fff640 1 -- 172.21.15.115:0/2281814167 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324007070 con 0x7ff32c0c7de0 2023-12-01T01:49:10.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.858+0000 7ff32bfff640 1 -- 172.21.15.115:0/2281814167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c1f5780 con 0x7ff32c0c7de0 2023-12-01T01:49:10.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.858+0000 7ff32bfff640 1 -- 172.21.15.115:0/2281814167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c144160 con 0x7ff32c0c7de0 2023-12-01T01:49:10.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.859+0000 7ff32bfff640 1 -- 172.21.15.115:0/2281814167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c0618d0 con 0x7ff32c0c7de0 2023-12-01T01:49:10.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.862+0000 7ff323fff640 1 -- 172.21.15.115:0/2281814167 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff3240071d0 con 0x7ff32c0c7de0 2023-12-01T01:49:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.862+0000 7ff323fff640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff30c06a550 0x7ff30c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.863+0000 7ff323fff640 1 -- 172.21.15.115:0/2281814167 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff324089690 con 0x7ff32c0c7de0 2023-12-01T01:49:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.863+0000 7ff32affd640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff30c06a550 0x7ff30c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.864+0000 7ff32affd640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff30c06a550 0x7ff30c06ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff31c036000 tx=0x7ff31c030000 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:10.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:10.867+0000 7ff323fff640 1 -- 172.21.15.115:0/2281814167 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff324054ba0 con 0x7ff32c0c7de0 2023-12-01T01:49:11.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.091+0000 7ff32bfff640 1 -- 172.21.15.115:0/2281814167 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff32c1f5f10 con 0x7ff32c0c7de0 2023-12-01T01:49:11.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.102+0000 7ff323fff640 1 -- 172.21.15.115:0/2281814167 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff3240581c0 con 0x7ff32c0c7de0 2023-12-01T01:49:11.104 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 6 and data pool 7 2023-12-01T01:49:11.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 -- 172.21.15.115:0/2281814167 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff30c06a550 msgr2=0x7ff30c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff30c06a550 0x7ff30c06ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff31c036000 tx=0x7ff31c030000 comp rx=0 tx=0).stop 2023-12-01T01:49:11.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 -- 172.21.15.115:0/2281814167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff32c0c7de0 msgr2=0x7ff32c06b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff32c0c7de0 0x7ff32c06b0e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff3240012e0 tx=0x7ff32400b4a0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 -- 172.21.15.115:0/2281814167 shutdown_connections 2023-12-01T01:49:11.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff30c06a550 0x7ff30c06ca10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff32c0c7de0 0x7ff32c06b0e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff32c1f1a10 0x7ff32c0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 --2- 172.21.15.115:0/2281814167 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3180039a0 0x7ff32c1f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 -- 172.21.15.115:0/2281814167 >> 172.21.15.115:0/2281814167 conn(0x7ff32c065960 msgr2=0x7ff32c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:11.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 -- 172.21.15.115:0/2281814167 shutdown_connections 2023-12-01T01:49:11.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.109+0000 7ff32bfff640 1 -- 172.21.15.115:0/2281814167 wait complete. 2023-12-01T01:49:11.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:49:11.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.315+0000 7faf87670640 1 Processor -- start 2023-12-01T01:49:11.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.315+0000 7faf87670640 1 -- start start 2023-12-01T01:49:11.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.316+0000 7faf87670640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faf8015d110 0x7faf801615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.316+0000 7faf87670640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cc880 0x7faf800ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.316+0000 7faf87670640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf800ac8a0 con 0x7faf800cf840 2023-12-01T01:49:11.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.316+0000 7faf87670640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf800ae2a0 con 0x7faf8015d110 2023-12-01T01:49:11.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.316+0000 7faf87670640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf80161b20 con 0x7faf800cc880 2023-12-01T01:49:11.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.317+0000 7faf85c0d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cc880 0x7faf800ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.317+0000 7faf8540c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faf8015d110 0x7faf801615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.317+0000 7faf85c0d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cc880 0x7faf800ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:40412/0 (socket says 172.21.15.115:40412) 2023-12-01T01:49:11.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.317+0000 7faf85c0d640 1 -- 172.21.15.115:0/1781152823 learned_addr learned my addr 172.21.15.115:0/1781152823 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:11.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.318+0000 7faf8540c640 1 -- 172.21.15.115:0/1781152823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cc880 msgr2=0x7faf800ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:11.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.318+0000 7faf8540c640 1 --2- 172.21.15.115:0/1781152823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cc880 0x7faf800ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.318+0000 7faf8540c640 1 -- 172.21.15.115:0/1781152823 >> v1:172.21.15.142:6789/0 conn(0x7faf800cf840 legacy=0x7faf800adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.318+0000 7faf8540c640 1 -- 172.21.15.115:0/1781152823 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8015d4f0 con 0x7faf8015d110 2023-12-01T01:49:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.318+0000 7faf85c0d640 1 --2- 172.21.15.115:0/1781152823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cc880 0x7faf800ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:49:11.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.319+0000 7faf8540c640 1 --2- 172.21.15.115:0/1781152823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf8015d110 0x7faf801615e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0097d0 tx=0x7faf7c02b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed7a964f605de736 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:11.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.320+0000 7faf77fff640 1 -- 172.21.15.115:0/1781152823 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c031800 con 0x7faf8015d110 2023-12-01T01:49:11.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.320+0000 7faf77fff640 1 -- 172.21.15.115:0/1781152823 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c031960 con 0x7faf8015d110 2023-12-01T01:49:11.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.320+0000 7faf77fff640 1 -- 172.21.15.115:0/1781152823 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c031cf0 con 0x7faf8015d110 2023-12-01T01:49:11.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.320+0000 7faf87670640 1 -- 172.21.15.115:0/1781152823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf8015d110 msgr2=0x7faf801615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.320+0000 7faf87670640 1 --2- 172.21.15.115:0/1781152823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf8015d110 0x7faf801615e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0097d0 tx=0x7faf7c02b8a0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.321+0000 7faf87670640 1 -- 172.21.15.115:0/1781152823 shutdown_connections 2023-12-01T01:49:11.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.321+0000 7faf87670640 1 --2- 172.21.15.115:0/1781152823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cc880 0x7faf800ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.321+0000 7faf87670640 1 --2- 172.21.15.115:0/1781152823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf8015d110 0x7faf801615e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.321+0000 7faf87670640 1 -- 172.21.15.115:0/1781152823 >> 172.21.15.115:0/1781152823 conn(0x7faf800cecb0 msgr2=0x7faf800cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:11.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.321+0000 7faf87670640 1 -- 172.21.15.115:0/1781152823 shutdown_connections 2023-12-01T01:49:11.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.321+0000 7faf87670640 1 -- 172.21.15.115:0/1781152823 wait complete. 2023-12-01T01:49:11.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.322+0000 7faf87670640 1 Processor -- start 2023-12-01T01:49:11.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.323+0000 7faf87670640 1 -- start start 2023-12-01T01:49:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.323+0000 7faf87670640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf800cc880 0x7faf800c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.323+0000 7faf87670640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cf840 0x7faf8015ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.323+0000 7faf87670640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf8015d110 0x7faf800b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf87670640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf800b30d0 con 0x7faf8015d110 2023-12-01T01:49:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf87670640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf800c7380 con 0x7faf800cc880 2023-12-01T01:49:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf87670640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf800c7660 con 0x7faf800cf840 2023-12-01T01:49:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf85c0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf8015d110 0x7faf800b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf85c0d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf8015d110 0x7faf800b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48490/0 (socket says 172.21.15.115:48490) 2023-12-01T01:49:11.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf85c0d640 1 -- 172.21.15.115:0/789045962 learned_addr learned my addr 172.21.15.115:0/789045962 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf8540c640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf800cc880 0x7faf800c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf84c0b640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cf840 0x7faf8015ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf85c0d640 1 -- 172.21.15.115:0/789045962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cf840 msgr2=0x7faf8015ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf85c0d640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cf840 0x7faf8015ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf85c0d640 1 -- 172.21.15.115:0/789045962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf800cc880 msgr2=0x7faf800c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf85c0d640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf800cc880 0x7faf800c27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf85c0d640 1 -- 172.21.15.115:0/789045962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf78009b30 con 0x7faf8015d110 2023-12-01T01:49:11.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.324+0000 7faf84c0b640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cf840 0x7faf8015ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.325+0000 7faf85c0d640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf8015d110 0x7faf800b2680 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faf7800efc0 tx=0x7faf7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.325+0000 7faf75ffb640 1 -- 172.21.15.115:0/789045962 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78018070 con 0x7faf8015d110 2023-12-01T01:49:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.325+0000 7faf75ffb640 1 -- 172.21.15.115:0/789045962 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7800f040 con 0x7faf8015d110 2023-12-01T01:49:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.325+0000 7faf75ffb640 1 -- 172.21.15.115:0/789045962 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7800ce00 con 0x7faf8015d110 2023-12-01T01:49:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.325+0000 7faf87670640 1 -- 172.21.15.115:0/789045962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c009400 con 0x7faf8015d110 2023-12-01T01:49:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.325+0000 7faf87670640 1 -- 172.21.15.115:0/789045962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf800c7c20 con 0x7faf8015d110 2023-12-01T01:49:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.326+0000 7faf87670640 1 -- 172.21.15.115:0/789045962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8006d520 con 0x7faf8015d110 2023-12-01T01:49:11.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.327+0000 7faf75ffb640 1 -- 172.21.15.115:0/789045962 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7faf78014480 con 0x7faf8015d110 2023-12-01T01:49:11.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.328+0000 7faf75ffb640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf6406a550 0x7faf6406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.329+0000 7faf75ffb640 1 -- 172.21.15.115:0/789045962 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf78088f30 con 0x7faf8015d110 2023-12-01T01:49:11.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.329+0000 7faf8540c640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf6406a550 0x7faf6406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.329+0000 7faf8540c640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf6406a550 0x7faf6406ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faf7c036000 tx=0x7faf7c03c040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:11.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.333+0000 7faf75ffb640 1 -- 172.21.15.115:0/789045962 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faf780542f0 con 0x7faf8015d110 2023-12-01T01:49:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.512+0000 7faf87670640 1 -- 172.21.15.115:0/789045962 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf8004e890 con 0x7faf8015d110 2023-12-01T01:49:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.513+0000 7faf75ffb640 1 -- 172.21.15.115:0/789045962 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16102 (secure 0 0 0) 0x7faf78057910 con 0x7faf8015d110 2023-12-01T01:49:11.515 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:11.515 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":63,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:49:11.093765+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:49:08.909721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-01T01:49:10.108096+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:49:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.515+0000 7faf87670640 1 -- 172.21.15.115:0/789045962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf6406a550 msgr2=0x7faf6406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.515+0000 7faf87670640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf6406a550 0x7faf6406ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faf7c036000 tx=0x7faf7c03c040 comp rx=0 tx=0).stop 2023-12-01T01:49:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.515+0000 7faf87670640 1 -- 172.21.15.115:0/789045962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf8015d110 msgr2=0x7faf800b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.515+0000 7faf87670640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf8015d110 0x7faf800b2680 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faf7800efc0 tx=0x7faf7800c490 comp rx=0 tx=0).stop 2023-12-01T01:49:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.515+0000 7faf87670640 1 -- 172.21.15.115:0/789045962 shutdown_connections 2023-12-01T01:49:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.515+0000 7faf87670640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf6406a550 0x7faf6406ca10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.515+0000 7faf87670640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf800cf840 0x7faf8015ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.515+0000 7faf87670640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf8015d110 0x7faf800b2680 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.515+0000 7faf87670640 1 --2- 172.21.15.115:0/789045962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf800cc880 0x7faf800c27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.515+0000 7faf87670640 1 -- 172.21.15.115:0/789045962 >> 172.21.15.115:0/789045962 conn(0x7faf800cecb0 msgr2=0x7faf800afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.516+0000 7faf87670640 1 -- 172.21.15.115:0/789045962 shutdown_connections 2023-12-01T01:49:11.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.516+0000 7faf87670640 1 -- 172.21.15.115:0/789045962 wait complete. 2023-12-01T01:49:11.534 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:49:11.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.730+0000 7fa3bdaaf640 1 Processor -- start 2023-12-01T01:49:11.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.730+0000 7fa3bdaaf640 1 -- start start 2023-12-01T01:49:11.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.731+0000 7fa3bdaaf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa3b806f2f0 0x7fa3b806bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.731+0000 7fa3bdaaf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b80add10 con 0x7fa3b80cad70 2023-12-01T01:49:11.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.731+0000 7fa3bdaaf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b80c5760 con 0x7fa3b815e2f0 2023-12-01T01:49:11.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.731+0000 7fa3bdaaf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b815e6d0 con 0x7fa3b806f2f0 2023-12-01T01:49:11.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b6ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa3b80cad70 0x7fa3b806b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58464/0 (socket says 172.21.15.115:58464) 2023-12-01T01:49:11.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b6ffd640 1 -- 172.21.15.115:0/4095201585 learned_addr learned my addr 172.21.15.115:0/4095201585 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:11.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116534832 0 0) 0x7fa3b80add10 con 0x7fa3b80cad70 2023-12-01T01:49:11.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b7fff640 1 --2- 172.21.15.115:0/4095201585 >> v2:172.21.15.142:3300/0 conn(0x7fa3b806f2f0 0x7fa3b806bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3a0003520 con 0x7fa3b80cad70 2023-12-01T01:49:11.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2687402799 0 0) 0x7fa3a0003520 con 0x7fa3b80cad70 2023-12-01T01:49:11.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 >> v2:172.21.15.142:3300/0 conn(0x7fa3b806f2f0 msgr2=0x7fa3b806bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:11.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b67fc640 1 --2- 172.21.15.115:0/4095201585 >> v2:172.21.15.142:3300/0 conn(0x7fa3b806f2f0 0x7fa3b806bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 >> v1:172.21.15.142:6789/0 conn(0x7fa3b815e2f0 legacy=0x7fa3b80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:11.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80cf7b0 con 0x7fa3b80cad70 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.732+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa3ac001690 con 0x7fa3b80cad70 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 >> v1:172.21.15.115:6789/0 conn(0x7fa3b80cad70 legacy=0x7fa3b806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b67fc640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3a0003a10 0x7fa3a0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b67fc640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a00047c0 0x7fa3a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b67fc640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3a0005380 0x7fa3a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b815e6d0 con 0x7fa3a00047c0 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a000c1f0 con 0x7fa3a0003a10 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a000c3b0 con 0x7fa3a0005380 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b6ffd640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a00047c0 0x7fa3a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b77fe640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3a0003a10 0x7fa3a0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b6ffd640 1 -- 172.21.15.115:0/4095201585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3a0005380 msgr2=0x7fa3a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b6ffd640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3a0005380 0x7fa3a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b6ffd640 1 -- 172.21.15.115:0/4095201585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3a0003a10 msgr2=0x7fa3a0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:11.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b6ffd640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3a0003a10 0x7fa3a0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b6ffd640 1 -- 172.21.15.115:0/4095201585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a0003520 con 0x7fa3a00047c0 2023-12-01T01:49:11.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b77fe640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3a0003a10 0x7fa3a0003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:11.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.733+0000 7fa3b7fff640 1 -- 172.21.15.115:0/4095201585 reap_dead start 2023-12-01T01:49:11.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.734+0000 7fa3b6ffd640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a00047c0 0x7fa3a0004c40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00c160 tx=0x7fa3a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9458c341cd959aca server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:11.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.734+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac005ad0 con 0x7fa3a00047c0 2023-12-01T01:49:11.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.734+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac005c50 con 0x7fa3a00047c0 2023-12-01T01:49:11.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.734+0000 7fa3b67fc640 1 -- 172.21.15.115:0/4095201585 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac007070 con 0x7fa3a00047c0 2023-12-01T01:49:11.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.735+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/4095201585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a00047c0 msgr2=0x7fa3a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.735+0000 7fa3bdaaf640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a00047c0 0x7fa3a0004c40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00c160 tx=0x7fa3a00057f0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.735+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/4095201585 shutdown_connections 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.735+0000 7fa3bdaaf640 1 --2- 172.21.15.115:0/4095201585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3a00047c0 0x7fa3a0004c40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.735+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/4095201585 >> 172.21.15.115:0/4095201585 conn(0x7fa3b80ce880 msgr2=0x7fa3b80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.735+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/4095201585 shutdown_connections 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.735+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/4095201585 wait complete. 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.737+0000 7fa3bdaaf640 1 Processor -- start 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.737+0000 7fa3bdaaf640 1 -- start start 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.738+0000 7fa3bdaaf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3a00047c0 0x7fa3b80b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.738+0000 7fa3bdaaf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3b814fc00 0x7fa3b8152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.738+0000 7fa3bdaaf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b81527c0 0x7fa3b80c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.738+0000 7fa3bdaaf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80c2a10 con 0x7fa3b81527c0 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.738+0000 7fa3bdaaf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80b4e30 con 0x7fa3a00047c0 2023-12-01T01:49:11.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.738+0000 7fa3bdaaf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8152c40 con 0x7fa3b814fc00 2023-12-01T01:49:11.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.738+0000 7fa3b7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b81527c0 0x7fa3b80c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.738+0000 7fa3b7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b81527c0 0x7fa3b80c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48508/0 (socket says 172.21.15.115:48508) 2023-12-01T01:49:11.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.738+0000 7fa3b7fff640 1 -- 172.21.15.115:0/1048154519 learned_addr learned my addr 172.21.15.115:0/1048154519 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:11.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa3b77fe640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3a00047c0 0x7fa3b80b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa3b6ffd640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3b814fc00 0x7fa3b8152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa3b7fff640 1 -- 172.21.15.115:0/1048154519 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3b814fc00 msgr2=0x7fa3b8152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa3b7fff640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3b814fc00 0x7fa3b8152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa3b7fff640 1 -- 172.21.15.115:0/1048154519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3a00047c0 msgr2=0x7fa3b80b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa3b7fff640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3a00047c0 0x7fa3b80b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa3b7fff640 1 -- 172.21.15.115:0/1048154519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80c5170 con 0x7fa3b81527c0 2023-12-01T01:49:11.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa3b7fff640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b81527c0 0x7fa3b80c5c30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa3b806f700 tx=0x7fa3b000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:11.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa39ffff640 1 -- 172.21.15.115:0/1048154519 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000c620 con 0x7fa3b81527c0 2023-12-01T01:49:11.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa39ffff640 1 -- 172.21.15.115:0/1048154519 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000c780 con 0x7fa3b81527c0 2023-12-01T01:49:11.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa39ffff640 1 -- 172.21.15.115:0/1048154519 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000cb30 con 0x7fa3b81527c0 2023-12-01T01:49:11.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/1048154519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b80c2450 con 0x7fa3b81527c0 2023-12-01T01:49:11.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.739+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/1048154519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b81503d0 con 0x7fa3b81527c0 2023-12-01T01:49:11.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.740+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/1048154519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b806fda0 con 0x7fa3b81527c0 2023-12-01T01:49:11.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.741+0000 7fa39ffff640 1 -- 172.21.15.115:0/1048154519 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa3b000cc90 con 0x7fa3b81527c0 2023-12-01T01:49:11.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.742+0000 7fa39ffff640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa3a406a550 0x7fa3a406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:11.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.742+0000 7fa39ffff640 1 -- 172.21.15.115:0/1048154519 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3b0088350 con 0x7fa3b81527c0 2023-12-01T01:49:11.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.742+0000 7fa3b77fe640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa3a406a550 0x7fa3a406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:11.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.743+0000 7fa3b77fe640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa3a406a550 0x7fa3a406ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8007fd0 tx=0x7fa3a8009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:11.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.746+0000 7fa39ffff640 1 -- 172.21.15.115:0/1048154519 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa3b0053790 con 0x7fa3b81527c0 2023-12-01T01:49:11.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.925+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/1048154519 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3b8150230 con 0x7fa3b81527c0 2023-12-01T01:49:11.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.926+0000 7fa39ffff640 1 -- 172.21.15.115:0/1048154519 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16102 (secure 0 0 0) 0x7fa3b8150230 con 0x7fa3b81527c0 2023-12-01T01:49:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:11.928 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":63,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:49:11.093765+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:49:08.909721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-01T01:49:10.108096+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:49:11.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/1048154519 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa3a406a550 msgr2=0x7fa3a406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa3a406a550 0x7fa3a406ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8007fd0 tx=0x7fa3a8009040 comp rx=0 tx=0).stop 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/1048154519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b81527c0 msgr2=0x7fa3b80c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b81527c0 0x7fa3b80c5c30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa3b806f700 tx=0x7fa3b000ec00 comp rx=0 tx=0).stop 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/1048154519 shutdown_connections 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa3a406a550 0x7fa3a406ca10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3b814fc00 0x7fa3b8152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b81527c0 0x7fa3b80c5c30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 --2- 172.21.15.115:0/1048154519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3a00047c0 0x7fa3b80b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/1048154519 >> 172.21.15.115:0/1048154519 conn(0x7fa3b80ce880 msgr2=0x7fa3b80ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/1048154519 shutdown_connections 2023-12-01T01:49:11.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:11.929+0000 7fa3bdaaf640 1 -- 172.21.15.115:0/1048154519 wait complete. 2023-12-01T01:49:11.950 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T01:49:12.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.158+0000 7f700e8d8640 1 Processor -- start 2023-12-01T01:49:12.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.158+0000 7f700e8d8640 1 -- start start 2023-12-01T01:49:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.160+0000 7f700e8d8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f70080b0e60 0x7f70080cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.160+0000 7f700e8d8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70080cc9a0 con 0x7f700815ac80 2023-12-01T01:49:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.160+0000 7f700e8d8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70080cbd90 con 0x7f70080ce1b0 2023-12-01T01:49:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.160+0000 7f700e8d8640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700815b060 con 0x7f70080b0e60 2023-12-01T01:49:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.160+0000 7f7007fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f700815ac80 0x7f70080cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58476/0 (socket says 172.21.15.115:58476) 2023-12-01T01:49:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.160+0000 7f7007fff640 1 -- 172.21.15.115:0/4044586652 learned_addr learned my addr 172.21.15.115:0/4044586652 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.160+0000 7f700ce75640 1 --2- 172.21.15.115:0/4044586652 >> v2:172.21.15.115:3300/0 conn(0x7f70080b0e60 0x7f70080cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.160+0000 7f7006ffd640 1 -- 172.21.15.115:0/4044586652 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089688725 0 0) 0x7f70080cc9a0 con 0x7f700815ac80 2023-12-01T01:49:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.161+0000 7f7006ffd640 1 -- 172.21.15.115:0/4044586652 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff0003520 con 0x7f700815ac80 2023-12-01T01:49:12.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.161+0000 7f700ce75640 1 -- 172.21.15.115:0/4044586652 >> v1:172.21.15.142:6789/0 conn(0x7f70080ce1b0 legacy=0x7f700806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:12.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.161+0000 7f700ce75640 1 -- 172.21.15.115:0/4044586652 >> v1:172.21.15.115:6789/0 conn(0x7f700815ac80 legacy=0x7f70080cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:12.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.161+0000 7f700ce75640 1 -- 172.21.15.115:0/4044586652 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080cc070 con 0x7f70080b0e60 2023-12-01T01:49:12.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.162+0000 7f700ce75640 1 --2- 172.21.15.115:0/4044586652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70080b0e60 0x7f70080cce30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7000009d90 tx=0x7f700002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c3fcd7a90e963 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:12.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.162+0000 7f7006ffd640 1 -- 172.21.15.115:0/4044586652 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700002cc60 con 0x7f70080b0e60 2023-12-01T01:49:12.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.162+0000 7f7006ffd640 1 -- 172.21.15.115:0/4044586652 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700002cdc0 con 0x7f70080b0e60 2023-12-01T01:49:12.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.162+0000 7f7006ffd640 1 -- 172.21.15.115:0/4044586652 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000034660 con 0x7f70080b0e60 2023-12-01T01:49:12.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.163+0000 7f700e8d8640 1 -- 172.21.15.115:0/4044586652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70080b0e60 msgr2=0x7f70080cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:12.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.163+0000 7f700e8d8640 1 --2- 172.21.15.115:0/4044586652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70080b0e60 0x7f70080cce30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7000009d90 tx=0x7f700002beb0 comp rx=0 tx=0).stop 2023-12-01T01:49:12.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.163+0000 7f700e8d8640 1 -- 172.21.15.115:0/4044586652 shutdown_connections 2023-12-01T01:49:12.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.163+0000 7f700e8d8640 1 --2- 172.21.15.115:0/4044586652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70080b0e60 0x7f70080cce30 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:12.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.163+0000 7f700e8d8640 1 -- 172.21.15.115:0/4044586652 >> 172.21.15.115:0/4044586652 conn(0x7f7008065960 msgr2=0x7f7008065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:12.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.163+0000 7f700e8d8640 1 -- 172.21.15.115:0/4044586652 shutdown_connections 2023-12-01T01:49:12.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.163+0000 7f700e8d8640 1 -- 172.21.15.115:0/4044586652 wait complete. 2023-12-01T01:49:12.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.164+0000 7f700e8d8640 1 Processor -- start 2023-12-01T01:49:12.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.165+0000 7f700e8d8640 1 -- start start 2023-12-01T01:49:12.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.165+0000 7f700e8d8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70080b0e60 0x7f700814f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:12.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700e8d8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70080ce1b0 0x7f700806b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:12.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700e8d8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f700815ac80 0x7f700806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700e8d8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008150b50 con 0x7f700815ac80 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700e8d8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70081502b0 con 0x7f70080ce1b0 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700e8d8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7008150590 con 0x7f70080b0e60 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700ce75640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f700815ac80 0x7f700806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700ce75640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f700815ac80 0x7f700806b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48528/0 (socket says 172.21.15.115:48528) 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700ce75640 1 -- 172.21.15.115:0/516481805 learned_addr learned my addr 172.21.15.115:0/516481805 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700ce75640 1 -- 172.21.15.115:0/516481805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70080b0e60 msgr2=0x7f700814f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700ce75640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70080b0e60 0x7f700814f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700ce75640 1 -- 172.21.15.115:0/516481805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70080ce1b0 msgr2=0x7f700806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700ce75640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70080ce1b0 0x7f700806b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:12.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.166+0000 7f700ce75640 1 -- 172.21.15.115:0/516481805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000009a70 con 0x7f700815ac80 2023-12-01T01:49:12.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.167+0000 7f700ce75640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f700815ac80 0x7f700806b7a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7000009d90 tx=0x7f7000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:12.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.167+0000 7f7004ff9640 1 -- 172.21.15.115:0/516481805 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000040650 con 0x7f700815ac80 2023-12-01T01:49:12.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.167+0000 7f7004ff9640 1 -- 172.21.15.115:0/516481805 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70000407b0 con 0x7f700815ac80 2023-12-01T01:49:12.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.167+0000 7f700e8d8640 1 -- 172.21.15.115:0/516481805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7008150870 con 0x7f700815ac80 2023-12-01T01:49:12.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.167+0000 7f7004ff9640 1 -- 172.21.15.115:0/516481805 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000040b60 con 0x7f700815ac80 2023-12-01T01:49:12.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.167+0000 7f700e8d8640 1 -- 172.21.15.115:0/516481805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7008150060 con 0x7f700815ac80 2023-12-01T01:49:12.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.168+0000 7f700e8d8640 1 -- 172.21.15.115:0/516481805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7008152ec0 con 0x7f700815ac80 2023-12-01T01:49:12.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.169+0000 7f7004ff9640 1 -- 172.21.15.115:0/516481805 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f7000040cc0 con 0x7f700815ac80 2023-12-01T01:49:12.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.170+0000 7f7004ff9640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ff406a550 0x7f6ff406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:12.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.170+0000 7f7004ff9640 1 -- 172.21.15.115:0/516481805 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70000ab900 con 0x7f700815ac80 2023-12-01T01:49:12.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.170+0000 7f7007fff640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ff406a550 0x7f6ff406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:12.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.171+0000 7f7007fff640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ff406a550 0x7f6ff406ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8001570 tx=0x7f6ff8007930 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:12.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.174+0000 7f7004ff9640 1 -- 172.21.15.115:0/516481805 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7008152ec0 con 0x7f700815ac80 2023-12-01T01:49:12.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:12.402+0000 7f700e8d8640 1 -- 172.21.15.115:0/516481805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7008152ec0 con 0x7f700815ac80 2023-12-01T01:49:13.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.116+0000 7f7004ff9640 1 -- 172.21.15.115:0/516481805 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7008152ec0 con 0x7f700815ac80 2023-12-01T01:49:13.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.119+0000 7f700e8d8640 1 -- 172.21.15.115:0/516481805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ff406a550 msgr2=0x7f6ff406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:13.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.119+0000 7f700e8d8640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ff406a550 0x7f6ff406ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8001570 tx=0x7f6ff8007930 comp rx=0 tx=0).stop 2023-12-01T01:49:13.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.119+0000 7f700e8d8640 1 -- 172.21.15.115:0/516481805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f700815ac80 msgr2=0x7f700806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:13.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.119+0000 7f700e8d8640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f700815ac80 0x7f700806b7a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7000009d90 tx=0x7f7000004a40 comp rx=0 tx=0).stop 2023-12-01T01:49:13.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.120+0000 7f700e8d8640 1 -- 172.21.15.115:0/516481805 shutdown_connections 2023-12-01T01:49:13.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.120+0000 7f700e8d8640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ff406a550 0x7f6ff406ca10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:13.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.120+0000 7f700e8d8640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f700815ac80 0x7f700806b7a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:13.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.120+0000 7f700e8d8640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70080ce1b0 0x7f700806b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:13.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.120+0000 7f700e8d8640 1 --2- 172.21.15.115:0/516481805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70080b0e60 0x7f700814f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:13.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.120+0000 7f700e8d8640 1 -- 172.21.15.115:0/516481805 >> 172.21.15.115:0/516481805 conn(0x7f7008065960 msgr2=0x7f70080b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:13.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.120+0000 7f700e8d8640 1 -- 172.21.15.115:0/516481805 shutdown_connections 2023-12-01T01:49:13.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.120+0000 7f700e8d8640 1 -- 172.21.15.115:0/516481805 wait complete. 2023-12-01T01:49:13.141 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T01:49:13.141 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T01:49:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.356+0000 7f8adeb11640 1 Processor -- start 2023-12-01T01:49:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.356+0000 7f8adeb11640 1 -- start start 2023-12-01T01:49:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.357+0000 7f8adeb11640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8ad815b0e0 0x7f8ad8164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.357+0000 7f8adeb11640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad80b2f30 con 0x7f8ad8142870 2023-12-01T01:49:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.357+0000 7f8adeb11640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad80b2320 con 0x7f8ad80cd1e0 2023-12-01T01:49:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.357+0000 7f8adeb11640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad80b3ae0 con 0x7f8ad815b0e0 2023-12-01T01:49:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.358+0000 7f8adc8ad640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8ad8142870 0x7f8ad80ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58492/0 (socket says 172.21.15.115:58492) 2023-12-01T01:49:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.358+0000 7f8adc8ad640 1 -- 172.21.15.115:0/3910509877 learned_addr learned my addr 172.21.15.115:0/3910509877 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:13.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.358+0000 7f8add0ae640 1 --2- 172.21.15.115:0/3910509877 >> v2:172.21.15.115:3300/0 conn(0x7f8ad815b0e0 0x7f8ad8164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.358+0000 7f8ad77fe640 1 -- 172.21.15.115:0/3910509877 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943577622 0 0) 0x7f8ad80b2f30 con 0x7f8ad8142870 2023-12-01T01:49:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.358+0000 7f8ad77fe640 1 -- 172.21.15.115:0/3910509877 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ac8003520 con 0x7f8ad8142870 2023-12-01T01:49:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.358+0000 7f8add0ae640 1 -- 172.21.15.115:0/3910509877 >> v1:172.21.15.142:6789/0 conn(0x7f8ad80cd1e0 legacy=0x7f8ad80b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.358+0000 7f8add0ae640 1 -- 172.21.15.115:0/3910509877 >> v1:172.21.15.115:6789/0 conn(0x7f8ad8142870 legacy=0x7f8ad80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:13.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.358+0000 7f8add0ae640 1 -- 172.21.15.115:0/3910509877 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad80b2600 con 0x7f8ad815b0e0 2023-12-01T01:49:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.359+0000 7f8add0ae640 1 --2- 172.21.15.115:0/3910509877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad815b0e0 0x7f8ad8164a20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009d90 tx=0x7f8ad002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9149aefab52eb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.360+0000 7f8ad77fe640 1 -- 172.21.15.115:0/3910509877 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad002cc60 con 0x7f8ad815b0e0 2023-12-01T01:49:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.360+0000 7f8ad77fe640 1 -- 172.21.15.115:0/3910509877 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad002cdc0 con 0x7f8ad815b0e0 2023-12-01T01:49:13.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.360+0000 7f8ad77fe640 1 -- 172.21.15.115:0/3910509877 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0034660 con 0x7f8ad815b0e0 2023-12-01T01:49:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.360+0000 7f8adeb11640 1 -- 172.21.15.115:0/3910509877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad815b0e0 msgr2=0x7f8ad8164a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.360+0000 7f8adeb11640 1 --2- 172.21.15.115:0/3910509877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad815b0e0 0x7f8ad8164a20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009d90 tx=0x7f8ad002beb0 comp rx=0 tx=0).stop 2023-12-01T01:49:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.360+0000 7f8adeb11640 1 -- 172.21.15.115:0/3910509877 shutdown_connections 2023-12-01T01:49:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.360+0000 7f8adeb11640 1 --2- 172.21.15.115:0/3910509877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad815b0e0 0x7f8ad8164a20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.360+0000 7f8adeb11640 1 -- 172.21.15.115:0/3910509877 >> 172.21.15.115:0/3910509877 conn(0x7f8ad8065960 msgr2=0x7f8ad8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.361+0000 7f8adeb11640 1 -- 172.21.15.115:0/3910509877 shutdown_connections 2023-12-01T01:49:13.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.361+0000 7f8adeb11640 1 -- 172.21.15.115:0/3910509877 wait complete. 2023-12-01T01:49:13.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.362+0000 7f8adeb11640 1 Processor -- start 2023-12-01T01:49:13.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.363+0000 7f8adeb11640 1 -- start start 2023-12-01T01:49:13.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.363+0000 7f8adeb11640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad80cd1e0 0x7f8ad813a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:13.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.363+0000 7f8adeb11640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad8142870 0x7f8ad814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:13.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.363+0000 7f8adeb11640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ad814a6b0 0x7f8ad8140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:13.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8adeb11640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80af890 con 0x7f8ad8142870 2023-12-01T01:49:13.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8adeb11640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80aec80 con 0x7f8ad814a6b0 2023-12-01T01:49:13.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8adeb11640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80aef60 con 0x7f8ad80cd1e0 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad8142870 0x7f8ad814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad8142870 0x7f8ad814a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48536/0 (socket says 172.21.15.115:48536) 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8ad7fff640 1 -- 172.21.15.115:0/2627156168 learned_addr learned my addr 172.21.15.115:0/2627156168 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8adc8ad640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad80cd1e0 0x7f8ad813a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8ad7fff640 1 -- 172.21.15.115:0/2627156168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad80cd1e0 msgr2=0x7f8ad813a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8ad7fff640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad80cd1e0 0x7f8ad813a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8ad7fff640 1 -- 172.21.15.115:0/2627156168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ad814a6b0 msgr2=0x7f8ad8140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8ad7fff640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ad814a6b0 0x7f8ad8140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8ad7fff640 1 -- 172.21.15.115:0/2627156168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad0009a70 con 0x7f8ad8142870 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.364+0000 7f8adc8ad640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad80cd1e0 0x7f8ad813a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.365+0000 7f8ad7fff640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad8142870 0x7f8ad814a170 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400c430 tx=0x7f8ac400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.365+0000 7f8ad57fa640 1 -- 172.21.15.115:0/2627156168 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4011790 con 0x7f8ad8142870 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.365+0000 7f8ad57fa640 1 -- 172.21.15.115:0/2627156168 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac40118f0 con 0x7f8ad8142870 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.365+0000 7f8adeb11640 1 -- 172.21.15.115:0/2627156168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad80af240 con 0x7f8ad8142870 2023-12-01T01:49:13.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.365+0000 7f8ad57fa640 1 -- 172.21.15.115:0/2627156168 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4011ce0 con 0x7f8ad8142870 2023-12-01T01:49:13.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.365+0000 7f8adeb11640 1 -- 172.21.15.115:0/2627156168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad813a630 con 0x7f8ad8142870 2023-12-01T01:49:13.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.366+0000 7f8adeb11640 1 -- 172.21.15.115:0/2627156168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad80afa40 con 0x7f8ad8142870 2023-12-01T01:49:13.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.367+0000 7f8ad57fa640 1 -- 172.21.15.115:0/2627156168 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8ac401c460 con 0x7f8ad8142870 2023-12-01T01:49:13.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.368+0000 7f8ad57fa640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8abc06a580 0x7f8abc06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:13.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.368+0000 7f8ad57fa640 1 -- 172.21.15.115:0/2627156168 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ac4088a70 con 0x7f8ad8142870 2023-12-01T01:49:13.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.368+0000 7f8adc8ad640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8abc06a580 0x7f8abc06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:13.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.369+0000 7f8adc8ad640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8abc06a580 0x7f8abc06ca40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0011e0 tx=0x7f8acc009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:13.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.372+0000 7f8ad57fa640 1 -- 172.21.15.115:0/2627156168 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8ac4053eb0 con 0x7f8ad8142870 2023-12-01T01:49:13.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:13.600+0000 7f8adeb11640 1 -- 172.21.15.115:0/2627156168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8ad804e890 con 0x7f8ad8142870 2023-12-01T01:49:14.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.118+0000 7f8ad57fa640 1 -- 172.21.15.115:0/2627156168 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8ac40574d0 con 0x7f8ad8142870 2023-12-01T01:49:14.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.122+0000 7f8adeb11640 1 -- 172.21.15.115:0/2627156168 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8abc06a580 msgr2=0x7f8abc06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.122+0000 7f8adeb11640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8abc06a580 0x7f8abc06ca40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0011e0 tx=0x7f8acc009040 comp rx=0 tx=0).stop 2023-12-01T01:49:14.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.122+0000 7f8adeb11640 1 -- 172.21.15.115:0/2627156168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad8142870 msgr2=0x7f8ad814a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.122+0000 7f8adeb11640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad8142870 0x7f8ad814a170 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400c430 tx=0x7f8ac400c8f0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.123+0000 7f8adeb11640 1 -- 172.21.15.115:0/2627156168 shutdown_connections 2023-12-01T01:49:14.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.123+0000 7f8adeb11640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8abc06a580 0x7f8abc06ca40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.123+0000 7f8adeb11640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ad814a6b0 0x7f8ad8140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.123+0000 7f8adeb11640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ad8142870 0x7f8ad814a170 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.123+0000 7f8adeb11640 1 --2- 172.21.15.115:0/2627156168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad80cd1e0 0x7f8ad813a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.123+0000 7f8adeb11640 1 -- 172.21.15.115:0/2627156168 >> 172.21.15.115:0/2627156168 conn(0x7f8ad8065960 msgr2=0x7f8ad813ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:14.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.123+0000 7f8adeb11640 1 -- 172.21.15.115:0/2627156168 shutdown_connections 2023-12-01T01:49:14.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.123+0000 7f8adeb11640 1 -- 172.21.15.115:0/2627156168 wait complete. 2023-12-01T01:49:14.142 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T01:49:14.143 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:49:14.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.353+0000 7f3b18327640 1 Processor -- start 2023-12-01T01:49:14.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.354+0000 7f3b18327640 1 -- start start 2023-12-01T01:49:14.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.355+0000 7f3b18327640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b1015cd20 0x7f3b100d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.355+0000 7f3b18327640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3b100d1230 0x7f3b100d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.355+0000 7f3b18327640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b100ce2e0 con 0x7f3b100b5d80 2023-12-01T01:49:14.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.355+0000 7f3b18327640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b100b2160 con 0x7f3b100d1230 2023-12-01T01:49:14.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.355+0000 7f3b18327640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100d1be0 con 0x7f3b1015cd20 2023-12-01T01:49:14.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.355+0000 7f3b168c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3b100d1230 0x7f3b100d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:14.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.356+0000 7f3b168c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3b100d1230 0x7f3b100d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48542/0 (socket says 172.21.15.115:48542) 2023-12-01T01:49:14.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.356+0000 7f3b168c4640 1 -- 172.21.15.115:0/502654328 learned_addr learned my addr 172.21.15.115:0/502654328 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:14.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.356+0000 7f3b160c3640 1 --2- 172.21.15.115:0/502654328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b1015cd20 0x7f3b100d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:14.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.356+0000 7f3b168c4640 1 -- 172.21.15.115:0/502654328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b1015cd20 msgr2=0x7f3b100d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:14.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.356+0000 7f3b168c4640 1 --2- 172.21.15.115:0/502654328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b1015cd20 0x7f3b100d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.356+0000 7f3b168c4640 1 -- 172.21.15.115:0/502654328 >> v1:172.21.15.142:6789/0 conn(0x7f3b100b5d80 legacy=0x7f3b100b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.356+0000 7f3b168c4640 1 -- 172.21.15.115:0/502654328 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b100ce000 con 0x7f3b100d1230 2023-12-01T01:49:14.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.357+0000 7f3b168c4640 1 --2- 172.21.15.115:0/502654328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 0x7f3b100d16a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009e50 tx=0x7f3b0802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db6d30224d6ba3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:14.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.358+0000 7f3b150c1640 1 -- 172.21.15.115:0/502654328 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0802ca70 con 0x7f3b100d1230 2023-12-01T01:49:14.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.358+0000 7f3b150c1640 1 -- 172.21.15.115:0/502654328 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0802cbd0 con 0x7f3b100d1230 2023-12-01T01:49:14.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.358+0000 7f3b150c1640 1 -- 172.21.15.115:0/502654328 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080343f0 con 0x7f3b100d1230 2023-12-01T01:49:14.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.358+0000 7f3b18327640 1 -- 172.21.15.115:0/502654328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 msgr2=0x7f3b100d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.358+0000 7f3b18327640 1 --2- 172.21.15.115:0/502654328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 0x7f3b100d16a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009e50 tx=0x7f3b0802bed0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.358+0000 7f3b18327640 1 -- 172.21.15.115:0/502654328 shutdown_connections 2023-12-01T01:49:14.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.358+0000 7f3b18327640 1 --2- 172.21.15.115:0/502654328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 0x7f3b100d16a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.359+0000 7f3b18327640 1 --2- 172.21.15.115:0/502654328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b1015cd20 0x7f3b100d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.359+0000 7f3b18327640 1 -- 172.21.15.115:0/502654328 >> 172.21.15.115:0/502654328 conn(0x7f3b1006a0e0 msgr2=0x7f3b1006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:14.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.359+0000 7f3b18327640 1 -- 172.21.15.115:0/502654328 shutdown_connections 2023-12-01T01:49:14.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.359+0000 7f3b18327640 1 -- 172.21.15.115:0/502654328 wait complete. 2023-12-01T01:49:14.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.360+0000 7f3b18327640 1 Processor -- start 2023-12-01T01:49:14.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.360+0000 7f3b18327640 1 -- start start 2023-12-01T01:49:14.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.361+0000 7f3b18327640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b100b5d80 0x7f3b100b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.361+0000 7f3b18327640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 0x7f3b100b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.361+0000 7f3b18327640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b1015cd20 0x7f3b100b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.361+0000 7f3b18327640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100c6440 con 0x7f3b100d1230 2023-12-01T01:49:14.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.361+0000 7f3b18327640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100b2de0 con 0x7f3b1015cd20 2023-12-01T01:49:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.361+0000 7f3b18327640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b100cc9a0 con 0x7f3b100b5d80 2023-12-01T01:49:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.362+0000 7f3b158c2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 0x7f3b100b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.362+0000 7f3b158c2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 0x7f3b100b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48554/0 (socket says 172.21.15.115:48554) 2023-12-01T01:49:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.362+0000 7f3b158c2640 1 -- 172.21.15.115:0/1515184133 learned_addr learned my addr 172.21.15.115:0/1515184133 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.362+0000 7f3b158c2640 1 -- 172.21.15.115:0/1515184133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b100b5d80 msgr2=0x7f3b100b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.362+0000 7f3b158c2640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b100b5d80 0x7f3b100b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.362+0000 7f3b158c2640 1 -- 172.21.15.115:0/1515184133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b1015cd20 msgr2=0x7f3b100b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.362+0000 7f3b158c2640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b1015cd20 0x7f3b100b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.362+0000 7f3b158c2640 1 -- 172.21.15.115:0/1515184133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08009b30 con 0x7f3b100d1230 2023-12-01T01:49:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.362+0000 7f3b158c2640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 0x7f3b100b41e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000c430 tx=0x7f3b0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.363+0000 7f3b06ffd640 1 -- 172.21.15.115:0/1515184133 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000052d0 con 0x7f3b100d1230 2023-12-01T01:49:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.363+0000 7f3b06ffd640 1 -- 172.21.15.115:0/1515184133 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b00005430 con 0x7f3b100d1230 2023-12-01T01:49:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.363+0000 7f3b06ffd640 1 -- 172.21.15.115:0/1515184133 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00003830 con 0x7f3b100d1230 2023-12-01T01:49:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.363+0000 7f3b18327640 1 -- 172.21.15.115:0/1515184133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b100cbd90 con 0x7f3b100d1230 2023-12-01T01:49:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.363+0000 7f3b18327640 1 -- 172.21.15.115:0/1515184133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b100cc350 con 0x7f3b100d1230 2023-12-01T01:49:14.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.364+0000 7f3b18327640 1 -- 172.21.15.115:0/1515184133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1004e890 con 0x7f3b100d1230 2023-12-01T01:49:14.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.365+0000 7f3b06ffd640 1 -- 172.21.15.115:0/1515184133 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3b00003990 con 0x7f3b100d1230 2023-12-01T01:49:14.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.366+0000 7f3b06ffd640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3af406a550 0x7f3af406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.367+0000 7f3b06ffd640 1 -- 172.21.15.115:0/1515184133 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b00088100 con 0x7f3b100d1230 2023-12-01T01:49:14.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.367+0000 7f3b160c3640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3af406a550 0x7f3af406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:14.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.367+0000 7f3b160c3640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3af406a550 0x7f3af406ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c006fd0 tx=0x7f3b0c008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:14.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.371+0000 7f3b06ffd640 1 -- 172.21.15.115:0/1515184133 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3b00053540 con 0x7f3b100d1230 2023-12-01T01:49:14.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.600+0000 7f3b18327640 1 -- 172.21.15.115:0/1515184133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b100ccb10 con 0x7f3b100d1230 2023-12-01T01:49:14.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.601+0000 7f3b06ffd640 1 -- 172.21.15.115:0/1515184133 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f3b00056b60 con 0x7f3b100d1230 2023-12-01T01:49:14.603 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:14.603 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/2548453767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2548453767},{"type":"v1","addr":"172.21.15.142:6834","nonce":2548453767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3727358840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3727358840},{"type":"v1","addr":"172.21.15.142:6835","nonce":3727358840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/4086473290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4086473290},{"type":"v1","addr":"172.21.15.115:6836","nonce":4086473290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:49:11.094285+0000","modified":"2023-12-01T01:49:14.112475+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1756585806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1756585806},{"type":"v1","addr":"172.21.15.115:6837","nonce":1756585806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-01T01:49:14.603 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 24 2023-12-01T01:49:14.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 -- 172.21.15.115:0/1515184133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3af406a550 msgr2=0x7f3af406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3af406a550 0x7f3af406ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c006fd0 tx=0x7f3b0c008040 comp rx=0 tx=0).stop 2023-12-01T01:49:14.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 -- 172.21.15.115:0/1515184133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 msgr2=0x7f3b100b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 0x7f3b100b41e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000c430 tx=0x7f3b0000c8f0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 -- 172.21.15.115:0/1515184133 shutdown_connections 2023-12-01T01:49:14.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3af406a550 0x7f3af406ca10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b1015cd20 0x7f3b100b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b100d1230 0x7f3b100b41e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 --2- 172.21.15.115:0/1515184133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b100b5d80 0x7f3b100b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 -- 172.21.15.115:0/1515184133 >> 172.21.15.115:0/1515184133 conn(0x7f3b1006a0e0 msgr2=0x7f3b10157500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:14.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 -- 172.21.15.115:0/1515184133 shutdown_connections 2023-12-01T01:49:14.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.604+0000 7f3b18327640 1 -- 172.21.15.115:0/1515184133 wait complete. 2023-12-01T01:49:14.625 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:49:14.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.836+0000 7f3690fab640 1 Processor -- start 2023-12-01T01:49:14.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.836+0000 7f3690fab640 1 -- start start 2023-12-01T01:49:14.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.837+0000 7f3690fab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f368c15e2f0 0x7f368c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.837+0000 7f3690fab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f368c0adf10 0x7f368c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.837+0000 7f3690fab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c0add10 con 0x7f368c06f2f0 2023-12-01T01:49:14.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.837+0000 7f3690fab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c0c5760 con 0x7f368c15e2f0 2023-12-01T01:49:14.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.837+0000 7f3690fab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c15e6d0 con 0x7f368c0adf10 2023-12-01T01:49:14.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f368a59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f368c15e2f0 0x7f368c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:14.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f3683fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f368c06f2f0 0x7f368c0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58498/0 (socket says 172.21.15.115:58498) 2023-12-01T01:49:14.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f368a59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f368c15e2f0 0x7f368c0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48558/0 (socket says 172.21.15.115:48558) 2023-12-01T01:49:14.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f3683fff640 1 -- 172.21.15.115:0/978891311 learned_addr learned my addr 172.21.15.115:0/978891311 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:14.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f368ad9d640 1 --2- 172.21.15.115:0/978891311 >> v2:172.21.15.142:3300/0 conn(0x7f368c0adf10 0x7f368c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:14.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f368a59c640 1 -- 172.21.15.115:0/978891311 >> v2:172.21.15.142:3300/0 conn(0x7f368c0adf10 msgr2=0x7f368c0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:14.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f368a59c640 1 --2- 172.21.15.115:0/978891311 >> v2:172.21.15.142:3300/0 conn(0x7f368c0adf10 0x7f368c0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f368a59c640 1 -- 172.21.15.115:0/978891311 >> v1:172.21.15.115:6789/0 conn(0x7f368c06f2f0 legacy=0x7f368c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f3689d9b640 1 -- 172.21.15.115:0/978891311 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287023888 0 0) 0x7f368c0add10 con 0x7f368c06f2f0 2023-12-01T01:49:14.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f368a59c640 1 -- 172.21.15.115:0/978891311 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0cf7b0 con 0x7f368c15e2f0 2023-12-01T01:49:14.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.838+0000 7f368ad9d640 1 --2- 172.21.15.115:0/978891311 >> v2:172.21.15.142:3300/0 conn(0x7f368c0adf10 0x7f368c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:14.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.840+0000 7f368a59c640 1 --2- 172.21.15.115:0/978891311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c15e2f0 0x7f368c0ca660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f368c0c8e80 tx=0x7f367c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e13db77fb0d91d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:14.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.840+0000 7f3689d9b640 1 -- 172.21.15.115:0/978891311 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c031710 con 0x7f368c15e2f0 2023-12-01T01:49:14.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.840+0000 7f3689d9b640 1 -- 172.21.15.115:0/978891311 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c02f050 con 0x7f368c15e2f0 2023-12-01T01:49:14.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.840+0000 7f3689d9b640 1 -- 172.21.15.115:0/978891311 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c031a90 con 0x7f368c15e2f0 2023-12-01T01:49:14.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.841+0000 7f3690fab640 1 -- 172.21.15.115:0/978891311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c15e2f0 msgr2=0x7f368c0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.841+0000 7f3690fab640 1 --2- 172.21.15.115:0/978891311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c15e2f0 0x7f368c0ca660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f368c0c8e80 tx=0x7f367c02b7b0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.841+0000 7f3690fab640 1 -- 172.21.15.115:0/978891311 shutdown_connections 2023-12-01T01:49:14.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.841+0000 7f3690fab640 1 --2- 172.21.15.115:0/978891311 >> v2:172.21.15.142:3300/0 conn(0x7f368c0adf10 0x7f368c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.841+0000 7f3690fab640 1 --2- 172.21.15.115:0/978891311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c15e2f0 0x7f368c0ca660 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.841+0000 7f3690fab640 1 -- 172.21.15.115:0/978891311 >> 172.21.15.115:0/978891311 conn(0x7f368c0ce880 msgr2=0x7f368c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:14.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.841+0000 7f3690fab640 1 -- 172.21.15.115:0/978891311 shutdown_connections 2023-12-01T01:49:14.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.841+0000 7f3690fab640 1 -- 172.21.15.115:0/978891311 wait complete. 2023-12-01T01:49:14.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.842+0000 7f3690fab640 1 Processor -- start 2023-12-01T01:49:14.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.843+0000 7f3690fab640 1 -- start start 2023-12-01T01:49:14.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.843+0000 7f3690fab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c06f2f0 0x7f368c1f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.843+0000 7f3690fab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f368c0adf10 0x7f368c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.843+0000 7f3690fab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f368c15e2f0 0x7f368c0acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f3690fab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0c2900 con 0x7f368c06f2f0 2023-12-01T01:49:14.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f3690fab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1f23a0 con 0x7f368c15e2f0 2023-12-01T01:49:14.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f3690fab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0c59f0 con 0x7f368c0adf10 2023-12-01T01:49:14.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f368a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c06f2f0 0x7f368c1f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:14.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f368a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c06f2f0 0x7f368c1f1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48560/0 (socket says 172.21.15.115:48560) 2023-12-01T01:49:14.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f368a59c640 1 -- 172.21.15.115:0/1854564660 learned_addr learned my addr 172.21.15.115:0/1854564660 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:14.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f368a59c640 1 -- 172.21.15.115:0/1854564660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f368c0adf10 msgr2=0x7f368c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f368a59c640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f368c0adf10 0x7f368c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f368a59c640 1 -- 172.21.15.115:0/1854564660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f368c15e2f0 msgr2=0x7f368c0acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:14.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f368a59c640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f368c15e2f0 0x7f368c0acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:14.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f368a59c640 1 -- 172.21.15.115:0/1854564660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c009400 con 0x7f368c06f2f0 2023-12-01T01:49:14.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.844+0000 7f368a59c640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c06f2f0 0x7f368c1f1e60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f367c009890 tx=0x7f367c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:14.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.845+0000 7f3682ffd640 1 -- 172.21.15.115:0/1854564660 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c004230 con 0x7f368c06f2f0 2023-12-01T01:49:14.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.845+0000 7f3682ffd640 1 -- 172.21.15.115:0/1854564660 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c004390 con 0x7f368c06f2f0 2023-12-01T01:49:14.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.845+0000 7f3690fab640 1 -- 172.21.15.115:0/1854564660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c0c5cd0 con 0x7f368c06f2f0 2023-12-01T01:49:14.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.845+0000 7f3682ffd640 1 -- 172.21.15.115:0/1854564660 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c033970 con 0x7f368c06f2f0 2023-12-01T01:49:14.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.845+0000 7f3690fab640 1 -- 172.21.15.115:0/1854564660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c0c6290 con 0x7f368c06f2f0 2023-12-01T01:49:14.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.846+0000 7f3690fab640 1 -- 172.21.15.115:0/1854564660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c06fda0 con 0x7f368c06f2f0 2023-12-01T01:49:14.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.847+0000 7f3682ffd640 1 -- 172.21.15.115:0/1854564660 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f367c03a050 con 0x7f368c06f2f0 2023-12-01T01:49:14.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.848+0000 7f3682ffd640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f366c06a580 0x7f366c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:14.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.848+0000 7f3682ffd640 1 -- 172.21.15.115:0/1854564660 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f367c0aa680 con 0x7f368c06f2f0 2023-12-01T01:49:14.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.848+0000 7f3683fff640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f366c06a580 0x7f366c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:14.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.849+0000 7f3683fff640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f366c06a580 0x7f366c06ca40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3674007fd0 tx=0x7f3674007960 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:14.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:14.852+0000 7f3682ffd640 1 -- 172.21.15.115:0/1854564660 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f367c075b60 con 0x7f368c06f2f0 2023-12-01T01:49:15.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.048+0000 7f3690fab640 1 -- 172.21.15.115:0/1854564660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f368c0c64b0 con 0x7f368c06f2f0 2023-12-01T01:49:15.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.051+0000 7f3682ffd640 1 -- 172.21.15.115:0/1854564660 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16089 (secure 0 0 0) 0x7f367c079180 con 0x7f368c06f2f0 2023-12-01T01:49:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:15.053 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":64,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:49:12.099143+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:49:08.909721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-01T01:49:10.108096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:49:15.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.053+0000 7f3690fab640 1 -- 172.21.15.115:0/1854564660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f366c06a580 msgr2=0x7f366c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.053+0000 7f3690fab640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f366c06a580 0x7f366c06ca40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3674007fd0 tx=0x7f3674007960 comp rx=0 tx=0).stop 2023-12-01T01:49:15.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.054+0000 7f3690fab640 1 -- 172.21.15.115:0/1854564660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c06f2f0 msgr2=0x7f368c1f1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.054+0000 7f3690fab640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c06f2f0 0x7f368c1f1e60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f367c009890 tx=0x7f367c0046c0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.054+0000 7f3690fab640 1 -- 172.21.15.115:0/1854564660 shutdown_connections 2023-12-01T01:49:15.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.054+0000 7f3690fab640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f366c06a580 0x7f366c06ca40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.054+0000 7f3690fab640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f368c0adf10 0x7f368c0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.054+0000 7f3690fab640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f368c15e2f0 0x7f368c0acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.054+0000 7f3690fab640 1 --2- 172.21.15.115:0/1854564660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c06f2f0 0x7f368c1f1e60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.054+0000 7f3690fab640 1 -- 172.21.15.115:0/1854564660 >> 172.21.15.115:0/1854564660 conn(0x7f368c0ce880 msgr2=0x7f368c06b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:15.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.054+0000 7f3690fab640 1 -- 172.21.15.115:0/1854564660 shutdown_connections 2023-12-01T01:49:15.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.054+0000 7f3690fab640 1 -- 172.21.15.115:0/1854564660 wait complete. 2023-12-01T01:49:15.074 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T01:49:15.074 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:49:15.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.287+0000 7fef7d084640 1 Processor -- start 2023-12-01T01:49:15.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.288+0000 7fef7d084640 1 -- start start 2023-12-01T01:49:15.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.289+0000 7fef7d084640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780b5d80 0x7fef780d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.289+0000 7fef7d084640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fef780d1230 0x7fef780d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.289+0000 7fef7d084640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef780ce2e0 con 0x7fef7815cd20 2023-12-01T01:49:15.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.289+0000 7fef7d084640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef780b2160 con 0x7fef780d1230 2023-12-01T01:49:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.289+0000 7fef7d084640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef780d1be0 con 0x7fef780b5d80 2023-12-01T01:49:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.290+0000 7fef7759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fef780d1230 0x7fef780d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.290+0000 7fef7759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fef780d1230 0x7fef780d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:54490/0 (socket says 172.21.15.115:54490) 2023-12-01T01:49:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.290+0000 7fef7759e640 1 -- 172.21.15.115:0/2138033557 learned_addr learned my addr 172.21.15.115:0/2138033557 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:15.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.290+0000 7fef6ffff640 1 --2- 172.21.15.115:0/2138033557 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780b5d80 0x7fef780d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.291+0000 7fef7759e640 1 -- 172.21.15.115:0/2138033557 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780b5d80 msgr2=0x7fef780d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.291+0000 7fef7759e640 1 --2- 172.21.15.115:0/2138033557 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780b5d80 0x7fef780d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.291+0000 7fef7759e640 1 -- 172.21.15.115:0/2138033557 >> v1:172.21.15.142:6789/0 conn(0x7fef7815cd20 legacy=0x7fef780b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.291+0000 7fef7759e640 1 -- 172.21.15.115:0/2138033557 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780ce000 con 0x7fef780d1230 2023-12-01T01:49:15.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.291+0000 7fef6ffff640 1 --2- 172.21.15.115:0/2138033557 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780b5d80 0x7fef780d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:49:15.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.292+0000 7fef7759e640 1 --2- 172.21.15.115:0/2138033557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef780d1230 0x7fef780d16a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fef70009e50 tx=0x7fef7002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18244f510f15bad4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.292+0000 7fef7659c640 1 -- 172.21.15.115:0/2138033557 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7002ca80 con 0x7fef780d1230 2023-12-01T01:49:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.292+0000 7fef7659c640 1 -- 172.21.15.115:0/2138033557 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7002cbe0 con 0x7fef780d1230 2023-12-01T01:49:15.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.292+0000 7fef7659c640 1 -- 172.21.15.115:0/2138033557 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef700343f0 con 0x7fef780d1230 2023-12-01T01:49:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.293+0000 7fef7d084640 1 -- 172.21.15.115:0/2138033557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef780d1230 msgr2=0x7fef780d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.293+0000 7fef7d084640 1 --2- 172.21.15.115:0/2138033557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef780d1230 0x7fef780d16a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fef70009e50 tx=0x7fef7002bee0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.293+0000 7fef7d084640 1 -- 172.21.15.115:0/2138033557 shutdown_connections 2023-12-01T01:49:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.293+0000 7fef7d084640 1 --2- 172.21.15.115:0/2138033557 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780b5d80 0x7fef780d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.293+0000 7fef7d084640 1 --2- 172.21.15.115:0/2138033557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef780d1230 0x7fef780d16a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.293+0000 7fef7d084640 1 -- 172.21.15.115:0/2138033557 >> 172.21.15.115:0/2138033557 conn(0x7fef7806a0e0 msgr2=0x7fef7806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.293+0000 7fef7d084640 1 -- 172.21.15.115:0/2138033557 shutdown_connections 2023-12-01T01:49:15.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.293+0000 7fef7d084640 1 -- 172.21.15.115:0/2138033557 wait complete. 2023-12-01T01:49:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.295+0000 7fef7d084640 1 Processor -- start 2023-12-01T01:49:15.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.295+0000 7fef7d084640 1 -- start start 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.296+0000 7fef7d084640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef780b5d80 0x7fef780b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.296+0000 7fef7d084640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780d1230 0x7fef781f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.296+0000 7fef7d084640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef7815cd20 0x7fef780cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.296+0000 7fef7d084640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef781f6460 con 0x7fef780b5d80 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.296+0000 7fef7d084640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef780ccb40 con 0x7fef7815cd20 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.296+0000 7fef7d084640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7815a970 con 0x7fef780d1230 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.296+0000 7fef76d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef780b5d80 0x7fef780b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.296+0000 7fef76d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef780b5d80 0x7fef780b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48576/0 (socket says 172.21.15.115:48576) 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.296+0000 7fef76d9d640 1 -- 172.21.15.115:0/2458277620 learned_addr learned my addr 172.21.15.115:0/2458277620 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.297+0000 7fef6ffff640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780d1230 0x7fef781f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:15.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.297+0000 7fef7759e640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef7815cd20 0x7fef780cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.297+0000 7fef76d9d640 1 -- 172.21.15.115:0/2458277620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780d1230 msgr2=0x7fef781f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.297+0000 7fef76d9d640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780d1230 0x7fef781f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.297+0000 7fef76d9d640 1 -- 172.21.15.115:0/2458277620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef7815cd20 msgr2=0x7fef780cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.297+0000 7fef76d9d640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef7815cd20 0x7fef780cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.297+0000 7fef76d9d640 1 -- 172.21.15.115:0/2458277620 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef60009400 con 0x7fef780b5d80 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.297+0000 7fef76d9d640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef780b5d80 0x7fef780b4c00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c400 tx=0x7fef6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.298+0000 7fef6f7fe640 1 -- 172.21.15.115:0/2458277620 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680052d0 con 0x7fef780b5d80 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.298+0000 7fef6f7fe640 1 -- 172.21.15.115:0/2458277620 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef68005430 con 0x7fef780b5d80 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.298+0000 7fef7d084640 1 -- 172.21.15.115:0/2458277620 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef70009b30 con 0x7fef780b5d80 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.298+0000 7fef6f7fe640 1 -- 172.21.15.115:0/2458277620 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68003830 con 0x7fef780b5d80 2023-12-01T01:49:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.298+0000 7fef7d084640 1 -- 172.21.15.115:0/2458277620 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7815af30 con 0x7fef780b5d80 2023-12-01T01:49:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.299+0000 7fef7d084640 1 -- 172.21.15.115:0/2458277620 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7804e890 con 0x7fef780b5d80 2023-12-01T01:49:15.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.300+0000 7fef6f7fe640 1 -- 172.21.15.115:0/2458277620 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fef68003990 con 0x7fef780b5d80 2023-12-01T01:49:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.301+0000 7fef6f7fe640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef5806a550 0x7fef5806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.302+0000 7fef6f7fe640 1 -- 172.21.15.115:0/2458277620 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef680880e0 con 0x7fef780b5d80 2023-12-01T01:49:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.302+0000 7fef6ffff640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef5806a550 0x7fef5806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:15.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.302+0000 7fef6ffff640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef5806a550 0x7fef5806ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef781545f0 tx=0x7fef60008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:15.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.306+0000 7fef6f7fe640 1 -- 172.21.15.115:0/2458277620 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fef68053520 con 0x7fef780b5d80 2023-12-01T01:49:15.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.495+0000 7fef7d084640 1 -- 172.21.15.115:0/2458277620 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef780cd3d0 con 0x7fef5806a550 2023-12-01T01:49:15.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.507+0000 7fef6f7fe640 1 -- 172.21.15.115:0/2458277620 <== mgr.5706 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275733 (secure 0 0 0) 0x7fef60011e60 con 0x7fef5806a550 2023-12-01T01:49:15.512 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:49:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 -- 172.21.15.115:0/2458277620 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef5806a550 msgr2=0x7fef5806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef5806a550 0x7fef5806ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef781545f0 tx=0x7fef60008040 comp rx=0 tx=0).stop 2023-12-01T01:49:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 -- 172.21.15.115:0/2458277620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef780b5d80 msgr2=0x7fef780b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef780b5d80 0x7fef780b4c00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c400 tx=0x7fef6800c8c0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 -- 172.21.15.115:0/2458277620 shutdown_connections 2023-12-01T01:49:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fef5806a550 0x7fef5806ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef780d1230 0x7fef781f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef7815cd20 0x7fef780cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 --2- 172.21.15.115:0/2458277620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef780b5d80 0x7fef780b4c00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 -- 172.21.15.115:0/2458277620 >> 172.21.15.115:0/2458277620 conn(0x7fef7806a0e0 msgr2=0x7fef78158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.513+0000 7fef7d084640 1 -- 172.21.15.115:0/2458277620 shutdown_connections 2023-12-01T01:49:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.514+0000 7fef7d084640 1 -- 172.21.15.115:0/2458277620 wait complete. 2023-12-01T01:49:15.533 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T01:49:13.893152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259232,"kb_used_data":6752,"kb_used_omap":0,"kb_used_meta":252416,"kb_avail":754715488,"statfs":{"total":773094113280,"available":772828659712,"internally_reserved":0,"allocated":6914048,"data_stored":3829298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":258473984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.293036"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.618015+0000","last_change":"2023-12-01T01:49:09.618015+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.618015+0000","last_undegraded":"2023-12-01T01:49:09.618015+0000","last_fullsized":"2023-12-01T01:49:09.618015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:42.308459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.621893+0000","last_change":"2023-12-01T01:49:09.621893+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.621893+0000","last_undegraded":"2023-12-01T01:49:09.621893+0000","last_fullsized":"2023-12-01T01:49:09.621893+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:28.123293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.616187+0000","last_change":"2023-12-01T01:49:09.616187+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.616187+0000","last_undegraded":"2023-12-01T01:49:09.616187+0000","last_fullsized":"2023-12-01T01:49:09.616187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:27:14.569892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.620966+0000","last_change":"2023-12-01T01:49:09.620966+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.620966+0000","last_undegraded":"2023-12-01T01:49:09.620966+0000","last_fullsized":"2023-12-01T01:49:09.620966+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:50:31.691006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.614895+0000","last_change":"2023-12-01T01:49:09.614895+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.614895+0000","last_undegraded":"2023-12-01T01:49:09.614895+0000","last_fullsized":"2023-12-01T01:49:09.614895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:17:03.520261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.622458+0000","last_change":"2023-12-01T01:49:09.622458+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.622458+0000","last_undegraded":"2023-12-01T01:49:09.622458+0000","last_fullsized":"2023-12-01T01:49:09.622458+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:26:01.897486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.609820+0000","last_change":"2023-12-01T01:49:09.609820+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.609820+0000","last_undegraded":"2023-12-01T01:49:09.609820+0000","last_fullsized":"2023-12-01T01:49:09.609820+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:27:14.569892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.610753+0000","last_change":"2023-12-01T01:49:09.610753+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.610753+0000","last_undegraded":"2023-12-01T01:49:09.610753+0000","last_fullsized":"2023-12-01T01:49:09.610753+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:24.143826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.605439+0000","last_change":"2023-12-01T01:49:09.605439+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.605439+0000","last_undegraded":"2023-12-01T01:49:09.605439+0000","last_fullsized":"2023-12-01T01:49:09.605439+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:20:16.537047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.610783+0000","last_change":"2023-12-01T01:49:09.610783+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.610783+0000","last_undegraded":"2023-12-01T01:49:09.610783+0000","last_fullsized":"2023-12-01T01:49:09.610783+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:42.308459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.607264+0000","last_change":"2023-12-01T01:49:09.607264+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.607264+0000","last_undegraded":"2023-12-01T01:49:09.607264+0000","last_fullsized":"2023-12-01T01:49:09.607264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:25:00.138088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.614393+0000","last_change":"2023-12-01T01:49:09.614393+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.614393+0000","last_undegraded":"2023-12-01T01:49:09.614393+0000","last_fullsized":"2023-12-01T01:49:09.614393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:10:57.107037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.611633+0000","last_change":"2023-12-01T01:49:09.611633+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.611633+0000","last_undegraded":"2023-12-01T01:49:09.611633+0000","last_fullsized":"2023-12-01T01:49:09.611633+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:07.421392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.609880+0000","last_change":"2023-12-01T01:49:09.609880+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.609880+0000","last_undegraded":"2023-12-01T01:49:09.609880+0000","last_fullsized":"2023-12-01T01:49:09.609880+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:42.308459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.618488+0000","last_change":"2023-12-01T01:49:09.618488+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.618488+0000","last_undegraded":"2023-12-01T01:49:09.618488+0000","last_fullsized":"2023-12-01T01:49:09.618488+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:55.774591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.612282+0000","last_change":"2023-12-01T01:49:09.612282+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.612282+0000","last_undegraded":"2023-12-01T01:49:09.612282+0000","last_fullsized":"2023-12-01T01:49:09.612282+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:14.976834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.620719+0000","last_change":"2023-12-01T01:49:09.620719+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.620719+0000","last_undegraded":"2023-12-01T01:49:09.620719+0000","last_fullsized":"2023-12-01T01:49:09.620719+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:44.215676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.605963+0000","last_change":"2023-12-01T01:49:09.605963+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.605963+0000","last_undegraded":"2023-12-01T01:49:09.605963+0000","last_fullsized":"2023-12-01T01:49:09.605963+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:18:13.563492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.605242+0000","last_change":"2023-12-01T01:49:09.605242+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.605242+0000","last_undegraded":"2023-12-01T01:49:09.605242+0000","last_fullsized":"2023-12-01T01:49:09.605242+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:26.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.615455+0000","last_change":"2023-12-01T01:49:09.615455+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.615455+0000","last_undegraded":"2023-12-01T01:49:09.615455+0000","last_fullsized":"2023-12-01T01:49:09.615455+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:41:28.863921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.613115+0000","last_change":"2023-12-01T01:49:09.613115+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.613115+0000","last_undegraded":"2023-12-01T01:49:09.613115+0000","last_fullsized":"2023-12-01T01:49:09.613115+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:43:54.993285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.610461+0000","last_change":"2023-12-01T01:49:09.610461+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.610461+0000","last_undegraded":"2023-12-01T01:49:09.610461+0000","last_fullsized":"2023-12-01T01:49:09.610461+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:31:01.633346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.610760+0000","last_change":"2023-12-01T01:49:09.610760+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.610760+0000","last_undegraded":"2023-12-01T01:49:09.610760+0000","last_fullsized":"2023-12-01T01:49:09.610760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:27:14.569892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.622199+0000","last_change":"2023-12-01T01:49:09.622199+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.622199+0000","last_undegraded":"2023-12-01T01:49:09.622199+0000","last_fullsized":"2023-12-01T01:49:09.622199+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:57:13.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.621205+0000","last_change":"2023-12-01T01:49:09.621205+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.621205+0000","last_undegraded":"2023-12-01T01:49:09.621205+0000","last_fullsized":"2023-12-01T01:49:09.621205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:59:28.255690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.610016+0000","last_change":"2023-12-01T01:49:09.610016+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.610016+0000","last_undegraded":"2023-12-01T01:49:09.610016+0000","last_fullsized":"2023-12-01T01:49:09.610016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:41:28.863921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.613245+0000","last_change":"2023-12-01T01:49:09.613245+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.613245+0000","last_undegraded":"2023-12-01T01:49:09.613245+0000","last_fullsized":"2023-12-01T01:49:09.613245+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:08:47.587116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.609876+0000","last_change":"2023-12-01T01:49:09.609876+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.609876+0000","last_undegraded":"2023-12-01T01:49:09.609876+0000","last_fullsized":"2023-12-01T01:49:09.609876+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:55.774591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.621311+0000","last_change":"2023-12-01T01:49:09.621311+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.621311+0000","last_undegraded":"2023-12-01T01:49:09.621311+0000","last_fullsized":"2023-12-01T01:49:09.621311+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:42:04.060079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.617136+0000","last_change":"2023-12-01T01:49:09.617136+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.617136+0000","last_undegraded":"2023-12-01T01:49:09.617136+0000","last_fullsized":"2023-12-01T01:49:09.617136+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:43:54.993285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.611451+0000","last_change":"2023-12-01T01:49:09.611451+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.611451+0000","last_undegraded":"2023-12-01T01:49:09.611451+0000","last_fullsized":"2023-12-01T01:49:09.611451+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:18.323184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:49:10.599665+0000","last_change":"2023-12-01T01:49:10.599665+0000","last_active":"2023-12-01T01:49:10.599665+0000","last_peered":"2023-12-01T01:49:10.599665+0000","last_clean":"2023-12-01T01:49:10.599665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:10.599665+0000","last_undegraded":"2023-12-01T01:49:10.599665+0000","last_fullsized":"2023-12-01T01:49:10.599665+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2023-12-01T01:49:09.622373+0000","last_change":"2023-12-01T01:49:09.622373+0000","last_active":"2023-12-01T01:49:09.584005+0000","last_peered":"2023-12-01T01:49:09.584005+0000","last_clean":"2023-12-01T01:49:09.584005+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:49:09.622373+0000","last_undegraded":"2023-12-01T01:49:09.622373+0000","last_fullsized":"2023-12-01T01:49:09.622373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:07:06.126404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"52'139","reported_seq":284,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-01T01:49:07.550022+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:49:07.550022+0000","last_peered":"2023-12-01T01:49:07.550022+0000","last_clean":"2023-12-01T01:49:07.550022+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:49:07.550022+0000","last_undegraded":"2023-12-01T01:49:07.550022+0000","last_fullsized":"2023-12-01T01:49:07.550022+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":724992,"data_stored":335433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0640000000000001,"15min":1.0569999999999999},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.472,"5min":1.627,"15min":1.8089999999999999},"last":0.876},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.0940000000000001,"15min":1.05},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.64000000000000001},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5660000000000001},"last":0.78700000000000003}]},{"osd":1,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.272,"15min":1.2070000000000001},"min":{"1min":0.78300000000000003,"5min":0.60899999999999999,"15min":0.56100000000000005},"max":{"1min":1.796,"5min":1.796,"15min":1.8420000000000001},"last":1.718},{"interface":"front","average":{"1min":1.345,"5min":1.292,"15min":1.262},"min":{"1min":1.0569999999999999,"5min":0.79900000000000004,"15min":0.78100000000000003},"max":{"1min":1.538,"5min":1.696,"15min":2.0870000000000002},"last":1.242}]},{"osd":2,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1319999999999999,"15min":1.1259999999999999},"min":{"1min":0.89600000000000002,"5min":0.76400000000000001,"15min":0.70699999999999996},"max":{"1min":1.5900000000000001,"5min":1.728,"15min":1.728},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.196,"5min":1.226,"15min":1.2350000000000001},"min":{"1min":0.90900000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.849}]},{"osd":3,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.2529999999999999,"15min":1.236},"min":{"1min":1.107,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.712,"5min":1.7250000000000001,"15min":2.0550000000000002},"last":1.21},{"interface":"front","average":{"1min":1.369,"5min":1.3680000000000001,"15min":1.3560000000000001},"min":{"1min":0.97399999999999998,"5min":0.83799999999999997,"15min":0.80000000000000004},"max":{"1min":1.738,"5min":1.9330000000000001,"15min":2.7050000000000001},"last":2.1880000000000002}]},{"osd":4,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2629999999999999,"15min":1.3089999999999999},"min":{"1min":0.83999999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7549999999999999,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.431,"15min":1.4299999999999999},"min":{"1min":1.1970000000000001,"5min":0.82399999999999995,"15min":0.78200000000000003},"max":{"1min":1.883,"5min":2.044,"15min":2.601},"last":2.0939999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.272,"15min":1.302},"min":{"1min":0.83099999999999996,"5min":0.76600000000000001,"15min":0.69099999999999995},"max":{"1min":1.661,"5min":2.0870000000000002,"15min":2.4500000000000002},"last":1.506},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5229999999999999,"15min":1.5349999999999999},"min":{"1min":1.2729999999999999,"5min":0.88100000000000001,"15min":0.80000000000000004},"max":{"1min":1.9179999999999999,"5min":2.1219999999999999,"15min":2.8069999999999999},"last":2.2930000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3959999999999999,"15min":1.4039999999999999},"min":{"1min":1.022,"5min":0.749,"15min":0.749},"max":{"1min":1.716,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.667},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.385,"15min":1.4259999999999999},"min":{"1min":0.91100000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8380000000000001,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.5860000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":712704,"data_stored":329616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.129,"15min":1.1080000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.66900000000000004},"max":{"1min":1.635,"5min":1.635,"15min":1.79},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.149,"15min":1.1279999999999999},"min":{"1min":0.82799999999999996,"5min":0.76600000000000001,"15min":0.61799999999999999},"max":{"1min":1.4810000000000001,"5min":1.6919999999999999,"15min":2.73},"last":1.097}]},{"osd":1,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1659999999999999,"15min":1.145},"min":{"1min":0.81899999999999995,"5min":0.80800000000000005,"15min":0.71199999999999997},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":2.2170000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.262,"5min":1.206,"15min":1.208},"min":{"1min":0.88700000000000001,"5min":0.77300000000000002,"15min":0.71199999999999997},"max":{"1min":1.5920000000000001,"5min":1.651,"15min":2.0920000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.1990000000000001,"15min":1.143},"min":{"1min":0.94599999999999995,"5min":0.83299999999999996,"15min":0.749},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.827},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.238,"15min":1.248},"min":{"1min":0.94599999999999995,"5min":0.90100000000000002,"15min":0.77200000000000002},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.3380000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2749999999999999,"15min":1.2809999999999999},"min":{"1min":1.0029999999999999,"5min":0.93799999999999994,"15min":0.76300000000000001},"max":{"1min":1.6879999999999999,"5min":1.774,"15min":2.1619999999999999},"last":1.958},{"interface":"front","average":{"1min":1.46,"5min":1.4059999999999999,"15min":1.409},"min":{"1min":1.1259999999999999,"5min":0.93899999999999995,"15min":0.81299999999999994},"max":{"1min":1.6970000000000001,"5min":1.758,"15min":2.4020000000000001},"last":1.4239999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.262,"15min":1.294},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.70699999999999996},"max":{"1min":1.748,"5min":1.748,"15min":2.3399999999999999},"last":1.538},{"interface":"front","average":{"1min":1.345,"5min":1.3260000000000001,"15min":1.3700000000000001},"min":{"1min":0.95299999999999996,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":2.6760000000000002},"last":1.627}]},{"osd":5,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.391,"15min":1.4219999999999999},"min":{"1min":1.0980000000000001,"5min":0.98099999999999998,"15min":0.82699999999999996},"max":{"1min":1.931,"5min":1.931,"15min":2.5539999999999998},"last":1.9059999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.468,"15min":1.4790000000000001},"min":{"1min":1.0589999999999999,"5min":0.98899999999999999,"15min":0.748},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.72}]},{"osd":7,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5129999999999999,"15min":1.528},"min":{"1min":1.194,"5min":1.1539999999999999,"15min":0.89500000000000002},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":2.0499999999999998},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.5720000000000001,"15min":1.5860000000000001},"min":{"1min":1.29,"5min":1.1479999999999999,"15min":0.78700000000000003},"max":{"1min":2.367,"5min":2.367,"15min":2.5219999999999998},"last":2.1059999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1306624,"data_stored":919984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1259999999999999,"15min":1.1180000000000001},"min":{"1min":0.92100000000000004,"5min":0.71399999999999997,"15min":0.52500000000000002},"max":{"1min":1.4319999999999999,"5min":2.2010000000000001,"15min":3.3980000000000001},"last":1.381},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.198,"15min":1.1399999999999999},"min":{"1min":0.91600000000000004,"5min":0.78100000000000003,"15min":0.60099999999999998},"max":{"1min":1.6479999999999999,"5min":2.153,"15min":2.153},"last":1.198}]},{"osd":1,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1619999999999999,"15min":1.1120000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.6879999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.24},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.196,"15min":1.147},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.64900000000000002},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.1459999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2450000000000001,"15min":1.21},"min":{"1min":0.93899999999999995,"5min":0.83899999999999997,"15min":0.73299999999999998},"max":{"1min":1.631,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2529999999999999,"15min":1.218},"min":{"1min":0.97699999999999998,"5min":0.90000000000000002,"15min":0.624},"max":{"1min":1.47,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.0620000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":1.03,"5min":0.90400000000000003,"15min":0.76900000000000002},"max":{"1min":1.5649999999999999,"5min":1.6799999999999999,"15min":1.833},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.3069999999999999,"15min":1.2989999999999999},"min":{"1min":1.077,"5min":0.84899999999999998,"15min":0.71699999999999997},"max":{"1min":1.625,"5min":1.8700000000000001,"15min":2.1440000000000001},"last":1.4159999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.228,"15min":1.2490000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.853,"5min":1.853,"15min":2.5129999999999999},"last":1.284},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.387,"15min":1.385},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.7230000000000001,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.3360000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.363,"15min":1.3480000000000001},"min":{"1min":0.93200000000000005,"5min":0.84999999999999998,"15min":0.79500000000000004},"max":{"1min":1.6970000000000001,"5min":1.9790000000000001,"15min":2.2970000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3939999999999999,"15min":1.415},"min":{"1min":0.93500000000000005,"5min":0.90900000000000003,"15min":0.78100000000000003},"max":{"1min":1.9179999999999999,"5min":2.198,"15min":3.23},"last":1.5409999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4970000000000001,"15min":1.518},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.89100000000000001},"max":{"1min":1.8029999999999999,"5min":2.0920000000000001,"15min":3.012},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.522,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.3140000000000001,"5min":0.998,"15min":0.875},"max":{"1min":1.887,"5min":2.1269999999999998,"15min":2.8420000000000001},"last":1.5880000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":712704,"data_stored":329616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1240000000000001,"15min":1.0920000000000001},"min":{"1min":0.81999999999999995,"5min":0.71799999999999997,"15min":0.64800000000000002},"max":{"1min":1.5549999999999999,"5min":1.6619999999999999,"15min":1.7529999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.181,"5min":1.2110000000000001,"15min":1.2010000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.65100000000000002},"max":{"1min":1.5580000000000001,"5min":2.1099999999999999,"15min":3.8069999999999999},"last":1.3740000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.2030000000000001},"min":{"1min":1.022,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6379999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.099,"5min":1.163,"15min":1.1319999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.753},"max":{"1min":1.524,"5min":1.544,"15min":1.637},"last":1.2390000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.228,"15min":1.1899999999999999},"min":{"1min":0.94999999999999996,"5min":0.79900000000000004,"15min":0.77600000000000002},"max":{"1min":1.548,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3049999999999999,"15min":1.2829999999999999},"min":{"1min":0.96399999999999997,"5min":0.93600000000000005,"15min":0.77900000000000003},"max":{"1min":1.5820000000000001,"5min":1.8149999999999999,"15min":2.0049999999999999},"last":1.524}]},{"osd":3,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2569999999999999,"15min":1.248},"min":{"1min":0.99299999999999999,"5min":0.85499999999999998,"15min":0.67900000000000005},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.891},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.331,"15min":1.321},"min":{"1min":0.995,"5min":0.94699999999999995,"15min":0.70399999999999996},"max":{"1min":1.5720000000000001,"5min":1.903,"15min":2.3010000000000002},"last":1.321}]},{"osd":5,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3,"15min":1.3260000000000001},"min":{"1min":0.91300000000000003,"5min":0.81000000000000005,"15min":0.61899999999999999},"max":{"1min":1.6140000000000001,"5min":1.946,"15min":2.1549999999999998},"last":1.071},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.266,"15min":1.3120000000000001},"min":{"1min":0.91400000000000003,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.7310000000000001,"5min":1.756,"15min":2.2029999999999998},"last":1.4510000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.353,"15min":1.3420000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.59099999999999997},"max":{"1min":1.6890000000000001,"5min":1.972,"15min":1.972},"last":1.117},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4710000000000001,"15min":1.466},"min":{"1min":1.097,"5min":0.89300000000000002,"15min":0.63200000000000001},"max":{"1min":1.712,"5min":2.54,"15min":2.54},"last":1.2789999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4570000000000001,"15min":1.484},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":0.60599999999999998},"max":{"1min":1.746,"5min":1.891,"15min":3.2789999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.577,"15min":1.542},"min":{"1min":1.091,"5min":1.091,"15min":0.64500000000000002},"max":{"1min":1.883,"5min":2.157,"15min":2.157},"last":1.4870000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35852,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335988,"statfs":{"total":96636764160,"available":96600051712,"internally_reserved":0,"allocated":724992,"data_stored":335433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.66300000000000003,"5min":0.65900000000000003,"15min":0.307},"max":{"1min":1.1659999999999999,"5min":1.3859999999999999,"15min":1.583},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.036,"5min":1.0329999999999999,"15min":1.018},"min":{"1min":0.72199999999999998,"5min":0.68899999999999995,"15min":0.40999999999999998},"max":{"1min":1.476,"5min":1.4990000000000001,"15min":1.9770000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1259999999999999,"15min":1.0609999999999999},"min":{"1min":0.92200000000000004,"5min":0.80100000000000005,"15min":0.38900000000000001},"max":{"1min":1.3839999999999999,"5min":1.5720000000000001,"15min":2.0779999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.008,"15min":0.95099999999999996},"min":{"1min":0.72699999999999998,"5min":0.65900000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.587,"15min":1.6519999999999999},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.101,"15min":1.0760000000000001},"min":{"1min":0.85399999999999998,"5min":0.70199999999999996,"15min":0.436},"max":{"1min":1.3859999999999999,"5min":1.536,"15min":1.766},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2509999999999999,"15min":1.171},"min":{"1min":1.0249999999999999,"5min":0.97499999999999998,"15min":0.48399999999999999},"max":{"1min":1.5760000000000001,"5min":1.601,"15min":1.661},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2689999999999999,"15min":1.274},"min":{"1min":1.075,"5min":1.012,"15min":0.79600000000000004},"max":{"1min":1.5820000000000001,"5min":1.635,"15min":2.9350000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.282,"5min":1.2689999999999999,"15min":1.256},"min":{"1min":1.044,"5min":0.97199999999999998,"15min":0.73299999999999998},"max":{"1min":1.637,"5min":1.637,"15min":2.2389999999999999},"last":1.0609999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.304,"15min":1.325},"min":{"1min":1.0669999999999999,"5min":1.014,"15min":0.63500000000000001},"max":{"1min":1.5009999999999999,"5min":1.806,"15min":3.476},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.3140000000000001,"15min":1.3640000000000001},"min":{"1min":1.0600000000000001,"5min":0.95499999999999996,"15min":0.64200000000000002},"max":{"1min":1.5920000000000001,"5min":1.9039999999999999,"15min":3.181},"last":0.93700000000000006}]},{"osd":6,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.3979999999999999,"15min":1.3759999999999999},"min":{"1min":1.149,"5min":0.92300000000000004,"15min":0.68300000000000005},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":2.532},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.4419999999999999,"15min":1.4670000000000001},"min":{"1min":1.2,"5min":1.171,"15min":0.872},"max":{"1min":1.7150000000000001,"5min":1.8100000000000001,"15min":3.0249999999999999},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4610000000000001,"15min":1.4430000000000001},"min":{"1min":1.25,"5min":1.1719999999999999,"15min":0.81499999999999995},"max":{"1min":1.825,"5min":1.984,"15min":2.1619999999999999},"last":1.27},{"interface":"front","average":{"1min":1.498,"5min":1.4770000000000001,"15min":1.498},"min":{"1min":1.268,"5min":1.139,"15min":0.83099999999999996},"max":{"1min":1.853,"5min":1.853,"15min":2.4590000000000001},"last":1.254}]}]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":712704,"data_stored":329616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.84599999999999997,"15min":0.81299999999999994},"min":{"1min":0.67100000000000004,"5min":0.59699999999999998,"15min":0.27100000000000002},"max":{"1min":1.107,"5min":1.482,"15min":1.738},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.081,"5min":1.0329999999999999,"15min":0.997},"min":{"1min":0.753,"5min":0.66800000000000004,"15min":0.375},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":0.999}]},{"osd":1,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.99199999999999999,"15min":1.012},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.41899999999999998},"max":{"1min":1.45,"5min":1.5780000000000001,"15min":1.7130000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1240000000000001,"15min":1.0700000000000001},"min":{"1min":0.81799999999999995,"5min":0.78300000000000003,"15min":0.504},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.252}]},{"osd":3,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0640000000000001,"15min":0.996},"min":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.45800000000000002},"max":{"1min":1.625,"5min":1.6479999999999999,"15min":1.883},"last":1.075},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1879999999999999,"15min":1.1619999999999999},"min":{"1min":0.93899999999999995,"5min":0.91900000000000004,"15min":0.39600000000000002},"max":{"1min":1.534,"5min":1.5660000000000001,"15min":2.6080000000000001},"last":1.1859999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.349,"15min":1.329},"min":{"1min":1.103,"5min":1.01,"15min":0.75},"max":{"1min":1.5840000000000001,"5min":1.97,"15min":2.2829999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2909999999999999,"15min":1.2629999999999999},"min":{"1min":1.075,"5min":0.97099999999999997,"15min":0.72299999999999998},"max":{"1min":1.6299999999999999,"5min":1.8080000000000001,"15min":1.964},"last":1.159}]},{"osd":5,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3320000000000001,"15min":1.339},"min":{"1min":1.161,"5min":1.0580000000000001,"15min":0.67600000000000005},"max":{"1min":1.804,"5min":1.804,"15min":2.5779999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.369,"15min":1.403},"min":{"1min":1.169,"5min":0.98699999999999999,"15min":0.69699999999999995},"max":{"1min":1.7390000000000001,"5min":1.75,"15min":3.0099999999999998},"last":1.3959999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.351,"15min":1.3100000000000001},"min":{"1min":1.0189999999999999,"5min":1.018,"15min":0.77500000000000002},"max":{"1min":1.5900000000000001,"5min":1.988,"15min":2.2799999999999998},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":0.77400000000000002},"max":{"1min":1.742,"5min":1.8939999999999999,"15min":2.5089999999999999},"last":1.532}]},{"osd":7,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4550000000000001,"15min":1.4610000000000001},"min":{"1min":1.1539999999999999,"5min":1.093,"15min":0.79400000000000004},"max":{"1min":1.849,"5min":2.214,"15min":2.448},"last":1.9430000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.403,"15min":1.387},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":0.77000000000000002},"max":{"1min":1.776,"5min":2.101,"15min":2.5049999999999999},"last":1.302}]}]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36420,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335420,"statfs":{"total":96636764160,"available":96599470080,"internally_reserved":0,"allocated":1306624,"data_stored":919984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.89400000000000002,"15min":0.89000000000000001},"min":{"1min":0.64800000000000002,"5min":0.60299999999999998,"15min":0.318},"max":{"1min":1.1919999999999999,"5min":1.5049999999999999,"15min":3.056},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0609999999999999,"15min":1.0209999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.32700000000000001},"max":{"1min":1.3640000000000001,"5min":1.6359999999999999,"15min":3.4129999999999998},"last":0.80100000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92300000000000004,"15min":0.92400000000000004},"min":{"1min":0.70899999999999996,"5min":0.66200000000000003,"15min":0.36399999999999999},"max":{"1min":1.276,"5min":1.54,"15min":3.3660000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.097,"5min":1.127,"15min":1.129},"min":{"1min":0.83999999999999997,"5min":0.76500000000000001,"15min":0.39000000000000001},"max":{"1min":1.492,"5min":1.7929999999999999,"15min":3.4980000000000002},"last":1.0820000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2529999999999999,"15min":1.2330000000000001},"min":{"1min":0.92500000000000004,"5min":0.92400000000000004,"15min":0.44},"max":{"1min":1.577,"5min":1.7290000000000001,"15min":3.464},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.173,"5min":1.107,"15min":1.087},"min":{"1min":0.88100000000000001,"5min":0.84399999999999997,"15min":0.39700000000000002},"max":{"1min":1.532,"5min":1.5840000000000001,"15min":3.0960000000000001},"last":1.5609999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2010000000000001,"15min":1.224},"min":{"1min":1.028,"5min":0.93200000000000005,"15min":0.63800000000000001},"max":{"1min":1.526,"5min":1.609,"15min":3.7810000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":1.157,"5min":1.083,"15min":0.76600000000000001},"max":{"1min":1.5509999999999999,"5min":1.863,"15min":4.7270000000000003},"last":1.369}]},{"osd":5,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3200000000000001,"15min":1.3120000000000001},"min":{"1min":0.97399999999999998,"5min":0.95699999999999996,"15min":0.61199999999999999},"max":{"1min":1.647,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3200000000000001,"15min":1.3720000000000001},"min":{"1min":1.107,"5min":1.0229999999999999,"15min":0.69199999999999995},"max":{"1min":1.6319999999999999,"5min":1.825,"15min":3.6669999999999998},"last":1.2410000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3620000000000001,"15min":1.377},"min":{"1min":1.026,"5min":0.97299999999999998,"15min":0.68400000000000005},"max":{"1min":1.7809999999999999,"5min":1.861,"15min":3.3300000000000001},"last":1.712},{"interface":"front","average":{"1min":1.387,"5min":1.3580000000000001,"15min":1.401},"min":{"1min":1.1479999999999999,"5min":1.1020000000000001,"15min":0.752},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.5129999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.472,"15min":1.506},"min":{"1min":1.3200000000000001,"5min":1.151,"15min":0.70599999999999996},"max":{"1min":1.764,"5min":1.9079999999999999,"15min":4.6749999999999998},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.45,"15min":1.4590000000000001},"min":{"1min":1.2330000000000001,"5min":1.0429999999999999,"15min":0.81699999999999995},"max":{"1min":1.992,"5min":1.992,"15min":3.5379999999999998},"last":1.7669999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":712704,"data_stored":329616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.89900000000000002,"15min":0.89100000000000001},"min":{"1min":0.68899999999999995,"5min":0.57199999999999995,"15min":0.29999999999999999},"max":{"1min":1.3640000000000001,"5min":1.4690000000000001,"15min":1.5620000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.441},"max":{"1min":1.2829999999999999,"5min":1.6850000000000001,"15min":1.6910000000000001},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.044,"15min":1.0209999999999999},"min":{"1min":0.81000000000000005,"5min":0.68500000000000005,"15min":0.41999999999999998},"max":{"1min":1.3660000000000001,"5min":1.6459999999999999,"15min":1.653},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.087,"15min":1.099},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.38400000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":2.8159999999999998},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.03,"15min":1.0449999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.45400000000000001},"max":{"1min":1.3859999999999999,"5min":1.7509999999999999,"15min":1.954},"last":0.38700000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.198,"15min":1.202},"min":{"1min":0.94799999999999995,"5min":0.85599999999999998,"15min":0.48999999999999999},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3220000000000001,"15min":1.3140000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.65700000000000003},"max":{"1min":1.637,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2769999999999999,"15min":1.266},"min":{"1min":1.0069999999999999,"5min":0.94899999999999995,"15min":0.76100000000000001},"max":{"1min":1.5489999999999999,"5min":1.708,"15min":2.0680000000000001},"last":1.032}]},{"osd":5,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.379,"15min":1.3779999999999999},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.71899999999999997},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.3400000000000001,"15min":1.3799999999999999},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.585,"5min":2.2759999999999998,"15min":2.7050000000000001},"last":0.86099999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.323,"15min":1.351},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.73099999999999998},"max":{"1min":1.5009999999999999,"5min":1.7989999999999999,"15min":2.157},"last":1.079},{"interface":"front","average":{"1min":1.47,"5min":1.4159999999999999,"15min":1.4450000000000001},"min":{"1min":1.109,"5min":1.0820000000000001,"15min":0.81000000000000005},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4139999999999999,"15min":1.4630000000000001},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.6479999999999999,"5min":2.4279999999999999,"15min":2.8999999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3939999999999999,"15min":1.4359999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.607,"5min":1.849,"15min":2.3039999999999998},"last":0.84299999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:49:15.538 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:49:15.570 INFO:tasks.ceph.mds.c.smithi115.stderr:2023-12-01T01:49:15.569+0000 7f6a65835640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:49:15.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.786+0000 7f4134dab640 1 Processor -- start 2023-12-01T01:49:15.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.787+0000 7f4134dab640 1 -- start start 2023-12-01T01:49:15.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.788+0000 7f4134dab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f413015dfd0 0x7f41300cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.788+0000 7f4134dab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f41300cf9a0 0x7f41300b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.788+0000 7f4134dab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4130160b20 con 0x7f413015dbf0 2023-12-01T01:49:15.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.788+0000 7f4134dab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4130160dc0 con 0x7f41300cf9a0 2023-12-01T01:49:15.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.788+0000 7f4134dab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41301610a0 con 0x7f413015dfd0 2023-12-01T01:49:15.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.788+0000 7f412e59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f413015dbf0 0x7f41300cdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58506/0 (socket says 172.21.15.115:58506) 2023-12-01T01:49:15.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.788+0000 7f412e59c640 1 -- 172.21.15.115:0/3548271938 learned_addr learned my addr 172.21.15.115:0/3548271938 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:15.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.789+0000 7f412ed9d640 1 --2- 172.21.15.115:0/3548271938 >> v2:172.21.15.115:3300/0 conn(0x7f41300cf9a0 0x7f41300b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:15.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.789+0000 7f412d59a640 1 -- 172.21.15.115:0/3548271938 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2489550998 0 0) 0x7f4130160b20 con 0x7f413015dbf0 2023-12-01T01:49:15.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.789+0000 7f412d59a640 1 -- 172.21.15.115:0/3548271938 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f411c003520 con 0x7f413015dbf0 2023-12-01T01:49:15.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.789+0000 7f412dd9b640 1 --2- 172.21.15.115:0/3548271938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f413015dfd0 0x7f41300cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:15.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.789+0000 7f412ed9d640 1 -- 172.21.15.115:0/3548271938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f413015dfd0 msgr2=0x7f41300cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:15.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.789+0000 7f412ed9d640 1 --2- 172.21.15.115:0/3548271938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f413015dfd0 0x7f41300cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.789+0000 7f412ed9d640 1 -- 172.21.15.115:0/3548271938 >> v1:172.21.15.115:6789/0 conn(0x7f413015dbf0 legacy=0x7f41300cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.789+0000 7f412ed9d640 1 -- 172.21.15.115:0/3548271938 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4130161380 con 0x7f41300cf9a0 2023-12-01T01:49:15.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.789+0000 7f412dd9b640 1 --2- 172.21.15.115:0/3548271938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f413015dfd0 0x7f41300cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:15.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.790+0000 7f412ed9d640 1 --2- 172.21.15.115:0/3548271938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 0x7f41300b7ef0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f41280097a0 tx=0x7f412802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcd8526c88ee647 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.790+0000 7f412d59a640 1 -- 172.21.15.115:0/3548271938 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412802d040 con 0x7f41300cf9a0 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.790+0000 7f412d59a640 1 -- 172.21.15.115:0/3548271938 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4128031810 con 0x7f41300cf9a0 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.791+0000 7f412d59a640 1 -- 172.21.15.115:0/3548271938 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128031ba0 con 0x7f41300cf9a0 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.791+0000 7f4134dab640 1 -- 172.21.15.115:0/3548271938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 msgr2=0x7f41300b7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.791+0000 7f4134dab640 1 --2- 172.21.15.115:0/3548271938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 0x7f41300b7ef0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f41280097a0 tx=0x7f412802b8b0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.791+0000 7f4134dab640 1 -- 172.21.15.115:0/3548271938 shutdown_connections 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.791+0000 7f4134dab640 1 --2- 172.21.15.115:0/3548271938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 0x7f41300b7ef0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.791+0000 7f4134dab640 1 --2- 172.21.15.115:0/3548271938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f413015dfd0 0x7f41300cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.791+0000 7f4134dab640 1 -- 172.21.15.115:0/3548271938 >> 172.21.15.115:0/3548271938 conn(0x7f4130065960 msgr2=0x7f4130065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.791+0000 7f4134dab640 1 -- 172.21.15.115:0/3548271938 shutdown_connections 2023-12-01T01:49:15.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.791+0000 7f4134dab640 1 -- 172.21.15.115:0/3548271938 wait complete. 2023-12-01T01:49:15.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.793+0000 7f4134dab640 1 Processor -- start 2023-12-01T01:49:15.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.793+0000 7f4134dab640 1 -- start start 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.794+0000 7f4134dab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 0x7f41300b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.794+0000 7f4134dab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413015dbf0 0x7f413015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.794+0000 7f4134dab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f413015dfd0 0x7f41301f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.794+0000 7f4134dab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413015aa20 con 0x7f41300cf9a0 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.794+0000 7f4134dab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300cbda0 con 0x7f413015dbf0 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.794+0000 7f4134dab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41300cbff0 con 0x7f413015dfd0 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.794+0000 7f412e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 0x7f41300b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.794+0000 7f412e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 0x7f41300b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48590/0 (socket says 172.21.15.115:48590) 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.794+0000 7f412e59c640 1 -- 172.21.15.115:0/3562348336 learned_addr learned my addr 172.21.15.115:0/3562348336 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f412e59c640 1 -- 172.21.15.115:0/3562348336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f413015dfd0 msgr2=0x7f41301f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f412e59c640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f413015dfd0 0x7f41301f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f412e59c640 1 -- 172.21.15.115:0/3562348336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413015dbf0 msgr2=0x7f413015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f412e59c640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413015dbf0 0x7f413015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:15.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f412e59c640 1 -- 172.21.15.115:0/3562348336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128009400 con 0x7f41300cf9a0 2023-12-01T01:49:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f412e59c640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 0x7f41300b0d70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f412000cbc0 tx=0x7f412000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f4126ffd640 1 -- 172.21.15.115:0/3562348336 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000ad40 con 0x7f41300cf9a0 2023-12-01T01:49:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f4126ffd640 1 -- 172.21.15.115:0/3562348336 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120012400 con 0x7f41300cf9a0 2023-12-01T01:49:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f4126ffd640 1 -- 172.21.15.115:0/3562348336 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000aea0 con 0x7f41300cf9a0 2023-12-01T01:49:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f4134dab640 1 -- 172.21.15.115:0/3562348336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41300cc2d0 con 0x7f41300cf9a0 2023-12-01T01:49:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.795+0000 7f4134dab640 1 -- 172.21.15.115:0/3562348336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41300cc890 con 0x7f41300cf9a0 2023-12-01T01:49:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.796+0000 7f4134dab640 1 -- 172.21.15.115:0/3562348336 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41300ccab0 con 0x7f41300cf9a0 2023-12-01T01:49:15.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.797+0000 7f4126ffd640 1 -- 172.21.15.115:0/3562348336 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4120007070 con 0x7f41300cf9a0 2023-12-01T01:49:15.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.798+0000 7f4126ffd640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f411006a580 0x7f411006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:15.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.798+0000 7f4126ffd640 1 -- 172.21.15.115:0/3562348336 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4120088aa0 con 0x7f41300cf9a0 2023-12-01T01:49:15.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.798+0000 7f412dd9b640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f411006a580 0x7f411006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:15.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.799+0000 7f412dd9b640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f411006a580 0x7f411006ca40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4118006fd0 tx=0x7f4118008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:15.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.803+0000 7f4126ffd640 1 -- 172.21.15.115:0/3562348336 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4120053f70 con 0x7f41300cf9a0 2023-12-01T01:49:15.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:15.989+0000 7f4134dab640 1 -- 172.21.15.115:0/3562348336 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41300cfdd0 con 0x7f411006a580 2023-12-01T01:49:16.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.012+0000 7f4126ffd640 1 -- 172.21.15.115:0/3562348336 <== mgr.5706 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286433 (secure 0 0 0) 0x7f411800ae40 con 0x7f411006a580 2023-12-01T01:49:16.016 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:16.019 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:49:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 -- 172.21.15.115:0/3562348336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f411006a580 msgr2=0x7f411006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f411006a580 0x7f411006ca40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4118006fd0 tx=0x7f4118008040 comp rx=0 tx=0).stop 2023-12-01T01:49:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 -- 172.21.15.115:0/3562348336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 msgr2=0x7f41300b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 0x7f41300b0d70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f412000cbc0 tx=0x7f412000a430 comp rx=0 tx=0).stop 2023-12-01T01:49:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 -- 172.21.15.115:0/3562348336 shutdown_connections 2023-12-01T01:49:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f411006a580 0x7f411006ca40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f413015dfd0 0x7f41301f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f413015dbf0 0x7f413015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 --2- 172.21.15.115:0/3562348336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f41300cf9a0 0x7f41300b0d70 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 -- 172.21.15.115:0/3562348336 >> 172.21.15.115:0/3562348336 conn(0x7f4130065960 msgr2=0x7f4130155910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:16.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.019+0000 7f4134dab640 1 -- 172.21.15.115:0/3562348336 shutdown_connections 2023-12-01T01:49:16.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.020+0000 7f4134dab640 1 -- 172.21.15.115:0/3562348336 wait complete. 2023-12-01T01:49:16.041 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T01:49:15.893776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259656,"kb_used_data":7176,"kb_used_omap":0,"kb_used_meta":252416,"kb_avail":754715064,"statfs":{"total":773094113280,"available":772828225536,"internally_reserved":0,"allocated":7348224,"data_stored":4023724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":258473984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001754"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.112811+0000","last_change":"2023-12-01T01:49:11.118775+0000","last_active":"2023-12-01T01:49:12.112811+0000","last_peered":"2023-12-01T01:49:12.112811+0000","last_clean":"2023-12-01T01:49:12.112811+0000","last_became_active":"2023-12-01T01:49:11.118495+0000","last_became_peered":"2023-12-01T01:49:11.118495+0000","last_unstale":"2023-12-01T01:49:12.112811+0000","last_undegraded":"2023-12-01T01:49:12.112811+0000","last_fullsized":"2023-12-01T01:49:12.112811+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:57.736237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.206825+0000","last_change":"2023-12-01T01:49:11.139193+0000","last_active":"2023-12-01T01:49:12.206825+0000","last_peered":"2023-12-01T01:49:12.206825+0000","last_clean":"2023-12-01T01:49:12.206825+0000","last_became_active":"2023-12-01T01:49:11.138977+0000","last_became_peered":"2023-12-01T01:49:11.138977+0000","last_unstale":"2023-12-01T01:49:12.206825+0000","last_undegraded":"2023-12-01T01:49:12.206825+0000","last_fullsized":"2023-12-01T01:49:12.206825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:43:56.008945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853833+0000","last_change":"2023-12-01T01:49:11.138005+0000","last_active":"2023-12-01T01:49:12.853833+0000","last_peered":"2023-12-01T01:49:12.853833+0000","last_clean":"2023-12-01T01:49:12.853833+0000","last_became_active":"2023-12-01T01:49:11.137591+0000","last_became_peered":"2023-12-01T01:49:11.137591+0000","last_unstale":"2023-12-01T01:49:12.853833+0000","last_undegraded":"2023-12-01T01:49:12.853833+0000","last_fullsized":"2023-12-01T01:49:12.853833+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:22:39.625242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854860+0000","last_change":"2023-12-01T01:49:11.134936+0000","last_active":"2023-12-01T01:49:12.854860+0000","last_peered":"2023-12-01T01:49:12.854860+0000","last_clean":"2023-12-01T01:49:12.854860+0000","last_became_active":"2023-12-01T01:49:11.134683+0000","last_became_peered":"2023-12-01T01:49:11.134683+0000","last_unstale":"2023-12-01T01:49:12.854860+0000","last_undegraded":"2023-12-01T01:49:12.854860+0000","last_fullsized":"2023-12-01T01:49:12.854860+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:57.736237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.112574+0000","last_change":"2023-12-01T01:49:11.130877+0000","last_active":"2023-12-01T01:49:12.112574+0000","last_peered":"2023-12-01T01:49:12.112574+0000","last_clean":"2023-12-01T01:49:12.112574+0000","last_became_active":"2023-12-01T01:49:11.130503+0000","last_became_peered":"2023-12-01T01:49:11.130503+0000","last_unstale":"2023-12-01T01:49:12.112574+0000","last_undegraded":"2023-12-01T01:49:12.112574+0000","last_fullsized":"2023-12-01T01:49:12.112574+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:26:02.913146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.852966+0000","last_change":"2023-12-01T01:49:11.142925+0000","last_active":"2023-12-01T01:49:12.852966+0000","last_peered":"2023-12-01T01:49:12.852966+0000","last_clean":"2023-12-01T01:49:12.852966+0000","last_became_active":"2023-12-01T01:49:11.141426+0000","last_became_peered":"2023-12-01T01:49:11.141426+0000","last_unstale":"2023-12-01T01:49:12.852966+0000","last_undegraded":"2023-12-01T01:49:12.852966+0000","last_fullsized":"2023-12-01T01:49:12.852966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:55.876144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853957+0000","last_change":"2023-12-01T01:49:11.133959+0000","last_active":"2023-12-01T01:49:12.853957+0000","last_peered":"2023-12-01T01:49:12.853957+0000","last_clean":"2023-12-01T01:49:12.853957+0000","last_became_active":"2023-12-01T01:49:11.133704+0000","last_became_peered":"2023-12-01T01:49:11.133704+0000","last_unstale":"2023-12-01T01:49:12.853957+0000","last_undegraded":"2023-12-01T01:49:12.853957+0000","last_fullsized":"2023-12-01T01:49:12.853957+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:47:00.893618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854046+0000","last_change":"2023-12-01T01:49:11.133401+0000","last_active":"2023-12-01T01:49:12.854046+0000","last_peered":"2023-12-01T01:49:12.854046+0000","last_clean":"2023-12-01T01:49:12.854046+0000","last_became_active":"2023-12-01T01:49:11.133116+0000","last_became_peered":"2023-12-01T01:49:11.133116+0000","last_unstale":"2023-12-01T01:49:12.854046+0000","last_undegraded":"2023-12-01T01:49:12.854046+0000","last_fullsized":"2023-12-01T01:49:12.854046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:56:23.370342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853906+0000","last_change":"2023-12-01T01:49:11.139637+0000","last_active":"2023-12-01T01:49:12.853906+0000","last_peered":"2023-12-01T01:49:12.853906+0000","last_clean":"2023-12-01T01:49:12.853906+0000","last_became_active":"2023-12-01T01:49:11.139259+0000","last_became_peered":"2023-12-01T01:49:11.139259+0000","last_unstale":"2023-12-01T01:49:12.853906+0000","last_undegraded":"2023-12-01T01:49:12.853906+0000","last_fullsized":"2023-12-01T01:49:12.853906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:26:02.913146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.120106+0000","last_change":"2023-12-01T01:49:11.140652+0000","last_active":"2023-12-01T01:49:12.120106+0000","last_peered":"2023-12-01T01:49:12.120106+0000","last_clean":"2023-12-01T01:49:12.120106+0000","last_became_active":"2023-12-01T01:49:11.138993+0000","last_became_peered":"2023-12-01T01:49:11.138993+0000","last_unstale":"2023-12-01T01:49:12.120106+0000","last_undegraded":"2023-12-01T01:49:12.120106+0000","last_fullsized":"2023-12-01T01:49:12.120106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:06.574323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.855407+0000","last_change":"2023-12-01T01:49:11.137476+0000","last_active":"2023-12-01T01:49:12.855407+0000","last_peered":"2023-12-01T01:49:12.855407+0000","last_clean":"2023-12-01T01:49:12.855407+0000","last_became_active":"2023-12-01T01:49:11.136909+0000","last_became_peered":"2023-12-01T01:49:11.136909+0000","last_unstale":"2023-12-01T01:49:12.855407+0000","last_undegraded":"2023-12-01T01:49:12.855407+0000","last_fullsized":"2023-12-01T01:49:12.855407+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:06.574323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.207107+0000","last_change":"2023-12-01T01:49:11.126372+0000","last_active":"2023-12-01T01:49:12.207107+0000","last_peered":"2023-12-01T01:49:12.207107+0000","last_clean":"2023-12-01T01:49:12.207107+0000","last_became_active":"2023-12-01T01:49:11.126065+0000","last_became_peered":"2023-12-01T01:49:11.126065+0000","last_unstale":"2023-12-01T01:49:12.207107+0000","last_undegraded":"2023-12-01T01:49:12.207107+0000","last_fullsized":"2023-12-01T01:49:12.207107+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:25.159487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853119+0000","last_change":"2023-12-01T01:49:11.142951+0000","last_active":"2023-12-01T01:49:12.853119+0000","last_peered":"2023-12-01T01:49:12.853119+0000","last_clean":"2023-12-01T01:49:12.853119+0000","last_became_active":"2023-12-01T01:49:11.142604+0000","last_became_peered":"2023-12-01T01:49:11.142604+0000","last_unstale":"2023-12-01T01:49:12.853119+0000","last_undegraded":"2023-12-01T01:49:12.853119+0000","last_fullsized":"2023-12-01T01:49:12.853119+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:56:23.370342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854644+0000","last_change":"2023-12-01T01:49:11.134325+0000","last_active":"2023-12-01T01:49:12.854644+0000","last_peered":"2023-12-01T01:49:12.854644+0000","last_clean":"2023-12-01T01:49:12.854644+0000","last_became_active":"2023-12-01T01:49:11.133869+0000","last_became_peered":"2023-12-01T01:49:11.133869+0000","last_unstale":"2023-12-01T01:49:12.854644+0000","last_undegraded":"2023-12-01T01:49:12.854644+0000","last_fullsized":"2023-12-01T01:49:12.854644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:32:53.588769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854049+0000","last_change":"2023-12-01T01:49:11.128972+0000","last_active":"2023-12-01T01:49:12.854049+0000","last_peered":"2023-12-01T01:49:12.854049+0000","last_clean":"2023-12-01T01:49:12.854049+0000","last_became_active":"2023-12-01T01:49:11.128623+0000","last_became_peered":"2023-12-01T01:49:11.128623+0000","last_unstale":"2023-12-01T01:49:12.854049+0000","last_undegraded":"2023-12-01T01:49:12.854049+0000","last_fullsized":"2023-12-01T01:49:12.854049+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:18.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.112752+0000","last_change":"2023-12-01T01:49:11.137806+0000","last_active":"2023-12-01T01:49:12.112752+0000","last_peered":"2023-12-01T01:49:12.112752+0000","last_clean":"2023-12-01T01:49:12.112752+0000","last_became_active":"2023-12-01T01:49:11.137558+0000","last_became_peered":"2023-12-01T01:49:11.137558+0000","last_unstale":"2023-12-01T01:49:12.112752+0000","last_undegraded":"2023-12-01T01:49:12.112752+0000","last_fullsized":"2023-12-01T01:49:12.112752+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:57:16.020220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.206948+0000","last_change":"2023-12-01T01:49:11.122070+0000","last_active":"2023-12-01T01:49:12.206948+0000","last_peered":"2023-12-01T01:49:12.206948+0000","last_clean":"2023-12-01T01:49:12.206948+0000","last_became_active":"2023-12-01T01:49:11.121087+0000","last_became_peered":"2023-12-01T01:49:11.121087+0000","last_unstale":"2023-12-01T01:49:12.206948+0000","last_undegraded":"2023-12-01T01:49:12.206948+0000","last_fullsized":"2023-12-01T01:49:12.206948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:10:58.122697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853433+0000","last_change":"2023-12-01T01:49:11.141391+0000","last_active":"2023-12-01T01:49:12.853433+0000","last_peered":"2023-12-01T01:49:12.853433+0000","last_clean":"2023-12-01T01:49:12.853433+0000","last_became_active":"2023-12-01T01:49:11.140542+0000","last_became_peered":"2023-12-01T01:49:11.140542+0000","last_unstale":"2023-12-01T01:49:12.853433+0000","last_undegraded":"2023-12-01T01:49:12.853433+0000","last_fullsized":"2023-12-01T01:49:12.853433+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:59:29.271350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.112827+0000","last_change":"2023-12-01T01:49:11.130381+0000","last_active":"2023-12-01T01:49:12.112827+0000","last_peered":"2023-12-01T01:49:12.112827+0000","last_clean":"2023-12-01T01:49:12.112827+0000","last_became_active":"2023-12-01T01:49:11.130087+0000","last_became_peered":"2023-12-01T01:49:11.130087+0000","last_unstale":"2023-12-01T01:49:12.112827+0000","last_undegraded":"2023-12-01T01:49:12.112827+0000","last_fullsized":"2023-12-01T01:49:12.112827+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:55.876144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.120060+0000","last_change":"2023-12-01T01:49:11.138187+0000","last_active":"2023-12-01T01:49:12.120060+0000","last_peered":"2023-12-01T01:49:12.120060+0000","last_clean":"2023-12-01T01:49:12.120060+0000","last_became_active":"2023-12-01T01:49:11.137911+0000","last_became_peered":"2023-12-01T01:49:11.137911+0000","last_unstale":"2023-12-01T01:49:12.120060+0000","last_undegraded":"2023-12-01T01:49:12.120060+0000","last_fullsized":"2023-12-01T01:49:12.120060+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:36:42.221953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.855037+0000","last_change":"2023-12-01T01:49:11.132316+0000","last_active":"2023-12-01T01:49:12.855037+0000","last_peered":"2023-12-01T01:49:12.855037+0000","last_clean":"2023-12-01T01:49:12.855037+0000","last_became_active":"2023-12-01T01:49:11.130352+0000","last_became_peered":"2023-12-01T01:49:11.130352+0000","last_unstale":"2023-12-01T01:49:12.855037+0000","last_undegraded":"2023-12-01T01:49:12.855037+0000","last_fullsized":"2023-12-01T01:49:12.855037+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:07:07.142065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.119297+0000","last_change":"2023-12-01T01:49:11.125526+0000","last_active":"2023-12-01T01:49:12.119297+0000","last_peered":"2023-12-01T01:49:12.119297+0000","last_clean":"2023-12-01T01:49:12.119297+0000","last_became_active":"2023-12-01T01:49:11.125252+0000","last_became_peered":"2023-12-01T01:49:11.125252+0000","last_unstale":"2023-12-01T01:49:12.119297+0000","last_undegraded":"2023-12-01T01:49:12.119297+0000","last_fullsized":"2023-12-01T01:49:12.119297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:10.774544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854231+0000","last_change":"2023-12-01T01:49:11.116416+0000","last_active":"2023-12-01T01:49:12.854231+0000","last_peered":"2023-12-01T01:49:12.854231+0000","last_clean":"2023-12-01T01:49:12.854231+0000","last_became_active":"2023-12-01T01:49:11.112915+0000","last_became_peered":"2023-12-01T01:49:11.112915+0000","last_unstale":"2023-12-01T01:49:12.854231+0000","last_undegraded":"2023-12-01T01:49:12.854231+0000","last_fullsized":"2023-12-01T01:49:12.854231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:41.205974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.206457+0000","last_change":"2023-12-01T01:49:11.139055+0000","last_active":"2023-12-01T01:49:12.206457+0000","last_peered":"2023-12-01T01:49:12.206457+0000","last_clean":"2023-12-01T01:49:12.206457+0000","last_became_active":"2023-12-01T01:49:11.138160+0000","last_became_peered":"2023-12-01T01:49:11.138160+0000","last_unstale":"2023-12-01T01:49:12.206457+0000","last_undegraded":"2023-12-01T01:49:12.206457+0000","last_fullsized":"2023-12-01T01:49:12.206457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:29.138954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.112792+0000","last_change":"2023-12-01T01:49:11.118712+0000","last_active":"2023-12-01T01:49:12.112792+0000","last_peered":"2023-12-01T01:49:12.112792+0000","last_clean":"2023-12-01T01:49:12.112792+0000","last_became_active":"2023-12-01T01:49:11.118152+0000","last_became_peered":"2023-12-01T01:49:11.118152+0000","last_unstale":"2023-12-01T01:49:12.112792+0000","last_undegraded":"2023-12-01T01:49:12.112792+0000","last_fullsized":"2023-12-01T01:49:12.112792+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:18.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.855612+0000","last_change":"2023-12-01T01:49:11.135295+0000","last_active":"2023-12-01T01:49:12.855612+0000","last_peered":"2023-12-01T01:49:12.855612+0000","last_clean":"2023-12-01T01:49:12.855612+0000","last_became_active":"2023-12-01T01:49:11.135018+0000","last_became_peered":"2023-12-01T01:49:11.135018+0000","last_unstale":"2023-12-01T01:49:12.855612+0000","last_undegraded":"2023-12-01T01:49:12.855612+0000","last_fullsized":"2023-12-01T01:49:12.855612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:36:42.221953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854881+0000","last_change":"2023-12-01T01:49:11.120597+0000","last_active":"2023-12-01T01:49:12.854881+0000","last_peered":"2023-12-01T01:49:12.854881+0000","last_clean":"2023-12-01T01:49:12.854881+0000","last_became_active":"2023-12-01T01:49:11.119782+0000","last_became_peered":"2023-12-01T01:49:11.119782+0000","last_unstale":"2023-12-01T01:49:12.854881+0000","last_undegraded":"2023-12-01T01:49:12.854881+0000","last_fullsized":"2023-12-01T01:49:12.854881+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:03:04.447914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.855294+0000","last_change":"2023-12-01T01:49:11.122379+0000","last_active":"2023-12-01T01:49:12.855294+0000","last_peered":"2023-12-01T01:49:12.855294+0000","last_clean":"2023-12-01T01:49:12.855294+0000","last_became_active":"2023-12-01T01:49:11.122128+0000","last_became_peered":"2023-12-01T01:49:11.122128+0000","last_unstale":"2023-12-01T01:49:12.855294+0000","last_undegraded":"2023-12-01T01:49:12.855294+0000","last_fullsized":"2023-12-01T01:49:12.855294+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:54.829854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.112666+0000","last_change":"2023-12-01T01:49:11.133244+0000","last_active":"2023-12-01T01:49:12.112666+0000","last_peered":"2023-12-01T01:49:12.112666+0000","last_clean":"2023-12-01T01:49:12.112666+0000","last_became_active":"2023-12-01T01:49:11.132979+0000","last_became_peered":"2023-12-01T01:49:11.132979+0000","last_unstale":"2023-12-01T01:49:12.112666+0000","last_undegraded":"2023-12-01T01:49:12.112666+0000","last_fullsized":"2023-12-01T01:49:12.112666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:56:23.370342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.206457+0000","last_change":"2023-12-01T01:49:11.127766+0000","last_active":"2023-12-01T01:49:12.206457+0000","last_peered":"2023-12-01T01:49:12.206457+0000","last_clean":"2023-12-01T01:49:12.206457+0000","last_became_active":"2023-12-01T01:49:11.127482+0000","last_became_peered":"2023-12-01T01:49:11.127482+0000","last_unstale":"2023-12-01T01:49:12.206457+0000","last_undegraded":"2023-12-01T01:49:12.206457+0000","last_fullsized":"2023-12-01T01:49:12.206457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:43.324119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853059+0000","last_change":"2023-12-01T01:49:11.127232+0000","last_active":"2023-12-01T01:49:12.853059+0000","last_peered":"2023-12-01T01:49:12.853059+0000","last_clean":"2023-12-01T01:49:12.853059+0000","last_became_active":"2023-12-01T01:49:11.126528+0000","last_became_peered":"2023-12-01T01:49:11.126528+0000","last_unstale":"2023-12-01T01:49:12.853059+0000","last_undegraded":"2023-12-01T01:49:12.853059+0000","last_fullsized":"2023-12-01T01:49:12.853059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:57:14.631183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854171+0000","last_change":"2023-12-01T01:49:11.129606+0000","last_active":"2023-12-01T01:49:12.854171+0000","last_peered":"2023-12-01T01:49:12.854171+0000","last_clean":"2023-12-01T01:49:12.854171+0000","last_became_active":"2023-12-01T01:49:11.129307+0000","last_became_peered":"2023-12-01T01:49:11.129307+0000","last_unstale":"2023-12-01T01:49:12.854171+0000","last_undegraded":"2023-12-01T01:49:12.854171+0000","last_fullsized":"2023-12-01T01:49:12.854171+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:45.674602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.119430+0000","last_change":"2023-12-01T01:49:11.138855+0000","last_active":"2023-12-01T01:49:12.119430+0000","last_peered":"2023-12-01T01:49:12.119430+0000","last_clean":"2023-12-01T01:49:12.119430+0000","last_became_active":"2023-12-01T01:49:11.138531+0000","last_became_peered":"2023-12-01T01:49:11.138531+0000","last_unstale":"2023-12-01T01:49:12.119430+0000","last_undegraded":"2023-12-01T01:49:12.119430+0000","last_fullsized":"2023-12-01T01:49:12.119430+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:52:59.232850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"63'4","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.132769+0000","last_change":"2023-12-01T01:49:10.650049+0000","last_active":"2023-12-01T01:49:11.132769+0000","last_peered":"2023-12-01T01:49:11.132769+0000","last_clean":"2023-12-01T01:49:11.132769+0000","last_became_active":"2023-12-01T01:49:10.647446+0000","last_became_peered":"2023-12-01T01:49:10.647446+0000","last_unstale":"2023-12-01T01:49:11.132769+0000","last_undegraded":"2023-12-01T01:49:11.132769+0000","last_fullsized":"2023-12-01T01:49:11.132769+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:42.308459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.112887+0000","last_change":"2023-12-01T01:49:11.128712+0000","last_active":"2023-12-01T01:49:12.112887+0000","last_peered":"2023-12-01T01:49:12.112887+0000","last_clean":"2023-12-01T01:49:12.112887+0000","last_became_active":"2023-12-01T01:49:11.128465+0000","last_became_peered":"2023-12-01T01:49:11.128465+0000","last_unstale":"2023-12-01T01:49:12.112887+0000","last_undegraded":"2023-12-01T01:49:12.112887+0000","last_fullsized":"2023-12-01T01:49:12.112887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:07:07.142065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"63'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.125422+0000","last_change":"2023-12-01T01:49:10.641179+0000","last_active":"2023-12-01T01:49:11.125422+0000","last_peered":"2023-12-01T01:49:11.125422+0000","last_clean":"2023-12-01T01:49:11.125422+0000","last_became_active":"2023-12-01T01:49:10.640899+0000","last_became_peered":"2023-12-01T01:49:10.640899+0000","last_unstale":"2023-12-01T01:49:11.125422+0000","last_undegraded":"2023-12-01T01:49:11.125422+0000","last_fullsized":"2023-12-01T01:49:11.125422+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:28.123293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.207147+0000","last_change":"2023-12-01T01:49:11.139628+0000","last_active":"2023-12-01T01:49:12.207147+0000","last_peered":"2023-12-01T01:49:12.207147+0000","last_clean":"2023-12-01T01:49:12.207147+0000","last_became_active":"2023-12-01T01:49:11.139418+0000","last_became_peered":"2023-12-01T01:49:11.139418+0000","last_unstale":"2023-12-01T01:49:12.207147+0000","last_undegraded":"2023-12-01T01:49:12.207147+0000","last_fullsized":"2023-12-01T01:49:12.207147+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.992494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"63'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.129204+0000","last_change":"2023-12-01T01:49:10.648425+0000","last_active":"2023-12-01T01:49:11.129204+0000","last_peered":"2023-12-01T01:49:11.129204+0000","last_clean":"2023-12-01T01:49:11.129204+0000","last_became_active":"2023-12-01T01:49:10.647786+0000","last_became_peered":"2023-12-01T01:49:10.647786+0000","last_unstale":"2023-12-01T01:49:11.129204+0000","last_undegraded":"2023-12-01T01:49:11.129204+0000","last_fullsized":"2023-12-01T01:49:11.129204+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:27:14.569892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.855337+0000","last_change":"2023-12-01T01:49:11.139914+0000","last_active":"2023-12-01T01:49:12.855337+0000","last_peered":"2023-12-01T01:49:12.855337+0000","last_clean":"2023-12-01T01:49:12.855337+0000","last_became_active":"2023-12-01T01:49:11.138033+0000","last_became_peered":"2023-12-01T01:49:11.138033+0000","last_unstale":"2023-12-01T01:49:12.855337+0000","last_undegraded":"2023-12-01T01:49:12.855337+0000","last_fullsized":"2023-12-01T01:49:12.855337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:26.761794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"63'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.126342+0000","last_change":"2023-12-01T01:49:10.642262+0000","last_active":"2023-12-01T01:49:11.126342+0000","last_peered":"2023-12-01T01:49:11.126342+0000","last_clean":"2023-12-01T01:49:11.126342+0000","last_became_active":"2023-12-01T01:49:10.641959+0000","last_became_peered":"2023-12-01T01:49:10.641959+0000","last_unstale":"2023-12-01T01:49:11.126342+0000","last_undegraded":"2023-12-01T01:49:11.126342+0000","last_fullsized":"2023-12-01T01:49:11.126342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:50:31.691006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854247+0000","last_change":"2023-12-01T01:49:11.120229+0000","last_active":"2023-12-01T01:49:12.854247+0000","last_peered":"2023-12-01T01:49:12.854247+0000","last_clean":"2023-12-01T01:49:12.854247+0000","last_became_active":"2023-12-01T01:49:11.118892+0000","last_became_peered":"2023-12-01T01:49:11.118892+0000","last_unstale":"2023-12-01T01:49:12.854247+0000","last_undegraded":"2023-12-01T01:49:12.854247+0000","last_fullsized":"2023-12-01T01:49:12.854247+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:22:28.356094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"63'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.132697+0000","last_change":"2023-12-01T01:49:10.642248+0000","last_active":"2023-12-01T01:49:11.132697+0000","last_peered":"2023-12-01T01:49:11.132697+0000","last_clean":"2023-12-01T01:49:11.132697+0000","last_became_active":"2023-12-01T01:49:10.641105+0000","last_became_peered":"2023-12-01T01:49:10.641105+0000","last_unstale":"2023-12-01T01:49:11.132697+0000","last_undegraded":"2023-12-01T01:49:11.132697+0000","last_fullsized":"2023-12-01T01:49:11.132697+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:17:03.520261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.120310+0000","last_change":"2023-12-01T01:49:11.141381+0000","last_active":"2023-12-01T01:49:12.120310+0000","last_peered":"2023-12-01T01:49:12.120310+0000","last_clean":"2023-12-01T01:49:12.120310+0000","last_became_active":"2023-12-01T01:49:11.140973+0000","last_became_peered":"2023-12-01T01:49:11.140973+0000","last_unstale":"2023-12-01T01:49:12.120310+0000","last_undegraded":"2023-12-01T01:49:12.120310+0000","last_fullsized":"2023-12-01T01:49:12.120310+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:57.736237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"63'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.126246+0000","last_change":"2023-12-01T01:49:10.634424+0000","last_active":"2023-12-01T01:49:11.126246+0000","last_peered":"2023-12-01T01:49:11.126246+0000","last_clean":"2023-12-01T01:49:11.126246+0000","last_became_active":"2023-12-01T01:49:10.634109+0000","last_became_peered":"2023-12-01T01:49:10.634109+0000","last_unstale":"2023-12-01T01:49:11.126246+0000","last_undegraded":"2023-12-01T01:49:11.126246+0000","last_fullsized":"2023-12-01T01:49:11.126246+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:26:01.897486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.119037+0000","last_change":"2023-12-01T01:49:11.126831+0000","last_active":"2023-12-01T01:49:12.119037+0000","last_peered":"2023-12-01T01:49:12.119037+0000","last_clean":"2023-12-01T01:49:12.119037+0000","last_became_active":"2023-12-01T01:49:11.126377+0000","last_became_peered":"2023-12-01T01:49:11.126377+0000","last_unstale":"2023-12-01T01:49:12.119037+0000","last_undegraded":"2023-12-01T01:49:12.119037+0000","last_fullsized":"2023-12-01T01:49:12.119037+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:57:16.020220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.106819+0000","last_change":"2023-12-01T01:49:10.651514+0000","last_active":"2023-12-01T01:49:11.106819+0000","last_peered":"2023-12-01T01:49:11.106819+0000","last_clean":"2023-12-01T01:49:11.106819+0000","last_became_active":"2023-12-01T01:49:10.651069+0000","last_became_peered":"2023-12-01T01:49:10.651069+0000","last_unstale":"2023-12-01T01:49:11.106819+0000","last_undegraded":"2023-12-01T01:49:11.106819+0000","last_fullsized":"2023-12-01T01:49:11.106819+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:27:14.569892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854311+0000","last_change":"2023-12-01T01:49:11.128777+0000","last_active":"2023-12-01T01:49:12.854311+0000","last_peered":"2023-12-01T01:49:12.854311+0000","last_clean":"2023-12-01T01:49:12.854311+0000","last_became_active":"2023-12-01T01:49:11.128501+0000","last_became_peered":"2023-12-01T01:49:11.128501+0000","last_unstale":"2023-12-01T01:49:12.854311+0000","last_undegraded":"2023-12-01T01:49:12.854311+0000","last_fullsized":"2023-12-01T01:49:12.854311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:39:45.849461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.111306+0000","last_change":"2023-12-01T01:49:10.647810+0000","last_active":"2023-12-01T01:49:11.111306+0000","last_peered":"2023-12-01T01:49:11.111306+0000","last_clean":"2023-12-01T01:49:11.111306+0000","last_became_active":"2023-12-01T01:49:10.647451+0000","last_became_peered":"2023-12-01T01:49:10.647451+0000","last_unstale":"2023-12-01T01:49:11.111306+0000","last_undegraded":"2023-12-01T01:49:11.111306+0000","last_fullsized":"2023-12-01T01:49:11.111306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:24.143826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853368+0000","last_change":"2023-12-01T01:49:11.124542+0000","last_active":"2023-12-01T01:49:12.853368+0000","last_peered":"2023-12-01T01:49:12.853368+0000","last_clean":"2023-12-01T01:49:12.853368+0000","last_became_active":"2023-12-01T01:49:11.124288+0000","last_became_peered":"2023-12-01T01:49:11.124288+0000","last_unstale":"2023-12-01T01:49:12.853368+0000","last_undegraded":"2023-12-01T01:49:12.853368+0000","last_fullsized":"2023-12-01T01:49:12.853368+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:25:01.153748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.109191+0000","last_change":"2023-12-01T01:49:10.646609+0000","last_active":"2023-12-01T01:49:11.109191+0000","last_peered":"2023-12-01T01:49:11.109191+0000","last_clean":"2023-12-01T01:49:11.109191+0000","last_became_active":"2023-12-01T01:49:10.646319+0000","last_became_peered":"2023-12-01T01:49:10.646319+0000","last_unstale":"2023-12-01T01:49:11.109191+0000","last_undegraded":"2023-12-01T01:49:11.109191+0000","last_fullsized":"2023-12-01T01:49:11.109191+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:20:16.537047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853533+0000","last_change":"2023-12-01T01:49:11.114715+0000","last_active":"2023-12-01T01:49:12.853533+0000","last_peered":"2023-12-01T01:49:12.853533+0000","last_clean":"2023-12-01T01:49:12.853533+0000","last_became_active":"2023-12-01T01:49:11.114493+0000","last_became_peered":"2023-12-01T01:49:11.114493+0000","last_unstale":"2023-12-01T01:49:12.853533+0000","last_undegraded":"2023-12-01T01:49:12.853533+0000","last_fullsized":"2023-12-01T01:49:12.853533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:52:59.232850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.125847+0000","last_change":"2023-12-01T01:49:10.644967+0000","last_active":"2023-12-01T01:49:11.125847+0000","last_peered":"2023-12-01T01:49:11.125847+0000","last_clean":"2023-12-01T01:49:11.125847+0000","last_became_active":"2023-12-01T01:49:10.643704+0000","last_became_peered":"2023-12-01T01:49:10.643704+0000","last_unstale":"2023-12-01T01:49:11.125847+0000","last_undegraded":"2023-12-01T01:49:11.125847+0000","last_fullsized":"2023-12-01T01:49:11.125847+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:42.308459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854952+0000","last_change":"2023-12-01T01:49:11.139228+0000","last_active":"2023-12-01T01:49:12.854952+0000","last_peered":"2023-12-01T01:49:12.854952+0000","last_clean":"2023-12-01T01:49:12.854952+0000","last_became_active":"2023-12-01T01:49:11.138947+0000","last_became_peered":"2023-12-01T01:49:11.138947+0000","last_unstale":"2023-12-01T01:49:12.854952+0000","last_undegraded":"2023-12-01T01:49:12.854952+0000","last_fullsized":"2023-12-01T01:49:12.854952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:57:16.020220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"63'3","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.124335+0000","last_change":"2023-12-01T01:49:10.645807+0000","last_active":"2023-12-01T01:49:11.124335+0000","last_peered":"2023-12-01T01:49:11.124335+0000","last_clean":"2023-12-01T01:49:11.124335+0000","last_became_active":"2023-12-01T01:49:10.645359+0000","last_became_peered":"2023-12-01T01:49:10.645359+0000","last_unstale":"2023-12-01T01:49:11.124335+0000","last_undegraded":"2023-12-01T01:49:11.124335+0000","last_fullsized":"2023-12-01T01:49:11.124335+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:25:00.138088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.855068+0000","last_change":"2023-12-01T01:49:11.141158+0000","last_active":"2023-12-01T01:49:12.855068+0000","last_peered":"2023-12-01T01:49:12.855068+0000","last_clean":"2023-12-01T01:49:12.855068+0000","last_became_active":"2023-12-01T01:49:11.140609+0000","last_became_peered":"2023-12-01T01:49:11.140609+0000","last_unstale":"2023-12-01T01:49:12.855068+0000","last_undegraded":"2023-12-01T01:49:12.855068+0000","last_fullsized":"2023-12-01T01:49:12.855068+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:04:37.172347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"63'6","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.147603+0000","last_change":"2023-12-01T01:49:10.646524+0000","last_active":"2023-12-01T01:49:11.147603+0000","last_peered":"2023-12-01T01:49:11.147603+0000","last_clean":"2023-12-01T01:49:11.147603+0000","last_became_active":"2023-12-01T01:49:10.646212+0000","last_became_peered":"2023-12-01T01:49:10.646212+0000","last_unstale":"2023-12-01T01:49:11.147603+0000","last_undegraded":"2023-12-01T01:49:11.147603+0000","last_fullsized":"2023-12-01T01:49:11.147603+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:10:57.107037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853197+0000","last_change":"2023-12-01T01:49:11.134042+0000","last_active":"2023-12-01T01:49:12.853197+0000","last_peered":"2023-12-01T01:49:12.853197+0000","last_clean":"2023-12-01T01:49:12.853197+0000","last_became_active":"2023-12-01T01:49:11.133171+0000","last_became_peered":"2023-12-01T01:49:11.133171+0000","last_unstale":"2023-12-01T01:49:12.853197+0000","last_undegraded":"2023-12-01T01:49:12.853197+0000","last_fullsized":"2023-12-01T01:49:12.853197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:29.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"63'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.133975+0000","last_change":"2023-12-01T01:49:10.649319+0000","last_active":"2023-12-01T01:49:11.133975+0000","last_peered":"2023-12-01T01:49:11.133975+0000","last_clean":"2023-12-01T01:49:11.133975+0000","last_became_active":"2023-12-01T01:49:10.648707+0000","last_became_peered":"2023-12-01T01:49:10.648707+0000","last_unstale":"2023-12-01T01:49:11.133975+0000","last_undegraded":"2023-12-01T01:49:11.133975+0000","last_fullsized":"2023-12-01T01:49:11.133975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:07.421392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.119059+0000","last_change":"2023-12-01T01:49:11.124789+0000","last_active":"2023-12-01T01:49:12.119059+0000","last_peered":"2023-12-01T01:49:12.119059+0000","last_clean":"2023-12-01T01:49:12.119059+0000","last_became_active":"2023-12-01T01:49:11.124407+0000","last_became_peered":"2023-12-01T01:49:11.124407+0000","last_unstale":"2023-12-01T01:49:12.119059+0000","last_undegraded":"2023-12-01T01:49:12.119059+0000","last_fullsized":"2023-12-01T01:49:12.119059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:59:29.271350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.107357+0000","last_change":"2023-12-01T01:49:10.641039+0000","last_active":"2023-12-01T01:49:11.107357+0000","last_peered":"2023-12-01T01:49:11.107357+0000","last_clean":"2023-12-01T01:49:11.107357+0000","last_became_active":"2023-12-01T01:49:10.640771+0000","last_became_peered":"2023-12-01T01:49:10.640771+0000","last_unstale":"2023-12-01T01:49:11.107357+0000","last_undegraded":"2023-12-01T01:49:11.107357+0000","last_fullsized":"2023-12-01T01:49:11.107357+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:42.308459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.118246+0000","last_change":"2023-12-01T01:49:11.145511+0000","last_active":"2023-12-01T01:49:12.118246+0000","last_peered":"2023-12-01T01:49:12.118246+0000","last_clean":"2023-12-01T01:49:12.118246+0000","last_became_active":"2023-12-01T01:49:11.145217+0000","last_became_peered":"2023-12-01T01:49:11.145217+0000","last_unstale":"2023-12-01T01:49:12.118246+0000","last_undegraded":"2023-12-01T01:49:12.118246+0000","last_fullsized":"2023-12-01T01:49:12.118246+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:45.231336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.122788+0000","last_change":"2023-12-01T01:49:10.650118+0000","last_active":"2023-12-01T01:49:11.122788+0000","last_peered":"2023-12-01T01:49:11.122788+0000","last_clean":"2023-12-01T01:49:11.122788+0000","last_became_active":"2023-12-01T01:49:10.649605+0000","last_became_peered":"2023-12-01T01:49:10.649605+0000","last_unstale":"2023-12-01T01:49:11.122788+0000","last_undegraded":"2023-12-01T01:49:11.122788+0000","last_fullsized":"2023-12-01T01:49:11.122788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:55.774591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.206506+0000","last_change":"2023-12-01T01:49:11.121621+0000","last_active":"2023-12-01T01:49:12.206506+0000","last_peered":"2023-12-01T01:49:12.206506+0000","last_clean":"2023-12-01T01:49:12.206506+0000","last_became_active":"2023-12-01T01:49:11.121344+0000","last_became_peered":"2023-12-01T01:49:11.121344+0000","last_unstale":"2023-12-01T01:49:12.206506+0000","last_undegraded":"2023-12-01T01:49:12.206506+0000","last_fullsized":"2023-12-01T01:49:12.206506+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:56.790251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"63'8","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.143961+0000","last_change":"2023-12-01T01:49:10.641266+0000","last_active":"2023-12-01T01:49:11.143961+0000","last_peered":"2023-12-01T01:49:11.143961+0000","last_clean":"2023-12-01T01:49:11.143961+0000","last_became_active":"2023-12-01T01:49:10.640939+0000","last_became_peered":"2023-12-01T01:49:10.640939+0000","last_unstale":"2023-12-01T01:49:11.143961+0000","last_undegraded":"2023-12-01T01:49:11.143961+0000","last_fullsized":"2023-12-01T01:49:11.143961+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:14.976834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.119679+0000","last_change":"2023-12-01T01:49:11.124478+0000","last_active":"2023-12-01T01:49:12.119679+0000","last_peered":"2023-12-01T01:49:12.119679+0000","last_clean":"2023-12-01T01:49:12.119679+0000","last_became_active":"2023-12-01T01:49:11.124122+0000","last_became_peered":"2023-12-01T01:49:11.124122+0000","last_unstale":"2023-12-01T01:49:12.119679+0000","last_undegraded":"2023-12-01T01:49:12.119679+0000","last_fullsized":"2023-12-01T01:49:12.119679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:23:05.700445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"63'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.139542+0000","last_change":"2023-12-01T01:49:10.648282+0000","last_active":"2023-12-01T01:49:11.139542+0000","last_peered":"2023-12-01T01:49:11.139542+0000","last_clean":"2023-12-01T01:49:11.139542+0000","last_became_active":"2023-12-01T01:49:10.648018+0000","last_became_peered":"2023-12-01T01:49:10.648018+0000","last_unstale":"2023-12-01T01:49:11.139542+0000","last_undegraded":"2023-12-01T01:49:11.139542+0000","last_fullsized":"2023-12-01T01:49:11.139542+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:44.215676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.120099+0000","last_change":"2023-12-01T01:49:11.137846+0000","last_active":"2023-12-01T01:49:12.120099+0000","last_peered":"2023-12-01T01:49:12.120099+0000","last_clean":"2023-12-01T01:49:12.120099+0000","last_became_active":"2023-12-01T01:49:11.137518+0000","last_became_peered":"2023-12-01T01:49:11.137518+0000","last_unstale":"2023-12-01T01:49:12.120099+0000","last_undegraded":"2023-12-01T01:49:12.120099+0000","last_fullsized":"2023-12-01T01:49:12.120099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:29.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"63'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.127986+0000","last_change":"2023-12-01T01:49:10.646854+0000","last_active":"2023-12-01T01:49:11.127986+0000","last_peered":"2023-12-01T01:49:11.127986+0000","last_clean":"2023-12-01T01:49:11.127986+0000","last_became_active":"2023-12-01T01:49:10.646259+0000","last_became_peered":"2023-12-01T01:49:10.646259+0000","last_unstale":"2023-12-01T01:49:11.127986+0000","last_undegraded":"2023-12-01T01:49:11.127986+0000","last_fullsized":"2023-12-01T01:49:11.127986+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:18:13.563492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.207351+0000","last_change":"2023-12-01T01:49:11.142276+0000","last_active":"2023-12-01T01:49:12.207351+0000","last_peered":"2023-12-01T01:49:12.207351+0000","last_clean":"2023-12-01T01:49:12.207351+0000","last_became_active":"2023-12-01T01:49:11.141956+0000","last_became_peered":"2023-12-01T01:49:11.141956+0000","last_unstale":"2023-12-01T01:49:12.207351+0000","last_undegraded":"2023-12-01T01:49:12.207351+0000","last_fullsized":"2023-12-01T01:49:12.207351+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:50:32.706667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"63'1","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.137343+0000","last_change":"2023-12-01T01:49:10.645863+0000","last_active":"2023-12-01T01:49:11.137343+0000","last_peered":"2023-12-01T01:49:11.137343+0000","last_clean":"2023-12-01T01:49:11.137343+0000","last_became_active":"2023-12-01T01:49:10.645568+0000","last_became_peered":"2023-12-01T01:49:10.645568+0000","last_unstale":"2023-12-01T01:49:11.137343+0000","last_undegraded":"2023-12-01T01:49:11.137343+0000","last_fullsized":"2023-12-01T01:49:11.137343+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:26.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853679+0000","last_change":"2023-12-01T01:49:11.137670+0000","last_active":"2023-12-01T01:49:12.853679+0000","last_peered":"2023-12-01T01:49:12.853679+0000","last_clean":"2023-12-01T01:49:12.853679+0000","last_became_active":"2023-12-01T01:49:11.137425+0000","last_became_peered":"2023-12-01T01:49:11.137425+0000","last_unstale":"2023-12-01T01:49:12.853679+0000","last_undegraded":"2023-12-01T01:49:12.853679+0000","last_fullsized":"2023-12-01T01:49:12.853679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:27:45.804045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.112895+0000","last_change":"2023-12-01T01:49:10.648757+0000","last_active":"2023-12-01T01:49:11.112895+0000","last_peered":"2023-12-01T01:49:11.112895+0000","last_clean":"2023-12-01T01:49:11.112895+0000","last_became_active":"2023-12-01T01:49:10.648465+0000","last_became_peered":"2023-12-01T01:49:10.648465+0000","last_unstale":"2023-12-01T01:49:11.112895+0000","last_undegraded":"2023-12-01T01:49:11.112895+0000","last_fullsized":"2023-12-01T01:49:11.112895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:41:28.863921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.119470+0000","last_change":"2023-12-01T01:49:11.133509+0000","last_active":"2023-12-01T01:49:12.119470+0000","last_peered":"2023-12-01T01:49:12.119470+0000","last_clean":"2023-12-01T01:49:12.119470+0000","last_became_active":"2023-12-01T01:49:11.133234+0000","last_became_peered":"2023-12-01T01:49:11.133234+0000","last_unstale":"2023-12-01T01:49:12.119470+0000","last_undegraded":"2023-12-01T01:49:12.119470+0000","last_fullsized":"2023-12-01T01:49:12.119470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:26.761794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"63'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.141661+0000","last_change":"2023-12-01T01:49:10.643086+0000","last_active":"2023-12-01T01:49:11.141661+0000","last_peered":"2023-12-01T01:49:11.141661+0000","last_clean":"2023-12-01T01:49:11.141661+0000","last_became_active":"2023-12-01T01:49:10.642811+0000","last_became_peered":"2023-12-01T01:49:10.642811+0000","last_unstale":"2023-12-01T01:49:11.141661+0000","last_undegraded":"2023-12-01T01:49:11.141661+0000","last_fullsized":"2023-12-01T01:49:11.141661+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:43:54.993285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.853259+0000","last_change":"2023-12-01T01:49:11.130256+0000","last_active":"2023-12-01T01:49:12.853259+0000","last_peered":"2023-12-01T01:49:12.853259+0000","last_clean":"2023-12-01T01:49:12.853259+0000","last_became_active":"2023-12-01T01:49:11.129936+0000","last_became_peered":"2023-12-01T01:49:11.129936+0000","last_unstale":"2023-12-01T01:49:12.853259+0000","last_undegraded":"2023-12-01T01:49:12.853259+0000","last_fullsized":"2023-12-01T01:49:12.853259+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:07:07.142065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.107278+0000","last_change":"2023-12-01T01:49:10.647932+0000","last_active":"2023-12-01T01:49:11.107278+0000","last_peered":"2023-12-01T01:49:11.107278+0000","last_clean":"2023-12-01T01:49:11.107278+0000","last_became_active":"2023-12-01T01:49:10.647616+0000","last_became_peered":"2023-12-01T01:49:10.647616+0000","last_unstale":"2023-12-01T01:49:11.107278+0000","last_undegraded":"2023-12-01T01:49:11.107278+0000","last_fullsized":"2023-12-01T01:49:11.107278+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:31:01.633346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854327+0000","last_change":"2023-12-01T01:49:11.128956+0000","last_active":"2023-12-01T01:49:12.854327+0000","last_peered":"2023-12-01T01:49:12.854327+0000","last_clean":"2023-12-01T01:49:12.854327+0000","last_became_active":"2023-12-01T01:49:11.128522+0000","last_became_peered":"2023-12-01T01:49:11.128522+0000","last_unstale":"2023-12-01T01:49:12.854327+0000","last_undegraded":"2023-12-01T01:49:12.854327+0000","last_fullsized":"2023-12-01T01:49:12.854327+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:41.205974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.118136+0000","last_change":"2023-12-01T01:49:10.643794+0000","last_active":"2023-12-01T01:49:11.118136+0000","last_peered":"2023-12-01T01:49:11.118136+0000","last_clean":"2023-12-01T01:49:11.118136+0000","last_became_active":"2023-12-01T01:49:10.643506+0000","last_became_peered":"2023-12-01T01:49:10.643506+0000","last_unstale":"2023-12-01T01:49:11.118136+0000","last_undegraded":"2023-12-01T01:49:11.118136+0000","last_fullsized":"2023-12-01T01:49:11.118136+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:27:14.569892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854237+0000","last_change":"2023-12-01T01:49:11.136049+0000","last_active":"2023-12-01T01:49:12.854237+0000","last_peered":"2023-12-01T01:49:12.854237+0000","last_clean":"2023-12-01T01:49:12.854237+0000","last_became_active":"2023-12-01T01:49:11.135295+0000","last_became_peered":"2023-12-01T01:49:11.135295+0000","last_unstale":"2023-12-01T01:49:12.854237+0000","last_undegraded":"2023-12-01T01:49:12.854237+0000","last_fullsized":"2023-12-01T01:49:12.854237+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:25.159487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.108402+0000","last_change":"2023-12-01T01:49:10.641681+0000","last_active":"2023-12-01T01:49:11.108402+0000","last_peered":"2023-12-01T01:49:11.108402+0000","last_clean":"2023-12-01T01:49:11.108402+0000","last_became_active":"2023-12-01T01:49:10.641409+0000","last_became_peered":"2023-12-01T01:49:10.641409+0000","last_unstale":"2023-12-01T01:49:11.108402+0000","last_undegraded":"2023-12-01T01:49:11.108402+0000","last_fullsized":"2023-12-01T01:49:11.108402+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:57:13.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.855134+0000","last_change":"2023-12-01T01:49:11.129137+0000","last_active":"2023-12-01T01:49:12.855134+0000","last_peered":"2023-12-01T01:49:12.855134+0000","last_clean":"2023-12-01T01:49:12.855134+0000","last_became_active":"2023-12-01T01:49:11.128855+0000","last_became_peered":"2023-12-01T01:49:11.128855+0000","last_unstale":"2023-12-01T01:49:12.855134+0000","last_undegraded":"2023-12-01T01:49:12.855134+0000","last_fullsized":"2023-12-01T01:49:12.855134+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:36.306753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.120892+0000","last_change":"2023-12-01T01:49:10.652821+0000","last_active":"2023-12-01T01:49:11.120892+0000","last_peered":"2023-12-01T01:49:11.120892+0000","last_clean":"2023-12-01T01:49:11.120892+0000","last_became_active":"2023-12-01T01:49:10.652558+0000","last_became_peered":"2023-12-01T01:49:10.652558+0000","last_unstale":"2023-12-01T01:49:11.120892+0000","last_undegraded":"2023-12-01T01:49:11.120892+0000","last_fullsized":"2023-12-01T01:49:11.120892+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:59:28.255690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.120191+0000","last_change":"2023-12-01T01:49:11.134232+0000","last_active":"2023-12-01T01:49:12.120191+0000","last_peered":"2023-12-01T01:49:12.120191+0000","last_clean":"2023-12-01T01:49:12.120191+0000","last_became_active":"2023-12-01T01:49:11.133177+0000","last_became_peered":"2023-12-01T01:49:11.133177+0000","last_unstale":"2023-12-01T01:49:12.120191+0000","last_undegraded":"2023-12-01T01:49:12.120191+0000","last_fullsized":"2023-12-01T01:49:12.120191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:18.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.108454+0000","last_change":"2023-12-01T01:49:10.649212+0000","last_active":"2023-12-01T01:49:11.108454+0000","last_peered":"2023-12-01T01:49:11.108454+0000","last_clean":"2023-12-01T01:49:11.108454+0000","last_became_active":"2023-12-01T01:49:10.648384+0000","last_became_peered":"2023-12-01T01:49:10.648384+0000","last_unstale":"2023-12-01T01:49:11.108454+0000","last_undegraded":"2023-12-01T01:49:11.108454+0000","last_fullsized":"2023-12-01T01:49:11.108454+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:41:28.863921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.855145+0000","last_change":"2023-12-01T01:49:11.141256+0000","last_active":"2023-12-01T01:49:12.855145+0000","last_peered":"2023-12-01T01:49:12.855145+0000","last_clean":"2023-12-01T01:49:12.855145+0000","last_became_active":"2023-12-01T01:49:11.140993+0000","last_became_peered":"2023-12-01T01:49:11.140993+0000","last_unstale":"2023-12-01T01:49:12.855145+0000","last_undegraded":"2023-12-01T01:49:12.855145+0000","last_fullsized":"2023-12-01T01:49:12.855145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:07:14.109770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.116619+0000","last_change":"2023-12-01T01:49:10.647075+0000","last_active":"2023-12-01T01:49:11.116619+0000","last_peered":"2023-12-01T01:49:11.116619+0000","last_clean":"2023-12-01T01:49:11.116619+0000","last_became_active":"2023-12-01T01:49:10.646773+0000","last_became_peered":"2023-12-01T01:49:10.646773+0000","last_unstale":"2023-12-01T01:49:11.116619+0000","last_undegraded":"2023-12-01T01:49:11.116619+0000","last_fullsized":"2023-12-01T01:49:11.116619+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:08:47.587116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.120288+0000","last_change":"2023-12-01T01:49:11.131904+0000","last_active":"2023-12-01T01:49:12.120288+0000","last_peered":"2023-12-01T01:49:12.120288+0000","last_clean":"2023-12-01T01:49:12.120288+0000","last_became_active":"2023-12-01T01:49:11.131636+0000","last_became_peered":"2023-12-01T01:49:11.131636+0000","last_unstale":"2023-12-01T01:49:12.120288+0000","last_undegraded":"2023-12-01T01:49:12.120288+0000","last_fullsized":"2023-12-01T01:49:12.120288+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:32:53.588769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.107094+0000","last_change":"2023-12-01T01:49:10.650224+0000","last_active":"2023-12-01T01:49:11.107094+0000","last_peered":"2023-12-01T01:49:11.107094+0000","last_clean":"2023-12-01T01:49:11.107094+0000","last_became_active":"2023-12-01T01:49:10.649919+0000","last_became_peered":"2023-12-01T01:49:10.649919+0000","last_unstale":"2023-12-01T01:49:11.107094+0000","last_undegraded":"2023-12-01T01:49:11.107094+0000","last_fullsized":"2023-12-01T01:49:11.107094+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:55.774591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.854794+0000","last_change":"2023-12-01T01:49:11.132525+0000","last_active":"2023-12-01T01:49:12.854794+0000","last_peered":"2023-12-01T01:49:12.854794+0000","last_clean":"2023-12-01T01:49:12.854794+0000","last_became_active":"2023-12-01T01:49:11.130705+0000","last_became_peered":"2023-12-01T01:49:11.130705+0000","last_unstale":"2023-12-01T01:49:12.854794+0000","last_undegraded":"2023-12-01T01:49:12.854794+0000","last_fullsized":"2023-12-01T01:49:12.854794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:26:02.913146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.111307+0000","last_change":"2023-12-01T01:49:10.636652+0000","last_active":"2023-12-01T01:49:11.111307+0000","last_peered":"2023-12-01T01:49:11.111307+0000","last_clean":"2023-12-01T01:49:11.111307+0000","last_became_active":"2023-12-01T01:49:10.636384+0000","last_became_peered":"2023-12-01T01:49:10.636384+0000","last_unstale":"2023-12-01T01:49:11.111307+0000","last_undegraded":"2023-12-01T01:49:11.111307+0000","last_fullsized":"2023-12-01T01:49:11.111307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:42:04.060079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.852638+0000","last_change":"2023-12-01T01:49:11.138441+0000","last_active":"2023-12-01T01:49:12.852638+0000","last_peered":"2023-12-01T01:49:12.852638+0000","last_clean":"2023-12-01T01:49:12.852638+0000","last_became_active":"2023-12-01T01:49:11.137879+0000","last_became_peered":"2023-12-01T01:49:11.137879+0000","last_unstale":"2023-12-01T01:49:12.852638+0000","last_undegraded":"2023-12-01T01:49:12.852638+0000","last_fullsized":"2023-12-01T01:49:12.852638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:45.231336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.122906+0000","last_change":"2023-12-01T01:49:10.640604+0000","last_active":"2023-12-01T01:49:11.122906+0000","last_peered":"2023-12-01T01:49:11.122906+0000","last_clean":"2023-12-01T01:49:11.122906+0000","last_became_active":"2023-12-01T01:49:10.639327+0000","last_became_peered":"2023-12-01T01:49:10.639327+0000","last_unstale":"2023-12-01T01:49:11.122906+0000","last_undegraded":"2023-12-01T01:49:11.122906+0000","last_fullsized":"2023-12-01T01:49:11.122906+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:43:54.993285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.206621+0000","last_change":"2023-12-01T01:49:11.118310+0000","last_active":"2023-12-01T01:49:12.206621+0000","last_peered":"2023-12-01T01:49:12.206621+0000","last_clean":"2023-12-01T01:49:12.206621+0000","last_became_active":"2023-12-01T01:49:11.118033+0000","last_became_peered":"2023-12-01T01:49:11.118033+0000","last_unstale":"2023-12-01T01:49:12.206621+0000","last_undegraded":"2023-12-01T01:49:12.206621+0000","last_fullsized":"2023-12-01T01:49:12.206621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:27:15.585553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.118146+0000","last_change":"2023-12-01T01:49:10.651697+0000","last_active":"2023-12-01T01:49:11.118146+0000","last_peered":"2023-12-01T01:49:11.118146+0000","last_clean":"2023-12-01T01:49:11.118146+0000","last_became_active":"2023-12-01T01:49:10.651195+0000","last_became_peered":"2023-12-01T01:49:10.651195+0000","last_unstale":"2023-12-01T01:49:11.118146+0000","last_undegraded":"2023-12-01T01:49:11.118146+0000","last_fullsized":"2023-12-01T01:49:11.118146+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:18.323184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-12-01T01:49:12.855673+0000","last_change":"2023-12-01T01:49:11.140244+0000","last_active":"2023-12-01T01:49:12.855673+0000","last_peered":"2023-12-01T01:49:12.855673+0000","last_clean":"2023-12-01T01:49:12.855673+0000","last_became_active":"2023-12-01T01:49:11.139893+0000","last_became_peered":"2023-12-01T01:49:11.139893+0000","last_unstale":"2023-12-01T01:49:12.855673+0000","last_undegraded":"2023-12-01T01:49:12.855673+0000","last_fullsized":"2023-12-01T01:49:12.855673+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:10.599665+0000","last_clean_scrub_stamp":"2023-12-01T01:49:10.599665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:55.876144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"63'2","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.141495+0000","last_change":"2023-12-01T01:49:10.649845+0000","last_active":"2023-12-01T01:49:11.141495+0000","last_peered":"2023-12-01T01:49:11.141495+0000","last_clean":"2023-12-01T01:49:11.141495+0000","last_became_active":"2023-12-01T01:49:10.649564+0000","last_became_peered":"2023-12-01T01:49:10.649564+0000","last_unstale":"2023-12-01T01:49:11.141495+0000","last_undegraded":"2023-12-01T01:49:11.141495+0000","last_fullsized":"2023-12-01T01:49:11.141495+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:49:09.584005+0000","last_clean_scrub_stamp":"2023-12-01T01:49:09.584005+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:07:06.126404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"52'139","reported_seq":287,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-01T01:49:11.107591+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:49:11.107591+0000","last_peered":"2023-12-01T01:49:11.107591+0000","last_clean":"2023-12-01T01:49:11.107591+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:49:11.107591+0000","last_undegraded":"2023-12-01T01:49:11.107591+0000","last_fullsized":"2023-12-01T01:49:11.107591+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":765952,"data_stored":354817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0640000000000001,"15min":1.0569999999999999},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":1.472,"5min":1.627,"15min":1.8089999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.0940000000000001,"15min":1.05},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.64000000000000001},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5660000000000001},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.272,"15min":1.2070000000000001},"min":{"1min":0.78300000000000003,"5min":0.60899999999999999,"15min":0.56100000000000005},"max":{"1min":1.796,"5min":1.796,"15min":1.8420000000000001},"last":1.175},{"interface":"front","average":{"1min":1.345,"5min":1.292,"15min":1.262},"min":{"1min":1.0569999999999999,"5min":0.79900000000000004,"15min":0.78100000000000003},"max":{"1min":1.538,"5min":1.696,"15min":2.0870000000000002},"last":1.3700000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1319999999999999,"15min":1.1259999999999999},"min":{"1min":0.89600000000000002,"5min":0.76400000000000001,"15min":0.70699999999999996},"max":{"1min":1.5900000000000001,"5min":1.728,"15min":1.728},"last":1.054},{"interface":"front","average":{"1min":1.196,"5min":1.226,"15min":1.2350000000000001},"min":{"1min":0.90900000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":0.999}]},{"osd":3,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.2529999999999999,"15min":1.236},"min":{"1min":1.107,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.712,"5min":1.7250000000000001,"15min":2.0550000000000002},"last":1.476},{"interface":"front","average":{"1min":1.369,"5min":1.3680000000000001,"15min":1.3560000000000001},"min":{"1min":0.97399999999999998,"5min":0.83799999999999997,"15min":0.80000000000000004},"max":{"1min":1.738,"5min":1.9330000000000001,"15min":2.7050000000000001},"last":1.335}]},{"osd":4,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2629999999999999,"15min":1.3089999999999999},"min":{"1min":0.83999999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7549999999999999,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.623},{"interface":"front","average":{"1min":1.462,"5min":1.431,"15min":1.4299999999999999},"min":{"1min":1.1970000000000001,"5min":0.82399999999999995,"15min":0.78200000000000003},"max":{"1min":1.883,"5min":2.044,"15min":2.601},"last":1.4359999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.272,"15min":1.302},"min":{"1min":0.83099999999999996,"5min":0.76600000000000001,"15min":0.69099999999999995},"max":{"1min":1.661,"5min":2.0870000000000002,"15min":2.4500000000000002},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5229999999999999,"15min":1.5349999999999999},"min":{"1min":1.2729999999999999,"5min":0.88100000000000001,"15min":0.80000000000000004},"max":{"1min":1.9179999999999999,"5min":2.1219999999999999,"15min":2.8069999999999999},"last":1.577}]},{"osd":6,"last update":"Fri Dec 1 01:48:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3959999999999999,"15min":1.4039999999999999},"min":{"1min":1.022,"5min":0.749,"15min":0.749},"max":{"1min":1.716,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.605},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.385,"15min":1.4259999999999999},"min":{"1min":0.91100000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8380000000000001,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.6830000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575011,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":770048,"data_stored":355399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.129,"15min":1.1080000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.66900000000000004},"max":{"1min":1.635,"5min":1.635,"15min":1.79},"last":1.115},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.149,"15min":1.1279999999999999},"min":{"1min":0.82799999999999996,"5min":0.76600000000000001,"15min":0.61799999999999999},"max":{"1min":1.4810000000000001,"5min":1.6919999999999999,"15min":2.73},"last":0.82999999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1659999999999999,"15min":1.145},"min":{"1min":0.81899999999999995,"5min":0.80800000000000005,"15min":0.71199999999999997},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":2.2170000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.206,"15min":1.208},"min":{"1min":0.88700000000000001,"5min":0.77300000000000002,"15min":0.71199999999999997},"max":{"1min":1.5920000000000001,"5min":1.651,"15min":2.0920000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.1990000000000001,"15min":1.143},"min":{"1min":0.94599999999999995,"5min":0.83299999999999996,"15min":0.749},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.238,"15min":1.248},"min":{"1min":0.94599999999999995,"5min":0.90100000000000002,"15min":0.77200000000000002},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.1679999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2749999999999999,"15min":1.2809999999999999},"min":{"1min":1.0029999999999999,"5min":0.93799999999999994,"15min":0.76300000000000001},"max":{"1min":1.6879999999999999,"5min":1.774,"15min":2.1619999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.46,"5min":1.4059999999999999,"15min":1.409},"min":{"1min":1.1259999999999999,"5min":0.93899999999999995,"15min":0.81299999999999994},"max":{"1min":1.6970000000000001,"5min":1.758,"15min":2.4020000000000001},"last":1.2929999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.262,"15min":1.294},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.70699999999999996},"max":{"1min":1.748,"5min":1.748,"15min":2.3399999999999999},"last":1.256},{"interface":"front","average":{"1min":1.345,"5min":1.3260000000000001,"15min":1.3700000000000001},"min":{"1min":0.95299999999999996,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":2.6760000000000002},"last":1.212}]},{"osd":5,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.391,"15min":1.4219999999999999},"min":{"1min":1.0980000000000001,"5min":0.98099999999999998,"15min":0.82699999999999996},"max":{"1min":1.931,"5min":1.931,"15min":2.5539999999999998},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.468,"15min":1.4790000000000001},"min":{"1min":1.0589999999999999,"5min":0.98899999999999999,"15min":0.748},"max":{"1min":2.8580000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.345}]},{"osd":7,"last update":"Fri Dec 1 01:48:27 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5129999999999999,"15min":1.528},"min":{"1min":1.194,"5min":1.1539999999999999,"15min":0.89500000000000002},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.5720000000000001,"15min":1.5860000000000001},"min":{"1min":1.29,"5min":1.1479999999999999,"15min":0.78700000000000003},"max":{"1min":2.367,"5min":2.367,"15min":2.5219999999999998},"last":1.3779999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":1359872,"data_stored":945173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1259999999999999,"15min":1.1180000000000001},"min":{"1min":0.92100000000000004,"5min":0.71399999999999997,"15min":0.52500000000000002},"max":{"1min":1.4319999999999999,"5min":2.2010000000000001,"15min":3.3980000000000001},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.198,"15min":1.1399999999999999},"min":{"1min":0.91600000000000004,"5min":0.78100000000000003,"15min":0.60099999999999998},"max":{"1min":1.6479999999999999,"5min":2.153,"15min":2.153},"last":1.3169999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1619999999999999,"15min":1.1120000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.6879999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.196,"15min":1.147},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.64900000000000002},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.2370000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2450000000000001,"15min":1.21},"min":{"1min":0.93899999999999995,"5min":0.83899999999999997,"15min":0.73299999999999998},"max":{"1min":1.631,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2529999999999999,"15min":1.218},"min":{"1min":0.97699999999999998,"5min":0.90000000000000002,"15min":0.624},"max":{"1min":1.47,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.153}]},{"osd":3,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":1.03,"5min":0.90400000000000003,"15min":0.76900000000000002},"max":{"1min":1.5649999999999999,"5min":1.6799999999999999,"15min":1.833},"last":1.024},{"interface":"front","average":{"1min":1.347,"5min":1.3069999999999999,"15min":1.2989999999999999},"min":{"1min":1.077,"5min":0.84899999999999998,"15min":0.71699999999999997},"max":{"1min":1.625,"5min":1.8700000000000001,"15min":2.1440000000000001},"last":1.5680000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.228,"15min":1.2490000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.853,"5min":1.853,"15min":2.5129999999999999},"last":1.377},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.387,"15min":1.385},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.7230000000000001,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.609}]},{"osd":6,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.363,"15min":1.3480000000000001},"min":{"1min":0.93200000000000005,"5min":0.84999999999999998,"15min":0.79500000000000004},"max":{"1min":1.6970000000000001,"5min":1.9790000000000001,"15min":2.2970000000000002},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3939999999999999,"15min":1.415},"min":{"1min":0.93500000000000005,"5min":0.90900000000000003,"15min":0.78100000000000003},"max":{"1min":1.9179999999999999,"5min":2.198,"15min":3.23},"last":1.6519999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4970000000000001,"15min":1.518},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.89100000000000001},"max":{"1min":1.8029999999999999,"5min":2.0920000000000001,"15min":3.012},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.5529999999999999,"15min":1.5529999999999999},"min":{"1min":1.3140000000000001,"5min":0.998,"15min":0.875},"max":{"1min":1.887,"5min":2.1269999999999998,"15min":2.8420000000000001},"last":1.6970000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575010,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35908,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335932,"statfs":{"total":96636764160,"available":96599994368,"internally_reserved":0,"allocated":782336,"data_stored":356439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1240000000000001,"15min":1.0920000000000001},"min":{"1min":0.81999999999999995,"5min":0.71799999999999997,"15min":0.64800000000000002},"max":{"1min":1.5549999999999999,"5min":1.6619999999999999,"15min":1.7529999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.181,"5min":1.2110000000000001,"15min":1.2010000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.65100000000000002},"max":{"1min":1.5580000000000001,"5min":2.1099999999999999,"15min":3.8069999999999999},"last":1.0800000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.2030000000000001},"min":{"1min":1.022,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6379999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.163,"15min":1.1319999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.753},"max":{"1min":1.524,"5min":1.544,"15min":1.637},"last":1.111}]},{"osd":2,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.228,"15min":1.1899999999999999},"min":{"1min":0.94999999999999996,"5min":0.79900000000000004,"15min":0.77600000000000002},"max":{"1min":1.548,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.996},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3049999999999999,"15min":1.2829999999999999},"min":{"1min":0.96399999999999997,"5min":0.93600000000000005,"15min":0.77900000000000003},"max":{"1min":1.5820000000000001,"5min":1.8149999999999999,"15min":2.0049999999999999},"last":0.96199999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2569999999999999,"15min":1.248},"min":{"1min":0.99299999999999999,"5min":0.85499999999999998,"15min":0.67900000000000005},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.891},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.331,"15min":1.321},"min":{"1min":0.995,"5min":0.94699999999999995,"15min":0.70399999999999996},"max":{"1min":1.5720000000000001,"5min":1.903,"15min":2.3010000000000002},"last":1.266}]},{"osd":5,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3,"15min":1.3260000000000001},"min":{"1min":0.91300000000000003,"5min":0.81000000000000005,"15min":0.61899999999999999},"max":{"1min":1.6140000000000001,"5min":1.946,"15min":2.1549999999999998},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.266,"15min":1.3120000000000001},"min":{"1min":0.91400000000000003,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.7310000000000001,"5min":1.756,"15min":2.2029999999999998},"last":1.1659999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.353,"15min":1.3420000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.59099999999999997},"max":{"1min":1.6890000000000001,"5min":1.972,"15min":1.972},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4710000000000001,"15min":1.466},"min":{"1min":1.097,"5min":0.89300000000000002,"15min":0.63200000000000001},"max":{"1min":1.712,"5min":2.54,"15min":2.54},"last":1.3169999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4570000000000001,"15min":1.484},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":0.60599999999999998},"max":{"1min":1.746,"5min":1.891,"15min":3.2789999999999999},"last":1.21},{"interface":"front","average":{"1min":1.552,"5min":1.577,"15min":1.542},"min":{"1min":1.091,"5min":1.091,"15min":0.64500000000000002},"max":{"1min":1.883,"5min":2.157,"15min":2.157},"last":1.4399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":774144,"data_stored":356347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.66300000000000003,"5min":0.65900000000000003,"15min":0.307},"max":{"1min":1.1659999999999999,"5min":1.3859999999999999,"15min":1.583},"last":0.46100000000000002},{"interface":"front","average":{"1min":1.036,"5min":1.0329999999999999,"15min":1.018},"min":{"1min":0.72199999999999998,"5min":0.68899999999999995,"15min":0.40999999999999998},"max":{"1min":1.476,"5min":1.4990000000000001,"15min":1.9770000000000001},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1259999999999999,"15min":1.0609999999999999},"min":{"1min":0.92200000000000004,"5min":0.80100000000000005,"15min":0.38900000000000001},"max":{"1min":1.3839999999999999,"5min":1.5720000000000001,"15min":2.0779999999999998},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.01,"5min":1.008,"15min":0.95099999999999996},"min":{"1min":0.72699999999999998,"5min":0.65900000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.587,"15min":1.6519999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.101,"15min":1.0760000000000001},"min":{"1min":0.85399999999999998,"5min":0.70199999999999996,"15min":0.436},"max":{"1min":1.3859999999999999,"5min":1.536,"15min":1.766},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2509999999999999,"15min":1.171},"min":{"1min":1.0249999999999999,"5min":0.97499999999999998,"15min":0.48399999999999999},"max":{"1min":1.5760000000000001,"5min":1.601,"15min":1.661},"last":0.85199999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2689999999999999,"15min":1.274},"min":{"1min":1.075,"5min":1.012,"15min":0.79600000000000004},"max":{"1min":1.5820000000000001,"5min":1.635,"15min":2.9350000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.2689999999999999,"15min":1.256},"min":{"1min":1.044,"5min":0.97199999999999998,"15min":0.73299999999999998},"max":{"1min":1.637,"5min":1.637,"15min":2.2389999999999999},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.304,"15min":1.325},"min":{"1min":1.0669999999999999,"5min":1.014,"15min":0.63500000000000001},"max":{"1min":1.5009999999999999,"5min":1.806,"15min":3.476},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.329,"5min":1.3140000000000001,"15min":1.3640000000000001},"min":{"1min":1.0600000000000001,"5min":0.95499999999999996,"15min":0.64200000000000002},"max":{"1min":1.5920000000000001,"5min":1.9039999999999999,"15min":3.181},"last":1.044}]},{"osd":6,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.3979999999999999,"15min":1.3759999999999999},"min":{"1min":1.149,"5min":0.92300000000000004,"15min":0.68300000000000005},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":2.532},"last":1.141},{"interface":"front","average":{"1min":1.462,"5min":1.4419999999999999,"15min":1.4670000000000001},"min":{"1min":1.2,"5min":1.171,"15min":0.872},"max":{"1min":1.7150000000000001,"5min":1.8100000000000001,"15min":3.0249999999999999},"last":1.1819999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:48:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4610000000000001,"15min":1.4430000000000001},"min":{"1min":1.25,"5min":1.1719999999999999,"15min":0.81499999999999995},"max":{"1min":1.825,"5min":1.984,"15min":2.1619999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.498,"5min":1.4770000000000001,"15min":1.498},"min":{"1min":1.268,"5min":1.139,"15min":0.83099999999999996},"max":{"1min":1.853,"5min":1.853,"15min":2.4590000000000001},"last":1.0720000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":765952,"data_stored":354873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.84599999999999997,"15min":0.81299999999999994},"min":{"1min":0.67100000000000004,"5min":0.59699999999999998,"15min":0.27100000000000002},"max":{"1min":1.107,"5min":1.482,"15min":1.738},"last":0.441},{"interface":"front","average":{"1min":1.081,"5min":1.0329999999999999,"15min":0.997},"min":{"1min":0.753,"5min":0.66800000000000004,"15min":0.375},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":1.0629999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.99199999999999999,"15min":1.012},"min":{"1min":0.83999999999999997,"5min":0.68899999999999995,"15min":0.41899999999999998},"max":{"1min":1.45,"5min":1.5780000000000001,"15min":1.7130000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1240000000000001,"15min":1.0700000000000001},"min":{"1min":0.81799999999999995,"5min":0.78300000000000003,"15min":0.504},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.86799999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0640000000000001,"15min":0.996},"min":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.45800000000000002},"max":{"1min":1.625,"5min":1.6479999999999999,"15min":1.883},"last":0.50700000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1879999999999999,"15min":1.1619999999999999},"min":{"1min":0.93899999999999995,"5min":0.91900000000000004,"15min":0.39600000000000002},"max":{"1min":1.534,"5min":1.5660000000000001,"15min":2.6080000000000001},"last":0.60399999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.349,"15min":1.329},"min":{"1min":1.103,"5min":1.01,"15min":0.75},"max":{"1min":1.5840000000000001,"5min":1.97,"15min":2.2829999999999999},"last":1.409},{"interface":"front","average":{"1min":1.242,"5min":1.2909999999999999,"15min":1.2629999999999999},"min":{"1min":1.075,"5min":0.97099999999999997,"15min":0.72299999999999998},"max":{"1min":1.6299999999999999,"5min":1.8080000000000001,"15min":1.964},"last":1.27}]},{"osd":5,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3320000000000001,"15min":1.339},"min":{"1min":1.161,"5min":1.0580000000000001,"15min":0.67600000000000005},"max":{"1min":1.804,"5min":1.804,"15min":2.5779999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.369,"15min":1.403},"min":{"1min":1.169,"5min":0.98699999999999999,"15min":0.69699999999999995},"max":{"1min":1.7390000000000001,"5min":1.75,"15min":3.0099999999999998},"last":1.452}]},{"osd":6,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.351,"15min":1.3100000000000001},"min":{"1min":1.0189999999999999,"5min":1.018,"15min":0.77500000000000002},"max":{"1min":1.5900000000000001,"5min":1.988,"15min":2.2799999999999998},"last":1.052},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":0.77400000000000002},"max":{"1min":1.742,"5min":1.8939999999999999,"15min":2.5089999999999999},"last":1.2110000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:48:16 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4550000000000001,"15min":1.4610000000000001},"min":{"1min":1.1539999999999999,"5min":1.093,"15min":0.79400000000000004},"max":{"1min":1.849,"5min":2.214,"15min":2.448},"last":1.353},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.403,"15min":1.387},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":0.77000000000000002},"max":{"1min":1.776,"5min":2.101,"15min":2.5049999999999999},"last":1.107}]}]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36472,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335368,"statfs":{"total":96636764160,"available":96599416832,"internally_reserved":0,"allocated":1359872,"data_stored":945221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.89400000000000002,"15min":0.89000000000000001},"min":{"1min":0.64800000000000002,"5min":0.60299999999999998,"15min":0.318},"max":{"1min":1.1919999999999999,"5min":1.5049999999999999,"15min":3.056},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0609999999999999,"15min":1.0209999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.32700000000000001},"max":{"1min":1.3640000000000001,"5min":1.6359999999999999,"15min":3.4129999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92300000000000004,"15min":0.92400000000000004},"min":{"1min":0.70899999999999996,"5min":0.66200000000000003,"15min":0.36399999999999999},"max":{"1min":1.276,"5min":1.54,"15min":3.3660000000000001},"last":0.745},{"interface":"front","average":{"1min":1.097,"5min":1.127,"15min":1.129},"min":{"1min":0.83999999999999997,"5min":0.76500000000000001,"15min":0.39000000000000001},"max":{"1min":1.492,"5min":1.7929999999999999,"15min":3.4980000000000002},"last":0.95899999999999996}]},{"osd":3,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2529999999999999,"15min":1.2330000000000001},"min":{"1min":0.92500000000000004,"5min":0.92400000000000004,"15min":0.44},"max":{"1min":1.577,"5min":1.7290000000000001,"15min":3.464},"last":1.111},{"interface":"front","average":{"1min":1.173,"5min":1.107,"15min":1.087},"min":{"1min":0.88100000000000001,"5min":0.84399999999999997,"15min":0.39700000000000002},"max":{"1min":1.532,"5min":1.5840000000000001,"15min":3.0960000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2010000000000001,"15min":1.224},"min":{"1min":1.028,"5min":0.93200000000000005,"15min":0.63800000000000001},"max":{"1min":1.526,"5min":1.609,"15min":3.7810000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":1.157,"5min":1.083,"15min":0.76600000000000001},"max":{"1min":1.5509999999999999,"5min":1.863,"15min":4.7270000000000003},"last":1.1950000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3200000000000001,"15min":1.3120000000000001},"min":{"1min":0.97399999999999998,"5min":0.95699999999999996,"15min":0.61199999999999999},"max":{"1min":1.647,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3200000000000001,"15min":1.3720000000000001},"min":{"1min":1.107,"5min":1.0229999999999999,"15min":0.69199999999999995},"max":{"1min":1.6319999999999999,"5min":1.825,"15min":3.6669999999999998},"last":1.0509999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3620000000000001,"15min":1.377},"min":{"1min":1.026,"5min":0.97299999999999998,"15min":0.68400000000000005},"max":{"1min":1.7809999999999999,"5min":1.861,"15min":3.3300000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.3580000000000001,"15min":1.401},"min":{"1min":1.1479999999999999,"5min":1.1020000000000001,"15min":0.752},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.0109999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:48:18 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.472,"15min":1.506},"min":{"1min":1.3200000000000001,"5min":1.151,"15min":0.70599999999999996},"max":{"1min":1.764,"5min":1.9079999999999999,"15min":4.6749999999999998},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.514,"5min":1.45,"15min":1.4590000000000001},"min":{"1min":1.2330000000000001,"5min":1.0429999999999999,"15min":0.81699999999999995},"max":{"1min":1.992,"5min":1.992,"15min":3.5379999999999998},"last":0.91700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":770048,"data_stored":355455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.89900000000000002,"15min":0.89100000000000001},"min":{"1min":0.68899999999999995,"5min":0.57199999999999995,"15min":0.29999999999999999},"max":{"1min":1.3640000000000001,"5min":1.4690000000000001,"15min":1.5620000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.441},"max":{"1min":1.2829999999999999,"5min":1.6850000000000001,"15min":1.6910000000000001},"last":0.91200000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.044,"15min":1.0209999999999999},"min":{"1min":0.81000000000000005,"5min":0.68500000000000005,"15min":0.41999999999999998},"max":{"1min":1.3660000000000001,"5min":1.6459999999999999,"15min":1.653},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.089,"5min":1.087,"15min":1.099},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.38400000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":2.8159999999999998},"last":0.83699999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.03,"15min":1.0449999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.45400000000000001},"max":{"1min":1.3859999999999999,"5min":1.7509999999999999,"15min":1.954},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.198,"15min":1.202},"min":{"1min":0.94799999999999995,"5min":0.85599999999999998,"15min":0.48999999999999999},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3220000000000001,"15min":1.3140000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.65700000000000003},"max":{"1min":1.637,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2769999999999999,"15min":1.266},"min":{"1min":1.0069999999999999,"5min":0.94899999999999995,"15min":0.76100000000000001},"max":{"1min":1.5489999999999999,"5min":1.708,"15min":2.0680000000000001},"last":1.3009999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.379,"15min":1.3779999999999999},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.71899999999999997},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.3400000000000001,"15min":1.3799999999999999},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.585,"5min":2.2759999999999998,"15min":2.7050000000000001},"last":1.1479999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.323,"15min":1.351},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.73099999999999998},"max":{"1min":1.5009999999999999,"5min":1.7989999999999999,"15min":2.157},"last":1.179},{"interface":"front","average":{"1min":1.47,"5min":1.4159999999999999,"15min":1.4450000000000001},"min":{"1min":1.109,"5min":1.0820000000000001,"15min":0.81000000000000005},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.3300000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4139999999999999,"15min":1.4630000000000001},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.6479999999999999,"5min":2.4279999999999999,"15min":2.8999999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3939999999999999,"15min":1.4359999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.607,"5min":1.849,"15min":2.3039999999999998},"last":1.401}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:49:16.045 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T01:49:16.045 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T01:49:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.310+0000 7f8723fff640 1 Processor -- start 2023-12-01T01:49:16.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.310+0000 7f8723fff640 1 -- start start 2023-12-01T01:49:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.312+0000 7f8723fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87240cdec0 0x7f87240b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.312+0000 7f8723fff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240ca080 con 0x7f87240cdae0 2023-12-01T01:49:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.312+0000 7f8723fff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240ca360 con 0x7f87240b0620 2023-12-01T01:49:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.312+0000 7f8723fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240b0ab0 con 0x7f87240cdec0 2023-12-01T01:49:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f8722ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f87240cdae0 0x7f872415dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58516/0 (socket says 172.21.15.115:58516) 2023-12-01T01:49:16.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f8722ffd640 1 -- 172.21.15.115:0/627530655 learned_addr learned my addr 172.21.15.115:0/627530655 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656451848 0 0) 0x7f87240ca080 con 0x7f87240cdae0 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8710003520 con 0x7f87240cdae0 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f871bfff640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87240cdec0 0x7f87240b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2566644366 0 0) 0x7f8710003520 con 0x7f87240cdae0 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87240cdec0 msgr2=0x7f87240b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87240cdec0 0x7f87240b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 >> v1:172.21.15.142:6789/0 conn(0x7f87240b0620 legacy=0x7f87240acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240ca640 con 0x7f87240cdae0 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f8714001670 con 0x7f87240cdae0 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 >> v1:172.21.15.115:6789/0 conn(0x7f87240cdae0 legacy=0x7f872415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87100039a0 0x7f8710003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.313+0000 7f87227fc640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87100047c0 0x7f8710004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f87227fc640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 0x7f871000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240b0ab0 con 0x7f8710005140 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871000bec0 con 0x7f87100039a0 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871000c050 con 0x7f87100047c0 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f87237fe640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 0x7f871000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f8722ffd640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87100039a0 0x7f8710003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f871bfff640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87100047c0 0x7f8710004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f87237fe640 1 -- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87100047c0 msgr2=0x7f8710004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f87237fe640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87100047c0 0x7f8710004c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f87237fe640 1 -- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87100039a0 msgr2=0x7f8710003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f87237fe640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87100039a0 0x7f8710003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.314+0000 7f87237fe640 1 -- 172.21.15.115:0/627530655 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8710003520 con 0x7f8710005140 2023-12-01T01:49:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.315+0000 7f87237fe640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 0x7f871000b980 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f871c001e50 tx=0x7f87100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480bdb77e4bd2930 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.316+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c003b70 con 0x7f8710005140 2023-12-01T01:49:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.316+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c003cd0 con 0x7f8710005140 2023-12-01T01:49:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.316+0000 7f87227fc640 1 -- 172.21.15.115:0/627530655 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c005310 con 0x7f8710005140 2023-12-01T01:49:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.316+0000 7f8723fff640 1 -- 172.21.15.115:0/627530655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 msgr2=0x7f871000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.316+0000 7f8723fff640 1 --2- 172.21.15.115:0/627530655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 0x7f871000b980 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f871c001e50 tx=0x7f87100057f0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.316+0000 7f87237fe640 1 -- 172.21.15.115:0/627530655 reap_dead start 2023-12-01T01:49:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.316+0000 7f8723fff640 1 -- 172.21.15.115:0/627530655 shutdown_connections 2023-12-01T01:49:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.316+0000 7f8723fff640 1 -- 172.21.15.115:0/627530655 >> 172.21.15.115:0/627530655 conn(0x7f8724066400 msgr2=0x7f87240cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.317+0000 7f8723fff640 1 -- 172.21.15.115:0/627530655 shutdown_connections 2023-12-01T01:49:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.317+0000 7f8723fff640 1 -- 172.21.15.115:0/627530655 wait complete. 2023-12-01T01:49:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.318+0000 7f8723fff640 1 Processor -- start 2023-12-01T01:49:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.318+0000 7f8723fff640 1 -- start start 2023-12-01T01:49:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.319+0000 7f8723fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 0x7f87240acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.319+0000 7f8723fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87240b1410 0x7f8724150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.319+0000 7f8723fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f872406fc90 0x7f8724070110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.319+0000 7f8723fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8724151000 con 0x7f8710005140 2023-12-01T01:49:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.319+0000 7f8723fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8724070650 con 0x7f872406fc90 2023-12-01T01:49:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.319+0000 7f8723fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240ad1e0 con 0x7f87240b1410 2023-12-01T01:49:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.320+0000 7f8722ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 0x7f87240acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.320+0000 7f8722ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 0x7f87240acc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48612/0 (socket says 172.21.15.115:48612) 2023-12-01T01:49:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.320+0000 7f8722ffd640 1 -- 172.21.15.115:0/2985570467 learned_addr learned my addr 172.21.15.115:0/2985570467 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.320+0000 7f87237fe640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f872406fc90 0x7f8724070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.320+0000 7f8722ffd640 1 -- 172.21.15.115:0/2985570467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87240b1410 msgr2=0x7f8724150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.320+0000 7f871bfff640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87240b1410 0x7f8724150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.320+0000 7f8722ffd640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87240b1410 0x7f8724150460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.320+0000 7f8722ffd640 1 -- 172.21.15.115:0/2985570467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f872406fc90 msgr2=0x7f8724070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.320+0000 7f8722ffd640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f872406fc90 0x7f8724070110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.320+0000 7f8722ffd640 1 -- 172.21.15.115:0/2985570467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240ad4c0 con 0x7f8710005140 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.321+0000 7f8722ffd640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 0x7f87240acc00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f871400e800 tx=0x7f871400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.321+0000 7f8720ff9640 1 -- 172.21.15.115:0/2985570467 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871400c8d0 con 0x7f8710005140 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.321+0000 7f8720ff9640 1 -- 172.21.15.115:0/2985570467 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871400ca30 con 0x7f8710005140 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.321+0000 7f8720ff9640 1 -- 172.21.15.115:0/2985570467 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871400cde0 con 0x7f8710005140 2023-12-01T01:49:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.321+0000 7f8723fff640 1 -- 172.21.15.115:0/2985570467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87240ad7a0 con 0x7f8710005140 2023-12-01T01:49:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.321+0000 7f8723fff640 1 -- 172.21.15.115:0/2985570467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87240ae1e0 con 0x7f8710005140 2023-12-01T01:49:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.322+0000 7f8723fff640 1 -- 172.21.15.115:0/2985570467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87240cabe0 con 0x7f8710005140 2023-12-01T01:49:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.323+0000 7f8720ff9640 1 -- 172.21.15.115:0/2985570467 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f871401d460 con 0x7f8710005140 2023-12-01T01:49:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.324+0000 7f8720ff9640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f870406a550 0x7f870406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.324+0000 7f8720ff9640 1 -- 172.21.15.115:0/2985570467 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8714088b60 con 0x7f8710005140 2023-12-01T01:49:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.324+0000 7f871bfff640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f870406a550 0x7f870406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.325+0000 7f871bfff640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f870406a550 0x7f870406ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87241512a0 tx=0x7f870c008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:16.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.328+0000 7f8720ff9640 1 -- 172.21.15.115:0/2985570467 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f87240cabe0 con 0x7f8710005140 2023-12-01T01:49:16.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.589+0000 7f8723fff640 1 -- 172.21.15.115:0/2985570467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f87240cabe0 con 0x7f8710005140 2023-12-01T01:49:16.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.598+0000 7f8720ff9640 1 -- 172.21.15.115:0/2985570467 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f87240cabe0 con 0x7f8710005140 2023-12-01T01:49:16.599 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T01:49:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 -- 172.21.15.115:0/2985570467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f870406a550 msgr2=0x7f870406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f870406a550 0x7f870406ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87241512a0 tx=0x7f870c008040 comp rx=0 tx=0).stop 2023-12-01T01:49:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 -- 172.21.15.115:0/2985570467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 msgr2=0x7f87240acc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 0x7f87240acc00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f871400e800 tx=0x7f871400ecc0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 -- 172.21.15.115:0/2985570467 shutdown_connections 2023-12-01T01:49:16.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f870406a550 0x7f870406ca10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87240b1410 0x7f8724150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f872406fc90 0x7f8724070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 --2- 172.21.15.115:0/2985570467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005140 0x7f87240acc00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 -- 172.21.15.115:0/2985570467 >> 172.21.15.115:0/2985570467 conn(0x7f8724066400 msgr2=0x7f872406f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:16.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 -- 172.21.15.115:0/2985570467 shutdown_connections 2023-12-01T01:49:16.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.601+0000 7f8723fff640 1 -- 172.21.15.115:0/2985570467 wait complete. 2023-12-01T01:49:16.619 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:49:16.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.830+0000 7f7d447e3640 1 Processor -- start 2023-12-01T01:49:16.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.831+0000 7f7d447e3640 1 -- start start 2023-12-01T01:49:16.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.832+0000 7f7d447e3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7d3c0cad70 0x7f7d3c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.832+0000 7f7d447e3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7d3c06f2f0 0x7f7d3c06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.832+0000 7f7d447e3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c0add10 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.832+0000 7f7d447e3640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c0c5760 con 0x7f7d3c0cad70 2023-12-01T01:49:16.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.832+0000 7f7d447e3640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c15e6d0 con 0x7f7d3c06f2f0 2023-12-01T01:49:16.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.832+0000 7f7d41d7e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7d3c0cad70 0x7f7d3c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.833+0000 7f7d41d7e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7d3c0cad70 0x7f7d3c06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48626/0 (socket says 172.21.15.115:48626) 2023-12-01T01:49:16.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.833+0000 7f7d41d7e640 1 -- 172.21.15.115:0/4207256647 learned_addr learned my addr 172.21.15.115:0/4207256647 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:16.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.833+0000 7f7d42d80640 1 --2- 172.21.15.115:0/4207256647 >> v2:172.21.15.142:3300/0 conn(0x7f7d3c06f2f0 0x7f7d3c06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.833+0000 7f7d41d7e640 1 -- 172.21.15.115:0/4207256647 >> v2:172.21.15.142:3300/0 conn(0x7f7d3c06f2f0 msgr2=0x7f7d3c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:16.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.833+0000 7f7d41d7e640 1 --2- 172.21.15.115:0/4207256647 >> v2:172.21.15.142:3300/0 conn(0x7f7d3c06f2f0 0x7f7d3c06bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.833+0000 7f7d41d7e640 1 -- 172.21.15.115:0/4207256647 >> v1:172.21.15.142:6789/0 conn(0x7f7d3c15e2f0 legacy=0x7f7d3c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:16.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.833+0000 7f7d41d7e640 1 -- 172.21.15.115:0/4207256647 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c0cf7b0 con 0x7f7d3c0cad70 2023-12-01T01:49:16.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.834+0000 7f7d41d7e640 1 --2- 172.21.15.115:0/4207256647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c0cad70 0x7f7d3c06b5c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c06de80 tx=0x7f7d3002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3392552aca15a16 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:16.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.834+0000 7f7d4157d640 1 -- 172.21.15.115:0/4207256647 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30031740 con 0x7f7d3c0cad70 2023-12-01T01:49:16.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.834+0000 7f7d4157d640 1 -- 172.21.15.115:0/4207256647 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d300318a0 con 0x7f7d3c0cad70 2023-12-01T01:49:16.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.834+0000 7f7d4157d640 1 -- 172.21.15.115:0/4207256647 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30031c30 con 0x7f7d3c0cad70 2023-12-01T01:49:16.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.835+0000 7f7d447e3640 1 -- 172.21.15.115:0/4207256647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c0cad70 msgr2=0x7f7d3c06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.835+0000 7f7d447e3640 1 --2- 172.21.15.115:0/4207256647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c0cad70 0x7f7d3c06b5c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c06de80 tx=0x7f7d3002b7e0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.835+0000 7f7d447e3640 1 -- 172.21.15.115:0/4207256647 shutdown_connections 2023-12-01T01:49:16.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.835+0000 7f7d447e3640 1 --2- 172.21.15.115:0/4207256647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c0cad70 0x7f7d3c06b5c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.835+0000 7f7d447e3640 1 --2- 172.21.15.115:0/4207256647 >> v2:172.21.15.142:3300/0 conn(0x7f7d3c06f2f0 0x7f7d3c06bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.835+0000 7f7d447e3640 1 -- 172.21.15.115:0/4207256647 >> 172.21.15.115:0/4207256647 conn(0x7f7d3c0ce880 msgr2=0x7f7d3c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:16.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.835+0000 7f7d447e3640 1 -- 172.21.15.115:0/4207256647 shutdown_connections 2023-12-01T01:49:16.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.835+0000 7f7d447e3640 1 -- 172.21.15.115:0/4207256647 wait complete. 2023-12-01T01:49:16.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.837+0000 7f7d447e3640 1 Processor -- start 2023-12-01T01:49:16.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.837+0000 7f7d447e3640 1 -- start start 2023-12-01T01:49:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.838+0000 7f7d447e3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d3c06f2f0 0x7f7d3c1579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.838+0000 7f7d447e3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d3c0cad70 0x7f7d3c1f0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.838+0000 7f7d447e3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c15e2f0 0x7f7d3c0c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.838+0000 7f7d447e3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0c5270 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.838+0000 7f7d447e3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0c23b0 con 0x7f7d3c06f2f0 2023-12-01T01:49:16.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.838+0000 7f7d447e3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0c2650 con 0x7f7d3c0cad70 2023-12-01T01:49:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.838+0000 7f7d42d80640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c15e2f0 0x7f7d3c0c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.838+0000 7f7d42d80640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c15e2f0 0x7f7d3c0c4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48634/0 (socket says 172.21.15.115:48634) 2023-12-01T01:49:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.838+0000 7f7d42d80640 1 -- 172.21.15.115:0/2248016409 learned_addr learned my addr 172.21.15.115:0/2248016409 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d4257f640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d3c06f2f0 0x7f7d3c1579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d42d80640 1 -- 172.21.15.115:0/2248016409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d3c0cad70 msgr2=0x7f7d3c1f0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d42d80640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d3c0cad70 0x7f7d3c1f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d42d80640 1 -- 172.21.15.115:0/2248016409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d3c06f2f0 msgr2=0x7f7d3c1579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d42d80640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d3c06f2f0 0x7f7d3c1579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d42d80640 1 -- 172.21.15.115:0/2248016409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d300094d0 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d42d80640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c15e2f0 0x7f7d3c0c4d30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400c4c0 tx=0x7f7d3400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d3affd640 1 -- 172.21.15.115:0/2248016409 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34004220 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d3affd640 1 -- 172.21.15.115:0/2248016409 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d340043a0 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d3affd640 1 -- 172.21.15.115:0/2248016409 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34003850 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d447e3640 1 -- 172.21.15.115:0/2248016409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c0c2930 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.839+0000 7f7d447e3640 1 -- 172.21.15.115:0/2248016409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c0c2ef0 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.840+0000 7f7d447e3640 1 -- 172.21.15.115:0/2248016409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3c06fda0 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.841+0000 7f7d3affd640 1 -- 172.21.15.115:0/2248016409 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f7d340039b0 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.842+0000 7f7d3affd640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d2806a550 0x7f7d2806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:16.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.842+0000 7f7d3affd640 1 -- 172.21.15.115:0/2248016409 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d34088820 con 0x7f7d3c15e2f0 2023-12-01T01:49:16.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.842+0000 7f7d4257f640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d2806a550 0x7f7d2806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:16.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.843+0000 7f7d4257f640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d2806a550 0x7f7d2806ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c007dc0 tx=0x7f7d2c0072f0 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:16.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:16.846+0000 7f7d3affd640 1 -- 172.21.15.115:0/2248016409 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7d34053be0 con 0x7f7d3c15e2f0 2023-12-01T01:49:17.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.071+0000 7f7d447e3640 1 -- 172.21.15.115:0/2248016409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d3c04e890 con 0x7f7d3c15e2f0 2023-12-01T01:49:17.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.072+0000 7f7d3affd640 1 -- 172.21.15.115:0/2248016409 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f7d3401e090 con 0x7f7d3c15e2f0 2023-12-01T01:49:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:17.074 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/2548453767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2548453767},{"type":"v1","addr":"172.21.15.142:6834","nonce":2548453767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3727358840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3727358840},{"type":"v1","addr":"172.21.15.142:6835","nonce":3727358840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/4086473290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4086473290},{"type":"v1","addr":"172.21.15.115:6836","nonce":4086473290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:49:11.094285+0000","modified":"2023-12-01T01:49:14.112475+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1756585806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1756585806},{"type":"v1","addr":"172.21.15.115:6837","nonce":1756585806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-01T01:49:17.074 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 24 2023-12-01T01:49:17.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 -- 172.21.15.115:0/2248016409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d2806a550 msgr2=0x7f7d2806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d2806a550 0x7f7d2806ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c007dc0 tx=0x7f7d2c0072f0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 -- 172.21.15.115:0/2248016409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c15e2f0 msgr2=0x7f7d3c0c4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c15e2f0 0x7f7d3c0c4d30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400c4c0 tx=0x7f7d3400c980 comp rx=0 tx=0).stop 2023-12-01T01:49:17.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 -- 172.21.15.115:0/2248016409 shutdown_connections 2023-12-01T01:49:17.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d2806a550 0x7f7d2806ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d3c0cad70 0x7f7d3c1f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d3c15e2f0 0x7f7d3c0c4d30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 --2- 172.21.15.115:0/2248016409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d3c06f2f0 0x7f7d3c1579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 -- 172.21.15.115:0/2248016409 >> 172.21.15.115:0/2248016409 conn(0x7f7d3c0ce880 msgr2=0x7f7d3c15d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:17.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 -- 172.21.15.115:0/2248016409 shutdown_connections 2023-12-01T01:49:17.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.074+0000 7f7d447e3640 1 -- 172.21.15.115:0/2248016409 wait complete. 2023-12-01T01:49:17.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T01:49:11.094285+0000', 'modified': '2023-12-01T01:49:14.112475+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5931}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5931': {'gid': 5931, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6837/1756585806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6835', 'nonce': 1756585806}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 1756585806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T01:49:17.497 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T01:49:17.497 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-01T01:49:17.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.719+0000 7ff9576f8640 1 Processor -- start 2023-12-01T01:49:17.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.720+0000 7ff9576f8640 1 -- start start 2023-12-01T01:49:17.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.721+0000 7ff9576f8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff9500cad70 0x7ff95006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.721+0000 7ff9576f8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff95006f2f0 0x7ff95006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.721+0000 7ff9576f8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9500add10 con 0x7ff95015e2f0 2023-12-01T01:49:17.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.721+0000 7ff9576f8640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9500c5760 con 0x7ff9500cad70 2023-12-01T01:49:17.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.721+0000 7ff9576f8640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95015e6d0 con 0x7ff95006f2f0 2023-12-01T01:49:17.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.721+0000 7ff955494640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff95015e2f0 0x7ff9500ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34760/0 (socket says 172.21.15.115:34760) 2023-12-01T01:49:17.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.721+0000 7ff955494640 1 -- 172.21.15.115:0/2701773970 learned_addr learned my addr 172.21.15.115:0/2701773970 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:17.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.721+0000 7ff954c93640 1 --2- 172.21.15.115:0/2701773970 >> v2:172.21.15.115:3300/0 conn(0x7ff9500cad70 0x7ff95006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.722+0000 7ff954c93640 1 -- 172.21.15.115:0/2701773970 >> v2:172.21.15.142:3300/0 conn(0x7ff95006f2f0 msgr2=0x7ff95006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:17.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.722+0000 7ff954c93640 1 --2- 172.21.15.115:0/2701773970 >> v2:172.21.15.142:3300/0 conn(0x7ff95006f2f0 0x7ff95006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.722+0000 7ff954c93640 1 -- 172.21.15.115:0/2701773970 >> v1:172.21.15.115:6789/0 conn(0x7ff95015e2f0 legacy=0x7ff9500ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.722+0000 7ff954c93640 1 -- 172.21.15.115:0/2701773970 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500cf7b0 con 0x7ff9500cad70 2023-12-01T01:49:17.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.722+0000 7ff947fff640 1 -- 172.21.15.115:0/2701773970 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129281371 0 0) 0x7ff9500add10 con 0x7ff95015e2f0 2023-12-01T01:49:17.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.723+0000 7ff954c93640 1 --2- 172.21.15.115:0/2701773970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9500cad70 0x7ff95006b5c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff9400097a0 tx=0x7ff94002b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3619b94fcd443beb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:17.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.723+0000 7ff947fff640 1 -- 172.21.15.115:0/2701773970 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940031520 con 0x7ff9500cad70 2023-12-01T01:49:17.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.723+0000 7ff947fff640 1 -- 172.21.15.115:0/2701773970 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff940031680 con 0x7ff9500cad70 2023-12-01T01:49:17.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.723+0000 7ff947fff640 1 -- 172.21.15.115:0/2701773970 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940031a10 con 0x7ff9500cad70 2023-12-01T01:49:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.724+0000 7ff9576f8640 1 -- 172.21.15.115:0/2701773970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9500cad70 msgr2=0x7ff95006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.724+0000 7ff9576f8640 1 --2- 172.21.15.115:0/2701773970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9500cad70 0x7ff95006b5c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff9400097a0 tx=0x7ff94002b7c0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.724+0000 7ff9576f8640 1 -- 172.21.15.115:0/2701773970 shutdown_connections 2023-12-01T01:49:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.724+0000 7ff9576f8640 1 --2- 172.21.15.115:0/2701773970 >> v2:172.21.15.142:3300/0 conn(0x7ff95006f2f0 0x7ff95006bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.724+0000 7ff9576f8640 1 --2- 172.21.15.115:0/2701773970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9500cad70 0x7ff95006b5c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.724+0000 7ff9576f8640 1 -- 172.21.15.115:0/2701773970 >> 172.21.15.115:0/2701773970 conn(0x7ff9500ce880 msgr2=0x7ff9500cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.724+0000 7ff9576f8640 1 -- 172.21.15.115:0/2701773970 shutdown_connections 2023-12-01T01:49:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.724+0000 7ff9576f8640 1 -- 172.21.15.115:0/2701773970 wait complete. 2023-12-01T01:49:17.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.726+0000 7ff9576f8640 1 Processor -- start 2023-12-01T01:49:17.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.726+0000 7ff9576f8640 1 -- start start 2023-12-01T01:49:17.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.727+0000 7ff9576f8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95006f2f0 0x7ff950159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.727+0000 7ff9576f8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9500cad70 0x7ff9500c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.727+0000 7ff9576f8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff95015e2f0 0x7ff950158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.727+0000 7ff9576f8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500c9630 con 0x7ff95015e2f0 2023-12-01T01:49:17.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.727+0000 7ff9576f8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500c7b80 con 0x7ff95006f2f0 2023-12-01T01:49:17.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.727+0000 7ff9576f8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff950159250 con 0x7ff9500cad70 2023-12-01T01:49:17.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.727+0000 7ff955c95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff95015e2f0 0x7ff950158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.727+0000 7ff955c95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff95015e2f0 0x7ff950158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52834/0 (socket says 172.21.15.115:52834) 2023-12-01T01:49:17.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.727+0000 7ff955c95640 1 -- 172.21.15.115:0/3267459786 learned_addr learned my addr 172.21.15.115:0/3267459786 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:17.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff955494640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95006f2f0 0x7ff950159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff954c93640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9500cad70 0x7ff9500c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff955c95640 1 -- 172.21.15.115:0/3267459786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9500cad70 msgr2=0x7ff9500c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff955c95640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9500cad70 0x7ff9500c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff955c95640 1 -- 172.21.15.115:0/3267459786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95006f2f0 msgr2=0x7ff950159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff955c95640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95006f2f0 0x7ff950159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff955c95640 1 -- 172.21.15.115:0/3267459786 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940009400 con 0x7ff95015e2f0 2023-12-01T01:49:17.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff955494640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95006f2f0 0x7ff950159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:17.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff955c95640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff95015e2f0 0x7ff950158d10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff94800be50 tx=0x7ff94800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:17.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff945ffb640 1 -- 172.21.15.115:0/3267459786 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff948014440 con 0x7ff95015e2f0 2023-12-01T01:49:17.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff945ffb640 1 -- 172.21.15.115:0/3267459786 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9480145a0 con 0x7ff95015e2f0 2023-12-01T01:49:17.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff945ffb640 1 -- 172.21.15.115:0/3267459786 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff948014950 con 0x7ff95015e2f0 2023-12-01T01:49:17.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.728+0000 7ff9576f8640 1 -- 172.21.15.115:0/3267459786 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9500c59f0 con 0x7ff95015e2f0 2023-12-01T01:49:17.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.729+0000 7ff9576f8640 1 -- 172.21.15.115:0/3267459786 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9500c5c90 con 0x7ff95015e2f0 2023-12-01T01:49:17.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.731+0000 7ff945ffb640 1 -- 172.21.15.115:0/3267459786 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff94801d3b0 con 0x7ff95015e2f0 2023-12-01T01:49:17.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.732+0000 7ff945ffb640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 0x7ff93406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.732+0000 7ff945ffb640 1 -- 172.21.15.115:0/3267459786 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff948088d90 con 0x7ff95015e2f0 2023-12-01T01:49:17.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.732+0000 7ff955494640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 0x7ff93406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.732+0000 7ff95a26a740 1 Processor -- start 2023-12-01T01:49:17.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.732+0000 7ff95a26a740 1 -- start start 2023-12-01T01:49:17.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.733+0000 7ff955494640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 0x7ff93406ca40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff94c000f60 tx=0x7ff94c009040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:17.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.733+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8e650 0x559b61c70730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.733+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8c170 0x559b61c70c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.733+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c6dd60 0x559b61c444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.733+0000 7ff95a26a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b61c1c2f0 con 0x559b61c8e650 2023-12-01T01:49:17.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.733+0000 7ff95a26a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b61bb0f80 con 0x559b61c6dd60 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.733+0000 7ff95a26a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b61bae710 con 0x559b61c8c170 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff955494640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8e650 0x559b61c70730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff955494640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8e650 0x559b61c70730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52848/0 (socket says 172.21.15.115:52848) 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff955494640 1 -- 172.21.15.115:0/1956434849 learned_addr learned my addr 172.21.15.115:0/1956434849 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff954c93640 1 --2- 172.21.15.115:0/1956434849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8c170 0x559b61c70c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff955c95640 1 --2- 172.21.15.115:0/1956434849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c6dd60 0x559b61c444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff955494640 1 -- 172.21.15.115:0/1956434849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8c170 msgr2=0x559b61c70c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff955494640 1 --2- 172.21.15.115:0/1956434849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8c170 0x559b61c70c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff955494640 1 -- 172.21.15.115:0/1956434849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c6dd60 msgr2=0x559b61c444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff955494640 1 --2- 172.21.15.115:0/1956434849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c6dd60 0x559b61c444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff955494640 1 -- 172.21.15.115:0/1956434849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b61b71690 con 0x559b61c8e650 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.734+0000 7ff955494640 1 --2- 172.21.15.115:0/1956434849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8e650 0x559b61c70730 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff94c005b90 tx=0x7ff94c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190008c93e82020b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.735+0000 7ff932ffd640 1 -- 172.21.15.115:0/1956434849 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00d9a0 con 0x559b61c8e650 2023-12-01T01:49:17.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.735+0000 7ff932ffd640 1 -- 172.21.15.115:0/1956434849 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c00db00 con 0x559b61c8e650 2023-12-01T01:49:17.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.735+0000 7ff932ffd640 1 -- 172.21.15.115:0/1956434849 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00de70 con 0x559b61c8e650 2023-12-01T01:49:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.735+0000 7ff95a26a740 1 -- 172.21.15.115:0/1956434849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8e650 msgr2=0x559b61c70730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.735+0000 7ff95a26a740 1 --2- 172.21.15.115:0/1956434849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8e650 0x559b61c70730 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff94c005b90 tx=0x7ff94c010000 comp rx=0 tx=0).stop 2023-12-01T01:49:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.735+0000 7ff95a26a740 1 -- 172.21.15.115:0/1956434849 shutdown_connections 2023-12-01T01:49:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.735+0000 7ff95a26a740 1 --2- 172.21.15.115:0/1956434849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8c170 0x559b61c70c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.735+0000 7ff95a26a740 1 --2- 172.21.15.115:0/1956434849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c6dd60 0x559b61c444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.735+0000 7ff95a26a740 1 --2- 172.21.15.115:0/1956434849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8e650 0x559b61c70730 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.735+0000 7ff95a26a740 1 -- 172.21.15.115:0/1956434849 >> 172.21.15.115:0/1956434849 conn(0x559b61c42bb0 msgr2=0x559b61c8b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.736+0000 7ff95a26a740 1 -- 172.21.15.115:0/1956434849 shutdown_connections 2023-12-01T01:49:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.736+0000 7ff95a26a740 1 -- 172.21.15.115:0/1956434849 wait complete. 2023-12-01T01:49:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.736+0000 7ff95a26a740 1 -- start start 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.736+0000 7ff95a26a740 1 Processor -- start 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.737+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c94f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.737+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8e650 0x559b61c678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.737+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c72e30 0x559b61c449b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.737+0000 7ff95a26a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b61baa000 con 0x559b61c8c170 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.737+0000 7ff95a26a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b61b4b7d0 con 0x559b61c72e30 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.737+0000 7ff95a26a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b61b4b470 con 0x559b61c8e650 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.737+0000 7ff95a26a740 10 client.0 _ll_register_callbacks cb 0x559b61b9a1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff955494640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c94f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff955494640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c94f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52860/0 (socket says 172.21.15.115:52860) 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff955494640 1 -- 172.21.15.115:0/3980922883 learned_addr learned my addr 172.21.15.115:0/3980922883 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff955c95640 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c72e30 0x559b61c449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff954c93640 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8e650 0x559b61c678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff955494640 1 -- 172.21.15.115:0/3980922883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8e650 msgr2=0x559b61c678f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff955494640 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8e650 0x559b61c678f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff955494640 1 -- 172.21.15.115:0/3980922883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c72e30 msgr2=0x559b61c449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff955494640 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c72e30 0x559b61c449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff955494640 1 -- 172.21.15.115:0/3980922883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c0058d0 con 0x559b61c8c170 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.738+0000 7ff955c95640 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c72e30 0x559b61c449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff955494640 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c94f10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0078c0 tx=0x7ff94c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b83a72852c2fac server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff92bfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff92bfff640 1 -- 172.21.15.115:0/3980922883 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c027070 con 0x559b61c8c170 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff92bfff640 1 -- 172.21.15.115:0/3980922883 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff94c02db10 con 0x559b61c8c170 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff95a26a740 1 -- 172.21.15.115:0/3980922883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b61bd4550 con 0x559b61c8c170 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff92bfff640 1 -- 172.21.15.115:0/3980922883 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff94c02dd10 con 0x559b61c8c170 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff95a26a740 10 client.6060 fetch_fsmap learned FSMap version 24 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff95a26a740 1 -- 172.21.15.115:0/3980922883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x559b61ba8680 con 0x559b61c8c170 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff92bfff640 1 -- 172.21.15.115:0/3980922883 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559b61bd4550 con 0x559b61c8c170 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff92bfff640 1 -- 172.21.15.115:0/3980922883 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff94c022570 con 0x559b61c8c170 2023-12-01T01:49:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff95a26a740 10 client.6060 fetch_fsmap finished waiting for FSMap version 24 2023-12-01T01:49:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.739+0000 7ff95a26a740 10 client.6060 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:49:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.740+0000 7ff95a26a740 20 client.6060 populate_metadata read hostname 'smithi115' 2023-12-01T01:49:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.740+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.115:6835/1756585806,v1:172.21.15.115:6837/1756585806] conn(0x559b61c9fdf0 0x559b61c71770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.740+0000 7ff95a26a740 4 client.6060 mds_command: new command op to 5931 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:49:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.740+0000 7ff95a26a740 1 -- 172.21.15.115:0/3980922883 --> [v2:172.21.15.115:6835/1756585806,v1:172.21.15.115:6837/1756585806] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b61b9cb50 con 0x559b61c9fdf0 2023-12-01T01:49:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.740+0000 7ff954c93640 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.115:6835/1756585806,v1:172.21.15.115:6837/1756585806] conn(0x559b61c9fdf0 0x559b61c71770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.740+0000 7ff954c93640 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.115:6835/1756585806,v1:172.21.15.115:6837/1756585806] conn(0x559b61c9fdf0 0x559b61c71770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26186e7a039907a4 server_cookie=48b132e130432933 in_seq=0 out_seq=0 2023-12-01T01:49:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.740+0000 7ff92bfff640 10 client.6060 ms_handle_connect on v2:172.21.15.115:6835/1756585806 2023-12-01T01:49:17.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff92bfff640 1 -- 172.21.15.115:0/3980922883 <== mds.0 v2:172.21.15.115:6835/1756585806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559b61b9cb50 con 0x559b61c9fdf0 2023-12-01T01:49:17.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff92bfff640 10 client.6060 handle_command_reply: tid=0 2023-12-01T01:49:17.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff95a26a740 1 client.6060 shutdown 2023-12-01T01:49:17.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff95a26a740 1 -- 172.21.15.115:0/3980922883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 msgr2=0x559b61c94f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c94f10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0078c0 tx=0x7ff94c012000 comp rx=0 tx=0).stop 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff95a26a740 1 -- 172.21.15.115:0/3980922883 shutdown_connections 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.115:6835/1756585806,v1:172.21.15.115:6837/1756585806] conn(0x559b61c9fdf0 0x559b61c71770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8e650 0x559b61c678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c72e30 0x559b61c449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff92bfff640 0 client.6060 ms_handle_reset on v2:172.21.15.115:6835/1756585806 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3980922883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c94f10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.741+0000 7ff95a26a740 1 -- 172.21.15.115:0/3980922883 >> 172.21.15.115:0/3980922883 conn(0x559b61c6dd60 msgr2=0x559b61c70620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.742+0000 7ff95a26a740 1 -- 172.21.15.115:0/3980922883 shutdown_connections 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.742+0000 7ff95a26a740 1 -- 172.21.15.115:0/3980922883 wait complete. 2023-12-01T01:49:17.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.742+0000 7ff95a26a740 20 client.6060 trim_cache size 0 max 16384 2023-12-01T01:49:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.758+0000 7ff95a26a740 1 Processor -- start 2023-12-01T01:49:17.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.758+0000 7ff95a26a740 1 -- start start 2023-12-01T01:49:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c72e30 0x559b61c70810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c9e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c8e650 0x559b61c9e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff95a26a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b61c46320 con 0x559b61c8c170 2023-12-01T01:49:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff95a26a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b61911f20 con 0x559b61c8e650 2023-12-01T01:49:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff95a26a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b61c441d0 con 0x559b61c72e30 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff954c93640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c9e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff954c93640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c9e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52866/0 (socket says 172.21.15.115:52866) 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff954c93640 1 -- 172.21.15.115:0/3383428565 learned_addr learned my addr 172.21.15.115:0/3383428565 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff954c93640 1 -- 172.21.15.115:0/3383428565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c72e30 msgr2=0x559b61c70810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.759+0000 7ff955494640 1 --2- 172.21.15.115:0/3383428565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c72e30 0x559b61c70810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.760+0000 7ff954c93640 1 --2- 172.21.15.115:0/3383428565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c72e30 0x559b61c70810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.760+0000 7ff954c93640 1 -- 172.21.15.115:0/3383428565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c8e650 msgr2=0x559b61c9e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.760+0000 7ff954c93640 1 --2- 172.21.15.115:0/3383428565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c8e650 0x559b61c9e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.760+0000 7ff954c93640 1 -- 172.21.15.115:0/3383428565 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b61b4a9e0 con 0x559b61c8c170 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.760+0000 7ff954c93640 1 --2- 172.21.15.115:0/3383428565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c9e3c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff9400097a0 tx=0x7ff94003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fee85a0e9934b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.760+0000 7ff9337fe640 1 -- 172.21.15.115:0/3383428565 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940049070 con 0x559b61c8c170 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.760+0000 7ff9337fe640 1 -- 172.21.15.115:0/3383428565 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff940045340 con 0x559b61c8c170 2023-12-01T01:49:17.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.760+0000 7ff9337fe640 1 -- 172.21.15.115:0/3383428565 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940031bb0 con 0x559b61c8c170 2023-12-01T01:49:17.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.761+0000 7ff95a26a740 1 -- 172.21.15.115:0/3383428565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 msgr2=0x559b61c9e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.761+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3383428565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c9e3c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff9400097a0 tx=0x7ff94003f040 comp rx=0 tx=0).stop 2023-12-01T01:49:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.761+0000 7ff95a26a740 1 -- 172.21.15.115:0/3383428565 shutdown_connections 2023-12-01T01:49:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.761+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3383428565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c8e650 0x559b61c9e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.761+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3383428565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c8c170 0x559b61c9e3c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.761+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3383428565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c72e30 0x559b61c70810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.761+0000 7ff95a26a740 1 -- 172.21.15.115:0/3383428565 >> 172.21.15.115:0/3383428565 conn(0x559b61c6dd60 msgr2=0x559b61c3c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.761+0000 7ff95a26a740 1 -- 172.21.15.115:0/3383428565 shutdown_connections 2023-12-01T01:49:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.761+0000 7ff95a26a740 1 -- 172.21.15.115:0/3383428565 wait complete. 2023-12-01T01:49:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.761+0000 7ff95a26a740 1 -- start start 2023-12-01T01:49:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.762+0000 7ff95a26a740 1 Processor -- start 2023-12-01T01:49:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c72e30 0x559b61c9e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c8c170 0x559b61ce86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff95a26a740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8e650 0x559b61ce8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff95a26a740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b61ce9120 con 0x559b61c72e30 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff95a26a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b61cf9100 con 0x559b61c8c170 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff95a26a740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b61c858e0 con 0x559b61c8e650 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff95a26a740 10 client.0 _ll_register_callbacks cb 0x7ff94c001da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff955494640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c72e30 0x559b61c9e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff955494640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c72e30 0x559b61c9e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52878/0 (socket says 172.21.15.115:52878) 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff955494640 1 -- 172.21.15.115:0/3584247666 learned_addr learned my addr 172.21.15.115:0/3584247666 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff955494640 1 -- 172.21.15.115:0/3584247666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8e650 msgr2=0x559b61ce8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff955494640 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8e650 0x559b61ce8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff955494640 1 -- 172.21.15.115:0/3584247666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c8c170 msgr2=0x559b61ce86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff955494640 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c8c170 0x559b61ce86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff955494640 1 -- 172.21.15.115:0/3584247666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9500cf7b0 con 0x559b61c72e30 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff955494640 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c72e30 0x559b61c9e7c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0011b0 tx=0x7ff94c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e68b67e5fc4124 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:17.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.763+0000 7ff92bfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff92bfff640 1 -- 172.21.15.115:0/3584247666 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c005a60 con 0x559b61c72e30 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff92bfff640 1 -- 172.21.15.115:0/3584247666 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff94c00dc70 con 0x559b61c72e30 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff95a26a740 1 -- 172.21.15.115:0/3584247666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff94c00db00 con 0x559b61c72e30 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff92bfff640 1 -- 172.21.15.115:0/3584247666 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff94c007c90 con 0x559b61c72e30 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff95a26a740 10 client.6066 fetch_fsmap learned FSMap version 24 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff95a26a740 1 -- 172.21.15.115:0/3584247666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x559b61b6fef0 con 0x559b61c72e30 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff92bfff640 1 -- 172.21.15.115:0/3584247666 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c00db00 con 0x559b61c72e30 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff92bfff640 1 -- 172.21.15.115:0/3584247666 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff94c022940 con 0x559b61c72e30 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff95a26a740 10 client.6066 fetch_fsmap finished waiting for FSMap version 24 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff95a26a740 10 client.6066 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff95a26a740 20 client.6066 populate_metadata read hostname 'smithi115' 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.115:6835/1756585806,v1:172.21.15.115:6837/1756585806] conn(0x559b61c645e0 0x559b61c649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff95a26a740 4 client.6066 mds_command: new command op to 5931 tid=0[{"prefix": "status"}] 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.764+0000 7ff95a26a740 1 -- 172.21.15.115:0/3584247666 --> [v2:172.21.15.115:6835/1756585806,v1:172.21.15.115:6837/1756585806] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559b61ba3080 con 0x559b61c645e0 2023-12-01T01:49:17.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.765+0000 7ff954c93640 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.115:6835/1756585806,v1:172.21.15.115:6837/1756585806] conn(0x559b61c645e0 0x559b61c649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.765+0000 7ff92bfff640 10 client.6066 ms_handle_connect on v2:172.21.15.115:6835/1756585806 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.765+0000 7ff954c93640 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.115:6835/1756585806,v1:172.21.15.115:6837/1756585806] conn(0x559b61c645e0 0x559b61c649c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf0de36dd567d8f6 server_cookie=6396dc127c3ac33f in_seq=0 out_seq=0 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.765+0000 7ff92bfff640 1 -- 172.21.15.115:0/3584247666 <== mds.0 v2:172.21.15.115:6835/1756585806 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x559b61ba3080 con 0x559b61c645e0 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.765+0000 7ff92bfff640 10 client.6066 handle_command_reply: tid=0 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.765+0000 7ff95a26a740 1 client.6066 shutdown 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 1 -- 172.21.15.115:0/3584247666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c72e30 msgr2=0x559b61c9e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c72e30 0x559b61c9e7c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0011b0 tx=0x7ff94c000ed0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 1 -- 172.21.15.115:0/3584247666 shutdown_connections 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.115:6835/1756585806,v1:172.21.15.115:6837/1756585806] conn(0x559b61c645e0 0x559b61c649c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x559b61c8e650 0x559b61ce8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b61c8c170 0x559b61ce86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff92bfff640 0 client.6066 ms_handle_reset on v2:172.21.15.115:6835/1756585806 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 1 --2- 172.21.15.115:0/3584247666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x559b61c72e30 0x559b61c9e7c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 1 -- 172.21.15.115:0/3584247666 >> 172.21.15.115:0/3584247666 conn(0x559b61c6dd60 msgr2=0x559b61c972e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:17.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 1 -- 172.21.15.115:0/3584247666 shutdown_connections 2023-12-01T01:49:17.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 1 -- 172.21.15.115:0/3584247666 wait complete. 2023-12-01T01:49:17.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff95a26a740 20 client.6066 trim_cache size 0 max 16384 2023-12-01T01:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T01:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T01:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 5931, 2023-12-01T01:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T01:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T01:49:17.771 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T01:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 6.6672177289999999, 2023-12-01T01:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 24, 2023-12-01T01:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 64, 2023-12-01T01:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 64, 2023-12-01T01:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 12.464732841 2023-12-01T01:49:17.772 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff9576f8640 1 -- 172.21.15.115:0/3267459786 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 msgr2=0x7ff93406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff9576f8640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 0x7ff93406ca40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff94c000f60 tx=0x7ff94c009040 comp rx=0 tx=0).stop 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff9576f8640 1 -- 172.21.15.115:0/3267459786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff95015e2f0 msgr2=0x7ff950158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.766+0000 7ff9576f8640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff95015e2f0 0x7ff950158d10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff94800be50 tx=0x7ff94800bf40 comp rx=0 tx=0).stop 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.767+0000 7ff9576f8640 1 -- 172.21.15.115:0/3267459786 shutdown_connections 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.767+0000 7ff9576f8640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 0x7ff93406ca40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.767+0000 7ff9576f8640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9500cad70 0x7ff9500c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.767+0000 7ff9576f8640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff95015e2f0 0x7ff950158d10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.767+0000 7ff9576f8640 1 --2- 172.21.15.115:0/3267459786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95006f2f0 0x7ff950159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.767+0000 7ff9576f8640 1 -- 172.21.15.115:0/3267459786 >> 172.21.15.115:0/3267459786 conn(0x7ff9500ce880 msgr2=0x7ff95015af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.767+0000 7ff9576f8640 1 -- 172.21.15.115:0/3267459786 shutdown_connections 2023-12-01T01:49:17.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.767+0000 7ff9576f8640 1 -- 172.21.15.115:0/3267459786 wait complete. 2023-12-01T01:49:17.784 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:49:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.994+0000 7f16ccbb1640 1 Processor -- start 2023-12-01T01:49:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.994+0000 7f16ccbb1640 1 -- start start 2023-12-01T01:49:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.996+0000 7f16ccbb1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80b5d80 0x7f16c80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.996+0000 7f16ccbb1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f16c80d1230 0x7f16c80d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.996+0000 7f16ccbb1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c80ce2e0 con 0x7f16c815cd20 2023-12-01T01:49:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.996+0000 7f16ccbb1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c80b2160 con 0x7f16c80d1230 2023-12-01T01:49:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.996+0000 7f16ccbb1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c80d1be0 con 0x7f16c80b5d80 2023-12-01T01:49:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.996+0000 7f16c6d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f16c80d1230 0x7f16c80d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.996+0000 7f16bdd9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80b5d80 0x7f16c80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.996+0000 7f16bdd9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80b5d80 0x7f16c80d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:37220/0 (socket says 172.21.15.115:37220) 2023-12-01T01:49:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.996+0000 7f16c6d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f16c80d1230 0x7f16c80d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:59636/0 (socket says 172.21.15.115:59636) 2023-12-01T01:49:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.996+0000 7f16bdd9b640 1 -- 172.21.15.115:0/4237150923 learned_addr learned my addr 172.21.15.115:0/4237150923 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.997+0000 7f16bdd9b640 1 -- 172.21.15.115:0/4237150923 >> v2:172.21.15.142:3300/0 conn(0x7f16c80d1230 msgr2=0x7f16c80d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.997+0000 7f16bdd9b640 1 --2- 172.21.15.115:0/4237150923 >> v2:172.21.15.142:3300/0 conn(0x7f16c80d1230 0x7f16c80d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.997+0000 7f16bdd9b640 1 -- 172.21.15.115:0/4237150923 >> v1:172.21.15.142:6789/0 conn(0x7f16c815cd20 legacy=0x7f16c80b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.997+0000 7f16bdd9b640 1 -- 172.21.15.115:0/4237150923 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c80ce000 con 0x7f16c80b5d80 2023-12-01T01:49:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.999+0000 7f16bdd9b640 1 --2- 172.21.15.115:0/4237150923 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80b5d80 0x7f16c80d24d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009870 tx=0x7f16b002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=461bdfc90793d99d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.999+0000 7f16c5d9b640 1 -- 172.21.15.115:0/4237150923 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0031640 con 0x7f16c80b5d80 2023-12-01T01:49:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.999+0000 7f16c5d9b640 1 -- 172.21.15.115:0/4237150923 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b00317a0 con 0x7f16c80b5d80 2023-12-01T01:49:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:17.999+0000 7f16c5d9b640 1 -- 172.21.15.115:0/4237150923 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0031b30 con 0x7f16c80b5d80 2023-12-01T01:49:18.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.000+0000 7f16ccbb1640 1 -- 172.21.15.115:0/4237150923 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80b5d80 msgr2=0x7f16c80d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:18.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.000+0000 7f16ccbb1640 1 --2- 172.21.15.115:0/4237150923 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80b5d80 0x7f16c80d24d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009870 tx=0x7f16b002b8f0 comp rx=0 tx=0).stop 2023-12-01T01:49:18.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.000+0000 7f16ccbb1640 1 -- 172.21.15.115:0/4237150923 shutdown_connections 2023-12-01T01:49:18.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.000+0000 7f16ccbb1640 1 --2- 172.21.15.115:0/4237150923 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80b5d80 0x7f16c80d24d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:18.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.000+0000 7f16ccbb1640 1 --2- 172.21.15.115:0/4237150923 >> v2:172.21.15.142:3300/0 conn(0x7f16c80d1230 0x7f16c80d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:18.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.000+0000 7f16ccbb1640 1 -- 172.21.15.115:0/4237150923 >> 172.21.15.115:0/4237150923 conn(0x7f16c806a0e0 msgr2=0x7f16c806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.000+0000 7f16ccbb1640 1 -- 172.21.15.115:0/4237150923 shutdown_connections 2023-12-01T01:49:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.000+0000 7f16ccbb1640 1 -- 172.21.15.115:0/4237150923 wait complete. 2023-12-01T01:49:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.002+0000 7f16ccbb1640 1 Processor -- start 2023-12-01T01:49:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.002+0000 7f16ccbb1640 1 -- start start 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16ccbb1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16c80b5d80 0x7f16c814ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16ccbb1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80d1230 0x7f16c814e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16ccbb1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16c815cd20 0x7f16c815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16ccbb1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c815f210 con 0x7f16c80b5d80 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16ccbb1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c814f700 con 0x7f16c815cd20 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16ccbb1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c815fd70 con 0x7f16c80d1230 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16c659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16c80b5d80 0x7f16c814ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16c659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16c80b5d80 0x7f16c814ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52884/0 (socket says 172.21.15.115:52884) 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16c659c640 1 -- 172.21.15.115:0/2304593917 learned_addr learned my addr 172.21.15.115:0/2304593917 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16bdd9b640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80d1230 0x7f16c814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:18.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.003+0000 7f16c6d9d640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16c815cd20 0x7f16c815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.004+0000 7f16c659c640 1 -- 172.21.15.115:0/2304593917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80d1230 msgr2=0x7f16c814e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.004+0000 7f16c659c640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80d1230 0x7f16c814e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.004+0000 7f16c659c640 1 -- 172.21.15.115:0/2304593917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16c815cd20 msgr2=0x7f16c815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.004+0000 7f16c659c640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16c815cd20 0x7f16c815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:18.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.004+0000 7f16c659c640 1 -- 172.21.15.115:0/2304593917 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b00094d0 con 0x7f16c80b5d80 2023-12-01T01:49:18.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.004+0000 7f16c659c640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16c80b5d80 0x7f16c814ed30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f16b800c450 tx=0x7f16b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:18.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.004+0000 7f16bf7fe640 1 -- 172.21.15.115:0/2304593917 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b80052d0 con 0x7f16c80b5d80 2023-12-01T01:49:18.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.004+0000 7f16bf7fe640 1 -- 172.21.15.115:0/2304593917 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8005430 con 0x7f16c80b5d80 2023-12-01T01:49:18.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.004+0000 7f16ccbb1640 1 -- 172.21.15.115:0/2304593917 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c814f270 con 0x7f16c80b5d80 2023-12-01T01:49:18.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.004+0000 7f16bf7fe640 1 -- 172.21.15.115:0/2304593917 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8012870 con 0x7f16c80b5d80 2023-12-01T01:49:18.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.005+0000 7f16ccbb1640 1 -- 172.21.15.115:0/2304593917 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c815f720 con 0x7f16c80b5d80 2023-12-01T01:49:18.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.006+0000 7f16ccbb1640 1 -- 172.21.15.115:0/2304593917 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c804e890 con 0x7f16c80b5d80 2023-12-01T01:49:18.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.006+0000 7f16bf7fe640 1 -- 172.21.15.115:0/2304593917 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f16b80129d0 con 0x7f16c80b5d80 2023-12-01T01:49:18.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.007+0000 7f16bf7fe640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16a806a550 0x7f16a806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:18.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.007+0000 7f16bf7fe640 1 -- 172.21.15.115:0/2304593917 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16b80886e0 con 0x7f16c80b5d80 2023-12-01T01:49:18.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.008+0000 7f16bdd9b640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16a806a550 0x7f16a806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:18.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.008+0000 7f16bdd9b640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16a806a550 0x7f16a806ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f16b0036000 tx=0x7f16b003c040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:18.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.012+0000 7f16bf7fe640 1 -- 172.21.15.115:0/2304593917 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f16b8053aa0 con 0x7f16c80b5d80 2023-12-01T01:49:18.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.241+0000 7f16ccbb1640 1 -- 172.21.15.115:0/2304593917 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f16c80cd680 con 0x7f16c80b5d80 2023-12-01T01:49:18.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.242+0000 7f16bf7fe640 1 -- 172.21.15.115:0/2304593917 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f16b80570c0 con 0x7f16c80b5d80 2023-12-01T01:49:18.245 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:49:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.244+0000 7f16ccbb1640 1 -- 172.21.15.115:0/2304593917 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16a806a550 msgr2=0x7f16a806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.244+0000 7f16ccbb1640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16a806a550 0x7f16a806ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f16b0036000 tx=0x7f16b003c040 comp rx=0 tx=0).stop 2023-12-01T01:49:18.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.244+0000 7f16ccbb1640 1 -- 172.21.15.115:0/2304593917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16c80b5d80 msgr2=0x7f16c814ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.244+0000 7f16ccbb1640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16c80b5d80 0x7f16c814ed30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f16b800c450 tx=0x7f16b800c910 comp rx=0 tx=0).stop 2023-12-01T01:49:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.244+0000 7f16ccbb1640 1 -- 172.21.15.115:0/2304593917 shutdown_connections 2023-12-01T01:49:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.244+0000 7f16ccbb1640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16a806a550 0x7f16a806ca10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.245+0000 7f16ccbb1640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16c80d1230 0x7f16c814e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.245+0000 7f16ccbb1640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16c815cd20 0x7f16c815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.245+0000 7f16ccbb1640 1 --2- 172.21.15.115:0/2304593917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16c80b5d80 0x7f16c814ed30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.245+0000 7f16ccbb1640 1 -- 172.21.15.115:0/2304593917 >> 172.21.15.115:0/2304593917 conn(0x7f16c806a0e0 msgr2=0x7f16c81599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.245+0000 7f16ccbb1640 1 -- 172.21.15.115:0/2304593917 shutdown_connections 2023-12-01T01:49:18.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:18.245+0000 7f16ccbb1640 1 -- 172.21.15.115:0/2304593917 wait complete. 2023-12-01T01:49:18.265 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T01:49:18.265 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:49:18.265 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T01:49:18.265 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T01:49:18.265 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:49:18.265 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T01:49:18.265 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T01:49:18.265 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:49:18.265 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:49:18.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:18.266 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T01:49:18.320 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:49:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:49:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:49:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:49:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:49:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 41599sec preferred_lft 41599sec 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:49:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T01:49:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:49:18.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:18.323 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:49:18.408 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T01:49:18.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:18.412 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T01:49:18.466 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T01:49:18.467 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:49:18.467 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:49:18.467 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:49:18.467 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:49:18.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:18.468 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:49:18.468 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:49:18.468 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:49:18.468 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:49:18.468 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:49:18.469 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:49:18.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:18.659 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:49:18.710 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T01:49:18.710 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T01:49:18.711 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:49:18.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:18.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T01:49:18.809 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:49:18.809 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T01:49:18.863 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:49:18.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:49:18.864 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:49:18.914 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:49:18.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:18.914 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:49:19.055 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:49:19.055 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:49:19.055 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:49:19.078 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:49:19.078 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:49:19.078 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:49:19.078 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:49:19.078 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:49:19.103 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:49:19.103 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:49:19.104 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:49:19.104 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:49:19.104 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:49:19.104 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:49:19.104 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:49:19.104 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:49:19.106 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T01:49:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:49:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:49:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:49:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:49:19.110 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:49:19.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:19.111 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:49:19.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:19.198 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:49:19.260 INFO:teuthology.orchestra.run.smithi115.stdout:client6072 2023-12-01T01:49:19.260 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:49:19.261 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:49:19.308 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T01:49:19.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.560+0000 7fbff3ece640 1 Processor -- start 2023-12-01T01:49:19.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.561+0000 7fbff3ece640 1 -- start start 2023-12-01T01:49:19.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.562+0000 7fbff3ece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfec15cd20 0x7fbfec0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:19.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.562+0000 7fbff3ece640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbfec0d1230 0x7fbfec0d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:19.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.562+0000 7fbff3ece640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfec0ce2e0 con 0x7fbfec0b5d80 2023-12-01T01:49:19.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.562+0000 7fbff3ece640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfec0b2160 con 0x7fbfec0d1230 2023-12-01T01:49:19.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.562+0000 7fbff3ece640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec0d1be0 con 0x7fbfec15cd20 2023-12-01T01:49:19.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.562+0000 7fbff246b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbfec0d1230 0x7fbfec0d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:19.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.562+0000 7fbff246b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbfec0d1230 0x7fbfec0d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52896/0 (socket says 172.21.15.115:52896) 2023-12-01T01:49:19.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.562+0000 7fbff246b640 1 -- 172.21.15.115:0/3772167707 learned_addr learned my addr 172.21.15.115:0/3772167707 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:19.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.563+0000 7fbff246b640 1 -- 172.21.15.115:0/3772167707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfec15cd20 msgr2=0x7fbfec0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:19.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.563+0000 7fbff246b640 1 --2- 172.21.15.115:0/3772167707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfec15cd20 0x7fbfec0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:19.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.563+0000 7fbff246b640 1 -- 172.21.15.115:0/3772167707 >> v1:172.21.15.142:6789/0 conn(0x7fbfec0b5d80 legacy=0x7fbfec0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:19.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.563+0000 7fbff246b640 1 -- 172.21.15.115:0/3772167707 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec0ce000 con 0x7fbfec0d1230 2023-12-01T01:49:19.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.564+0000 7fbff246b640 1 --2- 172.21.15.115:0/3772167707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 0x7fbfec0d16a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009870 tx=0x7fbfe402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb97aefb7866a886 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:19.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.564+0000 7fbff0c68640 1 -- 172.21.15.115:0/3772167707 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40317c0 con 0x7fbfec0d1230 2023-12-01T01:49:19.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.564+0000 7fbff0c68640 1 -- 172.21.15.115:0/3772167707 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4031920 con 0x7fbfec0d1230 2023-12-01T01:49:19.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.564+0000 7fbff0c68640 1 -- 172.21.15.115:0/3772167707 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4031cb0 con 0x7fbfec0d1230 2023-12-01T01:49:19.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.565+0000 7fbff3ece640 1 -- 172.21.15.115:0/3772167707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 msgr2=0x7fbfec0d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:19.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.565+0000 7fbff3ece640 1 --2- 172.21.15.115:0/3772167707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 0x7fbfec0d16a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009870 tx=0x7fbfe402b860 comp rx=0 tx=0).stop 2023-12-01T01:49:19.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.565+0000 7fbff3ece640 1 -- 172.21.15.115:0/3772167707 shutdown_connections 2023-12-01T01:49:19.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.565+0000 7fbff3ece640 1 --2- 172.21.15.115:0/3772167707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 0x7fbfec0d16a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:19.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.565+0000 7fbff3ece640 1 --2- 172.21.15.115:0/3772167707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfec15cd20 0x7fbfec0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:19.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.565+0000 7fbff3ece640 1 -- 172.21.15.115:0/3772167707 >> 172.21.15.115:0/3772167707 conn(0x7fbfec06a0e0 msgr2=0x7fbfec06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:19.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.565+0000 7fbff3ece640 1 -- 172.21.15.115:0/3772167707 shutdown_connections 2023-12-01T01:49:19.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.565+0000 7fbff3ece640 1 -- 172.21.15.115:0/3772167707 wait complete. 2023-12-01T01:49:19.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.567+0000 7fbff3ece640 1 Processor -- start 2023-12-01T01:49:19.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.567+0000 7fbff3ece640 1 -- start start 2023-12-01T01:49:19.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.568+0000 7fbff3ece640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfec0b5d80 0x7fbfec15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:19.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.568+0000 7fbff3ece640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 0x7fbfec151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:19.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.568+0000 7fbff3ece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfec15cd20 0x7fbfec0b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.568+0000 7fbff3ece640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec0b32b0 con 0x7fbfec0d1230 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.568+0000 7fbff3ece640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15f0a0 con 0x7fbfec0b5d80 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.568+0000 7fbff3ece640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15f2d0 con 0x7fbfec15cd20 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.568+0000 7fbff1469640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 0x7fbfec151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbff1469640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 0x7fbfec151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52904/0 (socket says 172.21.15.115:52904) 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbff1469640 1 -- 172.21.15.115:0/2860872229 learned_addr learned my addr 172.21.15.115:0/2860872229 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbff246b640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfec15cd20 0x7fbfec0b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbff1469640 1 -- 172.21.15.115:0/2860872229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfec15cd20 msgr2=0x7fbfec0b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbff1469640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfec15cd20 0x7fbfec0b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbff1469640 1 -- 172.21.15.115:0/2860872229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfec0b5d80 msgr2=0x7fbfec15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbff1469640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfec0b5d80 0x7fbfec15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbff1469640 1 -- 172.21.15.115:0/2860872229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40094d0 con 0x7fbfec0d1230 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbff246b640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfec15cd20 0x7fbfec0b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:19.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbff1469640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 0x7fbfec151140 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800b3f0 tx=0x7fbfd800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:19.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.569+0000 7fbfea7fc640 1 -- 172.21.15.115:0/2860872229 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8011830 con 0x7fbfec0d1230 2023-12-01T01:49:19.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.570+0000 7fbfea7fc640 1 -- 172.21.15.115:0/2860872229 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd8011990 con 0x7fbfec0d1230 2023-12-01T01:49:19.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.570+0000 7fbfea7fc640 1 -- 172.21.15.115:0/2860872229 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8011d20 con 0x7fbfec0d1230 2023-12-01T01:49:19.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.570+0000 7fbff3ece640 1 -- 172.21.15.115:0/2860872229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec15f5b0 con 0x7fbfec0d1230 2023-12-01T01:49:19.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.570+0000 7fbff3ece640 1 -- 172.21.15.115:0/2860872229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec15fb70 con 0x7fbfec0d1230 2023-12-01T01:49:19.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.571+0000 7fbff3ece640 1 -- 172.21.15.115:0/2860872229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec0d2990 con 0x7fbfec0d1230 2023-12-01T01:49:19.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.572+0000 7fbfea7fc640 1 -- 172.21.15.115:0/2860872229 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fbfd801a400 con 0x7fbfec0d1230 2023-12-01T01:49:19.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.574+0000 7fbfea7fc640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfd006a550 0x7fbfd006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:19.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.574+0000 7fbfea7fc640 1 -- 172.21.15.115:0/2860872229 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfd8088c60 con 0x7fbfec0d1230 2023-12-01T01:49:19.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.574+0000 7fbff1c6a640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfd006a550 0x7fbfd006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:19.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.575+0000 7fbff1c6a640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfd006a550 0x7fbfd006ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000ab90 tx=0x7fbfe0009250 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:19.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.580+0000 7fbfea7fc640 1 -- 172.21.15.115:0/2860872229 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbfd8054130 con 0x7fbfec0d1230 2023-12-01T01:49:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.796+0000 7fbff3ece640 1 -- 172.21.15.115:0/2860872229 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfec0cd3d0 con 0x7fbfd006a550 2023-12-01T01:49:19.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.799+0000 7fbfea7fc640 1 -- 172.21.15.115:0/2860872229 <== mgr.5706 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbfe00163f0 con 0x7fbfd006a550 2023-12-01T01:49:19.800 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:49:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:49:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T01:49:19.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:49:19.801 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:49:19.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.801+0000 7fbff3ece640 1 -- 172.21.15.115:0/2860872229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfd006a550 msgr2=0x7fbfd006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:19.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.801+0000 7fbff3ece640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfd006a550 0x7fbfd006ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000ab90 tx=0x7fbfe0009250 comp rx=0 tx=0).stop 2023-12-01T01:49:19.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.801+0000 7fbff3ece640 1 -- 172.21.15.115:0/2860872229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 msgr2=0x7fbfec151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:19.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.801+0000 7fbff3ece640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 0x7fbfec151140 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800b3f0 tx=0x7fbfd800b8b0 comp rx=0 tx=0).stop 2023-12-01T01:49:19.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.802+0000 7fbff3ece640 1 -- 172.21.15.115:0/2860872229 shutdown_connections 2023-12-01T01:49:19.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.802+0000 7fbff3ece640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfd006a550 0x7fbfd006ca10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:19.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.802+0000 7fbff3ece640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfec15cd20 0x7fbfec0b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:19.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.802+0000 7fbff3ece640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfec0d1230 0x7fbfec151140 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:19.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.802+0000 7fbff3ece640 1 --2- 172.21.15.115:0/2860872229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfec0b5d80 0x7fbfec15eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:19.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.802+0000 7fbff3ece640 1 -- 172.21.15.115:0/2860872229 >> 172.21.15.115:0/2860872229 conn(0x7fbfec06a0e0 msgr2=0x7fbfec0b8420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:19.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.802+0000 7fbff3ece640 1 -- 172.21.15.115:0/2860872229 shutdown_connections 2023-12-01T01:49:19.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:19.802+0000 7fbff3ece640 1 -- 172.21.15.115:0/2860872229 wait complete. 2023-12-01T01:49:19.822 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T01:49:20.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.036+0000 7f5332a5f640 1 Processor -- start 2023-12-01T01:49:20.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.036+0000 7f5332a5f640 1 -- start start 2023-12-01T01:49:20.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.038+0000 7f5332a5f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f532c160200 0x7f532c0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.038+0000 7f5332a5f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f532c0cbd30 0x7f532c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.038+0000 7f5332a5f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c0cb290 con 0x7f532c0b0f80 2023-12-01T01:49:20.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.038+0000 7f5332a5f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c0b35f0 con 0x7f532c0cbd30 2023-12-01T01:49:20.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.038+0000 7f5332a5f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c0cc6a0 con 0x7f532c160200 2023-12-01T01:49:20.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.038+0000 7f5330ffc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f532c0cbd30 0x7f532c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:20.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.038+0000 7f5323fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f532c0b0f80 0x7f532c06b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34764/0 (socket says 172.21.15.115:34764) 2023-12-01T01:49:20.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.038+0000 7f5323fff640 1 -- 172.21.15.115:0/3292999372 learned_addr learned my addr 172.21.15.115:0/3292999372 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:20.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.038+0000 7f532bfff640 1 --2- 172.21.15.115:0/3292999372 >> v2:172.21.15.142:3300/0 conn(0x7f532c160200 0x7f532c0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:20.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.039+0000 7f532b7fe640 1 -- 172.21.15.115:0/3292999372 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463373737 0 0) 0x7f532c0cb290 con 0x7f532c0b0f80 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.039+0000 7f5330ffc640 1 -- 172.21.15.115:0/3292999372 >> v2:172.21.15.142:3300/0 conn(0x7f532c160200 msgr2=0x7f532c0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.039+0000 7f5330ffc640 1 --2- 172.21.15.115:0/3292999372 >> v2:172.21.15.142:3300/0 conn(0x7f532c160200 0x7f532c0b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.039+0000 7f5330ffc640 1 -- 172.21.15.115:0/3292999372 >> v1:172.21.15.115:6789/0 conn(0x7f532c0b0f80 legacy=0x7f532c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.039+0000 7f5330ffc640 1 -- 172.21.15.115:0/3292999372 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c0b0ae0 con 0x7f532c0cbd30 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.040+0000 7f5330ffc640 1 --2- 172.21.15.115:0/3292999372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 0x7f532c0cc160 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f532c06ce00 tx=0x7f532402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2c0fde9cde1d26 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.040+0000 7f532b7fe640 1 -- 172.21.15.115:0/3292999372 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532402cc80 con 0x7f532c0cbd30 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.040+0000 7f532b7fe640 1 -- 172.21.15.115:0/3292999372 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532402cde0 con 0x7f532c0cbd30 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.040+0000 7f532b7fe640 1 -- 172.21.15.115:0/3292999372 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324034660 con 0x7f532c0cbd30 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.041+0000 7f5332a5f640 1 -- 172.21.15.115:0/3292999372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 msgr2=0x7f532c0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.041+0000 7f5332a5f640 1 --2- 172.21.15.115:0/3292999372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 0x7f532c0cc160 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f532c06ce00 tx=0x7f532402bed0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.041+0000 7f5332a5f640 1 -- 172.21.15.115:0/3292999372 shutdown_connections 2023-12-01T01:49:20.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.041+0000 7f5332a5f640 1 --2- 172.21.15.115:0/3292999372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 0x7f532c0cc160 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.041+0000 7f5332a5f640 1 --2- 172.21.15.115:0/3292999372 >> v2:172.21.15.142:3300/0 conn(0x7f532c160200 0x7f532c0b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.041+0000 7f5332a5f640 1 -- 172.21.15.115:0/3292999372 >> 172.21.15.115:0/3292999372 conn(0x7f532c065960 msgr2=0x7f532c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:20.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.041+0000 7f5332a5f640 1 -- 172.21.15.115:0/3292999372 shutdown_connections 2023-12-01T01:49:20.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.041+0000 7f5332a5f640 1 -- 172.21.15.115:0/3292999372 wait complete. 2023-12-01T01:49:20.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.043+0000 7f5332a5f640 1 Processor -- start 2023-12-01T01:49:20.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.043+0000 7f5332a5f640 1 -- start start 2023-12-01T01:49:20.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.044+0000 7f5332a5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c0b0f80 0x7f532c06b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.044+0000 7f5332a5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 0x7f532c1f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.044+0000 7f5332a5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f532c160200 0x7f532c1f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.044+0000 7f5332a5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c06b6a0 con 0x7f532c0cbd30 2023-12-01T01:49:20.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.044+0000 7f5332a5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c06b940 con 0x7f532c0b0f80 2023-12-01T01:49:20.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.044+0000 7f5332a5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c06bc20 con 0x7f532c160200 2023-12-01T01:49:20.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.044+0000 7f5323fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 0x7f532c1f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.044+0000 7f5323fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 0x7f532c1f5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52908/0 (socket says 172.21.15.115:52908) 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.044+0000 7f5323fff640 1 -- 172.21.15.115:0/1958610007 learned_addr learned my addr 172.21.15.115:0/1958610007 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f532bfff640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c0b0f80 0x7f532c06b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f5323fff640 1 -- 172.21.15.115:0/1958610007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f532c160200 msgr2=0x7f532c1f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f5330ffc640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f532c160200 0x7f532c1f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f5323fff640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f532c160200 0x7f532c1f5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f5323fff640 1 -- 172.21.15.115:0/1958610007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c0b0f80 msgr2=0x7f532c06b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f5323fff640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c0b0f80 0x7f532c06b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f5323fff640 1 -- 172.21.15.115:0/1958610007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5324009a70 con 0x7f532c0cbd30 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f532bfff640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c0b0f80 0x7f532c06b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:20.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f5323fff640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 0x7f532c1f5630 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f531400c4d0 tx=0x7f531400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:20.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f53297fa640 1 -- 172.21.15.115:0/1958610007 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53140074e0 con 0x7f532c0cbd30 2023-12-01T01:49:20.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f53297fa640 1 -- 172.21.15.115:0/1958610007 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314007660 con 0x7f532c0cbd30 2023-12-01T01:49:20.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f5332a5f640 1 -- 172.21.15.115:0/1958610007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c06bf00 con 0x7f532c0cbd30 2023-12-01T01:49:20.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f53297fa640 1 -- 172.21.15.115:0/1958610007 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314004a20 con 0x7f532c0cbd30 2023-12-01T01:49:20.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.045+0000 7f5332a5f640 1 -- 172.21.15.115:0/1958610007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c1f2670 con 0x7f532c0cbd30 2023-12-01T01:49:20.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.046+0000 7f5332a5f640 1 -- 172.21.15.115:0/1958610007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c04e890 con 0x7f532c0cbd30 2023-12-01T01:49:20.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.047+0000 7f53297fa640 1 -- 172.21.15.115:0/1958610007 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 19) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f5314004b80 con 0x7f532c0cbd30 2023-12-01T01:49:20.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.048+0000 7f53297fa640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f530c06a550 0x7f530c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.048+0000 7f53297fa640 1 -- 172.21.15.115:0/1958610007 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5314087fe0 con 0x7f532c0cbd30 2023-12-01T01:49:20.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.049+0000 7f532bfff640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f530c06a550 0x7f530c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:20.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.049+0000 7f532bfff640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f530c06a550 0x7f530c06ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f531c006fd0 tx=0x7f531c008040 comp rx=0 tx=0).ready entity=mgr.5706 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:20.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.053+0000 7f53297fa640 1 -- 172.21.15.115:0/1958610007 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f53140534f0 con 0x7f532c0cbd30 2023-12-01T01:49:20.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.291+0000 7f5332a5f640 1 -- 172.21.15.115:0/1958610007 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f532c0b1e20 con 0x7f532c0cbd30 2023-12-01T01:49:20.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.463+0000 7f53297fa640 1 -- 172.21.15.115:0/1958610007 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 20) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f53140538c0 con 0x7f532c0cbd30 2023-12-01T01:49:20.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.465+0000 7f53297fa640 1 -- 172.21.15.115:0/1958610007 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7f5314072c70 con 0x7f532c0cbd30 2023-12-01T01:49:20.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 -- 172.21.15.115:0/1958610007 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f530c06a550 msgr2=0x7f530c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f530c06a550 0x7f530c06ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f531c006fd0 tx=0x7f531c008040 comp rx=0 tx=0).stop 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 -- 172.21.15.115:0/1958610007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 msgr2=0x7f532c1f5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 0x7f532c1f5630 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f531400c4d0 tx=0x7f531400c990 comp rx=0 tx=0).stop 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 -- 172.21.15.115:0/1958610007 shutdown_connections 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f530c06a550 0x7f530c06ca10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f532c160200 0x7f532c1f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f532c0cbd30 0x7f532c1f5630 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 --2- 172.21.15.115:0/1958610007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c0b0f80 0x7f532c06b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 -- 172.21.15.115:0/1958610007 >> 172.21.15.115:0/1958610007 conn(0x7f532c065960 msgr2=0x7f532c152f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 -- 172.21.15.115:0/1958610007 shutdown_connections 2023-12-01T01:49:20.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.468+0000 7f5332a5f640 1 -- 172.21.15.115:0/1958610007 wait complete. 2023-12-01T01:49:20.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T01:49:20.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.685+0000 7effd8b45640 1 Processor -- start 2023-12-01T01:49:20.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.685+0000 7effd8b45640 1 -- start start 2023-12-01T01:49:20.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.687+0000 7effd8b45640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7effd4144120 0x7effd415e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.687+0000 7effd8b45640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7effd4070b10 0x7effd4070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.687+0000 7effd8b45640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd415bf00 con 0x7effd4065960 2023-12-01T01:49:20.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.687+0000 7effd8b45640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd415eab0 con 0x7effd4144120 2023-12-01T01:49:20.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.687+0000 7effd8b45640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd415b2f0 con 0x7effd4070b10 2023-12-01T01:49:20.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.687+0000 7effd37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7effd4144120 0x7effd415e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:20.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.687+0000 7effd37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7effd4144120 0x7effd415e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52938/0 (socket says 172.21.15.115:52938) 2023-12-01T01:49:20.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.687+0000 7effd37fe640 1 -- 172.21.15.115:0/3032703513 learned_addr learned my addr 172.21.15.115:0/3032703513 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:20.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.688+0000 7effd2ffd640 1 --2- 172.21.15.115:0/3032703513 >> v2:172.21.15.142:3300/0 conn(0x7effd4070b10 0x7effd4070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:20.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.688+0000 7effd37fe640 1 -- 172.21.15.115:0/3032703513 >> v2:172.21.15.142:3300/0 conn(0x7effd4070b10 msgr2=0x7effd4070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:20.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.688+0000 7effd37fe640 1 --2- 172.21.15.115:0/3032703513 >> v2:172.21.15.142:3300/0 conn(0x7effd4070b10 0x7effd4070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.688+0000 7effd37fe640 1 -- 172.21.15.115:0/3032703513 >> v1:172.21.15.142:6789/0 conn(0x7effd4065960 legacy=0x7effd4065d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:20.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.688+0000 7effd37fe640 1 -- 172.21.15.115:0/3032703513 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd415b5d0 con 0x7effd4144120 2023-12-01T01:49:20.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.688+0000 7effd2ffd640 1 --2- 172.21.15.115:0/3032703513 >> v2:172.21.15.142:3300/0 conn(0x7effd4070b10 0x7effd4070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:20.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.689+0000 7effd37fe640 1 --2- 172.21.15.115:0/3032703513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4144120 0x7effd415e570 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7effd40cf540 tx=0x7effc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e45ae1fa83f786d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:20.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.689+0000 7effd27fc640 1 -- 172.21.15.115:0/3032703513 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4031870 con 0x7effd4144120 2023-12-01T01:49:20.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.689+0000 7effd27fc640 1 -- 172.21.15.115:0/3032703513 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc40319d0 con 0x7effd4144120 2023-12-01T01:49:20.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.689+0000 7effd27fc640 1 -- 172.21.15.115:0/3032703513 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4031d60 con 0x7effd4144120 2023-12-01T01:49:20.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.690+0000 7effd8b45640 1 -- 172.21.15.115:0/3032703513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4144120 msgr2=0x7effd415e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:20.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.690+0000 7effd8b45640 1 --2- 172.21.15.115:0/3032703513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4144120 0x7effd415e570 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7effd40cf540 tx=0x7effc402b910 comp rx=0 tx=0).stop 2023-12-01T01:49:20.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.690+0000 7effd8b45640 1 -- 172.21.15.115:0/3032703513 shutdown_connections 2023-12-01T01:49:20.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.690+0000 7effd8b45640 1 --2- 172.21.15.115:0/3032703513 >> v2:172.21.15.142:3300/0 conn(0x7effd4070b10 0x7effd4070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.690+0000 7effd8b45640 1 --2- 172.21.15.115:0/3032703513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4144120 0x7effd415e570 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.690+0000 7effd8b45640 1 -- 172.21.15.115:0/3032703513 >> 172.21.15.115:0/3032703513 conn(0x7effd4073330 msgr2=0x7effd40cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:20.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.690+0000 7effd8b45640 1 -- 172.21.15.115:0/3032703513 shutdown_connections 2023-12-01T01:49:20.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.690+0000 7effd8b45640 1 -- 172.21.15.115:0/3032703513 wait complete. 2023-12-01T01:49:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.692+0000 7effd8b45640 1 Processor -- start 2023-12-01T01:49:20.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.692+0000 7effd8b45640 1 -- start start 2023-12-01T01:49:20.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.693+0000 7effd8b45640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4065960 0x7effd41f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.693+0000 7effd8b45640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effd4070b10 0x7effd40c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.693+0000 7effd8b45640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effd4144120 0x7effd4141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.693+0000 7effd8b45640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd415e8f0 con 0x7effd4065960 2023-12-01T01:49:20.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.693+0000 7effd8b45640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd40cc660 con 0x7effd4144120 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.693+0000 7effd8b45640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd41f20f0 con 0x7effd4070b10 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.693+0000 7effd37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4065960 0x7effd41f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.693+0000 7effd37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4065960 0x7effd41f1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52948/0 (socket says 172.21.15.115:52948) 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.693+0000 7effd37fe640 1 -- 172.21.15.115:0/112634028 learned_addr learned my addr 172.21.15.115:0/112634028 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.694+0000 7effd37fe640 1 -- 172.21.15.115:0/112634028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effd4070b10 msgr2=0x7effd40c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.694+0000 7effd37fe640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effd4070b10 0x7effd40c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.694+0000 7effd37fe640 1 -- 172.21.15.115:0/112634028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effd4144120 msgr2=0x7effd4141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.694+0000 7effd3fff640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effd4144120 0x7effd4141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.694+0000 7effd37fe640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effd4144120 0x7effd4141b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.694+0000 7effd37fe640 1 -- 172.21.15.115:0/112634028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc40094d0 con 0x7effd4065960 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.694+0000 7effd37fe640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4065960 0x7effd41f1bb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7effc4000c00 tx=0x7effc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.694+0000 7effcbfff640 1 -- 172.21.15.115:0/112634028 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4031840 con 0x7effd4065960 2023-12-01T01:49:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.694+0000 7effcbfff640 1 -- 172.21.15.115:0/112634028 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc40443f0 con 0x7effd4065960 2023-12-01T01:49:20.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.694+0000 7effcbfff640 1 -- 172.21.15.115:0/112634028 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40447c0 con 0x7effd4065960 2023-12-01T01:49:20.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.695+0000 7effd8b45640 1 -- 172.21.15.115:0/112634028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd415e330 con 0x7effd4065960 2023-12-01T01:49:20.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.695+0000 7effd8b45640 1 -- 172.21.15.115:0/112634028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd40cc2f0 con 0x7effd4065960 2023-12-01T01:49:20.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.696+0000 7effd8b45640 1 -- 172.21.15.115:0/112634028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd415bb50 con 0x7effd4065960 2023-12-01T01:49:20.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.697+0000 7effcbfff640 1 -- 172.21.15.115:0/112634028 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 88726+0+0 (secure 0 0 0) 0x7effc4044920 con 0x7effd4065960 2023-12-01T01:49:20.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.698+0000 7effcbfff640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7effb406a5a0 0x7effb406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:20.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.698+0000 7effcbfff640 1 -- 172.21.15.115:0/112634028 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effc402d040 con 0x7effd4065960 2023-12-01T01:49:20.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.698+0000 7effd2ffd640 1 -- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7effb406a5a0 msgr2=0x7effb406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:49:20.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.698+0000 7effd2ffd640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7effb406a5a0 0x7effb406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:49:20.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.707+0000 7effcbfff640 1 -- 172.21.15.115:0/112634028 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7effc4076980 con 0x7effd4065960 2023-12-01T01:49:20.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.899+0000 7effd2ffd640 1 -- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7effb406a5a0 msgr2=0x7effb406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:49:20.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.899+0000 7effd2ffd640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7effb406a5a0 0x7effb406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:49:20.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.901+0000 7effd8b45640 1 -- 172.21.15.115:0/112634028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7effd40618d0 con 0x7effd4065960 2023-12-01T01:49:20.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.908+0000 7effcbfff640 1 -- 172.21.15.115:0/112634028 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7effc4079fa0 con 0x7effd4065960 2023-12-01T01:49:20.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.910+0000 7effd8b45640 1 -- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7effb406a5a0 msgr2=0x7effb406ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:49:20.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.910+0000 7effd8b45640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7effb406a5a0 0x7effb406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.910+0000 7effd8b45640 1 -- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4065960 msgr2=0x7effd41f1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:20.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.910+0000 7effd8b45640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4065960 0x7effd41f1bb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7effc4000c00 tx=0x7effc40046c0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.911+0000 7effd8b45640 1 -- 172.21.15.115:0/112634028 shutdown_connections 2023-12-01T01:49:20.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.911+0000 7effd8b45640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7effb406a5a0 0x7effb406ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.911+0000 7effd8b45640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effd4070b10 0x7effd40c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.911+0000 7effd8b45640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effd4144120 0x7effd4141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.911+0000 7effd8b45640 1 --2- 172.21.15.115:0/112634028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7effd4065960 0x7effd41f1bb0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:20.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.911+0000 7effd8b45640 1 -- 172.21.15.115:0/112634028 >> 172.21.15.115:0/112634028 conn(0x7effd4073330 msgr2=0x7effd415ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:20.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.911+0000 7effd8b45640 1 -- 172.21.15.115:0/112634028 shutdown_connections 2023-12-01T01:49:20.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:20.911+0000 7effd8b45640 1 -- 172.21.15.115:0/112634028 wait complete. 2023-12-01T01:49:20.929 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T01:49:20.971 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:20.969+0000 7f93ebe46040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:49:21.004 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:21.003+0000 7fbd10246040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:49:21.022 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:21.021+0000 7f93ebe46040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:49:21.060 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:21.058+0000 7fbd10246040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:49:21.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.126+0000 7f313f7ab640 1 Processor -- start 2023-12-01T01:49:21.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.127+0000 7f313f7ab640 1 -- start start 2023-12-01T01:49:21.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.128+0000 7f313f7ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31380ccf90 0x7f31380cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.128+0000 7f313f7ab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f31380cd410 0x7f31380cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.128+0000 7f313f7ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313815d8b0 con 0x7f313815ab40 2023-12-01T01:49:21.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.128+0000 7f313f7ab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313815e4c0 con 0x7f31380cd410 2023-12-01T01:49:21.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.128+0000 7f313f7ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313815af20 con 0x7f31380ccf90 2023-12-01T01:49:21.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.129+0000 7f313dd48640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f31380cd410 0x7f31380cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:21.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.129+0000 7f313dd48640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f31380cd410 0x7f31380cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52954/0 (socket says 172.21.15.115:52954) 2023-12-01T01:49:21.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.129+0000 7f313dd48640 1 -- 172.21.15.115:0/887284016 learned_addr learned my addr 172.21.15.115:0/887284016 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:21.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.129+0000 7f313dd48640 1 -- 172.21.15.115:0/887284016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31380ccf90 msgr2=0x7f31380cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:21.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.129+0000 7f313dd48640 1 --2- 172.21.15.115:0/887284016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31380ccf90 0x7f31380cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.129+0000 7f313dd48640 1 -- 172.21.15.115:0/887284016 >> v1:172.21.15.142:6789/0 conn(0x7f313815ab40 legacy=0x7f31380ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:21.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.129+0000 7f313dd48640 1 -- 172.21.15.115:0/887284016 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313815db90 con 0x7f31380cd410 2023-12-01T01:49:21.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.130+0000 7f313dd48640 1 --2- 172.21.15.115:0/887284016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31380cd410 0x7f31380cf560 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3130009e80 tx=0x7f313002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a336d1a73b23c3e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:21.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f312ffff640 1 -- 172.21.15.115:0/887284016 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313002c9c0 con 0x7f31380cd410 2023-12-01T01:49:21.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f312ffff640 1 -- 172.21.15.115:0/887284016 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313002cb20 con 0x7f31380cd410 2023-12-01T01:49:21.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f312ffff640 1 -- 172.21.15.115:0/887284016 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31300343f0 con 0x7f31380cd410 2023-12-01T01:49:21.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f313f7ab640 1 -- 172.21.15.115:0/887284016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31380cd410 msgr2=0x7f31380cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:21.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f313f7ab640 1 --2- 172.21.15.115:0/887284016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31380cd410 0x7f31380cf560 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3130009e80 tx=0x7f313002be20 comp rx=0 tx=0).stop 2023-12-01T01:49:21.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f313f7ab640 1 -- 172.21.15.115:0/887284016 shutdown_connections 2023-12-01T01:49:21.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f313f7ab640 1 --2- 172.21.15.115:0/887284016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31380cd410 0x7f31380cf560 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f313f7ab640 1 --2- 172.21.15.115:0/887284016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31380ccf90 0x7f31380cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f313f7ab640 1 -- 172.21.15.115:0/887284016 >> 172.21.15.115:0/887284016 conn(0x7f313806bd00 msgr2=0x7f313806c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:21.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f313f7ab640 1 -- 172.21.15.115:0/887284016 shutdown_connections 2023-12-01T01:49:21.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.131+0000 7f313f7ab640 1 -- 172.21.15.115:0/887284016 wait complete. 2023-12-01T01:49:21.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.133+0000 7f313f7ab640 1 Processor -- start 2023-12-01T01:49:21.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.133+0000 7f313f7ab640 1 -- start start 2023-12-01T01:49:21.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.134+0000 7f313f7ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31380ccf90 0x7f313814fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.134+0000 7f313f7ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31380cd410 0x7f31381585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.134+0000 7f313f7ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f313815ab40 0x7f313815f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.134+0000 7f313f7ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3138157f40 con 0x7f313815ab40 2023-12-01T01:49:21.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.134+0000 7f313f7ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3138157330 con 0x7f31380cd410 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.134+0000 7f313f7ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3138157610 con 0x7f31380ccf90 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.134+0000 7f313dd48640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f313815ab40 0x7f313815f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.134+0000 7f313dd48640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f313815ab40 0x7f313815f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52968/0 (socket says 172.21.15.115:52968) 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.134+0000 7f313dd48640 1 -- 172.21.15.115:0/594793646 learned_addr learned my addr 172.21.15.115:0/594793646 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f313d547640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31380ccf90 0x7f313814fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f313cd46640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31380cd410 0x7f31381585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f313dd48640 1 -- 172.21.15.115:0/594793646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31380ccf90 msgr2=0x7f313814fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f313dd48640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31380ccf90 0x7f313814fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f313dd48640 1 -- 172.21.15.115:0/594793646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31380cd410 msgr2=0x7f31381585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f313dd48640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31380cd410 0x7f31381585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f313dd48640 1 -- 172.21.15.115:0/594793646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3130009b30 con 0x7f313815ab40 2023-12-01T01:49:21.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f313dd48640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f313815ab40 0x7f313815f7f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3130009fa0 tx=0x7f3130004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:21.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f312dffb640 1 -- 172.21.15.115:0/594793646 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130009510 con 0x7f313815ab40 2023-12-01T01:49:21.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f312dffb640 1 -- 172.21.15.115:0/594793646 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313003e450 con 0x7f313815ab40 2023-12-01T01:49:21.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f313f7ab640 1 -- 172.21.15.115:0/594793646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31381578f0 con 0x7f313815ab40 2023-12-01T01:49:21.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.135+0000 7f312dffb640 1 -- 172.21.15.115:0/594793646 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313003e7e0 con 0x7f313815ab40 2023-12-01T01:49:21.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.136+0000 7f313f7ab640 1 -- 172.21.15.115:0/594793646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3138150090 con 0x7f313815ab40 2023-12-01T01:49:21.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.137+0000 7f313f7ab640 1 -- 172.21.15.115:0/594793646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31381e5ab0 con 0x7f313815ab40 2023-12-01T01:49:21.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.137+0000 7f312dffb640 1 -- 172.21.15.115:0/594793646 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f313003e940 con 0x7f313815ab40 2023-12-01T01:49:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.138+0000 7f312dffb640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f311c06a5a0 0x7f311c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.138+0000 7f312dffb640 1 -- 172.21.15.115:0/594793646 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31300ab9f0 con 0x7f313815ab40 2023-12-01T01:49:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.138+0000 7f313d547640 1 -- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f311c06a5a0 msgr2=0x7f311c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:49:21.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.139+0000 7f313d547640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f311c06a5a0 0x7f311c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:49:21.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.142+0000 7f312dffb640 1 -- 172.21.15.115:0/594793646 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f31381e5ab0 con 0x7f313815ab40 2023-12-01T01:49:21.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.331+0000 7f313f7ab640 1 -- 172.21.15.115:0/594793646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f31381e5ab0 con 0x7f313815ab40 2023-12-01T01:49:21.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.338+0000 7f312dffb640 1 -- 172.21.15.115:0/594793646 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f31381e5ab0 con 0x7f313815ab40 2023-12-01T01:49:21.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313d547640 1 -- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f311c06a5a0 msgr2=0x7f311c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:49:21.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313d547640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f311c06a5a0 0x7f311c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:49:21.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313f7ab640 1 -- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f311c06a5a0 msgr2=0x7f311c06ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:49:21.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313f7ab640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f311c06a5a0 0x7f311c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313f7ab640 1 -- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f313815ab40 msgr2=0x7f313815f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:21.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313f7ab640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f313815ab40 0x7f313815f7f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3130009fa0 tx=0x7f3130004a40 comp rx=0 tx=0).stop 2023-12-01T01:49:21.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313f7ab640 1 -- 172.21.15.115:0/594793646 shutdown_connections 2023-12-01T01:49:21.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313f7ab640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f311c06a5a0 0x7f311c06ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313f7ab640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f313815ab40 0x7f313815f7f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313f7ab640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31380cd410 0x7f31381585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313f7ab640 1 --2- 172.21.15.115:0/594793646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31380ccf90 0x7f313814fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.340+0000 7f313f7ab640 1 -- 172.21.15.115:0/594793646 >> 172.21.15.115:0/594793646 conn(0x7f313806bd00 msgr2=0x7f313806c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:21.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.341+0000 7f313f7ab640 1 -- 172.21.15.115:0/594793646 shutdown_connections 2023-12-01T01:49:21.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.341+0000 7f313f7ab640 1 -- 172.21.15.115:0/594793646 wait complete. 2023-12-01T01:49:21.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:21.361 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-01T01:49:21.413 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-01T01:49:21.470 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:21.469+0000 7f93ebe46040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:49:21.524 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:21.522+0000 7f93ebe46040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:49:21.533 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:21.532+0000 7fbd10246040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:49:21.574 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:21.572+0000 7f93ebe46040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:49:21.588 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:21.586+0000 7fbd10246040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:49:21.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.620+0000 7f64f8684640 1 Processor -- start 2023-12-01T01:49:21.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.620+0000 7f64f8684640 1 -- start start 2023-12-01T01:49:21.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.621+0000 7f64f8684640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f64f00d0260 0x7f64f00b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.621+0000 7f64f8684640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f00b7760 con 0x7f64f00b18f0 2023-12-01T01:49:21.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.621+0000 7f64f8684640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f00b8b70 con 0x7f64f015cf70 2023-12-01T01:49:21.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.621+0000 7f64f8684640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f00b2170 con 0x7f64f00d0260 2023-12-01T01:49:21.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.621+0000 7f64f5c1f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f64f00d0260 0x7f64f00b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:21.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.621+0000 7f64f6c21640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f64f00b18f0 0x7f64f00d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34774/0 (socket says 172.21.15.115:34774) 2023-12-01T01:49:21.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.621+0000 7f64f6c21640 1 -- 172.21.15.115:0/1203900787 learned_addr learned my addr 172.21.15.115:0/1203900787 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:21.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.621+0000 7f64f5c1f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f64f00d0260 0x7f64f00b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52974/0 (socket says 172.21.15.115:52974) 2023-12-01T01:49:21.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.622+0000 7f64f5c1f640 1 -- 172.21.15.115:0/1203900787 >> v1:172.21.15.142:6789/0 conn(0x7f64f015cf70 legacy=0x7f64f00cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:21.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.622+0000 7f64f5c1f640 1 -- 172.21.15.115:0/1203900787 >> v1:172.21.15.115:6789/0 conn(0x7f64f00b18f0 legacy=0x7f64f00d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:21.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.622+0000 7f64f5c1f640 1 -- 172.21.15.115:0/1203900787 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f00b6e30 con 0x7f64f00d0260 2023-12-01T01:49:21.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.622+0000 7f64f541e640 1 -- 172.21.15.115:0/1203900787 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936458620 0 0) 0x7f64f00b7760 con 0x7f64f00b18f0 2023-12-01T01:49:21.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.623+0000 7f64f5c1f640 1 --2- 172.21.15.115:0/1203900787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 0x7f64f00b8470 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f64e00097a0 tx=0x7f64e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c343ec203392150 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:21.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.624+0000 7f64f541e640 1 -- 172.21.15.115:0/1203900787 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0031830 con 0x7f64f00d0260 2023-12-01T01:49:21.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.624+0000 7f64f541e640 1 -- 172.21.15.115:0/1203900787 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e0031990 con 0x7f64f00d0260 2023-12-01T01:49:21.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.624+0000 7f64f541e640 1 -- 172.21.15.115:0/1203900787 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0031d20 con 0x7f64f00d0260 2023-12-01T01:49:21.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.624+0000 7f64f8684640 1 -- 172.21.15.115:0/1203900787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 msgr2=0x7f64f00b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:21.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.624+0000 7f64f8684640 1 --2- 172.21.15.115:0/1203900787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 0x7f64f00b8470 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f64e00097a0 tx=0x7f64e002b8d0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.625+0000 7f64f8684640 1 -- 172.21.15.115:0/1203900787 shutdown_connections 2023-12-01T01:49:21.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.625+0000 7f64f8684640 1 --2- 172.21.15.115:0/1203900787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 0x7f64f00b8470 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.625+0000 7f64f8684640 1 -- 172.21.15.115:0/1203900787 >> 172.21.15.115:0/1203900787 conn(0x7f64f006a0e0 msgr2=0x7f64f006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:21.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.625+0000 7f64f8684640 1 -- 172.21.15.115:0/1203900787 shutdown_connections 2023-12-01T01:49:21.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.625+0000 7f64f8684640 1 -- 172.21.15.115:0/1203900787 wait complete. 2023-12-01T01:49:21.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.626+0000 7f64f8684640 1 Processor -- start 2023-12-01T01:49:21.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.626+0000 7f64f8684640 1 -- start start 2023-12-01T01:49:21.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.627+0000 7f64f8684640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f00b18f0 0x7f64f0156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.627+0000 7f64f8684640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 0x7f64f00cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.627+0000 7f64f8684640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f015cf70 0x7f64f00cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.627+0000 7f64f8684640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0160150 con 0x7f64f00d0260 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.627+0000 7f64f8684640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0156c50 con 0x7f64f00b18f0 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.627+0000 7f64f8684640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f015df00 con 0x7f64f015cf70 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f5c1f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 0x7f64f00cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f5c1f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 0x7f64f00cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52986/0 (socket says 172.21.15.115:52986) 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f5c1f640 1 -- 172.21.15.115:0/2783962913 learned_addr learned my addr 172.21.15.115:0/2783962913 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f6c21640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f015cf70 0x7f64f00cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f6420640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f00b18f0 0x7f64f0156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f5c1f640 1 -- 172.21.15.115:0/2783962913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f015cf70 msgr2=0x7f64f00cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f5c1f640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f015cf70 0x7f64f00cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f5c1f640 1 -- 172.21.15.115:0/2783962913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f00b18f0 msgr2=0x7f64f0156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f5c1f640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f00b18f0 0x7f64f0156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:21.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f5c1f640 1 -- 172.21.15.115:0/2783962913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e0009400 con 0x7f64f00d0260 2023-12-01T01:49:21.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f6c21640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f015cf70 0x7f64f00cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:21.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.628+0000 7f64f5c1f640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 0x7f64f00cbaa0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f64e002bee0 tx=0x7f64e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:21.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.629+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e003ba70 con 0x7f64f00d0260 2023-12-01T01:49:21.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.629+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e003bbd0 con 0x7f64f00d0260 2023-12-01T01:49:21.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.629+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e00453f0 con 0x7f64f00d0260 2023-12-01T01:49:21.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.629+0000 7f64f8684640 1 -- 172.21.15.115:0/2783962913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f015e1e0 con 0x7f64f00d0260 2023-12-01T01:49:21.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.629+0000 7f64f8684640 1 -- 172.21.15.115:0/2783962913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f015e7a0 con 0x7f64f00d0260 2023-12-01T01:49:21.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.630+0000 7f64f8684640 1 -- 172.21.15.115:0/2783962913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f015e990 con 0x7f64f00d0260 2023-12-01T01:49:21.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.631+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 20) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f64e0045550 con 0x7f64f00d0260 2023-12-01T01:49:21.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.632+0000 7f64e6ffd640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:21.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.632+0000 7f64f6420640 1 -- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 msgr2=0x7f64d406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:49:21.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.632+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64e00aaf70 con 0x7f64f00d0260 2023-12-01T01:49:21.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.632+0000 7f64f6420640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:49:21.634 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:21.632+0000 7f93ebe46040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:49:21.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.637+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f64e00ddc80 con 0x7f64f00d0260 2023-12-01T01:49:21.643 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:21.642+0000 7fbd10246040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:49:21.683 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:21.681+0000 7f93ebe46040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:49:21.707 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:21.706+0000 7fbd10246040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:49:21.758 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:21.757+0000 7fbd10246040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:49:21.791 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:21.790+0000 7f93ebe46040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:49:21.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.833+0000 7f64f6420640 1 -- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 msgr2=0x7f64d406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:49:21.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.833+0000 7f64f6420640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:49:21.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:21.849+0000 7f64f8684640 1 -- 172.21.15.115:0/2783962913 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f64f00b73f0 con 0x7f64d406a5a0 2023-12-01T01:49:21.853 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:21.851+0000 7f93ebe46040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:49:21.873 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:21.871+0000 7fbd10246040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:49:21.902 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:21.900+0000 7f93ebe46040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:49:21.935 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:21.934+0000 7fbd10246040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:49:21.959 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:21.957+0000 7f93ebe46040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:49:21.985 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:21.984+0000 7fbd10246040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:49:22.009 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:22.008+0000 7f93ebe46040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:49:22.042 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:22.041+0000 7fbd10246040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:49:22.092 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:22.091+0000 7fbd10246040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:49:22.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:22.234+0000 7f64f6420640 1 -- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 msgr2=0x7f64d406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:49:22.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:22.234+0000 7f64f6420640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:49:22.386 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:22.385+0000 7f93ebe46040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:49:22.474 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:22.473+0000 7fbd10246040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:49:22.545 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:22.543+0000 7f93ebe46040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:49:22.634 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:22.633+0000 7fbd10246040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:49:22.652 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:22.650+0000 7f93ebe46040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:49:22.702 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:22.700+0000 7f93ebe46040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:49:22.741 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:22.739+0000 7fbd10246040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:49:22.791 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:22.790+0000 7fbd10246040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:49:22.808 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:22.807+0000 7f93ebe46040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:49:22.899 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:22.898+0000 7fbd10246040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:49:22.915 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:22.913+0000 7f93ebe46040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:49:23.005 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:23.003+0000 7fbd10246040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:49:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:23.035+0000 7f64f6420640 1 -- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 msgr2=0x7f64d406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:49:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:23.035+0000 7f64f6420640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:49:23.070 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:23.068+0000 7f93ebe46040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:49:23.119 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:23.117+0000 7f93ebe46040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:49:23.165 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:23.163+0000 7fbd10246040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:49:23.214 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:23.212+0000 7fbd10246040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:49:23.237 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:23.236+0000 7f93ebe46040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:49:23.333 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:23.331+0000 7fbd10246040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:49:24.058 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:24.056+0000 7f93ebe46040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:49:24.164 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.163+0000 7fbd10246040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:49:24.351 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:24.350+0000 7f93ebe46040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:49:24.408 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:24.406+0000 7f93ebe46040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:49:24.455 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.453+0000 7fbd10246040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:49:24.512 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.510+0000 7fbd10246040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:49:24.522 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:49:24.520+0000 7f93ebe46040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:49:24.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:24.592+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 21) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f64e00a9490 con 0x7f64f00d0260 2023-12-01T01:49:24.634 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.633+0000 7fbd10246040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:49:24.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:24.637+0000 7f64f6420640 1 -- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 msgr2=0x7f64d406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:49:24.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:24.637+0000 7f64f6420640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-01T01:49:24.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:24.649+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 22) v1 ==== 88335+0+0 (secure 0 0 0) 0x7f64e007dca0 con 0x7f64f00d0260 2023-12-01T01:49:24.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:24.650+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 msgr2=0x7f64d406ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:49:24.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:24.650+0000 7f64e6ffd640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:24.831 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.830+0000 7fbc8c5d1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:49:24.831 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.830+0000 7fbc8c5d1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:49:24.832 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.830+0000 7fbc8fdd8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:49:24.832 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.830+0000 7fbc8c5d1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:49:24.832 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.830+0000 7fbc8c5d1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:49:24.832 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.830+0000 7fbc8fdd8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:49:24.832 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.830+0000 7fbc8fdd8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:49:24.832 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.830+0000 7fbc8fdd8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:49:24.832 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.830+0000 7fbc8fdd8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:49:24.832 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:49:24.830+0000 7fbc8c5d1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:49:25.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.665+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 <== mon.0 v2:172.21.15.115:3300/0 9 ==== mgrmap(e 23) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f64e0077190 con 0x7f64f00d0260 2023-12-01T01:49:25.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.667+0000 7f64e6ffd640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:25.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.667+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f64d4036460 con 0x7f64d406a5a0 2023-12-01T01:49:25.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.674+0000 7f64f6420640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:25.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.684+0000 7f64f6420640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406c9e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006a50 tx=0x7f64ec0091f0 comp rx=0 tx=0).ready entity=mgr.6105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:25.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.847+0000 7f64e6ffd640 1 -- 172.21.15.115:0/2783962913 <== mgr.6105 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f64ec011070 con 0x7f64d406a5a0 2023-12-01T01:49:25.849 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path snap_test_dir1 2023-12-01T01:49:25.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.850+0000 7f64f8684640 1 -- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 msgr2=0x7f64d406c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:25.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.850+0000 7f64f8684640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406c9e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006a50 tx=0x7f64ec0091f0 comp rx=0 tx=0).stop 2023-12-01T01:49:25.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.850+0000 7f64f8684640 1 -- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 msgr2=0x7f64f00cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:25.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.850+0000 7f64f8684640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 0x7f64f00cbaa0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f64e002bee0 tx=0x7f64e00046c0 comp rx=0 tx=0).stop 2023-12-01T01:49:25.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.851+0000 7f64f8684640 1 -- 172.21.15.115:0/2783962913 shutdown_connections 2023-12-01T01:49:25.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.851+0000 7f64f8684640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f64d406a5a0 0x7f64d406c9e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:25.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.851+0000 7f64f8684640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f015cf70 0x7f64f00cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:25.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.851+0000 7f64f8684640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f00d0260 0x7f64f00cbaa0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:25.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.851+0000 7f64f8684640 1 --2- 172.21.15.115:0/2783962913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f00b18f0 0x7f64f0156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:25.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.851+0000 7f64f8684640 1 -- 172.21.15.115:0/2783962913 >> 172.21.15.115:0/2783962913 conn(0x7f64f006a0e0 msgr2=0x7f64f015a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:25.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.851+0000 7f64f8684640 1 -- 172.21.15.115:0/2783962913 shutdown_connections 2023-12-01T01:49:25.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:25.851+0000 7f64f8684640 1 -- 172.21.15.115:0/2783962913 wait complete. 2023-12-01T01:49:25.875 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-01T01:49:25.875 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 2M 2023-12-01T01:49:26.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.083+0000 7f9fe1aa0640 1 Processor -- start 2023-12-01T01:49:26.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.084+0000 7f9fe1aa0640 1 -- start start 2023-12-01T01:49:26.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.085+0000 7f9fe1aa0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9fdc15cd90 0x7f9fdc0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.085+0000 7f9fe1aa0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0ce5e0 con 0x7f9fdc0b5fa0 2023-12-01T01:49:26.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.085+0000 7f9fe1aa0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0ce300 con 0x7f9fdc0cfd60 2023-12-01T01:49:26.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.085+0000 7f9fe1aa0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc0d0ec0 con 0x7f9fdc15cd90 2023-12-01T01:49:26.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.086+0000 7f9fd2ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9fdc0b5fa0 0x7f9fdc0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34788/0 (socket says 172.21.15.115:34788) 2023-12-01T01:49:26.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.086+0000 7f9fd2ffd640 1 -- 172.21.15.115:0/927933991 learned_addr learned my addr 172.21.15.115:0/927933991 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:26.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.086+0000 7f9fdb7fe640 1 --2- 172.21.15.115:0/927933991 >> v2:172.21.15.142:3300/0 conn(0x7f9fdc15cd90 0x7f9fdc0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.086+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827301400 0 0) 0x7f9fc4001110 con 0x7f9fdc0b5fa0 2023-12-01T01:49:26.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.086+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc8003520 con 0x7f9fdc0b5fa0 2023-12-01T01:49:26.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2665583502 0 0) 0x7f9fc8003520 con 0x7f9fdc0b5fa0 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 >> v2:172.21.15.142:3300/0 conn(0x7f9fdc15cd90 msgr2=0x7f9fdc0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 --2- 172.21.15.115:0/927933991 >> v2:172.21.15.142:3300/0 conn(0x7f9fdc15cd90 0x7f9fdc0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 >> v1:172.21.15.142:6789/0 conn(0x7f9fdc0cfd60 legacy=0x7f9fdc0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0d10a0 con 0x7f9fdc0b5fa0 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f9fc4001800 con 0x7f9fdc0b5fa0 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 >> v1:172.21.15.115:6789/0 conn(0x7f9fdc0b5fa0 legacy=0x7f9fdc0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fc8003970 0x7f9fc8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc80047c0 0x7f9fc8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fc800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0d0ec0 con 0x7f9fc8005180 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc800c020 con 0x7f9fc80047c0 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc800c1e0 con 0x7f9fc8003970 2023-12-01T01:49:26.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdbfff640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fc800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fd2ffd640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc80047c0 0x7f9fc8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdbfff640 1 -- 172.21.15.115:0/927933991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fc8003970 msgr2=0x7f9fc8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.087+0000 7f9fdb7fe640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fc8003970 0x7f9fc8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.088+0000 7f9fdbfff640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fc8003970 0x7f9fc8003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.088+0000 7f9fdbfff640 1 -- 172.21.15.115:0/927933991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc80047c0 msgr2=0x7f9fc8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.088+0000 7f9fdbfff640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc80047c0 0x7f9fc8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.088+0000 7f9fdbfff640 1 -- 172.21.15.115:0/927933991 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc8003520 con 0x7f9fc8005180 2023-12-01T01:49:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.088+0000 7f9fd2ffd640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fc80047c0 0x7f9fc8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:49:26.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.089+0000 7f9fdbfff640 1 -- 172.21.15.115:0/927933991 reap_dead start 2023-12-01T01:49:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.089+0000 7f9fdbfff640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fc800b980 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4001750 tx=0x7f9fc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261771fc821bd2dc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.089+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40052d0 con 0x7f9fc8005180 2023-12-01T01:49:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.089+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd402e050 con 0x7f9fc8005180 2023-12-01T01:49:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.089+0000 7f9fdaffd640 1 -- 172.21.15.115:0/927933991 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4003830 con 0x7f9fc8005180 2023-12-01T01:49:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.089+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/927933991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 msgr2=0x7f9fc800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.089+0000 7f9fe1aa0640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fc800b980 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4001750 tx=0x7f9fc80057f0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.090+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/927933991 shutdown_connections 2023-12-01T01:49:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.090+0000 7f9fe1aa0640 1 --2- 172.21.15.115:0/927933991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fc800b980 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.090+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/927933991 >> 172.21.15.115:0/927933991 conn(0x7f9fdc065960 msgr2=0x7f9fdc065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.090+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/927933991 shutdown_connections 2023-12-01T01:49:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.090+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/927933991 wait complete. 2023-12-01T01:49:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.091+0000 7f9fe1aa0640 1 Processor -- start 2023-12-01T01:49:26.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.092+0000 7f9fe1aa0640 1 -- start start 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.092+0000 7f9fe1aa0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fdc158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fe1aa0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fdc153280 0x7f9fdc142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fe1aa0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fdc144160 0x7f9fdc1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fe1aa0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0cb6f0 con 0x7f9fc8005180 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fe1aa0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc15c270 con 0x7f9fdc144160 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fe1aa0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc153700 con 0x7f9fdc153280 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fdb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fdc158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fdb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fdc158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53110/0 (socket says 172.21.15.115:53110) 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fdb7fe640 1 -- 172.21.15.115:0/2342933782 learned_addr learned my addr 172.21.15.115:0/2342933782 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fd2ffd640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fdc153280 0x7f9fdc142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fdb7fe640 1 -- 172.21.15.115:0/2342933782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fdc153280 msgr2=0x7f9fdc142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fdb7fe640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fdc153280 0x7f9fdc142100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fdb7fe640 1 -- 172.21.15.115:0/2342933782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fdc144160 msgr2=0x7f9fdc1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fdb7fe640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fdc144160 0x7f9fdc1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.093+0000 7f9fdb7fe640 1 -- 172.21.15.115:0/2342933782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc144b20 con 0x7f9fc8005180 2023-12-01T01:49:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.094+0000 7f9fdb7fe640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fdc158ad0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00f3b0 tx=0x7f9fcc00f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:26.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.094+0000 7f9fd8ff9640 1 -- 172.21.15.115:0/2342933782 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc017600 con 0x7f9fc8005180 2023-12-01T01:49:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.094+0000 7f9fd8ff9640 1 -- 172.21.15.115:0/2342933782 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc011040 con 0x7f9fc8005180 2023-12-01T01:49:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.094+0000 7f9fd8ff9640 1 -- 172.21.15.115:0/2342933782 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0179c0 con 0x7f9fc8005180 2023-12-01T01:49:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.094+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/2342933782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc0cade0 con 0x7f9fc8005180 2023-12-01T01:49:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.094+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/2342933782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc1590b0 con 0x7f9fc8005180 2023-12-01T01:49:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.095+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/2342933782 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc0cb960 con 0x7f9fc8005180 2023-12-01T01:49:26.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.096+0000 7f9fd8ff9640 1 -- 172.21.15.115:0/2342933782 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 23) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f9fcc017b20 con 0x7f9fc8005180 2023-12-01T01:49:26.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.098+0000 7f9fd8ff9640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9fbc06a2d0 0x7f9fbc06c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.098+0000 7f9fd8ff9640 1 -- 172.21.15.115:0/2342933782 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f9fcc088c60 con 0x7f9fc8005180 2023-12-01T01:49:26.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.098+0000 7f9fd2ffd640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9fbc06a2d0 0x7f9fbc06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.098+0000 7f9fd2ffd640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9fbc06a2d0 0x7f9fbc06c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4000f60 tx=0x7f9fc4001320 comp rx=0 tx=0).ready entity=mgr.6105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:26.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.102+0000 7f9fd8ff9640 1 -- 172.21.15.115:0/2342933782 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9fcc0544f0 con 0x7f9fc8005180 2023-12-01T01:49:26.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.314+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/2342933782 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fdc0d1660 con 0x7f9fbc06a2d0 2023-12-01T01:49:26.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.448+0000 7f9fd8ff9640 1 -- 172.21.15.115:0/2342933782 <== mgr.6105 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9fc400d040 con 0x7f9fbc06a2d0 2023-12-01T01:49:26.450 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path snap_test_dir1 2023-12-01T01:49:26.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/2342933782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9fbc06a2d0 msgr2=0x7f9fbc06c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9fbc06a2d0 0x7f9fbc06c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4000f60 tx=0x7f9fc4001320 comp rx=0 tx=0).stop 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/2342933782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 msgr2=0x7f9fdc158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fdc158ad0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00f3b0 tx=0x7f9fcc00f870 comp rx=0 tx=0).stop 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/2342933782 shutdown_connections 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9fbc06a2d0 0x7f9fbc06c790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fdc153280 0x7f9fdc142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fdc144160 0x7f9fdc1445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 --2- 172.21.15.115:0/2342933782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fc8005180 0x7f9fdc158ad0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/2342933782 >> 172.21.15.115:0/2342933782 conn(0x7f9fdc065960 msgr2=0x7f9fdc15fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/2342933782 shutdown_connections 2023-12-01T01:49:26.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.452+0000 7f9fe1aa0640 1 -- 172.21.15.115:0/2342933782 wait complete. 2023-12-01T01:49:26.476 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-01T01:49:26.476 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-01-01_50_00 in ~34s... 2023-12-01T01:49:26.476 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-01-01_51_00 in ~94s... 2023-12-01T01:49:26.476 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-12-01T01:49:26.476 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-01T01:49:26.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.691+0000 7f10026d8640 1 Processor -- start 2023-12-01T01:49:26.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.692+0000 7f10026d8640 1 -- start start 2023-12-01T01:49:26.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.693+0000 7f10026d8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0cdb30 0x7f0ffc0ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.693+0000 7f10026d8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ffc0cb290 con 0x7f0ffc06af70 2023-12-01T01:49:26.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.693+0000 7f10026d8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ffc0cb570 con 0x7f0ffc15d090 2023-12-01T01:49:26.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.693+0000 7f10026d8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc0cb850 con 0x7f0ffc0cdb30 2023-12-01T01:49:26.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.693+0000 7f1000c75640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0ffc06af70 0x7f0ffc0cbd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34794/0 (socket says 172.21.15.115:34794) 2023-12-01T01:49:26.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.693+0000 7f1000c75640 1 -- 172.21.15.115:0/3715287273 learned_addr learned my addr 172.21.15.115:0/3715287273 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:49:26.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffb7fe640 1 --2- 172.21.15.115:0/3715287273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0cdb30 0x7f0ffc0ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674707907 0 0) 0x7f0ffc0cb290 con 0x7f0ffc06af70 2023-12-01T01:49:26.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fe4003520 con 0x7f0ffc06af70 2023-12-01T01:49:26.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 321869994 0 0) 0x7f0fe4003520 con 0x7f0ffc06af70 2023-12-01T01:49:26.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0cdb30 msgr2=0x7f0ffc0ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:26.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffaffd640 1 --2- 172.21.15.115:0/3715287273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0cdb30 0x7f0ffc0ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 >> v1:172.21.15.142:6789/0 conn(0x7f0ffc15d090 legacy=0x7f0ffc0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc0cfec0 con 0x7f0ffc06af70 2023-12-01T01:49:26.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0ff40019f0 con 0x7f0ffc06af70 2023-12-01T01:49:26.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 >> v1:172.21.15.115:6789/0 conn(0x7f0ffc06af70 legacy=0x7f0ffc0cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.694+0000 7f0ffaffd640 1 --2- 172.21.15.115:0/3715287273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fe40039a0 0x7f0fe4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffaffd640 1 --2- 172.21.15.115:0/3715287273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 0x7f0fe4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffaffd640 1 --2- 172.21.15.115:0/3715287273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0cdb30 0x7f0fe4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc0cb850 con 0x7f0fe40047c0 2023-12-01T01:49:26.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe400ba30 con 0x7f0fe40039a0 2023-12-01T01:49:26.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe400bbf0 con 0x7f0ffc0cdb30 2023-12-01T01:49:26.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffb7fe640 1 --2- 172.21.15.115:0/3715287273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 0x7f0fe4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffb7fe640 1 -- 172.21.15.115:0/3715287273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0cdb30 msgr2=0x7f0fe4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:26.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffb7fe640 1 --2- 172.21.15.115:0/3715287273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0cdb30 0x7f0fe4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffb7fe640 1 -- 172.21.15.115:0/3715287273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fe40039a0 msgr2=0x7f0fe4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:49:26.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffb7fe640 1 --2- 172.21.15.115:0/3715287273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fe40039a0 0x7f0fe4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.695+0000 7f0ffb7fe640 1 -- 172.21.15.115:0/3715287273 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe4003520 con 0x7f0fe40047c0 2023-12-01T01:49:26.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.696+0000 7f0ffb7fe640 1 --2- 172.21.15.115:0/3715287273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 0x7f0fe4004c40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009a90 tx=0x7f0fe40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8575536b9dd354d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.697+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff002c6b0 con 0x7f0fe40047c0 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.697+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff002c810 con 0x7f0fe40047c0 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.697+0000 7f0ffaffd640 1 -- 172.21.15.115:0/3715287273 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff002cba0 con 0x7f0fe40047c0 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.697+0000 7f10026d8640 1 -- 172.21.15.115:0/3715287273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 msgr2=0x7f0fe4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.697+0000 7f10026d8640 1 --2- 172.21.15.115:0/3715287273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 0x7f0fe4004c40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009a90 tx=0x7f0fe40057f0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.697+0000 7f1000c75640 1 -- 172.21.15.115:0/3715287273 reap_dead start 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.698+0000 7f10026d8640 1 -- 172.21.15.115:0/3715287273 shutdown_connections 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.698+0000 7f10026d8640 1 -- 172.21.15.115:0/3715287273 >> 172.21.15.115:0/3715287273 conn(0x7f0ffc0d0ba0 msgr2=0x7f0ffc0d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.698+0000 7f10026d8640 1 -- 172.21.15.115:0/3715287273 shutdown_connections 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.698+0000 7f10026d8640 1 -- 172.21.15.115:0/3715287273 wait complete. 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.700+0000 7f10026d8640 1 Processor -- start 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.700+0000 7f10026d8640 1 -- start start 2023-12-01T01:49:26.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.701+0000 7f10026d8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 0x7f0ffc14c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.701+0000 7f10026d8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0ca580 0x7f0ffc1f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.701+0000 7f10026d8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc1f5ea0 0x7f0ffc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.701+0000 7f10026d8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc14c800 con 0x7f0fe40047c0 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.701+0000 7f10026d8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc14b870 con 0x7f0ffc1f5ea0 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.701+0000 7f10026d8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc1f6420 con 0x7f0ffc0ca580 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.701+0000 7f0ffbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 0x7f0ffc14c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.701+0000 7f0ffbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 0x7f0ffc14c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:53140/0 (socket says 172.21.15.115:53140) 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.701+0000 7f0ffbfff640 1 -- 172.21.15.115:0/1445329771 learned_addr learned my addr 172.21.15.115:0/1445329771 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f1000c75640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc1f5ea0 0x7f0ffc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f0ffb7fe640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0ca580 0x7f0ffc1f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f0ffbfff640 1 -- 172.21.15.115:0/1445329771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0ca580 msgr2=0x7f0ffc1f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f0ffbfff640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0ca580 0x7f0ffc1f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f0ffbfff640 1 -- 172.21.15.115:0/1445329771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc1f5ea0 msgr2=0x7f0ffc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f0ffbfff640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc1f5ea0 0x7f0ffc0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f0ffbfff640 1 -- 172.21.15.115:0/1445329771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc1f6700 con 0x7f0fe40047c0 2023-12-01T01:49:26.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f0ffbfff640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 0x7f0ffc14c2c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c480 tx=0x7f0fec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:26.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f0ff8ff9640 1 -- 172.21.15.115:0/1445329771 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0052d0 con 0x7f0fe40047c0 2023-12-01T01:49:26.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f0ff8ff9640 1 -- 172.21.15.115:0/1445329771 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec005470 con 0x7f0fe40047c0 2023-12-01T01:49:26.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f0ff8ff9640 1 -- 172.21.15.115:0/1445329771 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec007260 con 0x7f0fe40047c0 2023-12-01T01:49:26.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.702+0000 7f10026d8640 1 -- 172.21.15.115:0/1445329771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ffc14bac0 con 0x7f0fe40047c0 2023-12-01T01:49:26.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.703+0000 7f10026d8640 1 -- 172.21.15.115:0/1445329771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ffc0caa00 con 0x7f0fe40047c0 2023-12-01T01:49:26.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.704+0000 7f10026d8640 1 -- 172.21.15.115:0/1445329771 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ffc067850 con 0x7f0fe40047c0 2023-12-01T01:49:26.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.705+0000 7f0ff8ff9640 1 -- 172.21.15.115:0/1445329771 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 24) v1 ==== 88727+0+0 (secure 0 0 0) 0x7f0fec0073c0 con 0x7f0fe40047c0 2023-12-01T01:49:26.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.706+0000 7f0ff8ff9640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fe806a5a0 0x7f0fe806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:49:26.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.706+0000 7f0ff8ff9640 1 -- 172.21.15.115:0/1445329771 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f0fec0886a0 con 0x7f0fe40047c0 2023-12-01T01:49:26.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.706+0000 7f0ffb7fe640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fe806a5a0 0x7f0fe806ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:49:26.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.707+0000 7f0ffb7fe640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fe806a5a0 0x7f0fe806ca60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0036000 tx=0x7f0ff0030000 comp rx=0 tx=0).ready entity=mgr.6105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:49:26.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.710+0000 7f0ff8ff9640 1 -- 172.21.15.115:0/1445329771 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0fec0574a0 con 0x7f0fe40047c0 2023-12-01T01:49:26.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.923+0000 7f10026d8640 1 -- 172.21.15.115:0/1445329771 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ffc0d01d0 con 0x7f0fe806a5a0 2023-12-01T01:49:26.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.939+0000 7f0ff8ff9640 1 -- 172.21.15.115:0/1445329771 <== mgr.6105 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7f0ff00040c0 con 0x7f0fe806a5a0 2023-12-01T01:49:26.941 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:49:26.941 INFO:teuthology.orchestra.run.smithi115.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-12-01T01:49:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 -- 172.21.15.115:0/1445329771 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fe806a5a0 msgr2=0x7f0fe806ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fe806a5a0 0x7f0fe806ca60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0036000 tx=0x7f0ff0030000 comp rx=0 tx=0).stop 2023-12-01T01:49:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 -- 172.21.15.115:0/1445329771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 msgr2=0x7f0ffc14c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:49:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 0x7f0ffc14c2c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c480 tx=0x7f0fec00c940 comp rx=0 tx=0).stop 2023-12-01T01:49:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 -- 172.21.15.115:0/1445329771 shutdown_connections 2023-12-01T01:49:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fe806a5a0 0x7f0fe806ca60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc0ca580 0x7f0ffc1f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc1f5ea0 0x7f0ffc0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 --2- 172.21.15.115:0/1445329771 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fe40047c0 0x7f0ffc14c2c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:49:26.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 -- 172.21.15.115:0/1445329771 >> 172.21.15.115:0/1445329771 conn(0x7f0ffc0d0ba0 msgr2=0x7f0ffc0cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:49:26.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 -- 172.21.15.115:0/1445329771 shutdown_connections 2023-12-01T01:49:26.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:49:26.942+0000 7f10026d8640 1 -- 172.21.15.115:0/1445329771 wait complete. 2023-12-01T01:49:26.963 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-12-01T01:49:26.963 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-12-01T01:49:26.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:26.964 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:27.014 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:27.014 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:27.015 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:28.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:28.016 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:28.081 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:28.081 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:28.081 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:28.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:49:28.376 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:49:29.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:29.083 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:29.145 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:29.146 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:29.146 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:30.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:30.147 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:30.210 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:30.210 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:30.210 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:31.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:31.211 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:31.275 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:31.275 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:31.275 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:32.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:32.276 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:32.338 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:32.338 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:32.338 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:33.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:33.340 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:33.404 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:33.404 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:33.404 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:34.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:34.405 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:34.463 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:34.463 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:34.463 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:35.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:35.464 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:35.527 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:35.527 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:35.527 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:36.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:36.528 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:36.591 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:36.591 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:36.591 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:37.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:37.592 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:37.655 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:37.655 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:37.656 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:38.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:38.656 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:38.720 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:38.720 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:38.720 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:39.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:39.722 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:39.785 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:39.785 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:39.785 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:40.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:40.788 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:40.852 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:40.852 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:40.852 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:41.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:41.853 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:41.919 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:41.919 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:41.919 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:42.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:42.920 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:42.982 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:42.982 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:42.982 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:43.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:43.984 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:44.047 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:44.047 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:44.047 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:45.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:45.048 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:45.112 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:45.112 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:45.113 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:46.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:46.114 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:46.175 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:46.175 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:46.175 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:47.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:47.177 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:47.240 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:47.240 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:47.240 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:48.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:48.241 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:48.305 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:48.306 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:48.306 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:49.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:49.307 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:49.371 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:49.371 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:49.371 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:50.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:50.373 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:50.438 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:50.438 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:50.438 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:51.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:51.439 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:51.501 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:51.501 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:51.501 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:52.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:52.503 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:52.566 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:52.567 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:52.567 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:53.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:53.567 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:53.632 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:53.632 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:53.632 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:54.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:54.633 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:54.698 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:54.698 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:54.698 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:55.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:55.699 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:55.760 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:55.760 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:55.760 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:56.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:56.761 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:56.825 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:56.825 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:56.825 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:57.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:57.826 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:57.888 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:57.888 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:57.888 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:58.522 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:49:58.532 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:49:58.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:58.889 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:49:58.949 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:49:58.949 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:49:58.949 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:49:59.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:49:59.950 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:00.014 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:50:00.014 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:00.014 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:00.474 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:50:00.473+0000 7fbc7edb6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:50:00.475 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:50:00.473+0000 7fbc7edb6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:50:00.475 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:50:00.473+0000 7fbc7edb6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:50:00.475 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:50:00.473+0000 7fbc7edb6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:50:00.475 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:50:00.473+0000 7fbc7edb6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:50:01.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:01.015 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:01.077 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:01.079 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:01.079 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:01.079 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:01.079 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-01-01_50_00_UTC'] 2023-12-01T01:50:01.079 DEBUG:tasks.cephfs.test_snap_schedules:exec=1701395366.4765854, now = 1701395401.0795336, timo = 34 2023-12-01T01:50:02.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:02.081 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:02.144 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:02.145 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:02.145 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:02.145 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:03.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:03.146 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:03.208 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:03.209 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:03.210 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:03.210 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:04.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:04.211 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:04.272 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:04.273 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:04.273 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:04.273 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:05.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:05.274 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:05.334 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:05.335 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:05.335 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:05.335 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:06.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:06.339 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:06.401 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:06.403 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:06.403 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:06.403 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:07.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:07.404 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:07.464 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:07.465 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:07.465 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:07.465 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:08.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:08.466 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:08.529 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:08.530 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:08.530 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:08.530 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:09.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:09.531 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:09.594 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:09.595 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:09.596 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:09.596 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:10.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:10.596 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:10.658 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:10.659 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:10.659 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:10.659 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:11.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:11.661 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:11.721 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:11.723 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:11.723 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:11.723 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:12.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:12.724 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:12.786 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:12.787 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:12.787 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:12.787 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:13.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:13.789 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:13.850 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:13.852 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:13.852 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:13.852 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:14.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:14.853 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:14.916 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:14.918 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:14.918 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:14.918 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:15.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:15.920 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:15.981 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:15.983 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:15.983 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:15.983 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:16.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:16.983 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:17.042 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:17.043 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:17.043 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:17.043 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:18.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:18.044 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:18.104 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:18.106 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:18.106 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:18.106 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:19.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:19.107 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:19.168 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:19.170 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:19.170 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:19.170 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:20.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:20.172 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:20.233 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:20.234 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:20.234 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:20.234 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:21.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:21.236 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:21.297 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:21.298 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:21.298 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:21.298 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:22.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:22.299 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:22.361 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:22.363 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:22.363 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:22.363 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:23.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:23.364 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:23.426 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:23.427 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:23.428 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:23.428 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:24.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:24.429 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:24.490 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:24.491 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:24.491 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:24.491 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:25.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:25.492 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:25.552 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:25.553 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:25.553 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:25.553 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:26.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:26.554 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:26.617 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:26.618 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:26.618 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:26.618 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:27.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:27.619 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:27.681 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:27.682 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:27.682 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:27.682 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:28.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:28.683 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:28.721 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:50:28.738 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:50:28.743 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:28.744 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:28.744 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:28.744 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:29.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:29.745 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:29.808 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:29.809 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:29.809 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:29.809 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:30.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:30.810 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:30.875 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:30.875 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:30.875 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:30.875 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:31.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:31.876 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:31.937 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:31.938 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:31.939 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:31.939 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:32.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:32.940 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:33.002 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:33.003 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:33.003 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:33.003 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:34.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:34.005 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:34.066 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:34.067 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:34.067 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:34.067 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:35.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:35.068 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:35.130 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:35.131 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:35.131 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:35.131 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:36.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:36.132 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:36.195 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:36.196 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:36.196 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:36.196 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:37.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:37.197 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:37.259 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:37.261 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:37.261 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:37.261 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:38.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:38.262 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:38.324 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:38.325 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:38.325 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:38.325 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:39.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:39.326 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:39.388 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:39.390 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:39.390 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:39.390 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:40.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:40.391 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:40.455 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:40.456 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:40.456 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:40.456 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:41.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:41.457 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:41.518 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:41.520 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:41.520 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:41.520 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:42.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:42.521 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:42.582 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:42.583 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:42.583 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:42.583 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:43.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:43.585 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:43.647 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:43.649 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:43.649 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:43.649 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:44.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:44.651 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:44.712 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:44.714 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:44.714 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:44.714 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:45.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:45.715 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:45.778 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:45.779 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:45.779 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:45.779 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:46.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:46.780 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:46.841 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:46.843 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:46.843 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:46.843 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:47.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:47.844 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:47.906 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:47.908 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:47.908 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:47.908 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:48.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:48.909 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:48.971 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:48.972 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:48.972 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:48.973 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:49.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:49.973 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:50.038 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:50.039 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:50.039 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:50.039 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:51.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:51.040 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:51.102 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:51.104 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:51.104 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:51.104 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:52.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:52.105 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:52.166 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:52.167 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:52.167 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:52.167 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:53.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:53.168 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:53.229 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:53.230 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:53.231 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:53.231 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:54.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:54.231 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:54.294 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:54.295 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:54.295 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:54.295 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:55.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:55.297 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:55.359 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:55.360 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:55.360 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:55.360 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:56.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:56.362 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:56.423 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:56.424 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:56.424 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:56.424 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:57.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:57.426 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:57.487 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:57.488 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:57.488 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:57.488 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:58.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:58.490 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:58.551 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:58.552 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:58.553 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:58.553 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:50:58.907 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:50:58.917 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:50:59.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:50:59.554 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:50:59.616 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:50:59.617 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC'} 2023-12-01T01:50:59.617 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:50:59.617 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:51:00.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:00.618 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:51:00.682 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:51:00.682 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_51_00_UTC 2023-12-01T01:51:00.683 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_50_00_UTC', 'scheduled-2023-12-01-01_51_00_UTC'} 2023-12-01T01:51:00.684 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-01-01_51_00_UTC'} 2023-12-01T01:51:00.684 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:51:00.684 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-01-01_51_00_UTC'] 2023-12-01T01:51:00.684 DEBUG:tasks.cephfs.test_snap_schedules:exec=1701395366.4765854, now = 1701395460.68451, timo = 94 2023-12-01T01:51:01.685 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-01T01:51:01.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.910+0000 7f2accd00640 1 Processor -- start 2023-12-01T01:51:01.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.911+0000 7f2accd00640 1 -- start start 2023-12-01T01:51:01.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.912+0000 7f2accd00640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2ac815cd90 0x7f2ac80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:01.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.912+0000 7f2accd00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ac80b5fa0 0x7f2ac8161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:01.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.912+0000 7f2accd00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2ac80cf650 0x7f2ac80cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:01.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.912+0000 7f2accd00640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac80d0ec0 con 0x7f2ac815cd90 2023-12-01T01:51:01.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.912+0000 7f2accd00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac80ce5e0 con 0x7f2ac80cf650 2023-12-01T01:51:01.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.912+0000 7f2accd00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac8161950 con 0x7f2ac80b5fa0 2023-12-01T01:51:01.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.912+0000 7f2ac659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2ac815cd90 0x7f2ac80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:01.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.913+0000 7f2ac659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2ac815cd90 0x7f2ac80cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52602/0 (socket says 172.21.15.115:52602) 2023-12-01T01:51:01.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.913+0000 7f2ac659c640 1 -- 172.21.15.115:0/716513082 learned_addr learned my addr 172.21.15.115:0/716513082 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:01.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.913+0000 7f2ac659c640 1 -- 172.21.15.115:0/716513082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ac80b5fa0 msgr2=0x7f2ac8161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:01.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.913+0000 7f2ac659c640 1 --2- 172.21.15.115:0/716513082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ac80b5fa0 0x7f2ac8161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:01.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.913+0000 7f2ac659c640 1 -- 172.21.15.115:0/716513082 >> v2:172.21.15.142:3300/0 conn(0x7f2ac80cf650 msgr2=0x7f2ac80cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:01.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.913+0000 7f2ac659c640 1 --2- 172.21.15.115:0/716513082 >> v2:172.21.15.142:3300/0 conn(0x7f2ac80cf650 0x7f2ac80cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:01.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.913+0000 7f2ac659c640 1 -- 172.21.15.115:0/716513082 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac80d0000 con 0x7f2ac815cd90 2023-12-01T01:51:01.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.914+0000 7f2ac659c640 1 --2- 172.21.15.115:0/716513082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac815cd90 0x7f2ac80cdd20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80cd3a0 tx=0x7f2ab802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cbdd9ed7587bff server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:01.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.914+0000 7f2ac559a640 1 -- 172.21.15.115:0/716513082 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab802ca70 con 0x7f2ac815cd90 2023-12-01T01:51:01.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.914+0000 7f2ac559a640 1 -- 172.21.15.115:0/716513082 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab802cbd0 con 0x7f2ac815cd90 2023-12-01T01:51:01.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.914+0000 7f2ac559a640 1 -- 172.21.15.115:0/716513082 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab80343f0 con 0x7f2ac815cd90 2023-12-01T01:51:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.915+0000 7f2accd00640 1 -- 172.21.15.115:0/716513082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac815cd90 msgr2=0x7f2ac80cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.915+0000 7f2accd00640 1 --2- 172.21.15.115:0/716513082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac815cd90 0x7f2ac80cdd20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80cd3a0 tx=0x7f2ab802bed0 comp rx=0 tx=0).stop 2023-12-01T01:51:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.915+0000 7f2accd00640 1 -- 172.21.15.115:0/716513082 shutdown_connections 2023-12-01T01:51:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.915+0000 7f2accd00640 1 --2- 172.21.15.115:0/716513082 >> v2:172.21.15.142:3300/0 conn(0x7f2ac80cf650 0x7f2ac80cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.915+0000 7f2accd00640 1 --2- 172.21.15.115:0/716513082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ac80b5fa0 0x7f2ac8161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.915+0000 7f2accd00640 1 --2- 172.21.15.115:0/716513082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac815cd90 0x7f2ac80cdd20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.915+0000 7f2accd00640 1 -- 172.21.15.115:0/716513082 >> 172.21.15.115:0/716513082 conn(0x7f2ac8065960 msgr2=0x7f2ac8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.915+0000 7f2accd00640 1 -- 172.21.15.115:0/716513082 shutdown_connections 2023-12-01T01:51:01.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.915+0000 7f2accd00640 1 -- 172.21.15.115:0/716513082 wait complete. 2023-12-01T01:51:01.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.917+0000 7f2accd00640 1 Processor -- start 2023-12-01T01:51:01.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.918+0000 7f2accd00640 1 -- start start 2023-12-01T01:51:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.919+0000 7f2accd00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ac80b5fa0 0x7f2ac8155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:01.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.919+0000 7f2accd00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac80cf650 0x7f2ac815f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:01.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.919+0000 7f2accd00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac815cd90 0x7f2ac80b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:01.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.919+0000 7f2accd00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac80aad20 con 0x7f2ac80cf650 2023-12-01T01:51:01.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.919+0000 7f2accd00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac81557a0 con 0x7f2ac815cd90 2023-12-01T01:51:01.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.919+0000 7f2accd00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac815dfd0 con 0x7f2ac80b5fa0 2023-12-01T01:51:01.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.919+0000 7f2ac5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac80cf650 0x7f2ac815f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:01.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.919+0000 7f2ac5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac80cf650 0x7f2ac815f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52612/0 (socket says 172.21.15.115:52612) 2023-12-01T01:51:01.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.919+0000 7f2ac5d9b640 1 -- 172.21.15.115:0/3587061387 learned_addr learned my addr 172.21.15.115:0/3587061387 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:01.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2ac6d9d640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac815cd90 0x7f2ac80b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:01.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2ac5d9b640 1 -- 172.21.15.115:0/3587061387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ac80b5fa0 msgr2=0x7f2ac8155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:01.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2ac659c640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ac80b5fa0 0x7f2ac8155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:01.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2ac5d9b640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ac80b5fa0 0x7f2ac8155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:01.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2ac5d9b640 1 -- 172.21.15.115:0/3587061387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac815cd90 msgr2=0x7f2ac80b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:01.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2ac5d9b640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac815cd90 0x7f2ac80b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:01.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2ac5d9b640 1 -- 172.21.15.115:0/3587061387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab8009b30 con 0x7f2ac80cf650 2023-12-01T01:51:01.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2ac6d9d640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac815cd90 0x7f2ac80b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:01.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2ac5d9b640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac80cf650 0x7f2ac815f940 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000b410 tx=0x7f2ab000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:01.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2abeffd640 1 -- 172.21.15.115:0/3587061387 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0011830 con 0x7f2ac80cf650 2023-12-01T01:51:01.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2abeffd640 1 -- 172.21.15.115:0/3587061387 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab00119b0 con 0x7f2ac80cf650 2023-12-01T01:51:01.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2accd00640 1 -- 172.21.15.115:0/3587061387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac815e2b0 con 0x7f2ac80cf650 2023-12-01T01:51:01.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.920+0000 7f2abeffd640 1 -- 172.21.15.115:0/3587061387 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0011d60 con 0x7f2ac80cf650 2023-12-01T01:51:01.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.921+0000 7f2accd00640 1 -- 172.21.15.115:0/3587061387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac815e870 con 0x7f2ac80cf650 2023-12-01T01:51:01.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.922+0000 7f2accd00640 1 -- 172.21.15.115:0/3587061387 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac80aaef0 con 0x7f2ac80cf650 2023-12-01T01:51:01.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.923+0000 7f2abeffd640 1 -- 172.21.15.115:0/3587061387 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 88738+0+0 (secure 0 0 0) 0x7f2ab001a400 con 0x7f2ac80cf650 2023-12-01T01:51:01.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.924+0000 7f2abeffd640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2aa806a5d0 0x7f2aa806ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:01.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.924+0000 7f2abeffd640 1 -- 172.21.15.115:0/3587061387 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f2ab0088e00 con 0x7f2ac80cf650 2023-12-01T01:51:01.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.924+0000 7f2ac659c640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2aa806a5d0 0x7f2aa806ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:01.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.924+0000 7f2ac659c640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2aa806a5d0 0x7f2aa806ca90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8035000 tx=0x7f2ab8030000 comp rx=0 tx=0).ready entity=mgr.6105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:01.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:01.930+0000 7f2abeffd640 1 -- 172.21.15.115:0/3587061387 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2ac80aaef0 con 0x7f2ac80cf650 2023-12-01T01:51:02.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.138+0000 7f2accd00640 1 -- 172.21.15.115:0/3587061387 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ac80d10a0 con 0x7f2aa806a5d0 2023-12-01T01:51:02.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.295+0000 7f2abeffd640 1 -- 172.21.15.115:0/3587061387 <== mgr.6105 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2ab8040450 con 0x7f2aa806a5d0 2023-12-01T01:51:02.297 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path snap_test_dir1 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 -- 172.21.15.115:0/3587061387 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2aa806a5d0 msgr2=0x7f2aa806ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2aa806a5d0 0x7f2aa806ca90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8035000 tx=0x7f2ab8030000 comp rx=0 tx=0).stop 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 -- 172.21.15.115:0/3587061387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac80cf650 msgr2=0x7f2ac815f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac80cf650 0x7f2ac815f940 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000b410 tx=0x7f2ab000b8d0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 -- 172.21.15.115:0/3587061387 shutdown_connections 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2aa806a5d0 0x7f2aa806ca90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac815cd90 0x7f2ac80b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac80cf650 0x7f2ac815f940 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 --2- 172.21.15.115:0/3587061387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ac80b5fa0 0x7f2ac8155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 -- 172.21.15.115:0/3587061387 >> 172.21.15.115:0/3587061387 conn(0x7f2ac8065960 msgr2=0x7f2ac80d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.298+0000 7f2accd00640 1 -- 172.21.15.115:0/3587061387 shutdown_connections 2023-12-01T01:51:02.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.299+0000 7f2accd00640 1 -- 172.21.15.115:0/3587061387 wait complete. 2023-12-01T01:51:02.321 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-01T01:51:02.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:02.321 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:51:02.375 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:51:02.375 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_51_00_UTC 2023-12-01T01:51:02.376 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-01_50_00_UTC 2023-12-01T01:51:02.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:02.377 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-01_50_00_UTC) 2023-12-01T01:51:02.437 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-01_51_00_UTC 2023-12-01T01:51:02.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:02.438 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-01_51_00_UTC) 2023-12-01T01:51:02.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:02.499 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-01T01:51:02.562 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T01:51:02.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.775+0000 7fbfc4ea0640 1 Processor -- start 2023-12-01T01:51:02.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.775+0000 7fbfc4ea0640 1 -- start start 2023-12-01T01:51:02.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.776+0000 7fbfc4ea0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfc0142870 0x7fbfc00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.776+0000 7fbfc4ea0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbfc00b1e20 0x7fbfc015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.776+0000 7fbfc4ea0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc00cd470 con 0x7fbfc00cd010 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.776+0000 7fbfc4ea0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc00b3ae0 con 0x7fbfc00b1e20 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.776+0000 7fbfc4ea0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0142c50 con 0x7fbfc0142870 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.777+0000 7fbfbed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbfc00b1e20 0x7fbfc015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.777+0000 7fbfbed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbfc00b1e20 0x7fbfc015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52628/0 (socket says 172.21.15.115:52628) 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.777+0000 7fbfbed9d640 1 -- 172.21.15.115:0/2437237215 learned_addr learned my addr 172.21.15.115:0/2437237215 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.777+0000 7fbfbe59c640 1 --2- 172.21.15.115:0/2437237215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfc0142870 0x7fbfc00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.777+0000 7fbfbed9d640 1 -- 172.21.15.115:0/2437237215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfc0142870 msgr2=0x7fbfc00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.777+0000 7fbfbed9d640 1 --2- 172.21.15.115:0/2437237215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfc0142870 0x7fbfc00ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.777+0000 7fbfbed9d640 1 -- 172.21.15.115:0/2437237215 >> v1:172.21.15.142:6789/0 conn(0x7fbfc00cd010 legacy=0x7fbfc00b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:02.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.777+0000 7fbfbed9d640 1 -- 172.21.15.115:0/2437237215 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00b2f30 con 0x7fbfc00b1e20 2023-12-01T01:51:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.778+0000 7fbfbed9d640 1 --2- 172.21.15.115:0/2437237215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 0x7fbfc015a0d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009e80 tx=0x7fbfb802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afac5d45fbab351 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfbdd9b640 1 -- 172.21.15.115:0/2437237215 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb802ca50 con 0x7fbfc00b1e20 2023-12-01T01:51:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfbdd9b640 1 -- 172.21.15.115:0/2437237215 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb802cbb0 con 0x7fbfc00b1e20 2023-12-01T01:51:02.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfbdd9b640 1 -- 172.21.15.115:0/2437237215 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80343f0 con 0x7fbfc00b1e20 2023-12-01T01:51:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/2437237215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 msgr2=0x7fbfc015a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfc4ea0640 1 --2- 172.21.15.115:0/2437237215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 0x7fbfc015a0d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009e80 tx=0x7fbfb802beb0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/2437237215 shutdown_connections 2023-12-01T01:51:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfc4ea0640 1 --2- 172.21.15.115:0/2437237215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 0x7fbfc015a0d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfc4ea0640 1 --2- 172.21.15.115:0/2437237215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfc0142870 0x7fbfc00ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/2437237215 >> 172.21.15.115:0/2437237215 conn(0x7fbfc0065960 msgr2=0x7fbfc0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/2437237215 shutdown_connections 2023-12-01T01:51:02.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.779+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/2437237215 wait complete. 2023-12-01T01:51:02.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.781+0000 7fbfc4ea0640 1 Processor -- start 2023-12-01T01:51:02.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.781+0000 7fbfc4ea0640 1 -- start start 2023-12-01T01:51:02.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.782+0000 7fbfc4ea0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 0x7fbfc00ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:02.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.782+0000 7fbfc4ea0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc00cd010 0x7fbfc013b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:02.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.782+0000 7fbfc4ea0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfc0142870 0x7fbfc00c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:02.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.782+0000 7fbfc4ea0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00c6650 con 0x7fbfc00b1e20 2023-12-01T01:51:02.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.782+0000 7fbfc4ea0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc013b860 con 0x7fbfc00cd010 2023-12-01T01:51:02.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.782+0000 7fbfc4ea0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00c5db0 con 0x7fbfc0142870 2023-12-01T01:51:02.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfbe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 0x7fbfc00ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:02.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfbe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 0x7fbfc00ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52634/0 (socket says 172.21.15.115:52634) 2023-12-01T01:51:02.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfbe59c640 1 -- 172.21.15.115:0/3279907526 learned_addr learned my addr 172.21.15.115:0/3279907526 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:02.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfb7fff640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc00cd010 0x7fbfc013b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfbed9d640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfc0142870 0x7fbfc00c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfbe59c640 1 -- 172.21.15.115:0/3279907526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfc0142870 msgr2=0x7fbfc00c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfbe59c640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfc0142870 0x7fbfc00c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfbe59c640 1 -- 172.21.15.115:0/3279907526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc00cd010 msgr2=0x7fbfc013b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfbe59c640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc00cd010 0x7fbfc013b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfbe59c640 1 -- 172.21.15.115:0/3279907526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb8009b30 con 0x7fbfc00b1e20 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.783+0000 7fbfbe59c640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 0x7fbfc00ca0e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000d620 tx=0x7fbfb000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.784+0000 7fbfb6ffd640 1 -- 172.21.15.115:0/3279907526 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb000b620 con 0x7fbfc00b1e20 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.784+0000 7fbfb6ffd640 1 -- 172.21.15.115:0/3279907526 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb000b780 con 0x7fbfc00b1e20 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.784+0000 7fbfb6ffd640 1 -- 172.21.15.115:0/3279907526 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb000bb30 con 0x7fbfc00b1e20 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.784+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/3279907526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc00c6090 con 0x7fbfc00b1e20 2023-12-01T01:51:02.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.784+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/3279907526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc014f2f0 con 0x7fbfc00b1e20 2023-12-01T01:51:02.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.785+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/3279907526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc004e890 con 0x7fbfc00b1e20 2023-12-01T01:51:02.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.786+0000 7fbfb6ffd640 1 -- 172.21.15.115:0/3279907526 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 27) v1 ==== 88738+0+0 (secure 0 0 0) 0x7fbfb000bc90 con 0x7fbfc00b1e20 2023-12-01T01:51:02.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.787+0000 7fbfb6ffd640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfa006a5a0 0x7fbfa006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:02.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.787+0000 7fbfb6ffd640 1 -- 172.21.15.115:0/3279907526 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6198+0+0 (secure 0 0 0) 0x7fbfb00891b0 con 0x7fbfc00b1e20 2023-12-01T01:51:02.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.787+0000 7fbfb7fff640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfa006a5a0 0x7fbfa006ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:02.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.788+0000 7fbfb7fff640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfa006a5a0 0x7fbfa006ca60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0152350 tx=0x7fbfa8009040 comp rx=0 tx=0).ready entity=mgr.6105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:02.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:02.791+0000 7fbfb6ffd640 1 -- 172.21.15.115:0/3279907526 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbfb0054490 con 0x7fbfc00b1e20 2023-12-01T01:51:03.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.033+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/3279907526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fbfc006b6d0 con 0x7fbfc00b1e20 2023-12-01T01:51:03.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.487+0000 7fbfb6ffd640 1 -- 172.21.15.115:0/3279907526 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 28) v1 ==== 88721+0+0 (secure 0 0 0) 0x7fbfb0054860 con 0x7fbfc00b1e20 2023-12-01T01:51:03.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.489+0000 7fbfb6ffd640 1 -- 172.21.15.115:0/3279907526 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v28) v1 ==== 93+0+0 (secure 0 0 0) 0x7fbfb0086aa0 con 0x7fbfc00b1e20 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/3279907526 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfa006a5a0 msgr2=0x7fbfa006ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfa006a5a0 0x7fbfa006ca60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0152350 tx=0x7fbfa8009040 comp rx=0 tx=0).stop 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/3279907526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 msgr2=0x7fbfc00ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 0x7fbfc00ca0e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000d620 tx=0x7fbfb000dae0 comp rx=0 tx=0).stop 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/3279907526 shutdown_connections 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbfa006a5a0 0x7fbfa006ca60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfc0142870 0x7fbfc00c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfc00cd010 0x7fbfc013b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 --2- 172.21.15.115:0/3279907526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfc00b1e20 0x7fbfc00ca0e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/3279907526 >> 172.21.15.115:0/3279907526 conn(0x7fbfc0065960 msgr2=0x7fbfc006f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:03.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/3279907526 shutdown_connections 2023-12-01T01:51:03.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:03.492+0000 7fbfc4ea0640 1 -- 172.21.15.115:0/3279907526 wait complete. 2023-12-01T01:51:03.513 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:51:03.514 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:51:03.604 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:51:03.705 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:51:03.705 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:51:03.705 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:03.756 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:51:03.756 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:03.846 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:04.016 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:04.015+0000 7f7eabe41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:51:04.057 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:04.056+0000 7f245b64a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:51:04.072 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:04.070+0000 7f7eabe41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:51:04.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.090+0000 7fd94e882640 1 Processor -- start 2023-12-01T01:51:04.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.090+0000 7fd94e882640 1 -- start start 2023-12-01T01:51:04.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.091+0000 7fd94e882640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd94815cd20 0x7fd9480d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.091+0000 7fd94e882640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd9480d1230 0x7fd9480d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.091+0000 7fd94e882640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9480d1b90 con 0x7fd9480b5d80 2023-12-01T01:51:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.091+0000 7fd94e882640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9480b2160 con 0x7fd94815cd20 2023-12-01T01:51:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.091+0000 7fd94e882640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9480b8c80 con 0x7fd9480d1230 2023-12-01T01:51:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.091+0000 7fd947fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd94815cd20 0x7fd9480d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.091+0000 7fd947fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd94815cd20 0x7fd9480d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52654/0 (socket says 172.21.15.115:52654) 2023-12-01T01:51:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.091+0000 7fd947fff640 1 -- 172.21.15.115:0/542093134 learned_addr learned my addr 172.21.15.115:0/542093134 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.092+0000 7fd94ce1f640 1 --2- 172.21.15.115:0/542093134 >> v2:172.21.15.142:3300/0 conn(0x7fd9480d1230 0x7fd9480d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.092+0000 7fd947fff640 1 -- 172.21.15.115:0/542093134 >> v2:172.21.15.142:3300/0 conn(0x7fd9480d1230 msgr2=0x7fd9480d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.092+0000 7fd947fff640 1 --2- 172.21.15.115:0/542093134 >> v2:172.21.15.142:3300/0 conn(0x7fd9480d1230 0x7fd9480d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.092+0000 7fd947fff640 1 -- 172.21.15.115:0/542093134 >> v1:172.21.15.142:6789/0 conn(0x7fd9480b5d80 legacy=0x7fd9480b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.092+0000 7fd947fff640 1 -- 172.21.15.115:0/542093134 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9480ce2e0 con 0x7fd94815cd20 2023-12-01T01:51:04.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.093+0000 7fd947fff640 1 --2- 172.21.15.115:0/542093134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94815cd20 0x7fd9480d24d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd9480c6720 tx=0x7fd93802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802526db0da19b4b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:04.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.093+0000 7fd9477fe640 1 -- 172.21.15.115:0/542093134 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93802c9f0 con 0x7fd94815cd20 2023-12-01T01:51:04.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.093+0000 7fd9477fe640 1 -- 172.21.15.115:0/542093134 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93802cb50 con 0x7fd94815cd20 2023-12-01T01:51:04.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.093+0000 7fd9477fe640 1 -- 172.21.15.115:0/542093134 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9380343f0 con 0x7fd94815cd20 2023-12-01T01:51:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.094+0000 7fd94e882640 1 -- 172.21.15.115:0/542093134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94815cd20 msgr2=0x7fd9480d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.094+0000 7fd94e882640 1 --2- 172.21.15.115:0/542093134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94815cd20 0x7fd9480d24d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd9480c6720 tx=0x7fd93802be50 comp rx=0 tx=0).stop 2023-12-01T01:51:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.094+0000 7fd94e882640 1 -- 172.21.15.115:0/542093134 shutdown_connections 2023-12-01T01:51:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.094+0000 7fd94e882640 1 --2- 172.21.15.115:0/542093134 >> v2:172.21.15.142:3300/0 conn(0x7fd9480d1230 0x7fd9480d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.094+0000 7fd94e882640 1 --2- 172.21.15.115:0/542093134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94815cd20 0x7fd9480d24d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.094+0000 7fd94e882640 1 -- 172.21.15.115:0/542093134 >> 172.21.15.115:0/542093134 conn(0x7fd94806a0e0 msgr2=0x7fd94806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.094+0000 7fd94e882640 1 -- 172.21.15.115:0/542093134 shutdown_connections 2023-12-01T01:51:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.094+0000 7fd94e882640 1 -- 172.21.15.115:0/542093134 wait complete. 2023-12-01T01:51:04.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.096+0000 7fd94e882640 1 Processor -- start 2023-12-01T01:51:04.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.096+0000 7fd94e882640 1 -- start start 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd94e882640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480b5d80 0x7fd9480cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd94e882640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9480d1230 0x7fd948159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd94e882640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd94815cd20 0x7fd948152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd94e882640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd948159fa0 con 0x7fd9480b5d80 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd94e882640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9481526b0 con 0x7fd9480d1230 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd94e882640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9480cb4a0 con 0x7fd94815cd20 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd947fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480b5d80 0x7fd9480cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd947fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480b5d80 0x7fd9480cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52666/0 (socket says 172.21.15.115:52666) 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd947fff640 1 -- 172.21.15.115:0/3478348203 learned_addr learned my addr 172.21.15.115:0/3478348203 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd93ffff640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9480d1230 0x7fd948159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.097+0000 7fd94ce1f640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd94815cd20 0x7fd948152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd947fff640 1 -- 172.21.15.115:0/3478348203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd94815cd20 msgr2=0x7fd948152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd947fff640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd94815cd20 0x7fd948152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd947fff640 1 -- 172.21.15.115:0/3478348203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9480d1230 msgr2=0x7fd948159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd947fff640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9480d1230 0x7fd948159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd947fff640 1 -- 172.21.15.115:0/3478348203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd938009b30 con 0x7fd9480b5d80 2023-12-01T01:51:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd94ce1f640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd94815cd20 0x7fd948152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd947fff640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480b5d80 0x7fd9480cada0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd938000f80 tx=0x7fd938004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd9457fa640 1 -- 172.21.15.115:0/3478348203 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93803e450 con 0x7fd9480b5d80 2023-12-01T01:51:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd9457fa640 1 -- 172.21.15.115:0/3478348203 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93803e5f0 con 0x7fd9480b5d80 2023-12-01T01:51:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd9457fa640 1 -- 172.21.15.115:0/3478348203 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93803e9a0 con 0x7fd9480b5d80 2023-12-01T01:51:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd94e882640 1 -- 172.21.15.115:0/3478348203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9480cb780 con 0x7fd9480b5d80 2023-12-01T01:51:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.098+0000 7fd94e882640 1 -- 172.21.15.115:0/3478348203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9480cbd40 con 0x7fd9480b5d80 2023-12-01T01:51:04.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.099+0000 7fd94e882640 1 -- 172.21.15.115:0/3478348203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9480b1b10 con 0x7fd9480b5d80 2023-12-01T01:51:04.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.100+0000 7fd9457fa640 1 -- 172.21.15.115:0/3478348203 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 88721+0+0 (secure 0 0 0) 0x7fd93803eb00 con 0x7fd9480b5d80 2023-12-01T01:51:04.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.101+0000 7fd9457fa640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd92806a550 0x7fd92806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.101+0000 7fd9457fa640 1 -- 172.21.15.115:0/3478348203 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6198+0+0 (secure 0 0 0) 0x7fd9380ab960 con 0x7fd9480b5d80 2023-12-01T01:51:04.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.101+0000 7fd93ffff640 1 -- 172.21.15.115:0/3478348203 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd92806a550 msgr2=0x7fd92806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:04.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.101+0000 7fd93ffff640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd92806a550 0x7fd92806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:04.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.106+0000 7fd9457fa640 1 -- 172.21.15.115:0/3478348203 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd938076ce0 con 0x7fd9480b5d80 2023-12-01T01:51:04.115 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:04.114+0000 7f245b64a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:51:04.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.296+0000 7fd94e882640 1 -- 172.21.15.115:0/3478348203 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9480d1b90 con 0x7fd9480b5d80 2023-12-01T01:51:04.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.298+0000 7fd9457fa640 1 -- 172.21.15.115:0/3478348203 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16817 (secure 0 0 0) 0x7fd9480d1b90 con 0x7fd9480b5d80 2023-12-01T01:51:04.300 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:04.300 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":67,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:50:25.758749+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:49:08.909721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-01T01:49:10.108096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.1b","mappings":[{"from":5,"to":0}]},{"pgid":"7.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:49:24.636862+0000","172.21.15.115:6822/88317":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/2910255832":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/844837827":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1628065986":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1021278773":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/3831003429":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1351821585":"2023-12-02T01:49:24.636862+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:04.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 -- 172.21.15.115:0/3478348203 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd92806a550 msgr2=0x7fd92806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:04.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd92806a550 0x7fd92806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 -- 172.21.15.115:0/3478348203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480b5d80 msgr2=0x7fd9480cada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480b5d80 0x7fd9480cada0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd938000f80 tx=0x7fd938004a40 comp rx=0 tx=0).stop 2023-12-01T01:51:04.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 -- 172.21.15.115:0/3478348203 shutdown_connections 2023-12-01T01:51:04.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd92806a550 0x7fd92806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd94815cd20 0x7fd948152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9480d1230 0x7fd948159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 --2- 172.21.15.115:0/3478348203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480b5d80 0x7fd9480cada0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 -- 172.21.15.115:0/3478348203 >> 172.21.15.115:0/3478348203 conn(0x7fd94806a0e0 msgr2=0x7fd9481563d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:04.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 -- 172.21.15.115:0/3478348203 shutdown_connections 2023-12-01T01:51:04.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.300+0000 7fd94e882640 1 -- 172.21.15.115:0/3478348203 wait complete. 2023-12-01T01:51:04.321 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:49:08.909721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:49:10.108096+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:51:04.321 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:51:04.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.523+0000 7f6950bab640 1 Processor -- start 2023-12-01T01:51:04.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.523+0000 7f6950bab640 1 -- start start 2023-12-01T01:51:04.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.524+0000 7f6950bab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f694c15cf70 0x7f694c0b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.525+0000 7f6950bab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0d0260 0x7f694c0b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.525+0000 7f6950bab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694c0b7760 con 0x7f694c0b18f0 2023-12-01T01:51:04.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.525+0000 7f6950bab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694c0b2170 con 0x7f694c15cf70 2023-12-01T01:51:04.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.525+0000 7f6950bab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694c0b8b70 con 0x7f694c0d0260 2023-12-01T01:51:04.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.525+0000 7f694a59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f694c15cf70 0x7f694c0b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.525+0000 7f694ad9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0d0260 0x7f694c0b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.525+0000 7f694a59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f694c15cf70 0x7f694c0b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:46262/0 (socket says 172.21.15.115:46262) 2023-12-01T01:51:04.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.525+0000 7f694ad9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0d0260 0x7f694c0b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:44366/0 (socket says 172.21.15.115:44366) 2023-12-01T01:51:04.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.525+0000 7f694a59c640 1 -- 172.21.15.115:0/4752017 learned_addr learned my addr 172.21.15.115:0/4752017 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.526+0000 7f694a59c640 1 -- 172.21.15.115:0/4752017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0d0260 msgr2=0x7f694c0b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.526+0000 7f694a59c640 1 --2- 172.21.15.115:0/4752017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0d0260 0x7f694c0b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.526+0000 7f694a59c640 1 -- 172.21.15.115:0/4752017 >> v1:172.21.15.142:6789/0 conn(0x7f694c0b18f0 legacy=0x7f694c0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.526+0000 7f694a59c640 1 -- 172.21.15.115:0/4752017 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0b6e30 con 0x7f694c15cf70 2023-12-01T01:51:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.526+0000 7f694ad9d640 1 --2- 172.21.15.115:0/4752017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0d0260 0x7f694c0b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:51:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.527+0000 7f694a59c640 1 --2- 172.21.15.115:0/4752017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694c15cf70 0x7f694c0b8470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f693c009870 tx=0x7f693c02ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19ba22c6af9a3e91 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.528+0000 7f6949d9b640 1 -- 172.21.15.115:0/4752017 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c02f830 con 0x7f694c15cf70 2023-12-01T01:51:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.528+0000 7f6949d9b640 1 -- 172.21.15.115:0/4752017 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c02f990 con 0x7f694c15cf70 2023-12-01T01:51:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.528+0000 7f6949d9b640 1 -- 172.21.15.115:0/4752017 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c02fd20 con 0x7f694c15cf70 2023-12-01T01:51:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.528+0000 7f6950bab640 1 -- 172.21.15.115:0/4752017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694c15cf70 msgr2=0x7f694c0b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.528+0000 7f6950bab640 1 --2- 172.21.15.115:0/4752017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694c15cf70 0x7f694c0b8470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f693c009870 tx=0x7f693c02ba10 comp rx=0 tx=0).stop 2023-12-01T01:51:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.528+0000 7f6950bab640 1 -- 172.21.15.115:0/4752017 shutdown_connections 2023-12-01T01:51:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.528+0000 7f6950bab640 1 --2- 172.21.15.115:0/4752017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0d0260 0x7f694c0b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.528+0000 7f6950bab640 1 --2- 172.21.15.115:0/4752017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694c15cf70 0x7f694c0b8470 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.528+0000 7f6950bab640 1 -- 172.21.15.115:0/4752017 >> 172.21.15.115:0/4752017 conn(0x7f694c06a0e0 msgr2=0x7f694c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.529+0000 7f6950bab640 1 -- 172.21.15.115:0/4752017 shutdown_connections 2023-12-01T01:51:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.529+0000 7f6950bab640 1 -- 172.21.15.115:0/4752017 wait complete. 2023-12-01T01:51:04.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.530+0000 7f6950bab640 1 Processor -- start 2023-12-01T01:51:04.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.530+0000 7f6950bab640 1 -- start start 2023-12-01T01:51:04.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.531+0000 7f6950bab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0b18f0 0x7f694c152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.531+0000 7f6950bab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694c0d0260 0x7f694c1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.531+0000 7f6950bab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f694c15cf70 0x7f694c0d2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.531+0000 7f6950bab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c1534a0 con 0x7f694c15cf70 2023-12-01T01:51:04.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.531+0000 7f6950bab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c156ad0 con 0x7f694c0d0260 2023-12-01T01:51:04.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.531+0000 7f6950bab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694c156640 con 0x7f694c0b18f0 2023-12-01T01:51:04.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.531+0000 7f694ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f694c15cf70 0x7f694c0d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f694ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f694c15cf70 0x7f694c0d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52672/0 (socket says 172.21.15.115:52672) 2023-12-01T01:51:04.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f694ad9d640 1 -- 172.21.15.115:0/2240365706 learned_addr learned my addr 172.21.15.115:0/2240365706 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:04.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f694a59c640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0b18f0 0x7f694c152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f6943fff640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694c0d0260 0x7f694c1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f694ad9d640 1 -- 172.21.15.115:0/2240365706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0b18f0 msgr2=0x7f694c152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f694ad9d640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0b18f0 0x7f694c152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f694ad9d640 1 -- 172.21.15.115:0/2240365706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694c0d0260 msgr2=0x7f694c1544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f694ad9d640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694c0d0260 0x7f694c1544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f694ad9d640 1 -- 172.21.15.115:0/2240365706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6944009a70 con 0x7f694c15cf70 2023-12-01T01:51:04.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f694ad9d640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f694c15cf70 0x7f694c0d2060 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6944009f00 tx=0x7f694400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:04.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.532+0000 7f6942ffd640 1 -- 172.21.15.115:0/2240365706 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6944009d70 con 0x7f694c15cf70 2023-12-01T01:51:04.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.533+0000 7f6942ffd640 1 -- 172.21.15.115:0/2240365706 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694400f040 con 0x7f694c15cf70 2023-12-01T01:51:04.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.533+0000 7f6950bab640 1 -- 172.21.15.115:0/2240365706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693c0094d0 con 0x7f694c15cf70 2023-12-01T01:51:04.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.533+0000 7f6942ffd640 1 -- 172.21.15.115:0/2240365706 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6944003bb0 con 0x7f694c15cf70 2023-12-01T01:51:04.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.533+0000 7f6950bab640 1 -- 172.21.15.115:0/2240365706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c155d10 con 0x7f694c15cf70 2023-12-01T01:51:04.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.534+0000 7f6950bab640 1 -- 172.21.15.115:0/2240365706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694c04e890 con 0x7f694c15cf70 2023-12-01T01:51:04.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.535+0000 7f6942ffd640 1 -- 172.21.15.115:0/2240365706 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f6944014480 con 0x7f694c15cf70 2023-12-01T01:51:04.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.535+0000 7f6942ffd640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f692c06a550 0x7f692c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.536+0000 7f6942ffd640 1 -- 172.21.15.115:0/2240365706 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f6944089a30 con 0x7f694c15cf70 2023-12-01T01:51:04.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.536+0000 7f694a59c640 1 -- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f692c06a550 msgr2=0x7f692c06ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:04.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.536+0000 7f694a59c640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f692c06a550 0x7f692c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:04.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.544+0000 7f6942ffd640 1 -- 172.21.15.115:0/2240365706 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6944054c00 con 0x7f694c15cf70 2023-12-01T01:51:04.545 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:04.544+0000 7f7eabe41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:51:04.572 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:04.571+0000 7f245b64a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:51:04.599 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:04.597+0000 7f7eabe41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:51:04.624 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:04.622+0000 7f245b64a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:51:04.650 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:04.649+0000 7f7eabe41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:51:04.674 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:04.673+0000 7f245b64a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:51:04.711 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:04.710+0000 7f7eabe41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:51:04.736 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:04.734+0000 7f245b64a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:51:04.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.736+0000 7f6950bab640 1 -- 172.21.15.115:0/2240365706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f694c156780 con 0x7f694c15cf70 2023-12-01T01:51:04.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.737+0000 7f6942ffd640 1 -- 172.21.15.115:0/2240365706 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69440180a0 con 0x7f694c15cf70 2023-12-01T01:51:04.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.737+0000 7f694a59c640 1 -- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f692c06a550 msgr2=0x7f692c06ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:04.738 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:51:04.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.737+0000 7f694a59c640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f692c06a550 0x7f692c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 -- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f692c06a550 msgr2=0x7f692c06ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f692c06a550 0x7f692c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 -- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f694c15cf70 msgr2=0x7f694c0d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f694c15cf70 0x7f694c0d2060 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6944009f00 tx=0x7f694400c490 comp rx=0 tx=0).stop 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 -- 172.21.15.115:0/2240365706 shutdown_connections 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f692c06a550 0x7f692c06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f694c15cf70 0x7f694c0d2060 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694c0d0260 0x7f694c1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 --2- 172.21.15.115:0/2240365706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f694c0b18f0 0x7f694c152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 -- 172.21.15.115:0/2240365706 >> 172.21.15.115:0/2240365706 conn(0x7f694c06a0e0 msgr2=0x7f694c0b8330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.739+0000 7f6950bab640 1 -- 172.21.15.115:0/2240365706 shutdown_connections 2023-12-01T01:51:04.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.740+0000 7f6950bab640 1 -- 172.21.15.115:0/2240365706 wait complete. 2023-12-01T01:51:04.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:51:04.763 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:04.761+0000 7f7eabe41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:51:04.785 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:04.783+0000 7f245b64a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:51:04.880 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:04.878+0000 7f7eabe41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:51:04.895 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:04.893+0000 7f245b64a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:51:04.945 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:04.944+0000 7f7eabe41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:04.957 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:04.955+0000 7f245b64a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.958+0000 7f06fe100640 1 Processor -- start 2023-12-01T01:51:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.958+0000 7f06fe100640 1 -- start start 2023-12-01T01:51:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.959+0000 7f06fe100640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f06f815cf70 0x7f06f80b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.959+0000 7f06fe100640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f80b7760 con 0x7f06f80b18f0 2023-12-01T01:51:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.959+0000 7f06fe100640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f80b8b70 con 0x7f06f80d0260 2023-12-01T01:51:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.959+0000 7f06fe100640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f80b2170 con 0x7f06f815cf70 2023-12-01T01:51:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.960+0000 7f06f77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f06f815cf70 0x7f06f80b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.960+0000 7f06f77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f06f815cf70 0x7f06f80b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52674/0 (socket says 172.21.15.115:52674) 2023-12-01T01:51:04.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.960+0000 7f06f77fe640 1 -- 172.21.15.115:0/2184713493 learned_addr learned my addr 172.21.15.115:0/2184713493 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:04.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.960+0000 7f06f77fe640 1 -- 172.21.15.115:0/2184713493 >> v1:172.21.15.142:6789/0 conn(0x7f06f80d0260 legacy=0x7f06f80d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:04.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.960+0000 7f06f77fe640 1 -- 172.21.15.115:0/2184713493 >> v1:172.21.15.115:6789/0 conn(0x7f06f80b18f0 legacy=0x7f06f80cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.960+0000 7f06f77fe640 1 -- 172.21.15.115:0/2184713493 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f80b6e30 con 0x7f06f815cf70 2023-12-01T01:51:04.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.961+0000 7f06f77fe640 1 --2- 172.21.15.115:0/2184713493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f815cf70 0x7f06f80b8470 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009870 tx=0x7f06e802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c5b7efce4a7687 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:04.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.961+0000 7f06f6ffd640 1 -- 172.21.15.115:0/2184713493 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e802f840 con 0x7f06f815cf70 2023-12-01T01:51:04.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.962+0000 7f06f6ffd640 1 -- 172.21.15.115:0/2184713493 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e802f9a0 con 0x7f06f815cf70 2023-12-01T01:51:04.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.962+0000 7f06f6ffd640 1 -- 172.21.15.115:0/2184713493 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e802fd30 con 0x7f06f815cf70 2023-12-01T01:51:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.962+0000 7f06fe100640 1 -- 172.21.15.115:0/2184713493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f815cf70 msgr2=0x7f06f80b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.962+0000 7f06fe100640 1 --2- 172.21.15.115:0/2184713493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f815cf70 0x7f06f80b8470 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009870 tx=0x7f06e802ba20 comp rx=0 tx=0).stop 2023-12-01T01:51:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.962+0000 7f06fe100640 1 -- 172.21.15.115:0/2184713493 shutdown_connections 2023-12-01T01:51:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.962+0000 7f06fe100640 1 --2- 172.21.15.115:0/2184713493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f815cf70 0x7f06f80b8470 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.962+0000 7f06fe100640 1 -- 172.21.15.115:0/2184713493 >> 172.21.15.115:0/2184713493 conn(0x7f06f806a0e0 msgr2=0x7f06f806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.962+0000 7f06fe100640 1 -- 172.21.15.115:0/2184713493 shutdown_connections 2023-12-01T01:51:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.962+0000 7f06fe100640 1 -- 172.21.15.115:0/2184713493 wait complete. 2023-12-01T01:51:04.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.964+0000 7f06fe100640 1 Processor -- start 2023-12-01T01:51:04.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.964+0000 7f06fe100640 1 -- start start 2023-12-01T01:51:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.965+0000 7f06fe100640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f80b18f0 0x7f06f814e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.965+0000 7f06fe100640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06f80d0260 0x7f06f80d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.965+0000 7f06fe100640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06f815cf70 0x7f06f80d2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.965+0000 7f06fe100640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f815e090 con 0x7f06f80b18f0 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.965+0000 7f06fe100640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f80d2870 con 0x7f06f80d0260 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.965+0000 7f06fe100640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f815d480 con 0x7f06f815cf70 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.966+0000 7f06f77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f80b18f0 0x7f06f814e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.966+0000 7f06f77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f80b18f0 0x7f06f814e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52682/0 (socket says 172.21.15.115:52682) 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.966+0000 7f06f77fe640 1 -- 172.21.15.115:0/3445420165 learned_addr learned my addr 172.21.15.115:0/3445420165 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.966+0000 7f06f77fe640 1 -- 172.21.15.115:0/3445420165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06f815cf70 msgr2=0x7f06f80d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.966+0000 7f06f77fe640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06f815cf70 0x7f06f80d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.966+0000 7f06f77fe640 1 -- 172.21.15.115:0/3445420165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06f80d0260 msgr2=0x7f06f80d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.966+0000 7f06f77fe640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06f80d0260 0x7f06f80d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.966+0000 7f06f77fe640 1 -- 172.21.15.115:0/3445420165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80094d0 con 0x7f06f80b18f0 2023-12-01T01:51:04.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.966+0000 7f06f77fe640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f80b18f0 0x7f06f814e140 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f06e8000c00 tx=0x7f06e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.967+0000 7f06f4ff9640 1 -- 172.21.15.115:0/3445420165 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e802f7f0 con 0x7f06f80b18f0 2023-12-01T01:51:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.967+0000 7f06f4ff9640 1 -- 172.21.15.115:0/3445420165 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8041400 con 0x7f06f80b18f0 2023-12-01T01:51:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.967+0000 7f06f4ff9640 1 -- 172.21.15.115:0/3445420165 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e80417b0 con 0x7f06f80b18f0 2023-12-01T01:51:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.967+0000 7f06fe100640 1 -- 172.21.15.115:0/3445420165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f815d760 con 0x7f06f80b18f0 2023-12-01T01:51:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.967+0000 7f06fe100640 1 -- 172.21.15.115:0/3445420165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f815dd20 con 0x7f06f80b18f0 2023-12-01T01:51:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.968+0000 7f06fe100640 1 -- 172.21.15.115:0/3445420165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f804e890 con 0x7f06f80b18f0 2023-12-01T01:51:04.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.969+0000 7f06f4ff9640 1 -- 172.21.15.115:0/3445420165 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f06e8041910 con 0x7f06f80b18f0 2023-12-01T01:51:04.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.970+0000 7f06f4ff9640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f06d806a550 0x7f06d806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:04.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.970+0000 7f06f4ff9640 1 -- 172.21.15.115:0/3445420165 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f06e803c070 con 0x7f06f80b18f0 2023-12-01T01:51:04.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.970+0000 7f06effff640 1 -- 172.21.15.115:0/3445420165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f06d806a550 msgr2=0x7f06d806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:04.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.970+0000 7f06effff640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f06d806a550 0x7f06d806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:04.974+0000 7f06f4ff9640 1 -- 172.21.15.115:0/3445420165 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f06e8076e60 con 0x7f06f80b18f0 2023-12-01T01:51:04.998 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:04.997+0000 7f7eabe41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:51:05.007 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:05.006+0000 7f245b64a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:51:05.057 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:05.056+0000 7f7eabe41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:51:05.064 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:05.063+0000 7f245b64a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:51:05.109 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:05.108+0000 7f7eabe41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:51:05.114 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:05.112+0000 7f245b64a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:51:05.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.163+0000 7f06fe100640 1 -- 172.21.15.115:0/3445420165 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06f80670e0 con 0x7f06f80b18f0 2023-12-01T01:51:05.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.164+0000 7f06f4ff9640 1 -- 172.21.15.115:0/3445420165 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06e807a480 con 0x7f06f80b18f0 2023-12-01T01:51:05.165 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:51:05.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.166+0000 7f06fe100640 1 -- 172.21.15.115:0/3445420165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f06d806a550 msgr2=0x7f06d806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:05.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.166+0000 7f06fe100640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f06d806a550 0x7f06d806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.166+0000 7f06fe100640 1 -- 172.21.15.115:0/3445420165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f80b18f0 msgr2=0x7f06f814e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.166+0000 7f06fe100640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f80b18f0 0x7f06f814e140 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f06e8000c00 tx=0x7f06e8004730 comp rx=0 tx=0).stop 2023-12-01T01:51:05.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.166+0000 7f06fe100640 1 -- 172.21.15.115:0/3445420165 shutdown_connections 2023-12-01T01:51:05.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.166+0000 7f06fe100640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f06d806a550 0x7f06d806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.166+0000 7f06fe100640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06f815cf70 0x7f06f80d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.166+0000 7f06fe100640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06f80d0260 0x7f06f80d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.166+0000 7f06fe100640 1 --2- 172.21.15.115:0/3445420165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06f80b18f0 0x7f06f814e140 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.166+0000 7f06fe100640 1 -- 172.21.15.115:0/3445420165 >> 172.21.15.115:0/3445420165 conn(0x7f06f806a0e0 msgr2=0x7f06f80b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:05.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.167+0000 7f06fe100640 1 -- 172.21.15.115:0/3445420165 shutdown_connections 2023-12-01T01:51:05.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.167+0000 7f06fe100640 1 -- 172.21.15.115:0/3445420165 wait complete. 2023-12-01T01:51:05.184 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:51:05.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.383+0000 7fb37742d640 1 Processor -- start 2023-12-01T01:51:05.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.383+0000 7fb37742d640 1 -- start start 2023-12-01T01:51:05.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.385+0000 7fb37742d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700cf140 0x7fb3700cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.385+0000 7fb37742d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3700ce5a0 con 0x7fb3700b37f0 2023-12-01T01:51:05.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.385+0000 7fb37742d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb37015b350 con 0x7fb37015af70 2023-12-01T01:51:05.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.385+0000 7fb37742d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb370072260 con 0x7fb3700cf140 2023-12-01T01:51:05.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.385+0000 7fb375c2a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb3700b37f0 0x7fb3700cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59566/0 (socket says 172.21.15.115:59566) 2023-12-01T01:51:05.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.385+0000 7fb375c2a640 1 -- 172.21.15.115:0/857320558 learned_addr learned my addr 172.21.15.115:0/857320558 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb376c2c640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700cf140 0x7fb3700cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767946569 0 0) 0x7fb3700ce5a0 con 0x7fb3700b37f0 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb35c003520 con 0x7fb3700b37f0 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 910711247 0 0) 0x7fb35c003520 con 0x7fb3700b37f0 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700cf140 msgr2=0x7fb3700cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700cf140 0x7fb3700cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 >> v1:172.21.15.142:6789/0 conn(0x7fb37015af70 legacy=0x7fb370071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700b2550 con 0x7fb3700b37f0 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb360001670 con 0x7fb3700b37f0 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 >> v1:172.21.15.115:6789/0 conn(0x7fb3700b37f0 legacy=0x7fb3700cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700cf140 0x7fb35c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb35c003f70 0x7fb35c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb35c004d00 0x7fb35c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb370072260 con 0x7fb35c003f70 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c00bac0 con 0x7fb35c004d00 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.386+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c010090 con 0x7fb3700cf140 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.387+0000 7fb375c2a640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb35c003f70 0x7fb35c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.387+0000 7fb376c2c640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb35c004d00 0x7fb35c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.387+0000 7fb37642b640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700cf140 0x7fb35c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.387+0000 7fb375c2a640 1 -- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700cf140 msgr2=0x7fb35c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.387+0000 7fb375c2a640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700cf140 0x7fb35c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.387+0000 7fb375c2a640 1 -- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb35c004d00 msgr2=0x7fb35c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.387+0000 7fb375c2a640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb35c004d00 0x7fb35c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.387+0000 7fb375c2a640 1 -- 172.21.15.115:0/857320558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c003520 con 0x7fb35c003f70 2023-12-01T01:51:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.387+0000 7fb37642b640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700cf140 0x7fb35c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.389+0000 7fb375c2a640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb35c003f70 0x7fb35c0047c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c0e0 tx=0x7fb35c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b220e9fa554195d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.389+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360005ad0 con 0x7fb35c003f70 2023-12-01T01:51:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.389+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb360005c30 con 0x7fb35c003f70 2023-12-01T01:51:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.389+0000 7fb375429640 1 -- 172.21.15.115:0/857320558 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3600072b0 con 0x7fb35c003f70 2023-12-01T01:51:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.390+0000 7fb37742d640 1 -- 172.21.15.115:0/857320558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb35c003f70 msgr2=0x7fb35c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.390+0000 7fb37742d640 1 --2- 172.21.15.115:0/857320558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb35c003f70 0x7fb35c0047c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c0e0 tx=0x7fb35c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.390+0000 7fb376c2c640 1 -- 172.21.15.115:0/857320558 reap_dead start 2023-12-01T01:51:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.390+0000 7fb37742d640 1 -- 172.21.15.115:0/857320558 shutdown_connections 2023-12-01T01:51:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.390+0000 7fb37742d640 1 -- 172.21.15.115:0/857320558 >> 172.21.15.115:0/857320558 conn(0x7fb3700cd420 msgr2=0x7fb3700cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.390+0000 7fb37742d640 1 -- 172.21.15.115:0/857320558 shutdown_connections 2023-12-01T01:51:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.390+0000 7fb37742d640 1 -- 172.21.15.115:0/857320558 wait complete. 2023-12-01T01:51:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.391+0000 7fb37742d640 1 Processor -- start 2023-12-01T01:51:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.392+0000 7fb37742d640 1 -- start start 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.392+0000 7fb37742d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb35c003f70 0x7fb370156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.392+0000 7fb37742d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3700c9510 0x7fb3700b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.392+0000 7fb37742d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700b6250 0x7fb370157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb37742d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700b66e0 con 0x7fb3700c9510 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb37742d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700c8560 con 0x7fb35c003f70 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb37742d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb370158aa0 con 0x7fb3700b6250 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb375c2a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3700c9510 0x7fb3700b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb375c2a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3700c9510 0x7fb3700b5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52708/0 (socket says 172.21.15.115:52708) 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb375c2a640 1 -- 172.21.15.115:0/1875915359 learned_addr learned my addr 172.21.15.115:0/1875915359 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb37642b640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb35c003f70 0x7fb370156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb375c2a640 1 -- 172.21.15.115:0/1875915359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700b6250 msgr2=0x7fb370157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb375c2a640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700b6250 0x7fb370157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb375c2a640 1 -- 172.21.15.115:0/1875915359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb35c003f70 msgr2=0x7fb370156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb375c2a640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb35c003f70 0x7fb370156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.393+0000 7fb375c2a640 1 -- 172.21.15.115:0/1875915359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700c8790 con 0x7fb3700c9510 2023-12-01T01:51:05.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.394+0000 7fb375c2a640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3700c9510 0x7fb3700b5d10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c0e0 tx=0x7fb360001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:05.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.394+0000 7fb366ffd640 1 -- 172.21.15.115:0/1875915359 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36003c040 con 0x7fb3700c9510 2023-12-01T01:51:05.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.394+0000 7fb366ffd640 1 -- 172.21.15.115:0/1875915359 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb360007480 con 0x7fb3700c9510 2023-12-01T01:51:05.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.394+0000 7fb366ffd640 1 -- 172.21.15.115:0/1875915359 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36002c740 con 0x7fb3700c9510 2023-12-01T01:51:05.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.394+0000 7fb37742d640 1 -- 172.21.15.115:0/1875915359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3700c8a30 con 0x7fb3700c9510 2023-12-01T01:51:05.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.394+0000 7fb37742d640 1 -- 172.21.15.115:0/1875915359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3700c8ff0 con 0x7fb3700c9510 2023-12-01T01:51:05.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.395+0000 7fb37742d640 1 -- 172.21.15.115:0/1875915359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3700618d0 con 0x7fb3700c9510 2023-12-01T01:51:05.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.397+0000 7fb366ffd640 1 -- 172.21.15.115:0/1875915359 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 88721+0+0 (secure 0 0 0) 0x7fb360032070 con 0x7fb3700c9510 2023-12-01T01:51:05.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.398+0000 7fb366ffd640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb35406a550 0x7fb35406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.398+0000 7fb366ffd640 1 -- 172.21.15.115:0/1875915359 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6198+0+0 (secure 0 0 0) 0x7fb3600aa2d0 con 0x7fb3700c9510 2023-12-01T01:51:05.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.398+0000 7fb37642b640 1 -- 172.21.15.115:0/1875915359 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb35406a550 msgr2=0x7fb35406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:05.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.398+0000 7fb37642b640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb35406a550 0x7fb35406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:05.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.402+0000 7fb366ffd640 1 -- 172.21.15.115:0/1875915359 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb360075490 con 0x7fb3700c9510 2023-12-01T01:51:05.507 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:05.506+0000 7f245b64a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:51:05.518 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:05.517+0000 7f7eabe41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:51:05.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.595+0000 7fb37742d640 1 -- 172.21.15.115:0/1875915359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3700b2dc0 con 0x7fb3700c9510 2023-12-01T01:51:05.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.596+0000 7fb366ffd640 1 -- 172.21.15.115:0/1875915359 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3600460a0 con 0x7fb3700c9510 2023-12-01T01:51:05.597 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 -- 172.21.15.115:0/1875915359 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb35406a550 msgr2=0x7fb35406ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb35406a550 0x7fb35406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 -- 172.21.15.115:0/1875915359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3700c9510 msgr2=0x7fb3700b5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3700c9510 0x7fb3700b5d10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c0e0 tx=0x7fb360001930 comp rx=0 tx=0).stop 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 -- 172.21.15.115:0/1875915359 shutdown_connections 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb35406a550 0x7fb35406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3700b6250 0x7fb370157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3700c9510 0x7fb3700b5d10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 --2- 172.21.15.115:0/1875915359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb35c003f70 0x7fb370156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 -- 172.21.15.115:0/1875915359 >> 172.21.15.115:0/1875915359 conn(0x7fb3700cd420 msgr2=0x7fb3700cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 -- 172.21.15.115:0/1875915359 shutdown_connections 2023-12-01T01:51:05.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.598+0000 7fb37742d640 1 -- 172.21.15.115:0/1875915359 wait complete. 2023-12-01T01:51:05.618 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:51:05.669 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:05.668+0000 7f245b64a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:51:05.691 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:05.689+0000 7f7eabe41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:51:05.785 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:05.784+0000 7f245b64a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:51:05.809 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:05.807+0000 7f7eabe41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:51:05.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.813+0000 7f1474dfb640 1 Processor -- start 2023-12-01T01:51:05.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.813+0000 7f1474dfb640 1 -- start start 2023-12-01T01:51:05.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.814+0000 7f1474dfb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f14700b5d80 0x7f14700d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.814+0000 7f1474dfb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f14700d1230 0x7f14700d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.814+0000 7f1474dfb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14700d1b90 con 0x7f147015cd20 2023-12-01T01:51:05.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.814+0000 7f1474dfb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14700b2160 con 0x7f14700d1230 2023-12-01T01:51:05.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.814+0000 7f1474dfb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14700b8c80 con 0x7f14700b5d80 2023-12-01T01:51:05.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.815+0000 7f146e59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f147015cd20 0x7f14700b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59574/0 (socket says 172.21.15.115:59574) 2023-12-01T01:51:05.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.815+0000 7f146e59c640 1 -- 172.21.15.115:0/3124370724 learned_addr learned my addr 172.21.15.115:0/3124370724 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:05.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.815+0000 7f146ed9d640 1 --2- 172.21.15.115:0/3124370724 >> v2:172.21.15.115:3300/0 conn(0x7f14700d1230 0x7f14700d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:05.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.815+0000 7f146dd9b640 1 -- 172.21.15.115:0/3124370724 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422657645 0 0) 0x7f1460001110 con 0x7f147015cd20 2023-12-01T01:51:05.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.815+0000 7f146dd9b640 1 -- 172.21.15.115:0/3124370724 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f145c003520 con 0x7f147015cd20 2023-12-01T01:51:05.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.815+0000 7f146ed9d640 1 -- 172.21.15.115:0/3124370724 >> v2:172.21.15.142:3300/0 conn(0x7f14700b5d80 msgr2=0x7f14700d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:05.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.815+0000 7f146ed9d640 1 --2- 172.21.15.115:0/3124370724 >> v2:172.21.15.142:3300/0 conn(0x7f14700b5d80 0x7f14700d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.815+0000 7f146ed9d640 1 -- 172.21.15.115:0/3124370724 >> v1:172.21.15.115:6789/0 conn(0x7f147015cd20 legacy=0x7f14700b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.815+0000 7f146ed9d640 1 -- 172.21.15.115:0/3124370724 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14700ce2e0 con 0x7f14700d1230 2023-12-01T01:51:05.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.817+0000 7f146ed9d640 1 --2- 172.21.15.115:0/3124370724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 0x7f14700d1650 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f14700cf630 tx=0x7f146802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603f67be446be1e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:05.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.817+0000 7f146dd9b640 1 -- 172.21.15.115:0/3124370724 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146802cbe0 con 0x7f14700d1230 2023-12-01T01:51:05.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.817+0000 7f146dd9b640 1 -- 172.21.15.115:0/3124370724 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146802cd40 con 0x7f14700d1230 2023-12-01T01:51:05.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.817+0000 7f146dd9b640 1 -- 172.21.15.115:0/3124370724 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468034660 con 0x7f14700d1230 2023-12-01T01:51:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.818+0000 7f1474dfb640 1 -- 172.21.15.115:0/3124370724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 msgr2=0x7f14700d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.818+0000 7f1474dfb640 1 --2- 172.21.15.115:0/3124370724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 0x7f14700d1650 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f14700cf630 tx=0x7f146802be30 comp rx=0 tx=0).stop 2023-12-01T01:51:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.818+0000 7f1474dfb640 1 -- 172.21.15.115:0/3124370724 shutdown_connections 2023-12-01T01:51:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.818+0000 7f1474dfb640 1 --2- 172.21.15.115:0/3124370724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 0x7f14700d1650 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.818+0000 7f1474dfb640 1 --2- 172.21.15.115:0/3124370724 >> v2:172.21.15.142:3300/0 conn(0x7f14700b5d80 0x7f14700d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.818+0000 7f1474dfb640 1 -- 172.21.15.115:0/3124370724 >> 172.21.15.115:0/3124370724 conn(0x7f147006a0e0 msgr2=0x7f147006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.818+0000 7f1474dfb640 1 -- 172.21.15.115:0/3124370724 shutdown_connections 2023-12-01T01:51:05.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.818+0000 7f1474dfb640 1 -- 172.21.15.115:0/3124370724 wait complete. 2023-12-01T01:51:05.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.820+0000 7f1474dfb640 1 Processor -- start 2023-12-01T01:51:05.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.820+0000 7f1474dfb640 1 -- start start 2023-12-01T01:51:05.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.821+0000 7f1474dfb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14700b5d80 0x7f14700b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.821+0000 7f1474dfb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 0x7f14701f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.821+0000 7f1474dfb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f147015cd20 0x7f14701f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.821+0000 7f1474dfb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1470160320 con 0x7f14700d1230 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.821+0000 7f1474dfb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14701f32c0 con 0x7f14700b5d80 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.821+0000 7f1474dfb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147015f710 con 0x7f147015cd20 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.821+0000 7f1467fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 0x7f14701f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.821+0000 7f1467fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 0x7f14701f59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52734/0 (socket says 172.21.15.115:52734) 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.821+0000 7f1467fff640 1 -- 172.21.15.115:0/2460737279 learned_addr learned my addr 172.21.15.115:0/2460737279 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1467fff640 1 -- 172.21.15.115:0/2460737279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f147015cd20 msgr2=0x7f14701f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f146e59c640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14700b5d80 0x7f14700b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1467fff640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f147015cd20 0x7f14701f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1467fff640 1 -- 172.21.15.115:0/2460737279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14700b5d80 msgr2=0x7f14700b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1467fff640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14700b5d80 0x7f14700b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1467fff640 1 -- 172.21.15.115:0/2460737279 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1468009a70 con 0x7f14700d1230 2023-12-01T01:51:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1467fff640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 0x7f14701f59c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f145800b4a0 tx=0x7f145800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:05.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1466ffd640 1 -- 172.21.15.115:0/2460737279 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458010610 con 0x7f14700d1230 2023-12-01T01:51:05.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1466ffd640 1 -- 172.21.15.115:0/2460737279 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1458010770 con 0x7f14700d1230 2023-12-01T01:51:05.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1466ffd640 1 -- 172.21.15.115:0/2460737279 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458010b20 con 0x7f14700d1230 2023-12-01T01:51:05.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1474dfb640 1 -- 172.21.15.115:0/2460737279 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147015f9f0 con 0x7f14700d1230 2023-12-01T01:51:05.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.822+0000 7f1474dfb640 1 -- 172.21.15.115:0/2460737279 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147015ffb0 con 0x7f14700d1230 2023-12-01T01:51:05.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.824+0000 7f1474dfb640 1 -- 172.21.15.115:0/2460737279 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147004e890 con 0x7f14700d1230 2023-12-01T01:51:05.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.824+0000 7f1466ffd640 1 -- 172.21.15.115:0/2460737279 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f1458010c80 con 0x7f14700d1230 2023-12-01T01:51:05.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.825+0000 7f1466ffd640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f145006a550 0x7f145006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:05.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.825+0000 7f1466ffd640 1 -- 172.21.15.115:0/2460737279 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f1458088980 con 0x7f14700d1230 2023-12-01T01:51:05.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.825+0000 7f146e59c640 1 -- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f145006a550 msgr2=0x7f145006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:05.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.825+0000 7f146e59c640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f145006a550 0x7f145006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:05.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:05.834+0000 7f1466ffd640 1 -- 172.21.15.115:0/2460737279 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1458053b80 con 0x7f14700d1230 2023-12-01T01:51:05.836 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:05.834+0000 7f245b64a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:51:05.867 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:05.865+0000 7f7eabe41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:51:05.942 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:05.941+0000 7f245b64a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:05.979 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:05.977+0000 7f7eabe41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:06.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.026+0000 7f146e59c640 1 -- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f145006a550 msgr2=0x7f145006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:06.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.026+0000 7f146e59c640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f145006a550 0x7f145006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:51:06.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.037+0000 7f1474dfb640 1 -- 172.21.15.115:0/2460737279 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f147015b0e0 con 0x7f14700d1230 2023-12-01T01:51:06.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.038+0000 7f1466ffd640 1 -- 172.21.15.115:0/2460737279 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v68) v1 ==== 80+0+467 (secure 0 0 0) 0x7f145801e080 con 0x7f14700d1230 2023-12-01T01:51:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/88317 2023-12-02T01:49:24.636862+0000 2023-12-01T01:51:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/88317 2023-12-02T01:49:24.636862+0000 2023-12-01T01:51:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2910255832 2023-12-02T01:49:24.636862+0000 2023-12-01T01:51:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/844837827 2023-12-02T01:49:24.636862+0000 2023-12-01T01:51:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1628065986 2023-12-02T01:49:24.636862+0000 2023-12-01T01:51:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1021278773 2023-12-02T01:49:24.636862+0000 2023-12-01T01:51:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3831003429 2023-12-02T01:49:24.636862+0000 2023-12-01T01:51:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1351821585 2023-12-02T01:49:24.636862+0000 2023-12-01T01:51:06.041 INFO:teuthology.orchestra.run.smithi115.stderr:listed 8 entries 2023-12-01T01:51:06.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 -- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f145006a550 msgr2=0x7f145006ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f145006a550 0x7f145006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 -- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 msgr2=0x7f14701f59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 0x7f14701f59c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f145800b4a0 tx=0x7f145800b960 comp rx=0 tx=0).stop 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 -- 172.21.15.115:0/2460737279 shutdown_connections 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f145006a550 0x7f145006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f147015cd20 0x7f14701f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14700d1230 0x7f14701f59c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 --2- 172.21.15.115:0/2460737279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14700b5d80 0x7f14700b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 -- 172.21.15.115:0/2460737279 >> 172.21.15.115:0/2460737279 conn(0x7f147006a0e0 msgr2=0x7f1470157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 -- 172.21.15.115:0/2460737279 shutdown_connections 2023-12-01T01:51:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.041+0000 7f1474dfb640 1 -- 172.21.15.115:0/2460737279 wait complete. 2023-12-01T01:51:06.049 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:06.048+0000 7f245b64a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:51:06.061 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T01:51:06.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:06.061 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T01:51:06.090 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:06.089+0000 7f7eabe41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:51:06.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:51:06.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:06.159 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T01:51:06.205 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:06.203+0000 7f245b64a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:06.249 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:51:06.249 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:06.258 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:06.253+0000 7f245b64a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:06.259 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:06.257+0000 7f7eabe41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:06.305 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:51:06.305 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:51:06.305 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:06.314 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:06.308+0000 7f7eabe41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:06.357 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:51:06.357 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:51:06.357 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:06.379 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:06.377+0000 7f245b64a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:51:06.413 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:51:06.413 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:06.433 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:06.432+0000 7f7eabe41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:51:06.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.618+0000 7f8704ba9640 1 Processor -- start 2023-12-01T01:51:06.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.619+0000 7f8704ba9640 1 -- start start 2023-12-01T01:51:06.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.620+0000 7f8704ba9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870015d7b0 0x7f870015db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:06.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.620+0000 7f8704ba9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8700160670 0x7f87000cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:06.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.620+0000 7f8704ba9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87000cd3f0 con 0x7f87000cddc0 2023-12-01T01:51:06.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.620+0000 7f8704ba9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87000cd0f0 con 0x7f8700160670 2023-12-01T01:51:06.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.620+0000 7f8704ba9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8700160ec0 con 0x7f870015d7b0 2023-12-01T01:51:06.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.620+0000 7f86f7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8700160670 0x7f87000cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:06.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.620+0000 7f86f7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8700160670 0x7f87000cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52748/0 (socket says 172.21.15.115:52748) 2023-12-01T01:51:06.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.620+0000 7f86f7fff640 1 -- 172.21.15.115:0/2541108725 learned_addr learned my addr 172.21.15.115:0/2541108725 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:06.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.620+0000 7f86fffff640 1 --1- 172.21.15.115:0/2541108725 >> v1:172.21.15.115:6789/0 conn(0x7f87000cddc0 0x7f87000d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59576/0 (socket says 172.21.15.115:59576) 2023-12-01T01:51:06.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.621+0000 7f86ff7fe640 1 --2- 172.21.15.115:0/2541108725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870015d7b0 0x7f870015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:06.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.621+0000 7f86f7fff640 1 -- 172.21.15.115:0/2541108725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870015d7b0 msgr2=0x7f870015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:06.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.621+0000 7f86f7fff640 1 --2- 172.21.15.115:0/2541108725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870015d7b0 0x7f870015db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.621+0000 7f86f7fff640 1 -- 172.21.15.115:0/2541108725 >> v1:172.21.15.115:6789/0 conn(0x7f87000cddc0 legacy=0x7f87000d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:06.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.621+0000 7f86f7fff640 1 -- 172.21.15.115:0/2541108725 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87001611a0 con 0x7f8700160670 2023-12-01T01:51:06.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.621+0000 7f86feffd640 1 -- 172.21.15.115:0/2541108725 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246512558 0 0) 0x7f87000cd3f0 con 0x7f87000cddc0 2023-12-01T01:51:06.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.621+0000 7f86ff7fe640 1 --2- 172.21.15.115:0/2541108725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870015d7b0 0x7f870015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:51:06.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.622+0000 7f86f7fff640 1 --2- 172.21.15.115:0/2541108725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8700160670 0x7f87000cd880 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009870 tx=0x7f86e802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ae649f4fe97707 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:06.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f86feffd640 1 -- 172.21.15.115:0/2541108725 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e802f840 con 0x7f8700160670 2023-12-01T01:51:06.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f86feffd640 1 -- 172.21.15.115:0/2541108725 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e802f9a0 con 0x7f8700160670 2023-12-01T01:51:06.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f86feffd640 1 -- 172.21.15.115:0/2541108725 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e802fd30 con 0x7f8700160670 2023-12-01T01:51:06.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f8704ba9640 1 -- 172.21.15.115:0/2541108725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8700160670 msgr2=0x7f87000cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:06.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f8704ba9640 1 --2- 172.21.15.115:0/2541108725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8700160670 0x7f87000cd880 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009870 tx=0x7f86e802ba20 comp rx=0 tx=0).stop 2023-12-01T01:51:06.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f8704ba9640 1 -- 172.21.15.115:0/2541108725 shutdown_connections 2023-12-01T01:51:06.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f8704ba9640 1 --2- 172.21.15.115:0/2541108725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8700160670 0x7f87000cd880 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f8704ba9640 1 --2- 172.21.15.115:0/2541108725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870015d7b0 0x7f870015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f8704ba9640 1 -- 172.21.15.115:0/2541108725 >> 172.21.15.115:0/2541108725 conn(0x7f870006a0e0 msgr2=0x7f870006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:06.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f8704ba9640 1 -- 172.21.15.115:0/2541108725 shutdown_connections 2023-12-01T01:51:06.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.623+0000 7f8704ba9640 1 -- 172.21.15.115:0/2541108725 wait complete. 2023-12-01T01:51:06.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.625+0000 7f8704ba9640 1 Processor -- start 2023-12-01T01:51:06.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.625+0000 7f8704ba9640 1 -- start start 2023-12-01T01:51:06.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.626+0000 7f8704ba9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000cddc0 0x7f870014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:06.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.626+0000 7f8704ba9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f870015d7b0 0x7f87000d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:06.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.626+0000 7f8704ba9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8700160670 0x7f87000d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:06.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.626+0000 7f8704ba9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870015a950 con 0x7f870015d7b0 2023-12-01T01:51:06.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.626+0000 7f8704ba9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000cab90 con 0x7f8700160670 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.626+0000 7f8704ba9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870015ab80 con 0x7f87000cddc0 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.626+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f870015d7b0 0x7f87000d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.626+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f870015d7b0 0x7f87000d0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52750/0 (socket says 172.21.15.115:52750) 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.626+0000 7f86f7fff640 1 -- 172.21.15.115:0/1282043086 learned_addr learned my addr 172.21.15.115:0/1282043086 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.627+0000 7f86ff7fe640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000cddc0 0x7f870014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.627+0000 7f86fffff640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8700160670 0x7f87000d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.627+0000 7f86f7fff640 1 -- 172.21.15.115:0/1282043086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000cddc0 msgr2=0x7f870014bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.627+0000 7f86f7fff640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000cddc0 0x7f870014bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.627+0000 7f86f7fff640 1 -- 172.21.15.115:0/1282043086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8700160670 msgr2=0x7f87000d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.627+0000 7f86f7fff640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8700160670 0x7f87000d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.627+0000 7f86f7fff640 1 -- 172.21.15.115:0/1282043086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80094d0 con 0x7f870015d7b0 2023-12-01T01:51:06.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.627+0000 7f86f7fff640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f870015d7b0 0x7f87000d0b70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f86e8000c00 tx=0x7f86e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:06.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.627+0000 7f86fcff9640 1 -- 172.21.15.115:0/1282043086 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e803b040 con 0x7f870015d7b0 2023-12-01T01:51:06.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.628+0000 7f86fcff9640 1 -- 172.21.15.115:0/1282043086 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e8037c40 con 0x7f870015d7b0 2023-12-01T01:51:06.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.628+0000 7f86fcff9640 1 -- 172.21.15.115:0/1282043086 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8004030 con 0x7f870015d7b0 2023-12-01T01:51:06.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.628+0000 7f8704ba9640 1 -- 172.21.15.115:0/1282043086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87000ca8b0 con 0x7f870015d7b0 2023-12-01T01:51:06.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.628+0000 7f8704ba9640 1 -- 172.21.15.115:0/1282043086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870015b100 con 0x7f870015d7b0 2023-12-01T01:51:06.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.629+0000 7f8704ba9640 1 -- 172.21.15.115:0/1282043086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87000673a0 con 0x7f870015d7b0 2023-12-01T01:51:06.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.630+0000 7f86fcff9640 1 -- 172.21.15.115:0/1282043086 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f86e80041b0 con 0x7f870015d7b0 2023-12-01T01:51:06.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.631+0000 7f86fcff9640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a550 0x7f86e006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:06.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.631+0000 7f86fcff9640 1 -- 172.21.15.115:0/1282043086 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6270+0+0 (secure 0 0 0) 0x7f86e80abbe0 con 0x7f870015d7b0 2023-12-01T01:51:06.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.631+0000 7f86ff7fe640 1 -- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a550 msgr2=0x7f86e006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:06.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.631+0000 7f86ff7fe640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a550 0x7f86e006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:06.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.635+0000 7f86fcff9640 1 -- 172.21.15.115:0/1282043086 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f86e8076d80 con 0x7f870015d7b0 2023-12-01T01:51:06.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.832+0000 7f86ff7fe640 1 -- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a550 msgr2=0x7f86e006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:06.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.832+0000 7f86ff7fe640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a550 0x7f86e006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:51:06.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.872+0000 7f8704ba9640 1 -- 172.21.15.115:0/1282043086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87000d3100 con 0x7f870015d7b0 2023-12-01T01:51:06.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.874+0000 7f86fcff9640 1 -- 172.21.15.115:0/1282043086 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f86e807a3a0 con 0x7f870015d7b0 2023-12-01T01:51:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:06.875 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/2548453767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2548453767},{"type":"v1","addr":"172.21.15.142:6834","nonce":2548453767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3727358840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3727358840},{"type":"v1","addr":"172.21.15.142:6835","nonce":3727358840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/4086473290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4086473290},{"type":"v1","addr":"172.21.15.115:6836","nonce":4086473290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:49:11.094285+0000","modified":"2023-12-01T01:49:14.112475+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1756585806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1756585806},{"type":"v1","addr":"172.21.15.115:6837","nonce":1756585806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-01T01:51:06.875 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 24 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 -- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a550 msgr2=0x7f86e006ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a550 0x7f86e006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 -- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f870015d7b0 msgr2=0x7f87000d0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f870015d7b0 0x7f87000d0b70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f86e8000c00 tx=0x7f86e80046c0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 -- 172.21.15.115:0/1282043086 shutdown_connections 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a550 0x7f86e006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8700160670 0x7f87000d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f870015d7b0 0x7f87000d0b70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 --2- 172.21.15.115:0/1282043086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000cddc0 0x7f870014bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 -- 172.21.15.115:0/1282043086 >> 172.21.15.115:0/1282043086 conn(0x7f870006a0e0 msgr2=0x7f870006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 -- 172.21.15.115:0/1282043086 shutdown_connections 2023-12-01T01:51:06.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:06.876+0000 7f8704ba9640 1 -- 172.21.15.115:0/1282043086 wait complete. 2023-12-01T01:51:06.895 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:07.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.123+0000 7fc5a2bdf640 1 Processor -- start 2023-12-01T01:51:07.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.123+0000 7fc5a2bdf640 1 -- start start 2023-12-01T01:51:07.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.124+0000 7fc5a2bdf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc59c0d05d0 0x7fc59c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.124+0000 7fc5a2bdf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59c15e020 con 0x7fc59c15d7b0 2023-12-01T01:51:07.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.124+0000 7fc5a2bdf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59c0cd3f0 con 0x7fc59c15db90 2023-12-01T01:51:07.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.124+0000 7fc5a2bdf640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59c0d0f60 con 0x7fc59c0d05d0 2023-12-01T01:51:07.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.124+0000 7fc5a1bdd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc59c15d7b0 0x7fc59c160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59588/0 (socket says 172.21.15.115:59588) 2023-12-01T01:51:07.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.124+0000 7fc5a1bdd640 1 -- 172.21.15.115:0/564014420 learned_addr learned my addr 172.21.15.115:0/564014420 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:07.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.125+0000 7fc5a23de640 1 --2- 172.21.15.115:0/564014420 >> v2:172.21.15.115:3300/0 conn(0x7fc59c0d05d0 0x7fc59c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.125+0000 7fc5a0bdb640 1 -- 172.21.15.115:0/564014420 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045816324 0 0) 0x7fc59c15e020 con 0x7fc59c15d7b0 2023-12-01T01:51:07.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.125+0000 7fc5a0bdb640 1 -- 172.21.15.115:0/564014420 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc588003520 con 0x7fc59c15d7b0 2023-12-01T01:51:07.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.125+0000 7fc5a23de640 1 -- 172.21.15.115:0/564014420 >> v1:172.21.15.142:6789/0 conn(0x7fc59c15db90 legacy=0x7fc59c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.125+0000 7fc5a23de640 1 -- 172.21.15.115:0/564014420 >> v1:172.21.15.115:6789/0 conn(0x7fc59c15d7b0 legacy=0x7fc59c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.125+0000 7fc5a23de640 1 -- 172.21.15.115:0/564014420 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c0cd0f0 con 0x7fc59c0d05d0 2023-12-01T01:51:07.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.126+0000 7fc5a23de640 1 --2- 172.21.15.115:0/564014420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c0d05d0 0x7fc59c0d0a20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc594009d90 tx=0x7fc59402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9463c673f3671d49 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:07.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.126+0000 7fc5a0bdb640 1 -- 172.21.15.115:0/564014420 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59402cd80 con 0x7fc59c0d05d0 2023-12-01T01:51:07.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.127+0000 7fc5a0bdb640 1 -- 172.21.15.115:0/564014420 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc594034440 con 0x7fc59c0d05d0 2023-12-01T01:51:07.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.127+0000 7fc5a0bdb640 1 -- 172.21.15.115:0/564014420 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5940347b0 con 0x7fc59c0d05d0 2023-12-01T01:51:07.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.127+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/564014420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c0d05d0 msgr2=0x7fc59c0d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.127+0000 7fc5a2bdf640 1 --2- 172.21.15.115:0/564014420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c0d05d0 0x7fc59c0d0a20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc594009d90 tx=0x7fc59402bfd0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.127+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/564014420 shutdown_connections 2023-12-01T01:51:07.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.127+0000 7fc5a2bdf640 1 --2- 172.21.15.115:0/564014420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c0d05d0 0x7fc59c0d0a20 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.127+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/564014420 >> 172.21.15.115:0/564014420 conn(0x7fc59c06a0e0 msgr2=0x7fc59c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:07.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.127+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/564014420 shutdown_connections 2023-12-01T01:51:07.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.127+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/564014420 wait complete. 2023-12-01T01:51:07.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.129+0000 7fc5a2bdf640 1 Processor -- start 2023-12-01T01:51:07.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.129+0000 7fc5a2bdf640 1 -- start start 2023-12-01T01:51:07.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.130+0000 7fc5a2bdf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc59c0d05d0 0x7fc59c14be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.130+0000 7fc5a2bdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c15d7b0 0x7fc59c0cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.130+0000 7fc5a2bdf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59c15db90 0x7fc59c0b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.130+0000 7fc5a2bdf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0cb110 con 0x7fc59c15d7b0 2023-12-01T01:51:07.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.130+0000 7fc5a2bdf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0cab20 con 0x7fc59c15db90 2023-12-01T01:51:07.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.130+0000 7fc5a2bdf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c15c070 con 0x7fc59c0d05d0 2023-12-01T01:51:07.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.130+0000 7fc5a13dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c15d7b0 0x7fc59c0cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.130+0000 7fc5a13dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c15d7b0 0x7fc59c0cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52764/0 (socket says 172.21.15.115:52764) 2023-12-01T01:51:07.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5a13dc640 1 -- 172.21.15.115:0/4226828246 learned_addr learned my addr 172.21.15.115:0/4226828246 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:07.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5a13dc640 1 -- 172.21.15.115:0/4226828246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc59c0d05d0 msgr2=0x7fc59c14be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5a13dc640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc59c0d05d0 0x7fc59c14be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5a23de640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59c15db90 0x7fc59c0b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5a13dc640 1 -- 172.21.15.115:0/4226828246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59c15db90 msgr2=0x7fc59c0b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5a13dc640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59c15db90 0x7fc59c0b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5a13dc640 1 -- 172.21.15.115:0/4226828246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc594009a70 con 0x7fc59c15d7b0 2023-12-01T01:51:07.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5a13dc640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c15d7b0 0x7fc59c0cb530 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00c400 tx=0x7fc58c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:07.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5927fc640 1 -- 172.21.15.115:0/4226828246 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0052d0 con 0x7fc59c15d7b0 2023-12-01T01:51:07.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5927fc640 1 -- 172.21.15.115:0/4226828246 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c005430 con 0x7fc59c15d7b0 2023-12-01T01:51:07.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.131+0000 7fc5927fc640 1 -- 172.21.15.115:0/4226828246 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c003830 con 0x7fc59c15d7b0 2023-12-01T01:51:07.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.132+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/4226828246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59c15c350 con 0x7fc59c15d7b0 2023-12-01T01:51:07.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.132+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/4226828246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59c14c390 con 0x7fc59c15d7b0 2023-12-01T01:51:07.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.133+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/4226828246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59c0b8aa0 con 0x7fc59c15d7b0 2023-12-01T01:51:07.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.134+0000 7fc5927fc640 1 -- 172.21.15.115:0/4226828246 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 88721+0+0 (secure 0 0 0) 0x7fc58c003990 con 0x7fc59c15d7b0 2023-12-01T01:51:07.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.134+0000 7fc5927fc640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc58006a580 0x7fc58006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.135+0000 7fc5927fc640 1 -- 172.21.15.115:0/4226828246 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6270+0+0 (secure 0 0 0) 0x7fc58c088280 con 0x7fc59c15d7b0 2023-12-01T01:51:07.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.135+0000 7fc5a1bdd640 1 -- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc58006a580 msgr2=0x7fc58006ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:07.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.135+0000 7fc5a1bdd640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc58006a580 0x7fc58006ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:07.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.143+0000 7fc5927fc640 1 -- 172.21.15.115:0/4226828246 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc58c053520 con 0x7fc59c15d7b0 2023-12-01T01:51:07.190 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:07.188+0000 7f245b64a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:51:07.291 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.289+0000 7f7eabe41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:51:07.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.334+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/4226828246 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc59c15c540 con 0x7fc59c15d7b0 2023-12-01T01:51:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.336+0000 7fc5a1bdd640 1 -- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc58006a580 msgr2=0x7fc58006ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.336+0000 7fc5a1bdd640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc58006a580 0x7fc58006ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:51:07.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.336+0000 7fc5927fc640 1 -- 172.21.15.115:0/4226828246 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16948 (secure 0 0 0) 0x7fc58c056b40 con 0x7fc59c15d7b0 2023-12-01T01:51:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:07.338 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":69,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:06.478255+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:49:08.909721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-01T01:49:10.108096+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.1b","mappings":[{"from":5,"to":0}]},{"pgid":"7.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:49:24.636862+0000","172.21.15.115:6822/88317":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/2910255832":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/844837827":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1628065986":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1021278773":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/3831003429":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1351821585":"2023-12-02T01:49:24.636862+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc58006a580 msgr2=0x7fc58006ca40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc58006a580 0x7fc58006ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c15d7b0 msgr2=0x7fc59c0cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c15d7b0 0x7fc59c0cb530 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00c400 tx=0x7fc58c00c8c0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/4226828246 shutdown_connections 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc58006a580 0x7fc58006ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59c15db90 0x7fc59c0b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc59c15d7b0 0x7fc59c0cb530 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 --2- 172.21.15.115:0/4226828246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc59c0d05d0 0x7fc59c14be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/4226828246 >> 172.21.15.115:0/4226828246 conn(0x7fc59c06a0e0 msgr2=0x7fc59c0b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/4226828246 shutdown_connections 2023-12-01T01:51:07.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.339+0000 7fc5a2bdf640 1 -- 172.21.15.115:0/4226828246 wait complete. 2023-12-01T01:51:07.359 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:49:08.909721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:49:10.108096+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:51:07.360 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:51:07.484 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:07.482+0000 7f245b64a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:51:07.543 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:07.541+0000 7f245b64a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:51:07.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.559+0000 7f82095d2640 1 Processor -- start 2023-12-01T01:51:07.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.559+0000 7f82095d2640 1 -- start start 2023-12-01T01:51:07.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.561+0000 7f82095d2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82040cfd60 0x7f82040cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.561+0000 7f82095d2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040ce5e0 con 0x7f82040b5fa0 2023-12-01T01:51:07.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.561+0000 7f82095d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040ce300 con 0x7f820415cd90 2023-12-01T01:51:07.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.561+0000 7f82095d2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82040d0ec0 con 0x7f82040cfd60 2023-12-01T01:51:07.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.561+0000 7f82037fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f82040b5fa0 0x7f82040d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59596/0 (socket says 172.21.15.115:59596) 2023-12-01T01:51:07.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.561+0000 7f82037fe640 1 -- 172.21.15.115:0/1884853706 learned_addr learned my addr 172.21.15.115:0/1884853706 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:07.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.561+0000 7f82027fc640 1 --2- 172.21.15.115:0/1884853706 >> v2:172.21.15.142:3300/0 conn(0x7f82040cfd60 0x7f82040cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957470799 0 0) 0x7f81fc001490 con 0x7f82040b5fa0 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ec003520 con 0x7f82040b5fa0 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2092776744 0 0) 0x7f81ec003520 con 0x7f82040b5fa0 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 >> v2:172.21.15.142:3300/0 conn(0x7f82040cfd60 msgr2=0x7f82040cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 --2- 172.21.15.115:0/1884853706 >> v2:172.21.15.142:3300/0 conn(0x7f82040cfd60 0x7f82040cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 >> v1:172.21.15.142:6789/0 conn(0x7f820415cd90 legacy=0x7f82040cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82040d10a0 con 0x7f82040b5fa0 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f81fc001b80 con 0x7f82040b5fa0 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 >> v1:172.21.15.115:6789/0 conn(0x7f82040b5fa0 legacy=0x7f82040d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81ec0039c0 0x7f81ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81ec0047c0 0x7f81ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f81ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82040d0ec0 con 0x7f81ec005180 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec00c080 con 0x7f81ec0039c0 2023-12-01T01:51:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.562+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec00c240 con 0x7f81ec0047c0 2023-12-01T01:51:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.563+0000 7f82037fe640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f81ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.563+0000 7f8202ffd640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81ec0039c0 0x7f81ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.563+0000 7f82027fc640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81ec0047c0 0x7f81ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.563+0000 7f82037fe640 1 -- 172.21.15.115:0/1884853706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81ec0047c0 msgr2=0x7f81ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.563+0000 7f82037fe640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81ec0047c0 0x7f81ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.563+0000 7f82037fe640 1 -- 172.21.15.115:0/1884853706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81ec0039c0 msgr2=0x7f81ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.563+0000 7f82037fe640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81ec0039c0 0x7f81ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.563+0000 7f82037fe640 1 -- 172.21.15.115:0/1884853706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec003520 con 0x7f81ec005180 2023-12-01T01:51:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.563+0000 7f82027fc640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81ec0047c0 0x7f81ec004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.564+0000 7f82037fe640 1 -- 172.21.15.115:0/1884853706 reap_dead start 2023-12-01T01:51:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.564+0000 7f82037fe640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f81ec00b980 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00a730 tx=0x7f81ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce5881acf008eb1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.564+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0072d0 con 0x7f81ec005180 2023-12-01T01:51:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.564+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc02f070 con 0x7f81ec005180 2023-12-01T01:51:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.564+0000 7f8201ffb640 1 -- 172.21.15.115:0/1884853706 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0076b0 con 0x7f81ec005180 2023-12-01T01:51:07.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.565+0000 7f82095d2640 1 -- 172.21.15.115:0/1884853706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 msgr2=0x7f81ec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.565+0000 7f82095d2640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f81ec00b980 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00a730 tx=0x7f81ec0057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.565+0000 7f82095d2640 1 -- 172.21.15.115:0/1884853706 shutdown_connections 2023-12-01T01:51:07.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.565+0000 7f82095d2640 1 --2- 172.21.15.115:0/1884853706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f81ec00b980 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.565+0000 7f82095d2640 1 -- 172.21.15.115:0/1884853706 >> 172.21.15.115:0/1884853706 conn(0x7f8204065960 msgr2=0x7f8204065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:07.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.565+0000 7f82095d2640 1 -- 172.21.15.115:0/1884853706 shutdown_connections 2023-12-01T01:51:07.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.565+0000 7f82095d2640 1 -- 172.21.15.115:0/1884853706 wait complete. 2023-12-01T01:51:07.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.567+0000 7f82095d2640 1 Processor -- start 2023-12-01T01:51:07.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.567+0000 7f82095d2640 1 -- start start 2023-12-01T01:51:07.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f82095d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f8204157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f82095d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8204158440 0x7f82041f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f82095d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8204157e60 0x7f82041f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f82095d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82041f68b0 con 0x7f81ec005180 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f82095d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820415d6c0 con 0x7f8204157e60 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f82095d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8204160120 con 0x7f8204158440 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f8204157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f8202ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f8204157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52790/0 (socket says 172.21.15.115:52790) 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f8202ffd640 1 -- 172.21.15.115:0/550999666 learned_addr learned my addr 172.21.15.115:0/550999666 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f82027fc640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8204158440 0x7f82041f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f82037fe640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8204157e60 0x7f82041f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.568+0000 7f8202ffd640 1 -- 172.21.15.115:0/550999666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8204158440 msgr2=0x7f82041f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f8202ffd640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8204158440 0x7f82041f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f8202ffd640 1 -- 172.21.15.115:0/550999666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8204157e60 msgr2=0x7f82041f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f8202ffd640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8204157e60 0x7f82041f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f8202ffd640 1 -- 172.21.15.115:0/550999666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204160400 con 0x7f81ec005180 2023-12-01T01:51:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f82027fc640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8204158440 0x7f82041f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f8202ffd640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f8204157680 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f81f400e410 tx=0x7f81f400e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f81eb7fe640 1 -- 172.21.15.115:0/550999666 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f40052e0 con 0x7f81ec005180 2023-12-01T01:51:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f81eb7fe640 1 -- 172.21.15.115:0/550999666 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f4005440 con 0x7f81ec005180 2023-12-01T01:51:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f81eb7fe640 1 -- 172.21.15.115:0/550999666 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4003810 con 0x7f81ec005180 2023-12-01T01:51:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f82095d2640 1 -- 172.21.15.115:0/550999666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82041606e0 con 0x7f81ec005180 2023-12-01T01:51:07.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.569+0000 7f82095d2640 1 -- 172.21.15.115:0/550999666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82041f63b0 con 0x7f81ec005180 2023-12-01T01:51:07.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.571+0000 7f82095d2640 1 -- 172.21.15.115:0/550999666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820404e890 con 0x7f81ec005180 2023-12-01T01:51:07.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.571+0000 7f81eb7fe640 1 -- 172.21.15.115:0/550999666 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 28) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f81f4003970 con 0x7f81ec005180 2023-12-01T01:51:07.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.572+0000 7f81eb7fe640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a580 0x7f81f006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.572+0000 7f81eb7fe640 1 -- 172.21.15.115:0/550999666 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6270+0+0 (secure 0 0 0) 0x7f81f4088420 con 0x7f81ec005180 2023-12-01T01:51:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.572+0000 7f82027fc640 1 -- 172.21.15.115:0/550999666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a580 msgr2=0x7f81f006ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:07.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.572+0000 7f82027fc640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a580 0x7f81f006ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:07.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.576+0000 7f81eb7fe640 1 -- 172.21.15.115:0/550999666 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f81f4053780 con 0x7f81ec005180 2023-12-01T01:51:07.621 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.619+0000 7f7eabe41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:51:07.657 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:07.656+0000 7f245b64a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:51:07.680 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.678+0000 7f7eabe41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:51:07.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.765+0000 7f82095d2640 1 -- 172.21.15.115:0/550999666 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82040d0a70 con 0x7f81ec005180 2023-12-01T01:51:07.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.766+0000 7f81eb7fe640 1 -- 172.21.15.115:0/550999666 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f81f4056da0 con 0x7f81ec005180 2023-12-01T01:51:07.768 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:51:07.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 -- 172.21.15.115:0/550999666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a580 msgr2=0x7f81f006ca40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:07.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a580 0x7f81f006ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 -- 172.21.15.115:0/550999666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 msgr2=0x7f8204157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f8204157680 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f81f400e410 tx=0x7f81f400e8d0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 -- 172.21.15.115:0/550999666 shutdown_connections 2023-12-01T01:51:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a580 0x7f81f006ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8204158440 0x7f82041f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8204157e60 0x7f82041f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 --2- 172.21.15.115:0/550999666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81ec005180 0x7f8204157680 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 -- 172.21.15.115:0/550999666 >> 172.21.15.115:0/550999666 conn(0x7f8204065960 msgr2=0x7f820415ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 -- 172.21.15.115:0/550999666 shutdown_connections 2023-12-01T01:51:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.769+0000 7f82095d2640 1 -- 172.21.15.115:0/550999666 wait complete. 2023-12-01T01:51:07.788 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:51:07.797 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.796+0000 7f7eabe41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:51:07.877 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.875+0000 7f7e2d921640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:07.973 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.875+0000 7f7e2d921640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:07.973 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.875+0000 7f7e2d921640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:07.973 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.875+0000 7f7e2d921640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:07.973 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.875+0000 7f7e2d921640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:07.974 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.875+0000 7f7e31128640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:07.974 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.875+0000 7f7e31128640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:07.974 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.875+0000 7f7e31128640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:07.974 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.875+0000 7f7e31128640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:07.974 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:07.875+0000 7f7e31128640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.977+0000 7f80304af640 1 Processor -- start 2023-12-01T01:51:07.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.977+0000 7f80304af640 1 -- start start 2023-12-01T01:51:07.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.978+0000 7f80304af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80280ce1b0 0x7f80280cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.978+0000 7f80304af640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f80280b0e60 0x7f80280b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.978+0000 7f80304af640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80280cc9a0 con 0x7f802815ac80 2023-12-01T01:51:07.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.978+0000 7f80304af640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802815b060 con 0x7f80280b0e60 2023-12-01T01:51:07.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.978+0000 7f80304af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80280b0ce0 con 0x7f80280ce1b0 2023-12-01T01:51:07.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.979+0000 7f802e24b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f802815ac80 0x7f80280cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44564/0 (socket says 172.21.15.115:44564) 2023-12-01T01:51:07.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.979+0000 7f802e24b640 1 -- 172.21.15.115:0/748673232 learned_addr learned my addr 172.21.15.115:0/748673232 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:07.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.979+0000 7f802ea4c640 1 --2- 172.21.15.115:0/748673232 >> v2:172.21.15.115:3300/0 conn(0x7f80280b0e60 0x7f80280b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.979+0000 7f802d249640 1 -- 172.21.15.115:0/748673232 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257843659 0 0) 0x7f80280cc9a0 con 0x7f802815ac80 2023-12-01T01:51:07.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.979+0000 7f802d249640 1 -- 172.21.15.115:0/748673232 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8014003520 con 0x7f802815ac80 2023-12-01T01:51:07.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.979+0000 7f802ea4c640 1 -- 172.21.15.115:0/748673232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80280ce1b0 msgr2=0x7f80280cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:07.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.979+0000 7f802ea4c640 1 --2- 172.21.15.115:0/748673232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80280ce1b0 0x7f80280cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.979+0000 7f802ea4c640 1 -- 172.21.15.115:0/748673232 >> v1:172.21.15.115:6789/0 conn(0x7f802815ac80 legacy=0x7f80280cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.979+0000 7f802ea4c640 1 -- 172.21.15.115:0/748673232 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80280cbd90 con 0x7f80280b0e60 2023-12-01T01:51:07.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.980+0000 7f802ea4c640 1 --2- 172.21.15.115:0/748673232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 0x7f80280b07a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f80200097a0 tx=0x7f802002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cad5fb4e21e42e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:07.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.980+0000 7f802d249640 1 -- 172.21.15.115:0/748673232 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020031830 con 0x7f80280b0e60 2023-12-01T01:51:07.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.980+0000 7f802d249640 1 -- 172.21.15.115:0/748673232 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020031990 con 0x7f80280b0e60 2023-12-01T01:51:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.980+0000 7f802d249640 1 -- 172.21.15.115:0/748673232 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020031d20 con 0x7f80280b0e60 2023-12-01T01:51:07.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.981+0000 7f80304af640 1 -- 172.21.15.115:0/748673232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 msgr2=0x7f80280b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.981+0000 7f80304af640 1 --2- 172.21.15.115:0/748673232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 0x7f80280b07a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f80200097a0 tx=0x7f802002b8d0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.981+0000 7f80304af640 1 -- 172.21.15.115:0/748673232 shutdown_connections 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.981+0000 7f80304af640 1 --2- 172.21.15.115:0/748673232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 0x7f80280b07a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.981+0000 7f80304af640 1 --2- 172.21.15.115:0/748673232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80280ce1b0 0x7f80280cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.981+0000 7f80304af640 1 -- 172.21.15.115:0/748673232 >> 172.21.15.115:0/748673232 conn(0x7f8028065960 msgr2=0x7f8028065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.981+0000 7f80304af640 1 -- 172.21.15.115:0/748673232 shutdown_connections 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.981+0000 7f80304af640 1 -- 172.21.15.115:0/748673232 wait complete. 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.983+0000 7f80304af640 1 Processor -- start 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.983+0000 7f80304af640 1 -- start start 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f80304af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 0x7f80280c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f80304af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80280ce1b0 0x7f8028152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f80304af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f802815ac80 0x7f8028159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f80304af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8028143e80 con 0x7f80280b0e60 2023-12-01T01:51:07.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f80304af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80280c9e90 con 0x7f802815ac80 2023-12-01T01:51:07.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f80304af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8028157510 con 0x7f80280ce1b0 2023-12-01T01:51:07.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f802e24b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 0x7f80280c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:07.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f802e24b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 0x7f80280c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39446/0 (socket says 172.21.15.115:39446) 2023-12-01T01:51:07.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f802e24b640 1 -- 172.21.15.115:0/4022760566 learned_addr learned my addr 172.21.15.115:0/4022760566 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:07.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f802e24b640 1 -- 172.21.15.115:0/4022760566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80280ce1b0 msgr2=0x7f8028152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f802e24b640 1 --2- 172.21.15.115:0/4022760566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80280ce1b0 0x7f8028152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f802e24b640 1 -- 172.21.15.115:0/4022760566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f802815ac80 msgr2=0x7f8028159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:07.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f802e24b640 1 --2- 172.21.15.115:0/4022760566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f802815ac80 0x7f8028159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:07.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.984+0000 7f802e24b640 1 -- 172.21.15.115:0/4022760566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8020009400 con 0x7f80280b0e60 2023-12-01T01:51:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.985+0000 7f802e24b640 1 --2- 172.21.15.115:0/4022760566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 0x7f80280c9950 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f802400cbc0 tx=0x7f802400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.985+0000 7f801effd640 1 -- 172.21.15.115:0/4022760566 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802400ace0 con 0x7f80280b0e60 2023-12-01T01:51:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.985+0000 7f801effd640 1 -- 172.21.15.115:0/4022760566 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802400ae40 con 0x7f80280b0e60 2023-12-01T01:51:07.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.985+0000 7f80304af640 1 -- 172.21.15.115:0/4022760566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80281577f0 con 0x7f80280b0e60 2023-12-01T01:51:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.985+0000 7f801effd640 1 -- 172.21.15.115:0/4022760566 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024012620 con 0x7f80280b0e60 2023-12-01T01:51:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.985+0000 7f80304af640 1 -- 172.21.15.115:0/4022760566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8028157db0 con 0x7f80280b0e60 2023-12-01T01:51:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.986+0000 7f80304af640 1 -- 172.21.15.115:0/4022760566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802804e890 con 0x7f80280b0e60 2023-12-01T01:51:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.987+0000 7f801effd640 1 -- 172.21.15.115:0/4022760566 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 29) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f8024007070 con 0x7f80280b0e60 2023-12-01T01:51:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.988+0000 7f801effd640 1 -- 172.21.15.115:0/4022760566 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f80240886a0 con 0x7f80280b0e60 2023-12-01T01:51:07.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:07.992+0000 7f801effd640 1 -- 172.21.15.115:0/4022760566 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8024053dd0 con 0x7f80280b0e60 2023-12-01T01:51:08.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.165+0000 7f80304af640 1 -- 172.21.15.115:0/4022760566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f802815b600 con 0x7f80280b0e60 2023-12-01T01:51:08.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.166+0000 7f801effd640 1 -- 172.21.15.115:0/4022760566 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8024017020 con 0x7f80280b0e60 2023-12-01T01:51:08.172 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:51:08.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.168+0000 7f80304af640 1 -- 172.21.15.115:0/4022760566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 msgr2=0x7f80280c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.168+0000 7f80304af640 1 --2- 172.21.15.115:0/4022760566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 0x7f80280c9950 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f802400cbc0 tx=0x7f802400a430 comp rx=0 tx=0).stop 2023-12-01T01:51:08.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.169+0000 7f80304af640 1 -- 172.21.15.115:0/4022760566 shutdown_connections 2023-12-01T01:51:08.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.169+0000 7f80304af640 1 --2- 172.21.15.115:0/4022760566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80280ce1b0 0x7f8028152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.169+0000 7f80304af640 1 --2- 172.21.15.115:0/4022760566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f802815ac80 0x7f8028159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.169+0000 7f80304af640 1 --2- 172.21.15.115:0/4022760566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80280b0e60 0x7f80280c9950 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.169+0000 7f80304af640 1 -- 172.21.15.115:0/4022760566 >> 172.21.15.115:0/4022760566 conn(0x7f8028065960 msgr2=0x7f80280ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:08.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.169+0000 7f80304af640 1 -- 172.21.15.115:0/4022760566 shutdown_connections 2023-12-01T01:51:08.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.169+0000 7f80304af640 1 -- 172.21.15.115:0/4022760566 wait complete. 2023-12-01T01:51:08.188 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:51:08.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.447+0000 7faf7e13e640 1 Processor -- start 2023-12-01T01:51:08.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.448+0000 7faf7e13e640 1 -- start start 2023-12-01T01:51:08.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.449+0000 7faf7e13e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faf780cfd60 0x7faf780cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.449+0000 7faf7e13e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf780ce5e0 con 0x7faf780b5fa0 2023-12-01T01:51:08.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.449+0000 7faf7e13e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf780ce300 con 0x7faf7815cd90 2023-12-01T01:51:08.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.449+0000 7faf7e13e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf780d0ec0 con 0x7faf780cfd60 2023-12-01T01:51:08.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.449+0000 7faf77fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7faf780b5fa0 0x7faf780d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44576/0 (socket says 172.21.15.115:44576) 2023-12-01T01:51:08.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.449+0000 7faf77fff640 1 -- 172.21.15.115:0/2198781470 learned_addr learned my addr 172.21.15.115:0/2198781470 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:08.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf76ffd640 1 --2- 172.21.15.115:0/2198781470 >> v2:172.21.15.142:3300/0 conn(0x7faf780cfd60 0x7faf780cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426723722 0 0) 0x7faf70001110 con 0x7faf780b5fa0 2023-12-01T01:51:08.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf64003520 con 0x7faf780b5fa0 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1188451472 0 0) 0x7faf64003520 con 0x7faf780b5fa0 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 >> v2:172.21.15.142:3300/0 conn(0x7faf780cfd60 msgr2=0x7faf780cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 --2- 172.21.15.115:0/2198781470 >> v2:172.21.15.142:3300/0 conn(0x7faf780cfd60 0x7faf780cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 >> v1:172.21.15.142:6789/0 conn(0x7faf7815cd90 legacy=0x7faf780cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf780d10a0 con 0x7faf780b5fa0 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7faf70001800 con 0x7faf780b5fa0 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 >> v1:172.21.15.115:6789/0 conn(0x7faf780b5fa0 legacy=0x7faf780d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 --2- 172.21.15.115:0/2198781470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf64003970 0x7faf64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 --2- 172.21.15.115:0/2198781470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf640047c0 0x7faf64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 --2- 172.21.15.115:0/2198781470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf64005180 0x7faf6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf780d0ec0 con 0x7faf640047c0 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6400c020 con 0x7faf64003970 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.450+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6400c1e0 con 0x7faf64005180 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.451+0000 7faf76ffd640 1 --2- 172.21.15.115:0/2198781470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf640047c0 0x7faf64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.451+0000 7faf76ffd640 1 -- 172.21.15.115:0/2198781470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf64005180 msgr2=0x7faf6400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.451+0000 7faf76ffd640 1 --2- 172.21.15.115:0/2198781470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf64005180 0x7faf6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.451+0000 7faf76ffd640 1 -- 172.21.15.115:0/2198781470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf64003970 msgr2=0x7faf64003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.451+0000 7faf76ffd640 1 --2- 172.21.15.115:0/2198781470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf64003970 0x7faf64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.451+0000 7faf76ffd640 1 -- 172.21.15.115:0/2198781470 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf64003520 con 0x7faf640047c0 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.451+0000 7faf77fff640 1 -- 172.21.15.115:0/2198781470 reap_dead start 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.452+0000 7faf76ffd640 1 --2- 172.21.15.115:0/2198781470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf640047c0 0x7faf64004c40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7faf780d0170 tx=0x7faf640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1af2e54019287d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.452+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6002c660 con 0x7faf640047c0 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.452+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6002c7c0 con 0x7faf640047c0 2023-12-01T01:51:08.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.452+0000 7faf767fc640 1 -- 172.21.15.115:0/2198781470 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6002cb50 con 0x7faf640047c0 2023-12-01T01:51:08.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.453+0000 7faf7e13e640 1 -- 172.21.15.115:0/2198781470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf640047c0 msgr2=0x7faf64004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.453+0000 7faf7e13e640 1 --2- 172.21.15.115:0/2198781470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf640047c0 0x7faf64004c40 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7faf780d0170 tx=0x7faf640057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.453+0000 7faf7e13e640 1 -- 172.21.15.115:0/2198781470 shutdown_connections 2023-12-01T01:51:08.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.453+0000 7faf7e13e640 1 --2- 172.21.15.115:0/2198781470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf640047c0 0x7faf64004c40 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.453+0000 7faf7e13e640 1 -- 172.21.15.115:0/2198781470 >> 172.21.15.115:0/2198781470 conn(0x7faf78065960 msgr2=0x7faf78065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:08.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.453+0000 7faf7e13e640 1 -- 172.21.15.115:0/2198781470 shutdown_connections 2023-12-01T01:51:08.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.453+0000 7faf7e13e640 1 -- 172.21.15.115:0/2198781470 wait complete. 2023-12-01T01:51:08.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.455+0000 7faf7e13e640 1 Processor -- start 2023-12-01T01:51:08.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.455+0000 7faf7e13e640 1 -- start start 2023-12-01T01:51:08.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf7e13e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf640047c0 0x7faf780c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf7e13e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf781f2dd0 0x7faf780ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf7e13e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf780caac0 0x7faf78156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf7e13e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf780caf50 con 0x7faf780caac0 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf7e13e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7815abe0 con 0x7faf640047c0 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf7e13e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf781524b0 con 0x7faf781f2dd0 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf77fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf780caac0 0x7faf78156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf77fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf780caac0 0x7faf78156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39474/0 (socket says 172.21.15.115:39474) 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf77fff640 1 -- 172.21.15.115:0/1641021634 learned_addr learned my addr 172.21.15.115:0/1641021634 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf76ffd640 1 --2- 172.21.15.115:0/1641021634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf781f2dd0 0x7faf780ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf777fe640 1 --2- 172.21.15.115:0/1641021634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf640047c0 0x7faf780c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf77fff640 1 -- 172.21.15.115:0/1641021634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf781f2dd0 msgr2=0x7faf780ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf77fff640 1 --2- 172.21.15.115:0/1641021634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf781f2dd0 0x7faf780ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf77fff640 1 -- 172.21.15.115:0/1641021634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf640047c0 msgr2=0x7faf780c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf77fff640 1 --2- 172.21.15.115:0/1641021634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf640047c0 0x7faf780c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.456+0000 7faf77fff640 1 -- 172.21.15.115:0/1641021634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7815a2f0 con 0x7faf780caac0 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.457+0000 7faf77fff640 1 --2- 172.21.15.115:0/1641021634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf780caac0 0x7faf78156390 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faf70001370 tx=0x7faf7000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.457+0000 7faf6ffff640 1 -- 172.21.15.115:0/1641021634 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf700072d0 con 0x7faf780caac0 2023-12-01T01:51:08.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.457+0000 7faf6ffff640 1 -- 172.21.15.115:0/1641021634 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7000ced0 con 0x7faf780caac0 2023-12-01T01:51:08.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.457+0000 7faf6ffff640 1 -- 172.21.15.115:0/1641021634 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70007670 con 0x7faf780caac0 2023-12-01T01:51:08.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.457+0000 7faf7e13e640 1 -- 172.21.15.115:0/1641021634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7815a590 con 0x7faf780caac0 2023-12-01T01:51:08.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.457+0000 7faf7e13e640 1 -- 172.21.15.115:0/1641021634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf780cb160 con 0x7faf780caac0 2023-12-01T01:51:08.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.458+0000 7faf7e13e640 1 -- 172.21.15.115:0/1641021634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf78073910 con 0x7faf780caac0 2023-12-01T01:51:08.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.459+0000 7faf6ffff640 1 -- 172.21.15.115:0/1641021634 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 29) v1 ==== 88318+0+0 (secure 0 0 0) 0x7faf700047e0 con 0x7faf780caac0 2023-12-01T01:51:08.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.460+0000 7faf6ffff640 1 -- 172.21.15.115:0/1641021634 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6484+0+0 (secure 0 0 0) 0x7faf70087990 con 0x7faf780caac0 2023-12-01T01:51:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.465+0000 7faf6ffff640 1 -- 172.21.15.115:0/1641021634 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faf70053090 con 0x7faf780caac0 2023-12-01T01:51:08.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.639+0000 7faf7e13e640 1 -- 172.21.15.115:0/1641021634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf780ce5e0 con 0x7faf780caac0 2023-12-01T01:51:08.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.640+0000 7faf6ffff640 1 -- 172.21.15.115:0/1641021634 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf780ce5e0 con 0x7faf780caac0 2023-12-01T01:51:08.642 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:51:08.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.642+0000 7faf7e13e640 1 -- 172.21.15.115:0/1641021634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf780caac0 msgr2=0x7faf78156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.642+0000 7faf7e13e640 1 --2- 172.21.15.115:0/1641021634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf780caac0 0x7faf78156390 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faf70001370 tx=0x7faf7000cb90 comp rx=0 tx=0).stop 2023-12-01T01:51:08.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.643+0000 7faf7e13e640 1 -- 172.21.15.115:0/1641021634 shutdown_connections 2023-12-01T01:51:08.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.643+0000 7faf7e13e640 1 --2- 172.21.15.115:0/1641021634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf781f2dd0 0x7faf780ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.643+0000 7faf7e13e640 1 --2- 172.21.15.115:0/1641021634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf780caac0 0x7faf78156390 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.643+0000 7faf7e13e640 1 --2- 172.21.15.115:0/1641021634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf640047c0 0x7faf780c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.643+0000 7faf7e13e640 1 -- 172.21.15.115:0/1641021634 >> 172.21.15.115:0/1641021634 conn(0x7faf78065960 msgr2=0x7faf780b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:08.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.643+0000 7faf7e13e640 1 -- 172.21.15.115:0/1641021634 shutdown_connections 2023-12-01T01:51:08.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.643+0000 7faf7e13e640 1 -- 172.21.15.115:0/1641021634 wait complete. 2023-12-01T01:51:08.662 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:08.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.872+0000 7fd2d739c640 1 Processor -- start 2023-12-01T01:51:08.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.873+0000 7fd2d739c640 1 -- start start 2023-12-01T01:51:08.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.874+0000 7fd2d739c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d015e2f0 0x7fd2d00ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.874+0000 7fd2d739c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d00add10 con 0x7fd2d006f2f0 2023-12-01T01:51:08.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.874+0000 7fd2d739c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d00c5760 con 0x7fd2d00adf10 2023-12-01T01:51:08.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.874+0000 7fd2d739c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d00c9cf0 con 0x7fd2d015e2f0 2023-12-01T01:51:08.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.874+0000 7fd2d4937640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd2d006f2f0 0x7fd2d00c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44586/0 (socket says 172.21.15.115:44586) 2023-12-01T01:51:08.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.874+0000 7fd2d4937640 1 -- 172.21.15.115:0/231576658 learned_addr learned my addr 172.21.15.115:0/231576658 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:08.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2d5138640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d015e2f0 0x7fd2d00ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612882643 0 0) 0x7fd2d00add10 con 0x7fd2d006f2f0 2023-12-01T01:51:08.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2bc003520 con 0x7fd2d006f2f0 2023-12-01T01:51:08.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3781155637 0 0) 0x7fd2bc003520 con 0x7fd2d006f2f0 2023-12-01T01:51:08.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d015e2f0 msgr2=0x7fd2d00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:08.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d015e2f0 0x7fd2d00ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 >> v1:172.21.15.142:6789/0 conn(0x7fd2d00adf10 legacy=0x7fd2d00c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d015e6d0 con 0x7fd2d006f2f0 2023-12-01T01:51:08.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd2c4001670 con 0x7fd2d006f2f0 2023-12-01T01:51:08.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 >> v1:172.21.15.115:6789/0 conn(0x7fd2d006f2f0 legacy=0x7fd2d00c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2bc003a10 0x7fd2bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2bc0047c0 0x7fd2bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.875+0000 7fd2c3fff640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d015e2f0 0x7fd2bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d00c9cf0 con 0x7fd2bc003a10 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc0055c0 con 0x7fd2bc0047c0 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc00bec0 con 0x7fd2d015e2f0 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2d5138640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2bc003a10 0x7fd2bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2d4937640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2bc0047c0 0x7fd2bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2d5138640 1 -- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d015e2f0 msgr2=0x7fd2bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2d5138640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d015e2f0 0x7fd2bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2d5138640 1 -- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2bc0047c0 msgr2=0x7fd2bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2d5138640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2bc0047c0 0x7fd2bc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2d5138640 1 -- 172.21.15.115:0/231576658 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc003520 con 0x7fd2bc003a10 2023-12-01T01:51:08.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.876+0000 7fd2d4937640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2bc0047c0 0x7fd2bc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:08.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.877+0000 7fd2d5138640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2bc003a10 0x7fd2bc003e30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00ba10 tx=0x7fd2bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa356992888a662 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:08.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.878+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc032680 con 0x7fd2bc003a10 2023-12-01T01:51:08.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.878+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc032800 con 0x7fd2bc003a10 2023-12-01T01:51:08.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.878+0000 7fd2c3fff640 1 -- 172.21.15.115:0/231576658 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc032bb0 con 0x7fd2bc003a10 2023-12-01T01:51:08.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.878+0000 7fd2d739c640 1 -- 172.21.15.115:0/231576658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2bc003a10 msgr2=0x7fd2bc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.878+0000 7fd2d739c640 1 --2- 172.21.15.115:0/231576658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2bc003a10 0x7fd2bc003e30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00ba10 tx=0x7fd2bc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.878+0000 7fd2d5939640 1 -- 172.21.15.115:0/231576658 reap_dead start 2023-12-01T01:51:08.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.878+0000 7fd2d739c640 1 -- 172.21.15.115:0/231576658 shutdown_connections 2023-12-01T01:51:08.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.878+0000 7fd2d739c640 1 -- 172.21.15.115:0/231576658 >> 172.21.15.115:0/231576658 conn(0x7fd2d00ce880 msgr2=0x7fd2d00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:08.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.878+0000 7fd2d739c640 1 -- 172.21.15.115:0/231576658 shutdown_connections 2023-12-01T01:51:08.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.879+0000 7fd2d739c640 1 -- 172.21.15.115:0/231576658 wait complete. 2023-12-01T01:51:08.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.880+0000 7fd2d739c640 1 Processor -- start 2023-12-01T01:51:08.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.880+0000 7fd2d739c640 1 -- start start 2023-12-01T01:51:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.881+0000 7fd2d739c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2bc003a10 0x7fd2d01f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.881+0000 7fd2d739c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d00cb210 0x7fd2d015c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.881+0000 7fd2d739c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d00ac8a0 0x7fd2d00acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.881+0000 7fd2d739c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d00ad260 con 0x7fd2d00cb210 2023-12-01T01:51:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.881+0000 7fd2d739c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d015cc20 con 0x7fd2bc003a10 2023-12-01T01:51:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.881+0000 7fd2d739c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d01f2450 con 0x7fd2d00ac8a0 2023-12-01T01:51:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.882+0000 7fd2d4937640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d00cb210 0x7fd2d015c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.882+0000 7fd2d4937640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d00cb210 0x7fd2d015c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39490/0 (socket says 172.21.15.115:39490) 2023-12-01T01:51:08.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.882+0000 7fd2d4937640 1 -- 172.21.15.115:0/1152978778 learned_addr learned my addr 172.21.15.115:0/1152978778 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:08.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.882+0000 7fd2d4937640 1 -- 172.21.15.115:0/1152978778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d00ac8a0 msgr2=0x7fd2d00acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.882+0000 7fd2d5138640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2bc003a10 0x7fd2d01f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.882+0000 7fd2d4937640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d00ac8a0 0x7fd2d00acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.882+0000 7fd2d4937640 1 -- 172.21.15.115:0/1152978778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2bc003a10 msgr2=0x7fd2d01f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.882+0000 7fd2d4937640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2bc003a10 0x7fd2d01f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.882+0000 7fd2d4937640 1 -- 172.21.15.115:0/1152978778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d01f6a00 con 0x7fd2d00cb210 2023-12-01T01:51:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.882+0000 7fd2d4937640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d00cb210 0x7fd2d015c3d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400d7d0 tx=0x7fd2c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.883+0000 7fd2c1ffb640 1 -- 172.21.15.115:0/1152978778 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c40074e0 con 0x7fd2d00cb210 2023-12-01T01:51:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.883+0000 7fd2c1ffb640 1 -- 172.21.15.115:0/1152978778 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c4007640 con 0x7fd2d00cb210 2023-12-01T01:51:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.883+0000 7fd2c1ffb640 1 -- 172.21.15.115:0/1152978778 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4004a20 con 0x7fd2d00cb210 2023-12-01T01:51:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.883+0000 7fd2d739c640 1 -- 172.21.15.115:0/1152978778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d01f5c40 con 0x7fd2d00cb210 2023-12-01T01:51:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.883+0000 7fd2d739c640 1 -- 172.21.15.115:0/1152978778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d01f6200 con 0x7fd2d00cb210 2023-12-01T01:51:08.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.884+0000 7fd2d739c640 1 -- 172.21.15.115:0/1152978778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d006fda0 con 0x7fd2d00cb210 2023-12-01T01:51:08.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.885+0000 7fd2c1ffb640 1 -- 172.21.15.115:0/1152978778 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 30) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fd2c4004b80 con 0x7fd2d00cb210 2023-12-01T01:51:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.885+0000 7fd2c1ffb640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd2b406a280 0x7fd2b406c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.886+0000 7fd2c1ffb640 1 -- 172.21.15.115:0/1152978778 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6484+0+0 (secure 0 0 0) 0x7fd2c4088470 con 0x7fd2d00cb210 2023-12-01T01:51:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.886+0000 7fd2d5138640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd2b406a280 0x7fd2b406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:08.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.886+0000 7fd2d5138640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd2b406a280 0x7fd2b406c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00b9e0 tx=0x7fd2cc03d040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:08.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:08.890+0000 7fd2c1ffb640 1 -- 172.21.15.115:0/1152978778 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd2c4053a70 con 0x7fd2d00cb210 2023-12-01T01:51:09.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.100+0000 7fd2d739c640 1 -- 172.21.15.115:0/1152978778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2d00b43a0 con 0x7fd2d00cb210 2023-12-01T01:51:09.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.101+0000 7fd2c1ffb640 1 -- 172.21.15.115:0/1152978778 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fd2c4019090 con 0x7fd2d00cb210 2023-12-01T01:51:09.102 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:09.103 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/2548453767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2548453767},{"type":"v1","addr":"172.21.15.142:6834","nonce":2548453767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3727358840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3727358840},{"type":"v1","addr":"172.21.15.142:6835","nonce":3727358840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/4086473290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4086473290},{"type":"v1","addr":"172.21.15.115:6836","nonce":4086473290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:49:11.094285+0000","modified":"2023-12-01T01:49:14.112475+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1756585806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1756585806},{"type":"v1","addr":"172.21.15.115:6837","nonce":1756585806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-01T01:51:09.103 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 24 2023-12-01T01:51:09.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 -- 172.21.15.115:0/1152978778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd2b406a280 msgr2=0x7fd2b406c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:09.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd2b406a280 0x7fd2b406c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00b9e0 tx=0x7fd2cc03d040 comp rx=0 tx=0).stop 2023-12-01T01:51:09.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 -- 172.21.15.115:0/1152978778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d00cb210 msgr2=0x7fd2d015c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:09.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d00cb210 0x7fd2d015c3d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400d7d0 tx=0x7fd2c400dc90 comp rx=0 tx=0).stop 2023-12-01T01:51:09.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 -- 172.21.15.115:0/1152978778 shutdown_connections 2023-12-01T01:51:09.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd2b406a280 0x7fd2b406c740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2d00ac8a0 0x7fd2d00acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d00cb210 0x7fd2d015c3d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 --2- 172.21.15.115:0/1152978778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2bc003a10 0x7fd2d01f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 -- 172.21.15.115:0/1152978778 >> 172.21.15.115:0/1152978778 conn(0x7fd2d00ce880 msgr2=0x7fd2d00ca330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:09.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 -- 172.21.15.115:0/1152978778 shutdown_connections 2023-12-01T01:51:09.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.104+0000 7fd2d739c640 1 -- 172.21.15.115:0/1152978778 wait complete. 2023-12-01T01:51:09.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:09.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.324+0000 7fd7f2bca640 1 Processor -- start 2023-12-01T01:51:09.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.325+0000 7fd7f2bca640 1 -- start start 2023-12-01T01:51:09.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.326+0000 7fd7f2bca640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec1597d0 0x7fd7ec0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.326+0000 7fd7f2bca640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd7ec06b270 0x7fd7ec15db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.326+0000 7fd7f2bca640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd7ec15e070 0x7fd7ec0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.326+0000 7fd7f2bca640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ec06b9c0 con 0x7fd7ec15e070 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.326+0000 7fd7f2bca640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ec06bca0 con 0x7fd7ec06b270 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.326+0000 7fd7f2bca640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec06bf80 con 0x7fd7ec1597d0 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.327+0000 7fd7f23c9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd7ec15e070 0x7fd7ec0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.327+0000 7fd7f23c9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd7ec15e070 0x7fd7ec0adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39506/0 (socket says 172.21.15.115:39506) 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.327+0000 7fd7f23c9640 1 -- 172.21.15.115:0/3348616473 learned_addr learned my addr 172.21.15.115:0/3348616473 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.327+0000 7fd7f1bc8640 1 --2- 172.21.15.115:0/3348616473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec1597d0 0x7fd7ec0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.327+0000 7fd7f13c7640 1 --2- 172.21.15.115:0/3348616473 >> v2:172.21.15.142:3300/0 conn(0x7fd7ec06b270 0x7fd7ec15db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.327+0000 7fd7f23c9640 1 -- 172.21.15.115:0/3348616473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec1597d0 msgr2=0x7fd7ec0cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.327+0000 7fd7f23c9640 1 --2- 172.21.15.115:0/3348616473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec1597d0 0x7fd7ec0cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.327+0000 7fd7f23c9640 1 -- 172.21.15.115:0/3348616473 >> v2:172.21.15.142:3300/0 conn(0x7fd7ec06b270 msgr2=0x7fd7ec15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.327+0000 7fd7f23c9640 1 --2- 172.21.15.115:0/3348616473 >> v2:172.21.15.142:3300/0 conn(0x7fd7ec06b270 0x7fd7ec15db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.327+0000 7fd7f23c9640 1 -- 172.21.15.115:0/3348616473 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec15d610 con 0x7fd7ec15e070 2023-12-01T01:51:09.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.329+0000 7fd7f23c9640 1 --2- 172.21.15.115:0/3348616473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 0x7fd7ec0adf00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec06c370 tx=0x7fd7e402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5434305d0c038db server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:09.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.329+0000 7fd7f0bc6640 1 -- 172.21.15.115:0/3348616473 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e402ea00 con 0x7fd7ec15e070 2023-12-01T01:51:09.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.329+0000 7fd7f0bc6640 1 -- 172.21.15.115:0/3348616473 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e402eb60 con 0x7fd7ec15e070 2023-12-01T01:51:09.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.329+0000 7fd7f0bc6640 1 -- 172.21.15.115:0/3348616473 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4009600 con 0x7fd7ec15e070 2023-12-01T01:51:09.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.329+0000 7fd7f2bca640 1 -- 172.21.15.115:0/3348616473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 msgr2=0x7fd7ec0adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:09.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.330+0000 7fd7f2bca640 1 --2- 172.21.15.115:0/3348616473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 0x7fd7ec0adf00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec06c370 tx=0x7fd7e402e060 comp rx=0 tx=0).stop 2023-12-01T01:51:09.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.330+0000 7fd7f2bca640 1 -- 172.21.15.115:0/3348616473 shutdown_connections 2023-12-01T01:51:09.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.330+0000 7fd7f2bca640 1 --2- 172.21.15.115:0/3348616473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 0x7fd7ec0adf00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.330+0000 7fd7f2bca640 1 --2- 172.21.15.115:0/3348616473 >> v2:172.21.15.142:3300/0 conn(0x7fd7ec06b270 0x7fd7ec15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.330+0000 7fd7f2bca640 1 --2- 172.21.15.115:0/3348616473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec1597d0 0x7fd7ec0cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.330+0000 7fd7f2bca640 1 -- 172.21.15.115:0/3348616473 >> 172.21.15.115:0/3348616473 conn(0x7fd7ec0ce640 msgr2=0x7fd7ec0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:09.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.330+0000 7fd7f2bca640 1 -- 172.21.15.115:0/3348616473 shutdown_connections 2023-12-01T01:51:09.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.330+0000 7fd7f2bca640 1 -- 172.21.15.115:0/3348616473 wait complete. 2023-12-01T01:51:09.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.331+0000 7fd7f2bca640 1 Processor -- start 2023-12-01T01:51:09.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.332+0000 7fd7f2bca640 1 -- start start 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f2bca640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec06b270 0x7fd7ec1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f2bca640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7ec1597d0 0x7fd7ec15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f2bca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 0x7fd7ec1f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f2bca640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec153bf0 con 0x7fd7ec15e070 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f2bca640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec152c60 con 0x7fd7ec1597d0 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f2bca640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec153220 con 0x7fd7ec06b270 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f23c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 0x7fd7ec1f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f23c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 0x7fd7ec1f1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39510/0 (socket says 172.21.15.115:39510) 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f23c9640 1 -- 172.21.15.115:0/1502215351 learned_addr learned my addr 172.21.15.115:0/1502215351 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f13c7640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7ec1597d0 0x7fd7ec15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f1bc8640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec06b270 0x7fd7ec1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f23c9640 1 -- 172.21.15.115:0/1502215351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec06b270 msgr2=0x7fd7ec1536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f23c9640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec06b270 0x7fd7ec1536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f23c9640 1 -- 172.21.15.115:0/1502215351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7ec1597d0 msgr2=0x7fd7ec15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f23c9640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7ec1597d0 0x7fd7ec15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f23c9640 1 -- 172.21.15.115:0/1502215351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e400bd40 con 0x7fd7ec15e070 2023-12-01T01:51:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f13c7640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7ec1597d0 0x7fd7ec15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.333+0000 7fd7f1bc8640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec06b270 0x7fd7ec1536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:09.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.334+0000 7fd7f23c9640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 0x7fd7ec1f1a40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400d240 tx=0x7fd7e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:09.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.334+0000 7fd7e27fc640 1 -- 172.21.15.115:0/1502215351 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e402ea00 con 0x7fd7ec15e070 2023-12-01T01:51:09.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.334+0000 7fd7e27fc640 1 -- 172.21.15.115:0/1502215351 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e403f850 con 0x7fd7ec15e070 2023-12-01T01:51:09.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.334+0000 7fd7f2bca640 1 -- 172.21.15.115:0/1502215351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ec15c2d0 con 0x7fd7ec15e070 2023-12-01T01:51:09.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.334+0000 7fd7e27fc640 1 -- 172.21.15.115:0/1502215351 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e403fc00 con 0x7fd7ec15e070 2023-12-01T01:51:09.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.334+0000 7fd7f2bca640 1 -- 172.21.15.115:0/1502215351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ec152eb0 con 0x7fd7ec15e070 2023-12-01T01:51:09.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.335+0000 7fd7f2bca640 1 -- 172.21.15.115:0/1502215351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ec0673a0 con 0x7fd7ec15e070 2023-12-01T01:51:09.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.336+0000 7fd7e27fc640 1 -- 172.21.15.115:0/1502215351 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 30) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fd7e4049400 con 0x7fd7ec15e070 2023-12-01T01:51:09.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.337+0000 7fd7e27fc640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7d006a280 0x7fd7d006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.337+0000 7fd7e27fc640 1 -- 172.21.15.115:0/1502215351 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6484+0+0 (secure 0 0 0) 0x7fd7e40ac110 con 0x7fd7ec15e070 2023-12-01T01:51:09.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.337+0000 7fd7f1bc8640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7d006a280 0x7fd7d006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.338+0000 7fd7f1bc8640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7d006a280 0x7fd7d006c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8006fd0 tx=0x7fd7e8008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:09.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.341+0000 7fd7e27fc640 1 -- 172.21.15.115:0/1502215351 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd7e4077700 con 0x7fd7ec15e070 2023-12-01T01:51:09.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.517+0000 7fd7f2bca640 1 -- 172.21.15.115:0/1502215351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7ec153360 con 0x7fd7ec15e070 2023-12-01T01:51:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.519+0000 7fd7e27fc640 1 -- 172.21.15.115:0/1502215351 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17258 (secure 0 0 0) 0x7fd7e400b720 con 0x7fd7ec15e070 2023-12-01T01:51:09.521 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:09.521 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":70,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:07.799127+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:49:08.909721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-01T01:49:10.108096+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.1b","mappings":[{"from":5,"to":0}]},{"pgid":"7.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/588694958":"2023-12-02T01:51:07.799096+0000","172.21.15.115:0/2121673645":"2023-12-02T01:51:07.799096+0000","172.21.15.115:0/3856436585":"2023-12-02T01:51:07.799096+0000","172.21.15.115:0/1630933528":"2023-12-02T01:51:07.799096+0000","172.21.15.115:0/914093312":"2023-12-02T01:51:07.799096+0000","172.21.15.115:0/1675402614":"2023-12-02T01:51:07.799096+0000","172.21.15.115:6824/88317":"2023-12-02T01:49:24.636862+0000","172.21.15.115:6822/88317":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/2910255832":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/844837827":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1628065986":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1021278773":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/3831003429":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1351821585":"2023-12-02T01:49:24.636862+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:09.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 -- 172.21.15.115:0/1502215351 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7d006a280 msgr2=0x7fd7d006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7d006a280 0x7fd7d006c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8006fd0 tx=0x7fd7e8008040 comp rx=0 tx=0).stop 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 -- 172.21.15.115:0/1502215351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 msgr2=0x7fd7ec1f1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 0x7fd7ec1f1a40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400d240 tx=0x7fd7e4004a40 comp rx=0 tx=0).stop 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 -- 172.21.15.115:0/1502215351 shutdown_connections 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd7d006a280 0x7fd7d006c740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7ec15e070 0x7fd7ec1f1a40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7ec1597d0 0x7fd7ec15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 --2- 172.21.15.115:0/1502215351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7ec06b270 0x7fd7ec1536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 -- 172.21.15.115:0/1502215351 >> 172.21.15.115:0/1502215351 conn(0x7fd7ec0ce640 msgr2=0x7fd7ec0cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 -- 172.21.15.115:0/1502215351 shutdown_connections 2023-12-01T01:51:09.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.522+0000 7fd7f2bca640 1 -- 172.21.15.115:0/1502215351 wait complete. 2023-12-01T01:51:09.541 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T01:51:09.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:09.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.749+0000 7f0280e5f640 1 Processor -- start 2023-12-01T01:51:09.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.750+0000 7f0280e5f640 1 -- start start 2023-12-01T01:51:09.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.751+0000 7f0280e5f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f027c15cd20 0x7f027c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.751+0000 7f0280e5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f027c0d1230 0x7f027c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.751+0000 7f0280e5f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027c0d1b90 con 0x7f027c0b5d80 2023-12-01T01:51:09.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.751+0000 7f0280e5f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027c0b2160 con 0x7f027c15cd20 2023-12-01T01:51:09.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.751+0000 7f0280e5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c0b8c80 con 0x7f027c0d1230 2023-12-01T01:51:09.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.751+0000 7f027a59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f027c15cd20 0x7f027c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.752+0000 7f027a59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f027c15cd20 0x7f027c0d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39526/0 (socket says 172.21.15.115:39526) 2023-12-01T01:51:09.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.752+0000 7f027a59c640 1 -- 172.21.15.115:0/3945360369 learned_addr learned my addr 172.21.15.115:0/3945360369 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:09.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.752+0000 7f027ad9d640 1 --2- 172.21.15.115:0/3945360369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f027c0d1230 0x7f027c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.752+0000 7f027a59c640 1 -- 172.21.15.115:0/3945360369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f027c0d1230 msgr2=0x7f027c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:09.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.752+0000 7f027a59c640 1 --2- 172.21.15.115:0/3945360369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f027c0d1230 0x7f027c0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.752+0000 7f027a59c640 1 -- 172.21.15.115:0/3945360369 >> v1:172.21.15.142:6789/0 conn(0x7f027c0b5d80 legacy=0x7f027c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:09.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.752+0000 7f027a59c640 1 -- 172.21.15.115:0/3945360369 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c0ce2e0 con 0x7f027c15cd20 2023-12-01T01:51:09.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.753+0000 7f027a59c640 1 --2- 172.21.15.115:0/3945360369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c15cd20 0x7f027c0d24d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f026c009870 tx=0x7f026c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d50d2f855538ba6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:09.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.753+0000 7f0279d9b640 1 -- 172.21.15.115:0/3945360369 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c0317c0 con 0x7f027c15cd20 2023-12-01T01:51:09.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.753+0000 7f0279d9b640 1 -- 172.21.15.115:0/3945360369 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c031920 con 0x7f027c15cd20 2023-12-01T01:51:09.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.753+0000 7f0279d9b640 1 -- 172.21.15.115:0/3945360369 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c031cb0 con 0x7f027c15cd20 2023-12-01T01:51:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.754+0000 7f0280e5f640 1 -- 172.21.15.115:0/3945360369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c15cd20 msgr2=0x7f027c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.754+0000 7f0280e5f640 1 --2- 172.21.15.115:0/3945360369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c15cd20 0x7f027c0d24d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f026c009870 tx=0x7f026c02b860 comp rx=0 tx=0).stop 2023-12-01T01:51:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.754+0000 7f0280e5f640 1 -- 172.21.15.115:0/3945360369 shutdown_connections 2023-12-01T01:51:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.754+0000 7f0280e5f640 1 --2- 172.21.15.115:0/3945360369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f027c0d1230 0x7f027c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.754+0000 7f0280e5f640 1 --2- 172.21.15.115:0/3945360369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c15cd20 0x7f027c0d24d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.754+0000 7f0280e5f640 1 -- 172.21.15.115:0/3945360369 >> 172.21.15.115:0/3945360369 conn(0x7f027c06a0e0 msgr2=0x7f027c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.754+0000 7f0280e5f640 1 -- 172.21.15.115:0/3945360369 shutdown_connections 2023-12-01T01:51:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.754+0000 7f0280e5f640 1 -- 172.21.15.115:0/3945360369 wait complete. 2023-12-01T01:51:09.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.756+0000 7f0280e5f640 1 Processor -- start 2023-12-01T01:51:09.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.756+0000 7f0280e5f640 1 -- start start 2023-12-01T01:51:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f0280e5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c0b5d80 0x7f027c153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f0280e5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f027c0d1230 0x7f027c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f0280e5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f027c15cd20 0x7f027c0cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f0280e5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c1f6920 con 0x7f027c0b5d80 2023-12-01T01:51:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f0280e5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c1f5a50 con 0x7f027c0d1230 2023-12-01T01:51:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f0280e5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c156780 con 0x7f027c15cd20 2023-12-01T01:51:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f027a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c0b5d80 0x7f027c153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f027a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c0b5d80 0x7f027c153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39528/0 (socket says 172.21.15.115:39528) 2023-12-01T01:51:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f027a59c640 1 -- 172.21.15.115:0/1083454740 learned_addr learned my addr 172.21.15.115:0/1083454740 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f027ad9d640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f027c15cd20 0x7f027c0cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.757+0000 7f0273fff640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f027c0d1230 0x7f027c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f027a59c640 1 -- 172.21.15.115:0/1083454740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f027c15cd20 msgr2=0x7f027c0cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f027a59c640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f027c15cd20 0x7f027c0cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f027a59c640 1 -- 172.21.15.115:0/1083454740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f027c0d1230 msgr2=0x7f027c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f027a59c640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f027c0d1230 0x7f027c0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f027a59c640 1 -- 172.21.15.115:0/1083454740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c0094d0 con 0x7f027c0b5d80 2023-12-01T01:51:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f027a59c640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c0b5d80 0x7f027c153a80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f026c02b8a0 tx=0x7f026c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f0272ffd640 1 -- 172.21.15.115:0/1083454740 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c032860 con 0x7f027c0b5d80 2023-12-01T01:51:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f0272ffd640 1 -- 172.21.15.115:0/1083454740 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c0329c0 con 0x7f027c0b5d80 2023-12-01T01:51:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f0280e5f640 1 -- 172.21.15.115:0/1083454740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c1f5ca0 con 0x7f027c0b5d80 2023-12-01T01:51:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f0272ffd640 1 -- 172.21.15.115:0/1083454740 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c032d70 con 0x7f027c0b5d80 2023-12-01T01:51:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.758+0000 7f0280e5f640 1 -- 172.21.15.115:0/1083454740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c1f6260 con 0x7f027c0b5d80 2023-12-01T01:51:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.760+0000 7f0280e5f640 1 -- 172.21.15.115:0/1083454740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f027c04e890 con 0x7f027c0b5d80 2023-12-01T01:51:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.760+0000 7f0272ffd640 1 -- 172.21.15.115:0/1083454740 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 30) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f026c04e450 con 0x7f027c0b5d80 2023-12-01T01:51:09.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.762+0000 7f0272ffd640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025c06a280 0x7f025c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:09.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.762+0000 7f0272ffd640 1 -- 172.21.15.115:0/1083454740 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f026c0aa9c0 con 0x7f027c0b5d80 2023-12-01T01:51:09.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.762+0000 7f0273fff640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025c06a280 0x7f025c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:09.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.763+0000 7f0273fff640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025c06a280 0x7f025c06c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0264006fd0 tx=0x7f0264008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:09.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:09.767+0000 7f0272ffd640 1 -- 172.21.15.115:0/1083454740 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f026c076050 con 0x7f027c0b5d80 2023-12-01T01:51:10.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.012+0000 7f0280e5f640 1 -- 172.21.15.115:0/1083454740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f027c0d1b90 con 0x7f027c0b5d80 2023-12-01T01:51:10.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.013+0000 7f0272ffd640 1 -- 172.21.15.115:0/1083454740 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f027c0d1b90 con 0x7f027c0b5d80 2023-12-01T01:51:10.015 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:10.015 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/2548453767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2548453767},{"type":"v1","addr":"172.21.15.142:6834","nonce":2548453767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3727358840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3727358840},{"type":"v1","addr":"172.21.15.142:6835","nonce":3727358840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/4086473290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4086473290},{"type":"v1","addr":"172.21.15.115:6836","nonce":4086473290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:49:11.094285+0000","modified":"2023-12-01T01:49:14.112475+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1756585806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1756585806},{"type":"v1","addr":"172.21.15.115:6837","nonce":1756585806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-01T01:51:10.015 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 24 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.016+0000 7f0280e5f640 1 -- 172.21.15.115:0/1083454740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025c06a280 msgr2=0x7f025c06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.016+0000 7f0280e5f640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025c06a280 0x7f025c06c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0264006fd0 tx=0x7f0264008040 comp rx=0 tx=0).stop 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.016+0000 7f0280e5f640 1 -- 172.21.15.115:0/1083454740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c0b5d80 msgr2=0x7f027c153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.016+0000 7f0280e5f640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c0b5d80 0x7f027c153a80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f026c02b8a0 tx=0x7f026c0046c0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.016+0000 7f0280e5f640 1 -- 172.21.15.115:0/1083454740 shutdown_connections 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.016+0000 7f0280e5f640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f025c06a280 0x7f025c06c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.016+0000 7f0280e5f640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f027c15cd20 0x7f027c0cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.016+0000 7f0280e5f640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f027c0d1230 0x7f027c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.016+0000 7f0280e5f640 1 --2- 172.21.15.115:0/1083454740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f027c0b5d80 0x7f027c153a80 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.016+0000 7f0280e5f640 1 -- 172.21.15.115:0/1083454740 >> 172.21.15.115:0/1083454740 conn(0x7f027c06a0e0 msgr2=0x7f027c158410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.017+0000 7f0280e5f640 1 -- 172.21.15.115:0/1083454740 shutdown_connections 2023-12-01T01:51:10.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.017+0000 7f0280e5f640 1 -- 172.21.15.115:0/1083454740 wait complete. 2023-12-01T01:51:10.036 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.242+0000 7fa6ae35c640 1 Processor -- start 2023-12-01T01:51:10.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.243+0000 7fa6ae35c640 1 -- start start 2023-12-01T01:51:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.244+0000 7fa6ae35c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa6a815cd20 0x7fa6a80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.244+0000 7fa6ae35c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a80b5d80 0x7fa6a80b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.244+0000 7fa6ae35c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a80ce2e0 con 0x7fa6a80d1230 2023-12-01T01:51:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.244+0000 7fa6ae35c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a80b2160 con 0x7fa6a815cd20 2023-12-01T01:51:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.244+0000 7fa6ae35c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a80d2a10 con 0x7fa6a80b5d80 2023-12-01T01:51:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.244+0000 7fa6ac8f9640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa6a80d1230 0x7fa6a80d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44598/0 (socket says 172.21.15.115:44598) 2023-12-01T01:51:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.244+0000 7fa6ac8f9640 1 -- 172.21.15.115:0/2855115311 learned_addr learned my addr 172.21.15.115:0/2855115311 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a7fff640 1 --2- 172.21.15.115:0/2855115311 >> v2:172.21.15.142:3300/0 conn(0x7fa6a815cd20 0x7fa6a80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa69ffff640 1 --2- 172.21.15.115:0/2855115311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a80b5d80 0x7fa6a80b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778869621 0 0) 0x7fa6a80ce2e0 con 0x7fa6a80d1230 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa694003520 con 0x7fa6a80d1230 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3004397123 0 0) 0x7fa694003520 con 0x7fa6a80d1230 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a80b5d80 msgr2=0x7fa6a80b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 --2- 172.21.15.115:0/2855115311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a80b5d80 0x7fa6a80b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 >> v2:172.21.15.142:3300/0 conn(0x7fa6a815cd20 msgr2=0x7fa6a80d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 --2- 172.21.15.115:0/2855115311 >> v2:172.21.15.142:3300/0 conn(0x7fa6a815cd20 0x7fa6a80d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a80ce000 con 0x7fa6a80d1230 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa6a00019f0 con 0x7fa6a80d1230 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 >> v1:172.21.15.115:6789/0 conn(0x7fa6a80d1230 legacy=0x7fa6a80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 --2- 172.21.15.115:0/2855115311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a80b5d80 0x7fa694003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 --2- 172.21.15.115:0/2855115311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 0x7fa6940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 --2- 172.21.15.115:0/2855115311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa694004d00 0x7fa694005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a80d2a10 con 0x7fa694003f90 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69400b9e0 con 0x7fa694004d00 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.245+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69400bba0 con 0x7fa6a80b5d80 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.246+0000 7fa69ffff640 1 --2- 172.21.15.115:0/2855115311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 0x7fa6940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.246+0000 7fa69ffff640 1 -- 172.21.15.115:0/2855115311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a80b5d80 msgr2=0x7fa694003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.246+0000 7fa69ffff640 1 --2- 172.21.15.115:0/2855115311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a80b5d80 0x7fa694003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.246+0000 7fa69ffff640 1 -- 172.21.15.115:0/2855115311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa694004d00 msgr2=0x7fa694005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.246+0000 7fa69ffff640 1 --2- 172.21.15.115:0/2855115311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa694004d00 0x7fa694005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.246+0000 7fa69ffff640 1 -- 172.21.15.115:0/2855115311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa694003520 con 0x7fa694003f90 2023-12-01T01:51:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.247+0000 7fa69ffff640 1 --2- 172.21.15.115:0/2855115311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 0x7fa6940047c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa690009a90 tx=0x7fa6940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4c9bae15d2d0de server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:10.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.247+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69002c6f0 con 0x7fa694003f90 2023-12-01T01:51:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.248+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69002c850 con 0x7fa694003f90 2023-12-01T01:51:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.248+0000 7fa6a77fe640 1 -- 172.21.15.115:0/2855115311 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69002cbe0 con 0x7fa694003f90 2023-12-01T01:51:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.248+0000 7fa6ae35c640 1 -- 172.21.15.115:0/2855115311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 msgr2=0x7fa6940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.248+0000 7fa6ae35c640 1 --2- 172.21.15.115:0/2855115311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 0x7fa6940047c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa690009a90 tx=0x7fa6940057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.248+0000 7fa6ac8f9640 1 -- 172.21.15.115:0/2855115311 reap_dead start 2023-12-01T01:51:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.248+0000 7fa6ae35c640 1 -- 172.21.15.115:0/2855115311 shutdown_connections 2023-12-01T01:51:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.248+0000 7fa6ae35c640 1 -- 172.21.15.115:0/2855115311 >> 172.21.15.115:0/2855115311 conn(0x7fa6a806a0e0 msgr2=0x7fa6a806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.248+0000 7fa6ae35c640 1 -- 172.21.15.115:0/2855115311 shutdown_connections 2023-12-01T01:51:10.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.248+0000 7fa6ae35c640 1 -- 172.21.15.115:0/2855115311 wait complete. 2023-12-01T01:51:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.250+0000 7fa6ae35c640 1 Processor -- start 2023-12-01T01:51:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.250+0000 7fa6ae35c640 1 -- start start 2023-12-01T01:51:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.251+0000 7fa6ae35c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 0x7fa6a81f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.251+0000 7fa6ae35c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6a81f28e0 0x7fa6a80b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.251+0000 7fa6ae35c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a814ec60 0x7fa6a80b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.251+0000 7fa6ae35c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a80c6c50 con 0x7fa694003f90 2023-12-01T01:51:10.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.251+0000 7fa6ae35c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a815f080 con 0x7fa6a81f28e0 2023-12-01T01:51:10.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.251+0000 7fa6ae35c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a815f360 con 0x7fa6a814ec60 2023-12-01T01:51:10.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.251+0000 7fa6a7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 0x7fa6a81f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.251+0000 7fa6a7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 0x7fa6a81f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39570/0 (socket says 172.21.15.115:39570) 2023-12-01T01:51:10.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.251+0000 7fa6a7fff640 1 -- 172.21.15.115:0/1919105172 learned_addr learned my addr 172.21.15.115:0/1919105172 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6a7fff640 1 -- 172.21.15.115:0/1919105172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a814ec60 msgr2=0x7fa6a80b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6a7fff640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a814ec60 0x7fa6a80b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6a7fff640 1 -- 172.21.15.115:0/1919105172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6a81f28e0 msgr2=0x7fa6a80b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6a7fff640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6a81f28e0 0x7fa6a80b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6a7fff640 1 -- 172.21.15.115:0/1919105172 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a80c6e60 con 0x7fa694003f90 2023-12-01T01:51:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6a7fff640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 0x7fa6a81f23a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa69800e860 tx=0x7fa69800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6a57fa640 1 -- 172.21.15.115:0/1919105172 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698009600 con 0x7fa694003f90 2023-12-01T01:51:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6a57fa640 1 -- 172.21.15.115:0/1919105172 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69800c940 con 0x7fa694003f90 2023-12-01T01:51:10.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6a57fa640 1 -- 172.21.15.115:0/1919105172 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69800ccd0 con 0x7fa694003f90 2023-12-01T01:51:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6ae35c640 1 -- 172.21.15.115:0/1919105172 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a80c7140 con 0x7fa694003f90 2023-12-01T01:51:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.252+0000 7fa6ae35c640 1 -- 172.21.15.115:0/1919105172 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a80c7840 con 0x7fa694003f90 2023-12-01T01:51:10.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.254+0000 7fa6ae35c640 1 -- 172.21.15.115:0/1919105172 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a815f520 con 0x7fa694003f90 2023-12-01T01:51:10.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.255+0000 7fa6a57fa640 1 -- 172.21.15.115:0/1919105172 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 30) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fa698013470 con 0x7fa694003f90 2023-12-01T01:51:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.256+0000 7fa6a57fa640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa68806a2b0 0x7fa68806c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.256+0000 7fa6a57fa640 1 -- 172.21.15.115:0/1919105172 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6484+0+0 (secure 0 0 0) 0x7fa698088a80 con 0x7fa694003f90 2023-12-01T01:51:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.256+0000 7fa69ffff640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa68806a2b0 0x7fa68806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.256+0000 7fa69ffff640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa68806a2b0 0x7fa68806c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa690036000 tx=0x7fa69002e000 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:10.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.260+0000 7fa6a57fa640 1 -- 172.21.15.115:0/1919105172 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa698057b30 con 0x7fa694003f90 2023-12-01T01:51:10.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.493+0000 7fa6ae35c640 1 -- 172.21.15.115:0/1919105172 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6a804e890 con 0x7fa694003f90 2023-12-01T01:51:10.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.494+0000 7fa6a57fa640 1 -- 172.21.15.115:0/1919105172 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fa6980576c0 con 0x7fa694003f90 2023-12-01T01:51:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:10.496 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/2548453767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2548453767},{"type":"v1","addr":"172.21.15.142:6834","nonce":2548453767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3727358840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3727358840},{"type":"v1","addr":"172.21.15.142:6835","nonce":3727358840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/4086473290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4086473290},{"type":"v1","addr":"172.21.15.115:6836","nonce":4086473290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:49:11.094285+0000","modified":"2023-12-01T01:49:14.112475+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1756585806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1756585806},{"type":"v1","addr":"172.21.15.115:6837","nonce":1756585806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-01T01:51:10.496 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 24 2023-12-01T01:51:10.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 -- 172.21.15.115:0/1919105172 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa68806a2b0 msgr2=0x7fa68806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa68806a2b0 0x7fa68806c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa690036000 tx=0x7fa69002e000 comp rx=0 tx=0).stop 2023-12-01T01:51:10.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 -- 172.21.15.115:0/1919105172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 msgr2=0x7fa6a81f23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 0x7fa6a81f23a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa69800e860 tx=0x7fa69800ed20 comp rx=0 tx=0).stop 2023-12-01T01:51:10.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 -- 172.21.15.115:0/1919105172 shutdown_connections 2023-12-01T01:51:10.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa68806a2b0 0x7fa68806c770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6a814ec60 0x7fa6a80b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6a81f28e0 0x7fa6a80b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 --2- 172.21.15.115:0/1919105172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa694003f90 0x7fa6a81f23a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 -- 172.21.15.115:0/1919105172 >> 172.21.15.115:0/1919105172 conn(0x7fa6a806a0e0 msgr2=0x7fa6a80cad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:10.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 -- 172.21.15.115:0/1919105172 shutdown_connections 2023-12-01T01:51:10.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.497+0000 7fa6ae35c640 1 -- 172.21.15.115:0/1919105172 wait complete. 2023-12-01T01:51:10.516 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:10.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.711+0000 7f370c096640 1 Processor -- start 2023-12-01T01:51:10.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.711+0000 7f370c096640 1 -- start start 2023-12-01T01:51:10.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.712+0000 7f370c096640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3704159dd0 0x7f37040c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.712+0000 7f370c096640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f370406b6f0 0x7f37040c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.712+0000 7f370c096640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370406f8b0 con 0x7f37040ade80 2023-12-01T01:51:10.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.712+0000 7f370c096640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37040adcb0 con 0x7f370406b6f0 2023-12-01T01:51:10.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.712+0000 7f370c096640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37040ad9d0 con 0x7f3704159dd0 2023-12-01T01:51:10.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.713+0000 7f370a633640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f37040ade80 0x7f37040cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44608/0 (socket says 172.21.15.115:44608) 2023-12-01T01:51:10.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.713+0000 7f370a633640 1 -- 172.21.15.115:0/2084924587 learned_addr learned my addr 172.21.15.115:0/2084924587 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.713+0000 7f3709e32640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3704159dd0 0x7f37040c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.713+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612039128 0 0) 0x7f370406f8b0 con 0x7f37040ade80 2023-12-01T01:51:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.713+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36f0003520 con 0x7f37040ade80 2023-12-01T01:51:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.713+0000 7f3709631640 1 --2- 172.21.15.115:0/2084924587 >> v2:172.21.15.142:3300/0 conn(0x7f370406b6f0 0x7f37040c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.713+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 347505761 0 0) 0x7f36f0003520 con 0x7f37040ade80 2023-12-01T01:51:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3704159dd0 msgr2=0x7f37040c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3704159dd0 0x7f37040c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 >> v2:172.21.15.142:3300/0 conn(0x7f370406b6f0 msgr2=0x7f37040c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 --2- 172.21.15.115:0/2084924587 >> v2:172.21.15.142:3300/0 conn(0x7f370406b6f0 0x7f37040c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37040c9000 con 0x7f37040ade80 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f36fc0019f0 con 0x7f37040ade80 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3709631640 1 --2- 172.21.15.115:0/2084924587 >> v2:172.21.15.142:3300/0 conn(0x7f370406b6f0 0x7f37040c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 >> v1:172.21.15.115:6789/0 conn(0x7f37040ade80 legacy=0x7f37040cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36f00039e0 0x7f36f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36f00047c0 0x7f36f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f00051d0 0x7f36f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040adcb0 con 0x7f36f00039e0 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040ad9d0 con 0x7f36f00051d0 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f000c150 con 0x7f36f00047c0 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3709e32640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3704159dd0 0x7f37040c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3709e32640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36f00039e0 0x7f36f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f370a633640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f00051d0 0x7f36f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3709e32640 1 -- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36f00047c0 msgr2=0x7f36f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3709e32640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36f00047c0 0x7f36f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3709e32640 1 -- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f00051d0 msgr2=0x7f36f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:10.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3709e32640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36f00051d0 0x7f36f000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.714+0000 7f3709e32640 1 -- 172.21.15.115:0/2084924587 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f0003520 con 0x7f36f00039e0 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.715+0000 7f3709e32640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36f00039e0 0x7f36f0003e00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3700006fd0 tx=0x7f36f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f6bbe9517b2527 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.716+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700033600 con 0x7f36f00039e0 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.716+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370002f050 con 0x7f36f00039e0 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.716+0000 7f3708e30640 1 -- 172.21.15.115:0/2084924587 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700033980 con 0x7f36f00039e0 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.716+0000 7f370c096640 1 -- 172.21.15.115:0/2084924587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36f00039e0 msgr2=0x7f36f0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.716+0000 7f370c096640 1 --2- 172.21.15.115:0/2084924587 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36f00039e0 0x7f36f0003e00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3700006fd0 tx=0x7f36f00057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.716+0000 7f370a633640 1 -- 172.21.15.115:0/2084924587 reap_dead start 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.717+0000 7f370c096640 1 -- 172.21.15.115:0/2084924587 shutdown_connections 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.717+0000 7f370c096640 1 -- 172.21.15.115:0/2084924587 >> 172.21.15.115:0/2084924587 conn(0x7f37040ce880 msgr2=0x7f37040cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.717+0000 7f370c096640 1 -- 172.21.15.115:0/2084924587 shutdown_connections 2023-12-01T01:51:10.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.717+0000 7f370c096640 1 -- 172.21.15.115:0/2084924587 wait complete. 2023-12-01T01:51:10.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.718+0000 7f370c096640 1 Processor -- start 2023-12-01T01:51:10.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.718+0000 7f370c096640 1 -- start start 2023-12-01T01:51:10.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.719+0000 7f370c096640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36f00039e0 0x7f37040c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.719+0000 7f370c096640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37040c3310 0x7f370414e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.719+0000 7f370c096640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f370414e5b0 0x7f370415c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.719+0000 7f370c096640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370414ea30 con 0x7f37040c3310 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.719+0000 7f370c096640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370414d060 con 0x7f370414e5b0 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.719+0000 7f370c096640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370414dbe0 con 0x7f36f00039e0 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.720+0000 7f3709631640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37040c3310 0x7f370414e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.720+0000 7f3709631640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37040c3310 0x7f370414e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39600/0 (socket says 172.21.15.115:39600) 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.720+0000 7f3709631640 1 -- 172.21.15.115:0/3820050541 learned_addr learned my addr 172.21.15.115:0/3820050541 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.720+0000 7f3709631640 1 -- 172.21.15.115:0/3820050541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36f00039e0 msgr2=0x7f37040c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.720+0000 7f3709631640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36f00039e0 0x7f37040c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.720+0000 7f3709631640 1 -- 172.21.15.115:0/3820050541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f370414e5b0 msgr2=0x7f370415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.720+0000 7f3709631640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f370414e5b0 0x7f370415c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.720+0000 7f3709631640 1 -- 172.21.15.115:0/3820050541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370414d2b0 con 0x7f37040c3310 2023-12-01T01:51:10.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.720+0000 7f3709631640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37040c3310 0x7f370414e070 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f36f400e7e0 tx=0x7f36f400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:10.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.720+0000 7f36fa7fc640 1 -- 172.21.15.115:0/3820050541 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4004230 con 0x7f37040c3310 2023-12-01T01:51:10.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.721+0000 7f36fa7fc640 1 -- 172.21.15.115:0/3820050541 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f400ee90 con 0x7f37040c3310 2023-12-01T01:51:10.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.721+0000 7f36fa7fc640 1 -- 172.21.15.115:0/3820050541 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400c6d0 con 0x7f37040c3310 2023-12-01T01:51:10.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.721+0000 7f370c096640 1 -- 172.21.15.115:0/3820050541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370414d590 con 0x7f37040c3310 2023-12-01T01:51:10.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.721+0000 7f370c096640 1 -- 172.21.15.115:0/3820050541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37040c4f50 con 0x7f37040c3310 2023-12-01T01:51:10.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.722+0000 7f370c096640 1 -- 172.21.15.115:0/3820050541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37040738e0 con 0x7f37040c3310 2023-12-01T01:51:10.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.723+0000 7f36fa7fc640 1 -- 172.21.15.115:0/3820050541 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 31) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f36f400c830 con 0x7f37040c3310 2023-12-01T01:51:10.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.723+0000 7f36fa7fc640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f36e806a4f0 0x7f36e806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:10.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.724+0000 7f36fa7fc640 1 -- 172.21.15.115:0/3820050541 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f36f4088910 con 0x7f37040c3310 2023-12-01T01:51:10.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.724+0000 7f3709e32640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f36e806a4f0 0x7f36e806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:10.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.725+0000 7f3709e32640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f36e806a4f0 0x7f36e806c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3700036000 tx=0x7f3700035040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:10.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.728+0000 7f36fa7fc640 1 -- 172.21.15.115:0/3820050541 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f36f4053a80 con 0x7f37040c3310 2023-12-01T01:51:10.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.927+0000 7f370c096640 1 -- 172.21.15.115:0/3820050541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f370404e890 con 0x7f37040c3310 2023-12-01T01:51:10.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.930+0000 7f36fa7fc640 1 -- 172.21.15.115:0/3820050541 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17258 (secure 0 0 0) 0x7f36f40570a0 con 0x7f37040c3310 2023-12-01T01:51:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:10.933 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":71,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:10.454787+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:49:08.909721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-01T01:49:10.108096+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.1b","mappings":[{"from":5,"to":0}]},{"pgid":"7.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/588694958":"2023-12-02T01:51:07.799096+0000","172.21.15.115:0/2121673645":"2023-12-02T01:51:07.799096+0000","172.21.15.115:0/3856436585":"2023-12-02T01:51:07.799096+0000","172.21.15.115:0/1630933528":"2023-12-02T01:51:07.799096+0000","172.21.15.115:0/914093312":"2023-12-02T01:51:07.799096+0000","172.21.15.115:0/1675402614":"2023-12-02T01:51:07.799096+0000","172.21.15.115:6824/88317":"2023-12-02T01:49:24.636862+0000","172.21.15.115:6822/88317":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/2910255832":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/844837827":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1628065986":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1021278773":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/3831003429":"2023-12-02T01:49:24.636862+0000","172.21.15.115:0/1351821585":"2023-12-02T01:49:24.636862+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":7,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:10.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 -- 172.21.15.115:0/3820050541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f36e806a4f0 msgr2=0x7f36e806c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f36e806a4f0 0x7f36e806c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3700036000 tx=0x7f3700035040 comp rx=0 tx=0).stop 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 -- 172.21.15.115:0/3820050541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37040c3310 msgr2=0x7f370414e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37040c3310 0x7f370414e070 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f36f400e7e0 tx=0x7f36f400eca0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 -- 172.21.15.115:0/3820050541 shutdown_connections 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f36e806a4f0 0x7f36e806c9b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f370414e5b0 0x7f370415c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37040c3310 0x7f370414e070 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 --2- 172.21.15.115:0/3820050541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36f00039e0 0x7f37040c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 -- 172.21.15.115:0/3820050541 >> 172.21.15.115:0/3820050541 conn(0x7f37040ce880 msgr2=0x7f37040cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 -- 172.21.15.115:0/3820050541 shutdown_connections 2023-12-01T01:51:10.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:10.933+0000 7f370c096640 1 -- 172.21.15.115:0/3820050541 wait complete. 2023-12-01T01:51:10.953 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T01:51:11.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.162+0000 7f729c5fb640 1 Processor -- start 2023-12-01T01:51:11.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.162+0000 7f729c5fb640 1 -- start start 2023-12-01T01:51:11.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.164+0000 7f729c5fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f729415d220 0x7f72940cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.164+0000 7f729c5fb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729415d600 con 0x7f7294070ac0 2023-12-01T01:51:11.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.164+0000 7f729c5fb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729406c170 con 0x7f72940ad2b0 2023-12-01T01:51:11.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.164+0000 7f729c5fb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729406cc10 con 0x7f729415d220 2023-12-01T01:51:11.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.164+0000 7f729a397640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f729415d220 0x7f72940cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.164+0000 7f729a397640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f729415d220 0x7f72940cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39608/0 (socket says 172.21.15.115:39608) 2023-12-01T01:51:11.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.164+0000 7f729a397640 1 -- 172.21.15.115:0/4178398939 learned_addr learned my addr 172.21.15.115:0/4178398939 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:11.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.164+0000 7f729a397640 1 -- 172.21.15.115:0/4178398939 >> v1:172.21.15.142:6789/0 conn(0x7f72940ad2b0 legacy=0x7f729406b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:11.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.165+0000 7f729a397640 1 -- 172.21.15.115:0/4178398939 >> v1:172.21.15.115:6789/0 conn(0x7f7294070ac0 legacy=0x7f729406ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:11.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.165+0000 7f729a397640 1 -- 172.21.15.115:0/4178398939 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729406c4b0 con 0x7f729415d220 2023-12-01T01:51:11.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.166+0000 7f729a397640 1 --2- 172.21.15.115:0/4178398939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 0x7f72940cc570 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7290009870 tx=0x7f729002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db42e2831b4235e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:11.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.166+0000 7f7299395640 1 -- 172.21.15.115:0/4178398939 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290031870 con 0x7f729415d220 2023-12-01T01:51:11.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.166+0000 7f7299395640 1 -- 172.21.15.115:0/4178398939 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72900319d0 con 0x7f729415d220 2023-12-01T01:51:11.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.166+0000 7f7299395640 1 -- 172.21.15.115:0/4178398939 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290031d60 con 0x7f729415d220 2023-12-01T01:51:11.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.167+0000 7f729c5fb640 1 -- 172.21.15.115:0/4178398939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 msgr2=0x7f72940cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:11.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.167+0000 7f729c5fb640 1 --2- 172.21.15.115:0/4178398939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 0x7f72940cc570 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7290009870 tx=0x7f729002b910 comp rx=0 tx=0).stop 2023-12-01T01:51:11.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.167+0000 7f729c5fb640 1 -- 172.21.15.115:0/4178398939 shutdown_connections 2023-12-01T01:51:11.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.167+0000 7f729c5fb640 1 --2- 172.21.15.115:0/4178398939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 0x7f72940cc570 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.167+0000 7f729c5fb640 1 -- 172.21.15.115:0/4178398939 >> 172.21.15.115:0/4178398939 conn(0x7f72940ce880 msgr2=0x7f72940cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:11.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.167+0000 7f729c5fb640 1 -- 172.21.15.115:0/4178398939 shutdown_connections 2023-12-01T01:51:11.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.167+0000 7f729c5fb640 1 -- 172.21.15.115:0/4178398939 wait complete. 2023-12-01T01:51:11.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.168+0000 7f729c5fb640 1 Processor -- start 2023-12-01T01:51:11.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.169+0000 7f729c5fb640 1 -- start start 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.169+0000 7f729c5fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7294070ac0 0x7f72941f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.169+0000 7f729c5fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72940ad2b0 0x7f72940c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729c5fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 0x7f72940ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729c5fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72941f21c0 con 0x7f729415d220 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729c5fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72940c92c0 con 0x7f7294070ac0 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729c5fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72940c95a0 con 0x7f72940ad2b0 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729ab98640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 0x7f72940ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729ab98640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 0x7f72940ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39612/0 (socket says 172.21.15.115:39612) 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729ab98640 1 -- 172.21.15.115:0/586784313 learned_addr learned my addr 172.21.15.115:0/586784313 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729a397640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7294070ac0 0x7f72941f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f7299b96640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72940ad2b0 0x7f72940c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729ab98640 1 -- 172.21.15.115:0/586784313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72940ad2b0 msgr2=0x7f72940c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:11.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729ab98640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72940ad2b0 0x7f72940c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729ab98640 1 -- 172.21.15.115:0/586784313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7294070ac0 msgr2=0x7f72941f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:11.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729ab98640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7294070ac0 0x7f72941f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.170+0000 7f729ab98640 1 -- 172.21.15.115:0/586784313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900094d0 con 0x7f729415d220 2023-12-01T01:51:11.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.171+0000 7f729ab98640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 0x7f72940ca360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c6b0 tx=0x7f728c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:11.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.171+0000 7f728affd640 1 -- 172.21.15.115:0/586784313 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c003b70 con 0x7f729415d220 2023-12-01T01:51:11.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.171+0000 7f728affd640 1 -- 172.21.15.115:0/586784313 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c003cf0 con 0x7f729415d220 2023-12-01T01:51:11.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.171+0000 7f728affd640 1 -- 172.21.15.115:0/586784313 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c003ea0 con 0x7f729415d220 2023-12-01T01:51:11.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.171+0000 7f729c5fb640 1 -- 172.21.15.115:0/586784313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72940c9880 con 0x7f729415d220 2023-12-01T01:51:11.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.171+0000 7f729c5fb640 1 -- 172.21.15.115:0/586784313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72940c9e40 con 0x7f729415d220 2023-12-01T01:51:11.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.172+0000 7f729c5fb640 1 -- 172.21.15.115:0/586784313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729404e890 con 0x7f729415d220 2023-12-01T01:51:11.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.173+0000 7f728affd640 1 -- 172.21.15.115:0/586784313 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 31) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f728c005310 con 0x7f729415d220 2023-12-01T01:51:11.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.174+0000 7f728affd640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f727806a4c0 0x7f727806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.175+0000 7f728affd640 1 -- 172.21.15.115:0/586784313 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f728c0888e0 con 0x7f729415d220 2023-12-01T01:51:11.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.175+0000 7f729a397640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f727806a4c0 0x7f727806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.176+0000 7f729a397640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f727806a4c0 0x7f727806c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7290036000 tx=0x7f729003c040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:11.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.179+0000 7f728affd640 1 -- 172.21.15.115:0/586784313 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f728c053b60 con 0x7f729415d220 2023-12-01T01:51:11.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.405+0000 7f729c5fb640 1 -- 172.21.15.115:0/586784313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f729415d600 con 0x7f729415d220 2023-12-01T01:51:11.407 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T01:51:11.406+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:51:11.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.422+0000 7f728affd640 1 -- 172.21.15.115:0/586784313 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f729415d600 con 0x7f729415d220 2023-12-01T01:51:11.423 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T01:51:11.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 -- 172.21.15.115:0/586784313 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f727806a4c0 msgr2=0x7f727806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:11.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f727806a4c0 0x7f727806c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7290036000 tx=0x7f729003c040 comp rx=0 tx=0).stop 2023-12-01T01:51:11.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 -- 172.21.15.115:0/586784313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 msgr2=0x7f72940ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:11.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 0x7f72940ca360 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c6b0 tx=0x7f728c00cb70 comp rx=0 tx=0).stop 2023-12-01T01:51:11.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 -- 172.21.15.115:0/586784313 shutdown_connections 2023-12-01T01:51:11.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f727806a4c0 0x7f727806c980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72940ad2b0 0x7f72940c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f729415d220 0x7f72940ca360 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 --2- 172.21.15.115:0/586784313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7294070ac0 0x7f72941f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 -- 172.21.15.115:0/586784313 >> 172.21.15.115:0/586784313 conn(0x7f72940ce880 msgr2=0x7f72940cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:11.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 -- 172.21.15.115:0/586784313 shutdown_connections 2023-12-01T01:51:11.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.425+0000 7f729c5fb640 1 -- 172.21.15.115:0/586784313 wait complete. 2023-12-01T01:51:11.444 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T01:51:11.488 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:51:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.634+0000 7fc7913dc640 1 Processor -- start 2023-12-01T01:51:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.635+0000 7fc7913dc640 1 -- start start 2023-12-01T01:51:11.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.636+0000 7fc7913dc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 0x7fc78c0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.636+0000 7fc7913dc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc78c0b0f80 0x7fc78c06b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.636+0000 7fc7913dc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc78c06b7a0 0x7fc78c0cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.636+0000 7fc7913dc640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78c0b35f0 con 0x7fc78c06b7a0 2023-12-01T01:51:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.636+0000 7fc7913dc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78c0cb290 con 0x7fc78c0b0f80 2023-12-01T01:51:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.636+0000 7fc7913dc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c0b0ae0 con 0x7fc78c160200 2023-12-01T01:51:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.636+0000 7fc78b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc78c06b7a0 0x7fc78c0cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.636+0000 7fc78b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc78c06b7a0 0x7fc78c0cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39652/0 (socket says 172.21.15.115:39652) 2023-12-01T01:51:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.636+0000 7fc78b7fe640 1 -- 172.21.15.115:0/1992758084 learned_addr learned my addr 172.21.15.115:0/1992758084 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.637+0000 7fc78affd640 1 --2- 172.21.15.115:0/1992758084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 0x7fc78c0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.637+0000 7fc78b7fe640 1 -- 172.21.15.115:0/1992758084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 msgr2=0x7fc78c0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.637+0000 7fc78b7fe640 1 --2- 172.21.15.115:0/1992758084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 0x7fc78c0b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.637+0000 7fc78a7fc640 1 --2- 172.21.15.115:0/1992758084 >> v2:172.21.15.142:3300/0 conn(0x7fc78c0b0f80 0x7fc78c06b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.637+0000 7fc78b7fe640 1 -- 172.21.15.115:0/1992758084 >> v2:172.21.15.142:3300/0 conn(0x7fc78c0b0f80 msgr2=0x7fc78c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.637+0000 7fc78b7fe640 1 --2- 172.21.15.115:0/1992758084 >> v2:172.21.15.142:3300/0 conn(0x7fc78c0b0f80 0x7fc78c06b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.637+0000 7fc78b7fe640 1 -- 172.21.15.115:0/1992758084 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c0b3310 con 0x7fc78c06b7a0 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.637+0000 7fc78affd640 1 --2- 172.21.15.115:0/1992758084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 0x7fc78c0b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.637+0000 7fc78a7fc640 1 --2- 172.21.15.115:0/1992758084 >> v2:172.21.15.142:3300/0 conn(0x7fc78c0b0f80 0x7fc78c06b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.638+0000 7fc78b7fe640 1 --2- 172.21.15.115:0/1992758084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 0x7fc78c0cbd30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc78c06ce60 tx=0x7fc78402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebe7f7dfe5428b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.638+0000 7fc789ffb640 1 -- 172.21.15.115:0/1992758084 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78402ca80 con 0x7fc78c06b7a0 2023-12-01T01:51:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.638+0000 7fc789ffb640 1 -- 172.21.15.115:0/1992758084 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78402cbe0 con 0x7fc78c06b7a0 2023-12-01T01:51:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.638+0000 7fc789ffb640 1 -- 172.21.15.115:0/1992758084 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7840343f0 con 0x7fc78c06b7a0 2023-12-01T01:51:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.639+0000 7fc7913dc640 1 -- 172.21.15.115:0/1992758084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 msgr2=0x7fc78c0cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.639+0000 7fc7913dc640 1 --2- 172.21.15.115:0/1992758084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 0x7fc78c0cbd30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc78c06ce60 tx=0x7fc78402bee0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.639+0000 7fc7913dc640 1 -- 172.21.15.115:0/1992758084 shutdown_connections 2023-12-01T01:51:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.639+0000 7fc7913dc640 1 --2- 172.21.15.115:0/1992758084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 0x7fc78c0cbd30 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.639+0000 7fc7913dc640 1 --2- 172.21.15.115:0/1992758084 >> v2:172.21.15.142:3300/0 conn(0x7fc78c0b0f80 0x7fc78c06b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.639+0000 7fc7913dc640 1 --2- 172.21.15.115:0/1992758084 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 0x7fc78c0b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.639+0000 7fc7913dc640 1 -- 172.21.15.115:0/1992758084 >> 172.21.15.115:0/1992758084 conn(0x7fc78c065960 msgr2=0x7fc78c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.640+0000 7fc7913dc640 1 -- 172.21.15.115:0/1992758084 shutdown_connections 2023-12-01T01:51:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.640+0000 7fc7913dc640 1 -- 172.21.15.115:0/1992758084 wait complete. 2023-12-01T01:51:11.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.642+0000 7fc7913dc640 1 Processor -- start 2023-12-01T01:51:11.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.642+0000 7fc7913dc640 1 -- start start 2023-12-01T01:51:11.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc7913dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 0x7fc78c1f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc7913dc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c0b0f80 0x7fc78c145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc7913dc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 0x7fc78c1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc7913dc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c145770 con 0x7fc78c06b7a0 2023-12-01T01:51:11.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc7913dc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c154070 con 0x7fc78c0b0f80 2023-12-01T01:51:11.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc7913dc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c154350 con 0x7fc78c160200 2023-12-01T01:51:11.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc78affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 0x7fc78c1f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc78affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 0x7fc78c1f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39654/0 (socket says 172.21.15.115:39654) 2023-12-01T01:51:11.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc78affd640 1 -- 172.21.15.115:0/3449645677 learned_addr learned my addr 172.21.15.115:0/3449645677 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:11.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc78b7fe640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 0x7fc78c1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc78a7fc640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c0b0f80 0x7fc78c145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.643+0000 7fc78affd640 1 -- 172.21.15.115:0/3449645677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 msgr2=0x7fc78c1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:11.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc78affd640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 0x7fc78c1558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc78affd640 1 -- 172.21.15.115:0/3449645677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c0b0f80 msgr2=0x7fc78c145230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:11.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc78affd640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c0b0f80 0x7fc78c145230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:11.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc78affd640 1 -- 172.21.15.115:0/3449645677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784009b30 con 0x7fc78c06b7a0 2023-12-01T01:51:11.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc78a7fc640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c0b0f80 0x7fc78c145230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:11.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc78affd640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 0x7fc78c1f2ea0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c7f0 tx=0x7fc77c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:11.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc7737fe640 1 -- 172.21.15.115:0/3449645677 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c007a30 con 0x7fc78c06b7a0 2023-12-01T01:51:11.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc7737fe640 1 -- 172.21.15.115:0/3449645677 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c007b90 con 0x7fc78c06b7a0 2023-12-01T01:51:11.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc7913dc640 1 -- 172.21.15.115:0/3449645677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78c154630 con 0x7fc78c06b7a0 2023-12-01T01:51:11.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc7737fe640 1 -- 172.21.15.115:0/3449645677 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00ce40 con 0x7fc78c06b7a0 2023-12-01T01:51:11.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.644+0000 7fc7913dc640 1 -- 172.21.15.115:0/3449645677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c154bf0 con 0x7fc78c06b7a0 2023-12-01T01:51:11.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.645+0000 7fc7913dc640 1 -- 172.21.15.115:0/3449645677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78c06b260 con 0x7fc78c06b7a0 2023-12-01T01:51:11.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.647+0000 7fc7737fe640 1 -- 172.21.15.115:0/3449645677 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 31) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fc77c0113f0 con 0x7fc78c06b7a0 2023-12-01T01:51:11.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.647+0000 7fc7737fe640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77806a4c0 0x7fc77806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:11.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.648+0000 7fc7737fe640 1 -- 172.21.15.115:0/3449645677 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6542+0+0 (secure 0 0 0) 0x7fc77c088e80 con 0x7fc78c06b7a0 2023-12-01T01:51:11.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.648+0000 7fc78a7fc640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77806a4c0 0x7fc77806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:11.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.648+0000 7fc78a7fc640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77806a4c0 0x7fc77806c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc780006fd0 tx=0x7fc780008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:11.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.652+0000 7fc7737fe640 1 -- 172.21.15.115:0/3449645677 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc77c0540c0 con 0x7fc78c06b7a0 2023-12-01T01:51:11.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:11.886+0000 7fc7913dc640 1 -- 172.21.15.115:0/3449645677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc78c0667e0 con 0x7fc78c06b7a0 2023-12-01T01:51:12.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.426+0000 7fc7737fe640 1 -- 172.21.15.115:0/3449645677 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc77c0576e0 con 0x7fc78c06b7a0 2023-12-01T01:51:12.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 -- 172.21.15.115:0/3449645677 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77806a4c0 msgr2=0x7fc77806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77806a4c0 0x7fc77806c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc780006fd0 tx=0x7fc780008040 comp rx=0 tx=0).stop 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 -- 172.21.15.115:0/3449645677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 msgr2=0x7fc78c1f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 0x7fc78c1f2ea0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c7f0 tx=0x7fc77c00cbf0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 -- 172.21.15.115:0/3449645677 shutdown_connections 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc77806a4c0 0x7fc77806c980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc78c160200 0x7fc78c1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc78c0b0f80 0x7fc78c145230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 --2- 172.21.15.115:0/3449645677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc78c06b7a0 0x7fc78c1f2ea0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 -- 172.21.15.115:0/3449645677 >> 172.21.15.115:0/3449645677 conn(0x7fc78c065960 msgr2=0x7fc78c15be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 -- 172.21.15.115:0/3449645677 shutdown_connections 2023-12-01T01:51:12.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.431+0000 7fc7913dc640 1 -- 172.21.15.115:0/3449645677 wait complete. 2023-12-01T01:51:12.456 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T01:51:12.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.710+0000 7f97d77d2640 1 Processor -- start 2023-12-01T01:51:12.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.710+0000 7f97d77d2640 1 -- start start 2023-12-01T01:51:12.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.712+0000 7f97d77d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00b4070 0x7f97d00cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:12.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.712+0000 7f97d77d2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d006c860 con 0x7f97d00cb8c0 2023-12-01T01:51:12.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.712+0000 7f97d77d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d015b050 con 0x7f97d015ac70 2023-12-01T01:51:12.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.712+0000 7f97d77d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d00ce0b0 con 0x7f97d00b4070 2023-12-01T01:51:12.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.712+0000 7f97d4d6d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f97d00cb8c0 0x7f97d00cc370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44638/0 (socket says 172.21.15.115:44638) 2023-12-01T01:51:12.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.712+0000 7f97d4d6d640 1 -- 172.21.15.115:0/1988823038 learned_addr learned my addr 172.21.15.115:0/1988823038 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:12.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.712+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960100759 0 0) 0x7f97d006c860 con 0x7f97d00cb8c0 2023-12-01T01:51:12.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.712+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97bc003520 con 0x7f97d00cb8c0 2023-12-01T01:51:12.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.712+0000 7f97d5d6f640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00b4070 0x7f97d00cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:12.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1404605874 0 0) 0x7f97bc003520 con 0x7f97d00cb8c0 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00b4070 msgr2=0x7f97d00cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00b4070 0x7f97d00cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 >> v1:172.21.15.142:6789/0 conn(0x7f97d015ac70 legacy=0x7f97d00cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d00ce390 con 0x7f97d00cb8c0 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f97c0001670 con 0x7f97d00cb8c0 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 >> v1:172.21.15.115:6789/0 conn(0x7f97d00cb8c0 legacy=0x7f97d00cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00b4070 0x7f97bc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc003fd0 0x7f97bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97bc004d00 0x7f97bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d00ce0b0 con 0x7f97bc004d00 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc00bb40 con 0x7f97bc003fd0 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc00bd00 con 0x7f97d00b4070 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97d5d6f640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97bc004d00 0x7f97bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:12.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.713+0000 7f97d556e640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00b4070 0x7f97bc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:12.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.714+0000 7f97d4d6d640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc003fd0 0x7f97bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:12.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.714+0000 7f97d5d6f640 1 -- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00b4070 msgr2=0x7f97bc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:12.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.714+0000 7f97d5d6f640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00b4070 0x7f97bc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.714+0000 7f97d5d6f640 1 -- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc003fd0 msgr2=0x7f97bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:12.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.714+0000 7f97d5d6f640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc003fd0 0x7f97bc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.714+0000 7f97d5d6f640 1 -- 172.21.15.115:0/1988823038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc003520 con 0x7f97bc004d00 2023-12-01T01:51:12.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.714+0000 7f97d556e640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00b4070 0x7f97bc0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:12.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.714+0000 7f97d4d6d640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc003fd0 0x7f97bc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:12.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.715+0000 7f97d5d6f640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97bc004d00 0x7f97bc005180 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f97c8007360 tx=0x7f97bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957d483f5231c5bd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:12.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.715+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c802ccb0 con 0x7f97bc004d00 2023-12-01T01:51:12.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.715+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c802ce10 con 0x7f97bc004d00 2023-12-01T01:51:12.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.715+0000 7f97c7fff640 1 -- 172.21.15.115:0/1988823038 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c8034660 con 0x7f97bc004d00 2023-12-01T01:51:12.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.715+0000 7f97d77d2640 1 -- 172.21.15.115:0/1988823038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97bc004d00 msgr2=0x7f97bc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:12.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.715+0000 7f97d77d2640 1 --2- 172.21.15.115:0/1988823038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97bc004d00 0x7f97bc005180 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f97c8007360 tx=0x7f97bc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.716+0000 7f97d5d6f640 1 -- 172.21.15.115:0/1988823038 reap_dead start 2023-12-01T01:51:12.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.716+0000 7f97d77d2640 1 -- 172.21.15.115:0/1988823038 shutdown_connections 2023-12-01T01:51:12.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.716+0000 7f97d77d2640 1 -- 172.21.15.115:0/1988823038 >> 172.21.15.115:0/1988823038 conn(0x7f97d00662c0 msgr2=0x7f97d00b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:12.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.716+0000 7f97d77d2640 1 -- 172.21.15.115:0/1988823038 shutdown_connections 2023-12-01T01:51:12.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.716+0000 7f97d77d2640 1 -- 172.21.15.115:0/1988823038 wait complete. 2023-12-01T01:51:12.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.718+0000 7f97d77d2640 1 Processor -- start 2023-12-01T01:51:12.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.718+0000 7f97d77d2640 1 -- start start 2023-12-01T01:51:12.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d77d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc004d00 0x7f97d01f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:12.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d77d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d015bca0 0x7f97d0158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:12.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d77d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00a8490 0x7f97d00a8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:12.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d77d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d00a8e50 con 0x7f97d015bca0 2023-12-01T01:51:12.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d77d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d015d520 con 0x7f97bc004d00 2023-12-01T01:51:12.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d77d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d015c480 con 0x7f97d00a8490 2023-12-01T01:51:12.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d4d6d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d015bca0 0x7f97d0158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:12.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d4d6d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d015bca0 0x7f97d0158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39678/0 (socket says 172.21.15.115:39678) 2023-12-01T01:51:12.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d4d6d640 1 -- 172.21.15.115:0/2325327474 learned_addr learned my addr 172.21.15.115:0/2325327474 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:12.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d4d6d640 1 -- 172.21.15.115:0/2325327474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00a8490 msgr2=0x7f97d00a8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:12.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d5d6f640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00a8490 0x7f97d00a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:12.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d556e640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc004d00 0x7f97d01f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:12.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d4d6d640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00a8490 0x7f97d00a8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d4d6d640 1 -- 172.21.15.115:0/2325327474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc004d00 msgr2=0x7f97d01f04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:12.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d4d6d640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc004d00 0x7f97d01f04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:12.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d4d6d640 1 -- 172.21.15.115:0/2325327474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d015c760 con 0x7f97d015bca0 2023-12-01T01:51:12.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.719+0000 7f97d556e640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc004d00 0x7f97d01f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:12.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.720+0000 7f97d4d6d640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d015bca0 0x7f97d0158c50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f97c000d8e0 tx=0x7f97c000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:12.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.720+0000 7f97c5ffb640 1 -- 172.21.15.115:0/2325327474 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00074e0 con 0x7f97d015bca0 2023-12-01T01:51:12.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.720+0000 7f97c5ffb640 1 -- 172.21.15.115:0/2325327474 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c0007640 con 0x7f97d015bca0 2023-12-01T01:51:12.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.720+0000 7f97c5ffb640 1 -- 172.21.15.115:0/2325327474 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00049e0 con 0x7f97d015bca0 2023-12-01T01:51:12.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.720+0000 7f97d77d2640 1 -- 172.21.15.115:0/2325327474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d015ca40 con 0x7f97d015bca0 2023-12-01T01:51:12.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.720+0000 7f97d77d2640 1 -- 172.21.15.115:0/2325327474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d015d000 con 0x7f97d015bca0 2023-12-01T01:51:12.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.721+0000 7f97d77d2640 1 -- 172.21.15.115:0/2325327474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d015d1c0 con 0x7f97d015bca0 2023-12-01T01:51:12.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.722+0000 7f97c5ffb640 1 -- 172.21.15.115:0/2325327474 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f97c0004b40 con 0x7f97d015bca0 2023-12-01T01:51:12.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.723+0000 7f97c5ffb640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f97b406a550 0x7f97b406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:12.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.723+0000 7f97c5ffb640 1 -- 172.21.15.115:0/2325327474 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6542+0+0 (secure 0 0 0) 0x7f97c0088c40 con 0x7f97d015bca0 2023-12-01T01:51:12.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.723+0000 7f97d556e640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f97b406a550 0x7f97b406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:12.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.724+0000 7f97d556e640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f97b406a550 0x7f97b406ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0016f0 tx=0x7f97cc009040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:12.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.728+0000 7f97c5ffb640 1 -- 172.21.15.115:0/2325327474 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f97c0057800 con 0x7f97d015bca0 2023-12-01T01:51:12.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:12.919+0000 7f97d77d2640 1 -- 172.21.15.115:0/2325327474 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f97d0074210 con 0x7f97d015bca0 2023-12-01T01:51:13.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.541+0000 7f97c5ffb640 1 -- 172.21.15.115:0/2325327474 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7f97c0019020 con 0x7f97d015bca0 2023-12-01T01:51:13.543 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T01:51:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 -- 172.21.15.115:0/2325327474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f97b406a550 msgr2=0x7f97b406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f97b406a550 0x7f97b406ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0016f0 tx=0x7f97cc009040 comp rx=0 tx=0).stop 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 -- 172.21.15.115:0/2325327474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d015bca0 msgr2=0x7f97d0158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d015bca0 0x7f97d0158c50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f97c000d8e0 tx=0x7f97c000dda0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 -- 172.21.15.115:0/2325327474 shutdown_connections 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f97b406a550 0x7f97b406ca10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d00a8490 0x7f97d00a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d015bca0 0x7f97d0158c50 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 --2- 172.21.15.115:0/2325327474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97bc004d00 0x7f97d01f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 -- 172.21.15.115:0/2325327474 >> 172.21.15.115:0/2325327474 conn(0x7f97d00662c0 msgr2=0x7f97d00cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 -- 172.21.15.115:0/2325327474 shutdown_connections 2023-12-01T01:51:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.545+0000 7f97d77d2640 1 -- 172.21.15.115:0/2325327474 wait complete. 2023-12-01T01:51:13.568 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T01:51:13.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.767+0000 7fe1ada5f640 1 Processor -- start 2023-12-01T01:51:13.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.767+0000 7fe1ada5f640 1 -- start start 2023-12-01T01:51:13.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.768+0000 7fe1ada5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1a8159bf0 0x7fe1a815e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:13.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.768+0000 7fe1ada5f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe1a80cb8c0 0x7fe1a806f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.768+0000 7fe1ada5f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a815e570 con 0x7fe1a80ca7c0 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.768+0000 7fe1ada5f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a8159fd0 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.768+0000 7fe1ada5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a80c9440 con 0x7fe1a8159bf0 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.769+0000 7fe1a77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe1a80cb8c0 0x7fe1a806f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.769+0000 7fe1a77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe1a80cb8c0 0x7fe1a806f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39690/0 (socket says 172.21.15.115:39690) 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.769+0000 7fe1a77fe640 1 -- 172.21.15.115:0/983781371 learned_addr learned my addr 172.21.15.115:0/983781371 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.769+0000 7fe1a6ffd640 1 --2- 172.21.15.115:0/983781371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1a8159bf0 0x7fe1a815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.769+0000 7fe1a77fe640 1 -- 172.21.15.115:0/983781371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1a8159bf0 msgr2=0x7fe1a815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.769+0000 7fe1a77fe640 1 --2- 172.21.15.115:0/983781371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1a8159bf0 0x7fe1a815e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.769+0000 7fe1a77fe640 1 -- 172.21.15.115:0/983781371 >> v1:172.21.15.142:6789/0 conn(0x7fe1a80ca7c0 legacy=0x7fe1a80cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:13.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.769+0000 7fe1a77fe640 1 -- 172.21.15.115:0/983781371 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a80c9720 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.770+0000 7fe1a77fe640 1 --2- 172.21.15.115:0/983781371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 0x7fe1a806f650 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009e50 tx=0x7fe1a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c385be7c75261e89 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1a5ffb640 1 -- 172.21.15.115:0/983781371 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a002ca70 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1a5ffb640 1 -- 172.21.15.115:0/983781371 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a002cbd0 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1a5ffb640 1 -- 172.21.15.115:0/983781371 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a00343f0 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1ada5f640 1 -- 172.21.15.115:0/983781371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 msgr2=0x7fe1a806f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1ada5f640 1 --2- 172.21.15.115:0/983781371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 0x7fe1a806f650 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009e50 tx=0x7fe1a002bed0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1ada5f640 1 -- 172.21.15.115:0/983781371 shutdown_connections 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1ada5f640 1 --2- 172.21.15.115:0/983781371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 0x7fe1a806f650 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1ada5f640 1 --2- 172.21.15.115:0/983781371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1a8159bf0 0x7fe1a815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1ada5f640 1 -- 172.21.15.115:0/983781371 >> 172.21.15.115:0/983781371 conn(0x7fe1a80662c0 msgr2=0x7fe1a80b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1ada5f640 1 -- 172.21.15.115:0/983781371 shutdown_connections 2023-12-01T01:51:13.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.771+0000 7fe1ada5f640 1 -- 172.21.15.115:0/983781371 wait complete. 2023-12-01T01:51:13.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.773+0000 7fe1ada5f640 1 Processor -- start 2023-12-01T01:51:13.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.773+0000 7fe1ada5f640 1 -- start start 2023-12-01T01:51:13.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.774+0000 7fe1ada5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1a80ca7c0 0x7fe1a81f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:13.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.774+0000 7fe1ada5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 0x7fe1a80c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:13.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.774+0000 7fe1ada5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a8159bf0 0x7fe1a80ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:13.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.774+0000 7fe1ada5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81f1420 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.774+0000 7fe1ada5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81f1680 con 0x7fe1a8159bf0 2023-12-01T01:51:13.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.774+0000 7fe1ada5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81f1920 con 0x7fe1a80ca7c0 2023-12-01T01:51:13.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 0x7fe1a80c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:13.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 0x7fe1a80c48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39696/0 (socket says 172.21.15.115:39696) 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a67fc640 1 -- 172.21.15.115:0/2727936853 learned_addr learned my addr 172.21.15.115:0/2727936853 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a77fe640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a8159bf0 0x7fe1a80ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a6ffd640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1a80ca7c0 0x7fe1a81f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a67fc640 1 -- 172.21.15.115:0/2727936853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1a80ca7c0 msgr2=0x7fe1a81f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a67fc640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1a80ca7c0 0x7fe1a81f0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a67fc640 1 -- 172.21.15.115:0/2727936853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a8159bf0 msgr2=0x7fe1a80ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a67fc640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a8159bf0 0x7fe1a80ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a67fc640 1 -- 172.21.15.115:0/2727936853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a0009b30 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.775+0000 7fe1a67fc640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 0x7fe1a80c48e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00c430 tx=0x7fe19c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.776+0000 7fe18ffff640 1 -- 172.21.15.115:0/2727936853 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c011790 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.776+0000 7fe18ffff640 1 -- 172.21.15.115:0/2727936853 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19c0118f0 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.776+0000 7fe18ffff640 1 -- 172.21.15.115:0/2727936853 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c011ce0 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.776+0000 7fe1ada5f640 1 -- 172.21.15.115:0/2727936853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a81f1c00 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.776+0000 7fe1ada5f640 1 -- 172.21.15.115:0/2727936853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a81f21c0 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.777+0000 7fe1ada5f640 1 -- 172.21.15.115:0/2727936853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a81f23e0 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.778+0000 7fe18ffff640 1 -- 172.21.15.115:0/2727936853 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe19c01c460 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.779+0000 7fe18ffff640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe19406a580 0x7fe19406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:13.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.779+0000 7fe18ffff640 1 -- 172.21.15.115:0/2727936853 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe19c088b10 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.779+0000 7fe1a6ffd640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe19406a580 0x7fe19406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:13.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.780+0000 7fe1a6ffd640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe19406a580 0x7fe19406ca40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe198006fd0 tx=0x7fe198008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:13.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.783+0000 7fe18ffff640 1 -- 172.21.15.115:0/2727936853 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe19c053e60 con 0x7fe1a80cb8c0 2023-12-01T01:51:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:13.968+0000 7fe1ada5f640 1 -- 172.21.15.115:0/2727936853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe1a8074210 con 0x7fe1a80cb8c0 2023-12-01T01:51:14.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.556+0000 7fe18ffff640 1 -- 172.21.15.115:0/2727936853 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe19c057480 con 0x7fe1a80cb8c0 2023-12-01T01:51:14.558 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T01:51:14.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.559+0000 7fe1ada5f640 1 -- 172.21.15.115:0/2727936853 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe19406a580 msgr2=0x7fe19406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:14.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.559+0000 7fe1ada5f640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe19406a580 0x7fe19406ca40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe198006fd0 tx=0x7fe198008040 comp rx=0 tx=0).stop 2023-12-01T01:51:14.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.560+0000 7fe1ada5f640 1 -- 172.21.15.115:0/2727936853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 msgr2=0x7fe1a80c48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:14.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.560+0000 7fe1ada5f640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 0x7fe1a80c48e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00c430 tx=0x7fe19c00c8f0 comp rx=0 tx=0).stop 2023-12-01T01:51:14.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.560+0000 7fe1ada5f640 1 -- 172.21.15.115:0/2727936853 shutdown_connections 2023-12-01T01:51:14.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.560+0000 7fe1ada5f640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe19406a580 0x7fe19406ca40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:14.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.560+0000 7fe1ada5f640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a8159bf0 0x7fe1a80ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:14.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.560+0000 7fe1ada5f640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1a80cb8c0 0x7fe1a80c48e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:14.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.560+0000 7fe1ada5f640 1 --2- 172.21.15.115:0/2727936853 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1a80ca7c0 0x7fe1a81f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:14.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.560+0000 7fe1ada5f640 1 -- 172.21.15.115:0/2727936853 >> 172.21.15.115:0/2727936853 conn(0x7fe1a80662c0 msgr2=0x7fe1a8159020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:14.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.560+0000 7fe1ada5f640 1 -- 172.21.15.115:0/2727936853 shutdown_connections 2023-12-01T01:51:14.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.560+0000 7fe1ada5f640 1 -- 172.21.15.115:0/2727936853 wait complete. 2023-12-01T01:51:14.587 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T01:51:14.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.804+0000 7fa524cec640 1 Processor -- start 2023-12-01T01:51:14.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.804+0000 7fa524cec640 1 -- start start 2023-12-01T01:51:14.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa524cec640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa5200ce1b0 0x7fa5200cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:14.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa524cec640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa5200b0e60 0x7fa5200b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:14.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa524cec640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5200cc9a0 con 0x7fa52015ac80 2023-12-01T01:51:14.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa524cec640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52015b060 con 0x7fa5200ce1b0 2023-12-01T01:51:14.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa524cec640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5200b0ce0 con 0x7fa5200b0e60 2023-12-01T01:51:14.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa517fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa5200ce1b0 0x7fa5200cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:14.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa517fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa5200ce1b0 0x7fa5200cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39704/0 (socket says 172.21.15.115:39704) 2023-12-01T01:51:14.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa517fff640 1 -- 172.21.15.115:0/1527663353 learned_addr learned my addr 172.21.15.115:0/1527663353 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa517fff640 1 -- 172.21.15.115:0/1527663353 >> v2:172.21.15.142:3300/0 conn(0x7fa5200b0e60 msgr2=0x7fa5200b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa517fff640 1 --2- 172.21.15.115:0/1527663353 >> v2:172.21.15.142:3300/0 conn(0x7fa5200b0e60 0x7fa5200b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa517fff640 1 -- 172.21.15.115:0/1527663353 >> v1:172.21.15.142:6789/0 conn(0x7fa52015ac80 legacy=0x7fa5200cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.806+0000 7fa517fff640 1 -- 172.21.15.115:0/1527663353 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200cbd90 con 0x7fa5200ce1b0 2023-12-01T01:51:14.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.807+0000 7fa517fff640 1 --2- 172.21.15.115:0/1527663353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200ce1b0 0x7fa5200cce30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa5200cd6e0 tx=0x7fa50802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83947a1a83692dac server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.808+0000 7fa51dd9b640 1 -- 172.21.15.115:0/1527663353 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508031870 con 0x7fa5200ce1b0 2023-12-01T01:51:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.808+0000 7fa51dd9b640 1 -- 172.21.15.115:0/1527663353 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5080319d0 con 0x7fa5200ce1b0 2023-12-01T01:51:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.808+0000 7fa51dd9b640 1 -- 172.21.15.115:0/1527663353 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508031d60 con 0x7fa5200ce1b0 2023-12-01T01:51:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.808+0000 7fa524cec640 1 -- 172.21.15.115:0/1527663353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200ce1b0 msgr2=0x7fa5200cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.808+0000 7fa524cec640 1 --2- 172.21.15.115:0/1527663353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200ce1b0 0x7fa5200cce30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa5200cd6e0 tx=0x7fa50802b910 comp rx=0 tx=0).stop 2023-12-01T01:51:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.808+0000 7fa524cec640 1 -- 172.21.15.115:0/1527663353 shutdown_connections 2023-12-01T01:51:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.808+0000 7fa524cec640 1 --2- 172.21.15.115:0/1527663353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200ce1b0 0x7fa5200cce30 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.808+0000 7fa524cec640 1 --2- 172.21.15.115:0/1527663353 >> v2:172.21.15.142:3300/0 conn(0x7fa5200b0e60 0x7fa5200b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:14.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.808+0000 7fa524cec640 1 -- 172.21.15.115:0/1527663353 >> 172.21.15.115:0/1527663353 conn(0x7fa520065960 msgr2=0x7fa520065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:14.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.809+0000 7fa524cec640 1 -- 172.21.15.115:0/1527663353 shutdown_connections 2023-12-01T01:51:14.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.809+0000 7fa524cec640 1 -- 172.21.15.115:0/1527663353 wait complete. 2023-12-01T01:51:14.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.810+0000 7fa524cec640 1 Processor -- start 2023-12-01T01:51:14.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.810+0000 7fa524cec640 1 -- start start 2023-12-01T01:51:14.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.811+0000 7fa524cec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200b0e60 0x7fa52014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:14.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.811+0000 7fa524cec640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5200ce1b0 0x7fa5200c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:14.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.811+0000 7fa524cec640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa52015ac80 0x7fa5200c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:14.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.811+0000 7fa524cec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200b2900 con 0x7fa5200b0e60 2023-12-01T01:51:14.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.811+0000 7fa524cec640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200b2be0 con 0x7fa52015ac80 2023-12-01T01:51:14.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.811+0000 7fa524cec640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200b2ec0 con 0x7fa5200ce1b0 2023-12-01T01:51:14.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.811+0000 7fa51e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200b0e60 0x7fa52014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:14.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.811+0000 7fa51e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200b0e60 0x7fa52014d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39712/0 (socket says 172.21.15.115:39712) 2023-12-01T01:51:14.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.811+0000 7fa51e59c640 1 -- 172.21.15.115:0/1155129604 learned_addr learned my addr 172.21.15.115:0/1155129604 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:14.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.812+0000 7fa51e59c640 1 -- 172.21.15.115:0/1155129604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5200ce1b0 msgr2=0x7fa5200c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:14.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.812+0000 7fa51e59c640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5200ce1b0 0x7fa5200c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:14.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.812+0000 7fa51e59c640 1 -- 172.21.15.115:0/1155129604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa52015ac80 msgr2=0x7fa5200c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:14.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.812+0000 7fa51e59c640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa52015ac80 0x7fa5200c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:14.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.812+0000 7fa51e59c640 1 -- 172.21.15.115:0/1155129604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5080094d0 con 0x7fa5200b0e60 2023-12-01T01:51:14.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.812+0000 7fa51e59c640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200b0e60 0x7fa52014d8e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa51000b3e0 tx=0x7fa51000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:14.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.812+0000 7fa516ffd640 1 -- 172.21.15.115:0/1155129604 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510011830 con 0x7fa5200b0e60 2023-12-01T01:51:14.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.812+0000 7fa516ffd640 1 -- 172.21.15.115:0/1155129604 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5100119b0 con 0x7fa5200b0e60 2023-12-01T01:51:14.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.812+0000 7fa524cec640 1 -- 172.21.15.115:0/1155129604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5200b31a0 con 0x7fa5200b0e60 2023-12-01T01:51:14.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.812+0000 7fa516ffd640 1 -- 172.21.15.115:0/1155129604 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510011d60 con 0x7fa5200b0e60 2023-12-01T01:51:14.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.813+0000 7fa524cec640 1 -- 172.21.15.115:0/1155129604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5200b1b60 con 0x7fa5200b0e60 2023-12-01T01:51:14.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.813+0000 7fa524cec640 1 -- 172.21.15.115:0/1155129604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5200b36c0 con 0x7fa5200b0e60 2023-12-01T01:51:14.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.814+0000 7fa516ffd640 1 -- 172.21.15.115:0/1155129604 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa51001a400 con 0x7fa5200b0e60 2023-12-01T01:51:14.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.815+0000 7fa516ffd640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa50006a550 0x7fa50006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:14.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.815+0000 7fa516ffd640 1 -- 172.21.15.115:0/1155129604 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5545+0+0 (secure 0 0 0) 0x7fa510088b90 con 0x7fa5200b0e60 2023-12-01T01:51:14.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.816+0000 7fa517fff640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa50006a550 0x7fa50006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:14.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.816+0000 7fa517fff640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa50006a550 0x7fa50006ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa508036000 tx=0x7fa508030000 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:14.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:14.820+0000 7fa516ffd640 1 -- 172.21.15.115:0/1155129604 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa5200b36c0 con 0x7fa5200b0e60 2023-12-01T01:51:15.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.006+0000 7fa524cec640 1 -- 172.21.15.115:0/1155129604 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa5200b36c0 con 0x7fa5200b0e60 2023-12-01T01:51:15.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.012+0000 7fa516ffd640 1 -- 172.21.15.115:0/1155129604 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa5200b36c0 con 0x7fa5200b0e60 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 -- 172.21.15.115:0/1155129604 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa50006a550 msgr2=0x7fa50006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa50006a550 0x7fa50006ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa508036000 tx=0x7fa508030000 comp rx=0 tx=0).stop 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 -- 172.21.15.115:0/1155129604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200b0e60 msgr2=0x7fa52014d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200b0e60 0x7fa52014d8e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa51000b3e0 tx=0x7fa51000b8a0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 -- 172.21.15.115:0/1155129604 shutdown_connections 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa50006a550 0x7fa50006ca10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5200ce1b0 0x7fa5200c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa52015ac80 0x7fa5200c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 --2- 172.21.15.115:0/1155129604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5200b0e60 0x7fa52014d8e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 -- 172.21.15.115:0/1155129604 >> 172.21.15.115:0/1155129604 conn(0x7fa520065960 msgr2=0x7fa52006b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 -- 172.21.15.115:0/1155129604 shutdown_connections 2023-12-01T01:51:15.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.015+0000 7fa524cec640 1 -- 172.21.15.115:0/1155129604 wait complete. 2023-12-01T01:51:15.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T01:51:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.240+0000 7f625f59e640 1 Processor -- start 2023-12-01T01:51:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.240+0000 7f625f59e640 1 -- start start 2023-12-01T01:51:15.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.242+0000 7f625f59e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f626015cb40 0x7f62600d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.242+0000 7f625f59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f62600b8400 0x7f62600cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.242+0000 7f625f59e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62600d1e00 con 0x7f62600cdc80 2023-12-01T01:51:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.242+0000 7f625f59e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62600cd250 con 0x7f62600b8400 2023-12-01T01:51:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.242+0000 7f625f59e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6260069cc0 con 0x7f626015cb40 2023-12-01T01:51:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.242+0000 7f6257fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f62600b8400 0x7f62600cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.242+0000 7f625ed9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f62600cdc80 0x7f62601607c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44648/0 (socket says 172.21.15.115:44648) 2023-12-01T01:51:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.242+0000 7f625ed9d640 1 -- 172.21.15.115:0/2414489644 learned_addr learned my addr 172.21.15.115:0/2414489644 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.243+0000 7f625e59c640 1 --2- 172.21.15.115:0/2414489644 >> v2:172.21.15.142:3300/0 conn(0x7f626015cb40 0x7f62600d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.243+0000 7f6257fff640 1 -- 172.21.15.115:0/2414489644 >> v2:172.21.15.142:3300/0 conn(0x7f626015cb40 msgr2=0x7f62600d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.243+0000 7f6257fff640 1 --2- 172.21.15.115:0/2414489644 >> v2:172.21.15.142:3300/0 conn(0x7f626015cb40 0x7f62600d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.243+0000 7f6257fff640 1 -- 172.21.15.115:0/2414489644 >> v1:172.21.15.115:6789/0 conn(0x7f62600cdc80 legacy=0x7f62601607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.243+0000 7f6257fff640 1 -- 172.21.15.115:0/2414489644 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62600ce0e0 con 0x7f62600b8400 2023-12-01T01:51:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.243+0000 7f625dd9b640 1 -- 172.21.15.115:0/2414489644 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925968164 0 0) 0x7f62600d1e00 con 0x7f62600cdc80 2023-12-01T01:51:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.243+0000 7f625e59c640 1 --2- 172.21.15.115:0/2414489644 >> v2:172.21.15.142:3300/0 conn(0x7f626015cb40 0x7f62600d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.244+0000 7f6257fff640 1 --2- 172.21.15.115:0/2414489644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600b8400 0x7f62600cd740 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f62600d3b70 tx=0x7f624802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68f97f6c0492172 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:15.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.244+0000 7f625dd9b640 1 -- 172.21.15.115:0/2414489644 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248031830 con 0x7f62600b8400 2023-12-01T01:51:15.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.244+0000 7f625dd9b640 1 -- 172.21.15.115:0/2414489644 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6248031990 con 0x7f62600b8400 2023-12-01T01:51:15.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.244+0000 7f625dd9b640 1 -- 172.21.15.115:0/2414489644 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248031d20 con 0x7f62600b8400 2023-12-01T01:51:15.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.245+0000 7f625f59e640 1 -- 172.21.15.115:0/2414489644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600b8400 msgr2=0x7f62600cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.245+0000 7f625f59e640 1 --2- 172.21.15.115:0/2414489644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600b8400 0x7f62600cd740 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f62600d3b70 tx=0x7f624802b8d0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.245+0000 7f625f59e640 1 -- 172.21.15.115:0/2414489644 shutdown_connections 2023-12-01T01:51:15.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.245+0000 7f625f59e640 1 --2- 172.21.15.115:0/2414489644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600b8400 0x7f62600cd740 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.245+0000 7f625f59e640 1 --2- 172.21.15.115:0/2414489644 >> v2:172.21.15.142:3300/0 conn(0x7f626015cb40 0x7f62600d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.245+0000 7f625f59e640 1 -- 172.21.15.115:0/2414489644 >> 172.21.15.115:0/2414489644 conn(0x7f62600d2a70 msgr2=0x7f62600d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:15.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.245+0000 7f625f59e640 1 -- 172.21.15.115:0/2414489644 shutdown_connections 2023-12-01T01:51:15.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.245+0000 7f625f59e640 1 -- 172.21.15.115:0/2414489644 wait complete. 2023-12-01T01:51:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.247+0000 7f625f59e640 1 Processor -- start 2023-12-01T01:51:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.247+0000 7f625f59e640 1 -- start start 2023-12-01T01:51:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f625f59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62600b8400 0x7f62601f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f625f59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600cdc80 0x7f6260154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f625f59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626015cb40 0x7f626015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f625f59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62600c7a00 con 0x7f62600cdc80 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f625f59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626015e8c0 con 0x7f62600b8400 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f625f59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626015eba0 con 0x7f626015cb40 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f6257fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600cdc80 0x7f6260154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f6257fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600cdc80 0x7f6260154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39734/0 (socket says 172.21.15.115:39734) 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f6257fff640 1 -- 172.21.15.115:0/2366427814 learned_addr learned my addr 172.21.15.115:0/2366427814 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f625e59c640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62600b8400 0x7f62601f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f6257fff640 1 -- 172.21.15.115:0/2366427814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626015cb40 msgr2=0x7f626015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f6257fff640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626015cb40 0x7f626015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f6257fff640 1 -- 172.21.15.115:0/2366427814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62600b8400 msgr2=0x7f62601f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f6257fff640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62600b8400 0x7f62601f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.248+0000 7f6257fff640 1 -- 172.21.15.115:0/2366427814 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248009400 con 0x7f62600cdc80 2023-12-01T01:51:15.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.249+0000 7f6257fff640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600cdc80 0x7f6260154ea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6248009890 tx=0x7f62480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.249+0000 7f6256ffd640 1 -- 172.21.15.115:0/2366427814 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624803ba70 con 0x7f62600cdc80 2023-12-01T01:51:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.249+0000 7f6256ffd640 1 -- 172.21.15.115:0/2366427814 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624803bbf0 con 0x7f62600cdc80 2023-12-01T01:51:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.249+0000 7f6256ffd640 1 -- 172.21.15.115:0/2366427814 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248045400 con 0x7f62600cdc80 2023-12-01T01:51:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.249+0000 7f625f59e640 1 -- 172.21.15.115:0/2366427814 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62600c7690 con 0x7f62600cdc80 2023-12-01T01:51:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.249+0000 7f625f59e640 1 -- 172.21.15.115:0/2366427814 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62601553e0 con 0x7f62600cdc80 2023-12-01T01:51:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.250+0000 7f625f59e640 1 -- 172.21.15.115:0/2366427814 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62600c7bb0 con 0x7f62600cdc80 2023-12-01T01:51:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.251+0000 7f6256ffd640 1 -- 172.21.15.115:0/2366427814 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f624804e440 con 0x7f62600cdc80 2023-12-01T01:51:15.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.252+0000 7f6256ffd640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f624006a550 0x7f624006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.252+0000 7f6256ffd640 1 -- 172.21.15.115:0/2366427814 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f62480aaea0 con 0x7f62600cdc80 2023-12-01T01:51:15.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.252+0000 7f625e59c640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f624006a550 0x7f624006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:15.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.253+0000 7f625e59c640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f624006a550 0x7f624006ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f625000b840 tx=0x7f6250008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:15.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.256+0000 7f6256ffd640 1 -- 172.21.15.115:0/2366427814 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f62600c7bb0 con 0x7f62600cdc80 2023-12-01T01:51:15.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.448+0000 7f625f59e640 1 -- 172.21.15.115:0/2366427814 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f62600618d0 con 0x7f62600cdc80 2023-12-01T01:51:15.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.457+0000 7f6256ffd640 1 -- 172.21.15.115:0/2366427814 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v20) v1 ==== 120+0+0 (secure 0 0 0) 0x7f6248076310 con 0x7f62600cdc80 2023-12-01T01:51:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.460+0000 7f625f59e640 1 -- 172.21.15.115:0/2366427814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f624006a550 msgr2=0x7f624006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.460+0000 7f625f59e640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f624006a550 0x7f624006ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f625000b840 tx=0x7f6250008040 comp rx=0 tx=0).stop 2023-12-01T01:51:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.460+0000 7f625f59e640 1 -- 172.21.15.115:0/2366427814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600cdc80 msgr2=0x7f6260154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.460+0000 7f625f59e640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600cdc80 0x7f6260154ea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6248009890 tx=0x7f62480046c0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.460+0000 7f625f59e640 1 -- 172.21.15.115:0/2366427814 shutdown_connections 2023-12-01T01:51:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.460+0000 7f625f59e640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f624006a550 0x7f624006ca10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.460+0000 7f625f59e640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f626015cb40 0x7f626015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.460+0000 7f625f59e640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62600cdc80 0x7f6260154ea0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.461+0000 7f625f59e640 1 --2- 172.21.15.115:0/2366427814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62600b8400 0x7f62601f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.461+0000 7f625f59e640 1 -- 172.21.15.115:0/2366427814 >> 172.21.15.115:0/2366427814 conn(0x7f62600d2a70 msgr2=0x7f62600b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.461+0000 7f625f59e640 1 -- 172.21.15.115:0/2366427814 shutdown_connections 2023-12-01T01:51:15.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.461+0000 7f625f59e640 1 -- 172.21.15.115:0/2366427814 wait complete. 2023-12-01T01:51:15.481 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T01:51:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.690+0000 7fb86a3cb640 1 Processor -- start 2023-12-01T01:51:15.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.690+0000 7fb86a3cb640 1 -- start start 2023-12-01T01:51:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.691+0000 7fb86a3cb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb864160200 0x7fb8640b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.691+0000 7fb86a3cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8640b0f80 0x7fb86406b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.691+0000 7fb86a3cb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8640cb290 con 0x7fb86406b7a0 2023-12-01T01:51:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.691+0000 7fb86a3cb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8640b35f0 con 0x7fb864160200 2023-12-01T01:51:15.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.691+0000 7fb86a3cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8640b0ae0 con 0x7fb8640b0f80 2023-12-01T01:51:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.692+0000 7fb868968640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb86406b7a0 0x7fb8640cbd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44658/0 (socket says 172.21.15.115:44658) 2023-12-01T01:51:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.692+0000 7fb868968640 1 -- 172.21.15.115:0/2522819548 learned_addr learned my addr 172.21.15.115:0/2522819548 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.692+0000 7fb863fff640 1 --2- 172.21.15.115:0/2522819548 >> v2:172.21.15.115:3300/0 conn(0x7fb864160200 0x7fb8640b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.692+0000 7fb85bfff640 1 --2- 172.21.15.115:0/2522819548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8640b0f80 0x7fb86406b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.692+0000 7fb8637fe640 1 -- 172.21.15.115:0/2522819548 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323017641 0 0) 0x7fb8640cb290 con 0x7fb86406b7a0 2023-12-01T01:51:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.692+0000 7fb8637fe640 1 -- 172.21.15.115:0/2522819548 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb850003520 con 0x7fb86406b7a0 2023-12-01T01:51:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.693+0000 7fb863fff640 1 -- 172.21.15.115:0/2522819548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8640b0f80 msgr2=0x7fb86406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.693+0000 7fb863fff640 1 --2- 172.21.15.115:0/2522819548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8640b0f80 0x7fb86406b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.693+0000 7fb863fff640 1 -- 172.21.15.115:0/2522819548 >> v1:172.21.15.115:6789/0 conn(0x7fb86406b7a0 legacy=0x7fb8640cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.693+0000 7fb8637fe640 1 -- 172.21.15.115:0/2522819548 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1143383977 0 0) 0x7fb850003520 con 0x7fb86406b7a0 2023-12-01T01:51:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.693+0000 7fb863fff640 1 -- 172.21.15.115:0/2522819548 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8640b3310 con 0x7fb864160200 2023-12-01T01:51:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.694+0000 7fb863fff640 1 --2- 172.21.15.115:0/2522819548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 0x7fb8640b2c80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb8540097a0 tx=0x7fb85402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632da48106613e44 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.694+0000 7fb8637fe640 1 -- 172.21.15.115:0/2522819548 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0019f0 con 0x7fb864160200 2023-12-01T01:51:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.694+0000 7fb8637fe640 1 -- 172.21.15.115:0/2522819548 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85402e050 con 0x7fb864160200 2023-12-01T01:51:15.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.694+0000 7fb8637fe640 1 -- 172.21.15.115:0/2522819548 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854030730 con 0x7fb864160200 2023-12-01T01:51:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.695+0000 7fb86a3cb640 1 -- 172.21.15.115:0/2522819548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 msgr2=0x7fb8640b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.695+0000 7fb86a3cb640 1 --2- 172.21.15.115:0/2522819548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 0x7fb8640b2c80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb8540097a0 tx=0x7fb85402b790 comp rx=0 tx=0).stop 2023-12-01T01:51:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.695+0000 7fb86a3cb640 1 -- 172.21.15.115:0/2522819548 shutdown_connections 2023-12-01T01:51:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.695+0000 7fb86a3cb640 1 --2- 172.21.15.115:0/2522819548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8640b0f80 0x7fb86406b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.695+0000 7fb86a3cb640 1 --2- 172.21.15.115:0/2522819548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 0x7fb8640b2c80 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.695+0000 7fb86a3cb640 1 -- 172.21.15.115:0/2522819548 >> 172.21.15.115:0/2522819548 conn(0x7fb864065960 msgr2=0x7fb864065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.695+0000 7fb86a3cb640 1 -- 172.21.15.115:0/2522819548 shutdown_connections 2023-12-01T01:51:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.695+0000 7fb86a3cb640 1 -- 172.21.15.115:0/2522819548 wait complete. 2023-12-01T01:51:15.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.697+0000 7fb86a3cb640 1 Processor -- start 2023-12-01T01:51:15.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.697+0000 7fb86a3cb640 1 -- start start 2023-12-01T01:51:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb86a3cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb86406b7a0 0x7fb8641f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb86a3cb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8640b0f80 0x7fb86415b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb86a3cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 0x7fb86415e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb86a3cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8640c98e0 con 0x7fb864160200 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb86a3cb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8640c8bd0 con 0x7fb8640b0f80 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb86a3cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8641f3790 con 0x7fb86406b7a0 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb868968640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 0x7fb86415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb868968640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 0x7fb86415e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39748/0 (socket says 172.21.15.115:39748) 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb868968640 1 -- 172.21.15.115:0/680286195 learned_addr learned my addr 172.21.15.115:0/680286195 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb868968640 1 -- 172.21.15.115:0/680286195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb86406b7a0 msgr2=0x7fb8641f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb868968640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb86406b7a0 0x7fb8641f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb868968640 1 -- 172.21.15.115:0/680286195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8640b0f80 msgr2=0x7fb86415b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb868968640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8640b0f80 0x7fb86415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.698+0000 7fb868968640 1 -- 172.21.15.115:0/680286195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb854009400 con 0x7fb864160200 2023-12-01T01:51:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.699+0000 7fb868968640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 0x7fb86415e190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00ccc0 tx=0x7fb85c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:15.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.699+0000 7fb8617fa640 1 -- 172.21.15.115:0/680286195 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c012400 con 0x7fb864160200 2023-12-01T01:51:15.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.699+0000 7fb8617fa640 1 -- 172.21.15.115:0/680286195 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c012560 con 0x7fb864160200 2023-12-01T01:51:15.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.699+0000 7fb8617fa640 1 -- 172.21.15.115:0/680286195 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0128d0 con 0x7fb864160200 2023-12-01T01:51:15.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.699+0000 7fb86a3cb640 1 -- 172.21.15.115:0/680286195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8640c7fc0 con 0x7fb864160200 2023-12-01T01:51:15.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.699+0000 7fb86a3cb640 1 -- 172.21.15.115:0/680286195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8640c8580 con 0x7fb864160200 2023-12-01T01:51:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.700+0000 7fb86a3cb640 1 -- 172.21.15.115:0/680286195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86404e890 con 0x7fb864160200 2023-12-01T01:51:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.701+0000 7fb8617fa640 1 -- 172.21.15.115:0/680286195 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb85c007070 con 0x7fb864160200 2023-12-01T01:51:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.702+0000 7fb8617fa640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 0x7fb84406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.702+0000 7fb8617fa640 1 -- 172.21.15.115:0/680286195 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5545+0+0 (secure 0 0 0) 0x7fb85c088c20 con 0x7fb864160200 2023-12-01T01:51:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.702+0000 7fb863fff640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 0x7fb84406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:15.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.703+0000 7fb863fff640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 0x7fb84406ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb854034000 tx=0x7fb85402d000 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:15.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.707+0000 7fb8617fa640 1 -- 172.21.15.115:0/680286195 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb85c054100 con 0x7fb864160200 2023-12-01T01:51:15.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.892+0000 7fb86a3cb640 1 -- 172.21.15.115:0/680286195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fb864065ff0 con 0x7fb864160200 2023-12-01T01:51:15.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.898+0000 7fb8617fa640 1 -- 172.21.15.115:0/680286195 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb85c017020 con 0x7fb864160200 2023-12-01T01:51:15.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.901+0000 7fb86a3cb640 1 -- 172.21.15.115:0/680286195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 msgr2=0x7fb84406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.901+0000 7fb86a3cb640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 0x7fb84406ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb854034000 tx=0x7fb85402d000 comp rx=0 tx=0).stop 2023-12-01T01:51:15.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.901+0000 7fb86a3cb640 1 -- 172.21.15.115:0/680286195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 msgr2=0x7fb86415e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:15.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.901+0000 7fb86a3cb640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 0x7fb86415e190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00ccc0 tx=0x7fb85c00a430 comp rx=0 tx=0).stop 2023-12-01T01:51:15.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.901+0000 7fb86a3cb640 1 -- 172.21.15.115:0/680286195 shutdown_connections 2023-12-01T01:51:15.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.901+0000 7fb86a3cb640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 0x7fb84406ca10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.902+0000 7fb86a3cb640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb864160200 0x7fb86415e190 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.902+0000 7fb86a3cb640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8640b0f80 0x7fb86415b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.902+0000 7fb86a3cb640 1 --2- 172.21.15.115:0/680286195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb86406b7a0 0x7fb8641f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:15.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.902+0000 7fb86a3cb640 1 -- 172.21.15.115:0/680286195 >> 172.21.15.115:0/680286195 conn(0x7fb864065960 msgr2=0x7fb86406bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:15.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.902+0000 7fb86a3cb640 1 -- 172.21.15.115:0/680286195 shutdown_connections 2023-12-01T01:51:15.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:15.902+0000 7fb86a3cb640 1 -- 172.21.15.115:0/680286195 wait complete. 2023-12-01T01:51:15.922 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-12-01T01:51:16.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.126+0000 7fcbfd0e0640 1 Processor -- start 2023-12-01T01:51:16.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.126+0000 7fcbfd0e0640 1 -- start start 2023-12-01T01:51:16.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbfd0e0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcbf815e2f0 0x7fcbf80ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbfd0e0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcbf80adf10 0x7fcbf80caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbfd0e0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf80add10 con 0x7fcbf806f2f0 2023-12-01T01:51:16.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbfd0e0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf80c5760 con 0x7fcbf815e2f0 2023-12-01T01:51:16.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbfd0e0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf815e6d0 con 0x7fcbf80adf10 2023-12-01T01:51:16.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbf6d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcbf815e2f0 0x7fcbf80ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbf6d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcbf815e2f0 0x7fcbf80ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39750/0 (socket says 172.21.15.115:39750) 2023-12-01T01:51:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbf6d9d640 1 -- 172.21.15.115:0/3594934270 learned_addr learned my addr 172.21.15.115:0/3594934270 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbf6d9d640 1 -- 172.21.15.115:0/3594934270 >> v2:172.21.15.142:3300/0 conn(0x7fcbf80adf10 msgr2=0x7fcbf80caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:16.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbf759e640 1 --2- 172.21.15.115:0/3594934270 >> v2:172.21.15.142:3300/0 conn(0x7fcbf80adf10 0x7fcbf80caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbf6d9d640 1 --2- 172.21.15.115:0/3594934270 >> v2:172.21.15.142:3300/0 conn(0x7fcbf80adf10 0x7fcbf80caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.128+0000 7fcbf6d9d640 1 -- 172.21.15.115:0/3594934270 >> v1:172.21.15.142:6789/0 conn(0x7fcbf806f2f0 legacy=0x7fcbf80c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:16.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.129+0000 7fcbf6d9d640 1 -- 172.21.15.115:0/3594934270 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf80cf7b0 con 0x7fcbf815e2f0 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.130+0000 7fcbf6d9d640 1 --2- 172.21.15.115:0/3594934270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 0x7fcbf80ca660 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80c8e80 tx=0x7fcbe802b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a42c84cc3cfb709 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.130+0000 7fcbf5d9b640 1 -- 172.21.15.115:0/3594934270 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8031760 con 0x7fcbf815e2f0 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.130+0000 7fcbf5d9b640 1 -- 172.21.15.115:0/3594934270 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe80318c0 con 0x7fcbf815e2f0 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.130+0000 7fcbf5d9b640 1 -- 172.21.15.115:0/3594934270 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8031c50 con 0x7fcbf815e2f0 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.130+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/3594934270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 msgr2=0x7fcbf80ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.130+0000 7fcbfd0e0640 1 --2- 172.21.15.115:0/3594934270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 0x7fcbf80ca660 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80c8e80 tx=0x7fcbe802b800 comp rx=0 tx=0).stop 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.131+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/3594934270 shutdown_connections 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.131+0000 7fcbfd0e0640 1 --2- 172.21.15.115:0/3594934270 >> v2:172.21.15.142:3300/0 conn(0x7fcbf80adf10 0x7fcbf80caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.131+0000 7fcbfd0e0640 1 --2- 172.21.15.115:0/3594934270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 0x7fcbf80ca660 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.131+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/3594934270 >> 172.21.15.115:0/3594934270 conn(0x7fcbf80ce880 msgr2=0x7fcbf80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:16.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.131+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/3594934270 shutdown_connections 2023-12-01T01:51:16.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.131+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/3594934270 wait complete. 2023-12-01T01:51:16.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.132+0000 7fcbfd0e0640 1 Processor -- start 2023-12-01T01:51:16.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.132+0000 7fcbfd0e0640 1 -- start start 2023-12-01T01:51:16.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.133+0000 7fcbfd0e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbf806f2f0 0x7fcbf81f0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.133+0000 7fcbfd0e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf80adf10 0x7fcbf81f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.133+0000 7fcbfd0e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 0x7fcbf806bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.133+0000 7fcbfd0e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80c5bd0 con 0x7fcbf815e2f0 2023-12-01T01:51:16.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.133+0000 7fcbfd0e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80c4430 con 0x7fcbf806f2f0 2023-12-01T01:51:16.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.133+0000 7fcbfd0e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf81f3790 con 0x7fcbf80adf10 2023-12-01T01:51:16.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.134+0000 7fcbf759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 0x7fcbf806bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:16.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.134+0000 7fcbf759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 0x7fcbf806bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39752/0 (socket says 172.21.15.115:39752) 2023-12-01T01:51:16.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.134+0000 7fcbf759e640 1 -- 172.21.15.115:0/672219082 learned_addr learned my addr 172.21.15.115:0/672219082 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.134+0000 7fcbf6d9d640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbf806f2f0 0x7fcbf81f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.134+0000 7fcbf759e640 1 -- 172.21.15.115:0/672219082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf80adf10 msgr2=0x7fcbf81f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.134+0000 7fcbf759e640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf80adf10 0x7fcbf81f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.134+0000 7fcbf759e640 1 -- 172.21.15.115:0/672219082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbf806f2f0 msgr2=0x7fcbf81f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.134+0000 7fcbf759e640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbf806f2f0 0x7fcbf81f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.134+0000 7fcbf759e640 1 -- 172.21.15.115:0/672219082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe80094d0 con 0x7fcbf815e2f0 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.134+0000 7fcbf759e640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 0x7fcbf806bd80 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000cc30 tx=0x7fcbf000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.135+0000 7fcbdf7fe640 1 -- 172.21.15.115:0/672219082 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000ce40 con 0x7fcbf815e2f0 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.135+0000 7fcbdf7fe640 1 -- 172.21.15.115:0/672219082 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf0003990 con 0x7fcbf815e2f0 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.135+0000 7fcbdf7fe640 1 -- 172.21.15.115:0/672219082 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0003d20 con 0x7fcbf815e2f0 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.135+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/672219082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf80c2fc0 con 0x7fcbf815e2f0 2023-12-01T01:51:16.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.135+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/672219082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf80c3580 con 0x7fcbf815e2f0 2023-12-01T01:51:16.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.136+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/672219082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf806fda0 con 0x7fcbf815e2f0 2023-12-01T01:51:16.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.137+0000 7fcbdf7fe640 1 -- 172.21.15.115:0/672219082 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fcbf0003e80 con 0x7fcbf815e2f0 2023-12-01T01:51:16.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.138+0000 7fcbdf7fe640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a550 0x7fcbe406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.138+0000 7fcbdf7fe640 1 -- 172.21.15.115:0/672219082 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5545+0+0 (secure 0 0 0) 0x7fcbf0088ed0 con 0x7fcbf815e2f0 2023-12-01T01:51:16.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.138+0000 7fcbf6d9d640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a550 0x7fcbe406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:16.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.139+0000 7fcbf6d9d640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a550 0x7fcbe406ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8036000 tx=0x7fcbe803d040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:16.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.142+0000 7fcbdf7fe640 1 -- 172.21.15.115:0/672219082 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcbf0054330 con 0x7fcbf815e2f0 2023-12-01T01:51:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.400+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/672219082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fcbf80c5d10 con 0x7fcbf815e2f0 2023-12-01T01:51:16.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.569+0000 7fcbdf7fe640 1 -- 172.21.15.115:0/672219082 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v560) v1 ==== 154+0+0 (secure 0 0 0) 0x7fcbf80c5d10 con 0x7fcbf815e2f0 2023-12-01T01:51:16.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.571+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/672219082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a550 msgr2=0x7fcbe406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:16.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.571+0000 7fcbfd0e0640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a550 0x7fcbe406ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8036000 tx=0x7fcbe803d040 comp rx=0 tx=0).stop 2023-12-01T01:51:16.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.571+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/672219082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 msgr2=0x7fcbf806bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:16.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.571+0000 7fcbfd0e0640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 0x7fcbf806bd80 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000cc30 tx=0x7fcbf000a5c0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.572+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/672219082 shutdown_connections 2023-12-01T01:51:16.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.572+0000 7fcbfd0e0640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a550 0x7fcbe406ca10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.572+0000 7fcbfd0e0640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf80adf10 0x7fcbf81f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.572+0000 7fcbfd0e0640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbf815e2f0 0x7fcbf806bd80 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.572+0000 7fcbfd0e0640 1 --2- 172.21.15.115:0/672219082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbf806f2f0 0x7fcbf81f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.572+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/672219082 >> 172.21.15.115:0/672219082 conn(0x7fcbf80ce880 msgr2=0x7fcbf80af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:16.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.572+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/672219082 shutdown_connections 2023-12-01T01:51:16.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.572+0000 7fcbfd0e0640 1 -- 172.21.15.115:0/672219082 wait complete. 2023-12-01T01:51:16.591 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-12-01T01:51:16.592 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-12-01T01:51:16.592 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-01T01:51:16.592 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-12-01T01:51:16.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.802+0000 7f88a6bc9640 1 Processor -- start 2023-12-01T01:51:16.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.802+0000 7f88a6bc9640 1 -- start start 2023-12-01T01:51:16.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.803+0000 7f88a6bc9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f88a015d220 0x7f88a00cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.803+0000 7f88a6bc9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f88a0070ac0 0x7f88a00ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.803+0000 7f88a6bc9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a015d600 con 0x7f88a00ad2b0 2023-12-01T01:51:16.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.803+0000 7f88a6bc9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a006cc10 con 0x7f88a015d220 2023-12-01T01:51:16.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.803+0000 7f88a6bc9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a006c720 con 0x7f88a0070ac0 2023-12-01T01:51:16.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.804+0000 7f88a4965640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f88a015d220 0x7f88a00cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:16.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.804+0000 7f88a4965640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f88a015d220 0x7f88a00cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39762/0 (socket says 172.21.15.115:39762) 2023-12-01T01:51:16.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.804+0000 7f88a4965640 1 -- 172.21.15.115:0/1249189281 learned_addr learned my addr 172.21.15.115:0/1249189281 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:16.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.804+0000 7f88a5166640 1 --2- 172.21.15.115:0/1249189281 >> v2:172.21.15.142:3300/0 conn(0x7f88a0070ac0 0x7f88a00ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:16.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.804+0000 7f88a4965640 1 -- 172.21.15.115:0/1249189281 >> v2:172.21.15.142:3300/0 conn(0x7f88a0070ac0 msgr2=0x7f88a00ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:16.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.804+0000 7f88a4965640 1 --2- 172.21.15.115:0/1249189281 >> v2:172.21.15.142:3300/0 conn(0x7f88a0070ac0 0x7f88a00ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.804+0000 7f88a4965640 1 -- 172.21.15.115:0/1249189281 >> v1:172.21.15.142:6789/0 conn(0x7f88a00ad2b0 legacy=0x7f88a006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:16.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.804+0000 7f88a4965640 1 -- 172.21.15.115:0/1249189281 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a006c4b0 con 0x7f88a015d220 2023-12-01T01:51:16.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.804+0000 7f88a5166640 1 --2- 172.21.15.115:0/1249189281 >> v2:172.21.15.142:3300/0 conn(0x7f88a0070ac0 0x7f88a00ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.805+0000 7f88a4965640 1 --2- 172.21.15.115:0/1249189281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a015d220 0x7f88a00cc570 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f88a00c8e60 tx=0x7f889402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46218f7f722307e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.806+0000 7f889f7fe640 1 -- 172.21.15.115:0/1249189281 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88940317e0 con 0x7f88a015d220 2023-12-01T01:51:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.806+0000 7f889f7fe640 1 -- 172.21.15.115:0/1249189281 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8894031940 con 0x7f88a015d220 2023-12-01T01:51:16.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.806+0000 7f889f7fe640 1 -- 172.21.15.115:0/1249189281 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894031cd0 con 0x7f88a015d220 2023-12-01T01:51:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.806+0000 7f88a6bc9640 1 -- 172.21.15.115:0/1249189281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a015d220 msgr2=0x7f88a00cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.806+0000 7f88a6bc9640 1 --2- 172.21.15.115:0/1249189281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a015d220 0x7f88a00cc570 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f88a00c8e60 tx=0x7f889402b880 comp rx=0 tx=0).stop 2023-12-01T01:51:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.807+0000 7f88a6bc9640 1 -- 172.21.15.115:0/1249189281 shutdown_connections 2023-12-01T01:51:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.807+0000 7f88a6bc9640 1 --2- 172.21.15.115:0/1249189281 >> v2:172.21.15.142:3300/0 conn(0x7f88a0070ac0 0x7f88a00ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.807+0000 7f88a6bc9640 1 --2- 172.21.15.115:0/1249189281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a015d220 0x7f88a00cc570 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.807+0000 7f88a6bc9640 1 -- 172.21.15.115:0/1249189281 >> 172.21.15.115:0/1249189281 conn(0x7f88a00ce880 msgr2=0x7f88a00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.807+0000 7f88a6bc9640 1 -- 172.21.15.115:0/1249189281 shutdown_connections 2023-12-01T01:51:16.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.807+0000 7f88a6bc9640 1 -- 172.21.15.115:0/1249189281 wait complete. 2023-12-01T01:51:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.809+0000 7f88a6bc9640 1 Processor -- start 2023-12-01T01:51:16.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.809+0000 7f88a6bc9640 1 -- start start 2023-12-01T01:51:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.810+0000 7f88a6bc9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a0070ac0 0x7f88a01571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.810+0000 7f88a6bc9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88a00ad2b0 0x7f88a00ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.810+0000 7f88a6bc9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88a015d220 0x7f88a00b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.810+0000 7f88a6bc9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a00cab40 con 0x7f88a0070ac0 2023-12-01T01:51:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.810+0000 7f88a6bc9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a00b3ee0 con 0x7f88a00ad2b0 2023-12-01T01:51:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.810+0000 7f88a6bc9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a00b41c0 con 0x7f88a015d220 2023-12-01T01:51:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.810+0000 7f88a4965640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a0070ac0 0x7f88a01571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.811+0000 7f88a4965640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a0070ac0 0x7f88a01571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39770/0 (socket says 172.21.15.115:39770) 2023-12-01T01:51:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.811+0000 7f88a4965640 1 -- 172.21.15.115:0/3380380627 learned_addr learned my addr 172.21.15.115:0/3380380627 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.811+0000 7f88a4965640 1 -- 172.21.15.115:0/3380380627 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88a015d220 msgr2=0x7f88a00b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.811+0000 7f88a4965640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88a015d220 0x7f88a00b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.811+0000 7f889ffff640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88a00ad2b0 0x7f88a00ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.811+0000 7f88a4965640 1 -- 172.21.15.115:0/3380380627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88a00ad2b0 msgr2=0x7f88a00ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.811+0000 7f88a4965640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88a00ad2b0 0x7f88a00ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.811+0000 7f88a4965640 1 -- 172.21.15.115:0/3380380627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940094d0 con 0x7f88a0070ac0 2023-12-01T01:51:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.811+0000 7f88a4965640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a0070ac0 0x7f88a01571c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f889402b8c0 tx=0x7f88940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.812+0000 7f889d7fa640 1 -- 172.21.15.115:0/3380380627 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88940329c0 con 0x7f88a0070ac0 2023-12-01T01:51:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.812+0000 7f889d7fa640 1 -- 172.21.15.115:0/3380380627 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8894032b40 con 0x7f88a0070ac0 2023-12-01T01:51:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.812+0000 7f889d7fa640 1 -- 172.21.15.115:0/3380380627 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88940453f0 con 0x7f88a0070ac0 2023-12-01T01:51:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.812+0000 7f88a6bc9640 1 -- 172.21.15.115:0/3380380627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a00b1660 con 0x7f88a0070ac0 2023-12-01T01:51:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.812+0000 7f88a6bc9640 1 -- 172.21.15.115:0/3380380627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a01f2920 con 0x7f88a0070ac0 2023-12-01T01:51:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.813+0000 7f88a6bc9640 1 -- 172.21.15.115:0/3380380627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a00b1b80 con 0x7f88a0070ac0 2023-12-01T01:51:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.814+0000 7f889d7fa640 1 -- 172.21.15.115:0/3380380627 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f889404d3f0 con 0x7f88a0070ac0 2023-12-01T01:51:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.815+0000 7f889d7fa640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f888406a550 0x7f888406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.815+0000 7f889d7fa640 1 -- 172.21.15.115:0/3380380627 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f88940aa840 con 0x7f88a0070ac0 2023-12-01T01:51:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.815+0000 7f889ffff640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f888406a550 0x7f888406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.816+0000 7f889ffff640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f888406a550 0x7f888406ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f88a00ae290 tx=0x7f888c008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:16.819+0000 7f889d7fa640 1 -- 172.21.15.115:0/3380380627 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f88a00b1b80 con 0x7f88a0070ac0 2023-12-01T01:51:17.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.082+0000 7f88a6bc9640 1 -- 172.21.15.115:0/3380380627 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f88a015d600 con 0x7f88a0070ac0 2023-12-01T01:51:17.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.573+0000 7f889d7fa640 1 -- 172.21.15.115:0/3380380627 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v561) v1 ==== 159+0+0 (secure 0 0 0) 0x7f88a015d600 con 0x7f88a0070ac0 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 -- 172.21.15.115:0/3380380627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f888406a550 msgr2=0x7f888406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f888406a550 0x7f888406ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f88a00ae290 tx=0x7f888c008040 comp rx=0 tx=0).stop 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 -- 172.21.15.115:0/3380380627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a0070ac0 msgr2=0x7f88a01571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a0070ac0 0x7f88a01571c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f889402b8c0 tx=0x7f88940046c0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 -- 172.21.15.115:0/3380380627 shutdown_connections 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f888406a550 0x7f888406ca10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88a015d220 0x7f88a00b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88a00ad2b0 0x7f88a00ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 --2- 172.21.15.115:0/3380380627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88a0070ac0 0x7f88a01571c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 -- 172.21.15.115:0/3380380627 >> 172.21.15.115:0/3380380627 conn(0x7f88a00ce880 msgr2=0x7f88a0159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 -- 172.21.15.115:0/3380380627 shutdown_connections 2023-12-01T01:51:17.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.576+0000 7f88a6bc9640 1 -- 172.21.15.115:0/3380380627 wait complete. 2023-12-01T01:51:17.597 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T01:51:17.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.816+0000 7f5c7b0d7640 1 Processor -- start 2023-12-01T01:51:17.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.816+0000 7f5c7b0d7640 1 -- start start 2023-12-01T01:51:17.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.817+0000 7f5c7b0d7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c7415ac00 0x7f5c740ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:17.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.818+0000 7f5c7b0d7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5c740cb800 0x7f5c7406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:17.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.818+0000 7f5c7b0d7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7415afe0 con 0x7f5c740aede0 2023-12-01T01:51:17.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.818+0000 7f5c7b0d7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c740b12b0 con 0x7f5c740cb800 2023-12-01T01:51:17.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.818+0000 7f5c7b0d7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740b0a10 con 0x7f5c7415ac00 2023-12-01T01:51:17.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.818+0000 7f5c798d4640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5c740aede0 0x7f5c740cb0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58732/0 (socket says 172.21.15.115:58732) 2023-12-01T01:51:17.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.818+0000 7f5c798d4640 1 -- 172.21.15.115:0/302277784 learned_addr learned my addr 172.21.15.115:0/302277784 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:17.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.818+0000 7f5c7a8d6640 1 --2- 172.21.15.115:0/302277784 >> v2:172.21.15.142:3300/0 conn(0x7f5c740cb800 0x7f5c7406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:17.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.818+0000 7f5c7a0d5640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c7415ac00 0x7f5c740ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.818+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980956118 0 0) 0x7f5c7415afe0 con 0x7f5c740aede0 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.818+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c60003520 con 0x7f5c740aede0 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1460429415 0 0) 0x7f5c60003520 con 0x7f5c740aede0 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c7415ac00 msgr2=0x7f5c740ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c7415ac00 0x7f5c740ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 >> v2:172.21.15.142:3300/0 conn(0x7f5c740cb800 msgr2=0x7f5c7406fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 --2- 172.21.15.115:0/302277784 >> v2:172.21.15.142:3300/0 conn(0x7f5c740cb800 0x7f5c7406fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c740b0cf0 con 0x7f5c740aede0 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5c64001670 con 0x7f5c740aede0 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 >> v1:172.21.15.115:6789/0 conn(0x7f5c740aede0 legacy=0x7f5c740cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c7415ac00 0x7f5c60003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 0x7f5c600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:17.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c60004d00 0x7f5c60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740b12b0 con 0x7f5c60003ef0 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740b0a10 con 0x7f5c60004d00 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6000b9d0 con 0x7f5c7415ac00 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.819+0000 7f5c798d4640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 0x7f5c600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.820+0000 7f5c7a8d6640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c60004d00 0x7f5c60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.820+0000 7f5c798d4640 1 -- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c7415ac00 msgr2=0x7f5c60003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.820+0000 7f5c798d4640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c7415ac00 0x7f5c60003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.820+0000 7f5c798d4640 1 -- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c60004d00 msgr2=0x7f5c60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.820+0000 7f5c798d4640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c60004d00 0x7f5c60005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.820+0000 7f5c798d4640 1 -- 172.21.15.115:0/302277784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c60003520 con 0x7f5c60003ef0 2023-12-01T01:51:17.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.820+0000 7f5c7a8d6640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c60004d00 0x7f5c60005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.821+0000 7f5c798d4640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 0x7f5c600047c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400c270 tx=0x7f5c600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e9467b1e830f8c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.821+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6402cae0 con 0x7f5c60003ef0 2023-12-01T01:51:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.821+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6402cc40 con 0x7f5c60003ef0 2023-12-01T01:51:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.821+0000 7f5c790d3640 1 -- 172.21.15.115:0/302277784 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64005af0 con 0x7f5c60003ef0 2023-12-01T01:51:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.822+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/302277784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 msgr2=0x7f5c600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.822+0000 7f5c7b0d7640 1 --2- 172.21.15.115:0/302277784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 0x7f5c600047c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400c270 tx=0x7f5c600057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.822+0000 7f5c7a8d6640 1 -- 172.21.15.115:0/302277784 reap_dead start 2023-12-01T01:51:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.822+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/302277784 shutdown_connections 2023-12-01T01:51:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.822+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/302277784 >> 172.21.15.115:0/302277784 conn(0x7f5c74065960 msgr2=0x7f5c74065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.822+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/302277784 shutdown_connections 2023-12-01T01:51:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.822+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/302277784 wait complete. 2023-12-01T01:51:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.824+0000 7f5c7b0d7640 1 Processor -- start 2023-12-01T01:51:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.824+0000 7f5c7b0d7640 1 -- start start 2023-12-01T01:51:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.825+0000 7f5c7b0d7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 0x7f5c741585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.825+0000 7f5c7b0d7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7415baa0 0x7f5c741f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.825+0000 7f5c7b0d7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c741f5c50 0x7f5c740704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.825+0000 7f5c7b0d7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7415b680 con 0x7f5c60003ef0 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.825+0000 7f5c7b0d7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740c7750 con 0x7f5c7415baa0 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.825+0000 7f5c7b0d7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7415afe0 con 0x7f5c741f5c50 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.825+0000 7f5c7a0d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 0x7f5c741585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.825+0000 7f5c7a0d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 0x7f5c741585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56118/0 (socket says 172.21.15.115:56118) 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.825+0000 7f5c7a0d5640 1 -- 172.21.15.115:0/3899116143 learned_addr learned my addr 172.21.15.115:0/3899116143 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c798d4640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7415baa0 0x7f5c741f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c7a0d5640 1 -- 172.21.15.115:0/3899116143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c741f5c50 msgr2=0x7f5c740704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c7a0d5640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c741f5c50 0x7f5c740704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c7a0d5640 1 -- 172.21.15.115:0/3899116143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7415baa0 msgr2=0x7f5c741f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c7a0d5640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7415baa0 0x7f5c741f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c7a0d5640 1 -- 172.21.15.115:0/3899116143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74158b30 con 0x7f5c60003ef0 2023-12-01T01:51:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c798d4640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7415baa0 0x7f5c741f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c7a0d5640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 0x7f5c741585f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000e7f0 tx=0x7f5c7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c6affd640 1 -- 172.21.15.115:0/3899116143 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000c8c0 con 0x7f5c60003ef0 2023-12-01T01:51:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c6affd640 1 -- 172.21.15.115:0/3899116143 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7000ca20 con 0x7f5c60003ef0 2023-12-01T01:51:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c6affd640 1 -- 172.21.15.115:0/3899116143 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000cdb0 con 0x7f5c60003ef0 2023-12-01T01:51:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/3899116143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c74159020 con 0x7f5c60003ef0 2023-12-01T01:51:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.826+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/3899116143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c741f6540 con 0x7f5c60003ef0 2023-12-01T01:51:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.828+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/3899116143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7415b8f0 con 0x7f5c60003ef0 2023-12-01T01:51:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.828+0000 7f5c6affd640 1 -- 172.21.15.115:0/3899116143 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f5c70013470 con 0x7f5c60003ef0 2023-12-01T01:51:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.829+0000 7f5c6affd640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5c5806a580 0x7f5c5806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.830+0000 7f5c6affd640 1 -- 172.21.15.115:0/3899116143 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f5c700889d0 con 0x7f5c60003ef0 2023-12-01T01:51:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.830+0000 7f5c798d4640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5c5806a580 0x7f5c5806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.830+0000 7f5c798d4640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5c5806a580 0x7f5c5806ca40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5c64034000 tx=0x7f5c64007000 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:17.834+0000 7f5c6affd640 1 -- 172.21.15.115:0/3899116143 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5c70053eb0 con 0x7f5c60003ef0 2023-12-01T01:51:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.024+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/3899116143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5c740618d0 con 0x7f5c60003ef0 2023-12-01T01:51:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.030+0000 7f5c6affd640 1 -- 172.21.15.115:0/3899116143 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5c700170c0 con 0x7f5c60003ef0 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/3899116143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5c5806a580 msgr2=0x7f5c5806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5c5806a580 0x7f5c5806ca40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5c64034000 tx=0x7f5c64007000 comp rx=0 tx=0).stop 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/3899116143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 msgr2=0x7f5c741585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 0x7f5c741585f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000e7f0 tx=0x7f5c7000ecb0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/3899116143 shutdown_connections 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5c5806a580 0x7f5c5806ca40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c741f5c50 0x7f5c740704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7415baa0 0x7f5c741f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 --2- 172.21.15.115:0/3899116143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5c60003ef0 0x7f5c741585f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/3899116143 >> 172.21.15.115:0/3899116143 conn(0x7f5c74065960 msgr2=0x7f5c740cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/3899116143 shutdown_connections 2023-12-01T01:51:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.033+0000 7f5c7b0d7640 1 -- 172.21.15.115:0/3899116143 wait complete. 2023-12-01T01:51:18.054 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:51:18.054 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:51:18.167 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:51:18.296 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:51:18.296 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:51:18.296 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:18.356 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:51:18.356 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:51:18.356 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:18.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:51:18.416 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:51:18.416 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:18.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:51:18.475 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:51:18.475 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:18.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:51:18.535 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T01:51:18.535 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:18.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.756+0000 7ff956fc9640 1 Processor -- start 2023-12-01T01:51:18.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.756+0000 7ff956fc9640 1 -- start start 2023-12-01T01:51:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.757+0000 7ff956fc9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff95015e2f0 0x7ff9500ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.758+0000 7ff956fc9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff9500adf10 0x7ff9500caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.758+0000 7ff956fc9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9500add10 con 0x7ff95006f2f0 2023-12-01T01:51:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.758+0000 7ff956fc9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9500c5760 con 0x7ff9500adf10 2023-12-01T01:51:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.758+0000 7ff956fc9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95015e6d0 con 0x7ff95015e2f0 2023-12-01T01:51:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.758+0000 7ff94ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff95006f2f0 0x7ff9500c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58742/0 (socket says 172.21.15.115:58742) 2023-12-01T01:51:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.758+0000 7ff94ffff640 1 -- 172.21.15.115:0/1021712743 learned_addr learned my addr 172.21.15.115:0/1021712743 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:18.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.758+0000 7ff955566640 1 --2- 172.21.15.115:0/1021712743 >> v2:172.21.15.142:3300/0 conn(0x7ff9500adf10 0x7ff9500caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:18.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.758+0000 7ff954d65640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff95015e2f0 0x7ff9500ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:18.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.758+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633061609 0 0) 0x7ff9500add10 con 0x7ff95006f2f0 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.758+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff940003520 con 0x7ff95006f2f0 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252844397 0 0) 0x7ff940003520 con 0x7ff95006f2f0 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff95015e2f0 msgr2=0x7ff9500ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff95015e2f0 0x7ff9500ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 >> v2:172.21.15.142:3300/0 conn(0x7ff9500adf10 msgr2=0x7ff9500caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 --2- 172.21.15.115:0/1021712743 >> v2:172.21.15.142:3300/0 conn(0x7ff9500adf10 0x7ff9500caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500cf7b0 con 0x7ff95006f2f0 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7ff93c001670 con 0x7ff95006f2f0 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 >> v1:172.21.15.115:6789/0 conn(0x7ff95006f2f0 legacy=0x7ff9500c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff95015e2f0 0x7ff9400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff940003f60 0x7ff9400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 0x7ff940005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500c5760 con 0x7ff940004d00 2023-12-01T01:51:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95015e6d0 con 0x7ff940003f60 2023-12-01T01:51:18.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94000bae0 con 0x7ff95015e2f0 2023-12-01T01:51:18.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.759+0000 7ff955566640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 0x7ff940005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:18.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.760+0000 7ff94ffff640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff940003f60 0x7ff9400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:18.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.760+0000 7ff955566640 1 -- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff95015e2f0 msgr2=0x7ff9400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:18.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.760+0000 7ff954d65640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff95015e2f0 0x7ff9400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:18.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.760+0000 7ff955566640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff95015e2f0 0x7ff9400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.760+0000 7ff955566640 1 -- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff940003f60 msgr2=0x7ff9400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:18.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.760+0000 7ff955566640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff940003f60 0x7ff9400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.760+0000 7ff955566640 1 -- 172.21.15.115:0/1021712743 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940003520 con 0x7ff940004d00 2023-12-01T01:51:18.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.760+0000 7ff954d65640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff95015e2f0 0x7ff9400039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:18.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.761+0000 7ff955566640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 0x7ff940005180 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff948009f90 tx=0x7ff9400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd15563eb3504c6b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:18.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.761+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94802cc20 con 0x7ff940004d00 2023-12-01T01:51:18.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.761+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94800f040 con 0x7ff940004d00 2023-12-01T01:51:18.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.761+0000 7ff94f7fe640 1 -- 172.21.15.115:0/1021712743 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff948034440 con 0x7ff940004d00 2023-12-01T01:51:18.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.762+0000 7ff956fc9640 1 -- 172.21.15.115:0/1021712743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 msgr2=0x7ff940005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:18.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.762+0000 7ff956fc9640 1 --2- 172.21.15.115:0/1021712743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 0x7ff940005180 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff948009f90 tx=0x7ff9400057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.762+0000 7ff955566640 1 -- 172.21.15.115:0/1021712743 reap_dead start 2023-12-01T01:51:18.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.762+0000 7ff956fc9640 1 -- 172.21.15.115:0/1021712743 shutdown_connections 2023-12-01T01:51:18.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.762+0000 7ff956fc9640 1 -- 172.21.15.115:0/1021712743 >> 172.21.15.115:0/1021712743 conn(0x7ff9500ce880 msgr2=0x7ff9500cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:18.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.762+0000 7ff956fc9640 1 -- 172.21.15.115:0/1021712743 shutdown_connections 2023-12-01T01:51:18.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.762+0000 7ff956fc9640 1 -- 172.21.15.115:0/1021712743 wait complete. 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.764+0000 7ff956fc9640 1 Processor -- start 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.764+0000 7ff956fc9640 1 -- start start 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff956fc9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 0x7ff95015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff956fc9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9500c4080 0x7ff950152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff956fc9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9500c7a10 0x7ff9500c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff956fc9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500c83d0 con 0x7ff940004d00 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff956fc9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500c45e0 con 0x7ff9500c7a10 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff956fc9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500c48c0 con 0x7ff9500c4080 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff954d65640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 0x7ff95015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff954d65640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 0x7ff95015d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56142/0 (socket says 172.21.15.115:56142) 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff954d65640 1 -- 172.21.15.115:0/2262648124 learned_addr learned my addr 172.21.15.115:0/2262648124 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff954d65640 1 -- 172.21.15.115:0/2262648124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9500c4080 msgr2=0x7ff950152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:18.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff954d65640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9500c4080 0x7ff950152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff954d65640 1 -- 172.21.15.115:0/2262648124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9500c7a10 msgr2=0x7ff9500c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:18.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff954d65640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9500c7a10 0x7ff9500c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:18.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.765+0000 7ff954d65640 1 -- 172.21.15.115:0/2262648124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500c4ba0 con 0x7ff940004d00 2023-12-01T01:51:18.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.766+0000 7ff954d65640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 0x7ff95015d660 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff9440108b0 tx=0x7ff944010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:18.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.766+0000 7ff94d7fa640 1 -- 172.21.15.115:0/2262648124 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94400eb60 con 0x7ff940004d00 2023-12-01T01:51:18.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.766+0000 7ff94d7fa640 1 -- 172.21.15.115:0/2262648124 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94400ecc0 con 0x7ff940004d00 2023-12-01T01:51:18.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.766+0000 7ff956fc9640 1 -- 172.21.15.115:0/2262648124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9500c4e80 con 0x7ff940004d00 2023-12-01T01:51:18.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.766+0000 7ff94d7fa640 1 -- 172.21.15.115:0/2262648124 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94401c400 con 0x7ff940004d00 2023-12-01T01:51:18.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.766+0000 7ff956fc9640 1 -- 172.21.15.115:0/2262648124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9500c5160 con 0x7ff940004d00 2023-12-01T01:51:18.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.767+0000 7ff956fc9640 1 -- 172.21.15.115:0/2262648124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95006fda0 con 0x7ff940004d00 2023-12-01T01:51:18.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.768+0000 7ff94d7fa640 1 -- 172.21.15.115:0/2262648124 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff94401c560 con 0x7ff940004d00 2023-12-01T01:51:18.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.769+0000 7ff94d7fa640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 0x7ff93406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:18.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.769+0000 7ff94d7fa640 1 -- 172.21.15.115:0/2262648124 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5545+0+0 (secure 0 0 0) 0x7ff944088f80 con 0x7ff940004d00 2023-12-01T01:51:18.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.769+0000 7ff94ffff640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 0x7ff93406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:18.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.770+0000 7ff94ffff640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 0x7ff93406ca40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff93c000f60 tx=0x7ff93c001df0 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:18.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.773+0000 7ff94d7fa640 1 -- 172.21.15.115:0/2262648124 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff9440545a0 con 0x7ff940004d00 2023-12-01T01:51:19.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:18.998+0000 7ff956fc9640 1 -- 172.21.15.115:0/2262648124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff95004e890 con 0x7ff940004d00 2023-12-01T01:51:19.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.000+0000 7ff94d7fa640 1 -- 172.21.15.115:0/2262648124 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3541 (secure 0 0 0) 0x7ff944021df0 con 0x7ff940004d00 2023-12-01T01:51:19.001 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:19.001 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/2548453767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2548453767},{"type":"v1","addr":"172.21.15.142:6834","nonce":2548453767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":4871,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3727358840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3727358840},{"type":"v1","addr":"172.21.15.142:6835","nonce":3727358840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5928,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/4086473290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":4086473290},{"type":"v1","addr":"172.21.15.115:6836","nonce":4086473290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6360,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3568275811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3568275811},{"type":"v1","addr":"172.21.15.115:6837","nonce":3568275811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-12-01T01:51:19.002 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 26 2023-12-01T01:51:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 -- 172.21.15.115:0/2262648124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 msgr2=0x7ff93406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 0x7ff93406ca40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff93c000f60 tx=0x7ff93c001df0 comp rx=0 tx=0).stop 2023-12-01T01:51:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 -- 172.21.15.115:0/2262648124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 msgr2=0x7ff95015d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 0x7ff95015d660 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff9440108b0 tx=0x7ff944010d70 comp rx=0 tx=0).stop 2023-12-01T01:51:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 -- 172.21.15.115:0/2262648124 shutdown_connections 2023-12-01T01:51:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff93406a580 0x7ff93406ca40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff9500c4080 0x7ff950152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9500c7a10 0x7ff9500c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 --2- 172.21.15.115:0/2262648124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff940004d00 0x7ff95015d660 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 -- 172.21.15.115:0/2262648124 >> 172.21.15.115:0/2262648124 conn(0x7ff9500ce880 msgr2=0x7ff9500ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:19.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 -- 172.21.15.115:0/2262648124 shutdown_connections 2023-12-01T01:51:19.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.002+0000 7ff956fc9640 1 -- 172.21.15.115:0/2262648124 wait complete. 2023-12-01T01:51:19.023 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T01:51:19.023 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T01:51:19.023 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T01:51:19.023 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:51:19.023 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T01:51:19.024 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T01:51:19.024 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T01:51:19.024 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:51:19.024 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T01:51:19.024 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T01:51:19.024 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T01:51:19.024 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:51:19.025 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T01:51:19.025 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T01:51:19.025 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T01:51:19.025 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:51:19.039 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T01:51:19.039 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T01:51:19.041 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T01:51:19.041 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T01:51:19.044 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T01:51:19.044 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T01:51:19.054 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T01:51:19.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T01:51:19.087 INFO:tasks.ceph.mds.a:Started 2023-12-01T01:51:19.088 DEBUG:teuthology.parallel:result is None 2023-12-01T01:51:19.093 INFO:tasks.ceph.mds.c:Started 2023-12-01T01:51:19.093 DEBUG:teuthology.parallel:result is None 2023-12-01T01:51:19.102 INFO:tasks.ceph.mds.b:Started 2023-12-01T01:51:19.106 DEBUG:teuthology.parallel:result is None 2023-12-01T01:51:19.113 INFO:tasks.ceph.mds.d:Started 2023-12-01T01:51:19.113 DEBUG:teuthology.parallel:result is None 2023-12-01T01:51:19.114 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T01:51:19.238 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T01:51:19.239 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:51:19.270 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T01:51:19.291 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T01:51:19.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.296+0000 7fdfb759e640 1 Processor -- start 2023-12-01T01:51:19.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.296+0000 7fdfb759e640 1 -- start start 2023-12-01T01:51:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.297+0000 7fdfb759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb815cfa0 0x7fdfb80c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.297+0000 7fdfb759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdfb806d920 0x7fdfb806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.297+0000 7fdfb759e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb80c4a00 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.297+0000 7fdfb759e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb806e310 con 0x7fdfb806d920 2023-12-01T01:51:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.297+0000 7fdfb759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80cecf0 con 0x7fdfb815cfa0 2023-12-01T01:51:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.297+0000 7fdfb659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb815cfa0 0x7fdfb80c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.297+0000 7fdfb6d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdfb806d920 0x7fdfb806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.298+0000 7fdfb659c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb815cfa0 0x7fdfb80c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:34910/0 (socket says 172.21.15.115:34910) 2023-12-01T01:51:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.298+0000 7fdfb6d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdfb806d920 0x7fdfb806ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:46008/0 (socket says 172.21.15.115:46008) 2023-12-01T01:51:19.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.298+0000 7fdfb659c640 1 -- 172.21.15.115:0/2705437692 learned_addr learned my addr 172.21.15.115:0/2705437692 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.298+0000 7fdfb659c640 1 -- 172.21.15.115:0/2705437692 >> v2:172.21.15.142:3300/0 conn(0x7fdfb806d920 msgr2=0x7fdfb806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.298+0000 7fdfb659c640 1 --2- 172.21.15.115:0/2705437692 >> v2:172.21.15.142:3300/0 conn(0x7fdfb806d920 0x7fdfb806ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.298+0000 7fdfb659c640 1 -- 172.21.15.115:0/2705437692 >> v1:172.21.15.142:6789/0 conn(0x7fdfb80cdfd0 legacy=0x7fdfb80ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.298+0000 7fdfb659c640 1 -- 172.21.15.115:0/2705437692 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb80cef50 con 0x7fdfb815cfa0 2023-12-01T01:51:19.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.299+0000 7fdfb659c640 1 --2- 172.21.15.115:0/2705437692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb815cfa0 0x7fdfb80c68c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009870 tx=0x7fdfa802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd76d9aa95c03bf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:19.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.304+0000 7fdfb559a640 1 -- 172.21.15.115:0/2705437692 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8030840 con 0x7fdfb815cfa0 2023-12-01T01:51:19.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.304+0000 7fdfb559a640 1 -- 172.21.15.115:0/2705437692 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa80309a0 con 0x7fdfb815cfa0 2023-12-01T01:51:19.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.304+0000 7fdfb559a640 1 -- 172.21.15.115:0/2705437692 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8030d30 con 0x7fdfb815cfa0 2023-12-01T01:51:19.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.305+0000 7fdfb759e640 1 -- 172.21.15.115:0/2705437692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb815cfa0 msgr2=0x7fdfb80c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:19.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.305+0000 7fdfb759e640 1 --2- 172.21.15.115:0/2705437692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb815cfa0 0x7fdfb80c68c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009870 tx=0x7fdfa802b980 comp rx=0 tx=0).stop 2023-12-01T01:51:19.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.305+0000 7fdfb759e640 1 -- 172.21.15.115:0/2705437692 shutdown_connections 2023-12-01T01:51:19.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.305+0000 7fdfb759e640 1 --2- 172.21.15.115:0/2705437692 >> v2:172.21.15.142:3300/0 conn(0x7fdfb806d920 0x7fdfb806ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:19.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.305+0000 7fdfb759e640 1 --2- 172.21.15.115:0/2705437692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb815cfa0 0x7fdfb80c68c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:19.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.305+0000 7fdfb759e640 1 -- 172.21.15.115:0/2705437692 >> 172.21.15.115:0/2705437692 conn(0x7fdfb80cf7f0 msgr2=0x7fdfb80cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:19.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.305+0000 7fdfb759e640 1 -- 172.21.15.115:0/2705437692 shutdown_connections 2023-12-01T01:51:19.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.305+0000 7fdfb759e640 1 -- 172.21.15.115:0/2705437692 wait complete. 2023-12-01T01:51:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.307+0000 7fdfb759e640 1 Processor -- start 2023-12-01T01:51:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.307+0000 7fdfb759e640 1 -- start start 2023-12-01T01:51:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb806d920 0x7fdfb8149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfb80cdfd0 0x7fdfb80b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfb815cfa0 0x7fdfb81548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb759e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80b4cd0 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb759e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80b40c0 con 0x7fdfb815cfa0 2023-12-01T01:51:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80b43a0 con 0x7fdfb806d920 2023-12-01T01:51:19.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfb80cdfd0 0x7fdfb80b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfb80cdfd0 0x7fdfb80b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56164/0 (socket says 172.21.15.115:56164) 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb5d9b640 1 -- 172.21.15.115:0/3453512849 learned_addr learned my addr 172.21.15.115:0/3453512849 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb659c640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb806d920 0x7fdfb8149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.308+0000 7fdfb6d9d640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfb815cfa0 0x7fdfb81548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfb5d9b640 1 -- 172.21.15.115:0/3453512849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb806d920 msgr2=0x7fdfb8149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfb5d9b640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb806d920 0x7fdfb8149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfb5d9b640 1 -- 172.21.15.115:0/3453512849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfb815cfa0 msgr2=0x7fdfb81548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfb5d9b640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfb815cfa0 0x7fdfb81548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfb5d9b640 1 -- 172.21.15.115:0/3453512849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa80094d0 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfb6d9d640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfb815cfa0 0x7fdfb81548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfb5d9b640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfb80cdfd0 0x7fdfb80b5160 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c450 tx=0x7fdfac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfa6ffd640 1 -- 172.21.15.115:0/3453512849 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0052d0 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfa6ffd640 1 -- 172.21.15.115:0/3453512849 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac005430 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfa6ffd640 1 -- 172.21.15.115:0/3453512849 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac012870 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfb759e640 1 -- 172.21.15.115:0/3453512849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb80b4680 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.309+0000 7fdfb759e640 1 -- 172.21.15.115:0/3453512849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb8149cc0 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.311+0000 7fdfb759e640 1 -- 172.21.15.115:0/3453512849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb8149e80 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.311+0000 7fdfa6ffd640 1 -- 172.21.15.115:0/3453512849 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fdfac0129d0 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.312+0000 7fdfa6ffd640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdf9806a550 0x7fdf9806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:19.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.312+0000 7fdfa6ffd640 1 -- 172.21.15.115:0/3453512849 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5545+0+0 (secure 0 0 0) 0x7fdfac088360 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.312+0000 7fdfb659c640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdf9806a550 0x7fdf9806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:19.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.313+0000 7fdfb659c640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdf9806a550 0x7fdf9806ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8036000 tx=0x7fdfa8008030 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:19.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.316+0000 7fdfa6ffd640 1 -- 172.21.15.115:0/3453512849 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdfac053a20 con 0x7fdfb80cdfd0 2023-12-01T01:51:19.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:19.518+0000 7fdfb759e640 1 -- 172.21.15.115:0/3453512849 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fdfb80673a0 con 0x7fdfb80cdfd0 2023-12-01T01:51:20.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.313+0000 7fdfa6ffd640 1 -- 172.21.15.115:0/3453512849 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdfac057040 con 0x7fdfb80cdfd0 2023-12-01T01:51:20.315 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T01:51:20.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 -- 172.21.15.115:0/3453512849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdf9806a550 msgr2=0x7fdf9806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:20.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdf9806a550 0x7fdf9806ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8036000 tx=0x7fdfa8008030 comp rx=0 tx=0).stop 2023-12-01T01:51:20.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 -- 172.21.15.115:0/3453512849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfb80cdfd0 msgr2=0x7fdfb80b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:20.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfb80cdfd0 0x7fdfb80b5160 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c450 tx=0x7fdfac00c910 comp rx=0 tx=0).stop 2023-12-01T01:51:20.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 -- 172.21.15.115:0/3453512849 shutdown_connections 2023-12-01T01:51:20.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdf9806a550 0x7fdf9806ca10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfb815cfa0 0x7fdfb81548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdfb80cdfd0 0x7fdfb80b5160 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 --2- 172.21.15.115:0/3453512849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdfb806d920 0x7fdfb8149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 -- 172.21.15.115:0/3453512849 >> 172.21.15.115:0/3453512849 conn(0x7fdfb80cf7f0 msgr2=0x7fdfb80ccaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:20.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.316+0000 7fdfb759e640 1 -- 172.21.15.115:0/3453512849 shutdown_connections 2023-12-01T01:51:20.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.317+0000 7fdfb759e640 1 -- 172.21.15.115:0/3453512849 wait complete. 2023-12-01T01:51:20.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T01:51:20.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.551+0000 7fa2dca17640 1 Processor -- start 2023-12-01T01:51:20.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.551+0000 7fa2dca17640 1 -- start start 2023-12-01T01:51:20.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.553+0000 7fa2dca17640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d815e2f0 0x7fa2d80ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:20.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.553+0000 7fa2dca17640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa2d806f2f0 0x7fa2d80caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:20.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.553+0000 7fa2dca17640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d80add10 con 0x7fa2d80adf10 2023-12-01T01:51:20.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.553+0000 7fa2dca17640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d80c5760 con 0x7fa2d806f2f0 2023-12-01T01:51:20.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.553+0000 7fa2dca17640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d815e6d0 con 0x7fa2d815e2f0 2023-12-01T01:51:20.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.553+0000 7fa2d5d9b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa2d806f2f0 0x7fa2d80caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:20.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.553+0000 7fa2d5d9b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa2d806f2f0 0x7fa2d80caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:46038/0 (socket says 172.21.15.115:46038) 2023-12-01T01:51:20.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.553+0000 7fa2d5d9b640 1 -- 172.21.15.115:0/2292910091 learned_addr learned my addr 172.21.15.115:0/2292910091 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:20.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.554+0000 7fa2d659c640 1 --2- 172.21.15.115:0/2292910091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d815e2f0 0x7fa2d80ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.554+0000 7fa2d5d9b640 1 -- 172.21.15.115:0/2292910091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d815e2f0 msgr2=0x7fa2d80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.554+0000 7fa2d5d9b640 1 --2- 172.21.15.115:0/2292910091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d815e2f0 0x7fa2d80ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.554+0000 7fa2d5d9b640 1 -- 172.21.15.115:0/2292910091 >> v1:172.21.15.142:6789/0 conn(0x7fa2d80adf10 legacy=0x7fa2d80c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.554+0000 7fa2d559a640 1 -- 172.21.15.115:0/2292910091 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253444341 0 0) 0x7fa2d80add10 con 0x7fa2d80adf10 2023-12-01T01:51:20.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.554+0000 7fa2d5d9b640 1 -- 172.21.15.115:0/2292910091 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80cf7b0 con 0x7fa2d806f2f0 2023-12-01T01:51:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.556+0000 7fa2d5d9b640 1 --2- 172.21.15.115:0/2292910091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d806f2f0 0x7fa2d80caba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009870 tx=0x7fa2cc02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd847e7415197eb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.556+0000 7fa2d559a640 1 -- 172.21.15.115:0/2292910091 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0317c0 con 0x7fa2d806f2f0 2023-12-01T01:51:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.556+0000 7fa2d559a640 1 -- 172.21.15.115:0/2292910091 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc031920 con 0x7fa2d806f2f0 2023-12-01T01:51:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.556+0000 7fa2d559a640 1 -- 172.21.15.115:0/2292910091 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc031cb0 con 0x7fa2d806f2f0 2023-12-01T01:51:20.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.557+0000 7fa2dca17640 1 -- 172.21.15.115:0/2292910091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d806f2f0 msgr2=0x7fa2d80caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:20.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.557+0000 7fa2dca17640 1 --2- 172.21.15.115:0/2292910091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d806f2f0 0x7fa2d80caba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009870 tx=0x7fa2cc02b860 comp rx=0 tx=0).stop 2023-12-01T01:51:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.557+0000 7fa2dca17640 1 -- 172.21.15.115:0/2292910091 shutdown_connections 2023-12-01T01:51:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.557+0000 7fa2dca17640 1 --2- 172.21.15.115:0/2292910091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d806f2f0 0x7fa2d80caba0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.557+0000 7fa2dca17640 1 --2- 172.21.15.115:0/2292910091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d815e2f0 0x7fa2d80ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.557+0000 7fa2dca17640 1 -- 172.21.15.115:0/2292910091 >> 172.21.15.115:0/2292910091 conn(0x7fa2d80ce880 msgr2=0x7fa2d80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.557+0000 7fa2dca17640 1 -- 172.21.15.115:0/2292910091 shutdown_connections 2023-12-01T01:51:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.557+0000 7fa2dca17640 1 -- 172.21.15.115:0/2292910091 wait complete. 2023-12-01T01:51:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.559+0000 7fa2dca17640 1 Processor -- start 2023-12-01T01:51:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.559+0000 7fa2dca17640 1 -- start start 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2dca17640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d806f2f0 0x7fa2d80c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2dca17640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d80adf10 0x7fa2d80c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2dca17640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d815e2f0 0x7fa2d8158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2dca17640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81f31a0 con 0x7fa2d806f2f0 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2dca17640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d815d940 con 0x7fa2d815e2f0 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2dca17640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81f2520 con 0x7fa2d80adf10 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2d659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d806f2f0 0x7fa2d80c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2d659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d806f2f0 0x7fa2d80c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56180/0 (socket says 172.21.15.115:56180) 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2d659c640 1 -- 172.21.15.115:0/1967320823 learned_addr learned my addr 172.21.15.115:0/1967320823 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2d5d9b640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d80adf10 0x7fa2d80c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.560+0000 7fa2d6d9d640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d815e2f0 0x7fa2d8158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2d659c640 1 -- 172.21.15.115:0/1967320823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d80adf10 msgr2=0x7fa2d80c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2d659c640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d80adf10 0x7fa2d80c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2d659c640 1 -- 172.21.15.115:0/1967320823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d815e2f0 msgr2=0x7fa2d8158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2d659c640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d815e2f0 0x7fa2d8158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2d659c640 1 -- 172.21.15.115:0/1967320823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0094d0 con 0x7fa2d806f2f0 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2d5d9b640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d80adf10 0x7fa2d80c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2d6d9d640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d815e2f0 0x7fa2d8158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2d659c640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d806f2f0 0x7fa2d80c4e80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e870 tx=0x7fa2c800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2c6ffd640 1 -- 172.21.15.115:0/1967320823 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80094f0 con 0x7fa2d806f2f0 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2c6ffd640 1 -- 172.21.15.115:0/1967320823 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c800c940 con 0x7fa2d806f2f0 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2c6ffd640 1 -- 172.21.15.115:0/1967320823 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800ccf0 con 0x7fa2d806f2f0 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2dca17640 1 -- 172.21.15.115:0/1967320823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d81f2800 con 0x7fa2d806f2f0 2023-12-01T01:51:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.561+0000 7fa2dca17640 1 -- 172.21.15.115:0/1967320823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d81f2dc0 con 0x7fa2d806f2f0 2023-12-01T01:51:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.563+0000 7fa2dca17640 1 -- 172.21.15.115:0/1967320823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d806fda0 con 0x7fa2d806f2f0 2023-12-01T01:51:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.563+0000 7fa2c6ffd640 1 -- 172.21.15.115:0/1967320823 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa2c8013470 con 0x7fa2d806f2f0 2023-12-01T01:51:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.564+0000 7fa2c6ffd640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2b806a550 0x7fa2b806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.564+0000 7fa2c6ffd640 1 -- 172.21.15.115:0/1967320823 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa2c8088910 con 0x7fa2d806f2f0 2023-12-01T01:51:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.565+0000 7fa2d5d9b640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2b806a550 0x7fa2b806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.565+0000 7fa2d5d9b640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2b806a550 0x7fa2b806ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc036000 tx=0x7fa2cc03d040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:20.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.569+0000 7fa2c6ffd640 1 -- 172.21.15.115:0/1967320823 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa2c80540a0 con 0x7fa2d806f2f0 2023-12-01T01:51:20.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.830+0000 7fa2dca17640 1 -- 172.21.15.115:0/1967320823 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa2d80cc450 con 0x7fa2d806f2f0 2023-12-01T01:51:20.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.830+0000 7fa2c6ffd640 1 -- 172.21.15.115:0/1967320823 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa2c8021090 con 0x7fa2d806f2f0 2023-12-01T01:51:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:20.832 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:51:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T01:51:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T01:51:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T01:51:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T01:51:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:51:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T01:51:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T01:51:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:51:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:51:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T01:51:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T01:51:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:51:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:51:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T01:51:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:51:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T01:51:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T01:51:20.846 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:51:20.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 -- 172.21.15.115:0/1967320823 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2b806a550 msgr2=0x7fa2b806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2b806a550 0x7fa2b806ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc036000 tx=0x7fa2cc03d040 comp rx=0 tx=0).stop 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 -- 172.21.15.115:0/1967320823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d806f2f0 msgr2=0x7fa2d80c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d806f2f0 0x7fa2d80c4e80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e870 tx=0x7fa2c800ed30 comp rx=0 tx=0).stop 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 -- 172.21.15.115:0/1967320823 shutdown_connections 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2b806a550 0x7fa2b806ca10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2d80adf10 0x7fa2d80c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d815e2f0 0x7fa2d8158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 --2- 172.21.15.115:0/1967320823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d806f2f0 0x7fa2d80c4e80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 -- 172.21.15.115:0/1967320823 >> 172.21.15.115:0/1967320823 conn(0x7fa2d80ce880 msgr2=0x7fa2d80afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 -- 172.21.15.115:0/1967320823 shutdown_connections 2023-12-01T01:51:20.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:20.833+0000 7fa2dca17640 1 -- 172.21.15.115:0/1967320823 wait complete. 2023-12-01T01:51:20.853 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:21.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.050+0000 7f8d74dcb640 1 Processor -- start 2023-12-01T01:51:21.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.051+0000 7f8d74dcb640 1 -- start start 2023-12-01T01:51:21.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.052+0000 7f8d74dcb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8d7015cd20 0x7f8d700d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.052+0000 7f8d74dcb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8d700b5d80 0x7f8d700b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.052+0000 7f8d74dcb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700ce2e0 con 0x7f8d700d1230 2023-12-01T01:51:21.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.052+0000 7f8d74dcb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d700b2160 con 0x7f8d7015cd20 2023-12-01T01:51:21.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.052+0000 7f8d74dcb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d700d2a10 con 0x7f8d700b5d80 2023-12-01T01:51:21.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.052+0000 7f8d6e59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8d7015cd20 0x7f8d700d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.052+0000 7f8d6e59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8d7015cd20 0x7f8d700d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56194/0 (socket says 172.21.15.115:56194) 2023-12-01T01:51:21.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.052+0000 7f8d6e59c640 1 -- 172.21.15.115:0/2398056319 learned_addr learned my addr 172.21.15.115:0/2398056319 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:21.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.053+0000 7f8d6e59c640 1 -- 172.21.15.115:0/2398056319 >> v2:172.21.15.142:3300/0 conn(0x7f8d700b5d80 msgr2=0x7f8d700b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:21.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.053+0000 7f8d6dd9b640 1 --2- 172.21.15.115:0/2398056319 >> v2:172.21.15.142:3300/0 conn(0x7f8d700b5d80 0x7f8d700b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.053+0000 7f8d6e59c640 1 --2- 172.21.15.115:0/2398056319 >> v2:172.21.15.142:3300/0 conn(0x7f8d700b5d80 0x7f8d700b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.053+0000 7f8d6e59c640 1 -- 172.21.15.115:0/2398056319 >> v1:172.21.15.142:6789/0 conn(0x7f8d700d1230 legacy=0x7f8d700d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:21.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.053+0000 7f8d6e59c640 1 -- 172.21.15.115:0/2398056319 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700ce000 con 0x7f8d7015cd20 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.054+0000 7f8d6e59c640 1 --2- 172.21.15.115:0/2398056319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 0x7f8d700d24d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8d700cf630 tx=0x7f8d6002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0054f01f8f94c43 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.054+0000 7f8d6d59a640 1 -- 172.21.15.115:0/2398056319 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6002c9f0 con 0x7f8d7015cd20 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.054+0000 7f8d6d59a640 1 -- 172.21.15.115:0/2398056319 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6002cb50 con 0x7f8d7015cd20 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.054+0000 7f8d6d59a640 1 -- 172.21.15.115:0/2398056319 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600343f0 con 0x7f8d7015cd20 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.055+0000 7f8d74dcb640 1 -- 172.21.15.115:0/2398056319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 msgr2=0x7f8d700d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.055+0000 7f8d74dcb640 1 --2- 172.21.15.115:0/2398056319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 0x7f8d700d24d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8d700cf630 tx=0x7f8d6002be50 comp rx=0 tx=0).stop 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.055+0000 7f8d74dcb640 1 -- 172.21.15.115:0/2398056319 shutdown_connections 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.055+0000 7f8d74dcb640 1 --2- 172.21.15.115:0/2398056319 >> v2:172.21.15.142:3300/0 conn(0x7f8d700b5d80 0x7f8d700b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.055+0000 7f8d74dcb640 1 --2- 172.21.15.115:0/2398056319 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 0x7f8d700d24d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.055+0000 7f8d74dcb640 1 -- 172.21.15.115:0/2398056319 >> 172.21.15.115:0/2398056319 conn(0x7f8d7006a0e0 msgr2=0x7f8d7006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.055+0000 7f8d74dcb640 1 -- 172.21.15.115:0/2398056319 shutdown_connections 2023-12-01T01:51:21.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.055+0000 7f8d74dcb640 1 -- 172.21.15.115:0/2398056319 wait complete. 2023-12-01T01:51:21.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.057+0000 7f8d74dcb640 1 Processor -- start 2023-12-01T01:51:21.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.058+0000 7f8d74dcb640 1 -- start start 2023-12-01T01:51:21.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d74dcb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d700b5d80 0x7f8d701f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d74dcb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d700d1230 0x7f8d70148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d74dcb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 0x7f8d70148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d74dcb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7015f770 con 0x7f8d7015cd20 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d74dcb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7015eb60 con 0x7f8d700d1230 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d74dcb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7015ee40 con 0x7f8d700b5d80 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d6ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 0x7f8d70148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d6ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 0x7f8d70148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56196/0 (socket says 172.21.15.115:56196) 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d6ed9d640 1 -- 172.21.15.115:0/798833844 learned_addr learned my addr 172.21.15.115:0/798833844 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d6dd9b640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d700d1230 0x7f8d70148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.059+0000 7f8d6e59c640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d700b5d80 0x7f8d701f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d6ed9d640 1 -- 172.21.15.115:0/798833844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d700b5d80 msgr2=0x7f8d701f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d6ed9d640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d700b5d80 0x7f8d701f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d6ed9d640 1 -- 172.21.15.115:0/798833844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d700d1230 msgr2=0x7f8d70148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d6ed9d640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d700d1230 0x7f8d70148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d6ed9d640 1 -- 172.21.15.115:0/798833844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d60009b30 con 0x7f8d7015cd20 2023-12-01T01:51:21.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d6ed9d640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 0x7f8d70148590 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c390 tx=0x7f8d6800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:21.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d66ffd640 1 -- 172.21.15.115:0/798833844 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d680035f0 con 0x7f8d7015cd20 2023-12-01T01:51:21.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d66ffd640 1 -- 172.21.15.115:0/798833844 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68003750 con 0x7f8d7015cd20 2023-12-01T01:51:21.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d66ffd640 1 -- 172.21.15.115:0/798833844 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68003b00 con 0x7f8d7015cd20 2023-12-01T01:51:21.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d74dcb640 1 -- 172.21.15.115:0/798833844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7015f120 con 0x7f8d7015cd20 2023-12-01T01:51:21.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.060+0000 7f8d74dcb640 1 -- 172.21.15.115:0/798833844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7015fc00 con 0x7f8d7015cd20 2023-12-01T01:51:21.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.061+0000 7f8d74dcb640 1 -- 172.21.15.115:0/798833844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7004e890 con 0x7f8d7015cd20 2023-12-01T01:51:21.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.062+0000 7f8d66ffd640 1 -- 172.21.15.115:0/798833844 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8d68003c60 con 0x7f8d7015cd20 2023-12-01T01:51:21.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.063+0000 7f8d66ffd640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8d5006a550 0x7f8d5006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.063+0000 7f8d66ffd640 1 -- 172.21.15.115:0/798833844 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d680879c0 con 0x7f8d7015cd20 2023-12-01T01:51:21.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.064+0000 7f8d6e59c640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8d5006a550 0x7f8d5006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.064+0000 7f8d6e59c640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8d5006a550 0x7f8d5006ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8d60035000 tx=0x7f8d60030000 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:21.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.067+0000 7f8d66ffd640 1 -- 172.21.15.115:0/798833844 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8d680531e0 con 0x7f8d7015cd20 2023-12-01T01:51:21.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.256+0000 7f8d74dcb640 1 -- 172.21.15.115:0/798833844 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d700cd3a0 con 0x7f8d7015cd20 2023-12-01T01:51:21.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.258+0000 7f8d66ffd640 1 -- 172.21.15.115:0/798833844 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+12437 (secure 0 0 0) 0x7f8d68017090 con 0x7f8d7015cd20 2023-12-01T01:51:21.259 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:21.260 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":79,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:20.305841+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:21.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.260+0000 7f8d74dcb640 1 -- 172.21.15.115:0/798833844 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8d5006a550 msgr2=0x7f8d5006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.260+0000 7f8d74dcb640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8d5006a550 0x7f8d5006ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8d60035000 tx=0x7f8d60030000 comp rx=0 tx=0).stop 2023-12-01T01:51:21.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.260+0000 7f8d74dcb640 1 -- 172.21.15.115:0/798833844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 msgr2=0x7f8d70148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.260+0000 7f8d74dcb640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 0x7f8d70148590 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c390 tx=0x7f8d6800c850 comp rx=0 tx=0).stop 2023-12-01T01:51:21.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.260+0000 7f8d74dcb640 1 -- 172.21.15.115:0/798833844 shutdown_connections 2023-12-01T01:51:21.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.260+0000 7f8d74dcb640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8d5006a550 0x7f8d5006ca10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.260+0000 7f8d74dcb640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d7015cd20 0x7f8d70148590 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.260+0000 7f8d74dcb640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d700d1230 0x7f8d70148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.260+0000 7f8d74dcb640 1 --2- 172.21.15.115:0/798833844 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d700b5d80 0x7f8d701f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.260+0000 7f8d74dcb640 1 -- 172.21.15.115:0/798833844 >> 172.21.15.115:0/798833844 conn(0x7f8d7006a0e0 msgr2=0x7f8d701597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:21.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.261+0000 7f8d74dcb640 1 -- 172.21.15.115:0/798833844 shutdown_connections 2023-12-01T01:51:21.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.261+0000 7f8d74dcb640 1 -- 172.21.15.115:0/798833844 wait complete. 2023-12-01T01:51:21.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:51:21.282 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:51:21.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.493+0000 7f280ec69640 1 Processor -- start 2023-12-01T01:51:21.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.493+0000 7f280ec69640 1 -- start start 2023-12-01T01:51:21.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.495+0000 7f280ec69640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f28080cd1e0 0x7f28080b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.495+0000 7f280ec69640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28080b2f30 con 0x7f28080b1c50 2023-12-01T01:51:21.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.495+0000 7f280ec69640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28080b2080 con 0x7f2808142870 2023-12-01T01:51:21.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.495+0000 7f280ec69640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28080b3ae0 con 0x7f28080cd1e0 2023-12-01T01:51:21.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.495+0000 7f280d206640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f28080b1c50 0x7f280815a0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58750/0 (socket says 172.21.15.115:58750) 2023-12-01T01:51:21.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.495+0000 7f2807fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f28080cd1e0 0x7f28080b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.495+0000 7f280d206640 1 -- 172.21.15.115:0/3976777230 learned_addr learned my addr 172.21.15.115:0/3976777230 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:21.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.496+0000 7f28077fe640 1 -- 172.21.15.115:0/3976777230 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426816221 0 0) 0x7f28080b2f30 con 0x7f28080b1c50 2023-12-01T01:51:21.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.496+0000 7f28077fe640 1 -- 172.21.15.115:0/3976777230 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27f8005730 con 0x7f28080b1c50 2023-12-01T01:51:21.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.496+0000 7f2807fff640 1 -- 172.21.15.115:0/3976777230 >> v1:172.21.15.142:6789/0 conn(0x7f2808142870 legacy=0x7f28080ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.496+0000 7f2807fff640 1 -- 172.21.15.115:0/3976777230 >> v1:172.21.15.115:6789/0 conn(0x7f28080b1c50 legacy=0x7f280815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.496+0000 7f2807fff640 1 -- 172.21.15.115:0/3976777230 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28080b2320 con 0x7f28080cd1e0 2023-12-01T01:51:21.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.497+0000 7f2807fff640 1 --2- 172.21.15.115:0/3976777230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080cd1e0 0x7f28080b1710 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f27f40097a0 tx=0x7f27f402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3947583010c4631 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.497+0000 7f28077fe640 1 -- 172.21.15.115:0/3976777230 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4030610 con 0x7f28080cd1e0 2023-12-01T01:51:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.497+0000 7f28077fe640 1 -- 172.21.15.115:0/3976777230 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f4030770 con 0x7f28080cd1e0 2023-12-01T01:51:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.497+0000 7f28077fe640 1 -- 172.21.15.115:0/3976777230 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4030b00 con 0x7f28080cd1e0 2023-12-01T01:51:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.498+0000 7f280ec69640 1 -- 172.21.15.115:0/3976777230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080cd1e0 msgr2=0x7f28080b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.498+0000 7f280ec69640 1 --2- 172.21.15.115:0/3976777230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080cd1e0 0x7f28080b1710 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f27f40097a0 tx=0x7f27f402b960 comp rx=0 tx=0).stop 2023-12-01T01:51:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.498+0000 7f280ec69640 1 -- 172.21.15.115:0/3976777230 shutdown_connections 2023-12-01T01:51:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.498+0000 7f280ec69640 1 --2- 172.21.15.115:0/3976777230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080cd1e0 0x7f28080b1710 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.498+0000 7f280ec69640 1 -- 172.21.15.115:0/3976777230 >> 172.21.15.115:0/3976777230 conn(0x7f2808065960 msgr2=0x7f2808065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.498+0000 7f280ec69640 1 -- 172.21.15.115:0/3976777230 shutdown_connections 2023-12-01T01:51:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.498+0000 7f280ec69640 1 -- 172.21.15.115:0/3976777230 wait complete. 2023-12-01T01:51:21.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.500+0000 7f280ec69640 1 Processor -- start 2023-12-01T01:51:21.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.500+0000 7f280ec69640 1 -- start start 2023-12-01T01:51:21.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ec69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080b1c50 0x7f28081587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ec69640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28080cd1e0 0x7f28081eea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ec69640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2808142870 0x7f280813cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ec69640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28080b0410 con 0x7f28080b1c50 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ec69640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2808158d30 con 0x7f28080cd1e0 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ec69640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28080af790 con 0x7f2808142870 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ca05640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080b1c50 0x7f28081587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ca05640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080b1c50 0x7f28081587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56204/0 (socket says 172.21.15.115:56204) 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ca05640 1 -- 172.21.15.115:0/4083113790 learned_addr learned my addr 172.21.15.115:0/4083113790 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ca05640 1 -- 172.21.15.115:0/4083113790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2808142870 msgr2=0x7f280813cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ca05640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2808142870 0x7f280813cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f2807fff640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28080cd1e0 0x7f28081eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ca05640 1 -- 172.21.15.115:0/4083113790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28080cd1e0 msgr2=0x7f28081eea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ca05640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28080cd1e0 0x7f28081eea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.501+0000 7f280ca05640 1 -- 172.21.15.115:0/4083113790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f4009400 con 0x7f28080b1c50 2023-12-01T01:51:21.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.502+0000 7f280ca05640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080b1c50 0x7f28081587f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00c400 tx=0x7f27fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:21.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.502+0000 7f28057fa640 1 -- 172.21.15.115:0/4083113790 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0052d0 con 0x7f28080b1c50 2023-12-01T01:51:21.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.502+0000 7f28057fa640 1 -- 172.21.15.115:0/4083113790 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc005430 con 0x7f28080b1c50 2023-12-01T01:51:21.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.502+0000 7f28057fa640 1 -- 172.21.15.115:0/4083113790 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc003830 con 0x7f28080b1c50 2023-12-01T01:51:21.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.502+0000 7f280ec69640 1 -- 172.21.15.115:0/4083113790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28080afa70 con 0x7f28080b1c50 2023-12-01T01:51:21.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.502+0000 7f280ec69640 1 -- 172.21.15.115:0/4083113790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28080b0030 con 0x7f28080b1c50 2023-12-01T01:51:21.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.503+0000 7f280ec69640 1 -- 172.21.15.115:0/4083113790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28080b0550 con 0x7f28080b1c50 2023-12-01T01:51:21.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.504+0000 7f28057fa640 1 -- 172.21.15.115:0/4083113790 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f27fc003990 con 0x7f28080b1c50 2023-12-01T01:51:21.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.505+0000 7f28057fa640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f27ec06a550 0x7f27ec06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.505+0000 7f28057fa640 1 -- 172.21.15.115:0/4083113790 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f27fc087c60 con 0x7f28080b1c50 2023-12-01T01:51:21.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.506+0000 7f2807fff640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f27ec06a550 0x7f27ec06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.506+0000 7f2807fff640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f27ec06a550 0x7f27ec06ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27f4036000 tx=0x7f27f403b040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:21.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.510+0000 7f28057fa640 1 -- 172.21.15.115:0/4083113790 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f27fc053520 con 0x7f28080b1c50 2023-12-01T01:51:21.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.700+0000 7f280ec69640 1 -- 172.21.15.115:0/4083113790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f280804e890 con 0x7f28080b1c50 2023-12-01T01:51:21.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.701+0000 7f28057fa640 1 -- 172.21.15.115:0/4083113790 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27fc017020 con 0x7f28080b1c50 2023-12-01T01:51:21.702 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:51:21.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.703+0000 7f280ec69640 1 -- 172.21.15.115:0/4083113790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f27ec06a550 msgr2=0x7f27ec06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.703+0000 7f280ec69640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f27ec06a550 0x7f27ec06ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27f4036000 tx=0x7f27f403b040 comp rx=0 tx=0).stop 2023-12-01T01:51:21.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.703+0000 7f280ec69640 1 -- 172.21.15.115:0/4083113790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080b1c50 msgr2=0x7f28081587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.703+0000 7f280ec69640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080b1c50 0x7f28081587f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00c400 tx=0x7f27fc00c8c0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.704+0000 7f280ec69640 1 -- 172.21.15.115:0/4083113790 shutdown_connections 2023-12-01T01:51:21.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.704+0000 7f280ec69640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f27ec06a550 0x7f27ec06ca10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.704+0000 7f280ec69640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2808142870 0x7f280813cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.704+0000 7f280ec69640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28080cd1e0 0x7f28081eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.704+0000 7f280ec69640 1 --2- 172.21.15.115:0/4083113790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28080b1c50 0x7f28081587f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.704+0000 7f280ec69640 1 -- 172.21.15.115:0/4083113790 >> 172.21.15.115:0/4083113790 conn(0x7f2808065960 msgr2=0x7f28080ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:21.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.704+0000 7f280ec69640 1 -- 172.21.15.115:0/4083113790 shutdown_connections 2023-12-01T01:51:21.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.704+0000 7f280ec69640 1 -- 172.21.15.115:0/4083113790 wait complete. 2023-12-01T01:51:21.724 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T01:51:21.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.927+0000 7f7779a05640 1 Processor -- start 2023-12-01T01:51:21.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.927+0000 7f7779a05640 1 -- start start 2023-12-01T01:51:21.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.929+0000 7f7779a05640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f777415cd20 0x7f77740d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.929+0000 7f7779a05640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77740ce2e0 con 0x7f77740b5d80 2023-12-01T01:51:21.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.929+0000 7f7779a05640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77740ce000 con 0x7f77740d1230 2023-12-01T01:51:21.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.929+0000 7f7779a05640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77740b2160 con 0x7f777415cd20 2023-12-01T01:51:21.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.929+0000 7f776bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f77740b5d80 0x7f77740b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58764/0 (socket says 172.21.15.115:58764) 2023-12-01T01:51:21.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.929+0000 7f776bfff640 1 -- 172.21.15.115:0/2984324728 learned_addr learned my addr 172.21.15.115:0/2984324728 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:21.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f7772ffd640 1 --2- 172.21.15.115:0/2984324728 >> v2:172.21.15.142:3300/0 conn(0x7f777415cd20 0x7f77740d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665966579 0 0) 0x7f77740ce2e0 con 0x7f77740b5d80 2023-12-01T01:51:21.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7760003520 con 0x7f77740b5d80 2023-12-01T01:51:21.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342069552 0 0) 0x7f7760003520 con 0x7f77740b5d80 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 >> v2:172.21.15.142:3300/0 conn(0x7f777415cd20 msgr2=0x7f77740d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 --2- 172.21.15.115:0/2984324728 >> v2:172.21.15.142:3300/0 conn(0x7f777415cd20 0x7f77740d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 >> v1:172.21.15.142:6789/0 conn(0x7f77740d1230 legacy=0x7f77740d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740cd3d0 con 0x7f77740b5d80 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f775c001670 con 0x7f77740b5d80 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 >> v1:172.21.15.115:6789/0 conn(0x7f77740b5d80 legacy=0x7f77740b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 --2- 172.21.15.115:0/2984324728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77600039e0 0x7f7760003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 --2- 172.21.15.115:0/2984324728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7760004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 --2- 172.21.15.115:0/2984324728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77600051f0 0x7f776000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776000c120 con 0x7f77600047c0 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776000c290 con 0x7f77600051f0 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.930+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776000c450 con 0x7f77600039e0 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.931+0000 7f776bfff640 1 --2- 172.21.15.115:0/2984324728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7760004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.931+0000 7f7772ffd640 1 --2- 172.21.15.115:0/2984324728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77600039e0 0x7f7760003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.931+0000 7f776bfff640 1 -- 172.21.15.115:0/2984324728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77600039e0 msgr2=0x7f7760003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:21.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.931+0000 7f776bfff640 1 --2- 172.21.15.115:0/2984324728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77600039e0 0x7f7760003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.931+0000 7f776bfff640 1 -- 172.21.15.115:0/2984324728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77600051f0 msgr2=0x7f776000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:21.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.931+0000 7f776bfff640 1 --2- 172.21.15.115:0/2984324728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77600051f0 0x7f776000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.931+0000 7f776bfff640 1 -- 172.21.15.115:0/2984324728 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7760003520 con 0x7f77600047c0 2023-12-01T01:51:21.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.931+0000 7f77737fe640 1 -- 172.21.15.115:0/2984324728 reap_dead start 2023-12-01T01:51:21.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.932+0000 7f776bfff640 1 --2- 172.21.15.115:0/2984324728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7760004c40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f775c00c240 tx=0x7f77600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c562f6e2b2f4e1a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:21.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.932+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c005ad0 con 0x7f77600047c0 2023-12-01T01:51:21.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.932+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775c005c50 con 0x7f77600047c0 2023-12-01T01:51:21.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.932+0000 7f77727fc640 1 -- 172.21.15.115:0/2984324728 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c02c8b0 con 0x7f77600047c0 2023-12-01T01:51:21.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.933+0000 7f7779a05640 1 -- 172.21.15.115:0/2984324728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 msgr2=0x7f7760004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.933+0000 7f7779a05640 1 --2- 172.21.15.115:0/2984324728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7760004c40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f775c00c240 tx=0x7f77600057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.933+0000 7f7779a05640 1 -- 172.21.15.115:0/2984324728 shutdown_connections 2023-12-01T01:51:21.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.933+0000 7f7779a05640 1 --2- 172.21.15.115:0/2984324728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7760004c40 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.933+0000 7f7779a05640 1 -- 172.21.15.115:0/2984324728 >> 172.21.15.115:0/2984324728 conn(0x7f777406a0e0 msgr2=0x7f777406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:21.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.933+0000 7f7779a05640 1 -- 172.21.15.115:0/2984324728 shutdown_connections 2023-12-01T01:51:21.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.933+0000 7f7779a05640 1 -- 172.21.15.115:0/2984324728 wait complete. 2023-12-01T01:51:21.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.935+0000 7f7779a05640 1 Processor -- start 2023-12-01T01:51:21.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.935+0000 7f7779a05640 1 -- start start 2023-12-01T01:51:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.936+0000 7f7779a05640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7774155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.936+0000 7f7779a05640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77740b4b90 0x7f77740c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.936+0000 7f7779a05640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77740cb800 0x7f77740cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.936+0000 7f7779a05640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740cc1c0 con 0x7f77600047c0 2023-12-01T01:51:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.936+0000 7f7779a05640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7774156b20 con 0x7f77740b4b90 2023-12-01T01:51:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.936+0000 7f7779a05640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7774156560 con 0x7f77740cb800 2023-12-01T01:51:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.936+0000 7f7772ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7774155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.936+0000 7f7772ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7774155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56216/0 (socket says 172.21.15.115:56216) 2023-12-01T01:51:21.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.936+0000 7f7772ffd640 1 -- 172.21.15.115:0/790765612 learned_addr learned my addr 172.21.15.115:0/790765612 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.936+0000 7f776bfff640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77740b4b90 0x7f77740c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7772ffd640 1 -- 172.21.15.115:0/790765612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77740cb800 msgr2=0x7f77740cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7772ffd640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77740cb800 0x7f77740cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7772ffd640 1 -- 172.21.15.115:0/790765612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77740b4b90 msgr2=0x7f77740c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7772ffd640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77740b4b90 0x7f77740c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7772ffd640 1 -- 172.21.15.115:0/790765612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7774156840 con 0x7f77600047c0 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f776bfff640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77740b4b90 0x7f77740c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7772ffd640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7774155e80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f77640107e0 tx=0x7f7764010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7770ff9640 1 -- 172.21.15.115:0/790765612 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764009590 con 0x7f77600047c0 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7770ff9640 1 -- 172.21.15.115:0/790765612 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776400eb60 con 0x7f77600047c0 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7770ff9640 1 -- 172.21.15.115:0/790765612 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776401c400 con 0x7f77600047c0 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7779a05640 1 -- 172.21.15.115:0/790765612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77740b21e0 con 0x7f77600047c0 2023-12-01T01:51:21.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.937+0000 7f7779a05640 1 -- 172.21.15.115:0/790765612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77740b52c0 con 0x7f77600047c0 2023-12-01T01:51:21.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.938+0000 7f7779a05640 1 -- 172.21.15.115:0/790765612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777404e890 con 0x7f77600047c0 2023-12-01T01:51:21.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.939+0000 7f7770ff9640 1 -- 172.21.15.115:0/790765612 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f776401c560 con 0x7f77600047c0 2023-12-01T01:51:21.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.940+0000 7f7770ff9640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f775406a580 0x7f775406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:21.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.940+0000 7f7770ff9640 1 -- 172.21.15.115:0/790765612 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7764088cb0 con 0x7f77600047c0 2023-12-01T01:51:21.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.940+0000 7f776bfff640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f775406a580 0x7f775406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:21.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.941+0000 7f776bfff640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f775406a580 0x7f775406ca40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f775c034000 tx=0x7f775c009030 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:21.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:21.945+0000 7f7770ff9640 1 -- 172.21.15.115:0/790765612 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7764054580 con 0x7f77600047c0 2023-12-01T01:51:22.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.137+0000 7f7779a05640 1 -- 172.21.15.115:0/790765612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7774157ee0 con 0x7f77600047c0 2023-12-01T01:51:22.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.138+0000 7f7770ff9640 1 -- 172.21.15.115:0/790765612 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7764057ba0 con 0x7f77600047c0 2023-12-01T01:51:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:22.140 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:51:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T01:51:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T01:51:22.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:22.140 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:51:22.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.141+0000 7f7779a05640 1 -- 172.21.15.115:0/790765612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f775406a580 msgr2=0x7f775406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.141+0000 7f7779a05640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f775406a580 0x7f775406ca40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f775c034000 tx=0x7f775c009030 comp rx=0 tx=0).stop 2023-12-01T01:51:22.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.141+0000 7f7779a05640 1 -- 172.21.15.115:0/790765612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 msgr2=0x7f7774155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.141+0000 7f7779a05640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7774155e80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f77640107e0 tx=0x7f7764010ca0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.141+0000 7f7779a05640 1 -- 172.21.15.115:0/790765612 shutdown_connections 2023-12-01T01:51:22.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.141+0000 7f7779a05640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f775406a580 0x7f775406ca40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.141+0000 7f7779a05640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f77740cb800 0x7f77740cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.141+0000 7f7779a05640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77740b4b90 0x7f77740c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.141+0000 7f7779a05640 1 --2- 172.21.15.115:0/790765612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77600047c0 0x7f7774155e80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.141+0000 7f7779a05640 1 -- 172.21.15.115:0/790765612 >> 172.21.15.115:0/790765612 conn(0x7f777406a0e0 msgr2=0x7f7774156df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:22.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.142+0000 7f7779a05640 1 -- 172.21.15.115:0/790765612 shutdown_connections 2023-12-01T01:51:22.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.142+0000 7f7779a05640 1 -- 172.21.15.115:0/790765612 wait complete. 2023-12-01T01:51:22.161 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T01:51:22.161 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T01:51:22.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.370+0000 7f0cec9ef640 1 Processor -- start 2023-12-01T01:51:22.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.371+0000 7f0cec9ef640 1 -- start start 2023-12-01T01:51:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.372+0000 7f0cec9ef640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0ce815d910 0x7f0ce815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.372+0000 7f0cec9ef640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0ce80d30e0 0x7f0ce80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.372+0000 7f0cec9ef640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce80b2300 con 0x7f0ce80cdaa0 2023-12-01T01:51:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.372+0000 7f0cec9ef640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce81612b0 con 0x7f0ce80d30e0 2023-12-01T01:51:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.372+0000 7f0cec9ef640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce81616d0 con 0x7f0ce815d910 2023-12-01T01:51:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.372+0000 7f0ce759e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0ce80d30e0 0x7f0ce80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.373+0000 7f0ce759e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0ce80d30e0 0x7f0ce80d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56218/0 (socket says 172.21.15.115:56218) 2023-12-01T01:51:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.373+0000 7f0ce759e640 1 -- 172.21.15.115:0/3136502837 learned_addr learned my addr 172.21.15.115:0/3136502837 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.373+0000 7f0ce6d9d640 1 --2- 172.21.15.115:0/3136502837 >> v2:172.21.15.142:3300/0 conn(0x7f0ce815d910 0x7f0ce815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.373+0000 7f0ce759e640 1 -- 172.21.15.115:0/3136502837 >> v2:172.21.15.142:3300/0 conn(0x7f0ce815d910 msgr2=0x7f0ce815dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:22.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.373+0000 7f0ce759e640 1 --2- 172.21.15.115:0/3136502837 >> v2:172.21.15.142:3300/0 conn(0x7f0ce815d910 0x7f0ce815dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.373+0000 7f0ce759e640 1 -- 172.21.15.115:0/3136502837 >> v1:172.21.15.142:6789/0 conn(0x7f0ce80cdaa0 legacy=0x7f0ce80d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:22.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.373+0000 7f0ce759e640 1 -- 172.21.15.115:0/3136502837 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80ce260 con 0x7f0ce80d30e0 2023-12-01T01:51:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.374+0000 7f0ce759e640 1 --2- 172.21.15.115:0/3136502837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce80d30e0 0x7f0ce80d0e10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80d0de0 tx=0x7f0ce002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c7e27b68af73e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.374+0000 7f0ce5d9b640 1 -- 172.21.15.115:0/3136502837 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00317e0 con 0x7f0ce80d30e0 2023-12-01T01:51:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.375+0000 7f0ce5d9b640 1 -- 172.21.15.115:0/3136502837 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce0031940 con 0x7f0ce80d30e0 2023-12-01T01:51:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.375+0000 7f0ce5d9b640 1 -- 172.21.15.115:0/3136502837 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0031cd0 con 0x7f0ce80d30e0 2023-12-01T01:51:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.375+0000 7f0cec9ef640 1 -- 172.21.15.115:0/3136502837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce80d30e0 msgr2=0x7f0ce80d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.375+0000 7f0cec9ef640 1 --2- 172.21.15.115:0/3136502837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce80d30e0 0x7f0ce80d0e10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80d0de0 tx=0x7f0ce002b880 comp rx=0 tx=0).stop 2023-12-01T01:51:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.375+0000 7f0cec9ef640 1 -- 172.21.15.115:0/3136502837 shutdown_connections 2023-12-01T01:51:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.375+0000 7f0cec9ef640 1 --2- 172.21.15.115:0/3136502837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce80d30e0 0x7f0ce80d0e10 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.375+0000 7f0cec9ef640 1 --2- 172.21.15.115:0/3136502837 >> v2:172.21.15.142:3300/0 conn(0x7f0ce815d910 0x7f0ce815dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.375+0000 7f0cec9ef640 1 -- 172.21.15.115:0/3136502837 >> 172.21.15.115:0/3136502837 conn(0x7f0ce806a0b0 msgr2=0x7f0ce806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.375+0000 7f0cec9ef640 1 -- 172.21.15.115:0/3136502837 shutdown_connections 2023-12-01T01:51:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.375+0000 7f0cec9ef640 1 -- 172.21.15.115:0/3136502837 wait complete. 2023-12-01T01:51:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.377+0000 7f0cec9ef640 1 Processor -- start 2023-12-01T01:51:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.377+0000 7f0cec9ef640 1 -- start start 2023-12-01T01:51:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.378+0000 7f0cec9ef640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce80cdaa0 0x7f0ce80ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.378+0000 7f0cec9ef640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce80d30e0 0x7f0ce8151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.378+0000 7f0cec9ef640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce815d910 0x7f0ce80b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.378+0000 7f0cec9ef640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80b24a0 con 0x7f0ce815d910 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.378+0000 7f0cec9ef640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80caff0 con 0x7f0ce80d30e0 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.378+0000 7f0cec9ef640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80cb2d0 con 0x7f0ce80cdaa0 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.378+0000 7f0ce759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce815d910 0x7f0ce80b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.379+0000 7f0ce759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce815d910 0x7f0ce80b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56226/0 (socket says 172.21.15.115:56226) 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.379+0000 7f0ce759e640 1 -- 172.21.15.115:0/1395342363 learned_addr learned my addr 172.21.15.115:0/1395342363 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.379+0000 7f0ce6d9d640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce80cdaa0 0x7f0ce80ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.379+0000 7f0ce659c640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce80d30e0 0x7f0ce8151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.379+0000 7f0ce759e640 1 -- 172.21.15.115:0/1395342363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce80cdaa0 msgr2=0x7f0ce80ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.379+0000 7f0ce759e640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce80cdaa0 0x7f0ce80ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.379+0000 7f0ce759e640 1 -- 172.21.15.115:0/1395342363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce80d30e0 msgr2=0x7f0ce8151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.379+0000 7f0ce759e640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce80d30e0 0x7f0ce8151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.379+0000 7f0ce759e640 1 -- 172.21.15.115:0/1395342363 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce00094d0 con 0x7f0ce815d910 2023-12-01T01:51:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.379+0000 7f0ce759e640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce815d910 0x7f0ce80b49d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0ce002b8c0 tx=0x7f0ce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.380+0000 7f0cdf7fe640 1 -- 172.21.15.115:0/1395342363 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0032880 con 0x7f0ce815d910 2023-12-01T01:51:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.380+0000 7f0cdf7fe640 1 -- 172.21.15.115:0/1395342363 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce0032a00 con 0x7f0ce815d910 2023-12-01T01:51:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.380+0000 7f0cdf7fe640 1 -- 172.21.15.115:0/1395342363 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0032db0 con 0x7f0ce815d910 2023-12-01T01:51:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.380+0000 7f0cec9ef640 1 -- 172.21.15.115:0/1395342363 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce80b26f0 con 0x7f0ce815d910 2023-12-01T01:51:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.380+0000 7f0cec9ef640 1 -- 172.21.15.115:0/1395342363 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce80c8d30 con 0x7f0ce815d910 2023-12-01T01:51:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.381+0000 7f0cec9ef640 1 -- 172.21.15.115:0/1395342363 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce80b2c10 con 0x7f0ce815d910 2023-12-01T01:51:22.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.382+0000 7f0cdf7fe640 1 -- 172.21.15.115:0/1395342363 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0ce004e450 con 0x7f0ce815d910 2023-12-01T01:51:22.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.383+0000 7f0cdf7fe640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0cc806a550 0x7f0cc806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.383+0000 7f0cdf7fe640 1 -- 172.21.15.115:0/1395342363 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0ce00aa880 con 0x7f0ce815d910 2023-12-01T01:51:22.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.383+0000 7f0ce6d9d640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0cc806a550 0x7f0cc806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.384+0000 7f0ce6d9d640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0cc806a550 0x7f0cc806ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80099c0 tx=0x7f0cd8003990 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:22.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.387+0000 7f0cdf7fe640 1 -- 172.21.15.115:0/1395342363 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0ce80b2c10 con 0x7f0ce815d910 2023-12-01T01:51:22.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.576+0000 7f0cec9ef640 1 -- 172.21.15.115:0/1395342363 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0ce80b2300 con 0x7f0ce815d910 2023-12-01T01:51:22.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.695+0000 7f0cdf7fe640 1 -- 172.21.15.115:0/1395342363 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0ce80b2300 con 0x7f0ce815d910 2023-12-01T01:51:22.698 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T01:51:22.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.701+0000 7f0cec9ef640 1 -- 172.21.15.115:0/1395342363 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0cc806a550 msgr2=0x7f0cc806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.701+0000 7f0cec9ef640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0cc806a550 0x7f0cc806ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80099c0 tx=0x7f0cd8003990 comp rx=0 tx=0).stop 2023-12-01T01:51:22.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.701+0000 7f0cec9ef640 1 -- 172.21.15.115:0/1395342363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce815d910 msgr2=0x7f0ce80b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.701+0000 7f0cec9ef640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce815d910 0x7f0ce80b49d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0ce002b8c0 tx=0x7f0ce00046c0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.704+0000 7f0cec9ef640 1 -- 172.21.15.115:0/1395342363 shutdown_connections 2023-12-01T01:51:22.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.704+0000 7f0cec9ef640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0cc806a550 0x7f0cc806ca10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.704+0000 7f0cec9ef640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce815d910 0x7f0ce80b49d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.704+0000 7f0cec9ef640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ce80d30e0 0x7f0ce8151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.704+0000 7f0cec9ef640 1 --2- 172.21.15.115:0/1395342363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce80cdaa0 0x7f0ce80ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.704+0000 7f0cec9ef640 1 -- 172.21.15.115:0/1395342363 >> 172.21.15.115:0/1395342363 conn(0x7f0ce806a0b0 msgr2=0x7f0ce80b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:22.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.705+0000 7f0cec9ef640 1 -- 172.21.15.115:0/1395342363 shutdown_connections 2023-12-01T01:51:22.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.705+0000 7f0cec9ef640 1 -- 172.21.15.115:0/1395342363 wait complete. 2023-12-01T01:51:22.727 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T01:51:22.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.930+0000 7f90c08ee640 1 Processor -- start 2023-12-01T01:51:22.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.931+0000 7f90c08ee640 1 -- start start 2023-12-01T01:51:22.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.932+0000 7f90c08ee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc15ced0 0x7f90bc0b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.932+0000 7f90c08ee640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f90bc0b4d70 0x7f90bc0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.932+0000 7f90c08ee640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90bc0d1480 con 0x7f90bc0d2750 2023-12-01T01:51:22.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.932+0000 7f90c08ee640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc0d16b0 con 0x7f90bc0b4d70 2023-12-01T01:51:22.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.932+0000 7f90c08ee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0d1990 con 0x7f90bc15ced0 2023-12-01T01:51:22.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.932+0000 7f90b9d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f90bc0d2750 0x7f90bc0b4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58778/0 (socket says 172.21.15.115:58778) 2023-12-01T01:51:22.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.932+0000 7f90b9d9b640 1 -- 172.21.15.115:0/2671216034 learned_addr learned my addr 172.21.15.115:0/2671216034 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:22.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.932+0000 7f90bad9d640 1 --2- 172.21.15.115:0/2671216034 >> v2:172.21.15.115:3300/0 conn(0x7f90bc0b4d70 0x7f90bc0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.933+0000 7f90ba59c640 1 --2- 172.21.15.115:0/2671216034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc15ced0 0x7f90bc0b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.933+0000 7f90b959a640 1 -- 172.21.15.115:0/2671216034 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527855328 0 0) 0x7f90bc0d1480 con 0x7f90bc0d2750 2023-12-01T01:51:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.933+0000 7f90b959a640 1 -- 172.21.15.115:0/2671216034 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90a8003520 con 0x7f90bc0d2750 2023-12-01T01:51:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.933+0000 7f90bad9d640 1 -- 172.21.15.115:0/2671216034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc15ced0 msgr2=0x7f90bc0b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.933+0000 7f90bad9d640 1 --2- 172.21.15.115:0/2671216034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc15ced0 0x7f90bc0b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.933+0000 7f90bad9d640 1 -- 172.21.15.115:0/2671216034 >> v1:172.21.15.115:6789/0 conn(0x7f90bc0d2750 legacy=0x7f90bc0b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.933+0000 7f90bad9d640 1 -- 172.21.15.115:0/2671216034 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc0d1c70 con 0x7f90bc0b4d70 2023-12-01T01:51:22.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.934+0000 7f90bad9d640 1 --2- 172.21.15.115:0/2671216034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0b4d70 0x7f90bc0d0f40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f90b40097a0 tx=0x7f90b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca2cade54e25518 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:22.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.934+0000 7f90b959a640 1 -- 172.21.15.115:0/2671216034 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4031830 con 0x7f90bc0b4d70 2023-12-01T01:51:22.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.934+0000 7f90b959a640 1 -- 172.21.15.115:0/2671216034 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b4031990 con 0x7f90bc0b4d70 2023-12-01T01:51:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.934+0000 7f90b959a640 1 -- 172.21.15.115:0/2671216034 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4031d20 con 0x7f90bc0b4d70 2023-12-01T01:51:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.935+0000 7f90c08ee640 1 -- 172.21.15.115:0/2671216034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0b4d70 msgr2=0x7f90bc0d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.935+0000 7f90c08ee640 1 --2- 172.21.15.115:0/2671216034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0b4d70 0x7f90bc0d0f40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f90b40097a0 tx=0x7f90b402b8d0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.935+0000 7f90c08ee640 1 -- 172.21.15.115:0/2671216034 shutdown_connections 2023-12-01T01:51:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.935+0000 7f90c08ee640 1 --2- 172.21.15.115:0/2671216034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0b4d70 0x7f90bc0d0f40 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.935+0000 7f90c08ee640 1 --2- 172.21.15.115:0/2671216034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc15ced0 0x7f90bc0b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.935+0000 7f90c08ee640 1 -- 172.21.15.115:0/2671216034 >> 172.21.15.115:0/2671216034 conn(0x7f90bc065f90 msgr2=0x7f90bc0cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.935+0000 7f90c08ee640 1 -- 172.21.15.115:0/2671216034 shutdown_connections 2023-12-01T01:51:22.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.935+0000 7f90c08ee640 1 -- 172.21.15.115:0/2671216034 wait complete. 2023-12-01T01:51:22.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.937+0000 7f90c08ee640 1 Processor -- start 2023-12-01T01:51:22.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.937+0000 7f90c08ee640 1 -- start start 2023-12-01T01:51:22.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.938+0000 7f90c08ee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc0b4d70 0x7f90bc15fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.938+0000 7f90c08ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0d2750 0x7f90bc0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.938+0000 7f90c08ee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc15ced0 0x7f90bc0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.938+0000 7f90c08ee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0cbd90 con 0x7f90bc0d2750 2023-12-01T01:51:22.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.938+0000 7f90c08ee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc15da10 con 0x7f90bc15ced0 2023-12-01T01:51:22.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.938+0000 7f90c08ee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0cb460 con 0x7f90bc0b4d70 2023-12-01T01:51:22.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.938+0000 7f90b9d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0d2750 0x7f90bc0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.938+0000 7f90b9d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0d2750 0x7f90bc0cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56252/0 (socket says 172.21.15.115:56252) 2023-12-01T01:51:22.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90b9d9b640 1 -- 172.21.15.115:0/954844944 learned_addr learned my addr 172.21.15.115:0/954844944 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:22.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90ba59c640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc0b4d70 0x7f90bc15fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90bad9d640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc15ced0 0x7f90bc0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90b9d9b640 1 -- 172.21.15.115:0/954844944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc0b4d70 msgr2=0x7f90bc15fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90b9d9b640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc0b4d70 0x7f90bc15fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90b9d9b640 1 -- 172.21.15.115:0/954844944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc15ced0 msgr2=0x7f90bc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:22.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90b9d9b640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc15ced0 0x7f90bc0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:22.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90b9d9b640 1 -- 172.21.15.115:0/954844944 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b4009400 con 0x7f90bc0d2750 2023-12-01T01:51:22.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90ba59c640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc0b4d70 0x7f90bc15fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:22.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90bad9d640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc15ced0 0x7f90bc0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:22.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.939+0000 7f90b9d9b640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0d2750 0x7f90bc0cc220 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c4d0 tx=0x7f90a400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:22.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.940+0000 7f90b2ffd640 1 -- 172.21.15.115:0/954844944 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4005ce0 con 0x7f90bc0d2750 2023-12-01T01:51:22.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.940+0000 7f90b2ffd640 1 -- 172.21.15.115:0/954844944 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a4005e40 con 0x7f90bc0d2750 2023-12-01T01:51:22.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.940+0000 7f90b2ffd640 1 -- 172.21.15.115:0/954844944 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4007270 con 0x7f90bc0d2750 2023-12-01T01:51:22.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.940+0000 7f90c08ee640 1 -- 172.21.15.115:0/954844944 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc0cb740 con 0x7f90bc0d2750 2023-12-01T01:51:22.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.940+0000 7f90c08ee640 1 -- 172.21.15.115:0/954844944 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc160110 con 0x7f90bc0d2750 2023-12-01T01:51:22.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.941+0000 7f90c08ee640 1 -- 172.21.15.115:0/954844944 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc0cbf00 con 0x7f90bc0d2750 2023-12-01T01:51:22.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.942+0000 7f90b2ffd640 1 -- 172.21.15.115:0/954844944 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f90a4007490 con 0x7f90bc0d2750 2023-12-01T01:51:22.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.943+0000 7f90b2ffd640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f909c06a550 0x7f909c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:22.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.943+0000 7f90b2ffd640 1 -- 172.21.15.115:0/954844944 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f90a4088510 con 0x7f90bc0d2750 2023-12-01T01:51:22.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.943+0000 7f90ba59c640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f909c06a550 0x7f909c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:22.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.944+0000 7f90ba59c640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f909c06a550 0x7f909c06ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0ce040 tx=0x7f90ac00b040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:22.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:22.947+0000 7f90b2ffd640 1 -- 172.21.15.115:0/954844944 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f90a4053cf0 con 0x7f90bc0d2750 2023-12-01T01:51:23.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.135+0000 7f90c08ee640 1 -- 172.21.15.115:0/954844944 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f90bc04e890 con 0x7f90bc0d2750 2023-12-01T01:51:23.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.703+0000 7f90b2ffd640 1 -- 172.21.15.115:0/954844944 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f90a4057310 con 0x7f90bc0d2750 2023-12-01T01:51:23.710 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T01:51:23.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.716+0000 7f90c08ee640 1 -- 172.21.15.115:0/954844944 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f909c06a550 msgr2=0x7f909c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:23.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.716+0000 7f90c08ee640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f909c06a550 0x7f909c06ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0ce040 tx=0x7f90ac00b040 comp rx=0 tx=0).stop 2023-12-01T01:51:23.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.716+0000 7f90c08ee640 1 -- 172.21.15.115:0/954844944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0d2750 msgr2=0x7f90bc0cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:23.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.716+0000 7f90c08ee640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0d2750 0x7f90bc0cc220 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c4d0 tx=0x7f90a400c990 comp rx=0 tx=0).stop 2023-12-01T01:51:23.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.717+0000 7f90c08ee640 1 -- 172.21.15.115:0/954844944 shutdown_connections 2023-12-01T01:51:23.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.717+0000 7f90c08ee640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f909c06a550 0x7f909c06ca10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:23.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.717+0000 7f90c08ee640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90bc15ced0 0x7f90bc0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:23.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.717+0000 7f90c08ee640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90bc0d2750 0x7f90bc0cc220 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:23.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.717+0000 7f90c08ee640 1 --2- 172.21.15.115:0/954844944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90bc0b4d70 0x7f90bc15fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:23.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.717+0000 7f90c08ee640 1 -- 172.21.15.115:0/954844944 >> 172.21.15.115:0/954844944 conn(0x7f90bc065f90 msgr2=0x7f90bc15b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:23.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.718+0000 7f90c08ee640 1 -- 172.21.15.115:0/954844944 shutdown_connections 2023-12-01T01:51:23.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.720+0000 7f90c08ee640 1 -- 172.21.15.115:0/954844944 wait complete. 2023-12-01T01:51:23.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T01:51:23.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.931+0000 7f1777fff640 1 Processor -- start 2023-12-01T01:51:23.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.932+0000 7f1777fff640 1 -- start start 2023-12-01T01:51:23.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.933+0000 7f1777fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f177815cb60 0x7f17780caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:23.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.933+0000 7f1777fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f17780ca2d0 0x7f17780ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:23.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.933+0000 7f1777fff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17780ca700 con 0x7f17780cb430 2023-12-01T01:51:23.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.933+0000 7f1777fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17780cb8c0 con 0x7f17780ca2d0 2023-12-01T01:51:23.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.933+0000 7f1777fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1778070000 con 0x7f177815cb60 2023-12-01T01:51:23.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.933+0000 7f176e7fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f17780cb430 0x7f17780c9bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58784/0 (socket says 172.21.15.115:58784) 2023-12-01T01:51:23.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.933+0000 7f176e7fc640 1 -- 172.21.15.115:0/3767517491 learned_addr learned my addr 172.21.15.115:0/3767517491 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:23.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.934+0000 7f17777fe640 1 --2- 172.21.15.115:0/3767517491 >> v2:172.21.15.115:3300/0 conn(0x7f17780ca2d0 0x7f17780ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:23.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.934+0000 7f1776ffd640 1 --2- 172.21.15.115:0/3767517491 >> v2:172.21.15.142:3300/0 conn(0x7f177815cb60 0x7f17780caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:23.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.934+0000 7f17767fc640 1 -- 172.21.15.115:0/3767517491 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124934593 0 0) 0x7f17780ca700 con 0x7f17780cb430 2023-12-01T01:51:23.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.934+0000 7f17767fc640 1 -- 172.21.15.115:0/3767517491 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1758003520 con 0x7f17780cb430 2023-12-01T01:51:23.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.934+0000 7f17777fe640 1 -- 172.21.15.115:0/3767517491 >> v2:172.21.15.142:3300/0 conn(0x7f177815cb60 msgr2=0x7f17780caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:23.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.934+0000 7f17777fe640 1 --2- 172.21.15.115:0/3767517491 >> v2:172.21.15.142:3300/0 conn(0x7f177815cb60 0x7f17780caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:23.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.934+0000 7f17777fe640 1 -- 172.21.15.115:0/3767517491 >> v1:172.21.15.115:6789/0 conn(0x7f17780cb430 legacy=0x7f17780c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:23.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.934+0000 7f17777fe640 1 -- 172.21.15.115:0/3767517491 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17780702e0 con 0x7f17780ca2d0 2023-12-01T01:51:23.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.936+0000 7f17777fe640 1 --2- 172.21.15.115:0/3767517491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17780ca2d0 0x7f17780ac8a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f17780667c0 tx=0x7f177002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d125e09456e24f73 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:23.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.936+0000 7f17767fc640 1 -- 172.21.15.115:0/3767517491 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177002cc80 con 0x7f17780ca2d0 2023-12-01T01:51:23.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.936+0000 7f17767fc640 1 -- 172.21.15.115:0/3767517491 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177002cde0 con 0x7f17780ca2d0 2023-12-01T01:51:23.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.936+0000 7f17767fc640 1 -- 172.21.15.115:0/3767517491 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1770034640 con 0x7f17780ca2d0 2023-12-01T01:51:23.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.936+0000 7f1777fff640 1 -- 172.21.15.115:0/3767517491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17780ca2d0 msgr2=0x7f17780ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:23.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.936+0000 7f1777fff640 1 --2- 172.21.15.115:0/3767517491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17780ca2d0 0x7f17780ac8a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f17780667c0 tx=0x7f177002bed0 comp rx=0 tx=0).stop 2023-12-01T01:51:23.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.937+0000 7f1777fff640 1 -- 172.21.15.115:0/3767517491 shutdown_connections 2023-12-01T01:51:23.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.937+0000 7f1777fff640 1 --2- 172.21.15.115:0/3767517491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17780ca2d0 0x7f17780ac8a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:23.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.937+0000 7f1777fff640 1 --2- 172.21.15.115:0/3767517491 >> v2:172.21.15.142:3300/0 conn(0x7f177815cb60 0x7f17780caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:23.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.937+0000 7f1777fff640 1 -- 172.21.15.115:0/3767517491 >> 172.21.15.115:0/3767517491 conn(0x7f1778065960 msgr2=0x7f1778065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:23.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.937+0000 7f1777fff640 1 -- 172.21.15.115:0/3767517491 shutdown_connections 2023-12-01T01:51:23.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.937+0000 7f1777fff640 1 -- 172.21.15.115:0/3767517491 wait complete. 2023-12-01T01:51:23.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.938+0000 7f1777fff640 1 Processor -- start 2023-12-01T01:51:23.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.939+0000 7f1777fff640 1 -- start start 2023-12-01T01:51:23.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.939+0000 7f1777fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17780ca2d0 0x7f1778146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:23.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.939+0000 7f1777fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17780cb430 0x7f17780c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:23.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.939+0000 7f1777fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f177815cb60 0x7f177815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:23.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f1777fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17780c8200 con 0x7f177815cb60 2023-12-01T01:51:23.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f1777fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1778146f60 con 0x7f17780cb430 2023-12-01T01:51:23.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f1777fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17780c7c40 con 0x7f17780ca2d0 2023-12-01T01:51:23.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f17777fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f177815cb60 0x7f177815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:23.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f17777fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f177815cb60 0x7f177815cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56266/0 (socket says 172.21.15.115:56266) 2023-12-01T01:51:23.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f17777fe640 1 -- 172.21.15.115:0/753379469 learned_addr learned my addr 172.21.15.115:0/753379469 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:23.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f1776ffd640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17780ca2d0 0x7f1778146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:23.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f17777fe640 1 -- 172.21.15.115:0/753379469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17780ca2d0 msgr2=0x7f1778146a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:23.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f17777fe640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17780ca2d0 0x7f1778146a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:23.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f176e7fc640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17780cb430 0x7f17780c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:23.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f17777fe640 1 -- 172.21.15.115:0/753379469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17780cb430 msgr2=0x7f17780c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:23.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f17777fe640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17780cb430 0x7f17780c7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:23.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.940+0000 7f17777fe640 1 -- 172.21.15.115:0/753379469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1770009a70 con 0x7f177815cb60 2023-12-01T01:51:23.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.941+0000 7f17777fe640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f177815cb60 0x7f177815cf40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f177002c4e0 tx=0x7f1770004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:23.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.941+0000 7f176ffff640 1 -- 172.21.15.115:0/753379469 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177003e610 con 0x7f177815cb60 2023-12-01T01:51:23.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.941+0000 7f176ffff640 1 -- 172.21.15.115:0/753379469 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177003e770 con 0x7f177815cb60 2023-12-01T01:51:23.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.941+0000 7f1777fff640 1 -- 172.21.15.115:0/753379469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17780c7f20 con 0x7f177815cb60 2023-12-01T01:51:23.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.941+0000 7f176ffff640 1 -- 172.21.15.115:0/753379469 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177003eb60 con 0x7f177815cb60 2023-12-01T01:51:23.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.941+0000 7f1777fff640 1 -- 172.21.15.115:0/753379469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17780c2b50 con 0x7f177815cb60 2023-12-01T01:51:23.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.942+0000 7f1777fff640 1 -- 172.21.15.115:0/753379469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17780c26d0 con 0x7f177815cb60 2023-12-01T01:51:23.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.944+0000 7f176ffff640 1 -- 172.21.15.115:0/753379469 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f177003b050 con 0x7f177815cb60 2023-12-01T01:51:23.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.945+0000 7f176ffff640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f175c06a550 0x7f175c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:23.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.945+0000 7f176ffff640 1 -- 172.21.15.115:0/753379469 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f17700aadb0 con 0x7f177815cb60 2023-12-01T01:51:23.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.945+0000 7f1776ffd640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f175c06a550 0x7f175c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:23.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.946+0000 7f1776ffd640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f175c06a550 0x7f175c06ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1768006fd0 tx=0x7f1768008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:23.949+0000 7f176ffff640 1 -- 172.21.15.115:0/753379469 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f17700763d0 con 0x7f177815cb60 2023-12-01T01:51:24.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.171+0000 7f1777fff640 1 -- 172.21.15.115:0/753379469 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f17780c26d0 con 0x7f177815cb60 2023-12-01T01:51:24.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.182+0000 7f176ffff640 1 -- 172.21.15.115:0/753379469 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7f17780c26d0 con 0x7f177815cb60 2023-12-01T01:51:24.184 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 8 and data pool 9 2023-12-01T01:51:24.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.185+0000 7f1777fff640 1 -- 172.21.15.115:0/753379469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f175c06a550 msgr2=0x7f175c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.185+0000 7f1777fff640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f175c06a550 0x7f175c06ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1768006fd0 tx=0x7f1768008040 comp rx=0 tx=0).stop 2023-12-01T01:51:24.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.185+0000 7f1777fff640 1 -- 172.21.15.115:0/753379469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f177815cb60 msgr2=0x7f177815cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.185+0000 7f1777fff640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f177815cb60 0x7f177815cf40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f177002c4e0 tx=0x7f1770004a40 comp rx=0 tx=0).stop 2023-12-01T01:51:24.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.186+0000 7f1777fff640 1 -- 172.21.15.115:0/753379469 shutdown_connections 2023-12-01T01:51:24.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.186+0000 7f1777fff640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f175c06a550 0x7f175c06ca10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.186+0000 7f1777fff640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f177815cb60 0x7f177815cf40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.186+0000 7f1777fff640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17780cb430 0x7f17780c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.186+0000 7f1777fff640 1 --2- 172.21.15.115:0/753379469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17780ca2d0 0x7f1778146a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.186+0000 7f1777fff640 1 -- 172.21.15.115:0/753379469 >> 172.21.15.115:0/753379469 conn(0x7f1778065960 msgr2=0x7f17780c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:24.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.191+0000 7f1777fff640 1 -- 172.21.15.115:0/753379469 shutdown_connections 2023-12-01T01:51:24.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.195+0000 7f1777fff640 1 -- 172.21.15.115:0/753379469 wait complete. 2023-12-01T01:51:24.226 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:24.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.399+0000 7f3bcb7d9640 1 Processor -- start 2023-12-01T01:51:24.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.399+0000 7f3bcb7d9640 1 -- start start 2023-12-01T01:51:24.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.401+0000 7f3bcb7d9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3bc415acd0 0x7f3bc415d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.401+0000 7f3bcb7d9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc415de90 con 0x7f3bc415a8f0 2023-12-01T01:51:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.401+0000 7f3bcb7d9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc40cdc10 con 0x7f3bc415da50 2023-12-01T01:51:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.401+0000 7f3bcb7d9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc415e0d0 con 0x7f3bc415acd0 2023-12-01T01:51:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.401+0000 7f3bc8d74640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3bc415acd0 0x7f3bc415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.401+0000 7f3bc9575640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3bc415a8f0 0x7f3bc40ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58794/0 (socket says 172.21.15.115:58794) 2023-12-01T01:51:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.401+0000 7f3bc9575640 1 -- 172.21.15.115:0/2029334021 learned_addr learned my addr 172.21.15.115:0/2029334021 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.402+0000 7f3bc8d74640 1 -- 172.21.15.115:0/2029334021 >> v1:172.21.15.142:6789/0 conn(0x7f3bc415da50 legacy=0x7f3bc40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.402+0000 7f3bc8d74640 1 -- 172.21.15.115:0/2029334021 >> v1:172.21.15.115:6789/0 conn(0x7f3bc415a8f0 legacy=0x7f3bc40ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.402+0000 7f3bc8d74640 1 -- 172.21.15.115:0/2029334021 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc415e3b0 con 0x7f3bc415acd0 2023-12-01T01:51:24.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.403+0000 7f3bc8d74640 1 --2- 172.21.15.115:0/2029334021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415acd0 0x7f3bc415d510 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40097a0 tx=0x7f3bb402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716b5f0cbaaedd58 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:24.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.403+0000 7f3bbbfff640 1 -- 172.21.15.115:0/2029334021 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb402e070 con 0x7f3bc415acd0 2023-12-01T01:51:24.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.403+0000 7f3bbbfff640 1 -- 172.21.15.115:0/2029334021 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb40315a0 con 0x7f3bc415acd0 2023-12-01T01:51:24.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.403+0000 7f3bbbfff640 1 -- 172.21.15.115:0/2029334021 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4031930 con 0x7f3bc415acd0 2023-12-01T01:51:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.403+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2029334021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415acd0 msgr2=0x7f3bc415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.403+0000 7f3bcb7d9640 1 --2- 172.21.15.115:0/2029334021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415acd0 0x7f3bc415d510 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40097a0 tx=0x7f3bb402b840 comp rx=0 tx=0).stop 2023-12-01T01:51:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.404+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2029334021 shutdown_connections 2023-12-01T01:51:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.404+0000 7f3bcb7d9640 1 --2- 172.21.15.115:0/2029334021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415acd0 0x7f3bc415d510 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.404+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2029334021 >> 172.21.15.115:0/2029334021 conn(0x7f3bc4065960 msgr2=0x7f3bc4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:24.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.404+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2029334021 shutdown_connections 2023-12-01T01:51:24.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.404+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2029334021 wait complete. 2023-12-01T01:51:24.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.405+0000 7f3bcb7d9640 1 Processor -- start 2023-12-01T01:51:24.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.406+0000 7f3bcb7d9640 1 -- start start 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bcb7d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415a8f0 0x7f3bc41595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bcb7d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc415acd0 0x7f3bc40ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bcb7d9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc415da50 0x7f3bc40b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bcb7d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40b3770 con 0x7f3bc415a8f0 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bcb7d9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40b3a10 con 0x7f3bc415da50 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bcb7d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40b3cf0 con 0x7f3bc415acd0 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bc9575640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415a8f0 0x7f3bc41595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bc9575640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415a8f0 0x7f3bc41595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56292/0 (socket says 172.21.15.115:56292) 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bc9575640 1 -- 172.21.15.115:0/2722658542 learned_addr learned my addr 172.21.15.115:0/2722658542 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bc9575640 1 -- 172.21.15.115:0/2722658542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc415acd0 msgr2=0x7f3bc40ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bc9575640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc415acd0 0x7f3bc40ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bc9575640 1 -- 172.21.15.115:0/2722658542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc415da50 msgr2=0x7f3bc40b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bc9575640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc415da50 0x7f3bc40b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.407+0000 7f3bc9575640 1 -- 172.21.15.115:0/2722658542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb4009400 con 0x7f3bc415a8f0 2023-12-01T01:51:24.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.408+0000 7f3bc9575640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415a8f0 0x7f3bc41595e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000c4a0 tx=0x7f3bc000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:24.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.408+0000 7f3bb9ffb640 1 -- 172.21.15.115:0/2722658542 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0005ce0 con 0x7f3bc415a8f0 2023-12-01T01:51:24.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.408+0000 7f3bb9ffb640 1 -- 172.21.15.115:0/2722658542 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0005e40 con 0x7f3bc415a8f0 2023-12-01T01:51:24.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.408+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2722658542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc40b3fd0 con 0x7f3bc415a8f0 2023-12-01T01:51:24.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.408+0000 7f3bb9ffb640 1 -- 172.21.15.115:0/2722658542 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0007270 con 0x7f3bc415a8f0 2023-12-01T01:51:24.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.408+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2722658542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc40b4590 con 0x7f3bc415a8f0 2023-12-01T01:51:24.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.409+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2722658542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc40b4780 con 0x7f3bc415a8f0 2023-12-01T01:51:24.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.410+0000 7f3bb9ffb640 1 -- 172.21.15.115:0/2722658542 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3bc0007490 con 0x7f3bc415a8f0 2023-12-01T01:51:24.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.411+0000 7f3bb9ffb640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3ba806a550 0x7f3ba806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.411+0000 7f3bb9ffb640 1 -- 172.21.15.115:0/2722658542 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bc0087d80 con 0x7f3bc415a8f0 2023-12-01T01:51:24.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.411+0000 7f3bc8d74640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3ba806a550 0x7f3ba806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:24.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.412+0000 7f3bc8d74640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3ba806a550 0x7f3ba806ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4036000 tx=0x7f3bb4030000 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:24.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.416+0000 7f3bb9ffb640 1 -- 172.21.15.115:0/2722658542 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3bc0053300 con 0x7f3bc415a8f0 2023-12-01T01:51:24.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.592+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2722658542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bc415de90 con 0x7f3bc415a8f0 2023-12-01T01:51:24.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.594+0000 7f3bb9ffb640 1 -- 172.21.15.115:0/2722658542 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16158 (secure 0 0 0) 0x7f3bc415de90 con 0x7f3bc415a8f0 2023-12-01T01:51:24.596 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:24.596 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":82,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:24.173567+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:51:22.633453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-01T01:51:23.191899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:24.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.597+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2722658542 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3ba806a550 msgr2=0x7f3ba806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.597+0000 7f3bcb7d9640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3ba806a550 0x7f3ba806ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4036000 tx=0x7f3bb4030000 comp rx=0 tx=0).stop 2023-12-01T01:51:24.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.597+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2722658542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415a8f0 msgr2=0x7f3bc41595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.597+0000 7f3bcb7d9640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415a8f0 0x7f3bc41595e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000c4a0 tx=0x7f3bc000c960 comp rx=0 tx=0).stop 2023-12-01T01:51:24.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.598+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2722658542 shutdown_connections 2023-12-01T01:51:24.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.598+0000 7f3bcb7d9640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3ba806a550 0x7f3ba806ca10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.598+0000 7f3bcb7d9640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc415acd0 0x7f3bc40ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.598+0000 7f3bcb7d9640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc415da50 0x7f3bc40b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.598+0000 7f3bcb7d9640 1 --2- 172.21.15.115:0/2722658542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc415a8f0 0x7f3bc41595e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.598+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2722658542 >> 172.21.15.115:0/2722658542 conn(0x7f3bc4065960 msgr2=0x7f3bc415bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:24.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.598+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2722658542 shutdown_connections 2023-12-01T01:51:24.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.598+0000 7f3bcb7d9640 1 -- 172.21.15.115:0/2722658542 wait complete. 2023-12-01T01:51:24.617 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:24.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.825+0000 7fb1aef70640 1 Processor -- start 2023-12-01T01:51:24.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.825+0000 7fb1aef70640 1 -- start start 2023-12-01T01:51:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.826+0000 7fb1aef70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a81584b0 0x7fb1a80cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.826+0000 7fb1aef70640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb1a815afe0 0x7fb1a806b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.826+0000 7fb1aef70640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a80cc180 con 0x7fb1a806b7b0 2023-12-01T01:51:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.826+0000 7fb1aef70640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a806bc40 con 0x7fb1a815afe0 2023-12-01T01:51:24.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.826+0000 7fb1aef70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a815b550 con 0x7fb1a81584b0 2023-12-01T01:51:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.827+0000 7fb1ad76d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb1a815afe0 0x7fb1a806b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.827+0000 7fb1ae76f640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb1a806b7b0 0x7fb1a815e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58804/0 (socket says 172.21.15.115:58804) 2023-12-01T01:51:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.827+0000 7fb1ae76f640 1 -- 172.21.15.115:0/1184902364 learned_addr learned my addr 172.21.15.115:0/1184902364 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.827+0000 7fb1adf6e640 1 --2- 172.21.15.115:0/1184902364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a81584b0 0x7fb1a80cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.827+0000 7fb1acf6c640 1 -- 172.21.15.115:0/1184902364 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288443509 0 0) 0x7fb1a80cc180 con 0x7fb1a806b7b0 2023-12-01T01:51:24.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.827+0000 7fb1ad76d640 1 -- 172.21.15.115:0/1184902364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a81584b0 msgr2=0x7fb1a80cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.827+0000 7fb1ad76d640 1 --2- 172.21.15.115:0/1184902364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a81584b0 0x7fb1a80cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.827+0000 7fb1ad76d640 1 -- 172.21.15.115:0/1184902364 >> v1:172.21.15.115:6789/0 conn(0x7fb1a806b7b0 legacy=0x7fb1a815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.827+0000 7fb1ad76d640 1 -- 172.21.15.115:0/1184902364 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a815b830 con 0x7fb1a815afe0 2023-12-01T01:51:24.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.828+0000 7fb1ad76d640 1 --2- 172.21.15.115:0/1184902364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 0x7fb1a806b270 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb1980097a0 tx=0x7fb19802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126adcd681e5ee0c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1acf6c640 1 -- 172.21.15.115:0/1184902364 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19802d040 con 0x7fb1a815afe0 2023-12-01T01:51:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1acf6c640 1 -- 172.21.15.115:0/1184902364 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb198031810 con 0x7fb1a815afe0 2023-12-01T01:51:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1acf6c640 1 -- 172.21.15.115:0/1184902364 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198031ba0 con 0x7fb1a815afe0 2023-12-01T01:51:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1aef70640 1 -- 172.21.15.115:0/1184902364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 msgr2=0x7fb1a806b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1aef70640 1 --2- 172.21.15.115:0/1184902364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 0x7fb1a806b270 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb1980097a0 tx=0x7fb19802b8b0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1aef70640 1 -- 172.21.15.115:0/1184902364 shutdown_connections 2023-12-01T01:51:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1aef70640 1 --2- 172.21.15.115:0/1184902364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 0x7fb1a806b270 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1aef70640 1 --2- 172.21.15.115:0/1184902364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a81584b0 0x7fb1a80cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1aef70640 1 -- 172.21.15.115:0/1184902364 >> 172.21.15.115:0/1184902364 conn(0x7fb1a80ce640 msgr2=0x7fb1a80cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1aef70640 1 -- 172.21.15.115:0/1184902364 shutdown_connections 2023-12-01T01:51:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.829+0000 7fb1aef70640 1 -- 172.21.15.115:0/1184902364 wait complete. 2023-12-01T01:51:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.831+0000 7fb1aef70640 1 Processor -- start 2023-12-01T01:51:24.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.831+0000 7fb1aef70640 1 -- start start 2023-12-01T01:51:24.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.832+0000 7fb1aef70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a806b7b0 0x7fb1a81f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.832+0000 7fb1aef70640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a81584b0 0x7fb1a80ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.832+0000 7fb1aef70640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 0x7fb1a80acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.832+0000 7fb1aef70640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a81f53d0 con 0x7fb1a815afe0 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.832+0000 7fb1aef70640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a80c98f0 con 0x7fb1a81584b0 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.832+0000 7fb1aef70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a80c9bd0 con 0x7fb1a806b7b0 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.832+0000 7fb1ae76f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 0x7fb1a80acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.833+0000 7fb1ae76f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 0x7fb1a80acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56310/0 (socket says 172.21.15.115:56310) 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.833+0000 7fb1ae76f640 1 -- 172.21.15.115:0/4169129121 learned_addr learned my addr 172.21.15.115:0/4169129121 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.833+0000 7fb1adf6e640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a806b7b0 0x7fb1a81f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.833+0000 7fb1ae76f640 1 -- 172.21.15.115:0/4169129121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a806b7b0 msgr2=0x7fb1a81f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.833+0000 7fb1ae76f640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a806b7b0 0x7fb1a81f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.833+0000 7fb1ae76f640 1 -- 172.21.15.115:0/4169129121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a81584b0 msgr2=0x7fb1a80ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.833+0000 7fb1ae76f640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a81584b0 0x7fb1a80ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.833+0000 7fb1ae76f640 1 -- 172.21.15.115:0/4169129121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb198009400 con 0x7fb1a815afe0 2023-12-01T01:51:24.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.833+0000 7fb1ae76f640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 0x7fb1a80acdd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000c7e0 tx=0x7fb1a000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.833+0000 7fb19e7fc640 1 -- 172.21.15.115:0/4169129121 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000a640 con 0x7fb1a815afe0 2023-12-01T01:51:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.834+0000 7fb19e7fc640 1 -- 172.21.15.115:0/4169129121 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a000a7c0 con 0x7fb1a815afe0 2023-12-01T01:51:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.834+0000 7fb19e7fc640 1 -- 172.21.15.115:0/4169129121 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000ab70 con 0x7fb1a815afe0 2023-12-01T01:51:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.834+0000 7fb1aef70640 1 -- 172.21.15.115:0/4169129121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a80c9eb0 con 0x7fb1a815afe0 2023-12-01T01:51:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.834+0000 7fb1aef70640 1 -- 172.21.15.115:0/4169129121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a80ca470 con 0x7fb1a815afe0 2023-12-01T01:51:24.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.835+0000 7fb1aef70640 1 -- 172.21.15.115:0/4169129121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a80673a0 con 0x7fb1a815afe0 2023-12-01T01:51:24.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.836+0000 7fb19e7fc640 1 -- 172.21.15.115:0/4169129121 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb1a000acd0 con 0x7fb1a815afe0 2023-12-01T01:51:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.836+0000 7fb19e7fc640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb18c06a550 0x7fb18c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.837+0000 7fb19e7fc640 1 -- 172.21.15.115:0/4169129121 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1a00880f0 con 0x7fb1a815afe0 2023-12-01T01:51:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.837+0000 7fb1adf6e640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb18c06a550 0x7fb18c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.837+0000 7fb1adf6e640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb18c06a550 0x7fb18c06ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4006fd0 tx=0x7fb1a4008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:24.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:24.841+0000 7fb19e7fc640 1 -- 172.21.15.115:0/4169129121 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb1a0053530 con 0x7fb1a815afe0 2023-12-01T01:51:25.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.030+0000 7fb1aef70640 1 -- 172.21.15.115:0/4169129121 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1a80ca660 con 0x7fb1a815afe0 2023-12-01T01:51:25.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.031+0000 7fb19e7fc640 1 -- 172.21.15.115:0/4169129121 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16158 (secure 0 0 0) 0x7fb1a0018090 con 0x7fb1a815afe0 2023-12-01T01:51:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:25.033 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":82,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:24.173567+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:51:22.633453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-01T01:51:23.191899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:25.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.033+0000 7fb1aef70640 1 -- 172.21.15.115:0/4169129121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb18c06a550 msgr2=0x7fb18c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:25.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.033+0000 7fb1aef70640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb18c06a550 0x7fb18c06ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4006fd0 tx=0x7fb1a4008040 comp rx=0 tx=0).stop 2023-12-01T01:51:25.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.034+0000 7fb1aef70640 1 -- 172.21.15.115:0/4169129121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 msgr2=0x7fb1a80acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:25.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.034+0000 7fb1aef70640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 0x7fb1a80acdd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000c7e0 tx=0x7fb1a000cca0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.034+0000 7fb1aef70640 1 -- 172.21.15.115:0/4169129121 shutdown_connections 2023-12-01T01:51:25.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.034+0000 7fb1aef70640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb18c06a550 0x7fb18c06ca10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.034+0000 7fb1aef70640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a815afe0 0x7fb1a80acdd0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.034+0000 7fb1aef70640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a81584b0 0x7fb1a80ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.034+0000 7fb1aef70640 1 --2- 172.21.15.115:0/4169129121 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a806b7b0 0x7fb1a81f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.034+0000 7fb1aef70640 1 -- 172.21.15.115:0/4169129121 >> 172.21.15.115:0/4169129121 conn(0x7fb1a80ce640 msgr2=0x7fb1a80cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:25.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.034+0000 7fb1aef70640 1 -- 172.21.15.115:0/4169129121 shutdown_connections 2023-12-01T01:51:25.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.034+0000 7fb1aef70640 1 -- 172.21.15.115:0/4169129121 wait complete. 2023-12-01T01:51:25.055 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T01:51:25.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.270+0000 7fba26a70640 1 Processor -- start 2023-12-01T01:51:25.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.271+0000 7fba26a70640 1 -- start start 2023-12-01T01:51:25.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.272+0000 7fba26a70640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fba20142870 0x7fba200ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:25.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.272+0000 7fba26a70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba200cd010 0x7fba200b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:25.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.272+0000 7fba26a70640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba200cd470 con 0x7fba200b1c50 2023-12-01T01:51:25.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.272+0000 7fba26a70640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba200b3ae0 con 0x7fba20142870 2023-12-01T01:51:25.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.272+0000 7fba26a70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba20142c50 con 0x7fba200cd010 2023-12-01T01:51:25.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.272+0000 7fba2500d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fba200b1c50 0x7fba2015d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58816/0 (socket says 172.21.15.115:58816) 2023-12-01T01:51:25.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.272+0000 7fba2500d640 1 -- 172.21.15.115:0/1934720189 learned_addr learned my addr 172.21.15.115:0/1934720189 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:25.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1ffff640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba200cd010 0x7fba200b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:25.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186251392 0 0) 0x7fba200cd470 con 0x7fba200b1c50 2023-12-01T01:51:25.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba10003520 con 0x7fba200b1c50 2023-12-01T01:51:25.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba2480c640 1 --2- 172.21.15.115:0/1934720189 >> v2:172.21.15.142:3300/0 conn(0x7fba20142870 0x7fba200ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:25.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3446578460 0 0) 0x7fba10003520 con 0x7fba200b1c50 2023-12-01T01:51:25.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba200cd010 msgr2=0x7fba200b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:25.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba200cd010 0x7fba200b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 >> v2:172.21.15.142:3300/0 conn(0x7fba20142870 msgr2=0x7fba200ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:25.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 --2- 172.21.15.115:0/1934720189 >> v2:172.21.15.142:3300/0 conn(0x7fba20142870 0x7fba200ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba200b20e0 con 0x7fba200b1c50 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fba180019f0 con 0x7fba200b1c50 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 >> v1:172.21.15.115:6789/0 conn(0x7fba200b1c50 legacy=0x7fba2015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba100039a0 0x7fba10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba200cd010 0x7fba100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba10004d00 0x7fba10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200b3ae0 con 0x7fba100039a0 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.273+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba20142c50 con 0x7fba10004d00 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.274+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1000ba00 con 0x7fba200cd010 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.274+0000 7fba2480c640 1 --2- 172.21.15.115:0/1934720189 >> v2:172.21.15.142:3300/0 conn(0x7fba20142870 0x7fba200ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.274+0000 7fba2480c640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba100039a0 0x7fba10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:25.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.274+0000 7fba1ffff640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba200cd010 0x7fba100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:25.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.274+0000 7fba2480c640 1 -- 172.21.15.115:0/1934720189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba200cd010 msgr2=0x7fba100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:25.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.274+0000 7fba2480c640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba200cd010 0x7fba100047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.274+0000 7fba2480c640 1 -- 172.21.15.115:0/1934720189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba10004d00 msgr2=0x7fba10005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:25.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.274+0000 7fba2480c640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba10004d00 0x7fba10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.274+0000 7fba2480c640 1 -- 172.21.15.115:0/1934720189 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10003520 con 0x7fba100039a0 2023-12-01T01:51:25.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.275+0000 7fba2480c640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba100039a0 0x7fba10003dc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fba140098a0 tx=0x7fba100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc2d2689fff5f3e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:25.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.276+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba140305f0 con 0x7fba100039a0 2023-12-01T01:51:25.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.276+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba14030750 con 0x7fba100039a0 2023-12-01T01:51:25.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.276+0000 7fba1f7fe640 1 -- 172.21.15.115:0/1934720189 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14030ac0 con 0x7fba100039a0 2023-12-01T01:51:25.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.276+0000 7fba26a70640 1 -- 172.21.15.115:0/1934720189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba100039a0 msgr2=0x7fba10003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:25.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.276+0000 7fba26a70640 1 --2- 172.21.15.115:0/1934720189 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba100039a0 0x7fba10003dc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fba140098a0 tx=0x7fba100057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.276+0000 7fba2500d640 1 -- 172.21.15.115:0/1934720189 reap_dead start 2023-12-01T01:51:25.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.276+0000 7fba26a70640 1 -- 172.21.15.115:0/1934720189 shutdown_connections 2023-12-01T01:51:25.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.276+0000 7fba26a70640 1 -- 172.21.15.115:0/1934720189 >> 172.21.15.115:0/1934720189 conn(0x7fba20065960 msgr2=0x7fba20065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:25.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.276+0000 7fba26a70640 1 -- 172.21.15.115:0/1934720189 shutdown_connections 2023-12-01T01:51:25.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.276+0000 7fba26a70640 1 -- 172.21.15.115:0/1934720189 wait complete. 2023-12-01T01:51:25.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.278+0000 7fba26a70640 1 Processor -- start 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.278+0000 7fba26a70640 1 -- start start 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.279+0000 7fba26a70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba100039a0 0x7fba200aa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.279+0000 7fba26a70640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba20159150 0x7fba200aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.279+0000 7fba26a70640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba20158480 0x7fba20158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.279+0000 7fba26a70640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200ab6e0 con 0x7fba20158480 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.279+0000 7fba26a70640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200aaa60 con 0x7fba20159150 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.279+0000 7fba26a70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200aad40 con 0x7fba100039a0 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.279+0000 7fba2500d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba20158480 0x7fba20158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.279+0000 7fba2500d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba20158480 0x7fba20158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56328/0 (socket says 172.21.15.115:56328) 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.279+0000 7fba2500d640 1 -- 172.21.15.115:0/500253415 learned_addr learned my addr 172.21.15.115:0/500253415 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba2480c640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba100039a0 0x7fba200aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:25.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba1ffff640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba20159150 0x7fba200aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba2500d640 1 -- 172.21.15.115:0/500253415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba100039a0 msgr2=0x7fba200aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba2500d640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba100039a0 0x7fba200aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba2500d640 1 -- 172.21.15.115:0/500253415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba20159150 msgr2=0x7fba200aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba2500d640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba20159150 0x7fba200aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba2500d640 1 -- 172.21.15.115:0/500253415 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba200ab020 con 0x7fba20158480 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba2500d640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba20158480 0x7fba20158910 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fba18001370 tx=0x7fba1800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba1d7fa640 1 -- 172.21.15.115:0/500253415 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1800cea0 con 0x7fba20158480 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba1d7fa640 1 -- 172.21.15.115:0/500253415 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1800a590 con 0x7fba20158480 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba1d7fa640 1 -- 172.21.15.115:0/500253415 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1800a960 con 0x7fba20158480 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.280+0000 7fba26a70640 1 -- 172.21.15.115:0/500253415 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba200ab300 con 0x7fba20158480 2023-12-01T01:51:25.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.281+0000 7fba26a70640 1 -- 172.21.15.115:0/500253415 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba200abb00 con 0x7fba20158480 2023-12-01T01:51:25.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.282+0000 7fba26a70640 1 -- 172.21.15.115:0/500253415 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba200ab820 con 0x7fba20158480 2023-12-01T01:51:25.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.283+0000 7fba1d7fa640 1 -- 172.21.15.115:0/500253415 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fba1800aac0 con 0x7fba20158480 2023-12-01T01:51:25.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.284+0000 7fba1d7fa640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba0406a580 0x7fba0406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:25.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.284+0000 7fba1d7fa640 1 -- 172.21.15.115:0/500253415 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba18088010 con 0x7fba20158480 2023-12-01T01:51:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.284+0000 7fba2480c640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba0406a580 0x7fba0406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:25.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.285+0000 7fba2480c640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba0406a580 0x7fba0406ca40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fba14036000 tx=0x7fba1403b040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:25.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.288+0000 7fba1d7fa640 1 -- 172.21.15.115:0/500253415 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fba200ab820 con 0x7fba20158480 2023-12-01T01:51:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:25.515+0000 7fba26a70640 1 -- 172.21.15.115:0/500253415 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fba200b20e0 con 0x7fba20158480 2023-12-01T01:51:26.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.474+0000 7fba1d7fa640 1 -- 172.21.15.115:0/500253415 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7fba200b20e0 con 0x7fba20158480 2023-12-01T01:51:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.477+0000 7fba26a70640 1 -- 172.21.15.115:0/500253415 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba0406a580 msgr2=0x7fba0406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.477+0000 7fba26a70640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba0406a580 0x7fba0406ca40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fba14036000 tx=0x7fba1403b040 comp rx=0 tx=0).stop 2023-12-01T01:51:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.477+0000 7fba26a70640 1 -- 172.21.15.115:0/500253415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba20158480 msgr2=0x7fba20158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.477+0000 7fba26a70640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba20158480 0x7fba20158910 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fba18001370 tx=0x7fba1800cc50 comp rx=0 tx=0).stop 2023-12-01T01:51:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.477+0000 7fba26a70640 1 -- 172.21.15.115:0/500253415 shutdown_connections 2023-12-01T01:51:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.477+0000 7fba26a70640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba0406a580 0x7fba0406ca40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.477+0000 7fba26a70640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba20158480 0x7fba20158910 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.477+0000 7fba26a70640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba20159150 0x7fba200aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.477+0000 7fba26a70640 1 --2- 172.21.15.115:0/500253415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba100039a0 0x7fba200aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.477+0000 7fba26a70640 1 -- 172.21.15.115:0/500253415 >> 172.21.15.115:0/500253415 conn(0x7fba20065960 msgr2=0x7fba20142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.478+0000 7fba26a70640 1 -- 172.21.15.115:0/500253415 shutdown_connections 2023-12-01T01:51:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.478+0000 7fba26a70640 1 -- 172.21.15.115:0/500253415 wait complete. 2023-12-01T01:51:26.500 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T01:51:26.500 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T01:51:26.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.714+0000 7f51c400b640 1 Processor -- start 2023-12-01T01:51:26.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.715+0000 7f51c400b640 1 -- start start 2023-12-01T01:51:26.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.716+0000 7f51c400b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f51bc15ac80 0x7f51bc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:26.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.716+0000 7f51c400b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f51bc0ce1b0 0x7f51bc0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:26.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.716+0000 7f51c400b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51bc0cc9a0 con 0x7f51bc0b0e60 2023-12-01T01:51:26.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.716+0000 7f51c400b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc15b060 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.716+0000 7f51c400b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc0b0ce0 con 0x7f51bc15ac80 2023-12-01T01:51:26.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.717+0000 7f51c15a6640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f51bc0b0e60 0x7f51bc0cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58824/0 (socket says 172.21.15.115:58824) 2023-12-01T01:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.717+0000 7f51c15a6640 1 -- 172.21.15.115:0/1058288613 learned_addr learned my addr 172.21.15.115:0/1058288613 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.717+0000 7f51c25a8640 1 --2- 172.21.15.115:0/1058288613 >> v2:172.21.15.115:3300/0 conn(0x7f51bc0ce1b0 0x7f51bc0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.717+0000 7f51c1da7640 1 --2- 172.21.15.115:0/1058288613 >> v2:172.21.15.142:3300/0 conn(0x7f51bc15ac80 0x7f51bc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.717+0000 7f51c25a8640 1 -- 172.21.15.115:0/1058288613 >> v2:172.21.15.142:3300/0 conn(0x7f51bc15ac80 msgr2=0x7f51bc0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.717+0000 7f51c25a8640 1 --2- 172.21.15.115:0/1058288613 >> v2:172.21.15.142:3300/0 conn(0x7f51bc15ac80 0x7f51bc0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.717+0000 7f51c25a8640 1 -- 172.21.15.115:0/1058288613 >> v1:172.21.15.115:6789/0 conn(0x7f51bc0b0e60 legacy=0x7f51bc0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.717+0000 7f51c25a8640 1 -- 172.21.15.115:0/1058288613 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0cbd90 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.718+0000 7f51c25a8640 1 --2- 172.21.15.115:0/1058288613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 0x7f51bc0b07a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0b3cc0 tx=0x7f51b402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175c4450f7a18c07 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c0da5640 1 -- 172.21.15.115:0/1058288613 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b402cc70 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c0da5640 1 -- 172.21.15.115:0/1058288613 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b402cdd0 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c0da5640 1 -- 172.21.15.115:0/1058288613 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4034660 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c400b640 1 -- 172.21.15.115:0/1058288613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 msgr2=0x7f51bc0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c400b640 1 --2- 172.21.15.115:0/1058288613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 0x7f51bc0b07a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0b3cc0 tx=0x7f51b402bec0 comp rx=0 tx=0).stop 2023-12-01T01:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c400b640 1 -- 172.21.15.115:0/1058288613 shutdown_connections 2023-12-01T01:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c400b640 1 --2- 172.21.15.115:0/1058288613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 0x7f51bc0b07a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c400b640 1 --2- 172.21.15.115:0/1058288613 >> v2:172.21.15.142:3300/0 conn(0x7f51bc15ac80 0x7f51bc0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c400b640 1 -- 172.21.15.115:0/1058288613 >> 172.21.15.115:0/1058288613 conn(0x7f51bc065960 msgr2=0x7f51bc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c400b640 1 -- 172.21.15.115:0/1058288613 shutdown_connections 2023-12-01T01:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.719+0000 7f51c400b640 1 -- 172.21.15.115:0/1058288613 wait complete. 2023-12-01T01:51:26.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.721+0000 7f51c400b640 1 Processor -- start 2023-12-01T01:51:26.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.721+0000 7f51c400b640 1 -- start start 2023-12-01T01:51:26.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.722+0000 7f51c400b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc0b0e60 0x7f51bc1edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:26.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.722+0000 7f51c400b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 0x7f51bc1f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:26.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.722+0000 7f51c400b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51bc15ac80 0x7f51bc153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:26.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.722+0000 7f51c400b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc1526a0 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.722+0000 7f51c400b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc1f1600 con 0x7f51bc15ac80 2023-12-01T01:51:26.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.722+0000 7f51c400b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc151a90 con 0x7f51bc0b0e60 2023-12-01T01:51:26.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.722+0000 7f51c15a6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 0x7f51bc1f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:26.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.723+0000 7f51c15a6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 0x7f51bc1f10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56354/0 (socket says 172.21.15.115:56354) 2023-12-01T01:51:26.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.723+0000 7f51c15a6640 1 -- 172.21.15.115:0/3437588600 learned_addr learned my addr 172.21.15.115:0/3437588600 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:26.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.723+0000 7f51c1da7640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc0b0e60 0x7f51bc1edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:26.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.723+0000 7f51c25a8640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51bc15ac80 0x7f51bc153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:26.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.723+0000 7f51c15a6640 1 -- 172.21.15.115:0/3437588600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc0b0e60 msgr2=0x7f51bc1edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:26.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.723+0000 7f51c15a6640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc0b0e60 0x7f51bc1edcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:26.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.723+0000 7f51c15a6640 1 -- 172.21.15.115:0/3437588600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51bc15ac80 msgr2=0x7f51bc153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:26.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.723+0000 7f51c15a6640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51bc15ac80 0x7f51bc153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:26.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.723+0000 7f51c15a6640 1 -- 172.21.15.115:0/3437588600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4009a70 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.723+0000 7f51c15a6640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 0x7f51bc1f10c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00c4a0 tx=0x7f51ac00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:26.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.724+0000 7f51b27fc640 1 -- 172.21.15.115:0/3437588600 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac005ce0 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.724+0000 7f51b27fc640 1 -- 172.21.15.115:0/3437588600 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac005e40 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.724+0000 7f51b27fc640 1 -- 172.21.15.115:0/3437588600 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac007270 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.724+0000 7f51c400b640 1 -- 172.21.15.115:0/3437588600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc151d70 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.724+0000 7f51c400b640 1 -- 172.21.15.115:0/3437588600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc152330 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.725+0000 7f51c400b640 1 -- 172.21.15.115:0/3437588600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc152810 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.726+0000 7f51b27fc640 1 -- 172.21.15.115:0/3437588600 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f51ac007490 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.727+0000 7f51b27fc640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a006a550 0x7f51a006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:26.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.727+0000 7f51b27fc640 1 -- 172.21.15.115:0/3437588600 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51ac087ff0 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.727+0000 7f51c1da7640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a006a550 0x7f51a006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:26.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.728+0000 7f51c1da7640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a006a550 0x7f51a006ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51b8006fd0 tx=0x7f51b8008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:26.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.732+0000 7f51b27fc640 1 -- 172.21.15.115:0/3437588600 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f51ac0534e0 con 0x7f51bc0ce1b0 2023-12-01T01:51:26.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:26.961+0000 7f51c400b640 1 -- 172.21.15.115:0/3437588600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f51bc04e890 con 0x7f51bc0ce1b0 2023-12-01T01:51:27.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.715+0000 7f51b27fc640 1 -- 172.21.15.115:0/3437588600 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51ac0170c0 con 0x7f51bc0ce1b0 2023-12-01T01:51:27.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 -- 172.21.15.115:0/3437588600 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a006a550 msgr2=0x7f51a006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a006a550 0x7f51a006ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51b8006fd0 tx=0x7f51b8008040 comp rx=0 tx=0).stop 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 -- 172.21.15.115:0/3437588600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 msgr2=0x7f51bc1f10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 0x7f51bc1f10c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00c4a0 tx=0x7f51ac00c960 comp rx=0 tx=0).stop 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 -- 172.21.15.115:0/3437588600 shutdown_connections 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f51a006a550 0x7f51a006ca10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51bc15ac80 0x7f51bc153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0ce1b0 0x7f51bc1f10c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 --2- 172.21.15.115:0/3437588600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc0b0e60 0x7f51bc1edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 -- 172.21.15.115:0/3437588600 >> 172.21.15.115:0/3437588600 conn(0x7f51bc065960 msgr2=0x7f51bc15d360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 -- 172.21.15.115:0/3437588600 shutdown_connections 2023-12-01T01:51:27.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.719+0000 7f51c400b640 1 -- 172.21.15.115:0/3437588600 wait complete. 2023-12-01T01:51:27.741 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T01:51:27.742 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:27.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.957+0000 7f5cd0563640 1 Processor -- start 2023-12-01T01:51:27.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.957+0000 7f5cd0563640 1 -- start start 2023-12-01T01:51:27.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.959+0000 7f5cd0563640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5cc80cad70 0x7f5cc806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:27.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.959+0000 7f5cd0563640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc806f2f0 0x7f5cc806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:27.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.959+0000 7f5cd0563640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc80add10 con 0x7f5cc815e2f0 2023-12-01T01:51:27.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.959+0000 7f5cd0563640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc80c5760 con 0x7f5cc80cad70 2023-12-01T01:51:27.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.959+0000 7f5cd0563640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc815e6d0 con 0x7f5cc806f2f0 2023-12-01T01:51:27.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.959+0000 7f5ccdafe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5cc80cad70 0x7f5cc806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:27.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.959+0000 7f5cce2ff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5cc815e2f0 0x7f5cc80ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59336/0 (socket says 172.21.15.115:59336) 2023-12-01T01:51:27.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.959+0000 7f5cce2ff640 1 -- 172.21.15.115:0/1386674552 learned_addr learned my addr 172.21.15.115:0/1386674552 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:27.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.960+0000 7f5cceb00640 1 --2- 172.21.15.115:0/1386674552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc806f2f0 0x7f5cc806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:27.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.960+0000 7f5ccdafe640 1 -- 172.21.15.115:0/1386674552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc806f2f0 msgr2=0x7f5cc806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:27.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.960+0000 7f5ccdafe640 1 --2- 172.21.15.115:0/1386674552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc806f2f0 0x7f5cc806bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:27.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.960+0000 7f5ccdafe640 1 -- 172.21.15.115:0/1386674552 >> v1:172.21.15.115:6789/0 conn(0x7f5cc815e2f0 legacy=0x7f5cc80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:27.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.960+0000 7f5ccdafe640 1 -- 172.21.15.115:0/1386674552 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc80cf7b0 con 0x7f5cc80cad70 2023-12-01T01:51:27.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.961+0000 7f5ccdafe640 1 --2- 172.21.15.115:0/1386674552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 0x7f5cc806b5c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097a0 tx=0x7f5cb402b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52290c84c17de92 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:27.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.961+0000 7f5ccd2fd640 1 -- 172.21.15.115:0/1386674552 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4030710 con 0x7f5cc80cad70 2023-12-01T01:51:27.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.961+0000 7f5ccd2fd640 1 -- 172.21.15.115:0/1386674552 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb402e050 con 0x7f5cc80cad70 2023-12-01T01:51:27.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.961+0000 7f5ccd2fd640 1 -- 172.21.15.115:0/1386674552 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4030a90 con 0x7f5cc80cad70 2023-12-01T01:51:27.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.962+0000 7f5cd0563640 1 -- 172.21.15.115:0/1386674552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 msgr2=0x7f5cc806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:27.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.962+0000 7f5cd0563640 1 --2- 172.21.15.115:0/1386674552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 0x7f5cc806b5c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097a0 tx=0x7f5cb402b5d0 comp rx=0 tx=0).stop 2023-12-01T01:51:27.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.962+0000 7f5cd0563640 1 -- 172.21.15.115:0/1386674552 shutdown_connections 2023-12-01T01:51:27.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.962+0000 7f5cd0563640 1 --2- 172.21.15.115:0/1386674552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc806f2f0 0x7f5cc806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:27.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.962+0000 7f5cd0563640 1 --2- 172.21.15.115:0/1386674552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 0x7f5cc806b5c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:27.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.962+0000 7f5cd0563640 1 -- 172.21.15.115:0/1386674552 >> 172.21.15.115:0/1386674552 conn(0x7f5cc80ce880 msgr2=0x7f5cc80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:27.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.962+0000 7f5cd0563640 1 -- 172.21.15.115:0/1386674552 shutdown_connections 2023-12-01T01:51:27.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.962+0000 7f5cd0563640 1 -- 172.21.15.115:0/1386674552 wait complete. 2023-12-01T01:51:27.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.964+0000 7f5cd0563640 1 Processor -- start 2023-12-01T01:51:27.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.964+0000 7f5cd0563640 1 -- start start 2023-12-01T01:51:27.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.965+0000 7f5cd0563640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc806f2f0 0x7f5cc8159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:27.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.965+0000 7f5cd0563640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 0x7f5cc80ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:27.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.965+0000 7f5cd0563640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc815e2f0 0x7f5cc80acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:27.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.965+0000 7f5cd0563640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8159670 con 0x7f5cc80cad70 2023-12-01T01:51:27.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.965+0000 7f5cd0563640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc81f5c80 con 0x7f5cc806f2f0 2023-12-01T01:51:27.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.965+0000 7f5cd0563640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc81f5f60 con 0x7f5cc815e2f0 2023-12-01T01:51:27.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.965+0000 7f5ccdafe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 0x7f5cc80ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:27.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.965+0000 7f5ccdafe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 0x7f5cc80ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39190/0 (socket says 172.21.15.115:39190) 2023-12-01T01:51:27.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.965+0000 7f5ccdafe640 1 -- 172.21.15.115:0/1906982748 learned_addr learned my addr 172.21.15.115:0/1906982748 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:27.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5cce2ff640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc806f2f0 0x7f5cc8159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:27.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5ccdafe640 1 -- 172.21.15.115:0/1906982748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc815e2f0 msgr2=0x7f5cc80acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:27.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5cceb00640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc815e2f0 0x7f5cc80acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:27.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5ccdafe640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc815e2f0 0x7f5cc80acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:27.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5ccdafe640 1 -- 172.21.15.115:0/1906982748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc806f2f0 msgr2=0x7f5cc8159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:27.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5ccdafe640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc806f2f0 0x7f5cc8159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:27.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5ccdafe640 1 -- 172.21.15.115:0/1906982748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb4009400 con 0x7f5cc80cad70 2023-12-01T01:51:27.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5ccdafe640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 0x7f5cc80ac8a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4000c00 tx=0x7f5cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:27.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5cc6ffd640 1 -- 172.21.15.115:0/1906982748 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb403b8d0 con 0x7f5cc80cad70 2023-12-01T01:51:27.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5cc6ffd640 1 -- 172.21.15.115:0/1906982748 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb403ba30 con 0x7f5cc80cad70 2023-12-01T01:51:27.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5cd0563640 1 -- 172.21.15.115:0/1906982748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc81f6240 con 0x7f5cc80cad70 2023-12-01T01:51:27.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.966+0000 7f5cc6ffd640 1 -- 172.21.15.115:0/1906982748 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb403bde0 con 0x7f5cc80cad70 2023-12-01T01:51:27.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.967+0000 7f5cd0563640 1 -- 172.21.15.115:0/1906982748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc81f6800 con 0x7f5cc80cad70 2023-12-01T01:51:27.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.968+0000 7f5cd0563640 1 -- 172.21.15.115:0/1906982748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc806fda0 con 0x7f5cc80cad70 2023-12-01T01:51:27.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.969+0000 7f5cc6ffd640 1 -- 172.21.15.115:0/1906982748 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f5cb4039040 con 0x7f5cc80cad70 2023-12-01T01:51:27.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.969+0000 7f5cc6ffd640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cac06a580 0x7f5cac06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:27.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.970+0000 7f5cc6ffd640 1 -- 172.21.15.115:0/1906982748 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cb40aa3e0 con 0x7f5cc80cad70 2023-12-01T01:51:27.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.970+0000 7f5cce2ff640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cac06a580 0x7f5cac06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:27.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.970+0000 7f5cce2ff640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cac06a580 0x7f5cac06ca40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0012e0 tx=0x7f5cbc009040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:27.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:27.974+0000 7f5cc6ffd640 1 -- 172.21.15.115:0/1906982748 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5cb40757a0 con 0x7f5cc80cad70 2023-12-01T01:51:28.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.200+0000 7f5cd0563640 1 -- 172.21.15.115:0/1906982748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cc804e890 con 0x7f5cc80cad70 2023-12-01T01:51:28.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.201+0000 7f5cc6ffd640 1 -- 172.21.15.115:0/1906982748 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f5cb4078dc0 con 0x7f5cc80cad70 2023-12-01T01:51:28.203 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:28.203 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1284508688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1284508688},{"type":"v1","addr":"172.21.15.142:6833","nonce":1284508688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1196894153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1196894153},{"type":"v1","addr":"172.21.15.142:6835","nonce":1196894153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6432,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/120786293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":120786293},{"type":"v1","addr":"172.21.15.115:6836","nonce":120786293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:51:24.174363+0000","modified":"2023-12-01T01:51:27.708969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6435},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6435":{"gid":6435,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3394497964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3394497964},{"type":"v1","addr":"172.21.15.115:6837","nonce":3394497964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-01T01:51:28.203 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 32 2023-12-01T01:51:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 -- 172.21.15.115:0/1906982748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cac06a580 msgr2=0x7f5cac06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cac06a580 0x7f5cac06ca40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0012e0 tx=0x7f5cbc009040 comp rx=0 tx=0).stop 2023-12-01T01:51:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 -- 172.21.15.115:0/1906982748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 msgr2=0x7f5cc80ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 0x7f5cc80ac8a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4000c00 tx=0x7f5cb40046c0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 -- 172.21.15.115:0/1906982748 shutdown_connections 2023-12-01T01:51:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cac06a580 0x7f5cac06ca40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc815e2f0 0x7f5cc80acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc80cad70 0x7f5cc80ac8a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 --2- 172.21.15.115:0/1906982748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc806f2f0 0x7f5cc8159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 -- 172.21.15.115:0/1906982748 >> 172.21.15.115:0/1906982748 conn(0x7f5cc80ce880 msgr2=0x7f5cc806ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 -- 172.21.15.115:0/1906982748 shutdown_connections 2023-12-01T01:51:28.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.204+0000 7f5cd0563640 1 -- 172.21.15.115:0/1906982748 wait complete. 2023-12-01T01:51:28.223 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.434+0000 7feddd120640 1 Processor -- start 2023-12-01T01:51:28.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.434+0000 7feddd120640 1 -- start start 2023-12-01T01:51:28.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.435+0000 7feddd120640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fedd8161510 0x7fedd80b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.435+0000 7feddd120640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd81618f0 con 0x7fedd80d17e0 2023-12-01T01:51:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.435+0000 7feddd120640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd80b0d50 con 0x7fedd80c4170 2023-12-01T01:51:28.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.435+0000 7feddd120640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd80b0a20 con 0x7fedd8161510 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.436+0000 7fedd659c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fedd80d17e0 0x7fedd80c3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59342/0 (socket says 172.21.15.115:59342) 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.436+0000 7fedd659c640 1 -- 172.21.15.115:0/3390325803 learned_addr learned my addr 172.21.15.115:0/3390325803 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.436+0000 7fedd6d9d640 1 --2- 172.21.15.115:0/3390325803 >> v2:172.21.15.142:3300/0 conn(0x7fedd8161510 0x7fedd80b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.436+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770489756 0 0) 0x7fedd81618f0 con 0x7fedd80d17e0 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.436+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedc4003520 con 0x7fedd80d17e0 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3827745319 0 0) 0x7fedc4003520 con 0x7fedd80d17e0 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 >> v2:172.21.15.142:3300/0 conn(0x7fedd8161510 msgr2=0x7fedd80b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 --2- 172.21.15.115:0/3390325803 >> v2:172.21.15.142:3300/0 conn(0x7fedd8161510 0x7fedd80b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 >> v1:172.21.15.142:6789/0 conn(0x7fedd80c4170 legacy=0x7fedd80ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd8069e60 con 0x7fedd80d17e0 2023-12-01T01:51:28.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fedc0001670 con 0x7fedd80d17e0 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 >> v1:172.21.15.115:6789/0 conn(0x7fedd80d17e0 legacy=0x7fedd80c3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc40039e0 0x7fedc4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc4005180 0x7fedc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd80b0a20 con 0x7fedc40047c0 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc400c0b0 con 0x7fedc40039e0 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc400c270 con 0x7fedc4005180 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd659c640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd6d9d640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc40039e0 0x7fedc4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd659c640 1 -- 172.21.15.115:0/3390325803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc4005180 msgr2=0x7fedc400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd659c640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc4005180 0x7fedc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd659c640 1 -- 172.21.15.115:0/3390325803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc40039e0 msgr2=0x7fedc4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:28.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd659c640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc40039e0 0x7fedc4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd659c640 1 -- 172.21.15.115:0/3390325803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc4003520 con 0x7fedc40047c0 2023-12-01T01:51:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd6d9d640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc40039e0 0x7fedc4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:28.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.437+0000 7fedd759e640 1 -- 172.21.15.115:0/3390325803 reap_dead start 2023-12-01T01:51:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.439+0000 7fedd659c640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedc4004c40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fedc000c240 tx=0x7fedc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123e87eb45a32e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.439+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0005ad0 con 0x7fedc40047c0 2023-12-01T01:51:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.439+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc0005c30 con 0x7fedc40047c0 2023-12-01T01:51:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.439+0000 7fedd5d9b640 1 -- 172.21.15.115:0/3390325803 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc002caf0 con 0x7fedc40047c0 2023-12-01T01:51:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.439+0000 7feddd120640 1 -- 172.21.15.115:0/3390325803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 msgr2=0x7fedc4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.439+0000 7feddd120640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedc4004c40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fedc000c240 tx=0x7fedc40057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.439+0000 7feddd120640 1 -- 172.21.15.115:0/3390325803 shutdown_connections 2023-12-01T01:51:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.439+0000 7feddd120640 1 --2- 172.21.15.115:0/3390325803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedc4004c40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.439+0000 7feddd120640 1 -- 172.21.15.115:0/3390325803 >> 172.21.15.115:0/3390325803 conn(0x7fedd80d0a90 msgr2=0x7fedd80d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:28.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.440+0000 7feddd120640 1 -- 172.21.15.115:0/3390325803 shutdown_connections 2023-12-01T01:51:28.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.440+0000 7feddd120640 1 -- 172.21.15.115:0/3390325803 wait complete. 2023-12-01T01:51:28.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.441+0000 7feddd120640 1 Processor -- start 2023-12-01T01:51:28.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.441+0000 7feddd120640 1 -- start start 2023-12-01T01:51:28.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.442+0000 7feddd120640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedd815cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.442+0000 7feddd120640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd815d400 0x7fedd815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.442+0000 7feddd120640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd80c7100 0x7fedd80c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.442+0000 7feddd120640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd80c9090 con 0x7fedc40047c0 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.442+0000 7feddd120640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd80c9370 con 0x7fedd80c7100 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.442+0000 7feddd120640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd80c6900 con 0x7fedd815d400 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.443+0000 7fedd6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedd815cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.443+0000 7fedd6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedd815cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39196/0 (socket says 172.21.15.115:39196) 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.443+0000 7fedd6d9d640 1 -- 172.21.15.115:0/3553410877 learned_addr learned my addr 172.21.15.115:0/3553410877 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.443+0000 7fedd6d9d640 1 -- 172.21.15.115:0/3553410877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd815d400 msgr2=0x7fedd815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.443+0000 7fedd6d9d640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd815d400 0x7fedd815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.443+0000 7fedd6d9d640 1 -- 172.21.15.115:0/3553410877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd80c7100 msgr2=0x7fedd80c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.443+0000 7fedd6d9d640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd80c7100 0x7fedd80c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.443+0000 7fedd6d9d640 1 -- 172.21.15.115:0/3553410877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd80c6be0 con 0x7fedc40047c0 2023-12-01T01:51:28.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.443+0000 7fedd6d9d640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedd815cdd0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fedc800f3b0 tx=0x7fedc800f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:28.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.444+0000 7fedcf7fe640 1 -- 172.21.15.115:0/3553410877 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc801a070 con 0x7fedc40047c0 2023-12-01T01:51:28.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.444+0000 7fedcf7fe640 1 -- 172.21.15.115:0/3553410877 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc8003990 con 0x7fedc40047c0 2023-12-01T01:51:28.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.444+0000 7feddd120640 1 -- 172.21.15.115:0/3553410877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd815cae0 con 0x7fedc40047c0 2023-12-01T01:51:28.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.444+0000 7fedcf7fe640 1 -- 172.21.15.115:0/3553410877 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8003d20 con 0x7fedc40047c0 2023-12-01T01:51:28.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.444+0000 7feddd120640 1 -- 172.21.15.115:0/3553410877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd80c6250 con 0x7fedc40047c0 2023-12-01T01:51:28.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.445+0000 7feddd120640 1 -- 172.21.15.115:0/3553410877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd804e890 con 0x7fedc40047c0 2023-12-01T01:51:28.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.446+0000 7fedcf7fe640 1 -- 172.21.15.115:0/3553410877 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fedc8004020 con 0x7fedc40047c0 2023-12-01T01:51:28.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.447+0000 7fedcf7fe640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedb806a580 0x7fedb806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.447+0000 7fedcf7fe640 1 -- 172.21.15.115:0/3553410877 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedc8089760 con 0x7fedc40047c0 2023-12-01T01:51:28.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.447+0000 7fedd659c640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedb806a580 0x7fedb806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.448+0000 7fedd659c640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedb806a580 0x7fedb806ca40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fedc0034000 tx=0x7fedc0039040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:28.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.451+0000 7fedcf7fe640 1 -- 172.21.15.115:0/3553410877 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fedc8054b20 con 0x7fedc40047c0 2023-12-01T01:51:28.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.642+0000 7feddd120640 1 -- 172.21.15.115:0/3553410877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedd80afcd0 con 0x7fedc40047c0 2023-12-01T01:51:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.644+0000 7fedcf7fe640 1 -- 172.21.15.115:0/3553410877 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16145 (secure 0 0 0) 0x7fedc8058140 con 0x7fedc40047c0 2023-12-01T01:51:28.646 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:28.646 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":83,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:25.179266+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:51:22.633453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-01T01:51:23.191899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 -- 172.21.15.115:0/3553410877 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedb806a580 msgr2=0x7fedb806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedb806a580 0x7fedb806ca40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fedc0034000 tx=0x7fedc0039040 comp rx=0 tx=0).stop 2023-12-01T01:51:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 -- 172.21.15.115:0/3553410877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 msgr2=0x7fedd815cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedd815cdd0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fedc800f3b0 tx=0x7fedc800f870 comp rx=0 tx=0).stop 2023-12-01T01:51:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 -- 172.21.15.115:0/3553410877 shutdown_connections 2023-12-01T01:51:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedb806a580 0x7fedb806ca40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd815d400 0x7fedd815c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd80c7100 0x7fedd80c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 --2- 172.21.15.115:0/3553410877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc40047c0 0x7fedd815cdd0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 -- 172.21.15.115:0/3553410877 >> 172.21.15.115:0/3553410877 conn(0x7fedd80d0a90 msgr2=0x7fedd80cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 -- 172.21.15.115:0/3553410877 shutdown_connections 2023-12-01T01:51:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.647+0000 7feddd120640 1 -- 172.21.15.115:0/3553410877 wait complete. 2023-12-01T01:51:28.668 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T01:51:28.668 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:51:28.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.883+0000 7f3dac769640 1 Processor -- start 2023-12-01T01:51:28.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.884+0000 7f3dac769640 1 -- start start 2023-12-01T01:51:28.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.885+0000 7f3dac769640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da406d220 0x7f3da406d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.885+0000 7f3dac769640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3da406de30 0x7f3da4070220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.885+0000 7f3dac769640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da406e2a0 con 0x7f3da40ae4f0 2023-12-01T01:51:28.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.885+0000 7f3dac769640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da40cb510 con 0x7f3da406de30 2023-12-01T01:51:28.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.885+0000 7f3dac769640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40ae950 con 0x7f3da406d220 2023-12-01T01:51:28.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.885+0000 7f3daa505640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da406d220 0x7f3da406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.885+0000 7f3daad06640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3da406de30 0x7f3da4070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.886+0000 7f3daad06640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3da406de30 0x7f3da4070220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:56058/0 (socket says 172.21.15.115:56058) 2023-12-01T01:51:28.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.886+0000 7f3daad06640 1 -- 172.21.15.115:0/3445269446 learned_addr learned my addr 172.21.15.115:0/3445269446 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:28.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.886+0000 7f3daa505640 1 -- 172.21.15.115:0/3445269446 >> v2:172.21.15.142:3300/0 conn(0x7f3da406de30 msgr2=0x7f3da4070220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:28.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.886+0000 7f3daa505640 1 --2- 172.21.15.115:0/3445269446 >> v2:172.21.15.142:3300/0 conn(0x7f3da406de30 0x7f3da4070220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.886+0000 7f3daa505640 1 -- 172.21.15.115:0/3445269446 >> v1:172.21.15.142:6789/0 conn(0x7f3da40ae4f0 legacy=0x7f3da406fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.886+0000 7f3daa505640 1 -- 172.21.15.115:0/3445269446 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da415d570 con 0x7f3da406d220 2023-12-01T01:51:28.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.886+0000 7f3daad06640 1 --2- 172.21.15.115:0/3445269446 >> v2:172.21.15.142:3300/0 conn(0x7f3da406de30 0x7f3da4070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:51:28.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.888+0000 7f3daa505640 1 --2- 172.21.15.115:0/3445269446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da406d220 0x7f3da406d600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009d90 tx=0x7f3d9802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c9c642eecb638b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:28.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.888+0000 7f3da9503640 1 -- 172.21.15.115:0/3445269446 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9802cbc0 con 0x7f3da406d220 2023-12-01T01:51:28.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.888+0000 7f3da9503640 1 -- 172.21.15.115:0/3445269446 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9802cd20 con 0x7f3da406d220 2023-12-01T01:51:28.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.888+0000 7f3da9503640 1 -- 172.21.15.115:0/3445269446 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98034660 con 0x7f3da406d220 2023-12-01T01:51:28.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.889+0000 7f3dac769640 1 -- 172.21.15.115:0/3445269446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da406d220 msgr2=0x7f3da406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.889+0000 7f3dac769640 1 --2- 172.21.15.115:0/3445269446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da406d220 0x7f3da406d600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009d90 tx=0x7f3d9802be10 comp rx=0 tx=0).stop 2023-12-01T01:51:28.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.889+0000 7f3dac769640 1 -- 172.21.15.115:0/3445269446 shutdown_connections 2023-12-01T01:51:28.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.889+0000 7f3dac769640 1 --2- 172.21.15.115:0/3445269446 >> v2:172.21.15.142:3300/0 conn(0x7f3da406de30 0x7f3da4070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.889+0000 7f3dac769640 1 --2- 172.21.15.115:0/3445269446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da406d220 0x7f3da406d600 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.889+0000 7f3dac769640 1 -- 172.21.15.115:0/3445269446 >> 172.21.15.115:0/3445269446 conn(0x7f3da40cbb10 msgr2=0x7f3da40cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:28.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.889+0000 7f3dac769640 1 -- 172.21.15.115:0/3445269446 shutdown_connections 2023-12-01T01:51:28.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.889+0000 7f3dac769640 1 -- 172.21.15.115:0/3445269446 wait complete. 2023-12-01T01:51:28.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.891+0000 7f3dac769640 1 Processor -- start 2023-12-01T01:51:28.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.891+0000 7f3dac769640 1 -- start start 2023-12-01T01:51:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3dac769640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da406d220 0x7f3da40c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3dac769640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da406de30 0x7f3da4147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3dac769640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40ae4f0 0x7f3da40c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3dac769640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40b2ff0 con 0x7f3da406de30 2023-12-01T01:51:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3dac769640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40b32d0 con 0x7f3da406d220 2023-12-01T01:51:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3dac769640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40c6e50 con 0x7f3da40ae4f0 2023-12-01T01:51:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3da9d04640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da406de30 0x7f3da4147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3da9d04640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da406de30 0x7f3da4147420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39202/0 (socket says 172.21.15.115:39202) 2023-12-01T01:51:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3da9d04640 1 -- 172.21.15.115:0/2705618298 learned_addr learned my addr 172.21.15.115:0/2705618298 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3daa505640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da406d220 0x7f3da40c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3daad06640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40ae4f0 0x7f3da40c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3da9d04640 1 -- 172.21.15.115:0/2705618298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40ae4f0 msgr2=0x7f3da40c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3da9d04640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40ae4f0 0x7f3da40c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3da9d04640 1 -- 172.21.15.115:0/2705618298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da406d220 msgr2=0x7f3da40c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:28.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3da9d04640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da406d220 0x7f3da40c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:28.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3da9d04640 1 -- 172.21.15.115:0/2705618298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0094d0 con 0x7f3da406de30 2023-12-01T01:51:28.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3daa505640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da406d220 0x7f3da40c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:28.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.892+0000 7f3daad06640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40ae4f0 0x7f3da40c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:28.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.893+0000 7f3da9d04640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da406de30 0x7f3da4147420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000c400 tx=0x7f3d9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.893+0000 7f3da2ffd640 1 -- 172.21.15.115:0/2705618298 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d900052d0 con 0x7f3da406de30 2023-12-01T01:51:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.893+0000 7f3da2ffd640 1 -- 172.21.15.115:0/2705618298 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d90005430 con 0x7f3da406de30 2023-12-01T01:51:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.893+0000 7f3dac769640 1 -- 172.21.15.115:0/2705618298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d98009a70 con 0x7f3da406de30 2023-12-01T01:51:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.893+0000 7f3da2ffd640 1 -- 172.21.15.115:0/2705618298 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90003830 con 0x7f3da406de30 2023-12-01T01:51:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.893+0000 7f3dac769640 1 -- 172.21.15.115:0/2705618298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da41547d0 con 0x7f3da406de30 2023-12-01T01:51:28.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.895+0000 7f3dac769640 1 -- 172.21.15.115:0/2705618298 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da40c7370 con 0x7f3da406de30 2023-12-01T01:51:28.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.896+0000 7f3da2ffd640 1 -- 172.21.15.115:0/2705618298 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3d90003990 con 0x7f3da406de30 2023-12-01T01:51:28.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.896+0000 7f3da2ffd640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d8806a550 0x7f3d8806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:28.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.897+0000 7f3da2ffd640 1 -- 172.21.15.115:0/2705618298 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d900880c0 con 0x7f3da406de30 2023-12-01T01:51:28.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.897+0000 7f3daa505640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d8806a550 0x7f3d8806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:28.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.897+0000 7f3daa505640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d8806a550 0x7f3d8806ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d98036000 tx=0x7f3d98030000 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:28.901+0000 7f3da2ffd640 1 -- 172.21.15.115:0/2705618298 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3da40c7370 con 0x7f3da406de30 2023-12-01T01:51:29.077 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:51:29.087 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:51:29.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.088+0000 7f3dac769640 1 -- 172.21.15.115:0/2705618298 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3da40aca40 con 0x7f3d8806a550 2023-12-01T01:51:29.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.102+0000 7f3da2ffd640 1 -- 172.21.15.115:0/2705618298 <== mgr.6201 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283112 (secure 0 0 0) 0x7f3d98044090 con 0x7f3d8806a550 2023-12-01T01:51:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:29.108 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:51:29.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 -- 172.21.15.115:0/2705618298 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d8806a550 msgr2=0x7f3d8806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:29.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d8806a550 0x7f3d8806ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d98036000 tx=0x7f3d98030000 comp rx=0 tx=0).stop 2023-12-01T01:51:29.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 -- 172.21.15.115:0/2705618298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da406de30 msgr2=0x7f3da4147420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:29.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da406de30 0x7f3da4147420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000c400 tx=0x7f3d9000c8c0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 -- 172.21.15.115:0/2705618298 shutdown_connections 2023-12-01T01:51:29.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d8806a550 0x7f3d8806ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40ae4f0 0x7f3da40c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da406de30 0x7f3da4147420 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 --2- 172.21.15.115:0/2705618298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da406d220 0x7f3da40c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 -- 172.21.15.115:0/2705618298 >> 172.21.15.115:0/2705618298 conn(0x7f3da40cbb10 msgr2=0x7f3da40ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:29.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 -- 172.21.15.115:0/2705618298 shutdown_connections 2023-12-01T01:51:29.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.107+0000 7f3dac769640 1 -- 172.21.15.115:0/2705618298 wait complete. 2023-12-01T01:51:29.126 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T01:51:27.824598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265324,"kb_used_data":8748,"kb_used_omap":0,"kb_used_meta":256512,"kb_avail":754709396,"statfs":{"total":773094113280,"available":772822421504,"internally_reserved":0,"allocated":8957952,"data_stored":4848865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":262668288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.133577"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.720663+0000","last_change":"2023-12-01T01:51:23.720663+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.720663+0000","last_undegraded":"2023-12-01T01:51:23.720663+0000","last_fullsized":"2023-12-01T01:51:23.720663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:21.413565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710355+0000","last_change":"2023-12-01T01:51:23.710355+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710355+0000","last_undegraded":"2023-12-01T01:51:23.710355+0000","last_fullsized":"2023-12-01T01:51:23.710355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:26.173230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.706388+0000","last_change":"2023-12-01T01:51:23.706388+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.706388+0000","last_undegraded":"2023-12-01T01:51:23.706388+0000","last_fullsized":"2023-12-01T01:51:23.706388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:50.301817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.721934+0000","last_change":"2023-12-01T01:51:23.721934+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.721934+0000","last_undegraded":"2023-12-01T01:51:23.721934+0000","last_fullsized":"2023-12-01T01:51:23.721934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:03:53.147122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719862+0000","last_change":"2023-12-01T01:51:23.719862+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719862+0000","last_undegraded":"2023-12-01T01:51:23.719862+0000","last_fullsized":"2023-12-01T01:51:23.719862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:16.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.718565+0000","last_change":"2023-12-01T01:51:23.718565+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.718565+0000","last_undegraded":"2023-12-01T01:51:23.718565+0000","last_fullsized":"2023-12-01T01:51:23.718565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:08.979742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.717382+0000","last_change":"2023-12-01T01:51:23.717382+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.717382+0000","last_undegraded":"2023-12-01T01:51:23.717382+0000","last_fullsized":"2023-12-01T01:51:23.717382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:41.379891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.725226+0000","last_change":"2023-12-01T01:51:23.725226+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.725226+0000","last_undegraded":"2023-12-01T01:51:23.725226+0000","last_fullsized":"2023-12-01T01:51:23.725226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:56:56.216365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719271+0000","last_change":"2023-12-01T01:51:23.719271+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719271+0000","last_undegraded":"2023-12-01T01:51:23.719271+0000","last_fullsized":"2023-12-01T01:51:23.719271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:55.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719205+0000","last_change":"2023-12-01T01:51:23.719205+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719205+0000","last_undegraded":"2023-12-01T01:51:23.719205+0000","last_fullsized":"2023-12-01T01:51:23.719205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:52.689385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.702860+0000","last_change":"2023-12-01T01:51:23.702860+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.702860+0000","last_undegraded":"2023-12-01T01:51:23.702860+0000","last_fullsized":"2023-12-01T01:51:23.702860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:21.413565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.720658+0000","last_change":"2023-12-01T01:51:23.720658+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.720658+0000","last_undegraded":"2023-12-01T01:51:23.720658+0000","last_fullsized":"2023-12-01T01:51:23.720658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:32.142316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.721909+0000","last_change":"2023-12-01T01:51:23.721909+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.721909+0000","last_undegraded":"2023-12-01T01:51:23.721909+0000","last_fullsized":"2023-12-01T01:51:23.721909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:01.177699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.702061+0000","last_change":"2023-12-01T01:51:23.702061+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.702061+0000","last_undegraded":"2023-12-01T01:51:23.702061+0000","last_fullsized":"2023-12-01T01:51:23.702061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:29.110060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.718688+0000","last_change":"2023-12-01T01:51:23.718688+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.718688+0000","last_undegraded":"2023-12-01T01:51:23.718688+0000","last_fullsized":"2023-12-01T01:51:23.718688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719266+0000","last_change":"2023-12-01T01:51:23.719266+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719266+0000","last_undegraded":"2023-12-01T01:51:23.719266+0000","last_fullsized":"2023-12-01T01:51:23.719266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:28.427573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.717782+0000","last_change":"2023-12-01T01:51:23.717782+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.717782+0000","last_undegraded":"2023-12-01T01:51:23.717782+0000","last_fullsized":"2023-12-01T01:51:23.717782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710478+0000","last_change":"2023-12-01T01:51:23.710478+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710478+0000","last_undegraded":"2023-12-01T01:51:23.710478+0000","last_fullsized":"2023-12-01T01:51:23.710478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:27.535710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.718246+0000","last_change":"2023-12-01T01:51:23.718246+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.718246+0000","last_undegraded":"2023-12-01T01:51:23.718246+0000","last_fullsized":"2023-12-01T01:51:23.718246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:14.157429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.701921+0000","last_change":"2023-12-01T01:51:23.701921+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.701921+0000","last_undegraded":"2023-12-01T01:51:23.701921+0000","last_fullsized":"2023-12-01T01:51:23.701921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:08.979742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719563+0000","last_change":"2023-12-01T01:51:23.719563+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719563+0000","last_undegraded":"2023-12-01T01:51:23.719563+0000","last_fullsized":"2023-12-01T01:51:23.719563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.702566+0000","last_change":"2023-12-01T01:51:23.702566+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.702566+0000","last_undegraded":"2023-12-01T01:51:23.702566+0000","last_fullsized":"2023-12-01T01:51:23.702566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:12.784193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.715707+0000","last_change":"2023-12-01T01:51:23.715707+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.715707+0000","last_undegraded":"2023-12-01T01:51:23.715707+0000","last_fullsized":"2023-12-01T01:51:23.715707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:39:57.857405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719436+0000","last_change":"2023-12-01T01:51:23.719436+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719436+0000","last_undegraded":"2023-12-01T01:51:23.719436+0000","last_fullsized":"2023-12-01T01:51:23.719436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:51.772038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710118+0000","last_change":"2023-12-01T01:51:23.710118+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710118+0000","last_undegraded":"2023-12-01T01:51:23.710118+0000","last_fullsized":"2023-12-01T01:51:23.710118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:06:06.433309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.718549+0000","last_change":"2023-12-01T01:51:23.718549+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.718549+0000","last_undegraded":"2023-12-01T01:51:23.718549+0000","last_fullsized":"2023-12-01T01:51:23.718549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:10.334292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.714681+0000","last_change":"2023-12-01T01:51:23.714681+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.714681+0000","last_undegraded":"2023-12-01T01:51:23.714681+0000","last_fullsized":"2023-12-01T01:51:23.714681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:10:04.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.727172+0000","last_change":"2023-12-01T01:51:23.727172+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.727172+0000","last_undegraded":"2023-12-01T01:51:23.727172+0000","last_fullsized":"2023-12-01T01:51:23.727172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:46.030193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.720918+0000","last_change":"2023-12-01T01:51:23.720918+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.720918+0000","last_undegraded":"2023-12-01T01:51:23.720918+0000","last_fullsized":"2023-12-01T01:51:23.720918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:49.877816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.714310+0000","last_change":"2023-12-01T01:51:23.714310+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.714310+0000","last_undegraded":"2023-12-01T01:51:23.714310+0000","last_fullsized":"2023-12-01T01:51:23.714310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:28.026095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710305+0000","last_change":"2023-12-01T01:51:23.710305+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710305+0000","last_undegraded":"2023-12-01T01:51:23.710305+0000","last_fullsized":"2023-12-01T01:51:23.710305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:52.879295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.701734+0000","last_change":"2023-12-01T01:51:23.701734+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.701734+0000","last_undegraded":"2023-12-01T01:51:23.701734+0000","last_fullsized":"2023-12-01T01:51:23.701734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:16.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.713643+0000","last_change":"2023-12-01T01:51:23.713643+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.713643+0000","last_undegraded":"2023-12-01T01:51:23.713643+0000","last_fullsized":"2023-12-01T01:51:23.713643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:01:44.563857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.728135+0000","last_change":"2023-12-01T01:51:23.722626+0000","last_active":"2023-12-01T01:51:23.728135+0000","last_peered":"2023-12-01T01:51:23.728135+0000","last_clean":"2023-12-01T01:51:23.728135+0000","last_became_active":"2023-12-01T01:51:23.722272+0000","last_became_peered":"2023-12-01T01:51:23.722272+0000","last_unstale":"2023-12-01T01:51:23.728135+0000","last_undegraded":"2023-12-01T01:51:23.728135+0000","last_fullsized":"2023-12-01T01:51:23.728135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:49.297425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.715745+0000","last_change":"2023-12-01T01:51:23.715745+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.715745+0000","last_undegraded":"2023-12-01T01:51:23.715745+0000","last_fullsized":"2023-12-01T01:51:23.715745+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:12.784193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.734229+0000","last_change":"2023-12-01T01:51:23.730769+0000","last_active":"2023-12-01T01:51:23.734229+0000","last_peered":"2023-12-01T01:51:23.734229+0000","last_clean":"2023-12-01T01:51:23.734229+0000","last_became_active":"2023-12-01T01:51:23.730468+0000","last_became_peered":"2023-12-01T01:51:23.730468+0000","last_unstale":"2023-12-01T01:51:23.734229+0000","last_undegraded":"2023-12-01T01:51:23.734229+0000","last_fullsized":"2023-12-01T01:51:23.734229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:56.584539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.726700+0000","last_change":"2023-12-01T01:51:23.726700+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.726700+0000","last_undegraded":"2023-12-01T01:51:23.726700+0000","last_fullsized":"2023-12-01T01:51:23.726700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:53:01.496510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.734707+0000","last_change":"2023-12-01T01:51:23.729205+0000","last_active":"2023-12-01T01:51:23.734707+0000","last_peered":"2023-12-01T01:51:23.734707+0000","last_clean":"2023-12-01T01:51:23.734707+0000","last_became_active":"2023-12-01T01:51:23.728920+0000","last_became_peered":"2023-12-01T01:51:23.728920+0000","last_unstale":"2023-12-01T01:51:23.734707+0000","last_undegraded":"2023-12-01T01:51:23.734707+0000","last_fullsized":"2023-12-01T01:51:23.734707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:21:58.877785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.721738+0000","last_change":"2023-12-01T01:51:23.721738+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.721738+0000","last_undegraded":"2023-12-01T01:51:23.721738+0000","last_fullsized":"2023-12-01T01:51:23.721738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:01.177699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.732154+0000","last_change":"2023-12-01T01:51:23.729190+0000","last_active":"2023-12-01T01:51:23.732154+0000","last_peered":"2023-12-01T01:51:23.732154+0000","last_clean":"2023-12-01T01:51:23.732154+0000","last_became_active":"2023-12-01T01:51:23.728867+0000","last_became_peered":"2023-12-01T01:51:23.728867+0000","last_unstale":"2023-12-01T01:51:23.732154+0000","last_undegraded":"2023-12-01T01:51:23.732154+0000","last_fullsized":"2023-12-01T01:51:23.732154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:43:10.966577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.718697+0000","last_change":"2023-12-01T01:51:23.718697+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.718697+0000","last_undegraded":"2023-12-01T01:51:23.718697+0000","last_fullsized":"2023-12-01T01:51:23.718697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:01:53.075668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.721289+0000","last_change":"2023-12-01T01:51:23.717746+0000","last_active":"2023-12-01T01:51:23.721289+0000","last_peered":"2023-12-01T01:51:23.721289+0000","last_clean":"2023-12-01T01:51:23.721289+0000","last_became_active":"2023-12-01T01:51:23.715918+0000","last_became_peered":"2023-12-01T01:51:23.715918+0000","last_unstale":"2023-12-01T01:51:23.721289+0000","last_undegraded":"2023-12-01T01:51:23.721289+0000","last_fullsized":"2023-12-01T01:51:23.721289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:35:50.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.701843+0000","last_change":"2023-12-01T01:51:23.701843+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.701843+0000","last_undegraded":"2023-12-01T01:51:23.701843+0000","last_fullsized":"2023-12-01T01:51:23.701843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:06.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.730693+0000","last_change":"2023-12-01T01:51:23.725607+0000","last_active":"2023-12-01T01:51:23.730693+0000","last_peered":"2023-12-01T01:51:23.730693+0000","last_clean":"2023-12-01T01:51:23.730693+0000","last_became_active":"2023-12-01T01:51:23.722833+0000","last_became_peered":"2023-12-01T01:51:23.722833+0000","last_unstale":"2023-12-01T01:51:23.730693+0000","last_undegraded":"2023-12-01T01:51:23.730693+0000","last_fullsized":"2023-12-01T01:51:23.730693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:10:03.788374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.701782+0000","last_change":"2023-12-01T01:51:23.701782+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.701782+0000","last_undegraded":"2023-12-01T01:51:23.701782+0000","last_fullsized":"2023-12-01T01:51:23.701782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:51.144418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.732197+0000","last_change":"2023-12-01T01:51:23.729822+0000","last_active":"2023-12-01T01:51:23.732197+0000","last_peered":"2023-12-01T01:51:23.732197+0000","last_clean":"2023-12-01T01:51:23.732197+0000","last_became_active":"2023-12-01T01:51:23.728147+0000","last_became_peered":"2023-12-01T01:51:23.728147+0000","last_unstale":"2023-12-01T01:51:23.732197+0000","last_undegraded":"2023-12-01T01:51:23.732197+0000","last_fullsized":"2023-12-01T01:51:23.732197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:43:10.667895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719104+0000","last_change":"2023-12-01T01:51:23.719104+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719104+0000","last_undegraded":"2023-12-01T01:51:23.719104+0000","last_fullsized":"2023-12-01T01:51:23.719104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.713786+0000","last_change":"2023-12-01T01:51:23.710030+0000","last_active":"2023-12-01T01:51:23.713786+0000","last_peered":"2023-12-01T01:51:23.713786+0000","last_clean":"2023-12-01T01:51:23.713786+0000","last_became_active":"2023-12-01T01:51:23.709381+0000","last_became_peered":"2023-12-01T01:51:23.709381+0000","last_unstale":"2023-12-01T01:51:23.713786+0000","last_undegraded":"2023-12-01T01:51:23.713786+0000","last_fullsized":"2023-12-01T01:51:23.713786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:25.168837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.712668+0000","last_change":"2023-12-01T01:51:23.712668+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.712668+0000","last_undegraded":"2023-12-01T01:51:23.712668+0000","last_fullsized":"2023-12-01T01:51:23.712668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:50.301817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.729093+0000","last_change":"2023-12-01T01:51:23.725370+0000","last_active":"2023-12-01T01:51:23.729093+0000","last_peered":"2023-12-01T01:51:23.729093+0000","last_clean":"2023-12-01T01:51:23.729093+0000","last_became_active":"2023-12-01T01:51:23.725113+0000","last_became_peered":"2023-12-01T01:51:23.725113+0000","last_unstale":"2023-12-01T01:51:23.729093+0000","last_undegraded":"2023-12-01T01:51:23.729093+0000","last_fullsized":"2023-12-01T01:51:23.729093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:17.602658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.714798+0000","last_change":"2023-12-01T01:51:23.714798+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.714798+0000","last_undegraded":"2023-12-01T01:51:23.714798+0000","last_fullsized":"2023-12-01T01:51:23.714798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736347+0000","last_change":"2023-12-01T01:51:23.733129+0000","last_active":"2023-12-01T01:51:23.736347+0000","last_peered":"2023-12-01T01:51:23.736347+0000","last_clean":"2023-12-01T01:51:23.736347+0000","last_became_active":"2023-12-01T01:51:23.732687+0000","last_became_peered":"2023-12-01T01:51:23.732687+0000","last_unstale":"2023-12-01T01:51:23.736347+0000","last_undegraded":"2023-12-01T01:51:23.736347+0000","last_fullsized":"2023-12-01T01:51:23.736347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:59.339210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.722932+0000","last_change":"2023-12-01T01:51:23.722932+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.722932+0000","last_undegraded":"2023-12-01T01:51:23.722932+0000","last_fullsized":"2023-12-01T01:51:23.722932+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:54:45.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.727027+0000","last_change":"2023-12-01T01:51:23.723356+0000","last_active":"2023-12-01T01:51:23.727027+0000","last_peered":"2023-12-01T01:51:23.727027+0000","last_clean":"2023-12-01T01:51:23.727027+0000","last_became_active":"2023-12-01T01:51:23.723040+0000","last_became_peered":"2023-12-01T01:51:23.723040+0000","last_unstale":"2023-12-01T01:51:23.727027+0000","last_undegraded":"2023-12-01T01:51:23.727027+0000","last_fullsized":"2023-12-01T01:51:23.727027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:15.500740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.717320+0000","last_change":"2023-12-01T01:51:23.717320+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.717320+0000","last_undegraded":"2023-12-01T01:51:23.717320+0000","last_fullsized":"2023-12-01T01:51:23.717320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:23.126569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.727807+0000","last_change":"2023-12-01T01:51:23.725813+0000","last_active":"2023-12-01T01:51:23.727807+0000","last_peered":"2023-12-01T01:51:23.727807+0000","last_clean":"2023-12-01T01:51:23.727807+0000","last_became_active":"2023-12-01T01:51:23.723452+0000","last_became_peered":"2023-12-01T01:51:23.723452+0000","last_unstale":"2023-12-01T01:51:23.727807+0000","last_undegraded":"2023-12-01T01:51:23.727807+0000","last_fullsized":"2023-12-01T01:51:23.727807+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:46.793866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.720422+0000","last_change":"2023-12-01T01:51:23.720422+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.720422+0000","last_undegraded":"2023-12-01T01:51:23.720422+0000","last_fullsized":"2023-12-01T01:51:23.720422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.469019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736241+0000","last_change":"2023-12-01T01:51:23.732400+0000","last_active":"2023-12-01T01:51:23.736241+0000","last_peered":"2023-12-01T01:51:23.736241+0000","last_clean":"2023-12-01T01:51:23.736241+0000","last_became_active":"2023-12-01T01:51:23.730066+0000","last_became_peered":"2023-12-01T01:51:23.730066+0000","last_unstale":"2023-12-01T01:51:23.736241+0000","last_undegraded":"2023-12-01T01:51:23.736241+0000","last_fullsized":"2023-12-01T01:51:23.736241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:49.297425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.703529+0000","last_change":"2023-12-01T01:51:23.703529+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.703529+0000","last_undegraded":"2023-12-01T01:51:23.703529+0000","last_fullsized":"2023-12-01T01:51:23.703529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:10.334292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.727316+0000","last_change":"2023-12-01T01:51:23.722740+0000","last_active":"2023-12-01T01:51:23.727316+0000","last_peered":"2023-12-01T01:51:23.727316+0000","last_clean":"2023-12-01T01:51:23.727316+0000","last_became_active":"2023-12-01T01:51:23.722478+0000","last_became_peered":"2023-12-01T01:51:23.722478+0000","last_unstale":"2023-12-01T01:51:23.727316+0000","last_undegraded":"2023-12-01T01:51:23.727316+0000","last_fullsized":"2023-12-01T01:51:23.727316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:01:52.071276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.716303+0000","last_change":"2023-12-01T01:51:23.716303+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.716303+0000","last_undegraded":"2023-12-01T01:51:23.716303+0000","last_fullsized":"2023-12-01T01:51:23.716303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:33.729872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.726783+0000","last_change":"2023-12-01T01:51:23.725460+0000","last_active":"2023-12-01T01:51:23.726783+0000","last_peered":"2023-12-01T01:51:23.726783+0000","last_clean":"2023-12-01T01:51:23.726783+0000","last_became_active":"2023-12-01T01:51:23.725192+0000","last_became_peered":"2023-12-01T01:51:23.725192+0000","last_unstale":"2023-12-01T01:51:23.726783+0000","last_undegraded":"2023-12-01T01:51:23.726783+0000","last_fullsized":"2023-12-01T01:51:23.726783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:14.033061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.715228+0000","last_change":"2023-12-01T01:51:23.715228+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.715228+0000","last_undegraded":"2023-12-01T01:51:23.715228+0000","last_fullsized":"2023-12-01T01:51:23.715228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:49.877816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.730771+0000","last_change":"2023-12-01T01:51:23.728019+0000","last_active":"2023-12-01T01:51:23.730771+0000","last_peered":"2023-12-01T01:51:23.730771+0000","last_clean":"2023-12-01T01:51:23.730771+0000","last_became_active":"2023-12-01T01:51:23.727690+0000","last_became_peered":"2023-12-01T01:51:23.727690+0000","last_unstale":"2023-12-01T01:51:23.730771+0000","last_undegraded":"2023-12-01T01:51:23.730771+0000","last_fullsized":"2023-12-01T01:51:23.730771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:54:44.691969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710032+0000","last_change":"2023-12-01T01:51:23.710032+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710032+0000","last_undegraded":"2023-12-01T01:51:23.710032+0000","last_fullsized":"2023-12-01T01:51:23.710032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:14.157429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736371+0000","last_change":"2023-12-01T01:51:23.733218+0000","last_active":"2023-12-01T01:51:23.736371+0000","last_peered":"2023-12-01T01:51:23.736371+0000","last_clean":"2023-12-01T01:51:23.736371+0000","last_became_active":"2023-12-01T01:51:23.732910+0000","last_became_peered":"2023-12-01T01:51:23.732910+0000","last_unstale":"2023-12-01T01:51:23.736371+0000","last_undegraded":"2023-12-01T01:51:23.736371+0000","last_fullsized":"2023-12-01T01:51:23.736371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:54.751334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.712022+0000","last_change":"2023-12-01T01:51:23.712022+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.712022+0000","last_undegraded":"2023-12-01T01:51:23.712022+0000","last_fullsized":"2023-12-01T01:51:23.712022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.738667+0000","last_change":"2023-12-01T01:51:23.733533+0000","last_active":"2023-12-01T01:51:23.738667+0000","last_peered":"2023-12-01T01:51:23.738667+0000","last_clean":"2023-12-01T01:51:23.738667+0000","last_became_active":"2023-12-01T01:51:23.733196+0000","last_became_peered":"2023-12-01T01:51:23.733196+0000","last_unstale":"2023-12-01T01:51:23.738667+0000","last_undegraded":"2023-12-01T01:51:23.738667+0000","last_fullsized":"2023-12-01T01:51:23.738667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:09.329899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710494+0000","last_change":"2023-12-01T01:51:23.710494+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710494+0000","last_undegraded":"2023-12-01T01:51:23.710494+0000","last_fullsized":"2023-12-01T01:51:23.710494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:12:27.401881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.731329+0000","last_change":"2023-12-01T01:51:23.728528+0000","last_active":"2023-12-01T01:51:23.731329+0000","last_peered":"2023-12-01T01:51:23.731329+0000","last_clean":"2023-12-01T01:51:23.731329+0000","last_became_active":"2023-12-01T01:51:23.728266+0000","last_became_peered":"2023-12-01T01:51:23.728266+0000","last_unstale":"2023-12-01T01:51:23.731329+0000","last_undegraded":"2023-12-01T01:51:23.731329+0000","last_fullsized":"2023-12-01T01:51:23.731329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:18:10.249852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.716734+0000","last_change":"2023-12-01T01:51:23.716734+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.716734+0000","last_undegraded":"2023-12-01T01:51:23.716734+0000","last_fullsized":"2023-12-01T01:51:23.716734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:03.279418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736360+0000","last_change":"2023-12-01T01:51:23.733364+0000","last_active":"2023-12-01T01:51:23.736360+0000","last_peered":"2023-12-01T01:51:23.736360+0000","last_clean":"2023-12-01T01:51:23.736360+0000","last_became_active":"2023-12-01T01:51:23.732811+0000","last_became_peered":"2023-12-01T01:51:23.732811+0000","last_unstale":"2023-12-01T01:51:23.736360+0000","last_undegraded":"2023-12-01T01:51:23.736360+0000","last_fullsized":"2023-12-01T01:51:23.736360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:17.784149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719794+0000","last_change":"2023-12-01T01:51:23.719794+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719794+0000","last_undegraded":"2023-12-01T01:51:23.719794+0000","last_fullsized":"2023-12-01T01:51:23.719794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:57.588932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.721237+0000","last_change":"2023-12-01T01:51:23.717087+0000","last_active":"2023-12-01T01:51:23.721237+0000","last_peered":"2023-12-01T01:51:23.721237+0000","last_clean":"2023-12-01T01:51:23.721237+0000","last_became_active":"2023-12-01T01:51:23.712593+0000","last_became_peered":"2023-12-01T01:51:23.712593+0000","last_unstale":"2023-12-01T01:51:23.721237+0000","last_undegraded":"2023-12-01T01:51:23.721237+0000","last_fullsized":"2023-12-01T01:51:23.721237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:09.329899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.720276+0000","last_change":"2023-12-01T01:51:23.720276+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.720276+0000","last_undegraded":"2023-12-01T01:51:23.720276+0000","last_fullsized":"2023-12-01T01:51:23.720276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:09.806552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736322+0000","last_change":"2023-12-01T01:51:23.732919+0000","last_active":"2023-12-01T01:51:23.736322+0000","last_peered":"2023-12-01T01:51:23.736322+0000","last_clean":"2023-12-01T01:51:23.736322+0000","last_became_active":"2023-12-01T01:51:23.732477+0000","last_became_peered":"2023-12-01T01:51:23.732477+0000","last_unstale":"2023-12-01T01:51:23.736322+0000","last_undegraded":"2023-12-01T01:51:23.736322+0000","last_fullsized":"2023-12-01T01:51:23.736322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:28.105668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.724998+0000","last_change":"2023-12-01T01:51:23.724998+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.724998+0000","last_undegraded":"2023-12-01T01:51:23.724998+0000","last_fullsized":"2023-12-01T01:51:23.724998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:55.560831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.733228+0000","last_change":"2023-12-01T01:51:23.732268+0000","last_active":"2023-12-01T01:51:23.733228+0000","last_peered":"2023-12-01T01:51:23.733228+0000","last_clean":"2023-12-01T01:51:23.733228+0000","last_became_active":"2023-12-01T01:51:23.730931+0000","last_became_peered":"2023-12-01T01:51:23.730931+0000","last_unstale":"2023-12-01T01:51:23.733228+0000","last_undegraded":"2023-12-01T01:51:23.733228+0000","last_fullsized":"2023-12-01T01:51:23.733228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:43.652247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.709575+0000","last_change":"2023-12-01T01:51:23.709575+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.709575+0000","last_undegraded":"2023-12-01T01:51:23.709575+0000","last_fullsized":"2023-12-01T01:51:23.709575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:55:18.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736391+0000","last_change":"2023-12-01T01:51:23.735050+0000","last_active":"2023-12-01T01:51:23.736391+0000","last_peered":"2023-12-01T01:51:23.736391+0000","last_clean":"2023-12-01T01:51:23.736391+0000","last_became_active":"2023-12-01T01:51:23.734653+0000","last_became_peered":"2023-12-01T01:51:23.734653+0000","last_unstale":"2023-12-01T01:51:23.736391+0000","last_undegraded":"2023-12-01T01:51:23.736391+0000","last_fullsized":"2023-12-01T01:51:23.736391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:06.069833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.711558+0000","last_change":"2023-12-01T01:51:23.711558+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.711558+0000","last_undegraded":"2023-12-01T01:51:23.711558+0000","last_fullsized":"2023-12-01T01:51:23.711558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:34:25.329128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.730297+0000","last_change":"2023-12-01T01:51:23.724706+0000","last_active":"2023-12-01T01:51:23.730297+0000","last_peered":"2023-12-01T01:51:23.730297+0000","last_clean":"2023-12-01T01:51:23.730297+0000","last_became_active":"2023-12-01T01:51:23.723272+0000","last_became_peered":"2023-12-01T01:51:23.723272+0000","last_unstale":"2023-12-01T01:51:23.730297+0000","last_undegraded":"2023-12-01T01:51:23.730297+0000","last_fullsized":"2023-12-01T01:51:23.730297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:51.874903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.709087+0000","last_change":"2023-12-01T01:51:23.709087+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.709087+0000","last_undegraded":"2023-12-01T01:51:23.709087+0000","last_fullsized":"2023-12-01T01:51:23.709087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:47.798259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.728265+0000","last_change":"2023-12-01T01:51:23.721602+0000","last_active":"2023-12-01T01:51:23.728265+0000","last_peered":"2023-12-01T01:51:23.728265+0000","last_clean":"2023-12-01T01:51:23.728265+0000","last_became_active":"2023-12-01T01:51:23.721352+0000","last_became_peered":"2023-12-01T01:51:23.721352+0000","last_unstale":"2023-12-01T01:51:23.728265+0000","last_undegraded":"2023-12-01T01:51:23.728265+0000","last_fullsized":"2023-12-01T01:51:23.728265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:44.064139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.726907+0000","last_change":"2023-12-01T01:51:23.726907+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.726907+0000","last_undegraded":"2023-12-01T01:51:23.726907+0000","last_fullsized":"2023-12-01T01:51:23.726907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:13.923540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.733406+0000","last_change":"2023-12-01T01:51:23.732583+0000","last_active":"2023-12-01T01:51:23.733406+0000","last_peered":"2023-12-01T01:51:23.733406+0000","last_clean":"2023-12-01T01:51:23.733406+0000","last_became_active":"2023-12-01T01:51:23.731083+0000","last_became_peered":"2023-12-01T01:51:23.731083+0000","last_unstale":"2023-12-01T01:51:23.733406+0000","last_undegraded":"2023-12-01T01:51:23.733406+0000","last_fullsized":"2023-12-01T01:51:23.733406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:36.702370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.713608+0000","last_change":"2023-12-01T01:51:23.713608+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.713608+0000","last_undegraded":"2023-12-01T01:51:23.713608+0000","last_fullsized":"2023-12-01T01:51:23.713608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:06.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.735924+0000","last_change":"2023-12-01T01:51:23.732282+0000","last_active":"2023-12-01T01:51:23.735924+0000","last_peered":"2023-12-01T01:51:23.735924+0000","last_clean":"2023-12-01T01:51:23.735924+0000","last_became_active":"2023-12-01T01:51:23.732016+0000","last_became_peered":"2023-12-01T01:51:23.732016+0000","last_unstale":"2023-12-01T01:51:23.735924+0000","last_undegraded":"2023-12-01T01:51:23.735924+0000","last_fullsized":"2023-12-01T01:51:23.735924+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:50.767646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.712390+0000","last_change":"2023-12-01T01:51:23.712390+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.712390+0000","last_undegraded":"2023-12-01T01:51:23.712390+0000","last_fullsized":"2023-12-01T01:51:23.712390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:29.110060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.724154+0000","last_change":"2023-12-01T01:51:23.720558+0000","last_active":"2023-12-01T01:51:23.724154+0000","last_peered":"2023-12-01T01:51:23.724154+0000","last_clean":"2023-12-01T01:51:23.724154+0000","last_became_active":"2023-12-01T01:51:23.720288+0000","last_became_peered":"2023-12-01T01:51:23.720288+0000","last_unstale":"2023-12-01T01:51:23.724154+0000","last_undegraded":"2023-12-01T01:51:23.724154+0000","last_fullsized":"2023-12-01T01:51:23.724154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:54:32.273772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.717335+0000","last_change":"2023-12-01T01:51:23.717335+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.717335+0000","last_undegraded":"2023-12-01T01:51:23.717335+0000","last_fullsized":"2023-12-01T01:51:23.717335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:15.037453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.734858+0000","last_change":"2023-12-01T01:51:23.730111+0000","last_active":"2023-12-01T01:51:23.734858+0000","last_peered":"2023-12-01T01:51:23.734858+0000","last_clean":"2023-12-01T01:51:23.734858+0000","last_became_active":"2023-12-01T01:51:23.729813+0000","last_became_peered":"2023-12-01T01:51:23.729813+0000","last_unstale":"2023-12-01T01:51:23.734858+0000","last_undegraded":"2023-12-01T01:51:23.734858+0000","last_fullsized":"2023-12-01T01:51:23.734858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:06.524585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.708922+0000","last_change":"2023-12-01T01:51:23.708922+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.708922+0000","last_undegraded":"2023-12-01T01:51:23.708922+0000","last_fullsized":"2023-12-01T01:51:23.708922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.360859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736201+0000","last_change":"2023-12-01T01:51:23.731063+0000","last_active":"2023-12-01T01:51:23.736201+0000","last_peered":"2023-12-01T01:51:23.736201+0000","last_clean":"2023-12-01T01:51:23.736201+0000","last_became_active":"2023-12-01T01:51:23.730608+0000","last_became_peered":"2023-12-01T01:51:23.730608+0000","last_unstale":"2023-12-01T01:51:23.736201+0000","last_undegraded":"2023-12-01T01:51:23.736201+0000","last_fullsized":"2023-12-01T01:51:23.736201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:34:24.324736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.715721+0000","last_change":"2023-12-01T01:51:23.715721+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.715721+0000","last_undegraded":"2023-12-01T01:51:23.715721+0000","last_fullsized":"2023-12-01T01:51:23.715721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:31:57.782696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.713619+0000","last_change":"2023-12-01T01:51:23.709986+0000","last_active":"2023-12-01T01:51:23.713619+0000","last_peered":"2023-12-01T01:51:23.713619+0000","last_clean":"2023-12-01T01:51:23.713619+0000","last_became_active":"2023-12-01T01:51:23.709276+0000","last_became_peered":"2023-12-01T01:51:23.709276+0000","last_unstale":"2023-12-01T01:51:23.713619+0000","last_undegraded":"2023-12-01T01:51:23.713619+0000","last_fullsized":"2023-12-01T01:51:23.713619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:06:05.428917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'161","reported_seq":380,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-12-01T01:51:22.703508+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:51:22.703508+0000","last_peered":"2023-12-01T01:51:22.703508+0000","last_clean":"2023-12-01T01:51:22.703508+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:51:22.703508+0000","last_undegraded":"2023-12-01T01:51:22.703508+0000","last_fullsized":"2023-12-01T01:51:22.703508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":966656,"data_stored":456163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.79700000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3979999999999999,"5min":1.5549999999999999,"15min":1.8089999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.073,"15min":1.05},"min":{"1min":0.82899999999999996,"5min":0.64700000000000002,"15min":0.64000000000000001},"max":{"1min":1.353,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.2669999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.246,"15min":1.218},"min":{"1min":1.008,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.802,"5min":1.8100000000000001,"15min":1.8420000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2629999999999999,"15min":1.2569999999999999},"min":{"1min":1.0669999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5529999999999999,"5min":1.744,"15min":2.0870000000000002},"last":1.5049999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.121,"15min":1.123},"min":{"1min":0.80100000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5329999999999999,"5min":1.728,"15min":1.728},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.218,"15min":1.2290000000000001},"min":{"1min":0.92400000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.6120000000000001,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":0.95799999999999996}]},{"osd":3,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.876,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.617,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.355,"15min":1.349},"min":{"1min":1.026,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6859999999999999,"5min":2.1880000000000002,"15min":2.7050000000000001},"last":1.4199999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.2769999999999999,"15min":1.296},"min":{"1min":0.83099999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.681,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4610000000000001,"15min":1.4199999999999999},"min":{"1min":1.0609999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7370000000000001,"5min":2.0939999999999999,"15min":2.601},"last":1.534}]},{"osd":5,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.274,"15min":1.27},"min":{"1min":0.83099999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.6439999999999999,"5min":2.0870000000000002,"15min":2.4500000000000002},"last":1.579},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5309999999999999,"15min":1.526},"min":{"1min":1,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8839999999999999,"5min":2.2930000000000001,"15min":2.8069999999999999},"last":1.6160000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.393,"15min":1.3859999999999999},"min":{"1min":1.004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.77,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.645},{"interface":"front","average":{"1min":1.403,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":1.002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7929999999999999,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.4690000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":978944,"data_stored":462438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.101,"15min":1.1060000000000001},"min":{"1min":0.78900000000000003,"5min":0.69699999999999995,"15min":0.66900000000000004},"max":{"1min":1.619,"5min":1.635,"15min":1.79},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.137,"15min":1.1299999999999999},"min":{"1min":0.89600000000000002,"5min":0.64400000000000002,"15min":0.61799999999999999},"max":{"1min":1.4690000000000001,"5min":2.5569999999999999,"15min":2.73},"last":1.254}]},{"osd":1,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.167,"15min":1.153},"min":{"1min":0.94999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6759999999999999,"5min":1.8620000000000001,"15min":2.2170000000000001},"last":1.411},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.196,"15min":1.21},"min":{"1min":0.93400000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6259999999999999,"5min":1.651,"15min":2.0920000000000001},"last":1.111}]},{"osd":2,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.232,"15min":1.163},"min":{"1min":0.92000000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6359999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.27,"15min":1.25},"min":{"1min":0.877,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7549999999999999,"5min":2.137,"15min":2.137},"last":1.2869999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3009999999999999,"15min":1.2849999999999999},"min":{"1min":1.111,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.548,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.4259999999999999,"15min":1.407},"min":{"1min":1.137,"5min":1.085,"15min":0.81299999999999994},"max":{"1min":1.7130000000000001,"5min":1.758,"15min":2.4020000000000001},"last":1.3180000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.321,"15min":1.2909999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.70699999999999996},"max":{"1min":1.6919999999999999,"5min":2.226,"15min":2.3399999999999999},"last":1.169},{"interface":"front","average":{"1min":1.331,"5min":1.365,"15min":1.3580000000000001},"min":{"1min":0.94699999999999995,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.694,"5min":1.931,"15min":2.6760000000000002},"last":1.208}]},{"osd":5,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4419999999999999,"15min":1.4179999999999999},"min":{"1min":1.0469999999999999,"5min":1.0229999999999999,"15min":0.82699999999999996},"max":{"1min":1.726,"5min":2.1899999999999999,"15min":2.5539999999999998},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5109999999999999,"15min":1.482},"min":{"1min":1.1870000000000001,"5min":0.98899999999999999,"15min":0.748},"max":{"1min":1.8129999999999999,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.3740000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5660000000000001,"15min":1.5249999999999999},"min":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":0.89500000000000002},"max":{"1min":1.978,"5min":2.762,"15min":2.762},"last":1.518},{"interface":"front","average":{"1min":1.587,"5min":1.621,"15min":1.585},"min":{"1min":1.218,"5min":1.1479999999999999,"15min":0.78700000000000003},"max":{"1min":1.9259999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.4570000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575035,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1524,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1560576,"data_stored":1046531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.1060000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.498,"5min":1.651,"15min":3.3980000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.127,"15min":1.141},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.60099999999999998},"max":{"1min":1.5720000000000001,"5min":1.6479999999999999,"15min":2.153},"last":0.85199999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.153,"15min":1.137},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.7010000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.73599999999999999,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4039999999999999,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.048},{"interface":"front","average":{"1min":1.155,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.6080000000000001,"5min":1.621,"15min":2.0630000000000002},"last":1.0940000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.286},"min":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.76900000000000002},"max":{"1min":1.5309999999999999,"5min":1.694,"15min":1.833},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3120000000000001,"15min":1.296},"min":{"1min":0.98099999999999998,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":1.6779999999999999,"5min":1.7110000000000001,"15min":2.1440000000000001},"last":1.367}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.238,"15min":1.2390000000000001},"min":{"1min":0.86399999999999999,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.6779999999999999,"5min":1.853,"15min":2.5129999999999999},"last":1.423},{"interface":"front","average":{"1min":1.282,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.84999999999999998,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.6319999999999999,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.139}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2989999999999999,"15min":1.329},"min":{"1min":0.88500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":2.2970000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":1.125,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":1.962,"5min":1.962,"15min":3.23},"last":1.617}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4530000000000001,"15min":1.48},"min":{"1min":0.92400000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2,"5min":2,"15min":3.012},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4970000000000001,"15min":1.524},"min":{"1min":1.2689999999999999,"5min":0.873,"15min":0.873},"max":{"1min":2.073,"5min":2.073,"15min":2.8420000000000001},"last":1.5549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575035,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36088,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335752,"statfs":{"total":96636764160,"available":96599810048,"internally_reserved":0,"allocated":966656,"data_stored":456163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.119,"15min":1.1040000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.53,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.53},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.734,"5min":1.827,"15min":3.8069999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2350000000000001,"15min":1.2090000000000001},"min":{"1min":0.84599999999999997,"5min":0.81999999999999995,"15min":0.65900000000000003},"max":{"1min":1.528,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.29},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.131},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.671,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.04}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2110000000000001,"15min":1.196},"min":{"1min":0.83999999999999997,"5min":0.78200000000000003,"15min":0.77600000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3140000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.871,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.385}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.234},"min":{"1min":0.85399999999999998,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.8220000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3200000000000001,"15min":1.319},"min":{"1min":0.753,"5min":0.753,"15min":0.70399999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.3010000000000002},"last":1.6910000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.96599999999999997,"5min":0.79200000000000004,"15min":0.61899999999999999},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.1549999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2829999999999999,"15min":1.292},"min":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.5189999999999999,"5min":1.8029999999999999,"15min":2.2029999999999998},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.94999999999999996,"5min":0.84799999999999998,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.5029999999999999,"15min":1.4730000000000001},"min":{"1min":1.1000000000000001,"5min":0.96699999999999997,"15min":0.63200000000000001},"max":{"1min":1.7949999999999999,"5min":2.4990000000000001,"15min":2.54},"last":1.468}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.458,"15min":1.464},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.60599999999999998},"max":{"1min":1.6850000000000001,"5min":1.972,"15min":3.2789999999999999},"last":1.349},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5720000000000001,"15min":1.5580000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.04,"15min":2.157},"last":1.8140000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575034,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36088,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335752,"statfs":{"total":96636764160,"available":96599810048,"internally_reserved":0,"allocated":966656,"data_stored":456163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82899999999999996,"15min":0.85199999999999998},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.307},"max":{"1min":1.135,"5min":1.302,"15min":1.583},"last":0.873},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.999,"15min":1.0009999999999999},"min":{"1min":0.52100000000000002,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.3999999999999999,"5min":1.4990000000000001,"15min":1.9770000000000001},"last":1.1870000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.069,"15min":1.0629999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.38900000000000001},"max":{"1min":1.556,"5min":1.571,"15min":2.0779999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.95399999999999996,"15min":0.95199999999999996},"min":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.40600000000000003},"max":{"1min":1.5129999999999999,"5min":1.587,"15min":1.6519999999999999},"last":0.89900000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.016,"15min":1.054},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.436},"max":{"1min":1.286,"5min":1.512,"15min":1.766},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.173,"15min":1.1759999999999999},"min":{"1min":0.63800000000000001,"5min":0.621,"15min":0.48399999999999999},"max":{"1min":1.4870000000000001,"5min":1.647,"15min":1.661},"last":1.0600000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.238,"15min":1.2569999999999999},"min":{"1min":0.96499999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.476,"5min":1.6240000000000001,"15min":2.9350000000000001},"last":1.429},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.246},"min":{"1min":0.94099999999999995,"5min":0.81000000000000005,"15min":0.73299999999999998},"max":{"1min":1.554,"5min":1.637,"15min":2.2389999999999999},"last":1.345}]},{"osd":5,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.272,"15min":1.302},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.63500000000000001},"max":{"1min":1.5900000000000001,"5min":1.806,"15min":3.476},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2729999999999999,"15min":1.329},"min":{"1min":0.86099999999999999,"5min":0.82099999999999995,"15min":0.64200000000000002},"max":{"1min":1.464,"5min":1.9039999999999999,"15min":3.181},"last":1.681}]},{"osd":6,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.88900000000000001,"5min":0.80400000000000005,"15min":0.68300000000000005},"max":{"1min":1.6499999999999999,"5min":1.8580000000000001,"15min":2.532},"last":1.407},{"interface":"front","average":{"1min":1.409,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":1.02,"5min":0.94299999999999995,"15min":0.872},"max":{"1min":1.573,"5min":1.8049999999999999,"15min":3.0249999999999999},"last":1.5860000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3919999999999999,"15min":1.4239999999999999},"min":{"1min":1.006,"5min":0.999,"15min":0.81499999999999995},"max":{"1min":1.504,"5min":1.851,"15min":2.1619999999999999},"last":1.272},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4259999999999999,"15min":1.472},"min":{"1min":1.048,"5min":0.88300000000000001,"15min":0.83099999999999996},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":2.4590000000000001},"last":1.474}]}]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":978944,"data_stored":462438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27100000000000002},"max":{"1min":1.0860000000000001,"5min":1.339,"15min":1.738},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.026,"15min":1.01},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.375},"max":{"1min":1.4159999999999999,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":1.7729999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96499999999999997,"15min":0.98999999999999999},"min":{"1min":0.54700000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.1779999999999999,"5min":1.45,"15min":1.7130000000000001},"last":2.5},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0720000000000001,"15min":1.0840000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.504},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.909}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97499999999999998,"15min":1.0029999999999999},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.377,"5min":1.625,"15min":1.883},"last":1.5},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.39600000000000002},"max":{"1min":1.415,"5min":1.613,"15min":2.6080000000000001},"last":2.4500000000000002}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.331,"15min":1.3180000000000001},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.75},"max":{"1min":1.7230000000000001,"5min":1.9950000000000001,"15min":2.2829999999999999},"last":1.869},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.246,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.964},"last":1.3029999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3009999999999999,"15min":1.319},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.647,"5min":1.804,"15min":2.5779999999999998},"last":2.948},{"interface":"front","average":{"1min":1.375,"5min":1.387,"15min":1.375},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.901,"5min":2.048,"15min":3.0099999999999998},"last":2.016}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.3200000000000001},"min":{"1min":0.84299999999999997,"5min":0.83399999999999996,"15min":0.77500000000000002},"max":{"1min":1.6439999999999999,"5min":1.78,"15min":2.2799999999999998},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.766,"15min":2.5089999999999999},"last":4.0529999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.456,"15min":1.448},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.79400000000000004},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.448},"last":2.1779999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7649999999999999,"5min":1.802,"15min":2.5049999999999999},"last":1.613}]}]},{"osd":1,"up_from":13,"seq":55834575033,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36680,"kb_used_data":1536,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335160,"statfs":{"total":96636764160,"available":96599203840,"internally_reserved":0,"allocated":1572864,"data_stored":1052806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85699999999999998,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.53800000000000003,"15min":0.318},"max":{"1min":1.1220000000000001,"5min":1.5049999999999999,"15min":3.056},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.999,"15min":1.012},"min":{"1min":0.75700000000000001,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":1.335,"5min":1.6359999999999999,"15min":3.4129999999999998},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.88200000000000001,"15min":0.90100000000000002},"min":{"1min":0.67100000000000004,"5min":0.497,"15min":0.36399999999999999},"max":{"1min":1.1779999999999999,"5min":1.54,"15min":3.3660000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.78100000000000003,"5min":0.47899999999999998,"15min":0.39000000000000001},"max":{"1min":1.4890000000000001,"5min":1.492,"15min":3.4980000000000002},"last":1.0600000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.21,"15min":1.224},"min":{"1min":1.0860000000000001,"5min":0.51900000000000002,"15min":0.44},"max":{"1min":1.6859999999999999,"5min":1.7290000000000001,"15min":3.464},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.077,"15min":1.0669999999999999},"min":{"1min":0.82599999999999996,"5min":0.499,"15min":0.39700000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":3.0960000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1719999999999999,"15min":1.196},"min":{"1min":1.014,"5min":0.745,"15min":0.63800000000000001},"max":{"1min":1.494,"5min":1.536,"15min":3.7810000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.403,"5min":1.3240000000000001,"15min":1.3600000000000001},"min":{"1min":1.077,"5min":0.77500000000000002,"15min":0.76600000000000001},"max":{"1min":1.585,"5min":1.8089999999999999,"15min":4.7270000000000003},"last":1.2290000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.296,"15min":1.2909999999999999},"min":{"1min":1.032,"5min":0.90600000000000003,"15min":0.61199999999999999},"max":{"1min":1.5760000000000001,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.278,"15min":1.3160000000000001},"min":{"1min":1.004,"5min":0.76900000000000002,"15min":0.69199999999999995},"max":{"1min":1.5,"5min":1.825,"15min":3.6669999999999998},"last":1.169}]},{"osd":6,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.349},"min":{"1min":1.0760000000000001,"5min":0.71099999999999997,"15min":0.68400000000000005},"max":{"1min":1.5549999999999999,"5min":1.7809999999999999,"15min":3.3300000000000001},"last":1.25},{"interface":"front","average":{"1min":1.383,"5min":1.343,"15min":1.363},"min":{"1min":1.0649999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.595,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.089}]},{"osd":7,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.409,"15min":1.4630000000000001},"min":{"1min":1.246,"5min":0.76600000000000001,"15min":0.70599999999999996},"max":{"1min":1.645,"5min":1.8939999999999999,"15min":4.6749999999999998},"last":1.288},{"interface":"front","average":{"1min":1.444,"5min":1.4179999999999999,"15min":1.4299999999999999},"min":{"1min":1.1639999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7589999999999999,"5min":1.992,"15min":3.5379999999999998},"last":1.3100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":966656,"data_stored":456163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.60699999999999998,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.3520000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.622},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.92600000000000005,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4390000000000001,"5min":1.5580000000000001,"15min":1.6910000000000001},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98499999999999999,"15min":1.006},"min":{"1min":0.67600000000000005,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.4690000000000001,"5min":2.157,"15min":2.157},"last":1.048},{"interface":"front","average":{"1min":1.113,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.84299999999999997,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.395,"5min":1.728,"15min":2.8159999999999998},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.58499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.954},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.143,"15min":1.169},"min":{"1min":0.54900000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7070000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.202}]},{"osd":4,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.3089999999999999},"min":{"1min":1.05,"5min":0.78500000000000003,"15min":0.65700000000000003},"max":{"1min":1.5289999999999999,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.22,"5min":1.214,"15min":1.2629999999999999},"min":{"1min":0.83299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.107}]},{"osd":5,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.337,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.71899999999999997},"max":{"1min":1.839,"5min":2.351,"15min":2.351},"last":1.268},{"interface":"front","average":{"1min":1.276,"5min":1.306,"15min":1.349},"min":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6299999999999999,"5min":2.2909999999999999,"15min":2.7050000000000001},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73099999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.157},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.399},"min":{"1min":1.0129999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.3120000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":1.127,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.706,"5min":1.798,"15min":2.8999999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.421,"5min":1.361,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.006}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:51:29.132 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:51:29.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.369+0000 7fed2c441640 1 Processor -- start 2023-12-01T01:51:29.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.369+0000 7fed2c441640 1 -- start start 2023-12-01T01:51:29.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed2c441640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fed2415e2f0 0x7fed240ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed2c441640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fed2406f2f0 0x7fed240caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed2c441640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed240adf10 0x7fed240c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed2c441640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed240add10 con 0x7fed2415e2f0 2023-12-01T01:51:29.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed2c441640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed240c5760 con 0x7fed2406f2f0 2023-12-01T01:51:29.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed2c441640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2415e6d0 con 0x7fed240adf10 2023-12-01T01:51:29.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed2a1dd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fed2415e2f0 0x7fed240ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed2a1dd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fed2415e2f0 0x7fed240ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39210/0 (socket says 172.21.15.115:39210) 2023-12-01T01:51:29.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed2a1dd640 1 -- 172.21.15.115:0/2421684440 learned_addr learned my addr 172.21.15.115:0/2421684440 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:29.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed299dc640 1 --2- 172.21.15.115:0/2421684440 >> v2:172.21.15.142:3300/0 conn(0x7fed2406f2f0 0x7fed240caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.371+0000 7fed2a9de640 1 --2- 172.21.15.115:0/2421684440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed240adf10 0x7fed240c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.372+0000 7fed2a1dd640 1 -- 172.21.15.115:0/2421684440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed240adf10 msgr2=0x7fed240c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:29.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.372+0000 7fed2a1dd640 1 --2- 172.21.15.115:0/2421684440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed240adf10 0x7fed240c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.372+0000 7fed2a1dd640 1 -- 172.21.15.115:0/2421684440 >> v2:172.21.15.142:3300/0 conn(0x7fed2406f2f0 msgr2=0x7fed240caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:29.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.372+0000 7fed2a1dd640 1 --2- 172.21.15.115:0/2421684440 >> v2:172.21.15.142:3300/0 conn(0x7fed2406f2f0 0x7fed240caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.372+0000 7fed2a1dd640 1 -- 172.21.15.115:0/2421684440 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed240cf7b0 con 0x7fed2415e2f0 2023-12-01T01:51:29.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.373+0000 7fed2a1dd640 1 --2- 172.21.15.115:0/2421684440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2415e2f0 0x7fed240ca660 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fed2406de80 tx=0x7fed2002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daa95fc3e11abb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:29.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.373+0000 7fed291db640 1 -- 172.21.15.115:0/2421684440 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20031750 con 0x7fed2415e2f0 2023-12-01T01:51:29.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed291db640 1 -- 172.21.15.115:0/2421684440 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed200318b0 con 0x7fed2415e2f0 2023-12-01T01:51:29.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed291db640 1 -- 172.21.15.115:0/2421684440 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20031c40 con 0x7fed2415e2f0 2023-12-01T01:51:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed2c441640 1 -- 172.21.15.115:0/2421684440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2415e2f0 msgr2=0x7fed240ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed2c441640 1 --2- 172.21.15.115:0/2421684440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2415e2f0 0x7fed240ca660 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fed2406de80 tx=0x7fed2002b7f0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed2c441640 1 -- 172.21.15.115:0/2421684440 shutdown_connections 2023-12-01T01:51:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed2c441640 1 --2- 172.21.15.115:0/2421684440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed240adf10 0x7fed240c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed2c441640 1 --2- 172.21.15.115:0/2421684440 >> v2:172.21.15.142:3300/0 conn(0x7fed2406f2f0 0x7fed240caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed2c441640 1 --2- 172.21.15.115:0/2421684440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed2415e2f0 0x7fed240ca660 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed2c441640 1 -- 172.21.15.115:0/2421684440 >> 172.21.15.115:0/2421684440 conn(0x7fed240ce880 msgr2=0x7fed240cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed2c441640 1 -- 172.21.15.115:0/2421684440 shutdown_connections 2023-12-01T01:51:29.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.374+0000 7fed2c441640 1 -- 172.21.15.115:0/2421684440 wait complete. 2023-12-01T01:51:29.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.376+0000 7fed2c441640 1 Processor -- start 2023-12-01T01:51:29.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.376+0000 7fed2c441640 1 -- start start 2023-12-01T01:51:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.377+0000 7fed2c441640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed2406f2f0 0x7fed240ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.377+0000 7fed2c441640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed240adf10 0x7fed241f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.377+0000 7fed2c441640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed2415e2f0 0x7fed240ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.377+0000 7fed2c441640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed240c32f0 con 0x7fed240adf10 2023-12-01T01:51:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.377+0000 7fed2c441640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed241f2790 con 0x7fed2406f2f0 2023-12-01T01:51:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.377+0000 7fed2c441640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed240c23b0 con 0x7fed2415e2f0 2023-12-01T01:51:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.377+0000 7fed299dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed240adf10 0x7fed241f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.377+0000 7fed299dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed240adf10 0x7fed241f2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39218/0 (socket says 172.21.15.115:39218) 2023-12-01T01:51:29.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.377+0000 7fed299dc640 1 -- 172.21.15.115:0/588588011 learned_addr learned my addr 172.21.15.115:0/588588011 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.378+0000 7fed299dc640 1 -- 172.21.15.115:0/588588011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed2415e2f0 msgr2=0x7fed240ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.378+0000 7fed299dc640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed2415e2f0 0x7fed240ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.378+0000 7fed299dc640 1 -- 172.21.15.115:0/588588011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed2406f2f0 msgr2=0x7fed240ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.378+0000 7fed299dc640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed2406f2f0 0x7fed240ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.378+0000 7fed299dc640 1 -- 172.21.15.115:0/588588011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200094d0 con 0x7fed240adf10 2023-12-01T01:51:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.378+0000 7fed299dc640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed240adf10 0x7fed241f2be0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fed1400d800 tx=0x7fed1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.378+0000 7fed1affd640 1 -- 172.21.15.115:0/588588011 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed140096a0 con 0x7fed240adf10 2023-12-01T01:51:29.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.378+0000 7fed1affd640 1 -- 172.21.15.115:0/588588011 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed14009800 con 0x7fed240adf10 2023-12-01T01:51:29.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.378+0000 7fed1affd640 1 -- 172.21.15.115:0/588588011 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14009990 con 0x7fed240adf10 2023-12-01T01:51:29.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.378+0000 7fed2c441640 1 -- 172.21.15.115:0/588588011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed240c2670 con 0x7fed240adf10 2023-12-01T01:51:29.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.379+0000 7fed2c441640 1 -- 172.21.15.115:0/588588011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed240c2c30 con 0x7fed240adf10 2023-12-01T01:51:29.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.380+0000 7fed2c441640 1 -- 172.21.15.115:0/588588011 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2406fda0 con 0x7fed240adf10 2023-12-01T01:51:29.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.381+0000 7fed1affd640 1 -- 172.21.15.115:0/588588011 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fed14009af0 con 0x7fed240adf10 2023-12-01T01:51:29.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.382+0000 7fed1affd640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed0806a550 0x7fed0806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.382+0000 7fed1affd640 1 -- 172.21.15.115:0/588588011 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed14088f10 con 0x7fed240adf10 2023-12-01T01:51:29.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.382+0000 7fed2a1dd640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed0806a550 0x7fed0806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.383+0000 7fed2a1dd640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed0806a550 0x7fed0806ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fed20035000 tx=0x7fed20030000 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:29.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.386+0000 7fed1affd640 1 -- 172.21.15.115:0/588588011 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fed140543e0 con 0x7fed240adf10 2023-12-01T01:51:29.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.573+0000 7fed2c441640 1 -- 172.21.15.115:0/588588011 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed240b43d0 con 0x7fed0806a550 2023-12-01T01:51:29.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.592+0000 7fed1affd640 1 -- 172.21.15.115:0/588588011 <== mgr.6201 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283112 (secure 0 0 0) 0x7fed20031a20 con 0x7fed0806a550 2023-12-01T01:51:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:29.599 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:51:29.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.599+0000 7fed2c441640 1 -- 172.21.15.115:0/588588011 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed0806a550 msgr2=0x7fed0806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:29.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.599+0000 7fed2c441640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed0806a550 0x7fed0806ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fed20035000 tx=0x7fed20030000 comp rx=0 tx=0).stop 2023-12-01T01:51:29.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.599+0000 7fed2c441640 1 -- 172.21.15.115:0/588588011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed240adf10 msgr2=0x7fed241f2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:29.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.599+0000 7fed2c441640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed240adf10 0x7fed241f2be0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fed1400d800 tx=0x7fed1400dcc0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.599+0000 7fed2c441640 1 -- 172.21.15.115:0/588588011 shutdown_connections 2023-12-01T01:51:29.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.599+0000 7fed2c441640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed0806a550 0x7fed0806ca10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.599+0000 7fed2c441640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed2415e2f0 0x7fed240ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.600+0000 7fed2c441640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed240adf10 0x7fed241f2be0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.600+0000 7fed2c441640 1 --2- 172.21.15.115:0/588588011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed2406f2f0 0x7fed240ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.600+0000 7fed2c441640 1 -- 172.21.15.115:0/588588011 >> 172.21.15.115:0/588588011 conn(0x7fed240ce880 msgr2=0x7fed240ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:29.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.600+0000 7fed2c441640 1 -- 172.21.15.115:0/588588011 shutdown_connections 2023-12-01T01:51:29.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.600+0000 7fed2c441640 1 -- 172.21.15.115:0/588588011 wait complete. 2023-12-01T01:51:29.605 INFO:tasks.ceph.mds.c.smithi115.stderr:2023-12-01T01:51:29.603+0000 7f0d4578f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:51:29.620 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T01:51:27.824598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265324,"kb_used_data":8748,"kb_used_omap":0,"kb_used_meta":256512,"kb_avail":754709396,"statfs":{"total":773094113280,"available":772822421504,"internally_reserved":0,"allocated":8957952,"data_stored":4848865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":262668288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.133577"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.720663+0000","last_change":"2023-12-01T01:51:23.720663+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.720663+0000","last_undegraded":"2023-12-01T01:51:23.720663+0000","last_fullsized":"2023-12-01T01:51:23.720663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:21.413565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710355+0000","last_change":"2023-12-01T01:51:23.710355+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710355+0000","last_undegraded":"2023-12-01T01:51:23.710355+0000","last_fullsized":"2023-12-01T01:51:23.710355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:26.173230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.706388+0000","last_change":"2023-12-01T01:51:23.706388+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.706388+0000","last_undegraded":"2023-12-01T01:51:23.706388+0000","last_fullsized":"2023-12-01T01:51:23.706388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:50.301817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.721934+0000","last_change":"2023-12-01T01:51:23.721934+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.721934+0000","last_undegraded":"2023-12-01T01:51:23.721934+0000","last_fullsized":"2023-12-01T01:51:23.721934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:03:53.147122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719862+0000","last_change":"2023-12-01T01:51:23.719862+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719862+0000","last_undegraded":"2023-12-01T01:51:23.719862+0000","last_fullsized":"2023-12-01T01:51:23.719862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:16.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.718565+0000","last_change":"2023-12-01T01:51:23.718565+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.718565+0000","last_undegraded":"2023-12-01T01:51:23.718565+0000","last_fullsized":"2023-12-01T01:51:23.718565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:08.979742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.717382+0000","last_change":"2023-12-01T01:51:23.717382+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.717382+0000","last_undegraded":"2023-12-01T01:51:23.717382+0000","last_fullsized":"2023-12-01T01:51:23.717382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:41.379891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.725226+0000","last_change":"2023-12-01T01:51:23.725226+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.725226+0000","last_undegraded":"2023-12-01T01:51:23.725226+0000","last_fullsized":"2023-12-01T01:51:23.725226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:56:56.216365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719271+0000","last_change":"2023-12-01T01:51:23.719271+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719271+0000","last_undegraded":"2023-12-01T01:51:23.719271+0000","last_fullsized":"2023-12-01T01:51:23.719271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:55.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719205+0000","last_change":"2023-12-01T01:51:23.719205+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719205+0000","last_undegraded":"2023-12-01T01:51:23.719205+0000","last_fullsized":"2023-12-01T01:51:23.719205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:52.689385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.702860+0000","last_change":"2023-12-01T01:51:23.702860+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.702860+0000","last_undegraded":"2023-12-01T01:51:23.702860+0000","last_fullsized":"2023-12-01T01:51:23.702860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:21.413565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.720658+0000","last_change":"2023-12-01T01:51:23.720658+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.720658+0000","last_undegraded":"2023-12-01T01:51:23.720658+0000","last_fullsized":"2023-12-01T01:51:23.720658+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:32.142316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.721909+0000","last_change":"2023-12-01T01:51:23.721909+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.721909+0000","last_undegraded":"2023-12-01T01:51:23.721909+0000","last_fullsized":"2023-12-01T01:51:23.721909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:01.177699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.702061+0000","last_change":"2023-12-01T01:51:23.702061+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.702061+0000","last_undegraded":"2023-12-01T01:51:23.702061+0000","last_fullsized":"2023-12-01T01:51:23.702061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:29.110060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.718688+0000","last_change":"2023-12-01T01:51:23.718688+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.718688+0000","last_undegraded":"2023-12-01T01:51:23.718688+0000","last_fullsized":"2023-12-01T01:51:23.718688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719266+0000","last_change":"2023-12-01T01:51:23.719266+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719266+0000","last_undegraded":"2023-12-01T01:51:23.719266+0000","last_fullsized":"2023-12-01T01:51:23.719266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:28.427573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.717782+0000","last_change":"2023-12-01T01:51:23.717782+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.717782+0000","last_undegraded":"2023-12-01T01:51:23.717782+0000","last_fullsized":"2023-12-01T01:51:23.717782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710478+0000","last_change":"2023-12-01T01:51:23.710478+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710478+0000","last_undegraded":"2023-12-01T01:51:23.710478+0000","last_fullsized":"2023-12-01T01:51:23.710478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:27.535710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.718246+0000","last_change":"2023-12-01T01:51:23.718246+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.718246+0000","last_undegraded":"2023-12-01T01:51:23.718246+0000","last_fullsized":"2023-12-01T01:51:23.718246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:14.157429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.701921+0000","last_change":"2023-12-01T01:51:23.701921+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.701921+0000","last_undegraded":"2023-12-01T01:51:23.701921+0000","last_fullsized":"2023-12-01T01:51:23.701921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:08.979742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719563+0000","last_change":"2023-12-01T01:51:23.719563+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719563+0000","last_undegraded":"2023-12-01T01:51:23.719563+0000","last_fullsized":"2023-12-01T01:51:23.719563+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.702566+0000","last_change":"2023-12-01T01:51:23.702566+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.702566+0000","last_undegraded":"2023-12-01T01:51:23.702566+0000","last_fullsized":"2023-12-01T01:51:23.702566+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:12.784193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.715707+0000","last_change":"2023-12-01T01:51:23.715707+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.715707+0000","last_undegraded":"2023-12-01T01:51:23.715707+0000","last_fullsized":"2023-12-01T01:51:23.715707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:39:57.857405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719436+0000","last_change":"2023-12-01T01:51:23.719436+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719436+0000","last_undegraded":"2023-12-01T01:51:23.719436+0000","last_fullsized":"2023-12-01T01:51:23.719436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:51.772038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710118+0000","last_change":"2023-12-01T01:51:23.710118+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710118+0000","last_undegraded":"2023-12-01T01:51:23.710118+0000","last_fullsized":"2023-12-01T01:51:23.710118+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:06:06.433309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.718549+0000","last_change":"2023-12-01T01:51:23.718549+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.718549+0000","last_undegraded":"2023-12-01T01:51:23.718549+0000","last_fullsized":"2023-12-01T01:51:23.718549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:10.334292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.714681+0000","last_change":"2023-12-01T01:51:23.714681+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.714681+0000","last_undegraded":"2023-12-01T01:51:23.714681+0000","last_fullsized":"2023-12-01T01:51:23.714681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:10:04.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.727172+0000","last_change":"2023-12-01T01:51:23.727172+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.727172+0000","last_undegraded":"2023-12-01T01:51:23.727172+0000","last_fullsized":"2023-12-01T01:51:23.727172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:46.030193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.720918+0000","last_change":"2023-12-01T01:51:23.720918+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.720918+0000","last_undegraded":"2023-12-01T01:51:23.720918+0000","last_fullsized":"2023-12-01T01:51:23.720918+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:49.877816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.714310+0000","last_change":"2023-12-01T01:51:23.714310+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.714310+0000","last_undegraded":"2023-12-01T01:51:23.714310+0000","last_fullsized":"2023-12-01T01:51:23.714310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:28.026095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710305+0000","last_change":"2023-12-01T01:51:23.710305+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710305+0000","last_undegraded":"2023-12-01T01:51:23.710305+0000","last_fullsized":"2023-12-01T01:51:23.710305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:52.879295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.701734+0000","last_change":"2023-12-01T01:51:23.701734+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.701734+0000","last_undegraded":"2023-12-01T01:51:23.701734+0000","last_fullsized":"2023-12-01T01:51:23.701734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:16.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.713643+0000","last_change":"2023-12-01T01:51:23.713643+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.713643+0000","last_undegraded":"2023-12-01T01:51:23.713643+0000","last_fullsized":"2023-12-01T01:51:23.713643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:01:44.563857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.728135+0000","last_change":"2023-12-01T01:51:23.722626+0000","last_active":"2023-12-01T01:51:23.728135+0000","last_peered":"2023-12-01T01:51:23.728135+0000","last_clean":"2023-12-01T01:51:23.728135+0000","last_became_active":"2023-12-01T01:51:23.722272+0000","last_became_peered":"2023-12-01T01:51:23.722272+0000","last_unstale":"2023-12-01T01:51:23.728135+0000","last_undegraded":"2023-12-01T01:51:23.728135+0000","last_fullsized":"2023-12-01T01:51:23.728135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:49.297425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.715745+0000","last_change":"2023-12-01T01:51:23.715745+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.715745+0000","last_undegraded":"2023-12-01T01:51:23.715745+0000","last_fullsized":"2023-12-01T01:51:23.715745+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:12.784193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.734229+0000","last_change":"2023-12-01T01:51:23.730769+0000","last_active":"2023-12-01T01:51:23.734229+0000","last_peered":"2023-12-01T01:51:23.734229+0000","last_clean":"2023-12-01T01:51:23.734229+0000","last_became_active":"2023-12-01T01:51:23.730468+0000","last_became_peered":"2023-12-01T01:51:23.730468+0000","last_unstale":"2023-12-01T01:51:23.734229+0000","last_undegraded":"2023-12-01T01:51:23.734229+0000","last_fullsized":"2023-12-01T01:51:23.734229+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:56.584539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.726700+0000","last_change":"2023-12-01T01:51:23.726700+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.726700+0000","last_undegraded":"2023-12-01T01:51:23.726700+0000","last_fullsized":"2023-12-01T01:51:23.726700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:53:01.496510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.734707+0000","last_change":"2023-12-01T01:51:23.729205+0000","last_active":"2023-12-01T01:51:23.734707+0000","last_peered":"2023-12-01T01:51:23.734707+0000","last_clean":"2023-12-01T01:51:23.734707+0000","last_became_active":"2023-12-01T01:51:23.728920+0000","last_became_peered":"2023-12-01T01:51:23.728920+0000","last_unstale":"2023-12-01T01:51:23.734707+0000","last_undegraded":"2023-12-01T01:51:23.734707+0000","last_fullsized":"2023-12-01T01:51:23.734707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:21:58.877785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.721738+0000","last_change":"2023-12-01T01:51:23.721738+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.721738+0000","last_undegraded":"2023-12-01T01:51:23.721738+0000","last_fullsized":"2023-12-01T01:51:23.721738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:01.177699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.732154+0000","last_change":"2023-12-01T01:51:23.729190+0000","last_active":"2023-12-01T01:51:23.732154+0000","last_peered":"2023-12-01T01:51:23.732154+0000","last_clean":"2023-12-01T01:51:23.732154+0000","last_became_active":"2023-12-01T01:51:23.728867+0000","last_became_peered":"2023-12-01T01:51:23.728867+0000","last_unstale":"2023-12-01T01:51:23.732154+0000","last_undegraded":"2023-12-01T01:51:23.732154+0000","last_fullsized":"2023-12-01T01:51:23.732154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:43:10.966577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.718697+0000","last_change":"2023-12-01T01:51:23.718697+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.718697+0000","last_undegraded":"2023-12-01T01:51:23.718697+0000","last_fullsized":"2023-12-01T01:51:23.718697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:01:53.075668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.721289+0000","last_change":"2023-12-01T01:51:23.717746+0000","last_active":"2023-12-01T01:51:23.721289+0000","last_peered":"2023-12-01T01:51:23.721289+0000","last_clean":"2023-12-01T01:51:23.721289+0000","last_became_active":"2023-12-01T01:51:23.715918+0000","last_became_peered":"2023-12-01T01:51:23.715918+0000","last_unstale":"2023-12-01T01:51:23.721289+0000","last_undegraded":"2023-12-01T01:51:23.721289+0000","last_fullsized":"2023-12-01T01:51:23.721289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:35:50.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.701843+0000","last_change":"2023-12-01T01:51:23.701843+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.701843+0000","last_undegraded":"2023-12-01T01:51:23.701843+0000","last_fullsized":"2023-12-01T01:51:23.701843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:06.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.730693+0000","last_change":"2023-12-01T01:51:23.725607+0000","last_active":"2023-12-01T01:51:23.730693+0000","last_peered":"2023-12-01T01:51:23.730693+0000","last_clean":"2023-12-01T01:51:23.730693+0000","last_became_active":"2023-12-01T01:51:23.722833+0000","last_became_peered":"2023-12-01T01:51:23.722833+0000","last_unstale":"2023-12-01T01:51:23.730693+0000","last_undegraded":"2023-12-01T01:51:23.730693+0000","last_fullsized":"2023-12-01T01:51:23.730693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:10:03.788374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.701782+0000","last_change":"2023-12-01T01:51:23.701782+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.701782+0000","last_undegraded":"2023-12-01T01:51:23.701782+0000","last_fullsized":"2023-12-01T01:51:23.701782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:51.144418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.732197+0000","last_change":"2023-12-01T01:51:23.729822+0000","last_active":"2023-12-01T01:51:23.732197+0000","last_peered":"2023-12-01T01:51:23.732197+0000","last_clean":"2023-12-01T01:51:23.732197+0000","last_became_active":"2023-12-01T01:51:23.728147+0000","last_became_peered":"2023-12-01T01:51:23.728147+0000","last_unstale":"2023-12-01T01:51:23.732197+0000","last_undegraded":"2023-12-01T01:51:23.732197+0000","last_fullsized":"2023-12-01T01:51:23.732197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:43:10.667895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719104+0000","last_change":"2023-12-01T01:51:23.719104+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719104+0000","last_undegraded":"2023-12-01T01:51:23.719104+0000","last_fullsized":"2023-12-01T01:51:23.719104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.713786+0000","last_change":"2023-12-01T01:51:23.710030+0000","last_active":"2023-12-01T01:51:23.713786+0000","last_peered":"2023-12-01T01:51:23.713786+0000","last_clean":"2023-12-01T01:51:23.713786+0000","last_became_active":"2023-12-01T01:51:23.709381+0000","last_became_peered":"2023-12-01T01:51:23.709381+0000","last_unstale":"2023-12-01T01:51:23.713786+0000","last_undegraded":"2023-12-01T01:51:23.713786+0000","last_fullsized":"2023-12-01T01:51:23.713786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:25.168837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.712668+0000","last_change":"2023-12-01T01:51:23.712668+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.712668+0000","last_undegraded":"2023-12-01T01:51:23.712668+0000","last_fullsized":"2023-12-01T01:51:23.712668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:50.301817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.729093+0000","last_change":"2023-12-01T01:51:23.725370+0000","last_active":"2023-12-01T01:51:23.729093+0000","last_peered":"2023-12-01T01:51:23.729093+0000","last_clean":"2023-12-01T01:51:23.729093+0000","last_became_active":"2023-12-01T01:51:23.725113+0000","last_became_peered":"2023-12-01T01:51:23.725113+0000","last_unstale":"2023-12-01T01:51:23.729093+0000","last_undegraded":"2023-12-01T01:51:23.729093+0000","last_fullsized":"2023-12-01T01:51:23.729093+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:17.602658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.714798+0000","last_change":"2023-12-01T01:51:23.714798+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.714798+0000","last_undegraded":"2023-12-01T01:51:23.714798+0000","last_fullsized":"2023-12-01T01:51:23.714798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736347+0000","last_change":"2023-12-01T01:51:23.733129+0000","last_active":"2023-12-01T01:51:23.736347+0000","last_peered":"2023-12-01T01:51:23.736347+0000","last_clean":"2023-12-01T01:51:23.736347+0000","last_became_active":"2023-12-01T01:51:23.732687+0000","last_became_peered":"2023-12-01T01:51:23.732687+0000","last_unstale":"2023-12-01T01:51:23.736347+0000","last_undegraded":"2023-12-01T01:51:23.736347+0000","last_fullsized":"2023-12-01T01:51:23.736347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:59.339210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.722932+0000","last_change":"2023-12-01T01:51:23.722932+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.722932+0000","last_undegraded":"2023-12-01T01:51:23.722932+0000","last_fullsized":"2023-12-01T01:51:23.722932+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:54:45.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.727027+0000","last_change":"2023-12-01T01:51:23.723356+0000","last_active":"2023-12-01T01:51:23.727027+0000","last_peered":"2023-12-01T01:51:23.727027+0000","last_clean":"2023-12-01T01:51:23.727027+0000","last_became_active":"2023-12-01T01:51:23.723040+0000","last_became_peered":"2023-12-01T01:51:23.723040+0000","last_unstale":"2023-12-01T01:51:23.727027+0000","last_undegraded":"2023-12-01T01:51:23.727027+0000","last_fullsized":"2023-12-01T01:51:23.727027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:15.500740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.717320+0000","last_change":"2023-12-01T01:51:23.717320+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.717320+0000","last_undegraded":"2023-12-01T01:51:23.717320+0000","last_fullsized":"2023-12-01T01:51:23.717320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:23.126569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.727807+0000","last_change":"2023-12-01T01:51:23.725813+0000","last_active":"2023-12-01T01:51:23.727807+0000","last_peered":"2023-12-01T01:51:23.727807+0000","last_clean":"2023-12-01T01:51:23.727807+0000","last_became_active":"2023-12-01T01:51:23.723452+0000","last_became_peered":"2023-12-01T01:51:23.723452+0000","last_unstale":"2023-12-01T01:51:23.727807+0000","last_undegraded":"2023-12-01T01:51:23.727807+0000","last_fullsized":"2023-12-01T01:51:23.727807+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:46.793866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.720422+0000","last_change":"2023-12-01T01:51:23.720422+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.720422+0000","last_undegraded":"2023-12-01T01:51:23.720422+0000","last_fullsized":"2023-12-01T01:51:23.720422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.469019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736241+0000","last_change":"2023-12-01T01:51:23.732400+0000","last_active":"2023-12-01T01:51:23.736241+0000","last_peered":"2023-12-01T01:51:23.736241+0000","last_clean":"2023-12-01T01:51:23.736241+0000","last_became_active":"2023-12-01T01:51:23.730066+0000","last_became_peered":"2023-12-01T01:51:23.730066+0000","last_unstale":"2023-12-01T01:51:23.736241+0000","last_undegraded":"2023-12-01T01:51:23.736241+0000","last_fullsized":"2023-12-01T01:51:23.736241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:49.297425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.703529+0000","last_change":"2023-12-01T01:51:23.703529+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.703529+0000","last_undegraded":"2023-12-01T01:51:23.703529+0000","last_fullsized":"2023-12-01T01:51:23.703529+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:10.334292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.727316+0000","last_change":"2023-12-01T01:51:23.722740+0000","last_active":"2023-12-01T01:51:23.727316+0000","last_peered":"2023-12-01T01:51:23.727316+0000","last_clean":"2023-12-01T01:51:23.727316+0000","last_became_active":"2023-12-01T01:51:23.722478+0000","last_became_peered":"2023-12-01T01:51:23.722478+0000","last_unstale":"2023-12-01T01:51:23.727316+0000","last_undegraded":"2023-12-01T01:51:23.727316+0000","last_fullsized":"2023-12-01T01:51:23.727316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:01:52.071276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.716303+0000","last_change":"2023-12-01T01:51:23.716303+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.716303+0000","last_undegraded":"2023-12-01T01:51:23.716303+0000","last_fullsized":"2023-12-01T01:51:23.716303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:33.729872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.726783+0000","last_change":"2023-12-01T01:51:23.725460+0000","last_active":"2023-12-01T01:51:23.726783+0000","last_peered":"2023-12-01T01:51:23.726783+0000","last_clean":"2023-12-01T01:51:23.726783+0000","last_became_active":"2023-12-01T01:51:23.725192+0000","last_became_peered":"2023-12-01T01:51:23.725192+0000","last_unstale":"2023-12-01T01:51:23.726783+0000","last_undegraded":"2023-12-01T01:51:23.726783+0000","last_fullsized":"2023-12-01T01:51:23.726783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:14.033061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.715228+0000","last_change":"2023-12-01T01:51:23.715228+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.715228+0000","last_undegraded":"2023-12-01T01:51:23.715228+0000","last_fullsized":"2023-12-01T01:51:23.715228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:49.877816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.730771+0000","last_change":"2023-12-01T01:51:23.728019+0000","last_active":"2023-12-01T01:51:23.730771+0000","last_peered":"2023-12-01T01:51:23.730771+0000","last_clean":"2023-12-01T01:51:23.730771+0000","last_became_active":"2023-12-01T01:51:23.727690+0000","last_became_peered":"2023-12-01T01:51:23.727690+0000","last_unstale":"2023-12-01T01:51:23.730771+0000","last_undegraded":"2023-12-01T01:51:23.730771+0000","last_fullsized":"2023-12-01T01:51:23.730771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:54:44.691969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710032+0000","last_change":"2023-12-01T01:51:23.710032+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710032+0000","last_undegraded":"2023-12-01T01:51:23.710032+0000","last_fullsized":"2023-12-01T01:51:23.710032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:14.157429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736371+0000","last_change":"2023-12-01T01:51:23.733218+0000","last_active":"2023-12-01T01:51:23.736371+0000","last_peered":"2023-12-01T01:51:23.736371+0000","last_clean":"2023-12-01T01:51:23.736371+0000","last_became_active":"2023-12-01T01:51:23.732910+0000","last_became_peered":"2023-12-01T01:51:23.732910+0000","last_unstale":"2023-12-01T01:51:23.736371+0000","last_undegraded":"2023-12-01T01:51:23.736371+0000","last_fullsized":"2023-12-01T01:51:23.736371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:54.751334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.712022+0000","last_change":"2023-12-01T01:51:23.712022+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.712022+0000","last_undegraded":"2023-12-01T01:51:23.712022+0000","last_fullsized":"2023-12-01T01:51:23.712022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.738667+0000","last_change":"2023-12-01T01:51:23.733533+0000","last_active":"2023-12-01T01:51:23.738667+0000","last_peered":"2023-12-01T01:51:23.738667+0000","last_clean":"2023-12-01T01:51:23.738667+0000","last_became_active":"2023-12-01T01:51:23.733196+0000","last_became_peered":"2023-12-01T01:51:23.733196+0000","last_unstale":"2023-12-01T01:51:23.738667+0000","last_undegraded":"2023-12-01T01:51:23.738667+0000","last_fullsized":"2023-12-01T01:51:23.738667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:09.329899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.710494+0000","last_change":"2023-12-01T01:51:23.710494+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.710494+0000","last_undegraded":"2023-12-01T01:51:23.710494+0000","last_fullsized":"2023-12-01T01:51:23.710494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:12:27.401881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.731329+0000","last_change":"2023-12-01T01:51:23.728528+0000","last_active":"2023-12-01T01:51:23.731329+0000","last_peered":"2023-12-01T01:51:23.731329+0000","last_clean":"2023-12-01T01:51:23.731329+0000","last_became_active":"2023-12-01T01:51:23.728266+0000","last_became_peered":"2023-12-01T01:51:23.728266+0000","last_unstale":"2023-12-01T01:51:23.731329+0000","last_undegraded":"2023-12-01T01:51:23.731329+0000","last_fullsized":"2023-12-01T01:51:23.731329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:18:10.249852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.716734+0000","last_change":"2023-12-01T01:51:23.716734+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.716734+0000","last_undegraded":"2023-12-01T01:51:23.716734+0000","last_fullsized":"2023-12-01T01:51:23.716734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:03.279418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736360+0000","last_change":"2023-12-01T01:51:23.733364+0000","last_active":"2023-12-01T01:51:23.736360+0000","last_peered":"2023-12-01T01:51:23.736360+0000","last_clean":"2023-12-01T01:51:23.736360+0000","last_became_active":"2023-12-01T01:51:23.732811+0000","last_became_peered":"2023-12-01T01:51:23.732811+0000","last_unstale":"2023-12-01T01:51:23.736360+0000","last_undegraded":"2023-12-01T01:51:23.736360+0000","last_fullsized":"2023-12-01T01:51:23.736360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:17.784149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.719794+0000","last_change":"2023-12-01T01:51:23.719794+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.719794+0000","last_undegraded":"2023-12-01T01:51:23.719794+0000","last_fullsized":"2023-12-01T01:51:23.719794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:57.588932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.721237+0000","last_change":"2023-12-01T01:51:23.717087+0000","last_active":"2023-12-01T01:51:23.721237+0000","last_peered":"2023-12-01T01:51:23.721237+0000","last_clean":"2023-12-01T01:51:23.721237+0000","last_became_active":"2023-12-01T01:51:23.712593+0000","last_became_peered":"2023-12-01T01:51:23.712593+0000","last_unstale":"2023-12-01T01:51:23.721237+0000","last_undegraded":"2023-12-01T01:51:23.721237+0000","last_fullsized":"2023-12-01T01:51:23.721237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:09.329899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.720276+0000","last_change":"2023-12-01T01:51:23.720276+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.720276+0000","last_undegraded":"2023-12-01T01:51:23.720276+0000","last_fullsized":"2023-12-01T01:51:23.720276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:09.806552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736322+0000","last_change":"2023-12-01T01:51:23.732919+0000","last_active":"2023-12-01T01:51:23.736322+0000","last_peered":"2023-12-01T01:51:23.736322+0000","last_clean":"2023-12-01T01:51:23.736322+0000","last_became_active":"2023-12-01T01:51:23.732477+0000","last_became_peered":"2023-12-01T01:51:23.732477+0000","last_unstale":"2023-12-01T01:51:23.736322+0000","last_undegraded":"2023-12-01T01:51:23.736322+0000","last_fullsized":"2023-12-01T01:51:23.736322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:28.105668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.724998+0000","last_change":"2023-12-01T01:51:23.724998+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.724998+0000","last_undegraded":"2023-12-01T01:51:23.724998+0000","last_fullsized":"2023-12-01T01:51:23.724998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:55.560831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.733228+0000","last_change":"2023-12-01T01:51:23.732268+0000","last_active":"2023-12-01T01:51:23.733228+0000","last_peered":"2023-12-01T01:51:23.733228+0000","last_clean":"2023-12-01T01:51:23.733228+0000","last_became_active":"2023-12-01T01:51:23.730931+0000","last_became_peered":"2023-12-01T01:51:23.730931+0000","last_unstale":"2023-12-01T01:51:23.733228+0000","last_undegraded":"2023-12-01T01:51:23.733228+0000","last_fullsized":"2023-12-01T01:51:23.733228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:43.652247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.709575+0000","last_change":"2023-12-01T01:51:23.709575+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.709575+0000","last_undegraded":"2023-12-01T01:51:23.709575+0000","last_fullsized":"2023-12-01T01:51:23.709575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:55:18.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736391+0000","last_change":"2023-12-01T01:51:23.735050+0000","last_active":"2023-12-01T01:51:23.736391+0000","last_peered":"2023-12-01T01:51:23.736391+0000","last_clean":"2023-12-01T01:51:23.736391+0000","last_became_active":"2023-12-01T01:51:23.734653+0000","last_became_peered":"2023-12-01T01:51:23.734653+0000","last_unstale":"2023-12-01T01:51:23.736391+0000","last_undegraded":"2023-12-01T01:51:23.736391+0000","last_fullsized":"2023-12-01T01:51:23.736391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:06.069833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.711558+0000","last_change":"2023-12-01T01:51:23.711558+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.711558+0000","last_undegraded":"2023-12-01T01:51:23.711558+0000","last_fullsized":"2023-12-01T01:51:23.711558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:34:25.329128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.730297+0000","last_change":"2023-12-01T01:51:23.724706+0000","last_active":"2023-12-01T01:51:23.730297+0000","last_peered":"2023-12-01T01:51:23.730297+0000","last_clean":"2023-12-01T01:51:23.730297+0000","last_became_active":"2023-12-01T01:51:23.723272+0000","last_became_peered":"2023-12-01T01:51:23.723272+0000","last_unstale":"2023-12-01T01:51:23.730297+0000","last_undegraded":"2023-12-01T01:51:23.730297+0000","last_fullsized":"2023-12-01T01:51:23.730297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:51.874903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.709087+0000","last_change":"2023-12-01T01:51:23.709087+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.709087+0000","last_undegraded":"2023-12-01T01:51:23.709087+0000","last_fullsized":"2023-12-01T01:51:23.709087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:47.798259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.728265+0000","last_change":"2023-12-01T01:51:23.721602+0000","last_active":"2023-12-01T01:51:23.728265+0000","last_peered":"2023-12-01T01:51:23.728265+0000","last_clean":"2023-12-01T01:51:23.728265+0000","last_became_active":"2023-12-01T01:51:23.721352+0000","last_became_peered":"2023-12-01T01:51:23.721352+0000","last_unstale":"2023-12-01T01:51:23.728265+0000","last_undegraded":"2023-12-01T01:51:23.728265+0000","last_fullsized":"2023-12-01T01:51:23.728265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:44.064139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.726907+0000","last_change":"2023-12-01T01:51:23.726907+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.726907+0000","last_undegraded":"2023-12-01T01:51:23.726907+0000","last_fullsized":"2023-12-01T01:51:23.726907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:13.923540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.733406+0000","last_change":"2023-12-01T01:51:23.732583+0000","last_active":"2023-12-01T01:51:23.733406+0000","last_peered":"2023-12-01T01:51:23.733406+0000","last_clean":"2023-12-01T01:51:23.733406+0000","last_became_active":"2023-12-01T01:51:23.731083+0000","last_became_peered":"2023-12-01T01:51:23.731083+0000","last_unstale":"2023-12-01T01:51:23.733406+0000","last_undegraded":"2023-12-01T01:51:23.733406+0000","last_fullsized":"2023-12-01T01:51:23.733406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:36.702370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.713608+0000","last_change":"2023-12-01T01:51:23.713608+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.713608+0000","last_undegraded":"2023-12-01T01:51:23.713608+0000","last_fullsized":"2023-12-01T01:51:23.713608+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:06.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.735924+0000","last_change":"2023-12-01T01:51:23.732282+0000","last_active":"2023-12-01T01:51:23.735924+0000","last_peered":"2023-12-01T01:51:23.735924+0000","last_clean":"2023-12-01T01:51:23.735924+0000","last_became_active":"2023-12-01T01:51:23.732016+0000","last_became_peered":"2023-12-01T01:51:23.732016+0000","last_unstale":"2023-12-01T01:51:23.735924+0000","last_undegraded":"2023-12-01T01:51:23.735924+0000","last_fullsized":"2023-12-01T01:51:23.735924+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:50.767646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.712390+0000","last_change":"2023-12-01T01:51:23.712390+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.712390+0000","last_undegraded":"2023-12-01T01:51:23.712390+0000","last_fullsized":"2023-12-01T01:51:23.712390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:29.110060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.724154+0000","last_change":"2023-12-01T01:51:23.720558+0000","last_active":"2023-12-01T01:51:23.724154+0000","last_peered":"2023-12-01T01:51:23.724154+0000","last_clean":"2023-12-01T01:51:23.724154+0000","last_became_active":"2023-12-01T01:51:23.720288+0000","last_became_peered":"2023-12-01T01:51:23.720288+0000","last_unstale":"2023-12-01T01:51:23.724154+0000","last_undegraded":"2023-12-01T01:51:23.724154+0000","last_fullsized":"2023-12-01T01:51:23.724154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:54:32.273772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.717335+0000","last_change":"2023-12-01T01:51:23.717335+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.717335+0000","last_undegraded":"2023-12-01T01:51:23.717335+0000","last_fullsized":"2023-12-01T01:51:23.717335+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:15.037453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.734858+0000","last_change":"2023-12-01T01:51:23.730111+0000","last_active":"2023-12-01T01:51:23.734858+0000","last_peered":"2023-12-01T01:51:23.734858+0000","last_clean":"2023-12-01T01:51:23.734858+0000","last_became_active":"2023-12-01T01:51:23.729813+0000","last_became_peered":"2023-12-01T01:51:23.729813+0000","last_unstale":"2023-12-01T01:51:23.734858+0000","last_undegraded":"2023-12-01T01:51:23.734858+0000","last_fullsized":"2023-12-01T01:51:23.734858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:06.524585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.708922+0000","last_change":"2023-12-01T01:51:23.708922+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.708922+0000","last_undegraded":"2023-12-01T01:51:23.708922+0000","last_fullsized":"2023-12-01T01:51:23.708922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.360859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.736201+0000","last_change":"2023-12-01T01:51:23.731063+0000","last_active":"2023-12-01T01:51:23.736201+0000","last_peered":"2023-12-01T01:51:23.736201+0000","last_clean":"2023-12-01T01:51:23.736201+0000","last_became_active":"2023-12-01T01:51:23.730608+0000","last_became_peered":"2023-12-01T01:51:23.730608+0000","last_unstale":"2023-12-01T01:51:23.736201+0000","last_undegraded":"2023-12-01T01:51:23.736201+0000","last_fullsized":"2023-12-01T01:51:23.736201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:34:24.324736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":81,"state":"creating+peering","last_fresh":"2023-12-01T01:51:23.715721+0000","last_change":"2023-12-01T01:51:23.715721+0000","last_active":"2023-12-01T01:51:23.688067+0000","last_peered":"2023-12-01T01:51:23.688067+0000","last_clean":"2023-12-01T01:51:23.688067+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:51:23.715721+0000","last_undegraded":"2023-12-01T01:51:23.715721+0000","last_fullsized":"2023-12-01T01:51:23.715721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:31:57.782696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:51:23.713619+0000","last_change":"2023-12-01T01:51:23.709986+0000","last_active":"2023-12-01T01:51:23.713619+0000","last_peered":"2023-12-01T01:51:23.713619+0000","last_clean":"2023-12-01T01:51:23.713619+0000","last_became_active":"2023-12-01T01:51:23.709276+0000","last_became_peered":"2023-12-01T01:51:23.709276+0000","last_unstale":"2023-12-01T01:51:23.713619+0000","last_undegraded":"2023-12-01T01:51:23.713619+0000","last_fullsized":"2023-12-01T01:51:23.713619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:06:05.428917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'161","reported_seq":380,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-12-01T01:51:22.703508+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:51:22.703508+0000","last_peered":"2023-12-01T01:51:22.703508+0000","last_clean":"2023-12-01T01:51:22.703508+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:51:22.703508+0000","last_undegraded":"2023-12-01T01:51:22.703508+0000","last_fullsized":"2023-12-01T01:51:22.703508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":966656,"data_stored":456163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.79700000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3979999999999999,"5min":1.5549999999999999,"15min":1.8089999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.073,"15min":1.05},"min":{"1min":0.82899999999999996,"5min":0.64700000000000002,"15min":0.64000000000000001},"max":{"1min":1.353,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.2669999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.246,"15min":1.218},"min":{"1min":1.008,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.802,"5min":1.8100000000000001,"15min":1.8420000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2629999999999999,"15min":1.2569999999999999},"min":{"1min":1.0669999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5529999999999999,"5min":1.744,"15min":2.0870000000000002},"last":1.5049999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.121,"15min":1.123},"min":{"1min":0.80100000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5329999999999999,"5min":1.728,"15min":1.728},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.218,"15min":1.2290000000000001},"min":{"1min":0.92400000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.6120000000000001,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":0.95799999999999996}]},{"osd":3,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.876,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.617,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.355,"15min":1.349},"min":{"1min":1.026,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6859999999999999,"5min":2.1880000000000002,"15min":2.7050000000000001},"last":1.4199999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.2769999999999999,"15min":1.296},"min":{"1min":0.83099999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.681,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4610000000000001,"15min":1.4199999999999999},"min":{"1min":1.0609999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7370000000000001,"5min":2.0939999999999999,"15min":2.601},"last":1.534}]},{"osd":5,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.274,"15min":1.27},"min":{"1min":0.83099999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.6439999999999999,"5min":2.0870000000000002,"15min":2.4500000000000002},"last":1.579},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5309999999999999,"15min":1.526},"min":{"1min":1,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8839999999999999,"5min":2.2930000000000001,"15min":2.8069999999999999},"last":1.6160000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.393,"15min":1.3859999999999999},"min":{"1min":1.004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.77,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.645},{"interface":"front","average":{"1min":1.403,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":1.002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7929999999999999,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.4690000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":978944,"data_stored":462438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.101,"15min":1.1060000000000001},"min":{"1min":0.78900000000000003,"5min":0.69699999999999995,"15min":0.66900000000000004},"max":{"1min":1.619,"5min":1.635,"15min":1.79},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.137,"15min":1.1299999999999999},"min":{"1min":0.89600000000000002,"5min":0.64400000000000002,"15min":0.61799999999999999},"max":{"1min":1.4690000000000001,"5min":2.5569999999999999,"15min":2.73},"last":1.254}]},{"osd":1,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.167,"15min":1.153},"min":{"1min":0.94999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6759999999999999,"5min":1.8620000000000001,"15min":2.2170000000000001},"last":1.411},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.196,"15min":1.21},"min":{"1min":0.93400000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6259999999999999,"5min":1.651,"15min":2.0920000000000001},"last":1.111}]},{"osd":2,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.232,"15min":1.163},"min":{"1min":0.92000000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6359999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.27,"15min":1.25},"min":{"1min":0.877,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7549999999999999,"5min":2.137,"15min":2.137},"last":1.2869999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3009999999999999,"15min":1.2849999999999999},"min":{"1min":1.111,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.548,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.4259999999999999,"15min":1.407},"min":{"1min":1.137,"5min":1.085,"15min":0.81299999999999994},"max":{"1min":1.7130000000000001,"5min":1.758,"15min":2.4020000000000001},"last":1.3180000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.321,"15min":1.2909999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.70699999999999996},"max":{"1min":1.6919999999999999,"5min":2.226,"15min":2.3399999999999999},"last":1.169},{"interface":"front","average":{"1min":1.331,"5min":1.365,"15min":1.3580000000000001},"min":{"1min":0.94699999999999995,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.694,"5min":1.931,"15min":2.6760000000000002},"last":1.208}]},{"osd":5,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4419999999999999,"15min":1.4179999999999999},"min":{"1min":1.0469999999999999,"5min":1.0229999999999999,"15min":0.82699999999999996},"max":{"1min":1.726,"5min":2.1899999999999999,"15min":2.5539999999999998},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5109999999999999,"15min":1.482},"min":{"1min":1.1870000000000001,"5min":0.98899999999999999,"15min":0.748},"max":{"1min":1.8129999999999999,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.3740000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5660000000000001,"15min":1.5249999999999999},"min":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":0.89500000000000002},"max":{"1min":1.978,"5min":2.762,"15min":2.762},"last":1.518},{"interface":"front","average":{"1min":1.587,"5min":1.621,"15min":1.585},"min":{"1min":1.218,"5min":1.1479999999999999,"15min":0.78700000000000003},"max":{"1min":1.9259999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.4570000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575035,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1524,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1560576,"data_stored":1046531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.1060000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.498,"5min":1.651,"15min":3.3980000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.127,"15min":1.141},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.60099999999999998},"max":{"1min":1.5720000000000001,"5min":1.6479999999999999,"15min":2.153},"last":0.85199999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.153,"15min":1.137},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.7010000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.73599999999999999,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4039999999999999,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.048},{"interface":"front","average":{"1min":1.155,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.6080000000000001,"5min":1.621,"15min":2.0630000000000002},"last":1.0940000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.286},"min":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.76900000000000002},"max":{"1min":1.5309999999999999,"5min":1.694,"15min":1.833},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3120000000000001,"15min":1.296},"min":{"1min":0.98099999999999998,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":1.6779999999999999,"5min":1.7110000000000001,"15min":2.1440000000000001},"last":1.367}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.238,"15min":1.2390000000000001},"min":{"1min":0.86399999999999999,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.6779999999999999,"5min":1.853,"15min":2.5129999999999999},"last":1.423},{"interface":"front","average":{"1min":1.282,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.84999999999999998,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.6319999999999999,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.139}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2989999999999999,"15min":1.329},"min":{"1min":0.88500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":2.2970000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":1.125,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":1.962,"5min":1.962,"15min":3.23},"last":1.617}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4530000000000001,"15min":1.48},"min":{"1min":0.92400000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2,"5min":2,"15min":3.012},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4970000000000001,"15min":1.524},"min":{"1min":1.2689999999999999,"5min":0.873,"15min":0.873},"max":{"1min":2.073,"5min":2.073,"15min":2.8420000000000001},"last":1.5549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575035,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36088,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335752,"statfs":{"total":96636764160,"available":96599810048,"internally_reserved":0,"allocated":966656,"data_stored":456163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.119,"15min":1.1040000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.53,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.53},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.734,"5min":1.827,"15min":3.8069999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2350000000000001,"15min":1.2090000000000001},"min":{"1min":0.84599999999999997,"5min":0.81999999999999995,"15min":0.65900000000000003},"max":{"1min":1.528,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.29},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.131},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.671,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.04}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2110000000000001,"15min":1.196},"min":{"1min":0.83999999999999997,"5min":0.78200000000000003,"15min":0.77600000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3140000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.871,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.385}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.234},"min":{"1min":0.85399999999999998,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.8220000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3200000000000001,"15min":1.319},"min":{"1min":0.753,"5min":0.753,"15min":0.70399999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.3010000000000002},"last":1.6910000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.96599999999999997,"5min":0.79200000000000004,"15min":0.61899999999999999},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.1549999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2829999999999999,"15min":1.292},"min":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.5189999999999999,"5min":1.8029999999999999,"15min":2.2029999999999998},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.94999999999999996,"5min":0.84799999999999998,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.5029999999999999,"15min":1.4730000000000001},"min":{"1min":1.1000000000000001,"5min":0.96699999999999997,"15min":0.63200000000000001},"max":{"1min":1.7949999999999999,"5min":2.4990000000000001,"15min":2.54},"last":1.468}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.458,"15min":1.464},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.60599999999999998},"max":{"1min":1.6850000000000001,"5min":1.972,"15min":3.2789999999999999},"last":1.349},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5720000000000001,"15min":1.5580000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.04,"15min":2.157},"last":1.8140000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575034,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36088,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335752,"statfs":{"total":96636764160,"available":96599810048,"internally_reserved":0,"allocated":966656,"data_stored":456163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82899999999999996,"15min":0.85199999999999998},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.307},"max":{"1min":1.135,"5min":1.302,"15min":1.583},"last":0.873},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.999,"15min":1.0009999999999999},"min":{"1min":0.52100000000000002,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.3999999999999999,"5min":1.4990000000000001,"15min":1.9770000000000001},"last":1.1870000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.069,"15min":1.0629999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.38900000000000001},"max":{"1min":1.556,"5min":1.571,"15min":2.0779999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.95399999999999996,"15min":0.95199999999999996},"min":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.40600000000000003},"max":{"1min":1.5129999999999999,"5min":1.587,"15min":1.6519999999999999},"last":0.89900000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.016,"15min":1.054},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.436},"max":{"1min":1.286,"5min":1.512,"15min":1.766},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.173,"15min":1.1759999999999999},"min":{"1min":0.63800000000000001,"5min":0.621,"15min":0.48399999999999999},"max":{"1min":1.4870000000000001,"5min":1.647,"15min":1.661},"last":1.0600000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.238,"15min":1.2569999999999999},"min":{"1min":0.96499999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.476,"5min":1.6240000000000001,"15min":2.9350000000000001},"last":1.429},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.246},"min":{"1min":0.94099999999999995,"5min":0.81000000000000005,"15min":0.73299999999999998},"max":{"1min":1.554,"5min":1.637,"15min":2.2389999999999999},"last":1.345}]},{"osd":5,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.272,"15min":1.302},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.63500000000000001},"max":{"1min":1.5900000000000001,"5min":1.806,"15min":3.476},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2729999999999999,"15min":1.329},"min":{"1min":0.86099999999999999,"5min":0.82099999999999995,"15min":0.64200000000000002},"max":{"1min":1.464,"5min":1.9039999999999999,"15min":3.181},"last":1.681}]},{"osd":6,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.88900000000000001,"5min":0.80400000000000005,"15min":0.68300000000000005},"max":{"1min":1.6499999999999999,"5min":1.8580000000000001,"15min":2.532},"last":1.407},{"interface":"front","average":{"1min":1.409,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":1.02,"5min":0.94299999999999995,"15min":0.872},"max":{"1min":1.573,"5min":1.8049999999999999,"15min":3.0249999999999999},"last":1.5860000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3919999999999999,"15min":1.4239999999999999},"min":{"1min":1.006,"5min":0.999,"15min":0.81499999999999995},"max":{"1min":1.504,"5min":1.851,"15min":2.1619999999999999},"last":1.272},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4259999999999999,"15min":1.472},"min":{"1min":1.048,"5min":0.88300000000000001,"15min":0.83099999999999996},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":2.4590000000000001},"last":1.474}]}]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":978944,"data_stored":462438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27100000000000002},"max":{"1min":1.0860000000000001,"5min":1.339,"15min":1.738},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.026,"15min":1.01},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.375},"max":{"1min":1.4159999999999999,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":1.7729999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96499999999999997,"15min":0.98999999999999999},"min":{"1min":0.54700000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.1779999999999999,"5min":1.45,"15min":1.7130000000000001},"last":2.5},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0720000000000001,"15min":1.0840000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.504},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.909}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97499999999999998,"15min":1.0029999999999999},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.377,"5min":1.625,"15min":1.883},"last":1.5},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.39600000000000002},"max":{"1min":1.415,"5min":1.613,"15min":2.6080000000000001},"last":2.4500000000000002}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.331,"15min":1.3180000000000001},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.75},"max":{"1min":1.7230000000000001,"5min":1.9950000000000001,"15min":2.2829999999999999},"last":1.869},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.246,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.964},"last":1.3029999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3009999999999999,"15min":1.319},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.647,"5min":1.804,"15min":2.5779999999999998},"last":2.948},{"interface":"front","average":{"1min":1.375,"5min":1.387,"15min":1.375},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.901,"5min":2.048,"15min":3.0099999999999998},"last":2.016}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.3200000000000001},"min":{"1min":0.84299999999999997,"5min":0.83399999999999996,"15min":0.77500000000000002},"max":{"1min":1.6439999999999999,"5min":1.78,"15min":2.2799999999999998},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.766,"15min":2.5089999999999999},"last":4.0529999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.456,"15min":1.448},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.79400000000000004},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.448},"last":2.1779999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7649999999999999,"5min":1.802,"15min":2.5049999999999999},"last":1.613}]}]},{"osd":1,"up_from":13,"seq":55834575033,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36680,"kb_used_data":1536,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335160,"statfs":{"total":96636764160,"available":96599203840,"internally_reserved":0,"allocated":1572864,"data_stored":1052806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85699999999999998,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.53800000000000003,"15min":0.318},"max":{"1min":1.1220000000000001,"5min":1.5049999999999999,"15min":3.056},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.999,"15min":1.012},"min":{"1min":0.75700000000000001,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":1.335,"5min":1.6359999999999999,"15min":3.4129999999999998},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.88200000000000001,"15min":0.90100000000000002},"min":{"1min":0.67100000000000004,"5min":0.497,"15min":0.36399999999999999},"max":{"1min":1.1779999999999999,"5min":1.54,"15min":3.3660000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.78100000000000003,"5min":0.47899999999999998,"15min":0.39000000000000001},"max":{"1min":1.4890000000000001,"5min":1.492,"15min":3.4980000000000002},"last":1.0600000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.21,"15min":1.224},"min":{"1min":1.0860000000000001,"5min":0.51900000000000002,"15min":0.44},"max":{"1min":1.6859999999999999,"5min":1.7290000000000001,"15min":3.464},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.077,"15min":1.0669999999999999},"min":{"1min":0.82599999999999996,"5min":0.499,"15min":0.39700000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":3.0960000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1719999999999999,"15min":1.196},"min":{"1min":1.014,"5min":0.745,"15min":0.63800000000000001},"max":{"1min":1.494,"5min":1.536,"15min":3.7810000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.403,"5min":1.3240000000000001,"15min":1.3600000000000001},"min":{"1min":1.077,"5min":0.77500000000000002,"15min":0.76600000000000001},"max":{"1min":1.585,"5min":1.8089999999999999,"15min":4.7270000000000003},"last":1.2290000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.296,"15min":1.2909999999999999},"min":{"1min":1.032,"5min":0.90600000000000003,"15min":0.61199999999999999},"max":{"1min":1.5760000000000001,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.278,"15min":1.3160000000000001},"min":{"1min":1.004,"5min":0.76900000000000002,"15min":0.69199999999999995},"max":{"1min":1.5,"5min":1.825,"15min":3.6669999999999998},"last":1.169}]},{"osd":6,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.349},"min":{"1min":1.0760000000000001,"5min":0.71099999999999997,"15min":0.68400000000000005},"max":{"1min":1.5549999999999999,"5min":1.7809999999999999,"15min":3.3300000000000001},"last":1.25},{"interface":"front","average":{"1min":1.383,"5min":1.343,"15min":1.363},"min":{"1min":1.0649999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.595,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.089}]},{"osd":7,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.409,"15min":1.4630000000000001},"min":{"1min":1.246,"5min":0.76600000000000001,"15min":0.70599999999999996},"max":{"1min":1.645,"5min":1.8939999999999999,"15min":4.6749999999999998},"last":1.288},{"interface":"front","average":{"1min":1.444,"5min":1.4179999999999999,"15min":1.4299999999999999},"min":{"1min":1.1639999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7589999999999999,"5min":1.992,"15min":3.5379999999999998},"last":1.3100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":966656,"data_stored":456163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.60699999999999998,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.3520000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.622},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.92600000000000005,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4390000000000001,"5min":1.5580000000000001,"15min":1.6910000000000001},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98499999999999999,"15min":1.006},"min":{"1min":0.67600000000000005,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.4690000000000001,"5min":2.157,"15min":2.157},"last":1.048},{"interface":"front","average":{"1min":1.113,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.84299999999999997,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.395,"5min":1.728,"15min":2.8159999999999998},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.58499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.954},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.143,"15min":1.169},"min":{"1min":0.54900000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7070000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.202}]},{"osd":4,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.3089999999999999},"min":{"1min":1.05,"5min":0.78500000000000003,"15min":0.65700000000000003},"max":{"1min":1.5289999999999999,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.22,"5min":1.214,"15min":1.2629999999999999},"min":{"1min":0.83299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.107}]},{"osd":5,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.337,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.71899999999999997},"max":{"1min":1.839,"5min":2.351,"15min":2.351},"last":1.268},{"interface":"front","average":{"1min":1.276,"5min":1.306,"15min":1.349},"min":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6299999999999999,"5min":2.2909999999999999,"15min":2.7050000000000001},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73099999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.157},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.399},"min":{"1min":1.0129999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.3120000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":1.127,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.706,"5min":1.798,"15min":2.8999999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.421,"5min":1.361,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.006}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:51:29.625 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:51:29.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.882+0000 7f824fe4a640 1 Processor -- start 2023-12-01T01:51:29.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.882+0000 7f824fe4a640 1 -- start start 2023-12-01T01:51:29.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.883+0000 7f824fe4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82480ccc80 0x7f824806fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.883+0000 7f824fe4a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82480ad2b0 0x7f824806fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.883+0000 7f824fe4a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824815d600 con 0x7f824815d220 2023-12-01T01:51:29.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.884+0000 7f824fe4a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824806cc10 con 0x7f82480ad2b0 2023-12-01T01:51:29.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.884+0000 7f824fe4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824806c720 con 0x7f82480ccc80 2023-12-01T01:51:29.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.884+0000 7f824e3e7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82480ad2b0 0x7f824806fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.884+0000 7f824e3e7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82480ad2b0 0x7f824806fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39222/0 (socket says 172.21.15.115:39222) 2023-12-01T01:51:29.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.884+0000 7f824e3e7640 1 -- 172.21.15.115:0/1554766017 learned_addr learned my addr 172.21.15.115:0/1554766017 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:29.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.884+0000 7f824d3e5640 1 --2- 172.21.15.115:0/1554766017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82480ccc80 0x7f824806fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.884+0000 7f824e3e7640 1 -- 172.21.15.115:0/1554766017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82480ccc80 msgr2=0x7f824806fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:29.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.884+0000 7f824e3e7640 1 --2- 172.21.15.115:0/1554766017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82480ccc80 0x7f824806fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.884+0000 7f824e3e7640 1 -- 172.21.15.115:0/1554766017 >> v1:172.21.15.142:6789/0 conn(0x7f824815d220 legacy=0x7f82480cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:29.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.884+0000 7f824e3e7640 1 -- 172.21.15.115:0/1554766017 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824806c4b0 con 0x7f82480ad2b0 2023-12-01T01:51:29.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.885+0000 7f824e3e7640 1 --2- 172.21.15.115:0/1554766017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ad2b0 0x7f824806fff0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8240009e50 tx=0x7f824002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aafe91edb25f5a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:29.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.886+0000 7f824cbe4640 1 -- 172.21.15.115:0/1554766017 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824002c9e0 con 0x7f82480ad2b0 2023-12-01T01:51:29.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.886+0000 7f824cbe4640 1 -- 172.21.15.115:0/1554766017 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824002cb40 con 0x7f82480ad2b0 2023-12-01T01:51:29.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.886+0000 7f824cbe4640 1 -- 172.21.15.115:0/1554766017 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82400343f0 con 0x7f82480ad2b0 2023-12-01T01:51:29.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.887+0000 7f824fe4a640 1 -- 172.21.15.115:0/1554766017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ad2b0 msgr2=0x7f824806fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:29.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.887+0000 7f824fe4a640 1 --2- 172.21.15.115:0/1554766017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ad2b0 0x7f824806fff0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8240009e50 tx=0x7f824002be40 comp rx=0 tx=0).stop 2023-12-01T01:51:29.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.887+0000 7f824fe4a640 1 -- 172.21.15.115:0/1554766017 shutdown_connections 2023-12-01T01:51:29.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.887+0000 7f824fe4a640 1 --2- 172.21.15.115:0/1554766017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ad2b0 0x7f824806fff0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.887+0000 7f824fe4a640 1 --2- 172.21.15.115:0/1554766017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82480ccc80 0x7f824806fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.887+0000 7f824fe4a640 1 -- 172.21.15.115:0/1554766017 >> 172.21.15.115:0/1554766017 conn(0x7f82480ce880 msgr2=0x7f82480cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:29.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.887+0000 7f824fe4a640 1 -- 172.21.15.115:0/1554766017 shutdown_connections 2023-12-01T01:51:29.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.887+0000 7f824fe4a640 1 -- 172.21.15.115:0/1554766017 wait complete. 2023-12-01T01:51:29.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.889+0000 7f824fe4a640 1 Processor -- start 2023-12-01T01:51:29.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.889+0000 7f824fe4a640 1 -- start start 2023-12-01T01:51:29.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824fe4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82480ad2b0 0x7f8248154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824fe4a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ccc80 0x7f82480c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824fe4a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824815d220 0x7f82481588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824fe4a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82481546d0 con 0x7f82480ccc80 2023-12-01T01:51:29.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824fe4a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82480c6db0 con 0x7f824815d220 2023-12-01T01:51:29.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824fe4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82480c3260 con 0x7f82480ad2b0 2023-12-01T01:51:29.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824d3e5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ccc80 0x7f82480c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824d3e5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ccc80 0x7f82480c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39238/0 (socket says 172.21.15.115:39238) 2023-12-01T01:51:29.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824d3e5640 1 -- 172.21.15.115:0/4089896768 learned_addr learned my addr 172.21.15.115:0/4089896768 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:29.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824e3e7640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824815d220 0x7f82481588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824d3e5640 1 -- 172.21.15.115:0/4089896768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82480ad2b0 msgr2=0x7f8248154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:29.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824dbe6640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82480ad2b0 0x7f8248154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824d3e5640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82480ad2b0 0x7f8248154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824d3e5640 1 -- 172.21.15.115:0/4089896768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824815d220 msgr2=0x7f82481588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:29.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824d3e5640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824815d220 0x7f82481588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:29.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824d3e5640 1 -- 172.21.15.115:0/4089896768 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8240009b30 con 0x7f82480ccc80 2023-12-01T01:51:29.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.890+0000 7f824e3e7640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824815d220 0x7f82481588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:29.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.891+0000 7f824d3e5640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ccc80 0x7f82480c4d70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f823800d5f0 tx=0x7f823800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:29.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.891+0000 7f823e7fc640 1 -- 172.21.15.115:0/4089896768 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82380125f0 con 0x7f82480ccc80 2023-12-01T01:51:29.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.891+0000 7f823e7fc640 1 -- 172.21.15.115:0/4089896768 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8238012750 con 0x7f82480ccc80 2023-12-01T01:51:29.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.891+0000 7f823e7fc640 1 -- 172.21.15.115:0/4089896768 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238012b00 con 0x7f82480ccc80 2023-12-01T01:51:29.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.891+0000 7f824fe4a640 1 -- 172.21.15.115:0/4089896768 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82480c3540 con 0x7f82480ccc80 2023-12-01T01:51:29.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.891+0000 7f824fe4a640 1 -- 172.21.15.115:0/4089896768 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82480c3820 con 0x7f82480ccc80 2023-12-01T01:51:29.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.892+0000 7f824fe4a640 1 -- 172.21.15.115:0/4089896768 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824804e890 con 0x7f82480ccc80 2023-12-01T01:51:29.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.893+0000 7f823e7fc640 1 -- 172.21.15.115:0/4089896768 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8238012c60 con 0x7f82480ccc80 2023-12-01T01:51:29.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.894+0000 7f823e7fc640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f822c06a550 0x7f822c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:29.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.894+0000 7f823e7fc640 1 -- 172.21.15.115:0/4089896768 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8238088f30 con 0x7f82480ccc80 2023-12-01T01:51:29.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.894+0000 7f824dbe6640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f822c06a550 0x7f822c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:29.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.895+0000 7f824dbe6640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f822c06a550 0x7f822c06ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f824400a750 tx=0x7f8244008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:29.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:29.898+0000 7f823e7fc640 1 -- 172.21.15.115:0/4089896768 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8238054400 con 0x7f82480ccc80 2023-12-01T01:51:30.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.084+0000 7f824fe4a640 1 -- 172.21.15.115:0/4089896768 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f824806ce10 con 0x7f822c06a550 2023-12-01T01:51:30.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.092+0000 7f823e7fc640 1 -- 172.21.15.115:0/4089896768 <== mgr.6201 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286247 (secure 0 0 0) 0x7f824400c520 con 0x7f822c06a550 2023-12-01T01:51:30.098 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:30.099 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:51:30.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.098+0000 7f824fe4a640 1 -- 172.21.15.115:0/4089896768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f822c06a550 msgr2=0x7f822c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.098+0000 7f824fe4a640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f822c06a550 0x7f822c06ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f824400a750 tx=0x7f8244008040 comp rx=0 tx=0).stop 2023-12-01T01:51:30.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.098+0000 7f824fe4a640 1 -- 172.21.15.115:0/4089896768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ccc80 msgr2=0x7f82480c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.098+0000 7f824fe4a640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ccc80 0x7f82480c4d70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f823800d5f0 tx=0x7f823800dab0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.099+0000 7f824fe4a640 1 -- 172.21.15.115:0/4089896768 shutdown_connections 2023-12-01T01:51:30.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.099+0000 7f824fe4a640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f822c06a550 0x7f822c06ca10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.099+0000 7f824fe4a640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824815d220 0x7f82481588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.099+0000 7f824fe4a640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82480ccc80 0x7f82480c4d70 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.099+0000 7f824fe4a640 1 --2- 172.21.15.115:0/4089896768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82480ad2b0 0x7f8248154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.099+0000 7f824fe4a640 1 -- 172.21.15.115:0/4089896768 >> 172.21.15.115:0/4089896768 conn(0x7f82480ce880 msgr2=0x7f8248159f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:30.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.099+0000 7f824fe4a640 1 -- 172.21.15.115:0/4089896768 shutdown_connections 2023-12-01T01:51:30.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.099+0000 7f824fe4a640 1 -- 172.21.15.115:0/4089896768 wait complete. 2023-12-01T01:51:30.118 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-12-01T01:51:29.825357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269736,"kb_used_data":9064,"kb_used_omap":0,"kb_used_meta":260608,"kb_avail":754704984,"statfs":{"total":773094113280,"available":772817903616,"internally_reserved":0,"allocated":9281536,"data_stored":4989564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":266862592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.128678"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228585+0000","last_change":"2023-12-01T01:51:24.224886+0000","last_active":"2023-12-01T01:51:24.228585+0000","last_peered":"2023-12-01T01:51:24.228585+0000","last_clean":"2023-12-01T01:51:24.228585+0000","last_became_active":"2023-12-01T01:51:24.222251+0000","last_became_peered":"2023-12-01T01:51:24.222251+0000","last_unstale":"2023-12-01T01:51:24.228585+0000","last_undegraded":"2023-12-01T01:51:24.228585+0000","last_fullsized":"2023-12-01T01:51:24.228585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:21.413565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228337+0000","last_change":"2023-12-01T01:51:24.223609+0000","last_active":"2023-12-01T01:51:24.228337+0000","last_peered":"2023-12-01T01:51:24.228337+0000","last_clean":"2023-12-01T01:51:24.228337+0000","last_became_active":"2023-12-01T01:51:24.222770+0000","last_became_peered":"2023-12-01T01:51:24.222770+0000","last_unstale":"2023-12-01T01:51:24.228337+0000","last_undegraded":"2023-12-01T01:51:24.228337+0000","last_fullsized":"2023-12-01T01:51:24.228337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:26.173230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.209949+0000","last_change":"2023-12-01T01:51:24.207820+0000","last_active":"2023-12-01T01:51:24.209949+0000","last_peered":"2023-12-01T01:51:24.209949+0000","last_clean":"2023-12-01T01:51:24.209949+0000","last_became_active":"2023-12-01T01:51:24.207535+0000","last_became_peered":"2023-12-01T01:51:24.207535+0000","last_unstale":"2023-12-01T01:51:24.209949+0000","last_undegraded":"2023-12-01T01:51:24.209949+0000","last_fullsized":"2023-12-01T01:51:24.209949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:50.301817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223930+0000","last_change":"2023-12-01T01:51:24.222179+0000","last_active":"2023-12-01T01:51:24.223930+0000","last_peered":"2023-12-01T01:51:24.223930+0000","last_clean":"2023-12-01T01:51:24.223930+0000","last_became_active":"2023-12-01T01:51:24.221936+0000","last_became_peered":"2023-12-01T01:51:24.221936+0000","last_unstale":"2023-12-01T01:51:24.223930+0000","last_undegraded":"2023-12-01T01:51:24.223930+0000","last_fullsized":"2023-12-01T01:51:24.223930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:03:53.147122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224064+0000","last_change":"2023-12-01T01:51:24.219076+0000","last_active":"2023-12-01T01:51:24.224064+0000","last_peered":"2023-12-01T01:51:24.224064+0000","last_clean":"2023-12-01T01:51:24.224064+0000","last_became_active":"2023-12-01T01:51:24.218809+0000","last_became_peered":"2023-12-01T01:51:24.218809+0000","last_unstale":"2023-12-01T01:51:24.224064+0000","last_undegraded":"2023-12-01T01:51:24.224064+0000","last_fullsized":"2023-12-01T01:51:24.224064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:16.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224174+0000","last_change":"2023-12-01T01:51:24.221005+0000","last_active":"2023-12-01T01:51:24.224174+0000","last_peered":"2023-12-01T01:51:24.224174+0000","last_clean":"2023-12-01T01:51:24.224174+0000","last_became_active":"2023-12-01T01:51:24.220737+0000","last_became_peered":"2023-12-01T01:51:24.220737+0000","last_unstale":"2023-12-01T01:51:24.224174+0000","last_undegraded":"2023-12-01T01:51:24.224174+0000","last_fullsized":"2023-12-01T01:51:24.224174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:08.979742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.212243+0000","last_change":"2023-12-01T01:51:24.205047+0000","last_active":"2023-12-01T01:51:24.212243+0000","last_peered":"2023-12-01T01:51:24.212243+0000","last_clean":"2023-12-01T01:51:24.212243+0000","last_became_active":"2023-12-01T01:51:24.204779+0000","last_became_peered":"2023-12-01T01:51:24.204779+0000","last_unstale":"2023-12-01T01:51:24.212243+0000","last_undegraded":"2023-12-01T01:51:24.212243+0000","last_fullsized":"2023-12-01T01:51:24.212243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:41.379891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.204987+0000","last_change":"2023-12-01T01:51:24.199691+0000","last_active":"2023-12-01T01:51:24.204987+0000","last_peered":"2023-12-01T01:51:24.204987+0000","last_clean":"2023-12-01T01:51:24.204987+0000","last_became_active":"2023-12-01T01:51:24.199343+0000","last_became_peered":"2023-12-01T01:51:24.199343+0000","last_unstale":"2023-12-01T01:51:24.204987+0000","last_undegraded":"2023-12-01T01:51:24.204987+0000","last_fullsized":"2023-12-01T01:51:24.204987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:56:56.216365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223463+0000","last_change":"2023-12-01T01:51:24.221268+0000","last_active":"2023-12-01T01:51:24.223463+0000","last_peered":"2023-12-01T01:51:24.223463+0000","last_clean":"2023-12-01T01:51:24.223463+0000","last_became_active":"2023-12-01T01:51:24.220971+0000","last_became_peered":"2023-12-01T01:51:24.220971+0000","last_unstale":"2023-12-01T01:51:24.223463+0000","last_undegraded":"2023-12-01T01:51:24.223463+0000","last_fullsized":"2023-12-01T01:51:24.223463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:55.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225919+0000","last_change":"2023-12-01T01:51:24.222276+0000","last_active":"2023-12-01T01:51:24.225919+0000","last_peered":"2023-12-01T01:51:24.225919+0000","last_clean":"2023-12-01T01:51:24.225919+0000","last_became_active":"2023-12-01T01:51:24.220691+0000","last_became_peered":"2023-12-01T01:51:24.220691+0000","last_unstale":"2023-12-01T01:51:24.225919+0000","last_undegraded":"2023-12-01T01:51:24.225919+0000","last_fullsized":"2023-12-01T01:51:24.225919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:52.689385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225715+0000","last_change":"2023-12-01T01:51:24.221827+0000","last_active":"2023-12-01T01:51:24.225715+0000","last_peered":"2023-12-01T01:51:24.225715+0000","last_clean":"2023-12-01T01:51:24.225715+0000","last_became_active":"2023-12-01T01:51:24.221267+0000","last_became_peered":"2023-12-01T01:51:24.221267+0000","last_unstale":"2023-12-01T01:51:24.225715+0000","last_undegraded":"2023-12-01T01:51:24.225715+0000","last_fullsized":"2023-12-01T01:51:24.225715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:21.413565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.197650+0000","last_change":"2023-12-01T01:51:24.193709+0000","last_active":"2023-12-01T01:51:24.197650+0000","last_peered":"2023-12-01T01:51:24.197650+0000","last_clean":"2023-12-01T01:51:24.197650+0000","last_became_active":"2023-12-01T01:51:24.193253+0000","last_became_peered":"2023-12-01T01:51:24.193253+0000","last_unstale":"2023-12-01T01:51:24.197650+0000","last_undegraded":"2023-12-01T01:51:24.197650+0000","last_fullsized":"2023-12-01T01:51:24.197650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:32.142316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217097+0000","last_change":"2023-12-01T01:51:24.211498+0000","last_active":"2023-12-01T01:51:24.217097+0000","last_peered":"2023-12-01T01:51:24.217097+0000","last_clean":"2023-12-01T01:51:24.217097+0000","last_became_active":"2023-12-01T01:51:24.211147+0000","last_became_peered":"2023-12-01T01:51:24.211147+0000","last_unstale":"2023-12-01T01:51:24.217097+0000","last_undegraded":"2023-12-01T01:51:24.217097+0000","last_fullsized":"2023-12-01T01:51:24.217097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:01.177699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211052+0000","last_change":"2023-12-01T01:51:24.208945+0000","last_active":"2023-12-01T01:51:24.211052+0000","last_peered":"2023-12-01T01:51:24.211052+0000","last_clean":"2023-12-01T01:51:24.211052+0000","last_became_active":"2023-12-01T01:51:24.208667+0000","last_became_peered":"2023-12-01T01:51:24.208667+0000","last_unstale":"2023-12-01T01:51:24.211052+0000","last_undegraded":"2023-12-01T01:51:24.211052+0000","last_fullsized":"2023-12-01T01:51:24.211052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:29.110060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223068+0000","last_change":"2023-12-01T01:51:24.221816+0000","last_active":"2023-12-01T01:51:24.223068+0000","last_peered":"2023-12-01T01:51:24.223068+0000","last_clean":"2023-12-01T01:51:24.223068+0000","last_became_active":"2023-12-01T01:51:24.221513+0000","last_became_peered":"2023-12-01T01:51:24.221513+0000","last_unstale":"2023-12-01T01:51:24.223068+0000","last_undegraded":"2023-12-01T01:51:24.223068+0000","last_fullsized":"2023-12-01T01:51:24.223068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.219844+0000","last_change":"2023-12-01T01:51:24.212262+0000","last_active":"2023-12-01T01:51:24.219844+0000","last_peered":"2023-12-01T01:51:24.219844+0000","last_clean":"2023-12-01T01:51:24.219844+0000","last_became_active":"2023-12-01T01:51:24.211984+0000","last_became_peered":"2023-12-01T01:51:24.211984+0000","last_unstale":"2023-12-01T01:51:24.219844+0000","last_undegraded":"2023-12-01T01:51:24.219844+0000","last_fullsized":"2023-12-01T01:51:24.219844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:28.427573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228598+0000","last_change":"2023-12-01T01:51:24.225154+0000","last_active":"2023-12-01T01:51:24.228598+0000","last_peered":"2023-12-01T01:51:24.228598+0000","last_clean":"2023-12-01T01:51:24.228598+0000","last_became_active":"2023-12-01T01:51:24.224886+0000","last_became_peered":"2023-12-01T01:51:24.224886+0000","last_unstale":"2023-12-01T01:51:24.228598+0000","last_undegraded":"2023-12-01T01:51:24.228598+0000","last_fullsized":"2023-12-01T01:51:24.228598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223814+0000","last_change":"2023-12-01T01:51:24.222850+0000","last_active":"2023-12-01T01:51:24.223814+0000","last_peered":"2023-12-01T01:51:24.223814+0000","last_clean":"2023-12-01T01:51:24.223814+0000","last_became_active":"2023-12-01T01:51:24.222475+0000","last_became_peered":"2023-12-01T01:51:24.222475+0000","last_unstale":"2023-12-01T01:51:24.223814+0000","last_undegraded":"2023-12-01T01:51:24.223814+0000","last_fullsized":"2023-12-01T01:51:24.223814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:27.535710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225788+0000","last_change":"2023-12-01T01:51:24.220362+0000","last_active":"2023-12-01T01:51:24.225788+0000","last_peered":"2023-12-01T01:51:24.225788+0000","last_clean":"2023-12-01T01:51:24.225788+0000","last_became_active":"2023-12-01T01:51:24.219702+0000","last_became_peered":"2023-12-01T01:51:24.219702+0000","last_unstale":"2023-12-01T01:51:24.225788+0000","last_undegraded":"2023-12-01T01:51:24.225788+0000","last_fullsized":"2023-12-01T01:51:24.225788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:14.157429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217404+0000","last_change":"2023-12-01T01:51:24.211275+0000","last_active":"2023-12-01T01:51:24.217404+0000","last_peered":"2023-12-01T01:51:24.217404+0000","last_clean":"2023-12-01T01:51:24.217404+0000","last_became_active":"2023-12-01T01:51:24.210964+0000","last_became_peered":"2023-12-01T01:51:24.210964+0000","last_unstale":"2023-12-01T01:51:24.217404+0000","last_undegraded":"2023-12-01T01:51:24.217404+0000","last_fullsized":"2023-12-01T01:51:24.217404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:08.979742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.212510+0000","last_change":"2023-12-01T01:51:24.206715+0000","last_active":"2023-12-01T01:51:24.212510+0000","last_peered":"2023-12-01T01:51:24.212510+0000","last_clean":"2023-12-01T01:51:24.212510+0000","last_became_active":"2023-12-01T01:51:24.206432+0000","last_became_peered":"2023-12-01T01:51:24.206432+0000","last_unstale":"2023-12-01T01:51:24.212510+0000","last_undegraded":"2023-12-01T01:51:24.212510+0000","last_fullsized":"2023-12-01T01:51:24.212510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217430+0000","last_change":"2023-12-01T01:51:24.212833+0000","last_active":"2023-12-01T01:51:24.217430+0000","last_peered":"2023-12-01T01:51:24.217430+0000","last_clean":"2023-12-01T01:51:24.217430+0000","last_became_active":"2023-12-01T01:51:24.212494+0000","last_became_peered":"2023-12-01T01:51:24.212494+0000","last_unstale":"2023-12-01T01:51:24.217430+0000","last_undegraded":"2023-12-01T01:51:24.217430+0000","last_fullsized":"2023-12-01T01:51:24.217430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:12.784193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217998+0000","last_change":"2023-12-01T01:51:24.215195+0000","last_active":"2023-12-01T01:51:24.217998+0000","last_peered":"2023-12-01T01:51:24.217998+0000","last_clean":"2023-12-01T01:51:24.217998+0000","last_became_active":"2023-12-01T01:51:24.214924+0000","last_became_peered":"2023-12-01T01:51:24.214924+0000","last_unstale":"2023-12-01T01:51:24.217998+0000","last_undegraded":"2023-12-01T01:51:24.217998+0000","last_fullsized":"2023-12-01T01:51:24.217998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:39:57.857405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211682+0000","last_change":"2023-12-01T01:51:24.210975+0000","last_active":"2023-12-01T01:51:24.211682+0000","last_peered":"2023-12-01T01:51:24.211682+0000","last_clean":"2023-12-01T01:51:24.211682+0000","last_became_active":"2023-12-01T01:51:24.210687+0000","last_became_peered":"2023-12-01T01:51:24.210687+0000","last_unstale":"2023-12-01T01:51:24.211682+0000","last_undegraded":"2023-12-01T01:51:24.211682+0000","last_fullsized":"2023-12-01T01:51:24.211682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:51.772038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225706+0000","last_change":"2023-12-01T01:51:24.221053+0000","last_active":"2023-12-01T01:51:24.225706+0000","last_peered":"2023-12-01T01:51:24.225706+0000","last_clean":"2023-12-01T01:51:24.225706+0000","last_became_active":"2023-12-01T01:51:24.219646+0000","last_became_peered":"2023-12-01T01:51:24.219646+0000","last_unstale":"2023-12-01T01:51:24.225706+0000","last_undegraded":"2023-12-01T01:51:24.225706+0000","last_fullsized":"2023-12-01T01:51:24.225706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:06:06.433309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.200813+0000","last_change":"2023-12-01T01:51:24.197255+0000","last_active":"2023-12-01T01:51:24.200813+0000","last_peered":"2023-12-01T01:51:24.200813+0000","last_clean":"2023-12-01T01:51:24.200813+0000","last_became_active":"2023-12-01T01:51:24.197005+0000","last_became_peered":"2023-12-01T01:51:24.197005+0000","last_unstale":"2023-12-01T01:51:24.200813+0000","last_undegraded":"2023-12-01T01:51:24.200813+0000","last_fullsized":"2023-12-01T01:51:24.200813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:10.334292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225808+0000","last_change":"2023-12-01T01:51:24.220582+0000","last_active":"2023-12-01T01:51:24.225808+0000","last_peered":"2023-12-01T01:51:24.225808+0000","last_clean":"2023-12-01T01:51:24.225808+0000","last_became_active":"2023-12-01T01:51:24.219677+0000","last_became_peered":"2023-12-01T01:51:24.219677+0000","last_unstale":"2023-12-01T01:51:24.225808+0000","last_undegraded":"2023-12-01T01:51:24.225808+0000","last_fullsized":"2023-12-01T01:51:24.225808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:10:04.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.203074+0000","last_change":"2023-12-01T01:51:24.198223+0000","last_active":"2023-12-01T01:51:24.203074+0000","last_peered":"2023-12-01T01:51:24.203074+0000","last_clean":"2023-12-01T01:51:24.203074+0000","last_became_active":"2023-12-01T01:51:24.197651+0000","last_became_peered":"2023-12-01T01:51:24.197651+0000","last_unstale":"2023-12-01T01:51:24.203074+0000","last_undegraded":"2023-12-01T01:51:24.203074+0000","last_fullsized":"2023-12-01T01:51:24.203074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:46.030193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.208866+0000","last_change":"2023-12-01T01:51:24.205197+0000","last_active":"2023-12-01T01:51:24.208866+0000","last_peered":"2023-12-01T01:51:24.208866+0000","last_clean":"2023-12-01T01:51:24.208866+0000","last_became_active":"2023-12-01T01:51:24.204894+0000","last_became_peered":"2023-12-01T01:51:24.204894+0000","last_unstale":"2023-12-01T01:51:24.208866+0000","last_undegraded":"2023-12-01T01:51:24.208866+0000","last_fullsized":"2023-12-01T01:51:24.208866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:49.877816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.197827+0000","last_change":"2023-12-01T01:51:24.196324+0000","last_active":"2023-12-01T01:51:24.197827+0000","last_peered":"2023-12-01T01:51:24.197827+0000","last_clean":"2023-12-01T01:51:24.197827+0000","last_became_active":"2023-12-01T01:51:24.194764+0000","last_became_peered":"2023-12-01T01:51:24.194764+0000","last_unstale":"2023-12-01T01:51:24.197827+0000","last_undegraded":"2023-12-01T01:51:24.197827+0000","last_fullsized":"2023-12-01T01:51:24.197827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:28.026095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221789+0000","last_change":"2023-12-01T01:51:24.213551+0000","last_active":"2023-12-01T01:51:24.221789+0000","last_peered":"2023-12-01T01:51:24.221789+0000","last_clean":"2023-12-01T01:51:24.221789+0000","last_became_active":"2023-12-01T01:51:24.212904+0000","last_became_peered":"2023-12-01T01:51:24.212904+0000","last_unstale":"2023-12-01T01:51:24.221789+0000","last_undegraded":"2023-12-01T01:51:24.221789+0000","last_fullsized":"2023-12-01T01:51:24.221789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:52.879295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217559+0000","last_change":"2023-12-01T01:51:24.212738+0000","last_active":"2023-12-01T01:51:24.217559+0000","last_peered":"2023-12-01T01:51:24.217559+0000","last_clean":"2023-12-01T01:51:24.217559+0000","last_became_active":"2023-12-01T01:51:24.212433+0000","last_became_peered":"2023-12-01T01:51:24.212433+0000","last_unstale":"2023-12-01T01:51:24.217559+0000","last_undegraded":"2023-12-01T01:51:24.217559+0000","last_fullsized":"2023-12-01T01:51:24.217559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:16.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220228+0000","last_change":"2023-12-01T01:51:24.211814+0000","last_active":"2023-12-01T01:51:24.220228+0000","last_peered":"2023-12-01T01:51:24.220228+0000","last_clean":"2023-12-01T01:51:24.220228+0000","last_became_active":"2023-12-01T01:51:24.211542+0000","last_became_peered":"2023-12-01T01:51:24.211542+0000","last_unstale":"2023-12-01T01:51:24.220228+0000","last_undegraded":"2023-12-01T01:51:24.220228+0000","last_fullsized":"2023-12-01T01:51:24.220228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:01:44.563857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"82'4","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218866+0000","last_change":"2023-12-01T01:51:23.722626+0000","last_active":"2023-12-01T01:51:24.218866+0000","last_peered":"2023-12-01T01:51:24.218866+0000","last_clean":"2023-12-01T01:51:24.218866+0000","last_became_active":"2023-12-01T01:51:23.722272+0000","last_became_peered":"2023-12-01T01:51:23.722272+0000","last_unstale":"2023-12-01T01:51:24.218866+0000","last_undegraded":"2023-12-01T01:51:24.218866+0000","last_fullsized":"2023-12-01T01:51:24.218866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:49.297425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223115+0000","last_change":"2023-12-01T01:51:24.218268+0000","last_active":"2023-12-01T01:51:24.223115+0000","last_peered":"2023-12-01T01:51:24.223115+0000","last_clean":"2023-12-01T01:51:24.223115+0000","last_became_active":"2023-12-01T01:51:24.217888+0000","last_became_peered":"2023-12-01T01:51:24.217888+0000","last_unstale":"2023-12-01T01:51:24.223115+0000","last_undegraded":"2023-12-01T01:51:24.223115+0000","last_fullsized":"2023-12-01T01:51:24.223115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:12.784193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.219239+0000","last_change":"2023-12-01T01:51:23.730769+0000","last_active":"2023-12-01T01:51:24.219239+0000","last_peered":"2023-12-01T01:51:24.219239+0000","last_clean":"2023-12-01T01:51:24.219239+0000","last_became_active":"2023-12-01T01:51:23.730468+0000","last_became_peered":"2023-12-01T01:51:23.730468+0000","last_unstale":"2023-12-01T01:51:24.219239+0000","last_undegraded":"2023-12-01T01:51:24.219239+0000","last_fullsized":"2023-12-01T01:51:24.219239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:56.584539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218966+0000","last_change":"2023-12-01T01:51:24.214183+0000","last_active":"2023-12-01T01:51:24.218966+0000","last_peered":"2023-12-01T01:51:24.218966+0000","last_clean":"2023-12-01T01:51:24.218966+0000","last_became_active":"2023-12-01T01:51:24.213872+0000","last_became_peered":"2023-12-01T01:51:24.213872+0000","last_unstale":"2023-12-01T01:51:24.218966+0000","last_undegraded":"2023-12-01T01:51:24.218966+0000","last_fullsized":"2023-12-01T01:51:24.218966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:53:01.496510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228599+0000","last_change":"2023-12-01T01:51:23.729205+0000","last_active":"2023-12-01T01:51:24.228599+0000","last_peered":"2023-12-01T01:51:24.228599+0000","last_clean":"2023-12-01T01:51:24.228599+0000","last_became_active":"2023-12-01T01:51:23.728920+0000","last_became_peered":"2023-12-01T01:51:23.728920+0000","last_unstale":"2023-12-01T01:51:24.228599+0000","last_undegraded":"2023-12-01T01:51:24.228599+0000","last_fullsized":"2023-12-01T01:51:24.228599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:21:58.877785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.226012+0000","last_change":"2023-12-01T01:51:24.223234+0000","last_active":"2023-12-01T01:51:24.226012+0000","last_peered":"2023-12-01T01:51:24.226012+0000","last_clean":"2023-12-01T01:51:24.226012+0000","last_became_active":"2023-12-01T01:51:24.222152+0000","last_became_peered":"2023-12-01T01:51:24.222152+0000","last_unstale":"2023-12-01T01:51:24.226012+0000","last_undegraded":"2023-12-01T01:51:24.226012+0000","last_fullsized":"2023-12-01T01:51:24.226012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:01.177699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205222+0000","last_change":"2023-12-01T01:51:23.729190+0000","last_active":"2023-12-01T01:51:24.205222+0000","last_peered":"2023-12-01T01:51:24.205222+0000","last_clean":"2023-12-01T01:51:24.205222+0000","last_became_active":"2023-12-01T01:51:23.728867+0000","last_became_peered":"2023-12-01T01:51:23.728867+0000","last_unstale":"2023-12-01T01:51:24.205222+0000","last_undegraded":"2023-12-01T01:51:24.205222+0000","last_fullsized":"2023-12-01T01:51:24.205222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:43:10.966577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.209477+0000","last_change":"2023-12-01T01:51:24.205594+0000","last_active":"2023-12-01T01:51:24.209477+0000","last_peered":"2023-12-01T01:51:24.209477+0000","last_clean":"2023-12-01T01:51:24.209477+0000","last_became_active":"2023-12-01T01:51:24.205042+0000","last_became_peered":"2023-12-01T01:51:24.205042+0000","last_unstale":"2023-12-01T01:51:24.209477+0000","last_undegraded":"2023-12-01T01:51:24.209477+0000","last_fullsized":"2023-12-01T01:51:24.209477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:01:53.075668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211402+0000","last_change":"2023-12-01T01:51:23.717746+0000","last_active":"2023-12-01T01:51:24.211402+0000","last_peered":"2023-12-01T01:51:24.211402+0000","last_clean":"2023-12-01T01:51:24.211402+0000","last_became_active":"2023-12-01T01:51:23.715918+0000","last_became_peered":"2023-12-01T01:51:23.715918+0000","last_unstale":"2023-12-01T01:51:24.211402+0000","last_undegraded":"2023-12-01T01:51:24.211402+0000","last_fullsized":"2023-12-01T01:51:24.211402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:35:50.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.219334+0000","last_change":"2023-12-01T01:51:24.214836+0000","last_active":"2023-12-01T01:51:24.219334+0000","last_peered":"2023-12-01T01:51:24.219334+0000","last_clean":"2023-12-01T01:51:24.219334+0000","last_became_active":"2023-12-01T01:51:24.214055+0000","last_became_peered":"2023-12-01T01:51:24.214055+0000","last_unstale":"2023-12-01T01:51:24.219334+0000","last_undegraded":"2023-12-01T01:51:24.219334+0000","last_fullsized":"2023-12-01T01:51:24.219334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:06.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221430+0000","last_change":"2023-12-01T01:51:23.725607+0000","last_active":"2023-12-01T01:51:24.221430+0000","last_peered":"2023-12-01T01:51:24.221430+0000","last_clean":"2023-12-01T01:51:24.221430+0000","last_became_active":"2023-12-01T01:51:23.722833+0000","last_became_peered":"2023-12-01T01:51:23.722833+0000","last_unstale":"2023-12-01T01:51:24.221430+0000","last_undegraded":"2023-12-01T01:51:24.221430+0000","last_fullsized":"2023-12-01T01:51:24.221430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:10:03.788374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.214615+0000","last_change":"2023-12-01T01:51:24.211805+0000","last_active":"2023-12-01T01:51:24.214615+0000","last_peered":"2023-12-01T01:51:24.214615+0000","last_clean":"2023-12-01T01:51:24.214615+0000","last_became_active":"2023-12-01T01:51:24.211532+0000","last_became_peered":"2023-12-01T01:51:24.211532+0000","last_unstale":"2023-12-01T01:51:24.214615+0000","last_undegraded":"2023-12-01T01:51:24.214615+0000","last_fullsized":"2023-12-01T01:51:24.214615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:51.144418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.186975+0000","last_change":"2023-12-01T01:51:23.729822+0000","last_active":"2023-12-01T01:51:24.186975+0000","last_peered":"2023-12-01T01:51:24.186975+0000","last_clean":"2023-12-01T01:51:24.186975+0000","last_became_active":"2023-12-01T01:51:23.728147+0000","last_became_peered":"2023-12-01T01:51:23.728147+0000","last_unstale":"2023-12-01T01:51:24.186975+0000","last_undegraded":"2023-12-01T01:51:24.186975+0000","last_fullsized":"2023-12-01T01:51:24.186975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:43:10.667895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225883+0000","last_change":"2023-12-01T01:51:24.222272+0000","last_active":"2023-12-01T01:51:24.225883+0000","last_peered":"2023-12-01T01:51:24.225883+0000","last_clean":"2023-12-01T01:51:24.225883+0000","last_became_active":"2023-12-01T01:51:24.221990+0000","last_became_peered":"2023-12-01T01:51:24.221990+0000","last_unstale":"2023-12-01T01:51:24.225883+0000","last_undegraded":"2023-12-01T01:51:24.225883+0000","last_fullsized":"2023-12-01T01:51:24.225883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187467+0000","last_change":"2023-12-01T01:51:23.710030+0000","last_active":"2023-12-01T01:51:24.187467+0000","last_peered":"2023-12-01T01:51:24.187467+0000","last_clean":"2023-12-01T01:51:24.187467+0000","last_became_active":"2023-12-01T01:51:23.709381+0000","last_became_peered":"2023-12-01T01:51:23.709381+0000","last_unstale":"2023-12-01T01:51:24.187467+0000","last_undegraded":"2023-12-01T01:51:24.187467+0000","last_fullsized":"2023-12-01T01:51:24.187467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:25.168837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.207004+0000","last_change":"2023-12-01T01:51:24.200936+0000","last_active":"2023-12-01T01:51:24.207004+0000","last_peered":"2023-12-01T01:51:24.207004+0000","last_clean":"2023-12-01T01:51:24.207004+0000","last_became_active":"2023-12-01T01:51:24.198684+0000","last_became_peered":"2023-12-01T01:51:24.198684+0000","last_unstale":"2023-12-01T01:51:24.207004+0000","last_undegraded":"2023-12-01T01:51:24.207004+0000","last_fullsized":"2023-12-01T01:51:24.207004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:50.301817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.188426+0000","last_change":"2023-12-01T01:51:23.725370+0000","last_active":"2023-12-01T01:51:24.188426+0000","last_peered":"2023-12-01T01:51:24.188426+0000","last_clean":"2023-12-01T01:51:24.188426+0000","last_became_active":"2023-12-01T01:51:23.725113+0000","last_became_peered":"2023-12-01T01:51:23.725113+0000","last_unstale":"2023-12-01T01:51:24.188426+0000","last_undegraded":"2023-12-01T01:51:24.188426+0000","last_fullsized":"2023-12-01T01:51:24.188426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:17.602658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217828+0000","last_change":"2023-12-01T01:51:24.216184+0000","last_active":"2023-12-01T01:51:24.217828+0000","last_peered":"2023-12-01T01:51:24.217828+0000","last_clean":"2023-12-01T01:51:24.217828+0000","last_became_active":"2023-12-01T01:51:24.215804+0000","last_became_peered":"2023-12-01T01:51:24.215804+0000","last_unstale":"2023-12-01T01:51:24.217828+0000","last_undegraded":"2023-12-01T01:51:24.217828+0000","last_fullsized":"2023-12-01T01:51:24.217828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187682+0000","last_change":"2023-12-01T01:51:23.733129+0000","last_active":"2023-12-01T01:51:24.187682+0000","last_peered":"2023-12-01T01:51:24.187682+0000","last_clean":"2023-12-01T01:51:24.187682+0000","last_became_active":"2023-12-01T01:51:23.732687+0000","last_became_peered":"2023-12-01T01:51:23.732687+0000","last_unstale":"2023-12-01T01:51:24.187682+0000","last_undegraded":"2023-12-01T01:51:24.187682+0000","last_fullsized":"2023-12-01T01:51:24.187682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:59.339210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218946+0000","last_change":"2023-12-01T01:51:24.213951+0000","last_active":"2023-12-01T01:51:24.218946+0000","last_peered":"2023-12-01T01:51:24.218946+0000","last_clean":"2023-12-01T01:51:24.218946+0000","last_became_active":"2023-12-01T01:51:24.212626+0000","last_became_peered":"2023-12-01T01:51:24.212626+0000","last_unstale":"2023-12-01T01:51:24.218946+0000","last_undegraded":"2023-12-01T01:51:24.218946+0000","last_fullsized":"2023-12-01T01:51:24.218946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:54:45.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"82'3","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205513+0000","last_change":"2023-12-01T01:51:23.723356+0000","last_active":"2023-12-01T01:51:24.205513+0000","last_peered":"2023-12-01T01:51:24.205513+0000","last_clean":"2023-12-01T01:51:24.205513+0000","last_became_active":"2023-12-01T01:51:23.723040+0000","last_became_peered":"2023-12-01T01:51:23.723040+0000","last_unstale":"2023-12-01T01:51:24.205513+0000","last_undegraded":"2023-12-01T01:51:24.205513+0000","last_fullsized":"2023-12-01T01:51:24.205513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:15.500740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.213047+0000","last_change":"2023-12-01T01:51:24.209588+0000","last_active":"2023-12-01T01:51:24.213047+0000","last_peered":"2023-12-01T01:51:24.213047+0000","last_clean":"2023-12-01T01:51:24.213047+0000","last_became_active":"2023-12-01T01:51:24.207327+0000","last_became_peered":"2023-12-01T01:51:24.207327+0000","last_unstale":"2023-12-01T01:51:24.213047+0000","last_undegraded":"2023-12-01T01:51:24.213047+0000","last_fullsized":"2023-12-01T01:51:24.213047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:23.126569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"82'6","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220840+0000","last_change":"2023-12-01T01:51:23.725813+0000","last_active":"2023-12-01T01:51:24.220840+0000","last_peered":"2023-12-01T01:51:24.220840+0000","last_clean":"2023-12-01T01:51:24.220840+0000","last_became_active":"2023-12-01T01:51:23.723452+0000","last_became_peered":"2023-12-01T01:51:23.723452+0000","last_unstale":"2023-12-01T01:51:24.220840+0000","last_undegraded":"2023-12-01T01:51:24.220840+0000","last_fullsized":"2023-12-01T01:51:24.220840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:46.793866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.214150+0000","last_change":"2023-12-01T01:51:24.212037+0000","last_active":"2023-12-01T01:51:24.214150+0000","last_peered":"2023-12-01T01:51:24.214150+0000","last_clean":"2023-12-01T01:51:24.214150+0000","last_became_active":"2023-12-01T01:51:24.211783+0000","last_became_peered":"2023-12-01T01:51:24.211783+0000","last_unstale":"2023-12-01T01:51:24.214150+0000","last_undegraded":"2023-12-01T01:51:24.214150+0000","last_fullsized":"2023-12-01T01:51:24.214150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.469019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215996+0000","last_change":"2023-12-01T01:51:23.732400+0000","last_active":"2023-12-01T01:51:24.215996+0000","last_peered":"2023-12-01T01:51:24.215996+0000","last_clean":"2023-12-01T01:51:24.215996+0000","last_became_active":"2023-12-01T01:51:23.730066+0000","last_became_peered":"2023-12-01T01:51:23.730066+0000","last_unstale":"2023-12-01T01:51:24.215996+0000","last_undegraded":"2023-12-01T01:51:24.215996+0000","last_fullsized":"2023-12-01T01:51:24.215996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:49.297425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211556+0000","last_change":"2023-12-01T01:51:24.209576+0000","last_active":"2023-12-01T01:51:24.211556+0000","last_peered":"2023-12-01T01:51:24.211556+0000","last_clean":"2023-12-01T01:51:24.211556+0000","last_became_active":"2023-12-01T01:51:24.209297+0000","last_became_peered":"2023-12-01T01:51:24.209297+0000","last_unstale":"2023-12-01T01:51:24.211556+0000","last_undegraded":"2023-12-01T01:51:24.211556+0000","last_fullsized":"2023-12-01T01:51:24.211556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:10.334292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.194995+0000","last_change":"2023-12-01T01:51:23.722740+0000","last_active":"2023-12-01T01:51:24.194995+0000","last_peered":"2023-12-01T01:51:24.194995+0000","last_clean":"2023-12-01T01:51:24.194995+0000","last_became_active":"2023-12-01T01:51:23.722478+0000","last_became_peered":"2023-12-01T01:51:23.722478+0000","last_unstale":"2023-12-01T01:51:24.194995+0000","last_undegraded":"2023-12-01T01:51:24.194995+0000","last_fullsized":"2023-12-01T01:51:24.194995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:01:52.071276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.197625+0000","last_change":"2023-12-01T01:51:24.193465+0000","last_active":"2023-12-01T01:51:24.197625+0000","last_peered":"2023-12-01T01:51:24.197625+0000","last_clean":"2023-12-01T01:51:24.197625+0000","last_became_active":"2023-12-01T01:51:24.193193+0000","last_became_peered":"2023-12-01T01:51:24.193193+0000","last_unstale":"2023-12-01T01:51:24.197625+0000","last_undegraded":"2023-12-01T01:51:24.197625+0000","last_fullsized":"2023-12-01T01:51:24.197625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:33.729872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.194040+0000","last_change":"2023-12-01T01:51:23.725460+0000","last_active":"2023-12-01T01:51:24.194040+0000","last_peered":"2023-12-01T01:51:24.194040+0000","last_clean":"2023-12-01T01:51:24.194040+0000","last_became_active":"2023-12-01T01:51:23.725192+0000","last_became_peered":"2023-12-01T01:51:23.725192+0000","last_unstale":"2023-12-01T01:51:24.194040+0000","last_undegraded":"2023-12-01T01:51:24.194040+0000","last_fullsized":"2023-12-01T01:51:24.194040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:14.033061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217980+0000","last_change":"2023-12-01T01:51:24.215107+0000","last_active":"2023-12-01T01:51:24.217980+0000","last_peered":"2023-12-01T01:51:24.217980+0000","last_clean":"2023-12-01T01:51:24.217980+0000","last_became_active":"2023-12-01T01:51:24.214837+0000","last_became_peered":"2023-12-01T01:51:24.214837+0000","last_unstale":"2023-12-01T01:51:24.217980+0000","last_undegraded":"2023-12-01T01:51:24.217980+0000","last_fullsized":"2023-12-01T01:51:24.217980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:49.877816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.210435+0000","last_change":"2023-12-01T01:51:23.728019+0000","last_active":"2023-12-01T01:51:24.210435+0000","last_peered":"2023-12-01T01:51:24.210435+0000","last_clean":"2023-12-01T01:51:24.210435+0000","last_became_active":"2023-12-01T01:51:23.727690+0000","last_became_peered":"2023-12-01T01:51:23.727690+0000","last_unstale":"2023-12-01T01:51:24.210435+0000","last_undegraded":"2023-12-01T01:51:24.210435+0000","last_fullsized":"2023-12-01T01:51:24.210435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:54:44.691969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215141+0000","last_change":"2023-12-01T01:51:24.213421+0000","last_active":"2023-12-01T01:51:24.215141+0000","last_peered":"2023-12-01T01:51:24.215141+0000","last_clean":"2023-12-01T01:51:24.215141+0000","last_became_active":"2023-12-01T01:51:24.213163+0000","last_became_peered":"2023-12-01T01:51:24.213163+0000","last_unstale":"2023-12-01T01:51:24.215141+0000","last_undegraded":"2023-12-01T01:51:24.215141+0000","last_fullsized":"2023-12-01T01:51:24.215141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:14.157429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224213+0000","last_change":"2023-12-01T01:51:23.733218+0000","last_active":"2023-12-01T01:51:24.224213+0000","last_peered":"2023-12-01T01:51:24.224213+0000","last_clean":"2023-12-01T01:51:24.224213+0000","last_became_active":"2023-12-01T01:51:23.732910+0000","last_became_peered":"2023-12-01T01:51:23.732910+0000","last_unstale":"2023-12-01T01:51:24.224213+0000","last_undegraded":"2023-12-01T01:51:24.224213+0000","last_fullsized":"2023-12-01T01:51:24.224213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:54.751334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215104+0000","last_change":"2023-12-01T01:51:24.212578+0000","last_active":"2023-12-01T01:51:24.215104+0000","last_peered":"2023-12-01T01:51:24.215104+0000","last_clean":"2023-12-01T01:51:24.215104+0000","last_became_active":"2023-12-01T01:51:24.212344+0000","last_became_peered":"2023-12-01T01:51:24.212344+0000","last_unstale":"2023-12-01T01:51:24.215104+0000","last_undegraded":"2023-12-01T01:51:24.215104+0000","last_fullsized":"2023-12-01T01:51:24.215104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218291+0000","last_change":"2023-12-01T01:51:23.733533+0000","last_active":"2023-12-01T01:51:24.218291+0000","last_peered":"2023-12-01T01:51:24.218291+0000","last_clean":"2023-12-01T01:51:24.218291+0000","last_became_active":"2023-12-01T01:51:23.733196+0000","last_became_peered":"2023-12-01T01:51:23.733196+0000","last_unstale":"2023-12-01T01:51:24.218291+0000","last_undegraded":"2023-12-01T01:51:24.218291+0000","last_fullsized":"2023-12-01T01:51:24.218291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:09.329899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.222453+0000","last_change":"2023-12-01T01:51:24.213793+0000","last_active":"2023-12-01T01:51:24.222453+0000","last_peered":"2023-12-01T01:51:24.222453+0000","last_clean":"2023-12-01T01:51:24.222453+0000","last_became_active":"2023-12-01T01:51:24.213365+0000","last_became_peered":"2023-12-01T01:51:24.213365+0000","last_unstale":"2023-12-01T01:51:24.222453+0000","last_undegraded":"2023-12-01T01:51:24.222453+0000","last_fullsized":"2023-12-01T01:51:24.222453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:12:27.401881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.222133+0000","last_change":"2023-12-01T01:51:23.728528+0000","last_active":"2023-12-01T01:51:24.222133+0000","last_peered":"2023-12-01T01:51:24.222133+0000","last_clean":"2023-12-01T01:51:24.222133+0000","last_became_active":"2023-12-01T01:51:23.728266+0000","last_became_peered":"2023-12-01T01:51:23.728266+0000","last_unstale":"2023-12-01T01:51:24.222133+0000","last_undegraded":"2023-12-01T01:51:24.222133+0000","last_fullsized":"2023-12-01T01:51:24.222133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:18:10.249852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223947+0000","last_change":"2023-12-01T01:51:24.222537+0000","last_active":"2023-12-01T01:51:24.223947+0000","last_peered":"2023-12-01T01:51:24.223947+0000","last_clean":"2023-12-01T01:51:24.223947+0000","last_became_active":"2023-12-01T01:51:24.222305+0000","last_became_peered":"2023-12-01T01:51:24.222305+0000","last_unstale":"2023-12-01T01:51:24.223947+0000","last_undegraded":"2023-12-01T01:51:24.223947+0000","last_fullsized":"2023-12-01T01:51:24.223947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:03.279418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.189727+0000","last_change":"2023-12-01T01:51:23.733364+0000","last_active":"2023-12-01T01:51:24.189727+0000","last_peered":"2023-12-01T01:51:24.189727+0000","last_clean":"2023-12-01T01:51:24.189727+0000","last_became_active":"2023-12-01T01:51:23.732811+0000","last_became_peered":"2023-12-01T01:51:23.732811+0000","last_unstale":"2023-12-01T01:51:24.189727+0000","last_undegraded":"2023-12-01T01:51:24.189727+0000","last_fullsized":"2023-12-01T01:51:24.189727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:17.784149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218065+0000","last_change":"2023-12-01T01:51:24.216189+0000","last_active":"2023-12-01T01:51:24.218065+0000","last_peered":"2023-12-01T01:51:24.218065+0000","last_clean":"2023-12-01T01:51:24.218065+0000","last_became_active":"2023-12-01T01:51:24.215930+0000","last_became_peered":"2023-12-01T01:51:24.215930+0000","last_unstale":"2023-12-01T01:51:24.218065+0000","last_undegraded":"2023-12-01T01:51:24.218065+0000","last_fullsized":"2023-12-01T01:51:24.218065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:57.588932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.216083+0000","last_change":"2023-12-01T01:51:23.717087+0000","last_active":"2023-12-01T01:51:24.216083+0000","last_peered":"2023-12-01T01:51:24.216083+0000","last_clean":"2023-12-01T01:51:24.216083+0000","last_became_active":"2023-12-01T01:51:23.712593+0000","last_became_peered":"2023-12-01T01:51:23.712593+0000","last_unstale":"2023-12-01T01:51:24.216083+0000","last_undegraded":"2023-12-01T01:51:24.216083+0000","last_fullsized":"2023-12-01T01:51:24.216083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:09.329899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205309+0000","last_change":"2023-12-01T01:51:24.200939+0000","last_active":"2023-12-01T01:51:24.205309+0000","last_peered":"2023-12-01T01:51:24.205309+0000","last_clean":"2023-12-01T01:51:24.205309+0000","last_became_active":"2023-12-01T01:51:24.200488+0000","last_became_peered":"2023-12-01T01:51:24.200488+0000","last_unstale":"2023-12-01T01:51:24.205309+0000","last_undegraded":"2023-12-01T01:51:24.205309+0000","last_fullsized":"2023-12-01T01:51:24.205309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:09.806552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187843+0000","last_change":"2023-12-01T01:51:23.732919+0000","last_active":"2023-12-01T01:51:24.187843+0000","last_peered":"2023-12-01T01:51:24.187843+0000","last_clean":"2023-12-01T01:51:24.187843+0000","last_became_active":"2023-12-01T01:51:23.732477+0000","last_became_peered":"2023-12-01T01:51:23.732477+0000","last_unstale":"2023-12-01T01:51:24.187843+0000","last_undegraded":"2023-12-01T01:51:24.187843+0000","last_fullsized":"2023-12-01T01:51:24.187843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:28.105668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.204853+0000","last_change":"2023-12-01T01:51:24.198835+0000","last_active":"2023-12-01T01:51:24.204853+0000","last_peered":"2023-12-01T01:51:24.204853+0000","last_clean":"2023-12-01T01:51:24.204853+0000","last_became_active":"2023-12-01T01:51:24.198426+0000","last_became_peered":"2023-12-01T01:51:24.198426+0000","last_unstale":"2023-12-01T01:51:24.204853+0000","last_undegraded":"2023-12-01T01:51:24.204853+0000","last_fullsized":"2023-12-01T01:51:24.204853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:55.560831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187402+0000","last_change":"2023-12-01T01:51:23.732268+0000","last_active":"2023-12-01T01:51:24.187402+0000","last_peered":"2023-12-01T01:51:24.187402+0000","last_clean":"2023-12-01T01:51:24.187402+0000","last_became_active":"2023-12-01T01:51:23.730931+0000","last_became_peered":"2023-12-01T01:51:23.730931+0000","last_unstale":"2023-12-01T01:51:24.187402+0000","last_undegraded":"2023-12-01T01:51:24.187402+0000","last_fullsized":"2023-12-01T01:51:24.187402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:43.652247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223795+0000","last_change":"2023-12-01T01:51:24.222725+0000","last_active":"2023-12-01T01:51:24.223795+0000","last_peered":"2023-12-01T01:51:24.223795+0000","last_clean":"2023-12-01T01:51:24.223795+0000","last_became_active":"2023-12-01T01:51:24.222166+0000","last_became_peered":"2023-12-01T01:51:24.222166+0000","last_unstale":"2023-12-01T01:51:24.223795+0000","last_undegraded":"2023-12-01T01:51:24.223795+0000","last_fullsized":"2023-12-01T01:51:24.223795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:55:18.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221810+0000","last_change":"2023-12-01T01:51:23.735050+0000","last_active":"2023-12-01T01:51:24.221810+0000","last_peered":"2023-12-01T01:51:24.221810+0000","last_clean":"2023-12-01T01:51:24.221810+0000","last_became_active":"2023-12-01T01:51:23.734653+0000","last_became_peered":"2023-12-01T01:51:23.734653+0000","last_unstale":"2023-12-01T01:51:24.221810+0000","last_undegraded":"2023-12-01T01:51:24.221810+0000","last_fullsized":"2023-12-01T01:51:24.221810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:06.069833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215432+0000","last_change":"2023-12-01T01:51:24.213857+0000","last_active":"2023-12-01T01:51:24.215432+0000","last_peered":"2023-12-01T01:51:24.215432+0000","last_clean":"2023-12-01T01:51:24.215432+0000","last_became_active":"2023-12-01T01:51:24.213485+0000","last_became_peered":"2023-12-01T01:51:24.213485+0000","last_unstale":"2023-12-01T01:51:24.215432+0000","last_undegraded":"2023-12-01T01:51:24.215432+0000","last_fullsized":"2023-12-01T01:51:24.215432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:34:25.329128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220448+0000","last_change":"2023-12-01T01:51:23.724706+0000","last_active":"2023-12-01T01:51:24.220448+0000","last_peered":"2023-12-01T01:51:24.220448+0000","last_clean":"2023-12-01T01:51:24.220448+0000","last_became_active":"2023-12-01T01:51:23.723272+0000","last_became_peered":"2023-12-01T01:51:23.723272+0000","last_unstale":"2023-12-01T01:51:24.220448+0000","last_undegraded":"2023-12-01T01:51:24.220448+0000","last_fullsized":"2023-12-01T01:51:24.220448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:51.874903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.222668+0000","last_change":"2023-12-01T01:51:24.216311+0000","last_active":"2023-12-01T01:51:24.222668+0000","last_peered":"2023-12-01T01:51:24.222668+0000","last_clean":"2023-12-01T01:51:24.222668+0000","last_became_active":"2023-12-01T01:51:24.213698+0000","last_became_peered":"2023-12-01T01:51:24.213698+0000","last_unstale":"2023-12-01T01:51:24.222668+0000","last_undegraded":"2023-12-01T01:51:24.222668+0000","last_fullsized":"2023-12-01T01:51:24.222668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:47.798259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187078+0000","last_change":"2023-12-01T01:51:23.721602+0000","last_active":"2023-12-01T01:51:24.187078+0000","last_peered":"2023-12-01T01:51:24.187078+0000","last_clean":"2023-12-01T01:51:24.187078+0000","last_became_active":"2023-12-01T01:51:23.721352+0000","last_became_peered":"2023-12-01T01:51:23.721352+0000","last_unstale":"2023-12-01T01:51:24.187078+0000","last_undegraded":"2023-12-01T01:51:24.187078+0000","last_fullsized":"2023-12-01T01:51:24.187078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:44.064139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205130+0000","last_change":"2023-12-01T01:51:24.201302+0000","last_active":"2023-12-01T01:51:24.205130+0000","last_peered":"2023-12-01T01:51:24.205130+0000","last_clean":"2023-12-01T01:51:24.205130+0000","last_became_active":"2023-12-01T01:51:24.200899+0000","last_became_peered":"2023-12-01T01:51:24.200899+0000","last_unstale":"2023-12-01T01:51:24.205130+0000","last_undegraded":"2023-12-01T01:51:24.205130+0000","last_fullsized":"2023-12-01T01:51:24.205130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:13.923540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.201716+0000","last_change":"2023-12-01T01:51:23.732583+0000","last_active":"2023-12-01T01:51:24.201716+0000","last_peered":"2023-12-01T01:51:24.201716+0000","last_clean":"2023-12-01T01:51:24.201716+0000","last_became_active":"2023-12-01T01:51:23.731083+0000","last_became_peered":"2023-12-01T01:51:23.731083+0000","last_unstale":"2023-12-01T01:51:24.201716+0000","last_undegraded":"2023-12-01T01:51:24.201716+0000","last_fullsized":"2023-12-01T01:51:24.201716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:36.702370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224156+0000","last_change":"2023-12-01T01:51:24.220076+0000","last_active":"2023-12-01T01:51:24.224156+0000","last_peered":"2023-12-01T01:51:24.224156+0000","last_clean":"2023-12-01T01:51:24.224156+0000","last_became_active":"2023-12-01T01:51:24.216516+0000","last_became_peered":"2023-12-01T01:51:24.216516+0000","last_unstale":"2023-12-01T01:51:24.224156+0000","last_undegraded":"2023-12-01T01:51:24.224156+0000","last_fullsized":"2023-12-01T01:51:24.224156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:06.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.199761+0000","last_change":"2023-12-01T01:51:23.732282+0000","last_active":"2023-12-01T01:51:24.199761+0000","last_peered":"2023-12-01T01:51:24.199761+0000","last_clean":"2023-12-01T01:51:24.199761+0000","last_became_active":"2023-12-01T01:51:23.732016+0000","last_became_peered":"2023-12-01T01:51:23.732016+0000","last_unstale":"2023-12-01T01:51:24.199761+0000","last_undegraded":"2023-12-01T01:51:24.199761+0000","last_fullsized":"2023-12-01T01:51:24.199761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:50.767646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.210347+0000","last_change":"2023-12-01T01:51:24.204531+0000","last_active":"2023-12-01T01:51:24.210347+0000","last_peered":"2023-12-01T01:51:24.210347+0000","last_clean":"2023-12-01T01:51:24.210347+0000","last_became_active":"2023-12-01T01:51:24.204155+0000","last_became_peered":"2023-12-01T01:51:24.204155+0000","last_unstale":"2023-12-01T01:51:24.210347+0000","last_undegraded":"2023-12-01T01:51:24.210347+0000","last_fullsized":"2023-12-01T01:51:24.210347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:29.110060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.201449+0000","last_change":"2023-12-01T01:51:23.720558+0000","last_active":"2023-12-01T01:51:24.201449+0000","last_peered":"2023-12-01T01:51:24.201449+0000","last_clean":"2023-12-01T01:51:24.201449+0000","last_became_active":"2023-12-01T01:51:23.720288+0000","last_became_peered":"2023-12-01T01:51:23.720288+0000","last_unstale":"2023-12-01T01:51:24.201449+0000","last_undegraded":"2023-12-01T01:51:24.201449+0000","last_fullsized":"2023-12-01T01:51:24.201449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:54:32.273772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215064+0000","last_change":"2023-12-01T01:51:24.209551+0000","last_active":"2023-12-01T01:51:24.215064+0000","last_peered":"2023-12-01T01:51:24.215064+0000","last_clean":"2023-12-01T01:51:24.215064+0000","last_became_active":"2023-12-01T01:51:24.209232+0000","last_became_peered":"2023-12-01T01:51:24.209232+0000","last_unstale":"2023-12-01T01:51:24.215064+0000","last_undegraded":"2023-12-01T01:51:24.215064+0000","last_fullsized":"2023-12-01T01:51:24.215064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:15.037453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.213057+0000","last_change":"2023-12-01T01:51:23.730111+0000","last_active":"2023-12-01T01:51:24.213057+0000","last_peered":"2023-12-01T01:51:24.213057+0000","last_clean":"2023-12-01T01:51:24.213057+0000","last_became_active":"2023-12-01T01:51:23.729813+0000","last_became_peered":"2023-12-01T01:51:23.729813+0000","last_unstale":"2023-12-01T01:51:24.213057+0000","last_undegraded":"2023-12-01T01:51:24.213057+0000","last_fullsized":"2023-12-01T01:51:24.213057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:06.524585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221809+0000","last_change":"2023-12-01T01:51:24.213668+0000","last_active":"2023-12-01T01:51:24.221809+0000","last_peered":"2023-12-01T01:51:24.221809+0000","last_clean":"2023-12-01T01:51:24.221809+0000","last_became_active":"2023-12-01T01:51:24.213225+0000","last_became_peered":"2023-12-01T01:51:24.213225+0000","last_unstale":"2023-12-01T01:51:24.221809+0000","last_undegraded":"2023-12-01T01:51:24.221809+0000","last_fullsized":"2023-12-01T01:51:24.221809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.360859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.188190+0000","last_change":"2023-12-01T01:51:23.731063+0000","last_active":"2023-12-01T01:51:24.188190+0000","last_peered":"2023-12-01T01:51:24.188190+0000","last_clean":"2023-12-01T01:51:24.188190+0000","last_became_active":"2023-12-01T01:51:23.730608+0000","last_became_peered":"2023-12-01T01:51:23.730608+0000","last_unstale":"2023-12-01T01:51:24.188190+0000","last_undegraded":"2023-12-01T01:51:24.188190+0000","last_fullsized":"2023-12-01T01:51:24.188190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:34:24.324736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220336+0000","last_change":"2023-12-01T01:51:24.217170+0000","last_active":"2023-12-01T01:51:24.220336+0000","last_peered":"2023-12-01T01:51:24.220336+0000","last_clean":"2023-12-01T01:51:24.220336+0000","last_became_active":"2023-12-01T01:51:24.215927+0000","last_became_peered":"2023-12-01T01:51:24.215927+0000","last_unstale":"2023-12-01T01:51:24.220336+0000","last_undegraded":"2023-12-01T01:51:24.220336+0000","last_fullsized":"2023-12-01T01:51:24.220336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:31:57.782696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.186053+0000","last_change":"2023-12-01T01:51:23.709986+0000","last_active":"2023-12-01T01:51:24.186053+0000","last_peered":"2023-12-01T01:51:24.186053+0000","last_clean":"2023-12-01T01:51:24.186053+0000","last_became_active":"2023-12-01T01:51:23.709276+0000","last_became_peered":"2023-12-01T01:51:23.709276+0000","last_unstale":"2023-12-01T01:51:24.186053+0000","last_undegraded":"2023-12-01T01:51:24.186053+0000","last_fullsized":"2023-12-01T01:51:24.186053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:06:05.428917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'161","reported_seq":382,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187487+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:51:24.187487+0000","last_peered":"2023-12-01T01:51:24.187487+0000","last_clean":"2023-12-01T01:51:24.187487+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:51:24.187487+0000","last_undegraded":"2023-12-01T01:51:24.187487+0000","last_fullsized":"2023-12-01T01:51:24.187487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":1011712,"data_stored":475605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.79700000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3979999999999999,"5min":1.5549999999999999,"15min":1.8089999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.073,"15min":1.05},"min":{"1min":0.82899999999999996,"5min":0.64700000000000002,"15min":0.64000000000000001},"max":{"1min":1.353,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.246,"15min":1.218},"min":{"1min":1.008,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.802,"5min":1.8100000000000001,"15min":1.8420000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2629999999999999,"15min":1.2569999999999999},"min":{"1min":1.0669999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5529999999999999,"5min":1.744,"15min":2.0870000000000002},"last":1.117}]},{"osd":2,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.121,"15min":1.123},"min":{"1min":0.80100000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5329999999999999,"5min":1.728,"15min":1.728},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.218,"15min":1.2290000000000001},"min":{"1min":0.92400000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.6120000000000001,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.1579999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.876,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.617,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.355,"15min":1.349},"min":{"1min":1.026,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6859999999999999,"5min":2.1880000000000002,"15min":2.7050000000000001},"last":1.254}]},{"osd":4,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.2769999999999999,"15min":1.296},"min":{"1min":0.83099999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.681,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.036},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4610000000000001,"15min":1.4199999999999999},"min":{"1min":1.0609999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7370000000000001,"5min":2.0939999999999999,"15min":2.601},"last":1.349}]},{"osd":5,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.274,"15min":1.27},"min":{"1min":0.83099999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.6439999999999999,"5min":2.0870000000000002,"15min":2.4500000000000002},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5309999999999999,"15min":1.526},"min":{"1min":1,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8839999999999999,"5min":2.2930000000000001,"15min":2.8069999999999999},"last":1.5620000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.393,"15min":1.3859999999999999},"min":{"1min":1.004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.77,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":1.002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7929999999999999,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.3080000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":475535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.101,"15min":1.1060000000000001},"min":{"1min":0.78900000000000003,"5min":0.69699999999999995,"15min":0.66900000000000004},"max":{"1min":1.619,"5min":1.635,"15min":1.79},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.139,"5min":1.137,"15min":1.1299999999999999},"min":{"1min":0.89600000000000002,"5min":0.64400000000000002,"15min":0.61799999999999999},"max":{"1min":1.4690000000000001,"5min":2.5569999999999999,"15min":2.73},"last":1.292}]},{"osd":1,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.167,"15min":1.153},"min":{"1min":0.94999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6759999999999999,"5min":1.8620000000000001,"15min":2.2170000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.196,"15min":1.21},"min":{"1min":0.93400000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6259999999999999,"5min":1.651,"15min":2.0920000000000001},"last":1.149}]},{"osd":2,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.232,"15min":1.163},"min":{"1min":0.92000000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6359999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.27,"15min":1.25},"min":{"1min":0.877,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7549999999999999,"5min":2.137,"15min":2.137},"last":1.5549999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3009999999999999,"15min":1.2849999999999999},"min":{"1min":1.111,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.548,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.4259999999999999,"15min":1.407},"min":{"1min":1.137,"5min":1.085,"15min":0.81299999999999994},"max":{"1min":1.7130000000000001,"5min":1.758,"15min":2.4020000000000001},"last":1.3320000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.321,"15min":1.2909999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.70699999999999996},"max":{"1min":1.6919999999999999,"5min":2.226,"15min":2.3399999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.365,"15min":1.3580000000000001},"min":{"1min":0.94699999999999995,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.694,"5min":1.931,"15min":2.6760000000000002},"last":1.413}]},{"osd":5,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4419999999999999,"15min":1.4179999999999999},"min":{"1min":1.0469999999999999,"5min":1.0229999999999999,"15min":0.82699999999999996},"max":{"1min":1.726,"5min":2.1899999999999999,"15min":2.5539999999999998},"last":1.605},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5109999999999999,"15min":1.482},"min":{"1min":1.1870000000000001,"5min":0.98899999999999999,"15min":0.748},"max":{"1min":1.8129999999999999,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.0389999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5660000000000001,"15min":1.5249999999999999},"min":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":0.89500000000000002},"max":{"1min":1.978,"5min":2.762,"15min":2.762},"last":1.651},{"interface":"front","average":{"1min":1.587,"5min":1.621,"15min":1.585},"min":{"1min":1.218,"5min":1.1479999999999999,"15min":0.78700000000000003},"max":{"1min":1.9259999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.234}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1564,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1601536,"data_stored":1065915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.1060000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.498,"5min":1.651,"15min":3.3980000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.127,"15min":1.141},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.60099999999999998},"max":{"1min":1.5720000000000001,"5min":1.6479999999999999,"15min":2.153},"last":1.5109999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.153,"15min":1.137},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.7010000000000001},"last":1.472},{"interface":"front","average":{"1min":1.179,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.73599999999999999,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.94099999999999995}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4039999999999999,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.093},{"interface":"front","average":{"1min":1.155,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.6080000000000001,"5min":1.621,"15min":2.0630000000000002},"last":1.2090000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.286},"min":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.76900000000000002},"max":{"1min":1.5309999999999999,"5min":1.694,"15min":1.833},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3120000000000001,"15min":1.296},"min":{"1min":0.98099999999999998,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":1.6779999999999999,"5min":1.7110000000000001,"15min":2.1440000000000001},"last":1.663}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.238,"15min":1.2390000000000001},"min":{"1min":0.86399999999999999,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.6779999999999999,"5min":1.853,"15min":2.5129999999999999},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.84999999999999998,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.6319999999999999,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.5900000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2989999999999999,"15min":1.329},"min":{"1min":0.88500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":2.2970000000000002},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":1.125,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":1.962,"5min":1.962,"15min":3.23},"last":1.625}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4530000000000001,"15min":1.48},"min":{"1min":0.92400000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2,"5min":2,"15min":3.012},"last":2.0499999999999998},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4970000000000001,"15min":1.524},"min":{"1min":1.2689999999999999,"5min":0.873,"15min":0.873},"max":{"1min":2.073,"5min":2.073,"15min":2.8420000000000001},"last":2.1299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":1011712,"data_stored":476129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.119,"15min":1.1040000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.53,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.734,"5min":1.827,"15min":3.8069999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2350000000000001,"15min":1.2090000000000001},"min":{"1min":0.84599999999999997,"5min":0.81999999999999995,"15min":0.65900000000000003},"max":{"1min":1.528,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.452},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.131},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.671,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.995}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2110000000000001,"15min":1.196},"min":{"1min":0.83999999999999997,"5min":0.78200000000000003,"15min":0.77600000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3140000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.871,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.5780000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.234},"min":{"1min":0.85399999999999998,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.8220000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.095},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3200000000000001,"15min":1.319},"min":{"1min":0.753,"5min":0.753,"15min":0.70399999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.3010000000000002},"last":1.5149999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.96599999999999997,"5min":0.79200000000000004,"15min":0.61899999999999999},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.1549999999999998},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.2829999999999999,"15min":1.292},"min":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.5189999999999999,"5min":1.8029999999999999,"15min":2.2029999999999998},"last":1.2130000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.94999999999999996,"5min":0.84799999999999998,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.619},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.5029999999999999,"15min":1.4730000000000001},"min":{"1min":1.1000000000000001,"5min":0.96699999999999997,"15min":0.63200000000000001},"max":{"1min":1.7949999999999999,"5min":2.4990000000000001,"15min":2.54},"last":1.6950000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.458,"15min":1.464},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.60599999999999998},"max":{"1min":1.6850000000000001,"5min":1.972,"15min":3.2789999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5720000000000001,"15min":1.5580000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.04,"15min":2.157},"last":1.728}]}]},{"osd":3,"up_from":13,"seq":55834575035,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40224,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331616,"statfs":{"total":96636764160,"available":96595574784,"internally_reserved":0,"allocated":1007616,"data_stored":475603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82899999999999996,"15min":0.85199999999999998},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.307},"max":{"1min":1.135,"5min":1.302,"15min":1.583},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.999,"15min":1.0009999999999999},"min":{"1min":0.52100000000000002,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.3999999999999999,"5min":1.4990000000000001,"15min":1.9770000000000001},"last":1.2230000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.069,"15min":1.0629999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.38900000000000001},"max":{"1min":1.556,"5min":1.571,"15min":2.0779999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.95399999999999996,"15min":0.95199999999999996},"min":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.40600000000000003},"max":{"1min":1.5129999999999999,"5min":1.587,"15min":1.6519999999999999},"last":0.93600000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.016,"15min":1.054},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.436},"max":{"1min":1.286,"5min":1.512,"15min":1.766},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.173,"15min":1.1759999999999999},"min":{"1min":0.63800000000000001,"5min":0.621,"15min":0.48399999999999999},"max":{"1min":1.4870000000000001,"5min":1.647,"15min":1.661},"last":0.97699999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.238,"15min":1.2569999999999999},"min":{"1min":0.96499999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.476,"5min":1.6240000000000001,"15min":2.9350000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.246},"min":{"1min":0.94099999999999995,"5min":0.81000000000000005,"15min":0.73299999999999998},"max":{"1min":1.554,"5min":1.637,"15min":2.2389999999999999},"last":1.427}]},{"osd":5,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.272,"15min":1.302},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.63500000000000001},"max":{"1min":1.5900000000000001,"5min":1.806,"15min":3.476},"last":1.123},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2729999999999999,"15min":1.329},"min":{"1min":0.86099999999999999,"5min":0.82099999999999995,"15min":0.64200000000000002},"max":{"1min":1.464,"5min":1.9039999999999999,"15min":3.181},"last":1.2569999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.88900000000000001,"5min":0.80400000000000005,"15min":0.68300000000000005},"max":{"1min":1.6499999999999999,"5min":1.8580000000000001,"15min":2.532},"last":1.294},{"interface":"front","average":{"1min":1.409,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":1.02,"5min":0.94299999999999995,"15min":0.872},"max":{"1min":1.573,"5min":1.8049999999999999,"15min":3.0249999999999999},"last":1.393}]},{"osd":7,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3919999999999999,"15min":1.4239999999999999},"min":{"1min":1.006,"5min":0.999,"15min":0.81499999999999995},"max":{"1min":1.504,"5min":1.851,"15min":2.1619999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4259999999999999,"15min":1.472},"min":{"1min":1.048,"5min":0.88300000000000001,"15min":0.83099999999999996},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":2.4590000000000001},"last":1.5189999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575035,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":1024000,"data_stored":477237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27100000000000002},"max":{"1min":1.0860000000000001,"5min":1.339,"15min":1.738},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.026,"15min":1.01},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.375},"max":{"1min":1.4159999999999999,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":0.71099999999999997}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96499999999999997,"15min":0.98999999999999999},"min":{"1min":0.54700000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.1779999999999999,"5min":1.45,"15min":1.7130000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0720000000000001,"15min":1.0840000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.504},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.1719999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97499999999999998,"15min":1.0029999999999999},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.377,"5min":1.625,"15min":1.883},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.39600000000000002},"max":{"1min":1.415,"5min":1.613,"15min":2.6080000000000001},"last":1.2789999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.331,"15min":1.3180000000000001},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.75},"max":{"1min":1.7230000000000001,"5min":1.9950000000000001,"15min":2.2829999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.246,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.964},"last":1.4850000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3009999999999999,"15min":1.319},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.647,"5min":1.804,"15min":2.5779999999999998},"last":1.194},{"interface":"front","average":{"1min":1.375,"5min":1.387,"15min":1.375},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.901,"5min":2.048,"15min":3.0099999999999998},"last":1.0620000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.3200000000000001},"min":{"1min":0.84299999999999997,"5min":0.83399999999999996,"15min":0.77500000000000002},"max":{"1min":1.6439999999999999,"5min":1.78,"15min":2.2799999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.766,"15min":2.5089999999999999},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.456,"15min":1.448},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.79400000000000004},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.448},"last":1.331},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7649999999999999,"5min":1.802,"15min":2.5049999999999999},"last":1.5429999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":1564,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":1601536,"data_stored":1066463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85699999999999998,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.53800000000000003,"15min":0.318},"max":{"1min":1.1220000000000001,"5min":1.5049999999999999,"15min":3.056},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.999,"15min":1.012},"min":{"1min":0.75700000000000001,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":1.335,"5min":1.6359999999999999,"15min":3.4129999999999998},"last":1.4570000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.88200000000000001,"15min":0.90100000000000002},"min":{"1min":0.67100000000000004,"5min":0.497,"15min":0.36399999999999999},"max":{"1min":1.1779999999999999,"5min":1.54,"15min":3.3660000000000001},"last":0.877},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.78100000000000003,"5min":0.47899999999999998,"15min":0.39000000000000001},"max":{"1min":1.4890000000000001,"5min":1.492,"15min":3.4980000000000002},"last":0.998}]},{"osd":3,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.21,"15min":1.224},"min":{"1min":1.0860000000000001,"5min":0.51900000000000002,"15min":0.44},"max":{"1min":1.6859999999999999,"5min":1.7290000000000001,"15min":3.464},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.077,"15min":1.0669999999999999},"min":{"1min":0.82599999999999996,"5min":0.499,"15min":0.39700000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":3.0960000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1719999999999999,"15min":1.196},"min":{"1min":1.014,"5min":0.745,"15min":0.63800000000000001},"max":{"1min":1.494,"5min":1.536,"15min":3.7810000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.3240000000000001,"15min":1.3600000000000001},"min":{"1min":1.077,"5min":0.77500000000000002,"15min":0.76600000000000001},"max":{"1min":1.585,"5min":1.8089999999999999,"15min":4.7270000000000003},"last":1.2490000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.296,"15min":1.2909999999999999},"min":{"1min":1.032,"5min":0.90600000000000003,"15min":0.61199999999999999},"max":{"1min":1.5760000000000001,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.681},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.278,"15min":1.3160000000000001},"min":{"1min":1.004,"5min":0.76900000000000002,"15min":0.69199999999999995},"max":{"1min":1.5,"5min":1.825,"15min":3.6669999999999998},"last":1.4259999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.349},"min":{"1min":1.0760000000000001,"5min":0.71099999999999997,"15min":0.68400000000000005},"max":{"1min":1.5549999999999999,"5min":1.7809999999999999,"15min":3.3300000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.343,"15min":1.363},"min":{"1min":1.0649999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.595,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.5389999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.409,"15min":1.4630000000000001},"min":{"1min":1.246,"5min":0.76600000000000001,"15min":0.70599999999999996},"max":{"1min":1.645,"5min":1.8939999999999999,"15min":4.6749999999999998},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.4179999999999999,"15min":1.4299999999999999},"min":{"1min":1.1639999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7589999999999999,"5min":1.992,"15min":3.5379999999999998},"last":1.49}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":1015808,"data_stored":477077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.60699999999999998,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.3520000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.92600000000000005,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4390000000000001,"5min":1.5580000000000001,"15min":1.6910000000000001},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98499999999999999,"15min":1.006},"min":{"1min":0.67600000000000005,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.4690000000000001,"5min":2.157,"15min":2.157},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.113,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.84299999999999997,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.395,"5min":1.728,"15min":2.8159999999999998},"last":1.016}]},{"osd":3,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.58499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.954},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.143,"15min":1.169},"min":{"1min":0.54900000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7070000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1060000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.3089999999999999},"min":{"1min":1.05,"5min":0.78500000000000003,"15min":0.65700000000000003},"max":{"1min":1.5289999999999999,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.214,"15min":1.2629999999999999},"min":{"1min":0.83299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.2829999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.337,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.71899999999999997},"max":{"1min":1.839,"5min":2.351,"15min":2.351},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.306,"15min":1.349},"min":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6299999999999999,"5min":2.2909999999999999,"15min":2.7050000000000001},"last":1.2310000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73099999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.157},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.399},"min":{"1min":1.0129999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.256}]},{"osd":7,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":1.127,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.706,"5min":1.798,"15min":2.8999999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.361,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:51:30.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-01T01:51:30.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.377+0000 7f0a87469640 1 Processor -- start 2023-12-01T01:51:30.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.377+0000 7f0a87469640 1 -- start start 2023-12-01T01:51:30.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.378+0000 7f0a87469640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8015cd20 0x7f0a800d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.378+0000 7f0a87469640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0a800b5d80 0x7f0a800b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.378+0000 7f0a87469640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0a800d1230 0x7f0a800d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.378+0000 7f0a87469640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a800b2160 con 0x7f0a800d1230 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.378+0000 7f0a87469640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a800d1be0 con 0x7f0a800b5d80 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.378+0000 7f0a87469640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800d2a10 con 0x7f0a8015cd20 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a85a06640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0a800d1230 0x7f0a800d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a85a06640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0a800d1230 0x7f0a800d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39250/0 (socket says 172.21.15.115:39250) 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a85a06640 1 -- 172.21.15.115:0/2748557377 learned_addr learned my addr 172.21.15.115:0/2748557377 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a85205640 1 --2- 172.21.15.115:0/2748557377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8015cd20 0x7f0a800d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a84a04640 1 --2- 172.21.15.115:0/2748557377 >> v2:172.21.15.142:3300/0 conn(0x7f0a800b5d80 0x7f0a800b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a85a06640 1 -- 172.21.15.115:0/2748557377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8015cd20 msgr2=0x7f0a800d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a85a06640 1 --2- 172.21.15.115:0/2748557377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8015cd20 0x7f0a800d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a85a06640 1 -- 172.21.15.115:0/2748557377 >> v2:172.21.15.142:3300/0 conn(0x7f0a800b5d80 msgr2=0x7f0a800b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a85a06640 1 --2- 172.21.15.115:0/2748557377 >> v2:172.21.15.142:3300/0 conn(0x7f0a800b5d80 0x7f0a800b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a85a06640 1 -- 172.21.15.115:0/2748557377 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800ce2e0 con 0x7f0a800d1230 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.379+0000 7f0a85205640 1 --2- 172.21.15.115:0/2748557377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8015cd20 0x7f0a800d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.381+0000 7f0a85a06640 1 --2- 172.21.15.115:0/2748557377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 0x7f0a800d16a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0a8008bcc0 tx=0x7f0a7802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3b4873bf38989 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.381+0000 7f0a73fff640 1 -- 172.21.15.115:0/2748557377 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7802c9e0 con 0x7f0a800d1230 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.381+0000 7f0a73fff640 1 -- 172.21.15.115:0/2748557377 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7802cb40 con 0x7f0a800d1230 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.381+0000 7f0a73fff640 1 -- 172.21.15.115:0/2748557377 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a780343f0 con 0x7f0a800d1230 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.382+0000 7f0a87469640 1 -- 172.21.15.115:0/2748557377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 msgr2=0x7f0a800d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.382+0000 7f0a87469640 1 --2- 172.21.15.115:0/2748557377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 0x7f0a800d16a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0a8008bcc0 tx=0x7f0a7802be40 comp rx=0 tx=0).stop 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.382+0000 7f0a87469640 1 -- 172.21.15.115:0/2748557377 shutdown_connections 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.382+0000 7f0a87469640 1 --2- 172.21.15.115:0/2748557377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 0x7f0a800d16a0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.382+0000 7f0a87469640 1 --2- 172.21.15.115:0/2748557377 >> v2:172.21.15.142:3300/0 conn(0x7f0a800b5d80 0x7f0a800b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.382+0000 7f0a87469640 1 --2- 172.21.15.115:0/2748557377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8015cd20 0x7f0a800d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.382+0000 7f0a87469640 1 -- 172.21.15.115:0/2748557377 >> 172.21.15.115:0/2748557377 conn(0x7f0a8006a0e0 msgr2=0x7f0a8006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:30.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.382+0000 7f0a87469640 1 -- 172.21.15.115:0/2748557377 shutdown_connections 2023-12-01T01:51:30.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.382+0000 7f0a87469640 1 -- 172.21.15.115:0/2748557377 wait complete. 2023-12-01T01:51:30.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.384+0000 7f0a87469640 1 Processor -- start 2023-12-01T01:51:30.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.384+0000 7f0a87469640 1 -- start start 2023-12-01T01:51:30.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a87469640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a800b5d80 0x7f0a80154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a87469640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 0x7f0a8015ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a87469640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8015cd20 0x7f0a800c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a87469640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800c76c0 con 0x7f0a800d1230 2023-12-01T01:51:30.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a87469640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800c6ab0 con 0x7f0a800b5d80 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a87469640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a800c6d90 con 0x7f0a8015cd20 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a84a04640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 0x7f0a8015ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a84a04640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 0x7f0a8015ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39264/0 (socket says 172.21.15.115:39264) 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a84a04640 1 -- 172.21.15.115:0/3323432008 learned_addr learned my addr 172.21.15.115:0/3323432008 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a84a04640 1 -- 172.21.15.115:0/3323432008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8015cd20 msgr2=0x7f0a800c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a84a04640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8015cd20 0x7f0a800c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a84a04640 1 -- 172.21.15.115:0/3323432008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a800b5d80 msgr2=0x7f0a80154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a84a04640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a800b5d80 0x7f0a80154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.385+0000 7f0a84a04640 1 -- 172.21.15.115:0/3323432008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a78009b30 con 0x7f0a800d1230 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.386+0000 7f0a84a04640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 0x7f0a8015ff30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400d5c0 tx=0x7f0a7400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.386+0000 7f0a71ffb640 1 -- 172.21.15.115:0/3323432008 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a740125f0 con 0x7f0a800d1230 2023-12-01T01:51:30.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.386+0000 7f0a71ffb640 1 -- 172.21.15.115:0/3323432008 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a74012750 con 0x7f0a800d1230 2023-12-01T01:51:30.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.386+0000 7f0a71ffb640 1 -- 172.21.15.115:0/3323432008 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74012b00 con 0x7f0a800d1230 2023-12-01T01:51:30.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.386+0000 7f0a87469640 1 -- 172.21.15.115:0/3323432008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a800c7070 con 0x7f0a800d1230 2023-12-01T01:51:30.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.386+0000 7f0a87469640 1 -- 172.21.15.115:0/3323432008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a80160470 con 0x7f0a800d1230 2023-12-01T01:51:30.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.387+0000 7f0a87469640 1 -- 172.21.15.115:0/3323432008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8004e890 con 0x7f0a800d1230 2023-12-01T01:51:30.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.388+0000 7f0a71ffb640 1 -- 172.21.15.115:0/3323432008 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0a74012c60 con 0x7f0a800d1230 2023-12-01T01:51:30.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.389+0000 7f0a71ffb640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a6406a550 0x7f0a6406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.390+0000 7f0a71ffb640 1 -- 172.21.15.115:0/3323432008 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a74088f20 con 0x7f0a800d1230 2023-12-01T01:51:30.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.390+0000 7f0a85205640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a6406a550 0x7f0a6406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:30.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.390+0000 7f0a85205640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a6406a550 0x7f0a6406ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a800d1850 tx=0x7f0a7c008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:30.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.394+0000 7f0a71ffb640 1 -- 172.21.15.115:0/3323432008 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0a740543f0 con 0x7f0a800d1230 2023-12-01T01:51:30.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.638+0000 7f0a87469640 1 -- 172.21.15.115:0/3323432008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0a800c7870 con 0x7f0a800d1230 2023-12-01T01:51:30.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.639+0000 7f0a71ffb640 1 -- 172.21.15.115:0/3323432008 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3229 (secure 0 0 0) 0x7f0a800c7870 con 0x7f0a800d1230 2023-12-01T01:51:30.640 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:30.640 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":945,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":83,"num_osds":8,"num_up_osds":8,"osd_up_since":1701394558,"num_in_osds":8,"osd_in_since":1701394549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":271691776,"bytes_avail":772822421504,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":32,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6435}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-12-01T01:51:11.820763+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-01T01:51:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.641+0000 7f0a87469640 1 -- 172.21.15.115:0/3323432008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a6406a550 msgr2=0x7f0a6406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.641+0000 7f0a87469640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a6406a550 0x7f0a6406ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a800d1850 tx=0x7f0a7c008040 comp rx=0 tx=0).stop 2023-12-01T01:51:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.641+0000 7f0a87469640 1 -- 172.21.15.115:0/3323432008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 msgr2=0x7f0a8015ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.641+0000 7f0a87469640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 0x7f0a8015ff30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400d5c0 tx=0x7f0a7400da80 comp rx=0 tx=0).stop 2023-12-01T01:51:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.641+0000 7f0a87469640 1 -- 172.21.15.115:0/3323432008 shutdown_connections 2023-12-01T01:51:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.641+0000 7f0a87469640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a6406a550 0x7f0a6406ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.641+0000 7f0a87469640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8015cd20 0x7f0a800c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.641+0000 7f0a87469640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a800d1230 0x7f0a8015ff30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.641+0000 7f0a87469640 1 --2- 172.21.15.115:0/3323432008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a800b5d80 0x7f0a80154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.642+0000 7f0a87469640 1 -- 172.21.15.115:0/3323432008 >> 172.21.15.115:0/3323432008 conn(0x7f0a8006a0e0 msgr2=0x7f0a8015a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.642+0000 7f0a87469640 1 -- 172.21.15.115:0/3323432008 shutdown_connections 2023-12-01T01:51:30.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.642+0000 7f0a87469640 1 -- 172.21.15.115:0/3323432008 wait complete. 2023-12-01T01:51:30.662 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-01T01:51:30.662 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:51:30.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.867+0000 7f76a1ef3640 1 Processor -- start 2023-12-01T01:51:30.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.867+0000 7f76a1ef3640 1 -- start start 2023-12-01T01:51:30.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.868+0000 7f76a1ef3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f769c0b5d80 0x7f769c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.868+0000 7f76a1ef3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c0ce2e0 con 0x7f769c0d1230 2023-12-01T01:51:30.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.868+0000 7f76a1ef3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c0ce000 con 0x7f769c15cd20 2023-12-01T01:51:30.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.868+0000 7f76a1ef3640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c0b2160 con 0x7f769c0b5d80 2023-12-01T01:51:30.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.868+0000 7f769affd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f769c0b5d80 0x7f769c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:30.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.868+0000 7f769bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f769c0d1230 0x7f769c0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59352/0 (socket says 172.21.15.115:59352) 2023-12-01T01:51:30.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.868+0000 7f769bfff640 1 -- 172.21.15.115:0/3141362750 learned_addr learned my addr 172.21.15.115:0/3141362750 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:30.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.869+0000 7f769a7fc640 1 -- 172.21.15.115:0/3141362750 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083814270 0 0) 0x7f769c0ce2e0 con 0x7f769c0d1230 2023-12-01T01:51:30.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.869+0000 7f769affd640 1 -- 172.21.15.115:0/3141362750 >> v1:172.21.15.142:6789/0 conn(0x7f769c15cd20 legacy=0x7f769c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:30.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.869+0000 7f769affd640 1 -- 172.21.15.115:0/3141362750 >> v1:172.21.15.115:6789/0 conn(0x7f769c0d1230 legacy=0x7f769c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.869+0000 7f769affd640 1 -- 172.21.15.115:0/3141362750 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c0cd3d0 con 0x7f769c0b5d80 2023-12-01T01:51:30.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.870+0000 7f769affd640 1 --2- 172.21.15.115:0/3141362750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 0x7f769c0d24d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f76900097a0 tx=0x7f769002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5882dbc60c4b3a9f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:30.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.870+0000 7f769a7fc640 1 -- 172.21.15.115:0/3141362750 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769002e070 con 0x7f769c0b5d80 2023-12-01T01:51:30.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.870+0000 7f769a7fc640 1 -- 172.21.15.115:0/3141362750 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76900315a0 con 0x7f769c0b5d80 2023-12-01T01:51:30.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.870+0000 7f769a7fc640 1 -- 172.21.15.115:0/3141362750 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690031930 con 0x7f769c0b5d80 2023-12-01T01:51:30.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.871+0000 7f76a1ef3640 1 -- 172.21.15.115:0/3141362750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 msgr2=0x7f769c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.871+0000 7f76a1ef3640 1 --2- 172.21.15.115:0/3141362750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 0x7f769c0d24d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f76900097a0 tx=0x7f769002b840 comp rx=0 tx=0).stop 2023-12-01T01:51:30.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.871+0000 7f76a1ef3640 1 -- 172.21.15.115:0/3141362750 shutdown_connections 2023-12-01T01:51:30.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.871+0000 7f76a1ef3640 1 --2- 172.21.15.115:0/3141362750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 0x7f769c0d24d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.871+0000 7f76a1ef3640 1 -- 172.21.15.115:0/3141362750 >> 172.21.15.115:0/3141362750 conn(0x7f769c06a0e0 msgr2=0x7f769c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:30.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.871+0000 7f76a1ef3640 1 -- 172.21.15.115:0/3141362750 shutdown_connections 2023-12-01T01:51:30.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.871+0000 7f76a1ef3640 1 -- 172.21.15.115:0/3141362750 wait complete. 2023-12-01T01:51:30.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.873+0000 7f76a1ef3640 1 Processor -- start 2023-12-01T01:51:30.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.873+0000 7f76a1ef3640 1 -- start start 2023-12-01T01:51:30.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f76a1ef3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 0x7f769c1f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f76a1ef3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f769c0d1230 0x7f769c0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f76a1ef3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f769c15cd20 0x7f769c0b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f76a1ef3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c1f2520 con 0x7f769c0b5d80 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f76a1ef3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c1f3990 con 0x7f769c0d1230 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f76a1ef3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c1f3380 con 0x7f769c15cd20 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f769b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 0x7f769c1f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f769b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 0x7f769c1f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39272/0 (socket says 172.21.15.115:39272) 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f769b7fe640 1 -- 172.21.15.115:0/1622510788 learned_addr learned my addr 172.21.15.115:0/1622510788 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f769b7fe640 1 -- 172.21.15.115:0/1622510788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f769c15cd20 msgr2=0x7f769c0b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f769b7fe640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f769c15cd20 0x7f769c0b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f769b7fe640 1 -- 172.21.15.115:0/1622510788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f769c0d1230 msgr2=0x7f769c0b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f769b7fe640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f769c0d1230 0x7f769c0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.874+0000 7f769b7fe640 1 -- 172.21.15.115:0/1622510788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009400 con 0x7f769c0b5d80 2023-12-01T01:51:30.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.875+0000 7f769b7fe640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 0x7f769c1f1fe0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f768c00b3f0 tx=0x7f768c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:30.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.875+0000 7f7683fff640 1 -- 172.21.15.115:0/1622510788 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c011830 con 0x7f769c0b5d80 2023-12-01T01:51:30.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.875+0000 7f7683fff640 1 -- 172.21.15.115:0/1622510788 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c011990 con 0x7f769c0b5d80 2023-12-01T01:51:30.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.875+0000 7f7683fff640 1 -- 172.21.15.115:0/1622510788 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c011d40 con 0x7f769c0b5d80 2023-12-01T01:51:30.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.875+0000 7f76a1ef3640 1 -- 172.21.15.115:0/1622510788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c0b3200 con 0x7f769c0b5d80 2023-12-01T01:51:30.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.875+0000 7f76a1ef3640 1 -- 172.21.15.115:0/1622510788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c0b34e0 con 0x7f769c0b5d80 2023-12-01T01:51:30.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.876+0000 7f76a1ef3640 1 -- 172.21.15.115:0/1622510788 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c04e890 con 0x7f769c0b5d80 2023-12-01T01:51:30.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.877+0000 7f7683fff640 1 -- 172.21.15.115:0/1622510788 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f768c01a400 con 0x7f769c0b5d80 2023-12-01T01:51:30.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.878+0000 7f7683fff640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f768806a550 0x7f768806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:30.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.878+0000 7f7683fff640 1 -- 172.21.15.115:0/1622510788 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f768c0889f0 con 0x7f769c0b5d80 2023-12-01T01:51:30.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.878+0000 7f769affd640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f768806a550 0x7f768806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:30.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.879+0000 7f769affd640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f768806a550 0x7f768806ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7690036000 tx=0x7f7690030000 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:30.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:30.882+0000 7f7683fff640 1 -- 172.21.15.115:0/1622510788 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f768c053e30 con 0x7f769c0b5d80 2023-12-01T01:51:31.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.054+0000 7f76a1ef3640 1 -- 172.21.15.115:0/1622510788 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f769c0cd6b0 con 0x7f768806a550 2023-12-01T01:51:31.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.075+0000 7f7683fff640 1 -- 172.21.15.115:0/1622510788 <== mgr.6201 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286247 (secure 0 0 0) 0x7f7690031990 con 0x7f768806a550 2023-12-01T01:51:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:31.082 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:51:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 -- 172.21.15.115:0/1622510788 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f768806a550 msgr2=0x7f768806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f768806a550 0x7f768806ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7690036000 tx=0x7f7690030000 comp rx=0 tx=0).stop 2023-12-01T01:51:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 -- 172.21.15.115:0/1622510788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 msgr2=0x7f769c1f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 0x7f769c1f1fe0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f768c00b3f0 tx=0x7f768c00b8b0 comp rx=0 tx=0).stop 2023-12-01T01:51:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 -- 172.21.15.115:0/1622510788 shutdown_connections 2023-12-01T01:51:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f768806a550 0x7f768806ca10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f769c15cd20 0x7f769c0b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f769c0d1230 0x7f769c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 --2- 172.21.15.115:0/1622510788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0b5d80 0x7f769c1f1fe0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 -- 172.21.15.115:0/1622510788 >> 172.21.15.115:0/1622510788 conn(0x7f769c06a0e0 msgr2=0x7f769c157370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 -- 172.21.15.115:0/1622510788 shutdown_connections 2023-12-01T01:51:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:31.081+0000 7f76a1ef3640 1 -- 172.21.15.115:0/1622510788 wait complete. 2023-12-01T01:51:31.099 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-12-01T01:51:29.825357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269736,"kb_used_data":9064,"kb_used_omap":0,"kb_used_meta":260608,"kb_avail":754704984,"statfs":{"total":773094113280,"available":772817903616,"internally_reserved":0,"allocated":9281536,"data_stored":4989564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":266862592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.128678"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228585+0000","last_change":"2023-12-01T01:51:24.224886+0000","last_active":"2023-12-01T01:51:24.228585+0000","last_peered":"2023-12-01T01:51:24.228585+0000","last_clean":"2023-12-01T01:51:24.228585+0000","last_became_active":"2023-12-01T01:51:24.222251+0000","last_became_peered":"2023-12-01T01:51:24.222251+0000","last_unstale":"2023-12-01T01:51:24.228585+0000","last_undegraded":"2023-12-01T01:51:24.228585+0000","last_fullsized":"2023-12-01T01:51:24.228585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:21.413565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228337+0000","last_change":"2023-12-01T01:51:24.223609+0000","last_active":"2023-12-01T01:51:24.228337+0000","last_peered":"2023-12-01T01:51:24.228337+0000","last_clean":"2023-12-01T01:51:24.228337+0000","last_became_active":"2023-12-01T01:51:24.222770+0000","last_became_peered":"2023-12-01T01:51:24.222770+0000","last_unstale":"2023-12-01T01:51:24.228337+0000","last_undegraded":"2023-12-01T01:51:24.228337+0000","last_fullsized":"2023-12-01T01:51:24.228337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:26.173230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.209949+0000","last_change":"2023-12-01T01:51:24.207820+0000","last_active":"2023-12-01T01:51:24.209949+0000","last_peered":"2023-12-01T01:51:24.209949+0000","last_clean":"2023-12-01T01:51:24.209949+0000","last_became_active":"2023-12-01T01:51:24.207535+0000","last_became_peered":"2023-12-01T01:51:24.207535+0000","last_unstale":"2023-12-01T01:51:24.209949+0000","last_undegraded":"2023-12-01T01:51:24.209949+0000","last_fullsized":"2023-12-01T01:51:24.209949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:50.301817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223930+0000","last_change":"2023-12-01T01:51:24.222179+0000","last_active":"2023-12-01T01:51:24.223930+0000","last_peered":"2023-12-01T01:51:24.223930+0000","last_clean":"2023-12-01T01:51:24.223930+0000","last_became_active":"2023-12-01T01:51:24.221936+0000","last_became_peered":"2023-12-01T01:51:24.221936+0000","last_unstale":"2023-12-01T01:51:24.223930+0000","last_undegraded":"2023-12-01T01:51:24.223930+0000","last_fullsized":"2023-12-01T01:51:24.223930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:03:53.147122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224064+0000","last_change":"2023-12-01T01:51:24.219076+0000","last_active":"2023-12-01T01:51:24.224064+0000","last_peered":"2023-12-01T01:51:24.224064+0000","last_clean":"2023-12-01T01:51:24.224064+0000","last_became_active":"2023-12-01T01:51:24.218809+0000","last_became_peered":"2023-12-01T01:51:24.218809+0000","last_unstale":"2023-12-01T01:51:24.224064+0000","last_undegraded":"2023-12-01T01:51:24.224064+0000","last_fullsized":"2023-12-01T01:51:24.224064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:16.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224174+0000","last_change":"2023-12-01T01:51:24.221005+0000","last_active":"2023-12-01T01:51:24.224174+0000","last_peered":"2023-12-01T01:51:24.224174+0000","last_clean":"2023-12-01T01:51:24.224174+0000","last_became_active":"2023-12-01T01:51:24.220737+0000","last_became_peered":"2023-12-01T01:51:24.220737+0000","last_unstale":"2023-12-01T01:51:24.224174+0000","last_undegraded":"2023-12-01T01:51:24.224174+0000","last_fullsized":"2023-12-01T01:51:24.224174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:08.979742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.212243+0000","last_change":"2023-12-01T01:51:24.205047+0000","last_active":"2023-12-01T01:51:24.212243+0000","last_peered":"2023-12-01T01:51:24.212243+0000","last_clean":"2023-12-01T01:51:24.212243+0000","last_became_active":"2023-12-01T01:51:24.204779+0000","last_became_peered":"2023-12-01T01:51:24.204779+0000","last_unstale":"2023-12-01T01:51:24.212243+0000","last_undegraded":"2023-12-01T01:51:24.212243+0000","last_fullsized":"2023-12-01T01:51:24.212243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:41.379891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.204987+0000","last_change":"2023-12-01T01:51:24.199691+0000","last_active":"2023-12-01T01:51:24.204987+0000","last_peered":"2023-12-01T01:51:24.204987+0000","last_clean":"2023-12-01T01:51:24.204987+0000","last_became_active":"2023-12-01T01:51:24.199343+0000","last_became_peered":"2023-12-01T01:51:24.199343+0000","last_unstale":"2023-12-01T01:51:24.204987+0000","last_undegraded":"2023-12-01T01:51:24.204987+0000","last_fullsized":"2023-12-01T01:51:24.204987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:56:56.216365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223463+0000","last_change":"2023-12-01T01:51:24.221268+0000","last_active":"2023-12-01T01:51:24.223463+0000","last_peered":"2023-12-01T01:51:24.223463+0000","last_clean":"2023-12-01T01:51:24.223463+0000","last_became_active":"2023-12-01T01:51:24.220971+0000","last_became_peered":"2023-12-01T01:51:24.220971+0000","last_unstale":"2023-12-01T01:51:24.223463+0000","last_undegraded":"2023-12-01T01:51:24.223463+0000","last_fullsized":"2023-12-01T01:51:24.223463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:55.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225919+0000","last_change":"2023-12-01T01:51:24.222276+0000","last_active":"2023-12-01T01:51:24.225919+0000","last_peered":"2023-12-01T01:51:24.225919+0000","last_clean":"2023-12-01T01:51:24.225919+0000","last_became_active":"2023-12-01T01:51:24.220691+0000","last_became_peered":"2023-12-01T01:51:24.220691+0000","last_unstale":"2023-12-01T01:51:24.225919+0000","last_undegraded":"2023-12-01T01:51:24.225919+0000","last_fullsized":"2023-12-01T01:51:24.225919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:52.689385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225715+0000","last_change":"2023-12-01T01:51:24.221827+0000","last_active":"2023-12-01T01:51:24.225715+0000","last_peered":"2023-12-01T01:51:24.225715+0000","last_clean":"2023-12-01T01:51:24.225715+0000","last_became_active":"2023-12-01T01:51:24.221267+0000","last_became_peered":"2023-12-01T01:51:24.221267+0000","last_unstale":"2023-12-01T01:51:24.225715+0000","last_undegraded":"2023-12-01T01:51:24.225715+0000","last_fullsized":"2023-12-01T01:51:24.225715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:21.413565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.197650+0000","last_change":"2023-12-01T01:51:24.193709+0000","last_active":"2023-12-01T01:51:24.197650+0000","last_peered":"2023-12-01T01:51:24.197650+0000","last_clean":"2023-12-01T01:51:24.197650+0000","last_became_active":"2023-12-01T01:51:24.193253+0000","last_became_peered":"2023-12-01T01:51:24.193253+0000","last_unstale":"2023-12-01T01:51:24.197650+0000","last_undegraded":"2023-12-01T01:51:24.197650+0000","last_fullsized":"2023-12-01T01:51:24.197650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:32.142316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217097+0000","last_change":"2023-12-01T01:51:24.211498+0000","last_active":"2023-12-01T01:51:24.217097+0000","last_peered":"2023-12-01T01:51:24.217097+0000","last_clean":"2023-12-01T01:51:24.217097+0000","last_became_active":"2023-12-01T01:51:24.211147+0000","last_became_peered":"2023-12-01T01:51:24.211147+0000","last_unstale":"2023-12-01T01:51:24.217097+0000","last_undegraded":"2023-12-01T01:51:24.217097+0000","last_fullsized":"2023-12-01T01:51:24.217097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:01.177699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211052+0000","last_change":"2023-12-01T01:51:24.208945+0000","last_active":"2023-12-01T01:51:24.211052+0000","last_peered":"2023-12-01T01:51:24.211052+0000","last_clean":"2023-12-01T01:51:24.211052+0000","last_became_active":"2023-12-01T01:51:24.208667+0000","last_became_peered":"2023-12-01T01:51:24.208667+0000","last_unstale":"2023-12-01T01:51:24.211052+0000","last_undegraded":"2023-12-01T01:51:24.211052+0000","last_fullsized":"2023-12-01T01:51:24.211052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:29.110060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223068+0000","last_change":"2023-12-01T01:51:24.221816+0000","last_active":"2023-12-01T01:51:24.223068+0000","last_peered":"2023-12-01T01:51:24.223068+0000","last_clean":"2023-12-01T01:51:24.223068+0000","last_became_active":"2023-12-01T01:51:24.221513+0000","last_became_peered":"2023-12-01T01:51:24.221513+0000","last_unstale":"2023-12-01T01:51:24.223068+0000","last_undegraded":"2023-12-01T01:51:24.223068+0000","last_fullsized":"2023-12-01T01:51:24.223068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.219844+0000","last_change":"2023-12-01T01:51:24.212262+0000","last_active":"2023-12-01T01:51:24.219844+0000","last_peered":"2023-12-01T01:51:24.219844+0000","last_clean":"2023-12-01T01:51:24.219844+0000","last_became_active":"2023-12-01T01:51:24.211984+0000","last_became_peered":"2023-12-01T01:51:24.211984+0000","last_unstale":"2023-12-01T01:51:24.219844+0000","last_undegraded":"2023-12-01T01:51:24.219844+0000","last_fullsized":"2023-12-01T01:51:24.219844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:28.427573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228598+0000","last_change":"2023-12-01T01:51:24.225154+0000","last_active":"2023-12-01T01:51:24.228598+0000","last_peered":"2023-12-01T01:51:24.228598+0000","last_clean":"2023-12-01T01:51:24.228598+0000","last_became_active":"2023-12-01T01:51:24.224886+0000","last_became_peered":"2023-12-01T01:51:24.224886+0000","last_unstale":"2023-12-01T01:51:24.228598+0000","last_undegraded":"2023-12-01T01:51:24.228598+0000","last_fullsized":"2023-12-01T01:51:24.228598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223814+0000","last_change":"2023-12-01T01:51:24.222850+0000","last_active":"2023-12-01T01:51:24.223814+0000","last_peered":"2023-12-01T01:51:24.223814+0000","last_clean":"2023-12-01T01:51:24.223814+0000","last_became_active":"2023-12-01T01:51:24.222475+0000","last_became_peered":"2023-12-01T01:51:24.222475+0000","last_unstale":"2023-12-01T01:51:24.223814+0000","last_undegraded":"2023-12-01T01:51:24.223814+0000","last_fullsized":"2023-12-01T01:51:24.223814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:27.535710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225788+0000","last_change":"2023-12-01T01:51:24.220362+0000","last_active":"2023-12-01T01:51:24.225788+0000","last_peered":"2023-12-01T01:51:24.225788+0000","last_clean":"2023-12-01T01:51:24.225788+0000","last_became_active":"2023-12-01T01:51:24.219702+0000","last_became_peered":"2023-12-01T01:51:24.219702+0000","last_unstale":"2023-12-01T01:51:24.225788+0000","last_undegraded":"2023-12-01T01:51:24.225788+0000","last_fullsized":"2023-12-01T01:51:24.225788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:14.157429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217404+0000","last_change":"2023-12-01T01:51:24.211275+0000","last_active":"2023-12-01T01:51:24.217404+0000","last_peered":"2023-12-01T01:51:24.217404+0000","last_clean":"2023-12-01T01:51:24.217404+0000","last_became_active":"2023-12-01T01:51:24.210964+0000","last_became_peered":"2023-12-01T01:51:24.210964+0000","last_unstale":"2023-12-01T01:51:24.217404+0000","last_undegraded":"2023-12-01T01:51:24.217404+0000","last_fullsized":"2023-12-01T01:51:24.217404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:08.979742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.212510+0000","last_change":"2023-12-01T01:51:24.206715+0000","last_active":"2023-12-01T01:51:24.212510+0000","last_peered":"2023-12-01T01:51:24.212510+0000","last_clean":"2023-12-01T01:51:24.212510+0000","last_became_active":"2023-12-01T01:51:24.206432+0000","last_became_peered":"2023-12-01T01:51:24.206432+0000","last_unstale":"2023-12-01T01:51:24.212510+0000","last_undegraded":"2023-12-01T01:51:24.212510+0000","last_fullsized":"2023-12-01T01:51:24.212510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217430+0000","last_change":"2023-12-01T01:51:24.212833+0000","last_active":"2023-12-01T01:51:24.217430+0000","last_peered":"2023-12-01T01:51:24.217430+0000","last_clean":"2023-12-01T01:51:24.217430+0000","last_became_active":"2023-12-01T01:51:24.212494+0000","last_became_peered":"2023-12-01T01:51:24.212494+0000","last_unstale":"2023-12-01T01:51:24.217430+0000","last_undegraded":"2023-12-01T01:51:24.217430+0000","last_fullsized":"2023-12-01T01:51:24.217430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:12.784193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217998+0000","last_change":"2023-12-01T01:51:24.215195+0000","last_active":"2023-12-01T01:51:24.217998+0000","last_peered":"2023-12-01T01:51:24.217998+0000","last_clean":"2023-12-01T01:51:24.217998+0000","last_became_active":"2023-12-01T01:51:24.214924+0000","last_became_peered":"2023-12-01T01:51:24.214924+0000","last_unstale":"2023-12-01T01:51:24.217998+0000","last_undegraded":"2023-12-01T01:51:24.217998+0000","last_fullsized":"2023-12-01T01:51:24.217998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:39:57.857405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211682+0000","last_change":"2023-12-01T01:51:24.210975+0000","last_active":"2023-12-01T01:51:24.211682+0000","last_peered":"2023-12-01T01:51:24.211682+0000","last_clean":"2023-12-01T01:51:24.211682+0000","last_became_active":"2023-12-01T01:51:24.210687+0000","last_became_peered":"2023-12-01T01:51:24.210687+0000","last_unstale":"2023-12-01T01:51:24.211682+0000","last_undegraded":"2023-12-01T01:51:24.211682+0000","last_fullsized":"2023-12-01T01:51:24.211682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:51.772038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225706+0000","last_change":"2023-12-01T01:51:24.221053+0000","last_active":"2023-12-01T01:51:24.225706+0000","last_peered":"2023-12-01T01:51:24.225706+0000","last_clean":"2023-12-01T01:51:24.225706+0000","last_became_active":"2023-12-01T01:51:24.219646+0000","last_became_peered":"2023-12-01T01:51:24.219646+0000","last_unstale":"2023-12-01T01:51:24.225706+0000","last_undegraded":"2023-12-01T01:51:24.225706+0000","last_fullsized":"2023-12-01T01:51:24.225706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:06:06.433309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.200813+0000","last_change":"2023-12-01T01:51:24.197255+0000","last_active":"2023-12-01T01:51:24.200813+0000","last_peered":"2023-12-01T01:51:24.200813+0000","last_clean":"2023-12-01T01:51:24.200813+0000","last_became_active":"2023-12-01T01:51:24.197005+0000","last_became_peered":"2023-12-01T01:51:24.197005+0000","last_unstale":"2023-12-01T01:51:24.200813+0000","last_undegraded":"2023-12-01T01:51:24.200813+0000","last_fullsized":"2023-12-01T01:51:24.200813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:10.334292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225808+0000","last_change":"2023-12-01T01:51:24.220582+0000","last_active":"2023-12-01T01:51:24.225808+0000","last_peered":"2023-12-01T01:51:24.225808+0000","last_clean":"2023-12-01T01:51:24.225808+0000","last_became_active":"2023-12-01T01:51:24.219677+0000","last_became_peered":"2023-12-01T01:51:24.219677+0000","last_unstale":"2023-12-01T01:51:24.225808+0000","last_undegraded":"2023-12-01T01:51:24.225808+0000","last_fullsized":"2023-12-01T01:51:24.225808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:10:04.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.203074+0000","last_change":"2023-12-01T01:51:24.198223+0000","last_active":"2023-12-01T01:51:24.203074+0000","last_peered":"2023-12-01T01:51:24.203074+0000","last_clean":"2023-12-01T01:51:24.203074+0000","last_became_active":"2023-12-01T01:51:24.197651+0000","last_became_peered":"2023-12-01T01:51:24.197651+0000","last_unstale":"2023-12-01T01:51:24.203074+0000","last_undegraded":"2023-12-01T01:51:24.203074+0000","last_fullsized":"2023-12-01T01:51:24.203074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:46.030193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.208866+0000","last_change":"2023-12-01T01:51:24.205197+0000","last_active":"2023-12-01T01:51:24.208866+0000","last_peered":"2023-12-01T01:51:24.208866+0000","last_clean":"2023-12-01T01:51:24.208866+0000","last_became_active":"2023-12-01T01:51:24.204894+0000","last_became_peered":"2023-12-01T01:51:24.204894+0000","last_unstale":"2023-12-01T01:51:24.208866+0000","last_undegraded":"2023-12-01T01:51:24.208866+0000","last_fullsized":"2023-12-01T01:51:24.208866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:49.877816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.197827+0000","last_change":"2023-12-01T01:51:24.196324+0000","last_active":"2023-12-01T01:51:24.197827+0000","last_peered":"2023-12-01T01:51:24.197827+0000","last_clean":"2023-12-01T01:51:24.197827+0000","last_became_active":"2023-12-01T01:51:24.194764+0000","last_became_peered":"2023-12-01T01:51:24.194764+0000","last_unstale":"2023-12-01T01:51:24.197827+0000","last_undegraded":"2023-12-01T01:51:24.197827+0000","last_fullsized":"2023-12-01T01:51:24.197827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:28.026095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221789+0000","last_change":"2023-12-01T01:51:24.213551+0000","last_active":"2023-12-01T01:51:24.221789+0000","last_peered":"2023-12-01T01:51:24.221789+0000","last_clean":"2023-12-01T01:51:24.221789+0000","last_became_active":"2023-12-01T01:51:24.212904+0000","last_became_peered":"2023-12-01T01:51:24.212904+0000","last_unstale":"2023-12-01T01:51:24.221789+0000","last_undegraded":"2023-12-01T01:51:24.221789+0000","last_fullsized":"2023-12-01T01:51:24.221789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:52.879295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217559+0000","last_change":"2023-12-01T01:51:24.212738+0000","last_active":"2023-12-01T01:51:24.217559+0000","last_peered":"2023-12-01T01:51:24.217559+0000","last_clean":"2023-12-01T01:51:24.217559+0000","last_became_active":"2023-12-01T01:51:24.212433+0000","last_became_peered":"2023-12-01T01:51:24.212433+0000","last_unstale":"2023-12-01T01:51:24.217559+0000","last_undegraded":"2023-12-01T01:51:24.217559+0000","last_fullsized":"2023-12-01T01:51:24.217559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:16.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220228+0000","last_change":"2023-12-01T01:51:24.211814+0000","last_active":"2023-12-01T01:51:24.220228+0000","last_peered":"2023-12-01T01:51:24.220228+0000","last_clean":"2023-12-01T01:51:24.220228+0000","last_became_active":"2023-12-01T01:51:24.211542+0000","last_became_peered":"2023-12-01T01:51:24.211542+0000","last_unstale":"2023-12-01T01:51:24.220228+0000","last_undegraded":"2023-12-01T01:51:24.220228+0000","last_fullsized":"2023-12-01T01:51:24.220228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:01:44.563857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"82'4","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218866+0000","last_change":"2023-12-01T01:51:23.722626+0000","last_active":"2023-12-01T01:51:24.218866+0000","last_peered":"2023-12-01T01:51:24.218866+0000","last_clean":"2023-12-01T01:51:24.218866+0000","last_became_active":"2023-12-01T01:51:23.722272+0000","last_became_peered":"2023-12-01T01:51:23.722272+0000","last_unstale":"2023-12-01T01:51:24.218866+0000","last_undegraded":"2023-12-01T01:51:24.218866+0000","last_fullsized":"2023-12-01T01:51:24.218866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:49.297425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223115+0000","last_change":"2023-12-01T01:51:24.218268+0000","last_active":"2023-12-01T01:51:24.223115+0000","last_peered":"2023-12-01T01:51:24.223115+0000","last_clean":"2023-12-01T01:51:24.223115+0000","last_became_active":"2023-12-01T01:51:24.217888+0000","last_became_peered":"2023-12-01T01:51:24.217888+0000","last_unstale":"2023-12-01T01:51:24.223115+0000","last_undegraded":"2023-12-01T01:51:24.223115+0000","last_fullsized":"2023-12-01T01:51:24.223115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:12.784193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.219239+0000","last_change":"2023-12-01T01:51:23.730769+0000","last_active":"2023-12-01T01:51:24.219239+0000","last_peered":"2023-12-01T01:51:24.219239+0000","last_clean":"2023-12-01T01:51:24.219239+0000","last_became_active":"2023-12-01T01:51:23.730468+0000","last_became_peered":"2023-12-01T01:51:23.730468+0000","last_unstale":"2023-12-01T01:51:24.219239+0000","last_undegraded":"2023-12-01T01:51:24.219239+0000","last_fullsized":"2023-12-01T01:51:24.219239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:56.584539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218966+0000","last_change":"2023-12-01T01:51:24.214183+0000","last_active":"2023-12-01T01:51:24.218966+0000","last_peered":"2023-12-01T01:51:24.218966+0000","last_clean":"2023-12-01T01:51:24.218966+0000","last_became_active":"2023-12-01T01:51:24.213872+0000","last_became_peered":"2023-12-01T01:51:24.213872+0000","last_unstale":"2023-12-01T01:51:24.218966+0000","last_undegraded":"2023-12-01T01:51:24.218966+0000","last_fullsized":"2023-12-01T01:51:24.218966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:53:01.496510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228599+0000","last_change":"2023-12-01T01:51:23.729205+0000","last_active":"2023-12-01T01:51:24.228599+0000","last_peered":"2023-12-01T01:51:24.228599+0000","last_clean":"2023-12-01T01:51:24.228599+0000","last_became_active":"2023-12-01T01:51:23.728920+0000","last_became_peered":"2023-12-01T01:51:23.728920+0000","last_unstale":"2023-12-01T01:51:24.228599+0000","last_undegraded":"2023-12-01T01:51:24.228599+0000","last_fullsized":"2023-12-01T01:51:24.228599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:21:58.877785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.226012+0000","last_change":"2023-12-01T01:51:24.223234+0000","last_active":"2023-12-01T01:51:24.226012+0000","last_peered":"2023-12-01T01:51:24.226012+0000","last_clean":"2023-12-01T01:51:24.226012+0000","last_became_active":"2023-12-01T01:51:24.222152+0000","last_became_peered":"2023-12-01T01:51:24.222152+0000","last_unstale":"2023-12-01T01:51:24.226012+0000","last_undegraded":"2023-12-01T01:51:24.226012+0000","last_fullsized":"2023-12-01T01:51:24.226012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:01.177699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205222+0000","last_change":"2023-12-01T01:51:23.729190+0000","last_active":"2023-12-01T01:51:24.205222+0000","last_peered":"2023-12-01T01:51:24.205222+0000","last_clean":"2023-12-01T01:51:24.205222+0000","last_became_active":"2023-12-01T01:51:23.728867+0000","last_became_peered":"2023-12-01T01:51:23.728867+0000","last_unstale":"2023-12-01T01:51:24.205222+0000","last_undegraded":"2023-12-01T01:51:24.205222+0000","last_fullsized":"2023-12-01T01:51:24.205222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:43:10.966577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.209477+0000","last_change":"2023-12-01T01:51:24.205594+0000","last_active":"2023-12-01T01:51:24.209477+0000","last_peered":"2023-12-01T01:51:24.209477+0000","last_clean":"2023-12-01T01:51:24.209477+0000","last_became_active":"2023-12-01T01:51:24.205042+0000","last_became_peered":"2023-12-01T01:51:24.205042+0000","last_unstale":"2023-12-01T01:51:24.209477+0000","last_undegraded":"2023-12-01T01:51:24.209477+0000","last_fullsized":"2023-12-01T01:51:24.209477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:01:53.075668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211402+0000","last_change":"2023-12-01T01:51:23.717746+0000","last_active":"2023-12-01T01:51:24.211402+0000","last_peered":"2023-12-01T01:51:24.211402+0000","last_clean":"2023-12-01T01:51:24.211402+0000","last_became_active":"2023-12-01T01:51:23.715918+0000","last_became_peered":"2023-12-01T01:51:23.715918+0000","last_unstale":"2023-12-01T01:51:24.211402+0000","last_undegraded":"2023-12-01T01:51:24.211402+0000","last_fullsized":"2023-12-01T01:51:24.211402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:35:50.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.219334+0000","last_change":"2023-12-01T01:51:24.214836+0000","last_active":"2023-12-01T01:51:24.219334+0000","last_peered":"2023-12-01T01:51:24.219334+0000","last_clean":"2023-12-01T01:51:24.219334+0000","last_became_active":"2023-12-01T01:51:24.214055+0000","last_became_peered":"2023-12-01T01:51:24.214055+0000","last_unstale":"2023-12-01T01:51:24.219334+0000","last_undegraded":"2023-12-01T01:51:24.219334+0000","last_fullsized":"2023-12-01T01:51:24.219334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:06.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221430+0000","last_change":"2023-12-01T01:51:23.725607+0000","last_active":"2023-12-01T01:51:24.221430+0000","last_peered":"2023-12-01T01:51:24.221430+0000","last_clean":"2023-12-01T01:51:24.221430+0000","last_became_active":"2023-12-01T01:51:23.722833+0000","last_became_peered":"2023-12-01T01:51:23.722833+0000","last_unstale":"2023-12-01T01:51:24.221430+0000","last_undegraded":"2023-12-01T01:51:24.221430+0000","last_fullsized":"2023-12-01T01:51:24.221430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:10:03.788374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.214615+0000","last_change":"2023-12-01T01:51:24.211805+0000","last_active":"2023-12-01T01:51:24.214615+0000","last_peered":"2023-12-01T01:51:24.214615+0000","last_clean":"2023-12-01T01:51:24.214615+0000","last_became_active":"2023-12-01T01:51:24.211532+0000","last_became_peered":"2023-12-01T01:51:24.211532+0000","last_unstale":"2023-12-01T01:51:24.214615+0000","last_undegraded":"2023-12-01T01:51:24.214615+0000","last_fullsized":"2023-12-01T01:51:24.214615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:51.144418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.186975+0000","last_change":"2023-12-01T01:51:23.729822+0000","last_active":"2023-12-01T01:51:24.186975+0000","last_peered":"2023-12-01T01:51:24.186975+0000","last_clean":"2023-12-01T01:51:24.186975+0000","last_became_active":"2023-12-01T01:51:23.728147+0000","last_became_peered":"2023-12-01T01:51:23.728147+0000","last_unstale":"2023-12-01T01:51:24.186975+0000","last_undegraded":"2023-12-01T01:51:24.186975+0000","last_fullsized":"2023-12-01T01:51:24.186975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:43:10.667895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225883+0000","last_change":"2023-12-01T01:51:24.222272+0000","last_active":"2023-12-01T01:51:24.225883+0000","last_peered":"2023-12-01T01:51:24.225883+0000","last_clean":"2023-12-01T01:51:24.225883+0000","last_became_active":"2023-12-01T01:51:24.221990+0000","last_became_peered":"2023-12-01T01:51:24.221990+0000","last_unstale":"2023-12-01T01:51:24.225883+0000","last_undegraded":"2023-12-01T01:51:24.225883+0000","last_fullsized":"2023-12-01T01:51:24.225883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187467+0000","last_change":"2023-12-01T01:51:23.710030+0000","last_active":"2023-12-01T01:51:24.187467+0000","last_peered":"2023-12-01T01:51:24.187467+0000","last_clean":"2023-12-01T01:51:24.187467+0000","last_became_active":"2023-12-01T01:51:23.709381+0000","last_became_peered":"2023-12-01T01:51:23.709381+0000","last_unstale":"2023-12-01T01:51:24.187467+0000","last_undegraded":"2023-12-01T01:51:24.187467+0000","last_fullsized":"2023-12-01T01:51:24.187467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:25.168837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.207004+0000","last_change":"2023-12-01T01:51:24.200936+0000","last_active":"2023-12-01T01:51:24.207004+0000","last_peered":"2023-12-01T01:51:24.207004+0000","last_clean":"2023-12-01T01:51:24.207004+0000","last_became_active":"2023-12-01T01:51:24.198684+0000","last_became_peered":"2023-12-01T01:51:24.198684+0000","last_unstale":"2023-12-01T01:51:24.207004+0000","last_undegraded":"2023-12-01T01:51:24.207004+0000","last_fullsized":"2023-12-01T01:51:24.207004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:50.301817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.188426+0000","last_change":"2023-12-01T01:51:23.725370+0000","last_active":"2023-12-01T01:51:24.188426+0000","last_peered":"2023-12-01T01:51:24.188426+0000","last_clean":"2023-12-01T01:51:24.188426+0000","last_became_active":"2023-12-01T01:51:23.725113+0000","last_became_peered":"2023-12-01T01:51:23.725113+0000","last_unstale":"2023-12-01T01:51:24.188426+0000","last_undegraded":"2023-12-01T01:51:24.188426+0000","last_fullsized":"2023-12-01T01:51:24.188426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:17.602658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217828+0000","last_change":"2023-12-01T01:51:24.216184+0000","last_active":"2023-12-01T01:51:24.217828+0000","last_peered":"2023-12-01T01:51:24.217828+0000","last_clean":"2023-12-01T01:51:24.217828+0000","last_became_active":"2023-12-01T01:51:24.215804+0000","last_became_peered":"2023-12-01T01:51:24.215804+0000","last_unstale":"2023-12-01T01:51:24.217828+0000","last_undegraded":"2023-12-01T01:51:24.217828+0000","last_fullsized":"2023-12-01T01:51:24.217828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187682+0000","last_change":"2023-12-01T01:51:23.733129+0000","last_active":"2023-12-01T01:51:24.187682+0000","last_peered":"2023-12-01T01:51:24.187682+0000","last_clean":"2023-12-01T01:51:24.187682+0000","last_became_active":"2023-12-01T01:51:23.732687+0000","last_became_peered":"2023-12-01T01:51:23.732687+0000","last_unstale":"2023-12-01T01:51:24.187682+0000","last_undegraded":"2023-12-01T01:51:24.187682+0000","last_fullsized":"2023-12-01T01:51:24.187682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:59.339210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218946+0000","last_change":"2023-12-01T01:51:24.213951+0000","last_active":"2023-12-01T01:51:24.218946+0000","last_peered":"2023-12-01T01:51:24.218946+0000","last_clean":"2023-12-01T01:51:24.218946+0000","last_became_active":"2023-12-01T01:51:24.212626+0000","last_became_peered":"2023-12-01T01:51:24.212626+0000","last_unstale":"2023-12-01T01:51:24.218946+0000","last_undegraded":"2023-12-01T01:51:24.218946+0000","last_fullsized":"2023-12-01T01:51:24.218946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:54:45.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"82'3","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205513+0000","last_change":"2023-12-01T01:51:23.723356+0000","last_active":"2023-12-01T01:51:24.205513+0000","last_peered":"2023-12-01T01:51:24.205513+0000","last_clean":"2023-12-01T01:51:24.205513+0000","last_became_active":"2023-12-01T01:51:23.723040+0000","last_became_peered":"2023-12-01T01:51:23.723040+0000","last_unstale":"2023-12-01T01:51:24.205513+0000","last_undegraded":"2023-12-01T01:51:24.205513+0000","last_fullsized":"2023-12-01T01:51:24.205513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:15.500740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.213047+0000","last_change":"2023-12-01T01:51:24.209588+0000","last_active":"2023-12-01T01:51:24.213047+0000","last_peered":"2023-12-01T01:51:24.213047+0000","last_clean":"2023-12-01T01:51:24.213047+0000","last_became_active":"2023-12-01T01:51:24.207327+0000","last_became_peered":"2023-12-01T01:51:24.207327+0000","last_unstale":"2023-12-01T01:51:24.213047+0000","last_undegraded":"2023-12-01T01:51:24.213047+0000","last_fullsized":"2023-12-01T01:51:24.213047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:23.126569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"82'6","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220840+0000","last_change":"2023-12-01T01:51:23.725813+0000","last_active":"2023-12-01T01:51:24.220840+0000","last_peered":"2023-12-01T01:51:24.220840+0000","last_clean":"2023-12-01T01:51:24.220840+0000","last_became_active":"2023-12-01T01:51:23.723452+0000","last_became_peered":"2023-12-01T01:51:23.723452+0000","last_unstale":"2023-12-01T01:51:24.220840+0000","last_undegraded":"2023-12-01T01:51:24.220840+0000","last_fullsized":"2023-12-01T01:51:24.220840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:46.793866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.214150+0000","last_change":"2023-12-01T01:51:24.212037+0000","last_active":"2023-12-01T01:51:24.214150+0000","last_peered":"2023-12-01T01:51:24.214150+0000","last_clean":"2023-12-01T01:51:24.214150+0000","last_became_active":"2023-12-01T01:51:24.211783+0000","last_became_peered":"2023-12-01T01:51:24.211783+0000","last_unstale":"2023-12-01T01:51:24.214150+0000","last_undegraded":"2023-12-01T01:51:24.214150+0000","last_fullsized":"2023-12-01T01:51:24.214150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.469019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215996+0000","last_change":"2023-12-01T01:51:23.732400+0000","last_active":"2023-12-01T01:51:24.215996+0000","last_peered":"2023-12-01T01:51:24.215996+0000","last_clean":"2023-12-01T01:51:24.215996+0000","last_became_active":"2023-12-01T01:51:23.730066+0000","last_became_peered":"2023-12-01T01:51:23.730066+0000","last_unstale":"2023-12-01T01:51:24.215996+0000","last_undegraded":"2023-12-01T01:51:24.215996+0000","last_fullsized":"2023-12-01T01:51:24.215996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:49.297425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211556+0000","last_change":"2023-12-01T01:51:24.209576+0000","last_active":"2023-12-01T01:51:24.211556+0000","last_peered":"2023-12-01T01:51:24.211556+0000","last_clean":"2023-12-01T01:51:24.211556+0000","last_became_active":"2023-12-01T01:51:24.209297+0000","last_became_peered":"2023-12-01T01:51:24.209297+0000","last_unstale":"2023-12-01T01:51:24.211556+0000","last_undegraded":"2023-12-01T01:51:24.211556+0000","last_fullsized":"2023-12-01T01:51:24.211556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:10.334292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.194995+0000","last_change":"2023-12-01T01:51:23.722740+0000","last_active":"2023-12-01T01:51:24.194995+0000","last_peered":"2023-12-01T01:51:24.194995+0000","last_clean":"2023-12-01T01:51:24.194995+0000","last_became_active":"2023-12-01T01:51:23.722478+0000","last_became_peered":"2023-12-01T01:51:23.722478+0000","last_unstale":"2023-12-01T01:51:24.194995+0000","last_undegraded":"2023-12-01T01:51:24.194995+0000","last_fullsized":"2023-12-01T01:51:24.194995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:01:52.071276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.197625+0000","last_change":"2023-12-01T01:51:24.193465+0000","last_active":"2023-12-01T01:51:24.197625+0000","last_peered":"2023-12-01T01:51:24.197625+0000","last_clean":"2023-12-01T01:51:24.197625+0000","last_became_active":"2023-12-01T01:51:24.193193+0000","last_became_peered":"2023-12-01T01:51:24.193193+0000","last_unstale":"2023-12-01T01:51:24.197625+0000","last_undegraded":"2023-12-01T01:51:24.197625+0000","last_fullsized":"2023-12-01T01:51:24.197625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:33.729872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.194040+0000","last_change":"2023-12-01T01:51:23.725460+0000","last_active":"2023-12-01T01:51:24.194040+0000","last_peered":"2023-12-01T01:51:24.194040+0000","last_clean":"2023-12-01T01:51:24.194040+0000","last_became_active":"2023-12-01T01:51:23.725192+0000","last_became_peered":"2023-12-01T01:51:23.725192+0000","last_unstale":"2023-12-01T01:51:24.194040+0000","last_undegraded":"2023-12-01T01:51:24.194040+0000","last_fullsized":"2023-12-01T01:51:24.194040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:14.033061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217980+0000","last_change":"2023-12-01T01:51:24.215107+0000","last_active":"2023-12-01T01:51:24.217980+0000","last_peered":"2023-12-01T01:51:24.217980+0000","last_clean":"2023-12-01T01:51:24.217980+0000","last_became_active":"2023-12-01T01:51:24.214837+0000","last_became_peered":"2023-12-01T01:51:24.214837+0000","last_unstale":"2023-12-01T01:51:24.217980+0000","last_undegraded":"2023-12-01T01:51:24.217980+0000","last_fullsized":"2023-12-01T01:51:24.217980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:49.877816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.210435+0000","last_change":"2023-12-01T01:51:23.728019+0000","last_active":"2023-12-01T01:51:24.210435+0000","last_peered":"2023-12-01T01:51:24.210435+0000","last_clean":"2023-12-01T01:51:24.210435+0000","last_became_active":"2023-12-01T01:51:23.727690+0000","last_became_peered":"2023-12-01T01:51:23.727690+0000","last_unstale":"2023-12-01T01:51:24.210435+0000","last_undegraded":"2023-12-01T01:51:24.210435+0000","last_fullsized":"2023-12-01T01:51:24.210435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:54:44.691969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215141+0000","last_change":"2023-12-01T01:51:24.213421+0000","last_active":"2023-12-01T01:51:24.215141+0000","last_peered":"2023-12-01T01:51:24.215141+0000","last_clean":"2023-12-01T01:51:24.215141+0000","last_became_active":"2023-12-01T01:51:24.213163+0000","last_became_peered":"2023-12-01T01:51:24.213163+0000","last_unstale":"2023-12-01T01:51:24.215141+0000","last_undegraded":"2023-12-01T01:51:24.215141+0000","last_fullsized":"2023-12-01T01:51:24.215141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:14.157429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224213+0000","last_change":"2023-12-01T01:51:23.733218+0000","last_active":"2023-12-01T01:51:24.224213+0000","last_peered":"2023-12-01T01:51:24.224213+0000","last_clean":"2023-12-01T01:51:24.224213+0000","last_became_active":"2023-12-01T01:51:23.732910+0000","last_became_peered":"2023-12-01T01:51:23.732910+0000","last_unstale":"2023-12-01T01:51:24.224213+0000","last_undegraded":"2023-12-01T01:51:24.224213+0000","last_fullsized":"2023-12-01T01:51:24.224213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:54.751334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215104+0000","last_change":"2023-12-01T01:51:24.212578+0000","last_active":"2023-12-01T01:51:24.215104+0000","last_peered":"2023-12-01T01:51:24.215104+0000","last_clean":"2023-12-01T01:51:24.215104+0000","last_became_active":"2023-12-01T01:51:24.212344+0000","last_became_peered":"2023-12-01T01:51:24.212344+0000","last_unstale":"2023-12-01T01:51:24.215104+0000","last_undegraded":"2023-12-01T01:51:24.215104+0000","last_fullsized":"2023-12-01T01:51:24.215104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218291+0000","last_change":"2023-12-01T01:51:23.733533+0000","last_active":"2023-12-01T01:51:24.218291+0000","last_peered":"2023-12-01T01:51:24.218291+0000","last_clean":"2023-12-01T01:51:24.218291+0000","last_became_active":"2023-12-01T01:51:23.733196+0000","last_became_peered":"2023-12-01T01:51:23.733196+0000","last_unstale":"2023-12-01T01:51:24.218291+0000","last_undegraded":"2023-12-01T01:51:24.218291+0000","last_fullsized":"2023-12-01T01:51:24.218291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:09.329899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.222453+0000","last_change":"2023-12-01T01:51:24.213793+0000","last_active":"2023-12-01T01:51:24.222453+0000","last_peered":"2023-12-01T01:51:24.222453+0000","last_clean":"2023-12-01T01:51:24.222453+0000","last_became_active":"2023-12-01T01:51:24.213365+0000","last_became_peered":"2023-12-01T01:51:24.213365+0000","last_unstale":"2023-12-01T01:51:24.222453+0000","last_undegraded":"2023-12-01T01:51:24.222453+0000","last_fullsized":"2023-12-01T01:51:24.222453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:12:27.401881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.222133+0000","last_change":"2023-12-01T01:51:23.728528+0000","last_active":"2023-12-01T01:51:24.222133+0000","last_peered":"2023-12-01T01:51:24.222133+0000","last_clean":"2023-12-01T01:51:24.222133+0000","last_became_active":"2023-12-01T01:51:23.728266+0000","last_became_peered":"2023-12-01T01:51:23.728266+0000","last_unstale":"2023-12-01T01:51:24.222133+0000","last_undegraded":"2023-12-01T01:51:24.222133+0000","last_fullsized":"2023-12-01T01:51:24.222133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:18:10.249852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223947+0000","last_change":"2023-12-01T01:51:24.222537+0000","last_active":"2023-12-01T01:51:24.223947+0000","last_peered":"2023-12-01T01:51:24.223947+0000","last_clean":"2023-12-01T01:51:24.223947+0000","last_became_active":"2023-12-01T01:51:24.222305+0000","last_became_peered":"2023-12-01T01:51:24.222305+0000","last_unstale":"2023-12-01T01:51:24.223947+0000","last_undegraded":"2023-12-01T01:51:24.223947+0000","last_fullsized":"2023-12-01T01:51:24.223947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:03.279418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.189727+0000","last_change":"2023-12-01T01:51:23.733364+0000","last_active":"2023-12-01T01:51:24.189727+0000","last_peered":"2023-12-01T01:51:24.189727+0000","last_clean":"2023-12-01T01:51:24.189727+0000","last_became_active":"2023-12-01T01:51:23.732811+0000","last_became_peered":"2023-12-01T01:51:23.732811+0000","last_unstale":"2023-12-01T01:51:24.189727+0000","last_undegraded":"2023-12-01T01:51:24.189727+0000","last_fullsized":"2023-12-01T01:51:24.189727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:17.784149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218065+0000","last_change":"2023-12-01T01:51:24.216189+0000","last_active":"2023-12-01T01:51:24.218065+0000","last_peered":"2023-12-01T01:51:24.218065+0000","last_clean":"2023-12-01T01:51:24.218065+0000","last_became_active":"2023-12-01T01:51:24.215930+0000","last_became_peered":"2023-12-01T01:51:24.215930+0000","last_unstale":"2023-12-01T01:51:24.218065+0000","last_undegraded":"2023-12-01T01:51:24.218065+0000","last_fullsized":"2023-12-01T01:51:24.218065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:57.588932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.216083+0000","last_change":"2023-12-01T01:51:23.717087+0000","last_active":"2023-12-01T01:51:24.216083+0000","last_peered":"2023-12-01T01:51:24.216083+0000","last_clean":"2023-12-01T01:51:24.216083+0000","last_became_active":"2023-12-01T01:51:23.712593+0000","last_became_peered":"2023-12-01T01:51:23.712593+0000","last_unstale":"2023-12-01T01:51:24.216083+0000","last_undegraded":"2023-12-01T01:51:24.216083+0000","last_fullsized":"2023-12-01T01:51:24.216083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:09.329899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205309+0000","last_change":"2023-12-01T01:51:24.200939+0000","last_active":"2023-12-01T01:51:24.205309+0000","last_peered":"2023-12-01T01:51:24.205309+0000","last_clean":"2023-12-01T01:51:24.205309+0000","last_became_active":"2023-12-01T01:51:24.200488+0000","last_became_peered":"2023-12-01T01:51:24.200488+0000","last_unstale":"2023-12-01T01:51:24.205309+0000","last_undegraded":"2023-12-01T01:51:24.205309+0000","last_fullsized":"2023-12-01T01:51:24.205309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:09.806552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187843+0000","last_change":"2023-12-01T01:51:23.732919+0000","last_active":"2023-12-01T01:51:24.187843+0000","last_peered":"2023-12-01T01:51:24.187843+0000","last_clean":"2023-12-01T01:51:24.187843+0000","last_became_active":"2023-12-01T01:51:23.732477+0000","last_became_peered":"2023-12-01T01:51:23.732477+0000","last_unstale":"2023-12-01T01:51:24.187843+0000","last_undegraded":"2023-12-01T01:51:24.187843+0000","last_fullsized":"2023-12-01T01:51:24.187843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:28.105668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.204853+0000","last_change":"2023-12-01T01:51:24.198835+0000","last_active":"2023-12-01T01:51:24.204853+0000","last_peered":"2023-12-01T01:51:24.204853+0000","last_clean":"2023-12-01T01:51:24.204853+0000","last_became_active":"2023-12-01T01:51:24.198426+0000","last_became_peered":"2023-12-01T01:51:24.198426+0000","last_unstale":"2023-12-01T01:51:24.204853+0000","last_undegraded":"2023-12-01T01:51:24.204853+0000","last_fullsized":"2023-12-01T01:51:24.204853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:55.560831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187402+0000","last_change":"2023-12-01T01:51:23.732268+0000","last_active":"2023-12-01T01:51:24.187402+0000","last_peered":"2023-12-01T01:51:24.187402+0000","last_clean":"2023-12-01T01:51:24.187402+0000","last_became_active":"2023-12-01T01:51:23.730931+0000","last_became_peered":"2023-12-01T01:51:23.730931+0000","last_unstale":"2023-12-01T01:51:24.187402+0000","last_undegraded":"2023-12-01T01:51:24.187402+0000","last_fullsized":"2023-12-01T01:51:24.187402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:43.652247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223795+0000","last_change":"2023-12-01T01:51:24.222725+0000","last_active":"2023-12-01T01:51:24.223795+0000","last_peered":"2023-12-01T01:51:24.223795+0000","last_clean":"2023-12-01T01:51:24.223795+0000","last_became_active":"2023-12-01T01:51:24.222166+0000","last_became_peered":"2023-12-01T01:51:24.222166+0000","last_unstale":"2023-12-01T01:51:24.223795+0000","last_undegraded":"2023-12-01T01:51:24.223795+0000","last_fullsized":"2023-12-01T01:51:24.223795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:55:18.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221810+0000","last_change":"2023-12-01T01:51:23.735050+0000","last_active":"2023-12-01T01:51:24.221810+0000","last_peered":"2023-12-01T01:51:24.221810+0000","last_clean":"2023-12-01T01:51:24.221810+0000","last_became_active":"2023-12-01T01:51:23.734653+0000","last_became_peered":"2023-12-01T01:51:23.734653+0000","last_unstale":"2023-12-01T01:51:24.221810+0000","last_undegraded":"2023-12-01T01:51:24.221810+0000","last_fullsized":"2023-12-01T01:51:24.221810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:06.069833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215432+0000","last_change":"2023-12-01T01:51:24.213857+0000","last_active":"2023-12-01T01:51:24.215432+0000","last_peered":"2023-12-01T01:51:24.215432+0000","last_clean":"2023-12-01T01:51:24.215432+0000","last_became_active":"2023-12-01T01:51:24.213485+0000","last_became_peered":"2023-12-01T01:51:24.213485+0000","last_unstale":"2023-12-01T01:51:24.215432+0000","last_undegraded":"2023-12-01T01:51:24.215432+0000","last_fullsized":"2023-12-01T01:51:24.215432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:34:25.329128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220448+0000","last_change":"2023-12-01T01:51:23.724706+0000","last_active":"2023-12-01T01:51:24.220448+0000","last_peered":"2023-12-01T01:51:24.220448+0000","last_clean":"2023-12-01T01:51:24.220448+0000","last_became_active":"2023-12-01T01:51:23.723272+0000","last_became_peered":"2023-12-01T01:51:23.723272+0000","last_unstale":"2023-12-01T01:51:24.220448+0000","last_undegraded":"2023-12-01T01:51:24.220448+0000","last_fullsized":"2023-12-01T01:51:24.220448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:51.874903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.222668+0000","last_change":"2023-12-01T01:51:24.216311+0000","last_active":"2023-12-01T01:51:24.222668+0000","last_peered":"2023-12-01T01:51:24.222668+0000","last_clean":"2023-12-01T01:51:24.222668+0000","last_became_active":"2023-12-01T01:51:24.213698+0000","last_became_peered":"2023-12-01T01:51:24.213698+0000","last_unstale":"2023-12-01T01:51:24.222668+0000","last_undegraded":"2023-12-01T01:51:24.222668+0000","last_fullsized":"2023-12-01T01:51:24.222668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:47.798259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187078+0000","last_change":"2023-12-01T01:51:23.721602+0000","last_active":"2023-12-01T01:51:24.187078+0000","last_peered":"2023-12-01T01:51:24.187078+0000","last_clean":"2023-12-01T01:51:24.187078+0000","last_became_active":"2023-12-01T01:51:23.721352+0000","last_became_peered":"2023-12-01T01:51:23.721352+0000","last_unstale":"2023-12-01T01:51:24.187078+0000","last_undegraded":"2023-12-01T01:51:24.187078+0000","last_fullsized":"2023-12-01T01:51:24.187078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:44.064139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205130+0000","last_change":"2023-12-01T01:51:24.201302+0000","last_active":"2023-12-01T01:51:24.205130+0000","last_peered":"2023-12-01T01:51:24.205130+0000","last_clean":"2023-12-01T01:51:24.205130+0000","last_became_active":"2023-12-01T01:51:24.200899+0000","last_became_peered":"2023-12-01T01:51:24.200899+0000","last_unstale":"2023-12-01T01:51:24.205130+0000","last_undegraded":"2023-12-01T01:51:24.205130+0000","last_fullsized":"2023-12-01T01:51:24.205130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:13.923540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.201716+0000","last_change":"2023-12-01T01:51:23.732583+0000","last_active":"2023-12-01T01:51:24.201716+0000","last_peered":"2023-12-01T01:51:24.201716+0000","last_clean":"2023-12-01T01:51:24.201716+0000","last_became_active":"2023-12-01T01:51:23.731083+0000","last_became_peered":"2023-12-01T01:51:23.731083+0000","last_unstale":"2023-12-01T01:51:24.201716+0000","last_undegraded":"2023-12-01T01:51:24.201716+0000","last_fullsized":"2023-12-01T01:51:24.201716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:36.702370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224156+0000","last_change":"2023-12-01T01:51:24.220076+0000","last_active":"2023-12-01T01:51:24.224156+0000","last_peered":"2023-12-01T01:51:24.224156+0000","last_clean":"2023-12-01T01:51:24.224156+0000","last_became_active":"2023-12-01T01:51:24.216516+0000","last_became_peered":"2023-12-01T01:51:24.216516+0000","last_unstale":"2023-12-01T01:51:24.224156+0000","last_undegraded":"2023-12-01T01:51:24.224156+0000","last_fullsized":"2023-12-01T01:51:24.224156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:06.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.199761+0000","last_change":"2023-12-01T01:51:23.732282+0000","last_active":"2023-12-01T01:51:24.199761+0000","last_peered":"2023-12-01T01:51:24.199761+0000","last_clean":"2023-12-01T01:51:24.199761+0000","last_became_active":"2023-12-01T01:51:23.732016+0000","last_became_peered":"2023-12-01T01:51:23.732016+0000","last_unstale":"2023-12-01T01:51:24.199761+0000","last_undegraded":"2023-12-01T01:51:24.199761+0000","last_fullsized":"2023-12-01T01:51:24.199761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:50.767646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.210347+0000","last_change":"2023-12-01T01:51:24.204531+0000","last_active":"2023-12-01T01:51:24.210347+0000","last_peered":"2023-12-01T01:51:24.210347+0000","last_clean":"2023-12-01T01:51:24.210347+0000","last_became_active":"2023-12-01T01:51:24.204155+0000","last_became_peered":"2023-12-01T01:51:24.204155+0000","last_unstale":"2023-12-01T01:51:24.210347+0000","last_undegraded":"2023-12-01T01:51:24.210347+0000","last_fullsized":"2023-12-01T01:51:24.210347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:29.110060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.201449+0000","last_change":"2023-12-01T01:51:23.720558+0000","last_active":"2023-12-01T01:51:24.201449+0000","last_peered":"2023-12-01T01:51:24.201449+0000","last_clean":"2023-12-01T01:51:24.201449+0000","last_became_active":"2023-12-01T01:51:23.720288+0000","last_became_peered":"2023-12-01T01:51:23.720288+0000","last_unstale":"2023-12-01T01:51:24.201449+0000","last_undegraded":"2023-12-01T01:51:24.201449+0000","last_fullsized":"2023-12-01T01:51:24.201449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:54:32.273772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215064+0000","last_change":"2023-12-01T01:51:24.209551+0000","last_active":"2023-12-01T01:51:24.215064+0000","last_peered":"2023-12-01T01:51:24.215064+0000","last_clean":"2023-12-01T01:51:24.215064+0000","last_became_active":"2023-12-01T01:51:24.209232+0000","last_became_peered":"2023-12-01T01:51:24.209232+0000","last_unstale":"2023-12-01T01:51:24.215064+0000","last_undegraded":"2023-12-01T01:51:24.215064+0000","last_fullsized":"2023-12-01T01:51:24.215064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:15.037453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.213057+0000","last_change":"2023-12-01T01:51:23.730111+0000","last_active":"2023-12-01T01:51:24.213057+0000","last_peered":"2023-12-01T01:51:24.213057+0000","last_clean":"2023-12-01T01:51:24.213057+0000","last_became_active":"2023-12-01T01:51:23.729813+0000","last_became_peered":"2023-12-01T01:51:23.729813+0000","last_unstale":"2023-12-01T01:51:24.213057+0000","last_undegraded":"2023-12-01T01:51:24.213057+0000","last_fullsized":"2023-12-01T01:51:24.213057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:06.524585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221809+0000","last_change":"2023-12-01T01:51:24.213668+0000","last_active":"2023-12-01T01:51:24.221809+0000","last_peered":"2023-12-01T01:51:24.221809+0000","last_clean":"2023-12-01T01:51:24.221809+0000","last_became_active":"2023-12-01T01:51:24.213225+0000","last_became_peered":"2023-12-01T01:51:24.213225+0000","last_unstale":"2023-12-01T01:51:24.221809+0000","last_undegraded":"2023-12-01T01:51:24.221809+0000","last_fullsized":"2023-12-01T01:51:24.221809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.360859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.188190+0000","last_change":"2023-12-01T01:51:23.731063+0000","last_active":"2023-12-01T01:51:24.188190+0000","last_peered":"2023-12-01T01:51:24.188190+0000","last_clean":"2023-12-01T01:51:24.188190+0000","last_became_active":"2023-12-01T01:51:23.730608+0000","last_became_peered":"2023-12-01T01:51:23.730608+0000","last_unstale":"2023-12-01T01:51:24.188190+0000","last_undegraded":"2023-12-01T01:51:24.188190+0000","last_fullsized":"2023-12-01T01:51:24.188190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:34:24.324736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220336+0000","last_change":"2023-12-01T01:51:24.217170+0000","last_active":"2023-12-01T01:51:24.220336+0000","last_peered":"2023-12-01T01:51:24.220336+0000","last_clean":"2023-12-01T01:51:24.220336+0000","last_became_active":"2023-12-01T01:51:24.215927+0000","last_became_peered":"2023-12-01T01:51:24.215927+0000","last_unstale":"2023-12-01T01:51:24.220336+0000","last_undegraded":"2023-12-01T01:51:24.220336+0000","last_fullsized":"2023-12-01T01:51:24.220336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:31:57.782696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.186053+0000","last_change":"2023-12-01T01:51:23.709986+0000","last_active":"2023-12-01T01:51:24.186053+0000","last_peered":"2023-12-01T01:51:24.186053+0000","last_clean":"2023-12-01T01:51:24.186053+0000","last_became_active":"2023-12-01T01:51:23.709276+0000","last_became_peered":"2023-12-01T01:51:23.709276+0000","last_unstale":"2023-12-01T01:51:24.186053+0000","last_undegraded":"2023-12-01T01:51:24.186053+0000","last_fullsized":"2023-12-01T01:51:24.186053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:06:05.428917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'161","reported_seq":382,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187487+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:51:24.187487+0000","last_peered":"2023-12-01T01:51:24.187487+0000","last_clean":"2023-12-01T01:51:24.187487+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:51:24.187487+0000","last_undegraded":"2023-12-01T01:51:24.187487+0000","last_fullsized":"2023-12-01T01:51:24.187487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":1011712,"data_stored":475605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.79700000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3979999999999999,"5min":1.5549999999999999,"15min":1.8089999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.073,"15min":1.05},"min":{"1min":0.82899999999999996,"5min":0.64700000000000002,"15min":0.64000000000000001},"max":{"1min":1.353,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.246,"15min":1.218},"min":{"1min":1.008,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.802,"5min":1.8100000000000001,"15min":1.8420000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2629999999999999,"15min":1.2569999999999999},"min":{"1min":1.0669999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5529999999999999,"5min":1.744,"15min":2.0870000000000002},"last":1.117}]},{"osd":2,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.121,"15min":1.123},"min":{"1min":0.80100000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5329999999999999,"5min":1.728,"15min":1.728},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.218,"15min":1.2290000000000001},"min":{"1min":0.92400000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.6120000000000001,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.1579999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.876,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.617,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.355,"15min":1.349},"min":{"1min":1.026,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6859999999999999,"5min":2.1880000000000002,"15min":2.7050000000000001},"last":1.254}]},{"osd":4,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.2769999999999999,"15min":1.296},"min":{"1min":0.83099999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.681,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.036},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4610000000000001,"15min":1.4199999999999999},"min":{"1min":1.0609999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7370000000000001,"5min":2.0939999999999999,"15min":2.601},"last":1.349}]},{"osd":5,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.274,"15min":1.27},"min":{"1min":0.83099999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.6439999999999999,"5min":2.0870000000000002,"15min":2.4500000000000002},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5309999999999999,"15min":1.526},"min":{"1min":1,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8839999999999999,"5min":2.2930000000000001,"15min":2.8069999999999999},"last":1.5620000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.393,"15min":1.3859999999999999},"min":{"1min":1.004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.77,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":1.002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7929999999999999,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.3080000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":475535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.101,"15min":1.1060000000000001},"min":{"1min":0.78900000000000003,"5min":0.69699999999999995,"15min":0.66900000000000004},"max":{"1min":1.619,"5min":1.635,"15min":1.79},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.139,"5min":1.137,"15min":1.1299999999999999},"min":{"1min":0.89600000000000002,"5min":0.64400000000000002,"15min":0.61799999999999999},"max":{"1min":1.4690000000000001,"5min":2.5569999999999999,"15min":2.73},"last":1.292}]},{"osd":1,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.167,"15min":1.153},"min":{"1min":0.94999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6759999999999999,"5min":1.8620000000000001,"15min":2.2170000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.196,"15min":1.21},"min":{"1min":0.93400000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6259999999999999,"5min":1.651,"15min":2.0920000000000001},"last":1.149}]},{"osd":2,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.232,"15min":1.163},"min":{"1min":0.92000000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6359999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.27,"15min":1.25},"min":{"1min":0.877,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7549999999999999,"5min":2.137,"15min":2.137},"last":1.5549999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3009999999999999,"15min":1.2849999999999999},"min":{"1min":1.111,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.548,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.4259999999999999,"15min":1.407},"min":{"1min":1.137,"5min":1.085,"15min":0.81299999999999994},"max":{"1min":1.7130000000000001,"5min":1.758,"15min":2.4020000000000001},"last":1.3320000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.321,"15min":1.2909999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.70699999999999996},"max":{"1min":1.6919999999999999,"5min":2.226,"15min":2.3399999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.365,"15min":1.3580000000000001},"min":{"1min":0.94699999999999995,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.694,"5min":1.931,"15min":2.6760000000000002},"last":1.413}]},{"osd":5,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4419999999999999,"15min":1.4179999999999999},"min":{"1min":1.0469999999999999,"5min":1.0229999999999999,"15min":0.82699999999999996},"max":{"1min":1.726,"5min":2.1899999999999999,"15min":2.5539999999999998},"last":1.605},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5109999999999999,"15min":1.482},"min":{"1min":1.1870000000000001,"5min":0.98899999999999999,"15min":0.748},"max":{"1min":1.8129999999999999,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.0389999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5660000000000001,"15min":1.5249999999999999},"min":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":0.89500000000000002},"max":{"1min":1.978,"5min":2.762,"15min":2.762},"last":1.651},{"interface":"front","average":{"1min":1.587,"5min":1.621,"15min":1.585},"min":{"1min":1.218,"5min":1.1479999999999999,"15min":0.78700000000000003},"max":{"1min":1.9259999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.234}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1564,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1601536,"data_stored":1065915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.1060000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.498,"5min":1.651,"15min":3.3980000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.127,"15min":1.141},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.60099999999999998},"max":{"1min":1.5720000000000001,"5min":1.6479999999999999,"15min":2.153},"last":1.5109999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.153,"15min":1.137},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.7010000000000001},"last":1.472},{"interface":"front","average":{"1min":1.179,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.73599999999999999,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.94099999999999995}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4039999999999999,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.093},{"interface":"front","average":{"1min":1.155,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.6080000000000001,"5min":1.621,"15min":2.0630000000000002},"last":1.2090000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.286},"min":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.76900000000000002},"max":{"1min":1.5309999999999999,"5min":1.694,"15min":1.833},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3120000000000001,"15min":1.296},"min":{"1min":0.98099999999999998,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":1.6779999999999999,"5min":1.7110000000000001,"15min":2.1440000000000001},"last":1.663}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.238,"15min":1.2390000000000001},"min":{"1min":0.86399999999999999,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.6779999999999999,"5min":1.853,"15min":2.5129999999999999},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.84999999999999998,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.6319999999999999,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.5900000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2989999999999999,"15min":1.329},"min":{"1min":0.88500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":2.2970000000000002},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":1.125,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":1.962,"5min":1.962,"15min":3.23},"last":1.625}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4530000000000001,"15min":1.48},"min":{"1min":0.92400000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2,"5min":2,"15min":3.012},"last":2.0499999999999998},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4970000000000001,"15min":1.524},"min":{"1min":1.2689999999999999,"5min":0.873,"15min":0.873},"max":{"1min":2.073,"5min":2.073,"15min":2.8420000000000001},"last":2.1299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":1011712,"data_stored":476129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.119,"15min":1.1040000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.53,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.734,"5min":1.827,"15min":3.8069999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2350000000000001,"15min":1.2090000000000001},"min":{"1min":0.84599999999999997,"5min":0.81999999999999995,"15min":0.65900000000000003},"max":{"1min":1.528,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.452},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.131},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.671,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.995}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2110000000000001,"15min":1.196},"min":{"1min":0.83999999999999997,"5min":0.78200000000000003,"15min":0.77600000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3140000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.871,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.5780000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.234},"min":{"1min":0.85399999999999998,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.8220000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.095},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3200000000000001,"15min":1.319},"min":{"1min":0.753,"5min":0.753,"15min":0.70399999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.3010000000000002},"last":1.5149999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.96599999999999997,"5min":0.79200000000000004,"15min":0.61899999999999999},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.1549999999999998},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.2829999999999999,"15min":1.292},"min":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.5189999999999999,"5min":1.8029999999999999,"15min":2.2029999999999998},"last":1.2130000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.94999999999999996,"5min":0.84799999999999998,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.619},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.5029999999999999,"15min":1.4730000000000001},"min":{"1min":1.1000000000000001,"5min":0.96699999999999997,"15min":0.63200000000000001},"max":{"1min":1.7949999999999999,"5min":2.4990000000000001,"15min":2.54},"last":1.6950000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.458,"15min":1.464},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.60599999999999998},"max":{"1min":1.6850000000000001,"5min":1.972,"15min":3.2789999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5720000000000001,"15min":1.5580000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.04,"15min":2.157},"last":1.728}]}]},{"osd":3,"up_from":13,"seq":55834575035,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40224,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331616,"statfs":{"total":96636764160,"available":96595574784,"internally_reserved":0,"allocated":1007616,"data_stored":475603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82899999999999996,"15min":0.85199999999999998},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.307},"max":{"1min":1.135,"5min":1.302,"15min":1.583},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.999,"15min":1.0009999999999999},"min":{"1min":0.52100000000000002,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.3999999999999999,"5min":1.4990000000000001,"15min":1.9770000000000001},"last":1.2230000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.069,"15min":1.0629999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.38900000000000001},"max":{"1min":1.556,"5min":1.571,"15min":2.0779999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.95399999999999996,"15min":0.95199999999999996},"min":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.40600000000000003},"max":{"1min":1.5129999999999999,"5min":1.587,"15min":1.6519999999999999},"last":0.93600000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.016,"15min":1.054},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.436},"max":{"1min":1.286,"5min":1.512,"15min":1.766},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.173,"15min":1.1759999999999999},"min":{"1min":0.63800000000000001,"5min":0.621,"15min":0.48399999999999999},"max":{"1min":1.4870000000000001,"5min":1.647,"15min":1.661},"last":0.97699999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.238,"15min":1.2569999999999999},"min":{"1min":0.96499999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.476,"5min":1.6240000000000001,"15min":2.9350000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.246},"min":{"1min":0.94099999999999995,"5min":0.81000000000000005,"15min":0.73299999999999998},"max":{"1min":1.554,"5min":1.637,"15min":2.2389999999999999},"last":1.427}]},{"osd":5,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.272,"15min":1.302},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.63500000000000001},"max":{"1min":1.5900000000000001,"5min":1.806,"15min":3.476},"last":1.123},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2729999999999999,"15min":1.329},"min":{"1min":0.86099999999999999,"5min":0.82099999999999995,"15min":0.64200000000000002},"max":{"1min":1.464,"5min":1.9039999999999999,"15min":3.181},"last":1.2569999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.88900000000000001,"5min":0.80400000000000005,"15min":0.68300000000000005},"max":{"1min":1.6499999999999999,"5min":1.8580000000000001,"15min":2.532},"last":1.294},{"interface":"front","average":{"1min":1.409,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":1.02,"5min":0.94299999999999995,"15min":0.872},"max":{"1min":1.573,"5min":1.8049999999999999,"15min":3.0249999999999999},"last":1.393}]},{"osd":7,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3919999999999999,"15min":1.4239999999999999},"min":{"1min":1.006,"5min":0.999,"15min":0.81499999999999995},"max":{"1min":1.504,"5min":1.851,"15min":2.1619999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4259999999999999,"15min":1.472},"min":{"1min":1.048,"5min":0.88300000000000001,"15min":0.83099999999999996},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":2.4590000000000001},"last":1.5189999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575035,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":1024000,"data_stored":477237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27100000000000002},"max":{"1min":1.0860000000000001,"5min":1.339,"15min":1.738},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.026,"15min":1.01},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.375},"max":{"1min":1.4159999999999999,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":0.71099999999999997}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96499999999999997,"15min":0.98999999999999999},"min":{"1min":0.54700000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.1779999999999999,"5min":1.45,"15min":1.7130000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0720000000000001,"15min":1.0840000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.504},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.1719999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97499999999999998,"15min":1.0029999999999999},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.377,"5min":1.625,"15min":1.883},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.39600000000000002},"max":{"1min":1.415,"5min":1.613,"15min":2.6080000000000001},"last":1.2789999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.331,"15min":1.3180000000000001},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.75},"max":{"1min":1.7230000000000001,"5min":1.9950000000000001,"15min":2.2829999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.246,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.964},"last":1.4850000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3009999999999999,"15min":1.319},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.647,"5min":1.804,"15min":2.5779999999999998},"last":1.194},{"interface":"front","average":{"1min":1.375,"5min":1.387,"15min":1.375},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.901,"5min":2.048,"15min":3.0099999999999998},"last":1.0620000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.3200000000000001},"min":{"1min":0.84299999999999997,"5min":0.83399999999999996,"15min":0.77500000000000002},"max":{"1min":1.6439999999999999,"5min":1.78,"15min":2.2799999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.766,"15min":2.5089999999999999},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.456,"15min":1.448},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.79400000000000004},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.448},"last":1.331},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7649999999999999,"5min":1.802,"15min":2.5049999999999999},"last":1.5429999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":1564,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":1601536,"data_stored":1066463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85699999999999998,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.53800000000000003,"15min":0.318},"max":{"1min":1.1220000000000001,"5min":1.5049999999999999,"15min":3.056},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.999,"15min":1.012},"min":{"1min":0.75700000000000001,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":1.335,"5min":1.6359999999999999,"15min":3.4129999999999998},"last":1.4570000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.88200000000000001,"15min":0.90100000000000002},"min":{"1min":0.67100000000000004,"5min":0.497,"15min":0.36399999999999999},"max":{"1min":1.1779999999999999,"5min":1.54,"15min":3.3660000000000001},"last":0.877},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.78100000000000003,"5min":0.47899999999999998,"15min":0.39000000000000001},"max":{"1min":1.4890000000000001,"5min":1.492,"15min":3.4980000000000002},"last":0.998}]},{"osd":3,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.21,"15min":1.224},"min":{"1min":1.0860000000000001,"5min":0.51900000000000002,"15min":0.44},"max":{"1min":1.6859999999999999,"5min":1.7290000000000001,"15min":3.464},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.077,"15min":1.0669999999999999},"min":{"1min":0.82599999999999996,"5min":0.499,"15min":0.39700000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":3.0960000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1719999999999999,"15min":1.196},"min":{"1min":1.014,"5min":0.745,"15min":0.63800000000000001},"max":{"1min":1.494,"5min":1.536,"15min":3.7810000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.3240000000000001,"15min":1.3600000000000001},"min":{"1min":1.077,"5min":0.77500000000000002,"15min":0.76600000000000001},"max":{"1min":1.585,"5min":1.8089999999999999,"15min":4.7270000000000003},"last":1.2490000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.296,"15min":1.2909999999999999},"min":{"1min":1.032,"5min":0.90600000000000003,"15min":0.61199999999999999},"max":{"1min":1.5760000000000001,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.681},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.278,"15min":1.3160000000000001},"min":{"1min":1.004,"5min":0.76900000000000002,"15min":0.69199999999999995},"max":{"1min":1.5,"5min":1.825,"15min":3.6669999999999998},"last":1.4259999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.349},"min":{"1min":1.0760000000000001,"5min":0.71099999999999997,"15min":0.68400000000000005},"max":{"1min":1.5549999999999999,"5min":1.7809999999999999,"15min":3.3300000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.343,"15min":1.363},"min":{"1min":1.0649999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.595,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.5389999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.409,"15min":1.4630000000000001},"min":{"1min":1.246,"5min":0.76600000000000001,"15min":0.70599999999999996},"max":{"1min":1.645,"5min":1.8939999999999999,"15min":4.6749999999999998},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.4179999999999999,"15min":1.4299999999999999},"min":{"1min":1.1639999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7589999999999999,"5min":1.992,"15min":3.5379999999999998},"last":1.49}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":1015808,"data_stored":477077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.60699999999999998,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.3520000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.92600000000000005,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4390000000000001,"5min":1.5580000000000001,"15min":1.6910000000000001},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98499999999999999,"15min":1.006},"min":{"1min":0.67600000000000005,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.4690000000000001,"5min":2.157,"15min":2.157},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.113,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.84299999999999997,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.395,"5min":1.728,"15min":2.8159999999999998},"last":1.016}]},{"osd":3,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.58499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.954},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.143,"15min":1.169},"min":{"1min":0.54900000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7070000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1060000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.3089999999999999},"min":{"1min":1.05,"5min":0.78500000000000003,"15min":0.65700000000000003},"max":{"1min":1.5289999999999999,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.214,"15min":1.2629999999999999},"min":{"1min":0.83299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.2829999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.337,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.71899999999999997},"max":{"1min":1.839,"5min":2.351,"15min":2.351},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.306,"15min":1.349},"min":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6299999999999999,"5min":2.2909999999999999,"15min":2.7050000000000001},"last":1.2310000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73099999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.157},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.399},"min":{"1min":1.0129999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.256}]},{"osd":7,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":1.127,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.706,"5min":1.798,"15min":2.8999999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.361,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:51:34.104 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:51:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.325+0000 7fd289b99640 1 Processor -- start 2023-12-01T01:51:34.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.325+0000 7fd289b99640 1 -- start start 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.326+0000 7fd289b99640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd2840c9ef0 0x7fd28406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.326+0000 7fd289b99640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd28406b6f0 0x7fd284070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.326+0000 7fd289b99640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28406f8b0 con 0x7fd284159dd0 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.326+0000 7fd289b99640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2840adcb0 con 0x7fd28406b6f0 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.326+0000 7fd289b99640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2840ad9d0 con 0x7fd2840c9ef0 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.327+0000 7fd283fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd28406b6f0 0x7fd284070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.327+0000 7fd283fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd28406b6f0 0x7fd284070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39288/0 (socket says 172.21.15.115:39288) 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.327+0000 7fd283fff640 1 -- 172.21.15.115:0/1279690921 learned_addr learned my addr 172.21.15.115:0/1279690921 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.327+0000 7fd27affd640 1 --2- 172.21.15.115:0/1279690921 >> v2:172.21.15.142:3300/0 conn(0x7fd2840c9ef0 0x7fd28406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.327+0000 7fd283fff640 1 -- 172.21.15.115:0/1279690921 >> v2:172.21.15.142:3300/0 conn(0x7fd2840c9ef0 msgr2=0x7fd28406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.327+0000 7fd283fff640 1 --2- 172.21.15.115:0/1279690921 >> v2:172.21.15.142:3300/0 conn(0x7fd2840c9ef0 0x7fd28406fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.327+0000 7fd283fff640 1 -- 172.21.15.115:0/1279690921 >> v1:172.21.15.142:6789/0 conn(0x7fd284159dd0 legacy=0x7fd2840c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:34.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.327+0000 7fd283fff640 1 -- 172.21.15.115:0/1279690921 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2840c9000 con 0x7fd28406b6f0 2023-12-01T01:51:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.328+0000 7fd283fff640 1 --2- 172.21.15.115:0/1279690921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd28406b6f0 0x7fd284070260 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd28406de80 tx=0x7fd27c02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c825949e95ed6f1b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd282ffd640 1 -- 172.21.15.115:0/1279690921 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c02c960 con 0x7fd28406b6f0 2023-12-01T01:51:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd282ffd640 1 -- 172.21.15.115:0/1279690921 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c02cac0 con 0x7fd28406b6f0 2023-12-01T01:51:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd282ffd640 1 -- 172.21.15.115:0/1279690921 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c02ce50 con 0x7fd28406b6f0 2023-12-01T01:51:34.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd289b99640 1 -- 172.21.15.115:0/1279690921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd28406b6f0 msgr2=0x7fd284070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd289b99640 1 --2- 172.21.15.115:0/1279690921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd28406b6f0 0x7fd284070260 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd28406de80 tx=0x7fd27c02bdc0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd289b99640 1 -- 172.21.15.115:0/1279690921 shutdown_connections 2023-12-01T01:51:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd289b99640 1 --2- 172.21.15.115:0/1279690921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd28406b6f0 0x7fd284070260 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd289b99640 1 --2- 172.21.15.115:0/1279690921 >> v2:172.21.15.142:3300/0 conn(0x7fd2840c9ef0 0x7fd28406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd289b99640 1 -- 172.21.15.115:0/1279690921 >> 172.21.15.115:0/1279690921 conn(0x7fd2840ce880 msgr2=0x7fd2840cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd289b99640 1 -- 172.21.15.115:0/1279690921 shutdown_connections 2023-12-01T01:51:34.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.329+0000 7fd289b99640 1 -- 172.21.15.115:0/1279690921 wait complete. 2023-12-01T01:51:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.331+0000 7fd289b99640 1 Processor -- start 2023-12-01T01:51:34.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.331+0000 7fd289b99640 1 -- start start 2023-12-01T01:51:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.332+0000 7fd289b99640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd28406b6f0 0x7fd284155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.332+0000 7fd289b99640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2840c9ef0 0x7fd2840c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.332+0000 7fd289b99640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd284159dd0 0x7fd28415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.332+0000 7fd289b99640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840b1520 con 0x7fd284159dd0 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.332+0000 7fd289b99640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840c3ff0 con 0x7fd28406b6f0 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.332+0000 7fd289b99640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2840c42d0 con 0x7fd2840c9ef0 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd283fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd284159dd0 0x7fd28415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd283fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd284159dd0 0x7fd28415ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39304/0 (socket says 172.21.15.115:39304) 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd283fff640 1 -- 172.21.15.115:0/726244566 learned_addr learned my addr 172.21.15.115:0/726244566 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd27affd640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2840c9ef0 0x7fd2840c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd2837fe640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd28406b6f0 0x7fd284155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd283fff640 1 -- 172.21.15.115:0/726244566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2840c9ef0 msgr2=0x7fd2840c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd283fff640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2840c9ef0 0x7fd2840c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd283fff640 1 -- 172.21.15.115:0/726244566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd28406b6f0 msgr2=0x7fd284155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:34.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd283fff640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd28406b6f0 0x7fd284155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd283fff640 1 -- 172.21.15.115:0/726244566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c009b30 con 0x7fd284159dd0 2023-12-01T01:51:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd2837fe640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd28406b6f0 0x7fd284155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd27affd640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2840c9ef0 0x7fd2840c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.333+0000 7fd283fff640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd284159dd0 0x7fd28415ca20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd27c036040 tx=0x7fd27c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.334+0000 7fd280ff9640 1 -- 172.21.15.115:0/726244566 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c009510 con 0x7fd284159dd0 2023-12-01T01:51:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.334+0000 7fd280ff9640 1 -- 172.21.15.115:0/726244566 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c03e450 con 0x7fd284159dd0 2023-12-01T01:51:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.334+0000 7fd280ff9640 1 -- 172.21.15.115:0/726244566 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c03e800 con 0x7fd284159dd0 2023-12-01T01:51:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.334+0000 7fd289b99640 1 -- 172.21.15.115:0/726244566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2840c45b0 con 0x7fd284159dd0 2023-12-01T01:51:34.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.334+0000 7fd289b99640 1 -- 172.21.15.115:0/726244566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd284156080 con 0x7fd284159dd0 2023-12-01T01:51:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.335+0000 7fd289b99640 1 -- 172.21.15.115:0/726244566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2840b19b0 con 0x7fd284159dd0 2023-12-01T01:51:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.336+0000 7fd280ff9640 1 -- 172.21.15.115:0/726244566 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd27c03e960 con 0x7fd284159dd0 2023-12-01T01:51:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.337+0000 7fd280ff9640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd26406a550 0x7fd26406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.337+0000 7fd280ff9640 1 -- 172.21.15.115:0/726244566 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd27c0ab870 con 0x7fd284159dd0 2023-12-01T01:51:34.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.337+0000 7fd2837fe640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd26406a550 0x7fd26406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:34.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.338+0000 7fd2837fe640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd26406a550 0x7fd26406ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00a810 tx=0x7fd26c008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:34.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.341+0000 7fd280ff9640 1 -- 172.21.15.115:0/726244566 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd2840b19b0 con 0x7fd284159dd0 2023-12-01T01:51:34.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.528+0000 7fd289b99640 1 -- 172.21.15.115:0/726244566 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd28406d0d0 con 0x7fd26406a550 2023-12-01T01:51:34.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.552+0000 7fd280ff9640 1 -- 172.21.15.115:0/726244566 <== mgr.6201 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286247 (secure 0 0 0) 0x7fd26c00ad90 con 0x7fd26406a550 2023-12-01T01:51:34.555 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:34.558 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:51:34.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.558+0000 7fd289b99640 1 -- 172.21.15.115:0/726244566 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd26406a550 msgr2=0x7fd26406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:34.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.558+0000 7fd289b99640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd26406a550 0x7fd26406ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00a810 tx=0x7fd26c008040 comp rx=0 tx=0).stop 2023-12-01T01:51:34.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.558+0000 7fd289b99640 1 -- 172.21.15.115:0/726244566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd284159dd0 msgr2=0x7fd28415ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:34.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.558+0000 7fd289b99640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd284159dd0 0x7fd28415ca20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd27c036040 tx=0x7fd27c004a40 comp rx=0 tx=0).stop 2023-12-01T01:51:34.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.559+0000 7fd289b99640 1 -- 172.21.15.115:0/726244566 shutdown_connections 2023-12-01T01:51:34.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.559+0000 7fd289b99640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd26406a550 0x7fd26406ca10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.559+0000 7fd289b99640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2840c9ef0 0x7fd2840c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.559+0000 7fd289b99640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd284159dd0 0x7fd28415ca20 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.559+0000 7fd289b99640 1 --2- 172.21.15.115:0/726244566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd28406b6f0 0x7fd284155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.559+0000 7fd289b99640 1 -- 172.21.15.115:0/726244566 >> 172.21.15.115:0/726244566 conn(0x7fd2840ce880 msgr2=0x7fd284070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:34.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.559+0000 7fd289b99640 1 -- 172.21.15.115:0/726244566 shutdown_connections 2023-12-01T01:51:34.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.559+0000 7fd289b99640 1 -- 172.21.15.115:0/726244566 wait complete. 2023-12-01T01:51:34.580 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-12-01T01:51:33.826914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269736,"kb_used_data":9064,"kb_used_omap":0,"kb_used_meta":260608,"kb_avail":754704984,"statfs":{"total":773094113280,"available":772817903616,"internally_reserved":0,"allocated":9281536,"data_stored":4989564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":266862592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.646769"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228585+0000","last_change":"2023-12-01T01:51:24.224886+0000","last_active":"2023-12-01T01:51:24.228585+0000","last_peered":"2023-12-01T01:51:24.228585+0000","last_clean":"2023-12-01T01:51:24.228585+0000","last_became_active":"2023-12-01T01:51:24.222251+0000","last_became_peered":"2023-12-01T01:51:24.222251+0000","last_unstale":"2023-12-01T01:51:24.228585+0000","last_undegraded":"2023-12-01T01:51:24.228585+0000","last_fullsized":"2023-12-01T01:51:24.228585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:21.413565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228337+0000","last_change":"2023-12-01T01:51:24.223609+0000","last_active":"2023-12-01T01:51:24.228337+0000","last_peered":"2023-12-01T01:51:24.228337+0000","last_clean":"2023-12-01T01:51:24.228337+0000","last_became_active":"2023-12-01T01:51:24.222770+0000","last_became_peered":"2023-12-01T01:51:24.222770+0000","last_unstale":"2023-12-01T01:51:24.228337+0000","last_undegraded":"2023-12-01T01:51:24.228337+0000","last_fullsized":"2023-12-01T01:51:24.228337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:26.173230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.209949+0000","last_change":"2023-12-01T01:51:24.207820+0000","last_active":"2023-12-01T01:51:24.209949+0000","last_peered":"2023-12-01T01:51:24.209949+0000","last_clean":"2023-12-01T01:51:24.209949+0000","last_became_active":"2023-12-01T01:51:24.207535+0000","last_became_peered":"2023-12-01T01:51:24.207535+0000","last_unstale":"2023-12-01T01:51:24.209949+0000","last_undegraded":"2023-12-01T01:51:24.209949+0000","last_fullsized":"2023-12-01T01:51:24.209949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:50.301817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223930+0000","last_change":"2023-12-01T01:51:24.222179+0000","last_active":"2023-12-01T01:51:24.223930+0000","last_peered":"2023-12-01T01:51:24.223930+0000","last_clean":"2023-12-01T01:51:24.223930+0000","last_became_active":"2023-12-01T01:51:24.221936+0000","last_became_peered":"2023-12-01T01:51:24.221936+0000","last_unstale":"2023-12-01T01:51:24.223930+0000","last_undegraded":"2023-12-01T01:51:24.223930+0000","last_fullsized":"2023-12-01T01:51:24.223930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:03:53.147122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224064+0000","last_change":"2023-12-01T01:51:24.219076+0000","last_active":"2023-12-01T01:51:24.224064+0000","last_peered":"2023-12-01T01:51:24.224064+0000","last_clean":"2023-12-01T01:51:24.224064+0000","last_became_active":"2023-12-01T01:51:24.218809+0000","last_became_peered":"2023-12-01T01:51:24.218809+0000","last_unstale":"2023-12-01T01:51:24.224064+0000","last_undegraded":"2023-12-01T01:51:24.224064+0000","last_fullsized":"2023-12-01T01:51:24.224064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:16.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224174+0000","last_change":"2023-12-01T01:51:24.221005+0000","last_active":"2023-12-01T01:51:24.224174+0000","last_peered":"2023-12-01T01:51:24.224174+0000","last_clean":"2023-12-01T01:51:24.224174+0000","last_became_active":"2023-12-01T01:51:24.220737+0000","last_became_peered":"2023-12-01T01:51:24.220737+0000","last_unstale":"2023-12-01T01:51:24.224174+0000","last_undegraded":"2023-12-01T01:51:24.224174+0000","last_fullsized":"2023-12-01T01:51:24.224174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:08.979742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.212243+0000","last_change":"2023-12-01T01:51:24.205047+0000","last_active":"2023-12-01T01:51:24.212243+0000","last_peered":"2023-12-01T01:51:24.212243+0000","last_clean":"2023-12-01T01:51:24.212243+0000","last_became_active":"2023-12-01T01:51:24.204779+0000","last_became_peered":"2023-12-01T01:51:24.204779+0000","last_unstale":"2023-12-01T01:51:24.212243+0000","last_undegraded":"2023-12-01T01:51:24.212243+0000","last_fullsized":"2023-12-01T01:51:24.212243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:41.379891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.204987+0000","last_change":"2023-12-01T01:51:24.199691+0000","last_active":"2023-12-01T01:51:24.204987+0000","last_peered":"2023-12-01T01:51:24.204987+0000","last_clean":"2023-12-01T01:51:24.204987+0000","last_became_active":"2023-12-01T01:51:24.199343+0000","last_became_peered":"2023-12-01T01:51:24.199343+0000","last_unstale":"2023-12-01T01:51:24.204987+0000","last_undegraded":"2023-12-01T01:51:24.204987+0000","last_fullsized":"2023-12-01T01:51:24.204987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:56:56.216365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223463+0000","last_change":"2023-12-01T01:51:24.221268+0000","last_active":"2023-12-01T01:51:24.223463+0000","last_peered":"2023-12-01T01:51:24.223463+0000","last_clean":"2023-12-01T01:51:24.223463+0000","last_became_active":"2023-12-01T01:51:24.220971+0000","last_became_peered":"2023-12-01T01:51:24.220971+0000","last_unstale":"2023-12-01T01:51:24.223463+0000","last_undegraded":"2023-12-01T01:51:24.223463+0000","last_fullsized":"2023-12-01T01:51:24.223463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:55.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225919+0000","last_change":"2023-12-01T01:51:24.222276+0000","last_active":"2023-12-01T01:51:24.225919+0000","last_peered":"2023-12-01T01:51:24.225919+0000","last_clean":"2023-12-01T01:51:24.225919+0000","last_became_active":"2023-12-01T01:51:24.220691+0000","last_became_peered":"2023-12-01T01:51:24.220691+0000","last_unstale":"2023-12-01T01:51:24.225919+0000","last_undegraded":"2023-12-01T01:51:24.225919+0000","last_fullsized":"2023-12-01T01:51:24.225919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:52.689385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225715+0000","last_change":"2023-12-01T01:51:24.221827+0000","last_active":"2023-12-01T01:51:24.225715+0000","last_peered":"2023-12-01T01:51:24.225715+0000","last_clean":"2023-12-01T01:51:24.225715+0000","last_became_active":"2023-12-01T01:51:24.221267+0000","last_became_peered":"2023-12-01T01:51:24.221267+0000","last_unstale":"2023-12-01T01:51:24.225715+0000","last_undegraded":"2023-12-01T01:51:24.225715+0000","last_fullsized":"2023-12-01T01:51:24.225715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:21.413565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.197650+0000","last_change":"2023-12-01T01:51:24.193709+0000","last_active":"2023-12-01T01:51:24.197650+0000","last_peered":"2023-12-01T01:51:24.197650+0000","last_clean":"2023-12-01T01:51:24.197650+0000","last_became_active":"2023-12-01T01:51:24.193253+0000","last_became_peered":"2023-12-01T01:51:24.193253+0000","last_unstale":"2023-12-01T01:51:24.197650+0000","last_undegraded":"2023-12-01T01:51:24.197650+0000","last_fullsized":"2023-12-01T01:51:24.197650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:32.142316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217097+0000","last_change":"2023-12-01T01:51:24.211498+0000","last_active":"2023-12-01T01:51:24.217097+0000","last_peered":"2023-12-01T01:51:24.217097+0000","last_clean":"2023-12-01T01:51:24.217097+0000","last_became_active":"2023-12-01T01:51:24.211147+0000","last_became_peered":"2023-12-01T01:51:24.211147+0000","last_unstale":"2023-12-01T01:51:24.217097+0000","last_undegraded":"2023-12-01T01:51:24.217097+0000","last_fullsized":"2023-12-01T01:51:24.217097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:01.177699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211052+0000","last_change":"2023-12-01T01:51:24.208945+0000","last_active":"2023-12-01T01:51:24.211052+0000","last_peered":"2023-12-01T01:51:24.211052+0000","last_clean":"2023-12-01T01:51:24.211052+0000","last_became_active":"2023-12-01T01:51:24.208667+0000","last_became_peered":"2023-12-01T01:51:24.208667+0000","last_unstale":"2023-12-01T01:51:24.211052+0000","last_undegraded":"2023-12-01T01:51:24.211052+0000","last_fullsized":"2023-12-01T01:51:24.211052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:29.110060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223068+0000","last_change":"2023-12-01T01:51:24.221816+0000","last_active":"2023-12-01T01:51:24.223068+0000","last_peered":"2023-12-01T01:51:24.223068+0000","last_clean":"2023-12-01T01:51:24.223068+0000","last_became_active":"2023-12-01T01:51:24.221513+0000","last_became_peered":"2023-12-01T01:51:24.221513+0000","last_unstale":"2023-12-01T01:51:24.223068+0000","last_undegraded":"2023-12-01T01:51:24.223068+0000","last_fullsized":"2023-12-01T01:51:24.223068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.219844+0000","last_change":"2023-12-01T01:51:24.212262+0000","last_active":"2023-12-01T01:51:24.219844+0000","last_peered":"2023-12-01T01:51:24.219844+0000","last_clean":"2023-12-01T01:51:24.219844+0000","last_became_active":"2023-12-01T01:51:24.211984+0000","last_became_peered":"2023-12-01T01:51:24.211984+0000","last_unstale":"2023-12-01T01:51:24.219844+0000","last_undegraded":"2023-12-01T01:51:24.219844+0000","last_fullsized":"2023-12-01T01:51:24.219844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:28.427573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228598+0000","last_change":"2023-12-01T01:51:24.225154+0000","last_active":"2023-12-01T01:51:24.228598+0000","last_peered":"2023-12-01T01:51:24.228598+0000","last_clean":"2023-12-01T01:51:24.228598+0000","last_became_active":"2023-12-01T01:51:24.224886+0000","last_became_peered":"2023-12-01T01:51:24.224886+0000","last_unstale":"2023-12-01T01:51:24.228598+0000","last_undegraded":"2023-12-01T01:51:24.228598+0000","last_fullsized":"2023-12-01T01:51:24.228598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223814+0000","last_change":"2023-12-01T01:51:24.222850+0000","last_active":"2023-12-01T01:51:24.223814+0000","last_peered":"2023-12-01T01:51:24.223814+0000","last_clean":"2023-12-01T01:51:24.223814+0000","last_became_active":"2023-12-01T01:51:24.222475+0000","last_became_peered":"2023-12-01T01:51:24.222475+0000","last_unstale":"2023-12-01T01:51:24.223814+0000","last_undegraded":"2023-12-01T01:51:24.223814+0000","last_fullsized":"2023-12-01T01:51:24.223814+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:27.535710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225788+0000","last_change":"2023-12-01T01:51:24.220362+0000","last_active":"2023-12-01T01:51:24.225788+0000","last_peered":"2023-12-01T01:51:24.225788+0000","last_clean":"2023-12-01T01:51:24.225788+0000","last_became_active":"2023-12-01T01:51:24.219702+0000","last_became_peered":"2023-12-01T01:51:24.219702+0000","last_unstale":"2023-12-01T01:51:24.225788+0000","last_undegraded":"2023-12-01T01:51:24.225788+0000","last_fullsized":"2023-12-01T01:51:24.225788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:14.157429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217404+0000","last_change":"2023-12-01T01:51:24.211275+0000","last_active":"2023-12-01T01:51:24.217404+0000","last_peered":"2023-12-01T01:51:24.217404+0000","last_clean":"2023-12-01T01:51:24.217404+0000","last_became_active":"2023-12-01T01:51:24.210964+0000","last_became_peered":"2023-12-01T01:51:24.210964+0000","last_unstale":"2023-12-01T01:51:24.217404+0000","last_undegraded":"2023-12-01T01:51:24.217404+0000","last_fullsized":"2023-12-01T01:51:24.217404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:08.979742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.212510+0000","last_change":"2023-12-01T01:51:24.206715+0000","last_active":"2023-12-01T01:51:24.212510+0000","last_peered":"2023-12-01T01:51:24.212510+0000","last_clean":"2023-12-01T01:51:24.212510+0000","last_became_active":"2023-12-01T01:51:24.206432+0000","last_became_peered":"2023-12-01T01:51:24.206432+0000","last_unstale":"2023-12-01T01:51:24.212510+0000","last_undegraded":"2023-12-01T01:51:24.212510+0000","last_fullsized":"2023-12-01T01:51:24.212510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217430+0000","last_change":"2023-12-01T01:51:24.212833+0000","last_active":"2023-12-01T01:51:24.217430+0000","last_peered":"2023-12-01T01:51:24.217430+0000","last_clean":"2023-12-01T01:51:24.217430+0000","last_became_active":"2023-12-01T01:51:24.212494+0000","last_became_peered":"2023-12-01T01:51:24.212494+0000","last_unstale":"2023-12-01T01:51:24.217430+0000","last_undegraded":"2023-12-01T01:51:24.217430+0000","last_fullsized":"2023-12-01T01:51:24.217430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:12.784193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217998+0000","last_change":"2023-12-01T01:51:24.215195+0000","last_active":"2023-12-01T01:51:24.217998+0000","last_peered":"2023-12-01T01:51:24.217998+0000","last_clean":"2023-12-01T01:51:24.217998+0000","last_became_active":"2023-12-01T01:51:24.214924+0000","last_became_peered":"2023-12-01T01:51:24.214924+0000","last_unstale":"2023-12-01T01:51:24.217998+0000","last_undegraded":"2023-12-01T01:51:24.217998+0000","last_fullsized":"2023-12-01T01:51:24.217998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:39:57.857405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211682+0000","last_change":"2023-12-01T01:51:24.210975+0000","last_active":"2023-12-01T01:51:24.211682+0000","last_peered":"2023-12-01T01:51:24.211682+0000","last_clean":"2023-12-01T01:51:24.211682+0000","last_became_active":"2023-12-01T01:51:24.210687+0000","last_became_peered":"2023-12-01T01:51:24.210687+0000","last_unstale":"2023-12-01T01:51:24.211682+0000","last_undegraded":"2023-12-01T01:51:24.211682+0000","last_fullsized":"2023-12-01T01:51:24.211682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:51.772038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225706+0000","last_change":"2023-12-01T01:51:24.221053+0000","last_active":"2023-12-01T01:51:24.225706+0000","last_peered":"2023-12-01T01:51:24.225706+0000","last_clean":"2023-12-01T01:51:24.225706+0000","last_became_active":"2023-12-01T01:51:24.219646+0000","last_became_peered":"2023-12-01T01:51:24.219646+0000","last_unstale":"2023-12-01T01:51:24.225706+0000","last_undegraded":"2023-12-01T01:51:24.225706+0000","last_fullsized":"2023-12-01T01:51:24.225706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:06:06.433309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.200813+0000","last_change":"2023-12-01T01:51:24.197255+0000","last_active":"2023-12-01T01:51:24.200813+0000","last_peered":"2023-12-01T01:51:24.200813+0000","last_clean":"2023-12-01T01:51:24.200813+0000","last_became_active":"2023-12-01T01:51:24.197005+0000","last_became_peered":"2023-12-01T01:51:24.197005+0000","last_unstale":"2023-12-01T01:51:24.200813+0000","last_undegraded":"2023-12-01T01:51:24.200813+0000","last_fullsized":"2023-12-01T01:51:24.200813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:10.334292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225808+0000","last_change":"2023-12-01T01:51:24.220582+0000","last_active":"2023-12-01T01:51:24.225808+0000","last_peered":"2023-12-01T01:51:24.225808+0000","last_clean":"2023-12-01T01:51:24.225808+0000","last_became_active":"2023-12-01T01:51:24.219677+0000","last_became_peered":"2023-12-01T01:51:24.219677+0000","last_unstale":"2023-12-01T01:51:24.225808+0000","last_undegraded":"2023-12-01T01:51:24.225808+0000","last_fullsized":"2023-12-01T01:51:24.225808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:10:04.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.203074+0000","last_change":"2023-12-01T01:51:24.198223+0000","last_active":"2023-12-01T01:51:24.203074+0000","last_peered":"2023-12-01T01:51:24.203074+0000","last_clean":"2023-12-01T01:51:24.203074+0000","last_became_active":"2023-12-01T01:51:24.197651+0000","last_became_peered":"2023-12-01T01:51:24.197651+0000","last_unstale":"2023-12-01T01:51:24.203074+0000","last_undegraded":"2023-12-01T01:51:24.203074+0000","last_fullsized":"2023-12-01T01:51:24.203074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:46.030193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.208866+0000","last_change":"2023-12-01T01:51:24.205197+0000","last_active":"2023-12-01T01:51:24.208866+0000","last_peered":"2023-12-01T01:51:24.208866+0000","last_clean":"2023-12-01T01:51:24.208866+0000","last_became_active":"2023-12-01T01:51:24.204894+0000","last_became_peered":"2023-12-01T01:51:24.204894+0000","last_unstale":"2023-12-01T01:51:24.208866+0000","last_undegraded":"2023-12-01T01:51:24.208866+0000","last_fullsized":"2023-12-01T01:51:24.208866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:49.877816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.197827+0000","last_change":"2023-12-01T01:51:24.196324+0000","last_active":"2023-12-01T01:51:24.197827+0000","last_peered":"2023-12-01T01:51:24.197827+0000","last_clean":"2023-12-01T01:51:24.197827+0000","last_became_active":"2023-12-01T01:51:24.194764+0000","last_became_peered":"2023-12-01T01:51:24.194764+0000","last_unstale":"2023-12-01T01:51:24.197827+0000","last_undegraded":"2023-12-01T01:51:24.197827+0000","last_fullsized":"2023-12-01T01:51:24.197827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:28.026095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221789+0000","last_change":"2023-12-01T01:51:24.213551+0000","last_active":"2023-12-01T01:51:24.221789+0000","last_peered":"2023-12-01T01:51:24.221789+0000","last_clean":"2023-12-01T01:51:24.221789+0000","last_became_active":"2023-12-01T01:51:24.212904+0000","last_became_peered":"2023-12-01T01:51:24.212904+0000","last_unstale":"2023-12-01T01:51:24.221789+0000","last_undegraded":"2023-12-01T01:51:24.221789+0000","last_fullsized":"2023-12-01T01:51:24.221789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:52.879295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217559+0000","last_change":"2023-12-01T01:51:24.212738+0000","last_active":"2023-12-01T01:51:24.217559+0000","last_peered":"2023-12-01T01:51:24.217559+0000","last_clean":"2023-12-01T01:51:24.217559+0000","last_became_active":"2023-12-01T01:51:24.212433+0000","last_became_peered":"2023-12-01T01:51:24.212433+0000","last_unstale":"2023-12-01T01:51:24.217559+0000","last_undegraded":"2023-12-01T01:51:24.217559+0000","last_fullsized":"2023-12-01T01:51:24.217559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:16.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220228+0000","last_change":"2023-12-01T01:51:24.211814+0000","last_active":"2023-12-01T01:51:24.220228+0000","last_peered":"2023-12-01T01:51:24.220228+0000","last_clean":"2023-12-01T01:51:24.220228+0000","last_became_active":"2023-12-01T01:51:24.211542+0000","last_became_peered":"2023-12-01T01:51:24.211542+0000","last_unstale":"2023-12-01T01:51:24.220228+0000","last_undegraded":"2023-12-01T01:51:24.220228+0000","last_fullsized":"2023-12-01T01:51:24.220228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:01:44.563857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"82'4","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218866+0000","last_change":"2023-12-01T01:51:23.722626+0000","last_active":"2023-12-01T01:51:24.218866+0000","last_peered":"2023-12-01T01:51:24.218866+0000","last_clean":"2023-12-01T01:51:24.218866+0000","last_became_active":"2023-12-01T01:51:23.722272+0000","last_became_peered":"2023-12-01T01:51:23.722272+0000","last_unstale":"2023-12-01T01:51:24.218866+0000","last_undegraded":"2023-12-01T01:51:24.218866+0000","last_fullsized":"2023-12-01T01:51:24.218866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:49.297425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223115+0000","last_change":"2023-12-01T01:51:24.218268+0000","last_active":"2023-12-01T01:51:24.223115+0000","last_peered":"2023-12-01T01:51:24.223115+0000","last_clean":"2023-12-01T01:51:24.223115+0000","last_became_active":"2023-12-01T01:51:24.217888+0000","last_became_peered":"2023-12-01T01:51:24.217888+0000","last_unstale":"2023-12-01T01:51:24.223115+0000","last_undegraded":"2023-12-01T01:51:24.223115+0000","last_fullsized":"2023-12-01T01:51:24.223115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:12.784193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.219239+0000","last_change":"2023-12-01T01:51:23.730769+0000","last_active":"2023-12-01T01:51:24.219239+0000","last_peered":"2023-12-01T01:51:24.219239+0000","last_clean":"2023-12-01T01:51:24.219239+0000","last_became_active":"2023-12-01T01:51:23.730468+0000","last_became_peered":"2023-12-01T01:51:23.730468+0000","last_unstale":"2023-12-01T01:51:24.219239+0000","last_undegraded":"2023-12-01T01:51:24.219239+0000","last_fullsized":"2023-12-01T01:51:24.219239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:56.584539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218966+0000","last_change":"2023-12-01T01:51:24.214183+0000","last_active":"2023-12-01T01:51:24.218966+0000","last_peered":"2023-12-01T01:51:24.218966+0000","last_clean":"2023-12-01T01:51:24.218966+0000","last_became_active":"2023-12-01T01:51:24.213872+0000","last_became_peered":"2023-12-01T01:51:24.213872+0000","last_unstale":"2023-12-01T01:51:24.218966+0000","last_undegraded":"2023-12-01T01:51:24.218966+0000","last_fullsized":"2023-12-01T01:51:24.218966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:53:01.496510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.228599+0000","last_change":"2023-12-01T01:51:23.729205+0000","last_active":"2023-12-01T01:51:24.228599+0000","last_peered":"2023-12-01T01:51:24.228599+0000","last_clean":"2023-12-01T01:51:24.228599+0000","last_became_active":"2023-12-01T01:51:23.728920+0000","last_became_peered":"2023-12-01T01:51:23.728920+0000","last_unstale":"2023-12-01T01:51:24.228599+0000","last_undegraded":"2023-12-01T01:51:24.228599+0000","last_fullsized":"2023-12-01T01:51:24.228599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:21:58.877785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.226012+0000","last_change":"2023-12-01T01:51:24.223234+0000","last_active":"2023-12-01T01:51:24.226012+0000","last_peered":"2023-12-01T01:51:24.226012+0000","last_clean":"2023-12-01T01:51:24.226012+0000","last_became_active":"2023-12-01T01:51:24.222152+0000","last_became_peered":"2023-12-01T01:51:24.222152+0000","last_unstale":"2023-12-01T01:51:24.226012+0000","last_undegraded":"2023-12-01T01:51:24.226012+0000","last_fullsized":"2023-12-01T01:51:24.226012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:01.177699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205222+0000","last_change":"2023-12-01T01:51:23.729190+0000","last_active":"2023-12-01T01:51:24.205222+0000","last_peered":"2023-12-01T01:51:24.205222+0000","last_clean":"2023-12-01T01:51:24.205222+0000","last_became_active":"2023-12-01T01:51:23.728867+0000","last_became_peered":"2023-12-01T01:51:23.728867+0000","last_unstale":"2023-12-01T01:51:24.205222+0000","last_undegraded":"2023-12-01T01:51:24.205222+0000","last_fullsized":"2023-12-01T01:51:24.205222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:43:10.966577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.209477+0000","last_change":"2023-12-01T01:51:24.205594+0000","last_active":"2023-12-01T01:51:24.209477+0000","last_peered":"2023-12-01T01:51:24.209477+0000","last_clean":"2023-12-01T01:51:24.209477+0000","last_became_active":"2023-12-01T01:51:24.205042+0000","last_became_peered":"2023-12-01T01:51:24.205042+0000","last_unstale":"2023-12-01T01:51:24.209477+0000","last_undegraded":"2023-12-01T01:51:24.209477+0000","last_fullsized":"2023-12-01T01:51:24.209477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:01:53.075668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211402+0000","last_change":"2023-12-01T01:51:23.717746+0000","last_active":"2023-12-01T01:51:24.211402+0000","last_peered":"2023-12-01T01:51:24.211402+0000","last_clean":"2023-12-01T01:51:24.211402+0000","last_became_active":"2023-12-01T01:51:23.715918+0000","last_became_peered":"2023-12-01T01:51:23.715918+0000","last_unstale":"2023-12-01T01:51:24.211402+0000","last_undegraded":"2023-12-01T01:51:24.211402+0000","last_fullsized":"2023-12-01T01:51:24.211402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:35:50.904010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.219334+0000","last_change":"2023-12-01T01:51:24.214836+0000","last_active":"2023-12-01T01:51:24.219334+0000","last_peered":"2023-12-01T01:51:24.219334+0000","last_clean":"2023-12-01T01:51:24.219334+0000","last_became_active":"2023-12-01T01:51:24.214055+0000","last_became_peered":"2023-12-01T01:51:24.214055+0000","last_unstale":"2023-12-01T01:51:24.219334+0000","last_undegraded":"2023-12-01T01:51:24.219334+0000","last_fullsized":"2023-12-01T01:51:24.219334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:06.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221430+0000","last_change":"2023-12-01T01:51:23.725607+0000","last_active":"2023-12-01T01:51:24.221430+0000","last_peered":"2023-12-01T01:51:24.221430+0000","last_clean":"2023-12-01T01:51:24.221430+0000","last_became_active":"2023-12-01T01:51:23.722833+0000","last_became_peered":"2023-12-01T01:51:23.722833+0000","last_unstale":"2023-12-01T01:51:24.221430+0000","last_undegraded":"2023-12-01T01:51:24.221430+0000","last_fullsized":"2023-12-01T01:51:24.221430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:10:03.788374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.214615+0000","last_change":"2023-12-01T01:51:24.211805+0000","last_active":"2023-12-01T01:51:24.214615+0000","last_peered":"2023-12-01T01:51:24.214615+0000","last_clean":"2023-12-01T01:51:24.214615+0000","last_became_active":"2023-12-01T01:51:24.211532+0000","last_became_peered":"2023-12-01T01:51:24.211532+0000","last_unstale":"2023-12-01T01:51:24.214615+0000","last_undegraded":"2023-12-01T01:51:24.214615+0000","last_fullsized":"2023-12-01T01:51:24.214615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:51.144418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.186975+0000","last_change":"2023-12-01T01:51:23.729822+0000","last_active":"2023-12-01T01:51:24.186975+0000","last_peered":"2023-12-01T01:51:24.186975+0000","last_clean":"2023-12-01T01:51:24.186975+0000","last_became_active":"2023-12-01T01:51:23.728147+0000","last_became_peered":"2023-12-01T01:51:23.728147+0000","last_unstale":"2023-12-01T01:51:24.186975+0000","last_undegraded":"2023-12-01T01:51:24.186975+0000","last_fullsized":"2023-12-01T01:51:24.186975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:43:10.667895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.225883+0000","last_change":"2023-12-01T01:51:24.222272+0000","last_active":"2023-12-01T01:51:24.225883+0000","last_peered":"2023-12-01T01:51:24.225883+0000","last_clean":"2023-12-01T01:51:24.225883+0000","last_became_active":"2023-12-01T01:51:24.221990+0000","last_became_peered":"2023-12-01T01:51:24.221990+0000","last_unstale":"2023-12-01T01:51:24.225883+0000","last_undegraded":"2023-12-01T01:51:24.225883+0000","last_fullsized":"2023-12-01T01:51:24.225883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187467+0000","last_change":"2023-12-01T01:51:23.710030+0000","last_active":"2023-12-01T01:51:24.187467+0000","last_peered":"2023-12-01T01:51:24.187467+0000","last_clean":"2023-12-01T01:51:24.187467+0000","last_became_active":"2023-12-01T01:51:23.709381+0000","last_became_peered":"2023-12-01T01:51:23.709381+0000","last_unstale":"2023-12-01T01:51:24.187467+0000","last_undegraded":"2023-12-01T01:51:24.187467+0000","last_fullsized":"2023-12-01T01:51:24.187467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:25.168837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.207004+0000","last_change":"2023-12-01T01:51:24.200936+0000","last_active":"2023-12-01T01:51:24.207004+0000","last_peered":"2023-12-01T01:51:24.207004+0000","last_clean":"2023-12-01T01:51:24.207004+0000","last_became_active":"2023-12-01T01:51:24.198684+0000","last_became_peered":"2023-12-01T01:51:24.198684+0000","last_unstale":"2023-12-01T01:51:24.207004+0000","last_undegraded":"2023-12-01T01:51:24.207004+0000","last_fullsized":"2023-12-01T01:51:24.207004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:50.301817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.188426+0000","last_change":"2023-12-01T01:51:23.725370+0000","last_active":"2023-12-01T01:51:24.188426+0000","last_peered":"2023-12-01T01:51:24.188426+0000","last_clean":"2023-12-01T01:51:24.188426+0000","last_became_active":"2023-12-01T01:51:23.725113+0000","last_became_peered":"2023-12-01T01:51:23.725113+0000","last_unstale":"2023-12-01T01:51:24.188426+0000","last_undegraded":"2023-12-01T01:51:24.188426+0000","last_fullsized":"2023-12-01T01:51:24.188426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:17.602658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217828+0000","last_change":"2023-12-01T01:51:24.216184+0000","last_active":"2023-12-01T01:51:24.217828+0000","last_peered":"2023-12-01T01:51:24.217828+0000","last_clean":"2023-12-01T01:51:24.217828+0000","last_became_active":"2023-12-01T01:51:24.215804+0000","last_became_peered":"2023-12-01T01:51:24.215804+0000","last_unstale":"2023-12-01T01:51:24.217828+0000","last_undegraded":"2023-12-01T01:51:24.217828+0000","last_fullsized":"2023-12-01T01:51:24.217828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:18.788541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187682+0000","last_change":"2023-12-01T01:51:23.733129+0000","last_active":"2023-12-01T01:51:24.187682+0000","last_peered":"2023-12-01T01:51:24.187682+0000","last_clean":"2023-12-01T01:51:24.187682+0000","last_became_active":"2023-12-01T01:51:23.732687+0000","last_became_peered":"2023-12-01T01:51:23.732687+0000","last_unstale":"2023-12-01T01:51:24.187682+0000","last_undegraded":"2023-12-01T01:51:24.187682+0000","last_fullsized":"2023-12-01T01:51:24.187682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:59.339210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218946+0000","last_change":"2023-12-01T01:51:24.213951+0000","last_active":"2023-12-01T01:51:24.218946+0000","last_peered":"2023-12-01T01:51:24.218946+0000","last_clean":"2023-12-01T01:51:24.218946+0000","last_became_active":"2023-12-01T01:51:24.212626+0000","last_became_peered":"2023-12-01T01:51:24.212626+0000","last_unstale":"2023-12-01T01:51:24.218946+0000","last_undegraded":"2023-12-01T01:51:24.218946+0000","last_fullsized":"2023-12-01T01:51:24.218946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:54:45.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"82'3","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205513+0000","last_change":"2023-12-01T01:51:23.723356+0000","last_active":"2023-12-01T01:51:24.205513+0000","last_peered":"2023-12-01T01:51:24.205513+0000","last_clean":"2023-12-01T01:51:24.205513+0000","last_became_active":"2023-12-01T01:51:23.723040+0000","last_became_peered":"2023-12-01T01:51:23.723040+0000","last_unstale":"2023-12-01T01:51:24.205513+0000","last_undegraded":"2023-12-01T01:51:24.205513+0000","last_fullsized":"2023-12-01T01:51:24.205513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:15.500740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.213047+0000","last_change":"2023-12-01T01:51:24.209588+0000","last_active":"2023-12-01T01:51:24.213047+0000","last_peered":"2023-12-01T01:51:24.213047+0000","last_clean":"2023-12-01T01:51:24.213047+0000","last_became_active":"2023-12-01T01:51:24.207327+0000","last_became_peered":"2023-12-01T01:51:24.207327+0000","last_unstale":"2023-12-01T01:51:24.213047+0000","last_undegraded":"2023-12-01T01:51:24.213047+0000","last_fullsized":"2023-12-01T01:51:24.213047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:23.126569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"82'6","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220840+0000","last_change":"2023-12-01T01:51:23.725813+0000","last_active":"2023-12-01T01:51:24.220840+0000","last_peered":"2023-12-01T01:51:24.220840+0000","last_clean":"2023-12-01T01:51:24.220840+0000","last_became_active":"2023-12-01T01:51:23.723452+0000","last_became_peered":"2023-12-01T01:51:23.723452+0000","last_unstale":"2023-12-01T01:51:24.220840+0000","last_undegraded":"2023-12-01T01:51:24.220840+0000","last_fullsized":"2023-12-01T01:51:24.220840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:46.793866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.214150+0000","last_change":"2023-12-01T01:51:24.212037+0000","last_active":"2023-12-01T01:51:24.214150+0000","last_peered":"2023-12-01T01:51:24.214150+0000","last_clean":"2023-12-01T01:51:24.214150+0000","last_became_active":"2023-12-01T01:51:24.211783+0000","last_became_peered":"2023-12-01T01:51:24.211783+0000","last_unstale":"2023-12-01T01:51:24.214150+0000","last_undegraded":"2023-12-01T01:51:24.214150+0000","last_fullsized":"2023-12-01T01:51:24.214150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.469019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215996+0000","last_change":"2023-12-01T01:51:23.732400+0000","last_active":"2023-12-01T01:51:24.215996+0000","last_peered":"2023-12-01T01:51:24.215996+0000","last_clean":"2023-12-01T01:51:24.215996+0000","last_became_active":"2023-12-01T01:51:23.730066+0000","last_became_peered":"2023-12-01T01:51:23.730066+0000","last_unstale":"2023-12-01T01:51:24.215996+0000","last_undegraded":"2023-12-01T01:51:24.215996+0000","last_fullsized":"2023-12-01T01:51:24.215996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:49.297425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.211556+0000","last_change":"2023-12-01T01:51:24.209576+0000","last_active":"2023-12-01T01:51:24.211556+0000","last_peered":"2023-12-01T01:51:24.211556+0000","last_clean":"2023-12-01T01:51:24.211556+0000","last_became_active":"2023-12-01T01:51:24.209297+0000","last_became_peered":"2023-12-01T01:51:24.209297+0000","last_unstale":"2023-12-01T01:51:24.211556+0000","last_undegraded":"2023-12-01T01:51:24.211556+0000","last_fullsized":"2023-12-01T01:51:24.211556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:10.334292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.194995+0000","last_change":"2023-12-01T01:51:23.722740+0000","last_active":"2023-12-01T01:51:24.194995+0000","last_peered":"2023-12-01T01:51:24.194995+0000","last_clean":"2023-12-01T01:51:24.194995+0000","last_became_active":"2023-12-01T01:51:23.722478+0000","last_became_peered":"2023-12-01T01:51:23.722478+0000","last_unstale":"2023-12-01T01:51:24.194995+0000","last_undegraded":"2023-12-01T01:51:24.194995+0000","last_fullsized":"2023-12-01T01:51:24.194995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:01:52.071276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.197625+0000","last_change":"2023-12-01T01:51:24.193465+0000","last_active":"2023-12-01T01:51:24.197625+0000","last_peered":"2023-12-01T01:51:24.197625+0000","last_clean":"2023-12-01T01:51:24.197625+0000","last_became_active":"2023-12-01T01:51:24.193193+0000","last_became_peered":"2023-12-01T01:51:24.193193+0000","last_unstale":"2023-12-01T01:51:24.197625+0000","last_undegraded":"2023-12-01T01:51:24.197625+0000","last_fullsized":"2023-12-01T01:51:24.197625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:33.729872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.194040+0000","last_change":"2023-12-01T01:51:23.725460+0000","last_active":"2023-12-01T01:51:24.194040+0000","last_peered":"2023-12-01T01:51:24.194040+0000","last_clean":"2023-12-01T01:51:24.194040+0000","last_became_active":"2023-12-01T01:51:23.725192+0000","last_became_peered":"2023-12-01T01:51:23.725192+0000","last_unstale":"2023-12-01T01:51:24.194040+0000","last_undegraded":"2023-12-01T01:51:24.194040+0000","last_fullsized":"2023-12-01T01:51:24.194040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:14.033061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.217980+0000","last_change":"2023-12-01T01:51:24.215107+0000","last_active":"2023-12-01T01:51:24.217980+0000","last_peered":"2023-12-01T01:51:24.217980+0000","last_clean":"2023-12-01T01:51:24.217980+0000","last_became_active":"2023-12-01T01:51:24.214837+0000","last_became_peered":"2023-12-01T01:51:24.214837+0000","last_unstale":"2023-12-01T01:51:24.217980+0000","last_undegraded":"2023-12-01T01:51:24.217980+0000","last_fullsized":"2023-12-01T01:51:24.217980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:49.877816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.210435+0000","last_change":"2023-12-01T01:51:23.728019+0000","last_active":"2023-12-01T01:51:24.210435+0000","last_peered":"2023-12-01T01:51:24.210435+0000","last_clean":"2023-12-01T01:51:24.210435+0000","last_became_active":"2023-12-01T01:51:23.727690+0000","last_became_peered":"2023-12-01T01:51:23.727690+0000","last_unstale":"2023-12-01T01:51:24.210435+0000","last_undegraded":"2023-12-01T01:51:24.210435+0000","last_fullsized":"2023-12-01T01:51:24.210435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:54:44.691969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215141+0000","last_change":"2023-12-01T01:51:24.213421+0000","last_active":"2023-12-01T01:51:24.215141+0000","last_peered":"2023-12-01T01:51:24.215141+0000","last_clean":"2023-12-01T01:51:24.215141+0000","last_became_active":"2023-12-01T01:51:24.213163+0000","last_became_peered":"2023-12-01T01:51:24.213163+0000","last_unstale":"2023-12-01T01:51:24.215141+0000","last_undegraded":"2023-12-01T01:51:24.215141+0000","last_fullsized":"2023-12-01T01:51:24.215141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:14.157429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224213+0000","last_change":"2023-12-01T01:51:23.733218+0000","last_active":"2023-12-01T01:51:24.224213+0000","last_peered":"2023-12-01T01:51:24.224213+0000","last_clean":"2023-12-01T01:51:24.224213+0000","last_became_active":"2023-12-01T01:51:23.732910+0000","last_became_peered":"2023-12-01T01:51:23.732910+0000","last_unstale":"2023-12-01T01:51:24.224213+0000","last_undegraded":"2023-12-01T01:51:24.224213+0000","last_fullsized":"2023-12-01T01:51:24.224213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:54.751334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215104+0000","last_change":"2023-12-01T01:51:24.212578+0000","last_active":"2023-12-01T01:51:24.215104+0000","last_peered":"2023-12-01T01:51:24.215104+0000","last_clean":"2023-12-01T01:51:24.215104+0000","last_became_active":"2023-12-01T01:51:24.212344+0000","last_became_peered":"2023-12-01T01:51:24.212344+0000","last_unstale":"2023-12-01T01:51:24.215104+0000","last_undegraded":"2023-12-01T01:51:24.215104+0000","last_fullsized":"2023-12-01T01:51:24.215104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:23:00.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218291+0000","last_change":"2023-12-01T01:51:23.733533+0000","last_active":"2023-12-01T01:51:24.218291+0000","last_peered":"2023-12-01T01:51:24.218291+0000","last_clean":"2023-12-01T01:51:24.218291+0000","last_became_active":"2023-12-01T01:51:23.733196+0000","last_became_peered":"2023-12-01T01:51:23.733196+0000","last_unstale":"2023-12-01T01:51:24.218291+0000","last_undegraded":"2023-12-01T01:51:24.218291+0000","last_fullsized":"2023-12-01T01:51:24.218291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:09.329899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.222453+0000","last_change":"2023-12-01T01:51:24.213793+0000","last_active":"2023-12-01T01:51:24.222453+0000","last_peered":"2023-12-01T01:51:24.222453+0000","last_clean":"2023-12-01T01:51:24.222453+0000","last_became_active":"2023-12-01T01:51:24.213365+0000","last_became_peered":"2023-12-01T01:51:24.213365+0000","last_unstale":"2023-12-01T01:51:24.222453+0000","last_undegraded":"2023-12-01T01:51:24.222453+0000","last_fullsized":"2023-12-01T01:51:24.222453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:12:27.401881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.222133+0000","last_change":"2023-12-01T01:51:23.728528+0000","last_active":"2023-12-01T01:51:24.222133+0000","last_peered":"2023-12-01T01:51:24.222133+0000","last_clean":"2023-12-01T01:51:24.222133+0000","last_became_active":"2023-12-01T01:51:23.728266+0000","last_became_peered":"2023-12-01T01:51:23.728266+0000","last_unstale":"2023-12-01T01:51:24.222133+0000","last_undegraded":"2023-12-01T01:51:24.222133+0000","last_fullsized":"2023-12-01T01:51:24.222133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:18:10.249852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223947+0000","last_change":"2023-12-01T01:51:24.222537+0000","last_active":"2023-12-01T01:51:24.223947+0000","last_peered":"2023-12-01T01:51:24.223947+0000","last_clean":"2023-12-01T01:51:24.223947+0000","last_became_active":"2023-12-01T01:51:24.222305+0000","last_became_peered":"2023-12-01T01:51:24.222305+0000","last_unstale":"2023-12-01T01:51:24.223947+0000","last_undegraded":"2023-12-01T01:51:24.223947+0000","last_fullsized":"2023-12-01T01:51:24.223947+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:03.279418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.189727+0000","last_change":"2023-12-01T01:51:23.733364+0000","last_active":"2023-12-01T01:51:24.189727+0000","last_peered":"2023-12-01T01:51:24.189727+0000","last_clean":"2023-12-01T01:51:24.189727+0000","last_became_active":"2023-12-01T01:51:23.732811+0000","last_became_peered":"2023-12-01T01:51:23.732811+0000","last_unstale":"2023-12-01T01:51:24.189727+0000","last_undegraded":"2023-12-01T01:51:24.189727+0000","last_fullsized":"2023-12-01T01:51:24.189727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:53:17.784149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.218065+0000","last_change":"2023-12-01T01:51:24.216189+0000","last_active":"2023-12-01T01:51:24.218065+0000","last_peered":"2023-12-01T01:51:24.218065+0000","last_clean":"2023-12-01T01:51:24.218065+0000","last_became_active":"2023-12-01T01:51:24.215930+0000","last_became_peered":"2023-12-01T01:51:24.215930+0000","last_unstale":"2023-12-01T01:51:24.218065+0000","last_undegraded":"2023-12-01T01:51:24.218065+0000","last_fullsized":"2023-12-01T01:51:24.218065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:57.588932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.216083+0000","last_change":"2023-12-01T01:51:23.717087+0000","last_active":"2023-12-01T01:51:24.216083+0000","last_peered":"2023-12-01T01:51:24.216083+0000","last_clean":"2023-12-01T01:51:24.216083+0000","last_became_active":"2023-12-01T01:51:23.712593+0000","last_became_peered":"2023-12-01T01:51:23.712593+0000","last_unstale":"2023-12-01T01:51:24.216083+0000","last_undegraded":"2023-12-01T01:51:24.216083+0000","last_fullsized":"2023-12-01T01:51:24.216083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:19:09.329899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205309+0000","last_change":"2023-12-01T01:51:24.200939+0000","last_active":"2023-12-01T01:51:24.205309+0000","last_peered":"2023-12-01T01:51:24.205309+0000","last_clean":"2023-12-01T01:51:24.205309+0000","last_became_active":"2023-12-01T01:51:24.200488+0000","last_became_peered":"2023-12-01T01:51:24.200488+0000","last_unstale":"2023-12-01T01:51:24.205309+0000","last_undegraded":"2023-12-01T01:51:24.205309+0000","last_fullsized":"2023-12-01T01:51:24.205309+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:09.806552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187843+0000","last_change":"2023-12-01T01:51:23.732919+0000","last_active":"2023-12-01T01:51:24.187843+0000","last_peered":"2023-12-01T01:51:24.187843+0000","last_clean":"2023-12-01T01:51:24.187843+0000","last_became_active":"2023-12-01T01:51:23.732477+0000","last_became_peered":"2023-12-01T01:51:23.732477+0000","last_unstale":"2023-12-01T01:51:24.187843+0000","last_undegraded":"2023-12-01T01:51:24.187843+0000","last_fullsized":"2023-12-01T01:51:24.187843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:28.105668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.204853+0000","last_change":"2023-12-01T01:51:24.198835+0000","last_active":"2023-12-01T01:51:24.204853+0000","last_peered":"2023-12-01T01:51:24.204853+0000","last_clean":"2023-12-01T01:51:24.204853+0000","last_became_active":"2023-12-01T01:51:24.198426+0000","last_became_peered":"2023-12-01T01:51:24.198426+0000","last_unstale":"2023-12-01T01:51:24.204853+0000","last_undegraded":"2023-12-01T01:51:24.204853+0000","last_fullsized":"2023-12-01T01:51:24.204853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:55.560831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187402+0000","last_change":"2023-12-01T01:51:23.732268+0000","last_active":"2023-12-01T01:51:24.187402+0000","last_peered":"2023-12-01T01:51:24.187402+0000","last_clean":"2023-12-01T01:51:24.187402+0000","last_became_active":"2023-12-01T01:51:23.730931+0000","last_became_peered":"2023-12-01T01:51:23.730931+0000","last_unstale":"2023-12-01T01:51:24.187402+0000","last_undegraded":"2023-12-01T01:51:24.187402+0000","last_fullsized":"2023-12-01T01:51:24.187402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:43.652247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.223795+0000","last_change":"2023-12-01T01:51:24.222725+0000","last_active":"2023-12-01T01:51:24.223795+0000","last_peered":"2023-12-01T01:51:24.223795+0000","last_clean":"2023-12-01T01:51:24.223795+0000","last_became_active":"2023-12-01T01:51:24.222166+0000","last_became_peered":"2023-12-01T01:51:24.222166+0000","last_unstale":"2023-12-01T01:51:24.223795+0000","last_undegraded":"2023-12-01T01:51:24.223795+0000","last_fullsized":"2023-12-01T01:51:24.223795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:55:18.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221810+0000","last_change":"2023-12-01T01:51:23.735050+0000","last_active":"2023-12-01T01:51:24.221810+0000","last_peered":"2023-12-01T01:51:24.221810+0000","last_clean":"2023-12-01T01:51:24.221810+0000","last_became_active":"2023-12-01T01:51:23.734653+0000","last_became_peered":"2023-12-01T01:51:23.734653+0000","last_unstale":"2023-12-01T01:51:24.221810+0000","last_undegraded":"2023-12-01T01:51:24.221810+0000","last_fullsized":"2023-12-01T01:51:24.221810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:06.069833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215432+0000","last_change":"2023-12-01T01:51:24.213857+0000","last_active":"2023-12-01T01:51:24.215432+0000","last_peered":"2023-12-01T01:51:24.215432+0000","last_clean":"2023-12-01T01:51:24.215432+0000","last_became_active":"2023-12-01T01:51:24.213485+0000","last_became_peered":"2023-12-01T01:51:24.213485+0000","last_unstale":"2023-12-01T01:51:24.215432+0000","last_undegraded":"2023-12-01T01:51:24.215432+0000","last_fullsized":"2023-12-01T01:51:24.215432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:34:25.329128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220448+0000","last_change":"2023-12-01T01:51:23.724706+0000","last_active":"2023-12-01T01:51:24.220448+0000","last_peered":"2023-12-01T01:51:24.220448+0000","last_clean":"2023-12-01T01:51:24.220448+0000","last_became_active":"2023-12-01T01:51:23.723272+0000","last_became_peered":"2023-12-01T01:51:23.723272+0000","last_unstale":"2023-12-01T01:51:24.220448+0000","last_undegraded":"2023-12-01T01:51:24.220448+0000","last_fullsized":"2023-12-01T01:51:24.220448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:51.874903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.222668+0000","last_change":"2023-12-01T01:51:24.216311+0000","last_active":"2023-12-01T01:51:24.222668+0000","last_peered":"2023-12-01T01:51:24.222668+0000","last_clean":"2023-12-01T01:51:24.222668+0000","last_became_active":"2023-12-01T01:51:24.213698+0000","last_became_peered":"2023-12-01T01:51:24.213698+0000","last_unstale":"2023-12-01T01:51:24.222668+0000","last_undegraded":"2023-12-01T01:51:24.222668+0000","last_fullsized":"2023-12-01T01:51:24.222668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:47.798259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187078+0000","last_change":"2023-12-01T01:51:23.721602+0000","last_active":"2023-12-01T01:51:24.187078+0000","last_peered":"2023-12-01T01:51:24.187078+0000","last_clean":"2023-12-01T01:51:24.187078+0000","last_became_active":"2023-12-01T01:51:23.721352+0000","last_became_peered":"2023-12-01T01:51:23.721352+0000","last_unstale":"2023-12-01T01:51:24.187078+0000","last_undegraded":"2023-12-01T01:51:24.187078+0000","last_fullsized":"2023-12-01T01:51:24.187078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:44.064139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.205130+0000","last_change":"2023-12-01T01:51:24.201302+0000","last_active":"2023-12-01T01:51:24.205130+0000","last_peered":"2023-12-01T01:51:24.205130+0000","last_clean":"2023-12-01T01:51:24.205130+0000","last_became_active":"2023-12-01T01:51:24.200899+0000","last_became_peered":"2023-12-01T01:51:24.200899+0000","last_unstale":"2023-12-01T01:51:24.205130+0000","last_undegraded":"2023-12-01T01:51:24.205130+0000","last_fullsized":"2023-12-01T01:51:24.205130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:13.923540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.201716+0000","last_change":"2023-12-01T01:51:23.732583+0000","last_active":"2023-12-01T01:51:24.201716+0000","last_peered":"2023-12-01T01:51:24.201716+0000","last_clean":"2023-12-01T01:51:24.201716+0000","last_became_active":"2023-12-01T01:51:23.731083+0000","last_became_peered":"2023-12-01T01:51:23.731083+0000","last_unstale":"2023-12-01T01:51:24.201716+0000","last_undegraded":"2023-12-01T01:51:24.201716+0000","last_fullsized":"2023-12-01T01:51:24.201716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:36.702370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.224156+0000","last_change":"2023-12-01T01:51:24.220076+0000","last_active":"2023-12-01T01:51:24.224156+0000","last_peered":"2023-12-01T01:51:24.224156+0000","last_clean":"2023-12-01T01:51:24.224156+0000","last_became_active":"2023-12-01T01:51:24.216516+0000","last_became_peered":"2023-12-01T01:51:24.216516+0000","last_unstale":"2023-12-01T01:51:24.224156+0000","last_undegraded":"2023-12-01T01:51:24.224156+0000","last_fullsized":"2023-12-01T01:51:24.224156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:06.708100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.199761+0000","last_change":"2023-12-01T01:51:23.732282+0000","last_active":"2023-12-01T01:51:24.199761+0000","last_peered":"2023-12-01T01:51:24.199761+0000","last_clean":"2023-12-01T01:51:24.199761+0000","last_became_active":"2023-12-01T01:51:23.732016+0000","last_became_peered":"2023-12-01T01:51:23.732016+0000","last_unstale":"2023-12-01T01:51:24.199761+0000","last_undegraded":"2023-12-01T01:51:24.199761+0000","last_fullsized":"2023-12-01T01:51:24.199761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:50.767646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.210347+0000","last_change":"2023-12-01T01:51:24.204531+0000","last_active":"2023-12-01T01:51:24.210347+0000","last_peered":"2023-12-01T01:51:24.210347+0000","last_clean":"2023-12-01T01:51:24.210347+0000","last_became_active":"2023-12-01T01:51:24.204155+0000","last_became_peered":"2023-12-01T01:51:24.204155+0000","last_unstale":"2023-12-01T01:51:24.210347+0000","last_undegraded":"2023-12-01T01:51:24.210347+0000","last_fullsized":"2023-12-01T01:51:24.210347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:02:29.110060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.201449+0000","last_change":"2023-12-01T01:51:23.720558+0000","last_active":"2023-12-01T01:51:24.201449+0000","last_peered":"2023-12-01T01:51:24.201449+0000","last_clean":"2023-12-01T01:51:24.201449+0000","last_became_active":"2023-12-01T01:51:23.720288+0000","last_became_peered":"2023-12-01T01:51:23.720288+0000","last_unstale":"2023-12-01T01:51:24.201449+0000","last_undegraded":"2023-12-01T01:51:24.201449+0000","last_fullsized":"2023-12-01T01:51:24.201449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:54:32.273772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.215064+0000","last_change":"2023-12-01T01:51:24.209551+0000","last_active":"2023-12-01T01:51:24.215064+0000","last_peered":"2023-12-01T01:51:24.215064+0000","last_clean":"2023-12-01T01:51:24.215064+0000","last_became_active":"2023-12-01T01:51:24.209232+0000","last_became_peered":"2023-12-01T01:51:24.209232+0000","last_unstale":"2023-12-01T01:51:24.215064+0000","last_undegraded":"2023-12-01T01:51:24.215064+0000","last_fullsized":"2023-12-01T01:51:24.215064+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:15.037453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.213057+0000","last_change":"2023-12-01T01:51:23.730111+0000","last_active":"2023-12-01T01:51:24.213057+0000","last_peered":"2023-12-01T01:51:24.213057+0000","last_clean":"2023-12-01T01:51:24.213057+0000","last_became_active":"2023-12-01T01:51:23.729813+0000","last_became_peered":"2023-12-01T01:51:23.729813+0000","last_unstale":"2023-12-01T01:51:24.213057+0000","last_undegraded":"2023-12-01T01:51:24.213057+0000","last_fullsized":"2023-12-01T01:51:24.213057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:06.524585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.221809+0000","last_change":"2023-12-01T01:51:24.213668+0000","last_active":"2023-12-01T01:51:24.221809+0000","last_peered":"2023-12-01T01:51:24.221809+0000","last_clean":"2023-12-01T01:51:24.221809+0000","last_became_active":"2023-12-01T01:51:24.213225+0000","last_became_peered":"2023-12-01T01:51:24.213225+0000","last_unstale":"2023-12-01T01:51:24.221809+0000","last_undegraded":"2023-12-01T01:51:24.221809+0000","last_fullsized":"2023-12-01T01:51:24.221809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.360859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.188190+0000","last_change":"2023-12-01T01:51:23.731063+0000","last_active":"2023-12-01T01:51:24.188190+0000","last_peered":"2023-12-01T01:51:24.188190+0000","last_clean":"2023-12-01T01:51:24.188190+0000","last_became_active":"2023-12-01T01:51:23.730608+0000","last_became_peered":"2023-12-01T01:51:23.730608+0000","last_unstale":"2023-12-01T01:51:24.188190+0000","last_undegraded":"2023-12-01T01:51:24.188190+0000","last_fullsized":"2023-12-01T01:51:24.188190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:34:24.324736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.220336+0000","last_change":"2023-12-01T01:51:24.217170+0000","last_active":"2023-12-01T01:51:24.220336+0000","last_peered":"2023-12-01T01:51:24.220336+0000","last_clean":"2023-12-01T01:51:24.220336+0000","last_became_active":"2023-12-01T01:51:24.215927+0000","last_became_peered":"2023-12-01T01:51:24.215927+0000","last_unstale":"2023-12-01T01:51:24.220336+0000","last_undegraded":"2023-12-01T01:51:24.220336+0000","last_fullsized":"2023-12-01T01:51:24.220336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:23.688067+0000","last_clean_scrub_stamp":"2023-12-01T01:51:23.688067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:31:57.782696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.186053+0000","last_change":"2023-12-01T01:51:23.709986+0000","last_active":"2023-12-01T01:51:24.186053+0000","last_peered":"2023-12-01T01:51:24.186053+0000","last_clean":"2023-12-01T01:51:24.186053+0000","last_became_active":"2023-12-01T01:51:23.709276+0000","last_became_peered":"2023-12-01T01:51:23.709276+0000","last_unstale":"2023-12-01T01:51:24.186053+0000","last_undegraded":"2023-12-01T01:51:24.186053+0000","last_fullsized":"2023-12-01T01:51:24.186053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:51:22.683675+0000","last_clean_scrub_stamp":"2023-12-01T01:51:22.683675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:06:05.428917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"70'161","reported_seq":382,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:51:24.187487+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:51:24.187487+0000","last_peered":"2023-12-01T01:51:24.187487+0000","last_clean":"2023-12-01T01:51:24.187487+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:51:24.187487+0000","last_undegraded":"2023-12-01T01:51:24.187487+0000","last_fullsized":"2023-12-01T01:51:24.187487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":1011712,"data_stored":475605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.79700000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3979999999999999,"5min":1.5549999999999999,"15min":1.8089999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.073,"15min":1.05},"min":{"1min":0.82899999999999996,"5min":0.64700000000000002,"15min":0.64000000000000001},"max":{"1min":1.353,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.246,"15min":1.218},"min":{"1min":1.008,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.802,"5min":1.8100000000000001,"15min":1.8420000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2629999999999999,"15min":1.2569999999999999},"min":{"1min":1.0669999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5529999999999999,"5min":1.744,"15min":2.0870000000000002},"last":1.117}]},{"osd":2,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.121,"15min":1.123},"min":{"1min":0.80100000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5329999999999999,"5min":1.728,"15min":1.728},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.218,"15min":1.2290000000000001},"min":{"1min":0.92400000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.6120000000000001,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.1579999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2430000000000001,"15min":1.234},"min":{"1min":0.876,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.617,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.355,"15min":1.349},"min":{"1min":1.026,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6859999999999999,"5min":2.1880000000000002,"15min":2.7050000000000001},"last":1.254}]},{"osd":4,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.2769999999999999,"15min":1.296},"min":{"1min":0.83099999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.681,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.036},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4610000000000001,"15min":1.4199999999999999},"min":{"1min":1.0609999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7370000000000001,"5min":2.0939999999999999,"15min":2.601},"last":1.349}]},{"osd":5,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.274,"15min":1.27},"min":{"1min":0.83099999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.6439999999999999,"5min":2.0870000000000002,"15min":2.4500000000000002},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5309999999999999,"15min":1.526},"min":{"1min":1,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8839999999999999,"5min":2.2930000000000001,"15min":2.8069999999999999},"last":1.5620000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:50:29 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.393,"15min":1.3859999999999999},"min":{"1min":1.004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.77,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":1.002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7929999999999999,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.3080000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":475535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.101,"15min":1.1060000000000001},"min":{"1min":0.78900000000000003,"5min":0.69699999999999995,"15min":0.66900000000000004},"max":{"1min":1.619,"5min":1.635,"15min":1.79},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.139,"5min":1.137,"15min":1.1299999999999999},"min":{"1min":0.89600000000000002,"5min":0.64400000000000002,"15min":0.61799999999999999},"max":{"1min":1.4690000000000001,"5min":2.5569999999999999,"15min":2.73},"last":1.292}]},{"osd":1,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.167,"15min":1.153},"min":{"1min":0.94999999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6759999999999999,"5min":1.8620000000000001,"15min":2.2170000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.196,"15min":1.21},"min":{"1min":0.93400000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6259999999999999,"5min":1.651,"15min":2.0920000000000001},"last":1.149}]},{"osd":2,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.232,"15min":1.163},"min":{"1min":0.92000000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6359999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.27,"15min":1.25},"min":{"1min":0.877,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7549999999999999,"5min":2.137,"15min":2.137},"last":1.5549999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3009999999999999,"15min":1.2849999999999999},"min":{"1min":1.111,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.548,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.4259999999999999,"15min":1.407},"min":{"1min":1.137,"5min":1.085,"15min":0.81299999999999994},"max":{"1min":1.7130000000000001,"5min":1.758,"15min":2.4020000000000001},"last":1.3320000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.321,"15min":1.2909999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.70699999999999996},"max":{"1min":1.6919999999999999,"5min":2.226,"15min":2.3399999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.365,"15min":1.3580000000000001},"min":{"1min":0.94699999999999995,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.694,"5min":1.931,"15min":2.6760000000000002},"last":1.413}]},{"osd":5,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4419999999999999,"15min":1.4179999999999999},"min":{"1min":1.0469999999999999,"5min":1.0229999999999999,"15min":0.82699999999999996},"max":{"1min":1.726,"5min":2.1899999999999999,"15min":2.5539999999999998},"last":1.605},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5109999999999999,"15min":1.482},"min":{"1min":1.1870000000000001,"5min":0.98899999999999999,"15min":0.748},"max":{"1min":1.8129999999999999,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.0389999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:50:30 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5660000000000001,"15min":1.5249999999999999},"min":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":0.89500000000000002},"max":{"1min":1.978,"5min":2.762,"15min":2.762},"last":1.651},{"interface":"front","average":{"1min":1.587,"5min":1.621,"15min":1.585},"min":{"1min":1.218,"5min":1.1479999999999999,"15min":0.78700000000000003},"max":{"1min":1.9259999999999999,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.234}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1564,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1601536,"data_stored":1065915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.1060000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.498,"5min":1.651,"15min":3.3980000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.127,"15min":1.141},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.60099999999999998},"max":{"1min":1.5720000000000001,"5min":1.6479999999999999,"15min":2.153},"last":1.5109999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.153,"15min":1.137},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.7010000000000001},"last":1.472},{"interface":"front","average":{"1min":1.179,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.73599999999999999,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.94099999999999995}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4039999999999999,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.093},{"interface":"front","average":{"1min":1.155,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.6080000000000001,"5min":1.621,"15min":2.0630000000000002},"last":1.2090000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.286},"min":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.76900000000000002},"max":{"1min":1.5309999999999999,"5min":1.694,"15min":1.833},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3120000000000001,"15min":1.296},"min":{"1min":0.98099999999999998,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":1.6779999999999999,"5min":1.7110000000000001,"15min":2.1440000000000001},"last":1.663}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.238,"15min":1.2390000000000001},"min":{"1min":0.86399999999999999,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.6779999999999999,"5min":1.853,"15min":2.5129999999999999},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.84999999999999998,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.6319999999999999,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.5900000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2989999999999999,"15min":1.329},"min":{"1min":0.88500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":2.2970000000000002},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":1.125,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":1.962,"5min":1.962,"15min":3.23},"last":1.625}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4530000000000001,"15min":1.48},"min":{"1min":0.92400000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2,"5min":2,"15min":3.012},"last":2.0499999999999998},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4970000000000001,"15min":1.524},"min":{"1min":1.2689999999999999,"5min":0.873,"15min":0.873},"max":{"1min":2.073,"5min":2.073,"15min":2.8420000000000001},"last":2.1299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":1011712,"data_stored":476129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.119,"15min":1.1040000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.53,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.734,"5min":1.827,"15min":3.8069999999999999},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2350000000000001,"15min":1.2090000000000001},"min":{"1min":0.84599999999999997,"5min":0.81999999999999995,"15min":0.65900000000000003},"max":{"1min":1.528,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.452},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.131},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.671,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.995}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2110000000000001,"15min":1.196},"min":{"1min":0.83999999999999997,"5min":0.78200000000000003,"15min":0.77600000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3140000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.871,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.5780000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.234},"min":{"1min":0.85399999999999998,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.8220000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.095},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3200000000000001,"15min":1.319},"min":{"1min":0.753,"5min":0.753,"15min":0.70399999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.3010000000000002},"last":1.5149999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.96599999999999997,"5min":0.79200000000000004,"15min":0.61899999999999999},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.1549999999999998},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.2829999999999999,"15min":1.292},"min":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.5189999999999999,"5min":1.8029999999999999,"15min":2.2029999999999998},"last":1.2130000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.94999999999999996,"5min":0.84799999999999998,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.619},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.5029999999999999,"15min":1.4730000000000001},"min":{"1min":1.1000000000000001,"5min":0.96699999999999997,"15min":0.63200000000000001},"max":{"1min":1.7949999999999999,"5min":2.4990000000000001,"15min":2.54},"last":1.6950000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.458,"15min":1.464},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.60599999999999998},"max":{"1min":1.6850000000000001,"5min":1.972,"15min":3.2789999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5720000000000001,"15min":1.5580000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.04,"15min":2.157},"last":1.728}]}]},{"osd":3,"up_from":13,"seq":55834575035,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40224,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331616,"statfs":{"total":96636764160,"available":96595574784,"internally_reserved":0,"allocated":1007616,"data_stored":475603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82899999999999996,"15min":0.85199999999999998},"min":{"1min":0.46400000000000002,"5min":0.42699999999999999,"15min":0.307},"max":{"1min":1.135,"5min":1.302,"15min":1.583},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.999,"15min":1.0009999999999999},"min":{"1min":0.52100000000000002,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.3999999999999999,"5min":1.4990000000000001,"15min":1.9770000000000001},"last":1.2230000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.069,"15min":1.0629999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.38900000000000001},"max":{"1min":1.556,"5min":1.571,"15min":2.0779999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.95399999999999996,"15min":0.95199999999999996},"min":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.40600000000000003},"max":{"1min":1.5129999999999999,"5min":1.587,"15min":1.6519999999999999},"last":0.93600000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.016,"15min":1.054},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.436},"max":{"1min":1.286,"5min":1.512,"15min":1.766},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.173,"15min":1.1759999999999999},"min":{"1min":0.63800000000000001,"5min":0.621,"15min":0.48399999999999999},"max":{"1min":1.4870000000000001,"5min":1.647,"15min":1.661},"last":0.97699999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.238,"15min":1.2569999999999999},"min":{"1min":0.96499999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.476,"5min":1.6240000000000001,"15min":2.9350000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.232,"15min":1.246},"min":{"1min":0.94099999999999995,"5min":0.81000000000000005,"15min":0.73299999999999998},"max":{"1min":1.554,"5min":1.637,"15min":2.2389999999999999},"last":1.427}]},{"osd":5,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.272,"15min":1.302},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.63500000000000001},"max":{"1min":1.5900000000000001,"5min":1.806,"15min":3.476},"last":1.123},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2729999999999999,"15min":1.329},"min":{"1min":0.86099999999999999,"5min":0.82099999999999995,"15min":0.64200000000000002},"max":{"1min":1.464,"5min":1.9039999999999999,"15min":3.181},"last":1.2569999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.88900000000000001,"5min":0.80400000000000005,"15min":0.68300000000000005},"max":{"1min":1.6499999999999999,"5min":1.8580000000000001,"15min":2.532},"last":1.294},{"interface":"front","average":{"1min":1.409,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":1.02,"5min":0.94299999999999995,"15min":0.872},"max":{"1min":1.573,"5min":1.8049999999999999,"15min":3.0249999999999999},"last":1.393}]},{"osd":7,"last update":"Fri Dec 1 01:50:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3919999999999999,"15min":1.4239999999999999},"min":{"1min":1.006,"5min":0.999,"15min":0.81499999999999995},"max":{"1min":1.504,"5min":1.851,"15min":2.1619999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4259999999999999,"15min":1.472},"min":{"1min":1.048,"5min":0.88300000000000001,"15min":0.83099999999999996},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":2.4590000000000001},"last":1.5189999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575035,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":1024000,"data_stored":477237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27100000000000002},"max":{"1min":1.0860000000000001,"5min":1.339,"15min":1.738},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.026,"15min":1.01},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.375},"max":{"1min":1.4159999999999999,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":0.71099999999999997}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96499999999999997,"15min":0.98999999999999999},"min":{"1min":0.54700000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.1779999999999999,"5min":1.45,"15min":1.7130000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0720000000000001,"15min":1.0840000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.504},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.1719999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97499999999999998,"15min":1.0029999999999999},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.377,"5min":1.625,"15min":1.883},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.39600000000000002},"max":{"1min":1.415,"5min":1.613,"15min":2.6080000000000001},"last":1.2789999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.331,"15min":1.3180000000000001},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.75},"max":{"1min":1.7230000000000001,"5min":1.9950000000000001,"15min":2.2829999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.246,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.964},"last":1.4850000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3009999999999999,"15min":1.319},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.647,"5min":1.804,"15min":2.5779999999999998},"last":1.194},{"interface":"front","average":{"1min":1.375,"5min":1.387,"15min":1.375},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.901,"5min":2.048,"15min":3.0099999999999998},"last":1.0620000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.3200000000000001},"min":{"1min":0.84299999999999997,"5min":0.83399999999999996,"15min":0.77500000000000002},"max":{"1min":1.6439999999999999,"5min":1.78,"15min":2.2799999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.766,"15min":2.5089999999999999},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.456,"15min":1.448},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.79400000000000004},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.448},"last":1.331},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7649999999999999,"5min":1.802,"15min":2.5049999999999999},"last":1.5429999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":1564,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":1601536,"data_stored":1066463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85699999999999998,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.53800000000000003,"15min":0.318},"max":{"1min":1.1220000000000001,"5min":1.5049999999999999,"15min":3.056},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.999,"15min":1.012},"min":{"1min":0.75700000000000001,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":1.335,"5min":1.6359999999999999,"15min":3.4129999999999998},"last":1.4570000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.88200000000000001,"15min":0.90100000000000002},"min":{"1min":0.67100000000000004,"5min":0.497,"15min":0.36399999999999999},"max":{"1min":1.1779999999999999,"5min":1.54,"15min":3.3660000000000001},"last":0.877},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.069,"15min":1.103},"min":{"1min":0.78100000000000003,"5min":0.47899999999999998,"15min":0.39000000000000001},"max":{"1min":1.4890000000000001,"5min":1.492,"15min":3.4980000000000002},"last":0.998}]},{"osd":3,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.21,"15min":1.224},"min":{"1min":1.0860000000000001,"5min":0.51900000000000002,"15min":0.44},"max":{"1min":1.6859999999999999,"5min":1.7290000000000001,"15min":3.464},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.077,"15min":1.0669999999999999},"min":{"1min":0.82599999999999996,"5min":0.499,"15min":0.39700000000000002},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":3.0960000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1719999999999999,"15min":1.196},"min":{"1min":1.014,"5min":0.745,"15min":0.63800000000000001},"max":{"1min":1.494,"5min":1.536,"15min":3.7810000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.3240000000000001,"15min":1.3600000000000001},"min":{"1min":1.077,"5min":0.77500000000000002,"15min":0.76600000000000001},"max":{"1min":1.585,"5min":1.8089999999999999,"15min":4.7270000000000003},"last":1.2490000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.296,"15min":1.2909999999999999},"min":{"1min":1.032,"5min":0.90600000000000003,"15min":0.61199999999999999},"max":{"1min":1.5760000000000001,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.681},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.278,"15min":1.3160000000000001},"min":{"1min":1.004,"5min":0.76900000000000002,"15min":0.69199999999999995},"max":{"1min":1.5,"5min":1.825,"15min":3.6669999999999998},"last":1.4259999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.349},"min":{"1min":1.0760000000000001,"5min":0.71099999999999997,"15min":0.68400000000000005},"max":{"1min":1.5549999999999999,"5min":1.7809999999999999,"15min":3.3300000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.343,"15min":1.363},"min":{"1min":1.0649999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.595,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.5389999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:50:28 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.409,"15min":1.4630000000000001},"min":{"1min":1.246,"5min":0.76600000000000001,"15min":0.70599999999999996},"max":{"1min":1.645,"5min":1.8939999999999999,"15min":4.6749999999999998},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.4179999999999999,"15min":1.4299999999999999},"min":{"1min":1.1639999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7589999999999999,"5min":1.992,"15min":3.5379999999999998},"last":1.49}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":1015808,"data_stored":477077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.60699999999999998,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.3520000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.92600000000000005,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4390000000000001,"5min":1.5580000000000001,"15min":1.6910000000000001},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98499999999999999,"15min":1.006},"min":{"1min":0.67600000000000005,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.4690000000000001,"5min":2.157,"15min":2.157},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.113,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.84299999999999997,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.395,"5min":1.728,"15min":2.8159999999999998},"last":1.016}]},{"osd":3,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.58499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.954},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.143,"15min":1.169},"min":{"1min":0.54900000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7070000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1060000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.3089999999999999},"min":{"1min":1.05,"5min":0.78500000000000003,"15min":0.65700000000000003},"max":{"1min":1.5289999999999999,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.214,"15min":1.2629999999999999},"min":{"1min":0.83299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.2829999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.337,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.71899999999999997},"max":{"1min":1.839,"5min":2.351,"15min":2.351},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.306,"15min":1.349},"min":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6299999999999999,"5min":2.2909999999999999,"15min":2.7050000000000001},"last":1.2310000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73099999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.157},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.399},"min":{"1min":1.0129999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.256}]},{"osd":7,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":1.127,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.706,"5min":1.798,"15min":2.8999999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.361,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:51:34.584 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T01:51:34.584 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T01:51:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.849+0000 7fb2f28fb640 1 Processor -- start 2023-12-01T01:51:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.850+0000 7fb2f28fb640 1 -- start start 2023-12-01T01:51:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.851+0000 7fb2f28fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb2ec143240 0x7fb2ec0b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.851+0000 7fb2f28fb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb2ec06caa0 0x7fb2ec0ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.851+0000 7fb2f28fb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ec0b2b00 con 0x7fb2ec0b25e0 2023-12-01T01:51:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.851+0000 7fb2f28fb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ec0b2de0 con 0x7fb2ec143240 2023-12-01T01:51:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.851+0000 7fb2f28fb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ec0b30c0 con 0x7fb2ec06caa0 2023-12-01T01:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.852+0000 7fb2eb7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb2ec0b25e0 0x7fb2ec06c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59360/0 (socket says 172.21.15.115:59360) 2023-12-01T01:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.852+0000 7fb2ebfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb2ec143240 0x7fb2ec0b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.852+0000 7fb2eb7fe640 1 -- 172.21.15.115:0/3881704210 learned_addr learned my addr 172.21.15.115:0/3881704210 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.852+0000 7fb2ebfff640 1 -- 172.21.15.115:0/3881704210 >> v2:172.21.15.142:3300/0 conn(0x7fb2ec06caa0 msgr2=0x7fb2ec0ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.852+0000 7fb2f0e98640 1 --2- 172.21.15.115:0/3881704210 >> v2:172.21.15.142:3300/0 conn(0x7fb2ec06caa0 0x7fb2ec0ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.852+0000 7fb2ebfff640 1 --2- 172.21.15.115:0/3881704210 >> v2:172.21.15.142:3300/0 conn(0x7fb2ec06caa0 0x7fb2ec0ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.852+0000 7fb2ebfff640 1 -- 172.21.15.115:0/3881704210 >> v1:172.21.15.115:6789/0 conn(0x7fb2ec0b25e0 legacy=0x7fb2ec06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.852+0000 7fb2ebfff640 1 -- 172.21.15.115:0/3881704210 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0b33a0 con 0x7fb2ec143240 2023-12-01T01:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.852+0000 7fb2f0e98640 1 --2- 172.21.15.115:0/3881704210 >> v2:172.21.15.142:3300/0 conn(0x7fb2ec06caa0 0x7fb2ec0ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.852+0000 7fb2eaffd640 1 -- 172.21.15.115:0/3881704210 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330214969 0 0) 0x7fb2ec0b2b00 con 0x7fb2ec0b25e0 2023-12-01T01:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.853+0000 7fb2ebfff640 1 --2- 172.21.15.115:0/3881704210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec143240 0x7fb2ec0b20a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec159fe0 tx=0x7fb2dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fcf540e2a03381 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2eaffd640 1 -- 172.21.15.115:0/3881704210 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc031830 con 0x7fb2ec143240 2023-12-01T01:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2eaffd640 1 -- 172.21.15.115:0/3881704210 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc031990 con 0x7fb2ec143240 2023-12-01T01:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2eaffd640 1 -- 172.21.15.115:0/3881704210 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc031d20 con 0x7fb2ec143240 2023-12-01T01:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2f28fb640 1 -- 172.21.15.115:0/3881704210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec143240 msgr2=0x7fb2ec0b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2f28fb640 1 --2- 172.21.15.115:0/3881704210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec143240 0x7fb2ec0b20a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec159fe0 tx=0x7fb2dc02b8d0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2f28fb640 1 -- 172.21.15.115:0/3881704210 shutdown_connections 2023-12-01T01:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2f28fb640 1 --2- 172.21.15.115:0/3881704210 >> v2:172.21.15.142:3300/0 conn(0x7fb2ec06caa0 0x7fb2ec0ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2f28fb640 1 --2- 172.21.15.115:0/3881704210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec143240 0x7fb2ec0b20a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2f28fb640 1 -- 172.21.15.115:0/3881704210 >> 172.21.15.115:0/3881704210 conn(0x7fb2ec065960 msgr2=0x7fb2ec065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2f28fb640 1 -- 172.21.15.115:0/3881704210 shutdown_connections 2023-12-01T01:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.854+0000 7fb2f28fb640 1 -- 172.21.15.115:0/3881704210 wait complete. 2023-12-01T01:51:34.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.856+0000 7fb2f28fb640 1 Processor -- start 2023-12-01T01:51:34.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.856+0000 7fb2f28fb640 1 -- start start 2023-12-01T01:51:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.857+0000 7fb2f28fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec06caa0 0x7fb2ec153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.857+0000 7fb2f28fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2ec0b25e0 0x7fb2ec158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.857+0000 7fb2f28fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2ec143240 0x7fb2ec15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.857+0000 7fb2f28fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec153460 con 0x7fb2ec06caa0 2023-12-01T01:51:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.857+0000 7fb2f28fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec15c400 con 0x7fb2ec0b25e0 2023-12-01T01:51:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.857+0000 7fb2f28fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec15b7f0 con 0x7fb2ec143240 2023-12-01T01:51:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.858+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec06caa0 0x7fb2ec153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.858+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec06caa0 0x7fb2ec153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39334/0 (socket says 172.21.15.115:39334) 2023-12-01T01:51:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.858+0000 7fb2ebfff640 1 -- 172.21.15.115:0/2387863981 learned_addr learned my addr 172.21.15.115:0/2387863981 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:34.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.858+0000 7fb2ebfff640 1 -- 172.21.15.115:0/2387863981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2ec143240 msgr2=0x7fb2ec15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:34.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.858+0000 7fb2ebfff640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2ec143240 0x7fb2ec15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.858+0000 7fb2ebfff640 1 -- 172.21.15.115:0/2387863981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2ec0b25e0 msgr2=0x7fb2ec158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:34.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.858+0000 7fb2ebfff640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2ec0b25e0 0x7fb2ec158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:34.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.858+0000 7fb2ebfff640 1 -- 172.21.15.115:0/2387863981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc009400 con 0x7fb2ec06caa0 2023-12-01T01:51:34.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.858+0000 7fb2ebfff640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec06caa0 0x7fb2ec153ae0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc000c00 tx=0x7fb2dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:34.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.859+0000 7fb2e8ff9640 1 -- 172.21.15.115:0/2387863981 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc03aa60 con 0x7fb2ec06caa0 2023-12-01T01:51:34.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.859+0000 7fb2e8ff9640 1 -- 172.21.15.115:0/2387863981 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc03abe0 con 0x7fb2ec06caa0 2023-12-01T01:51:34.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.859+0000 7fb2e8ff9640 1 -- 172.21.15.115:0/2387863981 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0443f0 con 0x7fb2ec06caa0 2023-12-01T01:51:34.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.859+0000 7fb2f28fb640 1 -- 172.21.15.115:0/2387863981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec15bad0 con 0x7fb2ec06caa0 2023-12-01T01:51:34.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.859+0000 7fb2f28fb640 1 -- 172.21.15.115:0/2387863981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec15c090 con 0x7fb2ec06caa0 2023-12-01T01:51:34.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.860+0000 7fb2f28fb640 1 -- 172.21.15.115:0/2387863981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec04e890 con 0x7fb2ec06caa0 2023-12-01T01:51:34.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.861+0000 7fb2e8ff9640 1 -- 172.21.15.115:0/2387863981 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb2dc04d450 con 0x7fb2ec06caa0 2023-12-01T01:51:34.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.862+0000 7fb2e8ff9640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb2d806a580 0x7fb2d806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:34.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.862+0000 7fb2e8ff9640 1 -- 172.21.15.115:0/2387863981 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2dc0aad90 con 0x7fb2ec06caa0 2023-12-01T01:51:34.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.862+0000 7fb2eb7fe640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb2d806a580 0x7fb2d806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:34.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.863+0000 7fb2eb7fe640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb2d806a580 0x7fb2d806ca40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0007fd0 tx=0x7fb2e0007920 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:34.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:34.866+0000 7fb2e8ff9640 1 -- 172.21.15.115:0/2387863981 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb2dc076270 con 0x7fb2ec06caa0 2023-12-01T01:51:35.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.122+0000 7fb2f28fb640 1 -- 172.21.15.115:0/2387863981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb2ec0b1e80 con 0x7fb2ec06caa0 2023-12-01T01:51:35.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.128+0000 7fb2e8ff9640 1 -- 172.21.15.115:0/2387863981 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb2dc035020 con 0x7fb2ec06caa0 2023-12-01T01:51:35.129 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T01:51:35.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.130+0000 7fb2f28fb640 1 -- 172.21.15.115:0/2387863981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb2d806a580 msgr2=0x7fb2d806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:35.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.130+0000 7fb2f28fb640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb2d806a580 0x7fb2d806ca40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0007fd0 tx=0x7fb2e0007920 comp rx=0 tx=0).stop 2023-12-01T01:51:35.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.130+0000 7fb2f28fb640 1 -- 172.21.15.115:0/2387863981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec06caa0 msgr2=0x7fb2ec153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:35.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.130+0000 7fb2f28fb640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec06caa0 0x7fb2ec153ae0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc000c00 tx=0x7fb2dc0046c0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.131+0000 7fb2f28fb640 1 -- 172.21.15.115:0/2387863981 shutdown_connections 2023-12-01T01:51:35.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.131+0000 7fb2f28fb640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb2d806a580 0x7fb2d806ca40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.131+0000 7fb2f28fb640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2ec143240 0x7fb2ec15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.131+0000 7fb2f28fb640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2ec0b25e0 0x7fb2ec158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.131+0000 7fb2f28fb640 1 --2- 172.21.15.115:0/2387863981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2ec06caa0 0x7fb2ec153ae0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.131+0000 7fb2f28fb640 1 -- 172.21.15.115:0/2387863981 >> 172.21.15.115:0/2387863981 conn(0x7fb2ec065960 msgr2=0x7fb2ec06fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:35.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.131+0000 7fb2f28fb640 1 -- 172.21.15.115:0/2387863981 shutdown_connections 2023-12-01T01:51:35.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.131+0000 7fb2f28fb640 1 -- 172.21.15.115:0/2387863981 wait complete. 2023-12-01T01:51:35.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:35.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.358+0000 7f2a4b3ab640 1 Processor -- start 2023-12-01T01:51:35.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.359+0000 7f2a4b3ab640 1 -- start start 2023-12-01T01:51:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.360+0000 7f2a4b3ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a440b5d80 0x7f2a440d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.360+0000 7f2a4b3ab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a440ce2e0 con 0x7f2a440d1230 2023-12-01T01:51:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.360+0000 7f2a4b3ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a440ce000 con 0x7f2a4415cd20 2023-12-01T01:51:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.360+0000 7f2a4b3ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440b2160 con 0x7f2a440b5d80 2023-12-01T01:51:35.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.360+0000 7f2a49948640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2a440d1230 0x7f2a440d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59366/0 (socket says 172.21.15.115:59366) 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.360+0000 7f2a49948640 1 -- 172.21.15.115:0/1184781375 learned_addr learned my addr 172.21.15.115:0/1184781375 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775430307 0 0) 0x7f2a440ce2e0 con 0x7f2a440d1230 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a2c003520 con 0x7f2a440d1230 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4228080177 0 0) 0x7f2a2c003520 con 0x7f2a440d1230 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a440b5d80 msgr2=0x7f2a440d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a440b5d80 0x7f2a440d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 >> v1:172.21.15.142:6789/0 conn(0x7f2a4415cd20 legacy=0x7f2a440b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440cd3d0 con 0x7f2a440d1230 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2a3c0019f0 con 0x7f2a440d1230 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 >> v1:172.21.15.115:6789/0 conn(0x7f2a440d1230 legacy=0x7f2a440d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a440b5d80 0x7f2a2c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c004120 0x7f2a2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c004e20 0x7f2a2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0055e0 con 0x7f2a2c004120 2023-12-01T01:51:35.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c00bec0 con 0x7f2a2c004e20 2023-12-01T01:51:35.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c00c000 con 0x7f2a440b5d80 2023-12-01T01:51:35.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.361+0000 7f2a48946640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c004120 0x7f2a2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:35.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.362+0000 7f2a49948640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c004e20 0x7f2a2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:35.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.362+0000 7f2a49147640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a440b5d80 0x7f2a2c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:35.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.362+0000 7f2a48946640 1 -- 172.21.15.115:0/1184781375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a440b5d80 msgr2=0x7f2a2c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:35.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.362+0000 7f2a48946640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a440b5d80 0x7f2a2c003a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.362+0000 7f2a48946640 1 -- 172.21.15.115:0/1184781375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c004e20 msgr2=0x7f2a2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:35.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.362+0000 7f2a48946640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c004e20 0x7f2a2c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.362+0000 7f2a48946640 1 -- 172.21.15.115:0/1184781375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c003520 con 0x7f2a2c004120 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.363+0000 7f2a48946640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c004120 0x7f2a2c0047c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2a440b6160 tx=0x7f2a2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7705fa74771ca65 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.363+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38032600 con 0x7f2a2c004120 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.363+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3802e050 con 0x7f2a2c004120 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.363+0000 7f2a43fff640 1 -- 172.21.15.115:0/1184781375 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a380329c0 con 0x7f2a2c004120 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.364+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/1184781375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c004120 msgr2=0x7f2a2c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.364+0000 7f2a4b3ab640 1 --2- 172.21.15.115:0/1184781375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a2c004120 0x7f2a2c0047c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2a440b6160 tx=0x7f2a2c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.364+0000 7f2a49948640 1 -- 172.21.15.115:0/1184781375 reap_dead start 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.364+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/1184781375 shutdown_connections 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.364+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/1184781375 >> 172.21.15.115:0/1184781375 conn(0x7f2a4406a0e0 msgr2=0x7f2a4406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.364+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/1184781375 shutdown_connections 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.364+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/1184781375 wait complete. 2023-12-01T01:51:35.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.366+0000 7f2a4b3ab640 1 Processor -- start 2023-12-01T01:51:35.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.366+0000 7f2a4b3ab640 1 -- start start 2023-12-01T01:51:35.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a4b3ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c004120 0x7f2a440c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:35.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a4b3ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4415e990 0x7f2a4415b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:35.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a4b3ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a4415ee10 0x7f2a4415b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:35.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a4b3ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4415fff0 con 0x7f2a4415e990 2023-12-01T01:51:35.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a4b3ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440c9a40 con 0x7f2a2c004120 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a4b3ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4415f3e0 con 0x7f2a4415ee10 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a48946640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4415e990 0x7f2a4415b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a48946640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4415e990 0x7f2a4415b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39348/0 (socket says 172.21.15.115:39348) 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a48946640 1 -- 172.21.15.115:0/163936421 learned_addr learned my addr 172.21.15.115:0/163936421 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a49147640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c004120 0x7f2a440c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a48946640 1 -- 172.21.15.115:0/163936421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a4415ee10 msgr2=0x7f2a4415b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a48946640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a4415ee10 0x7f2a4415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a48946640 1 -- 172.21.15.115:0/163936421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c004120 msgr2=0x7f2a440c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a48946640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c004120 0x7f2a440c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a48946640 1 -- 172.21.15.115:0/163936421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4415f6c0 con 0x7f2a4415e990 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.367+0000 7f2a49147640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c004120 0x7f2a440c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.368+0000 7f2a48946640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4415e990 0x7f2a4415b1d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2a3802bb50 tx=0x7f2a380047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.368+0000 7f2a41ffb640 1 -- 172.21.15.115:0/163936421 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3803d890 con 0x7f2a4415e990 2023-12-01T01:51:35.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.368+0000 7f2a41ffb640 1 -- 172.21.15.115:0/163936421 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3802be90 con 0x7f2a4415e990 2023-12-01T01:51:35.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.368+0000 7f2a41ffb640 1 -- 172.21.15.115:0/163936421 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3803dc10 con 0x7f2a4415e990 2023-12-01T01:51:35.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.368+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/163936421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4415f9a0 con 0x7f2a4415e990 2023-12-01T01:51:35.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.368+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/163936421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a440b2790 con 0x7f2a4415e990 2023-12-01T01:51:35.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.369+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/163936421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a44160160 con 0x7f2a4415e990 2023-12-01T01:51:35.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.370+0000 7f2a41ffb640 1 -- 172.21.15.115:0/163936421 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f2a38039040 con 0x7f2a4415e990 2023-12-01T01:51:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.371+0000 7f2a41ffb640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a3006a580 0x7f2a3006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.371+0000 7f2a41ffb640 1 -- 172.21.15.115:0/163936421 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a380aa620 con 0x7f2a4415e990 2023-12-01T01:51:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.372+0000 7f2a49147640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a3006a580 0x7f2a3006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:35.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.372+0000 7f2a49147640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a3006a580 0x7f2a3006ca40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a34007fd0 tx=0x7f2a34009040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:35.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.376+0000 7f2a41ffb640 1 -- 172.21.15.115:0/163936421 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2a38075b20 con 0x7f2a4415e990 2023-12-01T01:51:35.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.603+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/163936421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a4404e890 con 0x7f2a4415e990 2023-12-01T01:51:35.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.603+0000 7f2a41ffb640 1 -- 172.21.15.115:0/163936421 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f2a38079140 con 0x7f2a4415e990 2023-12-01T01:51:35.604 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:35.604 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1284508688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1284508688},{"type":"v1","addr":"172.21.15.142:6833","nonce":1284508688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1196894153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1196894153},{"type":"v1","addr":"172.21.15.142:6835","nonce":1196894153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6432,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/120786293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":120786293},{"type":"v1","addr":"172.21.15.115:6836","nonce":120786293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:51:24.174363+0000","modified":"2023-12-01T01:51:27.708969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6435},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6435":{"gid":6435,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3394497964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3394497964},{"type":"v1","addr":"172.21.15.115:6837","nonce":3394497964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-01T01:51:35.604 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 32 2023-12-01T01:51:35.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/163936421 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a3006a580 msgr2=0x7f2a3006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:35.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a3006a580 0x7f2a3006ca40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a34007fd0 tx=0x7f2a34009040 comp rx=0 tx=0).stop 2023-12-01T01:51:35.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/163936421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4415e990 msgr2=0x7f2a4415b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:35.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4415e990 0x7f2a4415b1d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2a3802bb50 tx=0x7f2a380047d0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/163936421 shutdown_connections 2023-12-01T01:51:35.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a3006a580 0x7f2a3006ca40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a4415ee10 0x7f2a4415b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4415e990 0x7f2a4415b1d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 --2- 172.21.15.115:0/163936421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a2c004120 0x7f2a440c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:35.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/163936421 >> 172.21.15.115:0/163936421 conn(0x7f2a4406a0e0 msgr2=0x7f2a4406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:35.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/163936421 shutdown_connections 2023-12-01T01:51:35.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:35.606+0000 7f2a4b3ab640 1 -- 172.21.15.115:0/163936421 wait complete. 2023-12-01T01:51:36.027 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T01:51:24.174363+0000', 'modified': '2023-12-01T01:51:27.708969+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6435}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6435': {'gid': 6435, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6837/3394497964', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6835', 'nonce': 3394497964}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 3394497964}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T01:51:36.027 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T01:51:36.027 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-01T01:51:36.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.250+0000 7fa9ba0af640 1 Processor -- start 2023-12-01T01:51:36.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.250+0000 7fa9ba0af640 1 -- start start 2023-12-01T01:51:36.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.251+0000 7fa9ba0af640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa9b4159dd0 0x7fa9b40c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9ba0af640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9b4070d30 0x7fa9b40c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9ba0af640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b406f8b0 con 0x7fa9b406b6f0 2023-12-01T01:51:36.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9ba0af640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b40adcb0 con 0x7fa9b4159dd0 2023-12-01T01:51:36.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9ba0af640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b40ad9d0 con 0x7fa9b4070d30 2023-12-01T01:51:36.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9b37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa9b4159dd0 0x7fa9b40c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9b37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa9b4159dd0 0x7fa9b40c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39352/0 (socket says 172.21.15.115:39352) 2023-12-01T01:51:36.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9b37fe640 1 -- 172.21.15.115:0/1309518197 learned_addr learned my addr 172.21.15.115:0/1309518197 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:36.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9b37fe640 1 -- 172.21.15.115:0/1309518197 >> v2:172.21.15.142:3300/0 conn(0x7fa9b4070d30 msgr2=0x7fa9b40c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9b3fff640 1 --2- 172.21.15.115:0/1309518197 >> v2:172.21.15.142:3300/0 conn(0x7fa9b4070d30 0x7fa9b40c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/1309518197 >> v2:172.21.15.142:3300/0 conn(0x7fa9b4070d30 0x7fa9b40c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9b37fe640 1 -- 172.21.15.115:0/1309518197 >> v1:172.21.15.142:6789/0 conn(0x7fa9b406b6f0 legacy=0x7fa9b40cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9b37fe640 1 -- 172.21.15.115:0/1309518197 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b40c9000 con 0x7fa9b4159dd0 2023-12-01T01:51:36.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.252+0000 7fa9b3fff640 1 --2- 172.21.15.115:0/1309518197 >> v2:172.21.15.142:3300/0 conn(0x7fa9b4070d30 0x7fa9b40c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:36.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.254+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/1309518197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 0x7fa9b40c97e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa9b406f280 tx=0x7fa9a402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b149a9f571468482 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:36.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.254+0000 7fa9b27fc640 1 -- 172.21.15.115:0/1309518197 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4031750 con 0x7fa9b4159dd0 2023-12-01T01:51:36.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.254+0000 7fa9b27fc640 1 -- 172.21.15.115:0/1309518197 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a40318b0 con 0x7fa9b4159dd0 2023-12-01T01:51:36.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.254+0000 7fa9b27fc640 1 -- 172.21.15.115:0/1309518197 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4031c40 con 0x7fa9b4159dd0 2023-12-01T01:51:36.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.255+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1309518197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 msgr2=0x7fa9b40c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.255+0000 7fa9ba0af640 1 --2- 172.21.15.115:0/1309518197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 0x7fa9b40c97e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa9b406f280 tx=0x7fa9a402b7f0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.255+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1309518197 shutdown_connections 2023-12-01T01:51:36.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.255+0000 7fa9ba0af640 1 --2- 172.21.15.115:0/1309518197 >> v2:172.21.15.142:3300/0 conn(0x7fa9b4070d30 0x7fa9b40c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.255+0000 7fa9ba0af640 1 --2- 172.21.15.115:0/1309518197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 0x7fa9b40c97e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.255+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1309518197 >> 172.21.15.115:0/1309518197 conn(0x7fa9b40ce880 msgr2=0x7fa9b40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:36.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.255+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1309518197 shutdown_connections 2023-12-01T01:51:36.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.255+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1309518197 wait complete. 2023-12-01T01:51:36.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.256+0000 7fa9ba0af640 1 Processor -- start 2023-12-01T01:51:36.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.257+0000 7fa9ba0af640 1 -- start start 2023-12-01T01:51:36.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.257+0000 7fa9ba0af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b406b6f0 0x7fa9b40cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.257+0000 7fa9ba0af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b4070d30 0x7fa9b40aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.257+0000 7fa9ba0af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 0x7fa9b414fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.257+0000 7fa9ba0af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41e3990 con 0x7fa9b4159dd0 2023-12-01T01:51:36.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.257+0000 7fa9ba0af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40c42f0 con 0x7fa9b4070d30 2023-12-01T01:51:36.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.257+0000 7fa9ba0af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40cd3c0 con 0x7fa9b406b6f0 2023-12-01T01:51:36.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.258+0000 7fa9b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 0x7fa9b414fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.258+0000 7fa9b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 0x7fa9b414fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39366/0 (socket says 172.21.15.115:39366) 2023-12-01T01:51:36.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.258+0000 7fa9b3fff640 1 -- 172.21.15.115:0/1783193467 learned_addr learned my addr 172.21.15.115:0/1783193467 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:36.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.258+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b406b6f0 0x7fa9b40cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.258+0000 7fa9b3fff640 1 -- 172.21.15.115:0/1783193467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b406b6f0 msgr2=0x7fa9b40cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.258+0000 7fa9b3fff640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b406b6f0 0x7fa9b40cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.258+0000 7fa9b3fff640 1 -- 172.21.15.115:0/1783193467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b4070d30 msgr2=0x7fa9b40aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.258+0000 7fa9b3fff640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b4070d30 0x7fa9b40aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.258+0000 7fa9b3fff640 1 -- 172.21.15.115:0/1783193467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a40094d0 con 0x7fa9b4159dd0 2023-12-01T01:51:36.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.259+0000 7fa9b3fff640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 0x7fa9b414fb30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00cf20 tx=0x7fa9ac00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:36.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.259+0000 7fa99bfff640 1 -- 172.21.15.115:0/1783193467 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00d840 con 0x7fa9b4159dd0 2023-12-01T01:51:36.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.259+0000 7fa99bfff640 1 -- 172.21.15.115:0/1783193467 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac00d9a0 con 0x7fa9b4159dd0 2023-12-01T01:51:36.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.259+0000 7fa99bfff640 1 -- 172.21.15.115:0/1783193467 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00dd50 con 0x7fa9b4159dd0 2023-12-01T01:51:36.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.259+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1783193467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b40c3400 con 0x7fa9b4159dd0 2023-12-01T01:51:36.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.259+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1783193467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b40c39c0 con 0x7fa9b4159dd0 2023-12-01T01:51:36.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.262+0000 7fa99bfff640 1 -- 172.21.15.115:0/1783193467 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa9ac01e3b0 con 0x7fa9b4159dd0 2023-12-01T01:51:36.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.263+0000 7fa99bfff640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa9a006a580 0x7fa9a006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.263+0000 7fa99bfff640 1 -- 172.21.15.115:0/1783193467 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9ac089490 con 0x7fa9b4159dd0 2023-12-01T01:51:36.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.263+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa9a006a580 0x7fa9a006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.264+0000 7fa9bcc1e740 1 Processor -- start 2023-12-01T01:51:36.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.264+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa9a006a580 0x7fa9a006ca40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4035000 tx=0x7fa9a4008030 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:36.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.264+0000 7fa9bcc1e740 1 -- start start 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c2d8a0 0x55e798c5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c314a0 0x55e798c5b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e798c092f0 con 0x55e798c5d920 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e798b9df80 con 0x55e798c314a0 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e798b9b710 con 0x55e798c2d8a0 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9b37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9b37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39374/0 (socket says 172.21.15.115:39374) 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9b37fe640 1 -- 172.21.15.115:0/1129361640 learned_addr learned my addr 172.21.15.115:0/1129361640 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9b37fe640 1 -- 172.21.15.115:0/1129361640 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c2d8a0 msgr2=0x55e798c5ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/1129361640 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c2d8a0 0x55e798c5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9b37fe640 1 -- 172.21.15.115:0/1129361640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c314a0 msgr2=0x55e798c5b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/1129361640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c314a0 0x55e798c5b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.265+0000 7fa9b37fe640 1 -- 172.21.15.115:0/1129361640 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e798b5e690 con 0x55e798c5d920 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.266+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/1129361640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5dd00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40090b0 tx=0x7fa9a4031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f5dd9027757620 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.266+0000 7fa9997fa640 1 -- 172.21.15.115:0/1129361640 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4043070 con 0x55e798c5d920 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.266+0000 7fa9997fa640 1 -- 172.21.15.115:0/1129361640 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a403dac0 con 0x55e798c5d920 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.266+0000 7fa9997fa640 1 -- 172.21.15.115:0/1129361640 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a403de30 con 0x55e798c5d920 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.267+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/1129361640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 msgr2=0x55e798c5dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.267+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/1129361640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5dd00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40090b0 tx=0x7fa9a4031000 comp rx=0 tx=0).stop 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.267+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/1129361640 shutdown_connections 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.267+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/1129361640 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c2d8a0 0x55e798c5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.267+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/1129361640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c314a0 0x55e798c5b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.267+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/1129361640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5dd00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.267+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/1129361640 >> 172.21.15.115:0/1129361640 conn(0x55e798c2fbb0 msgr2=0x55e798c78730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:36.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.267+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/1129361640 shutdown_connections 2023-12-01T01:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.267+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/1129361640 wait complete. 2023-12-01T01:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.267+0000 7fa9bcc1e740 1 -- start start 2023-12-01T01:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.268+0000 7fa9bcc1e740 1 Processor -- start 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c314a0 0x55e798c526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c5d920 0x55e798c5ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c52d30 0x55e798c5bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e798b97000 con 0x55e798c314a0 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e798b387d0 con 0x55e798c52d30 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e798b38470 con 0x55e798c5d920 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9bcc1e740 10 client.0 _ll_register_callbacks cb 0x55e798b871b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9b37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c314a0 0x55e798c526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9b37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c314a0 0x55e798c526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39380/0 (socket says 172.21.15.115:39380) 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9b37fe640 1 -- 172.21.15.115:0/824150707 learned_addr learned my addr 172.21.15.115:0/824150707 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9b37fe640 1 -- 172.21.15.115:0/824150707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c5d920 msgr2=0x55e798c5ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c5d920 0x55e798c5ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9b37fe640 1 -- 172.21.15.115:0/824150707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c52d30 msgr2=0x55e798c5bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c52d30 0x55e798c5bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.269+0000 7fa9b37fe640 1 -- 172.21.15.115:0/824150707 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b40c9000 con 0x55e798c314a0 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.270+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c314a0 0x55e798c526a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009960 tx=0x7fa9a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53bf8647373f13b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.270+0000 7fa9927fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.270+0000 7fa9927fc640 1 -- 172.21.15.115:0/824150707 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4043040 con 0x55e798c314a0 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.270+0000 7fa9927fc640 1 -- 172.21.15.115:0/824150707 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9a4054680 con 0x55e798c314a0 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.270+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/824150707 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e798bc1550 con 0x55e798c314a0 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.270+0000 7fa9927fc640 1 -- 172.21.15.115:0/824150707 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9a4054840 con 0x55e798c314a0 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.270+0000 7fa9bcc1e740 10 client.6612 fetch_fsmap learned FSMap version 32 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.270+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/824150707 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e798b95680 con 0x55e798c314a0 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.270+0000 7fa9927fc640 1 -- 172.21.15.115:0/824150707 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e798bc1550 con 0x55e798c314a0 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.270+0000 7fa9927fc640 1 -- 172.21.15.115:0/824150707 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa9a40549a0 con 0x55e798c314a0 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.271+0000 7fa9bcc1e740 10 client.6612 fetch_fsmap finished waiting for FSMap version 32 2023-12-01T01:51:36.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.271+0000 7fa9bcc1e740 10 client.6612 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:51:36.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.271+0000 7fa9bcc1e740 20 client.6612 populate_metadata read hostname 'smithi115' 2023-12-01T01:51:36.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.271+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.115:6835/3394497964,v1:172.21.15.115:6837/3394497964] conn(0x55e798c78040 0x55e798c95870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.271+0000 7fa9bcc1e740 4 client.6612 mds_command: new command op to 6435 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:51:36.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.271+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/824150707 --> [v2:172.21.15.115:6835/3394497964,v1:172.21.15.115:6837/3394497964] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e798b89b50 con 0x55e798c78040 2023-12-01T01:51:36.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.271+0000 7fa9b2ffd640 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.115:6835/3394497964,v1:172.21.15.115:6837/3394497964] conn(0x55e798c78040 0x55e798c95870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.271+0000 7fa9b2ffd640 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.115:6835/3394497964,v1:172.21.15.115:6837/3394497964] conn(0x55e798c78040 0x55e798c95870 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d7cdcd76b4bb94 server_cookie=aa193f4916b161c5 in_seq=0 out_seq=0 2023-12-01T01:51:36.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.271+0000 7fa9927fc640 10 client.6612 ms_handle_connect on v2:172.21.15.115:6835/3394497964 2023-12-01T01:51:36.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.272+0000 7fa9927fc640 1 -- 172.21.15.115:0/824150707 <== mds.0 v2:172.21.15.115:6835/3394497964 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e798b89b50 con 0x55e798c78040 2023-12-01T01:51:36.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.272+0000 7fa9927fc640 10 client.6612 handle_command_reply: tid=0 2023-12-01T01:51:36.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.272+0000 7fa9bcc1e740 1 client.6612 shutdown 2023-12-01T01:51:36.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/824150707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c314a0 msgr2=0x55e798c526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c314a0 0x55e798c526a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009960 tx=0x7fa9a4004730 comp rx=0 tx=0).stop 2023-12-01T01:51:36.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/824150707 shutdown_connections 2023-12-01T01:51:36.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.115:6835/3394497964,v1:172.21.15.115:6837/3394497964] conn(0x55e798c78040 0x55e798c95870 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c5d920 0x55e798c5ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c52d30 0x55e798c5bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/824150707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c314a0 0x55e798c526a0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9927fc640 0 client.6612 ms_handle_reset on v2:172.21.15.115:6835/3394497964 2023-12-01T01:51:36.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/824150707 >> 172.21.15.115:0/824150707 conn(0x55e798c2d8a0 msgr2=0x55e798c31d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:36.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/824150707 shutdown_connections 2023-12-01T01:51:36.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/824150707 wait complete. 2023-12-01T01:51:36.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.273+0000 7fa9bcc1e740 20 client.6612 trim_cache size 0 max 16384 2023-12-01T01:51:36.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.289+0000 7fa9bcc1e740 1 Processor -- start 2023-12-01T01:51:36.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.289+0000 7fa9bcc1e740 1 -- start start 2023-12-01T01:51:36.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.290+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c314a0 0x55e798c5b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.290+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c52d30 0x55e798c5be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.290+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c5d920 0x55e798c5c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.290+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7988fef20 con 0x55e798c52d30 2023-12-01T01:51:36.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.290+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e798c52650 con 0x55e798c5d920 2023-12-01T01:51:36.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.290+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e798c7c080 con 0x55e798c314a0 2023-12-01T01:51:36.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.290+0000 7fa9b2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c52d30 0x55e798c5be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.290+0000 7fa9b2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c52d30 0x55e798c5be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39396/0 (socket says 172.21.15.115:39396) 2023-12-01T01:51:36.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.290+0000 7fa9b2ffd640 1 -- 172.21.15.115:0/4174592691 learned_addr learned my addr 172.21.15.115:0/4174592691 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:36.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa9b2ffd640 1 -- 172.21.15.115:0/4174592691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c314a0 msgr2=0x55e798c5b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa9b3fff640 1 --2- 172.21.15.115:0/4174592691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c5d920 0x55e798c5c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/4174592691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c314a0 0x55e798c5b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa9b2ffd640 1 --2- 172.21.15.115:0/4174592691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c314a0 0x55e798c5b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa9b2ffd640 1 -- 172.21.15.115:0/4174592691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c5d920 msgr2=0x55e798c5c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa9b2ffd640 1 --2- 172.21.15.115:0/4174592691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c5d920 0x55e798c5c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa9b2ffd640 1 -- 172.21.15.115:0/4174592691 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e798b379e0 con 0x55e798c52d30 2023-12-01T01:51:36.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa9b2ffd640 1 --2- 172.21.15.115:0/4174592691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c52d30 0x55e798c5be90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa9b406cdd0 tx=0x7fa9a8001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3b36854c048116 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:36.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa999ffb640 1 -- 172.21.15.115:0/4174592691 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8001580 con 0x55e798c52d30 2023-12-01T01:51:36.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa999ffb640 1 -- 172.21.15.115:0/4174592691 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a8001700 con 0x55e798c52d30 2023-12-01T01:51:36.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.291+0000 7fa999ffb640 1 -- 172.21.15.115:0/4174592691 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8001ab0 con 0x55e798c52d30 2023-12-01T01:51:36.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.292+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/4174592691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c52d30 msgr2=0x55e798c5be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.292+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/4174592691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c52d30 0x55e798c5be90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa9b406cdd0 tx=0x7fa9a8001090 comp rx=0 tx=0).stop 2023-12-01T01:51:36.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.292+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/4174592691 shutdown_connections 2023-12-01T01:51:36.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.292+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/4174592691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c5d920 0x55e798c5c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.292+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/4174592691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c52d30 0x55e798c5be90 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.292+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/4174592691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c314a0 0x55e798c5b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.292+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/4174592691 >> 172.21.15.115:0/4174592691 conn(0x55e798c2d8a0 msgr2=0x55e798c53420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:36.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.292+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/4174592691 shutdown_connections 2023-12-01T01:51:36.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.292+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/4174592691 wait complete. 2023-12-01T01:51:36.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.293+0000 7fa9bcc1e740 1 -- start start 2023-12-01T01:51:36.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.293+0000 7fa9bcc1e740 1 Processor -- start 2023-12-01T01:51:36.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.294+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c314a0 0x55e798cdd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.294+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c52d30 0x55e798c2dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.294+0000 7fa9bcc1e740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.294+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e798c8a220 con 0x55e798c5d920 2023-12-01T01:51:36.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.294+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e798c59460 con 0x55e798c314a0 2023-12-01T01:51:36.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.294+0000 7fa9bcc1e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e798c2ca40 con 0x55e798c52d30 2023-12-01T01:51:36.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.294+0000 7fa9bcc1e740 10 client.0 _ll_register_callbacks cb 0x7fa9a4036070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.294+0000 7fa9b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.294+0000 7fa9b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39410/0 (socket says 172.21.15.115:39410) 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.294+0000 7fa9b3fff640 1 -- 172.21.15.115:0/2205140441 learned_addr learned my addr 172.21.15.115:0/2205140441 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c314a0 0x55e798cdd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9b3fff640 1 -- 172.21.15.115:0/2205140441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c52d30 msgr2=0x55e798c2dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9b3fff640 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c52d30 0x55e798c2dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9b3fff640 1 -- 172.21.15.115:0/2205140441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c314a0 msgr2=0x55e798cdd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9b3fff640 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c314a0 0x55e798cdd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9b3fff640 1 -- 172.21.15.115:0/2205140441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a80132d0 con 0x55e798c5d920 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9b3fff640 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5b950 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac054bc0 tx=0x7fa9ac058e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5483728db4a9c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9927fc640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9927fc640 1 -- 172.21.15.115:0/2205140441 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac05c320 con 0x55e798c5d920 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9927fc640 1 -- 172.21.15.115:0/2205140441 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9ac06e020 con 0x55e798c5d920 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/2205140441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9a403dac0 con 0x55e798c5d920 2023-12-01T01:51:36.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9927fc640 1 -- 172.21.15.115:0/2205140441 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9ac0606a0 con 0x55e798c5d920 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9bcc1e740 10 client.6618 fetch_fsmap learned FSMap version 32 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/2205140441 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e798b5cef0 con 0x55e798c5d920 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.295+0000 7fa9927fc640 1 -- 172.21.15.115:0/2205140441 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a403dac0 con 0x55e798c5d920 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.296+0000 7fa9927fc640 1 -- 172.21.15.115:0/2205140441 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa9ac008040 con 0x55e798c5d920 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.296+0000 7fa9bcc1e740 10 client.6618 fetch_fsmap finished waiting for FSMap version 32 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.296+0000 7fa9bcc1e740 10 client.6618 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.296+0000 7fa9bcc1e740 20 client.6618 populate_metadata read hostname 'smithi115' 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.296+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.115:6835/3394497964,v1:172.21.15.115:6837/3394497964] conn(0x55e798c8ac00 0x55e798c8afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.296+0000 7fa9bcc1e740 4 client.6618 mds_command: new command op to 6435 tid=0[{"prefix": "status"}] 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.296+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/2205140441 --> [v2:172.21.15.115:6835/3394497964,v1:172.21.15.115:6837/3394497964] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e798b90080 con 0x55e798c8ac00 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.296+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.115:6835/3394497964,v1:172.21.15.115:6837/3394497964] conn(0x55e798c8ac00 0x55e798c8afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.296+0000 7fa9b37fe640 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.115:6835/3394497964,v1:172.21.15.115:6837/3394497964] conn(0x55e798c8ac00 0x55e798c8afe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da495bbaa328a574 server_cookie=c680698417206842 in_seq=0 out_seq=0 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.296+0000 7fa9927fc640 10 client.6618 ms_handle_connect on v2:172.21.15.115:6835/3394497964 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9927fc640 1 -- 172.21.15.115:0/2205140441 <== mds.0 v2:172.21.15.115:6835/3394497964 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e798b90080 con 0x55e798c8ac00 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9927fc640 10 client.6618 handle_command_reply: tid=0 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 client.6618 shutdown 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/2205140441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 msgr2=0x55e798c5b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5b950 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac054bc0 tx=0x7fa9ac058e80 comp rx=0 tx=0).stop 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/2205140441 shutdown_connections 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.115:6835/3394497964,v1:172.21.15.115:6837/3394497964] conn(0x55e798c8ac00 0x55e798c8afe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e798c52d30 0x55e798c2dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e798c5d920 0x55e798c5b950 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 --2- 172.21.15.115:0/2205140441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e798c314a0 0x55e798cdd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9927fc640 0 client.6618 ms_handle_reset on v2:172.21.15.115:6835/3394497964 2023-12-01T01:51:36.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/2205140441 >> 172.21.15.115:0/2205140441 conn(0x55e798c2d8a0 msgr2=0x55e798c53420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:36.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/2205140441 shutdown_connections 2023-12-01T01:51:36.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 1 -- 172.21.15.115:0/2205140441 wait complete. 2023-12-01T01:51:36.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.297+0000 7fa9bcc1e740 20 client.6618 trim_cache size 0 max 16384 2023-12-01T01:51:36.307 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:51:36.307 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T01:51:36.308 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T01:51:36.308 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 6435, 2023-12-01T01:51:36.308 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T01:51:36.308 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T01:51:36.308 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T01:51:36.309 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 12.117028976, 2023-12-01T01:51:36.309 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 32, 2023-12-01T01:51:36.309 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 83, 2023-12-01T01:51:36.309 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 83, 2023-12-01T01:51:36.309 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 17.053332039000001 2023-12-01T01:51:36.309 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:51:36.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1783193467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa9a006a580 msgr2=0x7fa9a006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa9a006a580 0x7fa9a006ca40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4035000 tx=0x7fa9a4008030 comp rx=0 tx=0).stop 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1783193467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 msgr2=0x7fa9b414fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 0x7fa9b414fb30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00cf20 tx=0x7fa9ac00a7d0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1783193467 shutdown_connections 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa9a006a580 0x7fa9a006ca40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9b4159dd0 0x7fa9b414fb30 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b4070d30 0x7fa9b40aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 --2- 172.21.15.115:0/1783193467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b406b6f0 0x7fa9b40cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1783193467 >> 172.21.15.115:0/1783193467 conn(0x7fa9b40ce880 msgr2=0x7fa9b40c1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1783193467 shutdown_connections 2023-12-01T01:51:36.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.298+0000 7fa9ba0af640 1 -- 172.21.15.115:0/1783193467 wait complete. 2023-12-01T01:51:36.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:51:36.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.511+0000 7f6e83fff640 1 Processor -- start 2023-12-01T01:51:36.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.511+0000 7f6e83fff640 1 -- start start 2023-12-01T01:51:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.513+0000 7f6e83fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6e8415afe0 0x7f6e8406b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.513+0000 7f6e83fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e8406b7b0 0x7f6e840cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.513+0000 7f6e83fff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e840ce4a0 con 0x7f6e841584b0 2023-12-01T01:51:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.513+0000 7f6e83fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8406bc40 con 0x7f6e8406b7b0 2023-12-01T01:51:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.513+0000 7f6e83fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e840cc180 con 0x7f6e8415afe0 2023-12-01T01:51:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.514+0000 7f6e837fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e8406b7b0 0x7f6e840cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.514+0000 7f6e837fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e8406b7b0 0x7f6e840cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39422/0 (socket says 172.21.15.115:39422) 2023-12-01T01:51:36.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.514+0000 7f6e837fe640 1 -- 172.21.15.115:0/1092602829 learned_addr learned my addr 172.21.15.115:0/1092602829 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.514+0000 7f6e7bfff640 1 --2- 172.21.15.115:0/1092602829 >> v2:172.21.15.142:3300/0 conn(0x7f6e8415afe0 0x7f6e8406b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.514+0000 7f6e837fe640 1 -- 172.21.15.115:0/1092602829 >> v2:172.21.15.142:3300/0 conn(0x7f6e8415afe0 msgr2=0x7f6e8406b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.514+0000 7f6e837fe640 1 --2- 172.21.15.115:0/1092602829 >> v2:172.21.15.142:3300/0 conn(0x7f6e8415afe0 0x7f6e8406b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.514+0000 7f6e837fe640 1 -- 172.21.15.115:0/1092602829 >> v1:172.21.15.142:6789/0 conn(0x7f6e841584b0 legacy=0x7f6e840cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.514+0000 7f6e837fe640 1 -- 172.21.15.115:0/1092602829 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8415b550 con 0x7f6e8406b7b0 2023-12-01T01:51:36.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.514+0000 7f6e7bfff640 1 --2- 172.21.15.115:0/1092602829 >> v2:172.21.15.142:3300/0 conn(0x7f6e8415afe0 0x7f6e8406b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.515+0000 7f6e837fe640 1 --2- 172.21.15.115:0/1092602829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8406b7b0 0x7f6e840cc680 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6e840caff0 tx=0x7f6e7c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35989e44971b8793 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e827fc640 1 -- 172.21.15.115:0/1092602829 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c02ca80 con 0x7f6e8406b7b0 2023-12-01T01:51:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e827fc640 1 -- 172.21.15.115:0/1092602829 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c02cbe0 con 0x7f6e8406b7b0 2023-12-01T01:51:36.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e827fc640 1 -- 172.21.15.115:0/1092602829 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0343f0 con 0x7f6e8406b7b0 2023-12-01T01:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e83fff640 1 -- 172.21.15.115:0/1092602829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8406b7b0 msgr2=0x7f6e840cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e83fff640 1 --2- 172.21.15.115:0/1092602829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8406b7b0 0x7f6e840cc680 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6e840caff0 tx=0x7f6e7c02bee0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e83fff640 1 -- 172.21.15.115:0/1092602829 shutdown_connections 2023-12-01T01:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e83fff640 1 --2- 172.21.15.115:0/1092602829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8406b7b0 0x7f6e840cc680 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e83fff640 1 --2- 172.21.15.115:0/1092602829 >> v2:172.21.15.142:3300/0 conn(0x7f6e8415afe0 0x7f6e8406b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e83fff640 1 -- 172.21.15.115:0/1092602829 >> 172.21.15.115:0/1092602829 conn(0x7f6e840ce640 msgr2=0x7f6e840cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e83fff640 1 -- 172.21.15.115:0/1092602829 shutdown_connections 2023-12-01T01:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.516+0000 7f6e83fff640 1 -- 172.21.15.115:0/1092602829 wait complete. 2023-12-01T01:51:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.518+0000 7f6e83fff640 1 Processor -- start 2023-12-01T01:51:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.518+0000 7f6e83fff640 1 -- start start 2023-12-01T01:51:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.519+0000 7f6e83fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e8406b7b0 0x7f6e841506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.519+0000 7f6e83fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e841584b0 0x7f6e840ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.519+0000 7f6e83fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8415afe0 0x7f6e840acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.519+0000 7f6e83fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e841f5d90 con 0x7f6e8415afe0 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.519+0000 7f6e83fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840b2430 con 0x7f6e841584b0 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.519+0000 7f6e83fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840b2710 con 0x7f6e8406b7b0 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e837fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8415afe0 0x7f6e840acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e837fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8415afe0 0x7f6e840acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39428/0 (socket says 172.21.15.115:39428) 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e837fe640 1 -- 172.21.15.115:0/2847759288 learned_addr learned my addr 172.21.15.115:0/2847759288 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e7bfff640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e841584b0 0x7f6e840ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e82ffd640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e8406b7b0 0x7f6e841506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e837fe640 1 -- 172.21.15.115:0/2847759288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e8406b7b0 msgr2=0x7f6e841506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e837fe640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e8406b7b0 0x7f6e841506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e837fe640 1 -- 172.21.15.115:0/2847759288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e841584b0 msgr2=0x7f6e840ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e837fe640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e841584b0 0x7f6e840ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e837fe640 1 -- 172.21.15.115:0/2847759288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c009b30 con 0x7f6e8415afe0 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e7bfff640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e841584b0 0x7f6e840ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.520+0000 7f6e837fe640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8415afe0 0x7f6e840acdd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c036040 tx=0x7f6e7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:36.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.521+0000 7f6e80ff9640 1 -- 172.21.15.115:0/2847759288 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c03e450 con 0x7f6e8415afe0 2023-12-01T01:51:36.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.521+0000 7f6e80ff9640 1 -- 172.21.15.115:0/2847759288 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c03e5d0 con 0x7f6e8415afe0 2023-12-01T01:51:36.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.521+0000 7f6e80ff9640 1 -- 172.21.15.115:0/2847759288 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c03e980 con 0x7f6e8415afe0 2023-12-01T01:51:36.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.521+0000 7f6e83fff640 1 -- 172.21.15.115:0/2847759288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e841f6070 con 0x7f6e8415afe0 2023-12-01T01:51:36.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.521+0000 7f6e83fff640 1 -- 172.21.15.115:0/2847759288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e840c5160 con 0x7f6e8415afe0 2023-12-01T01:51:36.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.522+0000 7f6e83fff640 1 -- 172.21.15.115:0/2847759288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e840673a0 con 0x7f6e8415afe0 2023-12-01T01:51:36.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.523+0000 7f6e80ff9640 1 -- 172.21.15.115:0/2847759288 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6e7c02cd50 con 0x7f6e8415afe0 2023-12-01T01:51:36.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.524+0000 7f6e80ff9640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e6406a580 0x7f6e6406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:36.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.524+0000 7f6e80ff9640 1 -- 172.21.15.115:0/2847759288 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e7c0ab250 con 0x7f6e8415afe0 2023-12-01T01:51:36.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.524+0000 7f6e82ffd640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e6406a580 0x7f6e6406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:36.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.525+0000 7f6e82ffd640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e6406a580 0x7f6e6406ca40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e74003d90 tx=0x7f6e74009040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:36.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.529+0000 7f6e80ff9640 1 -- 172.21.15.115:0/2847759288 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6e7c076720 con 0x7f6e8415afe0 2023-12-01T01:51:36.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.757+0000 7f6e83fff640 1 -- 172.21.15.115:0/2847759288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6e840ce4a0 con 0x7f6e8415afe0 2023-12-01T01:51:36.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.757+0000 7f6e80ff9640 1 -- 172.21.15.115:0/2847759288 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6e840ce4a0 con 0x7f6e8415afe0 2023-12-01T01:51:36.758 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:51:36.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 -- 172.21.15.115:0/2847759288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e6406a580 msgr2=0x7f6e6406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e6406a580 0x7f6e6406ca40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e74003d90 tx=0x7f6e74009040 comp rx=0 tx=0).stop 2023-12-01T01:51:36.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 -- 172.21.15.115:0/2847759288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8415afe0 msgr2=0x7f6e840acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:36.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8415afe0 0x7f6e840acdd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c036040 tx=0x7f6e7c004a40 comp rx=0 tx=0).stop 2023-12-01T01:51:36.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 -- 172.21.15.115:0/2847759288 shutdown_connections 2023-12-01T01:51:36.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e6406a580 0x7f6e6406ca40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e8415afe0 0x7f6e840acdd0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e841584b0 0x7f6e840ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 --2- 172.21.15.115:0/2847759288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e8406b7b0 0x7f6e841506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:36.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 -- 172.21.15.115:0/2847759288 >> 172.21.15.115:0/2847759288 conn(0x7f6e840ce640 msgr2=0x7f6e840c3500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:36.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 -- 172.21.15.115:0/2847759288 shutdown_connections 2023-12-01T01:51:36.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:36.760+0000 7f6e83fff640 1 -- 172.21.15.115:0/2847759288 wait complete. 2023-12-01T01:51:36.779 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T01:51:36.779 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:51:36.780 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T01:51:36.780 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T01:51:36.780 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:51:36.780 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T01:51:36.780 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T01:51:36.780 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:51:36.780 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:51:36.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:36.780 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T01:51:36.833 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:51:36.833 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:51:36.833 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:51:36.833 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:51:36.833 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:51:36.833 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:51:36.833 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 41460sec preferred_lft 41460sec 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:51:36.834 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:51:36.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:36.836 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:51:36.925 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T01:51:36.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:36.929 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T01:51:36.983 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T01:51:36.983 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:51:36.983 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:51:36.983 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:51:36.984 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:51:36.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:36.984 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:51:36.984 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:51:36.984 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:51:36.984 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:51:36.984 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:51:36.984 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:51:37.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:37.170 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:51:37.221 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T01:51:37.222 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T01:51:37.222 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:51:37.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:37.222 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T01:51:37.314 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:51:37.314 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T01:51:37.368 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:51:37.368 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:51:37.368 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:51:37.417 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:51:37.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:37.417 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:51:37.553 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:51:37.554 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:51:37.554 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:51:37.574 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:51:37.574 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:51:37.574 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:51:37.574 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:51:37.574 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:51:37.599 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:51:37.599 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:51:37.599 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:51:37.599 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:51:37.599 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:51:37.599 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:51:37.599 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:51:37.599 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:51:37.603 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T01:51:37.603 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:51:37.603 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:51:37.603 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:51:37.603 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:51:37.608 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:51:37.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:37.609 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:51:37.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:37.696 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:51:37.761 INFO:teuthology.orchestra.run.smithi115.stdout:client6627 2023-12-01T01:51:37.761 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:51:37.761 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:37.813 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T01:51:38.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.024+0000 7fc99bdbf640 1 Processor -- start 2023-12-01T01:51:38.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.024+0000 7fc99bdbf640 1 -- start start 2023-12-01T01:51:38.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.026+0000 7fc99bdbf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc9940ccc80 0x7fc99406fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.026+0000 7fc99bdbf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9940ad2b0 0x7fc99406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.026+0000 7fc99bdbf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99415d600 con 0x7fc99415d220 2023-12-01T01:51:38.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.026+0000 7fc99bdbf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99406cc10 con 0x7fc9940ccc80 2023-12-01T01:51:38.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.026+0000 7fc99bdbf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99406c720 con 0x7fc9940ad2b0 2023-12-01T01:51:38.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.027+0000 7fc99935a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc9940ccc80 0x7fc99406fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.027+0000 7fc99935a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc9940ccc80 0x7fc99406fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:60878/0 (socket says 172.21.15.115:60878) 2023-12-01T01:51:38.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.027+0000 7fc99935a640 1 -- 172.21.15.115:0/2838801900 learned_addr learned my addr 172.21.15.115:0/2838801900 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:38.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.027+0000 7fc99a35c640 1 --2- 172.21.15.115:0/2838801900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9940ad2b0 0x7fc99406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.027+0000 7fc99935a640 1 -- 172.21.15.115:0/2838801900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9940ad2b0 msgr2=0x7fc99406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:38.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.027+0000 7fc99935a640 1 --2- 172.21.15.115:0/2838801900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9940ad2b0 0x7fc99406fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.027+0000 7fc99935a640 1 -- 172.21.15.115:0/2838801900 >> v1:172.21.15.142:6789/0 conn(0x7fc99415d220 legacy=0x7fc9940cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:38.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.027+0000 7fc99935a640 1 -- 172.21.15.115:0/2838801900 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99406c4b0 con 0x7fc9940ccc80 2023-12-01T01:51:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.029+0000 7fc99935a640 1 --2- 172.21.15.115:0/2838801900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9940ccc80 0x7fc99406fab0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc984009870 tx=0x7fc98402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be5a457ab7a8d700 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.029+0000 7fc998b59640 1 -- 172.21.15.115:0/2838801900 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984031640 con 0x7fc9940ccc80 2023-12-01T01:51:38.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.029+0000 7fc998b59640 1 -- 172.21.15.115:0/2838801900 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9840317a0 con 0x7fc9940ccc80 2023-12-01T01:51:38.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.029+0000 7fc998b59640 1 -- 172.21.15.115:0/2838801900 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984031b30 con 0x7fc9940ccc80 2023-12-01T01:51:38.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.030+0000 7fc99bdbf640 1 -- 172.21.15.115:0/2838801900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9940ccc80 msgr2=0x7fc99406fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:38.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.030+0000 7fc99bdbf640 1 --2- 172.21.15.115:0/2838801900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9940ccc80 0x7fc99406fab0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc984009870 tx=0x7fc98402b8f0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.030+0000 7fc99bdbf640 1 -- 172.21.15.115:0/2838801900 shutdown_connections 2023-12-01T01:51:38.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.030+0000 7fc99bdbf640 1 --2- 172.21.15.115:0/2838801900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9940ad2b0 0x7fc99406fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.030+0000 7fc99bdbf640 1 --2- 172.21.15.115:0/2838801900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9940ccc80 0x7fc99406fab0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.030+0000 7fc99bdbf640 1 -- 172.21.15.115:0/2838801900 >> 172.21.15.115:0/2838801900 conn(0x7fc9940ce880 msgr2=0x7fc9940cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:38.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.030+0000 7fc99bdbf640 1 -- 172.21.15.115:0/2838801900 shutdown_connections 2023-12-01T01:51:38.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.030+0000 7fc99bdbf640 1 -- 172.21.15.115:0/2838801900 wait complete. 2023-12-01T01:51:38.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.032+0000 7fc99bdbf640 1 Processor -- start 2023-12-01T01:51:38.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.032+0000 7fc99bdbf640 1 -- start start 2023-12-01T01:51:38.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.033+0000 7fc99bdbf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9940ad2b0 0x7fc9940cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.033+0000 7fc99bdbf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9940ccc80 0x7fc9940b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.033+0000 7fc99bdbf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc99415d220 0x7fc9940c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.033+0000 7fc99bdbf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940c8670 con 0x7fc99415d220 2023-12-01T01:51:38.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.033+0000 7fc99bdbf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940b4260 con 0x7fc9940ccc80 2023-12-01T01:51:38.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.033+0000 7fc99bdbf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940c79f0 con 0x7fc9940ad2b0 2023-12-01T01:51:38.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.033+0000 7fc99a35c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc99415d220 0x7fc9940c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.033+0000 7fc99a35c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc99415d220 0x7fc9940c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50316/0 (socket says 172.21.15.115:50316) 2023-12-01T01:51:38.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.033+0000 7fc99a35c640 1 -- 172.21.15.115:0/3071353062 learned_addr learned my addr 172.21.15.115:0/3071353062 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:38.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc99935a640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9940ccc80 0x7fc9940b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc99a35c640 1 -- 172.21.15.115:0/3071353062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9940ad2b0 msgr2=0x7fc9940cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:38.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc99a35c640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9940ad2b0 0x7fc9940cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc99a35c640 1 -- 172.21.15.115:0/3071353062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9940ccc80 msgr2=0x7fc9940b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:38.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc99a35c640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9940ccc80 0x7fc9940b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc99a35c640 1 -- 172.21.15.115:0/3071353062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840094d0 con 0x7fc99415d220 2023-12-01T01:51:38.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc99935a640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9940ccc80 0x7fc9940b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:38.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc99a35c640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc99415d220 0x7fc9940c71a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00efc0 tx=0x7fc98c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:38.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc98a7fc640 1 -- 172.21.15.115:0/3071353062 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c009ba0 con 0x7fc99415d220 2023-12-01T01:51:38.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc98a7fc640 1 -- 172.21.15.115:0/3071353062 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c00f040 con 0x7fc99415d220 2023-12-01T01:51:38.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.034+0000 7fc98a7fc640 1 -- 172.21.15.115:0/3071353062 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0146c0 con 0x7fc99415d220 2023-12-01T01:51:38.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.035+0000 7fc99bdbf640 1 -- 172.21.15.115:0/3071353062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9940c7cd0 con 0x7fc99415d220 2023-12-01T01:51:38.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.035+0000 7fc99bdbf640 1 -- 172.21.15.115:0/3071353062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9940c8290 con 0x7fc99415d220 2023-12-01T01:51:38.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.036+0000 7fc99bdbf640 1 -- 172.21.15.115:0/3071353062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99406ce10 con 0x7fc99415d220 2023-12-01T01:51:38.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.037+0000 7fc98a7fc640 1 -- 172.21.15.115:0/3071353062 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc98c014820 con 0x7fc99415d220 2023-12-01T01:51:38.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.038+0000 7fc98a7fc640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc97806a550 0x7fc97806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.038+0000 7fc98a7fc640 1 -- 172.21.15.115:0/3071353062 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc98c089a10 con 0x7fc99415d220 2023-12-01T01:51:38.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.038+0000 7fc999b5b640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc97806a550 0x7fc97806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.038+0000 7fc999b5b640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc97806a550 0x7fc97806ca10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc990000f60 tx=0x7fc9900079b0 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:38.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.042+0000 7fc98a7fc640 1 -- 172.21.15.115:0/3071353062 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc98c054de0 con 0x7fc99415d220 2023-12-01T01:51:38.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.256+0000 7fc99bdbf640 1 -- 172.21.15.115:0/3071353062 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc99406f2f0 con 0x7fc97806a550 2023-12-01T01:51:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.260+0000 7fc98a7fc640 1 -- 172.21.15.115:0/3071353062 <== mgr.6201 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc990005080 con 0x7fc97806a550 2023-12-01T01:51:38.260 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:51:38.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:51:38.261 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T01:51:38.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:51:38.261 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:51:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.262+0000 7fc99bdbf640 1 -- 172.21.15.115:0/3071353062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc97806a550 msgr2=0x7fc97806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.262+0000 7fc99bdbf640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc97806a550 0x7fc97806ca10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc990000f60 tx=0x7fc9900079b0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.262+0000 7fc99bdbf640 1 -- 172.21.15.115:0/3071353062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc99415d220 msgr2=0x7fc9940c71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.262+0000 7fc99bdbf640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc99415d220 0x7fc9940c71a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00efc0 tx=0x7fc98c00c490 comp rx=0 tx=0).stop 2023-12-01T01:51:38.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.262+0000 7fc99bdbf640 1 -- 172.21.15.115:0/3071353062 shutdown_connections 2023-12-01T01:51:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.263+0000 7fc99bdbf640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc97806a550 0x7fc97806ca10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.263+0000 7fc99bdbf640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc99415d220 0x7fc9940c71a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.263+0000 7fc99bdbf640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9940ccc80 0x7fc9940b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.263+0000 7fc99bdbf640 1 --2- 172.21.15.115:0/3071353062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9940ad2b0 0x7fc9940cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.263+0000 7fc99bdbf640 1 -- 172.21.15.115:0/3071353062 >> 172.21.15.115:0/3071353062 conn(0x7fc9940ce880 msgr2=0x7fc99415bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.263+0000 7fc99bdbf640 1 -- 172.21.15.115:0/3071353062 shutdown_connections 2023-12-01T01:51:38.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.263+0000 7fc99bdbf640 1 -- 172.21.15.115:0/3071353062 wait complete. 2023-12-01T01:51:38.283 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T01:51:38.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.492+0000 7fadb142c640 1 Processor -- start 2023-12-01T01:51:38.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.492+0000 7fadb142c640 1 -- start start 2023-12-01T01:51:38.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.494+0000 7fadb142c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fadac0ccca0 0x7fadac06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.494+0000 7fadb142c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadac0cb100 0x7fadac06a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.494+0000 7fadb142c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fadac0d0c40 0x7fadac0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.494+0000 7fadb142c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadac0d07b0 con 0x7fadac0d0c40 2023-12-01T01:51:38.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.494+0000 7fadb142c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadac0cb560 con 0x7fadac0ccca0 2023-12-01T01:51:38.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.494+0000 7fadb142c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac06af00 con 0x7fadac0cb100 2023-12-01T01:51:38.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fadab7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fadac0d0c40 0x7fadac0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fadab7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fadac0d0c40 0x7fadac0d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50326/0 (socket says 172.21.15.115:50326) 2023-12-01T01:51:38.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fadab7fe640 1 -- 172.21.15.115:0/2919825089 learned_addr learned my addr 172.21.15.115:0/2919825089 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fada27fc640 1 --2- 172.21.15.115:0/2919825089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadac0cb100 0x7fadac06a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fadaaffd640 1 --2- 172.21.15.115:0/2919825089 >> v2:172.21.15.142:3300/0 conn(0x7fadac0ccca0 0x7fadac06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fadab7fe640 1 -- 172.21.15.115:0/2919825089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadac0cb100 msgr2=0x7fadac06a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fadab7fe640 1 --2- 172.21.15.115:0/2919825089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadac0cb100 0x7fadac06a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fadab7fe640 1 -- 172.21.15.115:0/2919825089 >> v2:172.21.15.142:3300/0 conn(0x7fadac0ccca0 msgr2=0x7fadac06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:38.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fadab7fe640 1 --2- 172.21.15.115:0/2919825089 >> v2:172.21.15.142:3300/0 conn(0x7fadac0ccca0 0x7fadac06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fadab7fe640 1 -- 172.21.15.115:0/2919825089 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac0d0160 con 0x7fadac0d0c40 2023-12-01T01:51:38.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.495+0000 7fadaaffd640 1 --2- 172.21.15.115:0/2919825089 >> v2:172.21.15.142:3300/0 conn(0x7fadac0ccca0 0x7fadac06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:38.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.496+0000 7fadab7fe640 1 --2- 172.21.15.115:0/2919825089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0d0c40 0x7fadac0d10d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fadac0d1b40 tx=0x7fada402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898690681ccd4eac server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.497+0000 7fadaa7fc640 1 -- 172.21.15.115:0/2919825089 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada402ca70 con 0x7fadac0d0c40 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.497+0000 7fadaa7fc640 1 -- 172.21.15.115:0/2919825089 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada402cbd0 con 0x7fadac0d0c40 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.497+0000 7fadaa7fc640 1 -- 172.21.15.115:0/2919825089 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada40343f0 con 0x7fadac0d0c40 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.497+0000 7fadb142c640 1 -- 172.21.15.115:0/2919825089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0d0c40 msgr2=0x7fadac0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.497+0000 7fadb142c640 1 --2- 172.21.15.115:0/2919825089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0d0c40 0x7fadac0d10d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fadac0d1b40 tx=0x7fada402bed0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.497+0000 7fadb142c640 1 -- 172.21.15.115:0/2919825089 shutdown_connections 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.497+0000 7fadb142c640 1 --2- 172.21.15.115:0/2919825089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0d0c40 0x7fadac0d10d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.497+0000 7fadb142c640 1 --2- 172.21.15.115:0/2919825089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadac0cb100 0x7fadac06a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.497+0000 7fadb142c640 1 --2- 172.21.15.115:0/2919825089 >> v2:172.21.15.142:3300/0 conn(0x7fadac0ccca0 0x7fadac06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.497+0000 7fadb142c640 1 -- 172.21.15.115:0/2919825089 >> 172.21.15.115:0/2919825089 conn(0x7fadac0713d0 msgr2=0x7fadac0717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:38.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.498+0000 7fadb142c640 1 -- 172.21.15.115:0/2919825089 shutdown_connections 2023-12-01T01:51:38.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.498+0000 7fadb142c640 1 -- 172.21.15.115:0/2919825089 wait complete. 2023-12-01T01:51:38.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.499+0000 7fadb142c640 1 Processor -- start 2023-12-01T01:51:38.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.499+0000 7fadb142c640 1 -- start start 2023-12-01T01:51:38.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.500+0000 7fadb142c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0cb100 0x7fadac160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.500+0000 7fadb142c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadac0ccca0 0x7fadac157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.500+0000 7fadb142c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac0d0c40 0x7fadac1580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.500+0000 7fadb142c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0caa50 con 0x7fadac0cb100 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.500+0000 7fadb142c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0b3640 con 0x7fadac0d0c40 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.500+0000 7fadb142c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0b3920 con 0x7fadac0ccca0 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fadaaffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0cb100 0x7fadac160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fadaaffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0cb100 0x7fadac160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50340/0 (socket says 172.21.15.115:50340) 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fadaaffd640 1 -- 172.21.15.115:0/1750903613 learned_addr learned my addr 172.21.15.115:0/1750903613 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fadaaffd640 1 -- 172.21.15.115:0/1750903613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadac0ccca0 msgr2=0x7fadac157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fadab7fe640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac0d0c40 0x7fadac1580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fada27fc640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadac0ccca0 0x7fadac157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fadaaffd640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadac0ccca0 0x7fadac157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fadaaffd640 1 -- 172.21.15.115:0/1750903613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac0d0c40 msgr2=0x7fadac1580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:38.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fadaaffd640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac0d0c40 0x7fadac1580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:38.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fadaaffd640 1 -- 172.21.15.115:0/1750903613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada4009b30 con 0x7fadac0cb100 2023-12-01T01:51:38.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.501+0000 7fadaaffd640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0cb100 0x7fadac160ce0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00c7f0 tx=0x7fad9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:38.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.502+0000 7fada3fff640 1 -- 172.21.15.115:0/1750903613 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c007820 con 0x7fadac0cb100 2023-12-01T01:51:38.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.502+0000 7fada3fff640 1 -- 172.21.15.115:0/1750903613 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9c007980 con 0x7fadac0cb100 2023-12-01T01:51:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.502+0000 7fada3fff640 1 -- 172.21.15.115:0/1750903613 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c007cf0 con 0x7fadac0cb100 2023-12-01T01:51:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.502+0000 7fadb142c640 1 -- 172.21.15.115:0/1750903613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac0b3c00 con 0x7fadac0cb100 2023-12-01T01:51:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.502+0000 7fadb142c640 1 -- 172.21.15.115:0/1750903613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac0b4400 con 0x7fadac0cb100 2023-12-01T01:51:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.503+0000 7fadb142c640 1 -- 172.21.15.115:0/1750903613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac06be50 con 0x7fadac0cb100 2023-12-01T01:51:38.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.504+0000 7fada3fff640 1 -- 172.21.15.115:0/1750903613 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 32) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fad9c0113f0 con 0x7fadac0cb100 2023-12-01T01:51:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.505+0000 7fada3fff640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad8c06a580 0x7fad8c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.505+0000 7fada3fff640 1 -- 172.21.15.115:0/1750903613 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad9c088df0 con 0x7fadac0cb100 2023-12-01T01:51:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.505+0000 7fada27fc640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad8c06a580 0x7fad8c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:38.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.506+0000 7fada27fc640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad8c06a580 0x7fad8c06ca40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fad94006fd0 tx=0x7fad94008040 comp rx=0 tx=0).ready entity=mgr.6201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:38.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.509+0000 7fada3fff640 1 -- 172.21.15.115:0/1750903613 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fad9c054250 con 0x7fadac0cb100 2023-12-01T01:51:38.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:38.751+0000 7fadb142c640 1 -- 172.21.15.115:0/1750903613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fadac04e890 con 0x7fadac0cb100 2023-12-01T01:51:39.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.531+0000 7fada3fff640 1 -- 172.21.15.115:0/1750903613 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 33) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fad9c087310 con 0x7fadac0cb100 2023-12-01T01:51:39.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.532+0000 7fada3fff640 1 -- 172.21.15.115:0/1750903613 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v33) v1 ==== 92+0+0 (secure 0 0 0) 0x7fad9c071580 con 0x7fadac0cb100 2023-12-01T01:51:39.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 -- 172.21.15.115:0/1750903613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad8c06a580 msgr2=0x7fad8c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:39.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad8c06a580 0x7fad8c06ca40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fad94006fd0 tx=0x7fad94008040 comp rx=0 tx=0).stop 2023-12-01T01:51:39.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 -- 172.21.15.115:0/1750903613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0cb100 msgr2=0x7fadac160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:39.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0cb100 0x7fadac160ce0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00c7f0 tx=0x7fad9c00ccb0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 -- 172.21.15.115:0/1750903613 shutdown_connections 2023-12-01T01:51:39.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad8c06a580 0x7fad8c06ca40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fadac0ccca0 0x7fadac157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadac0d0c40 0x7fadac1580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 --2- 172.21.15.115:0/1750903613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fadac0cb100 0x7fadac160ce0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 -- 172.21.15.115:0/1750903613 >> 172.21.15.115:0/1750903613 conn(0x7fadac0713d0 msgr2=0x7fadac0b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:39.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 -- 172.21.15.115:0/1750903613 shutdown_connections 2023-12-01T01:51:39.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.535+0000 7fadb142c640 1 -- 172.21.15.115:0/1750903613 wait complete. 2023-12-01T01:51:39.555 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T01:51:39.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.756+0000 7f0e9e5a7640 1 Processor -- start 2023-12-01T01:51:39.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.756+0000 7f0e9e5a7640 1 -- start start 2023-12-01T01:51:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.757+0000 7f0e9e5a7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0e980cfd60 0x7f0e980cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.757+0000 7f0e9e5a7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e980ce5e0 con 0x7f0e980b5fa0 2023-12-01T01:51:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.757+0000 7f0e9e5a7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e980ce300 con 0x7f0e9815cd90 2023-12-01T01:51:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.757+0000 7f0e9e5a7640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e980d0ec0 con 0x7f0e980cfd60 2023-12-01T01:51:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.758+0000 7f0e977fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0e980cfd60 0x7f0e980cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.758+0000 7f0e9cb44640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0e980b5fa0 0x7f0e980d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52496/0 (socket says 172.21.15.115:52496) 2023-12-01T01:51:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.758+0000 7f0e9cb44640 1 -- 172.21.15.115:0/3360012642 learned_addr learned my addr 172.21.15.115:0/3360012642 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:39.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.758+0000 7f0e96ffd640 1 -- 172.21.15.115:0/3360012642 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586924119 0 0) 0x7f0e90001110 con 0x7f0e980b5fa0 2023-12-01T01:51:39.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.758+0000 7f0e96ffd640 1 -- 172.21.15.115:0/3360012642 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e84003520 con 0x7f0e980b5fa0 2023-12-01T01:51:39.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.758+0000 7f0e977fe640 1 -- 172.21.15.115:0/3360012642 >> v1:172.21.15.142:6789/0 conn(0x7f0e9815cd90 legacy=0x7f0e980cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:39.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.758+0000 7f0e977fe640 1 -- 172.21.15.115:0/3360012642 >> v1:172.21.15.115:6789/0 conn(0x7f0e980b5fa0 legacy=0x7f0e980d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:39.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.758+0000 7f0e977fe640 1 -- 172.21.15.115:0/3360012642 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e980d10a0 con 0x7f0e980cfd60 2023-12-01T01:51:39.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.759+0000 7f0e977fe640 1 --2- 172.21.15.115:0/3360012642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e980cfd60 0x7f0e980cdd20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0e800097a0 tx=0x7f0e8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f030ac02561ac8bc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:39.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.760+0000 7f0e96ffd640 1 -- 172.21.15.115:0/3360012642 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80031830 con 0x7f0e980cfd60 2023-12-01T01:51:39.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.760+0000 7f0e96ffd640 1 -- 172.21.15.115:0/3360012642 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e80031990 con 0x7f0e980cfd60 2023-12-01T01:51:39.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.760+0000 7f0e96ffd640 1 -- 172.21.15.115:0/3360012642 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80031d00 con 0x7f0e980cfd60 2023-12-01T01:51:39.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.760+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3360012642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e980cfd60 msgr2=0x7f0e980cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:39.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.760+0000 7f0e9e5a7640 1 --2- 172.21.15.115:0/3360012642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e980cfd60 0x7f0e980cdd20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0e800097a0 tx=0x7f0e8002b8d0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.760+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3360012642 shutdown_connections 2023-12-01T01:51:39.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.760+0000 7f0e9e5a7640 1 --2- 172.21.15.115:0/3360012642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e980cfd60 0x7f0e980cdd20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.760+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3360012642 >> 172.21.15.115:0/3360012642 conn(0x7f0e98065960 msgr2=0x7f0e98065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:39.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.760+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3360012642 shutdown_connections 2023-12-01T01:51:39.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.761+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3360012642 wait complete. 2023-12-01T01:51:39.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.762+0000 7f0e9e5a7640 1 Processor -- start 2023-12-01T01:51:39.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.762+0000 7f0e9e5a7640 1 -- start start 2023-12-01T01:51:39.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.763+0000 7f0e9e5a7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980b5fa0 0x7f0e9815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.763+0000 7f0e9e5a7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cfd60 0x7f0e9815fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.763+0000 7f0e9e5a7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815cd90 0x7f0e980cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.763+0000 7f0e9e5a7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9815d9e0 con 0x7f0e9815cd90 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.763+0000 7f0e9e5a7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9815ddc0 con 0x7f0e980b5fa0 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.763+0000 7f0e9e5a7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9815e0a0 con 0x7f0e980cfd60 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.764+0000 7f0e9cb44640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815cd90 0x7f0e980cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.764+0000 7f0e9cb44640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815cd90 0x7f0e980cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50362/0 (socket says 172.21.15.115:50362) 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.764+0000 7f0e9cb44640 1 -- 172.21.15.115:0/3443951739 learned_addr learned my addr 172.21.15.115:0/3443951739 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.764+0000 7f0e97fff640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980b5fa0 0x7f0e9815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.764+0000 7f0e977fe640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cfd60 0x7f0e9815fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.764+0000 7f0e9cb44640 1 -- 172.21.15.115:0/3443951739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cfd60 msgr2=0x7f0e9815fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.764+0000 7f0e9cb44640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cfd60 0x7f0e9815fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.764+0000 7f0e9cb44640 1 -- 172.21.15.115:0/3443951739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980b5fa0 msgr2=0x7f0e9815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.764+0000 7f0e9cb44640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980b5fa0 0x7f0e9815d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.764+0000 7f0e9cb44640 1 -- 172.21.15.115:0/3443951739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e80009400 con 0x7f0e9815cd90 2023-12-01T01:51:39.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.765+0000 7f0e9cb44640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815cd90 0x7f0e980cb380 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000c700 tx=0x7f0e9000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:39.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.765+0000 7f0e94ff9640 1 -- 172.21.15.115:0/3443951739 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90005ab0 con 0x7f0e9815cd90 2023-12-01T01:51:39.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.765+0000 7f0e94ff9640 1 -- 172.21.15.115:0/3443951739 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e90005c10 con 0x7f0e9815cd90 2023-12-01T01:51:39.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.765+0000 7f0e94ff9640 1 -- 172.21.15.115:0/3443951739 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90001d50 con 0x7f0e9815cd90 2023-12-01T01:51:39.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.765+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3443951739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9815e380 con 0x7f0e9815cd90 2023-12-01T01:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.765+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3443951739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e981f2490 con 0x7f0e9815cd90 2023-12-01T01:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.766+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3443951739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e981f2a30 con 0x7f0e9815cd90 2023-12-01T01:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.767+0000 7f0e94ff9640 1 -- 172.21.15.115:0/3443951739 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 33) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f0e90007070 con 0x7f0e9815cd90 2023-12-01T01:51:39.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.768+0000 7f0e94ff9640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a5a0 0x7f0e7806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:39.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.768+0000 7f0e94ff9640 1 -- 172.21.15.115:0/3443951739 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e90087f90 con 0x7f0e9815cd90 2023-12-01T01:51:39.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.768+0000 7f0e97fff640 1 -- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a5a0 msgr2=0x7f0e7806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:39.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.768+0000 7f0e97fff640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a5a0 0x7f0e7806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:39.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.773+0000 7f0e94ff9640 1 -- 172.21.15.115:0/3443951739 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0e981f2a30 con 0x7f0e9815cd90 2023-12-01T01:51:39.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.966+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3443951739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f0e980ce5e0 con 0x7f0e9815cd90 2023-12-01T01:51:39.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.969+0000 7f0e97fff640 1 -- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a5a0 msgr2=0x7f0e7806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:39.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.969+0000 7f0e97fff640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a5a0 0x7f0e7806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:51:39.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.972+0000 7f0e94ff9640 1 -- 172.21.15.115:0/3443951739 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0e980ce5e0 con 0x7f0e9815cd90 2023-12-01T01:51:39.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a5a0 msgr2=0x7f0e7806ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:39.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a5a0 0x7f0e7806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815cd90 msgr2=0x7f0e980cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:39.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815cd90 0x7f0e980cb380 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000c700 tx=0x7f0e9000cbc0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3443951739 shutdown_connections 2023-12-01T01:51:39.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a5a0 0x7f0e7806ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cfd60 0x7f0e9815fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815cd90 0x7f0e980cb380 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 --2- 172.21.15.115:0/3443951739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980b5fa0 0x7f0e9815d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:39.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3443951739 >> 172.21.15.115:0/3443951739 conn(0x7f0e98065960 msgr2=0x7f0e980b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:39.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3443951739 shutdown_connections 2023-12-01T01:51:39.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:39.975+0000 7f0e9e5a7640 1 -- 172.21.15.115:0/3443951739 wait complete. 2023-12-01T01:51:39.993 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T01:51:40.032 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:40.031+0000 7eff4c23f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:51:40.063 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:40.062+0000 7f9d6c2da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:51:40.085 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:40.083+0000 7eff4c23f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:51:40.118 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:40.118+0000 7f9d6c2da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:51:40.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.191+0000 7fa487fff640 1 Processor -- start 2023-12-01T01:51:40.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.191+0000 7fa487fff640 1 -- start start 2023-12-01T01:51:40.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.193+0000 7fa487fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4881597e0 0x7fa4880cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.193+0000 7fa487fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa4880b30e0 0x7fa48806d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.193+0000 7fa487fff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4880cd860 con 0x7fa48806deb0 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.193+0000 7fa487fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48806e340 con 0x7fa4880b30e0 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.193+0000 7fa487fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4880cdb40 con 0x7fa4881597e0 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.194+0000 7fa4867fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa4880b30e0 0x7fa48806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.194+0000 7fa4867fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa4880b30e0 0x7fa48806d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50368/0 (socket says 172.21.15.115:50368) 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.194+0000 7fa4867fc640 1 -- 172.21.15.115:0/35012016 learned_addr learned my addr 172.21.15.115:0/35012016 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.194+0000 7fa4877fe640 1 --1- 172.21.15.115:0/35012016 >> v1:172.21.15.115:6789/0 conn(0x7fa48806deb0 0x7fa48815d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52500/0 (socket says 172.21.15.115:52500) 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.194+0000 7fa486ffd640 1 --2- 172.21.15.115:0/35012016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4881597e0 0x7fa4880cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.194+0000 7fa4867fc640 1 -- 172.21.15.115:0/35012016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4881597e0 msgr2=0x7fa4880cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.194+0000 7fa4867fc640 1 --2- 172.21.15.115:0/35012016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4881597e0 0x7fa4880cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.194+0000 7fa4867fc640 1 -- 172.21.15.115:0/35012016 >> v1:172.21.15.115:6789/0 conn(0x7fa48806deb0 legacy=0x7fa48815d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.194+0000 7fa4867fc640 1 -- 172.21.15.115:0/35012016 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4880cde20 con 0x7fa4880b30e0 2023-12-01T01:51:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.194+0000 7fa485ffb640 1 -- 172.21.15.115:0/35012016 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348195855 0 0) 0x7fa4880cd860 con 0x7fa48806deb0 2023-12-01T01:51:40.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.195+0000 7fa4867fc640 1 --2- 172.21.15.115:0/35012016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4880b30e0 0x7fa48806d970 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009870 tx=0x7fa47c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dc7ceca5ac8db8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:40.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.196+0000 7fa485ffb640 1 -- 172.21.15.115:0/35012016 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c030840 con 0x7fa4880b30e0 2023-12-01T01:51:40.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.196+0000 7fa485ffb640 1 -- 172.21.15.115:0/35012016 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c0309a0 con 0x7fa4880b30e0 2023-12-01T01:51:40.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.196+0000 7fa485ffb640 1 -- 172.21.15.115:0/35012016 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c030d30 con 0x7fa4880b30e0 2023-12-01T01:51:40.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.196+0000 7fa487fff640 1 -- 172.21.15.115:0/35012016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4880b30e0 msgr2=0x7fa48806d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:40.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.196+0000 7fa487fff640 1 --2- 172.21.15.115:0/35012016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4880b30e0 0x7fa48806d970 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009870 tx=0x7fa47c02b980 comp rx=0 tx=0).stop 2023-12-01T01:51:40.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.196+0000 7fa487fff640 1 -- 172.21.15.115:0/35012016 shutdown_connections 2023-12-01T01:51:40.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.196+0000 7fa487fff640 1 --2- 172.21.15.115:0/35012016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4880b30e0 0x7fa48806d970 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.196+0000 7fa487fff640 1 --2- 172.21.15.115:0/35012016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4881597e0 0x7fa4880cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.196+0000 7fa487fff640 1 -- 172.21.15.115:0/35012016 >> 172.21.15.115:0/35012016 conn(0x7fa48806a7b0 msgr2=0x7fa48806c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:40.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.197+0000 7fa487fff640 1 -- 172.21.15.115:0/35012016 shutdown_connections 2023-12-01T01:51:40.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.197+0000 7fa487fff640 1 -- 172.21.15.115:0/35012016 wait complete. 2023-12-01T01:51:40.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.198+0000 7fa487fff640 1 Processor -- start 2023-12-01T01:51:40.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.199+0000 7fa487fff640 1 -- start start 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa487fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa48806deb0 0x7fa48813dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa487fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880b30e0 0x7fa48814bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa487fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4881597e0 0x7fa48814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa487fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa488147850 con 0x7fa4881597e0 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa487fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48813e870 con 0x7fa48806deb0 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa487fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa488147a60 con 0x7fa4880b30e0 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa4877fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4881597e0 0x7fa48814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa4877fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4881597e0 0x7fa48814c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50384/0 (socket says 172.21.15.115:50384) 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa4877fe640 1 -- 172.21.15.115:0/2393534987 learned_addr learned my addr 172.21.15.115:0/2393534987 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa4867fc640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880b30e0 0x7fa48814bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa486ffd640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa48806deb0 0x7fa48813dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa4877fe640 1 -- 172.21.15.115:0/2393534987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880b30e0 msgr2=0x7fa48814bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa4877fe640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880b30e0 0x7fa48814bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa4877fe640 1 -- 172.21.15.115:0/2393534987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa48806deb0 msgr2=0x7fa48813dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa4877fe640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa48806deb0 0x7fa48813dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa4877fe640 1 -- 172.21.15.115:0/2393534987 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0094d0 con 0x7fa4881597e0 2023-12-01T01:51:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.200+0000 7fa4867fc640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880b30e0 0x7fa48814bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.201+0000 7fa486ffd640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa48806deb0 0x7fa48813dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:40.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.201+0000 7fa4877fe640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4881597e0 0x7fa48814c060 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c7a0 tx=0x7fa48000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.201+0000 7fa46f7fe640 1 -- 172.21.15.115:0/2393534987 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48000a640 con 0x7fa4881597e0 2023-12-01T01:51:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.201+0000 7fa46f7fe640 1 -- 172.21.15.115:0/2393534987 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48000a7a0 con 0x7fa4881597e0 2023-12-01T01:51:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.201+0000 7fa487fff640 1 -- 172.21.15.115:0/2393534987 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa488147d40 con 0x7fa4881597e0 2023-12-01T01:51:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.201+0000 7fa46f7fe640 1 -- 172.21.15.115:0/2393534987 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48000abd0 con 0x7fa4881597e0 2023-12-01T01:51:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.201+0000 7fa487fff640 1 -- 172.21.15.115:0/2393534987 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4881480d0 con 0x7fa4881597e0 2023-12-01T01:51:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.203+0000 7fa487fff640 1 -- 172.21.15.115:0/2393534987 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4880618d0 con 0x7fa4881597e0 2023-12-01T01:51:40.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.203+0000 7fa46f7fe640 1 -- 172.21.15.115:0/2393534987 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 33) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fa48000ad30 con 0x7fa4881597e0 2023-12-01T01:51:40.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.204+0000 7fa46f7fe640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa47406a5a0 0x7fa47406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.204+0000 7fa46f7fe640 1 -- 172.21.15.115:0/2393534987 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4800882f0 con 0x7fa4881597e0 2023-12-01T01:51:40.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.204+0000 7fa486ffd640 1 -- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa47406a5a0 msgr2=0x7fa47406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:40.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.204+0000 7fa486ffd640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa47406a5a0 0x7fa47406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.209+0000 7fa46f7fe640 1 -- 172.21.15.115:0/2393534987 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa4800537c0 con 0x7fa4881597e0 2023-12-01T01:51:40.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.405+0000 7fa486ffd640 1 -- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa47406a5a0 msgr2=0x7fa47406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:40.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.405+0000 7fa486ffd640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa47406a5a0 0x7fa47406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:51:40.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.411+0000 7fa487fff640 1 -- 172.21.15.115:0/2393534987 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fa4880b3d10 con 0x7fa4881597e0 2023-12-01T01:51:40.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.417+0000 7fa46f7fe640 1 -- 172.21.15.115:0/2393534987 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa480056de0 con 0x7fa4881597e0 2023-12-01T01:51:40.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.419+0000 7fa487fff640 1 -- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa47406a5a0 msgr2=0x7fa47406ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.419+0000 7fa487fff640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa47406a5a0 0x7fa47406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.419+0000 7fa487fff640 1 -- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4881597e0 msgr2=0x7fa48814c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.419+0000 7fa487fff640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4881597e0 0x7fa48814c060 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c7a0 tx=0x7fa48000cc60 comp rx=0 tx=0).stop 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.420+0000 7fa487fff640 1 -- 172.21.15.115:0/2393534987 shutdown_connections 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.420+0000 7fa487fff640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa47406a5a0 0x7fa47406ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.420+0000 7fa487fff640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4880b30e0 0x7fa48814bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.420+0000 7fa487fff640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4881597e0 0x7fa48814c060 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.420+0000 7fa487fff640 1 --2- 172.21.15.115:0/2393534987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa48806deb0 0x7fa48813dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.420+0000 7fa487fff640 1 -- 172.21.15.115:0/2393534987 >> 172.21.15.115:0/2393534987 conn(0x7fa48806a7b0 msgr2=0x7fa48814e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.420+0000 7fa487fff640 1 -- 172.21.15.115:0/2393534987 shutdown_connections 2023-12-01T01:51:40.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.420+0000 7fa487fff640 1 -- 172.21.15.115:0/2393534987 wait complete. 2023-12-01T01:51:40.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:40.438 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-01T01:51:40.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-12-01T01:51:40.539 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:40.537+0000 7eff4c23f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:51:40.594 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:40.592+0000 7eff4c23f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:51:40.621 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:40.620+0000 7f9d6c2da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:51:40.644 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:40.642+0000 7eff4c23f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:51:40.675 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:40.675+0000 7f9d6c2da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:51:40.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.688+0000 7f35d4a2b640 1 Processor -- start 2023-12-01T01:51:40.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.689+0000 7f35d4a2b640 1 -- start start 2023-12-01T01:51:40.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.690+0000 7f35d4a2b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d015cd80 0x7f35d01612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.690+0000 7f35d4a2b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f35d0069cc0 0x7f35d006a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.690+0000 7f35d4a2b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d00d0f20 con 0x7f35d00b8400 2023-12-01T01:51:40.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.690+0000 7f35d4a2b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d0161820 con 0x7f35d0069cc0 2023-12-01T01:51:40.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.690+0000 7f35d4a2b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d015e170 con 0x7f35d015cd80 2023-12-01T01:51:40.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.690+0000 7f35c7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f35d00b8400 0x7f35d00d1f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52510/0 (socket says 172.21.15.115:52510) 2023-12-01T01:51:40.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.690+0000 7f35c7fff640 1 -- 172.21.15.115:0/3054702211 learned_addr learned my addr 172.21.15.115:0/3054702211 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:40.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35cf59e640 1 --2- 172.21.15.115:0/3054702211 >> v2:172.21.15.142:3300/0 conn(0x7f35d0069cc0 0x7f35d006a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ced9d640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d015cd80 0x7f35d01612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046384085 0 0) 0x7f35d00d0f20 con 0x7f35d00b8400 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35bc003520 con 0x7f35d00b8400 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 738870571 0 0) 0x7f35bc003520 con 0x7f35d00b8400 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d015cd80 msgr2=0x7f35d01612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d015cd80 0x7f35d01612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 >> v2:172.21.15.142:3300/0 conn(0x7f35d0069cc0 msgr2=0x7f35d006a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 --2- 172.21.15.115:0/3054702211 >> v2:172.21.15.142:3300/0 conn(0x7f35d0069cc0 0x7f35d006a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00d1170 con 0x7f35d00b8400 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f35b8001670 con 0x7f35d00b8400 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 >> v1:172.21.15.115:6789/0 conn(0x7f35d00b8400 legacy=0x7f35d00d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35bc0039a0 0x7f35bc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d015cd80 0x7f35bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35bc004d00 0x7f35bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d015e170 con 0x7f35bc004d00 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc00b980 con 0x7f35bc0039a0 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.691+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc00bac0 con 0x7f35d015cd80 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.692+0000 7f35cf59e640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35bc004d00 0x7f35bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.692+0000 7f35c7fff640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d015cd80 0x7f35bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.692+0000 7f35ced9d640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35bc0039a0 0x7f35bc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.692+0000 7f35cf59e640 1 -- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d015cd80 msgr2=0x7f35bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.692+0000 7f35cf59e640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d015cd80 0x7f35bc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.692+0000 7f35cf59e640 1 -- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35bc0039a0 msgr2=0x7f35bc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.692+0000 7f35cf59e640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35bc0039a0 0x7f35bc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.692+0000 7f35cf59e640 1 -- 172.21.15.115:0/3054702211 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc003520 con 0x7f35bc004d00 2023-12-01T01:51:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.693+0000 7f35cf59e640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35bc004d00 0x7f35bc005180 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f35c800e170 tx=0x7f35bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ac674ac8b34da4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.694+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c802cc20 con 0x7f35bc004d00 2023-12-01T01:51:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.694+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c800f040 con 0x7f35bc004d00 2023-12-01T01:51:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.694+0000 7f35ce59c640 1 -- 172.21.15.115:0/3054702211 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8034440 con 0x7f35bc004d00 2023-12-01T01:51:40.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.694+0000 7f35d4a2b640 1 -- 172.21.15.115:0/3054702211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35bc004d00 msgr2=0x7f35bc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:40.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.694+0000 7f35d4a2b640 1 --2- 172.21.15.115:0/3054702211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35bc004d00 0x7f35bc005180 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f35c800e170 tx=0x7f35bc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.694+0000 7f35cf59e640 1 -- 172.21.15.115:0/3054702211 reap_dead start 2023-12-01T01:51:40.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.694+0000 7f35d4a2b640 1 -- 172.21.15.115:0/3054702211 shutdown_connections 2023-12-01T01:51:40.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.694+0000 7f35d4a2b640 1 -- 172.21.15.115:0/3054702211 >> 172.21.15.115:0/3054702211 conn(0x7f35d00d2a70 msgr2=0x7f35d00d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:40.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.694+0000 7f35d4a2b640 1 -- 172.21.15.115:0/3054702211 shutdown_connections 2023-12-01T01:51:40.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.694+0000 7f35d4a2b640 1 -- 172.21.15.115:0/3054702211 wait complete. 2023-12-01T01:51:40.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.697+0000 7f35d4a2b640 1 Processor -- start 2023-12-01T01:51:40.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.697+0000 7f35d4a2b640 1 -- start start 2023-12-01T01:51:40.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.699+0000 7f35d4a2b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35bc004d00 0x7f35d014bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.699+0000 7f35d4a2b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d00cc6b0 0x7f35d00c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.699+0000 7f35d4a2b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35d00b2450 0x7f35d00b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.699+0000 7f35d4a2b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00b2e40 con 0x7f35d00b2450 2023-12-01T01:51:40.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.699+0000 7f35d4a2b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00ca860 con 0x7f35bc004d00 2023-12-01T01:51:40.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.699+0000 7f35d4a2b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d014c3e0 con 0x7f35d00cc6b0 2023-12-01T01:51:40.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.699+0000 7f35cf59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35d00b2450 0x7f35d00b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.699+0000 7f35cf59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35d00b2450 0x7f35d00b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50400/0 (socket says 172.21.15.115:50400) 2023-12-01T01:51:40.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.699+0000 7f35cf59e640 1 -- 172.21.15.115:0/3906065402 learned_addr learned my addr 172.21.15.115:0/3906065402 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:40.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35ced9d640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35bc004d00 0x7f35d014bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35c7fff640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d00cc6b0 0x7f35d00c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:40.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35cf59e640 1 -- 172.21.15.115:0/3906065402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d00cc6b0 msgr2=0x7f35d00c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:40.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35cf59e640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35d00cc6b0 0x7f35d00c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35cf59e640 1 -- 172.21.15.115:0/3906065402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35bc004d00 msgr2=0x7f35d014bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:40.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35cf59e640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35bc004d00 0x7f35d014bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:40.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35cf59e640 1 -- 172.21.15.115:0/3906065402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00c9c50 con 0x7f35d00b2450 2023-12-01T01:51:40.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35cf59e640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35d00b2450 0x7f35d00b2900 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f35c8037040 tx=0x7f35c8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:40.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8034d50 con 0x7f35d00b2450 2023-12-01T01:51:40.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c8003990 con 0x7f35d00b2450 2023-12-01T01:51:40.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35d4a2b640 1 -- 172.21.15.115:0/3906065402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d00c9f30 con 0x7f35d00b2450 2023-12-01T01:51:40.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.700+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8003d80 con 0x7f35d00b2450 2023-12-01T01:51:40.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.701+0000 7f35d4a2b640 1 -- 172.21.15.115:0/3906065402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d00ca4f0 con 0x7f35d00b2450 2023-12-01T01:51:40.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.702+0000 7f35d4a2b640 1 -- 172.21.15.115:0/3906065402 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d00caa10 con 0x7f35d00b2450 2023-12-01T01:51:40.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.703+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 33) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f35c8004030 con 0x7f35d00b2450 2023-12-01T01:51:40.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.704+0000 7f35c77fe640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 0x7f35b006cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:40.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.704+0000 7f35ced9d640 1 -- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 msgr2=0x7f35b006cae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:40.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.704+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35c80aa510 con 0x7f35d00b2450 2023-12-01T01:51:40.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.704+0000 7f35ced9d640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 0x7f35b006cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:40.705 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:40.703+0000 7eff4c23f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:51:40.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.708+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f35d00caa10 con 0x7f35d00b2450 2023-12-01T01:51:40.732 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:40.731+0000 7f9d6c2da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:51:40.754 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:40.752+0000 7eff4c23f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:51:40.794 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:40.794+0000 7f9d6c2da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:51:40.845 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:40.845+0000 7f9d6c2da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:51:40.864 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:40.862+0000 7eff4c23f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:51:40.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.905+0000 7f35ced9d640 1 -- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 msgr2=0x7f35b006cae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:40.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.905+0000 7f35ced9d640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 0x7f35b006cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:51:40.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:40.922+0000 7f35d4a2b640 1 -- 172.21.15.115:0/3906065402 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f35d00d1730 con 0x7f35b006a620 2023-12-01T01:51:40.926 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:40.924+0000 7eff4c23f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:40.958 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:40.958+0000 7f9d6c2da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:51:40.975 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:40.973+0000 7eff4c23f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:51:41.021 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:41.020+0000 7f9d6c2da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:41.033 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:41.031+0000 7eff4c23f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:51:41.070 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:41.069+0000 7f9d6c2da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:51:41.082 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:41.081+0000 7eff4c23f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:51:41.126 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:41.125+0000 7f9d6c2da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:51:41.175 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:41.175+0000 7f9d6c2da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:51:41.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:41.306+0000 7f35ced9d640 1 -- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 msgr2=0x7f35b006cae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:41.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:41.306+0000 7f35ced9d640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 0x7f35b006cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:51:41.459 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:41.458+0000 7eff4c23f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:51:41.553 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:41.553+0000 7f9d6c2da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:51:41.616 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:41.615+0000 7eff4c23f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:51:41.712 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:41.712+0000 7f9d6c2da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:51:41.725 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:41.723+0000 7eff4c23f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:51:41.775 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:41.773+0000 7eff4c23f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:51:41.820 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:41.819+0000 7f9d6c2da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:51:41.870 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:41.870+0000 7f9d6c2da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:51:41.883 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:41.881+0000 7eff4c23f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:41.976 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:41.975+0000 7f9d6c2da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:41.990 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:41.989+0000 7eff4c23f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:51:42.080 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:42.080+0000 7f9d6c2da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:51:42.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:42.107+0000 7f35ced9d640 1 -- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 msgr2=0x7f35b006cae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:42.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:42.107+0000 7f35ced9d640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 0x7f35b006cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:51:42.148 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:42.146+0000 7eff4c23f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:42.197 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:42.195+0000 7eff4c23f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:42.237 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:42.236+0000 7f9d6c2da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:42.285 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:42.285+0000 7f9d6c2da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:42.315 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:42.313+0000 7eff4c23f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:51:42.402 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:42.400+0000 7f9d6c2da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:51:43.144 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:43.142+0000 7eff4c23f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:51:43.209 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.207+0000 7f9d6c2da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:51:43.438 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:43.436+0000 7eff4c23f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:51:43.495 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:43.494+0000 7eff4c23f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:51:43.499 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.498+0000 7f9d6c2da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:51:43.556 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.554+0000 7f9d6c2da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:51:43.608 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:43.606+0000 7eff4c23f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:51:43.669 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.668+0000 7f9d6c2da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:51:43.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:43.677+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 34) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f35c8075f50 con 0x7f35d00b2450 2023-12-01T01:51:43.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:43.705+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 35) v1 ==== 88335+0+0 (secure 0 0 0) 0x7f35c8090470 con 0x7f35d00b2450 2023-12-01T01:51:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:43.706+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 msgr2=0x7f35b006cae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:43.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:43.706+0000 7f35c77fe640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 0x7f35b006cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:43.883 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.882+0000 7f9ce95bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:43.883 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.882+0000 7f9ce95bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:43.883 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.882+0000 7f9ce95bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:43.883 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.882+0000 7f9ce95bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:43.883 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.882+0000 7f9ce95bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:43.884 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.883+0000 7f9cec5c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:43.884 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.883+0000 7f9cec5c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:43.884 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.883+0000 7f9cec5c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:43.884 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.883+0000 7f9cec5c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:43.884 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:43.883+0000 7f9cec5c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:44.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:44.720+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 <== mon.0 v2:172.21.15.115:3300/0 9 ==== mgrmap(e 36) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f35c8076820 con 0x7f35d00b2450 2023-12-01T01:51:44.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:44.721+0000 7f35c77fe640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 0x7f35b006cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:44.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:44.721+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f35b0034d70 con 0x7f35b006a620 2023-12-01T01:51:44.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:44.726+0000 7f35ced9d640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 0x7f35b006cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:44.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:44.737+0000 7f35ced9d640 1 --2- 172.21.15.115:0/3906065402 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f35b006a620 0x7f35b006cae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009160 tx=0x7f35c0008040 comp rx=0 tx=0).ready entity=mgr.6648 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:44.757 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:44.756+0000 7f9cf3610640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-01T01:51:44.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:44.756+0000 7f35c77fe640 1 -- 172.21.15.115:0/3906065402 <== mgr.6648 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f35b0034d70 con 0x7f35b006a620 2023-12-01T01:51:44.758 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-01T01:51:44.779 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-01T01:51:44.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:44.780 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-01T01:51:44.832 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T01:51:45.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.038+0000 7f2fcb3c9640 1 Processor -- start 2023-12-01T01:51:45.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.038+0000 7f2fcb3c9640 1 -- start start 2023-12-01T01:51:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.039+0000 7f2fcb3c9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2fc4142870 0x7f2fc40ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.039+0000 7f2fcb3c9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc40b2f30 con 0x7f2fc40cd010 2023-12-01T01:51:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.039+0000 7f2fcb3c9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc40b2320 con 0x7f2fc40b1e20 2023-12-01T01:51:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.039+0000 7f2fcb3c9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc40b3ae0 con 0x7f2fc4142870 2023-12-01T01:51:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.040+0000 7f2fc9165640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2fc4142870 0x7f2fc40ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.040+0000 7f2fc8964640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2fc40cd010 0x7f2fc40b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52520/0 (socket says 172.21.15.115:52520) 2023-12-01T01:51:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.040+0000 7f2fc8964640 1 -- 172.21.15.115:0/2052248320 learned_addr learned my addr 172.21.15.115:0/2052248320 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:45.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.040+0000 7f2fc9165640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2fc4142870 0x7f2fc40ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50526/0 (socket says 172.21.15.115:50526) 2023-12-01T01:51:45.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.040+0000 7f2fc9165640 1 -- 172.21.15.115:0/2052248320 >> v1:172.21.15.142:6789/0 conn(0x7f2fc40b1e20 legacy=0x7f2fc415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:45.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.040+0000 7f2fc9165640 1 -- 172.21.15.115:0/2052248320 >> v1:172.21.15.115:6789/0 conn(0x7f2fc40cd010 legacy=0x7f2fc40b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:45.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.040+0000 7f2fc9165640 1 -- 172.21.15.115:0/2052248320 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc40b2600 con 0x7f2fc4142870 2023-12-01T01:51:45.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.040+0000 7f2fbbfff640 1 -- 172.21.15.115:0/2052248320 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239455951 0 0) 0x7f2fc40b2f30 con 0x7f2fc40cd010 2023-12-01T01:51:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.041+0000 7f2fc9165640 1 --2- 172.21.15.115:0/2052248320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc4142870 0x7f2fc40ccad0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009d90 tx=0x7f2fc002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb4fecc08b6599 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.042+0000 7f2fbbfff640 1 -- 172.21.15.115:0/2052248320 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc002cc60 con 0x7f2fc4142870 2023-12-01T01:51:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.042+0000 7f2fbbfff640 1 -- 172.21.15.115:0/2052248320 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc002cdc0 con 0x7f2fc4142870 2023-12-01T01:51:45.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.042+0000 7f2fbbfff640 1 -- 172.21.15.115:0/2052248320 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0034660 con 0x7f2fc4142870 2023-12-01T01:51:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.042+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/2052248320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc4142870 msgr2=0x7f2fc40ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.042+0000 7f2fcb3c9640 1 --2- 172.21.15.115:0/2052248320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc4142870 0x7f2fc40ccad0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009d90 tx=0x7f2fc002beb0 comp rx=0 tx=0).stop 2023-12-01T01:51:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.042+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/2052248320 shutdown_connections 2023-12-01T01:51:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.042+0000 7f2fcb3c9640 1 --2- 172.21.15.115:0/2052248320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc4142870 0x7f2fc40ccad0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.042+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/2052248320 >> 172.21.15.115:0/2052248320 conn(0x7f2fc4065960 msgr2=0x7f2fc4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.043+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/2052248320 shutdown_connections 2023-12-01T01:51:45.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.043+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/2052248320 wait complete. 2023-12-01T01:51:45.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.044+0000 7f2fcb3c9640 1 Processor -- start 2023-12-01T01:51:45.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.044+0000 7f2fcb3c9640 1 -- start start 2023-12-01T01:51:45.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.045+0000 7f2fcb3c9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc40b1e20 0x7f2fc4155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:45.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.045+0000 7f2fcb3c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc40cd010 0x7f2fc415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.045+0000 7f2fcb3c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc4142870 0x7f2fc413f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.045+0000 7f2fcb3c9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc415e660 con 0x7f2fc40cd010 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.045+0000 7f2fcb3c9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc4155a80 con 0x7f2fc40b1e20 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.045+0000 7f2fcb3c9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc415dba0 con 0x7f2fc4142870 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.046+0000 7f2fc8964640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc40cd010 0x7f2fc415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.046+0000 7f2fc8964640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc40cd010 0x7f2fc415c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50530/0 (socket says 172.21.15.115:50530) 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.046+0000 7f2fc8964640 1 -- 172.21.15.115:0/3749843847 learned_addr learned my addr 172.21.15.115:0/3749843847 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.046+0000 7f2fc8964640 1 -- 172.21.15.115:0/3749843847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc4142870 msgr2=0x7f2fc413f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.046+0000 7f2fc8964640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc4142870 0x7f2fc413f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.046+0000 7f2fc9165640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc40b1e20 0x7f2fc4155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:45.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.046+0000 7f2fc8964640 1 -- 172.21.15.115:0/3749843847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc40b1e20 msgr2=0x7f2fc4155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:45.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.046+0000 7f2fc8964640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc40b1e20 0x7f2fc4155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:45.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.046+0000 7f2fc8964640 1 -- 172.21.15.115:0/3749843847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc0009a70 con 0x7f2fc40cd010 2023-12-01T01:51:45.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.046+0000 7f2fc8964640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc40cd010 0x7f2fc415c7f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c500 tx=0x7f2fb400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:45.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.047+0000 7f2fb9ffb640 1 -- 172.21.15.115:0/3749843847 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4007560 con 0x7f2fc40cd010 2023-12-01T01:51:45.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.047+0000 7f2fb9ffb640 1 -- 172.21.15.115:0/3749843847 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb40047e0 con 0x7f2fc40cd010 2023-12-01T01:51:45.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.047+0000 7f2fb9ffb640 1 -- 172.21.15.115:0/3749843847 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4004b90 con 0x7f2fc40cd010 2023-12-01T01:51:45.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.047+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/3749843847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc415d020 con 0x7f2fc40cd010 2023-12-01T01:51:45.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.047+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/3749843847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc415d5e0 con 0x7f2fc40cd010 2023-12-01T01:51:45.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.048+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/3749843847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc413b190 con 0x7f2fc40cd010 2023-12-01T01:51:45.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.049+0000 7f2fb9ffb640 1 -- 172.21.15.115:0/3749843847 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f2fb4004cf0 con 0x7f2fc40cd010 2023-12-01T01:51:45.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.050+0000 7f2fb9ffb640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fa806a300 0x7f2fa806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:45.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.050+0000 7f2fb9ffb640 1 -- 172.21.15.115:0/3749843847 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f2fb4087910 con 0x7f2fc40cd010 2023-12-01T01:51:45.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.050+0000 7f2fc9165640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fa806a300 0x7f2fa806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:45.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.051+0000 7f2fc9165640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fa806a300 0x7f2fa806c7c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0036000 tx=0x7f2fc0030000 comp rx=0 tx=0).ready entity=mgr.6648 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:45.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.055+0000 7f2fb9ffb640 1 -- 172.21.15.115:0/3749843847 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2fc413b190 con 0x7f2fc40cd010 2023-12-01T01:51:45.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.296+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/3749843847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f2fc413b190 con 0x7f2fc40cd010 2023-12-01T01:51:45.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.733+0000 7f2fb9ffb640 1 -- 172.21.15.115:0/3749843847 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 37) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f2fb40533e0 con 0x7f2fc40cd010 2023-12-01T01:51:45.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.735+0000 7f2fb9ffb640 1 -- 172.21.15.115:0/3749843847 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v37) v1 ==== 93+0+0 (secure 0 0 0) 0x7f2fb4050900 con 0x7f2fc40cd010 2023-12-01T01:51:45.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/3749843847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fa806a300 msgr2=0x7f2fa806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:45.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fa806a300 0x7f2fa806c7c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0036000 tx=0x7f2fc0030000 comp rx=0 tx=0).stop 2023-12-01T01:51:45.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/3749843847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc40cd010 msgr2=0x7f2fc415c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:45.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc40cd010 0x7f2fc415c7f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400c500 tx=0x7f2fb400c9c0 comp rx=0 tx=0).stop 2023-12-01T01:51:45.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/3749843847 shutdown_connections 2023-12-01T01:51:45.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2fa806a300 0x7f2fa806c7c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:45.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fc4142870 0x7f2fc413f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:45.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2fc40cd010 0x7f2fc415c7f0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:45.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 --2- 172.21.15.115:0/3749843847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fc40b1e20 0x7f2fc4155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:45.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/3749843847 >> 172.21.15.115:0/3749843847 conn(0x7f2fc4065960 msgr2=0x7f2fc406bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:45.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/3749843847 shutdown_connections 2023-12-01T01:51:45.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:45.738+0000 7f2fcb3c9640 1 -- 172.21.15.115:0/3749843847 wait complete. 2023-12-01T01:51:45.759 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:51:45.759 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:51:45.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:51:45.975 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:51:45.975 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:51:45.975 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:46.029 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:51:46.029 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:46.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:46.265 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:46.264+0000 7fcd4c031040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:51:46.268 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:46.266+0000 7f8cdb220040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:51:46.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.314+0000 7efd0359e640 1 Processor -- start 2023-12-01T01:51:46.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.314+0000 7efd0359e640 1 -- start start 2023-12-01T01:51:46.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.316+0000 7efd0359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd0406c9d0 0x7efd04071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.316+0000 7efd0359e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd04070720 con 0x7efd041596d0 2023-12-01T01:51:46.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.316+0000 7efd0359e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd040cd690 con 0x7efd040cd200 2023-12-01T01:51:46.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.316+0000 7efd0359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd04071cc0 con 0x7efd0406c9d0 2023-12-01T01:51:46.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.316+0000 7efd0259c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efd041596d0 0x7efd04070010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52544/0 (socket says 172.21.15.115:52544) 2023-12-01T01:51:46.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.316+0000 7efd0259c640 1 -- 172.21.15.115:0/1834706169 learned_addr learned my addr 172.21.15.115:0/1834706169 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd02d9d640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd0406c9d0 0x7efd04071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532913372 0 0) 0x7efd04070720 con 0x7efd041596d0 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf0003520 con 0x7efd041596d0 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2167943928 0 0) 0x7efcf0003520 con 0x7efd041596d0 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd0406c9d0 msgr2=0x7efd04071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd0406c9d0 0x7efd04071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 >> v1:172.21.15.142:6789/0 conn(0x7efd040cd200 legacy=0x7efd0406c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd040ce1a0 con 0x7efd041596d0 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7efcf4001670 con 0x7efd041596d0 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 >> v1:172.21.15.115:6789/0 conn(0x7efd041596d0 legacy=0x7efd04070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 0x7efcf0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcf00047c0 0x7efcf0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd0406c9d0 0x7efcf0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd04071cc0 con 0x7efcf0003a10 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf000ba80 con 0x7efcf00047c0 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf000bc40 con 0x7efd0406c9d0 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.317+0000 7efd0259c640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 0x7efcf0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.318+0000 7efd02d9d640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd0406c9d0 0x7efcf0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.318+0000 7efcfbfff640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcf00047c0 0x7efcf0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.318+0000 7efd0259c640 1 -- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd0406c9d0 msgr2=0x7efcf0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.318+0000 7efd0259c640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd0406c9d0 0x7efcf0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.318+0000 7efd0259c640 1 -- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcf00047c0 msgr2=0x7efcf0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.318+0000 7efd0259c640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcf00047c0 0x7efcf0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.318+0000 7efd0259c640 1 -- 172.21.15.115:0/1834706169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf0003520 con 0x7efcf0003a10 2023-12-01T01:51:46.322 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:46.320+0000 7f8cdb220040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:51:46.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.320+0000 7efd0259c640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 0x7efcf0003e30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7efcf4007ca0 tx=0x7efcf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea0668eaedca9a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:46.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.320+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf402ece0 con 0x7efcf0003a10 2023-12-01T01:51:46.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.320+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf402ee40 con 0x7efcf0003a10 2023-12-01T01:51:46.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.320+0000 7efd01d9b640 1 -- 172.21.15.115:0/1834706169 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4005b10 con 0x7efcf0003a10 2023-12-01T01:51:46.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.321+0000 7efd0359e640 1 -- 172.21.15.115:0/1834706169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 msgr2=0x7efcf0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.321+0000 7efd0359e640 1 --2- 172.21.15.115:0/1834706169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 0x7efcf0003e30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7efcf4007ca0 tx=0x7efcf00057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.321+0000 7efd02d9d640 1 -- 172.21.15.115:0/1834706169 reap_dead start 2023-12-01T01:51:46.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.321+0000 7efd0359e640 1 -- 172.21.15.115:0/1834706169 shutdown_connections 2023-12-01T01:51:46.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.321+0000 7efd0359e640 1 -- 172.21.15.115:0/1834706169 >> 172.21.15.115:0/1834706169 conn(0x7efd0406a0e0 msgr2=0x7efd0406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:46.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.321+0000 7efd0359e640 1 -- 172.21.15.115:0/1834706169 shutdown_connections 2023-12-01T01:51:46.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.321+0000 7efd0359e640 1 -- 172.21.15.115:0/1834706169 wait complete. 2023-12-01T01:51:46.324 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:46.322+0000 7fcd4c031040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:51:46.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.323+0000 7efd0359e640 1 Processor -- start 2023-12-01T01:51:46.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.323+0000 7efd0359e640 1 -- start start 2023-12-01T01:51:46.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 0x7efd0414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd040c74d0 0x7efd040c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd04157540 0x7efd041579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0359e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd04157f00 con 0x7efcf0003a10 2023-12-01T01:51:46.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c8b80 con 0x7efd04157540 2023-12-01T01:51:46.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c8e60 con 0x7efd040c74d0 2023-12-01T01:51:46.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 0x7efd0414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 0x7efd0414db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50556/0 (socket says 172.21.15.115:50556) 2023-12-01T01:51:46.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0259c640 1 -- 172.21.15.115:0/964755825 learned_addr learned my addr 172.21.15.115:0/964755825 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efcfbfff640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd040c74d0 0x7efd040c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd02d9d640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd04157540 0x7efd041579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0259c640 1 -- 172.21.15.115:0/964755825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd040c74d0 msgr2=0x7efd040c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0259c640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd040c74d0 0x7efd040c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0259c640 1 -- 172.21.15.115:0/964755825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd04157540 msgr2=0x7efd041579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0259c640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd04157540 0x7efd041579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efd0259c640 1 -- 172.21.15.115:0/964755825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd040c9140 con 0x7efcf0003a10 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.324+0000 7efcfbfff640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd040c74d0 0x7efd040c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.325+0000 7efd0259c640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 0x7efd0414db10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7efcf400c330 tx=0x7efcf4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.325+0000 7efcfaffd640 1 -- 172.21.15.115:0/964755825 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4007270 con 0x7efcf0003a10 2023-12-01T01:51:46.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.325+0000 7efcfaffd640 1 -- 172.21.15.115:0/964755825 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf40073d0 con 0x7efcf0003a10 2023-12-01T01:51:46.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.325+0000 7efd0359e640 1 -- 172.21.15.115:0/964755825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd040c9420 con 0x7efcf0003a10 2023-12-01T01:51:46.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.325+0000 7efcfaffd640 1 -- 172.21.15.115:0/964755825 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4007780 con 0x7efcf0003a10 2023-12-01T01:51:46.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.325+0000 7efd0359e640 1 -- 172.21.15.115:0/964755825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd040c8830 con 0x7efcf0003a10 2023-12-01T01:51:46.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.326+0000 7efd0359e640 1 -- 172.21.15.115:0/964755825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd04152ec0 con 0x7efcf0003a10 2023-12-01T01:51:46.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.327+0000 7efcfaffd640 1 -- 172.21.15.115:0/964755825 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 88710+0+0 (secure 0 0 0) 0x7efcf4034070 con 0x7efcf0003a10 2023-12-01T01:51:46.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.328+0000 7efcfaffd640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efce406a550 0x7efce406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.329+0000 7efcfbfff640 1 -- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efce406a550 msgr2=0x7efce406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:46.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.329+0000 7efcfaffd640 1 -- 172.21.15.115:0/964755825 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6053+0+0 (secure 0 0 0) 0x7efcf403e070 con 0x7efcf0003a10 2023-12-01T01:51:46.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.329+0000 7efcfbfff640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efce406a550 0x7efce406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:46.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.333+0000 7efcfaffd640 1 -- 172.21.15.115:0/964755825 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efcf4075a90 con 0x7efcf0003a10 2023-12-01T01:51:46.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.524+0000 7efd0359e640 1 -- 172.21.15.115:0/964755825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd040673a0 con 0x7efcf0003a10 2023-12-01T01:51:46.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.526+0000 7efcfaffd640 1 -- 172.21.15.115:0/964755825 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16641 (secure 0 0 0) 0x7efcf40790b0 con 0x7efcf0003a10 2023-12-01T01:51:46.528 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:46.528 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":84,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:43.682749+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:51:22.633453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-01T01:51:23.191899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:51:43.682671+0000","172.21.15.115:6822/88317":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/3840278706":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/3824153221":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/86374784":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/871551054":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/3669975107":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/2326628113":"2023-12-02T01:51:43.682671+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 -- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efce406a550 msgr2=0x7efce406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efcfbfff640 1 -- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efce406a550 msgr2=0x7efce406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efcfbfff640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efce406a550 0x7efce406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efce406a550 0x7efce406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 -- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 msgr2=0x7efd0414db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 0x7efd0414db10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7efcf400c330 tx=0x7efcf4001a00 comp rx=0 tx=0).stop 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 -- 172.21.15.115:0/964755825 shutdown_connections 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efce406a550 0x7efce406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd040c74d0 0x7efd040c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd04157540 0x7efd041579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 --2- 172.21.15.115:0/964755825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcf0003a10 0x7efd0414db10 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 -- 172.21.15.115:0/964755825 >> 172.21.15.115:0/964755825 conn(0x7efd0406a0e0 msgr2=0x7efd040c52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:46.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 -- 172.21.15.115:0/964755825 shutdown_connections 2023-12-01T01:51:46.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.530+0000 7efd0359e640 1 -- 172.21.15.115:0/964755825 wait complete. 2023-12-01T01:51:46.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:51:22.633453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:51:23.191899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:51:46.551 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:51:46.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.749+0000 7f19844db640 1 Processor -- start 2023-12-01T01:51:46.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.749+0000 7f19844db640 1 -- start start 2023-12-01T01:51:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.750+0000 7f19844db640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cfd60 0x7f197c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.751+0000 7f19844db640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c0ce5e0 con 0x7f197c15cd90 2023-12-01T01:51:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.751+0000 7f19844db640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c0ce300 con 0x7f197c0b5fa0 2023-12-01T01:51:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.751+0000 7f19844db640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0d0ec0 con 0x7f197c0cfd60 2023-12-01T01:51:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.751+0000 7f1982277640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f197c15cd90 0x7f197c0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52560/0 (socket says 172.21.15.115:52560) 2023-12-01T01:51:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.751+0000 7f1982277640 1 -- 172.21.15.115:0/3323426380 learned_addr learned my addr 172.21.15.115:0/3323426380 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.751+0000 7f1981a76640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cfd60 0x7f197c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.751+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896601714 0 0) 0x7f1978001110 con 0x7f197c15cd90 2023-12-01T01:51:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.751+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1968003520 con 0x7f197c15cd90 2023-12-01T01:51:46.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4038077541 0 0) 0x7f1968003520 con 0x7f197c15cd90 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cfd60 msgr2=0x7f197c0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cfd60 0x7f197c0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 >> v1:172.21.15.142:6789/0 conn(0x7f197c0b5fa0 legacy=0x7f197c0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0d10a0 con 0x7f197c15cd90 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f1978001800 con 0x7f197c15cd90 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 >> v1:172.21.15.115:6789/0 conn(0x7f197c15cd90 legacy=0x7f197c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cfd60 0x7f19680039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968003f70 0x7f19680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1968004d00 0x7f1968005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0d0ec0 con 0x7f1968004d00 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196800bb70 con 0x7f1968003f70 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196800bd30 con 0x7f197c0cfd60 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1982a78640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1968004d00 0x7f1968005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1982277640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cfd60 0x7f19680039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1981a76640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968003f70 0x7f19680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1982a78640 1 -- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cfd60 msgr2=0x7f19680039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:46.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1982a78640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cfd60 0x7f19680039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1982a78640 1 -- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968003f70 msgr2=0x7f19680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:46.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1982a78640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968003f70 0x7f19680047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.752+0000 7f1982a78640 1 -- 172.21.15.115:0/3323426380 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968003520 con 0x7f1968004d00 2023-12-01T01:51:46.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.753+0000 7f1982a78640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1968004d00 0x7f1968005180 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1974001e50 tx=0x7f19680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9679e773e0afeb2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.754+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974003b70 con 0x7f1968004d00 2023-12-01T01:51:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.754+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1974003cd0 con 0x7f1968004d00 2023-12-01T01:51:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.754+0000 7f1981275640 1 -- 172.21.15.115:0/3323426380 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974005310 con 0x7f1968004d00 2023-12-01T01:51:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.754+0000 7f19844db640 1 -- 172.21.15.115:0/3323426380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1968004d00 msgr2=0x7f1968005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.754+0000 7f19844db640 1 --2- 172.21.15.115:0/3323426380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1968004d00 0x7f1968005180 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1974001e50 tx=0x7f19680057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.754+0000 7f1982a78640 1 -- 172.21.15.115:0/3323426380 reap_dead start 2023-12-01T01:51:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.754+0000 7f19844db640 1 -- 172.21.15.115:0/3323426380 shutdown_connections 2023-12-01T01:51:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.754+0000 7f19844db640 1 -- 172.21.15.115:0/3323426380 >> 172.21.15.115:0/3323426380 conn(0x7f197c065960 msgr2=0x7f197c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.755+0000 7f19844db640 1 -- 172.21.15.115:0/3323426380 shutdown_connections 2023-12-01T01:51:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.755+0000 7f19844db640 1 -- 172.21.15.115:0/3323426380 wait complete. 2023-12-01T01:51:46.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.756+0000 7f19844db640 1 Processor -- start 2023-12-01T01:51:46.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.756+0000 7f19844db640 1 -- start start 2023-12-01T01:51:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.757+0000 7f19844db640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968004d00 0x7f197c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.757+0000 7f19844db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f197c14dbb0 0x7f197c0cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.757+0000 7f19844db640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cb780 0x7f197c14cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.757+0000 7f19844db640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1e8a90 con 0x7f197c14dbb0 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.757+0000 7f19844db640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c1e8d70 con 0x7f1968004d00 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.757+0000 7f19844db640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0cbc80 con 0x7f197c0cb780 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1981a76640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f197c14dbb0 0x7f197c0cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1981a76640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f197c14dbb0 0x7f197c0cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50576/0 (socket says 172.21.15.115:50576) 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1981a76640 1 -- 172.21.15.115:0/729531098 learned_addr learned my addr 172.21.15.115:0/729531098 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1982a78640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cb780 0x7f197c14cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1982277640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968004d00 0x7f197c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1981a76640 1 -- 172.21.15.115:0/729531098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cb780 msgr2=0x7f197c14cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1981a76640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cb780 0x7f197c14cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1981a76640 1 -- 172.21.15.115:0/729531098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968004d00 msgr2=0x7f197c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1981a76640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968004d00 0x7f197c0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1981a76640 1 -- 172.21.15.115:0/729531098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0cbf60 con 0x7f197c14dbb0 2023-12-01T01:51:46.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1982277640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968004d00 0x7f197c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:46.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.758+0000 7f1981a76640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f197c14dbb0 0x7f197c0cb240 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f196c010860 tx=0x7f196c010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:46.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.759+0000 7f1972ffd640 1 -- 172.21.15.115:0/729531098 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c009610 con 0x7f197c14dbb0 2023-12-01T01:51:46.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.759+0000 7f1972ffd640 1 -- 172.21.15.115:0/729531098 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c00eb60 con 0x7f197c14dbb0 2023-12-01T01:51:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.759+0000 7f1972ffd640 1 -- 172.21.15.115:0/729531098 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c01c400 con 0x7f197c14dbb0 2023-12-01T01:51:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.759+0000 7f19844db640 1 -- 172.21.15.115:0/729531098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c0cc240 con 0x7f197c14dbb0 2023-12-01T01:51:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.759+0000 7f19844db640 1 -- 172.21.15.115:0/729531098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c14d620 con 0x7f197c14dbb0 2023-12-01T01:51:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.760+0000 7f19844db640 1 -- 172.21.15.115:0/729531098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c04e890 con 0x7f197c14dbb0 2023-12-01T01:51:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.761+0000 7f1972ffd640 1 -- 172.21.15.115:0/729531098 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f196c01c560 con 0x7f197c14dbb0 2023-12-01T01:51:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.761+0000 7f1972ffd640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f196006a580 0x7f196006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.762+0000 7f1972ffd640 1 -- 172.21.15.115:0/729531098 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f196c089170 con 0x7f197c14dbb0 2023-12-01T01:51:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.762+0000 7f1982277640 1 -- 172.21.15.115:0/729531098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f196006a580 msgr2=0x7f196006ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.762+0000 7f1982277640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f196006a580 0x7f196006ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:46.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.766+0000 7f1972ffd640 1 -- 172.21.15.115:0/729531098 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f196c054590 con 0x7f197c14dbb0 2023-12-01T01:51:46.772 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:46.770+0000 7f8cdb220040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:51:46.793 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:46.791+0000 7fcd4c031040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:51:46.824 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:46.822+0000 7f8cdb220040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:51:46.847 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:46.846+0000 7fcd4c031040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:51:46.874 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:46.872+0000 7f8cdb220040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:51:46.900 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:46.899+0000 7fcd4c031040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:51:46.936 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:46.934+0000 7f8cdb220040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:51:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.955+0000 7f19844db640 1 -- 172.21.15.115:0/729531098 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f197c0d0a70 con 0x7f197c14dbb0 2023-12-01T01:51:46.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.956+0000 7f1972ffd640 1 -- 172.21.15.115:0/729531098 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f196c057bb0 con 0x7f197c14dbb0 2023-12-01T01:51:46.957 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:51:46.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.958+0000 7f19844db640 1 -- 172.21.15.115:0/729531098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f196006a580 msgr2=0x7f196006ca40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:46.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.958+0000 7f19844db640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f196006a580 0x7f196006ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.958+0000 7f19844db640 1 -- 172.21.15.115:0/729531098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f197c14dbb0 msgr2=0x7f197c0cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.958+0000 7f19844db640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f197c14dbb0 0x7f197c0cb240 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f196c010860 tx=0x7f196c010d20 comp rx=0 tx=0).stop 2023-12-01T01:51:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.959+0000 7f19844db640 1 -- 172.21.15.115:0/729531098 shutdown_connections 2023-12-01T01:51:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.959+0000 7f19844db640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f196006a580 0x7f196006ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.959+0000 7f19844db640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f197c0cb780 0x7f197c14cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.959+0000 7f19844db640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f197c14dbb0 0x7f197c0cb240 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.959+0000 7f19844db640 1 --2- 172.21.15.115:0/729531098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1968004d00 0x7f197c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.959+0000 7f19844db640 1 -- 172.21.15.115:0/729531098 >> 172.21.15.115:0/729531098 conn(0x7f197c065960 msgr2=0x7f197c15a380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.959+0000 7f19844db640 1 -- 172.21.15.115:0/729531098 shutdown_connections 2023-12-01T01:51:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:46.959+0000 7f19844db640 1 -- 172.21.15.115:0/729531098 wait complete. 2023-12-01T01:51:46.963 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:46.962+0000 7fcd4c031040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:51:46.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:51:46.985 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:46.984+0000 7f8cdb220040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:51:47.016 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:47.015+0000 7fcd4c031040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:51:47.097 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:47.095+0000 7f8cdb220040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:51:47.133 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:47.132+0000 7fcd4c031040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:51:47.158 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:47.157+0000 7f8cdb220040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:47.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.173+0000 7f22ecd16640 1 Processor -- start 2023-12-01T01:51:47.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.173+0000 7f22ecd16640 1 -- start start 2023-12-01T01:51:47.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.174+0000 7f22ecd16640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 0x7f22e80b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.174+0000 7f22ecd16640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f22e80b7e70 0x7f22e80b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22ecd16640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e80b6890 con 0x7f22e80b6210 2023-12-01T01:51:47.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22ecd16640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e80b6b70 con 0x7f22e80b7e70 2023-12-01T01:51:47.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22ecd16640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80b6e50 con 0x7f22e815cfa0 2023-12-01T01:51:47.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22e5d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f22e80b7e70 0x7f22e80b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:47.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22e5d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f22e80b7e70 0x7f22e80b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50588/0 (socket says 172.21.15.115:50588) 2023-12-01T01:51:47.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22e5d9b640 1 -- 172.21.15.115:0/3938784825 learned_addr learned my addr 172.21.15.115:0/3938784825 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:47.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22e6d9d640 1 --1- 172.21.15.115:0/3938784825 >> v1:172.21.15.115:6789/0 conn(0x7f22e80b6210 0x7f22e80cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52570/0 (socket says 172.21.15.115:52570) 2023-12-01T01:51:47.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22e659c640 1 --2- 172.21.15.115:0/3938784825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 0x7f22e80b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:47.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22e5d9b640 1 -- 172.21.15.115:0/3938784825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 msgr2=0x7f22e80b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:47.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22e5d9b640 1 --2- 172.21.15.115:0/3938784825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 0x7f22e80b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22e5d9b640 1 -- 172.21.15.115:0/3938784825 >> v1:172.21.15.115:6789/0 conn(0x7f22e80b6210 legacy=0x7f22e80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:47.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.175+0000 7f22e5d9b640 1 -- 172.21.15.115:0/3938784825 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e80b7130 con 0x7f22e80b7e70 2023-12-01T01:51:47.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.176+0000 7f22e5d9b640 1 --2- 172.21.15.115:0/3938784825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 0x7f22e80b5cd0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009870 tx=0x7f22dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d009251850a3a877 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:47.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.177+0000 7f22e559a640 1 -- 172.21.15.115:0/3938784825 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc031640 con 0x7f22e80b7e70 2023-12-01T01:51:47.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.177+0000 7f22e559a640 1 -- 172.21.15.115:0/3938784825 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc0317a0 con 0x7f22e80b7e70 2023-12-01T01:51:47.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.177+0000 7f22e559a640 1 -- 172.21.15.115:0/3938784825 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc031b30 con 0x7f22e80b7e70 2023-12-01T01:51:47.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.177+0000 7f22ecd16640 1 -- 172.21.15.115:0/3938784825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 msgr2=0x7f22e80b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:47.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.177+0000 7f22ecd16640 1 --2- 172.21.15.115:0/3938784825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 0x7f22e80b5cd0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009870 tx=0x7f22dc02b8f0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.178+0000 7f22ecd16640 1 -- 172.21.15.115:0/3938784825 shutdown_connections 2023-12-01T01:51:47.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.178+0000 7f22ecd16640 1 --2- 172.21.15.115:0/3938784825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 0x7f22e80b5cd0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.178+0000 7f22ecd16640 1 --2- 172.21.15.115:0/3938784825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 0x7f22e80b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.178+0000 7f22ecd16640 1 -- 172.21.15.115:0/3938784825 >> 172.21.15.115:0/3938784825 conn(0x7f22e806a0b0 msgr2=0x7f22e806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:47.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.178+0000 7f22ecd16640 1 -- 172.21.15.115:0/3938784825 shutdown_connections 2023-12-01T01:51:47.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.178+0000 7f22ecd16640 1 -- 172.21.15.115:0/3938784825 wait complete. 2023-12-01T01:51:47.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.179+0000 7f22ecd16640 1 Processor -- start 2023-12-01T01:51:47.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.179+0000 7f22ecd16640 1 -- start start 2023-12-01T01:51:47.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.180+0000 7f22ecd16640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22e80b6210 0x7f22e80c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.180+0000 7f22ecd16640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 0x7f22e815fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.180+0000 7f22ecd16640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 0x7f22e81f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.180+0000 7f22ecd16640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e81f6ad0 con 0x7f22e80b7e70 2023-12-01T01:51:47.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.180+0000 7f22ecd16640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80c8ed0 con 0x7f22e80b6210 2023-12-01T01:51:47.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.180+0000 7f22ecd16640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e815e120 con 0x7f22e815cfa0 2023-12-01T01:51:47.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 0x7f22e815fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:47.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 0x7f22e815fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50602/0 (socket says 172.21.15.115:50602) 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e5d9b640 1 -- 172.21.15.115:0/3732106797 learned_addr learned my addr 172.21.15.115:0/3732106797 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e6d9d640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 0x7f22e81f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e659c640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22e80b6210 0x7f22e80c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e5d9b640 1 -- 172.21.15.115:0/3732106797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 msgr2=0x7f22e81f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e5d9b640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 0x7f22e81f5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e5d9b640 1 -- 172.21.15.115:0/3732106797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22e80b6210 msgr2=0x7f22e80c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e5d9b640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22e80b6210 0x7f22e80c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e5d9b640 1 -- 172.21.15.115:0/3732106797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc0094d0 con 0x7f22e80b7e70 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e6d9d640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 0x7f22e81f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.181+0000 7f22e5d9b640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 0x7f22e815fa50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f22dc000c00 tx=0x7f22dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.182+0000 7f22d6ffd640 1 -- 172.21.15.115:0/3732106797 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc03ac60 con 0x7f22e80b7e70 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.182+0000 7f22d6ffd640 1 -- 172.21.15.115:0/3732106797 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc03adc0 con 0x7f22e80b7e70 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.182+0000 7f22ecd16640 1 -- 172.21.15.115:0/3732106797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e80b8c00 con 0x7f22e80b7e70 2023-12-01T01:51:47.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.182+0000 7f22d6ffd640 1 -- 172.21.15.115:0/3732106797 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc044630 con 0x7f22e80b7e70 2023-12-01T01:51:47.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.182+0000 7f22ecd16640 1 -- 172.21.15.115:0/3732106797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e81570c0 con 0x7f22e80b7e70 2023-12-01T01:51:47.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.183+0000 7f22ecd16640 1 -- 172.21.15.115:0/3732106797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e80b4430 con 0x7f22e80b7e70 2023-12-01T01:51:47.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.184+0000 7f22d6ffd640 1 -- 172.21.15.115:0/3732106797 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f22dc02f050 con 0x7f22e80b7e70 2023-12-01T01:51:47.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.185+0000 7f22d6ffd640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f22c806a550 0x7f22c806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.186+0000 7f22e659c640 1 -- 172.21.15.115:0/3732106797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f22c806a550 msgr2=0x7f22c806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:47.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.186+0000 7f22d6ffd640 1 -- 172.21.15.115:0/3732106797 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f22dc0aac60 con 0x7f22e80b7e70 2023-12-01T01:51:47.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.186+0000 7f22e659c640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f22c806a550 0x7f22c806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:47.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.189+0000 7f22d6ffd640 1 -- 172.21.15.115:0/3732106797 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f22dc075e30 con 0x7f22e80b7e70 2023-12-01T01:51:47.201 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:47.200+0000 7fcd4c031040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:47.209 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:47.207+0000 7f8cdb220040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:51:47.254 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:47.252+0000 7fcd4c031040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:51:47.265 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:47.264+0000 7f8cdb220040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:51:47.313 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:47.311+0000 7fcd4c031040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:51:47.316 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:47.314+0000 7f8cdb220040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:51:47.364 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:47.363+0000 7fcd4c031040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:51:47.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.373+0000 7f22ecd16640 1 -- 172.21.15.115:0/3732106797 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22e804e890 con 0x7f22e80b7e70 2023-12-01T01:51:47.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.374+0000 7f22d6ffd640 1 -- 172.21.15.115:0/3732106797 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f22dc031e60 con 0x7f22e80b7e70 2023-12-01T01:51:47.375 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:51:47.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.376+0000 7f22ecd16640 1 -- 172.21.15.115:0/3732106797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f22c806a550 msgr2=0x7f22c806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:47.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.376+0000 7f22ecd16640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f22c806a550 0x7f22c806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.376+0000 7f22ecd16640 1 -- 172.21.15.115:0/3732106797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 msgr2=0x7f22e815fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:47.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.376+0000 7f22ecd16640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 0x7f22e815fa50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f22dc000c00 tx=0x7f22dc0046c0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.376+0000 7f22ecd16640 1 -- 172.21.15.115:0/3732106797 shutdown_connections 2023-12-01T01:51:47.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.376+0000 7f22ecd16640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f22c806a550 0x7f22c806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.376+0000 7f22ecd16640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22e815cfa0 0x7f22e81f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.376+0000 7f22ecd16640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22e80b7e70 0x7f22e815fa50 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.376+0000 7f22ecd16640 1 --2- 172.21.15.115:0/3732106797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22e80b6210 0x7f22e80c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.376+0000 7f22ecd16640 1 -- 172.21.15.115:0/3732106797 >> 172.21.15.115:0/3732106797 conn(0x7f22e806a0b0 msgr2=0x7f22e8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:47.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.377+0000 7f22ecd16640 1 -- 172.21.15.115:0/3732106797 shutdown_connections 2023-12-01T01:51:47.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.377+0000 7f22ecd16640 1 -- 172.21.15.115:0/3732106797 wait complete. 2023-12-01T01:51:47.394 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:51:47.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.641+0000 7fc86604a640 1 Processor -- start 2023-12-01T01:51:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.641+0000 7fc86604a640 1 -- start start 2023-12-01T01:51:47.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc86604a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd470 0x7fc8600b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc86604a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc8600b2040 0x7fc86015e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc86604a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc8600cd850 0x7fc86015e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc86604a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8600cc2b0 con 0x7fc8600cd850 2023-12-01T01:51:47.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc86604a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8600ce0b0 con 0x7fc8600b2040 2023-12-01T01:51:47.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc86604a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8600cddd0 con 0x7fc8600cd470 2023-12-01T01:51:47.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc85ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc8600cd850 0x7fc86015e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:47.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc85ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc8600cd850 0x7fc86015e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58814/0 (socket says 172.21.15.115:58814) 2023-12-01T01:51:47.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc85ffff640 1 -- 172.21.15.115:0/2566332496 learned_addr learned my addr 172.21.15.115:0/2566332496 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:47.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc85f7fe640 1 --2- 172.21.15.115:0/2566332496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd470 0x7fc8600b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:47.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.643+0000 7fc85effd640 1 --2- 172.21.15.115:0/2566332496 >> v2:172.21.15.142:3300/0 conn(0x7fc8600b2040 0x7fc86015e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.644+0000 7fc85ffff640 1 -- 172.21.15.115:0/2566332496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd470 msgr2=0x7fc8600b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.644+0000 7fc85ffff640 1 --2- 172.21.15.115:0/2566332496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd470 0x7fc8600b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.644+0000 7fc85ffff640 1 -- 172.21.15.115:0/2566332496 >> v2:172.21.15.142:3300/0 conn(0x7fc8600b2040 msgr2=0x7fc86015e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.644+0000 7fc85ffff640 1 --2- 172.21.15.115:0/2566332496 >> v2:172.21.15.142:3300/0 conn(0x7fc8600b2040 0x7fc86015e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.644+0000 7fc85ffff640 1 -- 172.21.15.115:0/2566332496 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8600d1780 con 0x7fc8600cd850 2023-12-01T01:51:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.644+0000 7fc85f7fe640 1 --2- 172.21.15.115:0/2566332496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd470 0x7fc8600b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:51:47.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.645+0000 7fc85ffff640 1 --2- 172.21.15.115:0/2566332496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600cd850 0x7fc86015e550 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc860070d80 tx=0x7fc85802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b0b3b7b10f4be server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:47.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.646+0000 7fc85e7fc640 1 -- 172.21.15.115:0/2566332496 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85802ca70 con 0x7fc8600cd850 2023-12-01T01:51:47.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.646+0000 7fc85e7fc640 1 -- 172.21.15.115:0/2566332496 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85802cbd0 con 0x7fc8600cd850 2023-12-01T01:51:47.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.646+0000 7fc85e7fc640 1 -- 172.21.15.115:0/2566332496 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8580343f0 con 0x7fc8600cd850 2023-12-01T01:51:47.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.646+0000 7fc86604a640 1 -- 172.21.15.115:0/2566332496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600cd850 msgr2=0x7fc86015e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:47.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.646+0000 7fc86604a640 1 --2- 172.21.15.115:0/2566332496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600cd850 0x7fc86015e550 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc860070d80 tx=0x7fc85802bed0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.646+0000 7fc86604a640 1 -- 172.21.15.115:0/2566332496 shutdown_connections 2023-12-01T01:51:47.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.646+0000 7fc86604a640 1 --2- 172.21.15.115:0/2566332496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600cd850 0x7fc86015e550 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.646+0000 7fc86604a640 1 --2- 172.21.15.115:0/2566332496 >> v2:172.21.15.142:3300/0 conn(0x7fc8600b2040 0x7fc86015e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.646+0000 7fc86604a640 1 --2- 172.21.15.115:0/2566332496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd470 0x7fc8600b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.646+0000 7fc86604a640 1 -- 172.21.15.115:0/2566332496 >> 172.21.15.115:0/2566332496 conn(0x7fc8600d0ba0 msgr2=0x7fc8600d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:47.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.647+0000 7fc86604a640 1 -- 172.21.15.115:0/2566332496 shutdown_connections 2023-12-01T01:51:47.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.647+0000 7fc86604a640 1 -- 172.21.15.115:0/2566332496 wait complete. 2023-12-01T01:51:47.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.648+0000 7fc86604a640 1 Processor -- start 2023-12-01T01:51:47.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.648+0000 7fc86604a640 1 -- start start 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.649+0000 7fc86604a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600b2040 0x7fc860069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.649+0000 7fc86604a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8600cd470 0x7fc8600c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.649+0000 7fc86604a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd850 0x7fc86006b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.649+0000 7fc86604a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8600ca090 con 0x7fc8600b2040 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.649+0000 7fc86604a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86006ab40 con 0x7fc8600cd470 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.649+0000 7fc86604a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86006ba80 con 0x7fc8600cd850 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.649+0000 7fc85f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600b2040 0x7fc860069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.649+0000 7fc85f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600b2040 0x7fc860069cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58822/0 (socket says 172.21.15.115:58822) 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.649+0000 7fc85f7fe640 1 -- 172.21.15.115:0/2382552243 learned_addr learned my addr 172.21.15.115:0/2382552243 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc85ffff640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd850 0x7fc86006b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc85f7fe640 1 -- 172.21.15.115:0/2382552243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd850 msgr2=0x7fc86006b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc85f7fe640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd850 0x7fc86006b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc85f7fe640 1 -- 172.21.15.115:0/2382552243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8600cd470 msgr2=0x7fc8600c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:47.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc85f7fe640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8600cd470 0x7fc8600c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc85f7fe640 1 -- 172.21.15.115:0/2382552243 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc858009b30 con 0x7fc8600b2040 2023-12-01T01:51:47.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc85ffff640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd850 0x7fc86006b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:47.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc85f7fe640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600b2040 0x7fc860069cc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d7d0 tx=0x7fc85000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:47.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc847fff640 1 -- 172.21.15.115:0/2382552243 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850004030 con 0x7fc8600b2040 2023-12-01T01:51:47.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc847fff640 1 -- 172.21.15.115:0/2382552243 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc850004190 con 0x7fc8600b2040 2023-12-01T01:51:47.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc847fff640 1 -- 172.21.15.115:0/2382552243 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850004520 con 0x7fc8600b2040 2023-12-01T01:51:47.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.650+0000 7fc86604a640 1 -- 172.21.15.115:0/2382552243 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86006a2a0 con 0x7fc8600b2040 2023-12-01T01:51:47.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.651+0000 7fc86604a640 1 -- 172.21.15.115:0/2382552243 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86006a860 con 0x7fc8600b2040 2023-12-01T01:51:47.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.652+0000 7fc86604a640 1 -- 172.21.15.115:0/2382552243 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc860067850 con 0x7fc8600b2040 2023-12-01T01:51:47.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.653+0000 7fc847fff640 1 -- 172.21.15.115:0/2382552243 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 88710+0+0 (secure 0 0 0) 0x7fc8500096a0 con 0x7fc8600b2040 2023-12-01T01:51:47.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.653+0000 7fc847fff640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc84c06a550 0x7fc84c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:47.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.653+0000 7fc85effd640 1 -- 172.21.15.115:0/2382552243 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc84c06a550 msgr2=0x7fc84c06ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:47.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.653+0000 7fc847fff640 1 -- 172.21.15.115:0/2382552243 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fc850088ea0 con 0x7fc8600b2040 2023-12-01T01:51:47.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.654+0000 7fc85effd640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc84c06a550 0x7fc84c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:47.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.661+0000 7fc847fff640 1 -- 172.21.15.115:0/2382552243 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc850054180 con 0x7fc8600b2040 2023-12-01T01:51:47.695 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:47.693+0000 7f8cdb220040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:51:47.770 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:47.769+0000 7fcd4c031040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:51:47.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.849+0000 7fc86604a640 1 -- 172.21.15.115:0/2382552243 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc86006be80 con 0x7fc8600b2040 2023-12-01T01:51:47.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.849+0000 7fc847fff640 1 -- 172.21.15.115:0/2382552243 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc86006be80 con 0x7fc8600b2040 2023-12-01T01:51:47.851 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 -- 172.21.15.115:0/2382552243 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc84c06a550 msgr2=0x7fc84c06ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc84c06a550 0x7fc84c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 -- 172.21.15.115:0/2382552243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600b2040 msgr2=0x7fc860069cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600b2040 0x7fc860069cc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d7d0 tx=0x7fc85000dc90 comp rx=0 tx=0).stop 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 -- 172.21.15.115:0/2382552243 shutdown_connections 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc84c06a550 0x7fc84c06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8600cd850 0x7fc86006b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8600cd470 0x7fc8600c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 --2- 172.21.15.115:0/2382552243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8600b2040 0x7fc860069cc0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 -- 172.21.15.115:0/2382552243 >> 172.21.15.115:0/2382552243 conn(0x7fc8600d0ba0 msgr2=0x7fc86015c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 -- 172.21.15.115:0/2382552243 shutdown_connections 2023-12-01T01:51:47.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:47.852+0000 7fc86604a640 1 -- 172.21.15.115:0/2382552243 wait complete. 2023-12-01T01:51:47.858 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:47.856+0000 7f8cdb220040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:51:47.870 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:51:47.937 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:47.935+0000 7fcd4c031040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:51:47.966 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:47.964+0000 7f8cdb220040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:51:48.015 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:48.014+0000 7f8cdb220040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:51:48.051 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:48.050+0000 7fcd4c031040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:51:48.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.068+0000 7fb0dcc2c640 1 Processor -- start 2023-12-01T01:51:48.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.068+0000 7fb0dcc2c640 1 -- start start 2023-12-01T01:51:48.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.069+0000 7fb0dcc2c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb0d815cd20 0x7fb0d80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.069+0000 7fb0dcc2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0d80b5d80 0x7fb0d80b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.069+0000 7fb0dcc2c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb0d80d1230 0x7fb0d80d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.069+0000 7fb0dcc2c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d80b2160 con 0x7fb0d80d1230 2023-12-01T01:51:48.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.069+0000 7fb0dcc2c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d80d1be0 con 0x7fb0d815cd20 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.069+0000 7fb0dcc2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d80d2a10 con 0x7fb0d80b5d80 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.070+0000 7fb0d6d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb0d80d1230 0x7fb0d80d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.070+0000 7fb0d6d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb0d80d1230 0x7fb0d80d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58830/0 (socket says 172.21.15.115:58830) 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.070+0000 7fb0d6d9d640 1 -- 172.21.15.115:0/3122634211 learned_addr learned my addr 172.21.15.115:0/3122634211 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.070+0000 7fb0d5d9b640 1 --2- 172.21.15.115:0/3122634211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0d80b5d80 0x7fb0d80b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.070+0000 7fb0d659c640 1 --2- 172.21.15.115:0/3122634211 >> v2:172.21.15.142:3300/0 conn(0x7fb0d815cd20 0x7fb0d80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.070+0000 7fb0d6d9d640 1 -- 172.21.15.115:0/3122634211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0d80b5d80 msgr2=0x7fb0d80b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.070+0000 7fb0d6d9d640 1 --2- 172.21.15.115:0/3122634211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0d80b5d80 0x7fb0d80b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.070+0000 7fb0d6d9d640 1 -- 172.21.15.115:0/3122634211 >> v2:172.21.15.142:3300/0 conn(0x7fb0d815cd20 msgr2=0x7fb0d80d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.070+0000 7fb0d6d9d640 1 --2- 172.21.15.115:0/3122634211 >> v2:172.21.15.142:3300/0 conn(0x7fb0d815cd20 0x7fb0d80d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.070+0000 7fb0d6d9d640 1 -- 172.21.15.115:0/3122634211 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d80ce2e0 con 0x7fb0d80d1230 2023-12-01T01:51:48.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.071+0000 7fb0d6d9d640 1 --2- 172.21.15.115:0/3122634211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 0x7fb0d80d16a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb0d808bdb0 tx=0x7fb0d002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422aad7ff71fafd2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:48.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.071+0000 7fb0d559a640 1 -- 172.21.15.115:0/3122634211 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00317c0 con 0x7fb0d80d1230 2023-12-01T01:51:48.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0d559a640 1 -- 172.21.15.115:0/3122634211 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d0031920 con 0x7fb0d80d1230 2023-12-01T01:51:48.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0d559a640 1 -- 172.21.15.115:0/3122634211 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0031cb0 con 0x7fb0d80d1230 2023-12-01T01:51:48.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/3122634211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 msgr2=0x7fb0d80d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:48.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0dcc2c640 1 --2- 172.21.15.115:0/3122634211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 0x7fb0d80d16a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb0d808bdb0 tx=0x7fb0d002b860 comp rx=0 tx=0).stop 2023-12-01T01:51:48.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/3122634211 shutdown_connections 2023-12-01T01:51:48.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0dcc2c640 1 --2- 172.21.15.115:0/3122634211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 0x7fb0d80d16a0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0dcc2c640 1 --2- 172.21.15.115:0/3122634211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0d80b5d80 0x7fb0d80b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0dcc2c640 1 --2- 172.21.15.115:0/3122634211 >> v2:172.21.15.142:3300/0 conn(0x7fb0d815cd20 0x7fb0d80d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/3122634211 >> 172.21.15.115:0/3122634211 conn(0x7fb0d806a0e0 msgr2=0x7fb0d806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:48.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/3122634211 shutdown_connections 2023-12-01T01:51:48.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.072+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/3122634211 wait complete. 2023-12-01T01:51:48.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.074+0000 7fb0dcc2c640 1 Processor -- start 2023-12-01T01:51:48.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.074+0000 7fb0dcc2c640 1 -- start start 2023-12-01T01:51:48.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.075+0000 7fb0dcc2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0d80b5d80 0x7fb0d80b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.075+0000 7fb0dcc2c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 0x7fb0d80b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.075+0000 7fb0dcc2c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0d815cd20 0x7fb0d80b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.075+0000 7fb0dcc2c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d81f22e0 con 0x7fb0d80d1230 2023-12-01T01:51:48.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.075+0000 7fb0dcc2c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d80b2e20 con 0x7fb0d815cd20 2023-12-01T01:51:48.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.075+0000 7fb0dcc2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d80b3100 con 0x7fb0d80b5d80 2023-12-01T01:51:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 0x7fb0d80b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 0x7fb0d80b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58846/0 (socket says 172.21.15.115:58846) 2023-12-01T01:51:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d5d9b640 1 -- 172.21.15.115:0/21911238 learned_addr learned my addr 172.21.15.115:0/21911238 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d659c640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0d80b5d80 0x7fb0d80b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d5d9b640 1 -- 172.21.15.115:0/21911238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0d80b5d80 msgr2=0x7fb0d80b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d5d9b640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0d80b5d80 0x7fb0d80b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d6d9d640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0d815cd20 0x7fb0d80b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d5d9b640 1 -- 172.21.15.115:0/21911238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0d815cd20 msgr2=0x7fb0d80b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d5d9b640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0d815cd20 0x7fb0d80b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d5d9b640 1 -- 172.21.15.115:0/21911238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d00094d0 con 0x7fb0d80d1230 2023-12-01T01:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d6d9d640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0d815cd20 0x7fb0d80b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0d5d9b640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 0x7fb0d80b27f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000d7d0 tx=0x7fb0c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.076+0000 7fb0ceffd640 1 -- 172.21.15.115:0/21911238 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000b6c0 con 0x7fb0d80d1230 2023-12-01T01:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.077+0000 7fb0ceffd640 1 -- 172.21.15.115:0/21911238 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c000b820 con 0x7fb0d80d1230 2023-12-01T01:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.077+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/21911238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d80b33e0 con 0x7fb0d80d1230 2023-12-01T01:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.077+0000 7fb0ceffd640 1 -- 172.21.15.115:0/21911238 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000bbb0 con 0x7fb0d80d1230 2023-12-01T01:51:48.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.077+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/21911238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d80b39a0 con 0x7fb0d80d1230 2023-12-01T01:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.078+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/21911238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d80b3b60 con 0x7fb0d80d1230 2023-12-01T01:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.079+0000 7fb0ceffd640 1 -- 172.21.15.115:0/21911238 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 88710+0+0 (secure 0 0 0) 0x7fb0c000bd10 con 0x7fb0d80d1230 2023-12-01T01:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.080+0000 7fb0ceffd640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb0b806a550 0x7fb0b806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.080+0000 7fb0ceffd640 1 -- 172.21.15.115:0/21911238 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fb0c0089130 con 0x7fb0d80d1230 2023-12-01T01:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.080+0000 7fb0d659c640 1 -- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb0b806a550 msgr2=0x7fb0b806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.080+0000 7fb0d659c640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb0b806a550 0x7fb0b806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:48.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.089+0000 7fb0ceffd640 1 -- 172.21.15.115:0/21911238 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb0c00544b0 con 0x7fb0d80d1230 2023-12-01T01:51:48.110 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:48.108+0000 7fcd4c031040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:51:48.124 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:48.123+0000 7f8cdb220040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:48.225 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:48.222+0000 7fcd4c031040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:48.236 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:48.234+0000 7f8cdb220040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:51:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.282+0000 7fb0d659c640 1 -- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb0b806a550 msgr2=0x7fb0b806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.282+0000 7fb0d659c640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb0b806a550 0x7fb0b806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:51:48.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.299+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/21911238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb0d804e890 con 0x7fb0d80d1230 2023-12-01T01:51:48.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.299+0000 7fb0ceffd640 1 -- 172.21.15.115:0/21911238 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v84) v1 ==== 80+0+465 (secure 0 0 0) 0x7fb0c0057ad0 con 0x7fb0d80d1230 2023-12-01T01:51:48.301 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/88317 2023-12-02T01:51:43.682671+0000 2023-12-01T01:51:48.301 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/88317 2023-12-02T01:51:43.682671+0000 2023-12-01T01:51:48.301 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3840278706 2023-12-02T01:51:43.682671+0000 2023-12-01T01:51:48.301 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3824153221 2023-12-02T01:51:43.682671+0000 2023-12-01T01:51:48.301 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/86374784 2023-12-02T01:51:43.682671+0000 2023-12-01T01:51:48.301 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/871551054 2023-12-02T01:51:43.682671+0000 2023-12-01T01:51:48.301 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3669975107 2023-12-02T01:51:43.682671+0000 2023-12-01T01:51:48.301 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2326628113 2023-12-02T01:51:43.682671+0000 2023-12-01T01:51:48.302 INFO:teuthology.orchestra.run.smithi115.stderr:listed 8 entries 2023-12-01T01:51:48.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb0b806a550 msgr2=0x7fb0b806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb0b806a550 0x7fb0b806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 msgr2=0x7fb0d80b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 0x7fb0d80b27f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000d7d0 tx=0x7fb0c000dc90 comp rx=0 tx=0).stop 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/21911238 shutdown_connections 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb0b806a550 0x7fb0b806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0d815cd20 0x7fb0d80b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0d80d1230 0x7fb0d80b27f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 --2- 172.21.15.115:0/21911238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0d80b5d80 0x7fb0d80b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/21911238 >> 172.21.15.115:0/21911238 conn(0x7fb0d806a0e0 msgr2=0x7fb0d815bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/21911238 shutdown_connections 2023-12-01T01:51:48.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.302+0000 7fb0dcc2c640 1 -- 172.21.15.115:0/21911238 wait complete. 2023-12-01T01:51:48.321 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T01:51:48.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:48.321 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T01:51:48.335 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:48.334+0000 7fcd4c031040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:51:48.394 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:48.392+0000 7f8cdb220040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:48.411 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:51:48.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:51:48.411 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T01:51:48.442 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:48.441+0000 7f8cdb220040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:48.502 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:48.500+0000 7fcd4c031040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:51:48.504 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:51:48.504 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:48.552 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:48.550+0000 7fcd4c031040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:51:48.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:51:48.561 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:51:48.561 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:48.564 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:48.563+0000 7f8cdb220040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:51:48.617 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:51:48.617 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:51:48.617 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:51:48.674 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:51:48.674 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:48.675 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:48.674+0000 7fcd4c031040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:51:48.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.880+0000 7f5139f01640 1 Processor -- start 2023-12-01T01:51:48.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.880+0000 7f5139f01640 1 -- start start 2023-12-01T01:51:48.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.882+0000 7f5139f01640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f513415db90 0x7f51340cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.882+0000 7f5139f01640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51340cd3f0 con 0x7f51340cddc0 2023-12-01T01:51:48.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.882+0000 7f5139f01640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513415dff0 con 0x7f513415d7b0 2023-12-01T01:51:48.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.882+0000 7f5139f01640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51340cd0f0 con 0x7f513415db90 2023-12-01T01:51:48.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.882+0000 7f5139700640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f51340cddc0 0x7f51340d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:49026/0 (socket says 172.21.15.115:49026) 2023-12-01T01:51:48.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.882+0000 7f5139700640 1 -- 172.21.15.115:0/533178180 learned_addr learned my addr 172.21.15.115:0/533178180 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:48.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.882+0000 7f5133fff640 1 --2- 172.21.15.115:0/533178180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f513415db90 0x7f51340cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:48.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.883+0000 7f51337fe640 1 -- 172.21.15.115:0/533178180 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059387118 0 0) 0x7f51340cd3f0 con 0x7f51340cddc0 2023-12-01T01:51:48.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.883+0000 7f51337fe640 1 -- 172.21.15.115:0/533178180 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5124003520 con 0x7f51340cddc0 2023-12-01T01:51:48.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.883+0000 7f51337fe640 1 -- 172.21.15.115:0/533178180 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701900051 0 0) 0x7f513415dff0 con 0x7f513415d7b0 2023-12-01T01:51:48.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.883+0000 7f51337fe640 1 -- 172.21.15.115:0/533178180 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51340cd3f0 con 0x7f513415d7b0 2023-12-01T01:51:48.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.883+0000 7f51337fe640 1 -- 172.21.15.115:0/533178180 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 605605666 0 0) 0x7f5124003520 con 0x7f51340cddc0 2023-12-01T01:51:48.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.883+0000 7f5133fff640 1 -- 172.21.15.115:0/533178180 >> v1:172.21.15.142:6789/0 conn(0x7f513415d7b0 legacy=0x7f5134160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:48.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.883+0000 7f5133fff640 1 -- 172.21.15.115:0/533178180 >> v1:172.21.15.115:6789/0 conn(0x7f51340cddc0 legacy=0x7f51340d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:48.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.883+0000 7f5133fff640 1 -- 172.21.15.115:0/533178180 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5134160ec0 con 0x7f513415db90 2023-12-01T01:51:48.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.884+0000 7f5133fff640 1 --2- 172.21.15.115:0/533178180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f513415db90 0x7f51340cd880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f51200097d0 tx=0x7f512002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34fb1403cbd418c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:48.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.885+0000 7f51337fe640 1 -- 172.21.15.115:0/533178180 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0019f0 con 0x7f513415db90 2023-12-01T01:51:48.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.885+0000 7f51337fe640 1 -- 172.21.15.115:0/533178180 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5120031580 con 0x7f513415db90 2023-12-01T01:51:48.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.885+0000 7f51337fe640 1 -- 172.21.15.115:0/533178180 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120031910 con 0x7f513415db90 2023-12-01T01:51:48.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.886+0000 7f5139f01640 1 -- 172.21.15.115:0/533178180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f513415db90 msgr2=0x7f51340cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:48.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.886+0000 7f5139f01640 1 --2- 172.21.15.115:0/533178180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f513415db90 0x7f51340cd880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f51200097d0 tx=0x7f512002b8a0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.886+0000 7f5139f01640 1 -- 172.21.15.115:0/533178180 shutdown_connections 2023-12-01T01:51:48.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.886+0000 7f5139f01640 1 --2- 172.21.15.115:0/533178180 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f513415db90 0x7f51340cd880 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.886+0000 7f5139f01640 1 -- 172.21.15.115:0/533178180 >> 172.21.15.115:0/533178180 conn(0x7f513406a0e0 msgr2=0x7f513406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:48.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.886+0000 7f5139f01640 1 -- 172.21.15.115:0/533178180 shutdown_connections 2023-12-01T01:51:48.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.886+0000 7f5139f01640 1 -- 172.21.15.115:0/533178180 wait complete. 2023-12-01T01:51:48.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.887+0000 7f5139f01640 1 Processor -- start 2023-12-01T01:51:48.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.887+0000 7f5139f01640 1 -- start start 2023-12-01T01:51:48.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.888+0000 7f5139f01640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51340cddc0 0x7f513414d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.888+0000 7f5139f01640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f513415d7b0 0x7f51340c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.888+0000 7f5139f01640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513415db90 0x7f5134153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.888+0000 7f5139f01640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340b3440 con 0x7f51340cddc0 2023-12-01T01:51:48.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.888+0000 7f5139f01640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340b3690 con 0x7f513415db90 2023-12-01T01:51:48.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.888+0000 7f5139f01640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51340b3970 con 0x7f513415d7b0 2023-12-01T01:51:48.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.889+0000 7f5138eff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51340cddc0 0x7f513414d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.889+0000 7f5138eff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51340cddc0 0x7f513414d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58850/0 (socket says 172.21.15.115:58850) 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.889+0000 7f5138eff640 1 -- 172.21.15.115:0/2884526051 learned_addr learned my addr 172.21.15.115:0/2884526051 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.889+0000 7f5138eff640 1 -- 172.21.15.115:0/2884526051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f513415d7b0 msgr2=0x7f51340c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.889+0000 7f5138eff640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f513415d7b0 0x7f51340c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.889+0000 7f5138eff640 1 -- 172.21.15.115:0/2884526051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513415db90 msgr2=0x7f5134153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.889+0000 7f5138eff640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513415db90 0x7f5134153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.889+0000 7f5138eff640 1 -- 172.21.15.115:0/2884526051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5120009400 con 0x7f51340cddc0 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.889+0000 7f5138eff640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51340cddc0 0x7f513414d520 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f512800c490 tx=0x7f512800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.889+0000 7f51317fa640 1 -- 172.21.15.115:0/2884526051 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280052d0 con 0x7f51340cddc0 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.890+0000 7f51317fa640 1 -- 172.21.15.115:0/2884526051 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5128005430 con 0x7f51340cddc0 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.890+0000 7f5139f01640 1 -- 172.21.15.115:0/2884526051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51340b3c50 con 0x7f51340cddc0 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.890+0000 7f51317fa640 1 -- 172.21.15.115:0/2884526051 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128012870 con 0x7f51340cddc0 2023-12-01T01:51:48.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.890+0000 7f5139f01640 1 -- 172.21.15.115:0/2884526051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51340b4210 con 0x7f51340cddc0 2023-12-01T01:51:48.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.891+0000 7f5139f01640 1 -- 172.21.15.115:0/2884526051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51340673a0 con 0x7f51340cddc0 2023-12-01T01:51:48.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.892+0000 7f51317fa640 1 -- 172.21.15.115:0/2884526051 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f51280129d0 con 0x7f51340cddc0 2023-12-01T01:51:48.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.893+0000 7f51317fa640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f511806a580 0x7f511806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:48.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.893+0000 7f51317fa640 1 -- 172.21.15.115:0/2884526051 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f51280887c0 con 0x7f51340cddc0 2023-12-01T01:51:48.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.893+0000 7f5133fff640 1 -- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f511806a580 msgr2=0x7f511806ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:48.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.893+0000 7f5133fff640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f511806a580 0x7f511806ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:48.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:48.901+0000 7f51317fa640 1 -- 172.21.15.115:0/2884526051 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5128053b50 con 0x7f51340cddc0 2023-12-01T01:51:49.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.094+0000 7f5133fff640 1 -- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f511806a580 msgr2=0x7f511806ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:49.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.094+0000 7f5133fff640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f511806a580 0x7f511806ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:51:49.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.127+0000 7f5139f01640 1 -- 172.21.15.115:0/2884526051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51340d3100 con 0x7f51340cddc0 2023-12-01T01:51:49.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.128+0000 7f51317fa640 1 -- 172.21.15.115:0/2884526051 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f5128057170 con 0x7f51340cddc0 2023-12-01T01:51:49.130 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:49.130 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1284508688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1284508688},{"type":"v1","addr":"172.21.15.142:6833","nonce":1284508688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1196894153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1196894153},{"type":"v1","addr":"172.21.15.142:6835","nonce":1196894153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6432,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/120786293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":120786293},{"type":"v1","addr":"172.21.15.115:6836","nonce":120786293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:51:24.174363+0000","modified":"2023-12-01T01:51:27.708969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6435},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6435":{"gid":6435,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3394497964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3394497964},{"type":"v1","addr":"172.21.15.115:6837","nonce":3394497964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-01T01:51:49.130 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 32 2023-12-01T01:51:49.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.130+0000 7f5139f01640 1 -- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f511806a580 msgr2=0x7f511806ca40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.130+0000 7f5139f01640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f511806a580 0x7f511806ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.130+0000 7f5139f01640 1 -- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51340cddc0 msgr2=0x7f513414d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.130+0000 7f5139f01640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51340cddc0 0x7f513414d520 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f512800c490 tx=0x7f512800c950 comp rx=0 tx=0).stop 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.131+0000 7f5139f01640 1 -- 172.21.15.115:0/2884526051 shutdown_connections 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.131+0000 7f5139f01640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f511806a580 0x7f511806ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.131+0000 7f5139f01640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f513415d7b0 0x7f51340c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.131+0000 7f5139f01640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f513415db90 0x7f5134153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.131+0000 7f5139f01640 1 --2- 172.21.15.115:0/2884526051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51340cddc0 0x7f513414d520 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.131+0000 7f5139f01640 1 -- 172.21.15.115:0/2884526051 >> 172.21.15.115:0/2884526051 conn(0x7f513406a0e0 msgr2=0x7f51340d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.131+0000 7f5139f01640 1 -- 172.21.15.115:0/2884526051 shutdown_connections 2023-12-01T01:51:49.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.131+0000 7f5139f01640 1 -- 172.21.15.115:0/2884526051 wait complete. 2023-12-01T01:51:49.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:49.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.334+0000 7f3770470640 1 Processor -- start 2023-12-01T01:51:49.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.334+0000 7f3770470640 1 -- start start 2023-12-01T01:51:49.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.335+0000 7f3770470640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37680d1940 0x7f37680d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.335+0000 7f3770470640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37680ce2e0 con 0x7f376815cd20 2023-12-01T01:51:49.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.335+0000 7f3770470640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37680ce000 con 0x7f37680b5d80 2023-12-01T01:51:49.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.335+0000 7f3770470640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37680b2160 con 0x7f37680d1940 2023-12-01T01:51:49.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.335+0000 7f376e20c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f376815cd20 0x7f37680b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:49028/0 (socket says 172.21.15.115:49028) 2023-12-01T01:51:49.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.335+0000 7f376e20c640 1 -- 172.21.15.115:0/1109268362 learned_addr learned my addr 172.21.15.115:0/1109268362 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:49.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.336+0000 7f376ea0d640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37680d1940 0x7f37680d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.336+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699397160 0 0) 0x7f37680ce2e0 con 0x7f376815cd20 2023-12-01T01:51:49.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.336+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3754003520 con 0x7f376815cd20 2023-12-01T01:51:49.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.336+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 543796560 0 0) 0x7f3754003520 con 0x7f376815cd20 2023-12-01T01:51:49.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.336+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37680d1940 msgr2=0x7f37680d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:49.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.336+0000 7f376d20a640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37680d1940 0x7f37680d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.336+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 >> v1:172.21.15.142:6789/0 conn(0x7f37680b5d80 legacy=0x7f37680d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.336+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680cd3d0 con 0x7f376815cd20 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.336+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3764001670 con 0x7f376815cd20 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.336+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 >> v1:172.21.15.115:6789/0 conn(0x7f376815cd20 legacy=0x7f37680b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376d20a640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3754003a10 0x7f3754003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376d20a640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37540047c0 0x7f3754004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376d20a640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3754005180 0x7f375400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375400bf70 con 0x7f3754005180 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375400c0e0 con 0x7f37540047c0 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375400c2a0 con 0x7f3754003a10 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376ea0d640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3754005180 0x7f375400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376e20c640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3754003a10 0x7f3754003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376ea0d640 1 -- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3754003a10 msgr2=0x7f3754003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:49.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376ea0d640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3754003a10 0x7f3754003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376ea0d640 1 -- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37540047c0 msgr2=0x7f3754004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:49.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376da0b640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37540047c0 0x7f3754004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376ea0d640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37540047c0 0x7f3754004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.337+0000 7f376ea0d640 1 -- 172.21.15.115:0/1109268362 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3754003520 con 0x7f3754005180 2023-12-01T01:51:49.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.338+0000 7f376e20c640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3754003a10 0x7f3754003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:51:49.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.338+0000 7f376da0b640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37540047c0 0x7f3754004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:49.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.339+0000 7f376ea0d640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3754005180 0x7f375400b980 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3760009fc0 tx=0x7f37540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c703e2c4842d76ee server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:49.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.339+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376002cc20 con 0x7f3754005180 2023-12-01T01:51:49.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.339+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376000f040 con 0x7f3754005180 2023-12-01T01:51:49.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.339+0000 7f376d20a640 1 -- 172.21.15.115:0/1109268362 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760034440 con 0x7f3754005180 2023-12-01T01:51:49.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.339+0000 7f3770470640 1 -- 172.21.15.115:0/1109268362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3754005180 msgr2=0x7f375400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.339+0000 7f3770470640 1 --2- 172.21.15.115:0/1109268362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3754005180 0x7f375400b980 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3760009fc0 tx=0x7f37540057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.340+0000 7f376ea0d640 1 -- 172.21.15.115:0/1109268362 reap_dead start 2023-12-01T01:51:49.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.340+0000 7f3770470640 1 -- 172.21.15.115:0/1109268362 shutdown_connections 2023-12-01T01:51:49.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.340+0000 7f3770470640 1 -- 172.21.15.115:0/1109268362 >> 172.21.15.115:0/1109268362 conn(0x7f376806a0e0 msgr2=0x7f376806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:49.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.340+0000 7f3770470640 1 -- 172.21.15.115:0/1109268362 shutdown_connections 2023-12-01T01:51:49.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.340+0000 7f3770470640 1 -- 172.21.15.115:0/1109268362 wait complete. 2023-12-01T01:51:49.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.342+0000 7f3770470640 1 Processor -- start 2023-12-01T01:51:49.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.342+0000 7f3770470640 1 -- start start 2023-12-01T01:51:49.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f3770470640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3754005180 0x7f37680c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f3770470640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37680c6780 0x7f3768157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f3770470640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37681f1860 0x7f37681f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f3770470640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37680c85a0 con 0x7f37680c6780 2023-12-01T01:51:49.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f3770470640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37680c6d40 con 0x7f37681f1860 2023-12-01T01:51:49.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f3770470640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37680c7020 con 0x7f3754005180 2023-12-01T01:51:49.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f376da0b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37680c6780 0x7f3768157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f376da0b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37680c6780 0x7f3768157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58876/0 (socket says 172.21.15.115:58876) 2023-12-01T01:51:49.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f376da0b640 1 -- 172.21.15.115:0/3775508635 learned_addr learned my addr 172.21.15.115:0/3775508635 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f376e20c640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3754005180 0x7f37680c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f376ea0d640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37681f1860 0x7f37681f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f376da0b640 1 -- 172.21.15.115:0/3775508635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3754005180 msgr2=0x7f37680c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f376da0b640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3754005180 0x7f37680c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f376da0b640 1 -- 172.21.15.115:0/3775508635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37681f1860 msgr2=0x7f37681f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f376da0b640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37681f1860 0x7f37681f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.343+0000 7f376da0b640 1 -- 172.21.15.115:0/3775508635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680c7300 con 0x7f37680c6780 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.344+0000 7f376da0b640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37680c6780 0x7f3768157ea0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f375800d7a0 tx=0x7f375800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.344+0000 7f375effd640 1 -- 172.21.15.115:0/3775508635 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758017070 con 0x7f37680c6780 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.344+0000 7f375effd640 1 -- 172.21.15.115:0/3775508635 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3758005070 con 0x7f37680c6780 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.344+0000 7f375effd640 1 -- 172.21.15.115:0/3775508635 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758005420 con 0x7f37680c6780 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.344+0000 7f3770470640 1 -- 172.21.15.115:0/3775508635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37680c75e0 con 0x7f37680c6780 2023-12-01T01:51:49.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.344+0000 7f3770470640 1 -- 172.21.15.115:0/3775508635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37681f2430 con 0x7f37680c6780 2023-12-01T01:51:49.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.345+0000 7f3770470640 1 -- 172.21.15.115:0/3775508635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f376804e890 con 0x7f37680c6780 2023-12-01T01:51:49.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.346+0000 7f375effd640 1 -- 172.21.15.115:0/3775508635 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f3758007030 con 0x7f37680c6780 2023-12-01T01:51:49.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.347+0000 7f375effd640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f374c06a550 0x7f374c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.347+0000 7f375effd640 1 -- 172.21.15.115:0/3775508635 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f3758088520 con 0x7f37680c6780 2023-12-01T01:51:49.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.347+0000 7f376e20c640 1 -- 172.21.15.115:0/3775508635 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f374c06a550 msgr2=0x7f374c06ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:49.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.347+0000 7f376e20c640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f374c06a550 0x7f374c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:49.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.355+0000 7f375effd640 1 -- 172.21.15.115:0/3775508635 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3758053840 con 0x7f37680c6780 2023-12-01T01:51:49.425 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:49.423+0000 7f8cdb220040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:51:49.501 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:49.500+0000 7fcd4c031040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:51:49.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.535+0000 7f3770470640 1 -- 172.21.15.115:0/3775508635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37680d2680 con 0x7f37680c6780 2023-12-01T01:51:49.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.537+0000 7f375effd640 1 -- 172.21.15.115:0/3775508635 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f3758056e60 con 0x7f37680c6780 2023-12-01T01:51:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:49.539 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":84,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:43.682749+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:51:22.633453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-01T01:51:23.191899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:51:43.682671+0000","172.21.15.115:6822/88317":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/3840278706":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/3824153221":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/86374784":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/871551054":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/3669975107":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/2326628113":"2023-12-02T01:51:43.682671+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:49.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 -- 172.21.15.115:0/3775508635 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f374c06a550 msgr2=0x7f374c06ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:49.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f374c06a550 0x7f374c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 -- 172.21.15.115:0/3775508635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37680c6780 msgr2=0x7f3768157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37680c6780 0x7f3768157ea0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f375800d7a0 tx=0x7f375800dc60 comp rx=0 tx=0).stop 2023-12-01T01:51:49.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 -- 172.21.15.115:0/3775508635 shutdown_connections 2023-12-01T01:51:49.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f374c06a550 0x7f374c06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37681f1860 0x7f37681f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37680c6780 0x7f3768157ea0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 --2- 172.21.15.115:0/3775508635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3754005180 0x7f37680c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 -- 172.21.15.115:0/3775508635 >> 172.21.15.115:0/3775508635 conn(0x7f376806a0e0 msgr2=0x7f3768065960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:49.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 -- 172.21.15.115:0/3775508635 shutdown_connections 2023-12-01T01:51:49.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.540+0000 7f3770470640 1 -- 172.21.15.115:0/3775508635 wait complete. 2023-12-01T01:51:49.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:51:22.633453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:51:23.191899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:51:49.561 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:51:49.720 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:49.718+0000 7f8cdb220040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:51:49.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.771+0000 7fb9decdc640 1 Processor -- start 2023-12-01T01:51:49.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.771+0000 7fb9decdc640 1 -- start start 2023-12-01T01:51:49.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.772+0000 7fb9decdc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 0x7fb9d80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.772+0000 7fb9decdc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d80cdcd0 con 0x7fb9d80cfb80 2023-12-01T01:51:49.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.772+0000 7fb9decdc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d8161830 con 0x7fb9d8154010 2023-12-01T01:51:49.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.772+0000 7fb9decdc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80cd3a0 con 0x7fb9d8153c30 2023-12-01T01:51:49.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.773+0000 7fb9de4db640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb9d80cfb80 0x7fb9d8160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:49036/0 (socket says 172.21.15.115:49036) 2023-12-01T01:51:49.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.773+0000 7fb9de4db640 1 -- 172.21.15.115:0/3449137747 learned_addr learned my addr 172.21.15.115:0/3449137747 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:49.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.773+0000 7fb9ddcda640 1 --2- 172.21.15.115:0/3449137747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 0x7fb9d80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.773+0000 7fb9dccd8640 1 -- 172.21.15.115:0/3449137747 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361613812 0 0) 0x7fb9d80cdcd0 con 0x7fb9d80cfb80 2023-12-01T01:51:49.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.773+0000 7fb9dccd8640 1 -- 172.21.15.115:0/3449137747 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c4003520 con 0x7fb9d80cfb80 2023-12-01T01:51:49.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.773+0000 7fb9dccd8640 1 -- 172.21.15.115:0/3449137747 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 830521896 0 0) 0x7fb9c4003520 con 0x7fb9d80cfb80 2023-12-01T01:51:49.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.773+0000 7fb9ddcda640 1 -- 172.21.15.115:0/3449137747 >> v1:172.21.15.142:6789/0 conn(0x7fb9d8154010 legacy=0x7fb9d80cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.773+0000 7fb9ddcda640 1 -- 172.21.15.115:0/3449137747 >> v1:172.21.15.115:6789/0 conn(0x7fb9d80cfb80 legacy=0x7fb9d8160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.773+0000 7fb9ddcda640 1 -- 172.21.15.115:0/3449137747 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80cd680 con 0x7fb9d8153c30 2023-12-01T01:51:49.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.775+0000 7fb9ddcda640 1 --2- 172.21.15.115:0/3449137747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 0x7fb9d80ce160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009dc0 tx=0x7fb9d402bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec92fe6e1ab39a99 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:49.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.775+0000 7fb9dccd8640 1 -- 172.21.15.115:0/3449137747 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0001670 con 0x7fb9d8153c30 2023-12-01T01:51:49.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.775+0000 7fb9dccd8640 1 -- 172.21.15.115:0/3449137747 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d402c910 con 0x7fb9d8153c30 2023-12-01T01:51:49.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.775+0000 7fb9dccd8640 1 -- 172.21.15.115:0/3449137747 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d402cc80 con 0x7fb9d8153c30 2023-12-01T01:51:49.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.776+0000 7fb9decdc640 1 -- 172.21.15.115:0/3449137747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 msgr2=0x7fb9d80ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.776+0000 7fb9decdc640 1 --2- 172.21.15.115:0/3449137747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 0x7fb9d80ce160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009dc0 tx=0x7fb9d402bde0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.776+0000 7fb9decdc640 1 -- 172.21.15.115:0/3449137747 shutdown_connections 2023-12-01T01:51:49.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.776+0000 7fb9decdc640 1 --2- 172.21.15.115:0/3449137747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 0x7fb9d80ce160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.776+0000 7fb9decdc640 1 -- 172.21.15.115:0/3449137747 >> 172.21.15.115:0/3449137747 conn(0x7fb9d806a0b0 msgr2=0x7fb9d806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:49.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.776+0000 7fb9decdc640 1 -- 172.21.15.115:0/3449137747 shutdown_connections 2023-12-01T01:51:49.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.776+0000 7fb9decdc640 1 -- 172.21.15.115:0/3449137747 wait complete. 2023-12-01T01:51:49.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.777+0000 7fb9decdc640 1 Processor -- start 2023-12-01T01:51:49.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.778+0000 7fb9decdc640 1 -- start start 2023-12-01T01:51:49.779 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:49.777+0000 7f8cdb220040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:51:49.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.778+0000 7fb9decdc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9d80cfb80 0x7fb9d80cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.778+0000 7fb9decdc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 0x7fb9d80cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.778+0000 7fb9decdc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9d8154010 0x7fb9d815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9decdc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d8157410 con 0x7fb9d80cfb80 2023-12-01T01:51:49.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9decdc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80cb960 con 0x7fb9d8154010 2023-12-01T01:51:49.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9decdc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80cbc40 con 0x7fb9d8153c30 2023-12-01T01:51:49.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9ddcda640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9d80cfb80 0x7fb9d80cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9ddcda640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9d80cfb80 0x7fb9d80cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58890/0 (socket says 172.21.15.115:58890) 2023-12-01T01:51:49.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9ddcda640 1 -- 172.21.15.115:0/751984433 learned_addr learned my addr 172.21.15.115:0/751984433 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:49.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9de4db640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9d8154010 0x7fb9d815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9dd4d9640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 0x7fb9d80cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:49.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9ddcda640 1 -- 172.21.15.115:0/751984433 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 msgr2=0x7fb9d80cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9ddcda640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 0x7fb9d80cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9ddcda640 1 -- 172.21.15.115:0/751984433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9d8154010 msgr2=0x7fb9d815c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9ddcda640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9d8154010 0x7fb9d815c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9ddcda640 1 -- 172.21.15.115:0/751984433 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4009a70 con 0x7fb9d80cfb80 2023-12-01T01:51:49.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9dd4d9640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 0x7fb9d80cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:49.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.779+0000 7fb9de4db640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9d8154010 0x7fb9d815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:49.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.780+0000 7fb9ddcda640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9d80cfb80 0x7fb9d80cb2d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009ee0 tx=0x7fb9d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:49.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.780+0000 7fb9ce7fc640 1 -- 172.21.15.115:0/751984433 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4004250 con 0x7fb9d80cfb80 2023-12-01T01:51:49.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.780+0000 7fb9ce7fc640 1 -- 172.21.15.115:0/751984433 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d40043b0 con 0x7fb9d80cfb80 2023-12-01T01:51:49.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.780+0000 7fb9decdc640 1 -- 172.21.15.115:0/751984433 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d80cbf20 con 0x7fb9d80cfb80 2023-12-01T01:51:49.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.780+0000 7fb9ce7fc640 1 -- 172.21.15.115:0/751984433 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4004740 con 0x7fb9d80cfb80 2023-12-01T01:51:49.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.780+0000 7fb9decdc640 1 -- 172.21.15.115:0/751984433 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d80cc4e0 con 0x7fb9d80cfb80 2023-12-01T01:51:49.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.781+0000 7fb9decdc640 1 -- 172.21.15.115:0/751984433 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d80618d0 con 0x7fb9d80cfb80 2023-12-01T01:51:49.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.782+0000 7fb9ce7fc640 1 -- 172.21.15.115:0/751984433 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 37) v1 ==== 88710+0+0 (secure 0 0 0) 0x7fb9d4034440 con 0x7fb9d80cfb80 2023-12-01T01:51:49.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.783+0000 7fb9ce7fc640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9bc06a550 0x7fb9bc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:49.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.783+0000 7fb9ce7fc640 1 -- 172.21.15.115:0/751984433 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fb9d40ab550 con 0x7fb9d80cfb80 2023-12-01T01:51:49.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.783+0000 7fb9dd4d9640 1 -- 172.21.15.115:0/751984433 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9bc06a550 msgr2=0x7fb9bc06ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:51:49.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.783+0000 7fb9dd4d9640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9bc06a550 0x7fb9bc06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:51:49.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.787+0000 7fb9ce7fc640 1 -- 172.21.15.115:0/751984433 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb9d40769a0 con 0x7fb9d80cfb80 2023-12-01T01:51:49.817 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:49.816+0000 7fcd4c031040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:51:49.878 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:49.876+0000 7fcd4c031040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:51:49.895 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:51:49.893+0000 7f8cdb220040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:51:49.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.970+0000 7fb9decdc640 1 -- 172.21.15.115:0/751984433 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9d815fb20 con 0x7fb9d80cfb80 2023-12-01T01:51:49.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.971+0000 7fb9ce7fc640 1 -- 172.21.15.115:0/751984433 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9d4079fc0 con 0x7fb9d80cfb80 2023-12-01T01:51:49.972 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:51:49.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 -- 172.21.15.115:0/751984433 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9bc06a550 msgr2=0x7fb9bc06ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:51:49.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9bc06a550 0x7fb9bc06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 -- 172.21.15.115:0/751984433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9d80cfb80 msgr2=0x7fb9d80cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:49.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9d80cfb80 0x7fb9d80cb2d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009ee0 tx=0x7fb9d4004a40 comp rx=0 tx=0).stop 2023-12-01T01:51:49.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 -- 172.21.15.115:0/751984433 shutdown_connections 2023-12-01T01:51:49.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9bc06a550 0x7fb9bc06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9d8153c30 0x7fb9d80cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9d8154010 0x7fb9d815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 --2- 172.21.15.115:0/751984433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9d80cfb80 0x7fb9d80cb2d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:49.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 -- 172.21.15.115:0/751984433 >> 172.21.15.115:0/751984433 conn(0x7fb9d806a0b0 msgr2=0x7fb9d806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:49.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 -- 172.21.15.115:0/751984433 shutdown_connections 2023-12-01T01:51:49.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:49.974+0000 7fb9decdc640 1 -- 172.21.15.115:0/751984433 wait complete. 2023-12-01T01:51:49.993 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:51:49.999 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:49.996+0000 7fcd4c031040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:51:50.079 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:50.077+0000 7fccd0cec640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:50.079 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:50.077+0000 7fccd0cec640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:50.079 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:50.077+0000 7fccd0cec640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:50.079 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:50.077+0000 7fccd0cec640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:50.079 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:50.077+0000 7fccd0cec640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:50.079 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:50.077+0000 7fcccdce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:50.079 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:50.077+0000 7fcccdce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:50.079 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:50.077+0000 7fcccdce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:50.080 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:50.077+0000 7fcccdce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:50.080 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:51:50.077+0000 7fcccdce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:51:50.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.189+0000 7fa6c8e34640 1 Processor -- start 2023-12-01T01:51:50.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.189+0000 7fa6c8e34640 1 -- start start 2023-12-01T01:51:50.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c8e34640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6c41599e0 0x7fa6c4159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:50.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c8e34640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa6c40c9770 0x7fa6c4070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:50.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c8e34640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c415e440 con 0x7fa6c40af370 2023-12-01T01:51:50.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c8e34640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c415a300 con 0x7fa6c40c9770 2023-12-01T01:51:50.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c8e34640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c415d550 con 0x7fa6c41599e0 2023-12-01T01:51:50.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa6c40c9770 0x7fa6c4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:50.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6baffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa6c40af370 0x7fa6c40c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:49050/0 (socket says 172.21.15.115:49050) 2023-12-01T01:51:50.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6baffd640 1 -- 172.21.15.115:0/2225402604 learned_addr learned my addr 172.21.15.115:0/2225402604 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:50.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c37fe640 1 --2- 172.21.15.115:0/2225402604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6c41599e0 0x7fa6c4159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:50.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c2ffd640 1 -- 172.21.15.115:0/2225402604 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1434165705 0 0) 0x7fa6c415e440 con 0x7fa6c40af370 2023-12-01T01:51:50.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c2ffd640 1 -- 172.21.15.115:0/2225402604 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b0003520 con 0x7fa6c40af370 2023-12-01T01:51:50.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c3fff640 1 -- 172.21.15.115:0/2225402604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6c41599e0 msgr2=0x7fa6c4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:50.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c3fff640 1 --2- 172.21.15.115:0/2225402604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6c41599e0 0x7fa6c4159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.191+0000 7fa6c3fff640 1 -- 172.21.15.115:0/2225402604 >> v1:172.21.15.115:6789/0 conn(0x7fa6c40af370 legacy=0x7fa6c40c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:50.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.192+0000 7fa6c3fff640 1 -- 172.21.15.115:0/2225402604 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c415d830 con 0x7fa6c40c9770 2023-12-01T01:51:50.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.193+0000 7fa6c3fff640 1 --2- 172.21.15.115:0/2225402604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 0x7fa6c4070300 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009d90 tx=0x7fa6bc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f1a1147349dad0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:50.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.193+0000 7fa6c2ffd640 1 -- 172.21.15.115:0/2225402604 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc02cc50 con 0x7fa6c40c9770 2023-12-01T01:51:50.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.193+0000 7fa6c2ffd640 1 -- 172.21.15.115:0/2225402604 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc02cdb0 con 0x7fa6c40c9770 2023-12-01T01:51:50.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.193+0000 7fa6c2ffd640 1 -- 172.21.15.115:0/2225402604 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc034660 con 0x7fa6c40c9770 2023-12-01T01:51:50.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.193+0000 7fa6c8e34640 1 -- 172.21.15.115:0/2225402604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 msgr2=0x7fa6c4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:50.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.193+0000 7fa6c8e34640 1 --2- 172.21.15.115:0/2225402604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 0x7fa6c4070300 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009d90 tx=0x7fa6bc02bea0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.194+0000 7fa6c8e34640 1 -- 172.21.15.115:0/2225402604 shutdown_connections 2023-12-01T01:51:50.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.194+0000 7fa6c8e34640 1 --2- 172.21.15.115:0/2225402604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 0x7fa6c4070300 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.194+0000 7fa6c8e34640 1 --2- 172.21.15.115:0/2225402604 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6c41599e0 0x7fa6c4159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.194+0000 7fa6c8e34640 1 -- 172.21.15.115:0/2225402604 >> 172.21.15.115:0/2225402604 conn(0x7fa6c4065960 msgr2=0x7fa6c4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:50.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.194+0000 7fa6c8e34640 1 -- 172.21.15.115:0/2225402604 shutdown_connections 2023-12-01T01:51:50.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.194+0000 7fa6c8e34640 1 -- 172.21.15.115:0/2225402604 wait complete. 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.195+0000 7fa6c8e34640 1 Processor -- start 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.195+0000 7fa6c8e34640 1 -- start start 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.196+0000 7fa6c8e34640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6c40af370 0x7fa6c415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.196+0000 7fa6c8e34640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 0x7fa6c40c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.196+0000 7fa6c8e34640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6c41599e0 0x7fa6c414ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.196+0000 7fa6c8e34640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c4155a70 con 0x7fa6c40c9770 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.196+0000 7fa6c8e34640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c415c060 con 0x7fa6c40af370 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.196+0000 7fa6c8e34640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c415c340 con 0x7fa6c41599e0 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6baffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 0x7fa6c40c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6baffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 0x7fa6c40c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59002/0 (socket says 172.21.15.115:59002) 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6baffd640 1 -- 172.21.15.115:0/211711254 learned_addr learned my addr 172.21.15.115:0/211711254 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6c37fe640 1 --2- 172.21.15.115:0/211711254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6c40af370 0x7fa6c415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:50.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6baffd640 1 -- 172.21.15.115:0/211711254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6c41599e0 msgr2=0x7fa6c414ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6baffd640 1 --2- 172.21.15.115:0/211711254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6c41599e0 0x7fa6c414ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6baffd640 1 -- 172.21.15.115:0/211711254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6c40af370 msgr2=0x7fa6c415ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6baffd640 1 --2- 172.21.15.115:0/211711254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6c40af370 0x7fa6c415ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6baffd640 1 -- 172.21.15.115:0/211711254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc009a70 con 0x7fa6c40c9770 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6baffd640 1 --2- 172.21.15.115:0/211711254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 0x7fa6c40c8080 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c410 tx=0x7fa6ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.197+0000 7fa6c0ff9640 1 -- 172.21.15.115:0/211711254 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac005ce0 con 0x7fa6c40c9770 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.198+0000 7fa6c0ff9640 1 -- 172.21.15.115:0/211711254 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac005e40 con 0x7fa6c40c9770 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.198+0000 7fa6c0ff9640 1 -- 172.21.15.115:0/211711254 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0072f0 con 0x7fa6c40c9770 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.198+0000 7fa6c8e34640 1 -- 172.21.15.115:0/211711254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c415c620 con 0x7fa6c40c9770 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.198+0000 7fa6c8e34640 1 -- 172.21.15.115:0/211711254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c40c85c0 con 0x7fa6c40c9770 2023-12-01T01:51:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.199+0000 7fa6c8e34640 1 -- 172.21.15.115:0/211711254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c4155bb0 con 0x7fa6c40c9770 2023-12-01T01:51:50.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.200+0000 7fa6c0ff9640 1 -- 172.21.15.115:0/211711254 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 38) v1 ==== 88318+0+0 (secure 0 0 0) 0x7fa6ac007510 con 0x7fa6c40c9770 2023-12-01T01:51:50.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.201+0000 7fa6c0ff9640 1 -- 172.21.15.115:0/211711254 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fa6ac087d50 con 0x7fa6c40c9770 2023-12-01T01:51:50.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.206+0000 7fa6c0ff9640 1 -- 172.21.15.115:0/211711254 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa6c4155bb0 con 0x7fa6c40c9770 2023-12-01T01:51:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.393+0000 7fa6c8e34640 1 -- 172.21.15.115:0/211711254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6c4155bb0 con 0x7fa6c40c9770 2023-12-01T01:51:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.393+0000 7fa6c0ff9640 1 -- 172.21.15.115:0/211711254 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6c4155bb0 con 0x7fa6c40c9770 2023-12-01T01:51:50.395 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:51:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.395+0000 7fa6c8e34640 1 -- 172.21.15.115:0/211711254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 msgr2=0x7fa6c40c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.395+0000 7fa6c8e34640 1 --2- 172.21.15.115:0/211711254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 0x7fa6c40c8080 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c410 tx=0x7fa6ac00c8d0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.396+0000 7fa6c8e34640 1 -- 172.21.15.115:0/211711254 shutdown_connections 2023-12-01T01:51:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.396+0000 7fa6c8e34640 1 --2- 172.21.15.115:0/211711254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6c41599e0 0x7fa6c414ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.396+0000 7fa6c8e34640 1 --2- 172.21.15.115:0/211711254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6c40c9770 0x7fa6c40c8080 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.396+0000 7fa6c8e34640 1 --2- 172.21.15.115:0/211711254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6c40af370 0x7fa6c415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.396+0000 7fa6c8e34640 1 -- 172.21.15.115:0/211711254 >> 172.21.15.115:0/211711254 conn(0x7fa6c4065960 msgr2=0x7fa6c40ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:50.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.396+0000 7fa6c8e34640 1 -- 172.21.15.115:0/211711254 shutdown_connections 2023-12-01T01:51:50.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.396+0000 7fa6c8e34640 1 -- 172.21.15.115:0/211711254 wait complete. 2023-12-01T01:51:50.414 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:51:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.611+0000 7f7a5da8b640 1 Processor -- start 2023-12-01T01:51:50.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.611+0000 7f7a5da8b640 1 -- start start 2023-12-01T01:51:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.613+0000 7f7a5da8b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7a5815cd90 0x7f7a580cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.613+0000 7f7a5da8b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7a580cfd60 0x7f7a58161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.613+0000 7f7a5da8b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a580ce5e0 con 0x7f7a580b5fa0 2023-12-01T01:51:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.613+0000 7f7a5da8b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a580d0ec0 con 0x7f7a580cfd60 2023-12-01T01:51:50.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.613+0000 7f7a5da8b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a58161950 con 0x7f7a5815cd90 2023-12-01T01:51:50.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.613+0000 7f7a57fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7a580cfd60 0x7f7a58161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:50.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.613+0000 7f7a57fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7a580cfd60 0x7f7a58161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59004/0 (socket says 172.21.15.115:59004) 2023-12-01T01:51:50.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.613+0000 7f7a57fff640 1 -- 172.21.15.115:0/2144282583 learned_addr learned my addr 172.21.15.115:0/2144282583 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:50.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.613+0000 7f7a577fe640 1 --2- 172.21.15.115:0/2144282583 >> v2:172.21.15.142:3300/0 conn(0x7f7a5815cd90 0x7f7a580cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:50.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.614+0000 7f7a57fff640 1 -- 172.21.15.115:0/2144282583 >> v2:172.21.15.142:3300/0 conn(0x7f7a5815cd90 msgr2=0x7f7a580cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:50.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.614+0000 7f7a57fff640 1 --2- 172.21.15.115:0/2144282583 >> v2:172.21.15.142:3300/0 conn(0x7f7a5815cd90 0x7f7a580cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.614+0000 7f7a57fff640 1 -- 172.21.15.115:0/2144282583 >> v1:172.21.15.115:6789/0 conn(0x7f7a580b5fa0 legacy=0x7f7a580cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:50.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.614+0000 7f7a57fff640 1 -- 172.21.15.115:0/2144282583 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a580ce300 con 0x7f7a580cfd60 2023-12-01T01:51:50.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.614+0000 7f7a567fc640 1 -- 172.21.15.115:0/2144282583 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908967822 0 0) 0x7f7a40001110 con 0x7f7a580b5fa0 2023-12-01T01:51:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.615+0000 7f7a57fff640 1 --2- 172.21.15.115:0/2144282583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580cfd60 0x7f7a58161410 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7a5806bd10 tx=0x7f7a5002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f953e5f80115557 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.615+0000 7f7a567fc640 1 -- 172.21.15.115:0/2144282583 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5002cb20 con 0x7f7a580cfd60 2023-12-01T01:51:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.615+0000 7f7a567fc640 1 -- 172.21.15.115:0/2144282583 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5002cc80 con 0x7f7a580cfd60 2023-12-01T01:51:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.615+0000 7f7a567fc640 1 -- 172.21.15.115:0/2144282583 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500343f0 con 0x7f7a580cfd60 2023-12-01T01:51:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.616+0000 7f7a5da8b640 1 -- 172.21.15.115:0/2144282583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580cfd60 msgr2=0x7f7a58161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.616+0000 7f7a5da8b640 1 --2- 172.21.15.115:0/2144282583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580cfd60 0x7f7a58161410 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7a5806bd10 tx=0x7f7a5002bf80 comp rx=0 tx=0).stop 2023-12-01T01:51:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.616+0000 7f7a5da8b640 1 -- 172.21.15.115:0/2144282583 shutdown_connections 2023-12-01T01:51:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.616+0000 7f7a5da8b640 1 --2- 172.21.15.115:0/2144282583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580cfd60 0x7f7a58161410 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.616+0000 7f7a5da8b640 1 --2- 172.21.15.115:0/2144282583 >> v2:172.21.15.142:3300/0 conn(0x7f7a5815cd90 0x7f7a580cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.616+0000 7f7a5da8b640 1 -- 172.21.15.115:0/2144282583 >> 172.21.15.115:0/2144282583 conn(0x7f7a58065960 msgr2=0x7f7a58065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:50.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.616+0000 7f7a5da8b640 1 -- 172.21.15.115:0/2144282583 shutdown_connections 2023-12-01T01:51:50.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.616+0000 7f7a5da8b640 1 -- 172.21.15.115:0/2144282583 wait complete. 2023-12-01T01:51:50.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.618+0000 7f7a5da8b640 1 Processor -- start 2023-12-01T01:51:50.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.618+0000 7f7a5da8b640 1 -- start start 2023-12-01T01:51:50.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.618+0000 7f7a5da8b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580b5fa0 0x7f7a581f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:50.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a5da8b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a580cfd60 0x7f7a5814f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:50.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a5da8b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5815cd90 0x7f7a58154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a5da8b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a581f6880 con 0x7f7a580b5fa0 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a5da8b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58150450 con 0x7f7a5815cd90 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a5da8b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58151280 con 0x7f7a580cfd60 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a577fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580b5fa0 0x7f7a581f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a577fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580b5fa0 0x7f7a581f6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59010/0 (socket says 172.21.15.115:59010) 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a577fe640 1 -- 172.21.15.115:0/3258160327 learned_addr learned my addr 172.21.15.115:0/3258160327 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a577fe640 1 -- 172.21.15.115:0/3258160327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a580cfd60 msgr2=0x7f7a5814f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a577fe640 1 --2- 172.21.15.115:0/3258160327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a580cfd60 0x7f7a5814f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a577fe640 1 -- 172.21.15.115:0/3258160327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5815cd90 msgr2=0x7f7a58154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a577fe640 1 --2- 172.21.15.115:0/3258160327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5815cd90 0x7f7a58154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a577fe640 1 -- 172.21.15.115:0/3258160327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50009b30 con 0x7f7a580b5fa0 2023-12-01T01:51:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.619+0000 7f7a577fe640 1 --2- 172.21.15.115:0/3258160327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580b5fa0 0x7f7a581f6340 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800d800 tx=0x7f7a4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.620+0000 7f7a4ffff640 1 -- 172.21.15.115:0/3258160327 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4800b6c0 con 0x7f7a580b5fa0 2023-12-01T01:51:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.620+0000 7f7a4ffff640 1 -- 172.21.15.115:0/3258160327 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4800b820 con 0x7f7a580b5fa0 2023-12-01T01:51:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.620+0000 7f7a4ffff640 1 -- 172.21.15.115:0/3258160327 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4800bbb0 con 0x7f7a580b5fa0 2023-12-01T01:51:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.620+0000 7f7a5da8b640 1 -- 172.21.15.115:0/3258160327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a58151560 con 0x7f7a580b5fa0 2023-12-01T01:51:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.620+0000 7f7a5da8b640 1 -- 172.21.15.115:0/3258160327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a58151b20 con 0x7f7a580b5fa0 2023-12-01T01:51:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.621+0000 7f7a5da8b640 1 -- 172.21.15.115:0/3258160327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5804e890 con 0x7f7a580b5fa0 2023-12-01T01:51:50.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.622+0000 7f7a4ffff640 1 -- 172.21.15.115:0/3258160327 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 38) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f7a4800bd10 con 0x7f7a580b5fa0 2023-12-01T01:51:50.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.623+0000 7f7a4ffff640 1 -- 172.21.15.115:0/3258160327 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f7a48088b50 con 0x7f7a580b5fa0 2023-12-01T01:51:50.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.627+0000 7f7a4ffff640 1 -- 172.21.15.115:0/3258160327 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7a48054330 con 0x7f7a580b5fa0 2023-12-01T01:51:50.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.815+0000 7f7a5da8b640 1 -- 172.21.15.115:0/3258160327 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a581506a0 con 0x7f7a580b5fa0 2023-12-01T01:51:50.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.816+0000 7f7a4ffff640 1 -- 172.21.15.115:0/3258160327 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a48057950 con 0x7f7a580b5fa0 2023-12-01T01:51:50.817 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:51:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.818+0000 7f7a5da8b640 1 -- 172.21.15.115:0/3258160327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580b5fa0 msgr2=0x7f7a581f6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.818+0000 7f7a5da8b640 1 --2- 172.21.15.115:0/3258160327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580b5fa0 0x7f7a581f6340 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800d800 tx=0x7f7a4800dcc0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.818+0000 7f7a5da8b640 1 -- 172.21.15.115:0/3258160327 shutdown_connections 2023-12-01T01:51:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.818+0000 7f7a5da8b640 1 --2- 172.21.15.115:0/3258160327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a580cfd60 0x7f7a5814f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.818+0000 7f7a5da8b640 1 --2- 172.21.15.115:0/3258160327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a5815cd90 0x7f7a58154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.818+0000 7f7a5da8b640 1 --2- 172.21.15.115:0/3258160327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a580b5fa0 0x7f7a581f6340 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.818+0000 7f7a5da8b640 1 -- 172.21.15.115:0/3258160327 >> 172.21.15.115:0/3258160327 conn(0x7f7a58065960 msgr2=0x7f7a580b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.818+0000 7f7a5da8b640 1 -- 172.21.15.115:0/3258160327 shutdown_connections 2023-12-01T01:51:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:50.819+0000 7f7a5da8b640 1 -- 172.21.15.115:0/3258160327 wait complete. 2023-12-01T01:51:50.838 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:51.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.053+0000 7f47d0d37640 1 Processor -- start 2023-12-01T01:51:51.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.053+0000 7f47d0d37640 1 -- start start 2023-12-01T01:51:51.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.054+0000 7f47d0d37640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47cc0cd200 0x7f47cc071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.054+0000 7f47d0d37640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f47cc06c2c0 0x7f47cc06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.054+0000 7f47d0d37640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47cc070720 con 0x7f47cc1596d0 2023-12-01T01:51:51.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.054+0000 7f47d0d37640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47cc071cc0 con 0x7f47cc06c2c0 2023-12-01T01:51:51.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.054+0000 7f47d0d37640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc06cc40 con 0x7f47cc0cd200 2023-12-01T01:51:51.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47cb7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f47cc1596d0 0x7f47cc070010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:49068/0 (socket says 172.21.15.115:49068) 2023-12-01T01:51:51.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47cb7fe640 1 -- 172.21.15.115:0/2922520976 learned_addr learned my addr 172.21.15.115:0/2922520976 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:51.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468190705 0 0) 0x7f47cc070720 con 0x7f47cc1596d0 2023-12-01T01:51:51.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b8003520 con 0x7f47cc1596d0 2023-12-01T01:51:51.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47cbfff640 1 --2- 172.21.15.115:0/2922520976 >> v2:172.21.15.142:3300/0 conn(0x7f47cc06c2c0 0x7f47cc06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47caffd640 1 --2- 172.21.15.115:0/2922520976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47cc0cd200 0x7f47cc071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1442328799 0 0) 0x7f47b8003520 con 0x7f47cc1596d0 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47cc0cd200 msgr2=0x7f47cc071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47ca7fc640 1 --2- 172.21.15.115:0/2922520976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47cc0cd200 0x7f47cc071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 >> v2:172.21.15.142:3300/0 conn(0x7f47cc06c2c0 msgr2=0x7f47cc06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47ca7fc640 1 --2- 172.21.15.115:0/2922520976 >> v2:172.21.15.142:3300/0 conn(0x7f47cc06c2c0 0x7f47cc06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc0cd660 con 0x7f47cc1596d0 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f47c40019f0 con 0x7f47cc1596d0 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.055+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 >> v1:172.21.15.115:6789/0 conn(0x7f47cc1596d0 legacy=0x7f47cc070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47ca7fc640 1 --2- 172.21.15.115:0/2922520976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b80039e0 0x7f47b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47ca7fc640 1 --2- 172.21.15.115:0/2922520976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47cc0cd200 0x7f47b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47ca7fc640 1 --2- 172.21.15.115:0/2922520976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b8004d00 0x7f47b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc071cc0 con 0x7f47b80039e0 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc06cc40 con 0x7f47b8004d00 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b800dd20 con 0x7f47cc0cd200 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47cb7fe640 1 --2- 172.21.15.115:0/2922520976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b80039e0 0x7f47b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47cb7fe640 1 -- 172.21.15.115:0/2922520976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47cc0cd200 msgr2=0x7f47b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:51.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47cb7fe640 1 --2- 172.21.15.115:0/2922520976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47cc0cd200 0x7f47b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47cb7fe640 1 -- 172.21.15.115:0/2922520976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b8004d00 msgr2=0x7f47b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:51.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47cb7fe640 1 --2- 172.21.15.115:0/2922520976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47b8004d00 0x7f47b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.056+0000 7f47cb7fe640 1 -- 172.21.15.115:0/2922520976 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b8003520 con 0x7f47b80039e0 2023-12-01T01:51:51.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.057+0000 7f47cb7fe640 1 --2- 172.21.15.115:0/2922520976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b80039e0 0x7f47b8003e00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f47c400a810 tx=0x7f47b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9692f31838e23494 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:51.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.057+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4034630 con 0x7f47b80039e0 2023-12-01T01:51:51.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.057+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c402f070 con 0x7f47b80039e0 2023-12-01T01:51:51.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.057+0000 7f47ca7fc640 1 -- 172.21.15.115:0/2922520976 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40349f0 con 0x7f47b80039e0 2023-12-01T01:51:51.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.058+0000 7f47d0d37640 1 -- 172.21.15.115:0/2922520976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b80039e0 msgr2=0x7f47b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.058+0000 7f47d0d37640 1 --2- 172.21.15.115:0/2922520976 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47b80039e0 0x7f47b8003e00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f47c400a810 tx=0x7f47b80057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.058+0000 7f47cbfff640 1 -- 172.21.15.115:0/2922520976 reap_dead start 2023-12-01T01:51:51.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.058+0000 7f47d0d37640 1 -- 172.21.15.115:0/2922520976 shutdown_connections 2023-12-01T01:51:51.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.058+0000 7f47d0d37640 1 -- 172.21.15.115:0/2922520976 >> 172.21.15.115:0/2922520976 conn(0x7f47cc06a0e0 msgr2=0x7f47cc06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:51.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.058+0000 7f47d0d37640 1 -- 172.21.15.115:0/2922520976 shutdown_connections 2023-12-01T01:51:51.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.058+0000 7f47d0d37640 1 -- 172.21.15.115:0/2922520976 wait complete. 2023-12-01T01:51:51.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.060+0000 7f47d0d37640 1 Processor -- start 2023-12-01T01:51:51.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.060+0000 7f47d0d37640 1 -- start start 2023-12-01T01:51:51.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.061+0000 7f47d0d37640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b80039e0 0x7f47cc15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.061+0000 7f47d0d37640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc15c720 0x7f47cc0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.061+0000 7f47d0d37640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47cc0b1720 0x7f47cc0b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.061+0000 7f47d0d37640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc14ea80 con 0x7f47cc15c720 2023-12-01T01:51:51.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.061+0000 7f47d0d37640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc15cba0 con 0x7f47cc0b1720 2023-12-01T01:51:51.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.061+0000 7f47d0d37640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0b20e0 con 0x7f47b80039e0 2023-12-01T01:51:51.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.061+0000 7f47caffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc15c720 0x7f47cc0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.061+0000 7f47caffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc15c720 0x7f47cc0b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59030/0 (socket says 172.21.15.115:59030) 2023-12-01T01:51:51.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.061+0000 7f47caffd640 1 -- 172.21.15.115:0/2285641568 learned_addr learned my addr 172.21.15.115:0/2285641568 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:51.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47caffd640 1 -- 172.21.15.115:0/2285641568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b80039e0 msgr2=0x7f47cc15b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47cb7fe640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b80039e0 0x7f47cc15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47caffd640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b80039e0 0x7f47cc15b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47caffd640 1 -- 172.21.15.115:0/2285641568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47cc0b1720 msgr2=0x7f47cc0b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47cbfff640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47cc0b1720 0x7f47cc0b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47caffd640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47cc0b1720 0x7f47cc0b1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47caffd640 1 -- 172.21.15.115:0/2285641568 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc1f58a0 con 0x7f47cc15c720 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47caffd640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc15c720 0x7f47cc0b4fc0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f47b400d5f0 tx=0x7f47b400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47c3fff640 1 -- 172.21.15.115:0/2285641568 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b40125f0 con 0x7f47cc15c720 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47c3fff640 1 -- 172.21.15.115:0/2285641568 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b4012750 con 0x7f47cc15c720 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47c3fff640 1 -- 172.21.15.115:0/2285641568 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4012b00 con 0x7f47cc15c720 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.062+0000 7f47d0d37640 1 -- 172.21.15.115:0/2285641568 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47cc1f5af0 con 0x7f47cc15c720 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.063+0000 7f47d0d37640 1 -- 172.21.15.115:0/2285641568 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47cc1f60b0 con 0x7f47cc15c720 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.064+0000 7f47d0d37640 1 -- 172.21.15.115:0/2285641568 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47cc0673a0 con 0x7f47cc15c720 2023-12-01T01:51:51.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.064+0000 7f47c3fff640 1 -- 172.21.15.115:0/2285641568 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 39) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f47b4012c60 con 0x7f47cc15c720 2023-12-01T01:51:51.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.065+0000 7f47c3fff640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f47ac06a280 0x7f47ac06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.065+0000 7f47c3fff640 1 -- 172.21.15.115:0/2285641568 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f47b4088d20 con 0x7f47cc15c720 2023-12-01T01:51:51.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.065+0000 7f47cb7fe640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f47ac06a280 0x7f47ac06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.066+0000 7f47cb7fe640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f47ac06a280 0x7f47ac06c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f47c4034000 tx=0x7f47c4009030 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:51.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.070+0000 7f47c3fff640 1 -- 172.21.15.115:0/2285641568 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f47b4054320 con 0x7f47cc15c720 2023-12-01T01:51:51.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.296+0000 7f47d0d37640 1 -- 172.21.15.115:0/2285641568 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47cc071240 con 0x7f47cc15c720 2023-12-01T01:51:51.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.297+0000 7f47c3fff640 1 -- 172.21.15.115:0/2285641568 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f47b4057940 con 0x7f47cc15c720 2023-12-01T01:51:51.299 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:51.299 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1284508688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1284508688},{"type":"v1","addr":"172.21.15.142:6833","nonce":1284508688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1196894153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1196894153},{"type":"v1","addr":"172.21.15.142:6835","nonce":1196894153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6432,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/120786293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":120786293},{"type":"v1","addr":"172.21.15.115:6836","nonce":120786293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:51:24.174363+0000","modified":"2023-12-01T01:51:27.708969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6435},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6435":{"gid":6435,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3394497964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3394497964},{"type":"v1","addr":"172.21.15.115:6837","nonce":3394497964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-01T01:51:51.299 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 32 2023-12-01T01:51:51.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.300+0000 7f47d0d37640 1 -- 172.21.15.115:0/2285641568 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f47ac06a280 msgr2=0x7f47ac06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.300+0000 7f47d0d37640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f47ac06a280 0x7f47ac06c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f47c4034000 tx=0x7f47c4009030 comp rx=0 tx=0).stop 2023-12-01T01:51:51.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.300+0000 7f47d0d37640 1 -- 172.21.15.115:0/2285641568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc15c720 msgr2=0x7f47cc0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.300+0000 7f47d0d37640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc15c720 0x7f47cc0b4fc0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f47b400d5f0 tx=0x7f47b400dab0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.301+0000 7f47d0d37640 1 -- 172.21.15.115:0/2285641568 shutdown_connections 2023-12-01T01:51:51.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.301+0000 7f47d0d37640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f47ac06a280 0x7f47ac06c740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.301+0000 7f47d0d37640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47cc0b1720 0x7f47cc0b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.301+0000 7f47d0d37640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47cc15c720 0x7f47cc0b4fc0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.301+0000 7f47d0d37640 1 --2- 172.21.15.115:0/2285641568 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47b80039e0 0x7f47cc15b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.301+0000 7f47d0d37640 1 -- 172.21.15.115:0/2285641568 >> 172.21.15.115:0/2285641568 conn(0x7f47cc06a0e0 msgr2=0x7f47cc0c4580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:51.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.301+0000 7f47d0d37640 1 -- 172.21.15.115:0/2285641568 shutdown_connections 2023-12-01T01:51:51.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.301+0000 7f47d0d37640 1 -- 172.21.15.115:0/2285641568 wait complete. 2023-12-01T01:51:51.319 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:51.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.532+0000 7f4548869640 1 Processor -- start 2023-12-01T01:51:51.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.532+0000 7f4548869640 1 -- start start 2023-12-01T01:51:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.534+0000 7f4548869640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f45440ccc80 0x7f454406fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.534+0000 7f4548869640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454415d600 con 0x7f45440ad2b0 2023-12-01T01:51:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.534+0000 7f4548869640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454406c4b0 con 0x7f454415d220 2023-12-01T01:51:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.534+0000 7f4548869640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454406cc10 con 0x7f45440ccc80 2023-12-01T01:51:51.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.534+0000 7f4542d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f45440ad2b0 0x7f454406fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:49072/0 (socket says 172.21.15.115:49072) 2023-12-01T01:51:51.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.534+0000 7f4542d9d640 1 -- 172.21.15.115:0/190059423 learned_addr learned my addr 172.21.15.115:0/190059423 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:51.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.534+0000 7f4539d9b640 1 --2- 172.21.15.115:0/190059423 >> v2:172.21.15.142:3300/0 conn(0x7f45440ccc80 0x7f454406fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710492287 0 0) 0x7f453c001110 con 0x7f45440ad2b0 2023-12-01T01:51:51.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4530003520 con 0x7f45440ad2b0 2023-12-01T01:51:51.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3331192939 0 0) 0x7f4530003520 con 0x7f45440ad2b0 2023-12-01T01:51:51.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 >> v2:172.21.15.142:3300/0 conn(0x7f45440ccc80 msgr2=0x7f454406fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:51.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 --2- 172.21.15.115:0/190059423 >> v2:172.21.15.142:3300/0 conn(0x7f45440ccc80 0x7f454406fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 >> v1:172.21.15.142:6789/0 conn(0x7f454415d220 legacy=0x7f45440cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454406f2f0 con 0x7f45440ad2b0 2023-12-01T01:51:51.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f453c001800 con 0x7f45440ad2b0 2023-12-01T01:51:51.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 >> v1:172.21.15.115:6789/0 conn(0x7f45440ad2b0 legacy=0x7f454406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45300039e0 0x7f4530003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45300047c0 0x7f4530004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45300051f0 0x7f453000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454406cc10 con 0x7f45300039e0 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453000c150 con 0x7f45300051f0 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.535+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453000c310 con 0x7f45300047c0 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.536+0000 7f454259c640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45300039e0 0x7f4530003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.536+0000 7f4542d9d640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45300051f0 0x7f453000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.536+0000 7f4539d9b640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45300047c0 0x7f4530004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.536+0000 7f454259c640 1 -- 172.21.15.115:0/190059423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45300047c0 msgr2=0x7f4530004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.536+0000 7f454259c640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45300047c0 0x7f4530004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.536+0000 7f454259c640 1 -- 172.21.15.115:0/190059423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45300051f0 msgr2=0x7f453000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.536+0000 7f454259c640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45300051f0 0x7f453000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.536+0000 7f4542d9d640 1 -- 172.21.15.115:0/190059423 reap_dead start 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.536+0000 7f454259c640 1 -- 172.21.15.115:0/190059423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4530003520 con 0x7f45300039e0 2023-12-01T01:51:51.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.536+0000 7f4539d9b640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45300047c0 0x7f4530004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:51.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.537+0000 7f454259c640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45300039e0 0x7f4530003e00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4534001e50 tx=0x7f45300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba04fbd0524f47c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:51.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.537+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534003b70 con 0x7f45300039e0 2023-12-01T01:51:51.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.538+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534003cd0 con 0x7f45300039e0 2023-12-01T01:51:51.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.538+0000 7f4541d9b640 1 -- 172.21.15.115:0/190059423 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534005310 con 0x7f45300039e0 2023-12-01T01:51:51.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.538+0000 7f4548869640 1 -- 172.21.15.115:0/190059423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45300039e0 msgr2=0x7f4530003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.538+0000 7f4548869640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45300039e0 0x7f4530003e00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4534001e50 tx=0x7f45300057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.538+0000 7f4548869640 1 -- 172.21.15.115:0/190059423 shutdown_connections 2023-12-01T01:51:51.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.538+0000 7f4548869640 1 --2- 172.21.15.115:0/190059423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45300039e0 0x7f4530003e00 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.538+0000 7f4548869640 1 -- 172.21.15.115:0/190059423 >> 172.21.15.115:0/190059423 conn(0x7f45440ce880 msgr2=0x7f45440cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:51.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.538+0000 7f4548869640 1 -- 172.21.15.115:0/190059423 shutdown_connections 2023-12-01T01:51:51.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.538+0000 7f4548869640 1 -- 172.21.15.115:0/190059423 wait complete. 2023-12-01T01:51:51.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.540+0000 7f4548869640 1 Processor -- start 2023-12-01T01:51:51.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.540+0000 7f4548869640 1 -- start start 2023-12-01T01:51:51.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.541+0000 7f4548869640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45300039e0 0x7f45440c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.541+0000 7f4548869640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454414ac50 0x7f45440c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.541+0000 7f4548869640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45440c93f0 0x7f454415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.541+0000 7f4548869640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45440c23b0 con 0x7f454414ac50 2023-12-01T01:51:51.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.541+0000 7f4548869640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45440c2690 con 0x7f45300039e0 2023-12-01T01:51:51.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.541+0000 7f4548869640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45440c6350 con 0x7f45440c93f0 2023-12-01T01:51:51.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.541+0000 7f4539d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454414ac50 0x7f45440c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f4539d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454414ac50 0x7f45440c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59062/0 (socket says 172.21.15.115:59062) 2023-12-01T01:51:51.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f4539d9b640 1 -- 172.21.15.115:0/3586998684 learned_addr learned my addr 172.21.15.115:0/3586998684 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:51.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f454259c640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45300039e0 0x7f45440c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f4539d9b640 1 -- 172.21.15.115:0/3586998684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45440c93f0 msgr2=0x7f454415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f4539d9b640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45440c93f0 0x7f454415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f4539d9b640 1 -- 172.21.15.115:0/3586998684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45300039e0 msgr2=0x7f45440c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f4539d9b640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45300039e0 0x7f45440c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f4539d9b640 1 -- 172.21.15.115:0/3586998684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440c9900 con 0x7f454414ac50 2023-12-01T01:51:51.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f4539d9b640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454414ac50 0x7f45440c8eb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f452c0107f0 tx=0x7f452c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:51.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f453b7fe640 1 -- 172.21.15.115:0/3586998684 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c00eb60 con 0x7f454414ac50 2023-12-01T01:51:51.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.542+0000 7f453b7fe640 1 -- 172.21.15.115:0/3586998684 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c00ecc0 con 0x7f454414ac50 2023-12-01T01:51:51.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.543+0000 7f4548869640 1 -- 172.21.15.115:0/3586998684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45440c9be0 con 0x7f454414ac50 2023-12-01T01:51:51.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.543+0000 7f453b7fe640 1 -- 172.21.15.115:0/3586998684 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c010ea0 con 0x7f454414ac50 2023-12-01T01:51:51.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.543+0000 7f4548869640 1 -- 172.21.15.115:0/3586998684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4544158450 con 0x7f454414ac50 2023-12-01T01:51:51.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.544+0000 7f4548869640 1 -- 172.21.15.115:0/3586998684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454404e890 con 0x7f454414ac50 2023-12-01T01:51:51.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.545+0000 7f453b7fe640 1 -- 172.21.15.115:0/3586998684 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 39) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f452c01c400 con 0x7f454414ac50 2023-12-01T01:51:51.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.546+0000 7f453b7fe640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f452406a280 0x7f452406c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.546+0000 7f453b7fe640 1 -- 172.21.15.115:0/3586998684 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f452c088ae0 con 0x7f454414ac50 2023-12-01T01:51:51.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.546+0000 7f454259c640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f452406a280 0x7f452406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.547+0000 7f454259c640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f452406a280 0x7f452406c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4534034000 tx=0x7f4534009030 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:51.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.550+0000 7f453b7fe640 1 -- 172.21.15.115:0/3586998684 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f452c054260 con 0x7f454414ac50 2023-12-01T01:51:51.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.743+0000 7f4548869640 1 -- 172.21.15.115:0/3586998684 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45440cc8f0 con 0x7f454414ac50 2023-12-01T01:51:51.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.750+0000 7f453b7fe640 1 -- 172.21.15.115:0/3586998684 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17015 (secure 0 0 0) 0x7f452c057880 con 0x7f454414ac50 2023-12-01T01:51:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:51.752 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":85,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:49.999481+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:51:22.633453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-01T01:51:23.191899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/60448999":"2023-12-02T01:51:49.999425+0000","172.21.15.115:0/1905595030":"2023-12-02T01:51:49.999425+0000","172.21.15.115:0/2838184043":"2023-12-02T01:51:49.999425+0000","172.21.15.115:0/573957418":"2023-12-02T01:51:49.999425+0000","172.21.15.115:0/3305720342":"2023-12-02T01:51:49.999425+0000","172.21.15.115:6824/88317":"2023-12-02T01:51:43.682671+0000","172.21.15.115:6822/88317":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/246605387":"2023-12-02T01:51:49.999425+0000","172.21.15.115:0/3840278706":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/3824153221":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/86374784":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/871551054":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/3669975107":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/2326628113":"2023-12-02T01:51:43.682671+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:51.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.752+0000 7f4548869640 1 -- 172.21.15.115:0/3586998684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f452406a280 msgr2=0x7f452406c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f452406a280 0x7f452406c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4534034000 tx=0x7f4534009030 comp rx=0 tx=0).stop 2023-12-01T01:51:51.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 -- 172.21.15.115:0/3586998684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454414ac50 msgr2=0x7f45440c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454414ac50 0x7f45440c8eb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f452c0107f0 tx=0x7f452c010cb0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 -- 172.21.15.115:0/3586998684 shutdown_connections 2023-12-01T01:51:51.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f452406a280 0x7f452406c740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45440c93f0 0x7f454415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454414ac50 0x7f45440c8eb0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 --2- 172.21.15.115:0/3586998684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45300039e0 0x7f45440c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 -- 172.21.15.115:0/3586998684 >> 172.21.15.115:0/3586998684 conn(0x7f45440ce880 msgr2=0x7f45440b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:51.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 -- 172.21.15.115:0/3586998684 shutdown_connections 2023-12-01T01:51:51.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.753+0000 7f4548869640 1 -- 172.21.15.115:0/3586998684 wait complete. 2023-12-01T01:51:51.773 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T01:51:51.774 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:51.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.977+0000 7f08c6189640 1 Processor -- start 2023-12-01T01:51:51.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.977+0000 7f08c6189640 1 -- start start 2023-12-01T01:51:51.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08c6189640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08c00c8ed0 0x7f08c00cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08c6189640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f08c006c2f0 0x7f08c006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08c6189640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c006f2f0 con 0x7f08c00aded0 2023-12-01T01:51:51.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08c6189640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c006f5d0 con 0x7f08c006c2f0 2023-12-01T01:51:51.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08c6189640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00c9350 con 0x7f08c00c8ed0 2023-12-01T01:51:51.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08bffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f08c006c2f0 0x7f08c006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08bffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f08c006c2f0 0x7f08c006fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59070/0 (socket says 172.21.15.115:59070) 2023-12-01T01:51:51.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08bffff640 1 -- 172.21.15.115:0/3820257909 learned_addr learned my addr 172.21.15.115:0/3820257909 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:51.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08bffff640 1 -- 172.21.15.115:0/3820257909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08c00c8ed0 msgr2=0x7f08c00cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:51.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08bffff640 1 --2- 172.21.15.115:0/3820257909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08c00c8ed0 0x7f08c00cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08bffff640 1 -- 172.21.15.115:0/3820257909 >> v1:172.21.15.142:6789/0 conn(0x7f08c00aded0 legacy=0x7f08c006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:51.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.979+0000 7f08bffff640 1 -- 172.21.15.115:0/3820257909 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00c9630 con 0x7f08c006c2f0 2023-12-01T01:51:51.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.980+0000 7f08bffff640 1 --2- 172.21.15.115:0/3820257909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c006c2f0 0x7f08c006fa60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f08b800af60 tx=0x7f08b802cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78c718d2cb7a8fe server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:51.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.981+0000 7f08be7fc640 1 -- 172.21.15.115:0/3820257909 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b802d8f0 con 0x7f08c006c2f0 2023-12-01T01:51:51.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.981+0000 7f08be7fc640 1 -- 172.21.15.115:0/3820257909 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b802da50 con 0x7f08c006c2f0 2023-12-01T01:51:51.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.981+0000 7f08be7fc640 1 -- 172.21.15.115:0/3820257909 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b802dde0 con 0x7f08c006c2f0 2023-12-01T01:51:51.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.981+0000 7f08c6189640 1 -- 172.21.15.115:0/3820257909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c006c2f0 msgr2=0x7f08c006fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.981+0000 7f08c6189640 1 --2- 172.21.15.115:0/3820257909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c006c2f0 0x7f08c006fa60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f08b800af60 tx=0x7f08b802cf50 comp rx=0 tx=0).stop 2023-12-01T01:51:51.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.981+0000 7f08c6189640 1 -- 172.21.15.115:0/3820257909 shutdown_connections 2023-12-01T01:51:51.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.981+0000 7f08c6189640 1 --2- 172.21.15.115:0/3820257909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c006c2f0 0x7f08c006fa60 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.981+0000 7f08c6189640 1 --2- 172.21.15.115:0/3820257909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08c00c8ed0 0x7f08c00cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.981+0000 7f08c6189640 1 -- 172.21.15.115:0/3820257909 >> 172.21.15.115:0/3820257909 conn(0x7f08c00ce880 msgr2=0x7f08c00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:51.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.982+0000 7f08c6189640 1 -- 172.21.15.115:0/3820257909 shutdown_connections 2023-12-01T01:51:51.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.982+0000 7f08c6189640 1 -- 172.21.15.115:0/3820257909 wait complete. 2023-12-01T01:51:51.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.983+0000 7f08c6189640 1 Processor -- start 2023-12-01T01:51:51.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.983+0000 7f08c6189640 1 -- start start 2023-12-01T01:51:51.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.984+0000 7f08c6189640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08c006c2f0 0x7f08c015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.984+0000 7f08c6189640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08c00aded0 0x7f08c01449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.984+0000 7f08c6189640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c00c8ed0 0x7f08c00c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.984+0000 7f08c6189640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0144f20 con 0x7f08c00c8ed0 2023-12-01T01:51:51.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.984+0000 7f08c6189640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c015ad30 con 0x7f08c00aded0 2023-12-01T01:51:51.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.984+0000 7f08c6189640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c015b010 con 0x7f08c006c2f0 2023-12-01T01:51:51.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08bffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c00c8ed0 0x7f08c00c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08bffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c00c8ed0 0x7f08c00c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59082/0 (socket says 172.21.15.115:59082) 2023-12-01T01:51:51.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08bffff640 1 -- 172.21.15.115:0/534403906 learned_addr learned my addr 172.21.15.115:0/534403906 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:51.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08bffff640 1 -- 172.21.15.115:0/534403906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08c006c2f0 msgr2=0x7f08c015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08bf7fe640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08c006c2f0 0x7f08c015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08beffd640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08c00aded0 0x7f08c01449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08bffff640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08c006c2f0 0x7f08c015a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08bffff640 1 -- 172.21.15.115:0/534403906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08c00aded0 msgr2=0x7f08c01449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08bffff640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08c00aded0 0x7f08c01449e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08bffff640 1 -- 172.21.15.115:0/534403906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b800ac40 con 0x7f08c00c8ed0 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.985+0000 7f08bffff640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c00c8ed0 0x7f08c00c8400 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f08b800af60 tx=0x7f08b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.986+0000 7f08a7fff640 1 -- 172.21.15.115:0/534403906 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8009cd0 con 0x7f08c00c8ed0 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.986+0000 7f08a7fff640 1 -- 172.21.15.115:0/534403906 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b8009e30 con 0x7f08c00c8ed0 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.986+0000 7f08a7fff640 1 -- 172.21.15.115:0/534403906 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b80406b0 con 0x7f08c00c8ed0 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.986+0000 7f08c6189640 1 -- 172.21.15.115:0/534403906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c015b2f0 con 0x7f08c00c8ed0 2023-12-01T01:51:51.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.986+0000 7f08c6189640 1 -- 172.21.15.115:0/534403906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c015b8b0 con 0x7f08c00c8ed0 2023-12-01T01:51:51.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.987+0000 7f08c6189640 1 -- 172.21.15.115:0/534403906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c006b2d0 con 0x7f08c00c8ed0 2023-12-01T01:51:51.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.988+0000 7f08a7fff640 1 -- 172.21.15.115:0/534403906 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 39) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f08b8040810 con 0x7f08c00c8ed0 2023-12-01T01:51:51.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.989+0000 7f08a7fff640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f08ac06a280 0x7f08ac06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:51.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.989+0000 7f08a7fff640 1 -- 172.21.15.115:0/534403906 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f08b80abb80 con 0x7f08c00c8ed0 2023-12-01T01:51:51.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.989+0000 7f08bf7fe640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f08ac06a280 0x7f08ac06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:51.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.990+0000 7f08bf7fe640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f08ac06a280 0x7f08ac06c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f08b00015d0 tx=0x7f08b0009040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:51.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:51.993+0000 7f08a7fff640 1 -- 172.21.15.115:0/534403906 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f08b8077210 con 0x7f08c00c8ed0 2023-12-01T01:51:52.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.234+0000 7f08c6189640 1 -- 172.21.15.115:0/534403906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08c004e890 con 0x7f08c00c8ed0 2023-12-01T01:51:52.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.235+0000 7f08a7fff640 1 -- 172.21.15.115:0/534403906 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f08b8044090 con 0x7f08c00c8ed0 2023-12-01T01:51:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:52.237 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1284508688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1284508688},{"type":"v1","addr":"172.21.15.142:6833","nonce":1284508688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1196894153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1196894153},{"type":"v1","addr":"172.21.15.142:6835","nonce":1196894153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6432,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/120786293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":120786293},{"type":"v1","addr":"172.21.15.115:6836","nonce":120786293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:51:24.174363+0000","modified":"2023-12-01T01:51:27.708969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6435},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6435":{"gid":6435,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3394497964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3394497964},{"type":"v1","addr":"172.21.15.115:6837","nonce":3394497964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-01T01:51:52.237 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 32 2023-12-01T01:51:52.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.237+0000 7f08c6189640 1 -- 172.21.15.115:0/534403906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f08ac06a280 msgr2=0x7f08ac06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:52.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.237+0000 7f08c6189640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f08ac06a280 0x7f08ac06c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f08b00015d0 tx=0x7f08b0009040 comp rx=0 tx=0).stop 2023-12-01T01:51:52.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.237+0000 7f08c6189640 1 -- 172.21.15.115:0/534403906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c00c8ed0 msgr2=0x7f08c00c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:52.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.237+0000 7f08c6189640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c00c8ed0 0x7f08c00c8400 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f08b800af60 tx=0x7f08b8004a40 comp rx=0 tx=0).stop 2023-12-01T01:51:52.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.237+0000 7f08c6189640 1 -- 172.21.15.115:0/534403906 shutdown_connections 2023-12-01T01:51:52.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.237+0000 7f08c6189640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f08ac06a280 0x7f08ac06c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.237+0000 7f08c6189640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08c00c8ed0 0x7f08c00c8400 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.237+0000 7f08c6189640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08c00aded0 0x7f08c01449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.238+0000 7f08c6189640 1 --2- 172.21.15.115:0/534403906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08c006c2f0 0x7f08c015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.238+0000 7f08c6189640 1 -- 172.21.15.115:0/534403906 >> 172.21.15.115:0/534403906 conn(0x7f08c00ce880 msgr2=0x7f08c00cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:52.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.238+0000 7f08c6189640 1 -- 172.21.15.115:0/534403906 shutdown_connections 2023-12-01T01:51:52.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.238+0000 7f08c6189640 1 -- 172.21.15.115:0/534403906 wait complete. 2023-12-01T01:51:52.256 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:51:52.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.457+0000 7f349a981640 1 Processor -- start 2023-12-01T01:51:52.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.457+0000 7f349a981640 1 -- start start 2023-12-01T01:51:52.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.459+0000 7f349a981640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3494159dd0 0x7f34940c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.459+0000 7f349a981640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f349406b6f0 0x7f34940c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.459+0000 7f349a981640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f34940ade80 0x7f34940cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.459+0000 7f349a981640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f349406f8b0 con 0x7f34940ade80 2023-12-01T01:51:52.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.459+0000 7f349a981640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34940adcb0 con 0x7f349406b6f0 2023-12-01T01:51:52.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.459+0000 7f349a981640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34940ad9d0 con 0x7f3494159dd0 2023-12-01T01:51:52.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.459+0000 7f3498f1e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f34940ade80 0x7f34940cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.459+0000 7f3498f1e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f34940ade80 0x7f34940cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59112/0 (socket says 172.21.15.115:59112) 2023-12-01T01:51:52.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.459+0000 7f3498f1e640 1 -- 172.21.15.115:0/1568410311 learned_addr learned my addr 172.21.15.115:0/1568410311 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:52.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.460+0000 7f3493fff640 1 --2- 172.21.15.115:0/1568410311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3494159dd0 0x7f34940c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.460+0000 7f348b7fe640 1 --2- 172.21.15.115:0/1568410311 >> v2:172.21.15.142:3300/0 conn(0x7f349406b6f0 0x7f34940c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.460+0000 7f3498f1e640 1 -- 172.21.15.115:0/1568410311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3494159dd0 msgr2=0x7f34940c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:52.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.460+0000 7f3498f1e640 1 --2- 172.21.15.115:0/1568410311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3494159dd0 0x7f34940c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.460+0000 7f3498f1e640 1 -- 172.21.15.115:0/1568410311 >> v2:172.21.15.142:3300/0 conn(0x7f349406b6f0 msgr2=0x7f34940c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:52.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.460+0000 7f3498f1e640 1 --2- 172.21.15.115:0/1568410311 >> v2:172.21.15.142:3300/0 conn(0x7f349406b6f0 0x7f34940c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.460+0000 7f3498f1e640 1 -- 172.21.15.115:0/1568410311 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34940c9000 con 0x7f34940ade80 2023-12-01T01:51:52.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.461+0000 7f3498f1e640 1 --2- 172.21.15.115:0/1568410311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 0x7f34940cc450 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f34940b1460 tx=0x7f348c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83299c5035067135 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:52.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.461+0000 7f34937fe640 1 -- 172.21.15.115:0/1568410311 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c031740 con 0x7f34940ade80 2023-12-01T01:51:52.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.461+0000 7f34937fe640 1 -- 172.21.15.115:0/1568410311 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c0318a0 con 0x7f34940ade80 2023-12-01T01:51:52.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.461+0000 7f34937fe640 1 -- 172.21.15.115:0/1568410311 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c031c30 con 0x7f34940ade80 2023-12-01T01:51:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.462+0000 7f349a981640 1 -- 172.21.15.115:0/1568410311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 msgr2=0x7f34940cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.462+0000 7f349a981640 1 --2- 172.21.15.115:0/1568410311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 0x7f34940cc450 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f34940b1460 tx=0x7f348c02b7e0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.462+0000 7f349a981640 1 -- 172.21.15.115:0/1568410311 shutdown_connections 2023-12-01T01:51:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.462+0000 7f349a981640 1 --2- 172.21.15.115:0/1568410311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 0x7f34940cc450 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.462+0000 7f349a981640 1 --2- 172.21.15.115:0/1568410311 >> v2:172.21.15.142:3300/0 conn(0x7f349406b6f0 0x7f34940c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.462+0000 7f349a981640 1 --2- 172.21.15.115:0/1568410311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3494159dd0 0x7f34940c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.462+0000 7f349a981640 1 -- 172.21.15.115:0/1568410311 >> 172.21.15.115:0/1568410311 conn(0x7f34940ce880 msgr2=0x7f34940cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.462+0000 7f349a981640 1 -- 172.21.15.115:0/1568410311 shutdown_connections 2023-12-01T01:51:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.462+0000 7f349a981640 1 -- 172.21.15.115:0/1568410311 wait complete. 2023-12-01T01:51:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.464+0000 7f349a981640 1 Processor -- start 2023-12-01T01:51:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.464+0000 7f349a981640 1 -- start start 2023-12-01T01:51:52.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f349a981640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349406b6f0 0x7f349406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f349a981640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 0x7f3494150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f349a981640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494159dd0 0x7f349414fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f349a981640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349415c320 con 0x7f34940ade80 2023-12-01T01:51:52.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f349a981640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349415b710 con 0x7f3494159dd0 2023-12-01T01:51:52.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f349a981640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349415b9f0 con 0x7f349406b6f0 2023-12-01T01:51:52.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f348b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 0x7f3494150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f348b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 0x7f3494150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59124/0 (socket says 172.21.15.115:59124) 2023-12-01T01:51:52.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f348b7fe640 1 -- 172.21.15.115:0/2336311910 learned_addr learned my addr 172.21.15.115:0/2336311910 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:52.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f3493fff640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349406b6f0 0x7f349406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.465+0000 7f348b7fe640 1 -- 172.21.15.115:0/2336311910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349406b6f0 msgr2=0x7f349406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:52.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f348b7fe640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349406b6f0 0x7f349406c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f3498f1e640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494159dd0 0x7f349414fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f348b7fe640 1 -- 172.21.15.115:0/2336311910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494159dd0 msgr2=0x7f349414fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:52.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f348b7fe640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494159dd0 0x7f349414fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f348b7fe640 1 -- 172.21.15.115:0/2336311910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0094d0 con 0x7f34940ade80 2023-12-01T01:51:52.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f3493fff640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349406b6f0 0x7f349406c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:52.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f348b7fe640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 0x7f3494150e80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f347c00c7f0 tx=0x7f347c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:52.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f34917fa640 1 -- 172.21.15.115:0/2336311910 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c004220 con 0x7f34940ade80 2023-12-01T01:51:52.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f34917fa640 1 -- 172.21.15.115:0/2336311910 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c004380 con 0x7f34940ade80 2023-12-01T01:51:52.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f34917fa640 1 -- 172.21.15.115:0/2336311910 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c007860 con 0x7f34940ade80 2023-12-01T01:51:52.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f349a981640 1 -- 172.21.15.115:0/2336311910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349415bcd0 con 0x7f34940ade80 2023-12-01T01:51:52.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.466+0000 7f349a981640 1 -- 172.21.15.115:0/2336311910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349415c7b0 con 0x7f34940ade80 2023-12-01T01:51:52.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.467+0000 7f349a981640 1 -- 172.21.15.115:0/2336311910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34940c97e0 con 0x7f34940ade80 2023-12-01T01:51:52.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.468+0000 7f34917fa640 1 -- 172.21.15.115:0/2336311910 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 39) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f347c007bd0 con 0x7f34940ade80 2023-12-01T01:51:52.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.470+0000 7f34917fa640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f347406a280 0x7f347406c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.470+0000 7f34917fa640 1 -- 172.21.15.115:0/2336311910 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f347c0886e0 con 0x7f34940ade80 2023-12-01T01:51:52.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.470+0000 7f3493fff640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f347406a280 0x7f347406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.471+0000 7f3493fff640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f347406a280 0x7f347406c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3484007eb0 tx=0x7f3484007400 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:52.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.474+0000 7f34917fa640 1 -- 172.21.15.115:0/2336311910 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f347c053db0 con 0x7f34940ade80 2023-12-01T01:51:52.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.701+0000 7f349a981640 1 -- 172.21.15.115:0/2336311910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34940c92b0 con 0x7f34940ade80 2023-12-01T01:51:52.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.702+0000 7f34917fa640 1 -- 172.21.15.115:0/2336311910 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f347c0573d0 con 0x7f34940ade80 2023-12-01T01:51:52.703 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:52.703 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1284508688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1284508688},{"type":"v1","addr":"172.21.15.142:6833","nonce":1284508688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1196894153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1196894153},{"type":"v1","addr":"172.21.15.142:6835","nonce":1196894153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6432,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/120786293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":120786293},{"type":"v1","addr":"172.21.15.115:6836","nonce":120786293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:51:24.174363+0000","modified":"2023-12-01T01:51:27.708969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6435},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6435":{"gid":6435,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3394497964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3394497964},{"type":"v1","addr":"172.21.15.115:6837","nonce":3394497964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-01T01:51:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 32 2023-12-01T01:51:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.704+0000 7f349a981640 1 -- 172.21.15.115:0/2336311910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f347406a280 msgr2=0x7f347406c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.704+0000 7f349a981640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f347406a280 0x7f347406c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3484007eb0 tx=0x7f3484007400 comp rx=0 tx=0).stop 2023-12-01T01:51:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.705+0000 7f349a981640 1 -- 172.21.15.115:0/2336311910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 msgr2=0x7f3494150e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.705+0000 7f349a981640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 0x7f3494150e80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f347c00c7f0 tx=0x7f347c00ccb0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.705+0000 7f349a981640 1 -- 172.21.15.115:0/2336311910 shutdown_connections 2023-12-01T01:51:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.705+0000 7f349a981640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f347406a280 0x7f347406c740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.705+0000 7f349a981640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494159dd0 0x7f349414fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.705+0000 7f349a981640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34940ade80 0x7f3494150e80 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.705+0000 7f349a981640 1 --2- 172.21.15.115:0/2336311910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349406b6f0 0x7f349406c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.705+0000 7f349a981640 1 -- 172.21.15.115:0/2336311910 >> 172.21.15.115:0/2336311910 conn(0x7f34940ce880 msgr2=0x7f34940cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.705+0000 7f349a981640 1 -- 172.21.15.115:0/2336311910 shutdown_connections 2023-12-01T01:51:52.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.705+0000 7f349a981640 1 -- 172.21.15.115:0/2336311910 wait complete. 2023-12-01T01:51:52.725 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:51:52.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.925+0000 7f81f9148640 1 Processor -- start 2023-12-01T01:51:52.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.925+0000 7f81f9148640 1 -- start start 2023-12-01T01:51:52.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.926+0000 7f81f9148640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81f4159dd0 0x7f81f40c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.926+0000 7f81f9148640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81f4070d30 0x7f81f40c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.926+0000 7f81f9148640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f406f8b0 con 0x7f81f406b6f0 2023-12-01T01:51:52.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.926+0000 7f81f9148640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f40adcb0 con 0x7f81f4159dd0 2023-12-01T01:51:52.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.926+0000 7f81f9148640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40ad9d0 con 0x7f81f4070d30 2023-12-01T01:51:52.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.927+0000 7f81f2d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81f4159dd0 0x7f81f40c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.927+0000 7f81f2d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81f4159dd0 0x7f81f40c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59140/0 (socket says 172.21.15.115:59140) 2023-12-01T01:51:52.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.927+0000 7f81f2d9d640 1 -- 172.21.15.115:0/3263636636 learned_addr learned my addr 172.21.15.115:0/3263636636 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:52.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.927+0000 7f81f2d9d640 1 -- 172.21.15.115:0/3263636636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81f4070d30 msgr2=0x7f81f40c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:52.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.927+0000 7f81f2d9d640 1 --2- 172.21.15.115:0/3263636636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81f4070d30 0x7f81f40c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.927+0000 7f81f2d9d640 1 -- 172.21.15.115:0/3263636636 >> v1:172.21.15.142:6789/0 conn(0x7f81f406b6f0 legacy=0x7f81f40cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:52.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.927+0000 7f81f2d9d640 1 -- 172.21.15.115:0/3263636636 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f40c9000 con 0x7f81f4159dd0 2023-12-01T01:51:52.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.928+0000 7f81f2d9d640 1 --2- 172.21.15.115:0/3263636636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4159dd0 0x7f81f40c97e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009870 tx=0x7f81e402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c8530f566c0ecd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:52.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.928+0000 7f81f1d9b640 1 -- 172.21.15.115:0/3263636636 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4031740 con 0x7f81f4159dd0 2023-12-01T01:51:52.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.928+0000 7f81f1d9b640 1 -- 172.21.15.115:0/3263636636 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e40318a0 con 0x7f81f4159dd0 2023-12-01T01:51:52.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.928+0000 7f81f1d9b640 1 -- 172.21.15.115:0/3263636636 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4031c30 con 0x7f81f4159dd0 2023-12-01T01:51:52.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.929+0000 7f81f9148640 1 -- 172.21.15.115:0/3263636636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4159dd0 msgr2=0x7f81f40c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:52.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.929+0000 7f81f9148640 1 --2- 172.21.15.115:0/3263636636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4159dd0 0x7f81f40c97e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009870 tx=0x7f81e402b7e0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.929+0000 7f81f9148640 1 -- 172.21.15.115:0/3263636636 shutdown_connections 2023-12-01T01:51:52.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.929+0000 7f81f9148640 1 --2- 172.21.15.115:0/3263636636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81f4070d30 0x7f81f40c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.929+0000 7f81f9148640 1 --2- 172.21.15.115:0/3263636636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4159dd0 0x7f81f40c97e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.929+0000 7f81f9148640 1 -- 172.21.15.115:0/3263636636 >> 172.21.15.115:0/3263636636 conn(0x7f81f40ce880 msgr2=0x7f81f40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:52.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.929+0000 7f81f9148640 1 -- 172.21.15.115:0/3263636636 shutdown_connections 2023-12-01T01:51:52.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.929+0000 7f81f9148640 1 -- 172.21.15.115:0/3263636636 wait complete. 2023-12-01T01:51:52.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.931+0000 7f81f9148640 1 Processor -- start 2023-12-01T01:51:52.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.931+0000 7f81f9148640 1 -- start start 2023-12-01T01:51:52.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.932+0000 7f81f9148640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f406b6f0 0x7f81f415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.932+0000 7f81f9148640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4070d30 0x7f81f41f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.932+0000 7f81f9148640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81f4159dd0 0x7f81f41e7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.932+0000 7f81f9148640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f415c250 con 0x7f81f4070d30 2023-12-01T01:51:52.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.932+0000 7f81f9148640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f41e75a0 con 0x7f81f406b6f0 2023-12-01T01:51:52.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.932+0000 7f81f9148640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f415bc90 con 0x7f81f4159dd0 2023-12-01T01:51:52.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.932+0000 7f81f259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4070d30 0x7f81f41f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.932+0000 7f81f259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4070d30 0x7f81f41f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59148/0 (socket says 172.21.15.115:59148) 2023-12-01T01:51:52.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.932+0000 7f81f259c640 1 -- 172.21.15.115:0/630973361 learned_addr learned my addr 172.21.15.115:0/630973361 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:52.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81f359e640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81f4159dd0 0x7f81f41e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81f2d9d640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f406b6f0 0x7f81f415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81f259c640 1 -- 172.21.15.115:0/630973361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81f4159dd0 msgr2=0x7f81f41e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:52.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81f259c640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81f4159dd0 0x7f81f41e7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81f259c640 1 -- 172.21.15.115:0/630973361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f406b6f0 msgr2=0x7f81f415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:52.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81f259c640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f406b6f0 0x7f81f415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:52.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81f259c640 1 -- 172.21.15.115:0/630973361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e40094d0 con 0x7f81f4070d30 2023-12-01T01:51:52.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81f2d9d640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f406b6f0 0x7f81f415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:52.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81f259c640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4070d30 0x7f81f41f2f00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f81e800c410 tx=0x7f81e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:52.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81db7fe640 1 -- 172.21.15.115:0/630973361 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e80035f0 con 0x7f81f4070d30 2023-12-01T01:51:52.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81db7fe640 1 -- 172.21.15.115:0/630973361 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e8003750 con 0x7f81f4070d30 2023-12-01T01:51:52.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81db7fe640 1 -- 172.21.15.115:0/630973361 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8003b40 con 0x7f81f4070d30 2023-12-01T01:51:52.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.933+0000 7f81f9148640 1 -- 172.21.15.115:0/630973361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f415bf70 con 0x7f81f4070d30 2023-12-01T01:51:52.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.934+0000 7f81f9148640 1 -- 172.21.15.115:0/630973361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f41f29e0 con 0x7f81f4070d30 2023-12-01T01:51:52.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.935+0000 7f81f9148640 1 -- 172.21.15.115:0/630973361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f404e890 con 0x7f81f4070d30 2023-12-01T01:51:52.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.935+0000 7f81db7fe640 1 -- 172.21.15.115:0/630973361 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 40) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f81e8003ca0 con 0x7f81f4070d30 2023-12-01T01:51:52.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.936+0000 7f81db7fe640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81e006a4f0 0x7f81e006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:52.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.936+0000 7f81db7fe640 1 -- 172.21.15.115:0/630973361 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f81e8087f60 con 0x7f81f4070d30 2023-12-01T01:51:52.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.937+0000 7f81f2d9d640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81e006a4f0 0x7f81e006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:52.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.937+0000 7f81f2d9d640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81e006a4f0 0x7f81e006c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f81e4035000 tx=0x7f81e4008030 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:52.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:52.941+0000 7f81db7fe640 1 -- 172.21.15.115:0/630973361 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f81e8053170 con 0x7f81f4070d30 2023-12-01T01:51:53.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.135+0000 7f81f9148640 1 -- 172.21.15.115:0/630973361 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81f41f2ba0 con 0x7f81f4070d30 2023-12-01T01:51:53.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.137+0000 7f81db7fe640 1 -- 172.21.15.115:0/630973361 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17015 (secure 0 0 0) 0x7f81e8017090 con 0x7f81f4070d30 2023-12-01T01:51:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:51:53.139 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":85,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:51:49.999481+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:51:22.633453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-01T01:51:23.191899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/60448999":"2023-12-02T01:51:49.999425+0000","172.21.15.115:0/1905595030":"2023-12-02T01:51:49.999425+0000","172.21.15.115:0/2838184043":"2023-12-02T01:51:49.999425+0000","172.21.15.115:0/573957418":"2023-12-02T01:51:49.999425+0000","172.21.15.115:0/3305720342":"2023-12-02T01:51:49.999425+0000","172.21.15.115:6824/88317":"2023-12-02T01:51:43.682671+0000","172.21.15.115:6822/88317":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/246605387":"2023-12-02T01:51:49.999425+0000","172.21.15.115:0/3840278706":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/3824153221":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/86374784":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/871551054":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/3669975107":"2023-12-02T01:51:43.682671+0000","172.21.15.115:0/2326628113":"2023-12-02T01:51:43.682671+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 -- 172.21.15.115:0/630973361 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81e006a4f0 msgr2=0x7f81e006c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81e006a4f0 0x7f81e006c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f81e4035000 tx=0x7f81e4008030 comp rx=0 tx=0).stop 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 -- 172.21.15.115:0/630973361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4070d30 msgr2=0x7f81f41f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4070d30 0x7f81f41f2f00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f81e800c410 tx=0x7f81e800c8d0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 -- 172.21.15.115:0/630973361 shutdown_connections 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81e006a4f0 0x7f81e006c9b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81f4159dd0 0x7f81f41e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81f4070d30 0x7f81f41f2f00 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 --2- 172.21.15.115:0/630973361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f406b6f0 0x7f81f415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 -- 172.21.15.115:0/630973361 >> 172.21.15.115:0/630973361 conn(0x7f81f40ce880 msgr2=0x7f81f4158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.140+0000 7f81f9148640 1 -- 172.21.15.115:0/630973361 shutdown_connections 2023-12-01T01:51:53.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.141+0000 7f81f9148640 1 -- 172.21.15.115:0/630973361 wait complete. 2023-12-01T01:51:53.160 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T01:51:53.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.370+0000 7f9d29a4a640 1 Processor -- start 2023-12-01T01:51:53.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.370+0000 7f9d29a4a640 1 -- start start 2023-12-01T01:51:53.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.372+0000 7f9d29a4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d2415aac0 0x7f9d240ae700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.372+0000 7f9d29a4a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9d24070300 0x7f9d24070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.372+0000 7f9d29a4a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2406fe70 con 0x7f9d240c89b0 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.372+0000 7f9d29a4a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d24070cb0 con 0x7f9d24070300 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.372+0000 7f9d29a4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2406f860 con 0x7f9d2415aac0 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.373+0000 7f9d237fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9d24070300 0x7f9d24070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.373+0000 7f9d237fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9d24070300 0x7f9d24070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:50996/0 (socket says 172.21.15.115:50996) 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.373+0000 7f9d237fe640 1 -- 172.21.15.115:0/1844099878 learned_addr learned my addr 172.21.15.115:0/1844099878 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.373+0000 7f9d22ffd640 1 --2- 172.21.15.115:0/1844099878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d2415aac0 0x7f9d240ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.374+0000 7f9d237fe640 1 -- 172.21.15.115:0/1844099878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d2415aac0 msgr2=0x7f9d240ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.374+0000 7f9d237fe640 1 --2- 172.21.15.115:0/1844099878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d2415aac0 0x7f9d240ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.374+0000 7f9d237fe640 1 -- 172.21.15.115:0/1844099878 >> v1:172.21.15.142:6789/0 conn(0x7f9d240c89b0 legacy=0x7f9d2406b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.374+0000 7f9d237fe640 1 -- 172.21.15.115:0/1844099878 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d240aef30 con 0x7f9d24070300 2023-12-01T01:51:53.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.375+0000 7f9d237fe640 1 --2- 172.21.15.115:0/1844099878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d24070300 0x7f9d24070770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009e50 tx=0x7f9d1c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21a0d6bd06616f6d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:53.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.376+0000 7f9d227fc640 1 -- 172.21.15.115:0/1844099878 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c02cb00 con 0x7f9d24070300 2023-12-01T01:51:53.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.376+0000 7f9d227fc640 1 -- 172.21.15.115:0/1844099878 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c02cc60 con 0x7f9d24070300 2023-12-01T01:51:53.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.376+0000 7f9d227fc640 1 -- 172.21.15.115:0/1844099878 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0343f0 con 0x7f9d24070300 2023-12-01T01:51:53.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.376+0000 7f9d29a4a640 1 -- 172.21.15.115:0/1844099878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d24070300 msgr2=0x7f9d24070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.376+0000 7f9d29a4a640 1 --2- 172.21.15.115:0/1844099878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d24070300 0x7f9d24070770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009e50 tx=0x7f9d1c02bf60 comp rx=0 tx=0).stop 2023-12-01T01:51:53.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.376+0000 7f9d29a4a640 1 -- 172.21.15.115:0/1844099878 shutdown_connections 2023-12-01T01:51:53.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.376+0000 7f9d29a4a640 1 --2- 172.21.15.115:0/1844099878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d24070300 0x7f9d24070770 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.376+0000 7f9d29a4a640 1 --2- 172.21.15.115:0/1844099878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d2415aac0 0x7f9d240ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.376+0000 7f9d29a4a640 1 -- 172.21.15.115:0/1844099878 >> 172.21.15.115:0/1844099878 conn(0x7f9d240ce820 msgr2=0x7f9d240cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:53.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.377+0000 7f9d29a4a640 1 -- 172.21.15.115:0/1844099878 shutdown_connections 2023-12-01T01:51:53.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.377+0000 7f9d29a4a640 1 -- 172.21.15.115:0/1844099878 wait complete. 2023-12-01T01:51:53.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.378+0000 7f9d29a4a640 1 Processor -- start 2023-12-01T01:51:53.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.378+0000 7f9d29a4a640 1 -- start start 2023-12-01T01:51:53.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.379+0000 7f9d29a4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d24070300 0x7f9d24148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.379+0000 7f9d29a4a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d240c89b0 0x7f9d24149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.379+0000 7f9d29a4a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d2415aac0 0x7f9d2406b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.379+0000 7f9d29a4a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240c3a20 con 0x7f9d2415aac0 2023-12-01T01:51:53.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.379+0000 7f9d29a4a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24149bc0 con 0x7f9d240c89b0 2023-12-01T01:51:53.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.379+0000 7f9d29a4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d241470d0 con 0x7f9d24070300 2023-12-01T01:51:53.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d2415aac0 0x7f9d2406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d2415aac0 0x7f9d2406b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59150/0 (socket says 172.21.15.115:59150) 2023-12-01T01:51:53.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d237fe640 1 -- 172.21.15.115:0/2500428874 learned_addr learned my addr 172.21.15.115:0/2500428874 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:53.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d22ffd640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d24070300 0x7f9d24148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d1a7fc640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d240c89b0 0x7f9d24149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d237fe640 1 -- 172.21.15.115:0/2500428874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d24070300 msgr2=0x7f9d24148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d237fe640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d24070300 0x7f9d24148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d237fe640 1 -- 172.21.15.115:0/2500428874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d240c89b0 msgr2=0x7f9d24149520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d237fe640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d240c89b0 0x7f9d24149520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d237fe640 1 -- 172.21.15.115:0/2500428874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c009b30 con 0x7f9d2415aac0 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d1a7fc640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d240c89b0 0x7f9d24149520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d22ffd640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d24070300 0x7f9d24148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.380+0000 7f9d237fe640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d2415aac0 0x7f9d2406b250 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009e50 tx=0x7f9d1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.381+0000 7f9d20ff9640 1 -- 172.21.15.115:0/2500428874 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c009510 con 0x7f9d2415aac0 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.381+0000 7f9d20ff9640 1 -- 172.21.15.115:0/2500428874 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c03e450 con 0x7f9d2415aac0 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.381+0000 7f9d20ff9640 1 -- 172.21.15.115:0/2500428874 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c03e800 con 0x7f9d2415aac0 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.381+0000 7f9d29a4a640 1 -- 172.21.15.115:0/2500428874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d241473b0 con 0x7f9d2415aac0 2023-12-01T01:51:53.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.381+0000 7f9d29a4a640 1 -- 172.21.15.115:0/2500428874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d241436b0 con 0x7f9d2415aac0 2023-12-01T01:51:53.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.382+0000 7f9d29a4a640 1 -- 172.21.15.115:0/2500428874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d24146f30 con 0x7f9d2415aac0 2023-12-01T01:51:53.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.383+0000 7f9d20ff9640 1 -- 172.21.15.115:0/2500428874 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 40) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f9d1c03e960 con 0x7f9d2415aac0 2023-12-01T01:51:53.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.384+0000 7f9d20ff9640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d0406a4c0 0x7f9d0406c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.384+0000 7f9d20ff9640 1 -- 172.21.15.115:0/2500428874 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f9d1c0ab820 con 0x7f9d2415aac0 2023-12-01T01:51:53.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.384+0000 7f9d22ffd640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d0406a4c0 0x7f9d0406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.385+0000 7f9d22ffd640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d0406a4c0 0x7f9d0406c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009ac0 tx=0x7f9d14008040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:53.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.388+0000 7f9d20ff9640 1 -- 172.21.15.115:0/2500428874 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9d24146f30 con 0x7f9d2415aac0 2023-12-01T01:51:53.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.614+0000 7f9d29a4a640 1 -- 172.21.15.115:0/2500428874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9d24146f30 con 0x7f9d2415aac0 2023-12-01T01:51:53.617 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T01:51:53.615+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:51:53.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.633+0000 7f9d20ff9640 1 -- 172.21.15.115:0/2500428874 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9d24146f30 con 0x7f9d2415aac0 2023-12-01T01:51:53.635 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T01:51:53.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.635+0000 7f9d29a4a640 1 -- 172.21.15.115:0/2500428874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d0406a4c0 msgr2=0x7f9d0406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d0406a4c0 0x7f9d0406c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009ac0 tx=0x7f9d14008040 comp rx=0 tx=0).stop 2023-12-01T01:51:53.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 -- 172.21.15.115:0/2500428874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d2415aac0 msgr2=0x7f9d2406b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d2415aac0 0x7f9d2406b250 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009e50 tx=0x7f9d1c004a40 comp rx=0 tx=0).stop 2023-12-01T01:51:53.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 -- 172.21.15.115:0/2500428874 shutdown_connections 2023-12-01T01:51:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d0406a4c0 0x7f9d0406c980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d2415aac0 0x7f9d2406b250 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d240c89b0 0x7f9d24149520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 --2- 172.21.15.115:0/2500428874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d24070300 0x7f9d24148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 -- 172.21.15.115:0/2500428874 >> 172.21.15.115:0/2500428874 conn(0x7f9d240ce820 msgr2=0x7f9d24155680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 -- 172.21.15.115:0/2500428874 shutdown_connections 2023-12-01T01:51:53.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.636+0000 7f9d29a4a640 1 -- 172.21.15.115:0/2500428874 wait complete. 2023-12-01T01:51:53.649 INFO:tasks.ceph.mds.c.smithi115.stderr: -3706> 2023-12-01T01:51:29.603+0000 7f0d4578f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:51:53.658 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T01:51:53.718 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:51:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.848+0000 7f8bb0755640 1 Processor -- start 2023-12-01T01:51:53.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.848+0000 7f8bb0755640 1 -- start start 2023-12-01T01:51:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.850+0000 7f8bb0755640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8ba80b5cd0 0x7f8ba80b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.850+0000 7f8bb0755640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba80cfc10 0x7f8ba80d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.850+0000 7f8bb0755640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba80b6890 con 0x7f8ba815cfa0 2023-12-01T01:51:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.850+0000 7f8bb0755640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba80b66a0 con 0x7f8ba80b5cd0 2023-12-01T01:51:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.850+0000 7f8bb0755640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba80d05e0 con 0x7f8ba80cfc10 2023-12-01T01:51:53.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.850+0000 7f8bae4f1640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8ba815cfa0 0x7f8ba80b7930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:49096/0 (socket says 172.21.15.115:49096) 2023-12-01T01:51:53.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.850+0000 7f8bae4f1640 1 -- 172.21.15.115:0/211346058 learned_addr learned my addr 172.21.15.115:0/211346058 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:53.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.850+0000 7f8badcf0640 1 --2- 172.21.15.115:0/211346058 >> v2:172.21.15.142:3300/0 conn(0x7f8ba80b5cd0 0x7f8ba80b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.850+0000 7f8baecf2640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba80cfc10 0x7f8ba80d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.850+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109827046 0 0) 0x7f8ba80b6890 con 0x7f8ba815cfa0 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b90003520 con 0x7f8ba815cfa0 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1270707957 0 0) 0x7f8b90003520 con 0x7f8ba815cfa0 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba80cfc10 msgr2=0x7f8ba80d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba80cfc10 0x7f8ba80d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 >> v2:172.21.15.142:3300/0 conn(0x7f8ba80b5cd0 msgr2=0x7f8ba80b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 --2- 172.21.15.115:0/211346058 >> v2:172.21.15.142:3300/0 conn(0x7f8ba80b5cd0 0x7f8ba80b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba80b6b70 con 0x7f8ba815cfa0 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f8b98001670 con 0x7f8ba815cfa0 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 >> v1:172.21.15.115:6789/0 conn(0x7f8ba815cfa0 legacy=0x7f8ba80b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba80cfc10 0x7f8b90003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b90003f40 0x7f8b900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b90004d00 0x7f8b90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba80d05e0 con 0x7f8b90003f40 2023-12-01T01:51:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9000b9b0 con 0x7f8b90004d00 2023-12-01T01:51:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9000bb70 con 0x7f8ba80cfc10 2023-12-01T01:51:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.851+0000 7f8badcf0640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b90003f40 0x7f8b900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.852+0000 7f8baecf2640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b90004d00 0x7f8b90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.852+0000 7f8badcf0640 1 -- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba80cfc10 msgr2=0x7f8b90003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.852+0000 7f8badcf0640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba80cfc10 0x7f8b90003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.852+0000 7f8badcf0640 1 -- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b90004d00 msgr2=0x7f8b90005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.852+0000 7f8badcf0640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b90004d00 0x7f8b90005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.852+0000 7f8badcf0640 1 -- 172.21.15.115:0/211346058 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b90003520 con 0x7f8b90003f40 2023-12-01T01:51:53.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.852+0000 7f8baecf2640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b90004d00 0x7f8b90005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.853+0000 7f8badcf0640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b90003f40 0x7f8b900047c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009ac0 tx=0x7f8b900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6182ec0c2edb2e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.853+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba402c710 con 0x7f8b90003f40 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.853+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba402c870 con 0x7f8b90003f40 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.853+0000 7f8bad4ef640 1 -- 172.21.15.115:0/211346058 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba402cc00 con 0x7f8b90003f40 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.853+0000 7f8bb0755640 1 -- 172.21.15.115:0/211346058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b90003f40 msgr2=0x7f8b900047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.853+0000 7f8bb0755640 1 --2- 172.21.15.115:0/211346058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b90003f40 0x7f8b900047c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009ac0 tx=0x7f8b900057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.854+0000 7f8baecf2640 1 -- 172.21.15.115:0/211346058 reap_dead start 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.854+0000 7f8bb0755640 1 -- 172.21.15.115:0/211346058 shutdown_connections 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.854+0000 7f8bb0755640 1 -- 172.21.15.115:0/211346058 >> 172.21.15.115:0/211346058 conn(0x7f8ba806a0b0 msgr2=0x7f8ba806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.854+0000 7f8bb0755640 1 -- 172.21.15.115:0/211346058 shutdown_connections 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.854+0000 7f8bb0755640 1 -- 172.21.15.115:0/211346058 wait complete. 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.855+0000 7f8bb0755640 1 Processor -- start 2023-12-01T01:51:53.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.856+0000 7f8bb0755640 1 -- start start 2023-12-01T01:51:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.856+0000 7f8bb0755640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b90003f40 0x7f8ba81f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.856+0000 7f8bb0755640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba8149d20 0x7f8ba8156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8bb0755640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba8156770 0x7f8ba81517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8bb0755640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba815e470 con 0x7f8ba8156770 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8bb0755640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8156bf0 con 0x7f8b90003f40 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8bb0755640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba81f62d0 con 0x7f8ba8149d20 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8baecf2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba8156770 0x7f8ba81517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8baecf2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba8156770 0x7f8ba81517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59188/0 (socket says 172.21.15.115:59188) 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8baecf2640 1 -- 172.21.15.115:0/3733463751 learned_addr learned my addr 172.21.15.115:0/3733463751 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8bae4f1640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b90003f40 0x7f8ba81f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8badcf0640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba8149d20 0x7f8ba8156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8baecf2640 1 -- 172.21.15.115:0/3733463751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba8149d20 msgr2=0x7f8ba8156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8baecf2640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba8149d20 0x7f8ba8156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8baecf2640 1 -- 172.21.15.115:0/3733463751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b90003f40 msgr2=0x7f8ba81f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8baecf2640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b90003f40 0x7f8ba81f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:53.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8baecf2640 1 -- 172.21.15.115:0/3733463751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba815d7f0 con 0x7f8ba8156770 2023-12-01T01:51:53.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.857+0000 7f8badcf0640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba8149d20 0x7f8ba8156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:53.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.858+0000 7f8baecf2640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba8156770 0x7f8ba81517a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8ba806ac40 tx=0x7f8ba000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:53.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.858+0000 7f8b9effd640 1 -- 172.21.15.115:0/3733463751 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000fe30 con 0x7f8ba8156770 2023-12-01T01:51:53.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.858+0000 7f8b9effd640 1 -- 172.21.15.115:0/3733463751 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0011040 con 0x7f8ba8156770 2023-12-01T01:51:53.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.858+0000 7f8b9effd640 1 -- 172.21.15.115:0/3733463751 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0017620 con 0x7f8ba8156770 2023-12-01T01:51:53.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.858+0000 7f8bb0755640 1 -- 172.21.15.115:0/3733463751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba815dad0 con 0x7f8ba8156770 2023-12-01T01:51:53.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.858+0000 7f8bb0755640 1 -- 172.21.15.115:0/3733463751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba815e090 con 0x7f8ba8156770 2023-12-01T01:51:53.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.859+0000 7f8bb0755640 1 -- 172.21.15.115:0/3733463751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba815e5b0 con 0x7f8ba8156770 2023-12-01T01:51:53.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.860+0000 7f8b9effd640 1 -- 172.21.15.115:0/3733463751 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 40) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f8ba00203b0 con 0x7f8ba8156770 2023-12-01T01:51:53.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.861+0000 7f8b9effd640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b9406a4c0 0x7f8b9406c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:53.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.861+0000 7f8b9effd640 1 -- 172.21.15.115:0/3733463751 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f8ba0088e20 con 0x7f8ba8156770 2023-12-01T01:51:53.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.861+0000 7f8bae4f1640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b9406a4c0 0x7f8b9406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:53.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.862+0000 7f8bae4f1640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b9406a4c0 0x7f8b9406c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8b98001800 tx=0x7f8b98001630 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:53.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:53.865+0000 7f8b9effd640 1 -- 172.21.15.115:0/3733463751 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8ba815e5b0 con 0x7f8ba8156770 2023-12-01T01:51:54.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.094+0000 7f8bb0755640 1 -- 172.21.15.115:0/3733463751 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8ba815e5b0 con 0x7f8ba8156770 2023-12-01T01:51:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.639+0000 7f8b9effd640 1 -- 172.21.15.115:0/3733463751 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8ba815e5b0 con 0x7f8ba8156770 2023-12-01T01:51:54.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.642+0000 7f8bb0755640 1 -- 172.21.15.115:0/3733463751 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b9406a4c0 msgr2=0x7f8b9406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.642+0000 7f8bb0755640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b9406a4c0 0x7f8b9406c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8b98001800 tx=0x7f8b98001630 comp rx=0 tx=0).stop 2023-12-01T01:51:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.642+0000 7f8bb0755640 1 -- 172.21.15.115:0/3733463751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba8156770 msgr2=0x7f8ba81517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.642+0000 7f8bb0755640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba8156770 0x7f8ba81517a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8ba806ac40 tx=0x7f8ba000f4a0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.642+0000 7f8bb0755640 1 -- 172.21.15.115:0/3733463751 shutdown_connections 2023-12-01T01:51:54.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.642+0000 7f8bb0755640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b9406a4c0 0x7f8b9406c980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.642+0000 7f8bb0755640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ba8149d20 0x7f8ba8156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.642+0000 7f8bb0755640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ba8156770 0x7f8ba81517a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.643+0000 7f8bb0755640 1 --2- 172.21.15.115:0/3733463751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b90003f40 0x7f8ba81f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.643+0000 7f8bb0755640 1 -- 172.21.15.115:0/3733463751 >> 172.21.15.115:0/3733463751 conn(0x7f8ba806a0b0 msgr2=0x7f8ba80b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:54.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.643+0000 7f8bb0755640 1 -- 172.21.15.115:0/3733463751 shutdown_connections 2023-12-01T01:51:54.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.643+0000 7f8bb0755640 1 -- 172.21.15.115:0/3733463751 wait complete. 2023-12-01T01:51:54.664 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T01:51:54.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.884+0000 7fd11239b640 1 Processor -- start 2023-12-01T01:51:54.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.885+0000 7fd11239b640 1 -- start start 2023-12-01T01:51:54.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.886+0000 7fd11239b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c0c9770 0x7fd10c1648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:54.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.886+0000 7fd11239b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c15e440 con 0x7fd10c0af370 2023-12-01T01:51:54.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.886+0000 7fd11239b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c15d550 con 0x7fd10c1599e0 2023-12-01T01:51:54.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.886+0000 7fd11239b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c15d830 con 0x7fd10c0c9770 2023-12-01T01:51:54.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.886+0000 7fd110b98640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd10c0af370 0x7fd10c0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:49102/0 (socket says 172.21.15.115:49102) 2023-12-01T01:51:54.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.886+0000 7fd110b98640 1 -- 172.21.15.115:0/3924328067 learned_addr learned my addr 172.21.15.115:0/3924328067 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:54.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd111b9a640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c0c9770 0x7fd10c1648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:54.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 788832877 0 0) 0x7fd10c15e440 con 0x7fd10c0af370 2023-12-01T01:51:54.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f8003520 con 0x7fd10c0af370 2023-12-01T01:51:54.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3004078849 0 0) 0x7fd0f8003520 con 0x7fd10c0af370 2023-12-01T01:51:54.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c0c9770 msgr2=0x7fd10c1648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:54.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c0c9770 0x7fd10c1648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 >> v1:172.21.15.142:6789/0 conn(0x7fd10c1599e0 legacy=0x7fd10c159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:54.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c15db10 con 0x7fd10c0af370 2023-12-01T01:51:54.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd0fc001670 con 0x7fd10c0af370 2023-12-01T01:51:54.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 >> v1:172.21.15.115:6789/0 conn(0x7fd10c0af370 legacy=0x7fd10c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:54.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0f80039a0 0x7fd0f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:54.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c0c9770 0x7fd0f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:54.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.887+0000 7fd103fff640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f8004d00 0x7fd0f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:54.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.888+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c15d830 con 0x7fd0f80039a0 2023-12-01T01:51:54.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.888+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f800ba60 con 0x7fd0f8004d00 2023-12-01T01:51:54.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.888+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f800de30 con 0x7fd10c0c9770 2023-12-01T01:51:54.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.888+0000 7fd111399640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0f80039a0 0x7fd0f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:54.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.888+0000 7fd111b9a640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f8004d00 0x7fd0f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:54.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.888+0000 7fd111399640 1 -- 172.21.15.115:0/3924328067 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c0c9770 msgr2=0x7fd0f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:54.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.888+0000 7fd111399640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c0c9770 0x7fd0f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.888+0000 7fd111399640 1 -- 172.21.15.115:0/3924328067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f8004d00 msgr2=0x7fd0f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:54.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.888+0000 7fd111399640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f8004d00 0x7fd0f8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.888+0000 7fd111399640 1 -- 172.21.15.115:0/3924328067 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f8003520 con 0x7fd0f80039a0 2023-12-01T01:51:54.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.889+0000 7fd111399640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0f80039a0 0x7fd0f8003dc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fd108001720 tx=0x7fd0f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fea426679b2fdb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:54.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.889+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1080052d0 con 0x7fd0f80039a0 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.889+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10802e050 con 0x7fd0f80039a0 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.889+0000 7fd103fff640 1 -- 172.21.15.115:0/3924328067 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1080035f0 con 0x7fd0f80039a0 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.890+0000 7fd11239b640 1 -- 172.21.15.115:0/3924328067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0f80039a0 msgr2=0x7fd0f8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.890+0000 7fd11239b640 1 --2- 172.21.15.115:0/3924328067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd0f80039a0 0x7fd0f8003dc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fd108001720 tx=0x7fd0f80057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.890+0000 7fd111b9a640 1 -- 172.21.15.115:0/3924328067 reap_dead start 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.890+0000 7fd11239b640 1 -- 172.21.15.115:0/3924328067 shutdown_connections 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.890+0000 7fd11239b640 1 -- 172.21.15.115:0/3924328067 >> 172.21.15.115:0/3924328067 conn(0x7fd10c065960 msgr2=0x7fd10c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.890+0000 7fd11239b640 1 -- 172.21.15.115:0/3924328067 shutdown_connections 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.890+0000 7fd11239b640 1 -- 172.21.15.115:0/3924328067 wait complete. 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.892+0000 7fd11239b640 1 Processor -- start 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.893+0000 7fd11239b640 1 -- start start 2023-12-01T01:51:54.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.893+0000 7fd11239b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f80039a0 0x7fd10c0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.893+0000 7fd11239b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd10c153900 0x7fd10c0c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.893+0000 7fd11239b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c1450b0 0x7fd10c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.893+0000 7fd11239b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c144c20 con 0x7fd10c153900 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd11239b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0c45d0 con 0x7fd0f80039a0 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd11239b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0c8040 con 0x7fd10c1450b0 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd110b98640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd10c153900 0x7fd10c0c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd110b98640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd10c153900 0x7fd10c0c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59216/0 (socket says 172.21.15.115:59216) 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd110b98640 1 -- 172.21.15.115:0/901308264 learned_addr learned my addr 172.21.15.115:0/901308264 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd110b98640 1 -- 172.21.15.115:0/901308264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c1450b0 msgr2=0x7fd10c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd110b98640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c1450b0 0x7fd10c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd110b98640 1 -- 172.21.15.115:0/901308264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f80039a0 msgr2=0x7fd10c0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd110b98640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f80039a0 0x7fd10c0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd110b98640 1 -- 172.21.15.115:0/901308264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c0c8320 con 0x7fd10c153900 2023-12-01T01:51:54.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.894+0000 7fd110b98640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd10c153900 0x7fd10c0c4090 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c650 tx=0x7fd0fc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:54.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.895+0000 7fd101ffb640 1 -- 172.21.15.115:0/901308264 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0072d0 con 0x7fd10c153900 2023-12-01T01:51:54.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.895+0000 7fd101ffb640 1 -- 172.21.15.115:0/901308264 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0fc007450 con 0x7fd10c153900 2023-12-01T01:51:54.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.895+0000 7fd101ffb640 1 -- 172.21.15.115:0/901308264 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0047e0 con 0x7fd10c153900 2023-12-01T01:51:54.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.895+0000 7fd11239b640 1 -- 172.21.15.115:0/901308264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c0c8600 con 0x7fd10c153900 2023-12-01T01:51:54.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.895+0000 7fd11239b640 1 -- 172.21.15.115:0/901308264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c154000 con 0x7fd10c153900 2023-12-01T01:51:54.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.896+0000 7fd11239b640 1 -- 172.21.15.115:0/901308264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c144a60 con 0x7fd10c153900 2023-12-01T01:51:54.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.897+0000 7fd101ffb640 1 -- 172.21.15.115:0/901308264 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd0fc004960 con 0x7fd10c153900 2023-12-01T01:51:54.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.898+0000 7fd101ffb640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd0f006a580 0x7fd0f006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:54.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.898+0000 7fd101ffb640 1 -- 172.21.15.115:0/901308264 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fd0fc088070 con 0x7fd10c153900 2023-12-01T01:51:54.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.898+0000 7fd111399640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd0f006a580 0x7fd0f006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:54.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.899+0000 7fd111399640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd0f006a580 0x7fd0f006ca40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd108033000 tx=0x7fd10802f000 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:54.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:54.902+0000 7fd101ffb640 1 -- 172.21.15.115:0/901308264 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd0fc0bac80 con 0x7fd10c153900 2023-12-01T01:51:55.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.094+0000 7fd11239b640 1 -- 172.21.15.115:0/901308264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd10c0cbfc0 con 0x7fd10c153900 2023-12-01T01:51:55.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.798+0000 7fd101ffb640 1 -- 172.21.15.115:0/901308264 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd0fc018020 con 0x7fd10c153900 2023-12-01T01:51:55.800 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T01:51:55.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.802+0000 7fd11239b640 1 -- 172.21.15.115:0/901308264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd0f006a580 msgr2=0x7fd0f006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:55.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.802+0000 7fd11239b640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd0f006a580 0x7fd0f006ca40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd108033000 tx=0x7fd10802f000 comp rx=0 tx=0).stop 2023-12-01T01:51:55.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.802+0000 7fd11239b640 1 -- 172.21.15.115:0/901308264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd10c153900 msgr2=0x7fd10c0c4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:55.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.802+0000 7fd11239b640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd10c153900 0x7fd10c0c4090 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c650 tx=0x7fd0fc00cb10 comp rx=0 tx=0).stop 2023-12-01T01:51:55.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.803+0000 7fd11239b640 1 -- 172.21.15.115:0/901308264 shutdown_connections 2023-12-01T01:51:55.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.803+0000 7fd11239b640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd0f006a580 0x7fd0f006ca40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:55.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.803+0000 7fd11239b640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd10c1450b0 0x7fd10c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:55.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.803+0000 7fd11239b640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd10c153900 0x7fd10c0c4090 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:55.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.803+0000 7fd11239b640 1 --2- 172.21.15.115:0/901308264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd0f80039a0 0x7fd10c0c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:55.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.803+0000 7fd11239b640 1 -- 172.21.15.115:0/901308264 >> 172.21.15.115:0/901308264 conn(0x7fd10c065960 msgr2=0x7fd10c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:55.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.803+0000 7fd11239b640 1 -- 172.21.15.115:0/901308264 shutdown_connections 2023-12-01T01:51:55.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:55.803+0000 7fd11239b640 1 -- 172.21.15.115:0/901308264 wait complete. 2023-12-01T01:51:55.824 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T01:51:56.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.038+0000 7f0e9c937640 1 Processor -- start 2023-12-01T01:51:56.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.038+0000 7f0e9c937640 1 -- start start 2023-12-01T01:51:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.039+0000 7f0e9c937640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0e9815d910 0x7f0e9815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.039+0000 7f0e9c937640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980d30e0 0x7f0e980d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.039+0000 7f0e9c937640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e980b2300 con 0x7f0e980cdaa0 2023-12-01T01:51:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.039+0000 7f0e9c937640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e981612b0 con 0x7f0e9815d910 2023-12-01T01:51:56.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.039+0000 7f0e9c937640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e981616d0 con 0x7f0e980d30e0 2023-12-01T01:51:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.040+0000 7f0e977fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0e9815d910 0x7f0e9815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.040+0000 7f0e977fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0e9815d910 0x7f0e9815dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:51054/0 (socket says 172.21.15.115:51054) 2023-12-01T01:51:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.040+0000 7f0e97fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980d30e0 0x7f0e980d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:56.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.040+0000 7f0e977fe640 1 -- 172.21.15.115:0/678324756 learned_addr learned my addr 172.21.15.115:0/678324756 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.041+0000 7f0e977fe640 1 -- 172.21.15.115:0/678324756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980d30e0 msgr2=0x7f0e980d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.041+0000 7f0e977fe640 1 --2- 172.21.15.115:0/678324756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980d30e0 0x7f0e980d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.041+0000 7f0e977fe640 1 -- 172.21.15.115:0/678324756 >> v1:172.21.15.142:6789/0 conn(0x7f0e980cdaa0 legacy=0x7f0e980d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:56.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.041+0000 7f0e977fe640 1 -- 172.21.15.115:0/678324756 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e980ce260 con 0x7f0e9815d910 2023-12-01T01:51:56.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.043+0000 7f0e977fe640 1 --2- 172.21.15.115:0/678324756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e9815d910 0x7f0e9815dcf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0e880098a0 tx=0x7f0e8802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36fab6e6b011cf78 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.043+0000 7f0e96ffd640 1 -- 172.21.15.115:0/678324756 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88031640 con 0x7f0e9815d910 2023-12-01T01:51:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.043+0000 7f0e96ffd640 1 -- 172.21.15.115:0/678324756 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e880317a0 con 0x7f0e9815d910 2023-12-01T01:51:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.043+0000 7f0e96ffd640 1 -- 172.21.15.115:0/678324756 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88031b30 con 0x7f0e9815d910 2023-12-01T01:51:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.044+0000 7f0e9c937640 1 -- 172.21.15.115:0/678324756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e9815d910 msgr2=0x7f0e9815dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.044+0000 7f0e9c937640 1 --2- 172.21.15.115:0/678324756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e9815d910 0x7f0e9815dcf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0e880098a0 tx=0x7f0e8802b8f0 comp rx=0 tx=0).stop 2023-12-01T01:51:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.044+0000 7f0e9c937640 1 -- 172.21.15.115:0/678324756 shutdown_connections 2023-12-01T01:51:56.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.044+0000 7f0e9c937640 1 --2- 172.21.15.115:0/678324756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980d30e0 0x7f0e980d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.044+0000 7f0e9c937640 1 --2- 172.21.15.115:0/678324756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e9815d910 0x7f0e9815dcf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.044+0000 7f0e9c937640 1 -- 172.21.15.115:0/678324756 >> 172.21.15.115:0/678324756 conn(0x7f0e9806a0b0 msgr2=0x7f0e9806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.044+0000 7f0e9c937640 1 -- 172.21.15.115:0/678324756 shutdown_connections 2023-12-01T01:51:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.044+0000 7f0e9c937640 1 -- 172.21.15.115:0/678324756 wait complete. 2023-12-01T01:51:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.046+0000 7f0e9c937640 1 Processor -- start 2023-12-01T01:51:56.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.046+0000 7f0e9c937640 1 -- start start 2023-12-01T01:51:56.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.047+0000 7f0e9c937640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cdaa0 0x7f0e981f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.047+0000 7f0e9c937640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980d30e0 0x7f0e980c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.047+0000 7f0e9c937640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815d910 0x7f0e980b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.047+0000 7f0e9c937640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980cab90 con 0x7f0e9815d910 2023-12-01T01:51:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.047+0000 7f0e9c937640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980ca5d0 con 0x7f0e980d30e0 2023-12-01T01:51:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.047+0000 7f0e9c937640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980ca8b0 con 0x7f0e980cdaa0 2023-12-01T01:51:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.047+0000 7f0e97fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815d910 0x7f0e980b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.047+0000 7f0e97fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815d910 0x7f0e980b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59222/0 (socket says 172.21.15.115:59222) 2023-12-01T01:51:56.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.047+0000 7f0e97fff640 1 -- 172.21.15.115:0/642176334 learned_addr learned my addr 172.21.15.115:0/642176334 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e8ffff640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980d30e0 0x7f0e980c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:56.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e977fe640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cdaa0 0x7f0e981f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e97fff640 1 -- 172.21.15.115:0/642176334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cdaa0 msgr2=0x7f0e981f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e97fff640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cdaa0 0x7f0e981f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e97fff640 1 -- 172.21.15.115:0/642176334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980d30e0 msgr2=0x7f0e980c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e97fff640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980d30e0 0x7f0e980c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e97fff640 1 -- 172.21.15.115:0/642176334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880094d0 con 0x7f0e9815d910 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e977fe640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cdaa0 0x7f0e981f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e8ffff640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980d30e0 0x7f0e980c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e97fff640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815d910 0x7f0e980b37d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000efc0 tx=0x7f0e9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e94ff9640 1 -- 172.21.15.115:0/642176334 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90009ca0 con 0x7f0e9815d910 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e94ff9640 1 -- 172.21.15.115:0/642176334 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9000f040 con 0x7f0e9815d910 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e94ff9640 1 -- 172.21.15.115:0/642176334 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e900146c0 con 0x7f0e9815d910 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e9c937640 1 -- 172.21.15.115:0/642176334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e980c6610 con 0x7f0e9815d910 2023-12-01T01:51:56.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.048+0000 7f0e9c937640 1 -- 172.21.15.115:0/642176334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e98150190 con 0x7f0e9815d910 2023-12-01T01:51:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.049+0000 7f0e9c937640 1 -- 172.21.15.115:0/642176334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e980c6b30 con 0x7f0e9815d910 2023-12-01T01:51:56.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.050+0000 7f0e94ff9640 1 -- 172.21.15.115:0/642176334 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0e90014820 con 0x7f0e9815d910 2023-12-01T01:51:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.051+0000 7f0e94ff9640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a550 0x7f0e7806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.051+0000 7f0e94ff9640 1 -- 172.21.15.115:0/642176334 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5959+0+0 (secure 0 0 0) 0x7f0e90089ca0 con 0x7f0e9815d910 2023-12-01T01:51:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.052+0000 7f0e977fe640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a550 0x7f0e7806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:56.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.052+0000 7f0e977fe640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a550 0x7f0e7806ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0e88036000 tx=0x7f0e8803c040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:56.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.056+0000 7f0e94ff9640 1 -- 172.21.15.115:0/642176334 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0e980c6b30 con 0x7f0e9815d910 2023-12-01T01:51:56.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.259+0000 7f0e9c937640 1 -- 172.21.15.115:0/642176334 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e980c6b30 con 0x7f0e9815d910 2023-12-01T01:51:56.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.820+0000 7f0e94ff9640 1 -- 172.21.15.115:0/642176334 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0e980c6b30 con 0x7f0e9815d910 2023-12-01T01:51:56.822 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T01:51:56.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 -- 172.21.15.115:0/642176334 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a550 msgr2=0x7f0e7806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:56.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a550 0x7f0e7806ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0e88036000 tx=0x7f0e8803c040 comp rx=0 tx=0).stop 2023-12-01T01:51:56.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 -- 172.21.15.115:0/642176334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815d910 msgr2=0x7f0e980b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:56.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815d910 0x7f0e980b37d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000efc0 tx=0x7f0e9000c490 comp rx=0 tx=0).stop 2023-12-01T01:51:56.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 -- 172.21.15.115:0/642176334 shutdown_connections 2023-12-01T01:51:56.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0e7806a550 0x7f0e7806ca10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:56.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e9815d910 0x7f0e980b37d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:56.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e980d30e0 0x7f0e980c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:56.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 --2- 172.21.15.115:0/642176334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e980cdaa0 0x7f0e981f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:56.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 -- 172.21.15.115:0/642176334 >> 172.21.15.115:0/642176334 conn(0x7f0e9806a0b0 msgr2=0x7f0e9806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:56.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.824+0000 7f0e9c937640 1 -- 172.21.15.115:0/642176334 shutdown_connections 2023-12-01T01:51:56.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:56.825+0000 7f0e9c937640 1 -- 172.21.15.115:0/642176334 wait complete. 2023-12-01T01:51:56.848 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T01:51:57.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.070+0000 7fe3ff59e640 1 Processor -- start 2023-12-01T01:51:57.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.070+0000 7fe3ff59e640 1 -- start start 2023-12-01T01:51:57.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.071+0000 7fe3ff59e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe40015ac00 0x7fe4000ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.071+0000 7fe3ff59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4000aede0 0x7fe4000cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.071+0000 7fe3ff59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe4000b04c0 0x7fe4000cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.071+0000 7fe3ff59e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4000b12b0 con 0x7fe4000b04c0 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.071+0000 7fe3ff59e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40015afe0 con 0x7fe40015ac00 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.071+0000 7fe3ff59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b0a10 con 0x7fe4000aede0 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.071+0000 7fe3fed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe4000b04c0 0x7fe4000cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.071+0000 7fe3fed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe4000b04c0 0x7fe4000cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59238/0 (socket says 172.21.15.115:59238) 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.071+0000 7fe3fed9d640 1 -- 172.21.15.115:0/3261247533 learned_addr learned my addr 172.21.15.115:0/3261247533 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.072+0000 7fe3fe59c640 1 --2- 172.21.15.115:0/3261247533 >> v2:172.21.15.142:3300/0 conn(0x7fe40015ac00 0x7fe4000ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.072+0000 7fe3f7fff640 1 --2- 172.21.15.115:0/3261247533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4000aede0 0x7fe4000cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.072+0000 7fe3fed9d640 1 -- 172.21.15.115:0/3261247533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4000aede0 msgr2=0x7fe4000cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.072+0000 7fe3fed9d640 1 --2- 172.21.15.115:0/3261247533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4000aede0 0x7fe4000cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.072+0000 7fe3fed9d640 1 -- 172.21.15.115:0/3261247533 >> v2:172.21.15.142:3300/0 conn(0x7fe40015ac00 msgr2=0x7fe4000ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.072+0000 7fe3fed9d640 1 --2- 172.21.15.115:0/3261247533 >> v2:172.21.15.142:3300/0 conn(0x7fe40015ac00 0x7fe4000ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.072+0000 7fe3fed9d640 1 -- 172.21.15.115:0/3261247533 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000b0cf0 con 0x7fe4000b04c0 2023-12-01T01:51:57.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.073+0000 7fe3fed9d640 1 --2- 172.21.15.115:0/3261247533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000b04c0 0x7fe4000cb630 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe400066750 tx=0x7fe3f802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de288123f1ce34c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.073+0000 7fe3fdd9b640 1 -- 172.21.15.115:0/3261247533 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f802c9e0 con 0x7fe4000b04c0 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.073+0000 7fe3fdd9b640 1 -- 172.21.15.115:0/3261247533 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f802cb40 con 0x7fe4000b04c0 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.073+0000 7fe3fdd9b640 1 -- 172.21.15.115:0/3261247533 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80343f0 con 0x7fe4000b04c0 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.074+0000 7fe3ff59e640 1 -- 172.21.15.115:0/3261247533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000b04c0 msgr2=0x7fe4000cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.074+0000 7fe3ff59e640 1 --2- 172.21.15.115:0/3261247533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000b04c0 0x7fe4000cb630 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe400066750 tx=0x7fe3f802be40 comp rx=0 tx=0).stop 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.074+0000 7fe3ff59e640 1 -- 172.21.15.115:0/3261247533 shutdown_connections 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.074+0000 7fe3ff59e640 1 --2- 172.21.15.115:0/3261247533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000b04c0 0x7fe4000cb630 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.074+0000 7fe3ff59e640 1 --2- 172.21.15.115:0/3261247533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4000aede0 0x7fe4000cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.074+0000 7fe3ff59e640 1 --2- 172.21.15.115:0/3261247533 >> v2:172.21.15.142:3300/0 conn(0x7fe40015ac00 0x7fe4000ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.074+0000 7fe3ff59e640 1 -- 172.21.15.115:0/3261247533 >> 172.21.15.115:0/3261247533 conn(0x7fe400065960 msgr2=0x7fe400065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.074+0000 7fe3ff59e640 1 -- 172.21.15.115:0/3261247533 shutdown_connections 2023-12-01T01:51:57.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.074+0000 7fe3ff59e640 1 -- 172.21.15.115:0/3261247533 wait complete. 2023-12-01T01:51:57.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.076+0000 7fe3ff59e640 1 Processor -- start 2023-12-01T01:51:57.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.076+0000 7fe3ff59e640 1 -- start start 2023-12-01T01:51:57.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3ff59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000aede0 0x7fe4001e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3ff59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4000b04c0 0x7fe40015d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3ff59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40015ac00 0x7fe40015caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3ff59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001e6870 con 0x7fe4000aede0 2023-12-01T01:51:57.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3ff59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001e6b50 con 0x7fe40015ac00 2023-12-01T01:51:57.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3ff59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40015c010 con 0x7fe4000b04c0 2023-12-01T01:51:57.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3fe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000aede0 0x7fe4001e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3fe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000aede0 0x7fe4001e5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59240/0 (socket says 172.21.15.115:59240) 2023-12-01T01:51:57.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3fe59c640 1 -- 172.21.15.115:0/161835374 learned_addr learned my addr 172.21.15.115:0/161835374 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:57.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3fed9d640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40015ac00 0x7fe40015caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.077+0000 7fe3f7fff640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4000b04c0 0x7fe40015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3fe59c640 1 -- 172.21.15.115:0/161835374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4000b04c0 msgr2=0x7fe40015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3fe59c640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4000b04c0 0x7fe40015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3fe59c640 1 -- 172.21.15.115:0/161835374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40015ac00 msgr2=0x7fe40015caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3fe59c640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40015ac00 0x7fe40015caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3fe59c640 1 -- 172.21.15.115:0/161835374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8009b30 con 0x7fe4000aede0 2023-12-01T01:51:57.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3fed9d640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40015ac00 0x7fe40015caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:57.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3fe59c640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000aede0 0x7fe4001e5f20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000c7f0 tx=0x7fe3f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:57.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3f6ffd640 1 -- 172.21.15.115:0/161835374 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0004220 con 0x7fe4000aede0 2023-12-01T01:51:57.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3f6ffd640 1 -- 172.21.15.115:0/161835374 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0004380 con 0x7fe4000aede0 2023-12-01T01:51:57.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3ff59e640 1 -- 172.21.15.115:0/161835374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40015c2f0 con 0x7fe4000aede0 2023-12-01T01:51:57.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3f6ffd640 1 -- 172.21.15.115:0/161835374 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0007840 con 0x7fe4000aede0 2023-12-01T01:51:57.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.078+0000 7fe3ff59e640 1 -- 172.21.15.115:0/161835374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4001e6460 con 0x7fe4000aede0 2023-12-01T01:51:57.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.080+0000 7fe3ff59e640 1 -- 172.21.15.115:0/161835374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40015be70 con 0x7fe4000aede0 2023-12-01T01:51:57.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.080+0000 7fe3f6ffd640 1 -- 172.21.15.115:0/161835374 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe3f0007bb0 con 0x7fe4000aede0 2023-12-01T01:51:57.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.081+0000 7fe3f6ffd640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e006a550 0x7fe3e006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.081+0000 7fe3f6ffd640 1 -- 172.21.15.115:0/161835374 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5545+0+0 (secure 0 0 0) 0x7fe3f0088a50 con 0x7fe4000aede0 2023-12-01T01:51:57.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.082+0000 7fe3f7fff640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e006a550 0x7fe3e006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.082+0000 7fe3f7fff640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e006a550 0x7fe3e006ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe4001f5510 tx=0x7fe3e8008040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:57.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.086+0000 7fe3f6ffd640 1 -- 172.21.15.115:0/161835374 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe40015be70 con 0x7fe4000aede0 2023-12-01T01:51:57.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.277+0000 7fe3ff59e640 1 -- 172.21.15.115:0/161835374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe40015c810 con 0x7fe4000aede0 2023-12-01T01:51:57.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.286+0000 7fe3f6ffd640 1 -- 172.21.15.115:0/161835374 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe40015c810 con 0x7fe4000aede0 2023-12-01T01:51:57.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.288+0000 7fe3ff59e640 1 -- 172.21.15.115:0/161835374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e006a550 msgr2=0x7fe3e006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e006a550 0x7fe3e006ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe4001f5510 tx=0x7fe3e8008040 comp rx=0 tx=0).stop 2023-12-01T01:51:57.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 -- 172.21.15.115:0/161835374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000aede0 msgr2=0x7fe4001e5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000aede0 0x7fe4001e5f20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000c7f0 tx=0x7fe3f000ccb0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 -- 172.21.15.115:0/161835374 shutdown_connections 2023-12-01T01:51:57.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe3e006a550 0x7fe3e006ca10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4000b04c0 0x7fe40015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40015ac00 0x7fe40015caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 --2- 172.21.15.115:0/161835374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4000aede0 0x7fe4001e5f20 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 -- 172.21.15.115:0/161835374 >> 172.21.15.115:0/161835374 conn(0x7fe400065960 msgr2=0x7fe4000cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:57.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 -- 172.21.15.115:0/161835374 shutdown_connections 2023-12-01T01:51:57.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.289+0000 7fe3ff59e640 1 -- 172.21.15.115:0/161835374 wait complete. 2023-12-01T01:51:57.310 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T01:51:57.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.521+0000 7f2912082640 1 Processor -- start 2023-12-01T01:51:57.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.521+0000 7f2912082640 1 -- start start 2023-12-01T01:51:57.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.522+0000 7f2912082640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f290c15b0e0 0x7f290c164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.522+0000 7f2912082640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290c0b2f30 con 0x7f290c142870 2023-12-01T01:51:57.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.522+0000 7f2912082640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290c0b2320 con 0x7f290c0cd1e0 2023-12-01T01:51:57.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.522+0000 7f2912082640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0b3ae0 con 0x7f290c15b0e0 2023-12-01T01:51:57.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.522+0000 7f290b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f290c142870 0x7f290c0ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:49104/0 (socket says 172.21.15.115:49104) 2023-12-01T01:51:57.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.522+0000 7f290b7fe640 1 -- 172.21.15.115:0/2629457768 learned_addr learned my addr 172.21.15.115:0/2629457768 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406448340 0 0) 0x7f290c0b2f30 con 0x7f290c142870 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28f8003520 con 0x7f290c142870 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290bfff640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f290c15b0e0 0x7f290c164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1595110342 0 0) 0x7f28f8003520 con 0x7f290c142870 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f290c15b0e0 msgr2=0x7f290c164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f290c15b0e0 0x7f290c164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 >> v1:172.21.15.142:6789/0 conn(0x7f290c0cd1e0 legacy=0x7f290c0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c0b2600 con 0x7f290c142870 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f28fc001670 con 0x7f290c142870 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 >> v1:172.21.15.115:6789/0 conn(0x7f290c142870 legacy=0x7f290c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f80039a0 0x7f28f8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f80047c0 0x7f28f8004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28f8005140 0x7f28f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0b3ae0 con 0x7f28f8005140 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f800bec0 con 0x7f28f80039a0 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.523+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f800c080 con 0x7f28f80047c0 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.524+0000 7f290bfff640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28f8005140 0x7f28f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.524+0000 7f290b7fe640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f80039a0 0x7f28f8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.524+0000 7f2902ffd640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f80047c0 0x7f28f8004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.524+0000 7f290bfff640 1 -- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f80047c0 msgr2=0x7f28f8004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.524+0000 7f290bfff640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f80047c0 0x7f28f8004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.524+0000 7f290bfff640 1 -- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f80039a0 msgr2=0x7f28f8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.524+0000 7f290bfff640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f80039a0 0x7f28f8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.524+0000 7f290bfff640 1 -- 172.21.15.115:0/2629457768 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f8003520 con 0x7f28f8005140 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.524+0000 7f2902ffd640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f80047c0 0x7f28f8004c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:57.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.524+0000 7f290b7fe640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f80039a0 0x7f28f8003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:57.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.525+0000 7f290bfff640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28f8005140 0x7f28f800b980 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2904007330 tx=0x7f28f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84ed8f45fd3428b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.525+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290402cca0 con 0x7f28f8005140 2023-12-01T01:51:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.525+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290402ce00 con 0x7f28f8005140 2023-12-01T01:51:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.525+0000 7f290affd640 1 -- 172.21.15.115:0/2629457768 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904034660 con 0x7f28f8005140 2023-12-01T01:51:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.526+0000 7f2912082640 1 -- 172.21.15.115:0/2629457768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28f8005140 msgr2=0x7f28f800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.526+0000 7f2912082640 1 --2- 172.21.15.115:0/2629457768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28f8005140 0x7f28f800b980 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2904007330 tx=0x7f28f80057f0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.526+0000 7f290bfff640 1 -- 172.21.15.115:0/2629457768 reap_dead start 2023-12-01T01:51:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.526+0000 7f2912082640 1 -- 172.21.15.115:0/2629457768 shutdown_connections 2023-12-01T01:51:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.526+0000 7f2912082640 1 -- 172.21.15.115:0/2629457768 >> 172.21.15.115:0/2629457768 conn(0x7f290c065960 msgr2=0x7f290c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.526+0000 7f2912082640 1 -- 172.21.15.115:0/2629457768 shutdown_connections 2023-12-01T01:51:57.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.526+0000 7f2912082640 1 -- 172.21.15.115:0/2629457768 wait complete. 2023-12-01T01:51:57.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.528+0000 7f2912082640 1 Processor -- start 2023-12-01T01:51:57.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.528+0000 7f2912082640 1 -- start start 2023-12-01T01:51:57.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f2912082640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f8005140 0x7f290c13cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f2912082640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f290c14a050 0x7f290c14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f2912082640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f290c0c3b60 0x7f290c0c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f2912082640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0c4550 con 0x7f290c0c3b60 2023-12-01T01:51:57.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f2912082640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0c47e0 con 0x7f290c14a050 2023-12-01T01:51:57.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f2912082640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0c4ac0 con 0x7f28f8005140 2023-12-01T01:51:57.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f290bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f290c0c3b60 0x7f290c0c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f290bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f290c0c3b60 0x7f290c0c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59260/0 (socket says 172.21.15.115:59260) 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f290bfff640 1 -- 172.21.15.115:0/4204386479 learned_addr learned my addr 172.21.15.115:0/4204386479 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f290b7fe640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f8005140 0x7f290c13cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.529+0000 7f2902ffd640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f290c14a050 0x7f290c14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f290bfff640 1 -- 172.21.15.115:0/4204386479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f8005140 msgr2=0x7f290c13cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f290bfff640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f8005140 0x7f290c13cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f290bfff640 1 -- 172.21.15.115:0/4204386479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f290c14a050 msgr2=0x7f290c14a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f290bfff640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f290c14a050 0x7f290c14a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f290bfff640 1 -- 172.21.15.115:0/4204386479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c0c4da0 con 0x7f290c0c3b60 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f290b7fe640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f8005140 0x7f290c13cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f2902ffd640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f290c14a050 0x7f290c14a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:57.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f290bfff640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f290c0c3b60 0x7f290c0c4010 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2904007330 tx=0x7f2904004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:57.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f2908ff9640 1 -- 172.21.15.115:0/4204386479 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290403e640 con 0x7f290c0c3b60 2023-12-01T01:51:57.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f2908ff9640 1 -- 172.21.15.115:0/4204386479 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290403e7a0 con 0x7f290c0c3b60 2023-12-01T01:51:57.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f2908ff9640 1 -- 172.21.15.115:0/4204386479 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290403eb30 con 0x7f290c0c3b60 2023-12-01T01:51:57.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f2912082640 1 -- 172.21.15.115:0/4204386479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290c0c5080 con 0x7f290c0c3b60 2023-12-01T01:51:57.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.530+0000 7f2912082640 1 -- 172.21.15.115:0/4204386479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290c13d2a0 con 0x7f290c0c3b60 2023-12-01T01:51:57.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.532+0000 7f2912082640 1 -- 172.21.15.115:0/4204386479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290c0c55a0 con 0x7f290c0c3b60 2023-12-01T01:51:57.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.532+0000 7f2908ff9640 1 -- 172.21.15.115:0/4204386479 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f2904034d50 con 0x7f290c0c3b60 2023-12-01T01:51:57.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.533+0000 7f2908ff9640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ec06a510 0x7f28ec06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.533+0000 7f2908ff9640 1 -- 172.21.15.115:0/4204386479 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f29040aac60 con 0x7f290c0c3b60 2023-12-01T01:51:57.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.534+0000 7f290b7fe640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ec06a510 0x7f28ec06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.534+0000 7f290b7fe640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ec06a510 0x7f28ec06c9d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0079f0 tx=0x7f28fc009040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:57.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.538+0000 7f2908ff9640 1 -- 172.21.15.115:0/4204386479 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f290c0c55a0 con 0x7f290c0c3b60 2023-12-01T01:51:57.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.728+0000 7f2912082640 1 -- 172.21.15.115:0/4204386479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f290c0c55a0 con 0x7f290c0c3b60 2023-12-01T01:51:57.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.737+0000 7f2908ff9640 1 -- 172.21.15.115:0/4204386479 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v26) v1 ==== 120+0+0 (secure 0 0 0) 0x7f290c0c55a0 con 0x7f290c0c3b60 2023-12-01T01:51:57.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.739+0000 7f2912082640 1 -- 172.21.15.115:0/4204386479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ec06a510 msgr2=0x7f28ec06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.739+0000 7f2912082640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ec06a510 0x7f28ec06c9d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0079f0 tx=0x7f28fc009040 comp rx=0 tx=0).stop 2023-12-01T01:51:57.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.739+0000 7f2912082640 1 -- 172.21.15.115:0/4204386479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f290c0c3b60 msgr2=0x7f290c0c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.739+0000 7f2912082640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f290c0c3b60 0x7f290c0c4010 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2904007330 tx=0x7f2904004ab0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.739+0000 7f2912082640 1 -- 172.21.15.115:0/4204386479 shutdown_connections 2023-12-01T01:51:57.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.739+0000 7f2912082640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f28ec06a510 0x7f28ec06c9d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.739+0000 7f2912082640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f290c0c3b60 0x7f290c0c4010 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.739+0000 7f2912082640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f290c14a050 0x7f290c14a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.739+0000 7f2912082640 1 --2- 172.21.15.115:0/4204386479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f8005140 0x7f290c13cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.739+0000 7f2912082640 1 -- 172.21.15.115:0/4204386479 >> 172.21.15.115:0/4204386479 conn(0x7f290c065960 msgr2=0x7f290c13f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:57.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.740+0000 7f2912082640 1 -- 172.21.15.115:0/4204386479 shutdown_connections 2023-12-01T01:51:57.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.740+0000 7f2912082640 1 -- 172.21.15.115:0/4204386479 wait complete. 2023-12-01T01:51:57.760 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T01:51:57.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.968+0000 7f9b23c2c640 1 Processor -- start 2023-12-01T01:51:57.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.968+0000 7f9b23c2c640 1 -- start start 2023-12-01T01:51:57.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.969+0000 7f9b23c2c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9b1c142870 0x7f9b1c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.969+0000 7f9b23c2c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9b1c0cd010 0x7f9b1c0b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.969+0000 7f9b23c2c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1c0b2f30 con 0x7f9b1c0b1c50 2023-12-01T01:51:57.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.969+0000 7f9b23c2c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c0b3ae0 con 0x7f9b1c142870 2023-12-01T01:51:57.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.969+0000 7f9b23c2c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c0cd4a0 con 0x7f9b1c0cd010 2023-12-01T01:51:57.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.970+0000 7f9b219c8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9b1c142870 0x7f9b1c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.970+0000 7f9b221c9640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9b1c0b1c50 0x7f9b1c15d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54956/0 (socket says 172.21.15.115:54956) 2023-12-01T01:51:57.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.970+0000 7f9b221c9640 1 -- 172.21.15.115:0/940435830 learned_addr learned my addr 172.21.15.115:0/940435830 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:51:57.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.970+0000 7f9b211c7640 1 --2- 172.21.15.115:0/940435830 >> v2:172.21.15.142:3300/0 conn(0x7f9b1c0cd010 0x7f9b1c0b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.970+0000 7f9b219c8640 1 -- 172.21.15.115:0/940435830 >> v2:172.21.15.142:3300/0 conn(0x7f9b1c0cd010 msgr2=0x7f9b1c0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:57.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.970+0000 7f9b219c8640 1 --2- 172.21.15.115:0/940435830 >> v2:172.21.15.142:3300/0 conn(0x7f9b1c0cd010 0x7f9b1c0b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.970+0000 7f9b219c8640 1 -- 172.21.15.115:0/940435830 >> v1:172.21.15.115:6789/0 conn(0x7f9b1c0b1c50 legacy=0x7f9b1c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.970+0000 7f9b219c8640 1 -- 172.21.15.115:0/940435830 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c0b20c0 con 0x7f9b1c142870 2023-12-01T01:51:57.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.970+0000 7f9b209c6640 1 -- 172.21.15.115:0/940435830 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604004509 0 0) 0x7f9b1c0b2f30 con 0x7f9b1c0b1c50 2023-12-01T01:51:57.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.971+0000 7f9b219c8640 1 --2- 172.21.15.115:0/940435830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c142870 0x7f9b1c0ccad0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0cf7f0 tx=0x7f9b1802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51a920ff270362d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:57.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.971+0000 7f9b209c6640 1 -- 172.21.15.115:0/940435830 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18031830 con 0x7f9b1c142870 2023-12-01T01:51:57.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.971+0000 7f9b209c6640 1 -- 172.21.15.115:0/940435830 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b18031990 con 0x7f9b1c142870 2023-12-01T01:51:57.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.971+0000 7f9b209c6640 1 -- 172.21.15.115:0/940435830 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18031d20 con 0x7f9b1c142870 2023-12-01T01:51:57.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.972+0000 7f9b23c2c640 1 -- 172.21.15.115:0/940435830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c142870 msgr2=0x7f9b1c0ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.972+0000 7f9b23c2c640 1 --2- 172.21.15.115:0/940435830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c142870 0x7f9b1c0ccad0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0cf7f0 tx=0x7f9b1802b8d0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.972+0000 7f9b23c2c640 1 -- 172.21.15.115:0/940435830 shutdown_connections 2023-12-01T01:51:57.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.972+0000 7f9b23c2c640 1 --2- 172.21.15.115:0/940435830 >> v2:172.21.15.142:3300/0 conn(0x7f9b1c0cd010 0x7f9b1c0b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.972+0000 7f9b23c2c640 1 --2- 172.21.15.115:0/940435830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c142870 0x7f9b1c0ccad0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.972+0000 7f9b23c2c640 1 -- 172.21.15.115:0/940435830 >> 172.21.15.115:0/940435830 conn(0x7f9b1c065960 msgr2=0x7f9b1c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:57.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.972+0000 7f9b23c2c640 1 -- 172.21.15.115:0/940435830 shutdown_connections 2023-12-01T01:51:57.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.972+0000 7f9b23c2c640 1 -- 172.21.15.115:0/940435830 wait complete. 2023-12-01T01:51:57.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.974+0000 7f9b23c2c640 1 Processor -- start 2023-12-01T01:51:57.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.974+0000 7f9b23c2c640 1 -- start start 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.975+0000 7f9b23c2c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0b1c50 0x7f9b1c0c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.975+0000 7f9b23c2c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c0cd010 0x7f9b1c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.975+0000 7f9b23c2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c142870 0x7f9b1c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.975+0000 7f9b23c2c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c0b0b80 con 0x7f9b1c0cd010 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.975+0000 7f9b23c2c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c0c8ab0 con 0x7f9b1c0b1c50 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.975+0000 7f9b23c2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c0cba00 con 0x7f9b1c142870 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.975+0000 7f9b211c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c0cd010 0x7f9b1c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.975+0000 7f9b211c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c0cd010 0x7f9b1c15d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34484/0 (socket says 172.21.15.115:34484) 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.975+0000 7f9b211c7640 1 -- 172.21.15.115:0/2236961229 learned_addr learned my addr 172.21.15.115:0/2236961229 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b221c9640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c142870 0x7f9b1c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b219c8640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0b1c50 0x7f9b1c0c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b211c7640 1 -- 172.21.15.115:0/2236961229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c142870 msgr2=0x7f9b1c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b211c7640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c142870 0x7f9b1c15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b211c7640 1 -- 172.21.15.115:0/2236961229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0b1c50 msgr2=0x7f9b1c0c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b211c7640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0b1c50 0x7f9b1c0c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b211c7640 1 -- 172.21.15.115:0/2236961229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b18009400 con 0x7f9b1c0cd010 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b219c8640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0b1c50 0x7f9b1c0c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b221c9640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c142870 0x7f9b1c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b211c7640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c0cd010 0x7f9b1c15d2b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c7f0 tx=0x7f9b0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b127fc640 1 -- 172.21.15.115:0/2236961229 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c004220 con 0x7f9b1c0cd010 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b127fc640 1 -- 172.21.15.115:0/2236961229 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c0043a0 con 0x7f9b1c0cd010 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b127fc640 1 -- 172.21.15.115:0/2236961229 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c003850 con 0x7f9b1c0cd010 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.976+0000 7f9b23c2c640 1 -- 172.21.15.115:0/2236961229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c0c9980 con 0x7f9b1c0cd010 2023-12-01T01:51:57.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.977+0000 7f9b23c2c640 1 -- 172.21.15.115:0/2236961229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c0c8210 con 0x7f9b1c0cd010 2023-12-01T01:51:57.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.978+0000 7f9b23c2c640 1 -- 172.21.15.115:0/2236961229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1c04e890 con 0x7f9b1c0cd010 2023-12-01T01:51:57.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.979+0000 7f9b127fc640 1 -- 172.21.15.115:0/2236961229 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9b0c003bc0 con 0x7f9b1c0cd010 2023-12-01T01:51:57.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.979+0000 7f9b127fc640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b0006a550 0x7f9b0006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:57.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.980+0000 7f9b127fc640 1 -- 172.21.15.115:0/2236961229 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f9b0c088ae0 con 0x7f9b1c0cd010 2023-12-01T01:51:57.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.980+0000 7f9b219c8640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b0006a550 0x7f9b0006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:57.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.980+0000 7f9b219c8640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b0006a550 0x7f9b0006ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9b18036000 tx=0x7f9b1803c040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:57.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:57.984+0000 7f9b127fc640 1 -- 172.21.15.115:0/2236961229 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9b0c053eb0 con 0x7f9b1c0cd010 2023-12-01T01:51:58.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.172+0000 7f9b23c2c640 1 -- 172.21.15.115:0/2236961229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f9b1c06b6d0 con 0x7f9b1c0cd010 2023-12-01T01:51:58.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.182+0000 7f9b127fc640 1 -- 172.21.15.115:0/2236961229 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v27) v1 ==== 115+0+0 (secure 0 0 0) 0x7f9b0c01e020 con 0x7f9b1c0cd010 2023-12-01T01:51:58.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 -- 172.21.15.115:0/2236961229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b0006a550 msgr2=0x7f9b0006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b0006a550 0x7f9b0006ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9b18036000 tx=0x7f9b1803c040 comp rx=0 tx=0).stop 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 -- 172.21.15.115:0/2236961229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c0cd010 msgr2=0x7f9b1c15d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c0cd010 0x7f9b1c15d2b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c7f0 tx=0x7f9b0c00ccb0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 -- 172.21.15.115:0/2236961229 shutdown_connections 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b0006a550 0x7f9b0006ca10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b1c142870 0x7f9b1c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b1c0cd010 0x7f9b1c15d2b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 --2- 172.21.15.115:0/2236961229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b1c0b1c50 0x7f9b1c0c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 -- 172.21.15.115:0/2236961229 >> 172.21.15.115:0/2236961229 conn(0x7f9b1c065960 msgr2=0x7f9b1c15a200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 -- 172.21.15.115:0/2236961229 shutdown_connections 2023-12-01T01:51:58.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.185+0000 7f9b23c2c640 1 -- 172.21.15.115:0/2236961229 wait complete. 2023-12-01T01:51:58.206 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-12-01T01:51:58.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.416+0000 7f8b40c17640 1 Processor -- start 2023-12-01T01:51:58.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.417+0000 7f8b40c17640 1 -- start start 2023-12-01T01:51:58.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.418+0000 7f8b40c17640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8b3c15d220 0x7f8b3c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:58.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.418+0000 7f8b40c17640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b3c0ad2b0 0x7f8b3c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:58.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.418+0000 7f8b40c17640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3c15d600 con 0x7f8b3c0ade80 2023-12-01T01:51:58.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.418+0000 7f8b40c17640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c06cc10 con 0x7f8b3c15d220 2023-12-01T01:51:58.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.418+0000 7f8b40c17640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c06c720 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.419+0000 7f8b33fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b3c0ad2b0 0x7f8b3c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:58.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.419+0000 7f8b3a59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8b3c15d220 0x7f8b3c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:58.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.419+0000 7f8b3a59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8b3c15d220 0x7f8b3c0cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:34930/0 (socket says 172.21.15.115:34930) 2023-12-01T01:51:58.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.419+0000 7f8b3a59c640 1 -- 172.21.15.115:0/4054433665 learned_addr learned my addr 172.21.15.115:0/4054433665 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:58.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.420+0000 7f8b33fff640 1 -- 172.21.15.115:0/4054433665 >> v2:172.21.15.142:3300/0 conn(0x7f8b3c15d220 msgr2=0x7f8b3c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:58.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.420+0000 7f8b33fff640 1 --2- 172.21.15.115:0/4054433665 >> v2:172.21.15.142:3300/0 conn(0x7f8b3c15d220 0x7f8b3c0cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.420+0000 7f8b33fff640 1 -- 172.21.15.115:0/4054433665 >> v1:172.21.15.142:6789/0 conn(0x7f8b3c0ade80 legacy=0x7f8b3c06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:58.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.420+0000 7f8b33fff640 1 -- 172.21.15.115:0/4054433665 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c06c4b0 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.420+0000 7f8b3a59c640 1 --2- 172.21.15.115:0/4054433665 >> v2:172.21.15.142:3300/0 conn(0x7f8b3c15d220 0x7f8b3c0cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:51:58.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.421+0000 7f8b33fff640 1 --2- 172.21.15.115:0/4054433665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b3c0ad2b0 0x7f8b3c0ccab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b240097a0 tx=0x7f8b2402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d528470139fe7b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:58.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.422+0000 7f8b39d9b640 1 -- 172.21.15.115:0/4054433665 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2402e070 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.422+0000 7f8b39d9b640 1 -- 172.21.15.115:0/4054433665 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b240315a0 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.422+0000 7f8b39d9b640 1 -- 172.21.15.115:0/4054433665 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24031930 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.423+0000 7f8b40c17640 1 -- 172.21.15.115:0/4054433665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b3c0ad2b0 msgr2=0x7f8b3c0ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:58.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.423+0000 7f8b40c17640 1 --2- 172.21.15.115:0/4054433665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b3c0ad2b0 0x7f8b3c0ccab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b240097a0 tx=0x7f8b2402b840 comp rx=0 tx=0).stop 2023-12-01T01:51:58.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.423+0000 7f8b40c17640 1 -- 172.21.15.115:0/4054433665 shutdown_connections 2023-12-01T01:51:58.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.423+0000 7f8b40c17640 1 --2- 172.21.15.115:0/4054433665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b3c0ad2b0 0x7f8b3c0ccab0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.423+0000 7f8b40c17640 1 --2- 172.21.15.115:0/4054433665 >> v2:172.21.15.142:3300/0 conn(0x7f8b3c15d220 0x7f8b3c0cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.423+0000 7f8b40c17640 1 -- 172.21.15.115:0/4054433665 >> 172.21.15.115:0/4054433665 conn(0x7f8b3c0ce880 msgr2=0x7f8b3c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:58.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.423+0000 7f8b40c17640 1 -- 172.21.15.115:0/4054433665 shutdown_connections 2023-12-01T01:51:58.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.423+0000 7f8b40c17640 1 -- 172.21.15.115:0/4054433665 wait complete. 2023-12-01T01:51:58.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.425+0000 7f8b40c17640 1 Processor -- start 2023-12-01T01:51:58.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.426+0000 7f8b40c17640 1 -- start start 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.427+0000 7f8b40c17640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3c0ad2b0 0x7f8b3c157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.427+0000 7f8b40c17640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b3c0ade80 0x7f8b3c0c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.427+0000 7f8b40c17640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b3c15d220 0x7f8b3c14f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.427+0000 7f8b40c17640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c1589c0 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.427+0000 7f8b40c17640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0cbd60 con 0x7f8b3c0ade80 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.427+0000 7f8b40c17640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0c2a40 con 0x7f8b3c15d220 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.427+0000 7f8b3a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3c0ad2b0 0x7f8b3c157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.427+0000 7f8b3a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3c0ad2b0 0x7f8b3c157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34490/0 (socket says 172.21.15.115:34490) 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.427+0000 7f8b3a59c640 1 -- 172.21.15.115:0/439506565 learned_addr learned my addr 172.21.15.115:0/439506565 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b3a59c640 1 -- 172.21.15.115:0/439506565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b3c15d220 msgr2=0x7f8b3c14f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:58.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b3a59c640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b3c15d220 0x7f8b3c14f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b3a59c640 1 -- 172.21.15.115:0/439506565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b3c0ade80 msgr2=0x7f8b3c0c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:58.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b3a59c640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b3c0ade80 0x7f8b3c0c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b3a59c640 1 -- 172.21.15.115:0/439506565 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c0094d0 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b3a59c640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3c0ad2b0 0x7f8b3c157060 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00e9b0 tx=0x7f8b2c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:58.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b32ffd640 1 -- 172.21.15.115:0/439506565 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00c9a0 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b32ffd640 1 -- 172.21.15.115:0/439506565 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c00cb00 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b40c17640 1 -- 172.21.15.115:0/439506565 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b24009400 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b32ffd640 1 -- 172.21.15.115:0/439506565 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00ce70 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.428+0000 7f8b40c17640 1 -- 172.21.15.115:0/439506565 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c158370 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.430+0000 7f8b40c17640 1 -- 172.21.15.115:0/439506565 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3c04e890 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.430+0000 7f8b32ffd640 1 -- 172.21.15.115:0/439506565 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8b2c013470 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.431+0000 7f8b32ffd640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b1c06a550 0x7f8b1c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:58.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.431+0000 7f8b32ffd640 1 -- 172.21.15.115:0/439506565 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f8b2c088fc0 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.431+0000 7f8b33fff640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b1c06a550 0x7f8b1c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:58.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.432+0000 7f8b33fff640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b1c06a550 0x7f8b1c06ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8b24036000 tx=0x7f8b24030000 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:58.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.436+0000 7f8b32ffd640 1 -- 172.21.15.115:0/439506565 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8b2c054530 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.692+0000 7f8b40c17640 1 -- 172.21.15.115:0/439506565 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f8b3c06a2f0 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.823+0000 7f8b32ffd640 1 -- 172.21.15.115:0/439506565 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v596) v1 ==== 156+0+0 (secure 0 0 0) 0x7f8b2c057b50 con 0x7f8b3c0ad2b0 2023-12-01T01:51:58.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 -- 172.21.15.115:0/439506565 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b1c06a550 msgr2=0x7f8b1c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b1c06a550 0x7f8b1c06ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8b24036000 tx=0x7f8b24030000 comp rx=0 tx=0).stop 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 -- 172.21.15.115:0/439506565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3c0ad2b0 msgr2=0x7f8b3c157060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3c0ad2b0 0x7f8b3c157060 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00e9b0 tx=0x7f8b2c00ee70 comp rx=0 tx=0).stop 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 -- 172.21.15.115:0/439506565 shutdown_connections 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b1c06a550 0x7f8b1c06ca10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b3c15d220 0x7f8b3c14f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b3c0ade80 0x7f8b3c0c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 --2- 172.21.15.115:0/439506565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b3c0ad2b0 0x7f8b3c157060 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 -- 172.21.15.115:0/439506565 >> 172.21.15.115:0/439506565 conn(0x7f8b3c0ce880 msgr2=0x7f8b3c0ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 -- 172.21.15.115:0/439506565 shutdown_connections 2023-12-01T01:51:58.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:58.826+0000 7f8b40c17640 1 -- 172.21.15.115:0/439506565 wait complete. 2023-12-01T01:51:58.847 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-12-01T01:51:58.847 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-12-01T01:51:58.847 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-01T01:51:58.847 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-12-01T01:51:59.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.056+0000 7efe10c69640 1 Processor -- start 2023-12-01T01:51:59.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.056+0000 7efe10c69640 1 -- start start 2023-12-01T01:51:59.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.058+0000 7efe10c69640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe0c15e2f0 0x7efe0c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:59.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.058+0000 7efe10c69640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efe0c06f2f0 0x7efe0c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:59.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.058+0000 7efe10c69640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efe0c0adf10 0x7efe0c0c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:59.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.058+0000 7efe10c69640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0c0add10 con 0x7efe0c06f2f0 2023-12-01T01:51:59.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.058+0000 7efe10c69640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0c0c5760 con 0x7efe0c0adf10 2023-12-01T01:51:59.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.058+0000 7efe10c69640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c15e6d0 con 0x7efe0c15e2f0 2023-12-01T01:51:59.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.058+0000 7efe09d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efe0c06f2f0 0x7efe0c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:59.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.058+0000 7efe09d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efe0c06f2f0 0x7efe0c0caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34500/0 (socket says 172.21.15.115:34500) 2023-12-01T01:51:59.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.058+0000 7efe09d9b640 1 -- 172.21.15.115:0/3119403971 learned_addr learned my addr 172.21.15.115:0/3119403971 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:59.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.059+0000 7efe0ad9d640 1 --2- 172.21.15.115:0/3119403971 >> v2:172.21.15.142:3300/0 conn(0x7efe0c0adf10 0x7efe0c0c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:59.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.059+0000 7efe0a59c640 1 --2- 172.21.15.115:0/3119403971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe0c15e2f0 0x7efe0c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:59.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.059+0000 7efe09d9b640 1 -- 172.21.15.115:0/3119403971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe0c15e2f0 msgr2=0x7efe0c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:59.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.059+0000 7efe09d9b640 1 --2- 172.21.15.115:0/3119403971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe0c15e2f0 0x7efe0c0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.059+0000 7efe09d9b640 1 -- 172.21.15.115:0/3119403971 >> v2:172.21.15.142:3300/0 conn(0x7efe0c0adf10 msgr2=0x7efe0c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:51:59.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.059+0000 7efe09d9b640 1 --2- 172.21.15.115:0/3119403971 >> v2:172.21.15.142:3300/0 conn(0x7efe0c0adf10 0x7efe0c0c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.059+0000 7efe09d9b640 1 -- 172.21.15.115:0/3119403971 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c0cf7b0 con 0x7efe0c06f2f0 2023-12-01T01:51:59.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.060+0000 7efe09d9b640 1 --2- 172.21.15.115:0/3119403971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c06f2f0 0x7efe0c0caba0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0cb2b0 tx=0x7efe0002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d868e950404294 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:59.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.060+0000 7efe0959a640 1 -- 172.21.15.115:0/3119403971 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00031740 con 0x7efe0c06f2f0 2023-12-01T01:51:59.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.060+0000 7efe0959a640 1 -- 172.21.15.115:0/3119403971 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe000318a0 con 0x7efe0c06f2f0 2023-12-01T01:51:59.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.060+0000 7efe0959a640 1 -- 172.21.15.115:0/3119403971 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00031c30 con 0x7efe0c06f2f0 2023-12-01T01:51:59.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.061+0000 7efe10c69640 1 -- 172.21.15.115:0/3119403971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c06f2f0 msgr2=0x7efe0c0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:59.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.061+0000 7efe10c69640 1 --2- 172.21.15.115:0/3119403971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c06f2f0 0x7efe0c0caba0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0cb2b0 tx=0x7efe0002b7e0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.061+0000 7efe10c69640 1 -- 172.21.15.115:0/3119403971 shutdown_connections 2023-12-01T01:51:59.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.061+0000 7efe10c69640 1 --2- 172.21.15.115:0/3119403971 >> v2:172.21.15.142:3300/0 conn(0x7efe0c0adf10 0x7efe0c0c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.061+0000 7efe10c69640 1 --2- 172.21.15.115:0/3119403971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c06f2f0 0x7efe0c0caba0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.061+0000 7efe10c69640 1 --2- 172.21.15.115:0/3119403971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe0c15e2f0 0x7efe0c0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.061+0000 7efe10c69640 1 -- 172.21.15.115:0/3119403971 >> 172.21.15.115:0/3119403971 conn(0x7efe0c0ce880 msgr2=0x7efe0c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:59.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.061+0000 7efe10c69640 1 -- 172.21.15.115:0/3119403971 shutdown_connections 2023-12-01T01:51:59.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.061+0000 7efe10c69640 1 -- 172.21.15.115:0/3119403971 wait complete. 2023-12-01T01:51:59.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.063+0000 7efe10c69640 1 Processor -- start 2023-12-01T01:51:59.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.063+0000 7efe10c69640 1 -- start start 2023-12-01T01:51:59.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.064+0000 7efe10c69640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe0c06f2f0 0x7efe0c1598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:59.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.064+0000 7efe10c69640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c0adf10 0x7efe0c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:59.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.064+0000 7efe10c69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c15e2f0 0x7efe0c0acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:59.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.064+0000 7efe10c69640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c0ce110 con 0x7efe0c15e2f0 2023-12-01T01:51:59.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.064+0000 7efe10c69640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c0cd500 con 0x7efe0c0adf10 2023-12-01T01:51:59.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.064+0000 7efe10c69640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c0cd7e0 con 0x7efe0c06f2f0 2023-12-01T01:51:59.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.064+0000 7efe0ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c15e2f0 0x7efe0c0acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:59.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.064+0000 7efe0ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c15e2f0 0x7efe0c0acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34502/0 (socket says 172.21.15.115:34502) 2023-12-01T01:51:59.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.064+0000 7efe0ad9d640 1 -- 172.21.15.115:0/589879046 learned_addr learned my addr 172.21.15.115:0/589879046 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:51:59.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efe0a59c640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe0c06f2f0 0x7efe0c1598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:59.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efe0ad9d640 1 -- 172.21.15.115:0/589879046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe0c06f2f0 msgr2=0x7efe0c1598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:59.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efe0ad9d640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe0c06f2f0 0x7efe0c1598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efe0ad9d640 1 -- 172.21.15.115:0/589879046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c0adf10 msgr2=0x7efe0c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:59.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efe0ad9d640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c0adf10 0x7efe0c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efe0ad9d640 1 -- 172.21.15.115:0/589879046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000094d0 con 0x7efe0c15e2f0 2023-12-01T01:51:59.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efe0ad9d640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c15e2f0 0x7efe0c0acde0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efe04007d10 tx=0x7efe0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:59.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efdfaffd640 1 -- 172.21.15.115:0/589879046 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04014420 con 0x7efe0c15e2f0 2023-12-01T01:51:59.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efdfaffd640 1 -- 172.21.15.115:0/589879046 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe040145a0 con 0x7efe0c15e2f0 2023-12-01T01:51:59.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efe10c69640 1 -- 172.21.15.115:0/589879046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c0cdac0 con 0x7efe0c15e2f0 2023-12-01T01:51:59.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efdfaffd640 1 -- 172.21.15.115:0/589879046 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04014950 con 0x7efe0c15e2f0 2023-12-01T01:51:59.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.065+0000 7efe10c69640 1 -- 172.21.15.115:0/589879046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c159e10 con 0x7efe0c15e2f0 2023-12-01T01:51:59.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.067+0000 7efe10c69640 1 -- 172.21.15.115:0/589879046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0c06fda0 con 0x7efe0c15e2f0 2023-12-01T01:51:59.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.068+0000 7efdfaffd640 1 -- 172.21.15.115:0/589879046 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7efe04014ab0 con 0x7efe0c15e2f0 2023-12-01T01:51:59.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.068+0000 7efdfaffd640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efdec06a550 0x7efdec06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:51:59.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.069+0000 7efdfaffd640 1 -- 172.21.15.115:0/589879046 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5545+0+0 (secure 0 0 0) 0x7efe04089360 con 0x7efe0c15e2f0 2023-12-01T01:51:59.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.069+0000 7efe0a59c640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efdec06a550 0x7efdec06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:51:59.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.069+0000 7efe0a59c640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efdec06a550 0x7efdec06ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efdfc006fd0 tx=0x7efdfc008040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:51:59.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.073+0000 7efdfaffd640 1 -- 172.21.15.115:0/589879046 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efe04054840 con 0x7efe0c15e2f0 2023-12-01T01:51:59.227 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:51:59.237 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.331+0000 7efe10c69640 1 -- 172.21.15.115:0/589879046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7efe0c159fa0 con 0x7efe0c15e2f0 2023-12-01T01:51:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.828+0000 7efdfaffd640 1 -- 172.21.15.115:0/589879046 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v597) v1 ==== 169+0+0 (secure 0 0 0) 0x7efe04057e60 con 0x7efe0c15e2f0 2023-12-01T01:51:59.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.831+0000 7efe10c69640 1 -- 172.21.15.115:0/589879046 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efdec06a550 msgr2=0x7efdec06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.831+0000 7efe10c69640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efdec06a550 0x7efdec06ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efdfc006fd0 tx=0x7efdfc008040 comp rx=0 tx=0).stop 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.832+0000 7efe10c69640 1 -- 172.21.15.115:0/589879046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c15e2f0 msgr2=0x7efe0c0acde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.832+0000 7efe10c69640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c15e2f0 0x7efe0c0acde0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efe04007d10 tx=0x7efe0400e4b0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.832+0000 7efe10c69640 1 -- 172.21.15.115:0/589879046 shutdown_connections 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.832+0000 7efe10c69640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efdec06a550 0x7efdec06ca10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.832+0000 7efe10c69640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe0c15e2f0 0x7efe0c0acde0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.832+0000 7efe10c69640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c0adf10 0x7efe0c0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.832+0000 7efe10c69640 1 --2- 172.21.15.115:0/589879046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe0c06f2f0 0x7efe0c1598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.832+0000 7efe10c69640 1 -- 172.21.15.115:0/589879046 >> 172.21.15.115:0/589879046 conn(0x7efe0c0ce880 msgr2=0x7efe0c15ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.832+0000 7efe10c69640 1 -- 172.21.15.115:0/589879046 shutdown_connections 2023-12-01T01:51:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:51:59.832+0000 7efe10c69640 1 -- 172.21.15.115:0/589879046 wait complete. 2023-12-01T01:51:59.856 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T01:52:00.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.070+0000 7fcdcafd5640 1 Processor -- start 2023-12-01T01:52:00.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.070+0000 7fcdcafd5640 1 -- start start 2023-12-01T01:52:00.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.071+0000 7fcdcafd5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcdc415cd90 0x7fcdc40cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:00.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.071+0000 7fcdcafd5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc40ce5e0 con 0x7fcdc40b5fa0 2023-12-01T01:52:00.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.071+0000 7fcdcafd5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc40ce300 con 0x7fcdc40cfd60 2023-12-01T01:52:00.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.071+0000 7fcdcafd5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc40d0ec0 con 0x7fcdc415cd90 2023-12-01T01:52:00.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc3fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fcdc40b5fa0 0x7fcdc40cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54962/0 (socket says 172.21.15.115:54962) 2023-12-01T01:52:00.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc3fff640 1 -- 172.21.15.115:0/955193928 learned_addr learned my addr 172.21.15.115:0/955193928 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc8d71640 1 --2- 172.21.15.115:0/955193928 >> v2:172.21.15.142:3300/0 conn(0x7fcdc415cd90 0x7fcdc40cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029703284 0 0) 0x7fcdac001490 con 0x7fcdc40b5fa0 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcda4003520 con 0x7fcdc40b5fa0 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1828409532 0 0) 0x7fcda4003520 con 0x7fcdc40b5fa0 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 >> v2:172.21.15.142:3300/0 conn(0x7fcdc415cd90 msgr2=0x7fcdc40cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc37fe640 1 --2- 172.21.15.115:0/955193928 >> v2:172.21.15.142:3300/0 conn(0x7fcdc415cd90 0x7fcdc40cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 >> v1:172.21.15.142:6789/0 conn(0x7fcdc40cfd60 legacy=0x7fcdc40d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40d10a0 con 0x7fcdc40b5fa0 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.072+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fcdac001b80 con 0x7fcdc40b5fa0 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 >> v1:172.21.15.115:6789/0 conn(0x7fcdc40b5fa0 legacy=0x7fcdc40cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc37fe640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcda4003970 0x7fcda4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc37fe640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcda40047c0 0x7fcda4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:00.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc37fe640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcda400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40d0ec0 con 0x7fcda4005180 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda400c0b0 con 0x7fcda40047c0 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda400c270 con 0x7fcda4003970 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc9572640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcda400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc3fff640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcda40047c0 0x7fcda4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc8d71640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcda4003970 0x7fcda4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc9572640 1 -- 172.21.15.115:0/955193928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcda4003970 msgr2=0x7fcda4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc9572640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcda4003970 0x7fcda4003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc9572640 1 -- 172.21.15.115:0/955193928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcda40047c0 msgr2=0x7fcda4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc9572640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcda40047c0 0x7fcda4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.073+0000 7fcdc9572640 1 -- 172.21.15.115:0/955193928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda4003520 con 0x7fcda4005180 2023-12-01T01:52:00.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.074+0000 7fcdc9572640 1 -- 172.21.15.115:0/955193928 reap_dead start 2023-12-01T01:52:00.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.074+0000 7fcdc9572640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcda400b980 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc003d80 tx=0x7fcda40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209120092d58824e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:00.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.075+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc007220 con 0x7fcda4005180 2023-12-01T01:52:00.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.075+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc02e050 con 0x7fcda4005180 2023-12-01T01:52:00.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.075+0000 7fcdc37fe640 1 -- 172.21.15.115:0/955193928 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0075e0 con 0x7fcda4005180 2023-12-01T01:52:00.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.075+0000 7fcdcafd5640 1 -- 172.21.15.115:0/955193928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 msgr2=0x7fcda400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:00.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.075+0000 7fcdcafd5640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcda400b980 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc003d80 tx=0x7fcda40057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.075+0000 7fcdcafd5640 1 -- 172.21.15.115:0/955193928 shutdown_connections 2023-12-01T01:52:00.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.075+0000 7fcdcafd5640 1 --2- 172.21.15.115:0/955193928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcda400b980 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.075+0000 7fcdcafd5640 1 -- 172.21.15.115:0/955193928 >> 172.21.15.115:0/955193928 conn(0x7fcdc4065960 msgr2=0x7fcdc4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:00.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.076+0000 7fcdcafd5640 1 -- 172.21.15.115:0/955193928 shutdown_connections 2023-12-01T01:52:00.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.076+0000 7fcdcafd5640 1 -- 172.21.15.115:0/955193928 wait complete. 2023-12-01T01:52:00.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.077+0000 7fcdcafd5640 1 Processor -- start 2023-12-01T01:52:00.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.078+0000 7fcdcafd5640 1 -- start start 2023-12-01T01:52:00.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdcafd5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcdc415bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:00.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdcafd5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdc40c83f0 0x7fcdc40cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:00.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdcafd5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdc4158940 0x7fcdc4158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdcafd5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc4159300 con 0x7fcda4005180 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdcafd5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40cb320 con 0x7fcdc40c83f0 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdcafd5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40c8870 con 0x7fcdc4158940 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdc8d71640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcdc415bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdc8d71640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcdc415bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34518/0 (socket says 172.21.15.115:34518) 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdc8d71640 1 -- 172.21.15.115:0/1320996137 learned_addr learned my addr 172.21.15.115:0/1320996137 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdc8d71640 1 -- 172.21.15.115:0/1320996137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdc4158940 msgr2=0x7fcdc4158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdc8d71640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdc4158940 0x7fcdc4158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdc8d71640 1 -- 172.21.15.115:0/1320996137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdc40c83f0 msgr2=0x7fcdc40cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdc8d71640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdc40c83f0 0x7fcdc40cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.079+0000 7fcdc8d71640 1 -- 172.21.15.115:0/1320996137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40cb5c0 con 0x7fcda4005180 2023-12-01T01:52:00.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.080+0000 7fcdc8d71640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcdc415bbe0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e7e0 tx=0x7fcdb400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:00.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.080+0000 7fcdc17fa640 1 -- 172.21.15.115:0/1320996137 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400c6d0 con 0x7fcda4005180 2023-12-01T01:52:00.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.080+0000 7fcdc17fa640 1 -- 172.21.15.115:0/1320996137 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb400c830 con 0x7fcda4005180 2023-12-01T01:52:00.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.080+0000 7fcdc17fa640 1 -- 172.21.15.115:0/1320996137 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400cba0 con 0x7fcda4005180 2023-12-01T01:52:00.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.080+0000 7fcdcafd5640 1 -- 172.21.15.115:0/1320996137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc40cb8a0 con 0x7fcda4005180 2023-12-01T01:52:00.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.080+0000 7fcdcafd5640 1 -- 172.21.15.115:0/1320996137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc40cbe60 con 0x7fcda4005180 2023-12-01T01:52:00.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.081+0000 7fcdcafd5640 1 -- 172.21.15.115:0/1320996137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc404e890 con 0x7fcda4005180 2023-12-01T01:52:00.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.082+0000 7fcdc17fa640 1 -- 172.21.15.115:0/1320996137 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fcdb400cd00 con 0x7fcda4005180 2023-12-01T01:52:00.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.083+0000 7fcdc17fa640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcda806a550 0x7fcda806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:00.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.083+0000 7fcdc17fa640 1 -- 172.21.15.115:0/1320996137 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5545+0+0 (secure 0 0 0) 0x7fcdb4088a80 con 0x7fcda4005180 2023-12-01T01:52:00.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.083+0000 7fcdc3fff640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcda806a550 0x7fcda806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:00.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.084+0000 7fcdc3fff640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcda806a550 0x7fcda806ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40b8d20 tx=0x7fcdac001490 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:00.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.087+0000 7fcdc17fa640 1 -- 172.21.15.115:0/1320996137 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcdb4053e50 con 0x7fcda4005180 2023-12-01T01:52:00.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.277+0000 7fcdcafd5640 1 -- 172.21.15.115:0/1320996137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcdc40d0a70 con 0x7fcda4005180 2023-12-01T01:52:00.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.284+0000 7fcdc17fa640 1 -- 172.21.15.115:0/1320996137 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcdb4057470 con 0x7fcda4005180 2023-12-01T01:52:00.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.286+0000 7fcdcafd5640 1 -- 172.21.15.115:0/1320996137 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcda806a550 msgr2=0x7fcda806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:00.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.286+0000 7fcdcafd5640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcda806a550 0x7fcda806ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40b8d20 tx=0x7fcdac001490 comp rx=0 tx=0).stop 2023-12-01T01:52:00.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.286+0000 7fcdcafd5640 1 -- 172.21.15.115:0/1320996137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 msgr2=0x7fcdc415bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:00.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.286+0000 7fcdcafd5640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcdc415bbe0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e7e0 tx=0x7fcdb400eca0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.286+0000 7fcdcafd5640 1 -- 172.21.15.115:0/1320996137 shutdown_connections 2023-12-01T01:52:00.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.286+0000 7fcdcafd5640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcda806a550 0x7fcda806ca10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.287+0000 7fcdcafd5640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdc4158940 0x7fcdc4158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.287+0000 7fcdcafd5640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdc40c83f0 0x7fcdc40cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.287+0000 7fcdcafd5640 1 --2- 172.21.15.115:0/1320996137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcda4005180 0x7fcdc415bbe0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:00.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.287+0000 7fcdcafd5640 1 -- 172.21.15.115:0/1320996137 >> 172.21.15.115:0/1320996137 conn(0x7fcdc4065960 msgr2=0x7fcdc40d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:00.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.287+0000 7fcdcafd5640 1 -- 172.21.15.115:0/1320996137 shutdown_connections 2023-12-01T01:52:00.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:00.287+0000 7fcdcafd5640 1 -- 172.21.15.115:0/1320996137 wait complete. 2023-12-01T01:52:00.307 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:52:00.307 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:52:00.432 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:52:00.553 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:52:00.553 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:52:00.553 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:00.614 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:52:00.614 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:52:00.614 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:00.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:52:00.673 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:52:00.674 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:00.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:52:00.735 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:52:00.735 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:00.795 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:52:00.795 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T01:52:00.795 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:52:01.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.016+0000 7f9a20bfa640 1 Processor -- start 2023-12-01T01:52:01.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.016+0000 7f9a20bfa640 1 -- start start 2023-12-01T01:52:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.018+0000 7f9a20bfa640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9a1c15e2f0 0x7f9a1c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.018+0000 7f9a20bfa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9a1c0adf10 0x7f9a1c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.018+0000 7f9a20bfa640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c0add10 con 0x7f9a1c06f2f0 2023-12-01T01:52:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.018+0000 7f9a20bfa640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c0c5760 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.018+0000 7f9a20bfa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c15e6d0 con 0x7f9a1c0adf10 2023-12-01T01:52:01.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.018+0000 7f9a19d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9a1c06f2f0 0x7f9a1c0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54968/0 (socket says 172.21.15.115:54968) 2023-12-01T01:52:01.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.018+0000 7f9a19d9b640 1 -- 172.21.15.115:0/2641827317 learned_addr learned my addr 172.21.15.115:0/2641827317 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:01.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.018+0000 7f9a1a59c640 1 --2- 172.21.15.115:0/2641827317 >> v2:172.21.15.115:3300/0 conn(0x7f9a1c15e2f0 0x7f9a1c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.019+0000 7f9a1959a640 1 -- 172.21.15.115:0/2641827317 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779962000 0 0) 0x7f9a1c0add10 con 0x7f9a1c06f2f0 2023-12-01T01:52:01.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.019+0000 7f9a1ad9d640 1 --2- 172.21.15.115:0/2641827317 >> v2:172.21.15.142:3300/0 conn(0x7f9a1c0adf10 0x7f9a1c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.019+0000 7f9a1959a640 1 -- 172.21.15.115:0/2641827317 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a04003520 con 0x7f9a1c06f2f0 2023-12-01T01:52:01.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.019+0000 7f9a1a59c640 1 -- 172.21.15.115:0/2641827317 >> v2:172.21.15.142:3300/0 conn(0x7f9a1c0adf10 msgr2=0x7f9a1c0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:01.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.019+0000 7f9a1a59c640 1 --2- 172.21.15.115:0/2641827317 >> v2:172.21.15.142:3300/0 conn(0x7f9a1c0adf10 0x7f9a1c0caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.019+0000 7f9a1a59c640 1 -- 172.21.15.115:0/2641827317 >> v1:172.21.15.115:6789/0 conn(0x7f9a1c06f2f0 legacy=0x7f9a1c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:01.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.019+0000 7f9a1a59c640 1 -- 172.21.15.115:0/2641827317 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c0cf7b0 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.020+0000 7f9a1a59c640 1 --2- 172.21.15.115:0/2641827317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 0x7f9a1c0ca660 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0c8e80 tx=0x7f9a0c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d236f748e85c6f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:01.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.020+0000 7f9a1959a640 1 -- 172.21.15.115:0/2641827317 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c031710 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.020+0000 7f9a1959a640 1 -- 172.21.15.115:0/2641827317 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c02f050 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.020+0000 7f9a1959a640 1 -- 172.21.15.115:0/2641827317 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c031a70 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.021+0000 7f9a20bfa640 1 -- 172.21.15.115:0/2641827317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 msgr2=0x7f9a1c0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:01.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.021+0000 7f9a20bfa640 1 --2- 172.21.15.115:0/2641827317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 0x7f9a1c0ca660 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0c8e80 tx=0x7f9a0c02b7b0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.021+0000 7f9a20bfa640 1 -- 172.21.15.115:0/2641827317 shutdown_connections 2023-12-01T01:52:01.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.021+0000 7f9a20bfa640 1 --2- 172.21.15.115:0/2641827317 >> v2:172.21.15.142:3300/0 conn(0x7f9a1c0adf10 0x7f9a1c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.021+0000 7f9a20bfa640 1 --2- 172.21.15.115:0/2641827317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 0x7f9a1c0ca660 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.021+0000 7f9a20bfa640 1 -- 172.21.15.115:0/2641827317 >> 172.21.15.115:0/2641827317 conn(0x7f9a1c0ce880 msgr2=0x7f9a1c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:01.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.021+0000 7f9a20bfa640 1 -- 172.21.15.115:0/2641827317 shutdown_connections 2023-12-01T01:52:01.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.021+0000 7f9a20bfa640 1 -- 172.21.15.115:0/2641827317 wait complete. 2023-12-01T01:52:01.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.023+0000 7f9a20bfa640 1 Processor -- start 2023-12-01T01:52:01.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.023+0000 7f9a20bfa640 1 -- start start 2023-12-01T01:52:01.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a20bfa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a1c06f2f0 0x7f9a1c14c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a20bfa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a1c0adf10 0x7f9a1c149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a20bfa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 0x7f9a1c0c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a20bfa640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c149b10 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a20bfa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c149750 con 0x7f9a1c06f2f0 2023-12-01T01:52:01.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a20bfa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0c4880 con 0x7f9a1c0adf10 2023-12-01T01:52:01.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a1ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 0x7f9a1c0c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a1ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 0x7f9a1c0c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34532/0 (socket says 172.21.15.115:34532) 2023-12-01T01:52:01.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a1ad9d640 1 -- 172.21.15.115:0/3198401939 learned_addr learned my addr 172.21.15.115:0/3198401939 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:01.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a1ad9d640 1 -- 172.21.15.115:0/3198401939 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a1c0adf10 msgr2=0x7f9a1c149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:01.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a1ad9d640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a1c0adf10 0x7f9a1c149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a1ad9d640 1 -- 172.21.15.115:0/3198401939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a1c06f2f0 msgr2=0x7f9a1c14c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:01.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a1a59c640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a1c06f2f0 0x7f9a1c14c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a1ad9d640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a1c06f2f0 0x7f9a1c14c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a1ad9d640 1 -- 172.21.15.115:0/3198401939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c009400 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.024+0000 7f9a1a59c640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a1c06f2f0 0x7f9a1c14c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:01.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.025+0000 7f9a1ad9d640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 0x7f9a1c0c4020 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400cc30 tx=0x7f9a1400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:01.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.025+0000 7f9a0affd640 1 -- 172.21.15.115:0/3198401939 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400ce40 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.025+0000 7f9a0affd640 1 -- 172.21.15.115:0/3198401939 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a14003990 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.025+0000 7f9a0affd640 1 -- 172.21.15.115:0/3198401939 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14003d20 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.025+0000 7f9a20bfa640 1 -- 172.21.15.115:0/3198401939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c0c4b60 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.025+0000 7f9a20bfa640 1 -- 172.21.15.115:0/3198401939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c0c4560 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.027+0000 7f9a20bfa640 1 -- 172.21.15.115:0/3198401939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c06fda0 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.027+0000 7f9a0affd640 1 -- 172.21.15.115:0/3198401939 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9a14003e80 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.028+0000 7f9a0affd640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f99fc06a550 0x7f99fc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.028+0000 7f9a0affd640 1 -- 172.21.15.115:0/3198401939 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f9a14088e80 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.028+0000 7f9a1a59c640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f99fc06a550 0x7f99fc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.029+0000 7f9a1a59c640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f99fc06a550 0x7f99fc06ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c035000 tx=0x7f9a0c008030 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:01.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.033+0000 7f9a0affd640 1 -- 172.21.15.115:0/3198401939 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9a14054380 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.260+0000 7f9a20bfa640 1 -- 172.21.15.115:0/3198401939 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a1c149c50 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.261+0000 7f9a0affd640 1 -- 172.21.15.115:0/3198401939 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f9a1c149c50 con 0x7f9a1c15e2f0 2023-12-01T01:52:01.263 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:01.263 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5036,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1284508688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1284508688},{"type":"v1","addr":"172.21.15.142:6833","nonce":1284508688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1196894153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1196894153},{"type":"v1","addr":"172.21.15.142:6835","nonce":1196894153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6432,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/120786293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":120786293},{"type":"v1","addr":"172.21.15.115:6836","nonce":120786293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6873,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1406163157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1406163157},{"type":"v1","addr":"172.21.15.115:6837","nonce":1406163157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[]} 2023-12-01T01:52:01.263 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 34 2023-12-01T01:52:01.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.263+0000 7f9a20bfa640 1 -- 172.21.15.115:0/3198401939 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f99fc06a550 msgr2=0x7f99fc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:01.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.263+0000 7f9a20bfa640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f99fc06a550 0x7f99fc06ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c035000 tx=0x7f9a0c008030 comp rx=0 tx=0).stop 2023-12-01T01:52:01.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.263+0000 7f9a20bfa640 1 -- 172.21.15.115:0/3198401939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 msgr2=0x7f9a1c0c4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:01.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.263+0000 7f9a20bfa640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 0x7f9a1c0c4020 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400cc30 tx=0x7f9a1400a5c0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.264+0000 7f9a20bfa640 1 -- 172.21.15.115:0/3198401939 shutdown_connections 2023-12-01T01:52:01.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.264+0000 7f9a20bfa640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f99fc06a550 0x7f99fc06ca10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.264+0000 7f9a20bfa640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a1c0adf10 0x7f9a1c149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.264+0000 7f9a20bfa640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a1c15e2f0 0x7f9a1c0c4020 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.264+0000 7f9a20bfa640 1 --2- 172.21.15.115:0/3198401939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a1c06f2f0 0x7f9a1c14c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.264+0000 7f9a20bfa640 1 -- 172.21.15.115:0/3198401939 >> 172.21.15.115:0/3198401939 conn(0x7f9a1c0ce880 msgr2=0x7f9a1c06c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:01.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.264+0000 7f9a20bfa640 1 -- 172.21.15.115:0/3198401939 shutdown_connections 2023-12-01T01:52:01.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.264+0000 7f9a20bfa640 1 -- 172.21.15.115:0/3198401939 wait complete. 2023-12-01T01:52:01.284 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T01:52:01.284 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T01:52:01.284 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T01:52:01.284 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:52:01.284 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T01:52:01.284 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T01:52:01.284 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T01:52:01.285 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:52:01.285 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T01:52:01.285 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T01:52:01.285 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T01:52:01.285 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:52:01.285 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T01:52:01.285 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T01:52:01.285 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T01:52:01.286 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:52:01.299 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T01:52:01.299 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T01:52:01.305 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T01:52:01.305 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T01:52:01.313 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T01:52:01.313 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T01:52:01.314 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T01:52:01.315 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T01:52:01.353 INFO:tasks.ceph.mds.c:Started 2023-12-01T01:52:01.354 DEBUG:teuthology.parallel:result is None 2023-12-01T01:52:01.361 INFO:tasks.ceph.mds.a:Started 2023-12-01T01:52:01.361 DEBUG:teuthology.parallel:result is None 2023-12-01T01:52:01.362 INFO:tasks.ceph.mds.b:Started 2023-12-01T01:52:01.362 DEBUG:teuthology.parallel:result is None 2023-12-01T01:52:01.369 INFO:tasks.ceph.mds.d:Started 2023-12-01T01:52:01.369 DEBUG:teuthology.parallel:result is None 2023-12-01T01:52:01.369 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T01:52:01.510 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:52:01.517 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-12-01T01:52:01.517 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-12-01T01:52:01.517 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-12-01T01:52:01.520 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T01:52:01.531 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T01:52:01.533 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T01:52:01.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.563+0000 7f1384569640 1 Processor -- start 2023-12-01T01:52:01.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.563+0000 7f1384569640 1 -- start start 2023-12-01T01:52:01.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.564+0000 7f1384569640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f137c15cf90 0x7f137c0d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.564+0000 7f1384569640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f137c0ce0d0 0x7f137c0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.564+0000 7f1384569640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137c15d370 con 0x7f137c0b8610 2023-12-01T01:52:01.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.564+0000 7f1384569640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137c0d19a0 con 0x7f137c0ce0d0 2023-12-01T01:52:01.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.564+0000 7f1384569640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137c0d1c80 con 0x7f137c15cf90 2023-12-01T01:52:01.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.564+0000 7f1382b06640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f137c0ce0d0 0x7f137c0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.564+0000 7f1382b06640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f137c0ce0d0 0x7f137c0b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34580/0 (socket says 172.21.15.115:34580) 2023-12-01T01:52:01.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.564+0000 7f1382b06640 1 -- 172.21.15.115:0/2929171981 learned_addr learned my addr 172.21.15.115:0/2929171981 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:01.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.564+0000 7f1381b04640 1 --1- 172.21.15.115:0/2929171981 >> v1:172.21.15.142:6789/0 conn(0x7f137c0b8610 0x7f137c0cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:36950/0 (socket says 172.21.15.115:36950) 2023-12-01T01:52:01.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.565+0000 7f1382305640 1 --2- 172.21.15.115:0/2929171981 >> v2:172.21.15.142:3300/0 conn(0x7f137c15cf90 0x7f137c0d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.565+0000 7f1382b06640 1 -- 172.21.15.115:0/2929171981 >> v2:172.21.15.142:3300/0 conn(0x7f137c15cf90 msgr2=0x7f137c0d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:01.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.565+0000 7f1382b06640 1 --2- 172.21.15.115:0/2929171981 >> v2:172.21.15.142:3300/0 conn(0x7f137c15cf90 0x7f137c0d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.565+0000 7f1382b06640 1 -- 172.21.15.115:0/2929171981 >> v1:172.21.15.142:6789/0 conn(0x7f137c0b8610 legacy=0x7f137c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:01.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.565+0000 7f1382b06640 1 -- 172.21.15.115:0/2929171981 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0d1f60 con 0x7f137c0ce0d0 2023-12-01T01:52:01.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.566+0000 7f1382b06640 1 --2- 172.21.15.115:0/2929171981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0ce0d0 0x7f137c0b5d90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f137c066fc0 tx=0x7f137402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a37fa3ab979f82 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:01.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.566+0000 7f1381303640 1 -- 172.21.15.115:0/2929171981 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137402cb20 con 0x7f137c0ce0d0 2023-12-01T01:52:01.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.566+0000 7f1381303640 1 -- 172.21.15.115:0/2929171981 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137402cc80 con 0x7f137c0ce0d0 2023-12-01T01:52:01.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.566+0000 7f1381303640 1 -- 172.21.15.115:0/2929171981 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13740343f0 con 0x7f137c0ce0d0 2023-12-01T01:52:01.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.567+0000 7f1384569640 1 -- 172.21.15.115:0/2929171981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0ce0d0 msgr2=0x7f137c0b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:01.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.567+0000 7f1384569640 1 --2- 172.21.15.115:0/2929171981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0ce0d0 0x7f137c0b5d90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f137c066fc0 tx=0x7f137402bf80 comp rx=0 tx=0).stop 2023-12-01T01:52:01.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.567+0000 7f1384569640 1 -- 172.21.15.115:0/2929171981 shutdown_connections 2023-12-01T01:52:01.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.567+0000 7f1384569640 1 --2- 172.21.15.115:0/2929171981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0ce0d0 0x7f137c0b5d90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.567+0000 7f1384569640 1 --2- 172.21.15.115:0/2929171981 >> v2:172.21.15.142:3300/0 conn(0x7f137c15cf90 0x7f137c0d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.567+0000 7f1384569640 1 -- 172.21.15.115:0/2929171981 >> 172.21.15.115:0/2929171981 conn(0x7f137c065960 msgr2=0x7f137c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:01.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.567+0000 7f1384569640 1 -- 172.21.15.115:0/2929171981 shutdown_connections 2023-12-01T01:52:01.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.567+0000 7f1384569640 1 -- 172.21.15.115:0/2929171981 wait complete. 2023-12-01T01:52:01.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.569+0000 7f1384569640 1 Processor -- start 2023-12-01T01:52:01.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.569+0000 7f1384569640 1 -- start start 2023-12-01T01:52:01.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.570+0000 7f1384569640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0b8610 0x7f137c15d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.570+0000 7f1384569640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f137c0ce0d0 0x7f137c1f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.570+0000 7f1384569640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f137c15cf90 0x7f137c1f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.570+0000 7f1384569640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c158370 con 0x7f137c0b8610 2023-12-01T01:52:01.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.570+0000 7f1384569640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c157d60 con 0x7f137c15cf90 2023-12-01T01:52:01.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.570+0000 7f1384569640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c155930 con 0x7f137c0ce0d0 2023-12-01T01:52:01.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.570+0000 7f1382305640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0b8610 0x7f137c15d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.570+0000 7f1382305640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0b8610 0x7f137c15d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34586/0 (socket says 172.21.15.115:34586) 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.570+0000 7f1382305640 1 -- 172.21.15.115:0/4106236836 learned_addr learned my addr 172.21.15.115:0/4106236836 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1382b06640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f137c15cf90 0x7f137c1f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1381b04640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f137c0ce0d0 0x7f137c1f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1382305640 1 -- 172.21.15.115:0/4106236836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f137c0ce0d0 msgr2=0x7f137c1f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1382305640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f137c0ce0d0 0x7f137c1f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1382305640 1 -- 172.21.15.115:0/4106236836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f137c15cf90 msgr2=0x7f137c1f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1382305640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f137c15cf90 0x7f137c1f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1382305640 1 -- 172.21.15.115:0/4106236836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1374009b30 con 0x7f137c0b8610 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1382b06640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f137c15cf90 0x7f137c1f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1381b04640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f137c0ce0d0 0x7f137c1f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:01.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1382305640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0b8610 0x7f137c15d920 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f137800e870 tx=0x7f137800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:01.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1372ffd640 1 -- 172.21.15.115:0/4106236836 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780094f0 con 0x7f137c0b8610 2023-12-01T01:52:01.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1372ffd640 1 -- 172.21.15.115:0/4106236836 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137800c940 con 0x7f137c0b8610 2023-12-01T01:52:01.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1384569640 1 -- 172.21.15.115:0/4106236836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c158000 con 0x7f137c0b8610 2023-12-01T01:52:01.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1372ffd640 1 -- 172.21.15.115:0/4106236836 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137800ccf0 con 0x7f137c0b8610 2023-12-01T01:52:01.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.571+0000 7f1384569640 1 -- 172.21.15.115:0/4106236836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c147e60 con 0x7f137c0b8610 2023-12-01T01:52:01.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.573+0000 7f1384569640 1 -- 172.21.15.115:0/4106236836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137c04e890 con 0x7f137c0b8610 2023-12-01T01:52:01.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.574+0000 7f1372ffd640 1 -- 172.21.15.115:0/4106236836 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1378013470 con 0x7f137c0b8610 2023-12-01T01:52:01.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.574+0000 7f1372ffd640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f136006a550 0x7f136006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:01.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.575+0000 7f1372ffd640 1 -- 172.21.15.115:0/4106236836 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f1378088ab0 con 0x7f137c0b8610 2023-12-01T01:52:01.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.575+0000 7f1381b04640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f136006a550 0x7f136006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:01.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.576+0000 7f1381b04640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f136006a550 0x7f136006ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f137c0c7310 tx=0x7f136c009040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:01.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.579+0000 7f1372ffd640 1 -- 172.21.15.115:0/4106236836 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f13780540c0 con 0x7f137c0b8610 2023-12-01T01:52:01.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:01.778+0000 7f1384569640 1 -- 172.21.15.115:0/4106236836 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f137c0b19b0 con 0x7f137c0b8610 2023-12-01T01:52:02.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.566+0000 7f1372ffd640 1 -- 172.21.15.115:0/4106236836 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v93) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1378021080 con 0x7f137c0b8610 2023-12-01T01:52:02.567 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T01:52:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.569+0000 7f1384569640 1 -- 172.21.15.115:0/4106236836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f136006a550 msgr2=0x7f136006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.569+0000 7f1384569640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f136006a550 0x7f136006ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f137c0c7310 tx=0x7f136c009040 comp rx=0 tx=0).stop 2023-12-01T01:52:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.569+0000 7f1384569640 1 -- 172.21.15.115:0/4106236836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0b8610 msgr2=0x7f137c15d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:02.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.569+0000 7f1384569640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0b8610 0x7f137c15d920 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f137800e870 tx=0x7f137800ed30 comp rx=0 tx=0).stop 2023-12-01T01:52:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.569+0000 7f1384569640 1 -- 172.21.15.115:0/4106236836 shutdown_connections 2023-12-01T01:52:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.569+0000 7f1384569640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f136006a550 0x7f136006ca10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.569+0000 7f1384569640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f137c0ce0d0 0x7f137c1f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.569+0000 7f1384569640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f137c15cf90 0x7f137c1f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.569+0000 7f1384569640 1 --2- 172.21.15.115:0/4106236836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f137c0b8610 0x7f137c15d920 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.569+0000 7f1384569640 1 -- 172.21.15.115:0/4106236836 >> 172.21.15.115:0/4106236836 conn(0x7f137c065960 msgr2=0x7f137c15b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.570+0000 7f1384569640 1 -- 172.21.15.115:0/4106236836 shutdown_connections 2023-12-01T01:52:02.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.570+0000 7f1384569640 1 -- 172.21.15.115:0/4106236836 wait complete. 2023-12-01T01:52:02.591 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T01:52:02.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.802+0000 7f81b04d5640 1 Processor -- start 2023-12-01T01:52:02.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.802+0000 7f81b04d5640 1 -- start start 2023-12-01T01:52:02.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.804+0000 7f81b04d5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f81a815ada0 0x7f81a80ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:02.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.804+0000 7f81b04d5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81a80d15b0 0x7f81a80cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:02.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.804+0000 7f81b04d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81a80cf5b0 0x7f81a80cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:02.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.804+0000 7f81b04d5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a80cc9b0 con 0x7f81a80d15b0 2023-12-01T01:52:02.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.804+0000 7f81b04d5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a80cc110 con 0x7f81a815ada0 2023-12-01T01:52:02.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.804+0000 7f81b04d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a80cc3f0 con 0x7f81a80cf5b0 2023-12-01T01:52:02.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.804+0000 7f81ada70640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81a80d15b0 0x7f81a80cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:02.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.804+0000 7f81ada70640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81a80d15b0 0x7f81a80cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34598/0 (socket says 172.21.15.115:34598) 2023-12-01T01:52:02.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.804+0000 7f81ada70640 1 -- 172.21.15.115:0/868613720 learned_addr learned my addr 172.21.15.115:0/868613720 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:02.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.804+0000 7f81ae271640 1 --2- 172.21.15.115:0/868613720 >> v2:172.21.15.142:3300/0 conn(0x7f81a815ada0 0x7f81a80ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:02.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.805+0000 7f81aea72640 1 --2- 172.21.15.115:0/868613720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81a80cf5b0 0x7f81a80cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:02.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.805+0000 7f81ada70640 1 -- 172.21.15.115:0/868613720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81a80cf5b0 msgr2=0x7f81a80cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:02.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.805+0000 7f81ada70640 1 --2- 172.21.15.115:0/868613720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81a80cf5b0 0x7f81a80cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.805+0000 7f81ada70640 1 -- 172.21.15.115:0/868613720 >> v2:172.21.15.142:3300/0 conn(0x7f81a815ada0 msgr2=0x7f81a80ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:02.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.805+0000 7f81ada70640 1 --2- 172.21.15.115:0/868613720 >> v2:172.21.15.142:3300/0 conn(0x7f81a815ada0 0x7f81a80ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.805+0000 7f81ada70640 1 -- 172.21.15.115:0/868613720 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a80cc6d0 con 0x7f81a80d15b0 2023-12-01T01:52:02.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.806+0000 7f81ada70640 1 --2- 172.21.15.115:0/868613720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a80d15b0 0x7f81a80cf070 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f81a80722e0 tx=0x7f819802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4590cb6534ae09 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:02.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.806+0000 7f81ad26f640 1 -- 172.21.15.115:0/868613720 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198031750 con 0x7f81a80d15b0 2023-12-01T01:52:02.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.806+0000 7f81ad26f640 1 -- 172.21.15.115:0/868613720 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81980318b0 con 0x7f81a80d15b0 2023-12-01T01:52:02.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.806+0000 7f81ad26f640 1 -- 172.21.15.115:0/868613720 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198031c40 con 0x7f81a80d15b0 2023-12-01T01:52:02.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.807+0000 7f81b04d5640 1 -- 172.21.15.115:0/868613720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a80d15b0 msgr2=0x7f81a80cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:02.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.807+0000 7f81b04d5640 1 --2- 172.21.15.115:0/868613720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a80d15b0 0x7f81a80cf070 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f81a80722e0 tx=0x7f819802b7f0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.807+0000 7f81b04d5640 1 -- 172.21.15.115:0/868613720 shutdown_connections 2023-12-01T01:52:02.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.807+0000 7f81b04d5640 1 --2- 172.21.15.115:0/868613720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81a80cf5b0 0x7f81a80cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.807+0000 7f81b04d5640 1 --2- 172.21.15.115:0/868613720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a80d15b0 0x7f81a80cf070 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.807+0000 7f81b04d5640 1 --2- 172.21.15.115:0/868613720 >> v2:172.21.15.142:3300/0 conn(0x7f81a815ada0 0x7f81a80ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.807+0000 7f81b04d5640 1 -- 172.21.15.115:0/868613720 >> 172.21.15.115:0/868613720 conn(0x7f81a80d0a90 msgr2=0x7f81a80d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:02.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.807+0000 7f81b04d5640 1 -- 172.21.15.115:0/868613720 shutdown_connections 2023-12-01T01:52:02.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.807+0000 7f81b04d5640 1 -- 172.21.15.115:0/868613720 wait complete. 2023-12-01T01:52:02.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.809+0000 7f81b04d5640 1 Processor -- start 2023-12-01T01:52:02.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.809+0000 7f81b04d5640 1 -- start start 2023-12-01T01:52:02.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.810+0000 7f81b04d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81a80cf5b0 0x7f81a806b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:02.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.810+0000 7f81b04d5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81a80d15b0 0x7f81a80b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:02.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.810+0000 7f81b04d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a815ada0 0x7f81a80b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:02.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.810+0000 7f81b04d5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a80b50e0 con 0x7f81a815ada0 2023-12-01T01:52:02.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.810+0000 7f81b04d5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a80b53c0 con 0x7f81a80d15b0 2023-12-01T01:52:02.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.810+0000 7f81b04d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a80b56a0 con 0x7f81a80cf5b0 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.810+0000 7f81aea72640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a815ada0 0x7f81a80b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.810+0000 7f81aea72640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a815ada0 0x7f81a80b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34614/0 (socket says 172.21.15.115:34614) 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.810+0000 7f81aea72640 1 -- 172.21.15.115:0/542058668 learned_addr learned my addr 172.21.15.115:0/542058668 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f81ae271640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81a80cf5b0 0x7f81a806b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f81ada70640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81a80d15b0 0x7f81a80b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f81aea72640 1 -- 172.21.15.115:0/542058668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81a80cf5b0 msgr2=0x7f81a806b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f81aea72640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81a80cf5b0 0x7f81a806b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f81aea72640 1 -- 172.21.15.115:0/542058668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81a80d15b0 msgr2=0x7f81a80b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f81aea72640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81a80d15b0 0x7f81a80b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f81aea72640 1 -- 172.21.15.115:0/542058668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81980094d0 con 0x7f81a815ada0 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f81aea72640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a815ada0 0x7f81a80b1fa0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f81a0007d20 tx=0x7f81a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:02.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f819effd640 1 -- 172.21.15.115:0/542058668 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0014420 con 0x7f81a815ada0 2023-12-01T01:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f819effd640 1 -- 172.21.15.115:0/542058668 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a00145a0 con 0x7f81a815ada0 2023-12-01T01:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f819effd640 1 -- 172.21.15.115:0/542058668 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0014950 con 0x7f81a815ada0 2023-12-01T01:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.811+0000 7f81b04d5640 1 -- 172.21.15.115:0/542058668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a80b5980 con 0x7f81a815ada0 2023-12-01T01:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.812+0000 7f81b04d5640 1 -- 172.21.15.115:0/542058668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a806baa0 con 0x7f81a815ada0 2023-12-01T01:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.813+0000 7f81b04d5640 1 -- 172.21.15.115:0/542058668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a806bc30 con 0x7f81a815ada0 2023-12-01T01:52:02.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.813+0000 7f819effd640 1 -- 172.21.15.115:0/542058668 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f81a0014ab0 con 0x7f81a815ada0 2023-12-01T01:52:02.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.814+0000 7f819effd640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f818c06a580 0x7f818c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:02.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.814+0000 7f819effd640 1 -- 172.21.15.115:0/542058668 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f81a0088fa0 con 0x7f81a815ada0 2023-12-01T01:52:02.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.814+0000 7f81ae271640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f818c06a580 0x7f818c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:02.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.815+0000 7f81ae271640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f818c06a580 0x7f818c06ca40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f81a4006fd0 tx=0x7f81a4008040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:02.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:02.818+0000 7f819effd640 1 -- 172.21.15.115:0/542058668 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f81a00547c0 con 0x7f81a815ada0 2023-12-01T01:52:03.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.075+0000 7f81b04d5640 1 -- 172.21.15.115:0/542058668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f81a80b5ea0 con 0x7f81a815ada0 2023-12-01T01:52:03.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.077+0000 7f819effd640 1 -- 172.21.15.115:0/542058668 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f81a80b5ea0 con 0x7f81a815ada0 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T01:52:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:52:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T01:52:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:52:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T01:52:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:52:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:52:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T01:52:03.092 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 -- 172.21.15.115:0/542058668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f818c06a580 msgr2=0x7f818c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f818c06a580 0x7f818c06ca40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f81a4006fd0 tx=0x7f81a4008040 comp rx=0 tx=0).stop 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 -- 172.21.15.115:0/542058668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a815ada0 msgr2=0x7f81a80b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a815ada0 0x7f81a80b1fa0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f81a0007d20 tx=0x7f81a000e4b0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 -- 172.21.15.115:0/542058668 shutdown_connections 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f818c06a580 0x7f818c06ca40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81a815ada0 0x7f81a80b1fa0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81a80d15b0 0x7f81a80b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 --2- 172.21.15.115:0/542058668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81a80cf5b0 0x7f81a806b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 -- 172.21.15.115:0/542058668 >> 172.21.15.115:0/542058668 conn(0x7f81a80d0a90 msgr2=0x7f81a8159060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:03.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 -- 172.21.15.115:0/542058668 shutdown_connections 2023-12-01T01:52:03.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.080+0000 7f81b04d5640 1 -- 172.21.15.115:0/542058668 wait complete. 2023-12-01T01:52:03.100 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:03.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.310+0000 7f948a569640 1 Processor -- start 2023-12-01T01:52:03.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.310+0000 7f948a569640 1 -- start start 2023-12-01T01:52:03.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.311+0000 7f948a569640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f948415e2f0 0x7f94840ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.311+0000 7f948a569640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f948406f2f0 0x7f94840caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.311+0000 7f948a569640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94840add10 con 0x7f94840adf10 2023-12-01T01:52:03.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.311+0000 7f948a569640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94840c5760 con 0x7f948415e2f0 2023-12-01T01:52:03.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.311+0000 7f948a569640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948415e6d0 con 0x7f948406f2f0 2023-12-01T01:52:03.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.312+0000 7f9483fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f948415e2f0 0x7f94840ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:03.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.312+0000 7f9488b06640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f94840adf10 0x7f94840c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54998/0 (socket says 172.21.15.115:54998) 2023-12-01T01:52:03.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.312+0000 7f9488b06640 1 -- 172.21.15.115:0/1926970629 learned_addr learned my addr 172.21.15.115:0/1926970629 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:03.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.312+0000 7f9483fff640 1 --2- 172.21.15.115:0/1926970629 >> v2:172.21.15.115:3300/0 conn(0x7f948415e2f0 0x7f94840ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34620/0 (socket says 172.21.15.115:34620) 2023-12-01T01:52:03.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.312+0000 7f9483fff640 1 -- 172.21.15.115:0/1926970629 >> v2:172.21.15.142:3300/0 conn(0x7f948406f2f0 msgr2=0x7f94840caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:03.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.312+0000 7f9483fff640 1 --2- 172.21.15.115:0/1926970629 >> v2:172.21.15.142:3300/0 conn(0x7f948406f2f0 0x7f94840caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.312+0000 7f9483fff640 1 -- 172.21.15.115:0/1926970629 >> v1:172.21.15.115:6789/0 conn(0x7f94840adf10 legacy=0x7f94840c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.312+0000 7f9483fff640 1 -- 172.21.15.115:0/1926970629 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840cf7b0 con 0x7f948415e2f0 2023-12-01T01:52:03.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.312+0000 7f9482ffd640 1 -- 172.21.15.115:0/1926970629 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239081758 0 0) 0x7f94840add10 con 0x7f94840adf10 2023-12-01T01:52:03.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.313+0000 7f9483fff640 1 --2- 172.21.15.115:0/1926970629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 0x7f94840ca660 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f948406de80 tx=0x7f947402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c641e42906d04ab4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.313+0000 7f9482ffd640 1 -- 172.21.15.115:0/1926970629 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474031710 con 0x7f948415e2f0 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.313+0000 7f9482ffd640 1 -- 172.21.15.115:0/1926970629 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947402f050 con 0x7f948415e2f0 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.313+0000 7f9482ffd640 1 -- 172.21.15.115:0/1926970629 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474031a90 con 0x7f948415e2f0 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.314+0000 7f948a569640 1 -- 172.21.15.115:0/1926970629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 msgr2=0x7f94840ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.314+0000 7f948a569640 1 --2- 172.21.15.115:0/1926970629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 0x7f94840ca660 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f948406de80 tx=0x7f947402b7b0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.314+0000 7f948a569640 1 -- 172.21.15.115:0/1926970629 shutdown_connections 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.314+0000 7f948a569640 1 --2- 172.21.15.115:0/1926970629 >> v2:172.21.15.142:3300/0 conn(0x7f948406f2f0 0x7f94840caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.314+0000 7f948a569640 1 --2- 172.21.15.115:0/1926970629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 0x7f94840ca660 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.314+0000 7f948a569640 1 -- 172.21.15.115:0/1926970629 >> 172.21.15.115:0/1926970629 conn(0x7f94840ce880 msgr2=0x7f94840cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.314+0000 7f948a569640 1 -- 172.21.15.115:0/1926970629 shutdown_connections 2023-12-01T01:52:03.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.314+0000 7f948a569640 1 -- 172.21.15.115:0/1926970629 wait complete. 2023-12-01T01:52:03.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.316+0000 7f948a569640 1 Processor -- start 2023-12-01T01:52:03.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.316+0000 7f948a569640 1 -- start start 2023-12-01T01:52:03.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.317+0000 7f948a569640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948406f2f0 0x7f9484158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.317+0000 7f948a569640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94840adf10 0x7f94841f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.317+0000 7f948a569640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 0x7f94841f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.317+0000 7f948a569640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94840ad1f0 con 0x7f948415e2f0 2023-12-01T01:52:03.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.317+0000 7f948a569640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948415c7f0 con 0x7f948406f2f0 2023-12-01T01:52:03.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.317+0000 7f948a569640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948415c510 con 0x7f94840adf10 2023-12-01T01:52:03.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.317+0000 7f9488b06640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 0x7f94841f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:03.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.317+0000 7f9488b06640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 0x7f94841f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34634/0 (socket says 172.21.15.115:34634) 2023-12-01T01:52:03.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.317+0000 7f9488b06640 1 -- 172.21.15.115:0/3691728140 learned_addr learned my addr 172.21.15.115:0/3691728140 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:03.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f9488b06640 1 -- 172.21.15.115:0/3691728140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94840adf10 msgr2=0x7f94841f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f9488b06640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94840adf10 0x7f94841f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f9483fff640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948406f2f0 0x7f9484158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:03.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f9488b06640 1 -- 172.21.15.115:0/3691728140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948406f2f0 msgr2=0x7f9484158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f9488b06640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948406f2f0 0x7f9484158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f9488b06640 1 -- 172.21.15.115:0/3691728140 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474009400 con 0x7f948415e2f0 2023-12-01T01:52:03.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f9488b06640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 0x7f94841f3c60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c780 tx=0x7f947c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:03.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f9480ff9640 1 -- 172.21.15.115:0/3691728140 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c00ce90 con 0x7f948415e2f0 2023-12-01T01:52:03.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f9480ff9640 1 -- 172.21.15.115:0/3691728140 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c00a640 con 0x7f948415e2f0 2023-12-01T01:52:03.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f9480ff9640 1 -- 172.21.15.115:0/3691728140 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c00a9d0 con 0x7f948415e2f0 2023-12-01T01:52:03.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f948a569640 1 -- 172.21.15.115:0/3691728140 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94840ace80 con 0x7f948415e2f0 2023-12-01T01:52:03.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.318+0000 7f948a569640 1 -- 172.21.15.115:0/3691728140 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94840af070 con 0x7f948415e2f0 2023-12-01T01:52:03.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.320+0000 7f948a569640 1 -- 172.21.15.115:0/3691728140 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948406fda0 con 0x7f948415e2f0 2023-12-01T01:52:03.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.320+0000 7f9480ff9640 1 -- 172.21.15.115:0/3691728140 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f947c00ab30 con 0x7f948415e2f0 2023-12-01T01:52:03.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.321+0000 7f9480ff9640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f947006a580 0x7f947006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.321+0000 7f9480ff9640 1 -- 172.21.15.115:0/3691728140 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f947c087bb0 con 0x7f948415e2f0 2023-12-01T01:52:03.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.322+0000 7f9483fff640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f947006a580 0x7f947006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:03.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.322+0000 7f9483fff640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f947006a580 0x7f947006ca40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9474035000 tx=0x7f9474008030 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:03.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.326+0000 7f9480ff9640 1 -- 172.21.15.115:0/3691728140 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f947c0533e0 con 0x7f948415e2f0 2023-12-01T01:52:03.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.516+0000 7f948a569640 1 -- 172.21.15.115:0/3691728140 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94840ad010 con 0x7f948415e2f0 2023-12-01T01:52:03.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.517+0000 7f9480ff9640 1 -- 172.21.15.115:0/3691728140 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+12437 (secure 0 0 0) 0x7f947c018090 con 0x7f948415e2f0 2023-12-01T01:52:03.518 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:03.518 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":93,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:02.558287+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:03.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.519+0000 7f948a569640 1 -- 172.21.15.115:0/3691728140 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f947006a580 msgr2=0x7f947006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.519+0000 7f948a569640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f947006a580 0x7f947006ca40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9474035000 tx=0x7f9474008030 comp rx=0 tx=0).stop 2023-12-01T01:52:03.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.519+0000 7f948a569640 1 -- 172.21.15.115:0/3691728140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 msgr2=0x7f94841f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.519+0000 7f948a569640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 0x7f94841f3c60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f947c00c780 tx=0x7f947c00cc40 comp rx=0 tx=0).stop 2023-12-01T01:52:03.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.520+0000 7f948a569640 1 -- 172.21.15.115:0/3691728140 shutdown_connections 2023-12-01T01:52:03.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.520+0000 7f948a569640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f947006a580 0x7f947006ca40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.520+0000 7f948a569640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94840adf10 0x7f94841f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.520+0000 7f948a569640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f948415e2f0 0x7f94841f3c60 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.520+0000 7f948a569640 1 --2- 172.21.15.115:0/3691728140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f948406f2f0 0x7f9484158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.520+0000 7f948a569640 1 -- 172.21.15.115:0/3691728140 >> 172.21.15.115:0/3691728140 conn(0x7f94840ce880 msgr2=0x7f94840ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:03.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.520+0000 7f948a569640 1 -- 172.21.15.115:0/3691728140 shutdown_connections 2023-12-01T01:52:03.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.520+0000 7f948a569640 1 -- 172.21.15.115:0/3691728140 wait complete. 2023-12-01T01:52:03.540 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:52:03.540 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:52:03.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.754+0000 7f60b3f00640 1 Processor -- start 2023-12-01T01:52:03.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.755+0000 7f60b3f00640 1 -- start start 2023-12-01T01:52:03.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.756+0000 7f60b3f00640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f60ac0cfd60 0x7f60ac0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.756+0000 7f60b3f00640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ac0ce5e0 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.756+0000 7f60b3f00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ac0ce300 con 0x7f60ac15cd90 2023-12-01T01:52:03.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.756+0000 7f60b3f00640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ac0d0ec0 con 0x7f60ac0cfd60 2023-12-01T01:52:03.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.756+0000 7f60b149b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f60ac0cfd60 0x7f60ac0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:03.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.756+0000 7f60b149b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f60ac0cfd60 0x7f60ac0cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34638/0 (socket says 172.21.15.115:34638) 2023-12-01T01:52:03.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.756+0000 7f60b149b640 1 -- 172.21.15.115:0/3523669487 learned_addr learned my addr 172.21.15.115:0/3523669487 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:03.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.757+0000 7f60b149b640 1 -- 172.21.15.115:0/3523669487 >> v1:172.21.15.142:6789/0 conn(0x7f60ac15cd90 legacy=0x7f60ac0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:03.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.757+0000 7f60b149b640 1 -- 172.21.15.115:0/3523669487 >> v1:172.21.15.115:6789/0 conn(0x7f60ac0b5fa0 legacy=0x7f60ac0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.757+0000 7f60b149b640 1 -- 172.21.15.115:0/3523669487 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac0d10a0 con 0x7f60ac0cfd60 2023-12-01T01:52:03.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.758+0000 7f60b149b640 1 --2- 172.21.15.115:0/3523669487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0cfd60 0x7f60ac0cdd20 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6098009870 tx=0x7f609802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f77213bee73ae0c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:03.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.758+0000 7f60b0c9a640 1 -- 172.21.15.115:0/3523669487 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098030860 con 0x7f60ac0cfd60 2023-12-01T01:52:03.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.758+0000 7f60b0c9a640 1 -- 172.21.15.115:0/3523669487 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60980309c0 con 0x7f60ac0cfd60 2023-12-01T01:52:03.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.758+0000 7f60b0c9a640 1 -- 172.21.15.115:0/3523669487 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098030d50 con 0x7f60ac0cfd60 2023-12-01T01:52:03.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.759+0000 7f60b3f00640 1 -- 172.21.15.115:0/3523669487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0cfd60 msgr2=0x7f60ac0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.759+0000 7f60b3f00640 1 --2- 172.21.15.115:0/3523669487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0cfd60 0x7f60ac0cdd20 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6098009870 tx=0x7f609802b9a0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.759+0000 7f60b3f00640 1 -- 172.21.15.115:0/3523669487 shutdown_connections 2023-12-01T01:52:03.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.759+0000 7f60b3f00640 1 --2- 172.21.15.115:0/3523669487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0cfd60 0x7f60ac0cdd20 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.759+0000 7f60b3f00640 1 -- 172.21.15.115:0/3523669487 >> 172.21.15.115:0/3523669487 conn(0x7f60ac065960 msgr2=0x7f60ac065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:03.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.759+0000 7f60b3f00640 1 -- 172.21.15.115:0/3523669487 shutdown_connections 2023-12-01T01:52:03.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.759+0000 7f60b3f00640 1 -- 172.21.15.115:0/3523669487 wait complete. 2023-12-01T01:52:03.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.761+0000 7f60b3f00640 1 Processor -- start 2023-12-01T01:52:03.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.761+0000 7f60b3f00640 1 -- start start 2023-12-01T01:52:03.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.762+0000 7f60b3f00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0b5fa0 0x7f60ac15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.762+0000 7f60b3f00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ac0cfd60 0x7f60ac14c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.762+0000 7f60b3f00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ac15cd90 0x7f60ac1f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.762+0000 7f60b3f00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac14d400 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.762+0000 7f60b3f00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac1f6a50 con 0x7f60ac15cd90 2023-12-01T01:52:03.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.762+0000 7f60b3f00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac1f6d30 con 0x7f60ac0cfd60 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.762+0000 7f60b1c9c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0b5fa0 0x7f60ac15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.762+0000 7f60b1c9c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0b5fa0 0x7f60ac15fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34642/0 (socket says 172.21.15.115:34642) 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.762+0000 7f60b1c9c640 1 -- 172.21.15.115:0/4245448404 learned_addr learned my addr 172.21.15.115:0/4245448404 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60b149b640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ac0cfd60 0x7f60ac14c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60b1c9c640 1 -- 172.21.15.115:0/4245448404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ac0cfd60 msgr2=0x7f60ac14c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60b249d640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ac15cd90 0x7f60ac1f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60b1c9c640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ac0cfd60 0x7f60ac14c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60b1c9c640 1 -- 172.21.15.115:0/4245448404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ac15cd90 msgr2=0x7f60ac1f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60b1c9c640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ac15cd90 0x7f60ac1f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60b1c9c640 1 -- 172.21.15.115:0/4245448404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980094d0 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60b1c9c640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0b5fa0 0x7f60ac15fc90 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f60a000ba50 tx=0x7f60a000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:03.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60aa7fc640 1 -- 172.21.15.115:0/4245448404 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a000cbe0 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60aa7fc640 1 -- 172.21.15.115:0/4245448404 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a000cd80 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60aa7fc640 1 -- 172.21.15.115:0/4245448404 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0014680 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.763+0000 7f60b3f00640 1 -- 172.21.15.115:0/4245448404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac15f1b0 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.764+0000 7f60b3f00640 1 -- 172.21.15.115:0/4245448404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac15f770 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.765+0000 7f60b3f00640 1 -- 172.21.15.115:0/4245448404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ac04e890 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.766+0000 7f60aa7fc640 1 -- 172.21.15.115:0/4245448404 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f60a00147e0 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.767+0000 7f60aa7fc640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f609006a550 0x7f609006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:03.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.767+0000 7f60aa7fc640 1 -- 172.21.15.115:0/4245448404 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f60a0088550 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.767+0000 7f60b149b640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f609006a550 0x7f609006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:03.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.768+0000 7f60b149b640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f609006a550 0x7f609006ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6098036000 tx=0x7f609803b040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:03.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.771+0000 7f60aa7fc640 1 -- 172.21.15.115:0/4245448404 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f60a0053ec0 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.962+0000 7f60b3f00640 1 -- 172.21.15.115:0/4245448404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60ac0d0a70 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.962+0000 7f60aa7fc640 1 -- 172.21.15.115:0/4245448404 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60a00574e0 con 0x7f60ac0b5fa0 2023-12-01T01:52:03.964 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:52:03.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 -- 172.21.15.115:0/4245448404 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f609006a550 msgr2=0x7f609006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f609006a550 0x7f609006ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6098036000 tx=0x7f609803b040 comp rx=0 tx=0).stop 2023-12-01T01:52:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 -- 172.21.15.115:0/4245448404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0b5fa0 msgr2=0x7f60ac15fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0b5fa0 0x7f60ac15fc90 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f60a000ba50 tx=0x7f60a000bf10 comp rx=0 tx=0).stop 2023-12-01T01:52:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 -- 172.21.15.115:0/4245448404 shutdown_connections 2023-12-01T01:52:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f609006a550 0x7f609006ca10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60ac0cfd60 0x7f60ac14c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60ac15cd90 0x7f60ac1f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 --2- 172.21.15.115:0/4245448404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60ac0b5fa0 0x7f60ac15fc90 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 -- 172.21.15.115:0/4245448404 >> 172.21.15.115:0/4245448404 conn(0x7f60ac065960 msgr2=0x7f60ac0b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 -- 172.21.15.115:0/4245448404 shutdown_connections 2023-12-01T01:52:03.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:03.965+0000 7f60b3f00640 1 -- 172.21.15.115:0/4245448404 wait complete. 2023-12-01T01:52:03.985 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T01:52:04.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.201+0000 7f0760acf640 1 Processor -- start 2023-12-01T01:52:04.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.201+0000 7f0760acf640 1 -- start start 2023-12-01T01:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.202+0000 7f0760acf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f075c0b0e70 0x7f075c0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.203+0000 7f0760acf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f075c0b3ba0 0x7f075c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.203+0000 7f0760acf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f075c06b6a0 0x7f075c06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.203+0000 7f0760acf640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f075c0c8af0 con 0x7f075c0b0e70 2023-12-01T01:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.203+0000 7f0760acf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f075c06b1c0 con 0x7f075c06b6a0 2023-12-01T01:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.203+0000 7f0760acf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075c15f630 con 0x7f075c0b3ba0 2023-12-01T01:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.203+0000 7f075b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f075c0b0e70 0x7f075c0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.203+0000 7f075b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f075c0b0e70 0x7f075c0c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34656/0 (socket says 172.21.15.115:34656) 2023-12-01T01:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.203+0000 7f075b7fe640 1 -- 172.21.15.115:0/1919535271 learned_addr learned my addr 172.21.15.115:0/1919535271 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.203+0000 7f075affd640 1 --2- 172.21.15.115:0/1919535271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f075c0b3ba0 0x7f075c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.203+0000 7f075bfff640 1 --2- 172.21.15.115:0/1919535271 >> v2:172.21.15.142:3300/0 conn(0x7f075c06b6a0 0x7f075c06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.204+0000 7f075b7fe640 1 -- 172.21.15.115:0/1919535271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f075c0b3ba0 msgr2=0x7f075c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.204+0000 7f075b7fe640 1 --2- 172.21.15.115:0/1919535271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f075c0b3ba0 0x7f075c0b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.204+0000 7f075b7fe640 1 -- 172.21.15.115:0/1919535271 >> v2:172.21.15.142:3300/0 conn(0x7f075c06b6a0 msgr2=0x7f075c06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.204+0000 7f075b7fe640 1 --2- 172.21.15.115:0/1919535271 >> v2:172.21.15.142:3300/0 conn(0x7f075c06b6a0 0x7f075c06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.204+0000 7f075b7fe640 1 -- 172.21.15.115:0/1919535271 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c0b2040 con 0x7f075c0b0e70 2023-12-01T01:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.205+0000 7f075b7fe640 1 --2- 172.21.15.115:0/1919535271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 0x7f075c0c85b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f075c06eed0 tx=0x7f074c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb65e1e3ce4a8c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.205+0000 7f075a7fc640 1 -- 172.21.15.115:0/1919535271 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c02c9e0 con 0x7f075c0b0e70 2023-12-01T01:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.205+0000 7f075a7fc640 1 -- 172.21.15.115:0/1919535271 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c02cb40 con 0x7f075c0b0e70 2023-12-01T01:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.205+0000 7f075a7fc640 1 -- 172.21.15.115:0/1919535271 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c0343f0 con 0x7f075c0b0e70 2023-12-01T01:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.206+0000 7f0760acf640 1 -- 172.21.15.115:0/1919535271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 msgr2=0x7f075c0c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.206+0000 7f0760acf640 1 --2- 172.21.15.115:0/1919535271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 0x7f075c0c85b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f075c06eed0 tx=0x7f074c02be40 comp rx=0 tx=0).stop 2023-12-01T01:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.206+0000 7f0760acf640 1 -- 172.21.15.115:0/1919535271 shutdown_connections 2023-12-01T01:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.206+0000 7f0760acf640 1 --2- 172.21.15.115:0/1919535271 >> v2:172.21.15.142:3300/0 conn(0x7f075c06b6a0 0x7f075c06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.206+0000 7f0760acf640 1 --2- 172.21.15.115:0/1919535271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f075c0b3ba0 0x7f075c0b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.206+0000 7f0760acf640 1 --2- 172.21.15.115:0/1919535271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 0x7f075c0c85b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.206+0000 7f0760acf640 1 -- 172.21.15.115:0/1919535271 >> 172.21.15.115:0/1919535271 conn(0x7f075c0ce620 msgr2=0x7f075c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:04.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.206+0000 7f0760acf640 1 -- 172.21.15.115:0/1919535271 shutdown_connections 2023-12-01T01:52:04.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.206+0000 7f0760acf640 1 -- 172.21.15.115:0/1919535271 wait complete. 2023-12-01T01:52:04.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.207+0000 7f0760acf640 1 Processor -- start 2023-12-01T01:52:04.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.208+0000 7f0760acf640 1 -- start start 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f0760acf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c06b6a0 0x7f075c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f0760acf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 0x7f075c14ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f0760acf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f075c0b3ba0 0x7f075c14bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f0760acf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c0c8e10 con 0x7f075c0b0e70 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f0760acf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c0af540 con 0x7f075c06b6a0 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f0760acf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075c0af820 con 0x7f075c0b3ba0 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f075affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 0x7f075c14ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f075affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 0x7f075c14ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34664/0 (socket says 172.21.15.115:34664) 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f075affd640 1 -- 172.21.15.115:0/3001054233 learned_addr learned my addr 172.21.15.115:0/3001054233 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f075affd640 1 -- 172.21.15.115:0/3001054233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f075c0b3ba0 msgr2=0x7f075c14bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f075affd640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f075c0b3ba0 0x7f075c14bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f075b7fe640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c06b6a0 0x7f075c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f075affd640 1 -- 172.21.15.115:0/3001054233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c06b6a0 msgr2=0x7f075c0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f075affd640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c06b6a0 0x7f075c0acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.209+0000 7f075affd640 1 -- 172.21.15.115:0/3001054233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c009b30 con 0x7f075c0b0e70 2023-12-01T01:52:04.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.210+0000 7f075affd640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 0x7f075c14ba90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f075000d800 tx=0x7f075000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:04.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.210+0000 7f0743fff640 1 -- 172.21.15.115:0/3001054233 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075000b6c0 con 0x7f075c0b0e70 2023-12-01T01:52:04.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.210+0000 7f0743fff640 1 -- 172.21.15.115:0/3001054233 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075000b820 con 0x7f075c0b0e70 2023-12-01T01:52:04.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.210+0000 7f0760acf640 1 -- 172.21.15.115:0/3001054233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075c0afb00 con 0x7f075c0b0e70 2023-12-01T01:52:04.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.210+0000 7f0743fff640 1 -- 172.21.15.115:0/3001054233 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075000bbb0 con 0x7f075c0b0e70 2023-12-01T01:52:04.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.210+0000 7f0760acf640 1 -- 172.21.15.115:0/3001054233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075c0ad050 con 0x7f075c0b0e70 2023-12-01T01:52:04.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.211+0000 7f0760acf640 1 -- 172.21.15.115:0/3001054233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075c067850 con 0x7f075c0b0e70 2023-12-01T01:52:04.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.212+0000 7f0743fff640 1 -- 172.21.15.115:0/3001054233 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f075000bd10 con 0x7f075c0b0e70 2023-12-01T01:52:04.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.213+0000 7f0743fff640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f074806a550 0x7f074806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.213+0000 7f0743fff640 1 -- 172.21.15.115:0/3001054233 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0750088c90 con 0x7f075c0b0e70 2023-12-01T01:52:04.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.213+0000 7f075b7fe640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f074806a550 0x7f074806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.214+0000 7f075b7fe640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f074806a550 0x7f074806ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f074c035000 tx=0x7f074c030000 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:04.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.217+0000 7f0743fff640 1 -- 172.21.15.115:0/3001054233 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f07500544c0 con 0x7f075c0b0e70 2023-12-01T01:52:04.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.410+0000 7f0760acf640 1 -- 172.21.15.115:0/3001054233 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f075c0c8f50 con 0x7f075c0b0e70 2023-12-01T01:52:04.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.411+0000 7f0743fff640 1 -- 172.21.15.115:0/3001054233 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) v1 ==== 84+0+65 (secure 0 0 0) 0x7f075c0c8f50 con 0x7f075c0b0e70 2023-12-01T01:52:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:04.413 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:52:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T01:52:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T01:52:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:04.414 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:52:04.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.413+0000 7f0760acf640 1 -- 172.21.15.115:0/3001054233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f074806a550 msgr2=0x7f074806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:04.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.413+0000 7f0760acf640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f074806a550 0x7f074806ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f074c035000 tx=0x7f074c030000 comp rx=0 tx=0).stop 2023-12-01T01:52:04.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.413+0000 7f0760acf640 1 -- 172.21.15.115:0/3001054233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 msgr2=0x7f075c14ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:04.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.413+0000 7f0760acf640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 0x7f075c14ba90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f075000d800 tx=0x7f075000dcc0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.414+0000 7f0760acf640 1 -- 172.21.15.115:0/3001054233 shutdown_connections 2023-12-01T01:52:04.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.414+0000 7f0760acf640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f074806a550 0x7f074806ca10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.414+0000 7f0760acf640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f075c0b3ba0 0x7f075c14bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.414+0000 7f0760acf640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f075c0b0e70 0x7f075c14ba90 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.414+0000 7f0760acf640 1 --2- 172.21.15.115:0/3001054233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c06b6a0 0x7f075c0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.414+0000 7f0760acf640 1 -- 172.21.15.115:0/3001054233 >> 172.21.15.115:0/3001054233 conn(0x7f075c0ce620 msgr2=0x7f075c06f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:04.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.414+0000 7f0760acf640 1 -- 172.21.15.115:0/3001054233 shutdown_connections 2023-12-01T01:52:04.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.414+0000 7f0760acf640 1 -- 172.21.15.115:0/3001054233 wait complete. 2023-12-01T01:52:04.434 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T01:52:04.434 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T01:52:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.651+0000 7f0162b81640 1 Processor -- start 2023-12-01T01:52:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.651+0000 7f0162b81640 1 -- start start 2023-12-01T01:52:04.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.652+0000 7f0162b81640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f015c15ce40 0x7f015c0b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f0162b81640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f015c0b4d50 0x7f015c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f0162b81640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c0d1460 con 0x7f015c0d2730 2023-12-01T01:52:04.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f0162b81640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c15d220 con 0x7f015c15ce40 2023-12-01T01:52:04.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f0162b81640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0d1690 con 0x7f015c0b4d50 2023-12-01T01:52:04.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f016091d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f015c15ce40 0x7f015c0b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f016091d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f015c15ce40 0x7f015c0b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34666/0 (socket says 172.21.15.115:34666) 2023-12-01T01:52:04.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f016091d640 1 -- 172.21.15.115:0/1683785540 learned_addr learned my addr 172.21.15.115:0/1683785540 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:04.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f016111e640 1 --2- 172.21.15.115:0/1683785540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f015c0b4d50 0x7f015c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f016091d640 1 -- 172.21.15.115:0/1683785540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f015c0b4d50 msgr2=0x7f015c0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:04.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f016091d640 1 --2- 172.21.15.115:0/1683785540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f015c0b4d50 0x7f015c0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f016091d640 1 -- 172.21.15.115:0/1683785540 >> v1:172.21.15.142:6789/0 conn(0x7f015c0d2730 legacy=0x7f015c0b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.653+0000 7f016091d640 1 -- 172.21.15.115:0/1683785540 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c0d1970 con 0x7f015c15ce40 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.655+0000 7f016091d640 1 --2- 172.21.15.115:0/1683785540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c15ce40 0x7f015c0b8710 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f015c0cf5a0 tx=0x7f015002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa17e37e847e588 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.655+0000 7f015b7fe640 1 -- 172.21.15.115:0/1683785540 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500317c0 con 0x7f015c15ce40 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.655+0000 7f015b7fe640 1 -- 172.21.15.115:0/1683785540 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0150031920 con 0x7f015c15ce40 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.655+0000 7f015b7fe640 1 -- 172.21.15.115:0/1683785540 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150031cb0 con 0x7f015c15ce40 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.655+0000 7f0162b81640 1 -- 172.21.15.115:0/1683785540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c15ce40 msgr2=0x7f015c0b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.655+0000 7f0162b81640 1 --2- 172.21.15.115:0/1683785540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c15ce40 0x7f015c0b8710 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f015c0cf5a0 tx=0x7f015002b860 comp rx=0 tx=0).stop 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.656+0000 7f0162b81640 1 -- 172.21.15.115:0/1683785540 shutdown_connections 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.656+0000 7f0162b81640 1 --2- 172.21.15.115:0/1683785540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f015c0b4d50 0x7f015c0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.656+0000 7f0162b81640 1 --2- 172.21.15.115:0/1683785540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c15ce40 0x7f015c0b8710 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.656+0000 7f0162b81640 1 -- 172.21.15.115:0/1683785540 >> 172.21.15.115:0/1683785540 conn(0x7f015c065f90 msgr2=0x7f015c0cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.656+0000 7f0162b81640 1 -- 172.21.15.115:0/1683785540 shutdown_connections 2023-12-01T01:52:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.656+0000 7f0162b81640 1 -- 172.21.15.115:0/1683785540 wait complete. 2023-12-01T01:52:04.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.657+0000 7f0162b81640 1 Processor -- start 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.658+0000 7f0162b81640 1 -- start start 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f0162b81640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c0b4d50 0x7f015c1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f0162b81640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c0d2730 0x7f015c143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f0162b81640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f015c15ce40 0x7f015c145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f0162b81640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c144ba0 con 0x7f015c0d2730 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f0162b81640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c143cc0 con 0x7f015c0b4d50 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f0162b81640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c145570 con 0x7f015c15ce40 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f015bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c0d2730 0x7f015c143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f015bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c0d2730 0x7f015c143780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34672/0 (socket says 172.21.15.115:34672) 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f015bfff640 1 -- 172.21.15.115:0/1987294426 learned_addr learned my addr 172.21.15.115:0/1987294426 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f016091d640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c0b4d50 0x7f015c1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f016111e640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f015c15ce40 0x7f015c145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f015bfff640 1 -- 172.21.15.115:0/1987294426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f015c15ce40 msgr2=0x7f015c145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f015bfff640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f015c15ce40 0x7f015c145030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f015bfff640 1 -- 172.21.15.115:0/1987294426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c0b4d50 msgr2=0x7f015c1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f015bfff640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c0b4d50 0x7f015c1555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f015bfff640 1 -- 172.21.15.115:0/1987294426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01500094d0 con 0x7f015c0d2730 2023-12-01T01:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.659+0000 7f016091d640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c0b4d50 0x7f015c1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.660+0000 7f015bfff640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c0d2730 0x7f015c143780 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f014800c400 tx=0x7f014800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.660+0000 7f01597fa640 1 -- 172.21.15.115:0/1987294426 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148011790 con 0x7f015c0d2730 2023-12-01T01:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.660+0000 7f01597fa640 1 -- 172.21.15.115:0/1987294426 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0148011930 con 0x7f015c0d2730 2023-12-01T01:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.660+0000 7f01597fa640 1 -- 172.21.15.115:0/1987294426 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148011ce0 con 0x7f015c0d2730 2023-12-01T01:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.660+0000 7f0162b81640 1 -- 172.21.15.115:0/1987294426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c143f90 con 0x7f015c0d2730 2023-12-01T01:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.660+0000 7f0162b81640 1 -- 172.21.15.115:0/1987294426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c144550 con 0x7f015c0d2730 2023-12-01T01:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.661+0000 7f0162b81640 1 -- 172.21.15.115:0/1987294426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c144d50 con 0x7f015c0d2730 2023-12-01T01:52:04.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.662+0000 7f01597fa640 1 -- 172.21.15.115:0/1987294426 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f014801c460 con 0x7f015c0d2730 2023-12-01T01:52:04.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.663+0000 7f01597fa640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f014006a580 0x7f014006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:04.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.663+0000 7f01597fa640 1 -- 172.21.15.115:0/1987294426 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0148088710 con 0x7f015c0d2730 2023-12-01T01:52:04.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.663+0000 7f016091d640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f014006a580 0x7f014006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:04.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.664+0000 7f016091d640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f014006a580 0x7f014006ca40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0150036000 tx=0x7f015003d040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:04.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.667+0000 7f01597fa640 1 -- 172.21.15.115:0/1987294426 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f015c144d50 con 0x7f015c0d2730 2023-12-01T01:52:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.852+0000 7f0162b81640 1 -- 172.21.15.115:0/1987294426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f015c074210 con 0x7f015c0d2730 2023-12-01T01:52:04.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.969+0000 7f01597fa640 1 -- 172.21.15.115:0/1987294426 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v94) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0148053ea0 con 0x7f015c0d2730 2023-12-01T01:52:04.973 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T01:52:04.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.972+0000 7f0162b81640 1 -- 172.21.15.115:0/1987294426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f014006a580 msgr2=0x7f014006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:04.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.972+0000 7f0162b81640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f014006a580 0x7f014006ca40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0150036000 tx=0x7f015003d040 comp rx=0 tx=0).stop 2023-12-01T01:52:04.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.972+0000 7f0162b81640 1 -- 172.21.15.115:0/1987294426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c0d2730 msgr2=0x7f015c143780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.972+0000 7f0162b81640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c0d2730 0x7f015c143780 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f014800c400 tx=0x7f014800c8c0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.973+0000 7f0162b81640 1 -- 172.21.15.115:0/1987294426 shutdown_connections 2023-12-01T01:52:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.973+0000 7f0162b81640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f014006a580 0x7f014006ca40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.973+0000 7f0162b81640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f015c15ce40 0x7f015c145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.973+0000 7f0162b81640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f015c0d2730 0x7f015c143780 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.973+0000 7f0162b81640 1 --2- 172.21.15.115:0/1987294426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f015c0b4d50 0x7f015c1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.973+0000 7f0162b81640 1 -- 172.21.15.115:0/1987294426 >> 172.21.15.115:0/1987294426 conn(0x7f015c065f90 msgr2=0x7f015c0b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.973+0000 7f0162b81640 1 -- 172.21.15.115:0/1987294426 shutdown_connections 2023-12-01T01:52:04.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:04.973+0000 7f0162b81640 1 -- 172.21.15.115:0/1987294426 wait complete. 2023-12-01T01:52:04.998 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T01:52:05.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.215+0000 7fbe4e5cb640 1 Processor -- start 2023-12-01T01:52:05.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.215+0000 7fbe4e5cb640 1 -- start start 2023-12-01T01:52:05.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.217+0000 7fbe4e5cb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbe48157250 0x7fbe4815dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:05.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.217+0000 7fbe4e5cb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbe480af820 0x7fbe480afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:05.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.217+0000 7fbe4e5cb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4815e2b0 con 0x7fbe481608e0 2023-12-01T01:52:05.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.217+0000 7fbe4e5cb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe480b0210 con 0x7fbe480af820 2023-12-01T01:52:05.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.217+0000 7fbe4e5cb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4806b280 con 0x7fbe48157250 2023-12-01T01:52:05.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.217+0000 7fbe477fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbe480af820 0x7fbe480afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:05.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.217+0000 7fbe477fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbe480af820 0x7fbe480afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34678/0 (socket says 172.21.15.115:34678) 2023-12-01T01:52:05.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.217+0000 7fbe477fe640 1 -- 172.21.15.115:0/3931029107 learned_addr learned my addr 172.21.15.115:0/3931029107 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:05.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.217+0000 7fbe47fff640 1 --2- 172.21.15.115:0/3931029107 >> v2:172.21.15.142:3300/0 conn(0x7fbe48157250 0x7fbe4815dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:05.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.218+0000 7fbe477fe640 1 -- 172.21.15.115:0/3931029107 >> v2:172.21.15.142:3300/0 conn(0x7fbe48157250 msgr2=0x7fbe4815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:05.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.218+0000 7fbe477fe640 1 --2- 172.21.15.115:0/3931029107 >> v2:172.21.15.142:3300/0 conn(0x7fbe48157250 0x7fbe4815dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:05.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.218+0000 7fbe477fe640 1 -- 172.21.15.115:0/3931029107 >> v1:172.21.15.115:6789/0 conn(0x7fbe481608e0 legacy=0x7fbe48160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:05.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.218+0000 7fbe477fe640 1 -- 172.21.15.115:0/3931029107 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4806b520 con 0x7fbe480af820 2023-12-01T01:52:05.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.218+0000 7fbe46ffd640 1 -- 172.21.15.115:0/3931029107 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891746045 0 0) 0x7fbe4815e2b0 con 0x7fbe481608e0 2023-12-01T01:52:05.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.219+0000 7fbe477fe640 1 --2- 172.21.15.115:0/3931029107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe480af820 0x7fbe480afcd0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fbe4806c3d0 tx=0x7fbe3c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd07d4c5a627a253 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:05.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe46ffd640 1 -- 172.21.15.115:0/3931029107 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c030860 con 0x7fbe480af820 2023-12-01T01:52:05.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe46ffd640 1 -- 172.21.15.115:0/3931029107 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c0309c0 con 0x7fbe480af820 2023-12-01T01:52:05.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe46ffd640 1 -- 172.21.15.115:0/3931029107 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c030d50 con 0x7fbe480af820 2023-12-01T01:52:05.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/3931029107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe480af820 msgr2=0x7fbe480afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:05.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe4e5cb640 1 --2- 172.21.15.115:0/3931029107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe480af820 0x7fbe480afcd0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fbe4806c3d0 tx=0x7fbe3c02b9a0 comp rx=0 tx=0).stop 2023-12-01T01:52:05.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/3931029107 shutdown_connections 2023-12-01T01:52:05.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe4e5cb640 1 --2- 172.21.15.115:0/3931029107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe480af820 0x7fbe480afcd0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:05.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe4e5cb640 1 --2- 172.21.15.115:0/3931029107 >> v2:172.21.15.142:3300/0 conn(0x7fbe48157250 0x7fbe4815dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:05.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/3931029107 >> 172.21.15.115:0/3931029107 conn(0x7fbe48065960 msgr2=0x7fbe48065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:05.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/3931029107 shutdown_connections 2023-12-01T01:52:05.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.220+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/3931029107 wait complete. 2023-12-01T01:52:05.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.222+0000 7fbe4e5cb640 1 Processor -- start 2023-12-01T01:52:05.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.222+0000 7fbe4e5cb640 1 -- start start 2023-12-01T01:52:05.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.223+0000 7fbe4e5cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe480af820 0x7fbe48145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:05.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.223+0000 7fbe4e5cb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe48157250 0x7fbe4806f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:05.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.223+0000 7fbe4e5cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe481608e0 0x7fbe4806fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:05.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.223+0000 7fbe4e5cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4815ee10 con 0x7fbe481608e0 2023-12-01T01:52:05.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.223+0000 7fbe4e5cb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe48145aa0 con 0x7fbe48157250 2023-12-01T01:52:05.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.223+0000 7fbe4e5cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4815f060 con 0x7fbe480af820 2023-12-01T01:52:05.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.223+0000 7fbe4cb68640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe481608e0 0x7fbe4806fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:05.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe4cb68640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe481608e0 0x7fbe4806fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34694/0 (socket says 172.21.15.115:34694) 2023-12-01T01:52:05.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe4cb68640 1 -- 172.21.15.115:0/724725916 learned_addr learned my addr 172.21.15.115:0/724725916 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:05.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe4cb68640 1 -- 172.21.15.115:0/724725916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe480af820 msgr2=0x7fbe48145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:05.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe4cb68640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe480af820 0x7fbe48145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:05.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe4cb68640 1 -- 172.21.15.115:0/724725916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe48157250 msgr2=0x7fbe4806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:05.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe4cb68640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe48157250 0x7fbe4806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:05.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe4cb68640 1 -- 172.21.15.115:0/724725916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0094d0 con 0x7fbe481608e0 2023-12-01T01:52:05.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe4cb68640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe481608e0 0x7fbe4806fe70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000c780 tx=0x7fbe4000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:05.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe44ff9640 1 -- 172.21.15.115:0/724725916 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40003b70 con 0x7fbe481608e0 2023-12-01T01:52:05.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe44ff9640 1 -- 172.21.15.115:0/724725916 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe40003cd0 con 0x7fbe481608e0 2023-12-01T01:52:05.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe44ff9640 1 -- 172.21.15.115:0/724725916 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40005290 con 0x7fbe481608e0 2023-12-01T01:52:05.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.224+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/724725916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4815f340 con 0x7fbe481608e0 2023-12-01T01:52:05.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.225+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/724725916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4815f900 con 0x7fbe481608e0 2023-12-01T01:52:05.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.226+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/724725916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4804e890 con 0x7fbe481608e0 2023-12-01T01:52:05.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.227+0000 7fbe44ff9640 1 -- 172.21.15.115:0/724725916 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fbe400053f0 con 0x7fbe481608e0 2023-12-01T01:52:05.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.229+0000 7fbe44ff9640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe3406a550 0x7fbe3406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:05.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.229+0000 7fbe44ff9640 1 -- 172.21.15.115:0/724725916 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbe400891c0 con 0x7fbe481608e0 2023-12-01T01:52:05.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.229+0000 7fbe47fff640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe3406a550 0x7fbe3406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:05.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.230+0000 7fbe47fff640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe3406a550 0x7fbe3406ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe380098f0 tx=0x7fbe38008040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:05.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.233+0000 7fbe44ff9640 1 -- 172.21.15.115:0/724725916 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbe400582f0 con 0x7fbe481608e0 2023-12-01T01:52:05.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.423+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/724725916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbe480b2d20 con 0x7fbe481608e0 2023-12-01T01:52:05.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.980+0000 7fbe44ff9640 1 -- 172.21.15.115:0/724725916 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v95) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbe40001730 con 0x7fbe481608e0 2023-12-01T01:52:05.983 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T01:52:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.987+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/724725916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe3406a550 msgr2=0x7fbe3406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.987+0000 7fbe4e5cb640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe3406a550 0x7fbe3406ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe380098f0 tx=0x7fbe38008040 comp rx=0 tx=0).stop 2023-12-01T01:52:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.987+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/724725916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe481608e0 msgr2=0x7fbe4806fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.987+0000 7fbe4e5cb640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe481608e0 0x7fbe4806fe70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000c780 tx=0x7fbe4000cb80 comp rx=0 tx=0).stop 2023-12-01T01:52:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.987+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/724725916 shutdown_connections 2023-12-01T01:52:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.987+0000 7fbe4e5cb640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe3406a550 0x7fbe3406ca10 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe380098f0 tx=0x7fbe38008040 comp rx=0 tx=0).stop 2023-12-01T01:52:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.987+0000 7fbe4e5cb640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe481608e0 0x7fbe4806fe70 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.987+0000 7fbe4e5cb640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe48157250 0x7fbe4806f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.987+0000 7fbe4e5cb640 1 --2- 172.21.15.115:0/724725916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe480af820 0x7fbe48145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:05.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.987+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/724725916 >> 172.21.15.115:0/724725916 conn(0x7fbe48065960 msgr2=0x7fbe48154590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:05.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.991+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/724725916 shutdown_connections 2023-12-01T01:52:05.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:05.991+0000 7fbe4e5cb640 1 -- 172.21.15.115:0/724725916 wait complete. 2023-12-01T01:52:06.022 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T01:52:06.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.206+0000 7fb1359d2640 1 Processor -- start 2023-12-01T01:52:06.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.207+0000 7fb1359d2640 1 -- start start 2023-12-01T01:52:06.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.208+0000 7fb1359d2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb130142870 0x7fb1300ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.208+0000 7fb1359d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1300cd010 0x7fb1300b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.208+0000 7fb1359d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1300cd470 con 0x7fb1300b1c50 2023-12-01T01:52:06.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.208+0000 7fb1359d2640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1300b3ae0 con 0x7fb130142870 2023-12-01T01:52:06.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.208+0000 7fb1359d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb130142c50 con 0x7fb1300cd010 2023-12-01T01:52:06.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.209+0000 7fb12effd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb130142870 0x7fb1300ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:06.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.209+0000 7fb12effd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb130142870 0x7fb1300ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34698/0 (socket says 172.21.15.115:34698) 2023-12-01T01:52:06.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.209+0000 7fb12effd640 1 -- 172.21.15.115:0/3063344274 learned_addr learned my addr 172.21.15.115:0/3063344274 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:06.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.209+0000 7fb1267fc640 1 --2- 172.21.15.115:0/3063344274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1300cd010 0x7fb1300b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:06.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.209+0000 7fb12effd640 1 -- 172.21.15.115:0/3063344274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1300cd010 msgr2=0x7fb1300b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:06.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.209+0000 7fb12effd640 1 --2- 172.21.15.115:0/3063344274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1300cd010 0x7fb1300b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.209+0000 7fb12effd640 1 -- 172.21.15.115:0/3063344274 >> v1:172.21.15.142:6789/0 conn(0x7fb1300b1c50 legacy=0x7fb13015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:06.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.209+0000 7fb12effd640 1 -- 172.21.15.115:0/3063344274 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1300b20e0 con 0x7fb130142870 2023-12-01T01:52:06.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.210+0000 7fb12effd640 1 --2- 172.21.15.115:0/3063344274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 0x7fb1300ccad0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb12000bb40 tx=0x7fb12002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d622ad93a4f15c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:06.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.211+0000 7fb12e7fc640 1 -- 172.21.15.115:0/3063344274 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12002ee70 con 0x7fb130142870 2023-12-01T01:52:06.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.211+0000 7fb12e7fc640 1 -- 172.21.15.115:0/3063344274 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb120036440 con 0x7fb130142870 2023-12-01T01:52:06.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.211+0000 7fb12e7fc640 1 -- 172.21.15.115:0/3063344274 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1200367d0 con 0x7fb130142870 2023-12-01T01:52:06.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.211+0000 7fb1359d2640 1 -- 172.21.15.115:0/3063344274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 msgr2=0x7fb1300ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.211+0000 7fb1359d2640 1 --2- 172.21.15.115:0/3063344274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 0x7fb1300ccad0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb12000bb40 tx=0x7fb12002e0c0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.212+0000 7fb1359d2640 1 -- 172.21.15.115:0/3063344274 shutdown_connections 2023-12-01T01:52:06.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.212+0000 7fb1359d2640 1 --2- 172.21.15.115:0/3063344274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1300cd010 0x7fb1300b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.212+0000 7fb1359d2640 1 --2- 172.21.15.115:0/3063344274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 0x7fb1300ccad0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.212+0000 7fb1359d2640 1 -- 172.21.15.115:0/3063344274 >> 172.21.15.115:0/3063344274 conn(0x7fb130065960 msgr2=0x7fb130065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:06.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.212+0000 7fb1359d2640 1 -- 172.21.15.115:0/3063344274 shutdown_connections 2023-12-01T01:52:06.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.212+0000 7fb1359d2640 1 -- 172.21.15.115:0/3063344274 wait complete. 2023-12-01T01:52:06.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.213+0000 7fb1359d2640 1 Processor -- start 2023-12-01T01:52:06.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.213+0000 7fb1359d2640 1 -- start start 2023-12-01T01:52:06.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.214+0000 7fb1359d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1300b1c50 0x7fb1300cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.214+0000 7fb1359d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1300cd010 0x7fb1300c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.214+0000 7fb1359d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 0x7fb13013b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.214+0000 7fb1359d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13013b740 con 0x7fb130142870 2023-12-01T01:52:06.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.214+0000 7fb1359d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb130158e10 con 0x7fb1300b1c50 2023-12-01T01:52:06.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.214+0000 7fb1359d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1300c9360 con 0x7fb1300cd010 2023-12-01T01:52:06.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.215+0000 7fb12f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 0x7fb13013b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:06.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.215+0000 7fb12f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 0x7fb13013b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34710/0 (socket says 172.21.15.115:34710) 2023-12-01T01:52:06.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.215+0000 7fb12f7fe640 1 -- 172.21.15.115:0/2325330268 learned_addr learned my addr 172.21.15.115:0/2325330268 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:06.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.215+0000 7fb12effd640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1300b1c50 0x7fb1300cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:06.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.215+0000 7fb12f7fe640 1 -- 172.21.15.115:0/2325330268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1300cd010 msgr2=0x7fb1300c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.215+0000 7fb12f7fe640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1300cd010 0x7fb1300c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.215+0000 7fb12f7fe640 1 -- 172.21.15.115:0/2325330268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1300b1c50 msgr2=0x7fb1300cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.215+0000 7fb12f7fe640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1300b1c50 0x7fb1300cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.215+0000 7fb12f7fe640 1 -- 172.21.15.115:0/2325330268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12000b770 con 0x7fb130142870 2023-12-01T01:52:06.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.215+0000 7fb12effd640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1300b1c50 0x7fb1300cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:06.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.216+0000 7fb12f7fe640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 0x7fb13013b200 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fb12800c7b0 tx=0x7fb12800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:06.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.216+0000 7fb12cff9640 1 -- 172.21.15.115:0/2325330268 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128004b60 con 0x7fb130142870 2023-12-01T01:52:06.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.216+0000 7fb12cff9640 1 -- 172.21.15.115:0/2325330268 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb128004cc0 con 0x7fb130142870 2023-12-01T01:52:06.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.216+0000 7fb1359d2640 1 -- 172.21.15.115:0/2325330268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1300c8750 con 0x7fb130142870 2023-12-01T01:52:06.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.216+0000 7fb12cff9640 1 -- 172.21.15.115:0/2325330268 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12800aa90 con 0x7fb130142870 2023-12-01T01:52:06.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.216+0000 7fb1359d2640 1 -- 172.21.15.115:0/2325330268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1300c8d10 con 0x7fb130142870 2023-12-01T01:52:06.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.217+0000 7fb1359d2640 1 -- 172.21.15.115:0/2325330268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13014f750 con 0x7fb130142870 2023-12-01T01:52:06.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.219+0000 7fb12cff9640 1 -- 172.21.15.115:0/2325330268 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb12800abf0 con 0x7fb130142870 2023-12-01T01:52:06.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.219+0000 7fb12cff9640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb11006a580 0x7fb11006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.219+0000 7fb12cff9640 1 -- 172.21.15.115:0/2325330268 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb128088ba0 con 0x7fb130142870 2023-12-01T01:52:06.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.220+0000 7fb12effd640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb11006a580 0x7fb11006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:06.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.220+0000 7fb12effd640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb11006a580 0x7fb11006ca40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb120038000 tx=0x7fb120008040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:06.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.224+0000 7fb12cff9640 1 -- 172.21.15.115:0/2325330268 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb1280540e0 con 0x7fb130142870 2023-12-01T01:52:06.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.451+0000 7fb1359d2640 1 -- 172.21.15.115:0/2325330268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb13004e890 con 0x7fb130142870 2023-12-01T01:52:06.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.461+0000 7fb12cff9640 1 -- 172.21.15.115:0/2325330268 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v36) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb128057700 con 0x7fb130142870 2023-12-01T01:52:06.466 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 10 and data pool 11 2023-12-01T01:52:06.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.468+0000 7fb1359d2640 1 -- 172.21.15.115:0/2325330268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb11006a580 msgr2=0x7fb11006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.468+0000 7fb1359d2640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb11006a580 0x7fb11006ca40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb120038000 tx=0x7fb120008040 comp rx=0 tx=0).stop 2023-12-01T01:52:06.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.468+0000 7fb1359d2640 1 -- 172.21.15.115:0/2325330268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 msgr2=0x7fb13013b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.468+0000 7fb1359d2640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 0x7fb13013b200 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fb12800c7b0 tx=0x7fb12800cc70 comp rx=0 tx=0).stop 2023-12-01T01:52:06.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.469+0000 7fb1359d2640 1 -- 172.21.15.115:0/2325330268 shutdown_connections 2023-12-01T01:52:06.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.469+0000 7fb1359d2640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb11006a580 0x7fb11006ca40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.469+0000 7fb1359d2640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1300cd010 0x7fb1300c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.469+0000 7fb1359d2640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb130142870 0x7fb13013b200 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.469+0000 7fb1359d2640 1 --2- 172.21.15.115:0/2325330268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1300b1c50 0x7fb1300cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.469+0000 7fb1359d2640 1 -- 172.21.15.115:0/2325330268 >> 172.21.15.115:0/2325330268 conn(0x7fb130065960 msgr2=0x7fb13006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:06.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.470+0000 7fb1359d2640 1 -- 172.21.15.115:0/2325330268 shutdown_connections 2023-12-01T01:52:06.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.470+0000 7fb1359d2640 1 -- 172.21.15.115:0/2325330268 wait complete. 2023-12-01T01:52:06.502 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.675+0000 7f6709955640 1 Processor -- start 2023-12-01T01:52:06.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.675+0000 7f6709955640 1 -- start start 2023-12-01T01:52:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.676+0000 7f6709955640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f67040c9ef0 0x7f670406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.676+0000 7f6709955640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f670406b6f0 0x7f6704070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.676+0000 7f6709955640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670406f8b0 con 0x7f6704159dd0 2023-12-01T01:52:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.676+0000 7f6709955640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67040adcb0 con 0x7f670406b6f0 2023-12-01T01:52:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.676+0000 7f6709955640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67040ad9d0 con 0x7f67040c9ef0 2023-12-01T01:52:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.677+0000 7f6702ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6704159dd0 0x7f67040c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:55026/0 (socket says 172.21.15.115:55026) 2023-12-01T01:52:06.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.677+0000 7f6702ffd640 1 -- 172.21.15.115:0/3014182470 learned_addr learned my addr 172.21.15.115:0/3014182470 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:06.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.677+0000 7f67037fe640 1 --2- 172.21.15.115:0/3014182470 >> v2:172.21.15.115:3300/0 conn(0x7f670406b6f0 0x7f6704070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.677+0000 7f67027fc640 1 -- 172.21.15.115:0/3014182470 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068415166 0 0) 0x7f670406f8b0 con 0x7f6704159dd0 2023-12-01T01:52:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.677+0000 7f67027fc640 1 -- 172.21.15.115:0/3014182470 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f0003520 con 0x7f6704159dd0 2023-12-01T01:52:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.677+0000 7f67037fe640 1 -- 172.21.15.115:0/3014182470 >> v2:172.21.15.142:3300/0 conn(0x7f67040c9ef0 msgr2=0x7f670406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.677+0000 7f67037fe640 1 --2- 172.21.15.115:0/3014182470 >> v2:172.21.15.142:3300/0 conn(0x7f67040c9ef0 0x7f670406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.677+0000 7f67037fe640 1 -- 172.21.15.115:0/3014182470 >> v1:172.21.15.115:6789/0 conn(0x7f6704159dd0 legacy=0x7f67040c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.677+0000 7f67037fe640 1 -- 172.21.15.115:0/3014182470 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67040c9000 con 0x7f670406b6f0 2023-12-01T01:52:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.678+0000 7f67037fe640 1 --2- 172.21.15.115:0/3014182470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f670406b6f0 0x7f6704070260 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f670406de80 tx=0x7f66fc02b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b72ad19bdbbcc54 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.678+0000 7f67027fc640 1 -- 172.21.15.115:0/3014182470 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc031520 con 0x7f670406b6f0 2023-12-01T01:52:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.678+0000 7f67027fc640 1 -- 172.21.15.115:0/3014182470 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66fc031680 con 0x7f670406b6f0 2023-12-01T01:52:06.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.679+0000 7f67027fc640 1 -- 172.21.15.115:0/3014182470 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0319f0 con 0x7f670406b6f0 2023-12-01T01:52:06.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.679+0000 7f6709955640 1 -- 172.21.15.115:0/3014182470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f670406b6f0 msgr2=0x7f6704070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.679+0000 7f6709955640 1 --2- 172.21.15.115:0/3014182470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f670406b6f0 0x7f6704070260 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f670406de80 tx=0x7f66fc02b7c0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.679+0000 7f6709955640 1 -- 172.21.15.115:0/3014182470 shutdown_connections 2023-12-01T01:52:06.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.679+0000 7f6709955640 1 --2- 172.21.15.115:0/3014182470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f670406b6f0 0x7f6704070260 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.679+0000 7f6709955640 1 --2- 172.21.15.115:0/3014182470 >> v2:172.21.15.142:3300/0 conn(0x7f67040c9ef0 0x7f670406fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.679+0000 7f6709955640 1 -- 172.21.15.115:0/3014182470 >> 172.21.15.115:0/3014182470 conn(0x7f67040ce880 msgr2=0x7f67040cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:06.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.679+0000 7f6709955640 1 -- 172.21.15.115:0/3014182470 shutdown_connections 2023-12-01T01:52:06.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.679+0000 7f6709955640 1 -- 172.21.15.115:0/3014182470 wait complete. 2023-12-01T01:52:06.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.681+0000 7f6709955640 1 Processor -- start 2023-12-01T01:52:06.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.681+0000 7f6709955640 1 -- start start 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.682+0000 7f6709955640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f670406b6f0 0x7f67040acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.682+0000 7f6709955640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67040c9ef0 0x7f6704149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.682+0000 7f6709955640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704159dd0 0x7f67040c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.682+0000 7f6709955640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67041e4480 con 0x7f67040c9ef0 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.682+0000 7f6709955640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67041e3870 con 0x7f6704159dd0 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.682+0000 7f6709955640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67041e3b50 con 0x7f670406b6f0 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.682+0000 7f66fa7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67040c9ef0 0x7f6704149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.682+0000 7f66fa7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67040c9ef0 0x7f6704149710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34734/0 (socket says 172.21.15.115:34734) 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.682+0000 7f66fa7fc640 1 -- 172.21.15.115:0/187502236 learned_addr learned my addr 172.21.15.115:0/187502236 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f6702ffd640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f670406b6f0 0x7f67040acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f67037fe640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704159dd0 0x7f67040c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f66fa7fc640 1 -- 172.21.15.115:0/187502236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f670406b6f0 msgr2=0x7f67040acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f66fa7fc640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f670406b6f0 0x7f67040acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f66fa7fc640 1 -- 172.21.15.115:0/187502236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704159dd0 msgr2=0x7f67040c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f66fa7fc640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704159dd0 0x7f67040c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f66fa7fc640 1 -- 172.21.15.115:0/187502236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc009400 con 0x7f67040c9ef0 2023-12-01T01:52:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f66fa7fc640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67040c9ef0 0x7f6704149710 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00b4a0 tx=0x7f66ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f6700ff9640 1 -- 172.21.15.115:0/187502236 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec010610 con 0x7f67040c9ef0 2023-12-01T01:52:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f6700ff9640 1 -- 172.21.15.115:0/187502236 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec010770 con 0x7f67040c9ef0 2023-12-01T01:52:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f6700ff9640 1 -- 172.21.15.115:0/187502236 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec010b20 con 0x7f67040c9ef0 2023-12-01T01:52:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f6709955640 1 -- 172.21.15.115:0/187502236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67041e3e30 con 0x7f67040c9ef0 2023-12-01T01:52:06.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.683+0000 7f6709955640 1 -- 172.21.15.115:0/187502236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67040ad1d0 con 0x7f67040c9ef0 2023-12-01T01:52:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.684+0000 7f6709955640 1 -- 172.21.15.115:0/187502236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670404e890 con 0x7f67040c9ef0 2023-12-01T01:52:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.686+0000 7f6700ff9640 1 -- 172.21.15.115:0/187502236 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f66ec010c80 con 0x7f67040c9ef0 2023-12-01T01:52:06.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.686+0000 7f6700ff9640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66e406a550 0x7f66e406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.687+0000 7f6700ff9640 1 -- 172.21.15.115:0/187502236 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66ec088690 con 0x7f67040c9ef0 2023-12-01T01:52:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.687+0000 7f6702ffd640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66e406a550 0x7f66e406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:06.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.687+0000 7f6702ffd640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66e406a550 0x7f66e406ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66f4005c00 tx=0x7f66f4009040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:06.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.691+0000 7f6700ff9640 1 -- 172.21.15.115:0/187502236 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f66ec053b70 con 0x7f67040c9ef0 2023-12-01T01:52:06.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.875+0000 7f6709955640 1 -- 172.21.15.115:0/187502236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f670406d2c0 con 0x7f67040c9ef0 2023-12-01T01:52:06.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.878+0000 7f6700ff9640 1 -- 172.21.15.115:0/187502236 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f66ec057190 con 0x7f67040c9ef0 2023-12-01T01:52:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:06.880 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":96,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:06.453651+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:04.909384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:05.480768+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:06.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.880+0000 7f6709955640 1 -- 172.21.15.115:0/187502236 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66e406a550 msgr2=0x7f66e406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.880+0000 7f6709955640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66e406a550 0x7f66e406ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66f4005c00 tx=0x7f66f4009040 comp rx=0 tx=0).stop 2023-12-01T01:52:06.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.880+0000 7f6709955640 1 -- 172.21.15.115:0/187502236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67040c9ef0 msgr2=0x7f6704149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:06.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.880+0000 7f6709955640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67040c9ef0 0x7f6704149710 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00b4a0 tx=0x7f66ec00b960 comp rx=0 tx=0).stop 2023-12-01T01:52:06.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.880+0000 7f6709955640 1 -- 172.21.15.115:0/187502236 shutdown_connections 2023-12-01T01:52:06.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.880+0000 7f6709955640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f66e406a550 0x7f66e406ca10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.880+0000 7f6709955640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704159dd0 0x7f67040c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.880+0000 7f6709955640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67040c9ef0 0x7f6704149710 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.880+0000 7f6709955640 1 --2- 172.21.15.115:0/187502236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f670406b6f0 0x7f67040acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:06.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.881+0000 7f6709955640 1 -- 172.21.15.115:0/187502236 >> 172.21.15.115:0/187502236 conn(0x7f67040ce880 msgr2=0x7f6704067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:06.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.881+0000 7f6709955640 1 -- 172.21.15.115:0/187502236 shutdown_connections 2023-12-01T01:52:06.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:06.881+0000 7f6709955640 1 -- 172.21.15.115:0/187502236 wait complete. 2023-12-01T01:52:06.900 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:07.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.098+0000 7f7df6dd2640 1 Processor -- start 2023-12-01T01:52:07.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.098+0000 7f7df6dd2640 1 -- start start 2023-12-01T01:52:07.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.099+0000 7f7df6dd2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00c9ef0 0x7f7df006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.099+0000 7f7df6dd2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df006f8b0 con 0x7f7df0159dd0 2023-12-01T01:52:07.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.099+0000 7f7df6dd2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df00adcb0 con 0x7f7df006b6f0 2023-12-01T01:52:07.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.099+0000 7f7df6dd2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00ad9d0 con 0x7f7df00c9ef0 2023-12-01T01:52:07.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7df4b6e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7df0159dd0 0x7f7df00c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:55036/0 (socket says 172.21.15.115:55036) 2023-12-01T01:52:07.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7df4b6e640 1 -- 172.21.15.115:0/325948276 learned_addr learned my addr 172.21.15.115:0/325948276 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171729526 0 0) 0x7f7df006f8b0 con 0x7f7df0159dd0 2023-12-01T01:52:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dd8005730 con 0x7f7df0159dd0 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de7fff640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00c9ef0 0x7f7df006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2436104480 0 0) 0x7f7dd8005730 con 0x7f7df0159dd0 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00c9ef0 msgr2=0x7f7df006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00c9ef0 0x7f7df006fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 >> v1:172.21.15.142:6789/0 conn(0x7f7df006b6f0 legacy=0x7f7df0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df00c9000 con 0x7f7df0159dd0 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f7dec001670 con 0x7f7df0159dd0 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 >> v1:172.21.15.115:6789/0 conn(0x7f7df0159dd0 legacy=0x7f7df00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd8005c20 0x7f7dd8006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd80069d0 0x7f7dd8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00c9ef0 0x7f7dd800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00ad9d0 con 0x7f7dd80069d0 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd80077d0 con 0x7f7dd8005c20 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.100+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd800e0d0 con 0x7f7df00c9ef0 2023-12-01T01:52:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.101+0000 7f7de7fff640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd80069d0 0x7f7dd8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:07.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.101+0000 7f7df4b6e640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd8005c20 0x7f7dd8006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:07.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.101+0000 7f7de7fff640 1 -- 172.21.15.115:0/325948276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00c9ef0 msgr2=0x7f7dd800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:07.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.101+0000 7f7de7fff640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df00c9ef0 0x7f7dd800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.101+0000 7f7de7fff640 1 -- 172.21.15.115:0/325948276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd8005c20 msgr2=0x7f7dd8006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:07.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.101+0000 7f7de7fff640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd8005c20 0x7f7dd8006020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.101+0000 7f7de7fff640 1 -- 172.21.15.115:0/325948276 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8005730 con 0x7f7dd80069d0 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.102+0000 7f7de7fff640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd80069d0 0x7f7dd8006e50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7de000ba30 tx=0x7f7dd8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e3413d9c15081c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.102+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0032680 con 0x7f7dd80069d0 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.102+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de00327e0 con 0x7f7dd80069d0 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.102+0000 7f7de77fe640 1 -- 172.21.15.115:0/325948276 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0032b70 con 0x7f7dd80069d0 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.103+0000 7f7df6dd2640 1 -- 172.21.15.115:0/325948276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd80069d0 msgr2=0x7f7dd8006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.103+0000 7f7df6dd2640 1 --2- 172.21.15.115:0/325948276 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dd80069d0 0x7f7dd8006e50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7de000ba30 tx=0x7f7dd8007a00 comp rx=0 tx=0).stop 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.103+0000 7f7df536f640 1 -- 172.21.15.115:0/325948276 reap_dead start 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.103+0000 7f7df6dd2640 1 -- 172.21.15.115:0/325948276 shutdown_connections 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.103+0000 7f7df6dd2640 1 -- 172.21.15.115:0/325948276 >> 172.21.15.115:0/325948276 conn(0x7f7df00ce880 msgr2=0x7f7df00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.103+0000 7f7df6dd2640 1 -- 172.21.15.115:0/325948276 shutdown_connections 2023-12-01T01:52:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.103+0000 7f7df6dd2640 1 -- 172.21.15.115:0/325948276 wait complete. 2023-12-01T01:52:07.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.105+0000 7f7df6dd2640 1 Processor -- start 2023-12-01T01:52:07.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.105+0000 7f7df6dd2640 1 -- start start 2023-12-01T01:52:07.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7df6dd2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd80069d0 0x7f7df00c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7df6dd2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7df0153d70 0x7f7df00af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7df6dd2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df01541d0 0x7f7df00afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7df6dd2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00c6700 con 0x7f7df0153d70 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7df6dd2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00c5a80 con 0x7f7dd80069d0 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7df6dd2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00c5d60 con 0x7f7df01541d0 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7de7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7df0153d70 0x7f7df00af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7de7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7df0153d70 0x7f7df00af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34748/0 (socket says 172.21.15.115:34748) 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7de7fff640 1 -- 172.21.15.115:0/607794630 learned_addr learned my addr 172.21.15.115:0/607794630 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7de7fff640 1 -- 172.21.15.115:0/607794630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df01541d0 msgr2=0x7f7df00afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7de7fff640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df01541d0 0x7f7df00afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7de7fff640 1 -- 172.21.15.115:0/607794630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd80069d0 msgr2=0x7f7df00c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7de7fff640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd80069d0 0x7f7df00c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.106+0000 7f7de7fff640 1 -- 172.21.15.115:0/607794630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df00c6040 con 0x7f7df0153d70 2023-12-01T01:52:07.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.107+0000 7f7de7fff640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7df0153d70 0x7f7df00af570 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f7de0035040 tx=0x7f7de0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:07.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.107+0000 7f7de57fa640 1 -- 172.21.15.115:0/607794630 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0030aa0 con 0x7f7df0153d70 2023-12-01T01:52:07.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.107+0000 7f7de57fa640 1 -- 172.21.15.115:0/607794630 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de0030c20 con 0x7f7df0153d70 2023-12-01T01:52:07.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.107+0000 7f7de57fa640 1 -- 172.21.15.115:0/607794630 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de003b420 con 0x7f7df0153d70 2023-12-01T01:52:07.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.107+0000 7f7df6dd2640 1 -- 172.21.15.115:0/607794630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df00c6320 con 0x7f7df0153d70 2023-12-01T01:52:07.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.107+0000 7f7df6dd2640 1 -- 172.21.15.115:0/607794630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df015c5b0 con 0x7f7df0153d70 2023-12-01T01:52:07.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.108+0000 7f7df6dd2640 1 -- 172.21.15.115:0/607794630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df004e890 con 0x7f7df0153d70 2023-12-01T01:52:07.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.109+0000 7f7de57fa640 1 -- 172.21.15.115:0/607794630 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f7de003b580 con 0x7f7df0153d70 2023-12-01T01:52:07.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.110+0000 7f7de57fa640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 0x7f7ddc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.110+0000 7f7de57fa640 1 -- 172.21.15.115:0/607794630 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7de00ab580 con 0x7f7df0153d70 2023-12-01T01:52:07.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.110+0000 7f7df4b6e640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 0x7f7ddc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:07.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.111+0000 7f7df4b6e640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 0x7f7ddc06ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7dec001120 tx=0x7f7dec009040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:07.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.114+0000 7f7de57fa640 1 -- 172.21.15.115:0/607794630 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7de007a4f0 con 0x7f7df0153d70 2023-12-01T01:52:07.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.302+0000 7f7df6dd2640 1 -- 172.21.15.115:0/607794630 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7df00ccaf0 con 0x7f7df0153d70 2023-12-01T01:52:07.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.304+0000 7f7de57fa640 1 -- 172.21.15.115:0/607794630 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16101 (secure 0 0 0) 0x7f7de007a080 con 0x7f7df0153d70 2023-12-01T01:52:07.306 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:07.306 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":96,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:06.453651+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:04.909384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:05.480768+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:07.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.306+0000 7f7df6dd2640 1 -- 172.21.15.115:0/607794630 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 msgr2=0x7f7ddc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:07.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.306+0000 7f7df6dd2640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 0x7f7ddc06ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7dec001120 tx=0x7f7dec009040 comp rx=0 tx=0).stop 2023-12-01T01:52:07.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.306+0000 7f7df6dd2640 1 -- 172.21.15.115:0/607794630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7df0153d70 msgr2=0x7f7df00af570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:07.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.306+0000 7f7df6dd2640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7df0153d70 0x7f7df00af570 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f7de0035040 tx=0x7f7de0004730 comp rx=0 tx=0).stop 2023-12-01T01:52:07.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.307+0000 7f7df6dd2640 1 -- 172.21.15.115:0/607794630 shutdown_connections 2023-12-01T01:52:07.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.307+0000 7f7df6dd2640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 0x7f7ddc06ca10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.307+0000 7f7df6dd2640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7df01541d0 0x7f7df00afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.307+0000 7f7df6dd2640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7df0153d70 0x7f7df00af570 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.307+0000 7f7df6dd2640 1 --2- 172.21.15.115:0/607794630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd80069d0 0x7f7df00c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.307+0000 7f7df6dd2640 1 -- 172.21.15.115:0/607794630 >> 172.21.15.115:0/607794630 conn(0x7f7df00ce880 msgr2=0x7f7df0158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:07.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.307+0000 7f7df6dd2640 1 -- 172.21.15.115:0/607794630 shutdown_connections 2023-12-01T01:52:07.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.307+0000 7f7df6dd2640 1 -- 172.21.15.115:0/607794630 wait complete. 2023-12-01T01:52:07.327 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T01:52:07.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.547+0000 7f82e7fff640 1 Processor -- start 2023-12-01T01:52:07.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.547+0000 7f82e7fff640 1 -- start start 2023-12-01T01:52:07.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.548+0000 7f82e7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82e81599e0 0x7f82e8159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.548+0000 7f82e7fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82e80af370 0x7f82e80c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.549+0000 7f82e7fff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e815e440 con 0x7f82e80c95a0 2023-12-01T01:52:07.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.549+0000 7f82e7fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e815a300 con 0x7f82e81599e0 2023-12-01T01:52:07.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.549+0000 7f82e7fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e815d550 con 0x7f82e80af370 2023-12-01T01:52:07.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.549+0000 7f82e6ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82e81599e0 0x7f82e8159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:07.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.549+0000 7f82e6ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82e81599e0 0x7f82e8159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34758/0 (socket says 172.21.15.115:34758) 2023-12-01T01:52:07.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.549+0000 7f82e6ffd640 1 -- 172.21.15.115:0/3737707899 learned_addr learned my addr 172.21.15.115:0/3737707899 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:07.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.549+0000 7f82e67fc640 1 --2- 172.21.15.115:0/3737707899 >> v2:172.21.15.142:3300/0 conn(0x7f82e80af370 0x7f82e80c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:07.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.549+0000 7f82e6ffd640 1 -- 172.21.15.115:0/3737707899 >> v2:172.21.15.142:3300/0 conn(0x7f82e80af370 msgr2=0x7f82e80c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:07.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.549+0000 7f82e6ffd640 1 --2- 172.21.15.115:0/3737707899 >> v2:172.21.15.142:3300/0 conn(0x7f82e80af370 0x7f82e80c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.549+0000 7f82e6ffd640 1 -- 172.21.15.115:0/3737707899 >> v1:172.21.15.115:6789/0 conn(0x7f82e80c95a0 legacy=0x7f82e806f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:07.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.550+0000 7f82e6ffd640 1 -- 172.21.15.115:0/3737707899 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e815d830 con 0x7f82e81599e0 2023-12-01T01:52:07.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.551+0000 7f82e6ffd640 1 --2- 172.21.15.115:0/3737707899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e81599e0 0x7f82e8159dc0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f82e80668e0 tx=0x7f82d802dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d53ccfbbc277de server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:07.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.551+0000 7f82e5ffb640 1 -- 172.21.15.115:0/3737707899 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d802e960 con 0x7f82e81599e0 2023-12-01T01:52:07.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.551+0000 7f82e5ffb640 1 -- 172.21.15.115:0/3737707899 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d802eac0 con 0x7f82e81599e0 2023-12-01T01:52:07.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.551+0000 7f82e5ffb640 1 -- 172.21.15.115:0/3737707899 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d802ee50 con 0x7f82e81599e0 2023-12-01T01:52:07.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.551+0000 7f82e7fff640 1 -- 172.21.15.115:0/3737707899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e81599e0 msgr2=0x7f82e8159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:07.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.551+0000 7f82e7fff640 1 --2- 172.21.15.115:0/3737707899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e81599e0 0x7f82e8159dc0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f82e80668e0 tx=0x7f82d802dbb0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.552+0000 7f82e7fff640 1 -- 172.21.15.115:0/3737707899 shutdown_connections 2023-12-01T01:52:07.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.552+0000 7f82e7fff640 1 --2- 172.21.15.115:0/3737707899 >> v2:172.21.15.142:3300/0 conn(0x7f82e80af370 0x7f82e80c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.552+0000 7f82e7fff640 1 --2- 172.21.15.115:0/3737707899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e81599e0 0x7f82e8159dc0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.552+0000 7f82e7fff640 1 -- 172.21.15.115:0/3737707899 >> 172.21.15.115:0/3737707899 conn(0x7f82e8065960 msgr2=0x7f82e8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:07.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.552+0000 7f82e7fff640 1 -- 172.21.15.115:0/3737707899 shutdown_connections 2023-12-01T01:52:07.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.552+0000 7f82e7fff640 1 -- 172.21.15.115:0/3737707899 wait complete. 2023-12-01T01:52:07.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.553+0000 7f82e7fff640 1 Processor -- start 2023-12-01T01:52:07.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.554+0000 7f82e7fff640 1 -- start start 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.554+0000 7f82e7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e80af370 0x7f82e81f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e80c95a0 0x7f82e814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82e81599e0 0x7f82e80ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e7fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80c6930 con 0x7f82e80c95a0 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e7fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80b1bd0 con 0x7f82e80af370 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e7fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80c6300 con 0x7f82e81599e0 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e80c95a0 0x7f82e814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e80c95a0 0x7f82e814f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34772/0 (socket says 172.21.15.115:34772) 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e67fc640 1 -- 172.21.15.115:0/4258797591 learned_addr learned my addr 172.21.15.115:0/4258797591 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e6ffd640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e80af370 0x7f82e81f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e67fc640 1 -- 172.21.15.115:0/4258797591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82e81599e0 msgr2=0x7f82e80ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e67fc640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82e81599e0 0x7f82e80ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e67fc640 1 -- 172.21.15.115:0/4258797591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e80af370 msgr2=0x7f82e81f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:07.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e67fc640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e80af370 0x7f82e81f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:07.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.555+0000 7f82e67fc640 1 -- 172.21.15.115:0/4258797591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80094d0 con 0x7f82e80c95a0 2023-12-01T01:52:07.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.556+0000 7f82e67fc640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e80c95a0 0x7f82e814f770 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00d800 tx=0x7f82dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:07.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.556+0000 7f82cf7fe640 1 -- 172.21.15.115:0/4258797591 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00b6c0 con 0x7f82e80c95a0 2023-12-01T01:52:07.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.556+0000 7f82cf7fe640 1 -- 172.21.15.115:0/4258797591 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc00b820 con 0x7f82e80c95a0 2023-12-01T01:52:07.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.556+0000 7f82cf7fe640 1 -- 172.21.15.115:0/4258797591 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00bbb0 con 0x7f82e80c95a0 2023-12-01T01:52:07.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.556+0000 7f82e7fff640 1 -- 172.21.15.115:0/4258797591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e80b1de0 con 0x7f82e80c95a0 2023-12-01T01:52:07.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.556+0000 7f82e7fff640 1 -- 172.21.15.115:0/4258797591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e80c6550 con 0x7f82e80c95a0 2023-12-01T01:52:07.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.557+0000 7f82e7fff640 1 -- 172.21.15.115:0/4258797591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e80b2300 con 0x7f82e80c95a0 2023-12-01T01:52:07.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.558+0000 7f82cf7fe640 1 -- 172.21.15.115:0/4258797591 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f82dc00bd10 con 0x7f82e80c95a0 2023-12-01T01:52:07.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.559+0000 7f82cf7fe640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82d406a550 0x7f82d406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:07.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.559+0000 7f82cf7fe640 1 -- 172.21.15.115:0/4258797591 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82dc088f10 con 0x7f82e80c95a0 2023-12-01T01:52:07.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.560+0000 7f82e6ffd640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82d406a550 0x7f82d406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:07.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.560+0000 7f82e6ffd640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82d406a550 0x7f82d406ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f82d8038000 tx=0x7f82d8032000 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:07.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.564+0000 7f82cf7fe640 1 -- 172.21.15.115:0/4258797591 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f82dc054480 con 0x7f82e80c95a0 2023-12-01T01:52:07.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:07.792+0000 7f82e7fff640 1 -- 172.21.15.115:0/4258797591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f82e80618d0 con 0x7f82e80c95a0 2023-12-01T01:52:08.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-12-01T01:52:08.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-12-01T01:52:08.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-12-01T01:52:08.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.480+0000 7f82cf7fe640 1 -- 172.21.15.115:0/4258797591 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7f82dc057aa0 con 0x7f82e80c95a0 2023-12-01T01:52:08.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.483+0000 7f82e7fff640 1 -- 172.21.15.115:0/4258797591 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82d406a550 msgr2=0x7f82d406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:08.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.483+0000 7f82e7fff640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82d406a550 0x7f82d406ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f82d8038000 tx=0x7f82d8032000 comp rx=0 tx=0).stop 2023-12-01T01:52:08.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.483+0000 7f82e7fff640 1 -- 172.21.15.115:0/4258797591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e80c95a0 msgr2=0x7f82e814f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:08.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.483+0000 7f82e7fff640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e80c95a0 0x7f82e814f770 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00d800 tx=0x7f82dc00dcc0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.483+0000 7f82e7fff640 1 -- 172.21.15.115:0/4258797591 shutdown_connections 2023-12-01T01:52:08.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.483+0000 7f82e7fff640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82d406a550 0x7f82d406ca10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.483+0000 7f82e7fff640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82e81599e0 0x7f82e80ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.483+0000 7f82e7fff640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82e80c95a0 0x7f82e814f770 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.483+0000 7f82e7fff640 1 --2- 172.21.15.115:0/4258797591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82e80af370 0x7f82e81f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.483+0000 7f82e7fff640 1 -- 172.21.15.115:0/4258797591 >> 172.21.15.115:0/4258797591 conn(0x7f82e8065960 msgr2=0x7f82e80ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:08.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.484+0000 7f82e7fff640 1 -- 172.21.15.115:0/4258797591 shutdown_connections 2023-12-01T01:52:08.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.484+0000 7f82e7fff640 1 -- 172.21.15.115:0/4258797591 wait complete. 2023-12-01T01:52:08.506 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T01:52:08.507 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T01:52:08.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.718+0000 7f9d979b5640 1 Processor -- start 2023-12-01T01:52:08.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.718+0000 7f9d979b5640 1 -- start start 2023-12-01T01:52:08.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.720+0000 7f9d979b5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900b0f80 0x7f9d900b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:08.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.720+0000 7f9d979b5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900cb290 con 0x7f9d90160200 2023-12-01T01:52:08.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.720+0000 7f9d979b5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900b0ae0 con 0x7f9d900cbd30 2023-12-01T01:52:08.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.720+0000 7f9d979b5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900b35f0 con 0x7f9d900b0f80 2023-12-01T01:52:08.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.720+0000 7f9d95751640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9d90160200 0x7f9d9006b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35598/0 (socket says 172.21.15.115:35598) 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.720+0000 7f9d95751640 1 -- 172.21.15.115:0/4111710859 learned_addr learned my addr 172.21.15.115:0/4111710859 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.720+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454583731 0 0) 0x7f9d900cb290 con 0x7f9d90160200 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.720+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d7c003520 con 0x7f9d90160200 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d94f50640 1 --2- 172.21.15.115:0/4111710859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900b0f80 0x7f9d900b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3377439013 0 0) 0x7f9d7c003520 con 0x7f9d90160200 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900b0f80 msgr2=0x7f9d900b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 --2- 172.21.15.115:0/4111710859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900b0f80 0x7f9d900b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 >> v1:172.21.15.142:6789/0 conn(0x7f9d900cbd30 legacy=0x7f9d900cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900b3310 con 0x7f9d90160200 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f9d8c001670 con 0x7f9d90160200 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 >> v1:172.21.15.115:6789/0 conn(0x7f9d90160200 legacy=0x7f9d9006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 --2- 172.21.15.115:0/4111710859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c0039a0 0x7f9d7c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 --2- 172.21.15.115:0/4111710859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 0x7f9d7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:08.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 --2- 172.21.15.115:0/4111710859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900b0f80 0x7f9d7c0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:08.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900b35f0 con 0x7f9d7c0047c0 2023-12-01T01:52:08.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c00ba90 con 0x7f9d7c0039a0 2023-12-01T01:52:08.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c00bc50 con 0x7f9d900b0f80 2023-12-01T01:52:08.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.721+0000 7f9d94f50640 1 --2- 172.21.15.115:0/4111710859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 0x7f9d7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:08.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.722+0000 7f9d94f50640 1 -- 172.21.15.115:0/4111710859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900b0f80 msgr2=0x7f9d7c0051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:08.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.722+0000 7f9d94f50640 1 --2- 172.21.15.115:0/4111710859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900b0f80 0x7f9d7c0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.722+0000 7f9d94f50640 1 -- 172.21.15.115:0/4111710859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c0039a0 msgr2=0x7f9d7c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:08.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.722+0000 7f9d94f50640 1 --2- 172.21.15.115:0/4111710859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d7c0039a0 0x7f9d7c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.722+0000 7f9d94f50640 1 -- 172.21.15.115:0/4111710859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c003520 con 0x7f9d7c0047c0 2023-12-01T01:52:08.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.723+0000 7f9d94f50640 1 --2- 172.21.15.115:0/4111710859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 0x7f9d7c004c40 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000ba30 tx=0x7f9d7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bd329b6d9a03b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:08.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.723+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80032680 con 0x7f9d7c0047c0 2023-12-01T01:52:08.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.723+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d800327e0 con 0x7f9d7c0047c0 2023-12-01T01:52:08.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.723+0000 7f9d87fff640 1 -- 172.21.15.115:0/4111710859 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80032b70 con 0x7f9d7c0047c0 2023-12-01T01:52:08.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.724+0000 7f9d979b5640 1 -- 172.21.15.115:0/4111710859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 msgr2=0x7f9d7c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:08.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.724+0000 7f9d979b5640 1 --2- 172.21.15.115:0/4111710859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 0x7f9d7c004c40 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000ba30 tx=0x7f9d7c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.724+0000 7f9d95f52640 1 -- 172.21.15.115:0/4111710859 reap_dead start 2023-12-01T01:52:08.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.724+0000 7f9d979b5640 1 -- 172.21.15.115:0/4111710859 shutdown_connections 2023-12-01T01:52:08.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.724+0000 7f9d979b5640 1 -- 172.21.15.115:0/4111710859 >> 172.21.15.115:0/4111710859 conn(0x7f9d90065960 msgr2=0x7f9d90065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:08.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.724+0000 7f9d979b5640 1 -- 172.21.15.115:0/4111710859 shutdown_connections 2023-12-01T01:52:08.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.724+0000 7f9d979b5640 1 -- 172.21.15.115:0/4111710859 wait complete. 2023-12-01T01:52:08.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.726+0000 7f9d979b5640 1 Processor -- start 2023-12-01T01:52:08.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.726+0000 7f9d979b5640 1 -- start start 2023-12-01T01:52:08.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.727+0000 7f9d979b5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 0x7f9d901505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:08.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.727+0000 7f9d979b5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900c83d0 0x7f9d901f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:08.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.727+0000 7f9d979b5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d901f2610 0x7f9d901f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:08.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.727+0000 7f9d979b5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90150b20 con 0x7f9d7c0047c0 2023-12-01T01:52:08.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.727+0000 7f9d979b5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d901f63d0 con 0x7f9d901f2610 2023-12-01T01:52:08.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.727+0000 7f9d979b5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d901f66b0 con 0x7f9d900c83d0 2023-12-01T01:52:08.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.727+0000 7f9d95751640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 0x7f9d901505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:08.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.727+0000 7f9d95751640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 0x7f9d901505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46886/0 (socket says 172.21.15.115:46886) 2023-12-01T01:52:08.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.727+0000 7f9d95751640 1 -- 172.21.15.115:0/2310377362 learned_addr learned my addr 172.21.15.115:0/2310377362 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:08.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.727+0000 7f9d94f50640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900c83d0 0x7f9d901f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:08.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d95f52640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d901f2610 0x7f9d901f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:08.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d95751640 1 -- 172.21.15.115:0/2310377362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900c83d0 msgr2=0x7f9d901f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:08.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d95751640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900c83d0 0x7f9d901f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d95751640 1 -- 172.21.15.115:0/2310377362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d901f2610 msgr2=0x7f9d901f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:08.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d95751640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d901f2610 0x7f9d901f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:08.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d95751640 1 -- 172.21.15.115:0/2310377362 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d901f6990 con 0x7f9d7c0047c0 2023-12-01T01:52:08.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d95f52640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d901f2610 0x7f9d901f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:08.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d95751640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 0x7f9d901505e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c7a0 tx=0x7f9d8c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:08.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d85ffb640 1 -- 172.21.15.115:0/2310377362 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0074e0 con 0x7f9d7c0047c0 2023-12-01T01:52:08.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d85ffb640 1 -- 172.21.15.115:0/2310377362 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c007640 con 0x7f9d7c0047c0 2023-12-01T01:52:08.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d979b5640 1 -- 172.21.15.115:0/2310377362 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d900c8d60 con 0x7f9d7c0047c0 2023-12-01T01:52:08.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d85ffb640 1 -- 172.21.15.115:0/2310377362 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0049e0 con 0x7f9d7c0047c0 2023-12-01T01:52:08.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.728+0000 7f9d979b5640 1 -- 172.21.15.115:0/2310377362 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d901f60c0 con 0x7f9d7c0047c0 2023-12-01T01:52:08.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.730+0000 7f9d979b5640 1 -- 172.21.15.115:0/2310377362 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9004e890 con 0x7f9d7c0047c0 2023-12-01T01:52:08.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.731+0000 7f9d85ffb640 1 -- 172.21.15.115:0/2310377362 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9d8c004c00 con 0x7f9d7c0047c0 2023-12-01T01:52:08.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.731+0000 7f9d85ffb640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d7406a550 0x7f9d7406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:08.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.732+0000 7f9d85ffb640 1 -- 172.21.15.115:0/2310377362 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d8c087f60 con 0x7f9d7c0047c0 2023-12-01T01:52:08.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.732+0000 7f9d94f50640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d7406a550 0x7f9d7406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:08.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.732+0000 7f9d94f50640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d7406a550 0x7f9d7406ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000ba00 tx=0x7f9d8003d040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:08.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.736+0000 7f9d85ffb640 1 -- 172.21.15.115:0/2310377362 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9d8c053450 con 0x7f9d7c0047c0 2023-12-01T01:52:08.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:08.969+0000 7f9d979b5640 1 -- 172.21.15.115:0/2310377362 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9d900b1e20 con 0x7f9d7c0047c0 2023-12-01T01:52:09.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.490+0000 7f9d85ffb640 1 -- 172.21.15.115:0/2310377362 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9d8c0180c0 con 0x7f9d7c0047c0 2023-12-01T01:52:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 -- 172.21.15.115:0/2310377362 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d7406a550 msgr2=0x7f9d7406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d7406a550 0x7f9d7406ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000ba00 tx=0x7f9d8003d040 comp rx=0 tx=0).stop 2023-12-01T01:52:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 -- 172.21.15.115:0/2310377362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 msgr2=0x7f9d901505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 0x7f9d901505e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c7a0 tx=0x7f9d8c00cc60 comp rx=0 tx=0).stop 2023-12-01T01:52:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 -- 172.21.15.115:0/2310377362 shutdown_connections 2023-12-01T01:52:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9d7406a550 0x7f9d7406ca10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d900c83d0 0x7f9d901f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d901f2610 0x7f9d901f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 --2- 172.21.15.115:0/2310377362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d7c0047c0 0x7f9d901505e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 -- 172.21.15.115:0/2310377362 >> 172.21.15.115:0/2310377362 conn(0x7f9d90065960 msgr2=0x7f9d9015a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:09.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 -- 172.21.15.115:0/2310377362 shutdown_connections 2023-12-01T01:52:09.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.494+0000 7f9d979b5640 1 -- 172.21.15.115:0/2310377362 wait complete. 2023-12-01T01:52:09.516 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T01:52:09.516 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:52:09.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.731+0000 7f0aa501f640 1 Processor -- start 2023-12-01T01:52:09.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.731+0000 7f0aa501f640 1 -- start start 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.732+0000 7f0aa501f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0aa006b4e0 0x7f0aa0161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.732+0000 7f0aa501f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0aa006b8c0 0x7f0aa00cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0aa501f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa0162530 con 0x7f0aa015b0d0 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0aa501f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa00caf40 con 0x7f0aa006b8c0 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0aa501f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa00cd7f0 con 0x7f0aa006b4e0 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0a9f7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0aa006b8c0 0x7f0aa00cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0a9f7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0aa006b8c0 0x7f0aa00cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46894/0 (socket says 172.21.15.115:46894) 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0a9f7fe640 1 -- 172.21.15.115:0/2250623883 learned_addr learned my addr 172.21.15.115:0/2250623883 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0aa481e640 1 --1- 172.21.15.115:0/2250623883 >> v1:172.21.15.115:6789/0 conn(0x7f0aa015b0d0 0x7f0aa015b4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35612/0 (socket says 172.21.15.115:35612) 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0a9ffff640 1 --2- 172.21.15.115:0/2250623883 >> v2:172.21.15.142:3300/0 conn(0x7f0aa006b4e0 0x7f0aa0161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0a9f7fe640 1 -- 172.21.15.115:0/2250623883 >> v2:172.21.15.142:3300/0 conn(0x7f0aa006b4e0 msgr2=0x7f0aa0161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0a9f7fe640 1 --2- 172.21.15.115:0/2250623883 >> v2:172.21.15.142:3300/0 conn(0x7f0aa006b4e0 0x7f0aa0161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0a9f7fe640 1 -- 172.21.15.115:0/2250623883 >> v1:172.21.15.115:6789/0 conn(0x7f0aa015b0d0 legacy=0x7f0aa015b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:09.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0a9f7fe640 1 -- 172.21.15.115:0/2250623883 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa00ca3c0 con 0x7f0aa006b8c0 2023-12-01T01:52:09.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.733+0000 7f0a9ffff640 1 --2- 172.21.15.115:0/2250623883 >> v2:172.21.15.142:3300/0 conn(0x7f0aa006b4e0 0x7f0aa0161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.734+0000 7f0a9f7fe640 1 --2- 172.21.15.115:0/2250623883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa006b8c0 0x7f0aa00cc180 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0aa006d300 tx=0x7f0a9402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123e0423020b6f7b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0a9effd640 1 -- 172.21.15.115:0/2250623883 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94030870 con 0x7f0aa006b8c0 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0a9effd640 1 -- 172.21.15.115:0/2250623883 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a940309d0 con 0x7f0aa006b8c0 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0a9effd640 1 -- 172.21.15.115:0/2250623883 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94030d60 con 0x7f0aa006b8c0 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0aa501f640 1 -- 172.21.15.115:0/2250623883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa006b8c0 msgr2=0x7f0aa00cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0aa501f640 1 --2- 172.21.15.115:0/2250623883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa006b8c0 0x7f0aa00cc180 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0aa006d300 tx=0x7f0a9402b9b0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0aa501f640 1 -- 172.21.15.115:0/2250623883 shutdown_connections 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0aa501f640 1 --2- 172.21.15.115:0/2250623883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa006b8c0 0x7f0aa00cc180 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0aa501f640 1 --2- 172.21.15.115:0/2250623883 >> v2:172.21.15.142:3300/0 conn(0x7f0aa006b4e0 0x7f0aa0161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0aa501f640 1 -- 172.21.15.115:0/2250623883 >> 172.21.15.115:0/2250623883 conn(0x7f0aa00ce640 msgr2=0x7f0aa00cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0aa501f640 1 -- 172.21.15.115:0/2250623883 shutdown_connections 2023-12-01T01:52:09.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.735+0000 7f0aa501f640 1 -- 172.21.15.115:0/2250623883 wait complete. 2023-12-01T01:52:09.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.737+0000 7f0aa501f640 1 Processor -- start 2023-12-01T01:52:09.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.737+0000 7f0aa501f640 1 -- start start 2023-12-01T01:52:09.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.738+0000 7f0aa501f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aa006b4e0 0x7f0aa01539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:09.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.738+0000 7f0aa501f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aa006b8c0 0x7f0aa0156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:09.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.738+0000 7f0aa501f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa015b0d0 0x7f0aa01532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:09.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.738+0000 7f0aa501f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa006f400 con 0x7f0aa015b0d0 2023-12-01T01:52:09.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.738+0000 7f0aa501f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa0153800 con 0x7f0aa006b4e0 2023-12-01T01:52:09.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.738+0000 7f0aa501f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa0156f70 con 0x7f0aa006b8c0 2023-12-01T01:52:09.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.738+0000 7f0aa481e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa015b0d0 0x7f0aa01532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:09.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.738+0000 7f0aa481e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa015b0d0 0x7f0aa01532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46898/0 (socket says 172.21.15.115:46898) 2023-12-01T01:52:09.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.738+0000 7f0aa481e640 1 -- 172.21.15.115:0/1595239174 learned_addr learned my addr 172.21.15.115:0/1595239174 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:09.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.739+0000 7f0aa481e640 1 -- 172.21.15.115:0/1595239174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aa006b8c0 msgr2=0x7f0aa0156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:09.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.739+0000 7f0aa481e640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aa006b8c0 0x7f0aa0156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.739+0000 7f0aa481e640 1 -- 172.21.15.115:0/1595239174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aa006b4e0 msgr2=0x7f0aa01539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:09.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.739+0000 7f0aa481e640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aa006b4e0 0x7f0aa01539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.739+0000 7f0aa481e640 1 -- 172.21.15.115:0/1595239174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a940094d0 con 0x7f0aa015b0d0 2023-12-01T01:52:09.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.739+0000 7f0aa481e640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa015b0d0 0x7f0aa01532c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800c6f0 tx=0x7f0a9800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:09.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.739+0000 7f0a9cff9640 1 -- 172.21.15.115:0/1595239174 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98005da0 con 0x7f0aa015b0d0 2023-12-01T01:52:09.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.739+0000 7f0a9cff9640 1 -- 172.21.15.115:0/1595239174 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9800a640 con 0x7f0aa015b0d0 2023-12-01T01:52:09.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.739+0000 7f0a9cff9640 1 -- 172.21.15.115:0/1595239174 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800a9b0 con 0x7f0aa015b0d0 2023-12-01T01:52:09.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.740+0000 7f0aa501f640 1 -- 172.21.15.115:0/1595239174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa0157250 con 0x7f0aa015b0d0 2023-12-01T01:52:09.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.740+0000 7f0aa501f640 1 -- 172.21.15.115:0/1595239174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa0164360 con 0x7f0aa015b0d0 2023-12-01T01:52:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.741+0000 7f0aa501f640 1 -- 172.21.15.115:0/1595239174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa00673a0 con 0x7f0aa015b0d0 2023-12-01T01:52:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.742+0000 7f0a9cff9640 1 -- 172.21.15.115:0/1595239174 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0a9800ab10 con 0x7f0aa015b0d0 2023-12-01T01:52:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.742+0000 7f0a9cff9640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a8c06a580 0x7f0a8c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.743+0000 7f0a9cff9640 1 -- 172.21.15.115:0/1595239174 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a98088190 con 0x7f0aa015b0d0 2023-12-01T01:52:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.743+0000 7f0a9ffff640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a8c06a580 0x7f0a8c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:09.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.743+0000 7f0a9ffff640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a8c06a580 0x7f0a8c06ca40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0a90006fd0 tx=0x7f0a90008040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:09.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.747+0000 7f0a9cff9640 1 -- 172.21.15.115:0/1595239174 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0a98053710 con 0x7f0aa015b0d0 2023-12-01T01:52:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.974+0000 7f0aa501f640 1 -- 172.21.15.115:0/1595239174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0aa0157440 con 0x7f0aa015b0d0 2023-12-01T01:52:09.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.976+0000 7f0a9cff9640 1 -- 172.21.15.115:0/1595239174 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f0a98056d30 con 0x7f0aa015b0d0 2023-12-01T01:52:09.977 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:09.978 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/542800961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":542800961},{"type":"v1","addr":"172.21.15.142:6833","nonce":542800961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6942,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2035657957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2035657957},{"type":"v1","addr":"172.21.15.115:6835","nonce":2035657957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3644451216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3644451216},{"type":"v1","addr":"172.21.15.115:6837","nonce":3644451216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:06.454774+0000","modified":"2023-12-01T01:52:09.476985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6957":{"gid":6957,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1311626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1311626228},{"type":"v1","addr":"172.21.15.142:6835","nonce":1311626228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-01T01:52:09.978 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2023-12-01T01:52:09.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 -- 172.21.15.115:0/1595239174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a8c06a580 msgr2=0x7f0a8c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:09.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a8c06a580 0x7f0a8c06ca40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0a90006fd0 tx=0x7f0a90008040 comp rx=0 tx=0).stop 2023-12-01T01:52:09.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 -- 172.21.15.115:0/1595239174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa015b0d0 msgr2=0x7f0aa01532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:09.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa015b0d0 0x7f0aa01532c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800c6f0 tx=0x7f0a9800cbb0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 -- 172.21.15.115:0/1595239174 shutdown_connections 2023-12-01T01:52:09.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a8c06a580 0x7f0a8c06ca40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0aa006b8c0 0x7f0aa0156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0aa015b0d0 0x7f0aa01532c0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 --2- 172.21.15.115:0/1595239174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aa006b4e0 0x7f0aa01539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:09.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 -- 172.21.15.115:0/1595239174 >> 172.21.15.115:0/1595239174 conn(0x7f0aa00ce640 msgr2=0x7f0aa015a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:09.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 -- 172.21.15.115:0/1595239174 shutdown_connections 2023-12-01T01:52:09.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:09.979+0000 7f0aa501f640 1 -- 172.21.15.115:0/1595239174 wait complete. 2023-12-01T01:52:09.999 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:10.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.202+0000 7ff528d39640 1 Processor -- start 2023-12-01T01:52:10.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.202+0000 7ff528d39640 1 -- start start 2023-12-01T01:52:10.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.204+0000 7ff528d39640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 0x7ff5240cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.204+0000 7ff528d39640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5240ad2b0 0x7ff5240ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.204+0000 7ff528d39640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52415d600 con 0x7ff5240ade80 2023-12-01T01:52:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.204+0000 7ff528d39640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52406cc10 con 0x7ff5240ad2b0 2023-12-01T01:52:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.204+0000 7ff528d39640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52406c720 con 0x7ff52415d220 2023-12-01T01:52:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.204+0000 7ff51bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5240ad2b0 0x7ff5240ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.204+0000 7ff51bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5240ad2b0 0x7ff5240ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46914/0 (socket says 172.21.15.115:46914) 2023-12-01T01:52:10.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.204+0000 7ff51bfff640 1 -- 172.21.15.115:0/2259275127 learned_addr learned my addr 172.21.15.115:0/2259275127 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.205+0000 7ff51bfff640 1 -- 172.21.15.115:0/2259275127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 msgr2=0x7ff5240cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:10.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.205+0000 7ff52259c640 1 --2- 172.21.15.115:0/2259275127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 0x7ff5240cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.205+0000 7ff51bfff640 1 --2- 172.21.15.115:0/2259275127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 0x7ff5240cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.205+0000 7ff51bfff640 1 -- 172.21.15.115:0/2259275127 >> v1:172.21.15.142:6789/0 conn(0x7ff5240ade80 legacy=0x7ff52406b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.205+0000 7ff51bfff640 1 -- 172.21.15.115:0/2259275127 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52406c4b0 con 0x7ff5240ad2b0 2023-12-01T01:52:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.206+0000 7ff51bfff640 1 --2- 172.21.15.115:0/2259275127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 0x7ff5240ccab0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009870 tx=0x7ff50c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22319b2544e36f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.206+0000 7ff521d9b640 1 -- 172.21.15.115:0/2259275127 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0317c0 con 0x7ff5240ad2b0 2023-12-01T01:52:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.206+0000 7ff521d9b640 1 -- 172.21.15.115:0/2259275127 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c031920 con 0x7ff5240ad2b0 2023-12-01T01:52:10.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.206+0000 7ff521d9b640 1 -- 172.21.15.115:0/2259275127 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c031cb0 con 0x7ff5240ad2b0 2023-12-01T01:52:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.207+0000 7ff528d39640 1 -- 172.21.15.115:0/2259275127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 msgr2=0x7ff5240ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.207+0000 7ff528d39640 1 --2- 172.21.15.115:0/2259275127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 0x7ff5240ccab0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009870 tx=0x7ff50c02b860 comp rx=0 tx=0).stop 2023-12-01T01:52:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.207+0000 7ff528d39640 1 -- 172.21.15.115:0/2259275127 shutdown_connections 2023-12-01T01:52:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.207+0000 7ff528d39640 1 --2- 172.21.15.115:0/2259275127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 0x7ff5240ccab0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.207+0000 7ff528d39640 1 --2- 172.21.15.115:0/2259275127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 0x7ff5240cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.207+0000 7ff528d39640 1 -- 172.21.15.115:0/2259275127 >> 172.21.15.115:0/2259275127 conn(0x7ff5240ce880 msgr2=0x7ff5240cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:10.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.207+0000 7ff528d39640 1 -- 172.21.15.115:0/2259275127 shutdown_connections 2023-12-01T01:52:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.207+0000 7ff528d39640 1 -- 172.21.15.115:0/2259275127 wait complete. 2023-12-01T01:52:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.209+0000 7ff528d39640 1 Processor -- start 2023-12-01T01:52:10.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.209+0000 7ff528d39640 1 -- start start 2023-12-01T01:52:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff528d39640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 0x7ff524152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff528d39640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5240ade80 0x7ff5240af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff528d39640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 0x7ff5240c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff528d39640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5240b3c60 con 0x7ff5240ad2b0 2023-12-01T01:52:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff528d39640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5240b46b0 con 0x7ff5240ade80 2023-12-01T01:52:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff528d39640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5240b3e50 con 0x7ff52415d220 2023-12-01T01:52:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff52259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 0x7ff524152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff52259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 0x7ff524152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46922/0 (socket says 172.21.15.115:46922) 2023-12-01T01:52:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff52259c640 1 -- 172.21.15.115:0/3716065148 learned_addr learned my addr 172.21.15.115:0/3716065148 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff51bfff640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5240ade80 0x7ff5240af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff522d9d640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 0x7ff5240c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff52259c640 1 -- 172.21.15.115:0/3716065148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 msgr2=0x7ff5240c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff52259c640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 0x7ff5240c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff52259c640 1 -- 172.21.15.115:0/3716065148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5240ade80 msgr2=0x7ff5240af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff52259c640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5240ade80 0x7ff5240af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff52259c640 1 -- 172.21.15.115:0/3716065148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0094d0 con 0x7ff5240ad2b0 2023-12-01T01:52:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff522d9d640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 0x7ff5240c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.210+0000 7ff51bfff640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5240ade80 0x7ff5240af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:10.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.211+0000 7ff52259c640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 0x7ff524152df0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff51400c820 tx=0x7ff51400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.211+0000 7ff51affd640 1 -- 172.21.15.115:0/3716065148 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514004020 con 0x7ff5240ad2b0 2023-12-01T01:52:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.211+0000 7ff51affd640 1 -- 172.21.15.115:0/3716065148 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff514004180 con 0x7ff5240ad2b0 2023-12-01T01:52:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.211+0000 7ff51affd640 1 -- 172.21.15.115:0/3716065148 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514007830 con 0x7ff5240ad2b0 2023-12-01T01:52:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.211+0000 7ff528d39640 1 -- 172.21.15.115:0/3716065148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5240b1780 con 0x7ff5240ad2b0 2023-12-01T01:52:10.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.211+0000 7ff528d39640 1 -- 172.21.15.115:0/3716065148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5240b43d0 con 0x7ff5240ad2b0 2023-12-01T01:52:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.212+0000 7ff528d39640 1 -- 172.21.15.115:0/3716065148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52404e890 con 0x7ff5240ad2b0 2023-12-01T01:52:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.213+0000 7ff51affd640 1 -- 172.21.15.115:0/3716065148 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff514007990 con 0x7ff5240ad2b0 2023-12-01T01:52:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.214+0000 7ff51affd640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff50406a550 0x7ff50406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.214+0000 7ff51affd640 1 -- 172.21.15.115:0/3716065148 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff514088870 con 0x7ff5240ad2b0 2023-12-01T01:52:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.214+0000 7ff51bfff640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff50406a550 0x7ff50406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.215+0000 7ff51bfff640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff50406a550 0x7ff50406ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff50c036000 tx=0x7ff50c03d040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:10.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.219+0000 7ff51affd640 1 -- 172.21.15.115:0/3716065148 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff514053c30 con 0x7ff5240ad2b0 2023-12-01T01:52:10.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.410+0000 7ff528d39640 1 -- 172.21.15.115:0/3716065148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff52415a4d0 con 0x7ff5240ad2b0 2023-12-01T01:52:10.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.413+0000 7ff51affd640 1 -- 172.21.15.115:0/3716065148 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16088 (secure 0 0 0) 0x7ff51401e090 con 0x7ff5240ad2b0 2023-12-01T01:52:10.415 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:10.415 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":97,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:07.459655+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:04.909384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:05.480768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 -- 172.21.15.115:0/3716065148 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff50406a550 msgr2=0x7ff50406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff50406a550 0x7ff50406ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff50c036000 tx=0x7ff50c03d040 comp rx=0 tx=0).stop 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 -- 172.21.15.115:0/3716065148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 msgr2=0x7ff524152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 0x7ff524152df0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff51400c820 tx=0x7ff51400cce0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 -- 172.21.15.115:0/3716065148 shutdown_connections 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff50406a550 0x7ff50406ca10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff52415d220 0x7ff5240c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5240ade80 0x7ff5240af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 --2- 172.21.15.115:0/3716065148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5240ad2b0 0x7ff524152df0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 -- 172.21.15.115:0/3716065148 >> 172.21.15.115:0/3716065148 conn(0x7ff5240ce880 msgr2=0x7ff5240ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 -- 172.21.15.115:0/3716065148 shutdown_connections 2023-12-01T01:52:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.416+0000 7ff528d39640 1 -- 172.21.15.115:0/3716065148 wait complete. 2023-12-01T01:52:10.436 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T01:52:10.436 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:52:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.640+0000 7f3d81022640 1 Processor -- start 2023-12-01T01:52:10.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.640+0000 7f3d81022640 1 -- start start 2023-12-01T01:52:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.642+0000 7f3d81022640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c06b270 0x7f3d7c0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.642+0000 7f3d81022640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0ce120 con 0x7f3d7c0cdcc0 2023-12-01T01:52:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.642+0000 7f3d81022640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c06b700 con 0x7f3d7c1597d0 2023-12-01T01:52:10.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.642+0000 7f3d81022640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c06b9c0 con 0x7f3d7c06b270 2023-12-01T01:52:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.642+0000 7f3d73fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3d7c0cdcc0 0x7f3d7c0adf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35624/0 (socket says 172.21.15.115:35624) 2023-12-01T01:52:10.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.642+0000 7f3d73fff640 1 -- 172.21.15.115:0/3157420896 learned_addr learned my addr 172.21.15.115:0/3157420896 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538563906 0 0) 0x7f3d7c0ce120 con 0x7f3d7c0cdcc0 2023-12-01T01:52:10.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d68003520 con 0x7f3d7c0cdcc0 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d80821640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c06b270 0x7f3d7c0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640791768 0 0) 0x7f3d68003520 con 0x7f3d7c0cdcc0 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c06b270 msgr2=0x7f3d7c0ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c06b270 0x7f3d7c0ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 >> v1:172.21.15.142:6789/0 conn(0x7f3d7c1597d0 legacy=0x7f3d7c15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c06bca0 con 0x7f3d7c0cdcc0 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3d64001670 con 0x7f3d7c0cdcc0 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 >> v1:172.21.15.115:6789/0 conn(0x7f3d7c0cdcc0 legacy=0x7f3d7c0adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c06b270 0x7f3d68003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d68003fe0 0x7f3d680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68004d00 0x7f3d68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c06b9c0 con 0x7f3d68004d00 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6800ddd0 con 0x7f3d68003fe0 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.643+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6800df90 con 0x7f3d7c06b270 2023-12-01T01:52:10.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.644+0000 7f3d80821640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68004d00 0x7f3d68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.644+0000 7f3d7bfff640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c06b270 0x7f3d68003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.644+0000 7f3d73fff640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d68003fe0 0x7f3d680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.644+0000 7f3d80821640 1 -- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c06b270 msgr2=0x7f3d68003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.644+0000 7f3d80821640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c06b270 0x7f3d68003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.644+0000 7f3d80821640 1 -- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d68003fe0 msgr2=0x7f3d680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.644+0000 7f3d80821640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d68003fe0 0x7f3d680047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.644+0000 7f3d80821640 1 -- 172.21.15.115:0/3157420896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68003520 con 0x7f3d68004d00 2023-12-01T01:52:10.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.644+0000 7f3d7bfff640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c06b270 0x7f3d68003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.645+0000 7f3d80821640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68004d00 0x7f3d68005180 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009fc0 tx=0x7f3d680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11765a4c9c9e59a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.645+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7402cc20 con 0x7f3d68004d00 2023-12-01T01:52:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.645+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7400f040 con 0x7f3d68004d00 2023-12-01T01:52:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.645+0000 7f3d7b7fe640 1 -- 172.21.15.115:0/3157420896 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74034440 con 0x7f3d68004d00 2023-12-01T01:52:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.646+0000 7f3d81022640 1 -- 172.21.15.115:0/3157420896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68004d00 msgr2=0x7f3d68005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.646+0000 7f3d81022640 1 --2- 172.21.15.115:0/3157420896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68004d00 0x7f3d68005180 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009fc0 tx=0x7f3d680057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.646+0000 7f3d80821640 1 -- 172.21.15.115:0/3157420896 reap_dead start 2023-12-01T01:52:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.646+0000 7f3d81022640 1 -- 172.21.15.115:0/3157420896 shutdown_connections 2023-12-01T01:52:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.646+0000 7f3d81022640 1 -- 172.21.15.115:0/3157420896 >> 172.21.15.115:0/3157420896 conn(0x7f3d7c0ce640 msgr2=0x7f3d7c0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.646+0000 7f3d81022640 1 -- 172.21.15.115:0/3157420896 shutdown_connections 2023-12-01T01:52:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.646+0000 7f3d81022640 1 -- 172.21.15.115:0/3157420896 wait complete. 2023-12-01T01:52:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.648+0000 7f3d81022640 1 Processor -- start 2023-12-01T01:52:10.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.648+0000 7f3d81022640 1 -- start start 2023-12-01T01:52:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.649+0000 7f3d81022640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d68004d00 0x7f3d7c1ef900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.649+0000 7f3d81022640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d7c0c4420 0x7f3d7c1f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.649+0000 7f3d81022640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c1f3650 0x7f3d7c0ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.649+0000 7f3d81022640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1f2d50 con 0x7f3d7c0c4420 2023-12-01T01:52:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.649+0000 7f3d81022640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1f3030 con 0x7f3d68004d00 2023-12-01T01:52:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.649+0000 7f3d81022640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c1f3310 con 0x7f3d7c1f3650 2023-12-01T01:52:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.649+0000 7f3d73fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d7c0c4420 0x7f3d7c1f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.649+0000 7f3d73fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d7c0c4420 0x7f3d7c1f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46946/0 (socket says 172.21.15.115:46946) 2023-12-01T01:52:10.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.649+0000 7f3d73fff640 1 -- 172.21.15.115:0/4018709166 learned_addr learned my addr 172.21.15.115:0/4018709166 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d80821640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c1f3650 0x7f3d7c0ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d7bfff640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d68004d00 0x7f3d7c1ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d73fff640 1 -- 172.21.15.115:0/4018709166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c1f3650 msgr2=0x7f3d7c0ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d73fff640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c1f3650 0x7f3d7c0ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d73fff640 1 -- 172.21.15.115:0/4018709166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d68004d00 msgr2=0x7f3d7c1ef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d73fff640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d68004d00 0x7f3d7c1ef900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d73fff640 1 -- 172.21.15.115:0/4018709166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c15be90 con 0x7f3d7c0c4420 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d73fff640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d7c0c4420 0x7f3d7c1f2490 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400d7a0 tx=0x7f3d6400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d797fa640 1 -- 172.21.15.115:0/4018709166 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d640074e0 con 0x7f3d7c0c4420 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d797fa640 1 -- 172.21.15.115:0/4018709166 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d64007640 con 0x7f3d7c0c4420 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d797fa640 1 -- 172.21.15.115:0/4018709166 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64004a20 con 0x7f3d7c0c4420 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.650+0000 7f3d81022640 1 -- 172.21.15.115:0/4018709166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c1efe40 con 0x7f3d7c0c4420 2023-12-01T01:52:10.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.651+0000 7f3d81022640 1 -- 172.21.15.115:0/4018709166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c15baf0 con 0x7f3d7c0c4420 2023-12-01T01:52:10.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.652+0000 7f3d81022640 1 -- 172.21.15.115:0/4018709166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c0673a0 con 0x7f3d7c0c4420 2023-12-01T01:52:10.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.653+0000 7f3d797fa640 1 -- 172.21.15.115:0/4018709166 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3d64004b80 con 0x7f3d7c0c4420 2023-12-01T01:52:10.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.653+0000 7f3d797fa640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d5c06a580 0x7f3d5c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:10.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.654+0000 7f3d797fa640 1 -- 172.21.15.115:0/4018709166 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d64088730 con 0x7f3d7c0c4420 2023-12-01T01:52:10.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.654+0000 7f3d7bfff640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d5c06a580 0x7f3d5c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:10.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.654+0000 7f3d7bfff640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d5c06a580 0x7f3d5c06ca40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c000f60 tx=0x7f3d6c009040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:10.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.658+0000 7f3d797fa640 1 -- 172.21.15.115:0/4018709166 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3d64053bb0 con 0x7f3d7c0c4420 2023-12-01T01:52:10.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.843+0000 7f3d81022640 1 -- 172.21.15.115:0/4018709166 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d7c15d610 con 0x7f3d5c06a580 2023-12-01T01:52:10.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.870+0000 7f3d797fa640 1 -- 172.21.15.115:0/4018709166 <== mgr.6723 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283803 (secure 0 0 0) 0x7f3d6c00d040 con 0x7f3d5c06a580 2023-12-01T01:52:10.876 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:10.877 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 -- 172.21.15.115:0/4018709166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d5c06a580 msgr2=0x7f3d5c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d5c06a580 0x7f3d5c06ca40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c000f60 tx=0x7f3d6c009040 comp rx=0 tx=0).stop 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 -- 172.21.15.115:0/4018709166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d7c0c4420 msgr2=0x7f3d7c1f2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d7c0c4420 0x7f3d7c1f2490 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400d7a0 tx=0x7f3d6400dc60 comp rx=0 tx=0).stop 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 -- 172.21.15.115:0/4018709166 shutdown_connections 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d5c06a580 0x7f3d5c06ca40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c1f3650 0x7f3d7c0ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d7c0c4420 0x7f3d7c1f2490 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 --2- 172.21.15.115:0/4018709166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d68004d00 0x7f3d7c1ef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 -- 172.21.15.115:0/4018709166 >> 172.21.15.115:0/4018709166 conn(0x7f3d7c0ce640 msgr2=0x7f3d7c158b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 -- 172.21.15.115:0/4018709166 shutdown_connections 2023-12-01T01:52:10.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:10.876+0000 7f3d81022640 1 -- 172.21.15.115:0/4018709166 wait complete. 2023-12-01T01:52:10.897 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T01:52:10.030629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270816,"kb_used_data":10144,"kb_used_omap":0,"kb_used_meta":260608,"kb_avail":754703904,"statfs":{"total":773094113280,"available":772816797696,"internally_reserved":0,"allocated":10387456,"data_stored":5535942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":266862592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.063369"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997898+0000","last_change":"2023-12-01T01:52:05.997898+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997898+0000","last_undegraded":"2023-12-01T01:52:05.997898+0000","last_fullsized":"2023-12-01T01:52:05.997898+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:12:04.610539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995940+0000","last_change":"2023-12-01T01:52:05.995940+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995940+0000","last_undegraded":"2023-12-01T01:52:05.995940+0000","last_fullsized":"2023-12-01T01:52:05.995940+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:53.713026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995242+0000","last_change":"2023-12-01T01:52:05.995242+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995242+0000","last_undegraded":"2023-12-01T01:52:05.995242+0000","last_fullsized":"2023-12-01T01:52:05.995242+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993787+0000","last_change":"2023-12-01T01:52:05.993787+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993787+0000","last_undegraded":"2023-12-01T01:52:05.993787+0000","last_fullsized":"2023-12-01T01:52:05.993787+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:54.640923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997994+0000","last_change":"2023-12-01T01:52:05.997994+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997994+0000","last_undegraded":"2023-12-01T01:52:05.997994+0000","last_fullsized":"2023-12-01T01:52:05.997994+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:11.568997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.983916+0000","last_change":"2023-12-01T01:52:05.983916+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.983916+0000","last_undegraded":"2023-12-01T01:52:05.983916+0000","last_fullsized":"2023-12-01T01:52:05.983916+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995670+0000","last_change":"2023-12-01T01:52:05.995670+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995670+0000","last_undegraded":"2023-12-01T01:52:05.995670+0000","last_fullsized":"2023-12-01T01:52:05.995670+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997463+0000","last_change":"2023-12-01T01:52:05.997463+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997463+0000","last_undegraded":"2023-12-01T01:52:05.997463+0000","last_fullsized":"2023-12-01T01:52:05.997463+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:19:57.884685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:06.001892+0000","last_change":"2023-12-01T01:52:06.001892+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:06.001892+0000","last_undegraded":"2023-12-01T01:52:06.001892+0000","last_fullsized":"2023-12-01T01:52:06.001892+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:44.957677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995914+0000","last_change":"2023-12-01T01:52:05.995914+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995914+0000","last_undegraded":"2023-12-01T01:52:05.995914+0000","last_fullsized":"2023-12-01T01:52:05.995914+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996024+0000","last_change":"2023-12-01T01:52:05.996024+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996024+0000","last_undegraded":"2023-12-01T01:52:05.996024+0000","last_fullsized":"2023-12-01T01:52:05.996024+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:58:13.647534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.980909+0000","last_change":"2023-12-01T01:52:05.980909+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.980909+0000","last_undegraded":"2023-12-01T01:52:05.980909+0000","last_fullsized":"2023-12-01T01:52:05.980909+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992885+0000","last_change":"2023-12-01T01:52:05.992885+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992885+0000","last_undegraded":"2023-12-01T01:52:05.992885+0000","last_fullsized":"2023-12-01T01:52:05.992885+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:03:27.876714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.978309+0000","last_change":"2023-12-01T01:52:05.978309+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.978309+0000","last_undegraded":"2023-12-01T01:52:05.978309+0000","last_fullsized":"2023-12-01T01:52:05.978309+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:22.669615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.982112+0000","last_change":"2023-12-01T01:52:05.982112+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.982112+0000","last_undegraded":"2023-12-01T01:52:05.982112+0000","last_fullsized":"2023-12-01T01:52:05.982112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996130+0000","last_change":"2023-12-01T01:52:05.996130+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996130+0000","last_undegraded":"2023-12-01T01:52:05.996130+0000","last_fullsized":"2023-12-01T01:52:05.996130+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:19.431492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997994+0000","last_change":"2023-12-01T01:52:05.997994+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997994+0000","last_undegraded":"2023-12-01T01:52:05.997994+0000","last_fullsized":"2023-12-01T01:52:05.997994+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:40.773157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.999346+0000","last_change":"2023-12-01T01:52:05.999346+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.999346+0000","last_undegraded":"2023-12-01T01:52:05.999346+0000","last_fullsized":"2023-12-01T01:52:05.999346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:43:04.482263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992107+0000","last_change":"2023-12-01T01:52:05.992107+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992107+0000","last_undegraded":"2023-12-01T01:52:05.992107+0000","last_fullsized":"2023-12-01T01:52:05.992107+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:23.538863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997895+0000","last_change":"2023-12-01T01:52:05.997895+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997895+0000","last_undegraded":"2023-12-01T01:52:05.997895+0000","last_fullsized":"2023-12-01T01:52:05.997895+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:07:56.031035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995149+0000","last_change":"2023-12-01T01:52:05.995149+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995149+0000","last_undegraded":"2023-12-01T01:52:05.995149+0000","last_fullsized":"2023-12-01T01:52:05.995149+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:11.391920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.981128+0000","last_change":"2023-12-01T01:52:05.981128+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.981128+0000","last_undegraded":"2023-12-01T01:52:05.981128+0000","last_fullsized":"2023-12-01T01:52:05.981128+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995097+0000","last_change":"2023-12-01T01:52:05.995097+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995097+0000","last_undegraded":"2023-12-01T01:52:05.995097+0000","last_fullsized":"2023-12-01T01:52:05.995097+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995030+0000","last_change":"2023-12-01T01:52:05.995030+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995030+0000","last_undegraded":"2023-12-01T01:52:05.995030+0000","last_fullsized":"2023-12-01T01:52:05.995030+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:51:42.770614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.998660+0000","last_change":"2023-12-01T01:52:05.998660+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.998660+0000","last_undegraded":"2023-12-01T01:52:05.998660+0000","last_fullsized":"2023-12-01T01:52:05.998660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:34:48.109363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.998980+0000","last_change":"2023-12-01T01:52:05.998980+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.998980+0000","last_undegraded":"2023-12-01T01:52:05.998980+0000","last_fullsized":"2023-12-01T01:52:05.998980+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:23.123384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.999166+0000","last_change":"2023-12-01T01:52:05.999166+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.999166+0000","last_undegraded":"2023-12-01T01:52:05.999166+0000","last_fullsized":"2023-12-01T01:52:05.999166+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:50:15.604117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993773+0000","last_change":"2023-12-01T01:52:05.993773+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993773+0000","last_undegraded":"2023-12-01T01:52:05.993773+0000","last_fullsized":"2023-12-01T01:52:05.993773+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:24.016226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992182+0000","last_change":"2023-12-01T01:52:05.992182+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992182+0000","last_undegraded":"2023-12-01T01:52:05.992182+0000","last_fullsized":"2023-12-01T01:52:05.992182+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993145+0000","last_change":"2023-12-01T01:52:05.993145+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993145+0000","last_undegraded":"2023-12-01T01:52:05.993145+0000","last_fullsized":"2023-12-01T01:52:05.993145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:26.262966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993080+0000","last_change":"2023-12-01T01:52:05.993080+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993080+0000","last_undegraded":"2023-12-01T01:52:05.993080+0000","last_fullsized":"2023-12-01T01:52:05.993080+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:38:19.366076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990723+0000","last_change":"2023-12-01T01:52:05.990723+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990723+0000","last_undegraded":"2023-12-01T01:52:05.990723+0000","last_fullsized":"2023-12-01T01:52:05.990723+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992259+0000","last_change":"2023-12-01T01:52:05.992259+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992259+0000","last_undegraded":"2023-12-01T01:52:05.992259+0000","last_fullsized":"2023-12-01T01:52:05.992259+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:42:00.920100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.004868+0000","last_change":"2023-12-01T01:52:05.999712+0000","last_active":"2023-12-01T01:52:06.004868+0000","last_peered":"2023-12-01T01:52:06.004868+0000","last_clean":"2023-12-01T01:52:06.004868+0000","last_became_active":"2023-12-01T01:52:05.999447+0000","last_became_peered":"2023-12-01T01:52:05.999447+0000","last_unstale":"2023-12-01T01:52:06.004868+0000","last_undegraded":"2023-12-01T01:52:06.004868+0000","last_fullsized":"2023-12-01T01:52:06.004868+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:51:43.565839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:06.000063+0000","last_change":"2023-12-01T01:52:06.000063+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:06.000063+0000","last_undegraded":"2023-12-01T01:52:06.000063+0000","last_fullsized":"2023-12-01T01:52:06.000063+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:28:10.371917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009405+0000","last_change":"2023-12-01T01:52:06.006120+0000","last_active":"2023-12-01T01:52:06.009405+0000","last_peered":"2023-12-01T01:52:06.009405+0000","last_clean":"2023-12-01T01:52:06.009405+0000","last_became_active":"2023-12-01T01:52:06.004701+0000","last_became_peered":"2023-12-01T01:52:06.004701+0000","last_unstale":"2023-12-01T01:52:06.009405+0000","last_undegraded":"2023-12-01T01:52:06.009405+0000","last_fullsized":"2023-12-01T01:52:06.009405+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:01.267746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.981512+0000","last_change":"2023-12-01T01:52:05.981512+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.981512+0000","last_undegraded":"2023-12-01T01:52:05.981512+0000","last_fullsized":"2023-12-01T01:52:05.981512+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007591+0000","last_change":"2023-12-01T01:52:05.999810+0000","last_active":"2023-12-01T01:52:06.007591+0000","last_peered":"2023-12-01T01:52:06.007591+0000","last_clean":"2023-12-01T01:52:06.007591+0000","last_became_active":"2023-12-01T01:52:05.999536+0000","last_became_peered":"2023-12-01T01:52:05.999536+0000","last_unstale":"2023-12-01T01:52:06.007591+0000","last_undegraded":"2023-12-01T01:52:06.007591+0000","last_fullsized":"2023-12-01T01:52:06.007591+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:38:18.361437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995865+0000","last_change":"2023-12-01T01:52:05.995865+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995865+0000","last_undegraded":"2023-12-01T01:52:05.995865+0000","last_fullsized":"2023-12-01T01:52:05.995865+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:42.937116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008002+0000","last_change":"2023-12-01T01:52:06.004649+0000","last_active":"2023-12-01T01:52:06.008002+0000","last_peered":"2023-12-01T01:52:06.008002+0000","last_clean":"2023-12-01T01:52:06.008002+0000","last_became_active":"2023-12-01T01:52:06.004356+0000","last_became_peered":"2023-12-01T01:52:06.004356+0000","last_unstale":"2023-12-01T01:52:06.008002+0000","last_undegraded":"2023-12-01T01:52:06.008002+0000","last_fullsized":"2023-12-01T01:52:06.008002+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992585+0000","last_change":"2023-12-01T01:52:05.992585+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992585+0000","last_undegraded":"2023-12-01T01:52:05.992585+0000","last_fullsized":"2023-12-01T01:52:05.992585+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:28.966506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.001044+0000","last_change":"2023-12-01T01:52:05.998965+0000","last_active":"2023-12-01T01:52:06.001044+0000","last_peered":"2023-12-01T01:52:06.001044+0000","last_clean":"2023-12-01T01:52:06.001044+0000","last_became_active":"2023-12-01T01:52:05.997814+0000","last_became_peered":"2023-12-01T01:52:05.997814+0000","last_unstale":"2023-12-01T01:52:06.001044+0000","last_undegraded":"2023-12-01T01:52:06.001044+0000","last_fullsized":"2023-12-01T01:52:06.001044+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:06:24.991705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.980164+0000","last_change":"2023-12-01T01:52:05.980164+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.980164+0000","last_undegraded":"2023-12-01T01:52:05.980164+0000","last_fullsized":"2023-12-01T01:52:05.980164+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008823+0000","last_change":"2023-12-01T01:52:06.006338+0000","last_active":"2023-12-01T01:52:06.008823+0000","last_peered":"2023-12-01T01:52:06.008823+0000","last_clean":"2023-12-01T01:52:06.008823+0000","last_became_active":"2023-12-01T01:52:06.006077+0000","last_became_peered":"2023-12-01T01:52:06.006077+0000","last_unstale":"2023-12-01T01:52:06.008823+0000","last_undegraded":"2023-12-01T01:52:06.008823+0000","last_fullsized":"2023-12-01T01:52:06.008823+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:07:16.419472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:06.000047+0000","last_change":"2023-12-01T01:52:06.000047+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:06.000047+0000","last_undegraded":"2023-12-01T01:52:06.000047+0000","last_fullsized":"2023-12-01T01:52:06.000047+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006519+0000","last_change":"2023-12-01T01:52:06.001988+0000","last_active":"2023-12-01T01:52:06.006519+0000","last_peered":"2023-12-01T01:52:06.006519+0000","last_clean":"2023-12-01T01:52:06.006519+0000","last_became_active":"2023-12-01T01:52:06.001633+0000","last_became_peered":"2023-12-01T01:52:06.001633+0000","last_unstale":"2023-12-01T01:52:06.006519+0000","last_undegraded":"2023-12-01T01:52:06.006519+0000","last_fullsized":"2023-12-01T01:52:06.006519+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.980933+0000","last_change":"2023-12-01T01:52:05.980933+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.980933+0000","last_undegraded":"2023-12-01T01:52:05.980933+0000","last_fullsized":"2023-12-01T01:52:05.980933+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008555+0000","last_change":"2023-12-01T01:52:06.005204+0000","last_active":"2023-12-01T01:52:06.008555+0000","last_peered":"2023-12-01T01:52:06.008555+0000","last_clean":"2023-12-01T01:52:06.008555+0000","last_became_active":"2023-12-01T01:52:06.004890+0000","last_became_peered":"2023-12-01T01:52:06.004890+0000","last_unstale":"2023-12-01T01:52:06.008555+0000","last_undegraded":"2023-12-01T01:52:06.008555+0000","last_fullsized":"2023-12-01T01:52:06.008555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995453+0000","last_change":"2023-12-01T01:52:05.995453+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995453+0000","last_undegraded":"2023-12-01T01:52:05.995453+0000","last_fullsized":"2023-12-01T01:52:05.995453+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:06:21.919614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008628+0000","last_change":"2023-12-01T01:52:06.005744+0000","last_active":"2023-12-01T01:52:06.008628+0000","last_peered":"2023-12-01T01:52:06.008628+0000","last_clean":"2023-12-01T01:52:06.008628+0000","last_became_active":"2023-12-01T01:52:06.005449+0000","last_became_peered":"2023-12-01T01:52:06.005449+0000","last_unstale":"2023-12-01T01:52:06.008628+0000","last_undegraded":"2023-12-01T01:52:06.008628+0000","last_fullsized":"2023-12-01T01:52:06.008628+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:23.011587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.991180+0000","last_change":"2023-12-01T01:52:05.991180+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.991180+0000","last_undegraded":"2023-12-01T01:52:05.991180+0000","last_fullsized":"2023-12-01T01:52:05.991180+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:00:34.778062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007590+0000","last_change":"2023-12-01T01:52:06.001972+0000","last_active":"2023-12-01T01:52:06.007590+0000","last_peered":"2023-12-01T01:52:06.007590+0000","last_clean":"2023-12-01T01:52:06.007590+0000","last_became_active":"2023-12-01T01:52:05.996697+0000","last_became_peered":"2023-12-01T01:52:05.996697+0000","last_unstale":"2023-12-01T01:52:06.007590+0000","last_undegraded":"2023-12-01T01:52:06.007590+0000","last_fullsized":"2023-12-01T01:52:06.007590+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994125+0000","last_change":"2023-12-01T01:52:05.994125+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994125+0000","last_undegraded":"2023-12-01T01:52:05.994125+0000","last_fullsized":"2023-12-01T01:52:05.994125+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:17:44.214762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009060+0000","last_change":"2023-12-01T01:52:06.006483+0000","last_active":"2023-12-01T01:52:06.009060+0000","last_peered":"2023-12-01T01:52:06.009060+0000","last_clean":"2023-12-01T01:52:06.009060+0000","last_became_active":"2023-12-01T01:52:06.004318+0000","last_became_peered":"2023-12-01T01:52:06.004318+0000","last_unstale":"2023-12-01T01:52:06.009060+0000","last_undegraded":"2023-12-01T01:52:06.009060+0000","last_fullsized":"2023-12-01T01:52:06.009060+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:48.962691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996433+0000","last_change":"2023-12-01T01:52:05.996433+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996433+0000","last_undegraded":"2023-12-01T01:52:05.996433+0000","last_fullsized":"2023-12-01T01:52:05.996433+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:40.656152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006559+0000","last_change":"2023-12-01T01:52:06.002932+0000","last_active":"2023-12-01T01:52:06.006559+0000","last_peered":"2023-12-01T01:52:06.006559+0000","last_clean":"2023-12-01T01:52:06.006559+0000","last_became_active":"2023-12-01T01:52:06.002600+0000","last_became_peered":"2023-12-01T01:52:06.002600+0000","last_unstale":"2023-12-01T01:52:06.006559+0000","last_undegraded":"2023-12-01T01:52:06.006559+0000","last_fullsized":"2023-12-01T01:52:06.006559+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.979785+0000","last_change":"2023-12-01T01:52:05.979785+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.979785+0000","last_undegraded":"2023-12-01T01:52:05.979785+0000","last_fullsized":"2023-12-01T01:52:05.979785+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008627+0000","last_change":"2023-12-01T01:52:06.007948+0000","last_active":"2023-12-01T01:52:06.008627+0000","last_peered":"2023-12-01T01:52:06.008627+0000","last_clean":"2023-12-01T01:52:06.008627+0000","last_became_active":"2023-12-01T01:52:06.007397+0000","last_became_peered":"2023-12-01T01:52:06.007397+0000","last_unstale":"2023-12-01T01:52:06.008627+0000","last_undegraded":"2023-12-01T01:52:06.008627+0000","last_fullsized":"2023-12-01T01:52:06.008627+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:59:33.535939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993720+0000","last_change":"2023-12-01T01:52:05.993720+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993720+0000","last_undegraded":"2023-12-01T01:52:05.993720+0000","last_fullsized":"2023-12-01T01:52:05.993720+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.010334+0000","last_change":"2023-12-01T01:52:06.009917+0000","last_active":"2023-12-01T01:52:06.010334+0000","last_peered":"2023-12-01T01:52:06.010334+0000","last_clean":"2023-12-01T01:52:06.010334+0000","last_became_active":"2023-12-01T01:52:06.009593+0000","last_became_peered":"2023-12-01T01:52:06.009593+0000","last_unstale":"2023-12-01T01:52:06.010334+0000","last_undegraded":"2023-12-01T01:52:06.010334+0000","last_fullsized":"2023-12-01T01:52:06.010334+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:11:33.399459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994026+0000","last_change":"2023-12-01T01:52:05.994026+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994026+0000","last_undegraded":"2023-12-01T01:52:05.994026+0000","last_fullsized":"2023-12-01T01:52:05.994026+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:58.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006463+0000","last_change":"2023-12-01T01:52:05.999943+0000","last_active":"2023-12-01T01:52:06.006463+0000","last_peered":"2023-12-01T01:52:06.006463+0000","last_clean":"2023-12-01T01:52:06.006463+0000","last_became_active":"2023-12-01T01:52:05.999649+0000","last_became_peered":"2023-12-01T01:52:05.999649+0000","last_unstale":"2023-12-01T01:52:06.006463+0000","last_undegraded":"2023-12-01T01:52:06.006463+0000","last_fullsized":"2023-12-01T01:52:06.006463+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:42.916412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996979+0000","last_change":"2023-12-01T01:52:05.996979+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996979+0000","last_undegraded":"2023-12-01T01:52:05.996979+0000","last_fullsized":"2023-12-01T01:52:05.996979+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:37:55.202441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007490+0000","last_change":"2023-12-01T01:52:06.002123+0000","last_active":"2023-12-01T01:52:06.007490+0000","last_peered":"2023-12-01T01:52:06.007490+0000","last_clean":"2023-12-01T01:52:06.007490+0000","last_became_active":"2023-12-01T01:52:06.001181+0000","last_became_peered":"2023-12-01T01:52:06.001181+0000","last_unstale":"2023-12-01T01:52:06.007490+0000","last_undegraded":"2023-12-01T01:52:06.007490+0000","last_fullsized":"2023-12-01T01:52:06.007490+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:27:36.883717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.989549+0000","last_change":"2023-12-01T01:52:05.989549+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.989549+0000","last_undegraded":"2023-12-01T01:52:05.989549+0000","last_fullsized":"2023-12-01T01:52:05.989549+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008299+0000","last_change":"2023-12-01T01:52:06.003482+0000","last_active":"2023-12-01T01:52:06.008299+0000","last_peered":"2023-12-01T01:52:06.008299+0000","last_clean":"2023-12-01T01:52:06.008299+0000","last_became_active":"2023-12-01T01:52:06.002884+0000","last_became_peered":"2023-12-01T01:52:06.002884+0000","last_unstale":"2023-12-01T01:52:06.008299+0000","last_undegraded":"2023-12-01T01:52:06.008299+0000","last_fullsized":"2023-12-01T01:52:06.008299+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:14:22.769057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.989532+0000","last_change":"2023-12-01T01:52:05.989532+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.989532+0000","last_undegraded":"2023-12-01T01:52:05.989532+0000","last_fullsized":"2023-12-01T01:52:05.989532+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007286+0000","last_change":"2023-12-01T01:52:05.997054+0000","last_active":"2023-12-01T01:52:06.007286+0000","last_peered":"2023-12-01T01:52:06.007286+0000","last_clean":"2023-12-01T01:52:06.007286+0000","last_became_active":"2023-12-01T01:52:05.996352+0000","last_became_peered":"2023-12-01T01:52:05.996352+0000","last_unstale":"2023-12-01T01:52:06.007286+0000","last_undegraded":"2023-12-01T01:52:06.007286+0000","last_fullsized":"2023-12-01T01:52:06.007286+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992916+0000","last_change":"2023-12-01T01:52:05.992916+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992916+0000","last_undegraded":"2023-12-01T01:52:05.992916+0000","last_fullsized":"2023-12-01T01:52:05.992916+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007780+0000","last_change":"2023-12-01T01:52:06.004398+0000","last_active":"2023-12-01T01:52:06.007780+0000","last_peered":"2023-12-01T01:52:06.007780+0000","last_clean":"2023-12-01T01:52:06.007780+0000","last_became_active":"2023-12-01T01:52:06.004055+0000","last_became_peered":"2023-12-01T01:52:06.004055+0000","last_unstale":"2023-12-01T01:52:06.007780+0000","last_undegraded":"2023-12-01T01:52:06.007780+0000","last_fullsized":"2023-12-01T01:52:06.007780+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:22.076125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990996+0000","last_change":"2023-12-01T01:52:05.990996+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990996+0000","last_undegraded":"2023-12-01T01:52:05.990996+0000","last_fullsized":"2023-12-01T01:52:05.990996+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006417+0000","last_change":"2023-12-01T01:52:06.003105+0000","last_active":"2023-12-01T01:52:06.006417+0000","last_peered":"2023-12-01T01:52:06.006417+0000","last_clean":"2023-12-01T01:52:06.006417+0000","last_became_active":"2023-12-01T01:52:06.001893+0000","last_became_peered":"2023-12-01T01:52:06.001893+0000","last_unstale":"2023-12-01T01:52:06.006417+0000","last_undegraded":"2023-12-01T01:52:06.006417+0000","last_fullsized":"2023-12-01T01:52:06.006417+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994654+0000","last_change":"2023-12-01T01:52:05.994654+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994654+0000","last_undegraded":"2023-12-01T01:52:05.994654+0000","last_fullsized":"2023-12-01T01:52:05.994654+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:05.997657+0000","last_change":"2023-12-01T01:52:05.990357+0000","last_active":"2023-12-01T01:52:05.997657+0000","last_peered":"2023-12-01T01:52:05.997657+0000","last_clean":"2023-12-01T01:52:05.997657+0000","last_became_active":"2023-12-01T01:52:05.988196+0000","last_became_peered":"2023-12-01T01:52:05.988196+0000","last_unstale":"2023-12-01T01:52:05.997657+0000","last_undegraded":"2023-12-01T01:52:05.997657+0000","last_fullsized":"2023-12-01T01:52:05.997657+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:53.636284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995954+0000","last_change":"2023-12-01T01:52:05.995954+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995954+0000","last_undegraded":"2023-12-01T01:52:05.995954+0000","last_fullsized":"2023-12-01T01:52:05.995954+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:54.842423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.002568+0000","last_change":"2023-12-01T01:52:05.990673+0000","last_active":"2023-12-01T01:52:06.002568+0000","last_peered":"2023-12-01T01:52:06.002568+0000","last_clean":"2023-12-01T01:52:06.002568+0000","last_became_active":"2023-12-01T01:52:05.989026+0000","last_became_peered":"2023-12-01T01:52:05.989026+0000","last_unstale":"2023-12-01T01:52:06.002568+0000","last_undegraded":"2023-12-01T01:52:06.002568+0000","last_fullsized":"2023-12-01T01:52:06.002568+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:51:43.565839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.979875+0000","last_change":"2023-12-01T01:52:05.979875+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.979875+0000","last_undegraded":"2023-12-01T01:52:05.979875+0000","last_fullsized":"2023-12-01T01:52:05.979875+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008997+0000","last_change":"2023-12-01T01:52:06.003576+0000","last_active":"2023-12-01T01:52:06.008997+0000","last_peered":"2023-12-01T01:52:06.008997+0000","last_clean":"2023-12-01T01:52:06.008997+0000","last_became_active":"2023-12-01T01:52:06.003254+0000","last_became_peered":"2023-12-01T01:52:06.003254+0000","last_unstale":"2023-12-01T01:52:06.008997+0000","last_undegraded":"2023-12-01T01:52:06.008997+0000","last_fullsized":"2023-12-01T01:52:06.008997+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:10.387281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995686+0000","last_change":"2023-12-01T01:52:05.995686+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995686+0000","last_undegraded":"2023-12-01T01:52:05.995686+0000","last_fullsized":"2023-12-01T01:52:05.995686+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:31.031923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008408+0000","last_change":"2023-12-01T01:52:06.003659+0000","last_active":"2023-12-01T01:52:06.008408+0000","last_peered":"2023-12-01T01:52:06.008408+0000","last_clean":"2023-12-01T01:52:06.008408+0000","last_became_active":"2023-12-01T01:52:06.003228+0000","last_became_peered":"2023-12-01T01:52:06.003228+0000","last_unstale":"2023-12-01T01:52:06.008408+0000","last_undegraded":"2023-12-01T01:52:06.008408+0000","last_fullsized":"2023-12-01T01:52:06.008408+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:14.620804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994032+0000","last_change":"2023-12-01T01:52:05.994032+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994032+0000","last_undegraded":"2023-12-01T01:52:05.994032+0000","last_fullsized":"2023-12-01T01:52:05.994032+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:35:33.172589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007657+0000","last_change":"2023-12-01T01:52:06.000889+0000","last_active":"2023-12-01T01:52:06.007657+0000","last_peered":"2023-12-01T01:52:06.007657+0000","last_clean":"2023-12-01T01:52:06.007657+0000","last_became_active":"2023-12-01T01:52:06.000644+0000","last_became_peered":"2023-12-01T01:52:06.000644+0000","last_unstale":"2023-12-01T01:52:06.007657+0000","last_undegraded":"2023-12-01T01:52:06.007657+0000","last_fullsized":"2023-12-01T01:52:06.007657+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:19:11.688232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.989769+0000","last_change":"2023-12-01T01:52:05.989769+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.989769+0000","last_undegraded":"2023-12-01T01:52:05.989769+0000","last_fullsized":"2023-12-01T01:52:05.989769+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009039+0000","last_change":"2023-12-01T01:52:06.006527+0000","last_active":"2023-12-01T01:52:06.009039+0000","last_peered":"2023-12-01T01:52:06.009039+0000","last_clean":"2023-12-01T01:52:06.009039+0000","last_became_active":"2023-12-01T01:52:06.005261+0000","last_became_peered":"2023-12-01T01:52:06.005261+0000","last_unstale":"2023-12-01T01:52:06.009039+0000","last_undegraded":"2023-12-01T01:52:06.009039+0000","last_fullsized":"2023-12-01T01:52:06.009039+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:51:05.076663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992926+0000","last_change":"2023-12-01T01:52:05.992926+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992926+0000","last_undegraded":"2023-12-01T01:52:05.992926+0000","last_fullsized":"2023-12-01T01:52:05.992926+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:33:59.577488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:05.999607+0000","last_change":"2023-12-01T01:52:05.997953+0000","last_active":"2023-12-01T01:52:05.999607+0000","last_peered":"2023-12-01T01:52:05.999607+0000","last_clean":"2023-12-01T01:52:05.999607+0000","last_became_active":"2023-12-01T01:52:05.997697+0000","last_became_peered":"2023-12-01T01:52:05.997697+0000","last_unstale":"2023-12-01T01:52:05.999607+0000","last_undegraded":"2023-12-01T01:52:05.999607+0000","last_fullsized":"2023-12-01T01:52:05.999607+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:41:32.705632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990662+0000","last_change":"2023-12-01T01:52:05.990662+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990662+0000","last_undegraded":"2023-12-01T01:52:05.990662+0000","last_fullsized":"2023-12-01T01:52:05.990662+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:35:55.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:05.999154+0000","last_change":"2023-12-01T01:52:05.996908+0000","last_active":"2023-12-01T01:52:05.999154+0000","last_peered":"2023-12-01T01:52:05.999154+0000","last_clean":"2023-12-01T01:52:05.999154+0000","last_became_active":"2023-12-01T01:52:05.995875+0000","last_became_peered":"2023-12-01T01:52:05.995875+0000","last_unstale":"2023-12-01T01:52:05.999154+0000","last_undegraded":"2023-12-01T01:52:05.999154+0000","last_fullsized":"2023-12-01T01:52:05.999154+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:37.605903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993946+0000","last_change":"2023-12-01T01:52:05.993946+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993946+0000","last_undegraded":"2023-12-01T01:52:05.993946+0000","last_fullsized":"2023-12-01T01:52:05.993946+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:49:44.833749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009144+0000","last_change":"2023-12-01T01:52:06.003362+0000","last_active":"2023-12-01T01:52:06.009144+0000","last_peered":"2023-12-01T01:52:06.009144+0000","last_clean":"2023-12-01T01:52:06.009144+0000","last_became_active":"2023-12-01T01:52:06.003048+0000","last_became_peered":"2023-12-01T01:52:06.003048+0000","last_unstale":"2023-12-01T01:52:06.009144+0000","last_undegraded":"2023-12-01T01:52:06.009144+0000","last_fullsized":"2023-12-01T01:52:06.009144+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:48.873308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995418+0000","last_change":"2023-12-01T01:52:05.995418+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995418+0000","last_undegraded":"2023-12-01T01:52:05.995418+0000","last_fullsized":"2023-12-01T01:52:05.995418+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.002548+0000","last_change":"2023-12-01T01:52:05.990037+0000","last_active":"2023-12-01T01:52:06.002548+0000","last_peered":"2023-12-01T01:52:06.002548+0000","last_clean":"2023-12-01T01:52:06.002548+0000","last_became_active":"2023-12-01T01:52:05.988261+0000","last_became_peered":"2023-12-01T01:52:05.988261+0000","last_unstale":"2023-12-01T01:52:06.002548+0000","last_undegraded":"2023-12-01T01:52:06.002548+0000","last_fullsized":"2023-12-01T01:52:06.002548+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:43.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990406+0000","last_change":"2023-12-01T01:52:05.990406+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990406+0000","last_undegraded":"2023-12-01T01:52:05.990406+0000","last_fullsized":"2023-12-01T01:52:05.990406+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:14:44.417471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.004673+0000","last_change":"2023-12-01T01:52:05.998968+0000","last_active":"2023-12-01T01:52:06.004673+0000","last_peered":"2023-12-01T01:52:06.004673+0000","last_clean":"2023-12-01T01:52:06.004673+0000","last_became_active":"2023-12-01T01:52:05.998647+0000","last_became_peered":"2023-12-01T01:52:05.998647+0000","last_unstale":"2023-12-01T01:52:06.004673+0000","last_undegraded":"2023-12-01T01:52:06.004673+0000","last_fullsized":"2023-12-01T01:52:06.004673+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:22.076125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.991444+0000","last_change":"2023-12-01T01:52:05.991444+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.991444+0000","last_undegraded":"2023-12-01T01:52:05.991444+0000","last_fullsized":"2023-12-01T01:52:05.991444+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:12.910138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007411+0000","last_change":"2023-12-01T01:52:06.000732+0000","last_active":"2023-12-01T01:52:06.007411+0000","last_peered":"2023-12-01T01:52:06.007411+0000","last_clean":"2023-12-01T01:52:06.007411+0000","last_became_active":"2023-12-01T01:52:06.000124+0000","last_became_peered":"2023-12-01T01:52:06.000124+0000","last_unstale":"2023-12-01T01:52:06.007411+0000","last_undegraded":"2023-12-01T01:52:06.007411+0000","last_fullsized":"2023-12-01T01:52:06.007411+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:39.158199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"85'172","reported_seq":430,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-12-01T01:52:04.991801+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:52:04.991801+0000","last_peered":"2023-12-01T01:52:04.991801+0000","last_clean":"2023-12-01T01:52:04.991801+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:52:04.991801+0000","last_undegraded":"2023-12-01T01:52:04.991801+0000","last_fullsized":"2023-12-01T01:52:04.991801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1159168,"data_stored":549101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0389999999999999,"15min":1.0640000000000001},"min":{"1min":0.65200000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2869999999999999,"5min":1.5449999999999999,"15min":1.8089999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.05,"15min":1.0489999999999999},"min":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.64000000000000001},"max":{"1min":1.2969999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.2010000000000001,"15min":1.2130000000000001},"min":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.351,"5min":1.8100000000000001,"15min":1.8420000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2410000000000001,"15min":1.252},"min":{"1min":0.78900000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.629,"5min":1.744,"15min":2.0870000000000002},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.127,"15min":1.121},"min":{"1min":0.73299999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.607,"5min":1.728,"15min":1.728},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.175,"5min":1.208,"15min":1.2230000000000001},"min":{"1min":0.89700000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5489999999999999,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2230000000000001,"15min":1.2310000000000001},"min":{"1min":0.75,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.3979999999999999,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.337,"5min":1.3440000000000001,"15min":1.347},"min":{"1min":1.123,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6259999999999999,"5min":2.1880000000000002,"15min":2.7050000000000001},"last":1.0740000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3160000000000001,"15min":1.296},"min":{"1min":1.036,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5840000000000001,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4550000000000001,"15min":1.4139999999999999},"min":{"1min":0.96399999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7210000000000001,"5min":2.0939999999999999,"15min":2.601},"last":1.2030000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3169999999999999,"15min":1.2689999999999999},"min":{"1min":0.84899999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.4500000000000002},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.532,"15min":1.5209999999999999},"min":{"1min":1.147,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.802,"5min":2.2930000000000001,"15min":2.8069999999999999},"last":1.2470000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4119999999999999,"15min":1.383},"min":{"1min":1.0129999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.294,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.397},"min":{"1min":1.115,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7110000000000001,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.1080000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.091,"15min":1.103},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.55,"5min":1.635,"15min":1.79},"last":1.21},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1399999999999999,"15min":1.127},"min":{"1min":0.77200000000000002,"5min":0.64400000000000002,"15min":0.61799999999999999},"max":{"1min":1.327,"5min":2.5569999999999999,"15min":2.73},"last":1.264}]},{"osd":1,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1639999999999999,"15min":1.1559999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.47,"5min":1.8620000000000001,"15min":2.2170000000000001},"last":0.88},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1950000000000001,"15min":1.2050000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.494,"5min":1.6259999999999999,"15min":2.0920000000000001},"last":1.327}]},{"osd":2,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.214,"15min":1.1659999999999999},"min":{"1min":0.72299999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4510000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.5},{"interface":"front","average":{"1min":1.131,"5min":1.2450000000000001,"15min":1.24},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5549999999999999,"5min":2.137,"15min":2.137},"last":1.6100000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2869999999999999,"15min":1.2789999999999999},"min":{"1min":0.91600000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4339999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.647},{"interface":"front","average":{"1min":1.365,"5min":1.4139999999999999,"15min":1.403},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.81299999999999994},"max":{"1min":1.5569999999999999,"5min":1.742,"15min":2.4020000000000001},"last":1.5840000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.298,"15min":1.28},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6830000000000001,"5min":2.226,"15min":2.3399999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.3460000000000001,"15min":1.347},"min":{"1min":0.93600000000000005,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.6060000000000001,"5min":1.931,"15min":2.6760000000000002},"last":1.46}]},{"osd":5,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4330000000000001,"15min":1.4099999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6779999999999999,"5min":2.1899999999999999,"15min":2.5539999999999998},"last":1.401},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.498,"15min":1.4750000000000001},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":0.748},"max":{"1min":1.786,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.6910000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.5389999999999999,"15min":1.514},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.89500000000000002},"max":{"1min":1.651,"5min":2.762,"15min":2.762},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.5960000000000001,"15min":1.5740000000000001},"min":{"1min":1.234,"5min":1.155,"15min":0.78700000000000003},"max":{"1min":1.7050000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.7569999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1712,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1753088,"data_stored":1139469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.1060000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.498,"5min":1.651,"15min":3.3980000000000001},"last":1.224},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.127,"15min":1.141},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.60099999999999998},"max":{"1min":1.5720000000000001,"5min":1.6479999999999999,"15min":2.153},"last":1.446}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.153,"15min":1.137},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.7010000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.73599999999999999,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.0109999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4039999999999999,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.6080000000000001,"5min":1.621,"15min":2.0630000000000002},"last":1.9239999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.286},"min":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.76900000000000002},"max":{"1min":1.5309999999999999,"5min":1.694,"15min":1.833},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3120000000000001,"15min":1.296},"min":{"1min":0.98099999999999998,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":1.6779999999999999,"5min":1.7110000000000001,"15min":2.1440000000000001},"last":2.0569999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.238,"15min":1.2390000000000001},"min":{"1min":0.86399999999999999,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.6779999999999999,"5min":1.853,"15min":2.5129999999999999},"last":1.421},{"interface":"front","average":{"1min":1.282,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.84999999999999998,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.6319999999999999,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.671}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2989999999999999,"15min":1.329},"min":{"1min":0.88500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":2.2970000000000002},"last":1.7889999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":1.125,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":1.962,"5min":1.962,"15min":3.23},"last":1.327}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4530000000000001,"15min":1.48},"min":{"1min":0.92400000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2,"5min":2,"15min":3.012},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4970000000000001,"15min":1.524},"min":{"1min":1.2689999999999999,"5min":0.873,"15min":0.873},"max":{"1min":2.073,"5min":2.073,"15min":2.8420000000000001},"last":2.1699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36264,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335576,"statfs":{"total":96636764160,"available":96599629824,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.119,"15min":1.1040000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.53,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.734,"5min":1.827,"15min":3.8069999999999999},"last":1.7130000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2350000000000001,"15min":1.2090000000000001},"min":{"1min":0.84599999999999997,"5min":0.81999999999999995,"15min":0.65900000000000003},"max":{"1min":1.528,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.127},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.131},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.671,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.2250000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2110000000000001,"15min":1.196},"min":{"1min":0.83999999999999997,"5min":0.78200000000000003,"15min":0.77600000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.3140000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.871,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.415}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.234},"min":{"1min":0.85399999999999998,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.8220000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3200000000000001,"15min":1.319},"min":{"1min":0.753,"5min":0.753,"15min":0.70399999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.3010000000000002},"last":1.288}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.96599999999999997,"5min":0.79200000000000004,"15min":0.61899999999999999},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.1549999999999998},"last":1.018},{"interface":"front","average":{"1min":1.248,"5min":1.2829999999999999,"15min":1.292},"min":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.5189999999999999,"5min":1.8029999999999999,"15min":2.2029999999999998},"last":1.3720000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.94999999999999996,"5min":0.84799999999999998,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.5029999999999999,"15min":1.4730000000000001},"min":{"1min":1.1000000000000001,"5min":0.96699999999999997,"15min":0.63200000000000001},"max":{"1min":1.7949999999999999,"5min":2.4990000000000001,"15min":2.54},"last":1.823}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.458,"15min":1.464},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.60599999999999998},"max":{"1min":1.6850000000000001,"5min":1.972,"15min":3.2789999999999999},"last":1.863},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5720000000000001,"15min":1.5580000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.04,"15min":2.157},"last":1.9490000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81599999999999995,"15min":0.84799999999999998},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.307},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.583},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97599999999999998,"15min":0.996},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.3160000000000001,"5min":1.476,"15min":1.9770000000000001},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.052,"15min":1.0660000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.38900000000000001},"max":{"1min":1.423,"5min":1.571,"15min":2.0779999999999998},"last":1.002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.92200000000000004,"15min":0.94999999999999996},"min":{"1min":0.504,"5min":0.48199999999999998,"15min":0.40600000000000003},"max":{"1min":1.385,"5min":1.587,"15min":1.6519999999999999},"last":1.1060000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99399999999999999,"15min":1.0469999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.436},"max":{"1min":1.4810000000000001,"5min":1.512,"15min":1.766},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.167,"15min":1.1850000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.48399999999999999},"max":{"1min":1.522,"5min":1.647,"15min":1.661},"last":0.92700000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.212,"15min":1.2450000000000001},"min":{"1min":0.76900000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.4550000000000001,"5min":1.5820000000000001,"15min":2.9350000000000001},"last":1.135},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2150000000000001,"15min":1.24},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.599,"5min":1.637,"15min":2.2389999999999999},"last":1.8480000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.24,"15min":1.29},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.63500000000000001},"max":{"1min":1.7669999999999999,"5min":1.806,"15min":3.476},"last":1.71},{"interface":"front","average":{"1min":1.244,"5min":1.266,"15min":1.3109999999999999},"min":{"1min":0.92100000000000004,"5min":0.82099999999999995,"15min":0.64200000000000002},"max":{"1min":1.681,"5min":1.9039999999999999,"15min":3.181},"last":1.048}]},{"osd":6,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.363,"15min":1.377},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.68300000000000005},"max":{"1min":1.6040000000000001,"5min":1.8580000000000001,"15min":2.532},"last":1.9410000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3919999999999999,"15min":1.4259999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.872},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":3.0249999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.389,"15min":1.421},"min":{"1min":1.0600000000000001,"5min":0.999,"15min":0.81499999999999995},"max":{"1min":1.998,"5min":1.998,"15min":2.1619999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.415,"15min":1.4610000000000001},"min":{"1min":0.91200000000000003,"5min":0.88300000000000001,"15min":0.83099999999999996},"max":{"1min":1.6879999999999999,"5min":1.8779999999999999,"15min":2.4590000000000001},"last":2.0259999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27100000000000002},"max":{"1min":1.0860000000000001,"5min":1.339,"15min":1.738},"last":1.286},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.026,"15min":1.01},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.375},"max":{"1min":1.4159999999999999,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96499999999999997,"15min":0.98999999999999999},"min":{"1min":0.54700000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.1779999999999999,"5min":1.45,"15min":1.7130000000000001},"last":1.202},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0720000000000001,"15min":1.0840000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.504},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.0249999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97499999999999998,"15min":1.0029999999999999},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.377,"5min":1.625,"15min":1.883},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.39600000000000002},"max":{"1min":1.415,"5min":1.613,"15min":2.6080000000000001},"last":1.3999999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.331,"15min":1.3180000000000001},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.75},"max":{"1min":1.7230000000000001,"5min":1.9950000000000001,"15min":2.2829999999999999},"last":1.155},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.246,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.964},"last":1.01}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3009999999999999,"15min":1.319},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.647,"5min":1.804,"15min":2.5779999999999998},"last":1.456},{"interface":"front","average":{"1min":1.375,"5min":1.387,"15min":1.375},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.901,"5min":2.048,"15min":3.0099999999999998},"last":1.1040000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.3200000000000001},"min":{"1min":0.84299999999999997,"5min":0.83399999999999996,"15min":0.77500000000000002},"max":{"1min":1.6439999999999999,"5min":1.78,"15min":2.2799999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.766,"15min":2.5089999999999999},"last":1.5149999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.456,"15min":1.448},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.79400000000000004},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.448},"last":1.327},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7649999999999999,"5min":1.802,"15min":2.5049999999999999},"last":1.244}]}]},{"osd":1,"up_from":13,"seq":55834575041,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1740800,"data_stored":1133202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85199999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.318},"max":{"1min":1.341,"5min":1.5049999999999999,"15min":3.056},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.052,"5min":0.97999999999999998,"15min":1.016},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":3.4129999999999998},"last":1.081}]},{"osd":2,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.88600000000000001,"15min":0.89600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.36399999999999999},"max":{"1min":1.1919999999999999,"5min":1.54,"15min":3.3660000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0680000000000001,"15min":1.095},"min":{"1min":0.71199999999999997,"5min":0.47899999999999998,"15min":0.39000000000000001},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":3.4980000000000002},"last":1.1930000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.194,"15min":1.216},"min":{"1min":0.55400000000000005,"5min":0.51900000000000002,"15min":0.44},"max":{"1min":1.827,"5min":1.827,"15min":3.464},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.048,"15min":1.0529999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39700000000000002},"max":{"1min":1.657,"5min":1.6699999999999999,"15min":3.0960000000000001},"last":0.98899999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.175,"15min":1.1870000000000001},"min":{"1min":0.871,"5min":0.745,"15min":0.63800000000000001},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":3.7810000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3160000000000001,"15min":1.3440000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6559999999999999,"5min":1.8089999999999999,"15min":4.7270000000000003},"last":1.254}]},{"osd":5,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2829999999999999,"15min":1.2809999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6839999999999999,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.448},{"interface":"front","average":{"1min":1.3,"5min":1.268,"15min":1.296},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":1.825,"15min":3.6669999999999998},"last":1.1259999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3089999999999999,"15min":1.3360000000000001},"min":{"1min":0.79400000000000004,"5min":0.71099999999999997,"15min":0.68400000000000005},"max":{"1min":1.7490000000000001,"5min":1.7809999999999999,"15min":3.3300000000000001},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.3460000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5389999999999999,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.1619999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3979999999999999,"15min":1.4419999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.774,"5min":1.8939999999999999,"15min":4.6749999999999998},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.4159999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8440000000000001,"5min":1.992,"15min":3.5379999999999998},"last":1.522}]}]},{"osd":0,"up_from":13,"seq":55834575043,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.60699999999999998,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.3520000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.92600000000000005,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4390000000000001,"5min":1.5580000000000001,"15min":1.6910000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98499999999999999,"15min":1.006},"min":{"1min":0.67600000000000005,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.4690000000000001,"5min":2.157,"15min":2.157},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.113,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.84299999999999997,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.395,"5min":1.728,"15min":2.8159999999999998},"last":1.0820000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.58499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.954},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.143,"15min":1.169},"min":{"1min":0.54900000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7070000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1279999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.3089999999999999},"min":{"1min":1.05,"5min":0.78500000000000003,"15min":0.65700000000000003},"max":{"1min":1.5289999999999999,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.196},{"interface":"front","average":{"1min":1.22,"5min":1.214,"15min":1.2629999999999999},"min":{"1min":0.83299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.391}]},{"osd":5,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.337,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.71899999999999997},"max":{"1min":1.839,"5min":2.351,"15min":2.351},"last":1.232},{"interface":"front","average":{"1min":1.276,"5min":1.306,"15min":1.349},"min":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6299999999999999,"5min":2.2909999999999999,"15min":2.7050000000000001},"last":1.3100000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73099999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.157},"last":1.341},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.399},"min":{"1min":1.0129999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.292}]},{"osd":7,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":1.127,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.706,"5min":1.798,"15min":2.8999999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.361,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.276}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:52:10.903 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:52:11.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.146+0000 7efce75d2640 1 Processor -- start 2023-12-01T01:52:11.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.146+0000 7efce75d2640 1 -- start start 2023-12-01T01:52:11.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.147+0000 7efce75d2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efce00c9ef0 0x7efce006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.147+0000 7efce75d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce006b6f0 0x7efce0070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.147+0000 7efce75d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce006f8b0 con 0x7efce0159dd0 2023-12-01T01:52:11.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.147+0000 7efce75d2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce00adcb0 con 0x7efce00c9ef0 2023-12-01T01:52:11.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.147+0000 7efce75d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00ad9d0 con 0x7efce006b6f0 2023-12-01T01:52:11.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.148+0000 7efce4b6d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efce00c9ef0 0x7efce006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.148+0000 7efce4b6d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efce00c9ef0 0x7efce006fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53582/0 (socket says 172.21.15.115:53582) 2023-12-01T01:52:11.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.148+0000 7efce4b6d640 1 -- 172.21.15.115:0/2538327751 learned_addr learned my addr 172.21.15.115:0/2538327751 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:11.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.148+0000 7efce5b6f640 1 --2- 172.21.15.115:0/2538327751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce006b6f0 0x7efce0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.149+0000 7efce5b6f640 1 -- 172.21.15.115:0/2538327751 >> v2:172.21.15.142:3300/0 conn(0x7efce00c9ef0 msgr2=0x7efce006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:11.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.149+0000 7efce5b6f640 1 --2- 172.21.15.115:0/2538327751 >> v2:172.21.15.142:3300/0 conn(0x7efce00c9ef0 0x7efce006fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.149+0000 7efce5b6f640 1 -- 172.21.15.115:0/2538327751 >> v1:172.21.15.142:6789/0 conn(0x7efce0159dd0 legacy=0x7efce00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.149+0000 7efce5b6f640 1 -- 172.21.15.115:0/2538327751 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00c9000 con 0x7efce006b6f0 2023-12-01T01:52:11.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.150+0000 7efce5b6f640 1 --2- 172.21.15.115:0/2538327751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce006b6f0 0x7efce0070260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009d90 tx=0x7efcd802bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=864abca310244569 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:11.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efcd7fff640 1 -- 172.21.15.115:0/2538327751 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd802cb40 con 0x7efce006b6f0 2023-12-01T01:52:11.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efcd7fff640 1 -- 172.21.15.115:0/2538327751 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd802cca0 con 0x7efce006b6f0 2023-12-01T01:52:11.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efcd7fff640 1 -- 172.21.15.115:0/2538327751 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8034660 con 0x7efce006b6f0 2023-12-01T01:52:11.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efce75d2640 1 -- 172.21.15.115:0/2538327751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce006b6f0 msgr2=0x7efce0070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efce75d2640 1 --2- 172.21.15.115:0/2538327751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce006b6f0 0x7efce0070260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009d90 tx=0x7efcd802bd90 comp rx=0 tx=0).stop 2023-12-01T01:52:11.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efce75d2640 1 -- 172.21.15.115:0/2538327751 shutdown_connections 2023-12-01T01:52:11.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efce75d2640 1 --2- 172.21.15.115:0/2538327751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce006b6f0 0x7efce0070260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efce75d2640 1 --2- 172.21.15.115:0/2538327751 >> v2:172.21.15.142:3300/0 conn(0x7efce00c9ef0 0x7efce006fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efce75d2640 1 -- 172.21.15.115:0/2538327751 >> 172.21.15.115:0/2538327751 conn(0x7efce00ce880 msgr2=0x7efce00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:11.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efce75d2640 1 -- 172.21.15.115:0/2538327751 shutdown_connections 2023-12-01T01:52:11.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.151+0000 7efce75d2640 1 -- 172.21.15.115:0/2538327751 wait complete. 2023-12-01T01:52:11.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.153+0000 7efce75d2640 1 Processor -- start 2023-12-01T01:52:11.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.153+0000 7efce75d2640 1 -- start start 2023-12-01T01:52:11.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.154+0000 7efce75d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efce006b6f0 0x7efce00c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.154+0000 7efce75d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce00c9ef0 0x7efce0150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.154+0000 7efce75d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce0159dd0 0x7efce01502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.154+0000 7efce75d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce0150830 con 0x7efce006b6f0 2023-12-01T01:52:11.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.154+0000 7efce75d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce01513c0 con 0x7efce0159dd0 2023-12-01T01:52:11.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.154+0000 7efce75d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c2ff0 con 0x7efce00c9ef0 2023-12-01T01:52:11.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.154+0000 7efce536e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efce006b6f0 0x7efce00c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.154+0000 7efce536e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efce006b6f0 0x7efce00c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46956/0 (socket says 172.21.15.115:46956) 2023-12-01T01:52:11.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.154+0000 7efce536e640 1 -- 172.21.15.115:0/3478361806 learned_addr learned my addr 172.21.15.115:0/3478361806 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:11.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.155+0000 7efce5b6f640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce0159dd0 0x7efce01502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.155+0000 7efce4b6d640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce00c9ef0 0x7efce0150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.155+0000 7efce536e640 1 -- 172.21.15.115:0/3478361806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce00c9ef0 msgr2=0x7efce0150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.155+0000 7efce536e640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce00c9ef0 0x7efce0150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.155+0000 7efce536e640 1 -- 172.21.15.115:0/3478361806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce0159dd0 msgr2=0x7efce01502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.155+0000 7efce536e640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce0159dd0 0x7efce01502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.155+0000 7efce536e640 1 -- 172.21.15.115:0/3478361806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd8009a70 con 0x7efce006b6f0 2023-12-01T01:52:11.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.155+0000 7efce536e640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efce006b6f0 0x7efce00c5020 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00c450 tx=0x7efcdc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:11.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.155+0000 7efcd5ffb640 1 -- 172.21.15.115:0/3478361806 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0052d0 con 0x7efce006b6f0 2023-12-01T01:52:11.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.156+0000 7efcd5ffb640 1 -- 172.21.15.115:0/3478361806 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc005430 con 0x7efce006b6f0 2023-12-01T01:52:11.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.156+0000 7efcd5ffb640 1 -- 172.21.15.115:0/3478361806 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc012870 con 0x7efce006b6f0 2023-12-01T01:52:11.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.156+0000 7efce75d2640 1 -- 172.21.15.115:0/3478361806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce00c32d0 con 0x7efce006b6f0 2023-12-01T01:52:11.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.156+0000 7efce75d2640 1 -- 172.21.15.115:0/3478361806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce00c3890 con 0x7efce006b6f0 2023-12-01T01:52:11.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.157+0000 7efce75d2640 1 -- 172.21.15.115:0/3478361806 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce004e890 con 0x7efce006b6f0 2023-12-01T01:52:11.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.158+0000 7efcd5ffb640 1 -- 172.21.15.115:0/3478361806 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7efcdc0129d0 con 0x7efce006b6f0 2023-12-01T01:52:11.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.158+0000 7efcd5ffb640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efcc406a550 0x7efcc406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.159+0000 7efcd5ffb640 1 -- 172.21.15.115:0/3478361806 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcdc088600 con 0x7efce006b6f0 2023-12-01T01:52:11.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.159+0000 7efce4b6d640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efcc406a550 0x7efcc406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.159+0000 7efce4b6d640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efcc406a550 0x7efcc406ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efce00c2bb0 tx=0x7efcd0008040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:11.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.163+0000 7efcd5ffb640 1 -- 172.21.15.115:0/3478361806 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efcdc053ae0 con 0x7efce006b6f0 2023-12-01T01:52:11.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.334+0000 7efce75d2640 1 -- 172.21.15.115:0/3478361806 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efce006d0d0 con 0x7efcc406a550 2023-12-01T01:52:11.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.355+0000 7efcd5ffb640 1 -- 172.21.15.115:0/3478361806 <== mgr.6723 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283803 (secure 0 0 0) 0x7efcd0011970 con 0x7efcc406a550 2023-12-01T01:52:11.361 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:11.363 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:52:11.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.361+0000 7efce75d2640 1 -- 172.21.15.115:0/3478361806 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efcc406a550 msgr2=0x7efcc406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.361+0000 7efce75d2640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efcc406a550 0x7efcc406ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efce00c2bb0 tx=0x7efcd0008040 comp rx=0 tx=0).stop 2023-12-01T01:52:11.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.361+0000 7efce75d2640 1 -- 172.21.15.115:0/3478361806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efce006b6f0 msgr2=0x7efce00c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.361+0000 7efce75d2640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efce006b6f0 0x7efce00c5020 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00c450 tx=0x7efcdc00c910 comp rx=0 tx=0).stop 2023-12-01T01:52:11.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.361+0000 7efce75d2640 1 -- 172.21.15.115:0/3478361806 shutdown_connections 2023-12-01T01:52:11.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.361+0000 7efce75d2640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efcc406a550 0x7efcc406ca10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.361+0000 7efce75d2640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efce00c9ef0 0x7efce0150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.361+0000 7efce75d2640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efce0159dd0 0x7efce01502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.361+0000 7efce75d2640 1 --2- 172.21.15.115:0/3478361806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efce006b6f0 0x7efce00c5020 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.361+0000 7efce75d2640 1 -- 172.21.15.115:0/3478361806 >> 172.21.15.115:0/3478361806 conn(0x7efce00ce880 msgr2=0x7efce0158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:11.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.362+0000 7efce75d2640 1 -- 172.21.15.115:0/3478361806 shutdown_connections 2023-12-01T01:52:11.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.362+0000 7efce75d2640 1 -- 172.21.15.115:0/3478361806 wait complete. 2023-12-01T01:52:11.382 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T01:52:10.030629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270816,"kb_used_data":10144,"kb_used_omap":0,"kb_used_meta":260608,"kb_avail":754703904,"statfs":{"total":773094113280,"available":772816797696,"internally_reserved":0,"allocated":10387456,"data_stored":5535942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":266862592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.063369"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997898+0000","last_change":"2023-12-01T01:52:05.997898+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997898+0000","last_undegraded":"2023-12-01T01:52:05.997898+0000","last_fullsized":"2023-12-01T01:52:05.997898+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:12:04.610539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995940+0000","last_change":"2023-12-01T01:52:05.995940+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995940+0000","last_undegraded":"2023-12-01T01:52:05.995940+0000","last_fullsized":"2023-12-01T01:52:05.995940+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:53.713026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995242+0000","last_change":"2023-12-01T01:52:05.995242+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995242+0000","last_undegraded":"2023-12-01T01:52:05.995242+0000","last_fullsized":"2023-12-01T01:52:05.995242+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993787+0000","last_change":"2023-12-01T01:52:05.993787+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993787+0000","last_undegraded":"2023-12-01T01:52:05.993787+0000","last_fullsized":"2023-12-01T01:52:05.993787+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:54.640923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997994+0000","last_change":"2023-12-01T01:52:05.997994+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997994+0000","last_undegraded":"2023-12-01T01:52:05.997994+0000","last_fullsized":"2023-12-01T01:52:05.997994+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:11.568997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.983916+0000","last_change":"2023-12-01T01:52:05.983916+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.983916+0000","last_undegraded":"2023-12-01T01:52:05.983916+0000","last_fullsized":"2023-12-01T01:52:05.983916+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995670+0000","last_change":"2023-12-01T01:52:05.995670+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995670+0000","last_undegraded":"2023-12-01T01:52:05.995670+0000","last_fullsized":"2023-12-01T01:52:05.995670+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997463+0000","last_change":"2023-12-01T01:52:05.997463+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997463+0000","last_undegraded":"2023-12-01T01:52:05.997463+0000","last_fullsized":"2023-12-01T01:52:05.997463+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:19:57.884685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:06.001892+0000","last_change":"2023-12-01T01:52:06.001892+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:06.001892+0000","last_undegraded":"2023-12-01T01:52:06.001892+0000","last_fullsized":"2023-12-01T01:52:06.001892+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:44.957677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995914+0000","last_change":"2023-12-01T01:52:05.995914+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995914+0000","last_undegraded":"2023-12-01T01:52:05.995914+0000","last_fullsized":"2023-12-01T01:52:05.995914+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996024+0000","last_change":"2023-12-01T01:52:05.996024+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996024+0000","last_undegraded":"2023-12-01T01:52:05.996024+0000","last_fullsized":"2023-12-01T01:52:05.996024+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:58:13.647534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.980909+0000","last_change":"2023-12-01T01:52:05.980909+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.980909+0000","last_undegraded":"2023-12-01T01:52:05.980909+0000","last_fullsized":"2023-12-01T01:52:05.980909+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992885+0000","last_change":"2023-12-01T01:52:05.992885+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992885+0000","last_undegraded":"2023-12-01T01:52:05.992885+0000","last_fullsized":"2023-12-01T01:52:05.992885+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:03:27.876714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.978309+0000","last_change":"2023-12-01T01:52:05.978309+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.978309+0000","last_undegraded":"2023-12-01T01:52:05.978309+0000","last_fullsized":"2023-12-01T01:52:05.978309+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:22.669615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.982112+0000","last_change":"2023-12-01T01:52:05.982112+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.982112+0000","last_undegraded":"2023-12-01T01:52:05.982112+0000","last_fullsized":"2023-12-01T01:52:05.982112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996130+0000","last_change":"2023-12-01T01:52:05.996130+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996130+0000","last_undegraded":"2023-12-01T01:52:05.996130+0000","last_fullsized":"2023-12-01T01:52:05.996130+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:19.431492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997994+0000","last_change":"2023-12-01T01:52:05.997994+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997994+0000","last_undegraded":"2023-12-01T01:52:05.997994+0000","last_fullsized":"2023-12-01T01:52:05.997994+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:40.773157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.999346+0000","last_change":"2023-12-01T01:52:05.999346+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.999346+0000","last_undegraded":"2023-12-01T01:52:05.999346+0000","last_fullsized":"2023-12-01T01:52:05.999346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:43:04.482263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992107+0000","last_change":"2023-12-01T01:52:05.992107+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992107+0000","last_undegraded":"2023-12-01T01:52:05.992107+0000","last_fullsized":"2023-12-01T01:52:05.992107+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:23.538863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997895+0000","last_change":"2023-12-01T01:52:05.997895+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997895+0000","last_undegraded":"2023-12-01T01:52:05.997895+0000","last_fullsized":"2023-12-01T01:52:05.997895+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:07:56.031035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995149+0000","last_change":"2023-12-01T01:52:05.995149+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995149+0000","last_undegraded":"2023-12-01T01:52:05.995149+0000","last_fullsized":"2023-12-01T01:52:05.995149+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:11.391920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.981128+0000","last_change":"2023-12-01T01:52:05.981128+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.981128+0000","last_undegraded":"2023-12-01T01:52:05.981128+0000","last_fullsized":"2023-12-01T01:52:05.981128+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995097+0000","last_change":"2023-12-01T01:52:05.995097+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995097+0000","last_undegraded":"2023-12-01T01:52:05.995097+0000","last_fullsized":"2023-12-01T01:52:05.995097+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995030+0000","last_change":"2023-12-01T01:52:05.995030+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995030+0000","last_undegraded":"2023-12-01T01:52:05.995030+0000","last_fullsized":"2023-12-01T01:52:05.995030+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:51:42.770614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.998660+0000","last_change":"2023-12-01T01:52:05.998660+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.998660+0000","last_undegraded":"2023-12-01T01:52:05.998660+0000","last_fullsized":"2023-12-01T01:52:05.998660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:34:48.109363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.998980+0000","last_change":"2023-12-01T01:52:05.998980+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.998980+0000","last_undegraded":"2023-12-01T01:52:05.998980+0000","last_fullsized":"2023-12-01T01:52:05.998980+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:23.123384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.999166+0000","last_change":"2023-12-01T01:52:05.999166+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.999166+0000","last_undegraded":"2023-12-01T01:52:05.999166+0000","last_fullsized":"2023-12-01T01:52:05.999166+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:50:15.604117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993773+0000","last_change":"2023-12-01T01:52:05.993773+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993773+0000","last_undegraded":"2023-12-01T01:52:05.993773+0000","last_fullsized":"2023-12-01T01:52:05.993773+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:24.016226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992182+0000","last_change":"2023-12-01T01:52:05.992182+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992182+0000","last_undegraded":"2023-12-01T01:52:05.992182+0000","last_fullsized":"2023-12-01T01:52:05.992182+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993145+0000","last_change":"2023-12-01T01:52:05.993145+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993145+0000","last_undegraded":"2023-12-01T01:52:05.993145+0000","last_fullsized":"2023-12-01T01:52:05.993145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:26.262966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993080+0000","last_change":"2023-12-01T01:52:05.993080+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993080+0000","last_undegraded":"2023-12-01T01:52:05.993080+0000","last_fullsized":"2023-12-01T01:52:05.993080+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:38:19.366076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990723+0000","last_change":"2023-12-01T01:52:05.990723+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990723+0000","last_undegraded":"2023-12-01T01:52:05.990723+0000","last_fullsized":"2023-12-01T01:52:05.990723+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992259+0000","last_change":"2023-12-01T01:52:05.992259+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992259+0000","last_undegraded":"2023-12-01T01:52:05.992259+0000","last_fullsized":"2023-12-01T01:52:05.992259+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:42:00.920100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.004868+0000","last_change":"2023-12-01T01:52:05.999712+0000","last_active":"2023-12-01T01:52:06.004868+0000","last_peered":"2023-12-01T01:52:06.004868+0000","last_clean":"2023-12-01T01:52:06.004868+0000","last_became_active":"2023-12-01T01:52:05.999447+0000","last_became_peered":"2023-12-01T01:52:05.999447+0000","last_unstale":"2023-12-01T01:52:06.004868+0000","last_undegraded":"2023-12-01T01:52:06.004868+0000","last_fullsized":"2023-12-01T01:52:06.004868+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:51:43.565839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:06.000063+0000","last_change":"2023-12-01T01:52:06.000063+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:06.000063+0000","last_undegraded":"2023-12-01T01:52:06.000063+0000","last_fullsized":"2023-12-01T01:52:06.000063+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:28:10.371917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009405+0000","last_change":"2023-12-01T01:52:06.006120+0000","last_active":"2023-12-01T01:52:06.009405+0000","last_peered":"2023-12-01T01:52:06.009405+0000","last_clean":"2023-12-01T01:52:06.009405+0000","last_became_active":"2023-12-01T01:52:06.004701+0000","last_became_peered":"2023-12-01T01:52:06.004701+0000","last_unstale":"2023-12-01T01:52:06.009405+0000","last_undegraded":"2023-12-01T01:52:06.009405+0000","last_fullsized":"2023-12-01T01:52:06.009405+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:01.267746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.981512+0000","last_change":"2023-12-01T01:52:05.981512+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.981512+0000","last_undegraded":"2023-12-01T01:52:05.981512+0000","last_fullsized":"2023-12-01T01:52:05.981512+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007591+0000","last_change":"2023-12-01T01:52:05.999810+0000","last_active":"2023-12-01T01:52:06.007591+0000","last_peered":"2023-12-01T01:52:06.007591+0000","last_clean":"2023-12-01T01:52:06.007591+0000","last_became_active":"2023-12-01T01:52:05.999536+0000","last_became_peered":"2023-12-01T01:52:05.999536+0000","last_unstale":"2023-12-01T01:52:06.007591+0000","last_undegraded":"2023-12-01T01:52:06.007591+0000","last_fullsized":"2023-12-01T01:52:06.007591+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:38:18.361437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995865+0000","last_change":"2023-12-01T01:52:05.995865+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995865+0000","last_undegraded":"2023-12-01T01:52:05.995865+0000","last_fullsized":"2023-12-01T01:52:05.995865+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:42.937116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008002+0000","last_change":"2023-12-01T01:52:06.004649+0000","last_active":"2023-12-01T01:52:06.008002+0000","last_peered":"2023-12-01T01:52:06.008002+0000","last_clean":"2023-12-01T01:52:06.008002+0000","last_became_active":"2023-12-01T01:52:06.004356+0000","last_became_peered":"2023-12-01T01:52:06.004356+0000","last_unstale":"2023-12-01T01:52:06.008002+0000","last_undegraded":"2023-12-01T01:52:06.008002+0000","last_fullsized":"2023-12-01T01:52:06.008002+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992585+0000","last_change":"2023-12-01T01:52:05.992585+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992585+0000","last_undegraded":"2023-12-01T01:52:05.992585+0000","last_fullsized":"2023-12-01T01:52:05.992585+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:28.966506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.001044+0000","last_change":"2023-12-01T01:52:05.998965+0000","last_active":"2023-12-01T01:52:06.001044+0000","last_peered":"2023-12-01T01:52:06.001044+0000","last_clean":"2023-12-01T01:52:06.001044+0000","last_became_active":"2023-12-01T01:52:05.997814+0000","last_became_peered":"2023-12-01T01:52:05.997814+0000","last_unstale":"2023-12-01T01:52:06.001044+0000","last_undegraded":"2023-12-01T01:52:06.001044+0000","last_fullsized":"2023-12-01T01:52:06.001044+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:06:24.991705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.980164+0000","last_change":"2023-12-01T01:52:05.980164+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.980164+0000","last_undegraded":"2023-12-01T01:52:05.980164+0000","last_fullsized":"2023-12-01T01:52:05.980164+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008823+0000","last_change":"2023-12-01T01:52:06.006338+0000","last_active":"2023-12-01T01:52:06.008823+0000","last_peered":"2023-12-01T01:52:06.008823+0000","last_clean":"2023-12-01T01:52:06.008823+0000","last_became_active":"2023-12-01T01:52:06.006077+0000","last_became_peered":"2023-12-01T01:52:06.006077+0000","last_unstale":"2023-12-01T01:52:06.008823+0000","last_undegraded":"2023-12-01T01:52:06.008823+0000","last_fullsized":"2023-12-01T01:52:06.008823+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:07:16.419472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:06.000047+0000","last_change":"2023-12-01T01:52:06.000047+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:06.000047+0000","last_undegraded":"2023-12-01T01:52:06.000047+0000","last_fullsized":"2023-12-01T01:52:06.000047+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006519+0000","last_change":"2023-12-01T01:52:06.001988+0000","last_active":"2023-12-01T01:52:06.006519+0000","last_peered":"2023-12-01T01:52:06.006519+0000","last_clean":"2023-12-01T01:52:06.006519+0000","last_became_active":"2023-12-01T01:52:06.001633+0000","last_became_peered":"2023-12-01T01:52:06.001633+0000","last_unstale":"2023-12-01T01:52:06.006519+0000","last_undegraded":"2023-12-01T01:52:06.006519+0000","last_fullsized":"2023-12-01T01:52:06.006519+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.980933+0000","last_change":"2023-12-01T01:52:05.980933+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.980933+0000","last_undegraded":"2023-12-01T01:52:05.980933+0000","last_fullsized":"2023-12-01T01:52:05.980933+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008555+0000","last_change":"2023-12-01T01:52:06.005204+0000","last_active":"2023-12-01T01:52:06.008555+0000","last_peered":"2023-12-01T01:52:06.008555+0000","last_clean":"2023-12-01T01:52:06.008555+0000","last_became_active":"2023-12-01T01:52:06.004890+0000","last_became_peered":"2023-12-01T01:52:06.004890+0000","last_unstale":"2023-12-01T01:52:06.008555+0000","last_undegraded":"2023-12-01T01:52:06.008555+0000","last_fullsized":"2023-12-01T01:52:06.008555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995453+0000","last_change":"2023-12-01T01:52:05.995453+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995453+0000","last_undegraded":"2023-12-01T01:52:05.995453+0000","last_fullsized":"2023-12-01T01:52:05.995453+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:06:21.919614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008628+0000","last_change":"2023-12-01T01:52:06.005744+0000","last_active":"2023-12-01T01:52:06.008628+0000","last_peered":"2023-12-01T01:52:06.008628+0000","last_clean":"2023-12-01T01:52:06.008628+0000","last_became_active":"2023-12-01T01:52:06.005449+0000","last_became_peered":"2023-12-01T01:52:06.005449+0000","last_unstale":"2023-12-01T01:52:06.008628+0000","last_undegraded":"2023-12-01T01:52:06.008628+0000","last_fullsized":"2023-12-01T01:52:06.008628+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:23.011587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.991180+0000","last_change":"2023-12-01T01:52:05.991180+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.991180+0000","last_undegraded":"2023-12-01T01:52:05.991180+0000","last_fullsized":"2023-12-01T01:52:05.991180+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:00:34.778062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007590+0000","last_change":"2023-12-01T01:52:06.001972+0000","last_active":"2023-12-01T01:52:06.007590+0000","last_peered":"2023-12-01T01:52:06.007590+0000","last_clean":"2023-12-01T01:52:06.007590+0000","last_became_active":"2023-12-01T01:52:05.996697+0000","last_became_peered":"2023-12-01T01:52:05.996697+0000","last_unstale":"2023-12-01T01:52:06.007590+0000","last_undegraded":"2023-12-01T01:52:06.007590+0000","last_fullsized":"2023-12-01T01:52:06.007590+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994125+0000","last_change":"2023-12-01T01:52:05.994125+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994125+0000","last_undegraded":"2023-12-01T01:52:05.994125+0000","last_fullsized":"2023-12-01T01:52:05.994125+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:17:44.214762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009060+0000","last_change":"2023-12-01T01:52:06.006483+0000","last_active":"2023-12-01T01:52:06.009060+0000","last_peered":"2023-12-01T01:52:06.009060+0000","last_clean":"2023-12-01T01:52:06.009060+0000","last_became_active":"2023-12-01T01:52:06.004318+0000","last_became_peered":"2023-12-01T01:52:06.004318+0000","last_unstale":"2023-12-01T01:52:06.009060+0000","last_undegraded":"2023-12-01T01:52:06.009060+0000","last_fullsized":"2023-12-01T01:52:06.009060+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:48.962691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996433+0000","last_change":"2023-12-01T01:52:05.996433+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996433+0000","last_undegraded":"2023-12-01T01:52:05.996433+0000","last_fullsized":"2023-12-01T01:52:05.996433+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:40.656152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006559+0000","last_change":"2023-12-01T01:52:06.002932+0000","last_active":"2023-12-01T01:52:06.006559+0000","last_peered":"2023-12-01T01:52:06.006559+0000","last_clean":"2023-12-01T01:52:06.006559+0000","last_became_active":"2023-12-01T01:52:06.002600+0000","last_became_peered":"2023-12-01T01:52:06.002600+0000","last_unstale":"2023-12-01T01:52:06.006559+0000","last_undegraded":"2023-12-01T01:52:06.006559+0000","last_fullsized":"2023-12-01T01:52:06.006559+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.979785+0000","last_change":"2023-12-01T01:52:05.979785+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.979785+0000","last_undegraded":"2023-12-01T01:52:05.979785+0000","last_fullsized":"2023-12-01T01:52:05.979785+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008627+0000","last_change":"2023-12-01T01:52:06.007948+0000","last_active":"2023-12-01T01:52:06.008627+0000","last_peered":"2023-12-01T01:52:06.008627+0000","last_clean":"2023-12-01T01:52:06.008627+0000","last_became_active":"2023-12-01T01:52:06.007397+0000","last_became_peered":"2023-12-01T01:52:06.007397+0000","last_unstale":"2023-12-01T01:52:06.008627+0000","last_undegraded":"2023-12-01T01:52:06.008627+0000","last_fullsized":"2023-12-01T01:52:06.008627+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:59:33.535939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993720+0000","last_change":"2023-12-01T01:52:05.993720+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993720+0000","last_undegraded":"2023-12-01T01:52:05.993720+0000","last_fullsized":"2023-12-01T01:52:05.993720+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.010334+0000","last_change":"2023-12-01T01:52:06.009917+0000","last_active":"2023-12-01T01:52:06.010334+0000","last_peered":"2023-12-01T01:52:06.010334+0000","last_clean":"2023-12-01T01:52:06.010334+0000","last_became_active":"2023-12-01T01:52:06.009593+0000","last_became_peered":"2023-12-01T01:52:06.009593+0000","last_unstale":"2023-12-01T01:52:06.010334+0000","last_undegraded":"2023-12-01T01:52:06.010334+0000","last_fullsized":"2023-12-01T01:52:06.010334+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:11:33.399459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994026+0000","last_change":"2023-12-01T01:52:05.994026+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994026+0000","last_undegraded":"2023-12-01T01:52:05.994026+0000","last_fullsized":"2023-12-01T01:52:05.994026+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:58.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006463+0000","last_change":"2023-12-01T01:52:05.999943+0000","last_active":"2023-12-01T01:52:06.006463+0000","last_peered":"2023-12-01T01:52:06.006463+0000","last_clean":"2023-12-01T01:52:06.006463+0000","last_became_active":"2023-12-01T01:52:05.999649+0000","last_became_peered":"2023-12-01T01:52:05.999649+0000","last_unstale":"2023-12-01T01:52:06.006463+0000","last_undegraded":"2023-12-01T01:52:06.006463+0000","last_fullsized":"2023-12-01T01:52:06.006463+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:42.916412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996979+0000","last_change":"2023-12-01T01:52:05.996979+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996979+0000","last_undegraded":"2023-12-01T01:52:05.996979+0000","last_fullsized":"2023-12-01T01:52:05.996979+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:37:55.202441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007490+0000","last_change":"2023-12-01T01:52:06.002123+0000","last_active":"2023-12-01T01:52:06.007490+0000","last_peered":"2023-12-01T01:52:06.007490+0000","last_clean":"2023-12-01T01:52:06.007490+0000","last_became_active":"2023-12-01T01:52:06.001181+0000","last_became_peered":"2023-12-01T01:52:06.001181+0000","last_unstale":"2023-12-01T01:52:06.007490+0000","last_undegraded":"2023-12-01T01:52:06.007490+0000","last_fullsized":"2023-12-01T01:52:06.007490+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:27:36.883717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.989549+0000","last_change":"2023-12-01T01:52:05.989549+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.989549+0000","last_undegraded":"2023-12-01T01:52:05.989549+0000","last_fullsized":"2023-12-01T01:52:05.989549+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008299+0000","last_change":"2023-12-01T01:52:06.003482+0000","last_active":"2023-12-01T01:52:06.008299+0000","last_peered":"2023-12-01T01:52:06.008299+0000","last_clean":"2023-12-01T01:52:06.008299+0000","last_became_active":"2023-12-01T01:52:06.002884+0000","last_became_peered":"2023-12-01T01:52:06.002884+0000","last_unstale":"2023-12-01T01:52:06.008299+0000","last_undegraded":"2023-12-01T01:52:06.008299+0000","last_fullsized":"2023-12-01T01:52:06.008299+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:14:22.769057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.989532+0000","last_change":"2023-12-01T01:52:05.989532+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.989532+0000","last_undegraded":"2023-12-01T01:52:05.989532+0000","last_fullsized":"2023-12-01T01:52:05.989532+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007286+0000","last_change":"2023-12-01T01:52:05.997054+0000","last_active":"2023-12-01T01:52:06.007286+0000","last_peered":"2023-12-01T01:52:06.007286+0000","last_clean":"2023-12-01T01:52:06.007286+0000","last_became_active":"2023-12-01T01:52:05.996352+0000","last_became_peered":"2023-12-01T01:52:05.996352+0000","last_unstale":"2023-12-01T01:52:06.007286+0000","last_undegraded":"2023-12-01T01:52:06.007286+0000","last_fullsized":"2023-12-01T01:52:06.007286+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992916+0000","last_change":"2023-12-01T01:52:05.992916+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992916+0000","last_undegraded":"2023-12-01T01:52:05.992916+0000","last_fullsized":"2023-12-01T01:52:05.992916+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007780+0000","last_change":"2023-12-01T01:52:06.004398+0000","last_active":"2023-12-01T01:52:06.007780+0000","last_peered":"2023-12-01T01:52:06.007780+0000","last_clean":"2023-12-01T01:52:06.007780+0000","last_became_active":"2023-12-01T01:52:06.004055+0000","last_became_peered":"2023-12-01T01:52:06.004055+0000","last_unstale":"2023-12-01T01:52:06.007780+0000","last_undegraded":"2023-12-01T01:52:06.007780+0000","last_fullsized":"2023-12-01T01:52:06.007780+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:22.076125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990996+0000","last_change":"2023-12-01T01:52:05.990996+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990996+0000","last_undegraded":"2023-12-01T01:52:05.990996+0000","last_fullsized":"2023-12-01T01:52:05.990996+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006417+0000","last_change":"2023-12-01T01:52:06.003105+0000","last_active":"2023-12-01T01:52:06.006417+0000","last_peered":"2023-12-01T01:52:06.006417+0000","last_clean":"2023-12-01T01:52:06.006417+0000","last_became_active":"2023-12-01T01:52:06.001893+0000","last_became_peered":"2023-12-01T01:52:06.001893+0000","last_unstale":"2023-12-01T01:52:06.006417+0000","last_undegraded":"2023-12-01T01:52:06.006417+0000","last_fullsized":"2023-12-01T01:52:06.006417+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994654+0000","last_change":"2023-12-01T01:52:05.994654+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994654+0000","last_undegraded":"2023-12-01T01:52:05.994654+0000","last_fullsized":"2023-12-01T01:52:05.994654+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:05.997657+0000","last_change":"2023-12-01T01:52:05.990357+0000","last_active":"2023-12-01T01:52:05.997657+0000","last_peered":"2023-12-01T01:52:05.997657+0000","last_clean":"2023-12-01T01:52:05.997657+0000","last_became_active":"2023-12-01T01:52:05.988196+0000","last_became_peered":"2023-12-01T01:52:05.988196+0000","last_unstale":"2023-12-01T01:52:05.997657+0000","last_undegraded":"2023-12-01T01:52:05.997657+0000","last_fullsized":"2023-12-01T01:52:05.997657+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:53.636284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995954+0000","last_change":"2023-12-01T01:52:05.995954+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995954+0000","last_undegraded":"2023-12-01T01:52:05.995954+0000","last_fullsized":"2023-12-01T01:52:05.995954+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:54.842423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.002568+0000","last_change":"2023-12-01T01:52:05.990673+0000","last_active":"2023-12-01T01:52:06.002568+0000","last_peered":"2023-12-01T01:52:06.002568+0000","last_clean":"2023-12-01T01:52:06.002568+0000","last_became_active":"2023-12-01T01:52:05.989026+0000","last_became_peered":"2023-12-01T01:52:05.989026+0000","last_unstale":"2023-12-01T01:52:06.002568+0000","last_undegraded":"2023-12-01T01:52:06.002568+0000","last_fullsized":"2023-12-01T01:52:06.002568+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:51:43.565839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.979875+0000","last_change":"2023-12-01T01:52:05.979875+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.979875+0000","last_undegraded":"2023-12-01T01:52:05.979875+0000","last_fullsized":"2023-12-01T01:52:05.979875+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008997+0000","last_change":"2023-12-01T01:52:06.003576+0000","last_active":"2023-12-01T01:52:06.008997+0000","last_peered":"2023-12-01T01:52:06.008997+0000","last_clean":"2023-12-01T01:52:06.008997+0000","last_became_active":"2023-12-01T01:52:06.003254+0000","last_became_peered":"2023-12-01T01:52:06.003254+0000","last_unstale":"2023-12-01T01:52:06.008997+0000","last_undegraded":"2023-12-01T01:52:06.008997+0000","last_fullsized":"2023-12-01T01:52:06.008997+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:10.387281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995686+0000","last_change":"2023-12-01T01:52:05.995686+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995686+0000","last_undegraded":"2023-12-01T01:52:05.995686+0000","last_fullsized":"2023-12-01T01:52:05.995686+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:31.031923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008408+0000","last_change":"2023-12-01T01:52:06.003659+0000","last_active":"2023-12-01T01:52:06.008408+0000","last_peered":"2023-12-01T01:52:06.008408+0000","last_clean":"2023-12-01T01:52:06.008408+0000","last_became_active":"2023-12-01T01:52:06.003228+0000","last_became_peered":"2023-12-01T01:52:06.003228+0000","last_unstale":"2023-12-01T01:52:06.008408+0000","last_undegraded":"2023-12-01T01:52:06.008408+0000","last_fullsized":"2023-12-01T01:52:06.008408+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:14.620804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994032+0000","last_change":"2023-12-01T01:52:05.994032+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994032+0000","last_undegraded":"2023-12-01T01:52:05.994032+0000","last_fullsized":"2023-12-01T01:52:05.994032+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:35:33.172589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007657+0000","last_change":"2023-12-01T01:52:06.000889+0000","last_active":"2023-12-01T01:52:06.007657+0000","last_peered":"2023-12-01T01:52:06.007657+0000","last_clean":"2023-12-01T01:52:06.007657+0000","last_became_active":"2023-12-01T01:52:06.000644+0000","last_became_peered":"2023-12-01T01:52:06.000644+0000","last_unstale":"2023-12-01T01:52:06.007657+0000","last_undegraded":"2023-12-01T01:52:06.007657+0000","last_fullsized":"2023-12-01T01:52:06.007657+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:19:11.688232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.989769+0000","last_change":"2023-12-01T01:52:05.989769+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.989769+0000","last_undegraded":"2023-12-01T01:52:05.989769+0000","last_fullsized":"2023-12-01T01:52:05.989769+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009039+0000","last_change":"2023-12-01T01:52:06.006527+0000","last_active":"2023-12-01T01:52:06.009039+0000","last_peered":"2023-12-01T01:52:06.009039+0000","last_clean":"2023-12-01T01:52:06.009039+0000","last_became_active":"2023-12-01T01:52:06.005261+0000","last_became_peered":"2023-12-01T01:52:06.005261+0000","last_unstale":"2023-12-01T01:52:06.009039+0000","last_undegraded":"2023-12-01T01:52:06.009039+0000","last_fullsized":"2023-12-01T01:52:06.009039+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:51:05.076663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992926+0000","last_change":"2023-12-01T01:52:05.992926+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992926+0000","last_undegraded":"2023-12-01T01:52:05.992926+0000","last_fullsized":"2023-12-01T01:52:05.992926+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:33:59.577488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:05.999607+0000","last_change":"2023-12-01T01:52:05.997953+0000","last_active":"2023-12-01T01:52:05.999607+0000","last_peered":"2023-12-01T01:52:05.999607+0000","last_clean":"2023-12-01T01:52:05.999607+0000","last_became_active":"2023-12-01T01:52:05.997697+0000","last_became_peered":"2023-12-01T01:52:05.997697+0000","last_unstale":"2023-12-01T01:52:05.999607+0000","last_undegraded":"2023-12-01T01:52:05.999607+0000","last_fullsized":"2023-12-01T01:52:05.999607+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:41:32.705632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990662+0000","last_change":"2023-12-01T01:52:05.990662+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990662+0000","last_undegraded":"2023-12-01T01:52:05.990662+0000","last_fullsized":"2023-12-01T01:52:05.990662+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:35:55.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:05.999154+0000","last_change":"2023-12-01T01:52:05.996908+0000","last_active":"2023-12-01T01:52:05.999154+0000","last_peered":"2023-12-01T01:52:05.999154+0000","last_clean":"2023-12-01T01:52:05.999154+0000","last_became_active":"2023-12-01T01:52:05.995875+0000","last_became_peered":"2023-12-01T01:52:05.995875+0000","last_unstale":"2023-12-01T01:52:05.999154+0000","last_undegraded":"2023-12-01T01:52:05.999154+0000","last_fullsized":"2023-12-01T01:52:05.999154+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:37.605903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993946+0000","last_change":"2023-12-01T01:52:05.993946+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993946+0000","last_undegraded":"2023-12-01T01:52:05.993946+0000","last_fullsized":"2023-12-01T01:52:05.993946+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:49:44.833749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009144+0000","last_change":"2023-12-01T01:52:06.003362+0000","last_active":"2023-12-01T01:52:06.009144+0000","last_peered":"2023-12-01T01:52:06.009144+0000","last_clean":"2023-12-01T01:52:06.009144+0000","last_became_active":"2023-12-01T01:52:06.003048+0000","last_became_peered":"2023-12-01T01:52:06.003048+0000","last_unstale":"2023-12-01T01:52:06.009144+0000","last_undegraded":"2023-12-01T01:52:06.009144+0000","last_fullsized":"2023-12-01T01:52:06.009144+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:48.873308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995418+0000","last_change":"2023-12-01T01:52:05.995418+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995418+0000","last_undegraded":"2023-12-01T01:52:05.995418+0000","last_fullsized":"2023-12-01T01:52:05.995418+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.002548+0000","last_change":"2023-12-01T01:52:05.990037+0000","last_active":"2023-12-01T01:52:06.002548+0000","last_peered":"2023-12-01T01:52:06.002548+0000","last_clean":"2023-12-01T01:52:06.002548+0000","last_became_active":"2023-12-01T01:52:05.988261+0000","last_became_peered":"2023-12-01T01:52:05.988261+0000","last_unstale":"2023-12-01T01:52:06.002548+0000","last_undegraded":"2023-12-01T01:52:06.002548+0000","last_fullsized":"2023-12-01T01:52:06.002548+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:43.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990406+0000","last_change":"2023-12-01T01:52:05.990406+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990406+0000","last_undegraded":"2023-12-01T01:52:05.990406+0000","last_fullsized":"2023-12-01T01:52:05.990406+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:14:44.417471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.004673+0000","last_change":"2023-12-01T01:52:05.998968+0000","last_active":"2023-12-01T01:52:06.004673+0000","last_peered":"2023-12-01T01:52:06.004673+0000","last_clean":"2023-12-01T01:52:06.004673+0000","last_became_active":"2023-12-01T01:52:05.998647+0000","last_became_peered":"2023-12-01T01:52:05.998647+0000","last_unstale":"2023-12-01T01:52:06.004673+0000","last_undegraded":"2023-12-01T01:52:06.004673+0000","last_fullsized":"2023-12-01T01:52:06.004673+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:22.076125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.991444+0000","last_change":"2023-12-01T01:52:05.991444+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.991444+0000","last_undegraded":"2023-12-01T01:52:05.991444+0000","last_fullsized":"2023-12-01T01:52:05.991444+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:12.910138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007411+0000","last_change":"2023-12-01T01:52:06.000732+0000","last_active":"2023-12-01T01:52:06.007411+0000","last_peered":"2023-12-01T01:52:06.007411+0000","last_clean":"2023-12-01T01:52:06.007411+0000","last_became_active":"2023-12-01T01:52:06.000124+0000","last_became_peered":"2023-12-01T01:52:06.000124+0000","last_unstale":"2023-12-01T01:52:06.007411+0000","last_undegraded":"2023-12-01T01:52:06.007411+0000","last_fullsized":"2023-12-01T01:52:06.007411+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:39.158199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"85'172","reported_seq":430,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-12-01T01:52:04.991801+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:52:04.991801+0000","last_peered":"2023-12-01T01:52:04.991801+0000","last_clean":"2023-12-01T01:52:04.991801+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:52:04.991801+0000","last_undegraded":"2023-12-01T01:52:04.991801+0000","last_fullsized":"2023-12-01T01:52:04.991801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1159168,"data_stored":549101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0389999999999999,"15min":1.0640000000000001},"min":{"1min":0.65200000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2869999999999999,"5min":1.5449999999999999,"15min":1.8089999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.05,"15min":1.0489999999999999},"min":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.64000000000000001},"max":{"1min":1.2969999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.2010000000000001,"15min":1.2130000000000001},"min":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.351,"5min":1.8100000000000001,"15min":1.8420000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2410000000000001,"15min":1.252},"min":{"1min":0.78900000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.629,"5min":1.744,"15min":2.0870000000000002},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.127,"15min":1.121},"min":{"1min":0.73299999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.607,"5min":1.728,"15min":1.728},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.175,"5min":1.208,"15min":1.2230000000000001},"min":{"1min":0.89700000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5489999999999999,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2230000000000001,"15min":1.2310000000000001},"min":{"1min":0.75,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.3979999999999999,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.337,"5min":1.3440000000000001,"15min":1.347},"min":{"1min":1.123,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6259999999999999,"5min":2.1880000000000002,"15min":2.7050000000000001},"last":1.0740000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3160000000000001,"15min":1.296},"min":{"1min":1.036,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5840000000000001,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4550000000000001,"15min":1.4139999999999999},"min":{"1min":0.96399999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7210000000000001,"5min":2.0939999999999999,"15min":2.601},"last":1.2030000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3169999999999999,"15min":1.2689999999999999},"min":{"1min":0.84899999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.4500000000000002},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.532,"15min":1.5209999999999999},"min":{"1min":1.147,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.802,"5min":2.2930000000000001,"15min":2.8069999999999999},"last":1.2470000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4119999999999999,"15min":1.383},"min":{"1min":1.0129999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.294,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.397},"min":{"1min":1.115,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7110000000000001,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.1080000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.091,"15min":1.103},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.55,"5min":1.635,"15min":1.79},"last":1.21},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1399999999999999,"15min":1.127},"min":{"1min":0.77200000000000002,"5min":0.64400000000000002,"15min":0.61799999999999999},"max":{"1min":1.327,"5min":2.5569999999999999,"15min":2.73},"last":1.264}]},{"osd":1,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1639999999999999,"15min":1.1559999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.47,"5min":1.8620000000000001,"15min":2.2170000000000001},"last":0.88},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1950000000000001,"15min":1.2050000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.494,"5min":1.6259999999999999,"15min":2.0920000000000001},"last":1.327}]},{"osd":2,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.214,"15min":1.1659999999999999},"min":{"1min":0.72299999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4510000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.5},{"interface":"front","average":{"1min":1.131,"5min":1.2450000000000001,"15min":1.24},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5549999999999999,"5min":2.137,"15min":2.137},"last":1.6100000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2869999999999999,"15min":1.2789999999999999},"min":{"1min":0.91600000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4339999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.647},{"interface":"front","average":{"1min":1.365,"5min":1.4139999999999999,"15min":1.403},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.81299999999999994},"max":{"1min":1.5569999999999999,"5min":1.742,"15min":2.4020000000000001},"last":1.5840000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.298,"15min":1.28},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6830000000000001,"5min":2.226,"15min":2.3399999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.3460000000000001,"15min":1.347},"min":{"1min":0.93600000000000005,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.6060000000000001,"5min":1.931,"15min":2.6760000000000002},"last":1.46}]},{"osd":5,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4330000000000001,"15min":1.4099999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6779999999999999,"5min":2.1899999999999999,"15min":2.5539999999999998},"last":1.401},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.498,"15min":1.4750000000000001},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":0.748},"max":{"1min":1.786,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.6910000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.5389999999999999,"15min":1.514},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.89500000000000002},"max":{"1min":1.651,"5min":2.762,"15min":2.762},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.5960000000000001,"15min":1.5740000000000001},"min":{"1min":1.234,"5min":1.155,"15min":0.78700000000000003},"max":{"1min":1.7050000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.7569999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1712,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1753088,"data_stored":1139469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.1060000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.498,"5min":1.651,"15min":3.3980000000000001},"last":1.224},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.127,"15min":1.141},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.60099999999999998},"max":{"1min":1.5720000000000001,"5min":1.6479999999999999,"15min":2.153},"last":1.446}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.153,"15min":1.137},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.7010000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.73599999999999999,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.0109999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4039999999999999,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.6080000000000001,"5min":1.621,"15min":2.0630000000000002},"last":1.9239999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.286},"min":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.76900000000000002},"max":{"1min":1.5309999999999999,"5min":1.694,"15min":1.833},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3120000000000001,"15min":1.296},"min":{"1min":0.98099999999999998,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":1.6779999999999999,"5min":1.7110000000000001,"15min":2.1440000000000001},"last":2.0569999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.238,"15min":1.2390000000000001},"min":{"1min":0.86399999999999999,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.6779999999999999,"5min":1.853,"15min":2.5129999999999999},"last":1.421},{"interface":"front","average":{"1min":1.282,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.84999999999999998,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.6319999999999999,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.671}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2989999999999999,"15min":1.329},"min":{"1min":0.88500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":2.2970000000000002},"last":1.7889999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":1.125,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":1.962,"5min":1.962,"15min":3.23},"last":1.327}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4530000000000001,"15min":1.48},"min":{"1min":0.92400000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2,"5min":2,"15min":3.012},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4970000000000001,"15min":1.524},"min":{"1min":1.2689999999999999,"5min":0.873,"15min":0.873},"max":{"1min":2.073,"5min":2.073,"15min":2.8420000000000001},"last":2.1699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36264,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335576,"statfs":{"total":96636764160,"available":96599629824,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.119,"15min":1.1040000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.53,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.734,"5min":1.827,"15min":3.8069999999999999},"last":1.7130000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2350000000000001,"15min":1.2090000000000001},"min":{"1min":0.84599999999999997,"5min":0.81999999999999995,"15min":0.65900000000000003},"max":{"1min":1.528,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.127},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.131},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.671,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.2250000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2110000000000001,"15min":1.196},"min":{"1min":0.83999999999999997,"5min":0.78200000000000003,"15min":0.77600000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.3140000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.871,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.415}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.234},"min":{"1min":0.85399999999999998,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.8220000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3200000000000001,"15min":1.319},"min":{"1min":0.753,"5min":0.753,"15min":0.70399999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.3010000000000002},"last":1.288}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.96599999999999997,"5min":0.79200000000000004,"15min":0.61899999999999999},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.1549999999999998},"last":1.018},{"interface":"front","average":{"1min":1.248,"5min":1.2829999999999999,"15min":1.292},"min":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.5189999999999999,"5min":1.8029999999999999,"15min":2.2029999999999998},"last":1.3720000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.94999999999999996,"5min":0.84799999999999998,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.5029999999999999,"15min":1.4730000000000001},"min":{"1min":1.1000000000000001,"5min":0.96699999999999997,"15min":0.63200000000000001},"max":{"1min":1.7949999999999999,"5min":2.4990000000000001,"15min":2.54},"last":1.823}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.458,"15min":1.464},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.60599999999999998},"max":{"1min":1.6850000000000001,"5min":1.972,"15min":3.2789999999999999},"last":1.863},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5720000000000001,"15min":1.5580000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.04,"15min":2.157},"last":1.9490000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81599999999999995,"15min":0.84799999999999998},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.307},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.583},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97599999999999998,"15min":0.996},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.3160000000000001,"5min":1.476,"15min":1.9770000000000001},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.052,"15min":1.0660000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.38900000000000001},"max":{"1min":1.423,"5min":1.571,"15min":2.0779999999999998},"last":1.002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.92200000000000004,"15min":0.94999999999999996},"min":{"1min":0.504,"5min":0.48199999999999998,"15min":0.40600000000000003},"max":{"1min":1.385,"5min":1.587,"15min":1.6519999999999999},"last":1.1060000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99399999999999999,"15min":1.0469999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.436},"max":{"1min":1.4810000000000001,"5min":1.512,"15min":1.766},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.167,"15min":1.1850000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.48399999999999999},"max":{"1min":1.522,"5min":1.647,"15min":1.661},"last":0.92700000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.212,"15min":1.2450000000000001},"min":{"1min":0.76900000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.4550000000000001,"5min":1.5820000000000001,"15min":2.9350000000000001},"last":1.135},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2150000000000001,"15min":1.24},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.599,"5min":1.637,"15min":2.2389999999999999},"last":1.8480000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.24,"15min":1.29},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.63500000000000001},"max":{"1min":1.7669999999999999,"5min":1.806,"15min":3.476},"last":1.71},{"interface":"front","average":{"1min":1.244,"5min":1.266,"15min":1.3109999999999999},"min":{"1min":0.92100000000000004,"5min":0.82099999999999995,"15min":0.64200000000000002},"max":{"1min":1.681,"5min":1.9039999999999999,"15min":3.181},"last":1.048}]},{"osd":6,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.363,"15min":1.377},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.68300000000000005},"max":{"1min":1.6040000000000001,"5min":1.8580000000000001,"15min":2.532},"last":1.9410000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3919999999999999,"15min":1.4259999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.872},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":3.0249999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.389,"15min":1.421},"min":{"1min":1.0600000000000001,"5min":0.999,"15min":0.81499999999999995},"max":{"1min":1.998,"5min":1.998,"15min":2.1619999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.415,"15min":1.4610000000000001},"min":{"1min":0.91200000000000003,"5min":0.88300000000000001,"15min":0.83099999999999996},"max":{"1min":1.6879999999999999,"5min":1.8779999999999999,"15min":2.4590000000000001},"last":2.0259999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27100000000000002},"max":{"1min":1.0860000000000001,"5min":1.339,"15min":1.738},"last":1.286},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.026,"15min":1.01},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.375},"max":{"1min":1.4159999999999999,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96499999999999997,"15min":0.98999999999999999},"min":{"1min":0.54700000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.1779999999999999,"5min":1.45,"15min":1.7130000000000001},"last":1.202},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0720000000000001,"15min":1.0840000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.504},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.0249999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97499999999999998,"15min":1.0029999999999999},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.377,"5min":1.625,"15min":1.883},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.39600000000000002},"max":{"1min":1.415,"5min":1.613,"15min":2.6080000000000001},"last":1.3999999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.331,"15min":1.3180000000000001},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.75},"max":{"1min":1.7230000000000001,"5min":1.9950000000000001,"15min":2.2829999999999999},"last":1.155},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.246,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.964},"last":1.01}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3009999999999999,"15min":1.319},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.647,"5min":1.804,"15min":2.5779999999999998},"last":1.456},{"interface":"front","average":{"1min":1.375,"5min":1.387,"15min":1.375},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.901,"5min":2.048,"15min":3.0099999999999998},"last":1.1040000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.3200000000000001},"min":{"1min":0.84299999999999997,"5min":0.83399999999999996,"15min":0.77500000000000002},"max":{"1min":1.6439999999999999,"5min":1.78,"15min":2.2799999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.766,"15min":2.5089999999999999},"last":1.5149999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.456,"15min":1.448},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.79400000000000004},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.448},"last":1.327},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7649999999999999,"5min":1.802,"15min":2.5049999999999999},"last":1.244}]}]},{"osd":1,"up_from":13,"seq":55834575041,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1740800,"data_stored":1133202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85199999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.318},"max":{"1min":1.341,"5min":1.5049999999999999,"15min":3.056},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.052,"5min":0.97999999999999998,"15min":1.016},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":3.4129999999999998},"last":1.081}]},{"osd":2,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.88600000000000001,"15min":0.89600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.36399999999999999},"max":{"1min":1.1919999999999999,"5min":1.54,"15min":3.3660000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0680000000000001,"15min":1.095},"min":{"1min":0.71199999999999997,"5min":0.47899999999999998,"15min":0.39000000000000001},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":3.4980000000000002},"last":1.1930000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.194,"15min":1.216},"min":{"1min":0.55400000000000005,"5min":0.51900000000000002,"15min":0.44},"max":{"1min":1.827,"5min":1.827,"15min":3.464},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.048,"15min":1.0529999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39700000000000002},"max":{"1min":1.657,"5min":1.6699999999999999,"15min":3.0960000000000001},"last":0.98899999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.175,"15min":1.1870000000000001},"min":{"1min":0.871,"5min":0.745,"15min":0.63800000000000001},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":3.7810000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3160000000000001,"15min":1.3440000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6559999999999999,"5min":1.8089999999999999,"15min":4.7270000000000003},"last":1.254}]},{"osd":5,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2829999999999999,"15min":1.2809999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6839999999999999,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.448},{"interface":"front","average":{"1min":1.3,"5min":1.268,"15min":1.296},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":1.825,"15min":3.6669999999999998},"last":1.1259999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3089999999999999,"15min":1.3360000000000001},"min":{"1min":0.79400000000000004,"5min":0.71099999999999997,"15min":0.68400000000000005},"max":{"1min":1.7490000000000001,"5min":1.7809999999999999,"15min":3.3300000000000001},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.3460000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5389999999999999,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.1619999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3979999999999999,"15min":1.4419999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.774,"5min":1.8939999999999999,"15min":4.6749999999999998},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.4159999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8440000000000001,"5min":1.992,"15min":3.5379999999999998},"last":1.522}]}]},{"osd":0,"up_from":13,"seq":55834575043,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.60699999999999998,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.3520000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.92600000000000005,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4390000000000001,"5min":1.5580000000000001,"15min":1.6910000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98499999999999999,"15min":1.006},"min":{"1min":0.67600000000000005,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.4690000000000001,"5min":2.157,"15min":2.157},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.113,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.84299999999999997,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.395,"5min":1.728,"15min":2.8159999999999998},"last":1.0820000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.58499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.954},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.143,"15min":1.169},"min":{"1min":0.54900000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7070000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1279999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.3089999999999999},"min":{"1min":1.05,"5min":0.78500000000000003,"15min":0.65700000000000003},"max":{"1min":1.5289999999999999,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.196},{"interface":"front","average":{"1min":1.22,"5min":1.214,"15min":1.2629999999999999},"min":{"1min":0.83299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.391}]},{"osd":5,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.337,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.71899999999999997},"max":{"1min":1.839,"5min":2.351,"15min":2.351},"last":1.232},{"interface":"front","average":{"1min":1.276,"5min":1.306,"15min":1.349},"min":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6299999999999999,"5min":2.2909999999999999,"15min":2.7050000000000001},"last":1.3100000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73099999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.157},"last":1.341},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.399},"min":{"1min":1.0129999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.292}]},{"osd":7,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":1.127,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.706,"5min":1.798,"15min":2.8999999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.361,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.276}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:52:11.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:52:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.627+0000 7f1b23869640 1 Processor -- start 2023-12-01T01:52:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.627+0000 7f1b23869640 1 -- start start 2023-12-01T01:52:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.628+0000 7f1b23869640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b1c15e2f0 0x7f1b1c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.629+0000 7f1b23869640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1b1c06f2f0 0x7f1b1c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.629+0000 7f1b23869640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c0add10 con 0x7f1b1c0adf10 2023-12-01T01:52:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.629+0000 7f1b23869640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c0c5760 con 0x7f1b1c06f2f0 2023-12-01T01:52:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.629+0000 7f1b23869640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c15e6d0 con 0x7f1b1c15e2f0 2023-12-01T01:52:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.629+0000 7f1b20e04640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1b1c06f2f0 0x7f1b1c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.629+0000 7f1b20e04640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1b1c06f2f0 0x7f1b1c0caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46968/0 (socket says 172.21.15.115:46968) 2023-12-01T01:52:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.629+0000 7f1b21e06640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1b1c0adf10 0x7f1b1c0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35630/0 (socket says 172.21.15.115:35630) 2023-12-01T01:52:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.629+0000 7f1b20e04640 1 -- 172.21.15.115:0/1240806585 learned_addr learned my addr 172.21.15.115:0/1240806585 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.629+0000 7f1b20e04640 1 -- 172.21.15.115:0/1240806585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b1c15e2f0 msgr2=0x7f1b1c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.630+0000 7f1b20e04640 1 --2- 172.21.15.115:0/1240806585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b1c15e2f0 0x7f1b1c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.630+0000 7f1b20e04640 1 -- 172.21.15.115:0/1240806585 >> v1:172.21.15.115:6789/0 conn(0x7f1b1c0adf10 legacy=0x7f1b1c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.630+0000 7f1b20e04640 1 -- 172.21.15.115:0/1240806585 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c0cf7b0 con 0x7f1b1c06f2f0 2023-12-01T01:52:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.630+0000 7f1b13fff640 1 -- 172.21.15.115:0/1240806585 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008562444 0 0) 0x7f1b1c0add10 con 0x7f1b1c0adf10 2023-12-01T01:52:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.631+0000 7f1b20e04640 1 --2- 172.21.15.115:0/1240806585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c06f2f0 0x7f1b1c0caba0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009870 tx=0x7f1b0c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac58610b0b37ef8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:11.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.631+0000 7f1b13fff640 1 -- 172.21.15.115:0/1240806585 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0317c0 con 0x7f1b1c06f2f0 2023-12-01T01:52:11.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.631+0000 7f1b13fff640 1 -- 172.21.15.115:0/1240806585 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c031920 con 0x7f1b1c06f2f0 2023-12-01T01:52:11.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.631+0000 7f1b13fff640 1 -- 172.21.15.115:0/1240806585 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c031cb0 con 0x7f1b1c06f2f0 2023-12-01T01:52:11.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.631+0000 7f1b23869640 1 -- 172.21.15.115:0/1240806585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c06f2f0 msgr2=0x7f1b1c0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.631+0000 7f1b23869640 1 --2- 172.21.15.115:0/1240806585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c06f2f0 0x7f1b1c0caba0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009870 tx=0x7f1b0c02b860 comp rx=0 tx=0).stop 2023-12-01T01:52:11.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.632+0000 7f1b23869640 1 -- 172.21.15.115:0/1240806585 shutdown_connections 2023-12-01T01:52:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.632+0000 7f1b23869640 1 --2- 172.21.15.115:0/1240806585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c06f2f0 0x7f1b1c0caba0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.632+0000 7f1b23869640 1 --2- 172.21.15.115:0/1240806585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b1c15e2f0 0x7f1b1c0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.632+0000 7f1b23869640 1 -- 172.21.15.115:0/1240806585 >> 172.21.15.115:0/1240806585 conn(0x7f1b1c0ce880 msgr2=0x7f1b1c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.632+0000 7f1b23869640 1 -- 172.21.15.115:0/1240806585 shutdown_connections 2023-12-01T01:52:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.632+0000 7f1b23869640 1 -- 172.21.15.115:0/1240806585 wait complete. 2023-12-01T01:52:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.633+0000 7f1b23869640 1 Processor -- start 2023-12-01T01:52:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.634+0000 7f1b23869640 1 -- start start 2023-12-01T01:52:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.634+0000 7f1b23869640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b1c06f2f0 0x7f1b1c0c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.634+0000 7f1b23869640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c0adf10 0x7f1b1c1494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.634+0000 7f1b23869640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b1c15e2f0 0x7f1b1c149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.634+0000 7f1b23869640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c1f4720 con 0x7f1b1c0adf10 2023-12-01T01:52:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.634+0000 7f1b23869640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c14a050 con 0x7f1b1c06f2f0 2023-12-01T01:52:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.634+0000 7f1b23869640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c14a330 con 0x7f1b1c15e2f0 2023-12-01T01:52:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b20e04640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c0adf10 0x7f1b1c1494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b20e04640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c0adf10 0x7f1b1c1494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46984/0 (socket says 172.21.15.115:46984) 2023-12-01T01:52:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b20e04640 1 -- 172.21.15.115:0/4095276984 learned_addr learned my addr 172.21.15.115:0/4095276984 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b21605640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b1c06f2f0 0x7f1b1c0c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b21e06640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b1c15e2f0 0x7f1b1c149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b20e04640 1 -- 172.21.15.115:0/4095276984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b1c15e2f0 msgr2=0x7f1b1c149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b20e04640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b1c15e2f0 0x7f1b1c149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b20e04640 1 -- 172.21.15.115:0/4095276984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b1c06f2f0 msgr2=0x7f1b1c0c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b20e04640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b1c06f2f0 0x7f1b1c0c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b20e04640 1 -- 172.21.15.115:0/4095276984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c0094d0 con 0x7f1b1c0adf10 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.635+0000 7f1b21605640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b1c06f2f0 0x7f1b1c0c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.636+0000 7f1b20e04640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c0adf10 0x7f1b1c1494e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c034040 tx=0x7f1b0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.636+0000 7f1b11ffb640 1 -- 172.21.15.115:0/4095276984 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c004220 con 0x7f1b1c0adf10 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.636+0000 7f1b11ffb640 1 -- 172.21.15.115:0/4095276984 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c004380 con 0x7f1b1c0adf10 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.636+0000 7f1b11ffb640 1 -- 172.21.15.115:0/4095276984 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c003850 con 0x7f1b1c0adf10 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.636+0000 7f1b23869640 1 -- 172.21.15.115:0/4095276984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1c14a610 con 0x7f1b1c0adf10 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.636+0000 7f1b23869640 1 -- 172.21.15.115:0/4095276984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c14abd0 con 0x7f1b1c0adf10 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.637+0000 7f1b23869640 1 -- 172.21.15.115:0/4095276984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c06fda0 con 0x7f1b1c0adf10 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.638+0000 7f1b11ffb640 1 -- 172.21.15.115:0/4095276984 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1b0c043070 con 0x7f1b1c0adf10 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.639+0000 7f1b11ffb640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1b0006a550 0x7f1b0006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.639+0000 7f1b11ffb640 1 -- 172.21.15.115:0/4095276984 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b0c0aaba0 con 0x7f1b1c0adf10 2023-12-01T01:52:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.639+0000 7f1b21605640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1b0006a550 0x7f1b0006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:11.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.640+0000 7f1b21605640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1b0006a550 0x7f1b0006ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800a870 tx=0x7f1b18008040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:11.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.643+0000 7f1b11ffb640 1 -- 172.21.15.115:0/4095276984 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1b0c076190 con 0x7f1b1c0adf10 2023-12-01T01:52:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.830+0000 7f1b23869640 1 -- 172.21.15.115:0/4095276984 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b1c0b43d0 con 0x7f1b0006a550 2023-12-01T01:52:11.850 INFO:tasks.ceph.mds.b.smithi142.stderr:2023-12-01T01:52:11.849+0000 7f9f153c6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:52:11.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.852+0000 7f1b11ffb640 1 -- 172.21.15.115:0/4095276984 <== mgr.6723 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283803 (secure 0 0 0) 0x7f1b1800c7d0 con 0x7f1b0006a550 2023-12-01T01:52:11.856 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:11.859 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 -- 172.21.15.115:0/4095276984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1b0006a550 msgr2=0x7f1b0006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1b0006a550 0x7f1b0006ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800a870 tx=0x7f1b18008040 comp rx=0 tx=0).stop 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 -- 172.21.15.115:0/4095276984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c0adf10 msgr2=0x7f1b1c1494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c0adf10 0x7f1b1c1494e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c034040 tx=0x7f1b0c0046c0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 -- 172.21.15.115:0/4095276984 shutdown_connections 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1b0006a550 0x7f1b0006ca10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b1c15e2f0 0x7f1b1c149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b1c0adf10 0x7f1b1c1494e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 --2- 172.21.15.115:0/4095276984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b1c06f2f0 0x7f1b1c0c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 -- 172.21.15.115:0/4095276984 >> 172.21.15.115:0/4095276984 conn(0x7f1b1c0ce880 msgr2=0x7f1b1c15d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 -- 172.21.15.115:0/4095276984 shutdown_connections 2023-12-01T01:52:11.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:11.860+0000 7f1b23869640 1 -- 172.21.15.115:0/4095276984 wait complete. 2023-12-01T01:52:11.881 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T01:52:10.030629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270816,"kb_used_data":10144,"kb_used_omap":0,"kb_used_meta":260608,"kb_avail":754703904,"statfs":{"total":773094113280,"available":772816797696,"internally_reserved":0,"allocated":10387456,"data_stored":5535942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":266862592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.063369"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997898+0000","last_change":"2023-12-01T01:52:05.997898+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997898+0000","last_undegraded":"2023-12-01T01:52:05.997898+0000","last_fullsized":"2023-12-01T01:52:05.997898+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:12:04.610539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995940+0000","last_change":"2023-12-01T01:52:05.995940+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995940+0000","last_undegraded":"2023-12-01T01:52:05.995940+0000","last_fullsized":"2023-12-01T01:52:05.995940+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:53.713026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995242+0000","last_change":"2023-12-01T01:52:05.995242+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995242+0000","last_undegraded":"2023-12-01T01:52:05.995242+0000","last_fullsized":"2023-12-01T01:52:05.995242+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993787+0000","last_change":"2023-12-01T01:52:05.993787+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993787+0000","last_undegraded":"2023-12-01T01:52:05.993787+0000","last_fullsized":"2023-12-01T01:52:05.993787+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:54.640923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997994+0000","last_change":"2023-12-01T01:52:05.997994+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997994+0000","last_undegraded":"2023-12-01T01:52:05.997994+0000","last_fullsized":"2023-12-01T01:52:05.997994+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:11.568997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.983916+0000","last_change":"2023-12-01T01:52:05.983916+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.983916+0000","last_undegraded":"2023-12-01T01:52:05.983916+0000","last_fullsized":"2023-12-01T01:52:05.983916+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995670+0000","last_change":"2023-12-01T01:52:05.995670+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995670+0000","last_undegraded":"2023-12-01T01:52:05.995670+0000","last_fullsized":"2023-12-01T01:52:05.995670+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997463+0000","last_change":"2023-12-01T01:52:05.997463+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997463+0000","last_undegraded":"2023-12-01T01:52:05.997463+0000","last_fullsized":"2023-12-01T01:52:05.997463+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:19:57.884685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:06.001892+0000","last_change":"2023-12-01T01:52:06.001892+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:06.001892+0000","last_undegraded":"2023-12-01T01:52:06.001892+0000","last_fullsized":"2023-12-01T01:52:06.001892+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:44.957677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995914+0000","last_change":"2023-12-01T01:52:05.995914+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995914+0000","last_undegraded":"2023-12-01T01:52:05.995914+0000","last_fullsized":"2023-12-01T01:52:05.995914+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996024+0000","last_change":"2023-12-01T01:52:05.996024+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996024+0000","last_undegraded":"2023-12-01T01:52:05.996024+0000","last_fullsized":"2023-12-01T01:52:05.996024+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:58:13.647534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.980909+0000","last_change":"2023-12-01T01:52:05.980909+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.980909+0000","last_undegraded":"2023-12-01T01:52:05.980909+0000","last_fullsized":"2023-12-01T01:52:05.980909+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992885+0000","last_change":"2023-12-01T01:52:05.992885+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992885+0000","last_undegraded":"2023-12-01T01:52:05.992885+0000","last_fullsized":"2023-12-01T01:52:05.992885+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:03:27.876714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.978309+0000","last_change":"2023-12-01T01:52:05.978309+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.978309+0000","last_undegraded":"2023-12-01T01:52:05.978309+0000","last_fullsized":"2023-12-01T01:52:05.978309+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:22.669615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.982112+0000","last_change":"2023-12-01T01:52:05.982112+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.982112+0000","last_undegraded":"2023-12-01T01:52:05.982112+0000","last_fullsized":"2023-12-01T01:52:05.982112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996130+0000","last_change":"2023-12-01T01:52:05.996130+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996130+0000","last_undegraded":"2023-12-01T01:52:05.996130+0000","last_fullsized":"2023-12-01T01:52:05.996130+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:19.431492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997994+0000","last_change":"2023-12-01T01:52:05.997994+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997994+0000","last_undegraded":"2023-12-01T01:52:05.997994+0000","last_fullsized":"2023-12-01T01:52:05.997994+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:40.773157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.999346+0000","last_change":"2023-12-01T01:52:05.999346+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.999346+0000","last_undegraded":"2023-12-01T01:52:05.999346+0000","last_fullsized":"2023-12-01T01:52:05.999346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:43:04.482263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992107+0000","last_change":"2023-12-01T01:52:05.992107+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992107+0000","last_undegraded":"2023-12-01T01:52:05.992107+0000","last_fullsized":"2023-12-01T01:52:05.992107+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:23.538863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.997895+0000","last_change":"2023-12-01T01:52:05.997895+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.997895+0000","last_undegraded":"2023-12-01T01:52:05.997895+0000","last_fullsized":"2023-12-01T01:52:05.997895+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:07:56.031035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995149+0000","last_change":"2023-12-01T01:52:05.995149+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995149+0000","last_undegraded":"2023-12-01T01:52:05.995149+0000","last_fullsized":"2023-12-01T01:52:05.995149+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:11.391920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.981128+0000","last_change":"2023-12-01T01:52:05.981128+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.981128+0000","last_undegraded":"2023-12-01T01:52:05.981128+0000","last_fullsized":"2023-12-01T01:52:05.981128+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995097+0000","last_change":"2023-12-01T01:52:05.995097+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995097+0000","last_undegraded":"2023-12-01T01:52:05.995097+0000","last_fullsized":"2023-12-01T01:52:05.995097+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995030+0000","last_change":"2023-12-01T01:52:05.995030+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995030+0000","last_undegraded":"2023-12-01T01:52:05.995030+0000","last_fullsized":"2023-12-01T01:52:05.995030+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:51:42.770614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.998660+0000","last_change":"2023-12-01T01:52:05.998660+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.998660+0000","last_undegraded":"2023-12-01T01:52:05.998660+0000","last_fullsized":"2023-12-01T01:52:05.998660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:34:48.109363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.998980+0000","last_change":"2023-12-01T01:52:05.998980+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.998980+0000","last_undegraded":"2023-12-01T01:52:05.998980+0000","last_fullsized":"2023-12-01T01:52:05.998980+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:23.123384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.999166+0000","last_change":"2023-12-01T01:52:05.999166+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.999166+0000","last_undegraded":"2023-12-01T01:52:05.999166+0000","last_fullsized":"2023-12-01T01:52:05.999166+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:50:15.604117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993773+0000","last_change":"2023-12-01T01:52:05.993773+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993773+0000","last_undegraded":"2023-12-01T01:52:05.993773+0000","last_fullsized":"2023-12-01T01:52:05.993773+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:24.016226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992182+0000","last_change":"2023-12-01T01:52:05.992182+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992182+0000","last_undegraded":"2023-12-01T01:52:05.992182+0000","last_fullsized":"2023-12-01T01:52:05.992182+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993145+0000","last_change":"2023-12-01T01:52:05.993145+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993145+0000","last_undegraded":"2023-12-01T01:52:05.993145+0000","last_fullsized":"2023-12-01T01:52:05.993145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:26.262966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993080+0000","last_change":"2023-12-01T01:52:05.993080+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993080+0000","last_undegraded":"2023-12-01T01:52:05.993080+0000","last_fullsized":"2023-12-01T01:52:05.993080+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:38:19.366076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990723+0000","last_change":"2023-12-01T01:52:05.990723+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990723+0000","last_undegraded":"2023-12-01T01:52:05.990723+0000","last_fullsized":"2023-12-01T01:52:05.990723+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992259+0000","last_change":"2023-12-01T01:52:05.992259+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992259+0000","last_undegraded":"2023-12-01T01:52:05.992259+0000","last_fullsized":"2023-12-01T01:52:05.992259+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:42:00.920100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.004868+0000","last_change":"2023-12-01T01:52:05.999712+0000","last_active":"2023-12-01T01:52:06.004868+0000","last_peered":"2023-12-01T01:52:06.004868+0000","last_clean":"2023-12-01T01:52:06.004868+0000","last_became_active":"2023-12-01T01:52:05.999447+0000","last_became_peered":"2023-12-01T01:52:05.999447+0000","last_unstale":"2023-12-01T01:52:06.004868+0000","last_undegraded":"2023-12-01T01:52:06.004868+0000","last_fullsized":"2023-12-01T01:52:06.004868+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:51:43.565839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:06.000063+0000","last_change":"2023-12-01T01:52:06.000063+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:06.000063+0000","last_undegraded":"2023-12-01T01:52:06.000063+0000","last_fullsized":"2023-12-01T01:52:06.000063+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:28:10.371917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009405+0000","last_change":"2023-12-01T01:52:06.006120+0000","last_active":"2023-12-01T01:52:06.009405+0000","last_peered":"2023-12-01T01:52:06.009405+0000","last_clean":"2023-12-01T01:52:06.009405+0000","last_became_active":"2023-12-01T01:52:06.004701+0000","last_became_peered":"2023-12-01T01:52:06.004701+0000","last_unstale":"2023-12-01T01:52:06.009405+0000","last_undegraded":"2023-12-01T01:52:06.009405+0000","last_fullsized":"2023-12-01T01:52:06.009405+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:01.267746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.981512+0000","last_change":"2023-12-01T01:52:05.981512+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.981512+0000","last_undegraded":"2023-12-01T01:52:05.981512+0000","last_fullsized":"2023-12-01T01:52:05.981512+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007591+0000","last_change":"2023-12-01T01:52:05.999810+0000","last_active":"2023-12-01T01:52:06.007591+0000","last_peered":"2023-12-01T01:52:06.007591+0000","last_clean":"2023-12-01T01:52:06.007591+0000","last_became_active":"2023-12-01T01:52:05.999536+0000","last_became_peered":"2023-12-01T01:52:05.999536+0000","last_unstale":"2023-12-01T01:52:06.007591+0000","last_undegraded":"2023-12-01T01:52:06.007591+0000","last_fullsized":"2023-12-01T01:52:06.007591+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:38:18.361437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995865+0000","last_change":"2023-12-01T01:52:05.995865+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995865+0000","last_undegraded":"2023-12-01T01:52:05.995865+0000","last_fullsized":"2023-12-01T01:52:05.995865+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:42.937116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008002+0000","last_change":"2023-12-01T01:52:06.004649+0000","last_active":"2023-12-01T01:52:06.008002+0000","last_peered":"2023-12-01T01:52:06.008002+0000","last_clean":"2023-12-01T01:52:06.008002+0000","last_became_active":"2023-12-01T01:52:06.004356+0000","last_became_peered":"2023-12-01T01:52:06.004356+0000","last_unstale":"2023-12-01T01:52:06.008002+0000","last_undegraded":"2023-12-01T01:52:06.008002+0000","last_fullsized":"2023-12-01T01:52:06.008002+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992585+0000","last_change":"2023-12-01T01:52:05.992585+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992585+0000","last_undegraded":"2023-12-01T01:52:05.992585+0000","last_fullsized":"2023-12-01T01:52:05.992585+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:28.966506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.001044+0000","last_change":"2023-12-01T01:52:05.998965+0000","last_active":"2023-12-01T01:52:06.001044+0000","last_peered":"2023-12-01T01:52:06.001044+0000","last_clean":"2023-12-01T01:52:06.001044+0000","last_became_active":"2023-12-01T01:52:05.997814+0000","last_became_peered":"2023-12-01T01:52:05.997814+0000","last_unstale":"2023-12-01T01:52:06.001044+0000","last_undegraded":"2023-12-01T01:52:06.001044+0000","last_fullsized":"2023-12-01T01:52:06.001044+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:06:24.991705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.980164+0000","last_change":"2023-12-01T01:52:05.980164+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.980164+0000","last_undegraded":"2023-12-01T01:52:05.980164+0000","last_fullsized":"2023-12-01T01:52:05.980164+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008823+0000","last_change":"2023-12-01T01:52:06.006338+0000","last_active":"2023-12-01T01:52:06.008823+0000","last_peered":"2023-12-01T01:52:06.008823+0000","last_clean":"2023-12-01T01:52:06.008823+0000","last_became_active":"2023-12-01T01:52:06.006077+0000","last_became_peered":"2023-12-01T01:52:06.006077+0000","last_unstale":"2023-12-01T01:52:06.008823+0000","last_undegraded":"2023-12-01T01:52:06.008823+0000","last_fullsized":"2023-12-01T01:52:06.008823+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:07:16.419472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:06.000047+0000","last_change":"2023-12-01T01:52:06.000047+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:06.000047+0000","last_undegraded":"2023-12-01T01:52:06.000047+0000","last_fullsized":"2023-12-01T01:52:06.000047+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006519+0000","last_change":"2023-12-01T01:52:06.001988+0000","last_active":"2023-12-01T01:52:06.006519+0000","last_peered":"2023-12-01T01:52:06.006519+0000","last_clean":"2023-12-01T01:52:06.006519+0000","last_became_active":"2023-12-01T01:52:06.001633+0000","last_became_peered":"2023-12-01T01:52:06.001633+0000","last_unstale":"2023-12-01T01:52:06.006519+0000","last_undegraded":"2023-12-01T01:52:06.006519+0000","last_fullsized":"2023-12-01T01:52:06.006519+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.980933+0000","last_change":"2023-12-01T01:52:05.980933+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.980933+0000","last_undegraded":"2023-12-01T01:52:05.980933+0000","last_fullsized":"2023-12-01T01:52:05.980933+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008555+0000","last_change":"2023-12-01T01:52:06.005204+0000","last_active":"2023-12-01T01:52:06.008555+0000","last_peered":"2023-12-01T01:52:06.008555+0000","last_clean":"2023-12-01T01:52:06.008555+0000","last_became_active":"2023-12-01T01:52:06.004890+0000","last_became_peered":"2023-12-01T01:52:06.004890+0000","last_unstale":"2023-12-01T01:52:06.008555+0000","last_undegraded":"2023-12-01T01:52:06.008555+0000","last_fullsized":"2023-12-01T01:52:06.008555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995453+0000","last_change":"2023-12-01T01:52:05.995453+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995453+0000","last_undegraded":"2023-12-01T01:52:05.995453+0000","last_fullsized":"2023-12-01T01:52:05.995453+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:06:21.919614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008628+0000","last_change":"2023-12-01T01:52:06.005744+0000","last_active":"2023-12-01T01:52:06.008628+0000","last_peered":"2023-12-01T01:52:06.008628+0000","last_clean":"2023-12-01T01:52:06.008628+0000","last_became_active":"2023-12-01T01:52:06.005449+0000","last_became_peered":"2023-12-01T01:52:06.005449+0000","last_unstale":"2023-12-01T01:52:06.008628+0000","last_undegraded":"2023-12-01T01:52:06.008628+0000","last_fullsized":"2023-12-01T01:52:06.008628+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:23.011587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.991180+0000","last_change":"2023-12-01T01:52:05.991180+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.991180+0000","last_undegraded":"2023-12-01T01:52:05.991180+0000","last_fullsized":"2023-12-01T01:52:05.991180+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:00:34.778062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007590+0000","last_change":"2023-12-01T01:52:06.001972+0000","last_active":"2023-12-01T01:52:06.007590+0000","last_peered":"2023-12-01T01:52:06.007590+0000","last_clean":"2023-12-01T01:52:06.007590+0000","last_became_active":"2023-12-01T01:52:05.996697+0000","last_became_peered":"2023-12-01T01:52:05.996697+0000","last_unstale":"2023-12-01T01:52:06.007590+0000","last_undegraded":"2023-12-01T01:52:06.007590+0000","last_fullsized":"2023-12-01T01:52:06.007590+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994125+0000","last_change":"2023-12-01T01:52:05.994125+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994125+0000","last_undegraded":"2023-12-01T01:52:05.994125+0000","last_fullsized":"2023-12-01T01:52:05.994125+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:17:44.214762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009060+0000","last_change":"2023-12-01T01:52:06.006483+0000","last_active":"2023-12-01T01:52:06.009060+0000","last_peered":"2023-12-01T01:52:06.009060+0000","last_clean":"2023-12-01T01:52:06.009060+0000","last_became_active":"2023-12-01T01:52:06.004318+0000","last_became_peered":"2023-12-01T01:52:06.004318+0000","last_unstale":"2023-12-01T01:52:06.009060+0000","last_undegraded":"2023-12-01T01:52:06.009060+0000","last_fullsized":"2023-12-01T01:52:06.009060+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:48.962691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996433+0000","last_change":"2023-12-01T01:52:05.996433+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996433+0000","last_undegraded":"2023-12-01T01:52:05.996433+0000","last_fullsized":"2023-12-01T01:52:05.996433+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:40.656152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006559+0000","last_change":"2023-12-01T01:52:06.002932+0000","last_active":"2023-12-01T01:52:06.006559+0000","last_peered":"2023-12-01T01:52:06.006559+0000","last_clean":"2023-12-01T01:52:06.006559+0000","last_became_active":"2023-12-01T01:52:06.002600+0000","last_became_peered":"2023-12-01T01:52:06.002600+0000","last_unstale":"2023-12-01T01:52:06.006559+0000","last_undegraded":"2023-12-01T01:52:06.006559+0000","last_fullsized":"2023-12-01T01:52:06.006559+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.979785+0000","last_change":"2023-12-01T01:52:05.979785+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.979785+0000","last_undegraded":"2023-12-01T01:52:05.979785+0000","last_fullsized":"2023-12-01T01:52:05.979785+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008627+0000","last_change":"2023-12-01T01:52:06.007948+0000","last_active":"2023-12-01T01:52:06.008627+0000","last_peered":"2023-12-01T01:52:06.008627+0000","last_clean":"2023-12-01T01:52:06.008627+0000","last_became_active":"2023-12-01T01:52:06.007397+0000","last_became_peered":"2023-12-01T01:52:06.007397+0000","last_unstale":"2023-12-01T01:52:06.008627+0000","last_undegraded":"2023-12-01T01:52:06.008627+0000","last_fullsized":"2023-12-01T01:52:06.008627+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:59:33.535939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993720+0000","last_change":"2023-12-01T01:52:05.993720+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993720+0000","last_undegraded":"2023-12-01T01:52:05.993720+0000","last_fullsized":"2023-12-01T01:52:05.993720+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.010334+0000","last_change":"2023-12-01T01:52:06.009917+0000","last_active":"2023-12-01T01:52:06.010334+0000","last_peered":"2023-12-01T01:52:06.010334+0000","last_clean":"2023-12-01T01:52:06.010334+0000","last_became_active":"2023-12-01T01:52:06.009593+0000","last_became_peered":"2023-12-01T01:52:06.009593+0000","last_unstale":"2023-12-01T01:52:06.010334+0000","last_undegraded":"2023-12-01T01:52:06.010334+0000","last_fullsized":"2023-12-01T01:52:06.010334+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:11:33.399459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994026+0000","last_change":"2023-12-01T01:52:05.994026+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994026+0000","last_undegraded":"2023-12-01T01:52:05.994026+0000","last_fullsized":"2023-12-01T01:52:05.994026+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:58.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006463+0000","last_change":"2023-12-01T01:52:05.999943+0000","last_active":"2023-12-01T01:52:06.006463+0000","last_peered":"2023-12-01T01:52:06.006463+0000","last_clean":"2023-12-01T01:52:06.006463+0000","last_became_active":"2023-12-01T01:52:05.999649+0000","last_became_peered":"2023-12-01T01:52:05.999649+0000","last_unstale":"2023-12-01T01:52:06.006463+0000","last_undegraded":"2023-12-01T01:52:06.006463+0000","last_fullsized":"2023-12-01T01:52:06.006463+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:42.916412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.996979+0000","last_change":"2023-12-01T01:52:05.996979+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.996979+0000","last_undegraded":"2023-12-01T01:52:05.996979+0000","last_fullsized":"2023-12-01T01:52:05.996979+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:37:55.202441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007490+0000","last_change":"2023-12-01T01:52:06.002123+0000","last_active":"2023-12-01T01:52:06.007490+0000","last_peered":"2023-12-01T01:52:06.007490+0000","last_clean":"2023-12-01T01:52:06.007490+0000","last_became_active":"2023-12-01T01:52:06.001181+0000","last_became_peered":"2023-12-01T01:52:06.001181+0000","last_unstale":"2023-12-01T01:52:06.007490+0000","last_undegraded":"2023-12-01T01:52:06.007490+0000","last_fullsized":"2023-12-01T01:52:06.007490+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:27:36.883717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.989549+0000","last_change":"2023-12-01T01:52:05.989549+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.989549+0000","last_undegraded":"2023-12-01T01:52:05.989549+0000","last_fullsized":"2023-12-01T01:52:05.989549+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008299+0000","last_change":"2023-12-01T01:52:06.003482+0000","last_active":"2023-12-01T01:52:06.008299+0000","last_peered":"2023-12-01T01:52:06.008299+0000","last_clean":"2023-12-01T01:52:06.008299+0000","last_became_active":"2023-12-01T01:52:06.002884+0000","last_became_peered":"2023-12-01T01:52:06.002884+0000","last_unstale":"2023-12-01T01:52:06.008299+0000","last_undegraded":"2023-12-01T01:52:06.008299+0000","last_fullsized":"2023-12-01T01:52:06.008299+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:14:22.769057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.989532+0000","last_change":"2023-12-01T01:52:05.989532+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.989532+0000","last_undegraded":"2023-12-01T01:52:05.989532+0000","last_fullsized":"2023-12-01T01:52:05.989532+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007286+0000","last_change":"2023-12-01T01:52:05.997054+0000","last_active":"2023-12-01T01:52:06.007286+0000","last_peered":"2023-12-01T01:52:06.007286+0000","last_clean":"2023-12-01T01:52:06.007286+0000","last_became_active":"2023-12-01T01:52:05.996352+0000","last_became_peered":"2023-12-01T01:52:05.996352+0000","last_unstale":"2023-12-01T01:52:06.007286+0000","last_undegraded":"2023-12-01T01:52:06.007286+0000","last_fullsized":"2023-12-01T01:52:06.007286+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992916+0000","last_change":"2023-12-01T01:52:05.992916+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992916+0000","last_undegraded":"2023-12-01T01:52:05.992916+0000","last_fullsized":"2023-12-01T01:52:05.992916+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007780+0000","last_change":"2023-12-01T01:52:06.004398+0000","last_active":"2023-12-01T01:52:06.007780+0000","last_peered":"2023-12-01T01:52:06.007780+0000","last_clean":"2023-12-01T01:52:06.007780+0000","last_became_active":"2023-12-01T01:52:06.004055+0000","last_became_peered":"2023-12-01T01:52:06.004055+0000","last_unstale":"2023-12-01T01:52:06.007780+0000","last_undegraded":"2023-12-01T01:52:06.007780+0000","last_fullsized":"2023-12-01T01:52:06.007780+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:22.076125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990996+0000","last_change":"2023-12-01T01:52:05.990996+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990996+0000","last_undegraded":"2023-12-01T01:52:05.990996+0000","last_fullsized":"2023-12-01T01:52:05.990996+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.006417+0000","last_change":"2023-12-01T01:52:06.003105+0000","last_active":"2023-12-01T01:52:06.006417+0000","last_peered":"2023-12-01T01:52:06.006417+0000","last_clean":"2023-12-01T01:52:06.006417+0000","last_became_active":"2023-12-01T01:52:06.001893+0000","last_became_peered":"2023-12-01T01:52:06.001893+0000","last_unstale":"2023-12-01T01:52:06.006417+0000","last_undegraded":"2023-12-01T01:52:06.006417+0000","last_fullsized":"2023-12-01T01:52:06.006417+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994654+0000","last_change":"2023-12-01T01:52:05.994654+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994654+0000","last_undegraded":"2023-12-01T01:52:05.994654+0000","last_fullsized":"2023-12-01T01:52:05.994654+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:05.997657+0000","last_change":"2023-12-01T01:52:05.990357+0000","last_active":"2023-12-01T01:52:05.997657+0000","last_peered":"2023-12-01T01:52:05.997657+0000","last_clean":"2023-12-01T01:52:05.997657+0000","last_became_active":"2023-12-01T01:52:05.988196+0000","last_became_peered":"2023-12-01T01:52:05.988196+0000","last_unstale":"2023-12-01T01:52:05.997657+0000","last_undegraded":"2023-12-01T01:52:05.997657+0000","last_fullsized":"2023-12-01T01:52:05.997657+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:53.636284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995954+0000","last_change":"2023-12-01T01:52:05.995954+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995954+0000","last_undegraded":"2023-12-01T01:52:05.995954+0000","last_fullsized":"2023-12-01T01:52:05.995954+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:54.842423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.002568+0000","last_change":"2023-12-01T01:52:05.990673+0000","last_active":"2023-12-01T01:52:06.002568+0000","last_peered":"2023-12-01T01:52:06.002568+0000","last_clean":"2023-12-01T01:52:06.002568+0000","last_became_active":"2023-12-01T01:52:05.989026+0000","last_became_peered":"2023-12-01T01:52:05.989026+0000","last_unstale":"2023-12-01T01:52:06.002568+0000","last_undegraded":"2023-12-01T01:52:06.002568+0000","last_fullsized":"2023-12-01T01:52:06.002568+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:51:43.565839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.979875+0000","last_change":"2023-12-01T01:52:05.979875+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.979875+0000","last_undegraded":"2023-12-01T01:52:05.979875+0000","last_fullsized":"2023-12-01T01:52:05.979875+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008997+0000","last_change":"2023-12-01T01:52:06.003576+0000","last_active":"2023-12-01T01:52:06.008997+0000","last_peered":"2023-12-01T01:52:06.008997+0000","last_clean":"2023-12-01T01:52:06.008997+0000","last_became_active":"2023-12-01T01:52:06.003254+0000","last_became_peered":"2023-12-01T01:52:06.003254+0000","last_unstale":"2023-12-01T01:52:06.008997+0000","last_undegraded":"2023-12-01T01:52:06.008997+0000","last_fullsized":"2023-12-01T01:52:06.008997+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:10.387281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995686+0000","last_change":"2023-12-01T01:52:05.995686+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995686+0000","last_undegraded":"2023-12-01T01:52:05.995686+0000","last_fullsized":"2023-12-01T01:52:05.995686+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:31.031923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.008408+0000","last_change":"2023-12-01T01:52:06.003659+0000","last_active":"2023-12-01T01:52:06.008408+0000","last_peered":"2023-12-01T01:52:06.008408+0000","last_clean":"2023-12-01T01:52:06.008408+0000","last_became_active":"2023-12-01T01:52:06.003228+0000","last_became_peered":"2023-12-01T01:52:06.003228+0000","last_unstale":"2023-12-01T01:52:06.008408+0000","last_undegraded":"2023-12-01T01:52:06.008408+0000","last_fullsized":"2023-12-01T01:52:06.008408+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:14.620804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.994032+0000","last_change":"2023-12-01T01:52:05.994032+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.994032+0000","last_undegraded":"2023-12-01T01:52:05.994032+0000","last_fullsized":"2023-12-01T01:52:05.994032+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:35:33.172589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007657+0000","last_change":"2023-12-01T01:52:06.000889+0000","last_active":"2023-12-01T01:52:06.007657+0000","last_peered":"2023-12-01T01:52:06.007657+0000","last_clean":"2023-12-01T01:52:06.007657+0000","last_became_active":"2023-12-01T01:52:06.000644+0000","last_became_peered":"2023-12-01T01:52:06.000644+0000","last_unstale":"2023-12-01T01:52:06.007657+0000","last_undegraded":"2023-12-01T01:52:06.007657+0000","last_fullsized":"2023-12-01T01:52:06.007657+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:19:11.688232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.989769+0000","last_change":"2023-12-01T01:52:05.989769+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.989769+0000","last_undegraded":"2023-12-01T01:52:05.989769+0000","last_fullsized":"2023-12-01T01:52:05.989769+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009039+0000","last_change":"2023-12-01T01:52:06.006527+0000","last_active":"2023-12-01T01:52:06.009039+0000","last_peered":"2023-12-01T01:52:06.009039+0000","last_clean":"2023-12-01T01:52:06.009039+0000","last_became_active":"2023-12-01T01:52:06.005261+0000","last_became_peered":"2023-12-01T01:52:06.005261+0000","last_unstale":"2023-12-01T01:52:06.009039+0000","last_undegraded":"2023-12-01T01:52:06.009039+0000","last_fullsized":"2023-12-01T01:52:06.009039+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:51:05.076663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.992926+0000","last_change":"2023-12-01T01:52:05.992926+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.992926+0000","last_undegraded":"2023-12-01T01:52:05.992926+0000","last_fullsized":"2023-12-01T01:52:05.992926+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:33:59.577488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:05.999607+0000","last_change":"2023-12-01T01:52:05.997953+0000","last_active":"2023-12-01T01:52:05.999607+0000","last_peered":"2023-12-01T01:52:05.999607+0000","last_clean":"2023-12-01T01:52:05.999607+0000","last_became_active":"2023-12-01T01:52:05.997697+0000","last_became_peered":"2023-12-01T01:52:05.997697+0000","last_unstale":"2023-12-01T01:52:05.999607+0000","last_undegraded":"2023-12-01T01:52:05.999607+0000","last_fullsized":"2023-12-01T01:52:05.999607+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:41:32.705632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990662+0000","last_change":"2023-12-01T01:52:05.990662+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990662+0000","last_undegraded":"2023-12-01T01:52:05.990662+0000","last_fullsized":"2023-12-01T01:52:05.990662+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:35:55.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:05.999154+0000","last_change":"2023-12-01T01:52:05.996908+0000","last_active":"2023-12-01T01:52:05.999154+0000","last_peered":"2023-12-01T01:52:05.999154+0000","last_clean":"2023-12-01T01:52:05.999154+0000","last_became_active":"2023-12-01T01:52:05.995875+0000","last_became_peered":"2023-12-01T01:52:05.995875+0000","last_unstale":"2023-12-01T01:52:05.999154+0000","last_undegraded":"2023-12-01T01:52:05.999154+0000","last_fullsized":"2023-12-01T01:52:05.999154+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:37.605903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.993946+0000","last_change":"2023-12-01T01:52:05.993946+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.993946+0000","last_undegraded":"2023-12-01T01:52:05.993946+0000","last_fullsized":"2023-12-01T01:52:05.993946+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:49:44.833749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.009144+0000","last_change":"2023-12-01T01:52:06.003362+0000","last_active":"2023-12-01T01:52:06.009144+0000","last_peered":"2023-12-01T01:52:06.009144+0000","last_clean":"2023-12-01T01:52:06.009144+0000","last_became_active":"2023-12-01T01:52:06.003048+0000","last_became_peered":"2023-12-01T01:52:06.003048+0000","last_unstale":"2023-12-01T01:52:06.009144+0000","last_undegraded":"2023-12-01T01:52:06.009144+0000","last_fullsized":"2023-12-01T01:52:06.009144+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:48.873308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.995418+0000","last_change":"2023-12-01T01:52:05.995418+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.995418+0000","last_undegraded":"2023-12-01T01:52:05.995418+0000","last_fullsized":"2023-12-01T01:52:05.995418+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.002548+0000","last_change":"2023-12-01T01:52:05.990037+0000","last_active":"2023-12-01T01:52:06.002548+0000","last_peered":"2023-12-01T01:52:06.002548+0000","last_clean":"2023-12-01T01:52:06.002548+0000","last_became_active":"2023-12-01T01:52:05.988261+0000","last_became_peered":"2023-12-01T01:52:05.988261+0000","last_unstale":"2023-12-01T01:52:06.002548+0000","last_undegraded":"2023-12-01T01:52:06.002548+0000","last_fullsized":"2023-12-01T01:52:06.002548+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:43.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.990406+0000","last_change":"2023-12-01T01:52:05.990406+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.990406+0000","last_undegraded":"2023-12-01T01:52:05.990406+0000","last_fullsized":"2023-12-01T01:52:05.990406+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:14:44.417471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.004673+0000","last_change":"2023-12-01T01:52:05.998968+0000","last_active":"2023-12-01T01:52:06.004673+0000","last_peered":"2023-12-01T01:52:06.004673+0000","last_clean":"2023-12-01T01:52:06.004673+0000","last_became_active":"2023-12-01T01:52:05.998647+0000","last_became_peered":"2023-12-01T01:52:05.998647+0000","last_unstale":"2023-12-01T01:52:06.004673+0000","last_undegraded":"2023-12-01T01:52:06.004673+0000","last_fullsized":"2023-12-01T01:52:06.004673+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:22.076125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2023-12-01T01:52:05.991444+0000","last_change":"2023-12-01T01:52:05.991444+0000","last_active":"2023-12-01T01:52:05.964277+0000","last_peered":"2023-12-01T01:52:05.964277+0000","last_clean":"2023-12-01T01:52:05.964277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:05.991444+0000","last_undegraded":"2023-12-01T01:52:05.991444+0000","last_fullsized":"2023-12-01T01:52:05.991444+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:12.910138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.007411+0000","last_change":"2023-12-01T01:52:06.000732+0000","last_active":"2023-12-01T01:52:06.007411+0000","last_peered":"2023-12-01T01:52:06.007411+0000","last_clean":"2023-12-01T01:52:06.007411+0000","last_became_active":"2023-12-01T01:52:06.000124+0000","last_became_peered":"2023-12-01T01:52:06.000124+0000","last_unstale":"2023-12-01T01:52:06.007411+0000","last_undegraded":"2023-12-01T01:52:06.007411+0000","last_fullsized":"2023-12-01T01:52:06.007411+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:39.158199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"85'172","reported_seq":430,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-12-01T01:52:04.991801+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:52:04.991801+0000","last_peered":"2023-12-01T01:52:04.991801+0000","last_clean":"2023-12-01T01:52:04.991801+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:52:04.991801+0000","last_undegraded":"2023-12-01T01:52:04.991801+0000","last_fullsized":"2023-12-01T01:52:04.991801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1132,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1159168,"data_stored":549101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0389999999999999,"15min":1.0640000000000001},"min":{"1min":0.65200000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2869999999999999,"5min":1.5449999999999999,"15min":1.8089999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.05,"15min":1.0489999999999999},"min":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.64000000000000001},"max":{"1min":1.2969999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.2010000000000001,"15min":1.2130000000000001},"min":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.351,"5min":1.8100000000000001,"15min":1.8420000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2410000000000001,"15min":1.252},"min":{"1min":0.78900000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.629,"5min":1.744,"15min":2.0870000000000002},"last":0.86199999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.127,"15min":1.121},"min":{"1min":0.73299999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.607,"5min":1.728,"15min":1.728},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.175,"5min":1.208,"15min":1.2230000000000001},"min":{"1min":0.89700000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5489999999999999,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2230000000000001,"15min":1.2310000000000001},"min":{"1min":0.75,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.3979999999999999,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.337,"5min":1.3440000000000001,"15min":1.347},"min":{"1min":1.123,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6259999999999999,"5min":2.1880000000000002,"15min":2.7050000000000001},"last":1.0740000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3160000000000001,"15min":1.296},"min":{"1min":1.036,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5840000000000001,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4550000000000001,"15min":1.4139999999999999},"min":{"1min":0.96399999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7210000000000001,"5min":2.0939999999999999,"15min":2.601},"last":1.2030000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3169999999999999,"15min":1.2689999999999999},"min":{"1min":0.84899999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.4500000000000002},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.532,"15min":1.5209999999999999},"min":{"1min":1.147,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.802,"5min":2.2930000000000001,"15min":2.8069999999999999},"last":1.2470000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4119999999999999,"15min":1.383},"min":{"1min":1.0129999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.294,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.397},"min":{"1min":1.115,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7110000000000001,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.1080000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.091,"15min":1.103},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.55,"5min":1.635,"15min":1.79},"last":1.21},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1399999999999999,"15min":1.127},"min":{"1min":0.77200000000000002,"5min":0.64400000000000002,"15min":0.61799999999999999},"max":{"1min":1.327,"5min":2.5569999999999999,"15min":2.73},"last":1.264}]},{"osd":1,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1639999999999999,"15min":1.1559999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.47,"5min":1.8620000000000001,"15min":2.2170000000000001},"last":0.88},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1950000000000001,"15min":1.2050000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.494,"5min":1.6259999999999999,"15min":2.0920000000000001},"last":1.327}]},{"osd":2,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.214,"15min":1.1659999999999999},"min":{"1min":0.72299999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4510000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.5},{"interface":"front","average":{"1min":1.131,"5min":1.2450000000000001,"15min":1.24},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5549999999999999,"5min":2.137,"15min":2.137},"last":1.6100000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2869999999999999,"15min":1.2789999999999999},"min":{"1min":0.91600000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4339999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.647},{"interface":"front","average":{"1min":1.365,"5min":1.4139999999999999,"15min":1.403},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.81299999999999994},"max":{"1min":1.5569999999999999,"5min":1.742,"15min":2.4020000000000001},"last":1.5840000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.298,"15min":1.28},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6830000000000001,"5min":2.226,"15min":2.3399999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.3460000000000001,"15min":1.347},"min":{"1min":0.93600000000000005,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.6060000000000001,"5min":1.931,"15min":2.6760000000000002},"last":1.46}]},{"osd":5,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4330000000000001,"15min":1.4099999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6779999999999999,"5min":2.1899999999999999,"15min":2.5539999999999998},"last":1.401},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.498,"15min":1.4750000000000001},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":0.748},"max":{"1min":1.786,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.6910000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.5389999999999999,"15min":1.514},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.89500000000000002},"max":{"1min":1.651,"5min":2.762,"15min":2.762},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.5960000000000001,"15min":1.5740000000000001},"min":{"1min":1.234,"5min":1.155,"15min":0.78700000000000003},"max":{"1min":1.7050000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.7569999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1712,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1753088,"data_stored":1139469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.1060000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.498,"5min":1.651,"15min":3.3980000000000001},"last":1.224},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.127,"15min":1.141},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.60099999999999998},"max":{"1min":1.5720000000000001,"5min":1.6479999999999999,"15min":2.153},"last":1.446}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.153,"15min":1.137},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.7010000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.73599999999999999,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.0109999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4039999999999999,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.6080000000000001,"5min":1.621,"15min":2.0630000000000002},"last":1.9239999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.286},"min":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.76900000000000002},"max":{"1min":1.5309999999999999,"5min":1.694,"15min":1.833},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3120000000000001,"15min":1.296},"min":{"1min":0.98099999999999998,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":1.6779999999999999,"5min":1.7110000000000001,"15min":2.1440000000000001},"last":2.0569999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.238,"15min":1.2390000000000001},"min":{"1min":0.86399999999999999,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.6779999999999999,"5min":1.853,"15min":2.5129999999999999},"last":1.421},{"interface":"front","average":{"1min":1.282,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.84999999999999998,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.6319999999999999,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.671}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2989999999999999,"15min":1.329},"min":{"1min":0.88500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":2.2970000000000002},"last":1.7889999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":1.125,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":1.962,"5min":1.962,"15min":3.23},"last":1.327}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4530000000000001,"15min":1.48},"min":{"1min":0.92400000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2,"5min":2,"15min":3.012},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4970000000000001,"15min":1.524},"min":{"1min":1.2689999999999999,"5min":0.873,"15min":0.873},"max":{"1min":2.073,"5min":2.073,"15min":2.8420000000000001},"last":2.1699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36264,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335576,"statfs":{"total":96636764160,"available":96599629824,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.119,"15min":1.1040000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.53,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.734,"5min":1.827,"15min":3.8069999999999999},"last":1.7130000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2350000000000001,"15min":1.2090000000000001},"min":{"1min":0.84599999999999997,"5min":0.81999999999999995,"15min":0.65900000000000003},"max":{"1min":1.528,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.127},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.131},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.671,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.2250000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2110000000000001,"15min":1.196},"min":{"1min":0.83999999999999997,"5min":0.78200000000000003,"15min":0.77600000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.3140000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.871,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.415}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.234},"min":{"1min":0.85399999999999998,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.8220000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3200000000000001,"15min":1.319},"min":{"1min":0.753,"5min":0.753,"15min":0.70399999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.3010000000000002},"last":1.288}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.96599999999999997,"5min":0.79200000000000004,"15min":0.61899999999999999},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.1549999999999998},"last":1.018},{"interface":"front","average":{"1min":1.248,"5min":1.2829999999999999,"15min":1.292},"min":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.5189999999999999,"5min":1.8029999999999999,"15min":2.2029999999999998},"last":1.3720000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.94999999999999996,"5min":0.84799999999999998,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.5029999999999999,"15min":1.4730000000000001},"min":{"1min":1.1000000000000001,"5min":0.96699999999999997,"15min":0.63200000000000001},"max":{"1min":1.7949999999999999,"5min":2.4990000000000001,"15min":2.54},"last":1.823}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.458,"15min":1.464},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.60599999999999998},"max":{"1min":1.6850000000000001,"5min":1.972,"15min":3.2789999999999999},"last":1.863},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5720000000000001,"15min":1.5580000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.04,"15min":2.157},"last":1.9490000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81599999999999995,"15min":0.84799999999999998},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.307},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.583},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97599999999999998,"15min":0.996},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.3160000000000001,"5min":1.476,"15min":1.9770000000000001},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.052,"15min":1.0660000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.38900000000000001},"max":{"1min":1.423,"5min":1.571,"15min":2.0779999999999998},"last":1.002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.92200000000000004,"15min":0.94999999999999996},"min":{"1min":0.504,"5min":0.48199999999999998,"15min":0.40600000000000003},"max":{"1min":1.385,"5min":1.587,"15min":1.6519999999999999},"last":1.1060000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99399999999999999,"15min":1.0469999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.436},"max":{"1min":1.4810000000000001,"5min":1.512,"15min":1.766},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.167,"15min":1.1850000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.48399999999999999},"max":{"1min":1.522,"5min":1.647,"15min":1.661},"last":0.92700000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.212,"15min":1.2450000000000001},"min":{"1min":0.76900000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.4550000000000001,"5min":1.5820000000000001,"15min":2.9350000000000001},"last":1.135},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2150000000000001,"15min":1.24},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.599,"5min":1.637,"15min":2.2389999999999999},"last":1.8480000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.24,"15min":1.29},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.63500000000000001},"max":{"1min":1.7669999999999999,"5min":1.806,"15min":3.476},"last":1.71},{"interface":"front","average":{"1min":1.244,"5min":1.266,"15min":1.3109999999999999},"min":{"1min":0.92100000000000004,"5min":0.82099999999999995,"15min":0.64200000000000002},"max":{"1min":1.681,"5min":1.9039999999999999,"15min":3.181},"last":1.048}]},{"osd":6,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.363,"15min":1.377},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.68300000000000005},"max":{"1min":1.6040000000000001,"5min":1.8580000000000001,"15min":2.532},"last":1.9410000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3919999999999999,"15min":1.4259999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.872},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":3.0249999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.389,"15min":1.421},"min":{"1min":1.0600000000000001,"5min":0.999,"15min":0.81499999999999995},"max":{"1min":1.998,"5min":1.998,"15min":2.1619999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.415,"15min":1.4610000000000001},"min":{"1min":0.91200000000000003,"5min":0.88300000000000001,"15min":0.83099999999999996},"max":{"1min":1.6879999999999999,"5min":1.8779999999999999,"15min":2.4590000000000001},"last":2.0259999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27100000000000002},"max":{"1min":1.0860000000000001,"5min":1.339,"15min":1.738},"last":1.286},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.026,"15min":1.01},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.375},"max":{"1min":1.4159999999999999,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96499999999999997,"15min":0.98999999999999999},"min":{"1min":0.54700000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.1779999999999999,"5min":1.45,"15min":1.7130000000000001},"last":1.202},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0720000000000001,"15min":1.0840000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.504},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.0249999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97499999999999998,"15min":1.0029999999999999},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.377,"5min":1.625,"15min":1.883},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.39600000000000002},"max":{"1min":1.415,"5min":1.613,"15min":2.6080000000000001},"last":1.3999999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.331,"15min":1.3180000000000001},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.75},"max":{"1min":1.7230000000000001,"5min":1.9950000000000001,"15min":2.2829999999999999},"last":1.155},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.246,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.964},"last":1.01}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3009999999999999,"15min":1.319},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.647,"5min":1.804,"15min":2.5779999999999998},"last":1.456},{"interface":"front","average":{"1min":1.375,"5min":1.387,"15min":1.375},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.901,"5min":2.048,"15min":3.0099999999999998},"last":1.1040000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.3200000000000001},"min":{"1min":0.84299999999999997,"5min":0.83399999999999996,"15min":0.77500000000000002},"max":{"1min":1.6439999999999999,"5min":1.78,"15min":2.2799999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.766,"15min":2.5089999999999999},"last":1.5149999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.456,"15min":1.448},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.79400000000000004},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.448},"last":1.327},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7649999999999999,"5min":1.802,"15min":2.5049999999999999},"last":1.244}]}]},{"osd":1,"up_from":13,"seq":55834575041,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1740800,"data_stored":1133202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85199999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.318},"max":{"1min":1.341,"5min":1.5049999999999999,"15min":3.056},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.052,"5min":0.97999999999999998,"15min":1.016},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":3.4129999999999998},"last":1.081}]},{"osd":2,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.88600000000000001,"15min":0.89600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.36399999999999999},"max":{"1min":1.1919999999999999,"5min":1.54,"15min":3.3660000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0680000000000001,"15min":1.095},"min":{"1min":0.71199999999999997,"5min":0.47899999999999998,"15min":0.39000000000000001},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":3.4980000000000002},"last":1.1930000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.194,"15min":1.216},"min":{"1min":0.55400000000000005,"5min":0.51900000000000002,"15min":0.44},"max":{"1min":1.827,"5min":1.827,"15min":3.464},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.048,"15min":1.0529999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39700000000000002},"max":{"1min":1.657,"5min":1.6699999999999999,"15min":3.0960000000000001},"last":0.98899999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.175,"15min":1.1870000000000001},"min":{"1min":0.871,"5min":0.745,"15min":0.63800000000000001},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":3.7810000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3160000000000001,"15min":1.3440000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6559999999999999,"5min":1.8089999999999999,"15min":4.7270000000000003},"last":1.254}]},{"osd":5,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2829999999999999,"15min":1.2809999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6839999999999999,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.448},{"interface":"front","average":{"1min":1.3,"5min":1.268,"15min":1.296},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":1.825,"15min":3.6669999999999998},"last":1.1259999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3089999999999999,"15min":1.3360000000000001},"min":{"1min":0.79400000000000004,"5min":0.71099999999999997,"15min":0.68400000000000005},"max":{"1min":1.7490000000000001,"5min":1.7809999999999999,"15min":3.3300000000000001},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.3460000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5389999999999999,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.1619999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3979999999999999,"15min":1.4419999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.774,"5min":1.8939999999999999,"15min":4.6749999999999998},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.4159999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8440000000000001,"5min":1.992,"15min":3.5379999999999998},"last":1.522}]}]},{"osd":0,"up_from":13,"seq":55834575043,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1146880,"data_stored":542834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.60699999999999998,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.3520000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.92600000000000005,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4390000000000001,"5min":1.5580000000000001,"15min":1.6910000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98499999999999999,"15min":1.006},"min":{"1min":0.67600000000000005,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.4690000000000001,"5min":2.157,"15min":2.157},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.113,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.84299999999999997,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.395,"5min":1.728,"15min":2.8159999999999998},"last":1.0820000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.58499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.954},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.143,"15min":1.169},"min":{"1min":0.54900000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7070000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1279999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.3089999999999999},"min":{"1min":1.05,"5min":0.78500000000000003,"15min":0.65700000000000003},"max":{"1min":1.5289999999999999,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.196},{"interface":"front","average":{"1min":1.22,"5min":1.214,"15min":1.2629999999999999},"min":{"1min":0.83299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.391}]},{"osd":5,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.337,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.71899999999999997},"max":{"1min":1.839,"5min":2.351,"15min":2.351},"last":1.232},{"interface":"front","average":{"1min":1.276,"5min":1.306,"15min":1.349},"min":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6299999999999999,"5min":2.2909999999999999,"15min":2.7050000000000001},"last":1.3100000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73099999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.157},"last":1.341},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.399},"min":{"1min":1.0129999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.292}]},{"osd":7,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":1.127,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.706,"5min":1.798,"15min":2.8999999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.361,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.276}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:52:11.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-12-01T01:52:11.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.997898+0000', 'last_change': '2023-12-01T01:52:05.997898+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.997898+0000', 'last_undegraded': '2023-12-01T01:52:05.997898+0000', 'last_fullsized': '2023-12-01T01:52:05.997898+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:12:04.610539+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-12-01T01:52:11.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995940+0000', 'last_change': '2023-12-01T01:52:05.995940+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995940+0000', 'last_undegraded': '2023-12-01T01:52:05.995940+0000', 'last_fullsized': '2023-12-01T01:52:05.995940+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:34:53.713026+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:11.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-12-01T01:52:11.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995242+0000', 'last_change': '2023-12-01T01:52:05.995242+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995242+0000', 'last_undegraded': '2023-12-01T01:52:05.995242+0000', 'last_fullsized': '2023-12-01T01:52:05.995242+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:11.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-12-01T01:52:11.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.993787+0000', 'last_change': '2023-12-01T01:52:05.993787+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.993787+0000', 'last_undegraded': '2023-12-01T01:52:05.993787+0000', 'last_fullsized': '2023-12-01T01:52:05.993787+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:24:54.640923+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.997994+0000', 'last_change': '2023-12-01T01:52:05.997994+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.997994+0000', 'last_undegraded': '2023-12-01T01:52:05.997994+0000', 'last_fullsized': '2023-12-01T01:52:05.997994+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:14:11.568997+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.983916+0000', 'last_change': '2023-12-01T01:52:05.983916+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.983916+0000', 'last_undegraded': '2023-12-01T01:52:05.983916+0000', 'last_fullsized': '2023-12-01T01:52:05.983916+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995670+0000', 'last_change': '2023-12-01T01:52:05.995670+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995670+0000', 'last_undegraded': '2023-12-01T01:52:05.995670+0000', 'last_fullsized': '2023-12-01T01:52:05.995670+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.997463+0000', 'last_change': '2023-12-01T01:52:05.997463+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.997463+0000', 'last_undegraded': '2023-12-01T01:52:05.997463+0000', 'last_fullsized': '2023-12-01T01:52:05.997463+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:19:57.884685+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:06.001892+0000', 'last_change': '2023-12-01T01:52:06.001892+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:06.001892+0000', 'last_undegraded': '2023-12-01T01:52:06.001892+0000', 'last_fullsized': '2023-12-01T01:52:06.001892+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:41:44.957677+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-12-01T01:52:11.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995914+0000', 'last_change': '2023-12-01T01:52:05.995914+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995914+0000', 'last_undegraded': '2023-12-01T01:52:05.995914+0000', 'last_fullsized': '2023-12-01T01:52:05.995914+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.996024+0000', 'last_change': '2023-12-01T01:52:05.996024+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.996024+0000', 'last_undegraded': '2023-12-01T01:52:05.996024+0000', 'last_fullsized': '2023-12-01T01:52:05.996024+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:58:13.647534+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.980909+0000', 'last_change': '2023-12-01T01:52:05.980909+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.980909+0000', 'last_undegraded': '2023-12-01T01:52:05.980909+0000', 'last_fullsized': '2023-12-01T01:52:05.980909+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.992885+0000', 'last_change': '2023-12-01T01:52:05.992885+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.992885+0000', 'last_undegraded': '2023-12-01T01:52:05.992885+0000', 'last_fullsized': '2023-12-01T01:52:05.992885+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:03:27.876714+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.978309+0000', 'last_change': '2023-12-01T01:52:05.978309+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.978309+0000', 'last_undegraded': '2023-12-01T01:52:05.978309+0000', 'last_fullsized': '2023-12-01T01:52:05.978309+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:16:22.669615+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.982112+0000', 'last_change': '2023-12-01T01:52:05.982112+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.982112+0000', 'last_undegraded': '2023-12-01T01:52:05.982112+0000', 'last_fullsized': '2023-12-01T01:52:05.982112+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-12-01T01:52:11.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.996130+0000', 'last_change': '2023-12-01T01:52:05.996130+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.996130+0000', 'last_undegraded': '2023-12-01T01:52:05.996130+0000', 'last_fullsized': '2023-12-01T01:52:05.996130+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:57:19.431492+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.997994+0000', 'last_change': '2023-12-01T01:52:05.997994+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.997994+0000', 'last_undegraded': '2023-12-01T01:52:05.997994+0000', 'last_fullsized': '2023-12-01T01:52:05.997994+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:37:40.773157+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.999346+0000', 'last_change': '2023-12-01T01:52:05.999346+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.999346+0000', 'last_undegraded': '2023-12-01T01:52:05.999346+0000', 'last_fullsized': '2023-12-01T01:52:05.999346+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:43:04.482263+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.992107+0000', 'last_change': '2023-12-01T01:52:05.992107+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.992107+0000', 'last_undegraded': '2023-12-01T01:52:05.992107+0000', 'last_fullsized': '2023-12-01T01:52:05.992107+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:10:23.538863+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.997895+0000', 'last_change': '2023-12-01T01:52:05.997895+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.997895+0000', 'last_undegraded': '2023-12-01T01:52:05.997895+0000', 'last_fullsized': '2023-12-01T01:52:05.997895+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:07:56.031035+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995149+0000', 'last_change': '2023-12-01T01:52:05.995149+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995149+0000', 'last_undegraded': '2023-12-01T01:52:05.995149+0000', 'last_fullsized': '2023-12-01T01:52:05.995149+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:23:11.391920+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:11.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.981128+0000', 'last_change': '2023-12-01T01:52:05.981128+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.981128+0000', 'last_undegraded': '2023-12-01T01:52:05.981128+0000', 'last_fullsized': '2023-12-01T01:52:05.981128+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995097+0000', 'last_change': '2023-12-01T01:52:05.995097+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995097+0000', 'last_undegraded': '2023-12-01T01:52:05.995097+0000', 'last_fullsized': '2023-12-01T01:52:05.995097+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995030+0000', 'last_change': '2023-12-01T01:52:05.995030+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995030+0000', 'last_undegraded': '2023-12-01T01:52:05.995030+0000', 'last_fullsized': '2023-12-01T01:52:05.995030+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:51:42.770614+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.998660+0000', 'last_change': '2023-12-01T01:52:05.998660+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.998660+0000', 'last_undegraded': '2023-12-01T01:52:05.998660+0000', 'last_fullsized': '2023-12-01T01:52:05.998660+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:34:48.109363+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.998980+0000', 'last_change': '2023-12-01T01:52:05.998980+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.998980+0000', 'last_undegraded': '2023-12-01T01:52:05.998980+0000', 'last_fullsized': '2023-12-01T01:52:05.998980+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:17:23.123384+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-12-01T01:52:11.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.999166+0000', 'last_change': '2023-12-01T01:52:05.999166+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.999166+0000', 'last_undegraded': '2023-12-01T01:52:05.999166+0000', 'last_fullsized': '2023-12-01T01:52:05.999166+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:50:15.604117+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.993773+0000', 'last_change': '2023-12-01T01:52:05.993773+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.993773+0000', 'last_undegraded': '2023-12-01T01:52:05.993773+0000', 'last_fullsized': '2023-12-01T01:52:05.993773+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:46:24.016226+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.992182+0000', 'last_change': '2023-12-01T01:52:05.992182+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.992182+0000', 'last_undegraded': '2023-12-01T01:52:05.992182+0000', 'last_fullsized': '2023-12-01T01:52:05.992182+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.993145+0000', 'last_change': '2023-12-01T01:52:05.993145+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.993145+0000', 'last_undegraded': '2023-12-01T01:52:05.993145+0000', 'last_fullsized': '2023-12-01T01:52:05.993145+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:02:26.262966+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.993080+0000', 'last_change': '2023-12-01T01:52:05.993080+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.993080+0000', 'last_undegraded': '2023-12-01T01:52:05.993080+0000', 'last_fullsized': '2023-12-01T01:52:05.993080+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:38:19.366076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.990723+0000', 'last_change': '2023-12-01T01:52:05.990723+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.990723+0000', 'last_undegraded': '2023-12-01T01:52:05.990723+0000', 'last_fullsized': '2023-12-01T01:52:05.990723+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:11.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-12-01T01:52:11.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.992259+0000', 'last_change': '2023-12-01T01:52:05.992259+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.992259+0000', 'last_undegraded': '2023-12-01T01:52:05.992259+0000', 'last_fullsized': '2023-12-01T01:52:05.992259+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:42:00.920100+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-12-01T01:52:11.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:06.000063+0000', 'last_change': '2023-12-01T01:52:06.000063+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:06.000063+0000', 'last_undegraded': '2023-12-01T01:52:06.000063+0000', 'last_fullsized': '2023-12-01T01:52:06.000063+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:28:10.371917+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:11.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-12-01T01:52:11.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.981512+0000', 'last_change': '2023-12-01T01:52:05.981512+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.981512+0000', 'last_undegraded': '2023-12-01T01:52:05.981512+0000', 'last_fullsized': '2023-12-01T01:52:05.981512+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:11.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-12-01T01:52:11.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995865+0000', 'last_change': '2023-12-01T01:52:05.995865+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995865+0000', 'last_undegraded': '2023-12-01T01:52:05.995865+0000', 'last_fullsized': '2023-12-01T01:52:05.995865+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:41:42.937116+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:11.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-12-01T01:52:11.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.992585+0000', 'last_change': '2023-12-01T01:52:05.992585+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.992585+0000', 'last_undegraded': '2023-12-01T01:52:05.992585+0000', 'last_fullsized': '2023-12-01T01:52:05.992585+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:41:28.966506+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.980164+0000', 'last_change': '2023-12-01T01:52:05.980164+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.980164+0000', 'last_undegraded': '2023-12-01T01:52:05.980164+0000', 'last_fullsized': '2023-12-01T01:52:05.980164+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:06.000047+0000', 'last_change': '2023-12-01T01:52:06.000047+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:06.000047+0000', 'last_undegraded': '2023-12-01T01:52:06.000047+0000', 'last_fullsized': '2023-12-01T01:52:06.000047+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.980933+0000', 'last_change': '2023-12-01T01:52:05.980933+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.980933+0000', 'last_undegraded': '2023-12-01T01:52:05.980933+0000', 'last_fullsized': '2023-12-01T01:52:05.980933+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995453+0000', 'last_change': '2023-12-01T01:52:05.995453+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995453+0000', 'last_undegraded': '2023-12-01T01:52:05.995453+0000', 'last_fullsized': '2023-12-01T01:52:05.995453+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:06:21.919614+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.991180+0000', 'last_change': '2023-12-01T01:52:05.991180+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.991180+0000', 'last_undegraded': '2023-12-01T01:52:05.991180+0000', 'last_fullsized': '2023-12-01T01:52:05.991180+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:00:34.778062+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:11.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.994125+0000', 'last_change': '2023-12-01T01:52:05.994125+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.994125+0000', 'last_undegraded': '2023-12-01T01:52:05.994125+0000', 'last_fullsized': '2023-12-01T01:52:05.994125+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:17:44.214762+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.996433+0000', 'last_change': '2023-12-01T01:52:05.996433+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.996433+0000', 'last_undegraded': '2023-12-01T01:52:05.996433+0000', 'last_fullsized': '2023-12-01T01:52:05.996433+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:38:40.656152+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.979785+0000', 'last_change': '2023-12-01T01:52:05.979785+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.979785+0000', 'last_undegraded': '2023-12-01T01:52:05.979785+0000', 'last_fullsized': '2023-12-01T01:52:05.979785+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.993720+0000', 'last_change': '2023-12-01T01:52:05.993720+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.993720+0000', 'last_undegraded': '2023-12-01T01:52:05.993720+0000', 'last_fullsized': '2023-12-01T01:52:05.993720+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.994026+0000', 'last_change': '2023-12-01T01:52:05.994026+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.994026+0000', 'last_undegraded': '2023-12-01T01:52:05.994026+0000', 'last_fullsized': '2023-12-01T01:52:05.994026+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:18:58.385658+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-12-01T01:52:11.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.996979+0000', 'last_change': '2023-12-01T01:52:05.996979+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.996979+0000', 'last_undegraded': '2023-12-01T01:52:05.996979+0000', 'last_fullsized': '2023-12-01T01:52:05.996979+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:37:55.202441+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.989549+0000', 'last_change': '2023-12-01T01:52:05.989549+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.989549+0000', 'last_undegraded': '2023-12-01T01:52:05.989549+0000', 'last_fullsized': '2023-12-01T01:52:05.989549+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.989532+0000', 'last_change': '2023-12-01T01:52:05.989532+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.989532+0000', 'last_undegraded': '2023-12-01T01:52:05.989532+0000', 'last_fullsized': '2023-12-01T01:52:05.989532+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.992916+0000', 'last_change': '2023-12-01T01:52:05.992916+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.992916+0000', 'last_undegraded': '2023-12-01T01:52:05.992916+0000', 'last_fullsized': '2023-12-01T01:52:05.992916+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.990996+0000', 'last_change': '2023-12-01T01:52:05.990996+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.990996+0000', 'last_undegraded': '2023-12-01T01:52:05.990996+0000', 'last_fullsized': '2023-12-01T01:52:05.990996+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.994654+0000', 'last_change': '2023-12-01T01:52:05.994654+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.994654+0000', 'last_undegraded': '2023-12-01T01:52:05.994654+0000', 'last_fullsized': '2023-12-01T01:52:05.994654+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-12-01T01:52:11.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995954+0000', 'last_change': '2023-12-01T01:52:05.995954+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995954+0000', 'last_undegraded': '2023-12-01T01:52:05.995954+0000', 'last_fullsized': '2023-12-01T01:52:05.995954+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:44:54.842423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:11.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-12-01T01:52:11.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.979875+0000', 'last_change': '2023-12-01T01:52:05.979875+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.979875+0000', 'last_undegraded': '2023-12-01T01:52:05.979875+0000', 'last_fullsized': '2023-12-01T01:52:05.979875+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:52:11.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-12-01T01:52:11.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995686+0000', 'last_change': '2023-12-01T01:52:05.995686+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995686+0000', 'last_undegraded': '2023-12-01T01:52:05.995686+0000', 'last_fullsized': '2023-12-01T01:52:05.995686+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:40:31.031923+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:11.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-12-01T01:52:11.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.994032+0000', 'last_change': '2023-12-01T01:52:05.994032+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.994032+0000', 'last_undegraded': '2023-12-01T01:52:05.994032+0000', 'last_fullsized': '2023-12-01T01:52:05.994032+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:35:33.172589+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:11.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-12-01T01:52:11.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.989769+0000', 'last_change': '2023-12-01T01:52:05.989769+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.989769+0000', 'last_undegraded': '2023-12-01T01:52:05.989769+0000', 'last_fullsized': '2023-12-01T01:52:05.989769+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:11.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.992926+0000', 'last_change': '2023-12-01T01:52:05.992926+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.992926+0000', 'last_undegraded': '2023-12-01T01:52:05.992926+0000', 'last_fullsized': '2023-12-01T01:52:05.992926+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:33:59.577488+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.990662+0000', 'last_change': '2023-12-01T01:52:05.990662+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.990662+0000', 'last_undegraded': '2023-12-01T01:52:05.990662+0000', 'last_fullsized': '2023-12-01T01:52:05.990662+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:35:55.572050+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.993946+0000', 'last_change': '2023-12-01T01:52:05.993946+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.993946+0000', 'last_undegraded': '2023-12-01T01:52:05.993946+0000', 'last_fullsized': '2023-12-01T01:52:05.993946+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:49:44.833749+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.995418+0000', 'last_change': '2023-12-01T01:52:05.995418+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.995418+0000', 'last_undegraded': '2023-12-01T01:52:05.995418+0000', 'last_fullsized': '2023-12-01T01:52:05.995418+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.990406+0000', 'last_change': '2023-12-01T01:52:05.990406+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.990406+0000', 'last_undegraded': '2023-12-01T01:52:05.990406+0000', 'last_fullsized': '2023-12-01T01:52:05.990406+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:14:44.417471+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-12-01T01:52:11.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:05.991444+0000', 'last_change': '2023-12-01T01:52:05.991444+0000', 'last_active': '2023-12-01T01:52:05.964277+0000', 'last_peered': '2023-12-01T01:52:05.964277+0000', 'last_clean': '2023-12-01T01:52:05.964277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:05.991444+0000', 'last_undegraded': '2023-12-01T01:52:05.991444+0000', 'last_fullsized': '2023-12-01T01:52:05.991444+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:05.964277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:49:12.910138+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:11.898 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-01T01:52:12.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.135+0000 7fab1db2e640 1 Processor -- start 2023-12-01T01:52:12.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.135+0000 7fab1db2e640 1 -- start start 2023-12-01T01:52:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.136+0000 7fab1db2e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fab180adac0 0x7fab180cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.136+0000 7fab1db2e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab180c8ed0 0x7fab180cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.137+0000 7fab1db2e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab180ce150 con 0x7fab180cced0 2023-12-01T01:52:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.137+0000 7fab1db2e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab180cf3a0 con 0x7fab180adac0 2023-12-01T01:52:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.137+0000 7fab1db2e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180ce5e0 con 0x7fab180c8ed0 2023-12-01T01:52:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.137+0000 7fab17fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fab180cced0 0x7fab18161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35644/0 (socket says 172.21.15.115:35644) 2023-12-01T01:52:12.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.137+0000 7fab17fff640 1 -- 172.21.15.115:0/468415344 learned_addr learned my addr 172.21.15.115:0/468415344 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.137+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077006341 0 0) 0x7fab180ce150 con 0x7fab180cced0 2023-12-01T01:52:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.137+0000 7fab177fe640 1 --2- 172.21.15.115:0/468415344 >> v2:172.21.15.142:3300/0 conn(0x7fab180adac0 0x7fab180cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.137+0000 7fab0ffff640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab180c8ed0 0x7fab180cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.137+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab04003520 con 0x7fab180cced0 2023-12-01T01:52:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 762693311 0 0) 0x7fab04003520 con 0x7fab180cced0 2023-12-01T01:52:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab180c8ed0 msgr2=0x7fab180cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab180c8ed0 0x7fab180cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 >> v2:172.21.15.142:3300/0 conn(0x7fab180adac0 msgr2=0x7fab180cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 --2- 172.21.15.115:0/468415344 >> v2:172.21.15.142:3300/0 conn(0x7fab180adac0 0x7fab180cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1806d620 con 0x7fab180cced0 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fab100019f0 con 0x7fab180cced0 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 >> v1:172.21.15.115:6789/0 conn(0x7fab180cced0 legacy=0x7fab18161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab040039e0 0x7fab04003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 0x7fab04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab04005180 0x7fab0400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180cf3a0 con 0x7fab040047c0 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180ce5e0 con 0x7fab04005180 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0400bfc0 con 0x7fab040039e0 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab0ffff640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab180c8ed0 0x7fab180cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab0ffff640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 0x7fab04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.138+0000 7fab17fff640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab04005180 0x7fab0400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.139+0000 7fab177fe640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab040039e0 0x7fab04003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.139+0000 7fab0ffff640 1 -- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab040039e0 msgr2=0x7fab04003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.139+0000 7fab0ffff640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab040039e0 0x7fab04003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.139+0000 7fab0ffff640 1 -- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab04005180 msgr2=0x7fab0400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.139+0000 7fab0ffff640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab04005180 0x7fab0400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.139+0000 7fab0ffff640 1 -- 172.21.15.115:0/468415344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab04003520 con 0x7fab040047c0 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.140+0000 7fab0ffff640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 0x7fab04004c40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fab00006fd0 tx=0x7fab040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c186287f97844a5a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.140+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00032600 con 0x7fab040047c0 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.140+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0002e050 con 0x7fab040047c0 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.140+0000 7fab16ffd640 1 -- 172.21.15.115:0/468415344 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab000329c0 con 0x7fab040047c0 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.141+0000 7fab1db2e640 1 -- 172.21.15.115:0/468415344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 msgr2=0x7fab04004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.141+0000 7fab1db2e640 1 --2- 172.21.15.115:0/468415344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 0x7fab04004c40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fab00006fd0 tx=0x7fab040057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.141+0000 7fab17fff640 1 -- 172.21.15.115:0/468415344 reap_dead start 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.141+0000 7fab1db2e640 1 -- 172.21.15.115:0/468415344 shutdown_connections 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.141+0000 7fab1db2e640 1 -- 172.21.15.115:0/468415344 >> 172.21.15.115:0/468415344 conn(0x7fab180ce880 msgr2=0x7fab180cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.141+0000 7fab1db2e640 1 -- 172.21.15.115:0/468415344 shutdown_connections 2023-12-01T01:52:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.141+0000 7fab1db2e640 1 -- 172.21.15.115:0/468415344 wait complete. 2023-12-01T01:52:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.143+0000 7fab1db2e640 1 Processor -- start 2023-12-01T01:52:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.143+0000 7fab1db2e640 1 -- start start 2023-12-01T01:52:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab1db2e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 0x7fab1815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab1db2e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab180c5100 0x7fab180c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab1db2e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab180c72f0 0x7fab18070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab1db2e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab181e7d20 con 0x7fab040047c0 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab1db2e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180c6250 con 0x7fab180c72f0 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab1db2e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180c6530 con 0x7fab180c5100 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab177fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 0x7fab1815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab177fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 0x7fab1815ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46996/0 (socket says 172.21.15.115:46996) 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab177fe640 1 -- 172.21.15.115:0/2365127114 learned_addr learned my addr 172.21.15.115:0/2365127114 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab177fe640 1 -- 172.21.15.115:0/2365127114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab180c5100 msgr2=0x7fab180c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab177fe640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab180c5100 0x7fab180c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab177fe640 1 -- 172.21.15.115:0/2365127114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab180c72f0 msgr2=0x7fab18070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab177fe640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab180c72f0 0x7fab18070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.144+0000 7fab177fe640 1 -- 172.21.15.115:0/2365127114 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180c6810 con 0x7fab040047c0 2023-12-01T01:52:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.145+0000 7fab177fe640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 0x7fab1815ffc0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fab080116d0 tx=0x7fab08011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.145+0000 7fab14ff9640 1 -- 172.21.15.115:0/2365127114 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800f9c0 con 0x7fab040047c0 2023-12-01T01:52:12.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.145+0000 7fab14ff9640 1 -- 172.21.15.115:0/2365127114 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0800fb20 con 0x7fab040047c0 2023-12-01T01:52:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.145+0000 7fab1db2e640 1 -- 172.21.15.115:0/2365127114 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab180c6af0 con 0x7fab040047c0 2023-12-01T01:52:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.145+0000 7fab14ff9640 1 -- 172.21.15.115:0/2365127114 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0800fe90 con 0x7fab040047c0 2023-12-01T01:52:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.145+0000 7fab1db2e640 1 -- 172.21.15.115:0/2365127114 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab180c5e90 con 0x7fab040047c0 2023-12-01T01:52:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.146+0000 7fab1db2e640 1 -- 172.21.15.115:0/2365127114 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1804e890 con 0x7fab040047c0 2023-12-01T01:52:12.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.147+0000 7fab14ff9640 1 -- 172.21.15.115:0/2365127114 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fab08015460 con 0x7fab040047c0 2023-12-01T01:52:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.148+0000 7fab14ff9640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faaf806a550 0x7faaf806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.148+0000 7fab14ff9640 1 -- 172.21.15.115:0/2365127114 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab080893c0 con 0x7fab040047c0 2023-12-01T01:52:12.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.148+0000 7fab0ffff640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faaf806a550 0x7faaf806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.149+0000 7fab0ffff640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faaf806a550 0x7faaf806ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fab00035000 tx=0x7fab0002f000 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:12.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.153+0000 7fab14ff9640 1 -- 172.21.15.115:0/2365127114 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fab08054890 con 0x7fab040047c0 2023-12-01T01:52:12.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.407+0000 7fab1db2e640 1 -- 172.21.15.115:0/2365127114 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fab180afe90 con 0x7fab040047c0 2023-12-01T01:52:12.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.408+0000 7fab14ff9640 1 -- 172.21.15.115:0/2365127114 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3229 (secure 0 0 0) 0x7fab08057eb0 con 0x7fab040047c0 2023-12-01T01:52:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:12.409 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":987,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1701394558,"num_in_osds":8,"osd_in_since":1701394549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":277315584,"bytes_avail":772816797696,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":40,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"b","status":"up:active","gid":6957}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-12-01T01:51:54.026233+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 -- 172.21.15.115:0/2365127114 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faaf806a550 msgr2=0x7faaf806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faaf806a550 0x7faaf806ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fab00035000 tx=0x7fab0002f000 comp rx=0 tx=0).stop 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 -- 172.21.15.115:0/2365127114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 msgr2=0x7fab1815ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 0x7fab1815ffc0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fab080116d0 tx=0x7fab08011b90 comp rx=0 tx=0).stop 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 -- 172.21.15.115:0/2365127114 shutdown_connections 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faaf806a550 0x7faaf806ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab180c5100 0x7fab180c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab180c72f0 0x7fab18070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 --2- 172.21.15.115:0/2365127114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab040047c0 0x7fab1815ffc0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 -- 172.21.15.115:0/2365127114 >> 172.21.15.115:0/2365127114 conn(0x7fab180ce880 msgr2=0x7fab180cb260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 -- 172.21.15.115:0/2365127114 shutdown_connections 2023-12-01T01:52:12.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.410+0000 7fab1db2e640 1 -- 172.21.15.115:0/2365127114 wait complete. 2023-12-01T01:52:12.428 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-01T01:52:12.428 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:52:12.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.617+0000 7f072ebbb640 1 Processor -- start 2023-12-01T01:52:12.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.617+0000 7f072ebbb640 1 -- start start 2023-12-01T01:52:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.618+0000 7f072ebbb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f072806b6f0 0x7f0728070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.618+0000 7f072ebbb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072806f8b0 con 0x7f07280c9ef0 2023-12-01T01:52:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.618+0000 7f072ebbb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07280adcb0 con 0x7f0728159dd0 2023-12-01T01:52:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.618+0000 7f072ebbb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07280ad9d0 con 0x7f072806b6f0 2023-12-01T01:52:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.619+0000 7f0727fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f07280c9ef0 0x7f072806fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35652/0 (socket says 172.21.15.115:35652) 2023-12-01T01:52:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.619+0000 7f0727fff640 1 -- 172.21.15.115:0/1960023446 learned_addr learned my addr 172.21.15.115:0/1960023446 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.619+0000 7f072d158640 1 --2- 172.21.15.115:0/1960023446 >> v2:172.21.15.142:3300/0 conn(0x7f072806b6f0 0x7f0728070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.619+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014129069 0 0) 0x7f072806f8b0 con 0x7f07280c9ef0 2023-12-01T01:52:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.619+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0718003520 con 0x7f07280c9ef0 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3139212012 0 0) 0x7f0718003520 con 0x7f07280c9ef0 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 >> v2:172.21.15.142:3300/0 conn(0x7f072806b6f0 msgr2=0x7f0728070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 --2- 172.21.15.115:0/1960023446 >> v2:172.21.15.142:3300/0 conn(0x7f072806b6f0 0x7f0728070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 >> v1:172.21.15.142:6789/0 conn(0x7f0728159dd0 legacy=0x7f07280c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07280c9000 con 0x7f07280c9ef0 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0714001670 con 0x7f07280c9ef0 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 >> v1:172.21.15.115:6789/0 conn(0x7f07280c9ef0 legacy=0x7f072806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 --2- 172.21.15.115:0/1960023446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07180039e0 0x7f0718003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 --2- 172.21.15.115:0/1960023446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07180047c0 0x7f0718004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 --2- 172.21.15.115:0/1960023446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f071800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07280ad9d0 con 0x7f0718005280 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071800c1c0 con 0x7f07180047c0 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071800e590 con 0x7f07180039e0 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f072d158640 1 --2- 172.21.15.115:0/1960023446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f071800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f072d158640 1 -- 172.21.15.115:0/1960023446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07180039e0 msgr2=0x7f0718003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f072d158640 1 --2- 172.21.15.115:0/1960023446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07180039e0 0x7f0718003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f072d158640 1 -- 172.21.15.115:0/1960023446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07180047c0 msgr2=0x7f0718004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f072d158640 1 --2- 172.21.15.115:0/1960023446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07180047c0 0x7f0718004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.620+0000 7f072d158640 1 -- 172.21.15.115:0/1960023446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718003520 con 0x7f0718005280 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.621+0000 7f072d158640 1 -- 172.21.15.115:0/1960023446 reap_dead start 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.621+0000 7f072d158640 1 --2- 172.21.15.115:0/1960023446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f071800b980 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0720009fc0 tx=0x7f07180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe500739f6d89bc3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.622+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072002cc60 con 0x7f0718005280 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.622+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072000f040 con 0x7f0718005280 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.622+0000 7f07277fe640 1 -- 172.21.15.115:0/1960023446 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720003990 con 0x7f0718005280 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.622+0000 7f072ebbb640 1 -- 172.21.15.115:0/1960023446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 msgr2=0x7f071800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.622+0000 7f072ebbb640 1 --2- 172.21.15.115:0/1960023446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f071800b980 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0720009fc0 tx=0x7f07180057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.622+0000 7f072ebbb640 1 -- 172.21.15.115:0/1960023446 shutdown_connections 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.622+0000 7f072ebbb640 1 --2- 172.21.15.115:0/1960023446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f071800b980 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.622+0000 7f072ebbb640 1 -- 172.21.15.115:0/1960023446 >> 172.21.15.115:0/1960023446 conn(0x7f07280ce880 msgr2=0x7f07280cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.622+0000 7f072ebbb640 1 -- 172.21.15.115:0/1960023446 shutdown_connections 2023-12-01T01:52:12.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.622+0000 7f072ebbb640 1 -- 172.21.15.115:0/1960023446 wait complete. 2023-12-01T01:52:12.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.624+0000 7f072ebbb640 1 Processor -- start 2023-12-01T01:52:12.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.624+0000 7f072ebbb640 1 -- start start 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.625+0000 7f072ebbb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f072815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.625+0000 7f072ebbb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f072815c0c0 0x7f07280c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.625+0000 7f072ebbb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07280c2820 0x7f07280ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.625+0000 7f072ebbb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07280b25b0 con 0x7f0718005280 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.625+0000 7f072ebbb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072815c8a0 con 0x7f07280c2820 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.625+0000 7f072ebbb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07280b1f30 con 0x7f072815c0c0 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.625+0000 7f072c957640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f072815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.626+0000 7f072c957640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f072815cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47020/0 (socket says 172.21.15.115:47020) 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.626+0000 7f072c957640 1 -- 172.21.15.115:0/1865295901 learned_addr learned my addr 172.21.15.115:0/1865295901 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.626+0000 7f072d158640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07280c2820 0x7f07280ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.626+0000 7f072c957640 1 -- 172.21.15.115:0/1865295901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f072815c0c0 msgr2=0x7f07280c4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.626+0000 7f072c957640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f072815c0c0 0x7f07280c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.626+0000 7f072c957640 1 -- 172.21.15.115:0/1865295901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07280c2820 msgr2=0x7f07280ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.626+0000 7f072c957640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07280c2820 0x7f07280ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.626+0000 7f072c957640 1 -- 172.21.15.115:0/1865295901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072815c5c0 con 0x7f0718005280 2023-12-01T01:52:12.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.626+0000 7f072c957640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f072815cfc0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f071c00c450 tx=0x7f071c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:12.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.626+0000 7f07257fa640 1 -- 172.21.15.115:0/1865295901 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c0052d0 con 0x7f0718005280 2023-12-01T01:52:12.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.627+0000 7f07257fa640 1 -- 172.21.15.115:0/1865295901 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c005430 con 0x7f0718005280 2023-12-01T01:52:12.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.627+0000 7f07257fa640 1 -- 172.21.15.115:0/1865295901 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c012870 con 0x7f0718005280 2023-12-01T01:52:12.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.627+0000 7f072ebbb640 1 -- 172.21.15.115:0/1865295901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07280b21d0 con 0x7f0718005280 2023-12-01T01:52:12.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.627+0000 7f072ebbb640 1 -- 172.21.15.115:0/1865295901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07280c4e50 con 0x7f0718005280 2023-12-01T01:52:12.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.628+0000 7f072ebbb640 1 -- 172.21.15.115:0/1865295901 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07280b2820 con 0x7f0718005280 2023-12-01T01:52:12.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.629+0000 7f07257fa640 1 -- 172.21.15.115:0/1865295901 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f071c0129d0 con 0x7f0718005280 2023-12-01T01:52:12.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.630+0000 7f07257fa640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f070c06a550 0x7f070c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:12.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.630+0000 7f07257fa640 1 -- 172.21.15.115:0/1865295901 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f071c088580 con 0x7f0718005280 2023-12-01T01:52:12.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.630+0000 7f0727fff640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f070c06a550 0x7f070c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:12.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.631+0000 7f0727fff640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f070c06a550 0x7f070c06ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f07140078d0 tx=0x7f07140014d0 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:12.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.634+0000 7f07257fa640 1 -- 172.21.15.115:0/1865295901 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f071c053a50 con 0x7f0718005280 2023-12-01T01:52:12.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.817+0000 7f072ebbb640 1 -- 172.21.15.115:0/1865295901 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07280b1dc0 con 0x7f070c06a550 2023-12-01T01:52:12.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.825+0000 7f07257fa640 1 -- 172.21.15.115:0/1865295901 <== mgr.6723 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286853 (secure 0 0 0) 0x7f07280b1dc0 con 0x7f070c06a550 2023-12-01T01:52:12.827 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:12.832 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:52:12.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 -- 172.21.15.115:0/1865295901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f070c06a550 msgr2=0x7f070c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f070c06a550 0x7f070c06ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f07140078d0 tx=0x7f07140014d0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 -- 172.21.15.115:0/1865295901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 msgr2=0x7f072815cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:12.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f072815cfc0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f071c00c450 tx=0x7f071c00c910 comp rx=0 tx=0).stop 2023-12-01T01:52:12.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 -- 172.21.15.115:0/1865295901 shutdown_connections 2023-12-01T01:52:12.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f070c06a550 0x7f070c06ca10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f072815c0c0 0x7f07280c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07280c2820 0x7f07280ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 --2- 172.21.15.115:0/1865295901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0718005280 0x7f072815cfc0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:12.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 -- 172.21.15.115:0/1865295901 >> 172.21.15.115:0/1865295901 conn(0x7f07280ce880 msgr2=0x7f07280704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:12.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 -- 172.21.15.115:0/1865295901 shutdown_connections 2023-12-01T01:52:12.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:12.831+0000 7f072ebbb640 1 -- 172.21.15.115:0/1865295901 wait complete. 2023-12-01T01:52:12.849 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T01:52:12.031247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":303,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":293288,"kb_used_data":10472,"kb_used_omap":0,"kb_used_meta":282752,"kb_avail":754681432,"statfs":{"total":773094113280,"available":772793786368,"internally_reserved":0,"allocated":10723328,"data_stored":5682868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":289538048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.058770"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.500421+0000","last_change":"2023-12-01T01:52:06.489980+0000","last_active":"2023-12-01T01:52:06.500421+0000","last_peered":"2023-12-01T01:52:06.500421+0000","last_clean":"2023-12-01T01:52:06.500421+0000","last_became_active":"2023-12-01T01:52:06.489658+0000","last_became_peered":"2023-12-01T01:52:06.489658+0000","last_unstale":"2023-12-01T01:52:06.500421+0000","last_undegraded":"2023-12-01T01:52:06.500421+0000","last_fullsized":"2023-12-01T01:52:06.500421+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:12:04.610539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.488816+0000","last_change":"2023-12-01T01:52:06.483199+0000","last_active":"2023-12-01T01:52:06.488816+0000","last_peered":"2023-12-01T01:52:06.488816+0000","last_clean":"2023-12-01T01:52:06.488816+0000","last_became_active":"2023-12-01T01:52:06.482954+0000","last_became_peered":"2023-12-01T01:52:06.482954+0000","last_unstale":"2023-12-01T01:52:06.488816+0000","last_undegraded":"2023-12-01T01:52:06.488816+0000","last_fullsized":"2023-12-01T01:52:06.488816+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:53.713026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485604+0000","last_change":"2023-12-01T01:52:06.477868+0000","last_active":"2023-12-01T01:52:06.485604+0000","last_peered":"2023-12-01T01:52:06.485604+0000","last_clean":"2023-12-01T01:52:06.485604+0000","last_became_active":"2023-12-01T01:52:06.477587+0000","last_became_peered":"2023-12-01T01:52:06.477587+0000","last_unstale":"2023-12-01T01:52:06.485604+0000","last_undegraded":"2023-12-01T01:52:06.485604+0000","last_fullsized":"2023-12-01T01:52:06.485604+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.489524+0000","last_change":"2023-12-01T01:52:06.477629+0000","last_active":"2023-12-01T01:52:06.489524+0000","last_peered":"2023-12-01T01:52:06.489524+0000","last_clean":"2023-12-01T01:52:06.489524+0000","last_became_active":"2023-12-01T01:52:06.477344+0000","last_became_peered":"2023-12-01T01:52:06.477344+0000","last_unstale":"2023-12-01T01:52:06.489524+0000","last_undegraded":"2023-12-01T01:52:06.489524+0000","last_fullsized":"2023-12-01T01:52:06.489524+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:54.640923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485248+0000","last_change":"2023-12-01T01:52:06.483451+0000","last_active":"2023-12-01T01:52:06.485248+0000","last_peered":"2023-12-01T01:52:06.485248+0000","last_clean":"2023-12-01T01:52:06.485248+0000","last_became_active":"2023-12-01T01:52:06.483171+0000","last_became_peered":"2023-12-01T01:52:06.483171+0000","last_unstale":"2023-12-01T01:52:06.485248+0000","last_undegraded":"2023-12-01T01:52:06.485248+0000","last_fullsized":"2023-12-01T01:52:06.485248+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:11.568997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.501575+0000","last_change":"2023-12-01T01:52:06.499765+0000","last_active":"2023-12-01T01:52:06.501575+0000","last_peered":"2023-12-01T01:52:06.501575+0000","last_clean":"2023-12-01T01:52:06.501575+0000","last_became_active":"2023-12-01T01:52:06.499532+0000","last_became_peered":"2023-12-01T01:52:06.499532+0000","last_unstale":"2023-12-01T01:52:06.501575+0000","last_undegraded":"2023-12-01T01:52:06.501575+0000","last_fullsized":"2023-12-01T01:52:06.501575+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.479667+0000","last_change":"2023-12-01T01:52:06.475695+0000","last_active":"2023-12-01T01:52:06.479667+0000","last_peered":"2023-12-01T01:52:06.479667+0000","last_clean":"2023-12-01T01:52:06.479667+0000","last_became_active":"2023-12-01T01:52:06.475397+0000","last_became_peered":"2023-12-01T01:52:06.475397+0000","last_unstale":"2023-12-01T01:52:06.479667+0000","last_undegraded":"2023-12-01T01:52:06.479667+0000","last_fullsized":"2023-12-01T01:52:06.479667+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.499034+0000","last_change":"2023-12-01T01:52:06.495425+0000","last_active":"2023-12-01T01:52:06.499034+0000","last_peered":"2023-12-01T01:52:06.499034+0000","last_clean":"2023-12-01T01:52:06.499034+0000","last_became_active":"2023-12-01T01:52:06.495171+0000","last_became_peered":"2023-12-01T01:52:06.495171+0000","last_unstale":"2023-12-01T01:52:06.499034+0000","last_undegraded":"2023-12-01T01:52:06.499034+0000","last_fullsized":"2023-12-01T01:52:06.499034+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:19:57.884685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492130+0000","last_change":"2023-12-01T01:52:06.487903+0000","last_active":"2023-12-01T01:52:06.492130+0000","last_peered":"2023-12-01T01:52:06.492130+0000","last_clean":"2023-12-01T01:52:06.492130+0000","last_became_active":"2023-12-01T01:52:06.487482+0000","last_became_peered":"2023-12-01T01:52:06.487482+0000","last_unstale":"2023-12-01T01:52:06.492130+0000","last_undegraded":"2023-12-01T01:52:06.492130+0000","last_fullsized":"2023-12-01T01:52:06.492130+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:44.957677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485517+0000","last_change":"2023-12-01T01:52:06.482106+0000","last_active":"2023-12-01T01:52:06.485517+0000","last_peered":"2023-12-01T01:52:06.485517+0000","last_clean":"2023-12-01T01:52:06.485517+0000","last_became_active":"2023-12-01T01:52:06.480984+0000","last_became_peered":"2023-12-01T01:52:06.480984+0000","last_unstale":"2023-12-01T01:52:06.485517+0000","last_undegraded":"2023-12-01T01:52:06.485517+0000","last_fullsized":"2023-12-01T01:52:06.485517+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.499869+0000","last_change":"2023-12-01T01:52:06.496643+0000","last_active":"2023-12-01T01:52:06.499869+0000","last_peered":"2023-12-01T01:52:06.499869+0000","last_clean":"2023-12-01T01:52:06.499869+0000","last_became_active":"2023-12-01T01:52:06.496359+0000","last_became_peered":"2023-12-01T01:52:06.496359+0000","last_unstale":"2023-12-01T01:52:06.499869+0000","last_undegraded":"2023-12-01T01:52:06.499869+0000","last_fullsized":"2023-12-01T01:52:06.499869+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:58:13.647534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.490540+0000","last_change":"2023-12-01T01:52:06.485229+0000","last_active":"2023-12-01T01:52:06.490540+0000","last_peered":"2023-12-01T01:52:06.490540+0000","last_clean":"2023-12-01T01:52:06.490540+0000","last_became_active":"2023-12-01T01:52:06.484939+0000","last_became_peered":"2023-12-01T01:52:06.484939+0000","last_unstale":"2023-12-01T01:52:06.490540+0000","last_undegraded":"2023-12-01T01:52:06.490540+0000","last_fullsized":"2023-12-01T01:52:06.490540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491177+0000","last_change":"2023-12-01T01:52:06.480235+0000","last_active":"2023-12-01T01:52:06.491177+0000","last_peered":"2023-12-01T01:52:06.491177+0000","last_clean":"2023-12-01T01:52:06.491177+0000","last_became_active":"2023-12-01T01:52:06.478382+0000","last_became_peered":"2023-12-01T01:52:06.478382+0000","last_unstale":"2023-12-01T01:52:06.491177+0000","last_undegraded":"2023-12-01T01:52:06.491177+0000","last_fullsized":"2023-12-01T01:52:06.491177+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:03:27.876714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.487300+0000","last_change":"2023-12-01T01:52:06.485215+0000","last_active":"2023-12-01T01:52:06.487300+0000","last_peered":"2023-12-01T01:52:06.487300+0000","last_clean":"2023-12-01T01:52:06.487300+0000","last_became_active":"2023-12-01T01:52:06.484900+0000","last_became_peered":"2023-12-01T01:52:06.484900+0000","last_unstale":"2023-12-01T01:52:06.487300+0000","last_undegraded":"2023-12-01T01:52:06.487300+0000","last_fullsized":"2023-12-01T01:52:06.487300+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:22.669615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.489281+0000","last_change":"2023-12-01T01:52:06.487149+0000","last_active":"2023-12-01T01:52:06.489281+0000","last_peered":"2023-12-01T01:52:06.489281+0000","last_clean":"2023-12-01T01:52:06.489281+0000","last_became_active":"2023-12-01T01:52:06.486307+0000","last_became_peered":"2023-12-01T01:52:06.486307+0000","last_unstale":"2023-12-01T01:52:06.489281+0000","last_undegraded":"2023-12-01T01:52:06.489281+0000","last_fullsized":"2023-12-01T01:52:06.489281+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485892+0000","last_change":"2023-12-01T01:52:06.479432+0000","last_active":"2023-12-01T01:52:06.485892+0000","last_peered":"2023-12-01T01:52:06.485892+0000","last_clean":"2023-12-01T01:52:06.485892+0000","last_became_active":"2023-12-01T01:52:06.479162+0000","last_became_peered":"2023-12-01T01:52:06.479162+0000","last_unstale":"2023-12-01T01:52:06.485892+0000","last_undegraded":"2023-12-01T01:52:06.485892+0000","last_fullsized":"2023-12-01T01:52:06.485892+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:19.431492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.481232+0000","last_change":"2023-12-01T01:52:06.475996+0000","last_active":"2023-12-01T01:52:06.481232+0000","last_peered":"2023-12-01T01:52:06.481232+0000","last_clean":"2023-12-01T01:52:06.481232+0000","last_became_active":"2023-12-01T01:52:06.474002+0000","last_became_peered":"2023-12-01T01:52:06.474002+0000","last_unstale":"2023-12-01T01:52:06.481232+0000","last_undegraded":"2023-12-01T01:52:06.481232+0000","last_fullsized":"2023-12-01T01:52:06.481232+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:40.773157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.483645+0000","last_change":"2023-12-01T01:52:06.478718+0000","last_active":"2023-12-01T01:52:06.483645+0000","last_peered":"2023-12-01T01:52:06.483645+0000","last_clean":"2023-12-01T01:52:06.483645+0000","last_became_active":"2023-12-01T01:52:06.478398+0000","last_became_peered":"2023-12-01T01:52:06.478398+0000","last_unstale":"2023-12-01T01:52:06.483645+0000","last_undegraded":"2023-12-01T01:52:06.483645+0000","last_fullsized":"2023-12-01T01:52:06.483645+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:43:04.482263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.496516+0000","last_change":"2023-12-01T01:52:06.486443+0000","last_active":"2023-12-01T01:52:06.496516+0000","last_peered":"2023-12-01T01:52:06.496516+0000","last_clean":"2023-12-01T01:52:06.496516+0000","last_became_active":"2023-12-01T01:52:06.486194+0000","last_became_peered":"2023-12-01T01:52:06.486194+0000","last_unstale":"2023-12-01T01:52:06.496516+0000","last_undegraded":"2023-12-01T01:52:06.496516+0000","last_fullsized":"2023-12-01T01:52:06.496516+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:23.538863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.486240+0000","last_change":"2023-12-01T01:52:06.484596+0000","last_active":"2023-12-01T01:52:06.486240+0000","last_peered":"2023-12-01T01:52:06.486240+0000","last_clean":"2023-12-01T01:52:06.486240+0000","last_became_active":"2023-12-01T01:52:06.484190+0000","last_became_peered":"2023-12-01T01:52:06.484190+0000","last_unstale":"2023-12-01T01:52:06.486240+0000","last_undegraded":"2023-12-01T01:52:06.486240+0000","last_fullsized":"2023-12-01T01:52:06.486240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:07:56.031035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.478306+0000","last_change":"2023-12-01T01:52:06.475250+0000","last_active":"2023-12-01T01:52:06.478306+0000","last_peered":"2023-12-01T01:52:06.478306+0000","last_clean":"2023-12-01T01:52:06.478306+0000","last_became_active":"2023-12-01T01:52:06.474957+0000","last_became_peered":"2023-12-01T01:52:06.474957+0000","last_unstale":"2023-12-01T01:52:06.478306+0000","last_undegraded":"2023-12-01T01:52:06.478306+0000","last_fullsized":"2023-12-01T01:52:06.478306+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:11.391920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491672+0000","last_change":"2023-12-01T01:52:06.487363+0000","last_active":"2023-12-01T01:52:06.491672+0000","last_peered":"2023-12-01T01:52:06.491672+0000","last_clean":"2023-12-01T01:52:06.491672+0000","last_became_active":"2023-12-01T01:52:06.487091+0000","last_became_peered":"2023-12-01T01:52:06.487091+0000","last_unstale":"2023-12-01T01:52:06.491672+0000","last_undegraded":"2023-12-01T01:52:06.491672+0000","last_fullsized":"2023-12-01T01:52:06.491672+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.496723+0000","last_change":"2023-12-01T01:52:06.489054+0000","last_active":"2023-12-01T01:52:06.496723+0000","last_peered":"2023-12-01T01:52:06.496723+0000","last_clean":"2023-12-01T01:52:06.496723+0000","last_became_active":"2023-12-01T01:52:06.487822+0000","last_became_peered":"2023-12-01T01:52:06.487822+0000","last_unstale":"2023-12-01T01:52:06.496723+0000","last_undegraded":"2023-12-01T01:52:06.496723+0000","last_fullsized":"2023-12-01T01:52:06.496723+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.499013+0000","last_change":"2023-12-01T01:52:06.482346+0000","last_active":"2023-12-01T01:52:06.499013+0000","last_peered":"2023-12-01T01:52:06.499013+0000","last_clean":"2023-12-01T01:52:06.499013+0000","last_became_active":"2023-12-01T01:52:06.482104+0000","last_became_peered":"2023-12-01T01:52:06.482104+0000","last_unstale":"2023-12-01T01:52:06.499013+0000","last_undegraded":"2023-12-01T01:52:06.499013+0000","last_fullsized":"2023-12-01T01:52:06.499013+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:51:42.770614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.496802+0000","last_change":"2023-12-01T01:52:06.491310+0000","last_active":"2023-12-01T01:52:06.496802+0000","last_peered":"2023-12-01T01:52:06.496802+0000","last_clean":"2023-12-01T01:52:06.496802+0000","last_became_active":"2023-12-01T01:52:06.490864+0000","last_became_peered":"2023-12-01T01:52:06.490864+0000","last_unstale":"2023-12-01T01:52:06.496802+0000","last_undegraded":"2023-12-01T01:52:06.496802+0000","last_fullsized":"2023-12-01T01:52:06.496802+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:34:48.109363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497825+0000","last_change":"2023-12-01T01:52:06.481690+0000","last_active":"2023-12-01T01:52:06.497825+0000","last_peered":"2023-12-01T01:52:06.497825+0000","last_clean":"2023-12-01T01:52:06.497825+0000","last_became_active":"2023-12-01T01:52:06.481407+0000","last_became_peered":"2023-12-01T01:52:06.481407+0000","last_unstale":"2023-12-01T01:52:06.497825+0000","last_undegraded":"2023-12-01T01:52:06.497825+0000","last_fullsized":"2023-12-01T01:52:06.497825+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:23.123384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.481684+0000","last_change":"2023-12-01T01:52:06.477974+0000","last_active":"2023-12-01T01:52:06.481684+0000","last_peered":"2023-12-01T01:52:06.481684+0000","last_clean":"2023-12-01T01:52:06.481684+0000","last_became_active":"2023-12-01T01:52:06.477696+0000","last_became_peered":"2023-12-01T01:52:06.477696+0000","last_unstale":"2023-12-01T01:52:06.481684+0000","last_undegraded":"2023-12-01T01:52:06.481684+0000","last_fullsized":"2023-12-01T01:52:06.481684+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:50:15.604117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.489986+0000","last_change":"2023-12-01T01:52:06.484026+0000","last_active":"2023-12-01T01:52:06.489986+0000","last_peered":"2023-12-01T01:52:06.489986+0000","last_clean":"2023-12-01T01:52:06.489986+0000","last_became_active":"2023-12-01T01:52:06.483746+0000","last_became_peered":"2023-12-01T01:52:06.483746+0000","last_unstale":"2023-12-01T01:52:06.489986+0000","last_undegraded":"2023-12-01T01:52:06.489986+0000","last_fullsized":"2023-12-01T01:52:06.489986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:24.016226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491196+0000","last_change":"2023-12-01T01:52:06.480783+0000","last_active":"2023-12-01T01:52:06.491196+0000","last_peered":"2023-12-01T01:52:06.491196+0000","last_clean":"2023-12-01T01:52:06.491196+0000","last_became_active":"2023-12-01T01:52:06.480328+0000","last_became_peered":"2023-12-01T01:52:06.480328+0000","last_unstale":"2023-12-01T01:52:06.491196+0000","last_undegraded":"2023-12-01T01:52:06.491196+0000","last_fullsized":"2023-12-01T01:52:06.491196+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.498229+0000","last_change":"2023-12-01T01:52:06.489459+0000","last_active":"2023-12-01T01:52:06.498229+0000","last_peered":"2023-12-01T01:52:06.498229+0000","last_clean":"2023-12-01T01:52:06.498229+0000","last_became_active":"2023-12-01T01:52:06.489171+0000","last_became_peered":"2023-12-01T01:52:06.489171+0000","last_unstale":"2023-12-01T01:52:06.498229+0000","last_undegraded":"2023-12-01T01:52:06.498229+0000","last_fullsized":"2023-12-01T01:52:06.498229+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:26.262966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492491+0000","last_change":"2023-12-01T01:52:06.485347+0000","last_active":"2023-12-01T01:52:06.492491+0000","last_peered":"2023-12-01T01:52:06.492491+0000","last_clean":"2023-12-01T01:52:06.492491+0000","last_became_active":"2023-12-01T01:52:06.484910+0000","last_became_peered":"2023-12-01T01:52:06.484910+0000","last_unstale":"2023-12-01T01:52:06.492491+0000","last_undegraded":"2023-12-01T01:52:06.492491+0000","last_fullsized":"2023-12-01T01:52:06.492491+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:38:19.366076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.490593+0000","last_change":"2023-12-01T01:52:06.483449+0000","last_active":"2023-12-01T01:52:06.490593+0000","last_peered":"2023-12-01T01:52:06.490593+0000","last_clean":"2023-12-01T01:52:06.490593+0000","last_became_active":"2023-12-01T01:52:06.483198+0000","last_became_peered":"2023-12-01T01:52:06.483198+0000","last_unstale":"2023-12-01T01:52:06.490593+0000","last_undegraded":"2023-12-01T01:52:06.490593+0000","last_fullsized":"2023-12-01T01:52:06.490593+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.500145+0000","last_change":"2023-12-01T01:52:06.498117+0000","last_active":"2023-12-01T01:52:06.500145+0000","last_peered":"2023-12-01T01:52:06.500145+0000","last_clean":"2023-12-01T01:52:06.500145+0000","last_became_active":"2023-12-01T01:52:06.497659+0000","last_became_peered":"2023-12-01T01:52:06.497659+0000","last_unstale":"2023-12-01T01:52:06.500145+0000","last_undegraded":"2023-12-01T01:52:06.500145+0000","last_fullsized":"2023-12-01T01:52:06.500145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:42:00.920100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"96'4","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497130+0000","last_change":"2023-12-01T01:52:05.999712+0000","last_active":"2023-12-01T01:52:06.497130+0000","last_peered":"2023-12-01T01:52:06.497130+0000","last_clean":"2023-12-01T01:52:06.497130+0000","last_became_active":"2023-12-01T01:52:05.999447+0000","last_became_peered":"2023-12-01T01:52:05.999447+0000","last_unstale":"2023-12-01T01:52:06.497130+0000","last_undegraded":"2023-12-01T01:52:06.497130+0000","last_fullsized":"2023-12-01T01:52:06.497130+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:51:43.565839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491769+0000","last_change":"2023-12-01T01:52:06.487733+0000","last_active":"2023-12-01T01:52:06.491769+0000","last_peered":"2023-12-01T01:52:06.491769+0000","last_clean":"2023-12-01T01:52:06.491769+0000","last_became_active":"2023-12-01T01:52:06.487144+0000","last_became_peered":"2023-12-01T01:52:06.487144+0000","last_unstale":"2023-12-01T01:52:06.491769+0000","last_undegraded":"2023-12-01T01:52:06.491769+0000","last_fullsized":"2023-12-01T01:52:06.491769+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:28:10.371917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"96'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.495474+0000","last_change":"2023-12-01T01:52:06.006120+0000","last_active":"2023-12-01T01:52:06.495474+0000","last_peered":"2023-12-01T01:52:06.495474+0000","last_clean":"2023-12-01T01:52:06.495474+0000","last_became_active":"2023-12-01T01:52:06.004701+0000","last_became_peered":"2023-12-01T01:52:06.004701+0000","last_unstale":"2023-12-01T01:52:06.495474+0000","last_undegraded":"2023-12-01T01:52:06.495474+0000","last_fullsized":"2023-12-01T01:52:06.495474+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:01.267746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492294+0000","last_change":"2023-12-01T01:52:06.489871+0000","last_active":"2023-12-01T01:52:06.492294+0000","last_peered":"2023-12-01T01:52:06.492294+0000","last_clean":"2023-12-01T01:52:06.492294+0000","last_became_active":"2023-12-01T01:52:06.489638+0000","last_became_peered":"2023-12-01T01:52:06.489638+0000","last_unstale":"2023-12-01T01:52:06.492294+0000","last_undegraded":"2023-12-01T01:52:06.492294+0000","last_fullsized":"2023-12-01T01:52:06.492294+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.503888+0000","last_change":"2023-12-01T01:52:05.999810+0000","last_active":"2023-12-01T01:52:06.503888+0000","last_peered":"2023-12-01T01:52:06.503888+0000","last_clean":"2023-12-01T01:52:06.503888+0000","last_became_active":"2023-12-01T01:52:05.999536+0000","last_became_peered":"2023-12-01T01:52:05.999536+0000","last_unstale":"2023-12-01T01:52:06.503888+0000","last_undegraded":"2023-12-01T01:52:06.503888+0000","last_fullsized":"2023-12-01T01:52:06.503888+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:38:18.361437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.487055+0000","last_change":"2023-12-01T01:52:06.483935+0000","last_active":"2023-12-01T01:52:06.487055+0000","last_peered":"2023-12-01T01:52:06.487055+0000","last_clean":"2023-12-01T01:52:06.487055+0000","last_became_active":"2023-12-01T01:52:06.482193+0000","last_became_peered":"2023-12-01T01:52:06.482193+0000","last_unstale":"2023-12-01T01:52:06.487055+0000","last_undegraded":"2023-12-01T01:52:06.487055+0000","last_fullsized":"2023-12-01T01:52:06.487055+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:42.937116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497177+0000","last_change":"2023-12-01T01:52:06.004649+0000","last_active":"2023-12-01T01:52:06.497177+0000","last_peered":"2023-12-01T01:52:06.497177+0000","last_clean":"2023-12-01T01:52:06.497177+0000","last_became_active":"2023-12-01T01:52:06.004356+0000","last_became_peered":"2023-12-01T01:52:06.004356+0000","last_unstale":"2023-12-01T01:52:06.497177+0000","last_undegraded":"2023-12-01T01:52:06.497177+0000","last_fullsized":"2023-12-01T01:52:06.497177+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.498445+0000","last_change":"2023-12-01T01:52:06.495979+0000","last_active":"2023-12-01T01:52:06.498445+0000","last_peered":"2023-12-01T01:52:06.498445+0000","last_clean":"2023-12-01T01:52:06.498445+0000","last_became_active":"2023-12-01T01:52:06.494890+0000","last_became_peered":"2023-12-01T01:52:06.494890+0000","last_unstale":"2023-12-01T01:52:06.498445+0000","last_undegraded":"2023-12-01T01:52:06.498445+0000","last_fullsized":"2023-12-01T01:52:06.498445+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:28.966506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"96'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497261+0000","last_change":"2023-12-01T01:52:05.998965+0000","last_active":"2023-12-01T01:52:06.497261+0000","last_peered":"2023-12-01T01:52:06.497261+0000","last_clean":"2023-12-01T01:52:06.497261+0000","last_became_active":"2023-12-01T01:52:05.997814+0000","last_became_peered":"2023-12-01T01:52:05.997814+0000","last_unstale":"2023-12-01T01:52:06.497261+0000","last_undegraded":"2023-12-01T01:52:06.497261+0000","last_fullsized":"2023-12-01T01:52:06.497261+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:06:24.991705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492331+0000","last_change":"2023-12-01T01:52:06.490530+0000","last_active":"2023-12-01T01:52:06.492331+0000","last_peered":"2023-12-01T01:52:06.492331+0000","last_clean":"2023-12-01T01:52:06.492331+0000","last_became_active":"2023-12-01T01:52:06.490291+0000","last_became_peered":"2023-12-01T01:52:06.490291+0000","last_unstale":"2023-12-01T01:52:06.492331+0000","last_undegraded":"2023-12-01T01:52:06.492331+0000","last_fullsized":"2023-12-01T01:52:06.492331+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"96'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.496988+0000","last_change":"2023-12-01T01:52:06.006338+0000","last_active":"2023-12-01T01:52:06.496988+0000","last_peered":"2023-12-01T01:52:06.496988+0000","last_clean":"2023-12-01T01:52:06.496988+0000","last_became_active":"2023-12-01T01:52:06.006077+0000","last_became_peered":"2023-12-01T01:52:06.006077+0000","last_unstale":"2023-12-01T01:52:06.496988+0000","last_undegraded":"2023-12-01T01:52:06.496988+0000","last_fullsized":"2023-12-01T01:52:06.496988+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:07:16.419472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485848+0000","last_change":"2023-12-01T01:52:06.483163+0000","last_active":"2023-12-01T01:52:06.485848+0000","last_peered":"2023-12-01T01:52:06.485848+0000","last_clean":"2023-12-01T01:52:06.485848+0000","last_became_active":"2023-12-01T01:52:06.481477+0000","last_became_peered":"2023-12-01T01:52:06.481477+0000","last_unstale":"2023-12-01T01:52:06.485848+0000","last_undegraded":"2023-12-01T01:52:06.485848+0000","last_fullsized":"2023-12-01T01:52:06.485848+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.477488+0000","last_change":"2023-12-01T01:52:06.001988+0000","last_active":"2023-12-01T01:52:06.477488+0000","last_peered":"2023-12-01T01:52:06.477488+0000","last_clean":"2023-12-01T01:52:06.477488+0000","last_became_active":"2023-12-01T01:52:06.001633+0000","last_became_peered":"2023-12-01T01:52:06.001633+0000","last_unstale":"2023-12-01T01:52:06.477488+0000","last_undegraded":"2023-12-01T01:52:06.477488+0000","last_fullsized":"2023-12-01T01:52:06.477488+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.489262+0000","last_change":"2023-12-01T01:52:06.486910+0000","last_active":"2023-12-01T01:52:06.489262+0000","last_peered":"2023-12-01T01:52:06.489262+0000","last_clean":"2023-12-01T01:52:06.489262+0000","last_became_active":"2023-12-01T01:52:06.485145+0000","last_became_peered":"2023-12-01T01:52:06.485145+0000","last_unstale":"2023-12-01T01:52:06.489262+0000","last_undegraded":"2023-12-01T01:52:06.489262+0000","last_fullsized":"2023-12-01T01:52:06.489262+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.476149+0000","last_change":"2023-12-01T01:52:06.005204+0000","last_active":"2023-12-01T01:52:06.476149+0000","last_peered":"2023-12-01T01:52:06.476149+0000","last_clean":"2023-12-01T01:52:06.476149+0000","last_became_active":"2023-12-01T01:52:06.004890+0000","last_became_peered":"2023-12-01T01:52:06.004890+0000","last_unstale":"2023-12-01T01:52:06.476149+0000","last_undegraded":"2023-12-01T01:52:06.476149+0000","last_fullsized":"2023-12-01T01:52:06.476149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497008+0000","last_change":"2023-12-01T01:52:06.492266+0000","last_active":"2023-12-01T01:52:06.497008+0000","last_peered":"2023-12-01T01:52:06.497008+0000","last_clean":"2023-12-01T01:52:06.497008+0000","last_became_active":"2023-12-01T01:52:06.491993+0000","last_became_peered":"2023-12-01T01:52:06.491993+0000","last_unstale":"2023-12-01T01:52:06.497008+0000","last_undegraded":"2023-12-01T01:52:06.497008+0000","last_fullsized":"2023-12-01T01:52:06.497008+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:06:21.919614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.476234+0000","last_change":"2023-12-01T01:52:06.005744+0000","last_active":"2023-12-01T01:52:06.476234+0000","last_peered":"2023-12-01T01:52:06.476234+0000","last_clean":"2023-12-01T01:52:06.476234+0000","last_became_active":"2023-12-01T01:52:06.005449+0000","last_became_peered":"2023-12-01T01:52:06.005449+0000","last_unstale":"2023-12-01T01:52:06.476234+0000","last_undegraded":"2023-12-01T01:52:06.476234+0000","last_fullsized":"2023-12-01T01:52:06.476234+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:23.011587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.490428+0000","last_change":"2023-12-01T01:52:06.480888+0000","last_active":"2023-12-01T01:52:06.490428+0000","last_peered":"2023-12-01T01:52:06.490428+0000","last_clean":"2023-12-01T01:52:06.490428+0000","last_became_active":"2023-12-01T01:52:06.480586+0000","last_became_peered":"2023-12-01T01:52:06.480586+0000","last_unstale":"2023-12-01T01:52:06.490428+0000","last_undegraded":"2023-12-01T01:52:06.490428+0000","last_fullsized":"2023-12-01T01:52:06.490428+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:00:34.778062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.467850+0000","last_change":"2023-12-01T01:52:06.001972+0000","last_active":"2023-12-01T01:52:06.467850+0000","last_peered":"2023-12-01T01:52:06.467850+0000","last_clean":"2023-12-01T01:52:06.467850+0000","last_became_active":"2023-12-01T01:52:05.996697+0000","last_became_peered":"2023-12-01T01:52:05.996697+0000","last_unstale":"2023-12-01T01:52:06.467850+0000","last_undegraded":"2023-12-01T01:52:06.467850+0000","last_fullsized":"2023-12-01T01:52:06.467850+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.493598+0000","last_change":"2023-12-01T01:52:06.481826+0000","last_active":"2023-12-01T01:52:06.493598+0000","last_peered":"2023-12-01T01:52:06.493598+0000","last_clean":"2023-12-01T01:52:06.493598+0000","last_became_active":"2023-12-01T01:52:06.480848+0000","last_became_peered":"2023-12-01T01:52:06.480848+0000","last_unstale":"2023-12-01T01:52:06.493598+0000","last_undegraded":"2023-12-01T01:52:06.493598+0000","last_fullsized":"2023-12-01T01:52:06.493598+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:17:44.214762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"96'3","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497853+0000","last_change":"2023-12-01T01:52:06.006483+0000","last_active":"2023-12-01T01:52:06.497853+0000","last_peered":"2023-12-01T01:52:06.497853+0000","last_clean":"2023-12-01T01:52:06.497853+0000","last_became_active":"2023-12-01T01:52:06.004318+0000","last_became_peered":"2023-12-01T01:52:06.004318+0000","last_unstale":"2023-12-01T01:52:06.497853+0000","last_undegraded":"2023-12-01T01:52:06.497853+0000","last_fullsized":"2023-12-01T01:52:06.497853+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:48.962691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.481665+0000","last_change":"2023-12-01T01:52:06.477011+0000","last_active":"2023-12-01T01:52:06.481665+0000","last_peered":"2023-12-01T01:52:06.481665+0000","last_clean":"2023-12-01T01:52:06.481665+0000","last_became_active":"2023-12-01T01:52:06.476673+0000","last_became_peered":"2023-12-01T01:52:06.476673+0000","last_unstale":"2023-12-01T01:52:06.481665+0000","last_undegraded":"2023-12-01T01:52:06.481665+0000","last_fullsized":"2023-12-01T01:52:06.481665+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:40.656152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"96'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.499960+0000","last_change":"2023-12-01T01:52:06.002932+0000","last_active":"2023-12-01T01:52:06.499960+0000","last_peered":"2023-12-01T01:52:06.499960+0000","last_clean":"2023-12-01T01:52:06.499960+0000","last_became_active":"2023-12-01T01:52:06.002600+0000","last_became_peered":"2023-12-01T01:52:06.002600+0000","last_unstale":"2023-12-01T01:52:06.499960+0000","last_undegraded":"2023-12-01T01:52:06.499960+0000","last_fullsized":"2023-12-01T01:52:06.499960+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491945+0000","last_change":"2023-12-01T01:52:06.488627+0000","last_active":"2023-12-01T01:52:06.491945+0000","last_peered":"2023-12-01T01:52:06.491945+0000","last_clean":"2023-12-01T01:52:06.491945+0000","last_became_active":"2023-12-01T01:52:06.488099+0000","last_became_peered":"2023-12-01T01:52:06.488099+0000","last_unstale":"2023-12-01T01:52:06.491945+0000","last_undegraded":"2023-12-01T01:52:06.491945+0000","last_fullsized":"2023-12-01T01:52:06.491945+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.498992+0000","last_change":"2023-12-01T01:52:06.007948+0000","last_active":"2023-12-01T01:52:06.498992+0000","last_peered":"2023-12-01T01:52:06.498992+0000","last_clean":"2023-12-01T01:52:06.498992+0000","last_became_active":"2023-12-01T01:52:06.007397+0000","last_became_peered":"2023-12-01T01:52:06.007397+0000","last_unstale":"2023-12-01T01:52:06.498992+0000","last_undegraded":"2023-12-01T01:52:06.498992+0000","last_fullsized":"2023-12-01T01:52:06.498992+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:59:33.535939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.481548+0000","last_change":"2023-12-01T01:52:06.479285+0000","last_active":"2023-12-01T01:52:06.481548+0000","last_peered":"2023-12-01T01:52:06.481548+0000","last_clean":"2023-12-01T01:52:06.481548+0000","last_became_active":"2023-12-01T01:52:06.478994+0000","last_became_peered":"2023-12-01T01:52:06.478994+0000","last_unstale":"2023-12-01T01:52:06.481548+0000","last_undegraded":"2023-12-01T01:52:06.481548+0000","last_fullsized":"2023-12-01T01:52:06.481548+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.469399+0000","last_change":"2023-12-01T01:52:06.009917+0000","last_active":"2023-12-01T01:52:06.469399+0000","last_peered":"2023-12-01T01:52:06.469399+0000","last_clean":"2023-12-01T01:52:06.469399+0000","last_became_active":"2023-12-01T01:52:06.009593+0000","last_became_peered":"2023-12-01T01:52:06.009593+0000","last_unstale":"2023-12-01T01:52:06.469399+0000","last_undegraded":"2023-12-01T01:52:06.469399+0000","last_fullsized":"2023-12-01T01:52:06.469399+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:11:33.399459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485481+0000","last_change":"2023-12-01T01:52:06.483989+0000","last_active":"2023-12-01T01:52:06.485481+0000","last_peered":"2023-12-01T01:52:06.485481+0000","last_clean":"2023-12-01T01:52:06.485481+0000","last_became_active":"2023-12-01T01:52:06.483660+0000","last_became_peered":"2023-12-01T01:52:06.483660+0000","last_unstale":"2023-12-01T01:52:06.485481+0000","last_undegraded":"2023-12-01T01:52:06.485481+0000","last_fullsized":"2023-12-01T01:52:06.485481+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:58.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.466654+0000","last_change":"2023-12-01T01:52:05.999943+0000","last_active":"2023-12-01T01:52:06.466654+0000","last_peered":"2023-12-01T01:52:06.466654+0000","last_clean":"2023-12-01T01:52:06.466654+0000","last_became_active":"2023-12-01T01:52:05.999649+0000","last_became_peered":"2023-12-01T01:52:05.999649+0000","last_unstale":"2023-12-01T01:52:06.466654+0000","last_undegraded":"2023-12-01T01:52:06.466654+0000","last_fullsized":"2023-12-01T01:52:06.466654+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:42.916412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491715+0000","last_change":"2023-12-01T01:52:06.485967+0000","last_active":"2023-12-01T01:52:06.491715+0000","last_peered":"2023-12-01T01:52:06.491715+0000","last_clean":"2023-12-01T01:52:06.491715+0000","last_became_active":"2023-12-01T01:52:06.484050+0000","last_became_peered":"2023-12-01T01:52:06.484050+0000","last_unstale":"2023-12-01T01:52:06.491715+0000","last_undegraded":"2023-12-01T01:52:06.491715+0000","last_fullsized":"2023-12-01T01:52:06.491715+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:37:55.202441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.503344+0000","last_change":"2023-12-01T01:52:06.002123+0000","last_active":"2023-12-01T01:52:06.503344+0000","last_peered":"2023-12-01T01:52:06.503344+0000","last_clean":"2023-12-01T01:52:06.503344+0000","last_became_active":"2023-12-01T01:52:06.001181+0000","last_became_peered":"2023-12-01T01:52:06.001181+0000","last_unstale":"2023-12-01T01:52:06.503344+0000","last_undegraded":"2023-12-01T01:52:06.503344+0000","last_fullsized":"2023-12-01T01:52:06.503344+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:27:36.883717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.482562+0000","last_change":"2023-12-01T01:52:06.478607+0000","last_active":"2023-12-01T01:52:06.482562+0000","last_peered":"2023-12-01T01:52:06.482562+0000","last_clean":"2023-12-01T01:52:06.482562+0000","last_became_active":"2023-12-01T01:52:06.477495+0000","last_became_peered":"2023-12-01T01:52:06.477495+0000","last_unstale":"2023-12-01T01:52:06.482562+0000","last_undegraded":"2023-12-01T01:52:06.482562+0000","last_fullsized":"2023-12-01T01:52:06.482562+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"96'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.501969+0000","last_change":"2023-12-01T01:52:06.003482+0000","last_active":"2023-12-01T01:52:06.501969+0000","last_peered":"2023-12-01T01:52:06.501969+0000","last_clean":"2023-12-01T01:52:06.501969+0000","last_became_active":"2023-12-01T01:52:06.002884+0000","last_became_peered":"2023-12-01T01:52:06.002884+0000","last_unstale":"2023-12-01T01:52:06.501969+0000","last_undegraded":"2023-12-01T01:52:06.501969+0000","last_fullsized":"2023-12-01T01:52:06.501969+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:14:22.769057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497326+0000","last_change":"2023-12-01T01:52:06.490533+0000","last_active":"2023-12-01T01:52:06.497326+0000","last_peered":"2023-12-01T01:52:06.497326+0000","last_clean":"2023-12-01T01:52:06.497326+0000","last_became_active":"2023-12-01T01:52:06.490269+0000","last_became_peered":"2023-12-01T01:52:06.490269+0000","last_unstale":"2023-12-01T01:52:06.497326+0000","last_undegraded":"2023-12-01T01:52:06.497326+0000","last_fullsized":"2023-12-01T01:52:06.497326+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497862+0000","last_change":"2023-12-01T01:52:05.997054+0000","last_active":"2023-12-01T01:52:06.497862+0000","last_peered":"2023-12-01T01:52:06.497862+0000","last_clean":"2023-12-01T01:52:06.497862+0000","last_became_active":"2023-12-01T01:52:05.996352+0000","last_became_peered":"2023-12-01T01:52:05.996352+0000","last_unstale":"2023-12-01T01:52:06.497862+0000","last_undegraded":"2023-12-01T01:52:06.497862+0000","last_fullsized":"2023-12-01T01:52:06.497862+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492708+0000","last_change":"2023-12-01T01:52:06.489678+0000","last_active":"2023-12-01T01:52:06.492708+0000","last_peered":"2023-12-01T01:52:06.492708+0000","last_clean":"2023-12-01T01:52:06.492708+0000","last_became_active":"2023-12-01T01:52:06.489332+0000","last_became_peered":"2023-12-01T01:52:06.489332+0000","last_unstale":"2023-12-01T01:52:06.492708+0000","last_undegraded":"2023-12-01T01:52:06.492708+0000","last_fullsized":"2023-12-01T01:52:06.492708+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"96'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.502003+0000","last_change":"2023-12-01T01:52:06.004398+0000","last_active":"2023-12-01T01:52:06.502003+0000","last_peered":"2023-12-01T01:52:06.502003+0000","last_clean":"2023-12-01T01:52:06.502003+0000","last_became_active":"2023-12-01T01:52:06.004055+0000","last_became_peered":"2023-12-01T01:52:06.004055+0000","last_unstale":"2023-12-01T01:52:06.502003+0000","last_undegraded":"2023-12-01T01:52:06.502003+0000","last_fullsized":"2023-12-01T01:52:06.502003+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:22.076125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492015+0000","last_change":"2023-12-01T01:52:06.485744+0000","last_active":"2023-12-01T01:52:06.492015+0000","last_peered":"2023-12-01T01:52:06.492015+0000","last_clean":"2023-12-01T01:52:06.492015+0000","last_became_active":"2023-12-01T01:52:06.482211+0000","last_became_peered":"2023-12-01T01:52:06.482211+0000","last_unstale":"2023-12-01T01:52:06.492015+0000","last_undegraded":"2023-12-01T01:52:06.492015+0000","last_fullsized":"2023-12-01T01:52:06.492015+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.481528+0000","last_change":"2023-12-01T01:52:06.003105+0000","last_active":"2023-12-01T01:52:06.481528+0000","last_peered":"2023-12-01T01:52:06.481528+0000","last_clean":"2023-12-01T01:52:06.481528+0000","last_became_active":"2023-12-01T01:52:06.001893+0000","last_became_peered":"2023-12-01T01:52:06.001893+0000","last_unstale":"2023-12-01T01:52:06.481528+0000","last_undegraded":"2023-12-01T01:52:06.481528+0000","last_fullsized":"2023-12-01T01:52:06.481528+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.482600+0000","last_change":"2023-12-01T01:52:06.479989+0000","last_active":"2023-12-01T01:52:06.482600+0000","last_peered":"2023-12-01T01:52:06.482600+0000","last_clean":"2023-12-01T01:52:06.482600+0000","last_became_active":"2023-12-01T01:52:06.476920+0000","last_became_peered":"2023-12-01T01:52:06.476920+0000","last_unstale":"2023-12-01T01:52:06.482600+0000","last_undegraded":"2023-12-01T01:52:06.482600+0000","last_fullsized":"2023-12-01T01:52:06.482600+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.479466+0000","last_change":"2023-12-01T01:52:05.990357+0000","last_active":"2023-12-01T01:52:06.479466+0000","last_peered":"2023-12-01T01:52:06.479466+0000","last_clean":"2023-12-01T01:52:06.479466+0000","last_became_active":"2023-12-01T01:52:05.988196+0000","last_became_peered":"2023-12-01T01:52:05.988196+0000","last_unstale":"2023-12-01T01:52:06.479466+0000","last_undegraded":"2023-12-01T01:52:06.479466+0000","last_fullsized":"2023-12-01T01:52:06.479466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:53.636284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492219+0000","last_change":"2023-12-01T01:52:06.489484+0000","last_active":"2023-12-01T01:52:06.492219+0000","last_peered":"2023-12-01T01:52:06.492219+0000","last_clean":"2023-12-01T01:52:06.492219+0000","last_became_active":"2023-12-01T01:52:06.489010+0000","last_became_peered":"2023-12-01T01:52:06.489010+0000","last_unstale":"2023-12-01T01:52:06.492219+0000","last_undegraded":"2023-12-01T01:52:06.492219+0000","last_fullsized":"2023-12-01T01:52:06.492219+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:54.842423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.469428+0000","last_change":"2023-12-01T01:52:05.990673+0000","last_active":"2023-12-01T01:52:06.469428+0000","last_peered":"2023-12-01T01:52:06.469428+0000","last_clean":"2023-12-01T01:52:06.469428+0000","last_became_active":"2023-12-01T01:52:05.989026+0000","last_became_peered":"2023-12-01T01:52:05.989026+0000","last_unstale":"2023-12-01T01:52:06.469428+0000","last_undegraded":"2023-12-01T01:52:06.469428+0000","last_fullsized":"2023-12-01T01:52:06.469428+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:51:43.565839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485001+0000","last_change":"2023-12-01T01:52:06.483031+0000","last_active":"2023-12-01T01:52:06.485001+0000","last_peered":"2023-12-01T01:52:06.485001+0000","last_clean":"2023-12-01T01:52:06.485001+0000","last_became_active":"2023-12-01T01:52:06.480967+0000","last_became_peered":"2023-12-01T01:52:06.480967+0000","last_unstale":"2023-12-01T01:52:06.485001+0000","last_undegraded":"2023-12-01T01:52:06.485001+0000","last_fullsized":"2023-12-01T01:52:06.485001+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.462983+0000","last_change":"2023-12-01T01:52:06.003576+0000","last_active":"2023-12-01T01:52:06.462983+0000","last_peered":"2023-12-01T01:52:06.462983+0000","last_clean":"2023-12-01T01:52:06.462983+0000","last_became_active":"2023-12-01T01:52:06.003254+0000","last_became_peered":"2023-12-01T01:52:06.003254+0000","last_unstale":"2023-12-01T01:52:06.462983+0000","last_undegraded":"2023-12-01T01:52:06.462983+0000","last_fullsized":"2023-12-01T01:52:06.462983+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:10.387281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.486039+0000","last_change":"2023-12-01T01:52:06.484497+0000","last_active":"2023-12-01T01:52:06.486039+0000","last_peered":"2023-12-01T01:52:06.486039+0000","last_clean":"2023-12-01T01:52:06.486039+0000","last_became_active":"2023-12-01T01:52:06.483710+0000","last_became_peered":"2023-12-01T01:52:06.483710+0000","last_unstale":"2023-12-01T01:52:06.486039+0000","last_undegraded":"2023-12-01T01:52:06.486039+0000","last_fullsized":"2023-12-01T01:52:06.486039+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:31.031923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.502109+0000","last_change":"2023-12-01T01:52:06.003659+0000","last_active":"2023-12-01T01:52:06.502109+0000","last_peered":"2023-12-01T01:52:06.502109+0000","last_clean":"2023-12-01T01:52:06.502109+0000","last_became_active":"2023-12-01T01:52:06.003228+0000","last_became_peered":"2023-12-01T01:52:06.003228+0000","last_unstale":"2023-12-01T01:52:06.502109+0000","last_undegraded":"2023-12-01T01:52:06.502109+0000","last_fullsized":"2023-12-01T01:52:06.502109+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:14.620804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.495897+0000","last_change":"2023-12-01T01:52:06.486528+0000","last_active":"2023-12-01T01:52:06.495897+0000","last_peered":"2023-12-01T01:52:06.495897+0000","last_clean":"2023-12-01T01:52:06.495897+0000","last_became_active":"2023-12-01T01:52:06.486273+0000","last_became_peered":"2023-12-01T01:52:06.486273+0000","last_unstale":"2023-12-01T01:52:06.495897+0000","last_undegraded":"2023-12-01T01:52:06.495897+0000","last_fullsized":"2023-12-01T01:52:06.495897+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:35:33.172589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.476790+0000","last_change":"2023-12-01T01:52:06.000889+0000","last_active":"2023-12-01T01:52:06.476790+0000","last_peered":"2023-12-01T01:52:06.476790+0000","last_clean":"2023-12-01T01:52:06.476790+0000","last_became_active":"2023-12-01T01:52:06.000644+0000","last_became_peered":"2023-12-01T01:52:06.000644+0000","last_unstale":"2023-12-01T01:52:06.476790+0000","last_undegraded":"2023-12-01T01:52:06.476790+0000","last_fullsized":"2023-12-01T01:52:06.476790+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:19:11.688232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.489967+0000","last_change":"2023-12-01T01:52:06.483516+0000","last_active":"2023-12-01T01:52:06.489967+0000","last_peered":"2023-12-01T01:52:06.489967+0000","last_clean":"2023-12-01T01:52:06.489967+0000","last_became_active":"2023-12-01T01:52:06.482601+0000","last_became_peered":"2023-12-01T01:52:06.482601+0000","last_unstale":"2023-12-01T01:52:06.489967+0000","last_undegraded":"2023-12-01T01:52:06.489967+0000","last_fullsized":"2023-12-01T01:52:06.489967+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.477607+0000","last_change":"2023-12-01T01:52:06.006527+0000","last_active":"2023-12-01T01:52:06.477607+0000","last_peered":"2023-12-01T01:52:06.477607+0000","last_clean":"2023-12-01T01:52:06.477607+0000","last_became_active":"2023-12-01T01:52:06.005261+0000","last_became_peered":"2023-12-01T01:52:06.005261+0000","last_unstale":"2023-12-01T01:52:06.477607+0000","last_undegraded":"2023-12-01T01:52:06.477607+0000","last_fullsized":"2023-12-01T01:52:06.477607+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:51:05.076663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.494244+0000","last_change":"2023-12-01T01:52:06.485932+0000","last_active":"2023-12-01T01:52:06.494244+0000","last_peered":"2023-12-01T01:52:06.494244+0000","last_clean":"2023-12-01T01:52:06.494244+0000","last_became_active":"2023-12-01T01:52:06.483597+0000","last_became_peered":"2023-12-01T01:52:06.483597+0000","last_unstale":"2023-12-01T01:52:06.494244+0000","last_undegraded":"2023-12-01T01:52:06.494244+0000","last_fullsized":"2023-12-01T01:52:06.494244+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:33:59.577488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.476504+0000","last_change":"2023-12-01T01:52:05.997953+0000","last_active":"2023-12-01T01:52:06.476504+0000","last_peered":"2023-12-01T01:52:06.476504+0000","last_clean":"2023-12-01T01:52:06.476504+0000","last_became_active":"2023-12-01T01:52:05.997697+0000","last_became_peered":"2023-12-01T01:52:05.997697+0000","last_unstale":"2023-12-01T01:52:06.476504+0000","last_undegraded":"2023-12-01T01:52:06.476504+0000","last_fullsized":"2023-12-01T01:52:06.476504+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:41:32.705632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.478779+0000","last_change":"2023-12-01T01:52:06.475303+0000","last_active":"2023-12-01T01:52:06.478779+0000","last_peered":"2023-12-01T01:52:06.478779+0000","last_clean":"2023-12-01T01:52:06.478779+0000","last_became_active":"2023-12-01T01:52:06.474510+0000","last_became_peered":"2023-12-01T01:52:06.474510+0000","last_unstale":"2023-12-01T01:52:06.478779+0000","last_undegraded":"2023-12-01T01:52:06.478779+0000","last_fullsized":"2023-12-01T01:52:06.478779+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:35:55.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.476598+0000","last_change":"2023-12-01T01:52:05.996908+0000","last_active":"2023-12-01T01:52:06.476598+0000","last_peered":"2023-12-01T01:52:06.476598+0000","last_clean":"2023-12-01T01:52:06.476598+0000","last_became_active":"2023-12-01T01:52:05.995875+0000","last_became_peered":"2023-12-01T01:52:05.995875+0000","last_unstale":"2023-12-01T01:52:06.476598+0000","last_undegraded":"2023-12-01T01:52:06.476598+0000","last_fullsized":"2023-12-01T01:52:06.476598+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:37.605903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485924+0000","last_change":"2023-12-01T01:52:06.479161+0000","last_active":"2023-12-01T01:52:06.485924+0000","last_peered":"2023-12-01T01:52:06.485924+0000","last_clean":"2023-12-01T01:52:06.485924+0000","last_became_active":"2023-12-01T01:52:06.478891+0000","last_became_peered":"2023-12-01T01:52:06.478891+0000","last_unstale":"2023-12-01T01:52:06.485924+0000","last_undegraded":"2023-12-01T01:52:06.485924+0000","last_fullsized":"2023-12-01T01:52:06.485924+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:49:44.833749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.463972+0000","last_change":"2023-12-01T01:52:06.003362+0000","last_active":"2023-12-01T01:52:06.463972+0000","last_peered":"2023-12-01T01:52:06.463972+0000","last_clean":"2023-12-01T01:52:06.463972+0000","last_became_active":"2023-12-01T01:52:06.003048+0000","last_became_peered":"2023-12-01T01:52:06.003048+0000","last_unstale":"2023-12-01T01:52:06.463972+0000","last_undegraded":"2023-12-01T01:52:06.463972+0000","last_fullsized":"2023-12-01T01:52:06.463972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:48.873308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485703+0000","last_change":"2023-12-01T01:52:06.482989+0000","last_active":"2023-12-01T01:52:06.485703+0000","last_peered":"2023-12-01T01:52:06.485703+0000","last_clean":"2023-12-01T01:52:06.485703+0000","last_became_active":"2023-12-01T01:52:06.480990+0000","last_became_peered":"2023-12-01T01:52:06.480990+0000","last_unstale":"2023-12-01T01:52:06.485703+0000","last_undegraded":"2023-12-01T01:52:06.485703+0000","last_fullsized":"2023-12-01T01:52:06.485703+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.469520+0000","last_change":"2023-12-01T01:52:05.990037+0000","last_active":"2023-12-01T01:52:06.469520+0000","last_peered":"2023-12-01T01:52:06.469520+0000","last_clean":"2023-12-01T01:52:06.469520+0000","last_became_active":"2023-12-01T01:52:05.988261+0000","last_became_peered":"2023-12-01T01:52:05.988261+0000","last_unstale":"2023-12-01T01:52:06.469520+0000","last_undegraded":"2023-12-01T01:52:06.469520+0000","last_fullsized":"2023-12-01T01:52:06.469520+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:43.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.488961+0000","last_change":"2023-12-01T01:52:06.485149+0000","last_active":"2023-12-01T01:52:06.488961+0000","last_peered":"2023-12-01T01:52:06.488961+0000","last_clean":"2023-12-01T01:52:06.488961+0000","last_became_active":"2023-12-01T01:52:06.484566+0000","last_became_peered":"2023-12-01T01:52:06.484566+0000","last_unstale":"2023-12-01T01:52:06.488961+0000","last_undegraded":"2023-12-01T01:52:06.488961+0000","last_fullsized":"2023-12-01T01:52:06.488961+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:14:44.417471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.478518+0000","last_change":"2023-12-01T01:52:05.998968+0000","last_active":"2023-12-01T01:52:06.478518+0000","last_peered":"2023-12-01T01:52:06.478518+0000","last_clean":"2023-12-01T01:52:06.478518+0000","last_became_active":"2023-12-01T01:52:05.998647+0000","last_became_peered":"2023-12-01T01:52:05.998647+0000","last_unstale":"2023-12-01T01:52:06.478518+0000","last_undegraded":"2023-12-01T01:52:06.478518+0000","last_fullsized":"2023-12-01T01:52:06.478518+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:22.076125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492485+0000","last_change":"2023-12-01T01:52:06.489260+0000","last_active":"2023-12-01T01:52:06.492485+0000","last_peered":"2023-12-01T01:52:06.492485+0000","last_clean":"2023-12-01T01:52:06.492485+0000","last_became_active":"2023-12-01T01:52:06.489006+0000","last_became_peered":"2023-12-01T01:52:06.489006+0000","last_unstale":"2023-12-01T01:52:06.492485+0000","last_undegraded":"2023-12-01T01:52:06.492485+0000","last_fullsized":"2023-12-01T01:52:06.492485+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:12.910138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.498261+0000","last_change":"2023-12-01T01:52:06.000732+0000","last_active":"2023-12-01T01:52:06.498261+0000","last_peered":"2023-12-01T01:52:06.498261+0000","last_clean":"2023-12-01T01:52:06.498261+0000","last_became_active":"2023-12-01T01:52:06.000124+0000","last_became_peered":"2023-12-01T01:52:06.000124+0000","last_unstale":"2023-12-01T01:52:06.498261+0000","last_undegraded":"2023-12-01T01:52:06.498261+0000","last_fullsized":"2023-12-01T01:52:06.498261+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:39.158199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"85'172","reported_seq":433,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-12-01T01:52:08.635077+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:52:08.635077+0000","last_peered":"2023-12-01T01:52:08.635077+0000","last_clean":"2023-12-01T01:52:08.635077+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:52:08.635077+0000","last_undegraded":"2023-12-01T01:52:08.635077+0000","last_fullsized":"2023-12-01T01:52:08.635077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1196032,"data_stored":563740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0389999999999999,"15min":1.0640000000000001},"min":{"1min":0.65200000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2869999999999999,"5min":1.5449999999999999,"15min":1.8089999999999999},"last":1.401},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.05,"15min":1.0489999999999999},"min":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.64000000000000001},"max":{"1min":1.2969999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.96499999999999997}]},{"osd":1,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.2010000000000001,"15min":1.2130000000000001},"min":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.351,"5min":1.8100000000000001,"15min":1.8420000000000001},"last":1.131},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2410000000000001,"15min":1.252},"min":{"1min":0.78900000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.629,"5min":1.744,"15min":2.0870000000000002},"last":1.4259999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.127,"15min":1.121},"min":{"1min":0.73299999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.607,"5min":1.728,"15min":1.728},"last":1.24},{"interface":"front","average":{"1min":1.175,"5min":1.208,"15min":1.2230000000000001},"min":{"1min":0.89700000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5489999999999999,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2230000000000001,"15min":1.2310000000000001},"min":{"1min":0.75,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.3979999999999999,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":1.347},{"interface":"front","average":{"1min":1.337,"5min":1.3440000000000001,"15min":1.347},"min":{"1min":1.123,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6259999999999999,"5min":2.1880000000000002,"15min":2.7050000000000001},"last":1.5229999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3160000000000001,"15min":1.296},"min":{"1min":1.036,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5840000000000001,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4550000000000001,"15min":1.4139999999999999},"min":{"1min":0.96399999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7210000000000001,"5min":2.0939999999999999,"15min":2.601},"last":1.286}]},{"osd":5,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3169999999999999,"15min":1.2689999999999999},"min":{"1min":0.84899999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.4500000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.532,"15min":1.5209999999999999},"min":{"1min":1.147,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.802,"5min":2.2930000000000001,"15min":2.8069999999999999},"last":1.6319999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4119999999999999,"15min":1.383},"min":{"1min":1.0129999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.294,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.397},"min":{"1min":1.115,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7110000000000001,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.4770000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1200128,"data_stored":563774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.091,"15min":1.103},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.55,"5min":1.635,"15min":1.79},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1399999999999999,"15min":1.127},"min":{"1min":0.77200000000000002,"5min":0.64400000000000002,"15min":0.61799999999999999},"max":{"1min":1.327,"5min":2.5569999999999999,"15min":2.73},"last":1.1559999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1639999999999999,"15min":1.1559999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.47,"5min":1.8620000000000001,"15min":2.2170000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1950000000000001,"15min":1.2050000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.494,"5min":1.6259999999999999,"15min":2.0920000000000001},"last":1.3600000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.214,"15min":1.1659999999999999},"min":{"1min":0.72299999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4510000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.131,"5min":1.2450000000000001,"15min":1.24},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5549999999999999,"5min":2.137,"15min":2.137},"last":1.2050000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2869999999999999,"15min":1.2789999999999999},"min":{"1min":0.91600000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4339999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.04},{"interface":"front","average":{"1min":1.365,"5min":1.4139999999999999,"15min":1.403},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.81299999999999994},"max":{"1min":1.5569999999999999,"5min":1.742,"15min":2.4020000000000001},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.298,"15min":1.28},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6830000000000001,"5min":2.226,"15min":2.3399999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.3460000000000001,"15min":1.347},"min":{"1min":0.93600000000000005,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.6060000000000001,"5min":1.931,"15min":2.6760000000000002},"last":1.655}]},{"osd":5,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4330000000000001,"15min":1.4099999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6779999999999999,"5min":2.1899999999999999,"15min":2.5539999999999998},"last":1.431},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.498,"15min":1.4750000000000001},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":0.748},"max":{"1min":1.786,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.5620000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.5389999999999999,"15min":1.514},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.89500000000000002},"max":{"1min":1.651,"5min":2.762,"15min":2.762},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.5960000000000001,"15min":1.5740000000000001},"min":{"1min":1.234,"5min":1.155,"15min":0.78700000000000003},"max":{"1min":1.7050000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.444}]}]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1781760,"data_stored":1152634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.1060000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.498,"5min":1.651,"15min":3.3980000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.127,"15min":1.141},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.60099999999999998},"max":{"1min":1.5720000000000001,"5min":1.6479999999999999,"15min":2.153},"last":1.7130000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.153,"15min":1.137},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.7010000000000001},"last":1.228},{"interface":"front","average":{"1min":1.179,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.73599999999999999,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.26}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4039999999999999,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.6080000000000001,"5min":1.621,"15min":2.0630000000000002},"last":1.3939999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.286},"min":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.76900000000000002},"max":{"1min":1.5309999999999999,"5min":1.694,"15min":1.833},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3120000000000001,"15min":1.296},"min":{"1min":0.98099999999999998,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":1.6779999999999999,"5min":1.7110000000000001,"15min":2.1440000000000001},"last":1.5029999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.238,"15min":1.2390000000000001},"min":{"1min":0.86399999999999999,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.6779999999999999,"5min":1.853,"15min":2.5129999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.84999999999999998,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.6319999999999999,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.5409999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2989999999999999,"15min":1.329},"min":{"1min":0.88500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":2.2970000000000002},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":1.125,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":1.962,"5min":1.962,"15min":3.23},"last":1.3560000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4530000000000001,"15min":1.48},"min":{"1min":0.92400000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2,"5min":2,"15min":3.012},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4970000000000001,"15min":1.524},"min":{"1min":1.2689999999999999,"5min":0.873,"15min":0.873},"max":{"1min":2.073,"5min":2.073,"15min":2.8420000000000001},"last":1.6699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":1196032,"data_stored":562870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.119,"15min":1.1040000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.53,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.734,"5min":1.827,"15min":3.8069999999999999},"last":0.86199999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2350000000000001,"15min":1.2090000000000001},"min":{"1min":0.84599999999999997,"5min":0.81999999999999995,"15min":0.65900000000000003},"max":{"1min":1.528,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.131},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.671,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2110000000000001,"15min":1.196},"min":{"1min":0.83999999999999997,"5min":0.78200000000000003,"15min":0.77600000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3140000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.871,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.97399999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.234},"min":{"1min":0.85399999999999998,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.8220000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.383},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3200000000000001,"15min":1.319},"min":{"1min":0.753,"5min":0.753,"15min":0.70399999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.3010000000000002},"last":1.1339999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.96599999999999997,"5min":0.79200000000000004,"15min":0.61899999999999999},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.1549999999999998},"last":1.087},{"interface":"front","average":{"1min":1.248,"5min":1.2829999999999999,"15min":1.292},"min":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.5189999999999999,"5min":1.8029999999999999,"15min":2.2029999999999998},"last":1.272}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.94999999999999996,"5min":0.84799999999999998,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.5029999999999999,"15min":1.4730000000000001},"min":{"1min":1.1000000000000001,"5min":0.96699999999999997,"15min":0.63200000000000001},"max":{"1min":1.7949999999999999,"5min":2.4990000000000001,"15min":2.54},"last":1.2470000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.458,"15min":1.464},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.60599999999999998},"max":{"1min":1.6850000000000001,"5min":1.972,"15min":3.2789999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5720000000000001,"15min":1.5580000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.04,"15min":2.157},"last":1.528}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40408,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331432,"statfs":{"total":96636764160,"available":96595386368,"internally_reserved":0,"allocated":1196032,"data_stored":562302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81599999999999995,"15min":0.84799999999999998},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.307},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.583},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97599999999999998,"15min":0.996},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.3160000000000001,"5min":1.476,"15min":1.9770000000000001},"last":0.53700000000000003}]},{"osd":1,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.052,"15min":1.0660000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.38900000000000001},"max":{"1min":1.423,"5min":1.571,"15min":2.0779999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.92200000000000004,"15min":0.94999999999999996},"min":{"1min":0.504,"5min":0.48199999999999998,"15min":0.40600000000000003},"max":{"1min":1.385,"5min":1.587,"15min":1.6519999999999999},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99399999999999999,"15min":1.0469999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.436},"max":{"1min":1.4810000000000001,"5min":1.512,"15min":1.766},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.163,"5min":1.167,"15min":1.1850000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.48399999999999999},"max":{"1min":1.522,"5min":1.647,"15min":1.661},"last":0.80200000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.212,"15min":1.2450000000000001},"min":{"1min":0.76900000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.4550000000000001,"5min":1.5820000000000001,"15min":2.9350000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2150000000000001,"15min":1.24},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.599,"5min":1.637,"15min":2.2389999999999999},"last":0.84999999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.24,"15min":1.29},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.63500000000000001},"max":{"1min":1.7669999999999999,"5min":1.806,"15min":3.476},"last":0.995},{"interface":"front","average":{"1min":1.244,"5min":1.266,"15min":1.3109999999999999},"min":{"1min":0.92100000000000004,"5min":0.82099999999999995,"15min":0.64200000000000002},"max":{"1min":1.681,"5min":1.9039999999999999,"15min":3.181},"last":0.97999999999999998}]},{"osd":6,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.363,"15min":1.377},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.68300000000000005},"max":{"1min":1.6040000000000001,"5min":1.8580000000000001,"15min":2.532},"last":1.198},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3919999999999999,"15min":1.4259999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.872},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":3.0249999999999999},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.389,"15min":1.421},"min":{"1min":1.0600000000000001,"5min":0.999,"15min":0.81499999999999995},"max":{"1min":1.998,"5min":1.998,"15min":2.1619999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.415,"15min":1.4610000000000001},"min":{"1min":0.91200000000000003,"5min":0.88300000000000001,"15min":0.83099999999999996},"max":{"1min":1.6879999999999999,"5min":1.8779999999999999,"15min":2.4590000000000001},"last":1.1080000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":562758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27100000000000002},"max":{"1min":1.0860000000000001,"5min":1.339,"15min":1.738},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.026,"15min":1.01},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.375},"max":{"1min":1.4159999999999999,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96499999999999997,"15min":0.98999999999999999},"min":{"1min":0.54700000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.1779999999999999,"5min":1.45,"15min":1.7130000000000001},"last":0.629},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0720000000000001,"15min":1.0840000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.504},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97499999999999998,"15min":1.0029999999999999},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.377,"5min":1.625,"15min":1.883},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.39600000000000002},"max":{"1min":1.415,"5min":1.613,"15min":2.6080000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.331,"15min":1.3180000000000001},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.75},"max":{"1min":1.7230000000000001,"5min":1.9950000000000001,"15min":2.2829999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.246,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.964},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3009999999999999,"15min":1.319},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.647,"5min":1.804,"15min":2.5779999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.375,"5min":1.387,"15min":1.375},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.901,"5min":2.048,"15min":3.0099999999999998},"last":1.2490000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.3200000000000001},"min":{"1min":0.84299999999999997,"5min":0.83399999999999996,"15min":0.77500000000000002},"max":{"1min":1.6439999999999999,"5min":1.78,"15min":2.2799999999999998},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.766,"15min":2.5089999999999999},"last":1.181}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.456,"15min":1.448},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.79400000000000004},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.448},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7649999999999999,"5min":1.802,"15min":2.5049999999999999},"last":1.323}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59024,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312816,"statfs":{"total":96636764160,"available":96576323584,"internally_reserved":0,"allocated":1777664,"data_stored":1152544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85199999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.318},"max":{"1min":1.341,"5min":1.5049999999999999,"15min":3.056},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.052,"5min":0.97999999999999998,"15min":1.016},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":3.4129999999999998},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.88600000000000001,"15min":0.89600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.36399999999999999},"max":{"1min":1.1919999999999999,"5min":1.54,"15min":3.3660000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0680000000000001,"15min":1.095},"min":{"1min":0.71199999999999997,"5min":0.47899999999999998,"15min":0.39000000000000001},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":3.4980000000000002},"last":1.23}]},{"osd":3,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.194,"15min":1.216},"min":{"1min":0.55400000000000005,"5min":0.51900000000000002,"15min":0.44},"max":{"1min":1.827,"5min":1.827,"15min":3.464},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.048,"15min":1.0529999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39700000000000002},"max":{"1min":1.657,"5min":1.6699999999999999,"15min":3.0960000000000001},"last":1.137}]},{"osd":4,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.175,"15min":1.1870000000000001},"min":{"1min":0.871,"5min":0.745,"15min":0.63800000000000001},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":3.7810000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3160000000000001,"15min":1.3440000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6559999999999999,"5min":1.8089999999999999,"15min":4.7270000000000003},"last":1.337}]},{"osd":5,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2829999999999999,"15min":1.2809999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6839999999999999,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.268,"15min":1.296},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":1.825,"15min":3.6669999999999998},"last":1.194}]},{"osd":6,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3089999999999999,"15min":1.3360000000000001},"min":{"1min":0.79400000000000004,"5min":0.71099999999999997,"15min":0.68400000000000005},"max":{"1min":1.7490000000000001,"5min":1.7809999999999999,"15min":3.3300000000000001},"last":1.381},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.3460000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5389999999999999,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.29}]},{"osd":7,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3979999999999999,"15min":1.4419999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.774,"5min":1.8939999999999999,"15min":4.6749999999999998},"last":1.423},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.4159999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8440000000000001,"5min":1.992,"15min":3.5379999999999998},"last":1.4690000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575044,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":562246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.60699999999999998,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.3520000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.92600000000000005,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4390000000000001,"5min":1.5580000000000001,"15min":1.6910000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98499999999999999,"15min":1.006},"min":{"1min":0.67600000000000005,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.4690000000000001,"5min":2.157,"15min":2.157},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.113,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.84299999999999997,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.395,"5min":1.728,"15min":2.8159999999999998},"last":1.0820000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.58499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.954},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.143,"15min":1.169},"min":{"1min":0.54900000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7070000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1279999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.3089999999999999},"min":{"1min":1.05,"5min":0.78500000000000003,"15min":0.65700000000000003},"max":{"1min":1.5289999999999999,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.196},{"interface":"front","average":{"1min":1.22,"5min":1.214,"15min":1.2629999999999999},"min":{"1min":0.83299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.391}]},{"osd":5,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.337,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.71899999999999997},"max":{"1min":1.839,"5min":2.351,"15min":2.351},"last":1.232},{"interface":"front","average":{"1min":1.276,"5min":1.306,"15min":1.349},"min":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6299999999999999,"5min":2.2909999999999999,"15min":2.7050000000000001},"last":1.3100000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73099999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.157},"last":1.341},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.399},"min":{"1min":1.0129999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.292}]},{"osd":7,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":1.127,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.706,"5min":1.798,"15min":2.8999999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.361,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.276}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:52:15.856 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:52:16.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.080+0000 7fc103651640 1 Processor -- start 2023-12-01T01:52:16.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.080+0000 7fc103651640 1 -- start start 2023-12-01T01:52:16.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.082+0000 7fc103651640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 0x7fc0fc0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.082+0000 7fc103651640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc0fc070d30 0x7fc0fc0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.082+0000 7fc103651640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0fc06f8b0 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.082+0000 7fc103651640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc0adcb0 con 0x7fc0fc070d30 2023-12-01T01:52:16.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.082+0000 7fc103651640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0ad9d0 con 0x7fc0fc159dd0 2023-12-01T01:52:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.082+0000 7fc101bee640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc0fc070d30 0x7fc0fc0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.082+0000 7fc1013ed640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 0x7fc0fc0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.083+0000 7fc101bee640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc0fc070d30 0x7fc0fc0c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53646/0 (socket says 172.21.15.115:53646) 2023-12-01T01:52:16.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.083+0000 7fc101bee640 1 -- 172.21.15.115:0/4176691560 learned_addr learned my addr 172.21.15.115:0/4176691560 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:16.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.083+0000 7fc101bee640 1 -- 172.21.15.115:0/4176691560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 msgr2=0x7fc0fc0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:16.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.083+0000 7fc101bee640 1 --2- 172.21.15.115:0/4176691560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 0x7fc0fc0c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.083+0000 7fc101bee640 1 -- 172.21.15.115:0/4176691560 >> v1:172.21.15.142:6789/0 conn(0x7fc0fc06b6f0 legacy=0x7fc0fc0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.083+0000 7fc101bee640 1 -- 172.21.15.115:0/4176691560 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc0c9000 con 0x7fc0fc070d30 2023-12-01T01:52:16.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.083+0000 7fc1013ed640 1 --2- 172.21.15.115:0/4176691560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 0x7fc0fc0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:52:16.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.085+0000 7fc101bee640 1 --2- 172.21.15.115:0/4176691560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0fc070d30 0x7fc0fc0c9d20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009e50 tx=0x7fc0f402e060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8512f7bf2c78bcf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.085+0000 7fc0f3fff640 1 -- 172.21.15.115:0/4176691560 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f402ec00 con 0x7fc0fc070d30 2023-12-01T01:52:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.085+0000 7fc0f3fff640 1 -- 172.21.15.115:0/4176691560 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f402ed60 con 0x7fc0fc070d30 2023-12-01T01:52:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.085+0000 7fc0f3fff640 1 -- 172.21.15.115:0/4176691560 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4036610 con 0x7fc0fc070d30 2023-12-01T01:52:16.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.086+0000 7fc103651640 1 -- 172.21.15.115:0/4176691560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0fc070d30 msgr2=0x7fc0fc0c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.086+0000 7fc103651640 1 --2- 172.21.15.115:0/4176691560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0fc070d30 0x7fc0fc0c9d20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009e50 tx=0x7fc0f402e060 comp rx=0 tx=0).stop 2023-12-01T01:52:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.086+0000 7fc103651640 1 -- 172.21.15.115:0/4176691560 shutdown_connections 2023-12-01T01:52:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.086+0000 7fc103651640 1 --2- 172.21.15.115:0/4176691560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0fc070d30 0x7fc0fc0c9d20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.086+0000 7fc103651640 1 --2- 172.21.15.115:0/4176691560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 0x7fc0fc0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.086+0000 7fc103651640 1 -- 172.21.15.115:0/4176691560 >> 172.21.15.115:0/4176691560 conn(0x7fc0fc0ce880 msgr2=0x7fc0fc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.086+0000 7fc103651640 1 -- 172.21.15.115:0/4176691560 shutdown_connections 2023-12-01T01:52:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.086+0000 7fc103651640 1 -- 172.21.15.115:0/4176691560 wait complete. 2023-12-01T01:52:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.088+0000 7fc103651640 1 Processor -- start 2023-12-01T01:52:16.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.088+0000 7fc103651640 1 -- start start 2023-12-01T01:52:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.089+0000 7fc103651640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0fc06b6f0 0x7fc0fc158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.089+0000 7fc103651640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0fc070d30 0x7fc0fc150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.089+0000 7fc103651640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 0x7fc0fc15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.089+0000 7fc103651640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0c2550 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.089+0000 7fc103651640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0c2830 con 0x7fc0fc070d30 2023-12-01T01:52:16.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.089+0000 7fc103651640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0c2b10 con 0x7fc0fc159dd0 2023-12-01T01:52:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.089+0000 7fc1013ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0fc06b6f0 0x7fc0fc158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.089+0000 7fc1013ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0fc06b6f0 0x7fc0fc158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47032/0 (socket says 172.21.15.115:47032) 2023-12-01T01:52:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.089+0000 7fc1013ed640 1 -- 172.21.15.115:0/1318208580 learned_addr learned my addr 172.21.15.115:0/1318208580 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc1013ed640 1 -- 172.21.15.115:0/1318208580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 msgr2=0x7fc0fc15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc101bee640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 0x7fc0fc15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc100bec640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0fc070d30 0x7fc0fc150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc1013ed640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 0x7fc0fc15a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc1013ed640 1 -- 172.21.15.115:0/1318208580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0fc070d30 msgr2=0x7fc0fc150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc1013ed640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0fc070d30 0x7fc0fc150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc1013ed640 1 -- 172.21.15.115:0/1318208580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f8009400 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc100bec640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0fc070d30 0x7fc0fc150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc101bee640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 0x7fc0fc15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc1013ed640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0fc06b6f0 0x7fc0fc158440 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800e960 tx=0x7fc0f800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc0f1ffb640 1 -- 172.21.15.115:0/1318208580 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800ca00 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc0f1ffb640 1 -- 172.21.15.115:0/1318208580 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f800cb60 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc0f1ffb640 1 -- 172.21.15.115:0/1318208580 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8013470 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.090+0000 7fc103651640 1 -- 172.21.15.115:0/1318208580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f4009b30 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.091+0000 7fc103651640 1 -- 172.21.15.115:0/1318208580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0fc0c30d0 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.092+0000 7fc103651640 1 -- 172.21.15.115:0/1318208580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0fc04e890 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.092+0000 7fc0f1ffb640 1 -- 172.21.15.115:0/1318208580 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc0f80135d0 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.093+0000 7fc0f1ffb640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc0e006a580 0x7fc0e006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.093+0000 7fc0f1ffb640 1 -- 172.21.15.115:0/1318208580 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0f8088e40 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.094+0000 7fc100bec640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc0e006a580 0x7fc0e006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.094+0000 7fc100bec640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc0e006a580 0x7fc0e006ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc151480 tx=0x7fc0ec009040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:16.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.098+0000 7fc0f1ffb640 1 -- 172.21.15.115:0/1318208580 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc0f80541f0 con 0x7fc0fc06b6f0 2023-12-01T01:52:16.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.282+0000 7fc103651640 1 -- 172.21.15.115:0/1318208580 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0fc0c29a0 con 0x7fc0e006a580 2023-12-01T01:52:16.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.305+0000 7fc0f1ffb640 1 -- 172.21.15.115:0/1318208580 <== mgr.6723 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286853 (secure 0 0 0) 0x7fc0fc0c29a0 con 0x7fc0e006a580 2023-12-01T01:52:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:16.315 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 -- 172.21.15.115:0/1318208580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc0e006a580 msgr2=0x7fc0e006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc0e006a580 0x7fc0e006ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc151480 tx=0x7fc0ec009040 comp rx=0 tx=0).stop 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 -- 172.21.15.115:0/1318208580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0fc06b6f0 msgr2=0x7fc0fc158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0fc06b6f0 0x7fc0fc158440 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800e960 tx=0x7fc0f800ee20 comp rx=0 tx=0).stop 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 -- 172.21.15.115:0/1318208580 shutdown_connections 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc0e006a580 0x7fc0e006ca40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0fc159dd0 0x7fc0fc15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0fc070d30 0x7fc0fc150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 --2- 172.21.15.115:0/1318208580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0fc06b6f0 0x7fc0fc158440 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 -- 172.21.15.115:0/1318208580 >> 172.21.15.115:0/1318208580 conn(0x7fc0fc0ce880 msgr2=0x7fc0fc0af910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:16.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 -- 172.21.15.115:0/1318208580 shutdown_connections 2023-12-01T01:52:16.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.312+0000 7fc103651640 1 -- 172.21.15.115:0/1318208580 wait complete. 2023-12-01T01:52:16.333 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-01T01:52:16.032826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":303,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":293288,"kb_used_data":10472,"kb_used_omap":0,"kb_used_meta":282752,"kb_avail":754681432,"statfs":{"total":773094113280,"available":772793786368,"internally_reserved":0,"allocated":10723328,"data_stored":5682868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":289538048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.573133"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.500421+0000","last_change":"2023-12-01T01:52:06.489980+0000","last_active":"2023-12-01T01:52:06.500421+0000","last_peered":"2023-12-01T01:52:06.500421+0000","last_clean":"2023-12-01T01:52:06.500421+0000","last_became_active":"2023-12-01T01:52:06.489658+0000","last_became_peered":"2023-12-01T01:52:06.489658+0000","last_unstale":"2023-12-01T01:52:06.500421+0000","last_undegraded":"2023-12-01T01:52:06.500421+0000","last_fullsized":"2023-12-01T01:52:06.500421+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:12:04.610539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.488816+0000","last_change":"2023-12-01T01:52:06.483199+0000","last_active":"2023-12-01T01:52:06.488816+0000","last_peered":"2023-12-01T01:52:06.488816+0000","last_clean":"2023-12-01T01:52:06.488816+0000","last_became_active":"2023-12-01T01:52:06.482954+0000","last_became_peered":"2023-12-01T01:52:06.482954+0000","last_unstale":"2023-12-01T01:52:06.488816+0000","last_undegraded":"2023-12-01T01:52:06.488816+0000","last_fullsized":"2023-12-01T01:52:06.488816+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:53.713026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485604+0000","last_change":"2023-12-01T01:52:06.477868+0000","last_active":"2023-12-01T01:52:06.485604+0000","last_peered":"2023-12-01T01:52:06.485604+0000","last_clean":"2023-12-01T01:52:06.485604+0000","last_became_active":"2023-12-01T01:52:06.477587+0000","last_became_peered":"2023-12-01T01:52:06.477587+0000","last_unstale":"2023-12-01T01:52:06.485604+0000","last_undegraded":"2023-12-01T01:52:06.485604+0000","last_fullsized":"2023-12-01T01:52:06.485604+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.489524+0000","last_change":"2023-12-01T01:52:06.477629+0000","last_active":"2023-12-01T01:52:06.489524+0000","last_peered":"2023-12-01T01:52:06.489524+0000","last_clean":"2023-12-01T01:52:06.489524+0000","last_became_active":"2023-12-01T01:52:06.477344+0000","last_became_peered":"2023-12-01T01:52:06.477344+0000","last_unstale":"2023-12-01T01:52:06.489524+0000","last_undegraded":"2023-12-01T01:52:06.489524+0000","last_fullsized":"2023-12-01T01:52:06.489524+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:54.640923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485248+0000","last_change":"2023-12-01T01:52:06.483451+0000","last_active":"2023-12-01T01:52:06.485248+0000","last_peered":"2023-12-01T01:52:06.485248+0000","last_clean":"2023-12-01T01:52:06.485248+0000","last_became_active":"2023-12-01T01:52:06.483171+0000","last_became_peered":"2023-12-01T01:52:06.483171+0000","last_unstale":"2023-12-01T01:52:06.485248+0000","last_undegraded":"2023-12-01T01:52:06.485248+0000","last_fullsized":"2023-12-01T01:52:06.485248+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:11.568997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.501575+0000","last_change":"2023-12-01T01:52:06.499765+0000","last_active":"2023-12-01T01:52:06.501575+0000","last_peered":"2023-12-01T01:52:06.501575+0000","last_clean":"2023-12-01T01:52:06.501575+0000","last_became_active":"2023-12-01T01:52:06.499532+0000","last_became_peered":"2023-12-01T01:52:06.499532+0000","last_unstale":"2023-12-01T01:52:06.501575+0000","last_undegraded":"2023-12-01T01:52:06.501575+0000","last_fullsized":"2023-12-01T01:52:06.501575+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.479667+0000","last_change":"2023-12-01T01:52:06.475695+0000","last_active":"2023-12-01T01:52:06.479667+0000","last_peered":"2023-12-01T01:52:06.479667+0000","last_clean":"2023-12-01T01:52:06.479667+0000","last_became_active":"2023-12-01T01:52:06.475397+0000","last_became_peered":"2023-12-01T01:52:06.475397+0000","last_unstale":"2023-12-01T01:52:06.479667+0000","last_undegraded":"2023-12-01T01:52:06.479667+0000","last_fullsized":"2023-12-01T01:52:06.479667+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:52:50.864412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.499034+0000","last_change":"2023-12-01T01:52:06.495425+0000","last_active":"2023-12-01T01:52:06.499034+0000","last_peered":"2023-12-01T01:52:06.499034+0000","last_clean":"2023-12-01T01:52:06.499034+0000","last_became_active":"2023-12-01T01:52:06.495171+0000","last_became_peered":"2023-12-01T01:52:06.495171+0000","last_unstale":"2023-12-01T01:52:06.499034+0000","last_undegraded":"2023-12-01T01:52:06.499034+0000","last_fullsized":"2023-12-01T01:52:06.499034+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:19:57.884685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492130+0000","last_change":"2023-12-01T01:52:06.487903+0000","last_active":"2023-12-01T01:52:06.492130+0000","last_peered":"2023-12-01T01:52:06.492130+0000","last_clean":"2023-12-01T01:52:06.492130+0000","last_became_active":"2023-12-01T01:52:06.487482+0000","last_became_peered":"2023-12-01T01:52:06.487482+0000","last_unstale":"2023-12-01T01:52:06.492130+0000","last_undegraded":"2023-12-01T01:52:06.492130+0000","last_fullsized":"2023-12-01T01:52:06.492130+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:44.957677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485517+0000","last_change":"2023-12-01T01:52:06.482106+0000","last_active":"2023-12-01T01:52:06.485517+0000","last_peered":"2023-12-01T01:52:06.485517+0000","last_clean":"2023-12-01T01:52:06.485517+0000","last_became_active":"2023-12-01T01:52:06.480984+0000","last_became_peered":"2023-12-01T01:52:06.480984+0000","last_unstale":"2023-12-01T01:52:06.485517+0000","last_undegraded":"2023-12-01T01:52:06.485517+0000","last_fullsized":"2023-12-01T01:52:06.485517+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.499869+0000","last_change":"2023-12-01T01:52:06.496643+0000","last_active":"2023-12-01T01:52:06.499869+0000","last_peered":"2023-12-01T01:52:06.499869+0000","last_clean":"2023-12-01T01:52:06.499869+0000","last_became_active":"2023-12-01T01:52:06.496359+0000","last_became_peered":"2023-12-01T01:52:06.496359+0000","last_unstale":"2023-12-01T01:52:06.499869+0000","last_undegraded":"2023-12-01T01:52:06.499869+0000","last_fullsized":"2023-12-01T01:52:06.499869+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:58:13.647534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.490540+0000","last_change":"2023-12-01T01:52:06.485229+0000","last_active":"2023-12-01T01:52:06.490540+0000","last_peered":"2023-12-01T01:52:06.490540+0000","last_clean":"2023-12-01T01:52:06.490540+0000","last_became_active":"2023-12-01T01:52:06.484939+0000","last_became_peered":"2023-12-01T01:52:06.484939+0000","last_unstale":"2023-12-01T01:52:06.490540+0000","last_undegraded":"2023-12-01T01:52:06.490540+0000","last_fullsized":"2023-12-01T01:52:06.490540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491177+0000","last_change":"2023-12-01T01:52:06.480235+0000","last_active":"2023-12-01T01:52:06.491177+0000","last_peered":"2023-12-01T01:52:06.491177+0000","last_clean":"2023-12-01T01:52:06.491177+0000","last_became_active":"2023-12-01T01:52:06.478382+0000","last_became_peered":"2023-12-01T01:52:06.478382+0000","last_unstale":"2023-12-01T01:52:06.491177+0000","last_undegraded":"2023-12-01T01:52:06.491177+0000","last_fullsized":"2023-12-01T01:52:06.491177+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:03:27.876714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.487300+0000","last_change":"2023-12-01T01:52:06.485215+0000","last_active":"2023-12-01T01:52:06.487300+0000","last_peered":"2023-12-01T01:52:06.487300+0000","last_clean":"2023-12-01T01:52:06.487300+0000","last_became_active":"2023-12-01T01:52:06.484900+0000","last_became_peered":"2023-12-01T01:52:06.484900+0000","last_unstale":"2023-12-01T01:52:06.487300+0000","last_undegraded":"2023-12-01T01:52:06.487300+0000","last_fullsized":"2023-12-01T01:52:06.487300+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:22.669615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.489281+0000","last_change":"2023-12-01T01:52:06.487149+0000","last_active":"2023-12-01T01:52:06.489281+0000","last_peered":"2023-12-01T01:52:06.489281+0000","last_clean":"2023-12-01T01:52:06.489281+0000","last_became_active":"2023-12-01T01:52:06.486307+0000","last_became_peered":"2023-12-01T01:52:06.486307+0000","last_unstale":"2023-12-01T01:52:06.489281+0000","last_undegraded":"2023-12-01T01:52:06.489281+0000","last_fullsized":"2023-12-01T01:52:06.489281+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:46.317067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485892+0000","last_change":"2023-12-01T01:52:06.479432+0000","last_active":"2023-12-01T01:52:06.485892+0000","last_peered":"2023-12-01T01:52:06.485892+0000","last_clean":"2023-12-01T01:52:06.485892+0000","last_became_active":"2023-12-01T01:52:06.479162+0000","last_became_peered":"2023-12-01T01:52:06.479162+0000","last_unstale":"2023-12-01T01:52:06.485892+0000","last_undegraded":"2023-12-01T01:52:06.485892+0000","last_fullsized":"2023-12-01T01:52:06.485892+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:19.431492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.481232+0000","last_change":"2023-12-01T01:52:06.475996+0000","last_active":"2023-12-01T01:52:06.481232+0000","last_peered":"2023-12-01T01:52:06.481232+0000","last_clean":"2023-12-01T01:52:06.481232+0000","last_became_active":"2023-12-01T01:52:06.474002+0000","last_became_peered":"2023-12-01T01:52:06.474002+0000","last_unstale":"2023-12-01T01:52:06.481232+0000","last_undegraded":"2023-12-01T01:52:06.481232+0000","last_fullsized":"2023-12-01T01:52:06.481232+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:40.773157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.483645+0000","last_change":"2023-12-01T01:52:06.478718+0000","last_active":"2023-12-01T01:52:06.483645+0000","last_peered":"2023-12-01T01:52:06.483645+0000","last_clean":"2023-12-01T01:52:06.483645+0000","last_became_active":"2023-12-01T01:52:06.478398+0000","last_became_peered":"2023-12-01T01:52:06.478398+0000","last_unstale":"2023-12-01T01:52:06.483645+0000","last_undegraded":"2023-12-01T01:52:06.483645+0000","last_fullsized":"2023-12-01T01:52:06.483645+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:43:04.482263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.496516+0000","last_change":"2023-12-01T01:52:06.486443+0000","last_active":"2023-12-01T01:52:06.496516+0000","last_peered":"2023-12-01T01:52:06.496516+0000","last_clean":"2023-12-01T01:52:06.496516+0000","last_became_active":"2023-12-01T01:52:06.486194+0000","last_became_peered":"2023-12-01T01:52:06.486194+0000","last_unstale":"2023-12-01T01:52:06.496516+0000","last_undegraded":"2023-12-01T01:52:06.496516+0000","last_fullsized":"2023-12-01T01:52:06.496516+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:23.538863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.486240+0000","last_change":"2023-12-01T01:52:06.484596+0000","last_active":"2023-12-01T01:52:06.486240+0000","last_peered":"2023-12-01T01:52:06.486240+0000","last_clean":"2023-12-01T01:52:06.486240+0000","last_became_active":"2023-12-01T01:52:06.484190+0000","last_became_peered":"2023-12-01T01:52:06.484190+0000","last_unstale":"2023-12-01T01:52:06.486240+0000","last_undegraded":"2023-12-01T01:52:06.486240+0000","last_fullsized":"2023-12-01T01:52:06.486240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:07:56.031035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.478306+0000","last_change":"2023-12-01T01:52:06.475250+0000","last_active":"2023-12-01T01:52:06.478306+0000","last_peered":"2023-12-01T01:52:06.478306+0000","last_clean":"2023-12-01T01:52:06.478306+0000","last_became_active":"2023-12-01T01:52:06.474957+0000","last_became_peered":"2023-12-01T01:52:06.474957+0000","last_unstale":"2023-12-01T01:52:06.478306+0000","last_undegraded":"2023-12-01T01:52:06.478306+0000","last_fullsized":"2023-12-01T01:52:06.478306+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:11.391920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491672+0000","last_change":"2023-12-01T01:52:06.487363+0000","last_active":"2023-12-01T01:52:06.491672+0000","last_peered":"2023-12-01T01:52:06.491672+0000","last_clean":"2023-12-01T01:52:06.491672+0000","last_became_active":"2023-12-01T01:52:06.487091+0000","last_became_peered":"2023-12-01T01:52:06.487091+0000","last_unstale":"2023-12-01T01:52:06.491672+0000","last_undegraded":"2023-12-01T01:52:06.491672+0000","last_fullsized":"2023-12-01T01:52:06.491672+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.496723+0000","last_change":"2023-12-01T01:52:06.489054+0000","last_active":"2023-12-01T01:52:06.496723+0000","last_peered":"2023-12-01T01:52:06.496723+0000","last_clean":"2023-12-01T01:52:06.496723+0000","last_became_active":"2023-12-01T01:52:06.487822+0000","last_became_peered":"2023-12-01T01:52:06.487822+0000","last_unstale":"2023-12-01T01:52:06.496723+0000","last_undegraded":"2023-12-01T01:52:06.496723+0000","last_fullsized":"2023-12-01T01:52:06.496723+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.499013+0000","last_change":"2023-12-01T01:52:06.482346+0000","last_active":"2023-12-01T01:52:06.499013+0000","last_peered":"2023-12-01T01:52:06.499013+0000","last_clean":"2023-12-01T01:52:06.499013+0000","last_became_active":"2023-12-01T01:52:06.482104+0000","last_became_peered":"2023-12-01T01:52:06.482104+0000","last_unstale":"2023-12-01T01:52:06.499013+0000","last_undegraded":"2023-12-01T01:52:06.499013+0000","last_fullsized":"2023-12-01T01:52:06.499013+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:51:42.770614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.496802+0000","last_change":"2023-12-01T01:52:06.491310+0000","last_active":"2023-12-01T01:52:06.496802+0000","last_peered":"2023-12-01T01:52:06.496802+0000","last_clean":"2023-12-01T01:52:06.496802+0000","last_became_active":"2023-12-01T01:52:06.490864+0000","last_became_peered":"2023-12-01T01:52:06.490864+0000","last_unstale":"2023-12-01T01:52:06.496802+0000","last_undegraded":"2023-12-01T01:52:06.496802+0000","last_fullsized":"2023-12-01T01:52:06.496802+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:34:48.109363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497825+0000","last_change":"2023-12-01T01:52:06.481690+0000","last_active":"2023-12-01T01:52:06.497825+0000","last_peered":"2023-12-01T01:52:06.497825+0000","last_clean":"2023-12-01T01:52:06.497825+0000","last_became_active":"2023-12-01T01:52:06.481407+0000","last_became_peered":"2023-12-01T01:52:06.481407+0000","last_unstale":"2023-12-01T01:52:06.497825+0000","last_undegraded":"2023-12-01T01:52:06.497825+0000","last_fullsized":"2023-12-01T01:52:06.497825+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:23.123384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.481684+0000","last_change":"2023-12-01T01:52:06.477974+0000","last_active":"2023-12-01T01:52:06.481684+0000","last_peered":"2023-12-01T01:52:06.481684+0000","last_clean":"2023-12-01T01:52:06.481684+0000","last_became_active":"2023-12-01T01:52:06.477696+0000","last_became_peered":"2023-12-01T01:52:06.477696+0000","last_unstale":"2023-12-01T01:52:06.481684+0000","last_undegraded":"2023-12-01T01:52:06.481684+0000","last_fullsized":"2023-12-01T01:52:06.481684+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:50:15.604117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.489986+0000","last_change":"2023-12-01T01:52:06.484026+0000","last_active":"2023-12-01T01:52:06.489986+0000","last_peered":"2023-12-01T01:52:06.489986+0000","last_clean":"2023-12-01T01:52:06.489986+0000","last_became_active":"2023-12-01T01:52:06.483746+0000","last_became_peered":"2023-12-01T01:52:06.483746+0000","last_unstale":"2023-12-01T01:52:06.489986+0000","last_undegraded":"2023-12-01T01:52:06.489986+0000","last_fullsized":"2023-12-01T01:52:06.489986+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:24.016226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491196+0000","last_change":"2023-12-01T01:52:06.480783+0000","last_active":"2023-12-01T01:52:06.491196+0000","last_peered":"2023-12-01T01:52:06.491196+0000","last_clean":"2023-12-01T01:52:06.491196+0000","last_became_active":"2023-12-01T01:52:06.480328+0000","last_became_peered":"2023-12-01T01:52:06.480328+0000","last_unstale":"2023-12-01T01:52:06.491196+0000","last_undegraded":"2023-12-01T01:52:06.491196+0000","last_fullsized":"2023-12-01T01:52:06.491196+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:48.285382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.498229+0000","last_change":"2023-12-01T01:52:06.489459+0000","last_active":"2023-12-01T01:52:06.498229+0000","last_peered":"2023-12-01T01:52:06.498229+0000","last_clean":"2023-12-01T01:52:06.498229+0000","last_became_active":"2023-12-01T01:52:06.489171+0000","last_became_peered":"2023-12-01T01:52:06.489171+0000","last_unstale":"2023-12-01T01:52:06.498229+0000","last_undegraded":"2023-12-01T01:52:06.498229+0000","last_fullsized":"2023-12-01T01:52:06.498229+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:26.262966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492491+0000","last_change":"2023-12-01T01:52:06.485347+0000","last_active":"2023-12-01T01:52:06.492491+0000","last_peered":"2023-12-01T01:52:06.492491+0000","last_clean":"2023-12-01T01:52:06.492491+0000","last_became_active":"2023-12-01T01:52:06.484910+0000","last_became_peered":"2023-12-01T01:52:06.484910+0000","last_unstale":"2023-12-01T01:52:06.492491+0000","last_undegraded":"2023-12-01T01:52:06.492491+0000","last_fullsized":"2023-12-01T01:52:06.492491+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:38:19.366076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.490593+0000","last_change":"2023-12-01T01:52:06.483449+0000","last_active":"2023-12-01T01:52:06.490593+0000","last_peered":"2023-12-01T01:52:06.490593+0000","last_clean":"2023-12-01T01:52:06.490593+0000","last_became_active":"2023-12-01T01:52:06.483198+0000","last_became_peered":"2023-12-01T01:52:06.483198+0000","last_unstale":"2023-12-01T01:52:06.490593+0000","last_undegraded":"2023-12-01T01:52:06.490593+0000","last_fullsized":"2023-12-01T01:52:06.490593+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:02.272385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.500145+0000","last_change":"2023-12-01T01:52:06.498117+0000","last_active":"2023-12-01T01:52:06.500145+0000","last_peered":"2023-12-01T01:52:06.500145+0000","last_clean":"2023-12-01T01:52:06.500145+0000","last_became_active":"2023-12-01T01:52:06.497659+0000","last_became_peered":"2023-12-01T01:52:06.497659+0000","last_unstale":"2023-12-01T01:52:06.500145+0000","last_undegraded":"2023-12-01T01:52:06.500145+0000","last_fullsized":"2023-12-01T01:52:06.500145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:42:00.920100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"96'4","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497130+0000","last_change":"2023-12-01T01:52:05.999712+0000","last_active":"2023-12-01T01:52:06.497130+0000","last_peered":"2023-12-01T01:52:06.497130+0000","last_clean":"2023-12-01T01:52:06.497130+0000","last_became_active":"2023-12-01T01:52:05.999447+0000","last_became_peered":"2023-12-01T01:52:05.999447+0000","last_unstale":"2023-12-01T01:52:06.497130+0000","last_undegraded":"2023-12-01T01:52:06.497130+0000","last_fullsized":"2023-12-01T01:52:06.497130+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:51:43.565839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491769+0000","last_change":"2023-12-01T01:52:06.487733+0000","last_active":"2023-12-01T01:52:06.491769+0000","last_peered":"2023-12-01T01:52:06.491769+0000","last_clean":"2023-12-01T01:52:06.491769+0000","last_became_active":"2023-12-01T01:52:06.487144+0000","last_became_peered":"2023-12-01T01:52:06.487144+0000","last_unstale":"2023-12-01T01:52:06.491769+0000","last_undegraded":"2023-12-01T01:52:06.491769+0000","last_fullsized":"2023-12-01T01:52:06.491769+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:28:10.371917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"96'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.495474+0000","last_change":"2023-12-01T01:52:06.006120+0000","last_active":"2023-12-01T01:52:06.495474+0000","last_peered":"2023-12-01T01:52:06.495474+0000","last_clean":"2023-12-01T01:52:06.495474+0000","last_became_active":"2023-12-01T01:52:06.004701+0000","last_became_peered":"2023-12-01T01:52:06.004701+0000","last_unstale":"2023-12-01T01:52:06.495474+0000","last_undegraded":"2023-12-01T01:52:06.495474+0000","last_fullsized":"2023-12-01T01:52:06.495474+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:01.267746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492294+0000","last_change":"2023-12-01T01:52:06.489871+0000","last_active":"2023-12-01T01:52:06.492294+0000","last_peered":"2023-12-01T01:52:06.492294+0000","last_clean":"2023-12-01T01:52:06.492294+0000","last_became_active":"2023-12-01T01:52:06.489638+0000","last_became_peered":"2023-12-01T01:52:06.489638+0000","last_unstale":"2023-12-01T01:52:06.492294+0000","last_undegraded":"2023-12-01T01:52:06.492294+0000","last_fullsized":"2023-12-01T01:52:06.492294+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.503888+0000","last_change":"2023-12-01T01:52:05.999810+0000","last_active":"2023-12-01T01:52:06.503888+0000","last_peered":"2023-12-01T01:52:06.503888+0000","last_clean":"2023-12-01T01:52:06.503888+0000","last_became_active":"2023-12-01T01:52:05.999536+0000","last_became_peered":"2023-12-01T01:52:05.999536+0000","last_unstale":"2023-12-01T01:52:06.503888+0000","last_undegraded":"2023-12-01T01:52:06.503888+0000","last_fullsized":"2023-12-01T01:52:06.503888+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:38:18.361437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.487055+0000","last_change":"2023-12-01T01:52:06.483935+0000","last_active":"2023-12-01T01:52:06.487055+0000","last_peered":"2023-12-01T01:52:06.487055+0000","last_clean":"2023-12-01T01:52:06.487055+0000","last_became_active":"2023-12-01T01:52:06.482193+0000","last_became_peered":"2023-12-01T01:52:06.482193+0000","last_unstale":"2023-12-01T01:52:06.487055+0000","last_undegraded":"2023-12-01T01:52:06.487055+0000","last_fullsized":"2023-12-01T01:52:06.487055+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:42.937116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497177+0000","last_change":"2023-12-01T01:52:06.004649+0000","last_active":"2023-12-01T01:52:06.497177+0000","last_peered":"2023-12-01T01:52:06.497177+0000","last_clean":"2023-12-01T01:52:06.497177+0000","last_became_active":"2023-12-01T01:52:06.004356+0000","last_became_peered":"2023-12-01T01:52:06.004356+0000","last_unstale":"2023-12-01T01:52:06.497177+0000","last_undegraded":"2023-12-01T01:52:06.497177+0000","last_fullsized":"2023-12-01T01:52:06.497177+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.498445+0000","last_change":"2023-12-01T01:52:06.495979+0000","last_active":"2023-12-01T01:52:06.498445+0000","last_peered":"2023-12-01T01:52:06.498445+0000","last_clean":"2023-12-01T01:52:06.498445+0000","last_became_active":"2023-12-01T01:52:06.494890+0000","last_became_peered":"2023-12-01T01:52:06.494890+0000","last_unstale":"2023-12-01T01:52:06.498445+0000","last_undegraded":"2023-12-01T01:52:06.498445+0000","last_fullsized":"2023-12-01T01:52:06.498445+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:28.966506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"96'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497261+0000","last_change":"2023-12-01T01:52:05.998965+0000","last_active":"2023-12-01T01:52:06.497261+0000","last_peered":"2023-12-01T01:52:06.497261+0000","last_clean":"2023-12-01T01:52:06.497261+0000","last_became_active":"2023-12-01T01:52:05.997814+0000","last_became_peered":"2023-12-01T01:52:05.997814+0000","last_unstale":"2023-12-01T01:52:06.497261+0000","last_undegraded":"2023-12-01T01:52:06.497261+0000","last_fullsized":"2023-12-01T01:52:06.497261+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:06:24.991705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492331+0000","last_change":"2023-12-01T01:52:06.490530+0000","last_active":"2023-12-01T01:52:06.492331+0000","last_peered":"2023-12-01T01:52:06.492331+0000","last_clean":"2023-12-01T01:52:06.492331+0000","last_became_active":"2023-12-01T01:52:06.490291+0000","last_became_peered":"2023-12-01T01:52:06.490291+0000","last_unstale":"2023-12-01T01:52:06.492331+0000","last_undegraded":"2023-12-01T01:52:06.492331+0000","last_fullsized":"2023-12-01T01:52:06.492331+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"96'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.496988+0000","last_change":"2023-12-01T01:52:06.006338+0000","last_active":"2023-12-01T01:52:06.496988+0000","last_peered":"2023-12-01T01:52:06.496988+0000","last_clean":"2023-12-01T01:52:06.496988+0000","last_became_active":"2023-12-01T01:52:06.006077+0000","last_became_peered":"2023-12-01T01:52:06.006077+0000","last_unstale":"2023-12-01T01:52:06.496988+0000","last_undegraded":"2023-12-01T01:52:06.496988+0000","last_fullsized":"2023-12-01T01:52:06.496988+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:07:16.419472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485848+0000","last_change":"2023-12-01T01:52:06.483163+0000","last_active":"2023-12-01T01:52:06.485848+0000","last_peered":"2023-12-01T01:52:06.485848+0000","last_clean":"2023-12-01T01:52:06.485848+0000","last_became_active":"2023-12-01T01:52:06.481477+0000","last_became_peered":"2023-12-01T01:52:06.481477+0000","last_unstale":"2023-12-01T01:52:06.485848+0000","last_undegraded":"2023-12-01T01:52:06.485848+0000","last_fullsized":"2023-12-01T01:52:06.485848+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.477488+0000","last_change":"2023-12-01T01:52:06.001988+0000","last_active":"2023-12-01T01:52:06.477488+0000","last_peered":"2023-12-01T01:52:06.477488+0000","last_clean":"2023-12-01T01:52:06.477488+0000","last_became_active":"2023-12-01T01:52:06.001633+0000","last_became_peered":"2023-12-01T01:52:06.001633+0000","last_unstale":"2023-12-01T01:52:06.477488+0000","last_undegraded":"2023-12-01T01:52:06.477488+0000","last_fullsized":"2023-12-01T01:52:06.477488+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.489262+0000","last_change":"2023-12-01T01:52:06.486910+0000","last_active":"2023-12-01T01:52:06.489262+0000","last_peered":"2023-12-01T01:52:06.489262+0000","last_clean":"2023-12-01T01:52:06.489262+0000","last_became_active":"2023-12-01T01:52:06.485145+0000","last_became_peered":"2023-12-01T01:52:06.485145+0000","last_unstale":"2023-12-01T01:52:06.489262+0000","last_undegraded":"2023-12-01T01:52:06.489262+0000","last_fullsized":"2023-12-01T01:52:06.489262+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.476149+0000","last_change":"2023-12-01T01:52:06.005204+0000","last_active":"2023-12-01T01:52:06.476149+0000","last_peered":"2023-12-01T01:52:06.476149+0000","last_clean":"2023-12-01T01:52:06.476149+0000","last_became_active":"2023-12-01T01:52:06.004890+0000","last_became_peered":"2023-12-01T01:52:06.004890+0000","last_unstale":"2023-12-01T01:52:06.476149+0000","last_undegraded":"2023-12-01T01:52:06.476149+0000","last_fullsized":"2023-12-01T01:52:06.476149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497008+0000","last_change":"2023-12-01T01:52:06.492266+0000","last_active":"2023-12-01T01:52:06.497008+0000","last_peered":"2023-12-01T01:52:06.497008+0000","last_clean":"2023-12-01T01:52:06.497008+0000","last_became_active":"2023-12-01T01:52:06.491993+0000","last_became_peered":"2023-12-01T01:52:06.491993+0000","last_unstale":"2023-12-01T01:52:06.497008+0000","last_undegraded":"2023-12-01T01:52:06.497008+0000","last_fullsized":"2023-12-01T01:52:06.497008+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:06:21.919614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.476234+0000","last_change":"2023-12-01T01:52:06.005744+0000","last_active":"2023-12-01T01:52:06.476234+0000","last_peered":"2023-12-01T01:52:06.476234+0000","last_clean":"2023-12-01T01:52:06.476234+0000","last_became_active":"2023-12-01T01:52:06.005449+0000","last_became_peered":"2023-12-01T01:52:06.005449+0000","last_unstale":"2023-12-01T01:52:06.476234+0000","last_undegraded":"2023-12-01T01:52:06.476234+0000","last_fullsized":"2023-12-01T01:52:06.476234+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:23.011587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.490428+0000","last_change":"2023-12-01T01:52:06.480888+0000","last_active":"2023-12-01T01:52:06.490428+0000","last_peered":"2023-12-01T01:52:06.490428+0000","last_clean":"2023-12-01T01:52:06.490428+0000","last_became_active":"2023-12-01T01:52:06.480586+0000","last_became_peered":"2023-12-01T01:52:06.480586+0000","last_unstale":"2023-12-01T01:52:06.490428+0000","last_undegraded":"2023-12-01T01:52:06.490428+0000","last_fullsized":"2023-12-01T01:52:06.490428+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:00:34.778062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.467850+0000","last_change":"2023-12-01T01:52:06.001972+0000","last_active":"2023-12-01T01:52:06.467850+0000","last_peered":"2023-12-01T01:52:06.467850+0000","last_clean":"2023-12-01T01:52:06.467850+0000","last_became_active":"2023-12-01T01:52:05.996697+0000","last_became_peered":"2023-12-01T01:52:05.996697+0000","last_unstale":"2023-12-01T01:52:06.467850+0000","last_undegraded":"2023-12-01T01:52:06.467850+0000","last_fullsized":"2023-12-01T01:52:06.467850+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:21.909792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.493598+0000","last_change":"2023-12-01T01:52:06.481826+0000","last_active":"2023-12-01T01:52:06.493598+0000","last_peered":"2023-12-01T01:52:06.493598+0000","last_clean":"2023-12-01T01:52:06.493598+0000","last_became_active":"2023-12-01T01:52:06.480848+0000","last_became_peered":"2023-12-01T01:52:06.480848+0000","last_unstale":"2023-12-01T01:52:06.493598+0000","last_undegraded":"2023-12-01T01:52:06.493598+0000","last_fullsized":"2023-12-01T01:52:06.493598+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:17:44.214762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"96'3","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497853+0000","last_change":"2023-12-01T01:52:06.006483+0000","last_active":"2023-12-01T01:52:06.497853+0000","last_peered":"2023-12-01T01:52:06.497853+0000","last_clean":"2023-12-01T01:52:06.497853+0000","last_became_active":"2023-12-01T01:52:06.004318+0000","last_became_peered":"2023-12-01T01:52:06.004318+0000","last_unstale":"2023-12-01T01:52:06.497853+0000","last_undegraded":"2023-12-01T01:52:06.497853+0000","last_fullsized":"2023-12-01T01:52:06.497853+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:48.962691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.481665+0000","last_change":"2023-12-01T01:52:06.477011+0000","last_active":"2023-12-01T01:52:06.481665+0000","last_peered":"2023-12-01T01:52:06.481665+0000","last_clean":"2023-12-01T01:52:06.481665+0000","last_became_active":"2023-12-01T01:52:06.476673+0000","last_became_peered":"2023-12-01T01:52:06.476673+0000","last_unstale":"2023-12-01T01:52:06.481665+0000","last_undegraded":"2023-12-01T01:52:06.481665+0000","last_fullsized":"2023-12-01T01:52:06.481665+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:40.656152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"96'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.499960+0000","last_change":"2023-12-01T01:52:06.002932+0000","last_active":"2023-12-01T01:52:06.499960+0000","last_peered":"2023-12-01T01:52:06.499960+0000","last_clean":"2023-12-01T01:52:06.499960+0000","last_became_active":"2023-12-01T01:52:06.002600+0000","last_became_peered":"2023-12-01T01:52:06.002600+0000","last_unstale":"2023-12-01T01:52:06.499960+0000","last_undegraded":"2023-12-01T01:52:06.499960+0000","last_fullsized":"2023-12-01T01:52:06.499960+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491945+0000","last_change":"2023-12-01T01:52:06.488627+0000","last_active":"2023-12-01T01:52:06.491945+0000","last_peered":"2023-12-01T01:52:06.491945+0000","last_clean":"2023-12-01T01:52:06.491945+0000","last_became_active":"2023-12-01T01:52:06.488099+0000","last_became_peered":"2023-12-01T01:52:06.488099+0000","last_unstale":"2023-12-01T01:52:06.491945+0000","last_undegraded":"2023-12-01T01:52:06.491945+0000","last_fullsized":"2023-12-01T01:52:06.491945+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:29.801679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.498992+0000","last_change":"2023-12-01T01:52:06.007948+0000","last_active":"2023-12-01T01:52:06.498992+0000","last_peered":"2023-12-01T01:52:06.498992+0000","last_clean":"2023-12-01T01:52:06.498992+0000","last_became_active":"2023-12-01T01:52:06.007397+0000","last_became_peered":"2023-12-01T01:52:06.007397+0000","last_unstale":"2023-12-01T01:52:06.498992+0000","last_undegraded":"2023-12-01T01:52:06.498992+0000","last_fullsized":"2023-12-01T01:52:06.498992+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:59:33.535939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.481548+0000","last_change":"2023-12-01T01:52:06.479285+0000","last_active":"2023-12-01T01:52:06.481548+0000","last_peered":"2023-12-01T01:52:06.481548+0000","last_clean":"2023-12-01T01:52:06.481548+0000","last_became_active":"2023-12-01T01:52:06.478994+0000","last_became_peered":"2023-12-01T01:52:06.478994+0000","last_unstale":"2023-12-01T01:52:06.481548+0000","last_undegraded":"2023-12-01T01:52:06.481548+0000","last_fullsized":"2023-12-01T01:52:06.481548+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.469399+0000","last_change":"2023-12-01T01:52:06.009917+0000","last_active":"2023-12-01T01:52:06.469399+0000","last_peered":"2023-12-01T01:52:06.469399+0000","last_clean":"2023-12-01T01:52:06.469399+0000","last_became_active":"2023-12-01T01:52:06.009593+0000","last_became_peered":"2023-12-01T01:52:06.009593+0000","last_unstale":"2023-12-01T01:52:06.469399+0000","last_undegraded":"2023-12-01T01:52:06.469399+0000","last_fullsized":"2023-12-01T01:52:06.469399+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:11:33.399459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485481+0000","last_change":"2023-12-01T01:52:06.483989+0000","last_active":"2023-12-01T01:52:06.485481+0000","last_peered":"2023-12-01T01:52:06.485481+0000","last_clean":"2023-12-01T01:52:06.485481+0000","last_became_active":"2023-12-01T01:52:06.483660+0000","last_became_peered":"2023-12-01T01:52:06.483660+0000","last_unstale":"2023-12-01T01:52:06.485481+0000","last_undegraded":"2023-12-01T01:52:06.485481+0000","last_fullsized":"2023-12-01T01:52:06.485481+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:58.385658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.466654+0000","last_change":"2023-12-01T01:52:05.999943+0000","last_active":"2023-12-01T01:52:06.466654+0000","last_peered":"2023-12-01T01:52:06.466654+0000","last_clean":"2023-12-01T01:52:06.466654+0000","last_became_active":"2023-12-01T01:52:05.999649+0000","last_became_peered":"2023-12-01T01:52:05.999649+0000","last_unstale":"2023-12-01T01:52:06.466654+0000","last_undegraded":"2023-12-01T01:52:06.466654+0000","last_fullsized":"2023-12-01T01:52:06.466654+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:42.916412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.491715+0000","last_change":"2023-12-01T01:52:06.485967+0000","last_active":"2023-12-01T01:52:06.491715+0000","last_peered":"2023-12-01T01:52:06.491715+0000","last_clean":"2023-12-01T01:52:06.491715+0000","last_became_active":"2023-12-01T01:52:06.484050+0000","last_became_peered":"2023-12-01T01:52:06.484050+0000","last_unstale":"2023-12-01T01:52:06.491715+0000","last_undegraded":"2023-12-01T01:52:06.491715+0000","last_fullsized":"2023-12-01T01:52:06.491715+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:37:55.202441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"96'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.503344+0000","last_change":"2023-12-01T01:52:06.002123+0000","last_active":"2023-12-01T01:52:06.503344+0000","last_peered":"2023-12-01T01:52:06.503344+0000","last_clean":"2023-12-01T01:52:06.503344+0000","last_became_active":"2023-12-01T01:52:06.001181+0000","last_became_peered":"2023-12-01T01:52:06.001181+0000","last_unstale":"2023-12-01T01:52:06.503344+0000","last_undegraded":"2023-12-01T01:52:06.503344+0000","last_fullsized":"2023-12-01T01:52:06.503344+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:27:36.883717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.482562+0000","last_change":"2023-12-01T01:52:06.478607+0000","last_active":"2023-12-01T01:52:06.482562+0000","last_peered":"2023-12-01T01:52:06.482562+0000","last_clean":"2023-12-01T01:52:06.482562+0000","last_became_active":"2023-12-01T01:52:06.477495+0000","last_became_peered":"2023-12-01T01:52:06.477495+0000","last_unstale":"2023-12-01T01:52:06.482562+0000","last_undegraded":"2023-12-01T01:52:06.482562+0000","last_fullsized":"2023-12-01T01:52:06.482562+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"96'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.501969+0000","last_change":"2023-12-01T01:52:06.003482+0000","last_active":"2023-12-01T01:52:06.501969+0000","last_peered":"2023-12-01T01:52:06.501969+0000","last_clean":"2023-12-01T01:52:06.501969+0000","last_became_active":"2023-12-01T01:52:06.002884+0000","last_became_peered":"2023-12-01T01:52:06.002884+0000","last_unstale":"2023-12-01T01:52:06.501969+0000","last_undegraded":"2023-12-01T01:52:06.501969+0000","last_fullsized":"2023-12-01T01:52:06.501969+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:14:22.769057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497326+0000","last_change":"2023-12-01T01:52:06.490533+0000","last_active":"2023-12-01T01:52:06.497326+0000","last_peered":"2023-12-01T01:52:06.497326+0000","last_clean":"2023-12-01T01:52:06.497326+0000","last_became_active":"2023-12-01T01:52:06.490269+0000","last_became_peered":"2023-12-01T01:52:06.490269+0000","last_unstale":"2023-12-01T01:52:06.497326+0000","last_undegraded":"2023-12-01T01:52:06.497326+0000","last_fullsized":"2023-12-01T01:52:06.497326+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:03.240174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.497862+0000","last_change":"2023-12-01T01:52:05.997054+0000","last_active":"2023-12-01T01:52:06.497862+0000","last_peered":"2023-12-01T01:52:06.497862+0000","last_clean":"2023-12-01T01:52:06.497862+0000","last_became_active":"2023-12-01T01:52:05.996352+0000","last_became_peered":"2023-12-01T01:52:05.996352+0000","last_unstale":"2023-12-01T01:52:06.497862+0000","last_undegraded":"2023-12-01T01:52:06.497862+0000","last_fullsized":"2023-12-01T01:52:06.497862+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492708+0000","last_change":"2023-12-01T01:52:06.489678+0000","last_active":"2023-12-01T01:52:06.492708+0000","last_peered":"2023-12-01T01:52:06.492708+0000","last_clean":"2023-12-01T01:52:06.492708+0000","last_became_active":"2023-12-01T01:52:06.489332+0000","last_became_peered":"2023-12-01T01:52:06.489332+0000","last_unstale":"2023-12-01T01:52:06.492708+0000","last_undegraded":"2023-12-01T01:52:06.492708+0000","last_fullsized":"2023-12-01T01:52:06.492708+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:41.587746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"96'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.502003+0000","last_change":"2023-12-01T01:52:06.004398+0000","last_active":"2023-12-01T01:52:06.502003+0000","last_peered":"2023-12-01T01:52:06.502003+0000","last_clean":"2023-12-01T01:52:06.502003+0000","last_became_active":"2023-12-01T01:52:06.004055+0000","last_became_peered":"2023-12-01T01:52:06.004055+0000","last_unstale":"2023-12-01T01:52:06.502003+0000","last_undegraded":"2023-12-01T01:52:06.502003+0000","last_fullsized":"2023-12-01T01:52:06.502003+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:22.076125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492015+0000","last_change":"2023-12-01T01:52:06.485744+0000","last_active":"2023-12-01T01:52:06.492015+0000","last_peered":"2023-12-01T01:52:06.492015+0000","last_clean":"2023-12-01T01:52:06.492015+0000","last_became_active":"2023-12-01T01:52:06.482211+0000","last_became_peered":"2023-12-01T01:52:06.482211+0000","last_unstale":"2023-12-01T01:52:06.492015+0000","last_undegraded":"2023-12-01T01:52:06.492015+0000","last_fullsized":"2023-12-01T01:52:06.492015+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:15:14.116044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.481528+0000","last_change":"2023-12-01T01:52:06.003105+0000","last_active":"2023-12-01T01:52:06.481528+0000","last_peered":"2023-12-01T01:52:06.481528+0000","last_clean":"2023-12-01T01:52:06.481528+0000","last_became_active":"2023-12-01T01:52:06.001893+0000","last_became_peered":"2023-12-01T01:52:06.001893+0000","last_unstale":"2023-12-01T01:52:06.481528+0000","last_undegraded":"2023-12-01T01:52:06.481528+0000","last_fullsized":"2023-12-01T01:52:06.481528+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:27:15.489898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.482600+0000","last_change":"2023-12-01T01:52:06.479989+0000","last_active":"2023-12-01T01:52:06.482600+0000","last_peered":"2023-12-01T01:52:06.482600+0000","last_clean":"2023-12-01T01:52:06.482600+0000","last_became_active":"2023-12-01T01:52:06.476920+0000","last_became_peered":"2023-12-01T01:52:06.476920+0000","last_unstale":"2023-12-01T01:52:06.482600+0000","last_undegraded":"2023-12-01T01:52:06.482600+0000","last_fullsized":"2023-12-01T01:52:06.482600+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:26:45.696776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.479466+0000","last_change":"2023-12-01T01:52:05.990357+0000","last_active":"2023-12-01T01:52:06.479466+0000","last_peered":"2023-12-01T01:52:06.479466+0000","last_clean":"2023-12-01T01:52:06.479466+0000","last_became_active":"2023-12-01T01:52:05.988196+0000","last_became_peered":"2023-12-01T01:52:05.988196+0000","last_unstale":"2023-12-01T01:52:06.479466+0000","last_undegraded":"2023-12-01T01:52:06.479466+0000","last_fullsized":"2023-12-01T01:52:06.479466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:53.636284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492219+0000","last_change":"2023-12-01T01:52:06.489484+0000","last_active":"2023-12-01T01:52:06.492219+0000","last_peered":"2023-12-01T01:52:06.492219+0000","last_clean":"2023-12-01T01:52:06.492219+0000","last_became_active":"2023-12-01T01:52:06.489010+0000","last_became_peered":"2023-12-01T01:52:06.489010+0000","last_unstale":"2023-12-01T01:52:06.492219+0000","last_undegraded":"2023-12-01T01:52:06.492219+0000","last_fullsized":"2023-12-01T01:52:06.492219+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:44:54.842423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.469428+0000","last_change":"2023-12-01T01:52:05.990673+0000","last_active":"2023-12-01T01:52:06.469428+0000","last_peered":"2023-12-01T01:52:06.469428+0000","last_clean":"2023-12-01T01:52:06.469428+0000","last_became_active":"2023-12-01T01:52:05.989026+0000","last_became_peered":"2023-12-01T01:52:05.989026+0000","last_unstale":"2023-12-01T01:52:06.469428+0000","last_undegraded":"2023-12-01T01:52:06.469428+0000","last_fullsized":"2023-12-01T01:52:06.469428+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:51:43.565839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485001+0000","last_change":"2023-12-01T01:52:06.483031+0000","last_active":"2023-12-01T01:52:06.485001+0000","last_peered":"2023-12-01T01:52:06.485001+0000","last_clean":"2023-12-01T01:52:06.485001+0000","last_became_active":"2023-12-01T01:52:06.480967+0000","last_became_peered":"2023-12-01T01:52:06.480967+0000","last_unstale":"2023-12-01T01:52:06.485001+0000","last_undegraded":"2023-12-01T01:52:06.485001+0000","last_fullsized":"2023-12-01T01:52:06.485001+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:49.877947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.462983+0000","last_change":"2023-12-01T01:52:06.003576+0000","last_active":"2023-12-01T01:52:06.462983+0000","last_peered":"2023-12-01T01:52:06.462983+0000","last_clean":"2023-12-01T01:52:06.462983+0000","last_became_active":"2023-12-01T01:52:06.003254+0000","last_became_peered":"2023-12-01T01:52:06.003254+0000","last_unstale":"2023-12-01T01:52:06.462983+0000","last_undegraded":"2023-12-01T01:52:06.462983+0000","last_fullsized":"2023-12-01T01:52:06.462983+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:10.387281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.486039+0000","last_change":"2023-12-01T01:52:06.484497+0000","last_active":"2023-12-01T01:52:06.486039+0000","last_peered":"2023-12-01T01:52:06.486039+0000","last_clean":"2023-12-01T01:52:06.486039+0000","last_became_active":"2023-12-01T01:52:06.483710+0000","last_became_peered":"2023-12-01T01:52:06.483710+0000","last_unstale":"2023-12-01T01:52:06.486039+0000","last_undegraded":"2023-12-01T01:52:06.486039+0000","last_fullsized":"2023-12-01T01:52:06.486039+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:31.031923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.502109+0000","last_change":"2023-12-01T01:52:06.003659+0000","last_active":"2023-12-01T01:52:06.502109+0000","last_peered":"2023-12-01T01:52:06.502109+0000","last_clean":"2023-12-01T01:52:06.502109+0000","last_became_active":"2023-12-01T01:52:06.003228+0000","last_became_peered":"2023-12-01T01:52:06.003228+0000","last_unstale":"2023-12-01T01:52:06.502109+0000","last_undegraded":"2023-12-01T01:52:06.502109+0000","last_fullsized":"2023-12-01T01:52:06.502109+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:46:14.620804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.495897+0000","last_change":"2023-12-01T01:52:06.486528+0000","last_active":"2023-12-01T01:52:06.495897+0000","last_peered":"2023-12-01T01:52:06.495897+0000","last_clean":"2023-12-01T01:52:06.495897+0000","last_became_active":"2023-12-01T01:52:06.486273+0000","last_became_peered":"2023-12-01T01:52:06.486273+0000","last_unstale":"2023-12-01T01:52:06.495897+0000","last_undegraded":"2023-12-01T01:52:06.495897+0000","last_fullsized":"2023-12-01T01:52:06.495897+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:35:33.172589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.476790+0000","last_change":"2023-12-01T01:52:06.000889+0000","last_active":"2023-12-01T01:52:06.476790+0000","last_peered":"2023-12-01T01:52:06.476790+0000","last_clean":"2023-12-01T01:52:06.476790+0000","last_became_active":"2023-12-01T01:52:06.000644+0000","last_became_peered":"2023-12-01T01:52:06.000644+0000","last_unstale":"2023-12-01T01:52:06.476790+0000","last_undegraded":"2023-12-01T01:52:06.476790+0000","last_fullsized":"2023-12-01T01:52:06.476790+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:19:11.688232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.489967+0000","last_change":"2023-12-01T01:52:06.483516+0000","last_active":"2023-12-01T01:52:06.489967+0000","last_peered":"2023-12-01T01:52:06.489967+0000","last_clean":"2023-12-01T01:52:06.489967+0000","last_became_active":"2023-12-01T01:52:06.482601+0000","last_became_peered":"2023-12-01T01:52:06.482601+0000","last_unstale":"2023-12-01T01:52:06.489967+0000","last_undegraded":"2023-12-01T01:52:06.489967+0000","last_fullsized":"2023-12-01T01:52:06.489967+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:42:27.616705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.477607+0000","last_change":"2023-12-01T01:52:06.006527+0000","last_active":"2023-12-01T01:52:06.477607+0000","last_peered":"2023-12-01T01:52:06.477607+0000","last_clean":"2023-12-01T01:52:06.477607+0000","last_became_active":"2023-12-01T01:52:06.005261+0000","last_became_peered":"2023-12-01T01:52:06.005261+0000","last_unstale":"2023-12-01T01:52:06.477607+0000","last_undegraded":"2023-12-01T01:52:06.477607+0000","last_fullsized":"2023-12-01T01:52:06.477607+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:51:05.076663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.494244+0000","last_change":"2023-12-01T01:52:06.485932+0000","last_active":"2023-12-01T01:52:06.494244+0000","last_peered":"2023-12-01T01:52:06.494244+0000","last_clean":"2023-12-01T01:52:06.494244+0000","last_became_active":"2023-12-01T01:52:06.483597+0000","last_became_peered":"2023-12-01T01:52:06.483597+0000","last_unstale":"2023-12-01T01:52:06.494244+0000","last_undegraded":"2023-12-01T01:52:06.494244+0000","last_fullsized":"2023-12-01T01:52:06.494244+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:33:59.577488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.476504+0000","last_change":"2023-12-01T01:52:05.997953+0000","last_active":"2023-12-01T01:52:06.476504+0000","last_peered":"2023-12-01T01:52:06.476504+0000","last_clean":"2023-12-01T01:52:06.476504+0000","last_became_active":"2023-12-01T01:52:05.997697+0000","last_became_peered":"2023-12-01T01:52:05.997697+0000","last_unstale":"2023-12-01T01:52:06.476504+0000","last_undegraded":"2023-12-01T01:52:06.476504+0000","last_fullsized":"2023-12-01T01:52:06.476504+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:41:32.705632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.478779+0000","last_change":"2023-12-01T01:52:06.475303+0000","last_active":"2023-12-01T01:52:06.478779+0000","last_peered":"2023-12-01T01:52:06.478779+0000","last_clean":"2023-12-01T01:52:06.478779+0000","last_became_active":"2023-12-01T01:52:06.474510+0000","last_became_peered":"2023-12-01T01:52:06.474510+0000","last_unstale":"2023-12-01T01:52:06.478779+0000","last_undegraded":"2023-12-01T01:52:06.478779+0000","last_fullsized":"2023-12-01T01:52:06.478779+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:35:55.572050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.476598+0000","last_change":"2023-12-01T01:52:05.996908+0000","last_active":"2023-12-01T01:52:06.476598+0000","last_peered":"2023-12-01T01:52:06.476598+0000","last_clean":"2023-12-01T01:52:06.476598+0000","last_became_active":"2023-12-01T01:52:05.995875+0000","last_became_peered":"2023-12-01T01:52:05.995875+0000","last_unstale":"2023-12-01T01:52:06.476598+0000","last_undegraded":"2023-12-01T01:52:06.476598+0000","last_fullsized":"2023-12-01T01:52:06.476598+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:37.605903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485924+0000","last_change":"2023-12-01T01:52:06.479161+0000","last_active":"2023-12-01T01:52:06.485924+0000","last_peered":"2023-12-01T01:52:06.485924+0000","last_clean":"2023-12-01T01:52:06.485924+0000","last_became_active":"2023-12-01T01:52:06.478891+0000","last_became_peered":"2023-12-01T01:52:06.478891+0000","last_unstale":"2023-12-01T01:52:06.485924+0000","last_undegraded":"2023-12-01T01:52:06.485924+0000","last_fullsized":"2023-12-01T01:52:06.485924+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:49:44.833749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.463972+0000","last_change":"2023-12-01T01:52:06.003362+0000","last_active":"2023-12-01T01:52:06.463972+0000","last_peered":"2023-12-01T01:52:06.463972+0000","last_clean":"2023-12-01T01:52:06.463972+0000","last_became_active":"2023-12-01T01:52:06.003048+0000","last_became_peered":"2023-12-01T01:52:06.003048+0000","last_unstale":"2023-12-01T01:52:06.463972+0000","last_undegraded":"2023-12-01T01:52:06.463972+0000","last_fullsized":"2023-12-01T01:52:06.463972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:59:48.873308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.485703+0000","last_change":"2023-12-01T01:52:06.482989+0000","last_active":"2023-12-01T01:52:06.485703+0000","last_peered":"2023-12-01T01:52:06.485703+0000","last_clean":"2023-12-01T01:52:06.485703+0000","last_became_active":"2023-12-01T01:52:06.480990+0000","last_became_peered":"2023-12-01T01:52:06.480990+0000","last_unstale":"2023-12-01T01:52:06.485703+0000","last_undegraded":"2023-12-01T01:52:06.485703+0000","last_fullsized":"2023-12-01T01:52:06.485703+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:52.703856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.469520+0000","last_change":"2023-12-01T01:52:05.990037+0000","last_active":"2023-12-01T01:52:06.469520+0000","last_peered":"2023-12-01T01:52:06.469520+0000","last_clean":"2023-12-01T01:52:06.469520+0000","last_became_active":"2023-12-01T01:52:05.988261+0000","last_became_peered":"2023-12-01T01:52:05.988261+0000","last_unstale":"2023-12-01T01:52:06.469520+0000","last_undegraded":"2023-12-01T01:52:06.469520+0000","last_fullsized":"2023-12-01T01:52:06.469520+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:43.953038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.488961+0000","last_change":"2023-12-01T01:52:06.485149+0000","last_active":"2023-12-01T01:52:06.488961+0000","last_peered":"2023-12-01T01:52:06.488961+0000","last_clean":"2023-12-01T01:52:06.488961+0000","last_became_active":"2023-12-01T01:52:06.484566+0000","last_became_peered":"2023-12-01T01:52:06.484566+0000","last_unstale":"2023-12-01T01:52:06.488961+0000","last_undegraded":"2023-12-01T01:52:06.488961+0000","last_fullsized":"2023-12-01T01:52:06.488961+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:14:44.417471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.478518+0000","last_change":"2023-12-01T01:52:05.998968+0000","last_active":"2023-12-01T01:52:06.478518+0000","last_peered":"2023-12-01T01:52:06.478518+0000","last_clean":"2023-12-01T01:52:06.478518+0000","last_became_active":"2023-12-01T01:52:05.998647+0000","last_became_peered":"2023-12-01T01:52:05.998647+0000","last_unstale":"2023-12-01T01:52:06.478518+0000","last_undegraded":"2023-12-01T01:52:06.478518+0000","last_fullsized":"2023-12-01T01:52:06.478518+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:22.076125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.492485+0000","last_change":"2023-12-01T01:52:06.489260+0000","last_active":"2023-12-01T01:52:06.492485+0000","last_peered":"2023-12-01T01:52:06.492485+0000","last_clean":"2023-12-01T01:52:06.492485+0000","last_became_active":"2023-12-01T01:52:06.489006+0000","last_became_peered":"2023-12-01T01:52:06.489006+0000","last_unstale":"2023-12-01T01:52:06.492485+0000","last_undegraded":"2023-12-01T01:52:06.492485+0000","last_fullsized":"2023-12-01T01:52:06.492485+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:05.964277+0000","last_clean_scrub_stamp":"2023-12-01T01:52:05.964277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:12.910138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"96'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-01T01:52:06.498261+0000","last_change":"2023-12-01T01:52:06.000732+0000","last_active":"2023-12-01T01:52:06.498261+0000","last_peered":"2023-12-01T01:52:06.498261+0000","last_clean":"2023-12-01T01:52:06.498261+0000","last_became_active":"2023-12-01T01:52:06.000124+0000","last_became_peered":"2023-12-01T01:52:06.000124+0000","last_unstale":"2023-12-01T01:52:06.498261+0000","last_undegraded":"2023-12-01T01:52:06.498261+0000","last_fullsized":"2023-12-01T01:52:06.498261+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:04.959638+0000","last_clean_scrub_stamp":"2023-12-01T01:52:04.959638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:39.158199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"85'172","reported_seq":433,"reported_epoch":97,"state":"active+clean","last_fresh":"2023-12-01T01:52:08.635077+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:52:08.635077+0000","last_peered":"2023-12-01T01:52:08.635077+0000","last_clean":"2023-12-01T01:52:08.635077+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:52:08.635077+0000","last_undegraded":"2023-12-01T01:52:08.635077+0000","last_fullsized":"2023-12-01T01:52:08.635077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1130,"num_write":259,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1196032,"data_stored":563740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.0389999999999999,"15min":1.0640000000000001},"min":{"1min":0.65200000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2869999999999999,"5min":1.5449999999999999,"15min":1.8089999999999999},"last":1.401},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.05,"15min":1.0489999999999999},"min":{"1min":0.65600000000000003,"5min":0.64700000000000002,"15min":0.64000000000000001},"max":{"1min":1.2969999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.96499999999999997}]},{"osd":1,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.2010000000000001,"15min":1.2130000000000001},"min":{"1min":0.65700000000000003,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.351,"5min":1.8100000000000001,"15min":1.8420000000000001},"last":1.131},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2410000000000001,"15min":1.252},"min":{"1min":0.78900000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.629,"5min":1.744,"15min":2.0870000000000002},"last":1.4259999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.127,"15min":1.121},"min":{"1min":0.73299999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.607,"5min":1.728,"15min":1.728},"last":1.24},{"interface":"front","average":{"1min":1.175,"5min":1.208,"15min":1.2230000000000001},"min":{"1min":0.89700000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5489999999999999,"5min":1.9690000000000001,"15min":2.0209999999999999},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.2230000000000001,"15min":1.2310000000000001},"min":{"1min":0.75,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.3979999999999999,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":1.347},{"interface":"front","average":{"1min":1.337,"5min":1.3440000000000001,"15min":1.347},"min":{"1min":1.123,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6259999999999999,"5min":2.1880000000000002,"15min":2.7050000000000001},"last":1.5229999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3160000000000001,"15min":1.296},"min":{"1min":1.036,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5840000000000001,"5min":1.8979999999999999,"15min":2.2130000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4550000000000001,"15min":1.4139999999999999},"min":{"1min":0.96399999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7210000000000001,"5min":2.0939999999999999,"15min":2.601},"last":1.286}]},{"osd":5,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3169999999999999,"15min":1.2689999999999999},"min":{"1min":0.84899999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.4500000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.532,"15min":1.5209999999999999},"min":{"1min":1.147,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.802,"5min":2.2930000000000001,"15min":2.8069999999999999},"last":1.6319999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:51:30 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4119999999999999,"15min":1.383},"min":{"1min":1.0129999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.294,"5min":2.3420000000000001,"15min":2.3570000000000002},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4099999999999999,"15min":1.397},"min":{"1min":1.115,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7110000000000001,"5min":2.3500000000000001,"15min":2.5009999999999999},"last":1.4770000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1200128,"data_stored":563774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.091,"15min":1.103},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.55,"5min":1.635,"15min":1.79},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1399999999999999,"15min":1.127},"min":{"1min":0.77200000000000002,"5min":0.64400000000000002,"15min":0.61799999999999999},"max":{"1min":1.327,"5min":2.5569999999999999,"15min":2.73},"last":1.1559999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1639999999999999,"15min":1.1559999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.47,"5min":1.8620000000000001,"15min":2.2170000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1950000000000001,"15min":1.2050000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.494,"5min":1.6259999999999999,"15min":2.0920000000000001},"last":1.3600000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.214,"15min":1.1659999999999999},"min":{"1min":0.72299999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.4510000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.131,"5min":1.2450000000000001,"15min":1.24},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5549999999999999,"5min":2.137,"15min":2.137},"last":1.2050000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2869999999999999,"15min":1.2789999999999999},"min":{"1min":0.91600000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.4339999999999999,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.04},{"interface":"front","average":{"1min":1.365,"5min":1.4139999999999999,"15min":1.403},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.81299999999999994},"max":{"1min":1.5569999999999999,"5min":1.742,"15min":2.4020000000000001},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.298,"15min":1.28},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6830000000000001,"5min":2.226,"15min":2.3399999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.3460000000000001,"15min":1.347},"min":{"1min":0.93600000000000005,"5min":0.78100000000000003,"15min":0.69999999999999996},"max":{"1min":1.6060000000000001,"5min":1.931,"15min":2.6760000000000002},"last":1.655}]},{"osd":5,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4330000000000001,"15min":1.4099999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6779999999999999,"5min":2.1899999999999999,"15min":2.5539999999999998},"last":1.431},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.498,"15min":1.4750000000000001},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":0.748},"max":{"1min":1.786,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.5620000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.5389999999999999,"15min":1.514},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.89500000000000002},"max":{"1min":1.651,"5min":2.762,"15min":2.762},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.5960000000000001,"15min":1.5740000000000001},"min":{"1min":1.234,"5min":1.155,"15min":0.78700000000000003},"max":{"1min":1.7050000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.444}]}]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1781760,"data_stored":1152634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.1060000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.498,"5min":1.651,"15min":3.3980000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.127,"15min":1.141},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.60099999999999998},"max":{"1min":1.5720000000000001,"5min":1.6479999999999999,"15min":2.153},"last":1.7130000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.153,"15min":1.137},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.63200000000000001},"max":{"1min":1.4770000000000001,"5min":1.6879999999999999,"15min":1.7010000000000001},"last":1.228},{"interface":"front","average":{"1min":1.179,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.73599999999999999,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.26}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.2,"15min":1.1990000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4039999999999999,"5min":1.6559999999999999,"15min":1.9410000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.1850000000000001,"15min":1.224},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.6080000000000001,"5min":1.621,"15min":2.0630000000000002},"last":1.3939999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.258,"15min":1.286},"min":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.76900000000000002},"max":{"1min":1.5309999999999999,"5min":1.694,"15min":1.833},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3120000000000001,"15min":1.296},"min":{"1min":0.98099999999999998,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":1.6779999999999999,"5min":1.7110000000000001,"15min":2.1440000000000001},"last":1.5029999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.238,"15min":1.2390000000000001},"min":{"1min":0.86399999999999999,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.6779999999999999,"5min":1.853,"15min":2.5129999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.84999999999999998,"5min":0.80300000000000005,"15min":0.57299999999999995},"max":{"1min":1.6319999999999999,"5min":1.7869999999999999,"15min":2.0899999999999999},"last":1.5409999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2989999999999999,"15min":1.329},"min":{"1min":0.88500000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":2.2970000000000002},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.3919999999999999,"15min":1.389},"min":{"1min":1.125,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":1.962,"5min":1.962,"15min":3.23},"last":1.3560000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4530000000000001,"15min":1.48},"min":{"1min":0.92400000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2,"5min":2,"15min":3.012},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.4970000000000001,"15min":1.524},"min":{"1min":1.2689999999999999,"5min":0.873,"15min":0.873},"max":{"1min":2.073,"5min":2.073,"15min":2.8420000000000001},"last":1.6699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":1196032,"data_stored":562870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.119,"15min":1.1040000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.53,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.734,"5min":1.827,"15min":3.8069999999999999},"last":0.86199999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2350000000000001,"15min":1.2090000000000001},"min":{"1min":0.84599999999999997,"5min":0.81999999999999995,"15min":0.65900000000000003},"max":{"1min":1.528,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.1160000000000001,"15min":1.131},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.671,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2110000000000001,"15min":1.196},"min":{"1min":0.83999999999999997,"5min":0.78200000000000003,"15min":0.77600000000000002},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3140000000000001,"15min":1.286},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.871,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.97399999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.234},"min":{"1min":0.85399999999999998,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.8220000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.383},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3200000000000001,"15min":1.319},"min":{"1min":0.753,"5min":0.753,"15min":0.70399999999999996},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.3010000000000002},"last":1.1339999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.96599999999999997,"5min":0.79200000000000004,"15min":0.61899999999999999},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.1549999999999998},"last":1.087},{"interface":"front","average":{"1min":1.248,"5min":1.2829999999999999,"15min":1.292},"min":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.60199999999999998},"max":{"1min":1.5189999999999999,"5min":1.8029999999999999,"15min":2.2029999999999998},"last":1.272}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.94999999999999996,"5min":0.84799999999999998,"15min":0.59099999999999997},"max":{"1min":1.8089999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.5029999999999999,"15min":1.4730000000000001},"min":{"1min":1.1000000000000001,"5min":0.96699999999999997,"15min":0.63200000000000001},"max":{"1min":1.7949999999999999,"5min":2.4990000000000001,"15min":2.54},"last":1.2470000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.458,"15min":1.464},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.60599999999999998},"max":{"1min":1.6850000000000001,"5min":1.972,"15min":3.2789999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5720000000000001,"15min":1.5580000000000001},"min":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":0.64500000000000002},"max":{"1min":2.0059999999999998,"5min":2.04,"15min":2.157},"last":1.528}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40408,"kb_used_data":1168,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331432,"statfs":{"total":96636764160,"available":96595386368,"internally_reserved":0,"allocated":1196032,"data_stored":562302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81599999999999995,"15min":0.84799999999999998},"min":{"1min":0.501,"5min":0.42699999999999999,"15min":0.307},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.583},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97599999999999998,"15min":0.996},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.3160000000000001,"5min":1.476,"15min":1.9770000000000001},"last":0.53700000000000003}]},{"osd":1,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.052,"15min":1.0660000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.38900000000000001},"max":{"1min":1.423,"5min":1.571,"15min":2.0779999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.92200000000000004,"15min":0.94999999999999996},"min":{"1min":0.504,"5min":0.48199999999999998,"15min":0.40600000000000003},"max":{"1min":1.385,"5min":1.587,"15min":1.6519999999999999},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99399999999999999,"15min":1.0469999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.436},"max":{"1min":1.4810000000000001,"5min":1.512,"15min":1.766},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.163,"5min":1.167,"15min":1.1850000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.48399999999999999},"max":{"1min":1.522,"5min":1.647,"15min":1.661},"last":0.80200000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.212,"15min":1.2450000000000001},"min":{"1min":0.76900000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.4550000000000001,"5min":1.5820000000000001,"15min":2.9350000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2150000000000001,"15min":1.24},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.599,"5min":1.637,"15min":2.2389999999999999},"last":0.84999999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.24,"15min":1.29},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.63500000000000001},"max":{"1min":1.7669999999999999,"5min":1.806,"15min":3.476},"last":0.995},{"interface":"front","average":{"1min":1.244,"5min":1.266,"15min":1.3109999999999999},"min":{"1min":0.92100000000000004,"5min":0.82099999999999995,"15min":0.64200000000000002},"max":{"1min":1.681,"5min":1.9039999999999999,"15min":3.181},"last":0.97999999999999998}]},{"osd":6,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.363,"15min":1.377},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.68300000000000005},"max":{"1min":1.6040000000000001,"5min":1.8580000000000001,"15min":2.532},"last":1.198},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3919999999999999,"15min":1.4259999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.872},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":3.0249999999999999},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.389,"15min":1.421},"min":{"1min":1.0600000000000001,"5min":0.999,"15min":0.81499999999999995},"max":{"1min":1.998,"5min":1.998,"15min":2.1619999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.415,"15min":1.4610000000000001},"min":{"1min":0.91200000000000003,"5min":0.88300000000000001,"15min":0.83099999999999996},"max":{"1min":1.6879999999999999,"5min":1.8779999999999999,"15min":2.4590000000000001},"last":1.1080000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":562758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27100000000000002},"max":{"1min":1.0860000000000001,"5min":1.339,"15min":1.738},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.026,"15min":1.01},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.375},"max":{"1min":1.4159999999999999,"5min":1.5940000000000001,"15min":2.3159999999999998},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.96499999999999997,"15min":0.98999999999999999},"min":{"1min":0.54700000000000004,"5min":0.373,"15min":0.373},"max":{"1min":1.1779999999999999,"5min":1.45,"15min":1.7130000000000001},"last":0.629},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0720000000000001,"15min":1.0840000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.504},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97499999999999998,"15min":1.0029999999999999},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.377,"5min":1.625,"15min":1.883},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.58999999999999997,"5min":0.51600000000000001,"15min":0.39600000000000002},"max":{"1min":1.415,"5min":1.613,"15min":2.6080000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.331,"15min":1.3180000000000001},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.75},"max":{"1min":1.7230000000000001,"5min":1.9950000000000001,"15min":2.2829999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.246,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.964},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3009999999999999,"15min":1.319},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.647,"5min":1.804,"15min":2.5779999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.375,"5min":1.387,"15min":1.375},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.901,"5min":2.048,"15min":3.0099999999999998},"last":1.2490000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.3200000000000001},"min":{"1min":0.84299999999999997,"5min":0.83399999999999996,"15min":0.77500000000000002},"max":{"1min":1.6439999999999999,"5min":1.78,"15min":2.2799999999999998},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.766,"15min":2.5089999999999999},"last":1.181}]},{"osd":7,"last update":"Fri Dec 1 01:51:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.456,"15min":1.448},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.79400000000000004},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":2.448},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.67100000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7649999999999999,"5min":1.802,"15min":2.5049999999999999},"last":1.323}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59024,"kb_used_data":1736,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312816,"statfs":{"total":96636764160,"available":96576323584,"internally_reserved":0,"allocated":1777664,"data_stored":1152544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85199999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.318},"max":{"1min":1.341,"5min":1.5049999999999999,"15min":3.056},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.052,"5min":0.97999999999999998,"15min":1.016},"min":{"1min":0.50900000000000001,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":3.4129999999999998},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.88600000000000001,"15min":0.89600000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.36399999999999999},"max":{"1min":1.1919999999999999,"5min":1.54,"15min":3.3660000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0680000000000001,"15min":1.095},"min":{"1min":0.71199999999999997,"5min":0.47899999999999998,"15min":0.39000000000000001},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":3.4980000000000002},"last":1.23}]},{"osd":3,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.194,"15min":1.216},"min":{"1min":0.55400000000000005,"5min":0.51900000000000002,"15min":0.44},"max":{"1min":1.827,"5min":1.827,"15min":3.464},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.048,"15min":1.0529999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39700000000000002},"max":{"1min":1.657,"5min":1.6699999999999999,"15min":3.0960000000000001},"last":1.137}]},{"osd":4,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.175,"15min":1.1870000000000001},"min":{"1min":0.871,"5min":0.745,"15min":0.63800000000000001},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":3.7810000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3160000000000001,"15min":1.3440000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6559999999999999,"5min":1.8089999999999999,"15min":4.7270000000000003},"last":1.337}]},{"osd":5,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2829999999999999,"15min":1.2809999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6839999999999999,"5min":1.7190000000000001,"15min":3.2839999999999998},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.268,"15min":1.296},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":1.825,"15min":3.6669999999999998},"last":1.194}]},{"osd":6,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3089999999999999,"15min":1.3360000000000001},"min":{"1min":0.79400000000000004,"5min":0.71099999999999997,"15min":0.68400000000000005},"max":{"1min":1.7490000000000001,"5min":1.7809999999999999,"15min":3.3300000000000001},"last":1.381},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.3460000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5389999999999999,"5min":1.7490000000000001,"15min":3.7400000000000002},"last":1.29}]},{"osd":7,"last update":"Fri Dec 1 01:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3979999999999999,"15min":1.4419999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.774,"5min":1.8939999999999999,"15min":4.6749999999999998},"last":1.423},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.4159999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8440000000000001,"5min":1.992,"15min":3.5379999999999998},"last":1.4690000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575044,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1187840,"data_stored":562246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.87,"15min":0.86599999999999999},"min":{"1min":0.60699999999999998,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.3520000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.92600000000000005,"15min":0.96899999999999997},"min":{"1min":0.57899999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4390000000000001,"5min":1.5580000000000001,"15min":1.6910000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.98499999999999999,"15min":1.006},"min":{"1min":0.67600000000000005,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.4690000000000001,"5min":2.157,"15min":2.157},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.113,"5min":1.034,"15min":1.0760000000000001},"min":{"1min":0.84299999999999997,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.395,"5min":1.728,"15min":2.8159999999999998},"last":1.0820000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97999999999999998,"15min":1.024},"min":{"1min":0.58499999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.954},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.143,"15min":1.169},"min":{"1min":0.54900000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7070000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1279999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2749999999999999,"15min":1.3089999999999999},"min":{"1min":1.05,"5min":0.78500000000000003,"15min":0.65700000000000003},"max":{"1min":1.5289999999999999,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.196},{"interface":"front","average":{"1min":1.22,"5min":1.214,"15min":1.2629999999999999},"min":{"1min":0.83299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.391}]},{"osd":5,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.337,"15min":1.351},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.71899999999999997},"max":{"1min":1.839,"5min":2.351,"15min":2.351},"last":1.232},{"interface":"front","average":{"1min":1.276,"5min":1.306,"15min":1.349},"min":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6299999999999999,"5min":2.2909999999999999,"15min":2.7050000000000001},"last":1.3100000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.73099999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.157},"last":1.341},{"interface":"front","average":{"1min":1.379,"5min":1.3680000000000001,"15min":1.399},"min":{"1min":1.0129999999999999,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.292}]},{"osd":7,"last update":"Fri Dec 1 01:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.417},"min":{"1min":1.127,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.706,"5min":1.798,"15min":2.8999999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.361,"15min":1.407},"min":{"1min":1.1120000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.276}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:52:16.340 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T01:52:16.340 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T01:52:16.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.597+0000 7f1bc3cb9640 1 Processor -- start 2023-12-01T01:52:16.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.598+0000 7f1bc3cb9640 1 -- start start 2023-12-01T01:52:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.599+0000 7f1bc3cb9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1bbc15ced0 0x7f1bbc0b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.599+0000 7f1bc3cb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0b4d70 0x7f1bbc0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.599+0000 7f1bc3cb9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bbc0d1480 con 0x7f1bbc0d2750 2023-12-01T01:52:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.599+0000 7f1bc3cb9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bbc0d16b0 con 0x7f1bbc15ced0 2023-12-01T01:52:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.599+0000 7f1bc3cb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc0d1990 con 0x7f1bbc0b4d70 2023-12-01T01:52:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.600+0000 7f1bc2256640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0b4d70 0x7f1bbc0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.600+0000 7f1bc1a55640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1bbc15ced0 0x7f1bbc0b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.600+0000 7f1bc2256640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0b4d70 0x7f1bbc0d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:39160/0 (socket says 172.21.15.115:39160) 2023-12-01T01:52:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.600+0000 7f1bc1a55640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1bbc15ced0 0x7f1bbc0b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53670/0 (socket says 172.21.15.115:53670) 2023-12-01T01:52:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.600+0000 7f1bc2256640 1 -- 172.21.15.115:0/3734359891 learned_addr learned my addr 172.21.15.115:0/3734359891 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.601+0000 7f1bc1a55640 1 -- 172.21.15.115:0/3734359891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0b4d70 msgr2=0x7f1bbc0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.601+0000 7f1bc1a55640 1 --2- 172.21.15.115:0/3734359891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0b4d70 0x7f1bbc0d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.601+0000 7f1bc1a55640 1 -- 172.21.15.115:0/3734359891 >> v1:172.21.15.142:6789/0 conn(0x7f1bbc0d2750 legacy=0x7f1bbc0b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.601+0000 7f1bc1a55640 1 -- 172.21.15.115:0/3734359891 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc0d1c70 con 0x7f1bbc15ced0 2023-12-01T01:52:16.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.601+0000 7f1bc2256640 1 --2- 172.21.15.115:0/3734359891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0b4d70 0x7f1bbc0d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:52:16.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.602+0000 7f1bc1a55640 1 --2- 172.21.15.115:0/3734359891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc15ced0 0x7f1bbc0b8730 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009d90 tx=0x7f1bb002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d8fdc08292f5a87 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:16.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.602+0000 7f1bc0a53640 1 -- 172.21.15.115:0/3734359891 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb002cc50 con 0x7f1bbc15ced0 2023-12-01T01:52:16.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.603+0000 7f1bc0a53640 1 -- 172.21.15.115:0/3734359891 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb002cdb0 con 0x7f1bbc15ced0 2023-12-01T01:52:16.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.603+0000 7f1bc0a53640 1 -- 172.21.15.115:0/3734359891 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0034660 con 0x7f1bbc15ced0 2023-12-01T01:52:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.603+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/3734359891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc15ced0 msgr2=0x7f1bbc0b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.603+0000 7f1bc3cb9640 1 --2- 172.21.15.115:0/3734359891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc15ced0 0x7f1bbc0b8730 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009d90 tx=0x7f1bb002bea0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.603+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/3734359891 shutdown_connections 2023-12-01T01:52:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.603+0000 7f1bc3cb9640 1 --2- 172.21.15.115:0/3734359891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc0b4d70 0x7f1bbc0d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.603+0000 7f1bc3cb9640 1 --2- 172.21.15.115:0/3734359891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc15ced0 0x7f1bbc0b8730 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.603+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/3734359891 >> 172.21.15.115:0/3734359891 conn(0x7f1bbc065f90 msgr2=0x7f1bbc0cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.603+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/3734359891 shutdown_connections 2023-12-01T01:52:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.603+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/3734359891 wait complete. 2023-12-01T01:52:16.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.605+0000 7f1bc3cb9640 1 Processor -- start 2023-12-01T01:52:16.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.605+0000 7f1bc3cb9640 1 -- start start 2023-12-01T01:52:16.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.606+0000 7f1bc3cb9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc0b4d70 0x7f1bbc1f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.606+0000 7f1bc3cb9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bbc0d2750 0x7f1bbc14a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.606+0000 7f1bc3cb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc15ced0 0x7f1bbc14b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.606+0000 7f1bc3cb9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc14b200 con 0x7f1bbc0d2750 2023-12-01T01:52:16.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.606+0000 7f1bc3cb9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc153020 con 0x7f1bbc0b4d70 2023-12-01T01:52:16.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.606+0000 7f1bc3cb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc1f35d0 con 0x7f1bbc15ced0 2023-12-01T01:52:16.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc1254640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bbc0d2750 0x7f1bbc14a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc1254640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bbc0d2750 0x7f1bbc14a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47044/0 (socket says 172.21.15.115:47044) 2023-12-01T01:52:16.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc1254640 1 -- 172.21.15.115:0/791866285 learned_addr learned my addr 172.21.15.115:0/791866285 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:16.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc1a55640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc0b4d70 0x7f1bbc1f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc2256640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc15ced0 0x7f1bbc14b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc1254640 1 -- 172.21.15.115:0/791866285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc15ced0 msgr2=0x7f1bbc14b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc1254640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc15ced0 0x7f1bbc14b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc1254640 1 -- 172.21.15.115:0/791866285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc0b4d70 msgr2=0x7f1bbc1f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc1254640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc0b4d70 0x7f1bbc1f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc1254640 1 -- 172.21.15.115:0/791866285 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40094d0 con 0x7f1bbc0d2750 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc1a55640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc0b4d70 0x7f1bbc1f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.607+0000 7f1bc2256640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc15ced0 0x7f1bbc14b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.608+0000 7f1bc1254640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bbc0d2750 0x7f1bbc14a080 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c480 tx=0x7f1ba800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.608+0000 7f1bba7fc640 1 -- 172.21.15.115:0/791866285 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80052d0 con 0x7f1bbc0d2750 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.608+0000 7f1bba7fc640 1 -- 172.21.15.115:0/791866285 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba8005430 con 0x7f1bbc0d2750 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.608+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/791866285 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb0009a70 con 0x7f1bbc0d2750 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.608+0000 7f1bba7fc640 1 -- 172.21.15.115:0/791866285 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8012870 con 0x7f1bbc0d2750 2023-12-01T01:52:16.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.608+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/791866285 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc14c440 con 0x7f1bbc0d2750 2023-12-01T01:52:16.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.609+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/791866285 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bbc04e890 con 0x7f1bbc0d2750 2023-12-01T01:52:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.611+0000 7f1bba7fc640 1 -- 172.21.15.115:0/791866285 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1ba80129d0 con 0x7f1bbc0d2750 2023-12-01T01:52:16.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.612+0000 7f1bba7fc640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ba006a550 0x7f1ba006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:16.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.612+0000 7f1bba7fc640 1 -- 172.21.15.115:0/791866285 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ba80885a0 con 0x7f1bbc0d2750 2023-12-01T01:52:16.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.612+0000 7f1bc1a55640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ba006a550 0x7f1ba006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:16.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.613+0000 7f1bc1a55640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ba006a550 0x7f1ba006ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0036000 tx=0x7f1bb0030000 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:16.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.618+0000 7f1bba7fc640 1 -- 172.21.15.115:0/791866285 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1ba8053b10 con 0x7f1bbc0d2750 2023-12-01T01:52:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.881+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/791866285 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1bbc0b6f70 con 0x7f1bbc0d2750 2023-12-01T01:52:16.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.887+0000 7f1bba7fc640 1 -- 172.21.15.115:0/791866285 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1ba8057130 con 0x7f1bbc0d2750 2023-12-01T01:52:16.889 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T01:52:16.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/791866285 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ba006a550 msgr2=0x7f1ba006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ba006a550 0x7f1ba006ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0036000 tx=0x7f1bb0030000 comp rx=0 tx=0).stop 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/791866285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bbc0d2750 msgr2=0x7f1bbc14a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bbc0d2750 0x7f1bbc14a080 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c480 tx=0x7f1ba800c940 comp rx=0 tx=0).stop 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/791866285 shutdown_connections 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ba006a550 0x7f1ba006ca10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bbc15ced0 0x7f1bbc14b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1bbc0d2750 0x7f1bbc14a080 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 --2- 172.21.15.115:0/791866285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bbc0b4d70 0x7f1bbc1f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/791866285 >> 172.21.15.115:0/791866285 conn(0x7f1bbc065f90 msgr2=0x7f1bbc0cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/791866285 shutdown_connections 2023-12-01T01:52:16.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:16.890+0000 7f1bc3cb9640 1 -- 172.21.15.115:0/791866285 wait complete. 2023-12-01T01:52:16.910 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:52:17.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.119+0000 7f59fca5f640 1 Processor -- start 2023-12-01T01:52:17.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.119+0000 7f59fca5f640 1 -- start start 2023-12-01T01:52:17.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.121+0000 7f59fca5f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f59f80c9ef0 0x7f59f806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:17.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.121+0000 7f59fca5f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f806f8b0 con 0x7f59f806b6f0 2023-12-01T01:52:17.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.121+0000 7f59fca5f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f80adcb0 con 0x7f59f8159dd0 2023-12-01T01:52:17.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.121+0000 7f59fca5f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f80ad9d0 con 0x7f59f80c9ef0 2023-12-01T01:52:17.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.121+0000 7f59f6d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f59f806b6f0 0x7f59f8070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35664/0 (socket says 172.21.15.115:35664) 2023-12-01T01:52:17.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.121+0000 7f59f6d9d640 1 -- 172.21.15.115:0/4157071175 learned_addr learned my addr 172.21.15.115:0/4157071175 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:17.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f5d9b640 1 --2- 172.21.15.115:0/4157071175 >> v2:172.21.15.142:3300/0 conn(0x7f59f80c9ef0 0x7f59f806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:17.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263957701 0 0) 0x7f59f806f8b0 con 0x7f59f806b6f0 2023-12-01T01:52:17.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59e4005730 con 0x7f59f806b6f0 2023-12-01T01:52:17.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 827119813 0 0) 0x7f59e4005730 con 0x7f59f806b6f0 2023-12-01T01:52:17.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 >> v2:172.21.15.142:3300/0 conn(0x7f59f80c9ef0 msgr2=0x7f59f806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:17.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 --2- 172.21.15.115:0/4157071175 >> v2:172.21.15.142:3300/0 conn(0x7f59f80c9ef0 0x7f59f806fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:17.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 >> v1:172.21.15.142:6789/0 conn(0x7f59f8159dd0 legacy=0x7f59f80c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:17.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f80c9000 con 0x7f59f806b6f0 2023-12-01T01:52:17.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f59f00019f0 con 0x7f59f806b6f0 2023-12-01T01:52:17.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 >> v1:172.21.15.115:6789/0 conn(0x7f59f806b6f0 legacy=0x7f59f8070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e4005bf0 0x7f59e4006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e40069d0 0x7f59e4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4007490 0x7f59e400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f80ad9d0 con 0x7f59e4007490 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e400e3d0 con 0x7f59e40069d0 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.122+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e400e590 con 0x7f59e4005bf0 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.123+0000 7f59f6d9d640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4007490 0x7f59e400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.123+0000 7f59f5d9b640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e40069d0 0x7f59e4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.123+0000 7f59f659c640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e4005bf0 0x7f59e4006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.123+0000 7f59f6d9d640 1 -- 172.21.15.115:0/4157071175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e4005bf0 msgr2=0x7f59e4006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.123+0000 7f59f6d9d640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e4005bf0 0x7f59e4006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.123+0000 7f59f6d9d640 1 -- 172.21.15.115:0/4157071175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e40069d0 msgr2=0x7f59e4006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.123+0000 7f59f6d9d640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e40069d0 0x7f59e4006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:17.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.123+0000 7f59f6d9d640 1 -- 172.21.15.115:0/4157071175 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e4005730 con 0x7f59e4007490 2023-12-01T01:52:17.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.123+0000 7f59f659c640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e4005bf0 0x7f59e4006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.124+0000 7f59f6d9d640 1 -- 172.21.15.115:0/4157071175 reap_dead start 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.124+0000 7f59f6d9d640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4007490 0x7f59e400db90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f59f000a830 tx=0x7f59e4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c17e9dd9bcedc8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.124+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f00072d0 con 0x7f59e4007490 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.124+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f002f070 con 0x7f59e4007490 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.125+0000 7f59f559a640 1 -- 172.21.15.115:0/4157071175 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f00076b0 con 0x7f59e4007490 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.125+0000 7f59fca5f640 1 -- 172.21.15.115:0/4157071175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4007490 msgr2=0x7f59e400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.125+0000 7f59fca5f640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4007490 0x7f59e400db90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f59f000a830 tx=0x7f59e4007a00 comp rx=0 tx=0).stop 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.125+0000 7f59fca5f640 1 -- 172.21.15.115:0/4157071175 shutdown_connections 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.125+0000 7f59fca5f640 1 --2- 172.21.15.115:0/4157071175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e4007490 0x7f59e400db90 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.125+0000 7f59fca5f640 1 -- 172.21.15.115:0/4157071175 >> 172.21.15.115:0/4157071175 conn(0x7f59f80ce880 msgr2=0x7f59f80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.125+0000 7f59fca5f640 1 -- 172.21.15.115:0/4157071175 shutdown_connections 2023-12-01T01:52:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.125+0000 7f59fca5f640 1 -- 172.21.15.115:0/4157071175 wait complete. 2023-12-01T01:52:17.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.127+0000 7f59fca5f640 1 Processor -- start 2023-12-01T01:52:17.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.127+0000 7f59fca5f640 1 -- start start 2023-12-01T01:52:17.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.128+0000 7f59fca5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e4007490 0x7f59f80b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:17.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.128+0000 7f59fca5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f806c110 0x7f59f81572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:17.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.128+0000 7f59fca5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59f815a4b0 0x7f59f80cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:17.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.128+0000 7f59fca5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f8157810 con 0x7f59f815a4b0 2023-12-01T01:52:17.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.128+0000 7f59fca5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f80c8c90 con 0x7f59f806c110 2023-12-01T01:52:17.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.128+0000 7f59fca5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f815ac20 con 0x7f59e4007490 2023-12-01T01:52:17.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.128+0000 7f59f6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59f815a4b0 0x7f59f80cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:17.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.128+0000 7f59f6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59f815a4b0 0x7f59f80cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47074/0 (socket says 172.21.15.115:47074) 2023-12-01T01:52:17.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.128+0000 7f59f6d9d640 1 -- 172.21.15.115:0/562429873 learned_addr learned my addr 172.21.15.115:0/562429873 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:17.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.129+0000 7f59f659c640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e4007490 0x7f59f80b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:17.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.129+0000 7f59f5d9b640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f806c110 0x7f59f81572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:17.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.129+0000 7f59f6d9d640 1 -- 172.21.15.115:0/562429873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e4007490 msgr2=0x7f59f80b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.129+0000 7f59f6d9d640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e4007490 0x7f59f80b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.129+0000 7f59f6d9d640 1 -- 172.21.15.115:0/562429873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f806c110 msgr2=0x7f59f81572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.129+0000 7f59f6d9d640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f806c110 0x7f59f81572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.129+0000 7f59f6d9d640 1 -- 172.21.15.115:0/562429873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f815af00 con 0x7f59f815a4b0 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.129+0000 7f59f6d9d640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59f815a4b0 0x7f59f80cd5f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f59f002cc40 tx=0x7f59f0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.129+0000 7f59eeffd640 1 -- 172.21.15.115:0/562429873 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f00019f0 con 0x7f59f815a4b0 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.130+0000 7f59eeffd640 1 -- 172.21.15.115:0/562429873 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f0004b60 con 0x7f59f815a4b0 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.130+0000 7f59fca5f640 1 -- 172.21.15.115:0/562429873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f80c6a00 con 0x7f59f815a4b0 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.130+0000 7f59eeffd640 1 -- 172.21.15.115:0/562429873 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0007190 con 0x7f59f815a4b0 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.130+0000 7f59fca5f640 1 -- 172.21.15.115:0/562429873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f815a930 con 0x7f59f815a4b0 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.131+0000 7f59fca5f640 1 -- 172.21.15.115:0/562429873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f80c6f20 con 0x7f59f815a4b0 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.132+0000 7f59eeffd640 1 -- 172.21.15.115:0/562429873 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f59f0031030 con 0x7f59f815a4b0 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.133+0000 7f59eeffd640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f59d806a580 0x7f59d806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:17.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.133+0000 7f59eeffd640 1 -- 172.21.15.115:0/562429873 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59f00a9de0 con 0x7f59f815a4b0 2023-12-01T01:52:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.133+0000 7f59f659c640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f59d806a580 0x7f59d806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.134+0000 7f59f659c640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f59d806a580 0x7f59d806ca40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59e80015b0 tx=0x7f59e8009040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:17.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.137+0000 7f59eeffd640 1 -- 172.21.15.115:0/562429873 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f59f80c6f20 con 0x7f59f815a4b0 2023-12-01T01:52:17.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.365+0000 7f59fca5f640 1 -- 172.21.15.115:0/562429873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59f80c6f20 con 0x7f59f815a4b0 2023-12-01T01:52:17.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.366+0000 7f59eeffd640 1 -- 172.21.15.115:0/562429873 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f59f80c6f20 con 0x7f59f815a4b0 2023-12-01T01:52:17.368 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:17.368 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/542800961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":542800961},{"type":"v1","addr":"172.21.15.142:6833","nonce":542800961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6942,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2035657957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2035657957},{"type":"v1","addr":"172.21.15.115:6835","nonce":2035657957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3644451216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3644451216},{"type":"v1","addr":"172.21.15.115:6837","nonce":3644451216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:06.454774+0000","modified":"2023-12-01T01:52:09.476985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6957":{"gid":6957,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1311626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1311626228},{"type":"v1","addr":"172.21.15.142:6835","nonce":1311626228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-01T01:52:17.368 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2023-12-01T01:52:17.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.368+0000 7f59fca5f640 1 -- 172.21.15.115:0/562429873 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f59d806a580 msgr2=0x7f59d806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:17.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.368+0000 7f59fca5f640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f59d806a580 0x7f59d806ca40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59e80015b0 tx=0x7f59e8009040 comp rx=0 tx=0).stop 2023-12-01T01:52:17.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.368+0000 7f59fca5f640 1 -- 172.21.15.115:0/562429873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59f815a4b0 msgr2=0x7f59f80cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:17.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.368+0000 7f59fca5f640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59f815a4b0 0x7f59f80cd5f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f59f002cc40 tx=0x7f59f0001d80 comp rx=0 tx=0).stop 2023-12-01T01:52:17.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.369+0000 7f59fca5f640 1 -- 172.21.15.115:0/562429873 shutdown_connections 2023-12-01T01:52:17.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.369+0000 7f59fca5f640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f59d806a580 0x7f59d806ca40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:17.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.369+0000 7f59fca5f640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59f815a4b0 0x7f59f80cd5f0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:17.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.369+0000 7f59fca5f640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f806c110 0x7f59f81572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:17.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.369+0000 7f59fca5f640 1 --2- 172.21.15.115:0/562429873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e4007490 0x7f59f80b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:17.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.369+0000 7f59fca5f640 1 -- 172.21.15.115:0/562429873 >> 172.21.15.115:0/562429873 conn(0x7f59f80ce880 msgr2=0x7f59f806c830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:17.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.369+0000 7f59fca5f640 1 -- 172.21.15.115:0/562429873 shutdown_connections 2023-12-01T01:52:17.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:17.369+0000 7f59fca5f640 1 -- 172.21.15.115:0/562429873 wait complete. 2023-12-01T01:52:17.790 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T01:52:06.454774+0000', 'modified': '2023-12-01T01:52:09.476985+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6957}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6957': {'gid': 6957, 'name': 'b', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6835/1311626228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 1311626228}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 1311626228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T01:52:17.790 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T01:52:17.790 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-01T01:52:18.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.014+0000 7f2012f88640 1 Processor -- start 2023-12-01T01:52:18.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.014+0000 7f2012f88640 1 -- start start 2023-12-01T01:52:18.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.016+0000 7f2012f88640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f200c0701b0 0x7f200c070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.016+0000 7f2012f88640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f200c0cac50 0x7f200c0cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.016+0000 7f2012f88640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200c06da60 con 0x7f200c159af0 2023-12-01T01:52:18.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.016+0000 7f2012f88640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200c0cd9c0 con 0x7f200c0701b0 2023-12-01T01:52:18.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.016+0000 7f2012f88640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0cb600 con 0x7f200c0cac50 2023-12-01T01:52:18.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.016+0000 7f2011785640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f200c159af0 0x7f200c0cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:45918/0 (socket says 172.21.15.115:45918) 2023-12-01T01:52:18.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.016+0000 7f2011785640 1 -- 172.21.15.115:0/296327017 learned_addr learned my addr 172.21.15.115:0/296327017 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2012787640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f200c0cac50 0x7f200c0cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2011f86640 1 --2- 172.21.15.115:0/296327017 >> v2:172.21.15.142:3300/0 conn(0x7f200c0701b0 0x7f200c070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587014246 0 0) 0x7f200c06da60 con 0x7f200c159af0 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ff8005730 con 0x7f200c159af0 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1654662970 0 0) 0x7f1ff8005730 con 0x7f200c159af0 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f200c0cac50 msgr2=0x7f200c0cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f200c0cac50 0x7f200c0cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 >> v2:172.21.15.142:3300/0 conn(0x7f200c0701b0 msgr2=0x7f200c070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 --2- 172.21.15.115:0/296327017 >> v2:172.21.15.142:3300/0 conn(0x7f200c0701b0 0x7f200c070590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c070ad0 con 0x7f200c159af0 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f1ffc001670 con 0x7f200c159af0 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2012787640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f200c0cac50 0x7f200c0cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:52:18.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 >> v1:172.21.15.115:6789/0 conn(0x7f200c159af0 legacy=0x7f200c0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8005bf0 0x7f1ff8006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f200c0cac50 0x7f1ff80069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ff8006f10 0x7f1ff8007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0cd9c0 con 0x7f1ff8005bf0 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0cb600 con 0x7f1ff8006f10 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.017+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff800ff90 con 0x7f200c0cac50 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.018+0000 7f2011f86640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8005bf0 0x7f1ff8006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.018+0000 7f2011f86640 1 -- 172.21.15.115:0/296327017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f200c0cac50 msgr2=0x7f1ff80069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.018+0000 7f2011f86640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f200c0cac50 0x7f1ff80069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.018+0000 7f2011f86640 1 -- 172.21.15.115:0/296327017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ff8006f10 msgr2=0x7f1ff8007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.018+0000 7f2011f86640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ff8006f10 0x7f1ff8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.018+0000 7f2011f86640 1 -- 172.21.15.115:0/296327017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff8005730 con 0x7f1ff8005bf0 2023-12-01T01:52:18.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.019+0000 7f2011f86640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8005bf0 0x7f1ff8006010 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2008009ac0 tx=0x7f1ff8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8721fff475d07055 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:18.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.019+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200802c710 con 0x7f1ff8005bf0 2023-12-01T01:52:18.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.019+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200802c870 con 0x7f1ff8005bf0 2023-12-01T01:52:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.019+0000 7f2010f84640 1 -- 172.21.15.115:0/296327017 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200802cc00 con 0x7f1ff8005bf0 2023-12-01T01:52:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.020+0000 7f2012f88640 1 -- 172.21.15.115:0/296327017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8005bf0 msgr2=0x7f1ff8006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.020+0000 7f2012f88640 1 --2- 172.21.15.115:0/296327017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ff8005bf0 0x7f1ff8006010 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2008009ac0 tx=0x7f1ff8007a00 comp rx=0 tx=0).stop 2023-12-01T01:52:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.020+0000 7f2012787640 1 -- 172.21.15.115:0/296327017 reap_dead start 2023-12-01T01:52:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.020+0000 7f2012f88640 1 -- 172.21.15.115:0/296327017 shutdown_connections 2023-12-01T01:52:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.020+0000 7f2012f88640 1 -- 172.21.15.115:0/296327017 >> 172.21.15.115:0/296327017 conn(0x7f200c0cc160 msgr2=0x7f200c0cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.020+0000 7f2012f88640 1 -- 172.21.15.115:0/296327017 shutdown_connections 2023-12-01T01:52:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.020+0000 7f2012f88640 1 -- 172.21.15.115:0/296327017 wait complete. 2023-12-01T01:52:18.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.022+0000 7f2012f88640 1 Processor -- start 2023-12-01T01:52:18.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.022+0000 7f2012f88640 1 -- start start 2023-12-01T01:52:18.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.023+0000 7f2012f88640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ff8005bf0 0x7f200c0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.023+0000 7f2012f88640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200c1e28e0 0x7f200c1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.023+0000 7f2012f88640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c15af20 0x7f200c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.023+0000 7f2012f88640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0b40c0 con 0x7f200c1e28e0 2023-12-01T01:52:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.023+0000 7f2012f88640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0b4950 con 0x7f200c15af20 2023-12-01T01:52:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.023+0000 7f2012f88640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200c15b8c0 con 0x7f1ff8005bf0 2023-12-01T01:52:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.023+0000 7f2011785640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200c1e28e0 0x7f200c1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.023+0000 7f2011785640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200c1e28e0 0x7f200c1527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40394/0 (socket says 172.21.15.115:40394) 2023-12-01T01:52:18.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.023+0000 7f2011785640 1 -- 172.21.15.115:0/2773914980 learned_addr learned my addr 172.21.15.115:0/2773914980 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:18.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f2012787640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c15af20 0x7f200c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f2011f86640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ff8005bf0 0x7f200c0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f2011785640 1 -- 172.21.15.115:0/2773914980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ff8005bf0 msgr2=0x7f200c0b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f2011785640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ff8005bf0 0x7f200c0b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f2011785640 1 -- 172.21.15.115:0/2773914980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c15af20 msgr2=0x7f200c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f2011785640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c15af20 0x7f200c15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f2011785640 1 -- 172.21.15.115:0/2773914980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c1e2fe0 con 0x7f200c1e28e0 2023-12-01T01:52:18.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f2011785640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200c1e28e0 0x7f200c1527c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c650 tx=0x7f1ffc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:18.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f20027fc640 1 -- 172.21.15.115:0/2773914980 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0072d0 con 0x7f200c1e28e0 2023-12-01T01:52:18.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f20027fc640 1 -- 172.21.15.115:0/2773914980 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc007450 con 0x7f200c1e28e0 2023-12-01T01:52:18.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f20027fc640 1 -- 172.21.15.115:0/2773914980 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0047e0 con 0x7f200c1e28e0 2023-12-01T01:52:18.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f2012f88640 1 -- 172.21.15.115:0/2773914980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c1e2d60 con 0x7f200c1e28e0 2023-12-01T01:52:18.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.024+0000 7f2012f88640 1 -- 172.21.15.115:0/2773914980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200c0b4b40 con 0x7f200c1e28e0 2023-12-01T01:52:18.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.027+0000 7f20027fc640 1 -- 172.21.15.115:0/2773914980 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1ffc004960 con 0x7f200c1e28e0 2023-12-01T01:52:18.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.028+0000 7f20027fc640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ff006a550 0x7f1ff006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.028+0000 7f20027fc640 1 -- 172.21.15.115:0/2773914980 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ffc087de0 con 0x7f200c1e28e0 2023-12-01T01:52:18.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.028+0000 7f2011f86640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ff006a550 0x7f1ff006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.028+0000 7f2016e34740 1 Processor -- start 2023-12-01T01:52:18.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.028+0000 7f2016e34740 1 -- start start 2023-12-01T01:52:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.029+0000 7f2011f86640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ff006a550 0x7f1ff006ca10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2008036000 tx=0x7f2008030000 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.029+0000 7f2016e34740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2baee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.029+0000 7f2016e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28aa80 0x55635e2b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.029+0000 7f2016e34740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e28e680 0x55635e2b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.029+0000 7f2016e34740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635e2662f0 con 0x55635e2bab00 2023-12-01T01:52:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.029+0000 7f2016e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635e1faf80 con 0x55635e28aa80 2023-12-01T01:52:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.029+0000 7f2016e34740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55635e1f8710 con 0x55635e28e680 2023-12-01T01:52:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.030+0000 7f2011f86640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2baee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.030+0000 7f2011f86640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2baee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40402/0 (socket says 172.21.15.115:40402) 2023-12-01T01:52:18.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.030+0000 7f2011f86640 1 -- 172.21.15.115:0/1690244134 learned_addr learned my addr 172.21.15.115:0/1690244134 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.030+0000 7f2011f86640 1 -- 172.21.15.115:0/1690244134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e28e680 msgr2=0x55635e2b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.030+0000 7f2011f86640 1 --2- 172.21.15.115:0/1690244134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e28e680 0x55635e2b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.030+0000 7f2011785640 1 --2- 172.21.15.115:0/1690244134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28aa80 0x55635e2b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.030+0000 7f2011f86640 1 -- 172.21.15.115:0/1690244134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28aa80 msgr2=0x55635e2b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.030+0000 7f2011f86640 1 --2- 172.21.15.115:0/1690244134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28aa80 0x55635e2b7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.030+0000 7f2011f86640 1 -- 172.21.15.115:0/1690244134 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55635e1bb690 con 0x55635e2bab00 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.030+0000 7f2011f86640 1 --2- 172.21.15.115:0/1690244134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2baee0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2008009be0 tx=0x7f200802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e68a1aa74f9adb0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f1fef7fe640 1 -- 172.21.15.115:0/1690244134 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080044c0 con 0x55635e2bab00 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f1fef7fe640 1 -- 172.21.15.115:0/1690244134 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2008003610 con 0x55635e2bab00 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f1fef7fe640 1 -- 172.21.15.115:0/1690244134 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008003980 con 0x55635e2bab00 2023-12-01T01:52:18.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f2016e34740 1 -- 172.21.15.115:0/1690244134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 msgr2=0x55635e2baee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f2016e34740 1 --2- 172.21.15.115:0/1690244134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2baee0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2008009be0 tx=0x7f200802e000 comp rx=0 tx=0).stop 2023-12-01T01:52:18.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f2016e34740 1 -- 172.21.15.115:0/1690244134 shutdown_connections 2023-12-01T01:52:18.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f2016e34740 1 --2- 172.21.15.115:0/1690244134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e28e680 0x55635e2b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f2016e34740 1 --2- 172.21.15.115:0/1690244134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28aa80 0x55635e2b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f2016e34740 1 --2- 172.21.15.115:0/1690244134 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2baee0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f2016e34740 1 -- 172.21.15.115:0/1690244134 >> 172.21.15.115:0/1690244134 conn(0x55635e28cd90 msgr2=0x55635e2d5910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:18.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f2016e34740 1 -- 172.21.15.115:0/1690244134 shutdown_connections 2023-12-01T01:52:18.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.031+0000 7f2016e34740 1 -- 172.21.15.115:0/1690244134 wait complete. 2023-12-01T01:52:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.032+0000 7f2016e34740 1 -- start start 2023-12-01T01:52:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.033+0000 7f2016e34740 1 Processor -- start 2023-12-01T01:52:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2016e34740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e28e680 0x55635e2b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2016e34740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2016e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e2baee0 0x55635e2ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2016e34740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635e1f4000 con 0x55635e2bab00 2023-12-01T01:52:18.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2016e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635e1957d0 con 0x55635e2baee0 2023-12-01T01:52:18.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2016e34740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55635e195470 con 0x55635e28e680 2023-12-01T01:52:18.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2016e34740 10 client.0 _ll_register_callbacks cb 0x55635e1e41b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:52:18.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2011785640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2011785640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40418/0 (socket says 172.21.15.115:40418) 2023-12-01T01:52:18.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2011785640 1 -- 172.21.15.115:0/4039116966 learned_addr learned my addr 172.21.15.115:0/4039116966 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:18.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2011785640 1 -- 172.21.15.115:0/4039116966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e28e680 msgr2=0x55635e2b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2011785640 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e28e680 0x55635e2b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2011785640 1 -- 172.21.15.115:0/4039116966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e2baee0 msgr2=0x55635e2ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2011785640 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e2baee0 0x55635e2ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.034+0000 7f2011785640 1 -- 172.21.15.115:0/4039116966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff8005730 con 0x55635e2bab00 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f2011785640 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2adb20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc01f030 tx=0x7f1ffc057720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a678e83d7dcd523a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f1fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f1fecff9640 1 -- 172.21.15.115:0/4039116966 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0565d0 con 0x55635e2bab00 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f1fecff9640 1 -- 172.21.15.115:0/4039116966 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ffc05eec0 con 0x55635e2bab00 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f2016e34740 1 -- 172.21.15.115:0/4039116966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55635e21e550 con 0x55635e2bab00 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f1fecff9640 1 -- 172.21.15.115:0/4039116966 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ffc05ea30 con 0x55635e2bab00 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f2016e34740 10 client.7113 fetch_fsmap learned FSMap version 40 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f2016e34740 1 -- 172.21.15.115:0/4039116966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55635e1f2680 con 0x55635e2bab00 2023-12-01T01:52:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f1fecff9640 1 -- 172.21.15.115:0/4039116966 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55635e21e550 con 0x55635e2bab00 2023-12-01T01:52:18.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f1fecff9640 1 -- 172.21.15.115:0/4039116966 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f1ffc06e1b0 con 0x55635e2bab00 2023-12-01T01:52:18.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f2016e34740 10 client.7113 fetch_fsmap finished waiting for FSMap version 40 2023-12-01T01:52:18.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f2016e34740 10 client.7113 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-01T01:52:18.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f2016e34740 20 client.7113 populate_metadata read hostname 'smithi115' 2023-12-01T01:52:18.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f2016e34740 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.142:6834/1311626228,v1:172.21.15.142:6835/1311626228] conn(0x55635e2bcdf0 0x55635e2bd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f2016e34740 4 client.7113 mds_command: new command op to 6957 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:52:18.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.035+0000 7f2016e34740 1 -- 172.21.15.115:0/4039116966 --> [v2:172.21.15.142:6834/1311626228,v1:172.21.15.142:6835/1311626228] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55635e1e6b50 con 0x55635e2bcdf0 2023-12-01T01:52:18.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.036+0000 7f2011f86640 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.142:6834/1311626228,v1:172.21.15.142:6835/1311626228] conn(0x55635e2bcdf0 0x55635e2bd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.037+0000 7f2011f86640 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.142:6834/1311626228,v1:172.21.15.142:6835/1311626228] conn(0x55635e2bcdf0 0x55635e2bd1d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f9d5e166aae3fd server_cookie=95f173e8e61e21b3 in_seq=0 out_seq=0 2023-12-01T01:52:18.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.037+0000 7f1fecff9640 10 client.7113 ms_handle_connect on v2:172.21.15.142:6834/1311626228 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f1fecff9640 1 -- 172.21.15.115:0/4039116966 <== mds.0 v2:172.21.15.142:6834/1311626228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55635e1e6b50 con 0x55635e2bcdf0 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f1fecff9640 10 client.7113 handle_command_reply: tid=0 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 client.7113 shutdown 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 -- 172.21.15.115:0/4039116966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 msgr2=0x55635e2adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2adb20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc01f030 tx=0x7f1ffc057720 comp rx=0 tx=0).stop 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 -- 172.21.15.115:0/4039116966 shutdown_connections 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.142:6834/1311626228,v1:172.21.15.142:6835/1311626228] conn(0x55635e2bcdf0 0x55635e2bd1d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e2baee0 0x55635e2ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2bab00 0x55635e2adb20 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 --2- 172.21.15.115:0/4039116966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e28e680 0x55635e2b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f1fecff9640 0 client.7113 ms_handle_reset on v2:172.21.15.142:6834/1311626228 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 -- 172.21.15.115:0/4039116966 >> 172.21.15.115:0/4039116966 conn(0x55635e28aa80 msgr2=0x55635e2be840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 -- 172.21.15.115:0/4039116966 shutdown_connections 2023-12-01T01:52:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 1 -- 172.21.15.115:0/4039116966 wait complete. 2023-12-01T01:52:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.039+0000 7f2016e34740 20 client.7113 trim_cache size 0 max 16384 2023-12-01T01:52:18.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.056+0000 7f2016e34740 1 Processor -- start 2023-12-01T01:52:18.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.056+0000 7f2016e34740 1 -- start start 2023-12-01T01:52:18.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2016e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28e680 0x55635e2b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2016e34740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e2bab00 0x55635e2bd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2016e34740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2baee0 0x55635e2bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2016e34740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635df5bf20 con 0x55635e2baee0 2023-12-01T01:52:18.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2016e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635e2af290 con 0x55635e28e680 2023-12-01T01:52:18.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2016e34740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55635e2d68d0 con 0x55635e2bab00 2023-12-01T01:52:18.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2012787640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2baee0 0x55635e2bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2012787640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2baee0 0x55635e2bb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40430/0 (socket says 172.21.15.115:40430) 2023-12-01T01:52:18.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2012787640 1 -- 172.21.15.115:0/2600686676 learned_addr learned my addr 172.21.15.115:0/2600686676 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2011f86640 1 --2- 172.21.15.115:0/2600686676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28e680 0x55635e2b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2012787640 1 -- 172.21.15.115:0/2600686676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e2bab00 msgr2=0x55635e2bd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2011785640 1 --2- 172.21.15.115:0/2600686676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e2bab00 0x55635e2bd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2012787640 1 --2- 172.21.15.115:0/2600686676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e2bab00 0x55635e2bd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2012787640 1 -- 172.21.15.115:0/2600686676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28e680 msgr2=0x55635e2b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2012787640 1 --2- 172.21.15.115:0/2600686676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28e680 0x55635e2b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2012787640 1 -- 172.21.15.115:0/2600686676 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55635e1949e0 con 0x55635e2baee0 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.057+0000 7f2011f86640 1 --2- 172.21.15.115:0/2600686676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28e680 0x55635e2b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.058+0000 7f2012787640 1 --2- 172.21.15.115:0/2600686676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2baee0 0x55635e2bb4b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f200c070d70 tx=0x7f2004007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9c4fb415fa31cb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.058+0000 7f1feffff640 1 -- 172.21.15.115:0/2600686676 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200400e050 con 0x55635e2baee0 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.058+0000 7f1feffff640 1 -- 172.21.15.115:0/2600686676 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200400f040 con 0x55635e2baee0 2023-12-01T01:52:18.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.058+0000 7f1feffff640 1 -- 172.21.15.115:0/2600686676 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20040148c0 con 0x55635e2baee0 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.059+0000 7f2016e34740 1 -- 172.21.15.115:0/2600686676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2baee0 msgr2=0x55635e2bb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.059+0000 7f2016e34740 1 --2- 172.21.15.115:0/2600686676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2baee0 0x55635e2bb4b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f200c070d70 tx=0x7f2004007700 comp rx=0 tx=0).stop 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.059+0000 7f2016e34740 1 -- 172.21.15.115:0/2600686676 shutdown_connections 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.059+0000 7f2016e34740 1 --2- 172.21.15.115:0/2600686676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e2bab00 0x55635e2bd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.059+0000 7f2016e34740 1 --2- 172.21.15.115:0/2600686676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e2baee0 0x55635e2bb4b0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.059+0000 7f2016e34740 1 --2- 172.21.15.115:0/2600686676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e28e680 0x55635e2b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.059+0000 7f2016e34740 1 -- 172.21.15.115:0/2600686676 >> 172.21.15.115:0/2600686676 conn(0x55635e28aa80 msgr2=0x55635e290cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.059+0000 7f2016e34740 1 -- 172.21.15.115:0/2600686676 shutdown_connections 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.059+0000 7f2016e34740 1 -- 172.21.15.115:0/2600686676 wait complete. 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.059+0000 7f2016e34740 1 -- start start 2023-12-01T01:52:18.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.060+0000 7f2016e34740 1 Processor -- start 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2016e34740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e28e680 0x55635e343610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2016e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e2bab00 0x55635e2bb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2016e34740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e2baee0 0x55635e2bb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2016e34740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635e2b7420 con 0x55635e28e680 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2016e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55635e345380 con 0x55635e2bab00 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2016e34740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55635e2d82c0 con 0x55635e2baee0 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2016e34740 10 client.0 _ll_register_callbacks cb 0x7f2008037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2011f86640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e28e680 0x55635e343610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2011f86640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e28e680 0x55635e343610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40434/0 (socket says 172.21.15.115:40434) 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2011f86640 1 -- 172.21.15.115:0/3656020924 learned_addr learned my addr 172.21.15.115:0/3656020924 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2011f86640 1 -- 172.21.15.115:0/3656020924 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e2baee0 msgr2=0x55635e2bb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2011f86640 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e2baee0 0x55635e2bb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2011f86640 1 -- 172.21.15.115:0/3656020924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e2bab00 msgr2=0x55635e2bb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2011f86640 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e2bab00 0x55635e2bb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.061+0000 7f2011f86640 1 -- 172.21.15.115:0/3656020924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2004009a20 con 0x55635e28e680 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f2011f86640 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e28e680 0x55635e343610 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2008009a90 tx=0x7f20080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7827110339017901 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f1fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f1fecff9640 1 -- 172.21.15.115:0/3656020924 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200805d070 con 0x55635e28e680 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f1fecff9640 1 -- 172.21.15.115:0/3656020924 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20080593f0 con 0x55635e28e680 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f2016e34740 1 -- 172.21.15.115:0/3656020924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2008003610 con 0x55635e28e680 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f1fecff9640 1 -- 172.21.15.115:0/3656020924 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2008055040 con 0x55635e28e680 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f2016e34740 10 client.7119 fetch_fsmap learned FSMap version 40 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f2016e34740 1 -- 172.21.15.115:0/3656020924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55635e1b9ef0 con 0x55635e28e680 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f1fecff9640 1 -- 172.21.15.115:0/3656020924 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2008003610 con 0x55635e28e680 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f1fecff9640 1 -- 172.21.15.115:0/3656020924 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 40) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2008058070 con 0x55635e28e680 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f2016e34740 10 client.7119 fetch_fsmap finished waiting for FSMap version 40 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f2016e34740 10 client.7119 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-01T01:52:18.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f2016e34740 20 client.7119 populate_metadata read hostname 'smithi115' 2023-12-01T01:52:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f2016e34740 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.142:6834/1311626228,v1:172.21.15.142:6835/1311626228] conn(0x55635e2ee050 0x55635e2ee450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.062+0000 7f2016e34740 4 client.7119 mds_command: new command op to 6957 tid=0[{"prefix": "status"}] 2023-12-01T01:52:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.063+0000 7f2016e34740 1 -- 172.21.15.115:0/3656020924 --> [v2:172.21.15.142:6834/1311626228,v1:172.21.15.142:6835/1311626228] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55635e1ed080 con 0x55635e2ee050 2023-12-01T01:52:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.063+0000 7f2011785640 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.142:6834/1311626228,v1:172.21.15.142:6835/1311626228] conn(0x55635e2ee050 0x55635e2ee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.064+0000 7f2011785640 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.142:6834/1311626228,v1:172.21.15.142:6835/1311626228] conn(0x55635e2ee050 0x55635e2ee450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed24f158f432141d server_cookie=92e9ea14aa6889aa in_seq=0 out_seq=0 2023-12-01T01:52:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.064+0000 7f1fecff9640 10 client.7119 ms_handle_connect on v2:172.21.15.142:6834/1311626228 2023-12-01T01:52:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f1fecff9640 1 -- 172.21.15.115:0/3656020924 <== mds.0 v2:172.21.15.142:6834/1311626228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55635e1ed080 con 0x55635e2ee050 2023-12-01T01:52:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f1fecff9640 10 client.7119 handle_command_reply: tid=0 2023-12-01T01:52:18.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 client.7119 shutdown 2023-12-01T01:52:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 -- 172.21.15.115:0/3656020924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e28e680 msgr2=0x55635e343610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e28e680 0x55635e343610 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2008009a90 tx=0x7f20080046c0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 -- 172.21.15.115:0/3656020924 shutdown_connections 2023-12-01T01:52:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.142:6834/1311626228,v1:172.21.15.142:6835/1311626228] conn(0x55635e2ee050 0x55635e2ee450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55635e2baee0 0x55635e2bb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f1fecff9640 0 client.7119 ms_handle_reset on v2:172.21.15.142:6834/1311626228 2023-12-01T01:52:18.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55635e2bab00 0x55635e2bb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 --2- 172.21.15.115:0/3656020924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55635e28e680 0x55635e343610 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 -- 172.21.15.115:0/3656020924 >> 172.21.15.115:0/3656020924 conn(0x55635e28aa80 msgr2=0x55635e2b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 -- 172.21.15.115:0/3656020924 shutdown_connections 2023-12-01T01:52:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 1 -- 172.21.15.115:0/3656020924 wait complete. 2023-12-01T01:52:18.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.065+0000 7f2016e34740 20 client.7119 trim_cache size 0 max 16384 2023-12-01T01:52:18.070 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:52:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T01:52:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T01:52:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 6957, 2023-12-01T01:52:18.070 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T01:52:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T01:52:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T01:52:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 11.605446713999999, 2023-12-01T01:52:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 40, 2023-12-01T01:52:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 97, 2023-12-01T01:52:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 97, 2023-12-01T01:52:18.071 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 16.527914008 2023-12-01T01:52:18.071 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:52:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 -- 172.21.15.115:0/2773914980 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ff006a550 msgr2=0x7f1ff006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ff006a550 0x7f1ff006ca10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2008036000 tx=0x7f2008030000 comp rx=0 tx=0).stop 2023-12-01T01:52:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 -- 172.21.15.115:0/2773914980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200c1e28e0 msgr2=0x7f200c1527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200c1e28e0 0x7f200c1527c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c650 tx=0x7f1ffc00cb10 comp rx=0 tx=0).stop 2023-12-01T01:52:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 -- 172.21.15.115:0/2773914980 shutdown_connections 2023-12-01T01:52:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ff006a550 0x7f1ff006ca10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c15af20 0x7f200c15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200c1e28e0 0x7f200c1527c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 --2- 172.21.15.115:0/2773914980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ff8005bf0 0x7f200c0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 -- 172.21.15.115:0/2773914980 >> 172.21.15.115:0/2773914980 conn(0x7f200c0cc160 msgr2=0x7f200c06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 -- 172.21.15.115:0/2773914980 shutdown_connections 2023-12-01T01:52:18.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.066+0000 7f2012f88640 1 -- 172.21.15.115:0/2773914980 wait complete. 2023-12-01T01:52:18.086 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:52:18.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.296+0000 7fcc5392a640 1 Processor -- start 2023-12-01T01:52:18.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.297+0000 7fcc5392a640 1 -- start start 2023-12-01T01:52:18.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.298+0000 7fcc5392a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcc4c0cad70 0x7fcc4c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.298+0000 7fcc5392a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc4c06f2f0 0x7fcc4c06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.298+0000 7fcc5392a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4c0add10 con 0x7fcc4c15e2f0 2023-12-01T01:52:18.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.298+0000 7fcc5392a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c5760 con 0x7fcc4c0cad70 2023-12-01T01:52:18.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.298+0000 7fcc5392a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c15e6d0 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.299+0000 7fcc50ec5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcc4c0cad70 0x7fcc4c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.299+0000 7fcc50ec5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcc4c0cad70 0x7fcc4c06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:60110/0 (socket says 172.21.15.115:60110) 2023-12-01T01:52:18.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.299+0000 7fcc50ec5640 1 -- 172.21.15.115:0/1682394442 learned_addr learned my addr 172.21.15.115:0/1682394442 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:18.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.299+0000 7fcc51ec7640 1 --2- 172.21.15.115:0/1682394442 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc4c06f2f0 0x7fcc4c06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.300+0000 7fcc50ec5640 1 -- 172.21.15.115:0/1682394442 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc4c06f2f0 msgr2=0x7fcc4c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:18.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.300+0000 7fcc50ec5640 1 --2- 172.21.15.115:0/1682394442 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc4c06f2f0 0x7fcc4c06bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.300+0000 7fcc50ec5640 1 -- 172.21.15.115:0/1682394442 >> v1:172.21.15.142:6789/0 conn(0x7fcc4c15e2f0 legacy=0x7fcc4c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.300+0000 7fcc50ec5640 1 -- 172.21.15.115:0/1682394442 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0cf7b0 con 0x7fcc4c0cad70 2023-12-01T01:52:18.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.301+0000 7fcc50ec5640 1 --2- 172.21.15.115:0/1682394442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c0cad70 0x7fcc4c06b5c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009870 tx=0x7fcc3c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a86fd1bfe8cc76 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:18.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.301+0000 7fcc43fff640 1 -- 172.21.15.115:0/1682394442 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0317c0 con 0x7fcc4c0cad70 2023-12-01T01:52:18.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.301+0000 7fcc43fff640 1 -- 172.21.15.115:0/1682394442 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c031920 con 0x7fcc4c0cad70 2023-12-01T01:52:18.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.301+0000 7fcc43fff640 1 -- 172.21.15.115:0/1682394442 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c031cb0 con 0x7fcc4c0cad70 2023-12-01T01:52:18.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.302+0000 7fcc5392a640 1 -- 172.21.15.115:0/1682394442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c0cad70 msgr2=0x7fcc4c06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.302+0000 7fcc5392a640 1 --2- 172.21.15.115:0/1682394442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c0cad70 0x7fcc4c06b5c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009870 tx=0x7fcc3c02b860 comp rx=0 tx=0).stop 2023-12-01T01:52:18.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.302+0000 7fcc5392a640 1 -- 172.21.15.115:0/1682394442 shutdown_connections 2023-12-01T01:52:18.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.302+0000 7fcc5392a640 1 --2- 172.21.15.115:0/1682394442 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc4c06f2f0 0x7fcc4c06bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.302+0000 7fcc5392a640 1 --2- 172.21.15.115:0/1682394442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c0cad70 0x7fcc4c06b5c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.302+0000 7fcc5392a640 1 -- 172.21.15.115:0/1682394442 >> 172.21.15.115:0/1682394442 conn(0x7fcc4c0ce880 msgr2=0x7fcc4c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:18.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.302+0000 7fcc5392a640 1 -- 172.21.15.115:0/1682394442 shutdown_connections 2023-12-01T01:52:18.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.302+0000 7fcc5392a640 1 -- 172.21.15.115:0/1682394442 wait complete. 2023-12-01T01:52:18.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.304+0000 7fcc5392a640 1 Processor -- start 2023-12-01T01:52:18.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.304+0000 7fcc5392a640 1 -- start start 2023-12-01T01:52:18.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.305+0000 7fcc5392a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc4c06f2f0 0x7fcc4c153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.305+0000 7fcc5392a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc4c0cad70 0x7fcc4c0c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.305+0000 7fcc5392a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c15e2f0 0x7fcc4c0c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.305+0000 7fcc5392a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c150780 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.305+0000 7fcc5392a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c4a60 con 0x7fcc4c15e2f0 2023-12-01T01:52:18.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc5392a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0c4d40 con 0x7fcc4c0cad70 2023-12-01T01:52:18.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc516c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc4c06f2f0 0x7fcc4c153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc516c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc4c06f2f0 0x7fcc4c153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40438/0 (socket says 172.21.15.115:40438) 2023-12-01T01:52:18.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc516c6640 1 -- 172.21.15.115:0/4022046195 learned_addr learned my addr 172.21.15.115:0/4022046195 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:18.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc50ec5640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc4c0cad70 0x7fcc4c0c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc51ec7640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c15e2f0 0x7fcc4c0c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc516c6640 1 -- 172.21.15.115:0/4022046195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc4c0cad70 msgr2=0x7fcc4c0c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc516c6640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc4c0cad70 0x7fcc4c0c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc516c6640 1 -- 172.21.15.115:0/4022046195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c15e2f0 msgr2=0x7fcc4c0c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc516c6640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c15e2f0 0x7fcc4c0c4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.306+0000 7fcc516c6640 1 -- 172.21.15.115:0/4022046195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c0094d0 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.307+0000 7fcc516c6640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc4c06f2f0 0x7fcc4c153340 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c3c0 tx=0x7fcc4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:18.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.307+0000 7fcc41ffb640 1 -- 172.21.15.115:0/4022046195 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc480052d0 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.307+0000 7fcc41ffb640 1 -- 172.21.15.115:0/4022046195 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48005450 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.307+0000 7fcc41ffb640 1 -- 172.21.15.115:0/4022046195 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48007260 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.307+0000 7fcc5392a640 1 -- 172.21.15.115:0/4022046195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c0c5020 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.307+0000 7fcc5392a640 1 -- 172.21.15.115:0/4022046195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c0ce110 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.308+0000 7fcc5392a640 1 -- 172.21.15.115:0/4022046195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c06fda0 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.309+0000 7fcc41ffb640 1 -- 172.21.15.115:0/4022046195 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fcc480073c0 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.310+0000 7fcc41ffb640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcc3006a550 0x7fcc3006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:18.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.310+0000 7fcc41ffb640 1 -- 172.21.15.115:0/4022046195 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc48087ec0 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.310+0000 7fcc50ec5640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcc3006a550 0x7fcc3006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:18.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.311+0000 7fcc50ec5640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcc3006a550 0x7fcc3006ca10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c036000 tx=0x7fcc3c03d040 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:18.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.314+0000 7fcc41ffb640 1 -- 172.21.15.115:0/4022046195 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcc48053310 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.543+0000 7fcc5392a640 1 -- 172.21.15.115:0/4022046195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcc4c0c5260 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.544+0000 7fcc41ffb640 1 -- 172.21.15.115:0/4022046195 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcc4c0c5260 con 0x7fcc4c06f2f0 2023-12-01T01:52:18.545 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:52:18.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 -- 172.21.15.115:0/4022046195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcc3006a550 msgr2=0x7fcc3006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcc3006a550 0x7fcc3006ca10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c036000 tx=0x7fcc3c03d040 comp rx=0 tx=0).stop 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 -- 172.21.15.115:0/4022046195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc4c06f2f0 msgr2=0x7fcc4c153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc4c06f2f0 0x7fcc4c153340 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c3c0 tx=0x7fcc4800c880 comp rx=0 tx=0).stop 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 -- 172.21.15.115:0/4022046195 shutdown_connections 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcc3006a550 0x7fcc3006ca10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc4c0cad70 0x7fcc4c0c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc4c15e2f0 0x7fcc4c0c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 --2- 172.21.15.115:0/4022046195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc4c06f2f0 0x7fcc4c153340 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 -- 172.21.15.115:0/4022046195 >> 172.21.15.115:0/4022046195 conn(0x7fcc4c0ce880 msgr2=0x7fcc4c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 -- 172.21.15.115:0/4022046195 shutdown_connections 2023-12-01T01:52:18.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:18.547+0000 7fcc5392a640 1 -- 172.21.15.115:0/4022046195 wait complete. 2023-12-01T01:52:18.567 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T01:52:18.567 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:52:18.567 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T01:52:18.567 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T01:52:18.567 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:52:18.567 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T01:52:18.567 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T01:52:18.567 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:52:18.567 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:52:18.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:18.567 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T01:52:18.619 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 41418sec preferred_lft 41418sec 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:52:18.620 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:52:18.621 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:52:18.621 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T01:52:18.621 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:52:18.621 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:52:18.621 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T01:52:18.621 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T01:52:18.621 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:52:18.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:18.621 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:52:18.713 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T01:52:18.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:18.717 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T01:52:18.771 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T01:52:18.771 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:52:18.771 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:52:18.771 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:52:18.772 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:52:18.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:18.772 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:52:18.772 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:52:18.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:52:18.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:52:18.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:52:18.772 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:52:18.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:18.959 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:52:19.008 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T01:52:19.009 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T01:52:19.009 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:52:19.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:19.009 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T01:52:19.103 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:52:19.104 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T01:52:19.161 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:52:19.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:52:19.161 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:52:19.213 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:52:19.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:19.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:52:19.354 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:52:19.354 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:52:19.354 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:52:19.375 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:52:19.375 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:52:19.375 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:52:19.375 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:52:19.375 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:52:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:52:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:52:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:52:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:52:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:52:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:52:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:52:19.402 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:52:19.405 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T01:52:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:52:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:52:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:52:19.405 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:52:19.409 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:52:19.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:19.409 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:52:19.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:19.498 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:52:19.577 INFO:teuthology.orchestra.run.smithi115.stdout:client7125 2023-12-01T01:52:19.577 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:52:19.577 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:19.634 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T01:52:19.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.843+0000 7f4831955640 1 Processor -- start 2023-12-01T01:52:19.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.843+0000 7f4831955640 1 -- start start 2023-12-01T01:52:19.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.844+0000 7f4831955640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f482c15e2f0 0x7f482c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:19.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.844+0000 7f4831955640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0add10 con 0x7f482c06f2f0 2023-12-01T01:52:19.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.844+0000 7f4831955640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0c5760 con 0x7f482c0adf10 2023-12-01T01:52:19.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.844+0000 7f4831955640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c0c9cf0 con 0x7f482c15e2f0 2023-12-01T01:52:19.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.845+0000 7f482a7fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f482c06f2f0 0x7f482c0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:45934/0 (socket says 172.21.15.115:45934) 2023-12-01T01:52:19.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.845+0000 7f482a7fc640 1 -- 172.21.15.115:0/1977405566 learned_addr learned my addr 172.21.15.115:0/1977405566 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:19.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.845+0000 7f482affd640 1 --2- 172.21.15.115:0/1977405566 >> v2:172.21.15.142:3300/0 conn(0x7f482c15e2f0 0x7f482c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:19.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.845+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914971416 0 0) 0x7f482c0add10 con 0x7f482c06f2f0 2023-12-01T01:52:19.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.845+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4818003520 con 0x7f482c06f2f0 2023-12-01T01:52:19.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3033658043 0 0) 0x7f4818003520 con 0x7f482c06f2f0 2023-12-01T01:52:19.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 >> v2:172.21.15.142:3300/0 conn(0x7f482c15e2f0 msgr2=0x7f482c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:19.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 --2- 172.21.15.115:0/1977405566 >> v2:172.21.15.142:3300/0 conn(0x7f482c15e2f0 0x7f482c0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 >> v1:172.21.15.142:6789/0 conn(0x7f482c0adf10 legacy=0x7f482c0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c15e6d0 con 0x7f482c06f2f0 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4814001670 con 0x7f482c06f2f0 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 >> v1:172.21.15.115:6789/0 conn(0x7f482c06f2f0 legacy=0x7f482c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 --2- 172.21.15.115:0/1977405566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48180039e0 0x7f4818003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 --2- 172.21.15.115:0/1977405566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48180047c0 0x7f4818004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 --2- 172.21.15.115:0/1977405566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4818005280 0x7f481800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0c9cf0 con 0x7f48180039e0 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481800c1c0 con 0x7f48180047c0 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481800c380 con 0x7f4818005280 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f482affd640 1 --2- 172.21.15.115:0/1977405566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48180039e0 0x7f4818003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f482affd640 1 -- 172.21.15.115:0/1977405566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4818005280 msgr2=0x7f481800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f482affd640 1 --2- 172.21.15.115:0/1977405566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4818005280 0x7f481800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f482affd640 1 -- 172.21.15.115:0/1977405566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48180047c0 msgr2=0x7f4818004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:19.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f482affd640 1 --2- 172.21.15.115:0/1977405566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48180047c0 0x7f4818004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:19.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f482affd640 1 -- 172.21.15.115:0/1977405566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818003520 con 0x7f48180039e0 2023-12-01T01:52:19.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.846+0000 7f482b7fe640 1 -- 172.21.15.115:0/1977405566 reap_dead start 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.847+0000 7f482affd640 1 --2- 172.21.15.115:0/1977405566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48180039e0 0x7f4818003e00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f481c009fc0 tx=0x7f48180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b956171ea0ff95 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.848+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c02cc60 con 0x7f48180039e0 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.848+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c00f040 con 0x7f48180039e0 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.848+0000 7f4829ffb640 1 -- 172.21.15.115:0/1977405566 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c034440 con 0x7f48180039e0 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.848+0000 7f4831955640 1 -- 172.21.15.115:0/1977405566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48180039e0 msgr2=0x7f4818003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.848+0000 7f4831955640 1 --2- 172.21.15.115:0/1977405566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48180039e0 0x7f4818003e00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f481c009fc0 tx=0x7f48180057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.848+0000 7f4831955640 1 -- 172.21.15.115:0/1977405566 shutdown_connections 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.848+0000 7f4831955640 1 --2- 172.21.15.115:0/1977405566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48180039e0 0x7f4818003e00 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.848+0000 7f4831955640 1 -- 172.21.15.115:0/1977405566 >> 172.21.15.115:0/1977405566 conn(0x7f482c0ce880 msgr2=0x7f482c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.848+0000 7f4831955640 1 -- 172.21.15.115:0/1977405566 shutdown_connections 2023-12-01T01:52:19.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.848+0000 7f4831955640 1 -- 172.21.15.115:0/1977405566 wait complete. 2023-12-01T01:52:19.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.850+0000 7f4831955640 1 Processor -- start 2023-12-01T01:52:19.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.850+0000 7f4831955640 1 -- start start 2023-12-01T01:52:19.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.851+0000 7f4831955640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48180039e0 0x7f482c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:19.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.851+0000 7f4831955640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f482c0c23b0 0x7f482c0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:19.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.851+0000 7f4831955640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c14dae0 0x7f482c15cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:19.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.851+0000 7f4831955640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c15be40 con 0x7f482c14dae0 2023-12-01T01:52:19.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.851+0000 7f4831955640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c15c120 con 0x7f48180039e0 2023-12-01T01:52:19.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.851+0000 7f4831955640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c15c400 con 0x7f482c0c23b0 2023-12-01T01:52:19.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.851+0000 7f482b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c14dae0 0x7f482c15cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:19.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.851+0000 7f482b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c14dae0 0x7f482c15cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40458/0 (socket says 172.21.15.115:40458) 2023-12-01T01:52:19.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f482b7fe640 1 -- 172.21.15.115:0/111833916 learned_addr learned my addr 172.21.15.115:0/111833916 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:19.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f482a7fc640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f482c0c23b0 0x7f482c0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:19.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f482affd640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48180039e0 0x7f482c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f482b7fe640 1 -- 172.21.15.115:0/111833916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f482c0c23b0 msgr2=0x7f482c0c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f482b7fe640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f482c0c23b0 0x7f482c0c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f482b7fe640 1 -- 172.21.15.115:0/111833916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48180039e0 msgr2=0x7f482c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f482b7fe640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48180039e0 0x7f482c15b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f482b7fe640 1 -- 172.21.15.115:0/111833916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c15c6e0 con 0x7f482c14dae0 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f482affd640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48180039e0 0x7f482c15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f482b7fe640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c14dae0 0x7f482c15cee0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f482c0c8e50 tx=0x7f482400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f4823fff640 1 -- 172.21.15.115:0/111833916 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48240074e0 con 0x7f482c14dae0 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f4823fff640 1 -- 172.21.15.115:0/111833916 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824007660 con 0x7f482c14dae0 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.852+0000 7f4823fff640 1 -- 172.21.15.115:0/111833916 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824004260 con 0x7f482c14dae0 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.853+0000 7f4831955640 1 -- 172.21.15.115:0/111833916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c15c9c0 con 0x7f482c14dae0 2023-12-01T01:52:19.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.853+0000 7f4831955640 1 -- 172.21.15.115:0/111833916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c0c2810 con 0x7f482c14dae0 2023-12-01T01:52:19.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.854+0000 7f4831955640 1 -- 172.21.15.115:0/111833916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c06fda0 con 0x7f482c14dae0 2023-12-01T01:52:19.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.855+0000 7f4823fff640 1 -- 172.21.15.115:0/111833916 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f48240057c0 con 0x7f482c14dae0 2023-12-01T01:52:19.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.856+0000 7f4823fff640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f480c06a550 0x7f480c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:19.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.856+0000 7f4823fff640 1 -- 172.21.15.115:0/111833916 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4824087c40 con 0x7f482c14dae0 2023-12-01T01:52:19.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.856+0000 7f482affd640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f480c06a550 0x7f480c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:19.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.857+0000 7f482affd640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f480c06a550 0x7f480c06ca10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f481c036000 tx=0x7f481c02e000 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:19.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:19.861+0000 7f4823fff640 1 -- 172.21.15.115:0/111833916 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4824052ff0 con 0x7f482c14dae0 2023-12-01T01:52:20.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.075+0000 7f4831955640 1 -- 172.21.15.115:0/111833916 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f482c0cf4d0 con 0x7f480c06a550 2023-12-01T01:52:20.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.077+0000 7f4823fff640 1 -- 172.21.15.115:0/111833916 <== mgr.6723 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f481c044080 con 0x7f480c06a550 2023-12-01T01:52:20.079 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:52:20.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:20.080 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T01:52:20.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:20.080 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:52:20.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.080+0000 7f4831955640 1 -- 172.21.15.115:0/111833916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f480c06a550 msgr2=0x7f480c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:20.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.080+0000 7f4831955640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f480c06a550 0x7f480c06ca10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f481c036000 tx=0x7f481c02e000 comp rx=0 tx=0).stop 2023-12-01T01:52:20.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.080+0000 7f4831955640 1 -- 172.21.15.115:0/111833916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c14dae0 msgr2=0x7f482c15cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:20.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.080+0000 7f4831955640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c14dae0 0x7f482c15cee0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f482c0c8e50 tx=0x7f482400c820 comp rx=0 tx=0).stop 2023-12-01T01:52:20.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.080+0000 7f4831955640 1 -- 172.21.15.115:0/111833916 shutdown_connections 2023-12-01T01:52:20.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.080+0000 7f4831955640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f480c06a550 0x7f480c06ca10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:20.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.080+0000 7f4831955640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f482c0c23b0 0x7f482c0c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:20.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.080+0000 7f4831955640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c14dae0 0x7f482c15cee0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:20.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.080+0000 7f4831955640 1 --2- 172.21.15.115:0/111833916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48180039e0 0x7f482c15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:20.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.081+0000 7f4831955640 1 -- 172.21.15.115:0/111833916 >> 172.21.15.115:0/111833916 conn(0x7f482c0ce880 msgr2=0x7f482c06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:20.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.081+0000 7f4831955640 1 -- 172.21.15.115:0/111833916 shutdown_connections 2023-12-01T01:52:20.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.081+0000 7f4831955640 1 -- 172.21.15.115:0/111833916 wait complete. 2023-12-01T01:52:20.111 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T01:52:20.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.322+0000 7f29378af640 1 Processor -- start 2023-12-01T01:52:20.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.322+0000 7f29378af640 1 -- start start 2023-12-01T01:52:20.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.324+0000 7f29378af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 0x7f29300ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:20.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.324+0000 7f29378af640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f29300cd010 0x7f29300b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:20.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.324+0000 7f29378af640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29300cd470 con 0x7f29300b1c50 2023-12-01T01:52:20.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.324+0000 7f29378af640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29300b3ae0 con 0x7f29300cd010 2023-12-01T01:52:20.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.324+0000 7f29378af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2930142c50 con 0x7f2930142870 2023-12-01T01:52:20.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.325+0000 7f2934e4a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f29300cd010 0x7f29300b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:20.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.325+0000 7f293564b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 0x7f29300ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:20.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.325+0000 7f293564b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 0x7f29300ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:33638/0 (socket says 172.21.15.115:33638) 2023-12-01T01:52:20.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.325+0000 7f293564b640 1 -- 172.21.15.115:0/3849768398 learned_addr learned my addr 172.21.15.115:0/3849768398 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:20.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.325+0000 7f2934e4a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f29300cd010 0x7f29300b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:60160/0 (socket says 172.21.15.115:60160) 2023-12-01T01:52:20.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.326+0000 7f2934e4a640 1 -- 172.21.15.115:0/3849768398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 msgr2=0x7f29300ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:20.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.326+0000 7f2934e4a640 1 --2- 172.21.15.115:0/3849768398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 0x7f29300ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:20.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.326+0000 7f2934e4a640 1 -- 172.21.15.115:0/3849768398 >> v1:172.21.15.142:6789/0 conn(0x7f29300b1c50 legacy=0x7f293015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:20.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.326+0000 7f2934e4a640 1 -- 172.21.15.115:0/3849768398 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29300b20e0 con 0x7f29300cd010 2023-12-01T01:52:20.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.326+0000 7f293564b640 1 --2- 172.21.15.115:0/3849768398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 0x7f29300ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:52:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.328+0000 7f2934e4a640 1 --2- 172.21.15.115:0/3849768398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29300cd010 0x7f29300b1710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f29200097d0 tx=0x7f292002b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c0a524de31a12f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.328+0000 7f2927fff640 1 -- 172.21.15.115:0/3849768398 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920031800 con 0x7f29300cd010 2023-12-01T01:52:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.328+0000 7f2927fff640 1 -- 172.21.15.115:0/3849768398 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2920031960 con 0x7f29300cd010 2023-12-01T01:52:20.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.328+0000 7f2927fff640 1 -- 172.21.15.115:0/3849768398 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920031cf0 con 0x7f29300cd010 2023-12-01T01:52:20.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.329+0000 7f29378af640 1 -- 172.21.15.115:0/3849768398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29300cd010 msgr2=0x7f29300b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:20.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.329+0000 7f29378af640 1 --2- 172.21.15.115:0/3849768398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29300cd010 0x7f29300b1710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f29200097d0 tx=0x7f292002b8a0 comp rx=0 tx=0).stop 2023-12-01T01:52:20.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.329+0000 7f29378af640 1 -- 172.21.15.115:0/3849768398 shutdown_connections 2023-12-01T01:52:20.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.329+0000 7f29378af640 1 --2- 172.21.15.115:0/3849768398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29300cd010 0x7f29300b1710 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:20.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.329+0000 7f29378af640 1 --2- 172.21.15.115:0/3849768398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 0x7f29300ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:20.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.329+0000 7f29378af640 1 -- 172.21.15.115:0/3849768398 >> 172.21.15.115:0/3849768398 conn(0x7f2930065960 msgr2=0x7f2930065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:20.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.329+0000 7f29378af640 1 -- 172.21.15.115:0/3849768398 shutdown_connections 2023-12-01T01:52:20.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.329+0000 7f29378af640 1 -- 172.21.15.115:0/3849768398 wait complete. 2023-12-01T01:52:20.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.331+0000 7f29378af640 1 Processor -- start 2023-12-01T01:52:20.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.331+0000 7f29378af640 1 -- start start 2023-12-01T01:52:20.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.332+0000 7f29378af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29300b1c50 0x7f29300b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:20.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.332+0000 7f29378af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29300cd010 0x7f29300704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:20.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.332+0000 7f29378af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 0x7f29301f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:20.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.332+0000 7f29378af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29301590c0 con 0x7f29300b1c50 2023-12-01T01:52:20.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.332+0000 7f29378af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293015aae0 con 0x7f29300cd010 2023-12-01T01:52:20.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.332+0000 7f29378af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29300b0670 con 0x7f2930142870 2023-12-01T01:52:20.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.332+0000 7f293564b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29300b1c50 0x7f29300b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:20.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.332+0000 7f293564b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29300b1c50 0x7f29300b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40460/0 (socket says 172.21.15.115:40460) 2023-12-01T01:52:20.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.332+0000 7f293564b640 1 -- 172.21.15.115:0/3130622410 learned_addr learned my addr 172.21.15.115:0/3130622410 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:20.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f293564b640 1 -- 172.21.15.115:0/3130622410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 msgr2=0x7f29301f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:20.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f2935e4c640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 0x7f29301f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:20.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f2934e4a640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29300cd010 0x7f29300704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:20.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f293564b640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 0x7f29301f3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:20.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f293564b640 1 -- 172.21.15.115:0/3130622410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29300cd010 msgr2=0x7f29300704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:20.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f293564b640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29300cd010 0x7f29300704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:20.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f293564b640 1 -- 172.21.15.115:0/3130622410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c0094d0 con 0x7f29300b1c50 2023-12-01T01:52:20.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f2934e4a640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29300cd010 0x7f29300704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:20.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f293564b640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29300b1c50 0x7f29300b0130 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f292c00e9b0 tx=0x7f292c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:20.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f2925ffb640 1 -- 172.21.15.115:0/3130622410 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c00c9a0 con 0x7f29300b1c50 2023-12-01T01:52:20.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f2925ffb640 1 -- 172.21.15.115:0/3130622410 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c00cb00 con 0x7f29300b1c50 2023-12-01T01:52:20.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f2925ffb640 1 -- 172.21.15.115:0/3130622410 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c00ce70 con 0x7f29300b1c50 2023-12-01T01:52:20.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.333+0000 7f29378af640 1 -- 172.21.15.115:0/3130622410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2920009400 con 0x7f29300b1c50 2023-12-01T01:52:20.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.334+0000 7f29378af640 1 -- 172.21.15.115:0/3130622410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293015a1b0 con 0x7f29300b1c50 2023-12-01T01:52:20.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.335+0000 7f29378af640 1 -- 172.21.15.115:0/3130622410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29300b2bc0 con 0x7f29300b1c50 2023-12-01T01:52:20.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.336+0000 7f2925ffb640 1 -- 172.21.15.115:0/3130622410 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 41) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f292c013470 con 0x7f29300b1c50 2023-12-01T01:52:20.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.336+0000 7f2925ffb640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f291406a550 0x7f291406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:20.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.337+0000 7f2925ffb640 1 -- 172.21.15.115:0/3130622410 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f292c089130 con 0x7f29300b1c50 2023-12-01T01:52:20.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.337+0000 7f2934e4a640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f291406a550 0x7f291406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:20.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.337+0000 7f2934e4a640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f291406a550 0x7f291406ca10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2920036000 tx=0x7f2920008030 comp rx=0 tx=0).ready entity=mgr.6723 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:20.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.341+0000 7f2925ffb640 1 -- 172.21.15.115:0/3130622410 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f292c054570 con 0x7f29300b1c50 2023-12-01T01:52:20.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:20.581+0000 7f29378af640 1 -- 172.21.15.115:0/3130622410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f293004e890 con 0x7f29300b1c50 2023-12-01T01:52:21.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.559+0000 7f2925ffb640 1 -- 172.21.15.115:0/3130622410 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 42) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f292c054940 con 0x7f29300b1c50 2023-12-01T01:52:21.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.561+0000 7f2925ffb640 1 -- 172.21.15.115:0/3130622410 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v42) v1 ==== 92+0+0 (secure 0 0 0) 0x7f292c0862b0 con 0x7f29300b1c50 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 -- 172.21.15.115:0/3130622410 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f291406a550 msgr2=0x7f291406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f291406a550 0x7f291406ca10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2920036000 tx=0x7f2920008030 comp rx=0 tx=0).stop 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 -- 172.21.15.115:0/3130622410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29300b1c50 msgr2=0x7f29300b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29300b1c50 0x7f29300b0130 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f292c00e9b0 tx=0x7f292c00ee70 comp rx=0 tx=0).stop 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 -- 172.21.15.115:0/3130622410 shutdown_connections 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f291406a550 0x7f291406ca10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2930142870 0x7f29301f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29300cd010 0x7f29300704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 --2- 172.21.15.115:0/3130622410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29300b1c50 0x7f29300b0130 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 -- 172.21.15.115:0/3130622410 >> 172.21.15.115:0/3130622410 conn(0x7f2930065960 msgr2=0x7f293013ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 -- 172.21.15.115:0/3130622410 shutdown_connections 2023-12-01T01:52:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.564+0000 7f29378af640 1 -- 172.21.15.115:0/3130622410 wait complete. 2023-12-01T01:52:21.585 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T01:52:21.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.772+0000 7fad3db3e640 1 Processor -- start 2023-12-01T01:52:21.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.772+0000 7fad3db3e640 1 -- start start 2023-12-01T01:52:21.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.773+0000 7fad3db3e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fad3815cd90 0x7fad380cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad3db3e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fad380cfd60 0x7fad38161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad3db3e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad380ce5e0 con 0x7fad380b5fa0 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad3db3e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad380d0ec0 con 0x7fad3815cd90 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad3db3e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad38161950 con 0x7fad380cfd60 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad377fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fad3815cd90 0x7fad380cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad2ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fad380b5fa0 0x7fad380cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:45946/0 (socket says 172.21.15.115:45946) 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad2ffff640 1 -- 172.21.15.115:0/2164016676 learned_addr learned my addr 172.21.15.115:0/2164016676 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad37fff640 1 --2- 172.21.15.115:0/2164016676 >> v2:172.21.15.142:3300/0 conn(0x7fad380cfd60 0x7fad38161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad377fe640 1 -- 172.21.15.115:0/2164016676 >> v2:172.21.15.142:3300/0 conn(0x7fad380cfd60 msgr2=0x7fad38161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad377fe640 1 --2- 172.21.15.115:0/2164016676 >> v2:172.21.15.142:3300/0 conn(0x7fad380cfd60 0x7fad38161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad377fe640 1 -- 172.21.15.115:0/2164016676 >> v1:172.21.15.115:6789/0 conn(0x7fad380b5fa0 legacy=0x7fad380cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:21.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad377fe640 1 -- 172.21.15.115:0/2164016676 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad380ce300 con 0x7fad3815cd90 2023-12-01T01:52:21.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.774+0000 7fad36ffd640 1 -- 172.21.15.115:0/2164016676 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330307255 0 0) 0x7fad20001110 con 0x7fad380b5fa0 2023-12-01T01:52:21.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.776+0000 7fad377fe640 1 --2- 172.21.15.115:0/2164016676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3815cd90 0x7fad380cdd20 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fad380d04f0 tx=0x7fad2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3734bb11f4520b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:21.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.776+0000 7fad36ffd640 1 -- 172.21.15.115:0/2164016676 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28031830 con 0x7fad3815cd90 2023-12-01T01:52:21.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.776+0000 7fad36ffd640 1 -- 172.21.15.115:0/2164016676 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad28031990 con 0x7fad3815cd90 2023-12-01T01:52:21.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.776+0000 7fad36ffd640 1 -- 172.21.15.115:0/2164016676 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28031d20 con 0x7fad3815cd90 2023-12-01T01:52:21.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.776+0000 7fad3db3e640 1 -- 172.21.15.115:0/2164016676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3815cd90 msgr2=0x7fad380cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:21.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.776+0000 7fad3db3e640 1 --2- 172.21.15.115:0/2164016676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3815cd90 0x7fad380cdd20 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fad380d04f0 tx=0x7fad2802b8d0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.777+0000 7fad3db3e640 1 -- 172.21.15.115:0/2164016676 shutdown_connections 2023-12-01T01:52:21.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.777+0000 7fad3db3e640 1 --2- 172.21.15.115:0/2164016676 >> v2:172.21.15.142:3300/0 conn(0x7fad380cfd60 0x7fad38161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.777+0000 7fad3db3e640 1 --2- 172.21.15.115:0/2164016676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad3815cd90 0x7fad380cdd20 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.777+0000 7fad3db3e640 1 -- 172.21.15.115:0/2164016676 >> 172.21.15.115:0/2164016676 conn(0x7fad38065960 msgr2=0x7fad38065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:21.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.777+0000 7fad3db3e640 1 -- 172.21.15.115:0/2164016676 shutdown_connections 2023-12-01T01:52:21.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.777+0000 7fad3db3e640 1 -- 172.21.15.115:0/2164016676 wait complete. 2023-12-01T01:52:21.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.778+0000 7fad3db3e640 1 Processor -- start 2023-12-01T01:52:21.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.779+0000 7fad3db3e640 1 -- start start 2023-12-01T01:52:21.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad3db3e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad380b5fa0 0x7fad381f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:21.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad3db3e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad380cfd60 0x7fad381effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:21.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad3db3e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3815cd90 0x7fad380a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad3db3e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380a9ab0 con 0x7fad380cfd60 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad3db3e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380a9d50 con 0x7fad3815cd90 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad3db3e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad380aa030 con 0x7fad380b5fa0 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad380cfd60 0x7fad381effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad380cfd60 0x7fad381effc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40496/0 (socket says 172.21.15.115:40496) 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad2ffff640 1 -- 172.21.15.115:0/2417916951 learned_addr learned my addr 172.21.15.115:0/2417916951 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad377fe640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad380b5fa0 0x7fad381f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad2ffff640 1 -- 172.21.15.115:0/2417916951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad380b5fa0 msgr2=0x7fad381f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad2ffff640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad380b5fa0 0x7fad381f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad2ffff640 1 -- 172.21.15.115:0/2417916951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3815cd90 msgr2=0x7fad380a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad2ffff640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3815cd90 0x7fad380a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.780+0000 7fad2ffff640 1 -- 172.21.15.115:0/2417916951 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad28009400 con 0x7fad380cfd60 2023-12-01T01:52:21.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.781+0000 7fad2ffff640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad380cfd60 0x7fad381effc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c780 tx=0x7fad2000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:21.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.781+0000 7fad34ff9640 1 -- 172.21.15.115:0/2417916951 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad200072d0 con 0x7fad380cfd60 2023-12-01T01:52:21.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.781+0000 7fad34ff9640 1 -- 172.21.15.115:0/2417916951 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20007430 con 0x7fad380cfd60 2023-12-01T01:52:21.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.781+0000 7fad34ff9640 1 -- 172.21.15.115:0/2417916951 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20001d10 con 0x7fad380cfd60 2023-12-01T01:52:21.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.781+0000 7fad3db3e640 1 -- 172.21.15.115:0/2417916951 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad380aa310 con 0x7fad380cfd60 2023-12-01T01:52:21.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.781+0000 7fad3db3e640 1 -- 172.21.15.115:0/2417916951 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad380aa8d0 con 0x7fad380cfd60 2023-12-01T01:52:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.782+0000 7fad3db3e640 1 -- 172.21.15.115:0/2417916951 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad380aaac0 con 0x7fad380cfd60 2023-12-01T01:52:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.783+0000 7fad34ff9640 1 -- 172.21.15.115:0/2417916951 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 42) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fad200047e0 con 0x7fad380cfd60 2023-12-01T01:52:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.784+0000 7fad34ff9640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad1806a5d0 0x7fad1806ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:21.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.784+0000 7fad34ff9640 1 -- 172.21.15.115:0/2417916951 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad20087c10 con 0x7fad380cfd60 2023-12-01T01:52:21.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.784+0000 7fad377fe640 1 -- 172.21.15.115:0/2417916951 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad1806a5d0 msgr2=0x7fad1806ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:21.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.784+0000 7fad377fe640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad1806a5d0 0x7fad1806ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:21.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.789+0000 7fad34ff9640 1 -- 172.21.15.115:0/2417916951 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fad200530f0 con 0x7fad380cfd60 2023-12-01T01:52:21.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.972+0000 7fad3db3e640 1 -- 172.21.15.115:0/2417916951 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fad380ce5e0 con 0x7fad380cfd60 2023-12-01T01:52:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.980+0000 7fad34ff9640 1 -- 172.21.15.115:0/2417916951 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7fad380ce5e0 con 0x7fad380cfd60 2023-12-01T01:52:21.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 -- 172.21.15.115:0/2417916951 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad1806a5d0 msgr2=0x7fad1806ca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:21.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad1806a5d0 0x7fad1806ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 -- 172.21.15.115:0/2417916951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad380cfd60 msgr2=0x7fad381effc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad380cfd60 0x7fad381effc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c780 tx=0x7fad2000cc40 comp rx=0 tx=0).stop 2023-12-01T01:52:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 -- 172.21.15.115:0/2417916951 shutdown_connections 2023-12-01T01:52:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fad1806a5d0 0x7fad1806ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3815cd90 0x7fad380a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fad380cfd60 0x7fad381effc0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 --2- 172.21.15.115:0/2417916951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad380b5fa0 0x7fad381f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 -- 172.21.15.115:0/2417916951 >> 172.21.15.115:0/2417916951 conn(0x7fad38065960 msgr2=0x7fad380d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 -- 172.21.15.115:0/2417916951 shutdown_connections 2023-12-01T01:52:21.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:21.983+0000 7fad3db3e640 1 -- 172.21.15.115:0/2417916951 wait complete. 2023-12-01T01:52:22.003 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T01:52:22.060 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:22.059+0000 7f7ec9432040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:52:22.112 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:22.112+0000 7f7ec9432040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:52:22.115 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:22.114+0000 7efdf2232040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:52:22.169 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:22.168+0000 7efdf2232040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:52:22.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.201+0000 7fb4d31ab640 1 Processor -- start 2023-12-01T01:52:22.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.201+0000 7fb4d31ab640 1 -- start start 2023-12-01T01:52:22.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.202+0000 7fb4d31ab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb4cc0cfd60 0x7fb4cc0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.202+0000 7fb4d31ab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc0ce5e0 con 0x7fb4cc15cd90 2023-12-01T01:52:22.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.202+0000 7fb4d31ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc0ce300 con 0x7fb4cc0b5fa0 2023-12-01T01:52:22.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.202+0000 7fb4d31ab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc0d0ec0 con 0x7fb4cc0cfd60 2023-12-01T01:52:22.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.202+0000 7fb4cbfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb4cc0cfd60 0x7fb4cc0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:22.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.202+0000 7fb4cbfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb4cc0cfd60 0x7fb4cc0cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40508/0 (socket says 172.21.15.115:40508) 2023-12-01T01:52:22.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.202+0000 7fb4cbfff640 1 -- 172.21.15.115:0/377011884 learned_addr learned my addr 172.21.15.115:0/377011884 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:22.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.203+0000 7fb4cbfff640 1 -- 172.21.15.115:0/377011884 >> v1:172.21.15.142:6789/0 conn(0x7fb4cc0b5fa0 legacy=0x7fb4cc0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:22.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.203+0000 7fb4cbfff640 1 -- 172.21.15.115:0/377011884 >> v1:172.21.15.115:6789/0 conn(0x7fb4cc15cd90 legacy=0x7fb4cc0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:22.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.203+0000 7fb4cb7fe640 1 -- 172.21.15.115:0/377011884 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96700134 0 0) 0x7fb4c0001110 con 0x7fb4cc15cd90 2023-12-01T01:52:22.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.203+0000 7fb4cbfff640 1 -- 172.21.15.115:0/377011884 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0d10a0 con 0x7fb4cc0cfd60 2023-12-01T01:52:22.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.205+0000 7fb4cbfff640 1 --2- 172.21.15.115:0/377011884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc0cfd60 0x7fb4cc0cdd20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009870 tx=0x7fb4b802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b96fbf1fcc6791 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:22.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.205+0000 7fb4cb7fe640 1 -- 172.21.15.115:0/377011884 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8030860 con 0x7fb4cc0cfd60 2023-12-01T01:52:22.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.205+0000 7fb4cb7fe640 1 -- 172.21.15.115:0/377011884 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b80309c0 con 0x7fb4cc0cfd60 2023-12-01T01:52:22.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.205+0000 7fb4cb7fe640 1 -- 172.21.15.115:0/377011884 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8030d50 con 0x7fb4cc0cfd60 2023-12-01T01:52:22.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.206+0000 7fb4d31ab640 1 -- 172.21.15.115:0/377011884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc0cfd60 msgr2=0x7fb4cc0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:22.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.206+0000 7fb4d31ab640 1 --2- 172.21.15.115:0/377011884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc0cfd60 0x7fb4cc0cdd20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009870 tx=0x7fb4b802b9a0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.206+0000 7fb4d31ab640 1 -- 172.21.15.115:0/377011884 shutdown_connections 2023-12-01T01:52:22.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.206+0000 7fb4d31ab640 1 --2- 172.21.15.115:0/377011884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc0cfd60 0x7fb4cc0cdd20 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.206+0000 7fb4d31ab640 1 -- 172.21.15.115:0/377011884 >> 172.21.15.115:0/377011884 conn(0x7fb4cc065960 msgr2=0x7fb4cc065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:22.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.206+0000 7fb4d31ab640 1 -- 172.21.15.115:0/377011884 shutdown_connections 2023-12-01T01:52:22.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.206+0000 7fb4d31ab640 1 -- 172.21.15.115:0/377011884 wait complete. 2023-12-01T01:52:22.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.207+0000 7fb4d31ab640 1 Processor -- start 2023-12-01T01:52:22.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.208+0000 7fb4d31ab640 1 -- start start 2023-12-01T01:52:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d31ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4cc0b5fa0 0x7fb4cc0cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d31ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc0cfd60 0x7fb4cc14ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d31ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc15cd90 0x7fb4cc1e6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d31ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc160620 con 0x7fb4cc15cd90 2023-12-01T01:52:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d31ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc160190 con 0x7fb4cc0cfd60 2023-12-01T01:52:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d31ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc15f580 con 0x7fb4cc0b5fa0 2023-12-01T01:52:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d1748640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc15cd90 0x7fb4cc1e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d1748640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc15cd90 0x7fb4cc1e6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40514/0 (socket says 172.21.15.115:40514) 2023-12-01T01:52:22.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d1748640 1 -- 172.21.15.115:0/1175843002 learned_addr learned my addr 172.21.15.115:0/1175843002 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:22.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4cbfff640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc0cfd60 0x7fb4cc14ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d1748640 1 -- 172.21.15.115:0/1175843002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4cc0b5fa0 msgr2=0x7fb4cc0cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d1748640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4cc0b5fa0 0x7fb4cc0cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d1748640 1 -- 172.21.15.115:0/1175843002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc0cfd60 msgr2=0x7fb4cc14ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d1748640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc0cfd60 0x7fb4cc14ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.209+0000 7fb4d1748640 1 -- 172.21.15.115:0/1175843002 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80094d0 con 0x7fb4cc15cd90 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.210+0000 7fb4d1748640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc15cd90 0x7fb4cc1e6810 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400c780 tx=0x7fb4c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.210+0000 7fb4c97fa640 1 -- 172.21.15.115:0/1175843002 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4004b60 con 0x7fb4cc15cd90 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.210+0000 7fb4c97fa640 1 -- 172.21.15.115:0/1175843002 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c400ce90 con 0x7fb4cc15cd90 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.210+0000 7fb4c97fa640 1 -- 172.21.15.115:0/1175843002 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c400a850 con 0x7fb4cc15cd90 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.210+0000 7fb4d31ab640 1 -- 172.21.15.115:0/1175843002 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc15f860 con 0x7fb4cc15cd90 2023-12-01T01:52:22.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.210+0000 7fb4d31ab640 1 -- 172.21.15.115:0/1175843002 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc15fe20 con 0x7fb4cc15cd90 2023-12-01T01:52:22.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.211+0000 7fb4d31ab640 1 -- 172.21.15.115:0/1175843002 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc0b8c00 con 0x7fb4cc15cd90 2023-12-01T01:52:22.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.212+0000 7fb4c97fa640 1 -- 172.21.15.115:0/1175843002 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 42) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fb4c400a9b0 con 0x7fb4cc15cd90 2023-12-01T01:52:22.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.213+0000 7fb4c97fa640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4b006a5a0 0x7fb4b006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.213+0000 7fb4c97fa640 1 -- 172.21.15.115:0/1175843002 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4c4088d90 con 0x7fb4cc15cd90 2023-12-01T01:52:22.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.213+0000 7fb4d0f47640 1 -- 172.21.15.115:0/1175843002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4b006a5a0 msgr2=0x7fb4b006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:22.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.213+0000 7fb4d0f47640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4b006a5a0 0x7fb4b006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:22.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.217+0000 7fb4c97fa640 1 -- 172.21.15.115:0/1175843002 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb4c40541d0 con 0x7fb4cc15cd90 2023-12-01T01:52:22.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.397+0000 7fb4d31ab640 1 -- 172.21.15.115:0/1175843002 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fb4cc04e890 con 0x7fb4cc15cd90 2023-12-01T01:52:22.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.406+0000 7fb4c97fa640 1 -- 172.21.15.115:0/1175843002 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb4c40577f0 con 0x7fb4cc15cd90 2023-12-01T01:52:22.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.408+0000 7fb4d31ab640 1 -- 172.21.15.115:0/1175843002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4b006a5a0 msgr2=0x7fb4b006ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:22.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.408+0000 7fb4d31ab640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4b006a5a0 0x7fb4b006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.408+0000 7fb4d31ab640 1 -- 172.21.15.115:0/1175843002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc15cd90 msgr2=0x7fb4cc1e6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:22.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.408+0000 7fb4d31ab640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc15cd90 0x7fb4cc1e6810 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400c780 tx=0x7fb4c400cc40 comp rx=0 tx=0).stop 2023-12-01T01:52:22.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.408+0000 7fb4d31ab640 1 -- 172.21.15.115:0/1175843002 shutdown_connections 2023-12-01T01:52:22.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.408+0000 7fb4d31ab640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb4b006a5a0 0x7fb4b006ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.408+0000 7fb4d31ab640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4cc15cd90 0x7fb4cc1e6810 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.408+0000 7fb4d31ab640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4cc0cfd60 0x7fb4cc14ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.408+0000 7fb4d31ab640 1 --2- 172.21.15.115:0/1175843002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4cc0b5fa0 0x7fb4cc0cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.408+0000 7fb4d31ab640 1 -- 172.21.15.115:0/1175843002 >> 172.21.15.115:0/1175843002 conn(0x7fb4cc065960 msgr2=0x7fb4cc0b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.409+0000 7fb4d31ab640 1 -- 172.21.15.115:0/1175843002 shutdown_connections 2023-12-01T01:52:22.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.409+0000 7fb4d31ab640 1 -- 172.21.15.115:0/1175843002 wait complete. 2023-12-01T01:52:22.429 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-12-01T01:52:22.570 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:22.569+0000 7f7ec9432040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:52:22.622 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:22.621+0000 7f7ec9432040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:52:22.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.633+0000 7fca220ce640 1 Processor -- start 2023-12-01T01:52:22.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.633+0000 7fca220ce640 1 -- start start 2023-12-01T01:52:22.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.634+0000 7fca220ce640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fca1c071740 0x7fca1c070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.634+0000 7fca220ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1c15b200 0x7fca1c0ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.634+0000 7fca220ce640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fca1c15b6b0 0x7fca1c0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca220ce640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1c0cf950 con 0x7fca1c15b6b0 2023-12-01T01:52:22.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca220ce640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1c15c1f0 con 0x7fca1c071740 2023-12-01T01:52:22.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca220ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0cba40 con 0x7fca1c15b200 2023-12-01T01:52:22.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca1bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fca1c15b6b0 0x7fca1c0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:22.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca1bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fca1c15b6b0 0x7fca1c0cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40518/0 (socket says 172.21.15.115:40518) 2023-12-01T01:52:22.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca1bfff640 1 -- 172.21.15.115:0/675711676 learned_addr learned my addr 172.21.15.115:0/675711676 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:22.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca13fff640 1 --2- 172.21.15.115:0/675711676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1c15b200 0x7fca1c0ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:22.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca1b7fe640 1 --2- 172.21.15.115:0/675711676 >> v2:172.21.15.142:3300/0 conn(0x7fca1c071740 0x7fca1c070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:22.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca1bfff640 1 -- 172.21.15.115:0/675711676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1c15b200 msgr2=0x7fca1c0ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:22.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca1bfff640 1 --2- 172.21.15.115:0/675711676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1c15b200 0x7fca1c0ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca1bfff640 1 -- 172.21.15.115:0/675711676 >> v2:172.21.15.142:3300/0 conn(0x7fca1c071740 msgr2=0x7fca1c070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:22.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca1bfff640 1 --2- 172.21.15.115:0/675711676 >> v2:172.21.15.142:3300/0 conn(0x7fca1c071740 0x7fca1c070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.635+0000 7fca1bfff640 1 -- 172.21.15.115:0/675711676 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c1556f0 con 0x7fca1c15b6b0 2023-12-01T01:52:22.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.636+0000 7fca1bfff640 1 --2- 172.21.15.115:0/675711676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca1c15b6b0 0x7fca1c0cf410 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0d0110 tx=0x7fca1402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a63b1580493a4ef server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:22.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.637+0000 7fca1affd640 1 -- 172.21.15.115:0/675711676 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1402c9e0 con 0x7fca1c15b6b0 2023-12-01T01:52:22.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.637+0000 7fca1affd640 1 -- 172.21.15.115:0/675711676 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1402cb40 con 0x7fca1c15b6b0 2023-12-01T01:52:22.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.637+0000 7fca1affd640 1 -- 172.21.15.115:0/675711676 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca140343f0 con 0x7fca1c15b6b0 2023-12-01T01:52:22.639 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:22.637+0000 7efdf2232040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:52:22.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.637+0000 7fca220ce640 1 -- 172.21.15.115:0/675711676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca1c15b6b0 msgr2=0x7fca1c0cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:22.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.637+0000 7fca220ce640 1 --2- 172.21.15.115:0/675711676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca1c15b6b0 0x7fca1c0cf410 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0d0110 tx=0x7fca1402be40 comp rx=0 tx=0).stop 2023-12-01T01:52:22.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.637+0000 7fca220ce640 1 -- 172.21.15.115:0/675711676 shutdown_connections 2023-12-01T01:52:22.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.637+0000 7fca220ce640 1 --2- 172.21.15.115:0/675711676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca1c15b6b0 0x7fca1c0cf410 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.637+0000 7fca220ce640 1 --2- 172.21.15.115:0/675711676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1c15b200 0x7fca1c0ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.637+0000 7fca220ce640 1 --2- 172.21.15.115:0/675711676 >> v2:172.21.15.142:3300/0 conn(0x7fca1c071740 0x7fca1c070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.637+0000 7fca220ce640 1 -- 172.21.15.115:0/675711676 >> 172.21.15.115:0/675711676 conn(0x7fca1c06bd00 msgr2=0x7fca1c06c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:22.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.638+0000 7fca220ce640 1 -- 172.21.15.115:0/675711676 shutdown_connections 2023-12-01T01:52:22.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.638+0000 7fca220ce640 1 -- 172.21.15.115:0/675711676 wait complete. 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.639+0000 7fca220ce640 1 Processor -- start 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.639+0000 7fca220ce640 1 -- start start 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.640+0000 7fca220ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca1c071740 0x7fca1c14b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.640+0000 7fca220ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1c15b200 0x7fca1c150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.640+0000 7fca220ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1c15b6b0 0x7fca1c142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.640+0000 7fca220ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c14db20 con 0x7fca1c071740 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.640+0000 7fca220ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c14e330 con 0x7fca1c15b6b0 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.640+0000 7fca220ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0b2240 con 0x7fca1c15b200 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca1b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca1c071740 0x7fca1c14b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca1b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca1c071740 0x7fca1c14b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40528/0 (socket says 172.21.15.115:40528) 2023-12-01T01:52:22.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca1b7fe640 1 -- 172.21.15.115:0/308626177 learned_addr learned my addr 172.21.15.115:0/308626177 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca13fff640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1c15b200 0x7fca1c150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca1bfff640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1c15b6b0 0x7fca1c142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca1b7fe640 1 -- 172.21.15.115:0/308626177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1c15b200 msgr2=0x7fca1c150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca1b7fe640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1c15b200 0x7fca1c150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca1b7fe640 1 -- 172.21.15.115:0/308626177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1c15b6b0 msgr2=0x7fca1c142bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca1b7fe640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1c15b6b0 0x7fca1c142bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca1b7fe640 1 -- 172.21.15.115:0/308626177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca14009b30 con 0x7fca1c071740 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca13fff640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1c15b200 0x7fca1c150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.641+0000 7fca1b7fe640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca1c071740 0x7fca1c14b7f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00d650 tx=0x7fca0c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.642+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00b6c0 con 0x7fca1c071740 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.642+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c00b820 con 0x7fca1c071740 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.642+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00bbd0 con 0x7fca1c071740 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.642+0000 7fca220ce640 1 -- 172.21.15.115:0/308626177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c1512c0 con 0x7fca1c071740 2023-12-01T01:52:22.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.642+0000 7fca220ce640 1 -- 172.21.15.115:0/308626177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c14e050 con 0x7fca1c071740 2023-12-01T01:52:22.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.643+0000 7fca220ce640 1 -- 172.21.15.115:0/308626177 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca1c073590 con 0x7fca1c071740 2023-12-01T01:52:22.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.644+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 42) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fca0c00bd30 con 0x7fca1c071740 2023-12-01T01:52:22.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.645+0000 7fca18ff9640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 0x7fc9fc06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:22.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.645+0000 7fca13fff640 1 -- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 msgr2=0x7fc9fc06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:22.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.645+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca0c088f70 con 0x7fca1c071740 2023-12-01T01:52:22.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.646+0000 7fca13fff640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 0x7fc9fc06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:22.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.650+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fca0c054440 con 0x7fca1c071740 2023-12-01T01:52:22.674 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:22.673+0000 7f7ec9432040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:52:22.693 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:22.692+0000 7efdf2232040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:52:22.736 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:22.735+0000 7f7ec9432040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:52:22.746 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:22.745+0000 7efdf2232040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:52:22.784 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:22.783+0000 7f7ec9432040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:52:22.809 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:22.807+0000 7efdf2232040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:52:22.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.847+0000 7fca13fff640 1 -- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 msgr2=0x7fc9fc06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:22.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.847+0000 7fca13fff640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 0x7fc9fc06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:52:22.859 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:22.858+0000 7efdf2232040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:52:22.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:22.864+0000 7fca220ce640 1 -- 172.21.15.115:0/308626177 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fca1c15bc30 con 0x7fc9fc06a5a0 2023-12-01T01:52:22.895 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:22.894+0000 7f7ec9432040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:52:22.957 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:22.956+0000 7f7ec9432040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:52:22.971 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:22.970+0000 7efdf2232040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:52:23.006 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:23.006+0000 7f7ec9432040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:52:23.033 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:23.032+0000 7efdf2232040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:52:23.064 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:23.063+0000 7f7ec9432040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:52:23.084 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:23.082+0000 7efdf2232040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:52:23.115 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:23.114+0000 7f7ec9432040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:52:23.141 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:23.139+0000 7efdf2232040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:52:23.191 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:23.189+0000 7efdf2232040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:52:23.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:23.248+0000 7fca13fff640 1 -- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 msgr2=0x7fc9fc06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:23.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:23.248+0000 7fca13fff640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 0x7fc9fc06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:52:23.499 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:23.498+0000 7f7ec9432040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:52:23.568 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:23.566+0000 7efdf2232040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:52:23.658 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:23.657+0000 7f7ec9432040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:52:23.725 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:23.724+0000 7efdf2232040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:52:23.766 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:23.765+0000 7f7ec9432040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:52:23.817 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:23.816+0000 7f7ec9432040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:52:23.832 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:23.831+0000 7efdf2232040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:52:23.882 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:23.880+0000 7efdf2232040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:52:23.925 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:23.924+0000 7f7ec9432040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:52:23.988 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:23.987+0000 7efdf2232040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:52:24.031 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:24.030+0000 7f7ec9432040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:52:24.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:24.049+0000 7fca13fff640 1 -- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 msgr2=0x7fc9fc06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:24.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:24.049+0000 7fca13fff640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 0x7fc9fc06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:52:24.094 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:24.093+0000 7efdf2232040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:52:24.187 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:24.186+0000 7f7ec9432040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:52:24.236 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:24.236+0000 7f7ec9432040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:52:24.250 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:24.248+0000 7efdf2232040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:52:24.298 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:24.296+0000 7efdf2232040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:52:24.355 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:24.354+0000 7f7ec9432040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:52:24.415 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:24.414+0000 7efdf2232040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:52:25.163 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:25.162+0000 7f7ec9432040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:52:25.242 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.241+0000 7efdf2232040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:52:25.457 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:25.456+0000 7f7ec9432040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:52:25.516 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:25.515+0000 7f7ec9432040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:52:25.562 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.561+0000 7efdf2232040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:52:25.618 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.618+0000 7efdf2232040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:52:25.630 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:25.629+0000 7f7ec9432040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:52:25.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:25.652+0000 7fca13fff640 1 -- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 msgr2=0x7fc9fc06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:25.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:25.652+0000 7fca13fff640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 0x7fc9fc06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-01T01:52:25.733 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.732+0000 7efdf2232040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:52:25.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:25.750+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 43) v1 ==== 88335+0+0 (secure 0 0 0) 0x7fca0c054810 con 0x7fca1c071740 2023-12-01T01:52:25.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:25.750+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 msgr2=0x7fc9fc06ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:25.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:25.750+0000 7fca18ff9640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 0x7fc9fc06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:25.866 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.865+0000 7efd7168a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:25.866 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.865+0000 7efd7168a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:25.866 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.865+0000 7efd7168a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:25.866 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.865+0000 7efd7168a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:25.866 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.865+0000 7efd7168a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:25.866 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.865+0000 7efd6e684640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:25.866 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.865+0000 7efd6e684640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:25.867 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.865+0000 7efd6e684640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:25.867 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.865+0000 7efd6e684640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:25.867 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:25.865+0000 7efd6e684640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:26.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:26.767+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 44) v1 ==== 88462+0+0 (secure 0 0 0) 0x7fca0c06f900 con 0x7fca1c071740 2023-12-01T01:52:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:26.768+0000 7fca18ff9640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 0x7fc9fc06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:26.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:26.768+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9fc027f80 con 0x7fc9fc06a5a0 2023-12-01T01:52:26.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:26.771+0000 7fca13fff640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 0x7fc9fc06ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:26.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:26.776+0000 7fca13fff640 1 --2- 172.21.15.115:0/308626177 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9fc06a5a0 0x7fc9fc06ca60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fca1c15e440 tx=0x7fca04008040 comp rx=0 tx=0).ready entity=mgr.7149 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:26.798 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:26.797+0000 7efd786d8640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-01T01:52:26.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:26.798+0000 7fca18ff9640 1 -- 172.21.15.115:0/308626177 <== mgr.7149 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fc9fc027f80 con 0x7fc9fc06a5a0 2023-12-01T01:52:26.799 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-01T01:52:26.812 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-01T01:52:26.813 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T01:52:27.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.016+0000 7f8cc34e5640 1 Processor -- start 2023-12-01T01:52:27.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.016+0000 7f8cc34e5640 1 -- start start 2023-12-01T01:52:27.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.017+0000 7f8cc34e5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8cbc15ac80 0x7f8cbc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:27.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.017+0000 7f8cc34e5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8cbc0ce1b0 0x7f8cbc0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:27.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.017+0000 7f8cc34e5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc0cc9a0 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.017+0000 7f8cc34e5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cbc15b060 con 0x7f8cbc15ac80 2023-12-01T01:52:27.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.017+0000 7f8cc34e5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cbc0b0ce0 con 0x7f8cbc0ce1b0 2023-12-01T01:52:27.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.018+0000 7f8cc1281640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8cbc15ac80 0x7f8cbc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:27.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.018+0000 7f8cc1281640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8cbc15ac80 0x7f8cbc0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40700/0 (socket says 172.21.15.115:40700) 2023-12-01T01:52:27.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.018+0000 7f8cc1281640 1 -- 172.21.15.115:0/2971237511 learned_addr learned my addr 172.21.15.115:0/2971237511 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:27.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.018+0000 7f8cc0a80640 1 --1- 172.21.15.115:0/2971237511 >> v1:172.21.15.115:6789/0 conn(0x7f8cbc0b0e60 0x7f8cbc0cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:45964/0 (socket says 172.21.15.115:45964) 2023-12-01T01:52:27.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.018+0000 7f8cc1281640 1 -- 172.21.15.115:0/2971237511 >> v2:172.21.15.142:3300/0 conn(0x7f8cbc0ce1b0 msgr2=0x7f8cbc0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:27.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.018+0000 7f8cc1281640 1 --2- 172.21.15.115:0/2971237511 >> v2:172.21.15.142:3300/0 conn(0x7f8cbc0ce1b0 0x7f8cbc0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.018+0000 7f8cc1281640 1 -- 172.21.15.115:0/2971237511 >> v1:172.21.15.115:6789/0 conn(0x7f8cbc0b0e60 legacy=0x7f8cbc0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:27.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.018+0000 7f8cc1281640 1 -- 172.21.15.115:0/2971237511 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc0cbd90 con 0x7f8cbc15ac80 2023-12-01T01:52:27.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.019+0000 7f8cc1281640 1 --2- 172.21.15.115:0/2971237511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc15ac80 0x7f8cbc0cce30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc06f1b0 tx=0x7f8cb802b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28927e10b11127c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:27.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.019+0000 7f8cb3fff640 1 -- 172.21.15.115:0/2971237511 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8030880 con 0x7f8cbc15ac80 2023-12-01T01:52:27.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.019+0000 7f8cb3fff640 1 -- 172.21.15.115:0/2971237511 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb80309e0 con 0x7f8cbc15ac80 2023-12-01T01:52:27.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.019+0000 7f8cb3fff640 1 -- 172.21.15.115:0/2971237511 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8030d70 con 0x7f8cbc15ac80 2023-12-01T01:52:27.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.020+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2971237511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc15ac80 msgr2=0x7f8cbc0cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:27.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.020+0000 7f8cc34e5640 1 --2- 172.21.15.115:0/2971237511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc15ac80 0x7f8cbc0cce30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc06f1b0 tx=0x7f8cb802b9c0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.020+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2971237511 shutdown_connections 2023-12-01T01:52:27.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.020+0000 7f8cc34e5640 1 --2- 172.21.15.115:0/2971237511 >> v2:172.21.15.142:3300/0 conn(0x7f8cbc0ce1b0 0x7f8cbc0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.020+0000 7f8cc34e5640 1 --2- 172.21.15.115:0/2971237511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc15ac80 0x7f8cbc0cce30 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.020+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2971237511 >> 172.21.15.115:0/2971237511 conn(0x7f8cbc065960 msgr2=0x7f8cbc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:27.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.020+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2971237511 shutdown_connections 2023-12-01T01:52:27.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.020+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2971237511 wait complete. 2023-12-01T01:52:27.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.022+0000 7f8cc34e5640 1 Processor -- start 2023-12-01T01:52:27.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.022+0000 7f8cc34e5640 1 -- start start 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc34e5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc0b0e60 0x7f8cbc1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc34e5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cbc0ce1b0 0x7f8cbc1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc34e5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cbc15ac80 0x7f8cbc157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc34e5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0a8990 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc34e5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc158920 con 0x7f8cbc15ac80 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc34e5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc155740 con 0x7f8cbc0ce1b0 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc1281640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc0b0e60 0x7f8cbc1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc1281640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc0b0e60 0x7f8cbc1591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40710/0 (socket says 172.21.15.115:40710) 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc1281640 1 -- 172.21.15.115:0/2756277343 learned_addr learned my addr 172.21.15.115:0/2756277343 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc1281640 1 -- 172.21.15.115:0/2756277343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cbc0ce1b0 msgr2=0x7f8cbc1577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc1281640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cbc0ce1b0 0x7f8cbc1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc1281640 1 -- 172.21.15.115:0/2756277343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cbc15ac80 msgr2=0x7f8cbc157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:27.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc1281640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cbc15ac80 0x7f8cbc157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.023+0000 7f8cc1281640 1 -- 172.21.15.115:0/2756277343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb80094d0 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.024+0000 7f8cc1281640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc0b0e60 0x7f8cbc1591f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8000c00 tx=0x7f8cb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:27.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.024+0000 7f8cb1ffb640 1 -- 172.21.15.115:0/2756277343 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8030850 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.024+0000 7f8cb1ffb640 1 -- 172.21.15.115:0/2756277343 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb8039d90 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.024+0000 7f8cb1ffb640 1 -- 172.21.15.115:0/2756277343 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8043630 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.024+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2756277343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc158360 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.024+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2756277343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc0a85b0 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.025+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2756277343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc04e890 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.026+0000 7f8cb1ffb640 1 -- 172.21.15.115:0/2756277343 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 44) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f8cb802f3d0 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.027+0000 7f8cb1ffb640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ca006a2d0 0x7f8ca006c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:27.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.027+0000 7f8cb1ffb640 1 -- 172.21.15.115:0/2756277343 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f8cb80aac90 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.027+0000 7f8cc0a80640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ca006a2d0 0x7f8ca006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:27.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.028+0000 7f8cc0a80640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ca006a2d0 0x7f8ca006c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0b3330 tx=0x7f8cac0079c0 comp rx=0 tx=0).ready entity=mgr.7149 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:27.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.032+0000 7f8cb1ffb640 1 -- 172.21.15.115:0/2756277343 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8cb8076390 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.271+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2756277343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f8cbc04e040 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.781+0000 7f8cb1ffb640 1 -- 172.21.15.115:0/2756277343 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 45) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f8cb8076760 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.782+0000 7f8cb1ffb640 1 -- 172.21.15.115:0/2756277343 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v45) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8cb808da80 con 0x7f8cbc0b0e60 2023-12-01T01:52:27.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.786+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2756277343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ca006a2d0 msgr2=0x7f8ca006c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:27.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.786+0000 7f8cc34e5640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ca006a2d0 0x7f8ca006c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0b3330 tx=0x7f8cac0079c0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.786+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2756277343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc0b0e60 msgr2=0x7f8cbc1591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:27.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.786+0000 7f8cc34e5640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc0b0e60 0x7f8cbc1591f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8000c00 tx=0x7f8cb8004730 comp rx=0 tx=0).stop 2023-12-01T01:52:27.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.786+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2756277343 shutdown_connections 2023-12-01T01:52:27.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.786+0000 7f8cc34e5640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ca006a2d0 0x7f8ca006c790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.786+0000 7f8cc34e5640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cbc0ce1b0 0x7f8cbc1577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.786+0000 7f8cc34e5640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cbc15ac80 0x7f8cbc157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.786+0000 7f8cc34e5640 1 --2- 172.21.15.115:0/2756277343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cbc0b0e60 0x7f8cbc1591f0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:27.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.786+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2756277343 >> 172.21.15.115:0/2756277343 conn(0x7f8cbc065960 msgr2=0x7f8cbc06fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:27.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.787+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2756277343 shutdown_connections 2023-12-01T01:52:27.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:27.787+0000 7f8cc34e5640 1 -- 172.21.15.115:0/2756277343 wait complete. 2023-12-01T01:52:27.809 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:52:27.809 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:52:27.915 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:52:28.019 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:52:28.020 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:52:28.020 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:28.072 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:52:28.072 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:28.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:28.277 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:28.277+0000 7f7ad0e46040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:52:28.317 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:28.317+0000 7fbee1631040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:52:28.329 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:28.329+0000 7f7ad0e46040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:52:28.371 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:28.371+0000 7fbee1631040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:52:28.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.405+0000 7fc3797d0640 1 Processor -- start 2023-12-01T01:52:28.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.405+0000 7fc3797d0640 1 -- start start 2023-12-01T01:52:28.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc3797d0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc3740cad70 0x7fc37406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc3797d0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc37406f2f0 0x7fc37406bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc3797d0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3740add10 con 0x7fc37415e2f0 2023-12-01T01:52:28.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc3797d0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3740c5760 con 0x7fc3740cad70 2023-12-01T01:52:28.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc3797d0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37415e6d0 con 0x7fc37406f2f0 2023-12-01T01:52:28.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc36bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc3740cad70 0x7fc37406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc36bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc3740cad70 0x7fc37406b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37376/0 (socket says 172.21.15.115:37376) 2023-12-01T01:52:28.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc36bfff640 1 -- 172.21.15.115:0/2357023699 learned_addr learned my addr 172.21.15.115:0/2357023699 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:28.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc36bfff640 1 -- 172.21.15.115:0/2357023699 >> v2:172.21.15.142:3300/0 conn(0x7fc37406f2f0 msgr2=0x7fc37406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:28.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc36bfff640 1 --2- 172.21.15.115:0/2357023699 >> v2:172.21.15.142:3300/0 conn(0x7fc37406f2f0 0x7fc37406bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.407+0000 7fc36bfff640 1 -- 172.21.15.115:0/2357023699 >> v1:172.21.15.142:6789/0 conn(0x7fc37415e2f0 legacy=0x7fc3740ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:28.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.408+0000 7fc36bfff640 1 -- 172.21.15.115:0/2357023699 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3740cf7b0 con 0x7fc3740cad70 2023-12-01T01:52:28.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.409+0000 7fc36bfff640 1 --2- 172.21.15.115:0/2357023699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 0x7fc37406b5c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009870 tx=0x7fc35c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd61b6925eaf2102 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:28.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.409+0000 7fc3727fc640 1 -- 172.21.15.115:0/2357023699 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c031800 con 0x7fc3740cad70 2023-12-01T01:52:28.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.409+0000 7fc3727fc640 1 -- 172.21.15.115:0/2357023699 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c031960 con 0x7fc3740cad70 2023-12-01T01:52:28.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.409+0000 7fc3727fc640 1 -- 172.21.15.115:0/2357023699 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c031cf0 con 0x7fc3740cad70 2023-12-01T01:52:28.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.410+0000 7fc3797d0640 1 -- 172.21.15.115:0/2357023699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 msgr2=0x7fc37406b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:28.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.410+0000 7fc3797d0640 1 --2- 172.21.15.115:0/2357023699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 0x7fc37406b5c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009870 tx=0x7fc35c02b8a0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.410+0000 7fc3797d0640 1 -- 172.21.15.115:0/2357023699 shutdown_connections 2023-12-01T01:52:28.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.410+0000 7fc3797d0640 1 --2- 172.21.15.115:0/2357023699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 0x7fc37406b5c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.410+0000 7fc3797d0640 1 --2- 172.21.15.115:0/2357023699 >> v2:172.21.15.142:3300/0 conn(0x7fc37406f2f0 0x7fc37406bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.410+0000 7fc3797d0640 1 -- 172.21.15.115:0/2357023699 >> 172.21.15.115:0/2357023699 conn(0x7fc3740ce880 msgr2=0x7fc3740cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:28.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.410+0000 7fc3797d0640 1 -- 172.21.15.115:0/2357023699 shutdown_connections 2023-12-01T01:52:28.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.410+0000 7fc3797d0640 1 -- 172.21.15.115:0/2357023699 wait complete. 2023-12-01T01:52:28.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.411+0000 7fc3797d0640 1 Processor -- start 2023-12-01T01:52:28.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.412+0000 7fc3797d0640 1 -- start start 2023-12-01T01:52:28.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc3797d0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37406f2f0 0x7fc3741f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc3797d0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 0x7fc3740c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc3797d0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37415e2f0 0x7fc3740c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc3797d0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c28f0 con 0x7fc3740cad70 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc3797d0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c2b00 con 0x7fc37406f2f0 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc3797d0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c2de0 con 0x7fc37415e2f0 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc36bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 0x7fc3740c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc36bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 0x7fc3740c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37382/0 (socket says 172.21.15.115:37382) 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc36bfff640 1 -- 172.21.15.115:0/3133997777 learned_addr learned my addr 172.21.15.115:0/3133997777 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc372ffd640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37406f2f0 0x7fc3741f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc3737fe640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37415e2f0 0x7fc3740c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc36bfff640 1 -- 172.21.15.115:0/3133997777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37415e2f0 msgr2=0x7fc3740c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc36bfff640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37415e2f0 0x7fc3740c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc36bfff640 1 -- 172.21.15.115:0/3133997777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37406f2f0 msgr2=0x7fc3741f0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc36bfff640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37406f2f0 0x7fc3741f0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc36bfff640 1 -- 172.21.15.115:0/3133997777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0094d0 con 0x7fc3740cad70 2023-12-01T01:52:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc3737fe640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37415e2f0 0x7fc3740c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.413+0000 7fc372ffd640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37406f2f0 0x7fc3741f0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.414+0000 7fc36bfff640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 0x7fc3740c23b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc35c000c00 tx=0x7fc35c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.414+0000 7fc36b7fe640 1 -- 172.21.15.115:0/3133997777 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c03ad30 con 0x7fc3740cad70 2023-12-01T01:52:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.414+0000 7fc36b7fe640 1 -- 172.21.15.115:0/3133997777 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c03ae90 con 0x7fc3740cad70 2023-12-01T01:52:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.414+0000 7fc36b7fe640 1 -- 172.21.15.115:0/3133997777 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c044610 con 0x7fc3740cad70 2023-12-01T01:52:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.414+0000 7fc3797d0640 1 -- 172.21.15.115:0/3133997777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3740c30c0 con 0x7fc3740cad70 2023-12-01T01:52:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.414+0000 7fc3797d0640 1 -- 172.21.15.115:0/3133997777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3741555c0 con 0x7fc3740cad70 2023-12-01T01:52:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.416+0000 7fc3797d0640 1 -- 172.21.15.115:0/3133997777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37406fda0 con 0x7fc3740cad70 2023-12-01T01:52:28.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.416+0000 7fc36b7fe640 1 -- 172.21.15.115:0/3133997777 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 45) v1 ==== 88710+0+0 (secure 0 0 0) 0x7fc35c04d450 con 0x7fc3740cad70 2023-12-01T01:52:28.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.417+0000 7fc36b7fe640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc35406a550 0x7fc35406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.417+0000 7fc36b7fe640 1 -- 172.21.15.115:0/3133997777 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fc35c02e070 con 0x7fc3740cad70 2023-12-01T01:52:28.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.417+0000 7fc372ffd640 1 -- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc35406a550 msgr2=0x7fc35406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:28.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.417+0000 7fc372ffd640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc35406a550 0x7fc35406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:28.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.426+0000 7fc36b7fe640 1 -- 172.21.15.115:0/3133997777 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc35c076450 con 0x7fc3740cad70 2023-12-01T01:52:28.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.618+0000 7fc372ffd640 1 -- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc35406a550 msgr2=0x7fc35406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:28.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.618+0000 7fc372ffd640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc35406a550 0x7fc35406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:52:28.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.618+0000 7fc3797d0640 1 -- 172.21.15.115:0/3133997777 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3740ca110 con 0x7fc3740cad70 2023-12-01T01:52:28.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.620+0000 7fc36b7fe640 1 -- 172.21.15.115:0/3133997777 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16586 (secure 0 0 0) 0x7fc35c079a70 con 0x7fc3740cad70 2023-12-01T01:52:28.621 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:28.621 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":98,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:25.735054+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:04.909384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:05.480768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6822/88317":"2023-12-02T01:52:25.734961+0000","172.21.15.115:6824/88317":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/3373285076":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/3569012860":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/709337731":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/1296968550":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/1490819667":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/1878415381":"2023-12-02T01:52:25.734961+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:28.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 -- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc35406a550 msgr2=0x7fc35406ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:28.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc35406a550 0x7fc35406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 -- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 msgr2=0x7fc3740c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:28.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 0x7fc3740c23b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc35c000c00 tx=0x7fc35c0046c0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 -- 172.21.15.115:0/3133997777 shutdown_connections 2023-12-01T01:52:28.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc35406a550 0x7fc35406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc37415e2f0 0x7fc3740c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3740cad70 0x7fc3740c23b0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 --2- 172.21.15.115:0/3133997777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc37406f2f0 0x7fc3741f0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 -- 172.21.15.115:0/3133997777 >> 172.21.15.115:0/3133997777 conn(0x7fc3740ce880 msgr2=0x7fc3740aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:28.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 -- 172.21.15.115:0/3133997777 shutdown_connections 2023-12-01T01:52:28.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.623+0000 7fc3797d0640 1 -- 172.21.15.115:0/3133997777 wait complete. 2023-12-01T01:52:28.642 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:52:04.909384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:52:05.480768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:52:28.642 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:52:28.775 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:28.775+0000 7f7ad0e46040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:52:28.828 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:28.827+0000 7f7ad0e46040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:52:28.845 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:28.844+0000 7fbee1631040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:52:28.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.854+0000 7f5f2217e640 1 Processor -- start 2023-12-01T01:52:28.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.854+0000 7f5f2217e640 1 -- start start 2023-12-01T01:52:28.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.855+0000 7f5f2217e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c153c30 0x7f5f1c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.856+0000 7f5f2217e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0cdcd0 con 0x7f5f1c154010 2023-12-01T01:52:28.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.856+0000 7f5f2217e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c161830 con 0x7f5f1c0cfb80 2023-12-01T01:52:28.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.856+0000 7f5f2217e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0cd3a0 con 0x7f5f1c153c30 2023-12-01T01:52:28.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.856+0000 7f5f2097b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5f1c154010 0x7f5f1c0cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51356/0 (socket says 172.21.15.115:51356) 2023-12-01T01:52:28.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.856+0000 7f5f2097b640 1 -- 172.21.15.115:0/783514840 learned_addr learned my addr 172.21.15.115:0/783514840 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.856+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194731731 0 0) 0x7f5f1c0cdcd0 con 0x7f5f1c154010 2023-12-01T01:52:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.856+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f08003520 con 0x7f5f1c154010 2023-12-01T01:52:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.856+0000 7f5f2117c640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c153c30 0x7f5f1c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1955703398 0 0) 0x7f5f08003520 con 0x7f5f1c154010 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c153c30 msgr2=0x7f5f1c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c153c30 0x7f5f1c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 >> v1:172.21.15.142:6789/0 conn(0x7f5f1c0cfb80 legacy=0x7f5f1c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0cd680 con 0x7f5f1c154010 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5f0c001670 con 0x7f5f1c154010 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f2117c640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c153c30 0x7f5f1c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 >> v1:172.21.15.115:6789/0 conn(0x7f5f1c154010 legacy=0x7f5f1c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 0x7f5f08003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f080047c0 0x7f5f08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c153c30 0x7f5f080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0cd3a0 con 0x7f5f080039a0 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0800bb10 con 0x7f5f080047c0 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0800bc80 con 0x7f5f1c153c30 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f2117c640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 0x7f5f08003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.858+0000 7f5f2097b640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f080047c0 0x7f5f08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.857+0000 7f5f2197d640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c153c30 0x7f5f080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.858+0000 7f5f2117c640 1 -- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c153c30 msgr2=0x7f5f080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:28.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.858+0000 7f5f2117c640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c153c30 0x7f5f080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.858+0000 7f5f2117c640 1 -- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f080047c0 msgr2=0x7f5f08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:28.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.858+0000 7f5f2117c640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f080047c0 0x7f5f08004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.858+0000 7f5f2117c640 1 -- 172.21.15.115:0/783514840 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f08003520 con 0x7f5f080039a0 2023-12-01T01:52:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.859+0000 7f5f2117c640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 0x7f5f08003dc0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5f180098a0 tx=0x7f5f080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82bbc141de66a7a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.860+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18004230 con 0x7f5f080039a0 2023-12-01T01:52:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.860+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18004390 con 0x7f5f080039a0 2023-12-01T01:52:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.860+0000 7f5f13fff640 1 -- 172.21.15.115:0/783514840 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18003830 con 0x7f5f080039a0 2023-12-01T01:52:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.860+0000 7f5f2217e640 1 -- 172.21.15.115:0/783514840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 msgr2=0x7f5f08003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.860+0000 7f5f2217e640 1 --2- 172.21.15.115:0/783514840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 0x7f5f08003dc0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5f180098a0 tx=0x7f5f080057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.860+0000 7f5f2197d640 1 -- 172.21.15.115:0/783514840 reap_dead start 2023-12-01T01:52:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.860+0000 7f5f2217e640 1 -- 172.21.15.115:0/783514840 shutdown_connections 2023-12-01T01:52:28.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.860+0000 7f5f2217e640 1 -- 172.21.15.115:0/783514840 >> 172.21.15.115:0/783514840 conn(0x7f5f1c06a0b0 msgr2=0x7f5f1c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:28.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.860+0000 7f5f2217e640 1 -- 172.21.15.115:0/783514840 shutdown_connections 2023-12-01T01:52:28.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.861+0000 7f5f2217e640 1 -- 172.21.15.115:0/783514840 wait complete. 2023-12-01T01:52:28.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.862+0000 7f5f2217e640 1 Processor -- start 2023-12-01T01:52:28.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.862+0000 7f5f2217e640 1 -- start start 2023-12-01T01:52:28.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.863+0000 7f5f2217e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 0x7f5f1c0c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.863+0000 7f5f2217e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c150270 0x7f5f1c0c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.863+0000 7f5f2217e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f1c0c56e0 0x7f5f1c0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.863+0000 7f5f2217e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c150af0 con 0x7f5f080039a0 2023-12-01T01:52:28.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.863+0000 7f5f2217e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c1562b0 con 0x7f5f1c0c56e0 2023-12-01T01:52:28.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.863+0000 7f5f2217e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c9450 con 0x7f5f1c150270 2023-12-01T01:52:28.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.863+0000 7f5f2117c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 0x7f5f1c0c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.864+0000 7f5f2117c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 0x7f5f1c0c8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37408/0 (socket says 172.21.15.115:37408) 2023-12-01T01:52:28.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.864+0000 7f5f2117c640 1 -- 172.21.15.115:0/788522906 learned_addr learned my addr 172.21.15.115:0/788522906 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:28.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.864+0000 7f5f2097b640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c150270 0x7f5f1c0c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.864+0000 7f5f2197d640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f1c0c56e0 0x7f5f1c0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.864+0000 7f5f2117c640 1 -- 172.21.15.115:0/788522906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c150270 msgr2=0x7f5f1c0c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.864+0000 7f5f2117c640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c150270 0x7f5f1c0c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.864+0000 7f5f2117c640 1 -- 172.21.15.115:0/788522906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f1c0c56e0 msgr2=0x7f5f1c0c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.864+0000 7f5f2117c640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f1c0c56e0 0x7f5f1c0c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.864+0000 7f5f2117c640 1 -- 172.21.15.115:0/788522906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0ca150 con 0x7f5f080039a0 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.864+0000 7f5f2117c640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 0x7f5f1c0c8f10 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5f180098a0 tx=0x7f5f18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.865+0000 7f5f11ffb640 1 -- 172.21.15.115:0/788522906 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18030630 con 0x7f5f080039a0 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.865+0000 7f5f11ffb640 1 -- 172.21.15.115:0/788522906 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18030790 con 0x7f5f080039a0 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.865+0000 7f5f11ffb640 1 -- 172.21.15.115:0/788522906 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18030b00 con 0x7f5f080039a0 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.865+0000 7f5f2217e640 1 -- 172.21.15.115:0/788522906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c1556a0 con 0x7f5f080039a0 2023-12-01T01:52:28.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.865+0000 7f5f2217e640 1 -- 172.21.15.115:0/788522906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c155c60 con 0x7f5f080039a0 2023-12-01T01:52:28.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.866+0000 7f5f2217e640 1 -- 172.21.15.115:0/788522906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c150af0 con 0x7f5f080039a0 2023-12-01T01:52:28.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.867+0000 7f5f11ffb640 1 -- 172.21.15.115:0/788522906 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 45) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f5f18030c60 con 0x7f5f080039a0 2023-12-01T01:52:28.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.868+0000 7f5f11ffb640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0006a580 0x7f5f0006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:28.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.868+0000 7f5f11ffb640 1 -- 172.21.15.115:0/788522906 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f5f180aa870 con 0x7f5f080039a0 2023-12-01T01:52:28.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.868+0000 7f5f2097b640 1 -- 172.21.15.115:0/788522906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0006a580 msgr2=0x7f5f0006ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:28.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.868+0000 7f5f2097b640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0006a580 0x7f5f0006ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:28.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:28.876+0000 7f5f11ffb640 1 -- 172.21.15.115:0/788522906 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5f18075ca0 con 0x7f5f080039a0 2023-12-01T01:52:28.879 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:28.879+0000 7f7ad0e46040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:52:28.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:28.904+0000 7fbee1631040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:52:28.940 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:28.939+0000 7f7ad0e46040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:52:28.956 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:28.956+0000 7fbee1631040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:52:28.988 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:28.987+0000 7f7ad0e46040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:52:29.019 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:29.018+0000 7fbee1631040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:52:29.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.065+0000 7f5f2217e640 1 -- 172.21.15.115:0/788522906 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f1c0618d0 con 0x7f5f080039a0 2023-12-01T01:52:29.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.066+0000 7f5f11ffb640 1 -- 172.21.15.115:0/788522906 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f18048020 con 0x7f5f080039a0 2023-12-01T01:52:29.066 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:52:29.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.068+0000 7f5f2217e640 1 -- 172.21.15.115:0/788522906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0006a580 msgr2=0x7f5f0006ca40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:29.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.068+0000 7f5f2217e640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0006a580 0x7f5f0006ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.068+0000 7f5f2217e640 1 -- 172.21.15.115:0/788522906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 msgr2=0x7f5f1c0c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.068+0000 7f5f2217e640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 0x7f5f1c0c8f10 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5f180098a0 tx=0x7f5f18004730 comp rx=0 tx=0).stop 2023-12-01T01:52:29.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.068+0000 7f5f2217e640 1 -- 172.21.15.115:0/788522906 shutdown_connections 2023-12-01T01:52:29.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.068+0000 7f5f2217e640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f0006a580 0x7f5f0006ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.068+0000 7f5f2217e640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f1c150270 0x7f5f1c0c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.068+0000 7f5f2217e640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f1c0c56e0 0x7f5f1c0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.068+0000 7f5f2217e640 1 --2- 172.21.15.115:0/788522906 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f080039a0 0x7f5f1c0c8f10 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.068+0000 7f5f2217e640 1 -- 172.21.15.115:0/788522906 >> 172.21.15.115:0/788522906 conn(0x7f5f1c06a0b0 msgr2=0x7f5f1c15c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:29.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.069+0000 7f5f2217e640 1 -- 172.21.15.115:0/788522906 shutdown_connections 2023-12-01T01:52:29.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.069+0000 7f5f2217e640 1 -- 172.21.15.115:0/788522906 wait complete. 2023-12-01T01:52:29.071 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:29.070+0000 7fbee1631040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:52:29.086 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:52:29.096 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:29.096+0000 7f7ad0e46040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:52:29.158 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:29.158+0000 7f7ad0e46040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:52:29.189 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:29.188+0000 7fbee1631040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:52:29.207 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:29.206+0000 7f7ad0e46040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:52:29.254 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:29.254+0000 7fbee1631040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:52:29.263 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:29.262+0000 7f7ad0e46040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:52:29.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.286+0000 7f57c0399640 1 Processor -- start 2023-12-01T01:52:29.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.286+0000 7f57c0399640 1 -- start start 2023-12-01T01:52:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.287+0000 7f57c0399640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80b5fa0 0x7f57b80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.287+0000 7f57c0399640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b80ce5e0 con 0x7f57b815cd90 2023-12-01T01:52:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.287+0000 7f57c0399640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b80ce300 con 0x7f57b80cfd60 2023-12-01T01:52:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.287+0000 7f57c0399640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80d0ec0 con 0x7f57b80b5fa0 2023-12-01T01:52:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57be135640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f57b815cd90 0x7f57b80cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51368/0 (socket says 172.21.15.115:51368) 2023-12-01T01:52:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57be135640 1 -- 172.21.15.115:0/1652888919 learned_addr learned my addr 172.21.15.115:0/1652888919 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57be936640 1 --2- 172.21.15.115:0/1652888919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80b5fa0 0x7f57b80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939461629 0 0) 0x7f57b4001110 con 0x7f57b815cd90 2023-12-01T01:52:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a4003520 con 0x7f57b815cd90 2023-12-01T01:52:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1535405288 0 0) 0x7f57a4003520 con 0x7f57b815cd90 2023-12-01T01:52:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80b5fa0 msgr2=0x7f57b80cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57bd133640 1 --2- 172.21.15.115:0/1652888919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80b5fa0 0x7f57b80cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 >> v1:172.21.15.142:6789/0 conn(0x7f57b80cfd60 legacy=0x7f57b80d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80d10a0 con 0x7f57b815cd90 2023-12-01T01:52:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f57b4001800 con 0x7f57b815cd90 2023-12-01T01:52:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 >> v1:172.21.15.115:6789/0 conn(0x7f57b815cd90 legacy=0x7f57b80cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.288+0000 7f57bd133640 1 --2- 172.21.15.115:0/1652888919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a40039a0 0x7f57a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57bd133640 1 --2- 172.21.15.115:0/1652888919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80b5fa0 0x7f57a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57bd133640 1 --2- 172.21.15.115:0/1652888919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4004d00 0x7f57a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80d0ec0 con 0x7f57a4004d00 2023-12-01T01:52:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a400bb60 con 0x7f57a40039a0 2023-12-01T01:52:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a400bd20 con 0x7f57b80b5fa0 2023-12-01T01:52:29.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57be936640 1 --2- 172.21.15.115:0/1652888919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4004d00 0x7f57a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:29.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57be936640 1 -- 172.21.15.115:0/1652888919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80b5fa0 msgr2=0x7f57a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:29.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57be936640 1 --2- 172.21.15.115:0/1652888919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80b5fa0 0x7f57a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57be936640 1 -- 172.21.15.115:0/1652888919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a40039a0 msgr2=0x7f57a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:29.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57be936640 1 --2- 172.21.15.115:0/1652888919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a40039a0 0x7f57a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.289+0000 7f57be936640 1 -- 172.21.15.115:0/1652888919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a4003520 con 0x7f57a4004d00 2023-12-01T01:52:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.290+0000 7f57be936640 1 --2- 172.21.15.115:0/1652888919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4004d00 0x7f57a4005180 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f57b0007360 tx=0x7f57a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc7d13d27d70d8e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.291+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b002ccf0 con 0x7f57a4004d00 2023-12-01T01:52:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.291+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b002ce50 con 0x7f57a4004d00 2023-12-01T01:52:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.291+0000 7f57bd133640 1 -- 172.21.15.115:0/1652888919 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0034660 con 0x7f57a4004d00 2023-12-01T01:52:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.291+0000 7f57c0399640 1 -- 172.21.15.115:0/1652888919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4004d00 msgr2=0x7f57a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.291+0000 7f57c0399640 1 --2- 172.21.15.115:0/1652888919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a4004d00 0x7f57a4005180 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f57b0007360 tx=0x7f57a40057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.291+0000 7f57be936640 1 -- 172.21.15.115:0/1652888919 reap_dead start 2023-12-01T01:52:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.291+0000 7f57c0399640 1 -- 172.21.15.115:0/1652888919 shutdown_connections 2023-12-01T01:52:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.291+0000 7f57c0399640 1 -- 172.21.15.115:0/1652888919 >> 172.21.15.115:0/1652888919 conn(0x7f57b8065960 msgr2=0x7f57b8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:29.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.291+0000 7f57c0399640 1 -- 172.21.15.115:0/1652888919 shutdown_connections 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.292+0000 7f57c0399640 1 -- 172.21.15.115:0/1652888919 wait complete. 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.293+0000 7f57c0399640 1 Processor -- start 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.293+0000 7f57c0399640 1 -- start start 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.294+0000 7f57c0399640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a4004d00 0x7f57b814c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.294+0000 7f57c0399640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80cc4f0 0x7f57b80cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.294+0000 7f57c0399640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b815a6d0 0x7f57b815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.294+0000 7f57c0399640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b815b090 con 0x7f57b815a6d0 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.294+0000 7f57c0399640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80ca840 con 0x7f57a4004d00 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.294+0000 7f57c0399640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b80cb8a0 con 0x7f57b80cc4f0 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be936640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b815a6d0 0x7f57b815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be936640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b815a6d0 0x7f57b815ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37432/0 (socket says 172.21.15.115:37432) 2023-12-01T01:52:29.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be936640 1 -- 172.21.15.115:0/940576068 learned_addr learned my addr 172.21.15.115:0/940576068 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be135640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a4004d00 0x7f57b814c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57bd934640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80cc4f0 0x7f57b80cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be936640 1 -- 172.21.15.115:0/940576068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80cc4f0 msgr2=0x7f57b80cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be936640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80cc4f0 0x7f57b80cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be936640 1 -- 172.21.15.115:0/940576068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a4004d00 msgr2=0x7f57b814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be936640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a4004d00 0x7f57b814c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be936640 1 -- 172.21.15.115:0/940576068 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80caae0 con 0x7f57b815a6d0 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57bd934640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80cc4f0 0x7f57b80cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be135640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a4004d00 0x7f57b814c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.295+0000 7f57be936640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b815a6d0 0x7f57b815ab50 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f57b002dfd0 tx=0x7f57b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.296+0000 7f57aeffd640 1 -- 172.21.15.115:0/940576068 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0034ea0 con 0x7f57b815a6d0 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.296+0000 7f57aeffd640 1 -- 172.21.15.115:0/940576068 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b0003bb0 con 0x7f57b815a6d0 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.296+0000 7f57aeffd640 1 -- 172.21.15.115:0/940576068 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0004020 con 0x7f57b815a6d0 2023-12-01T01:52:29.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.296+0000 7f57c0399640 1 -- 172.21.15.115:0/940576068 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b80cadc0 con 0x7f57b815a6d0 2023-12-01T01:52:29.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.296+0000 7f57c0399640 1 -- 172.21.15.115:0/940576068 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b80cb380 con 0x7f57b815a6d0 2023-12-01T01:52:29.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.297+0000 7f57c0399640 1 -- 172.21.15.115:0/940576068 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b804e890 con 0x7f57b815a6d0 2023-12-01T01:52:29.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.298+0000 7f57aeffd640 1 -- 172.21.15.115:0/940576068 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 45) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f57b003b040 con 0x7f57b815a6d0 2023-12-01T01:52:29.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.299+0000 7f57aeffd640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f579c06a550 0x7f579c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.299+0000 7f57be135640 1 -- 172.21.15.115:0/940576068 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f579c06a550 msgr2=0x7f579c06ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:29.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.299+0000 7f57aeffd640 1 -- 172.21.15.115:0/940576068 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f57b00aa070 con 0x7f57b815a6d0 2023-12-01T01:52:29.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.299+0000 7f57be135640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f579c06a550 0x7f579c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:29.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.304+0000 7f57aeffd640 1 -- 172.21.15.115:0/940576068 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f57b00753e0 con 0x7f57b815a6d0 2023-12-01T01:52:29.308 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:29.307+0000 7fbee1631040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:52:29.315 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:29.314+0000 7f7ad0e46040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:52:29.367 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:29.367+0000 7fbee1631040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:52:29.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:52:29.397 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:52:29.421 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:29.421+0000 7fbee1631040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:52:29.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.492+0000 7f57c0399640 1 -- 172.21.15.115:0/940576068 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57b80d0a70 con 0x7f57b815a6d0 2023-12-01T01:52:29.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.493+0000 7f57aeffd640 1 -- 172.21.15.115:0/940576068 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57b0045020 con 0x7f57b815a6d0 2023-12-01T01:52:29.493 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:52:29.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.495+0000 7f57c0399640 1 -- 172.21.15.115:0/940576068 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f579c06a550 msgr2=0x7f579c06ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:29.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.495+0000 7f57c0399640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f579c06a550 0x7f579c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.495+0000 7f57c0399640 1 -- 172.21.15.115:0/940576068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b815a6d0 msgr2=0x7f57b815ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.495+0000 7f57c0399640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b815a6d0 0x7f57b815ab50 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f57b002dfd0 tx=0x7f57b0004ab0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.495+0000 7f57c0399640 1 -- 172.21.15.115:0/940576068 shutdown_connections 2023-12-01T01:52:29.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.495+0000 7f57c0399640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f579c06a550 0x7f579c06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.495+0000 7f57c0399640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57b80cc4f0 0x7f57b80cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.495+0000 7f57c0399640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57b815a6d0 0x7f57b815ab50 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.495+0000 7f57c0399640 1 --2- 172.21.15.115:0/940576068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a4004d00 0x7f57b814c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.495+0000 7f57c0399640 1 -- 172.21.15.115:0/940576068 >> 172.21.15.115:0/940576068 conn(0x7f57b8065960 msgr2=0x7f57b80d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:29.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.496+0000 7f57c0399640 1 -- 172.21.15.115:0/940576068 shutdown_connections 2023-12-01T01:52:29.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.496+0000 7f57c0399640 1 -- 172.21.15.115:0/940576068 wait complete. 2023-12-01T01:52:29.514 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:52:29.699 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:29.697+0000 7f7ad0e46040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:52:29.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.768+0000 7f37f0d15640 1 Processor -- start 2023-12-01T01:52:29.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.768+0000 7f37f0d15640 1 -- start start 2023-12-01T01:52:29.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.769+0000 7f37f0d15640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec153c30 0x7f37ec0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.769+0000 7f37f0d15640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f37ec0cfb80 0x7f37ec160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.769+0000 7f37f0d15640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec0cdcd0 con 0x7f37ec154010 2023-12-01T01:52:29.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.769+0000 7f37f0d15640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ec161830 con 0x7f37ec0cfb80 2023-12-01T01:52:29.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.769+0000 7f37f0d15640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0cd3a0 con 0x7f37ec153c30 2023-12-01T01:52:29.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.770+0000 7f37eaffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f37ec154010 0x7f37ec0cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51370/0 (socket says 172.21.15.115:51370) 2023-12-01T01:52:29.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.770+0000 7f37eaffd640 1 -- 172.21.15.115:0/3777015013 learned_addr learned my addr 172.21.15.115:0/3777015013 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:29.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.770+0000 7f37ebfff640 1 --2- 172.21.15.115:0/3777015013 >> v2:172.21.15.115:3300/0 conn(0x7f37ec0cfb80 0x7f37ec160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:29.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.770+0000 7f37eb7fe640 1 --2- 172.21.15.115:0/3777015013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec153c30 0x7f37ec0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:29.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.770+0000 7f37ea7fc640 1 -- 172.21.15.115:0/3777015013 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243822764 0 0) 0x7f37ec0cdcd0 con 0x7f37ec154010 2023-12-01T01:52:29.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.770+0000 7f37ea7fc640 1 -- 172.21.15.115:0/3777015013 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37d4003520 con 0x7f37ec154010 2023-12-01T01:52:29.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.770+0000 7f37ebfff640 1 -- 172.21.15.115:0/3777015013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec153c30 msgr2=0x7f37ec0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:29.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.770+0000 7f37ebfff640 1 --2- 172.21.15.115:0/3777015013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec153c30 0x7f37ec0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.770+0000 7f37ebfff640 1 -- 172.21.15.115:0/3777015013 >> v1:172.21.15.115:6789/0 conn(0x7f37ec154010 legacy=0x7f37ec0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.770+0000 7f37ebfff640 1 -- 172.21.15.115:0/3777015013 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec0cd680 con 0x7f37ec0cfb80 2023-12-01T01:52:29.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.771+0000 7f37ebfff640 1 --2- 172.21.15.115:0/3777015013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec0cfb80 0x7f37ec160ac0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009d90 tx=0x7f37e402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e6111c2f44fd74 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:29.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.772+0000 7f37ea7fc640 1 -- 172.21.15.115:0/3777015013 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e402cc50 con 0x7f37ec0cfb80 2023-12-01T01:52:29.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.772+0000 7f37ea7fc640 1 -- 172.21.15.115:0/3777015013 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e402cdb0 con 0x7f37ec0cfb80 2023-12-01T01:52:29.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.772+0000 7f37ea7fc640 1 -- 172.21.15.115:0/3777015013 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4034660 con 0x7f37ec0cfb80 2023-12-01T01:52:29.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.772+0000 7f37f0d15640 1 -- 172.21.15.115:0/3777015013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec0cfb80 msgr2=0x7f37ec160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.772+0000 7f37f0d15640 1 --2- 172.21.15.115:0/3777015013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec0cfb80 0x7f37ec160ac0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009d90 tx=0x7f37e402bea0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.772+0000 7f37f0d15640 1 -- 172.21.15.115:0/3777015013 shutdown_connections 2023-12-01T01:52:29.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.772+0000 7f37f0d15640 1 --2- 172.21.15.115:0/3777015013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec0cfb80 0x7f37ec160ac0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.772+0000 7f37f0d15640 1 --2- 172.21.15.115:0/3777015013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec153c30 0x7f37ec0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.772+0000 7f37f0d15640 1 -- 172.21.15.115:0/3777015013 >> 172.21.15.115:0/3777015013 conn(0x7f37ec06a0b0 msgr2=0x7f37ec06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:29.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.773+0000 7f37f0d15640 1 -- 172.21.15.115:0/3777015013 shutdown_connections 2023-12-01T01:52:29.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.773+0000 7f37f0d15640 1 -- 172.21.15.115:0/3777015013 wait complete. 2023-12-01T01:52:29.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.774+0000 7f37f0d15640 1 Processor -- start 2023-12-01T01:52:29.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.774+0000 7f37f0d15640 1 -- start start 2023-12-01T01:52:29.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.775+0000 7f37f0d15640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cfb80 0x7f37ec15a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.775+0000 7f37f0d15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec153c30 0x7f37ec0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.775+0000 7f37f0d15640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec154010 0x7f37ec14e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.775+0000 7f37f0d15640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec15ad50 con 0x7f37ec154010 2023-12-01T01:52:29.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.775+0000 7f37f0d15640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0ca1c0 con 0x7f37ec153c30 2023-12-01T01:52:29.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.775+0000 7f37f0d15640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0c7580 con 0x7f37ec0cfb80 2023-12-01T01:52:29.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.775+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec154010 0x7f37ec14e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:29.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec154010 0x7f37ec14e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37452/0 (socket says 172.21.15.115:37452) 2023-12-01T01:52:29.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37ebfff640 1 -- 172.21.15.115:0/1216606711 learned_addr learned my addr 172.21.15.115:0/1216606711 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:29.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37eb7fe640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cfb80 0x7f37ec15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:29.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37eaffd640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec153c30 0x7f37ec0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:29.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37ebfff640 1 -- 172.21.15.115:0/1216606711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cfb80 msgr2=0x7f37ec15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37ebfff640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cfb80 0x7f37ec15a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37ebfff640 1 -- 172.21.15.115:0/1216606711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec153c30 msgr2=0x7f37ec0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37ebfff640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec153c30 0x7f37ec0c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37ebfff640 1 -- 172.21.15.115:0/1216606711 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e4009a70 con 0x7f37ec154010 2023-12-01T01:52:29.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37eb7fe640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cfb80 0x7f37ec15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:29.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37eaffd640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec153c30 0x7f37ec0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:29.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37ebfff640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec154010 0x7f37ec14e9b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009eb0 tx=0x7f37e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:29.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.776+0000 7f37d3fff640 1 -- 172.21.15.115:0/1216606711 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4040650 con 0x7f37ec154010 2023-12-01T01:52:29.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.777+0000 7f37d3fff640 1 -- 172.21.15.115:0/1216606711 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e40407b0 con 0x7f37ec154010 2023-12-01T01:52:29.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.777+0000 7f37f0d15640 1 -- 172.21.15.115:0/1216606711 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec0c7860 con 0x7f37ec154010 2023-12-01T01:52:29.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.777+0000 7f37d3fff640 1 -- 172.21.15.115:0/1216606711 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4040b60 con 0x7f37ec154010 2023-12-01T01:52:29.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.777+0000 7f37f0d15640 1 -- 172.21.15.115:0/1216606711 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec0c7e20 con 0x7f37ec154010 2023-12-01T01:52:29.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.778+0000 7f37f0d15640 1 -- 172.21.15.115:0/1216606711 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec0618d0 con 0x7f37ec154010 2023-12-01T01:52:29.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.779+0000 7f37d3fff640 1 -- 172.21.15.115:0/1216606711 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 45) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f37e4040cc0 con 0x7f37ec154010 2023-12-01T01:52:29.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.780+0000 7f37d3fff640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37d806a550 0x7f37d806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:29.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.780+0000 7f37d3fff640 1 -- 172.21.15.115:0/1216606711 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f37e40ab960 con 0x7f37ec154010 2023-12-01T01:52:29.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.780+0000 7f37eb7fe640 1 -- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37d806a550 msgr2=0x7f37d806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:29.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.780+0000 7f37eb7fe640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37d806a550 0x7f37d806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:29.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.788+0000 7f37d3fff640 1 -- 172.21.15.115:0/1216606711 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f37e4076e00 con 0x7f37ec154010 2023-12-01T01:52:29.833 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:29.832+0000 7fbee1631040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:52:29.858 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:29.857+0000 7f7ad0e46040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:52:29.966 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:29.965+0000 7f7ad0e46040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:52:29.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.978+0000 7f37f0d15640 1 -- 172.21.15.115:0/1216606711 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37ec0b5920 con 0x7f37ec154010 2023-12-01T01:52:29.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.979+0000 7f37d3fff640 1 -- 172.21.15.115:0/1216606711 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37e4044020 con 0x7f37ec154010 2023-12-01T01:52:29.979 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37eb7fe640 1 -- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37d806a550 msgr2=0x7f37d806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37eb7fe640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37d806a550 0x7f37d806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 -- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37d806a550 msgr2=0x7f37d806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37d806a550 0x7f37d806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 -- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec154010 msgr2=0x7f37ec14e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec154010 0x7f37ec14e9b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009eb0 tx=0x7f37e4004a40 comp rx=0 tx=0).stop 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 -- 172.21.15.115:0/1216606711 shutdown_connections 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37d806a550 0x7f37d806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37ec154010 0x7f37ec14e9b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec153c30 0x7f37ec0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 --2- 172.21.15.115:0/1216606711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cfb80 0x7f37ec15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 -- 172.21.15.115:0/1216606711 >> 172.21.15.115:0/1216606711 conn(0x7f37ec06a0b0 msgr2=0x7f37ec06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 -- 172.21.15.115:0/1216606711 shutdown_connections 2023-12-01T01:52:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:29.981+0000 7f37f0d15640 1 -- 172.21.15.115:0/1216606711 wait complete. 2023-12-01T01:52:29.999 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:52:30.000 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:29.998+0000 7fbee1631040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:52:30.019 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:30.017+0000 7f7ad0e46040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:52:30.116 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:30.114+0000 7fbee1631040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:52:30.126 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:30.125+0000 7f7ad0e46040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:52:30.167 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:30.167+0000 7fbee1631040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:52:30.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.202+0000 7fda9cf1b640 1 Processor -- start 2023-12-01T01:52:30.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.203+0000 7fda9cf1b640 1 -- start start 2023-12-01T01:52:30.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.204+0000 7fda9cf1b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fda980ad0f0 0x7fda98167d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.204+0000 7fda9cf1b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda980aef60 con 0x7fda98157380 2023-12-01T01:52:30.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.204+0000 7fda9cf1b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda980ad5c0 con 0x7fda98066070 2023-12-01T01:52:30.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.204+0000 7fda9cf1b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda98161990 con 0x7fda980ad0f0 2023-12-01T01:52:30.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.204+0000 7fda9659c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fda98157380 0x7fda98065960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51378/0 (socket says 172.21.15.115:51378) 2023-12-01T01:52:30.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.204+0000 7fda9659c640 1 -- 172.21.15.115:0/479478117 learned_addr learned my addr 172.21.15.115:0/479478117 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:30.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.204+0000 7fda96d9d640 1 --2- 172.21.15.115:0/479478117 >> v2:172.21.15.115:3300/0 conn(0x7fda980ad0f0 0x7fda98167d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:30.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.205+0000 7fda96d9d640 1 -- 172.21.15.115:0/479478117 >> v1:172.21.15.142:6789/0 conn(0x7fda98066070 legacy=0x7fda980ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:30.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.205+0000 7fda96d9d640 1 -- 172.21.15.115:0/479478117 >> v1:172.21.15.115:6789/0 conn(0x7fda98157380 legacy=0x7fda98065960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:30.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.205+0000 7fda96d9d640 1 -- 172.21.15.115:0/479478117 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9815ddd0 con 0x7fda980ad0f0 2023-12-01T01:52:30.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.206+0000 7fda96d9d640 1 --2- 172.21.15.115:0/479478117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980ad0f0 0x7fda98167d10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fda90009d90 tx=0x7fda9002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4d978bd6fbc963 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:30.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.206+0000 7fda95d9b640 1 -- 172.21.15.115:0/479478117 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9002cc60 con 0x7fda980ad0f0 2023-12-01T01:52:30.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.206+0000 7fda95d9b640 1 -- 172.21.15.115:0/479478117 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9002cdc0 con 0x7fda980ad0f0 2023-12-01T01:52:30.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.206+0000 7fda95d9b640 1 -- 172.21.15.115:0/479478117 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90034660 con 0x7fda980ad0f0 2023-12-01T01:52:30.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.207+0000 7fda9cf1b640 1 -- 172.21.15.115:0/479478117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980ad0f0 msgr2=0x7fda98167d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:30.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.207+0000 7fda9cf1b640 1 --2- 172.21.15.115:0/479478117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980ad0f0 0x7fda98167d10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fda90009d90 tx=0x7fda9002beb0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.207+0000 7fda9cf1b640 1 -- 172.21.15.115:0/479478117 shutdown_connections 2023-12-01T01:52:30.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.207+0000 7fda9cf1b640 1 --2- 172.21.15.115:0/479478117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980ad0f0 0x7fda98167d10 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.207+0000 7fda9cf1b640 1 -- 172.21.15.115:0/479478117 >> 172.21.15.115:0/479478117 conn(0x7fda9806a3d0 msgr2=0x7fda980b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:30.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.207+0000 7fda9cf1b640 1 -- 172.21.15.115:0/479478117 shutdown_connections 2023-12-01T01:52:30.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.207+0000 7fda9cf1b640 1 -- 172.21.15.115:0/479478117 wait complete. 2023-12-01T01:52:30.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.209+0000 7fda9cf1b640 1 Processor -- start 2023-12-01T01:52:30.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.209+0000 7fda9cf1b640 1 -- start start 2023-12-01T01:52:30.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.210+0000 7fda9cf1b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98066070 0x7fda9814c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.210+0000 7fda9cf1b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda980ad0f0 0x7fda98151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.210+0000 7fda9cf1b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda98157380 0x7fda98155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.210+0000 7fda9cf1b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9814d750 con 0x7fda98066070 2023-12-01T01:52:30.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.210+0000 7fda9cf1b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9814cb40 con 0x7fda98157380 2023-12-01T01:52:30.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.210+0000 7fda9cf1b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9814ce20 con 0x7fda980ad0f0 2023-12-01T01:52:30.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.210+0000 7fda9659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98066070 0x7fda9814c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:30.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.210+0000 7fda9659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98066070 0x7fda9814c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37462/0 (socket says 172.21.15.115:37462) 2023-12-01T01:52:30.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.210+0000 7fda9659c640 1 -- 172.21.15.115:0/651173371 learned_addr learned my addr 172.21.15.115:0/651173371 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda96d9d640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda98157380 0x7fda98155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda8ffff640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda980ad0f0 0x7fda98151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda9659c640 1 -- 172.21.15.115:0/651173371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda980ad0f0 msgr2=0x7fda98151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda9659c640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda980ad0f0 0x7fda98151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda9659c640 1 -- 172.21.15.115:0/651173371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda98157380 msgr2=0x7fda98155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda9659c640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda98157380 0x7fda98155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda9659c640 1 -- 172.21.15.115:0/651173371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90009a70 con 0x7fda98066070 2023-12-01T01:52:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda96d9d640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda98157380 0x7fda98155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda9659c640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98066070 0x7fda9814c240 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fda8800c400 tx=0x7fda8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda8effd640 1 -- 172.21.15.115:0/651173371 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88005ce0 con 0x7fda98066070 2023-12-01T01:52:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda8effd640 1 -- 172.21.15.115:0/651173371 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda88005e40 con 0x7fda98066070 2023-12-01T01:52:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda8effd640 1 -- 172.21.15.115:0/651173371 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88007310 con 0x7fda98066070 2023-12-01T01:52:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda9cf1b640 1 -- 172.21.15.115:0/651173371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9814d100 con 0x7fda98066070 2023-12-01T01:52:30.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.211+0000 7fda9cf1b640 1 -- 172.21.15.115:0/651173371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9814c780 con 0x7fda98066070 2023-12-01T01:52:30.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.213+0000 7fda9cf1b640 1 -- 172.21.15.115:0/651173371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9804e890 con 0x7fda98066070 2023-12-01T01:52:30.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.213+0000 7fda8effd640 1 -- 172.21.15.115:0/651173371 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 45) v1 ==== 88710+0+0 (secure 0 0 0) 0x7fda88007470 con 0x7fda98066070 2023-12-01T01:52:30.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.214+0000 7fda8effd640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda7806a550 0x7fda7806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.214+0000 7fda8effd640 1 -- 172.21.15.115:0/651173371 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fda880881e0 con 0x7fda98066070 2023-12-01T01:52:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.214+0000 7fda8ffff640 1 -- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda7806a550 msgr2=0x7fda7806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:30.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.214+0000 7fda8ffff640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda7806a550 0x7fda7806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:30.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.218+0000 7fda8effd640 1 -- 172.21.15.115:0/651173371 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fda88053560 con 0x7fda98066070 2023-12-01T01:52:30.233 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:30.232+0000 7f7ad0e46040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:52:30.282 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:30.281+0000 7fbee1631040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:52:30.390 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:30.388+0000 7f7ad0e46040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:52:30.393 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:30.392+0000 7fbee1631040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:52:30.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.415+0000 7fda8ffff640 1 -- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda7806a550 msgr2=0x7fda7806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:30.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.415+0000 7fda8ffff640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda7806a550 0x7fda7806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:52:30.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.421+0000 7fda9cf1b640 1 -- 172.21.15.115:0/651173371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fda980b2990 con 0x7fda98066070 2023-12-01T01:52:30.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.422+0000 7fda8effd640 1 -- 172.21.15.115:0/651173371 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v98) v1 ==== 80+0+467 (secure 0 0 0) 0x7fda88056b80 con 0x7fda98066070 2023-12-01T01:52:30.423 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/88317 2023-12-02T01:52:25.734961+0000 2023-12-01T01:52:30.423 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/88317 2023-12-02T01:52:25.734961+0000 2023-12-01T01:52:30.423 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3373285076 2023-12-02T01:52:25.734961+0000 2023-12-01T01:52:30.423 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3569012860 2023-12-02T01:52:25.734961+0000 2023-12-01T01:52:30.423 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/709337731 2023-12-02T01:52:25.734961+0000 2023-12-01T01:52:30.423 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1296968550 2023-12-02T01:52:25.734961+0000 2023-12-01T01:52:30.424 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1490819667 2023-12-02T01:52:25.734961+0000 2023-12-01T01:52:30.424 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1878415381 2023-12-02T01:52:25.734961+0000 2023-12-01T01:52:30.424 INFO:teuthology.orchestra.run.smithi115.stderr:listed 8 entries 2023-12-01T01:52:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.424+0000 7fda9cf1b640 1 -- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda7806a550 msgr2=0x7fda7806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.424+0000 7fda9cf1b640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda7806a550 0x7fda7806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.424+0000 7fda9cf1b640 1 -- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98066070 msgr2=0x7fda9814c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.424+0000 7fda9cf1b640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98066070 0x7fda9814c240 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fda8800c400 tx=0x7fda8800c8c0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.424+0000 7fda9cf1b640 1 -- 172.21.15.115:0/651173371 shutdown_connections 2023-12-01T01:52:30.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.424+0000 7fda9cf1b640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda7806a550 0x7fda7806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.424+0000 7fda9cf1b640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda980ad0f0 0x7fda98151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.424+0000 7fda9cf1b640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda98157380 0x7fda98155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.425+0000 7fda9cf1b640 1 --2- 172.21.15.115:0/651173371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda98066070 0x7fda9814c240 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.425+0000 7fda9cf1b640 1 -- 172.21.15.115:0/651173371 >> 172.21.15.115:0/651173371 conn(0x7fda9806a3d0 msgr2=0x7fda980b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:30.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.425+0000 7fda9cf1b640 1 -- 172.21.15.115:0/651173371 shutdown_connections 2023-12-01T01:52:30.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.425+0000 7fda9cf1b640 1 -- 172.21.15.115:0/651173371 wait complete. 2023-12-01T01:52:30.439 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:30.437+0000 7f7ad0e46040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:52:30.443 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T01:52:30.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:30.443 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T01:52:30.541 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:52:30.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:52:30.542 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T01:52:30.567 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:30.565+0000 7fbee1631040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:52:30.568 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:30.566+0000 7f7ad0e46040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:52:30.585 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:52:30.585 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:30.618 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:30.617+0000 7fbee1631040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:52:30.639 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:52:30.639 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:52:30.639 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:30.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:52:30.691 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:52:30.691 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:30.740 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:30.739+0000 7fbee1631040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:52:30.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:52:30.743 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:52:30.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.953+0000 7fc2e70da640 1 Processor -- start 2023-12-01T01:52:30.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.953+0000 7fc2e70da640 1 -- start start 2023-12-01T01:52:30.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.955+0000 7fc2e70da640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc2e01596d0 0x7fc2e00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.955+0000 7fc2e70da640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2e0071780 0x7fc2e0070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.955+0000 7fc2e70da640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e00ce1a0 con 0x7fc2e006c2c0 2023-12-01T01:52:30.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.955+0000 7fc2e70da640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e00cd900 con 0x7fc2e01596d0 2023-12-01T01:52:30.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.955+0000 7fc2e70da640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00cdbe0 con 0x7fc2e0071780 2023-12-01T01:52:30.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.955+0000 7fc2e60d8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc2e01596d0 0x7fc2e00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:30.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.955+0000 7fc2e58d7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2e0071780 0x7fc2e0070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:30.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.956+0000 7fc2e60d8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc2e01596d0 0x7fc2e00cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:34318/0 (socket says 172.21.15.115:34318) 2023-12-01T01:52:30.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.956+0000 7fc2e60d8640 1 -- 172.21.15.115:0/3143608523 learned_addr learned my addr 172.21.15.115:0/3143608523 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:30.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.956+0000 7fc2e68d9640 1 --1- 172.21.15.115:0/3143608523 >> v1:172.21.15.142:6789/0 conn(0x7fc2e006c2c0 0x7fc2e00cb530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:46610/0 (socket says 172.21.15.115:46610) 2023-12-01T01:52:30.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.956+0000 7fc2e60d8640 1 -- 172.21.15.115:0/3143608523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2e0071780 msgr2=0x7fc2e0070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:30.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.956+0000 7fc2e60d8640 1 --2- 172.21.15.115:0/3143608523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2e0071780 0x7fc2e0070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.956+0000 7fc2e60d8640 1 -- 172.21.15.115:0/3143608523 >> v1:172.21.15.142:6789/0 conn(0x7fc2e006c2c0 legacy=0x7fc2e00cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:30.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.956+0000 7fc2e60d8640 1 -- 172.21.15.115:0/3143608523 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00cdec0 con 0x7fc2e01596d0 2023-12-01T01:52:30.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.958+0000 7fc2e60d8640 1 --2- 172.21.15.115:0/3143608523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e01596d0 0x7fc2e00cd200 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009870 tx=0x7fc2dc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4f9b92b79f48615 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:30.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.958+0000 7fc2e50d6640 1 -- 172.21.15.115:0/3143608523 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc030840 con 0x7fc2e01596d0 2023-12-01T01:52:30.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.958+0000 7fc2e50d6640 1 -- 172.21.15.115:0/3143608523 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc0309a0 con 0x7fc2e01596d0 2023-12-01T01:52:30.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.958+0000 7fc2e50d6640 1 -- 172.21.15.115:0/3143608523 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc030d30 con 0x7fc2e01596d0 2023-12-01T01:52:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.959+0000 7fc2e70da640 1 -- 172.21.15.115:0/3143608523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e01596d0 msgr2=0x7fc2e00cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.959+0000 7fc2e70da640 1 --2- 172.21.15.115:0/3143608523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e01596d0 0x7fc2e00cd200 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009870 tx=0x7fc2dc02b980 comp rx=0 tx=0).stop 2023-12-01T01:52:30.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.959+0000 7fc2e70da640 1 -- 172.21.15.115:0/3143608523 shutdown_connections 2023-12-01T01:52:30.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.959+0000 7fc2e70da640 1 --2- 172.21.15.115:0/3143608523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2e0071780 0x7fc2e0070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.959+0000 7fc2e70da640 1 --2- 172.21.15.115:0/3143608523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e01596d0 0x7fc2e00cd200 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.959+0000 7fc2e70da640 1 -- 172.21.15.115:0/3143608523 >> 172.21.15.115:0/3143608523 conn(0x7fc2e006a0e0 msgr2=0x7fc2e006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:30.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.959+0000 7fc2e70da640 1 -- 172.21.15.115:0/3143608523 shutdown_connections 2023-12-01T01:52:30.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.959+0000 7fc2e70da640 1 -- 172.21.15.115:0/3143608523 wait complete. 2023-12-01T01:52:30.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.961+0000 7fc2e70da640 1 Processor -- start 2023-12-01T01:52:30.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.961+0000 7fc2e70da640 1 -- start start 2023-12-01T01:52:30.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e70da640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e006c2c0 0x7fc2e00b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e70da640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2e0071780 0x7fc2e00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e70da640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2e01596d0 0x7fc2e0158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e70da640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00af820 con 0x7fc2e0071780 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e70da640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00afb00 con 0x7fc2e006c2c0 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e70da640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00afde0 con 0x7fc2e01596d0 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e58d7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2e0071780 0x7fc2e00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e58d7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2e0071780 0x7fc2e00b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37472/0 (socket says 172.21.15.115:37472) 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e58d7640 1 -- 172.21.15.115:0/3837085916 learned_addr learned my addr 172.21.15.115:0/3837085916 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e68d9640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2e01596d0 0x7fc2e0158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.962+0000 7fc2e60d8640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e006c2c0 0x7fc2e00b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2e58d7640 1 -- 172.21.15.115:0/3837085916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2e01596d0 msgr2=0x7fc2e0158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2e58d7640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2e01596d0 0x7fc2e0158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2e58d7640 1 -- 172.21.15.115:0/3837085916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e006c2c0 msgr2=0x7fc2e00b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2e58d7640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e006c2c0 0x7fc2e00b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2e58d7640 1 -- 172.21.15.115:0/3837085916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0094d0 con 0x7fc2e0071780 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2e58d7640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2e0071780 0x7fc2e00b08c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000e9b0 tx=0x7fc2d000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:30.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2d6ffd640 1 -- 172.21.15.115:0/3837085916 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0009670 con 0x7fc2e0071780 2023-12-01T01:52:30.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2d6ffd640 1 -- 172.21.15.115:0/3837085916 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d000c9a0 con 0x7fc2e0071780 2023-12-01T01:52:30.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2d6ffd640 1 -- 172.21.15.115:0/3837085916 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000cd50 con 0x7fc2e0071780 2023-12-01T01:52:30.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2e70da640 1 -- 172.21.15.115:0/3837085916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e00b00c0 con 0x7fc2e0071780 2023-12-01T01:52:30.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.963+0000 7fc2e70da640 1 -- 172.21.15.115:0/3837085916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e00ad3e0 con 0x7fc2e0071780 2023-12-01T01:52:30.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.964+0000 7fc2e70da640 1 -- 172.21.15.115:0/3837085916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e00673a0 con 0x7fc2e0071780 2023-12-01T01:52:30.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.965+0000 7fc2d6ffd640 1 -- 172.21.15.115:0/3837085916 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 45) v1 ==== 88710+0+0 (secure 0 0 0) 0x7fc2d0013470 con 0x7fc2e0071780 2023-12-01T01:52:30.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.966+0000 7fc2d6ffd640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2c406a550 0x7fc2c406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:30.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.966+0000 7fc2d6ffd640 1 -- 172.21.15.115:0/3837085916 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fc2d0088e80 con 0x7fc2e0071780 2023-12-01T01:52:30.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.966+0000 7fc2e60d8640 1 -- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2c406a550 msgr2=0x7fc2c406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:30.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.966+0000 7fc2e60d8640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2c406a550 0x7fc2c406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:30.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:30.975+0000 7fc2d6ffd640 1 -- 172.21.15.115:0/3837085916 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc2d0054200 con 0x7fc2e0071780 2023-12-01T01:52:31.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.167+0000 7fc2e60d8640 1 -- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2c406a550 msgr2=0x7fc2c406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:31.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.167+0000 7fc2e60d8640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2c406a550 0x7fc2c406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:52:31.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.201+0000 7fc2e70da640 1 -- 172.21.15.115:0/3837085916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2e0071240 con 0x7fc2e0071780 2023-12-01T01:52:31.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.202+0000 7fc2d6ffd640 1 -- 172.21.15.115:0/3837085916 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4659 (secure 0 0 0) 0x7fc2d0017090 con 0x7fc2e0071780 2023-12-01T01:52:31.202 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:31.202 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/542800961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":542800961},{"type":"v1","addr":"172.21.15.142:6833","nonce":542800961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6942,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2035657957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2035657957},{"type":"v1","addr":"172.21.15.115:6835","nonce":2035657957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3644451216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3644451216},{"type":"v1","addr":"172.21.15.115:6837","nonce":3644451216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:06.454774+0000","modified":"2023-12-01T01:52:09.476985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6957":{"gid":6957,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1311626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1311626228},{"type":"v1","addr":"172.21.15.142:6835","nonce":1311626228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-01T01:52:31.203 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2023-12-01T01:52:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.204+0000 7fc2e70da640 1 -- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2c406a550 msgr2=0x7fc2c406ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.204+0000 7fc2e70da640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2c406a550 0x7fc2c406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.204+0000 7fc2e70da640 1 -- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2e0071780 msgr2=0x7fc2e00b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.204+0000 7fc2e70da640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2e0071780 0x7fc2e00b08c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000e9b0 tx=0x7fc2d000ee70 comp rx=0 tx=0).stop 2023-12-01T01:52:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.204+0000 7fc2e70da640 1 -- 172.21.15.115:0/3837085916 shutdown_connections 2023-12-01T01:52:31.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.204+0000 7fc2e70da640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc2c406a550 0x7fc2c406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.204+0000 7fc2e70da640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2e01596d0 0x7fc2e0158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.204+0000 7fc2e70da640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2e0071780 0x7fc2e00b08c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.204+0000 7fc2e70da640 1 --2- 172.21.15.115:0/3837085916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2e006c2c0 0x7fc2e00b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.204+0000 7fc2e70da640 1 -- 172.21.15.115:0/3837085916 >> 172.21.15.115:0/3837085916 conn(0x7fc2e006a0e0 msgr2=0x7fc2e006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.205+0000 7fc2e70da640 1 -- 172.21.15.115:0/3837085916 shutdown_connections 2023-12-01T01:52:31.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.205+0000 7fc2e70da640 1 -- 172.21.15.115:0/3837085916 wait complete. 2023-12-01T01:52:31.224 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:31.402 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:31.399+0000 7f7ad0e46040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:52:31.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.408+0000 7f8e329fd640 1 Processor -- start 2023-12-01T01:52:31.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.408+0000 7f8e329fd640 1 -- start start 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.409+0000 7f8e329fd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8e2c160130 0x7f8e2c0d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.409+0000 7f8e329fd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8e2c0d26e0 0x7f8e2c0b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.409+0000 7f8e329fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c0cfb20 0x7f8e2c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.409+0000 7f8e329fd640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2c0d0510 con 0x7f8e2c160130 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.409+0000 7f8e329fd640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2c0b8b60 con 0x7f8e2c0d26e0 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.409+0000 7f8e329fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c160510 con 0x7f8e2c0cfb20 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.410+0000 7f8e319fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8e2c160130 0x7f8e2c0d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.410+0000 7f8e319fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8e2c160130 0x7f8e2c0d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37474/0 (socket says 172.21.15.115:37474) 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.410+0000 7f8e319fb640 1 -- 172.21.15.115:0/1424336738 learned_addr learned my addr 172.21.15.115:0/1424336738 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.410+0000 7f8e311fa640 1 --2- 172.21.15.115:0/1424336738 >> v2:172.21.15.142:3300/0 conn(0x7f8e2c0d26e0 0x7f8e2c0b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:31.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.410+0000 7f8e319fb640 1 -- 172.21.15.115:0/1424336738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c0cfb20 msgr2=0x7f8e2c0cffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-01T01:52:31.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.410+0000 7f8e319fb640 1 --2- 172.21.15.115:0/1424336738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c0cfb20 0x7f8e2c0cffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.410+0000 7f8e319fb640 1 -- 172.21.15.115:0/1424336738 >> v2:172.21.15.142:3300/0 conn(0x7f8e2c0d26e0 msgr2=0x7f8e2c0b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:31.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.410+0000 7f8e319fb640 1 --2- 172.21.15.115:0/1424336738 >> v2:172.21.15.142:3300/0 conn(0x7f8e2c0d26e0 0x7f8e2c0b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.410+0000 7f8e319fb640 1 -- 172.21.15.115:0/1424336738 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c0d1640 con 0x7f8e2c160130 2023-12-01T01:52:31.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.412+0000 7f8e319fb640 1 --2- 172.21.15.115:0/1424336738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 0x7f8e2c0d0fb0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0d0d10 tx=0x7f8e2802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd11b68df796a0f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:31.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.412+0000 7f8e309f9640 1 -- 172.21.15.115:0/1424336738 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28004230 con 0x7f8e2c160130 2023-12-01T01:52:31.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.412+0000 7f8e309f9640 1 -- 172.21.15.115:0/1424336738 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e28004390 con 0x7f8e2c160130 2023-12-01T01:52:31.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.412+0000 7f8e309f9640 1 -- 172.21.15.115:0/1424336738 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28031870 con 0x7f8e2c160130 2023-12-01T01:52:31.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.412+0000 7f8e329fd640 1 -- 172.21.15.115:0/1424336738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 msgr2=0x7f8e2c0d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:31.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.412+0000 7f8e329fd640 1 --2- 172.21.15.115:0/1424336738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 0x7f8e2c0d0fb0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0d0d10 tx=0x7f8e2802b910 comp rx=0 tx=0).stop 2023-12-01T01:52:31.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.413+0000 7f8e329fd640 1 -- 172.21.15.115:0/1424336738 shutdown_connections 2023-12-01T01:52:31.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.413+0000 7f8e329fd640 1 --2- 172.21.15.115:0/1424336738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c0cfb20 0x7f8e2c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.413+0000 7f8e329fd640 1 --2- 172.21.15.115:0/1424336738 >> v2:172.21.15.142:3300/0 conn(0x7f8e2c0d26e0 0x7f8e2c0b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.413+0000 7f8e329fd640 1 --2- 172.21.15.115:0/1424336738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 0x7f8e2c0d0fb0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.413+0000 7f8e329fd640 1 -- 172.21.15.115:0/1424336738 >> 172.21.15.115:0/1424336738 conn(0x7f8e2c06a0b0 msgr2=0x7f8e2c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:31.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.413+0000 7f8e329fd640 1 -- 172.21.15.115:0/1424336738 shutdown_connections 2023-12-01T01:52:31.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.413+0000 7f8e329fd640 1 -- 172.21.15.115:0/1424336738 wait complete. 2023-12-01T01:52:31.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.414+0000 7f8e329fd640 1 Processor -- start 2023-12-01T01:52:31.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.415+0000 7f8e329fd640 1 -- start start 2023-12-01T01:52:31.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.415+0000 7f8e329fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c0cfb20 0x7f8e2c14cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.415+0000 7f8e329fd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e2c0d26e0 0x7f8e2c0cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.415+0000 7f8e329fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 0x7f8e2c0c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.415+0000 7f8e329fd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c1f2420 con 0x7f8e2c160130 2023-12-01T01:52:31.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.415+0000 7f8e329fd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c150c40 con 0x7f8e2c0d26e0 2023-12-01T01:52:31.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.415+0000 7f8e329fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c150f20 con 0x7f8e2c0cfb20 2023-12-01T01:52:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.416+0000 7f8e321fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 0x7f8e2c0c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.416+0000 7f8e321fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 0x7f8e2c0c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37478/0 (socket says 172.21.15.115:37478) 2023-12-01T01:52:31.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.416+0000 7f8e321fc640 1 -- 172.21.15.115:0/2737310231 learned_addr learned my addr 172.21.15.115:0/2737310231 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.416+0000 7f8e321fc640 1 -- 172.21.15.115:0/2737310231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c0cfb20 msgr2=0x7f8e2c14cfb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.416+0000 7f8e321fc640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c0cfb20 0x7f8e2c14cfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.416+0000 7f8e321fc640 1 -- 172.21.15.115:0/2737310231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e2c0d26e0 msgr2=0x7f8e2c0cc4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.416+0000 7f8e321fc640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e2c0d26e0 0x7f8e2c0cc4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.416+0000 7f8e321fc640 1 -- 172.21.15.115:0/2737310231 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e28009460 con 0x7f8e2c160130 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.417+0000 7f8e321fc640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 0x7f8e2c0c7720 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400b990 tx=0x7f8e2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.417+0000 7f8e227fc640 1 -- 172.21.15.115:0/2737310231 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2400cba0 con 0x7f8e2c160130 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.417+0000 7f8e227fc640 1 -- 172.21.15.115:0/2737310231 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2400cd00 con 0x7f8e2c160130 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.417+0000 7f8e227fc640 1 -- 172.21.15.115:0/2737310231 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24014680 con 0x7f8e2c160130 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.417+0000 7f8e329fd640 1 -- 172.21.15.115:0/2737310231 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2c151200 con 0x7f8e2c160130 2023-12-01T01:52:31.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.417+0000 7f8e329fd640 1 -- 172.21.15.115:0/2737310231 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2c0ca120 con 0x7f8e2c160130 2023-12-01T01:52:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.418+0000 7f8e329fd640 1 -- 172.21.15.115:0/2737310231 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2c0ca340 con 0x7f8e2c160130 2023-12-01T01:52:31.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.419+0000 7f8e227fc640 1 -- 172.21.15.115:0/2737310231 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 45) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f8e2401d3b0 con 0x7f8e2c160130 2023-12-01T01:52:31.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.420+0000 7f8e227fc640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e1006a550 0x7f8e1006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.420+0000 7f8e227fc640 1 -- 172.21.15.115:0/2737310231 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f8e24088c90 con 0x7f8e2c160130 2023-12-01T01:52:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.420+0000 7f8e319fb640 1 -- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e1006a550 msgr2=0x7f8e1006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:31.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.420+0000 7f8e319fb640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e1006a550 0x7f8e1006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:31.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.424+0000 7f8e227fc640 1 -- 172.21.15.115:0/2737310231 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8e24053f70 con 0x7f8e2c160130 2023-12-01T01:52:31.580 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:31.579+0000 7fbee1631040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:52:31.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.616+0000 7f8e329fd640 1 -- 172.21.15.115:0/2737310231 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e2c0673a0 con 0x7f8e2c160130 2023-12-01T01:52:31.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.619+0000 7f8e227fc640 1 -- 172.21.15.115:0/2737310231 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16586 (secure 0 0 0) 0x7f8e24019020 con 0x7f8e2c160130 2023-12-01T01:52:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:31.620 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":98,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:25.735054+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:04.909384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:05.480768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6822/88317":"2023-12-02T01:52:25.734961+0000","172.21.15.115:6824/88317":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/3373285076":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/3569012860":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/709337731":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/1296968550":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/1490819667":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/1878415381":"2023-12-02T01:52:25.734961+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:31.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.621+0000 7f8e319fb640 1 -- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e1006a550 msgr2=0x7f8e1006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:31.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.621+0000 7f8e319fb640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e1006a550 0x7f8e1006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 -- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e1006a550 msgr2=0x7f8e1006ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e1006a550 0x7f8e1006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 -- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 msgr2=0x7f8e2c0c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 0x7f8e2c0c7720 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400b990 tx=0x7f8e2400be50 comp rx=0 tx=0).stop 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 -- 172.21.15.115:0/2737310231 shutdown_connections 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e1006a550 0x7f8e1006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c160130 0x7f8e2c0c7720 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e2c0d26e0 0x7f8e2c0cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 --2- 172.21.15.115:0/2737310231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c0cfb20 0x7f8e2c14cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 -- 172.21.15.115:0/2737310231 >> 172.21.15.115:0/2737310231 conn(0x7f8e2c06a0b0 msgr2=0x7f8e2c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:31.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 -- 172.21.15.115:0/2737310231 shutdown_connections 2023-12-01T01:52:31.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.622+0000 7f8e329fd640 1 -- 172.21.15.115:0/2737310231 wait complete. 2023-12-01T01:52:31.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:52:04.909384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:52:05.480768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:52:31.642 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:52:31.696 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:31.695+0000 7f7ad0e46040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:52:31.753 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:31.751+0000 7f7ad0e46040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:52:31.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.843+0000 7f1fde421640 1 Processor -- start 2023-12-01T01:52:31.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.843+0000 7f1fde421640 1 -- start start 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.845+0000 7f1fde421640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1fd815cf10 0x7f1fd80cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.845+0000 7f1fde421640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fd8070d30 0x7f1fd8065960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.845+0000 7f1fde421640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1fd80705e0 0x7f1fd8065ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.845+0000 7f1fde421640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd815d2f0 con 0x7f1fd80705e0 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.845+0000 7f1fde421640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd8071640 con 0x7f1fd815cf10 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.845+0000 7f1fde421640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b4b30 con 0x7f1fd8070d30 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.845+0000 7f1fdc9be640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1fd80705e0 0x7f1fd8065ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.845+0000 7f1fdc9be640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1fd80705e0 0x7f1fd8065ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37480/0 (socket says 172.21.15.115:37480) 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.845+0000 7f1fdc9be640 1 -- 172.21.15.115:0/805960690 learned_addr learned my addr 172.21.15.115:0/805960690 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.846+0000 7f1fd77fe640 1 --2- 172.21.15.115:0/805960690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fd8070d30 0x7f1fd8065960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:31.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.846+0000 7f1fd7fff640 1 --2- 172.21.15.115:0/805960690 >> v2:172.21.15.142:3300/0 conn(0x7f1fd815cf10 0x7f1fd80cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:31.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.846+0000 7f1fdc9be640 1 -- 172.21.15.115:0/805960690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fd8070d30 msgr2=0x7f1fd8065960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:31.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.846+0000 7f1fdc9be640 1 --2- 172.21.15.115:0/805960690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fd8070d30 0x7f1fd8065960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.846+0000 7f1fdc9be640 1 -- 172.21.15.115:0/805960690 >> v2:172.21.15.142:3300/0 conn(0x7f1fd815cf10 msgr2=0x7f1fd80cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:31.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.846+0000 7f1fdc9be640 1 --2- 172.21.15.115:0/805960690 >> v2:172.21.15.142:3300/0 conn(0x7f1fd815cf10 0x7f1fd80cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.846+0000 7f1fdc9be640 1 -- 172.21.15.115:0/805960690 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80b3fb0 con 0x7f1fd80705e0 2023-12-01T01:52:31.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.847+0000 7f1fdc9be640 1 --2- 172.21.15.115:0/805960690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 0x7f1fd8065ea0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8070380 tx=0x7f1fd002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fbeb36a4323cd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:31.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.847+0000 7f1fd6ffd640 1 -- 172.21.15.115:0/805960690 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd002c9e0 con 0x7f1fd80705e0 2023-12-01T01:52:31.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.847+0000 7f1fd6ffd640 1 -- 172.21.15.115:0/805960690 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd002cb40 con 0x7f1fd80705e0 2023-12-01T01:52:31.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.847+0000 7f1fd6ffd640 1 -- 172.21.15.115:0/805960690 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00343f0 con 0x7f1fd80705e0 2023-12-01T01:52:31.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.848+0000 7f1fde421640 1 -- 172.21.15.115:0/805960690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 msgr2=0x7f1fd8065ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:31.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.848+0000 7f1fde421640 1 --2- 172.21.15.115:0/805960690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 0x7f1fd8065ea0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8070380 tx=0x7f1fd002be40 comp rx=0 tx=0).stop 2023-12-01T01:52:31.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.848+0000 7f1fde421640 1 -- 172.21.15.115:0/805960690 shutdown_connections 2023-12-01T01:52:31.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.848+0000 7f1fde421640 1 --2- 172.21.15.115:0/805960690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 0x7f1fd8065ea0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.848+0000 7f1fde421640 1 --2- 172.21.15.115:0/805960690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fd8070d30 0x7f1fd8065960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.848+0000 7f1fde421640 1 --2- 172.21.15.115:0/805960690 >> v2:172.21.15.142:3300/0 conn(0x7f1fd815cf10 0x7f1fd80cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.848+0000 7f1fde421640 1 -- 172.21.15.115:0/805960690 >> 172.21.15.115:0/805960690 conn(0x7f1fd80b1720 msgr2=0x7f1fd80b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:31.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.848+0000 7f1fde421640 1 -- 172.21.15.115:0/805960690 shutdown_connections 2023-12-01T01:52:31.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.848+0000 7f1fde421640 1 -- 172.21.15.115:0/805960690 wait complete. 2023-12-01T01:52:31.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.850+0000 7f1fde421640 1 Processor -- start 2023-12-01T01:52:31.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.850+0000 7f1fde421640 1 -- start start 2023-12-01T01:52:31.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fde421640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 0x7f1fd815fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fde421640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fd8070d30 0x7f1fd80cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fde421640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd815cf10 0x7f1fd815ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fde421640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd815e600 con 0x7f1fd80705e0 2023-12-01T01:52:31.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fde421640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd81603b0 con 0x7f1fd815cf10 2023-12-01T01:52:31.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fde421640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd814c280 con 0x7f1fd8070d30 2023-12-01T01:52:31.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fd7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 0x7f1fd815fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:31.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fd7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 0x7f1fd815fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37494/0 (socket says 172.21.15.115:37494) 2023-12-01T01:52:31.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fd7fff640 1 -- 172.21.15.115:0/1451278064 learned_addr learned my addr 172.21.15.115:0/1451278064 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:31.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fd77fe640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fd8070d30 0x7f1fd80cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.851+0000 7f1fdc9be640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd815cf10 0x7f1fd815ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fd7fff640 1 -- 172.21.15.115:0/1451278064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fd8070d30 msgr2=0x7f1fd80cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fd7fff640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fd8070d30 0x7f1fd80cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fd7fff640 1 -- 172.21.15.115:0/1451278064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd815cf10 msgr2=0x7f1fd815ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fd7fff640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd815cf10 0x7f1fd815ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fd7fff640 1 -- 172.21.15.115:0/1451278064 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd0009b30 con 0x7f1fd80705e0 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fd7fff640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 0x7f1fd815fe70 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800d830 tx=0x7f1fc800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fd4ff9640 1 -- 172.21.15.115:0/1451278064 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800b6c0 con 0x7f1fd80705e0 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fd4ff9640 1 -- 172.21.15.115:0/1451278064 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc800b820 con 0x7f1fd80705e0 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fd4ff9640 1 -- 172.21.15.115:0/1451278064 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800bbb0 con 0x7f1fd80705e0 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fde421640 1 -- 172.21.15.115:0/1451278064 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd815d9f0 con 0x7f1fd80705e0 2023-12-01T01:52:31.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.852+0000 7f1fde421640 1 -- 172.21.15.115:0/1451278064 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd815dfb0 con 0x7f1fd80705e0 2023-12-01T01:52:31.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.854+0000 7f1fde421640 1 -- 172.21.15.115:0/1451278064 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd804e890 con 0x7f1fd80705e0 2023-12-01T01:52:31.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.854+0000 7f1fd4ff9640 1 -- 172.21.15.115:0/1451278064 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 45) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f1fc801e460 con 0x7f1fd80705e0 2023-12-01T01:52:31.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.855+0000 7f1fd4ff9640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fc406a550 0x7f1fc406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:31.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.855+0000 7f1fd4ff9640 1 -- 172.21.15.115:0/1451278064 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f1fc8089270 con 0x7f1fd80705e0 2023-12-01T01:52:31.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.855+0000 7f1fd77fe640 1 -- 172.21.15.115:0/1451278064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fc406a550 msgr2=0x7f1fc406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:52:31.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.855+0000 7f1fd77fe640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fc406a550 0x7f1fc406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:52:31.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:31.860+0000 7f1fd4ff9640 1 -- 172.21.15.115:0/1451278064 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1fc80545e0 con 0x7f1fd80705e0 2023-12-01T01:52:31.870 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:52:31.869+0000 7f7ad0e46040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:52:31.894 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:31.894+0000 7fbee1631040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:52:31.953 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:31.953+0000 7fbee1631040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:52:32.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.050+0000 7f1fde421640 1 -- 172.21.15.115:0/1451278064 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fd8070af0 con 0x7f1fd80705e0 2023-12-01T01:52:32.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.051+0000 7f1fd4ff9640 1 -- 172.21.15.115:0/1451278064 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1fc8057c00 con 0x7f1fd80705e0 2023-12-01T01:52:32.051 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:52:32.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.053+0000 7f1fde421640 1 -- 172.21.15.115:0/1451278064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fc406a550 msgr2=0x7f1fc406ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:52:32.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.053+0000 7f1fde421640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fc406a550 0x7f1fc406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.053+0000 7f1fde421640 1 -- 172.21.15.115:0/1451278064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 msgr2=0x7f1fd815fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:32.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.053+0000 7f1fde421640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 0x7f1fd815fe70 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800d830 tx=0x7f1fc800dcf0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.053+0000 7f1fde421640 1 -- 172.21.15.115:0/1451278064 shutdown_connections 2023-12-01T01:52:32.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.053+0000 7f1fde421640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fc406a550 0x7f1fc406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.053+0000 7f1fde421640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fd8070d30 0x7f1fd80cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.053+0000 7f1fde421640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fd815cf10 0x7f1fd815ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.053+0000 7f1fde421640 1 --2- 172.21.15.115:0/1451278064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fd80705e0 0x7f1fd815fe70 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.053+0000 7f1fde421640 1 -- 172.21.15.115:0/1451278064 >> 172.21.15.115:0/1451278064 conn(0x7f1fd80b1720 msgr2=0x7f1fd80adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:32.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.054+0000 7f1fde421640 1 -- 172.21.15.115:0/1451278064 shutdown_connections 2023-12-01T01:52:32.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.054+0000 7f1fde421640 1 -- 172.21.15.115:0/1451278064 wait complete. 2023-12-01T01:52:32.070 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:52:32.078 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.072+0000 7fbee1631040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:52:32.155 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.154+0000 7fbe662fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:32.156 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.154+0000 7fbe662fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:32.156 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.154+0000 7fbe662fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:32.156 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.154+0000 7fbe662fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:32.156 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.155+0000 7fbe662fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:32.156 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.155+0000 7fbe62af3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:32.156 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.155+0000 7fbe62af3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:32.156 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.155+0000 7fbe62af3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:32.156 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.155+0000 7fbe62af3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:32.156 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:52:32.155+0000 7fbe62af3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:52:32.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.256+0000 7f83d15fd640 1 Processor -- start 2023-12-01T01:52:32.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.256+0000 7f83d15fd640 1 -- start start 2023-12-01T01:52:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83d15fd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f83cc153c30 0x7f83cc0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83d15fd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f83cc154010 0x7f83cc0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83d15fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83cc0cf9b0 0x7f83cc160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83d15fd640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83cc0cdcd0 con 0x7f83cc154010 2023-12-01T01:52:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83d15fd640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83cc0cfe20 con 0x7f83cc153c30 2023-12-01T01:52:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83d15fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc161830 con 0x7f83cc0cf9b0 2023-12-01T01:52:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83c3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f83cc154010 0x7f83cc0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83c3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f83cc154010 0x7f83cc0cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37630/0 (socket says 172.21.15.115:37630) 2023-12-01T01:52:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83c3fff640 1 -- 172.21.15.115:0/780560073 learned_addr learned my addr 172.21.15.115:0/780560073 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83c3fff640 1 -- 172.21.15.115:0/780560073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83cc0cf9b0 msgr2=0x7f83cc160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:32.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83c3fff640 1 --2- 172.21.15.115:0/780560073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83cc0cf9b0 0x7f83cc160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83c3fff640 1 -- 172.21.15.115:0/780560073 >> v2:172.21.15.142:3300/0 conn(0x7f83cc153c30 msgr2=0x7f83cc0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:32.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83c3fff640 1 --2- 172.21.15.115:0/780560073 >> v2:172.21.15.142:3300/0 conn(0x7f83cc153c30 0x7f83cc0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.258+0000 7f83c3fff640 1 -- 172.21.15.115:0/780560073 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc0cd3a0 con 0x7f83cc154010 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.259+0000 7f83c3fff640 1 --2- 172.21.15.115:0/780560073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc154010 0x7f83cc0cf470 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0d3800 tx=0x7f83b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f7af3e8049761b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83cb7fe640 1 -- 172.21.15.115:0/780560073 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4031870 con 0x7f83cc154010 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83cb7fe640 1 -- 172.21.15.115:0/780560073 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b40319d0 con 0x7f83cc154010 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83cb7fe640 1 -- 172.21.15.115:0/780560073 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4031d60 con 0x7f83cc154010 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83d15fd640 1 -- 172.21.15.115:0/780560073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc154010 msgr2=0x7f83cc0cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83d15fd640 1 --2- 172.21.15.115:0/780560073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc154010 0x7f83cc0cf470 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0d3800 tx=0x7f83b402b910 comp rx=0 tx=0).stop 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83d15fd640 1 -- 172.21.15.115:0/780560073 shutdown_connections 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83d15fd640 1 --2- 172.21.15.115:0/780560073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83cc0cf9b0 0x7f83cc160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83d15fd640 1 --2- 172.21.15.115:0/780560073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc154010 0x7f83cc0cf470 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83d15fd640 1 --2- 172.21.15.115:0/780560073 >> v2:172.21.15.142:3300/0 conn(0x7f83cc153c30 0x7f83cc0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83d15fd640 1 -- 172.21.15.115:0/780560073 >> 172.21.15.115:0/780560073 conn(0x7f83cc06a0b0 msgr2=0x7f83cc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:32.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83d15fd640 1 -- 172.21.15.115:0/780560073 shutdown_connections 2023-12-01T01:52:32.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.260+0000 7f83d15fd640 1 -- 172.21.15.115:0/780560073 wait complete. 2023-12-01T01:52:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.262+0000 7f83d15fd640 1 Processor -- start 2023-12-01T01:52:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.262+0000 7f83d15fd640 1 -- start start 2023-12-01T01:52:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.263+0000 7f83d15fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83cc0cf9b0 0x7f83cc0c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.263+0000 7f83d15fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc153c30 0x7f83cc144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.263+0000 7f83d15fd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83cc154010 0x7f83cc0d1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.263+0000 7f83d15fd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc1464d0 con 0x7f83cc153c30 2023-12-01T01:52:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.263+0000 7f83d15fd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0c5340 con 0x7f83cc154010 2023-12-01T01:52:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.263+0000 7f83d15fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc145850 con 0x7f83cc0cf9b0 2023-12-01T01:52:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.263+0000 7f83c3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc153c30 0x7f83cc144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.263+0000 7f83c3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc153c30 0x7f83cc144e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37640/0 (socket says 172.21.15.115:37640) 2023-12-01T01:52:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.263+0000 7f83c3fff640 1 -- 172.21.15.115:0/2496463496 learned_addr learned my addr 172.21.15.115:0/2496463496 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.264+0000 7f83c3fff640 1 -- 172.21.15.115:0/2496463496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83cc0cf9b0 msgr2=0x7f83cc0c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.264+0000 7f83cbfff640 1 --2- 172.21.15.115:0/2496463496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83cc0cf9b0 0x7f83cc0c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.264+0000 7f83c3fff640 1 --2- 172.21.15.115:0/2496463496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83cc0cf9b0 0x7f83cc0c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.264+0000 7f83c3fff640 1 -- 172.21.15.115:0/2496463496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83cc154010 msgr2=0x7f83cc0d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.264+0000 7f83c3fff640 1 --2- 172.21.15.115:0/2496463496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83cc154010 0x7f83cc0d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.264+0000 7f83c3fff640 1 -- 172.21.15.115:0/2496463496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b40094d0 con 0x7f83cc153c30 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.264+0000 7f83cbfff640 1 --2- 172.21.15.115:0/2496463496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83cc0cf9b0 0x7f83cc0c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.264+0000 7f83c3fff640 1 --2- 172.21.15.115:0/2496463496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc153c30 0x7f83cc144e00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f83b40090b0 tx=0x7f83b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.264+0000 7f83c97fa640 1 -- 172.21.15.115:0/2496463496 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b403ad30 con 0x7f83cc153c30 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.265+0000 7f83c97fa640 1 -- 172.21.15.115:0/2496463496 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b40443f0 con 0x7f83cc153c30 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.265+0000 7f83c97fa640 1 -- 172.21.15.115:0/2496463496 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b40447a0 con 0x7f83cc153c30 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.265+0000 7f83d15fd640 1 -- 172.21.15.115:0/2496463496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83cc145b30 con 0x7f83cc153c30 2023-12-01T01:52:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.265+0000 7f83d15fd640 1 -- 172.21.15.115:0/2496463496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83cc1460f0 con 0x7f83cc153c30 2023-12-01T01:52:32.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.266+0000 7f83d15fd640 1 -- 172.21.15.115:0/2496463496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83cc146610 con 0x7f83cc153c30 2023-12-01T01:52:32.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.267+0000 7f83c97fa640 1 -- 172.21.15.115:0/2496463496 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 46) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f83b4044900 con 0x7f83cc153c30 2023-12-01T01:52:32.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.268+0000 7f83c97fa640 1 -- 172.21.15.115:0/2496463496 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f83b402d040 con 0x7f83cc153c30 2023-12-01T01:52:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.273+0000 7f83c97fa640 1 -- 172.21.15.115:0/2496463496 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f83cc146610 con 0x7f83cc153c30 2023-12-01T01:52:32.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.464+0000 7f83d15fd640 1 -- 172.21.15.115:0/2496463496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83cc146610 con 0x7f83cc153c30 2023-12-01T01:52:32.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.465+0000 7f83c97fa640 1 -- 172.21.15.115:0/2496463496 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83cc146610 con 0x7f83cc153c30 2023-12-01T01:52:32.466 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:52:32.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.468+0000 7f83d15fd640 1 -- 172.21.15.115:0/2496463496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc153c30 msgr2=0x7f83cc144e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:32.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.468+0000 7f83d15fd640 1 --2- 172.21.15.115:0/2496463496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc153c30 0x7f83cc144e00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f83b40090b0 tx=0x7f83b40046c0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.468+0000 7f83d15fd640 1 -- 172.21.15.115:0/2496463496 shutdown_connections 2023-12-01T01:52:32.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.468+0000 7f83d15fd640 1 --2- 172.21.15.115:0/2496463496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83cc154010 0x7f83cc0d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.468+0000 7f83d15fd640 1 --2- 172.21.15.115:0/2496463496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83cc153c30 0x7f83cc144e00 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.468+0000 7f83d15fd640 1 --2- 172.21.15.115:0/2496463496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83cc0cf9b0 0x7f83cc0c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.468+0000 7f83d15fd640 1 -- 172.21.15.115:0/2496463496 >> 172.21.15.115:0/2496463496 conn(0x7f83cc06a0b0 msgr2=0x7f83cc0c5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:32.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.468+0000 7f83d15fd640 1 -- 172.21.15.115:0/2496463496 shutdown_connections 2023-12-01T01:52:32.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.468+0000 7f83d15fd640 1 -- 172.21.15.115:0/2496463496 wait complete. 2023-12-01T01:52:32.486 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:52:32.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.696+0000 7f15db500640 1 Processor -- start 2023-12-01T01:52:32.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.696+0000 7f15db500640 1 -- start start 2023-12-01T01:52:32.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.698+0000 7f15db500640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f15d40ce1b0 0x7f15d40cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.698+0000 7f15db500640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40b0e60 0x7f15d40b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.698+0000 7f15db500640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d40cc9a0 con 0x7f15d415ac80 2023-12-01T01:52:32.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.698+0000 7f15db500640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d415b060 con 0x7f15d40ce1b0 2023-12-01T01:52:32.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.698+0000 7f15db500640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40b0ce0 con 0x7f15d40b0e60 2023-12-01T01:52:32.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.698+0000 7f15d929c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f15d415ac80 0x7f15d40cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51392/0 (socket says 172.21.15.115:51392) 2023-12-01T01:52:32.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.698+0000 7f15d929c640 1 -- 172.21.15.115:0/1839594663 learned_addr learned my addr 172.21.15.115:0/1839594663 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:32.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020171115 0 0) 0x7f15d40cc9a0 con 0x7f15d415ac80 2023-12-01T01:52:32.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15c0003520 con 0x7f15d415ac80 2023-12-01T01:52:32.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15d9a9d640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40b0e60 0x7f15d40b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:32.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15d8a9b640 1 --2- 172.21.15.115:0/1839594663 >> v2:172.21.15.142:3300/0 conn(0x7f15d40ce1b0 0x7f15d40cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:32.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3301739385 0 0) 0x7f15c0003520 con 0x7f15d415ac80 2023-12-01T01:52:32.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40b0e60 msgr2=0x7f15d40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40b0e60 0x7f15d40b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 >> v2:172.21.15.142:3300/0 conn(0x7f15d40ce1b0 msgr2=0x7f15d40cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 --2- 172.21.15.115:0/1839594663 >> v2:172.21.15.142:3300/0 conn(0x7f15d40ce1b0 0x7f15d40cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40cbd90 con 0x7f15d415ac80 2023-12-01T01:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f15d0001670 con 0x7f15d415ac80 2023-12-01T01:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 >> v1:172.21.15.115:6789/0 conn(0x7f15d415ac80 legacy=0x7f15d40cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40b0e60 0x7f15c0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c0003ff0 0x7f15c00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0004d00 0x7f15c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d415b060 con 0x7f15c0003ff0 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40b0ce0 con 0x7f15c0004d00 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c000bae0 con 0x7f15d40b0e60 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.699+0000 7f15d8a9b640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c0003ff0 0x7f15c00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.700+0000 7f15d929c640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40b0e60 0x7f15c0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.700+0000 7f15d9a9d640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0004d00 0x7f15c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.700+0000 7f15d8a9b640 1 -- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40b0e60 msgr2=0x7f15c0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.700+0000 7f15d8a9b640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15d40b0e60 0x7f15c0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.700+0000 7f15d8a9b640 1 -- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0004d00 msgr2=0x7f15c0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.700+0000 7f15d8a9b640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0004d00 0x7f15c0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.700+0000 7f15d8a9b640 1 -- 172.21.15.115:0/1839594663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c0003520 con 0x7f15c0003ff0 2023-12-01T01:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.700+0000 7f15d9a9d640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15c0004d00 0x7f15c0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.701+0000 7f15d8a9b640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c0003ff0 0x7f15c00047c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f15c400ba30 tx=0x7f15c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a7e115c0b76454 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:32.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.701+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4032680 con 0x7f15c0003ff0 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.701+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c40327e0 con 0x7f15c0003ff0 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.701+0000 7f15cbfff640 1 -- 172.21.15.115:0/1839594663 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4032b70 con 0x7f15c0003ff0 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.702+0000 7f15db500640 1 -- 172.21.15.115:0/1839594663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c0003ff0 msgr2=0x7f15c00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.702+0000 7f15db500640 1 --2- 172.21.15.115:0/1839594663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15c0003ff0 0x7f15c00047c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f15c400ba30 tx=0x7f15c00057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.702+0000 7f15d9a9d640 1 -- 172.21.15.115:0/1839594663 reap_dead start 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.702+0000 7f15db500640 1 -- 172.21.15.115:0/1839594663 shutdown_connections 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.702+0000 7f15db500640 1 -- 172.21.15.115:0/1839594663 >> 172.21.15.115:0/1839594663 conn(0x7f15d4065960 msgr2=0x7f15d4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.702+0000 7f15db500640 1 -- 172.21.15.115:0/1839594663 shutdown_connections 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.702+0000 7f15db500640 1 -- 172.21.15.115:0/1839594663 wait complete. 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.704+0000 7f15db500640 1 Processor -- start 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.704+0000 7f15db500640 1 -- start start 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15db500640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15c0003ff0 0x7f15d41e6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15db500640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d41e6a70 0x7f15d41e4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15db500640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15d41e4ea0 0x7f15d413fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15db500640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4152eb0 con 0x7f15d41e4ea0 2023-12-01T01:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15db500640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d41e5490 con 0x7f15d41e6a70 2023-12-01T01:52:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15db500640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d41e5770 con 0x7f15c0003ff0 2023-12-01T01:52:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15d9a9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15d41e4ea0 0x7f15d413fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15d9a9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15d41e4ea0 0x7f15d413fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37648/0 (socket says 172.21.15.115:37648) 2023-12-01T01:52:32.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15d9a9d640 1 -- 172.21.15.115:0/1000521350 learned_addr learned my addr 172.21.15.115:0/1000521350 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:32.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15d9a9d640 1 -- 172.21.15.115:0/1000521350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15c0003ff0 msgr2=0x7f15d41e6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:32.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15d9a9d640 1 --2- 172.21.15.115:0/1000521350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15c0003ff0 0x7f15d41e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15d9a9d640 1 -- 172.21.15.115:0/1000521350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d41e6a70 msgr2=0x7f15d41e4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:32.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15d9a9d640 1 --2- 172.21.15.115:0/1000521350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d41e6a70 0x7f15d41e4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.705+0000 7f15d9a9d640 1 -- 172.21.15.115:0/1000521350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d41e5a50 con 0x7f15d41e4ea0 2023-12-01T01:52:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.706+0000 7f15d9a9d640 1 --2- 172.21.15.115:0/1000521350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15d41e4ea0 0x7f15d413fcb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f15d40ca270 tx=0x7f15cc009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.706+0000 7f15c9ffb640 1 -- 172.21.15.115:0/1000521350 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc007cf0 con 0x7f15d41e4ea0 2023-12-01T01:52:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.706+0000 7f15c9ffb640 1 -- 172.21.15.115:0/1000521350 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc007e70 con 0x7f15d41e4ea0 2023-12-01T01:52:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.706+0000 7f15c9ffb640 1 -- 172.21.15.115:0/1000521350 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc014620 con 0x7f15d41e4ea0 2023-12-01T01:52:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.706+0000 7f15db500640 1 -- 172.21.15.115:0/1000521350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d41e5d30 con 0x7f15d41e4ea0 2023-12-01T01:52:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.706+0000 7f15db500640 1 -- 172.21.15.115:0/1000521350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d41401f0 con 0x7f15d41e4ea0 2023-12-01T01:52:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.707+0000 7f15db500640 1 -- 172.21.15.115:0/1000521350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d41e6250 con 0x7f15d41e4ea0 2023-12-01T01:52:32.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.708+0000 7f15c9ffb640 1 -- 172.21.15.115:0/1000521350 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 46) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f15cc01d3b0 con 0x7f15d41e4ea0 2023-12-01T01:52:32.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.709+0000 7f15c9ffb640 1 -- 172.21.15.115:0/1000521350 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f15cc088360 con 0x7f15d41e4ea0 2023-12-01T01:52:32.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.713+0000 7f15c9ffb640 1 -- 172.21.15.115:0/1000521350 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f15d41e6250 con 0x7f15d41e4ea0 2023-12-01T01:52:32.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.895+0000 7f15db500640 1 -- 172.21.15.115:0/1000521350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15d41e6250 con 0x7f15d41e4ea0 2023-12-01T01:52:32.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.896+0000 7f15c9ffb640 1 -- 172.21.15.115:0/1000521350 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15d41e6250 con 0x7f15d41e4ea0 2023-12-01T01:52:32.896 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:52:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.898+0000 7f15db500640 1 -- 172.21.15.115:0/1000521350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15d41e4ea0 msgr2=0x7f15d413fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.898+0000 7f15db500640 1 --2- 172.21.15.115:0/1000521350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15d41e4ea0 0x7f15d413fcb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f15d40ca270 tx=0x7f15cc009d20 comp rx=0 tx=0).stop 2023-12-01T01:52:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.899+0000 7f15db500640 1 -- 172.21.15.115:0/1000521350 shutdown_connections 2023-12-01T01:52:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.899+0000 7f15db500640 1 --2- 172.21.15.115:0/1000521350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15d41e4ea0 0x7f15d413fcb0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.899+0000 7f15db500640 1 --2- 172.21.15.115:0/1000521350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15d41e6a70 0x7f15d41e4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.899+0000 7f15db500640 1 --2- 172.21.15.115:0/1000521350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15c0003ff0 0x7f15d41e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.899+0000 7f15db500640 1 -- 172.21.15.115:0/1000521350 >> 172.21.15.115:0/1000521350 conn(0x7f15d4065960 msgr2=0x7f15d4070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.899+0000 7f15db500640 1 -- 172.21.15.115:0/1000521350 shutdown_connections 2023-12-01T01:52:32.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:32.899+0000 7f15db500640 1 -- 172.21.15.115:0/1000521350 wait complete. 2023-12-01T01:52:32.917 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:52:33.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.125+0000 7f478ffff640 1 Processor -- start 2023-12-01T01:52:33.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.125+0000 7f478ffff640 1 -- start start 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.126+0000 7f478ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f47900ae1a0 0x7f47900ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.126+0000 7f478ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f479006c470 0x7f4790069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.126+0000 7f478ffff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47900ccfe0 con 0x7f479006a200 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.126+0000 7f478ffff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47900c9520 con 0x7f47900ae1a0 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.126+0000 7f478ffff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479006a620 con 0x7f479006c470 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.127+0000 7f478effd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f47900ae1a0 0x7f47900ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.127+0000 7f478effd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f47900ae1a0 0x7f47900ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37662/0 (socket says 172.21.15.115:37662) 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.127+0000 7f478effd640 1 -- 172.21.15.115:0/949550804 learned_addr learned my addr 172.21.15.115:0/949550804 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.127+0000 7f478f7fe640 1 --1- 172.21.15.115:0/949550804 >> v1:172.21.15.115:6789/0 conn(0x7f479006a200 0x7f4790160730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51402/0 (socket says 172.21.15.115:51402) 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.127+0000 7f478effd640 1 -- 172.21.15.115:0/949550804 >> v2:172.21.15.142:3300/0 conn(0x7f479006c470 msgr2=0x7f4790069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.127+0000 7f47867fc640 1 --2- 172.21.15.115:0/949550804 >> v2:172.21.15.142:3300/0 conn(0x7f479006c470 0x7f4790069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:33.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.127+0000 7f478e7fc640 1 -- 172.21.15.115:0/949550804 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410211110 0 0) 0x7f4788001110 con 0x7f479006a200 2023-12-01T01:52:33.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.127+0000 7f478effd640 1 --2- 172.21.15.115:0/949550804 >> v2:172.21.15.142:3300/0 conn(0x7f479006c470 0x7f4790069cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.127+0000 7f478effd640 1 -- 172.21.15.115:0/949550804 >> v1:172.21.15.115:6789/0 conn(0x7f479006a200 legacy=0x7f4790160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.127+0000 7f478effd640 1 -- 172.21.15.115:0/949550804 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47900ce5a0 con 0x7f47900ae1a0 2023-12-01T01:52:33.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.128+0000 7f478effd640 1 --2- 172.21.15.115:0/949550804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47900ae1a0 0x7f47900ccaa0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f479006ae70 tx=0x7f478002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956672551c20f05c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:33.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.128+0000 7f478e7fc640 1 -- 172.21.15.115:0/949550804 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478002cb20 con 0x7f47900ae1a0 2023-12-01T01:52:33.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.128+0000 7f478e7fc640 1 -- 172.21.15.115:0/949550804 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478002cc80 con 0x7f47900ae1a0 2023-12-01T01:52:33.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.128+0000 7f478e7fc640 1 -- 172.21.15.115:0/949550804 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47800343f0 con 0x7f47900ae1a0 2023-12-01T01:52:33.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.129+0000 7f478ffff640 1 -- 172.21.15.115:0/949550804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47900ae1a0 msgr2=0x7f47900ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.129+0000 7f478ffff640 1 --2- 172.21.15.115:0/949550804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47900ae1a0 0x7f47900ccaa0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f479006ae70 tx=0x7f478002bf80 comp rx=0 tx=0).stop 2023-12-01T01:52:33.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.129+0000 7f478ffff640 1 -- 172.21.15.115:0/949550804 shutdown_connections 2023-12-01T01:52:33.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.129+0000 7f478ffff640 1 --2- 172.21.15.115:0/949550804 >> v2:172.21.15.142:3300/0 conn(0x7f479006c470 0x7f4790069cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.129+0000 7f478ffff640 1 --2- 172.21.15.115:0/949550804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47900ae1a0 0x7f47900ccaa0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.129+0000 7f478ffff640 1 -- 172.21.15.115:0/949550804 >> 172.21.15.115:0/949550804 conn(0x7f47900ceed0 msgr2=0x7f47900cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:33.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.129+0000 7f478ffff640 1 -- 172.21.15.115:0/949550804 shutdown_connections 2023-12-01T01:52:33.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.129+0000 7f478ffff640 1 -- 172.21.15.115:0/949550804 wait complete. 2023-12-01T01:52:33.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.131+0000 7f478ffff640 1 Processor -- start 2023-12-01T01:52:33.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.131+0000 7f478ffff640 1 -- start start 2023-12-01T01:52:33.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f479006a200 0x7f47901f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479006c470 0x7f47900c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47900ae1a0 0x7f47900c1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478ffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47900c3760 con 0x7f479006a200 2023-12-01T01:52:33.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478ffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47901f3300 con 0x7f47900ae1a0 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47900c2b50 con 0x7f479006c470 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f479006a200 0x7f47901f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f479006a200 0x7f47901f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37668/0 (socket says 172.21.15.115:37668) 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478effd640 1 -- 172.21.15.115:0/2999014495 learned_addr learned my addr 172.21.15.115:0/2999014495 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478effd640 1 -- 172.21.15.115:0/2999014495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479006c470 msgr2=0x7f47900c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478effd640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479006c470 0x7f47900c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478effd640 1 -- 172.21.15.115:0/2999014495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47900ae1a0 msgr2=0x7f47900c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478effd640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47900ae1a0 0x7f47900c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.132+0000 7f478effd640 1 -- 172.21.15.115:0/2999014495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4780009b30 con 0x7f479006a200 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.133+0000 7f478effd640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f479006a200 0x7f47901f2dc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4780000f80 tx=0x7f4780004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:33.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.133+0000 7f4787fff640 1 -- 172.21.15.115:0/2999014495 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780009510 con 0x7f479006a200 2023-12-01T01:52:33.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.133+0000 7f4787fff640 1 -- 172.21.15.115:0/2999014495 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478003e450 con 0x7f479006a200 2023-12-01T01:52:33.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.133+0000 7f4787fff640 1 -- 172.21.15.115:0/2999014495 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478003e800 con 0x7f479006a200 2023-12-01T01:52:33.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.133+0000 7f478ffff640 1 -- 172.21.15.115:0/2999014495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47900c2e30 con 0x7f479006a200 2023-12-01T01:52:33.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.133+0000 7f478ffff640 1 -- 172.21.15.115:0/2999014495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47900c33f0 con 0x7f479006a200 2023-12-01T01:52:33.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.134+0000 7f478ffff640 1 -- 172.21.15.115:0/2999014495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47900673a0 con 0x7f479006a200 2023-12-01T01:52:33.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.135+0000 7f4787fff640 1 -- 172.21.15.115:0/2999014495 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 47) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f478003e960 con 0x7f479006a200 2023-12-01T01:52:33.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.136+0000 7f4787fff640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f477006a280 0x7f477006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.136+0000 7f4787fff640 1 -- 172.21.15.115:0/2999014495 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f47800ab5b0 con 0x7f479006a200 2023-12-01T01:52:33.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.137+0000 7f47867fc640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f477006a280 0x7f477006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:33.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.137+0000 7f47867fc640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f477006a280 0x7f477006c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4778006fd0 tx=0x7f4778008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:33.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.140+0000 7f4787fff640 1 -- 172.21.15.115:0/2999014495 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4780076bb0 con 0x7f479006a200 2023-12-01T01:52:33.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.366+0000 7f478ffff640 1 -- 172.21.15.115:0/2999014495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f479006ceb0 con 0x7f479006a200 2023-12-01T01:52:33.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.367+0000 7f4787fff640 1 -- 172.21.15.115:0/2999014495 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f478007a1d0 con 0x7f479006a200 2023-12-01T01:52:33.368 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:33.368 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/542800961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":542800961},{"type":"v1","addr":"172.21.15.142:6833","nonce":542800961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6942,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2035657957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2035657957},{"type":"v1","addr":"172.21.15.115:6835","nonce":2035657957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3644451216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3644451216},{"type":"v1","addr":"172.21.15.115:6837","nonce":3644451216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:06.454774+0000","modified":"2023-12-01T01:52:09.476985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6957":{"gid":6957,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1311626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1311626228},{"type":"v1","addr":"172.21.15.142:6835","nonce":1311626228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-01T01:52:33.368 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2023-12-01T01:52:33.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 -- 172.21.15.115:0/2999014495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f477006a280 msgr2=0x7f477006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f477006a280 0x7f477006c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4778006fd0 tx=0x7f4778008040 comp rx=0 tx=0).stop 2023-12-01T01:52:33.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 -- 172.21.15.115:0/2999014495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f479006a200 msgr2=0x7f47901f2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f479006a200 0x7f47901f2dc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4780000f80 tx=0x7f4780004a40 comp rx=0 tx=0).stop 2023-12-01T01:52:33.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 -- 172.21.15.115:0/2999014495 shutdown_connections 2023-12-01T01:52:33.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f477006a280 0x7f477006c740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f479006c470 0x7f47900c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47900ae1a0 0x7f47900c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 --2- 172.21.15.115:0/2999014495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f479006a200 0x7f47901f2dc0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 -- 172.21.15.115:0/2999014495 >> 172.21.15.115:0/2999014495 conn(0x7f47900ceed0 msgr2=0x7f47900716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:33.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 -- 172.21.15.115:0/2999014495 shutdown_connections 2023-12-01T01:52:33.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.370+0000 7f478ffff640 1 -- 172.21.15.115:0/2999014495 wait complete. 2023-12-01T01:52:33.389 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.596+0000 7f62015e3640 1 Processor -- start 2023-12-01T01:52:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.596+0000 7f62015e3640 1 -- start start 2023-12-01T01:52:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.597+0000 7f62015e3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f61fc15e2f0 0x7f61fc0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.597+0000 7f62015e3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc0add10 con 0x7f61fc06f2f0 2023-12-01T01:52:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.597+0000 7f62015e3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61fc0c5760 con 0x7f61fc0adf10 2023-12-01T01:52:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.597+0000 7f62015e3640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61fc0c9cf0 con 0x7f61fc15e2f0 2023-12-01T01:52:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.598+0000 7f61f3fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f61fc06f2f0 0x7f61fc0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51404/0 (socket says 172.21.15.115:51404) 2023-12-01T01:52:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.598+0000 7f61f3fff640 1 -- 172.21.15.115:0/2715440284 learned_addr learned my addr 172.21.15.115:0/2715440284 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.598+0000 7f61faffd640 1 --2- 172.21.15.115:0/2715440284 >> v2:172.21.15.142:3300/0 conn(0x7f61fc15e2f0 0x7f61fc0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.598+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292553586 0 0) 0x7f61fc0add10 con 0x7f61fc06f2f0 2023-12-01T01:52:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.598+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e8003520 con 0x7f61fc06f2f0 2023-12-01T01:52:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 852109076 0 0) 0x7f61e8003520 con 0x7f61fc06f2f0 2023-12-01T01:52:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 >> v2:172.21.15.142:3300/0 conn(0x7f61fc15e2f0 msgr2=0x7f61fc0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 --2- 172.21.15.115:0/2715440284 >> v2:172.21.15.142:3300/0 conn(0x7f61fc15e2f0 0x7f61fc0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 >> v1:172.21.15.142:6789/0 conn(0x7f61fc0adf10 legacy=0x7f61fc0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc15e6d0 con 0x7f61fc06f2f0 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f61e4001670 con 0x7f61fc06f2f0 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 >> v1:172.21.15.115:6789/0 conn(0x7f61fc06f2f0 legacy=0x7f61fc0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 --2- 172.21.15.115:0/2715440284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e80039e0 0x7f61e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 --2- 172.21.15.115:0/2715440284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61e80047c0 0x7f61e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 --2- 172.21.15.115:0/2715440284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e8005280 0x7f61e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0c9cf0 con 0x7f61e80039e0 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e800c1c0 con 0x7f61e8005280 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e800c380 con 0x7f61e80047c0 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61faffd640 1 --2- 172.21.15.115:0/2715440284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e80039e0 0x7f61e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61faffd640 1 -- 172.21.15.115:0/2715440284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61e80047c0 msgr2=0x7f61e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.599+0000 7f61faffd640 1 --2- 172.21.15.115:0/2715440284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61e80047c0 0x7f61e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.600+0000 7f61faffd640 1 -- 172.21.15.115:0/2715440284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e8005280 msgr2=0x7f61e800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.600+0000 7f61faffd640 1 --2- 172.21.15.115:0/2715440284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e8005280 0x7f61e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.600+0000 7f61faffd640 1 -- 172.21.15.115:0/2715440284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e8003520 con 0x7f61e80039e0 2023-12-01T01:52:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.600+0000 7f61fb7fe640 1 -- 172.21.15.115:0/2715440284 reap_dead start 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.601+0000 7f61faffd640 1 --2- 172.21.15.115:0/2715440284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e80039e0 0x7f61e8003e00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009ac0 tx=0x7f61e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b8ff27940c535e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.601+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec02c710 con 0x7f61e80039e0 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.601+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec02c870 con 0x7f61e80039e0 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.601+0000 7f61fa7fc640 1 -- 172.21.15.115:0/2715440284 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec02cc00 con 0x7f61e80039e0 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.601+0000 7f62015e3640 1 -- 172.21.15.115:0/2715440284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e80039e0 msgr2=0x7f61e8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.601+0000 7f62015e3640 1 --2- 172.21.15.115:0/2715440284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e80039e0 0x7f61e8003e00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009ac0 tx=0x7f61e80057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.602+0000 7f62015e3640 1 -- 172.21.15.115:0/2715440284 shutdown_connections 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.602+0000 7f62015e3640 1 --2- 172.21.15.115:0/2715440284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e80039e0 0x7f61e8003e00 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.602+0000 7f62015e3640 1 -- 172.21.15.115:0/2715440284 >> 172.21.15.115:0/2715440284 conn(0x7f61fc0ce880 msgr2=0x7f61fc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.602+0000 7f62015e3640 1 -- 172.21.15.115:0/2715440284 shutdown_connections 2023-12-01T01:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.602+0000 7f62015e3640 1 -- 172.21.15.115:0/2715440284 wait complete. 2023-12-01T01:52:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.604+0000 7f62015e3640 1 Processor -- start 2023-12-01T01:52:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.604+0000 7f62015e3640 1 -- start start 2023-12-01T01:52:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f62015e3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80039e0 0x7f61fc1f03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f62015e3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61fc0c2680 0x7f61fc0af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f62015e3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61fc0c5950 0x7f61fc0af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f62015e3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0c2ad0 con 0x7f61fc0c2680 2023-12-01T01:52:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f62015e3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0c8420 con 0x7f61e80039e0 2023-12-01T01:52:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f62015e3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61fc0c8700 con 0x7f61fc0c5950 2023-12-01T01:52:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61fc0c2680 0x7f61fc0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f61f3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61fc0c2680 0x7f61fc0af310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37678/0 (socket says 172.21.15.115:37678) 2023-12-01T01:52:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f61f3fff640 1 -- 172.21.15.115:0/1015518510 learned_addr learned my addr 172.21.15.115:0/1015518510 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f61faffd640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80039e0 0x7f61fc1f03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f61f3fff640 1 -- 172.21.15.115:0/1015518510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61fc0c5950 msgr2=0x7f61fc0af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f61f3fff640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61fc0c5950 0x7f61fc0af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f61f3fff640 1 -- 172.21.15.115:0/1015518510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80039e0 msgr2=0x7f61fc1f03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f61f3fff640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80039e0 0x7f61fc1f03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f61f3fff640 1 -- 172.21.15.115:0/1015518510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc0c2d20 con 0x7f61fc0c2680 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.605+0000 7f61faffd640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80039e0 0x7f61fc1f03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.606+0000 7f61f3fff640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61fc0c2680 0x7f61fc0af310 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f61e40010e0 tx=0x7f61e400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.606+0000 7f61f37fe640 1 -- 172.21.15.115:0/1015518510 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40074e0 con 0x7f61fc0c2680 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.606+0000 7f61f37fe640 1 -- 172.21.15.115:0/1015518510 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e4007660 con 0x7f61fc0c2680 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.606+0000 7f61f37fe640 1 -- 172.21.15.115:0/1015518510 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4004a20 con 0x7f61fc0c2680 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.606+0000 7f62015e3640 1 -- 172.21.15.115:0/1015518510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61fc0c3000 con 0x7f61fc0c2680 2023-12-01T01:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.606+0000 7f62015e3640 1 -- 172.21.15.115:0/1015518510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61fc0c8060 con 0x7f61fc0c2680 2023-12-01T01:52:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.607+0000 7f62015e3640 1 -- 172.21.15.115:0/1015518510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61fc06fda0 con 0x7f61fc0c2680 2023-12-01T01:52:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.608+0000 7f61f37fe640 1 -- 172.21.15.115:0/1015518510 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 47) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f61e4004b80 con 0x7f61fc0c2680 2023-12-01T01:52:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.609+0000 7f61f37fe640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61dc06a280 0x7f61dc06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.609+0000 7f61f37fe640 1 -- 172.21.15.115:0/1015518510 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f61e4087cb0 con 0x7f61fc0c2680 2023-12-01T01:52:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.609+0000 7f61faffd640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61dc06a280 0x7f61dc06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:33.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.610+0000 7f61faffd640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61dc06a280 0x7f61dc06c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f61ec036000 tx=0x7f61ec030000 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:33.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.613+0000 7f61f37fe640 1 -- 172.21.15.115:0/1015518510 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f61e40532b0 con 0x7f61fc0c2680 2023-12-01T01:52:33.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.808+0000 7f62015e3640 1 -- 172.21.15.115:0/1015518510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61fc0c3240 con 0x7f61fc0c2680 2023-12-01T01:52:33.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.811+0000 7f61f37fe640 1 -- 172.21.15.115:0/1015518510 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16963 (secure 0 0 0) 0x7f61fc0c3240 con 0x7f61fc0c2680 2023-12-01T01:52:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:33.812 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":99,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:32.074823+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:04.909384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:05.480768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/1457241066":"2023-12-02T01:52:32.074792+0000","172.21.15.115:0/2170465963":"2023-12-02T01:52:32.074792+0000","172.21.15.115:0/1482402703":"2023-12-02T01:52:32.074792+0000","172.21.15.115:0/202387737":"2023-12-02T01:52:32.074792+0000","172.21.15.115:6822/88317":"2023-12-02T01:52:25.734961+0000","172.21.15.115:6824/88317":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/3373285076":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/3569012860":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/709337731":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/1296968550":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/3605273682":"2023-12-02T01:52:32.074792+0000","172.21.15.115:0/1490819667":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/1208012610":"2023-12-02T01:52:32.074792+0000","172.21.15.115:0/1878415381":"2023-12-02T01:52:25.734961+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:33.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 -- 172.21.15.115:0/1015518510 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61dc06a280 msgr2=0x7f61dc06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61dc06a280 0x7f61dc06c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f61ec036000 tx=0x7f61ec030000 comp rx=0 tx=0).stop 2023-12-01T01:52:33.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 -- 172.21.15.115:0/1015518510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61fc0c2680 msgr2=0x7f61fc0af310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:33.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61fc0c2680 0x7f61fc0af310 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f61e40010e0 tx=0x7f61e400cae0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 -- 172.21.15.115:0/1015518510 shutdown_connections 2023-12-01T01:52:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61dc06a280 0x7f61dc06c740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61fc0c5950 0x7f61fc0af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61fc0c2680 0x7f61fc0af310 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 --2- 172.21.15.115:0/1015518510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80039e0 0x7f61fc1f03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 -- 172.21.15.115:0/1015518510 >> 172.21.15.115:0/1015518510 conn(0x7f61fc0ce880 msgr2=0x7f61fc0c8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 -- 172.21.15.115:0/1015518510 shutdown_connections 2023-12-01T01:52:33.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:33.814+0000 7f62015e3640 1 -- 172.21.15.115:0/1015518510 wait complete. 2023-12-01T01:52:33.834 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T01:52:33.834 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:52:34.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.049+0000 7f2ae2643640 1 Processor -- start 2023-12-01T01:52:34.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.049+0000 7f2ae2643640 1 -- start start 2023-12-01T01:52:34.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.050+0000 7f2ae2643640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0cad70 0x7f2adc06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.050+0000 7f2ae2643640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2adc06f2f0 0x7f2adc06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.050+0000 7f2ae2643640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2adc0add10 con 0x7f2adc15e2f0 2023-12-01T01:52:34.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.050+0000 7f2ae2643640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2adc0c5760 con 0x7f2adc06f2f0 2023-12-01T01:52:34.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.050+0000 7f2ae2643640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc15e6d0 con 0x7f2adc0cad70 2023-12-01T01:52:34.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adbfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2adc15e2f0 0x7f2adc0ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51412/0 (socket says 172.21.15.115:51412) 2023-12-01T01:52:34.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adbfff640 1 -- 172.21.15.115:0/3084705972 learned_addr learned my addr 172.21.15.115:0/3084705972 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:34.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2ae0be0640 1 --2- 172.21.15.115:0/3084705972 >> v2:172.21.15.142:3300/0 conn(0x7f2adc06f2f0 0x7f2adc06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adb7fe640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0cad70 0x7f2adc06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678286716 0 0) 0x7f2adc0add10 con 0x7f2adc15e2f0 2023-12-01T01:52:34.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ac4003520 con 0x7f2adc15e2f0 2023-12-01T01:52:34.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4123227126 0 0) 0x7f2ac4003520 con 0x7f2adc15e2f0 2023-12-01T01:52:34.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0cad70 msgr2=0x7f2adc06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:34.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adaffd640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0cad70 0x7f2adc06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 >> v2:172.21.15.142:3300/0 conn(0x7f2adc06f2f0 msgr2=0x7f2adc06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:34.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adaffd640 1 --2- 172.21.15.115:0/3084705972 >> v2:172.21.15.142:3300/0 conn(0x7f2adc06f2f0 0x7f2adc06bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc0cf7b0 con 0x7f2adc15e2f0 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.051+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2acc001670 con 0x7f2adc15e2f0 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2ae0be0640 1 --2- 172.21.15.115:0/3084705972 >> v2:172.21.15.142:3300/0 conn(0x7f2adc06f2f0 0x7f2adc06bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 >> v1:172.21.15.115:6789/0 conn(0x7f2adc15e2f0 legacy=0x7f2adc0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adaffd640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 0x7f2ac4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adaffd640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0cad70 0x7f2ac40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adaffd640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4004d00 0x7f2ac4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc0c5760 con 0x7f2ac40039e0 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc15e6d0 con 0x7f2ac4004d00 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac400fff0 con 0x7f2adc0cad70 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adbfff640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 0x7f2ac4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2ae0be0640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4004d00 0x7f2ac4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adbfff640 1 -- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0cad70 msgr2=0x7f2ac40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adb7fe640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0cad70 0x7f2ac40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adbfff640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0cad70 0x7f2ac40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adbfff640 1 -- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4004d00 msgr2=0x7f2ac4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:34.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adbfff640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4004d00 0x7f2ac4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2adbfff640 1 -- 172.21.15.115:0/3084705972 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac4003520 con 0x7f2ac40039e0 2023-12-01T01:52:34.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.052+0000 7f2ae0be0640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ac4004d00 0x7f2ac4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:34.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.053+0000 7f2adbfff640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 0x7f2ac4003e00 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00c240 tx=0x7f2ac40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b97d7486d4b16a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:34.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.054+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc02cb00 con 0x7f2ac40039e0 2023-12-01T01:52:34.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.054+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2acc02cc60 con 0x7f2ac40039e0 2023-12-01T01:52:34.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.054+0000 7f2adaffd640 1 -- 172.21.15.115:0/3084705972 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc005af0 con 0x7f2ac40039e0 2023-12-01T01:52:34.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.054+0000 7f2ae2643640 1 -- 172.21.15.115:0/3084705972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 msgr2=0x7f2ac4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.054+0000 7f2ae2643640 1 --2- 172.21.15.115:0/3084705972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 0x7f2ac4003e00 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00c240 tx=0x7f2ac40057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.054+0000 7f2ae0be0640 1 -- 172.21.15.115:0/3084705972 reap_dead start 2023-12-01T01:52:34.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.054+0000 7f2ae2643640 1 -- 172.21.15.115:0/3084705972 shutdown_connections 2023-12-01T01:52:34.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.054+0000 7f2ae2643640 1 -- 172.21.15.115:0/3084705972 >> 172.21.15.115:0/3084705972 conn(0x7f2adc0ce880 msgr2=0x7f2adc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:34.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.054+0000 7f2ae2643640 1 -- 172.21.15.115:0/3084705972 shutdown_connections 2023-12-01T01:52:34.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.054+0000 7f2ae2643640 1 -- 172.21.15.115:0/3084705972 wait complete. 2023-12-01T01:52:34.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.056+0000 7f2ae2643640 1 Processor -- start 2023-12-01T01:52:34.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.056+0000 7f2ae2643640 1 -- start start 2023-12-01T01:52:34.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.057+0000 7f2ae2643640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 0x7f2adc159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.057+0000 7f2ae2643640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2adc0c2860 0x7f2adc0c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.057+0000 7f2ae2643640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0c3220 0x7f2adc0c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.057+0000 7f2ae2643640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc0c6d00 con 0x7f2ac40039e0 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.057+0000 7f2ae2643640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc0b37f0 con 0x7f2adc0c2860 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.057+0000 7f2ae2643640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc0b2e90 con 0x7f2adc0c3220 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.057+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 0x7f2adc159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2adbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 0x7f2adc159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37684/0 (socket says 172.21.15.115:37684) 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2adbfff640 1 -- 172.21.15.115:0/2875035009 learned_addr learned my addr 172.21.15.115:0/2875035009 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2adb7fe640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2adc0c2860 0x7f2adc0c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2ae0be0640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0c3220 0x7f2adc0c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2adbfff640 1 -- 172.21.15.115:0/2875035009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0c3220 msgr2=0x7f2adc0c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2adbfff640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0c3220 0x7f2adc0c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2adbfff640 1 -- 172.21.15.115:0/2875035009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2adc0c2860 msgr2=0x7f2adc0c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2adbfff640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2adc0c2860 0x7f2adc0c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2adbfff640 1 -- 172.21.15.115:0/2875035009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc0c36a0 con 0x7f2ac40039e0 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2ae0be0640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0c3220 0x7f2adc0c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:34.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2adb7fe640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2adc0c2860 0x7f2adc0c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:34.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2adbfff640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 0x7f2adc159050 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00c360 tx=0x7f2acc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:34.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.058+0000 7f2ad8ff9640 1 -- 172.21.15.115:0/2875035009 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc039070 con 0x7f2ac40039e0 2023-12-01T01:52:34.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.059+0000 7f2ad8ff9640 1 -- 172.21.15.115:0/2875035009 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2acc001630 con 0x7f2ac40039e0 2023-12-01T01:52:34.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.059+0000 7f2ad8ff9640 1 -- 172.21.15.115:0/2875035009 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc03c040 con 0x7f2ac40039e0 2023-12-01T01:52:34.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.059+0000 7f2ae2643640 1 -- 172.21.15.115:0/2875035009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2adc0b2310 con 0x7f2ac40039e0 2023-12-01T01:52:34.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.059+0000 7f2ae2643640 1 -- 172.21.15.115:0/2875035009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc0b28d0 con 0x7f2ac40039e0 2023-12-01T01:52:34.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.060+0000 7f2ae2643640 1 -- 172.21.15.115:0/2875035009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2adc06fda0 con 0x7f2ac40039e0 2023-12-01T01:52:34.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.061+0000 7f2ad8ff9640 1 -- 172.21.15.115:0/2875035009 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 47) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f2acc032070 con 0x7f2ac40039e0 2023-12-01T01:52:34.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.061+0000 7f2ad8ff9640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ac806a280 0x7f2ac806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.062+0000 7f2ad8ff9640 1 -- 172.21.15.115:0/2875035009 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f2acc0a9c00 con 0x7f2ac40039e0 2023-12-01T01:52:34.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.062+0000 7f2adb7fe640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ac806a280 0x7f2ac806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.062+0000 7f2adb7fe640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ac806a280 0x7f2ac806c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0c8a40 tx=0x7f2ad0008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:34.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.066+0000 7f2ad8ff9640 1 -- 172.21.15.115:0/2875035009 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2acc0752b0 con 0x7f2ac40039e0 2023-12-01T01:52:34.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.310+0000 7f2ae2643640 1 -- 172.21.15.115:0/2875035009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2adc04e890 con 0x7f2ac40039e0 2023-12-01T01:52:34.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.311+0000 7f2ad8ff9640 1 -- 172.21.15.115:0/2875035009 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f2acc046070 con 0x7f2ac40039e0 2023-12-01T01:52:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:34.312 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/542800961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":542800961},{"type":"v1","addr":"172.21.15.142:6833","nonce":542800961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6942,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2035657957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2035657957},{"type":"v1","addr":"172.21.15.115:6835","nonce":2035657957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3644451216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3644451216},{"type":"v1","addr":"172.21.15.115:6837","nonce":3644451216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:06.454774+0000","modified":"2023-12-01T01:52:09.476985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6957":{"gid":6957,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1311626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1311626228},{"type":"v1","addr":"172.21.15.142:6835","nonce":1311626228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-01T01:52:34.312 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2023-12-01T01:52:34.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.313+0000 7f2ae2643640 1 -- 172.21.15.115:0/2875035009 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ac806a280 msgr2=0x7f2ac806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.313+0000 7f2ae2643640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ac806a280 0x7f2ac806c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0c8a40 tx=0x7f2ad0008040 comp rx=0 tx=0).stop 2023-12-01T01:52:34.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.313+0000 7f2ae2643640 1 -- 172.21.15.115:0/2875035009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 msgr2=0x7f2adc159050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.313+0000 7f2ae2643640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 0x7f2adc159050 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00c360 tx=0x7f2acc001a00 comp rx=0 tx=0).stop 2023-12-01T01:52:34.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.313+0000 7f2ae2643640 1 -- 172.21.15.115:0/2875035009 shutdown_connections 2023-12-01T01:52:34.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.313+0000 7f2ae2643640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ac806a280 0x7f2ac806c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.313+0000 7f2ae2643640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2adc0c3220 0x7f2adc0c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.313+0000 7f2ae2643640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2adc0c2860 0x7f2adc0c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.313+0000 7f2ae2643640 1 --2- 172.21.15.115:0/2875035009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ac40039e0 0x7f2adc159050 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.313+0000 7f2ae2643640 1 -- 172.21.15.115:0/2875035009 >> 172.21.15.115:0/2875035009 conn(0x7f2adc0ce880 msgr2=0x7f2adc15acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:34.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.314+0000 7f2ae2643640 1 -- 172.21.15.115:0/2875035009 shutdown_connections 2023-12-01T01:52:34.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.314+0000 7f2ae2643640 1 -- 172.21.15.115:0/2875035009 wait complete. 2023-12-01T01:52:34.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:52:34.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.539+0000 7f939bf47640 1 Processor -- start 2023-12-01T01:52:34.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.539+0000 7f939bf47640 1 -- start start 2023-12-01T01:52:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.541+0000 7f939bf47640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f93940ccc80 0x7f939406fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.541+0000 7f939bf47640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f93940ad2b0 0x7f939406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.541+0000 7f939bf47640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939415d600 con 0x7f939415d220 2023-12-01T01:52:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.541+0000 7f939bf47640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939406cc10 con 0x7f93940ad2b0 2023-12-01T01:52:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.541+0000 7f939bf47640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939406c720 con 0x7f93940ccc80 2023-12-01T01:52:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.541+0000 7f9399ce3640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f939415d220 0x7f93940cc570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51414/0 (socket says 172.21.15.115:51414) 2023-12-01T01:52:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.541+0000 7f9399ce3640 1 -- 172.21.15.115:0/2786371669 learned_addr learned my addr 172.21.15.115:0/2786371669 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:34.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.541+0000 7f939a4e4640 1 --2- 172.21.15.115:0/2786371669 >> v2:172.21.15.115:3300/0 conn(0x7f93940ad2b0 0x7f939406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.541+0000 7f93994e2640 1 --2- 172.21.15.115:0/2786371669 >> v2:172.21.15.142:3300/0 conn(0x7f93940ccc80 0x7f939406fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.541+0000 7f9398ce1640 1 -- 172.21.15.115:0/2786371669 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183846438 0 0) 0x7f9390001110 con 0x7f939415d220 2023-12-01T01:52:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.542+0000 7f9398ce1640 1 -- 172.21.15.115:0/2786371669 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9380003520 con 0x7f939415d220 2023-12-01T01:52:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.542+0000 7f939a4e4640 1 -- 172.21.15.115:0/2786371669 >> v2:172.21.15.142:3300/0 conn(0x7f93940ccc80 msgr2=0x7f939406fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.542+0000 7f939a4e4640 1 --2- 172.21.15.115:0/2786371669 >> v2:172.21.15.142:3300/0 conn(0x7f93940ccc80 0x7f939406fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.542+0000 7f939a4e4640 1 -- 172.21.15.115:0/2786371669 >> v1:172.21.15.115:6789/0 conn(0x7f939415d220 legacy=0x7f93940cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.542+0000 7f939a4e4640 1 -- 172.21.15.115:0/2786371669 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939406c4b0 con 0x7f93940ad2b0 2023-12-01T01:52:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.543+0000 7f939a4e4640 1 --2- 172.21.15.115:0/2786371669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93940ad2b0 0x7f939406fff0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f93940cc4e0 tx=0x7f938c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2c6de13f149680 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.543+0000 7f9398ce1640 1 -- 172.21.15.115:0/2786371669 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c02cbf0 con 0x7f93940ad2b0 2023-12-01T01:52:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.543+0000 7f9398ce1640 1 -- 172.21.15.115:0/2786371669 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c02cd50 con 0x7f93940ad2b0 2023-12-01T01:52:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.543+0000 7f9398ce1640 1 -- 172.21.15.115:0/2786371669 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c034660 con 0x7f93940ad2b0 2023-12-01T01:52:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.544+0000 7f939bf47640 1 -- 172.21.15.115:0/2786371669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93940ad2b0 msgr2=0x7f939406fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.544+0000 7f939bf47640 1 --2- 172.21.15.115:0/2786371669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93940ad2b0 0x7f939406fff0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f93940cc4e0 tx=0x7f938c02be40 comp rx=0 tx=0).stop 2023-12-01T01:52:34.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.544+0000 7f939bf47640 1 -- 172.21.15.115:0/2786371669 shutdown_connections 2023-12-01T01:52:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.544+0000 7f939bf47640 1 --2- 172.21.15.115:0/2786371669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93940ad2b0 0x7f939406fff0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.544+0000 7f939bf47640 1 --2- 172.21.15.115:0/2786371669 >> v2:172.21.15.142:3300/0 conn(0x7f93940ccc80 0x7f939406fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.544+0000 7f939bf47640 1 -- 172.21.15.115:0/2786371669 >> 172.21.15.115:0/2786371669 conn(0x7f93940ce880 msgr2=0x7f93940cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.544+0000 7f939bf47640 1 -- 172.21.15.115:0/2786371669 shutdown_connections 2023-12-01T01:52:34.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.544+0000 7f939bf47640 1 -- 172.21.15.115:0/2786371669 wait complete. 2023-12-01T01:52:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.546+0000 7f939bf47640 1 Processor -- start 2023-12-01T01:52:34.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.546+0000 7f939bf47640 1 -- start start 2023-12-01T01:52:34.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939bf47640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93940ad2b0 0x7f93941f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939bf47640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93940ccc80 0x7f93940b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939bf47640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939415d220 0x7f9394152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939bf47640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93940b3390 con 0x7f939415d220 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939bf47640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93940ca6c0 con 0x7f93940ad2b0 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939bf47640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93940ca3d0 con 0x7f93940ccc80 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939a4e4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939415d220 0x7f9394152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939a4e4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939415d220 0x7f9394152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37726/0 (socket says 172.21.15.115:37726) 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939a4e4640 1 -- 172.21.15.115:0/2477775784 learned_addr learned my addr 172.21.15.115:0/2477775784 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f9399ce3640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93940ad2b0 0x7f93941f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939a4e4640 1 -- 172.21.15.115:0/2477775784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93940ccc80 msgr2=0x7f93940b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939a4e4640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93940ccc80 0x7f93940b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939a4e4640 1 -- 172.21.15.115:0/2477775784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93940ad2b0 msgr2=0x7f93941f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939a4e4640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93940ad2b0 0x7f93941f2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.547+0000 7f939a4e4640 1 -- 172.21.15.115:0/2477775784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c009a70 con 0x7f939415d220 2023-12-01T01:52:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.548+0000 7f939a4e4640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939415d220 0x7f9394152e60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f938c009e80 tx=0x7f938c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.548+0000 7f93867fc640 1 -- 172.21.15.115:0/2477775784 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c040650 con 0x7f939415d220 2023-12-01T01:52:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.548+0000 7f93867fc640 1 -- 172.21.15.115:0/2477775784 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c0407b0 con 0x7f939415d220 2023-12-01T01:52:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.548+0000 7f93867fc640 1 -- 172.21.15.115:0/2477775784 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c040b60 con 0x7f939415d220 2023-12-01T01:52:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.548+0000 7f939bf47640 1 -- 172.21.15.115:0/2477775784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93940b35c0 con 0x7f939415d220 2023-12-01T01:52:34.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.548+0000 7f939bf47640 1 -- 172.21.15.115:0/2477775784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93940b3b80 con 0x7f939415d220 2023-12-01T01:52:34.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.549+0000 7f939bf47640 1 -- 172.21.15.115:0/2477775784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93940ca850 con 0x7f939415d220 2023-12-01T01:52:34.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.550+0000 7f93867fc640 1 -- 172.21.15.115:0/2477775784 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 47) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f938c040cc0 con 0x7f939415d220 2023-12-01T01:52:34.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.552+0000 7f93867fc640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f937806a280 0x7f937806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:34.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.552+0000 7f93867fc640 1 -- 172.21.15.115:0/2477775784 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f938c0ab610 con 0x7f939415d220 2023-12-01T01:52:34.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.552+0000 7f9399ce3640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f937806a280 0x7f937806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:34.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.553+0000 7f9399ce3640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f937806a280 0x7f937806c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9390001270 tx=0x7f9390009040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:34.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.556+0000 7f93867fc640 1 -- 172.21.15.115:0/2477775784 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f938c076c10 con 0x7f939415d220 2023-12-01T01:52:34.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.782+0000 7f939bf47640 1 -- 172.21.15.115:0/2477775784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f939404e890 con 0x7f939415d220 2023-12-01T01:52:34.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.783+0000 7f93867fc640 1 -- 172.21.15.115:0/2477775784 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4659 (secure 0 0 0) 0x7f938c009450 con 0x7f939415d220 2023-12-01T01:52:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:34.784 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/542800961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":542800961},{"type":"v1","addr":"172.21.15.142:6833","nonce":542800961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6942,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2035657957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2035657957},{"type":"v1","addr":"172.21.15.115:6835","nonce":2035657957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3644451216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3644451216},{"type":"v1","addr":"172.21.15.115:6837","nonce":3644451216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:06.454774+0000","modified":"2023-12-01T01:52:09.476985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6957":{"gid":6957,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1311626228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1311626228},{"type":"v1","addr":"172.21.15.142:6835","nonce":1311626228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-01T01:52:34.784 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 40 2023-12-01T01:52:34.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 -- 172.21.15.115:0/2477775784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f937806a280 msgr2=0x7f937806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f937806a280 0x7f937806c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9390001270 tx=0x7f9390009040 comp rx=0 tx=0).stop 2023-12-01T01:52:34.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 -- 172.21.15.115:0/2477775784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939415d220 msgr2=0x7f9394152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:34.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939415d220 0x7f9394152e60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f938c009e80 tx=0x7f938c004a40 comp rx=0 tx=0).stop 2023-12-01T01:52:34.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 -- 172.21.15.115:0/2477775784 shutdown_connections 2023-12-01T01:52:34.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f937806a280 0x7f937806c740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93940ccc80 0x7f93940b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f939415d220 0x7f9394152e60 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 --2- 172.21.15.115:0/2477775784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93940ad2b0 0x7f93941f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:34.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 -- 172.21.15.115:0/2477775784 >> 172.21.15.115:0/2477775784 conn(0x7f93940ce880 msgr2=0x7f939406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:34.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 -- 172.21.15.115:0/2477775784 shutdown_connections 2023-12-01T01:52:34.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:34.786+0000 7f939bf47640 1 -- 172.21.15.115:0/2477775784 wait complete. 2023-12-01T01:52:34.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:35.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.000+0000 7fc907fab640 1 Processor -- start 2023-12-01T01:52:35.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.000+0000 7fc907fab640 1 -- start start 2023-12-01T01:52:35.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc907fab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc900159dd0 0x7fc9000c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc907fab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc90006b6f0 0x7fc9000c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc907fab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc9000ade80 0x7fc9000cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc907fab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90006f8b0 con 0x7fc900159dd0 2023-12-01T01:52:35.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc907fab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9000adcb0 con 0x7fc9000ade80 2023-12-01T01:52:35.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc907fab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9000ad9d0 con 0x7fc90006b6f0 2023-12-01T01:52:35.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc905d47640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc900159dd0 0x7fc9000c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc905d47640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc900159dd0 0x7fc9000c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37732/0 (socket says 172.21.15.115:37732) 2023-12-01T01:52:35.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc905d47640 1 -- 172.21.15.115:0/348986406 learned_addr learned my addr 172.21.15.115:0/348986406 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:35.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc905d47640 1 -- 172.21.15.115:0/348986406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc90006b6f0 msgr2=0x7fc9000c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:35.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc905d47640 1 --2- 172.21.15.115:0/348986406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc90006b6f0 0x7fc9000c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc905d47640 1 -- 172.21.15.115:0/348986406 >> v2:172.21.15.142:3300/0 conn(0x7fc9000ade80 msgr2=0x7fc9000cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:35.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc905d47640 1 --2- 172.21.15.115:0/348986406 >> v2:172.21.15.142:3300/0 conn(0x7fc9000ade80 0x7fc9000cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.002+0000 7fc905d47640 1 -- 172.21.15.115:0/348986406 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9000c9000 con 0x7fc900159dd0 2023-12-01T01:52:35.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.003+0000 7fc905d47640 1 --2- 172.21.15.115:0/348986406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc900159dd0 0x7fc9000c97e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc90006b490 tx=0x7fc8fc02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f343c517ff760c9a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:35.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.004+0000 7fc904d45640 1 -- 172.21.15.115:0/348986406 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc031750 con 0x7fc900159dd0 2023-12-01T01:52:35.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.004+0000 7fc904d45640 1 -- 172.21.15.115:0/348986406 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc0318b0 con 0x7fc900159dd0 2023-12-01T01:52:35.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.004+0000 7fc904d45640 1 -- 172.21.15.115:0/348986406 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc031c40 con 0x7fc900159dd0 2023-12-01T01:52:35.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.004+0000 7fc907fab640 1 -- 172.21.15.115:0/348986406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc900159dd0 msgr2=0x7fc9000c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.004+0000 7fc907fab640 1 --2- 172.21.15.115:0/348986406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc900159dd0 0x7fc9000c97e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc90006b490 tx=0x7fc8fc02b7f0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.005+0000 7fc907fab640 1 -- 172.21.15.115:0/348986406 shutdown_connections 2023-12-01T01:52:35.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.005+0000 7fc907fab640 1 --2- 172.21.15.115:0/348986406 >> v2:172.21.15.142:3300/0 conn(0x7fc9000ade80 0x7fc9000cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.005+0000 7fc907fab640 1 --2- 172.21.15.115:0/348986406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc90006b6f0 0x7fc9000c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.005+0000 7fc907fab640 1 --2- 172.21.15.115:0/348986406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc900159dd0 0x7fc9000c97e0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.005+0000 7fc907fab640 1 -- 172.21.15.115:0/348986406 >> 172.21.15.115:0/348986406 conn(0x7fc9000ce880 msgr2=0x7fc9000cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:35.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.005+0000 7fc907fab640 1 -- 172.21.15.115:0/348986406 shutdown_connections 2023-12-01T01:52:35.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.005+0000 7fc907fab640 1 -- 172.21.15.115:0/348986406 wait complete. 2023-12-01T01:52:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.006+0000 7fc907fab640 1 Processor -- start 2023-12-01T01:52:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.006+0000 7fc907fab640 1 -- start start 2023-12-01T01:52:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.007+0000 7fc907fab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc90006b6f0 0x7fc90014e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.007+0000 7fc907fab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9000ade80 0x7fc9000c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.007+0000 7fc907fab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc900159dd0 0x7fc90006c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.007+0000 7fc907fab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9000c8120 con 0x7fc90006b6f0 2023-12-01T01:52:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.007+0000 7fc907fab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9000c7510 con 0x7fc9000ade80 2023-12-01T01:52:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.007+0000 7fc907fab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9000c77f0 con 0x7fc900159dd0 2023-12-01T01:52:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.008+0000 7fc905d47640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc90006b6f0 0x7fc90014e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.008+0000 7fc905d47640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc90006b6f0 0x7fc90014e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37738/0 (socket says 172.21.15.115:37738) 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.008+0000 7fc905d47640 1 -- 172.21.15.115:0/2606368150 learned_addr learned my addr 172.21.15.115:0/2606368150 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.008+0000 7fc905d47640 1 -- 172.21.15.115:0/2606368150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc900159dd0 msgr2=0x7fc90006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.008+0000 7fc905d47640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc900159dd0 0x7fc90006c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.008+0000 7fc905d47640 1 -- 172.21.15.115:0/2606368150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9000ade80 msgr2=0x7fc9000c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.008+0000 7fc905d47640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9000ade80 0x7fc9000c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.008+0000 7fc905d47640 1 -- 172.21.15.115:0/2606368150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc0094d0 con 0x7fc90006b6f0 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.008+0000 7fc905d47640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc90006b6f0 0x7fc90014e8d0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000c00 tx=0x7fc8fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.009+0000 7fc8f67fc640 1 -- 172.21.15.115:0/2606368150 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc033c80 con 0x7fc90006b6f0 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.009+0000 7fc8f67fc640 1 -- 172.21.15.115:0/2606368150 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc033de0 con 0x7fc90006b6f0 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.009+0000 7fc8f67fc640 1 -- 172.21.15.115:0/2606368150 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc03d6b0 con 0x7fc90006b6f0 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.009+0000 7fc907fab640 1 -- 172.21.15.115:0/2606368150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9000c7ad0 con 0x7fc90006b6f0 2023-12-01T01:52:35.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.009+0000 7fc907fab640 1 -- 172.21.15.115:0/2606368150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9000c6e60 con 0x7fc90006b6f0 2023-12-01T01:52:35.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.010+0000 7fc907fab640 1 -- 172.21.15.115:0/2606368150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9000c5d60 con 0x7fc90006b6f0 2023-12-01T01:52:35.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.011+0000 7fc8f67fc640 1 -- 172.21.15.115:0/2606368150 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 48) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fc8fc03a040 con 0x7fc90006b6f0 2023-12-01T01:52:35.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.012+0000 7fc8f67fc640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc8e406a4c0 0x7fc8e406c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.012+0000 7fc8f67fc640 1 -- 172.21.15.115:0/2606368150 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fc8fc0aac70 con 0x7fc90006b6f0 2023-12-01T01:52:35.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.012+0000 7fc905546640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc8e406a4c0 0x7fc8e406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.013+0000 7fc905546640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc8e406a4c0 0x7fc8e406c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc9000c24d0 tx=0x7fc8f0006c90 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:35.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.016+0000 7fc8f67fc640 1 -- 172.21.15.115:0/2606368150 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc8fc0760a0 con 0x7fc90006b6f0 2023-12-01T01:52:35.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.207+0000 7fc907fab640 1 -- 172.21.15.115:0/2606368150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9000738e0 con 0x7fc90006b6f0 2023-12-01T01:52:35.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.210+0000 7fc8f67fc640 1 -- 172.21.15.115:0/2606368150 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16963 (secure 0 0 0) 0x7fc8fc038090 con 0x7fc90006b6f0 2023-12-01T01:52:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:35.213 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":99,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:32.074823+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:04.909384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:05.480768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/1457241066":"2023-12-02T01:52:32.074792+0000","172.21.15.115:0/2170465963":"2023-12-02T01:52:32.074792+0000","172.21.15.115:0/1482402703":"2023-12-02T01:52:32.074792+0000","172.21.15.115:0/202387737":"2023-12-02T01:52:32.074792+0000","172.21.15.115:6822/88317":"2023-12-02T01:52:25.734961+0000","172.21.15.115:6824/88317":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/3373285076":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/3569012860":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/709337731":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/1296968550":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/3605273682":"2023-12-02T01:52:32.074792+0000","172.21.15.115:0/1490819667":"2023-12-02T01:52:25.734961+0000","172.21.15.115:0/1208012610":"2023-12-02T01:52:32.074792+0000","172.21.15.115:0/1878415381":"2023-12-02T01:52:25.734961+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.213+0000 7fc907fab640 1 -- 172.21.15.115:0/2606368150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc8e406a4c0 msgr2=0x7fc8e406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.213+0000 7fc907fab640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc8e406a4c0 0x7fc8e406c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc9000c24d0 tx=0x7fc8f0006c90 comp rx=0 tx=0).stop 2023-12-01T01:52:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.213+0000 7fc907fab640 1 -- 172.21.15.115:0/2606368150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc90006b6f0 msgr2=0x7fc90014e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.213+0000 7fc907fab640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc90006b6f0 0x7fc90014e8d0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000c00 tx=0x7fc8fc0046c0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.213+0000 7fc907fab640 1 -- 172.21.15.115:0/2606368150 shutdown_connections 2023-12-01T01:52:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.213+0000 7fc907fab640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc8e406a4c0 0x7fc8e406c980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.213+0000 7fc907fab640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc900159dd0 0x7fc90006c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.213+0000 7fc907fab640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9000ade80 0x7fc9000c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.213+0000 7fc907fab640 1 --2- 172.21.15.115:0/2606368150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc90006b6f0 0x7fc90014e8d0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.213+0000 7fc907fab640 1 -- 172.21.15.115:0/2606368150 >> 172.21.15.115:0/2606368150 conn(0x7fc9000ce880 msgr2=0x7fc90006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.214+0000 7fc907fab640 1 -- 172.21.15.115:0/2606368150 shutdown_connections 2023-12-01T01:52:35.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.214+0000 7fc907fab640 1 -- 172.21.15.115:0/2606368150 wait complete. 2023-12-01T01:52:35.233 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T01:52:35.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.435+0000 7f0fdc5cb640 1 Processor -- start 2023-12-01T01:52:35.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.435+0000 7f0fdc5cb640 1 -- start start 2023-12-01T01:52:35.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.436+0000 7f0fdc5cb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0fd40d15b0 0x7f0fd40cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.436+0000 7f0fdc5cb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0fd40cf5b0 0x7f0fd40c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.436+0000 7f0fdc5cb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd40cc110 con 0x7f0fd415ada0 2023-12-01T01:52:35.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.436+0000 7f0fdc5cb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd40cc9b0 con 0x7f0fd40d15b0 2023-12-01T01:52:35.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.436+0000 7f0fdc5cb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd40cc3f0 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.437+0000 7f0fd9b66640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0fd40d15b0 0x7f0fd40cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.437+0000 7f0fd9b66640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0fd40d15b0 0x7f0fd40cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37748/0 (socket says 172.21.15.115:37748) 2023-12-01T01:52:35.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.437+0000 7f0fd9b66640 1 -- 172.21.15.115:0/369736063 learned_addr learned my addr 172.21.15.115:0/369736063 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:35.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.437+0000 7f0fdab68640 1 --2- 172.21.15.115:0/369736063 >> v2:172.21.15.142:3300/0 conn(0x7f0fd40cf5b0 0x7f0fd40c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.437+0000 7f0fd9b66640 1 -- 172.21.15.115:0/369736063 >> v2:172.21.15.142:3300/0 conn(0x7f0fd40cf5b0 msgr2=0x7f0fd40c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:35.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.437+0000 7f0fd9b66640 1 --2- 172.21.15.115:0/369736063 >> v2:172.21.15.142:3300/0 conn(0x7f0fd40cf5b0 0x7f0fd40c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.437+0000 7f0fd9b66640 1 -- 172.21.15.115:0/369736063 >> v1:172.21.15.142:6789/0 conn(0x7f0fd415ada0 legacy=0x7f0fd40ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:35.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.437+0000 7f0fd9b66640 1 -- 172.21.15.115:0/369736063 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd40cc6d0 con 0x7f0fd40d15b0 2023-12-01T01:52:35.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.438+0000 7f0fd9b66640 1 --2- 172.21.15.115:0/369736063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40d15b0 0x7f0fd40cf070 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4071980 tx=0x7f0fc402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ccd6fe493b6ea4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:35.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fd9365640 1 -- 172.21.15.115:0/369736063 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4031750 con 0x7f0fd40d15b0 2023-12-01T01:52:35.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fd9365640 1 -- 172.21.15.115:0/369736063 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc40318b0 con 0x7f0fd40d15b0 2023-12-01T01:52:35.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fd9365640 1 -- 172.21.15.115:0/369736063 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4031c40 con 0x7f0fd40d15b0 2023-12-01T01:52:35.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/369736063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40d15b0 msgr2=0x7f0fd40cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fdc5cb640 1 --2- 172.21.15.115:0/369736063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40d15b0 0x7f0fd40cf070 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4071980 tx=0x7f0fc402b7f0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/369736063 shutdown_connections 2023-12-01T01:52:35.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fdc5cb640 1 --2- 172.21.15.115:0/369736063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40d15b0 0x7f0fd40cf070 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fdc5cb640 1 --2- 172.21.15.115:0/369736063 >> v2:172.21.15.142:3300/0 conn(0x7f0fd40cf5b0 0x7f0fd40c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/369736063 >> 172.21.15.115:0/369736063 conn(0x7f0fd40d0a90 msgr2=0x7f0fd40d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:35.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/369736063 shutdown_connections 2023-12-01T01:52:35.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.439+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/369736063 wait complete. 2023-12-01T01:52:35.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.441+0000 7f0fdc5cb640 1 Processor -- start 2023-12-01T01:52:35.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.441+0000 7f0fdc5cb640 1 -- start start 2023-12-01T01:52:35.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.442+0000 7f0fdc5cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40cf5b0 0x7f0fd409dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.442+0000 7f0fdc5cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd40d15b0 0x7f0fd40b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.442+0000 7f0fdc5cb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd415ada0 0x7f0fd40c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.442+0000 7f0fdc5cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd409d230 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.442+0000 7f0fdc5cb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd409d510 con 0x7f0fd415ada0 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.442+0000 7f0fdc5cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd409d7f0 con 0x7f0fd40d15b0 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.442+0000 7f0fda367640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40cf5b0 0x7f0fd409dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.442+0000 7f0fda367640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40cf5b0 0x7f0fd409dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37756/0 (socket says 172.21.15.115:37756) 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.442+0000 7f0fda367640 1 -- 172.21.15.115:0/2598004690 learned_addr learned my addr 172.21.15.115:0/2598004690 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.443+0000 7f0fda367640 1 -- 172.21.15.115:0/2598004690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd40d15b0 msgr2=0x7f0fd40b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.443+0000 7f0fda367640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd40d15b0 0x7f0fd40b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.443+0000 7f0fda367640 1 -- 172.21.15.115:0/2598004690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd415ada0 msgr2=0x7f0fd40c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.443+0000 7f0fda367640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd415ada0 0x7f0fd40c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.443+0000 7f0fda367640 1 -- 172.21.15.115:0/2598004690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40094d0 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.443+0000 7f0fda367640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40cf5b0 0x7f0fd409dff0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b3f0 tx=0x7f0fd000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.443+0000 7f0fcaffd640 1 -- 172.21.15.115:0/2598004690 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0011830 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.443+0000 7f0fcaffd640 1 -- 172.21.15.115:0/2598004690 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0011990 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.444+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/2598004690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd409dad0 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.444+0000 7f0fcaffd640 1 -- 172.21.15.115:0/2598004690 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0011d40 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.444+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/2598004690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd40b63d0 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.445+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/2598004690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd40c79b0 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.446+0000 7f0fcaffd640 1 -- 172.21.15.115:0/2598004690 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 48) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f0fd001a400 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.446+0000 7f0fcaffd640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fb806a4c0 0x7f0fb806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.447+0000 7f0fcaffd640 1 -- 172.21.15.115:0/2598004690 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f0fd0088a70 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.447+0000 7f0fd9b66640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fb806a4c0 0x7f0fb806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.447+0000 7f0fd9b66640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fb806a4c0 0x7f0fb806c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4035000 tx=0x7f0fc4030000 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:35.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.451+0000 7f0fcaffd640 1 -- 172.21.15.115:0/2598004690 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0fd0053da0 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.681+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/2598004690 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0fd409d370 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.683 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T01:52:35.682+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:52:35.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.704+0000 7f0fcaffd640 1 -- 172.21.15.115:0/2598004690 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0fd409d370 con 0x7f0fd40cf5b0 2023-12-01T01:52:35.706 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/2598004690 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fb806a4c0 msgr2=0x7f0fb806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fb806a4c0 0x7f0fb806c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4035000 tx=0x7f0fc4030000 comp rx=0 tx=0).stop 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/2598004690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40cf5b0 msgr2=0x7f0fd409dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40cf5b0 0x7f0fd409dff0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b3f0 tx=0x7f0fd000b8b0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/2598004690 shutdown_connections 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0fb806a4c0 0x7f0fb806c980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd40d15b0 0x7f0fd40b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd415ada0 0x7f0fd40c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 --2- 172.21.15.115:0/2598004690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0fd40cf5b0 0x7f0fd409dff0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/2598004690 >> 172.21.15.115:0/2598004690 conn(0x7f0fd40d0a90 msgr2=0x7f0fd40c9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/2598004690 shutdown_connections 2023-12-01T01:52:35.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.707+0000 7f0fdc5cb640 1 -- 172.21.15.115:0/2598004690 wait complete. 2023-12-01T01:52:35.724 INFO:tasks.ceph.mds.b.smithi142.stderr: -2727> 2023-12-01T01:52:11.849+0000 7f9f153c6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:52:35.727 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T01:52:35.812 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T01:52:35.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.946+0000 7f722326c640 1 Processor -- start 2023-12-01T01:52:35.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.946+0000 7f722326c640 1 -- start start 2023-12-01T01:52:35.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.947+0000 7f722326c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f721c0b04c0 0x7f721c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.947+0000 7f722326c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f721c0cb630 0x7f721c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.947+0000 7f722326c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c0cbaa0 con 0x7f721c15ac00 2023-12-01T01:52:35.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.947+0000 7f722326c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c0b12b0 con 0x7f721c0b04c0 2023-12-01T01:52:35.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.947+0000 7f722326c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c15afe0 con 0x7f721c0cb630 2023-12-01T01:52:35.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.948+0000 7f7221a69640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f721c0b04c0 0x7f721c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.948+0000 7f7221a69640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f721c0b04c0 0x7f721c0cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37768/0 (socket says 172.21.15.115:37768) 2023-12-01T01:52:35.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.948+0000 7f7221a69640 1 -- 172.21.15.115:0/830906897 learned_addr learned my addr 172.21.15.115:0/830906897 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:35.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.948+0000 7f7222a6b640 1 --2- 172.21.15.115:0/830906897 >> v2:172.21.15.142:3300/0 conn(0x7f721c0cb630 0x7f721c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.948+0000 7f7221a69640 1 -- 172.21.15.115:0/830906897 >> v2:172.21.15.142:3300/0 conn(0x7f721c0cb630 msgr2=0x7f721c06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:35.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.948+0000 7f7221a69640 1 --2- 172.21.15.115:0/830906897 >> v2:172.21.15.142:3300/0 conn(0x7f721c0cb630 0x7f721c06fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.948+0000 7f7221a69640 1 -- 172.21.15.115:0/830906897 >> v1:172.21.15.142:6789/0 conn(0x7f721c15ac00 legacy=0x7f721c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:35.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.948+0000 7f7221a69640 1 -- 172.21.15.115:0/830906897 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c0b0a10 con 0x7f721c0b04c0 2023-12-01T01:52:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.949+0000 7f7221a69640 1 --2- 172.21.15.115:0/830906897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0b04c0 0x7f721c0cb0f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f721c06c2a0 tx=0x7f720c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f63f9fcfbd4850c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.950+0000 7f7221268640 1 -- 172.21.15.115:0/830906897 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c0317e0 con 0x7f721c0b04c0 2023-12-01T01:52:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.950+0000 7f7221268640 1 -- 172.21.15.115:0/830906897 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c031940 con 0x7f721c0b04c0 2023-12-01T01:52:35.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.950+0000 7f7221268640 1 -- 172.21.15.115:0/830906897 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c031cd0 con 0x7f721c0b04c0 2023-12-01T01:52:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.950+0000 7f722326c640 1 -- 172.21.15.115:0/830906897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0b04c0 msgr2=0x7f721c0cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.950+0000 7f722326c640 1 --2- 172.21.15.115:0/830906897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0b04c0 0x7f721c0cb0f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f721c06c2a0 tx=0x7f720c02b880 comp rx=0 tx=0).stop 2023-12-01T01:52:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.950+0000 7f722326c640 1 -- 172.21.15.115:0/830906897 shutdown_connections 2023-12-01T01:52:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.950+0000 7f722326c640 1 --2- 172.21.15.115:0/830906897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0b04c0 0x7f721c0cb0f0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.950+0000 7f722326c640 1 --2- 172.21.15.115:0/830906897 >> v2:172.21.15.142:3300/0 conn(0x7f721c0cb630 0x7f721c06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.950+0000 7f722326c640 1 -- 172.21.15.115:0/830906897 >> 172.21.15.115:0/830906897 conn(0x7f721c065960 msgr2=0x7f721c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.951+0000 7f722326c640 1 -- 172.21.15.115:0/830906897 shutdown_connections 2023-12-01T01:52:35.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.951+0000 7f722326c640 1 -- 172.21.15.115:0/830906897 wait complete. 2023-12-01T01:52:35.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.952+0000 7f722326c640 1 Processor -- start 2023-12-01T01:52:35.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.952+0000 7f722326c640 1 -- start start 2023-12-01T01:52:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.953+0000 7f722326c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c0b04c0 0x7f721c1498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.953+0000 7f722326c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0cb630 0x7f721c0c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.953+0000 7f722326c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f721c15ac00 0x7f721c147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.953+0000 7f722326c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c15d190 con 0x7f721c0cb630 2023-12-01T01:52:35.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.953+0000 7f722326c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c14a110 con 0x7f721c0b04c0 2023-12-01T01:52:35.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.953+0000 7f722326c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c14a3f0 con 0x7f721c15ac00 2023-12-01T01:52:35.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f7221a69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0cb630 0x7f721c0c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f7221a69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0cb630 0x7f721c0c4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37772/0 (socket says 172.21.15.115:37772) 2023-12-01T01:52:35.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f7221a69640 1 -- 172.21.15.115:0/237403560 learned_addr learned my addr 172.21.15.115:0/237403560 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:35.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f7221a69640 1 -- 172.21.15.115:0/237403560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f721c15ac00 msgr2=0x7f721c147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f7221a69640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f721c15ac00 0x7f721c147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f7221a69640 1 -- 172.21.15.115:0/237403560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c0b04c0 msgr2=0x7f721c1498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:35.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f722226a640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c0b04c0 0x7f721c1498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f7221a69640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c0b04c0 0x7f721c1498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:35.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f7221a69640 1 -- 172.21.15.115:0/237403560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c0094d0 con 0x7f721c0cb630 2023-12-01T01:52:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f722226a640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c0b04c0 0x7f721c1498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.954+0000 7f7221a69640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0cb630 0x7f721c0c4aa0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f720c02b8c0 tx=0x7f720c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.955+0000 7f7212ffd640 1 -- 172.21.15.115:0/237403560 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c032880 con 0x7f721c0cb630 2023-12-01T01:52:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.955+0000 7f7212ffd640 1 -- 172.21.15.115:0/237403560 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c0329e0 con 0x7f721c0cb630 2023-12-01T01:52:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.955+0000 7f722326c640 1 -- 172.21.15.115:0/237403560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c14a6d0 con 0x7f721c0cb630 2023-12-01T01:52:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.955+0000 7f7212ffd640 1 -- 172.21.15.115:0/237403560 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c032d90 con 0x7f721c0cb630 2023-12-01T01:52:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.955+0000 7f722326c640 1 -- 172.21.15.115:0/237403560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c0c4fe0 con 0x7f721c0cb630 2023-12-01T01:52:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.956+0000 7f722326c640 1 -- 172.21.15.115:0/237403560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c15d300 con 0x7f721c0cb630 2023-12-01T01:52:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.957+0000 7f7212ffd640 1 -- 172.21.15.115:0/237403560 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 48) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f720c04e450 con 0x7f721c0cb630 2023-12-01T01:52:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.958+0000 7f7212ffd640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f720006a4f0 0x7f720006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.958+0000 7f7212ffd640 1 -- 172.21.15.115:0/237403560 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f720c0aad40 con 0x7f721c0cb630 2023-12-01T01:52:35.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.958+0000 7f722226a640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f720006a4f0 0x7f720006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:35.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.959+0000 7f722226a640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f720006a4f0 0x7f720006c9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f721c1578b0 tx=0x7f7218008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:35.962+0000 7f7212ffd640 1 -- 172.21.15.115:0/237403560 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f720c0760c0 con 0x7f721c0cb630 2023-12-01T01:52:36.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.190+0000 7f722326c640 1 -- 172.21.15.115:0/237403560 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f721c0618d0 con 0x7f721c0cb630 2023-12-01T01:52:36.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.821+0000 7f7212ffd640 1 -- 172.21.15.115:0/237403560 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7f720c0796e0 con 0x7f721c0cb630 2023-12-01T01:52:36.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 -- 172.21.15.115:0/237403560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f720006a4f0 msgr2=0x7f720006c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f720006a4f0 0x7f720006c9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f721c1578b0 tx=0x7f7218008040 comp rx=0 tx=0).stop 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 -- 172.21.15.115:0/237403560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0cb630 msgr2=0x7f721c0c4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0cb630 0x7f721c0c4aa0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f720c02b8c0 tx=0x7f720c0046c0 comp rx=0 tx=0).stop 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 -- 172.21.15.115:0/237403560 shutdown_connections 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f720006a4f0 0x7f720006c9b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f721c15ac00 0x7f721c147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f721c0cb630 0x7f721c0c4aa0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 --2- 172.21.15.115:0/237403560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f721c0b04c0 0x7f721c1498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 -- 172.21.15.115:0/237403560 >> 172.21.15.115:0/237403560 conn(0x7f721c065960 msgr2=0x7f721c0709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 -- 172.21.15.115:0/237403560 shutdown_connections 2023-12-01T01:52:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:36.825+0000 7f722326c640 1 -- 172.21.15.115:0/237403560 wait complete. 2023-12-01T01:52:36.855 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T01:52:37.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.064+0000 7f8b827f0640 1 Processor -- start 2023-12-01T01:52:37.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.064+0000 7f8b827f0640 1 -- start start 2023-12-01T01:52:37.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.065+0000 7f8b827f0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8b7c0ca7c0 0x7f8b7c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:37.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.066+0000 7f8b827f0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c0c9440 con 0x7f8b7c159bf0 2023-12-01T01:52:37.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.066+0000 7f8b827f0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c0c9720 con 0x7f8b7c0cb6f0 2023-12-01T01:52:37.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.066+0000 7f8b827f0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c159fd0 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.066+0000 7f8b7b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8b7c0ca7c0 0x7f8b7c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:37.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.066+0000 7f8b7bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8b7c159bf0 0x7f8b7c15e030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51430/0 (socket says 172.21.15.115:51430) 2023-12-01T01:52:37.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.066+0000 7f8b7bfff640 1 -- 172.21.15.115:0/3763812181 learned_addr learned my addr 172.21.15.115:0/3763812181 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:37.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.067+0000 7f8b7b7fe640 1 -- 172.21.15.115:0/3763812181 >> v1:172.21.15.142:6789/0 conn(0x7f8b7c0cb6f0 legacy=0x7f8b7c06f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:37.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.067+0000 7f8b7b7fe640 1 -- 172.21.15.115:0/3763812181 >> v1:172.21.15.115:6789/0 conn(0x7f8b7c159bf0 legacy=0x7f8b7c15e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:37.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.067+0000 7f8b7affd640 1 -- 172.21.15.115:0/3763812181 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479179784 0 0) 0x7f8b7c0c9440 con 0x7f8b7c159bf0 2023-12-01T01:52:37.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.067+0000 7f8b7b7fe640 1 -- 172.21.15.115:0/3763812181 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0c9a00 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.068+0000 7f8b7b7fe640 1 --2- 172.21.15.115:0/3763812181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 0x7f8b7c0cb1b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8b640097a0 tx=0x7f8b6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c6ef3c61f089c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:37.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.068+0000 7f8b7affd640 1 -- 172.21.15.115:0/3763812181 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64031830 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.068+0000 7f8b7affd640 1 -- 172.21.15.115:0/3763812181 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b64031990 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.068+0000 7f8b7affd640 1 -- 172.21.15.115:0/3763812181 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64031d20 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.069+0000 7f8b827f0640 1 -- 172.21.15.115:0/3763812181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 msgr2=0x7f8b7c0cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:37.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.069+0000 7f8b827f0640 1 --2- 172.21.15.115:0/3763812181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 0x7f8b7c0cb1b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8b640097a0 tx=0x7f8b6402b8d0 comp rx=0 tx=0).stop 2023-12-01T01:52:37.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.069+0000 7f8b827f0640 1 -- 172.21.15.115:0/3763812181 shutdown_connections 2023-12-01T01:52:37.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.069+0000 7f8b827f0640 1 --2- 172.21.15.115:0/3763812181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 0x7f8b7c0cb1b0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:37.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.069+0000 7f8b827f0640 1 -- 172.21.15.115:0/3763812181 >> 172.21.15.115:0/3763812181 conn(0x7f8b7c0662c0 msgr2=0x7f8b7c0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:37.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.069+0000 7f8b827f0640 1 -- 172.21.15.115:0/3763812181 shutdown_connections 2023-12-01T01:52:37.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.069+0000 7f8b827f0640 1 -- 172.21.15.115:0/3763812181 wait complete. 2023-12-01T01:52:37.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.071+0000 7f8b827f0640 1 Processor -- start 2023-12-01T01:52:37.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.071+0000 7f8b827f0640 1 -- start start 2023-12-01T01:52:37.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b827f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 0x7f8b7c14f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:37.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b827f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b7c0cb6f0 0x7f8b7c14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:37.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b827f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b7c159bf0 0x7f8b7c0c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:37.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b827f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1f5b90 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b827f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c15d080 con 0x7f8b7c159bf0 2023-12-01T01:52:37.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b827f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c15af80 con 0x7f8b7c0cb6f0 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b7bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 0x7f8b7c14f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b7bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 0x7f8b7c14f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37796/0 (socket says 172.21.15.115:37796) 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b7bfff640 1 -- 172.21.15.115:0/2760301686 learned_addr learned my addr 172.21.15.115:0/2760301686 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b7bfff640 1 -- 172.21.15.115:0/2760301686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b7c0cb6f0 msgr2=0x7f8b7c14fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b7bfff640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b7c0cb6f0 0x7f8b7c14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b7bfff640 1 -- 172.21.15.115:0/2760301686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b7c159bf0 msgr2=0x7f8b7c0c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b7bfff640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b7c159bf0 0x7f8b7c0c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.072+0000 7f8b7bfff640 1 -- 172.21.15.115:0/2760301686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b64009400 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.073+0000 7f8b7bfff640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 0x7f8b7c14f5d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00c430 tx=0x7f8b6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.073+0000 7f8b78ff9640 1 -- 172.21.15.115:0/2760301686 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0074e0 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.073+0000 7f8b78ff9640 1 -- 172.21.15.115:0/2760301686 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c007660 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.073+0000 7f8b78ff9640 1 -- 172.21.15.115:0/2760301686 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c004a20 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.073+0000 7f8b827f0640 1 -- 172.21.15.115:0/2760301686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c151450 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.073+0000 7f8b827f0640 1 -- 172.21.15.115:0/2760301686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c15a400 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.074+0000 7f8b827f0640 1 -- 172.21.15.115:0/2760301686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c1f5cd0 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.075+0000 7f8b78ff9640 1 -- 172.21.15.115:0/2760301686 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8b6c004b80 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.076+0000 7f8b78ff9640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b5c06a550 0x7f8b5c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:37.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.076+0000 7f8b78ff9640 1 -- 172.21.15.115:0/2760301686 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f8b6c088bf0 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.076+0000 7f8b7b7fe640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b5c06a550 0x7f8b5c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:37.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.077+0000 7f8b7b7fe640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b5c06a550 0x7f8b5c06ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8b64036000 tx=0x7f8b64030000 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:37.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.081+0000 7f8b78ff9640 1 -- 172.21.15.115:0/2760301686 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8b7c1f5cd0 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.271+0000 7f8b827f0640 1 -- 172.21.15.115:0/2760301686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b7c1f5cd0 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.837+0000 7f8b78ff9640 1 -- 172.21.15.115:0/2760301686 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8b7c1f5cd0 con 0x7f8b7c0ca7c0 2023-12-01T01:52:37.839 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T01:52:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.842+0000 7f8b827f0640 1 -- 172.21.15.115:0/2760301686 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b5c06a550 msgr2=0x7f8b5c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.842+0000 7f8b827f0640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b5c06a550 0x7f8b5c06ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8b64036000 tx=0x7f8b64030000 comp rx=0 tx=0).stop 2023-12-01T01:52:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.842+0000 7f8b827f0640 1 -- 172.21.15.115:0/2760301686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 msgr2=0x7f8b7c14f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:37.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.842+0000 7f8b827f0640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 0x7f8b7c14f5d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00c430 tx=0x7f8b6c00c8f0 comp rx=0 tx=0).stop 2023-12-01T01:52:37.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.842+0000 7f8b827f0640 1 -- 172.21.15.115:0/2760301686 shutdown_connections 2023-12-01T01:52:37.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.842+0000 7f8b827f0640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8b5c06a550 0x7f8b5c06ca10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:37.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.842+0000 7f8b827f0640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b7c0cb6f0 0x7f8b7c14fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:37.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.842+0000 7f8b827f0640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b7c159bf0 0x7f8b7c0c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:37.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.842+0000 7f8b827f0640 1 --2- 172.21.15.115:0/2760301686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b7c0ca7c0 0x7f8b7c14f5d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:37.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.842+0000 7f8b827f0640 1 -- 172.21.15.115:0/2760301686 >> 172.21.15.115:0/2760301686 conn(0x7f8b7c0662c0 msgr2=0x7f8b7c0ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:37.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.843+0000 7f8b827f0640 1 -- 172.21.15.115:0/2760301686 shutdown_connections 2023-12-01T01:52:37.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:37.843+0000 7f8b827f0640 1 -- 172.21.15.115:0/2760301686 wait complete. 2023-12-01T01:52:37.873 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T01:52:38.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.063+0000 7f1a7cbae640 1 Processor -- start 2023-12-01T01:52:38.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.064+0000 7f1a7cbae640 1 -- start start 2023-12-01T01:52:38.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.065+0000 7f1a7cbae640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1a780cb8c0 0x7f1a780cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:38.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.065+0000 7f1a7cbae640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7806c860 con 0x7f1a7815ac70 2023-12-01T01:52:38.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.065+0000 7f1a7cbae640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7815b050 con 0x7f1a780b4070 2023-12-01T01:52:38.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.065+0000 7f1a7cbae640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a780ce0b0 con 0x7f1a780cb8c0 2023-12-01T01:52:38.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.066+0000 7f1a7659c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1a7815ac70 0x7f1a780cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40404/0 (socket says 172.21.15.115:40404) 2023-12-01T01:52:38.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.066+0000 7f1a7659c640 1 -- 172.21.15.115:0/3562532604 learned_addr learned my addr 172.21.15.115:0/3562532604 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:38.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.066+0000 7f1a75d9b640 1 --2- 172.21.15.115:0/3562532604 >> v2:172.21.15.115:3300/0 conn(0x7f1a780cb8c0 0x7f1a780cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:38.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.066+0000 7f1a7559a640 1 -- 172.21.15.115:0/3562532604 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317590096 0 0) 0x7f1a7806c860 con 0x7f1a7815ac70 2023-12-01T01:52:38.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.066+0000 7f1a7559a640 1 -- 172.21.15.115:0/3562532604 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a64003520 con 0x7f1a7815ac70 2023-12-01T01:52:38.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.066+0000 7f1a75d9b640 1 -- 172.21.15.115:0/3562532604 >> v1:172.21.15.142:6789/0 conn(0x7f1a780b4070 legacy=0x7f1a780cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:38.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.066+0000 7f1a75d9b640 1 -- 172.21.15.115:0/3562532604 >> v1:172.21.15.115:6789/0 conn(0x7f1a7815ac70 legacy=0x7f1a780cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:38.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.066+0000 7f1a75d9b640 1 -- 172.21.15.115:0/3562532604 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a780ce390 con 0x7f1a780cb8c0 2023-12-01T01:52:38.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.068+0000 7f1a75d9b640 1 --2- 172.21.15.115:0/3562532604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780cb8c0 0x7f1a780cc370 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1a600097a0 tx=0x7f1a6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0fa2befad03d81 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:38.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.068+0000 7f1a7559a640 1 -- 172.21.15.115:0/3562532604 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60031830 con 0x7f1a780cb8c0 2023-12-01T01:52:38.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.068+0000 7f1a7559a640 1 -- 172.21.15.115:0/3562532604 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60031990 con 0x7f1a780cb8c0 2023-12-01T01:52:38.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.068+0000 7f1a7559a640 1 -- 172.21.15.115:0/3562532604 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60031d20 con 0x7f1a780cb8c0 2023-12-01T01:52:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.068+0000 7f1a7cbae640 1 -- 172.21.15.115:0/3562532604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780cb8c0 msgr2=0x7f1a780cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.068+0000 7f1a7cbae640 1 --2- 172.21.15.115:0/3562532604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780cb8c0 0x7f1a780cc370 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f1a600097a0 tx=0x7f1a6002b8d0 comp rx=0 tx=0).stop 2023-12-01T01:52:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.069+0000 7f1a7cbae640 1 -- 172.21.15.115:0/3562532604 shutdown_connections 2023-12-01T01:52:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.069+0000 7f1a7cbae640 1 --2- 172.21.15.115:0/3562532604 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780cb8c0 0x7f1a780cc370 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:38.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.069+0000 7f1a7cbae640 1 -- 172.21.15.115:0/3562532604 >> 172.21.15.115:0/3562532604 conn(0x7f1a780662c0 msgr2=0x7f1a780b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:38.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.069+0000 7f1a7cbae640 1 -- 172.21.15.115:0/3562532604 shutdown_connections 2023-12-01T01:52:38.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.069+0000 7f1a7cbae640 1 -- 172.21.15.115:0/3562532604 wait complete. 2023-12-01T01:52:38.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.070+0000 7f1a7cbae640 1 Processor -- start 2023-12-01T01:52:38.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.071+0000 7f1a7cbae640 1 -- start start 2023-12-01T01:52:38.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.071+0000 7f1a7cbae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780b4070 0x7f1a78159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:38.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.071+0000 7f1a7cbae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a780cb8c0 0x7f1a780c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:38.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.071+0000 7f1a7cbae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a7815ac70 0x7f1a7814f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:38.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.071+0000 7f1a7cbae640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a780b1be0 con 0x7f1a780b4070 2023-12-01T01:52:38.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.071+0000 7f1a7cbae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a780b1ec0 con 0x7f1a780cb8c0 2023-12-01T01:52:38.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.071+0000 7f1a7cbae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a780b21a0 con 0x7f1a7815ac70 2023-12-01T01:52:38.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.072+0000 7f1a7659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780b4070 0x7f1a78159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.072+0000 7f1a7659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780b4070 0x7f1a78159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46952/0 (socket says 172.21.15.115:46952) 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.072+0000 7f1a7659c640 1 -- 172.21.15.115:0/66331791 learned_addr learned my addr 172.21.15.115:0/66331791 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.072+0000 7f1a7659c640 1 -- 172.21.15.115:0/66331791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a7815ac70 msgr2=0x7f1a7814f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.072+0000 7f1a7659c640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a7815ac70 0x7f1a7814f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.072+0000 7f1a7659c640 1 -- 172.21.15.115:0/66331791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a780cb8c0 msgr2=0x7f1a780c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.072+0000 7f1a7659c640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a780cb8c0 0x7f1a780c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.072+0000 7f1a7659c640 1 -- 172.21.15.115:0/66331791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60009400 con 0x7f1a780b4070 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.072+0000 7f1a7659c640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780b4070 0x7f1a78159120 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800cbc0 tx=0x7f1a6800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.073+0000 7f1a6effd640 1 -- 172.21.15.115:0/66331791 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6800ad40 con 0x7f1a780b4070 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.073+0000 7f1a6effd640 1 -- 172.21.15.115:0/66331791 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a68012400 con 0x7f1a780b4070 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.073+0000 7f1a6effd640 1 -- 172.21.15.115:0/66331791 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6800aea0 con 0x7f1a780b4070 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.073+0000 7f1a7cbae640 1 -- 172.21.15.115:0/66331791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a780b2480 con 0x7f1a780b4070 2023-12-01T01:52:38.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.073+0000 7f1a7cbae640 1 -- 172.21.15.115:0/66331791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a780b1840 con 0x7f1a780b4070 2023-12-01T01:52:38.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.074+0000 7f1a7cbae640 1 -- 172.21.15.115:0/66331791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a780c5850 con 0x7f1a780b4070 2023-12-01T01:52:38.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.075+0000 7f1a6effd640 1 -- 172.21.15.115:0/66331791 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1a68007070 con 0x7f1a780b4070 2023-12-01T01:52:38.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.076+0000 7f1a6effd640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1a5806a550 0x7f1a5806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:38.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.076+0000 7f1a6effd640 1 -- 172.21.15.115:0/66331791 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5959+0+0 (secure 0 0 0) 0x7f1a68088bd0 con 0x7f1a780b4070 2023-12-01T01:52:38.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.076+0000 7f1a75d9b640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1a5806a550 0x7f1a5806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:38.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.077+0000 7f1a75d9b640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1a5806a550 0x7f1a5806ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1a60036000 tx=0x7f1a60030000 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:38.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.080+0000 7f1a6effd640 1 -- 172.21.15.115:0/66331791 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1a68057a50 con 0x7f1a780b4070 2023-12-01T01:52:38.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.268+0000 7f1a7cbae640 1 -- 172.21.15.115:0/66331791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1a78074210 con 0x7f1a780b4070 2023-12-01T01:52:38.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.859+0000 7f1a6effd640 1 -- 172.21.15.115:0/66331791 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1a68017020 con 0x7f1a780b4070 2023-12-01T01:52:38.862 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T01:52:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 -- 172.21.15.115:0/66331791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1a5806a550 msgr2=0x7f1a5806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:38.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1a5806a550 0x7f1a5806ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1a60036000 tx=0x7f1a60030000 comp rx=0 tx=0).stop 2023-12-01T01:52:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 -- 172.21.15.115:0/66331791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780b4070 msgr2=0x7f1a78159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780b4070 0x7f1a78159120 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800cbc0 tx=0x7f1a6800a430 comp rx=0 tx=0).stop 2023-12-01T01:52:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 -- 172.21.15.115:0/66331791 shutdown_connections 2023-12-01T01:52:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1a5806a550 0x7f1a5806ca10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a7815ac70 0x7f1a7814f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a780cb8c0 0x7f1a780c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 --2- 172.21.15.115:0/66331791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a780b4070 0x7f1a78159120 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 -- 172.21.15.115:0/66331791 >> 172.21.15.115:0/66331791 conn(0x7f1a780662c0 msgr2=0x7f1a78159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 -- 172.21.15.115:0/66331791 shutdown_connections 2023-12-01T01:52:38.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:38.863+0000 7f1a7cbae640 1 -- 172.21.15.115:0/66331791 wait complete. 2023-12-01T01:52:38.884 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T01:52:39.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.090+0000 7fed4a081640 1 Processor -- start 2023-12-01T01:52:39.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.091+0000 7fed4a081640 1 -- start start 2023-12-01T01:52:39.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.092+0000 7fed4a081640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fed4406c3a0 0x7fed44065960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.093+0000 7fed4a081640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 0x7fed440706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.093+0000 7fed4a081640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4406c830 con 0x7fed44160520 2023-12-01T01:52:39.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.093+0000 7fed4a081640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed44161b30 con 0x7fed4406c3a0 2023-12-01T01:52:39.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.093+0000 7fed4a081640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed44066330 con 0x7fed44065ea0 2023-12-01T01:52:39.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.093+0000 7fed43fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 0x7fed440706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.093+0000 7fed42ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fed4406c3a0 0x7fed44065960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.093+0000 7fed43fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 0x7fed440706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:56196/0 (socket says 172.21.15.115:56196) 2023-12-01T01:52:39.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.093+0000 7fed437fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fed44160520 0x7fed440c6080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:55862/0 (socket says 172.21.15.115:55862) 2023-12-01T01:52:39.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.093+0000 7fed43fff640 1 -- 172.21.15.115:0/1594388670 learned_addr learned my addr 172.21.15.115:0/1594388670 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:39.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.094+0000 7fed43fff640 1 -- 172.21.15.115:0/1594388670 >> v2:172.21.15.142:3300/0 conn(0x7fed4406c3a0 msgr2=0x7fed44065960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:39.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.094+0000 7fed43fff640 1 --2- 172.21.15.115:0/1594388670 >> v2:172.21.15.142:3300/0 conn(0x7fed4406c3a0 0x7fed44065960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.094+0000 7fed43fff640 1 -- 172.21.15.115:0/1594388670 >> v1:172.21.15.142:6789/0 conn(0x7fed44160520 legacy=0x7fed440c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.094+0000 7fed43fff640 1 -- 172.21.15.115:0/1594388670 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed44160900 con 0x7fed44065ea0 2023-12-01T01:52:39.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.096+0000 7fed43fff640 1 --2- 172.21.15.115:0/1594388670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 0x7fed440706d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009870 tx=0x7fed3c02b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4afd0466a6650ca5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:39.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.096+0000 7fed427fc640 1 -- 172.21.15.115:0/1594388670 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c031860 con 0x7fed44065ea0 2023-12-01T01:52:39.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.096+0000 7fed427fc640 1 -- 172.21.15.115:0/1594388670 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3c0319c0 con 0x7fed44065ea0 2023-12-01T01:52:39.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.096+0000 7fed427fc640 1 -- 172.21.15.115:0/1594388670 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c031d50 con 0x7fed44065ea0 2023-12-01T01:52:39.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.097+0000 7fed4a081640 1 -- 172.21.15.115:0/1594388670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 msgr2=0x7fed440706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.097+0000 7fed4a081640 1 --2- 172.21.15.115:0/1594388670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 0x7fed440706d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009870 tx=0x7fed3c02b900 comp rx=0 tx=0).stop 2023-12-01T01:52:39.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.097+0000 7fed4a081640 1 -- 172.21.15.115:0/1594388670 shutdown_connections 2023-12-01T01:52:39.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.097+0000 7fed4a081640 1 --2- 172.21.15.115:0/1594388670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 0x7fed440706d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.097+0000 7fed4a081640 1 --2- 172.21.15.115:0/1594388670 >> v2:172.21.15.142:3300/0 conn(0x7fed4406c3a0 0x7fed44065960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.097+0000 7fed4a081640 1 -- 172.21.15.115:0/1594388670 >> 172.21.15.115:0/1594388670 conn(0x7fed440cf980 msgr2=0x7fed440cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:39.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.097+0000 7fed4a081640 1 -- 172.21.15.115:0/1594388670 shutdown_connections 2023-12-01T01:52:39.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.097+0000 7fed4a081640 1 -- 172.21.15.115:0/1594388670 wait complete. 2023-12-01T01:52:39.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.099+0000 7fed4a081640 1 Processor -- start 2023-12-01T01:52:39.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.099+0000 7fed4a081640 1 -- start start 2023-12-01T01:52:39.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed4a081640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 0x7fed440b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed4a081640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4406c3a0 0x7fed440c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed4a081640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed44160520 0x7fed4414d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed4a081640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed440ca370 con 0x7fed44160520 2023-12-01T01:52:39.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed4a081640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed440ca650 con 0x7fed4406c3a0 2023-12-01T01:52:39.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed4a081640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed440ca930 con 0x7fed44065ea0 2023-12-01T01:52:39.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed43fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed44160520 0x7fed4414d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed43fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed44160520 0x7fed4414d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46958/0 (socket says 172.21.15.115:46958) 2023-12-01T01:52:39.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed43fff640 1 -- 172.21.15.115:0/2612893004 learned_addr learned my addr 172.21.15.115:0/2612893004 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:39.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed437fe640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 0x7fed440b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.100+0000 7fed42ffd640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4406c3a0 0x7fed440c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed43fff640 1 -- 172.21.15.115:0/2612893004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 msgr2=0x7fed440b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed43fff640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 0x7fed440b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed43fff640 1 -- 172.21.15.115:0/2612893004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4406c3a0 msgr2=0x7fed440c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed43fff640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4406c3a0 0x7fed440c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed43fff640 1 -- 172.21.15.115:0/2612893004 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c0094d0 con 0x7fed44160520 2023-12-01T01:52:39.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed43fff640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed44160520 0x7fed4414d610 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009870 tx=0x7fed3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:39.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed3bfff640 1 -- 172.21.15.115:0/2612893004 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c03ac60 con 0x7fed44160520 2023-12-01T01:52:39.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed3bfff640 1 -- 172.21.15.115:0/2612893004 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3c03ade0 con 0x7fed44160520 2023-12-01T01:52:39.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed4a081640 1 -- 172.21.15.115:0/2612893004 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed440cac10 con 0x7fed44160520 2023-12-01T01:52:39.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed3bfff640 1 -- 172.21.15.115:0/2612893004 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c044630 con 0x7fed44160520 2023-12-01T01:52:39.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.101+0000 7fed4a081640 1 -- 172.21.15.115:0/2612893004 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed440c9fb0 con 0x7fed44160520 2023-12-01T01:52:39.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.103+0000 7fed4a081640 1 -- 172.21.15.115:0/2612893004 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed440cb130 con 0x7fed44160520 2023-12-01T01:52:39.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.104+0000 7fed3bfff640 1 -- 172.21.15.115:0/2612893004 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fed3c04d450 con 0x7fed44160520 2023-12-01T01:52:39.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.104+0000 7fed3bfff640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed2406a550 0x7fed2406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.104+0000 7fed3bfff640 1 -- 172.21.15.115:0/2612893004 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5545+0+0 (secure 0 0 0) 0x7fed3c0aaf70 con 0x7fed44160520 2023-12-01T01:52:39.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.105+0000 7fed437fe640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed2406a550 0x7fed2406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.105+0000 7fed437fe640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed2406a550 0x7fed2406ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fed340012e0 tx=0x7fed34005b70 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:39.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.109+0000 7fed3bfff640 1 -- 172.21.15.115:0/2612893004 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fed3c0ddc80 con 0x7fed44160520 2023-12-01T01:52:39.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.306+0000 7fed4a081640 1 -- 172.21.15.115:0/2612893004 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fed4404e890 con 0x7fed44160520 2023-12-01T01:52:39.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.314+0000 7fed3bfff640 1 -- 172.21.15.115:0/2612893004 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fed440cb130 con 0x7fed44160520 2023-12-01T01:52:39.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.317+0000 7fed4a081640 1 -- 172.21.15.115:0/2612893004 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed2406a550 msgr2=0x7fed2406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.317+0000 7fed4a081640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed2406a550 0x7fed2406ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fed340012e0 tx=0x7fed34005b70 comp rx=0 tx=0).stop 2023-12-01T01:52:39.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.317+0000 7fed4a081640 1 -- 172.21.15.115:0/2612893004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed44160520 msgr2=0x7fed4414d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.317+0000 7fed4a081640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed44160520 0x7fed4414d610 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009870 tx=0x7fed3c0046c0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.318+0000 7fed4a081640 1 -- 172.21.15.115:0/2612893004 shutdown_connections 2023-12-01T01:52:39.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.318+0000 7fed4a081640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed2406a550 0x7fed2406ca10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.318+0000 7fed4a081640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed44160520 0x7fed4414d610 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.318+0000 7fed4a081640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4406c3a0 0x7fed440c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.318+0000 7fed4a081640 1 --2- 172.21.15.115:0/2612893004 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed44065ea0 0x7fed440b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.318+0000 7fed4a081640 1 -- 172.21.15.115:0/2612893004 >> 172.21.15.115:0/2612893004 conn(0x7fed440cf980 msgr2=0x7fed440c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.318+0000 7fed4a081640 1 -- 172.21.15.115:0/2612893004 shutdown_connections 2023-12-01T01:52:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.318+0000 7fed4a081640 1 -- 172.21.15.115:0/2612893004 wait complete. 2023-12-01T01:52:39.337 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T01:52:39.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.550+0000 7f86ac0ab640 1 Processor -- start 2023-12-01T01:52:39.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.550+0000 7f86ac0ab640 1 -- start start 2023-12-01T01:52:39.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.552+0000 7f86ac0ab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86a40b29d0 0x7f86a40cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.552+0000 7f86ac0ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86a40afae0 0x7f86a40cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.552+0000 7f86ac0ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a40b33e0 con 0x7f86a415d0b0 2023-12-01T01:52:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.552+0000 7f86ac0ab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a415d490 con 0x7f86a40b29d0 2023-12-01T01:52:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.552+0000 7f86ac0ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40cb720 con 0x7f86a40afae0 2023-12-01T01:52:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.552+0000 7f86a9646640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86a40b29d0 0x7f86a40cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.552+0000 7f86a9646640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86a40b29d0 0x7f86a40cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46974/0 (socket says 172.21.15.115:46974) 2023-12-01T01:52:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.552+0000 7f86a9646640 1 -- 172.21.15.115:0/4148285869 learned_addr learned my addr 172.21.15.115:0/4148285869 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.553+0000 7f86aa648640 1 --2- 172.21.15.115:0/4148285869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86a40afae0 0x7f86a40cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.553+0000 7f86a9646640 1 -- 172.21.15.115:0/4148285869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86a40afae0 msgr2=0x7f86a40cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:39.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.553+0000 7f86a9646640 1 --2- 172.21.15.115:0/4148285869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86a40afae0 0x7f86a40cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.553+0000 7f86a9646640 1 -- 172.21.15.115:0/4148285869 >> v1:172.21.15.142:6789/0 conn(0x7f86a415d0b0 legacy=0x7f86a40cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:39.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.553+0000 7f86a9646640 1 -- 172.21.15.115:0/4148285869 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a4069df0 con 0x7f86a40b29d0 2023-12-01T01:52:39.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.554+0000 7f86a9646640 1 --2- 172.21.15.115:0/4148285869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40b29d0 0x7f86a40cd100 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f86940098a0 tx=0x7f869402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b369c2e27b4f0232 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.555+0000 7f86a8e45640 1 -- 172.21.15.115:0/4148285869 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694031640 con 0x7f86a40b29d0 2023-12-01T01:52:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.555+0000 7f86a8e45640 1 -- 172.21.15.115:0/4148285869 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86940317a0 con 0x7f86a40b29d0 2023-12-01T01:52:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.555+0000 7f86a8e45640 1 -- 172.21.15.115:0/4148285869 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694031b30 con 0x7f86a40b29d0 2023-12-01T01:52:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.555+0000 7f86ac0ab640 1 -- 172.21.15.115:0/4148285869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40b29d0 msgr2=0x7f86a40cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.555+0000 7f86ac0ab640 1 --2- 172.21.15.115:0/4148285869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40b29d0 0x7f86a40cd100 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f86940098a0 tx=0x7f869402b8f0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.555+0000 7f86ac0ab640 1 -- 172.21.15.115:0/4148285869 shutdown_connections 2023-12-01T01:52:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.555+0000 7f86ac0ab640 1 --2- 172.21.15.115:0/4148285869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86a40afae0 0x7f86a40cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.555+0000 7f86ac0ab640 1 --2- 172.21.15.115:0/4148285869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40b29d0 0x7f86a40cd100 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.555+0000 7f86ac0ab640 1 -- 172.21.15.115:0/4148285869 >> 172.21.15.115:0/4148285869 conn(0x7f86a4071430 msgr2=0x7f86a4071840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.555+0000 7f86ac0ab640 1 -- 172.21.15.115:0/4148285869 shutdown_connections 2023-12-01T01:52:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.556+0000 7f86ac0ab640 1 -- 172.21.15.115:0/4148285869 wait complete. 2023-12-01T01:52:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.557+0000 7f86ac0ab640 1 Processor -- start 2023-12-01T01:52:39.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.557+0000 7f86ac0ab640 1 -- start start 2023-12-01T01:52:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.558+0000 7f86ac0ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40afae0 0x7f86a415d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.558+0000 7f86ac0ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a40b29d0 0x7f86a415f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.558+0000 7f86ac0ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86a415d0b0 0x7f86a4154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.558+0000 7f86ac0ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a415e490 con 0x7f86a40afae0 2023-12-01T01:52:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.558+0000 7f86ac0ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a415dbf0 con 0x7f86a40b29d0 2023-12-01T01:52:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.558+0000 7f86ac0ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a415ded0 con 0x7f86a415d0b0 2023-12-01T01:52:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.558+0000 7f86a9e47640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40afae0 0x7f86a415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f86a9e47640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40afae0 0x7f86a415d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46976/0 (socket says 172.21.15.115:46976) 2023-12-01T01:52:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f86a9e47640 1 -- 172.21.15.115:0/1198308378 learned_addr learned my addr 172.21.15.115:0/1198308378 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f86a9e47640 1 -- 172.21.15.115:0/1198308378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86a415d0b0 msgr2=0x7f86a4154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f86a9e47640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86a415d0b0 0x7f86a4154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f86a9e47640 1 -- 172.21.15.115:0/1198308378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a40b29d0 msgr2=0x7f86a415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f86a9e47640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a40b29d0 0x7f86a415f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f86a9e47640 1 -- 172.21.15.115:0/1198308378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940094d0 con 0x7f86a40afae0 2023-12-01T01:52:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f86a9e47640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40afae0 0x7f86a415d610 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f86a000c430 tx=0x7f86a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f869a7fc640 1 -- 172.21.15.115:0/1198308378 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0011790 con 0x7f86a40afae0 2023-12-01T01:52:39.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f869a7fc640 1 -- 172.21.15.115:0/1198308378 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a00118f0 con 0x7f86a40afae0 2023-12-01T01:52:39.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.559+0000 7f869a7fc640 1 -- 172.21.15.115:0/1198308378 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0011ce0 con 0x7f86a40afae0 2023-12-01T01:52:39.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.560+0000 7f86ac0ab640 1 -- 172.21.15.115:0/1198308378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a415e1b0 con 0x7f86a40afae0 2023-12-01T01:52:39.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.560+0000 7f86ac0ab640 1 -- 172.21.15.115:0/1198308378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a415f720 con 0x7f86a40afae0 2023-12-01T01:52:39.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.561+0000 7f86ac0ab640 1 -- 172.21.15.115:0/1198308378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a404e890 con 0x7f86a40afae0 2023-12-01T01:52:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.562+0000 7f869a7fc640 1 -- 172.21.15.115:0/1198308378 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f86a001c460 con 0x7f86a40afae0 2023-12-01T01:52:39.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.562+0000 7f869a7fc640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f868806a550 0x7f868806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.563+0000 7f869a7fc640 1 -- 172.21.15.115:0/1198308378 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f86a0088950 con 0x7f86a40afae0 2023-12-01T01:52:39.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.563+0000 7f86a9646640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f868806a550 0x7f868806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.563+0000 7f86a9646640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f868806a550 0x7f868806ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8694036000 tx=0x7f8694030000 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.567+0000 7f869a7fc640 1 -- 172.21.15.115:0/1198308378 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f86a0053f40 con 0x7f86a40afae0 2023-12-01T01:52:39.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.744+0000 7f86ac0ab640 1 -- 172.21.15.115:0/1198308378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f86a40b6940 con 0x7f86a40afae0 2023-12-01T01:52:39.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.752+0000 7f869a7fc640 1 -- 172.21.15.115:0/1198308378 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v32) v1 ==== 120+0+0 (secure 0 0 0) 0x7f86a0057560 con 0x7f86a40afae0 2023-12-01T01:52:39.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.755+0000 7f86ac0ab640 1 -- 172.21.15.115:0/1198308378 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f868806a550 msgr2=0x7f868806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.755+0000 7f86ac0ab640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f868806a550 0x7f868806ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8694036000 tx=0x7f8694030000 comp rx=0 tx=0).stop 2023-12-01T01:52:39.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.755+0000 7f86ac0ab640 1 -- 172.21.15.115:0/1198308378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40afae0 msgr2=0x7f86a415d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.755+0000 7f86ac0ab640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40afae0 0x7f86a415d610 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f86a000c430 tx=0x7f86a000c8f0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.756+0000 7f86ac0ab640 1 -- 172.21.15.115:0/1198308378 shutdown_connections 2023-12-01T01:52:39.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.756+0000 7f86ac0ab640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f868806a550 0x7f868806ca10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.756+0000 7f86ac0ab640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86a415d0b0 0x7f86a4154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.756+0000 7f86ac0ab640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86a40b29d0 0x7f86a415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.756+0000 7f86ac0ab640 1 --2- 172.21.15.115:0/1198308378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86a40afae0 0x7f86a415d610 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.756+0000 7f86ac0ab640 1 -- 172.21.15.115:0/1198308378 >> 172.21.15.115:0/1198308378 conn(0x7f86a4071430 msgr2=0x7f86a4071810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.756+0000 7f86ac0ab640 1 -- 172.21.15.115:0/1198308378 shutdown_connections 2023-12-01T01:52:39.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.756+0000 7f86ac0ab640 1 -- 172.21.15.115:0/1198308378 wait complete. 2023-12-01T01:52:39.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T01:52:39.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.985+0000 7ff1e095c640 1 Processor -- start 2023-12-01T01:52:39.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.986+0000 7ff1e095c640 1 -- start start 2023-12-01T01:52:39.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.987+0000 7ff1e095c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1dc15ac80 0x7ff1dc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.987+0000 7ff1e095c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff1dc0b0e60 0x7ff1dc0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.987+0000 7ff1e095c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1dc0cc9a0 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.987+0000 7ff1e095c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1dc15b060 con 0x7ff1dc0b0e60 2023-12-01T01:52:39.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.987+0000 7ff1e095c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0b0ce0 con 0x7ff1dc15ac80 2023-12-01T01:52:39.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.988+0000 7ff1dad9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff1dc0cdaa0 0x7ff1dc06b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40420/0 (socket says 172.21.15.115:40420) 2023-12-01T01:52:39.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.988+0000 7ff1dad9d640 1 -- 172.21.15.115:0/1933577838 learned_addr learned my addr 172.21.15.115:0/1933577838 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:39.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.988+0000 7ff1d3fff640 1 --2- 172.21.15.115:0/1933577838 >> v2:172.21.15.115:3300/0 conn(0x7ff1dc0b0e60 0x7ff1dc0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.988+0000 7ff1da59c640 1 --2- 172.21.15.115:0/1933577838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1dc15ac80 0x7ff1dc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.988+0000 7ff1d3fff640 1 -- 172.21.15.115:0/1933577838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1dc15ac80 msgr2=0x7ff1dc0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:39.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.988+0000 7ff1d3fff640 1 --2- 172.21.15.115:0/1933577838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1dc15ac80 0x7ff1dc0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.988+0000 7ff1d3fff640 1 -- 172.21.15.115:0/1933577838 >> v1:172.21.15.115:6789/0 conn(0x7ff1dc0cdaa0 legacy=0x7ff1dc06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.988+0000 7ff1d3fff640 1 -- 172.21.15.115:0/1933577838 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc0cbd90 con 0x7ff1dc0b0e60 2023-12-01T01:52:39.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.989+0000 7ff1d3fff640 1 --2- 172.21.15.115:0/1933577838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0b0e60 0x7ff1dc0b07a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40097a0 tx=0x7ff1c402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b920b121400be3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:39.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.990+0000 7ff1d9d9b640 1 -- 172.21.15.115:0/1933577838 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c402d040 con 0x7ff1dc0b0e60 2023-12-01T01:52:39.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.990+0000 7ff1d9d9b640 1 -- 172.21.15.115:0/1933577838 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c4031810 con 0x7ff1dc0b0e60 2023-12-01T01:52:39.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.990+0000 7ff1d9d9b640 1 -- 172.21.15.115:0/1933577838 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4031ba0 con 0x7ff1dc0b0e60 2023-12-01T01:52:39.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.990+0000 7ff1e095c640 1 -- 172.21.15.115:0/1933577838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0b0e60 msgr2=0x7ff1dc0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.990+0000 7ff1e095c640 1 --2- 172.21.15.115:0/1933577838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0b0e60 0x7ff1dc0b07a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40097a0 tx=0x7ff1c402b8b0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.990+0000 7ff1e095c640 1 -- 172.21.15.115:0/1933577838 shutdown_connections 2023-12-01T01:52:39.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.990+0000 7ff1e095c640 1 --2- 172.21.15.115:0/1933577838 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0b0e60 0x7ff1dc0b07a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.990+0000 7ff1e095c640 1 --2- 172.21.15.115:0/1933577838 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1dc15ac80 0x7ff1dc0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.990+0000 7ff1e095c640 1 -- 172.21.15.115:0/1933577838 >> 172.21.15.115:0/1933577838 conn(0x7ff1dc065960 msgr2=0x7ff1dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:39.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.991+0000 7ff1e095c640 1 -- 172.21.15.115:0/1933577838 shutdown_connections 2023-12-01T01:52:39.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.991+0000 7ff1e095c640 1 -- 172.21.15.115:0/1933577838 wait complete. 2023-12-01T01:52:39.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.992+0000 7ff1e095c640 1 Processor -- start 2023-12-01T01:52:39.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.993+0000 7ff1e095c640 1 -- start start 2023-12-01T01:52:39.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.993+0000 7ff1e095c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1dc0b0e60 0x7ff1dc1f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1e095c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0cdaa0 0x7ff1dc14e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1e095c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1dc15ac80 0x7ff1dc151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1e095c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc152ef0 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1e095c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc14e6d0 con 0x7ff1dc0b0e60 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1e095c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc153190 con 0x7ff1dc15ac80 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0cdaa0 0x7ff1dc14e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0cdaa0 0x7ff1dc14e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46988/0 (socket says 172.21.15.115:46988) 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1d3fff640 1 -- 172.21.15.115:0/3635026789 learned_addr learned my addr 172.21.15.115:0/3635026789 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1d3fff640 1 -- 172.21.15.115:0/3635026789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1dc15ac80 msgr2=0x7ff1dc151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1dad9d640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1dc15ac80 0x7ff1dc151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1da59c640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1dc0b0e60 0x7ff1dc1f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1d3fff640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1dc15ac80 0x7ff1dc151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1d3fff640 1 -- 172.21.15.115:0/3635026789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1dc0b0e60 msgr2=0x7ff1dc1f2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1d3fff640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1dc0b0e60 0x7ff1dc1f2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.994+0000 7ff1d3fff640 1 -- 172.21.15.115:0/3635026789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c4009400 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.995+0000 7ff1d3fff640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0cdaa0 0x7ff1dc14e190 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40098c0 tx=0x7ff1c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.995+0000 7ff1d2ffd640 1 -- 172.21.15.115:0/3635026789 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c403ad60 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.995+0000 7ff1d2ffd640 1 -- 172.21.15.115:0/3635026789 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c40443f0 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.995+0000 7ff1d2ffd640 1 -- 172.21.15.115:0/3635026789 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40447a0 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.995+0000 7ff1e095c640 1 -- 172.21.15.115:0/3635026789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc153470 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.995+0000 7ff1e095c640 1 -- 172.21.15.115:0/3635026789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc151a10 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.996+0000 7ff1e095c640 1 -- 172.21.15.115:0/3635026789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc04e890 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.997+0000 7ff1d2ffd640 1 -- 172.21.15.115:0/3635026789 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff1c404d450 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.998+0000 7ff1d2ffd640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff1bc06a550 0x7ff1bc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:39.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.998+0000 7ff1d2ffd640 1 -- 172.21.15.115:0/3635026789 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5545+0+0 (secure 0 0 0) 0x7ff1c40aaee0 con 0x7ff1dc0cdaa0 2023-12-01T01:52:39.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.998+0000 7ff1da59c640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff1bc06a550 0x7ff1bc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:40.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:39.999+0000 7ff1da59c640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff1bc06a550 0x7ff1bc06ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc006fd0 tx=0x7ff1cc008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.002+0000 7ff1d2ffd640 1 -- 172.21.15.115:0/3635026789 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff1c40762b0 con 0x7ff1dc0cdaa0 2023-12-01T01:52:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.205+0000 7ff1e095c640 1 -- 172.21.15.115:0/3635026789 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ff1dc0c3cf0 con 0x7ff1dc0cdaa0 2023-12-01T01:52:40.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.213+0000 7ff1d2ffd640 1 -- 172.21.15.115:0/3635026789 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v33) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff1c40798d0 con 0x7ff1dc0cdaa0 2023-12-01T01:52:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 -- 172.21.15.115:0/3635026789 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff1bc06a550 msgr2=0x7ff1bc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff1bc06a550 0x7ff1bc06ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc006fd0 tx=0x7ff1cc008040 comp rx=0 tx=0).stop 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 -- 172.21.15.115:0/3635026789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0cdaa0 msgr2=0x7ff1dc14e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0cdaa0 0x7ff1dc14e190 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40098c0 tx=0x7ff1c40046c0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 -- 172.21.15.115:0/3635026789 shutdown_connections 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff1bc06a550 0x7ff1bc06ca10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1dc15ac80 0x7ff1dc151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1dc0cdaa0 0x7ff1dc14e190 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 --2- 172.21.15.115:0/3635026789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1dc0b0e60 0x7ff1dc1f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 -- 172.21.15.115:0/3635026789 >> 172.21.15.115:0/3635026789 conn(0x7ff1dc065960 msgr2=0x7ff1dc14eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 -- 172.21.15.115:0/3635026789 shutdown_connections 2023-12-01T01:52:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.216+0000 7ff1e095c640 1 -- 172.21.15.115:0/3635026789 wait complete. 2023-12-01T01:52:40.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-12-01T01:52:40.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.438+0000 7f62415d9640 1 Processor -- start 2023-12-01T01:52:40.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.439+0000 7f62415d9640 1 -- start start 2023-12-01T01:52:40.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.440+0000 7f62415d9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f623c0aded0 0x7f623c0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:40.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.440+0000 7f62415d9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c06f2f0 con 0x7f623c0c8ed0 2023-12-01T01:52:40.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.440+0000 7f62415d9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c06f5d0 con 0x7f623c06c2f0 2023-12-01T01:52:40.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.440+0000 7f62415d9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c0c9350 con 0x7f623c0aded0 2023-12-01T01:52:40.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.440+0000 7f62327fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f623c0c8ed0 0x7f623c06bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40426/0 (socket says 172.21.15.115:40426) 2023-12-01T01:52:40.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.440+0000 7f62327fc640 1 -- 172.21.15.115:0/1106645230 learned_addr learned my addr 172.21.15.115:0/1106645230 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497764867 0 0) 0x7f623c06f2f0 con 0x7f623c0c8ed0 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6228003520 con 0x7f623c0c8ed0 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623affd640 1 --2- 172.21.15.115:0/1106645230 >> v2:172.21.15.142:3300/0 conn(0x7f623c0aded0 0x7f623c0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3061031809 0 0) 0x7f6228003520 con 0x7f623c0c8ed0 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 >> v2:172.21.15.142:3300/0 conn(0x7f623c0aded0 msgr2=0x7f623c0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 --2- 172.21.15.115:0/1106645230 >> v2:172.21.15.142:3300/0 conn(0x7f623c0aded0 0x7f623c0cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 >> v1:172.21.15.142:6789/0 conn(0x7f623c06c2f0 legacy=0x7f623c06fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c0c9630 con 0x7f623c0c8ed0 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6224001670 con 0x7f623c0c8ed0 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 >> v1:172.21.15.115:6789/0 conn(0x7f623c0c8ed0 legacy=0x7f623c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 --2- 172.21.15.115:0/1106645230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62280039e0 0x7f6228003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 --2- 172.21.15.115:0/1106645230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62280047c0 0x7f6228004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 --2- 172.21.15.115:0/1106645230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62280052a0 0x7f622800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0c9350 con 0x7f62280047c0 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622800c1c0 con 0x7f62280052a0 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.441+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622800e590 con 0x7f62280039e0 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.442+0000 7f62327fc640 1 --2- 172.21.15.115:0/1106645230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62280047c0 0x7f6228004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.442+0000 7f62327fc640 1 -- 172.21.15.115:0/1106645230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62280039e0 msgr2=0x7f6228003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.442+0000 7f62327fc640 1 --2- 172.21.15.115:0/1106645230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62280039e0 0x7f6228003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.442+0000 7f62327fc640 1 -- 172.21.15.115:0/1106645230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62280052a0 msgr2=0x7f622800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.442+0000 7f62327fc640 1 --2- 172.21.15.115:0/1106645230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62280052a0 0x7f622800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.442+0000 7f62327fc640 1 -- 172.21.15.115:0/1106645230 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228003520 con 0x7f62280047c0 2023-12-01T01:52:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.442+0000 7f623b7fe640 1 -- 172.21.15.115:0/1106645230 reap_dead start 2023-12-01T01:52:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.443+0000 7f62327fc640 1 --2- 172.21.15.115:0/1106645230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62280047c0 0x7f6228004c40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f623c0cf450 tx=0x7f62280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240e0d131cc128c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.443+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224005ad0 con 0x7f62280047c0 2023-12-01T01:52:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.443+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6224005c50 con 0x7f62280047c0 2023-12-01T01:52:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.443+0000 7f623a7fc640 1 -- 172.21.15.115:0/1106645230 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622402c8b0 con 0x7f62280047c0 2023-12-01T01:52:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.444+0000 7f62415d9640 1 -- 172.21.15.115:0/1106645230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62280047c0 msgr2=0x7f6228004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.444+0000 7f62415d9640 1 --2- 172.21.15.115:0/1106645230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62280047c0 0x7f6228004c40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f623c0cf450 tx=0x7f62280057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.444+0000 7f62415d9640 1 -- 172.21.15.115:0/1106645230 shutdown_connections 2023-12-01T01:52:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.444+0000 7f62415d9640 1 --2- 172.21.15.115:0/1106645230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62280047c0 0x7f6228004c40 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.444+0000 7f62415d9640 1 -- 172.21.15.115:0/1106645230 >> 172.21.15.115:0/1106645230 conn(0x7f623c0ce880 msgr2=0x7f623c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.444+0000 7f62415d9640 1 -- 172.21.15.115:0/1106645230 shutdown_connections 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.444+0000 7f62415d9640 1 -- 172.21.15.115:0/1106645230 wait complete. 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.446+0000 7f62415d9640 1 Processor -- start 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.446+0000 7f62415d9640 1 -- start start 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62415d9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62280047c0 0x7f623c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62415d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f623c0b4070 0x7f623c157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62415d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f623c0c23b0 0x7f623c0c2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62415d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0b4530 con 0x7f623c0b4070 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62415d9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0c4fe0 con 0x7f62280047c0 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62415d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0c46b0 con 0x7f623c0c23b0 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62327fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f623c0b4070 0x7f623c157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62327fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f623c0b4070 0x7f623c157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46998/0 (socket says 172.21.15.115:46998) 2023-12-01T01:52:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62327fc640 1 -- 172.21.15.115:0/125705126 learned_addr learned my addr 172.21.15.115:0/125705126 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f623affd640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62280047c0 0x7f623c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62327fc640 1 -- 172.21.15.115:0/125705126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f623c0c23b0 msgr2=0x7f623c0c2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62327fc640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f623c0c23b0 0x7f623c0c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62327fc640 1 -- 172.21.15.115:0/125705126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62280047c0 msgr2=0x7f623c1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62327fc640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62280047c0 0x7f623c1574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.447+0000 7f62327fc640 1 -- 172.21.15.115:0/125705126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c0c4990 con 0x7f623c0b4070 2023-12-01T01:52:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.448+0000 7f62327fc640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f623c0b4070 0x7f623c157a00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6224001930 tx=0x7f622402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.448+0000 7f6233fff640 1 -- 172.21.15.115:0/125705126 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622403c040 con 0x7f623c0b4070 2023-12-01T01:52:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.448+0000 7f6233fff640 1 -- 172.21.15.115:0/125705126 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622402cd80 con 0x7f623c0b4070 2023-12-01T01:52:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.448+0000 7f62415d9640 1 -- 172.21.15.115:0/125705126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c0c4c70 con 0x7f623c0b4070 2023-12-01T01:52:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.448+0000 7f6233fff640 1 -- 172.21.15.115:0/125705126 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224007290 con 0x7f623c0b4070 2023-12-01T01:52:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.448+0000 7f62415d9640 1 -- 172.21.15.115:0/125705126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c0c2d60 con 0x7f623c0b4070 2023-12-01T01:52:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.449+0000 7f62415d9640 1 -- 172.21.15.115:0/125705126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c0c5170 con 0x7f623c0b4070 2023-12-01T01:52:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.450+0000 7f6233fff640 1 -- 172.21.15.115:0/125705126 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6224032070 con 0x7f623c0b4070 2023-12-01T01:52:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.451+0000 7f6233fff640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f621c06a580 0x7f621c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.451+0000 7f6233fff640 1 -- 172.21.15.115:0/125705126 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f62240aa030 con 0x7f623c0b4070 2023-12-01T01:52:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.451+0000 7f623affd640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f621c06a580 0x7f621c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:40.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.452+0000 7f623affd640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f621c06a580 0x7f621c06ca40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f623c04f940 tx=0x7f622c00b040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.455+0000 7f6233fff640 1 -- 172.21.15.115:0/125705126 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f62240756e0 con 0x7f623c0b4070 2023-12-01T01:52:40.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.717+0000 7f62415d9640 1 -- 172.21.15.115:0/125705126 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f623c04e890 con 0x7f623c0b4070 2023-12-01T01:52:40.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.872+0000 7f6233fff640 1 -- 172.21.15.115:0/125705126 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v630) v1 ==== 166+0+0 (secure 0 0 0) 0x7f6224078d00 con 0x7f623c0b4070 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 -- 172.21.15.115:0/125705126 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f621c06a580 msgr2=0x7f621c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f621c06a580 0x7f621c06ca40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f623c04f940 tx=0x7f622c00b040 comp rx=0 tx=0).stop 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 -- 172.21.15.115:0/125705126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f623c0b4070 msgr2=0x7f623c157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f623c0b4070 0x7f623c157a00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6224001930 tx=0x7f622402e070 comp rx=0 tx=0).stop 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 -- 172.21.15.115:0/125705126 shutdown_connections 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f621c06a580 0x7f621c06ca40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f623c0c23b0 0x7f623c0c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f623c0b4070 0x7f623c157a00 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 --2- 172.21.15.115:0/125705126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62280047c0 0x7f623c1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 -- 172.21.15.115:0/125705126 >> 172.21.15.115:0/125705126 conn(0x7f623c0ce880 msgr2=0x7f623c06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 -- 172.21.15.115:0/125705126 shutdown_connections 2023-12-01T01:52:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:40.875+0000 7f62415d9640 1 -- 172.21.15.115:0/125705126 wait complete. 2023-12-01T01:52:40.898 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-12-01T01:52:40.898 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-12-01T01:52:40.898 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-01T01:52:40.899 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-12-01T01:52:41.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.110+0000 7f5ce11ab640 1 Processor -- start 2023-12-01T01:52:41.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.110+0000 7f5ce11ab640 1 -- start start 2023-12-01T01:52:41.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.111+0000 7f5ce11ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc15e2f0 0x7f5cdc0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:41.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.111+0000 7f5ce11ab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5cdc06f2f0 0x7f5cdc0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:41.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.111+0000 7f5ce11ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cdc0add10 con 0x7f5cdc0adf10 2023-12-01T01:52:41.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.111+0000 7f5ce11ab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cdc0c5760 con 0x7f5cdc06f2f0 2023-12-01T01:52:41.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.111+0000 7f5ce11ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc15e6d0 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.112+0000 7f5cdad9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc15e2f0 0x7f5cdc0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:41.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.112+0000 7f5cda59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5cdc06f2f0 0x7f5cdc0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:41.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.112+0000 7f5cdad9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc15e2f0 0x7f5cdc0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:56262/0 (socket says 172.21.15.115:56262) 2023-12-01T01:52:41.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.112+0000 7f5cda59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5cdc06f2f0 0x7f5cdc0caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:52802/0 (socket says 172.21.15.115:52802) 2023-12-01T01:52:41.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.112+0000 7f5cdad9d640 1 -- 172.21.15.115:0/2416346554 learned_addr learned my addr 172.21.15.115:0/2416346554 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:41.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.113+0000 7f5cd9d9b640 1 -- 172.21.15.115:0/2416346554 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624323630 0 0) 0x7f5cdc0add10 con 0x7f5cdc0adf10 2023-12-01T01:52:41.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.113+0000 7f5cd9d9b640 1 -- 172.21.15.115:0/2416346554 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cc4003520 con 0x7f5cdc0adf10 2023-12-01T01:52:41.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.113+0000 7f5cdad9d640 1 -- 172.21.15.115:0/2416346554 >> v2:172.21.15.142:3300/0 conn(0x7f5cdc06f2f0 msgr2=0x7f5cdc0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:41.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.113+0000 7f5cdad9d640 1 --2- 172.21.15.115:0/2416346554 >> v2:172.21.15.142:3300/0 conn(0x7f5cdc06f2f0 0x7f5cdc0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:41.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.113+0000 7f5cdad9d640 1 -- 172.21.15.115:0/2416346554 >> v1:172.21.15.142:6789/0 conn(0x7f5cdc0adf10 legacy=0x7f5cdc0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:41.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.113+0000 7f5cdad9d640 1 -- 172.21.15.115:0/2416346554 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc0cf7b0 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.113+0000 7f5cda59c640 1 --2- 172.21.15.115:0/2416346554 >> v2:172.21.15.142:3300/0 conn(0x7f5cdc06f2f0 0x7f5cdc0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:52:41.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.115+0000 7f5cdad9d640 1 --2- 172.21.15.115:0/2416346554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc15e2f0 0x7f5cdc0ca660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009870 tx=0x7f5ccc02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c0e87248b63514 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:41.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.115+0000 7f5cd9d9b640 1 -- 172.21.15.115:0/2416346554 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0317c0 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.116+0000 7f5cd9d9b640 1 -- 172.21.15.115:0/2416346554 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc031920 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.116+0000 7f5cd9d9b640 1 -- 172.21.15.115:0/2416346554 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc031c90 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.116+0000 7f5ce11ab640 1 -- 172.21.15.115:0/2416346554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc15e2f0 msgr2=0x7f5cdc0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.116+0000 7f5ce11ab640 1 --2- 172.21.15.115:0/2416346554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc15e2f0 0x7f5cdc0ca660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009870 tx=0x7f5ccc02b860 comp rx=0 tx=0).stop 2023-12-01T01:52:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.116+0000 7f5ce11ab640 1 -- 172.21.15.115:0/2416346554 shutdown_connections 2023-12-01T01:52:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.116+0000 7f5ce11ab640 1 --2- 172.21.15.115:0/2416346554 >> v2:172.21.15.142:3300/0 conn(0x7f5cdc06f2f0 0x7f5cdc0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.116+0000 7f5ce11ab640 1 --2- 172.21.15.115:0/2416346554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc15e2f0 0x7f5cdc0ca660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.116+0000 7f5ce11ab640 1 -- 172.21.15.115:0/2416346554 >> 172.21.15.115:0/2416346554 conn(0x7f5cdc0ce880 msgr2=0x7f5cdc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.116+0000 7f5ce11ab640 1 -- 172.21.15.115:0/2416346554 shutdown_connections 2023-12-01T01:52:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.116+0000 7f5ce11ab640 1 -- 172.21.15.115:0/2416346554 wait complete. 2023-12-01T01:52:41.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.118+0000 7f5ce11ab640 1 Processor -- start 2023-12-01T01:52:41.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.118+0000 7f5ce11ab640 1 -- start start 2023-12-01T01:52:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.119+0000 7f5ce11ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cdc06f2f0 0x7f5cdc0c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.119+0000 7f5ce11ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc0adf10 0x7f5cdc0c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.119+0000 7f5ce11ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cdc15e2f0 0x7f5cdc14f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.119+0000 7f5ce11ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc0c6280 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.119+0000 7f5ce11ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc0b3af0 con 0x7f5cdc06f2f0 2023-12-01T01:52:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.119+0000 7f5ce11ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc0c5950 con 0x7f5cdc0adf10 2023-12-01T01:52:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cdb59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cdc15e2f0 0x7f5cdc14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cdb59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cdc15e2f0 0x7f5cdc14f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47002/0 (socket says 172.21.15.115:47002) 2023-12-01T01:52:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cdb59e640 1 -- 172.21.15.115:0/436395075 learned_addr learned my addr 172.21.15.115:0/436395075 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cda59c640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc0adf10 0x7f5cdc0c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cdad9d640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cdc06f2f0 0x7f5cdc0c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cdb59e640 1 -- 172.21.15.115:0/436395075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc0adf10 msgr2=0x7f5cdc0c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cdb59e640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc0adf10 0x7f5cdc0c40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cdb59e640 1 -- 172.21.15.115:0/436395075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cdc06f2f0 msgr2=0x7f5cdc0c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cdb59e640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cdc06f2f0 0x7f5cdc0c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cdb59e640 1 -- 172.21.15.115:0/436395075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd0009400 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.120+0000 7f5cdb59e640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cdc15e2f0 0x7f5cdc14f920 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400c730 tx=0x7f5cd400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.121+0000 7f5cc37fe640 1 -- 172.21.15.115:0/436395075 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4003b70 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.121+0000 7f5cc37fe640 1 -- 172.21.15.115:0/436395075 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd4003cf0 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.121+0000 7f5cc37fe640 1 -- 172.21.15.115:0/436395075 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4005310 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.121+0000 7f5ce11ab640 1 -- 172.21.15.115:0/436395075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ccc0094d0 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.121+0000 7f5ce11ab640 1 -- 172.21.15.115:0/436395075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cdc0c5f10 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.122+0000 7f5ce11ab640 1 -- 172.21.15.115:0/436395075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cdc06fda0 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.123+0000 7f5cc37fe640 1 -- 172.21.15.115:0/436395075 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f5cd4005680 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.124+0000 7f5cc37fe640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cc806a550 0x7f5cc806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.124+0000 7f5cc37fe640 1 -- 172.21.15.115:0/436395075 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f5cd4088a70 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.124+0000 7f5cdad9d640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cc806a550 0x7f5cc806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:41.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.125+0000 7f5cdad9d640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cc806a550 0x7f5cc806ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc036000 tx=0x7f5ccc03d040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:41.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.128+0000 7f5cc37fe640 1 -- 172.21.15.115:0/436395075 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5cd4053f50 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.386+0000 7f5ce11ab640 1 -- 172.21.15.115:0/436395075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f5cdc04e890 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.876+0000 7f5cc37fe640 1 -- 172.21.15.115:0/436395075 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v631) v1 ==== 186+0+0 (secure 0 0 0) 0x7f5cd4003e60 con 0x7f5cdc15e2f0 2023-12-01T01:52:41.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 -- 172.21.15.115:0/436395075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cc806a550 msgr2=0x7f5cc806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:41.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cc806a550 0x7f5cc806ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc036000 tx=0x7f5ccc03d040 comp rx=0 tx=0).stop 2023-12-01T01:52:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 -- 172.21.15.115:0/436395075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cdc15e2f0 msgr2=0x7f5cdc14f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cdc15e2f0 0x7f5cdc14f920 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400c730 tx=0x7f5cd400cbf0 comp rx=0 tx=0).stop 2023-12-01T01:52:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 -- 172.21.15.115:0/436395075 shutdown_connections 2023-12-01T01:52:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5cc806a550 0x7f5cc806ca10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cdc0adf10 0x7f5cdc0c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cdc15e2f0 0x7f5cdc14f920 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 --2- 172.21.15.115:0/436395075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cdc06f2f0 0x7f5cdc0c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 -- 172.21.15.115:0/436395075 >> 172.21.15.115:0/436395075 conn(0x7f5cdc0ce880 msgr2=0x7f5cdc0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 -- 172.21.15.115:0/436395075 shutdown_connections 2023-12-01T01:52:41.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:41.880+0000 7f5ce11ab640 1 -- 172.21.15.115:0/436395075 wait complete. 2023-12-01T01:52:41.903 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T01:52:42.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.112+0000 7f6c215ea640 1 Processor -- start 2023-12-01T01:52:42.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.112+0000 7f6c215ea640 1 -- start start 2023-12-01T01:52:42.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.114+0000 7f6c215ea640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6c1c160200 0x7f6c1c0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:42.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.114+0000 7f6c215ea640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1c0cb290 con 0x7f6c1c0cbd30 2023-12-01T01:52:42.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.114+0000 7f6c215ea640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1c0b0ae0 con 0x7f6c1c0b0f80 2023-12-01T01:52:42.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.114+0000 7f6c215ea640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1c0b35f0 con 0x7f6c1c160200 2023-12-01T01:52:42.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.114+0000 7f6c1b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6c1c0cbd30 0x7f6c1c0cc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40440/0 (socket says 172.21.15.115:40440) 2023-12-01T01:52:42.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.114+0000 7f6c1b7fe640 1 -- 172.21.15.115:0/3219724202 learned_addr learned my addr 172.21.15.115:0/3219724202 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:42.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.114+0000 7f6c1affd640 1 --2- 172.21.15.115:0/3219724202 >> v2:172.21.15.142:3300/0 conn(0x7f6c1c160200 0x7f6c1c0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:42.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.114+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909925580 0 0) 0x7f6c1c0cb290 con 0x7f6c1c0cbd30 2023-12-01T01:52:42.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.114+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c08003520 con 0x7f6c1c0cbd30 2023-12-01T01:52:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 277220654 0 0) 0x7f6c08003520 con 0x7f6c1c0cbd30 2023-12-01T01:52:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 >> v2:172.21.15.142:3300/0 conn(0x7f6c1c160200 msgr2=0x7f6c1c0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 --2- 172.21.15.115:0/3219724202 >> v2:172.21.15.142:3300/0 conn(0x7f6c1c160200 0x7f6c1c0b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 >> v1:172.21.15.142:6789/0 conn(0x7f6c1c0b0f80 legacy=0x7f6c1c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0b3310 con 0x7f6c1c0cbd30 2023-12-01T01:52:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6c14001670 con 0x7f6c1c0cbd30 2023-12-01T01:52:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 >> v1:172.21.15.115:6789/0 conn(0x7f6c1c0cbd30 legacy=0x7f6c1c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c080039c0 0x7f6c08003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c080047c0 0x7f6c08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:42.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c080051d0 0x7f6c0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0b35f0 con 0x7f6c080039c0 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0800c100 con 0x7f6c080051d0 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0800c2c0 con 0x7f6c080047c0 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.115+0000 7f6c1affd640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c080039c0 0x7f6c08003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.116+0000 7f6c1affd640 1 -- 172.21.15.115:0/3219724202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c080047c0 msgr2=0x7f6c08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.116+0000 7f6c1a7fc640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c080047c0 0x7f6c08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.116+0000 7f6c1b7fe640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c080051d0 0x7f6c0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.116+0000 7f6c1affd640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c080047c0 0x7f6c08004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.116+0000 7f6c1affd640 1 -- 172.21.15.115:0/3219724202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c080051d0 msgr2=0x7f6c0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.116+0000 7f6c1affd640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c080051d0 0x7f6c0800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.116+0000 7f6c1affd640 1 -- 172.21.15.115:0/3219724202 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c08003520 con 0x7f6c080039c0 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.116+0000 7f6c1b7fe640 1 -- 172.21.15.115:0/3219724202 reap_dead start 2023-12-01T01:52:42.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.116+0000 7f6c1a7fc640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c080047c0 0x7f6c08004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:42.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.117+0000 7f6c1affd640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c080039c0 0x7f6c08003dc0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c007330 tx=0x7f6c080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20f1482afdd790e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.117+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c02cca0 con 0x7f6c080039c0 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.117+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c02ce00 con 0x7f6c080039c0 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.117+0000 7f6c19ffb640 1 -- 172.21.15.115:0/3219724202 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c034660 con 0x7f6c080039c0 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.118+0000 7f6c215ea640 1 -- 172.21.15.115:0/3219724202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c080039c0 msgr2=0x7f6c08003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.118+0000 7f6c215ea640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c080039c0 0x7f6c08003dc0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c007330 tx=0x7f6c080057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.118+0000 7f6c215ea640 1 -- 172.21.15.115:0/3219724202 shutdown_connections 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.118+0000 7f6c215ea640 1 --2- 172.21.15.115:0/3219724202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c080039c0 0x7f6c08003dc0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.118+0000 7f6c215ea640 1 -- 172.21.15.115:0/3219724202 >> 172.21.15.115:0/3219724202 conn(0x7f6c1c065960 msgr2=0x7f6c1c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.118+0000 7f6c215ea640 1 -- 172.21.15.115:0/3219724202 shutdown_connections 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.118+0000 7f6c215ea640 1 -- 172.21.15.115:0/3219724202 wait complete. 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.119+0000 7f6c215ea640 1 Processor -- start 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.120+0000 7f6c215ea640 1 -- start start 2023-12-01T01:52:42.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.120+0000 7f6c215ea640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c080039c0 0x7f6c1c1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:42.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.120+0000 7f6c215ea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c1c14f9f0 0x7f6c1c0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:42.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c215ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1c0ca1a0 0x7f6c1c1f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:42.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c215ea640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c1f69a0 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c215ea640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0ca620 con 0x7f6c080039c0 2023-12-01T01:52:42.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c215ea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c153f40 con 0x7f6c1c0ca1a0 2023-12-01T01:52:42.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c1a7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c1c14f9f0 0x7f6c1c0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:42.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c1a7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c1c14f9f0 0x7f6c1c0c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47028/0 (socket says 172.21.15.115:47028) 2023-12-01T01:52:42.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c1a7fc640 1 -- 172.21.15.115:0/283548739 learned_addr learned my addr 172.21.15.115:0/283548739 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:42.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c1affd640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c080039c0 0x7f6c1c1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:42.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c1b7fe640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1c0ca1a0 0x7f6c1c1f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c1a7fc640 1 -- 172.21.15.115:0/283548739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1c0ca1a0 msgr2=0x7f6c1c1f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c1a7fc640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1c0ca1a0 0x7f6c1c1f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c1a7fc640 1 -- 172.21.15.115:0/283548739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c080039c0 msgr2=0x7f6c1c1543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c1a7fc640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c080039c0 0x7f6c1c1543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.121+0000 7f6c1a7fc640 1 -- 172.21.15.115:0/283548739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c1f66c0 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.122+0000 7f6c1a7fc640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c1c14f9f0 0x7f6c1c0c9c60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400d460 tx=0x7f6c0400d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.122+0000 7f6c137fe640 1 -- 172.21.15.115:0/283548739 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04005230 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.122+0000 7f6c137fe640 1 -- 172.21.15.115:0/283548739 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c040053d0 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.122+0000 7f6c137fe640 1 -- 172.21.15.115:0/283548739 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04007260 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.122+0000 7f6c215ea640 1 -- 172.21.15.115:0/283548739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c1538f0 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.122+0000 7f6c215ea640 1 -- 172.21.15.115:0/283548739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c14fe70 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.123+0000 7f6c215ea640 1 -- 172.21.15.115:0/283548739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c150090 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.124+0000 7f6c137fe640 1 -- 172.21.15.115:0/283548739 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6c040073c0 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.126+0000 7f6c137fe640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bfc06a580 0x7f6bfc06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:42.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.126+0000 7f6c137fe640 1 -- 172.21.15.115:0/283548739 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f6c04088540 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.126+0000 7f6c1affd640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bfc06a580 0x7f6bfc06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:42.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.126+0000 7f6c1affd640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bfc06a580 0x7f6bfc06ca40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c036000 tx=0x7f6c0c030000 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:42.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.130+0000 7f6c137fe640 1 -- 172.21.15.115:0/283548739 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6c04053910 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.308+0000 7f6c215ea640 1 -- 172.21.15.115:0/283548739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6c1c04e890 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.315+0000 7f6c137fe640 1 -- 172.21.15.115:0/283548739 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6c04056f30 con 0x7f6c1c14f9f0 2023-12-01T01:52:42.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.317+0000 7f6c215ea640 1 -- 172.21.15.115:0/283548739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bfc06a580 msgr2=0x7f6bfc06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:42.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.317+0000 7f6c215ea640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bfc06a580 0x7f6bfc06ca40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c036000 tx=0x7f6c0c030000 comp rx=0 tx=0).stop 2023-12-01T01:52:42.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.317+0000 7f6c215ea640 1 -- 172.21.15.115:0/283548739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c1c14f9f0 msgr2=0x7f6c1c0c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:42.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.317+0000 7f6c215ea640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c1c14f9f0 0x7f6c1c0c9c60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400d460 tx=0x7f6c0400d920 comp rx=0 tx=0).stop 2023-12-01T01:52:42.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.318+0000 7f6c215ea640 1 -- 172.21.15.115:0/283548739 shutdown_connections 2023-12-01T01:52:42.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.318+0000 7f6c215ea640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bfc06a580 0x7f6bfc06ca40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.318+0000 7f6c215ea640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1c0ca1a0 0x7f6c1c1f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.318+0000 7f6c215ea640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c1c14f9f0 0x7f6c1c0c9c60 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.318+0000 7f6c215ea640 1 --2- 172.21.15.115:0/283548739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c080039c0 0x7f6c1c1543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:42.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.318+0000 7f6c215ea640 1 -- 172.21.15.115:0/283548739 >> 172.21.15.115:0/283548739 conn(0x7f6c1c065960 msgr2=0x7f6c1c06c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:42.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.318+0000 7f6c215ea640 1 -- 172.21.15.115:0/283548739 shutdown_connections 2023-12-01T01:52:42.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:42.318+0000 7f6c215ea640 1 -- 172.21.15.115:0/283548739 wait complete. 2023-12-01T01:52:42.337 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:52:42.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:52:42.454 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:52:42.581 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:52:42.581 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:52:42.581 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:42.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:52:42.645 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:52:42.645 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:42.705 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:52:42.705 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:52:42.705 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:42.763 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:52:42.764 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:52:42.764 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:52:42.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:52:42.824 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T01:52:42.824 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:52:43.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.044+0000 7fc08efd9640 1 Processor -- start 2023-12-01T01:52:43.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.045+0000 7fc08efd9640 1 -- start start 2023-12-01T01:52:43.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.046+0000 7fc08efd9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc08815e2f0 0x7fc0880ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.046+0000 7fc08efd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc08806f2f0 0x7fc0880caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.046+0000 7fc08efd9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0880add10 con 0x7fc0880adf10 2023-12-01T01:52:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.046+0000 7fc08efd9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0880c5760 con 0x7fc08815e2f0 2023-12-01T01:52:43.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.046+0000 7fc08efd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08815e6d0 con 0x7fc08806f2f0 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc08d576640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc0880adf10 0x7fc0880c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40446/0 (socket says 172.21.15.115:40446) 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc08d576640 1 -- 172.21.15.115:0/3182164876 learned_addr learned my addr 172.21.15.115:0/3182164876 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc08cd75640 1 --2- 172.21.15.115:0/3182164876 >> v2:172.21.15.142:3300/0 conn(0x7fc08815e2f0 0x7fc0880ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc087fff640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc08806f2f0 0x7fc0880caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185335012 0 0) 0x7fc0880add10 con 0x7fc0880adf10 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc078003520 con 0x7fc0880adf10 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 762313111 0 0) 0x7fc078003520 con 0x7fc0880adf10 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc08806f2f0 msgr2=0x7fc0880caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc0877fe640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc08806f2f0 0x7fc0880caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 >> v2:172.21.15.142:3300/0 conn(0x7fc08815e2f0 msgr2=0x7fc0880ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc0877fe640 1 --2- 172.21.15.115:0/3182164876 >> v2:172.21.15.142:3300/0 conn(0x7fc08815e2f0 0x7fc0880ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.047+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0880cf7b0 con 0x7fc0880adf10 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc0800019f0 con 0x7fc0880adf10 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 >> v1:172.21.15.115:6789/0 conn(0x7fc0880adf10 legacy=0x7fc0880c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc0877fe640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0780039e0 0x7fc078003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc0877fe640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc08806f2f0 0x7fc0780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc0877fe640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc078004d00 0x7fc078005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0880c5760 con 0x7fc078004d00 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08815e6d0 con 0x7fc0780039e0 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07800bae0 con 0x7fc08806f2f0 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc08d576640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc078004d00 0x7fc078005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc08cd75640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0780039e0 0x7fc078003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc087fff640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc08806f2f0 0x7fc0780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc08d576640 1 -- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc08806f2f0 msgr2=0x7fc0780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc08d576640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc08806f2f0 0x7fc0780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc08d576640 1 -- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0780039e0 msgr2=0x7fc078003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:43.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc08d576640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0780039e0 0x7fc078003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.048+0000 7fc08d576640 1 -- 172.21.15.115:0/3182164876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078003520 con 0x7fc078004d00 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.050+0000 7fc08d576640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc078004d00 0x7fc078005180 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc0880ae370 tx=0x7fc0780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941b9beaf0c6b665 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.050+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080034630 con 0x7fc078004d00 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.050+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08002f070 con 0x7fc078004d00 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.050+0000 7fc0877fe640 1 -- 172.21.15.115:0/3182164876 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800349d0 con 0x7fc078004d00 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.051+0000 7fc08efd9640 1 -- 172.21.15.115:0/3182164876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc078004d00 msgr2=0x7fc078005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.051+0000 7fc08efd9640 1 --2- 172.21.15.115:0/3182164876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc078004d00 0x7fc078005180 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc0880ae370 tx=0x7fc0780057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.051+0000 7fc08d576640 1 -- 172.21.15.115:0/3182164876 reap_dead start 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.051+0000 7fc08efd9640 1 -- 172.21.15.115:0/3182164876 shutdown_connections 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.051+0000 7fc08efd9640 1 -- 172.21.15.115:0/3182164876 >> 172.21.15.115:0/3182164876 conn(0x7fc0880ce880 msgr2=0x7fc0880cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.051+0000 7fc08efd9640 1 -- 172.21.15.115:0/3182164876 shutdown_connections 2023-12-01T01:52:43.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.051+0000 7fc08efd9640 1 -- 172.21.15.115:0/3182164876 wait complete. 2023-12-01T01:52:43.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.053+0000 7fc08efd9640 1 Processor -- start 2023-12-01T01:52:43.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.053+0000 7fc08efd9640 1 -- start start 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08efd9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc078004d00 0x7fc088150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08efd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0880c2650 0x7fc0880c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08efd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881e4600 0x7fc0881e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08efd9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0881e4fc0 con 0x7fc0881e4600 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08efd9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0881511f0 con 0x7fc078004d00 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08efd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0881f16f0 con 0x7fc0880c2650 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08d576640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881e4600 0x7fc0881e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08d576640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881e4600 0x7fc0881e4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47034/0 (socket says 172.21.15.115:47034) 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08d576640 1 -- 172.21.15.115:0/1828893144 learned_addr learned my addr 172.21.15.115:0/1828893144 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08cd75640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc078004d00 0x7fc088150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc087fff640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0880c2650 0x7fc0880c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08d576640 1 -- 172.21.15.115:0/1828893144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0880c2650 msgr2=0x7fc0880c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08d576640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0880c2650 0x7fc0880c2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08d576640 1 -- 172.21.15.115:0/1828893144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc078004d00 msgr2=0x7fc088150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08d576640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc078004d00 0x7fc088150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.054+0000 7fc08d576640 1 -- 172.21.15.115:0/1828893144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0880c3110 con 0x7fc0881e4600 2023-12-01T01:52:43.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.055+0000 7fc08d576640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881e4600 0x7fc0881e4a80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fc08000a870 tx=0x7fc080001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:43.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.055+0000 7fc0857fa640 1 -- 172.21.15.115:0/1828893144 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080001f30 con 0x7fc0881e4600 2023-12-01T01:52:43.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.055+0000 7fc0857fa640 1 -- 172.21.15.115:0/1828893144 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc080034430 con 0x7fc0881e4600 2023-12-01T01:52:43.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.055+0000 7fc0857fa640 1 -- 172.21.15.115:0/1828893144 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800347a0 con 0x7fc0881e4600 2023-12-01T01:52:43.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.055+0000 7fc08efd9640 1 -- 172.21.15.115:0/1828893144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0880c33f0 con 0x7fc0881e4600 2023-12-01T01:52:43.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.055+0000 7fc08efd9640 1 -- 172.21.15.115:0/1828893144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0881f1960 con 0x7fc0881e4600 2023-12-01T01:52:43.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.057+0000 7fc08efd9640 1 -- 172.21.15.115:0/1828893144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08806fda0 con 0x7fc0881e4600 2023-12-01T01:52:43.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.057+0000 7fc0857fa640 1 -- 172.21.15.115:0/1828893144 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc080031030 con 0x7fc0881e4600 2023-12-01T01:52:43.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.058+0000 7fc0857fa640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc06c06a580 0x7fc06c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.058+0000 7fc0857fa640 1 -- 172.21.15.115:0/1828893144 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5545+0+0 (secure 0 0 0) 0x7fc0800aa000 con 0x7fc0881e4600 2023-12-01T01:52:43.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.059+0000 7fc08cd75640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc06c06a580 0x7fc06c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.059+0000 7fc08cd75640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc06c06a580 0x7fc06c06ca40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009a10 tx=0x7fc07c008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:43.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.062+0000 7fc0857fa640 1 -- 172.21.15.115:0/1828893144 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc080075500 con 0x7fc0881e4600 2023-12-01T01:52:43.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.291+0000 7fc08efd9640 1 -- 172.21.15.115:0/1828893144 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc08804e890 con 0x7fc0881e4600 2023-12-01T01:52:43.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.292+0000 7fc0857fa640 1 -- 172.21.15.115:0/1828893144 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fc080045090 con 0x7fc0881e4600 2023-12-01T01:52:43.294 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:43.294 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5354,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1559849716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1559849716},{"type":"v1","addr":"172.21.15.142:6835","nonce":1559849716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/542800961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":542800961},{"type":"v1","addr":"172.21.15.142:6833","nonce":542800961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6942,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/2035657957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2035657957},{"type":"v1","addr":"172.21.15.115:6835","nonce":2035657957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3644451216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3644451216},{"type":"v1","addr":"172.21.15.115:6837","nonce":3644451216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[]} 2023-12-01T01:52:43.294 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 42 2023-12-01T01:52:43.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 -- 172.21.15.115:0/1828893144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc06c06a580 msgr2=0x7fc06c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc06c06a580 0x7fc06c06ca40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009a10 tx=0x7fc07c008040 comp rx=0 tx=0).stop 2023-12-01T01:52:43.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 -- 172.21.15.115:0/1828893144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881e4600 msgr2=0x7fc0881e4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881e4600 0x7fc0881e4a80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fc08000a870 tx=0x7fc080001360 comp rx=0 tx=0).stop 2023-12-01T01:52:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 -- 172.21.15.115:0/1828893144 shutdown_connections 2023-12-01T01:52:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc06c06a580 0x7fc06c06ca40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0880c2650 0x7fc0880c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0881e4600 0x7fc0881e4a80 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 --2- 172.21.15.115:0/1828893144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc078004d00 0x7fc088150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 -- 172.21.15.115:0/1828893144 >> 172.21.15.115:0/1828893144 conn(0x7fc0880ce880 msgr2=0x7fc08815d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 -- 172.21.15.115:0/1828893144 shutdown_connections 2023-12-01T01:52:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.295+0000 7fc08efd9640 1 -- 172.21.15.115:0/1828893144 wait complete. 2023-12-01T01:52:43.314 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T01:52:43.314 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T01:52:43.314 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T01:52:43.314 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:52:43.314 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T01:52:43.314 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T01:52:43.315 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T01:52:43.315 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:52:43.315 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T01:52:43.315 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T01:52:43.315 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T01:52:43.315 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:52:43.315 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T01:52:43.315 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T01:52:43.316 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T01:52:43.316 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:52:43.328 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T01:52:43.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T01:52:43.334 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T01:52:43.334 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T01:52:43.338 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T01:52:43.338 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T01:52:43.342 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T01:52:43.342 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T01:52:43.382 INFO:tasks.ceph.mds.a:Started 2023-12-01T01:52:43.382 DEBUG:teuthology.parallel:result is None 2023-12-01T01:52:43.389 INFO:tasks.ceph.mds.c:Started 2023-12-01T01:52:43.390 DEBUG:teuthology.parallel:result is None 2023-12-01T01:52:43.390 INFO:tasks.ceph.mds.d:Started 2023-12-01T01:52:43.391 DEBUG:teuthology.parallel:result is None 2023-12-01T01:52:43.398 INFO:tasks.ceph.mds.b:Started 2023-12-01T01:52:43.398 DEBUG:teuthology.parallel:result is None 2023-12-01T01:52:43.398 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T01:52:43.542 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T01:52:43.547 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:52:43.559 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T01:52:43.559 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T01:52:43.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.597+0000 7f391800b640 1 Processor -- start 2023-12-01T01:52:43.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.597+0000 7f391800b640 1 -- start start 2023-12-01T01:52:43.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.598+0000 7f391800b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39100d1c90 0x7f39100b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.598+0000 7f391800b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3910160b20 con 0x7f391015dfd0 2023-12-01T01:52:43.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.598+0000 7f391800b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3910160dc0 con 0x7f391015dbf0 2023-12-01T01:52:43.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.598+0000 7f391800b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39101610a0 con 0x7f39100d1c90 2023-12-01T01:52:43.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.598+0000 7f39155a6640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f391015dfd0 0x7f39100cf460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40464/0 (socket says 172.21.15.115:40464) 2023-12-01T01:52:43.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.598+0000 7f39155a6640 1 -- 172.21.15.115:0/3883445990 learned_addr learned my addr 172.21.15.115:0/3883445990 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:43.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.599+0000 7f39165a8640 1 --2- 172.21.15.115:0/3883445990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39100d1c90 0x7f39100b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.599+0000 7f3914da5640 1 -- 172.21.15.115:0/3883445990 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372017978 0 0) 0x7f3910160b20 con 0x7f391015dfd0 2023-12-01T01:52:43.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.599+0000 7f3914da5640 1 -- 172.21.15.115:0/3883445990 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38fc003520 con 0x7f391015dfd0 2023-12-01T01:52:43.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.599+0000 7f3914da5640 1 -- 172.21.15.115:0/3883445990 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797941024 0 0) 0x7f3910160dc0 con 0x7f391015dbf0 2023-12-01T01:52:43.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.599+0000 7f3914da5640 1 -- 172.21.15.115:0/3883445990 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3910160b20 con 0x7f391015dbf0 2023-12-01T01:52:43.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.599+0000 7f39165a8640 1 -- 172.21.15.115:0/3883445990 >> v1:172.21.15.142:6789/0 conn(0x7f391015dbf0 legacy=0x7f39100cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.599+0000 7f39165a8640 1 -- 172.21.15.115:0/3883445990 >> v1:172.21.15.115:6789/0 conn(0x7f391015dfd0 legacy=0x7f39100cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.599+0000 7f39165a8640 1 -- 172.21.15.115:0/3883445990 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3910161380 con 0x7f39100d1c90 2023-12-01T01:52:43.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.600+0000 7f39165a8640 1 --2- 172.21.15.115:0/3883445990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39100d1c90 0x7f39100b7ef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3908009dc0 tx=0x7f390802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b65ce47640228d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:43.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.601+0000 7f3914da5640 1 -- 172.21.15.115:0/3883445990 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390802ccd0 con 0x7f39100d1c90 2023-12-01T01:52:43.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.601+0000 7f3914da5640 1 -- 172.21.15.115:0/3883445990 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390802ce30 con 0x7f39100d1c90 2023-12-01T01:52:43.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.601+0000 7f3914da5640 1 -- 172.21.15.115:0/3883445990 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908034660 con 0x7f39100d1c90 2023-12-01T01:52:43.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.601+0000 7f391800b640 1 -- 172.21.15.115:0/3883445990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39100d1c90 msgr2=0x7f39100b7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.601+0000 7f391800b640 1 --2- 172.21.15.115:0/3883445990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39100d1c90 0x7f39100b7ef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3908009dc0 tx=0x7f390802bf20 comp rx=0 tx=0).stop 2023-12-01T01:52:43.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.601+0000 7f391800b640 1 -- 172.21.15.115:0/3883445990 shutdown_connections 2023-12-01T01:52:43.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.601+0000 7f391800b640 1 --2- 172.21.15.115:0/3883445990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39100d1c90 0x7f39100b7ef0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.601+0000 7f391800b640 1 -- 172.21.15.115:0/3883445990 >> 172.21.15.115:0/3883445990 conn(0x7f3910065960 msgr2=0x7f3910065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:43.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.601+0000 7f391800b640 1 -- 172.21.15.115:0/3883445990 shutdown_connections 2023-12-01T01:52:43.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.601+0000 7f391800b640 1 -- 172.21.15.115:0/3883445990 wait complete. 2023-12-01T01:52:43.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.603+0000 7f391800b640 1 Processor -- start 2023-12-01T01:52:43.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.603+0000 7f391800b640 1 -- start start 2023-12-01T01:52:43.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f391800b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39100d1c90 0x7f3910152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f391800b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f391015dbf0 0x7f3910153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f391800b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f391015dfd0 0x7f39101542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f391800b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3910157ce0 con 0x7f391015dfd0 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f391800b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100cc4a0 con 0x7f39100d1c90 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f391800b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100cc780 con 0x7f391015dbf0 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f39165a8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f391015dfd0 0x7f39101542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f39165a8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f391015dfd0 0x7f39101542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47056/0 (socket says 172.21.15.115:47056) 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f39165a8640 1 -- 172.21.15.115:0/445368224 learned_addr learned my addr 172.21.15.115:0/445368224 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f39155a6640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f391015dbf0 0x7f3910153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.604+0000 7f3915da7640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39100d1c90 0x7f3910152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.605+0000 7f39165a8640 1 -- 172.21.15.115:0/445368224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f391015dbf0 msgr2=0x7f3910153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.605+0000 7f39165a8640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f391015dbf0 0x7f3910153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.605+0000 7f39165a8640 1 -- 172.21.15.115:0/445368224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39100d1c90 msgr2=0x7f3910152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.605+0000 7f39165a8640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39100d1c90 0x7f3910152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.605+0000 7f39165a8640 1 -- 172.21.15.115:0/445368224 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3908009a70 con 0x7f391015dfd0 2023-12-01T01:52:43.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.605+0000 7f39165a8640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f391015dfd0 0x7f39101542f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f390802dfd0 tx=0x7f3908004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:43.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.605+0000 7f39067fc640 1 -- 172.21.15.115:0/445368224 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390803d600 con 0x7f391015dfd0 2023-12-01T01:52:43.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.606+0000 7f39067fc640 1 -- 172.21.15.115:0/445368224 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390803d760 con 0x7f391015dfd0 2023-12-01T01:52:43.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.606+0000 7f39067fc640 1 -- 172.21.15.115:0/445368224 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390803db10 con 0x7f391015dfd0 2023-12-01T01:52:43.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.606+0000 7f391800b640 1 -- 172.21.15.115:0/445368224 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39100cca60 con 0x7f391015dfd0 2023-12-01T01:52:43.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.606+0000 7f391800b640 1 -- 172.21.15.115:0/445368224 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3910152940 con 0x7f391015dfd0 2023-12-01T01:52:43.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.607+0000 7f391800b640 1 -- 172.21.15.115:0/445368224 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3910157ee0 con 0x7f391015dfd0 2023-12-01T01:52:43.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.608+0000 7f39067fc640 1 -- 172.21.15.115:0/445368224 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f390803b040 con 0x7f391015dfd0 2023-12-01T01:52:43.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.609+0000 7f39067fc640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f38f406a550 0x7f38f406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:43.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.609+0000 7f39067fc640 1 -- 172.21.15.115:0/445368224 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f39080aad70 con 0x7f391015dfd0 2023-12-01T01:52:43.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.609+0000 7f3915da7640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f38f406a550 0x7f38f406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:43.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.610+0000 7f3915da7640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f38f406a550 0x7f38f406ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f390c0011e0 tx=0x7f390c009040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.613+0000 7f39067fc640 1 -- 172.21.15.115:0/445368224 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3908076390 con 0x7f391015dfd0 2023-12-01T01:52:43.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:43.817+0000 7f391800b640 1 -- 172.21.15.115:0/445368224 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f391004e890 con 0x7f391015dfd0 2023-12-01T01:52:44.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.593+0000 7f39067fc640 1 -- 172.21.15.115:0/445368224 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v107) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3908039080 con 0x7f391015dfd0 2023-12-01T01:52:44.595 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T01:52:44.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.596+0000 7f391800b640 1 -- 172.21.15.115:0/445368224 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f38f406a550 msgr2=0x7f38f406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:44.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.596+0000 7f391800b640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f38f406a550 0x7f38f406ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f390c0011e0 tx=0x7f390c009040 comp rx=0 tx=0).stop 2023-12-01T01:52:44.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.596+0000 7f391800b640 1 -- 172.21.15.115:0/445368224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f391015dfd0 msgr2=0x7f39101542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:44.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.596+0000 7f391800b640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f391015dfd0 0x7f39101542f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f390802dfd0 tx=0x7f3908004a40 comp rx=0 tx=0).stop 2023-12-01T01:52:44.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.597+0000 7f391800b640 1 -- 172.21.15.115:0/445368224 shutdown_connections 2023-12-01T01:52:44.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.597+0000 7f391800b640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f38f406a550 0x7f38f406ca10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:44.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.597+0000 7f391800b640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f391015dbf0 0x7f3910153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:44.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.597+0000 7f391800b640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f391015dfd0 0x7f39101542f0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:44.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.597+0000 7f391800b640 1 --2- 172.21.15.115:0/445368224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39100d1c90 0x7f3910152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:44.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.597+0000 7f391800b640 1 -- 172.21.15.115:0/445368224 >> 172.21.15.115:0/445368224 conn(0x7f3910065960 msgr2=0x7f39100b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:44.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.597+0000 7f391800b640 1 -- 172.21.15.115:0/445368224 shutdown_connections 2023-12-01T01:52:44.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.597+0000 7f391800b640 1 -- 172.21.15.115:0/445368224 wait complete. 2023-12-01T01:52:44.618 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T01:52:44.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.835+0000 7f4e460e0640 1 Processor -- start 2023-12-01T01:52:44.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.835+0000 7f4e460e0640 1 -- start start 2023-12-01T01:52:44.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.837+0000 7f4e460e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e400cad70 0x7f4e4006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:44.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.837+0000 7f4e460e0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4e4006f2f0 0x7f4e4006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:44.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.837+0000 7f4e460e0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e400add10 con 0x7f4e4015e2f0 2023-12-01T01:52:44.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.837+0000 7f4e460e0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e400c5760 con 0x7f4e4006f2f0 2023-12-01T01:52:44.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.837+0000 7f4e460e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4015e6d0 con 0x7f4e400cad70 2023-12-01T01:52:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.837+0000 7f4e3ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4e4006f2f0 0x7f4e4006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.837+0000 7f4e37fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e400cad70 0x7f4e4006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.838+0000 7f4e3ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4e4006f2f0 0x7f4e4006bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:52900/0 (socket says 172.21.15.115:52900) 2023-12-01T01:52:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.838+0000 7f4e3ffff640 1 -- 172.21.15.115:0/1650233508 learned_addr learned my addr 172.21.15.115:0/1650233508 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:44.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.838+0000 7f4e3f7fe640 1 --1- 172.21.15.115:0/1650233508 >> v1:172.21.15.142:6789/0 conn(0x7f4e4015e2f0 0x7f4e400ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:55944/0 (socket says 172.21.15.115:55944) 2023-12-01T01:52:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.838+0000 7f4e3ffff640 1 -- 172.21.15.115:0/1650233508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e400cad70 msgr2=0x7f4e4006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.838+0000 7f4e3ffff640 1 --2- 172.21.15.115:0/1650233508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e400cad70 0x7f4e4006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.838+0000 7f4e3ffff640 1 -- 172.21.15.115:0/1650233508 >> v1:172.21.15.142:6789/0 conn(0x7f4e4015e2f0 legacy=0x7f4e400ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:44.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.838+0000 7f4e3ffff640 1 -- 172.21.15.115:0/1650233508 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e400cf7b0 con 0x7f4e4006f2f0 2023-12-01T01:52:44.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.840+0000 7f4e3ffff640 1 --2- 172.21.15.115:0/1650233508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4006f2f0 0x7f4e4006bb00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009e50 tx=0x7f4e3802be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df1e1a44e62b702 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:44.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.840+0000 7f4e3effd640 1 -- 172.21.15.115:0/1650233508 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3802c9f0 con 0x7f4e4006f2f0 2023-12-01T01:52:44.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.840+0000 7f4e3effd640 1 -- 172.21.15.115:0/1650233508 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3802cb50 con 0x7f4e4006f2f0 2023-12-01T01:52:44.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.840+0000 7f4e3effd640 1 -- 172.21.15.115:0/1650233508 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380343f0 con 0x7f4e4006f2f0 2023-12-01T01:52:44.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.841+0000 7f4e460e0640 1 -- 172.21.15.115:0/1650233508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4006f2f0 msgr2=0x7f4e4006bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:44.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.841+0000 7f4e460e0640 1 --2- 172.21.15.115:0/1650233508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4006f2f0 0x7f4e4006bb00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009e50 tx=0x7f4e3802be50 comp rx=0 tx=0).stop 2023-12-01T01:52:44.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.841+0000 7f4e460e0640 1 -- 172.21.15.115:0/1650233508 shutdown_connections 2023-12-01T01:52:44.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.841+0000 7f4e460e0640 1 --2- 172.21.15.115:0/1650233508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e400cad70 0x7f4e4006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:44.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.841+0000 7f4e460e0640 1 --2- 172.21.15.115:0/1650233508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4006f2f0 0x7f4e4006bb00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:44.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.841+0000 7f4e460e0640 1 -- 172.21.15.115:0/1650233508 >> 172.21.15.115:0/1650233508 conn(0x7f4e400ce880 msgr2=0x7f4e400cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:44.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.841+0000 7f4e460e0640 1 -- 172.21.15.115:0/1650233508 shutdown_connections 2023-12-01T01:52:44.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.841+0000 7f4e460e0640 1 -- 172.21.15.115:0/1650233508 wait complete. 2023-12-01T01:52:44.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.843+0000 7f4e460e0640 1 Processor -- start 2023-12-01T01:52:44.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.843+0000 7f4e460e0640 1 -- start start 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.844+0000 7f4e460e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4006f2f0 0x7f4e4015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.844+0000 7f4e460e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e400cad70 0x7f4e400c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.844+0000 7f4e460e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4015e2f0 0x7f4e400c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.844+0000 7f4e460e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400c8a70 con 0x7f4e400cad70 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.844+0000 7f4e460e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e401f2310 con 0x7f4e4006f2f0 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.844+0000 7f4e460e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4015aed0 con 0x7f4e4015e2f0 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.844+0000 7f4e37fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e400cad70 0x7f4e400c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.844+0000 7f4e37fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e400cad70 0x7f4e400c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47070/0 (socket says 172.21.15.115:47070) 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.844+0000 7f4e37fff640 1 -- 172.21.15.115:0/2668385834 learned_addr learned my addr 172.21.15.115:0/2668385834 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e3ffff640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4015e2f0 0x7f4e400c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e3f7fe640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4006f2f0 0x7f4e4015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:44.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e37fff640 1 -- 172.21.15.115:0/2668385834 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4015e2f0 msgr2=0x7f4e400c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e37fff640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4015e2f0 0x7f4e400c4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e37fff640 1 -- 172.21.15.115:0/2668385834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4006f2f0 msgr2=0x7f4e4015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e37fff640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4006f2f0 0x7f4e4015a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e37fff640 1 -- 172.21.15.115:0/2668385834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38009b30 con 0x7f4e400cad70 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e3f7fe640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4006f2f0 0x7f4e4015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e37fff640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e400cad70 0x7f4e400c5ed0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800e8a0 tx=0x7f4e2800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e3cff9640 1 -- 172.21.15.115:0/2668385834 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e280094f0 con 0x7f4e400cad70 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e3cff9640 1 -- 172.21.15.115:0/2668385834 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2800c940 con 0x7f4e400cad70 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e3cff9640 1 -- 172.21.15.115:0/2668385834 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800ccf0 con 0x7f4e400cad70 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.845+0000 7f4e460e0640 1 -- 172.21.15.115:0/2668385834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e400c7b70 con 0x7f4e400cad70 2023-12-01T01:52:44.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.846+0000 7f4e460e0640 1 -- 172.21.15.115:0/2668385834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e400c83b0 con 0x7f4e400cad70 2023-12-01T01:52:44.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.847+0000 7f4e460e0640 1 -- 172.21.15.115:0/2668385834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4006fda0 con 0x7f4e400cad70 2023-12-01T01:52:44.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.847+0000 7f4e3cff9640 1 -- 172.21.15.115:0/2668385834 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4e28013470 con 0x7f4e400cad70 2023-12-01T01:52:44.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.848+0000 7f4e3cff9640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e2006a550 0x7f4e2006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:44.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.848+0000 7f4e3cff9640 1 -- 172.21.15.115:0/2668385834 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4e280887e0 con 0x7f4e400cad70 2023-12-01T01:52:44.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.849+0000 7f4e3f7fe640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e2006a550 0x7f4e2006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:44.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.849+0000 7f4e3f7fe640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e2006a550 0x7f4e2006ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e30000f60 tx=0x7f4e30009040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:44.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:44.853+0000 7f4e3cff9640 1 -- 172.21.15.115:0/2668385834 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4e28054010 con 0x7f4e400cad70 2023-12-01T01:52:45.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.109+0000 7f4e460e0640 1 -- 172.21.15.115:0/2668385834 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4e400c8bb0 con 0x7f4e400cad70 2023-12-01T01:52:45.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.110+0000 7f4e3cff9640 1 -- 172.21.15.115:0/2668385834 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4e28017090 con 0x7f4e400cad70 2023-12-01T01:52:45.111 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:45.111 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:52:45.111 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T01:52:45.112 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:45.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.114 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.115 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:45.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T01:52:45.119 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T01:52:45.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.121 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T01:52:45.122 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:52:45.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:52:45.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:45.125 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T01:52:45.125 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T01:52:45.125 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:52:45.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:52:45.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:52:45.125 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:52:45.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:45.125 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T01:52:45.125 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:52:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.112+0000 7f4e460e0640 1 -- 172.21.15.115:0/2668385834 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e2006a550 msgr2=0x7f4e2006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.112+0000 7f4e460e0640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e2006a550 0x7f4e2006ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e30000f60 tx=0x7f4e30009040 comp rx=0 tx=0).stop 2023-12-01T01:52:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.112+0000 7f4e460e0640 1 -- 172.21.15.115:0/2668385834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e400cad70 msgr2=0x7f4e400c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.112+0000 7f4e460e0640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e400cad70 0x7f4e400c5ed0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800e8a0 tx=0x7f4e2800ed60 comp rx=0 tx=0).stop 2023-12-01T01:52:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.112+0000 7f4e460e0640 1 -- 172.21.15.115:0/2668385834 shutdown_connections 2023-12-01T01:52:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.112+0000 7f4e460e0640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e2006a550 0x7f4e2006ca10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.112+0000 7f4e460e0640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4015e2f0 0x7f4e400c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.112+0000 7f4e460e0640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e400cad70 0x7f4e400c5ed0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.112+0000 7f4e460e0640 1 --2- 172.21.15.115:0/2668385834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4006f2f0 0x7f4e4015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.112+0000 7f4e460e0640 1 -- 172.21.15.115:0/2668385834 >> 172.21.15.115:0/2668385834 conn(0x7f4e400ce880 msgr2=0x7f4e4006b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.113+0000 7f4e460e0640 1 -- 172.21.15.115:0/2668385834 shutdown_connections 2023-12-01T01:52:45.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.113+0000 7f4e460e0640 1 -- 172.21.15.115:0/2668385834 wait complete. 2023-12-01T01:52:45.133 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:45.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.341+0000 7f1f45021640 1 Processor -- start 2023-12-01T01:52:45.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.341+0000 7f1f45021640 1 -- start start 2023-12-01T01:52:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.343+0000 7f1f45021640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1f40159dd0 0x7f1f400c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.343+0000 7f1f45021640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1f40070d30 0x7f1f400c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.343+0000 7f1f45021640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4006f8b0 con 0x7f1f4006b6f0 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.343+0000 7f1f45021640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f400adcb0 con 0x7f1f40159dd0 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.343+0000 7f1f45021640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f400ad9d0 con 0x7f1f40070d30 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.343+0000 7f1f3ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1f40159dd0 0x7f1f400c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.343+0000 7f1f3ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1f40159dd0 0x7f1f400c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47078/0 (socket says 172.21.15.115:47078) 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.343+0000 7f1f3ed9d640 1 -- 172.21.15.115:0/3200033010 learned_addr learned my addr 172.21.15.115:0/3200033010 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.343+0000 7f1f3e59c640 1 --1- 172.21.15.115:0/3200033010 >> v1:172.21.15.115:6789/0 conn(0x7f1f4006b6f0 0x7f1f400cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40476/0 (socket says 172.21.15.115:40476) 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.344+0000 7f1f3f59e640 1 --2- 172.21.15.115:0/3200033010 >> v2:172.21.15.142:3300/0 conn(0x7f1f40070d30 0x7f1f400c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.344+0000 7f1f3ed9d640 1 -- 172.21.15.115:0/3200033010 >> v2:172.21.15.142:3300/0 conn(0x7f1f40070d30 msgr2=0x7f1f400c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.344+0000 7f1f3ed9d640 1 --2- 172.21.15.115:0/3200033010 >> v2:172.21.15.142:3300/0 conn(0x7f1f40070d30 0x7f1f400c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.344+0000 7f1f3ed9d640 1 -- 172.21.15.115:0/3200033010 >> v1:172.21.15.115:6789/0 conn(0x7f1f4006b6f0 legacy=0x7f1f400cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.344+0000 7f1f3ed9d640 1 -- 172.21.15.115:0/3200033010 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f400c9000 con 0x7f1f40159dd0 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.344+0000 7f1f3dd9b640 1 -- 172.21.15.115:0/3200033010 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385766951 0 0) 0x7f1f4006f8b0 con 0x7f1f4006b6f0 2023-12-01T01:52:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.344+0000 7f1f3f59e640 1 --2- 172.21.15.115:0/3200033010 >> v2:172.21.15.142:3300/0 conn(0x7f1f40070d30 0x7f1f400c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:45.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.345+0000 7f1f3ed9d640 1 --2- 172.21.15.115:0/3200033010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40159dd0 0x7f1f400c97e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1f4006f280 tx=0x7f1f3002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cae0732b821a4e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.345+0000 7f1f3dd9b640 1 -- 172.21.15.115:0/3200033010 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f300317e0 con 0x7f1f40159dd0 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.345+0000 7f1f3dd9b640 1 -- 172.21.15.115:0/3200033010 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f30031940 con 0x7f1f40159dd0 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.345+0000 7f1f3dd9b640 1 -- 172.21.15.115:0/3200033010 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30031cd0 con 0x7f1f40159dd0 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.346+0000 7f1f45021640 1 -- 172.21.15.115:0/3200033010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40159dd0 msgr2=0x7f1f400c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.346+0000 7f1f45021640 1 --2- 172.21.15.115:0/3200033010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40159dd0 0x7f1f400c97e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1f4006f280 tx=0x7f1f3002b880 comp rx=0 tx=0).stop 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.346+0000 7f1f45021640 1 -- 172.21.15.115:0/3200033010 shutdown_connections 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.346+0000 7f1f45021640 1 --2- 172.21.15.115:0/3200033010 >> v2:172.21.15.142:3300/0 conn(0x7f1f40070d30 0x7f1f400c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.346+0000 7f1f45021640 1 --2- 172.21.15.115:0/3200033010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40159dd0 0x7f1f400c97e0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.346+0000 7f1f45021640 1 -- 172.21.15.115:0/3200033010 >> 172.21.15.115:0/3200033010 conn(0x7f1f400ce880 msgr2=0x7f1f400cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.346+0000 7f1f45021640 1 -- 172.21.15.115:0/3200033010 shutdown_connections 2023-12-01T01:52:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.346+0000 7f1f45021640 1 -- 172.21.15.115:0/3200033010 wait complete. 2023-12-01T01:52:45.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.348+0000 7f1f45021640 1 Processor -- start 2023-12-01T01:52:45.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.348+0000 7f1f45021640 1 -- start start 2023-12-01T01:52:45.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.349+0000 7f1f45021640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f4006b6f0 0x7f1f4014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.349+0000 7f1f45021640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40070d30 0x7f1f400c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.349+0000 7f1f45021640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f40159dd0 0x7f1f400c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.349+0000 7f1f45021640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f400c6670 con 0x7f1f40070d30 2023-12-01T01:52:45.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.349+0000 7f1f45021640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f400c5a60 con 0x7f1f4006b6f0 2023-12-01T01:52:45.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.349+0000 7f1f45021640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f400c5d40 con 0x7f1f40159dd0 2023-12-01T01:52:45.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.349+0000 7f1f3e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40070d30 0x7f1f400c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:45.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.349+0000 7f1f3e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40070d30 0x7f1f400c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47092/0 (socket says 172.21.15.115:47092) 2023-12-01T01:52:45.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.349+0000 7f1f3e59c640 1 -- 172.21.15.115:0/2719905578 learned_addr learned my addr 172.21.15.115:0/2719905578 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.350+0000 7f1f3e59c640 1 -- 172.21.15.115:0/2719905578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f40159dd0 msgr2=0x7f1f400c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.350+0000 7f1f3e59c640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f40159dd0 0x7f1f400c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.350+0000 7f1f3e59c640 1 -- 172.21.15.115:0/2719905578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f4006b6f0 msgr2=0x7f1f4014c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.350+0000 7f1f3e59c640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f4006b6f0 0x7f1f4014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.350+0000 7f1f3e59c640 1 -- 172.21.15.115:0/2719905578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f300094d0 con 0x7f1f40070d30 2023-12-01T01:52:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.350+0000 7f1f3e59c640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40070d30 0x7f1f400c6b00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400c4d0 tx=0x7f1f3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.350+0000 7f1f277fe640 1 -- 172.21.15.115:0/2719905578 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f340074e0 con 0x7f1f40070d30 2023-12-01T01:52:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.350+0000 7f1f277fe640 1 -- 172.21.15.115:0/2719905578 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f34007660 con 0x7f1f40070d30 2023-12-01T01:52:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.350+0000 7f1f277fe640 1 -- 172.21.15.115:0/2719905578 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34004a20 con 0x7f1f40070d30 2023-12-01T01:52:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.350+0000 7f1f45021640 1 -- 172.21.15.115:0/2719905578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f400c6020 con 0x7f1f40070d30 2023-12-01T01:52:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.351+0000 7f1f45021640 1 -- 172.21.15.115:0/2719905578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4014c950 con 0x7f1f40070d30 2023-12-01T01:52:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.352+0000 7f1f45021640 1 -- 172.21.15.115:0/2719905578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4014cb70 con 0x7f1f40070d30 2023-12-01T01:52:45.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.353+0000 7f1f277fe640 1 -- 172.21.15.115:0/2719905578 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1f34004b80 con 0x7f1f40070d30 2023-12-01T01:52:45.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.353+0000 7f1f277fe640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f2c06a550 0x7f1f2c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.354+0000 7f1f277fe640 1 -- 172.21.15.115:0/2719905578 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1f340878f0 con 0x7f1f40070d30 2023-12-01T01:52:45.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.354+0000 7f1f3ed9d640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f2c06a550 0x7f1f2c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:45.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.355+0000 7f1f3ed9d640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f2c06a550 0x7f1f2c06ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1f30036000 tx=0x7f1f30030000 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.358+0000 7f1f277fe640 1 -- 172.21.15.115:0/2719905578 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1f34053120 con 0x7f1f40070d30 2023-12-01T01:52:45.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.549+0000 7f1f45021640 1 -- 172.21.15.115:0/2719905578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f4004e890 con 0x7f1f40070d30 2023-12-01T01:52:45.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.551+0000 7f1f277fe640 1 -- 172.21.15.115:0/2719905578 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+12439 (secure 0 0 0) 0x7f1f34018020 con 0x7f1f40070d30 2023-12-01T01:52:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:45.552 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":107,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:44.586079+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:45.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.553+0000 7f1f45021640 1 -- 172.21.15.115:0/2719905578 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f2c06a550 msgr2=0x7f1f2c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.553+0000 7f1f45021640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f2c06a550 0x7f1f2c06ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1f30036000 tx=0x7f1f30030000 comp rx=0 tx=0).stop 2023-12-01T01:52:45.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.553+0000 7f1f45021640 1 -- 172.21.15.115:0/2719905578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40070d30 msgr2=0x7f1f400c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.553+0000 7f1f45021640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40070d30 0x7f1f400c6b00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400c4d0 tx=0x7f1f3400c990 comp rx=0 tx=0).stop 2023-12-01T01:52:45.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.554+0000 7f1f45021640 1 -- 172.21.15.115:0/2719905578 shutdown_connections 2023-12-01T01:52:45.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.554+0000 7f1f45021640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f2c06a550 0x7f1f2c06ca10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.554+0000 7f1f45021640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f40159dd0 0x7f1f400c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.554+0000 7f1f45021640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f40070d30 0x7f1f400c6b00 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.554+0000 7f1f45021640 1 --2- 172.21.15.115:0/2719905578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f4006b6f0 0x7f1f4014c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.554+0000 7f1f45021640 1 -- 172.21.15.115:0/2719905578 >> 172.21.15.115:0/2719905578 conn(0x7f1f400ce880 msgr2=0x7f1f40152710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:45.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.554+0000 7f1f45021640 1 -- 172.21.15.115:0/2719905578 shutdown_connections 2023-12-01T01:52:45.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.554+0000 7f1f45021640 1 -- 172.21.15.115:0/2719905578 wait complete. 2023-12-01T01:52:45.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:52:45.574 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:52:45.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.794+0000 7fef16f9c640 1 Processor -- start 2023-12-01T01:52:45.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.794+0000 7fef16f9c640 1 -- start start 2023-12-01T01:52:45.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.796+0000 7fef16f9c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fef10160200 0x7fef100b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.796+0000 7fef16f9c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef100b0f80 0x7fef1006b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.796+0000 7fef16f9c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef100cb290 con 0x7fef1006b7a0 2023-12-01T01:52:45.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.796+0000 7fef16f9c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef100b35f0 con 0x7fef10160200 2023-12-01T01:52:45.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.796+0000 7fef16f9c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef100b0ae0 con 0x7fef100b0f80 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.797+0000 7fef15539640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fef1006b7a0 0x7fef100cbd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40486/0 (socket says 172.21.15.115:40486) 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.797+0000 7fef15539640 1 -- 172.21.15.115:0/2878930596 learned_addr learned my addr 172.21.15.115:0/2878930596 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.797+0000 7fef0ffff640 1 --2- 172.21.15.115:0/2878930596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef100b0f80 0x7fef1006b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.797+0000 7fef14d38640 1 --2- 172.21.15.115:0/2878930596 >> v2:172.21.15.142:3300/0 conn(0x7fef10160200 0x7fef100b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.797+0000 7fef0f7fe640 1 -- 172.21.15.115:0/2878930596 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714518918 0 0) 0x7fef100cb290 con 0x7fef1006b7a0 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.797+0000 7fef0f7fe640 1 -- 172.21.15.115:0/2878930596 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef00003520 con 0x7fef1006b7a0 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.797+0000 7fef0f7fe640 1 -- 172.21.15.115:0/2878930596 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3531934065 0 0) 0x7fef00003520 con 0x7fef1006b7a0 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.798+0000 7fef0ffff640 1 -- 172.21.15.115:0/2878930596 >> v2:172.21.15.142:3300/0 conn(0x7fef10160200 msgr2=0x7fef100b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.798+0000 7fef0ffff640 1 --2- 172.21.15.115:0/2878930596 >> v2:172.21.15.142:3300/0 conn(0x7fef10160200 0x7fef100b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.798+0000 7fef0ffff640 1 -- 172.21.15.115:0/2878930596 >> v1:172.21.15.115:6789/0 conn(0x7fef1006b7a0 legacy=0x7fef100cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.798+0000 7fef0ffff640 1 -- 172.21.15.115:0/2878930596 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef100b3310 con 0x7fef100b0f80 2023-12-01T01:52:45.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.799+0000 7fef0ffff640 1 --2- 172.21.15.115:0/2878930596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef100b0f80 0x7fef1006b260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feefc0097a0 tx=0x7feefc02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b4d52bf55b4444a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:45.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.799+0000 7fef0f7fe640 1 -- 172.21.15.115:0/2878930596 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef080019f0 con 0x7fef100b0f80 2023-12-01T01:52:45.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.800+0000 7fef0f7fe640 1 -- 172.21.15.115:0/2878930596 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc02e050 con 0x7fef100b0f80 2023-12-01T01:52:45.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.800+0000 7fef0f7fe640 1 -- 172.21.15.115:0/2878930596 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc030710 con 0x7fef100b0f80 2023-12-01T01:52:45.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.800+0000 7fef16f9c640 1 -- 172.21.15.115:0/2878930596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef100b0f80 msgr2=0x7fef1006b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.800+0000 7fef16f9c640 1 --2- 172.21.15.115:0/2878930596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef100b0f80 0x7fef1006b260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feefc0097a0 tx=0x7feefc02b790 comp rx=0 tx=0).stop 2023-12-01T01:52:45.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.800+0000 7fef16f9c640 1 -- 172.21.15.115:0/2878930596 shutdown_connections 2023-12-01T01:52:45.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.800+0000 7fef16f9c640 1 --2- 172.21.15.115:0/2878930596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef100b0f80 0x7fef1006b260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.800+0000 7fef16f9c640 1 --2- 172.21.15.115:0/2878930596 >> v2:172.21.15.142:3300/0 conn(0x7fef10160200 0x7fef100b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.800+0000 7fef16f9c640 1 -- 172.21.15.115:0/2878930596 >> 172.21.15.115:0/2878930596 conn(0x7fef10065960 msgr2=0x7fef10065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:45.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.800+0000 7fef16f9c640 1 -- 172.21.15.115:0/2878930596 shutdown_connections 2023-12-01T01:52:45.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.800+0000 7fef16f9c640 1 -- 172.21.15.115:0/2878930596 wait complete. 2023-12-01T01:52:45.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.802+0000 7fef16f9c640 1 Processor -- start 2023-12-01T01:52:45.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.802+0000 7fef16f9c640 1 -- start start 2023-12-01T01:52:45.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.803+0000 7fef16f9c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1006b7a0 0x7fef1014f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.803+0000 7fef16f9c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef100b0f80 0x7fef1006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.803+0000 7fef16f9c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef10160200 0x7fef1006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.803+0000 7fef16f9c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef101f2260 con 0x7fef100b0f80 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.803+0000 7fef16f9c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef101f6630 con 0x7fef10160200 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.803+0000 7fef16f9c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef101f2520 con 0x7fef1006b7a0 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.804+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef100b0f80 0x7fef1006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.804+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef100b0f80 0x7fef1006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47108/0 (socket says 172.21.15.115:47108) 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.804+0000 7fef0ffff640 1 -- 172.21.15.115:0/2268834507 learned_addr learned my addr 172.21.15.115:0/2268834507 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.804+0000 7fef15539640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef10160200 0x7fef1006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.804+0000 7fef14d38640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1006b7a0 0x7fef1014f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.804+0000 7fef0ffff640 1 -- 172.21.15.115:0/2268834507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1006b7a0 msgr2=0x7fef1014f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.804+0000 7fef0ffff640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1006b7a0 0x7fef1014f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.804+0000 7fef0ffff640 1 -- 172.21.15.115:0/2268834507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef10160200 msgr2=0x7fef1006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.804+0000 7fef0ffff640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef10160200 0x7fef1006ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:45.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.804+0000 7fef0ffff640 1 -- 172.21.15.115:0/2268834507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc009400 con 0x7fef100b0f80 2023-12-01T01:52:45.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.805+0000 7fef0ffff640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef100b0f80 0x7fef1006f9e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c00 tx=0x7feefc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:45.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.805+0000 7fef0d7fa640 1 -- 172.21.15.115:0/2268834507 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc03c620 con 0x7fef100b0f80 2023-12-01T01:52:45.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.805+0000 7fef0d7fa640 1 -- 172.21.15.115:0/2268834507 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc03c7a0 con 0x7fef100b0f80 2023-12-01T01:52:45.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.805+0000 7fef0d7fa640 1 -- 172.21.15.115:0/2268834507 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc03cb50 con 0x7fef100b0f80 2023-12-01T01:52:45.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.805+0000 7fef16f9c640 1 -- 172.21.15.115:0/2268834507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef101f2800 con 0x7fef100b0f80 2023-12-01T01:52:45.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.805+0000 7fef16f9c640 1 -- 172.21.15.115:0/2268834507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1014f780 con 0x7fef100b0f80 2023-12-01T01:52:45.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.806+0000 7fef16f9c640 1 -- 172.21.15.115:0/2268834507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1014f910 con 0x7fef100b0f80 2023-12-01T01:52:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.807+0000 7fef0d7fa640 1 -- 172.21.15.115:0/2268834507 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7feefc039070 con 0x7fef100b0f80 2023-12-01T01:52:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.808+0000 7fef0d7fa640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feef406a550 0x7feef406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.808+0000 7fef0d7fa640 1 -- 172.21.15.115:0/2268834507 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feefc0aa020 con 0x7fef100b0f80 2023-12-01T01:52:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.808+0000 7fef14d38640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feef406a550 0x7feef406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.809+0000 7fef14d38640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feef406a550 0x7feef406ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef100caf30 tx=0x7fef04008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:45.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:45.812+0000 7fef0d7fa640 1 -- 172.21.15.115:0/2268834507 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fef1014f910 con 0x7fef100b0f80 2023-12-01T01:52:46.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.003+0000 7fef16f9c640 1 -- 172.21.15.115:0/2268834507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef101f2d20 con 0x7fef100b0f80 2023-12-01T01:52:46.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.004+0000 7fef0d7fa640 1 -- 172.21.15.115:0/2268834507 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef101f2d20 con 0x7fef100b0f80 2023-12-01T01:52:46.006 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:52:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.006+0000 7fef16f9c640 1 -- 172.21.15.115:0/2268834507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feef406a550 msgr2=0x7feef406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.006+0000 7fef16f9c640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feef406a550 0x7feef406ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef100caf30 tx=0x7fef04008040 comp rx=0 tx=0).stop 2023-12-01T01:52:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.006+0000 7fef16f9c640 1 -- 172.21.15.115:0/2268834507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef100b0f80 msgr2=0x7fef1006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.006+0000 7fef16f9c640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef100b0f80 0x7fef1006f9e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c00 tx=0x7feefc0046c0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.007+0000 7fef16f9c640 1 -- 172.21.15.115:0/2268834507 shutdown_connections 2023-12-01T01:52:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.007+0000 7fef16f9c640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7feef406a550 0x7feef406ca10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.007+0000 7fef16f9c640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef10160200 0x7fef1006ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.007+0000 7fef16f9c640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fef100b0f80 0x7fef1006f9e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.007+0000 7fef16f9c640 1 --2- 172.21.15.115:0/2268834507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef1006b7a0 0x7fef1014f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.007+0000 7fef16f9c640 1 -- 172.21.15.115:0/2268834507 >> 172.21.15.115:0/2268834507 conn(0x7fef10065960 msgr2=0x7fef100cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.007+0000 7fef16f9c640 1 -- 172.21.15.115:0/2268834507 shutdown_connections 2023-12-01T01:52:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.007+0000 7fef16f9c640 1 -- 172.21.15.115:0/2268834507 wait complete. 2023-12-01T01:52:46.026 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T01:52:46.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.235+0000 7f9d1e4a0640 1 Processor -- start 2023-12-01T01:52:46.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.235+0000 7f9d1e4a0640 1 -- start start 2023-12-01T01:52:46.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.236+0000 7f9d1e4a0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9d1815cf90 0x7f9d180d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.236+0000 7f9d1e4a0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9d180b8610 0x7f9d180cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.236+0000 7f9d1e4a0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d180ce330 con 0x7f9d180cdf00 2023-12-01T01:52:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.236+0000 7f9d1e4a0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1815d370 con 0x7f9d180b8610 2023-12-01T01:52:46.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.236+0000 7f9d1e4a0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d180d19a0 con 0x7f9d1815cf90 2023-12-01T01:52:46.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.237+0000 7f9d177fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9d180b8610 0x7f9d180cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:46.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.237+0000 7f9d177fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9d180b8610 0x7f9d180cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47114/0 (socket says 172.21.15.115:47114) 2023-12-01T01:52:46.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.237+0000 7f9d177fe640 1 -- 172.21.15.115:0/4112667017 learned_addr learned my addr 172.21.15.115:0/4112667017 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:46.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.237+0000 7f9d17fff640 1 --2- 172.21.15.115:0/4112667017 >> v2:172.21.15.142:3300/0 conn(0x7f9d1815cf90 0x7f9d180d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:46.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.237+0000 7f9d177fe640 1 -- 172.21.15.115:0/4112667017 >> v2:172.21.15.142:3300/0 conn(0x7f9d1815cf90 msgr2=0x7f9d180d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:46.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.237+0000 7f9d177fe640 1 --2- 172.21.15.115:0/4112667017 >> v2:172.21.15.142:3300/0 conn(0x7f9d1815cf90 0x7f9d180d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.237+0000 7f9d177fe640 1 -- 172.21.15.115:0/4112667017 >> v1:172.21.15.142:6789/0 conn(0x7f9d180cdf00 legacy=0x7f9d180b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:46.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.237+0000 7f9d177fe640 1 -- 172.21.15.115:0/4112667017 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d180d1c80 con 0x7f9d180b8610 2023-12-01T01:52:46.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.238+0000 7f9d177fe640 1 --2- 172.21.15.115:0/4112667017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d180b8610 0x7f9d180cd9c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9d1808b560 tx=0x7f9d0002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671d77c5e2d0619a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:46.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.239+0000 7f9d16ffd640 1 -- 172.21.15.115:0/4112667017 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00031640 con 0x7f9d180b8610 2023-12-01T01:52:46.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.239+0000 7f9d16ffd640 1 -- 172.21.15.115:0/4112667017 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d000317a0 con 0x7f9d180b8610 2023-12-01T01:52:46.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.239+0000 7f9d16ffd640 1 -- 172.21.15.115:0/4112667017 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00031b30 con 0x7f9d180b8610 2023-12-01T01:52:46.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.239+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/4112667017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d180b8610 msgr2=0x7f9d180cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.239+0000 7f9d1e4a0640 1 --2- 172.21.15.115:0/4112667017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d180b8610 0x7f9d180cd9c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9d1808b560 tx=0x7f9d0002b8f0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.239+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/4112667017 shutdown_connections 2023-12-01T01:52:46.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.239+0000 7f9d1e4a0640 1 --2- 172.21.15.115:0/4112667017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d180b8610 0x7f9d180cd9c0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.239+0000 7f9d1e4a0640 1 --2- 172.21.15.115:0/4112667017 >> v2:172.21.15.142:3300/0 conn(0x7f9d1815cf90 0x7f9d180d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.240+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/4112667017 >> 172.21.15.115:0/4112667017 conn(0x7f9d18065960 msgr2=0x7f9d18065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:46.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.240+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/4112667017 shutdown_connections 2023-12-01T01:52:46.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.240+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/4112667017 wait complete. 2023-12-01T01:52:46.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.241+0000 7f9d1e4a0640 1 Processor -- start 2023-12-01T01:52:46.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.241+0000 7f9d1e4a0640 1 -- start start 2023-12-01T01:52:46.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.242+0000 7f9d1e4a0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d180b8610 0x7f9d180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.242+0000 7f9d1e4a0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d180cdf00 0x7f9d180cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.242+0000 7f9d1e4a0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d1815cf90 0x7f9d181f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.242+0000 7f9d1e4a0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180cbe00 con 0x7f9d1815cf90 2023-12-01T01:52:46.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.242+0000 7f9d1e4a0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180cc060 con 0x7f9d180b8610 2023-12-01T01:52:46.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.242+0000 7f9d1e4a0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180c79e0 con 0x7f9d180cdf00 2023-12-01T01:52:46.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d1ca3d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d1815cf90 0x7f9d181f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:46.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d1ca3d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d1815cf90 0x7f9d181f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47126/0 (socket says 172.21.15.115:47126) 2023-12-01T01:52:46.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d1ca3d640 1 -- 172.21.15.115:0/2258569178 learned_addr learned my addr 172.21.15.115:0/2258569178 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:46.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d1ca3d640 1 -- 172.21.15.115:0/2258569178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d180cdf00 msgr2=0x7f9d180cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d17fff640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d180b8610 0x7f9d180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:46.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d177fe640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d180cdf00 0x7f9d180cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:46.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d1ca3d640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d180cdf00 0x7f9d180cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d1ca3d640 1 -- 172.21.15.115:0/2258569178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d180b8610 msgr2=0x7f9d180b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d1ca3d640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d180b8610 0x7f9d180b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d1ca3d640 1 -- 172.21.15.115:0/2258569178 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d000094d0 con 0x7f9d1815cf90 2023-12-01T01:52:46.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.243+0000 7f9d1ca3d640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d1815cf90 0x7f9d181f5d30 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000c410 tx=0x7f9d1000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:46.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.244+0000 7f9d14ff9640 1 -- 172.21.15.115:0/2258569178 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10011790 con 0x7f9d1815cf90 2023-12-01T01:52:46.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.244+0000 7f9d14ff9640 1 -- 172.21.15.115:0/2258569178 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d100118f0 con 0x7f9d1815cf90 2023-12-01T01:52:46.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.244+0000 7f9d14ff9640 1 -- 172.21.15.115:0/2258569178 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10011ce0 con 0x7f9d1815cf90 2023-12-01T01:52:46.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.244+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/2258569178 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d180c7cc0 con 0x7f9d1815cf90 2023-12-01T01:52:46.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.244+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/2258569178 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d180cc340 con 0x7f9d1815cf90 2023-12-01T01:52:46.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.245+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/2258569178 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d180c81e0 con 0x7f9d1815cf90 2023-12-01T01:52:46.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.246+0000 7f9d14ff9640 1 -- 172.21.15.115:0/2258569178 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9d1001c460 con 0x7f9d1815cf90 2023-12-01T01:52:46.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.247+0000 7f9d14ff9640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9cf806a550 0x7f9cf806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.247+0000 7f9d14ff9640 1 -- 172.21.15.115:0/2258569178 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9d100884b0 con 0x7f9d1815cf90 2023-12-01T01:52:46.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.247+0000 7f9d17fff640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9cf806a550 0x7f9cf806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:46.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.248+0000 7f9d17fff640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9cf806a550 0x7f9cf806ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009e90 tx=0x7f9d0800b040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:46.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.251+0000 7f9d14ff9640 1 -- 172.21.15.115:0/2258569178 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9d180c81e0 con 0x7f9d1815cf90 2023-12-01T01:52:46.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.445+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/2258569178 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9d180c81e0 con 0x7f9d1815cf90 2023-12-01T01:52:46.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.446+0000 7f9d14ff9640 1 -- 172.21.15.115:0/2258569178 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v107) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9d180c81e0 con 0x7f9d1815cf90 2023-12-01T01:52:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:46.448 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:52:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:52:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T01:52:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T01:52:46.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:52:46.448 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:52:46.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/2258569178 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9cf806a550 msgr2=0x7f9cf806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9cf806a550 0x7f9cf806ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009e90 tx=0x7f9d0800b040 comp rx=0 tx=0).stop 2023-12-01T01:52:46.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/2258569178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d1815cf90 msgr2=0x7f9d181f5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d1815cf90 0x7f9d181f5d30 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000c410 tx=0x7f9d1000c8d0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/2258569178 shutdown_connections 2023-12-01T01:52:46.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9cf806a550 0x7f9cf806ca10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9d180cdf00 0x7f9d180cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9d1815cf90 0x7f9d181f5d30 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 --2- 172.21.15.115:0/2258569178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d180b8610 0x7f9d180b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/2258569178 >> 172.21.15.115:0/2258569178 conn(0x7f9d18065960 msgr2=0x7f9d18065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:46.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/2258569178 shutdown_connections 2023-12-01T01:52:46.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.449+0000 7f9d1e4a0640 1 -- 172.21.15.115:0/2258569178 wait complete. 2023-12-01T01:52:46.470 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T01:52:46.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T01:52:46.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.690+0000 7f3d959c6640 1 Processor -- start 2023-12-01T01:52:46.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.690+0000 7f3d959c6640 1 -- start start 2023-12-01T01:52:46.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.691+0000 7f3d959c6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3d9015ac00 0x7f3d900ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d959c6640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3d900cb800 0x7f3d9006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d959c6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9015afe0 con 0x7f3d900aede0 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d959c6640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d900b12b0 con 0x7f3d900cb800 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d959c6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d900b0a10 con 0x7f3d9015ac00 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d951c5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3d900cb800 0x7f3d9006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d951c5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3d900cb800 0x7f3d9006fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47128/0 (socket says 172.21.15.115:47128) 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d951c5640 1 -- 172.21.15.115:0/1561461640 learned_addr learned my addr 172.21.15.115:0/1561461640 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d949c4640 1 --2- 172.21.15.115:0/1561461640 >> v2:172.21.15.142:3300/0 conn(0x7f3d9015ac00 0x7f3d900ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d951c5640 1 -- 172.21.15.115:0/1561461640 >> v2:172.21.15.142:3300/0 conn(0x7f3d9015ac00 msgr2=0x7f3d900ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d951c5640 1 --2- 172.21.15.115:0/1561461640 >> v2:172.21.15.142:3300/0 conn(0x7f3d9015ac00 0x7f3d900ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d951c5640 1 -- 172.21.15.115:0/1561461640 >> v1:172.21.15.115:6789/0 conn(0x7f3d900aede0 legacy=0x7f3d900cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.692+0000 7f3d951c5640 1 -- 172.21.15.115:0/1561461640 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900b0cf0 con 0x7f3d900cb800 2023-12-01T01:52:46.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.693+0000 7f3d951c5640 1 --2- 172.21.15.115:0/1561461640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900cb800 0x7f3d9006fcd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3d90066750 tx=0x7f3d8802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a60a38fb3cd381 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:46.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.694+0000 7f3d8f7fe640 1 -- 172.21.15.115:0/1561461640 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8802ca80 con 0x7f3d900cb800 2023-12-01T01:52:46.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.694+0000 7f3d8f7fe640 1 -- 172.21.15.115:0/1561461640 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8802cbe0 con 0x7f3d900cb800 2023-12-01T01:52:46.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.694+0000 7f3d8f7fe640 1 -- 172.21.15.115:0/1561461640 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d880343f0 con 0x7f3d900cb800 2023-12-01T01:52:46.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.694+0000 7f3d959c6640 1 -- 172.21.15.115:0/1561461640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900cb800 msgr2=0x7f3d9006fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.694+0000 7f3d959c6640 1 --2- 172.21.15.115:0/1561461640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900cb800 0x7f3d9006fcd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3d90066750 tx=0x7f3d8802bee0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.694+0000 7f3d959c6640 1 -- 172.21.15.115:0/1561461640 shutdown_connections 2023-12-01T01:52:46.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.694+0000 7f3d959c6640 1 --2- 172.21.15.115:0/1561461640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900cb800 0x7f3d9006fcd0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.694+0000 7f3d959c6640 1 --2- 172.21.15.115:0/1561461640 >> v2:172.21.15.142:3300/0 conn(0x7f3d9015ac00 0x7f3d900ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.694+0000 7f3d959c6640 1 -- 172.21.15.115:0/1561461640 >> 172.21.15.115:0/1561461640 conn(0x7f3d90065960 msgr2=0x7f3d90065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:46.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.695+0000 7f3d959c6640 1 -- 172.21.15.115:0/1561461640 shutdown_connections 2023-12-01T01:52:46.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.695+0000 7f3d959c6640 1 -- 172.21.15.115:0/1561461640 wait complete. 2023-12-01T01:52:46.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.696+0000 7f3d959c6640 1 Processor -- start 2023-12-01T01:52:46.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.696+0000 7f3d959c6640 1 -- start start 2023-12-01T01:52:46.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.697+0000 7f3d959c6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900aede0 0x7f3d9014ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.697+0000 7f3d959c6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d900cb800 0x7f3d900ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.697+0000 7f3d959c6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9015ac00 0x7f3d900cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.697+0000 7f3d959c6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9014f110 con 0x7f3d900aede0 2023-12-01T01:52:46.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.697+0000 7f3d959c6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9015bfb0 con 0x7f3d9015ac00 2023-12-01T01:52:46.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.697+0000 7f3d959c6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9015c590 con 0x7f3d900cb800 2023-12-01T01:52:46.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.697+0000 7f3d949c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900aede0 0x7f3d9014ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:46.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d949c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900aede0 0x7f3d9014ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47136/0 (socket says 172.21.15.115:47136) 2023-12-01T01:52:46.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d949c4640 1 -- 172.21.15.115:0/3933716869 learned_addr learned my addr 172.21.15.115:0/3933716869 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:46.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d949c4640 1 -- 172.21.15.115:0/3933716869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d900cb800 msgr2=0x7f3d900ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d949c4640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d900cb800 0x7f3d900ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d949c4640 1 -- 172.21.15.115:0/3933716869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9015ac00 msgr2=0x7f3d900cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:46.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d949c4640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9015ac00 0x7f3d900cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:46.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d949c4640 1 -- 172.21.15.115:0/3933716869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d88009b30 con 0x7f3d900aede0 2023-12-01T01:52:46.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d949c4640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900aede0 0x7f3d9014ebd0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400d5f0 tx=0x7f3d8400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:46.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d8d7fa640 1 -- 172.21.15.115:0/3933716869 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d840125f0 con 0x7f3d900aede0 2023-12-01T01:52:46.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d8d7fa640 1 -- 172.21.15.115:0/3933716869 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d84012750 con 0x7f3d900aede0 2023-12-01T01:52:46.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d8d7fa640 1 -- 172.21.15.115:0/3933716869 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84012b00 con 0x7f3d900aede0 2023-12-01T01:52:46.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.698+0000 7f3d959c6640 1 -- 172.21.15.115:0/3933716869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9015c870 con 0x7f3d900aede0 2023-12-01T01:52:46.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.699+0000 7f3d959c6640 1 -- 172.21.15.115:0/3933716869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9015c290 con 0x7f3d900aede0 2023-12-01T01:52:46.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.700+0000 7f3d959c6640 1 -- 172.21.15.115:0/3933716869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d900618d0 con 0x7f3d900aede0 2023-12-01T01:52:46.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.701+0000 7f3d8d7fa640 1 -- 172.21.15.115:0/3933716869 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3d84012c60 con 0x7f3d900aede0 2023-12-01T01:52:46.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.702+0000 7f3d8d7fa640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d7406a550 0x7f3d7406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:46.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.702+0000 7f3d8d7fa640 1 -- 172.21.15.115:0/3933716869 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3d84088c10 con 0x7f3d900aede0 2023-12-01T01:52:46.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.702+0000 7f3d8ffff640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d7406a550 0x7f3d7406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:46.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.703+0000 7f3d8ffff640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d7406a550 0x7f3d7406ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d900cd790 tx=0x7f3d7c007950 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:46.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.706+0000 7f3d8d7fa640 1 -- 172.21.15.115:0/3933716869 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3d84054440 con 0x7f3d900aede0 2023-12-01T01:52:46.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:46.895+0000 7f3d959c6640 1 -- 172.21.15.115:0/3933716869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3d90065f50 con 0x7f3d900aede0 2023-12-01T01:52:47.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.007+0000 7f3d8d7fa640 1 -- 172.21.15.115:0/3933716869 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v108) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3d84057a60 con 0x7f3d900aede0 2023-12-01T01:52:47.009 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T01:52:47.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.012+0000 7f3d959c6640 1 -- 172.21.15.115:0/3933716869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d7406a550 msgr2=0x7f3d7406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.012+0000 7f3d959c6640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d7406a550 0x7f3d7406ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d900cd790 tx=0x7f3d7c007950 comp rx=0 tx=0).stop 2023-12-01T01:52:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.012+0000 7f3d959c6640 1 -- 172.21.15.115:0/3933716869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900aede0 msgr2=0x7f3d9014ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.012+0000 7f3d959c6640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900aede0 0x7f3d9014ebd0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400d5f0 tx=0x7f3d8400dab0 comp rx=0 tx=0).stop 2023-12-01T01:52:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.012+0000 7f3d959c6640 1 -- 172.21.15.115:0/3933716869 shutdown_connections 2023-12-01T01:52:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.012+0000 7f3d959c6640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d7406a550 0x7f3d7406ca10 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3d900cd790 tx=0x7f3d7c007950 comp rx=0 tx=0).stop 2023-12-01T01:52:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.012+0000 7f3d959c6640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d900cb800 0x7f3d900ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.012+0000 7f3d959c6640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d9015ac00 0x7f3d900cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.012+0000 7f3d959c6640 1 --2- 172.21.15.115:0/3933716869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d900aede0 0x7f3d9014ebd0 secure :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400d5f0 tx=0x7f3d8400dab0 comp rx=0 tx=0).stop 2023-12-01T01:52:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.012+0000 7f3d959c6640 1 -- 172.21.15.115:0/3933716869 >> 172.21.15.115:0/3933716869 conn(0x7f3d90065960 msgr2=0x7f3d90159eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:47.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.013+0000 7f3d959c6640 1 -- 172.21.15.115:0/3933716869 shutdown_connections 2023-12-01T01:52:47.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.013+0000 7f3d959c6640 1 -- 172.21.15.115:0/3933716869 wait complete. 2023-12-01T01:52:47.037 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T01:52:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.243+0000 7fccd8199640 1 Processor -- start 2023-12-01T01:52:47.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.243+0000 7fccd8199640 1 -- start start 2023-12-01T01:52:47.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.245+0000 7fccd8199640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fccd015ced0 0x7fccd00b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:47.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.245+0000 7fccd8199640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fccd00b4d70 0x7fccd00d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:47.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.245+0000 7fccd8199640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd00d1480 con 0x7fccd00d2750 2023-12-01T01:52:47.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.245+0000 7fccd8199640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd00d16b0 con 0x7fccd015ced0 2023-12-01T01:52:47.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.245+0000 7fccd8199640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd00d1990 con 0x7fccd00b4d70 2023-12-01T01:52:47.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.245+0000 7fccd5f35640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fccd015ced0 0x7fccd00b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:47.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.245+0000 7fccd5734640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fccd00d2750 0x7fccd00b4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40512/0 (socket says 172.21.15.115:40512) 2023-12-01T01:52:47.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.245+0000 7fccd5734640 1 -- 172.21.15.115:0/426145847 learned_addr learned my addr 172.21.15.115:0/426145847 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:47.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.245+0000 7fccd5f35640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fccd015ced0 0x7fccd00b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47146/0 (socket says 172.21.15.115:47146) 2023-12-01T01:52:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.246+0000 7fccd6736640 1 --2- 172.21.15.115:0/426145847 >> v2:172.21.15.142:3300/0 conn(0x7fccd00b4d70 0x7fccd00d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.246+0000 7fccd5f35640 1 -- 172.21.15.115:0/426145847 >> v2:172.21.15.142:3300/0 conn(0x7fccd00b4d70 msgr2=0x7fccd00d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.246+0000 7fccd5f35640 1 --2- 172.21.15.115:0/426145847 >> v2:172.21.15.142:3300/0 conn(0x7fccd00b4d70 0x7fccd00d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.246+0000 7fccd5f35640 1 -- 172.21.15.115:0/426145847 >> v1:172.21.15.115:6789/0 conn(0x7fccd00d2750 legacy=0x7fccd00b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.246+0000 7fccd5f35640 1 -- 172.21.15.115:0/426145847 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd00d1c70 con 0x7fccd015ced0 2023-12-01T01:52:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.246+0000 7fccd4f33640 1 -- 172.21.15.115:0/426145847 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048463543 0 0) 0x7fccd00d1480 con 0x7fccd00d2750 2023-12-01T01:52:47.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.247+0000 7fccd5f35640 1 --2- 172.21.15.115:0/426145847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd015ced0 0x7fccd00b8730 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fccd00d0740 tx=0x7fcccc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df092a482965a7a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:47.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.247+0000 7fccd4f33640 1 -- 172.21.15.115:0/426145847 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc031830 con 0x7fccd015ced0 2023-12-01T01:52:47.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.247+0000 7fccd4f33640 1 -- 172.21.15.115:0/426145847 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc031990 con 0x7fccd015ced0 2023-12-01T01:52:47.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.247+0000 7fccd4f33640 1 -- 172.21.15.115:0/426145847 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc031d20 con 0x7fccd015ced0 2023-12-01T01:52:47.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.248+0000 7fccd8199640 1 -- 172.21.15.115:0/426145847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd015ced0 msgr2=0x7fccd00b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:47.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.248+0000 7fccd8199640 1 --2- 172.21.15.115:0/426145847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd015ced0 0x7fccd00b8730 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fccd00d0740 tx=0x7fcccc02b8d0 comp rx=0 tx=0).stop 2023-12-01T01:52:47.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.248+0000 7fccd8199640 1 -- 172.21.15.115:0/426145847 shutdown_connections 2023-12-01T01:52:47.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.248+0000 7fccd8199640 1 --2- 172.21.15.115:0/426145847 >> v2:172.21.15.142:3300/0 conn(0x7fccd00b4d70 0x7fccd00d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:47.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.248+0000 7fccd8199640 1 --2- 172.21.15.115:0/426145847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd015ced0 0x7fccd00b8730 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:47.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.248+0000 7fccd8199640 1 -- 172.21.15.115:0/426145847 >> 172.21.15.115:0/426145847 conn(0x7fccd0065f90 msgr2=0x7fccd00cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:47.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.248+0000 7fccd8199640 1 -- 172.21.15.115:0/426145847 shutdown_connections 2023-12-01T01:52:47.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.248+0000 7fccd8199640 1 -- 172.21.15.115:0/426145847 wait complete. 2023-12-01T01:52:47.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.250+0000 7fccd8199640 1 Processor -- start 2023-12-01T01:52:47.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.250+0000 7fccd8199640 1 -- start start 2023-12-01T01:52:47.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.251+0000 7fccd8199640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd00b4d70 0x7fccd00cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.251+0000 7fccd8199640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd00d2750 0x7fccd0159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.251+0000 7fccd8199640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd015ced0 0x7fccd015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.251+0000 7fccd8199640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd015e1d0 con 0x7fccd00d2750 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.251+0000 7fccd8199640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd015a400 con 0x7fccd00b4d70 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.251+0000 7fccd8199640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00ca630 con 0x7fccd015ced0 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd5734640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd00d2750 0x7fccd0159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd5734640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd00d2750 0x7fccd0159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47158/0 (socket says 172.21.15.115:47158) 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd5734640 1 -- 172.21.15.115:0/433021064 learned_addr learned my addr 172.21.15.115:0/433021064 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd6736640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd015ced0 0x7fccd015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd5f35640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd00b4d70 0x7fccd00cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd5734640 1 -- 172.21.15.115:0/433021064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd015ced0 msgr2=0x7fccd015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd5734640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd015ced0 0x7fccd015b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:47.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd5734640 1 -- 172.21.15.115:0/433021064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd00b4d70 msgr2=0x7fccd00cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd5734640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd00b4d70 0x7fccd00cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd5734640 1 -- 172.21.15.115:0/433021064 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc009400 con 0x7fccd00d2750 2023-12-01T01:52:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.252+0000 7fccd5734640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd00d2750 0x7fccd0159ec0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c500 tx=0x7fccc400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.253+0000 7fccc27fc640 1 -- 172.21.15.115:0/433021064 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4007560 con 0x7fccd00d2750 2023-12-01T01:52:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.253+0000 7fccc27fc640 1 -- 172.21.15.115:0/433021064 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc40047e0 con 0x7fccd00d2750 2023-12-01T01:52:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.253+0000 7fccc27fc640 1 -- 172.21.15.115:0/433021064 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4004b90 con 0x7fccd00d2750 2023-12-01T01:52:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.253+0000 7fccd8199640 1 -- 172.21.15.115:0/433021064 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd00ca880 con 0x7fccd00d2750 2023-12-01T01:52:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.253+0000 7fccd8199640 1 -- 172.21.15.115:0/433021064 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd015ddf0 con 0x7fccd00d2750 2023-12-01T01:52:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.254+0000 7fccd8199640 1 -- 172.21.15.115:0/433021064 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd015e310 con 0x7fccd00d2750 2023-12-01T01:52:47.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.255+0000 7fccc27fc640 1 -- 172.21.15.115:0/433021064 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fccc4004cf0 con 0x7fccd00d2750 2023-12-01T01:52:47.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.256+0000 7fccc27fc640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fccb406a550 0x7fccb406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:47.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.256+0000 7fccc27fc640 1 -- 172.21.15.115:0/433021064 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fccc4087a40 con 0x7fccd00d2750 2023-12-01T01:52:47.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.257+0000 7fccd5f35640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fccb406a550 0x7fccb406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:47.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.258+0000 7fccd5f35640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fccb406a550 0x7fccb406ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcccc036000 tx=0x7fcccc03c040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:47.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.262+0000 7fccc27fc640 1 -- 172.21.15.115:0/433021064 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fccd015e310 con 0x7fccd00d2750 2023-12-01T01:52:47.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:47.455+0000 7fccd8199640 1 -- 172.21.15.115:0/433021064 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fccd015e310 con 0x7fccd00d2750 2023-12-01T01:52:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.013+0000 7fccc27fc640 1 -- 172.21.15.115:0/433021064 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v109) v1 ==== 170+0+0 (secure 0 0 0) 0x7fccd015e310 con 0x7fccd00d2750 2023-12-01T01:52:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T01:52:48.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.020+0000 7fccd8199640 1 -- 172.21.15.115:0/433021064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fccb406a550 msgr2=0x7fccb406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.020+0000 7fccd8199640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fccb406a550 0x7fccb406ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcccc036000 tx=0x7fcccc03c040 comp rx=0 tx=0).stop 2023-12-01T01:52:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.020+0000 7fccd8199640 1 -- 172.21.15.115:0/433021064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd00d2750 msgr2=0x7fccd0159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.020+0000 7fccd8199640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd00d2750 0x7fccd0159ec0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c500 tx=0x7fccc400c9c0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.021+0000 7fccd8199640 1 -- 172.21.15.115:0/433021064 shutdown_connections 2023-12-01T01:52:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.021+0000 7fccd8199640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fccb406a550 0x7fccb406ca10 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcccc036000 tx=0x7fcccc03c040 comp rx=0 tx=0).stop 2023-12-01T01:52:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.021+0000 7fccd8199640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd015ced0 0x7fccd015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.021+0000 7fccd8199640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccd00d2750 0x7fccd0159ec0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.021+0000 7fccd8199640 1 --2- 172.21.15.115:0/433021064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd00b4d70 0x7fccd00cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.021+0000 7fccd8199640 1 -- 172.21.15.115:0/433021064 >> 172.21.15.115:0/433021064 conn(0x7fccd0065f90 msgr2=0x7fccd00c8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.023+0000 7fccd8199640 1 -- 172.21.15.115:0/433021064 shutdown_connections 2023-12-01T01:52:48.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.026+0000 7fccd8199640 1 -- 172.21.15.115:0/433021064 wait complete. 2023-12-01T01:52:48.058 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T01:52:48.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.243+0000 7f5e94147640 1 Processor -- start 2023-12-01T01:52:48.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.244+0000 7f5e94147640 1 -- start start 2023-12-01T01:52:48.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.245+0000 7f5e94147640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e8c142870 0x7f5e8c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.245+0000 7f5e94147640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5e8c0b1e20 0x7f5e8c15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.245+0000 7f5e94147640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8c0cd470 con 0x7f5e8c0cd010 2023-12-01T01:52:48.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.245+0000 7f5e94147640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8c0b3ae0 con 0x7f5e8c0b1e20 2023-12-01T01:52:48.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.245+0000 7f5e94147640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c142c50 con 0x7f5e8c142870 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.246+0000 7f5e916e2640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5e8c0cd010 0x7f5e8c0b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:33080/0 (socket says 172.21.15.115:33080) 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.246+0000 7f5e916e2640 1 -- 172.21.15.115:0/2390400625 learned_addr learned my addr 172.21.15.115:0/2390400625 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.246+0000 7f5e91ee3640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e8c142870 0x7f5e8c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.246+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425729702 0 0) 0x7f5e8c0cd470 con 0x7f5e8c0cd010 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.246+0000 7f5e926e4640 1 --2- 172.21.15.115:0/2390400625 >> v2:172.21.15.142:3300/0 conn(0x7f5e8c0b1e20 0x7f5e8c15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.246+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e78003520 con 0x7f5e8c0cd010 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.246+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1968666707 0 0) 0x7f5e78003520 con 0x7f5e8c0cd010 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e8c142870 msgr2=0x7f5e8c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e8c142870 0x7f5e8c0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 >> v2:172.21.15.142:3300/0 conn(0x7f5e8c0b1e20 msgr2=0x7f5e8c15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 --2- 172.21.15.115:0/2390400625 >> v2:172.21.15.142:3300/0 conn(0x7f5e8c0b1e20 0x7f5e8c15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c0b2f30 con 0x7f5e8c0cd010 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5e80001670 con 0x7f5e8c0cd010 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 >> v1:172.21.15.115:6789/0 conn(0x7f5e8c0cd010 legacy=0x7f5e8c0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e780039a0 0x7f5e78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e780047c0 0x7f5e78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e8c142870 0x7f5e78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0b3ae0 con 0x7f5e780039a0 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c142c50 con 0x7f5e780047c0 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7800b980 con 0x7f5e8c142870 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e91ee3640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e780039a0 0x7f5e78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e91ee3640 1 -- 172.21.15.115:0/2390400625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e8c142870 msgr2=0x7f5e78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e926e4640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e8c142870 0x7f5e78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e91ee3640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e8c142870 0x7f5e78005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e91ee3640 1 -- 172.21.15.115:0/2390400625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e780047c0 msgr2=0x7f5e78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:48.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e91ee3640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e780047c0 0x7f5e78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.247+0000 7f5e91ee3640 1 -- 172.21.15.115:0/2390400625 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78003520 con 0x7f5e780039a0 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.249+0000 7f5e91ee3640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e780039a0 0x7f5e78003dc0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009ac0 tx=0x7f5e780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1035adec27ad22a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.249+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8802c6b0 con 0x7f5e780039a0 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.249+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8802c810 con 0x7f5e780039a0 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.249+0000 7f5e90ee1640 1 -- 172.21.15.115:0/2390400625 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8802cba0 con 0x7f5e780039a0 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.249+0000 7f5e94147640 1 -- 172.21.15.115:0/2390400625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e780039a0 msgr2=0x7f5e78003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.249+0000 7f5e94147640 1 --2- 172.21.15.115:0/2390400625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e780039a0 0x7f5e78003dc0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009ac0 tx=0x7f5e780057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.249+0000 7f5e926e4640 1 -- 172.21.15.115:0/2390400625 reap_dead start 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.250+0000 7f5e94147640 1 -- 172.21.15.115:0/2390400625 shutdown_connections 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.250+0000 7f5e94147640 1 -- 172.21.15.115:0/2390400625 >> 172.21.15.115:0/2390400625 conn(0x7f5e8c065960 msgr2=0x7f5e8c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.250+0000 7f5e94147640 1 -- 172.21.15.115:0/2390400625 shutdown_connections 2023-12-01T01:52:48.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.250+0000 7f5e94147640 1 -- 172.21.15.115:0/2390400625 wait complete. 2023-12-01T01:52:48.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.251+0000 7f5e94147640 1 Processor -- start 2023-12-01T01:52:48.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.252+0000 7f5e94147640 1 -- start start 2023-12-01T01:52:48.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.252+0000 7f5e94147640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e780039a0 0x7f5e8c13c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.252+0000 7f5e94147640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e8c1f2650 0x7f5e8c1f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.252+0000 7f5e94147640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e8c1413c0 0x7f5e8c141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.252+0000 7f5e94147640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0ab240 con 0x7f5e8c1413c0 2023-12-01T01:52:48.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.252+0000 7f5e94147640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c141db0 con 0x7f5e8c1f2650 2023-12-01T01:52:48.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.252+0000 7f5e94147640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c13c770 con 0x7f5e780039a0 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.253+0000 7f5e926e4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e8c1413c0 0x7f5e8c141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.253+0000 7f5e926e4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e8c1413c0 0x7f5e8c141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43722/0 (socket says 172.21.15.115:43722) 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.253+0000 7f5e926e4640 1 -- 172.21.15.115:0/1969506981 learned_addr learned my addr 172.21.15.115:0/1969506981 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.253+0000 7f5e91ee3640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e780039a0 0x7f5e8c13c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.253+0000 7f5e916e2640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e8c1f2650 0x7f5e8c1f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.253+0000 7f5e926e4640 1 -- 172.21.15.115:0/1969506981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e780039a0 msgr2=0x7f5e8c13c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.253+0000 7f5e926e4640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e780039a0 0x7f5e8c13c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.253+0000 7f5e926e4640 1 -- 172.21.15.115:0/1969506981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e8c1f2650 msgr2=0x7f5e8c1f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.253+0000 7f5e926e4640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e8c1f2650 0x7f5e8c1f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.253+0000 7f5e926e4640 1 -- 172.21.15.115:0/1969506981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c0aa5c0 con 0x7f5e8c1413c0 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.254+0000 7f5e926e4640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e8c1413c0 0x7f5e8c141870 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c06f1b0 tx=0x7f5e8400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.254+0000 7f5e7e7fc640 1 -- 172.21.15.115:0/1969506981 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84017400 con 0x7f5e8c1413c0 2023-12-01T01:52:48.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.254+0000 7f5e7e7fc640 1 -- 172.21.15.115:0/1969506981 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e84011040 con 0x7f5e8c1413c0 2023-12-01T01:52:48.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.254+0000 7f5e7e7fc640 1 -- 172.21.15.115:0/1969506981 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e840177c0 con 0x7f5e8c1413c0 2023-12-01T01:52:48.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.254+0000 7f5e94147640 1 -- 172.21.15.115:0/1969506981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c0aa8a0 con 0x7f5e8c1413c0 2023-12-01T01:52:48.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.254+0000 7f5e94147640 1 -- 172.21.15.115:0/1969506981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c0aae60 con 0x7f5e8c1413c0 2023-12-01T01:52:48.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.255+0000 7f5e94147640 1 -- 172.21.15.115:0/1969506981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c0cc940 con 0x7f5e8c1413c0 2023-12-01T01:52:48.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.256+0000 7f5e7e7fc640 1 -- 172.21.15.115:0/1969506981 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f5e84017920 con 0x7f5e8c1413c0 2023-12-01T01:52:48.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.257+0000 7f5e7e7fc640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e7006a550 0x7f5e7006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.258+0000 7f5e7e7fc640 1 -- 172.21.15.115:0/1969506981 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5e84088fe0 con 0x7f5e8c1413c0 2023-12-01T01:52:48.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.258+0000 7f5e91ee3640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e7006a550 0x7f5e7006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.258+0000 7f5e91ee3640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e7006a550 0x7f5e7006ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5e88036000 tx=0x7f5e88030000 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:48.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.262+0000 7f5e7e7fc640 1 -- 172.21.15.115:0/1969506981 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5e84054420 con 0x7f5e8c1413c0 2023-12-01T01:52:48.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.488+0000 7f5e94147640 1 -- 172.21.15.115:0/1969506981 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5e8c0ab380 con 0x7f5e8c1413c0 2023-12-01T01:52:48.498 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 12 and data pool 13 2023-12-01T01:52:48.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.497+0000 7f5e7e7fc640 1 -- 172.21.15.115:0/1969506981 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v44) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5e84057a40 con 0x7f5e8c1413c0 2023-12-01T01:52:48.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.502+0000 7f5e94147640 1 -- 172.21.15.115:0/1969506981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e7006a550 msgr2=0x7f5e7006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.502+0000 7f5e94147640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e7006a550 0x7f5e7006ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5e88036000 tx=0x7f5e88030000 comp rx=0 tx=0).stop 2023-12-01T01:52:48.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.502+0000 7f5e94147640 1 -- 172.21.15.115:0/1969506981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e8c1413c0 msgr2=0x7f5e8c141870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.502+0000 7f5e94147640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e8c1413c0 0x7f5e8c141870 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c06f1b0 tx=0x7f5e8400f4a0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.504+0000 7f5e94147640 1 -- 172.21.15.115:0/1969506981 shutdown_connections 2023-12-01T01:52:48.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.504+0000 7f5e94147640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5e7006a550 0x7f5e7006ca10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.504+0000 7f5e94147640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e8c1413c0 0x7f5e8c141870 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.504+0000 7f5e94147640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e8c1f2650 0x7f5e8c1f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.504+0000 7f5e94147640 1 --2- 172.21.15.115:0/1969506981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e780039a0 0x7f5e8c13c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.504+0000 7f5e94147640 1 -- 172.21.15.115:0/1969506981 >> 172.21.15.115:0/1969506981 conn(0x7f5e8c065960 msgr2=0x7f5e8c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:48.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.506+0000 7f5e94147640 1 -- 172.21.15.115:0/1969506981 shutdown_connections 2023-12-01T01:52:48.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.506+0000 7f5e94147640 1 -- 172.21.15.115:0/1969506981 wait complete. 2023-12-01T01:52:48.536 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:48.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.711+0000 7f03c23bc640 1 Processor -- start 2023-12-01T01:52:48.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.711+0000 7f03c23bc640 1 -- start start 2023-12-01T01:52:48.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.712+0000 7f03c23bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc15afe0 0x7f03bc06b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.712+0000 7f03c23bc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03bc06b7b0 0x7f03bc0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.712+0000 7f03c23bc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc0ce4a0 con 0x7f03bc1584b0 2023-12-01T01:52:48.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.712+0000 7f03c23bc640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03bc06bc40 con 0x7f03bc06b7b0 2023-12-01T01:52:48.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.712+0000 7f03c23bc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0cc180 con 0x7f03bc15afe0 2023-12-01T01:52:48.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.713+0000 7f03c1bbb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03bc06b7b0 0x7f03bc0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.713+0000 7f03c1bbb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03bc06b7b0 0x7f03bc0cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43738/0 (socket says 172.21.15.115:43738) 2023-12-01T01:52:48.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.713+0000 7f03c1bbb640 1 -- 172.21.15.115:0/1205546623 learned_addr learned my addr 172.21.15.115:0/1205546623 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:48.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.713+0000 7f03c1bbb640 1 -- 172.21.15.115:0/1205546623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc15afe0 msgr2=0x7f03bc06b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:48.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.713+0000 7f03c1bbb640 1 --2- 172.21.15.115:0/1205546623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc15afe0 0x7f03bc06b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.713+0000 7f03c1bbb640 1 -- 172.21.15.115:0/1205546623 >> v1:172.21.15.142:6789/0 conn(0x7f03bc1584b0 legacy=0x7f03bc0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:48.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.713+0000 7f03c1bbb640 1 -- 172.21.15.115:0/1205546623 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc15b550 con 0x7f03bc06b7b0 2023-12-01T01:52:48.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.714+0000 7f03c1bbb640 1 --2- 172.21.15.115:0/1205546623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc06b7b0 0x7f03bc0cc680 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009e50 tx=0x7f03b402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b789635a5d03e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:48.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.715+0000 7f03b3fff640 1 -- 172.21.15.115:0/1205546623 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b402ca70 con 0x7f03bc06b7b0 2023-12-01T01:52:48.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.715+0000 7f03b3fff640 1 -- 172.21.15.115:0/1205546623 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b402cbd0 con 0x7f03bc06b7b0 2023-12-01T01:52:48.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.715+0000 7f03b3fff640 1 -- 172.21.15.115:0/1205546623 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40343f0 con 0x7f03bc06b7b0 2023-12-01T01:52:48.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.715+0000 7f03c23bc640 1 -- 172.21.15.115:0/1205546623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc06b7b0 msgr2=0x7f03bc0cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.715+0000 7f03c23bc640 1 --2- 172.21.15.115:0/1205546623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc06b7b0 0x7f03bc0cc680 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009e50 tx=0x7f03b402bed0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.716+0000 7f03c23bc640 1 -- 172.21.15.115:0/1205546623 shutdown_connections 2023-12-01T01:52:48.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.716+0000 7f03c23bc640 1 --2- 172.21.15.115:0/1205546623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc06b7b0 0x7f03bc0cc680 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.716+0000 7f03c23bc640 1 --2- 172.21.15.115:0/1205546623 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc15afe0 0x7f03bc06b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.716+0000 7f03c23bc640 1 -- 172.21.15.115:0/1205546623 >> 172.21.15.115:0/1205546623 conn(0x7f03bc0ce640 msgr2=0x7f03bc0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:48.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.716+0000 7f03c23bc640 1 -- 172.21.15.115:0/1205546623 shutdown_connections 2023-12-01T01:52:48.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.716+0000 7f03c23bc640 1 -- 172.21.15.115:0/1205546623 wait complete. 2023-12-01T01:52:48.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.717+0000 7f03c23bc640 1 Processor -- start 2023-12-01T01:52:48.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.717+0000 7f03c23bc640 1 -- start start 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.718+0000 7f03c23bc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03bc06b7b0 0x7f03bc1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.718+0000 7f03c23bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc1584b0 0x7f03bc0c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.718+0000 7f03c23bc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc15afe0 0x7f03bc157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.718+0000 7f03c23bc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0c6c90 con 0x7f03bc15afe0 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.718+0000 7f03c23bc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc1507e0 con 0x7f03bc06b7b0 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.718+0000 7f03c23bc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0c6610 con 0x7f03bc1584b0 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c1bbb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc15afe0 0x7f03bc157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c1bbb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc15afe0 0x7f03bc157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43748/0 (socket says 172.21.15.115:43748) 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c1bbb640 1 -- 172.21.15.115:0/341384235 learned_addr learned my addr 172.21.15.115:0/341384235 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c13ba640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03bc06b7b0 0x7f03bc1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c0bb9640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc1584b0 0x7f03bc0c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c1bbb640 1 -- 172.21.15.115:0/341384235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc1584b0 msgr2=0x7f03bc0c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c1bbb640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc1584b0 0x7f03bc0c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c1bbb640 1 -- 172.21.15.115:0/341384235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03bc06b7b0 msgr2=0x7f03bc1502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c1bbb640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03bc06b7b0 0x7f03bc1502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c1bbb640 1 -- 172.21.15.115:0/341384235 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b4009b30 con 0x7f03bc15afe0 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c13ba640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03bc06b7b0 0x7f03bc1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c0bb9640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc1584b0 0x7f03bc0c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.719+0000 7f03c1bbb640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc15afe0 0x7f03bc157000 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f03b4036040 tx=0x7f03b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.720+0000 7f03b1ffb640 1 -- 172.21.15.115:0/341384235 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b403e450 con 0x7f03bc15afe0 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.720+0000 7f03b1ffb640 1 -- 172.21.15.115:0/341384235 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b403e5d0 con 0x7f03bc15afe0 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.720+0000 7f03c23bc640 1 -- 172.21.15.115:0/341384235 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc152990 con 0x7f03bc15afe0 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.720+0000 7f03b1ffb640 1 -- 172.21.15.115:0/341384235 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b403e980 con 0x7f03bc15afe0 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.720+0000 7f03c23bc640 1 -- 172.21.15.115:0/341384235 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc152c70 con 0x7f03bc15afe0 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.721+0000 7f03c23bc640 1 -- 172.21.15.115:0/341384235 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc0c6470 con 0x7f03bc15afe0 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.722+0000 7f03b1ffb640 1 -- 172.21.15.115:0/341384235 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f03b403eae0 con 0x7f03bc15afe0 2023-12-01T01:52:48.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.723+0000 7f03b1ffb640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f03a006a580 0x7f03a006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.723+0000 7f03b1ffb640 1 -- 172.21.15.115:0/341384235 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03b40ab500 con 0x7f03bc15afe0 2023-12-01T01:52:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.723+0000 7f03c13ba640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f03a006a580 0x7f03a006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.724+0000 7f03c13ba640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f03a006a580 0x7f03a006ca40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f03b8007fd0 tx=0x7f03b8009040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:48.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.727+0000 7f03b1ffb640 1 -- 172.21.15.115:0/341384235 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f03bc0c6470 con 0x7f03bc15afe0 2023-12-01T01:52:48.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.906+0000 7f03c23bc640 1 -- 172.21.15.115:0/341384235 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03bc0c6dd0 con 0x7f03bc15afe0 2023-12-01T01:52:48.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.909+0000 7f03b1ffb640 1 -- 172.21.15.115:0/341384235 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16176 (secure 0 0 0) 0x7f03bc0c6dd0 con 0x7f03bc15afe0 2023-12-01T01:52:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:48.911 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":110,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:48.490789+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:46.948807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:47.513140+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:48.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.911+0000 7f03c23bc640 1 -- 172.21.15.115:0/341384235 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f03a006a580 msgr2=0x7f03a006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.911+0000 7f03c23bc640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f03a006a580 0x7f03a006ca40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f03b8007fd0 tx=0x7f03b8009040 comp rx=0 tx=0).stop 2023-12-01T01:52:48.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.911+0000 7f03c23bc640 1 -- 172.21.15.115:0/341384235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc15afe0 msgr2=0x7f03bc157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:48.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.911+0000 7f03c23bc640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc15afe0 0x7f03bc157000 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f03b4036040 tx=0x7f03b4004a40 comp rx=0 tx=0).stop 2023-12-01T01:52:48.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.911+0000 7f03c23bc640 1 -- 172.21.15.115:0/341384235 shutdown_connections 2023-12-01T01:52:48.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.911+0000 7f03c23bc640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f03a006a580 0x7f03a006ca40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.911+0000 7f03c23bc640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc1584b0 0x7f03bc0c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.911+0000 7f03c23bc640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03bc15afe0 0x7f03bc157000 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.911+0000 7f03c23bc640 1 --2- 172.21.15.115:0/341384235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03bc06b7b0 0x7f03bc1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:48.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.911+0000 7f03c23bc640 1 -- 172.21.15.115:0/341384235 >> 172.21.15.115:0/341384235 conn(0x7f03bc0ce640 msgr2=0x7f03bc0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:48.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.912+0000 7f03c23bc640 1 -- 172.21.15.115:0/341384235 shutdown_connections 2023-12-01T01:52:48.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:48.912+0000 7f03c23bc640 1 -- 172.21.15.115:0/341384235 wait complete. 2023-12-01T01:52:48.931 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:49.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.138+0000 7fd17f21e640 1 Processor -- start 2023-12-01T01:52:49.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.139+0000 7fd17f21e640 1 -- start start 2023-12-01T01:52:49.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.140+0000 7fd17f21e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780ca3f0 0x7fd1781648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.140+0000 7fd17f21e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17815add0 con 0x7fd178159bc0 2023-12-01T01:52:49.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.140+0000 7fd17f21e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17815a1c0 con 0x7fd1780cb8c0 2023-12-01T01:52:49.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.140+0000 7fd17f21e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd178159fb0 con 0x7fd1780ca3f0 2023-12-01T01:52:49.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.140+0000 7fd17cfba640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd178159bc0 0x7fd1780cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:33096/0 (socket says 172.21.15.115:33096) 2023-12-01T01:52:49.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.140+0000 7fd17cfba640 1 -- 172.21.15.115:0/827096082 learned_addr learned my addr 172.21.15.115:0/827096082 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596385079 0 0) 0x7fd17815add0 con 0x7fd178159bc0 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd168005730 con 0x7fd178159bc0 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd17d7bb640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780ca3f0 0x7fd1781648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 291814400 0 0) 0x7fd168005730 con 0x7fd178159bc0 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780ca3f0 msgr2=0x7fd1781648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780ca3f0 0x7fd1781648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 >> v1:172.21.15.142:6789/0 conn(0x7fd1780cb8c0 legacy=0x7fd17806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17815a4a0 con 0x7fd178159bc0 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd16c001670 con 0x7fd178159bc0 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 >> v1:172.21.15.115:6789/0 conn(0x7fd178159bc0 legacy=0x7fd1780cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780ca3f0 0x7fd168005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd168006230 0x7fd1680069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 0x7fd168007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178159fb0 con 0x7fd168006f10 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16800ddd0 con 0x7fd168006230 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.141+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16800df90 con 0x7fd1780ca3f0 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.142+0000 7fd17d7bb640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 0x7fd168007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.142+0000 7fd17cfba640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780ca3f0 0x7fd168005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.142+0000 7fd177fff640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd168006230 0x7fd1680069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.142+0000 7fd17d7bb640 1 -- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780ca3f0 msgr2=0x7fd168005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.142+0000 7fd17d7bb640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780ca3f0 0x7fd168005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.142+0000 7fd17d7bb640 1 -- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd168006230 msgr2=0x7fd1680069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.142+0000 7fd17d7bb640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd168006230 0x7fd1680069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.142+0000 7fd17d7bb640 1 -- 172.21.15.115:0/827096082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168005730 con 0x7fd168006f10 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.142+0000 7fd17cfba640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780ca3f0 0x7fd168005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:49.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.142+0000 7fd177fff640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd168006230 0x7fd1680069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:49.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.143+0000 7fd17d7bb640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 0x7fd168007390 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd1780c22a0 tx=0x7fd168007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4185c0a4623ee536 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:49.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.143+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17002cc50 con 0x7fd168006f10 2023-12-01T01:52:49.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.143+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17002cdb0 con 0x7fd168006f10 2023-12-01T01:52:49.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.143+0000 7fd1777fe640 1 -- 172.21.15.115:0/827096082 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170034680 con 0x7fd168006f10 2023-12-01T01:52:49.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.144+0000 7fd17f21e640 1 -- 172.21.15.115:0/827096082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 msgr2=0x7fd168007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.144+0000 7fd17f21e640 1 --2- 172.21.15.115:0/827096082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 0x7fd168007390 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd1780c22a0 tx=0x7fd168007a00 comp rx=0 tx=0).stop 2023-12-01T01:52:49.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.144+0000 7fd17d7bb640 1 -- 172.21.15.115:0/827096082 reap_dead start 2023-12-01T01:52:49.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.144+0000 7fd17f21e640 1 -- 172.21.15.115:0/827096082 shutdown_connections 2023-12-01T01:52:49.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.144+0000 7fd17f21e640 1 -- 172.21.15.115:0/827096082 >> 172.21.15.115:0/827096082 conn(0x7fd178066210 msgr2=0x7fd1780b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:49.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.144+0000 7fd17f21e640 1 -- 172.21.15.115:0/827096082 shutdown_connections 2023-12-01T01:52:49.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.144+0000 7fd17f21e640 1 -- 172.21.15.115:0/827096082 wait complete. 2023-12-01T01:52:49.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.146+0000 7fd17f21e640 1 Processor -- start 2023-12-01T01:52:49.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.146+0000 7fd17f21e640 1 -- start start 2023-12-01T01:52:49.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.147+0000 7fd17f21e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 0x7fd1781f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.147+0000 7fd17f21e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1780b3d70 0x7fd1781483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.147+0000 7fd17f21e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780b41f0 0x7fd1780ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.147+0000 7fd17f21e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1781f2970 con 0x7fd168006f10 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.147+0000 7fd17f21e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1780c2e60 con 0x7fd1780b3d70 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.147+0000 7fd17f21e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1780c3140 con 0x7fd1780b41f0 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.147+0000 7fd17cfba640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 0x7fd1781f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.147+0000 7fd17cfba640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 0x7fd1781f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43766/0 (socket says 172.21.15.115:43766) 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.147+0000 7fd17cfba640 1 -- 172.21.15.115:0/2178942101 learned_addr learned my addr 172.21.15.115:0/2178942101 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd17cfba640 1 -- 172.21.15.115:0/2178942101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780b41f0 msgr2=0x7fd1780ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd17cfba640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780b41f0 0x7fd1780ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd17cfba640 1 -- 172.21.15.115:0/2178942101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1780b3d70 msgr2=0x7fd1781483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd17cfba640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1780b3d70 0x7fd1781483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd17cfba640 1 -- 172.21.15.115:0/2178942101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780c3420 con 0x7fd168006f10 2023-12-01T01:52:49.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd17cfba640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 0x7fd1781f2430 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00d8b0 tx=0x7fd16c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:49.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd1757fa640 1 -- 172.21.15.115:0/2178942101 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0074e0 con 0x7fd168006f10 2023-12-01T01:52:49.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd1757fa640 1 -- 172.21.15.115:0/2178942101 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c007640 con 0x7fd168006f10 2023-12-01T01:52:49.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd1757fa640 1 -- 172.21.15.115:0/2178942101 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0049e0 con 0x7fd168006f10 2023-12-01T01:52:49.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd17f21e640 1 -- 172.21.15.115:0/2178942101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1780c3770 con 0x7fd168006f10 2023-12-01T01:52:49.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.148+0000 7fd17f21e640 1 -- 172.21.15.115:0/2178942101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1780c3d30 con 0x7fd168006f10 2023-12-01T01:52:49.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.149+0000 7fd17f21e640 1 -- 172.21.15.115:0/2178942101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd178074210 con 0x7fd168006f10 2023-12-01T01:52:49.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.150+0000 7fd1757fa640 1 -- 172.21.15.115:0/2178942101 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd16c004b40 con 0x7fd168006f10 2023-12-01T01:52:49.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.151+0000 7fd1757fa640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd15c06a550 0x7fd15c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.151+0000 7fd1757fa640 1 -- 172.21.15.115:0/2178942101 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd16c088600 con 0x7fd168006f10 2023-12-01T01:52:49.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.151+0000 7fd177fff640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd15c06a550 0x7fd15c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.152+0000 7fd177fff640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd15c06a550 0x7fd15c06ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd164000f60 tx=0x7fd164009040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:49.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.156+0000 7fd1757fa640 1 -- 172.21.15.115:0/2178942101 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd16c0539d0 con 0x7fd168006f10 2023-12-01T01:52:49.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.347+0000 7fd17f21e640 1 -- 172.21.15.115:0/2178942101 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1780b00e0 con 0x7fd168006f10 2023-12-01T01:52:49.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.349+0000 7fd1757fa640 1 -- 172.21.15.115:0/2178942101 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16176 (secure 0 0 0) 0x7fd16c056ff0 con 0x7fd168006f10 2023-12-01T01:52:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:49.351 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":110,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:48.490789+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:46.948807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:47.513140+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:49.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 -- 172.21.15.115:0/2178942101 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd15c06a550 msgr2=0x7fd15c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd15c06a550 0x7fd15c06ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd164000f60 tx=0x7fd164009040 comp rx=0 tx=0).stop 2023-12-01T01:52:49.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 -- 172.21.15.115:0/2178942101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 msgr2=0x7fd1781f2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 0x7fd1781f2430 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00d8b0 tx=0x7fd16c00dd70 comp rx=0 tx=0).stop 2023-12-01T01:52:49.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 -- 172.21.15.115:0/2178942101 shutdown_connections 2023-12-01T01:52:49.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd15c06a550 0x7fd15c06ca10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1780b41f0 0x7fd1780ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1780b3d70 0x7fd1781483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 --2- 172.21.15.115:0/2178942101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd168006f10 0x7fd1781f2430 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 -- 172.21.15.115:0/2178942101 >> 172.21.15.115:0/2178942101 conn(0x7fd178066210 msgr2=0x7fd178070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:49.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 -- 172.21.15.115:0/2178942101 shutdown_connections 2023-12-01T01:52:49.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.352+0000 7fd17f21e640 1 -- 172.21.15.115:0/2178942101 wait complete. 2023-12-01T01:52:49.374 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T01:52:49.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.585+0000 7f2c08773640 1 Processor -- start 2023-12-01T01:52:49.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.585+0000 7f2c08773640 1 -- start start 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.586+0000 7f2c08773640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2c00142870 0x7f2c000ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.586+0000 7f2c08773640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2c000b1e20 0x7f2c0015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.586+0000 7f2c08773640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c000b2f30 con 0x7f2c000cd010 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.586+0000 7f2c08773640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c000b3ae0 con 0x7f2c000b1e20 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.586+0000 7f2c08773640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c000cd4a0 con 0x7f2c00142870 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.587+0000 7f2c06d10640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2c000b1e20 0x7f2c0015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.587+0000 7f2c06d10640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2c000b1e20 0x7f2c0015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43770/0 (socket says 172.21.15.115:43770) 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.587+0000 7f2c06d10640 1 -- 172.21.15.115:0/4004436017 learned_addr learned my addr 172.21.15.115:0/4004436017 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.587+0000 7f2c0650f640 1 --2- 172.21.15.115:0/4004436017 >> v2:172.21.15.142:3300/0 conn(0x7f2c00142870 0x7f2c000ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.587+0000 7f2c06d10640 1 -- 172.21.15.115:0/4004436017 >> v2:172.21.15.142:3300/0 conn(0x7f2c00142870 msgr2=0x7f2c000ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:49.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.587+0000 7f2c06d10640 1 --2- 172.21.15.115:0/4004436017 >> v2:172.21.15.142:3300/0 conn(0x7f2c00142870 0x7f2c000ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.587+0000 7f2c06d10640 1 -- 172.21.15.115:0/4004436017 >> v1:172.21.15.115:6789/0 conn(0x7f2c000cd010 legacy=0x7f2c000b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.587+0000 7f2c06d10640 1 -- 172.21.15.115:0/4004436017 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c000b2320 con 0x7f2c000b1e20 2023-12-01T01:52:49.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.587+0000 7f2c0550d640 1 -- 172.21.15.115:0/4004436017 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166081386 0 0) 0x7f2c000b2f30 con 0x7f2c000cd010 2023-12-01T01:52:49.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.587+0000 7f2c0650f640 1 --2- 172.21.15.115:0/4004436017 >> v2:172.21.15.142:3300/0 conn(0x7f2c00142870 0x7f2c000ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:49.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.588+0000 7f2c06d10640 1 --2- 172.21.15.115:0/4004436017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000b1e20 0x7f2c0015a0d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2c000b14d0 tx=0x7f2bf802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c91170d00e5343 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:49.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.588+0000 7f2c0550d640 1 -- 172.21.15.115:0/4004436017 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf802cb20 con 0x7f2c000b1e20 2023-12-01T01:52:49.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.589+0000 7f2c0550d640 1 -- 172.21.15.115:0/4004436017 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf802cc80 con 0x7f2c000b1e20 2023-12-01T01:52:49.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.589+0000 7f2c0550d640 1 -- 172.21.15.115:0/4004436017 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80343f0 con 0x7f2c000b1e20 2023-12-01T01:52:49.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.589+0000 7f2c08773640 1 -- 172.21.15.115:0/4004436017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000b1e20 msgr2=0x7f2c0015a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.589+0000 7f2c08773640 1 --2- 172.21.15.115:0/4004436017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000b1e20 0x7f2c0015a0d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2c000b14d0 tx=0x7f2bf802bf80 comp rx=0 tx=0).stop 2023-12-01T01:52:49.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.589+0000 7f2c08773640 1 -- 172.21.15.115:0/4004436017 shutdown_connections 2023-12-01T01:52:49.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.589+0000 7f2c08773640 1 --2- 172.21.15.115:0/4004436017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000b1e20 0x7f2c0015a0d0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.589+0000 7f2c08773640 1 --2- 172.21.15.115:0/4004436017 >> v2:172.21.15.142:3300/0 conn(0x7f2c00142870 0x7f2c000ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.589+0000 7f2c08773640 1 -- 172.21.15.115:0/4004436017 >> 172.21.15.115:0/4004436017 conn(0x7f2c00065960 msgr2=0x7f2c00065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:49.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.589+0000 7f2c08773640 1 -- 172.21.15.115:0/4004436017 shutdown_connections 2023-12-01T01:52:49.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.589+0000 7f2c08773640 1 -- 172.21.15.115:0/4004436017 wait complete. 2023-12-01T01:52:49.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.591+0000 7f2c08773640 1 Processor -- start 2023-12-01T01:52:49.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.591+0000 7f2c08773640 1 -- start start 2023-12-01T01:52:49.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.592+0000 7f2c08773640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c000b1e20 0x7f2c00153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.592+0000 7f2c08773640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000cd010 0x7f2c00153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.592+0000 7f2c08773640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c00142870 0x7f2c00157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.592+0000 7f2c08773640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0015a8e0 con 0x7f2c000cd010 2023-12-01T01:52:49.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.592+0000 7f2c08773640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00153ca0 con 0x7f2c000b1e20 2023-12-01T01:52:49.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.592+0000 7f2c08773640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00158750 con 0x7f2c00142870 2023-12-01T01:52:49.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.592+0000 7f2c05d0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000cd010 0x7f2c00153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.592+0000 7f2c05d0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000cd010 0x7f2c00153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43778/0 (socket says 172.21.15.115:43778) 2023-12-01T01:52:49.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.592+0000 7f2c05d0e640 1 -- 172.21.15.115:0/1171330573 learned_addr learned my addr 172.21.15.115:0/1171330573 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2c0650f640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c000b1e20 0x7f2c00153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2c05d0e640 1 -- 172.21.15.115:0/1171330573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c00142870 msgr2=0x7f2c00157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2c05d0e640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c00142870 0x7f2c00157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2c05d0e640 1 -- 172.21.15.115:0/1171330573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c000b1e20 msgr2=0x7f2c00153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2c05d0e640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c000b1e20 0x7f2c00153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2c05d0e640 1 -- 172.21.15.115:0/1171330573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf8009b30 con 0x7f2c000cd010 2023-12-01T01:52:49.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2c0650f640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c000b1e20 0x7f2c00153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2c05d0e640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000cd010 0x7f2c00153760 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000c4b0 tx=0x7f2bf000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2bf6ffd640 1 -- 172.21.15.115:0/1171330573 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0005480 con 0x7f2c000cd010 2023-12-01T01:52:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2bf6ffd640 1 -- 172.21.15.115:0/1171330573 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf00035f0 con 0x7f2c000cd010 2023-12-01T01:52:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2bf6ffd640 1 -- 172.21.15.115:0/1171330573 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00039e0 con 0x7f2c000cd010 2023-12-01T01:52:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2c08773640 1 -- 172.21.15.115:0/1171330573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0015ab80 con 0x7f2c000cd010 2023-12-01T01:52:49.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.593+0000 7f2c08773640 1 -- 172.21.15.115:0/1171330573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0015b140 con 0x7f2c000cd010 2023-12-01T01:52:49.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.595+0000 7f2c08773640 1 -- 172.21.15.115:0/1171330573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c000cc940 con 0x7f2c000cd010 2023-12-01T01:52:49.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.595+0000 7f2bf6ffd640 1 -- 172.21.15.115:0/1171330573 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f2bf0003b40 con 0x7f2c000cd010 2023-12-01T01:52:49.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.596+0000 7f2bf6ffd640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2be406a550 0x7f2be406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:49.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.596+0000 7f2bf6ffd640 1 -- 172.21.15.115:0/1171330573 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bf00880c0 con 0x7f2c000cd010 2023-12-01T01:52:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.597+0000 7f2c0650f640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2be406a550 0x7f2be406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:49.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.597+0000 7f2c0650f640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2be406a550 0x7f2be406ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc006fd0 tx=0x7f2bfc008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:49.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.600+0000 7f2bf6ffd640 1 -- 172.21.15.115:0/1171330573 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2bf00535a0 con 0x7f2c000cd010 2023-12-01T01:52:49.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:49.830+0000 7f2c08773640 1 -- 172.21.15.115:0/1171330573 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2c0004e890 con 0x7f2c000cd010 2023-12-01T01:52:50.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.515+0000 7f2bf6ffd640 1 -- 172.21.15.115:0/1171330573 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2bf0056bc0 con 0x7f2c000cd010 2023-12-01T01:52:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.518+0000 7f2c08773640 1 -- 172.21.15.115:0/1171330573 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2be406a550 msgr2=0x7f2be406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.518+0000 7f2c08773640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2be406a550 0x7f2be406ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc006fd0 tx=0x7f2bfc008040 comp rx=0 tx=0).stop 2023-12-01T01:52:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.518+0000 7f2c08773640 1 -- 172.21.15.115:0/1171330573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000cd010 msgr2=0x7f2c00153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.518+0000 7f2c08773640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000cd010 0x7f2c00153760 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000c4b0 tx=0x7f2bf000c970 comp rx=0 tx=0).stop 2023-12-01T01:52:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.518+0000 7f2c08773640 1 -- 172.21.15.115:0/1171330573 shutdown_connections 2023-12-01T01:52:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.518+0000 7f2c08773640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2be406a550 0x7f2be406ca10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.518+0000 7f2c08773640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c00142870 0x7f2c00157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.518+0000 7f2c08773640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000cd010 0x7f2c00153760 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.518+0000 7f2c08773640 1 --2- 172.21.15.115:0/1171330573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c000b1e20 0x7f2c00153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.518+0000 7f2c08773640 1 -- 172.21.15.115:0/1171330573 >> 172.21.15.115:0/1171330573 conn(0x7f2c00065960 msgr2=0x7f2c0013f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:50.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.519+0000 7f2c08773640 1 -- 172.21.15.115:0/1171330573 shutdown_connections 2023-12-01T01:52:50.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.519+0000 7f2c08773640 1 -- 172.21.15.115:0/1171330573 wait complete. 2023-12-01T01:52:50.540 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T01:52:50.540 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T01:52:50.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.755+0000 7f43f7fff640 1 Processor -- start 2023-12-01T01:52:50.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.756+0000 7f43f7fff640 1 -- start start 2023-12-01T01:52:50.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.757+0000 7f43f7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f815cdf0 0x7f43f8065960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:50.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.757+0000 7f43f7fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f43f80ae6f0 0x7f43f8065ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:50.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.757+0000 7f43f7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f43f80aedf0 0x7f43f8070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:50.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.757+0000 7f43f7fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f80cd4f0 con 0x7f43f80aedf0 2023-12-01T01:52:50.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.757+0000 7f43f7fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f80cdf40 con 0x7f43f80ae6f0 2023-12-01T01:52:50.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.757+0000 7f43f7fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f80cdc60 con 0x7f43f815cdf0 2023-12-01T01:52:50.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.758+0000 7f43f77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f43f80aedf0 0x7f43f8070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:50.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.758+0000 7f43f77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f43f80aedf0 0x7f43f8070140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43786/0 (socket says 172.21.15.115:43786) 2023-12-01T01:52:50.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.758+0000 7f43f77fe640 1 -- 172.21.15.115:0/313681317 learned_addr learned my addr 172.21.15.115:0/313681317 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:50.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.758+0000 7f43f6ffd640 1 --2- 172.21.15.115:0/313681317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f815cdf0 0x7f43f8065960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:50.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.758+0000 7f43f67fc640 1 --2- 172.21.15.115:0/313681317 >> v2:172.21.15.142:3300/0 conn(0x7f43f80ae6f0 0x7f43f8065ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:50.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.758+0000 7f43f77fe640 1 -- 172.21.15.115:0/313681317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f815cdf0 msgr2=0x7f43f8065960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:50.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.758+0000 7f43f77fe640 1 --2- 172.21.15.115:0/313681317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f815cdf0 0x7f43f8065960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.758+0000 7f43f77fe640 1 -- 172.21.15.115:0/313681317 >> v2:172.21.15.142:3300/0 conn(0x7f43f80ae6f0 msgr2=0x7f43f8065ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:50.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.758+0000 7f43f77fe640 1 --2- 172.21.15.115:0/313681317 >> v2:172.21.15.142:3300/0 conn(0x7f43f80ae6f0 0x7f43f8065ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.758+0000 7f43f77fe640 1 -- 172.21.15.115:0/313681317 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f80c8aa0 con 0x7f43f80aedf0 2023-12-01T01:52:50.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.759+0000 7f43f77fe640 1 --2- 172.21.15.115:0/313681317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f80aedf0 0x7f43f8070140 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f43f806f870 tx=0x7f43f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6a16de0060a228 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:50.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.760+0000 7f43f5ffb640 1 -- 172.21.15.115:0/313681317 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0031870 con 0x7f43f80aedf0 2023-12-01T01:52:50.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.760+0000 7f43f5ffb640 1 -- 172.21.15.115:0/313681317 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f00319d0 con 0x7f43f80aedf0 2023-12-01T01:52:50.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.760+0000 7f43f5ffb640 1 -- 172.21.15.115:0/313681317 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0031d60 con 0x7f43f80aedf0 2023-12-01T01:52:50.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.760+0000 7f43f7fff640 1 -- 172.21.15.115:0/313681317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f80aedf0 msgr2=0x7f43f8070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:50.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.760+0000 7f43f7fff640 1 --2- 172.21.15.115:0/313681317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f80aedf0 0x7f43f8070140 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f43f806f870 tx=0x7f43f002b910 comp rx=0 tx=0).stop 2023-12-01T01:52:50.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.761+0000 7f43f7fff640 1 -- 172.21.15.115:0/313681317 shutdown_connections 2023-12-01T01:52:50.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.761+0000 7f43f7fff640 1 --2- 172.21.15.115:0/313681317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f80aedf0 0x7f43f8070140 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.761+0000 7f43f7fff640 1 --2- 172.21.15.115:0/313681317 >> v2:172.21.15.142:3300/0 conn(0x7f43f80ae6f0 0x7f43f8065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.761+0000 7f43f7fff640 1 --2- 172.21.15.115:0/313681317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f815cdf0 0x7f43f8065960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.761+0000 7f43f7fff640 1 -- 172.21.15.115:0/313681317 >> 172.21.15.115:0/313681317 conn(0x7f43f80cf6f0 msgr2=0x7f43f80cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:50.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.761+0000 7f43f7fff640 1 -- 172.21.15.115:0/313681317 shutdown_connections 2023-12-01T01:52:50.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.761+0000 7f43f7fff640 1 -- 172.21.15.115:0/313681317 wait complete. 2023-12-01T01:52:50.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.762+0000 7f43f7fff640 1 Processor -- start 2023-12-01T01:52:50.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.763+0000 7f43f7fff640 1 -- start start 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.763+0000 7f43f7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f80ae6f0 0x7f43f81e6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.763+0000 7f43f7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80aedf0 0x7f43f815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.763+0000 7f43f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f815cdf0 0x7f43f815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.763+0000 7f43f7fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f81e72f0 con 0x7f43f815cdf0 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.763+0000 7f43f7fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f80b2b40 con 0x7f43f80aedf0 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.763+0000 7f43f7fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f80b2d90 con 0x7f43f80ae6f0 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.764+0000 7f43f77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f815cdf0 0x7f43f815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.764+0000 7f43f77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f815cdf0 0x7f43f815c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43794/0 (socket says 172.21.15.115:43794) 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.764+0000 7f43f77fe640 1 -- 172.21.15.115:0/4159024026 learned_addr learned my addr 172.21.15.115:0/4159024026 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.764+0000 7f43f6ffd640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f80ae6f0 0x7f43f81e6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.764+0000 7f43f77fe640 1 -- 172.21.15.115:0/4159024026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f80ae6f0 msgr2=0x7f43f81e6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:50.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.764+0000 7f43f77fe640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f80ae6f0 0x7f43f81e6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.764+0000 7f43f77fe640 1 -- 172.21.15.115:0/4159024026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80aedf0 msgr2=0x7f43f815baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:50.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.764+0000 7f43f77fe640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80aedf0 0x7f43f815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:50.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.764+0000 7f43f77fe640 1 -- 172.21.15.115:0/4159024026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f00094d0 con 0x7f43f815cdf0 2023-12-01T01:52:50.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.764+0000 7f43f6ffd640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f80ae6f0 0x7f43f81e6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:50.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.765+0000 7f43f77fe640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f815cdf0 0x7f43f815c030 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f43f00090b0 tx=0x7f43f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:50.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.765+0000 7f43ef7fe640 1 -- 172.21.15.115:0/4159024026 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f003ac60 con 0x7f43f815cdf0 2023-12-01T01:52:50.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.765+0000 7f43ef7fe640 1 -- 172.21.15.115:0/4159024026 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f003adc0 con 0x7f43f815cdf0 2023-12-01T01:52:50.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.765+0000 7f43f7fff640 1 -- 172.21.15.115:0/4159024026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f80b3070 con 0x7f43f815cdf0 2023-12-01T01:52:50.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.765+0000 7f43ef7fe640 1 -- 172.21.15.115:0/4159024026 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0044630 con 0x7f43f815cdf0 2023-12-01T01:52:50.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.765+0000 7f43f7fff640 1 -- 172.21.15.115:0/4159024026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f80b3630 con 0x7f43f815cdf0 2023-12-01T01:52:50.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.766+0000 7f43f7fff640 1 -- 172.21.15.115:0/4159024026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f80cbd40 con 0x7f43f815cdf0 2023-12-01T01:52:50.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.767+0000 7f43ef7fe640 1 -- 172.21.15.115:0/4159024026 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f43f0044790 con 0x7f43f815cdf0 2023-12-01T01:52:50.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.768+0000 7f43ef7fe640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f43d806a5a0 0x7f43d806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:50.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.768+0000 7f43ef7fe640 1 -- 172.21.15.115:0/4159024026 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43f002d040 con 0x7f43f815cdf0 2023-12-01T01:52:50.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.768+0000 7f43f6ffd640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f43d806a5a0 0x7f43d806ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:50.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.769+0000 7f43f6ffd640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f43d806a5a0 0x7f43d806ca60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43f81f56e0 tx=0x7f43e800b040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:50.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:50.773+0000 7f43ef7fe640 1 -- 172.21.15.115:0/4159024026 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f43f0076650 con 0x7f43f815cdf0 2023-12-01T01:52:51.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.004+0000 7f43f7fff640 1 -- 172.21.15.115:0/4159024026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f43f80618d0 con 0x7f43f815cdf0 2023-12-01T01:52:51.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.530+0000 7f43ef7fe640 1 -- 172.21.15.115:0/4159024026 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7f43f00350c0 con 0x7f43f815cdf0 2023-12-01T01:52:51.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 -- 172.21.15.115:0/4159024026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f43d806a5a0 msgr2=0x7f43d806ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f43d806a5a0 0x7f43d806ca60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43f81f56e0 tx=0x7f43e800b040 comp rx=0 tx=0).stop 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 -- 172.21.15.115:0/4159024026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f815cdf0 msgr2=0x7f43f815c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f815cdf0 0x7f43f815c030 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f43f00090b0 tx=0x7f43f00046c0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 -- 172.21.15.115:0/4159024026 shutdown_connections 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f43d806a5a0 0x7f43d806ca60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43f815cdf0 0x7f43f815c030 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43f80aedf0 0x7f43f815baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 --2- 172.21.15.115:0/4159024026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43f80ae6f0 0x7f43f81e6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 -- 172.21.15.115:0/4159024026 >> 172.21.15.115:0/4159024026 conn(0x7f43f80cf6f0 msgr2=0x7f43f80cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 -- 172.21.15.115:0/4159024026 shutdown_connections 2023-12-01T01:52:51.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.533+0000 7f43f7fff640 1 -- 172.21.15.115:0/4159024026 wait complete. 2023-12-01T01:52:51.555 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T01:52:51.555 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:52:51.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.765+0000 7f4a20421640 1 Processor -- start 2023-12-01T01:52:51.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.765+0000 7f4a20421640 1 -- start start 2023-12-01T01:52:51.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.766+0000 7f4a20421640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4a180c9ef0 0x7f4a1806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:51.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.766+0000 7f4a20421640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1806f8b0 con 0x7f4a18159dd0 2023-12-01T01:52:51.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.766+0000 7f4a20421640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a180adcb0 con 0x7f4a1806b6f0 2023-12-01T01:52:51.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.766+0000 7f4a20421640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a180ad9d0 con 0x7f4a180c9ef0 2023-12-01T01:52:51.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.767+0000 7f4a1e1bd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4a18159dd0 0x7f4a180c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:33108/0 (socket says 172.21.15.115:33108) 2023-12-01T01:52:51.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.767+0000 7f4a1e1bd640 1 -- 172.21.15.115:0/1269374522 learned_addr learned my addr 172.21.15.115:0/1269374522 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:51.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.767+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221579592 0 0) 0x7f4a1806f8b0 con 0x7f4a18159dd0 2023-12-01T01:52:51.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.767+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a04003520 con 0x7f4a18159dd0 2023-12-01T01:52:51.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.767+0000 7f4a1d9bc640 1 --2- 172.21.15.115:0/1269374522 >> v2:172.21.15.142:3300/0 conn(0x7f4a180c9ef0 0x7f4a1806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:51.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2525371873 0 0) 0x7f4a04003520 con 0x7f4a18159dd0 2023-12-01T01:52:51.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 >> v2:172.21.15.142:3300/0 conn(0x7f4a180c9ef0 msgr2=0x7f4a1806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:51.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 --2- 172.21.15.115:0/1269374522 >> v2:172.21.15.142:3300/0 conn(0x7f4a180c9ef0 0x7f4a1806fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 >> v1:172.21.15.142:6789/0 conn(0x7f4a1806b6f0 legacy=0x7f4a18070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:51.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180c9000 con 0x7f4a18159dd0 2023-12-01T01:52:51.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4a14001670 con 0x7f4a18159dd0 2023-12-01T01:52:51.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 >> v1:172.21.15.115:6789/0 conn(0x7f4a18159dd0 legacy=0x7f4a180c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a040039e0 0x7f4a04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a040047c0 0x7f4a04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a04005380 0x7f4a0400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180ad9d0 con 0x7f4a04005380 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0400e470 con 0x7f4a040039e0 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0400e630 con 0x7f4a040047c0 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1e9be640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a04005380 0x7f4a0400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1e9be640 1 -- 172.21.15.115:0/1269374522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a040047c0 msgr2=0x7f4a04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1d9bc640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a040047c0 0x7f4a04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1e9be640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a040047c0 0x7f4a04004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1e1bd640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a040039e0 0x7f4a04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:51.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.768+0000 7f4a1e9be640 1 -- 172.21.15.115:0/1269374522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a040039e0 msgr2=0x7f4a04003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:51.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.769+0000 7f4a1e9be640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a040039e0 0x7f4a04003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.769+0000 7f4a1e9be640 1 -- 172.21.15.115:0/1269374522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a04003520 con 0x7f4a04005380 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.769+0000 7f4a1e9be640 1 -- 172.21.15.115:0/1269374522 reap_dead start 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.770+0000 7f4a1e9be640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a04005380 0x7f4a0400db90 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4a100044b0 tx=0x7f4a040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534189e29a83ea9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.770+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10006d50 con 0x7f4a04005380 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.770+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a10007020 con 0x7f4a04005380 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.770+0000 7f4a1d1bb640 1 -- 172.21.15.115:0/1269374522 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a100073d0 con 0x7f4a04005380 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.770+0000 7f4a20421640 1 -- 172.21.15.115:0/1269374522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a04005380 msgr2=0x7f4a0400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.770+0000 7f4a20421640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a04005380 0x7f4a0400db90 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4a100044b0 tx=0x7f4a040057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.771+0000 7f4a20421640 1 -- 172.21.15.115:0/1269374522 shutdown_connections 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.771+0000 7f4a20421640 1 --2- 172.21.15.115:0/1269374522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a04005380 0x7f4a0400db90 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.771+0000 7f4a20421640 1 -- 172.21.15.115:0/1269374522 >> 172.21.15.115:0/1269374522 conn(0x7f4a180ce880 msgr2=0x7f4a180cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:51.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.771+0000 7f4a20421640 1 -- 172.21.15.115:0/1269374522 shutdown_connections 2023-12-01T01:52:51.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.771+0000 7f4a20421640 1 -- 172.21.15.115:0/1269374522 wait complete. 2023-12-01T01:52:51.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.772+0000 7f4a20421640 1 Processor -- start 2023-12-01T01:52:51.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.773+0000 7f4a20421640 1 -- start start 2023-12-01T01:52:51.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.773+0000 7f4a20421640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a04005380 0x7f4a180af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:51.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.773+0000 7f4a20421640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a180c2dc0 0x7f4a181f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:51.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.773+0000 7f4a20421640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a180c3240 0x7f4a181f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:51.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.773+0000 7f4a20421640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c4490 con 0x7f4a180c3240 2023-12-01T01:52:51.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.773+0000 7f4a20421640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c3810 con 0x7f4a04005380 2023-12-01T01:52:51.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.773+0000 7f4a20421640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c3af0 con 0x7f4a180c2dc0 2023-12-01T01:52:51.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.774+0000 7f4a1e9be640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a180c3240 0x7f4a181f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:51.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.774+0000 7f4a1e9be640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a180c3240 0x7f4a181f0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43806/0 (socket says 172.21.15.115:43806) 2023-12-01T01:52:51.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.774+0000 7f4a1e9be640 1 -- 172.21.15.115:0/2466050750 learned_addr learned my addr 172.21.15.115:0/2466050750 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.774+0000 7f4a1d9bc640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a180c2dc0 0x7f4a181f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.774+0000 7f4a1e1bd640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a04005380 0x7f4a180af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.774+0000 7f4a1e9be640 1 -- 172.21.15.115:0/2466050750 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a180c2dc0 msgr2=0x7f4a181f04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.774+0000 7f4a1e9be640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a180c2dc0 0x7f4a181f04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.774+0000 7f4a1e9be640 1 -- 172.21.15.115:0/2466050750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a04005380 msgr2=0x7f4a180af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.774+0000 7f4a1e9be640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a04005380 0x7f4a180af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.774+0000 7f4a1e9be640 1 -- 172.21.15.115:0/2466050750 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180c3dd0 con 0x7f4a180c3240 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.775+0000 7f4a1e9be640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a180c3240 0x7f4a181f0a10 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4a10000f80 tx=0x7f4a10003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.775+0000 7f4a0effd640 1 -- 172.21.15.115:0/2466050750 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1003e070 con 0x7f4a180c3240 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.775+0000 7f4a0effd640 1 -- 172.21.15.115:0/2466050750 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a10006eb0 con 0x7f4a180c3240 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.775+0000 7f4a0effd640 1 -- 172.21.15.115:0/2466050750 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a100078d0 con 0x7f4a180c3240 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.775+0000 7f4a20421640 1 -- 172.21.15.115:0/2466050750 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a180c40b0 con 0x7f4a180c3240 2023-12-01T01:52:51.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.775+0000 7f4a20421640 1 -- 172.21.15.115:0/2466050750 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a180afe80 con 0x7f4a180c3240 2023-12-01T01:52:51.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.776+0000 7f4a20421640 1 -- 172.21.15.115:0/2466050750 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1806d0d0 con 0x7f4a180c3240 2023-12-01T01:52:51.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.777+0000 7f4a0effd640 1 -- 172.21.15.115:0/2466050750 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4a10030420 con 0x7f4a180c3240 2023-12-01T01:52:51.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.778+0000 7f4a0effd640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f49fc06a550 0x7f49fc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:51.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.778+0000 7f4a0effd640 1 -- 172.21.15.115:0/2466050750 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a100a9d80 con 0x7f4a180c3240 2023-12-01T01:52:51.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.778+0000 7f4a1e1bd640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f49fc06a550 0x7f49fc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:51.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.779+0000 7f4a1e1bd640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f49fc06a550 0x7f49fc06ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4a140018e0 tx=0x7f4a14009040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:51.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:51.782+0000 7f4a0effd640 1 -- 172.21.15.115:0/2466050750 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4a10075160 con 0x7f4a180c3240 2023-12-01T01:52:52.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.012+0000 7f4a20421640 1 -- 172.21.15.115:0/2466050750 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a180ca2d0 con 0x7f4a180c3240 2023-12-01T01:52:52.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.013+0000 7f4a0effd640 1 -- 172.21.15.115:0/2466050750 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f4a10034090 con 0x7f4a180c3240 2023-12-01T01:52:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:52.015 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3538792232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3538792232},{"type":"v1","addr":"172.21.15.142:6835","nonce":3538792232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7449,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3171906828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3171906828},{"type":"v1","addr":"172.21.15.115:6835","nonce":3171906828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7452,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2130806226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2130806226},{"type":"v1","addr":"172.21.15.115:6837","nonce":2130806226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:48.491310+0000","modified":"2023-12-01T01:52:51.513496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7455":{"gid":7455,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/3108729597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3108729597},{"type":"v1","addr":"172.21.15.142:6834","nonce":3108729597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-01T01:52:52.015 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 48 2023-12-01T01:52:52.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 -- 172.21.15.115:0/2466050750 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f49fc06a550 msgr2=0x7f49fc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f49fc06a550 0x7f49fc06ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4a140018e0 tx=0x7f4a14009040 comp rx=0 tx=0).stop 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 -- 172.21.15.115:0/2466050750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a180c3240 msgr2=0x7f4a181f0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a180c3240 0x7f4a181f0a10 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4a10000f80 tx=0x7f4a10003a70 comp rx=0 tx=0).stop 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 -- 172.21.15.115:0/2466050750 shutdown_connections 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f49fc06a550 0x7f49fc06ca10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4a180c2dc0 0x7f4a181f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a180c3240 0x7f4a181f0a10 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 --2- 172.21.15.115:0/2466050750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a04005380 0x7f4a180af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 -- 172.21.15.115:0/2466050750 >> 172.21.15.115:0/2466050750 conn(0x7f4a180ce880 msgr2=0x7f4a18158e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 -- 172.21.15.115:0/2466050750 shutdown_connections 2023-12-01T01:52:52.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.016+0000 7f4a20421640 1 -- 172.21.15.115:0/2466050750 wait complete. 2023-12-01T01:52:52.036 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:52:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.246+0000 7f078ff00640 1 Processor -- start 2023-12-01T01:52:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.246+0000 7f078ff00640 1 -- start start 2023-12-01T01:52:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.248+0000 7f078ff00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f07880c9ef0 0x7f078806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.248+0000 7f078ff00640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078806f8b0 con 0x7f078806b6f0 2023-12-01T01:52:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.248+0000 7f078ff00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07880adcb0 con 0x7f0788159dd0 2023-12-01T01:52:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.248+0000 7f078ff00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07880ad9d0 con 0x7f07880c9ef0 2023-12-01T01:52:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.248+0000 7f078e49d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f078806b6f0 0x7f0788070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:33118/0 (socket says 172.21.15.115:33118) 2023-12-01T01:52:52.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.248+0000 7f078e49d640 1 -- 172.21.15.115:0/1101388540 learned_addr learned my addr 172.21.15.115:0/1101388540 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078d49b640 1 --2- 172.21.15.115:0/1101388540 >> v2:172.21.15.142:3300/0 conn(0x7f07880c9ef0 0x7f078806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466952098 0 0) 0x7f078806f8b0 con 0x7f078806b6f0 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0774003520 con 0x7f078806b6f0 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688210502 0 0) 0x7f0774003520 con 0x7f078806b6f0 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 >> v2:172.21.15.142:3300/0 conn(0x7f07880c9ef0 msgr2=0x7f078806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 --2- 172.21.15.115:0/1101388540 >> v2:172.21.15.142:3300/0 conn(0x7f07880c9ef0 0x7f078806fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 >> v1:172.21.15.142:6789/0 conn(0x7f0788159dd0 legacy=0x7f07880c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880c9000 con 0x7f078806b6f0 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0780001670 con 0x7f078806b6f0 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 >> v1:172.21.15.115:6789/0 conn(0x7f078806b6f0 legacy=0x7f0788070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 --2- 172.21.15.115:0/1101388540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07740039e0 0x7f0774003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.249+0000 7f078cc9a640 1 --2- 172.21.15.115:0/1101388540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07740047c0 0x7f0774004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078cc9a640 1 --2- 172.21.15.115:0/1101388540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0774005280 0x7f077400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07880ad9d0 con 0x7f07740039e0 2023-12-01T01:52:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077400c1c0 con 0x7f0774005280 2023-12-01T01:52:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077400e590 con 0x7f07740047c0 2023-12-01T01:52:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078dc9c640 1 --2- 172.21.15.115:0/1101388540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07740039e0 0x7f0774003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078dc9c640 1 -- 172.21.15.115:0/1101388540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07740047c0 msgr2=0x7f0774004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078dc9c640 1 --2- 172.21.15.115:0/1101388540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07740047c0 0x7f0774004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078dc9c640 1 -- 172.21.15.115:0/1101388540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0774005280 msgr2=0x7f077400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078dc9c640 1 --2- 172.21.15.115:0/1101388540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0774005280 0x7f077400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078dc9c640 1 -- 172.21.15.115:0/1101388540 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0774003520 con 0x7f07740039e0 2023-12-01T01:52:52.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.250+0000 7f078e49d640 1 -- 172.21.15.115:0/1101388540 reap_dead start 2023-12-01T01:52:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.251+0000 7f078dc9c640 1 --2- 172.21.15.115:0/1101388540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07740039e0 0x7f0774003e00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0784001e50 tx=0x7f07740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db093f948c38cf97 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.251+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784003b70 con 0x7f07740039e0 2023-12-01T01:52:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.251+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0784003cd0 con 0x7f07740039e0 2023-12-01T01:52:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.251+0000 7f078cc9a640 1 -- 172.21.15.115:0/1101388540 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784005310 con 0x7f07740039e0 2023-12-01T01:52:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.252+0000 7f078ff00640 1 -- 172.21.15.115:0/1101388540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07740039e0 msgr2=0x7f0774003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.252+0000 7f078ff00640 1 --2- 172.21.15.115:0/1101388540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07740039e0 0x7f0774003e00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0784001e50 tx=0x7f07740057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.252+0000 7f078ff00640 1 -- 172.21.15.115:0/1101388540 shutdown_connections 2023-12-01T01:52:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.252+0000 7f078ff00640 1 --2- 172.21.15.115:0/1101388540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07740039e0 0x7f0774003e00 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.252+0000 7f078ff00640 1 -- 172.21.15.115:0/1101388540 >> 172.21.15.115:0/1101388540 conn(0x7f07880ce880 msgr2=0x7f07880cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.252+0000 7f078ff00640 1 -- 172.21.15.115:0/1101388540 shutdown_connections 2023-12-01T01:52:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.252+0000 7f078ff00640 1 -- 172.21.15.115:0/1101388540 wait complete. 2023-12-01T01:52:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.254+0000 7f078ff00640 1 Processor -- start 2023-12-01T01:52:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.254+0000 7f078ff00640 1 -- start start 2023-12-01T01:52:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.255+0000 7f078ff00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07740039e0 0x7f07880c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.255+0000 7f078ff00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078815a4b0 0x7f07881568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.255+0000 7f078ff00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07881f5c30 0x7f0788157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.255+0000 7f078ff00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07880c8d40 con 0x7f07881f5c30 2023-12-01T01:52:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.255+0000 7f078ff00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078815a9b0 con 0x7f07740039e0 2023-12-01T01:52:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.255+0000 7f078ff00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078815ac90 con 0x7f078815a4b0 2023-12-01T01:52:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.255+0000 7f078e49d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07881f5c30 0x7f0788157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.255+0000 7f078e49d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07881f5c30 0x7f0788157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43828/0 (socket says 172.21.15.115:43828) 2023-12-01T01:52:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.255+0000 7f078e49d640 1 -- 172.21.15.115:0/1519748329 learned_addr learned my addr 172.21.15.115:0/1519748329 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f078d49b640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078815a4b0 0x7f07881568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f078dc9c640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07740039e0 0x7f07880c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f078e49d640 1 -- 172.21.15.115:0/1519748329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078815a4b0 msgr2=0x7f07881568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f078e49d640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078815a4b0 0x7f07881568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f078e49d640 1 -- 172.21.15.115:0/1519748329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07740039e0 msgr2=0x7f07880c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f078e49d640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07740039e0 0x7f07880c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f078e49d640 1 -- 172.21.15.115:0/1519748329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078815af70 con 0x7f07881f5c30 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f078e49d640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07881f5c30 0x7f0788157c60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f07880cacc0 tx=0x7f078000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f077e7fc640 1 -- 172.21.15.115:0/1519748329 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780007440 con 0x7f07881f5c30 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f077e7fc640 1 -- 172.21.15.115:0/1519748329 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07800075a0 con 0x7f07881f5c30 2023-12-01T01:52:52.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f077e7fc640 1 -- 172.21.15.115:0/1519748329 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780004a20 con 0x7f07881f5c30 2023-12-01T01:52:52.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.256+0000 7f078ff00640 1 -- 172.21.15.115:0/1519748329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078815b250 con 0x7f07881f5c30 2023-12-01T01:52:52.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.257+0000 7f078ff00640 1 -- 172.21.15.115:0/1519748329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07881573b0 con 0x7f07881f5c30 2023-12-01T01:52:52.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.258+0000 7f078ff00640 1 -- 172.21.15.115:0/1519748329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078815c6b0 con 0x7f07881f5c30 2023-12-01T01:52:52.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.259+0000 7f077e7fc640 1 -- 172.21.15.115:0/1519748329 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0780004b80 con 0x7f07881f5c30 2023-12-01T01:52:52.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.260+0000 7f077e7fc640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f076c06a550 0x7f076c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.260+0000 7f077e7fc640 1 -- 172.21.15.115:0/1519748329 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0780087c70 con 0x7f07881f5c30 2023-12-01T01:52:52.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.260+0000 7f078dc9c640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f076c06a550 0x7f076c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.261+0000 7f078dc9c640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f076c06a550 0x7f076c06ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0784034000 tx=0x7f0784009030 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:52.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.264+0000 7f077e7fc640 1 -- 172.21.15.115:0/1519748329 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f078815c6b0 con 0x7f07881f5c30 2023-12-01T01:52:52.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.457+0000 7f078ff00640 1 -- 172.21.15.115:0/1519748329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f078815c6b0 con 0x7f07881f5c30 2023-12-01T01:52:52.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.460+0000 7f077e7fc640 1 -- 172.21.15.115:0/1519748329 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16163 (secure 0 0 0) 0x7f078815c6b0 con 0x7f07881f5c30 2023-12-01T01:52:52.461 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:52.462 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":111,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:52:49.495660+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:46.948807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:47.513140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:52:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.462+0000 7f078ff00640 1 -- 172.21.15.115:0/1519748329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f076c06a550 msgr2=0x7f076c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.462+0000 7f078ff00640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f076c06a550 0x7f076c06ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0784034000 tx=0x7f0784009030 comp rx=0 tx=0).stop 2023-12-01T01:52:52.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.462+0000 7f078ff00640 1 -- 172.21.15.115:0/1519748329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07881f5c30 msgr2=0x7f0788157c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.462+0000 7f078ff00640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07881f5c30 0x7f0788157c60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f07880cacc0 tx=0x7f078000c890 comp rx=0 tx=0).stop 2023-12-01T01:52:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.463+0000 7f078ff00640 1 -- 172.21.15.115:0/1519748329 shutdown_connections 2023-12-01T01:52:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.463+0000 7f078ff00640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f076c06a550 0x7f076c06ca10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.463+0000 7f078ff00640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f078815a4b0 0x7f07881568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.463+0000 7f078ff00640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07881f5c30 0x7f0788157c60 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.463+0000 7f078ff00640 1 --2- 172.21.15.115:0/1519748329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07740039e0 0x7f07880c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.463+0000 7f078ff00640 1 -- 172.21.15.115:0/1519748329 >> 172.21.15.115:0/1519748329 conn(0x7f07880ce880 msgr2=0x7f07880cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.463+0000 7f078ff00640 1 -- 172.21.15.115:0/1519748329 shutdown_connections 2023-12-01T01:52:52.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.463+0000 7f078ff00640 1 -- 172.21.15.115:0/1519748329 wait complete. 2023-12-01T01:52:52.484 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T01:52:52.485 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:52:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.697+0000 7f7dd3fff640 1 Processor -- start 2023-12-01T01:52:52.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.698+0000 7f7dd3fff640 1 -- start start 2023-12-01T01:52:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.699+0000 7f7dd3fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7dd415cb40 0x7f7dd40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.699+0000 7f7dd3fff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd40d17c0 con 0x7f7dd40cf980 2023-12-01T01:52:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.699+0000 7f7dd3fff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd40d19f0 con 0x7f7dd40d13b0 2023-12-01T01:52:52.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.699+0000 7f7dd3fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd40d1f90 con 0x7f7dd415cb40 2023-12-01T01:52:52.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.700+0000 7f7dd27fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7dd40cf980 0x7f7dd40d0ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:33132/0 (socket says 172.21.15.115:33132) 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.700+0000 7f7dd27fc640 1 -- 172.21.15.115:0/2709435103 learned_addr learned my addr 172.21.15.115:0/2709435103 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.700+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/2709435103 >> v2:172.21.15.142:3300/0 conn(0x7f7dd415cb40 0x7f7dd40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.700+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787073397 0 0) 0x7f7dd40d17c0 con 0x7f7dd40cf980 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.700+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc0003520 con 0x7f7dd40cf980 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 594685777 0 0) 0x7f7dc0003520 con 0x7f7dd40cf980 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 >> v2:172.21.15.142:3300/0 conn(0x7f7dd415cb40 msgr2=0x7f7dd40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 --2- 172.21.15.115:0/2709435103 >> v2:172.21.15.142:3300/0 conn(0x7f7dd415cb40 0x7f7dd40b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 >> v1:172.21.15.142:6789/0 conn(0x7f7dd40d13b0 legacy=0x7f7dd40d2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd40d21a0 con 0x7f7dd40cf980 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f7dbc001670 con 0x7f7dd40cf980 2023-12-01T01:52:52.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 >> v1:172.21.15.115:6789/0 conn(0x7f7dd40cf980 legacy=0x7f7dd40d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dc0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dc00047c0 0x7f7dc0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dc00052a0 0x7f7dc000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40d1f90 con 0x7f7dc0003a30 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc000c170 con 0x7f7dc00052a0 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc000c330 con 0x7f7dc00047c0 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.701+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dc0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.702+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/2709435103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dc00047c0 msgr2=0x7f7dc0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.702+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dc00047c0 0x7f7dc0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.702+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/2709435103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dc00052a0 msgr2=0x7f7dc000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.702+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dc00052a0 0x7f7dc000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.702+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dc00052a0 0x7f7dc000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.702+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/2709435103 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc0003520 con 0x7f7dc0003a30 2023-12-01T01:52:52.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.702+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dc00052a0 0x7f7dc000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:52.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.702+0000 7f7dd37fe640 1 -- 172.21.15.115:0/2709435103 reap_dead start 2023-12-01T01:52:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.703+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dc0003e50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009fc0 tx=0x7f7dc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8dd31d8a0a90b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.703+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc402cc20 con 0x7f7dc0003a30 2023-12-01T01:52:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.703+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc400f040 con 0x7f7dc0003a30 2023-12-01T01:52:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.703+0000 7f7dd1ffb640 1 -- 172.21.15.115:0/2709435103 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4034440 con 0x7f7dc0003a30 2023-12-01T01:52:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.703+0000 7f7dd3fff640 1 -- 172.21.15.115:0/2709435103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 msgr2=0x7f7dc0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.703+0000 7f7dd3fff640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dc0003e50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009fc0 tx=0x7f7dc00057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.704+0000 7f7dd3fff640 1 -- 172.21.15.115:0/2709435103 shutdown_connections 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.704+0000 7f7dd3fff640 1 --2- 172.21.15.115:0/2709435103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dc0003e50 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.704+0000 7f7dd3fff640 1 -- 172.21.15.115:0/2709435103 >> 172.21.15.115:0/2709435103 conn(0x7f7dd406a0e0 msgr2=0x7f7dd406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.704+0000 7f7dd3fff640 1 -- 172.21.15.115:0/2709435103 shutdown_connections 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.704+0000 7f7dd3fff640 1 -- 172.21.15.115:0/2709435103 wait complete. 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.705+0000 7f7dd3fff640 1 Processor -- start 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.706+0000 7f7dd3fff640 1 -- start start 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.706+0000 7f7dd3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dd4153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.706+0000 7f7dd3fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd4152200 0x7f7dd415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd3fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd415fbe0 0x7f7dd40cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd3fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd415e600 con 0x7f7dc0003a30 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd3fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd415e8e0 con 0x7f7dd4152200 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd3fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd415ebc0 con 0x7f7dd415fbe0 2023-12-01T01:52:52.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dd4153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dd4153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43850/0 (socket says 172.21.15.115:43850) 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/838320779 learned_addr learned my addr 172.21.15.115:0/838320779 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd37fe640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd415fbe0 0x7f7dd40cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/838320779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd415fbe0 msgr2=0x7f7dd40cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd415fbe0 0x7f7dd40cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/838320779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd4152200 msgr2=0x7f7dd415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd4152200 0x7f7dd415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.707+0000 7f7dd2ffd640 1 -- 172.21.15.115:0/838320779 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd415eea0 con 0x7f7dc0003a30 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.708+0000 7f7dd2ffd640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dd4153940 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7dc402c5c0 tx=0x7f7dc4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.708+0000 7f7dcb7fe640 1 -- 172.21.15.115:0/838320779 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4034e50 con 0x7f7dc0003a30 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.708+0000 7f7dcb7fe640 1 -- 172.21.15.115:0/838320779 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc4003990 con 0x7f7dc0003a30 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.708+0000 7f7dcb7fe640 1 -- 172.21.15.115:0/838320779 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4003d00 con 0x7f7dc0003a30 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.708+0000 7f7dd3fff640 1 -- 172.21.15.115:0/838320779 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd415f180 con 0x7f7dc0003a30 2023-12-01T01:52:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.708+0000 7f7dd3fff640 1 -- 172.21.15.115:0/838320779 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd4153550 con 0x7f7dc0003a30 2023-12-01T01:52:52.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.709+0000 7f7dd3fff640 1 -- 172.21.15.115:0/838320779 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd415ea20 con 0x7f7dc0003a30 2023-12-01T01:52:52.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.710+0000 7f7dcb7fe640 1 -- 172.21.15.115:0/838320779 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f7dc4004030 con 0x7f7dc0003a30 2023-12-01T01:52:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.711+0000 7f7dcb7fe640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7db406a550 0x7f7db406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.711+0000 7f7dcb7fe640 1 -- 172.21.15.115:0/838320779 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dc40aa550 con 0x7f7dc0003a30 2023-12-01T01:52:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.711+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7db406a550 0x7f7db406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.712+0000 7f7dd27fc640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7db406a550 0x7f7db406ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc000fd0 tx=0x7f7dbc001510 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:52.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.715+0000 7f7dcb7fe640 1 -- 172.21.15.115:0/838320779 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7dd415ea20 con 0x7f7dc0003a30 2023-12-01T01:52:52.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.901+0000 7f7dd3fff640 1 -- 172.21.15.115:0/838320779 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7dd415ed30 con 0x7f7db406a550 2023-12-01T01:52:52.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.914+0000 7f7dcb7fe640 1 -- 172.21.15.115:0/838320779 <== mgr.7230 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284309 (secure 0 0 0) 0x7f7dd415ed30 con 0x7f7db406a550 2023-12-01T01:52:52.919 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:52.921 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:52:52.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 -- 172.21.15.115:0/838320779 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7db406a550 msgr2=0x7f7db406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7db406a550 0x7f7db406ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc000fd0 tx=0x7f7dbc001510 comp rx=0 tx=0).stop 2023-12-01T01:52:52.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 -- 172.21.15.115:0/838320779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 msgr2=0x7f7dd4153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:52.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dd4153940 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7dc402c5c0 tx=0x7f7dc4009750 comp rx=0 tx=0).stop 2023-12-01T01:52:52.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 -- 172.21.15.115:0/838320779 shutdown_connections 2023-12-01T01:52:52.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7db406a550 0x7f7db406ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd415fbe0 0x7f7dd40cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd4152200 0x7f7dd415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 --2- 172.21.15.115:0/838320779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7dc0003a30 0x7f7dd4153940 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:52.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 -- 172.21.15.115:0/838320779 >> 172.21.15.115:0/838320779 conn(0x7f7dd406a0e0 msgr2=0x7f7dd415c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:52.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 -- 172.21.15.115:0/838320779 shutdown_connections 2023-12-01T01:52:52.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:52.920+0000 7f7dd3fff640 1 -- 172.21.15.115:0/838320779 wait complete. 2023-12-01T01:52:52.942 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T01:52:52.106719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316488,"kb_used_data":11528,"kb_used_omap":0,"kb_used_meta":304896,"kb_avail":754658232,"statfs":{"total":773094113280,"available":772770029568,"internally_reserved":0,"allocated":11804672,"data_stored":6216712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312213504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.102080"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.031286+0000","last_change":"2023-12-01T01:52:48.031286+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.031286+0000","last_undegraded":"2023-12-01T01:52:48.031286+0000","last_fullsized":"2023-12-01T01:52:48.031286+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:07:01.922435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.022478+0000","last_change":"2023-12-01T01:52:48.022478+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.022478+0000","last_undegraded":"2023-12-01T01:52:48.022478+0000","last_fullsized":"2023-12-01T01:52:48.022478+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:20.199417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.034568+0000","last_change":"2023-12-01T01:52:48.034568+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.034568+0000","last_undegraded":"2023-12-01T01:52:48.034568+0000","last_fullsized":"2023-12-01T01:52:48.034568+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029059+0000","last_change":"2023-12-01T01:52:48.029059+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029059+0000","last_undegraded":"2023-12-01T01:52:48.029059+0000","last_fullsized":"2023-12-01T01:52:48.029059+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:06:59.600929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.022456+0000","last_change":"2023-12-01T01:52:48.022456+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.022456+0000","last_undegraded":"2023-12-01T01:52:48.022456+0000","last_fullsized":"2023-12-01T01:52:48.022456+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:48:34.887669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.033630+0000","last_change":"2023-12-01T01:52:48.033630+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.033630+0000","last_undegraded":"2023-12-01T01:52:48.033630+0000","last_fullsized":"2023-12-01T01:52:48.033630+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:39:18.420163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020737+0000","last_change":"2023-12-01T01:52:48.020737+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020737+0000","last_undegraded":"2023-12-01T01:52:48.020737+0000","last_fullsized":"2023-12-01T01:52:48.020737+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:51.221046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024562+0000","last_change":"2023-12-01T01:52:48.024562+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024562+0000","last_undegraded":"2023-12-01T01:52:48.024562+0000","last_fullsized":"2023-12-01T01:52:48.024562+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:21.099958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026842+0000","last_change":"2023-12-01T01:52:48.026842+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026842+0000","last_undegraded":"2023-12-01T01:52:48.026842+0000","last_fullsized":"2023-12-01T01:52:48.026842+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:52.282595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.034705+0000","last_change":"2023-12-01T01:52:48.034705+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.034705+0000","last_undegraded":"2023-12-01T01:52:48.034705+0000","last_fullsized":"2023-12-01T01:52:48.034705+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029077+0000","last_change":"2023-12-01T01:52:48.029077+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029077+0000","last_undegraded":"2023-12-01T01:52:48.029077+0000","last_fullsized":"2023-12-01T01:52:48.029077+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025888+0000","last_change":"2023-12-01T01:52:48.025888+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025888+0000","last_undegraded":"2023-12-01T01:52:48.025888+0000","last_fullsized":"2023-12-01T01:52:48.025888+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021972+0000","last_change":"2023-12-01T01:52:48.021972+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021972+0000","last_undegraded":"2023-12-01T01:52:48.021972+0000","last_fullsized":"2023-12-01T01:52:48.021972+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:45:17.044474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.037434+0000","last_change":"2023-12-01T01:52:48.037434+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.037434+0000","last_undegraded":"2023-12-01T01:52:48.037434+0000","last_fullsized":"2023-12-01T01:52:48.037434+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:40:59.300637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.019592+0000","last_change":"2023-12-01T01:52:48.019592+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.019592+0000","last_undegraded":"2023-12-01T01:52:48.019592+0000","last_fullsized":"2023-12-01T01:52:48.019592+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:58:23.518943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029503+0000","last_change":"2023-12-01T01:52:48.029503+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029503+0000","last_undegraded":"2023-12-01T01:52:48.029503+0000","last_fullsized":"2023-12-01T01:52:48.029503+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024646+0000","last_change":"2023-12-01T01:52:48.024646+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024646+0000","last_undegraded":"2023-12-01T01:52:48.024646+0000","last_fullsized":"2023-12-01T01:52:48.024646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029391+0000","last_change":"2023-12-01T01:52:48.029391+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029391+0000","last_undegraded":"2023-12-01T01:52:48.029391+0000","last_fullsized":"2023-12-01T01:52:48.029391+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.038499+0000","last_change":"2023-12-01T01:52:48.038499+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.038499+0000","last_undegraded":"2023-12-01T01:52:48.038499+0000","last_fullsized":"2023-12-01T01:52:48.038499+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:13:33.646688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035539+0000","last_change":"2023-12-01T01:52:48.035539+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035539+0000","last_undegraded":"2023-12-01T01:52:48.035539+0000","last_fullsized":"2023-12-01T01:52:48.035539+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:42.942148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026846+0000","last_change":"2023-12-01T01:52:48.026846+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026846+0000","last_undegraded":"2023-12-01T01:52:48.026846+0000","last_fullsized":"2023-12-01T01:52:48.026846+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.036892+0000","last_change":"2023-12-01T01:52:48.036892+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.036892+0000","last_undegraded":"2023-12-01T01:52:48.036892+0000","last_fullsized":"2023-12-01T01:52:48.036892+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027150+0000","last_change":"2023-12-01T01:52:48.027150+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027150+0000","last_undegraded":"2023-12-01T01:52:48.027150+0000","last_fullsized":"2023-12-01T01:52:48.027150+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.030146+0000","last_change":"2023-12-01T01:52:48.030146+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.030146+0000","last_undegraded":"2023-12-01T01:52:48.030146+0000","last_fullsized":"2023-12-01T01:52:48.030146+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.031458+0000","last_change":"2023-12-01T01:52:48.031458+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.031458+0000","last_undegraded":"2023-12-01T01:52:48.031458+0000","last_fullsized":"2023-12-01T01:52:48.031458+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023595+0000","last_change":"2023-12-01T01:52:48.023595+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023595+0000","last_undegraded":"2023-12-01T01:52:48.023595+0000","last_fullsized":"2023-12-01T01:52:48.023595+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:22.154067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027885+0000","last_change":"2023-12-01T01:52:48.027885+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027885+0000","last_undegraded":"2023-12-01T01:52:48.027885+0000","last_fullsized":"2023-12-01T01:52:48.027885+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:12:02.919659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021893+0000","last_change":"2023-12-01T01:52:48.021893+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021893+0000","last_undegraded":"2023-12-01T01:52:48.021893+0000","last_fullsized":"2023-12-01T01:52:48.021893+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:25:02.674823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020506+0000","last_change":"2023-12-01T01:52:48.020506+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020506+0000","last_undegraded":"2023-12-01T01:52:48.020506+0000","last_fullsized":"2023-12-01T01:52:48.020506+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035671+0000","last_change":"2023-12-01T01:52:48.035671+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035671+0000","last_undegraded":"2023-12-01T01:52:48.035671+0000","last_fullsized":"2023-12-01T01:52:48.035671+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:00.516734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.034799+0000","last_change":"2023-12-01T01:52:48.034799+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.034799+0000","last_undegraded":"2023-12-01T01:52:48.034799+0000","last_fullsized":"2023-12-01T01:52:48.034799+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.028380+0000","last_change":"2023-12-01T01:52:48.028380+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.028380+0000","last_undegraded":"2023-12-01T01:52:48.028380+0000","last_fullsized":"2023-12-01T01:52:48.028380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026751+0000","last_change":"2023-12-01T01:52:48.026751+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026751+0000","last_undegraded":"2023-12-01T01:52:48.026751+0000","last_fullsized":"2023-12-01T01:52:48.026751+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:02:42.893820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041182+0000","last_change":"2023-12-01T01:52:48.038820+0000","last_active":"2023-12-01T01:52:48.041182+0000","last_peered":"2023-12-01T01:52:48.041182+0000","last_clean":"2023-12-01T01:52:48.041182+0000","last_became_active":"2023-12-01T01:52:48.038539+0000","last_became_peered":"2023-12-01T01:52:48.038539+0000","last_unstale":"2023-12-01T01:52:48.041182+0000","last_undegraded":"2023-12-01T01:52:48.041182+0000","last_fullsized":"2023-12-01T01:52:48.041182+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:21.090681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026112+0000","last_change":"2023-12-01T01:52:48.026112+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026112+0000","last_undegraded":"2023-12-01T01:52:48.026112+0000","last_fullsized":"2023-12-01T01:52:48.026112+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:01.801261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.050678+0000","last_change":"2023-12-01T01:52:48.049643+0000","last_active":"2023-12-01T01:52:48.050678+0000","last_peered":"2023-12-01T01:52:48.050678+0000","last_clean":"2023-12-01T01:52:48.050678+0000","last_became_active":"2023-12-01T01:52:48.049288+0000","last_became_peered":"2023-12-01T01:52:48.049288+0000","last_unstale":"2023-12-01T01:52:48.050678+0000","last_undegraded":"2023-12-01T01:52:48.050678+0000","last_fullsized":"2023-12-01T01:52:48.050678+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:41.937860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024925+0000","last_change":"2023-12-01T01:52:48.024925+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024925+0000","last_undegraded":"2023-12-01T01:52:48.024925+0000","last_fullsized":"2023-12-01T01:52:48.024925+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:50:17.533885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044029+0000","last_change":"2023-12-01T01:52:48.039238+0000","last_active":"2023-12-01T01:52:48.044029+0000","last_peered":"2023-12-01T01:52:48.044029+0000","last_clean":"2023-12-01T01:52:48.044029+0000","last_became_active":"2023-12-01T01:52:48.038835+0000","last_became_peered":"2023-12-01T01:52:48.038835+0000","last_unstale":"2023-12-01T01:52:48.044029+0000","last_undegraded":"2023-12-01T01:52:48.044029+0000","last_fullsized":"2023-12-01T01:52:48.044029+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023506+0000","last_change":"2023-12-01T01:52:48.023506+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023506+0000","last_undegraded":"2023-12-01T01:52:48.023506+0000","last_fullsized":"2023-12-01T01:52:48.023506+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.048403+0000","last_change":"2023-12-01T01:52:48.045716+0000","last_active":"2023-12-01T01:52:48.048403+0000","last_peered":"2023-12-01T01:52:48.048403+0000","last_clean":"2023-12-01T01:52:48.048403+0000","last_became_active":"2023-12-01T01:52:48.045176+0000","last_became_peered":"2023-12-01T01:52:48.045176+0000","last_unstale":"2023-12-01T01:52:48.048403+0000","last_undegraded":"2023-12-01T01:52:48.048403+0000","last_fullsized":"2023-12-01T01:52:48.048403+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:55:21.695057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026841+0000","last_change":"2023-12-01T01:52:48.026841+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026841+0000","last_undegraded":"2023-12-01T01:52:48.026841+0000","last_fullsized":"2023-12-01T01:52:48.026841+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.036263+0000","last_change":"2023-12-01T01:52:48.033877+0000","last_active":"2023-12-01T01:52:48.036263+0000","last_peered":"2023-12-01T01:52:48.036263+0000","last_clean":"2023-12-01T01:52:48.036263+0000","last_became_active":"2023-12-01T01:52:48.033605+0000","last_became_peered":"2023-12-01T01:52:48.033605+0000","last_unstale":"2023-12-01T01:52:48.036263+0000","last_undegraded":"2023-12-01T01:52:48.036263+0000","last_fullsized":"2023-12-01T01:52:48.036263+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:55.538925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.032197+0000","last_change":"2023-12-01T01:52:48.032197+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.032197+0000","last_undegraded":"2023-12-01T01:52:48.032197+0000","last_fullsized":"2023-12-01T01:52:48.032197+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:53:05.715094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.046497+0000","last_change":"2023-12-01T01:52:48.043431+0000","last_active":"2023-12-01T01:52:48.046497+0000","last_peered":"2023-12-01T01:52:48.046497+0000","last_clean":"2023-12-01T01:52:48.046497+0000","last_became_active":"2023-12-01T01:52:48.043162+0000","last_became_peered":"2023-12-01T01:52:48.043162+0000","last_unstale":"2023-12-01T01:52:48.046497+0000","last_undegraded":"2023-12-01T01:52:48.046497+0000","last_fullsized":"2023-12-01T01:52:48.046497+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.022665+0000","last_change":"2023-12-01T01:52:48.022665+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.022665+0000","last_undegraded":"2023-12-01T01:52:48.022665+0000","last_fullsized":"2023-12-01T01:52:48.022665+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:10:51.868019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049031+0000","last_change":"2023-12-01T01:52:48.046040+0000","last_active":"2023-12-01T01:52:48.049031+0000","last_peered":"2023-12-01T01:52:48.049031+0000","last_clean":"2023-12-01T01:52:48.049031+0000","last_became_active":"2023-12-01T01:52:48.045722+0000","last_became_peered":"2023-12-01T01:52:48.045722+0000","last_unstale":"2023-12-01T01:52:48.049031+0000","last_undegraded":"2023-12-01T01:52:48.049031+0000","last_fullsized":"2023-12-01T01:52:48.049031+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:25:01.670535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035505+0000","last_change":"2023-12-01T01:52:48.035505+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035505+0000","last_undegraded":"2023-12-01T01:52:48.035505+0000","last_fullsized":"2023-12-01T01:52:48.035505+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:38.690885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.051042+0000","last_change":"2023-12-01T01:52:48.049260+0000","last_active":"2023-12-01T01:52:48.051042+0000","last_peered":"2023-12-01T01:52:48.051042+0000","last_clean":"2023-12-01T01:52:48.051042+0000","last_became_active":"2023-12-01T01:52:48.048998+0000","last_became_peered":"2023-12-01T01:52:48.048998+0000","last_unstale":"2023-12-01T01:52:48.051042+0000","last_undegraded":"2023-12-01T01:52:48.051042+0000","last_fullsized":"2023-12-01T01:52:48.051042+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:06.546333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035689+0000","last_change":"2023-12-01T01:52:48.035689+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035689+0000","last_undegraded":"2023-12-01T01:52:48.035689+0000","last_fullsized":"2023-12-01T01:52:48.035689+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:12.516160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.037274+0000","last_change":"2023-12-01T01:52:48.035560+0000","last_active":"2023-12-01T01:52:48.037274+0000","last_peered":"2023-12-01T01:52:48.037274+0000","last_clean":"2023-12-01T01:52:48.037274+0000","last_became_active":"2023-12-01T01:52:48.035303+0000","last_became_peered":"2023-12-01T01:52:48.035303+0000","last_unstale":"2023-12-01T01:52:48.037274+0000","last_undegraded":"2023-12-01T01:52:48.037274+0000","last_fullsized":"2023-12-01T01:52:48.037274+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:04.573692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025380+0000","last_change":"2023-12-01T01:52:48.025380+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025380+0000","last_undegraded":"2023-12-01T01:52:48.025380+0000","last_fullsized":"2023-12-01T01:52:48.025380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:22.643458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041963+0000","last_change":"2023-12-01T01:52:48.040913+0000","last_active":"2023-12-01T01:52:48.041963+0000","last_peered":"2023-12-01T01:52:48.041963+0000","last_clean":"2023-12-01T01:52:48.041963+0000","last_became_active":"2023-12-01T01:52:48.040402+0000","last_became_peered":"2023-12-01T01:52:48.040402+0000","last_unstale":"2023-12-01T01:52:48.041963+0000","last_undegraded":"2023-12-01T01:52:48.041963+0000","last_fullsized":"2023-12-01T01:52:48.041963+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027539+0000","last_change":"2023-12-01T01:52:48.027539+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027539+0000","last_undegraded":"2023-12-01T01:52:48.027539+0000","last_fullsized":"2023-12-01T01:52:48.027539+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.047008+0000","last_change":"2023-12-01T01:52:48.045583+0000","last_active":"2023-12-01T01:52:48.047008+0000","last_peered":"2023-12-01T01:52:48.047008+0000","last_clean":"2023-12-01T01:52:48.047008+0000","last_became_active":"2023-12-01T01:52:48.045146+0000","last_became_peered":"2023-12-01T01:52:48.045146+0000","last_unstale":"2023-12-01T01:52:48.047008+0000","last_undegraded":"2023-12-01T01:52:48.047008+0000","last_fullsized":"2023-12-01T01:52:48.047008+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:25.868578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.028382+0000","last_change":"2023-12-01T01:52:48.028382+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.028382+0000","last_undegraded":"2023-12-01T01:52:48.028382+0000","last_fullsized":"2023-12-01T01:52:48.028382+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:03.971637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044477+0000","last_change":"2023-12-01T01:52:48.042396+0000","last_active":"2023-12-01T01:52:48.044477+0000","last_peered":"2023-12-01T01:52:48.044477+0000","last_clean":"2023-12-01T01:52:48.044477+0000","last_became_active":"2023-12-01T01:52:48.041958+0000","last_became_peered":"2023-12-01T01:52:48.041958+0000","last_unstale":"2023-12-01T01:52:48.044477+0000","last_undegraded":"2023-12-01T01:52:48.044477+0000","last_fullsized":"2023-12-01T01:52:48.044477+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:33.265613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024486+0000","last_change":"2023-12-01T01:52:48.024486+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024486+0000","last_undegraded":"2023-12-01T01:52:48.024486+0000","last_fullsized":"2023-12-01T01:52:48.024486+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.039978+0000","last_change":"2023-12-01T01:52:48.035637+0000","last_active":"2023-12-01T01:52:48.039978+0000","last_peered":"2023-12-01T01:52:48.039978+0000","last_clean":"2023-12-01T01:52:48.039978+0000","last_became_active":"2023-12-01T01:52:48.034955+0000","last_became_peered":"2023-12-01T01:52:48.034955+0000","last_unstale":"2023-12-01T01:52:48.039978+0000","last_undegraded":"2023-12-01T01:52:48.039978+0000","last_fullsized":"2023-12-01T01:52:48.039978+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:02.433217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025388+0000","last_change":"2023-12-01T01:52:48.025388+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025388+0000","last_undegraded":"2023-12-01T01:52:48.025388+0000","last_fullsized":"2023-12-01T01:52:48.025388+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:58:00.440400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.037568+0000","last_change":"2023-12-01T01:52:48.033331+0000","last_active":"2023-12-01T01:52:48.037568+0000","last_peered":"2023-12-01T01:52:48.037568+0000","last_clean":"2023-12-01T01:52:48.037568+0000","last_became_active":"2023-12-01T01:52:48.032852+0000","last_became_peered":"2023-12-01T01:52:48.032852+0000","last_unstale":"2023-12-01T01:52:48.037568+0000","last_undegraded":"2023-12-01T01:52:48.037568+0000","last_fullsized":"2023-12-01T01:52:48.037568+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:37.686598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.036643+0000","last_change":"2023-12-01T01:52:48.036643+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.036643+0000","last_undegraded":"2023-12-01T01:52:48.036643+0000","last_fullsized":"2023-12-01T01:52:48.036643+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:45.833443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.042163+0000","last_change":"2023-12-01T01:52:48.041648+0000","last_active":"2023-12-01T01:52:48.042163+0000","last_peered":"2023-12-01T01:52:48.042163+0000","last_clean":"2023-12-01T01:52:48.042163+0000","last_became_active":"2023-12-01T01:52:48.040913+0000","last_became_peered":"2023-12-01T01:52:48.040913+0000","last_unstale":"2023-12-01T01:52:48.042163+0000","last_undegraded":"2023-12-01T01:52:48.042163+0000","last_fullsized":"2023-12-01T01:52:48.042163+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:37.686598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023509+0000","last_change":"2023-12-01T01:52:48.023509+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023509+0000","last_undegraded":"2023-12-01T01:52:48.023509+0000","last_fullsized":"2023-12-01T01:52:48.023509+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:56.543213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.034765+0000","last_change":"2023-12-01T01:52:48.032822+0000","last_active":"2023-12-01T01:52:48.034765+0000","last_peered":"2023-12-01T01:52:48.034765+0000","last_clean":"2023-12-01T01:52:48.034765+0000","last_became_active":"2023-12-01T01:52:48.032368+0000","last_became_peered":"2023-12-01T01:52:48.032368+0000","last_unstale":"2023-12-01T01:52:48.034765+0000","last_undegraded":"2023-12-01T01:52:48.034765+0000","last_fullsized":"2023-12-01T01:52:48.034765+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:10:50.863731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026337+0000","last_change":"2023-12-01T01:52:48.026337+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026337+0000","last_undegraded":"2023-12-01T01:52:48.026337+0000","last_fullsized":"2023-12-01T01:52:48.026337+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:09:36.618385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041816+0000","last_change":"2023-12-01T01:52:48.040258+0000","last_active":"2023-12-01T01:52:48.041816+0000","last_peered":"2023-12-01T01:52:48.041816+0000","last_clean":"2023-12-01T01:52:48.041816+0000","last_became_active":"2023-12-01T01:52:48.037376+0000","last_became_peered":"2023-12-01T01:52:48.037376+0000","last_unstale":"2023-12-01T01:52:48.041816+0000","last_undegraded":"2023-12-01T01:52:48.041816+0000","last_fullsized":"2023-12-01T01:52:48.041816+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:13.111537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.033821+0000","last_change":"2023-12-01T01:52:48.033821+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.033821+0000","last_undegraded":"2023-12-01T01:52:48.033821+0000","last_fullsized":"2023-12-01T01:52:48.033821+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.051066+0000","last_change":"2023-12-01T01:52:48.050044+0000","last_active":"2023-12-01T01:52:48.051066+0000","last_peered":"2023-12-01T01:52:48.051066+0000","last_clean":"2023-12-01T01:52:48.051066+0000","last_became_active":"2023-12-01T01:52:48.049779+0000","last_became_peered":"2023-12-01T01:52:48.049779+0000","last_unstale":"2023-12-01T01:52:48.051066+0000","last_undegraded":"2023-12-01T01:52:48.051066+0000","last_fullsized":"2023-12-01T01:52:48.051066+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:20:19.462886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026154+0000","last_change":"2023-12-01T01:52:48.026154+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026154+0000","last_undegraded":"2023-12-01T01:52:48.026154+0000","last_fullsized":"2023-12-01T01:52:48.026154+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:17.235352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.050226+0000","last_change":"2023-12-01T01:52:48.048859+0000","last_active":"2023-12-01T01:52:48.050226+0000","last_peered":"2023-12-01T01:52:48.050226+0000","last_clean":"2023-12-01T01:52:48.050226+0000","last_became_active":"2023-12-01T01:52:48.048546+0000","last_became_peered":"2023-12-01T01:52:48.048546+0000","last_unstale":"2023-12-01T01:52:48.050226+0000","last_undegraded":"2023-12-01T01:52:48.050226+0000","last_fullsized":"2023-12-01T01:52:48.050226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:43.307966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021314+0000","last_change":"2023-12-01T01:52:48.021314+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021314+0000","last_undegraded":"2023-12-01T01:52:48.021314+0000","last_fullsized":"2023-12-01T01:52:48.021314+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049328+0000","last_change":"2023-12-01T01:52:48.046685+0000","last_active":"2023-12-01T01:52:48.049328+0000","last_peered":"2023-12-01T01:52:48.049328+0000","last_clean":"2023-12-01T01:52:48.049328+0000","last_became_active":"2023-12-01T01:52:48.046395+0000","last_became_peered":"2023-12-01T01:52:48.046395+0000","last_unstale":"2023-12-01T01:52:48.049328+0000","last_undegraded":"2023-12-01T01:52:48.049328+0000","last_fullsized":"2023-12-01T01:52:48.049328+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:29:10.130713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021393+0000","last_change":"2023-12-01T01:52:48.021393+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021393+0000","last_undegraded":"2023-12-01T01:52:48.021393+0000","last_fullsized":"2023-12-01T01:52:48.021393+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:52:45.545702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041831+0000","last_change":"2023-12-01T01:52:48.038141+0000","last_active":"2023-12-01T01:52:48.041831+0000","last_peered":"2023-12-01T01:52:48.041831+0000","last_clean":"2023-12-01T01:52:48.041831+0000","last_became_active":"2023-12-01T01:52:48.037797+0000","last_became_peered":"2023-12-01T01:52:48.037797+0000","last_unstale":"2023-12-01T01:52:48.041831+0000","last_undegraded":"2023-12-01T01:52:48.041831+0000","last_fullsized":"2023-12-01T01:52:48.041831+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021667+0000","last_change":"2023-12-01T01:52:48.021667+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021667+0000","last_undegraded":"2023-12-01T01:52:48.021667+0000","last_fullsized":"2023-12-01T01:52:48.021667+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.043878+0000","last_change":"2023-12-01T01:52:48.042643+0000","last_active":"2023-12-01T01:52:48.043878+0000","last_peered":"2023-12-01T01:52:48.043878+0000","last_clean":"2023-12-01T01:52:48.043878+0000","last_became_active":"2023-12-01T01:52:48.042174+0000","last_became_peered":"2023-12-01T01:52:48.042174+0000","last_unstale":"2023-12-01T01:52:48.043878+0000","last_undegraded":"2023-12-01T01:52:48.043878+0000","last_fullsized":"2023-12-01T01:52:48.043878+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:59.436113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026638+0000","last_change":"2023-12-01T01:52:48.026638+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026638+0000","last_undegraded":"2023-12-01T01:52:48.026638+0000","last_fullsized":"2023-12-01T01:52:48.026638+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049368+0000","last_change":"2023-12-01T01:52:48.047235+0000","last_active":"2023-12-01T01:52:48.049368+0000","last_peered":"2023-12-01T01:52:48.049368+0000","last_clean":"2023-12-01T01:52:48.049368+0000","last_became_active":"2023-12-01T01:52:48.046953+0000","last_became_peered":"2023-12-01T01:52:48.046953+0000","last_unstale":"2023-12-01T01:52:48.049368+0000","last_undegraded":"2023-12-01T01:52:48.049368+0000","last_fullsized":"2023-12-01T01:52:48.049368+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:13.492174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023160+0000","last_change":"2023-12-01T01:52:48.023160+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023160+0000","last_undegraded":"2023-12-01T01:52:48.023160+0000","last_fullsized":"2023-12-01T01:52:48.023160+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:39.608737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.051849+0000","last_change":"2023-12-01T01:52:48.050705+0000","last_active":"2023-12-01T01:52:48.051849+0000","last_peered":"2023-12-01T01:52:48.051849+0000","last_clean":"2023-12-01T01:52:48.051849+0000","last_became_active":"2023-12-01T01:52:48.050445+0000","last_became_peered":"2023-12-01T01:52:48.050445+0000","last_unstale":"2023-12-01T01:52:48.051849+0000","last_undegraded":"2023-12-01T01:52:48.051849+0000","last_fullsized":"2023-12-01T01:52:48.051849+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:02:56.065815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025759+0000","last_change":"2023-12-01T01:52:48.025759+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025759+0000","last_undegraded":"2023-12-01T01:52:48.025759+0000","last_fullsized":"2023-12-01T01:52:48.025759+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:55:14.304687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.042923+0000","last_change":"2023-12-01T01:52:48.040799+0000","last_active":"2023-12-01T01:52:48.042923+0000","last_peered":"2023-12-01T01:52:48.042923+0000","last_clean":"2023-12-01T01:52:48.042923+0000","last_became_active":"2023-12-01T01:52:48.040471+0000","last_became_peered":"2023-12-01T01:52:48.040471+0000","last_unstale":"2023-12-01T01:52:48.042923+0000","last_undegraded":"2023-12-01T01:52:48.042923+0000","last_fullsized":"2023-12-01T01:52:48.042923+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021428+0000","last_change":"2023-12-01T01:52:48.021428+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021428+0000","last_undegraded":"2023-12-01T01:52:48.021428+0000","last_fullsized":"2023-12-01T01:52:48.021428+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:35:01.791105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044954+0000","last_change":"2023-12-01T01:52:48.043497+0000","last_active":"2023-12-01T01:52:48.044954+0000","last_peered":"2023-12-01T01:52:48.044954+0000","last_clean":"2023-12-01T01:52:48.044954+0000","last_became_active":"2023-12-01T01:52:48.043198+0000","last_became_peered":"2023-12-01T01:52:48.043198+0000","last_unstale":"2023-12-01T01:52:48.044954+0000","last_undegraded":"2023-12-01T01:52:48.044954+0000","last_fullsized":"2023-12-01T01:52:48.044954+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:44.930335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.033844+0000","last_change":"2023-12-01T01:52:48.033844+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.033844+0000","last_undegraded":"2023-12-01T01:52:48.033844+0000","last_fullsized":"2023-12-01T01:52:48.033844+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:03.437505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049785+0000","last_change":"2023-12-01T01:52:48.046182+0000","last_active":"2023-12-01T01:52:48.049785+0000","last_peered":"2023-12-01T01:52:48.049785+0000","last_clean":"2023-12-01T01:52:48.049785+0000","last_became_active":"2023-12-01T01:52:48.045696+0000","last_became_peered":"2023-12-01T01:52:48.045696+0000","last_unstale":"2023-12-01T01:52:48.049785+0000","last_undegraded":"2023-12-01T01:52:48.049785+0000","last_fullsized":"2023-12-01T01:52:48.049785+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:44.829155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020256+0000","last_change":"2023-12-01T01:52:48.020256+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020256+0000","last_undegraded":"2023-12-01T01:52:48.020256+0000","last_fullsized":"2023-12-01T01:52:48.020256+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:34:49.188317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044461+0000","last_change":"2023-12-01T01:52:48.042910+0000","last_active":"2023-12-01T01:52:48.044461+0000","last_peered":"2023-12-01T01:52:48.044461+0000","last_clean":"2023-12-01T01:52:48.044461+0000","last_became_active":"2023-12-01T01:52:48.041897+0000","last_became_peered":"2023-12-01T01:52:48.041897+0000","last_unstale":"2023-12-01T01:52:48.044461+0000","last_undegraded":"2023-12-01T01:52:48.044461+0000","last_fullsized":"2023-12-01T01:52:48.044461+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027872+0000","last_change":"2023-12-01T01:52:48.027872+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027872+0000","last_undegraded":"2023-12-01T01:52:48.027872+0000","last_fullsized":"2023-12-01T01:52:48.027872+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:45.934623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.047598+0000","last_change":"2023-12-01T01:52:48.044493+0000","last_active":"2023-12-01T01:52:48.047598+0000","last_peered":"2023-12-01T01:52:48.047598+0000","last_clean":"2023-12-01T01:52:48.047598+0000","last_became_active":"2023-12-01T01:52:48.044221+0000","last_became_peered":"2023-12-01T01:52:48.044221+0000","last_unstale":"2023-12-01T01:52:48.047598+0000","last_undegraded":"2023-12-01T01:52:48.047598+0000","last_fullsized":"2023-12-01T01:52:48.047598+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:59.436113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.019339+0000","last_change":"2023-12-01T01:52:48.019339+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.019339+0000","last_undegraded":"2023-12-01T01:52:48.019339+0000","last_fullsized":"2023-12-01T01:52:48.019339+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:20:23.997177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.046763+0000","last_change":"2023-12-01T01:52:48.045085+0000","last_active":"2023-12-01T01:52:48.046763+0000","last_peered":"2023-12-01T01:52:48.046763+0000","last_clean":"2023-12-01T01:52:48.046763+0000","last_became_active":"2023-12-01T01:52:48.044789+0000","last_became_peered":"2023-12-01T01:52:48.044789+0000","last_unstale":"2023-12-01T01:52:48.046763+0000","last_undegraded":"2023-12-01T01:52:48.046763+0000","last_fullsized":"2023-12-01T01:52:48.046763+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:58:00.466321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020747+0000","last_change":"2023-12-01T01:52:48.020747+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020747+0000","last_undegraded":"2023-12-01T01:52:48.020747+0000","last_fullsized":"2023-12-01T01:52:48.020747+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:02:57.070103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.039567+0000","last_change":"2023-12-01T01:52:48.034635+0000","last_active":"2023-12-01T01:52:48.039567+0000","last_peered":"2023-12-01T01:52:48.039567+0000","last_clean":"2023-12-01T01:52:48.039567+0000","last_became_active":"2023-12-01T01:52:48.034401+0000","last_became_peered":"2023-12-01T01:52:48.034401+0000","last_unstale":"2023-12-01T01:52:48.039567+0000","last_undegraded":"2023-12-01T01:52:48.039567+0000","last_fullsized":"2023-12-01T01:52:48.039567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:19.195130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026074+0000","last_change":"2023-12-01T01:52:48.026074+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026074+0000","last_undegraded":"2023-12-01T01:52:48.026074+0000","last_fullsized":"2023-12-01T01:52:48.026074+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.045370+0000","last_change":"2023-12-01T01:52:48.043879+0000","last_active":"2023-12-01T01:52:48.045370+0000","last_peered":"2023-12-01T01:52:48.045370+0000","last_clean":"2023-12-01T01:52:48.045370+0000","last_became_active":"2023-12-01T01:52:48.043357+0000","last_became_peered":"2023-12-01T01:52:48.043357+0000","last_unstale":"2023-12-01T01:52:48.045370+0000","last_undegraded":"2023-12-01T01:52:48.045370+0000","last_fullsized":"2023-12-01T01:52:48.045370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:59.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'183","reported_seq":481,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-12-01T01:52:47.018333+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:52:47.018333+0000","last_peered":"2023-12-01T01:52:47.018333+0000","last_clean":"2023-12-01T01:52:47.018333+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:52:47.018333+0000","last_undegraded":"2023-12-01T01:52:47.018333+0000","last_fullsized":"2023-12-01T01:52:47.018333+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575051,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.02,"15min":1.0660000000000001},"min":{"1min":0.54900000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.401,"5min":1.514,"15min":1.8089999999999999},"last":1.145},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0309999999999999,"15min":1.0469999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5289999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.0469999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1499999999999999,"15min":1.2070000000000001},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.5669999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2150000000000001,"15min":1.248},"min":{"1min":0.66800000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6279999999999999,"5min":1.744,"15min":1.744},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1040000000000001,"15min":1.117},"min":{"1min":0.749,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4910000000000001,"5min":1.607,"15min":1.728},"last":1.244},{"interface":"front","average":{"1min":1.2,"5min":1.2010000000000001,"15min":1.2190000000000001},"min":{"1min":0.92500000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.641,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.274}]},{"osd":3,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.1910000000000001,"15min":1.2230000000000001},"min":{"1min":0.64200000000000002,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.347,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3180000000000001,"15min":1.3420000000000001},"min":{"1min":0.63700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7490000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.3360000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.3009999999999999,"15min":1.2849999999999999},"min":{"1min":0.72699999999999998,"5min":0.56100000000000005,"15min":0.55500000000000005},"max":{"1min":1.4450000000000001,"5min":1.7549999999999999,"15min":2.2130000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4350000000000001,"15min":1.4059999999999999},"min":{"1min":0.77500000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8100000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.008}]},{"osd":5,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.292,"15min":1.258},"min":{"1min":0.625,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.0379999999999998,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.47,"5min":1.514,"15min":1.514},"min":{"1min":0.69399999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.861,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.3859999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.395,"15min":1.377},"min":{"1min":0.753,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1549999999999998,"5min":2.294,"15min":2.3420000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.387},"min":{"1min":0.82599999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.893,"5min":1.893,"15min":2.3500000000000001},"last":1.0920000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.1020000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.79},"last":1.034},{"interface":"front","average":{"1min":1.036,"5min":1.105,"15min":1.1220000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5349999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.999}]},{"osd":1,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.161,"15min":1.159},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1950000000000001,"15min":1.2050000000000001},"min":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0920000000000001},"last":1.204}]},{"osd":2,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1859999999999999,"15min":1.171},"min":{"1min":0.69699999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5760000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.754},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.242,"15min":1.2350000000000001},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":3.024,"5min":3.024,"15min":3.024},"last":1.1499999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":1.0289999999999999,"5min":0.93700000000000006,"15min":0.82399999999999995},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.48}]},{"osd":4,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3,"15min":1.2709999999999999},"min":{"1min":0.94299999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6819999999999999,"5min":2.226,"15min":2.226},"last":1.238},{"interface":"front","average":{"1min":1.339,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":1.028,"5min":0.91400000000000003,"15min":0.78100000000000003},"max":{"1min":1.885,"5min":1.931,"15min":2.2839999999999998},"last":1.5800000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.415,"15min":1.397},"min":{"1min":0.755,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.74,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3},{"interface":"front","average":{"1min":1.47,"5min":1.502,"15min":1.472},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.84099999999999997},"max":{"1min":1.9790000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.6799999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.524,"15min":1.5009999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6870000000000001,"5min":2.762,"15min":2.762},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.577,"15min":1.5629999999999999},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":1.9330000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.742}]}]},{"osd":5,"up_from":13,"seq":55834575051,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32924,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338916,"statfs":{"total":96636764160,"available":96603049984,"internally_reserved":0,"allocated":1921024,"data_stored":1219865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1259999999999999,"15min":1.1040000000000001},"min":{"1min":0.63200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.2010000000000001},"last":1.099},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1180000000000001,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.60099999999999998},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":2.153},"last":1.3700000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.117,"15min":1.1399999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.63200000000000001},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.1399999999999999,"15min":1.159},"min":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.6519999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.2430000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.173,"15min":1.196},"min":{"1min":0.65000000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.702,"5min":1.702,"15min":1.7110000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.208,"15min":1.2330000000000001},"min":{"1min":0.79100000000000004,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":2.0630000000000002},"last":1.1990000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.248,"15min":1.282},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.8169999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.302},"min":{"1min":1.159,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.95099999999999996}]},{"osd":4,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2410000000000001},"min":{"1min":0.83499999999999996,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.018},{"interface":"front","average":{"1min":1.409,"5min":1.319,"15min":1.3480000000000001},"min":{"1min":0.84399999999999997,"5min":0.80300000000000005,"15min":0.79900000000000004},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.452}]},{"osd":6,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3129999999999999,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.859,"5min":1.885,"15min":1.984},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.3939999999999999,"15min":1.3839999999999999},"min":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.149}]},{"osd":7,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.462,"15min":1.4750000000000001},"min":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0920000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.522,"15min":1.5269999999999999},"min":{"1min":1.304,"5min":0.873,"15min":0.873},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.411}]}]},{"osd":4,"up_from":13,"seq":55834575051,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1120000000000001,"15min":1.105},"min":{"1min":0.72199999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.753,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.7130000000000001,"5min":1.7529999999999999,"15min":2.1099999999999999},"last":1.657}]},{"osd":1,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.216,"15min":1.204},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.65900000000000003},"max":{"1min":1.5409999999999999,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.534},{"interface":"front","average":{"1min":1.087,"5min":1.101,"15min":1.129},"min":{"1min":0.76600000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3979999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.458}]},{"osd":2,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.234,"15min":1.2},"min":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.716,"5min":1.871,"15min":1.871},"last":1.204},{"interface":"front","average":{"1min":1.327,"5min":1.3180000000000001,"15min":1.288},"min":{"1min":0.94599999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.581,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.76}]},{"osd":3,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.258,"15min":1.2310000000000001},"min":{"1min":0.81899999999999995,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.6140000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.139},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3200000000000001,"15min":1.3160000000000001},"min":{"1min":0.88500000000000001,"5min":0.753,"15min":0.753},"max":{"1min":1.712,"5min":1.7270000000000001,"15min":1.903},"last":1.3009999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.319,"15min":1.3129999999999999},"min":{"1min":0.94899999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.288,"15min":1.2869999999999999},"min":{"1min":0.97199999999999998,"5min":0.90300000000000002,"15min":0.85399999999999998},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.24}]},{"osd":6,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.381,"15min":1.3500000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.81299999999999994},"max":{"1min":1.8799999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.502,"15min":1.472},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.0960000000000001,"5min":2.4990000000000001,"15min":2.54},"last":1.71}]},{"osd":7,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.472,"15min":1.4610000000000001},"min":{"1min":1.274,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5780000000000001,"15min":1.5620000000000001},"min":{"1min":1.28,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":1.96,"5min":2.04,"15min":2.157},"last":1.486}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58584,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313256,"statfs":{"total":96636764160,"available":96576774144,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.84199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.307},"max":{"1min":1.1899999999999999,"5min":1.3320000000000001,"15min":1.583},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.93500000000000005,"15min":0.97999999999999998},"min":{"1min":0.48099999999999998,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.99199999999999999,"15min":1.056},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.38900000000000001},"max":{"1min":1.5669999999999999,"5min":1.571,"15min":2.0779999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.89300000000000002,"15min":0.94799999999999995},"min":{"1min":0.48399999999999999,"5min":0.48199999999999998,"15min":0.42699999999999999},"max":{"1min":1.1339999999999999,"5min":1.55,"15min":1.6519999999999999},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.96699999999999997,"15min":1.0349999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3360000000000001,"5min":1.4810000000000001,"15min":1.5609999999999999},"last":1.167},{"interface":"front","average":{"1min":0.995,"5min":1.125,"15min":1.1819999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1970000000000001,"15min":1.232},"min":{"1min":0.84999999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.524,"5min":1.5820000000000001,"15min":1.9319999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.198,"15min":1.232},"min":{"1min":0.79300000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.909,"5min":1.909,"15min":2.0419999999999998},"last":0.96099999999999997}]},{"osd":5,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.228,"15min":1.2789999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.71,"5min":1.7669999999999999,"15min":2.0510000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2529999999999999,"15min":1.288},"min":{"1min":0.745,"5min":0.745,"15min":0.64200000000000002},"max":{"1min":1.6759999999999999,"5min":1.681,"15min":2.1880000000000002},"last":1.274}]},{"osd":6,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.89600000000000002,"5min":0.76300000000000001,"15min":0.68300000000000005},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.995},{"interface":"front","average":{"1min":1.264,"5min":1.3700000000000001,"15min":1.405},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8260000000000001,"5min":1.9119999999999999,"15min":1.99},"last":1.4259999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.365,"15min":1.413},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.81499999999999995},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.1619999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4019999999999999,"15min":1.4470000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.83099999999999996},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.2130000000000001},"last":1.0940000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":0.81699999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.5529999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.98399999999999999,"15min":1},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.97099999999999997}]},{"osd":1,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95799999999999996,"15min":0.98699999999999999},"min":{"1min":0.54900000000000004,"5min":0.373,"15min":0.373},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":1.042},{"interface":"front","average":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.081},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.278}]},{"osd":3,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93899999999999995,"15min":1.002},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5189999999999999,"5min":1.625,"15min":1.6479999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.139}]},{"osd":4,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.869,"5min":1.9950000000000001,"15min":2.024},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.111,"5min":1.202,"15min":1.258},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.964},"last":1.2210000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.3129999999999999},"min":{"1min":0.72199999999999998,"5min":0.62,"15min":0.62},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.357,"15min":1.353},"min":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.016,"5min":2.048,"15min":2.323},"last":0.88900000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2849999999999999,"15min":1.3180000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.579,"5min":1.78,"15min":2.2799999999999998},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.90300000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.383}]},{"osd":7,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.448},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3440000000000001,"15min":1.381},"min":{"1min":0.80600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":2.1080000000000001},"last":1.4019999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575049,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59164,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312676,"statfs":{"total":96636764160,"available":96576180224,"internally_reserved":0,"allocated":1921024,"data_stored":1219865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.83499999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.318},"max":{"1min":1.1639999999999999,"5min":1.341,"15min":1.5049999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.876,"5min":0.94999999999999996,"15min":1.0069999999999999},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.597,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85299999999999998,"15min":0.88100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.54},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0349999999999999,"15min":1.081},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.44800000000000001},"max":{"1min":1.472,"5min":1.7250000000000001,"15min":1.804},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.5},"max":{"1min":1.718,"5min":1.827,"15min":1.827},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1,"15min":1.038},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.39700000000000002},"max":{"1min":1.381,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.626}]},{"osd":4,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1579999999999999,"15min":1.173},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.65300000000000002},"max":{"1min":1.425,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.288,"15min":1.3240000000000001},"min":{"1min":0.85599999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.863},"last":1.131}]},{"osd":5,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":0.77700000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.244,"15min":1.27},"min":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":2.1619999999999999},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.294,"15min":1.3220000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.68400000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.861},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.306,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.742,"5min":1.7490000000000001,"15min":1.8839999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3720000000000001,"15min":1.417},"min":{"1min":0.91200000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.782,"5min":1.782,"15min":2.0609999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3859999999999999,"15min":1.399},"min":{"1min":0.80900000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.931,"5min":1.992,"15min":1.992},"last":0.81799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575051,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84099999999999997,"15min":0.85399999999999998},"min":{"1min":0.435,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.194,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89900000000000002,"15min":0.96499999999999997},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3859999999999999,"5min":1.544,"15min":1.6910000000000001},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.94799999999999995,"15min":0.998},"min":{"1min":0.51500000000000001,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.431,"5min":2.157,"15min":2.157},"last":0.874},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0089999999999999,"15min":1.0649999999999999},"min":{"1min":0.57399999999999995,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.3400000000000001,"5min":1.728,"15min":1.728},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":1.0149999999999999},"min":{"1min":0.623,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.954},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1080000000000001,"15min":1.1539999999999999},"min":{"1min":0.54200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4199999999999999,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.246}]},{"osd":4,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2589999999999999,"15min":1.3029999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5840000000000001,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1899999999999999,"15min":1.258},"min":{"1min":0.90200000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.55,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.323,"15min":1.343},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.71899999999999997},"max":{"1min":1.841,"5min":2.351,"15min":2.351},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2769999999999999,"15min":1.333},"min":{"1min":0.81599999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.546,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.7509999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2789999999999999,"15min":1.319},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7529999999999999,"5min":1.8680000000000001,"15min":2.157},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.351,"15min":1.3839999999999999},"min":{"1min":0.94499999999999995,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.7809999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.5940000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.339,"15min":1.3979999999999999},"min":{"1min":0.94999999999999996,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.5700000000000001,"5min":1.798,"15min":2.4279999999999999},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.3320000000000001,"15min":1.3899999999999999},"min":{"1min":0.98999999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.667,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":0.995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:52:52.950 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:52:53.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.202+0000 7fe914755640 1 Processor -- start 2023-12-01T01:52:53.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.202+0000 7fe914755640 1 -- start start 2023-12-01T01:52:53.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.203+0000 7fe914755640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe90c15d220 0x7fe90c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.203+0000 7fe914755640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0ad2b0 0x7fe90c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.204+0000 7fe914755640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c15d600 con 0x7fe90c0ade80 2023-12-01T01:52:53.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.204+0000 7fe914755640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90c06cc10 con 0x7fe90c15d220 2023-12-01T01:52:53.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.204+0000 7fe914755640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c06c720 con 0x7fe90c0ad2b0 2023-12-01T01:52:53.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.204+0000 7fe912cf2640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe90c0ade80 0x7fe90c06b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:33146/0 (socket says 172.21.15.115:33146) 2023-12-01T01:52:53.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.204+0000 7fe912cf2640 1 -- 172.21.15.115:0/2424138238 learned_addr learned my addr 172.21.15.115:0/2424138238 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:53.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.204+0000 7fe911cf0640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0ad2b0 0x7fe90c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.204+0000 7fe9124f1640 1 --2- 172.21.15.115:0/2424138238 >> v2:172.21.15.142:3300/0 conn(0x7fe90c15d220 0x7fe90c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.204+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492100609 0 0) 0x7fe904001490 con 0x7fe90c0ade80 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.204+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f8003520 con 0x7fe90c0ade80 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4173399251 0 0) 0x7fe8f8003520 con 0x7fe90c0ade80 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0ad2b0 msgr2=0x7fe90c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0ad2b0 0x7fe90c0ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 >> v2:172.21.15.142:3300/0 conn(0x7fe90c15d220 msgr2=0x7fe90c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 --2- 172.21.15.115:0/2424138238 >> v2:172.21.15.142:3300/0 conn(0x7fe90c15d220 0x7fe90c0cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c06c4b0 con 0x7fe90c0ade80 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fe904001b80 con 0x7fe90c0ade80 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 >> v1:172.21.15.115:6789/0 conn(0x7fe90c0ade80 legacy=0x7fe90c06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f80039e0 0x7fe8f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0ad2b0 0x7fe8f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8f8004d00 0x7fe8f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c06cc10 con 0x7fe8f80039e0 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c06c720 con 0x7fe8f8004d00 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f800ba60 con 0x7fe90c0ad2b0 2023-12-01T01:52:53.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe9124f1640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f80039e0 0x7fe8f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.205+0000 7fe911cf0640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0ad2b0 0x7fe8f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.206+0000 7fe9124f1640 1 -- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0ad2b0 msgr2=0x7fe8f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:53.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.206+0000 7fe9124f1640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0ad2b0 0x7fe8f80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.206+0000 7fe9124f1640 1 -- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8f8004d00 msgr2=0x7fe8f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:53.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.206+0000 7fe9124f1640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8f8004d00 0x7fe8f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.206+0000 7fe9124f1640 1 -- 172.21.15.115:0/2424138238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8003520 con 0x7fe8f80039e0 2023-12-01T01:52:53.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.206+0000 7fe911cf0640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0ad2b0 0x7fe8f80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:53.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.207+0000 7fe9124f1640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f80039e0 0x7fe8f8003e00 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe908009a90 tx=0x7fe8f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0501c6f35bb7f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:53.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.207+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90802c6d0 con 0x7fe8f80039e0 2023-12-01T01:52:53.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.207+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90802c830 con 0x7fe8f80039e0 2023-12-01T01:52:53.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.207+0000 7fe9114ef640 1 -- 172.21.15.115:0/2424138238 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90802cbc0 con 0x7fe8f80039e0 2023-12-01T01:52:53.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.207+0000 7fe914755640 1 -- 172.21.15.115:0/2424138238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f80039e0 msgr2=0x7fe8f8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.207+0000 7fe914755640 1 --2- 172.21.15.115:0/2424138238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f80039e0 0x7fe8f8003e00 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe908009a90 tx=0x7fe8f80057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.207+0000 7fe912cf2640 1 -- 172.21.15.115:0/2424138238 reap_dead start 2023-12-01T01:52:53.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.208+0000 7fe914755640 1 -- 172.21.15.115:0/2424138238 shutdown_connections 2023-12-01T01:52:53.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.208+0000 7fe914755640 1 -- 172.21.15.115:0/2424138238 >> 172.21.15.115:0/2424138238 conn(0x7fe90c0ce880 msgr2=0x7fe90c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:53.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.208+0000 7fe914755640 1 -- 172.21.15.115:0/2424138238 shutdown_connections 2023-12-01T01:52:53.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.208+0000 7fe914755640 1 -- 172.21.15.115:0/2424138238 wait complete. 2023-12-01T01:52:53.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.209+0000 7fe914755640 1 Processor -- start 2023-12-01T01:52:53.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.210+0000 7fe914755640 1 -- start start 2023-12-01T01:52:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.210+0000 7fe914755640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8f80039e0 0x7fe90c0c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe914755640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe90c0c7e00 0x7fe90c152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe914755640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0af370 0x7fe90c0af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe914755640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c153230 con 0x7fe90c0c7e00 2023-12-01T01:52:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe914755640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0afd30 con 0x7fe8f80039e0 2023-12-01T01:52:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe914755640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0c8280 con 0x7fe90c0af370 2023-12-01T01:52:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe911cf0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe90c0c7e00 0x7fe90c152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe911cf0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe90c0c7e00 0x7fe90c152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43854/0 (socket says 172.21.15.115:43854) 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe911cf0640 1 -- 172.21.15.115:0/171912266 learned_addr learned my addr 172.21.15.115:0/171912266 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe9124f1640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8f80039e0 0x7fe90c0c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe911cf0640 1 -- 172.21.15.115:0/171912266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0af370 msgr2=0x7fe90c0af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe911cf0640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0af370 0x7fe90c0af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe911cf0640 1 -- 172.21.15.115:0/171912266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8f80039e0 msgr2=0x7fe90c0c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe911cf0640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8f80039e0 0x7fe90c0c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.211+0000 7fe911cf0640 1 -- 172.21.15.115:0/171912266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0c94e0 con 0x7fe90c0c7e00 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.212+0000 7fe911cf0640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe90c0c7e00 0x7fe90c152170 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0108b0 tx=0x7fe8fc010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.212+0000 7fe902ffd640 1 -- 172.21.15.115:0/171912266 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00eb60 con 0x7fe90c0c7e00 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.212+0000 7fe902ffd640 1 -- 172.21.15.115:0/171912266 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc00ecc0 con 0x7fe90c0c7e00 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.212+0000 7fe902ffd640 1 -- 172.21.15.115:0/171912266 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc01c400 con 0x7fe90c0c7e00 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.212+0000 7fe914755640 1 -- 172.21.15.115:0/171912266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90c0c84c0 con 0x7fe90c0c7e00 2023-12-01T01:52:53.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.212+0000 7fe914755640 1 -- 172.21.15.115:0/171912266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c0c8e90 con 0x7fe90c0c7e00 2023-12-01T01:52:53.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.213+0000 7fe914755640 1 -- 172.21.15.115:0/171912266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90c0c9670 con 0x7fe90c0c7e00 2023-12-01T01:52:53.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.214+0000 7fe902ffd640 1 -- 172.21.15.115:0/171912266 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe8fc01c560 con 0x7fe90c0c7e00 2023-12-01T01:52:53.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.215+0000 7fe902ffd640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe8f006a550 0x7fe8f006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.215+0000 7fe902ffd640 1 -- 172.21.15.115:0/171912266 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8fc089120 con 0x7fe90c0c7e00 2023-12-01T01:52:53.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.215+0000 7fe9124f1640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe8f006a550 0x7fe8f006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.216+0000 7fe9124f1640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe8f006a550 0x7fe8f006ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe908036000 tx=0x7fe908030000 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:53.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.220+0000 7fe902ffd640 1 -- 172.21.15.115:0/171912266 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe8fc0546a0 con 0x7fe90c0c7e00 2023-12-01T01:52:53.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.405+0000 7fe914755640 1 -- 172.21.15.115:0/171912266 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe90c06ce10 con 0x7fe8f006a550 2023-12-01T01:52:53.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.424+0000 7fe902ffd640 1 -- 172.21.15.115:0/171912266 <== mgr.7230 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284309 (secure 0 0 0) 0x7fe9080040c0 con 0x7fe8f006a550 2023-12-01T01:52:53.430 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:53.431 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:52:53.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.429+0000 7fe914755640 1 -- 172.21.15.115:0/171912266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe8f006a550 msgr2=0x7fe8f006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.429+0000 7fe914755640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe8f006a550 0x7fe8f006ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe908036000 tx=0x7fe908030000 comp rx=0 tx=0).stop 2023-12-01T01:52:53.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.430+0000 7fe914755640 1 -- 172.21.15.115:0/171912266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe90c0c7e00 msgr2=0x7fe90c152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.430+0000 7fe914755640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe90c0c7e00 0x7fe90c152170 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0108b0 tx=0x7fe8fc010d70 comp rx=0 tx=0).stop 2023-12-01T01:52:53.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.430+0000 7fe914755640 1 -- 172.21.15.115:0/171912266 shutdown_connections 2023-12-01T01:52:53.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.430+0000 7fe914755640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe8f006a550 0x7fe8f006ca10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.430+0000 7fe914755640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe90c0af370 0x7fe90c0af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.430+0000 7fe914755640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe90c0c7e00 0x7fe90c152170 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.430+0000 7fe914755640 1 --2- 172.21.15.115:0/171912266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8f80039e0 0x7fe90c0c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.430+0000 7fe914755640 1 -- 172.21.15.115:0/171912266 >> 172.21.15.115:0/171912266 conn(0x7fe90c0ce880 msgr2=0x7fe90c1e3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:53.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.430+0000 7fe914755640 1 -- 172.21.15.115:0/171912266 shutdown_connections 2023-12-01T01:52:53.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.430+0000 7fe914755640 1 -- 172.21.15.115:0/171912266 wait complete. 2023-12-01T01:52:53.449 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T01:52:52.106719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316488,"kb_used_data":11528,"kb_used_omap":0,"kb_used_meta":304896,"kb_avail":754658232,"statfs":{"total":773094113280,"available":772770029568,"internally_reserved":0,"allocated":11804672,"data_stored":6216712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312213504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.102080"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.031286+0000","last_change":"2023-12-01T01:52:48.031286+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.031286+0000","last_undegraded":"2023-12-01T01:52:48.031286+0000","last_fullsized":"2023-12-01T01:52:48.031286+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:07:01.922435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.022478+0000","last_change":"2023-12-01T01:52:48.022478+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.022478+0000","last_undegraded":"2023-12-01T01:52:48.022478+0000","last_fullsized":"2023-12-01T01:52:48.022478+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:20.199417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.034568+0000","last_change":"2023-12-01T01:52:48.034568+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.034568+0000","last_undegraded":"2023-12-01T01:52:48.034568+0000","last_fullsized":"2023-12-01T01:52:48.034568+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029059+0000","last_change":"2023-12-01T01:52:48.029059+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029059+0000","last_undegraded":"2023-12-01T01:52:48.029059+0000","last_fullsized":"2023-12-01T01:52:48.029059+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:06:59.600929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.022456+0000","last_change":"2023-12-01T01:52:48.022456+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.022456+0000","last_undegraded":"2023-12-01T01:52:48.022456+0000","last_fullsized":"2023-12-01T01:52:48.022456+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:48:34.887669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.033630+0000","last_change":"2023-12-01T01:52:48.033630+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.033630+0000","last_undegraded":"2023-12-01T01:52:48.033630+0000","last_fullsized":"2023-12-01T01:52:48.033630+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:39:18.420163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020737+0000","last_change":"2023-12-01T01:52:48.020737+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020737+0000","last_undegraded":"2023-12-01T01:52:48.020737+0000","last_fullsized":"2023-12-01T01:52:48.020737+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:51.221046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024562+0000","last_change":"2023-12-01T01:52:48.024562+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024562+0000","last_undegraded":"2023-12-01T01:52:48.024562+0000","last_fullsized":"2023-12-01T01:52:48.024562+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:21.099958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026842+0000","last_change":"2023-12-01T01:52:48.026842+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026842+0000","last_undegraded":"2023-12-01T01:52:48.026842+0000","last_fullsized":"2023-12-01T01:52:48.026842+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:52.282595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.034705+0000","last_change":"2023-12-01T01:52:48.034705+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.034705+0000","last_undegraded":"2023-12-01T01:52:48.034705+0000","last_fullsized":"2023-12-01T01:52:48.034705+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029077+0000","last_change":"2023-12-01T01:52:48.029077+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029077+0000","last_undegraded":"2023-12-01T01:52:48.029077+0000","last_fullsized":"2023-12-01T01:52:48.029077+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025888+0000","last_change":"2023-12-01T01:52:48.025888+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025888+0000","last_undegraded":"2023-12-01T01:52:48.025888+0000","last_fullsized":"2023-12-01T01:52:48.025888+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021972+0000","last_change":"2023-12-01T01:52:48.021972+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021972+0000","last_undegraded":"2023-12-01T01:52:48.021972+0000","last_fullsized":"2023-12-01T01:52:48.021972+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:45:17.044474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.037434+0000","last_change":"2023-12-01T01:52:48.037434+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.037434+0000","last_undegraded":"2023-12-01T01:52:48.037434+0000","last_fullsized":"2023-12-01T01:52:48.037434+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:40:59.300637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.019592+0000","last_change":"2023-12-01T01:52:48.019592+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.019592+0000","last_undegraded":"2023-12-01T01:52:48.019592+0000","last_fullsized":"2023-12-01T01:52:48.019592+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:58:23.518943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029503+0000","last_change":"2023-12-01T01:52:48.029503+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029503+0000","last_undegraded":"2023-12-01T01:52:48.029503+0000","last_fullsized":"2023-12-01T01:52:48.029503+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024646+0000","last_change":"2023-12-01T01:52:48.024646+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024646+0000","last_undegraded":"2023-12-01T01:52:48.024646+0000","last_fullsized":"2023-12-01T01:52:48.024646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029391+0000","last_change":"2023-12-01T01:52:48.029391+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029391+0000","last_undegraded":"2023-12-01T01:52:48.029391+0000","last_fullsized":"2023-12-01T01:52:48.029391+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.038499+0000","last_change":"2023-12-01T01:52:48.038499+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.038499+0000","last_undegraded":"2023-12-01T01:52:48.038499+0000","last_fullsized":"2023-12-01T01:52:48.038499+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:13:33.646688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035539+0000","last_change":"2023-12-01T01:52:48.035539+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035539+0000","last_undegraded":"2023-12-01T01:52:48.035539+0000","last_fullsized":"2023-12-01T01:52:48.035539+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:42.942148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026846+0000","last_change":"2023-12-01T01:52:48.026846+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026846+0000","last_undegraded":"2023-12-01T01:52:48.026846+0000","last_fullsized":"2023-12-01T01:52:48.026846+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.036892+0000","last_change":"2023-12-01T01:52:48.036892+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.036892+0000","last_undegraded":"2023-12-01T01:52:48.036892+0000","last_fullsized":"2023-12-01T01:52:48.036892+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027150+0000","last_change":"2023-12-01T01:52:48.027150+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027150+0000","last_undegraded":"2023-12-01T01:52:48.027150+0000","last_fullsized":"2023-12-01T01:52:48.027150+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.030146+0000","last_change":"2023-12-01T01:52:48.030146+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.030146+0000","last_undegraded":"2023-12-01T01:52:48.030146+0000","last_fullsized":"2023-12-01T01:52:48.030146+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.031458+0000","last_change":"2023-12-01T01:52:48.031458+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.031458+0000","last_undegraded":"2023-12-01T01:52:48.031458+0000","last_fullsized":"2023-12-01T01:52:48.031458+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023595+0000","last_change":"2023-12-01T01:52:48.023595+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023595+0000","last_undegraded":"2023-12-01T01:52:48.023595+0000","last_fullsized":"2023-12-01T01:52:48.023595+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:22.154067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027885+0000","last_change":"2023-12-01T01:52:48.027885+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027885+0000","last_undegraded":"2023-12-01T01:52:48.027885+0000","last_fullsized":"2023-12-01T01:52:48.027885+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:12:02.919659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021893+0000","last_change":"2023-12-01T01:52:48.021893+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021893+0000","last_undegraded":"2023-12-01T01:52:48.021893+0000","last_fullsized":"2023-12-01T01:52:48.021893+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:25:02.674823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020506+0000","last_change":"2023-12-01T01:52:48.020506+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020506+0000","last_undegraded":"2023-12-01T01:52:48.020506+0000","last_fullsized":"2023-12-01T01:52:48.020506+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035671+0000","last_change":"2023-12-01T01:52:48.035671+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035671+0000","last_undegraded":"2023-12-01T01:52:48.035671+0000","last_fullsized":"2023-12-01T01:52:48.035671+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:00.516734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.034799+0000","last_change":"2023-12-01T01:52:48.034799+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.034799+0000","last_undegraded":"2023-12-01T01:52:48.034799+0000","last_fullsized":"2023-12-01T01:52:48.034799+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.028380+0000","last_change":"2023-12-01T01:52:48.028380+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.028380+0000","last_undegraded":"2023-12-01T01:52:48.028380+0000","last_fullsized":"2023-12-01T01:52:48.028380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026751+0000","last_change":"2023-12-01T01:52:48.026751+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026751+0000","last_undegraded":"2023-12-01T01:52:48.026751+0000","last_fullsized":"2023-12-01T01:52:48.026751+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:02:42.893820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041182+0000","last_change":"2023-12-01T01:52:48.038820+0000","last_active":"2023-12-01T01:52:48.041182+0000","last_peered":"2023-12-01T01:52:48.041182+0000","last_clean":"2023-12-01T01:52:48.041182+0000","last_became_active":"2023-12-01T01:52:48.038539+0000","last_became_peered":"2023-12-01T01:52:48.038539+0000","last_unstale":"2023-12-01T01:52:48.041182+0000","last_undegraded":"2023-12-01T01:52:48.041182+0000","last_fullsized":"2023-12-01T01:52:48.041182+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:21.090681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026112+0000","last_change":"2023-12-01T01:52:48.026112+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026112+0000","last_undegraded":"2023-12-01T01:52:48.026112+0000","last_fullsized":"2023-12-01T01:52:48.026112+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:01.801261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.050678+0000","last_change":"2023-12-01T01:52:48.049643+0000","last_active":"2023-12-01T01:52:48.050678+0000","last_peered":"2023-12-01T01:52:48.050678+0000","last_clean":"2023-12-01T01:52:48.050678+0000","last_became_active":"2023-12-01T01:52:48.049288+0000","last_became_peered":"2023-12-01T01:52:48.049288+0000","last_unstale":"2023-12-01T01:52:48.050678+0000","last_undegraded":"2023-12-01T01:52:48.050678+0000","last_fullsized":"2023-12-01T01:52:48.050678+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:41.937860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024925+0000","last_change":"2023-12-01T01:52:48.024925+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024925+0000","last_undegraded":"2023-12-01T01:52:48.024925+0000","last_fullsized":"2023-12-01T01:52:48.024925+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:50:17.533885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044029+0000","last_change":"2023-12-01T01:52:48.039238+0000","last_active":"2023-12-01T01:52:48.044029+0000","last_peered":"2023-12-01T01:52:48.044029+0000","last_clean":"2023-12-01T01:52:48.044029+0000","last_became_active":"2023-12-01T01:52:48.038835+0000","last_became_peered":"2023-12-01T01:52:48.038835+0000","last_unstale":"2023-12-01T01:52:48.044029+0000","last_undegraded":"2023-12-01T01:52:48.044029+0000","last_fullsized":"2023-12-01T01:52:48.044029+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023506+0000","last_change":"2023-12-01T01:52:48.023506+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023506+0000","last_undegraded":"2023-12-01T01:52:48.023506+0000","last_fullsized":"2023-12-01T01:52:48.023506+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.048403+0000","last_change":"2023-12-01T01:52:48.045716+0000","last_active":"2023-12-01T01:52:48.048403+0000","last_peered":"2023-12-01T01:52:48.048403+0000","last_clean":"2023-12-01T01:52:48.048403+0000","last_became_active":"2023-12-01T01:52:48.045176+0000","last_became_peered":"2023-12-01T01:52:48.045176+0000","last_unstale":"2023-12-01T01:52:48.048403+0000","last_undegraded":"2023-12-01T01:52:48.048403+0000","last_fullsized":"2023-12-01T01:52:48.048403+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:55:21.695057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026841+0000","last_change":"2023-12-01T01:52:48.026841+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026841+0000","last_undegraded":"2023-12-01T01:52:48.026841+0000","last_fullsized":"2023-12-01T01:52:48.026841+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.036263+0000","last_change":"2023-12-01T01:52:48.033877+0000","last_active":"2023-12-01T01:52:48.036263+0000","last_peered":"2023-12-01T01:52:48.036263+0000","last_clean":"2023-12-01T01:52:48.036263+0000","last_became_active":"2023-12-01T01:52:48.033605+0000","last_became_peered":"2023-12-01T01:52:48.033605+0000","last_unstale":"2023-12-01T01:52:48.036263+0000","last_undegraded":"2023-12-01T01:52:48.036263+0000","last_fullsized":"2023-12-01T01:52:48.036263+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:55.538925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.032197+0000","last_change":"2023-12-01T01:52:48.032197+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.032197+0000","last_undegraded":"2023-12-01T01:52:48.032197+0000","last_fullsized":"2023-12-01T01:52:48.032197+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:53:05.715094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.046497+0000","last_change":"2023-12-01T01:52:48.043431+0000","last_active":"2023-12-01T01:52:48.046497+0000","last_peered":"2023-12-01T01:52:48.046497+0000","last_clean":"2023-12-01T01:52:48.046497+0000","last_became_active":"2023-12-01T01:52:48.043162+0000","last_became_peered":"2023-12-01T01:52:48.043162+0000","last_unstale":"2023-12-01T01:52:48.046497+0000","last_undegraded":"2023-12-01T01:52:48.046497+0000","last_fullsized":"2023-12-01T01:52:48.046497+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.022665+0000","last_change":"2023-12-01T01:52:48.022665+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.022665+0000","last_undegraded":"2023-12-01T01:52:48.022665+0000","last_fullsized":"2023-12-01T01:52:48.022665+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:10:51.868019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049031+0000","last_change":"2023-12-01T01:52:48.046040+0000","last_active":"2023-12-01T01:52:48.049031+0000","last_peered":"2023-12-01T01:52:48.049031+0000","last_clean":"2023-12-01T01:52:48.049031+0000","last_became_active":"2023-12-01T01:52:48.045722+0000","last_became_peered":"2023-12-01T01:52:48.045722+0000","last_unstale":"2023-12-01T01:52:48.049031+0000","last_undegraded":"2023-12-01T01:52:48.049031+0000","last_fullsized":"2023-12-01T01:52:48.049031+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:25:01.670535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035505+0000","last_change":"2023-12-01T01:52:48.035505+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035505+0000","last_undegraded":"2023-12-01T01:52:48.035505+0000","last_fullsized":"2023-12-01T01:52:48.035505+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:38.690885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.051042+0000","last_change":"2023-12-01T01:52:48.049260+0000","last_active":"2023-12-01T01:52:48.051042+0000","last_peered":"2023-12-01T01:52:48.051042+0000","last_clean":"2023-12-01T01:52:48.051042+0000","last_became_active":"2023-12-01T01:52:48.048998+0000","last_became_peered":"2023-12-01T01:52:48.048998+0000","last_unstale":"2023-12-01T01:52:48.051042+0000","last_undegraded":"2023-12-01T01:52:48.051042+0000","last_fullsized":"2023-12-01T01:52:48.051042+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:06.546333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035689+0000","last_change":"2023-12-01T01:52:48.035689+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035689+0000","last_undegraded":"2023-12-01T01:52:48.035689+0000","last_fullsized":"2023-12-01T01:52:48.035689+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:12.516160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.037274+0000","last_change":"2023-12-01T01:52:48.035560+0000","last_active":"2023-12-01T01:52:48.037274+0000","last_peered":"2023-12-01T01:52:48.037274+0000","last_clean":"2023-12-01T01:52:48.037274+0000","last_became_active":"2023-12-01T01:52:48.035303+0000","last_became_peered":"2023-12-01T01:52:48.035303+0000","last_unstale":"2023-12-01T01:52:48.037274+0000","last_undegraded":"2023-12-01T01:52:48.037274+0000","last_fullsized":"2023-12-01T01:52:48.037274+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:04.573692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025380+0000","last_change":"2023-12-01T01:52:48.025380+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025380+0000","last_undegraded":"2023-12-01T01:52:48.025380+0000","last_fullsized":"2023-12-01T01:52:48.025380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:22.643458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041963+0000","last_change":"2023-12-01T01:52:48.040913+0000","last_active":"2023-12-01T01:52:48.041963+0000","last_peered":"2023-12-01T01:52:48.041963+0000","last_clean":"2023-12-01T01:52:48.041963+0000","last_became_active":"2023-12-01T01:52:48.040402+0000","last_became_peered":"2023-12-01T01:52:48.040402+0000","last_unstale":"2023-12-01T01:52:48.041963+0000","last_undegraded":"2023-12-01T01:52:48.041963+0000","last_fullsized":"2023-12-01T01:52:48.041963+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027539+0000","last_change":"2023-12-01T01:52:48.027539+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027539+0000","last_undegraded":"2023-12-01T01:52:48.027539+0000","last_fullsized":"2023-12-01T01:52:48.027539+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.047008+0000","last_change":"2023-12-01T01:52:48.045583+0000","last_active":"2023-12-01T01:52:48.047008+0000","last_peered":"2023-12-01T01:52:48.047008+0000","last_clean":"2023-12-01T01:52:48.047008+0000","last_became_active":"2023-12-01T01:52:48.045146+0000","last_became_peered":"2023-12-01T01:52:48.045146+0000","last_unstale":"2023-12-01T01:52:48.047008+0000","last_undegraded":"2023-12-01T01:52:48.047008+0000","last_fullsized":"2023-12-01T01:52:48.047008+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:25.868578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.028382+0000","last_change":"2023-12-01T01:52:48.028382+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.028382+0000","last_undegraded":"2023-12-01T01:52:48.028382+0000","last_fullsized":"2023-12-01T01:52:48.028382+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:03.971637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044477+0000","last_change":"2023-12-01T01:52:48.042396+0000","last_active":"2023-12-01T01:52:48.044477+0000","last_peered":"2023-12-01T01:52:48.044477+0000","last_clean":"2023-12-01T01:52:48.044477+0000","last_became_active":"2023-12-01T01:52:48.041958+0000","last_became_peered":"2023-12-01T01:52:48.041958+0000","last_unstale":"2023-12-01T01:52:48.044477+0000","last_undegraded":"2023-12-01T01:52:48.044477+0000","last_fullsized":"2023-12-01T01:52:48.044477+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:33.265613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024486+0000","last_change":"2023-12-01T01:52:48.024486+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024486+0000","last_undegraded":"2023-12-01T01:52:48.024486+0000","last_fullsized":"2023-12-01T01:52:48.024486+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.039978+0000","last_change":"2023-12-01T01:52:48.035637+0000","last_active":"2023-12-01T01:52:48.039978+0000","last_peered":"2023-12-01T01:52:48.039978+0000","last_clean":"2023-12-01T01:52:48.039978+0000","last_became_active":"2023-12-01T01:52:48.034955+0000","last_became_peered":"2023-12-01T01:52:48.034955+0000","last_unstale":"2023-12-01T01:52:48.039978+0000","last_undegraded":"2023-12-01T01:52:48.039978+0000","last_fullsized":"2023-12-01T01:52:48.039978+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:02.433217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025388+0000","last_change":"2023-12-01T01:52:48.025388+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025388+0000","last_undegraded":"2023-12-01T01:52:48.025388+0000","last_fullsized":"2023-12-01T01:52:48.025388+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:58:00.440400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.037568+0000","last_change":"2023-12-01T01:52:48.033331+0000","last_active":"2023-12-01T01:52:48.037568+0000","last_peered":"2023-12-01T01:52:48.037568+0000","last_clean":"2023-12-01T01:52:48.037568+0000","last_became_active":"2023-12-01T01:52:48.032852+0000","last_became_peered":"2023-12-01T01:52:48.032852+0000","last_unstale":"2023-12-01T01:52:48.037568+0000","last_undegraded":"2023-12-01T01:52:48.037568+0000","last_fullsized":"2023-12-01T01:52:48.037568+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:37.686598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.036643+0000","last_change":"2023-12-01T01:52:48.036643+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.036643+0000","last_undegraded":"2023-12-01T01:52:48.036643+0000","last_fullsized":"2023-12-01T01:52:48.036643+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:45.833443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.042163+0000","last_change":"2023-12-01T01:52:48.041648+0000","last_active":"2023-12-01T01:52:48.042163+0000","last_peered":"2023-12-01T01:52:48.042163+0000","last_clean":"2023-12-01T01:52:48.042163+0000","last_became_active":"2023-12-01T01:52:48.040913+0000","last_became_peered":"2023-12-01T01:52:48.040913+0000","last_unstale":"2023-12-01T01:52:48.042163+0000","last_undegraded":"2023-12-01T01:52:48.042163+0000","last_fullsized":"2023-12-01T01:52:48.042163+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:37.686598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023509+0000","last_change":"2023-12-01T01:52:48.023509+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023509+0000","last_undegraded":"2023-12-01T01:52:48.023509+0000","last_fullsized":"2023-12-01T01:52:48.023509+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:56.543213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.034765+0000","last_change":"2023-12-01T01:52:48.032822+0000","last_active":"2023-12-01T01:52:48.034765+0000","last_peered":"2023-12-01T01:52:48.034765+0000","last_clean":"2023-12-01T01:52:48.034765+0000","last_became_active":"2023-12-01T01:52:48.032368+0000","last_became_peered":"2023-12-01T01:52:48.032368+0000","last_unstale":"2023-12-01T01:52:48.034765+0000","last_undegraded":"2023-12-01T01:52:48.034765+0000","last_fullsized":"2023-12-01T01:52:48.034765+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:10:50.863731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026337+0000","last_change":"2023-12-01T01:52:48.026337+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026337+0000","last_undegraded":"2023-12-01T01:52:48.026337+0000","last_fullsized":"2023-12-01T01:52:48.026337+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:09:36.618385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041816+0000","last_change":"2023-12-01T01:52:48.040258+0000","last_active":"2023-12-01T01:52:48.041816+0000","last_peered":"2023-12-01T01:52:48.041816+0000","last_clean":"2023-12-01T01:52:48.041816+0000","last_became_active":"2023-12-01T01:52:48.037376+0000","last_became_peered":"2023-12-01T01:52:48.037376+0000","last_unstale":"2023-12-01T01:52:48.041816+0000","last_undegraded":"2023-12-01T01:52:48.041816+0000","last_fullsized":"2023-12-01T01:52:48.041816+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:13.111537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.033821+0000","last_change":"2023-12-01T01:52:48.033821+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.033821+0000","last_undegraded":"2023-12-01T01:52:48.033821+0000","last_fullsized":"2023-12-01T01:52:48.033821+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.051066+0000","last_change":"2023-12-01T01:52:48.050044+0000","last_active":"2023-12-01T01:52:48.051066+0000","last_peered":"2023-12-01T01:52:48.051066+0000","last_clean":"2023-12-01T01:52:48.051066+0000","last_became_active":"2023-12-01T01:52:48.049779+0000","last_became_peered":"2023-12-01T01:52:48.049779+0000","last_unstale":"2023-12-01T01:52:48.051066+0000","last_undegraded":"2023-12-01T01:52:48.051066+0000","last_fullsized":"2023-12-01T01:52:48.051066+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:20:19.462886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026154+0000","last_change":"2023-12-01T01:52:48.026154+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026154+0000","last_undegraded":"2023-12-01T01:52:48.026154+0000","last_fullsized":"2023-12-01T01:52:48.026154+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:17.235352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.050226+0000","last_change":"2023-12-01T01:52:48.048859+0000","last_active":"2023-12-01T01:52:48.050226+0000","last_peered":"2023-12-01T01:52:48.050226+0000","last_clean":"2023-12-01T01:52:48.050226+0000","last_became_active":"2023-12-01T01:52:48.048546+0000","last_became_peered":"2023-12-01T01:52:48.048546+0000","last_unstale":"2023-12-01T01:52:48.050226+0000","last_undegraded":"2023-12-01T01:52:48.050226+0000","last_fullsized":"2023-12-01T01:52:48.050226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:43.307966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021314+0000","last_change":"2023-12-01T01:52:48.021314+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021314+0000","last_undegraded":"2023-12-01T01:52:48.021314+0000","last_fullsized":"2023-12-01T01:52:48.021314+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049328+0000","last_change":"2023-12-01T01:52:48.046685+0000","last_active":"2023-12-01T01:52:48.049328+0000","last_peered":"2023-12-01T01:52:48.049328+0000","last_clean":"2023-12-01T01:52:48.049328+0000","last_became_active":"2023-12-01T01:52:48.046395+0000","last_became_peered":"2023-12-01T01:52:48.046395+0000","last_unstale":"2023-12-01T01:52:48.049328+0000","last_undegraded":"2023-12-01T01:52:48.049328+0000","last_fullsized":"2023-12-01T01:52:48.049328+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:29:10.130713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021393+0000","last_change":"2023-12-01T01:52:48.021393+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021393+0000","last_undegraded":"2023-12-01T01:52:48.021393+0000","last_fullsized":"2023-12-01T01:52:48.021393+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:52:45.545702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041831+0000","last_change":"2023-12-01T01:52:48.038141+0000","last_active":"2023-12-01T01:52:48.041831+0000","last_peered":"2023-12-01T01:52:48.041831+0000","last_clean":"2023-12-01T01:52:48.041831+0000","last_became_active":"2023-12-01T01:52:48.037797+0000","last_became_peered":"2023-12-01T01:52:48.037797+0000","last_unstale":"2023-12-01T01:52:48.041831+0000","last_undegraded":"2023-12-01T01:52:48.041831+0000","last_fullsized":"2023-12-01T01:52:48.041831+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021667+0000","last_change":"2023-12-01T01:52:48.021667+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021667+0000","last_undegraded":"2023-12-01T01:52:48.021667+0000","last_fullsized":"2023-12-01T01:52:48.021667+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.043878+0000","last_change":"2023-12-01T01:52:48.042643+0000","last_active":"2023-12-01T01:52:48.043878+0000","last_peered":"2023-12-01T01:52:48.043878+0000","last_clean":"2023-12-01T01:52:48.043878+0000","last_became_active":"2023-12-01T01:52:48.042174+0000","last_became_peered":"2023-12-01T01:52:48.042174+0000","last_unstale":"2023-12-01T01:52:48.043878+0000","last_undegraded":"2023-12-01T01:52:48.043878+0000","last_fullsized":"2023-12-01T01:52:48.043878+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:59.436113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026638+0000","last_change":"2023-12-01T01:52:48.026638+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026638+0000","last_undegraded":"2023-12-01T01:52:48.026638+0000","last_fullsized":"2023-12-01T01:52:48.026638+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049368+0000","last_change":"2023-12-01T01:52:48.047235+0000","last_active":"2023-12-01T01:52:48.049368+0000","last_peered":"2023-12-01T01:52:48.049368+0000","last_clean":"2023-12-01T01:52:48.049368+0000","last_became_active":"2023-12-01T01:52:48.046953+0000","last_became_peered":"2023-12-01T01:52:48.046953+0000","last_unstale":"2023-12-01T01:52:48.049368+0000","last_undegraded":"2023-12-01T01:52:48.049368+0000","last_fullsized":"2023-12-01T01:52:48.049368+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:13.492174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023160+0000","last_change":"2023-12-01T01:52:48.023160+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023160+0000","last_undegraded":"2023-12-01T01:52:48.023160+0000","last_fullsized":"2023-12-01T01:52:48.023160+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:39.608737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.051849+0000","last_change":"2023-12-01T01:52:48.050705+0000","last_active":"2023-12-01T01:52:48.051849+0000","last_peered":"2023-12-01T01:52:48.051849+0000","last_clean":"2023-12-01T01:52:48.051849+0000","last_became_active":"2023-12-01T01:52:48.050445+0000","last_became_peered":"2023-12-01T01:52:48.050445+0000","last_unstale":"2023-12-01T01:52:48.051849+0000","last_undegraded":"2023-12-01T01:52:48.051849+0000","last_fullsized":"2023-12-01T01:52:48.051849+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:02:56.065815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025759+0000","last_change":"2023-12-01T01:52:48.025759+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025759+0000","last_undegraded":"2023-12-01T01:52:48.025759+0000","last_fullsized":"2023-12-01T01:52:48.025759+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:55:14.304687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.042923+0000","last_change":"2023-12-01T01:52:48.040799+0000","last_active":"2023-12-01T01:52:48.042923+0000","last_peered":"2023-12-01T01:52:48.042923+0000","last_clean":"2023-12-01T01:52:48.042923+0000","last_became_active":"2023-12-01T01:52:48.040471+0000","last_became_peered":"2023-12-01T01:52:48.040471+0000","last_unstale":"2023-12-01T01:52:48.042923+0000","last_undegraded":"2023-12-01T01:52:48.042923+0000","last_fullsized":"2023-12-01T01:52:48.042923+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021428+0000","last_change":"2023-12-01T01:52:48.021428+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021428+0000","last_undegraded":"2023-12-01T01:52:48.021428+0000","last_fullsized":"2023-12-01T01:52:48.021428+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:35:01.791105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044954+0000","last_change":"2023-12-01T01:52:48.043497+0000","last_active":"2023-12-01T01:52:48.044954+0000","last_peered":"2023-12-01T01:52:48.044954+0000","last_clean":"2023-12-01T01:52:48.044954+0000","last_became_active":"2023-12-01T01:52:48.043198+0000","last_became_peered":"2023-12-01T01:52:48.043198+0000","last_unstale":"2023-12-01T01:52:48.044954+0000","last_undegraded":"2023-12-01T01:52:48.044954+0000","last_fullsized":"2023-12-01T01:52:48.044954+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:44.930335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.033844+0000","last_change":"2023-12-01T01:52:48.033844+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.033844+0000","last_undegraded":"2023-12-01T01:52:48.033844+0000","last_fullsized":"2023-12-01T01:52:48.033844+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:03.437505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049785+0000","last_change":"2023-12-01T01:52:48.046182+0000","last_active":"2023-12-01T01:52:48.049785+0000","last_peered":"2023-12-01T01:52:48.049785+0000","last_clean":"2023-12-01T01:52:48.049785+0000","last_became_active":"2023-12-01T01:52:48.045696+0000","last_became_peered":"2023-12-01T01:52:48.045696+0000","last_unstale":"2023-12-01T01:52:48.049785+0000","last_undegraded":"2023-12-01T01:52:48.049785+0000","last_fullsized":"2023-12-01T01:52:48.049785+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:44.829155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020256+0000","last_change":"2023-12-01T01:52:48.020256+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020256+0000","last_undegraded":"2023-12-01T01:52:48.020256+0000","last_fullsized":"2023-12-01T01:52:48.020256+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:34:49.188317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044461+0000","last_change":"2023-12-01T01:52:48.042910+0000","last_active":"2023-12-01T01:52:48.044461+0000","last_peered":"2023-12-01T01:52:48.044461+0000","last_clean":"2023-12-01T01:52:48.044461+0000","last_became_active":"2023-12-01T01:52:48.041897+0000","last_became_peered":"2023-12-01T01:52:48.041897+0000","last_unstale":"2023-12-01T01:52:48.044461+0000","last_undegraded":"2023-12-01T01:52:48.044461+0000","last_fullsized":"2023-12-01T01:52:48.044461+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027872+0000","last_change":"2023-12-01T01:52:48.027872+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027872+0000","last_undegraded":"2023-12-01T01:52:48.027872+0000","last_fullsized":"2023-12-01T01:52:48.027872+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:45.934623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.047598+0000","last_change":"2023-12-01T01:52:48.044493+0000","last_active":"2023-12-01T01:52:48.047598+0000","last_peered":"2023-12-01T01:52:48.047598+0000","last_clean":"2023-12-01T01:52:48.047598+0000","last_became_active":"2023-12-01T01:52:48.044221+0000","last_became_peered":"2023-12-01T01:52:48.044221+0000","last_unstale":"2023-12-01T01:52:48.047598+0000","last_undegraded":"2023-12-01T01:52:48.047598+0000","last_fullsized":"2023-12-01T01:52:48.047598+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:59.436113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.019339+0000","last_change":"2023-12-01T01:52:48.019339+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.019339+0000","last_undegraded":"2023-12-01T01:52:48.019339+0000","last_fullsized":"2023-12-01T01:52:48.019339+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:20:23.997177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.046763+0000","last_change":"2023-12-01T01:52:48.045085+0000","last_active":"2023-12-01T01:52:48.046763+0000","last_peered":"2023-12-01T01:52:48.046763+0000","last_clean":"2023-12-01T01:52:48.046763+0000","last_became_active":"2023-12-01T01:52:48.044789+0000","last_became_peered":"2023-12-01T01:52:48.044789+0000","last_unstale":"2023-12-01T01:52:48.046763+0000","last_undegraded":"2023-12-01T01:52:48.046763+0000","last_fullsized":"2023-12-01T01:52:48.046763+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:58:00.466321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020747+0000","last_change":"2023-12-01T01:52:48.020747+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020747+0000","last_undegraded":"2023-12-01T01:52:48.020747+0000","last_fullsized":"2023-12-01T01:52:48.020747+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:02:57.070103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.039567+0000","last_change":"2023-12-01T01:52:48.034635+0000","last_active":"2023-12-01T01:52:48.039567+0000","last_peered":"2023-12-01T01:52:48.039567+0000","last_clean":"2023-12-01T01:52:48.039567+0000","last_became_active":"2023-12-01T01:52:48.034401+0000","last_became_peered":"2023-12-01T01:52:48.034401+0000","last_unstale":"2023-12-01T01:52:48.039567+0000","last_undegraded":"2023-12-01T01:52:48.039567+0000","last_fullsized":"2023-12-01T01:52:48.039567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:19.195130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026074+0000","last_change":"2023-12-01T01:52:48.026074+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026074+0000","last_undegraded":"2023-12-01T01:52:48.026074+0000","last_fullsized":"2023-12-01T01:52:48.026074+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.045370+0000","last_change":"2023-12-01T01:52:48.043879+0000","last_active":"2023-12-01T01:52:48.045370+0000","last_peered":"2023-12-01T01:52:48.045370+0000","last_clean":"2023-12-01T01:52:48.045370+0000","last_became_active":"2023-12-01T01:52:48.043357+0000","last_became_peered":"2023-12-01T01:52:48.043357+0000","last_unstale":"2023-12-01T01:52:48.045370+0000","last_undegraded":"2023-12-01T01:52:48.045370+0000","last_fullsized":"2023-12-01T01:52:48.045370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:59.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'183","reported_seq":481,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-12-01T01:52:47.018333+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:52:47.018333+0000","last_peered":"2023-12-01T01:52:47.018333+0000","last_clean":"2023-12-01T01:52:47.018333+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:52:47.018333+0000","last_undegraded":"2023-12-01T01:52:47.018333+0000","last_fullsized":"2023-12-01T01:52:47.018333+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575051,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.02,"15min":1.0660000000000001},"min":{"1min":0.54900000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.401,"5min":1.514,"15min":1.8089999999999999},"last":1.145},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0309999999999999,"15min":1.0469999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5289999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.0469999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1499999999999999,"15min":1.2070000000000001},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.5669999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2150000000000001,"15min":1.248},"min":{"1min":0.66800000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6279999999999999,"5min":1.744,"15min":1.744},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1040000000000001,"15min":1.117},"min":{"1min":0.749,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4910000000000001,"5min":1.607,"15min":1.728},"last":1.244},{"interface":"front","average":{"1min":1.2,"5min":1.2010000000000001,"15min":1.2190000000000001},"min":{"1min":0.92500000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.641,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.274}]},{"osd":3,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.1910000000000001,"15min":1.2230000000000001},"min":{"1min":0.64200000000000002,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.347,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3180000000000001,"15min":1.3420000000000001},"min":{"1min":0.63700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7490000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.3360000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.3009999999999999,"15min":1.2849999999999999},"min":{"1min":0.72699999999999998,"5min":0.56100000000000005,"15min":0.55500000000000005},"max":{"1min":1.4450000000000001,"5min":1.7549999999999999,"15min":2.2130000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4350000000000001,"15min":1.4059999999999999},"min":{"1min":0.77500000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8100000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.008}]},{"osd":5,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.292,"15min":1.258},"min":{"1min":0.625,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.0379999999999998,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.47,"5min":1.514,"15min":1.514},"min":{"1min":0.69399999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.861,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.3859999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.395,"15min":1.377},"min":{"1min":0.753,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1549999999999998,"5min":2.294,"15min":2.3420000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.387},"min":{"1min":0.82599999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.893,"5min":1.893,"15min":2.3500000000000001},"last":1.0920000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.1020000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.79},"last":1.034},{"interface":"front","average":{"1min":1.036,"5min":1.105,"15min":1.1220000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5349999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.999}]},{"osd":1,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.161,"15min":1.159},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1950000000000001,"15min":1.2050000000000001},"min":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0920000000000001},"last":1.204}]},{"osd":2,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1859999999999999,"15min":1.171},"min":{"1min":0.69699999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5760000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.754},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.242,"15min":1.2350000000000001},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":3.024,"5min":3.024,"15min":3.024},"last":1.1499999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":1.0289999999999999,"5min":0.93700000000000006,"15min":0.82399999999999995},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.48}]},{"osd":4,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3,"15min":1.2709999999999999},"min":{"1min":0.94299999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6819999999999999,"5min":2.226,"15min":2.226},"last":1.238},{"interface":"front","average":{"1min":1.339,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":1.028,"5min":0.91400000000000003,"15min":0.78100000000000003},"max":{"1min":1.885,"5min":1.931,"15min":2.2839999999999998},"last":1.5800000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.415,"15min":1.397},"min":{"1min":0.755,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.74,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3},{"interface":"front","average":{"1min":1.47,"5min":1.502,"15min":1.472},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.84099999999999997},"max":{"1min":1.9790000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.6799999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.524,"15min":1.5009999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6870000000000001,"5min":2.762,"15min":2.762},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.577,"15min":1.5629999999999999},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":1.9330000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.742}]}]},{"osd":5,"up_from":13,"seq":55834575051,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32924,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338916,"statfs":{"total":96636764160,"available":96603049984,"internally_reserved":0,"allocated":1921024,"data_stored":1219865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1259999999999999,"15min":1.1040000000000001},"min":{"1min":0.63200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.2010000000000001},"last":1.099},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1180000000000001,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.60099999999999998},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":2.153},"last":1.3700000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.117,"15min":1.1399999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.63200000000000001},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.1399999999999999,"15min":1.159},"min":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.6519999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.2430000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.173,"15min":1.196},"min":{"1min":0.65000000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.702,"5min":1.702,"15min":1.7110000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.208,"15min":1.2330000000000001},"min":{"1min":0.79100000000000004,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":2.0630000000000002},"last":1.1990000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.248,"15min":1.282},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.8169999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.302},"min":{"1min":1.159,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.95099999999999996}]},{"osd":4,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2410000000000001},"min":{"1min":0.83499999999999996,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.018},{"interface":"front","average":{"1min":1.409,"5min":1.319,"15min":1.3480000000000001},"min":{"1min":0.84399999999999997,"5min":0.80300000000000005,"15min":0.79900000000000004},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.452}]},{"osd":6,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3129999999999999,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.859,"5min":1.885,"15min":1.984},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.3939999999999999,"15min":1.3839999999999999},"min":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.149}]},{"osd":7,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.462,"15min":1.4750000000000001},"min":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0920000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.522,"15min":1.5269999999999999},"min":{"1min":1.304,"5min":0.873,"15min":0.873},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.411}]}]},{"osd":4,"up_from":13,"seq":55834575051,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1120000000000001,"15min":1.105},"min":{"1min":0.72199999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.753,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.7130000000000001,"5min":1.7529999999999999,"15min":2.1099999999999999},"last":1.657}]},{"osd":1,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.216,"15min":1.204},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.65900000000000003},"max":{"1min":1.5409999999999999,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.534},{"interface":"front","average":{"1min":1.087,"5min":1.101,"15min":1.129},"min":{"1min":0.76600000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3979999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.458}]},{"osd":2,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.234,"15min":1.2},"min":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.716,"5min":1.871,"15min":1.871},"last":1.204},{"interface":"front","average":{"1min":1.327,"5min":1.3180000000000001,"15min":1.288},"min":{"1min":0.94599999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.581,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.76}]},{"osd":3,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.258,"15min":1.2310000000000001},"min":{"1min":0.81899999999999995,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.6140000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.139},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3200000000000001,"15min":1.3160000000000001},"min":{"1min":0.88500000000000001,"5min":0.753,"15min":0.753},"max":{"1min":1.712,"5min":1.7270000000000001,"15min":1.903},"last":1.3009999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.319,"15min":1.3129999999999999},"min":{"1min":0.94899999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.288,"15min":1.2869999999999999},"min":{"1min":0.97199999999999998,"5min":0.90300000000000002,"15min":0.85399999999999998},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.24}]},{"osd":6,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.381,"15min":1.3500000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.81299999999999994},"max":{"1min":1.8799999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.502,"15min":1.472},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.0960000000000001,"5min":2.4990000000000001,"15min":2.54},"last":1.71}]},{"osd":7,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.472,"15min":1.4610000000000001},"min":{"1min":1.274,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5780000000000001,"15min":1.5620000000000001},"min":{"1min":1.28,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":1.96,"5min":2.04,"15min":2.157},"last":1.486}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58584,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313256,"statfs":{"total":96636764160,"available":96576774144,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.84199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.307},"max":{"1min":1.1899999999999999,"5min":1.3320000000000001,"15min":1.583},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.93500000000000005,"15min":0.97999999999999998},"min":{"1min":0.48099999999999998,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.99199999999999999,"15min":1.056},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.38900000000000001},"max":{"1min":1.5669999999999999,"5min":1.571,"15min":2.0779999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.89300000000000002,"15min":0.94799999999999995},"min":{"1min":0.48399999999999999,"5min":0.48199999999999998,"15min":0.42699999999999999},"max":{"1min":1.1339999999999999,"5min":1.55,"15min":1.6519999999999999},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.96699999999999997,"15min":1.0349999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3360000000000001,"5min":1.4810000000000001,"15min":1.5609999999999999},"last":1.167},{"interface":"front","average":{"1min":0.995,"5min":1.125,"15min":1.1819999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1970000000000001,"15min":1.232},"min":{"1min":0.84999999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.524,"5min":1.5820000000000001,"15min":1.9319999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.198,"15min":1.232},"min":{"1min":0.79300000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.909,"5min":1.909,"15min":2.0419999999999998},"last":0.96099999999999997}]},{"osd":5,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.228,"15min":1.2789999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.71,"5min":1.7669999999999999,"15min":2.0510000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2529999999999999,"15min":1.288},"min":{"1min":0.745,"5min":0.745,"15min":0.64200000000000002},"max":{"1min":1.6759999999999999,"5min":1.681,"15min":2.1880000000000002},"last":1.274}]},{"osd":6,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.89600000000000002,"5min":0.76300000000000001,"15min":0.68300000000000005},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.995},{"interface":"front","average":{"1min":1.264,"5min":1.3700000000000001,"15min":1.405},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8260000000000001,"5min":1.9119999999999999,"15min":1.99},"last":1.4259999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.365,"15min":1.413},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.81499999999999995},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.1619999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4019999999999999,"15min":1.4470000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.83099999999999996},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.2130000000000001},"last":1.0940000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":0.81699999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.5529999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.98399999999999999,"15min":1},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.97099999999999997}]},{"osd":1,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95799999999999996,"15min":0.98699999999999999},"min":{"1min":0.54900000000000004,"5min":0.373,"15min":0.373},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":1.042},{"interface":"front","average":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.081},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.278}]},{"osd":3,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93899999999999995,"15min":1.002},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5189999999999999,"5min":1.625,"15min":1.6479999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.139}]},{"osd":4,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.869,"5min":1.9950000000000001,"15min":2.024},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.111,"5min":1.202,"15min":1.258},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.964},"last":1.2210000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.3129999999999999},"min":{"1min":0.72199999999999998,"5min":0.62,"15min":0.62},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.357,"15min":1.353},"min":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.016,"5min":2.048,"15min":2.323},"last":0.88900000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2849999999999999,"15min":1.3180000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.579,"5min":1.78,"15min":2.2799999999999998},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.90300000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.383}]},{"osd":7,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.448},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3440000000000001,"15min":1.381},"min":{"1min":0.80600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":2.1080000000000001},"last":1.4019999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575049,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59164,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312676,"statfs":{"total":96636764160,"available":96576180224,"internally_reserved":0,"allocated":1921024,"data_stored":1219865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.83499999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.318},"max":{"1min":1.1639999999999999,"5min":1.341,"15min":1.5049999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.876,"5min":0.94999999999999996,"15min":1.0069999999999999},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.597,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85299999999999998,"15min":0.88100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.54},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0349999999999999,"15min":1.081},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.44800000000000001},"max":{"1min":1.472,"5min":1.7250000000000001,"15min":1.804},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.5},"max":{"1min":1.718,"5min":1.827,"15min":1.827},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1,"15min":1.038},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.39700000000000002},"max":{"1min":1.381,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.626}]},{"osd":4,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1579999999999999,"15min":1.173},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.65300000000000002},"max":{"1min":1.425,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.288,"15min":1.3240000000000001},"min":{"1min":0.85599999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.863},"last":1.131}]},{"osd":5,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":0.77700000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.244,"15min":1.27},"min":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":2.1619999999999999},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.294,"15min":1.3220000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.68400000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.861},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.306,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.742,"5min":1.7490000000000001,"15min":1.8839999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3720000000000001,"15min":1.417},"min":{"1min":0.91200000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.782,"5min":1.782,"15min":2.0609999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3859999999999999,"15min":1.399},"min":{"1min":0.80900000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.931,"5min":1.992,"15min":1.992},"last":0.81799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575051,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84099999999999997,"15min":0.85399999999999998},"min":{"1min":0.435,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.194,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89900000000000002,"15min":0.96499999999999997},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3859999999999999,"5min":1.544,"15min":1.6910000000000001},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.94799999999999995,"15min":0.998},"min":{"1min":0.51500000000000001,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.431,"5min":2.157,"15min":2.157},"last":0.874},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0089999999999999,"15min":1.0649999999999999},"min":{"1min":0.57399999999999995,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.3400000000000001,"5min":1.728,"15min":1.728},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":1.0149999999999999},"min":{"1min":0.623,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.954},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1080000000000001,"15min":1.1539999999999999},"min":{"1min":0.54200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4199999999999999,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.246}]},{"osd":4,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2589999999999999,"15min":1.3029999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5840000000000001,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1899999999999999,"15min":1.258},"min":{"1min":0.90200000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.55,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.323,"15min":1.343},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.71899999999999997},"max":{"1min":1.841,"5min":2.351,"15min":2.351},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2769999999999999,"15min":1.333},"min":{"1min":0.81599999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.546,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.7509999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2789999999999999,"15min":1.319},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7529999999999999,"5min":1.8680000000000001,"15min":2.157},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.351,"15min":1.3839999999999999},"min":{"1min":0.94499999999999995,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.7809999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.5940000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.339,"15min":1.3979999999999999},"min":{"1min":0.94999999999999996,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.5700000000000001,"5min":1.798,"15min":2.4279999999999999},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.3320000000000001,"15min":1.3899999999999999},"min":{"1min":0.98999999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.667,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":0.995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:52:53.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:52:53.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.704+0000 7fbe6ba66640 1 Processor -- start 2023-12-01T01:52:53.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.704+0000 7fbe6ba66640 1 -- start start 2023-12-01T01:52:53.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.705+0000 7fbe6ba66640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbe6415e2f0 0x7fbe640ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.705+0000 7fbe6ba66640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe6406f2f0 0x7fbe640caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.706+0000 7fbe6ba66640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe640add10 con 0x7fbe640adf10 2023-12-01T01:52:53.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.706+0000 7fbe6ba66640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe640c5760 con 0x7fbe6415e2f0 2023-12-01T01:52:53.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.706+0000 7fbe6ba66640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6415e6d0 con 0x7fbe6406f2f0 2023-12-01T01:52:53.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.706+0000 7fbe6a003640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fbe640adf10 0x7fbe640c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:33160/0 (socket says 172.21.15.115:33160) 2023-12-01T01:52:53.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.706+0000 7fbe6a003640 1 -- 172.21.15.115:0/2227394426 learned_addr learned my addr 172.21.15.115:0/2227394426 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:53.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.706+0000 7fbe69001640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe6406f2f0 0x7fbe640caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.706+0000 7fbe69802640 1 --2- 172.21.15.115:0/2227394426 >> v2:172.21.15.142:3300/0 conn(0x7fbe6415e2f0 0x7fbe640ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920206027 0 0) 0x7fbe640add10 con 0x7fbe640adf10 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe50003520 con 0x7fbe640adf10 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1913605632 0 0) 0x7fbe50003520 con 0x7fbe640adf10 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe6406f2f0 msgr2=0x7fbe640caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe6406f2f0 0x7fbe640caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 >> v2:172.21.15.142:3300/0 conn(0x7fbe6415e2f0 msgr2=0x7fbe640ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 --2- 172.21.15.115:0/2227394426 >> v2:172.21.15.142:3300/0 conn(0x7fbe6415e2f0 0x7fbe640ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe640cf7b0 con 0x7fbe640adf10 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fbe5c0019f0 con 0x7fbe640adf10 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 >> v1:172.21.15.115:6789/0 conn(0x7fbe640adf10 legacy=0x7fbe640c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe500039e0 0x7fbe50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe6406f2f0 0x7fbe500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe50004d00 0x7fbe50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640c5760 con 0x7fbe50004d00 2023-12-01T01:52:53.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6415e6d0 con 0x7fbe500039e0 2023-12-01T01:52:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5000ba80 con 0x7fbe6406f2f0 2023-12-01T01:52:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.707+0000 7fbe6a003640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe50004d00 0x7fbe50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.708+0000 7fbe69802640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe500039e0 0x7fbe50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.708+0000 7fbe69001640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe6406f2f0 0x7fbe500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.708+0000 7fbe6a003640 1 -- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe6406f2f0 msgr2=0x7fbe500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.708+0000 7fbe6a003640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe6406f2f0 0x7fbe500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.708+0000 7fbe6a003640 1 -- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe500039e0 msgr2=0x7fbe50003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.708+0000 7fbe6a003640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe500039e0 0x7fbe50003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.708+0000 7fbe6a003640 1 -- 172.21.15.115:0/2227394426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe50003520 con 0x7fbe50004d00 2023-12-01T01:52:53.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.709+0000 7fbe6a003640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe50004d00 0x7fbe50005180 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00a860 tx=0x7fbe500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae731e9f56ac2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:53.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.709+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c034630 con 0x7fbe50004d00 2023-12-01T01:52:53.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.709+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c02f070 con 0x7fbe50004d00 2023-12-01T01:52:53.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.709+0000 7fbe68800640 1 -- 172.21.15.115:0/2227394426 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0349f0 con 0x7fbe50004d00 2023-12-01T01:52:53.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.710+0000 7fbe6ba66640 1 -- 172.21.15.115:0/2227394426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe50004d00 msgr2=0x7fbe50005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.710+0000 7fbe6ba66640 1 --2- 172.21.15.115:0/2227394426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe50004d00 0x7fbe50005180 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00a860 tx=0x7fbe500057f0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.710+0000 7fbe6a003640 1 -- 172.21.15.115:0/2227394426 reap_dead start 2023-12-01T01:52:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.710+0000 7fbe6ba66640 1 -- 172.21.15.115:0/2227394426 shutdown_connections 2023-12-01T01:52:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.710+0000 7fbe6ba66640 1 -- 172.21.15.115:0/2227394426 >> 172.21.15.115:0/2227394426 conn(0x7fbe640ce880 msgr2=0x7fbe640cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.710+0000 7fbe6ba66640 1 -- 172.21.15.115:0/2227394426 shutdown_connections 2023-12-01T01:52:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.710+0000 7fbe6ba66640 1 -- 172.21.15.115:0/2227394426 wait complete. 2023-12-01T01:52:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.712+0000 7fbe6ba66640 1 Processor -- start 2023-12-01T01:52:53.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.712+0000 7fbe6ba66640 1 -- start start 2023-12-01T01:52:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.713+0000 7fbe6ba66640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe50004d00 0x7fbe640c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.713+0000 7fbe6ba66640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe64158cb0 0x7fbe64155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.713+0000 7fbe6ba66640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe6415ada0 0x7fbe6415b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.713+0000 7fbe6ba66640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe641555b0 con 0x7fbe6415ada0 2023-12-01T01:52:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.713+0000 7fbe6ba66640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6415b760 con 0x7fbe50004d00 2023-12-01T01:52:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.713+0000 7fbe6ba66640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640c3cf0 con 0x7fbe64158cb0 2023-12-01T01:52:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.713+0000 7fbe6a003640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe6415ada0 0x7fbe6415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.713+0000 7fbe6a003640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe6415ada0 0x7fbe6415b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43874/0 (socket says 172.21.15.115:43874) 2023-12-01T01:52:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.713+0000 7fbe6a003640 1 -- 172.21.15.115:0/1291695423 learned_addr learned my addr 172.21.15.115:0/1291695423 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:53.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe69001640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe64158cb0 0x7fbe64155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe6a003640 1 -- 172.21.15.115:0/1291695423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe64158cb0 msgr2=0x7fbe64155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe6a003640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe64158cb0 0x7fbe64155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe6a003640 1 -- 172.21.15.115:0/1291695423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe50004d00 msgr2=0x7fbe640c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe6a003640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe50004d00 0x7fbe640c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe6a003640 1 -- 172.21.15.115:0/1291695423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe640c3450 con 0x7fbe6415ada0 2023-12-01T01:52:53.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe69001640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe64158cb0 0x7fbe64155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:53.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe6a003640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe6415ada0 0x7fbe6415b220 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00a860 tx=0x7fbe5c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:53.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe5a7fc640 1 -- 172.21.15.115:0/1291695423 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c034840 con 0x7fbe6415ada0 2023-12-01T01:52:53.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe5a7fc640 1 -- 172.21.15.115:0/1291695423 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c0349a0 con 0x7fbe6415ada0 2023-12-01T01:52:53.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe6ba66640 1 -- 172.21.15.115:0/1291695423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe640c3730 con 0x7fbe6415ada0 2023-12-01T01:52:53.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.714+0000 7fbe5a7fc640 1 -- 172.21.15.115:0/1291695423 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c034d30 con 0x7fbe6415ada0 2023-12-01T01:52:53.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.715+0000 7fbe6ba66640 1 -- 172.21.15.115:0/1291695423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe640ca3a0 con 0x7fbe6415ada0 2023-12-01T01:52:53.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.716+0000 7fbe6ba66640 1 -- 172.21.15.115:0/1291695423 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6406fda0 con 0x7fbe6415ada0 2023-12-01T01:52:53.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.716+0000 7fbe5a7fc640 1 -- 172.21.15.115:0/1291695423 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fbe5c031030 con 0x7fbe6415ada0 2023-12-01T01:52:53.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.717+0000 7fbe5a7fc640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe4806a580 0x7fbe4806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:53.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.717+0000 7fbe5a7fc640 1 -- 172.21.15.115:0/1291695423 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe5c0aa460 con 0x7fbe6415ada0 2023-12-01T01:52:53.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.718+0000 7fbe69802640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe4806a580 0x7fbe4806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:53.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.718+0000 7fbe69802640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe4806a580 0x7fbe4806ca40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009990 tx=0x7fbe60008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:53.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.722+0000 7fbe5a7fc640 1 -- 172.21.15.115:0/1291695423 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbe5c075790 con 0x7fbe6415ada0 2023-12-01T01:52:53.903 INFO:tasks.ceph.mds.d.smithi142.stderr:2023-12-01T01:52:53.901+0000 7fdfe6e5d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:52:53.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.907+0000 7fbe6ba66640 1 -- 172.21.15.115:0/1291695423 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe640ca220 con 0x7fbe4806a580 2023-12-01T01:52:53.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.917+0000 7fbe5a7fc640 1 -- 172.21.15.115:0/1291695423 <== mgr.7230 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284309 (secure 0 0 0) 0x7fbe60011860 con 0x7fbe4806a580 2023-12-01T01:52:53.920 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:53.924 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:52:53.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.924+0000 7fbe6ba66640 1 -- 172.21.15.115:0/1291695423 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe4806a580 msgr2=0x7fbe4806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.924+0000 7fbe6ba66640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe4806a580 0x7fbe4806ca40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009990 tx=0x7fbe60008040 comp rx=0 tx=0).stop 2023-12-01T01:52:53.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.924+0000 7fbe6ba66640 1 -- 172.21.15.115:0/1291695423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe6415ada0 msgr2=0x7fbe6415b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:53.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.924+0000 7fbe6ba66640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe6415ada0 0x7fbe6415b220 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00a860 tx=0x7fbe5c001d80 comp rx=0 tx=0).stop 2023-12-01T01:52:53.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.924+0000 7fbe6ba66640 1 -- 172.21.15.115:0/1291695423 shutdown_connections 2023-12-01T01:52:53.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.924+0000 7fbe6ba66640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbe4806a580 0x7fbe4806ca40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.924+0000 7fbe6ba66640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbe64158cb0 0x7fbe64155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.924+0000 7fbe6ba66640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbe6415ada0 0x7fbe6415b220 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.924+0000 7fbe6ba66640 1 --2- 172.21.15.115:0/1291695423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe50004d00 0x7fbe640c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:53.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.924+0000 7fbe6ba66640 1 -- 172.21.15.115:0/1291695423 >> 172.21.15.115:0/1291695423 conn(0x7fbe640ce880 msgr2=0x7fbe640af770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:53.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.925+0000 7fbe6ba66640 1 -- 172.21.15.115:0/1291695423 shutdown_connections 2023-12-01T01:52:53.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:53.925+0000 7fbe6ba66640 1 -- 172.21.15.115:0/1291695423 wait complete. 2023-12-01T01:52:53.945 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T01:52:52.106719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316488,"kb_used_data":11528,"kb_used_omap":0,"kb_used_meta":304896,"kb_avail":754658232,"statfs":{"total":773094113280,"available":772770029568,"internally_reserved":0,"allocated":11804672,"data_stored":6216712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312213504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.102080"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.031286+0000","last_change":"2023-12-01T01:52:48.031286+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.031286+0000","last_undegraded":"2023-12-01T01:52:48.031286+0000","last_fullsized":"2023-12-01T01:52:48.031286+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:07:01.922435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.022478+0000","last_change":"2023-12-01T01:52:48.022478+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.022478+0000","last_undegraded":"2023-12-01T01:52:48.022478+0000","last_fullsized":"2023-12-01T01:52:48.022478+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:20.199417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.034568+0000","last_change":"2023-12-01T01:52:48.034568+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.034568+0000","last_undegraded":"2023-12-01T01:52:48.034568+0000","last_fullsized":"2023-12-01T01:52:48.034568+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029059+0000","last_change":"2023-12-01T01:52:48.029059+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029059+0000","last_undegraded":"2023-12-01T01:52:48.029059+0000","last_fullsized":"2023-12-01T01:52:48.029059+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:06:59.600929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.022456+0000","last_change":"2023-12-01T01:52:48.022456+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.022456+0000","last_undegraded":"2023-12-01T01:52:48.022456+0000","last_fullsized":"2023-12-01T01:52:48.022456+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:48:34.887669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.033630+0000","last_change":"2023-12-01T01:52:48.033630+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.033630+0000","last_undegraded":"2023-12-01T01:52:48.033630+0000","last_fullsized":"2023-12-01T01:52:48.033630+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:39:18.420163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020737+0000","last_change":"2023-12-01T01:52:48.020737+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020737+0000","last_undegraded":"2023-12-01T01:52:48.020737+0000","last_fullsized":"2023-12-01T01:52:48.020737+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:51.221046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024562+0000","last_change":"2023-12-01T01:52:48.024562+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024562+0000","last_undegraded":"2023-12-01T01:52:48.024562+0000","last_fullsized":"2023-12-01T01:52:48.024562+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:21.099958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026842+0000","last_change":"2023-12-01T01:52:48.026842+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026842+0000","last_undegraded":"2023-12-01T01:52:48.026842+0000","last_fullsized":"2023-12-01T01:52:48.026842+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:52.282595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.034705+0000","last_change":"2023-12-01T01:52:48.034705+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.034705+0000","last_undegraded":"2023-12-01T01:52:48.034705+0000","last_fullsized":"2023-12-01T01:52:48.034705+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029077+0000","last_change":"2023-12-01T01:52:48.029077+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029077+0000","last_undegraded":"2023-12-01T01:52:48.029077+0000","last_fullsized":"2023-12-01T01:52:48.029077+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025888+0000","last_change":"2023-12-01T01:52:48.025888+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025888+0000","last_undegraded":"2023-12-01T01:52:48.025888+0000","last_fullsized":"2023-12-01T01:52:48.025888+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021972+0000","last_change":"2023-12-01T01:52:48.021972+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021972+0000","last_undegraded":"2023-12-01T01:52:48.021972+0000","last_fullsized":"2023-12-01T01:52:48.021972+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:45:17.044474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.037434+0000","last_change":"2023-12-01T01:52:48.037434+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.037434+0000","last_undegraded":"2023-12-01T01:52:48.037434+0000","last_fullsized":"2023-12-01T01:52:48.037434+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:40:59.300637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.019592+0000","last_change":"2023-12-01T01:52:48.019592+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.019592+0000","last_undegraded":"2023-12-01T01:52:48.019592+0000","last_fullsized":"2023-12-01T01:52:48.019592+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:58:23.518943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029503+0000","last_change":"2023-12-01T01:52:48.029503+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029503+0000","last_undegraded":"2023-12-01T01:52:48.029503+0000","last_fullsized":"2023-12-01T01:52:48.029503+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024646+0000","last_change":"2023-12-01T01:52:48.024646+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024646+0000","last_undegraded":"2023-12-01T01:52:48.024646+0000","last_fullsized":"2023-12-01T01:52:48.024646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.029391+0000","last_change":"2023-12-01T01:52:48.029391+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.029391+0000","last_undegraded":"2023-12-01T01:52:48.029391+0000","last_fullsized":"2023-12-01T01:52:48.029391+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.038499+0000","last_change":"2023-12-01T01:52:48.038499+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.038499+0000","last_undegraded":"2023-12-01T01:52:48.038499+0000","last_fullsized":"2023-12-01T01:52:48.038499+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:13:33.646688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035539+0000","last_change":"2023-12-01T01:52:48.035539+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035539+0000","last_undegraded":"2023-12-01T01:52:48.035539+0000","last_fullsized":"2023-12-01T01:52:48.035539+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:42.942148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026846+0000","last_change":"2023-12-01T01:52:48.026846+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026846+0000","last_undegraded":"2023-12-01T01:52:48.026846+0000","last_fullsized":"2023-12-01T01:52:48.026846+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.036892+0000","last_change":"2023-12-01T01:52:48.036892+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.036892+0000","last_undegraded":"2023-12-01T01:52:48.036892+0000","last_fullsized":"2023-12-01T01:52:48.036892+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027150+0000","last_change":"2023-12-01T01:52:48.027150+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027150+0000","last_undegraded":"2023-12-01T01:52:48.027150+0000","last_fullsized":"2023-12-01T01:52:48.027150+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.030146+0000","last_change":"2023-12-01T01:52:48.030146+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.030146+0000","last_undegraded":"2023-12-01T01:52:48.030146+0000","last_fullsized":"2023-12-01T01:52:48.030146+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.031458+0000","last_change":"2023-12-01T01:52:48.031458+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.031458+0000","last_undegraded":"2023-12-01T01:52:48.031458+0000","last_fullsized":"2023-12-01T01:52:48.031458+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023595+0000","last_change":"2023-12-01T01:52:48.023595+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023595+0000","last_undegraded":"2023-12-01T01:52:48.023595+0000","last_fullsized":"2023-12-01T01:52:48.023595+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:22.154067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027885+0000","last_change":"2023-12-01T01:52:48.027885+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027885+0000","last_undegraded":"2023-12-01T01:52:48.027885+0000","last_fullsized":"2023-12-01T01:52:48.027885+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:12:02.919659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021893+0000","last_change":"2023-12-01T01:52:48.021893+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021893+0000","last_undegraded":"2023-12-01T01:52:48.021893+0000","last_fullsized":"2023-12-01T01:52:48.021893+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:25:02.674823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020506+0000","last_change":"2023-12-01T01:52:48.020506+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020506+0000","last_undegraded":"2023-12-01T01:52:48.020506+0000","last_fullsized":"2023-12-01T01:52:48.020506+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035671+0000","last_change":"2023-12-01T01:52:48.035671+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035671+0000","last_undegraded":"2023-12-01T01:52:48.035671+0000","last_fullsized":"2023-12-01T01:52:48.035671+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:00.516734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.034799+0000","last_change":"2023-12-01T01:52:48.034799+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.034799+0000","last_undegraded":"2023-12-01T01:52:48.034799+0000","last_fullsized":"2023-12-01T01:52:48.034799+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.028380+0000","last_change":"2023-12-01T01:52:48.028380+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.028380+0000","last_undegraded":"2023-12-01T01:52:48.028380+0000","last_fullsized":"2023-12-01T01:52:48.028380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026751+0000","last_change":"2023-12-01T01:52:48.026751+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026751+0000","last_undegraded":"2023-12-01T01:52:48.026751+0000","last_fullsized":"2023-12-01T01:52:48.026751+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:02:42.893820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041182+0000","last_change":"2023-12-01T01:52:48.038820+0000","last_active":"2023-12-01T01:52:48.041182+0000","last_peered":"2023-12-01T01:52:48.041182+0000","last_clean":"2023-12-01T01:52:48.041182+0000","last_became_active":"2023-12-01T01:52:48.038539+0000","last_became_peered":"2023-12-01T01:52:48.038539+0000","last_unstale":"2023-12-01T01:52:48.041182+0000","last_undegraded":"2023-12-01T01:52:48.041182+0000","last_fullsized":"2023-12-01T01:52:48.041182+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:21.090681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026112+0000","last_change":"2023-12-01T01:52:48.026112+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026112+0000","last_undegraded":"2023-12-01T01:52:48.026112+0000","last_fullsized":"2023-12-01T01:52:48.026112+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:01.801261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.050678+0000","last_change":"2023-12-01T01:52:48.049643+0000","last_active":"2023-12-01T01:52:48.050678+0000","last_peered":"2023-12-01T01:52:48.050678+0000","last_clean":"2023-12-01T01:52:48.050678+0000","last_became_active":"2023-12-01T01:52:48.049288+0000","last_became_peered":"2023-12-01T01:52:48.049288+0000","last_unstale":"2023-12-01T01:52:48.050678+0000","last_undegraded":"2023-12-01T01:52:48.050678+0000","last_fullsized":"2023-12-01T01:52:48.050678+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:41.937860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024925+0000","last_change":"2023-12-01T01:52:48.024925+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024925+0000","last_undegraded":"2023-12-01T01:52:48.024925+0000","last_fullsized":"2023-12-01T01:52:48.024925+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:50:17.533885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044029+0000","last_change":"2023-12-01T01:52:48.039238+0000","last_active":"2023-12-01T01:52:48.044029+0000","last_peered":"2023-12-01T01:52:48.044029+0000","last_clean":"2023-12-01T01:52:48.044029+0000","last_became_active":"2023-12-01T01:52:48.038835+0000","last_became_peered":"2023-12-01T01:52:48.038835+0000","last_unstale":"2023-12-01T01:52:48.044029+0000","last_undegraded":"2023-12-01T01:52:48.044029+0000","last_fullsized":"2023-12-01T01:52:48.044029+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023506+0000","last_change":"2023-12-01T01:52:48.023506+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023506+0000","last_undegraded":"2023-12-01T01:52:48.023506+0000","last_fullsized":"2023-12-01T01:52:48.023506+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.048403+0000","last_change":"2023-12-01T01:52:48.045716+0000","last_active":"2023-12-01T01:52:48.048403+0000","last_peered":"2023-12-01T01:52:48.048403+0000","last_clean":"2023-12-01T01:52:48.048403+0000","last_became_active":"2023-12-01T01:52:48.045176+0000","last_became_peered":"2023-12-01T01:52:48.045176+0000","last_unstale":"2023-12-01T01:52:48.048403+0000","last_undegraded":"2023-12-01T01:52:48.048403+0000","last_fullsized":"2023-12-01T01:52:48.048403+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:55:21.695057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026841+0000","last_change":"2023-12-01T01:52:48.026841+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026841+0000","last_undegraded":"2023-12-01T01:52:48.026841+0000","last_fullsized":"2023-12-01T01:52:48.026841+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.036263+0000","last_change":"2023-12-01T01:52:48.033877+0000","last_active":"2023-12-01T01:52:48.036263+0000","last_peered":"2023-12-01T01:52:48.036263+0000","last_clean":"2023-12-01T01:52:48.036263+0000","last_became_active":"2023-12-01T01:52:48.033605+0000","last_became_peered":"2023-12-01T01:52:48.033605+0000","last_unstale":"2023-12-01T01:52:48.036263+0000","last_undegraded":"2023-12-01T01:52:48.036263+0000","last_fullsized":"2023-12-01T01:52:48.036263+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:55.538925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.032197+0000","last_change":"2023-12-01T01:52:48.032197+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.032197+0000","last_undegraded":"2023-12-01T01:52:48.032197+0000","last_fullsized":"2023-12-01T01:52:48.032197+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:53:05.715094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.046497+0000","last_change":"2023-12-01T01:52:48.043431+0000","last_active":"2023-12-01T01:52:48.046497+0000","last_peered":"2023-12-01T01:52:48.046497+0000","last_clean":"2023-12-01T01:52:48.046497+0000","last_became_active":"2023-12-01T01:52:48.043162+0000","last_became_peered":"2023-12-01T01:52:48.043162+0000","last_unstale":"2023-12-01T01:52:48.046497+0000","last_undegraded":"2023-12-01T01:52:48.046497+0000","last_fullsized":"2023-12-01T01:52:48.046497+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.022665+0000","last_change":"2023-12-01T01:52:48.022665+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.022665+0000","last_undegraded":"2023-12-01T01:52:48.022665+0000","last_fullsized":"2023-12-01T01:52:48.022665+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:10:51.868019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049031+0000","last_change":"2023-12-01T01:52:48.046040+0000","last_active":"2023-12-01T01:52:48.049031+0000","last_peered":"2023-12-01T01:52:48.049031+0000","last_clean":"2023-12-01T01:52:48.049031+0000","last_became_active":"2023-12-01T01:52:48.045722+0000","last_became_peered":"2023-12-01T01:52:48.045722+0000","last_unstale":"2023-12-01T01:52:48.049031+0000","last_undegraded":"2023-12-01T01:52:48.049031+0000","last_fullsized":"2023-12-01T01:52:48.049031+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:25:01.670535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035505+0000","last_change":"2023-12-01T01:52:48.035505+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035505+0000","last_undegraded":"2023-12-01T01:52:48.035505+0000","last_fullsized":"2023-12-01T01:52:48.035505+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:38.690885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.051042+0000","last_change":"2023-12-01T01:52:48.049260+0000","last_active":"2023-12-01T01:52:48.051042+0000","last_peered":"2023-12-01T01:52:48.051042+0000","last_clean":"2023-12-01T01:52:48.051042+0000","last_became_active":"2023-12-01T01:52:48.048998+0000","last_became_peered":"2023-12-01T01:52:48.048998+0000","last_unstale":"2023-12-01T01:52:48.051042+0000","last_undegraded":"2023-12-01T01:52:48.051042+0000","last_fullsized":"2023-12-01T01:52:48.051042+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:06.546333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.035689+0000","last_change":"2023-12-01T01:52:48.035689+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.035689+0000","last_undegraded":"2023-12-01T01:52:48.035689+0000","last_fullsized":"2023-12-01T01:52:48.035689+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:12.516160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.037274+0000","last_change":"2023-12-01T01:52:48.035560+0000","last_active":"2023-12-01T01:52:48.037274+0000","last_peered":"2023-12-01T01:52:48.037274+0000","last_clean":"2023-12-01T01:52:48.037274+0000","last_became_active":"2023-12-01T01:52:48.035303+0000","last_became_peered":"2023-12-01T01:52:48.035303+0000","last_unstale":"2023-12-01T01:52:48.037274+0000","last_undegraded":"2023-12-01T01:52:48.037274+0000","last_fullsized":"2023-12-01T01:52:48.037274+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:04.573692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025380+0000","last_change":"2023-12-01T01:52:48.025380+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025380+0000","last_undegraded":"2023-12-01T01:52:48.025380+0000","last_fullsized":"2023-12-01T01:52:48.025380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:22.643458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041963+0000","last_change":"2023-12-01T01:52:48.040913+0000","last_active":"2023-12-01T01:52:48.041963+0000","last_peered":"2023-12-01T01:52:48.041963+0000","last_clean":"2023-12-01T01:52:48.041963+0000","last_became_active":"2023-12-01T01:52:48.040402+0000","last_became_peered":"2023-12-01T01:52:48.040402+0000","last_unstale":"2023-12-01T01:52:48.041963+0000","last_undegraded":"2023-12-01T01:52:48.041963+0000","last_fullsized":"2023-12-01T01:52:48.041963+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027539+0000","last_change":"2023-12-01T01:52:48.027539+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027539+0000","last_undegraded":"2023-12-01T01:52:48.027539+0000","last_fullsized":"2023-12-01T01:52:48.027539+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.047008+0000","last_change":"2023-12-01T01:52:48.045583+0000","last_active":"2023-12-01T01:52:48.047008+0000","last_peered":"2023-12-01T01:52:48.047008+0000","last_clean":"2023-12-01T01:52:48.047008+0000","last_became_active":"2023-12-01T01:52:48.045146+0000","last_became_peered":"2023-12-01T01:52:48.045146+0000","last_unstale":"2023-12-01T01:52:48.047008+0000","last_undegraded":"2023-12-01T01:52:48.047008+0000","last_fullsized":"2023-12-01T01:52:48.047008+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:25.868578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.028382+0000","last_change":"2023-12-01T01:52:48.028382+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.028382+0000","last_undegraded":"2023-12-01T01:52:48.028382+0000","last_fullsized":"2023-12-01T01:52:48.028382+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:03.971637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044477+0000","last_change":"2023-12-01T01:52:48.042396+0000","last_active":"2023-12-01T01:52:48.044477+0000","last_peered":"2023-12-01T01:52:48.044477+0000","last_clean":"2023-12-01T01:52:48.044477+0000","last_became_active":"2023-12-01T01:52:48.041958+0000","last_became_peered":"2023-12-01T01:52:48.041958+0000","last_unstale":"2023-12-01T01:52:48.044477+0000","last_undegraded":"2023-12-01T01:52:48.044477+0000","last_fullsized":"2023-12-01T01:52:48.044477+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:33.265613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.024486+0000","last_change":"2023-12-01T01:52:48.024486+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.024486+0000","last_undegraded":"2023-12-01T01:52:48.024486+0000","last_fullsized":"2023-12-01T01:52:48.024486+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.039978+0000","last_change":"2023-12-01T01:52:48.035637+0000","last_active":"2023-12-01T01:52:48.039978+0000","last_peered":"2023-12-01T01:52:48.039978+0000","last_clean":"2023-12-01T01:52:48.039978+0000","last_became_active":"2023-12-01T01:52:48.034955+0000","last_became_peered":"2023-12-01T01:52:48.034955+0000","last_unstale":"2023-12-01T01:52:48.039978+0000","last_undegraded":"2023-12-01T01:52:48.039978+0000","last_fullsized":"2023-12-01T01:52:48.039978+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:02.433217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025388+0000","last_change":"2023-12-01T01:52:48.025388+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025388+0000","last_undegraded":"2023-12-01T01:52:48.025388+0000","last_fullsized":"2023-12-01T01:52:48.025388+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:58:00.440400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.037568+0000","last_change":"2023-12-01T01:52:48.033331+0000","last_active":"2023-12-01T01:52:48.037568+0000","last_peered":"2023-12-01T01:52:48.037568+0000","last_clean":"2023-12-01T01:52:48.037568+0000","last_became_active":"2023-12-01T01:52:48.032852+0000","last_became_peered":"2023-12-01T01:52:48.032852+0000","last_unstale":"2023-12-01T01:52:48.037568+0000","last_undegraded":"2023-12-01T01:52:48.037568+0000","last_fullsized":"2023-12-01T01:52:48.037568+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:37.686598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.036643+0000","last_change":"2023-12-01T01:52:48.036643+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.036643+0000","last_undegraded":"2023-12-01T01:52:48.036643+0000","last_fullsized":"2023-12-01T01:52:48.036643+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:45.833443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.042163+0000","last_change":"2023-12-01T01:52:48.041648+0000","last_active":"2023-12-01T01:52:48.042163+0000","last_peered":"2023-12-01T01:52:48.042163+0000","last_clean":"2023-12-01T01:52:48.042163+0000","last_became_active":"2023-12-01T01:52:48.040913+0000","last_became_peered":"2023-12-01T01:52:48.040913+0000","last_unstale":"2023-12-01T01:52:48.042163+0000","last_undegraded":"2023-12-01T01:52:48.042163+0000","last_fullsized":"2023-12-01T01:52:48.042163+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:37.686598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023509+0000","last_change":"2023-12-01T01:52:48.023509+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023509+0000","last_undegraded":"2023-12-01T01:52:48.023509+0000","last_fullsized":"2023-12-01T01:52:48.023509+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:56.543213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.034765+0000","last_change":"2023-12-01T01:52:48.032822+0000","last_active":"2023-12-01T01:52:48.034765+0000","last_peered":"2023-12-01T01:52:48.034765+0000","last_clean":"2023-12-01T01:52:48.034765+0000","last_became_active":"2023-12-01T01:52:48.032368+0000","last_became_peered":"2023-12-01T01:52:48.032368+0000","last_unstale":"2023-12-01T01:52:48.034765+0000","last_undegraded":"2023-12-01T01:52:48.034765+0000","last_fullsized":"2023-12-01T01:52:48.034765+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:10:50.863731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026337+0000","last_change":"2023-12-01T01:52:48.026337+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026337+0000","last_undegraded":"2023-12-01T01:52:48.026337+0000","last_fullsized":"2023-12-01T01:52:48.026337+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:09:36.618385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041816+0000","last_change":"2023-12-01T01:52:48.040258+0000","last_active":"2023-12-01T01:52:48.041816+0000","last_peered":"2023-12-01T01:52:48.041816+0000","last_clean":"2023-12-01T01:52:48.041816+0000","last_became_active":"2023-12-01T01:52:48.037376+0000","last_became_peered":"2023-12-01T01:52:48.037376+0000","last_unstale":"2023-12-01T01:52:48.041816+0000","last_undegraded":"2023-12-01T01:52:48.041816+0000","last_fullsized":"2023-12-01T01:52:48.041816+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:13.111537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.033821+0000","last_change":"2023-12-01T01:52:48.033821+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.033821+0000","last_undegraded":"2023-12-01T01:52:48.033821+0000","last_fullsized":"2023-12-01T01:52:48.033821+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.051066+0000","last_change":"2023-12-01T01:52:48.050044+0000","last_active":"2023-12-01T01:52:48.051066+0000","last_peered":"2023-12-01T01:52:48.051066+0000","last_clean":"2023-12-01T01:52:48.051066+0000","last_became_active":"2023-12-01T01:52:48.049779+0000","last_became_peered":"2023-12-01T01:52:48.049779+0000","last_unstale":"2023-12-01T01:52:48.051066+0000","last_undegraded":"2023-12-01T01:52:48.051066+0000","last_fullsized":"2023-12-01T01:52:48.051066+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:20:19.462886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026154+0000","last_change":"2023-12-01T01:52:48.026154+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026154+0000","last_undegraded":"2023-12-01T01:52:48.026154+0000","last_fullsized":"2023-12-01T01:52:48.026154+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:17.235352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.050226+0000","last_change":"2023-12-01T01:52:48.048859+0000","last_active":"2023-12-01T01:52:48.050226+0000","last_peered":"2023-12-01T01:52:48.050226+0000","last_clean":"2023-12-01T01:52:48.050226+0000","last_became_active":"2023-12-01T01:52:48.048546+0000","last_became_peered":"2023-12-01T01:52:48.048546+0000","last_unstale":"2023-12-01T01:52:48.050226+0000","last_undegraded":"2023-12-01T01:52:48.050226+0000","last_fullsized":"2023-12-01T01:52:48.050226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:43.307966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021314+0000","last_change":"2023-12-01T01:52:48.021314+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021314+0000","last_undegraded":"2023-12-01T01:52:48.021314+0000","last_fullsized":"2023-12-01T01:52:48.021314+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049328+0000","last_change":"2023-12-01T01:52:48.046685+0000","last_active":"2023-12-01T01:52:48.049328+0000","last_peered":"2023-12-01T01:52:48.049328+0000","last_clean":"2023-12-01T01:52:48.049328+0000","last_became_active":"2023-12-01T01:52:48.046395+0000","last_became_peered":"2023-12-01T01:52:48.046395+0000","last_unstale":"2023-12-01T01:52:48.049328+0000","last_undegraded":"2023-12-01T01:52:48.049328+0000","last_fullsized":"2023-12-01T01:52:48.049328+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:29:10.130713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021393+0000","last_change":"2023-12-01T01:52:48.021393+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021393+0000","last_undegraded":"2023-12-01T01:52:48.021393+0000","last_fullsized":"2023-12-01T01:52:48.021393+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:52:45.545702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.041831+0000","last_change":"2023-12-01T01:52:48.038141+0000","last_active":"2023-12-01T01:52:48.041831+0000","last_peered":"2023-12-01T01:52:48.041831+0000","last_clean":"2023-12-01T01:52:48.041831+0000","last_became_active":"2023-12-01T01:52:48.037797+0000","last_became_peered":"2023-12-01T01:52:48.037797+0000","last_unstale":"2023-12-01T01:52:48.041831+0000","last_undegraded":"2023-12-01T01:52:48.041831+0000","last_fullsized":"2023-12-01T01:52:48.041831+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021667+0000","last_change":"2023-12-01T01:52:48.021667+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021667+0000","last_undegraded":"2023-12-01T01:52:48.021667+0000","last_fullsized":"2023-12-01T01:52:48.021667+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.043878+0000","last_change":"2023-12-01T01:52:48.042643+0000","last_active":"2023-12-01T01:52:48.043878+0000","last_peered":"2023-12-01T01:52:48.043878+0000","last_clean":"2023-12-01T01:52:48.043878+0000","last_became_active":"2023-12-01T01:52:48.042174+0000","last_became_peered":"2023-12-01T01:52:48.042174+0000","last_unstale":"2023-12-01T01:52:48.043878+0000","last_undegraded":"2023-12-01T01:52:48.043878+0000","last_fullsized":"2023-12-01T01:52:48.043878+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:59.436113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026638+0000","last_change":"2023-12-01T01:52:48.026638+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026638+0000","last_undegraded":"2023-12-01T01:52:48.026638+0000","last_fullsized":"2023-12-01T01:52:48.026638+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049368+0000","last_change":"2023-12-01T01:52:48.047235+0000","last_active":"2023-12-01T01:52:48.049368+0000","last_peered":"2023-12-01T01:52:48.049368+0000","last_clean":"2023-12-01T01:52:48.049368+0000","last_became_active":"2023-12-01T01:52:48.046953+0000","last_became_peered":"2023-12-01T01:52:48.046953+0000","last_unstale":"2023-12-01T01:52:48.049368+0000","last_undegraded":"2023-12-01T01:52:48.049368+0000","last_fullsized":"2023-12-01T01:52:48.049368+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:13.492174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.023160+0000","last_change":"2023-12-01T01:52:48.023160+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.023160+0000","last_undegraded":"2023-12-01T01:52:48.023160+0000","last_fullsized":"2023-12-01T01:52:48.023160+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:39.608737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.051849+0000","last_change":"2023-12-01T01:52:48.050705+0000","last_active":"2023-12-01T01:52:48.051849+0000","last_peered":"2023-12-01T01:52:48.051849+0000","last_clean":"2023-12-01T01:52:48.051849+0000","last_became_active":"2023-12-01T01:52:48.050445+0000","last_became_peered":"2023-12-01T01:52:48.050445+0000","last_unstale":"2023-12-01T01:52:48.051849+0000","last_undegraded":"2023-12-01T01:52:48.051849+0000","last_fullsized":"2023-12-01T01:52:48.051849+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:02:56.065815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.025759+0000","last_change":"2023-12-01T01:52:48.025759+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.025759+0000","last_undegraded":"2023-12-01T01:52:48.025759+0000","last_fullsized":"2023-12-01T01:52:48.025759+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:55:14.304687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.042923+0000","last_change":"2023-12-01T01:52:48.040799+0000","last_active":"2023-12-01T01:52:48.042923+0000","last_peered":"2023-12-01T01:52:48.042923+0000","last_clean":"2023-12-01T01:52:48.042923+0000","last_became_active":"2023-12-01T01:52:48.040471+0000","last_became_peered":"2023-12-01T01:52:48.040471+0000","last_unstale":"2023-12-01T01:52:48.042923+0000","last_undegraded":"2023-12-01T01:52:48.042923+0000","last_fullsized":"2023-12-01T01:52:48.042923+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.021428+0000","last_change":"2023-12-01T01:52:48.021428+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.021428+0000","last_undegraded":"2023-12-01T01:52:48.021428+0000","last_fullsized":"2023-12-01T01:52:48.021428+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:35:01.791105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044954+0000","last_change":"2023-12-01T01:52:48.043497+0000","last_active":"2023-12-01T01:52:48.044954+0000","last_peered":"2023-12-01T01:52:48.044954+0000","last_clean":"2023-12-01T01:52:48.044954+0000","last_became_active":"2023-12-01T01:52:48.043198+0000","last_became_peered":"2023-12-01T01:52:48.043198+0000","last_unstale":"2023-12-01T01:52:48.044954+0000","last_undegraded":"2023-12-01T01:52:48.044954+0000","last_fullsized":"2023-12-01T01:52:48.044954+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:44.930335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.033844+0000","last_change":"2023-12-01T01:52:48.033844+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.033844+0000","last_undegraded":"2023-12-01T01:52:48.033844+0000","last_fullsized":"2023-12-01T01:52:48.033844+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:03.437505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.049785+0000","last_change":"2023-12-01T01:52:48.046182+0000","last_active":"2023-12-01T01:52:48.049785+0000","last_peered":"2023-12-01T01:52:48.049785+0000","last_clean":"2023-12-01T01:52:48.049785+0000","last_became_active":"2023-12-01T01:52:48.045696+0000","last_became_peered":"2023-12-01T01:52:48.045696+0000","last_unstale":"2023-12-01T01:52:48.049785+0000","last_undegraded":"2023-12-01T01:52:48.049785+0000","last_fullsized":"2023-12-01T01:52:48.049785+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:44.829155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020256+0000","last_change":"2023-12-01T01:52:48.020256+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020256+0000","last_undegraded":"2023-12-01T01:52:48.020256+0000","last_fullsized":"2023-12-01T01:52:48.020256+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:34:49.188317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.044461+0000","last_change":"2023-12-01T01:52:48.042910+0000","last_active":"2023-12-01T01:52:48.044461+0000","last_peered":"2023-12-01T01:52:48.044461+0000","last_clean":"2023-12-01T01:52:48.044461+0000","last_became_active":"2023-12-01T01:52:48.041897+0000","last_became_peered":"2023-12-01T01:52:48.041897+0000","last_unstale":"2023-12-01T01:52:48.044461+0000","last_undegraded":"2023-12-01T01:52:48.044461+0000","last_fullsized":"2023-12-01T01:52:48.044461+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.027872+0000","last_change":"2023-12-01T01:52:48.027872+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.027872+0000","last_undegraded":"2023-12-01T01:52:48.027872+0000","last_fullsized":"2023-12-01T01:52:48.027872+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:45.934623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.047598+0000","last_change":"2023-12-01T01:52:48.044493+0000","last_active":"2023-12-01T01:52:48.047598+0000","last_peered":"2023-12-01T01:52:48.047598+0000","last_clean":"2023-12-01T01:52:48.047598+0000","last_became_active":"2023-12-01T01:52:48.044221+0000","last_became_peered":"2023-12-01T01:52:48.044221+0000","last_unstale":"2023-12-01T01:52:48.047598+0000","last_undegraded":"2023-12-01T01:52:48.047598+0000","last_fullsized":"2023-12-01T01:52:48.047598+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:59.436113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.019339+0000","last_change":"2023-12-01T01:52:48.019339+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.019339+0000","last_undegraded":"2023-12-01T01:52:48.019339+0000","last_fullsized":"2023-12-01T01:52:48.019339+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:20:23.997177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.046763+0000","last_change":"2023-12-01T01:52:48.045085+0000","last_active":"2023-12-01T01:52:48.046763+0000","last_peered":"2023-12-01T01:52:48.046763+0000","last_clean":"2023-12-01T01:52:48.046763+0000","last_became_active":"2023-12-01T01:52:48.044789+0000","last_became_peered":"2023-12-01T01:52:48.044789+0000","last_unstale":"2023-12-01T01:52:48.046763+0000","last_undegraded":"2023-12-01T01:52:48.046763+0000","last_fullsized":"2023-12-01T01:52:48.046763+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:58:00.466321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.020747+0000","last_change":"2023-12-01T01:52:48.020747+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.020747+0000","last_undegraded":"2023-12-01T01:52:48.020747+0000","last_fullsized":"2023-12-01T01:52:48.020747+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:02:57.070103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.039567+0000","last_change":"2023-12-01T01:52:48.034635+0000","last_active":"2023-12-01T01:52:48.039567+0000","last_peered":"2023-12-01T01:52:48.039567+0000","last_clean":"2023-12-01T01:52:48.039567+0000","last_became_active":"2023-12-01T01:52:48.034401+0000","last_became_peered":"2023-12-01T01:52:48.034401+0000","last_unstale":"2023-12-01T01:52:48.039567+0000","last_undegraded":"2023-12-01T01:52:48.039567+0000","last_fullsized":"2023-12-01T01:52:48.039567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:19.195130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2023-12-01T01:52:48.026074+0000","last_change":"2023-12-01T01:52:48.026074+0000","last_active":"2023-12-01T01:52:48.003394+0000","last_peered":"2023-12-01T01:52:48.003394+0000","last_clean":"2023-12-01T01:52:48.003394+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:52:48.026074+0000","last_undegraded":"2023-12-01T01:52:48.026074+0000","last_fullsized":"2023-12-01T01:52:48.026074+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.045370+0000","last_change":"2023-12-01T01:52:48.043879+0000","last_active":"2023-12-01T01:52:48.045370+0000","last_peered":"2023-12-01T01:52:48.045370+0000","last_clean":"2023-12-01T01:52:48.045370+0000","last_became_active":"2023-12-01T01:52:48.043357+0000","last_became_peered":"2023-12-01T01:52:48.043357+0000","last_unstale":"2023-12-01T01:52:48.045370+0000","last_undegraded":"2023-12-01T01:52:48.045370+0000","last_fullsized":"2023-12-01T01:52:48.045370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:59.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'183","reported_seq":481,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-12-01T01:52:47.018333+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:52:47.018333+0000","last_peered":"2023-12-01T01:52:47.018333+0000","last_clean":"2023-12-01T01:52:47.018333+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:52:47.018333+0000","last_undegraded":"2023-12-01T01:52:47.018333+0000","last_fullsized":"2023-12-01T01:52:47.018333+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575051,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.02,"15min":1.0660000000000001},"min":{"1min":0.54900000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.401,"5min":1.514,"15min":1.8089999999999999},"last":1.145},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0309999999999999,"15min":1.0469999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5289999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.0469999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1499999999999999,"15min":1.2070000000000001},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.5669999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2150000000000001,"15min":1.248},"min":{"1min":0.66800000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6279999999999999,"5min":1.744,"15min":1.744},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1040000000000001,"15min":1.117},"min":{"1min":0.749,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4910000000000001,"5min":1.607,"15min":1.728},"last":1.244},{"interface":"front","average":{"1min":1.2,"5min":1.2010000000000001,"15min":1.2190000000000001},"min":{"1min":0.92500000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.641,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.274}]},{"osd":3,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.1910000000000001,"15min":1.2230000000000001},"min":{"1min":0.64200000000000002,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.347,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3180000000000001,"15min":1.3420000000000001},"min":{"1min":0.63700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7490000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.3360000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.3009999999999999,"15min":1.2849999999999999},"min":{"1min":0.72699999999999998,"5min":0.56100000000000005,"15min":0.55500000000000005},"max":{"1min":1.4450000000000001,"5min":1.7549999999999999,"15min":2.2130000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4350000000000001,"15min":1.4059999999999999},"min":{"1min":0.77500000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8100000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.008}]},{"osd":5,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.292,"15min":1.258},"min":{"1min":0.625,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.0379999999999998,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.47,"5min":1.514,"15min":1.514},"min":{"1min":0.69399999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.861,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.3859999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.395,"15min":1.377},"min":{"1min":0.753,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1549999999999998,"5min":2.294,"15min":2.3420000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.387},"min":{"1min":0.82599999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.893,"5min":1.893,"15min":2.3500000000000001},"last":1.0920000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.1020000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.79},"last":1.034},{"interface":"front","average":{"1min":1.036,"5min":1.105,"15min":1.1220000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5349999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":0.999}]},{"osd":1,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.161,"15min":1.159},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1950000000000001,"15min":1.2050000000000001},"min":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0920000000000001},"last":1.204}]},{"osd":2,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1859999999999999,"15min":1.171},"min":{"1min":0.69699999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5760000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.754},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.242,"15min":1.2350000000000001},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":3.024,"5min":3.024,"15min":3.024},"last":1.1499999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":1.0289999999999999,"5min":0.93700000000000006,"15min":0.82399999999999995},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.48}]},{"osd":4,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3,"15min":1.2709999999999999},"min":{"1min":0.94299999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6819999999999999,"5min":2.226,"15min":2.226},"last":1.238},{"interface":"front","average":{"1min":1.339,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":1.028,"5min":0.91400000000000003,"15min":0.78100000000000003},"max":{"1min":1.885,"5min":1.931,"15min":2.2839999999999998},"last":1.5800000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.415,"15min":1.397},"min":{"1min":0.755,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.74,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3},{"interface":"front","average":{"1min":1.47,"5min":1.502,"15min":1.472},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.84099999999999997},"max":{"1min":1.9790000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.6799999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.524,"15min":1.5009999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6870000000000001,"5min":2.762,"15min":2.762},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.577,"15min":1.5629999999999999},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":1.9330000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.742}]}]},{"osd":5,"up_from":13,"seq":55834575051,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32924,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338916,"statfs":{"total":96636764160,"available":96603049984,"internally_reserved":0,"allocated":1921024,"data_stored":1219865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1259999999999999,"15min":1.1040000000000001},"min":{"1min":0.63200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.2010000000000001},"last":1.099},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1180000000000001,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.60099999999999998},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":2.153},"last":1.3700000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.117,"15min":1.1399999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.63200000000000001},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.1399999999999999,"15min":1.159},"min":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.6519999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.2430000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.173,"15min":1.196},"min":{"1min":0.65000000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.702,"5min":1.702,"15min":1.7110000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.208,"15min":1.2330000000000001},"min":{"1min":0.79100000000000004,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":2.0630000000000002},"last":1.1990000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.248,"15min":1.282},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.8169999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.302},"min":{"1min":1.159,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.95099999999999996}]},{"osd":4,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2410000000000001},"min":{"1min":0.83499999999999996,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.018},{"interface":"front","average":{"1min":1.409,"5min":1.319,"15min":1.3480000000000001},"min":{"1min":0.84399999999999997,"5min":0.80300000000000005,"15min":0.79900000000000004},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.452}]},{"osd":6,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3129999999999999,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.859,"5min":1.885,"15min":1.984},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.3939999999999999,"15min":1.3839999999999999},"min":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.149}]},{"osd":7,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.462,"15min":1.4750000000000001},"min":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0920000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.522,"15min":1.5269999999999999},"min":{"1min":1.304,"5min":0.873,"15min":0.873},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.411}]}]},{"osd":4,"up_from":13,"seq":55834575051,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1120000000000001,"15min":1.105},"min":{"1min":0.72199999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.753,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.7130000000000001,"5min":1.7529999999999999,"15min":2.1099999999999999},"last":1.657}]},{"osd":1,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.216,"15min":1.204},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.65900000000000003},"max":{"1min":1.5409999999999999,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.534},{"interface":"front","average":{"1min":1.087,"5min":1.101,"15min":1.129},"min":{"1min":0.76600000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3979999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.458}]},{"osd":2,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.234,"15min":1.2},"min":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.716,"5min":1.871,"15min":1.871},"last":1.204},{"interface":"front","average":{"1min":1.327,"5min":1.3180000000000001,"15min":1.288},"min":{"1min":0.94599999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.581,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.76}]},{"osd":3,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.258,"15min":1.2310000000000001},"min":{"1min":0.81899999999999995,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.6140000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.139},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3200000000000001,"15min":1.3160000000000001},"min":{"1min":0.88500000000000001,"5min":0.753,"15min":0.753},"max":{"1min":1.712,"5min":1.7270000000000001,"15min":1.903},"last":1.3009999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.319,"15min":1.3129999999999999},"min":{"1min":0.94899999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.288,"15min":1.2869999999999999},"min":{"1min":0.97199999999999998,"5min":0.90300000000000002,"15min":0.85399999999999998},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.24}]},{"osd":6,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.381,"15min":1.3500000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.81299999999999994},"max":{"1min":1.8799999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.502,"15min":1.472},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.0960000000000001,"5min":2.4990000000000001,"15min":2.54},"last":1.71}]},{"osd":7,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.472,"15min":1.4610000000000001},"min":{"1min":1.274,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5780000000000001,"15min":1.5620000000000001},"min":{"1min":1.28,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":1.96,"5min":2.04,"15min":2.157},"last":1.486}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58584,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313256,"statfs":{"total":96636764160,"available":96576774144,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.84199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.307},"max":{"1min":1.1899999999999999,"5min":1.3320000000000001,"15min":1.583},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.93500000000000005,"15min":0.97999999999999998},"min":{"1min":0.48099999999999998,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.99199999999999999,"15min":1.056},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.38900000000000001},"max":{"1min":1.5669999999999999,"5min":1.571,"15min":2.0779999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.89300000000000002,"15min":0.94799999999999995},"min":{"1min":0.48399999999999999,"5min":0.48199999999999998,"15min":0.42699999999999999},"max":{"1min":1.1339999999999999,"5min":1.55,"15min":1.6519999999999999},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.96699999999999997,"15min":1.0349999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3360000000000001,"5min":1.4810000000000001,"15min":1.5609999999999999},"last":1.167},{"interface":"front","average":{"1min":0.995,"5min":1.125,"15min":1.1819999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1970000000000001,"15min":1.232},"min":{"1min":0.84999999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.524,"5min":1.5820000000000001,"15min":1.9319999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.198,"15min":1.232},"min":{"1min":0.79300000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.909,"5min":1.909,"15min":2.0419999999999998},"last":0.96099999999999997}]},{"osd":5,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.228,"15min":1.2789999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.71,"5min":1.7669999999999999,"15min":2.0510000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2529999999999999,"15min":1.288},"min":{"1min":0.745,"5min":0.745,"15min":0.64200000000000002},"max":{"1min":1.6759999999999999,"5min":1.681,"15min":2.1880000000000002},"last":1.274}]},{"osd":6,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.89600000000000002,"5min":0.76300000000000001,"15min":0.68300000000000005},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.995},{"interface":"front","average":{"1min":1.264,"5min":1.3700000000000001,"15min":1.405},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8260000000000001,"5min":1.9119999999999999,"15min":1.99},"last":1.4259999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.365,"15min":1.413},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.81499999999999995},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.1619999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4019999999999999,"15min":1.4470000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.83099999999999996},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.2130000000000001},"last":1.0940000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":0.81699999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.5529999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.98399999999999999,"15min":1},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.97099999999999997}]},{"osd":1,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95799999999999996,"15min":0.98699999999999999},"min":{"1min":0.54900000000000004,"5min":0.373,"15min":0.373},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":1.042},{"interface":"front","average":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.081},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.278}]},{"osd":3,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93899999999999995,"15min":1.002},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5189999999999999,"5min":1.625,"15min":1.6479999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.139}]},{"osd":4,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.869,"5min":1.9950000000000001,"15min":2.024},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.111,"5min":1.202,"15min":1.258},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.964},"last":1.2210000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.3129999999999999},"min":{"1min":0.72199999999999998,"5min":0.62,"15min":0.62},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.357,"15min":1.353},"min":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.016,"5min":2.048,"15min":2.323},"last":0.88900000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2849999999999999,"15min":1.3180000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.579,"5min":1.78,"15min":2.2799999999999998},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.90300000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.383}]},{"osd":7,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.448},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3440000000000001,"15min":1.381},"min":{"1min":0.80600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":2.1080000000000001},"last":1.4019999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575049,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59164,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312676,"statfs":{"total":96636764160,"available":96576180224,"internally_reserved":0,"allocated":1921024,"data_stored":1219865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.83499999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.318},"max":{"1min":1.1639999999999999,"5min":1.341,"15min":1.5049999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.876,"5min":0.94999999999999996,"15min":1.0069999999999999},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.597,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85299999999999998,"15min":0.88100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.54},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0349999999999999,"15min":1.081},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.44800000000000001},"max":{"1min":1.472,"5min":1.7250000000000001,"15min":1.804},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.5},"max":{"1min":1.718,"5min":1.827,"15min":1.827},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1,"15min":1.038},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.39700000000000002},"max":{"1min":1.381,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.626}]},{"osd":4,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1579999999999999,"15min":1.173},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.65300000000000002},"max":{"1min":1.425,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.288,"15min":1.3240000000000001},"min":{"1min":0.85599999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.863},"last":1.131}]},{"osd":5,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":0.77700000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.244,"15min":1.27},"min":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":2.1619999999999999},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.294,"15min":1.3220000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.68400000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.861},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.306,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.742,"5min":1.7490000000000001,"15min":1.8839999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3720000000000001,"15min":1.417},"min":{"1min":0.91200000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.782,"5min":1.782,"15min":2.0609999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3859999999999999,"15min":1.399},"min":{"1min":0.80900000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.931,"5min":1.992,"15min":1.992},"last":0.81799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575051,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1327104,"data_stored":629497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84099999999999997,"15min":0.85399999999999998},"min":{"1min":0.435,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.194,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89900000000000002,"15min":0.96499999999999997},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3859999999999999,"5min":1.544,"15min":1.6910000000000001},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.94799999999999995,"15min":0.998},"min":{"1min":0.51500000000000001,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.431,"5min":2.157,"15min":2.157},"last":0.874},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0089999999999999,"15min":1.0649999999999999},"min":{"1min":0.57399999999999995,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.3400000000000001,"5min":1.728,"15min":1.728},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":1.0149999999999999},"min":{"1min":0.623,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.954},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1080000000000001,"15min":1.1539999999999999},"min":{"1min":0.54200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4199999999999999,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.246}]},{"osd":4,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2589999999999999,"15min":1.3029999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5840000000000001,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1899999999999999,"15min":1.258},"min":{"1min":0.90200000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.55,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.323,"15min":1.343},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.71899999999999997},"max":{"1min":1.841,"5min":2.351,"15min":2.351},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2769999999999999,"15min":1.333},"min":{"1min":0.81599999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.546,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.7509999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2789999999999999,"15min":1.319},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7529999999999999,"5min":1.8680000000000001,"15min":2.157},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.351,"15min":1.3839999999999999},"min":{"1min":0.94499999999999995,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.7809999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.5940000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.339,"15min":1.3979999999999999},"min":{"1min":0.94999999999999996,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.5700000000000001,"5min":1.798,"15min":2.4279999999999999},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.3320000000000001,"15min":1.3899999999999999},"min":{"1min":0.98999999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.667,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":0.995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:52:53.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2023-12-01T01:52:53.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.031286+0000', 'last_change': '2023-12-01T01:52:48.031286+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.031286+0000', 'last_undegraded': '2023-12-01T01:52:48.031286+0000', 'last_fullsized': '2023-12-01T01:52:48.031286+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:07:01.922435+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2023-12-01T01:52:53.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.022478+0000', 'last_change': '2023-12-01T01:52:48.022478+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.022478+0000', 'last_undegraded': '2023-12-01T01:52:48.022478+0000', 'last_fullsized': '2023-12-01T01:52:48.022478+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:17:20.199417+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:52:53.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2023-12-01T01:52:53.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.034568+0000', 'last_change': '2023-12-01T01:52:48.034568+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.034568+0000', 'last_undegraded': '2023-12-01T01:52:48.034568+0000', 'last_fullsized': '2023-12-01T01:52:48.034568+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:53.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2023-12-01T01:52:53.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.029059+0000', 'last_change': '2023-12-01T01:52:48.029059+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.029059+0000', 'last_undegraded': '2023-12-01T01:52:48.029059+0000', 'last_fullsized': '2023-12-01T01:52:48.029059+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:06:59.600929+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.022456+0000', 'last_change': '2023-12-01T01:52:48.022456+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.022456+0000', 'last_undegraded': '2023-12-01T01:52:48.022456+0000', 'last_fullsized': '2023-12-01T01:52:48.022456+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:48:34.887669+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.033630+0000', 'last_change': '2023-12-01T01:52:48.033630+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.033630+0000', 'last_undegraded': '2023-12-01T01:52:48.033630+0000', 'last_fullsized': '2023-12-01T01:52:48.033630+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:39:18.420163+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.020737+0000', 'last_change': '2023-12-01T01:52:48.020737+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.020737+0000', 'last_undegraded': '2023-12-01T01:52:48.020737+0000', 'last_fullsized': '2023-12-01T01:52:48.020737+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:08:51.221046+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.024562+0000', 'last_change': '2023-12-01T01:52:48.024562+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.024562+0000', 'last_undegraded': '2023-12-01T01:52:48.024562+0000', 'last_fullsized': '2023-12-01T01:52:48.024562+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:06:21.099958+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.026842+0000', 'last_change': '2023-12-01T01:52:48.026842+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.026842+0000', 'last_undegraded': '2023-12-01T01:52:48.026842+0000', 'last_fullsized': '2023-12-01T01:52:48.026842+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:08:52.282595+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.034705+0000', 'last_change': '2023-12-01T01:52:48.034705+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.034705+0000', 'last_undegraded': '2023-12-01T01:52:48.034705+0000', 'last_fullsized': '2023-12-01T01:52:48.034705+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.029077+0000', 'last_change': '2023-12-01T01:52:48.029077+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.029077+0000', 'last_undegraded': '2023-12-01T01:52:48.029077+0000', 'last_fullsized': '2023-12-01T01:52:48.029077+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.025888+0000', 'last_change': '2023-12-01T01:52:48.025888+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.025888+0000', 'last_undegraded': '2023-12-01T01:52:48.025888+0000', 'last_fullsized': '2023-12-01T01:52:48.025888+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.021972+0000', 'last_change': '2023-12-01T01:52:48.021972+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.021972+0000', 'last_undegraded': '2023-12-01T01:52:48.021972+0000', 'last_fullsized': '2023-12-01T01:52:48.021972+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:45:17.044474+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.037434+0000', 'last_change': '2023-12-01T01:52:48.037434+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.037434+0000', 'last_undegraded': '2023-12-01T01:52:48.037434+0000', 'last_fullsized': '2023-12-01T01:52:48.037434+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:40:59.300637+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2023-12-01T01:52:53.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.019592+0000', 'last_change': '2023-12-01T01:52:48.019592+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.019592+0000', 'last_undegraded': '2023-12-01T01:52:48.019592+0000', 'last_fullsized': '2023-12-01T01:52:48.019592+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:58:23.518943+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.029503+0000', 'last_change': '2023-12-01T01:52:48.029503+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.029503+0000', 'last_undegraded': '2023-12-01T01:52:48.029503+0000', 'last_fullsized': '2023-12-01T01:52:48.029503+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.024646+0000', 'last_change': '2023-12-01T01:52:48.024646+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.024646+0000', 'last_undegraded': '2023-12-01T01:52:48.024646+0000', 'last_fullsized': '2023-12-01T01:52:48.024646+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.029391+0000', 'last_change': '2023-12-01T01:52:48.029391+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.029391+0000', 'last_undegraded': '2023-12-01T01:52:48.029391+0000', 'last_fullsized': '2023-12-01T01:52:48.029391+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.038499+0000', 'last_change': '2023-12-01T01:52:48.038499+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.038499+0000', 'last_undegraded': '2023-12-01T01:52:48.038499+0000', 'last_fullsized': '2023-12-01T01:52:48.038499+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:13:33.646688+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.035539+0000', 'last_change': '2023-12-01T01:52:48.035539+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.035539+0000', 'last_undegraded': '2023-12-01T01:52:48.035539+0000', 'last_fullsized': '2023-12-01T01:52:48.035539+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:50:42.942148+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2023-12-01T01:52:53.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.026846+0000', 'last_change': '2023-12-01T01:52:48.026846+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.026846+0000', 'last_undegraded': '2023-12-01T01:52:48.026846+0000', 'last_fullsized': '2023-12-01T01:52:48.026846+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.036892+0000', 'last_change': '2023-12-01T01:52:48.036892+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.036892+0000', 'last_undegraded': '2023-12-01T01:52:48.036892+0000', 'last_fullsized': '2023-12-01T01:52:48.036892+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.027150+0000', 'last_change': '2023-12-01T01:52:48.027150+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.027150+0000', 'last_undegraded': '2023-12-01T01:52:48.027150+0000', 'last_fullsized': '2023-12-01T01:52:48.027150+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.030146+0000', 'last_change': '2023-12-01T01:52:48.030146+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.030146+0000', 'last_undegraded': '2023-12-01T01:52:48.030146+0000', 'last_fullsized': '2023-12-01T01:52:48.030146+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.031458+0000', 'last_change': '2023-12-01T01:52:48.031458+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.031458+0000', 'last_undegraded': '2023-12-01T01:52:48.031458+0000', 'last_fullsized': '2023-12-01T01:52:48.031458+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.023595+0000', 'last_change': '2023-12-01T01:52:48.023595+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.023595+0000', 'last_undegraded': '2023-12-01T01:52:48.023595+0000', 'last_fullsized': '2023-12-01T01:52:48.023595+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:35:22.154067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.027885+0000', 'last_change': '2023-12-01T01:52:48.027885+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.027885+0000', 'last_undegraded': '2023-12-01T01:52:48.027885+0000', 'last_fullsized': '2023-12-01T01:52:48.027885+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:12:02.919659+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.021893+0000', 'last_change': '2023-12-01T01:52:48.021893+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.021893+0000', 'last_undegraded': '2023-12-01T01:52:48.021893+0000', 'last_fullsized': '2023-12-01T01:52:48.021893+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:25:02.674823+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.020506+0000', 'last_change': '2023-12-01T01:52:48.020506+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.020506+0000', 'last_undegraded': '2023-12-01T01:52:48.020506+0000', 'last_fullsized': '2023-12-01T01:52:48.020506+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.035671+0000', 'last_change': '2023-12-01T01:52:48.035671+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.035671+0000', 'last_undegraded': '2023-12-01T01:52:48.035671+0000', 'last_fullsized': '2023-12-01T01:52:48.035671+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:09:00.516734+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.034799+0000', 'last_change': '2023-12-01T01:52:48.034799+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.034799+0000', 'last_undegraded': '2023-12-01T01:52:48.034799+0000', 'last_fullsized': '2023-12-01T01:52:48.034799+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.028380+0000', 'last_change': '2023-12-01T01:52:48.028380+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.028380+0000', 'last_undegraded': '2023-12-01T01:52:48.028380+0000', 'last_fullsized': '2023-12-01T01:52:48.028380+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:53.955 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.026751+0000', 'last_change': '2023-12-01T01:52:48.026751+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.026751+0000', 'last_undegraded': '2023-12-01T01:52:48.026751+0000', 'last_fullsized': '2023-12-01T01:52:48.026751+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:02:42.893820+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.026112+0000', 'last_change': '2023-12-01T01:52:48.026112+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.026112+0000', 'last_undegraded': '2023-12-01T01:52:48.026112+0000', 'last_fullsized': '2023-12-01T01:52:48.026112+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:55:01.801261+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.024925+0000', 'last_change': '2023-12-01T01:52:48.024925+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.024925+0000', 'last_undegraded': '2023-12-01T01:52:48.024925+0000', 'last_fullsized': '2023-12-01T01:52:48.024925+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:50:17.533885+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.023506+0000', 'last_change': '2023-12-01T01:52:48.023506+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.023506+0000', 'last_undegraded': '2023-12-01T01:52:48.023506+0000', 'last_fullsized': '2023-12-01T01:52:48.023506+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.026841+0000', 'last_change': '2023-12-01T01:52:48.026841+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.026841+0000', 'last_undegraded': '2023-12-01T01:52:48.026841+0000', 'last_fullsized': '2023-12-01T01:52:48.026841+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.032197+0000', 'last_change': '2023-12-01T01:52:48.032197+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.032197+0000', 'last_undegraded': '2023-12-01T01:52:48.032197+0000', 'last_fullsized': '2023-12-01T01:52:48.032197+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:53:05.715094+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:53.956 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.022665+0000', 'last_change': '2023-12-01T01:52:48.022665+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.022665+0000', 'last_undegraded': '2023-12-01T01:52:48.022665+0000', 'last_fullsized': '2023-12-01T01:52:48.022665+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:10:51.868019+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.035505+0000', 'last_change': '2023-12-01T01:52:48.035505+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.035505+0000', 'last_undegraded': '2023-12-01T01:52:48.035505+0000', 'last_fullsized': '2023-12-01T01:52:48.035505+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:23:38.690885+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.035689+0000', 'last_change': '2023-12-01T01:52:48.035689+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.035689+0000', 'last_undegraded': '2023-12-01T01:52:48.035689+0000', 'last_fullsized': '2023-12-01T01:52:48.035689+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:11:12.516160+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.025380+0000', 'last_change': '2023-12-01T01:52:48.025380+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.025380+0000', 'last_undegraded': '2023-12-01T01:52:48.025380+0000', 'last_fullsized': '2023-12-01T01:52:48.025380+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:40:22.643458+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.027539+0000', 'last_change': '2023-12-01T01:52:48.027539+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.027539+0000', 'last_undegraded': '2023-12-01T01:52:48.027539+0000', 'last_fullsized': '2023-12-01T01:52:48.027539+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2023-12-01T01:52:53.957 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.028382+0000', 'last_change': '2023-12-01T01:52:48.028382+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.028382+0000', 'last_undegraded': '2023-12-01T01:52:48.028382+0000', 'last_fullsized': '2023-12-01T01:52:48.028382+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:34:03.971637+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.024486+0000', 'last_change': '2023-12-01T01:52:48.024486+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.024486+0000', 'last_undegraded': '2023-12-01T01:52:48.024486+0000', 'last_fullsized': '2023-12-01T01:52:48.024486+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.025388+0000', 'last_change': '2023-12-01T01:52:48.025388+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.025388+0000', 'last_undegraded': '2023-12-01T01:52:48.025388+0000', 'last_fullsized': '2023-12-01T01:52:48.025388+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:58:00.440400+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.036643+0000', 'last_change': '2023-12-01T01:52:48.036643+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.036643+0000', 'last_undegraded': '2023-12-01T01:52:48.036643+0000', 'last_fullsized': '2023-12-01T01:52:48.036643+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:35:45.833443+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.023509+0000', 'last_change': '2023-12-01T01:52:48.023509+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.023509+0000', 'last_undegraded': '2023-12-01T01:52:48.023509+0000', 'last_fullsized': '2023-12-01T01:52:48.023509+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:22:56.543213+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.026337+0000', 'last_change': '2023-12-01T01:52:48.026337+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.026337+0000', 'last_undegraded': '2023-12-01T01:52:48.026337+0000', 'last_fullsized': '2023-12-01T01:52:48.026337+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:09:36.618385+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2023-12-01T01:52:53.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.033821+0000', 'last_change': '2023-12-01T01:52:48.033821+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.033821+0000', 'last_undegraded': '2023-12-01T01:52:48.033821+0000', 'last_fullsized': '2023-12-01T01:52:48.033821+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.026154+0000', 'last_change': '2023-12-01T01:52:48.026154+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.026154+0000', 'last_undegraded': '2023-12-01T01:52:48.026154+0000', 'last_fullsized': '2023-12-01T01:52:48.026154+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:48:17.235352+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.021314+0000', 'last_change': '2023-12-01T01:52:48.021314+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.021314+0000', 'last_undegraded': '2023-12-01T01:52:48.021314+0000', 'last_fullsized': '2023-12-01T01:52:48.021314+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.021393+0000', 'last_change': '2023-12-01T01:52:48.021393+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.021393+0000', 'last_undegraded': '2023-12-01T01:52:48.021393+0000', 'last_fullsized': '2023-12-01T01:52:48.021393+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:52:45.545702+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.021667+0000', 'last_change': '2023-12-01T01:52:48.021667+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.021667+0000', 'last_undegraded': '2023-12-01T01:52:48.021667+0000', 'last_fullsized': '2023-12-01T01:52:48.021667+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.026638+0000', 'last_change': '2023-12-01T01:52:48.026638+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.026638+0000', 'last_undegraded': '2023-12-01T01:52:48.026638+0000', 'last_fullsized': '2023-12-01T01:52:48.026638+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2023-12-01T01:52:53.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.023160+0000', 'last_change': '2023-12-01T01:52:48.023160+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.023160+0000', 'last_undegraded': '2023-12-01T01:52:48.023160+0000', 'last_fullsized': '2023-12-01T01:52:48.023160+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:52:39.608737+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.025759+0000', 'last_change': '2023-12-01T01:52:48.025759+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.025759+0000', 'last_undegraded': '2023-12-01T01:52:48.025759+0000', 'last_fullsized': '2023-12-01T01:52:48.025759+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:55:14.304687+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.021428+0000', 'last_change': '2023-12-01T01:52:48.021428+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.021428+0000', 'last_undegraded': '2023-12-01T01:52:48.021428+0000', 'last_fullsized': '2023-12-01T01:52:48.021428+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:35:01.791105+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.033844+0000', 'last_change': '2023-12-01T01:52:48.033844+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.033844+0000', 'last_undegraded': '2023-12-01T01:52:48.033844+0000', 'last_fullsized': '2023-12-01T01:52:48.033844+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:46:03.437505+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.020256+0000', 'last_change': '2023-12-01T01:52:48.020256+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.020256+0000', 'last_undegraded': '2023-12-01T01:52:48.020256+0000', 'last_fullsized': '2023-12-01T01:52:48.020256+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:34:49.188317+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.027872+0000', 'last_change': '2023-12-01T01:52:48.027872+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.027872+0000', 'last_undegraded': '2023-12-01T01:52:48.027872+0000', 'last_fullsized': '2023-12-01T01:52:48.027872+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:17:45.934623+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2023-12-01T01:52:53.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.019339+0000', 'last_change': '2023-12-01T01:52:48.019339+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.019339+0000', 'last_undegraded': '2023-12-01T01:52:48.019339+0000', 'last_fullsized': '2023-12-01T01:52:48.019339+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:20:23.997177+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2023-12-01T01:52:53.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.020747+0000', 'last_change': '2023-12-01T01:52:48.020747+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.020747+0000', 'last_undegraded': '2023-12-01T01:52:48.020747+0000', 'last_fullsized': '2023-12-01T01:52:48.020747+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:02:57.070103+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T01:52:53.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2023-12-01T01:52:53.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2023-12-01T01:52:48.026074+0000', 'last_change': '2023-12-01T01:52:48.026074+0000', 'last_active': '2023-12-01T01:52:48.003394+0000', 'last_peered': '2023-12-01T01:52:48.003394+0000', 'last_clean': '2023-12-01T01:52:48.003394+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T01:52:48.026074+0000', 'last_undegraded': '2023-12-01T01:52:48.026074+0000', 'last_fullsized': '2023-12-01T01:52:48.026074+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'last_clean_scrub_stamp': '2023-12-01T01:52:48.003394+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T01:52:53.961 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-01T01:52:54.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.204+0000 7fa1457c9640 1 Processor -- start 2023-12-01T01:52:54.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.205+0000 7fa1457c9640 1 -- start start 2023-12-01T01:52:54.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.206+0000 7fa1457c9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa14015cd20 0x7fa1400d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.206+0000 7fa1457c9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa1400b5d80 0x7fa1400b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.206+0000 7fa1457c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400d1230 0x7fa1400d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.206+0000 7fa1457c9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400b2160 con 0x7fa1400b5d80 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.206+0000 7fa1457c9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400d2a10 con 0x7fa14015cd20 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.206+0000 7fa1457c9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400d1b90 con 0x7fa1400d1230 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.207+0000 7fa13e7fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa1400b5d80 0x7fa1400b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.207+0000 7fa13e7fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa1400b5d80 0x7fa1400b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43880/0 (socket says 172.21.15.115:43880) 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.207+0000 7fa13e7fc640 1 -- 172.21.15.115:0/3322706423 learned_addr learned my addr 172.21.15.115:0/3322706423 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.207+0000 7fa13effd640 1 --2- 172.21.15.115:0/3322706423 >> v2:172.21.15.142:3300/0 conn(0x7fa14015cd20 0x7fa1400d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.207+0000 7fa13f7fe640 1 --2- 172.21.15.115:0/3322706423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400d1230 0x7fa1400d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.207+0000 7fa13e7fc640 1 -- 172.21.15.115:0/3322706423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400d1230 msgr2=0x7fa1400d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.207+0000 7fa13e7fc640 1 --2- 172.21.15.115:0/3322706423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400d1230 0x7fa1400d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.207+0000 7fa13e7fc640 1 -- 172.21.15.115:0/3322706423 >> v2:172.21.15.142:3300/0 conn(0x7fa14015cd20 msgr2=0x7fa1400d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.207+0000 7fa13e7fc640 1 --2- 172.21.15.115:0/3322706423 >> v2:172.21.15.142:3300/0 conn(0x7fa14015cd20 0x7fa1400d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.207+0000 7fa13e7fc640 1 -- 172.21.15.115:0/3322706423 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400ce2e0 con 0x7fa1400b5d80 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.208+0000 7fa13e7fc640 1 --2- 172.21.15.115:0/3322706423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400b5d80 0x7fa1400b8570 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa14008bdb0 tx=0x7fa13402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b22ea96762830b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:54.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.209+0000 7fa13dffb640 1 -- 172.21.15.115:0/3322706423 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13402c9e0 con 0x7fa1400b5d80 2023-12-01T01:52:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.209+0000 7fa13dffb640 1 -- 172.21.15.115:0/3322706423 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13402cb40 con 0x7fa1400b5d80 2023-12-01T01:52:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.209+0000 7fa13dffb640 1 -- 172.21.15.115:0/3322706423 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1340343f0 con 0x7fa1400b5d80 2023-12-01T01:52:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.209+0000 7fa1457c9640 1 -- 172.21.15.115:0/3322706423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400b5d80 msgr2=0x7fa1400b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.209+0000 7fa1457c9640 1 --2- 172.21.15.115:0/3322706423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400b5d80 0x7fa1400b8570 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa14008bdb0 tx=0x7fa13402be40 comp rx=0 tx=0).stop 2023-12-01T01:52:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.209+0000 7fa1457c9640 1 -- 172.21.15.115:0/3322706423 shutdown_connections 2023-12-01T01:52:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.209+0000 7fa1457c9640 1 --2- 172.21.15.115:0/3322706423 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400d1230 0x7fa1400d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.209+0000 7fa1457c9640 1 --2- 172.21.15.115:0/3322706423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400b5d80 0x7fa1400b8570 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.209+0000 7fa1457c9640 1 --2- 172.21.15.115:0/3322706423 >> v2:172.21.15.142:3300/0 conn(0x7fa14015cd20 0x7fa1400d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.209+0000 7fa1457c9640 1 -- 172.21.15.115:0/3322706423 >> 172.21.15.115:0/3322706423 conn(0x7fa14006a0e0 msgr2=0x7fa14006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.210+0000 7fa1457c9640 1 -- 172.21.15.115:0/3322706423 shutdown_connections 2023-12-01T01:52:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.210+0000 7fa1457c9640 1 -- 172.21.15.115:0/3322706423 wait complete. 2023-12-01T01:52:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.212+0000 7fa1457c9640 1 Processor -- start 2023-12-01T01:52:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.212+0000 7fa1457c9640 1 -- start start 2023-12-01T01:52:54.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.213+0000 7fa1457c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400b5d80 0x7fa1400c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.213+0000 7fa1457c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400d1230 0x7fa1400b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.213+0000 7fa1457c9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14015cd20 0x7fa1401f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.213+0000 7fa1457c9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140158ee0 con 0x7fa1400d1230 2023-12-01T01:52:54.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.213+0000 7fa1457c9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140158640 con 0x7fa14015cd20 2023-12-01T01:52:54.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.213+0000 7fa1457c9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa140158920 con 0x7fa1400b5d80 2023-12-01T01:52:54.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.213+0000 7fa13e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400d1230 0x7fa1400b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.213+0000 7fa13e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400d1230 0x7fa1400b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43886/0 (socket says 172.21.15.115:43886) 2023-12-01T01:52:54.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.213+0000 7fa13e7fc640 1 -- 172.21.15.115:0/3719327546 learned_addr learned my addr 172.21.15.115:0/3719327546 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:54.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa13effd640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400b5d80 0x7fa1400c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa13f7fe640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14015cd20 0x7fa1401f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa13e7fc640 1 -- 172.21.15.115:0/3719327546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400b5d80 msgr2=0x7fa1400c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa13e7fc640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400b5d80 0x7fa1400c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa13e7fc640 1 -- 172.21.15.115:0/3719327546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14015cd20 msgr2=0x7fa1401f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa13e7fc640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14015cd20 0x7fa1401f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa13e7fc640 1 -- 172.21.15.115:0/3719327546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa134009b30 con 0x7fa1400d1230 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa13effd640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400b5d80 0x7fa1400c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa13e7fc640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400d1230 0x7fa1400b4ca0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa134000f80 tx=0x7fa134004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa1277fe640 1 -- 172.21.15.115:0/3719327546 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134009510 con 0x7fa1400d1230 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa1277fe640 1 -- 172.21.15.115:0/3719327546 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13403e450 con 0x7fa1400d1230 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa1277fe640 1 -- 172.21.15.115:0/3719327546 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13403e800 con 0x7fa1400d1230 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.214+0000 7fa1457c9640 1 -- 172.21.15.115:0/3719327546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa140158c00 con 0x7fa1400d1230 2023-12-01T01:52:54.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.215+0000 7fa1457c9640 1 -- 172.21.15.115:0/3719327546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1400c8f80 con 0x7fa1400d1230 2023-12-01T01:52:54.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.216+0000 7fa1457c9640 1 -- 172.21.15.115:0/3719327546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14015f470 con 0x7fa1400d1230 2023-12-01T01:52:54.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.217+0000 7fa1277fe640 1 -- 172.21.15.115:0/3719327546 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa13403e960 con 0x7fa1400d1230 2023-12-01T01:52:54.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.218+0000 7fa1277fe640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa12c06a550 0x7fa12c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.218+0000 7fa1277fe640 1 -- 172.21.15.115:0/3719327546 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1340ab920 con 0x7fa1400d1230 2023-12-01T01:52:54.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.218+0000 7fa13effd640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa12c06a550 0x7fa12c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.219+0000 7fa13effd640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa12c06a550 0x7fa12c06ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa130006fd0 tx=0x7fa130008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:54.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.222+0000 7fa1277fe640 1 -- 172.21.15.115:0/3719327546 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa134076d60 con 0x7fa1400d1230 2023-12-01T01:52:54.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.483+0000 7fa1457c9640 1 -- 172.21.15.115:0/3719327546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa14004e890 con 0x7fa1400d1230 2023-12-01T01:52:54.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.484+0000 7fa1277fe640 1 -- 172.21.15.115:0/3719327546 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3231 (secure 0 0 0) 0x7fa13407a380 con 0x7fa1400d1230 2023-12-01T01:52:54.486 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:54.486 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1029,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1701394558,"num_in_osds":8,"osd_in_since":1701394549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":324083712,"bytes_avail":772770029568,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":48,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7455}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2023-12-01T01:52:36.102931+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.487+0000 7fa1457c9640 1 -- 172.21.15.115:0/3719327546 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa12c06a550 msgr2=0x7fa12c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.487+0000 7fa1457c9640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa12c06a550 0x7fa12c06ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa130006fd0 tx=0x7fa130008040 comp rx=0 tx=0).stop 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.487+0000 7fa1457c9640 1 -- 172.21.15.115:0/3719327546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400d1230 msgr2=0x7fa1400b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.487+0000 7fa1457c9640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400d1230 0x7fa1400b4ca0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa134000f80 tx=0x7fa134004a40 comp rx=0 tx=0).stop 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.487+0000 7fa1457c9640 1 -- 172.21.15.115:0/3719327546 shutdown_connections 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.487+0000 7fa1457c9640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa12c06a550 0x7fa12c06ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.487+0000 7fa1457c9640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa14015cd20 0x7fa1401f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.487+0000 7fa1457c9640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400d1230 0x7fa1400b4ca0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.487+0000 7fa1457c9640 1 --2- 172.21.15.115:0/3719327546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1400b5d80 0x7fa1400c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.487+0000 7fa1457c9640 1 -- 172.21.15.115:0/3719327546 >> 172.21.15.115:0/3719327546 conn(0x7fa14006a0e0 msgr2=0x7fa1400c94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:54.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.488+0000 7fa1457c9640 1 -- 172.21.15.115:0/3719327546 shutdown_connections 2023-12-01T01:52:54.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.488+0000 7fa1457c9640 1 -- 172.21.15.115:0/3719327546 wait complete. 2023-12-01T01:52:54.507 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-01T01:52:54.507 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:52:54.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.724+0000 7fb8585b9640 1 Processor -- start 2023-12-01T01:52:54.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.724+0000 7fb8585b9640 1 -- start start 2023-12-01T01:52:54.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.726+0000 7fb8585b9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb85015e2f0 0x7fb8500ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.726+0000 7fb8585b9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb85006f2f0 0x7fb8500caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.726+0000 7fb8585b9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8500add10 con 0x7fb8500adf10 2023-12-01T01:52:54.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.726+0000 7fb8585b9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8500c5760 con 0x7fb85015e2f0 2023-12-01T01:52:54.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.726+0000 7fb8585b9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85015e6d0 con 0x7fb85006f2f0 2023-12-01T01:52:54.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.726+0000 7fb856355640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb85015e2f0 0x7fb8500ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.726+0000 7fb856355640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb85015e2f0 0x7fb8500ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43888/0 (socket says 172.21.15.115:43888) 2023-12-01T01:52:54.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.726+0000 7fb856355640 1 -- 172.21.15.115:0/3087666494 learned_addr learned my addr 172.21.15.115:0/3087666494 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:54.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.726+0000 7fb855b54640 1 --2- 172.21.15.115:0/3087666494 >> v2:172.21.15.142:3300/0 conn(0x7fb85006f2f0 0x7fb8500caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.727+0000 7fb856355640 1 -- 172.21.15.115:0/3087666494 >> v2:172.21.15.142:3300/0 conn(0x7fb85006f2f0 msgr2=0x7fb8500caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:54.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.727+0000 7fb856355640 1 --2- 172.21.15.115:0/3087666494 >> v2:172.21.15.142:3300/0 conn(0x7fb85006f2f0 0x7fb8500caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.727+0000 7fb856355640 1 -- 172.21.15.115:0/3087666494 >> v1:172.21.15.142:6789/0 conn(0x7fb8500adf10 legacy=0x7fb8500c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:54.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.727+0000 7fb856355640 1 -- 172.21.15.115:0/3087666494 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8500cf7b0 con 0x7fb85015e2f0 2023-12-01T01:52:54.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.728+0000 7fb856355640 1 --2- 172.21.15.115:0/3087666494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85015e2f0 0x7fb8500ca660 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb85006de80 tx=0x7fb84c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a390d4f664da2613 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:54.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.728+0000 7fb855353640 1 -- 172.21.15.115:0/3087666494 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c031750 con 0x7fb85015e2f0 2023-12-01T01:52:54.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.728+0000 7fb855353640 1 -- 172.21.15.115:0/3087666494 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c0318b0 con 0x7fb85015e2f0 2023-12-01T01:52:54.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.728+0000 7fb855353640 1 -- 172.21.15.115:0/3087666494 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c031c40 con 0x7fb85015e2f0 2023-12-01T01:52:54.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.728+0000 7fb8585b9640 1 -- 172.21.15.115:0/3087666494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85015e2f0 msgr2=0x7fb8500ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:54.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.729+0000 7fb8585b9640 1 --2- 172.21.15.115:0/3087666494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85015e2f0 0x7fb8500ca660 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb85006de80 tx=0x7fb84c02b7f0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.729+0000 7fb8585b9640 1 -- 172.21.15.115:0/3087666494 shutdown_connections 2023-12-01T01:52:54.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.729+0000 7fb8585b9640 1 --2- 172.21.15.115:0/3087666494 >> v2:172.21.15.142:3300/0 conn(0x7fb85006f2f0 0x7fb8500caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.729+0000 7fb8585b9640 1 --2- 172.21.15.115:0/3087666494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85015e2f0 0x7fb8500ca660 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.729+0000 7fb8585b9640 1 -- 172.21.15.115:0/3087666494 >> 172.21.15.115:0/3087666494 conn(0x7fb8500ce880 msgr2=0x7fb8500cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:54.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.729+0000 7fb8585b9640 1 -- 172.21.15.115:0/3087666494 shutdown_connections 2023-12-01T01:52:54.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.729+0000 7fb8585b9640 1 -- 172.21.15.115:0/3087666494 wait complete. 2023-12-01T01:52:54.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.731+0000 7fb8585b9640 1 Processor -- start 2023-12-01T01:52:54.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.731+0000 7fb8585b9640 1 -- start start 2023-12-01T01:52:54.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.732+0000 7fb8585b9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb85006f2f0 0x7fb8500acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.732+0000 7fb8585b9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8500adf10 0x7fb8500c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.732+0000 7fb8585b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85015e2f0 0x7fb8500c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.732+0000 7fb8585b9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500c5e90 con 0x7fb8500adf10 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.732+0000 7fb8585b9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500c6110 con 0x7fb85015e2f0 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.732+0000 7fb8585b9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8500c63f0 con 0x7fb85006f2f0 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb855b54640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8500adf10 0x7fb8500c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb855b54640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8500adf10 0x7fb8500c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43890/0 (socket says 172.21.15.115:43890) 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb855b54640 1 -- 172.21.15.115:0/3212735825 learned_addr learned my addr 172.21.15.115:0/3212735825 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb856355640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb85006f2f0 0x7fb8500acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb856b56640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85015e2f0 0x7fb8500c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb855b54640 1 -- 172.21.15.115:0/3212735825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb85006f2f0 msgr2=0x7fb8500acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb855b54640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb85006f2f0 0x7fb8500acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb855b54640 1 -- 172.21.15.115:0/3212735825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85015e2f0 msgr2=0x7fb8500c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb855b54640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85015e2f0 0x7fb8500c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb855b54640 1 -- 172.21.15.115:0/3212735825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c0094d0 con 0x7fb8500adf10 2023-12-01T01:52:54.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb856b56640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85015e2f0 0x7fb8500c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:54.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.733+0000 7fb855b54640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8500adf10 0x7fb8500c5950 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb84000d5f0 tx=0x7fb84000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:54.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.734+0000 7fb846ffd640 1 -- 172.21.15.115:0/3212735825 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8400125f0 con 0x7fb8500adf10 2023-12-01T01:52:54.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.734+0000 7fb846ffd640 1 -- 172.21.15.115:0/3212735825 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb840012750 con 0x7fb8500adf10 2023-12-01T01:52:54.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.734+0000 7fb846ffd640 1 -- 172.21.15.115:0/3212735825 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb840012b00 con 0x7fb8500adf10 2023-12-01T01:52:54.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.734+0000 7fb8585b9640 1 -- 172.21.15.115:0/3212735825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8500c66d0 con 0x7fb8500adf10 2023-12-01T01:52:54.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.734+0000 7fb8585b9640 1 -- 172.21.15.115:0/3212735825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8500c7da0 con 0x7fb8500adf10 2023-12-01T01:52:54.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.735+0000 7fb8585b9640 1 -- 172.21.15.115:0/3212735825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85006fda0 con 0x7fb8500adf10 2023-12-01T01:52:54.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.736+0000 7fb846ffd640 1 -- 172.21.15.115:0/3212735825 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb840012c60 con 0x7fb8500adf10 2023-12-01T01:52:54.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.737+0000 7fb846ffd640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb83406a550 0x7fb83406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:54.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.737+0000 7fb846ffd640 1 -- 172.21.15.115:0/3212735825 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb840088f30 con 0x7fb8500adf10 2023-12-01T01:52:54.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.738+0000 7fb856355640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb83406a550 0x7fb83406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:54.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.738+0000 7fb856355640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb83406a550 0x7fb83406ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb84c035000 tx=0x7fb84c008030 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:54.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.742+0000 7fb846ffd640 1 -- 172.21.15.115:0/3212735825 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb840054400 con 0x7fb8500adf10 2023-12-01T01:52:54.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.931+0000 7fb8585b9640 1 -- 172.21.15.115:0/3212735825 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8500b43d0 con 0x7fb83406a550 2023-12-01T01:52:54.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.955+0000 7fb846ffd640 1 -- 172.21.15.115:0/3212735825 <== mgr.7230 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287293 (secure 0 0 0) 0x7fb84c031b20 con 0x7fb83406a550 2023-12-01T01:52:54.959 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:54.961 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:52:54.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.961+0000 7fb8585b9640 1 -- 172.21.15.115:0/3212735825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb83406a550 msgr2=0x7fb83406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:54.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.961+0000 7fb8585b9640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb83406a550 0x7fb83406ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb84c035000 tx=0x7fb84c008030 comp rx=0 tx=0).stop 2023-12-01T01:52:54.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.961+0000 7fb8585b9640 1 -- 172.21.15.115:0/3212735825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8500adf10 msgr2=0x7fb8500c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:54.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.962+0000 7fb8585b9640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8500adf10 0x7fb8500c5950 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb84000d5f0 tx=0x7fb84000dab0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.962+0000 7fb8585b9640 1 -- 172.21.15.115:0/3212735825 shutdown_connections 2023-12-01T01:52:54.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.962+0000 7fb8585b9640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb83406a550 0x7fb83406ca10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.962+0000 7fb8585b9640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85015e2f0 0x7fb8500c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.962+0000 7fb8585b9640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8500adf10 0x7fb8500c5950 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.962+0000 7fb8585b9640 1 --2- 172.21.15.115:0/3212735825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb85006f2f0 0x7fb8500acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:54.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.962+0000 7fb8585b9640 1 -- 172.21.15.115:0/3212735825 >> 172.21.15.115:0/3212735825 conn(0x7fb8500ce880 msgr2=0x7fb85015d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:54.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.962+0000 7fb8585b9640 1 -- 172.21.15.115:0/3212735825 shutdown_connections 2023-12-01T01:52:54.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:54.962+0000 7fb8585b9640 1 -- 172.21.15.115:0/3212735825 wait complete. 2023-12-01T01:52:54.984 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T01:52:54.107102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316840,"kb_used_data":11880,"kb_used_omap":0,"kb_used_meta":304896,"kb_avail":754657880,"statfs":{"total":773094113280,"available":772769669120,"internally_reserved":0,"allocated":12165120,"data_stored":6376236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312213504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.096625"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525276+0000","last_change":"2023-12-01T01:52:48.519541+0000","last_active":"2023-12-01T01:52:48.525276+0000","last_peered":"2023-12-01T01:52:48.525276+0000","last_clean":"2023-12-01T01:52:48.525276+0000","last_became_active":"2023-12-01T01:52:48.518919+0000","last_became_peered":"2023-12-01T01:52:48.518919+0000","last_unstale":"2023-12-01T01:52:48.525276+0000","last_undegraded":"2023-12-01T01:52:48.525276+0000","last_fullsized":"2023-12-01T01:52:48.525276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:07:01.922435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.524811+0000","last_change":"2023-12-01T01:52:48.522421+0000","last_active":"2023-12-01T01:52:48.524811+0000","last_peered":"2023-12-01T01:52:48.524811+0000","last_clean":"2023-12-01T01:52:48.524811+0000","last_became_active":"2023-12-01T01:52:48.522158+0000","last_became_peered":"2023-12-01T01:52:48.522158+0000","last_unstale":"2023-12-01T01:52:48.524811+0000","last_undegraded":"2023-12-01T01:52:48.524811+0000","last_fullsized":"2023-12-01T01:52:48.524811+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:20.199417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533775+0000","last_change":"2023-12-01T01:52:48.529730+0000","last_active":"2023-12-01T01:52:48.533775+0000","last_peered":"2023-12-01T01:52:48.533775+0000","last_clean":"2023-12-01T01:52:48.533775+0000","last_became_active":"2023-12-01T01:52:48.529453+0000","last_became_peered":"2023-12-01T01:52:48.529453+0000","last_unstale":"2023-12-01T01:52:48.533775+0000","last_undegraded":"2023-12-01T01:52:48.533775+0000","last_fullsized":"2023-12-01T01:52:48.533775+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.526329+0000","last_change":"2023-12-01T01:52:48.520005+0000","last_active":"2023-12-01T01:52:48.526329+0000","last_peered":"2023-12-01T01:52:48.526329+0000","last_clean":"2023-12-01T01:52:48.526329+0000","last_became_active":"2023-12-01T01:52:48.519744+0000","last_became_peered":"2023-12-01T01:52:48.519744+0000","last_unstale":"2023-12-01T01:52:48.526329+0000","last_undegraded":"2023-12-01T01:52:48.526329+0000","last_fullsized":"2023-12-01T01:52:48.526329+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:06:59.600929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527655+0000","last_change":"2023-12-01T01:52:48.524456+0000","last_active":"2023-12-01T01:52:48.527655+0000","last_peered":"2023-12-01T01:52:48.527655+0000","last_clean":"2023-12-01T01:52:48.527655+0000","last_became_active":"2023-12-01T01:52:48.524192+0000","last_became_peered":"2023-12-01T01:52:48.524192+0000","last_unstale":"2023-12-01T01:52:48.527655+0000","last_undegraded":"2023-12-01T01:52:48.527655+0000","last_fullsized":"2023-12-01T01:52:48.527655+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:48:34.887669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.535085+0000","last_change":"2023-12-01T01:52:48.530842+0000","last_active":"2023-12-01T01:52:48.535085+0000","last_peered":"2023-12-01T01:52:48.535085+0000","last_clean":"2023-12-01T01:52:48.535085+0000","last_became_active":"2023-12-01T01:52:48.529072+0000","last_became_peered":"2023-12-01T01:52:48.529072+0000","last_unstale":"2023-12-01T01:52:48.535085+0000","last_undegraded":"2023-12-01T01:52:48.535085+0000","last_fullsized":"2023-12-01T01:52:48.535085+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:39:18.420163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.524471+0000","last_change":"2023-12-01T01:52:48.517258+0000","last_active":"2023-12-01T01:52:48.524471+0000","last_peered":"2023-12-01T01:52:48.524471+0000","last_clean":"2023-12-01T01:52:48.524471+0000","last_became_active":"2023-12-01T01:52:48.516997+0000","last_became_peered":"2023-12-01T01:52:48.516997+0000","last_unstale":"2023-12-01T01:52:48.524471+0000","last_undegraded":"2023-12-01T01:52:48.524471+0000","last_fullsized":"2023-12-01T01:52:48.524471+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:51.221046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532821+0000","last_change":"2023-12-01T01:52:48.528652+0000","last_active":"2023-12-01T01:52:48.532821+0000","last_peered":"2023-12-01T01:52:48.532821+0000","last_clean":"2023-12-01T01:52:48.532821+0000","last_became_active":"2023-12-01T01:52:48.528108+0000","last_became_peered":"2023-12-01T01:52:48.528108+0000","last_unstale":"2023-12-01T01:52:48.532821+0000","last_undegraded":"2023-12-01T01:52:48.532821+0000","last_fullsized":"2023-12-01T01:52:48.532821+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:21.099958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.515844+0000","last_change":"2023-12-01T01:52:48.511194+0000","last_active":"2023-12-01T01:52:48.515844+0000","last_peered":"2023-12-01T01:52:48.515844+0000","last_clean":"2023-12-01T01:52:48.515844+0000","last_became_active":"2023-12-01T01:52:48.510913+0000","last_became_peered":"2023-12-01T01:52:48.510913+0000","last_unstale":"2023-12-01T01:52:48.515844+0000","last_undegraded":"2023-12-01T01:52:48.515844+0000","last_fullsized":"2023-12-01T01:52:48.515844+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:52.282595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527630+0000","last_change":"2023-12-01T01:52:48.526094+0000","last_active":"2023-12-01T01:52:48.527630+0000","last_peered":"2023-12-01T01:52:48.527630+0000","last_clean":"2023-12-01T01:52:48.527630+0000","last_became_active":"2023-12-01T01:52:48.525631+0000","last_became_peered":"2023-12-01T01:52:48.525631+0000","last_unstale":"2023-12-01T01:52:48.527630+0000","last_undegraded":"2023-12-01T01:52:48.527630+0000","last_fullsized":"2023-12-01T01:52:48.527630+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532749+0000","last_change":"2023-12-01T01:52:48.523827+0000","last_active":"2023-12-01T01:52:48.532749+0000","last_peered":"2023-12-01T01:52:48.532749+0000","last_clean":"2023-12-01T01:52:48.532749+0000","last_became_active":"2023-12-01T01:52:48.521531+0000","last_became_peered":"2023-12-01T01:52:48.521531+0000","last_unstale":"2023-12-01T01:52:48.532749+0000","last_undegraded":"2023-12-01T01:52:48.532749+0000","last_fullsized":"2023-12-01T01:52:48.532749+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.520324+0000","last_change":"2023-12-01T01:52:48.517956+0000","last_active":"2023-12-01T01:52:48.520324+0000","last_peered":"2023-12-01T01:52:48.520324+0000","last_clean":"2023-12-01T01:52:48.520324+0000","last_became_active":"2023-12-01T01:52:48.517665+0000","last_became_peered":"2023-12-01T01:52:48.517665+0000","last_unstale":"2023-12-01T01:52:48.520324+0000","last_undegraded":"2023-12-01T01:52:48.520324+0000","last_fullsized":"2023-12-01T01:52:48.520324+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527638+0000","last_change":"2023-12-01T01:52:48.523339+0000","last_active":"2023-12-01T01:52:48.527638+0000","last_peered":"2023-12-01T01:52:48.527638+0000","last_clean":"2023-12-01T01:52:48.527638+0000","last_became_active":"2023-12-01T01:52:48.523056+0000","last_became_peered":"2023-12-01T01:52:48.523056+0000","last_unstale":"2023-12-01T01:52:48.527638+0000","last_undegraded":"2023-12-01T01:52:48.527638+0000","last_fullsized":"2023-12-01T01:52:48.527638+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:45:17.044474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532841+0000","last_change":"2023-12-01T01:52:48.527584+0000","last_active":"2023-12-01T01:52:48.532841+0000","last_peered":"2023-12-01T01:52:48.532841+0000","last_clean":"2023-12-01T01:52:48.532841+0000","last_became_active":"2023-12-01T01:52:48.526884+0000","last_became_peered":"2023-12-01T01:52:48.526884+0000","last_unstale":"2023-12-01T01:52:48.532841+0000","last_undegraded":"2023-12-01T01:52:48.532841+0000","last_fullsized":"2023-12-01T01:52:48.532841+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:40:59.300637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.524491+0000","last_change":"2023-12-01T01:52:48.518153+0000","last_active":"2023-12-01T01:52:48.524491+0000","last_peered":"2023-12-01T01:52:48.524491+0000","last_clean":"2023-12-01T01:52:48.524491+0000","last_became_active":"2023-12-01T01:52:48.517878+0000","last_became_peered":"2023-12-01T01:52:48.517878+0000","last_unstale":"2023-12-01T01:52:48.524491+0000","last_undegraded":"2023-12-01T01:52:48.524491+0000","last_fullsized":"2023-12-01T01:52:48.524491+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:58:23.518943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.521765+0000","last_change":"2023-12-01T01:52:48.520173+0000","last_active":"2023-12-01T01:52:48.521765+0000","last_peered":"2023-12-01T01:52:48.521765+0000","last_clean":"2023-12-01T01:52:48.521765+0000","last_became_active":"2023-12-01T01:52:48.518959+0000","last_became_peered":"2023-12-01T01:52:48.518959+0000","last_unstale":"2023-12-01T01:52:48.521765+0000","last_undegraded":"2023-12-01T01:52:48.521765+0000","last_fullsized":"2023-12-01T01:52:48.521765+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.526378+0000","last_change":"2023-12-01T01:52:48.523081+0000","last_active":"2023-12-01T01:52:48.526378+0000","last_peered":"2023-12-01T01:52:48.526378+0000","last_clean":"2023-12-01T01:52:48.526378+0000","last_became_active":"2023-12-01T01:52:48.522578+0000","last_became_peered":"2023-12-01T01:52:48.522578+0000","last_unstale":"2023-12-01T01:52:48.526378+0000","last_undegraded":"2023-12-01T01:52:48.526378+0000","last_fullsized":"2023-12-01T01:52:48.526378+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.528907+0000","last_change":"2023-12-01T01:52:48.520828+0000","last_active":"2023-12-01T01:52:48.528907+0000","last_peered":"2023-12-01T01:52:48.528907+0000","last_clean":"2023-12-01T01:52:48.528907+0000","last_became_active":"2023-12-01T01:52:48.520448+0000","last_became_peered":"2023-12-01T01:52:48.520448+0000","last_unstale":"2023-12-01T01:52:48.528907+0000","last_undegraded":"2023-12-01T01:52:48.528907+0000","last_fullsized":"2023-12-01T01:52:48.528907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532779+0000","last_change":"2023-12-01T01:52:48.528240+0000","last_active":"2023-12-01T01:52:48.532779+0000","last_peered":"2023-12-01T01:52:48.532779+0000","last_clean":"2023-12-01T01:52:48.532779+0000","last_became_active":"2023-12-01T01:52:48.527401+0000","last_became_peered":"2023-12-01T01:52:48.527401+0000","last_unstale":"2023-12-01T01:52:48.532779+0000","last_undegraded":"2023-12-01T01:52:48.532779+0000","last_fullsized":"2023-12-01T01:52:48.532779+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:13:33.646688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.536524+0000","last_change":"2023-12-01T01:52:48.532223+0000","last_active":"2023-12-01T01:52:48.536524+0000","last_peered":"2023-12-01T01:52:48.536524+0000","last_clean":"2023-12-01T01:52:48.536524+0000","last_became_active":"2023-12-01T01:52:48.530936+0000","last_became_peered":"2023-12-01T01:52:48.530936+0000","last_unstale":"2023-12-01T01:52:48.536524+0000","last_undegraded":"2023-12-01T01:52:48.536524+0000","last_fullsized":"2023-12-01T01:52:48.536524+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:42.942148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.539847+0000","last_change":"2023-12-01T01:52:48.531631+0000","last_active":"2023-12-01T01:52:48.539847+0000","last_peered":"2023-12-01T01:52:48.539847+0000","last_clean":"2023-12-01T01:52:48.539847+0000","last_became_active":"2023-12-01T01:52:48.531336+0000","last_became_peered":"2023-12-01T01:52:48.531336+0000","last_unstale":"2023-12-01T01:52:48.539847+0000","last_undegraded":"2023-12-01T01:52:48.539847+0000","last_fullsized":"2023-12-01T01:52:48.539847+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.517000+0000","last_change":"2023-12-01T01:52:48.513602+0000","last_active":"2023-12-01T01:52:48.517000+0000","last_peered":"2023-12-01T01:52:48.517000+0000","last_clean":"2023-12-01T01:52:48.517000+0000","last_became_active":"2023-12-01T01:52:48.513269+0000","last_became_peered":"2023-12-01T01:52:48.513269+0000","last_unstale":"2023-12-01T01:52:48.517000+0000","last_undegraded":"2023-12-01T01:52:48.517000+0000","last_fullsized":"2023-12-01T01:52:48.517000+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525839+0000","last_change":"2023-12-01T01:52:48.522640+0000","last_active":"2023-12-01T01:52:48.525839+0000","last_peered":"2023-12-01T01:52:48.525839+0000","last_clean":"2023-12-01T01:52:48.525839+0000","last_became_active":"2023-12-01T01:52:48.521392+0000","last_became_peered":"2023-12-01T01:52:48.521392+0000","last_unstale":"2023-12-01T01:52:48.525839+0000","last_undegraded":"2023-12-01T01:52:48.525839+0000","last_fullsized":"2023-12-01T01:52:48.525839+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.530244+0000","last_change":"2023-12-01T01:52:48.521211+0000","last_active":"2023-12-01T01:52:48.530244+0000","last_peered":"2023-12-01T01:52:48.530244+0000","last_clean":"2023-12-01T01:52:48.530244+0000","last_became_active":"2023-12-01T01:52:48.520691+0000","last_became_peered":"2023-12-01T01:52:48.520691+0000","last_unstale":"2023-12-01T01:52:48.530244+0000","last_undegraded":"2023-12-01T01:52:48.530244+0000","last_fullsized":"2023-12-01T01:52:48.530244+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.528579+0000","last_change":"2023-12-01T01:52:48.517997+0000","last_active":"2023-12-01T01:52:48.528579+0000","last_peered":"2023-12-01T01:52:48.528579+0000","last_clean":"2023-12-01T01:52:48.528579+0000","last_became_active":"2023-12-01T01:52:48.516730+0000","last_became_peered":"2023-12-01T01:52:48.516730+0000","last_unstale":"2023-12-01T01:52:48.528579+0000","last_undegraded":"2023-12-01T01:52:48.528579+0000","last_fullsized":"2023-12-01T01:52:48.528579+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.514112+0000","last_change":"2023-12-01T01:52:48.511348+0000","last_active":"2023-12-01T01:52:48.514112+0000","last_peered":"2023-12-01T01:52:48.514112+0000","last_clean":"2023-12-01T01:52:48.514112+0000","last_became_active":"2023-12-01T01:52:48.510938+0000","last_became_peered":"2023-12-01T01:52:48.510938+0000","last_unstale":"2023-12-01T01:52:48.514112+0000","last_undegraded":"2023-12-01T01:52:48.514112+0000","last_fullsized":"2023-12-01T01:52:48.514112+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:22.154067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527586+0000","last_change":"2023-12-01T01:52:48.525606+0000","last_active":"2023-12-01T01:52:48.527586+0000","last_peered":"2023-12-01T01:52:48.527586+0000","last_clean":"2023-12-01T01:52:48.527586+0000","last_became_active":"2023-12-01T01:52:48.524366+0000","last_became_peered":"2023-12-01T01:52:48.524366+0000","last_unstale":"2023-12-01T01:52:48.527586+0000","last_undegraded":"2023-12-01T01:52:48.527586+0000","last_fullsized":"2023-12-01T01:52:48.527586+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:12:02.919659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.516130+0000","last_change":"2023-12-01T01:52:48.511789+0000","last_active":"2023-12-01T01:52:48.516130+0000","last_peered":"2023-12-01T01:52:48.516130+0000","last_clean":"2023-12-01T01:52:48.516130+0000","last_became_active":"2023-12-01T01:52:48.511449+0000","last_became_peered":"2023-12-01T01:52:48.511449+0000","last_unstale":"2023-12-01T01:52:48.516130+0000","last_undegraded":"2023-12-01T01:52:48.516130+0000","last_fullsized":"2023-12-01T01:52:48.516130+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:25:02.674823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525530+0000","last_change":"2023-12-01T01:52:48.521420+0000","last_active":"2023-12-01T01:52:48.525530+0000","last_peered":"2023-12-01T01:52:48.525530+0000","last_clean":"2023-12-01T01:52:48.525530+0000","last_became_active":"2023-12-01T01:52:48.519768+0000","last_became_peered":"2023-12-01T01:52:48.519768+0000","last_unstale":"2023-12-01T01:52:48.525530+0000","last_undegraded":"2023-12-01T01:52:48.525530+0000","last_fullsized":"2023-12-01T01:52:48.525530+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523402+0000","last_change":"2023-12-01T01:52:48.517145+0000","last_active":"2023-12-01T01:52:48.523402+0000","last_peered":"2023-12-01T01:52:48.523402+0000","last_clean":"2023-12-01T01:52:48.523402+0000","last_became_active":"2023-12-01T01:52:48.516469+0000","last_became_peered":"2023-12-01T01:52:48.516469+0000","last_unstale":"2023-12-01T01:52:48.523402+0000","last_undegraded":"2023-12-01T01:52:48.523402+0000","last_fullsized":"2023-12-01T01:52:48.523402+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:00.516734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533549+0000","last_change":"2023-12-01T01:52:48.527651+0000","last_active":"2023-12-01T01:52:48.533549+0000","last_peered":"2023-12-01T01:52:48.533549+0000","last_clean":"2023-12-01T01:52:48.533549+0000","last_became_active":"2023-12-01T01:52:48.527285+0000","last_became_peered":"2023-12-01T01:52:48.527285+0000","last_unstale":"2023-12-01T01:52:48.533549+0000","last_undegraded":"2023-12-01T01:52:48.533549+0000","last_fullsized":"2023-12-01T01:52:48.533549+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533783+0000","last_change":"2023-12-01T01:52:48.529942+0000","last_active":"2023-12-01T01:52:48.533783+0000","last_peered":"2023-12-01T01:52:48.533783+0000","last_clean":"2023-12-01T01:52:48.533783+0000","last_became_active":"2023-12-01T01:52:48.529681+0000","last_became_peered":"2023-12-01T01:52:48.529681+0000","last_unstale":"2023-12-01T01:52:48.533783+0000","last_undegraded":"2023-12-01T01:52:48.533783+0000","last_fullsized":"2023-12-01T01:52:48.533783+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.519637+0000","last_change":"2023-12-01T01:52:48.516192+0000","last_active":"2023-12-01T01:52:48.519637+0000","last_peered":"2023-12-01T01:52:48.519637+0000","last_clean":"2023-12-01T01:52:48.519637+0000","last_became_active":"2023-12-01T01:52:48.514199+0000","last_became_peered":"2023-12-01T01:52:48.514199+0000","last_unstale":"2023-12-01T01:52:48.519637+0000","last_undegraded":"2023-12-01T01:52:48.519637+0000","last_fullsized":"2023-12-01T01:52:48.519637+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:02:42.893820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"110'4","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.540248+0000","last_change":"2023-12-01T01:52:48.038820+0000","last_active":"2023-12-01T01:52:48.540248+0000","last_peered":"2023-12-01T01:52:48.540248+0000","last_clean":"2023-12-01T01:52:48.540248+0000","last_became_active":"2023-12-01T01:52:48.038539+0000","last_became_peered":"2023-12-01T01:52:48.038539+0000","last_unstale":"2023-12-01T01:52:48.540248+0000","last_undegraded":"2023-12-01T01:52:48.540248+0000","last_fullsized":"2023-12-01T01:52:48.540248+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:21.090681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525259+0000","last_change":"2023-12-01T01:52:48.518682+0000","last_active":"2023-12-01T01:52:48.525259+0000","last_peered":"2023-12-01T01:52:48.525259+0000","last_clean":"2023-12-01T01:52:48.525259+0000","last_became_active":"2023-12-01T01:52:48.518415+0000","last_became_peered":"2023-12-01T01:52:48.518415+0000","last_unstale":"2023-12-01T01:52:48.525259+0000","last_undegraded":"2023-12-01T01:52:48.525259+0000","last_fullsized":"2023-12-01T01:52:48.525259+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:01.801261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.534050+0000","last_change":"2023-12-01T01:52:48.049643+0000","last_active":"2023-12-01T01:52:48.534050+0000","last_peered":"2023-12-01T01:52:48.534050+0000","last_clean":"2023-12-01T01:52:48.534050+0000","last_became_active":"2023-12-01T01:52:48.049288+0000","last_became_peered":"2023-12-01T01:52:48.049288+0000","last_unstale":"2023-12-01T01:52:48.534050+0000","last_undegraded":"2023-12-01T01:52:48.534050+0000","last_fullsized":"2023-12-01T01:52:48.534050+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:41.937860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.526343+0000","last_change":"2023-12-01T01:52:48.521987+0000","last_active":"2023-12-01T01:52:48.526343+0000","last_peered":"2023-12-01T01:52:48.526343+0000","last_clean":"2023-12-01T01:52:48.526343+0000","last_became_active":"2023-12-01T01:52:48.521595+0000","last_became_peered":"2023-12-01T01:52:48.521595+0000","last_unstale":"2023-12-01T01:52:48.526343+0000","last_undegraded":"2023-12-01T01:52:48.526343+0000","last_fullsized":"2023-12-01T01:52:48.526343+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:50:17.533885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.536817+0000","last_change":"2023-12-01T01:52:48.039238+0000","last_active":"2023-12-01T01:52:48.536817+0000","last_peered":"2023-12-01T01:52:48.536817+0000","last_clean":"2023-12-01T01:52:48.536817+0000","last_became_active":"2023-12-01T01:52:48.038835+0000","last_became_peered":"2023-12-01T01:52:48.038835+0000","last_unstale":"2023-12-01T01:52:48.536817+0000","last_undegraded":"2023-12-01T01:52:48.536817+0000","last_fullsized":"2023-12-01T01:52:48.536817+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525630+0000","last_change":"2023-12-01T01:52:48.522088+0000","last_active":"2023-12-01T01:52:48.525630+0000","last_peered":"2023-12-01T01:52:48.525630+0000","last_clean":"2023-12-01T01:52:48.525630+0000","last_became_active":"2023-12-01T01:52:48.521834+0000","last_became_peered":"2023-12-01T01:52:48.521834+0000","last_unstale":"2023-12-01T01:52:48.525630+0000","last_undegraded":"2023-12-01T01:52:48.525630+0000","last_fullsized":"2023-12-01T01:52:48.525630+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.534961+0000","last_change":"2023-12-01T01:52:48.045716+0000","last_active":"2023-12-01T01:52:48.534961+0000","last_peered":"2023-12-01T01:52:48.534961+0000","last_clean":"2023-12-01T01:52:48.534961+0000","last_became_active":"2023-12-01T01:52:48.045176+0000","last_became_peered":"2023-12-01T01:52:48.045176+0000","last_unstale":"2023-12-01T01:52:48.534961+0000","last_undegraded":"2023-12-01T01:52:48.534961+0000","last_fullsized":"2023-12-01T01:52:48.534961+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:55:21.695057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523967+0000","last_change":"2023-12-01T01:52:48.519484+0000","last_active":"2023-12-01T01:52:48.523967+0000","last_peered":"2023-12-01T01:52:48.523967+0000","last_clean":"2023-12-01T01:52:48.523967+0000","last_became_active":"2023-12-01T01:52:48.519075+0000","last_became_peered":"2023-12-01T01:52:48.519075+0000","last_unstale":"2023-12-01T01:52:48.523967+0000","last_undegraded":"2023-12-01T01:52:48.523967+0000","last_fullsized":"2023-12-01T01:52:48.523967+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533011+0000","last_change":"2023-12-01T01:52:48.033877+0000","last_active":"2023-12-01T01:52:48.533011+0000","last_peered":"2023-12-01T01:52:48.533011+0000","last_clean":"2023-12-01T01:52:48.533011+0000","last_became_active":"2023-12-01T01:52:48.033605+0000","last_became_peered":"2023-12-01T01:52:48.033605+0000","last_unstale":"2023-12-01T01:52:48.533011+0000","last_undegraded":"2023-12-01T01:52:48.533011+0000","last_fullsized":"2023-12-01T01:52:48.533011+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:55.538925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527179+0000","last_change":"2023-12-01T01:52:48.525907+0000","last_active":"2023-12-01T01:52:48.527179+0000","last_peered":"2023-12-01T01:52:48.527179+0000","last_clean":"2023-12-01T01:52:48.527179+0000","last_became_active":"2023-12-01T01:52:48.525555+0000","last_became_peered":"2023-12-01T01:52:48.525555+0000","last_unstale":"2023-12-01T01:52:48.527179+0000","last_undegraded":"2023-12-01T01:52:48.527179+0000","last_fullsized":"2023-12-01T01:52:48.527179+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:53:05.715094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.531149+0000","last_change":"2023-12-01T01:52:48.043431+0000","last_active":"2023-12-01T01:52:48.531149+0000","last_peered":"2023-12-01T01:52:48.531149+0000","last_clean":"2023-12-01T01:52:48.531149+0000","last_became_active":"2023-12-01T01:52:48.043162+0000","last_became_peered":"2023-12-01T01:52:48.043162+0000","last_unstale":"2023-12-01T01:52:48.531149+0000","last_undegraded":"2023-12-01T01:52:48.531149+0000","last_fullsized":"2023-12-01T01:52:48.531149+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523481+0000","last_change":"2023-12-01T01:52:48.521106+0000","last_active":"2023-12-01T01:52:48.523481+0000","last_peered":"2023-12-01T01:52:48.523481+0000","last_clean":"2023-12-01T01:52:48.523481+0000","last_became_active":"2023-12-01T01:52:48.520854+0000","last_became_peered":"2023-12-01T01:52:48.520854+0000","last_unstale":"2023-12-01T01:52:48.523481+0000","last_undegraded":"2023-12-01T01:52:48.523481+0000","last_fullsized":"2023-12-01T01:52:48.523481+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:10:51.868019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.503557+0000","last_change":"2023-12-01T01:52:48.046040+0000","last_active":"2023-12-01T01:52:48.503557+0000","last_peered":"2023-12-01T01:52:48.503557+0000","last_clean":"2023-12-01T01:52:48.503557+0000","last_became_active":"2023-12-01T01:52:48.045722+0000","last_became_peered":"2023-12-01T01:52:48.045722+0000","last_unstale":"2023-12-01T01:52:48.503557+0000","last_undegraded":"2023-12-01T01:52:48.503557+0000","last_fullsized":"2023-12-01T01:52:48.503557+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:25:01.670535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525082+0000","last_change":"2023-12-01T01:52:48.515054+0000","last_active":"2023-12-01T01:52:48.525082+0000","last_peered":"2023-12-01T01:52:48.525082+0000","last_clean":"2023-12-01T01:52:48.525082+0000","last_became_active":"2023-12-01T01:52:48.514726+0000","last_became_peered":"2023-12-01T01:52:48.514726+0000","last_unstale":"2023-12-01T01:52:48.525082+0000","last_undegraded":"2023-12-01T01:52:48.525082+0000","last_fullsized":"2023-12-01T01:52:48.525082+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:38.690885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.502994+0000","last_change":"2023-12-01T01:52:48.049260+0000","last_active":"2023-12-01T01:52:48.502994+0000","last_peered":"2023-12-01T01:52:48.502994+0000","last_clean":"2023-12-01T01:52:48.502994+0000","last_became_active":"2023-12-01T01:52:48.048998+0000","last_became_peered":"2023-12-01T01:52:48.048998+0000","last_unstale":"2023-12-01T01:52:48.502994+0000","last_undegraded":"2023-12-01T01:52:48.502994+0000","last_fullsized":"2023-12-01T01:52:48.502994+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:06.546333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523078+0000","last_change":"2023-12-01T01:52:48.516163+0000","last_active":"2023-12-01T01:52:48.523078+0000","last_peered":"2023-12-01T01:52:48.523078+0000","last_clean":"2023-12-01T01:52:48.523078+0000","last_became_active":"2023-12-01T01:52:48.515895+0000","last_became_peered":"2023-12-01T01:52:48.515895+0000","last_unstale":"2023-12-01T01:52:48.523078+0000","last_undegraded":"2023-12-01T01:52:48.523078+0000","last_fullsized":"2023-12-01T01:52:48.523078+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:12.516160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.505335+0000","last_change":"2023-12-01T01:52:48.035560+0000","last_active":"2023-12-01T01:52:48.505335+0000","last_peered":"2023-12-01T01:52:48.505335+0000","last_clean":"2023-12-01T01:52:48.505335+0000","last_became_active":"2023-12-01T01:52:48.035303+0000","last_became_peered":"2023-12-01T01:52:48.035303+0000","last_unstale":"2023-12-01T01:52:48.505335+0000","last_undegraded":"2023-12-01T01:52:48.505335+0000","last_fullsized":"2023-12-01T01:52:48.505335+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:04.573692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525296+0000","last_change":"2023-12-01T01:52:48.519705+0000","last_active":"2023-12-01T01:52:48.525296+0000","last_peered":"2023-12-01T01:52:48.525296+0000","last_clean":"2023-12-01T01:52:48.525296+0000","last_became_active":"2023-12-01T01:52:48.519304+0000","last_became_peered":"2023-12-01T01:52:48.519304+0000","last_unstale":"2023-12-01T01:52:48.525296+0000","last_undegraded":"2023-12-01T01:52:48.525296+0000","last_fullsized":"2023-12-01T01:52:48.525296+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:22.643458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.513060+0000","last_change":"2023-12-01T01:52:48.040913+0000","last_active":"2023-12-01T01:52:48.513060+0000","last_peered":"2023-12-01T01:52:48.513060+0000","last_clean":"2023-12-01T01:52:48.513060+0000","last_became_active":"2023-12-01T01:52:48.040402+0000","last_became_peered":"2023-12-01T01:52:48.040402+0000","last_unstale":"2023-12-01T01:52:48.513060+0000","last_undegraded":"2023-12-01T01:52:48.513060+0000","last_fullsized":"2023-12-01T01:52:48.513060+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.529300+0000","last_change":"2023-12-01T01:52:48.520445+0000","last_active":"2023-12-01T01:52:48.529300+0000","last_peered":"2023-12-01T01:52:48.529300+0000","last_clean":"2023-12-01T01:52:48.529300+0000","last_became_active":"2023-12-01T01:52:48.520113+0000","last_became_peered":"2023-12-01T01:52:48.520113+0000","last_unstale":"2023-12-01T01:52:48.529300+0000","last_undegraded":"2023-12-01T01:52:48.529300+0000","last_fullsized":"2023-12-01T01:52:48.529300+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"110'3","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.535325+0000","last_change":"2023-12-01T01:52:48.045583+0000","last_active":"2023-12-01T01:52:48.535325+0000","last_peered":"2023-12-01T01:52:48.535325+0000","last_clean":"2023-12-01T01:52:48.535325+0000","last_became_active":"2023-12-01T01:52:48.045146+0000","last_became_peered":"2023-12-01T01:52:48.045146+0000","last_unstale":"2023-12-01T01:52:48.535325+0000","last_undegraded":"2023-12-01T01:52:48.535325+0000","last_fullsized":"2023-12-01T01:52:48.535325+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:25.868578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527568+0000","last_change":"2023-12-01T01:52:48.524728+0000","last_active":"2023-12-01T01:52:48.527568+0000","last_peered":"2023-12-01T01:52:48.527568+0000","last_clean":"2023-12-01T01:52:48.527568+0000","last_became_active":"2023-12-01T01:52:48.524228+0000","last_became_peered":"2023-12-01T01:52:48.524228+0000","last_unstale":"2023-12-01T01:52:48.527568+0000","last_undegraded":"2023-12-01T01:52:48.527568+0000","last_fullsized":"2023-12-01T01:52:48.527568+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:03.971637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"110'6","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.541971+0000","last_change":"2023-12-01T01:52:48.042396+0000","last_active":"2023-12-01T01:52:48.541971+0000","last_peered":"2023-12-01T01:52:48.541971+0000","last_clean":"2023-12-01T01:52:48.541971+0000","last_became_active":"2023-12-01T01:52:48.041958+0000","last_became_peered":"2023-12-01T01:52:48.041958+0000","last_unstale":"2023-12-01T01:52:48.541971+0000","last_undegraded":"2023-12-01T01:52:48.541971+0000","last_fullsized":"2023-12-01T01:52:48.541971+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:33.265613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532665+0000","last_change":"2023-12-01T01:52:48.523304+0000","last_active":"2023-12-01T01:52:48.532665+0000","last_peered":"2023-12-01T01:52:48.532665+0000","last_clean":"2023-12-01T01:52:48.532665+0000","last_became_active":"2023-12-01T01:52:48.523050+0000","last_became_peered":"2023-12-01T01:52:48.523050+0000","last_unstale":"2023-12-01T01:52:48.532665+0000","last_undegraded":"2023-12-01T01:52:48.532665+0000","last_fullsized":"2023-12-01T01:52:48.532665+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.539919+0000","last_change":"2023-12-01T01:52:48.035637+0000","last_active":"2023-12-01T01:52:48.539919+0000","last_peered":"2023-12-01T01:52:48.539919+0000","last_clean":"2023-12-01T01:52:48.539919+0000","last_became_active":"2023-12-01T01:52:48.034955+0000","last_became_peered":"2023-12-01T01:52:48.034955+0000","last_unstale":"2023-12-01T01:52:48.539919+0000","last_undegraded":"2023-12-01T01:52:48.539919+0000","last_fullsized":"2023-12-01T01:52:48.539919+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:02.433217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.530823+0000","last_change":"2023-12-01T01:52:48.527142+0000","last_active":"2023-12-01T01:52:48.530823+0000","last_peered":"2023-12-01T01:52:48.530823+0000","last_clean":"2023-12-01T01:52:48.530823+0000","last_became_active":"2023-12-01T01:52:48.526832+0000","last_became_peered":"2023-12-01T01:52:48.526832+0000","last_unstale":"2023-12-01T01:52:48.530823+0000","last_undegraded":"2023-12-01T01:52:48.530823+0000","last_fullsized":"2023-12-01T01:52:48.530823+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:58:00.440400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.504235+0000","last_change":"2023-12-01T01:52:48.033331+0000","last_active":"2023-12-01T01:52:48.504235+0000","last_peered":"2023-12-01T01:52:48.504235+0000","last_clean":"2023-12-01T01:52:48.504235+0000","last_became_active":"2023-12-01T01:52:48.032852+0000","last_became_peered":"2023-12-01T01:52:48.032852+0000","last_unstale":"2023-12-01T01:52:48.504235+0000","last_undegraded":"2023-12-01T01:52:48.504235+0000","last_fullsized":"2023-12-01T01:52:48.504235+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:37.686598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523961+0000","last_change":"2023-12-01T01:52:48.521021+0000","last_active":"2023-12-01T01:52:48.523961+0000","last_peered":"2023-12-01T01:52:48.523961+0000","last_clean":"2023-12-01T01:52:48.523961+0000","last_became_active":"2023-12-01T01:52:48.520767+0000","last_became_peered":"2023-12-01T01:52:48.520767+0000","last_unstale":"2023-12-01T01:52:48.523961+0000","last_undegraded":"2023-12-01T01:52:48.523961+0000","last_fullsized":"2023-12-01T01:52:48.523961+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:45.833443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.511826+0000","last_change":"2023-12-01T01:52:48.041648+0000","last_active":"2023-12-01T01:52:48.511826+0000","last_peered":"2023-12-01T01:52:48.511826+0000","last_clean":"2023-12-01T01:52:48.511826+0000","last_became_active":"2023-12-01T01:52:48.040913+0000","last_became_peered":"2023-12-01T01:52:48.040913+0000","last_unstale":"2023-12-01T01:52:48.511826+0000","last_undegraded":"2023-12-01T01:52:48.511826+0000","last_fullsized":"2023-12-01T01:52:48.511826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:37.686598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.528672+0000","last_change":"2023-12-01T01:52:48.520423+0000","last_active":"2023-12-01T01:52:48.528672+0000","last_peered":"2023-12-01T01:52:48.528672+0000","last_clean":"2023-12-01T01:52:48.528672+0000","last_became_active":"2023-12-01T01:52:48.518361+0000","last_became_peered":"2023-12-01T01:52:48.518361+0000","last_unstale":"2023-12-01T01:52:48.528672+0000","last_undegraded":"2023-12-01T01:52:48.528672+0000","last_fullsized":"2023-12-01T01:52:48.528672+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:56.543213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"110'8","reported_seq":19,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.541959+0000","last_change":"2023-12-01T01:52:48.032822+0000","last_active":"2023-12-01T01:52:48.541959+0000","last_peered":"2023-12-01T01:52:48.541959+0000","last_clean":"2023-12-01T01:52:48.541959+0000","last_became_active":"2023-12-01T01:52:48.032368+0000","last_became_peered":"2023-12-01T01:52:48.032368+0000","last_unstale":"2023-12-01T01:52:48.541959+0000","last_undegraded":"2023-12-01T01:52:48.541959+0000","last_fullsized":"2023-12-01T01:52:48.541959+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:10:50.863731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.516793+0000","last_change":"2023-12-01T01:52:48.512359+0000","last_active":"2023-12-01T01:52:48.516793+0000","last_peered":"2023-12-01T01:52:48.516793+0000","last_clean":"2023-12-01T01:52:48.516793+0000","last_became_active":"2023-12-01T01:52:48.511971+0000","last_became_peered":"2023-12-01T01:52:48.511971+0000","last_unstale":"2023-12-01T01:52:48.516793+0000","last_undegraded":"2023-12-01T01:52:48.516793+0000","last_fullsized":"2023-12-01T01:52:48.516793+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:09:36.618385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.535408+0000","last_change":"2023-12-01T01:52:48.040258+0000","last_active":"2023-12-01T01:52:48.535408+0000","last_peered":"2023-12-01T01:52:48.535408+0000","last_clean":"2023-12-01T01:52:48.535408+0000","last_became_active":"2023-12-01T01:52:48.037376+0000","last_became_peered":"2023-12-01T01:52:48.037376+0000","last_unstale":"2023-12-01T01:52:48.535408+0000","last_undegraded":"2023-12-01T01:52:48.535408+0000","last_fullsized":"2023-12-01T01:52:48.535408+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:13.111537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533742+0000","last_change":"2023-12-01T01:52:48.529537+0000","last_active":"2023-12-01T01:52:48.533742+0000","last_peered":"2023-12-01T01:52:48.533742+0000","last_clean":"2023-12-01T01:52:48.533742+0000","last_became_active":"2023-12-01T01:52:48.529283+0000","last_became_peered":"2023-12-01T01:52:48.529283+0000","last_unstale":"2023-12-01T01:52:48.533742+0000","last_undegraded":"2023-12-01T01:52:48.533742+0000","last_fullsized":"2023-12-01T01:52:48.533742+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.539256+0000","last_change":"2023-12-01T01:52:48.050044+0000","last_active":"2023-12-01T01:52:48.539256+0000","last_peered":"2023-12-01T01:52:48.539256+0000","last_clean":"2023-12-01T01:52:48.539256+0000","last_became_active":"2023-12-01T01:52:48.049779+0000","last_became_peered":"2023-12-01T01:52:48.049779+0000","last_unstale":"2023-12-01T01:52:48.539256+0000","last_undegraded":"2023-12-01T01:52:48.539256+0000","last_fullsized":"2023-12-01T01:52:48.539256+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:20:19.462886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.519394+0000","last_change":"2023-12-01T01:52:48.515405+0000","last_active":"2023-12-01T01:52:48.519394+0000","last_peered":"2023-12-01T01:52:48.519394+0000","last_clean":"2023-12-01T01:52:48.519394+0000","last_became_active":"2023-12-01T01:52:48.515166+0000","last_became_peered":"2023-12-01T01:52:48.515166+0000","last_unstale":"2023-12-01T01:52:48.519394+0000","last_undegraded":"2023-12-01T01:52:48.519394+0000","last_fullsized":"2023-12-01T01:52:48.519394+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:17.235352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.502669+0000","last_change":"2023-12-01T01:52:48.048859+0000","last_active":"2023-12-01T01:52:48.502669+0000","last_peered":"2023-12-01T01:52:48.502669+0000","last_clean":"2023-12-01T01:52:48.502669+0000","last_became_active":"2023-12-01T01:52:48.048546+0000","last_became_peered":"2023-12-01T01:52:48.048546+0000","last_unstale":"2023-12-01T01:52:48.502669+0000","last_undegraded":"2023-12-01T01:52:48.502669+0000","last_fullsized":"2023-12-01T01:52:48.502669+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:43.307966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.531262+0000","last_change":"2023-12-01T01:52:48.527737+0000","last_active":"2023-12-01T01:52:48.531262+0000","last_peered":"2023-12-01T01:52:48.531262+0000","last_clean":"2023-12-01T01:52:48.531262+0000","last_became_active":"2023-12-01T01:52:48.527227+0000","last_became_peered":"2023-12-01T01:52:48.527227+0000","last_unstale":"2023-12-01T01:52:48.531262+0000","last_undegraded":"2023-12-01T01:52:48.531262+0000","last_fullsized":"2023-12-01T01:52:48.531262+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.539606+0000","last_change":"2023-12-01T01:52:48.046685+0000","last_active":"2023-12-01T01:52:48.539606+0000","last_peered":"2023-12-01T01:52:48.539606+0000","last_clean":"2023-12-01T01:52:48.539606+0000","last_became_active":"2023-12-01T01:52:48.046395+0000","last_became_peered":"2023-12-01T01:52:48.046395+0000","last_unstale":"2023-12-01T01:52:48.539606+0000","last_undegraded":"2023-12-01T01:52:48.539606+0000","last_fullsized":"2023-12-01T01:52:48.539606+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:29:10.130713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533217+0000","last_change":"2023-12-01T01:52:48.531261+0000","last_active":"2023-12-01T01:52:48.533217+0000","last_peered":"2023-12-01T01:52:48.533217+0000","last_clean":"2023-12-01T01:52:48.533217+0000","last_became_active":"2023-12-01T01:52:48.529433+0000","last_became_peered":"2023-12-01T01:52:48.529433+0000","last_unstale":"2023-12-01T01:52:48.533217+0000","last_undegraded":"2023-12-01T01:52:48.533217+0000","last_fullsized":"2023-12-01T01:52:48.533217+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:52:45.545702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.504332+0000","last_change":"2023-12-01T01:52:48.038141+0000","last_active":"2023-12-01T01:52:48.504332+0000","last_peered":"2023-12-01T01:52:48.504332+0000","last_clean":"2023-12-01T01:52:48.504332+0000","last_became_active":"2023-12-01T01:52:48.037797+0000","last_became_peered":"2023-12-01T01:52:48.037797+0000","last_unstale":"2023-12-01T01:52:48.504332+0000","last_undegraded":"2023-12-01T01:52:48.504332+0000","last_fullsized":"2023-12-01T01:52:48.504332+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.524044+0000","last_change":"2023-12-01T01:52:48.519490+0000","last_active":"2023-12-01T01:52:48.524044+0000","last_peered":"2023-12-01T01:52:48.524044+0000","last_clean":"2023-12-01T01:52:48.524044+0000","last_became_active":"2023-12-01T01:52:48.519131+0000","last_became_peered":"2023-12-01T01:52:48.519131+0000","last_unstale":"2023-12-01T01:52:48.524044+0000","last_undegraded":"2023-12-01T01:52:48.524044+0000","last_fullsized":"2023-12-01T01:52:48.524044+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.509529+0000","last_change":"2023-12-01T01:52:48.042643+0000","last_active":"2023-12-01T01:52:48.509529+0000","last_peered":"2023-12-01T01:52:48.509529+0000","last_clean":"2023-12-01T01:52:48.509529+0000","last_became_active":"2023-12-01T01:52:48.042174+0000","last_became_peered":"2023-12-01T01:52:48.042174+0000","last_unstale":"2023-12-01T01:52:48.509529+0000","last_undegraded":"2023-12-01T01:52:48.509529+0000","last_fullsized":"2023-12-01T01:52:48.509529+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:59.436113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.522735+0000","last_change":"2023-12-01T01:52:48.520519+0000","last_active":"2023-12-01T01:52:48.522735+0000","last_peered":"2023-12-01T01:52:48.522735+0000","last_clean":"2023-12-01T01:52:48.522735+0000","last_became_active":"2023-12-01T01:52:48.520259+0000","last_became_peered":"2023-12-01T01:52:48.520259+0000","last_unstale":"2023-12-01T01:52:48.522735+0000","last_undegraded":"2023-12-01T01:52:48.522735+0000","last_fullsized":"2023-12-01T01:52:48.522735+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.502961+0000","last_change":"2023-12-01T01:52:48.047235+0000","last_active":"2023-12-01T01:52:48.502961+0000","last_peered":"2023-12-01T01:52:48.502961+0000","last_clean":"2023-12-01T01:52:48.502961+0000","last_became_active":"2023-12-01T01:52:48.046953+0000","last_became_peered":"2023-12-01T01:52:48.046953+0000","last_unstale":"2023-12-01T01:52:48.502961+0000","last_undegraded":"2023-12-01T01:52:48.502961+0000","last_fullsized":"2023-12-01T01:52:48.502961+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:13.492174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.513572+0000","last_change":"2023-12-01T01:52:48.509005+0000","last_active":"2023-12-01T01:52:48.513572+0000","last_peered":"2023-12-01T01:52:48.513572+0000","last_clean":"2023-12-01T01:52:48.513572+0000","last_became_active":"2023-12-01T01:52:48.508718+0000","last_became_peered":"2023-12-01T01:52:48.508718+0000","last_unstale":"2023-12-01T01:52:48.513572+0000","last_undegraded":"2023-12-01T01:52:48.513572+0000","last_fullsized":"2023-12-01T01:52:48.513572+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:39.608737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.535636+0000","last_change":"2023-12-01T01:52:48.050705+0000","last_active":"2023-12-01T01:52:48.535636+0000","last_peered":"2023-12-01T01:52:48.535636+0000","last_clean":"2023-12-01T01:52:48.535636+0000","last_became_active":"2023-12-01T01:52:48.050445+0000","last_became_peered":"2023-12-01T01:52:48.050445+0000","last_unstale":"2023-12-01T01:52:48.535636+0000","last_undegraded":"2023-12-01T01:52:48.535636+0000","last_fullsized":"2023-12-01T01:52:48.535636+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:02:56.065815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523934+0000","last_change":"2023-12-01T01:52:48.519761+0000","last_active":"2023-12-01T01:52:48.523934+0000","last_peered":"2023-12-01T01:52:48.523934+0000","last_clean":"2023-12-01T01:52:48.523934+0000","last_became_active":"2023-12-01T01:52:48.517974+0000","last_became_peered":"2023-12-01T01:52:48.517974+0000","last_unstale":"2023-12-01T01:52:48.523934+0000","last_undegraded":"2023-12-01T01:52:48.523934+0000","last_fullsized":"2023-12-01T01:52:48.523934+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:55:14.304687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.526121+0000","last_change":"2023-12-01T01:52:48.040799+0000","last_active":"2023-12-01T01:52:48.526121+0000","last_peered":"2023-12-01T01:52:48.526121+0000","last_clean":"2023-12-01T01:52:48.526121+0000","last_became_active":"2023-12-01T01:52:48.040471+0000","last_became_peered":"2023-12-01T01:52:48.040471+0000","last_unstale":"2023-12-01T01:52:48.526121+0000","last_undegraded":"2023-12-01T01:52:48.526121+0000","last_fullsized":"2023-12-01T01:52:48.526121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.517427+0000","last_change":"2023-12-01T01:52:48.515269+0000","last_active":"2023-12-01T01:52:48.517427+0000","last_peered":"2023-12-01T01:52:48.517427+0000","last_clean":"2023-12-01T01:52:48.517427+0000","last_became_active":"2023-12-01T01:52:48.514457+0000","last_became_peered":"2023-12-01T01:52:48.514457+0000","last_unstale":"2023-12-01T01:52:48.517427+0000","last_undegraded":"2023-12-01T01:52:48.517427+0000","last_fullsized":"2023-12-01T01:52:48.517427+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:35:01.791105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.510349+0000","last_change":"2023-12-01T01:52:48.043497+0000","last_active":"2023-12-01T01:52:48.510349+0000","last_peered":"2023-12-01T01:52:48.510349+0000","last_clean":"2023-12-01T01:52:48.510349+0000","last_became_active":"2023-12-01T01:52:48.043198+0000","last_became_peered":"2023-12-01T01:52:48.043198+0000","last_unstale":"2023-12-01T01:52:48.510349+0000","last_undegraded":"2023-12-01T01:52:48.510349+0000","last_fullsized":"2023-12-01T01:52:48.510349+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:44.930335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527813+0000","last_change":"2023-12-01T01:52:48.525650+0000","last_active":"2023-12-01T01:52:48.527813+0000","last_peered":"2023-12-01T01:52:48.527813+0000","last_clean":"2023-12-01T01:52:48.527813+0000","last_became_active":"2023-12-01T01:52:48.525192+0000","last_became_peered":"2023-12-01T01:52:48.525192+0000","last_unstale":"2023-12-01T01:52:48.527813+0000","last_undegraded":"2023-12-01T01:52:48.527813+0000","last_fullsized":"2023-12-01T01:52:48.527813+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:03.437505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.513774+0000","last_change":"2023-12-01T01:52:48.046182+0000","last_active":"2023-12-01T01:52:48.513774+0000","last_peered":"2023-12-01T01:52:48.513774+0000","last_clean":"2023-12-01T01:52:48.513774+0000","last_became_active":"2023-12-01T01:52:48.045696+0000","last_became_peered":"2023-12-01T01:52:48.045696+0000","last_unstale":"2023-12-01T01:52:48.513774+0000","last_undegraded":"2023-12-01T01:52:48.513774+0000","last_fullsized":"2023-12-01T01:52:48.513774+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:44.829155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523516+0000","last_change":"2023-12-01T01:52:48.519955+0000","last_active":"2023-12-01T01:52:48.523516+0000","last_peered":"2023-12-01T01:52:48.523516+0000","last_clean":"2023-12-01T01:52:48.523516+0000","last_became_active":"2023-12-01T01:52:48.519705+0000","last_became_peered":"2023-12-01T01:52:48.519705+0000","last_unstale":"2023-12-01T01:52:48.523516+0000","last_undegraded":"2023-12-01T01:52:48.523516+0000","last_fullsized":"2023-12-01T01:52:48.523516+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:34:49.188317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.505474+0000","last_change":"2023-12-01T01:52:48.042910+0000","last_active":"2023-12-01T01:52:48.505474+0000","last_peered":"2023-12-01T01:52:48.505474+0000","last_clean":"2023-12-01T01:52:48.505474+0000","last_became_active":"2023-12-01T01:52:48.041897+0000","last_became_peered":"2023-12-01T01:52:48.041897+0000","last_unstale":"2023-12-01T01:52:48.505474+0000","last_undegraded":"2023-12-01T01:52:48.505474+0000","last_fullsized":"2023-12-01T01:52:48.505474+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532905+0000","last_change":"2023-12-01T01:52:48.525217+0000","last_active":"2023-12-01T01:52:48.532905+0000","last_peered":"2023-12-01T01:52:48.532905+0000","last_clean":"2023-12-01T01:52:48.532905+0000","last_became_active":"2023-12-01T01:52:48.524951+0000","last_became_peered":"2023-12-01T01:52:48.524951+0000","last_unstale":"2023-12-01T01:52:48.532905+0000","last_undegraded":"2023-12-01T01:52:48.532905+0000","last_fullsized":"2023-12-01T01:52:48.532905+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:45.934623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.515888+0000","last_change":"2023-12-01T01:52:48.044493+0000","last_active":"2023-12-01T01:52:48.515888+0000","last_peered":"2023-12-01T01:52:48.515888+0000","last_clean":"2023-12-01T01:52:48.515888+0000","last_became_active":"2023-12-01T01:52:48.044221+0000","last_became_peered":"2023-12-01T01:52:48.044221+0000","last_unstale":"2023-12-01T01:52:48.515888+0000","last_undegraded":"2023-12-01T01:52:48.515888+0000","last_fullsized":"2023-12-01T01:52:48.515888+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:59.436113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523313+0000","last_change":"2023-12-01T01:52:48.517859+0000","last_active":"2023-12-01T01:52:48.523313+0000","last_peered":"2023-12-01T01:52:48.523313+0000","last_clean":"2023-12-01T01:52:48.523313+0000","last_became_active":"2023-12-01T01:52:48.517615+0000","last_became_peered":"2023-12-01T01:52:48.517615+0000","last_unstale":"2023-12-01T01:52:48.523313+0000","last_undegraded":"2023-12-01T01:52:48.523313+0000","last_fullsized":"2023-12-01T01:52:48.523313+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:20:23.997177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.504533+0000","last_change":"2023-12-01T01:52:48.045085+0000","last_active":"2023-12-01T01:52:48.504533+0000","last_peered":"2023-12-01T01:52:48.504533+0000","last_clean":"2023-12-01T01:52:48.504533+0000","last_became_active":"2023-12-01T01:52:48.044789+0000","last_became_peered":"2023-12-01T01:52:48.044789+0000","last_unstale":"2023-12-01T01:52:48.504533+0000","last_undegraded":"2023-12-01T01:52:48.504533+0000","last_fullsized":"2023-12-01T01:52:48.504533+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:58:00.466321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.522830+0000","last_change":"2023-12-01T01:52:48.513454+0000","last_active":"2023-12-01T01:52:48.522830+0000","last_peered":"2023-12-01T01:52:48.522830+0000","last_clean":"2023-12-01T01:52:48.522830+0000","last_became_active":"2023-12-01T01:52:48.513192+0000","last_became_peered":"2023-12-01T01:52:48.513192+0000","last_unstale":"2023-12-01T01:52:48.522830+0000","last_undegraded":"2023-12-01T01:52:48.522830+0000","last_fullsized":"2023-12-01T01:52:48.522830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:02:57.070103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.503936+0000","last_change":"2023-12-01T01:52:48.034635+0000","last_active":"2023-12-01T01:52:48.503936+0000","last_peered":"2023-12-01T01:52:48.503936+0000","last_clean":"2023-12-01T01:52:48.503936+0000","last_became_active":"2023-12-01T01:52:48.034401+0000","last_became_peered":"2023-12-01T01:52:48.034401+0000","last_unstale":"2023-12-01T01:52:48.503936+0000","last_undegraded":"2023-12-01T01:52:48.503936+0000","last_fullsized":"2023-12-01T01:52:48.503936+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:19.195130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.529281+0000","last_change":"2023-12-01T01:52:48.519960+0000","last_active":"2023-12-01T01:52:48.529281+0000","last_peered":"2023-12-01T01:52:48.529281+0000","last_clean":"2023-12-01T01:52:48.529281+0000","last_became_active":"2023-12-01T01:52:48.518314+0000","last_became_peered":"2023-12-01T01:52:48.518314+0000","last_unstale":"2023-12-01T01:52:48.529281+0000","last_undegraded":"2023-12-01T01:52:48.529281+0000","last_fullsized":"2023-12-01T01:52:48.529281+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.536821+0000","last_change":"2023-12-01T01:52:48.043879+0000","last_active":"2023-12-01T01:52:48.536821+0000","last_peered":"2023-12-01T01:52:48.536821+0000","last_clean":"2023-12-01T01:52:48.536821+0000","last_became_active":"2023-12-01T01:52:48.043357+0000","last_became_peered":"2023-12-01T01:52:48.043357+0000","last_unstale":"2023-12-01T01:52:48.536821+0000","last_undegraded":"2023-12-01T01:52:48.536821+0000","last_fullsized":"2023-12-01T01:52:48.536821+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:59.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'183","reported_seq":483,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.501694+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:52:48.501694+0000","last_peered":"2023-12-01T01:52:48.501694+0000","last_clean":"2023-12-01T01:52:48.501694+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:52:48.501694+0000","last_undegraded":"2023-12-01T01:52:48.501694+0000","last_fullsized":"2023-12-01T01:52:48.501694+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575052,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28288,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343552,"statfs":{"total":96636764160,"available":96607797248,"internally_reserved":0,"allocated":1368064,"data_stored":648917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.02,"15min":1.0660000000000001},"min":{"1min":0.54900000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.401,"5min":1.514,"15min":1.8089999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0309999999999999,"15min":1.0469999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5289999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.90300000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1499999999999999,"15min":1.2070000000000001},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.5669999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2150000000000001,"15min":1.248},"min":{"1min":0.66800000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6279999999999999,"5min":1.744,"15min":1.744},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1040000000000001,"15min":1.117},"min":{"1min":0.749,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4910000000000001,"5min":1.607,"15min":1.728},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.2,"5min":1.2010000000000001,"15min":1.2190000000000001},"min":{"1min":0.92500000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.641,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.0700000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.1910000000000001,"15min":1.2230000000000001},"min":{"1min":0.64200000000000002,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.347,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":1.246},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3180000000000001,"15min":1.3420000000000001},"min":{"1min":0.63700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7490000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.875}]},{"osd":4,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.3009999999999999,"15min":1.2849999999999999},"min":{"1min":0.72699999999999998,"5min":0.56100000000000005,"15min":0.55500000000000005},"max":{"1min":1.4450000000000001,"5min":1.7549999999999999,"15min":2.2130000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4350000000000001,"15min":1.4059999999999999},"min":{"1min":0.77500000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8100000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3129999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.292,"15min":1.258},"min":{"1min":0.625,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.0379999999999998,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.351},{"interface":"front","average":{"1min":1.47,"5min":1.514,"15min":1.514},"min":{"1min":0.69399999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.861,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4339999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.395,"15min":1.377},"min":{"1min":0.753,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1549999999999998,"5min":2.294,"15min":2.3420000000000001},"last":1.139},{"interface":"front","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.387},"min":{"1min":0.82599999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.893,"5min":1.893,"15min":2.3500000000000001},"last":1.4590000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575053,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1372160,"data_stored":649519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.1020000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.79},"last":1.204},{"interface":"front","average":{"1min":1.036,"5min":1.105,"15min":1.1220000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5349999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.49}]},{"osd":1,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.161,"15min":1.159},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1950000000000001,"15min":1.2050000000000001},"min":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0920000000000001},"last":1.3460000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1859999999999999,"15min":1.171},"min":{"1min":0.69699999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5760000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.242,"15min":1.2350000000000001},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":3.024,"5min":3.024,"15min":3.024},"last":1.599}]},{"osd":3,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":1.0289999999999999,"5min":0.93700000000000006,"15min":0.82399999999999995},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.4419999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3,"15min":1.2709999999999999},"min":{"1min":0.94299999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6819999999999999,"5min":2.226,"15min":2.226},"last":1.016},{"interface":"front","average":{"1min":1.339,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":1.028,"5min":0.91400000000000003,"15min":0.78100000000000003},"max":{"1min":1.885,"5min":1.931,"15min":2.2839999999999998},"last":1.726}]},{"osd":5,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.415,"15min":1.397},"min":{"1min":0.755,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.74,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.7529999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.502,"15min":1.472},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.84099999999999997},"max":{"1min":1.9790000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.669}]},{"osd":7,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.524,"15min":1.5009999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6870000000000001,"5min":2.762,"15min":2.762},"last":1.923},{"interface":"front","average":{"1min":1.488,"5min":1.577,"15min":1.5629999999999999},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":1.9330000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.55}]}]},{"osd":5,"up_from":13,"seq":55834575052,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1928,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1974272,"data_stored":1240813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1259999999999999,"15min":1.1040000000000001},"min":{"1min":0.63200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.2010000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1180000000000001,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.60099999999999998},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":2.153},"last":1.2509999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.117,"15min":1.1399999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.63200000000000001},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.171},{"interface":"front","average":{"1min":1.105,"5min":1.1399999999999999,"15min":1.159},"min":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.6519999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.3480000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.173,"15min":1.196},"min":{"1min":0.65000000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.702,"5min":1.702,"15min":1.7110000000000001},"last":1.004},{"interface":"front","average":{"1min":1.298,"5min":1.208,"15min":1.2330000000000001},"min":{"1min":0.79100000000000004,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":2.0630000000000002},"last":1.069}]},{"osd":3,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.248,"15min":1.282},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.8169999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.302},"min":{"1min":1.159,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.302}]},{"osd":4,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2410000000000001},"min":{"1min":0.83499999999999996,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.319,"15min":1.3480000000000001},"min":{"1min":0.84399999999999997,"5min":0.80300000000000005,"15min":0.79900000000000004},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.3300000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3129999999999999,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.859,"5min":1.885,"15min":1.984},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.3939999999999999,"15min":1.3839999999999999},"min":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.6140000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.462,"15min":1.4750000000000001},"min":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0920000000000001},"last":1.9319999999999999},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.522,"15min":1.5269999999999999},"min":{"1min":1.304,"5min":0.873,"15min":0.873},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.79}]}]},{"osd":4,"up_from":13,"seq":55834575052,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1368064,"data_stored":648881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1120000000000001,"15min":1.105},"min":{"1min":0.72199999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.753,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.7130000000000001,"5min":1.7529999999999999,"15min":2.1099999999999999},"last":1.28}]},{"osd":1,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.216,"15min":1.204},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.65900000000000003},"max":{"1min":1.5409999999999999,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.087,"5min":1.101,"15min":1.129},"min":{"1min":0.76600000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3979999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.93799999999999994}]},{"osd":2,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.234,"15min":1.2},"min":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.716,"5min":1.871,"15min":1.871},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.3180000000000001,"15min":1.288},"min":{"1min":0.94599999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.581,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.258,"15min":1.2310000000000001},"min":{"1min":0.81899999999999995,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.6140000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3200000000000001,"15min":1.3160000000000001},"min":{"1min":0.88500000000000001,"5min":0.753,"15min":0.753},"max":{"1min":1.712,"5min":1.7270000000000001,"15min":1.903},"last":0.88500000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.319,"15min":1.3129999999999999},"min":{"1min":0.94899999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.288,"15min":1.2869999999999999},"min":{"1min":0.97199999999999998,"5min":0.90300000000000002,"15min":0.85399999999999998},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.151}]},{"osd":6,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.381,"15min":1.3500000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.81299999999999994},"max":{"1min":1.8799999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.502,"15min":1.472},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.0960000000000001,"5min":2.4990000000000001,"15min":2.54},"last":1.3600000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.472,"15min":1.4610000000000001},"min":{"1min":1.274,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.306},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5780000000000001,"15min":1.5620000000000001},"min":{"1min":1.28,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":1.96,"5min":2.04,"15min":2.157},"last":1.395}]}]},{"osd":3,"up_from":13,"seq":55834575051,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58632,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313208,"statfs":{"total":96636764160,"available":96576724992,"internally_reserved":0,"allocated":1376256,"data_stored":649521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.84199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.307},"max":{"1min":1.1899999999999999,"5min":1.3320000000000001,"15min":1.583},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.93500000000000005,"15min":0.97999999999999998},"min":{"1min":0.48099999999999998,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.86699999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.99199999999999999,"15min":1.056},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.38900000000000001},"max":{"1min":1.5669999999999999,"5min":1.571,"15min":2.0779999999999998},"last":0.754},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.89300000000000002,"15min":0.94799999999999995},"min":{"1min":0.48399999999999999,"5min":0.48199999999999998,"15min":0.42699999999999999},"max":{"1min":1.1339999999999999,"5min":1.55,"15min":1.6519999999999999},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.96699999999999997,"15min":1.0349999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3360000000000001,"5min":1.4810000000000001,"15min":1.5609999999999999},"last":1.079},{"interface":"front","average":{"1min":0.995,"5min":1.125,"15min":1.1819999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.024}]},{"osd":4,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1970000000000001,"15min":1.232},"min":{"1min":0.84999999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.524,"5min":1.5820000000000001,"15min":1.9319999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.198,"15min":1.232},"min":{"1min":0.79300000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.909,"5min":1.909,"15min":2.0419999999999998},"last":1.2050000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.228,"15min":1.2789999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.71,"5min":1.7669999999999999,"15min":2.0510000000000002},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2529999999999999,"15min":1.288},"min":{"1min":0.745,"5min":0.745,"15min":0.64200000000000002},"max":{"1min":1.6759999999999999,"5min":1.681,"15min":2.1880000000000002},"last":1.702}]},{"osd":6,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.89600000000000002,"5min":0.76300000000000001,"15min":0.68300000000000005},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.3700000000000001,"15min":1.405},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8260000000000001,"5min":1.9119999999999999,"15min":1.99},"last":1.758}]},{"osd":7,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.365,"15min":1.413},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.81499999999999995},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.1619999999999999},"last":1.359},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4019999999999999,"15min":1.4470000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.83099999999999996},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.2130000000000001},"last":1.4399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575051,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36476,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335364,"statfs":{"total":96636764160,"available":96599412736,"internally_reserved":0,"allocated":1363968,"data_stored":648847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":0.81699999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.5529999999999999},"last":0.748},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.98399999999999999,"15min":1},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95799999999999996,"15min":0.98699999999999999},"min":{"1min":0.54900000000000004,"5min":0.373,"15min":0.373},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.081},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93899999999999995,"15min":1.002},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5189999999999999,"5min":1.625,"15min":1.6479999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.98799999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.869,"5min":1.9950000000000001,"15min":2.024},"last":0.872},{"interface":"front","average":{"1min":1.111,"5min":1.202,"15min":1.258},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.964},"last":0.91800000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.3129999999999999},"min":{"1min":0.72199999999999998,"5min":0.62,"15min":0.62},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.357,"15min":1.353},"min":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.016,"5min":2.048,"15min":2.323},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2849999999999999,"15min":1.3180000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.579,"5min":1.78,"15min":2.2799999999999998},"last":1.014},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.90300000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.3680000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.448},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3440000000000001,"15min":1.381},"min":{"1min":0.80600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":2.1080000000000001},"last":1.075}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59204,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312636,"statfs":{"total":96636764160,"available":96576139264,"internally_reserved":0,"allocated":1961984,"data_stored":1239305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.83499999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.318},"max":{"1min":1.1639999999999999,"5min":1.341,"15min":1.5049999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.94999999999999996,"15min":1.0069999999999999},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.597,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85299999999999998,"15min":0.88100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.54},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0349999999999999,"15min":1.081},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.44800000000000001},"max":{"1min":1.472,"5min":1.7250000000000001,"15min":1.804},"last":1.089}]},{"osd":3,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.5},"max":{"1min":1.718,"5min":1.827,"15min":1.827},"last":1.5129999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1,"15min":1.038},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.39700000000000002},"max":{"1min":1.381,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.4570000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1579999999999999,"15min":1.173},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.65300000000000002},"max":{"1min":1.425,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.361},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.288,"15min":1.3240000000000001},"min":{"1min":0.85599999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.863},"last":1.3169999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":0.77700000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.76},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.244,"15min":1.27},"min":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":2.1619999999999999},"last":1.1910000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.294,"15min":1.3220000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.68400000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.861},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.306,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.742,"5min":1.7490000000000001,"15min":1.8839999999999999},"last":1.5820000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3720000000000001,"15min":1.417},"min":{"1min":0.91200000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.782,"5min":1.782,"15min":2.0609999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3859999999999999,"15min":1.399},"min":{"1min":0.80900000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.931,"5min":1.992,"15min":1.992},"last":1.278}]}]},{"osd":0,"up_from":13,"seq":55834575052,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1380352,"data_stored":650433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84099999999999997,"15min":0.85399999999999998},"min":{"1min":0.435,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.194,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89900000000000002,"15min":0.96499999999999997},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3859999999999999,"5min":1.544,"15min":1.6910000000000001},"last":0.71299999999999997}]},{"osd":2,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.94799999999999995,"15min":0.998},"min":{"1min":0.51500000000000001,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.431,"5min":2.157,"15min":2.157},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0089999999999999,"15min":1.0649999999999999},"min":{"1min":0.57399999999999995,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.3400000000000001,"5min":1.728,"15min":1.728},"last":1.238}]},{"osd":3,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":1.0149999999999999},"min":{"1min":0.623,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.954},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1080000000000001,"15min":1.1539999999999999},"min":{"1min":0.54200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4199999999999999,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.90400000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2589999999999999,"15min":1.3029999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5840000000000001,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1899999999999999,"15min":1.258},"min":{"1min":0.90200000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.55,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.02}]},{"osd":5,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.323,"15min":1.343},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.71899999999999997},"max":{"1min":1.841,"5min":2.351,"15min":2.351},"last":1.323},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2769999999999999,"15min":1.333},"min":{"1min":0.81599999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.546,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.083}]},{"osd":6,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2789999999999999,"15min":1.319},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7529999999999999,"5min":1.8680000000000001,"15min":2.157},"last":1.119},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.351,"15min":1.3839999999999999},"min":{"1min":0.94499999999999995,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.7809999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.2689999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.339,"15min":1.3979999999999999},"min":{"1min":0.94999999999999996,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.5700000000000001,"5min":1.798,"15min":2.4279999999999999},"last":1.302},{"interface":"front","average":{"1min":1.25,"5min":1.3320000000000001,"15min":1.3899999999999999},"min":{"1min":0.98999999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.667,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.0700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:52:57.990 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:52:58.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.215+0000 7fea47269640 1 Processor -- start 2023-12-01T01:52:58.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.215+0000 7fea47269640 1 -- start start 2023-12-01T01:52:58.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.216+0000 7fea47269640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fea4015e2f0 0x7fea400ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.216+0000 7fea47269640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea4006f2f0 0x7fea400caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.216+0000 7fea47269640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fea400adf10 0x7fea400c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.216+0000 7fea47269640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea400add10 con 0x7fea4015e2f0 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea47269640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea400c5760 con 0x7fea400adf10 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea47269640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4015e6d0 con 0x7fea4006f2f0 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea45005640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fea4015e2f0 0x7fea400ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea45005640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fea4015e2f0 0x7fea400ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50308/0 (socket says 172.21.15.115:50308) 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea45005640 1 -- 172.21.15.115:0/3751608898 learned_addr learned my addr 172.21.15.115:0/3751608898 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea45806640 1 --2- 172.21.15.115:0/3751608898 >> v2:172.21.15.142:3300/0 conn(0x7fea400adf10 0x7fea400c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea45005640 1 -- 172.21.15.115:0/3751608898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea4006f2f0 msgr2=0x7fea400caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea45005640 1 --2- 172.21.15.115:0/3751608898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea4006f2f0 0x7fea400caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea45005640 1 -- 172.21.15.115:0/3751608898 >> v2:172.21.15.142:3300/0 conn(0x7fea400adf10 msgr2=0x7fea400c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea45005640 1 --2- 172.21.15.115:0/3751608898 >> v2:172.21.15.142:3300/0 conn(0x7fea400adf10 0x7fea400c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea45005640 1 -- 172.21.15.115:0/3751608898 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea400cf7b0 con 0x7fea4015e2f0 2023-12-01T01:52:58.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.217+0000 7fea45806640 1 --2- 172.21.15.115:0/3751608898 >> v2:172.21.15.142:3300/0 conn(0x7fea400adf10 0x7fea400c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:58.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.218+0000 7fea45005640 1 --2- 172.21.15.115:0/3751608898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 0x7fea400ca660 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fea400cb2b0 tx=0x7fea3c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a3aaed8dc9a632 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:58.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea37fff640 1 -- 172.21.15.115:0/3751608898 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c02c950 con 0x7fea4015e2f0 2023-12-01T01:52:58.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea37fff640 1 -- 172.21.15.115:0/3751608898 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c02cab0 con 0x7fea4015e2f0 2023-12-01T01:52:58.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea37fff640 1 -- 172.21.15.115:0/3751608898 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c02ce40 con 0x7fea4015e2f0 2023-12-01T01:52:58.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea47269640 1 -- 172.21.15.115:0/3751608898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 msgr2=0x7fea400ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:58.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea47269640 1 --2- 172.21.15.115:0/3751608898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 0x7fea400ca660 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fea400cb2b0 tx=0x7fea3c02bdb0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea47269640 1 -- 172.21.15.115:0/3751608898 shutdown_connections 2023-12-01T01:52:58.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea47269640 1 --2- 172.21.15.115:0/3751608898 >> v2:172.21.15.142:3300/0 conn(0x7fea400adf10 0x7fea400c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea47269640 1 --2- 172.21.15.115:0/3751608898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea4006f2f0 0x7fea400caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea47269640 1 --2- 172.21.15.115:0/3751608898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 0x7fea400ca660 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea47269640 1 -- 172.21.15.115:0/3751608898 >> 172.21.15.115:0/3751608898 conn(0x7fea400ce880 msgr2=0x7fea400cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:58.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea47269640 1 -- 172.21.15.115:0/3751608898 shutdown_connections 2023-12-01T01:52:58.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.219+0000 7fea47269640 1 -- 172.21.15.115:0/3751608898 wait complete. 2023-12-01T01:52:58.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.221+0000 7fea47269640 1 Processor -- start 2023-12-01T01:52:58.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.221+0000 7fea47269640 1 -- start start 2023-12-01T01:52:58.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.222+0000 7fea47269640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea4006f2f0 0x7fea401f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.222+0000 7fea47269640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea400adf10 0x7fea401f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.222+0000 7fea47269640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 0x7fea401f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.222+0000 7fea47269640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea401f2980 con 0x7fea4015e2f0 2023-12-01T01:52:58.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.222+0000 7fea47269640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea401f2c60 con 0x7fea4006f2f0 2023-12-01T01:52:58.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.222+0000 7fea47269640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea401f2f40 con 0x7fea400adf10 2023-12-01T01:52:58.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.223+0000 7fea45806640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 0x7fea401f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.223+0000 7fea45806640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 0x7fea401f1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50324/0 (socket says 172.21.15.115:50324) 2023-12-01T01:52:58.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.223+0000 7fea45806640 1 -- 172.21.15.115:0/2267269612 learned_addr learned my addr 172.21.15.115:0/2267269612 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:58.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.223+0000 7fea45005640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea4006f2f0 0x7fea401f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.223+0000 7fea44804640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea400adf10 0x7fea401f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.223+0000 7fea45806640 1 -- 172.21.15.115:0/2267269612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea400adf10 msgr2=0x7fea401f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:58.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.223+0000 7fea45806640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea400adf10 0x7fea401f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.223+0000 7fea45806640 1 -- 172.21.15.115:0/2267269612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea4006f2f0 msgr2=0x7fea401f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:58.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.223+0000 7fea45806640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea4006f2f0 0x7fea401f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.223+0000 7fea45806640 1 -- 172.21.15.115:0/2267269612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c009b30 con 0x7fea4015e2f0 2023-12-01T01:52:58.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.224+0000 7fea45806640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 0x7fea401f1c10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fea3800c5c0 tx=0x7fea3800ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:58.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.224+0000 7fea35ffb640 1 -- 172.21.15.115:0/2267269612 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38011800 con 0x7fea4015e2f0 2023-12-01T01:52:58.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.224+0000 7fea35ffb640 1 -- 172.21.15.115:0/2267269612 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea38011980 con 0x7fea4015e2f0 2023-12-01T01:52:58.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.224+0000 7fea35ffb640 1 -- 172.21.15.115:0/2267269612 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38011d30 con 0x7fea4015e2f0 2023-12-01T01:52:58.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.224+0000 7fea47269640 1 -- 172.21.15.115:0/2267269612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea401f3220 con 0x7fea4015e2f0 2023-12-01T01:52:58.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.224+0000 7fea47269640 1 -- 172.21.15.115:0/2267269612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea400c3fb0 con 0x7fea4015e2f0 2023-12-01T01:52:58.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.225+0000 7fea47269640 1 -- 172.21.15.115:0/2267269612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4006fda0 con 0x7fea4015e2f0 2023-12-01T01:52:58.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.226+0000 7fea35ffb640 1 -- 172.21.15.115:0/2267269612 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fea38022350 con 0x7fea4015e2f0 2023-12-01T01:52:58.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.227+0000 7fea35ffb640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea2406a550 0x7fea2406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.227+0000 7fea35ffb640 1 -- 172.21.15.115:0/2267269612 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea38088c30 con 0x7fea4015e2f0 2023-12-01T01:52:58.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.227+0000 7fea45005640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea2406a550 0x7fea2406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.228+0000 7fea45005640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea2406a550 0x7fea2406ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea3c035000 tx=0x7fea3c030000 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:58.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.232+0000 7fea35ffb640 1 -- 172.21.15.115:0/2267269612 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fea38054010 con 0x7fea4015e2f0 2023-12-01T01:52:58.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.419+0000 7fea47269640 1 -- 172.21.15.115:0/2267269612 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea400b43d0 con 0x7fea2406a550 2023-12-01T01:52:58.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.443+0000 7fea35ffb640 1 -- 172.21.15.115:0/2267269612 <== mgr.7230 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287293 (secure 0 0 0) 0x7fea3c034470 con 0x7fea2406a550 2023-12-01T01:52:58.448 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:58.450 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.448+0000 7fea47269640 1 -- 172.21.15.115:0/2267269612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea2406a550 msgr2=0x7fea2406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.448+0000 7fea47269640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea2406a550 0x7fea2406ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea3c035000 tx=0x7fea3c030000 comp rx=0 tx=0).stop 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.448+0000 7fea47269640 1 -- 172.21.15.115:0/2267269612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 msgr2=0x7fea401f1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.448+0000 7fea47269640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 0x7fea401f1c10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fea3800c5c0 tx=0x7fea3800ca80 comp rx=0 tx=0).stop 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.449+0000 7fea47269640 1 -- 172.21.15.115:0/2267269612 shutdown_connections 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.449+0000 7fea47269640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fea2406a550 0x7fea2406ca10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.449+0000 7fea47269640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea400adf10 0x7fea401f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.449+0000 7fea47269640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea4015e2f0 0x7fea401f1c10 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.449+0000 7fea47269640 1 --2- 172.21.15.115:0/2267269612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea4006f2f0 0x7fea401f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.449+0000 7fea47269640 1 -- 172.21.15.115:0/2267269612 >> 172.21.15.115:0/2267269612 conn(0x7fea400ce880 msgr2=0x7fea400cb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:58.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.449+0000 7fea47269640 1 -- 172.21.15.115:0/2267269612 shutdown_connections 2023-12-01T01:52:58.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.449+0000 7fea47269640 1 -- 172.21.15.115:0/2267269612 wait complete. 2023-12-01T01:52:58.471 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-01T01:52:58.108503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316840,"kb_used_data":11880,"kb_used_omap":0,"kb_used_meta":304896,"kb_avail":754657880,"statfs":{"total":773094113280,"available":772769669120,"internally_reserved":0,"allocated":12165120,"data_stored":6376236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312213504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.612758"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525276+0000","last_change":"2023-12-01T01:52:48.519541+0000","last_active":"2023-12-01T01:52:48.525276+0000","last_peered":"2023-12-01T01:52:48.525276+0000","last_clean":"2023-12-01T01:52:48.525276+0000","last_became_active":"2023-12-01T01:52:48.518919+0000","last_became_peered":"2023-12-01T01:52:48.518919+0000","last_unstale":"2023-12-01T01:52:48.525276+0000","last_undegraded":"2023-12-01T01:52:48.525276+0000","last_fullsized":"2023-12-01T01:52:48.525276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:07:01.922435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.524811+0000","last_change":"2023-12-01T01:52:48.522421+0000","last_active":"2023-12-01T01:52:48.524811+0000","last_peered":"2023-12-01T01:52:48.524811+0000","last_clean":"2023-12-01T01:52:48.524811+0000","last_became_active":"2023-12-01T01:52:48.522158+0000","last_became_peered":"2023-12-01T01:52:48.522158+0000","last_unstale":"2023-12-01T01:52:48.524811+0000","last_undegraded":"2023-12-01T01:52:48.524811+0000","last_fullsized":"2023-12-01T01:52:48.524811+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:20.199417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533775+0000","last_change":"2023-12-01T01:52:48.529730+0000","last_active":"2023-12-01T01:52:48.533775+0000","last_peered":"2023-12-01T01:52:48.533775+0000","last_clean":"2023-12-01T01:52:48.533775+0000","last_became_active":"2023-12-01T01:52:48.529453+0000","last_became_peered":"2023-12-01T01:52:48.529453+0000","last_unstale":"2023-12-01T01:52:48.533775+0000","last_undegraded":"2023-12-01T01:52:48.533775+0000","last_fullsized":"2023-12-01T01:52:48.533775+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.526329+0000","last_change":"2023-12-01T01:52:48.520005+0000","last_active":"2023-12-01T01:52:48.526329+0000","last_peered":"2023-12-01T01:52:48.526329+0000","last_clean":"2023-12-01T01:52:48.526329+0000","last_became_active":"2023-12-01T01:52:48.519744+0000","last_became_peered":"2023-12-01T01:52:48.519744+0000","last_unstale":"2023-12-01T01:52:48.526329+0000","last_undegraded":"2023-12-01T01:52:48.526329+0000","last_fullsized":"2023-12-01T01:52:48.526329+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:06:59.600929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527655+0000","last_change":"2023-12-01T01:52:48.524456+0000","last_active":"2023-12-01T01:52:48.527655+0000","last_peered":"2023-12-01T01:52:48.527655+0000","last_clean":"2023-12-01T01:52:48.527655+0000","last_became_active":"2023-12-01T01:52:48.524192+0000","last_became_peered":"2023-12-01T01:52:48.524192+0000","last_unstale":"2023-12-01T01:52:48.527655+0000","last_undegraded":"2023-12-01T01:52:48.527655+0000","last_fullsized":"2023-12-01T01:52:48.527655+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:48:34.887669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.535085+0000","last_change":"2023-12-01T01:52:48.530842+0000","last_active":"2023-12-01T01:52:48.535085+0000","last_peered":"2023-12-01T01:52:48.535085+0000","last_clean":"2023-12-01T01:52:48.535085+0000","last_became_active":"2023-12-01T01:52:48.529072+0000","last_became_peered":"2023-12-01T01:52:48.529072+0000","last_unstale":"2023-12-01T01:52:48.535085+0000","last_undegraded":"2023-12-01T01:52:48.535085+0000","last_fullsized":"2023-12-01T01:52:48.535085+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:39:18.420163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.524471+0000","last_change":"2023-12-01T01:52:48.517258+0000","last_active":"2023-12-01T01:52:48.524471+0000","last_peered":"2023-12-01T01:52:48.524471+0000","last_clean":"2023-12-01T01:52:48.524471+0000","last_became_active":"2023-12-01T01:52:48.516997+0000","last_became_peered":"2023-12-01T01:52:48.516997+0000","last_unstale":"2023-12-01T01:52:48.524471+0000","last_undegraded":"2023-12-01T01:52:48.524471+0000","last_fullsized":"2023-12-01T01:52:48.524471+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:51.221046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532821+0000","last_change":"2023-12-01T01:52:48.528652+0000","last_active":"2023-12-01T01:52:48.532821+0000","last_peered":"2023-12-01T01:52:48.532821+0000","last_clean":"2023-12-01T01:52:48.532821+0000","last_became_active":"2023-12-01T01:52:48.528108+0000","last_became_peered":"2023-12-01T01:52:48.528108+0000","last_unstale":"2023-12-01T01:52:48.532821+0000","last_undegraded":"2023-12-01T01:52:48.532821+0000","last_fullsized":"2023-12-01T01:52:48.532821+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:06:21.099958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.515844+0000","last_change":"2023-12-01T01:52:48.511194+0000","last_active":"2023-12-01T01:52:48.515844+0000","last_peered":"2023-12-01T01:52:48.515844+0000","last_clean":"2023-12-01T01:52:48.515844+0000","last_became_active":"2023-12-01T01:52:48.510913+0000","last_became_peered":"2023-12-01T01:52:48.510913+0000","last_unstale":"2023-12-01T01:52:48.515844+0000","last_undegraded":"2023-12-01T01:52:48.515844+0000","last_fullsized":"2023-12-01T01:52:48.515844+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:52.282595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527630+0000","last_change":"2023-12-01T01:52:48.526094+0000","last_active":"2023-12-01T01:52:48.527630+0000","last_peered":"2023-12-01T01:52:48.527630+0000","last_clean":"2023-12-01T01:52:48.527630+0000","last_became_active":"2023-12-01T01:52:48.525631+0000","last_became_peered":"2023-12-01T01:52:48.525631+0000","last_unstale":"2023-12-01T01:52:48.527630+0000","last_undegraded":"2023-12-01T01:52:48.527630+0000","last_fullsized":"2023-12-01T01:52:48.527630+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532749+0000","last_change":"2023-12-01T01:52:48.523827+0000","last_active":"2023-12-01T01:52:48.532749+0000","last_peered":"2023-12-01T01:52:48.532749+0000","last_clean":"2023-12-01T01:52:48.532749+0000","last_became_active":"2023-12-01T01:52:48.521531+0000","last_became_peered":"2023-12-01T01:52:48.521531+0000","last_unstale":"2023-12-01T01:52:48.532749+0000","last_undegraded":"2023-12-01T01:52:48.532749+0000","last_fullsized":"2023-12-01T01:52:48.532749+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:19.674839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.520324+0000","last_change":"2023-12-01T01:52:48.517956+0000","last_active":"2023-12-01T01:52:48.520324+0000","last_peered":"2023-12-01T01:52:48.520324+0000","last_clean":"2023-12-01T01:52:48.520324+0000","last_became_active":"2023-12-01T01:52:48.517665+0000","last_became_peered":"2023-12-01T01:52:48.517665+0000","last_unstale":"2023-12-01T01:52:48.520324+0000","last_undegraded":"2023-12-01T01:52:48.520324+0000","last_fullsized":"2023-12-01T01:52:48.520324+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527638+0000","last_change":"2023-12-01T01:52:48.523339+0000","last_active":"2023-12-01T01:52:48.527638+0000","last_peered":"2023-12-01T01:52:48.527638+0000","last_clean":"2023-12-01T01:52:48.527638+0000","last_became_active":"2023-12-01T01:52:48.523056+0000","last_became_peered":"2023-12-01T01:52:48.523056+0000","last_unstale":"2023-12-01T01:52:48.527638+0000","last_undegraded":"2023-12-01T01:52:48.527638+0000","last_fullsized":"2023-12-01T01:52:48.527638+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:45:17.044474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532841+0000","last_change":"2023-12-01T01:52:48.527584+0000","last_active":"2023-12-01T01:52:48.532841+0000","last_peered":"2023-12-01T01:52:48.532841+0000","last_clean":"2023-12-01T01:52:48.532841+0000","last_became_active":"2023-12-01T01:52:48.526884+0000","last_became_peered":"2023-12-01T01:52:48.526884+0000","last_unstale":"2023-12-01T01:52:48.532841+0000","last_undegraded":"2023-12-01T01:52:48.532841+0000","last_fullsized":"2023-12-01T01:52:48.532841+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:40:59.300637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.524491+0000","last_change":"2023-12-01T01:52:48.518153+0000","last_active":"2023-12-01T01:52:48.524491+0000","last_peered":"2023-12-01T01:52:48.524491+0000","last_clean":"2023-12-01T01:52:48.524491+0000","last_became_active":"2023-12-01T01:52:48.517878+0000","last_became_peered":"2023-12-01T01:52:48.517878+0000","last_unstale":"2023-12-01T01:52:48.524491+0000","last_undegraded":"2023-12-01T01:52:48.524491+0000","last_fullsized":"2023-12-01T01:52:48.524491+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:58:23.518943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.521765+0000","last_change":"2023-12-01T01:52:48.520173+0000","last_active":"2023-12-01T01:52:48.521765+0000","last_peered":"2023-12-01T01:52:48.521765+0000","last_clean":"2023-12-01T01:52:48.521765+0000","last_became_active":"2023-12-01T01:52:48.518959+0000","last_became_peered":"2023-12-01T01:52:48.518959+0000","last_unstale":"2023-12-01T01:52:48.521765+0000","last_undegraded":"2023-12-01T01:52:48.521765+0000","last_fullsized":"2023-12-01T01:52:48.521765+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.526378+0000","last_change":"2023-12-01T01:52:48.523081+0000","last_active":"2023-12-01T01:52:48.526378+0000","last_peered":"2023-12-01T01:52:48.526378+0000","last_clean":"2023-12-01T01:52:48.526378+0000","last_became_active":"2023-12-01T01:52:48.522578+0000","last_became_peered":"2023-12-01T01:52:48.522578+0000","last_unstale":"2023-12-01T01:52:48.526378+0000","last_undegraded":"2023-12-01T01:52:48.526378+0000","last_fullsized":"2023-12-01T01:52:48.526378+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.528907+0000","last_change":"2023-12-01T01:52:48.520828+0000","last_active":"2023-12-01T01:52:48.528907+0000","last_peered":"2023-12-01T01:52:48.528907+0000","last_clean":"2023-12-01T01:52:48.528907+0000","last_became_active":"2023-12-01T01:52:48.520448+0000","last_became_peered":"2023-12-01T01:52:48.520448+0000","last_unstale":"2023-12-01T01:52:48.528907+0000","last_undegraded":"2023-12-01T01:52:48.528907+0000","last_fullsized":"2023-12-01T01:52:48.528907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:14.707263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532779+0000","last_change":"2023-12-01T01:52:48.528240+0000","last_active":"2023-12-01T01:52:48.532779+0000","last_peered":"2023-12-01T01:52:48.532779+0000","last_clean":"2023-12-01T01:52:48.532779+0000","last_became_active":"2023-12-01T01:52:48.527401+0000","last_became_peered":"2023-12-01T01:52:48.527401+0000","last_unstale":"2023-12-01T01:52:48.532779+0000","last_undegraded":"2023-12-01T01:52:48.532779+0000","last_fullsized":"2023-12-01T01:52:48.532779+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:13:33.646688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.536524+0000","last_change":"2023-12-01T01:52:48.532223+0000","last_active":"2023-12-01T01:52:48.536524+0000","last_peered":"2023-12-01T01:52:48.536524+0000","last_clean":"2023-12-01T01:52:48.536524+0000","last_became_active":"2023-12-01T01:52:48.530936+0000","last_became_peered":"2023-12-01T01:52:48.530936+0000","last_unstale":"2023-12-01T01:52:48.536524+0000","last_undegraded":"2023-12-01T01:52:48.536524+0000","last_fullsized":"2023-12-01T01:52:48.536524+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:42.942148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.539847+0000","last_change":"2023-12-01T01:52:48.531631+0000","last_active":"2023-12-01T01:52:48.539847+0000","last_peered":"2023-12-01T01:52:48.539847+0000","last_clean":"2023-12-01T01:52:48.539847+0000","last_became_active":"2023-12-01T01:52:48.531336+0000","last_became_peered":"2023-12-01T01:52:48.531336+0000","last_unstale":"2023-12-01T01:52:48.539847+0000","last_undegraded":"2023-12-01T01:52:48.539847+0000","last_fullsized":"2023-12-01T01:52:48.539847+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.517000+0000","last_change":"2023-12-01T01:52:48.513602+0000","last_active":"2023-12-01T01:52:48.517000+0000","last_peered":"2023-12-01T01:52:48.517000+0000","last_clean":"2023-12-01T01:52:48.517000+0000","last_became_active":"2023-12-01T01:52:48.513269+0000","last_became_peered":"2023-12-01T01:52:48.513269+0000","last_unstale":"2023-12-01T01:52:48.517000+0000","last_undegraded":"2023-12-01T01:52:48.517000+0000","last_fullsized":"2023-12-01T01:52:48.517000+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525839+0000","last_change":"2023-12-01T01:52:48.522640+0000","last_active":"2023-12-01T01:52:48.525839+0000","last_peered":"2023-12-01T01:52:48.525839+0000","last_clean":"2023-12-01T01:52:48.525839+0000","last_became_active":"2023-12-01T01:52:48.521392+0000","last_became_peered":"2023-12-01T01:52:48.521392+0000","last_unstale":"2023-12-01T01:52:48.525839+0000","last_undegraded":"2023-12-01T01:52:48.525839+0000","last_fullsized":"2023-12-01T01:52:48.525839+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:39.742863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.530244+0000","last_change":"2023-12-01T01:52:48.521211+0000","last_active":"2023-12-01T01:52:48.530244+0000","last_peered":"2023-12-01T01:52:48.530244+0000","last_clean":"2023-12-01T01:52:48.530244+0000","last_became_active":"2023-12-01T01:52:48.520691+0000","last_became_peered":"2023-12-01T01:52:48.520691+0000","last_unstale":"2023-12-01T01:52:48.530244+0000","last_undegraded":"2023-12-01T01:52:48.530244+0000","last_fullsized":"2023-12-01T01:52:48.530244+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.528579+0000","last_change":"2023-12-01T01:52:48.517997+0000","last_active":"2023-12-01T01:52:48.528579+0000","last_peered":"2023-12-01T01:52:48.528579+0000","last_clean":"2023-12-01T01:52:48.528579+0000","last_became_active":"2023-12-01T01:52:48.516730+0000","last_became_peered":"2023-12-01T01:52:48.516730+0000","last_unstale":"2023-12-01T01:52:48.528579+0000","last_undegraded":"2023-12-01T01:52:48.528579+0000","last_fullsized":"2023-12-01T01:52:48.528579+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.514112+0000","last_change":"2023-12-01T01:52:48.511348+0000","last_active":"2023-12-01T01:52:48.514112+0000","last_peered":"2023-12-01T01:52:48.514112+0000","last_clean":"2023-12-01T01:52:48.514112+0000","last_became_active":"2023-12-01T01:52:48.510938+0000","last_became_peered":"2023-12-01T01:52:48.510938+0000","last_unstale":"2023-12-01T01:52:48.514112+0000","last_undegraded":"2023-12-01T01:52:48.514112+0000","last_fullsized":"2023-12-01T01:52:48.514112+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:22.154067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527586+0000","last_change":"2023-12-01T01:52:48.525606+0000","last_active":"2023-12-01T01:52:48.527586+0000","last_peered":"2023-12-01T01:52:48.527586+0000","last_clean":"2023-12-01T01:52:48.527586+0000","last_became_active":"2023-12-01T01:52:48.524366+0000","last_became_peered":"2023-12-01T01:52:48.524366+0000","last_unstale":"2023-12-01T01:52:48.527586+0000","last_undegraded":"2023-12-01T01:52:48.527586+0000","last_fullsized":"2023-12-01T01:52:48.527586+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:12:02.919659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.516130+0000","last_change":"2023-12-01T01:52:48.511789+0000","last_active":"2023-12-01T01:52:48.516130+0000","last_peered":"2023-12-01T01:52:48.516130+0000","last_clean":"2023-12-01T01:52:48.516130+0000","last_became_active":"2023-12-01T01:52:48.511449+0000","last_became_peered":"2023-12-01T01:52:48.511449+0000","last_unstale":"2023-12-01T01:52:48.516130+0000","last_undegraded":"2023-12-01T01:52:48.516130+0000","last_fullsized":"2023-12-01T01:52:48.516130+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:25:02.674823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525530+0000","last_change":"2023-12-01T01:52:48.521420+0000","last_active":"2023-12-01T01:52:48.525530+0000","last_peered":"2023-12-01T01:52:48.525530+0000","last_clean":"2023-12-01T01:52:48.525530+0000","last_became_active":"2023-12-01T01:52:48.519768+0000","last_became_peered":"2023-12-01T01:52:48.519768+0000","last_unstale":"2023-12-01T01:52:48.525530+0000","last_undegraded":"2023-12-01T01:52:48.525530+0000","last_fullsized":"2023-12-01T01:52:48.525530+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523402+0000","last_change":"2023-12-01T01:52:48.517145+0000","last_active":"2023-12-01T01:52:48.523402+0000","last_peered":"2023-12-01T01:52:48.523402+0000","last_clean":"2023-12-01T01:52:48.523402+0000","last_became_active":"2023-12-01T01:52:48.516469+0000","last_became_peered":"2023-12-01T01:52:48.516469+0000","last_unstale":"2023-12-01T01:52:48.523402+0000","last_undegraded":"2023-12-01T01:52:48.523402+0000","last_fullsized":"2023-12-01T01:52:48.523402+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:00.516734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533549+0000","last_change":"2023-12-01T01:52:48.527651+0000","last_active":"2023-12-01T01:52:48.533549+0000","last_peered":"2023-12-01T01:52:48.533549+0000","last_clean":"2023-12-01T01:52:48.533549+0000","last_became_active":"2023-12-01T01:52:48.527285+0000","last_became_peered":"2023-12-01T01:52:48.527285+0000","last_unstale":"2023-12-01T01:52:48.533549+0000","last_undegraded":"2023-12-01T01:52:48.533549+0000","last_fullsized":"2023-12-01T01:52:48.533549+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:03.748734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533783+0000","last_change":"2023-12-01T01:52:48.529942+0000","last_active":"2023-12-01T01:52:48.533783+0000","last_peered":"2023-12-01T01:52:48.533783+0000","last_clean":"2023-12-01T01:52:48.533783+0000","last_became_active":"2023-12-01T01:52:48.529681+0000","last_became_peered":"2023-12-01T01:52:48.529681+0000","last_unstale":"2023-12-01T01:52:48.533783+0000","last_undegraded":"2023-12-01T01:52:48.533783+0000","last_fullsized":"2023-12-01T01:52:48.533783+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.519637+0000","last_change":"2023-12-01T01:52:48.516192+0000","last_active":"2023-12-01T01:52:48.519637+0000","last_peered":"2023-12-01T01:52:48.519637+0000","last_clean":"2023-12-01T01:52:48.519637+0000","last_became_active":"2023-12-01T01:52:48.514199+0000","last_became_peered":"2023-12-01T01:52:48.514199+0000","last_unstale":"2023-12-01T01:52:48.519637+0000","last_undegraded":"2023-12-01T01:52:48.519637+0000","last_fullsized":"2023-12-01T01:52:48.519637+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:02:42.893820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"110'4","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.540248+0000","last_change":"2023-12-01T01:52:48.038820+0000","last_active":"2023-12-01T01:52:48.540248+0000","last_peered":"2023-12-01T01:52:48.540248+0000","last_clean":"2023-12-01T01:52:48.540248+0000","last_became_active":"2023-12-01T01:52:48.038539+0000","last_became_peered":"2023-12-01T01:52:48.038539+0000","last_unstale":"2023-12-01T01:52:48.540248+0000","last_undegraded":"2023-12-01T01:52:48.540248+0000","last_fullsized":"2023-12-01T01:52:48.540248+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:21.090681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525259+0000","last_change":"2023-12-01T01:52:48.518682+0000","last_active":"2023-12-01T01:52:48.525259+0000","last_peered":"2023-12-01T01:52:48.525259+0000","last_clean":"2023-12-01T01:52:48.525259+0000","last_became_active":"2023-12-01T01:52:48.518415+0000","last_became_peered":"2023-12-01T01:52:48.518415+0000","last_unstale":"2023-12-01T01:52:48.525259+0000","last_undegraded":"2023-12-01T01:52:48.525259+0000","last_fullsized":"2023-12-01T01:52:48.525259+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:01.801261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.534050+0000","last_change":"2023-12-01T01:52:48.049643+0000","last_active":"2023-12-01T01:52:48.534050+0000","last_peered":"2023-12-01T01:52:48.534050+0000","last_clean":"2023-12-01T01:52:48.534050+0000","last_became_active":"2023-12-01T01:52:48.049288+0000","last_became_peered":"2023-12-01T01:52:48.049288+0000","last_unstale":"2023-12-01T01:52:48.534050+0000","last_undegraded":"2023-12-01T01:52:48.534050+0000","last_fullsized":"2023-12-01T01:52:48.534050+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:41.937860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.526343+0000","last_change":"2023-12-01T01:52:48.521987+0000","last_active":"2023-12-01T01:52:48.526343+0000","last_peered":"2023-12-01T01:52:48.526343+0000","last_clean":"2023-12-01T01:52:48.526343+0000","last_became_active":"2023-12-01T01:52:48.521595+0000","last_became_peered":"2023-12-01T01:52:48.521595+0000","last_unstale":"2023-12-01T01:52:48.526343+0000","last_undegraded":"2023-12-01T01:52:48.526343+0000","last_fullsized":"2023-12-01T01:52:48.526343+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:50:17.533885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.536817+0000","last_change":"2023-12-01T01:52:48.039238+0000","last_active":"2023-12-01T01:52:48.536817+0000","last_peered":"2023-12-01T01:52:48.536817+0000","last_clean":"2023-12-01T01:52:48.536817+0000","last_became_active":"2023-12-01T01:52:48.038835+0000","last_became_peered":"2023-12-01T01:52:48.038835+0000","last_unstale":"2023-12-01T01:52:48.536817+0000","last_undegraded":"2023-12-01T01:52:48.536817+0000","last_fullsized":"2023-12-01T01:52:48.536817+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525630+0000","last_change":"2023-12-01T01:52:48.522088+0000","last_active":"2023-12-01T01:52:48.525630+0000","last_peered":"2023-12-01T01:52:48.525630+0000","last_clean":"2023-12-01T01:52:48.525630+0000","last_became_active":"2023-12-01T01:52:48.521834+0000","last_became_peered":"2023-12-01T01:52:48.521834+0000","last_unstale":"2023-12-01T01:52:48.525630+0000","last_undegraded":"2023-12-01T01:52:48.525630+0000","last_fullsized":"2023-12-01T01:52:48.525630+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:28.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.534961+0000","last_change":"2023-12-01T01:52:48.045716+0000","last_active":"2023-12-01T01:52:48.534961+0000","last_peered":"2023-12-01T01:52:48.534961+0000","last_clean":"2023-12-01T01:52:48.534961+0000","last_became_active":"2023-12-01T01:52:48.045176+0000","last_became_peered":"2023-12-01T01:52:48.045176+0000","last_unstale":"2023-12-01T01:52:48.534961+0000","last_undegraded":"2023-12-01T01:52:48.534961+0000","last_fullsized":"2023-12-01T01:52:48.534961+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:55:21.695057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523967+0000","last_change":"2023-12-01T01:52:48.519484+0000","last_active":"2023-12-01T01:52:48.523967+0000","last_peered":"2023-12-01T01:52:48.523967+0000","last_clean":"2023-12-01T01:52:48.523967+0000","last_became_active":"2023-12-01T01:52:48.519075+0000","last_became_peered":"2023-12-01T01:52:48.519075+0000","last_unstale":"2023-12-01T01:52:48.523967+0000","last_undegraded":"2023-12-01T01:52:48.523967+0000","last_fullsized":"2023-12-01T01:52:48.523967+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:48:12.472639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533011+0000","last_change":"2023-12-01T01:52:48.033877+0000","last_active":"2023-12-01T01:52:48.533011+0000","last_peered":"2023-12-01T01:52:48.533011+0000","last_clean":"2023-12-01T01:52:48.533011+0000","last_became_active":"2023-12-01T01:52:48.033605+0000","last_became_peered":"2023-12-01T01:52:48.033605+0000","last_unstale":"2023-12-01T01:52:48.533011+0000","last_undegraded":"2023-12-01T01:52:48.533011+0000","last_fullsized":"2023-12-01T01:52:48.533011+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:55.538925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527179+0000","last_change":"2023-12-01T01:52:48.525907+0000","last_active":"2023-12-01T01:52:48.527179+0000","last_peered":"2023-12-01T01:52:48.527179+0000","last_clean":"2023-12-01T01:52:48.527179+0000","last_became_active":"2023-12-01T01:52:48.525555+0000","last_became_peered":"2023-12-01T01:52:48.525555+0000","last_unstale":"2023-12-01T01:52:48.527179+0000","last_undegraded":"2023-12-01T01:52:48.527179+0000","last_fullsized":"2023-12-01T01:52:48.527179+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:53:05.715094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.531149+0000","last_change":"2023-12-01T01:52:48.043431+0000","last_active":"2023-12-01T01:52:48.531149+0000","last_peered":"2023-12-01T01:52:48.531149+0000","last_clean":"2023-12-01T01:52:48.531149+0000","last_became_active":"2023-12-01T01:52:48.043162+0000","last_became_peered":"2023-12-01T01:52:48.043162+0000","last_unstale":"2023-12-01T01:52:48.531149+0000","last_undegraded":"2023-12-01T01:52:48.531149+0000","last_fullsized":"2023-12-01T01:52:48.531149+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523481+0000","last_change":"2023-12-01T01:52:48.521106+0000","last_active":"2023-12-01T01:52:48.523481+0000","last_peered":"2023-12-01T01:52:48.523481+0000","last_clean":"2023-12-01T01:52:48.523481+0000","last_became_active":"2023-12-01T01:52:48.520854+0000","last_became_peered":"2023-12-01T01:52:48.520854+0000","last_unstale":"2023-12-01T01:52:48.523481+0000","last_undegraded":"2023-12-01T01:52:48.523481+0000","last_fullsized":"2023-12-01T01:52:48.523481+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:10:51.868019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.503557+0000","last_change":"2023-12-01T01:52:48.046040+0000","last_active":"2023-12-01T01:52:48.503557+0000","last_peered":"2023-12-01T01:52:48.503557+0000","last_clean":"2023-12-01T01:52:48.503557+0000","last_became_active":"2023-12-01T01:52:48.045722+0000","last_became_peered":"2023-12-01T01:52:48.045722+0000","last_unstale":"2023-12-01T01:52:48.503557+0000","last_undegraded":"2023-12-01T01:52:48.503557+0000","last_fullsized":"2023-12-01T01:52:48.503557+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:25:01.670535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525082+0000","last_change":"2023-12-01T01:52:48.515054+0000","last_active":"2023-12-01T01:52:48.525082+0000","last_peered":"2023-12-01T01:52:48.525082+0000","last_clean":"2023-12-01T01:52:48.525082+0000","last_became_active":"2023-12-01T01:52:48.514726+0000","last_became_peered":"2023-12-01T01:52:48.514726+0000","last_unstale":"2023-12-01T01:52:48.525082+0000","last_undegraded":"2023-12-01T01:52:48.525082+0000","last_fullsized":"2023-12-01T01:52:48.525082+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:38.690885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.502994+0000","last_change":"2023-12-01T01:52:48.049260+0000","last_active":"2023-12-01T01:52:48.502994+0000","last_peered":"2023-12-01T01:52:48.502994+0000","last_clean":"2023-12-01T01:52:48.502994+0000","last_became_active":"2023-12-01T01:52:48.048998+0000","last_became_peered":"2023-12-01T01:52:48.048998+0000","last_unstale":"2023-12-01T01:52:48.502994+0000","last_undegraded":"2023-12-01T01:52:48.502994+0000","last_fullsized":"2023-12-01T01:52:48.502994+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:51:06.546333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523078+0000","last_change":"2023-12-01T01:52:48.516163+0000","last_active":"2023-12-01T01:52:48.523078+0000","last_peered":"2023-12-01T01:52:48.523078+0000","last_clean":"2023-12-01T01:52:48.523078+0000","last_became_active":"2023-12-01T01:52:48.515895+0000","last_became_peered":"2023-12-01T01:52:48.515895+0000","last_unstale":"2023-12-01T01:52:48.523078+0000","last_undegraded":"2023-12-01T01:52:48.523078+0000","last_fullsized":"2023-12-01T01:52:48.523078+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:12.516160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.505335+0000","last_change":"2023-12-01T01:52:48.035560+0000","last_active":"2023-12-01T01:52:48.505335+0000","last_peered":"2023-12-01T01:52:48.505335+0000","last_clean":"2023-12-01T01:52:48.505335+0000","last_became_active":"2023-12-01T01:52:48.035303+0000","last_became_peered":"2023-12-01T01:52:48.035303+0000","last_unstale":"2023-12-01T01:52:48.505335+0000","last_undegraded":"2023-12-01T01:52:48.505335+0000","last_fullsized":"2023-12-01T01:52:48.505335+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:11:04.573692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.525296+0000","last_change":"2023-12-01T01:52:48.519705+0000","last_active":"2023-12-01T01:52:48.525296+0000","last_peered":"2023-12-01T01:52:48.525296+0000","last_clean":"2023-12-01T01:52:48.525296+0000","last_became_active":"2023-12-01T01:52:48.519304+0000","last_became_peered":"2023-12-01T01:52:48.519304+0000","last_unstale":"2023-12-01T01:52:48.525296+0000","last_undegraded":"2023-12-01T01:52:48.525296+0000","last_fullsized":"2023-12-01T01:52:48.525296+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:22.643458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.513060+0000","last_change":"2023-12-01T01:52:48.040913+0000","last_active":"2023-12-01T01:52:48.513060+0000","last_peered":"2023-12-01T01:52:48.513060+0000","last_clean":"2023-12-01T01:52:48.513060+0000","last_became_active":"2023-12-01T01:52:48.040402+0000","last_became_peered":"2023-12-01T01:52:48.040402+0000","last_unstale":"2023-12-01T01:52:48.513060+0000","last_undegraded":"2023-12-01T01:52:48.513060+0000","last_fullsized":"2023-12-01T01:52:48.513060+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.529300+0000","last_change":"2023-12-01T01:52:48.520445+0000","last_active":"2023-12-01T01:52:48.529300+0000","last_peered":"2023-12-01T01:52:48.529300+0000","last_clean":"2023-12-01T01:52:48.529300+0000","last_became_active":"2023-12-01T01:52:48.520113+0000","last_became_peered":"2023-12-01T01:52:48.520113+0000","last_unstale":"2023-12-01T01:52:48.529300+0000","last_undegraded":"2023-12-01T01:52:48.529300+0000","last_fullsized":"2023-12-01T01:52:48.529300+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:18.398402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"110'3","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.535325+0000","last_change":"2023-12-01T01:52:48.045583+0000","last_active":"2023-12-01T01:52:48.535325+0000","last_peered":"2023-12-01T01:52:48.535325+0000","last_clean":"2023-12-01T01:52:48.535325+0000","last_became_active":"2023-12-01T01:52:48.045146+0000","last_became_peered":"2023-12-01T01:52:48.045146+0000","last_unstale":"2023-12-01T01:52:48.535325+0000","last_undegraded":"2023-12-01T01:52:48.535325+0000","last_fullsized":"2023-12-01T01:52:48.535325+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:50:25.868578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527568+0000","last_change":"2023-12-01T01:52:48.524728+0000","last_active":"2023-12-01T01:52:48.527568+0000","last_peered":"2023-12-01T01:52:48.527568+0000","last_clean":"2023-12-01T01:52:48.527568+0000","last_became_active":"2023-12-01T01:52:48.524228+0000","last_became_peered":"2023-12-01T01:52:48.524228+0000","last_unstale":"2023-12-01T01:52:48.527568+0000","last_undegraded":"2023-12-01T01:52:48.527568+0000","last_fullsized":"2023-12-01T01:52:48.527568+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:03.971637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"110'6","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.541971+0000","last_change":"2023-12-01T01:52:48.042396+0000","last_active":"2023-12-01T01:52:48.541971+0000","last_peered":"2023-12-01T01:52:48.541971+0000","last_clean":"2023-12-01T01:52:48.541971+0000","last_became_active":"2023-12-01T01:52:48.041958+0000","last_became_peered":"2023-12-01T01:52:48.041958+0000","last_unstale":"2023-12-01T01:52:48.541971+0000","last_undegraded":"2023-12-01T01:52:48.541971+0000","last_fullsized":"2023-12-01T01:52:48.541971+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:33.265613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532665+0000","last_change":"2023-12-01T01:52:48.523304+0000","last_active":"2023-12-01T01:52:48.532665+0000","last_peered":"2023-12-01T01:52:48.532665+0000","last_clean":"2023-12-01T01:52:48.532665+0000","last_became_active":"2023-12-01T01:52:48.523050+0000","last_became_peered":"2023-12-01T01:52:48.523050+0000","last_unstale":"2023-12-01T01:52:48.532665+0000","last_undegraded":"2023-12-01T01:52:48.532665+0000","last_fullsized":"2023-12-01T01:52:48.532665+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.539919+0000","last_change":"2023-12-01T01:52:48.035637+0000","last_active":"2023-12-01T01:52:48.539919+0000","last_peered":"2023-12-01T01:52:48.539919+0000","last_clean":"2023-12-01T01:52:48.539919+0000","last_became_active":"2023-12-01T01:52:48.034955+0000","last_became_peered":"2023-12-01T01:52:48.034955+0000","last_unstale":"2023-12-01T01:52:48.539919+0000","last_undegraded":"2023-12-01T01:52:48.539919+0000","last_fullsized":"2023-12-01T01:52:48.539919+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:02.433217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.530823+0000","last_change":"2023-12-01T01:52:48.527142+0000","last_active":"2023-12-01T01:52:48.530823+0000","last_peered":"2023-12-01T01:52:48.530823+0000","last_clean":"2023-12-01T01:52:48.530823+0000","last_became_active":"2023-12-01T01:52:48.526832+0000","last_became_peered":"2023-12-01T01:52:48.526832+0000","last_unstale":"2023-12-01T01:52:48.530823+0000","last_undegraded":"2023-12-01T01:52:48.530823+0000","last_fullsized":"2023-12-01T01:52:48.530823+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:58:00.440400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.504235+0000","last_change":"2023-12-01T01:52:48.033331+0000","last_active":"2023-12-01T01:52:48.504235+0000","last_peered":"2023-12-01T01:52:48.504235+0000","last_clean":"2023-12-01T01:52:48.504235+0000","last_became_active":"2023-12-01T01:52:48.032852+0000","last_became_peered":"2023-12-01T01:52:48.032852+0000","last_unstale":"2023-12-01T01:52:48.504235+0000","last_undegraded":"2023-12-01T01:52:48.504235+0000","last_fullsized":"2023-12-01T01:52:48.504235+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:37.686598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523961+0000","last_change":"2023-12-01T01:52:48.521021+0000","last_active":"2023-12-01T01:52:48.523961+0000","last_peered":"2023-12-01T01:52:48.523961+0000","last_clean":"2023-12-01T01:52:48.523961+0000","last_became_active":"2023-12-01T01:52:48.520767+0000","last_became_peered":"2023-12-01T01:52:48.520767+0000","last_unstale":"2023-12-01T01:52:48.523961+0000","last_undegraded":"2023-12-01T01:52:48.523961+0000","last_fullsized":"2023-12-01T01:52:48.523961+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:45.833443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.511826+0000","last_change":"2023-12-01T01:52:48.041648+0000","last_active":"2023-12-01T01:52:48.511826+0000","last_peered":"2023-12-01T01:52:48.511826+0000","last_clean":"2023-12-01T01:52:48.511826+0000","last_became_active":"2023-12-01T01:52:48.040913+0000","last_became_peered":"2023-12-01T01:52:48.040913+0000","last_unstale":"2023-12-01T01:52:48.511826+0000","last_undegraded":"2023-12-01T01:52:48.511826+0000","last_fullsized":"2023-12-01T01:52:48.511826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:23:37.686598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.528672+0000","last_change":"2023-12-01T01:52:48.520423+0000","last_active":"2023-12-01T01:52:48.528672+0000","last_peered":"2023-12-01T01:52:48.528672+0000","last_clean":"2023-12-01T01:52:48.528672+0000","last_became_active":"2023-12-01T01:52:48.518361+0000","last_became_peered":"2023-12-01T01:52:48.518361+0000","last_unstale":"2023-12-01T01:52:48.528672+0000","last_undegraded":"2023-12-01T01:52:48.528672+0000","last_fullsized":"2023-12-01T01:52:48.528672+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:22:56.543213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"110'8","reported_seq":19,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.541959+0000","last_change":"2023-12-01T01:52:48.032822+0000","last_active":"2023-12-01T01:52:48.541959+0000","last_peered":"2023-12-01T01:52:48.541959+0000","last_clean":"2023-12-01T01:52:48.541959+0000","last_became_active":"2023-12-01T01:52:48.032368+0000","last_became_peered":"2023-12-01T01:52:48.032368+0000","last_unstale":"2023-12-01T01:52:48.541959+0000","last_undegraded":"2023-12-01T01:52:48.541959+0000","last_fullsized":"2023-12-01T01:52:48.541959+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:10:50.863731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.516793+0000","last_change":"2023-12-01T01:52:48.512359+0000","last_active":"2023-12-01T01:52:48.516793+0000","last_peered":"2023-12-01T01:52:48.516793+0000","last_clean":"2023-12-01T01:52:48.516793+0000","last_became_active":"2023-12-01T01:52:48.511971+0000","last_became_peered":"2023-12-01T01:52:48.511971+0000","last_unstale":"2023-12-01T01:52:48.516793+0000","last_undegraded":"2023-12-01T01:52:48.516793+0000","last_fullsized":"2023-12-01T01:52:48.516793+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:09:36.618385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.535408+0000","last_change":"2023-12-01T01:52:48.040258+0000","last_active":"2023-12-01T01:52:48.535408+0000","last_peered":"2023-12-01T01:52:48.535408+0000","last_clean":"2023-12-01T01:52:48.535408+0000","last_became_active":"2023-12-01T01:52:48.037376+0000","last_became_peered":"2023-12-01T01:52:48.037376+0000","last_unstale":"2023-12-01T01:52:48.535408+0000","last_undegraded":"2023-12-01T01:52:48.535408+0000","last_fullsized":"2023-12-01T01:52:48.535408+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:11:13.111537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533742+0000","last_change":"2023-12-01T01:52:48.529537+0000","last_active":"2023-12-01T01:52:48.533742+0000","last_peered":"2023-12-01T01:52:48.533742+0000","last_clean":"2023-12-01T01:52:48.533742+0000","last_became_active":"2023-12-01T01:52:48.529283+0000","last_became_peered":"2023-12-01T01:52:48.529283+0000","last_unstale":"2023-12-01T01:52:48.533742+0000","last_undegraded":"2023-12-01T01:52:48.533742+0000","last_fullsized":"2023-12-01T01:52:48.533742+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:39.222047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.539256+0000","last_change":"2023-12-01T01:52:48.050044+0000","last_active":"2023-12-01T01:52:48.539256+0000","last_peered":"2023-12-01T01:52:48.539256+0000","last_clean":"2023-12-01T01:52:48.539256+0000","last_became_active":"2023-12-01T01:52:48.049779+0000","last_became_peered":"2023-12-01T01:52:48.049779+0000","last_unstale":"2023-12-01T01:52:48.539256+0000","last_undegraded":"2023-12-01T01:52:48.539256+0000","last_fullsized":"2023-12-01T01:52:48.539256+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:20:19.462886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.519394+0000","last_change":"2023-12-01T01:52:48.515405+0000","last_active":"2023-12-01T01:52:48.519394+0000","last_peered":"2023-12-01T01:52:48.519394+0000","last_clean":"2023-12-01T01:52:48.519394+0000","last_became_active":"2023-12-01T01:52:48.515166+0000","last_became_peered":"2023-12-01T01:52:48.515166+0000","last_unstale":"2023-12-01T01:52:48.519394+0000","last_undegraded":"2023-12-01T01:52:48.519394+0000","last_fullsized":"2023-12-01T01:52:48.519394+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:17.235352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.502669+0000","last_change":"2023-12-01T01:52:48.048859+0000","last_active":"2023-12-01T01:52:48.502669+0000","last_peered":"2023-12-01T01:52:48.502669+0000","last_clean":"2023-12-01T01:52:48.502669+0000","last_became_active":"2023-12-01T01:52:48.048546+0000","last_became_peered":"2023-12-01T01:52:48.048546+0000","last_unstale":"2023-12-01T01:52:48.502669+0000","last_undegraded":"2023-12-01T01:52:48.502669+0000","last_fullsized":"2023-12-01T01:52:48.502669+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:43.307966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.531262+0000","last_change":"2023-12-01T01:52:48.527737+0000","last_active":"2023-12-01T01:52:48.531262+0000","last_peered":"2023-12-01T01:52:48.531262+0000","last_clean":"2023-12-01T01:52:48.531262+0000","last_became_active":"2023-12-01T01:52:48.527227+0000","last_became_peered":"2023-12-01T01:52:48.527227+0000","last_unstale":"2023-12-01T01:52:48.531262+0000","last_undegraded":"2023-12-01T01:52:48.531262+0000","last_fullsized":"2023-12-01T01:52:48.531262+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:34.269901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.539606+0000","last_change":"2023-12-01T01:52:48.046685+0000","last_active":"2023-12-01T01:52:48.539606+0000","last_peered":"2023-12-01T01:52:48.539606+0000","last_clean":"2023-12-01T01:52:48.539606+0000","last_became_active":"2023-12-01T01:52:48.046395+0000","last_became_peered":"2023-12-01T01:52:48.046395+0000","last_unstale":"2023-12-01T01:52:48.539606+0000","last_undegraded":"2023-12-01T01:52:48.539606+0000","last_fullsized":"2023-12-01T01:52:48.539606+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:29:10.130713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.533217+0000","last_change":"2023-12-01T01:52:48.531261+0000","last_active":"2023-12-01T01:52:48.533217+0000","last_peered":"2023-12-01T01:52:48.533217+0000","last_clean":"2023-12-01T01:52:48.533217+0000","last_became_active":"2023-12-01T01:52:48.529433+0000","last_became_peered":"2023-12-01T01:52:48.529433+0000","last_unstale":"2023-12-01T01:52:48.533217+0000","last_undegraded":"2023-12-01T01:52:48.533217+0000","last_fullsized":"2023-12-01T01:52:48.533217+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:52:45.545702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.504332+0000","last_change":"2023-12-01T01:52:48.038141+0000","last_active":"2023-12-01T01:52:48.504332+0000","last_peered":"2023-12-01T01:52:48.504332+0000","last_clean":"2023-12-01T01:52:48.504332+0000","last_became_active":"2023-12-01T01:52:48.037797+0000","last_became_peered":"2023-12-01T01:52:48.037797+0000","last_unstale":"2023-12-01T01:52:48.504332+0000","last_undegraded":"2023-12-01T01:52:48.504332+0000","last_fullsized":"2023-12-01T01:52:48.504332+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.524044+0000","last_change":"2023-12-01T01:52:48.519490+0000","last_active":"2023-12-01T01:52:48.524044+0000","last_peered":"2023-12-01T01:52:48.524044+0000","last_clean":"2023-12-01T01:52:48.524044+0000","last_became_active":"2023-12-01T01:52:48.519131+0000","last_became_peered":"2023-12-01T01:52:48.519131+0000","last_unstale":"2023-12-01T01:52:48.524044+0000","last_undegraded":"2023-12-01T01:52:48.524044+0000","last_fullsized":"2023-12-01T01:52:48.524044+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:24:01.770900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.509529+0000","last_change":"2023-12-01T01:52:48.042643+0000","last_active":"2023-12-01T01:52:48.509529+0000","last_peered":"2023-12-01T01:52:48.509529+0000","last_clean":"2023-12-01T01:52:48.509529+0000","last_became_active":"2023-12-01T01:52:48.042174+0000","last_became_peered":"2023-12-01T01:52:48.042174+0000","last_unstale":"2023-12-01T01:52:48.509529+0000","last_undegraded":"2023-12-01T01:52:48.509529+0000","last_fullsized":"2023-12-01T01:52:48.509529+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:59.436113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.522735+0000","last_change":"2023-12-01T01:52:48.520519+0000","last_active":"2023-12-01T01:52:48.522735+0000","last_peered":"2023-12-01T01:52:48.522735+0000","last_clean":"2023-12-01T01:52:48.522735+0000","last_became_active":"2023-12-01T01:52:48.520259+0000","last_became_peered":"2023-12-01T01:52:48.520259+0000","last_unstale":"2023-12-01T01:52:48.522735+0000","last_undegraded":"2023-12-01T01:52:48.522735+0000","last_fullsized":"2023-12-01T01:52:48.522735+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:49.055372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.502961+0000","last_change":"2023-12-01T01:52:48.047235+0000","last_active":"2023-12-01T01:52:48.502961+0000","last_peered":"2023-12-01T01:52:48.502961+0000","last_clean":"2023-12-01T01:52:48.502961+0000","last_became_active":"2023-12-01T01:52:48.046953+0000","last_became_peered":"2023-12-01T01:52:48.046953+0000","last_unstale":"2023-12-01T01:52:48.502961+0000","last_undegraded":"2023-12-01T01:52:48.502961+0000","last_fullsized":"2023-12-01T01:52:48.502961+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:13.492174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.513572+0000","last_change":"2023-12-01T01:52:48.509005+0000","last_active":"2023-12-01T01:52:48.513572+0000","last_peered":"2023-12-01T01:52:48.513572+0000","last_clean":"2023-12-01T01:52:48.513572+0000","last_became_active":"2023-12-01T01:52:48.508718+0000","last_became_peered":"2023-12-01T01:52:48.508718+0000","last_unstale":"2023-12-01T01:52:48.513572+0000","last_undegraded":"2023-12-01T01:52:48.513572+0000","last_fullsized":"2023-12-01T01:52:48.513572+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:39.608737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.535636+0000","last_change":"2023-12-01T01:52:48.050705+0000","last_active":"2023-12-01T01:52:48.535636+0000","last_peered":"2023-12-01T01:52:48.535636+0000","last_clean":"2023-12-01T01:52:48.535636+0000","last_became_active":"2023-12-01T01:52:48.050445+0000","last_became_peered":"2023-12-01T01:52:48.050445+0000","last_unstale":"2023-12-01T01:52:48.535636+0000","last_undegraded":"2023-12-01T01:52:48.535636+0000","last_fullsized":"2023-12-01T01:52:48.535636+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:02:56.065815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523934+0000","last_change":"2023-12-01T01:52:48.519761+0000","last_active":"2023-12-01T01:52:48.523934+0000","last_peered":"2023-12-01T01:52:48.523934+0000","last_clean":"2023-12-01T01:52:48.523934+0000","last_became_active":"2023-12-01T01:52:48.517974+0000","last_became_peered":"2023-12-01T01:52:48.517974+0000","last_unstale":"2023-12-01T01:52:48.523934+0000","last_undegraded":"2023-12-01T01:52:48.523934+0000","last_fullsized":"2023-12-01T01:52:48.523934+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:55:14.304687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.526121+0000","last_change":"2023-12-01T01:52:48.040799+0000","last_active":"2023-12-01T01:52:48.526121+0000","last_peered":"2023-12-01T01:52:48.526121+0000","last_clean":"2023-12-01T01:52:48.526121+0000","last_became_active":"2023-12-01T01:52:48.040471+0000","last_became_peered":"2023-12-01T01:52:48.040471+0000","last_unstale":"2023-12-01T01:52:48.526121+0000","last_undegraded":"2023-12-01T01:52:48.526121+0000","last_fullsized":"2023-12-01T01:52:48.526121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:27.670048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.517427+0000","last_change":"2023-12-01T01:52:48.515269+0000","last_active":"2023-12-01T01:52:48.517427+0000","last_peered":"2023-12-01T01:52:48.517427+0000","last_clean":"2023-12-01T01:52:48.517427+0000","last_became_active":"2023-12-01T01:52:48.514457+0000","last_became_peered":"2023-12-01T01:52:48.514457+0000","last_unstale":"2023-12-01T01:52:48.517427+0000","last_undegraded":"2023-12-01T01:52:48.517427+0000","last_fullsized":"2023-12-01T01:52:48.517427+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:35:01.791105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.510349+0000","last_change":"2023-12-01T01:52:48.043497+0000","last_active":"2023-12-01T01:52:48.510349+0000","last_peered":"2023-12-01T01:52:48.510349+0000","last_clean":"2023-12-01T01:52:48.510349+0000","last_became_active":"2023-12-01T01:52:48.043198+0000","last_became_peered":"2023-12-01T01:52:48.043198+0000","last_unstale":"2023-12-01T01:52:48.510349+0000","last_undegraded":"2023-12-01T01:52:48.510349+0000","last_fullsized":"2023-12-01T01:52:48.510349+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:44.930335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.527813+0000","last_change":"2023-12-01T01:52:48.525650+0000","last_active":"2023-12-01T01:52:48.527813+0000","last_peered":"2023-12-01T01:52:48.527813+0000","last_clean":"2023-12-01T01:52:48.527813+0000","last_became_active":"2023-12-01T01:52:48.525192+0000","last_became_peered":"2023-12-01T01:52:48.525192+0000","last_unstale":"2023-12-01T01:52:48.527813+0000","last_undegraded":"2023-12-01T01:52:48.527813+0000","last_fullsized":"2023-12-01T01:52:48.527813+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:03.437505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.513774+0000","last_change":"2023-12-01T01:52:48.046182+0000","last_active":"2023-12-01T01:52:48.513774+0000","last_peered":"2023-12-01T01:52:48.513774+0000","last_clean":"2023-12-01T01:52:48.513774+0000","last_became_active":"2023-12-01T01:52:48.045696+0000","last_became_peered":"2023-12-01T01:52:48.045696+0000","last_unstale":"2023-12-01T01:52:48.513774+0000","last_undegraded":"2023-12-01T01:52:48.513774+0000","last_fullsized":"2023-12-01T01:52:48.513774+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:35:44.829155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523516+0000","last_change":"2023-12-01T01:52:48.519955+0000","last_active":"2023-12-01T01:52:48.523516+0000","last_peered":"2023-12-01T01:52:48.523516+0000","last_clean":"2023-12-01T01:52:48.523516+0000","last_became_active":"2023-12-01T01:52:48.519705+0000","last_became_peered":"2023-12-01T01:52:48.519705+0000","last_unstale":"2023-12-01T01:52:48.523516+0000","last_undegraded":"2023-12-01T01:52:48.523516+0000","last_fullsized":"2023-12-01T01:52:48.523516+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:34:49.188317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.505474+0000","last_change":"2023-12-01T01:52:48.042910+0000","last_active":"2023-12-01T01:52:48.505474+0000","last_peered":"2023-12-01T01:52:48.505474+0000","last_clean":"2023-12-01T01:52:48.505474+0000","last_became_active":"2023-12-01T01:52:48.041897+0000","last_became_peered":"2023-12-01T01:52:48.041897+0000","last_unstale":"2023-12-01T01:52:48.505474+0000","last_undegraded":"2023-12-01T01:52:48.505474+0000","last_fullsized":"2023-12-01T01:52:48.505474+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:40:21.639171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.532905+0000","last_change":"2023-12-01T01:52:48.525217+0000","last_active":"2023-12-01T01:52:48.532905+0000","last_peered":"2023-12-01T01:52:48.532905+0000","last_clean":"2023-12-01T01:52:48.532905+0000","last_became_active":"2023-12-01T01:52:48.524951+0000","last_became_peered":"2023-12-01T01:52:48.524951+0000","last_unstale":"2023-12-01T01:52:48.532905+0000","last_undegraded":"2023-12-01T01:52:48.532905+0000","last_fullsized":"2023-12-01T01:52:48.532905+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:45.934623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.515888+0000","last_change":"2023-12-01T01:52:48.044493+0000","last_active":"2023-12-01T01:52:48.515888+0000","last_peered":"2023-12-01T01:52:48.515888+0000","last_clean":"2023-12-01T01:52:48.515888+0000","last_became_active":"2023-12-01T01:52:48.044221+0000","last_became_peered":"2023-12-01T01:52:48.044221+0000","last_unstale":"2023-12-01T01:52:48.515888+0000","last_undegraded":"2023-12-01T01:52:48.515888+0000","last_fullsized":"2023-12-01T01:52:48.515888+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:59.436113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.523313+0000","last_change":"2023-12-01T01:52:48.517859+0000","last_active":"2023-12-01T01:52:48.523313+0000","last_peered":"2023-12-01T01:52:48.523313+0000","last_clean":"2023-12-01T01:52:48.523313+0000","last_became_active":"2023-12-01T01:52:48.517615+0000","last_became_peered":"2023-12-01T01:52:48.517615+0000","last_unstale":"2023-12-01T01:52:48.523313+0000","last_undegraded":"2023-12-01T01:52:48.523313+0000","last_fullsized":"2023-12-01T01:52:48.523313+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:20:23.997177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.504533+0000","last_change":"2023-12-01T01:52:48.045085+0000","last_active":"2023-12-01T01:52:48.504533+0000","last_peered":"2023-12-01T01:52:48.504533+0000","last_clean":"2023-12-01T01:52:48.504533+0000","last_became_active":"2023-12-01T01:52:48.044789+0000","last_became_peered":"2023-12-01T01:52:48.044789+0000","last_unstale":"2023-12-01T01:52:48.504533+0000","last_undegraded":"2023-12-01T01:52:48.504533+0000","last_fullsized":"2023-12-01T01:52:48.504533+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:58:00.466321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.522830+0000","last_change":"2023-12-01T01:52:48.513454+0000","last_active":"2023-12-01T01:52:48.522830+0000","last_peered":"2023-12-01T01:52:48.522830+0000","last_clean":"2023-12-01T01:52:48.522830+0000","last_became_active":"2023-12-01T01:52:48.513192+0000","last_became_peered":"2023-12-01T01:52:48.513192+0000","last_unstale":"2023-12-01T01:52:48.522830+0000","last_undegraded":"2023-12-01T01:52:48.522830+0000","last_fullsized":"2023-12-01T01:52:48.522830+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:02:57.070103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.503936+0000","last_change":"2023-12-01T01:52:48.034635+0000","last_active":"2023-12-01T01:52:48.503936+0000","last_peered":"2023-12-01T01:52:48.503936+0000","last_clean":"2023-12-01T01:52:48.503936+0000","last_became_active":"2023-12-01T01:52:48.034401+0000","last_became_peered":"2023-12-01T01:52:48.034401+0000","last_unstale":"2023-12-01T01:52:48.503936+0000","last_undegraded":"2023-12-01T01:52:48.503936+0000","last_fullsized":"2023-12-01T01:52:48.503936+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:19.195130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.529281+0000","last_change":"2023-12-01T01:52:48.519960+0000","last_active":"2023-12-01T01:52:48.529281+0000","last_peered":"2023-12-01T01:52:48.529281+0000","last_clean":"2023-12-01T01:52:48.529281+0000","last_became_active":"2023-12-01T01:52:48.518314+0000","last_became_peered":"2023-12-01T01:52:48.518314+0000","last_unstale":"2023-12-01T01:52:48.529281+0000","last_undegraded":"2023-12-01T01:52:48.529281+0000","last_fullsized":"2023-12-01T01:52:48.529281+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:48.003394+0000","last_clean_scrub_stamp":"2023-12-01T01:52:48.003394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:54.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.536821+0000","last_change":"2023-12-01T01:52:48.043879+0000","last_active":"2023-12-01T01:52:48.536821+0000","last_peered":"2023-12-01T01:52:48.536821+0000","last_clean":"2023-12-01T01:52:48.536821+0000","last_became_active":"2023-12-01T01:52:48.043357+0000","last_became_peered":"2023-12-01T01:52:48.043357+0000","last_unstale":"2023-12-01T01:52:48.536821+0000","last_undegraded":"2023-12-01T01:52:48.536821+0000","last_fullsized":"2023-12-01T01:52:48.536821+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:52:46.999106+0000","last_clean_scrub_stamp":"2023-12-01T01:52:46.999106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:59.497221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"99'183","reported_seq":483,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-01T01:52:48.501694+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:52:48.501694+0000","last_peered":"2023-12-01T01:52:48.501694+0000","last_clean":"2023-12-01T01:52:48.501694+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:52:48.501694+0000","last_undegraded":"2023-12-01T01:52:48.501694+0000","last_fullsized":"2023-12-01T01:52:48.501694+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575052,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28288,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343552,"statfs":{"total":96636764160,"available":96607797248,"internally_reserved":0,"allocated":1368064,"data_stored":648917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.02,"15min":1.0660000000000001},"min":{"1min":0.54900000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.401,"5min":1.514,"15min":1.8089999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0309999999999999,"15min":1.0469999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5289999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.90300000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1499999999999999,"15min":1.2070000000000001},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.56100000000000005},"max":{"1min":1.5669999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2150000000000001,"15min":1.248},"min":{"1min":0.66800000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6279999999999999,"5min":1.744,"15min":1.744},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1040000000000001,"15min":1.117},"min":{"1min":0.749,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4910000000000001,"5min":1.607,"15min":1.728},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.2,"5min":1.2010000000000001,"15min":1.2190000000000001},"min":{"1min":0.92500000000000004,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.641,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.0700000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.1910000000000001,"15min":1.2230000000000001},"min":{"1min":0.64200000000000002,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.347,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":1.246},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3180000000000001,"15min":1.3420000000000001},"min":{"1min":0.63700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7490000000000001,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.875}]},{"osd":4,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.3009999999999999,"15min":1.2849999999999999},"min":{"1min":0.72699999999999998,"5min":0.56100000000000005,"15min":0.55500000000000005},"max":{"1min":1.4450000000000001,"5min":1.7549999999999999,"15min":2.2130000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4350000000000001,"15min":1.4059999999999999},"min":{"1min":0.77500000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8100000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3129999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.292,"15min":1.258},"min":{"1min":0.625,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.0379999999999998,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.351},{"interface":"front","average":{"1min":1.47,"5min":1.514,"15min":1.514},"min":{"1min":0.69399999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.861,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4339999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:52:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.395,"15min":1.377},"min":{"1min":0.753,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1549999999999998,"5min":2.294,"15min":2.3420000000000001},"last":1.139},{"interface":"front","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.387},"min":{"1min":0.82599999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.893,"5min":1.893,"15min":2.3500000000000001},"last":1.4590000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575053,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1372160,"data_stored":649519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.1020000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.79},"last":1.204},{"interface":"front","average":{"1min":1.036,"5min":1.105,"15min":1.1220000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5349999999999999,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.49}]},{"osd":1,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.161,"15min":1.159},"min":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1950000000000001,"15min":1.2050000000000001},"min":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0920000000000001},"last":1.3460000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1859999999999999,"15min":1.171},"min":{"1min":0.69699999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5760000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.242,"15min":1.2350000000000001},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":3.024,"5min":3.024,"15min":3.024},"last":1.599}]},{"osd":3,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":1.0289999999999999,"5min":0.93700000000000006,"15min":0.82399999999999995},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.4419999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3,"15min":1.2709999999999999},"min":{"1min":0.94299999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6819999999999999,"5min":2.226,"15min":2.226},"last":1.016},{"interface":"front","average":{"1min":1.339,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":1.028,"5min":0.91400000000000003,"15min":0.78100000000000003},"max":{"1min":1.885,"5min":1.931,"15min":2.2839999999999998},"last":1.726}]},{"osd":5,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.415,"15min":1.397},"min":{"1min":0.755,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.74,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.7529999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.502,"15min":1.472},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.84099999999999997},"max":{"1min":1.9790000000000001,"5min":2.8580000000000001,"15min":2.8580000000000001},"last":1.669}]},{"osd":7,"last update":"Fri Dec 1 01:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.524,"15min":1.5009999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6870000000000001,"5min":2.762,"15min":2.762},"last":1.923},{"interface":"front","average":{"1min":1.488,"5min":1.577,"15min":1.5629999999999999},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":1.9330000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":1.55}]}]},{"osd":5,"up_from":13,"seq":55834575052,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1928,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1974272,"data_stored":1240813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1259999999999999,"15min":1.1040000000000001},"min":{"1min":0.63200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.2010000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1180000000000001,"15min":1.143},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.60099999999999998},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":2.153},"last":1.2509999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.117,"15min":1.1399999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.63200000000000001},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.171},{"interface":"front","average":{"1min":1.105,"5min":1.1399999999999999,"15min":1.159},"min":{"1min":0.72999999999999998,"5min":0.67800000000000005,"15min":0.64900000000000002},"max":{"1min":1.6519999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.3480000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.173,"15min":1.196},"min":{"1min":0.65000000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.702,"5min":1.702,"15min":1.7110000000000001},"last":1.004},{"interface":"front","average":{"1min":1.298,"5min":1.208,"15min":1.2330000000000001},"min":{"1min":0.79100000000000004,"5min":0.77300000000000002,"15min":0.624},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":2.0630000000000002},"last":1.069}]},{"osd":3,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.248,"15min":1.282},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.8169999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.302},"min":{"1min":1.159,"5min":0.91200000000000003,"15min":0.71699999999999997},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.302}]},{"osd":4,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2490000000000001,"15min":1.2410000000000001},"min":{"1min":0.83499999999999996,"5min":0.72999999999999998,"15min":0.67600000000000005},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.319,"15min":1.3480000000000001},"min":{"1min":0.84399999999999997,"5min":0.80300000000000005,"15min":0.79900000000000004},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.3300000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3129999999999999,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.859,"5min":1.885,"15min":1.984},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.3939999999999999,"15min":1.3839999999999999},"min":{"1min":0.95699999999999996,"5min":0.93500000000000005,"15min":0.78100000000000003},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.6140000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.462,"15min":1.4750000000000001},"min":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0920000000000001},"last":1.9319999999999999},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.522,"15min":1.5269999999999999},"min":{"1min":1.304,"5min":0.873,"15min":0.873},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.79}]}]},{"osd":4,"up_from":13,"seq":55834575052,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36480,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335360,"statfs":{"total":96636764160,"available":96599408640,"internally_reserved":0,"allocated":1368064,"data_stored":648881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.1120000000000001,"15min":1.105},"min":{"1min":0.72199999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.753,"5min":0.66900000000000004,"15min":0.65100000000000002},"max":{"1min":1.7130000000000001,"5min":1.7529999999999999,"15min":2.1099999999999999},"last":1.28}]},{"osd":1,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.216,"15min":1.204},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.65900000000000003},"max":{"1min":1.5409999999999999,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.087,"5min":1.101,"15min":1.129},"min":{"1min":0.76600000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3979999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.93799999999999994}]},{"osd":2,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.234,"15min":1.2},"min":{"1min":0.80900000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.716,"5min":1.871,"15min":1.871},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.3180000000000001,"15min":1.288},"min":{"1min":0.94599999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.581,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.258,"15min":1.2310000000000001},"min":{"1min":0.81899999999999995,"5min":0.77000000000000002,"15min":0.67900000000000005},"max":{"1min":1.6140000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3200000000000001,"15min":1.3160000000000001},"min":{"1min":0.88500000000000001,"5min":0.753,"15min":0.753},"max":{"1min":1.712,"5min":1.7270000000000001,"15min":1.903},"last":0.88500000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.319,"15min":1.3129999999999999},"min":{"1min":0.94899999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.288,"15min":1.2869999999999999},"min":{"1min":0.97199999999999998,"5min":0.90300000000000002,"15min":0.85399999999999998},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.151}]},{"osd":6,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.381,"15min":1.3500000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.81299999999999994},"max":{"1min":1.8799999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.502,"15min":1.472},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.0960000000000001,"5min":2.4990000000000001,"15min":2.54},"last":1.3600000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.472,"15min":1.4610000000000001},"min":{"1min":1.274,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.306},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5780000000000001,"15min":1.5620000000000001},"min":{"1min":1.28,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":1.96,"5min":2.04,"15min":2.157},"last":1.395}]}]},{"osd":3,"up_from":13,"seq":55834575051,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58632,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313208,"statfs":{"total":96636764160,"available":96576724992,"internally_reserved":0,"allocated":1376256,"data_stored":649521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.84199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.307},"max":{"1min":1.1899999999999999,"5min":1.3320000000000001,"15min":1.583},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.93500000000000005,"15min":0.97999999999999998},"min":{"1min":0.48099999999999998,"5min":0.435,"15min":0.40999999999999998},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.86699999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.99199999999999999,"15min":1.056},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.38900000000000001},"max":{"1min":1.5669999999999999,"5min":1.571,"15min":2.0779999999999998},"last":0.754},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.89300000000000002,"15min":0.94799999999999995},"min":{"1min":0.48399999999999999,"5min":0.48199999999999998,"15min":0.42699999999999999},"max":{"1min":1.1339999999999999,"5min":1.55,"15min":1.6519999999999999},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.96699999999999997,"15min":1.0349999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3360000000000001,"5min":1.4810000000000001,"15min":1.5609999999999999},"last":1.079},{"interface":"front","average":{"1min":0.995,"5min":1.125,"15min":1.1819999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.024}]},{"osd":4,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1970000000000001,"15min":1.232},"min":{"1min":0.84999999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.524,"5min":1.5820000000000001,"15min":1.9319999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.198,"15min":1.232},"min":{"1min":0.79300000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.909,"5min":1.909,"15min":2.0419999999999998},"last":1.2050000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.228,"15min":1.2789999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.71,"5min":1.7669999999999999,"15min":2.0510000000000002},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2529999999999999,"15min":1.288},"min":{"1min":0.745,"5min":0.745,"15min":0.64200000000000002},"max":{"1min":1.6759999999999999,"5min":1.681,"15min":2.1880000000000002},"last":1.702}]},{"osd":6,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.89600000000000002,"5min":0.76300000000000001,"15min":0.68300000000000005},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.3700000000000001,"15min":1.405},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8260000000000001,"5min":1.9119999999999999,"15min":1.99},"last":1.758}]},{"osd":7,"last update":"Fri Dec 1 01:52:30 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.365,"15min":1.413},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.81499999999999995},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.1619999999999999},"last":1.359},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4019999999999999,"15min":1.4470000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.83099999999999996},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.2130000000000001},"last":1.4399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575051,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36476,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335364,"statfs":{"total":96636764160,"available":96599412736,"internally_reserved":0,"allocated":1363968,"data_stored":648847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":0.81699999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.5529999999999999},"last":0.748},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.98399999999999999,"15min":1},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95799999999999996,"15min":0.98699999999999999},"min":{"1min":0.54900000000000004,"5min":0.373,"15min":0.373},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":1.0600000000000001,"15min":1.081},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.93899999999999995,"15min":1.002},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5189999999999999,"5min":1.625,"15min":1.6479999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.98799999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.869,"5min":1.9950000000000001,"15min":2.024},"last":0.872},{"interface":"front","average":{"1min":1.111,"5min":1.202,"15min":1.258},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.964},"last":0.91800000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3,"15min":1.3129999999999999},"min":{"1min":0.72199999999999998,"5min":0.62,"15min":0.62},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.357,"15min":1.353},"min":{"1min":0.70099999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.016,"5min":2.048,"15min":2.323},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2849999999999999,"15min":1.3180000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.579,"5min":1.78,"15min":2.2799999999999998},"last":1.014},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.90300000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.3680000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.4410000000000001,"15min":1.4330000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.448},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3440000000000001,"15min":1.381},"min":{"1min":0.80600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":2.1080000000000001},"last":1.075}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59204,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312636,"statfs":{"total":96636764160,"available":96576139264,"internally_reserved":0,"allocated":1961984,"data_stored":1239305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.80400000000000005,"15min":0.83499999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.318},"max":{"1min":1.1639999999999999,"5min":1.341,"15min":1.5049999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.94999999999999996,"15min":1.0069999999999999},"min":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.597,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85299999999999998,"15min":0.88100000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.54},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.0349999999999999,"15min":1.081},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.44800000000000001},"max":{"1min":1.472,"5min":1.7250000000000001,"15min":1.804},"last":1.089}]},{"osd":3,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.5},"max":{"1min":1.718,"5min":1.827,"15min":1.827},"last":1.5129999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1,"15min":1.038},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.39700000000000002},"max":{"1min":1.381,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.4570000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1579999999999999,"15min":1.173},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.65300000000000002},"max":{"1min":1.425,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.361},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.288,"15min":1.3240000000000001},"min":{"1min":0.85599999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.863},"last":1.3169999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":0.77700000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":1.76},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.244,"15min":1.27},"min":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":2.1619999999999999},"last":1.1910000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.294,"15min":1.3220000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.68400000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.861},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.306,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.742,"5min":1.7490000000000001,"15min":1.8839999999999999},"last":1.5820000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3720000000000001,"15min":1.417},"min":{"1min":0.91200000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.782,"5min":1.782,"15min":2.0609999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3859999999999999,"15min":1.399},"min":{"1min":0.80900000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.931,"5min":1.992,"15min":1.992},"last":1.278}]}]},{"osd":0,"up_from":13,"seq":55834575052,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1380352,"data_stored":650433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84099999999999997,"15min":0.85399999999999998},"min":{"1min":0.435,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":1.194,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89900000000000002,"15min":0.96499999999999997},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3859999999999999,"5min":1.544,"15min":1.6910000000000001},"last":0.71299999999999997}]},{"osd":2,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.94799999999999995,"15min":0.998},"min":{"1min":0.51500000000000001,"5min":0.42699999999999999,"15min":0.41999999999999998},"max":{"1min":1.431,"5min":2.157,"15min":2.157},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.94999999999999996,"5min":1.0089999999999999,"15min":1.0649999999999999},"min":{"1min":0.57399999999999995,"5min":0.46400000000000002,"15min":0.38400000000000001},"max":{"1min":1.3400000000000001,"5min":1.728,"15min":1.728},"last":1.238}]},{"osd":3,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":1.0149999999999999},"min":{"1min":0.623,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.954},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.1080000000000001,"15min":1.1539999999999999},"min":{"1min":0.54200000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4199999999999999,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.90400000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2589999999999999,"15min":1.3029999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5840000000000001,"5min":1.9379999999999999,"15min":2.3719999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1899999999999999,"15min":1.258},"min":{"1min":0.90200000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.55,"5min":1.7989999999999999,"15min":2.0680000000000001},"last":1.02}]},{"osd":5,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.323,"15min":1.343},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.71899999999999997},"max":{"1min":1.841,"5min":2.351,"15min":2.351},"last":1.323},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2769999999999999,"15min":1.333},"min":{"1min":0.81599999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.546,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.083}]},{"osd":6,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2789999999999999,"15min":1.319},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7529999999999999,"5min":1.8680000000000001,"15min":2.157},"last":1.119},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.351,"15min":1.3839999999999999},"min":{"1min":0.94499999999999995,"5min":0.81699999999999995,"15min":0.81000000000000005},"max":{"1min":1.7809999999999999,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.2689999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.339,"15min":1.3979999999999999},"min":{"1min":0.94999999999999996,"5min":0.77300000000000002,"15min":0.74199999999999999},"max":{"1min":1.5700000000000001,"5min":1.798,"15min":2.4279999999999999},"last":1.302},{"interface":"front","average":{"1min":1.25,"5min":1.3320000000000001,"15min":1.3899999999999999},"min":{"1min":0.98999999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.667,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.0700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:52:58.476 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T01:52:58.477 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T01:52:58.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.739+0000 7f51c2e77640 1 Processor -- start 2023-12-01T01:52:58.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.740+0000 7f51c2e77640 1 -- start start 2023-12-01T01:52:58.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.741+0000 7f51c2e77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc15ced0 0x7f51bc0b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.741+0000 7f51c2e77640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f51bc0d2750 0x7f51bc0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.741+0000 7f51c2e77640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51bc0d1480 con 0x7f51bc0b4660 2023-12-01T01:52:58.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.741+0000 7f51c2e77640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc0d16b0 con 0x7f51bc0d2750 2023-12-01T01:52:58.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.741+0000 7f51c2e77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0d1990 con 0x7f51bc15ced0 2023-12-01T01:52:58.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.741+0000 7f51bbfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f51bc0d2750 0x7f51bc0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.741+0000 7f51c1414640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f51bc0b4660 0x7f51bc069cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57200/0 (socket says 172.21.15.115:57200) 2023-12-01T01:52:58.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.741+0000 7f51c1414640 1 -- 172.21.15.115:0/3637082243 learned_addr learned my addr 172.21.15.115:0/3637082243 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:52:58.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.742+0000 7f51bbfff640 1 -- 172.21.15.115:0/3637082243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc15ced0 msgr2=0x7f51bc0b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:58.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.742+0000 7f51c0c13640 1 --2- 172.21.15.115:0/3637082243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc15ced0 0x7f51bc0b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.742+0000 7f51bb7fe640 1 -- 172.21.15.115:0/3637082243 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010916704 0 0) 0x7f51bc0d1480 con 0x7f51bc0b4660 2023-12-01T01:52:58.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.742+0000 7f51bbfff640 1 --2- 172.21.15.115:0/3637082243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc15ced0 0x7f51bc0b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.742+0000 7f51bbfff640 1 -- 172.21.15.115:0/3637082243 >> v1:172.21.15.115:6789/0 conn(0x7f51bc0b4660 legacy=0x7f51bc069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:58.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.742+0000 7f51bbfff640 1 -- 172.21.15.115:0/3637082243 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0d1c70 con 0x7f51bc0d2750 2023-12-01T01:52:58.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.742+0000 7f51c0c13640 1 --2- 172.21.15.115:0/3637082243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc15ced0 0x7f51bc0b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:52:58.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.743+0000 7f51bbfff640 1 --2- 172.21.15.115:0/3637082243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0d2750 0x7f51bc0d0f40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f51a40097a0 tx=0x7f51a402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4c226981a16488 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.743+0000 7f51bb7fe640 1 -- 172.21.15.115:0/3637082243 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a402d040 con 0x7f51bc0d2750 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.743+0000 7f51bb7fe640 1 -- 172.21.15.115:0/3637082243 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a4031810 con 0x7f51bc0d2750 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.743+0000 7f51bb7fe640 1 -- 172.21.15.115:0/3637082243 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4031ba0 con 0x7f51bc0d2750 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.744+0000 7f51c2e77640 1 -- 172.21.15.115:0/3637082243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0d2750 msgr2=0x7f51bc0d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.744+0000 7f51c2e77640 1 --2- 172.21.15.115:0/3637082243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0d2750 0x7f51bc0d0f40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f51a40097a0 tx=0x7f51a402b8b0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.744+0000 7f51c2e77640 1 -- 172.21.15.115:0/3637082243 shutdown_connections 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.744+0000 7f51c2e77640 1 --2- 172.21.15.115:0/3637082243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0d2750 0x7f51bc0d0f40 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.744+0000 7f51c2e77640 1 --2- 172.21.15.115:0/3637082243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc15ced0 0x7f51bc0b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.744+0000 7f51c2e77640 1 -- 172.21.15.115:0/3637082243 >> 172.21.15.115:0/3637082243 conn(0x7f51bc065f90 msgr2=0x7f51bc0cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.744+0000 7f51c2e77640 1 -- 172.21.15.115:0/3637082243 shutdown_connections 2023-12-01T01:52:58.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.744+0000 7f51c2e77640 1 -- 172.21.15.115:0/3637082243 wait complete. 2023-12-01T01:52:58.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.746+0000 7f51c2e77640 1 Processor -- start 2023-12-01T01:52:58.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.746+0000 7f51c2e77640 1 -- start start 2023-12-01T01:52:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.747+0000 7f51c2e77640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0b4660 0x7f51bc156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.747+0000 7f51c2e77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc0d2750 0x7f51bc1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.747+0000 7f51c2e77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51bc15ced0 0x7f51bc151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.747+0000 7f51c2e77640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0c8440 con 0x7f51bc0b4660 2023-12-01T01:52:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.747+0000 7f51c2e77640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc156d40 con 0x7f51bc15ced0 2023-12-01T01:52:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.747+0000 7f51c2e77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc1520d0 con 0x7f51bc0d2750 2023-12-01T01:52:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.747+0000 7f51c0c13640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0b4660 0x7f51bc156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.747+0000 7f51c0c13640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0b4660 0x7f51bc156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50346/0 (socket says 172.21.15.115:50346) 2023-12-01T01:52:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.747+0000 7f51c0c13640 1 -- 172.21.15.115:0/260540277 learned_addr learned my addr 172.21.15.115:0/260540277 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:58.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51bbfff640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc0d2750 0x7f51bc1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51c0c13640 1 -- 172.21.15.115:0/260540277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc0d2750 msgr2=0x7f51bc1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:58.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51c0c13640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc0d2750 0x7f51bc1549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51c0c13640 1 -- 172.21.15.115:0/260540277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51bc15ced0 msgr2=0x7f51bc151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:58.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51c0c13640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51bc15ced0 0x7f51bc151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:58.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51c0c13640 1 -- 172.21.15.115:0/260540277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4009400 con 0x7f51bc0b4660 2023-12-01T01:52:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51c0c13640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0b4660 0x7f51bc156800 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00ce00 tx=0x7f51ac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51b97fa640 1 -- 172.21.15.115:0/260540277 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00f890 con 0x7f51bc0b4660 2023-12-01T01:52:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51b97fa640 1 -- 172.21.15.115:0/260540277 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac00f9f0 con 0x7f51bc0b4660 2023-12-01T01:52:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51b97fa640 1 -- 172.21.15.115:0/260540277 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00fdc0 con 0x7f51bc0b4660 2023-12-01T01:52:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.748+0000 7f51c2e77640 1 -- 172.21.15.115:0/260540277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc1523b0 con 0x7f51bc0b4660 2023-12-01T01:52:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.749+0000 7f51c2e77640 1 -- 172.21.15.115:0/260540277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc0c80d0 con 0x7f51bc0b4660 2023-12-01T01:52:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.750+0000 7f51c2e77640 1 -- 172.21.15.115:0/260540277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc0c7c50 con 0x7f51bc0b4660 2023-12-01T01:52:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.750+0000 7f51b97fa640 1 -- 172.21.15.115:0/260540277 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f51ac00b040 con 0x7f51bc0b4660 2023-12-01T01:52:58.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.751+0000 7f51b97fa640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f519c06a550 0x7f519c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:58.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.751+0000 7f51b97fa640 1 -- 172.21.15.115:0/260540277 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51ac0895e0 con 0x7f51bc0b4660 2023-12-01T01:52:58.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.752+0000 7f51bbfff640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f519c06a550 0x7f519c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:58.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.752+0000 7f51bbfff640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f519c06a550 0x7f519c06ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f51a4036000 tx=0x7f51a4008030 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:58.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:58.755+0000 7f51b97fa640 1 -- 172.21.15.115:0/260540277 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f51bc0c7c50 con 0x7f51bc0b4660 2023-12-01T01:52:59.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.018+0000 7f51c2e77640 1 -- 172.21.15.115:0/260540277 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f51bc0c7c50 con 0x7f51bc0b4660 2023-12-01T01:52:59.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.028+0000 7f51b97fa640 1 -- 172.21.15.115:0/260540277 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f51bc0c7c50 con 0x7f51bc0b4660 2023-12-01T01:52:59.029 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T01:52:59.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 -- 172.21.15.115:0/260540277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f519c06a550 msgr2=0x7f519c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:59.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f519c06a550 0x7f519c06ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f51a4036000 tx=0x7f51a4008030 comp rx=0 tx=0).stop 2023-12-01T01:52:59.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 -- 172.21.15.115:0/260540277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0b4660 msgr2=0x7f51bc156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:59.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0b4660 0x7f51bc156800 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00ce00 tx=0x7f51ac00a650 comp rx=0 tx=0).stop 2023-12-01T01:52:59.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 -- 172.21.15.115:0/260540277 shutdown_connections 2023-12-01T01:52:59.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f519c06a550 0x7f519c06ca10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51bc0d2750 0x7f51bc1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51bc15ced0 0x7f51bc151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 --2- 172.21.15.115:0/260540277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51bc0b4660 0x7f51bc156800 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 -- 172.21.15.115:0/260540277 >> 172.21.15.115:0/260540277 conn(0x7f51bc065f90 msgr2=0x7f51bc15abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:59.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 -- 172.21.15.115:0/260540277 shutdown_connections 2023-12-01T01:52:59.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.031+0000 7f51c2e77640 1 -- 172.21.15.115:0/260540277 wait complete. 2023-12-01T01:52:59.052 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:52:59.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.262+0000 7f1d9aa12640 1 Processor -- start 2023-12-01T01:52:59.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.262+0000 7f1d9aa12640 1 -- start start 2023-12-01T01:52:59.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.263+0000 7f1d9aa12640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d9415e2f0 0x7f1d940ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:59.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.263+0000 7f1d9aa12640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1d940adf10 0x7f1d940caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:59.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.263+0000 7f1d9aa12640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d940add10 con 0x7f1d9406f2f0 2023-12-01T01:52:59.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.263+0000 7f1d9aa12640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d940c5760 con 0x7f1d940adf10 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.263+0000 7f1d9aa12640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9415e6d0 con 0x7f1d9415e2f0 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.264+0000 7f1d98faf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1d940adf10 0x7f1d940caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.264+0000 7f1d98faf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1d940adf10 0x7f1d940caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50352/0 (socket says 172.21.15.115:50352) 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.264+0000 7f1d8bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1d9406f2f0 0x7f1d940c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57212/0 (socket says 172.21.15.115:57212) 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.264+0000 7f1d98faf640 1 -- 172.21.15.115:0/313493262 learned_addr learned my addr 172.21.15.115:0/313493262 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.264+0000 7f1d93fff640 1 --2- 172.21.15.115:0/313493262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d9415e2f0 0x7f1d940ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.264+0000 7f1d98faf640 1 -- 172.21.15.115:0/313493262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d9415e2f0 msgr2=0x7f1d940ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.264+0000 7f1d98faf640 1 --2- 172.21.15.115:0/313493262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d9415e2f0 0x7f1d940ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.264+0000 7f1d98faf640 1 -- 172.21.15.115:0/313493262 >> v1:172.21.15.115:6789/0 conn(0x7f1d9406f2f0 legacy=0x7f1d940c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.264+0000 7f1d98faf640 1 -- 172.21.15.115:0/313493262 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940cf7b0 con 0x7f1d940adf10 2023-12-01T01:52:59.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.264+0000 7f1d937fe640 1 -- 172.21.15.115:0/313493262 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202977146 0 0) 0x7f1d940add10 con 0x7f1d9406f2f0 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.265+0000 7f1d98faf640 1 --2- 172.21.15.115:0/313493262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 0x7f1d940caba0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009e50 tx=0x7f1d8c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c126f4294436e577 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.265+0000 7f1d937fe640 1 -- 172.21.15.115:0/313493262 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c02c9e0 con 0x7f1d940adf10 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.265+0000 7f1d937fe640 1 -- 172.21.15.115:0/313493262 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c02cb40 con 0x7f1d940adf10 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.265+0000 7f1d937fe640 1 -- 172.21.15.115:0/313493262 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c0343f0 con 0x7f1d940adf10 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.266+0000 7f1d9aa12640 1 -- 172.21.15.115:0/313493262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 msgr2=0x7f1d940caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.266+0000 7f1d9aa12640 1 --2- 172.21.15.115:0/313493262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 0x7f1d940caba0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009e50 tx=0x7f1d8c02be40 comp rx=0 tx=0).stop 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.266+0000 7f1d9aa12640 1 -- 172.21.15.115:0/313493262 shutdown_connections 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.266+0000 7f1d9aa12640 1 --2- 172.21.15.115:0/313493262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 0x7f1d940caba0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.266+0000 7f1d9aa12640 1 --2- 172.21.15.115:0/313493262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d9415e2f0 0x7f1d940ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.266+0000 7f1d9aa12640 1 -- 172.21.15.115:0/313493262 >> 172.21.15.115:0/313493262 conn(0x7f1d940ce880 msgr2=0x7f1d940cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.266+0000 7f1d9aa12640 1 -- 172.21.15.115:0/313493262 shutdown_connections 2023-12-01T01:52:59.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.266+0000 7f1d9aa12640 1 -- 172.21.15.115:0/313493262 wait complete. 2023-12-01T01:52:59.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.268+0000 7f1d9aa12640 1 Processor -- start 2023-12-01T01:52:59.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.268+0000 7f1d9aa12640 1 -- start start 2023-12-01T01:52:59.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.269+0000 7f1d9aa12640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d9406f2f0 0x7f1d94151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:59.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.269+0000 7f1d9aa12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 0x7f1d940cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:59.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.269+0000 7f1d9aa12640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9415e2f0 0x7f1d940ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:59.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.269+0000 7f1d9aa12640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940c3060 con 0x7f1d940adf10 2023-12-01T01:52:59.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.269+0000 7f1d9aa12640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940c34f0 con 0x7f1d9415e2f0 2023-12-01T01:52:59.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.269+0000 7f1d9aa12640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94152270 con 0x7f1d9406f2f0 2023-12-01T01:52:59.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.269+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 0x7f1d940cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:59.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.269+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 0x7f1d940cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50360/0 (socket says 172.21.15.115:50360) 2023-12-01T01:52:59.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.269+0000 7f1d8bfff640 1 -- 172.21.15.115:0/3711434417 learned_addr learned my addr 172.21.15.115:0/3711434417 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:52:59.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.269+0000 7f1d93fff640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d9406f2f0 0x7f1d94151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:59.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d8bfff640 1 -- 172.21.15.115:0/3711434417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d9406f2f0 msgr2=0x7f1d94151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:59.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d8bfff640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d9406f2f0 0x7f1d94151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d98faf640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9415e2f0 0x7f1d940ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:59.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d8bfff640 1 -- 172.21.15.115:0/3711434417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9415e2f0 msgr2=0x7f1d940ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:59.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d8bfff640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9415e2f0 0x7f1d940ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d8bfff640 1 -- 172.21.15.115:0/3711434417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c009b30 con 0x7f1d940adf10 2023-12-01T01:52:59.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d8bfff640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 0x7f1d940cdc60 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00c430 tx=0x7f1d7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:59.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d917fa640 1 -- 172.21.15.115:0/3711434417 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c0074e0 con 0x7f1d940adf10 2023-12-01T01:52:59.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d917fa640 1 -- 172.21.15.115:0/3711434417 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c007660 con 0x7f1d940adf10 2023-12-01T01:52:59.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d917fa640 1 -- 172.21.15.115:0/3711434417 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c004a20 con 0x7f1d940adf10 2023-12-01T01:52:59.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d9aa12640 1 -- 172.21.15.115:0/3711434417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d940c2450 con 0x7f1d940adf10 2023-12-01T01:52:59.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.270+0000 7f1d9aa12640 1 -- 172.21.15.115:0/3711434417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d940c2a10 con 0x7f1d940adf10 2023-12-01T01:52:59.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.271+0000 7f1d9aa12640 1 -- 172.21.15.115:0/3711434417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9406fda0 con 0x7f1d940adf10 2023-12-01T01:52:59.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.272+0000 7f1d917fa640 1 -- 172.21.15.115:0/3711434417 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1d7c004b80 con 0x7f1d940adf10 2023-12-01T01:52:59.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.273+0000 7f1d917fa640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1d7406a550 0x7f1d7406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:52:59.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.273+0000 7f1d917fa640 1 -- 172.21.15.115:0/3711434417 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d7c088950 con 0x7f1d940adf10 2023-12-01T01:52:59.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.273+0000 7f1d93fff640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1d7406a550 0x7f1d7406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:52:59.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.274+0000 7f1d93fff640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1d7406a550 0x7f1d7406ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1d84006fd0 tx=0x7f1d84008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:52:59.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.277+0000 7f1d917fa640 1 -- 172.21.15.115:0/3711434417 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1d7c053e20 con 0x7f1d940adf10 2023-12-01T01:52:59.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.505+0000 7f1d9aa12640 1 -- 172.21.15.115:0/3711434417 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d940ca110 con 0x7f1d940adf10 2023-12-01T01:52:59.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.506+0000 7f1d917fa640 1 -- 172.21.15.115:0/3711434417 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f1d7c057440 con 0x7f1d940adf10 2023-12-01T01:52:59.508 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:52:59.508 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3538792232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3538792232},{"type":"v1","addr":"172.21.15.142:6835","nonce":3538792232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7449,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3171906828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3171906828},{"type":"v1","addr":"172.21.15.115:6835","nonce":3171906828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7452,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2130806226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2130806226},{"type":"v1","addr":"172.21.15.115:6837","nonce":2130806226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:48.491310+0000","modified":"2023-12-01T01:52:51.513496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7455":{"gid":7455,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/3108729597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3108729597},{"type":"v1","addr":"172.21.15.142:6834","nonce":3108729597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-01T01:52:59.508 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 48 2023-12-01T01:52:59.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 -- 172.21.15.115:0/3711434417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1d7406a550 msgr2=0x7f1d7406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:59.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1d7406a550 0x7f1d7406ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1d84006fd0 tx=0x7f1d84008040 comp rx=0 tx=0).stop 2023-12-01T01:52:59.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 -- 172.21.15.115:0/3711434417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 msgr2=0x7f1d940cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:52:59.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 0x7f1d940cdc60 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00c430 tx=0x7f1d7c00c8f0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 -- 172.21.15.115:0/3711434417 shutdown_connections 2023-12-01T01:52:59.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1d7406a550 0x7f1d7406ca10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9415e2f0 0x7f1d940ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d940adf10 0x7f1d940cdc60 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 --2- 172.21.15.115:0/3711434417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d9406f2f0 0x7f1d94151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:52:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 -- 172.21.15.115:0/3711434417 >> 172.21.15.115:0/3711434417 conn(0x7f1d940ce880 msgr2=0x7f1d9415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:52:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 -- 172.21.15.115:0/3711434417 shutdown_connections 2023-12-01T01:52:59.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:52:59.509+0000 7f1d9aa12640 1 -- 172.21.15.115:0/3711434417 wait complete. 2023-12-01T01:52:59.528 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:52:59.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:52:59.933 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T01:52:48.491310+0000', 'modified': '2023-12-01T01:52:51.513496+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7455}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7455': {'gid': 7455, 'name': 'd', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6834/3108729597', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 3108729597}, {'type': 'v1', 'addr': '172.21.15.142:6834', 'nonce': 3108729597}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T01:52:59.933 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T01:52:59.933 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-01T01:53:00.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.151+0000 7f55e54af640 1 Processor -- start 2023-12-01T01:53:00.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.151+0000 7f55e54af640 1 -- start start 2023-12-01T01:53:00.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.153+0000 7f55e54af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55e015cd20 0x7f55e00d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.153+0000 7f55e54af640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f55e00d1230 0x7f55e00d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.153+0000 7f55e54af640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e00ce2e0 con 0x7f55e00b5d80 2023-12-01T01:53:00.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.153+0000 7f55e54af640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e00b2160 con 0x7f55e00d1230 2023-12-01T01:53:00.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.153+0000 7f55e54af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00d1be0 con 0x7f55e015cd20 2023-12-01T01:53:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.153+0000 7f55df7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f55e00d1230 0x7f55e00d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.153+0000 7f55de7fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f55e00b5d80 0x7f55e00b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57220/0 (socket says 172.21.15.115:57220) 2023-12-01T01:53:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.154+0000 7f55df7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f55e00d1230 0x7f55e00d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50366/0 (socket says 172.21.15.115:50366) 2023-12-01T01:53:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.154+0000 7f55de7fc640 1 -- 172.21.15.115:0/1837216178 learned_addr learned my addr 172.21.15.115:0/1837216178 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:53:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.154+0000 7f55deffd640 1 --2- 172.21.15.115:0/1837216178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55e015cd20 0x7f55e00d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.154+0000 7f55df7fe640 1 -- 172.21.15.115:0/1837216178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55e015cd20 msgr2=0x7f55e00d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.154+0000 7f55df7fe640 1 --2- 172.21.15.115:0/1837216178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55e015cd20 0x7f55e00d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.154+0000 7f55df7fe640 1 -- 172.21.15.115:0/1837216178 >> v1:172.21.15.115:6789/0 conn(0x7f55e00b5d80 legacy=0x7f55e00b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.154+0000 7f55df7fe640 1 -- 172.21.15.115:0/1837216178 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00ce000 con 0x7f55e00d1230 2023-12-01T01:53:00.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.154+0000 7f55ddffb640 1 -- 172.21.15.115:0/1837216178 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117023511 0 0) 0x7f55e00ce2e0 con 0x7f55e00b5d80 2023-12-01T01:53:00.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.155+0000 7f55df7fe640 1 --2- 172.21.15.115:0/1837216178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 0x7f55e00d16a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009d90 tx=0x7f55d802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922aef2fb8d1155a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.155+0000 7f55ddffb640 1 -- 172.21.15.115:0/1837216178 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d802cbc0 con 0x7f55e00d1230 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.155+0000 7f55ddffb640 1 -- 172.21.15.115:0/1837216178 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d802cd20 con 0x7f55e00d1230 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.155+0000 7f55ddffb640 1 -- 172.21.15.115:0/1837216178 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8034660 con 0x7f55e00d1230 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.156+0000 7f55e54af640 1 -- 172.21.15.115:0/1837216178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 msgr2=0x7f55e00d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.156+0000 7f55e54af640 1 --2- 172.21.15.115:0/1837216178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 0x7f55e00d16a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009d90 tx=0x7f55d802be10 comp rx=0 tx=0).stop 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.156+0000 7f55e54af640 1 -- 172.21.15.115:0/1837216178 shutdown_connections 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.156+0000 7f55e54af640 1 --2- 172.21.15.115:0/1837216178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 0x7f55e00d16a0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.156+0000 7f55e54af640 1 --2- 172.21.15.115:0/1837216178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55e015cd20 0x7f55e00d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.156+0000 7f55e54af640 1 -- 172.21.15.115:0/1837216178 >> 172.21.15.115:0/1837216178 conn(0x7f55e006a0e0 msgr2=0x7f55e006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.156+0000 7f55e54af640 1 -- 172.21.15.115:0/1837216178 shutdown_connections 2023-12-01T01:53:00.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.156+0000 7f55e54af640 1 -- 172.21.15.115:0/1837216178 wait complete. 2023-12-01T01:53:00.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.158+0000 7f55e54af640 1 Processor -- start 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.158+0000 7f55e54af640 1 -- start start 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55e54af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55e00b5d80 0x7f55e015fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55e54af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 0x7f55e00c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55e54af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55e015cd20 0x7f55e00b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55e54af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00c8a20 con 0x7f55e00d1230 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55e54af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00c7bb0 con 0x7f55e00b5d80 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55e54af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e0156a60 con 0x7f55e015cd20 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55de7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 0x7f55e00c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55de7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 0x7f55e00c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50374/0 (socket says 172.21.15.115:50374) 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55de7fc640 1 -- 172.21.15.115:0/3357728763 learned_addr learned my addr 172.21.15.115:0/3357728763 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55de7fc640 1 -- 172.21.15.115:0/3357728763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55e015cd20 msgr2=0x7f55e00b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55de7fc640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55e015cd20 0x7f55e00b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55de7fc640 1 -- 172.21.15.115:0/3357728763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55e00b5d80 msgr2=0x7f55e015fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55de7fc640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55e00b5d80 0x7f55e015fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.159+0000 7f55de7fc640 1 -- 172.21.15.115:0/3357728763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d8009a70 con 0x7f55e00d1230 2023-12-01T01:53:00.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.160+0000 7f55de7fc640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 0x7f55e00c7670 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f55c800c410 tx=0x7f55c800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:00.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.160+0000 7f55d77fe640 1 -- 172.21.15.115:0/3357728763 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80074e0 con 0x7f55e00d1230 2023-12-01T01:53:00.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.160+0000 7f55d77fe640 1 -- 172.21.15.115:0/3357728763 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c8007660 con 0x7f55e00d1230 2023-12-01T01:53:00.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.160+0000 7f55d77fe640 1 -- 172.21.15.115:0/3357728763 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8004a20 con 0x7f55e00d1230 2023-12-01T01:53:00.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.160+0000 7f55e54af640 1 -- 172.21.15.115:0/3357728763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e0156ec0 con 0x7f55e00d1230 2023-12-01T01:53:00.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.160+0000 7f55e54af640 1 -- 172.21.15.115:0/3357728763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e00c8080 con 0x7f55e00d1230 2023-12-01T01:53:00.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.162+0000 7f55d77fe640 1 -- 172.21.15.115:0/3357728763 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f55c8004b80 con 0x7f55e00d1230 2023-12-01T01:53:00.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.163+0000 7f55d77fe640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f55c006a550 0x7f55c006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.163+0000 7f55d77fe640 1 -- 172.21.15.115:0/3357728763 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55c8088990 con 0x7f55e00d1230 2023-12-01T01:53:00.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.163+0000 7f55deffd640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f55c006a550 0x7f55c006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.164+0000 7f55deffd640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f55c006a550 0x7f55c006ca10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55d00098f0 tx=0x7f55d0008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:00.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.164+0000 7f55e8029740 1 Processor -- start 2023-12-01T01:53:00.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.164+0000 7f55e8029740 1 -- start start 2023-12-01T01:53:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.165+0000 7f55e8029740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190b9920 0x55c6190b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.165+0000 7f55e8029740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190898a0 0x55c6190b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.165+0000 7f55e8029740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.165+0000 7f55e8029740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6190652f0 con 0x55c61908d4a0 2023-12-01T01:53:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.165+0000 7f55e8029740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c618ff9f80 con 0x55c6190898a0 2023-12-01T01:53:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.165+0000 7f55e8029740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c618ff7710 con 0x55c6190b9920 2023-12-01T01:53:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.165+0000 7f55df7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.165+0000 7f55df7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190b70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50378/0 (socket says 172.21.15.115:50378) 2023-12-01T01:53:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.165+0000 7f55df7fe640 1 -- 172.21.15.115:0/3226245523 learned_addr learned my addr 172.21.15.115:0/3226245523 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55deffd640 1 --2- 172.21.15.115:0/3226245523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190b9920 0x55c6190b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55de7fc640 1 --2- 172.21.15.115:0/3226245523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190898a0 0x55c6190b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55df7fe640 1 -- 172.21.15.115:0/3226245523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190b9920 msgr2=0x55c6190b9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55df7fe640 1 --2- 172.21.15.115:0/3226245523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190b9920 0x55c6190b9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55df7fe640 1 -- 172.21.15.115:0/3226245523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190898a0 msgr2=0x55c6190b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55df7fe640 1 --2- 172.21.15.115:0/3226245523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190898a0 0x55c6190b6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55df7fe640 1 -- 172.21.15.115:0/3226245523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c618fba690 con 0x55c61908d4a0 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55de7fc640 1 --2- 172.21.15.115:0/3226245523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190898a0 0x55c6190b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55df7fe640 1 --2- 172.21.15.115:0/3226245523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190b70d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009eb0 tx=0x7f55d802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de72f3611fc8b35a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55d4ff9640 1 -- 172.21.15.115:0/3226245523 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d802c9c0 con 0x55c61908d4a0 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55d4ff9640 1 -- 172.21.15.115:0/3226245523 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d802ce90 con 0x55c61908d4a0 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.166+0000 7f55d4ff9640 1 -- 172.21.15.115:0/3226245523 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8034440 con 0x55c61908d4a0 2023-12-01T01:53:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.167+0000 7f55e8029740 1 -- 172.21.15.115:0/3226245523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 msgr2=0x55c6190b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.167+0000 7f55e8029740 1 --2- 172.21.15.115:0/3226245523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190b70d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009eb0 tx=0x7f55d802e070 comp rx=0 tx=0).stop 2023-12-01T01:53:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.167+0000 7f55e8029740 1 -- 172.21.15.115:0/3226245523 shutdown_connections 2023-12-01T01:53:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.167+0000 7f55e8029740 1 --2- 172.21.15.115:0/3226245523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190b70d0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.167+0000 7f55e8029740 1 --2- 172.21.15.115:0/3226245523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190898a0 0x55c6190b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.167+0000 7f55e8029740 1 --2- 172.21.15.115:0/3226245523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190b9920 0x55c6190b9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.167+0000 7f55e8029740 1 -- 172.21.15.115:0/3226245523 >> 172.21.15.115:0/3226245523 conn(0x55c61908bbb0 msgr2=0x55c6190d4730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.167+0000 7f55e8029740 1 -- 172.21.15.115:0/3226245523 shutdown_connections 2023-12-01T01:53:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.167+0000 7f55e8029740 1 -- 172.21.15.115:0/3226245523 wait complete. 2023-12-01T01:53:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.168+0000 7f55e8029740 1 -- start start 2023-12-01T01:53:00.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.168+0000 7f55e8029740 1 Processor -- start 2023-12-01T01:53:00.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.169+0000 7f55e8029740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.169+0000 7f55e8029740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190b9920 0x55c6190b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.169+0000 7f55e8029740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190bca80 0x55c6190bcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.169+0000 7f55e8029740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c618ff3000 con 0x55c61908d4a0 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.169+0000 7f55e8029740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c618f947d0 con 0x55c6190b9920 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.169+0000 7f55e8029740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c618f94470 con 0x55c6190bca80 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.169+0000 7f55e8029740 10 client.0 _ll_register_callbacks cb 0x55c618fe31b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.169+0000 7f55deffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.169+0000 7f55deffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50386/0 (socket says 172.21.15.115:50386) 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.169+0000 7f55deffd640 1 -- 172.21.15.115:0/2857538942 learned_addr learned my addr 172.21.15.115:0/2857538942 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55de7fc640 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190b9920 0x55c6190b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55df7fe640 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190bca80 0x55c6190bcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55deffd640 1 -- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190bca80 msgr2=0x55c6190bcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:00.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55deffd640 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190bca80 0x55c6190bcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55deffd640 1 -- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190b9920 msgr2=0x55c6190b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55deffd640 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190b9920 0x55c6190b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55deffd640 1 -- 172.21.15.115:0/2857538942 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e00ce000 con 0x55c61908d4a0 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55deffd640 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190ae8d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009530 tx=0x7f55d0006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0b54865d88023e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55bdffb640 1 -- 172.21.15.115:0/2857538942 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d001f410 con 0x55c61908d4a0 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55bdffb640 1 -- 172.21.15.115:0/2857538942 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55d001f570 con 0x55c61908d4a0 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.170+0000 7f55e8029740 1 -- 172.21.15.115:0/2857538942 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c61901d550 con 0x55c61908d4a0 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55bdffb640 1 -- 172.21.15.115:0/2857538942 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55d0005990 con 0x55c61908d4a0 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55e8029740 10 client.7620 fetch_fsmap learned FSMap version 48 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55e8029740 1 -- 172.21.15.115:0/2857538942 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55c618ff1680 con 0x55c61908d4a0 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55bdffb640 1 -- 172.21.15.115:0/2857538942 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c61901d550 con 0x55c61908d4a0 2023-12-01T01:53:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55bdffb640 1 -- 172.21.15.115:0/2857538942 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f55d0005af0 con 0x55c61908d4a0 2023-12-01T01:53:00.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55e8029740 10 client.7620 fetch_fsmap finished waiting for FSMap version 48 2023-12-01T01:53:00.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55e8029740 10 client.7620 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-01T01:53:00.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55e8029740 20 client.7620 populate_metadata read hostname 'smithi115' 2023-12-01T01:53:00.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55e8029740 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:6832/3108729597,v1:172.21.15.142:6834/3108729597] conn(0x55c6190b3190 0x55c6190b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55e8029740 4 client.7620 mds_command: new command op to 7455 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:53:00.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.171+0000 7f55e8029740 1 -- 172.21.15.115:0/2857538942 --> [v2:172.21.15.142:6832/3108729597,v1:172.21.15.142:6834/3108729597] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c618fe5b50 con 0x55c6190b3190 2023-12-01T01:53:00.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.172+0000 7f55de7fc640 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:6832/3108729597,v1:172.21.15.142:6834/3108729597] conn(0x55c6190b3190 0x55c6190b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.172+0000 7f55de7fc640 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:6832/3108729597,v1:172.21.15.142:6834/3108729597] conn(0x55c6190b3190 0x55c6190b3570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdc8b0c10e79f2e4 server_cookie=2a6053d78ac0318f in_seq=0 out_seq=0 2023-12-01T01:53:00.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.172+0000 7f55bdffb640 10 client.7620 ms_handle_connect on v2:172.21.15.142:6832/3108729597 2023-12-01T01:53:00.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.174+0000 7f55bdffb640 1 -- 172.21.15.115:0/2857538942 <== mds.0 v2:172.21.15.142:6832/3108729597 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c618fe5b50 con 0x55c6190b3190 2023-12-01T01:53:00.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.174+0000 7f55bdffb640 10 client.7620 handle_command_reply: tid=0 2023-12-01T01:53:00.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.174+0000 7f55e8029740 1 client.7620 shutdown 2023-12-01T01:53:00.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 1 -- 172.21.15.115:0/2857538942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 msgr2=0x55c6190ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190ae8d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009530 tx=0x7f55d0006e10 comp rx=0 tx=0).stop 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 1 -- 172.21.15.115:0/2857538942 shutdown_connections 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:6832/3108729597,v1:172.21.15.142:6834/3108729597] conn(0x55c6190b3190 0x55c6190b3570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190bca80 0x55c6190bcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190b9920 0x55c6190b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55bdffb640 0 client.7620 ms_handle_reset on v2:172.21.15.142:6832/3108729597 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 1 --2- 172.21.15.115:0/2857538942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190ae8d0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 1 -- 172.21.15.115:0/2857538942 >> 172.21.15.115:0/2857538942 conn(0x55c6190898a0 msgr2=0x55c61908de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 1 -- 172.21.15.115:0/2857538942 shutdown_connections 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 1 -- 172.21.15.115:0/2857538942 wait complete. 2023-12-01T01:53:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.175+0000 7f55e8029740 20 client.7620 trim_cache size 0 max 16384 2023-12-01T01:53:00.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.191+0000 7f55e8029740 1 Processor -- start 2023-12-01T01:53:00.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.191+0000 7f55e8029740 1 -- start start 2023-12-01T01:53:00.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55e8029740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55e8029740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190b9920 0x55c6190d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55e8029740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190bca80 0x55c6190bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55e8029740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c618d5af20 con 0x55c61908d4a0 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55e8029740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c61908a590 con 0x55c6190bca80 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55e8029740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6190b2140 con 0x55c6190b9920 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55deffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55deffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190851e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50388/0 (socket says 172.21.15.115:50388) 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55deffd640 1 -- 172.21.15.115:0/3716068914 learned_addr learned my addr 172.21.15.115:0/3716068914 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55deffd640 1 -- 172.21.15.115:0/3716068914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190b9920 msgr2=0x55c6190d7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55deffd640 1 --2- 172.21.15.115:0/3716068914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190b9920 0x55c6190d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55deffd640 1 -- 172.21.15.115:0/3716068914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190bca80 msgr2=0x55c6190bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:00.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55deffd640 1 --2- 172.21.15.115:0/3716068914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190bca80 0x55c6190bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.192+0000 7f55deffd640 1 -- 172.21.15.115:0/3716068914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c618f939e0 con 0x55c61908d4a0 2023-12-01T01:53:00.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.193+0000 7f55deffd640 1 --2- 172.21.15.115:0/3716068914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190851e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009f10 tx=0x7f55d001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f010c85dfde415 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:00.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.193+0000 7f55d57fa640 1 -- 172.21.15.115:0/3716068914 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0014c90 con 0x55c61908d4a0 2023-12-01T01:53:00.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.193+0000 7f55d57fa640 1 -- 172.21.15.115:0/3716068914 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d001a050 con 0x55c61908d4a0 2023-12-01T01:53:00.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.193+0000 7f55d57fa640 1 -- 172.21.15.115:0/3716068914 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d001f610 con 0x55c61908d4a0 2023-12-01T01:53:00.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 -- 172.21.15.115:0/3716068914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 msgr2=0x55c6190851e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 --2- 172.21.15.115:0/3716068914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190851e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009f10 tx=0x7f55d001d000 comp rx=0 tx=0).stop 2023-12-01T01:53:00.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 -- 172.21.15.115:0/3716068914 shutdown_connections 2023-12-01T01:53:00.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 --2- 172.21.15.115:0/3716068914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190b9920 0x55c6190d7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 --2- 172.21.15.115:0/3716068914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6190bca80 0x55c6190bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 --2- 172.21.15.115:0/3716068914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c61908d4a0 0x55c6190851e0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 -- 172.21.15.115:0/3716068914 >> 172.21.15.115:0/3716068914 conn(0x55c6190898a0 msgr2=0x55c6190aec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:00.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 -- 172.21.15.115:0/3716068914 shutdown_connections 2023-12-01T01:53:00.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 -- 172.21.15.115:0/3716068914 wait complete. 2023-12-01T01:53:00.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 -- start start 2023-12-01T01:53:00.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.194+0000 7f55e8029740 1 Processor -- start 2023-12-01T01:53:00.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.195+0000 7f55e8029740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c61908d4a0 0x55c6190bd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.195+0000 7f55e8029740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c6190b9920 0x55c6190d4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.195+0000 7f55e8029740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190bca80 0x55c61913ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.195+0000 7f55e8029740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6190e3df0 con 0x55c6190b9920 2023-12-01T01:53:00.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.195+0000 7f55e8029740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6190d4c50 con 0x55c61908d4a0 2023-12-01T01:53:00.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.195+0000 7f55e8029740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6190d5180 con 0x55c6190bca80 2023-12-01T01:53:00.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.195+0000 7f55e8029740 10 client.0 _ll_register_callbacks cb 0x7f55d8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55de7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c6190b9920 0x55c6190d4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55de7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c6190b9920 0x55c6190d4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50394/0 (socket says 172.21.15.115:50394) 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55de7fc640 1 -- 172.21.15.115:0/3710502546 learned_addr learned my addr 172.21.15.115:0/3710502546 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55de7fc640 1 -- 172.21.15.115:0/3710502546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190bca80 msgr2=0x55c61913ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55de7fc640 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190bca80 0x55c61913ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55de7fc640 1 -- 172.21.15.115:0/3710502546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c61908d4a0 msgr2=0x55c6190bd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55de7fc640 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c61908d4a0 0x55c6190bd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55de7fc640 1 -- 172.21.15.115:0/3710502546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c618ff1680 con 0x55c6190b9920 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55de7fc640 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c6190b9920 0x55c6190d4320 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f55c8000fb0 tx=0x7f55c800e300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a25810a00038d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.196+0000 7f55bdffb640 1 -- 172.21.15.115:0/3710502546 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8054110 con 0x55c6190b9920 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55bdffb640 1 -- 172.21.15.115:0/3710502546 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55c8054270 con 0x55c6190b9920 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55e8029740 1 -- 172.21.15.115:0/3710502546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55d802ce90 con 0x55c6190b9920 2023-12-01T01:53:00.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55bdffb640 1 -- 172.21.15.115:0/3710502546 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c8086e10 con 0x55c6190b9920 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55e8029740 10 client.7626 fetch_fsmap learned FSMap version 48 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55e8029740 1 -- 172.21.15.115:0/3710502546 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55c618fb8ef0 con 0x55c6190b9920 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55bdffb640 1 -- 172.21.15.115:0/3710502546 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d802ce90 con 0x55c6190b9920 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55bdffb640 1 -- 172.21.15.115:0/3710502546 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 48) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f55c801b060 con 0x55c6190b9920 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55e8029740 10 client.7626 fetch_fsmap finished waiting for FSMap version 48 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55e8029740 10 client.7626 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55e8029740 20 client.7626 populate_metadata read hostname 'smithi115' 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55e8029740 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.142:6832/3108729597,v1:172.21.15.142:6834/3108729597] conn(0x55c6190b6e10 0x55c61913cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55e8029740 4 client.7626 mds_command: new command op to 7455 tid=0[{"prefix": "status"}] 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.197+0000 7f55e8029740 1 -- 172.21.15.115:0/3710502546 --> [v2:172.21.15.142:6832/3108729597,v1:172.21.15.142:6834/3108729597] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c618fec080 con 0x55c6190b6e10 2023-12-01T01:53:00.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.198+0000 7f55deffd640 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.142:6832/3108729597,v1:172.21.15.142:6834/3108729597] conn(0x55c6190b6e10 0x55c61913cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.199+0000 7f55deffd640 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.142:6832/3108729597,v1:172.21.15.142:6834/3108729597] conn(0x55c6190b6e10 0x55c61913cfc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32364a0fcb984f6e server_cookie=a3c516533167a22b in_seq=0 out_seq=0 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.199+0000 7f55bdffb640 10 client.7626 ms_handle_connect on v2:172.21.15.142:6832/3108729597 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.199+0000 7f55bdffb640 1 -- 172.21.15.115:0/3710502546 <== mds.0 v2:172.21.15.142:6832/3108729597 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55c618fec080 con 0x55c6190b6e10 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.199+0000 7f55bdffb640 10 client.7626 handle_command_reply: tid=0 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.199+0000 7f55e8029740 1 client.7626 shutdown 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 1 -- 172.21.15.115:0/3710502546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c6190b9920 msgr2=0x55c6190d4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c6190b9920 0x55c6190d4320 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f55c8000fb0 tx=0x7f55c800e300 comp rx=0 tx=0).stop 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 1 -- 172.21.15.115:0/3710502546 shutdown_connections 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.142:6832/3108729597,v1:172.21.15.142:6834/3108729597] conn(0x55c6190b6e10 0x55c61913cfc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c6190bca80 0x55c61913ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55c6190b9920 0x55c6190d4320 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55bdffb640 0 client.7626 ms_handle_reset on v2:172.21.15.142:6832/3108729597 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 1 --2- 172.21.15.115:0/3710502546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c61908d4a0 0x55c6190bd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 1 -- 172.21.15.115:0/3710502546 >> 172.21.15.115:0/3710502546 conn(0x55c6190898a0 msgr2=0x55c6190d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 1 -- 172.21.15.115:0/3710502546 shutdown_connections 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 1 -- 172.21.15.115:0/3710502546 wait complete. 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.200+0000 7f55e8029740 20 client.7626 trim_cache size 0 max 16384 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 7455, 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 11.703861635000001, 2023-12-01T01:53:00.204 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 48, 2023-12-01T01:53:00.205 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 111, 2023-12-01T01:53:00.205 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 111, 2023-12-01T01:53:00.205 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 16.637537764000001 2023-12-01T01:53:00.205 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:53:00.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 -- 172.21.15.115:0/3357728763 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f55c006a550 msgr2=0x7f55c006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f55c006a550 0x7f55c006ca10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55d00098f0 tx=0x7f55d0008040 comp rx=0 tx=0).stop 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 -- 172.21.15.115:0/3357728763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 msgr2=0x7f55e00c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 0x7f55e00c7670 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f55c800c410 tx=0x7f55c800c8d0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 -- 172.21.15.115:0/3357728763 shutdown_connections 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f55c006a550 0x7f55c006ca10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55e015cd20 0x7f55e00b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55e00d1230 0x7f55e00c7670 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 --2- 172.21.15.115:0/3357728763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55e00b5d80 0x7f55e015fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 -- 172.21.15.115:0/3357728763 >> 172.21.15.115:0/3357728763 conn(0x7f55e006a0e0 msgr2=0x7f55e0158b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 -- 172.21.15.115:0/3357728763 shutdown_connections 2023-12-01T01:53:00.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.201+0000 7f55e54af640 1 -- 172.21.15.115:0/3357728763 wait complete. 2023-12-01T01:53:00.218 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:53:00.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.421+0000 7fca3b84a640 1 Processor -- start 2023-12-01T01:53:00.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.421+0000 7fca3b84a640 1 -- start start 2023-12-01T01:53:00.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.422+0000 7fca3b84a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fca340cad70 0x7fca3406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.422+0000 7fca3b84a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca340add10 con 0x7fca3415e2f0 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.422+0000 7fca3b84a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca340c5760 con 0x7fca3406f2f0 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.422+0000 7fca3b84a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3415e6d0 con 0x7fca340cad70 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.423+0000 7fca395e6640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fca3415e2f0 0x7fca340ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57232/0 (socket says 172.21.15.115:57232) 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.423+0000 7fca395e6640 1 -- 172.21.15.115:0/525544212 learned_addr learned my addr 172.21.15.115:0/525544212 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.423+0000 7fca38de5640 1 --2- 172.21.15.115:0/525544212 >> v2:172.21.15.115:3300/0 conn(0x7fca340cad70 0x7fca3406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.423+0000 7fca2bfff640 1 -- 172.21.15.115:0/525544212 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347612680 0 0) 0x7fca340add10 con 0x7fca3415e2f0 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.423+0000 7fca2bfff640 1 -- 172.21.15.115:0/525544212 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca20003520 con 0x7fca3415e2f0 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.423+0000 7fca38de5640 1 -- 172.21.15.115:0/525544212 >> v1:172.21.15.142:6789/0 conn(0x7fca3406f2f0 legacy=0x7fca3406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.423+0000 7fca38de5640 1 -- 172.21.15.115:0/525544212 >> v1:172.21.15.115:6789/0 conn(0x7fca3415e2f0 legacy=0x7fca340ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.423+0000 7fca38de5640 1 -- 172.21.15.115:0/525544212 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca340cf7b0 con 0x7fca340cad70 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.424+0000 7fca38de5640 1 --2- 172.21.15.115:0/525544212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca340cad70 0x7fca3406b5c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fca240097d0 tx=0x7fca2402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246fdb29457bfd86 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.425+0000 7fca2bfff640 1 -- 172.21.15.115:0/525544212 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2402f050 con 0x7fca340cad70 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.425+0000 7fca2bfff640 1 -- 172.21.15.115:0/525544212 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca24031700 con 0x7fca340cad70 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.425+0000 7fca2bfff640 1 -- 172.21.15.115:0/525544212 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24031a70 con 0x7fca340cad70 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.425+0000 7fca3b84a640 1 -- 172.21.15.115:0/525544212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca340cad70 msgr2=0x7fca3406b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.425+0000 7fca3b84a640 1 --2- 172.21.15.115:0/525544212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca340cad70 0x7fca3406b5c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fca240097d0 tx=0x7fca2402b7a0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.425+0000 7fca3b84a640 1 -- 172.21.15.115:0/525544212 shutdown_connections 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.425+0000 7fca3b84a640 1 --2- 172.21.15.115:0/525544212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca340cad70 0x7fca3406b5c0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.425+0000 7fca3b84a640 1 -- 172.21.15.115:0/525544212 >> 172.21.15.115:0/525544212 conn(0x7fca340ce880 msgr2=0x7fca340cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.425+0000 7fca3b84a640 1 -- 172.21.15.115:0/525544212 shutdown_connections 2023-12-01T01:53:00.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.425+0000 7fca3b84a640 1 -- 172.21.15.115:0/525544212 wait complete. 2023-12-01T01:53:00.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.427+0000 7fca3b84a640 1 Processor -- start 2023-12-01T01:53:00.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.427+0000 7fca3b84a640 1 -- start start 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.428+0000 7fca3b84a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca3406f2f0 0x7fca34151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.428+0000 7fca3b84a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca340cad70 0x7fca340c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.428+0000 7fca3b84a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3415e2f0 0x7fca340ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.428+0000 7fca3b84a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca340c4030 con 0x7fca3406f2f0 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.428+0000 7fca3b84a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca340c4310 con 0x7fca3415e2f0 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.428+0000 7fca3b84a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca340c45f0 con 0x7fca340cad70 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.428+0000 7fca395e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca3406f2f0 0x7fca34151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.428+0000 7fca395e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca3406f2f0 0x7fca34151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50412/0 (socket says 172.21.15.115:50412) 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.428+0000 7fca395e6640 1 -- 172.21.15.115:0/123523984 learned_addr learned my addr 172.21.15.115:0/123523984 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca38de5640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca340cad70 0x7fca340c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca395e6640 1 -- 172.21.15.115:0/123523984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca340cad70 msgr2=0x7fca340c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca39de7640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3415e2f0 0x7fca340ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca395e6640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca340cad70 0x7fca340c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca395e6640 1 -- 172.21.15.115:0/123523984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3415e2f0 msgr2=0x7fca340ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca395e6640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3415e2f0 0x7fca340ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca395e6640 1 -- 172.21.15.115:0/123523984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24009400 con 0x7fca3406f2f0 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca395e6640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca3406f2f0 0x7fca34151ce0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fca300010e0 tx=0x7fca3000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca29ffb640 1 -- 172.21.15.115:0/123523984 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30005ce0 con 0x7fca3406f2f0 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca29ffb640 1 -- 172.21.15.115:0/123523984 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca30005e60 con 0x7fca3406f2f0 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca29ffb640 1 -- 172.21.15.115:0/123523984 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300072f0 con 0x7fca3406f2f0 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.429+0000 7fca3b84a640 1 -- 172.21.15.115:0/123523984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca340c48d0 con 0x7fca3406f2f0 2023-12-01T01:53:00.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.430+0000 7fca3b84a640 1 -- 172.21.15.115:0/123523984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca34152220 con 0x7fca3406f2f0 2023-12-01T01:53:00.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.431+0000 7fca3b84a640 1 -- 172.21.15.115:0/123523984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3406fda0 con 0x7fca3406f2f0 2023-12-01T01:53:00.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.431+0000 7fca29ffb640 1 -- 172.21.15.115:0/123523984 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fca30007450 con 0x7fca3406f2f0 2023-12-01T01:53:00.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.432+0000 7fca29ffb640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fca1806a580 0x7fca1806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:00.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.432+0000 7fca29ffb640 1 -- 172.21.15.115:0/123523984 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca300881a0 con 0x7fca3406f2f0 2023-12-01T01:53:00.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.433+0000 7fca38de5640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fca1806a580 0x7fca1806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:00.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.433+0000 7fca38de5640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fca1806a580 0x7fca1806ca40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca24035000 tx=0x7fca24008030 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:00.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.437+0000 7fca29ffb640 1 -- 172.21.15.115:0/123523984 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fca30053550 con 0x7fca3406f2f0 2023-12-01T01:53:00.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.667+0000 7fca3b84a640 1 -- 172.21.15.115:0/123523984 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fca340c4ac0 con 0x7fca3406f2f0 2023-12-01T01:53:00.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.668+0000 7fca29ffb640 1 -- 172.21.15.115:0/123523984 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7fca30056b70 con 0x7fca3406f2f0 2023-12-01T01:53:00.669 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:53:00.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.670+0000 7fca3b84a640 1 -- 172.21.15.115:0/123523984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fca1806a580 msgr2=0x7fca1806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.670+0000 7fca3b84a640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fca1806a580 0x7fca1806ca40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca24035000 tx=0x7fca24008030 comp rx=0 tx=0).stop 2023-12-01T01:53:00.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.670+0000 7fca3b84a640 1 -- 172.21.15.115:0/123523984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca3406f2f0 msgr2=0x7fca34151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:00.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.670+0000 7fca3b84a640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca3406f2f0 0x7fca34151ce0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fca300010e0 tx=0x7fca3000c890 comp rx=0 tx=0).stop 2023-12-01T01:53:00.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.670+0000 7fca3b84a640 1 -- 172.21.15.115:0/123523984 shutdown_connections 2023-12-01T01:53:00.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.670+0000 7fca3b84a640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fca1806a580 0x7fca1806ca40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.670+0000 7fca3b84a640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca340cad70 0x7fca340c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.670+0000 7fca3b84a640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3415e2f0 0x7fca340ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.670+0000 7fca3b84a640 1 --2- 172.21.15.115:0/123523984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca3406f2f0 0x7fca34151ce0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:00.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.670+0000 7fca3b84a640 1 -- 172.21.15.115:0/123523984 >> 172.21.15.115:0/123523984 conn(0x7fca340ce880 msgr2=0x7fca340af430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:00.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.671+0000 7fca3b84a640 1 -- 172.21.15.115:0/123523984 shutdown_connections 2023-12-01T01:53:00.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:00.671+0000 7fca3b84a640 1 -- 172.21.15.115:0/123523984 wait complete. 2023-12-01T01:53:00.690 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T01:53:00.690 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:53:00.690 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T01:53:00.690 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T01:53:00.691 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:53:00.691 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T01:53:00.691 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T01:53:00.691 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:53:00.691 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:53:00.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:53:00.691 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T01:53:00.743 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:53:00.743 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:53:00.743 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:53:00.743 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:53:00.743 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 41376sec preferred_lft 41376sec 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T01:53:00.744 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T01:53:00.745 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:53:00.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:53:00.747 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:53:00.840 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T01:53:00.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:53:00.844 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T01:53:00.900 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T01:53:00.900 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:53:00.900 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:53:00.900 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:53:00.900 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:53:00.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:53:00.901 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:53:00.901 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:53:00.901 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:53:00.901 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:53:00.901 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:53:00.902 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:53:01.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:53:01.084 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:53:01.132 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T01:53:01.132 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T01:53:01.132 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:53:01.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:53:01.132 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T01:53:01.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:53:01.223 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T01:53:01.276 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:53:01.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:53:01.277 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:53:01.328 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:53:01.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:53:01.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:53:01.468 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:53:01.468 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:53:01.468 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:53:01.488 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:53:01.489 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:53:01.489 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:53:01.489 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:53:01.489 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:53:01.518 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:53:01.518 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:53:01.519 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:53:01.519 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:53:01.519 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:53:01.519 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:53:01.519 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:53:01.519 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:53:01.522 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T01:53:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:53:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:53:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:53:01.522 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:53:01.527 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:53:01.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:53:01.527 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:53:01.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:53:01.613 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:53:01.692 INFO:teuthology.orchestra.run.smithi115.stdout:client5456 2023-12-01T01:53:01.692 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:53:01.693 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:53:01.749 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T01:53:01.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.959+0000 7fb541ece640 1 Processor -- start 2023-12-01T01:53:01.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.960+0000 7fb541ece640 1 -- start start 2023-12-01T01:53:01.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.961+0000 7fb541ece640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb53c15cd20 0x7fb53c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:01.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.961+0000 7fb541ece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb53c0b5d80 0x7fb53c0b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:01.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.961+0000 7fb541ece640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53c0ce2e0 con 0x7fb53c0d1230 2023-12-01T01:53:01.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.961+0000 7fb541ece640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53c0b2160 con 0x7fb53c15cd20 2023-12-01T01:53:01.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.961+0000 7fb541ece640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0d2a10 con 0x7fb53c0b5d80 2023-12-01T01:53:01.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.962+0000 7fb53bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb53c0d1230 0x7fb53c0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57238/0 (socket says 172.21.15.115:57238) 2023-12-01T01:53:01.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.962+0000 7fb53bfff640 1 -- 172.21.15.115:0/1948564447 learned_addr learned my addr 172.21.15.115:0/1948564447 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:53:01.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.962+0000 7fb533fff640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb53c0b5d80 0x7fb53c0b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:01.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.962+0000 7fb53b7fe640 1 --2- 172.21.15.115:0/1948564447 >> v2:172.21.15.142:3300/0 conn(0x7fb53c15cd20 0x7fb53c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:01.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.962+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104383025 0 0) 0x7fb53c0ce2e0 con 0x7fb53c0d1230 2023-12-01T01:53:01.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.962+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb528003520 con 0x7fb53c0d1230 2023-12-01T01:53:01.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.962+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 893953668 0 0) 0x7fb528003520 con 0x7fb53c0d1230 2023-12-01T01:53:01.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.962+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb53c0b5d80 msgr2=0x7fb53c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:01.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb53c0b5d80 0x7fb53c0b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 >> v2:172.21.15.142:3300/0 conn(0x7fb53c15cd20 msgr2=0x7fb53c0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 --2- 172.21.15.115:0/1948564447 >> v2:172.21.15.142:3300/0 conn(0x7fb53c15cd20 0x7fb53c0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c0ce000 con 0x7fb53c0d1230 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb5340019f0 con 0x7fb53c0d1230 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 >> v1:172.21.15.115:6789/0 conn(0x7fb53c0d1230 legacy=0x7fb53c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb528003a10 0x7fb528003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5280047c0 0x7fb528004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb53c0b5d80 0x7fb528005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0d2a10 con 0x7fb5280047c0 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52800b980 con 0x7fb528003a10 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52800bb40 con 0x7fb53c0b5d80 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb533fff640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5280047c0 0x7fb528004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53b7fe640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb528003a10 0x7fb528003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:01.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb533fff640 1 -- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb53c0b5d80 msgr2=0x7fb528005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:01.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb533fff640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb53c0b5d80 0x7fb528005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:01.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb533fff640 1 -- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb528003a10 msgr2=0x7fb528003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:01.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb533fff640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb528003a10 0x7fb528003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:01.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb533fff640 1 -- 172.21.15.115:0/1948564447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528003520 con 0x7fb5280047c0 2023-12-01T01:53:01.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.963+0000 7fb53b7fe640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb528003a10 0x7fb528003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:53:01.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.965+0000 7fb533fff640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5280047c0 0x7fb528004c40 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fb524009a90 tx=0x7fb5280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e572b07f037bbf34 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:01.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.965+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52402c6b0 con 0x7fb5280047c0 2023-12-01T01:53:01.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.965+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52402c810 con 0x7fb5280047c0 2023-12-01T01:53:01.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.965+0000 7fb53affd640 1 -- 172.21.15.115:0/1948564447 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52402cba0 con 0x7fb5280047c0 2023-12-01T01:53:01.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.965+0000 7fb541ece640 1 -- 172.21.15.115:0/1948564447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5280047c0 msgr2=0x7fb528004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:01.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.965+0000 7fb541ece640 1 --2- 172.21.15.115:0/1948564447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5280047c0 0x7fb528004c40 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fb524009a90 tx=0x7fb5280057f0 comp rx=0 tx=0).stop 2023-12-01T01:53:01.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.966+0000 7fb53bfff640 1 -- 172.21.15.115:0/1948564447 reap_dead start 2023-12-01T01:53:01.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.966+0000 7fb541ece640 1 -- 172.21.15.115:0/1948564447 shutdown_connections 2023-12-01T01:53:01.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.966+0000 7fb541ece640 1 -- 172.21.15.115:0/1948564447 >> 172.21.15.115:0/1948564447 conn(0x7fb53c06a0e0 msgr2=0x7fb53c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:01.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.966+0000 7fb541ece640 1 -- 172.21.15.115:0/1948564447 shutdown_connections 2023-12-01T01:53:01.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.966+0000 7fb541ece640 1 -- 172.21.15.115:0/1948564447 wait complete. 2023-12-01T01:53:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.967+0000 7fb541ece640 1 Processor -- start 2023-12-01T01:53:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.968+0000 7fb541ece640 1 -- start start 2023-12-01T01:53:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.968+0000 7fb541ece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5280047c0 0x7fb53c0b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb541ece640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb53c0c6780 0x7fb53c0c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb541ece640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb53c0c9080 0x7fb53c0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb541ece640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0c9a40 con 0x7fb53c0c6780 2023-12-01T01:53:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb541ece640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0c7f00 con 0x7fb53c0c9080 2023-12-01T01:53:01.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb541ece640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c1f2360 con 0x7fb5280047c0 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb533fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb53c0c6780 0x7fb53c0c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb533fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb53c0c6780 0x7fb53c0c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50432/0 (socket says 172.21.15.115:50432) 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb533fff640 1 -- 172.21.15.115:0/765046790 learned_addr learned my addr 172.21.15.115:0/765046790 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb53b7fe640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5280047c0 0x7fb53c0b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb53bfff640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb53c0c9080 0x7fb53c0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb533fff640 1 -- 172.21.15.115:0/765046790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5280047c0 msgr2=0x7fb53c0b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb533fff640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5280047c0 0x7fb53c0b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb533fff640 1 -- 172.21.15.115:0/765046790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb53c0c9080 msgr2=0x7fb53c0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb533fff640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb53c0c9080 0x7fb53c0c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb533fff640 1 -- 172.21.15.115:0/765046790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c0c72f0 con 0x7fb53c0c6780 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.969+0000 7fb53bfff640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb53c0c9080 0x7fb53c0c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:53:01.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.970+0000 7fb533fff640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb53c0c6780 0x7fb53c0c6c00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fb524037040 tx=0x7fb524004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.970+0000 7fb538ff9640 1 -- 172.21.15.115:0/765046790 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524003810 con 0x7fb53c0c6780 2023-12-01T01:53:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.970+0000 7fb538ff9640 1 -- 172.21.15.115:0/765046790 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb524003970 con 0x7fb53c0c6780 2023-12-01T01:53:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.970+0000 7fb538ff9640 1 -- 172.21.15.115:0/765046790 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524003ce0 con 0x7fb53c0c6780 2023-12-01T01:53:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.970+0000 7fb541ece640 1 -- 172.21.15.115:0/765046790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c0c75d0 con 0x7fb53c0c6780 2023-12-01T01:53:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.970+0000 7fb541ece640 1 -- 172.21.15.115:0/765046790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c0c7b90 con 0x7fb53c0c6780 2023-12-01T01:53:01.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.971+0000 7fb541ece640 1 -- 172.21.15.115:0/765046790 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53c04e890 con 0x7fb53c0c6780 2023-12-01T01:53:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.972+0000 7fb538ff9640 1 -- 172.21.15.115:0/765046790 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb524034440 con 0x7fb53c0c6780 2023-12-01T01:53:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.973+0000 7fb538ff9640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb51c06a580 0x7fb51c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.973+0000 7fb538ff9640 1 -- 172.21.15.115:0/765046790 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5240aac20 con 0x7fb53c0c6780 2023-12-01T01:53:01.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.973+0000 7fb53b7fe640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb51c06a580 0x7fb51c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:01.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.974+0000 7fb53b7fe640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb51c06a580 0x7fb51c06ca40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009b20 tx=0x7fb52c008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:01.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:01.978+0000 7fb538ff9640 1 -- 172.21.15.115:0/765046790 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb5240762d0 con 0x7fb53c0c6780 2023-12-01T01:53:02.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.193+0000 7fb541ece640 1 -- 172.21.15.115:0/765046790 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb53c0c9a40 con 0x7fb51c06a580 2023-12-01T01:53:02.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.195+0000 7fb538ff9640 1 -- 172.21.15.115:0/765046790 <== mgr.7230 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb53c0c9a40 con 0x7fb51c06a580 2023-12-01T01:53:02.196 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:53:02.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:53:02.197 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T01:53:02.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:53:02.197 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:53:02.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 -- 172.21.15.115:0/765046790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb51c06a580 msgr2=0x7fb51c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:02.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb51c06a580 0x7fb51c06ca40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009b20 tx=0x7fb52c008040 comp rx=0 tx=0).stop 2023-12-01T01:53:02.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 -- 172.21.15.115:0/765046790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb53c0c6780 msgr2=0x7fb53c0c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:02.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb53c0c6780 0x7fb53c0c6c00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fb524037040 tx=0x7fb524004730 comp rx=0 tx=0).stop 2023-12-01T01:53:02.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 -- 172.21.15.115:0/765046790 shutdown_connections 2023-12-01T01:53:02.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb51c06a580 0x7fb51c06ca40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:02.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb53c0c9080 0x7fb53c0c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:02.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb53c0c6780 0x7fb53c0c6c00 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:02.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 --2- 172.21.15.115:0/765046790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5280047c0 0x7fb53c0b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:02.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 -- 172.21.15.115:0/765046790 >> 172.21.15.115:0/765046790 conn(0x7fb53c06a0e0 msgr2=0x7fb53c0ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:02.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 -- 172.21.15.115:0/765046790 shutdown_connections 2023-12-01T01:53:02.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.198+0000 7fb541ece640 1 -- 172.21.15.115:0/765046790 wait complete. 2023-12-01T01:53:02.218 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T01:53:02.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.432+0000 7fe23eece640 1 Processor -- start 2023-12-01T01:53:02.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.432+0000 7fe23eece640 1 -- start start 2023-12-01T01:53:02.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23eece640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe2380cd1e0 0x7fe2380b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:02.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23eece640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe2380b1c50 0x7fe23815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:02.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23eece640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2380b2f30 con 0x7fe238142870 2023-12-01T01:53:02.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23eece640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2380b3ae0 con 0x7fe2380b1c50 2023-12-01T01:53:02.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23eece640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe238142c50 con 0x7fe2380cd1e0 2023-12-01T01:53:02.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23cc6a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe238142870 0x7fe2380ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57246/0 (socket says 172.21.15.115:57246) 2023-12-01T01:53:02.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23cc6a640 1 -- 172.21.15.115:0/1335054655 learned_addr learned my addr 172.21.15.115:0/1335054655 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:53:02.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23d46b640 1 --2- 172.21.15.115:0/1335054655 >> v2:172.21.15.115:3300/0 conn(0x7fe2380b1c50 0x7fe23815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:02.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe2377fe640 1 -- 172.21.15.115:0/1335054655 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915602909 0 0) 0x7fe2380b2f30 con 0x7fe238142870 2023-12-01T01:53:02.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe2377fe640 1 -- 172.21.15.115:0/1335054655 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe228003520 con 0x7fe238142870 2023-12-01T01:53:02.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe237fff640 1 --2- 172.21.15.115:0/1335054655 >> v2:172.21.15.142:3300/0 conn(0x7fe2380cd1e0 0x7fe2380b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:02.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23d46b640 1 -- 172.21.15.115:0/1335054655 >> v2:172.21.15.142:3300/0 conn(0x7fe2380cd1e0 msgr2=0x7fe2380b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:02.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23d46b640 1 --2- 172.21.15.115:0/1335054655 >> v2:172.21.15.142:3300/0 conn(0x7fe2380cd1e0 0x7fe2380b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:02.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23d46b640 1 -- 172.21.15.115:0/1335054655 >> v1:172.21.15.115:6789/0 conn(0x7fe238142870 legacy=0x7fe2380ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:02.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.434+0000 7fe23d46b640 1 -- 172.21.15.115:0/1335054655 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2380b2080 con 0x7fe2380b1c50 2023-12-01T01:53:02.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.435+0000 7fe237fff640 1 --2- 172.21.15.115:0/1335054655 >> v2:172.21.15.142:3300/0 conn(0x7fe2380cd1e0 0x7fe2380b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:53:02.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.436+0000 7fe23d46b640 1 --2- 172.21.15.115:0/1335054655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 0x7fe23815a0d0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fe2380cf7f0 tx=0x7fe23002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48be9dc411d637df server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:02.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.436+0000 7fe2377fe640 1 -- 172.21.15.115:0/1335054655 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23002cc80 con 0x7fe2380b1c50 2023-12-01T01:53:02.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.436+0000 7fe2377fe640 1 -- 172.21.15.115:0/1335054655 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23002cde0 con 0x7fe2380b1c50 2023-12-01T01:53:02.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.436+0000 7fe2377fe640 1 -- 172.21.15.115:0/1335054655 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230034640 con 0x7fe2380b1c50 2023-12-01T01:53:02.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.436+0000 7fe23eece640 1 -- 172.21.15.115:0/1335054655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 msgr2=0x7fe23815a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:02.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.436+0000 7fe23eece640 1 --2- 172.21.15.115:0/1335054655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 0x7fe23815a0d0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fe2380cf7f0 tx=0x7fe23002bed0 comp rx=0 tx=0).stop 2023-12-01T01:53:02.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.436+0000 7fe23eece640 1 -- 172.21.15.115:0/1335054655 shutdown_connections 2023-12-01T01:53:02.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.436+0000 7fe23eece640 1 --2- 172.21.15.115:0/1335054655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 0x7fe23815a0d0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:02.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.436+0000 7fe23eece640 1 --2- 172.21.15.115:0/1335054655 >> v2:172.21.15.142:3300/0 conn(0x7fe2380cd1e0 0x7fe2380b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:02.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.436+0000 7fe23eece640 1 -- 172.21.15.115:0/1335054655 >> 172.21.15.115:0/1335054655 conn(0x7fe238065960 msgr2=0x7fe238065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:02.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.437+0000 7fe23eece640 1 -- 172.21.15.115:0/1335054655 shutdown_connections 2023-12-01T01:53:02.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.437+0000 7fe23eece640 1 -- 172.21.15.115:0/1335054655 wait complete. 2023-12-01T01:53:02.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.438+0000 7fe23eece640 1 Processor -- start 2023-12-01T01:53:02.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.438+0000 7fe23eece640 1 -- start start 2023-12-01T01:53:02.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.439+0000 7fe23eece640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 0x7fe23813dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:02.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.439+0000 7fe23eece640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2380cd1e0 0x7fe2380aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:02.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.439+0000 7fe23eece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe238142870 0x7fe2380aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:02.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.439+0000 7fe23eece640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23813f2b0 con 0x7fe2380b1c50 2023-12-01T01:53:02.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.439+0000 7fe23eece640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23813e630 con 0x7fe2380cd1e0 2023-12-01T01:53:02.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.439+0000 7fe23eece640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23813e910 con 0x7fe238142870 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.439+0000 7fe23cc6a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 0x7fe23813dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe23cc6a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 0x7fe23813dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50452/0 (socket says 172.21.15.115:50452) 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe23cc6a640 1 -- 172.21.15.115:0/2540904169 learned_addr learned my addr 172.21.15.115:0/2540904169 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe23cc6a640 1 -- 172.21.15.115:0/2540904169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe238142870 msgr2=0x7fe2380aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe23cc6a640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe238142870 0x7fe2380aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe23cc6a640 1 -- 172.21.15.115:0/2540904169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2380cd1e0 msgr2=0x7fe2380aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe23cc6a640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2380cd1e0 0x7fe2380aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe23cc6a640 1 -- 172.21.15.115:0/2540904169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230009a70 con 0x7fe2380b1c50 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe23cc6a640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 0x7fe23813dde0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00c390 tx=0x7fe22c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe2357fa640 1 -- 172.21.15.115:0/2540904169 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c005ce0 con 0x7fe2380b1c50 2023-12-01T01:53:02.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe2357fa640 1 -- 172.21.15.115:0/2540904169 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c005e40 con 0x7fe2380b1c50 2023-12-01T01:53:02.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.440+0000 7fe2357fa640 1 -- 172.21.15.115:0/2540904169 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0072f0 con 0x7fe2380b1c50 2023-12-01T01:53:02.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.441+0000 7fe23eece640 1 -- 172.21.15.115:0/2540904169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23813ebf0 con 0x7fe2380b1c50 2023-12-01T01:53:02.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.441+0000 7fe23eece640 1 -- 172.21.15.115:0/2540904169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23813e320 con 0x7fe2380b1c50 2023-12-01T01:53:02.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.442+0000 7fe23eece640 1 -- 172.21.15.115:0/2540904169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23813f3f0 con 0x7fe2380b1c50 2023-12-01T01:53:02.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.443+0000 7fe2357fa640 1 -- 172.21.15.115:0/2540904169 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 49) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe22c007450 con 0x7fe2380b1c50 2023-12-01T01:53:02.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.444+0000 7fe2357fa640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe21c06a550 0x7fe21c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:02.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.444+0000 7fe2357fa640 1 -- 172.21.15.115:0/2540904169 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe22c0880e0 con 0x7fe2380b1c50 2023-12-01T01:53:02.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.444+0000 7fe237fff640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe21c06a550 0x7fe21c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:02.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.445+0000 7fe237fff640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe21c06a550 0x7fe21c06ca10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe224006fd0 tx=0x7fe224008040 comp rx=0 tx=0).ready entity=mgr.7230 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:02.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.448+0000 7fe2357fa640 1 -- 172.21.15.115:0/2540904169 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe23813f3f0 con 0x7fe2380b1c50 2023-12-01T01:53:02.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:02.686+0000 7fe23eece640 1 -- 172.21.15.115:0/2540904169 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe23813f3f0 con 0x7fe2380b1c50 2023-12-01T01:53:03.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.609+0000 7fe2357fa640 1 -- 172.21.15.115:0/2540904169 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 50) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fe22c053980 con 0x7fe2380b1c50 2023-12-01T01:53:03.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.611+0000 7fe2357fa640 1 -- 172.21.15.115:0/2540904169 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v50) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe22c072270 con 0x7fe2380b1c50 2023-12-01T01:53:03.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 -- 172.21.15.115:0/2540904169 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe21c06a550 msgr2=0x7fe21c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe21c06a550 0x7fe21c06ca10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe224006fd0 tx=0x7fe224008040 comp rx=0 tx=0).stop 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 -- 172.21.15.115:0/2540904169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 msgr2=0x7fe23813dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 0x7fe23813dde0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00c390 tx=0x7fe22c00c850 comp rx=0 tx=0).stop 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 -- 172.21.15.115:0/2540904169 shutdown_connections 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe21c06a550 0x7fe21c06ca10 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe224006fd0 tx=0x7fe224008040 comp rx=0 tx=0).stop 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe238142870 0x7fe2380aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2380cd1e0 0x7fe2380aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 --2- 172.21.15.115:0/2540904169 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2380b1c50 0x7fe23813dde0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 -- 172.21.15.115:0/2540904169 >> 172.21.15.115:0/2540904169 conn(0x7fe238065960 msgr2=0x7fe238067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 -- 172.21.15.115:0/2540904169 shutdown_connections 2023-12-01T01:53:03.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.614+0000 7fe23eece640 1 -- 172.21.15.115:0/2540904169 wait complete. 2023-12-01T01:53:03.636 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T01:53:03.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.830+0000 7f036dff0640 1 Processor -- start 2023-12-01T01:53:03.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.830+0000 7f036dff0640 1 -- start start 2023-12-01T01:53:03.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.831+0000 7f036dff0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0368142870 0x7f03680ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.831+0000 7f036dff0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f03680cd010 0x7f03680b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.831+0000 7f036dff0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03680b1c50 0x7f036815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.831+0000 7f036dff0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03680b3ae0 con 0x7f03680b1c50 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.831+0000 7f036dff0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0368142c50 con 0x7f03680cd010 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.831+0000 7f036dff0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03680cd470 con 0x7f0368142870 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.832+0000 7f0367fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03680b1c50 0x7f036815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.832+0000 7f0367fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03680b1c50 0x7f036815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50490/0 (socket says 172.21.15.115:50490) 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.832+0000 7f0367fff640 1 -- 172.21.15.115:0/2385936178 learned_addr learned my addr 172.21.15.115:0/2385936178 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.832+0000 7f035ffff640 1 --2- 172.21.15.115:0/2385936178 >> v2:172.21.15.142:3300/0 conn(0x7f03680cd010 0x7f03680b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.832+0000 7f03677fe640 1 --2- 172.21.15.115:0/2385936178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0368142870 0x7f03680ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.832+0000 7f0367fff640 1 -- 172.21.15.115:0/2385936178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0368142870 msgr2=0x7f03680ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:03.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.832+0000 7f0367fff640 1 --2- 172.21.15.115:0/2385936178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0368142870 0x7f03680ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.832+0000 7f0367fff640 1 -- 172.21.15.115:0/2385936178 >> v2:172.21.15.142:3300/0 conn(0x7f03680cd010 msgr2=0x7f03680b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:03.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.832+0000 7f0367fff640 1 --2- 172.21.15.115:0/2385936178 >> v2:172.21.15.142:3300/0 conn(0x7f03680cd010 0x7f03680b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.832+0000 7f0367fff640 1 -- 172.21.15.115:0/2385936178 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03680b20e0 con 0x7f03680b1c50 2023-12-01T01:53:03.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.833+0000 7f0367fff640 1 --2- 172.21.15.115:0/2385936178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680b1c50 0x7f036815d690 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f036806d4c0 tx=0x7f036002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be049ee94aaa3c19 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:03.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.834+0000 7f0366ffd640 1 -- 172.21.15.115:0/2385936178 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036002ca70 con 0x7f03680b1c50 2023-12-01T01:53:03.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.834+0000 7f0366ffd640 1 -- 172.21.15.115:0/2385936178 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036002cbd0 con 0x7f03680b1c50 2023-12-01T01:53:03.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.834+0000 7f0366ffd640 1 -- 172.21.15.115:0/2385936178 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600343f0 con 0x7f03680b1c50 2023-12-01T01:53:03.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.834+0000 7f036dff0640 1 -- 172.21.15.115:0/2385936178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680b1c50 msgr2=0x7f036815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:03.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.834+0000 7f036dff0640 1 --2- 172.21.15.115:0/2385936178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680b1c50 0x7f036815d690 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f036806d4c0 tx=0x7f036002bed0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.835+0000 7f036dff0640 1 -- 172.21.15.115:0/2385936178 shutdown_connections 2023-12-01T01:53:03.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.835+0000 7f036dff0640 1 --2- 172.21.15.115:0/2385936178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680b1c50 0x7f036815d690 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.835+0000 7f036dff0640 1 --2- 172.21.15.115:0/2385936178 >> v2:172.21.15.142:3300/0 conn(0x7f03680cd010 0x7f03680b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.835+0000 7f036dff0640 1 --2- 172.21.15.115:0/2385936178 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0368142870 0x7f03680ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.835+0000 7f036dff0640 1 -- 172.21.15.115:0/2385936178 >> 172.21.15.115:0/2385936178 conn(0x7f0368065960 msgr2=0x7f0368065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:03.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.835+0000 7f036dff0640 1 -- 172.21.15.115:0/2385936178 shutdown_connections 2023-12-01T01:53:03.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.835+0000 7f036dff0640 1 -- 172.21.15.115:0/2385936178 wait complete. 2023-12-01T01:53:03.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.836+0000 7f036dff0640 1 Processor -- start 2023-12-01T01:53:03.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.837+0000 7f036dff0640 1 -- start start 2023-12-01T01:53:03.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f036dff0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03680b1c50 0x7f0368151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:03.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f036dff0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680cd010 0x7f03680c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:03.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f036dff0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0368142870 0x7f03680c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:03.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f036dff0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036813dd20 con 0x7f03680cd010 2023-12-01T01:53:03.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f036dff0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0368151cb0 con 0x7f03680b1c50 2023-12-01T01:53:03.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f036dff0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03680ac4f0 con 0x7f0368142870 2023-12-01T01:53:03.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f035ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680cd010 0x7f03680c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f035ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680cd010 0x7f03680c3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50506/0 (socket says 172.21.15.115:50506) 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f035ffff640 1 -- 172.21.15.115:0/3428474675 learned_addr learned my addr 172.21.15.115:0/3428474675 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f03677fe640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03680b1c50 0x7f0368151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f0367fff640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0368142870 0x7f03680c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f035ffff640 1 -- 172.21.15.115:0/3428474675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0368142870 msgr2=0x7f03680c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f035ffff640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0368142870 0x7f03680c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.838+0000 7f035ffff640 1 -- 172.21.15.115:0/3428474675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03680b1c50 msgr2=0x7f0368151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.839+0000 7f035ffff640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03680b1c50 0x7f0368151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.839+0000 7f035ffff640 1 -- 172.21.15.115:0/3428474675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0360009b30 con 0x7f03680cd010 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.839+0000 7f03677fe640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03680b1c50 0x7f0368151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:53:03.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.839+0000 7f035ffff640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680cd010 0x7f03680c3b20 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f035000d800 tx=0x7f035000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.839+0000 7f0364ff9640 1 -- 172.21.15.115:0/3428474675 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03500096a0 con 0x7f03680cd010 2023-12-01T01:53:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.839+0000 7f0364ff9640 1 -- 172.21.15.115:0/3428474675 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0350009800 con 0x7f03680cd010 2023-12-01T01:53:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.839+0000 7f0364ff9640 1 -- 172.21.15.115:0/3428474675 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350009b90 con 0x7f03680cd010 2023-12-01T01:53:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.839+0000 7f036dff0640 1 -- 172.21.15.115:0/3428474675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03680ac7d0 con 0x7f03680cd010 2023-12-01T01:53:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.839+0000 7f036dff0640 1 -- 172.21.15.115:0/3428474675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03680acd90 con 0x7f03680cd010 2023-12-01T01:53:03.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.841+0000 7f036dff0640 1 -- 172.21.15.115:0/3428474675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036804e890 con 0x7f03680cd010 2023-12-01T01:53:03.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.842+0000 7f0364ff9640 1 -- 172.21.15.115:0/3428474675 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 50) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f0350009cf0 con 0x7f03680cd010 2023-12-01T01:53:03.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.844+0000 7f0364ff9640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034806a5d0 0x7f034806ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:03.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.844+0000 7f03677fe640 1 -- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034806a5d0 msgr2=0x7f034806ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:53:03.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.844+0000 7f0364ff9640 1 -- 172.21.15.115:0/3428474675 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0350089150 con 0x7f03680cd010 2023-12-01T01:53:03.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.844+0000 7f03677fe640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034806a5d0 0x7f034806ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:53:03.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:03.848+0000 7f0364ff9640 1 -- 172.21.15.115:0/3428474675 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0350054630 con 0x7f03680cd010 2023-12-01T01:53:04.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.039+0000 7f036dff0640 1 -- 172.21.15.115:0/3428474675 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f036806b6d0 con 0x7f03680cd010 2023-12-01T01:53:04.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.045+0000 7f03677fe640 1 -- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034806a5d0 msgr2=0x7f034806ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:53:04.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.045+0000 7f03677fe640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034806a5d0 0x7f034806ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:53:04.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.047+0000 7f0364ff9640 1 -- 172.21.15.115:0/3428474675 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f035005ce60 con 0x7f03680cd010 2023-12-01T01:53:04.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 -- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034806a5d0 msgr2=0x7f034806ca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:53:04.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034806a5d0 0x7f034806ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 -- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680cd010 msgr2=0x7f03680c3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:04.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680cd010 0x7f03680c3b20 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f035000d800 tx=0x7f035000dcc0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 -- 172.21.15.115:0/3428474675 shutdown_connections 2023-12-01T01:53:04.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f034806a5d0 0x7f034806ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0368142870 0x7f03680c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03680cd010 0x7f03680c3b20 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 --2- 172.21.15.115:0/3428474675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03680b1c50 0x7f0368151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 -- 172.21.15.115:0/3428474675 >> 172.21.15.115:0/3428474675 conn(0x7f0368065960 msgr2=0x7f0368070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:04.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 -- 172.21.15.115:0/3428474675 shutdown_connections 2023-12-01T01:53:04.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.049+0000 7f036dff0640 1 -- 172.21.15.115:0/3428474675 wait complete. 2023-12-01T01:53:04.067 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T01:53:04.147 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:04.145+0000 7fa5056d3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:53:04.171 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:04.170+0000 7fa4a120e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:53:04.202 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:04.201+0000 7fa5056d3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:53:04.229 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:04.228+0000 7fa4a120e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:53:04.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.260+0000 7f1067fff640 1 Processor -- start 2023-12-01T01:53:04.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.260+0000 7f1067fff640 1 -- start start 2023-12-01T01:53:04.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.263+0000 7f1067fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f10680cfb80 0x7f10680ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.263+0000 7f1067fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1068154010 0x7f1068160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.263+0000 7f1067fff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10680cdcd0 con 0x7f1068153c30 2023-12-01T01:53:04.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.263+0000 7f1067fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1068161830 con 0x7f10680cfb80 2023-12-01T01:53:04.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.263+0000 7f1067fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10680cd3a0 con 0x7f1068154010 2023-12-01T01:53:04.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.263+0000 7f1066ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1068153c30 0x7f10680cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57256/0 (socket says 172.21.15.115:57256) 2023-12-01T01:53:04.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.263+0000 7f1066ffd640 1 -- 172.21.15.115:0/849707980 learned_addr learned my addr 172.21.15.115:0/849707980 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:53:04.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.263+0000 7f105ffff640 1 --2- 172.21.15.115:0/849707980 >> v2:172.21.15.115:3300/0 conn(0x7f10680cfb80 0x7f10680ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:04.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.263+0000 7f10677fe640 1 --2- 172.21.15.115:0/849707980 >> v2:172.21.15.142:3300/0 conn(0x7f1068154010 0x7f1068160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:04.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.263+0000 7f10667fc640 1 -- 172.21.15.115:0/849707980 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059938926 0 0) 0x7f10680cdcd0 con 0x7f1068153c30 2023-12-01T01:53:04.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.264+0000 7f10667fc640 1 -- 172.21.15.115:0/849707980 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1054003520 con 0x7f1068153c30 2023-12-01T01:53:04.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.264+0000 7f105ffff640 1 -- 172.21.15.115:0/849707980 >> v2:172.21.15.142:3300/0 conn(0x7f1068154010 msgr2=0x7f1068160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:04.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.264+0000 7f105ffff640 1 --2- 172.21.15.115:0/849707980 >> v2:172.21.15.142:3300/0 conn(0x7f1068154010 0x7f1068160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.264+0000 7f105ffff640 1 -- 172.21.15.115:0/849707980 >> v1:172.21.15.115:6789/0 conn(0x7f1068153c30 legacy=0x7f10680cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:04.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.264+0000 7f105ffff640 1 -- 172.21.15.115:0/849707980 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10680cd680 con 0x7f10680cfb80 2023-12-01T01:53:04.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.264+0000 7f10677fe640 1 --2- 172.21.15.115:0/849707980 >> v2:172.21.15.142:3300/0 conn(0x7f1068154010 0x7f1068160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:53:04.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.265+0000 7f105ffff640 1 --2- 172.21.15.115:0/849707980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 0x7f10680ce160 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f106815af90 tx=0x7f105002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab8f2f5fdccd96b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:04.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.265+0000 7f10667fc640 1 -- 172.21.15.115:0/849707980 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050031830 con 0x7f10680cfb80 2023-12-01T01:53:04.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.265+0000 7f10667fc640 1 -- 172.21.15.115:0/849707980 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1050031990 con 0x7f10680cfb80 2023-12-01T01:53:04.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.265+0000 7f10667fc640 1 -- 172.21.15.115:0/849707980 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050031d20 con 0x7f10680cfb80 2023-12-01T01:53:04.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.266+0000 7f1067fff640 1 -- 172.21.15.115:0/849707980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 msgr2=0x7f10680ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:04.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.266+0000 7f1067fff640 1 --2- 172.21.15.115:0/849707980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 0x7f10680ce160 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f106815af90 tx=0x7f105002b8d0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.266+0000 7f1067fff640 1 -- 172.21.15.115:0/849707980 shutdown_connections 2023-12-01T01:53:04.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.266+0000 7f1067fff640 1 --2- 172.21.15.115:0/849707980 >> v2:172.21.15.142:3300/0 conn(0x7f1068154010 0x7f1068160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.266+0000 7f1067fff640 1 --2- 172.21.15.115:0/849707980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 0x7f10680ce160 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.266+0000 7f1067fff640 1 -- 172.21.15.115:0/849707980 >> 172.21.15.115:0/849707980 conn(0x7f106806a0b0 msgr2=0x7f106806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:04.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.266+0000 7f1067fff640 1 -- 172.21.15.115:0/849707980 shutdown_connections 2023-12-01T01:53:04.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.266+0000 7f1067fff640 1 -- 172.21.15.115:0/849707980 wait complete. 2023-12-01T01:53:04.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.267+0000 7f1067fff640 1 Processor -- start 2023-12-01T01:53:04.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.268+0000 7f1067fff640 1 -- start start 2023-12-01T01:53:04.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.268+0000 7f1067fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 0x7f106815a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1067fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1068153c30 0x7f1068144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1067fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1068154010 0x7f10681517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1067fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681f6470 con 0x7f10680cfb80 2023-12-01T01:53:04.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1067fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1068159af0 con 0x7f1068153c30 2023-12-01T01:53:04.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1067fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106815adb0 con 0x7f1068154010 2023-12-01T01:53:04.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1066ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 0x7f106815a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:04.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1066ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 0x7f106815a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50518/0 (socket says 172.21.15.115:50518) 2023-12-01T01:53:04.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1066ffd640 1 -- 172.21.15.115:0/3051824113 learned_addr learned my addr 172.21.15.115:0/3051824113 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:04.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1066ffd640 1 -- 172.21.15.115:0/3051824113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1068154010 msgr2=0x7f10681517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:04.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1066ffd640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1068154010 0x7f10681517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1066ffd640 1 -- 172.21.15.115:0/3051824113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1068153c30 msgr2=0x7f1068144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:04.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1066ffd640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1068153c30 0x7f1068144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1066ffd640 1 -- 172.21.15.115:0/3051824113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1050009400 con 0x7f10680cfb80 2023-12-01T01:53:04.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.269+0000 7f1066ffd640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 0x7f106815a870 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f105800c500 tx=0x7f105800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:04.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.270+0000 7f1064ff9640 1 -- 172.21.15.115:0/3051824113 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058005ce0 con 0x7f10680cfb80 2023-12-01T01:53:04.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.270+0000 7f1064ff9640 1 -- 172.21.15.115:0/3051824113 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1058005e40 con 0x7f10680cfb80 2023-12-01T01:53:04.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.270+0000 7f1064ff9640 1 -- 172.21.15.115:0/3051824113 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058007330 con 0x7f10680cfb80 2023-12-01T01:53:04.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.270+0000 7f1067fff640 1 -- 172.21.15.115:0/3051824113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10681f6190 con 0x7f10680cfb80 2023-12-01T01:53:04.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.270+0000 7f1067fff640 1 -- 172.21.15.115:0/3051824113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106815a070 con 0x7f10680cfb80 2023-12-01T01:53:04.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.271+0000 7f1067fff640 1 -- 172.21.15.115:0/3051824113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10680618d0 con 0x7f10680cfb80 2023-12-01T01:53:04.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.272+0000 7f1064ff9640 1 -- 172.21.15.115:0/3051824113 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 50) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f1058007550 con 0x7f10680cfb80 2023-12-01T01:53:04.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.273+0000 7f1064ff9640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f104806a5a0 0x7f104806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.273+0000 7f1064ff9640 1 -- 172.21.15.115:0/3051824113 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1058088110 con 0x7f10680cfb80 2023-12-01T01:53:04.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.273+0000 7f105ffff640 1 -- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f104806a5a0 msgr2=0x7f104806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:53:04.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.273+0000 7f105ffff640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f104806a5a0 0x7f104806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:53:04.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.277+0000 7f1064ff9640 1 -- 172.21.15.115:0/3051824113 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1058053740 con 0x7f10680cfb80 2023-12-01T01:53:04.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.469+0000 7f1067fff640 1 -- 172.21.15.115:0/3051824113 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f10680d1ba0 con 0x7f10680cfb80 2023-12-01T01:53:04.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.474+0000 7f105ffff640 1 -- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f104806a5a0 msgr2=0x7f104806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:53:04.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.474+0000 7f105ffff640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f104806a5a0 0x7f104806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:53:04.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.475+0000 7f1064ff9640 1 -- 172.21.15.115:0/3051824113 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1058056d60 con 0x7f10680cfb80 2023-12-01T01:53:04.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 -- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f104806a5a0 msgr2=0x7f104806ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f104806a5a0 0x7f104806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 -- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 msgr2=0x7f106815a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 0x7f106815a870 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f105800c500 tx=0x7f105800c9c0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 -- 172.21.15.115:0/3051824113 shutdown_connections 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f104806a5a0 0x7f104806ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1068154010 0x7f10681517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1068153c30 0x7f1068144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 --2- 172.21.15.115:0/3051824113 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10680cfb80 0x7f106815a870 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 -- 172.21.15.115:0/3051824113 >> 172.21.15.115:0/3051824113 conn(0x7f106806a0b0 msgr2=0x7f1068150fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 -- 172.21.15.115:0/3051824113 shutdown_connections 2023-12-01T01:53:04.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.478+0000 7f1067fff640 1 -- 172.21.15.115:0/3051824113 wait complete. 2023-12-01T01:53:04.497 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /bad-path 1M 2023-12-01T01:53:04.676 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:04.675+0000 7fa5056d3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:53:04.687 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:04.686+0000 7fa4a120e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:53:04.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.703+0000 7fcd6f577640 1 Processor -- start 2023-12-01T01:53:04.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.703+0000 7fcd6f577640 1 -- start start 2023-12-01T01:53:04.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.704+0000 7fcd6f577640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd6815cd90 0x7fcd680cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.704+0000 7fcd6f577640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd680cfd60 0x7fcd68161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.704+0000 7fcd6f577640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd680ce5e0 con 0x7fcd680b5fa0 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.704+0000 7fcd6f577640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd680d0ec0 con 0x7fcd680cfd60 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.704+0000 7fcd6f577640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd68161950 con 0x7fcd6815cd90 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.705+0000 7fcd6db14640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd680cfd60 0x7fcd68161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.705+0000 7fcd6db14640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd680cfd60 0x7fcd68161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50522/0 (socket says 172.21.15.115:50522) 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.705+0000 7fcd6db14640 1 -- 172.21.15.115:0/896412705 learned_addr learned my addr 172.21.15.115:0/896412705 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.705+0000 7fcd6d313640 1 --2- 172.21.15.115:0/896412705 >> v2:172.21.15.142:3300/0 conn(0x7fcd6815cd90 0x7fcd680cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.705+0000 7fcd6db14640 1 -- 172.21.15.115:0/896412705 >> v2:172.21.15.142:3300/0 conn(0x7fcd6815cd90 msgr2=0x7fcd680cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.705+0000 7fcd6db14640 1 --2- 172.21.15.115:0/896412705 >> v2:172.21.15.142:3300/0 conn(0x7fcd6815cd90 0x7fcd680cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.705+0000 7fcd6db14640 1 -- 172.21.15.115:0/896412705 >> v1:172.21.15.142:6789/0 conn(0x7fcd680b5fa0 legacy=0x7fcd680cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.705+0000 7fcd6db14640 1 -- 172.21.15.115:0/896412705 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680ce300 con 0x7fcd680cfd60 2023-12-01T01:53:04.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.705+0000 7fcd6d313640 1 --2- 172.21.15.115:0/896412705 >> v2:172.21.15.142:3300/0 conn(0x7fcd6815cd90 0x7fcd680cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:53:04.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.706+0000 7fcd6db14640 1 --2- 172.21.15.115:0/896412705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 0x7fcd68161410 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fcd6806bd10 tx=0x7fcd6002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7d28091bab32a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:04.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.707+0000 7fcd5ffff640 1 -- 172.21.15.115:0/896412705 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6002ca80 con 0x7fcd680cfd60 2023-12-01T01:53:04.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.707+0000 7fcd5ffff640 1 -- 172.21.15.115:0/896412705 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6002cbe0 con 0x7fcd680cfd60 2023-12-01T01:53:04.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.707+0000 7fcd5ffff640 1 -- 172.21.15.115:0/896412705 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd600343f0 con 0x7fcd680cfd60 2023-12-01T01:53:04.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.707+0000 7fcd6f577640 1 -- 172.21.15.115:0/896412705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 msgr2=0x7fcd68161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:04.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.707+0000 7fcd6f577640 1 --2- 172.21.15.115:0/896412705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 0x7fcd68161410 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fcd6806bd10 tx=0x7fcd6002bee0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.708+0000 7fcd6f577640 1 -- 172.21.15.115:0/896412705 shutdown_connections 2023-12-01T01:53:04.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.708+0000 7fcd6f577640 1 --2- 172.21.15.115:0/896412705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 0x7fcd68161410 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.708+0000 7fcd6f577640 1 --2- 172.21.15.115:0/896412705 >> v2:172.21.15.142:3300/0 conn(0x7fcd6815cd90 0x7fcd680cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.708+0000 7fcd6f577640 1 -- 172.21.15.115:0/896412705 >> 172.21.15.115:0/896412705 conn(0x7fcd68065960 msgr2=0x7fcd68065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:04.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.708+0000 7fcd6f577640 1 -- 172.21.15.115:0/896412705 shutdown_connections 2023-12-01T01:53:04.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.708+0000 7fcd6f577640 1 -- 172.21.15.115:0/896412705 wait complete. 2023-12-01T01:53:04.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.709+0000 7fcd6f577640 1 Processor -- start 2023-12-01T01:53:04.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.709+0000 7fcd6f577640 1 -- start start 2023-12-01T01:53:04.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.710+0000 7fcd6f577640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd680b5fa0 0x7fcd681531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.710+0000 7fcd6f577640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 0x7fcd68154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.710+0000 7fcd6f577640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd6815cd90 0x7fcd6815fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.710+0000 7fcd6f577640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd68155750 con 0x7fcd680cfd60 2023-12-01T01:53:04.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.710+0000 7fcd6f577640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680cb5b0 con 0x7fcd680b5fa0 2023-12-01T01:53:04.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.710+0000 7fcd6f577640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680cb890 con 0x7fcd6815cd90 2023-12-01T01:53:04.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.711+0000 7fcd6cb12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 0x7fcd68154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:04.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.711+0000 7fcd6cb12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 0x7fcd68154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50530/0 (socket says 172.21.15.115:50530) 2023-12-01T01:53:04.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.711+0000 7fcd6cb12640 1 -- 172.21.15.115:0/579897103 learned_addr learned my addr 172.21.15.115:0/579897103 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.711+0000 7fcd6d313640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd680b5fa0 0x7fcd681531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.711+0000 7fcd6cb12640 1 -- 172.21.15.115:0/579897103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd6815cd90 msgr2=0x7fcd6815fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.711+0000 7fcd6cb12640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd6815cd90 0x7fcd6815fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.711+0000 7fcd6cb12640 1 -- 172.21.15.115:0/579897103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd680b5fa0 msgr2=0x7fcd681531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.711+0000 7fcd6cb12640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd680b5fa0 0x7fcd681531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.711+0000 7fcd6cb12640 1 -- 172.21.15.115:0/579897103 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd60009b30 con 0x7fcd680cfd60 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.711+0000 7fcd6cb12640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 0x7fcd68154de0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800b3c0 tx=0x7fcd5800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.712+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58011830 con 0x7fcd680cfd60 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.712+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58011990 con 0x7fcd680cfd60 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.712+0000 7fcd6f577640 1 -- 172.21.15.115:0/579897103 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd680cbb70 con 0x7fcd680cfd60 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.712+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58011d80 con 0x7fcd680cfd60 2023-12-01T01:53:04.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.712+0000 7fcd6f577640 1 -- 172.21.15.115:0/579897103 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd68155500 con 0x7fcd680cfd60 2023-12-01T01:53:04.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.713+0000 7fcd6f577640 1 -- 172.21.15.115:0/579897103 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd680cb410 con 0x7fcd680cfd60 2023-12-01T01:53:04.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.714+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 50) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fcd5801b460 con 0x7fcd680cfd60 2023-12-01T01:53:04.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.715+0000 7fcd5dffb640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:04.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.715+0000 7fcd6d313640 1 -- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 msgr2=0x7fcd4c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:53:04.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.715+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd58088700 con 0x7fcd680cfd60 2023-12-01T01:53:04.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.715+0000 7fcd6d313640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:53:04.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.723+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcd680cb410 con 0x7fcd680cfd60 2023-12-01T01:53:04.736 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:04.735+0000 7fa5056d3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:53:04.741 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:04.740+0000 7fa4a120e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:53:04.789 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:04.787+0000 7fa5056d3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:53:04.790 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:04.789+0000 7fa4a120e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:53:04.850 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:04.849+0000 7fa4a120e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:53:04.852 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:04.850+0000 7fa5056d3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:53:04.899 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:04.898+0000 7fa4a120e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:53:04.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:04.902+0000 7fa5056d3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:53:04.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.916+0000 7fcd6d313640 1 -- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 msgr2=0x7fcd4c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:53:04.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.916+0000 7fcd6d313640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:53:04.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:04.935+0000 7fcd6f577640 1 -- 172.21.15.115:0/579897103 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd680d1380 con 0x7fcd4c06a5a0 2023-12-01T01:53:05.009 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:05.009+0000 7fa4a120e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:53:05.016 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:05.014+0000 7fa5056d3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:53:05.072 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:05.071+0000 7fa4a120e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:53:05.079 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:05.077+0000 7fa5056d3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:53:05.122 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:05.122+0000 7fa4a120e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:53:05.128 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:05.127+0000 7fa5056d3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:53:05.179 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:05.178+0000 7fa4a120e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:53:05.184 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:05.183+0000 7fa5056d3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:53:05.228 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:05.227+0000 7fa4a120e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:53:05.234 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:05.233+0000 7fa5056d3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:53:05.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:05.317+0000 7fcd6d313640 1 -- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 msgr2=0x7fcd4c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:53:05.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:05.317+0000 7fcd6d313640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:53:05.608 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:05.607+0000 7fa4a120e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:53:05.615 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:05.614+0000 7fa5056d3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:53:05.765 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:05.764+0000 7fa4a120e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:53:05.774 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:05.773+0000 7fa5056d3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:53:05.873 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:05.872+0000 7fa4a120e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:53:05.883 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:05.881+0000 7fa5056d3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:53:05.923 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:05.922+0000 7fa4a120e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:53:05.933 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:05.931+0000 7fa5056d3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:53:06.029 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:06.029+0000 7fa4a120e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:53:06.040 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:06.039+0000 7fa5056d3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:53:06.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:06.118+0000 7fcd6d313640 1 -- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 msgr2=0x7fcd4c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:53:06.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:06.118+0000 7fcd6d313640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:53:06.135 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:06.134+0000 7fa4a120e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:53:06.146 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:06.144+0000 7fa5056d3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:53:06.289 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:06.288+0000 7fa4a120e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:53:06.301 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:06.299+0000 7fa5056d3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:53:06.338 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:06.337+0000 7fa4a120e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:53:06.349 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:06.348+0000 7fa5056d3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:53:06.458 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:06.455+0000 7fa4a120e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:53:06.467 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:06.465+0000 7fa5056d3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:53:07.269 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:07.268+0000 7fa4a120e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:53:07.297 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:07.295+0000 7fa5056d3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:53:07.564 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:07.563+0000 7fa4a120e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:53:07.618 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:07.616+0000 7fa5056d3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:53:07.621 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:07.620+0000 7fa4a120e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:53:07.676 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:07.674+0000 7fa5056d3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:53:07.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:07.720+0000 7fcd6d313640 1 -- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 msgr2=0x7fcd4c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:53:07.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:07.720+0000 7fcd6d313640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-01T01:53:07.733 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:53:07.733+0000 7fa4a120e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:53:07.790 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:07.789+0000 7fa5056d3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:53:07.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:07.800+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 51) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fcd58053f10 con 0x7fcd680cfd60 2023-12-01T01:53:07.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:07.816+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 52) v1 ==== 88335+0+0 (secure 0 0 0) 0x7fcd5805f350 con 0x7fcd680cfd60 2023-12-01T01:53:07.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:07.816+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 msgr2=0x7fcd4c06ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:53:07.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:07.816+0000 7fcd5dffb640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:08.016 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:08.015+0000 7fa4829bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:53:08.017 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:08.015+0000 7fa4829bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:53:08.017 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:08.015+0000 7fa4829bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:53:08.017 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:08.015+0000 7fa4829bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:53:08.017 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:08.015+0000 7fa4829bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:53:08.017 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:08.015+0000 7fa4859c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:53:08.017 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:08.015+0000 7fa4859c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:53:08.017 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:08.015+0000 7fa4859c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:53:08.017 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:08.015+0000 7fa4859c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:53:08.017 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:53:08.015+0000 7fa4859c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:53:08.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:08.841+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 <== mon.0 v2:172.21.15.115:3300/0 9 ==== mgrmap(e 53) v1 ==== 88462+0+0 (secure 0 0 0) 0x7fcd580547e0 con 0x7fcd680cfd60 2023-12-01T01:53:08.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:08.842+0000 7fcd5dffb640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:08.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:08.842+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd4c027760 con 0x7fcd4c06a5a0 2023-12-01T01:53:08.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:08.851+0000 7fcd6d313640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:08.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:08.854+0000 7fcd6d313640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06c9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd64008010 tx=0x7fcd64008040 comp rx=0 tx=0).ready entity=mgr.7656 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:09.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.014+0000 7fcd5dffb640 1 -- 172.21.15.115:0/579897103 <== mgr.7656 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fcd4c027760 con 0x7fcd4c06a5a0 2023-12-01T01:53:09.016 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /bad-path 2023-12-01T01:53:09.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.017+0000 7fcd6f577640 1 -- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 msgr2=0x7fcd4c06c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:09.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.017+0000 7fcd6f577640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06c9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd64008010 tx=0x7fcd64008040 comp rx=0 tx=0).stop 2023-12-01T01:53:09.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.017+0000 7fcd6f577640 1 -- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 msgr2=0x7fcd68154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:09.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.017+0000 7fcd6f577640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 0x7fcd68154de0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800b3c0 tx=0x7fcd5800b880 comp rx=0 tx=0).stop 2023-12-01T01:53:09.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.018+0000 7fcd6f577640 1 -- 172.21.15.115:0/579897103 shutdown_connections 2023-12-01T01:53:09.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.018+0000 7fcd6f577640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd4c06a5a0 0x7fcd4c06c9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.018+0000 7fcd6f577640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd6815cd90 0x7fcd6815fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.018+0000 7fcd6f577640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd680cfd60 0x7fcd68154de0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.018+0000 7fcd6f577640 1 --2- 172.21.15.115:0/579897103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd680b5fa0 0x7fcd681531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.018+0000 7fcd6f577640 1 -- 172.21.15.115:0/579897103 >> 172.21.15.115:0/579897103 conn(0x7fcd68065960 msgr2=0x7fcd680b6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:09.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.018+0000 7fcd6f577640 1 -- 172.21.15.115:0/579897103 shutdown_connections 2023-12-01T01:53:09.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.018+0000 7fcd6f577640 1 -- 172.21.15.115:0/579897103 wait complete. 2023-12-01T01:53:09.041 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-12-01T01:53:09.041 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-12-01T01:53:09.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.254+0000 7ff5c86bd640 1 Processor -- start 2023-12-01T01:53:09.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.254+0000 7ff5c86bd640 1 -- start start 2023-12-01T01:53:09.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.256+0000 7ff5c86bd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5c0142870 0x7ff5c00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:09.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.256+0000 7ff5c86bd640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c00b2f30 con 0x7ff5c00b1e20 2023-12-01T01:53:09.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.256+0000 7ff5c86bd640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c00b2320 con 0x7ff5c00cd010 2023-12-01T01:53:09.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.256+0000 7ff5c86bd640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c00b3ae0 con 0x7ff5c0142870 2023-12-01T01:53:09.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.256+0000 7ff5c6459640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5c0142870 0x7ff5c00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:09.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.256+0000 7ff5c6459640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5c0142870 0x7ff5c00ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44358/0 (socket says 172.21.15.115:44358) 2023-12-01T01:53:09.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.256+0000 7ff5c6459640 1 -- 172.21.15.115:0/1314752229 learned_addr learned my addr 172.21.15.115:0/1314752229 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:09.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.256+0000 7ff5c6c5a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff5c00b1e20 0x7ff5c015a0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37788/0 (socket says 172.21.15.115:37788) 2023-12-01T01:53:09.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.256+0000 7ff5c6459640 1 -- 172.21.15.115:0/1314752229 >> v1:172.21.15.142:6789/0 conn(0x7ff5c00cd010 legacy=0x7ff5c00b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:53:09.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.256+0000 7ff5c6459640 1 -- 172.21.15.115:0/1314752229 >> v1:172.21.15.115:6789/0 conn(0x7ff5c00b1e20 legacy=0x7ff5c015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:09.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.257+0000 7ff5c6459640 1 -- 172.21.15.115:0/1314752229 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c00b2600 con 0x7ff5c0142870 2023-12-01T01:53:09.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.258+0000 7ff5c6459640 1 --2- 172.21.15.115:0/1314752229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c0142870 0x7ff5c00ccad0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009870 tx=0x7ff5bc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ea251b2eaeea68 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:09.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.258+0000 7ff5c5457640 1 -- 172.21.15.115:0/1314752229 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc030860 con 0x7ff5c0142870 2023-12-01T01:53:09.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.258+0000 7ff5c5457640 1 -- 172.21.15.115:0/1314752229 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc0309c0 con 0x7ff5c0142870 2023-12-01T01:53:09.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.258+0000 7ff5c5457640 1 -- 172.21.15.115:0/1314752229 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc030d50 con 0x7ff5c0142870 2023-12-01T01:53:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.258+0000 7ff5c86bd640 1 -- 172.21.15.115:0/1314752229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c0142870 msgr2=0x7ff5c00ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.258+0000 7ff5c86bd640 1 --2- 172.21.15.115:0/1314752229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c0142870 0x7ff5c00ccad0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009870 tx=0x7ff5bc02b9a0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.259+0000 7ff5c86bd640 1 -- 172.21.15.115:0/1314752229 shutdown_connections 2023-12-01T01:53:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.259+0000 7ff5c86bd640 1 --2- 172.21.15.115:0/1314752229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c0142870 0x7ff5c00ccad0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.259+0000 7ff5c86bd640 1 -- 172.21.15.115:0/1314752229 >> 172.21.15.115:0/1314752229 conn(0x7ff5c0065960 msgr2=0x7ff5c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:09.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.259+0000 7ff5c86bd640 1 -- 172.21.15.115:0/1314752229 shutdown_connections 2023-12-01T01:53:09.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.259+0000 7ff5c86bd640 1 -- 172.21.15.115:0/1314752229 wait complete. 2023-12-01T01:53:09.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.260+0000 7ff5c86bd640 1 Processor -- start 2023-12-01T01:53:09.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.261+0000 7ff5c86bd640 1 -- start start 2023-12-01T01:53:09.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.261+0000 7ff5c86bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c00b1e20 0x7ff5c013dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:09.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.261+0000 7ff5c86bd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c00cd010 0x7ff5c00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:09.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.261+0000 7ff5c86bd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5c0142870 0x7ff5c00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.261+0000 7ff5c86bd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c013e200 con 0x7ff5c00b1e20 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.261+0000 7ff5c86bd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c015de60 con 0x7ff5c00cd010 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.261+0000 7ff5c86bd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00aef40 con 0x7ff5c0142870 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.262+0000 7ff5c6459640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c00b1e20 0x7ff5c013dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.262+0000 7ff5c6459640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c00b1e20 0x7ff5c013dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44364/0 (socket says 172.21.15.115:44364) 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.262+0000 7ff5c6459640 1 -- 172.21.15.115:0/3431999492 learned_addr learned my addr 172.21.15.115:0/3431999492 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.262+0000 7ff5c5c58640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c00cd010 0x7ff5c00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.262+0000 7ff5c6459640 1 -- 172.21.15.115:0/3431999492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5c0142870 msgr2=0x7ff5c00707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.262+0000 7ff5c6459640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5c0142870 0x7ff5c00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.262+0000 7ff5c6459640 1 -- 172.21.15.115:0/3431999492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c00cd010 msgr2=0x7ff5c00702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.262+0000 7ff5c6459640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c00cd010 0x7ff5c00702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.262+0000 7ff5c6459640 1 -- 172.21.15.115:0/3431999492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc0094d0 con 0x7ff5c00b1e20 2023-12-01T01:53:09.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.262+0000 7ff5c6459640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c00b1e20 0x7ff5c013dcc0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009870 tx=0x7ff5bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:09.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.263+0000 7ff5b6ffd640 1 -- 172.21.15.115:0/3431999492 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc039b60 con 0x7ff5c00b1e20 2023-12-01T01:53:09.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.263+0000 7ff5b6ffd640 1 -- 172.21.15.115:0/3431999492 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc039cc0 con 0x7ff5c00b1e20 2023-12-01T01:53:09.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.263+0000 7ff5b6ffd640 1 -- 172.21.15.115:0/3431999492 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc043630 con 0x7ff5c00b1e20 2023-12-01T01:53:09.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.263+0000 7ff5c86bd640 1 -- 172.21.15.115:0/3431999492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c00af220 con 0x7ff5c00b1e20 2023-12-01T01:53:09.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.263+0000 7ff5c86bd640 1 -- 172.21.15.115:0/3431999492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c00af7e0 con 0x7ff5c00b1e20 2023-12-01T01:53:09.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.264+0000 7ff5c86bd640 1 -- 172.21.15.115:0/3431999492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c00afa20 con 0x7ff5c00b1e20 2023-12-01T01:53:09.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.265+0000 7ff5b6ffd640 1 -- 172.21.15.115:0/3431999492 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 53) v1 ==== 88462+0+0 (secure 0 0 0) 0x7ff5bc02f3b0 con 0x7ff5c00b1e20 2023-12-01T01:53:09.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.266+0000 7ff5b6ffd640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5a406a2d0 0x7ff5a406c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:53:09.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.266+0000 7ff5b6ffd640 1 -- 172.21.15.115:0/3431999492 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7ff5bc0aab50 con 0x7ff5c00b1e20 2023-12-01T01:53:09.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.266+0000 7ff5c5c58640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5a406a2d0 0x7ff5a406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:53:09.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.267+0000 7ff5c5c58640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5a406a2d0 0x7ff5a406c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00ae710 tx=0x7ff5b0009040 comp rx=0 tx=0).ready entity=mgr.7656 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:53:09.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.270+0000 7ff5b6ffd640 1 -- 172.21.15.115:0/3431999492 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff5c00afa20 con 0x7ff5c00b1e20 2023-12-01T01:53:09.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.485+0000 7ff5c86bd640 1 -- 172.21.15.115:0/3431999492 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5c00b28e0 con 0x7ff5a406a2d0 2023-12-01T01:53:09.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.501+0000 7ff5b6ffd640 1 -- 172.21.15.115:0/3431999492 <== mgr.7656 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7ff5b00050d0 con 0x7ff5a406a2d0 2023-12-01T01:53:09.503 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:53:09.503 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:53:08", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-01T01:53:09.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.504+0000 7ff5c86bd640 1 -- 172.21.15.115:0/3431999492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5a406a2d0 msgr2=0x7ff5a406c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.504+0000 7ff5c86bd640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5a406a2d0 0x7ff5a406c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00ae710 tx=0x7ff5b0009040 comp rx=0 tx=0).stop 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.504+0000 7ff5c86bd640 1 -- 172.21.15.115:0/3431999492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c00b1e20 msgr2=0x7ff5c013dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.504+0000 7ff5c86bd640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c00b1e20 0x7ff5c013dcc0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009870 tx=0x7ff5bc004730 comp rx=0 tx=0).stop 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.505+0000 7ff5c86bd640 1 -- 172.21.15.115:0/3431999492 shutdown_connections 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.505+0000 7ff5c86bd640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff5a406a2d0 0x7ff5a406c790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.505+0000 7ff5c86bd640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5c0142870 0x7ff5c00707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.505+0000 7ff5c86bd640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c00cd010 0x7ff5c00702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.505+0000 7ff5c86bd640 1 --2- 172.21.15.115:0/3431999492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5c00b1e20 0x7ff5c013dcc0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.505+0000 7ff5c86bd640 1 -- 172.21.15.115:0/3431999492 >> 172.21.15.115:0/3431999492 conn(0x7ff5c0065960 msgr2=0x7ff5c006f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.505+0000 7ff5c86bd640 1 -- 172.21.15.115:0/3431999492 shutdown_connections 2023-12-01T01:53:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:53:09.505+0000 7ff5c86bd640 1 -- 172.21.15.115:0/3431999492 wait complete. 2023-12-01T01:53:09.525 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:53:08", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-01T01:53:29.733 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:53:29.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:53:59.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:53:59.890 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:54:00.038 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:00.037+0000 7fa4751a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:00.039 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:00.037+0000 7fa4751a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:00.039 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:00.037+0000 7fa4751a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:00.039 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:00.037+0000 7fa4751a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:00.039 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:00.037+0000 7fa4751a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:09.527 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-12-01T01:54:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.755+0000 7f7e8d884640 1 Processor -- start 2023-12-01T01:54:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.756+0000 7f7e8d884640 1 -- start start 2023-12-01T01:54:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.758+0000 7f7e8d884640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7e88142870 0x7f7e880ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.758+0000 7f7e8d884640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7e880cd010 0x7f7e880b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.758+0000 7f7e8d884640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e880b1c50 0x7f7e8815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.758+0000 7f7e8d884640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e880b3ae0 con 0x7f7e88142870 2023-12-01T01:54:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.758+0000 7f7e8d884640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e880cd4a0 con 0x7f7e880cd010 2023-12-01T01:54:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.758+0000 7f7e8d884640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e88142c50 con 0x7f7e880b1c50 2023-12-01T01:54:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.758+0000 7f7e86ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7e88142870 0x7f7e880ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.758+0000 7f7e86ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7e88142870 0x7f7e880ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43646/0 (socket says 172.21.15.115:43646) 2023-12-01T01:54:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.758+0000 7f7e86ffd640 1 -- 172.21.15.115:0/149938852 learned_addr learned my addr 172.21.15.115:0/149938852 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.759+0000 7f7e877fe640 1 --2- 172.21.15.115:0/149938852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e880b1c50 0x7f7e8815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.759+0000 7f7e86ffd640 1 -- 172.21.15.115:0/149938852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e880b1c50 msgr2=0x7f7e8815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.759+0000 7f7e86ffd640 1 --2- 172.21.15.115:0/149938852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e880b1c50 0x7f7e8815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.759+0000 7f7e7ffff640 1 --2- 172.21.15.115:0/149938852 >> v2:172.21.15.142:3300/0 conn(0x7f7e880cd010 0x7f7e880b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.759+0000 7f7e86ffd640 1 -- 172.21.15.115:0/149938852 >> v2:172.21.15.142:3300/0 conn(0x7f7e880cd010 msgr2=0x7f7e880b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.759+0000 7f7e86ffd640 1 --2- 172.21.15.115:0/149938852 >> v2:172.21.15.142:3300/0 conn(0x7f7e880cd010 0x7f7e880b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.759+0000 7f7e86ffd640 1 -- 172.21.15.115:0/149938852 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e880b20e0 con 0x7f7e88142870 2023-12-01T01:54:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.759+0000 7f7e7ffff640 1 --2- 172.21.15.115:0/149938852 >> v2:172.21.15.142:3300/0 conn(0x7f7e880cd010 0x7f7e880b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.760+0000 7f7e86ffd640 1 --2- 172.21.15.115:0/149938852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e88142870 0x7f7e880ccad0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7e880cf7f0 tx=0x7f7e7802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aadf02b382e0c18 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.760+0000 7f7e867fc640 1 -- 172.21.15.115:0/149938852 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78031640 con 0x7f7e88142870 2023-12-01T01:54:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.760+0000 7f7e867fc640 1 -- 172.21.15.115:0/149938852 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e780317a0 con 0x7f7e88142870 2023-12-01T01:54:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.760+0000 7f7e867fc640 1 -- 172.21.15.115:0/149938852 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78031b30 con 0x7f7e88142870 2023-12-01T01:54:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.761+0000 7f7e8d884640 1 -- 172.21.15.115:0/149938852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e88142870 msgr2=0x7f7e880ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.761+0000 7f7e8d884640 1 --2- 172.21.15.115:0/149938852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e88142870 0x7f7e880ccad0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7e880cf7f0 tx=0x7f7e7802b8f0 comp rx=0 tx=0).stop 2023-12-01T01:54:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.761+0000 7f7e8d884640 1 -- 172.21.15.115:0/149938852 shutdown_connections 2023-12-01T01:54:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.761+0000 7f7e8d884640 1 --2- 172.21.15.115:0/149938852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e880b1c50 0x7f7e8815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.761+0000 7f7e8d884640 1 --2- 172.21.15.115:0/149938852 >> v2:172.21.15.142:3300/0 conn(0x7f7e880cd010 0x7f7e880b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.761+0000 7f7e8d884640 1 --2- 172.21.15.115:0/149938852 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e88142870 0x7f7e880ccad0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.761+0000 7f7e8d884640 1 -- 172.21.15.115:0/149938852 >> 172.21.15.115:0/149938852 conn(0x7f7e88065960 msgr2=0x7f7e88065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.761+0000 7f7e8d884640 1 -- 172.21.15.115:0/149938852 shutdown_connections 2023-12-01T01:54:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.761+0000 7f7e8d884640 1 -- 172.21.15.115:0/149938852 wait complete. 2023-12-01T01:54:09.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.763+0000 7f7e8d884640 1 Processor -- start 2023-12-01T01:54:09.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.763+0000 7f7e8d884640 1 -- start start 2023-12-01T01:54:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.764+0000 7f7e8d884640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e880b1c50 0x7f7e8815d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.764+0000 7f7e8d884640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e880cd010 0x7f7e88159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.764+0000 7f7e8d884640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e88142870 0x7f7e88159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.764+0000 7f7e8d884640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880cb710 con 0x7f7e880cd010 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.764+0000 7f7e8d884640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880cb430 con 0x7f7e880b1c50 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.764+0000 7f7e8d884640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880c5350 con 0x7f7e88142870 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.764+0000 7f7e7ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e880cd010 0x7f7e88159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.764+0000 7f7e7ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e880cd010 0x7f7e88159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43648/0 (socket says 172.21.15.115:43648) 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.764+0000 7f7e7ffff640 1 -- 172.21.15.115:0/137015381 learned_addr learned my addr 172.21.15.115:0/137015381 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.764+0000 7f7e86ffd640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e880b1c50 0x7f7e8815d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e877fe640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e88142870 0x7f7e88159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e7ffff640 1 -- 172.21.15.115:0/137015381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e88142870 msgr2=0x7f7e88159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e7ffff640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e88142870 0x7f7e88159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e7ffff640 1 -- 172.21.15.115:0/137015381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e880b1c50 msgr2=0x7f7e8815d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e7ffff640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e880b1c50 0x7f7e8815d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e7ffff640 1 -- 172.21.15.115:0/137015381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e780094d0 con 0x7f7e880cd010 2023-12-01T01:54:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e86ffd640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e880b1c50 0x7f7e8815d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e877fe640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e88142870 0x7f7e88159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e7ffff640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e880cd010 0x7f7e88159100 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000c7f0 tx=0x7f7e7000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e84ff9640 1 -- 172.21.15.115:0/137015381 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70007a30 con 0x7f7e880cd010 2023-12-01T01:54:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e84ff9640 1 -- 172.21.15.115:0/137015381 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e70007b90 con 0x7f7e880cd010 2023-12-01T01:54:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e8d884640 1 -- 172.21.15.115:0/137015381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e880c5630 con 0x7f7e880cd010 2023-12-01T01:54:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e84ff9640 1 -- 172.21.15.115:0/137015381 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000ce40 con 0x7f7e880cd010 2023-12-01T01:54:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.765+0000 7f7e8d884640 1 -- 172.21.15.115:0/137015381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8815d6e0 con 0x7f7e880cd010 2023-12-01T01:54:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.766+0000 7f7e8d884640 1 -- 172.21.15.115:0/137015381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8804e890 con 0x7f7e880cd010 2023-12-01T01:54:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.767+0000 7f7e84ff9640 1 -- 172.21.15.115:0/137015381 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 56) v1 ==== 88840+0+0 (secure 0 0 0) 0x7f7e700113f0 con 0x7f7e880cd010 2023-12-01T01:54:09.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.768+0000 7f7e84ff9640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7e6806a6c0 0x7f7e6806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:09.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.768+0000 7f7e84ff9640 1 -- 172.21.15.115:0/137015381 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f7e700892c0 con 0x7f7e880cd010 2023-12-01T01:54:09.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.769+0000 7f7e86ffd640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7e6806a6c0 0x7f7e6806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:09.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.769+0000 7f7e86ffd640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7e6806a6c0 0x7f7e6806cb80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7e78036000 tx=0x7f7e7803c040 comp rx=0 tx=0).ready entity=mgr.7656 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:09.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.772+0000 7f7e84ff9640 1 -- 172.21.15.115:0/137015381 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7e70054240 con 0x7f7e880cd010 2023-12-01T01:54:09.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:09.984+0000 7f7e8d884640 1 -- 172.21.15.115:0/137015381 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e880b2320 con 0x7f7e6806a6c0 2023-12-01T01:54:10.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.000+0000 7f7e84ff9640 1 -- 172.21.15.115:0/137015381 <== mgr.7656 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f7e78031ae0 con 0x7f7e6806a6c0 2023-12-01T01:54:10.000 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:10.000 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:53:08", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-12-01T01:54:10.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.002+0000 7f7e8d884640 1 -- 172.21.15.115:0/137015381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7e6806a6c0 msgr2=0x7f7e6806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.002+0000 7f7e8d884640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7e6806a6c0 0x7f7e6806cb80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7e78036000 tx=0x7f7e7803c040 comp rx=0 tx=0).stop 2023-12-01T01:54:10.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.002+0000 7f7e8d884640 1 -- 172.21.15.115:0/137015381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e880cd010 msgr2=0x7f7e88159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.002+0000 7f7e8d884640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e880cd010 0x7f7e88159100 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000c7f0 tx=0x7f7e7000cbf0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.002+0000 7f7e8d884640 1 -- 172.21.15.115:0/137015381 shutdown_connections 2023-12-01T01:54:10.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.002+0000 7f7e8d884640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7e6806a6c0 0x7f7e6806cb80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.002+0000 7f7e8d884640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7e88142870 0x7f7e88159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.002+0000 7f7e8d884640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7e880cd010 0x7f7e88159100 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.002+0000 7f7e8d884640 1 --2- 172.21.15.115:0/137015381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e880b1c50 0x7f7e8815d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.002+0000 7f7e8d884640 1 -- 172.21.15.115:0/137015381 >> 172.21.15.115:0/137015381 conn(0x7f7e88065960 msgr2=0x7f7e8806f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:10.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.003+0000 7f7e8d884640 1 -- 172.21.15.115:0/137015381 shutdown_connections 2023-12-01T01:54:10.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.003+0000 7f7e8d884640 1 -- 172.21.15.115:0/137015381 wait complete. 2023-12-01T01:54:10.023 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:53:08", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-12-01T01:54:10.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /bad-path 2023-12-01T01:54:10.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.229+0000 7f961ac69640 1 Processor -- start 2023-12-01T01:54:10.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.229+0000 7f961ac69640 1 -- start start 2023-12-01T01:54:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.231+0000 7f961ac69640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f961415cfa0 0x7f96140b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.231+0000 7f961ac69640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f96140b63e0 0x7f96140cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.231+0000 7f961ac69640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140b6890 con 0x7f96140b7e70 2023-12-01T01:54:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.231+0000 7f961ac69640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96140b6b70 con 0x7f96140b63e0 2023-12-01T01:54:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.231+0000 7f961ac69640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140b6e50 con 0x7f961415cfa0 2023-12-01T01:54:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.231+0000 7f9619206640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f96140b63e0 0x7f96140cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.231+0000 7f9619206640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f96140b63e0 0x7f96140cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43660/0 (socket says 172.21.15.115:43660) 2023-12-01T01:54:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.231+0000 7f9619206640 1 -- 172.21.15.115:0/1829372156 learned_addr learned my addr 172.21.15.115:0/1829372156 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.232+0000 7f9618a05640 1 --2- 172.21.15.115:0/1829372156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f961415cfa0 0x7f96140b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.232+0000 7f9619206640 1 -- 172.21.15.115:0/1829372156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f961415cfa0 msgr2=0x7f96140b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.232+0000 7f9619206640 1 --2- 172.21.15.115:0/1829372156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f961415cfa0 0x7f96140b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.232+0000 7f9619206640 1 -- 172.21.15.115:0/1829372156 >> v1:172.21.15.142:6789/0 conn(0x7f96140b7e70 legacy=0x7f96140b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.232+0000 7f9619206640 1 -- 172.21.15.115:0/1829372156 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140b7130 con 0x7f96140b63e0 2023-12-01T01:54:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.233+0000 7f9619206640 1 --2- 172.21.15.115:0/1829372156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96140b63e0 0x7f96140cfc10 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f960c009e50 tx=0x7f960c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375e5543c6e72dc0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.233+0000 7f96137fe640 1 -- 172.21.15.115:0/1829372156 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c02c9e0 con 0x7f96140b63e0 2023-12-01T01:54:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.233+0000 7f96137fe640 1 -- 172.21.15.115:0/1829372156 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c02cb40 con 0x7f96140b63e0 2023-12-01T01:54:10.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.233+0000 7f96137fe640 1 -- 172.21.15.115:0/1829372156 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0343f0 con 0x7f96140b63e0 2023-12-01T01:54:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.234+0000 7f961ac69640 1 -- 172.21.15.115:0/1829372156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96140b63e0 msgr2=0x7f96140cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.234+0000 7f961ac69640 1 --2- 172.21.15.115:0/1829372156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96140b63e0 0x7f96140cfc10 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f960c009e50 tx=0x7f960c02be40 comp rx=0 tx=0).stop 2023-12-01T01:54:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.234+0000 7f961ac69640 1 -- 172.21.15.115:0/1829372156 shutdown_connections 2023-12-01T01:54:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.234+0000 7f961ac69640 1 --2- 172.21.15.115:0/1829372156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96140b63e0 0x7f96140cfc10 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.234+0000 7f961ac69640 1 --2- 172.21.15.115:0/1829372156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f961415cfa0 0x7f96140b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.234+0000 7f961ac69640 1 -- 172.21.15.115:0/1829372156 >> 172.21.15.115:0/1829372156 conn(0x7f961406a0b0 msgr2=0x7f961406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.234+0000 7f961ac69640 1 -- 172.21.15.115:0/1829372156 shutdown_connections 2023-12-01T01:54:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.234+0000 7f961ac69640 1 -- 172.21.15.115:0/1829372156 wait complete. 2023-12-01T01:54:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.236+0000 7f961ac69640 1 Processor -- start 2023-12-01T01:54:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.236+0000 7f961ac69640 1 -- start start 2023-12-01T01:54:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.237+0000 7f961ac69640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96140b63e0 0x7f96140cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.237+0000 7f961ac69640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96140b7e70 0x7f961414fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.237+0000 7f961ac69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f961415cfa0 0x7f96140caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.237+0000 7f961ac69640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140cbdd0 con 0x7f961415cfa0 2023-12-01T01:54:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.237+0000 7f961ac69640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96141503c0 con 0x7f96140b63e0 2023-12-01T01:54:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.237+0000 7f961ac69640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140cb010 con 0x7f96140b7e70 2023-12-01T01:54:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.237+0000 7f9619206640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f961415cfa0 0x7f96140caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.237+0000 7f9619206640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f961415cfa0 0x7f96140caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43676/0 (socket says 172.21.15.115:43676) 2023-12-01T01:54:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.237+0000 7f9619206640 1 -- 172.21.15.115:0/80195724 learned_addr learned my addr 172.21.15.115:0/80195724 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f9618a05640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96140b63e0 0x7f96140cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f9613fff640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96140b7e70 0x7f961414fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f9619206640 1 -- 172.21.15.115:0/80195724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96140b7e70 msgr2=0x7f961414fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f9619206640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96140b7e70 0x7f961414fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f9619206640 1 -- 172.21.15.115:0/80195724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96140b63e0 msgr2=0x7f96140cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f9619206640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96140b63e0 0x7f96140cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f9619206640 1 -- 172.21.15.115:0/80195724 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c009b30 con 0x7f961415cfa0 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f9613fff640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96140b7e70 0x7f961414fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f9619206640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f961415cfa0 0x7f96140caad0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f960c009e50 tx=0x7f960c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f96117fa640 1 -- 172.21.15.115:0/80195724 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c009510 con 0x7f961415cfa0 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f96117fa640 1 -- 172.21.15.115:0/80195724 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c03e450 con 0x7f961415cfa0 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f96117fa640 1 -- 172.21.15.115:0/80195724 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c03e7e0 con 0x7f961415cfa0 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.238+0000 7f961ac69640 1 -- 172.21.15.115:0/80195724 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96140b8c00 con 0x7f961415cfa0 2023-12-01T01:54:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.239+0000 7f961ac69640 1 -- 172.21.15.115:0/80195724 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96140c9d10 con 0x7f961415cfa0 2023-12-01T01:54:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.240+0000 7f961ac69640 1 -- 172.21.15.115:0/80195724 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961404e890 con 0x7f961415cfa0 2023-12-01T01:54:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.240+0000 7f96117fa640 1 -- 172.21.15.115:0/80195724 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 56) v1 ==== 88840+0+0 (secure 0 0 0) 0x7f960c03e940 con 0x7f961415cfa0 2023-12-01T01:54:10.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.241+0000 7f96117fa640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f95f806a6c0 0x7f95f806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.241+0000 7f96117fa640 1 -- 172.21.15.115:0/80195724 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f960c0abf80 con 0x7f961415cfa0 2023-12-01T01:54:10.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.242+0000 7f9618a05640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f95f806a6c0 0x7f95f806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:10.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.242+0000 7f9618a05640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f95f806a6c0 0x7f95f806cb80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9608006fd0 tx=0x7f9608008040 comp rx=0 tx=0).ready entity=mgr.7656 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.246+0000 7f96117fa640 1 -- 172.21.15.115:0/80195724 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f960c077010 con 0x7f961415cfa0 2023-12-01T01:54:10.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.457+0000 7f961ac69640 1 -- 172.21.15.115:0/80195724 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f96140b8a80 con 0x7f95f806a6c0 2023-12-01T01:54:10.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.614+0000 7f96117fa640 1 -- 172.21.15.115:0/80195724 <== mgr.7656 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f960800b200 con 0x7f95f806a6c0 2023-12-01T01:54:10.615 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /bad-path 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 -- 172.21.15.115:0/80195724 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f95f806a6c0 msgr2=0x7f95f806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f95f806a6c0 0x7f95f806cb80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9608006fd0 tx=0x7f9608008040 comp rx=0 tx=0).stop 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 -- 172.21.15.115:0/80195724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f961415cfa0 msgr2=0x7f96140caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f961415cfa0 0x7f96140caad0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f960c009e50 tx=0x7f960c004a40 comp rx=0 tx=0).stop 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 -- 172.21.15.115:0/80195724 shutdown_connections 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f95f806a6c0 0x7f95f806cb80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96140b7e70 0x7f961414fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f961415cfa0 0x7f96140caad0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 --2- 172.21.15.115:0/80195724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96140b63e0 0x7f96140cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 -- 172.21.15.115:0/80195724 >> 172.21.15.115:0/80195724 conn(0x7f961406a0b0 msgr2=0x7f9614151170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:10.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 -- 172.21.15.115:0/80195724 shutdown_connections 2023-12-01T01:54:10.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.618+0000 7f961ac69640 1 -- 172.21.15.115:0/80195724 wait complete. 2023-12-01T01:54:10.640 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-12-01T01:54:10.640 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T01:54:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.854+0000 7ff44c318640 1 Processor -- start 2023-12-01T01:54:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.854+0000 7ff44c318640 1 -- start start 2023-12-01T01:54:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff44c318640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff4440cc440 0x7ff4440b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff44c318640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4440cb290 con 0x7ff444160200 2023-12-01T01:54:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff44c318640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4440b0ae0 con 0x7ff4440b0f80 2023-12-01T01:54:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff44c318640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4440b35f0 con 0x7ff4440cc440 2023-12-01T01:54:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff44a0b4640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff444160200 0x7ff44406b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44484/0 (socket says 172.21.15.115:44484) 2023-12-01T01:54:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff44a0b4640 1 -- 172.21.15.115:0/3896944685 learned_addr learned my addr 172.21.15.115:0/3896944685 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff44a8b5640 1 --2- 172.21.15.115:0/3896944685 >> v2:172.21.15.115:3300/0 conn(0x7ff4440cc440 0x7ff4440b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff44a8b5640 1 -- 172.21.15.115:0/3896944685 >> v1:172.21.15.142:6789/0 conn(0x7ff4440b0f80 legacy=0x7ff4440cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff4490b2640 1 -- 172.21.15.115:0/3896944685 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857174332 0 0) 0x7ff4440cb290 con 0x7ff444160200 2023-12-01T01:54:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff44a8b5640 1 -- 172.21.15.115:0/3896944685 >> v1:172.21.15.115:6789/0 conn(0x7ff444160200 legacy=0x7ff44406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.856+0000 7ff44a8b5640 1 -- 172.21.15.115:0/3896944685 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440b3310 con 0x7ff4440cc440 2023-12-01T01:54:10.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.857+0000 7ff44a8b5640 1 --2- 172.21.15.115:0/3896944685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440cc440 0x7ff4440b2c80 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009d90 tx=0x7ff43c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fd72d5d7091a48 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:10.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.858+0000 7ff4490b2640 1 -- 172.21.15.115:0/3896944685 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c02cc60 con 0x7ff4440cc440 2023-12-01T01:54:10.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.858+0000 7ff4490b2640 1 -- 172.21.15.115:0/3896944685 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c02cdc0 con 0x7ff4440cc440 2023-12-01T01:54:10.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.858+0000 7ff4490b2640 1 -- 172.21.15.115:0/3896944685 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c034660 con 0x7ff4440cc440 2023-12-01T01:54:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.858+0000 7ff44c318640 1 -- 172.21.15.115:0/3896944685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440cc440 msgr2=0x7ff4440b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.858+0000 7ff44c318640 1 --2- 172.21.15.115:0/3896944685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440cc440 0x7ff4440b2c80 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009d90 tx=0x7ff43c02beb0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.858+0000 7ff44c318640 1 -- 172.21.15.115:0/3896944685 shutdown_connections 2023-12-01T01:54:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.858+0000 7ff44c318640 1 --2- 172.21.15.115:0/3896944685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440cc440 0x7ff4440b2c80 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.858+0000 7ff44c318640 1 -- 172.21.15.115:0/3896944685 >> 172.21.15.115:0/3896944685 conn(0x7ff444065960 msgr2=0x7ff444065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.858+0000 7ff44c318640 1 -- 172.21.15.115:0/3896944685 shutdown_connections 2023-12-01T01:54:10.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.858+0000 7ff44c318640 1 -- 172.21.15.115:0/3896944685 wait complete. 2023-12-01T01:54:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.860+0000 7ff44c318640 1 Processor -- start 2023-12-01T01:54:10.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.860+0000 7ff44c318640 1 -- start start 2023-12-01T01:54:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.861+0000 7ff44c318640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440b0f80 0x7ff44414d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.861+0000 7ff44c318640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4440cc440 0x7ff444138e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.861+0000 7ff44c318640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff444160200 0x7ff4441393d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.861+0000 7ff44c318640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444139910 con 0x7ff4440b0f80 2023-12-01T01:54:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.861+0000 7ff44c318640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44414b910 con 0x7ff4440cc440 2023-12-01T01:54:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.861+0000 7ff44c318640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44414bbf0 con 0x7ff444160200 2023-12-01T01:54:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.861+0000 7ff44a0b4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440b0f80 0x7ff44414d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.862+0000 7ff44a0b4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440b0f80 0x7ff44414d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43692/0 (socket says 172.21.15.115:43692) 2023-12-01T01:54:10.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.862+0000 7ff44a0b4640 1 -- 172.21.15.115:0/3714274945 learned_addr learned my addr 172.21.15.115:0/3714274945 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:10.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.862+0000 7ff44a0b4640 1 -- 172.21.15.115:0/3714274945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff444160200 msgr2=0x7ff4441393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.862+0000 7ff44a0b4640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff444160200 0x7ff4441393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.862+0000 7ff4498b3640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4440cc440 0x7ff444138e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.862+0000 7ff44a0b4640 1 -- 172.21.15.115:0/3714274945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4440cc440 msgr2=0x7ff444138e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.862+0000 7ff44a0b4640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4440cc440 0x7ff444138e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.862+0000 7ff44a0b4640 1 -- 172.21.15.115:0/3714274945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c009a70 con 0x7ff4440b0f80 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.862+0000 7ff44a0b4640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440b0f80 0x7ff44414d3a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff44000c4d0 tx=0x7ff44000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.862+0000 7ff43affd640 1 -- 172.21.15.115:0/3714274945 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4400074e0 con 0x7ff4440b0f80 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.863+0000 7ff43affd640 1 -- 172.21.15.115:0/3714274945 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff440007660 con 0x7ff4440b0f80 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.863+0000 7ff43affd640 1 -- 172.21.15.115:0/3714274945 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440004a20 con 0x7ff4440b0f80 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.863+0000 7ff44c318640 1 -- 172.21.15.115:0/3714274945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44414bed0 con 0x7ff4440b0f80 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.863+0000 7ff44c318640 1 -- 172.21.15.115:0/3714274945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44414c490 con 0x7ff4440b0f80 2023-12-01T01:54:10.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.864+0000 7ff44c318640 1 -- 172.21.15.115:0/3714274945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44404e890 con 0x7ff4440b0f80 2023-12-01T01:54:10.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.865+0000 7ff43affd640 1 -- 172.21.15.115:0/3714274945 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 56) v1 ==== 88840+0+0 (secure 0 0 0) 0x7ff440004b80 con 0x7ff4440b0f80 2023-12-01T01:54:10.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.866+0000 7ff43affd640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff42806a6c0 0x7ff42806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:10.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.866+0000 7ff43affd640 1 -- 172.21.15.115:0/3714274945 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7ff440088190 con 0x7ff4440b0f80 2023-12-01T01:54:10.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.867+0000 7ff4498b3640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff42806a6c0 0x7ff42806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:10.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.867+0000 7ff4498b3640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff42806a6c0 0x7ff42806cb80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4440cb240 tx=0x7ff434008040 comp rx=0 tx=0).ready entity=mgr.7656 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:10.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:10.871+0000 7ff43affd640 1 -- 172.21.15.115:0/3714274945 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff440053220 con 0x7ff4440b0f80 2023-12-01T01:54:11.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.111+0000 7ff44c318640 1 -- 172.21.15.115:0/3714274945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7ff44415dfe0 con 0x7ff4440b0f80 2023-12-01T01:54:11.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.720+0000 7ff43affd640 1 -- 172.21.15.115:0/3714274945 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 57) v1 ==== 88823+0+0 (secure 0 0 0) 0x7ff4400535f0 con 0x7ff4440b0f80 2023-12-01T01:54:11.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.721+0000 7ff43affd640 1 -- 172.21.15.115:0/3714274945 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v57) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff440072cd0 con 0x7ff4440b0f80 2023-12-01T01:54:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.724+0000 7ff44c318640 1 -- 172.21.15.115:0/3714274945 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff42806a6c0 msgr2=0x7ff42806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.724+0000 7ff44c318640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff42806a6c0 0x7ff42806cb80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff4440cb240 tx=0x7ff434008040 comp rx=0 tx=0).stop 2023-12-01T01:54:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.724+0000 7ff44c318640 1 -- 172.21.15.115:0/3714274945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440b0f80 msgr2=0x7ff44414d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.724+0000 7ff44c318640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440b0f80 0x7ff44414d3a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff44000c4d0 tx=0x7ff44000c990 comp rx=0 tx=0).stop 2023-12-01T01:54:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.724+0000 7ff44c318640 1 -- 172.21.15.115:0/3714274945 shutdown_connections 2023-12-01T01:54:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.724+0000 7ff44c318640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff42806a6c0 0x7ff42806cb80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.724+0000 7ff44c318640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff444160200 0x7ff4441393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.724+0000 7ff44c318640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4440cc440 0x7ff444138e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:11.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.724+0000 7ff44c318640 1 --2- 172.21.15.115:0/3714274945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4440b0f80 0x7ff44414d3a0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:11.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.724+0000 7ff44c318640 1 -- 172.21.15.115:0/3714274945 >> 172.21.15.115:0/3714274945 conn(0x7ff444065960 msgr2=0x7ff44415d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:11.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.725+0000 7ff44c318640 1 -- 172.21.15.115:0/3714274945 shutdown_connections 2023-12-01T01:54:11.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:11.725+0000 7ff44c318640 1 -- 172.21.15.115:0/3714274945 wait complete. 2023-12-01T01:54:11.746 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:54:11.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:54:11.861 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:54:11.951 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:54:11.952 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:54:11.952 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:54:12.001 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:54:12.002 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:54:12.096 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:54:12.243 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:12.242+0000 7fc14303c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:54:12.276 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:12.275+0000 7f5564e50040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:54:12.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.292+0000 7f2f1bfff640 1 Processor -- start 2023-12-01T01:54:12.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.292+0000 7f2f1bfff640 1 -- start start 2023-12-01T01:54:12.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.294+0000 7f2f1bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 0x7f2f1c15db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.294+0000 7f2f1bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2f1c0d05d0 0x7f2f1c0d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.294+0000 7f2f1bfff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c0cd3f0 con 0x7f2f1c160670 2023-12-01T01:54:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.294+0000 7f2f1bfff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c0cd0f0 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.294+0000 7f2f1bfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c160ec0 con 0x7f2f1c15d7b0 2023-12-01T01:54:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.294+0000 7f2f1b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2f1c0d05d0 0x7f2f1c0d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.294+0000 7f2f1b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2f1c0d05d0 0x7f2f1c0d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43716/0 (socket says 172.21.15.115:43716) 2023-12-01T01:54:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.294+0000 7f2f1b7fe640 1 -- 172.21.15.115:0/399827979 learned_addr learned my addr 172.21.15.115:0/399827979 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:12.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.294+0000 7f2f1affd640 1 --2- 172.21.15.115:0/399827979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 0x7f2f1c15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:12.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.295+0000 7f2f1b7fe640 1 -- 172.21.15.115:0/399827979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 msgr2=0x7f2f1c15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:12.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.295+0000 7f2f1b7fe640 1 --2- 172.21.15.115:0/399827979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 0x7f2f1c15db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.295+0000 7f2f1b7fe640 1 -- 172.21.15.115:0/399827979 >> v1:172.21.15.142:6789/0 conn(0x7f2f1c160670 legacy=0x7f2f1c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:12.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.295+0000 7f2f1b7fe640 1 -- 172.21.15.115:0/399827979 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c1611a0 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.296+0000 7f2f1b7fe640 1 --2- 172.21.15.115:0/399827979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 0x7f2f1c0d0a60 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009e50 tx=0x7f2f1402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1cc5d4238e5309 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:12.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.297+0000 7f2f19ffb640 1 -- 172.21.15.115:0/399827979 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1402cb00 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.297+0000 7f2f19ffb640 1 -- 172.21.15.115:0/399827979 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1402cc60 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.297+0000 7f2f19ffb640 1 -- 172.21.15.115:0/399827979 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140343f0 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.297+0000 7f2f1bfff640 1 -- 172.21.15.115:0/399827979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 msgr2=0x7f2f1c0d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.297+0000 7f2f1bfff640 1 --2- 172.21.15.115:0/399827979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 0x7f2f1c0d0a60 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009e50 tx=0x7f2f1402bf60 comp rx=0 tx=0).stop 2023-12-01T01:54:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.298+0000 7f2f1bfff640 1 -- 172.21.15.115:0/399827979 shutdown_connections 2023-12-01T01:54:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.298+0000 7f2f1bfff640 1 --2- 172.21.15.115:0/399827979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 0x7f2f1c0d0a60 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.298+0000 7f2f1bfff640 1 --2- 172.21.15.115:0/399827979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 0x7f2f1c15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.298+0000 7f2f1bfff640 1 -- 172.21.15.115:0/399827979 >> 172.21.15.115:0/399827979 conn(0x7f2f1c06a0e0 msgr2=0x7f2f1c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.298+0000 7f2f1bfff640 1 -- 172.21.15.115:0/399827979 shutdown_connections 2023-12-01T01:54:12.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.298+0000 7f2f1bfff640 1 -- 172.21.15.115:0/399827979 wait complete. 2023-12-01T01:54:12.299 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:12.299+0000 7fc14303c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:54:12.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.299+0000 7f2f1bfff640 1 Processor -- start 2023-12-01T01:54:12.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.299+0000 7f2f1bfff640 1 -- start start 2023-12-01T01:54:12.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.300+0000 7f2f1bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 0x7f2f1c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.300+0000 7f2f1bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 0x7f2f1c0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.300+0000 7f2f1bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c160670 0x7f2f1c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.300+0000 7f2f1bfff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0b6fc0 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.300+0000 7f2f1bfff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0b6320 con 0x7f2f1c160670 2023-12-01T01:54:12.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.300+0000 7f2f1bfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0b6600 con 0x7f2f1c15d7b0 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 0x7f2f1c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 0x7f2f1c1517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43728/0 (socket says 172.21.15.115:43728) 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1affd640 1 -- 172.21.15.115:0/3750336099 learned_addr learned my addr 172.21.15.115:0/3750336099 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1a7fc640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 0x7f2f1c0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1b7fe640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c160670 0x7f2f1c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1affd640 1 -- 172.21.15.115:0/3750336099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 msgr2=0x7f2f1c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1affd640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 0x7f2f1c0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1affd640 1 -- 172.21.15.115:0/3750336099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c160670 msgr2=0x7f2f1c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1affd640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c160670 0x7f2f1c0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1affd640 1 -- 172.21.15.115:0/3750336099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f14009b30 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1a7fc640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 0x7f2f1c0b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1b7fe640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c160670 0x7f2f1c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:12.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.301+0000 7f2f1affd640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 0x7f2f1c1517f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00d800 tx=0x7f2f0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:12.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.302+0000 7f2f037fe640 1 -- 172.21.15.115:0/3750336099 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c00b6c0 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.302+0000 7f2f037fe640 1 -- 172.21.15.115:0/3750336099 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c00b820 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.302+0000 7f2f037fe640 1 -- 172.21.15.115:0/3750336099 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c00bbb0 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.302+0000 7f2f1bfff640 1 -- 172.21.15.115:0/3750336099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c0b6c50 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.302+0000 7f2f1bfff640 1 -- 172.21.15.115:0/3750336099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c0b68e0 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.303+0000 7f2f1bfff640 1 -- 172.21.15.115:0/3750336099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c0673a0 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.304+0000 7f2f037fe640 1 -- 172.21.15.115:0/3750336099 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 57) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f2f0c00bd10 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.305+0000 7f2f037fe640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2f0806a6a0 0x7f2f0806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.306+0000 7f2f037fe640 1 -- 172.21.15.115:0/3750336099 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f2f0c0893c0 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.306+0000 7f2f1a7fc640 1 -- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2f0806a6a0 msgr2=0x7f2f0806cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:12.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.306+0000 7f2f1a7fc640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2f0806a6a0 0x7f2f0806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:12.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.310+0000 7f2f037fe640 1 -- 172.21.15.115:0/3750336099 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2f0c0544f0 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.334 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:12.334+0000 7f5564e50040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:54:12.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.503+0000 7f2f1bfff640 1 -- 172.21.15.115:0/3750336099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f1c0d3100 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.505+0000 7f2f037fe640 1 -- 172.21.15.115:0/3750336099 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16662 (secure 0 0 0) 0x7f2f0c057b10 con 0x7f2f1c0d05d0 2023-12-01T01:54:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:12.508 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":112,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:53:07.800915+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:46.948807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:47.513140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:53:07.800872+0000","172.21.15.115:6822/88317":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/2605263330":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3479742140":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/1863268774":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/1925101108":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3510224424":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3799042057":"2023-12-02T01:53:07.800872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:54:12.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.507+0000 7f2f1a7fc640 1 -- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2f0806a6a0 msgr2=0x7f2f0806cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:12.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.507+0000 7f2f1a7fc640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2f0806a6a0 0x7f2f0806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:54:12.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 -- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2f0806a6a0 msgr2=0x7f2f0806cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:12.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2f0806a6a0 0x7f2f0806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 -- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 msgr2=0x7f2f1c1517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:12.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 0x7f2f1c1517f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00d800 tx=0x7f2f0c00dcc0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 -- 172.21.15.115:0/3750336099 shutdown_connections 2023-12-01T01:54:12.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2f0806a6a0 0x7f2f0806cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f1c15d7b0 0x7f2f1c0b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c160670 0x7f2f1c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 --2- 172.21.15.115:0/3750336099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f1c0d05d0 0x7f2f1c1517f0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 -- 172.21.15.115:0/3750336099 >> 172.21.15.115:0/3750336099 conn(0x7f2f1c06a0e0 msgr2=0x7f2f1c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:12.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 -- 172.21.15.115:0/3750336099 shutdown_connections 2023-12-01T01:54:12.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.508+0000 7f2f1bfff640 1 -- 172.21.15.115:0/3750336099 wait complete. 2023-12-01T01:54:12.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:52:46.948807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:52:47.513140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:54:12.528 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:54:12.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.732+0000 7f720a096640 1 Processor -- start 2023-12-01T01:54:12.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.733+0000 7f720a096640 1 -- start start 2023-12-01T01:54:12.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.734+0000 7f720a096640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f720415cd90 0x7f72040cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.734+0000 7f720a096640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f72040cfd60 0x7f7204161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.734+0000 7f720a096640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72040ce5e0 con 0x7f72040b5fa0 2023-12-01T01:54:12.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.734+0000 7f720a096640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72040d0ec0 con 0x7f72040cfd60 2023-12-01T01:54:12.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.734+0000 7f720a096640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7204161950 con 0x7f720415cd90 2023-12-01T01:54:12.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.734+0000 7f7203fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f72040cfd60 0x7f7204161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:12.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.734+0000 7f7203fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f72040cfd60 0x7f7204161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43732/0 (socket says 172.21.15.115:43732) 2023-12-01T01:54:12.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.734+0000 7f7203fff640 1 -- 172.21.15.115:0/3346686575 learned_addr learned my addr 172.21.15.115:0/3346686575 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:12.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.735+0000 7f72037fe640 1 --2- 172.21.15.115:0/3346686575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f720415cd90 0x7f72040cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:12.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.735+0000 7f7203fff640 1 -- 172.21.15.115:0/3346686575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f720415cd90 msgr2=0x7f72040cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:12.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.735+0000 7f7203fff640 1 --2- 172.21.15.115:0/3346686575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f720415cd90 0x7f72040cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.735+0000 7f7203fff640 1 -- 172.21.15.115:0/3346686575 >> v1:172.21.15.142:6789/0 conn(0x7f72040b5fa0 legacy=0x7f72040cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:12.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.735+0000 7f7203fff640 1 -- 172.21.15.115:0/3346686575 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72040ce300 con 0x7f72040cfd60 2023-12-01T01:54:12.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.736+0000 7f7203fff640 1 --2- 172.21.15.115:0/3346686575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040cfd60 0x7f7204161410 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009e50 tx=0x7f71fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7599c4caae28848e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:12.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.736+0000 7f7202ffd640 1 -- 172.21.15.115:0/3346686575 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc02ca70 con 0x7f72040cfd60 2023-12-01T01:54:12.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.736+0000 7f7202ffd640 1 -- 172.21.15.115:0/3346686575 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc02cbd0 con 0x7f72040cfd60 2023-12-01T01:54:12.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.736+0000 7f7202ffd640 1 -- 172.21.15.115:0/3346686575 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0343f0 con 0x7f72040cfd60 2023-12-01T01:54:12.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.737+0000 7f720a096640 1 -- 172.21.15.115:0/3346686575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040cfd60 msgr2=0x7f7204161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:12.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.737+0000 7f720a096640 1 --2- 172.21.15.115:0/3346686575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040cfd60 0x7f7204161410 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009e50 tx=0x7f71fc02bed0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.737+0000 7f720a096640 1 -- 172.21.15.115:0/3346686575 shutdown_connections 2023-12-01T01:54:12.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.737+0000 7f720a096640 1 --2- 172.21.15.115:0/3346686575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040cfd60 0x7f7204161410 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.737+0000 7f720a096640 1 --2- 172.21.15.115:0/3346686575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f720415cd90 0x7f72040cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.737+0000 7f720a096640 1 -- 172.21.15.115:0/3346686575 >> 172.21.15.115:0/3346686575 conn(0x7f7204065960 msgr2=0x7f7204065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:12.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.737+0000 7f720a096640 1 -- 172.21.15.115:0/3346686575 shutdown_connections 2023-12-01T01:54:12.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.737+0000 7f720a096640 1 -- 172.21.15.115:0/3346686575 wait complete. 2023-12-01T01:54:12.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.739+0000 7f720a096640 1 Processor -- start 2023-12-01T01:54:12.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.740+0000 7f720a096640 1 -- start start 2023-12-01T01:54:12.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f720a096640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040b5fa0 0x7f720415d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f720a096640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040cfd60 0x7f72040cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f720a096640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f720415cd90 0x7f720414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f720a096640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72041f25e0 con 0x7f72040b5fa0 2023-12-01T01:54:12.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f720a096640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040cb780 con 0x7f720415cd90 2023-12-01T01:54:12.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f720a096640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720415d850 con 0x7f72040cfd60 2023-12-01T01:54:12.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f72037fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040b5fa0 0x7f720415d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:12.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f72037fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040b5fa0 0x7f720415d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43748/0 (socket says 172.21.15.115:43748) 2023-12-01T01:54:12.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f72037fe640 1 -- 172.21.15.115:0/2598869095 learned_addr learned my addr 172.21.15.115:0/2598869095 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:12.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f71fbfff640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040cfd60 0x7f72040cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:12.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f7203fff640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f720415cd90 0x7f720414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:12.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.741+0000 7f72037fe640 1 -- 172.21.15.115:0/2598869095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040cfd60 msgr2=0x7f72040cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:12.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f72037fe640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040cfd60 0x7f72040cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f72037fe640 1 -- 172.21.15.115:0/2598869095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f720415cd90 msgr2=0x7f720414fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:12.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f72037fe640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f720415cd90 0x7f720414fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f72037fe640 1 -- 172.21.15.115:0/2598869095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc009b30 con 0x7f72040b5fa0 2023-12-01T01:54:12.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f7203fff640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f720415cd90 0x7f720414fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:12.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f72037fe640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040b5fa0 0x7f720415d1c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f71f400d800 tx=0x7f71f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:12.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f7200ff9640 1 -- 172.21.15.115:0/2598869095 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f400b660 con 0x7f72040b5fa0 2023-12-01T01:54:12.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f7200ff9640 1 -- 172.21.15.115:0/2598869095 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f400b7c0 con 0x7f72040b5fa0 2023-12-01T01:54:12.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f7200ff9640 1 -- 172.21.15.115:0/2598869095 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f400bb50 con 0x7f72040b5fa0 2023-12-01T01:54:12.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f720a096640 1 -- 172.21.15.115:0/2598869095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720415db30 con 0x7f72040b5fa0 2023-12-01T01:54:12.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.742+0000 7f720a096640 1 -- 172.21.15.115:0/2598869095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720415e0f0 con 0x7f72040b5fa0 2023-12-01T01:54:12.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.743+0000 7f720a096640 1 -- 172.21.15.115:0/2598869095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720404e890 con 0x7f72040b5fa0 2023-12-01T01:54:12.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.745+0000 7f7200ff9640 1 -- 172.21.15.115:0/2598869095 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 57) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f71f400bcb0 con 0x7f72040b5fa0 2023-12-01T01:54:12.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.746+0000 7f7200ff9640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f71e406a6a0 0x7f71e406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:12.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.746+0000 7f7200ff9640 1 -- 172.21.15.115:0/2598869095 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f71f4089390 con 0x7f72040b5fa0 2023-12-01T01:54:12.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.746+0000 7f71fbfff640 1 -- 172.21.15.115:0/2598869095 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f71e406a6a0 msgr2=0x7f71e406cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:12.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.746+0000 7f71fbfff640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f71e406a6a0 0x7f71e406cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:12.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.750+0000 7f7200ff9640 1 -- 172.21.15.115:0/2598869095 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f71f40544b0 con 0x7f72040b5fa0 2023-12-01T01:54:12.767 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:12.767+0000 7fc14303c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:54:12.786 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:12.786+0000 7f5564e50040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:54:12.821 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:12.821+0000 7fc14303c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:54:12.838 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:12.837+0000 7f5564e50040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:54:12.873 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:12.873+0000 7fc14303c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:54:12.888 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:12.887+0000 7f5564e50040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:54:12.936 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:12.936+0000 7fc14303c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:54:12.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.941+0000 7f720a096640 1 -- 172.21.15.115:0/2598869095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72040d0a70 con 0x7f72040b5fa0 2023-12-01T01:54:12.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.942+0000 7f7200ff9640 1 -- 172.21.15.115:0/2598869095 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71f4057ad0 con 0x7f72040b5fa0 2023-12-01T01:54:12.942 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:54:12.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.944+0000 7f720a096640 1 -- 172.21.15.115:0/2598869095 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f71e406a6a0 msgr2=0x7f71e406cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:12.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.944+0000 7f720a096640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f71e406a6a0 0x7f71e406cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.944+0000 7f720a096640 1 -- 172.21.15.115:0/2598869095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040b5fa0 msgr2=0x7f720415d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:12.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.944+0000 7f720a096640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040b5fa0 0x7f720415d1c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f71f400d800 tx=0x7f71f400dcc0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.944+0000 7f720a096640 1 -- 172.21.15.115:0/2598869095 shutdown_connections 2023-12-01T01:54:12.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.944+0000 7f720a096640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f71e406a6a0 0x7f71e406cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.944+0000 7f720a096640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040cfd60 0x7f72040cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.944+0000 7f720a096640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f720415cd90 0x7f720414fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.944+0000 7f720a096640 1 --2- 172.21.15.115:0/2598869095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040b5fa0 0x7f720415d1c0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:12.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.945+0000 7f720a096640 1 -- 172.21.15.115:0/2598869095 >> 172.21.15.115:0/2598869095 conn(0x7f7204065960 msgr2=0x7f720415a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:12.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.945+0000 7f720a096640 1 -- 172.21.15.115:0/2598869095 shutdown_connections 2023-12-01T01:54:12.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:12.945+0000 7f720a096640 1 -- 172.21.15.115:0/2598869095 wait complete. 2023-12-01T01:54:12.947 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:12.947+0000 7f5564e50040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:54:12.962 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:54:12.989 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:12.989+0000 7fc14303c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:54:12.996 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:12.996+0000 7f5564e50040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:54:13.106 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:13.105+0000 7fc14303c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:54:13.107 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:13.107+0000 7f5564e50040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:54:13.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.164+0000 7ff2abfff640 1 Processor -- start 2023-12-01T01:54:13.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.164+0000 7ff2abfff640 1 -- start start 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.165+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2ac071e30 0x7ff2ac15bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.166+0000 7ff2abfff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac06fba0 con 0x7ff2ac15acb0 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.166+0000 7ff2abfff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac06f5e0 con 0x7ff2ac0cc620 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.166+0000 7ff2abfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac06f8c0 con 0x7ff2ac071e30 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.166+0000 7ff2aaffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff2ac15acb0 0x7ff2ac06ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44502/0 (socket says 172.21.15.115:44502) 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.166+0000 7ff2aaffd640 1 -- 172.21.15.115:0/1651449629 learned_addr learned my addr 172.21.15.115:0/1651449629 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.166+0000 7ff2a27fc640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2ac071e30 0x7ff2ac15bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.166+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392747301 0 0) 0x7ff2ac06fba0 con 0x7ff2ac15acb0 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.166+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff298003520 con 0x7ff2ac15acb0 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2898678748 0 0) 0x7ff298003520 con 0x7ff2ac15acb0 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2ac071e30 msgr2=0x7ff2ac15bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:13.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2ac071e30 0x7ff2ac15bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 >> v1:172.21.15.142:6789/0 conn(0x7ff2ac0cc620 legacy=0x7ff2ac0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac15b500 con 0x7ff2ac15acb0 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7ff29c001670 con 0x7ff2ac15acb0 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 >> v1:172.21.15.115:6789/0 conn(0x7ff2ac15acb0 legacy=0x7ff2ac06ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980039a0 0x7ff298003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2980047c0 0x7ff298004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2ac071e30 0x7ff298005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac06f8c0 con 0x7ff2980039a0 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29800bb40 con 0x7ff2980047c0 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29800bd00 con 0x7ff2ac071e30 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.167+0000 7ff2aaffd640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980039a0 0x7ff298003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:13.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.168+0000 7ff2a27fc640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2980047c0 0x7ff298004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.168+0000 7ff2aaffd640 1 -- 172.21.15.115:0/1651449629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2ac071e30 msgr2=0x7ff298005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.168+0000 7ff2aaffd640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2ac071e30 0x7ff298005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.168+0000 7ff2aaffd640 1 -- 172.21.15.115:0/1651449629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2980047c0 msgr2=0x7ff298004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.168+0000 7ff2aaffd640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2980047c0 0x7ff298004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.168+0000 7ff2aaffd640 1 -- 172.21.15.115:0/1651449629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298003520 con 0x7ff2980039a0 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.169+0000 7ff2aaffd640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980039a0 0x7ff298003dc0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00c0e0 tx=0x7ff2980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63bdf20b07fdf31 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.169+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c005ad0 con 0x7ff2980039a0 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.169+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c005c50 con 0x7ff2980039a0 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.169+0000 7ff2aa7fc640 1 -- 172.21.15.115:0/1651449629 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c02c7d0 con 0x7ff2980039a0 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.169+0000 7ff2abfff640 1 -- 172.21.15.115:0/1651449629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980039a0 msgr2=0x7ff298003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.169+0000 7ff2abfff640 1 --2- 172.21.15.115:0/1651449629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980039a0 0x7ff298003dc0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00c0e0 tx=0x7ff2980057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.169+0000 7ff2ab7fe640 1 -- 172.21.15.115:0/1651449629 reap_dead start 2023-12-01T01:54:13.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.170+0000 7ff2abfff640 1 -- 172.21.15.115:0/1651449629 shutdown_connections 2023-12-01T01:54:13.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.170+0000 7ff2abfff640 1 -- 172.21.15.115:0/1651449629 >> 172.21.15.115:0/1651449629 conn(0x7ff2ac0cd420 msgr2=0x7ff2ac0cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:13.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.170+0000 7ff2abfff640 1 -- 172.21.15.115:0/1651449629 shutdown_connections 2023-12-01T01:54:13.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.170+0000 7ff2abfff640 1 -- 172.21.15.115:0/1651449629 wait complete. 2023-12-01T01:54:13.174 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:13.169+0000 7f5564e50040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:54:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.171+0000 7ff2abfff640 1 Processor -- start 2023-12-01T01:54:13.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.172+0000 7ff2abfff640 1 -- start start 2023-12-01T01:54:13.175 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:13.172+0000 7fc14303c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:54:13.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.172+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2980039a0 0x7ff2ac0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.172+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac0caa60 0x7ff2ac157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.172+0000 7ff2abfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2ac1582c0 0x7ff2ac1f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2abfff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac1f2910 con 0x7ff2ac0caa60 2023-12-01T01:54:13.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2abfff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac156ce0 con 0x7ff2ac1582c0 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2abfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac156fc0 con 0x7ff2980039a0 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2a27fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac0caa60 0x7ff2ac157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2a27fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac0caa60 0x7ff2ac157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43762/0 (socket says 172.21.15.115:43762) 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2a27fc640 1 -- 172.21.15.115:0/1021732264 learned_addr learned my addr 172.21.15.115:0/1021732264 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2aaffd640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2980039a0 0x7ff2ac0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2ab7fe640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2ac1582c0 0x7ff2ac1f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2a27fc640 1 -- 172.21.15.115:0/1021732264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2980039a0 msgr2=0x7ff2ac0ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2a27fc640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2980039a0 0x7ff2ac0ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2a27fc640 1 -- 172.21.15.115:0/1021732264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2ac1582c0 msgr2=0x7ff2ac1f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2a27fc640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2ac1582c0 0x7ff2ac1f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2a27fc640 1 -- 172.21.15.115:0/1021732264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac1572a0 con 0x7ff2ac0caa60 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.173+0000 7ff2ab7fe640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2ac1582c0 0x7ff2ac1f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.174+0000 7ff2a27fc640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac0caa60 0x7ff2ac157d80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ff294010890 tx=0x7ff294010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.174+0000 7ff2a8ff9640 1 -- 172.21.15.115:0/1021732264 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29400eb60 con 0x7ff2ac0caa60 2023-12-01T01:54:13.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.174+0000 7ff2a8ff9640 1 -- 172.21.15.115:0/1021732264 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29400ed00 con 0x7ff2ac0caa60 2023-12-01T01:54:13.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.174+0000 7ff2abfff640 1 -- 172.21.15.115:0/1021732264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac157580 con 0x7ff2ac0caa60 2023-12-01T01:54:13.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.174+0000 7ff2a8ff9640 1 -- 172.21.15.115:0/1021732264 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29401c640 con 0x7ff2ac0caa60 2023-12-01T01:54:13.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.174+0000 7ff2abfff640 1 -- 172.21.15.115:0/1021732264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac156990 con 0x7ff2ac0caa60 2023-12-01T01:54:13.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.175+0000 7ff2abfff640 1 -- 172.21.15.115:0/1021732264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac0618d0 con 0x7ff2ac0caa60 2023-12-01T01:54:13.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.176+0000 7ff2a8ff9640 1 -- 172.21.15.115:0/1021732264 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 57) v1 ==== 88823+0+0 (secure 0 0 0) 0x7ff2940143f0 con 0x7ff2ac0caa60 2023-12-01T01:54:13.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.177+0000 7ff2a8ff9640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff28c06a6a0 0x7ff28c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.177+0000 7ff2a8ff9640 1 -- 172.21.15.115:0/1021732264 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7ff294089280 con 0x7ff2ac0caa60 2023-12-01T01:54:13.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.177+0000 7ff2aaffd640 1 -- 172.21.15.115:0/1021732264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff28c06a6a0 msgr2=0x7ff28c06cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:13.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.177+0000 7ff2aaffd640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff28c06a6a0 0x7ff28c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:13.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.181+0000 7ff2a8ff9640 1 -- 172.21.15.115:0/1021732264 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff294054440 con 0x7ff2ac0caa60 2023-12-01T01:54:13.221 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:13.220+0000 7f5564e50040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:54:13.226 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:13.224+0000 7fc14303c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:54:13.277 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:13.277+0000 7f5564e50040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:54:13.284 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:13.283+0000 7fc14303c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:54:13.327 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:13.326+0000 7f5564e50040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:54:13.335 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:13.335+0000 7fc14303c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:54:13.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.371+0000 7ff2abfff640 1 -- 172.21.15.115:0/1021732264 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2ac0b29f0 con 0x7ff2ac0caa60 2023-12-01T01:54:13.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.372+0000 7ff2a8ff9640 1 -- 172.21.15.115:0/1021732264 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff294057a60 con 0x7ff2ac0caa60 2023-12-01T01:54:13.375 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 -- 172.21.15.115:0/1021732264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff28c06a6a0 msgr2=0x7ff28c06cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff28c06a6a0 0x7ff28c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 -- 172.21.15.115:0/1021732264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac0caa60 msgr2=0x7ff2ac157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac0caa60 0x7ff2ac157d80 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ff294010890 tx=0x7ff294010d50 comp rx=0 tx=0).stop 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 -- 172.21.15.115:0/1021732264 shutdown_connections 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff28c06a6a0 0x7ff28c06cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2ac1582c0 0x7ff2ac1f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac0caa60 0x7ff2ac157d80 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 --2- 172.21.15.115:0/1021732264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2980039a0 0x7ff2ac0ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 -- 172.21.15.115:0/1021732264 >> 172.21.15.115:0/1021732264 conn(0x7ff2ac0cd420 msgr2=0x7ff2ac0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 -- 172.21.15.115:0/1021732264 shutdown_connections 2023-12-01T01:54:13.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.375+0000 7ff2abfff640 1 -- 172.21.15.115:0/1021732264 wait complete. 2023-12-01T01:54:13.393 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:54:13.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.643+0000 7f9575dea640 1 Processor -- start 2023-12-01T01:54:13.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.643+0000 7f9575dea640 1 -- start start 2023-12-01T01:54:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.644+0000 7f9575dea640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f957015cf70 0x7f95700b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f9575dea640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f95700d0260 0x7f95700b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f9575dea640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95700b7760 con 0x7f95700b18f0 2023-12-01T01:54:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f9575dea640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95700b2170 con 0x7f957015cf70 2023-12-01T01:54:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f9575dea640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95700b8b70 con 0x7f95700d0260 2023-12-01T01:54:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f956f7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f957015cf70 0x7f95700b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f956f7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f957015cf70 0x7f95700b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43778/0 (socket says 172.21.15.115:43778) 2023-12-01T01:54:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f956f7fe640 1 -- 172.21.15.115:0/719773238 learned_addr learned my addr 172.21.15.115:0/719773238 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f956f7fe640 1 -- 172.21.15.115:0/719773238 >> v2:172.21.15.142:3300/0 conn(0x7f95700d0260 msgr2=0x7f95700b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:13.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f956ffff640 1 --2- 172.21.15.115:0/719773238 >> v2:172.21.15.142:3300/0 conn(0x7f95700d0260 0x7f95700b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:13.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f956f7fe640 1 --2- 172.21.15.115:0/719773238 >> v2:172.21.15.142:3300/0 conn(0x7f95700d0260 0x7f95700b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f956f7fe640 1 -- 172.21.15.115:0/719773238 >> v1:172.21.15.142:6789/0 conn(0x7f95700b18f0 legacy=0x7f95700cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:13.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.645+0000 7f956f7fe640 1 -- 172.21.15.115:0/719773238 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95700b6e30 con 0x7f957015cf70 2023-12-01T01:54:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.646+0000 7f956f7fe640 1 --2- 172.21.15.115:0/719773238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f957015cf70 0x7f95700b8470 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f95700d0970 tx=0x7f956002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225b58652ca3e2bc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.647+0000 7f956e7fc640 1 -- 172.21.15.115:0/719773238 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560030860 con 0x7f957015cf70 2023-12-01T01:54:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.647+0000 7f956e7fc640 1 -- 172.21.15.115:0/719773238 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95600309c0 con 0x7f957015cf70 2023-12-01T01:54:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.647+0000 7f956e7fc640 1 -- 172.21.15.115:0/719773238 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560030d50 con 0x7f957015cf70 2023-12-01T01:54:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.647+0000 7f9575dea640 1 -- 172.21.15.115:0/719773238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f957015cf70 msgr2=0x7f95700b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.647+0000 7f9575dea640 1 --2- 172.21.15.115:0/719773238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f957015cf70 0x7f95700b8470 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f95700d0970 tx=0x7f956002b9a0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.648+0000 7f9575dea640 1 -- 172.21.15.115:0/719773238 shutdown_connections 2023-12-01T01:54:13.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.648+0000 7f9575dea640 1 --2- 172.21.15.115:0/719773238 >> v2:172.21.15.142:3300/0 conn(0x7f95700d0260 0x7f95700b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.648+0000 7f9575dea640 1 --2- 172.21.15.115:0/719773238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f957015cf70 0x7f95700b8470 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.648+0000 7f9575dea640 1 -- 172.21.15.115:0/719773238 >> 172.21.15.115:0/719773238 conn(0x7f957006a0e0 msgr2=0x7f957006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:13.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.648+0000 7f9575dea640 1 -- 172.21.15.115:0/719773238 shutdown_connections 2023-12-01T01:54:13.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.648+0000 7f9575dea640 1 -- 172.21.15.115:0/719773238 wait complete. 2023-12-01T01:54:13.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.649+0000 7f9575dea640 1 Processor -- start 2023-12-01T01:54:13.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.650+0000 7f9575dea640 1 -- start start 2023-12-01T01:54:13.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.650+0000 7f9575dea640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95700b18f0 0x7f9570155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f9575dea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95700d0260 0x7f9570149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f9575dea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f957015cf70 0x7f9570153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f9575dea640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95700c9470 con 0x7f95700d0260 2023-12-01T01:54:13.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f9575dea640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9570154490 con 0x7f95700b18f0 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f9575dea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9570154770 con 0x7f957015cf70 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f956effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95700d0260 0x7f9570149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f956effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95700d0260 0x7f9570149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43784/0 (socket says 172.21.15.115:43784) 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f956effd640 1 -- 172.21.15.115:0/4069785802 learned_addr learned my addr 172.21.15.115:0/4069785802 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f956f7fe640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95700b18f0 0x7f9570155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f956effd640 1 -- 172.21.15.115:0/4069785802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f957015cf70 msgr2=0x7f9570153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f956effd640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f957015cf70 0x7f9570153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f956effd640 1 -- 172.21.15.115:0/4069785802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95700b18f0 msgr2=0x7f9570155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f956effd640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95700b18f0 0x7f9570155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f956effd640 1 -- 172.21.15.115:0/4069785802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600094d0 con 0x7f95700d0260 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.651+0000 7f956f7fe640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95700b18f0 0x7f9570155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.652+0000 7f956effd640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95700d0260 0x7f9570149d60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f956400c410 tx=0x7f956400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.652+0000 7f9557fff640 1 -- 172.21.15.115:0/4069785802 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640035f0 con 0x7f95700d0260 2023-12-01T01:54:13.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.652+0000 7f9557fff640 1 -- 172.21.15.115:0/4069785802 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9564003750 con 0x7f95700d0260 2023-12-01T01:54:13.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.652+0000 7f9557fff640 1 -- 172.21.15.115:0/4069785802 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9564003b40 con 0x7f95700d0260 2023-12-01T01:54:13.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.652+0000 7f9575dea640 1 -- 172.21.15.115:0/4069785802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9570154a50 con 0x7f95700d0260 2023-12-01T01:54:13.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.652+0000 7f9575dea640 1 -- 172.21.15.115:0/4069785802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9570155010 con 0x7f95700d0260 2023-12-01T01:54:13.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.653+0000 7f9575dea640 1 -- 172.21.15.115:0/4069785802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957004e890 con 0x7f95700d0260 2023-12-01T01:54:13.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.654+0000 7f9557fff640 1 -- 172.21.15.115:0/4069785802 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 57) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f9564003ca0 con 0x7f95700d0260 2023-12-01T01:54:13.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.655+0000 7f9557fff640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f955c06a670 0x7f955c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:13.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.655+0000 7f9557fff640 1 -- 172.21.15.115:0/4069785802 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f95640881a0 con 0x7f95700d0260 2023-12-01T01:54:13.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.655+0000 7f956f7fe640 1 -- 172.21.15.115:0/4069785802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f955c06a670 msgr2=0x7f955c06cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:13.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.655+0000 7f956f7fe640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f955c06a670 0x7f955c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:13.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.659+0000 7f9557fff640 1 -- 172.21.15.115:0/4069785802 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9564053230 con 0x7f95700d0260 2023-12-01T01:54:13.708 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:13.707+0000 7f5564e50040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:54:13.736 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:13.735+0000 7fc14303c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:54:13.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.848+0000 7f9575dea640 1 -- 172.21.15.115:0/4069785802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f957015b0f0 con 0x7f95700d0260 2023-12-01T01:54:13.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.849+0000 7f9557fff640 1 -- 172.21.15.115:0/4069785802 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9564056850 con 0x7f95700d0260 2023-12-01T01:54:13.849 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:54:13.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.851+0000 7f9575dea640 1 -- 172.21.15.115:0/4069785802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f955c06a670 msgr2=0x7f955c06cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:13.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.851+0000 7f9575dea640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f955c06a670 0x7f955c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.851+0000 7f9575dea640 1 -- 172.21.15.115:0/4069785802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95700d0260 msgr2=0x7f9570149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:13.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.851+0000 7f9575dea640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95700d0260 0x7f9570149d60 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f956400c410 tx=0x7f956400c8d0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.852+0000 7f9575dea640 1 -- 172.21.15.115:0/4069785802 shutdown_connections 2023-12-01T01:54:13.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.852+0000 7f9575dea640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f955c06a670 0x7f955c06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.852+0000 7f9575dea640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f957015cf70 0x7f9570153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.852+0000 7f9575dea640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95700d0260 0x7f9570149d60 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.852+0000 7f9575dea640 1 --2- 172.21.15.115:0/4069785802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95700b18f0 0x7f9570155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:13.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.852+0000 7f9575dea640 1 -- 172.21.15.115:0/4069785802 >> 172.21.15.115:0/4069785802 conn(0x7f957006a0e0 msgr2=0x7f95700cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:13.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.852+0000 7f9575dea640 1 -- 172.21.15.115:0/4069785802 shutdown_connections 2023-12-01T01:54:13.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:13.852+0000 7f9575dea640 1 -- 172.21.15.115:0/4069785802 wait complete. 2023-12-01T01:54:13.866 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:13.866+0000 7f5564e50040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:54:13.870 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:54:13.900 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:13.900+0000 7fc14303c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:54:13.974 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:13.973+0000 7f5564e50040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:54:14.013 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:14.013+0000 7fc14303c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:54:14.023 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:14.023+0000 7f5564e50040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:54:14.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.064+0000 7f7f44c38640 1 Processor -- start 2023-12-01T01:54:14.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.064+0000 7f7f44c38640 1 -- start start 2023-12-01T01:54:14.065 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:14.064+0000 7fc14303c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:54:14.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.066+0000 7f7f44c38640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7f4006f2f0 0x7f7f4006bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.066+0000 7f7f44c38640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f400add10 con 0x7f7f4015e2f0 2023-12-01T01:54:14.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.066+0000 7f7f44c38640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f400c5760 con 0x7f7f400cad70 2023-12-01T01:54:14.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.066+0000 7f7f44c38640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4015e6d0 con 0x7f7f4006f2f0 2023-12-01T01:54:14.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.066+0000 7f7f3ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7f4006f2f0 0x7f7f4006bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:14.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.066+0000 7f7f3ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7f4006f2f0 0x7f7f4006bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43798/0 (socket says 172.21.15.115:43798) 2023-12-01T01:54:14.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.066+0000 7f7f3ed9d640 1 -- 172.21.15.115:0/1714397697 learned_addr learned my addr 172.21.15.115:0/1714397697 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:14.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.066+0000 7f7f3ed9d640 1 -- 172.21.15.115:0/1714397697 >> v1:172.21.15.142:6789/0 conn(0x7f7f400cad70 legacy=0x7f7f4006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:14.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.066+0000 7f7f3ed9d640 1 -- 172.21.15.115:0/1714397697 >> v1:172.21.15.115:6789/0 conn(0x7f7f4015e2f0 legacy=0x7f7f400ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:14.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.066+0000 7f7f3ed9d640 1 -- 172.21.15.115:0/1714397697 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f400cf7b0 con 0x7f7f4006f2f0 2023-12-01T01:54:14.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.067+0000 7f7f3ed9d640 1 --2- 172.21.15.115:0/1714397697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4006f2f0 0x7f7f4006bcd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009e80 tx=0x7f7f3802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd73032e9dfcf04 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:14.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.068+0000 7f7f3dd9b640 1 -- 172.21.15.115:0/1714397697 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3802c9e0 con 0x7f7f4006f2f0 2023-12-01T01:54:14.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.068+0000 7f7f3dd9b640 1 -- 172.21.15.115:0/1714397697 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3802cb40 con 0x7f7f4006f2f0 2023-12-01T01:54:14.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.068+0000 7f7f3dd9b640 1 -- 172.21.15.115:0/1714397697 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f380343f0 con 0x7f7f4006f2f0 2023-12-01T01:54:14.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.068+0000 7f7f44c38640 1 -- 172.21.15.115:0/1714397697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4006f2f0 msgr2=0x7f7f4006bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:14.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.068+0000 7f7f44c38640 1 --2- 172.21.15.115:0/1714397697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4006f2f0 0x7f7f4006bcd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009e80 tx=0x7f7f3802be40 comp rx=0 tx=0).stop 2023-12-01T01:54:14.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.068+0000 7f7f44c38640 1 -- 172.21.15.115:0/1714397697 shutdown_connections 2023-12-01T01:54:14.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.068+0000 7f7f44c38640 1 --2- 172.21.15.115:0/1714397697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4006f2f0 0x7f7f4006bcd0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.068+0000 7f7f44c38640 1 -- 172.21.15.115:0/1714397697 >> 172.21.15.115:0/1714397697 conn(0x7f7f400ce880 msgr2=0x7f7f400cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:14.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.069+0000 7f7f44c38640 1 -- 172.21.15.115:0/1714397697 shutdown_connections 2023-12-01T01:54:14.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.069+0000 7f7f44c38640 1 -- 172.21.15.115:0/1714397697 wait complete. 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.070+0000 7f7f44c38640 1 Processor -- start 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.070+0000 7f7f44c38640 1 -- start start 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.071+0000 7f7f44c38640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4006f2f0 0x7f7f4014dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.071+0000 7f7f44c38640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f400cad70 0x7f7f400c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f44c38640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4015e2f0 0x7f7f400c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f44c38640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4014e2e0 con 0x7f7f4015e2f0 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f44c38640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f400c2e00 con 0x7f7f4006f2f0 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f44c38640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f400c30e0 con 0x7f7f400cad70 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f3ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4015e2f0 0x7f7f400c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f3ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4015e2f0 0x7f7f400c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43814/0 (socket says 172.21.15.115:43814) 2023-12-01T01:54:14.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f3ed9d640 1 -- 172.21.15.115:0/1457016157 learned_addr learned my addr 172.21.15.115:0/1457016157 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f3e59c640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4006f2f0 0x7f7f4014dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f37fff640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f400cad70 0x7f7f400c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f3ed9d640 1 -- 172.21.15.115:0/1457016157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f400cad70 msgr2=0x7f7f400c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f3ed9d640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f400cad70 0x7f7f400c3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f3ed9d640 1 -- 172.21.15.115:0/1457016157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4006f2f0 msgr2=0x7f7f4014dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f3ed9d640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4006f2f0 0x7f7f4014dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f3ed9d640 1 -- 172.21.15.115:0/1457016157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38009b30 con 0x7f7f4015e2f0 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.072+0000 7f7f3e59c640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4006f2f0 0x7f7f4014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.073+0000 7f7f3ed9d640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4015e2f0 0x7f7f400c23b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009fa0 tx=0x7f7f38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.073+0000 7f7f36ffd640 1 -- 172.21.15.115:0/1457016157 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38009510 con 0x7f7f4015e2f0 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.073+0000 7f7f36ffd640 1 -- 172.21.15.115:0/1457016157 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3803e450 con 0x7f7f4015e2f0 2023-12-01T01:54:14.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.073+0000 7f7f36ffd640 1 -- 172.21.15.115:0/1457016157 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3803e7e0 con 0x7f7f4015e2f0 2023-12-01T01:54:14.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.073+0000 7f7f44c38640 1 -- 172.21.15.115:0/1457016157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f400c33c0 con 0x7f7f4015e2f0 2023-12-01T01:54:14.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.073+0000 7f7f44c38640 1 -- 172.21.15.115:0/1457016157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f400c3980 con 0x7f7f4015e2f0 2023-12-01T01:54:14.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.074+0000 7f7f44c38640 1 -- 172.21.15.115:0/1457016157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4006fda0 con 0x7f7f4015e2f0 2023-12-01T01:54:14.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.075+0000 7f7f36ffd640 1 -- 172.21.15.115:0/1457016157 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 57) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f7f3803e940 con 0x7f7f4015e2f0 2023-12-01T01:54:14.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.076+0000 7f7f36ffd640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7f2006a670 0x7f7f2006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.076+0000 7f7f36ffd640 1 -- 172.21.15.115:0/1457016157 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f7f380abd50 con 0x7f7f4015e2f0 2023-12-01T01:54:14.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.076+0000 7f7f3e59c640 1 -- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7f2006a670 msgr2=0x7f7f2006cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:14.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.076+0000 7f7f3e59c640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7f2006a670 0x7f7f2006cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:14.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.081+0000 7f7f36ffd640 1 -- 172.21.15.115:0/1457016157 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7f38076e70 con 0x7f7f4015e2f0 2023-12-01T01:54:14.131 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:14.129+0000 7f5564e50040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:54:14.177 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:14.177+0000 7fc14303c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:54:14.237 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:14.235+0000 7f5564e50040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:54:14.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.277+0000 7f7f3e59c640 1 -- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7f2006a670 msgr2=0x7f7f2006cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:14.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.277+0000 7f7f3e59c640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7f2006a670 0x7f7f2006cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:54:14.285 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:14.285+0000 7fc14303c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:54:14.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.285+0000 7f7f44c38640 1 -- 172.21.15.115:0/1457016157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7f400b43b0 con 0x7f7f4015e2f0 2023-12-01T01:54:14.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.286+0000 7f7f36ffd640 1 -- 172.21.15.115:0/1457016157 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v112) v1 ==== 80+0+468 (secure 0 0 0) 0x7f7f400b43b0 con 0x7f7f4015e2f0 2023-12-01T01:54:14.286 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/88317 2023-12-02T01:53:07.800872+0000 2023-12-01T01:54:14.286 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/88317 2023-12-02T01:53:07.800872+0000 2023-12-01T01:54:14.286 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2605263330 2023-12-02T01:53:07.800872+0000 2023-12-01T01:54:14.286 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3479742140 2023-12-02T01:53:07.800872+0000 2023-12-01T01:54:14.287 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1863268774 2023-12-02T01:53:07.800872+0000 2023-12-01T01:54:14.287 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1925101108 2023-12-02T01:53:07.800872+0000 2023-12-01T01:54:14.287 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3510224424 2023-12-02T01:53:07.800872+0000 2023-12-01T01:54:14.287 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3799042057 2023-12-02T01:53:07.800872+0000 2023-12-01T01:54:14.287 INFO:teuthology.orchestra.run.smithi115.stderr:listed 8 entries 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.288+0000 7f7f44c38640 1 -- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7f2006a670 msgr2=0x7f7f2006cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.288+0000 7f7f44c38640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7f2006a670 0x7f7f2006cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.288+0000 7f7f44c38640 1 -- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4015e2f0 msgr2=0x7f7f400c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.288+0000 7f7f44c38640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4015e2f0 0x7f7f400c23b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009fa0 tx=0x7f7f38004a40 comp rx=0 tx=0).stop 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.288+0000 7f7f44c38640 1 -- 172.21.15.115:0/1457016157 shutdown_connections 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.288+0000 7f7f44c38640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7f2006a670 0x7f7f2006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.288+0000 7f7f44c38640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f400cad70 0x7f7f400c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.288+0000 7f7f44c38640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f4015e2f0 0x7f7f400c23b0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.288+0000 7f7f44c38640 1 --2- 172.21.15.115:0/1457016157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4006f2f0 0x7f7f4014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.288+0000 7f7f44c38640 1 -- 172.21.15.115:0/1457016157 >> 172.21.15.115:0/1457016157 conn(0x7f7f400ce880 msgr2=0x7f7f4006c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.289+0000 7f7f44c38640 1 -- 172.21.15.115:0/1457016157 shutdown_connections 2023-12-01T01:54:14.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.289+0000 7f7f44c38640 1 -- 172.21.15.115:0/1457016157 wait complete. 2023-12-01T01:54:14.308 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T01:54:14.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:14.308 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T01:54:14.392 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:14.390+0000 7f5564e50040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:54:14.441 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:14.439+0000 7f5564e50040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:54:14.449 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:54:14.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:14.449 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T01:54:14.454 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:14.451+0000 7fc14303c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:54:14.491 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:54:14.491 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:54:14.505 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:14.504+0000 7fc14303c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:54:14.544 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:54:14.544 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:54:14.545 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:54:14.566 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:14.564+0000 7f5564e50040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:54:14.596 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:54:14.597 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:54:14.597 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:54:14.628 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:14.628+0000 7fc14303c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:54:14.650 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:54:14.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:54:14.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.856+0000 7ffa51c0e640 1 Processor -- start 2023-12-01T01:54:14.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.857+0000 7ffa51c0e640 1 -- start start 2023-12-01T01:54:14.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.858+0000 7ffa51c0e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa4c160130 0x7ffa4c0d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.858+0000 7ffa51c0e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffa4c0d26e0 0x7ffa4c0b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.858+0000 7ffa51c0e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4c0b8b60 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.858+0000 7ffa51c0e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4c160510 con 0x7ffa4c0d26e0 2023-12-01T01:54:14.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.858+0000 7ffa51c0e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0b6d20 con 0x7ffa4c160130 2023-12-01T01:54:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.858+0000 7ffa4bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffa4c0d26e0 0x7ffa4c0b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.858+0000 7ffa4bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffa4c0d26e0 0x7ffa4c0b8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43830/0 (socket says 172.21.15.115:43830) 2023-12-01T01:54:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.859+0000 7ffa4bfff640 1 -- 172.21.15.115:0/4013357104 learned_addr learned my addr 172.21.15.115:0/4013357104 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.859+0000 7ffa50c0c640 1 --2- 172.21.15.115:0/4013357104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa4c160130 0x7ffa4c0d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.859+0000 7ffa4bfff640 1 -- 172.21.15.115:0/4013357104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa4c160130 msgr2=0x7ffa4c0d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.859+0000 7ffa4bfff640 1 --2- 172.21.15.115:0/4013357104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa4c160130 0x7ffa4c0d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.859+0000 7ffa4bfff640 1 -- 172.21.15.115:0/4013357104 >> v1:172.21.15.115:6789/0 conn(0x7ffa4c0cfb20 legacy=0x7ffa4c0d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.859+0000 7ffa4bfff640 1 -- 172.21.15.115:0/4013357104 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c0d1640 con 0x7ffa4c0d26e0 2023-12-01T01:54:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.861+0000 7ffa4bfff640 1 --2- 172.21.15.115:0/4013357104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0d26e0 0x7ffa4c0b8620 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009870 tx=0x7ffa3802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687ba21677b0116c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:14.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.861+0000 7ffa4b7fe640 1 -- 172.21.15.115:0/4013357104 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38030840 con 0x7ffa4c0d26e0 2023-12-01T01:54:14.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.861+0000 7ffa4b7fe640 1 -- 172.21.15.115:0/4013357104 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa380309a0 con 0x7ffa4c0d26e0 2023-12-01T01:54:14.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.861+0000 7ffa4b7fe640 1 -- 172.21.15.115:0/4013357104 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38030d30 con 0x7ffa4c0d26e0 2023-12-01T01:54:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.862+0000 7ffa51c0e640 1 -- 172.21.15.115:0/4013357104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0d26e0 msgr2=0x7ffa4c0b8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.862+0000 7ffa51c0e640 1 --2- 172.21.15.115:0/4013357104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0d26e0 0x7ffa4c0b8620 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009870 tx=0x7ffa3802b980 comp rx=0 tx=0).stop 2023-12-01T01:54:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.862+0000 7ffa51c0e640 1 -- 172.21.15.115:0/4013357104 shutdown_connections 2023-12-01T01:54:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.862+0000 7ffa51c0e640 1 --2- 172.21.15.115:0/4013357104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0d26e0 0x7ffa4c0b8620 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.862+0000 7ffa51c0e640 1 --2- 172.21.15.115:0/4013357104 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa4c160130 0x7ffa4c0d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.862+0000 7ffa51c0e640 1 -- 172.21.15.115:0/4013357104 >> 172.21.15.115:0/4013357104 conn(0x7ffa4c06a0b0 msgr2=0x7ffa4c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.862+0000 7ffa51c0e640 1 -- 172.21.15.115:0/4013357104 shutdown_connections 2023-12-01T01:54:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.862+0000 7ffa51c0e640 1 -- 172.21.15.115:0/4013357104 wait complete. 2023-12-01T01:54:14.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.864+0000 7ffa51c0e640 1 Processor -- start 2023-12-01T01:54:14.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.864+0000 7ffa51c0e640 1 -- start start 2023-12-01T01:54:14.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa51c0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0cfb20 0x7ffa4c0c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa51c0e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c0d26e0 0x7ffa4c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa51c0e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa4c160130 0x7ffa4c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa51c0e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c150990 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa51c0e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0cb510 con 0x7ffa4c0d26e0 2023-12-01T01:54:14.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa51c0e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0cb7f0 con 0x7ffa4c160130 2023-12-01T01:54:14.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa50c0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0cfb20 0x7ffa4c0c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:14.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa50c0c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0cfb20 0x7ffa4c0c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43842/0 (socket says 172.21.15.115:43842) 2023-12-01T01:54:14.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa50c0c640 1 -- 172.21.15.115:0/2547225502 learned_addr learned my addr 172.21.15.115:0/2547225502 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:14.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa5140d640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa4c160130 0x7ffa4c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa4bfff640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c0d26e0 0x7ffa4c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa50c0c640 1 -- 172.21.15.115:0/2547225502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa4c160130 msgr2=0x7ffa4c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.865+0000 7ffa50c0c640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa4c160130 0x7ffa4c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.866+0000 7ffa50c0c640 1 -- 172.21.15.115:0/2547225502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c0d26e0 msgr2=0x7ffa4c0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.866+0000 7ffa50c0c640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c0d26e0 0x7ffa4c0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.866+0000 7ffa50c0c640 1 -- 172.21.15.115:0/2547225502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa380094d0 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.866+0000 7ffa50c0c640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0cfb20 0x7ffa4c0c8bb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000e870 tx=0x7ffa4000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.866+0000 7ffa497fa640 1 -- 172.21.15.115:0/2547225502 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa400095c0 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.866+0000 7ffa497fa640 1 -- 172.21.15.115:0/2547225502 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4000c940 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.866+0000 7ffa51c0e640 1 -- 172.21.15.115:0/2547225502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c0cbad0 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.866+0000 7ffa497fa640 1 -- 172.21.15.115:0/2547225502 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4000ccb0 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.866+0000 7ffa51c0e640 1 -- 172.21.15.115:0/2547225502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c0cc090 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.867+0000 7ffa51c0e640 1 -- 172.21.15.115:0/2547225502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c0673a0 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.868+0000 7ffa497fa640 1 -- 172.21.15.115:0/2547225502 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 57) v1 ==== 88823+0+0 (secure 0 0 0) 0x7ffa40013470 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.869+0000 7ffa497fa640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa3006a6a0 0x7ffa3006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.869+0000 7ffa497fa640 1 -- 172.21.15.115:0/2547225502 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7ffa40089440 con 0x7ffa4c0cfb20 2023-12-01T01:54:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.869+0000 7ffa4bfff640 1 -- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa3006a6a0 msgr2=0x7ffa3006cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:14.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.869+0000 7ffa4bfff640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa3006a6a0 0x7ffa3006cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:14.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:14.873+0000 7ffa497fa640 1 -- 172.21.15.115:0/2547225502 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ffa40054590 con 0x7ffa4c0cfb20 2023-12-01T01:54:15.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.070+0000 7ffa4bfff640 1 -- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa3006a6a0 msgr2=0x7ffa3006cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:15.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.070+0000 7ffa4bfff640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa3006a6a0 0x7ffa3006cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:54:15.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.098+0000 7ffa51c0e640 1 -- 172.21.15.115:0/2547225502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa4c0d21b0 con 0x7ffa4c0cfb20 2023-12-01T01:54:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.099+0000 7ffa497fa640 1 -- 172.21.15.115:0/2547225502 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4662 (secure 0 0 0) 0x7ffa40057bb0 con 0x7ffa4c0cfb20 2023-12-01T01:54:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:15.099 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3538792232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3538792232},{"type":"v1","addr":"172.21.15.142:6835","nonce":3538792232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7449,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3171906828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3171906828},{"type":"v1","addr":"172.21.15.115:6835","nonce":3171906828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7452,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2130806226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2130806226},{"type":"v1","addr":"172.21.15.115:6837","nonce":2130806226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:48.491310+0000","modified":"2023-12-01T01:52:51.513496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7455":{"gid":7455,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/3108729597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3108729597},{"type":"v1","addr":"172.21.15.142:6834","nonce":3108729597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-01T01:54:15.099 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 48 2023-12-01T01:54:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 -- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa3006a6a0 msgr2=0x7ffa3006cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa3006a6a0 0x7ffa3006cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 -- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0cfb20 msgr2=0x7ffa4c0c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0cfb20 0x7ffa4c0c8bb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000e870 tx=0x7ffa4000ed30 comp rx=0 tx=0).stop 2023-12-01T01:54:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 -- 172.21.15.115:0/2547225502 shutdown_connections 2023-12-01T01:54:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffa3006a6a0 0x7ffa3006cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa4c160130 0x7ffa4c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa4c0d26e0 0x7ffa4c0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 --2- 172.21.15.115:0/2547225502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa4c0cfb20 0x7ffa4c0c8bb0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 -- 172.21.15.115:0/2547225502 >> 172.21.15.115:0/2547225502 conn(0x7ffa4c06a0b0 msgr2=0x7ffa4c15a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 -- 172.21.15.115:0/2547225502 shutdown_connections 2023-12-01T01:54:15.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.101+0000 7ffa51c0e640 1 -- 172.21.15.115:0/2547225502 wait complete. 2023-12-01T01:54:15.124 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:54:15.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.315+0000 7fba69769640 1 Processor -- start 2023-12-01T01:54:15.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.315+0000 7fba69769640 1 -- start start 2023-12-01T01:54:15.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.316+0000 7fba69769640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fba6415cd20 0x7fba640d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba69769640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba640b5d80 0x7fba640b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba69769640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba640ce2e0 con 0x7fba640d1230 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba69769640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba640b2160 con 0x7fba6415cd20 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba69769640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640d2a10 con 0x7fba640b5d80 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba62ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fba6415cd20 0x7fba640d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba62ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fba6415cd20 0x7fba640d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43858/0 (socket says 172.21.15.115:43858) 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba62ffd640 1 -- 172.21.15.115:0/1939237994 learned_addr learned my addr 172.21.15.115:0/1939237994 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba627fc640 1 --2- 172.21.15.115:0/1939237994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba640b5d80 0x7fba640b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba62ffd640 1 -- 172.21.15.115:0/1939237994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba640b5d80 msgr2=0x7fba640b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba62ffd640 1 --2- 172.21.15.115:0/1939237994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba640b5d80 0x7fba640b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba62ffd640 1 -- 172.21.15.115:0/1939237994 >> v1:172.21.15.115:6789/0 conn(0x7fba640d1230 legacy=0x7fba640d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.317+0000 7fba62ffd640 1 -- 172.21.15.115:0/1939237994 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba640ce000 con 0x7fba6415cd20 2023-12-01T01:54:15.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.319+0000 7fba62ffd640 1 --2- 172.21.15.115:0/1939237994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba6415cd20 0x7fba640d24d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fba540098a0 tx=0x7fba5402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4110bbee5f026622 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:15.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.319+0000 7fba61ffb640 1 -- 172.21.15.115:0/1939237994 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54031640 con 0x7fba6415cd20 2023-12-01T01:54:15.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.319+0000 7fba61ffb640 1 -- 172.21.15.115:0/1939237994 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba540317a0 con 0x7fba6415cd20 2023-12-01T01:54:15.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.319+0000 7fba61ffb640 1 -- 172.21.15.115:0/1939237994 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54031b30 con 0x7fba6415cd20 2023-12-01T01:54:15.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.319+0000 7fba69769640 1 -- 172.21.15.115:0/1939237994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba6415cd20 msgr2=0x7fba640d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.319+0000 7fba69769640 1 --2- 172.21.15.115:0/1939237994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba6415cd20 0x7fba640d24d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fba540098a0 tx=0x7fba5402b8f0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.320+0000 7fba69769640 1 -- 172.21.15.115:0/1939237994 shutdown_connections 2023-12-01T01:54:15.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.320+0000 7fba69769640 1 --2- 172.21.15.115:0/1939237994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba640b5d80 0x7fba640b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.320+0000 7fba69769640 1 --2- 172.21.15.115:0/1939237994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba6415cd20 0x7fba640d24d0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.320+0000 7fba69769640 1 -- 172.21.15.115:0/1939237994 >> 172.21.15.115:0/1939237994 conn(0x7fba6406a0e0 msgr2=0x7fba6406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:15.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.320+0000 7fba69769640 1 -- 172.21.15.115:0/1939237994 shutdown_connections 2023-12-01T01:54:15.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.320+0000 7fba69769640 1 -- 172.21.15.115:0/1939237994 wait complete. 2023-12-01T01:54:15.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.321+0000 7fba69769640 1 Processor -- start 2023-12-01T01:54:15.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.321+0000 7fba69769640 1 -- start start 2023-12-01T01:54:15.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.322+0000 7fba69769640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba640b5d80 0x7fba6414f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.322+0000 7fba69769640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba640d1230 0x7fba640c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.322+0000 7fba69769640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba6415cd20 0x7fba641541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.322+0000 7fba69769640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6415ab50 con 0x7fba640d1230 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.322+0000 7fba69769640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6414f6f0 con 0x7fba640b5d80 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.322+0000 7fba69769640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba64159ed0 con 0x7fba6415cd20 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.323+0000 7fba627fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba640d1230 0x7fba640c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.323+0000 7fba627fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba640d1230 0x7fba640c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43866/0 (socket says 172.21.15.115:43866) 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.323+0000 7fba627fc640 1 -- 172.21.15.115:0/4203831034 learned_addr learned my addr 172.21.15.115:0/4203831034 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.323+0000 7fba637fe640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba6415cd20 0x7fba641541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.323+0000 7fba62ffd640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba640b5d80 0x7fba6414f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.323+0000 7fba627fc640 1 -- 172.21.15.115:0/4203831034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba6415cd20 msgr2=0x7fba641541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.323+0000 7fba627fc640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba6415cd20 0x7fba641541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.323+0000 7fba627fc640 1 -- 172.21.15.115:0/4203831034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba640b5d80 msgr2=0x7fba6414f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.323+0000 7fba627fc640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba640b5d80 0x7fba6414f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.323+0000 7fba627fc640 1 -- 172.21.15.115:0/4203831034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540094d0 con 0x7fba640d1230 2023-12-01T01:54:15.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.324+0000 7fba627fc640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba640d1230 0x7fba640c6780 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fba5800d7d0 tx=0x7fba5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:15.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.324+0000 7fba4b7fe640 1 -- 172.21.15.115:0/4203831034 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5800b6c0 con 0x7fba640d1230 2023-12-01T01:54:15.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.324+0000 7fba4b7fe640 1 -- 172.21.15.115:0/4203831034 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5800b840 con 0x7fba640d1230 2023-12-01T01:54:15.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.324+0000 7fba4b7fe640 1 -- 172.21.15.115:0/4203831034 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5800bbf0 con 0x7fba640d1230 2023-12-01T01:54:15.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.324+0000 7fba69769640 1 -- 172.21.15.115:0/4203831034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6415a1b0 con 0x7fba640d1230 2023-12-01T01:54:15.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.324+0000 7fba69769640 1 -- 172.21.15.115:0/4203831034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6415a770 con 0x7fba640d1230 2023-12-01T01:54:15.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.325+0000 7fba69769640 1 -- 172.21.15.115:0/4203831034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba6415ac90 con 0x7fba640d1230 2023-12-01T01:54:15.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.326+0000 7fba4b7fe640 1 -- 172.21.15.115:0/4203831034 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 57) v1 ==== 88823+0+0 (secure 0 0 0) 0x7fba5800bd50 con 0x7fba640d1230 2023-12-01T01:54:15.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.327+0000 7fba4b7fe640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba5006a670 0x7fba5006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.327+0000 7fba4b7fe640 1 -- 172.21.15.115:0/4203831034 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fba58089460 con 0x7fba640d1230 2023-12-01T01:54:15.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.327+0000 7fba62ffd640 1 -- 172.21.15.115:0/4203831034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba5006a670 msgr2=0x7fba5006cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:15.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.327+0000 7fba62ffd640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba5006a670 0x7fba5006cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:15.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.331+0000 7fba4b7fe640 1 -- 172.21.15.115:0/4203831034 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fba6415ac90 con 0x7fba640d1230 2023-12-01T01:54:15.416 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:15.414+0000 7f5564e50040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:54:15.457 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:15.457+0000 7fc14303c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:54:15.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.519+0000 7fba69769640 1 -- 172.21.15.115:0/4203831034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba6415ac90 con 0x7fba640d1230 2023-12-01T01:54:15.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.520+0000 7fba4b7fe640 1 -- 172.21.15.115:0/4203831034 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16662 (secure 0 0 0) 0x7fba6415ac90 con 0x7fba640d1230 2023-12-01T01:54:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:15.521 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":112,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:53:07.800915+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:46.948807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:47.513140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:53:07.800872+0000","172.21.15.115:6822/88317":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/2605263330":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3479742140":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/1863268774":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/1925101108":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3510224424":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3799042057":"2023-12-02T01:53:07.800872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:54:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 -- 172.21.15.115:0/4203831034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba5006a670 msgr2=0x7fba5006cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:15.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba5006a670 0x7fba5006cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 -- 172.21.15.115:0/4203831034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba640d1230 msgr2=0x7fba640c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba640d1230 0x7fba640c6780 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fba5800d7d0 tx=0x7fba5800dc90 comp rx=0 tx=0).stop 2023-12-01T01:54:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 -- 172.21.15.115:0/4203831034 shutdown_connections 2023-12-01T01:54:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba5006a670 0x7fba5006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba6415cd20 0x7fba641541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba640d1230 0x7fba640c6780 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 --2- 172.21.15.115:0/4203831034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba640b5d80 0x7fba6414f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 -- 172.21.15.115:0/4203831034 >> 172.21.15.115:0/4203831034 conn(0x7fba6406a0e0 msgr2=0x7fba641e58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 -- 172.21.15.115:0/4203831034 shutdown_connections 2023-12-01T01:54:15.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.523+0000 7fba69769640 1 -- 172.21.15.115:0/4203831034 wait complete. 2023-12-01T01:54:15.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:52:46.948807+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:52:47.513140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:54:15.542 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:54:15.736 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:15.734+0000 7f5564e50040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:54:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.745+0000 7ffbeb59e640 1 Processor -- start 2023-12-01T01:54:15.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.746+0000 7ffbeb59e640 1 -- start start 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbeb59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffbec15acb0 0x7ffbec06ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbeb59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbec0cc620 0x7ffbec0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbeb59e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbec06fba0 con 0x7ffbec071e30 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbeb59e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbec15c410 con 0x7ffbec15acb0 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbeb59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec06f5e0 con 0x7ffbec0cc620 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbea59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffbec15acb0 0x7ffbec06ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbea59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffbec15acb0 0x7ffbec06ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43870/0 (socket says 172.21.15.115:43870) 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbea59c640 1 -- 172.21.15.115:0/2189833829 learned_addr learned my addr 172.21.15.115:0/2189833829 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbead9d640 1 --2- 172.21.15.115:0/2189833829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbec0cc620 0x7ffbec0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbea59c640 1 -- 172.21.15.115:0/2189833829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbec0cc620 msgr2=0x7ffbec0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbea59c640 1 --2- 172.21.15.115:0/2189833829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbec0cc620 0x7ffbec0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.747+0000 7ffbea59c640 1 -- 172.21.15.115:0/2189833829 >> v1:172.21.15.115:6789/0 conn(0x7ffbec071e30 legacy=0x7ffbec15bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.748+0000 7ffbea59c640 1 -- 172.21.15.115:0/2189833829 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbec06f8c0 con 0x7ffbec15acb0 2023-12-01T01:54:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.749+0000 7ffbea59c640 1 --2- 172.21.15.115:0/2189833829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 0x7ffbec06ed90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009870 tx=0x7ffbdc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c8c264df823417 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.749+0000 7ffbe9d9b640 1 -- 172.21.15.115:0/2189833829 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc031640 con 0x7ffbec15acb0 2023-12-01T01:54:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.749+0000 7ffbe9d9b640 1 -- 172.21.15.115:0/2189833829 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc0317a0 con 0x7ffbec15acb0 2023-12-01T01:54:15.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.749+0000 7ffbe9d9b640 1 -- 172.21.15.115:0/2189833829 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc031b30 con 0x7ffbec15acb0 2023-12-01T01:54:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.749+0000 7ffbeb59e640 1 -- 172.21.15.115:0/2189833829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 msgr2=0x7ffbec06ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.749+0000 7ffbeb59e640 1 --2- 172.21.15.115:0/2189833829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 0x7ffbec06ed90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009870 tx=0x7ffbdc02b8f0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.750+0000 7ffbeb59e640 1 -- 172.21.15.115:0/2189833829 shutdown_connections 2023-12-01T01:54:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.750+0000 7ffbeb59e640 1 --2- 172.21.15.115:0/2189833829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbec0cc620 0x7ffbec0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.750+0000 7ffbeb59e640 1 --2- 172.21.15.115:0/2189833829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 0x7ffbec06ed90 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.750+0000 7ffbeb59e640 1 -- 172.21.15.115:0/2189833829 >> 172.21.15.115:0/2189833829 conn(0x7ffbec0cd420 msgr2=0x7ffbec0cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.750+0000 7ffbeb59e640 1 -- 172.21.15.115:0/2189833829 shutdown_connections 2023-12-01T01:54:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.750+0000 7ffbeb59e640 1 -- 172.21.15.115:0/2189833829 wait complete. 2023-12-01T01:54:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.751+0000 7ffbeb59e640 1 Processor -- start 2023-12-01T01:54:15.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.751+0000 7ffbeb59e640 1 -- start start 2023-12-01T01:54:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.752+0000 7ffbeb59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbec071e30 0x7ffbec0c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.752+0000 7ffbeb59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbec0cc620 0x7ffbec151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.752+0000 7ffbeb59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 0x7ffbec0c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.752+0000 7ffbeb59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec152420 con 0x7ffbec15acb0 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.752+0000 7ffbeb59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0c8990 con 0x7ffbec0cc620 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.752+0000 7ffbeb59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0c9360 con 0x7ffbec071e30 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbead9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 0x7ffbec0c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbead9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 0x7ffbec0c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43884/0 (socket says 172.21.15.115:43884) 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbead9d640 1 -- 172.21.15.115:0/867406567 learned_addr learned my addr 172.21.15.115:0/867406567 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbea59c640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbec071e30 0x7ffbec0c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbe3fff640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbec0cc620 0x7ffbec151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbead9d640 1 -- 172.21.15.115:0/867406567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbec071e30 msgr2=0x7ffbec0c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbead9d640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbec071e30 0x7ffbec0c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbead9d640 1 -- 172.21.15.115:0/867406567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbec0cc620 msgr2=0x7ffbec151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbead9d640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbec0cc620 0x7ffbec151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbead9d640 1 -- 172.21.15.115:0/867406567 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc0094d0 con 0x7ffbec15acb0 2023-12-01T01:54:15.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbead9d640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 0x7ffbec0c8e20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4007e60 tx=0x7ffbe400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.753+0000 7ffbe2ffd640 1 -- 172.21.15.115:0/867406567 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4014420 con 0x7ffbec15acb0 2023-12-01T01:54:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.754+0000 7ffbe2ffd640 1 -- 172.21.15.115:0/867406567 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe40145a0 con 0x7ffbec15acb0 2023-12-01T01:54:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.754+0000 7ffbeb59e640 1 -- 172.21.15.115:0/867406567 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbec0c7d80 con 0x7ffbec15acb0 2023-12-01T01:54:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.754+0000 7ffbe2ffd640 1 -- 172.21.15.115:0/867406567 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4014950 con 0x7ffbec15acb0 2023-12-01T01:54:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.754+0000 7ffbeb59e640 1 -- 172.21.15.115:0/867406567 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbec0c8340 con 0x7ffbec15acb0 2023-12-01T01:54:15.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.755+0000 7ffbeb59e640 1 -- 172.21.15.115:0/867406567 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbec0c8b00 con 0x7ffbec15acb0 2023-12-01T01:54:15.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.756+0000 7ffbe2ffd640 1 -- 172.21.15.115:0/867406567 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 57) v1 ==== 88823+0+0 (secure 0 0 0) 0x7ffbe4014ab0 con 0x7ffbec15acb0 2023-12-01T01:54:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.757+0000 7ffbe2ffd640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffbcc06a670 0x7ffbcc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:15.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.757+0000 7ffbe2ffd640 1 -- 172.21.15.115:0/867406567 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6053+0+0 (secure 0 0 0) 0x7ffbe40897a0 con 0x7ffbec15acb0 2023-12-01T01:54:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.757+0000 7ffbea59c640 1 -- 172.21.15.115:0/867406567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffbcc06a670 msgr2=0x7ffbcc06cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:15.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.758+0000 7ffbea59c640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffbcc06a670 0x7ffbcc06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:15.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.761+0000 7ffbe2ffd640 1 -- 172.21.15.115:0/867406567 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ffbe4054900 con 0x7ffbec15acb0 2023-12-01T01:54:15.772 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:15.772+0000 7fc14303c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:54:15.795 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:15.793+0000 7f5564e50040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:54:15.831 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:15.830+0000 7fc14303c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:54:15.907 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:15.906+0000 7f5564e50040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:54:15.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.940+0000 7ffbeb59e640 1 -- 172.21.15.115:0/867406567 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbec0618d0 con 0x7ffbec15acb0 2023-12-01T01:54:15.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.941+0000 7ffbe2ffd640 1 -- 172.21.15.115:0/867406567 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbe4057f20 con 0x7ffbec15acb0 2023-12-01T01:54:15.941 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:54:15.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 -- 172.21.15.115:0/867406567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffbcc06a670 msgr2=0x7ffbcc06cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:15.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffbcc06a670 0x7ffbcc06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 -- 172.21.15.115:0/867406567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 msgr2=0x7ffbec0c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:15.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 0x7ffbec0c8e20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4007e60 tx=0x7ffbe400e4b0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 -- 172.21.15.115:0/867406567 shutdown_connections 2023-12-01T01:54:15.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ffbcc06a670 0x7ffbcc06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbec15acb0 0x7ffbec0c8e20 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbec0cc620 0x7ffbec151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 --2- 172.21.15.115:0/867406567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbec071e30 0x7ffbec0c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:15.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 -- 172.21.15.115:0/867406567 >> 172.21.15.115:0/867406567 conn(0x7ffbec0cd420 msgr2=0x7ffbec0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:15.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.943+0000 7ffbeb59e640 1 -- 172.21.15.115:0/867406567 shutdown_connections 2023-12-01T01:54:15.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:15.944+0000 7ffbeb59e640 1 -- 172.21.15.115:0/867406567 wait complete. 2023-12-01T01:54:15.947 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:15.946+0000 7fc14303c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:54:15.963 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:54:16.030 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:16.029+0000 7fc0c43cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:16.030 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:16.029+0000 7fc0c43cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:16.030 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:16.029+0000 7fc0c43cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:16.030 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:16.029+0000 7fc0c43cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:16.030 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:16.029+0000 7fc0c43cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:16.031 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:16.031+0000 7fc0c73d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:16.031 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:16.031+0000 7fc0c73d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:16.031 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:16.031+0000 7fc0c73d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:16.031 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:16.031+0000 7fc0c73d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:16.031 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:16.031+0000 7fc0c73d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:16.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.150+0000 7f685695c640 1 Processor -- start 2023-12-01T01:54:16.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.150+0000 7f685695c640 1 -- start start 2023-12-01T01:54:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.151+0000 7f685695c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f68500b5fa0 0x7f68500cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.152+0000 7f685695c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500ce5e0 con 0x7f68500cfd60 2023-12-01T01:54:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.152+0000 7f685695c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500ce300 con 0x7f685015cd90 2023-12-01T01:54:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.152+0000 7f685695c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500d0ec0 con 0x7f68500b5fa0 2023-12-01T01:54:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.152+0000 7f6854ef9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f68500b5fa0 0x7f68500cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.152+0000 7f68477fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f68500cfd60 0x7f68500d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44552/0 (socket says 172.21.15.115:44552) 2023-12-01T01:54:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.152+0000 7f68477fe640 1 -- 172.21.15.115:0/1395727123 learned_addr learned my addr 172.21.15.115:0/1395727123 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.152+0000 7f6854ef9640 1 -- 172.21.15.115:0/1395727123 >> v1:172.21.15.142:6789/0 conn(0x7f685015cd90 legacy=0x7f68500cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.152+0000 7f6854ef9640 1 -- 172.21.15.115:0/1395727123 >> v1:172.21.15.115:6789/0 conn(0x7f68500cfd60 legacy=0x7f68500d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.152+0000 7f6854ef9640 1 -- 172.21.15.115:0/1395727123 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500d10a0 con 0x7f68500b5fa0 2023-12-01T01:54:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.152+0000 7f684f7fe640 1 -- 172.21.15.115:0/1395727123 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568334789 0 0) 0x7f6838001110 con 0x7f68500cfd60 2023-12-01T01:54:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.153+0000 7f6854ef9640 1 --2- 172.21.15.115:0/1395727123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500b5fa0 0x7f68500cdd20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6848009d90 tx=0x7f684802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee6d36f4cecf725 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.154+0000 7f684f7fe640 1 -- 172.21.15.115:0/1395727123 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684802cc60 con 0x7f68500b5fa0 2023-12-01T01:54:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.154+0000 7f684f7fe640 1 -- 172.21.15.115:0/1395727123 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684802cdc0 con 0x7f68500b5fa0 2023-12-01T01:54:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.154+0000 7f684f7fe640 1 -- 172.21.15.115:0/1395727123 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848034660 con 0x7f68500b5fa0 2023-12-01T01:54:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.154+0000 7f685695c640 1 -- 172.21.15.115:0/1395727123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500b5fa0 msgr2=0x7f68500cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.154+0000 7f685695c640 1 --2- 172.21.15.115:0/1395727123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500b5fa0 0x7f68500cdd20 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6848009d90 tx=0x7f684802beb0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.154+0000 7f685695c640 1 -- 172.21.15.115:0/1395727123 shutdown_connections 2023-12-01T01:54:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.154+0000 7f685695c640 1 --2- 172.21.15.115:0/1395727123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500b5fa0 0x7f68500cdd20 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.154+0000 7f685695c640 1 -- 172.21.15.115:0/1395727123 >> 172.21.15.115:0/1395727123 conn(0x7f6850065960 msgr2=0x7f6850065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.154+0000 7f685695c640 1 -- 172.21.15.115:0/1395727123 shutdown_connections 2023-12-01T01:54:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.154+0000 7f685695c640 1 -- 172.21.15.115:0/1395727123 wait complete. 2023-12-01T01:54:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.156+0000 7f685695c640 1 Processor -- start 2023-12-01T01:54:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.156+0000 7f685695c640 1 -- start start 2023-12-01T01:54:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.157+0000 7f685695c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500b5fa0 0x7f68501f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.157+0000 7f685695c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500cfd60 0x7f68501505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.157+0000 7f685695c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f685015cd90 0x7f6850140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.157+0000 7f685695c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685014f7a0 con 0x7f68500cfd60 2023-12-01T01:54:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.157+0000 7f685695c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6850141ab0 con 0x7f68500b5fa0 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.157+0000 7f685695c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6850141d90 con 0x7f685015cd90 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.157+0000 7f68477fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500cfd60 0x7f68501505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.157+0000 7f68477fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500cfd60 0x7f68501505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44000/0 (socket says 172.21.15.115:44000) 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.157+0000 7f68477fe640 1 -- 172.21.15.115:0/341671592 learned_addr learned my addr 172.21.15.115:0/341671592 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f68477fe640 1 -- 172.21.15.115:0/341671592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f685015cd90 msgr2=0x7f6850140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f68477fe640 1 --2- 172.21.15.115:0/341671592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f685015cd90 0x7f6850140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f68477fe640 1 -- 172.21.15.115:0/341671592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500b5fa0 msgr2=0x7f68501f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f684ffff640 1 --2- 172.21.15.115:0/341671592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500b5fa0 0x7f68501f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f68477fe640 1 --2- 172.21.15.115:0/341671592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500b5fa0 0x7f68501f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f68477fe640 1 -- 172.21.15.115:0/341671592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848009a70 con 0x7f68500cfd60 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f684ffff640 1 --2- 172.21.15.115:0/341671592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500b5fa0 0x7f68501f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f68477fe640 1 --2- 172.21.15.115:0/341671592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500cfd60 0x7f68501505e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f683800c780 tx=0x7f683800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f684d7fa640 1 -- 172.21.15.115:0/341671592 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68380072d0 con 0x7f68500cfd60 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f684d7fa640 1 -- 172.21.15.115:0/341671592 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6838007430 con 0x7f68500cfd60 2023-12-01T01:54:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f684d7fa640 1 -- 172.21.15.115:0/341671592 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838001d10 con 0x7f68500cfd60 2023-12-01T01:54:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f685695c640 1 -- 172.21.15.115:0/341671592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68501f5b90 con 0x7f68500cfd60 2023-12-01T01:54:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.158+0000 7f685695c640 1 -- 172.21.15.115:0/341671592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68501f6150 con 0x7f68500cfd60 2023-12-01T01:54:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.159+0000 7f685695c640 1 -- 172.21.15.115:0/341671592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685004e890 con 0x7f68500cfd60 2023-12-01T01:54:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.160+0000 7f684d7fa640 1 -- 172.21.15.115:0/341671592 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 58) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f68380047e0 con 0x7f68500cfd60 2023-12-01T01:54:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.161+0000 7f684d7fa640 1 -- 172.21.15.115:0/341671592 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f6838087810 con 0x7f68500cfd60 2023-12-01T01:54:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.165+0000 7f684d7fa640 1 -- 172.21.15.115:0/341671592 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6838052eb0 con 0x7f68500cfd60 2023-12-01T01:54:16.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.353+0000 7f685695c640 1 -- 172.21.15.115:0/341671592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68500ce5e0 con 0x7f68500cfd60 2023-12-01T01:54:16.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.354+0000 7f684d7fa640 1 -- 172.21.15.115:0/341671592 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68500ce5e0 con 0x7f68500cfd60 2023-12-01T01:54:16.357 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:54:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.357+0000 7f685695c640 1 -- 172.21.15.115:0/341671592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500cfd60 msgr2=0x7f68501505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.357+0000 7f685695c640 1 --2- 172.21.15.115:0/341671592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500cfd60 0x7f68501505e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f683800c780 tx=0x7f683800cc40 comp rx=0 tx=0).stop 2023-12-01T01:54:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.357+0000 7f685695c640 1 -- 172.21.15.115:0/341671592 shutdown_connections 2023-12-01T01:54:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.357+0000 7f685695c640 1 --2- 172.21.15.115:0/341671592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f685015cd90 0x7f6850140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.357+0000 7f685695c640 1 --2- 172.21.15.115:0/341671592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68500cfd60 0x7f68501505e0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.357+0000 7f685695c640 1 --2- 172.21.15.115:0/341671592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500b5fa0 0x7f68501f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.357+0000 7f685695c640 1 -- 172.21.15.115:0/341671592 >> 172.21.15.115:0/341671592 conn(0x7f6850065960 msgr2=0x7f68500b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.357+0000 7f685695c640 1 -- 172.21.15.115:0/341671592 shutdown_connections 2023-12-01T01:54:16.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.357+0000 7f685695c640 1 -- 172.21.15.115:0/341671592 wait complete. 2023-12-01T01:54:16.376 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:54:16.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.590+0000 7fdc0a20b640 1 Processor -- start 2023-12-01T01:54:16.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.590+0000 7fdc0a20b640 1 -- start start 2023-12-01T01:54:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.592+0000 7fdc0a20b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc0406d310 0x7fdc040cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.592+0000 7fdc0a20b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc040b09c0 0x7fdc040b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.592+0000 7fdc0a20b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0415d140 con 0x7fdc0414ff20 2023-12-01T01:54:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.592+0000 7fdc0a20b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc040b13b0 con 0x7fdc040b09c0 2023-12-01T01:54:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.592+0000 7fdc0a20b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0415d420 con 0x7fdc0406d310 2023-12-01T01:54:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.592+0000 7fdc03fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc040b09c0 0x7fdc040b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.593+0000 7fdc03fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc040b09c0 0x7fdc040b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44008/0 (socket says 172.21.15.115:44008) 2023-12-01T01:54:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.593+0000 7fdc03fff640 1 -- 172.21.15.115:0/403400329 learned_addr learned my addr 172.21.15.115:0/403400329 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.593+0000 7fdc037fe640 1 --2- 172.21.15.115:0/403400329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc0406d310 0x7fdc040cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:16.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.593+0000 7fdc03fff640 1 -- 172.21.15.115:0/403400329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc0406d310 msgr2=0x7fdc040cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:16.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.593+0000 7fdc03fff640 1 --2- 172.21.15.115:0/403400329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc0406d310 0x7fdc040cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.593+0000 7fdc03fff640 1 -- 172.21.15.115:0/403400329 >> v1:172.21.15.142:6789/0 conn(0x7fdc0414ff20 legacy=0x7fdc0415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:16.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.593+0000 7fdc03fff640 1 -- 172.21.15.115:0/403400329 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0415d700 con 0x7fdc040b09c0 2023-12-01T01:54:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.594+0000 7fdc03fff640 1 --2- 172.21.15.115:0/403400329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc040b09c0 0x7fdc040b0e70 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009e50 tx=0x7fdbfc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ac7f02d6dbf518 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc02ffd640 1 -- 172.21.15.115:0/403400329 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc02cb00 con 0x7fdc040b09c0 2023-12-01T01:54:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc02ffd640 1 -- 172.21.15.115:0/403400329 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbfc02cc60 con 0x7fdc040b09c0 2023-12-01T01:54:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc02ffd640 1 -- 172.21.15.115:0/403400329 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0343f0 con 0x7fdc040b09c0 2023-12-01T01:54:16.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc0a20b640 1 -- 172.21.15.115:0/403400329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc040b09c0 msgr2=0x7fdc040b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:16.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc0a20b640 1 --2- 172.21.15.115:0/403400329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc040b09c0 0x7fdc040b0e70 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009e50 tx=0x7fdbfc02bf60 comp rx=0 tx=0).stop 2023-12-01T01:54:16.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc0a20b640 1 -- 172.21.15.115:0/403400329 shutdown_connections 2023-12-01T01:54:16.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc0a20b640 1 --2- 172.21.15.115:0/403400329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc040b09c0 0x7fdc040b0e70 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc0a20b640 1 --2- 172.21.15.115:0/403400329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc0406d310 0x7fdc040cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc0a20b640 1 -- 172.21.15.115:0/403400329 >> 172.21.15.115:0/403400329 conn(0x7fdc04065960 msgr2=0x7fdc04065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:16.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc0a20b640 1 -- 172.21.15.115:0/403400329 shutdown_connections 2023-12-01T01:54:16.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.595+0000 7fdc0a20b640 1 -- 172.21.15.115:0/403400329 wait complete. 2023-12-01T01:54:16.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.597+0000 7fdc0a20b640 1 Processor -- start 2023-12-01T01:54:16.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.597+0000 7fdc0a20b640 1 -- start start 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.598+0000 7fdc0a20b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc0406d310 0x7fdc040cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.598+0000 7fdc0a20b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040b09c0 0x7fdc04141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.598+0000 7fdc0a20b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc0414ff20 0x7fdc040ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.598+0000 7fdc0a20b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc04071080 con 0x7fdc0414ff20 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.598+0000 7fdc0a20b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040af820 con 0x7fdc040b09c0 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.598+0000 7fdc0a20b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc041e4bb0 con 0x7fdc0406d310 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.598+0000 7fdc03fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc0414ff20 0x7fdc040ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.598+0000 7fdc03fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc0414ff20 0x7fdc040ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44020/0 (socket says 172.21.15.115:44020) 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.598+0000 7fdc03fff640 1 -- 172.21.15.115:0/2793471305 learned_addr learned my addr 172.21.15.115:0/2793471305 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdbfbfff640 1 --2- 172.21.15.115:0/2793471305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040b09c0 0x7fdc04141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc037fe640 1 --2- 172.21.15.115:0/2793471305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc0406d310 0x7fdc040cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc03fff640 1 -- 172.21.15.115:0/2793471305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc0406d310 msgr2=0x7fdc040cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:16.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc03fff640 1 --2- 172.21.15.115:0/2793471305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc0406d310 0x7fdc040cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc03fff640 1 -- 172.21.15.115:0/2793471305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040b09c0 msgr2=0x7fdc04141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc03fff640 1 --2- 172.21.15.115:0/2793471305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040b09c0 0x7fdc04141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc03fff640 1 -- 172.21.15.115:0/2793471305 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc009b30 con 0x7fdc0414ff20 2023-12-01T01:54:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc03fff640 1 --2- 172.21.15.115:0/2793471305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc0414ff20 0x7fdc040ac890 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009f70 tx=0x7fdbfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc00ff9640 1 -- 172.21.15.115:0/2793471305 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc009510 con 0x7fdc0414ff20 2023-12-01T01:54:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc00ff9640 1 -- 172.21.15.115:0/2793471305 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbfc03e450 con 0x7fdc0414ff20 2023-12-01T01:54:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc00ff9640 1 -- 172.21.15.115:0/2793471305 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc03e7e0 con 0x7fdc0414ff20 2023-12-01T01:54:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.599+0000 7fdc0a20b640 1 -- 172.21.15.115:0/2793471305 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc04070820 con 0x7fdc0414ff20 2023-12-01T01:54:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.600+0000 7fdc0a20b640 1 -- 172.21.15.115:0/2793471305 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc04070da0 con 0x7fdc0414ff20 2023-12-01T01:54:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.601+0000 7fdc0a20b640 1 -- 172.21.15.115:0/2793471305 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc041e4d60 con 0x7fdc0414ff20 2023-12-01T01:54:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.601+0000 7fdc00ff9640 1 -- 172.21.15.115:0/2793471305 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 58) v1 ==== 88318+0+0 (secure 0 0 0) 0x7fdbfc03e940 con 0x7fdc0414ff20 2023-12-01T01:54:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.602+0000 7fdc00ff9640 1 -- 172.21.15.115:0/2793471305 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fdbfc0ab770 con 0x7fdc0414ff20 2023-12-01T01:54:16.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.606+0000 7fdc00ff9640 1 -- 172.21.15.115:0/2793471305 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdc041e4d60 con 0x7fdc0414ff20 2023-12-01T01:54:16.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.794+0000 7fdc0a20b640 1 -- 172.21.15.115:0/2793471305 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc041e4d20 con 0x7fdc0414ff20 2023-12-01T01:54:16.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.794+0000 7fdc00ff9640 1 -- 172.21.15.115:0/2793471305 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdbfc076f30 con 0x7fdc0414ff20 2023-12-01T01:54:16.795 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:54:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.797+0000 7fdc0a20b640 1 -- 172.21.15.115:0/2793471305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc0414ff20 msgr2=0x7fdc040ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.797+0000 7fdc0a20b640 1 --2- 172.21.15.115:0/2793471305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc0414ff20 0x7fdc040ac890 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009f70 tx=0x7fdbfc004a40 comp rx=0 tx=0).stop 2023-12-01T01:54:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.797+0000 7fdc0a20b640 1 -- 172.21.15.115:0/2793471305 shutdown_connections 2023-12-01T01:54:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.797+0000 7fdc0a20b640 1 --2- 172.21.15.115:0/2793471305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc0414ff20 0x7fdc040ac890 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.797+0000 7fdc0a20b640 1 --2- 172.21.15.115:0/2793471305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc040b09c0 0x7fdc04141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.797+0000 7fdc0a20b640 1 --2- 172.21.15.115:0/2793471305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc0406d310 0x7fdc040cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.797+0000 7fdc0a20b640 1 -- 172.21.15.115:0/2793471305 >> 172.21.15.115:0/2793471305 conn(0x7fdc04065960 msgr2=0x7fdc0414f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.797+0000 7fdc0a20b640 1 -- 172.21.15.115:0/2793471305 shutdown_connections 2023-12-01T01:54:16.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:16.797+0000 7fdc0a20b640 1 -- 172.21.15.115:0/2793471305 wait complete. 2023-12-01T01:54:16.816 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:54:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.035+0000 7f8a10d05640 1 Processor -- start 2023-12-01T01:54:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.035+0000 7f8a10d05640 1 -- start start 2023-12-01T01:54:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.037+0000 7f8a10d05640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c0b5d80 0x7f8a0c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.037+0000 7f8a10d05640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0ce2e0 con 0x7f8a0c15cd20 2023-12-01T01:54:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.037+0000 7f8a10d05640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0ce000 con 0x7f8a0c0d1230 2023-12-01T01:54:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.037+0000 7f8a10d05640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0b2160 con 0x7f8a0c0b5d80 2023-12-01T01:54:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.037+0000 7f8a0a59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8a0c15cd20 0x7f8a0c0b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44564/0 (socket says 172.21.15.115:44564) 2023-12-01T01:54:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.037+0000 7f8a0a59c640 1 -- 172.21.15.115:0/774043027 learned_addr learned my addr 172.21.15.115:0/774043027 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.037+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274552077 0 0) 0x7f8a0c0ce2e0 con 0x7f8a0c15cd20 2023-12-01T01:54:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.037+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f8003520 con 0x7f8a0c15cd20 2023-12-01T01:54:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.037+0000 7f8a01d9b640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c0b5d80 0x7f8a0c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1320762746 0 0) 0x7f89f8003520 con 0x7f8a0c15cd20 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c0b5d80 msgr2=0x7f8a0c0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c0b5d80 0x7f8a0c0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 >> v1:172.21.15.142:6789/0 conn(0x7f8a0c0d1230 legacy=0x7f8a0c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0cd3d0 con 0x7f8a0c15cd20 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f89fc0019f0 con 0x7f8a0c15cd20 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 >> v1:172.21.15.115:6789/0 conn(0x7f8a0c15cd20 legacy=0x7f8a0c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f8003a10 0x7f89f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f80047c0 0x7f89f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c0b5d80 0x7f89f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f8005480 con 0x7f89f8003a10 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80055f0 con 0x7f89f80047c0 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f800bec0 con 0x7f8a0c0b5d80 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a0a59c640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f8003a10 0x7f89f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a01d9b640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f80047c0 0x7f89f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a0a59c640 1 -- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c0b5d80 msgr2=0x7f89f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a0a59c640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c0b5d80 0x7f89f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a0a59c640 1 -- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f80047c0 msgr2=0x7f89f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.038+0000 7f8a0a59c640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f80047c0 0x7f89f8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.039+0000 7f8a0a59c640 1 -- 172.21.15.115:0/774043027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f8003520 con 0x7f89f8003a10 2023-12-01T01:54:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.039+0000 7f8a01d9b640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f80047c0 0x7f89f8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.040+0000 7f8a0a59c640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f8003a10 0x7f89f8003e10 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00a830 tx=0x7f89f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976d3c9e2b9b0830 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.040+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc034630 con 0x7f89f8003a10 2023-12-01T01:54:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.040+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc02f070 con 0x7f89f8003a10 2023-12-01T01:54:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.040+0000 7f8a09d9b640 1 -- 172.21.15.115:0/774043027 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0349f0 con 0x7f89f8003a10 2023-12-01T01:54:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.040+0000 7f8a10d05640 1 -- 172.21.15.115:0/774043027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f8003a10 msgr2=0x7f89f8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.040+0000 7f8a10d05640 1 --2- 172.21.15.115:0/774043027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f8003a10 0x7f89f8003e10 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00a830 tx=0x7f89f80057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.040+0000 7f8a0ad9d640 1 -- 172.21.15.115:0/774043027 reap_dead start 2023-12-01T01:54:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.040+0000 7f8a10d05640 1 -- 172.21.15.115:0/774043027 shutdown_connections 2023-12-01T01:54:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.040+0000 7f8a10d05640 1 -- 172.21.15.115:0/774043027 >> 172.21.15.115:0/774043027 conn(0x7f8a0c06a0e0 msgr2=0x7f8a0c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.040+0000 7f8a10d05640 1 -- 172.21.15.115:0/774043027 shutdown_connections 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.041+0000 7f8a10d05640 1 -- 172.21.15.115:0/774043027 wait complete. 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.042+0000 7f8a10d05640 1 Processor -- start 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.042+0000 7f8a10d05640 1 -- start start 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.043+0000 7f8a10d05640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f8003a10 0x7f8a0c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.043+0000 7f8a10d05640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c1f5760 0x7f8a0c15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.043+0000 7f8a10d05640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a0c15d430 0x7f8a0c15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.043+0000 7f8a10d05640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c1f3330 con 0x7f8a0c15d430 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.043+0000 7f8a10d05640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c15de20 con 0x7f89f8003a10 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.043+0000 7f8a10d05640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c1e51f0 con 0x7f8a0c1f5760 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.044+0000 7f8a0ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a0c15d430 0x7f8a0c15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.044+0000 7f8a0ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a0c15d430 0x7f8a0c15d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44034/0 (socket says 172.21.15.115:44034) 2023-12-01T01:54:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.044+0000 7f8a0ad9d640 1 -- 172.21.15.115:0/1164661650 learned_addr learned my addr 172.21.15.115:0/1164661650 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.044+0000 7f8a0a59c640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f8003a10 0x7f8a0c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.044+0000 7f8a0ad9d640 1 -- 172.21.15.115:0/1164661650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c1f5760 msgr2=0x7f8a0c15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.044+0000 7f8a0ad9d640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c1f5760 0x7f8a0c15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.044+0000 7f8a0ad9d640 1 -- 172.21.15.115:0/1164661650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f8003a10 msgr2=0x7f8a0c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.044+0000 7f8a0ad9d640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f8003a10 0x7f8a0c15f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.044+0000 7f8a0ad9d640 1 -- 172.21.15.115:0/1164661650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c1f5fb0 con 0x7f8a0c15d430 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.044+0000 7f8a0ad9d640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a0c15d430 0x7f8a0c15d8e0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0b1d50 tx=0x7f8a0400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.045+0000 7f8a037fe640 1 -- 172.21.15.115:0/1164661650 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040074e0 con 0x7f8a0c15d430 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.045+0000 7f8a037fe640 1 -- 172.21.15.115:0/1164661650 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a04007640 con 0x7f8a0c15d430 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.045+0000 7f8a10d05640 1 -- 172.21.15.115:0/1164661650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c1f6290 con 0x7f8a0c15d430 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.045+0000 7f8a037fe640 1 -- 172.21.15.115:0/1164661650 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040059e0 con 0x7f8a0c15d430 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.045+0000 7f8a10d05640 1 -- 172.21.15.115:0/1164661650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c1e5490 con 0x7f8a0c15d430 2023-12-01T01:54:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.046+0000 7f8a10d05640 1 -- 172.21.15.115:0/1164661650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c04e890 con 0x7f8a0c15d430 2023-12-01T01:54:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.047+0000 7f8a037fe640 1 -- 172.21.15.115:0/1164661650 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 59) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f8a04005b40 con 0x7f8a0c15d430 2023-12-01T01:54:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.048+0000 7f8a037fe640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f89ec06a2b0 0x7f89ec06c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.048+0000 7f8a037fe640 1 -- 172.21.15.115:0/1164661650 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f8a04087980 con 0x7f8a0c15d430 2023-12-01T01:54:17.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.048+0000 7f8a0a59c640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f89ec06a2b0 0x7f89ec06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.049+0000 7f8a0a59c640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f89ec06a2b0 0x7f89ec06c770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f89fc034000 tx=0x7f89fc02f000 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.052+0000 7f8a037fe640 1 -- 172.21.15.115:0/1164661650 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8a04053000 con 0x7f8a0c15d430 2023-12-01T01:54:17.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.282+0000 7f8a10d05640 1 -- 172.21.15.115:0/1164661650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a0c1e5650 con 0x7f8a0c15d430 2023-12-01T01:54:17.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.283+0000 7f8a037fe640 1 -- 172.21.15.115:0/1164661650 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f8a04017090 con 0x7f8a0c15d430 2023-12-01T01:54:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:17.283 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3538792232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3538792232},{"type":"v1","addr":"172.21.15.142:6835","nonce":3538792232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7449,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3171906828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3171906828},{"type":"v1","addr":"172.21.15.115:6835","nonce":3171906828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7452,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2130806226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2130806226},{"type":"v1","addr":"172.21.15.115:6837","nonce":2130806226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:48.491310+0000","modified":"2023-12-01T01:52:51.513496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7455":{"gid":7455,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/3108729597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3108729597},{"type":"v1","addr":"172.21.15.142:6834","nonce":3108729597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-01T01:54:17.284 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 48 2023-12-01T01:54:17.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.285+0000 7f8a10d05640 1 -- 172.21.15.115:0/1164661650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f89ec06a2b0 msgr2=0x7f89ec06c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.285+0000 7f8a10d05640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f89ec06a2b0 0x7f89ec06c770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f89fc034000 tx=0x7f89fc02f000 comp rx=0 tx=0).stop 2023-12-01T01:54:17.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.285+0000 7f8a10d05640 1 -- 172.21.15.115:0/1164661650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a0c15d430 msgr2=0x7f8a0c15d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.285+0000 7f8a10d05640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a0c15d430 0x7f8a0c15d8e0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0b1d50 tx=0x7f8a0400c820 comp rx=0 tx=0).stop 2023-12-01T01:54:17.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.285+0000 7f8a10d05640 1 -- 172.21.15.115:0/1164661650 shutdown_connections 2023-12-01T01:54:17.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.285+0000 7f8a10d05640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f89ec06a2b0 0x7f89ec06c770 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.285+0000 7f8a10d05640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a0c1f5760 0x7f8a0c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.285+0000 7f8a10d05640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a0c15d430 0x7f8a0c15d8e0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.285+0000 7f8a10d05640 1 --2- 172.21.15.115:0/1164661650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f8003a10 0x7f8a0c15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.285+0000 7f8a10d05640 1 -- 172.21.15.115:0/1164661650 >> 172.21.15.115:0/1164661650 conn(0x7f8a0c06a0e0 msgr2=0x7f8a0c0c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:17.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.286+0000 7f8a10d05640 1 -- 172.21.15.115:0/1164661650 shutdown_connections 2023-12-01T01:54:17.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.286+0000 7f8a10d05640 1 -- 172.21.15.115:0/1164661650 wait complete. 2023-12-01T01:54:17.305 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:54:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.518+0000 7f76f1032640 1 Processor -- start 2023-12-01T01:54:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.518+0000 7f76f1032640 1 -- start start 2023-12-01T01:54:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.519+0000 7f76f1032640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f76ec06c2f0 0x7f76ec0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.519+0000 7f76f1032640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ec06f2f0 con 0x7f76ec0c8ed0 2023-12-01T01:54:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.520+0000 7f76f1032640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ec06f5d0 con 0x7f76ec0aded0 2023-12-01T01:54:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.520+0000 7f76f1032640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ec0c9350 con 0x7f76ec06c2f0 2023-12-01T01:54:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.520+0000 7f76ea59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f76ec0c8ed0 0x7f76ec06fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:44568/0 (socket says 172.21.15.115:44568) 2023-12-01T01:54:17.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.520+0000 7f76ea59c640 1 -- 172.21.15.115:0/884466170 learned_addr learned my addr 172.21.15.115:0/884466170 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:17.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.520+0000 7f76eb59e640 1 --2- 172.21.15.115:0/884466170 >> v2:172.21.15.142:3300/0 conn(0x7f76ec06c2f0 0x7f76ec0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.520+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487703719 0 0) 0x7f76ec06f2f0 con 0x7f76ec0c8ed0 2023-12-01T01:54:17.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.520+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76d4003520 con 0x7f76ec0c8ed0 2023-12-01T01:54:17.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1452211173 0 0) 0x7f76d4003520 con 0x7f76ec0c8ed0 2023-12-01T01:54:17.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 >> v2:172.21.15.142:3300/0 conn(0x7f76ec06c2f0 msgr2=0x7f76ec0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:17.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 --2- 172.21.15.115:0/884466170 >> v2:172.21.15.142:3300/0 conn(0x7f76ec06c2f0 0x7f76ec0cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 >> v1:172.21.15.142:6789/0 conn(0x7f76ec0aded0 legacy=0x7f76ec06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec0c9630 con 0x7f76ec0c8ed0 2023-12-01T01:54:17.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f76e0001670 con 0x7f76ec0c8ed0 2023-12-01T01:54:17.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76eb59e640 1 --2- 172.21.15.115:0/884466170 >> v2:172.21.15.142:3300/0 conn(0x7f76ec06c2f0 0x7f76ec0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:54:17.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 >> v1:172.21.15.115:6789/0 conn(0x7f76ec0c8ed0 legacy=0x7f76ec06fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40039e0 0x7f76d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76d40047c0 0x7f76d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76d40052a0 0x7f76d400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec0c9350 con 0x7f76d40052a0 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d400e400 con 0x7f76d40039e0 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d400e5c0 con 0x7f76d40047c0 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76eb59e640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76d40052a0 0x7f76d400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76ead9d640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40039e0 0x7f76d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76ea59c640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76d40047c0 0x7f76d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76eb59e640 1 -- 172.21.15.115:0/884466170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76d40047c0 msgr2=0x7f76d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.521+0000 7f76eb59e640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76d40047c0 0x7f76d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.522+0000 7f76eb59e640 1 -- 172.21.15.115:0/884466170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40039e0 msgr2=0x7f76d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.522+0000 7f76eb59e640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d40039e0 0x7f76d4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.522+0000 7f76eb59e640 1 -- 172.21.15.115:0/884466170 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4003520 con 0x7f76d40052a0 2023-12-01T01:54:17.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.522+0000 7f76ea59c640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76d40047c0 0x7f76d4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:17.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.522+0000 7f76eb59e640 1 -- 172.21.15.115:0/884466170 reap_dead start 2023-12-01T01:54:17.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.523+0000 7f76eb59e640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76d40052a0 0x7f76d400db90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0cb2d0 tx=0x7f76d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7ae553dce3ab98 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:17.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.523+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e402cd60 con 0x7f76d40052a0 2023-12-01T01:54:17.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.523+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e4034440 con 0x7f76d40052a0 2023-12-01T01:54:17.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.523+0000 7f76e9d9b640 1 -- 172.21.15.115:0/884466170 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e40347f0 con 0x7f76d40052a0 2023-12-01T01:54:17.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.523+0000 7f76f1032640 1 -- 172.21.15.115:0/884466170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76d40052a0 msgr2=0x7f76d400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.523+0000 7f76f1032640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76d40052a0 0x7f76d400db90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0cb2d0 tx=0x7f76d40057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.524+0000 7f76f1032640 1 -- 172.21.15.115:0/884466170 shutdown_connections 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.524+0000 7f76f1032640 1 --2- 172.21.15.115:0/884466170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76d40052a0 0x7f76d400db90 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.524+0000 7f76f1032640 1 -- 172.21.15.115:0/884466170 >> 172.21.15.115:0/884466170 conn(0x7f76ec0ce880 msgr2=0x7f76ec0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.524+0000 7f76f1032640 1 -- 172.21.15.115:0/884466170 shutdown_connections 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.524+0000 7f76f1032640 1 -- 172.21.15.115:0/884466170 wait complete. 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.525+0000 7f76f1032640 1 Processor -- start 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.526+0000 7f76f1032640 1 -- start start 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.526+0000 7f76f1032640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76d40052a0 0x7f76ec15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.526+0000 7f76f1032640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec0c4b50 0x7f76ec147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.526+0000 7f76f1032640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ec15bb80 0x7f76ec0c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.526+0000 7f76f1032640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec148280 con 0x7f76ec0c4b50 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.526+0000 7f76f1032640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec15a110 con 0x7f76ec15bb80 2023-12-01T01:54:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76f1032640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec15a9b0 con 0x7f76d40052a0 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76ea59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec0c4b50 0x7f76ec147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76ea59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec0c4b50 0x7f76ec147d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44050/0 (socket says 172.21.15.115:44050) 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76ea59c640 1 -- 172.21.15.115:0/3956194006 learned_addr learned my addr 172.21.15.115:0/3956194006 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76ead9d640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76d40052a0 0x7f76ec15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76ea59c640 1 -- 172.21.15.115:0/3956194006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76d40052a0 msgr2=0x7f76ec15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76eb59e640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ec15bb80 0x7f76ec0c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76ea59c640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76d40052a0 0x7f76ec15ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76ea59c640 1 -- 172.21.15.115:0/3956194006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ec15bb80 msgr2=0x7f76ec0c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76ea59c640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ec15bb80 0x7f76ec0c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.527+0000 7f76ea59c640 1 -- 172.21.15.115:0/3956194006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec15c040 con 0x7f76ec0c4b50 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.528+0000 7f76ea59c640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec0c4b50 0x7f76ec147d40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f76e000d7e0 tx=0x7f76e000dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.528+0000 7f76d37fe640 1 -- 172.21.15.115:0/3956194006 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e00074e0 con 0x7f76ec0c4b50 2023-12-01T01:54:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.528+0000 7f76d37fe640 1 -- 172.21.15.115:0/3956194006 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e000de90 con 0x7f76ec0c4b50 2023-12-01T01:54:17.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.528+0000 7f76f1032640 1 -- 172.21.15.115:0/3956194006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ec15a360 con 0x7f76ec0c4b50 2023-12-01T01:54:17.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.528+0000 7f76d37fe640 1 -- 172.21.15.115:0/3956194006 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0004a20 con 0x7f76ec0c4b50 2023-12-01T01:54:17.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.528+0000 7f76f1032640 1 -- 172.21.15.115:0/3956194006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec154c60 con 0x7f76ec0c4b50 2023-12-01T01:54:17.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.529+0000 7f76f1032640 1 -- 172.21.15.115:0/3956194006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ec15c320 con 0x7f76ec0c4b50 2023-12-01T01:54:17.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.530+0000 7f76d37fe640 1 -- 172.21.15.115:0/3956194006 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 59) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f76e0004b80 con 0x7f76ec0c4b50 2023-12-01T01:54:17.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.531+0000 7f76d37fe640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f76d806a2b0 0x7f76d806c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.531+0000 7f76d37fe640 1 -- 172.21.15.115:0/3956194006 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f76e00882c0 con 0x7f76ec0c4b50 2023-12-01T01:54:17.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.532+0000 7f76ead9d640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f76d806a2b0 0x7f76d806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.532+0000 7f76ead9d640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f76d806a2b0 0x7f76d806c770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f76dc005490 tx=0x7f76dc009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:17.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.536+0000 7f76d37fe640 1 -- 172.21.15.115:0/3956194006 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f76e0053960 con 0x7f76ec0c4b50 2023-12-01T01:54:17.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.730+0000 7f76f1032640 1 -- 172.21.15.115:0/3956194006 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76ec04e890 con 0x7f76ec0c4b50 2023-12-01T01:54:17.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.733+0000 7f76d37fe640 1 -- 172.21.15.115:0/3956194006 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17165 (secure 0 0 0) 0x7f76e0019020 con 0x7f76ec0c4b50 2023-12-01T01:54:17.734 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:17.734 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":113,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:54:15.949036+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:46.948807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:47.513140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2630261754":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/1784916208":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/2568072626":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/425252523":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/4226313375":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/2179794808":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/2366841392":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/2335792152":"2023-12-02T01:54:15.948999+0000","172.21.15.115:6824/88317":"2023-12-02T01:53:07.800872+0000","172.21.15.115:6822/88317":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/2605263330":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3479742140":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/1863268774":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/1925101108":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3510224424":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3799042057":"2023-12-02T01:53:07.800872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:54:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 -- 172.21.15.115:0/3956194006 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f76d806a2b0 msgr2=0x7f76d806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f76d806a2b0 0x7f76d806c770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f76dc005490 tx=0x7f76dc009040 comp rx=0 tx=0).stop 2023-12-01T01:54:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 -- 172.21.15.115:0/3956194006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec0c4b50 msgr2=0x7f76ec147d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec0c4b50 0x7f76ec147d40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f76e000d7e0 tx=0x7f76e000dca0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 -- 172.21.15.115:0/3956194006 shutdown_connections 2023-12-01T01:54:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f76d806a2b0 0x7f76d806c770 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ec15bb80 0x7f76ec0c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76ec0c4b50 0x7f76ec147d40 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 --2- 172.21.15.115:0/3956194006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76d40052a0 0x7f76ec15ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 -- 172.21.15.115:0/3956194006 >> 172.21.15.115:0/3956194006 conn(0x7f76ec0ce880 msgr2=0x7f76ec0af560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:17.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 -- 172.21.15.115:0/3956194006 shutdown_connections 2023-12-01T01:54:17.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.736+0000 7f76f1032640 1 -- 172.21.15.115:0/3956194006 wait complete. 2023-12-01T01:54:17.755 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T01:54:17.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:54:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.967+0000 7f9151f73640 1 Processor -- start 2023-12-01T01:54:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.967+0000 7f9151f73640 1 -- start start 2023-12-01T01:54:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.968+0000 7f9151f73640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f914c0c94c0 0x7f914c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.969+0000 7f9151f73640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914c06b7a0 con 0x7f914c0ccfc0 2023-12-01T01:54:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.969+0000 7f9151f73640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914c0709f0 con 0x7f914c159bb0 2023-12-01T01:54:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.969+0000 7f9151f73640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914c0ce290 con 0x7f914c0c94c0 2023-12-01T01:54:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.969+0000 7f914bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f914c0ccfc0 0x7f914c15d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59854/0 (socket says 172.21.15.115:59854) 2023-12-01T01:54:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.969+0000 7f914bfff640 1 -- 172.21.15.115:0/2672234135 learned_addr learned my addr 172.21.15.115:0/2672234135 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:17.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.969+0000 7f9143fff640 1 --2- 172.21.15.115:0/2672234135 >> v2:172.21.15.142:3300/0 conn(0x7f914c0c94c0 0x7f914c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.969+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610718230 0 0) 0x7f914c06b7a0 con 0x7f914c0ccfc0 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.969+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9138003520 con 0x7f914c0ccfc0 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3393648264 0 0) 0x7f9138003520 con 0x7f914c0ccfc0 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 >> v2:172.21.15.142:3300/0 conn(0x7f914c0c94c0 msgr2=0x7f914c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 --2- 172.21.15.115:0/2672234135 >> v2:172.21.15.142:3300/0 conn(0x7f914c0c94c0 0x7f914c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 >> v1:172.21.15.142:6789/0 conn(0x7f914c159bb0 legacy=0x7f914c0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c0ce970 con 0x7f914c0ccfc0 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f91440019f0 con 0x7f914c0ccfc0 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 >> v1:172.21.15.115:6789/0 conn(0x7f914c0ccfc0 legacy=0x7f914c15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 --2- 172.21.15.115:0/2672234135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9138003a60 0x7f9138003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 --2- 172.21.15.115:0/2672234135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91380047c0 0x7f9138004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 --2- 172.21.15.115:0/2672234135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91380051d0 0x7f913800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c0ce290 con 0x7f91380047c0 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913800e330 con 0x7f91380051d0 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913800e4f0 con 0x7f9138003a60 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.970+0000 7f9143fff640 1 --2- 172.21.15.115:0/2672234135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91380047c0 0x7f9138004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.971+0000 7f914bfff640 1 --2- 172.21.15.115:0/2672234135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91380051d0 0x7f913800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.971+0000 7f9143fff640 1 -- 172.21.15.115:0/2672234135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9138003a60 msgr2=0x7f9138003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.971+0000 7f9143fff640 1 --2- 172.21.15.115:0/2672234135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9138003a60 0x7f9138003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.971+0000 7f9143fff640 1 -- 172.21.15.115:0/2672234135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91380051d0 msgr2=0x7f913800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.971+0000 7f9143fff640 1 --2- 172.21.15.115:0/2672234135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91380051d0 0x7f913800db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.971+0000 7f9143fff640 1 -- 172.21.15.115:0/2672234135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138003520 con 0x7f91380047c0 2023-12-01T01:54:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.971+0000 7f914bfff640 1 -- 172.21.15.115:0/2672234135 reap_dead start 2023-12-01T01:54:17.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.972+0000 7f9143fff640 1 --2- 172.21.15.115:0/2672234135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91380047c0 0x7f9138004c40 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9134009a90 tx=0x7f91380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d2a08a5f09f5bc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:17.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.972+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913402c680 con 0x7f91380047c0 2023-12-01T01:54:17.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.972+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913402c7e0 con 0x7f91380047c0 2023-12-01T01:54:17.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.972+0000 7f914affd640 1 -- 172.21.15.115:0/2672234135 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913402cb70 con 0x7f91380047c0 2023-12-01T01:54:17.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.973+0000 7f9151f73640 1 -- 172.21.15.115:0/2672234135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91380047c0 msgr2=0x7f9138004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.973+0000 7f9151f73640 1 --2- 172.21.15.115:0/2672234135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91380047c0 0x7f9138004c40 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9134009a90 tx=0x7f91380057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.973+0000 7f9151f73640 1 -- 172.21.15.115:0/2672234135 shutdown_connections 2023-12-01T01:54:17.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.973+0000 7f9151f73640 1 --2- 172.21.15.115:0/2672234135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91380047c0 0x7f9138004c40 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.973+0000 7f9151f73640 1 -- 172.21.15.115:0/2672234135 >> 172.21.15.115:0/2672234135 conn(0x7f914c0cece0 msgr2=0x7f914c0cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:17.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.973+0000 7f9151f73640 1 -- 172.21.15.115:0/2672234135 shutdown_connections 2023-12-01T01:54:17.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.973+0000 7f9151f73640 1 -- 172.21.15.115:0/2672234135 wait complete. 2023-12-01T01:54:17.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.975+0000 7f9151f73640 1 Processor -- start 2023-12-01T01:54:17.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.975+0000 7f9151f73640 1 -- start start 2023-12-01T01:54:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.976+0000 7f9151f73640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91380047c0 0x7f914c0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.976+0000 7f9151f73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c1f28a0 0x7f914c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.976+0000 7f9151f73640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f914c0c8a70 0x7f914c15c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.976+0000 7f9151f73640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c0c7380 con 0x7f914c0c8a70 2023-12-01T01:54:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.976+0000 7f9151f73640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c0ac8a0 con 0x7f914c1f28a0 2023-12-01T01:54:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.976+0000 7f9151f73640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914c1f2e10 con 0x7f91380047c0 2023-12-01T01:54:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.976+0000 7f914bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f914c0c8a70 0x7f914c15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.976+0000 7f914bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f914c0c8a70 0x7f914c15c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38820/0 (socket says 172.21.15.115:38820) 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.976+0000 7f914bfff640 1 -- 172.21.15.115:0/2101884863 learned_addr learned my addr 172.21.15.115:0/2101884863 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.976+0000 7f914b7fe640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91380047c0 0x7f914c0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f9143fff640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c1f28a0 0x7f914c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f914bfff640 1 -- 172.21.15.115:0/2101884863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91380047c0 msgr2=0x7f914c0c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f914bfff640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91380047c0 0x7f914c0c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f914bfff640 1 -- 172.21.15.115:0/2101884863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c1f28a0 msgr2=0x7f914c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f914bfff640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c1f28a0 0x7f914c0b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f914bfff640 1 -- 172.21.15.115:0/2101884863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c1f30f0 con 0x7f914c0c8a70 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f914bfff640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f914c0c8a70 0x7f914c15c160 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f914c069d10 tx=0x7f914400df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f9148ff9640 1 -- 172.21.15.115:0/2101884863 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914400b7a0 con 0x7f914c0c8a70 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f9148ff9640 1 -- 172.21.15.115:0/2101884863 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914400b900 con 0x7f914c0c8a70 2023-12-01T01:54:17.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f9151f73640 1 -- 172.21.15.115:0/2101884863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914c1f33d0 con 0x7f914c0c8a70 2023-12-01T01:54:17.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f9148ff9640 1 -- 172.21.15.115:0/2101884863 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914400bcb0 con 0x7f914c0c8a70 2023-12-01T01:54:17.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.977+0000 7f9151f73640 1 -- 172.21.15.115:0/2101884863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914c0b46b0 con 0x7f914c0c8a70 2023-12-01T01:54:17.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.978+0000 7f9151f73640 1 -- 172.21.15.115:0/2101884863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914c04e890 con 0x7f914c0c8a70 2023-12-01T01:54:17.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.979+0000 7f9148ff9640 1 -- 172.21.15.115:0/2101884863 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 59) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f9144007070 con 0x7f914c0c8a70 2023-12-01T01:54:17.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.980+0000 7f9148ff9640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f912c06a280 0x7f912c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:17.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.980+0000 7f9148ff9640 1 -- 172.21.15.115:0/2101884863 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f9144088b00 con 0x7f914c0c8a70 2023-12-01T01:54:17.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.981+0000 7f914b7fe640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f912c06a280 0x7f912c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:17.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.981+0000 7f914b7fe640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f912c06a280 0x7f912c06c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f914c06fae0 tx=0x7f913c009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:17.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:17.985+0000 7f9148ff9640 1 -- 172.21.15.115:0/2101884863 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f914408ae60 con 0x7f914c0c8a70 2023-12-01T01:54:18.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.222+0000 7f9151f73640 1 -- 172.21.15.115:0/2101884863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f914c0b4c60 con 0x7f914c0c8a70 2023-12-01T01:54:18.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.223+0000 7f9148ff9640 1 -- 172.21.15.115:0/2101884863 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f91440540b0 con 0x7f914c0c8a70 2023-12-01T01:54:18.224 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:18.224 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3538792232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3538792232},{"type":"v1","addr":"172.21.15.142:6835","nonce":3538792232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7449,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3171906828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3171906828},{"type":"v1","addr":"172.21.15.115:6835","nonce":3171906828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7452,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2130806226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2130806226},{"type":"v1","addr":"172.21.15.115:6837","nonce":2130806226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:48.491310+0000","modified":"2023-12-01T01:52:51.513496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7455":{"gid":7455,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/3108729597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3108729597},{"type":"v1","addr":"172.21.15.142:6834","nonce":3108729597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-01T01:54:18.224 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 48 2023-12-01T01:54:18.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 -- 172.21.15.115:0/2101884863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f912c06a280 msgr2=0x7f912c06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f912c06a280 0x7f912c06c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f914c06fae0 tx=0x7f913c009040 comp rx=0 tx=0).stop 2023-12-01T01:54:18.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 -- 172.21.15.115:0/2101884863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f914c0c8a70 msgr2=0x7f914c15c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f914c0c8a70 0x7f914c15c160 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f914c069d10 tx=0x7f914400df70 comp rx=0 tx=0).stop 2023-12-01T01:54:18.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 -- 172.21.15.115:0/2101884863 shutdown_connections 2023-12-01T01:54:18.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f912c06a280 0x7f912c06c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f914c0c8a70 0x7f914c15c160 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f914c1f28a0 0x7f914c0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 --2- 172.21.15.115:0/2101884863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91380047c0 0x7f914c0c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 -- 172.21.15.115:0/2101884863 >> 172.21.15.115:0/2101884863 conn(0x7f914c0cece0 msgr2=0x7f914c0c4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:18.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 -- 172.21.15.115:0/2101884863 shutdown_connections 2023-12-01T01:54:18.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.226+0000 7f9151f73640 1 -- 172.21.15.115:0/2101884863 wait complete. 2023-12-01T01:54:18.245 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:54:18.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.454+0000 7f69569a9640 1 Processor -- start 2023-12-01T01:54:18.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.455+0000 7f69569a9640 1 -- start start 2023-12-01T01:54:18.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.456+0000 7f69569a9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6950159dd0 0x7f69500c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.456+0000 7f69569a9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f695006b6f0 0x7f69500c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.456+0000 7f69569a9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f695006f8b0 con 0x7f69500ade80 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.456+0000 7f69569a9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69500adcb0 con 0x7f6950159dd0 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.456+0000 7f69569a9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69500ad9d0 con 0x7f695006b6f0 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.457+0000 7f694ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6950159dd0 0x7f69500c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.457+0000 7f694ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6950159dd0 0x7f69500c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38854/0 (socket says 172.21.15.115:38854) 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.457+0000 7f694ffff640 1 -- 172.21.15.115:0/3263221408 learned_addr learned my addr 172.21.15.115:0/3263221408 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.457+0000 7f6954f46640 1 --1- 172.21.15.115:0/3263221408 >> v1:172.21.15.115:6789/0 conn(0x7f69500ade80 0x7f69500cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59866/0 (socket says 172.21.15.115:59866) 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.457+0000 7f6947fff640 1 --2- 172.21.15.115:0/3263221408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f695006b6f0 0x7f69500c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.457+0000 7f694ffff640 1 -- 172.21.15.115:0/3263221408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f695006b6f0 msgr2=0x7f69500c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:18.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.457+0000 7f694ffff640 1 --2- 172.21.15.115:0/3263221408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f695006b6f0 0x7f69500c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.457+0000 7f694ffff640 1 -- 172.21.15.115:0/3263221408 >> v1:172.21.15.115:6789/0 conn(0x7f69500ade80 legacy=0x7f69500cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.457+0000 7f694ffff640 1 -- 172.21.15.115:0/3263221408 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69500c9000 con 0x7f6950159dd0 2023-12-01T01:54:18.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.458+0000 7f694ffff640 1 --2- 172.21.15.115:0/3263221408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6950159dd0 0x7f69500c97e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6940009870 tx=0x7f694002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324edc06e563e5ed server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:18.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.459+0000 7f694f7fe640 1 -- 172.21.15.115:0/3263221408 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69400317c0 con 0x7f6950159dd0 2023-12-01T01:54:18.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.459+0000 7f694f7fe640 1 -- 172.21.15.115:0/3263221408 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6940031920 con 0x7f6950159dd0 2023-12-01T01:54:18.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.459+0000 7f694f7fe640 1 -- 172.21.15.115:0/3263221408 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940031cb0 con 0x7f6950159dd0 2023-12-01T01:54:18.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.459+0000 7f69569a9640 1 -- 172.21.15.115:0/3263221408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6950159dd0 msgr2=0x7f69500c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.459+0000 7f69569a9640 1 --2- 172.21.15.115:0/3263221408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6950159dd0 0x7f69500c97e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6940009870 tx=0x7f694002b860 comp rx=0 tx=0).stop 2023-12-01T01:54:18.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.459+0000 7f69569a9640 1 -- 172.21.15.115:0/3263221408 shutdown_connections 2023-12-01T01:54:18.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.459+0000 7f69569a9640 1 --2- 172.21.15.115:0/3263221408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f695006b6f0 0x7f69500c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.459+0000 7f69569a9640 1 --2- 172.21.15.115:0/3263221408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6950159dd0 0x7f69500c97e0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.459+0000 7f69569a9640 1 -- 172.21.15.115:0/3263221408 >> 172.21.15.115:0/3263221408 conn(0x7f69500ce880 msgr2=0x7f69500cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:18.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.459+0000 7f69569a9640 1 -- 172.21.15.115:0/3263221408 shutdown_connections 2023-12-01T01:54:18.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.460+0000 7f69569a9640 1 -- 172.21.15.115:0/3263221408 wait complete. 2023-12-01T01:54:18.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.461+0000 7f69569a9640 1 Processor -- start 2023-12-01T01:54:18.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.461+0000 7f69569a9640 1 -- start start 2023-12-01T01:54:18.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.462+0000 7f69569a9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f695006b6f0 0x7f69501f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.462+0000 7f69569a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69500ade80 0x7f695015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.462+0000 7f69569a9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6950159dd0 0x7f69500ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.462+0000 7f69569a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69500ae3a0 con 0x7f69500ade80 2023-12-01T01:54:18.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.462+0000 7f69569a9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69500aef20 con 0x7f6950159dd0 2023-12-01T01:54:18.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.462+0000 7f69569a9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69500ae5f0 con 0x7f695006b6f0 2023-12-01T01:54:18.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.462+0000 7f6947fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69500ade80 0x7f695015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.462+0000 7f6947fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69500ade80 0x7f695015ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38856/0 (socket says 172.21.15.115:38856) 2023-12-01T01:54:18.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.463+0000 7f6947fff640 1 -- 172.21.15.115:0/2460302865 learned_addr learned my addr 172.21.15.115:0/2460302865 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:18.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.463+0000 7f6947fff640 1 -- 172.21.15.115:0/2460302865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f695006b6f0 msgr2=0x7f69501f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.463+0000 7f6947fff640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f695006b6f0 0x7f69501f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.463+0000 7f6947fff640 1 -- 172.21.15.115:0/2460302865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6950159dd0 msgr2=0x7f69500ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.463+0000 7f6947fff640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6950159dd0 0x7f69500ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.463+0000 7f6947fff640 1 -- 172.21.15.115:0/2460302865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69400094d0 con 0x7f69500ade80 2023-12-01T01:54:18.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.463+0000 7f6947fff640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69500ade80 0x7f695015ade0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f693800d830 tx=0x7f693800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:18.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.463+0000 7f694d7fa640 1 -- 172.21.15.115:0/2460302865 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693800b6c0 con 0x7f69500ade80 2023-12-01T01:54:18.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.463+0000 7f694d7fa640 1 -- 172.21.15.115:0/2460302865 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693800b820 con 0x7f69500ade80 2023-12-01T01:54:18.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.464+0000 7f69569a9640 1 -- 172.21.15.115:0/2460302865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69500ae8d0 con 0x7f69500ade80 2023-12-01T01:54:18.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.464+0000 7f694d7fa640 1 -- 172.21.15.115:0/2460302865 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693800bbb0 con 0x7f69500ade80 2023-12-01T01:54:18.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.464+0000 7f69569a9640 1 -- 172.21.15.115:0/2460302865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695015b320 con 0x7f69500ade80 2023-12-01T01:54:18.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.465+0000 7f69569a9640 1 -- 172.21.15.115:0/2460302865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695004e890 con 0x7f69500ade80 2023-12-01T01:54:18.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.466+0000 7f694d7fa640 1 -- 172.21.15.115:0/2460302865 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 59) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f693801e460 con 0x7f69500ade80 2023-12-01T01:54:18.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.466+0000 7f694d7fa640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f693006a280 0x7f693006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.467+0000 7f694d7fa640 1 -- 172.21.15.115:0/2460302865 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f6938088f60 con 0x7f69500ade80 2023-12-01T01:54:18.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.467+0000 7f694ffff640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f693006a280 0x7f693006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.467+0000 7f694ffff640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f693006a280 0x7f693006c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6940036000 tx=0x7f6940030000 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:18.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.471+0000 7f694d7fa640 1 -- 172.21.15.115:0/2460302865 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f69380bbc80 con 0x7f69500ade80 2023-12-01T01:54:18.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.686+0000 7f69569a9640 1 -- 172.21.15.115:0/2460302865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69500af060 con 0x7f69500ade80 2023-12-01T01:54:18.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.687+0000 7f694d7fa640 1 -- 172.21.15.115:0/2460302865 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f69380545e0 con 0x7f69500ade80 2023-12-01T01:54:18.687 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:18.688 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3538792232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3538792232},{"type":"v1","addr":"172.21.15.142:6835","nonce":3538792232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7449,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3171906828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3171906828},{"type":"v1","addr":"172.21.15.115:6835","nonce":3171906828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7452,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2130806226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2130806226},{"type":"v1","addr":"172.21.15.115:6837","nonce":2130806226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:52:48.491310+0000","modified":"2023-12-01T01:52:51.513496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7455":{"gid":7455,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/3108729597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3108729597},{"type":"v1","addr":"172.21.15.142:6834","nonce":3108729597}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-01T01:54:18.688 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 48 2023-12-01T01:54:18.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 -- 172.21.15.115:0/2460302865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f693006a280 msgr2=0x7f693006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f693006a280 0x7f693006c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6940036000 tx=0x7f6940030000 comp rx=0 tx=0).stop 2023-12-01T01:54:18.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 -- 172.21.15.115:0/2460302865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69500ade80 msgr2=0x7f695015ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69500ade80 0x7f695015ade0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f693800d830 tx=0x7f693800dcf0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 -- 172.21.15.115:0/2460302865 shutdown_connections 2023-12-01T01:54:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f693006a280 0x7f693006c740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6950159dd0 0x7f69500ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69500ade80 0x7f695015ade0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 --2- 172.21.15.115:0/2460302865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f695006b6f0 0x7f69501f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 -- 172.21.15.115:0/2460302865 >> 172.21.15.115:0/2460302865 conn(0x7f69500ce880 msgr2=0x7f69500702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 -- 172.21.15.115:0/2460302865 shutdown_connections 2023-12-01T01:54:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.690+0000 7f69569a9640 1 -- 172.21.15.115:0/2460302865 wait complete. 2023-12-01T01:54:18.707 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:54:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.910+0000 7f07a32b6640 1 Processor -- start 2023-12-01T01:54:18.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.910+0000 7f07a32b6640 1 -- start start 2023-12-01T01:54:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.911+0000 7f07a32b6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f079c0cad70 0x7f079c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.911+0000 7f07a32b6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0add10 con 0x7f079c15e2f0 2023-12-01T01:54:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.911+0000 7f07a32b6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0c5760 con 0x7f079c06f2f0 2023-12-01T01:54:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.911+0000 7f07a32b6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079c15e6d0 con 0x7f079c0cad70 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.912+0000 7f07a1052640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f079c15e2f0 0x7f079c0ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59872/0 (socket says 172.21.15.115:59872) 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.912+0000 7f07a1052640 1 -- 172.21.15.115:0/2145310270 learned_addr learned my addr 172.21.15.115:0/2145310270 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.912+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949191405 0 0) 0x7f079c0add10 con 0x7f079c15e2f0 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.912+0000 7f07a0851640 1 --2- 172.21.15.115:0/2145310270 >> v2:172.21.15.142:3300/0 conn(0x7f079c0cad70 0x7f079c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.912+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0788003520 con 0x7f079c15e2f0 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 962654531 0 0) 0x7f0788003520 con 0x7f079c15e2f0 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 >> v2:172.21.15.142:3300/0 conn(0x7f079c0cad70 msgr2=0x7f079c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 --2- 172.21.15.115:0/2145310270 >> v2:172.21.15.142:3300/0 conn(0x7f079c0cad70 0x7f079c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 >> v1:172.21.15.142:6789/0 conn(0x7f079c06f2f0 legacy=0x7f079c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0cf7b0 con 0x7f079c15e2f0 2023-12-01T01:54:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0798001690 con 0x7f079c15e2f0 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 >> v1:172.21.15.115:6789/0 conn(0x7f079c15e2f0 legacy=0x7f079c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07880039e0 0x7f0788003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07880047c0 0x7f0788004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0788005380 0x7f078800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c15e6d0 con 0x7f0788005380 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078800c260 con 0x7f07880047c0 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078800c420 con 0x7f07880039e0 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f07a1853640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0788005380 0x7f078800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f07a0851640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07880047c0 0x7f0788004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f07a1052640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07880039e0 0x7f0788003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f07a1853640 1 -- 172.21.15.115:0/2145310270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07880039e0 msgr2=0x7f0788003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f07a1853640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07880039e0 0x7f0788003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f07a1853640 1 -- 172.21.15.115:0/2145310270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07880047c0 msgr2=0x7f0788004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f07a1853640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07880047c0 0x7f0788004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.913+0000 7f07a1853640 1 -- 172.21.15.115:0/2145310270 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0788003520 con 0x7f0788005380 2023-12-01T01:54:18.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.914+0000 7f07a1853640 1 -- 172.21.15.115:0/2145310270 reap_dead start 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.914+0000 7f07a1853640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0788005380 0x7f078800b980 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f079c06f700 tx=0x7f07880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ecb3839448ec33 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.915+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794006b50 con 0x7f0788005380 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.915+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0794006cb0 con 0x7f0788005380 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.915+0000 7f0793fff640 1 -- 172.21.15.115:0/2145310270 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794007260 con 0x7f0788005380 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.915+0000 7f07a32b6640 1 -- 172.21.15.115:0/2145310270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0788005380 msgr2=0x7f078800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.915+0000 7f07a32b6640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0788005380 0x7f078800b980 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f079c06f700 tx=0x7f07880057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.915+0000 7f07a32b6640 1 -- 172.21.15.115:0/2145310270 shutdown_connections 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.915+0000 7f07a32b6640 1 --2- 172.21.15.115:0/2145310270 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0788005380 0x7f078800b980 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.915+0000 7f07a32b6640 1 -- 172.21.15.115:0/2145310270 >> 172.21.15.115:0/2145310270 conn(0x7f079c0ce880 msgr2=0x7f079c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.915+0000 7f07a32b6640 1 -- 172.21.15.115:0/2145310270 shutdown_connections 2023-12-01T01:54:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.915+0000 7f07a32b6640 1 -- 172.21.15.115:0/2145310270 wait complete. 2023-12-01T01:54:18.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.917+0000 7f07a32b6640 1 Processor -- start 2023-12-01T01:54:18.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.917+0000 7f07a32b6640 1 -- start start 2023-12-01T01:54:18.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.918+0000 7f07a32b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0788005380 0x7f079c14dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.918+0000 7f07a32b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c0ac8a0 0x7f079c0acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.918+0000 7f07a32b6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c0ae860 0x7f079c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.918+0000 7f07a32b6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0ad260 con 0x7f079c0ac8a0 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.918+0000 7f07a32b6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0af5d0 con 0x7f079c0ae860 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.918+0000 7f07a32b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0af8b0 con 0x7f0788005380 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.918+0000 7f07a0851640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c0ac8a0 0x7f079c0acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.918+0000 7f07a0851640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c0ac8a0 0x7f079c0acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38884/0 (socket says 172.21.15.115:38884) 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.918+0000 7f07a0851640 1 -- 172.21.15.115:0/2658653038 learned_addr learned my addr 172.21.15.115:0/2658653038 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f07a1853640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c0ae860 0x7f079c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f07a1052640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0788005380 0x7f079c14dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f07a0851640 1 -- 172.21.15.115:0/2658653038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0788005380 msgr2=0x7f079c14dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f07a0851640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0788005380 0x7f079c14dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f07a0851640 1 -- 172.21.15.115:0/2658653038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c0ae860 msgr2=0x7f079c0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f07a0851640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c0ae860 0x7f079c0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:18.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f07a0851640 1 -- 172.21.15.115:0/2658653038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0afb90 con 0x7f079c0ac8a0 2023-12-01T01:54:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f07a1853640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c0ae860 0x7f079c0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f07a0851640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c0ac8a0 0x7f079c0acd20 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f078c00f580 tx=0x7f078c00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f0791ffb640 1 -- 172.21.15.115:0/2658653038 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c013810 con 0x7f079c0ac8a0 2023-12-01T01:54:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f0791ffb640 1 -- 172.21.15.115:0/2658653038 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c013970 con 0x7f079c0ac8a0 2023-12-01T01:54:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f0791ffb640 1 -- 172.21.15.115:0/2658653038 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c013ce0 con 0x7f079c0ac8a0 2023-12-01T01:54:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.919+0000 7f07a32b6640 1 -- 172.21.15.115:0/2658653038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079c0afe70 con 0x7f079c0ac8a0 2023-12-01T01:54:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.920+0000 7f07a32b6640 1 -- 172.21.15.115:0/2658653038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c0af220 con 0x7f079c0ac8a0 2023-12-01T01:54:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.921+0000 7f07a32b6640 1 -- 172.21.15.115:0/2658653038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079c06fda0 con 0x7f079c0ac8a0 2023-12-01T01:54:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.921+0000 7f0791ffb640 1 -- 172.21.15.115:0/2658653038 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 60) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f078c01f460 con 0x7f079c0ac8a0 2023-12-01T01:54:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.922+0000 7f0791ffb640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f078006a4c0 0x7f078006c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.922+0000 7f0791ffb640 1 -- 172.21.15.115:0/2658653038 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f078c0890a0 con 0x7f079c0ac8a0 2023-12-01T01:54:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.923+0000 7f07a1052640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f078006a4c0 0x7f078006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.923+0000 7f07a1052640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f078006a4c0 0x7f078006c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0798000ff0 tx=0x7f0798009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:18.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:18.926+0000 7f0791ffb640 1 -- 172.21.15.115:0/2658653038 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f078c054300 con 0x7f079c0ac8a0 2023-12-01T01:54:19.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.102+0000 7f07a32b6640 1 -- 172.21.15.115:0/2658653038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f079c0ad740 con 0x7f079c0ac8a0 2023-12-01T01:54:19.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.104+0000 7f0791ffb640 1 -- 172.21.15.115:0/2658653038 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17165 (secure 0 0 0) 0x7f078c057920 con 0x7f079c0ac8a0 2023-12-01T01:54:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:19.105 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":113,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:54:15.949036+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:52:46.948807+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-01T01:52:47.513140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2630261754":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/1784916208":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/2568072626":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/425252523":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/4226313375":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/2179794808":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/2366841392":"2023-12-02T01:54:15.948999+0000","172.21.15.115:0/2335792152":"2023-12-02T01:54:15.948999+0000","172.21.15.115:6824/88317":"2023-12-02T01:53:07.800872+0000","172.21.15.115:6822/88317":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/2605263330":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3479742140":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/1863268774":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/1925101108":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3510224424":"2023-12-02T01:53:07.800872+0000","172.21.15.115:0/3799042057":"2023-12-02T01:53:07.800872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 -- 172.21.15.115:0/2658653038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f078006a4c0 msgr2=0x7f078006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f078006a4c0 0x7f078006c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0798000ff0 tx=0x7f0798009040 comp rx=0 tx=0).stop 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 -- 172.21.15.115:0/2658653038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c0ac8a0 msgr2=0x7f079c0acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c0ac8a0 0x7f079c0acd20 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f078c00f580 tx=0x7f078c00fa40 comp rx=0 tx=0).stop 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 -- 172.21.15.115:0/2658653038 shutdown_connections 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f078006a4c0 0x7f078006c980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079c0ae860 0x7f079c0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f079c0ac8a0 0x7f079c0acd20 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 --2- 172.21.15.115:0/2658653038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0788005380 0x7f079c14dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 -- 172.21.15.115:0/2658653038 >> 172.21.15.115:0/2658653038 conn(0x7f079c0ce880 msgr2=0x7f079c06b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:19.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 -- 172.21.15.115:0/2658653038 shutdown_connections 2023-12-01T01:54:19.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.107+0000 7f07a32b6640 1 -- 172.21.15.115:0/2658653038 wait complete. 2023-12-01T01:54:19.124 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T01:54:19.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.323+0000 7f4b7900b640 1 Processor -- start 2023-12-01T01:54:19.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.324+0000 7f4b7900b640 1 -- start start 2023-12-01T01:54:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.325+0000 7f4b7900b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b7415cd20 0x7f4b740d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.325+0000 7f4b7900b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4b740b5d80 0x7f4b740b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.325+0000 7f4b7900b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b740ce2e0 con 0x7f4b740d1230 2023-12-01T01:54:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.325+0000 7f4b7900b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b740b2160 con 0x7f4b740b5d80 2023-12-01T01:54:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.325+0000 7f4b7900b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740d2a10 con 0x7f4b7415cd20 2023-12-01T01:54:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.326+0000 7f4b7259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4b740b5d80 0x7f4b740b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.326+0000 7f4b7359e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4b740d1230 0x7f4b740d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59880/0 (socket says 172.21.15.115:59880) 2023-12-01T01:54:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.326+0000 7f4b7359e640 1 -- 172.21.15.115:0/3950491097 learned_addr learned my addr 172.21.15.115:0/3950491097 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.326+0000 7f4b72d9d640 1 --2- 172.21.15.115:0/3950491097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b7415cd20 0x7f4b740d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.326+0000 7f4b7259c640 1 -- 172.21.15.115:0/3950491097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b7415cd20 msgr2=0x7f4b740d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.326+0000 7f4b7259c640 1 --2- 172.21.15.115:0/3950491097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b7415cd20 0x7f4b740d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.326+0000 7f4b7259c640 1 -- 172.21.15.115:0/3950491097 >> v1:172.21.15.115:6789/0 conn(0x7f4b740d1230 legacy=0x7f4b740d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.326+0000 7f4b7259c640 1 -- 172.21.15.115:0/3950491097 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b740ce000 con 0x7f4b740b5d80 2023-12-01T01:54:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.327+0000 7f4b7259c640 1 --2- 172.21.15.115:0/3950491097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 0x7f4b740b8570 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0097d0 tx=0x7f4b5c02b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5e72adb2268f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b71d9b640 1 -- 172.21.15.115:0/3950491097 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c031770 con 0x7f4b740b5d80 2023-12-01T01:54:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b71d9b640 1 -- 172.21.15.115:0/3950491097 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c0318d0 con 0x7f4b740b5d80 2023-12-01T01:54:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b71d9b640 1 -- 172.21.15.115:0/3950491097 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c031c60 con 0x7f4b740b5d80 2023-12-01T01:54:19.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b7900b640 1 -- 172.21.15.115:0/3950491097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 msgr2=0x7f4b740b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b7900b640 1 --2- 172.21.15.115:0/3950491097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 0x7f4b740b8570 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0097d0 tx=0x7f4b5c02b810 comp rx=0 tx=0).stop 2023-12-01T01:54:19.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b7900b640 1 -- 172.21.15.115:0/3950491097 shutdown_connections 2023-12-01T01:54:19.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b7900b640 1 --2- 172.21.15.115:0/3950491097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 0x7f4b740b8570 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b7900b640 1 --2- 172.21.15.115:0/3950491097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b7415cd20 0x7f4b740d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b7900b640 1 -- 172.21.15.115:0/3950491097 >> 172.21.15.115:0/3950491097 conn(0x7f4b7406a0e0 msgr2=0x7f4b7406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:19.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b7900b640 1 -- 172.21.15.115:0/3950491097 shutdown_connections 2023-12-01T01:54:19.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.328+0000 7f4b7900b640 1 -- 172.21.15.115:0/3950491097 wait complete. 2023-12-01T01:54:19.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.330+0000 7f4b7900b640 1 Processor -- start 2023-12-01T01:54:19.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.330+0000 7f4b7900b640 1 -- start start 2023-12-01T01:54:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.331+0000 7f4b7900b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 0x7f4b741f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.331+0000 7f4b7900b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b740d1230 0x7f4b740c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.331+0000 7f4b7900b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7415cd20 0x7f4b740c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.331+0000 7f4b7900b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740b55d0 con 0x7f4b740b5d80 2023-12-01T01:54:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.331+0000 7f4b7900b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740b5050 con 0x7f4b7415cd20 2023-12-01T01:54:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.331+0000 7f4b7900b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740cb920 con 0x7f4b740d1230 2023-12-01T01:54:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.331+0000 7f4b72d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 0x7f4b741f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.331+0000 7f4b72d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 0x7f4b741f1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38898/0 (socket says 172.21.15.115:38898) 2023-12-01T01:54:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.331+0000 7f4b72d9d640 1 -- 172.21.15.115:0/1907090048 learned_addr learned my addr 172.21.15.115:0/1907090048 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.332+0000 7f4b72d9d640 1 -- 172.21.15.115:0/1907090048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b740d1230 msgr2=0x7f4b740c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.332+0000 7f4b72d9d640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b740d1230 0x7f4b740c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.332+0000 7f4b72d9d640 1 -- 172.21.15.115:0/1907090048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7415cd20 msgr2=0x7f4b740c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.332+0000 7f4b72d9d640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7415cd20 0x7f4b740c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.332+0000 7f4b72d9d640 1 -- 172.21.15.115:0/1907090048 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c009400 con 0x7f4b740b5d80 2023-12-01T01:54:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.332+0000 7f4b72d9d640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 0x7f4b741f1dc0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4b64007e40 tx=0x7f4b6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:19.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.332+0000 7f4b6b7fe640 1 -- 172.21.15.115:0/1907090048 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64014620 con 0x7f4b740b5d80 2023-12-01T01:54:19.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.332+0000 7f4b6b7fe640 1 -- 172.21.15.115:0/1907090048 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6400ee90 con 0x7f4b740b5d80 2023-12-01T01:54:19.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.332+0000 7f4b7900b640 1 -- 172.21.15.115:0/1907090048 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b740b52f0 con 0x7f4b740b5d80 2023-12-01T01:54:19.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.333+0000 7f4b6b7fe640 1 -- 172.21.15.115:0/1907090048 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640149c0 con 0x7f4b740b5d80 2023-12-01T01:54:19.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.333+0000 7f4b7900b640 1 -- 172.21.15.115:0/1907090048 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b740c6e00 con 0x7f4b740b5d80 2023-12-01T01:54:19.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.334+0000 7f4b7900b640 1 -- 172.21.15.115:0/1907090048 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b740cb780 con 0x7f4b740b5d80 2023-12-01T01:54:19.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.335+0000 7f4b6b7fe640 1 -- 172.21.15.115:0/1907090048 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 60) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f4b64014b20 con 0x7f4b740b5d80 2023-12-01T01:54:19.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.336+0000 7f4b6b7fe640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4b5406a4f0 0x7f4b5406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.336+0000 7f4b6b7fe640 1 -- 172.21.15.115:0/1907090048 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f4b64089580 con 0x7f4b740b5d80 2023-12-01T01:54:19.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.336+0000 7f4b7259c640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4b5406a4f0 0x7f4b5406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:19.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.337+0000 7f4b7259c640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4b5406a4f0 0x7f4b5406c9b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c036000 tx=0x7f4b5c030000 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:19.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.340+0000 7f4b6b7fe640 1 -- 172.21.15.115:0/1907090048 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4b740cb780 con 0x7f4b740b5d80 2023-12-01T01:54:19.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.575+0000 7f4b7900b640 1 -- 172.21.15.115:0/1907090048 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4b740cb780 con 0x7f4b740b5d80 2023-12-01T01:54:19.577 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T01:54:19.577+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:54:19.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.591+0000 7f4b6b7fe640 1 -- 172.21.15.115:0/1907090048 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4b740cb780 con 0x7f4b740b5d80 2023-12-01T01:54:19.592 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T01:54:19.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.594+0000 7f4b7900b640 1 -- 172.21.15.115:0/1907090048 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4b5406a4f0 msgr2=0x7f4b5406c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.594+0000 7f4b7900b640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4b5406a4f0 0x7f4b5406c9b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c036000 tx=0x7f4b5c030000 comp rx=0 tx=0).stop 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.594+0000 7f4b7900b640 1 -- 172.21.15.115:0/1907090048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 msgr2=0x7f4b741f1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.594+0000 7f4b7900b640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 0x7f4b741f1dc0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4b64007e40 tx=0x7f4b6400e4b0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.594+0000 7f4b7900b640 1 -- 172.21.15.115:0/1907090048 shutdown_connections 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.594+0000 7f4b7900b640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4b5406a4f0 0x7f4b5406c9b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.594+0000 7f4b7900b640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4b740d1230 0x7f4b740c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.594+0000 7f4b7900b640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b7415cd20 0x7f4b740c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.594+0000 7f4b7900b640 1 --2- 172.21.15.115:0/1907090048 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b740b5d80 0x7f4b741f1dc0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.594+0000 7f4b7900b640 1 -- 172.21.15.115:0/1907090048 >> 172.21.15.115:0/1907090048 conn(0x7f4b7406a0e0 msgr2=0x7f4b7415bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.595+0000 7f4b7900b640 1 -- 172.21.15.115:0/1907090048 shutdown_connections 2023-12-01T01:54:19.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.595+0000 7f4b7900b640 1 -- 172.21.15.115:0/1907090048 wait complete. 2023-12-01T01:54:19.612 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T01:54:19.626 INFO:tasks.ceph.mds.d.smithi142.stderr: -7047> 2023-12-01T01:52:53.901+0000 7fdfe6e5d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:54:19.705 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T01:54:19.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.819+0000 7fba9759e640 1 Processor -- start 2023-12-01T01:54:19.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.819+0000 7fba9759e640 1 -- start start 2023-12-01T01:54:19.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.821+0000 7fba9759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fba98157360 0x7fba980cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.821+0000 7fba9759e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fba980cdab0 0x7fba980d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.821+0000 7fba9759e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba980cfe60 con 0x7fba980cf9b0 2023-12-01T01:54:19.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.821+0000 7fba9759e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba980ce090 con 0x7fba980cdab0 2023-12-01T01:54:19.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.821+0000 7fba9759e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba980d0f10 con 0x7fba98157360 2023-12-01T01:54:19.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.821+0000 7fba95d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fba980cf9b0 0x7fba980cd3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59882/0 (socket says 172.21.15.115:59882) 2023-12-01T01:54:19.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.821+0000 7fba95d9b640 1 -- 172.21.15.115:0/84255903 learned_addr learned my addr 172.21.15.115:0/84255903 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:19.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.821+0000 7fba96d9d640 1 --2- 172.21.15.115:0/84255903 >> v2:172.21.15.115:3300/0 conn(0x7fba980cdab0 0x7fba980d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:19.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.822+0000 7fba9659c640 1 --2- 172.21.15.115:0/84255903 >> v2:172.21.15.142:3300/0 conn(0x7fba98157360 0x7fba980cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:19.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.822+0000 7fba96d9d640 1 -- 172.21.15.115:0/84255903 >> v2:172.21.15.142:3300/0 conn(0x7fba98157360 msgr2=0x7fba980cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:19.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.822+0000 7fba96d9d640 1 --2- 172.21.15.115:0/84255903 >> v2:172.21.15.142:3300/0 conn(0x7fba98157360 0x7fba980cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.822+0000 7fba96d9d640 1 -- 172.21.15.115:0/84255903 >> v1:172.21.15.115:6789/0 conn(0x7fba980cf9b0 legacy=0x7fba980cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.822+0000 7fba96d9d640 1 -- 172.21.15.115:0/84255903 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba980d11f0 con 0x7fba980cdab0 2023-12-01T01:54:19.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.823+0000 7fba96d9d640 1 --2- 172.21.15.115:0/84255903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 0x7fba980d19f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fba980d3820 tx=0x7fba9002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8be83cc4486373f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9559a640 1 -- 172.21.15.115:0/84255903 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90031830 con 0x7fba980cdab0 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9559a640 1 -- 172.21.15.115:0/84255903 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba90031990 con 0x7fba980cdab0 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9559a640 1 -- 172.21.15.115:0/84255903 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90031d20 con 0x7fba980cdab0 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9759e640 1 -- 172.21.15.115:0/84255903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 msgr2=0x7fba980d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9759e640 1 --2- 172.21.15.115:0/84255903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 0x7fba980d19f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fba980d3820 tx=0x7fba9002b8d0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9759e640 1 -- 172.21.15.115:0/84255903 shutdown_connections 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9759e640 1 --2- 172.21.15.115:0/84255903 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 0x7fba980d19f0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9759e640 1 --2- 172.21.15.115:0/84255903 >> v2:172.21.15.142:3300/0 conn(0x7fba98157360 0x7fba980cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9759e640 1 -- 172.21.15.115:0/84255903 >> 172.21.15.115:0/84255903 conn(0x7fba9806a0b0 msgr2=0x7fba9806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9759e640 1 -- 172.21.15.115:0/84255903 shutdown_connections 2023-12-01T01:54:19.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.824+0000 7fba9759e640 1 -- 172.21.15.115:0/84255903 wait complete. 2023-12-01T01:54:19.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.826+0000 7fba9759e640 1 Processor -- start 2023-12-01T01:54:19.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.826+0000 7fba9759e640 1 -- start start 2023-12-01T01:54:19.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.827+0000 7fba9759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 0x7fba980cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.827+0000 7fba9759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba980cf9b0 0x7fba981f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.827+0000 7fba9759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba98157360 0x7fba980cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.827+0000 7fba9759e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba980ccb60 con 0x7fba980cdab0 2023-12-01T01:54:19.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.827+0000 7fba9759e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba98159930 con 0x7fba980cf9b0 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.827+0000 7fba9759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba981f2010 con 0x7fba98157360 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.827+0000 7fba9659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 0x7fba980cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.827+0000 7fba9659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 0x7fba980cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38916/0 (socket says 172.21.15.115:38916) 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.827+0000 7fba9659c640 1 -- 172.21.15.115:0/4150442870 learned_addr learned my addr 172.21.15.115:0/4150442870 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.828+0000 7fba9659c640 1 -- 172.21.15.115:0/4150442870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba98157360 msgr2=0x7fba980cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.828+0000 7fba9659c640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba98157360 0x7fba980cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.828+0000 7fba9659c640 1 -- 172.21.15.115:0/4150442870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba980cf9b0 msgr2=0x7fba981f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.828+0000 7fba9659c640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba980cf9b0 0x7fba981f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.828+0000 7fba9659c640 1 -- 172.21.15.115:0/4150442870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba90009400 con 0x7fba980cdab0 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.828+0000 7fba9659c640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 0x7fba980cc620 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fba88007ef0 tx=0x7fba8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.828+0000 7fba8effd640 1 -- 172.21.15.115:0/4150442870 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88014620 con 0x7fba980cdab0 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.828+0000 7fba8effd640 1 -- 172.21.15.115:0/4150442870 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba88014780 con 0x7fba980cdab0 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.828+0000 7fba9759e640 1 -- 172.21.15.115:0/4150442870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba98159bd0 con 0x7fba980cdab0 2023-12-01T01:54:19.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.828+0000 7fba8effd640 1 -- 172.21.15.115:0/4150442870 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88014b10 con 0x7fba980cdab0 2023-12-01T01:54:19.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.829+0000 7fba9759e640 1 -- 172.21.15.115:0/4150442870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9815a190 con 0x7fba980cdab0 2023-12-01T01:54:19.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.829+0000 7fba9759e640 1 -- 172.21.15.115:0/4150442870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba980618d0 con 0x7fba980cdab0 2023-12-01T01:54:19.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.830+0000 7fba8effd640 1 -- 172.21.15.115:0/4150442870 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 60) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fba88014c70 con 0x7fba980cdab0 2023-12-01T01:54:19.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.831+0000 7fba8effd640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba7806a4c0 0x7fba7806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:19.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.831+0000 7fba8effd640 1 -- 172.21.15.115:0/4150442870 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fba88089730 con 0x7fba980cdab0 2023-12-01T01:54:19.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.832+0000 7fba95d9b640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba7806a4c0 0x7fba7806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:19.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.832+0000 7fba95d9b640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba7806a4c0 0x7fba7806c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fba9814b800 tx=0x7fba80007a40 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:19.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:19.835+0000 7fba8effd640 1 -- 172.21.15.115:0/4150442870 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fba88054990 con 0x7fba980cdab0 2023-12-01T01:54:20.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.066+0000 7fba9759e640 1 -- 172.21.15.115:0/4150442870 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fba980c5aa0 con 0x7fba980cdab0 2023-12-01T01:54:20.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.596+0000 7fba8effd640 1 -- 172.21.15.115:0/4150442870 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7fba88057fb0 con 0x7fba980cdab0 2023-12-01T01:54:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 -- 172.21.15.115:0/4150442870 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba7806a4c0 msgr2=0x7fba7806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba7806a4c0 0x7fba7806c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fba9814b800 tx=0x7fba80007a40 comp rx=0 tx=0).stop 2023-12-01T01:54:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 -- 172.21.15.115:0/4150442870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 msgr2=0x7fba980cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 0x7fba980cc620 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fba88007ef0 tx=0x7fba8800e4b0 comp rx=0 tx=0).stop 2023-12-01T01:54:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 -- 172.21.15.115:0/4150442870 shutdown_connections 2023-12-01T01:54:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fba7806a4c0 0x7fba7806c980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba98157360 0x7fba980cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba980cf9b0 0x7fba981f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 --2- 172.21.15.115:0/4150442870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba980cdab0 0x7fba980cc620 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 -- 172.21.15.115:0/4150442870 >> 172.21.15.115:0/4150442870 conn(0x7fba9806a0b0 msgr2=0x7fba980d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 -- 172.21.15.115:0/4150442870 shutdown_connections 2023-12-01T01:54:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.601+0000 7fba9759e640 1 -- 172.21.15.115:0/4150442870 wait complete. 2023-12-01T01:54:20.625 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T01:54:20.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.852+0000 7f1fd29d6640 1 Processor -- start 2023-12-01T01:54:20.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.852+0000 7f1fd29d6640 1 -- start start 2023-12-01T01:54:20.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.853+0000 7f1fd29d6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fcc0af370 0x7f1fcc0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:20.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.853+0000 7f1fd29d6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1fcc0c95a0 0x7f1fcc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:20.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.853+0000 7f1fd29d6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc15e440 con 0x7f1fcc1599e0 2023-12-01T01:54:20.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.853+0000 7f1fd29d6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc15d550 con 0x7f1fcc0c95a0 2023-12-01T01:54:20.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.853+0000 7f1fd29d6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15d830 con 0x7f1fcc0af370 2023-12-01T01:54:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.854+0000 7f1fd21d5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1fcc0c95a0 0x7f1fcc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.854+0000 7f1fd11d3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fcc0af370 0x7f1fcc0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.854+0000 7f1fd21d5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1fcc0c95a0 0x7f1fcc070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:59310/0 (socket says 172.21.15.115:59310) 2023-12-01T01:54:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.854+0000 7f1fd21d5640 1 -- 172.21.15.115:0/2543211993 learned_addr learned my addr 172.21.15.115:0/2543211993 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:20.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.855+0000 7f1fd21d5640 1 -- 172.21.15.115:0/2543211993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fcc0af370 msgr2=0x7f1fcc0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:20.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.855+0000 7f1fd21d5640 1 --2- 172.21.15.115:0/2543211993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fcc0af370 0x7f1fcc0c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:20.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.855+0000 7f1fd21d5640 1 -- 172.21.15.115:0/2543211993 >> v1:172.21.15.142:6789/0 conn(0x7f1fcc1599e0 legacy=0x7f1fcc159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:20.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.855+0000 7f1fd21d5640 1 -- 172.21.15.115:0/2543211993 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc15db10 con 0x7f1fcc0c95a0 2023-12-01T01:54:20.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.857+0000 7f1fd21d5640 1 --2- 172.21.15.115:0/2543211993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fcc0c95a0 0x7f1fcc070300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400d150 tx=0x7f1fc402e160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f61032736a9885a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:20.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.857+0000 7f1fd09d2640 1 -- 172.21.15.115:0/2543211993 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400b0d0 con 0x7f1fcc0c95a0 2023-12-01T01:54:20.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.857+0000 7f1fd09d2640 1 -- 172.21.15.115:0/2543211993 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc400b230 con 0x7f1fcc0c95a0 2023-12-01T01:54:20.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.857+0000 7f1fd09d2640 1 -- 172.21.15.115:0/2543211993 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4009600 con 0x7f1fcc0c95a0 2023-12-01T01:54:20.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.858+0000 7f1fd29d6640 1 -- 172.21.15.115:0/2543211993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fcc0c95a0 msgr2=0x7f1fcc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:20.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.858+0000 7f1fd29d6640 1 --2- 172.21.15.115:0/2543211993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fcc0c95a0 0x7f1fcc070300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400d150 tx=0x7f1fc402e160 comp rx=0 tx=0).stop 2023-12-01T01:54:20.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.858+0000 7f1fd29d6640 1 -- 172.21.15.115:0/2543211993 shutdown_connections 2023-12-01T01:54:20.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.858+0000 7f1fd29d6640 1 --2- 172.21.15.115:0/2543211993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fcc0af370 0x7f1fcc0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:20.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.858+0000 7f1fd29d6640 1 --2- 172.21.15.115:0/2543211993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fcc0c95a0 0x7f1fcc070300 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:20.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.858+0000 7f1fd29d6640 1 -- 172.21.15.115:0/2543211993 >> 172.21.15.115:0/2543211993 conn(0x7f1fcc065960 msgr2=0x7f1fcc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:20.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.858+0000 7f1fd29d6640 1 -- 172.21.15.115:0/2543211993 shutdown_connections 2023-12-01T01:54:20.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.858+0000 7f1fd29d6640 1 -- 172.21.15.115:0/2543211993 wait complete. 2023-12-01T01:54:20.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.860+0000 7f1fd29d6640 1 Processor -- start 2023-12-01T01:54:20.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.860+0000 7f1fd29d6640 1 -- start start 2023-12-01T01:54:20.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.861+0000 7f1fd29d6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fcc0af370 0x7f1fcc0c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:20.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.861+0000 7f1fd29d6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fcc0c95a0 0x7f1fcc154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:20.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.861+0000 7f1fd29d6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fcc1599e0 0x7f1fcc155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:20.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.861+0000 7f1fd29d6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15c740 con 0x7f1fcc1599e0 2023-12-01T01:54:20.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.861+0000 7f1fd29d6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15ca20 con 0x7f1fcc0c95a0 2023-12-01T01:54:20.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.861+0000 7f1fd29d6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15cd00 con 0x7f1fcc0af370 2023-12-01T01:54:20.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.862+0000 7f1fd21d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fcc1599e0 0x7f1fcc155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:20.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.862+0000 7f1fd21d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fcc1599e0 0x7f1fcc155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38928/0 (socket says 172.21.15.115:38928) 2023-12-01T01:54:20.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.862+0000 7f1fd21d5640 1 -- 172.21.15.115:0/245289075 learned_addr learned my addr 172.21.15.115:0/245289075 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:20.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.862+0000 7f1fd21d5640 1 -- 172.21.15.115:0/245289075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fcc0af370 msgr2=0x7f1fcc0c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:20.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.862+0000 7f1fd21d5640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fcc0af370 0x7f1fcc0c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.862+0000 7f1fd21d5640 1 -- 172.21.15.115:0/245289075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fcc0c95a0 msgr2=0x7f1fcc154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.862+0000 7f1fd11d3640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fcc0c95a0 0x7f1fcc154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.862+0000 7f1fd21d5640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fcc0c95a0 0x7f1fcc154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.862+0000 7f1fd21d5640 1 -- 172.21.15.115:0/245289075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc400bd40 con 0x7f1fcc1599e0 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.862+0000 7f1fd21d5640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fcc1599e0 0x7f1fcc155150 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4038040 tx=0x7f1fc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.863+0000 7f1fc27fc640 1 -- 172.21.15.115:0/245289075 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4036800 con 0x7f1fcc1599e0 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.863+0000 7f1fc27fc640 1 -- 172.21.15.115:0/245289075 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc4036980 con 0x7f1fcc1599e0 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.863+0000 7f1fc27fc640 1 -- 172.21.15.115:0/245289075 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4036d30 con 0x7f1fcc1599e0 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.863+0000 7f1fd29d6640 1 -- 172.21.15.115:0/245289075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc0c5980 con 0x7f1fcc1599e0 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.863+0000 7f1fd29d6640 1 -- 172.21.15.115:0/245289075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc155e80 con 0x7f1fcc1599e0 2023-12-01T01:54:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.864+0000 7f1fd29d6640 1 -- 172.21.15.115:0/245289075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc0618d0 con 0x7f1fcc1599e0 2023-12-01T01:54:20.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.865+0000 7f1fc27fc640 1 -- 172.21.15.115:0/245289075 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1fc404a460 con 0x7f1fcc1599e0 2023-12-01T01:54:20.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.866+0000 7f1fc27fc640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fb006a580 0x7f1fb006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:20.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.866+0000 7f1fc27fc640 1 -- 172.21.15.115:0/245289075 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f1fc40abfb0 con 0x7f1fcc1599e0 2023-12-01T01:54:20.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.866+0000 7f1fd19d4640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fb006a580 0x7f1fb006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:20.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.866+0000 7f1fd19d4640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fb006a580 0x7f1fb006ca40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8000f60 tx=0x7f1fc80079b0 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:20.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:20.870+0000 7f1fc27fc640 1 -- 172.21.15.115:0/245289075 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1fc40dec80 con 0x7f1fcc1599e0 2023-12-01T01:54:21.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.060+0000 7f1fd29d6640 1 -- 172.21.15.115:0/245289075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1fcc0c5b40 con 0x7f1fcc1599e0 2023-12-01T01:54:21.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.786+0000 7f1fc27fc640 1 -- 172.21.15.115:0/245289075 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1fc4077330 con 0x7f1fcc1599e0 2023-12-01T01:54:21.792 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T01:54:21.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.793+0000 7f1fd29d6640 1 -- 172.21.15.115:0/245289075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fb006a580 msgr2=0x7f1fb006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:21.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.793+0000 7f1fd29d6640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fb006a580 0x7f1fb006ca40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8000f60 tx=0x7f1fc80079b0 comp rx=0 tx=0).stop 2023-12-01T01:54:21.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.793+0000 7f1fd29d6640 1 -- 172.21.15.115:0/245289075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fcc1599e0 msgr2=0x7f1fcc155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:21.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.793+0000 7f1fd29d6640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fcc1599e0 0x7f1fcc155150 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4038040 tx=0x7f1fc4004a40 comp rx=0 tx=0).stop 2023-12-01T01:54:21.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.793+0000 7f1fd29d6640 1 -- 172.21.15.115:0/245289075 shutdown_connections 2023-12-01T01:54:21.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.793+0000 7f1fd29d6640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fb006a580 0x7f1fb006ca40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:21.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.793+0000 7f1fd29d6640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1fcc1599e0 0x7f1fcc155150 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:21.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.793+0000 7f1fd29d6640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1fcc0c95a0 0x7f1fcc154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:21.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.793+0000 7f1fd29d6640 1 --2- 172.21.15.115:0/245289075 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1fcc0af370 0x7f1fcc0c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:21.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.793+0000 7f1fd29d6640 1 -- 172.21.15.115:0/245289075 >> 172.21.15.115:0/245289075 conn(0x7f1fcc065960 msgr2=0x7f1fcc0ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:21.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.794+0000 7f1fd29d6640 1 -- 172.21.15.115:0/245289075 shutdown_connections 2023-12-01T01:54:21.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:21.794+0000 7f1fd29d6640 1 -- 172.21.15.115:0/245289075 wait complete. 2023-12-01T01:54:21.817 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T01:54:22.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.017+0000 7f9c2e2b9640 1 Processor -- start 2023-12-01T01:54:22.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.018+0000 7f9c2e2b9640 1 -- start start 2023-12-01T01:54:22.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.019+0000 7f9c2e2b9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c2815d910 0x7f9c2815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:22.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.019+0000 7f9c2e2b9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9c280cdaa0 0x7f9c280d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:22.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.019+0000 7f9c2e2b9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c280b2300 con 0x7f9c280d2f10 2023-12-01T01:54:22.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.019+0000 7f9c2e2b9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c281612b0 con 0x7f9c280cdaa0 2023-12-01T01:54:22.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.019+0000 7f9c2e2b9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c281616d0 con 0x7f9c2815d910 2023-12-01T01:54:22.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.020+0000 7f9c2cab6640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9c280cdaa0 0x7f9c280d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:22.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.020+0000 7f9c2cab6640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9c280cdaa0 0x7f9c280d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38934/0 (socket says 172.21.15.115:38934) 2023-12-01T01:54:22.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.020+0000 7f9c2cab6640 1 -- 172.21.15.115:0/1115371006 learned_addr learned my addr 172.21.15.115:0/1115371006 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:22.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.020+0000 7f9c2d2b7640 1 --2- 172.21.15.115:0/1115371006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c2815d910 0x7f9c2815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:22.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.020+0000 7f9c2cab6640 1 -- 172.21.15.115:0/1115371006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c2815d910 msgr2=0x7f9c2815dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:22.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.020+0000 7f9c2cab6640 1 --2- 172.21.15.115:0/1115371006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c2815d910 0x7f9c2815dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:22.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.020+0000 7f9c2cab6640 1 -- 172.21.15.115:0/1115371006 >> v1:172.21.15.142:6789/0 conn(0x7f9c280d2f10 legacy=0x7f9c280b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:22.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.020+0000 7f9c2cab6640 1 -- 172.21.15.115:0/1115371006 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c280ce260 con 0x7f9c280cdaa0 2023-12-01T01:54:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.022+0000 7f9c2cab6640 1 --2- 172.21.15.115:0/1115371006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 0x7f9c280d29d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9c140098a0 tx=0x7f9c1402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61f5c1d31368108 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.022+0000 7f9c27fff640 1 -- 172.21.15.115:0/1115371006 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c140317c0 con 0x7f9c280cdaa0 2023-12-01T01:54:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.022+0000 7f9c27fff640 1 -- 172.21.15.115:0/1115371006 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c14031920 con 0x7f9c280cdaa0 2023-12-01T01:54:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.022+0000 7f9c27fff640 1 -- 172.21.15.115:0/1115371006 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14031cb0 con 0x7f9c280cdaa0 2023-12-01T01:54:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.023+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/1115371006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 msgr2=0x7f9c280d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.023+0000 7f9c2e2b9640 1 --2- 172.21.15.115:0/1115371006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 0x7f9c280d29d0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9c140098a0 tx=0x7f9c1402b860 comp rx=0 tx=0).stop 2023-12-01T01:54:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.023+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/1115371006 shutdown_connections 2023-12-01T01:54:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.023+0000 7f9c2e2b9640 1 --2- 172.21.15.115:0/1115371006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 0x7f9c280d29d0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.023+0000 7f9c2e2b9640 1 --2- 172.21.15.115:0/1115371006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c2815d910 0x7f9c2815dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:22.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.023+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/1115371006 >> 172.21.15.115:0/1115371006 conn(0x7f9c2806a0b0 msgr2=0x7f9c2806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:22.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.023+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/1115371006 shutdown_connections 2023-12-01T01:54:22.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.023+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/1115371006 wait complete. 2023-12-01T01:54:22.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.025+0000 7f9c2e2b9640 1 Processor -- start 2023-12-01T01:54:22.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.025+0000 7f9c2e2b9640 1 -- start start 2023-12-01T01:54:22.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.026+0000 7f9c2e2b9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 0x7f9c2814ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:22.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.026+0000 7f9c2e2b9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c280d2f10 0x7f9c2815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:22.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.026+0000 7f9c2e2b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2815d910 0x7f9c2815f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:22.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.026+0000 7f9c2e2b9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c280c6570 con 0x7f9c280cdaa0 2023-12-01T01:54:22.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.026+0000 7f9c2e2b9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c281508e0 con 0x7f9c2815d910 2023-12-01T01:54:22.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.026+0000 7f9c2e2b9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c280cbfe0 con 0x7f9c280d2f10 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.026+0000 7f9c2d2b7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 0x7f9c2814ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.026+0000 7f9c2d2b7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 0x7f9c2814ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38950/0 (socket says 172.21.15.115:38950) 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.026+0000 7f9c2d2b7640 1 -- 172.21.15.115:0/3912707501 learned_addr learned my addr 172.21.15.115:0/3912707501 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2cab6640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c280d2f10 0x7f9c2815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2dab8640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2815d910 0x7f9c2815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2d2b7640 1 -- 172.21.15.115:0/3912707501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c280d2f10 msgr2=0x7f9c2815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2d2b7640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c280d2f10 0x7f9c2815f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2d2b7640 1 -- 172.21.15.115:0/3912707501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2815d910 msgr2=0x7f9c2815f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2d2b7640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2815d910 0x7f9c2815f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2d2b7640 1 -- 172.21.15.115:0/3912707501 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c140094d0 con 0x7f9c280cdaa0 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2cab6640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c280d2f10 0x7f9c2815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2dab8640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2815d910 0x7f9c2815f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2d2b7640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 0x7f9c2814ce60 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009d10 tx=0x7f9c1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c25ffb640 1 -- 172.21.15.115:0/3912707501 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c009b80 con 0x7f9c280cdaa0 2023-12-01T01:54:22.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c25ffb640 1 -- 172.21.15.115:0/3912707501 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c00f040 con 0x7f9c280cdaa0 2023-12-01T01:54:22.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c25ffb640 1 -- 172.21.15.115:0/3912707501 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c014680 con 0x7f9c280cdaa0 2023-12-01T01:54:22.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.027+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/3912707501 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c280cc2c0 con 0x7f9c280cdaa0 2023-12-01T01:54:22.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.028+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/3912707501 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c280c6750 con 0x7f9c280cdaa0 2023-12-01T01:54:22.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.028+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/3912707501 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c280cc7e0 con 0x7f9c280cdaa0 2023-12-01T01:54:22.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.029+0000 7f9c25ffb640 1 -- 172.21.15.115:0/3912707501 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9c1c0147e0 con 0x7f9c280cdaa0 2023-12-01T01:54:22.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.030+0000 7f9c25ffb640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9c0c06a550 0x7f9c0c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:22.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.031+0000 7f9c25ffb640 1 -- 172.21.15.115:0/3912707501 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f9c1c089800 con 0x7f9c280cdaa0 2023-12-01T01:54:22.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.031+0000 7f9c2cab6640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9c0c06a550 0x7f9c0c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:22.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.031+0000 7f9c2cab6640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9c0c06a550 0x7f9c0c06ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c14036000 tx=0x7f9c1403d040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:22.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.035+0000 7f9c25ffb640 1 -- 172.21.15.115:0/3912707501 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9c1c054a70 con 0x7f9c280cdaa0 2023-12-01T01:54:22.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.222+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/3912707501 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9c280618d0 con 0x7f9c280cdaa0 2023-12-01T01:54:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.792+0000 7f9c25ffb640 1 -- 172.21.15.115:0/3912707501 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9c1c058090 con 0x7f9c280cdaa0 2023-12-01T01:54:22.794 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.796+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/3912707501 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9c0c06a550 msgr2=0x7f9c0c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.796+0000 7f9c2e2b9640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9c0c06a550 0x7f9c0c06ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c14036000 tx=0x7f9c1403d040 comp rx=0 tx=0).stop 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.797+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/3912707501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 msgr2=0x7f9c2814ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.797+0000 7f9c2e2b9640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 0x7f9c2814ce60 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009d10 tx=0x7f9c1c00c490 comp rx=0 tx=0).stop 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.797+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/3912707501 shutdown_connections 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.797+0000 7f9c2e2b9640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9c0c06a550 0x7f9c0c06ca10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.797+0000 7f9c2e2b9640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c280d2f10 0x7f9c2815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.797+0000 7f9c2e2b9640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2815d910 0x7f9c2815f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.797+0000 7f9c2e2b9640 1 --2- 172.21.15.115:0/3912707501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c280cdaa0 0x7f9c2814ce60 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.797+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/3912707501 >> 172.21.15.115:0/3912707501 conn(0x7f9c2806a0b0 msgr2=0x7f9c280d1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.797+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/3912707501 shutdown_connections 2023-12-01T01:54:22.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:22.797+0000 7f9c2e2b9640 1 -- 172.21.15.115:0/3912707501 wait complete. 2023-12-01T01:54:22.819 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T01:54:23.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.032+0000 7f0854dab640 1 Processor -- start 2023-12-01T01:54:23.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.033+0000 7f0854dab640 1 -- start start 2023-12-01T01:54:23.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.034+0000 7f0854dab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0850143eb0 0x7f085006b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.034+0000 7f0854dab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f08500cbc80 0x7f0850164c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.034+0000 7f0854dab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08500ce1b0 con 0x7f085006b7a0 2023-12-01T01:54:23.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.034+0000 7f0854dab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08500ce490 con 0x7f0850143eb0 2023-12-01T01:54:23.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.034+0000 7f0854dab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08500caa90 con 0x7f08500cbc80 2023-12-01T01:54:23.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.034+0000 7f084e59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0850143eb0 0x7f085006b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.035+0000 7f084e59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0850143eb0 0x7f085006b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38958/0 (socket says 172.21.15.115:38958) 2023-12-01T01:54:23.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.035+0000 7f084e59c640 1 -- 172.21.15.115:0/1107788046 learned_addr learned my addr 172.21.15.115:0/1107788046 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.035+0000 7f084e59c640 1 -- 172.21.15.115:0/1107788046 >> v2:172.21.15.142:3300/0 conn(0x7f08500cbc80 msgr2=0x7f0850164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.035+0000 7f084e59c640 1 --2- 172.21.15.115:0/1107788046 >> v2:172.21.15.142:3300/0 conn(0x7f08500cbc80 0x7f0850164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.035+0000 7f084e59c640 1 -- 172.21.15.115:0/1107788046 >> v1:172.21.15.142:6789/0 conn(0x7f085006b7a0 legacy=0x7f08500cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.035+0000 7f084e59c640 1 -- 172.21.15.115:0/1107788046 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08500cad70 con 0x7f0850143eb0 2023-12-01T01:54:23.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.036+0000 7f084e59c640 1 --2- 172.21.15.115:0/1107788046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0850143eb0 0x7f085006b260 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f08500668d0 tx=0x7f084002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca0ad00fef9e44e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:23.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.036+0000 7f084dd9b640 1 -- 172.21.15.115:0/1107788046 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840031870 con 0x7f0850143eb0 2023-12-01T01:54:23.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.036+0000 7f084dd9b640 1 -- 172.21.15.115:0/1107788046 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08400319d0 con 0x7f0850143eb0 2023-12-01T01:54:23.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.036+0000 7f084dd9b640 1 -- 172.21.15.115:0/1107788046 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840031d60 con 0x7f0850143eb0 2023-12-01T01:54:23.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.037+0000 7f0854dab640 1 -- 172.21.15.115:0/1107788046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0850143eb0 msgr2=0x7f085006b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.037+0000 7f0854dab640 1 --2- 172.21.15.115:0/1107788046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0850143eb0 0x7f085006b260 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f08500668d0 tx=0x7f084002b910 comp rx=0 tx=0).stop 2023-12-01T01:54:23.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.037+0000 7f0854dab640 1 -- 172.21.15.115:0/1107788046 shutdown_connections 2023-12-01T01:54:23.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.037+0000 7f0854dab640 1 --2- 172.21.15.115:0/1107788046 >> v2:172.21.15.142:3300/0 conn(0x7f08500cbc80 0x7f0850164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.037+0000 7f0854dab640 1 --2- 172.21.15.115:0/1107788046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0850143eb0 0x7f085006b260 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.037+0000 7f0854dab640 1 -- 172.21.15.115:0/1107788046 >> 172.21.15.115:0/1107788046 conn(0x7f0850065960 msgr2=0x7f0850065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:23.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.037+0000 7f0854dab640 1 -- 172.21.15.115:0/1107788046 shutdown_connections 2023-12-01T01:54:23.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.037+0000 7f0854dab640 1 -- 172.21.15.115:0/1107788046 wait complete. 2023-12-01T01:54:23.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.039+0000 7f0854dab640 1 Processor -- start 2023-12-01T01:54:23.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.039+0000 7f0854dab640 1 -- start start 2023-12-01T01:54:23.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.040+0000 7f0854dab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f085006b7a0 0x7f085015de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.040+0000 7f0854dab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0850143eb0 0x7f08501f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.040+0000 7f0854dab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08501572f0 0x7f085015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.040+0000 7f0854dab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085015c5f0 con 0x7f08501572f0 2023-12-01T01:54:23.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.040+0000 7f0854dab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08501f79d0 con 0x7f0850143eb0 2023-12-01T01:54:23.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.040+0000 7f0854dab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085015eac0 con 0x7f085006b7a0 2023-12-01T01:54:23.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.040+0000 7f084ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08501572f0 0x7f085015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.040+0000 7f084ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08501572f0 0x7f085015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38970/0 (socket says 172.21.15.115:38970) 2023-12-01T01:54:23.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.040+0000 7f084ed9d640 1 -- 172.21.15.115:0/2383659071 learned_addr learned my addr 172.21.15.115:0/2383659071 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:23.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f0847fff640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0850143eb0 0x7f08501f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f084e59c640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f085006b7a0 0x7f085015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f084ed9d640 1 -- 172.21.15.115:0/2383659071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f085006b7a0 msgr2=0x7f085015de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f084ed9d640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f085006b7a0 0x7f085015de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f084ed9d640 1 -- 172.21.15.115:0/2383659071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0850143eb0 msgr2=0x7f08501f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f084ed9d640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0850143eb0 0x7f08501f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f084ed9d640 1 -- 172.21.15.115:0/2383659071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08400094d0 con 0x7f08501572f0 2023-12-01T01:54:23.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f084ed9d640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08501572f0 0x7f085015ca10 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f084800b970 tx=0x7f084800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:23.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f0846ffd640 1 -- 172.21.15.115:0/2383659071 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084800cb40 con 0x7f08501572f0 2023-12-01T01:54:23.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f0846ffd640 1 -- 172.21.15.115:0/2383659071 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084800ccc0 con 0x7f08501572f0 2023-12-01T01:54:23.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f0846ffd640 1 -- 172.21.15.115:0/2383659071 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848014680 con 0x7f08501572f0 2023-12-01T01:54:23.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.041+0000 7f0854dab640 1 -- 172.21.15.115:0/2383659071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085015e500 con 0x7f08501572f0 2023-12-01T01:54:23.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.042+0000 7f0854dab640 1 -- 172.21.15.115:0/2383659071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085015c210 con 0x7f08501572f0 2023-12-01T01:54:23.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.043+0000 7f0854dab640 1 -- 172.21.15.115:0/2383659071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085015c730 con 0x7f08501572f0 2023-12-01T01:54:23.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.044+0000 7f0846ffd640 1 -- 172.21.15.115:0/2383659071 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f084801d3b0 con 0x7f08501572f0 2023-12-01T01:54:23.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.044+0000 7f0846ffd640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f083006a550 0x7f083006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.045+0000 7f0846ffd640 1 -- 172.21.15.115:0/2383659071 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f08480889a0 con 0x7f08501572f0 2023-12-01T01:54:23.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.045+0000 7f084e59c640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f083006a550 0x7f083006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.046+0000 7f084e59c640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f083006a550 0x7f083006ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0840036000 tx=0x7f084003c040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:23.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.049+0000 7f0846ffd640 1 -- 172.21.15.115:0/2383659071 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f085015c730 con 0x7f08501572f0 2023-12-01T01:54:23.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.237+0000 7f0854dab640 1 -- 172.21.15.115:0/2383659071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f085015c730 con 0x7f08501572f0 2023-12-01T01:54:23.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.243+0000 7f0846ffd640 1 -- 172.21.15.115:0/2383659071 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f085015c730 con 0x7f08501572f0 2023-12-01T01:54:23.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 -- 172.21.15.115:0/2383659071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f083006a550 msgr2=0x7f083006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f083006a550 0x7f083006ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0840036000 tx=0x7f084003c040 comp rx=0 tx=0).stop 2023-12-01T01:54:23.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 -- 172.21.15.115:0/2383659071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08501572f0 msgr2=0x7f085015ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08501572f0 0x7f085015ca10 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f084800b970 tx=0x7f084800be30 comp rx=0 tx=0).stop 2023-12-01T01:54:23.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 -- 172.21.15.115:0/2383659071 shutdown_connections 2023-12-01T01:54:23.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f083006a550 0x7f083006ca10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08501572f0 0x7f085015ca10 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0850143eb0 0x7f08501f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 --2- 172.21.15.115:0/2383659071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f085006b7a0 0x7f085015de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 -- 172.21.15.115:0/2383659071 >> 172.21.15.115:0/2383659071 conn(0x7f0850065960 msgr2=0x7f0850157bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:23.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 -- 172.21.15.115:0/2383659071 shutdown_connections 2023-12-01T01:54:23.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.246+0000 7f0854dab640 1 -- 172.21.15.115:0/2383659071 wait complete. 2023-12-01T01:54:23.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T01:54:23.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.477+0000 7fa174b9f640 1 Processor -- start 2023-12-01T01:54:23.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.477+0000 7fa174b9f640 1 -- start start 2023-12-01T01:54:23.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.479+0000 7fa174b9f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa17015de10 0x7fa1700c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.479+0000 7fa174b9f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1700ca670 con 0x7fa1700ae6e0 2023-12-01T01:54:23.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.479+0000 7fa174b9f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1700ca390 con 0x7fa17015e1f0 2023-12-01T01:54:23.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.479+0000 7fa174b9f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa170160c20 con 0x7fa17015de10 2023-12-01T01:54:23.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.479+0000 7fa16ed9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa1700ae6e0 0x7fa1700ccfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59892/0 (socket says 172.21.15.115:59892) 2023-12-01T01:54:23.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.479+0000 7fa16ed9d640 1 -- 172.21.15.115:0/3341920405 learned_addr learned my addr 172.21.15.115:0/3341920405 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:23.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.479+0000 7fa16e59c640 1 --2- 172.21.15.115:0/3341920405 >> v2:172.21.15.142:3300/0 conn(0x7fa17015de10 0x7fa1700c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934244423 0 0) 0x7fa1700ca670 con 0x7fa1700ae6e0 2023-12-01T01:54:23.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa15c003520 con 0x7fa1700ae6e0 2023-12-01T01:54:23.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1350275783 0 0) 0x7fa15c003520 con 0x7fa1700ae6e0 2023-12-01T01:54:23.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 >> v2:172.21.15.142:3300/0 conn(0x7fa17015de10 msgr2=0x7fa1700c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:23.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 --2- 172.21.15.115:0/3341920405 >> v2:172.21.15.142:3300/0 conn(0x7fa17015de10 0x7fa1700c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 >> v1:172.21.15.142:6789/0 conn(0x7fa17015e1f0 legacy=0x7fa1700adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa170160f00 con 0x7fa1700ae6e0 2023-12-01T01:54:23.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa168001670 con 0x7fa1700ae6e0 2023-12-01T01:54:23.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 >> v1:172.21.15.115:6789/0 conn(0x7fa1700ae6e0 legacy=0x7fa1700ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 --2- 172.21.15.115:0/3341920405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa15c003970 0x7fa15c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 --2- 172.21.15.115:0/3341920405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa15c0047c0 0x7fa15c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 --2- 172.21.15.115:0/3341920405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa15c005180 0x7fa15c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa170160c20 con 0x7fa15c005180 2023-12-01T01:54:23.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c00c020 con 0x7fa15c003970 2023-12-01T01:54:23.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.480+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c00c1e0 con 0x7fa15c0047c0 2023-12-01T01:54:23.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.481+0000 7fa16ed9d640 1 --2- 172.21.15.115:0/3341920405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa15c005180 0x7fa15c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.481+0000 7fa16ed9d640 1 -- 172.21.15.115:0/3341920405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa15c0047c0 msgr2=0x7fa15c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:23.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.481+0000 7fa16e59c640 1 --2- 172.21.15.115:0/3341920405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa15c003970 0x7fa15c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.481+0000 7fa16ed9d640 1 --2- 172.21.15.115:0/3341920405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa15c0047c0 0x7fa15c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.481+0000 7fa16ed9d640 1 -- 172.21.15.115:0/3341920405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa15c003970 msgr2=0x7fa15c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:23.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.481+0000 7fa16ed9d640 1 --2- 172.21.15.115:0/3341920405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa15c003970 0x7fa15c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.481+0000 7fa16ed9d640 1 -- 172.21.15.115:0/3341920405 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c003520 con 0x7fa15c005180 2023-12-01T01:54:23.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.482+0000 7fa16ed9d640 1 -- 172.21.15.115:0/3341920405 reap_dead start 2023-12-01T01:54:23.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.482+0000 7fa16ed9d640 1 --2- 172.21.15.115:0/3341920405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa15c005180 0x7fa15c00b980 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fa16800c240 tx=0x7fa15c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4361833de059b3fd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:23.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.482+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168005ad0 con 0x7fa15c005180 2023-12-01T01:54:23.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.482+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa168005c70 con 0x7fa15c005180 2023-12-01T01:54:23.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.482+0000 7fa16d59a640 1 -- 172.21.15.115:0/3341920405 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16802c8b0 con 0x7fa15c005180 2023-12-01T01:54:23.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.483+0000 7fa174b9f640 1 -- 172.21.15.115:0/3341920405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa15c005180 msgr2=0x7fa15c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.483+0000 7fa174b9f640 1 --2- 172.21.15.115:0/3341920405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa15c005180 0x7fa15c00b980 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fa16800c240 tx=0x7fa15c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.483+0000 7fa174b9f640 1 -- 172.21.15.115:0/3341920405 shutdown_connections 2023-12-01T01:54:23.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.483+0000 7fa174b9f640 1 --2- 172.21.15.115:0/3341920405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa15c005180 0x7fa15c00b980 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.483+0000 7fa174b9f640 1 -- 172.21.15.115:0/3341920405 >> 172.21.15.115:0/3341920405 conn(0x7fa1700cf0b0 msgr2=0x7fa1700cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:23.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.483+0000 7fa174b9f640 1 -- 172.21.15.115:0/3341920405 shutdown_connections 2023-12-01T01:54:23.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.483+0000 7fa174b9f640 1 -- 172.21.15.115:0/3341920405 wait complete. 2023-12-01T01:54:23.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.485+0000 7fa174b9f640 1 Processor -- start 2023-12-01T01:54:23.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.485+0000 7fa174b9f640 1 -- start start 2023-12-01T01:54:23.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.486+0000 7fa174b9f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa15c005180 0x7fa1701f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.486+0000 7fa174b9f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1700c2be0 0x7fa1700c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.486+0000 7fa174b9f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1700c6180 0x7fa1700c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.486+0000 7fa174b9f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700c30e0 con 0x7fa1700c2be0 2023-12-01T01:54:23.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.486+0000 7fa174b9f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700c33c0 con 0x7fa1700c6180 2023-12-01T01:54:23.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.486+0000 7fa174b9f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700c36a0 con 0x7fa15c005180 2023-12-01T01:54:23.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.486+0000 7fa16dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1700c2be0 0x7fa1700c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.486+0000 7fa16dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1700c2be0 0x7fa1700c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38988/0 (socket says 172.21.15.115:38988) 2023-12-01T01:54:23.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.486+0000 7fa16dd9b640 1 -- 172.21.15.115:0/1786060335 learned_addr learned my addr 172.21.15.115:0/1786060335 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:23.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.486+0000 7fa16ed9d640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1700c6180 0x7fa1700c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa16e59c640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa15c005180 0x7fa1701f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa16dd9b640 1 -- 172.21.15.115:0/1786060335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa15c005180 msgr2=0x7fa1701f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa16dd9b640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa15c005180 0x7fa1701f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa16dd9b640 1 -- 172.21.15.115:0/1786060335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1700c6180 msgr2=0x7fa1700c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa16dd9b640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1700c6180 0x7fa1700c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa16dd9b640 1 -- 172.21.15.115:0/1786060335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1700c3980 con 0x7fa1700c2be0 2023-12-01T01:54:23.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa16dd9b640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1700c2be0 0x7fa1700c96e0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c450 tx=0x7fa15800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:23.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa166ffd640 1 -- 172.21.15.115:0/1786060335 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1580052d0 con 0x7fa1700c2be0 2023-12-01T01:54:23.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa166ffd640 1 -- 172.21.15.115:0/1786060335 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158005450 con 0x7fa1700c2be0 2023-12-01T01:54:23.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa166ffd640 1 -- 172.21.15.115:0/1786060335 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158007260 con 0x7fa1700c2be0 2023-12-01T01:54:23.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa174b9f640 1 -- 172.21.15.115:0/1786060335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1700c3c60 con 0x7fa1700c2be0 2023-12-01T01:54:23.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.487+0000 7fa174b9f640 1 -- 172.21.15.115:0/1786060335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1700c4180 con 0x7fa1700c2be0 2023-12-01T01:54:23.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.488+0000 7fa174b9f640 1 -- 172.21.15.115:0/1786060335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17004e890 con 0x7fa1700c2be0 2023-12-01T01:54:23.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.489+0000 7fa166ffd640 1 -- 172.21.15.115:0/1786060335 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa1580073c0 con 0x7fa1700c2be0 2023-12-01T01:54:23.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.491+0000 7fa166ffd640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa15006a580 0x7fa15006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.491+0000 7fa166ffd640 1 -- 172.21.15.115:0/1786060335 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fa1580882c0 con 0x7fa1700c2be0 2023-12-01T01:54:23.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.491+0000 7fa16e59c640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa15006a580 0x7fa15006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.491+0000 7fa16e59c640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa15006a580 0x7fa15006ca40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa160003a50 tx=0x7fa16000b040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:23.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.495+0000 7fa166ffd640 1 -- 172.21.15.115:0/1786060335 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa1580537e0 con 0x7fa1700c2be0 2023-12-01T01:54:23.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.685+0000 7fa174b9f640 1 -- 172.21.15.115:0/1786060335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fa170073170 con 0x7fa1700c2be0 2023-12-01T01:54:23.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.691+0000 7fa166ffd640 1 -- 172.21.15.115:0/1786060335 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v38) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa158056e00 con 0x7fa1700c2be0 2023-12-01T01:54:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 -- 172.21.15.115:0/1786060335 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa15006a580 msgr2=0x7fa15006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa15006a580 0x7fa15006ca40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa160003a50 tx=0x7fa16000b040 comp rx=0 tx=0).stop 2023-12-01T01:54:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 -- 172.21.15.115:0/1786060335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1700c2be0 msgr2=0x7fa1700c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1700c2be0 0x7fa1700c96e0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c450 tx=0x7fa15800c910 comp rx=0 tx=0).stop 2023-12-01T01:54:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 -- 172.21.15.115:0/1786060335 shutdown_connections 2023-12-01T01:54:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa15006a580 0x7fa15006ca40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1700c6180 0x7fa1700c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1700c2be0 0x7fa1700c96e0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 --2- 172.21.15.115:0/1786060335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa15c005180 0x7fa1701f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 -- 172.21.15.115:0/1786060335 >> 172.21.15.115:0/1786060335 conn(0x7fa1700cf0b0 msgr2=0x7fa17006c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:23.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 -- 172.21.15.115:0/1786060335 shutdown_connections 2023-12-01T01:54:23.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.695+0000 7fa174b9f640 1 -- 172.21.15.115:0/1786060335 wait complete. 2023-12-01T01:54:23.722 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T01:54:23.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.942+0000 7f679205f640 1 Processor -- start 2023-12-01T01:54:23.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.943+0000 7f679205f640 1 -- start start 2023-12-01T01:54:23.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.944+0000 7f679205f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f678c06b900 0x7f678c0cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.944+0000 7f679205f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c0d07b0 con 0x7f678c0ccca0 2023-12-01T01:54:23.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.944+0000 7f679205f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c06af00 con 0x7f678c0d0c40 2023-12-01T01:54:23.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.944+0000 7f679205f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c0d0160 con 0x7f678c06b900 2023-12-01T01:54:23.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.944+0000 7f678b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f678c0ccca0 0x7f678c06a6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59908/0 (socket says 172.21.15.115:59908) 2023-12-01T01:54:23.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.944+0000 7f678b7fe640 1 -- 172.21.15.115:0/3320657232 learned_addr learned my addr 172.21.15.115:0/3320657232 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:23.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678affd640 1 --2- 172.21.15.115:0/3320657232 >> v2:172.21.15.142:3300/0 conn(0x7f678c06b900 0x7f678c0cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669809810 0 0) 0x7f678c0d07b0 con 0x7f678c0ccca0 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6778003520 con 0x7f678c0ccca0 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3918435449 0 0) 0x7f6778003520 con 0x7f678c0ccca0 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 >> v2:172.21.15.142:3300/0 conn(0x7f678c06b900 msgr2=0x7f678c0cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 --2- 172.21.15.115:0/3320657232 >> v2:172.21.15.142:3300/0 conn(0x7f678c06b900 0x7f678c0cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 >> v1:172.21.15.142:6789/0 conn(0x7f678c0d0c40 legacy=0x7f678c0d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c0d0440 con 0x7f678c0ccca0 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f677c0019f0 con 0x7f678c0ccca0 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 >> v1:172.21.15.115:6789/0 conn(0x7f678c0ccca0 legacy=0x7f678c06a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 --2- 172.21.15.115:0/3320657232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67780039c0 0x7f6778003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 --2- 172.21.15.115:0/3320657232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67780047c0 0x7f6778004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 --2- 172.21.15.115:0/3320657232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f677800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0d0160 con 0x7f6778005180 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677800c080 con 0x7f67780039c0 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.945+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677800c240 con 0x7f67780047c0 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.946+0000 7f678bfff640 1 --2- 172.21.15.115:0/3320657232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f677800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.946+0000 7f678b7fe640 1 --2- 172.21.15.115:0/3320657232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67780039c0 0x7f6778003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.946+0000 7f678bfff640 1 -- 172.21.15.115:0/3320657232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67780047c0 msgr2=0x7f6778004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.946+0000 7f678bfff640 1 --2- 172.21.15.115:0/3320657232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67780047c0 0x7f6778004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.946+0000 7f678bfff640 1 -- 172.21.15.115:0/3320657232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67780039c0 msgr2=0x7f6778003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.946+0000 7f678bfff640 1 --2- 172.21.15.115:0/3320657232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67780039c0 0x7f6778003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.946+0000 7f678bfff640 1 -- 172.21.15.115:0/3320657232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778003520 con 0x7f6778005180 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.947+0000 7f678bfff640 1 -- 172.21.15.115:0/3320657232 reap_dead start 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.947+0000 7f678bfff640 1 --2- 172.21.15.115:0/3320657232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f677800b980 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f6784001750 tx=0x7f67780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e92fec40ee965c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.947+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840052d0 con 0x7f6778005180 2023-12-01T01:54:23.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.948+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678402e050 con 0x7f6778005180 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.948+0000 7f678a7fc640 1 -- 172.21.15.115:0/3320657232 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840035f0 con 0x7f6778005180 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.948+0000 7f679205f640 1 -- 172.21.15.115:0/3320657232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 msgr2=0x7f677800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.948+0000 7f679205f640 1 --2- 172.21.15.115:0/3320657232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f677800b980 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f6784001750 tx=0x7f67780057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.948+0000 7f679205f640 1 -- 172.21.15.115:0/3320657232 shutdown_connections 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.948+0000 7f679205f640 1 --2- 172.21.15.115:0/3320657232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f677800b980 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.948+0000 7f679205f640 1 -- 172.21.15.115:0/3320657232 >> 172.21.15.115:0/3320657232 conn(0x7f678c0713d0 msgr2=0x7f678c0717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.948+0000 7f679205f640 1 -- 172.21.15.115:0/3320657232 shutdown_connections 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.948+0000 7f679205f640 1 -- 172.21.15.115:0/3320657232 wait complete. 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.950+0000 7f679205f640 1 Processor -- start 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.950+0000 7f679205f640 1 -- start start 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.951+0000 7f679205f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f678c0c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.951+0000 7f679205f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f678c0b47e0 0x7f678c1f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.951+0000 7f679205f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f678c069f00 0x7f678c06a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.951+0000 7f679205f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0b3f70 con 0x7f6778005180 2023-12-01T01:54:23.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.951+0000 7f679205f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c06a8c0 con 0x7f678c069f00 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.951+0000 7f679205f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0ca430 con 0x7f678c0b47e0 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.951+0000 7f678b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f678c0c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.951+0000 7f678b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f678c0c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39016/0 (socket says 172.21.15.115:39016) 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.951+0000 7f678b7fe640 1 -- 172.21.15.115:0/1325777667 learned_addr learned my addr 172.21.15.115:0/1325777667 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f678b7fe640 1 -- 172.21.15.115:0/1325777667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f678c0b47e0 msgr2=0x7f678c1f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f678b7fe640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f678c0b47e0 0x7f678c1f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f678b7fe640 1 -- 172.21.15.115:0/1325777667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f678c069f00 msgr2=0x7f678c06a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f678b7fe640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f678c069f00 0x7f678c06a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f678b7fe640 1 -- 172.21.15.115:0/1325777667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c0ca160 con 0x7f6778005180 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f678b7fe640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f678c0c9c20 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f677c0079e0 tx=0x7f677c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f6783fff640 1 -- 172.21.15.115:0/1325777667 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c007ad0 con 0x7f6778005180 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f6783fff640 1 -- 172.21.15.115:0/1325777667 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c00ee50 con 0x7f6778005180 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f679205f640 1 -- 172.21.15.115:0/1325777667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c0b4390 con 0x7f6778005180 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f6783fff640 1 -- 172.21.15.115:0/1325777667 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c013640 con 0x7f6778005180 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.952+0000 7f679205f640 1 -- 172.21.15.115:0/1325777667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c0b38b0 con 0x7f6778005180 2023-12-01T01:54:23.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.954+0000 7f679205f640 1 -- 172.21.15.115:0/1325777667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c06be50 con 0x7f6778005180 2023-12-01T01:54:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.955+0000 7f6783fff640 1 -- 172.21.15.115:0/1325777667 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f677c0137a0 con 0x7f6778005180 2023-12-01T01:54:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.955+0000 7f6783fff640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f676c06a550 0x7f676c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.956+0000 7f6783fff640 1 -- 172.21.15.115:0/1325777667 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f677c089130 con 0x7f6778005180 2023-12-01T01:54:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.956+0000 7f678affd640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f676c06a550 0x7f676c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:23.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.956+0000 7f678affd640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f676c06a550 0x7f676c06ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6774006a90 tx=0x7f6774006a20 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:23.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:23.960+0000 7f6783fff640 1 -- 172.21.15.115:0/1325777667 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f677c054700 con 0x7f6778005180 2023-12-01T01:54:24.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.148+0000 7f679205f640 1 -- 172.21.15.115:0/1325777667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f678c04e890 con 0x7f6778005180 2023-12-01T01:54:24.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.155+0000 7f6783fff640 1 -- 172.21.15.115:0/1325777667 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v39) v1 ==== 115+0+0 (secure 0 0 0) 0x7f677c057d20 con 0x7f6778005180 2023-12-01T01:54:24.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.158+0000 7f679205f640 1 -- 172.21.15.115:0/1325777667 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f676c06a550 msgr2=0x7f676c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:24.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.158+0000 7f679205f640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f676c06a550 0x7f676c06ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6774006a90 tx=0x7f6774006a20 comp rx=0 tx=0).stop 2023-12-01T01:54:24.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.158+0000 7f679205f640 1 -- 172.21.15.115:0/1325777667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 msgr2=0x7f678c0c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:24.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.158+0000 7f679205f640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f678c0c9c20 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f677c0079e0 tx=0x7f677c00e530 comp rx=0 tx=0).stop 2023-12-01T01:54:24.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.158+0000 7f679205f640 1 -- 172.21.15.115:0/1325777667 shutdown_connections 2023-12-01T01:54:24.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.158+0000 7f679205f640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f676c06a550 0x7f676c06ca10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.158+0000 7f679205f640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f678c0b47e0 0x7f678c1f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.158+0000 7f679205f640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f678c069f00 0x7f678c06a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.158+0000 7f679205f640 1 --2- 172.21.15.115:0/1325777667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6778005180 0x7f678c0c9c20 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.158+0000 7f679205f640 1 -- 172.21.15.115:0/1325777667 >> 172.21.15.115:0/1325777667 conn(0x7f678c0713d0 msgr2=0x7f678c0cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:24.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.159+0000 7f679205f640 1 -- 172.21.15.115:0/1325777667 shutdown_connections 2023-12-01T01:54:24.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.159+0000 7f679205f640 1 -- 172.21.15.115:0/1325777667 wait complete. 2023-12-01T01:54:24.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-12-01T01:54:24.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.393+0000 7f6bc84ab640 1 Processor -- start 2023-12-01T01:54:24.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.393+0000 7f6bc84ab640 1 -- start start 2023-12-01T01:54:24.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc84ab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6bc00c9ef0 0x7f6bc006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:24.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc84ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 0x7f6bc0070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:24.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc84ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc006f8b0 con 0x7f6bc0159dd0 2023-12-01T01:54:24.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc84ab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc00adcb0 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc84ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00ad9d0 con 0x7f6bc006b6f0 2023-12-01T01:54:24.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc5a46640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6bc00c9ef0 0x7f6bc006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:24.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc5a46640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6bc00c9ef0 0x7f6bc006fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39020/0 (socket says 172.21.15.115:39020) 2023-12-01T01:54:24.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc5a46640 1 -- 172.21.15.115:0/699346858 learned_addr learned my addr 172.21.15.115:0/699346858 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:24.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc6a48640 1 --2- 172.21.15.115:0/699346858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 0x7f6bc0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:24.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc5a46640 1 -- 172.21.15.115:0/699346858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 msgr2=0x7f6bc0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:24.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc5a46640 1 --2- 172.21.15.115:0/699346858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 0x7f6bc0070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc5a46640 1 -- 172.21.15.115:0/699346858 >> v1:172.21.15.142:6789/0 conn(0x7f6bc0159dd0 legacy=0x7f6bc00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:24.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc5a46640 1 -- 172.21.15.115:0/699346858 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00c9000 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.395+0000 7f6bc6a48640 1 --2- 172.21.15.115:0/699346858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 0x7f6bc0070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.397+0000 7f6bc5a46640 1 --2- 172.21.15.115:0/699346858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 0x7f6bc006fd20 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009870 tx=0x7f6bb002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba1e04792927a78 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.397+0000 7f6bc5245640 1 -- 172.21.15.115:0/699346858 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0031740 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.397+0000 7f6bc5245640 1 -- 172.21.15.115:0/699346858 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb00318a0 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.397+0000 7f6bc5245640 1 -- 172.21.15.115:0/699346858 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0031c30 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.397+0000 7f6bc84ab640 1 -- 172.21.15.115:0/699346858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 msgr2=0x7f6bc006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.397+0000 7f6bc84ab640 1 --2- 172.21.15.115:0/699346858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 0x7f6bc006fd20 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009870 tx=0x7f6bb002b7e0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.398+0000 7f6bc84ab640 1 -- 172.21.15.115:0/699346858 shutdown_connections 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.398+0000 7f6bc84ab640 1 --2- 172.21.15.115:0/699346858 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 0x7f6bc0070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.398+0000 7f6bc84ab640 1 --2- 172.21.15.115:0/699346858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 0x7f6bc006fd20 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.398+0000 7f6bc84ab640 1 -- 172.21.15.115:0/699346858 >> 172.21.15.115:0/699346858 conn(0x7f6bc00ce880 msgr2=0x7f6bc00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:24.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.398+0000 7f6bc84ab640 1 -- 172.21.15.115:0/699346858 shutdown_connections 2023-12-01T01:54:24.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.398+0000 7f6bc84ab640 1 -- 172.21.15.115:0/699346858 wait complete. 2023-12-01T01:54:24.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.399+0000 7f6bc84ab640 1 Processor -- start 2023-12-01T01:54:24.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.399+0000 7f6bc84ab640 1 -- start start 2023-12-01T01:54:24.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.400+0000 7f6bc84ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 0x7f6bc0156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:24.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.400+0000 7f6bc84ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 0x7f6bc01e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.400+0000 7f6bc84ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bc0159dd0 0x7f6bc00cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.400+0000 7f6bc84ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc015ad00 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.400+0000 7f6bc84ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc015afe0 con 0x7f6bc0159dd0 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.400+0000 7f6bc84ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc015b2c0 con 0x7f6bc006b6f0 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.401+0000 7f6bc5a46640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 0x7f6bc01e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.401+0000 7f6bc5a46640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 0x7f6bc01e6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39026/0 (socket says 172.21.15.115:39026) 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.401+0000 7f6bc5a46640 1 -- 172.21.15.115:0/3550897732 learned_addr learned my addr 172.21.15.115:0/3550897732 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.401+0000 7f6bc6a48640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bc0159dd0 0x7f6bc00cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.401+0000 7f6bc6247640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 0x7f6bc0156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.401+0000 7f6bc5a46640 1 -- 172.21.15.115:0/3550897732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 msgr2=0x7f6bc0156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:24.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.401+0000 7f6bc5a46640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 0x7f6bc0156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.401+0000 7f6bc5a46640 1 -- 172.21.15.115:0/3550897732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bc0159dd0 msgr2=0x7f6bc00cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:24.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.401+0000 7f6bc5a46640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bc0159dd0 0x7f6bc00cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.401+0000 7f6bc5a46640 1 -- 172.21.15.115:0/3550897732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00094d0 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.402+0000 7f6bc5a46640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 0x7f6bc01e6fd0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009870 tx=0x7f6bb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.402+0000 7f6bb6ffd640 1 -- 172.21.15.115:0/3550897732 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0033c70 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.402+0000 7f6bb6ffd640 1 -- 172.21.15.115:0/3550897732 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb0033dd0 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.402+0000 7f6bb6ffd640 1 -- 172.21.15.115:0/3550897732 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb003c640 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.402+0000 7f6bc84ab640 1 -- 172.21.15.115:0/3550897732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc015b5a0 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.402+0000 7f6bc84ab640 1 -- 172.21.15.115:0/3550897732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc01e7510 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.403+0000 7f6bc84ab640 1 -- 172.21.15.115:0/3550897732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc004e890 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.404+0000 7f6bb6ffd640 1 -- 172.21.15.115:0/3550897732 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6bb003a040 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.405+0000 7f6bb6ffd640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ba406a550 0x7f6ba406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:24.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.405+0000 7f6bb6ffd640 1 -- 172.21.15.115:0/3550897732 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f6bb00aa700 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.406+0000 7f6bc6247640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ba406a550 0x7f6ba406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:24.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.406+0000 7f6bc6247640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ba406a550 0x7f6ba406ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc004ea0 tx=0x7f6bbc009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:24.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.410+0000 7f6bb6ffd640 1 -- 172.21.15.115:0/3550897732 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6bb0075c40 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.665+0000 7f6bc84ab640 1 -- 172.21.15.115:0/3550897732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f6bc0071b00 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.816+0000 7f6bb6ffd640 1 -- 172.21.15.115:0/3550897732 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v697) v1 ==== 183+0+0 (secure 0 0 0) 0x7f6bb00380f0 con 0x7f6bc00c9ef0 2023-12-01T01:54:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.818+0000 7f6bc84ab640 1 -- 172.21.15.115:0/3550897732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ba406a550 msgr2=0x7f6ba406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.818+0000 7f6bc84ab640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ba406a550 0x7f6ba406ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc004ea0 tx=0x7f6bbc009040 comp rx=0 tx=0).stop 2023-12-01T01:54:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.818+0000 7f6bc84ab640 1 -- 172.21.15.115:0/3550897732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 msgr2=0x7f6bc01e6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.819+0000 7f6bc84ab640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 0x7f6bc01e6fd0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009870 tx=0x7f6bb00046c0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.819+0000 7f6bc84ab640 1 -- 172.21.15.115:0/3550897732 shutdown_connections 2023-12-01T01:54:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.819+0000 7f6bc84ab640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ba406a550 0x7f6ba406ca10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.819+0000 7f6bc84ab640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bc0159dd0 0x7f6bc00cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.819+0000 7f6bc84ab640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bc00c9ef0 0x7f6bc01e6fd0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.819+0000 7f6bc84ab640 1 --2- 172.21.15.115:0/3550897732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bc006b6f0 0x7f6bc0156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.819+0000 7f6bc84ab640 1 -- 172.21.15.115:0/3550897732 >> 172.21.15.115:0/3550897732 conn(0x7f6bc00ce880 msgr2=0x7f6bc006fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.819+0000 7f6bc84ab640 1 -- 172.21.15.115:0/3550897732 shutdown_connections 2023-12-01T01:54:24.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:24.819+0000 7f6bc84ab640 1 -- 172.21.15.115:0/3550897732 wait complete. 2023-12-01T01:54:24.843 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-12-01T01:54:24.844 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-12-01T01:54:24.844 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-01T01:54:24.844 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-12-01T01:54:25.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.057+0000 7fec18b37640 1 Processor -- start 2023-12-01T01:54:25.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.057+0000 7fec18b37640 1 -- start start 2023-12-01T01:54:25.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.059+0000 7fec18b37640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fec141584b0 0x7fec140cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:25.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.059+0000 7fec18b37640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec1415afe0 0x7fec1406b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:25.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.059+0000 7fec18b37640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec140cc180 con 0x7fec1406b7b0 2023-12-01T01:54:25.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.059+0000 7fec18b37640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1406bc40 con 0x7fec141584b0 2023-12-01T01:54:25.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.059+0000 7fec18b37640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1415b550 con 0x7fec1415afe0 2023-12-01T01:54:25.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.059+0000 7fec137fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fec141584b0 0x7fec140cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:25.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.060+0000 7fec12ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec1415afe0 0x7fec1406b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.060+0000 7fec137fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fec141584b0 0x7fec140cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:59454/0 (socket says 172.21.15.115:59454) 2023-12-01T01:54:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.060+0000 7fec137fe640 1 -- 172.21.15.115:0/156738921 learned_addr learned my addr 172.21.15.115:0/156738921 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.060+0000 7fec137fe640 1 -- 172.21.15.115:0/156738921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec1415afe0 msgr2=0x7fec1406b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.060+0000 7fec137fe640 1 --2- 172.21.15.115:0/156738921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec1415afe0 0x7fec1406b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.060+0000 7fec137fe640 1 -- 172.21.15.115:0/156738921 >> v1:172.21.15.142:6789/0 conn(0x7fec1406b7b0 legacy=0x7fec1415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.060+0000 7fec137fe640 1 -- 172.21.15.115:0/156738921 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1415b830 con 0x7fec141584b0 2023-12-01T01:54:25.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.061+0000 7fec137fe640 1 --2- 172.21.15.115:0/156738921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec141584b0 0x7fec140cdd90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fec04009870 tx=0x7fec0402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b43fe341cbc53d29 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.062+0000 7fec127fc640 1 -- 172.21.15.115:0/156738921 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04030840 con 0x7fec141584b0 2023-12-01T01:54:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.062+0000 7fec127fc640 1 -- 172.21.15.115:0/156738921 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec040309a0 con 0x7fec141584b0 2023-12-01T01:54:25.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.062+0000 7fec127fc640 1 -- 172.21.15.115:0/156738921 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04030d30 con 0x7fec141584b0 2023-12-01T01:54:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.062+0000 7fec18b37640 1 -- 172.21.15.115:0/156738921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec141584b0 msgr2=0x7fec140cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.062+0000 7fec18b37640 1 --2- 172.21.15.115:0/156738921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec141584b0 0x7fec140cdd90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fec04009870 tx=0x7fec0402b980 comp rx=0 tx=0).stop 2023-12-01T01:54:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.063+0000 7fec18b37640 1 -- 172.21.15.115:0/156738921 shutdown_connections 2023-12-01T01:54:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.063+0000 7fec18b37640 1 --2- 172.21.15.115:0/156738921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec1415afe0 0x7fec1406b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.063+0000 7fec18b37640 1 --2- 172.21.15.115:0/156738921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec141584b0 0x7fec140cdd90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.063+0000 7fec18b37640 1 -- 172.21.15.115:0/156738921 >> 172.21.15.115:0/156738921 conn(0x7fec140ce640 msgr2=0x7fec140cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:25.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.063+0000 7fec18b37640 1 -- 172.21.15.115:0/156738921 shutdown_connections 2023-12-01T01:54:25.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.063+0000 7fec18b37640 1 -- 172.21.15.115:0/156738921 wait complete. 2023-12-01T01:54:25.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.064+0000 7fec18b37640 1 Processor -- start 2023-12-01T01:54:25.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.064+0000 7fec18b37640 1 -- start start 2023-12-01T01:54:25.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.065+0000 7fec18b37640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1406b7b0 0x7fec1414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:25.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.065+0000 7fec18b37640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec141584b0 0x7fec141f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:25.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.065+0000 7fec18b37640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec1415afe0 0x7fec140ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:25.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.065+0000 7fec18b37640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1414c710 con 0x7fec141584b0 2023-12-01T01:54:25.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.065+0000 7fec18b37640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec140b2e20 con 0x7fec1406b7b0 2023-12-01T01:54:25.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.065+0000 7fec18b37640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec140b3100 con 0x7fec1415afe0 2023-12-01T01:54:25.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.066+0000 7fec12ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec141584b0 0x7fec141f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:25.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.066+0000 7fec12ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec141584b0 0x7fec141f25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39042/0 (socket says 172.21.15.115:39042) 2023-12-01T01:54:25.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.066+0000 7fec12ffd640 1 -- 172.21.15.115:0/315898147 learned_addr learned my addr 172.21.15.115:0/315898147 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:25.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.066+0000 7fec137fe640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1406b7b0 0x7fec1414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.066+0000 7fec12ffd640 1 -- 172.21.15.115:0/315898147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec1415afe0 msgr2=0x7fec140ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.066+0000 7fec12ffd640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec1415afe0 0x7fec140ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.066+0000 7fec12ffd640 1 -- 172.21.15.115:0/315898147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1406b7b0 msgr2=0x7fec1414c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.066+0000 7fec12ffd640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1406b7b0 0x7fec1414c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.066+0000 7fec12ffd640 1 -- 172.21.15.115:0/315898147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec040094d0 con 0x7fec141584b0 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.066+0000 7fec137fe640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1406b7b0 0x7fec1414c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.067+0000 7fec12ffd640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec141584b0 0x7fec141f25d0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fec0800e870 tx=0x7fec0800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.067+0000 7febfbfff640 1 -- 172.21.15.115:0/315898147 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec080094f0 con 0x7fec141584b0 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.067+0000 7febfbfff640 1 -- 172.21.15.115:0/315898147 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0800c940 con 0x7fec141584b0 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.067+0000 7febfbfff640 1 -- 172.21.15.115:0/315898147 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0800ccf0 con 0x7fec141584b0 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.067+0000 7fec18b37640 1 -- 172.21.15.115:0/315898147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec140b33e0 con 0x7fec141584b0 2023-12-01T01:54:25.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.067+0000 7fec18b37640 1 -- 172.21.15.115:0/315898147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec140b39a0 con 0x7fec141584b0 2023-12-01T01:54:25.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.068+0000 7fec18b37640 1 -- 172.21.15.115:0/315898147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec140673a0 con 0x7fec141584b0 2023-12-01T01:54:25.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.069+0000 7febfbfff640 1 -- 172.21.15.115:0/315898147 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fec08013470 con 0x7fec141584b0 2023-12-01T01:54:25.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.070+0000 7febfbfff640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fec0006a550 0x7fec0006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:25.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.070+0000 7febfbfff640 1 -- 172.21.15.115:0/315898147 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fec08088af0 con 0x7fec141584b0 2023-12-01T01:54:25.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.070+0000 7fec137fe640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fec0006a550 0x7fec0006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:25.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.071+0000 7fec137fe640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fec0006a550 0x7fec0006ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fec04036000 tx=0x7fec0403b040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:25.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.074+0000 7febfbfff640 1 -- 172.21.15.115:0/315898147 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fec08054020 con 0x7fec141584b0 2023-12-01T01:54:25.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.318+0000 7fec18b37640 1 -- 172.21.15.115:0/315898147 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fec1406b480 con 0x7fec141584b0 2023-12-01T01:54:25.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.819+0000 7febfbfff640 1 -- 172.21.15.115:0/315898147 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v698) v1 ==== 151+0+0 (secure 0 0 0) 0x7fec080170d0 con 0x7fec141584b0 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 -- 172.21.15.115:0/315898147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fec0006a550 msgr2=0x7fec0006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fec0006a550 0x7fec0006ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fec04036000 tx=0x7fec0403b040 comp rx=0 tx=0).stop 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 -- 172.21.15.115:0/315898147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec141584b0 msgr2=0x7fec141f25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec141584b0 0x7fec141f25d0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fec0800e870 tx=0x7fec0800ed30 comp rx=0 tx=0).stop 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 -- 172.21.15.115:0/315898147 shutdown_connections 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fec0006a550 0x7fec0006ca10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec1415afe0 0x7fec140ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec141584b0 0x7fec141f25d0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 --2- 172.21.15.115:0/315898147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1406b7b0 0x7fec1414c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 -- 172.21.15.115:0/315898147 >> 172.21.15.115:0/315898147 conn(0x7fec140ce640 msgr2=0x7fec140cb050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:25.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 -- 172.21.15.115:0/315898147 shutdown_connections 2023-12-01T01:54:25.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:25.822+0000 7fec18b37640 1 -- 172.21.15.115:0/315898147 wait complete. 2023-12-01T01:54:25.844 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T01:54:26.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.062+0000 7f7fc7ef7640 1 Processor -- start 2023-12-01T01:54:26.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.062+0000 7f7fc7ef7640 1 -- start start 2023-12-01T01:54:26.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.064+0000 7f7fc7ef7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7fc015ac80 0x7f7fc00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:26.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.064+0000 7f7fc7ef7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc00cc9a0 con 0x7f7fc00b0e60 2023-12-01T01:54:26.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.064+0000 7f7fc7ef7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc00cbd90 con 0x7f7fc00ce1b0 2023-12-01T01:54:26.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.064+0000 7f7fc7ef7640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc015b060 con 0x7f7fc015ac80 2023-12-01T01:54:26.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.064+0000 7f7fc5c93640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7fc015ac80 0x7f7fc00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:26.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.064+0000 7f7fc5c93640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7fc015ac80 0x7f7fc00cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39046/0 (socket says 172.21.15.115:39046) 2023-12-01T01:54:26.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.064+0000 7f7fc5c93640 1 -- 172.21.15.115:0/1800831914 learned_addr learned my addr 172.21.15.115:0/1800831914 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:26.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.064+0000 7f7fc5c93640 1 -- 172.21.15.115:0/1800831914 >> v1:172.21.15.142:6789/0 conn(0x7f7fc00ce1b0 legacy=0x7f7fc006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:26.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.064+0000 7f7fc5c93640 1 -- 172.21.15.115:0/1800831914 >> v1:172.21.15.115:6789/0 conn(0x7f7fc00b0e60 legacy=0x7f7fc00cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:26.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.065+0000 7f7fc5c93640 1 -- 172.21.15.115:0/1800831914 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc00cc070 con 0x7f7fc015ac80 2023-12-01T01:54:26.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.066+0000 7f7fc5c93640 1 --2- 172.21.15.115:0/1800831914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 0x7f7fc00cce30 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009870 tx=0x7f7fbc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede826fa1110f86f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:26.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.066+0000 7f7fc4c91640 1 -- 172.21.15.115:0/1800831914 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc030860 con 0x7f7fc015ac80 2023-12-01T01:54:26.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.066+0000 7f7fc4c91640 1 -- 172.21.15.115:0/1800831914 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc0309c0 con 0x7f7fc015ac80 2023-12-01T01:54:26.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.066+0000 7f7fc4c91640 1 -- 172.21.15.115:0/1800831914 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc030d50 con 0x7f7fc015ac80 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.066+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/1800831914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 msgr2=0x7f7fc00cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.066+0000 7f7fc7ef7640 1 --2- 172.21.15.115:0/1800831914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 0x7f7fc00cce30 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009870 tx=0x7f7fbc02b9a0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.067+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/1800831914 shutdown_connections 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.067+0000 7f7fc7ef7640 1 --2- 172.21.15.115:0/1800831914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 0x7f7fc00cce30 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.067+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/1800831914 >> 172.21.15.115:0/1800831914 conn(0x7f7fc0065960 msgr2=0x7f7fc0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.067+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/1800831914 shutdown_connections 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.067+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/1800831914 wait complete. 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.068+0000 7f7fc7ef7640 1 Processor -- start 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.068+0000 7f7fc7ef7640 1 -- start start 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.069+0000 7f7fc7ef7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fc00b0e60 0x7f7fc0150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.069+0000 7f7fc7ef7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fc00ce1b0 0x7f7fc01f0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.069+0000 7f7fc7ef7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 0x7f7fc01f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.069+0000 7f7fc7ef7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc0150dc0 con 0x7f7fc015ac80 2023-12-01T01:54:26.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.069+0000 7f7fc7ef7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc0151060 con 0x7f7fc00b0e60 2023-12-01T01:54:26.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.069+0000 7f7fc7ef7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc0151340 con 0x7f7fc00ce1b0 2023-12-01T01:54:26.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.070+0000 7f7fc6494640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 0x7f7fc01f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:26.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.070+0000 7f7fc6494640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 0x7f7fc01f0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39050/0 (socket says 172.21.15.115:39050) 2023-12-01T01:54:26.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.070+0000 7f7fc6494640 1 -- 172.21.15.115:0/3865287241 learned_addr learned my addr 172.21.15.115:0/3865287241 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:26.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.070+0000 7f7fc5c93640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fc00b0e60 0x7f7fc0150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:26.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.070+0000 7f7fc5492640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fc00ce1b0 0x7f7fc01f0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:26.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.070+0000 7f7fc6494640 1 -- 172.21.15.115:0/3865287241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fc00ce1b0 msgr2=0x7f7fc01f0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:26.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.070+0000 7f7fc6494640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fc00ce1b0 0x7f7fc01f0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.070+0000 7f7fc6494640 1 -- 172.21.15.115:0/3865287241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fc00b0e60 msgr2=0x7f7fc0150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:26.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.070+0000 7f7fc6494640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fc00b0e60 0x7f7fc0150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.070+0000 7f7fc6494640 1 -- 172.21.15.115:0/3865287241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc0094d0 con 0x7f7fc015ac80 2023-12-01T01:54:26.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.071+0000 7f7fc6494640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 0x7f7fc01f0bb0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800c780 tx=0x7f7fb800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:26.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.071+0000 7f7fb67fc640 1 -- 172.21.15.115:0/3865287241 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8003d80 con 0x7f7fc015ac80 2023-12-01T01:54:26.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.071+0000 7f7fb67fc640 1 -- 172.21.15.115:0/3865287241 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb800ce30 con 0x7f7fc015ac80 2023-12-01T01:54:26.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.071+0000 7f7fb67fc640 1 -- 172.21.15.115:0/3865287241 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8005290 con 0x7f7fc015ac80 2023-12-01T01:54:26.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.071+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/3865287241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc0151620 con 0x7f7fc015ac80 2023-12-01T01:54:26.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.071+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/3865287241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc0151e20 con 0x7f7fc015ac80 2023-12-01T01:54:26.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.072+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/3865287241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc0151b40 con 0x7f7fc015ac80 2023-12-01T01:54:26.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.073+0000 7f7fb67fc640 1 -- 172.21.15.115:0/3865287241 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f7fb8006b10 con 0x7f7fc015ac80 2023-12-01T01:54:26.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.074+0000 7f7fb67fc640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7fa406a550 0x7f7fa406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:26.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.074+0000 7f7fb67fc640 1 -- 172.21.15.115:0/3865287241 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f7fb8087e60 con 0x7f7fc015ac80 2023-12-01T01:54:26.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.074+0000 7f7fc5c93640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7fa406a550 0x7f7fa406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:26.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.075+0000 7f7fc5c93640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7fa406a550 0x7f7fa406ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc036000 tx=0x7f7fbc03b040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:26.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.078+0000 7f7fb67fc640 1 -- 172.21.15.115:0/3865287241 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7fc0151b40 con 0x7f7fc015ac80 2023-12-01T01:54:26.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.268+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/3865287241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7fc0151b40 con 0x7f7fc015ac80 2023-12-01T01:54:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.277+0000 7f7fb67fc640 1 -- 172.21.15.115:0/3865287241 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7fc0151b40 con 0x7f7fc015ac80 2023-12-01T01:54:26.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/3865287241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7fa406a550 msgr2=0x7f7fa406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:26.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7fa406a550 0x7f7fa406ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc036000 tx=0x7f7fbc03b040 comp rx=0 tx=0).stop 2023-12-01T01:54:26.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/3865287241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 msgr2=0x7f7fc01f0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:26.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 0x7f7fc01f0bb0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800c780 tx=0x7f7fb800cc40 comp rx=0 tx=0).stop 2023-12-01T01:54:26.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/3865287241 shutdown_connections 2023-12-01T01:54:26.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7fa406a550 0x7f7fa406ca10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fc00ce1b0 0x7f7fc01f0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fc015ac80 0x7f7fc01f0bb0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 --2- 172.21.15.115:0/3865287241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fc00b0e60 0x7f7fc0150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/3865287241 >> 172.21.15.115:0/3865287241 conn(0x7f7fc0065960 msgr2=0x7f7fc0155f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/3865287241 shutdown_connections 2023-12-01T01:54:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.280+0000 7f7fc7ef7640 1 -- 172.21.15.115:0/3865287241 wait complete. 2023-12-01T01:54:26.299 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:54:26.299 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:54:26.407 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:54:26.529 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:54:26.529 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:54:26.529 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:54:26.589 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:54:26.589 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:54:26.589 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:54:26.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:54:26.650 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:54:26.650 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:54:26.711 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:54:26.711 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:54:26.712 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:54:26.772 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:54:26.772 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T01:54:26.773 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:54:26.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.990+0000 7fedeba12640 1 Processor -- start 2023-12-01T01:54:26.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.991+0000 7fedeba12640 1 -- start start 2023-12-01T01:54:26.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.992+0000 7fedeba12640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede40cad70 0x7fede406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:26.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.992+0000 7fedeba12640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fede406f2f0 0x7fede406bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:26.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.992+0000 7fedeba12640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede40add10 con 0x7fede415e2f0 2023-12-01T01:54:26.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.992+0000 7fedeba12640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede40c5760 con 0x7fede406f2f0 2023-12-01T01:54:26.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.992+0000 7fedeba12640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede415e6d0 con 0x7fede40cad70 2023-12-01T01:54:26.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.993+0000 7fede8fad640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede40cad70 0x7fede406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:26.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.993+0000 7fede9faf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fede406f2f0 0x7fede406bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:26.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.993+0000 7fede8fad640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede40cad70 0x7fede406b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:34020/0 (socket says 172.21.15.115:34020) 2023-12-01T01:54:26.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.993+0000 7fede8fad640 1 -- 172.21.15.115:0/4035283110 learned_addr learned my addr 172.21.15.115:0/4035283110 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:26.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.994+0000 7fede8fad640 1 -- 172.21.15.115:0/4035283110 >> v2:172.21.15.142:3300/0 conn(0x7fede406f2f0 msgr2=0x7fede406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:26.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.994+0000 7fede8fad640 1 --2- 172.21.15.115:0/4035283110 >> v2:172.21.15.142:3300/0 conn(0x7fede406f2f0 0x7fede406bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.994+0000 7fede8fad640 1 -- 172.21.15.115:0/4035283110 >> v1:172.21.15.142:6789/0 conn(0x7fede415e2f0 legacy=0x7fede40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:26.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.994+0000 7fede8fad640 1 -- 172.21.15.115:0/4035283110 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede40cf7b0 con 0x7fede40cad70 2023-12-01T01:54:26.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.994+0000 7fede9faf640 1 --2- 172.21.15.115:0/4035283110 >> v2:172.21.15.142:3300/0 conn(0x7fede406f2f0 0x7fede406bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:54:26.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.995+0000 7fede8fad640 1 --2- 172.21.15.115:0/4035283110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede40cad70 0x7fede406b5c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009870 tx=0x7fedd402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3106f30aba61561f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:26.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.996+0000 7feddbfff640 1 -- 172.21.15.115:0/4035283110 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd40317c0 con 0x7fede40cad70 2023-12-01T01:54:26.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.996+0000 7feddbfff640 1 -- 172.21.15.115:0/4035283110 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd4031920 con 0x7fede40cad70 2023-12-01T01:54:26.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.996+0000 7feddbfff640 1 -- 172.21.15.115:0/4035283110 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4031cb0 con 0x7fede40cad70 2023-12-01T01:54:26.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.996+0000 7fedeba12640 1 -- 172.21.15.115:0/4035283110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede40cad70 msgr2=0x7fede406b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:26.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.996+0000 7fedeba12640 1 --2- 172.21.15.115:0/4035283110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede40cad70 0x7fede406b5c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009870 tx=0x7fedd402b860 comp rx=0 tx=0).stop 2023-12-01T01:54:26.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.996+0000 7fedeba12640 1 -- 172.21.15.115:0/4035283110 shutdown_connections 2023-12-01T01:54:26.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.996+0000 7fedeba12640 1 --2- 172.21.15.115:0/4035283110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede40cad70 0x7fede406b5c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.996+0000 7fedeba12640 1 --2- 172.21.15.115:0/4035283110 >> v2:172.21.15.142:3300/0 conn(0x7fede406f2f0 0x7fede406bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:26.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.996+0000 7fedeba12640 1 -- 172.21.15.115:0/4035283110 >> 172.21.15.115:0/4035283110 conn(0x7fede40ce880 msgr2=0x7fede40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.997+0000 7fedeba12640 1 -- 172.21.15.115:0/4035283110 shutdown_connections 2023-12-01T01:54:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.997+0000 7fedeba12640 1 -- 172.21.15.115:0/4035283110 wait complete. 2023-12-01T01:54:26.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.998+0000 7fedeba12640 1 Processor -- start 2023-12-01T01:54:27.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.998+0000 7fedeba12640 1 -- start start 2023-12-01T01:54:27.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.999+0000 7fedeba12640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede406f2f0 0x7fede4159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.999+0000 7fedeba12640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fede40cad70 0x7fede40c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.999+0000 7fedeba12640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fede415e2f0 0x7fede40aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.999+0000 7fedeba12640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede40c2af0 con 0x7fede40cad70 2023-12-01T01:54:27.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.999+0000 7fedeba12640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede415cfd0 con 0x7fede415e2f0 2023-12-01T01:54:27.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:26.999+0000 7fedeba12640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede40c24e0 con 0x7fede406f2f0 2023-12-01T01:54:27.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.000+0000 7fede8fad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fede40cad70 0x7fede40c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.000+0000 7fede8fad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fede40cad70 0x7fede40c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39066/0 (socket says 172.21.15.115:39066) 2023-12-01T01:54:27.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.000+0000 7fede8fad640 1 -- 172.21.15.115:0/312772143 learned_addr learned my addr 172.21.15.115:0/312772143 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.000+0000 7fede9faf640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fede415e2f0 0x7fede40aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.000+0000 7fede97ae640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede406f2f0 0x7fede4159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.000+0000 7fede8fad640 1 -- 172.21.15.115:0/312772143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede406f2f0 msgr2=0x7fede4159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.000+0000 7fede8fad640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede406f2f0 0x7fede4159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.000+0000 7fede8fad640 1 -- 172.21.15.115:0/312772143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fede415e2f0 msgr2=0x7fede40aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.000+0000 7fede8fad640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fede415e2f0 0x7fede40aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.000+0000 7fede8fad640 1 -- 172.21.15.115:0/312772143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc009a70 con 0x7fede40cad70 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.001+0000 7fede8fad640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fede40cad70 0x7fede40c6e20 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fedd402b8a0 tx=0x7fedd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.001+0000 7fedd9ffb640 1 -- 172.21.15.115:0/312772143 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4032860 con 0x7fede40cad70 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.001+0000 7fedd9ffb640 1 -- 172.21.15.115:0/312772143 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd4032a00 con 0x7fede40cad70 2023-12-01T01:54:27.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.001+0000 7fedd9ffb640 1 -- 172.21.15.115:0/312772143 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4032db0 con 0x7fede40cad70 2023-12-01T01:54:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.001+0000 7fedeba12640 1 -- 172.21.15.115:0/312772143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd40094d0 con 0x7fede40cad70 2023-12-01T01:54:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.001+0000 7fedeba12640 1 -- 172.21.15.115:0/312772143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede40c2780 con 0x7fede40cad70 2023-12-01T01:54:27.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.002+0000 7fedeba12640 1 -- 172.21.15.115:0/312772143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede406fda0 con 0x7fede40cad70 2023-12-01T01:54:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.003+0000 7fedd9ffb640 1 -- 172.21.15.115:0/312772143 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fedd404e450 con 0x7fede40cad70 2023-12-01T01:54:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.004+0000 7fedd9ffb640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedc806a550 0x7fedc806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.004+0000 7fedd9ffb640 1 -- 172.21.15.115:0/312772143 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fedd40aaaf0 con 0x7fede40cad70 2023-12-01T01:54:27.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.004+0000 7fede97ae640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedc806a550 0x7fedc806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.005+0000 7fede97ae640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedc806a550 0x7fedc806ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fede0000f60 tx=0x7fede0009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:27.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.008+0000 7fedd9ffb640 1 -- 172.21.15.115:0/312772143 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fedd4076130 con 0x7fede40cad70 2023-12-01T01:54:27.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.234+0000 7fedeba12640 1 -- 172.21.15.115:0/312772143 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fede415ab30 con 0x7fede40cad70 2023-12-01T01:54:27.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.234+0000 7fedd9ffb640 1 -- 172.21.15.115:0/312772143 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fedd4079750 con 0x7fede40cad70 2023-12-01T01:54:27.236 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:27.236 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5588,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3678441703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3678441703},{"type":"v1","addr":"172.21.15.142:6834","nonce":3678441703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":5827,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3538792232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3538792232},{"type":"v1","addr":"172.21.15.142:6835","nonce":3538792232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7449,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3171906828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3171906828},{"type":"v1","addr":"172.21.15.115:6835","nonce":3171906828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7452,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2130806226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2130806226},{"type":"v1","addr":"172.21.15.115:6837","nonce":2130806226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[]} 2023-12-01T01:54:27.236 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 50 2023-12-01T01:54:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.237+0000 7fedeba12640 1 -- 172.21.15.115:0/312772143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedc806a550 msgr2=0x7fedc806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.237+0000 7fedeba12640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedc806a550 0x7fedc806ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fede0000f60 tx=0x7fede0009040 comp rx=0 tx=0).stop 2023-12-01T01:54:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.237+0000 7fedeba12640 1 -- 172.21.15.115:0/312772143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fede40cad70 msgr2=0x7fede40c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.237+0000 7fedeba12640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fede40cad70 0x7fede40c6e20 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fedd402b8a0 tx=0x7fedd40046c0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.237+0000 7fedeba12640 1 -- 172.21.15.115:0/312772143 shutdown_connections 2023-12-01T01:54:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.237+0000 7fedeba12640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fedc806a550 0x7fedc806ca10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.237+0000 7fedeba12640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fede415e2f0 0x7fede40aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.237+0000 7fedeba12640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fede40cad70 0x7fede40c6e20 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.237+0000 7fedeba12640 1 --2- 172.21.15.115:0/312772143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fede406f2f0 0x7fede4159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.237+0000 7fedeba12640 1 -- 172.21.15.115:0/312772143 >> 172.21.15.115:0/312772143 conn(0x7fede40ce880 msgr2=0x7fede40cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.238+0000 7fedeba12640 1 -- 172.21.15.115:0/312772143 shutdown_connections 2023-12-01T01:54:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.238+0000 7fedeba12640 1 -- 172.21.15.115:0/312772143 wait complete. 2023-12-01T01:54:27.259 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T01:54:27.259 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T01:54:27.259 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T01:54:27.259 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:54:27.259 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T01:54:27.259 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T01:54:27.260 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T01:54:27.260 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:54:27.260 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T01:54:27.260 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T01:54:27.260 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T01:54:27.260 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:54:27.260 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T01:54:27.260 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T01:54:27.261 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T01:54:27.261 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:54:27.276 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T01:54:27.277 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T01:54:27.278 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T01:54:27.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T01:54:27.282 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T01:54:27.282 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T01:54:27.287 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T01:54:27.287 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T01:54:27.325 INFO:tasks.ceph.mds.a:Started 2023-12-01T01:54:27.326 DEBUG:teuthology.parallel:result is None 2023-12-01T01:54:27.333 INFO:tasks.ceph.mds.c:Started 2023-12-01T01:54:27.334 DEBUG:teuthology.parallel:result is None 2023-12-01T01:54:27.335 INFO:tasks.ceph.mds.d:Started 2023-12-01T01:54:27.335 DEBUG:teuthology.parallel:result is None 2023-12-01T01:54:27.343 INFO:tasks.ceph.mds.b:Started 2023-12-01T01:54:27.343 DEBUG:teuthology.parallel:result is None 2023-12-01T01:54:27.343 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T01:54:27.483 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T01:54:27.483 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:54:27.505 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T01:54:27.506 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T01:54:27.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.533+0000 7f3a6b3e9640 1 Processor -- start 2023-12-01T01:54:27.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.534+0000 7f3a6b3e9640 1 -- start start 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a6b3e9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3a640b0e70 0x7f3a640c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a6b3e9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a640b3ba0 0x7f3a640b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a6b3e9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a6406b6a0 0x7f3a6406bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a6b3e9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a640c8af0 con 0x7f3a640b0e70 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a6b3e9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6406b1c0 con 0x7f3a640b3ba0 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a6b3e9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6415f630 con 0x7f3a6406b6a0 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a6a3e7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3a640b0e70 0x7f3a640c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a6a3e7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3a640b0e70 0x7f3a640c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39108/0 (socket says 172.21.15.115:39108) 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a6a3e7640 1 -- 172.21.15.115:0/1134455111 learned_addr learned my addr 172.21.15.115:0/1134455111 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a69be6640 1 --2- 172.21.15.115:0/1134455111 >> v2:172.21.15.142:3300/0 conn(0x7f3a640b3ba0 0x7f3a640b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.535+0000 7f3a6abe8640 1 --2- 172.21.15.115:0/1134455111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a6406b6a0 0x7f3a6406bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.536+0000 7f3a6a3e7640 1 -- 172.21.15.115:0/1134455111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a6406b6a0 msgr2=0x7f3a6406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.536+0000 7f3a6a3e7640 1 --2- 172.21.15.115:0/1134455111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a6406b6a0 0x7f3a6406bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.536+0000 7f3a6a3e7640 1 -- 172.21.15.115:0/1134455111 >> v2:172.21.15.142:3300/0 conn(0x7f3a640b3ba0 msgr2=0x7f3a640b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.536+0000 7f3a6a3e7640 1 --2- 172.21.15.115:0/1134455111 >> v2:172.21.15.142:3300/0 conn(0x7f3a640b3ba0 0x7f3a640b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.536+0000 7f3a6a3e7640 1 -- 172.21.15.115:0/1134455111 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a640b2040 con 0x7f3a640b0e70 2023-12-01T01:54:27.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.537+0000 7f3a6a3e7640 1 --2- 172.21.15.115:0/1134455111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b0e70 0x7f3a640c85b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f3a640b1250 tx=0x7f3a6002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea9b11d52affe31 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:27.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.537+0000 7f3a693e5640 1 -- 172.21.15.115:0/1134455111 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600317d0 con 0x7f3a640b0e70 2023-12-01T01:54:27.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.537+0000 7f3a693e5640 1 -- 172.21.15.115:0/1134455111 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60031930 con 0x7f3a640b0e70 2023-12-01T01:54:27.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.537+0000 7f3a693e5640 1 -- 172.21.15.115:0/1134455111 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60031cc0 con 0x7f3a640b0e70 2023-12-01T01:54:27.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.538+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/1134455111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b0e70 msgr2=0x7f3a640c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:27.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.538+0000 7f3a6b3e9640 1 --2- 172.21.15.115:0/1134455111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b0e70 0x7f3a640c85b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f3a640b1250 tx=0x7f3a6002b870 comp rx=0 tx=0).stop 2023-12-01T01:54:27.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.538+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/1134455111 shutdown_connections 2023-12-01T01:54:27.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.538+0000 7f3a6b3e9640 1 --2- 172.21.15.115:0/1134455111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a6406b6a0 0x7f3a6406bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.538+0000 7f3a6b3e9640 1 --2- 172.21.15.115:0/1134455111 >> v2:172.21.15.142:3300/0 conn(0x7f3a640b3ba0 0x7f3a640b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.538+0000 7f3a6b3e9640 1 --2- 172.21.15.115:0/1134455111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b0e70 0x7f3a640c85b0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.538+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/1134455111 >> 172.21.15.115:0/1134455111 conn(0x7f3a640ce620 msgr2=0x7f3a640cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:27.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.538+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/1134455111 shutdown_connections 2023-12-01T01:54:27.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.538+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/1134455111 wait complete. 2023-12-01T01:54:27.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.539+0000 7f3a6b3e9640 1 Processor -- start 2023-12-01T01:54:27.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.540+0000 7f3a6b3e9640 1 -- start start 2023-12-01T01:54:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.540+0000 7f3a6b3e9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a6406b6a0 0x7f3a641f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6b3e9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a640b0e70 0x7f3a640c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6b3e9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b3ba0 0x7f3a640caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6b3e9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640c8d30 con 0x7f3a640b3ba0 2023-12-01T01:54:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6b3e9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a641f2ba0 con 0x7f3a640b0e70 2023-12-01T01:54:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6b3e9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a641f2e80 con 0x7f3a6406b6a0 2023-12-01T01:54:27.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6abe8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b3ba0 0x7f3a640caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6abe8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b3ba0 0x7f3a640caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39118/0 (socket says 172.21.15.115:39118) 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6abe8640 1 -- 172.21.15.115:0/2473356865 learned_addr learned my addr 172.21.15.115:0/2473356865 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a69be6640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a640b0e70 0x7f3a640c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6a3e7640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a6406b6a0 0x7f3a641f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6abe8640 1 -- 172.21.15.115:0/2473356865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a6406b6a0 msgr2=0x7f3a641f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6abe8640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a6406b6a0 0x7f3a641f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6abe8640 1 -- 172.21.15.115:0/2473356865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a640b0e70 msgr2=0x7f3a640c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6abe8640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a640b0e70 0x7f3a640c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.541+0000 7f3a6abe8640 1 -- 172.21.15.115:0/2473356865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a600094d0 con 0x7f3a640b3ba0 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.542+0000 7f3a6abe8640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b3ba0 0x7f3a640caf70 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00eed0 tx=0x7f3a5c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.542+0000 7f3a5affd640 1 -- 172.21.15.115:0/2473356865 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c010040 con 0x7f3a640b3ba0 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.542+0000 7f3a5affd640 1 -- 172.21.15.115:0/2473356865 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c00bd20 con 0x7f3a640b3ba0 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.542+0000 7f3a5affd640 1 -- 172.21.15.115:0/2473356865 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c014640 con 0x7f3a640b3ba0 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.542+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/2473356865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a641f3160 con 0x7f3a640b3ba0 2023-12-01T01:54:27.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.542+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/2473356865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a641f3440 con 0x7f3a640b3ba0 2023-12-01T01:54:27.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.543+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/2473356865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a64067850 con 0x7f3a640b3ba0 2023-12-01T01:54:27.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.544+0000 7f3a5affd640 1 -- 172.21.15.115:0/2473356865 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3a5c0147a0 con 0x7f3a640b3ba0 2023-12-01T01:54:27.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.545+0000 7f3a5affd640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a4806a580 0x7f3a4806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:27.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.545+0000 7f3a5affd640 1 -- 172.21.15.115:0/2473356865 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f3a5c089360 con 0x7f3a640b3ba0 2023-12-01T01:54:27.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.545+0000 7f3a6a3e7640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a4806a580 0x7f3a4806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:27.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.546+0000 7f3a6a3e7640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a4806a580 0x7f3a4806ca40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a60036000 tx=0x7f3a6003d040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.549+0000 7f3a5affd640 1 -- 172.21.15.115:0/2473356865 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3a5c054820 con 0x7f3a640b3ba0 2023-12-01T01:54:27.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:27.746+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/2473356865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3a640673a0 con 0x7f3a640b3ba0 2023-12-01T01:54:28.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.535+0000 7f3a5affd640 1 -- 172.21.15.115:0/2473356865 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3a5c057e40 con 0x7f3a640b3ba0 2023-12-01T01:54:28.537 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T01:54:28.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.538+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/2473356865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a4806a580 msgr2=0x7f3a4806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:28.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.538+0000 7f3a6b3e9640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a4806a580 0x7f3a4806ca40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a60036000 tx=0x7f3a6003d040 comp rx=0 tx=0).stop 2023-12-01T01:54:28.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.538+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/2473356865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b3ba0 msgr2=0x7f3a640caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:28.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.538+0000 7f3a6b3e9640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b3ba0 0x7f3a640caf70 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00eed0 tx=0x7f3a5c00b480 comp rx=0 tx=0).stop 2023-12-01T01:54:28.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.539+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/2473356865 shutdown_connections 2023-12-01T01:54:28.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.539+0000 7f3a6b3e9640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a4806a580 0x7f3a4806ca40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.539+0000 7f3a6b3e9640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a640b3ba0 0x7f3a640caf70 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.539+0000 7f3a6b3e9640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a640b0e70 0x7f3a640c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.539+0000 7f3a6b3e9640 1 --2- 172.21.15.115:0/2473356865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a6406b6a0 0x7f3a641f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.539+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/2473356865 >> 172.21.15.115:0/2473356865 conn(0x7f3a640ce620 msgr2=0x7f3a6406f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:28.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.539+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/2473356865 shutdown_connections 2023-12-01T01:54:28.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.539+0000 7f3a6b3e9640 1 -- 172.21.15.115:0/2473356865 wait complete. 2023-12-01T01:54:28.560 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T01:54:28.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.770+0000 7f3e24c5f640 1 Processor -- start 2023-12-01T01:54:28.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.770+0000 7f3e24c5f640 1 -- start start 2023-12-01T01:54:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e24c5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e2015d220 0x7f3e200cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:28.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e24c5f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3e200ad2b0 0x7f3e200ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e24c5f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3e200ade80 0x7f3e2006b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e24c5f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2006cc10 con 0x7f3e200ade80 2023-12-01T01:54:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e24c5f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2006c720 con 0x7f3e200ad2b0 2023-12-01T01:54:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e24c5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2015d600 con 0x7f3e2015d220 2023-12-01T01:54:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e1ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3e200ade80 0x7f3e2006b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e1ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3e200ade80 0x7f3e2006b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58414/0 (socket says 172.21.15.115:58414) 2023-12-01T01:54:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e1ed9d640 1 -- 172.21.15.115:0/3295624956 learned_addr learned my addr 172.21.15.115:0/3295624956 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e1e59c640 1 --2- 172.21.15.115:0/3295624956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e2015d220 0x7f3e200cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:28.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.772+0000 7f3e15d9b640 1 --2- 172.21.15.115:0/3295624956 >> v2:172.21.15.142:3300/0 conn(0x7f3e200ad2b0 0x7f3e200ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.773+0000 7f3e1ed9d640 1 -- 172.21.15.115:0/3295624956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e2015d220 msgr2=0x7f3e200cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.773+0000 7f3e1ed9d640 1 --2- 172.21.15.115:0/3295624956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e2015d220 0x7f3e200cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.773+0000 7f3e1ed9d640 1 -- 172.21.15.115:0/3295624956 >> v2:172.21.15.142:3300/0 conn(0x7f3e200ad2b0 msgr2=0x7f3e200ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.773+0000 7f3e1ed9d640 1 --2- 172.21.15.115:0/3295624956 >> v2:172.21.15.142:3300/0 conn(0x7f3e200ad2b0 0x7f3e200ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.773+0000 7f3e1ed9d640 1 -- 172.21.15.115:0/3295624956 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2006c4b0 con 0x7f3e200ade80 2023-12-01T01:54:28.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.774+0000 7f3e1ed9d640 1 --2- 172.21.15.115:0/3295624956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e200ade80 0x7f3e2006b350 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f3e200b1370 tx=0x7f3e1802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de210f372063340 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:28.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.774+0000 7f3e1dd9b640 1 -- 172.21.15.115:0/3295624956 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e180317e0 con 0x7f3e200ade80 2023-12-01T01:54:28.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e1dd9b640 1 -- 172.21.15.115:0/3295624956 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e18031940 con 0x7f3e200ade80 2023-12-01T01:54:28.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e1dd9b640 1 -- 172.21.15.115:0/3295624956 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18031cd0 con 0x7f3e200ade80 2023-12-01T01:54:28.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e24c5f640 1 -- 172.21.15.115:0/3295624956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e200ade80 msgr2=0x7f3e2006b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:28.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e24c5f640 1 --2- 172.21.15.115:0/3295624956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e200ade80 0x7f3e2006b350 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f3e200b1370 tx=0x7f3e1802b880 comp rx=0 tx=0).stop 2023-12-01T01:54:28.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e24c5f640 1 -- 172.21.15.115:0/3295624956 shutdown_connections 2023-12-01T01:54:28.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e24c5f640 1 --2- 172.21.15.115:0/3295624956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e200ade80 0x7f3e2006b350 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e24c5f640 1 --2- 172.21.15.115:0/3295624956 >> v2:172.21.15.142:3300/0 conn(0x7f3e200ad2b0 0x7f3e200ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e24c5f640 1 --2- 172.21.15.115:0/3295624956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e2015d220 0x7f3e200cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e24c5f640 1 -- 172.21.15.115:0/3295624956 >> 172.21.15.115:0/3295624956 conn(0x7f3e200ce880 msgr2=0x7f3e200cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:28.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e24c5f640 1 -- 172.21.15.115:0/3295624956 shutdown_connections 2023-12-01T01:54:28.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.775+0000 7f3e24c5f640 1 -- 172.21.15.115:0/3295624956 wait complete. 2023-12-01T01:54:28.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.777+0000 7f3e24c5f640 1 Processor -- start 2023-12-01T01:54:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.777+0000 7f3e24c5f640 1 -- start start 2023-12-01T01:54:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.778+0000 7f3e24c5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e200ad2b0 0x7f3e201e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.778+0000 7f3e24c5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e200ade80 0x7f3e200b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.778+0000 7f3e24c5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e2015d220 0x7f3e20152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.778+0000 7f3e24c5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e201e7470 con 0x7f3e2015d220 2023-12-01T01:54:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.778+0000 7f3e24c5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e201e5e90 con 0x7f3e200ade80 2023-12-01T01:54:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.778+0000 7f3e24c5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e201e6170 con 0x7f3e200ad2b0 2023-12-01T01:54:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.778+0000 7f3e1ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e2015d220 0x7f3e20152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.779+0000 7f3e1ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e2015d220 0x7f3e20152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58426/0 (socket says 172.21.15.115:58426) 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.779+0000 7f3e1ed9d640 1 -- 172.21.15.115:0/2435026754 learned_addr learned my addr 172.21.15.115:0/2435026754 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.779+0000 7f3e1e59c640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e200ad2b0 0x7f3e201e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.779+0000 7f3e1ed9d640 1 -- 172.21.15.115:0/2435026754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e200ad2b0 msgr2=0x7f3e201e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.779+0000 7f3e1ed9d640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e200ad2b0 0x7f3e201e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.779+0000 7f3e1ed9d640 1 -- 172.21.15.115:0/2435026754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e200ade80 msgr2=0x7f3e200b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.779+0000 7f3e1ed9d640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e200ade80 0x7f3e200b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.779+0000 7f3e1ed9d640 1 -- 172.21.15.115:0/2435026754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e180094d0 con 0x7f3e2015d220 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.779+0000 7f3e1e59c640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e200ad2b0 0x7f3e201e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.779+0000 7f3e1ed9d640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e2015d220 0x7f3e20152c30 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f3e180090b0 tx=0x7f3e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.780+0000 7f3e177fe640 1 -- 172.21.15.115:0/2435026754 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18032880 con 0x7f3e2015d220 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.780+0000 7f3e177fe640 1 -- 172.21.15.115:0/2435026754 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e18032a20 con 0x7f3e2015d220 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.780+0000 7f3e177fe640 1 -- 172.21.15.115:0/2435026754 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18032dd0 con 0x7f3e2015d220 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.780+0000 7f3e24c5f640 1 -- 172.21.15.115:0/2435026754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e201e6450 con 0x7f3e2015d220 2023-12-01T01:54:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.780+0000 7f3e24c5f640 1 -- 172.21.15.115:0/2435026754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e201e6a10 con 0x7f3e2015d220 2023-12-01T01:54:28.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.781+0000 7f3e24c5f640 1 -- 172.21.15.115:0/2435026754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e201e6c00 con 0x7f3e2015d220 2023-12-01T01:54:28.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.782+0000 7f3e177fe640 1 -- 172.21.15.115:0/2435026754 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3e1804e450 con 0x7f3e2015d220 2023-12-01T01:54:28.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.782+0000 7f3e177fe640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e0006a550 0x7f3e0006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:28.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.783+0000 7f3e177fe640 1 -- 172.21.15.115:0/2435026754 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3e180aa9b0 con 0x7f3e2015d220 2023-12-01T01:54:28.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.783+0000 7f3e1e59c640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e0006a550 0x7f3e0006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:28.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.783+0000 7f3e1e59c640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e0006a550 0x7f3e0006ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009f50 tx=0x7f3e1000b040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:28.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:28.787+0000 7f3e177fe640 1 -- 172.21.15.115:0/2435026754 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3e18076030 con 0x7f3e2015d220 2023-12-01T01:54:29.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.055+0000 7f3e24c5f640 1 -- 172.21.15.115:0/2435026754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3e2004e890 con 0x7f3e2015d220 2023-12-01T01:54:29.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.056+0000 7f3e177fe640 1 -- 172.21.15.115:0/2435026754 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3e18079650 con 0x7f3e2015d220 2023-12-01T01:54:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:29.057 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:54:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T01:54:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T01:54:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T01:54:29.057 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:54:29.058 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T01:54:29.059 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T01:54:29.061 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:54:29.062 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T01:54:29.063 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T01:54:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T01:54:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:54:29.065 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T01:54:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:54:29.070 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:54:29.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:29.072 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T01:54:29.072 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:54:29.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 -- 172.21.15.115:0/2435026754 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e0006a550 msgr2=0x7f3e0006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e0006a550 0x7f3e0006ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009f50 tx=0x7f3e1000b040 comp rx=0 tx=0).stop 2023-12-01T01:54:29.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 -- 172.21.15.115:0/2435026754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e2015d220 msgr2=0x7f3e20152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e2015d220 0x7f3e20152c30 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f3e180090b0 tx=0x7f3e180046c0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 -- 172.21.15.115:0/2435026754 shutdown_connections 2023-12-01T01:54:29.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3e0006a550 0x7f3e0006ca10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e2015d220 0x7f3e20152c30 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e200ade80 0x7f3e200b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 --2- 172.21.15.115:0/2435026754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e200ad2b0 0x7f3e201e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 -- 172.21.15.115:0/2435026754 >> 172.21.15.115:0/2435026754 conn(0x7f3e200ce880 msgr2=0x7f3e200ae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:29.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 -- 172.21.15.115:0/2435026754 shutdown_connections 2023-12-01T01:54:29.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.058+0000 7f3e24c5f640 1 -- 172.21.15.115:0/2435026754 wait complete. 2023-12-01T01:54:29.077 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:54:29.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.275+0000 7f2ed59b1640 1 Processor -- start 2023-12-01T01:54:29.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.275+0000 7f2ed59b1640 1 -- start start 2023-12-01T01:54:29.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.276+0000 7f2ed59b1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2ed015e2f0 0x7f2ed00ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.276+0000 7f2ed59b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed00adf10 0x7f2ed00caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.276+0000 7f2ed59b1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed00add10 con 0x7f2ed006f2f0 2023-12-01T01:54:29.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.276+0000 7f2ed59b1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed00c5760 con 0x7f2ed015e2f0 2023-12-01T01:54:29.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.276+0000 7f2ed59b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed015e6d0 con 0x7f2ed00adf10 2023-12-01T01:54:29.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.277+0000 7f2ec7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2ed006f2f0 0x7f2ed00c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59062/0 (socket says 172.21.15.115:59062) 2023-12-01T01:54:29.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.277+0000 7f2ec7fff640 1 -- 172.21.15.115:0/1083215257 learned_addr learned my addr 172.21.15.115:0/1083215257 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.277+0000 7f2ecf7fe640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed00adf10 0x7f2ed00caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.277+0000 7f2eceffd640 1 --2- 172.21.15.115:0/1083215257 >> v2:172.21.15.142:3300/0 conn(0x7f2ed015e2f0 0x7f2ed00ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.277+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764827520 0 0) 0x7f2ed00add10 con 0x7f2ed006f2f0 2023-12-01T01:54:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.277+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ebc003520 con 0x7f2ed006f2f0 2023-12-01T01:54:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.277+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3240502168 0 0) 0x7f2ebc003520 con 0x7f2ed006f2f0 2023-12-01T01:54:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.277+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed00adf10 msgr2=0x7f2ed00caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.277+0000 7f2ece7fc640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed00adf10 0x7f2ed00caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 >> v2:172.21.15.142:3300/0 conn(0x7f2ed015e2f0 msgr2=0x7f2ed00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 --2- 172.21.15.115:0/1083215257 >> v2:172.21.15.142:3300/0 conn(0x7f2ed015e2f0 0x7f2ed00ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed00cf7b0 con 0x7f2ed006f2f0 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2eb8001690 con 0x7f2ed006f2f0 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 >> v1:172.21.15.115:6789/0 conn(0x7f2ed006f2f0 legacy=0x7f2ed00c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc0039e0 0x7f2ebc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed00adf10 0x7f2ebc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc004d00 0x7f2ebc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed00c5760 con 0x7f2ebc0039e0 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed015e6d0 con 0x7f2ebc004d00 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc00bab0 con 0x7f2ed00adf10 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2eceffd640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc0039e0 0x7f2ebc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ec7fff640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed00adf10 0x7f2ebc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2eceffd640 1 -- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed00adf10 msgr2=0x7f2ebc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2eceffd640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed00adf10 0x7f2ebc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ecf7fe640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc004d00 0x7f2ebc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2eceffd640 1 -- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc004d00 msgr2=0x7f2ebc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:29.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2eceffd640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ebc004d00 0x7f2ebc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2eceffd640 1 -- 172.21.15.115:0/1083215257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc003520 con 0x7f2ebc0039e0 2023-12-01T01:54:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.278+0000 7f2ec7fff640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed00adf10 0x7f2ebc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.280+0000 7f2eceffd640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc0039e0 0x7f2ebc003e00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009ac0 tx=0x7f2ebc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6028a8e89a6544a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.280+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec002c710 con 0x7f2ebc0039e0 2023-12-01T01:54:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.280+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec002c870 con 0x7f2ebc0039e0 2023-12-01T01:54:29.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.280+0000 7f2ece7fc640 1 -- 172.21.15.115:0/1083215257 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec002cc00 con 0x7f2ebc0039e0 2023-12-01T01:54:29.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.280+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1083215257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc0039e0 msgr2=0x7f2ebc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.280+0000 7f2ed59b1640 1 --2- 172.21.15.115:0/1083215257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ebc0039e0 0x7f2ebc003e00 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009ac0 tx=0x7f2ebc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.280+0000 7f2ecf7fe640 1 -- 172.21.15.115:0/1083215257 reap_dead start 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.280+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1083215257 shutdown_connections 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.280+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1083215257 >> 172.21.15.115:0/1083215257 conn(0x7f2ed00ce880 msgr2=0x7f2ed00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.280+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1083215257 shutdown_connections 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.281+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1083215257 wait complete. 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.282+0000 7f2ed59b1640 1 Processor -- start 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.282+0000 7f2ed59b1640 1 -- start start 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.283+0000 7f2ed59b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ebc0039e0 0x7f2ed015c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.283+0000 7f2ed59b1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed015b020 0x7f2ed014bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.283+0000 7f2ed59b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ed014c250 0x7f2ed014c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.283+0000 7f2ed59b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed014c6e0 con 0x7f2ed014c250 2023-12-01T01:54:29.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.283+0000 7f2ed59b1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed015c0f0 con 0x7f2ed015b020 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.283+0000 7f2ed59b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed015b4e0 con 0x7f2ebc0039e0 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.284+0000 7f2ecf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ed014c250 0x7f2ed014c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.284+0000 7f2ecf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ed014c250 0x7f2ed014c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58448/0 (socket says 172.21.15.115:58448) 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.284+0000 7f2ecf7fe640 1 -- 172.21.15.115:0/1489465060 learned_addr learned my addr 172.21.15.115:0/1489465060 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.284+0000 7f2eceffd640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ebc0039e0 0x7f2ed015c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.284+0000 7f2ec7fff640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed015b020 0x7f2ed014bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.284+0000 7f2ecf7fe640 1 -- 172.21.15.115:0/1489465060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ebc0039e0 msgr2=0x7f2ed015c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.284+0000 7f2ecf7fe640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ebc0039e0 0x7f2ed015c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.284+0000 7f2ecf7fe640 1 -- 172.21.15.115:0/1489465060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed015b020 msgr2=0x7f2ed014bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.284+0000 7f2ecf7fe640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed015b020 0x7f2ed014bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.284+0000 7f2ecf7fe640 1 -- 172.21.15.115:0/1489465060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed015b7c0 con 0x7f2ed014c250 2023-12-01T01:54:29.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.285+0000 7f2ecf7fe640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ed014c250 0x7f2ed014c980 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f2ed006fba0 tx=0x7f2ec800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.285+0000 7f2eccff9640 1 -- 172.21.15.115:0/1489465060 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8017400 con 0x7f2ed014c250 2023-12-01T01:54:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.285+0000 7f2eccff9640 1 -- 172.21.15.115:0/1489465060 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec8011040 con 0x7f2ed014c250 2023-12-01T01:54:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.285+0000 7f2eccff9640 1 -- 172.21.15.115:0/1489465060 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec80177c0 con 0x7f2ed014c250 2023-12-01T01:54:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.285+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1489465060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed015baa0 con 0x7f2ed014c250 2023-12-01T01:54:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.285+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1489465060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed015cac0 con 0x7f2ed014c250 2023-12-01T01:54:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.286+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1489465060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed006fda0 con 0x7f2ed014c250 2023-12-01T01:54:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.287+0000 7f2eccff9640 1 -- 172.21.15.115:0/1489465060 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f2ec8017920 con 0x7f2ed014c250 2023-12-01T01:54:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.288+0000 7f2eccff9640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2eb006a550 0x7f2eb006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.288+0000 7f2eccff9640 1 -- 172.21.15.115:0/1489465060 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2ec8088cf0 con 0x7f2ed014c250 2023-12-01T01:54:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.288+0000 7f2eceffd640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2eb006a550 0x7f2eb006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.289+0000 7f2eceffd640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2eb006a550 0x7f2eb006ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009ac0 tx=0x7f2ec0030000 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:29.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.292+0000 7f2eccff9640 1 -- 172.21.15.115:0/1489465060 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2ec808d070 con 0x7f2ed014c250 2023-12-01T01:54:29.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.484+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1489465060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ed015c280 con 0x7f2ed014c250 2023-12-01T01:54:29.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.485+0000 7f2eccff9640 1 -- 172.21.15.115:0/1489465060 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+12447 (secure 0 0 0) 0x7f2ec8054410 con 0x7f2ed014c250 2023-12-01T01:54:29.487 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:29.487 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":121,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:54:28.528464+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:54:29.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.488+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1489465060 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2eb006a550 msgr2=0x7f2eb006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.488+0000 7f2ed59b1640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2eb006a550 0x7f2eb006ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009ac0 tx=0x7f2ec0030000 comp rx=0 tx=0).stop 2023-12-01T01:54:29.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.488+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1489465060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ed014c250 msgr2=0x7f2ed014c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.488+0000 7f2ed59b1640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ed014c250 0x7f2ed014c980 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f2ed006fba0 tx=0x7f2ec800f4a0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.488+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1489465060 shutdown_connections 2023-12-01T01:54:29.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.488+0000 7f2ed59b1640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2eb006a550 0x7f2eb006ca10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.488+0000 7f2ed59b1640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ed014c250 0x7f2ed014c980 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.488+0000 7f2ed59b1640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed015b020 0x7f2ed014bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.488+0000 7f2ed59b1640 1 --2- 172.21.15.115:0/1489465060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ebc0039e0 0x7f2ed015c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.488+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1489465060 >> 172.21.15.115:0/1489465060 conn(0x7f2ed00ce880 msgr2=0x7f2ed00caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:29.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.489+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1489465060 shutdown_connections 2023-12-01T01:54:29.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.489+0000 7f2ed59b1640 1 -- 172.21.15.115:0/1489465060 wait complete. 2023-12-01T01:54:29.508 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:54:29.508 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:54:29.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.723+0000 7f86ddac4640 1 Processor -- start 2023-12-01T01:54:29.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.723+0000 7f86ddac4640 1 -- start start 2023-12-01T01:54:29.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.725+0000 7f86ddac4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86d80cfd60 0x7f86d80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.725+0000 7f86ddac4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80b5fa0 0x7f86d8161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.725+0000 7f86ddac4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d80ce5e0 con 0x7f86d815cd90 2023-12-01T01:54:29.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.725+0000 7f86ddac4640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d80d0ec0 con 0x7f86d80cfd60 2023-12-01T01:54:29.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.725+0000 7f86ddac4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d8161950 con 0x7f86d80b5fa0 2023-12-01T01:54:29.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.725+0000 7f86d77fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f86d815cd90 0x7f86d80cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59068/0 (socket says 172.21.15.115:59068) 2023-12-01T01:54:29.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.725+0000 7f86d77fe640 1 -- 172.21.15.115:0/4051416106 learned_addr learned my addr 172.21.15.115:0/4051416106 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:29.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.725+0000 7f86cffff640 1 --2- 172.21.15.115:0/4051416106 >> v2:172.21.15.115:3300/0 conn(0x7f86d80cfd60 0x7f86d80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.726+0000 7f86d6ffd640 1 -- 172.21.15.115:0/4051416106 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902575966 0 0) 0x7f86c8001110 con 0x7f86d815cd90 2023-12-01T01:54:29.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.726+0000 7f86d7fff640 1 --2- 172.21.15.115:0/4051416106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80b5fa0 0x7f86d8161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.726+0000 7f86d6ffd640 1 -- 172.21.15.115:0/4051416106 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c4003520 con 0x7f86d815cd90 2023-12-01T01:54:29.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.726+0000 7f86cffff640 1 -- 172.21.15.115:0/4051416106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80b5fa0 msgr2=0x7f86d8161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:29.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.726+0000 7f86cffff640 1 --2- 172.21.15.115:0/4051416106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80b5fa0 0x7f86d8161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.726+0000 7f86cffff640 1 -- 172.21.15.115:0/4051416106 >> v1:172.21.15.115:6789/0 conn(0x7f86d815cd90 legacy=0x7f86d80cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.726+0000 7f86cffff640 1 -- 172.21.15.115:0/4051416106 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80ce300 con 0x7f86d80cfd60 2023-12-01T01:54:29.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.726+0000 7f86d7fff640 1 --2- 172.21.15.115:0/4051416106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80b5fa0 0x7f86d8161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:29.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.727+0000 7f86cffff640 1 --2- 172.21.15.115:0/4051416106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d80cfd60 0x7f86d80cdd20 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f86c00097a0 tx=0x7f86c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb41952a304db1a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:29.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.727+0000 7f86d6ffd640 1 -- 172.21.15.115:0/4051416106 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0031830 con 0x7f86d80cfd60 2023-12-01T01:54:29.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.727+0000 7f86d6ffd640 1 -- 172.21.15.115:0/4051416106 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0031990 con 0x7f86d80cfd60 2023-12-01T01:54:29.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.727+0000 7f86d6ffd640 1 -- 172.21.15.115:0/4051416106 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0031d20 con 0x7f86d80cfd60 2023-12-01T01:54:29.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.728+0000 7f86ddac4640 1 -- 172.21.15.115:0/4051416106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d80cfd60 msgr2=0x7f86d80cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.728+0000 7f86ddac4640 1 --2- 172.21.15.115:0/4051416106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d80cfd60 0x7f86d80cdd20 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f86c00097a0 tx=0x7f86c002b8d0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.728+0000 7f86ddac4640 1 -- 172.21.15.115:0/4051416106 shutdown_connections 2023-12-01T01:54:29.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.728+0000 7f86ddac4640 1 --2- 172.21.15.115:0/4051416106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80b5fa0 0x7f86d8161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.728+0000 7f86ddac4640 1 --2- 172.21.15.115:0/4051416106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d80cfd60 0x7f86d80cdd20 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.728+0000 7f86ddac4640 1 -- 172.21.15.115:0/4051416106 >> 172.21.15.115:0/4051416106 conn(0x7f86d8065960 msgr2=0x7f86d8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:29.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.728+0000 7f86ddac4640 1 -- 172.21.15.115:0/4051416106 shutdown_connections 2023-12-01T01:54:29.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.728+0000 7f86ddac4640 1 -- 172.21.15.115:0/4051416106 wait complete. 2023-12-01T01:54:29.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.730+0000 7f86ddac4640 1 Processor -- start 2023-12-01T01:54:29.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.730+0000 7f86ddac4640 1 -- start start 2023-12-01T01:54:29.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.731+0000 7f86ddac4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86d80b5fa0 0x7f86d81544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.731+0000 7f86ddac4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80cfd60 0x7f86d815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.731+0000 7f86ddac4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d815cd90 0x7f86d8151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.731+0000 7f86ddac4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d81525a0 con 0x7f86d815cd90 2023-12-01T01:54:29.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.731+0000 7f86ddac4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d815a8f0 con 0x7f86d80b5fa0 2023-12-01T01:54:29.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.731+0000 7f86ddac4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d815ab00 con 0x7f86d80cfd60 2023-12-01T01:54:29.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d815cd90 0x7f86d8151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d815cd90 0x7f86d8151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58476/0 (socket says 172.21.15.115:58476) 2023-12-01T01:54:29.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d7fff640 1 -- 172.21.15.115:0/2304498567 learned_addr learned my addr 172.21.15.115:0/2304498567 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:29.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86cffff640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80cfd60 0x7f86d815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d7fff640 1 -- 172.21.15.115:0/2304498567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80cfd60 msgr2=0x7f86d815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d77fe640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86d80b5fa0 0x7f86d81544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d7fff640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80cfd60 0x7f86d815a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d7fff640 1 -- 172.21.15.115:0/2304498567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86d80b5fa0 msgr2=0x7f86d81544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d7fff640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86d80b5fa0 0x7f86d81544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d7fff640 1 -- 172.21.15.115:0/2304498567 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c0009400 con 0x7f86d815cd90 2023-12-01T01:54:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d7fff640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d815cd90 0x7f86d8151a20 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f86d000cef0 tx=0x7f86d000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.732+0000 7f86d4ff9640 1 -- 172.21.15.115:0/2304498567 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d000d840 con 0x7f86d815cd90 2023-12-01T01:54:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.733+0000 7f86d4ff9640 1 -- 172.21.15.115:0/2304498567 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d000d9c0 con 0x7f86d815cd90 2023-12-01T01:54:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.733+0000 7f86d4ff9640 1 -- 172.21.15.115:0/2304498567 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d000dd70 con 0x7f86d815cd90 2023-12-01T01:54:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.733+0000 7f86ddac4640 1 -- 172.21.15.115:0/2304498567 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d815ade0 con 0x7f86d815cd90 2023-12-01T01:54:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.733+0000 7f86ddac4640 1 -- 172.21.15.115:0/2304498567 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d815b3a0 con 0x7f86d815cd90 2023-12-01T01:54:29.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.734+0000 7f86ddac4640 1 -- 172.21.15.115:0/2304498567 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d804e890 con 0x7f86d815cd90 2023-12-01T01:54:29.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.735+0000 7f86d4ff9640 1 -- 172.21.15.115:0/2304498567 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f86d001e3b0 con 0x7f86d815cd90 2023-12-01T01:54:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.736+0000 7f86d4ff9640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86b806a550 0x7f86b806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.736+0000 7f86d4ff9640 1 -- 172.21.15.115:0/2304498567 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f86d0089180 con 0x7f86d815cd90 2023-12-01T01:54:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.736+0000 7f86d77fe640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86b806a550 0x7f86b806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.737+0000 7f86d77fe640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86b806a550 0x7f86b806ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f86c8001270 tx=0x7f86c8009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.740+0000 7f86d4ff9640 1 -- 172.21.15.115:0/2304498567 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f86d0054b00 con 0x7f86d815cd90 2023-12-01T01:54:29.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.928+0000 7f86ddac4640 1 -- 172.21.15.115:0/2304498567 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86d80d0a70 con 0x7f86d815cd90 2023-12-01T01:54:29.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.929+0000 7f86d4ff9640 1 -- 172.21.15.115:0/2304498567 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86d001a020 con 0x7f86d815cd90 2023-12-01T01:54:29.931 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:54:29.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.931+0000 7f86ddac4640 1 -- 172.21.15.115:0/2304498567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86b806a550 msgr2=0x7f86b806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86b806a550 0x7f86b806ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f86c8001270 tx=0x7f86c8009040 comp rx=0 tx=0).stop 2023-12-01T01:54:29.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 -- 172.21.15.115:0/2304498567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d815cd90 msgr2=0x7f86d8151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:29.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d815cd90 0x7f86d8151a20 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f86d000cef0 tx=0x7f86d000a710 comp rx=0 tx=0).stop 2023-12-01T01:54:29.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 -- 172.21.15.115:0/2304498567 shutdown_connections 2023-12-01T01:54:29.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86b806a550 0x7f86b806ca10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86d80cfd60 0x7f86d815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86d815cd90 0x7f86d8151a20 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 --2- 172.21.15.115:0/2304498567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86d80b5fa0 0x7f86d81544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:29.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 -- 172.21.15.115:0/2304498567 >> 172.21.15.115:0/2304498567 conn(0x7f86d8065960 msgr2=0x7f86d8157050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:29.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 -- 172.21.15.115:0/2304498567 shutdown_connections 2023-12-01T01:54:29.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:29.932+0000 7f86ddac4640 1 -- 172.21.15.115:0/2304498567 wait complete. 2023-12-01T01:54:29.953 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T01:54:30.028 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:54:30.039 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:54:30.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.158+0000 7ff008038640 1 Processor -- start 2023-12-01T01:54:30.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.159+0000 7ff008038640 1 -- start start 2023-12-01T01:54:30.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.160+0000 7ff008038640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff000159dd0 0x7ff0000c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.160+0000 7ff008038640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00006b6f0 0x7ff0000c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.160+0000 7ff008038640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00006f8b0 con 0x7ff0000ade80 2023-12-01T01:54:30.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.160+0000 7ff008038640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0000adcb0 con 0x7ff000159dd0 2023-12-01T01:54:30.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.160+0000 7ff008038640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000ad9d0 con 0x7ff00006b6f0 2023-12-01T01:54:30.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff0065d5640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff0000ade80 0x7ff0000cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59070/0 (socket says 172.21.15.115:59070) 2023-12-01T01:54:30.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff0065d5640 1 -- 172.21.15.115:0/1161763220 learned_addr learned my addr 172.21.15.115:0/1161763220 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:30.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff005dd4640 1 --2- 172.21.15.115:0/1161763220 >> v2:172.21.15.142:3300/0 conn(0x7ff000159dd0 0x7ff0000c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:30.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff0055d3640 1 --2- 172.21.15.115:0/1161763220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00006b6f0 0x7ff0000c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:30.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff004dd2640 1 -- 172.21.15.115:0/1161763220 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336001244 0 0) 0x7ff00006f8b0 con 0x7ff0000ade80 2023-12-01T01:54:30.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff004dd2640 1 -- 172.21.15.115:0/1161763220 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefec003520 con 0x7ff0000ade80 2023-12-01T01:54:30.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff005dd4640 1 -- 172.21.15.115:0/1161763220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00006b6f0 msgr2=0x7ff0000c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff005dd4640 1 --2- 172.21.15.115:0/1161763220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00006b6f0 0x7ff0000c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff005dd4640 1 -- 172.21.15.115:0/1161763220 >> v1:172.21.15.115:6789/0 conn(0x7ff0000ade80 legacy=0x7ff0000cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff004dd2640 1 -- 172.21.15.115:0/1161763220 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2834425116 0 0) 0x7fefec003520 con 0x7ff0000ade80 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.161+0000 7ff005dd4640 1 -- 172.21.15.115:0/1161763220 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000c9000 con 0x7ff000159dd0 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.163+0000 7ff005dd4640 1 --2- 172.21.15.115:0/1161763220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff000159dd0 0x7ff0000c97e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feffc0097a0 tx=0x7feffc02b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a16bbbf8ca69673 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.163+0000 7ff004dd2640 1 -- 172.21.15.115:0/1161763220 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff80019f0 con 0x7ff000159dd0 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.163+0000 7ff004dd2640 1 -- 172.21.15.115:0/1161763220 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feffc02e050 con 0x7ff000159dd0 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.163+0000 7ff004dd2640 1 -- 172.21.15.115:0/1161763220 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc030710 con 0x7ff000159dd0 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.164+0000 7ff008038640 1 -- 172.21.15.115:0/1161763220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff000159dd0 msgr2=0x7ff0000c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.164+0000 7ff008038640 1 --2- 172.21.15.115:0/1161763220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff000159dd0 0x7ff0000c97e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feffc0097a0 tx=0x7feffc02b660 comp rx=0 tx=0).stop 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.164+0000 7ff008038640 1 -- 172.21.15.115:0/1161763220 shutdown_connections 2023-12-01T01:54:30.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.164+0000 7ff008038640 1 --2- 172.21.15.115:0/1161763220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00006b6f0 0x7ff0000c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.164+0000 7ff008038640 1 --2- 172.21.15.115:0/1161763220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff000159dd0 0x7ff0000c97e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.164+0000 7ff008038640 1 -- 172.21.15.115:0/1161763220 >> 172.21.15.115:0/1161763220 conn(0x7ff0000ce880 msgr2=0x7ff0000cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:30.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.164+0000 7ff008038640 1 -- 172.21.15.115:0/1161763220 shutdown_connections 2023-12-01T01:54:30.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.164+0000 7ff008038640 1 -- 172.21.15.115:0/1161763220 wait complete. 2023-12-01T01:54:30.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.166+0000 7ff008038640 1 Processor -- start 2023-12-01T01:54:30.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.166+0000 7ff008038640 1 -- start start 2023-12-01T01:54:30.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.167+0000 7ff008038640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00006b6f0 0x7ff000150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.167+0000 7ff008038640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0000ade80 0x7ff00014e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.167+0000 7ff008038640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff000159dd0 0x7ff00014d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.167+0000 7ff008038640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00014ed50 con 0x7ff000159dd0 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.167+0000 7ff008038640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001f5f30 con 0x7ff0000ade80 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.167+0000 7ff008038640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001f5600 con 0x7ff00006b6f0 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.167+0000 7ff0065d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff000159dd0 0x7ff00014d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.167+0000 7ff0065d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff000159dd0 0x7ff00014d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58490/0 (socket says 172.21.15.115:58490) 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.167+0000 7ff0065d5640 1 -- 172.21.15.115:0/3539406258 learned_addr learned my addr 172.21.15.115:0/3539406258 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7ff0065d5640 1 -- 172.21.15.115:0/3539406258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00006b6f0 msgr2=0x7ff000150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7ff0065d5640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00006b6f0 0x7ff000150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7ff0065d5640 1 -- 172.21.15.115:0/3539406258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0000ade80 msgr2=0x7ff00014e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7ff0065d5640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0000ade80 0x7ff00014e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7ff0065d5640 1 -- 172.21.15.115:0/3539406258 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc009400 con 0x7ff000159dd0 2023-12-01T01:54:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7ff0065d5640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff000159dd0 0x7ff00014d650 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7feff8001830 tx=0x7feff800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7feff67fc640 1 -- 172.21.15.115:0/3539406258 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8012400 con 0x7ff000159dd0 2023-12-01T01:54:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7feff67fc640 1 -- 172.21.15.115:0/3539406258 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff8012560 con 0x7ff000159dd0 2023-12-01T01:54:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7feff67fc640 1 -- 172.21.15.115:0/3539406258 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8012910 con 0x7ff000159dd0 2023-12-01T01:54:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7ff008038640 1 -- 172.21.15.115:0/3539406258 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0001f58e0 con 0x7ff000159dd0 2023-12-01T01:54:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.168+0000 7ff008038640 1 -- 172.21.15.115:0/3539406258 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00014ea00 con 0x7ff000159dd0 2023-12-01T01:54:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.169+0000 7ff008038640 1 -- 172.21.15.115:0/3539406258 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00004e890 con 0x7ff000159dd0 2023-12-01T01:54:30.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.170+0000 7feff67fc640 1 -- 172.21.15.115:0/3539406258 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7feff8012a70 con 0x7ff000159dd0 2023-12-01T01:54:30.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.172+0000 7feff67fc640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fefe406a550 0x7fefe406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.172+0000 7feff67fc640 1 -- 172.21.15.115:0/3539406258 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feff8088f80 con 0x7ff000159dd0 2023-12-01T01:54:30.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.172+0000 7ff005dd4640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fefe406a550 0x7fefe406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:30.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.173+0000 7ff005dd4640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fefe406a550 0x7fefe406ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feffc034000 tx=0x7feffc02f000 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:30.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.176+0000 7feff67fc640 1 -- 172.21.15.115:0/3539406258 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7feff8054710 con 0x7ff000159dd0 2023-12-01T01:54:30.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.369+0000 7ff008038640 1 -- 172.21.15.115:0/3539406258 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff0001f6070 con 0x7ff000159dd0 2023-12-01T01:54:30.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.370+0000 7feff67fc640 1 -- 172.21.15.115:0/3539406258 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v121) v1 ==== 84+0+65 (secure 0 0 0) 0x7feff8057d30 con 0x7ff000159dd0 2023-12-01T01:54:30.371 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:30.371 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:54:30.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:30.371 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T01:54:30.371 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T01:54:30.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:30.371 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:54:30.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 -- 172.21.15.115:0/3539406258 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fefe406a550 msgr2=0x7fefe406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:30.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fefe406a550 0x7fefe406ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feffc034000 tx=0x7feffc02f000 comp rx=0 tx=0).stop 2023-12-01T01:54:30.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 -- 172.21.15.115:0/3539406258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff000159dd0 msgr2=0x7ff00014d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:30.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff000159dd0 0x7ff00014d650 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7feff8001830 tx=0x7feff800a430 comp rx=0 tx=0).stop 2023-12-01T01:54:30.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 -- 172.21.15.115:0/3539406258 shutdown_connections 2023-12-01T01:54:30.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fefe406a550 0x7fefe406ca10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff000159dd0 0x7ff00014d650 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0000ade80 0x7ff00014e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 --2- 172.21.15.115:0/3539406258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff00006b6f0 0x7ff000150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 -- 172.21.15.115:0/3539406258 >> 172.21.15.115:0/3539406258 conn(0x7ff0000ce880 msgr2=0x7ff0000af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:30.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 -- 172.21.15.115:0/3539406258 shutdown_connections 2023-12-01T01:54:30.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.372+0000 7ff008038640 1 -- 172.21.15.115:0/3539406258 wait complete. 2023-12-01T01:54:30.393 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T01:54:30.393 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T01:54:30.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.613+0000 7fdc17aee640 1 Processor -- start 2023-12-01T01:54:30.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.613+0000 7fdc17aee640 1 -- start start 2023-12-01T01:54:30.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc17aee640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc1015ac70 0x7fdc100cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc17aee640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdc100adca0 0x7fdc100cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc17aee640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1006c860 con 0x7fdc100b4070 2023-12-01T01:54:30.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc17aee640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1015b050 con 0x7fdc1015ac70 2023-12-01T01:54:30.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc17aee640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc100ce0b0 con 0x7fdc100adca0 2023-12-01T01:54:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc1588a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc1015ac70 0x7fdc100cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc1588a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc1015ac70 0x7fdc100cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58504/0 (socket says 172.21.15.115:58504) 2023-12-01T01:54:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc1588a640 1 -- 172.21.15.115:0/3184995207 learned_addr learned my addr 172.21.15.115:0/3184995207 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc1588a640 1 -- 172.21.15.115:0/3184995207 >> v2:172.21.15.142:3300/0 conn(0x7fdc100adca0 msgr2=0x7fdc100cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc1588a640 1 --2- 172.21.15.115:0/3184995207 >> v2:172.21.15.142:3300/0 conn(0x7fdc100adca0 0x7fdc100cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc1588a640 1 -- 172.21.15.115:0/3184995207 >> v1:172.21.15.142:6789/0 conn(0x7fdc100b4070 legacy=0x7fdc1006b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:30.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.615+0000 7fdc1588a640 1 -- 172.21.15.115:0/3184995207 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc100ce390 con 0x7fdc1015ac70 2023-12-01T01:54:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.617+0000 7fdc1588a640 1 --2- 172.21.15.115:0/3184995207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc1015ac70 0x7fdc100cb1b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fdc100c2060 tx=0x7fdc0c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77229825241451e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.617+0000 7fdc14888640 1 -- 172.21.15.115:0/3184995207 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0317e0 con 0x7fdc1015ac70 2023-12-01T01:54:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.617+0000 7fdc14888640 1 -- 172.21.15.115:0/3184995207 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c031940 con 0x7fdc1015ac70 2023-12-01T01:54:30.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.617+0000 7fdc14888640 1 -- 172.21.15.115:0/3184995207 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c031cd0 con 0x7fdc1015ac70 2023-12-01T01:54:30.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.617+0000 7fdc17aee640 1 -- 172.21.15.115:0/3184995207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc1015ac70 msgr2=0x7fdc100cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:30.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.617+0000 7fdc17aee640 1 --2- 172.21.15.115:0/3184995207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc1015ac70 0x7fdc100cb1b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fdc100c2060 tx=0x7fdc0c02b880 comp rx=0 tx=0).stop 2023-12-01T01:54:30.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.618+0000 7fdc17aee640 1 -- 172.21.15.115:0/3184995207 shutdown_connections 2023-12-01T01:54:30.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.618+0000 7fdc17aee640 1 --2- 172.21.15.115:0/3184995207 >> v2:172.21.15.142:3300/0 conn(0x7fdc100adca0 0x7fdc100cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.618+0000 7fdc17aee640 1 --2- 172.21.15.115:0/3184995207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc1015ac70 0x7fdc100cb1b0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.618+0000 7fdc17aee640 1 -- 172.21.15.115:0/3184995207 >> 172.21.15.115:0/3184995207 conn(0x7fdc100662c0 msgr2=0x7fdc100b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:30.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.618+0000 7fdc17aee640 1 -- 172.21.15.115:0/3184995207 shutdown_connections 2023-12-01T01:54:30.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.618+0000 7fdc17aee640 1 -- 172.21.15.115:0/3184995207 wait complete. 2023-12-01T01:54:30.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.619+0000 7fdc17aee640 1 Processor -- start 2023-12-01T01:54:30.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.620+0000 7fdc17aee640 1 -- start start 2023-12-01T01:54:30.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.620+0000 7fdc17aee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc100adca0 0x7fdc10149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.620+0000 7fdc17aee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc100b4070 0x7fdc100c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.620+0000 7fdc17aee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1015ac70 0x7fdc100b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.620+0000 7fdc17aee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100c5a40 con 0x7fdc100b4070 2023-12-01T01:54:30.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.620+0000 7fdc17aee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100c4fb0 con 0x7fdc1015ac70 2023-12-01T01:54:30.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.620+0000 7fdc17aee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100c5c90 con 0x7fdc100adca0 2023-12-01T01:54:30.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.621+0000 7fdc15089640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc100b4070 0x7fdc100c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:30.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.621+0000 7fdc15089640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc100b4070 0x7fdc100c22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58518/0 (socket says 172.21.15.115:58518) 2023-12-01T01:54:30.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.621+0000 7fdc15089640 1 -- 172.21.15.115:0/2236045056 learned_addr learned my addr 172.21.15.115:0/2236045056 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:30.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.621+0000 7fdc15089640 1 -- 172.21.15.115:0/2236045056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc100adca0 msgr2=0x7fdc10149ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.621+0000 7fdc15089640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc100adca0 0x7fdc10149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.621+0000 7fdc15089640 1 -- 172.21.15.115:0/2236045056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1015ac70 msgr2=0x7fdc100b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.621+0000 7fdc15089640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1015ac70 0x7fdc100b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.621+0000 7fdc15089640 1 -- 172.21.15.115:0/2236045056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c0094d0 con 0x7fdc100b4070 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.621+0000 7fdc15089640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc100b4070 0x7fdc100c22d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000b3c0 tx=0x7fdc0000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.622+0000 7fdc067fc640 1 -- 172.21.15.115:0/2236045056 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00011830 con 0x7fdc100b4070 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.622+0000 7fdc067fc640 1 -- 172.21.15.115:0/2236045056 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc00011990 con 0x7fdc100b4070 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.622+0000 7fdc067fc640 1 -- 172.21.15.115:0/2236045056 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00011d60 con 0x7fdc100b4070 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.622+0000 7fdc17aee640 1 -- 172.21.15.115:0/2236045056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc100c5f70 con 0x7fdc100b4070 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.622+0000 7fdc17aee640 1 -- 172.21.15.115:0/2236045056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1014a000 con 0x7fdc100b4070 2023-12-01T01:54:30.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.623+0000 7fdc17aee640 1 -- 172.21.15.115:0/2236045056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc100c6490 con 0x7fdc100b4070 2023-12-01T01:54:30.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.624+0000 7fdc067fc640 1 -- 172.21.15.115:0/2236045056 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fdc0001b460 con 0x7fdc100b4070 2023-12-01T01:54:30.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.625+0000 7fdc067fc640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbf406a550 0x7fdbf406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:30.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.625+0000 7fdc067fc640 1 -- 172.21.15.115:0/2236045056 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdc00088410 con 0x7fdc100b4070 2023-12-01T01:54:30.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.625+0000 7fdc1588a640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbf406a550 0x7fdbf406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:30.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.626+0000 7fdc1588a640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbf406a550 0x7fdbf406ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c036000 tx=0x7fdc0c030000 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:30.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.629+0000 7fdc067fc640 1 -- 172.21.15.115:0/2236045056 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdc100c6490 con 0x7fdc100b4070 2023-12-01T01:54:30.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:30.818+0000 7fdc17aee640 1 -- 172.21.15.115:0/2236045056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdc100c6490 con 0x7fdc100b4070 2023-12-01T01:54:31.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.565+0000 7fdc067fc640 1 -- 172.21.15.115:0/2236045056 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v122) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdc100c6490 con 0x7fdc100b4070 2023-12-01T01:54:31.567 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T01:54:31.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 -- 172.21.15.115:0/2236045056 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbf406a550 msgr2=0x7fdbf406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbf406a550 0x7fdbf406ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c036000 tx=0x7fdc0c030000 comp rx=0 tx=0).stop 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 -- 172.21.15.115:0/2236045056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc100b4070 msgr2=0x7fdc100c22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc100b4070 0x7fdc100c22d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000b3c0 tx=0x7fdc0000b880 comp rx=0 tx=0).stop 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 -- 172.21.15.115:0/2236045056 shutdown_connections 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdbf406a550 0x7fdbf406ca10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1015ac70 0x7fdc100b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc100b4070 0x7fdc100c22d0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 --2- 172.21.15.115:0/2236045056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc100adca0 0x7fdc10149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 -- 172.21.15.115:0/2236045056 >> 172.21.15.115:0/2236045056 conn(0x7fdc100662c0 msgr2=0x7fdc100ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 -- 172.21.15.115:0/2236045056 shutdown_connections 2023-12-01T01:54:31.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.569+0000 7fdc17aee640 1 -- 172.21.15.115:0/2236045056 wait complete. 2023-12-01T01:54:31.597 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T01:54:31.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.809+0000 7f2831c77640 1 Processor -- start 2023-12-01T01:54:31.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.809+0000 7f2831c77640 1 -- start start 2023-12-01T01:54:31.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f2831c77640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f282c1552b0 0x7f282c0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:31.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f2831c77640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f282c06c1c0 0x7f282c0ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:31.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f2831c77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f282c0af700 0x7f282c0afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:31.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f2831c77640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282c155690 con 0x7f282c1552b0 2023-12-01T01:54:31.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f2831c77640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282c06bda0 con 0x7f282c06c1c0 2023-12-01T01:54:31.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f2831c77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282c06b400 con 0x7f282c0af700 2023-12-01T01:54:31.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f282b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f282c1552b0 0x7f282c0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:31.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f282b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f282c1552b0 0x7f282c0ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58526/0 (socket says 172.21.15.115:58526) 2023-12-01T01:54:31.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f282b7fe640 1 -- 172.21.15.115:0/1024617118 learned_addr learned my addr 172.21.15.115:0/1024617118 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:31.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f282bfff640 1 --2- 172.21.15.115:0/1024617118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f282c0af700 0x7f282c0afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:31.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f282b7fe640 1 -- 172.21.15.115:0/1024617118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f282c0af700 msgr2=0x7f282c0afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:31.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f2823fff640 1 --2- 172.21.15.115:0/1024617118 >> v2:172.21.15.142:3300/0 conn(0x7f282c06c1c0 0x7f282c0ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:31.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f282b7fe640 1 --2- 172.21.15.115:0/1024617118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f282c0af700 0x7f282c0afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.811+0000 7f282b7fe640 1 -- 172.21.15.115:0/1024617118 >> v2:172.21.15.142:3300/0 conn(0x7f282c06c1c0 msgr2=0x7f282c0ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:31.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.812+0000 7f282b7fe640 1 --2- 172.21.15.115:0/1024617118 >> v2:172.21.15.142:3300/0 conn(0x7f282c06c1c0 0x7f282c0ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.812+0000 7f282b7fe640 1 -- 172.21.15.115:0/1024617118 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c06b6e0 con 0x7f282c1552b0 2023-12-01T01:54:31.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.813+0000 7f282b7fe640 1 --2- 172.21.15.115:0/1024617118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 0x7f282c0ce760 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f282c0b3ca0 tx=0x7f281c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6122137083d06c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:31.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.813+0000 7f282affd640 1 -- 172.21.15.115:0/1024617118 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c031870 con 0x7f282c1552b0 2023-12-01T01:54:31.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.813+0000 7f282affd640 1 -- 172.21.15.115:0/1024617118 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c0319d0 con 0x7f282c1552b0 2023-12-01T01:54:31.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.813+0000 7f282affd640 1 -- 172.21.15.115:0/1024617118 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c031d60 con 0x7f282c1552b0 2023-12-01T01:54:31.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.814+0000 7f2831c77640 1 -- 172.21.15.115:0/1024617118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 msgr2=0x7f282c0ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:31.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.814+0000 7f2831c77640 1 --2- 172.21.15.115:0/1024617118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 0x7f282c0ce760 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f282c0b3ca0 tx=0x7f281c02b910 comp rx=0 tx=0).stop 2023-12-01T01:54:31.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.814+0000 7f2831c77640 1 -- 172.21.15.115:0/1024617118 shutdown_connections 2023-12-01T01:54:31.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.814+0000 7f2831c77640 1 --2- 172.21.15.115:0/1024617118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f282c0af700 0x7f282c0afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.814+0000 7f2831c77640 1 --2- 172.21.15.115:0/1024617118 >> v2:172.21.15.142:3300/0 conn(0x7f282c06c1c0 0x7f282c0ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.814+0000 7f2831c77640 1 --2- 172.21.15.115:0/1024617118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 0x7f282c0ce760 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.814+0000 7f2831c77640 1 -- 172.21.15.115:0/1024617118 >> 172.21.15.115:0/1024617118 conn(0x7f282c065960 msgr2=0x7f282c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:31.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.814+0000 7f2831c77640 1 -- 172.21.15.115:0/1024617118 shutdown_connections 2023-12-01T01:54:31.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.814+0000 7f2831c77640 1 -- 172.21.15.115:0/1024617118 wait complete. 2023-12-01T01:54:31.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.816+0000 7f2831c77640 1 Processor -- start 2023-12-01T01:54:31.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.816+0000 7f2831c77640 1 -- start start 2023-12-01T01:54:31.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f2831c77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c06c1c0 0x7f282c1f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:31.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f2831c77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f282c0af700 0x7f282c1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:31.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f2831c77640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 0x7f282c1e24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:31.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f2831c77640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c0b1880 con 0x7f282c1552b0 2023-12-01T01:54:31.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f2831c77640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c1f5970 con 0x7f282c06c1c0 2023-12-01T01:54:31.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f2831c77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282c1f2440 con 0x7f282c0af700 2023-12-01T01:54:31.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f282bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 0x7f282c1e24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:31.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f282bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 0x7f282c1e24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58536/0 (socket says 172.21.15.115:58536) 2023-12-01T01:54:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f282bfff640 1 -- 172.21.15.115:0/2280038663 learned_addr learned my addr 172.21.15.115:0/2280038663 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f2823fff640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f282c0af700 0x7f282c1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f282b7fe640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c06c1c0 0x7f282c1f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f282bfff640 1 -- 172.21.15.115:0/2280038663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f282c0af700 msgr2=0x7f282c1f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f282bfff640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f282c0af700 0x7f282c1f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f282bfff640 1 -- 172.21.15.115:0/2280038663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c06c1c0 msgr2=0x7f282c1f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f282bfff640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c06c1c0 0x7f282c1f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.817+0000 7f282bfff640 1 -- 172.21.15.115:0/2280038663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0094d0 con 0x7f282c1552b0 2023-12-01T01:54:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.818+0000 7f282bfff640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 0x7f282c1e24e0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f282400cda0 tx=0x7f282400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:31.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.818+0000 7f2828ff9640 1 -- 172.21.15.115:0/2280038663 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824003b90 con 0x7f282c1552b0 2023-12-01T01:54:31.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.818+0000 7f2828ff9640 1 -- 172.21.15.115:0/2280038663 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2824003cf0 con 0x7f282c1552b0 2023-12-01T01:54:31.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.818+0000 7f2828ff9640 1 -- 172.21.15.115:0/2280038663 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824004240 con 0x7f282c1552b0 2023-12-01T01:54:31.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.818+0000 7f2831c77640 1 -- 172.21.15.115:0/2280038663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282c0b1ad0 con 0x7f282c1552b0 2023-12-01T01:54:31.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.818+0000 7f2831c77640 1 -- 172.21.15.115:0/2280038663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c0b2090 con 0x7f282c1552b0 2023-12-01T01:54:31.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.819+0000 7f2831c77640 1 -- 172.21.15.115:0/2280038663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282c154050 con 0x7f282c1552b0 2023-12-01T01:54:31.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.822+0000 7f2828ff9640 1 -- 172.21.15.115:0/2280038663 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f28240043a0 con 0x7f282c1552b0 2023-12-01T01:54:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.822+0000 7f2828ff9640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f280c06a550 0x7f280c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.823+0000 7f2828ff9640 1 -- 172.21.15.115:0/2280038663 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2824012070 con 0x7f282c1552b0 2023-12-01T01:54:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.823+0000 7f282b7fe640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f280c06a550 0x7f280c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.824+0000 7f282b7fe640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f280c06a550 0x7f280c06ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f281c036000 tx=0x7f281c03c040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:31.826+0000 7f2828ff9640 1 -- 172.21.15.115:0/2280038663 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2824054e10 con 0x7f282c1552b0 2023-12-01T01:54:32.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.021+0000 7f2831c77640 1 -- 172.21.15.115:0/2280038663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f282c04e890 con 0x7f282c1552b0 2023-12-01T01:54:32.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.581+0000 7f2828ff9640 1 -- 172.21.15.115:0/2280038663 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v123) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2824058430 con 0x7f282c1552b0 2023-12-01T01:54:32.587 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T01:54:32.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.588+0000 7f2831c77640 1 -- 172.21.15.115:0/2280038663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f280c06a550 msgr2=0x7f280c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:32.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.588+0000 7f2831c77640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f280c06a550 0x7f280c06ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f281c036000 tx=0x7f281c03c040 comp rx=0 tx=0).stop 2023-12-01T01:54:32.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.588+0000 7f2831c77640 1 -- 172.21.15.115:0/2280038663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 msgr2=0x7f282c1e24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:32.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.588+0000 7f2831c77640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 0x7f282c1e24e0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f282400cda0 tx=0x7f282400a660 comp rx=0 tx=0).stop 2023-12-01T01:54:32.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.589+0000 7f2831c77640 1 -- 172.21.15.115:0/2280038663 shutdown_connections 2023-12-01T01:54:32.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.589+0000 7f2831c77640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f280c06a550 0x7f280c06ca10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.589+0000 7f2831c77640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f282c0af700 0x7f282c1f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.589+0000 7f2831c77640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f282c1552b0 0x7f282c1e24e0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.589+0000 7f2831c77640 1 --2- 172.21.15.115:0/2280038663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c06c1c0 0x7f282c1f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.589+0000 7f2831c77640 1 -- 172.21.15.115:0/2280038663 >> 172.21.15.115:0/2280038663 conn(0x7f282c065960 msgr2=0x7f282c153940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:32.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.592+0000 7f2831c77640 1 -- 172.21.15.115:0/2280038663 shutdown_connections 2023-12-01T01:54:32.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.592+0000 7f2831c77640 1 -- 172.21.15.115:0/2280038663 wait complete. 2023-12-01T01:54:32.620 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T01:54:32.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.806+0000 7fe037405640 1 Processor -- start 2023-12-01T01:54:32.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.806+0000 7fe037405640 1 -- start start 2023-12-01T01:54:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.807+0000 7fe037405640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe030142870 0x7fe0300ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.807+0000 7fe037405640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe0300cd010 0x7fe0300b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.807+0000 7fe037405640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe0300b1c50 0x7fe03015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.807+0000 7fe037405640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0300b3ae0 con 0x7fe0300b1c50 2023-12-01T01:54:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.807+0000 7fe037405640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe030142c50 con 0x7fe0300cd010 2023-12-01T01:54:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.807+0000 7fe037405640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300cd470 con 0x7fe030142870 2023-12-01T01:54:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.808+0000 7fe0359a2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe0300b1c50 0x7fe03015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.808+0000 7fe0359a2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe0300b1c50 0x7fe03015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58540/0 (socket says 172.21.15.115:58540) 2023-12-01T01:54:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.808+0000 7fe0359a2640 1 -- 172.21.15.115:0/2742315776 learned_addr learned my addr 172.21.15.115:0/2742315776 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:32.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.808+0000 7fe0349a0640 1 --2- 172.21.15.115:0/2742315776 >> v2:172.21.15.142:3300/0 conn(0x7fe0300cd010 0x7fe0300b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:32.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.808+0000 7fe0351a1640 1 --2- 172.21.15.115:0/2742315776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe030142870 0x7fe0300ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:32.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.808+0000 7fe0359a2640 1 -- 172.21.15.115:0/2742315776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe030142870 msgr2=0x7fe0300ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:32.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.808+0000 7fe0359a2640 1 --2- 172.21.15.115:0/2742315776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe030142870 0x7fe0300ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.808+0000 7fe0359a2640 1 -- 172.21.15.115:0/2742315776 >> v2:172.21.15.142:3300/0 conn(0x7fe0300cd010 msgr2=0x7fe0300b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:32.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.809+0000 7fe0359a2640 1 --2- 172.21.15.115:0/2742315776 >> v2:172.21.15.142:3300/0 conn(0x7fe0300cd010 0x7fe0300b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.809+0000 7fe0359a2640 1 -- 172.21.15.115:0/2742315776 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300b20e0 con 0x7fe0300b1c50 2023-12-01T01:54:32.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.810+0000 7fe0359a2640 1 --2- 172.21.15.115:0/2742315776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300b1c50 0x7fe03015d690 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe03006d4c0 tx=0x7fe02802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27382f03964d0d0a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:32.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.810+0000 7fe027fff640 1 -- 172.21.15.115:0/2742315776 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02802ea80 con 0x7fe0300b1c50 2023-12-01T01:54:32.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.810+0000 7fe027fff640 1 -- 172.21.15.115:0/2742315776 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02802ebe0 con 0x7fe0300b1c50 2023-12-01T01:54:32.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.810+0000 7fe027fff640 1 -- 172.21.15.115:0/2742315776 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028009600 con 0x7fe0300b1c50 2023-12-01T01:54:32.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.811+0000 7fe037405640 1 -- 172.21.15.115:0/2742315776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300b1c50 msgr2=0x7fe03015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:32.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.811+0000 7fe037405640 1 --2- 172.21.15.115:0/2742315776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300b1c50 0x7fe03015d690 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fe03006d4c0 tx=0x7fe02802e0e0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.811+0000 7fe037405640 1 -- 172.21.15.115:0/2742315776 shutdown_connections 2023-12-01T01:54:32.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.811+0000 7fe037405640 1 --2- 172.21.15.115:0/2742315776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300b1c50 0x7fe03015d690 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.811+0000 7fe037405640 1 --2- 172.21.15.115:0/2742315776 >> v2:172.21.15.142:3300/0 conn(0x7fe0300cd010 0x7fe0300b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.811+0000 7fe037405640 1 --2- 172.21.15.115:0/2742315776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe030142870 0x7fe0300ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.811+0000 7fe037405640 1 -- 172.21.15.115:0/2742315776 >> 172.21.15.115:0/2742315776 conn(0x7fe030065960 msgr2=0x7fe030065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:32.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.811+0000 7fe037405640 1 -- 172.21.15.115:0/2742315776 shutdown_connections 2023-12-01T01:54:32.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.811+0000 7fe037405640 1 -- 172.21.15.115:0/2742315776 wait complete. 2023-12-01T01:54:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.813+0000 7fe037405640 1 Processor -- start 2023-12-01T01:54:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.813+0000 7fe037405640 1 -- start start 2023-12-01T01:54:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.814+0000 7fe037405640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0300b1c50 0x7fe0301f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.814+0000 7fe037405640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300cd010 0x7fe030141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.814+0000 7fe037405640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe030142870 0x7fe03014b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.814+0000 7fe037405640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300c6b80 con 0x7fe0300cd010 2023-12-01T01:54:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.814+0000 7fe037405640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300c6d60 con 0x7fe0300b1c50 2023-12-01T01:54:32.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.814+0000 7fe037405640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300c7040 con 0x7fe030142870 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.814+0000 7fe0349a0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300cd010 0x7fe030141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.814+0000 7fe0349a0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300cd010 0x7fe030141880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58542/0 (socket says 172.21.15.115:58542) 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.814+0000 7fe0349a0640 1 -- 172.21.15.115:0/4190536534 learned_addr learned my addr 172.21.15.115:0/4190536534 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe0349a0640 1 -- 172.21.15.115:0/4190536534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe030142870 msgr2=0x7fe03014b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe0349a0640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe030142870 0x7fe03014b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe0349a0640 1 -- 172.21.15.115:0/4190536534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0300b1c50 msgr2=0x7fe0301f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe0349a0640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0300b1c50 0x7fe0301f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe0349a0640 1 -- 172.21.15.115:0/4190536534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02800bd40 con 0x7fe0300cd010 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe0349a0640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300cd010 0x7fe030141880 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe02000d800 tx=0x7fe02000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe025ffb640 1 -- 172.21.15.115:0/4190536534 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200096a0 con 0x7fe0300cd010 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe025ffb640 1 -- 172.21.15.115:0/4190536534 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020009800 con 0x7fe0300cd010 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe037405640 1 -- 172.21.15.115:0/4190536534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0300c7320 con 0x7fe0300cd010 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe025ffb640 1 -- 172.21.15.115:0/4190536534 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020009b90 con 0x7fe0300cd010 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.815+0000 7fe037405640 1 -- 172.21.15.115:0/4190536534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0300c78e0 con 0x7fe0300cd010 2023-12-01T01:54:32.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.816+0000 7fe037405640 1 -- 172.21.15.115:0/4190536534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03004e890 con 0x7fe0300cd010 2023-12-01T01:54:32.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.818+0000 7fe025ffb640 1 -- 172.21.15.115:0/4190536534 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe020009cf0 con 0x7fe0300cd010 2023-12-01T01:54:32.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.819+0000 7fe025ffb640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe01406a550 0x7fe01406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:32.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.819+0000 7fe025ffb640 1 -- 172.21.15.115:0/4190536534 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe0200895d0 con 0x7fe0300cd010 2023-12-01T01:54:32.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.819+0000 7fe0351a1640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe01406a550 0x7fe01406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:32.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.820+0000 7fe0351a1640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe01406a550 0x7fe01406ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c008040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:32.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:32.823+0000 7fe025ffb640 1 -- 172.21.15.115:0/4190536534 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe020054c80 con 0x7fe0300cd010 2023-12-01T01:54:33.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.045+0000 7fe037405640 1 -- 172.21.15.115:0/4190536534 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe0300707f0 con 0x7fe0300cd010 2023-12-01T01:54:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.056+0000 7fe025ffb640 1 -- 172.21.15.115:0/4190536534 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe02000b440 con 0x7fe0300cd010 2023-12-01T01:54:33.058 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 14 and data pool 15 2023-12-01T01:54:33.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.060+0000 7fe037405640 1 -- 172.21.15.115:0/4190536534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe01406a550 msgr2=0x7fe01406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.060+0000 7fe037405640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe01406a550 0x7fe01406ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c008040 comp rx=0 tx=0).stop 2023-12-01T01:54:33.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.060+0000 7fe037405640 1 -- 172.21.15.115:0/4190536534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300cd010 msgr2=0x7fe030141880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.060+0000 7fe037405640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300cd010 0x7fe030141880 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe02000d800 tx=0x7fe02000dcc0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.061+0000 7fe037405640 1 -- 172.21.15.115:0/4190536534 shutdown_connections 2023-12-01T01:54:33.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.061+0000 7fe037405640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe01406a550 0x7fe01406ca10 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe02c006fd0 tx=0x7fe02c008040 comp rx=0 tx=0).stop 2023-12-01T01:54:33.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.061+0000 7fe037405640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe030142870 0x7fe03014b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.061+0000 7fe037405640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300cd010 0x7fe030141880 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.061+0000 7fe037405640 1 --2- 172.21.15.115:0/4190536534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0300b1c50 0x7fe0301f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.061+0000 7fe037405640 1 -- 172.21.15.115:0/4190536534 >> 172.21.15.115:0/4190536534 conn(0x7fe030065960 msgr2=0x7fe03006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:33.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.062+0000 7fe037405640 1 -- 172.21.15.115:0/4190536534 shutdown_connections 2023-12-01T01:54:33.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.063+0000 7fe037405640 1 -- 172.21.15.115:0/4190536534 wait complete. 2023-12-01T01:54:33.092 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:54:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.269+0000 7fe7d23b0640 1 Processor -- start 2023-12-01T01:54:33.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.269+0000 7fe7d23b0640 1 -- start start 2023-12-01T01:54:33.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.271+0000 7fe7d23b0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe7cc1584b0 0x7fe7cc0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.271+0000 7fe7d23b0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc15afe0 0x7fe7cc06b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.271+0000 7fe7d23b0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc0cc180 con 0x7fe7cc06b7b0 2023-12-01T01:54:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.271+0000 7fe7d23b0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc06bc40 con 0x7fe7cc1584b0 2023-12-01T01:54:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.271+0000 7fe7d23b0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc15b550 con 0x7fe7cc15afe0 2023-12-01T01:54:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.271+0000 7fe7d13ae640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe7cc1584b0 0x7fe7cc0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.271+0000 7fe7d13ae640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe7cc1584b0 0x7fe7cc0cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58548/0 (socket says 172.21.15.115:58548) 2023-12-01T01:54:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.271+0000 7fe7d13ae640 1 -- 172.21.15.115:0/3326507935 learned_addr learned my addr 172.21.15.115:0/3326507935 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.271+0000 7fe7d0bad640 1 --2- 172.21.15.115:0/3326507935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc15afe0 0x7fe7cc06b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.272+0000 7fe7d13ae640 1 -- 172.21.15.115:0/3326507935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc15afe0 msgr2=0x7fe7cc06b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.272+0000 7fe7d13ae640 1 --2- 172.21.15.115:0/3326507935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc15afe0 0x7fe7cc06b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.272+0000 7fe7d13ae640 1 -- 172.21.15.115:0/3326507935 >> v1:172.21.15.142:6789/0 conn(0x7fe7cc06b7b0 legacy=0x7fe7cc15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.272+0000 7fe7d13ae640 1 -- 172.21.15.115:0/3326507935 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc15b830 con 0x7fe7cc1584b0 2023-12-01T01:54:33.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.273+0000 7fe7d13ae640 1 --2- 172.21.15.115:0/3326507935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 0x7fe7cc0cdd90 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009870 tx=0x7fe7c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fefd790f6a5f283 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.273+0000 7fe7bffff640 1 -- 172.21.15.115:0/3326507935 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0030840 con 0x7fe7cc1584b0 2023-12-01T01:54:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.273+0000 7fe7bffff640 1 -- 172.21.15.115:0/3326507935 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c00309a0 con 0x7fe7cc1584b0 2023-12-01T01:54:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.273+0000 7fe7bffff640 1 -- 172.21.15.115:0/3326507935 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0030d30 con 0x7fe7cc1584b0 2023-12-01T01:54:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.274+0000 7fe7d23b0640 1 -- 172.21.15.115:0/3326507935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 msgr2=0x7fe7cc0cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.274+0000 7fe7d23b0640 1 --2- 172.21.15.115:0/3326507935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 0x7fe7cc0cdd90 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009870 tx=0x7fe7c002b980 comp rx=0 tx=0).stop 2023-12-01T01:54:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.274+0000 7fe7d23b0640 1 -- 172.21.15.115:0/3326507935 shutdown_connections 2023-12-01T01:54:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.274+0000 7fe7d23b0640 1 --2- 172.21.15.115:0/3326507935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc15afe0 0x7fe7cc06b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.274+0000 7fe7d23b0640 1 --2- 172.21.15.115:0/3326507935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 0x7fe7cc0cdd90 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.274+0000 7fe7d23b0640 1 -- 172.21.15.115:0/3326507935 >> 172.21.15.115:0/3326507935 conn(0x7fe7cc0ce640 msgr2=0x7fe7cc0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.274+0000 7fe7d23b0640 1 -- 172.21.15.115:0/3326507935 shutdown_connections 2023-12-01T01:54:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.274+0000 7fe7d23b0640 1 -- 172.21.15.115:0/3326507935 wait complete. 2023-12-01T01:54:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.276+0000 7fe7d23b0640 1 Processor -- start 2023-12-01T01:54:33.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.276+0000 7fe7d23b0640 1 -- start start 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.277+0000 7fe7d23b0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc06b7b0 0x7fe7cc14b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.277+0000 7fe7d23b0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 0x7fe7cc158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.277+0000 7fe7d23b0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc15afe0 0x7fe7cc0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.277+0000 7fe7d23b0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc14b770 con 0x7fe7cc1584b0 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.277+0000 7fe7d23b0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc1594f0 con 0x7fe7cc06b7b0 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.277+0000 7fe7d23b0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc1597d0 con 0x7fe7cc15afe0 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.277+0000 7fe7d0bad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 0x7fe7cc158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.277+0000 7fe7d0bad640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 0x7fe7cc158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58562/0 (socket says 172.21.15.115:58562) 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.277+0000 7fe7d0bad640 1 -- 172.21.15.115:0/1595996280 learned_addr learned my addr 172.21.15.115:0/1595996280 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7d13ae640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc06b7b0 0x7fe7cc14b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:33.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7d0bad640 1 -- 172.21.15.115:0/1595996280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc15afe0 msgr2=0x7fe7cc0c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7d0bad640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc15afe0 0x7fe7cc0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7d0bad640 1 -- 172.21.15.115:0/1595996280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc06b7b0 msgr2=0x7fe7cc14b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7d0bad640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc06b7b0 0x7fe7cc14b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7d0bad640 1 -- 172.21.15.115:0/1595996280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c00094d0 con 0x7fe7cc1584b0 2023-12-01T01:54:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7d0bad640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 0x7fe7cc158b30 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800d5d0 tx=0x7fe7c800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7bdffb640 1 -- 172.21.15.115:0/1595996280 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80125f0 con 0x7fe7cc1584b0 2023-12-01T01:54:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7bdffb640 1 -- 172.21.15.115:0/1595996280 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c8012770 con 0x7fe7cc1584b0 2023-12-01T01:54:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7bdffb640 1 -- 172.21.15.115:0/1595996280 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8012b20 con 0x7fe7cc1584b0 2023-12-01T01:54:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.278+0000 7fe7d23b0640 1 -- 172.21.15.115:0/1595996280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc0c6af0 con 0x7fe7cc1584b0 2023-12-01T01:54:33.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.279+0000 7fe7d23b0640 1 -- 172.21.15.115:0/1595996280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc159ab0 con 0x7fe7cc1584b0 2023-12-01T01:54:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.280+0000 7fe7d23b0640 1 -- 172.21.15.115:0/1595996280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc0c7010 con 0x7fe7cc1584b0 2023-12-01T01:54:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.281+0000 7fe7bdffb640 1 -- 172.21.15.115:0/1595996280 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe7c8012c80 con 0x7fe7cc1584b0 2023-12-01T01:54:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.281+0000 7fe7bdffb640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7b006a550 0x7fe7b006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.282+0000 7fe7bdffb640 1 -- 172.21.15.115:0/1595996280 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7c8088fb0 con 0x7fe7cc1584b0 2023-12-01T01:54:33.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.282+0000 7fe7d13ae640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7b006a550 0x7fe7b006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:33.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.282+0000 7fe7d13ae640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7b006a550 0x7fe7b006ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0036000 tx=0x7fe7c003b040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:33.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.286+0000 7fe7bdffb640 1 -- 172.21.15.115:0/1595996280 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe7c80bbc80 con 0x7fe7cc1584b0 2023-12-01T01:54:33.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.463+0000 7fe7d23b0640 1 -- 172.21.15.115:0/1595996280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7cc0673a0 con 0x7fe7cc1584b0 2023-12-01T01:54:33.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.466+0000 7fe7bdffb640 1 -- 172.21.15.115:0/1595996280 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16112 (secure 0 0 0) 0x7fe7cc0c7010 con 0x7fe7cc1584b0 2023-12-01T01:54:33.468 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:33.468 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":124,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:54:33.047359+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:54:30.872492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-01T01:54:32.074780+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:54:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.468+0000 7fe7d23b0640 1 -- 172.21.15.115:0/1595996280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7b006a550 msgr2=0x7fe7b006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.468+0000 7fe7d23b0640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7b006a550 0x7fe7b006ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0036000 tx=0x7fe7c003b040 comp rx=0 tx=0).stop 2023-12-01T01:54:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.468+0000 7fe7d23b0640 1 -- 172.21.15.115:0/1595996280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 msgr2=0x7fe7cc158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.468+0000 7fe7d23b0640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 0x7fe7cc158b30 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800d5d0 tx=0x7fe7c800da90 comp rx=0 tx=0).stop 2023-12-01T01:54:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.468+0000 7fe7d23b0640 1 -- 172.21.15.115:0/1595996280 shutdown_connections 2023-12-01T01:54:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.468+0000 7fe7d23b0640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7b006a550 0x7fe7b006ca10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.468+0000 7fe7d23b0640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7cc15afe0 0x7fe7cc0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.468+0000 7fe7d23b0640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7cc1584b0 0x7fe7cc158b30 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.468+0000 7fe7d23b0640 1 --2- 172.21.15.115:0/1595996280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7cc06b7b0 0x7fe7cc14b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.468+0000 7fe7d23b0640 1 -- 172.21.15.115:0/1595996280 >> 172.21.15.115:0/1595996280 conn(0x7fe7cc0ce640 msgr2=0x7fe7cc0cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.469+0000 7fe7d23b0640 1 -- 172.21.15.115:0/1595996280 shutdown_connections 2023-12-01T01:54:33.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.469+0000 7fe7d23b0640 1 -- 172.21.15.115:0/1595996280 wait complete. 2023-12-01T01:54:33.488 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:54:33.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.691+0000 7fc6dcc84640 1 Processor -- start 2023-12-01T01:54:33.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.691+0000 7fc6dcc84640 1 -- start start 2023-12-01T01:54:33.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6dcc84640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc6d8159dd0 0x7fc6d80c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6dcc84640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc6d806b6f0 0x7fc6d80c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6dcc84640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d80ade80 0x7fc6d80cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6dcc84640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d806f8b0 con 0x7fc6d8159dd0 2023-12-01T01:54:33.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6dcc84640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d80adcb0 con 0x7fc6d806b6f0 2023-12-01T01:54:33.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6dcc84640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d80ad9d0 con 0x7fc6d80ade80 2023-12-01T01:54:33.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6d659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc6d8159dd0 0x7fc6d80c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:33.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6d659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc6d8159dd0 0x7fc6d80c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58564/0 (socket says 172.21.15.115:58564) 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6d659c640 1 -- 172.21.15.115:0/4150810875 learned_addr learned my addr 172.21.15.115:0/4150810875 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6d5d9b640 1 --2- 172.21.15.115:0/4150810875 >> v2:172.21.15.142:3300/0 conn(0x7fc6d806b6f0 0x7fc6d80c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6d659c640 1 -- 172.21.15.115:0/4150810875 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d80ade80 msgr2=0x7fc6d80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6d659c640 1 --2- 172.21.15.115:0/4150810875 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d80ade80 0x7fc6d80cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6d659c640 1 -- 172.21.15.115:0/4150810875 >> v2:172.21.15.142:3300/0 conn(0x7fc6d806b6f0 msgr2=0x7fc6d80c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6d659c640 1 --2- 172.21.15.115:0/4150810875 >> v2:172.21.15.142:3300/0 conn(0x7fc6d806b6f0 0x7fc6d80c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6d659c640 1 -- 172.21.15.115:0/4150810875 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80c9000 con 0x7fc6d8159dd0 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.693+0000 7fc6d5d9b640 1 --2- 172.21.15.115:0/4150810875 >> v2:172.21.15.142:3300/0 conn(0x7fc6d806b6f0 0x7fc6d80c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.694+0000 7fc6d659c640 1 --2- 172.21.15.115:0/4150810875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d8159dd0 0x7fc6d80c97e0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc6d806de80 tx=0x7fc6c802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194f5c9aed54716b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.695+0000 7fc6d559a640 1 -- 172.21.15.115:0/4150810875 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8031750 con 0x7fc6d8159dd0 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.695+0000 7fc6d559a640 1 -- 172.21.15.115:0/4150810875 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c80318b0 con 0x7fc6d8159dd0 2023-12-01T01:54:33.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.695+0000 7fc6d559a640 1 -- 172.21.15.115:0/4150810875 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8031c40 con 0x7fc6d8159dd0 2023-12-01T01:54:33.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.695+0000 7fc6dcc84640 1 -- 172.21.15.115:0/4150810875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d8159dd0 msgr2=0x7fc6d80c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.695+0000 7fc6dcc84640 1 --2- 172.21.15.115:0/4150810875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d8159dd0 0x7fc6d80c97e0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc6d806de80 tx=0x7fc6c802b7f0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.696+0000 7fc6dcc84640 1 -- 172.21.15.115:0/4150810875 shutdown_connections 2023-12-01T01:54:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.696+0000 7fc6dcc84640 1 --2- 172.21.15.115:0/4150810875 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d80ade80 0x7fc6d80cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.696+0000 7fc6dcc84640 1 --2- 172.21.15.115:0/4150810875 >> v2:172.21.15.142:3300/0 conn(0x7fc6d806b6f0 0x7fc6d80c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.696+0000 7fc6dcc84640 1 --2- 172.21.15.115:0/4150810875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d8159dd0 0x7fc6d80c97e0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.696+0000 7fc6dcc84640 1 -- 172.21.15.115:0/4150810875 >> 172.21.15.115:0/4150810875 conn(0x7fc6d80ce880 msgr2=0x7fc6d80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.696+0000 7fc6dcc84640 1 -- 172.21.15.115:0/4150810875 shutdown_connections 2023-12-01T01:54:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.696+0000 7fc6dcc84640 1 -- 172.21.15.115:0/4150810875 wait complete. 2023-12-01T01:54:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.698+0000 7fc6dcc84640 1 Processor -- start 2023-12-01T01:54:33.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.698+0000 7fc6dcc84640 1 -- start start 2023-12-01T01:54:33.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.699+0000 7fc6dcc84640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d806b6f0 0x7fc6d80c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.699+0000 7fc6dcc84640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d80ade80 0x7fc6d80aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.699+0000 7fc6dcc84640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d8159dd0 0x7fc6d81f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.699+0000 7fc6dcc84640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d815a4a0 con 0x7fc6d806b6f0 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.699+0000 7fc6dcc84640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d80c5950 con 0x7fc6d8159dd0 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.699+0000 7fc6dcc84640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d81f3120 con 0x7fc6d80ade80 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.700+0000 7fc6d659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d806b6f0 0x7fc6d80c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.700+0000 7fc6d659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d806b6f0 0x7fc6d80c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58566/0 (socket says 172.21.15.115:58566) 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.700+0000 7fc6d659c640 1 -- 172.21.15.115:0/2782789082 learned_addr learned my addr 172.21.15.115:0/2782789082 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.700+0000 7fc6d659c640 1 -- 172.21.15.115:0/2782789082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d80ade80 msgr2=0x7fc6d80aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.700+0000 7fc6d659c640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d80ade80 0x7fc6d80aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.700+0000 7fc6d659c640 1 -- 172.21.15.115:0/2782789082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d8159dd0 msgr2=0x7fc6d81f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.700+0000 7fc6d6d9d640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d8159dd0 0x7fc6d81f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.700+0000 7fc6d659c640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d8159dd0 0x7fc6d81f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.700+0000 7fc6d659c640 1 -- 172.21.15.115:0/2782789082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c80094d0 con 0x7fc6d806b6f0 2023-12-01T01:54:33.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.700+0000 7fc6d6d9d640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d8159dd0 0x7fc6d81f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:33.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.701+0000 7fc6d659c640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d806b6f0 0x7fc6d80c6340 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8000c00 tx=0x7fc6c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:33.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.701+0000 7fc6c6ffd640 1 -- 172.21.15.115:0/2782789082 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8033c80 con 0x7fc6d806b6f0 2023-12-01T01:54:33.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.701+0000 7fc6c6ffd640 1 -- 172.21.15.115:0/2782789082 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c8033de0 con 0x7fc6d806b6f0 2023-12-01T01:54:33.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.701+0000 7fc6c6ffd640 1 -- 172.21.15.115:0/2782789082 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c803c640 con 0x7fc6d806b6f0 2023-12-01T01:54:33.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.701+0000 7fc6dcc84640 1 -- 172.21.15.115:0/2782789082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d81f3400 con 0x7fc6d806b6f0 2023-12-01T01:54:33.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.701+0000 7fc6dcc84640 1 -- 172.21.15.115:0/2782789082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d80c5e40 con 0x7fc6d806b6f0 2023-12-01T01:54:33.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.702+0000 7fc6dcc84640 1 -- 172.21.15.115:0/2782789082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d80c5fd0 con 0x7fc6d806b6f0 2023-12-01T01:54:33.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.703+0000 7fc6c6ffd640 1 -- 172.21.15.115:0/2782789082 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc6c803a040 con 0x7fc6d806b6f0 2023-12-01T01:54:33.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.704+0000 7fc6c6ffd640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6b806a550 0x7fc6b806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:33.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.704+0000 7fc6c6ffd640 1 -- 172.21.15.115:0/2782789082 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6c80aa820 con 0x7fc6d806b6f0 2023-12-01T01:54:33.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.704+0000 7fc6d5d9b640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6b806a550 0x7fc6b806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:33.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.705+0000 7fc6d5d9b640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6b806a550 0x7fc6b806ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc006fd0 tx=0x7fc6cc008040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:33.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.709+0000 7fc6c6ffd640 1 -- 172.21.15.115:0/2782789082 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc6c8075e20 con 0x7fc6d806b6f0 2023-12-01T01:54:33.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.900+0000 7fc6dcc84640 1 -- 172.21.15.115:0/2782789082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6d804e890 con 0x7fc6d806b6f0 2023-12-01T01:54:33.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.903+0000 7fc6c6ffd640 1 -- 172.21.15.115:0/2782789082 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16112 (secure 0 0 0) 0x7fc6c8038090 con 0x7fc6d806b6f0 2023-12-01T01:54:33.905 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:33.906 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":124,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:54:33.047359+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:54:30.872492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-01T01:54:32.074780+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:54:33.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 -- 172.21.15.115:0/2782789082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6b806a550 msgr2=0x7fc6b806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6b806a550 0x7fc6b806ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc006fd0 tx=0x7fc6cc008040 comp rx=0 tx=0).stop 2023-12-01T01:54:33.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 -- 172.21.15.115:0/2782789082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d806b6f0 msgr2=0x7fc6d80c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:33.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d806b6f0 0x7fc6d80c6340 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8000c00 tx=0x7fc6c80046c0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 -- 172.21.15.115:0/2782789082 shutdown_connections 2023-12-01T01:54:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc6b806a550 0x7fc6b806ca10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d80ade80 0x7fc6d80aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d8159dd0 0x7fc6d81f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 --2- 172.21.15.115:0/2782789082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d806b6f0 0x7fc6d80c6340 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 -- 172.21.15.115:0/2782789082 >> 172.21.15.115:0/2782789082 conn(0x7fc6d80ce880 msgr2=0x7fc6d80c1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 -- 172.21.15.115:0/2782789082 shutdown_connections 2023-12-01T01:54:33.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:33.906+0000 7fc6dcc84640 1 -- 172.21.15.115:0/2782789082 wait complete. 2023-12-01T01:54:33.927 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T01:54:34.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.135+0000 7f043338e640 1 Processor -- start 2023-12-01T01:54:34.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.135+0000 7f043338e640 1 -- start start 2023-12-01T01:54:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f043338e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f042c0cb800 0x7f042c1656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f043338e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042c15afe0 con 0x7f042c0b04c0 2023-12-01T01:54:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f043338e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042c0b0a10 con 0x7f042c15ac00 2023-12-01T01:54:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f043338e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042c0b12b0 con 0x7f042c0cb800 2023-12-01T01:54:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f0431b8b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f042c0b04c0 0x7f042c0cb0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59074/0 (socket says 172.21.15.115:59074) 2023-12-01T01:54:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f0431b8b640 1 -- 172.21.15.115:0/1239255210 learned_addr learned my addr 172.21.15.115:0/1239255210 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f0432b8d640 1 --2- 172.21.15.115:0/1239255210 >> v2:172.21.15.115:3300/0 conn(0x7f042c0cb800 0x7f042c1656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f043138a640 1 -- 172.21.15.115:0/1239255210 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18936261 0 0) 0x7f042c15afe0 con 0x7f042c0b04c0 2023-12-01T01:54:34.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f043138a640 1 -- 172.21.15.115:0/1239255210 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0418003520 con 0x7f042c0b04c0 2023-12-01T01:54:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f0432b8d640 1 -- 172.21.15.115:0/1239255210 >> v1:172.21.15.142:6789/0 conn(0x7f042c15ac00 legacy=0x7f042c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.137+0000 7f0432b8d640 1 -- 172.21.15.115:0/1239255210 >> v1:172.21.15.115:6789/0 conn(0x7f042c0b04c0 legacy=0x7f042c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:34.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.138+0000 7f0432b8d640 1 -- 172.21.15.115:0/1239255210 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c0b0cf0 con 0x7f042c0cb800 2023-12-01T01:54:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.139+0000 7f0432b8d640 1 --2- 172.21.15.115:0/1239255210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c0cb800 0x7f042c1656e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f0424009d90 tx=0x7f042402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f88a349c90ed5dc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.139+0000 7f043138a640 1 -- 172.21.15.115:0/1239255210 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042402cbd0 con 0x7f042c0cb800 2023-12-01T01:54:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.139+0000 7f043138a640 1 -- 172.21.15.115:0/1239255210 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042402cd30 con 0x7f042c0cb800 2023-12-01T01:54:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.139+0000 7f043138a640 1 -- 172.21.15.115:0/1239255210 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042402cea0 con 0x7f042c0cb800 2023-12-01T01:54:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.139+0000 7f043338e640 1 -- 172.21.15.115:0/1239255210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c0cb800 msgr2=0x7f042c1656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.139+0000 7f043338e640 1 --2- 172.21.15.115:0/1239255210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c0cb800 0x7f042c1656e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f0424009d90 tx=0x7f042402be20 comp rx=0 tx=0).stop 2023-12-01T01:54:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.140+0000 7f043338e640 1 -- 172.21.15.115:0/1239255210 shutdown_connections 2023-12-01T01:54:34.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.140+0000 7f043338e640 1 --2- 172.21.15.115:0/1239255210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c0cb800 0x7f042c1656e0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:34.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.140+0000 7f043338e640 1 -- 172.21.15.115:0/1239255210 >> 172.21.15.115:0/1239255210 conn(0x7f042c065960 msgr2=0x7f042c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:34.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.140+0000 7f043338e640 1 -- 172.21.15.115:0/1239255210 shutdown_connections 2023-12-01T01:54:34.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.140+0000 7f043338e640 1 -- 172.21.15.115:0/1239255210 wait complete. 2023-12-01T01:54:34.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.141+0000 7f043338e640 1 Processor -- start 2023-12-01T01:54:34.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.141+0000 7f043338e640 1 -- start start 2023-12-01T01:54:34.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.142+0000 7f043338e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f042c0b04c0 0x7f042c146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.142+0000 7f043338e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c0cb800 0x7f042c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.142+0000 7f043338e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c15ac00 0x7f042c070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.142+0000 7f043338e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c15bcc0 con 0x7f042c15ac00 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.142+0000 7f043338e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c15b320 con 0x7f042c0cb800 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.142+0000 7f043338e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c15b600 con 0x7f042c0b04c0 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.143+0000 7f0432b8d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c15ac00 0x7f042c070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.143+0000 7f0432b8d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c15ac00 0x7f042c070210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58588/0 (socket says 172.21.15.115:58588) 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.143+0000 7f0432b8d640 1 -- 172.21.15.115:0/3626732802 learned_addr learned my addr 172.21.15.115:0/3626732802 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.143+0000 7f0432b8d640 1 -- 172.21.15.115:0/3626732802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f042c0b04c0 msgr2=0x7f042c146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.143+0000 7f0432b8d640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f042c0b04c0 0x7f042c146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.143+0000 7f0432b8d640 1 -- 172.21.15.115:0/3626732802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c0cb800 msgr2=0x7f042c06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.143+0000 7f0432b8d640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c0cb800 0x7f042c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:34.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.143+0000 7f0432b8d640 1 -- 172.21.15.115:0/3626732802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0424009a70 con 0x7f042c15ac00 2023-12-01T01:54:34.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.143+0000 7f0432b8d640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c15ac00 0x7f042c070210 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0424009d90 tx=0x7f0424004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:34.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.144+0000 7f0422ffd640 1 -- 172.21.15.115:0/3626732802 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042402cea0 con 0x7f042c15ac00 2023-12-01T01:54:34.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.144+0000 7f0422ffd640 1 -- 172.21.15.115:0/3626732802 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0424040650 con 0x7f042c15ac00 2023-12-01T01:54:34.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.144+0000 7f043338e640 1 -- 172.21.15.115:0/3626732802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c15b8e0 con 0x7f042c15ac00 2023-12-01T01:54:34.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.144+0000 7f0422ffd640 1 -- 172.21.15.115:0/3626732802 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424040a00 con 0x7f042c15ac00 2023-12-01T01:54:34.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.144+0000 7f043338e640 1 -- 172.21.15.115:0/3626732802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c0c7970 con 0x7f042c15ac00 2023-12-01T01:54:34.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.145+0000 7f043338e640 1 -- 172.21.15.115:0/3626732802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042c15be00 con 0x7f042c15ac00 2023-12-01T01:54:34.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.146+0000 7f0422ffd640 1 -- 172.21.15.115:0/3626732802 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0424040b60 con 0x7f042c15ac00 2023-12-01T01:54:34.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.147+0000 7f0422ffd640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f041006a550 0x7f041006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:34.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.147+0000 7f0422ffd640 1 -- 172.21.15.115:0/3626732802 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04240ab660 con 0x7f042c15ac00 2023-12-01T01:54:34.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.147+0000 7f043238c640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f041006a550 0x7f041006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:34.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.147+0000 7f043238c640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f041006a550 0x7f041006ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0428006fd0 tx=0x7f0428006bc0 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:34.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.151+0000 7f0422ffd640 1 -- 172.21.15.115:0/3626732802 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f042c15be00 con 0x7f042c15ac00 2023-12-01T01:54:34.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:34.365+0000 7f043338e640 1 -- 172.21.15.115:0/3626732802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f042c15be00 con 0x7f042c15ac00 2023-12-01T01:54:35.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.071+0000 7f0422ffd640 1 -- 172.21.15.115:0/3626732802 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f042c15be00 con 0x7f042c15ac00 2023-12-01T01:54:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.074+0000 7f043338e640 1 -- 172.21.15.115:0/3626732802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f041006a550 msgr2=0x7f041006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.074+0000 7f043338e640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f041006a550 0x7f041006ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0428006fd0 tx=0x7f0428006bc0 comp rx=0 tx=0).stop 2023-12-01T01:54:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.074+0000 7f043338e640 1 -- 172.21.15.115:0/3626732802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c15ac00 msgr2=0x7f042c070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.074+0000 7f043338e640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c15ac00 0x7f042c070210 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0424009d90 tx=0x7f0424004a40 comp rx=0 tx=0).stop 2023-12-01T01:54:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.074+0000 7f043338e640 1 -- 172.21.15.115:0/3626732802 shutdown_connections 2023-12-01T01:54:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.074+0000 7f043338e640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f041006a550 0x7f041006ca10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.074+0000 7f043338e640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f042c15ac00 0x7f042c070210 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.074+0000 7f043338e640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042c0cb800 0x7f042c06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.074+0000 7f043338e640 1 --2- 172.21.15.115:0/3626732802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f042c0b04c0 0x7f042c146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.074+0000 7f043338e640 1 -- 172.21.15.115:0/3626732802 >> 172.21.15.115:0/3626732802 conn(0x7f042c065960 msgr2=0x7f042c0ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.075+0000 7f043338e640 1 -- 172.21.15.115:0/3626732802 shutdown_connections 2023-12-01T01:54:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.075+0000 7f043338e640 1 -- 172.21.15.115:0/3626732802 wait complete. 2023-12-01T01:54:35.095 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T01:54:35.095 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T01:54:35.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.314+0000 7fa7392e0640 1 Processor -- start 2023-12-01T01:54:35.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.314+0000 7fa7392e0640 1 -- start start 2023-12-01T01:54:35.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.316+0000 7fa7392e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 0x7fa7340ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:35.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.316+0000 7fa7392e0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa7340cd010 0x7fa7340b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:35.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.316+0000 7fa7392e0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7340cd470 con 0x7fa7340b1c50 2023-12-01T01:54:35.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.316+0000 7fa7392e0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7340b3ae0 con 0x7fa7340cd010 2023-12-01T01:54:35.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.316+0000 7fa7392e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa734142c50 con 0x7fa734142870 2023-12-01T01:54:35.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.317+0000 7fa7327fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa7340cd010 0x7fa7340b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:35.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.317+0000 7fa732ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 0x7fa7340ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:35.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.317+0000 7fa7327fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa7340cd010 0x7fa7340b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:41292/0 (socket says 172.21.15.115:41292) 2023-12-01T01:54:35.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.317+0000 7fa7327fc640 1 -- 172.21.15.115:0/2683994018 learned_addr learned my addr 172.21.15.115:0/2683994018 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:35.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.317+0000 7fa7327fc640 1 -- 172.21.15.115:0/2683994018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 msgr2=0x7fa7340ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:35.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.317+0000 7fa7327fc640 1 --2- 172.21.15.115:0/2683994018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 0x7fa7340ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:35.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.317+0000 7fa7327fc640 1 -- 172.21.15.115:0/2683994018 >> v1:172.21.15.142:6789/0 conn(0x7fa7340b1c50 legacy=0x7fa73415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:35.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.317+0000 7fa7327fc640 1 -- 172.21.15.115:0/2683994018 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7340b20e0 con 0x7fa7340cd010 2023-12-01T01:54:35.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.319+0000 7fa7327fc640 1 --2- 172.21.15.115:0/2683994018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7340cd010 0x7fa7340b1710 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa7280098a0 tx=0x7fa72802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=298d28da3eb60aef server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:35.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.320+0000 7fa731ffb640 1 -- 172.21.15.115:0/2683994018 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728030840 con 0x7fa7340cd010 2023-12-01T01:54:35.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.320+0000 7fa731ffb640 1 -- 172.21.15.115:0/2683994018 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7280309a0 con 0x7fa7340cd010 2023-12-01T01:54:35.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.320+0000 7fa731ffb640 1 -- 172.21.15.115:0/2683994018 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728030d30 con 0x7fa7340cd010 2023-12-01T01:54:35.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.320+0000 7fa7392e0640 1 -- 172.21.15.115:0/2683994018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7340cd010 msgr2=0x7fa7340b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:35.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.320+0000 7fa7392e0640 1 --2- 172.21.15.115:0/2683994018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7340cd010 0x7fa7340b1710 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa7280098a0 tx=0x7fa72802b980 comp rx=0 tx=0).stop 2023-12-01T01:54:35.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.320+0000 7fa7392e0640 1 -- 172.21.15.115:0/2683994018 shutdown_connections 2023-12-01T01:54:35.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.320+0000 7fa7392e0640 1 --2- 172.21.15.115:0/2683994018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7340cd010 0x7fa7340b1710 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:35.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.320+0000 7fa7392e0640 1 --2- 172.21.15.115:0/2683994018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 0x7fa7340ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:35.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.320+0000 7fa7392e0640 1 -- 172.21.15.115:0/2683994018 >> 172.21.15.115:0/2683994018 conn(0x7fa734065960 msgr2=0x7fa734065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:35.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.321+0000 7fa7392e0640 1 -- 172.21.15.115:0/2683994018 shutdown_connections 2023-12-01T01:54:35.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.321+0000 7fa7392e0640 1 -- 172.21.15.115:0/2683994018 wait complete. 2023-12-01T01:54:35.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.323+0000 7fa7392e0640 1 Processor -- start 2023-12-01T01:54:35.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.323+0000 7fa7392e0640 1 -- start start 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7392e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7340b1c50 0x7fa73413e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7392e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340cd010 0x7fa7341e7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7392e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 0x7fa7341e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7392e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340ab7f0 con 0x7fa7340cd010 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7392e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340aa310 con 0x7fa7340b1c50 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7392e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340aa5f0 con 0x7fa734142870 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7327fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340cd010 0x7fa7341e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7327fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340cd010 0x7fa7341e7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58592/0 (socket says 172.21.15.115:58592) 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7327fc640 1 -- 172.21.15.115:0/149419857 learned_addr learned my addr 172.21.15.115:0/149419857 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7337fe640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 0x7fa7341e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7327fc640 1 -- 172.21.15.115:0/149419857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 msgr2=0x7fa7341e7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7327fc640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 0x7fa7341e7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7327fc640 1 -- 172.21.15.115:0/149419857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7340b1c50 msgr2=0x7fa73413e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:35.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.325+0000 7fa7327fc640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7340b1c50 0x7fa73413e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:35.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.326+0000 7fa7327fc640 1 -- 172.21.15.115:0/149419857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7280094d0 con 0x7fa7340cd010 2023-12-01T01:54:35.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.326+0000 7fa7337fe640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 0x7fa7341e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:35.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.326+0000 7fa7327fc640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340cd010 0x7fa7341e7950 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fa7280098a0 tx=0x7fa728004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:35.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.326+0000 7fa71b7fe640 1 -- 172.21.15.115:0/149419857 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728039c90 con 0x7fa7340cd010 2023-12-01T01:54:35.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.326+0000 7fa71b7fe640 1 -- 172.21.15.115:0/149419857 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa728039df0 con 0x7fa7340cd010 2023-12-01T01:54:35.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.326+0000 7fa71b7fe640 1 -- 172.21.15.115:0/149419857 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728043640 con 0x7fa7340cd010 2023-12-01T01:54:35.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.326+0000 7fa7392e0640 1 -- 172.21.15.115:0/149419857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7340aa8d0 con 0x7fa7340cd010 2023-12-01T01:54:35.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.326+0000 7fa7392e0640 1 -- 172.21.15.115:0/149419857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7340aae90 con 0x7fa7340cd010 2023-12-01T01:54:35.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.328+0000 7fa7392e0640 1 -- 172.21.15.115:0/149419857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7340b2bc0 con 0x7fa7340cd010 2023-12-01T01:54:35.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.328+0000 7fa71b7fe640 1 -- 172.21.15.115:0/149419857 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa7280437a0 con 0x7fa7340cd010 2023-12-01T01:54:35.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.329+0000 7fa71b7fe640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa72006a580 0x7fa72006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:35.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.329+0000 7fa71b7fe640 1 -- 172.21.15.115:0/149419857 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa72803e070 con 0x7fa7340cd010 2023-12-01T01:54:35.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.330+0000 7fa732ffd640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa72006a580 0x7fa72006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:35.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.330+0000 7fa732ffd640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa72006a580 0x7fa72006ca40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa724006a90 tx=0x7fa724006a20 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:35.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.334+0000 7fa71b7fe640 1 -- 172.21.15.115:0/149419857 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa728076db0 con 0x7fa7340cd010 2023-12-01T01:54:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:35.560+0000 7fa7392e0640 1 -- 172.21.15.115:0/149419857 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa7340b20e0 con 0x7fa7340cd010 2023-12-01T01:54:36.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.565+0000 7fa71b7fe640 1 -- 172.21.15.115:0/149419857 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa7340b20e0 con 0x7fa7340cd010 2023-12-01T01:54:36.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 -- 172.21.15.115:0/149419857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa72006a580 msgr2=0x7fa72006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:36.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa72006a580 0x7fa72006ca40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa724006a90 tx=0x7fa724006a20 comp rx=0 tx=0).stop 2023-12-01T01:54:36.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 -- 172.21.15.115:0/149419857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340cd010 msgr2=0x7fa7341e7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:36.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340cd010 0x7fa7341e7950 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fa7280098a0 tx=0x7fa728004730 comp rx=0 tx=0).stop 2023-12-01T01:54:36.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 -- 172.21.15.115:0/149419857 shutdown_connections 2023-12-01T01:54:36.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa72006a580 0x7fa72006ca40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:36.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa734142870 0x7fa7341e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:36.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340cd010 0x7fa7341e7950 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:36.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 --2- 172.21.15.115:0/149419857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7340b1c50 0x7fa73413e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:36.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 -- 172.21.15.115:0/149419857 >> 172.21.15.115:0/149419857 conn(0x7fa734065960 msgr2=0x7fa73413f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:36.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 -- 172.21.15.115:0/149419857 shutdown_connections 2023-12-01T01:54:36.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.569+0000 7fa7392e0640 1 -- 172.21.15.115:0/149419857 wait complete. 2023-12-01T01:54:36.591 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T01:54:36.592 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:54:36.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.805+0000 7f79aa277640 1 Processor -- start 2023-12-01T01:54:36.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.806+0000 7f79aa277640 1 -- start start 2023-12-01T01:54:36.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.807+0000 7f79aa277640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79a415cd20 0x7f79a40d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:36.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.807+0000 7f79aa277640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a40d1230 0x7f79a40d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:36.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.807+0000 7f79aa277640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a40d1b90 con 0x7f79a40b5d80 2023-12-01T01:54:36.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.807+0000 7f79aa277640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a40b2160 con 0x7f79a415cd20 2023-12-01T01:54:36.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.807+0000 7f79aa277640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a40b8c80 con 0x7f79a40d1230 2023-12-01T01:54:36.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.808+0000 7f79a3fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79a415cd20 0x7f79a40d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:36.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.808+0000 7f79a3fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79a415cd20 0x7f79a40d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:41302/0 (socket says 172.21.15.115:41302) 2023-12-01T01:54:36.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.808+0000 7f79a3fff640 1 -- 172.21.15.115:0/3751043507 learned_addr learned my addr 172.21.15.115:0/3751043507 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:36.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.808+0000 7f79a8814640 1 --2- 172.21.15.115:0/3751043507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a40d1230 0x7f79a40d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:36.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.809+0000 7f79a3fff640 1 -- 172.21.15.115:0/3751043507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a40d1230 msgr2=0x7f79a40d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:36.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.809+0000 7f79a3fff640 1 --2- 172.21.15.115:0/3751043507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a40d1230 0x7f79a40d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:36.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.809+0000 7f79a3fff640 1 -- 172.21.15.115:0/3751043507 >> v1:172.21.15.142:6789/0 conn(0x7f79a40b5d80 legacy=0x7f79a40b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:36.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.809+0000 7f79a3fff640 1 -- 172.21.15.115:0/3751043507 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40ce2e0 con 0x7f79a415cd20 2023-12-01T01:54:36.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.810+0000 7f79a3fff640 1 --2- 172.21.15.115:0/3751043507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a415cd20 0x7f79a40d24d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7994009e50 tx=0x7f799402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6751bafcb051781 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:36.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.810+0000 7f79a2ffd640 1 -- 172.21.15.115:0/3751043507 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799402ca70 con 0x7f79a415cd20 2023-12-01T01:54:36.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.811+0000 7f79a2ffd640 1 -- 172.21.15.115:0/3751043507 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799402cbd0 con 0x7f79a415cd20 2023-12-01T01:54:36.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.811+0000 7f79a2ffd640 1 -- 172.21.15.115:0/3751043507 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940343f0 con 0x7f79a415cd20 2023-12-01T01:54:36.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.811+0000 7f79aa277640 1 -- 172.21.15.115:0/3751043507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a415cd20 msgr2=0x7f79a40d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:36.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.811+0000 7f79aa277640 1 --2- 172.21.15.115:0/3751043507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a415cd20 0x7f79a40d24d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7994009e50 tx=0x7f799402bed0 comp rx=0 tx=0).stop 2023-12-01T01:54:36.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.811+0000 7f79aa277640 1 -- 172.21.15.115:0/3751043507 shutdown_connections 2023-12-01T01:54:36.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.811+0000 7f79aa277640 1 --2- 172.21.15.115:0/3751043507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a40d1230 0x7f79a40d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:36.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.811+0000 7f79aa277640 1 --2- 172.21.15.115:0/3751043507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a415cd20 0x7f79a40d24d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:36.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.811+0000 7f79aa277640 1 -- 172.21.15.115:0/3751043507 >> 172.21.15.115:0/3751043507 conn(0x7f79a406a0e0 msgr2=0x7f79a406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:36.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.811+0000 7f79aa277640 1 -- 172.21.15.115:0/3751043507 shutdown_connections 2023-12-01T01:54:36.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.811+0000 7f79aa277640 1 -- 172.21.15.115:0/3751043507 wait complete. 2023-12-01T01:54:36.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.813+0000 7f79aa277640 1 Processor -- start 2023-12-01T01:54:36.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.814+0000 7f79aa277640 1 -- start start 2023-12-01T01:54:36.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.815+0000 7f79aa277640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a40b5d80 0x7f79a41531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:36.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.815+0000 7f79aa277640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40d1230 0x7f79a415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:36.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.815+0000 7f79aa277640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415cd20 0x7f79a415b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:36.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.815+0000 7f79aa277640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4154a40 con 0x7f79a40b5d80 2023-12-01T01:54:36.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.815+0000 7f79aa277640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4153a30 con 0x7f79a40d1230 2023-12-01T01:54:36.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.815+0000 7f79aa277640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4153c80 con 0x7f79a415cd20 2023-12-01T01:54:36.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.815+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a40b5d80 0x7f79a41531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:36.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.815+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a40b5d80 0x7f79a41531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58608/0 (socket says 172.21.15.115:58608) 2023-12-01T01:54:36.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.815+0000 7f79a3fff640 1 -- 172.21.15.115:0/1351797572 learned_addr learned my addr 172.21.15.115:0/1351797572 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:36.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a37fe640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40d1230 0x7f79a415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:36.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a8814640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415cd20 0x7f79a415b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:36.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a3fff640 1 -- 172.21.15.115:0/1351797572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415cd20 msgr2=0x7f79a415b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:36.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a3fff640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415cd20 0x7f79a415b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:36.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a3fff640 1 -- 172.21.15.115:0/1351797572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40d1230 msgr2=0x7f79a415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:36.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a3fff640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40d1230 0x7f79a415aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:36.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a3fff640 1 -- 172.21.15.115:0/1351797572 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7994009b30 con 0x7f79a40b5d80 2023-12-01T01:54:36.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a8814640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415cd20 0x7f79a415b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:36.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a3fff640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a40b5d80 0x7f79a41531d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7994000f80 tx=0x7f7994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:36.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a0ff9640 1 -- 172.21.15.115:0/1351797572 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994009510 con 0x7f79a40b5d80 2023-12-01T01:54:36.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a0ff9640 1 -- 172.21.15.115:0/1351797572 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799403e450 con 0x7f79a40b5d80 2023-12-01T01:54:36.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.816+0000 7f79a0ff9640 1 -- 172.21.15.115:0/1351797572 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799403e800 con 0x7f79a40b5d80 2023-12-01T01:54:36.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.817+0000 7f79aa277640 1 -- 172.21.15.115:0/1351797572 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a4153f60 con 0x7f79a40b5d80 2023-12-01T01:54:36.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.817+0000 7f79aa277640 1 -- 172.21.15.115:0/1351797572 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a4154520 con 0x7f79a40b5d80 2023-12-01T01:54:36.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.818+0000 7f79aa277640 1 -- 172.21.15.115:0/1351797572 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a40cd3b0 con 0x7f79a40b5d80 2023-12-01T01:54:36.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.819+0000 7f79a0ff9640 1 -- 172.21.15.115:0/1351797572 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f799403e960 con 0x7f79a40b5d80 2023-12-01T01:54:36.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.820+0000 7f79a0ff9640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 0x7f798406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:36.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.820+0000 7f79a0ff9640 1 -- 172.21.15.115:0/1351797572 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79940ab790 con 0x7f79a40b5d80 2023-12-01T01:54:36.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.820+0000 7f79a37fe640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 0x7f798406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:36.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.820+0000 7f79a37fe640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 0x7f798406ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f79a4156250 tx=0x7f798c009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:36.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:36.824+0000 7f79a0ff9640 1 -- 172.21.15.115:0/1351797572 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f79a40cd3b0 con 0x7f79a40b5d80 2023-12-01T01:54:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.054+0000 7f79aa277640 1 -- 172.21.15.115:0/1351797572 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79a40cd3b0 con 0x7f79a40b5d80 2023-12-01T01:54:37.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.055+0000 7f79a0ff9640 1 -- 172.21.15.115:0/1351797572 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f79a40cd3b0 con 0x7f79a40b5d80 2023-12-01T01:54:37.057 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:37.057 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/4122088547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":4122088547},{"type":"v1","addr":"172.21.15.142:6834","nonce":4122088547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/299645728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":299645728},{"type":"v1","addr":"172.21.15.142:6835","nonce":299645728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7983,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/150770274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":150770274},{"type":"v1","addr":"172.21.15.115:6836","nonce":150770274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:54:33.047892+0000","modified":"2023-12-01T01:54:36.558949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/2398273191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":2398273191},{"type":"v1","addr":"172.21.15.115:6837","nonce":2398273191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-01T01:54:37.057 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 56 2023-12-01T01:54:37.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 -- 172.21.15.115:0/1351797572 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 msgr2=0x7f798406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 0x7f798406ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f79a4156250 tx=0x7f798c009040 comp rx=0 tx=0).stop 2023-12-01T01:54:37.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 -- 172.21.15.115:0/1351797572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a40b5d80 msgr2=0x7f79a41531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a40b5d80 0x7f79a41531d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7994000f80 tx=0x7f7994004a40 comp rx=0 tx=0).stop 2023-12-01T01:54:37.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 -- 172.21.15.115:0/1351797572 shutdown_connections 2023-12-01T01:54:37.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 0x7f798406ca10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415cd20 0x7f79a415b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40d1230 0x7f79a415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 --2- 172.21.15.115:0/1351797572 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a40b5d80 0x7f79a41531d0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 -- 172.21.15.115:0/1351797572 >> 172.21.15.115:0/1351797572 conn(0x7f79a406a0e0 msgr2=0x7f79a40b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:37.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 -- 172.21.15.115:0/1351797572 shutdown_connections 2023-12-01T01:54:37.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.058+0000 7f79aa277640 1 -- 172.21.15.115:0/1351797572 wait complete. 2023-12-01T01:54:37.079 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:54:37.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.290+0000 7f67c9b92640 1 Processor -- start 2023-12-01T01:54:37.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.290+0000 7f67c9b92640 1 -- start start 2023-12-01T01:54:37.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.292+0000 7f67c9b92640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 0x7f67c40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.292+0000 7f67c9b92640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f67c40adf10 0x7f67c40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.292+0000 7f67c9b92640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c40add10 con 0x7f67c406f2f0 2023-12-01T01:54:37.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.292+0000 7f67c9b92640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c40c5760 con 0x7f67c40adf10 2023-12-01T01:54:37.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.292+0000 7f67c9b92640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c415e6d0 con 0x7f67c415e2f0 2023-12-01T01:54:37.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.292+0000 7f67c3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f67c40adf10 0x7f67c40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.292+0000 7f67c3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f67c40adf10 0x7f67c40caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58616/0 (socket says 172.21.15.115:58616) 2023-12-01T01:54:37.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.292+0000 7f67c3fff640 1 -- 172.21.15.115:0/2618531347 learned_addr learned my addr 172.21.15.115:0/2618531347 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:37.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.293+0000 7f67c3fff640 1 -- 172.21.15.115:0/2618531347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 msgr2=0x7f67c40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:37.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.293+0000 7f67c37fe640 1 --2- 172.21.15.115:0/2618531347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 0x7f67c40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.293+0000 7f67c3fff640 1 --2- 172.21.15.115:0/2618531347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 0x7f67c40ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.293+0000 7f67c3fff640 1 -- 172.21.15.115:0/2618531347 >> v1:172.21.15.142:6789/0 conn(0x7f67c406f2f0 legacy=0x7f67c40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:37.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.293+0000 7f67c3fff640 1 -- 172.21.15.115:0/2618531347 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c40cf7b0 con 0x7f67c40adf10 2023-12-01T01:54:37.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.293+0000 7f67c37fe640 1 --2- 172.21.15.115:0/2618531347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 0x7f67c40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:37.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.294+0000 7f67c3fff640 1 --2- 172.21.15.115:0/2618531347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 0x7f67c40caba0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009e50 tx=0x7f67bc02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc275765451ba627 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:37.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.294+0000 7f67c2ffd640 1 -- 172.21.15.115:0/2618531347 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc02c960 con 0x7f67c40adf10 2023-12-01T01:54:37.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.294+0000 7f67c2ffd640 1 -- 172.21.15.115:0/2618531347 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67bc02cac0 con 0x7f67c40adf10 2023-12-01T01:54:37.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.294+0000 7f67c2ffd640 1 -- 172.21.15.115:0/2618531347 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc02ce50 con 0x7f67c40adf10 2023-12-01T01:54:37.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.295+0000 7f67c9b92640 1 -- 172.21.15.115:0/2618531347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 msgr2=0x7f67c40caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.295+0000 7f67c9b92640 1 --2- 172.21.15.115:0/2618531347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 0x7f67c40caba0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009e50 tx=0x7f67bc02bdc0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.295+0000 7f67c9b92640 1 -- 172.21.15.115:0/2618531347 shutdown_connections 2023-12-01T01:54:37.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.295+0000 7f67c9b92640 1 --2- 172.21.15.115:0/2618531347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 0x7f67c40caba0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.295+0000 7f67c9b92640 1 --2- 172.21.15.115:0/2618531347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 0x7f67c40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.295+0000 7f67c9b92640 1 -- 172.21.15.115:0/2618531347 >> 172.21.15.115:0/2618531347 conn(0x7f67c40ce880 msgr2=0x7f67c40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:37.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.295+0000 7f67c9b92640 1 -- 172.21.15.115:0/2618531347 shutdown_connections 2023-12-01T01:54:37.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.295+0000 7f67c9b92640 1 -- 172.21.15.115:0/2618531347 wait complete. 2023-12-01T01:54:37.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.297+0000 7f67c9b92640 1 Processor -- start 2023-12-01T01:54:37.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.297+0000 7f67c9b92640 1 -- start start 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.298+0000 7f67c9b92640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c406f2f0 0x7f67c40cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.298+0000 7f67c9b92640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 0x7f67c4150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.298+0000 7f67c9b92640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 0x7f67c40c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.298+0000 7f67c9b92640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c40cd6b0 con 0x7f67c40adf10 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.298+0000 7f67c9b92640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41514a0 con 0x7f67c406f2f0 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.298+0000 7f67c9b92640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c40cca30 con 0x7f67c415e2f0 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.298+0000 7f67bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 0x7f67c4150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.298+0000 7f67bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 0x7f67c4150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58632/0 (socket says 172.21.15.115:58632) 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.298+0000 7f67bbfff640 1 -- 172.21.15.115:0/177998008 learned_addr learned my addr 172.21.15.115:0/177998008 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67c3fff640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 0x7f67c40c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67c37fe640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c406f2f0 0x7f67c40cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67bbfff640 1 -- 172.21.15.115:0/177998008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 msgr2=0x7f67c40c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67bbfff640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 0x7f67c40c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67bbfff640 1 -- 172.21.15.115:0/177998008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c406f2f0 msgr2=0x7f67c40cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67bbfff640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c406f2f0 0x7f67c40cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67bbfff640 1 -- 172.21.15.115:0/177998008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc009b30 con 0x7f67c40adf10 2023-12-01T01:54:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67bbfff640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 0x7f67c4150f60 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00b3f0 tx=0x7f67ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67c0ff9640 1 -- 172.21.15.115:0/177998008 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac011830 con 0x7f67c40adf10 2023-12-01T01:54:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67c0ff9640 1 -- 172.21.15.115:0/177998008 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac011990 con 0x7f67c40adf10 2023-12-01T01:54:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67c9b92640 1 -- 172.21.15.115:0/177998008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c40ccd10 con 0x7f67c40adf10 2023-12-01T01:54:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.299+0000 7f67c0ff9640 1 -- 172.21.15.115:0/177998008 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac011d40 con 0x7f67c40adf10 2023-12-01T01:54:37.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.300+0000 7f67c9b92640 1 -- 172.21.15.115:0/177998008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c40cd2d0 con 0x7f67c40adf10 2023-12-01T01:54:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.301+0000 7f67c9b92640 1 -- 172.21.15.115:0/177998008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c406fda0 con 0x7f67c40adf10 2023-12-01T01:54:37.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.301+0000 7f67c0ff9640 1 -- 172.21.15.115:0/177998008 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f67ac01a400 con 0x7f67c40adf10 2023-12-01T01:54:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.303+0000 7f67c0ff9640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f67a406a550 0x7f67a406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.303+0000 7f67c0ff9640 1 -- 172.21.15.115:0/177998008 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67ac088920 con 0x7f67c40adf10 2023-12-01T01:54:37.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.303+0000 7f67c37fe640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f67a406a550 0x7f67a406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.304+0000 7f67c37fe640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f67a406a550 0x7f67a406ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f67b400b840 tx=0x7f67b4008040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:37.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.308+0000 7f67c0ff9640 1 -- 172.21.15.115:0/177998008 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f67ac053e00 con 0x7f67c40adf10 2023-12-01T01:54:37.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.500+0000 7f67c9b92640 1 -- 172.21.15.115:0/177998008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67c40cd7f0 con 0x7f67c40adf10 2023-12-01T01:54:37.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.501+0000 7f67c0ff9640 1 -- 172.21.15.115:0/177998008 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16099 (secure 0 0 0) 0x7f67c40cd7f0 con 0x7f67c40adf10 2023-12-01T01:54:37.503 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:37.503 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":125,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:54:34.052626+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:54:30.872492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-01T01:54:32.074780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:54:37.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 -- 172.21.15.115:0/177998008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f67a406a550 msgr2=0x7f67a406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f67a406a550 0x7f67a406ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f67b400b840 tx=0x7f67b4008040 comp rx=0 tx=0).stop 2023-12-01T01:54:37.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 -- 172.21.15.115:0/177998008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 msgr2=0x7f67c4150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 0x7f67c4150f60 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00b3f0 tx=0x7f67ac00b8b0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 -- 172.21.15.115:0/177998008 shutdown_connections 2023-12-01T01:54:37.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f67a406a550 0x7f67a406ca10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67c415e2f0 0x7f67c40c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67c40adf10 0x7f67c4150f60 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 --2- 172.21.15.115:0/177998008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c406f2f0 0x7f67c40cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 -- 172.21.15.115:0/177998008 >> 172.21.15.115:0/177998008 conn(0x7f67c40ce880 msgr2=0x7f67c406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:37.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 -- 172.21.15.115:0/177998008 shutdown_connections 2023-12-01T01:54:37.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.505+0000 7f67c9b92640 1 -- 172.21.15.115:0/177998008 wait complete. 2023-12-01T01:54:37.525 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T01:54:37.526 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:54:37.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.725+0000 7f18af299640 1 Processor -- start 2023-12-01T01:54:37.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.725+0000 7f18af299640 1 -- start start 2023-12-01T01:54:37.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18af299640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f18a80b5d80 0x7f18a80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18af299640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a80d1230 0x7f18a80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18af299640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a80d1b90 con 0x7f18a815cd20 2023-12-01T01:54:37.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18af299640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a80b2160 con 0x7f18a80b5d80 2023-12-01T01:54:37.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18af299640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80b8c80 con 0x7f18a80d1230 2023-12-01T01:54:37.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18ac834640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f18a80b5d80 0x7f18a80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18ac834640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f18a80b5d80 0x7f18a80d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49116/0 (socket says 172.21.15.115:49116) 2023-12-01T01:54:37.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18ac834640 1 -- 172.21.15.115:0/3042089434 learned_addr learned my addr 172.21.15.115:0/3042089434 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:37.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18ad836640 1 --2- 172.21.15.115:0/3042089434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a80d1230 0x7f18a80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18ac834640 1 -- 172.21.15.115:0/3042089434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a80d1230 msgr2=0x7f18a80d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:37.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18ac834640 1 --2- 172.21.15.115:0/3042089434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a80d1230 0x7f18a80d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18ac834640 1 -- 172.21.15.115:0/3042089434 >> v1:172.21.15.142:6789/0 conn(0x7f18a815cd20 legacy=0x7f18a80b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:37.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18ac834640 1 -- 172.21.15.115:0/3042089434 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80ce2e0 con 0x7f18a80b5d80 2023-12-01T01:54:37.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.727+0000 7f18ad836640 1 --2- 172.21.15.115:0/3042089434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a80d1230 0x7f18a80d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:37.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.729+0000 7f18ac834640 1 --2- 172.21.15.115:0/3042089434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80b5d80 0x7f18a80d24d0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1898009870 tx=0x7f189802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb79ba115b86b69 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:37.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.729+0000 7f189ffff640 1 -- 172.21.15.115:0/3042089434 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18980317c0 con 0x7f18a80b5d80 2023-12-01T01:54:37.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.729+0000 7f189ffff640 1 -- 172.21.15.115:0/3042089434 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1898031920 con 0x7f18a80b5d80 2023-12-01T01:54:37.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.729+0000 7f189ffff640 1 -- 172.21.15.115:0/3042089434 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898031cb0 con 0x7f18a80b5d80 2023-12-01T01:54:37.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.730+0000 7f18af299640 1 -- 172.21.15.115:0/3042089434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80b5d80 msgr2=0x7f18a80d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.730+0000 7f18af299640 1 --2- 172.21.15.115:0/3042089434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80b5d80 0x7f18a80d24d0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1898009870 tx=0x7f189802b860 comp rx=0 tx=0).stop 2023-12-01T01:54:37.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.730+0000 7f18af299640 1 -- 172.21.15.115:0/3042089434 shutdown_connections 2023-12-01T01:54:37.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.730+0000 7f18af299640 1 --2- 172.21.15.115:0/3042089434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a80d1230 0x7f18a80d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.730+0000 7f18af299640 1 --2- 172.21.15.115:0/3042089434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80b5d80 0x7f18a80d24d0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.730+0000 7f18af299640 1 -- 172.21.15.115:0/3042089434 >> 172.21.15.115:0/3042089434 conn(0x7f18a806a0e0 msgr2=0x7f18a806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:37.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.730+0000 7f18af299640 1 -- 172.21.15.115:0/3042089434 shutdown_connections 2023-12-01T01:54:37.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.730+0000 7f18af299640 1 -- 172.21.15.115:0/3042089434 wait complete. 2023-12-01T01:54:37.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.732+0000 7f18af299640 1 Processor -- start 2023-12-01T01:54:37.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.732+0000 7f18af299640 1 -- start start 2023-12-01T01:54:37.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.733+0000 7f18af299640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80b5d80 0x7f18a80b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.733+0000 7f18af299640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80d1230 0x7f18a80cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.733+0000 7f18af299640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815cd20 0x7f18a80c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.733+0000 7f18af299640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80ccb50 con 0x7f18a80d1230 2023-12-01T01:54:37.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.733+0000 7f18af299640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80b4b60 con 0x7f18a80b5d80 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.733+0000 7f18af299640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80b2350 con 0x7f18a815cd20 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.733+0000 7f18ac834640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80d1230 0x7f18a80cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.734+0000 7f18ac834640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80d1230 0x7f18a80cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49118/0 (socket says 172.21.15.115:49118) 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.734+0000 7f18ac834640 1 -- 172.21.15.115:0/789464488 learned_addr learned my addr 172.21.15.115:0/789464488 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.734+0000 7f18ad035640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80b5d80 0x7f18a80b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.734+0000 7f18ad836640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815cd20 0x7f18a80c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.734+0000 7f18ac834640 1 -- 172.21.15.115:0/789464488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815cd20 msgr2=0x7f18a80c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.734+0000 7f18ac834640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815cd20 0x7f18a80c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.734+0000 7f18ac834640 1 -- 172.21.15.115:0/789464488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80b5d80 msgr2=0x7f18a80b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.734+0000 7f18ac834640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80b5d80 0x7f18a80b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.734+0000 7f18ac834640 1 -- 172.21.15.115:0/789464488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980094d0 con 0x7f18a80d1230 2023-12-01T01:54:37.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.734+0000 7f18ac834640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80d1230 0x7f18a80cc610 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f1898000c00 tx=0x7f18980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:37.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.735+0000 7f189dffb640 1 -- 172.21.15.115:0/789464488 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18980329a0 con 0x7f18a80d1230 2023-12-01T01:54:37.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.735+0000 7f189dffb640 1 -- 172.21.15.115:0/789464488 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1898032b20 con 0x7f18a80d1230 2023-12-01T01:54:37.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.735+0000 7f189dffb640 1 -- 172.21.15.115:0/789464488 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18980453f0 con 0x7f18a80d1230 2023-12-01T01:54:37.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.735+0000 7f18af299640 1 -- 172.21.15.115:0/789464488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a80b2630 con 0x7f18a80d1230 2023-12-01T01:54:37.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.735+0000 7f18af299640 1 -- 172.21.15.115:0/789464488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a80c9010 con 0x7f18a80d1230 2023-12-01T01:54:37.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.736+0000 7f18af299640 1 -- 172.21.15.115:0/789464488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a80b2b50 con 0x7f18a80d1230 2023-12-01T01:54:37.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.737+0000 7f189dffb640 1 -- 172.21.15.115:0/789464488 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f189804d3f0 con 0x7f18a80d1230 2023-12-01T01:54:37.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.738+0000 7f189dffb640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f188c06a550 0x7f188c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:37.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.738+0000 7f189dffb640 1 -- 172.21.15.115:0/789464488 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18980aa980 con 0x7f18a80d1230 2023-12-01T01:54:37.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.738+0000 7f18ad035640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f188c06a550 0x7f188c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:37.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.739+0000 7f18ad035640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f188c06a550 0x7f188c06ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18a4004ea0 tx=0x7f18a4009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:37.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.743+0000 7f189dffb640 1 -- 172.21.15.115:0/789464488 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f18a80b2b50 con 0x7f18a80d1230 2023-12-01T01:54:37.846 INFO:tasks.ceph.mds.c.smithi115.stderr:2023-12-01T01:54:37.844+0000 7f1cf50d0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:54:37.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.928+0000 7f18af299640 1 -- 172.21.15.115:0/789464488 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18a80b24c0 con 0x7f188c06a550 2023-12-01T01:54:37.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.955+0000 7f189dffb640 1 -- 172.21.15.115:0/789464488 <== mgr.7767 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276268 (secure 0 0 0) 0x7f18a80b24c0 con 0x7f188c06a550 2023-12-01T01:54:37.958 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:37.962 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.962+0000 7f18af299640 1 -- 172.21.15.115:0/789464488 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f188c06a550 msgr2=0x7f188c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.962+0000 7f18af299640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f188c06a550 0x7f188c06ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18a4004ea0 tx=0x7f18a4009040 comp rx=0 tx=0).stop 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.962+0000 7f18af299640 1 -- 172.21.15.115:0/789464488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80d1230 msgr2=0x7f18a80cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.962+0000 7f18af299640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80d1230 0x7f18a80cc610 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f1898000c00 tx=0x7f18980046c0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.962+0000 7f18af299640 1 -- 172.21.15.115:0/789464488 shutdown_connections 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.962+0000 7f18af299640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f188c06a550 0x7f188c06ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.962+0000 7f18af299640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18a815cd20 0x7f18a80c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.962+0000 7f18af299640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80d1230 0x7f18a80cc610 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.962+0000 7f18af299640 1 --2- 172.21.15.115:0/789464488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a80b5d80 0x7f18a80b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.962+0000 7f18af299640 1 -- 172.21.15.115:0/789464488 >> 172.21.15.115:0/789464488 conn(0x7f18a806a0e0 msgr2=0x7f18a815b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:37.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.963+0000 7f18af299640 1 -- 172.21.15.115:0/789464488 shutdown_connections 2023-12-01T01:54:37.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:37.963+0000 7f18af299640 1 -- 172.21.15.115:0/789464488 wait complete. 2023-12-01T01:54:37.983 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T01:54:35.981775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":326028,"kb_used_data":12876,"kb_used_omap":0,"kb_used_meta":313088,"kb_avail":754648692,"statfs":{"total":773094113280,"available":772760260608,"internally_reserved":0,"allocated":13185024,"data_stored":6887747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":320602112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.419033"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.576834+0000","last_change":"2023-12-01T01:54:31.576834+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.576834+0000","last_undegraded":"2023-12-01T01:54:31.576834+0000","last_fullsized":"2023-12-01T01:54:31.576834+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:27:48.189318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.586559+0000","last_change":"2023-12-01T01:54:31.586559+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.586559+0000","last_undegraded":"2023-12-01T01:54:31.586559+0000","last_fullsized":"2023-12-01T01:54:31.586559+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:24.464526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.572874+0000","last_change":"2023-12-01T01:54:31.572874+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.572874+0000","last_undegraded":"2023-12-01T01:54:31.572874+0000","last_fullsized":"2023-12-01T01:54:31.572874+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:57.646717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.571570+0000","last_change":"2023-12-01T01:54:31.571570+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.571570+0000","last_undegraded":"2023-12-01T01:54:31.571570+0000","last_fullsized":"2023-12-01T01:54:31.571570+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:10.478463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.574497+0000","last_change":"2023-12-01T01:54:31.574497+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.574497+0000","last_undegraded":"2023-12-01T01:54:31.574497+0000","last_fullsized":"2023-12-01T01:54:31.574497+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:04.006938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.571319+0000","last_change":"2023-12-01T01:54:31.571319+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.571319+0000","last_undegraded":"2023-12-01T01:54:31.571319+0000","last_fullsized":"2023-12-01T01:54:31.571319+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:03.501711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.586952+0000","last_change":"2023-12-01T01:54:31.586952+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.586952+0000","last_undegraded":"2023-12-01T01:54:31.586952+0000","last_fullsized":"2023-12-01T01:54:31.586952+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:39:22.365281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.580989+0000","last_change":"2023-12-01T01:54:31.580989+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.580989+0000","last_undegraded":"2023-12-01T01:54:31.580989+0000","last_fullsized":"2023-12-01T01:54:31.580989+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:07:39.620156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.572515+0000","last_change":"2023-12-01T01:54:31.572515+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.572515+0000","last_undegraded":"2023-12-01T01:54:31.572515+0000","last_fullsized":"2023-12-01T01:54:31.572515+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:45:39.640112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.573179+0000","last_change":"2023-12-01T01:54:31.573179+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.573179+0000","last_undegraded":"2023-12-01T01:54:31.573179+0000","last_fullsized":"2023-12-01T01:54:31.573179+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:09.456228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.587722+0000","last_change":"2023-12-01T01:54:31.587722+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.587722+0000","last_undegraded":"2023-12-01T01:54:31.587722+0000","last_fullsized":"2023-12-01T01:54:31.587722+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:36:47.997971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.573675+0000","last_change":"2023-12-01T01:54:31.573675+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.573675+0000","last_undegraded":"2023-12-01T01:54:31.573675+0000","last_fullsized":"2023-12-01T01:54:31.573675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:40:30.130936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.582463+0000","last_change":"2023-12-01T01:54:31.582463+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.582463+0000","last_undegraded":"2023-12-01T01:54:31.582463+0000","last_fullsized":"2023-12-01T01:54:31.582463+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:34.393085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.569694+0000","last_change":"2023-12-01T01:54:31.569694+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.569694+0000","last_undegraded":"2023-12-01T01:54:31.569694+0000","last_fullsized":"2023-12-01T01:54:31.569694+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:15:41.132592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.573820+0000","last_change":"2023-12-01T01:54:31.573820+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.573820+0000","last_undegraded":"2023-12-01T01:54:31.573820+0000","last_fullsized":"2023-12-01T01:54:31.573820+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:31:14.275675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.569671+0000","last_change":"2023-12-01T01:54:31.569671+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.569671+0000","last_undegraded":"2023-12-01T01:54:31.569671+0000","last_fullsized":"2023-12-01T01:54:31.569671+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:41:01.381821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.574076+0000","last_change":"2023-12-01T01:54:31.574076+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.574076+0000","last_undegraded":"2023-12-01T01:54:31.574076+0000","last_fullsized":"2023-12-01T01:54:31.574076+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:06.389420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.587032+0000","last_change":"2023-12-01T01:54:31.587032+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.587032+0000","last_undegraded":"2023-12-01T01:54:31.587032+0000","last_fullsized":"2023-12-01T01:54:31.587032+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:48:53.395467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.581136+0000","last_change":"2023-12-01T01:54:31.581136+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.581136+0000","last_undegraded":"2023-12-01T01:54:31.581136+0000","last_fullsized":"2023-12-01T01:54:31.581136+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:39:32.315098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.573390+0000","last_change":"2023-12-01T01:54:31.573390+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.573390+0000","last_undegraded":"2023-12-01T01:54:31.573390+0000","last_fullsized":"2023-12-01T01:54:31.573390+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:16.042920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.579281+0000","last_change":"2023-12-01T01:54:31.579281+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.579281+0000","last_undegraded":"2023-12-01T01:54:31.579281+0000","last_fullsized":"2023-12-01T01:54:31.579281+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:15:41.132592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.746068+0000","last_change":"2023-12-01T01:54:31.746068+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.746068+0000","last_undegraded":"2023-12-01T01:54:31.746068+0000","last_fullsized":"2023-12-01T01:54:31.746068+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:13:26.203448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.575864+0000","last_change":"2023-12-01T01:54:31.575864+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.575864+0000","last_undegraded":"2023-12-01T01:54:31.575864+0000","last_fullsized":"2023-12-01T01:54:31.575864+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:10:19.855954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.574073+0000","last_change":"2023-12-01T01:54:31.574073+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.574073+0000","last_undegraded":"2023-12-01T01:54:31.574073+0000","last_fullsized":"2023-12-01T01:54:31.574073+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:58.343037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.582327+0000","last_change":"2023-12-01T01:54:31.582327+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.582327+0000","last_undegraded":"2023-12-01T01:54:31.582327+0000","last_fullsized":"2023-12-01T01:54:31.582327+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:24.407114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.581200+0000","last_change":"2023-12-01T01:54:31.581200+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.581200+0000","last_undegraded":"2023-12-01T01:54:31.581200+0000","last_fullsized":"2023-12-01T01:54:31.581200+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:29:36.651831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.577539+0000","last_change":"2023-12-01T01:54:31.577539+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.577539+0000","last_undegraded":"2023-12-01T01:54:31.577539+0000","last_fullsized":"2023-12-01T01:54:31.577539+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:41:01.381821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.572675+0000","last_change":"2023-12-01T01:54:31.572675+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.572675+0000","last_undegraded":"2023-12-01T01:54:31.572675+0000","last_fullsized":"2023-12-01T01:54:31.572675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:45:44.823359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.577905+0000","last_change":"2023-12-01T01:54:31.577905+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.577905+0000","last_undegraded":"2023-12-01T01:54:31.577905+0000","last_fullsized":"2023-12-01T01:54:31.577905+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:10.478463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.578361+0000","last_change":"2023-12-01T01:54:31.578361+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.578361+0000","last_undegraded":"2023-12-01T01:54:31.578361+0000","last_fullsized":"2023-12-01T01:54:31.578361+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:52:01.375954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.572297+0000","last_change":"2023-12-01T01:54:31.572297+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.572297+0000","last_undegraded":"2023-12-01T01:54:31.572297+0000","last_fullsized":"2023-12-01T01:54:31.572297+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:07:53.810438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:54:32.561300+0000","last_change":"2023-12-01T01:54:32.561300+0000","last_active":"2023-12-01T01:54:32.561300+0000","last_peered":"2023-12-01T01:54:32.561300+0000","last_clean":"2023-12-01T01:54:32.561300+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:32.561300+0000","last_undegraded":"2023-12-01T01:54:32.561300+0000","last_fullsized":"2023-12-01T01:54:32.561300+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":122,"state":"creating+peering","last_fresh":"2023-12-01T01:54:31.575178+0000","last_change":"2023-12-01T01:54:31.575178+0000","last_active":"2023-12-01T01:54:31.556692+0000","last_peered":"2023-12-01T01:54:31.556692+0000","last_clean":"2023-12-01T01:54:31.556692+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:54:31.575178+0000","last_undegraded":"2023-12-01T01:54:31.575178+0000","last_fullsized":"2023-12-01T01:54:31.575178+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:27:48.367833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"113'205","reported_seq":571,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-12-01T01:54:28.676048+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:54:28.676048+0000","last_peered":"2023-12-01T01:54:28.676048+0000","last_clean":"2023-12-01T01:54:28.676048+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:54:28.676048+0000","last_undegraded":"2023-12-01T01:54:28.676048+0000","last_fullsized":"2023-12-01T01:54:28.676048+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575071,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1460,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1495040,"data_stored":711195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:53:31 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0329999999999999,"15min":1.0700000000000001},"min":{"1min":0.70999999999999996,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.8089999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0129999999999999,"15min":1.054},"min":{"1min":0.79300000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.381,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.046}]},{"osd":1,"last update":"Fri Dec 1 01:53:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1240000000000001,"15min":1.212},"min":{"1min":0.73699999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.5860000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.161},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.1890000000000001,"15min":1.254},"min":{"1min":0.77900000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5840000000000001,"5min":1.744,"15min":1.744},"last":1.123}]},{"osd":2,"last update":"Fri Dec 1 01:53:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0920000000000001,"15min":1.123},"min":{"1min":0.64100000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5,"5min":1.607,"15min":1.728},"last":1.083},{"interface":"front","average":{"1min":1.194,"5min":1.2010000000000001,"15min":1.2190000000000001},"min":{"1min":0.84199999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.681,"5min":1.849,"15min":1.9690000000000001},"last":1.2649999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:53:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.1759999999999999,"15min":1.2350000000000001},"min":{"1min":0.69699999999999995,"5min":0.58599999999999997,"15min":0.57499999999999996},"max":{"1min":1.637,"5min":1.7350000000000001,"15min":2.0550000000000002},"last":1.216},{"interface":"front","average":{"1min":1.216,"5min":1.288,"15min":1.337},"min":{"1min":0.82999999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.478,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.347}]},{"osd":4,"last update":"Fri Dec 1 01:53:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2989999999999999,"15min":1.288},"min":{"1min":0.79300000000000004,"5min":0.56100000000000005,"15min":0.55500000000000005},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":2.2130000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.82799999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.8380000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4059999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:53:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.2689999999999999,"15min":1.2549999999999999},"min":{"1min":0.78000000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.577,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.478,"5min":1.5,"15min":1.5189999999999999},"min":{"1min":1.2,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.9379999999999999,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4530000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:53:31 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3819999999999999,"15min":1.381},"min":{"1min":1.03,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.7250000000000001,"5min":2.294,"15min":2.3420000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.375,"15min":1.3879999999999999},"min":{"1min":0.875,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.954,"5min":1.954,"15min":2.3500000000000001},"last":1.288}]}]},{"osd":6,"up_from":13,"seq":55834575072,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36604,"kb_used_data":1460,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335236,"statfs":{"total":96636764160,"available":96599281664,"internally_reserved":0,"allocated":1495040,"data_stored":711195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0569999999999999,"15min":1.1020000000000001},"min":{"1min":0.73199999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.431,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.105,"15min":1.1299999999999999},"min":{"1min":0.79000000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.579,"5min":2.5569999999999999,"15min":2.5569999999999999},"last":1.163}]},{"osd":1,"last update":"Fri Dec 1 01:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.161,"15min":1.1679999999999999},"min":{"1min":0.92200000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.5329999999999999,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.173,"15min":1.206},"min":{"1min":0.86799999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5980000000000001,"5min":2.0720000000000001,"15min":2.0920000000000001},"last":1.117}]},{"osd":2,"last update":"Fri Dec 1 01:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.141,"15min":1.171},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.4850000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.234,"15min":1.2390000000000001},"min":{"1min":0.81899999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.599,"5min":3.024,"15min":3.024},"last":1.0700000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.26,"15min":1.2749999999999999},"min":{"1min":0.88900000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.611,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.385,"15min":1.401},"min":{"1min":1.1439999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.5900000000000001,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.6020000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.304,"15min":1.268},"min":{"1min":0.85099999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6319999999999999,"5min":2.226,"15min":2.226},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.367,"15min":1.341},"min":{"1min":0.97799999999999998,"5min":0.91400000000000003,"15min":0.78100000000000003},"max":{"1min":1.736,"5min":1.931,"15min":2.2839999999999998},"last":1.712}]},{"osd":5,"last update":"Fri Dec 1 01:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.427,"15min":1.397},"min":{"1min":1.1679999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7529999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":2.2250000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.4970000000000001,"15min":1.474},"min":{"1min":1.0029999999999999,"5min":0.96499999999999997,"15min":0.84099999999999997},"max":{"1min":1.9059999999999999,"5min":2.6819999999999999,"15min":2.8580000000000001},"last":1.8109999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5229999999999999,"15min":1.504},"min":{"1min":1.0269999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.9330000000000001,"5min":2.762,"15min":2.762},"last":1.9430000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.569,"15min":1.5629999999999999},"min":{"1min":1.238,"5min":1.038,"15min":1.038},"max":{"1min":1.9810000000000001,"5min":2.8479999999999999,"15min":2.8479999999999999},"last":2.1379999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575071,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":2101248,"data_stored":1307380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.111,"15min":1.1080000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.522,"5min":1.8240000000000001,"15min":2.2010000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1060000000000001,"15min":1.1439999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5609999999999999,"5min":1.7130000000000001,"15min":2.153},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.149,"15min":1.1499999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6040000000000001,"5min":1.77,"15min":1.77},"last":1.171},{"interface":"front","average":{"1min":1.139,"5min":1.145,"15min":1.157},"min":{"1min":0.754,"5min":0.72999999999999998,"15min":0.67800000000000005},"max":{"1min":1.4890000000000001,"5min":1.6519999999999999,"15min":1.7330000000000001},"last":0.89000000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.147,"15min":1.1879999999999999},"min":{"1min":0.71999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.532,"5min":1.702,"15min":1.702},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.248,"5min":1.226,"15min":1.2370000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.605,"5min":1.9239999999999999,"15min":2.0630000000000002},"last":0.83899999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2190000000000001,"15min":1.2629999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.651,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3109999999999999,"15min":1.304},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.3129999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.226,"15min":1.224},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.474,"5min":1.984,"15min":1.984},"last":1.01},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.3009999999999999,"15min":1.3340000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.6299999999999999,"5min":1.927,"15min":1.927},"last":1.2230000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3300000000000001,"15min":1.325},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.694,"5min":1.885,"15min":1.9790000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3799999999999999,"15min":1.371},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.746,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.2669999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.45,"15min":1.4670000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.7210000000000001,"5min":2.0499999999999998,"15min":2.0920000000000001},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.532,"15min":1.5229999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.861,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.383}]}]},{"osd":4,"up_from":13,"seq":55834575071,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36616,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335224,"statfs":{"total":96636764160,"available":96599269376,"internally_reserved":0,"allocated":1507328,"data_stored":717012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.77000000000000002,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5509999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1599999999999999,"15min":1.1739999999999999},"min":{"1min":0.80600000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.603,"5min":1.778,"15min":2.1099999999999999},"last":0.78100000000000003}]},{"osd":1,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.222,"15min":1.214},"min":{"1min":0.83699999999999997,"5min":0.72399999999999998,"15min":0.65900000000000003},"max":{"1min":1.8460000000000001,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.147,"15min":1.135},"min":{"1min":0.72999999999999998,"5min":0.67300000000000004,"15min":0.64200000000000002},"max":{"1min":1.6060000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2150000000000001,"15min":1.202},"min":{"1min":0.91800000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4330000000000001,"5min":1.871,"15min":1.871},"last":1.171},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.288,"15min":1.29},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.599,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.006}]},{"osd":3,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.246,"15min":1.244},"min":{"1min":0.93300000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5169999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.3180000000000001,"15min":1.3169999999999999},"min":{"1min":1.071,"5min":0.753,"15min":0.753},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.903},"last":1.266}]},{"osd":5,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.337,"15min":1.321},"min":{"1min":0.95999999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.7010000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.3080000000000001,"15min":1.292},"min":{"1min":1.0189999999999999,"5min":0.90300000000000002,"15min":0.85399999999999998},"max":{"1min":1.627,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.05}]},{"osd":6,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":1.3180000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.7649999999999999,"5min":1.8799999999999999,"15min":1.9950000000000001},"last":1.371},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.47,"15min":1.4790000000000001},"min":{"1min":1.2330000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.96,"5min":2.306,"15min":2.54},"last":1.4590000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.468,"15min":1.4630000000000001},"min":{"1min":1.2989999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.8799999999999999,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.5620000000000001,"15min":1.5660000000000001},"min":{"1min":1.1299999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":1.9830000000000001,"5min":2.04,"15min":2.157},"last":1.5940000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575070,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58748,"kb_used_data":1460,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313092,"statfs":{"total":96636764160,"available":96576606208,"internally_reserved":0,"allocated":1495040,"data_stored":711195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:53:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.81699999999999995,"15min":0.85399999999999998},"min":{"1min":0.61599999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.3049999999999999,"5min":1.3320000000000001,"15min":1.583},"last":1.472},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.90600000000000003,"15min":0.98599999999999999},"min":{"1min":0.59299999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.4770000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.83399999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.96399999999999997,"15min":1.0600000000000001},"min":{"1min":0.753,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6339999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.88500000000000001,"15min":0.95599999999999996},"min":{"1min":0.66300000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.4079999999999999,"5min":1.5129999999999999,"15min":1.6519999999999999},"last":0.91100000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.96699999999999997,"15min":1.044},"min":{"1min":0.502,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.075,"15min":1.1839999999999999},"min":{"1min":0.64700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.405,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.21}]},{"osd":4,"last update":"Fri Dec 1 01:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1759999999999999,"15min":1.232},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.546,"5min":1.573,"15min":1.9319999999999999},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.194,"15min":1.232},"min":{"1min":0.96099999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5580000000000001,"5min":1.909,"15min":2.0419999999999998},"last":1.506}]},{"osd":5,"last update":"Fri Dec 1 01:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2110000000000001,"15min":1.28},"min":{"1min":0.84699999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5029999999999999,"5min":1.7669999999999999,"15min":2.0510000000000002},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.2569999999999999,"15min":1.2989999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.831,"5min":1.831,"15min":2.1880000000000002},"last":1.252}]},{"osd":6,"last update":"Fri Dec 1 01:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.3360000000000001,"15min":1.3759999999999999},"min":{"1min":0.95299999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.694,"5min":1.984,"15min":1.984},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.3600000000000001,"15min":1.4099999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.76,"5min":1.9119999999999999,"15min":1.99},"last":1.6859999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.333,"15min":1.415},"min":{"1min":0.879,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.845,"5min":2.0720000000000001,"15min":2.129},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.387,"15min":1.452},"min":{"1min":0.95599999999999996,"5min":0.87,"15min":0.87},"max":{"1min":1.8660000000000001,"5min":2.0430000000000001,"15min":2.2130000000000001},"last":1.8879999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575070,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36616,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335224,"statfs":{"total":96636764160,"available":96599269376,"internally_reserved":0,"allocated":1507328,"data_stored":717012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:53:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77800000000000002,"15min":0.81899999999999995},"min":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.5529999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.94899999999999995,"15min":1.002},"min":{"1min":0.504,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3480000000000001,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Dec 1 01:53:28 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.93200000000000005,"15min":0.98499999999999999},"min":{"1min":0.375,"5min":0.373,"15min":0.373},"max":{"1min":1.4990000000000001,"5min":2.5,"15min":2.5},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.03,"5min":1.032,"15min":1.0840000000000001},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3300000000000001,"5min":1.909,"15min":1.909},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Dec 1 01:53:28 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.90500000000000003,"15min":1.002},"min":{"1min":0.47699999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.278,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.071,"15min":1.151},"min":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3360000000000001,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.95699999999999996}]},{"osd":4,"last update":"Fri Dec 1 01:53:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2569999999999999,"15min":1.294},"min":{"1min":0.81299999999999994,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5509999999999999,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1850000000000001,"15min":1.252},"min":{"1min":0.79100000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.411,"5min":1.6879999999999999,"15min":1.8080000000000001},"last":1.4319999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:53:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.268,"15min":1.3089999999999999},"min":{"1min":0.79800000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.8149999999999999,"5min":2.948,"15min":2.948},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.325,"15min":1.3500000000000001},"min":{"1min":0.70799999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.988,"5min":2.048,"15min":2.048},"last":1.1439999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:53:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2649999999999999,"15min":1.3109999999999999},"min":{"1min":0.96899999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.544,"5min":1.78,"15min":1.988},"last":1.593},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.385,"15min":1.421},"min":{"1min":0.85599999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.825,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.2150000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:53:28 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.415,"15min":1.429},"min":{"1min":0.80000000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.0880000000000001,"5min":2.1779999999999999,"15min":2.214},"last":1.272},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3149999999999999,"15min":1.375},"min":{"1min":0.75,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.631,"5min":1.9259999999999999,"15min":2.101},"last":1.534}]}]},{"osd":1,"up_from":13,"seq":55834575069,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59328,"kb_used_data":2040,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312512,"statfs":{"total":96636764160,"available":96576012288,"internally_reserved":0,"allocated":2088960,"data_stored":1301563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:53:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77300000000000002,"15min":0.83999999999999997},"min":{"1min":0.51600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.175,"5min":1.341,"15min":1.5049999999999999},"last":1.369},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":1.016},"min":{"1min":0.41099999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4350000000000001,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:53:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84099999999999997,"15min":0.89200000000000002},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.246,"5min":1.2949999999999999,"15min":1.54},"last":0.879},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.999,"15min":1.083},"min":{"1min":0.45700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.349,"5min":1.7250000000000001,"15min":1.804},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1299999999999999,"15min":1.2110000000000001},"min":{"1min":0.67700000000000005,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.5129999999999999,"5min":1.827,"15min":1.827},"last":1.234},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95899999999999996,"15min":1.042},"min":{"1min":0.58599999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.466,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.92800000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1439999999999999,"15min":1.1850000000000001},"min":{"1min":0.85899999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.4850000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.292,"15min":1.335},"min":{"1min":0.90400000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.403}]},{"osd":5,"last update":"Fri Dec 1 01:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2470000000000001,"15min":1.2749999999999999},"min":{"1min":0.84399999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.76,"5min":1.988,"15min":1.988},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.224,"15min":1.2749999999999999},"min":{"1min":0.79200000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.587,"5min":1.6799999999999999,"15min":1.825},"last":1.27}]},{"osd":6,"last update":"Fri Dec 1 01:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.288,"15min":1.3320000000000001},"min":{"1min":1.0189999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6699999999999999,"5min":1.839,"15min":1.861},"last":1.3},{"interface":"front","average":{"1min":1.335,"5min":1.296,"15min":1.3360000000000001},"min":{"1min":0.89300000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.657,"5min":1.742,"15min":1.8280000000000001},"last":1.111}]},{"osd":7,"last update":"Fri Dec 1 01:53:30 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.357,"15min":1.4259999999999999},"min":{"1min":0.97799999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":2.0609999999999999},"last":1.446},{"interface":"front","average":{"1min":1.325,"5min":1.3480000000000001,"15min":1.403},"min":{"1min":0.81799999999999995,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.681,"5min":1.931,"15min":1.992},"last":1.4159999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575071,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1460,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1495040,"data_stored":711195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.80700000000000005,"15min":0.83999999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.081,"5min":1.3520000000000001,"15min":1.8109999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.93500000000000005,"15min":0.96499999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":1.5620000000000001,"5min":1.5860000000000001,"15min":1.6910000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95499999999999996,"15min":0.98799999999999999},"min":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.42699999999999999},"max":{"1min":1.3799999999999999,"5min":1.528,"15min":2.157},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.018,"5min":1.0089999999999999,"15min":1.054},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.389,"5min":1.3999999999999999,"15min":1.728},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97599999999999998,"15min":1.004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.331,"5min":1.4850000000000001,"15min":1.7509999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.137,"15min":1.155},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.46500000000000002},"max":{"1min":1.4830000000000001,"5min":1.7070000000000001,"15min":2.2610000000000001},"last":0.83999999999999997}]},{"osd":4,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.248,"15min":1.2949999999999999},"min":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6140000000000001,"5min":1.6240000000000001,"15min":2.3719999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.2,"5min":1.1930000000000001,"15min":1.248},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5800000000000001,"5min":1.7989999999999999,"15min":1.823},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3,"15min":1.333},"min":{"1min":0.88700000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.5600000000000001,"5min":1.841,"15min":2.351},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.278,"15min":1.3220000000000001},"min":{"1min":0.89800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6890000000000001,"5min":1.7509999999999999,"15min":2.2909999999999999},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.272,"15min":1.3049999999999999},"min":{"1min":0.69299999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5700000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3600000000000001,"15min":1.381},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7290000000000001,"5min":1.9159999999999999,"15min":2.4380000000000002},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.353,"15min":1.3939999999999999},"min":{"1min":0.70699999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":2.4279999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3500000000000001,"15min":1.383},"min":{"1min":0.82699999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.7669999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.1120000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:54:37.988 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:54:38.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.242+0000 7fe37314e640 1 Processor -- start 2023-12-01T01:54:38.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.242+0000 7fe37314e640 1 -- start start 2023-12-01T01:54:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.243+0000 7fe37314e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0cad70 0x7fe36c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.243+0000 7fe37314e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c0add10 con 0x7fe36c15e2f0 2023-12-01T01:54:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.244+0000 7fe37314e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c0c5760 con 0x7fe36c06f2f0 2023-12-01T01:54:38.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.244+0000 7fe37314e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c15e6d0 con 0x7fe36c0cad70 2023-12-01T01:54:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.244+0000 7fe370eea640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe36c15e2f0 0x7fe36c0ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48308/0 (socket says 172.21.15.115:48308) 2023-12-01T01:54:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.244+0000 7fe370eea640 1 -- 172.21.15.115:0/1039402781 learned_addr learned my addr 172.21.15.115:0/1039402781 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.244+0000 7fe36bfff640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0cad70 0x7fe36c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.244+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167314150 0 0) 0x7fe36c0add10 con 0x7fe36c15e2f0 2023-12-01T01:54:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.244+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe35c003520 con 0x7fe36c15e2f0 2023-12-01T01:54:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4064958305 0 0) 0x7fe35c003520 con 0x7fe36c15e2f0 2023-12-01T01:54:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0cad70 msgr2=0x7fe36c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0cad70 0x7fe36c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 >> v1:172.21.15.142:6789/0 conn(0x7fe36c06f2f0 legacy=0x7fe36c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0cf7b0 con 0x7fe36c15e2f0 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fe360001670 con 0x7fe36c15e2f0 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 >> v1:172.21.15.115:6789/0 conn(0x7fe36c15e2f0 legacy=0x7fe36c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35c003a10 0x7fe35c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe35c0047c0 0x7fe35c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0cad70 0x7fe35c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c15e6d0 con 0x7fe35c0047c0 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0054c0 con 0x7fe35c003a10 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c005680 con 0x7fe36c0cad70 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.245+0000 7fe36bfff640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe35c0047c0 0x7fe35c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.246+0000 7fe370eea640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35c003a10 0x7fe35c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.246+0000 7fe3716eb640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0cad70 0x7fe35c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.246+0000 7fe36bfff640 1 -- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0cad70 msgr2=0x7fe35c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.246+0000 7fe36bfff640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0cad70 0x7fe35c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.246+0000 7fe36bfff640 1 -- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35c003a10 msgr2=0x7fe35c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.246+0000 7fe36bfff640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35c003a10 0x7fe35c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.246+0000 7fe36bfff640 1 -- 172.21.15.115:0/1039402781 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c003520 con 0x7fe35c0047c0 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.247+0000 7fe36bfff640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe35c0047c0 0x7fe35c004c40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe358009ac0 tx=0x7fe35c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbb87c7aeb10860 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.248+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35802c6b0 con 0x7fe35c0047c0 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.248+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35802c810 con 0x7fe35c0047c0 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.248+0000 7fe36b7fe640 1 -- 172.21.15.115:0/1039402781 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35802cba0 con 0x7fe35c0047c0 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.248+0000 7fe37314e640 1 -- 172.21.15.115:0/1039402781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe35c0047c0 msgr2=0x7fe35c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.248+0000 7fe37314e640 1 --2- 172.21.15.115:0/1039402781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe35c0047c0 0x7fe35c004c40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe358009ac0 tx=0x7fe35c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.248+0000 7fe3716eb640 1 -- 172.21.15.115:0/1039402781 reap_dead start 2023-12-01T01:54:38.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.249+0000 7fe37314e640 1 -- 172.21.15.115:0/1039402781 shutdown_connections 2023-12-01T01:54:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.249+0000 7fe37314e640 1 -- 172.21.15.115:0/1039402781 >> 172.21.15.115:0/1039402781 conn(0x7fe36c0ce880 msgr2=0x7fe36c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.249+0000 7fe37314e640 1 -- 172.21.15.115:0/1039402781 shutdown_connections 2023-12-01T01:54:38.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.249+0000 7fe37314e640 1 -- 172.21.15.115:0/1039402781 wait complete. 2023-12-01T01:54:38.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.250+0000 7fe37314e640 1 Processor -- start 2023-12-01T01:54:38.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.251+0000 7fe37314e640 1 -- start start 2023-12-01T01:54:38.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.251+0000 7fe37314e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35c0047c0 0x7fe36c159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.251+0000 7fe37314e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0c23b0 0x7fe36c0c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe37314e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe36c0c2d70 0x7fe36c0c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe37314e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c15a3a0 con 0x7fe36c0c2d70 2023-12-01T01:54:38.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe37314e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0c7760 con 0x7fe35c0047c0 2023-12-01T01:54:38.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe37314e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0c31f0 con 0x7fe36c0c23b0 2023-12-01T01:54:38.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe3716eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe36c0c2d70 0x7fe36c0c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe3716eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe36c0c2d70 0x7fe36c0c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49140/0 (socket says 172.21.15.115:49140) 2023-12-01T01:54:38.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe3716eb640 1 -- 172.21.15.115:0/3017320553 learned_addr learned my addr 172.21.15.115:0/3017320553 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:38.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe370eea640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35c0047c0 0x7fe36c159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe36bfff640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0c23b0 0x7fe36c0c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe3716eb640 1 -- 172.21.15.115:0/3017320553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0c23b0 msgr2=0x7fe36c0c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe3716eb640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0c23b0 0x7fe36c0c2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe3716eb640 1 -- 172.21.15.115:0/3017320553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35c0047c0 msgr2=0x7fe36c159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe3716eb640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35c0047c0 0x7fe36c159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe3716eb640 1 -- 172.21.15.115:0/3017320553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0c79b0 con 0x7fe36c0c2d70 2023-12-01T01:54:38.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.252+0000 7fe370eea640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35c0047c0 0x7fe36c159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.253+0000 7fe3716eb640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe36c0c2d70 0x7fe36c0c4030 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe36c06fba0 tx=0x7fe36400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.253+0000 7fe3697fa640 1 -- 172.21.15.115:0/3017320553 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364003970 con 0x7fe36c0c2d70 2023-12-01T01:54:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.253+0000 7fe3697fa640 1 -- 172.21.15.115:0/3017320553 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe364003ad0 con 0x7fe36c0c2d70 2023-12-01T01:54:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.253+0000 7fe3697fa640 1 -- 172.21.15.115:0/3017320553 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364003e80 con 0x7fe36c0c2d70 2023-12-01T01:54:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.253+0000 7fe37314e640 1 -- 172.21.15.115:0/3017320553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c0c7c90 con 0x7fe36c0c2d70 2023-12-01T01:54:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.253+0000 7fe37314e640 1 -- 172.21.15.115:0/3017320553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c0c8250 con 0x7fe36c0c2d70 2023-12-01T01:54:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.254+0000 7fe37314e640 1 -- 172.21.15.115:0/3017320553 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c06fda0 con 0x7fe36c0c2d70 2023-12-01T01:54:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.255+0000 7fe3697fa640 1 -- 172.21.15.115:0/3017320553 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe3640050d0 con 0x7fe36c0c2d70 2023-12-01T01:54:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.257+0000 7fe3697fa640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe35006a580 0x7fe35006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.257+0000 7fe3697fa640 1 -- 172.21.15.115:0/3017320553 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe364088800 con 0x7fe36c0c2d70 2023-12-01T01:54:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.257+0000 7fe370eea640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe35006a580 0x7fe35006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.257+0000 7fe370eea640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe35006a580 0x7fe35006ca40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe360001c90 tx=0x7fe360009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:38.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.261+0000 7fe3697fa640 1 -- 172.21.15.115:0/3017320553 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe364053ce0 con 0x7fe36c0c2d70 2023-12-01T01:54:38.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.447+0000 7fe37314e640 1 -- 172.21.15.115:0/3017320553 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe36c0b43d0 con 0x7fe35006a580 2023-12-01T01:54:38.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.465+0000 7fe3697fa640 1 -- 172.21.15.115:0/3017320553 <== mgr.7767 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287365 (secure 0 0 0) 0x7fe360007030 con 0x7fe35006a580 2023-12-01T01:54:38.469 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:38.472 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:54:38.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 -- 172.21.15.115:0/3017320553 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe35006a580 msgr2=0x7fe35006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe35006a580 0x7fe35006ca40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe360001c90 tx=0x7fe360009040 comp rx=0 tx=0).stop 2023-12-01T01:54:38.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 -- 172.21.15.115:0/3017320553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe36c0c2d70 msgr2=0x7fe36c0c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe36c0c2d70 0x7fe36c0c4030 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe36c06fba0 tx=0x7fe36400ec10 comp rx=0 tx=0).stop 2023-12-01T01:54:38.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 -- 172.21.15.115:0/3017320553 shutdown_connections 2023-12-01T01:54:38.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe35006a580 0x7fe35006ca40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe36c0c23b0 0x7fe36c0c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe36c0c2d70 0x7fe36c0c4030 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 --2- 172.21.15.115:0/3017320553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35c0047c0 0x7fe36c159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 -- 172.21.15.115:0/3017320553 >> 172.21.15.115:0/3017320553 conn(0x7fe36c0ce880 msgr2=0x7fe36c0b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:38.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 -- 172.21.15.115:0/3017320553 shutdown_connections 2023-12-01T01:54:38.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.472+0000 7fe37314e640 1 -- 172.21.15.115:0/3017320553 wait complete. 2023-12-01T01:54:38.493 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T01:54:37.982524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":326440,"kb_used_data":13288,"kb_used_omap":0,"kb_used_meta":313088,"kb_avail":754648280,"statfs":{"total":773094113280,"available":772759838720,"internally_reserved":0,"allocated":13606912,"data_stored":7076356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":320602112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001892"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.324965+0000","last_change":"2023-12-01T01:54:33.095006+0000","last_active":"2023-12-01T01:54:34.324965+0000","last_peered":"2023-12-01T01:54:34.324965+0000","last_clean":"2023-12-01T01:54:34.324965+0000","last_became_active":"2023-12-01T01:54:33.094499+0000","last_became_peered":"2023-12-01T01:54:33.094499+0000","last_unstale":"2023-12-01T01:54:34.324965+0000","last_undegraded":"2023-12-01T01:54:34.324965+0000","last_fullsized":"2023-12-01T01:54:34.324965+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:22:59.409731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.322232+0000","last_change":"2023-12-01T01:54:33.099131+0000","last_active":"2023-12-01T01:54:34.322232+0000","last_peered":"2023-12-01T01:54:34.322232+0000","last_clean":"2023-12-01T01:54:34.322232+0000","last_became_active":"2023-12-01T01:54:33.098337+0000","last_became_peered":"2023-12-01T01:54:33.098337+0000","last_unstale":"2023-12-01T01:54:34.322232+0000","last_undegraded":"2023-12-01T01:54:34.322232+0000","last_fullsized":"2023-12-01T01:54:34.322232+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:03:57.356742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.075343+0000","last_change":"2023-12-01T01:54:33.077758+0000","last_active":"2023-12-01T01:54:34.075343+0000","last_peered":"2023-12-01T01:54:34.075343+0000","last_clean":"2023-12-01T01:54:34.075343+0000","last_became_active":"2023-12-01T01:54:33.077426+0000","last_became_peered":"2023-12-01T01:54:33.077426+0000","last_unstale":"2023-12-01T01:54:34.075343+0000","last_undegraded":"2023-12-01T01:54:34.075343+0000","last_fullsized":"2023-12-01T01:54:34.075343+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:29.476536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.325453+0000","last_change":"2023-12-01T01:54:33.099263+0000","last_active":"2023-12-01T01:54:34.325453+0000","last_peered":"2023-12-01T01:54:34.325453+0000","last_clean":"2023-12-01T01:54:34.325453+0000","last_became_active":"2023-12-01T01:54:33.097482+0000","last_became_peered":"2023-12-01T01:54:33.097482+0000","last_unstale":"2023-12-01T01:54:34.325453+0000","last_undegraded":"2023-12-01T01:54:34.325453+0000","last_fullsized":"2023-12-01T01:54:34.325453+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:21:37.993719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.069781+0000","last_change":"2023-12-01T01:54:33.076126+0000","last_active":"2023-12-01T01:54:34.069781+0000","last_peered":"2023-12-01T01:54:34.069781+0000","last_clean":"2023-12-01T01:54:34.069781+0000","last_became_active":"2023-12-01T01:54:33.075882+0000","last_became_peered":"2023-12-01T01:54:33.075882+0000","last_unstale":"2023-12-01T01:54:34.069781+0000","last_undegraded":"2023-12-01T01:54:34.069781+0000","last_fullsized":"2023-12-01T01:54:34.069781+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:41:18.928991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.069932+0000","last_change":"2023-12-01T01:54:33.077287+0000","last_active":"2023-12-01T01:54:34.069932+0000","last_peered":"2023-12-01T01:54:34.069932+0000","last_clean":"2023-12-01T01:54:34.069932+0000","last_became_active":"2023-12-01T01:54:33.076642+0000","last_became_peered":"2023-12-01T01:54:33.076642+0000","last_unstale":"2023-12-01T01:54:34.069932+0000","last_undegraded":"2023-12-01T01:54:34.069932+0000","last_fullsized":"2023-12-01T01:54:34.069932+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:14:13.496592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.069403+0000","last_change":"2023-12-01T01:54:33.068438+0000","last_active":"2023-12-01T01:54:34.069403+0000","last_peered":"2023-12-01T01:54:34.069403+0000","last_clean":"2023-12-01T01:54:34.069403+0000","last_became_active":"2023-12-01T01:54:33.068184+0000","last_became_peered":"2023-12-01T01:54:33.068184+0000","last_unstale":"2023-12-01T01:54:34.069403+0000","last_undegraded":"2023-12-01T01:54:34.069403+0000","last_fullsized":"2023-12-01T01:54:34.069403+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:25:09.751693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.074718+0000","last_change":"2023-12-01T01:54:33.078693+0000","last_active":"2023-12-01T01:54:34.074718+0000","last_peered":"2023-12-01T01:54:34.074718+0000","last_clean":"2023-12-01T01:54:34.074718+0000","last_became_active":"2023-12-01T01:54:33.078377+0000","last_became_peered":"2023-12-01T01:54:33.078377+0000","last_unstale":"2023-12-01T01:54:34.074718+0000","last_undegraded":"2023-12-01T01:54:34.074718+0000","last_fullsized":"2023-12-01T01:54:34.074718+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:36:53.994382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.070225+0000","last_change":"2023-12-01T01:54:33.086044+0000","last_active":"2023-12-01T01:54:34.070225+0000","last_peered":"2023-12-01T01:54:34.070225+0000","last_clean":"2023-12-01T01:54:34.070225+0000","last_became_active":"2023-12-01T01:54:33.085823+0000","last_became_peered":"2023-12-01T01:54:33.085823+0000","last_unstale":"2023-12-01T01:54:34.070225+0000","last_undegraded":"2023-12-01T01:54:34.070225+0000","last_fullsized":"2023-12-01T01:54:34.070225+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:04.075901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.320423+0000","last_change":"2023-12-01T01:54:33.097187+0000","last_active":"2023-12-01T01:54:34.320423+0000","last_peered":"2023-12-01T01:54:34.320423+0000","last_clean":"2023-12-01T01:54:34.320423+0000","last_became_active":"2023-12-01T01:54:33.096904+0000","last_became_peered":"2023-12-01T01:54:33.096904+0000","last_unstale":"2023-12-01T01:54:34.320423+0000","last_undegraded":"2023-12-01T01:54:34.320423+0000","last_fullsized":"2023-12-01T01:54:34.320423+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:13.646456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.069978+0000","last_change":"2023-12-01T01:54:33.076032+0000","last_active":"2023-12-01T01:54:34.069978+0000","last_peered":"2023-12-01T01:54:34.069978+0000","last_clean":"2023-12-01T01:54:34.069978+0000","last_became_active":"2023-12-01T01:54:33.075760+0000","last_became_peered":"2023-12-01T01:54:33.075760+0000","last_unstale":"2023-12-01T01:54:34.069978+0000","last_undegraded":"2023-12-01T01:54:34.069978+0000","last_fullsized":"2023-12-01T01:54:34.069978+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:01:48.381427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.280911+0000","last_change":"2023-12-01T01:54:33.095739+0000","last_active":"2023-12-01T01:54:34.280911+0000","last_peered":"2023-12-01T01:54:34.280911+0000","last_clean":"2023-12-01T01:54:34.280911+0000","last_became_active":"2023-12-01T01:54:33.095476+0000","last_became_peered":"2023-12-01T01:54:33.095476+0000","last_unstale":"2023-12-01T01:54:34.280911+0000","last_undegraded":"2023-12-01T01:54:34.280911+0000","last_fullsized":"2023-12-01T01:54:34.280911+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:10:20.860563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.324794+0000","last_change":"2023-12-01T01:54:33.082241+0000","last_active":"2023-12-01T01:54:34.324794+0000","last_peered":"2023-12-01T01:54:34.324794+0000","last_clean":"2023-12-01T01:54:34.324794+0000","last_became_active":"2023-12-01T01:54:33.081415+0000","last_became_peered":"2023-12-01T01:54:33.081415+0000","last_unstale":"2023-12-01T01:54:34.324794+0000","last_undegraded":"2023-12-01T01:54:34.324794+0000","last_fullsized":"2023-12-01T01:54:34.324794+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:10:20.860563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.322328+0000","last_change":"2023-12-01T01:54:33.097009+0000","last_active":"2023-12-01T01:54:34.322328+0000","last_peered":"2023-12-01T01:54:34.322328+0000","last_clean":"2023-12-01T01:54:34.322328+0000","last_became_active":"2023-12-01T01:54:33.095114+0000","last_became_peered":"2023-12-01T01:54:33.095114+0000","last_unstale":"2023-12-01T01:54:34.322328+0000","last_undegraded":"2023-12-01T01:54:34.322328+0000","last_fullsized":"2023-12-01T01:54:34.322328+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:33:48.664360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.323804+0000","last_change":"2023-12-01T01:54:33.089292+0000","last_active":"2023-12-01T01:54:34.323804+0000","last_peered":"2023-12-01T01:54:34.323804+0000","last_clean":"2023-12-01T01:54:34.323804+0000","last_became_active":"2023-12-01T01:54:33.089040+0000","last_became_peered":"2023-12-01T01:54:33.089040+0000","last_unstale":"2023-12-01T01:54:34.323804+0000","last_undegraded":"2023-12-01T01:54:34.323804+0000","last_fullsized":"2023-12-01T01:54:34.323804+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:05.440162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.069446+0000","last_change":"2023-12-01T01:54:33.087626+0000","last_active":"2023-12-01T01:54:34.069446+0000","last_peered":"2023-12-01T01:54:34.069446+0000","last_clean":"2023-12-01T01:54:34.069446+0000","last_became_active":"2023-12-01T01:54:33.087384+0000","last_became_peered":"2023-12-01T01:54:33.087384+0000","last_unstale":"2023-12-01T01:54:34.069446+0000","last_undegraded":"2023-12-01T01:54:34.069446+0000","last_fullsized":"2023-12-01T01:54:34.069446+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:06:50.439904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.323784+0000","last_change":"2023-12-01T01:54:33.083547+0000","last_active":"2023-12-01T01:54:34.323784+0000","last_peered":"2023-12-01T01:54:34.323784+0000","last_clean":"2023-12-01T01:54:34.323784+0000","last_became_active":"2023-12-01T01:54:33.082810+0000","last_became_peered":"2023-12-01T01:54:33.082810+0000","last_unstale":"2023-12-01T01:54:34.323784+0000","last_undegraded":"2023-12-01T01:54:34.323784+0000","last_fullsized":"2023-12-01T01:54:34.323784+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:39:23.369889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.322535+0000","last_change":"2023-12-01T01:54:33.097154+0000","last_active":"2023-12-01T01:54:34.322535+0000","last_peered":"2023-12-01T01:54:34.322535+0000","last_clean":"2023-12-01T01:54:34.322535+0000","last_became_active":"2023-12-01T01:54:33.096457+0000","last_became_peered":"2023-12-01T01:54:33.096457+0000","last_unstale":"2023-12-01T01:54:34.322535+0000","last_undegraded":"2023-12-01T01:54:34.322535+0000","last_fullsized":"2023-12-01T01:54:34.322535+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:36:21.663477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.322855+0000","last_change":"2023-12-01T01:54:33.095393+0000","last_active":"2023-12-01T01:54:34.322855+0000","last_peered":"2023-12-01T01:54:34.322855+0000","last_clean":"2023-12-01T01:54:34.322855+0000","last_became_active":"2023-12-01T01:54:33.094191+0000","last_became_peered":"2023-12-01T01:54:33.094191+0000","last_unstale":"2023-12-01T01:54:34.322855+0000","last_undegraded":"2023-12-01T01:54:34.322855+0000","last_fullsized":"2023-12-01T01:54:34.322855+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:11.025147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.281044+0000","last_change":"2023-12-01T01:54:33.075371+0000","last_active":"2023-12-01T01:54:34.281044+0000","last_peered":"2023-12-01T01:54:34.281044+0000","last_clean":"2023-12-01T01:54:34.281044+0000","last_became_active":"2023-12-01T01:54:33.074012+0000","last_became_peered":"2023-12-01T01:54:33.074012+0000","last_unstale":"2023-12-01T01:54:34.281044+0000","last_undegraded":"2023-12-01T01:54:34.281044+0000","last_fullsized":"2023-12-01T01:54:34.281044+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:58.651325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.075799+0000","last_change":"2023-12-01T01:54:33.077951+0000","last_active":"2023-12-01T01:54:34.075799+0000","last_peered":"2023-12-01T01:54:34.075799+0000","last_clean":"2023-12-01T01:54:34.075799+0000","last_became_active":"2023-12-01T01:54:33.077607+0000","last_became_peered":"2023-12-01T01:54:33.077607+0000","last_unstale":"2023-12-01T01:54:34.075799+0000","last_undegraded":"2023-12-01T01:54:34.075799+0000","last_fullsized":"2023-12-01T01:54:34.075799+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:18.790802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.074732+0000","last_change":"2023-12-01T01:54:33.079670+0000","last_active":"2023-12-01T01:54:34.074732+0000","last_peered":"2023-12-01T01:54:34.074732+0000","last_clean":"2023-12-01T01:54:34.074732+0000","last_became_active":"2023-12-01T01:54:33.079380+0000","last_became_peered":"2023-12-01T01:54:33.079380+0000","last_unstale":"2023-12-01T01:54:34.074732+0000","last_undegraded":"2023-12-01T01:54:34.074732+0000","last_fullsized":"2023-12-01T01:54:34.074732+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:20.216558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.324047+0000","last_change":"2023-12-01T01:54:33.089054+0000","last_active":"2023-12-01T01:54:34.324047+0000","last_peered":"2023-12-01T01:54:34.324047+0000","last_clean":"2023-12-01T01:54:34.324047+0000","last_became_active":"2023-12-01T01:54:33.088626+0000","last_became_peered":"2023-12-01T01:54:33.088626+0000","last_unstale":"2023-12-01T01:54:34.324047+0000","last_undegraded":"2023-12-01T01:54:34.324047+0000","last_fullsized":"2023-12-01T01:54:34.324047+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:35.397693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.071968+0000","last_change":"2023-12-01T01:54:33.095076+0000","last_active":"2023-12-01T01:54:34.071968+0000","last_peered":"2023-12-01T01:54:34.071968+0000","last_clean":"2023-12-01T01:54:34.071968+0000","last_became_active":"2023-12-01T01:54:33.094842+0000","last_became_peered":"2023-12-01T01:54:33.094842+0000","last_unstale":"2023-12-01T01:54:34.071968+0000","last_undegraded":"2023-12-01T01:54:34.071968+0000","last_fullsized":"2023-12-01T01:54:34.071968+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:13.646456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.320504+0000","last_change":"2023-12-01T01:54:33.087933+0000","last_active":"2023-12-01T01:54:34.320504+0000","last_peered":"2023-12-01T01:54:34.320504+0000","last_clean":"2023-12-01T01:54:34.320504+0000","last_became_active":"2023-12-01T01:54:33.087469+0000","last_became_peered":"2023-12-01T01:54:33.087469+0000","last_unstale":"2023-12-01T01:54:34.320504+0000","last_undegraded":"2023-12-01T01:54:34.320504+0000","last_fullsized":"2023-12-01T01:54:34.320504+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:10.805523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.325208+0000","last_change":"2023-12-01T01:54:33.082903+0000","last_active":"2023-12-01T01:54:34.325208+0000","last_peered":"2023-12-01T01:54:34.325208+0000","last_clean":"2023-12-01T01:54:34.325208+0000","last_became_active":"2023-12-01T01:54:33.082636+0000","last_became_peered":"2023-12-01T01:54:33.082636+0000","last_unstale":"2023-12-01T01:54:34.325208+0000","last_undegraded":"2023-12-01T01:54:34.325208+0000","last_fullsized":"2023-12-01T01:54:34.325208+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:47:36.639317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.072761+0000","last_change":"2023-12-01T01:54:33.091271+0000","last_active":"2023-12-01T01:54:34.072761+0000","last_peered":"2023-12-01T01:54:34.072761+0000","last_clean":"2023-12-01T01:54:34.072761+0000","last_became_active":"2023-12-01T01:54:33.091002+0000","last_became_peered":"2023-12-01T01:54:33.091002+0000","last_unstale":"2023-12-01T01:54:34.072761+0000","last_undegraded":"2023-12-01T01:54:34.072761+0000","last_fullsized":"2023-12-01T01:54:34.072761+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:10.805523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.072454+0000","last_change":"2023-12-01T01:54:33.083477+0000","last_active":"2023-12-01T01:54:34.072454+0000","last_peered":"2023-12-01T01:54:34.072454+0000","last_clean":"2023-12-01T01:54:34.072454+0000","last_became_active":"2023-12-01T01:54:33.082923+0000","last_became_peered":"2023-12-01T01:54:33.082923+0000","last_unstale":"2023-12-01T01:54:34.072454+0000","last_undegraded":"2023-12-01T01:54:34.072454+0000","last_fullsized":"2023-12-01T01:54:34.072454+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:29:37.656439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.324548+0000","last_change":"2023-12-01T01:54:33.074866+0000","last_active":"2023-12-01T01:54:34.324548+0000","last_peered":"2023-12-01T01:54:34.324548+0000","last_clean":"2023-12-01T01:54:34.324548+0000","last_became_active":"2023-12-01T01:54:33.074519+0000","last_became_peered":"2023-12-01T01:54:33.074519+0000","last_unstale":"2023-12-01T01:54:34.324548+0000","last_undegraded":"2023-12-01T01:54:34.324548+0000","last_fullsized":"2023-12-01T01:54:34.324548+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:39:48.891522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.325155+0000","last_change":"2023-12-01T01:54:33.089339+0000","last_active":"2023-12-01T01:54:34.325155+0000","last_peered":"2023-12-01T01:54:34.325155+0000","last_clean":"2023-12-01T01:54:34.325155+0000","last_became_active":"2023-12-01T01:54:33.089016+0000","last_became_peered":"2023-12-01T01:54:33.089016+0000","last_unstale":"2023-12-01T01:54:34.325155+0000","last_undegraded":"2023-12-01T01:54:34.325155+0000","last_fullsized":"2023-12-01T01:54:34.325155+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:05.011546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.074772+0000","last_change":"2023-12-01T01:54:33.078595+0000","last_active":"2023-12-01T01:54:34.074772+0000","last_peered":"2023-12-01T01:54:34.074772+0000","last_clean":"2023-12-01T01:54:34.074772+0000","last_became_active":"2023-12-01T01:54:33.078267+0000","last_became_peered":"2023-12-01T01:54:33.078267+0000","last_unstale":"2023-12-01T01:54:34.074772+0000","last_undegraded":"2023-12-01T01:54:34.074772+0000","last_fullsized":"2023-12-01T01:54:34.074772+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:13.646456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.321765+0000","last_change":"2023-12-01T01:54:33.089868+0000","last_active":"2023-12-01T01:54:34.321765+0000","last_peered":"2023-12-01T01:54:34.321765+0000","last_clean":"2023-12-01T01:54:34.321765+0000","last_became_active":"2023-12-01T01:54:33.088800+0000","last_became_peered":"2023-12-01T01:54:33.088800+0000","last_unstale":"2023-12-01T01:54:34.321765+0000","last_undegraded":"2023-12-01T01:54:34.321765+0000","last_fullsized":"2023-12-01T01:54:34.321765+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:49:36.963623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.323910+0000","last_change":"2023-12-01T01:54:33.082887+0000","last_active":"2023-12-01T01:54:34.323910+0000","last_peered":"2023-12-01T01:54:34.323910+0000","last_clean":"2023-12-01T01:54:34.323910+0000","last_became_active":"2023-12-01T01:54:33.082589+0000","last_became_peered":"2023-12-01T01:54:33.082589+0000","last_unstale":"2023-12-01T01:54:34.323910+0000","last_undegraded":"2023-12-01T01:54:34.323910+0000","last_fullsized":"2023-12-01T01:54:34.323910+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:36:49.002580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"124'4","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.086428+0000","last_change":"2023-12-01T01:54:32.599896+0000","last_active":"2023-12-01T01:54:33.086428+0000","last_peered":"2023-12-01T01:54:33.086428+0000","last_clean":"2023-12-01T01:54:33.086428+0000","last_became_active":"2023-12-01T01:54:32.599595+0000","last_became_peered":"2023-12-01T01:54:32.599595+0000","last_unstale":"2023-12-01T01:54:33.086428+0000","last_undegraded":"2023-12-01T01:54:33.086428+0000","last_fullsized":"2023-12-01T01:54:33.086428+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:27:48.189318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.325476+0000","last_change":"2023-12-01T01:54:33.075587+0000","last_active":"2023-12-01T01:54:34.325476+0000","last_peered":"2023-12-01T01:54:34.325476+0000","last_clean":"2023-12-01T01:54:34.325476+0000","last_became_active":"2023-12-01T01:54:33.075100+0000","last_became_peered":"2023-12-01T01:54:33.075100+0000","last_unstale":"2023-12-01T01:54:34.325476+0000","last_undegraded":"2023-12-01T01:54:34.325476+0000","last_fullsized":"2023-12-01T01:54:34.325476+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:41:02.386430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.093904+0000","last_change":"2023-12-01T01:54:32.613559+0000","last_active":"2023-12-01T01:54:33.093904+0000","last_peered":"2023-12-01T01:54:33.093904+0000","last_clean":"2023-12-01T01:54:33.093904+0000","last_became_active":"2023-12-01T01:54:32.613265+0000","last_became_peered":"2023-12-01T01:54:32.613265+0000","last_unstale":"2023-12-01T01:54:33.093904+0000","last_undegraded":"2023-12-01T01:54:33.093904+0000","last_fullsized":"2023-12-01T01:54:33.093904+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:24.464526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.325697+0000","last_change":"2023-12-01T01:54:33.085333+0000","last_active":"2023-12-01T01:54:34.325697+0000","last_peered":"2023-12-01T01:54:34.325697+0000","last_clean":"2023-12-01T01:54:34.325697+0000","last_became_active":"2023-12-01T01:54:33.084274+0000","last_became_peered":"2023-12-01T01:54:33.084274+0000","last_unstale":"2023-12-01T01:54:34.325697+0000","last_undegraded":"2023-12-01T01:54:34.325697+0000","last_fullsized":"2023-12-01T01:54:34.325697+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:29:37.656439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.095663+0000","last_change":"2023-12-01T01:54:32.608282+0000","last_active":"2023-12-01T01:54:33.095663+0000","last_peered":"2023-12-01T01:54:33.095663+0000","last_clean":"2023-12-01T01:54:33.095663+0000","last_became_active":"2023-12-01T01:54:32.607567+0000","last_became_peered":"2023-12-01T01:54:32.607567+0000","last_unstale":"2023-12-01T01:54:33.095663+0000","last_undegraded":"2023-12-01T01:54:33.095663+0000","last_fullsized":"2023-12-01T01:54:33.095663+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:37:57.646717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.320187+0000","last_change":"2023-12-01T01:54:33.067132+0000","last_active":"2023-12-01T01:54:34.320187+0000","last_peered":"2023-12-01T01:54:34.320187+0000","last_clean":"2023-12-01T01:54:34.320187+0000","last_became_active":"2023-12-01T01:54:33.066351+0000","last_became_peered":"2023-12-01T01:54:33.066351+0000","last_unstale":"2023-12-01T01:54:34.320187+0000","last_undegraded":"2023-12-01T01:54:34.320187+0000","last_fullsized":"2023-12-01T01:54:34.320187+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:18.790802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.098963+0000","last_change":"2023-12-01T01:54:32.610725+0000","last_active":"2023-12-01T01:54:33.098963+0000","last_peered":"2023-12-01T01:54:33.098963+0000","last_clean":"2023-12-01T01:54:33.098963+0000","last_became_active":"2023-12-01T01:54:32.610424+0000","last_became_peered":"2023-12-01T01:54:32.610424+0000","last_unstale":"2023-12-01T01:54:33.098963+0000","last_undegraded":"2023-12-01T01:54:33.098963+0000","last_fullsized":"2023-12-01T01:54:33.098963+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:10.478463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.280792+0000","last_change":"2023-12-01T01:54:33.070392+0000","last_active":"2023-12-01T01:54:34.280792+0000","last_peered":"2023-12-01T01:54:34.280792+0000","last_clean":"2023-12-01T01:54:34.280792+0000","last_became_active":"2023-12-01T01:54:33.068985+0000","last_became_peered":"2023-12-01T01:54:33.068985+0000","last_unstale":"2023-12-01T01:54:34.280792+0000","last_undegraded":"2023-12-01T01:54:34.280792+0000","last_fullsized":"2023-12-01T01:54:34.280792+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:45:40.644721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.096946+0000","last_change":"2023-12-01T01:54:32.603266+0000","last_active":"2023-12-01T01:54:33.096946+0000","last_peered":"2023-12-01T01:54:33.096946+0000","last_clean":"2023-12-01T01:54:33.096946+0000","last_became_active":"2023-12-01T01:54:32.602987+0000","last_became_peered":"2023-12-01T01:54:32.602987+0000","last_unstale":"2023-12-01T01:54:33.096946+0000","last_undegraded":"2023-12-01T01:54:33.096946+0000","last_fullsized":"2023-12-01T01:54:33.096946+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:57:04.006938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.069883+0000","last_change":"2023-12-01T01:54:33.074859+0000","last_active":"2023-12-01T01:54:34.069883+0000","last_peered":"2023-12-01T01:54:34.069883+0000","last_clean":"2023-12-01T01:54:34.069883+0000","last_became_active":"2023-12-01T01:54:33.074633+0000","last_became_peered":"2023-12-01T01:54:33.074633+0000","last_unstale":"2023-12-01T01:54:34.069883+0000","last_undegraded":"2023-12-01T01:54:34.069883+0000","last_fullsized":"2023-12-01T01:54:34.069883+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:41.032779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.082576+0000","last_change":"2023-12-01T01:54:32.592269+0000","last_active":"2023-12-01T01:54:33.082576+0000","last_peered":"2023-12-01T01:54:33.082576+0000","last_clean":"2023-12-01T01:54:33.082576+0000","last_became_active":"2023-12-01T01:54:32.591945+0000","last_became_peered":"2023-12-01T01:54:32.591945+0000","last_unstale":"2023-12-01T01:54:33.082576+0000","last_undegraded":"2023-12-01T01:54:33.082576+0000","last_fullsized":"2023-12-01T01:54:33.082576+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:03.501711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.325142+0000","last_change":"2023-12-01T01:54:33.087453+0000","last_active":"2023-12-01T01:54:34.325142+0000","last_peered":"2023-12-01T01:54:34.325142+0000","last_clean":"2023-12-01T01:54:34.325142+0000","last_became_active":"2023-12-01T01:54:33.086795+0000","last_became_peered":"2023-12-01T01:54:33.086795+0000","last_unstale":"2023-12-01T01:54:34.325142+0000","last_undegraded":"2023-12-01T01:54:34.325142+0000","last_fullsized":"2023-12-01T01:54:34.325142+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:07:54.815046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.063469+0000","last_change":"2023-12-01T01:54:32.601996+0000","last_active":"2023-12-01T01:54:33.063469+0000","last_peered":"2023-12-01T01:54:33.063469+0000","last_clean":"2023-12-01T01:54:33.063469+0000","last_became_active":"2023-12-01T01:54:32.601667+0000","last_became_peered":"2023-12-01T01:54:32.601667+0000","last_unstale":"2023-12-01T01:54:33.063469+0000","last_undegraded":"2023-12-01T01:54:33.063469+0000","last_fullsized":"2023-12-01T01:54:33.063469+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:39:22.365281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.280378+0000","last_change":"2023-12-01T01:54:33.069961+0000","last_active":"2023-12-01T01:54:34.280378+0000","last_peered":"2023-12-01T01:54:34.280378+0000","last_clean":"2023-12-01T01:54:34.280378+0000","last_became_active":"2023-12-01T01:54:33.069692+0000","last_became_peered":"2023-12-01T01:54:33.069692+0000","last_unstale":"2023-12-01T01:54:34.280378+0000","last_undegraded":"2023-12-01T01:54:34.280378+0000","last_fullsized":"2023-12-01T01:54:34.280378+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:45:45.827967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.072493+0000","last_change":"2023-12-01T01:54:32.609579+0000","last_active":"2023-12-01T01:54:33.072493+0000","last_peered":"2023-12-01T01:54:33.072493+0000","last_clean":"2023-12-01T01:54:33.072493+0000","last_became_active":"2023-12-01T01:54:32.609297+0000","last_became_peered":"2023-12-01T01:54:32.609297+0000","last_unstale":"2023-12-01T01:54:33.072493+0000","last_undegraded":"2023-12-01T01:54:33.072493+0000","last_fullsized":"2023-12-01T01:54:33.072493+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:07:39.620156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.322776+0000","last_change":"2023-12-01T01:54:33.099791+0000","last_active":"2023-12-01T01:54:34.322776+0000","last_peered":"2023-12-01T01:54:34.322776+0000","last_clean":"2023-12-01T01:54:34.322776+0000","last_became_active":"2023-12-01T01:54:33.099365+0000","last_became_peered":"2023-12-01T01:54:33.099365+0000","last_unstale":"2023-12-01T01:54:34.322776+0000","last_undegraded":"2023-12-01T01:54:34.322776+0000","last_fullsized":"2023-12-01T01:54:34.322776+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:48:09.518199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.062085+0000","last_change":"2023-12-01T01:54:32.596533+0000","last_active":"2023-12-01T01:54:33.062085+0000","last_peered":"2023-12-01T01:54:33.062085+0000","last_clean":"2023-12-01T01:54:33.062085+0000","last_became_active":"2023-12-01T01:54:32.595240+0000","last_became_peered":"2023-12-01T01:54:32.595240+0000","last_unstale":"2023-12-01T01:54:33.062085+0000","last_undegraded":"2023-12-01T01:54:33.062085+0000","last_fullsized":"2023-12-01T01:54:33.062085+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:45:39.640112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.072845+0000","last_change":"2023-12-01T01:54:33.079961+0000","last_active":"2023-12-01T01:54:34.072845+0000","last_peered":"2023-12-01T01:54:34.072845+0000","last_clean":"2023-12-01T01:54:34.072845+0000","last_became_active":"2023-12-01T01:54:33.079632+0000","last_became_peered":"2023-12-01T01:54:33.079632+0000","last_unstale":"2023-12-01T01:54:34.072845+0000","last_undegraded":"2023-12-01T01:54:34.072845+0000","last_fullsized":"2023-12-01T01:54:34.072845+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:39:33.319706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.086524+0000","last_change":"2023-12-01T01:54:32.610079+0000","last_active":"2023-12-01T01:54:33.086524+0000","last_peered":"2023-12-01T01:54:33.086524+0000","last_clean":"2023-12-01T01:54:33.086524+0000","last_became_active":"2023-12-01T01:54:32.609800+0000","last_became_peered":"2023-12-01T01:54:32.609800+0000","last_unstale":"2023-12-01T01:54:33.086524+0000","last_undegraded":"2023-12-01T01:54:33.086524+0000","last_fullsized":"2023-12-01T01:54:33.086524+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:09.456228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.069768+0000","last_change":"2023-12-01T01:54:33.081653+0000","last_active":"2023-12-01T01:54:34.069768+0000","last_peered":"2023-12-01T01:54:34.069768+0000","last_clean":"2023-12-01T01:54:34.069768+0000","last_became_active":"2023-12-01T01:54:33.081377+0000","last_became_peered":"2023-12-01T01:54:33.081377+0000","last_unstale":"2023-12-01T01:54:34.069768+0000","last_undegraded":"2023-12-01T01:54:34.069768+0000","last_fullsized":"2023-12-01T01:54:34.069768+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:16:31.197019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"124'3","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.080682+0000","last_change":"2023-12-01T01:54:32.600662+0000","last_active":"2023-12-01T01:54:33.080682+0000","last_peered":"2023-12-01T01:54:33.080682+0000","last_clean":"2023-12-01T01:54:33.080682+0000","last_became_active":"2023-12-01T01:54:32.600408+0000","last_became_peered":"2023-12-01T01:54:32.600408+0000","last_unstale":"2023-12-01T01:54:33.080682+0000","last_undegraded":"2023-12-01T01:54:33.080682+0000","last_fullsized":"2023-12-01T01:54:33.080682+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:36:47.997971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.072702+0000","last_change":"2023-12-01T01:54:33.088419+0000","last_active":"2023-12-01T01:54:34.072702+0000","last_peered":"2023-12-01T01:54:34.072702+0000","last_clean":"2023-12-01T01:54:34.072702+0000","last_became_active":"2023-12-01T01:54:33.088110+0000","last_became_peered":"2023-12-01T01:54:33.088110+0000","last_unstale":"2023-12-01T01:54:34.072702+0000","last_undegraded":"2023-12-01T01:54:34.072702+0000","last_fullsized":"2023-12-01T01:54:34.072702+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:25.411723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"124'6","reported_seq":17,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.099780+0000","last_change":"2023-12-01T01:54:32.609480+0000","last_active":"2023-12-01T01:54:33.099780+0000","last_peered":"2023-12-01T01:54:33.099780+0000","last_clean":"2023-12-01T01:54:33.099780+0000","last_became_active":"2023-12-01T01:54:32.608986+0000","last_became_peered":"2023-12-01T01:54:32.608986+0000","last_unstale":"2023-12-01T01:54:33.099780+0000","last_undegraded":"2023-12-01T01:54:33.099780+0000","last_fullsized":"2023-12-01T01:54:33.099780+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:40:30.130936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.322615+0000","last_change":"2023-12-01T01:54:33.094212+0000","last_active":"2023-12-01T01:54:34.322615+0000","last_peered":"2023-12-01T01:54:34.322615+0000","last_clean":"2023-12-01T01:54:34.322615+0000","last_became_active":"2023-12-01T01:54:33.093161+0000","last_became_peered":"2023-12-01T01:54:33.093161+0000","last_unstale":"2023-12-01T01:54:34.322615+0000","last_undegraded":"2023-12-01T01:54:34.322615+0000","last_fullsized":"2023-12-01T01:54:34.322615+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:10.805523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.095940+0000","last_change":"2023-12-01T01:54:32.610239+0000","last_active":"2023-12-01T01:54:33.095940+0000","last_peered":"2023-12-01T01:54:33.095940+0000","last_clean":"2023-12-01T01:54:33.095940+0000","last_became_active":"2023-12-01T01:54:32.609930+0000","last_became_peered":"2023-12-01T01:54:32.609930+0000","last_unstale":"2023-12-01T01:54:33.095940+0000","last_undegraded":"2023-12-01T01:54:33.095940+0000","last_fullsized":"2023-12-01T01:54:33.095940+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:34.393085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.075822+0000","last_change":"2023-12-01T01:54:33.093853+0000","last_active":"2023-12-01T01:54:34.075822+0000","last_peered":"2023-12-01T01:54:34.075822+0000","last_clean":"2023-12-01T01:54:34.075822+0000","last_became_active":"2023-12-01T01:54:33.093579+0000","last_became_peered":"2023-12-01T01:54:33.093579+0000","last_unstale":"2023-12-01T01:54:34.075822+0000","last_undegraded":"2023-12-01T01:54:34.075822+0000","last_fullsized":"2023-12-01T01:54:34.075822+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:10.805523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.086117+0000","last_change":"2023-12-01T01:54:32.605824+0000","last_active":"2023-12-01T01:54:33.086117+0000","last_peered":"2023-12-01T01:54:33.086117+0000","last_clean":"2023-12-01T01:54:33.086117+0000","last_became_active":"2023-12-01T01:54:32.603788+0000","last_became_peered":"2023-12-01T01:54:32.603788+0000","last_unstale":"2023-12-01T01:54:33.086117+0000","last_undegraded":"2023-12-01T01:54:33.086117+0000","last_fullsized":"2023-12-01T01:54:33.086117+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:15:41.132592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.069697+0000","last_change":"2023-12-01T01:54:33.074373+0000","last_active":"2023-12-01T01:54:34.069697+0000","last_peered":"2023-12-01T01:54:34.069697+0000","last_clean":"2023-12-01T01:54:34.069697+0000","last_became_active":"2023-12-01T01:54:33.074128+0000","last_became_peered":"2023-12-01T01:54:33.074128+0000","last_unstale":"2023-12-01T01:54:34.069697+0000","last_undegraded":"2023-12-01T01:54:34.069697+0000","last_fullsized":"2023-12-01T01:54:34.069697+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:32:24.394396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.060102+0000","last_change":"2023-12-01T01:54:32.592599+0000","last_active":"2023-12-01T01:54:33.060102+0000","last_peered":"2023-12-01T01:54:33.060102+0000","last_clean":"2023-12-01T01:54:33.060102+0000","last_became_active":"2023-12-01T01:54:32.592268+0000","last_became_peered":"2023-12-01T01:54:32.592268+0000","last_unstale":"2023-12-01T01:54:33.060102+0000","last_undegraded":"2023-12-01T01:54:33.060102+0000","last_fullsized":"2023-12-01T01:54:33.060102+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:31:14.275675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.074728+0000","last_change":"2023-12-01T01:54:33.082374+0000","last_active":"2023-12-01T01:54:34.074728+0000","last_peered":"2023-12-01T01:54:34.074728+0000","last_clean":"2023-12-01T01:54:34.074728+0000","last_became_active":"2023-12-01T01:54:33.081894+0000","last_became_peered":"2023-12-01T01:54:33.081894+0000","last_unstale":"2023-12-01T01:54:34.074728+0000","last_undegraded":"2023-12-01T01:54:34.074728+0000","last_fullsized":"2023-12-01T01:54:34.074728+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:33:38.813748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"124'8","reported_seq":19,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.097994+0000","last_change":"2023-12-01T01:54:32.606806+0000","last_active":"2023-12-01T01:54:33.097994+0000","last_peered":"2023-12-01T01:54:33.097994+0000","last_clean":"2023-12-01T01:54:33.097994+0000","last_became_active":"2023-12-01T01:54:32.606519+0000","last_became_peered":"2023-12-01T01:54:32.606519+0000","last_unstale":"2023-12-01T01:54:33.097994+0000","last_undegraded":"2023-12-01T01:54:33.097994+0000","last_fullsized":"2023-12-01T01:54:33.097994+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:41:01.381821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.072974+0000","last_change":"2023-12-01T01:54:33.084352+0000","last_active":"2023-12-01T01:54:34.072974+0000","last_peered":"2023-12-01T01:54:34.072974+0000","last_clean":"2023-12-01T01:54:34.072974+0000","last_became_active":"2023-12-01T01:54:33.084091+0000","last_became_peered":"2023-12-01T01:54:33.084091+0000","last_unstale":"2023-12-01T01:54:34.072974+0000","last_undegraded":"2023-12-01T01:54:34.072974+0000","last_fullsized":"2023-12-01T01:54:34.072974+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:21.023813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.094145+0000","last_change":"2023-12-01T01:54:32.612265+0000","last_active":"2023-12-01T01:54:33.094145+0000","last_peered":"2023-12-01T01:54:33.094145+0000","last_clean":"2023-12-01T01:54:33.094145+0000","last_became_active":"2023-12-01T01:54:32.611791+0000","last_became_peered":"2023-12-01T01:54:32.611791+0000","last_unstale":"2023-12-01T01:54:33.094145+0000","last_undegraded":"2023-12-01T01:54:33.094145+0000","last_fullsized":"2023-12-01T01:54:33.094145+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:06.389420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.072650+0000","last_change":"2023-12-01T01:54:33.084274+0000","last_active":"2023-12-01T01:54:34.072650+0000","last_peered":"2023-12-01T01:54:34.072650+0000","last_clean":"2023-12-01T01:54:34.072650+0000","last_became_active":"2023-12-01T01:54:33.083907+0000","last_became_peered":"2023-12-01T01:54:33.083907+0000","last_unstale":"2023-12-01T01:54:34.072650+0000","last_undegraded":"2023-12-01T01:54:34.072650+0000","last_fullsized":"2023-12-01T01:54:34.072650+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:51:09.772313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.082609+0000","last_change":"2023-12-01T01:54:32.595531+0000","last_active":"2023-12-01T01:54:33.082609+0000","last_peered":"2023-12-01T01:54:33.082609+0000","last_clean":"2023-12-01T01:54:33.082609+0000","last_became_active":"2023-12-01T01:54:32.595199+0000","last_became_peered":"2023-12-01T01:54:32.595199+0000","last_unstale":"2023-12-01T01:54:33.082609+0000","last_undegraded":"2023-12-01T01:54:33.082609+0000","last_fullsized":"2023-12-01T01:54:33.082609+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:48:53.395467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.281416+0000","last_change":"2023-12-01T01:54:33.075257+0000","last_active":"2023-12-01T01:54:34.281416+0000","last_peered":"2023-12-01T01:54:34.281416+0000","last_clean":"2023-12-01T01:54:34.281416+0000","last_became_active":"2023-12-01T01:54:33.074523+0000","last_became_peered":"2023-12-01T01:54:33.074523+0000","last_unstale":"2023-12-01T01:54:34.281416+0000","last_undegraded":"2023-12-01T01:54:34.281416+0000","last_fullsized":"2023-12-01T01:54:34.281416+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:07:30.302554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.080772+0000","last_change":"2023-12-01T01:54:32.611404+0000","last_active":"2023-12-01T01:54:33.080772+0000","last_peered":"2023-12-01T01:54:33.080772+0000","last_clean":"2023-12-01T01:54:33.080772+0000","last_became_active":"2023-12-01T01:54:32.608805+0000","last_became_peered":"2023-12-01T01:54:32.608805+0000","last_unstale":"2023-12-01T01:54:33.080772+0000","last_undegraded":"2023-12-01T01:54:33.080772+0000","last_fullsized":"2023-12-01T01:54:33.080772+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:39:32.315098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.324684+0000","last_change":"2023-12-01T01:54:33.086569+0000","last_active":"2023-12-01T01:54:34.324684+0000","last_peered":"2023-12-01T01:54:34.324684+0000","last_clean":"2023-12-01T01:54:34.324684+0000","last_became_active":"2023-12-01T01:54:33.086258+0000","last_became_peered":"2023-12-01T01:54:33.086258+0000","last_unstale":"2023-12-01T01:54:34.324684+0000","last_undegraded":"2023-12-01T01:54:34.324684+0000","last_fullsized":"2023-12-01T01:54:34.324684+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:48:09.518199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.099116+0000","last_change":"2023-12-01T01:54:32.610921+0000","last_active":"2023-12-01T01:54:33.099116+0000","last_peered":"2023-12-01T01:54:33.099116+0000","last_clean":"2023-12-01T01:54:33.099116+0000","last_became_active":"2023-12-01T01:54:32.610583+0000","last_became_peered":"2023-12-01T01:54:32.610583+0000","last_unstale":"2023-12-01T01:54:33.099116+0000","last_undegraded":"2023-12-01T01:54:33.099116+0000","last_fullsized":"2023-12-01T01:54:33.099116+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:16.042920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.323237+0000","last_change":"2023-12-01T01:54:33.092967+0000","last_active":"2023-12-01T01:54:34.323237+0000","last_peered":"2023-12-01T01:54:34.323237+0000","last_clean":"2023-12-01T01:54:34.323237+0000","last_became_active":"2023-12-01T01:54:33.092599+0000","last_became_peered":"2023-12-01T01:54:33.092599+0000","last_unstale":"2023-12-01T01:54:34.323237+0000","last_undegraded":"2023-12-01T01:54:34.323237+0000","last_fullsized":"2023-12-01T01:54:34.323237+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:48:54.400075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.075980+0000","last_change":"2023-12-01T01:54:32.594780+0000","last_active":"2023-12-01T01:54:33.075980+0000","last_peered":"2023-12-01T01:54:33.075980+0000","last_clean":"2023-12-01T01:54:33.075980+0000","last_became_active":"2023-12-01T01:54:32.594339+0000","last_became_peered":"2023-12-01T01:54:32.594339+0000","last_unstale":"2023-12-01T01:54:33.075980+0000","last_undegraded":"2023-12-01T01:54:33.075980+0000","last_fullsized":"2023-12-01T01:54:33.075980+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:15:41.132592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.322785+0000","last_change":"2023-12-01T01:54:33.087854+0000","last_active":"2023-12-01T01:54:34.322785+0000","last_peered":"2023-12-01T01:54:34.322785+0000","last_clean":"2023-12-01T01:54:34.322785+0000","last_became_active":"2023-12-01T01:54:33.087524+0000","last_became_peered":"2023-12-01T01:54:33.087524+0000","last_unstale":"2023-12-01T01:54:34.322785+0000","last_undegraded":"2023-12-01T01:54:34.322785+0000","last_fullsized":"2023-12-01T01:54:34.322785+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:25.411723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.063933+0000","last_change":"2023-12-01T01:54:32.601967+0000","last_active":"2023-12-01T01:54:33.063933+0000","last_peered":"2023-12-01T01:54:33.063933+0000","last_clean":"2023-12-01T01:54:33.063933+0000","last_became_active":"2023-12-01T01:54:32.601703+0000","last_became_peered":"2023-12-01T01:54:32.601703+0000","last_unstale":"2023-12-01T01:54:33.063933+0000","last_undegraded":"2023-12-01T01:54:33.063933+0000","last_fullsized":"2023-12-01T01:54:33.063933+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:13:26.203448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.072190+0000","last_change":"2023-12-01T01:54:33.083340+0000","last_active":"2023-12-01T01:54:34.072190+0000","last_peered":"2023-12-01T01:54:34.072190+0000","last_clean":"2023-12-01T01:54:34.072190+0000","last_became_active":"2023-12-01T01:54:33.083088+0000","last_became_peered":"2023-12-01T01:54:33.083088+0000","last_unstale":"2023-12-01T01:54:34.072190+0000","last_undegraded":"2023-12-01T01:54:34.072190+0000","last_fullsized":"2023-12-01T01:54:34.072190+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:07:40.624764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.066745+0000","last_change":"2023-12-01T01:54:32.601251+0000","last_active":"2023-12-01T01:54:33.066745+0000","last_peered":"2023-12-01T01:54:33.066745+0000","last_clean":"2023-12-01T01:54:33.066745+0000","last_became_active":"2023-12-01T01:54:32.600980+0000","last_became_peered":"2023-12-01T01:54:32.600980+0000","last_unstale":"2023-12-01T01:54:33.066745+0000","last_undegraded":"2023-12-01T01:54:33.066745+0000","last_fullsized":"2023-12-01T01:54:33.066745+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:10:19.855954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.324190+0000","last_change":"2023-12-01T01:54:33.093739+0000","last_active":"2023-12-01T01:54:34.324190+0000","last_peered":"2023-12-01T01:54:34.324190+0000","last_clean":"2023-12-01T01:54:34.324190+0000","last_became_active":"2023-12-01T01:54:33.093135+0000","last_became_peered":"2023-12-01T01:54:33.093135+0000","last_unstale":"2023-12-01T01:54:34.324190+0000","last_undegraded":"2023-12-01T01:54:34.324190+0000","last_fullsized":"2023-12-01T01:54:34.324190+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:19:26.454653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"124'2","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.084418+0000","last_change":"2023-12-01T01:54:32.594504+0000","last_active":"2023-12-01T01:54:33.084418+0000","last_peered":"2023-12-01T01:54:33.084418+0000","last_clean":"2023-12-01T01:54:33.084418+0000","last_became_active":"2023-12-01T01:54:32.593697+0000","last_became_peered":"2023-12-01T01:54:32.593697+0000","last_unstale":"2023-12-01T01:54:33.084418+0000","last_undegraded":"2023-12-01T01:54:33.084418+0000","last_fullsized":"2023-12-01T01:54:33.084418+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:58.343037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.324039+0000","last_change":"2023-12-01T01:54:33.096535+0000","last_active":"2023-12-01T01:54:34.324039+0000","last_peered":"2023-12-01T01:54:34.324039+0000","last_clean":"2023-12-01T01:54:34.324039+0000","last_became_active":"2023-12-01T01:54:33.095775+0000","last_became_peered":"2023-12-01T01:54:33.095775+0000","last_unstale":"2023-12-01T01:54:34.324039+0000","last_undegraded":"2023-12-01T01:54:34.324039+0000","last_fullsized":"2023-12-01T01:54:34.324039+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:07:49.046812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.078982+0000","last_change":"2023-12-01T01:54:32.610571+0000","last_active":"2023-12-01T01:54:33.078982+0000","last_peered":"2023-12-01T01:54:33.078982+0000","last_clean":"2023-12-01T01:54:33.078982+0000","last_became_active":"2023-12-01T01:54:32.610250+0000","last_became_peered":"2023-12-01T01:54:32.610250+0000","last_unstale":"2023-12-01T01:54:33.078982+0000","last_undegraded":"2023-12-01T01:54:33.078982+0000","last_fullsized":"2023-12-01T01:54:33.078982+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:24.407114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.075081+0000","last_change":"2023-12-01T01:54:33.077959+0000","last_active":"2023-12-01T01:54:34.075081+0000","last_peered":"2023-12-01T01:54:34.075081+0000","last_clean":"2023-12-01T01:54:34.075081+0000","last_became_active":"2023-12-01T01:54:33.077723+0000","last_became_peered":"2023-12-01T01:54:33.077723+0000","last_unstale":"2023-12-01T01:54:34.075081+0000","last_undegraded":"2023-12-01T01:54:34.075081+0000","last_fullsized":"2023-12-01T01:54:34.075081+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:13:27.208057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.075380+0000","last_change":"2023-12-01T01:54:32.605401+0000","last_active":"2023-12-01T01:54:33.075380+0000","last_peered":"2023-12-01T01:54:33.075380+0000","last_clean":"2023-12-01T01:54:33.075380+0000","last_became_active":"2023-12-01T01:54:32.604697+0000","last_became_peered":"2023-12-01T01:54:32.604697+0000","last_unstale":"2023-12-01T01:54:33.075380+0000","last_undegraded":"2023-12-01T01:54:33.075380+0000","last_fullsized":"2023-12-01T01:54:33.075380+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:29:36.651831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.070009+0000","last_change":"2023-12-01T01:54:33.066841+0000","last_active":"2023-12-01T01:54:34.070009+0000","last_peered":"2023-12-01T01:54:34.070009+0000","last_clean":"2023-12-01T01:54:34.070009+0000","last_became_active":"2023-12-01T01:54:33.066578+0000","last_became_peered":"2023-12-01T01:54:33.066578+0000","last_unstale":"2023-12-01T01:54:34.070009+0000","last_undegraded":"2023-12-01T01:54:34.070009+0000","last_fullsized":"2023-12-01T01:54:34.070009+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:14.153995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.078645+0000","last_change":"2023-12-01T01:54:32.596549+0000","last_active":"2023-12-01T01:54:33.078645+0000","last_peered":"2023-12-01T01:54:33.078645+0000","last_clean":"2023-12-01T01:54:33.078645+0000","last_became_active":"2023-12-01T01:54:32.596130+0000","last_became_peered":"2023-12-01T01:54:32.596130+0000","last_unstale":"2023-12-01T01:54:33.078645+0000","last_undegraded":"2023-12-01T01:54:33.078645+0000","last_fullsized":"2023-12-01T01:54:33.078645+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:41:01.381821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.322286+0000","last_change":"2023-12-01T01:54:33.085541+0000","last_active":"2023-12-01T01:54:34.322286+0000","last_peered":"2023-12-01T01:54:34.322286+0000","last_clean":"2023-12-01T01:54:34.322286+0000","last_became_active":"2023-12-01T01:54:33.084949+0000","last_became_peered":"2023-12-01T01:54:33.084949+0000","last_unstale":"2023-12-01T01:54:34.322286+0000","last_undegraded":"2023-12-01T01:54:34.322286+0000","last_fullsized":"2023-12-01T01:54:34.322286+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:05.440162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.064825+0000","last_change":"2023-12-01T01:54:32.606639+0000","last_active":"2023-12-01T01:54:33.064825+0000","last_peered":"2023-12-01T01:54:33.064825+0000","last_clean":"2023-12-01T01:54:33.064825+0000","last_became_active":"2023-12-01T01:54:32.606361+0000","last_became_peered":"2023-12-01T01:54:32.606361+0000","last_unstale":"2023-12-01T01:54:33.064825+0000","last_undegraded":"2023-12-01T01:54:33.064825+0000","last_fullsized":"2023-12-01T01:54:33.064825+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:45:44.823359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.070082+0000","last_change":"2023-12-01T01:54:33.086083+0000","last_active":"2023-12-01T01:54:34.070082+0000","last_peered":"2023-12-01T01:54:34.070082+0000","last_clean":"2023-12-01T01:54:34.070082+0000","last_became_active":"2023-12-01T01:54:33.085819+0000","last_became_peered":"2023-12-01T01:54:33.085819+0000","last_unstale":"2023-12-01T01:54:34.070082+0000","last_undegraded":"2023-12-01T01:54:34.070082+0000","last_fullsized":"2023-12-01T01:54:34.070082+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:45:53.648633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.076094+0000","last_change":"2023-12-01T01:54:32.598842+0000","last_active":"2023-12-01T01:54:33.076094+0000","last_peered":"2023-12-01T01:54:33.076094+0000","last_clean":"2023-12-01T01:54:33.076094+0000","last_became_active":"2023-12-01T01:54:32.598491+0000","last_became_peered":"2023-12-01T01:54:32.598491+0000","last_unstale":"2023-12-01T01:54:33.076094+0000","last_undegraded":"2023-12-01T01:54:33.076094+0000","last_fullsized":"2023-12-01T01:54:33.076094+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:10.478463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.321896+0000","last_change":"2023-12-01T01:54:33.085570+0000","last_active":"2023-12-01T01:54:34.321896+0000","last_peered":"2023-12-01T01:54:34.321896+0000","last_clean":"2023-12-01T01:54:34.321896+0000","last_became_active":"2023-12-01T01:54:33.085189+0000","last_became_peered":"2023-12-01T01:54:33.085189+0000","last_unstale":"2023-12-01T01:54:34.321896+0000","last_undegraded":"2023-12-01T01:54:34.321896+0000","last_fullsized":"2023-12-01T01:54:34.321896+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:34:25.469135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.066581+0000","last_change":"2023-12-01T01:54:32.603847+0000","last_active":"2023-12-01T01:54:33.066581+0000","last_peered":"2023-12-01T01:54:33.066581+0000","last_clean":"2023-12-01T01:54:33.066581+0000","last_became_active":"2023-12-01T01:54:32.603564+0000","last_became_peered":"2023-12-01T01:54:32.603564+0000","last_unstale":"2023-12-01T01:54:33.066581+0000","last_undegraded":"2023-12-01T01:54:33.066581+0000","last_fullsized":"2023-12-01T01:54:33.066581+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:52:01.375954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.280605+0000","last_change":"2023-12-01T01:54:33.095819+0000","last_active":"2023-12-01T01:54:34.280605+0000","last_peered":"2023-12-01T01:54:34.280605+0000","last_clean":"2023-12-01T01:54:34.280605+0000","last_became_active":"2023-12-01T01:54:33.095456+0000","last_became_peered":"2023-12-01T01:54:33.095456+0000","last_unstale":"2023-12-01T01:54:34.280605+0000","last_undegraded":"2023-12-01T01:54:34.280605+0000","last_fullsized":"2023-12-01T01:54:34.280605+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:07:54.815046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.066953+0000","last_change":"2023-12-01T01:54:32.599314+0000","last_active":"2023-12-01T01:54:33.066953+0000","last_peered":"2023-12-01T01:54:33.066953+0000","last_clean":"2023-12-01T01:54:33.066953+0000","last_became_active":"2023-12-01T01:54:32.599051+0000","last_became_peered":"2023-12-01T01:54:32.599051+0000","last_unstale":"2023-12-01T01:54:33.066953+0000","last_undegraded":"2023-12-01T01:54:33.066953+0000","last_fullsized":"2023-12-01T01:54:33.066953+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:07:53.810438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-12-01T01:54:34.075817+0000","last_change":"2023-12-01T01:54:33.092893+0000","last_active":"2023-12-01T01:54:34.075817+0000","last_peered":"2023-12-01T01:54:34.075817+0000","last_clean":"2023-12-01T01:54:34.075817+0000","last_became_active":"2023-12-01T01:54:33.092622+0000","last_became_peered":"2023-12-01T01:54:33.092622+0000","last_unstale":"2023-12-01T01:54:34.075817+0000","last_undegraded":"2023-12-01T01:54:34.075817+0000","last_fullsized":"2023-12-01T01:54:34.075817+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:32.561300+0000","last_clean_scrub_stamp":"2023-12-01T01:54:32.561300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:21.023813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"124'1","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.092169+0000","last_change":"2023-12-01T01:54:32.609383+0000","last_active":"2023-12-01T01:54:33.092169+0000","last_peered":"2023-12-01T01:54:33.092169+0000","last_clean":"2023-12-01T01:54:33.092169+0000","last_became_active":"2023-12-01T01:54:32.609021+0000","last_became_peered":"2023-12-01T01:54:32.609021+0000","last_unstale":"2023-12-01T01:54:33.092169+0000","last_undegraded":"2023-12-01T01:54:33.092169+0000","last_fullsized":"2023-12-01T01:54:33.092169+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:54:31.556692+0000","last_clean_scrub_stamp":"2023-12-01T01:54:31.556692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:27:48.367833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"113'205","reported_seq":574,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-12-01T01:54:33.059791+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:54:33.059791+0000","last_peered":"2023-12-01T01:54:33.059791+0000","last_clean":"2023-12-01T01:54:33.059791+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:54:33.059791+0000","last_undegraded":"2023-12-01T01:54:33.059791+0000","last_fullsized":"2023-12-01T01:54:33.059791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575072,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1548288,"data_stored":736944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0549999999999999,"15min":1.069},"min":{"1min":0.77600000000000002,"5min":0.54900000000000004,"15min":0.41299999999999998},"max":{"1min":1.6359999999999999,"5min":1.7430000000000001,"15min":1.8089999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.387,"5min":1.5289999999999999,"15min":2.2029999999999998},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1419999999999999,"15min":1.208},"min":{"1min":0.753,"5min":0.59799999999999998,"15min":0.57299999999999995},"max":{"1min":1.6000000000000001,"5min":1.802,"15min":1.8100000000000001},"last":1.284},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2010000000000001,"15min":1.2509999999999999},"min":{"1min":0.84199999999999997,"5min":0.66800000000000004,"15min":0.59199999999999997},"max":{"1min":1.456,"5min":1.629,"15min":1.744},"last":1.2070000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1080000000000001,"15min":1.117},"min":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.50600000000000001},"max":{"1min":1.3740000000000001,"5min":1.607,"15min":1.728},"last":1.258},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1910000000000001,"15min":1.21},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.36599999999999999},"max":{"1min":1.6080000000000001,"5min":1.681,"15min":1.9690000000000001},"last":1.1659999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1890000000000001,"15min":1.2310000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.496,"5min":1.637,"15min":2.0550000000000002},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.284,"15min":1.3280000000000001},"min":{"1min":0.90900000000000003,"5min":0.63700000000000001,"15min":0.44900000000000001},"max":{"1min":1.5580000000000001,"5min":1.7490000000000001,"15min":2.1880000000000002},"last":1.379}]},{"osd":4,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3,"15min":1.288},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.55500000000000005},"max":{"1min":1.663,"5min":1.8300000000000001,"15min":2.2130000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.385,"15min":1.399},"min":{"1min":0.93600000000000005,"5min":0.77500000000000002,"15min":0.63300000000000001},"max":{"1min":1.694,"5min":1.8380000000000001,"15min":2.0939999999999999},"last":1.4379999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.232,"15min":1.2450000000000001},"min":{"1min":0.72999999999999998,"5min":0.625,"15min":0.42799999999999999},"max":{"1min":1.5449999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"min":{"1min":1.111,"5min":0.69399999999999995,"15min":0.66900000000000004},"max":{"1min":1.6679999999999999,"5min":1.9379999999999999,"15min":2.2930000000000001},"last":1.4930000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3640000000000001,"15min":1.373},"min":{"1min":0.92800000000000005,"5min":0.753,"15min":0.51200000000000001},"max":{"1min":1.5309999999999999,"5min":2.294,"15min":2.3420000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.3540000000000001,"15min":1.3799999999999999},"min":{"1min":0.82899999999999996,"5min":0.82599999999999996,"15min":0.60899999999999999},"max":{"1min":1.5960000000000001,"5min":1.954,"15min":2.3500000000000001},"last":1.337}]}]},{"osd":6,"up_from":13,"seq":55834575073,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36652,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335188,"statfs":{"total":96636764160,"available":96599232512,"internally_reserved":0,"allocated":1544192,"data_stored":736362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.69799999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4139999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.1080000000000001,"15min":1.1319999999999999},"min":{"1min":0.78700000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4570000000000001,"5min":1.579,"15min":2.5569999999999999},"last":0.85099999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.181,"15min":1.167},"min":{"1min":0.80500000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.5489999999999999,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.256,"5min":1.198,"15min":1.208},"min":{"1min":0.90400000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5860000000000001,"5min":2.0720000000000001,"15min":2.0920000000000001},"last":1.2849999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1240000000000001,"15min":1.1659999999999999},"min":{"1min":0.84899999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.462,"5min":1.6359999999999999,"15min":2.3460000000000001},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.216,"15min":1.238},"min":{"1min":0.82199999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.619,"5min":3.024,"15min":3.024},"last":1.4670000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.23,"15min":1.2709999999999999},"min":{"1min":0.81799999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.556,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.008},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4019999999999999,"15min":1.4039999999999999},"min":{"1min":0.95899999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.7450000000000001,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.401}]},{"osd":4,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.2949999999999999,"15min":1.278},"min":{"1min":0.92500000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":2.226},"last":1.9390000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.385,"15min":1.355},"min":{"1min":1.0600000000000001,"5min":0.93600000000000005,"15min":0.78100000000000003},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":2.2839999999999998},"last":1.5189999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.421,"15min":1.409},"min":{"1min":1.1699999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.837},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.496,"15min":1.486},"min":{"1min":1.3180000000000001,"5min":0.96499999999999997,"15min":0.84099999999999997},"max":{"1min":1.9770000000000001,"5min":1.9790000000000001,"15min":2.8580000000000001},"last":1.597}]},{"osd":7,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":1.522,"15min":1.516},"min":{"1min":1.3759999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.958,"5min":1.978,"15min":2.762},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.748,"5min":1.577,"15min":1.5780000000000001},"min":{"1min":1.4159999999999999,"5min":1.038,"15min":1.038},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.8479999999999999},"last":2.016}]}]},{"osd":5,"up_from":13,"seq":55834575072,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":2108,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2158592,"data_stored":1328350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.111,"15min":1.1080000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.522,"5min":1.8240000000000001,"15min":2.2010000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1060000000000001,"15min":1.1439999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5609999999999999,"5min":1.7130000000000001,"15min":2.153},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.149,"15min":1.1499999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6040000000000001,"5min":1.77,"15min":1.77},"last":1.214},{"interface":"front","average":{"1min":1.139,"5min":1.145,"15min":1.157},"min":{"1min":0.754,"5min":0.72999999999999998,"15min":0.67800000000000005},"max":{"1min":1.4890000000000001,"5min":1.6519999999999999,"15min":1.7330000000000001},"last":1.091}]},{"osd":2,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.147,"15min":1.1879999999999999},"min":{"1min":0.71999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.532,"5min":1.702,"15min":1.702},"last":0.872},{"interface":"front","average":{"1min":1.248,"5min":1.226,"15min":1.2370000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.605,"5min":1.9239999999999999,"15min":2.0630000000000002},"last":1.0209999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2190000000000001,"15min":1.2629999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.651,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3109999999999999,"15min":1.304},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.141}]},{"osd":4,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.226,"15min":1.224},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.474,"5min":1.984,"15min":1.984},"last":1.248},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.3009999999999999,"15min":1.3340000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.6299999999999999,"5min":1.927,"15min":1.927},"last":0.96699999999999997}]},{"osd":6,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3300000000000001,"15min":1.325},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.694,"5min":1.885,"15min":1.9790000000000001},"last":1.321},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3799999999999999,"15min":1.371},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.746,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.506}]},{"osd":7,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.45,"15min":1.4670000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.7210000000000001,"5min":2.0499999999999998,"15min":2.0920000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.532,"15min":1.5229999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.861,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.601}]}]},{"osd":4,"up_from":13,"seq":55834575072,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":1548288,"data_stored":736944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.77000000000000002,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5509999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1599999999999999,"15min":1.1739999999999999},"min":{"1min":0.80600000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.603,"5min":1.778,"15min":2.1099999999999999},"last":1.5409999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.222,"15min":1.214},"min":{"1min":0.83699999999999997,"5min":0.72399999999999998,"15min":0.65900000000000003},"max":{"1min":1.8460000000000001,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.044},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.147,"15min":1.135},"min":{"1min":0.72999999999999998,"5min":0.67300000000000004,"15min":0.64200000000000002},"max":{"1min":1.6060000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.94099999999999995}]},{"osd":2,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2150000000000001,"15min":1.202},"min":{"1min":0.91800000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4330000000000001,"5min":1.871,"15min":1.871},"last":1.494},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.288,"15min":1.29},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.599,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.3819999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.246,"15min":1.244},"min":{"1min":0.93300000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5169999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.103},{"interface":"front","average":{"1min":1.383,"5min":1.3180000000000001,"15min":1.3169999999999999},"min":{"1min":1.071,"5min":0.753,"15min":0.753},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.903},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.337,"15min":1.321},"min":{"1min":0.95999999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.7010000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.3080000000000001,"15min":1.292},"min":{"1min":1.0189999999999999,"5min":0.90300000000000002,"15min":0.85399999999999998},"max":{"1min":1.627,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.3080000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":1.3180000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.7649999999999999,"5min":1.8799999999999999,"15min":1.9950000000000001},"last":1.425},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.47,"15min":1.4790000000000001},"min":{"1min":1.2330000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.96,"5min":2.306,"15min":2.54},"last":1.6200000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.468,"15min":1.4630000000000001},"min":{"1min":1.2989999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.8799999999999999,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.804},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.5620000000000001,"15min":1.5660000000000001},"min":{"1min":1.1299999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":1.9830000000000001,"5min":2.04,"15min":2.157},"last":1.5780000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575071,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58808,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313032,"statfs":{"total":96636764160,"available":96576544768,"internally_reserved":0,"allocated":1556480,"data_stored":736556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.84699999999999998,"15min":0.85799999999999998},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.472,"5min":1.472,"15min":1.583},"last":0.41799999999999998},{"interface":"front","average":{"1min":1.032,"5min":0.94299999999999995,"15min":0.98699999999999999},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.435},"max":{"1min":1.395,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.99199999999999999,"15min":1.0580000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.573,"5min":1.6200000000000001,"15min":1.6339999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.91300000000000003,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.48399999999999999,"15min":0.48199999999999998},"max":{"1min":1.3049999999999999,"5min":1.5129999999999999,"15min":1.6519999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.999,"15min":1.0429999999999999},"min":{"1min":0.59399999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3759999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1060000000000001,"15min":1.1799999999999999},"min":{"1min":0.495,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.575,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.495}]},{"osd":4,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1930000000000001,"15min":1.232},"min":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.9319999999999999},"last":0.627},{"interface":"front","average":{"1min":1.228,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":0.85499999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.627,"5min":1.909,"15min":2.0419999999999998},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2370000000000001,"15min":1.2709999999999999},"min":{"1min":1.071,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.706,"5min":1.7669999999999999,"15min":1.806},"last":1.071},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.28,"15min":1.2989999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7549999999999999,"5min":1.831,"15min":2.1880000000000002},"last":0.66500000000000004}]},{"osd":6,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3500000000000001,"15min":1.3700000000000001},"min":{"1min":1.034,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.649,"5min":1.984,"15min":1.984},"last":1.034},{"interface":"front","average":{"1min":1.427,"5min":1.3839999999999999,"15min":1.411},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8109999999999999,"5min":1.9119999999999999,"15min":1.99},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.3660000000000001,"15min":1.411},"min":{"1min":0.873,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8799999999999999,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.873},{"interface":"front","average":{"1min":1.464,"5min":1.419,"15min":1.4430000000000001},"min":{"1min":1.121,"5min":0.87,"15min":0.87},"max":{"1min":1.8879999999999999,"5min":2.0430000000000001,"15min":2.1230000000000002},"last":1.121}]}]},{"osd":2,"up_from":13,"seq":55834575071,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":1548288,"data_stored":736432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2729999999999999,"5min":1.411,"15min":1.5529999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.95299999999999996,"15min":0.99099999999999999},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.502,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.94599999999999995,"15min":0.97899999999999998},"min":{"1min":0.434,"5min":0.375,"15min":0.373},"max":{"1min":1.5,"5min":2.5,"15min":2.5},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.036,"15min":1.075},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.5549999999999999,"5min":1.909,"15min":1.909},"last":1.4390000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.91000000000000003,"15min":0.99099999999999999},"min":{"1min":0.48799999999999999,"5min":0.45800000000000002,"15min":0.45200000000000001},"max":{"1min":1.3500000000000001,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0649999999999999,"15min":1.1419999999999999},"min":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.571,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.071}]},{"osd":4,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2290000000000001,"15min":1.2909999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.571,"5min":1.869,"15min":1.9950000000000001},"last":1.258},{"interface":"front","average":{"1min":1.202,"5min":1.1870000000000001,"15min":1.2470000000000001},"min":{"1min":0.71099999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5069999999999999,"5min":1.6879999999999999,"15min":1.8080000000000001},"last":0.995}]},{"osd":5,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2669999999999999,"15min":1.3069999999999999},"min":{"1min":0.79200000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.6220000000000001,"5min":2.948,"15min":2.948},"last":1.403},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.294,"15min":1.343},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7370000000000001,"5min":2.016,"15min":2.048},"last":1.0249999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.264,"15min":1.3120000000000001},"min":{"1min":0.77600000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.609,"5min":1.6439999999999999,"15min":1.988},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.381,"15min":1.4139999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.2309999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.3089999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.379,"15min":1.4259999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.349},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3160000000000001,"15min":1.371},"min":{"1min":0.83299999999999996,"5min":0.67100000000000004,"15min":0.67000000000000004},"max":{"1min":1.7110000000000001,"5min":1.9259999999999999,"15min":2.101},"last":1.159}]}]},{"osd":1,"up_from":13,"seq":55834575070,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59396,"kb_used_data":2108,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312444,"statfs":{"total":96636764160,"available":96575942656,"internally_reserved":0,"allocated":2158592,"data_stored":1328406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.77900000000000003,"15min":0.83899999999999997},"min":{"1min":0.48199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.5049999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.95699999999999996,"15min":1.0069999999999999},"min":{"1min":0.52100000000000002,"5min":0.40500000000000003,"15min":0.39600000000000002},"max":{"1min":1.2749999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.54400000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.54},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.026,"15min":1.0780000000000001},"min":{"1min":0.64700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.363,"5min":1.7250000000000001,"15min":1.804},"last":0.94099999999999995}]},{"osd":3,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.165,"15min":1.21},"min":{"1min":0.73799999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.51,"5min":1.827,"15min":1.827},"last":1.415},{"interface":"front","average":{"1min":1.036,"5min":0.99299999999999999,"15min":1.0409999999999999},"min":{"1min":0.71099999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5069999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.3959999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.175,"15min":1.1839999999999999},"min":{"1min":0.89800000000000002,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.4850000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.319,"15min":1.335},"min":{"1min":0.90800000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8360000000000001,"5min":1.905,"15min":1.905},"last":1.3080000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2649999999999999,"15min":1.28},"min":{"1min":1.0489999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.706,"5min":1.988,"15min":1.988},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.26,"15min":1.2769999999999999},"min":{"1min":0.95599999999999996,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6439999999999999,"5min":1.6799999999999999,"15min":1.825},"last":1.04}]},{"osd":6,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3089999999999999,"15min":1.333},"min":{"1min":1.087,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6100000000000001,"5min":1.839,"15min":1.861},"last":1.548},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.321,"15min":1.3360000000000001},"min":{"1min":0.88100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.587,"5min":1.742,"15min":1.8109999999999999},"last":1.0109999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3899999999999999,"15min":1.4259999999999999},"min":{"1min":0.99099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.823,"5min":1.9099999999999999,"15min":2.0609999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.371,"15min":1.4039999999999999},"min":{"1min":0.93300000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7869999999999999,"5min":1.931,"15min":1.992},"last":1.4750000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575072,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1544192,"data_stored":736362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.80700000000000005,"15min":0.83999999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.081,"5min":1.3520000000000001,"15min":1.8109999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.93500000000000005,"15min":0.96499999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":1.5620000000000001,"5min":1.5860000000000001,"15min":1.6910000000000001},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95499999999999996,"15min":0.98799999999999999},"min":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.42699999999999999},"max":{"1min":1.3799999999999999,"5min":1.528,"15min":2.157},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.018,"5min":1.0089999999999999,"15min":1.054},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.389,"5min":1.3999999999999999,"15min":1.728},"last":0.90700000000000003}]},{"osd":3,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97599999999999998,"15min":1.004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.331,"5min":1.4850000000000001,"15min":1.7509999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.137,"15min":1.155},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.46500000000000002},"max":{"1min":1.4830000000000001,"5min":1.7070000000000001,"15min":2.2610000000000001},"last":1.3899999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.248,"15min":1.2949999999999999},"min":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6140000000000001,"5min":1.6240000000000001,"15min":2.3719999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.1930000000000001,"15min":1.248},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5800000000000001,"5min":1.7989999999999999,"15min":1.823},"last":1.454}]},{"osd":5,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3,"15min":1.333},"min":{"1min":0.88700000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.5600000000000001,"5min":1.841,"15min":2.351},"last":1.131},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.278,"15min":1.3220000000000001},"min":{"1min":0.89800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6890000000000001,"5min":1.7509999999999999,"15min":2.2909999999999999},"last":1.2450000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.272,"15min":1.3049999999999999},"min":{"1min":0.69299999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5700000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.827},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3600000000000001,"15min":1.381},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7290000000000001,"5min":1.9159999999999999,"15min":2.4380000000000002},"last":1.0760000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.353,"15min":1.3939999999999999},"min":{"1min":0.70699999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":2.4279999999999999},"last":1.353},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3500000000000001,"15min":1.383},"min":{"1min":0.82699999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.7669999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.7609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:54:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T01:54:38.499 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T01:54:38.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.757+0000 7f64584cf640 1 Processor -- start 2023-12-01T01:54:38.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.757+0000 7f64584cf640 1 -- start start 2023-12-01T01:54:38.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.759+0000 7f64584cf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6450143240 0x7f64500b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.759+0000 7f64584cf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f645006caa0 0x7f64500ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.759+0000 7f64584cf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64500b2b00 con 0x7f64500b25e0 2023-12-01T01:54:38.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.759+0000 7f64584cf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64500b2de0 con 0x7f645006caa0 2023-12-01T01:54:38.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.759+0000 7f64584cf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64500b30c0 con 0x7f6450143240 2023-12-01T01:54:38.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.759+0000 7f645626b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6450143240 0x7f64500b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.759+0000 7f6456a6c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f645006caa0 0x7f64500ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.759+0000 7f6456a6c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f645006caa0 0x7f64500ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:36442/0 (socket says 172.21.15.115:36442) 2023-12-01T01:54:38.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.759+0000 7f6456a6c640 1 -- 172.21.15.115:0/4158544845 learned_addr learned my addr 172.21.15.115:0/4158544845 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:38.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.760+0000 7f6456a6c640 1 -- 172.21.15.115:0/4158544845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6450143240 msgr2=0x7f64500b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:38.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.760+0000 7f6456a6c640 1 --2- 172.21.15.115:0/4158544845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6450143240 0x7f64500b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.760+0000 7f6456a6c640 1 -- 172.21.15.115:0/4158544845 >> v1:172.21.15.142:6789/0 conn(0x7f64500b25e0 legacy=0x7f645006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.760+0000 7f6456a6c640 1 -- 172.21.15.115:0/4158544845 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64500b33a0 con 0x7f645006caa0 2023-12-01T01:54:38.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.762+0000 7f6456a6c640 1 --2- 172.21.15.115:0/4158544845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645006caa0 0x7f64500ce0c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6448009e50 tx=0x7f644802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8181c7d6dcb302 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:38.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.762+0000 7f6455269640 1 -- 172.21.15.115:0/4158544845 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644802caf0 con 0x7f645006caa0 2023-12-01T01:54:38.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.762+0000 7f6455269640 1 -- 172.21.15.115:0/4158544845 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644802cc50 con 0x7f645006caa0 2023-12-01T01:54:38.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.762+0000 7f6455269640 1 -- 172.21.15.115:0/4158544845 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480343f0 con 0x7f645006caa0 2023-12-01T01:54:38.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.763+0000 7f64584cf640 1 -- 172.21.15.115:0/4158544845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645006caa0 msgr2=0x7f64500ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.763+0000 7f64584cf640 1 --2- 172.21.15.115:0/4158544845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645006caa0 0x7f64500ce0c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6448009e50 tx=0x7f644802bf50 comp rx=0 tx=0).stop 2023-12-01T01:54:38.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.763+0000 7f64584cf640 1 -- 172.21.15.115:0/4158544845 shutdown_connections 2023-12-01T01:54:38.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.763+0000 7f64584cf640 1 --2- 172.21.15.115:0/4158544845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645006caa0 0x7f64500ce0c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.763+0000 7f64584cf640 1 --2- 172.21.15.115:0/4158544845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6450143240 0x7f64500b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.763+0000 7f64584cf640 1 -- 172.21.15.115:0/4158544845 >> 172.21.15.115:0/4158544845 conn(0x7f6450065960 msgr2=0x7f6450065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:38.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.763+0000 7f64584cf640 1 -- 172.21.15.115:0/4158544845 shutdown_connections 2023-12-01T01:54:38.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.763+0000 7f64584cf640 1 -- 172.21.15.115:0/4158544845 wait complete. 2023-12-01T01:54:38.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.765+0000 7f64584cf640 1 Processor -- start 2023-12-01T01:54:38.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.765+0000 7f64584cf640 1 -- start start 2023-12-01T01:54:38.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.766+0000 7f64584cf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645006caa0 0x7f6450158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.766+0000 7f64584cf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64500b25e0 0x7f6450158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.766+0000 7f64584cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6450143240 0x7f6450151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.766+0000 7f64584cf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64501339c0 con 0x7f6450143240 2023-12-01T01:54:38.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.766+0000 7f64584cf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6450152ab0 con 0x7f645006caa0 2023-12-01T01:54:38.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.766+0000 7f64584cf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6450152d90 con 0x7f64500b25e0 2023-12-01T01:54:38.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.766+0000 7f6456a6c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6450143240 0x7f6450151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.766+0000 7f6456a6c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6450143240 0x7f6450151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49146/0 (socket says 172.21.15.115:49146) 2023-12-01T01:54:38.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.766+0000 7f6456a6c640 1 -- 172.21.15.115:0/2484769516 learned_addr learned my addr 172.21.15.115:0/2484769516 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:38.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.767+0000 7f645626b640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645006caa0 0x7f6450158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.767+0000 7f6455a6a640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64500b25e0 0x7f6450158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.767+0000 7f6456a6c640 1 -- 172.21.15.115:0/2484769516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64500b25e0 msgr2=0x7f6450158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.767+0000 7f6456a6c640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64500b25e0 0x7f6450158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.767+0000 7f6456a6c640 1 -- 172.21.15.115:0/2484769516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645006caa0 msgr2=0x7f6450158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.767+0000 7f6456a6c640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645006caa0 0x7f6450158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.767+0000 7f6456a6c640 1 -- 172.21.15.115:0/2484769516 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6448009b30 con 0x7f6450143240 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.767+0000 7f645626b640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645006caa0 0x7f6450158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.767+0000 7f6456a6c640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6450143240 0x7f6450151ea0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6448009e50 tx=0x7f6448004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.767+0000 7f6446ffd640 1 -- 172.21.15.115:0/2484769516 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448009510 con 0x7f6450143240 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.768+0000 7f6446ffd640 1 -- 172.21.15.115:0/2484769516 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644803e450 con 0x7f6450143240 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.768+0000 7f6446ffd640 1 -- 172.21.15.115:0/2484769516 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644803e800 con 0x7f6450143240 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.768+0000 7f64584cf640 1 -- 172.21.15.115:0/2484769516 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64501e5250 con 0x7f6450143240 2023-12-01T01:54:38.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.768+0000 7f64584cf640 1 -- 172.21.15.115:0/2484769516 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6450153070 con 0x7f6450143240 2023-12-01T01:54:38.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.769+0000 7f64584cf640 1 -- 172.21.15.115:0/2484769516 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64501e50b0 con 0x7f6450143240 2023-12-01T01:54:38.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.770+0000 7f6446ffd640 1 -- 172.21.15.115:0/2484769516 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f644803e960 con 0x7f6450143240 2023-12-01T01:54:38.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.771+0000 7f6446ffd640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643406a550 0x7f643406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:38.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.771+0000 7f6446ffd640 1 -- 172.21.15.115:0/2484769516 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64480ab7a0 con 0x7f6450143240 2023-12-01T01:54:38.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.771+0000 7f645626b640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643406a550 0x7f643406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:38.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.772+0000 7f645626b640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643406a550 0x7f643406ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f644c009b90 tx=0x7f644c008040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:38.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:38.776+0000 7f6446ffd640 1 -- 172.21.15.115:0/2484769516 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f64501e50b0 con 0x7f6450143240 2023-12-01T01:54:39.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.035+0000 7f64584cf640 1 -- 172.21.15.115:0/2484769516 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f64501e50b0 con 0x7f6450143240 2023-12-01T01:54:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.043+0000 7f6446ffd640 1 -- 172.21.15.115:0/2484769516 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f64501e50b0 con 0x7f6450143240 2023-12-01T01:54:39.044 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T01:54:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.045+0000 7f64584cf640 1 -- 172.21.15.115:0/2484769516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643406a550 msgr2=0x7f643406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:39.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.045+0000 7f64584cf640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643406a550 0x7f643406ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f644c009b90 tx=0x7f644c008040 comp rx=0 tx=0).stop 2023-12-01T01:54:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.045+0000 7f64584cf640 1 -- 172.21.15.115:0/2484769516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6450143240 msgr2=0x7f6450151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.045+0000 7f64584cf640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6450143240 0x7f6450151ea0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6448009e50 tx=0x7f6448004a40 comp rx=0 tx=0).stop 2023-12-01T01:54:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.046+0000 7f64584cf640 1 -- 172.21.15.115:0/2484769516 shutdown_connections 2023-12-01T01:54:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.046+0000 7f64584cf640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643406a550 0x7f643406ca10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.046+0000 7f64584cf640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64500b25e0 0x7f6450158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.046+0000 7f64584cf640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6450143240 0x7f6450151ea0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.046+0000 7f64584cf640 1 --2- 172.21.15.115:0/2484769516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645006caa0 0x7f6450158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.046+0000 7f64584cf640 1 -- 172.21.15.115:0/2484769516 >> 172.21.15.115:0/2484769516 conn(0x7f6450065960 msgr2=0x7f645006c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.046+0000 7f64584cf640 1 -- 172.21.15.115:0/2484769516 shutdown_connections 2023-12-01T01:54:39.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.046+0000 7f64584cf640 1 -- 172.21.15.115:0/2484769516 wait complete. 2023-12-01T01:54:39.067 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:54:39.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.278+0000 7f107e22c640 1 Processor -- start 2023-12-01T01:54:39.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.279+0000 7f107e22c640 1 -- start start 2023-12-01T01:54:39.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.280+0000 7f107e22c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f107815e2f0 0x7f10780ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:39.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.280+0000 7f107e22c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f107806f2f0 0x7f10780caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:39.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.280+0000 7f107e22c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10780add10 con 0x7f10780adf10 2023-12-01T01:54:39.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.280+0000 7f107e22c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10780c5760 con 0x7f107806f2f0 2023-12-01T01:54:39.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.280+0000 7f107e22c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107815e6d0 con 0x7f107815e2f0 2023-12-01T01:54:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.280+0000 7f106ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f107806f2f0 0x7f10780caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.280+0000 7f106ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f107806f2f0 0x7f10780caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49148/0 (socket says 172.21.15.115:49148) 2023-12-01T01:54:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.280+0000 7f106ffff640 1 -- 172.21.15.115:0/1402287940 learned_addr learned my addr 172.21.15.115:0/1402287940 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.281+0000 7f10777fe640 1 --2- 172.21.15.115:0/1402287940 >> v2:172.21.15.142:3300/0 conn(0x7f107815e2f0 0x7f10780ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.281+0000 7f106ffff640 1 -- 172.21.15.115:0/1402287940 >> v2:172.21.15.142:3300/0 conn(0x7f107815e2f0 msgr2=0x7f10780ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.281+0000 7f106ffff640 1 --2- 172.21.15.115:0/1402287940 >> v2:172.21.15.142:3300/0 conn(0x7f107815e2f0 0x7f10780ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.281+0000 7f106ffff640 1 -- 172.21.15.115:0/1402287940 >> v1:172.21.15.142:6789/0 conn(0x7f10780adf10 legacy=0x7f10780c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:39.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.281+0000 7f106ffff640 1 -- 172.21.15.115:0/1402287940 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10780cf7b0 con 0x7f107806f2f0 2023-12-01T01:54:39.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.282+0000 7f106ffff640 1 --2- 172.21.15.115:0/1402287940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f107806f2f0 0x7f10780caba0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f107806ef60 tx=0x7f106002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d4c82bbbf11c43 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:39.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.282+0000 7f1076ffd640 1 -- 172.21.15.115:0/1402287940 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060031750 con 0x7f107806f2f0 2023-12-01T01:54:39.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.282+0000 7f1076ffd640 1 -- 172.21.15.115:0/1402287940 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10600318b0 con 0x7f107806f2f0 2023-12-01T01:54:39.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.282+0000 7f1076ffd640 1 -- 172.21.15.115:0/1402287940 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060031c40 con 0x7f107806f2f0 2023-12-01T01:54:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.283+0000 7f107e22c640 1 -- 172.21.15.115:0/1402287940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f107806f2f0 msgr2=0x7f10780caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.283+0000 7f107e22c640 1 --2- 172.21.15.115:0/1402287940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f107806f2f0 0x7f10780caba0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f107806ef60 tx=0x7f106002b7f0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.283+0000 7f107e22c640 1 -- 172.21.15.115:0/1402287940 shutdown_connections 2023-12-01T01:54:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.283+0000 7f107e22c640 1 --2- 172.21.15.115:0/1402287940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f107806f2f0 0x7f10780caba0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.283+0000 7f107e22c640 1 --2- 172.21.15.115:0/1402287940 >> v2:172.21.15.142:3300/0 conn(0x7f107815e2f0 0x7f10780ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.283+0000 7f107e22c640 1 -- 172.21.15.115:0/1402287940 >> 172.21.15.115:0/1402287940 conn(0x7f10780ce880 msgr2=0x7f10780cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.283+0000 7f107e22c640 1 -- 172.21.15.115:0/1402287940 shutdown_connections 2023-12-01T01:54:39.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.283+0000 7f107e22c640 1 -- 172.21.15.115:0/1402287940 wait complete. 2023-12-01T01:54:39.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.285+0000 7f107e22c640 1 Processor -- start 2023-12-01T01:54:39.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.285+0000 7f107e22c640 1 -- start start 2023-12-01T01:54:39.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.286+0000 7f107e22c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f107806f2f0 0x7f107814f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:39.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.286+0000 7f107e22c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10780adf10 0x7f10780af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:39.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.286+0000 7f107e22c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f107815e2f0 0x7f10780afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:39.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.286+0000 7f107e22c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1078150290 con 0x7f10780adf10 2023-12-01T01:54:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.286+0000 7f107e22c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10780c6700 con 0x7f107806f2f0 2023-12-01T01:54:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.286+0000 7f107e22c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10780c69e0 con 0x7f107815e2f0 2023-12-01T01:54:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.287+0000 7f106ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10780adf10 0x7f10780af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.287+0000 7f106ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10780adf10 0x7f10780af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49162/0 (socket says 172.21.15.115:49162) 2023-12-01T01:54:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.287+0000 7f106ffff640 1 -- 172.21.15.115:0/840654774 learned_addr learned my addr 172.21.15.115:0/840654774 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.287+0000 7f10777fe640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f107806f2f0 0x7f107814f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.287+0000 7f106ffff640 1 -- 172.21.15.115:0/840654774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f107815e2f0 msgr2=0x7f10780afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.287+0000 7f1077fff640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f107815e2f0 0x7f10780afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.287+0000 7f106ffff640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f107815e2f0 0x7f10780afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.287+0000 7f106ffff640 1 -- 172.21.15.115:0/840654774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f107806f2f0 msgr2=0x7f107814f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.287+0000 7f106ffff640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f107806f2f0 0x7f107814f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.287+0000 7f106ffff640 1 -- 172.21.15.115:0/840654774 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600094d0 con 0x7f10780adf10 2023-12-01T01:54:39.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.288+0000 7f106ffff640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10780adf10 0x7f10780af520 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f1060000c00 tx=0x7f10600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:39.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.288+0000 7f1074ff9640 1 -- 172.21.15.115:0/840654774 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060033c80 con 0x7f10780adf10 2023-12-01T01:54:39.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.288+0000 7f1074ff9640 1 -- 172.21.15.115:0/840654774 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060033de0 con 0x7f10780adf10 2023-12-01T01:54:39.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.288+0000 7f1074ff9640 1 -- 172.21.15.115:0/840654774 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106003c640 con 0x7f10780adf10 2023-12-01T01:54:39.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.288+0000 7f107e22c640 1 -- 172.21.15.115:0/840654774 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10780c6cc0 con 0x7f10780adf10 2023-12-01T01:54:39.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.288+0000 7f107e22c640 1 -- 172.21.15.115:0/840654774 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10780c7280 con 0x7f10780adf10 2023-12-01T01:54:39.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.289+0000 7f107e22c640 1 -- 172.21.15.115:0/840654774 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107806fda0 con 0x7f10780adf10 2023-12-01T01:54:39.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.290+0000 7f1074ff9640 1 -- 172.21.15.115:0/840654774 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f106003a040 con 0x7f10780adf10 2023-12-01T01:54:39.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.291+0000 7f1074ff9640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f105806a550 0x7f105806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:39.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.292+0000 7f1074ff9640 1 -- 172.21.15.115:0/840654774 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10600aa6e0 con 0x7f10780adf10 2023-12-01T01:54:39.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.292+0000 7f10777fe640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f105806a550 0x7f105806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:39.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.292+0000 7f10777fe640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f105806a550 0x7f105806ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1068006fd0 tx=0x7f1068008040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:39.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.296+0000 7f1074ff9640 1 -- 172.21.15.115:0/840654774 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1060075c40 con 0x7f10780adf10 2023-12-01T01:54:39.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.520+0000 7f107e22c640 1 -- 172.21.15.115:0/840654774 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f107804e890 con 0x7f10780adf10 2023-12-01T01:54:39.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.522+0000 7f1074ff9640 1 -- 172.21.15.115:0/840654774 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f1060079260 con 0x7f10780adf10 2023-12-01T01:54:39.523 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:39.523 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/4122088547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":4122088547},{"type":"v1","addr":"172.21.15.142:6834","nonce":4122088547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/299645728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":299645728},{"type":"v1","addr":"172.21.15.142:6835","nonce":299645728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7983,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/150770274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":150770274},{"type":"v1","addr":"172.21.15.115:6836","nonce":150770274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:54:33.047892+0000","modified":"2023-12-01T01:54:36.558949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/2398273191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":2398273191},{"type":"v1","addr":"172.21.15.115:6837","nonce":2398273191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-01T01:54:39.523 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 56 2023-12-01T01:54:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.524+0000 7f107e22c640 1 -- 172.21.15.115:0/840654774 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f105806a550 msgr2=0x7f105806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.524+0000 7f107e22c640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f105806a550 0x7f105806ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1068006fd0 tx=0x7f1068008040 comp rx=0 tx=0).stop 2023-12-01T01:54:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.525+0000 7f107e22c640 1 -- 172.21.15.115:0/840654774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10780adf10 msgr2=0x7f10780af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.525+0000 7f107e22c640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10780adf10 0x7f10780af520 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f1060000c00 tx=0x7f10600046c0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.525+0000 7f107e22c640 1 -- 172.21.15.115:0/840654774 shutdown_connections 2023-12-01T01:54:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.525+0000 7f107e22c640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f105806a550 0x7f105806ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.525+0000 7f107e22c640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f107815e2f0 0x7f10780afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.525+0000 7f107e22c640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10780adf10 0x7f10780af520 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.525+0000 7f107e22c640 1 --2- 172.21.15.115:0/840654774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f107806f2f0 0x7f107814f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:39.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.525+0000 7f107e22c640 1 -- 172.21.15.115:0/840654774 >> 172.21.15.115:0/840654774 conn(0x7f10780ce880 msgr2=0x7f107815cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:39.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.525+0000 7f107e22c640 1 -- 172.21.15.115:0/840654774 shutdown_connections 2023-12-01T01:54:39.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:39.525+0000 7f107e22c640 1 -- 172.21.15.115:0/840654774 wait complete. 2023-12-01T01:54:39.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T01:54:33.047892+0000', 'modified': '2023-12-01T01:54:36.558949+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7986}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7986': {'gid': 7986, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6837/2398273191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6835', 'nonce': 2398273191}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 2398273191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T01:54:39.947 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T01:54:39.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-01T01:54:40.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.165+0000 7f9f03992640 1 Processor -- start 2023-12-01T01:54:40.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.165+0000 7f9f03992640 1 -- start start 2023-12-01T01:54:40.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.166+0000 7f9f03992640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 0x7f9efc0d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.166+0000 7f9f03992640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9efc0ce0d0 0x7f9efc0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f03992640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9efc15d370 con 0x7f9efc0b8610 2023-12-01T01:54:40.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f03992640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9efc0d19a0 con 0x7f9efc0ce0d0 2023-12-01T01:54:40.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f03992640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0d1c80 con 0x7f9efc15cf90 2023-12-01T01:54:40.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f01f2f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9efc0ce0d0 0x7f9efc0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f01f2f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9efc0ce0d0 0x7f9efc0b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49168/0 (socket says 172.21.15.115:49168) 2023-12-01T01:54:40.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f01f2f640 1 -- 172.21.15.115:0/893662216 learned_addr learned my addr 172.21.15.115:0/893662216 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:40.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f0172e640 1 --2- 172.21.15.115:0/893662216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 0x7f9efc0d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f01f2f640 1 -- 172.21.15.115:0/893662216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 msgr2=0x7f9efc0d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f01f2f640 1 --2- 172.21.15.115:0/893662216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 0x7f9efc0d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f01f2f640 1 -- 172.21.15.115:0/893662216 >> v1:172.21.15.142:6789/0 conn(0x7f9efc0b8610 legacy=0x7f9efc0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.167+0000 7f9f01f2f640 1 -- 172.21.15.115:0/893662216 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc0d1f60 con 0x7f9efc0ce0d0 2023-12-01T01:54:40.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.168+0000 7f9f01f2f640 1 --2- 172.21.15.115:0/893662216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0ce0d0 0x7f9efc0b5d90 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009e50 tx=0x7f9ef402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b244034c556c059 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:40.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.169+0000 7f9ef3fff640 1 -- 172.21.15.115:0/893662216 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef402ca70 con 0x7f9efc0ce0d0 2023-12-01T01:54:40.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.169+0000 7f9ef3fff640 1 -- 172.21.15.115:0/893662216 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef402cbd0 con 0x7f9efc0ce0d0 2023-12-01T01:54:40.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.169+0000 7f9ef3fff640 1 -- 172.21.15.115:0/893662216 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40343f0 con 0x7f9efc0ce0d0 2023-12-01T01:54:40.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.169+0000 7f9f03992640 1 -- 172.21.15.115:0/893662216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0ce0d0 msgr2=0x7f9efc0b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.170+0000 7f9f03992640 1 --2- 172.21.15.115:0/893662216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0ce0d0 0x7f9efc0b5d90 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009e50 tx=0x7f9ef402bed0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.170+0000 7f9f03992640 1 -- 172.21.15.115:0/893662216 shutdown_connections 2023-12-01T01:54:40.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.170+0000 7f9f03992640 1 --2- 172.21.15.115:0/893662216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0ce0d0 0x7f9efc0b5d90 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.170+0000 7f9f03992640 1 --2- 172.21.15.115:0/893662216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 0x7f9efc0d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.170+0000 7f9f03992640 1 -- 172.21.15.115:0/893662216 >> 172.21.15.115:0/893662216 conn(0x7f9efc065960 msgr2=0x7f9efc065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:40.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.170+0000 7f9f03992640 1 -- 172.21.15.115:0/893662216 shutdown_connections 2023-12-01T01:54:40.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.170+0000 7f9f03992640 1 -- 172.21.15.115:0/893662216 wait complete. 2023-12-01T01:54:40.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.172+0000 7f9f03992640 1 Processor -- start 2023-12-01T01:54:40.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.172+0000 7f9f03992640 1 -- start start 2023-12-01T01:54:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f03992640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0b8610 0x7f9efc153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f03992640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9efc0ce0d0 0x7f9efc151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f03992640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 0x7f9efc1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f03992640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0b3970 con 0x7f9efc0b8610 2023-12-01T01:54:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f03992640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0b3c50 con 0x7f9efc0ce0d0 2023-12-01T01:54:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f03992640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0b3f30 con 0x7f9efc15cf90 2023-12-01T01:54:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f0172e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0b8610 0x7f9efc153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f0172e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0b8610 0x7f9efc153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49174/0 (socket says 172.21.15.115:49174) 2023-12-01T01:54:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f0172e640 1 -- 172.21.15.115:0/210784458 learned_addr learned my addr 172.21.15.115:0/210784458 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:40.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9efc0ce0d0 0x7f9efc151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.173+0000 7f9f01f2f640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 0x7f9efc1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9f0172e640 1 -- 172.21.15.115:0/210784458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 msgr2=0x7f9efc1509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9f0172e640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 0x7f9efc1509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9f0172e640 1 -- 172.21.15.115:0/210784458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9efc0ce0d0 msgr2=0x7f9efc151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9f0172e640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9efc0ce0d0 0x7f9efc151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9f0172e640 1 -- 172.21.15.115:0/210784458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4009b30 con 0x7f9efc0b8610 2023-12-01T01:54:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9f01f2f640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 0x7f9efc1509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:40.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9f0172e640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0b8610 0x7f9efc153c60 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800d5f0 tx=0x7f9ef800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9ef1ffb640 1 -- 172.21.15.115:0/210784458 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80125f0 con 0x7f9efc0b8610 2023-12-01T01:54:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9ef1ffb640 1 -- 172.21.15.115:0/210784458 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef8012750 con 0x7f9efc0b8610 2023-12-01T01:54:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9f03992640 1 -- 172.21.15.115:0/210784458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc0b4210 con 0x7f9efc0b8610 2023-12-01T01:54:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9ef1ffb640 1 -- 172.21.15.115:0/210784458 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8012b00 con 0x7f9efc0b8610 2023-12-01T01:54:40.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.174+0000 7f9f03992640 1 -- 172.21.15.115:0/210784458 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc154d20 con 0x7f9efc0b8610 2023-12-01T01:54:40.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.177+0000 7f9ef1ffb640 1 -- 172.21.15.115:0/210784458 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9ef8012c60 con 0x7f9efc0b8610 2023-12-01T01:54:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.178+0000 7f9ef1ffb640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ee006a550 0x7f9ee006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.178+0000 7f9ef1ffb640 1 -- 172.21.15.115:0/210784458 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ef8088fc0 con 0x7f9efc0b8610 2023-12-01T01:54:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.179+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ee006a550 0x7f9ee006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.179+0000 7f9f065dd740 1 Processor -- start 2023-12-01T01:54:40.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.179+0000 7f9f065dd740 1 -- start start 2023-12-01T01:54:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.179+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ee006a550 0x7f9ee006ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0b83e0 tx=0x7f9eec009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de80fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7df8a0 0x5564de80cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de7e34a0 0x5564de80d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f065dd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564de7bb2f0 con 0x5564de7df8a0 2023-12-01T01:54:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f065dd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564de74ff80 con 0x5564de7e34a0 2023-12-01T01:54:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f065dd740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564de74d710 con 0x5564de80f920 2023-12-01T01:54:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f00f2d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7df8a0 0x5564de80cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f00f2d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7df8a0 0x5564de80cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49190/0 (socket says 172.21.15.115:49190) 2023-12-01T01:54:40.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f00f2d640 1 -- 172.21.15.115:0/3163511486 learned_addr learned my addr 172.21.15.115:0/3163511486 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f01f2f640 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de7e34a0 0x5564de80d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f00f2d640 1 -- 172.21.15.115:0/3163511486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 msgr2=0x5564de80fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.180+0000 7f9f0172e640 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de80fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.181+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de80fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.181+0000 7f9f00f2d640 1 -- 172.21.15.115:0/3163511486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de7e34a0 msgr2=0x5564de80d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.181+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de7e34a0 0x5564de80d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.181+0000 7f9f00f2d640 1 -- 172.21.15.115:0/3163511486 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564de710690 con 0x5564de7df8a0 2023-12-01T01:54:40.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.181+0000 7f9f01f2f640 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de7e34a0 0x5564de80d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.181+0000 7f9f0172e640 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de80fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.181+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7df8a0 0x5564de80cb90 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0055b0 tx=0x7f9eec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f93223d6b3df7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.181+0000 7f9ee6ffd640 1 -- 172.21.15.115:0/3163511486 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec015880 con 0x5564de7df8a0 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.181+0000 7f9ee6ffd640 1 -- 172.21.15.115:0/3163511486 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec0159e0 con 0x5564de7df8a0 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.181+0000 7f9ee6ffd640 1 -- 172.21.15.115:0/3163511486 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec015d70 con 0x5564de7df8a0 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.182+0000 7f9f065dd740 1 -- 172.21.15.115:0/3163511486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7df8a0 msgr2=0x5564de80cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.182+0000 7f9f065dd740 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7df8a0 0x5564de80cb90 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0055b0 tx=0x7f9eec011000 comp rx=0 tx=0).stop 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.182+0000 7f9f065dd740 1 -- 172.21.15.115:0/3163511486 shutdown_connections 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.182+0000 7f9f065dd740 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de7e34a0 0x5564de80d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.182+0000 7f9f065dd740 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7df8a0 0x5564de80cb90 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.182+0000 7f9f065dd740 1 --2- 172.21.15.115:0/3163511486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de80fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.182+0000 7f9f065dd740 1 -- 172.21.15.115:0/3163511486 >> 172.21.15.115:0/3163511486 conn(0x5564de7e1bb0 msgr2=0x5564de82a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.182+0000 7f9f065dd740 1 -- 172.21.15.115:0/3163511486 shutdown_connections 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.182+0000 7f9f065dd740 1 -- 172.21.15.115:0/3163511486 wait complete. 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.183+0000 7f9f065dd740 1 -- start start 2023-12-01T01:54:40.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.183+0000 7f9f065dd740 1 Processor -- start 2023-12-01T01:54:40.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de7e34a0 0x5564de807070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de80f920 0x5564de7f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de805250 0x5564de82cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f065dd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564de749000 con 0x5564de805250 2023-12-01T01:54:40.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f065dd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564de6ea7d0 con 0x5564de80f920 2023-12-01T01:54:40.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f065dd740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564de6ea470 con 0x5564de7e34a0 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f065dd740 10 client.0 _ll_register_callbacks cb 0x5564de7391b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f01f2f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de805250 0x5564de82cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f01f2f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de805250 0x5564de82cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49200/0 (socket says 172.21.15.115:49200) 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f01f2f640 1 -- 172.21.15.115:0/1292518386 learned_addr learned my addr 172.21.15.115:0/1292518386 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.184+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de80f920 0x5564de7f9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9f0172e640 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de7e34a0 0x5564de807070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9f01f2f640 1 -- 172.21.15.115:0/1292518386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de7e34a0 msgr2=0x5564de807070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9f01f2f640 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de7e34a0 0x5564de807070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9f01f2f640 1 -- 172.21.15.115:0/1292518386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de80f920 msgr2=0x5564de7f9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9f01f2f640 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de80f920 0x5564de7f9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9f01f2f640 1 -- 172.21.15.115:0/1292518386 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eec00f040 con 0x5564de805250 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de80f920 0x5564de7f9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9f01f2f640 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de805250 0x5564de82cdb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4036040 tx=0x7f9ef4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fcf06df4271936 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9edbfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:54:40.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9edbfff640 1 -- 172.21.15.115:0/1292518386 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4034b30 con 0x5564de805250 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9edbfff640 1 -- 172.21.15.115:0/1292518386 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ef4034c90 con 0x5564de805250 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.185+0000 7f9f065dd740 1 -- 172.21.15.115:0/1292518386 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564de773550 con 0x5564de805250 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9edbfff640 1 -- 172.21.15.115:0/1292518386 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ef4034e50 con 0x5564de805250 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9f065dd740 10 client.8118 fetch_fsmap learned FSMap version 56 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9f065dd740 1 -- 172.21.15.115:0/1292518386 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5564de747680 con 0x5564de805250 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9edbfff640 1 -- 172.21.15.115:0/1292518386 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5564de773550 con 0x5564de805250 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9edbfff640 1 -- 172.21.15.115:0/1292518386 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9ef403c040 con 0x5564de805250 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9f065dd740 10 client.8118 fetch_fsmap finished waiting for FSMap version 56 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9f065dd740 10 client.8118 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9f065dd740 20 client.8118 populate_metadata read hostname 'smithi115' 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9f065dd740 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.115:6835/2398273191,v1:172.21.15.115:6837/2398273191] conn(0x5564de810c30 0x5564de808da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9f065dd740 4 client.8118 mds_command: new command op to 7986 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9f065dd740 1 -- 172.21.15.115:0/1292518386 --> [v2:172.21.15.115:6835/2398273191,v1:172.21.15.115:6837/2398273191] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564de73bb50 con 0x5564de810c30 2023-12-01T01:54:40.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.186+0000 7f9f0172e640 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.115:6835/2398273191,v1:172.21.15.115:6837/2398273191] conn(0x5564de810c30 0x5564de808da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.187+0000 7f9f0172e640 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.115:6835/2398273191,v1:172.21.15.115:6837/2398273191] conn(0x5564de810c30 0x5564de808da0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=978d9491c1f60f34 server_cookie=a63d164cd0144f52 in_seq=0 out_seq=0 2023-12-01T01:54:40.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.187+0000 7f9edbfff640 10 client.8118 ms_handle_connect on v2:172.21.15.115:6835/2398273191 2023-12-01T01:54:40.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.187+0000 7f9edbfff640 1 -- 172.21.15.115:0/1292518386 <== mds.0 v2:172.21.15.115:6835/2398273191 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5564de73bb50 con 0x5564de810c30 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.187+0000 7f9edbfff640 10 client.8118 handle_command_reply: tid=0 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 client.8118 shutdown 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 -- 172.21.15.115:0/1292518386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de805250 msgr2=0x5564de82cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de805250 0x5564de82cdb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4036040 tx=0x7f9ef4004ab0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 -- 172.21.15.115:0/1292518386 shutdown_connections 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.115:6835/2398273191,v1:172.21.15.115:6837/2398273191] conn(0x5564de810c30 0x5564de808da0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de805250 0x5564de82cdb0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de80f920 0x5564de7f9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9edbfff640 0 client.8118 ms_handle_reset on v2:172.21.15.115:6835/2398273191 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 --2- 172.21.15.115:0/1292518386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de7e34a0 0x5564de807070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 -- 172.21.15.115:0/1292518386 >> 172.21.15.115:0/1292518386 conn(0x5564de7df8a0 msgr2=0x5564de8136c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 -- 172.21.15.115:0/1292518386 shutdown_connections 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 1 -- 172.21.15.115:0/1292518386 wait complete. 2023-12-01T01:54:40.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.188+0000 7f9f065dd740 20 client.8118 trim_cache size 0 max 16384 2023-12-01T01:54:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.204+0000 7f9f065dd740 1 Processor -- start 2023-12-01T01:54:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.204+0000 7f9f065dd740 1 -- start start 2023-12-01T01:54:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.205+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de80fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.205+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de805250 0x5564de80d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.205+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de7e39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.205+0000 7f9f065dd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564de4b0f20 con 0x5564de7e34a0 2023-12-01T01:54:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.205+0000 7f9f065dd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564de7e5ab0 con 0x5564de805250 2023-12-01T01:54:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.205+0000 7f9f065dd740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564de7e13b0 con 0x5564de80f920 2023-12-01T01:54:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.205+0000 7f9f0172e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de80fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9f0172e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de80fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49206/0 (socket says 172.21.15.115:49206) 2023-12-01T01:54:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9f0172e640 1 -- 172.21.15.115:0/348868378 learned_addr learned my addr 172.21.15.115:0/348868378 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9f0172e640 1 -- 172.21.15.115:0/348868378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 msgr2=0x5564de7e39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9f0172e640 1 --2- 172.21.15.115:0/348868378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de7e39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9f0172e640 1 -- 172.21.15.115:0/348868378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de805250 msgr2=0x5564de80d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9f0172e640 1 --2- 172.21.15.115:0/348868378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de805250 0x5564de80d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9f0172e640 1 -- 172.21.15.115:0/348868378 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564de7e1200 con 0x5564de7e34a0 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9f0172e640 1 --2- 172.21.15.115:0/348868378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de80fd00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8013040 tx=0x7f9ef805e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add379f4768c1dab server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9ee77fe640 1 -- 172.21.15.115:0/348868378 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef805bef0 con 0x5564de7e34a0 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9ee77fe640 1 -- 172.21.15.115:0/348868378 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef805b030 con 0x5564de7e34a0 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.206+0000 7f9ee77fe640 1 -- 172.21.15.115:0/348868378 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80600a0 con 0x5564de7e34a0 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.207+0000 7f9f065dd740 1 -- 172.21.15.115:0/348868378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 msgr2=0x5564de80fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.207+0000 7f9f065dd740 1 --2- 172.21.15.115:0/348868378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de80fd00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8013040 tx=0x7f9ef805e040 comp rx=0 tx=0).stop 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.207+0000 7f9f065dd740 1 -- 172.21.15.115:0/348868378 shutdown_connections 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.207+0000 7f9f065dd740 1 --2- 172.21.15.115:0/348868378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de7e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.207+0000 7f9f065dd740 1 --2- 172.21.15.115:0/348868378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de805250 0x5564de80d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.207+0000 7f9f065dd740 1 --2- 172.21.15.115:0/348868378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de80fd00 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.207+0000 7f9f065dd740 1 -- 172.21.15.115:0/348868378 >> 172.21.15.115:0/348868378 conn(0x5564de7df8a0 msgr2=0x5564de807070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.207+0000 7f9f065dd740 1 -- 172.21.15.115:0/348868378 shutdown_connections 2023-12-01T01:54:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.207+0000 7f9f065dd740 1 -- 172.21.15.115:0/348868378 wait complete. 2023-12-01T01:54:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.208+0000 7f9f065dd740 1 -- start start 2023-12-01T01:54:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.208+0000 7f9f065dd740 1 Processor -- start 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.209+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de7e3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.209+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de805250 0x5564de82dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.209+0000 7f9f065dd740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de82e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.209+0000 7f9f065dd740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564de891c20 con 0x5564de7e34a0 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.209+0000 7f9f065dd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564de803b50 con 0x5564de805250 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.209+0000 7f9f065dd740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564de803d00 con 0x5564de80f920 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.209+0000 7f9f065dd740 10 client.0 _ll_register_callbacks cb 0x7f9eec007df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.209+0000 7f9f0172e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de7e3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.209+0000 7f9f0172e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de7e3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49220/0 (socket says 172.21.15.115:49220) 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.209+0000 7f9f0172e640 1 -- 172.21.15.115:0/3926177381 learned_addr learned my addr 172.21.15.115:0/3926177381 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de805250 0x5564de82dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9f0172e640 1 -- 172.21.15.115:0/3926177381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 msgr2=0x5564de82e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9f0172e640 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de82e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9f0172e640 1 -- 172.21.15.115:0/3926177381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de805250 msgr2=0x5564de82dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9f0172e640 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de805250 0x5564de82dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9f0172e640 1 -- 172.21.15.115:0/3926177381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc154d20 con 0x5564de7e34a0 2023-12-01T01:54:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9f0172e640 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de7e3970 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8006fa0 tx=0x7f9ef806d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f778d19e5f89f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9edbfff640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:54:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9edbfff640 1 -- 172.21.15.115:0/3926177381 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef805bef0 con 0x5564de7e34a0 2023-12-01T01:54:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9edbfff640 1 -- 172.21.15.115:0/3926177381 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ef80547d0 con 0x5564de7e34a0 2023-12-01T01:54:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9f065dd740 1 -- 172.21.15.115:0/3926177381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9eec0159e0 con 0x5564de7e34a0 2023-12-01T01:54:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9edbfff640 1 -- 172.21.15.115:0/3926177381 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ef8054990 con 0x5564de7e34a0 2023-12-01T01:54:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9f065dd740 10 client.8124 fetch_fsmap learned FSMap version 56 2023-12-01T01:54:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.210+0000 7f9f065dd740 1 -- 172.21.15.115:0/3926177381 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5564de6e99e0 con 0x5564de7e34a0 2023-12-01T01:54:40.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.211+0000 7f9edbfff640 1 -- 172.21.15.115:0/3926177381 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec0159e0 con 0x5564de7e34a0 2023-12-01T01:54:40.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.211+0000 7f9edbfff640 1 -- 172.21.15.115:0/3926177381 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 56) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9ef8081120 con 0x5564de7e34a0 2023-12-01T01:54:40.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.211+0000 7f9f065dd740 10 client.8124 fetch_fsmap finished waiting for FSMap version 56 2023-12-01T01:54:40.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.211+0000 7f9f065dd740 10 client.8124 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.211+0000 7f9f065dd740 20 client.8124 populate_metadata read hostname 'smithi115' 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.211+0000 7f9f065dd740 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.115:6835/2398273191,v1:172.21.15.115:6837/2398273191] conn(0x5564de82a1d0 0x5564de82a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.211+0000 7f9f065dd740 4 client.8124 mds_command: new command op to 7986 tid=0[{"prefix": "status"}] 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.211+0000 7f9f065dd740 1 -- 172.21.15.115:0/3926177381 --> [v2:172.21.15.115:6835/2398273191,v1:172.21.15.115:6837/2398273191] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5564de742080 con 0x5564de82a1d0 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.211+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.115:6835/2398273191,v1:172.21.15.115:6837/2398273191] conn(0x5564de82a1d0 0x5564de82a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9edbfff640 10 client.8124 ms_handle_connect on v2:172.21.15.115:6835/2398273191 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f00f2d640 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.115:6835/2398273191,v1:172.21.15.115:6837/2398273191] conn(0x5564de82a1d0 0x5564de82a5b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f458ca7613b172 server_cookie=95eddcc70ebe0bdb in_seq=0 out_seq=0 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9edbfff640 1 -- 172.21.15.115:0/3926177381 <== mds.0 v2:172.21.15.115:6835/2398273191 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5564de742080 con 0x5564de82a1d0 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9edbfff640 10 client.8124 handle_command_reply: tid=0 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 client.8124 shutdown 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 -- 172.21.15.115:0/3926177381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 msgr2=0x5564de7e3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de7e3970 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8006fa0 tx=0x7f9ef806d040 comp rx=0 tx=0).stop 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 -- 172.21.15.115:0/3926177381 shutdown_connections 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.115:6835/2398273191,v1:172.21.15.115:6837/2398273191] conn(0x5564de82a1d0 0x5564de82a5b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5564de80f920 0x5564de82e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564de805250 0x5564de82dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9edbfff640 0 client.8124 ms_handle_reset on v2:172.21.15.115:6835/2398273191 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 --2- 172.21.15.115:0/3926177381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5564de7e34a0 0x5564de7e3970 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 -- 172.21.15.115:0/3926177381 >> 172.21.15.115:0/3926177381 conn(0x5564de7df8a0 msgr2=0x5564de80ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 -- 172.21.15.115:0/3926177381 shutdown_connections 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.212+0000 7f9f065dd740 1 -- 172.21.15.115:0/3926177381 wait complete. 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f065dd740 20 client.8124 trim_cache size 0 max 16384 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T01:54:40.217 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 7986, 2023-12-01T01:54:40.218 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T01:54:40.218 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T01:54:40.218 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T01:54:40.218 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 7.1606612810000003, 2023-12-01T01:54:40.218 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 56, 2023-12-01T01:54:40.218 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 125, 2023-12-01T01:54:40.218 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 125, 2023-12-01T01:54:40.218 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 12.724825568 2023-12-01T01:54:40.218 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 -- 172.21.15.115:0/210784458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ee006a550 msgr2=0x7f9ee006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ee006a550 0x7f9ee006ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0b83e0 tx=0x7f9eec009040 comp rx=0 tx=0).stop 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 -- 172.21.15.115:0/210784458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0b8610 msgr2=0x7f9efc153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0b8610 0x7f9efc153c60 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800d5f0 tx=0x7f9ef800dab0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 -- 172.21.15.115:0/210784458 shutdown_connections 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9ee006a550 0x7f9ee006ca10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9efc15cf90 0x7f9efc1509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9efc0ce0d0 0x7f9efc151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 --2- 172.21.15.115:0/210784458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9efc0b8610 0x7f9efc153c60 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 -- 172.21.15.115:0/210784458 >> 172.21.15.115:0/210784458 conn(0x7f9efc065960 msgr2=0x7f9efc15b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 -- 172.21.15.115:0/210784458 shutdown_connections 2023-12-01T01:54:40.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.213+0000 7f9f03992640 1 -- 172.21.15.115:0/210784458 wait complete. 2023-12-01T01:54:40.232 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:54:40.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.441+0000 7f4c1b59e640 1 Processor -- start 2023-12-01T01:54:40.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.441+0000 7f4c1b59e640 1 -- start start 2023-12-01T01:54:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.443+0000 7f4c1b59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4c1c0d05d0 0x7f4c1c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.443+0000 7f4c1b59e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c15e020 con 0x7f4c1c15db90 2023-12-01T01:54:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.443+0000 7f4c1b59e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c0cd3f0 con 0x7f4c1c15d7b0 2023-12-01T01:54:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.443+0000 7f4c1b59e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c0d0f60 con 0x7f4c1c0d05d0 2023-12-01T01:54:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.443+0000 7f4c13fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4c1c15db90 0x7f4c1c0cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48312/0 (socket says 172.21.15.115:48312) 2023-12-01T01:54:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.443+0000 7f4c13fff640 1 -- 172.21.15.115:0/3796374716 learned_addr learned my addr 172.21.15.115:0/3796374716 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.443+0000 7f4c1ad9d640 1 --2- 172.21.15.115:0/3796374716 >> v2:172.21.15.115:3300/0 conn(0x7f4c1c0d05d0 0x7f4c1c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.444+0000 7f4c19d9b640 1 -- 172.21.15.115:0/3796374716 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457697630 0 0) 0x7f4c1c15e020 con 0x7f4c1c15db90 2023-12-01T01:54:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.444+0000 7f4c19d9b640 1 -- 172.21.15.115:0/3796374716 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c08003520 con 0x7f4c1c15db90 2023-12-01T01:54:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.444+0000 7f4c1ad9d640 1 -- 172.21.15.115:0/3796374716 >> v1:172.21.15.142:6789/0 conn(0x7f4c1c15d7b0 legacy=0x7f4c1c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.444+0000 7f4c1ad9d640 1 -- 172.21.15.115:0/3796374716 >> v1:172.21.15.115:6789/0 conn(0x7f4c1c15db90 legacy=0x7f4c1c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.444+0000 7f4c1ad9d640 1 -- 172.21.15.115:0/3796374716 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c0cd0f0 con 0x7f4c1c0d05d0 2023-12-01T01:54:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.445+0000 7f4c1ad9d640 1 --2- 172.21.15.115:0/3796374716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0d05d0 0x7f4c1c0d0a20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009d90 tx=0x7f4c1402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd145e649469e33 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.445+0000 7f4c19d9b640 1 -- 172.21.15.115:0/3796374716 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1402cd00 con 0x7f4c1c0d05d0 2023-12-01T01:54:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.445+0000 7f4c19d9b640 1 -- 172.21.15.115:0/3796374716 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1402ce60 con 0x7f4c1c0d05d0 2023-12-01T01:54:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.445+0000 7f4c19d9b640 1 -- 172.21.15.115:0/3796374716 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14034660 con 0x7f4c1c0d05d0 2023-12-01T01:54:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.446+0000 7f4c1b59e640 1 -- 172.21.15.115:0/3796374716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0d05d0 msgr2=0x7f4c1c0d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.446+0000 7f4c1b59e640 1 --2- 172.21.15.115:0/3796374716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0d05d0 0x7f4c1c0d0a20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009d90 tx=0x7f4c1402bf50 comp rx=0 tx=0).stop 2023-12-01T01:54:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.446+0000 7f4c1b59e640 1 -- 172.21.15.115:0/3796374716 shutdown_connections 2023-12-01T01:54:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.446+0000 7f4c1b59e640 1 --2- 172.21.15.115:0/3796374716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0d05d0 0x7f4c1c0d0a20 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.446+0000 7f4c1b59e640 1 -- 172.21.15.115:0/3796374716 >> 172.21.15.115:0/3796374716 conn(0x7f4c1c06a0e0 msgr2=0x7f4c1c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.446+0000 7f4c1b59e640 1 -- 172.21.15.115:0/3796374716 shutdown_connections 2023-12-01T01:54:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.446+0000 7f4c1b59e640 1 -- 172.21.15.115:0/3796374716 wait complete. 2023-12-01T01:54:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.448+0000 7f4c1b59e640 1 Processor -- start 2023-12-01T01:54:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.448+0000 7f4c1b59e640 1 -- start start 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1b59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c1c0d05d0 0x7f4c1c1f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1b59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c1c15d7b0 0x7f4c1c0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1b59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15db90 0x7f4c1c15b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1b59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0b5690 con 0x7f4c1c15db90 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1b59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c15c660 con 0x7f4c1c15d7b0 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1b59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0c8c50 con 0x7f4c1c0d05d0 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15db90 0x7f4c1c15b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15db90 0x7f4c1c15b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49232/0 (socket says 172.21.15.115:49232) 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1ad9d640 1 -- 172.21.15.115:0/787208685 learned_addr learned my addr 172.21.15.115:0/787208685 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c13fff640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c1c15d7b0 0x7f4c1c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1a59c640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c1c0d05d0 0x7f4c1c1f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1ad9d640 1 -- 172.21.15.115:0/787208685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c1c0d05d0 msgr2=0x7f4c1c1f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.449+0000 7f4c1ad9d640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c1c0d05d0 0x7f4c1c1f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.450+0000 7f4c1ad9d640 1 -- 172.21.15.115:0/787208685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c1c15d7b0 msgr2=0x7f4c1c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.450+0000 7f4c1ad9d640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c1c15d7b0 0x7f4c1c0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.450+0000 7f4c1ad9d640 1 -- 172.21.15.115:0/787208685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c14009a70 con 0x7f4c1c15db90 2023-12-01T01:54:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.450+0000 7f4c1ad9d640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15db90 0x7f4c1c15b570 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009d90 tx=0x7f4c14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.450+0000 7f4c12ffd640 1 -- 172.21.15.115:0/787208685 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1403d600 con 0x7f4c1c15db90 2023-12-01T01:54:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.450+0000 7f4c12ffd640 1 -- 172.21.15.115:0/787208685 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1403d780 con 0x7f4c1c15db90 2023-12-01T01:54:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.450+0000 7f4c12ffd640 1 -- 172.21.15.115:0/787208685 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1403db30 con 0x7f4c1c15db90 2023-12-01T01:54:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.450+0000 7f4c1b59e640 1 -- 172.21.15.115:0/787208685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c15bdc0 con 0x7f4c1c15db90 2023-12-01T01:54:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.450+0000 7f4c1b59e640 1 -- 172.21.15.115:0/787208685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c15c380 con 0x7f4c1c15db90 2023-12-01T01:54:40.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.452+0000 7f4c1b59e640 1 -- 172.21.15.115:0/787208685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c0673a0 con 0x7f4c1c15db90 2023-12-01T01:54:40.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.453+0000 7f4c12ffd640 1 -- 172.21.15.115:0/787208685 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4c1403b040 con 0x7f4c1c15db90 2023-12-01T01:54:40.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.454+0000 7f4c12ffd640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bfc06a550 0x7f4bfc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:40.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.454+0000 7f4c12ffd640 1 -- 172.21.15.115:0/787208685 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c140aa8d0 con 0x7f4c1c15db90 2023-12-01T01:54:40.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.454+0000 7f4c1a59c640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bfc06a550 0x7f4bfc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.455+0000 7f4c1a59c640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bfc06a550 0x7f4bfc06ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c004ea0 tx=0x7f4c0c009040 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:40.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.458+0000 7f4c12ffd640 1 -- 172.21.15.115:0/787208685 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4c14075e40 con 0x7f4c1c15db90 2023-12-01T01:54:40.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.690+0000 7f4c1b59e640 1 -- 172.21.15.115:0/787208685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4c1c0d3100 con 0x7f4c1c15db90 2023-12-01T01:54:40.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.690+0000 7f4c12ffd640 1 -- 172.21.15.115:0/787208685 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4c1404d060 con 0x7f4c1c15db90 2023-12-01T01:54:40.692 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:54:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 -- 172.21.15.115:0/787208685 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bfc06a550 msgr2=0x7f4bfc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bfc06a550 0x7f4bfc06ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c004ea0 tx=0x7f4c0c009040 comp rx=0 tx=0).stop 2023-12-01T01:54:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 -- 172.21.15.115:0/787208685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15db90 msgr2=0x7f4c1c15b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:40.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15db90 0x7f4c1c15b570 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009d90 tx=0x7f4c14004a40 comp rx=0 tx=0).stop 2023-12-01T01:54:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 -- 172.21.15.115:0/787208685 shutdown_connections 2023-12-01T01:54:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bfc06a550 0x7f4bfc06ca10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15db90 0x7f4c1c15b570 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c1c15d7b0 0x7f4c1c0c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 --2- 172.21.15.115:0/787208685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c1c0d05d0 0x7f4c1c1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 -- 172.21.15.115:0/787208685 >> 172.21.15.115:0/787208685 conn(0x7f4c1c06a0e0 msgr2=0x7f4c1c15f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 -- 172.21.15.115:0/787208685 shutdown_connections 2023-12-01T01:54:40.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:40.693+0000 7f4c1b59e640 1 -- 172.21.15.115:0/787208685 wait complete. 2023-12-01T01:54:40.713 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T01:54:40.713 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:54:40.714 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T01:54:40.714 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T01:54:40.714 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:54:40.714 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T01:54:40.714 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T01:54:40.714 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:54:40.714 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:54:40.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:40.714 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T01:54:40.770 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:54:40.770 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:54:40.770 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:54:40.770 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:54:40.770 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:54:40.771 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 41276sec preferred_lft 41276sec 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:54:40.772 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T01:54:40.773 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T01:54:40.773 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:54:40.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:40.773 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:54:40.863 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T01:54:40.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:40.868 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T01:54:40.920 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T01:54:40.920 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:54:40.920 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:54:40.921 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:54:40.921 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:54:40.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:40.921 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:54:40.921 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:54:40.921 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:54:40.921 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:54:40.921 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:54:40.921 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:54:41.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:41.109 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:54:41.159 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T01:54:41.159 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T01:54:41.159 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:54:41.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:41.159 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T01:54:41.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:54:41.246 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T01:54:41.304 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:54:41.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:54:41.304 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:54:41.356 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:54:41.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:41.357 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:54:41.496 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:54:41.496 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:54:41.496 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:54:41.517 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:54:41.517 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:54:41.517 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:54:41.517 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:54:41.517 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:54:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:54:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:54:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:54:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:54:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:54:41.544 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:54:41.545 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:54:41.545 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:54:41.549 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T01:54:41.549 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:54:41.549 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:54:41.549 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:54:41.549 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:54:41.553 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:54:41.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:41.553 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:54:41.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:41.634 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:54:41.698 INFO:teuthology.orchestra.run.smithi115.stdout:client6154 2023-12-01T01:54:41.698 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:54:41.698 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:54:41.748 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T01:54:42.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.011+0000 7fa3b49fa640 1 Processor -- start 2023-12-01T01:54:42.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.011+0000 7fa3b49fa640 1 -- start start 2023-12-01T01:54:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.012+0000 7fa3b49fa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa3b006b190 0x7fa3b00709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3b49fa640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa3b00702a0 0x7fa3b0070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3b49fa640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b00707a0 con 0x7fa3b0159d90 2023-12-01T01:54:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3b49fa640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b00c9380 con 0x7fa3b00702a0 2023-12-01T01:54:42.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3b49fa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b00b2f70 con 0x7fa3b006b190 2023-12-01T01:54:42.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3af59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa3b00702a0 0x7fa3b0070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:42.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3af59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa3b00702a0 0x7fa3b0070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49248/0 (socket says 172.21.15.115:49248) 2023-12-01T01:54:42.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3af59e640 1 -- 172.21.15.115:0/987853630 learned_addr learned my addr 172.21.15.115:0/987853630 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:42.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3af59e640 1 -- 172.21.15.115:0/987853630 >> v2:172.21.15.142:3300/0 conn(0x7fa3b006b190 msgr2=0x7fa3b00709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:42.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3af59e640 1 --2- 172.21.15.115:0/987853630 >> v2:172.21.15.142:3300/0 conn(0x7fa3b006b190 0x7fa3b00709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3af59e640 1 -- 172.21.15.115:0/987853630 >> v1:172.21.15.142:6789/0 conn(0x7fa3b0159d90 legacy=0x7fa3b00c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:42.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.013+0000 7fa3af59e640 1 -- 172.21.15.115:0/987853630 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b015e2e0 con 0x7fa3b00702a0 2023-12-01T01:54:42.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.014+0000 7fa3af59e640 1 --2- 172.21.15.115:0/987853630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b00702a0 0x7fa3b0070ee0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00b41f0 tx=0x7fa3a802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb6d564100fa1e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:42.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.015+0000 7fa3ae59c640 1 -- 172.21.15.115:0/987853630 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a802c9f0 con 0x7fa3b00702a0 2023-12-01T01:54:42.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.015+0000 7fa3ae59c640 1 -- 172.21.15.115:0/987853630 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a802cb50 con 0x7fa3b00702a0 2023-12-01T01:54:42.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.015+0000 7fa3ae59c640 1 -- 172.21.15.115:0/987853630 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80343f0 con 0x7fa3b00702a0 2023-12-01T01:54:42.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.015+0000 7fa3b49fa640 1 -- 172.21.15.115:0/987853630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b00702a0 msgr2=0x7fa3b0070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:42.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.015+0000 7fa3b49fa640 1 --2- 172.21.15.115:0/987853630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b00702a0 0x7fa3b0070ee0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00b41f0 tx=0x7fa3a802be50 comp rx=0 tx=0).stop 2023-12-01T01:54:42.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.015+0000 7fa3b49fa640 1 -- 172.21.15.115:0/987853630 shutdown_connections 2023-12-01T01:54:42.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.015+0000 7fa3b49fa640 1 --2- 172.21.15.115:0/987853630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b00702a0 0x7fa3b0070ee0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.015+0000 7fa3b49fa640 1 --2- 172.21.15.115:0/987853630 >> v2:172.21.15.142:3300/0 conn(0x7fa3b006b190 0x7fa3b00709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.015+0000 7fa3b49fa640 1 -- 172.21.15.115:0/987853630 >> 172.21.15.115:0/987853630 conn(0x7fa3b00ce620 msgr2=0x7fa3b00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:42.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.016+0000 7fa3b49fa640 1 -- 172.21.15.115:0/987853630 shutdown_connections 2023-12-01T01:54:42.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.016+0000 7fa3b49fa640 1 -- 172.21.15.115:0/987853630 wait complete. 2023-12-01T01:54:42.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.017+0000 7fa3b49fa640 1 Processor -- start 2023-12-01T01:54:42.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.017+0000 7fa3b49fa640 1 -- start start 2023-12-01T01:54:42.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.018+0000 7fa3b49fa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b006b190 0x7fa3b00cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.018+0000 7fa3b49fa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3b00702a0 0x7fa3b00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.018+0000 7fa3b49fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3b0159d90 0x7fa3b014e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.018+0000 7fa3b49fa640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00c21e0 con 0x7fa3b006b190 2023-12-01T01:54:42.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.018+0000 7fa3b49fa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00cae40 con 0x7fa3b00702a0 2023-12-01T01:54:42.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.018+0000 7fa3b49fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00cb090 con 0x7fa3b0159d90 2023-12-01T01:54:42.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.019+0000 7fa3aed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b006b190 0x7fa3b00cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:42.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.019+0000 7fa3aed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b006b190 0x7fa3b00cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49262/0 (socket says 172.21.15.115:49262) 2023-12-01T01:54:42.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.019+0000 7fa3aed9d640 1 -- 172.21.15.115:0/4217533016 learned_addr learned my addr 172.21.15.115:0/4217533016 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:42.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.019+0000 7fa3aed9d640 1 -- 172.21.15.115:0/4217533016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3b0159d90 msgr2=0x7fa3b014e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:42.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.019+0000 7fa3aed9d640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3b0159d90 0x7fa3b014e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.019+0000 7fa3aed9d640 1 -- 172.21.15.115:0/4217533016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3b00702a0 msgr2=0x7fa3b00ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:42.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.019+0000 7fa3aed9d640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3b00702a0 0x7fa3b00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.019+0000 7fa3aed9d640 1 -- 172.21.15.115:0/4217533016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a8009b30 con 0x7fa3b006b190 2023-12-01T01:54:42.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.019+0000 7fa3aed9d640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b006b190 0x7fa3b00cbe50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000c430 tx=0x7fa3a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:42.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.020+0000 7fa3a77fe640 1 -- 172.21.15.115:0/4217533016 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0011790 con 0x7fa3b006b190 2023-12-01T01:54:42.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.020+0000 7fa3a77fe640 1 -- 172.21.15.115:0/4217533016 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a00118f0 con 0x7fa3b006b190 2023-12-01T01:54:42.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.020+0000 7fa3b49fa640 1 -- 172.21.15.115:0/4217533016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b00cb370 con 0x7fa3b006b190 2023-12-01T01:54:42.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.020+0000 7fa3a77fe640 1 -- 172.21.15.115:0/4217533016 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0011ce0 con 0x7fa3b006b190 2023-12-01T01:54:42.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.020+0000 7fa3b49fa640 1 -- 172.21.15.115:0/4217533016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b014e5b0 con 0x7fa3b006b190 2023-12-01T01:54:42.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.021+0000 7fa3b49fa640 1 -- 172.21.15.115:0/4217533016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b014e770 con 0x7fa3b006b190 2023-12-01T01:54:42.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.022+0000 7fa3a77fe640 1 -- 172.21.15.115:0/4217533016 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa3a001c460 con 0x7fa3b006b190 2023-12-01T01:54:42.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.023+0000 7fa3a77fe640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa39006a550 0x7fa39006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.023+0000 7fa3a77fe640 1 -- 172.21.15.115:0/4217533016 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3a0088a30 con 0x7fa3b006b190 2023-12-01T01:54:42.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.023+0000 7fa3a7fff640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa39006a550 0x7fa39006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:42.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.024+0000 7fa3a7fff640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa39006a550 0x7fa39006ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00b0ad0 tx=0x7fa398006bf0 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:42.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.028+0000 7fa3a77fe640 1 -- 172.21.15.115:0/4217533016 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa3a0053f10 con 0x7fa3b006b190 2023-12-01T01:54:42.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.243+0000 7fa3b49fa640 1 -- 172.21.15.115:0/4217533016 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3b0067850 con 0x7fa39006a550 2023-12-01T01:54:42.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.247+0000 7fa3a77fe640 1 -- 172.21.15.115:0/4217533016 <== mgr.7767 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa3b0067850 con 0x7fa39006a550 2023-12-01T01:54:42.248 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:54:42.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:54:42.249 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T01:54:42.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:54:42.249 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.249+0000 7fa3b49fa640 1 -- 172.21.15.115:0/4217533016 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa39006a550 msgr2=0x7fa39006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.249+0000 7fa3b49fa640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa39006a550 0x7fa39006ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00b0ad0 tx=0x7fa398006bf0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.249+0000 7fa3b49fa640 1 -- 172.21.15.115:0/4217533016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b006b190 msgr2=0x7fa3b00cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.249+0000 7fa3b49fa640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b006b190 0x7fa3b00cbe50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000c430 tx=0x7fa3a000c8f0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.249+0000 7fa3b49fa640 1 -- 172.21.15.115:0/4217533016 shutdown_connections 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.249+0000 7fa3b49fa640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa39006a550 0x7fa39006ca10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.249+0000 7fa3b49fa640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3b0159d90 0x7fa3b014e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.249+0000 7fa3b49fa640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3b00702a0 0x7fa3b00ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.249+0000 7fa3b49fa640 1 --2- 172.21.15.115:0/4217533016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3b006b190 0x7fa3b00cbe50 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.249+0000 7fa3b49fa640 1 -- 172.21.15.115:0/4217533016 >> 172.21.15.115:0/4217533016 conn(0x7fa3b00ce620 msgr2=0x7fa3b01556b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.250+0000 7fa3b49fa640 1 -- 172.21.15.115:0/4217533016 shutdown_connections 2023-12-01T01:54:42.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.250+0000 7fa3b49fa640 1 -- 172.21.15.115:0/4217533016 wait complete. 2023-12-01T01:54:42.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T01:54:42.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.484+0000 7fbb811b1640 1 Processor -- start 2023-12-01T01:54:42.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.484+0000 7fbb811b1640 1 -- start start 2023-12-01T01:54:42.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.486+0000 7fbb811b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c160200 0x7fbb7c0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.486+0000 7fbb811b1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbb7c0b0f80 0x7fbb7c06b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.486+0000 7fbb811b1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c0cb290 con 0x7fbb7c06b7a0 2023-12-01T01:54:42.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.486+0000 7fbb811b1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b35f0 con 0x7fbb7c0b0f80 2023-12-01T01:54:42.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.486+0000 7fbb811b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b0ae0 con 0x7fbb7c160200 2023-12-01T01:54:42.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.486+0000 7fbb7b59e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fbb7c06b7a0 0x7fbb7c0cbd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48324/0 (socket says 172.21.15.115:48324) 2023-12-01T01:54:42.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.486+0000 7fbb7b59e640 1 -- 172.21.15.115:0/262078957 learned_addr learned my addr 172.21.15.115:0/262078957 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.486+0000 7fbb7ad9d640 1 --2- 172.21.15.115:0/262078957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c160200 0x7fbb7c0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb7a59c640 1 --2- 172.21.15.115:0/262078957 >> v2:172.21.15.142:3300/0 conn(0x7fbb7c0b0f80 0x7fbb7c06b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856838959 0 0) 0x7fbb7c0cb290 con 0x7fbb7c06b7a0 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb64003520 con 0x7fbb7c06b7a0 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1786306352 0 0) 0x7fbb64003520 con 0x7fbb7c06b7a0 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c160200 msgr2=0x7fbb7c0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 --2- 172.21.15.115:0/262078957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c160200 0x7fbb7c0b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 >> v2:172.21.15.142:3300/0 conn(0x7fbb7c0b0f80 msgr2=0x7fbb7c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 --2- 172.21.15.115:0/262078957 >> v2:172.21.15.142:3300/0 conn(0x7fbb7c0b0f80 0x7fbb7c06b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0b3310 con 0x7fbb7c06b7a0 2023-12-01T01:54:42.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fbb740019f0 con 0x7fbb7c06b7a0 2023-12-01T01:54:42.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 >> v1:172.21.15.115:6789/0 conn(0x7fbb7c06b7a0 legacy=0x7fbb7c0cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:42.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 --2- 172.21.15.115:0/262078957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 0x7fbb64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 --2- 172.21.15.115:0/262078957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb640047c0 0x7fbb64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 --2- 172.21.15.115:0/262078957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c160200 0x7fbb64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b35f0 con 0x7fbb64003970 2023-12-01T01:54:42.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b0ae0 con 0x7fbb640047c0 2023-12-01T01:54:42.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6400b980 con 0x7fbb7c160200 2023-12-01T01:54:42.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.487+0000 7fbb7ad9d640 1 --2- 172.21.15.115:0/262078957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 0x7fbb64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:42.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.488+0000 7fbb7ad9d640 1 -- 172.21.15.115:0/262078957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c160200 msgr2=0x7fbb64005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:42.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.488+0000 7fbb7ad9d640 1 --2- 172.21.15.115:0/262078957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c160200 0x7fbb64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.488+0000 7fbb7ad9d640 1 -- 172.21.15.115:0/262078957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb640047c0 msgr2=0x7fbb64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:42.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.488+0000 7fbb7ad9d640 1 --2- 172.21.15.115:0/262078957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb640047c0 0x7fbb64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.488+0000 7fbb7ad9d640 1 -- 172.21.15.115:0/262078957 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb64003520 con 0x7fbb64003970 2023-12-01T01:54:42.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.489+0000 7fbb7ad9d640 1 --2- 172.21.15.115:0/262078957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 0x7fbb64003d90 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009ac0 tx=0x7fbb640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e048cfc1175f88fc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:42.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.489+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c02c6d0 con 0x7fbb64003970 2023-12-01T01:54:42.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.489+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c02c830 con 0x7fbb64003970 2023-12-01T01:54:42.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.489+0000 7fbb79d9b640 1 -- 172.21.15.115:0/262078957 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c02cbc0 con 0x7fbb64003970 2023-12-01T01:54:42.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.490+0000 7fbb811b1640 1 -- 172.21.15.115:0/262078957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 msgr2=0x7fbb64003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:42.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.490+0000 7fbb811b1640 1 --2- 172.21.15.115:0/262078957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 0x7fbb64003d90 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009ac0 tx=0x7fbb640057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.490+0000 7fbb7b59e640 1 -- 172.21.15.115:0/262078957 reap_dead start 2023-12-01T01:54:42.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.490+0000 7fbb811b1640 1 -- 172.21.15.115:0/262078957 shutdown_connections 2023-12-01T01:54:42.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.490+0000 7fbb811b1640 1 -- 172.21.15.115:0/262078957 >> 172.21.15.115:0/262078957 conn(0x7fbb7c065960 msgr2=0x7fbb7c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:42.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.490+0000 7fbb811b1640 1 -- 172.21.15.115:0/262078957 shutdown_connections 2023-12-01T01:54:42.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.490+0000 7fbb811b1640 1 -- 172.21.15.115:0/262078957 wait complete. 2023-12-01T01:54:42.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.492+0000 7fbb811b1640 1 Processor -- start 2023-12-01T01:54:42.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.492+0000 7fbb811b1640 1 -- start start 2023-12-01T01:54:42.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.493+0000 7fbb811b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 0x7fbb7c0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.493+0000 7fbb811b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c0c8530 0x7fbb7c0c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.493+0000 7fbb811b1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c156aa0 0x7fbb7c156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.493+0000 7fbb811b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c155d70 con 0x7fbb64003970 2023-12-01T01:54:42.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.493+0000 7fbb811b1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0c92f0 con 0x7fbb7c156aa0 2023-12-01T01:54:42.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.493+0000 7fbb811b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c155fc0 con 0x7fbb7c0c8530 2023-12-01T01:54:42.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.493+0000 7fbb7ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 0x7fbb7c0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:42.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.493+0000 7fbb7ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 0x7fbb7c0c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49278/0 (socket says 172.21.15.115:49278) 2023-12-01T01:54:42.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.493+0000 7fbb7ad9d640 1 -- 172.21.15.115:0/2123685099 learned_addr learned my addr 172.21.15.115:0/2123685099 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:42.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.494+0000 7fbb7ad9d640 1 -- 172.21.15.115:0/2123685099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c0c8530 msgr2=0x7fbb7c0c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:42.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.494+0000 7fbb7ad9d640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c0c8530 0x7fbb7c0c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.494+0000 7fbb7ad9d640 1 -- 172.21.15.115:0/2123685099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c156aa0 msgr2=0x7fbb7c156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:42.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.494+0000 7fbb7ad9d640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c156aa0 0x7fbb7c156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:42.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.494+0000 7fbb7ad9d640 1 -- 172.21.15.115:0/2123685099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c1562a0 con 0x7fbb64003970 2023-12-01T01:54:42.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.494+0000 7fbb7ad9d640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 0x7fbb7c0c9da0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c02c2d0 tx=0x7fbb6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:42.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.494+0000 7fbb637fe640 1 -- 172.21.15.115:0/2123685099 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c003810 con 0x7fbb64003970 2023-12-01T01:54:42.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.495+0000 7fbb637fe640 1 -- 172.21.15.115:0/2123685099 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c003970 con 0x7fbb64003970 2023-12-01T01:54:42.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.495+0000 7fbb637fe640 1 -- 172.21.15.115:0/2123685099 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c003ce0 con 0x7fbb64003970 2023-12-01T01:54:42.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.495+0000 7fbb811b1640 1 -- 172.21.15.115:0/2123685099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c156580 con 0x7fbb64003970 2023-12-01T01:54:42.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.495+0000 7fbb811b1640 1 -- 172.21.15.115:0/2123685099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c0ca2e0 con 0x7fbb64003970 2023-12-01T01:54:42.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.496+0000 7fbb811b1640 1 -- 172.21.15.115:0/2123685099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c04e890 con 0x7fbb64003970 2023-12-01T01:54:42.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.497+0000 7fbb637fe640 1 -- 172.21.15.115:0/2123685099 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 61) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fbb6c040660 con 0x7fbb64003970 2023-12-01T01:54:42.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.498+0000 7fbb637fe640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb6806a580 0x7fbb6806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:42.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.498+0000 7fbb637fe640 1 -- 172.21.15.115:0/2123685099 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb6c0aaa80 con 0x7fbb64003970 2023-12-01T01:54:42.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.498+0000 7fbb7a59c640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb6806a580 0x7fbb6806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:42.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.499+0000 7fbb7a59c640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb6806a580 0x7fbb6806ca40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb700093d0 tx=0x7fbb70009300 comp rx=0 tx=0).ready entity=mgr.7767 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:42.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.502+0000 7fbb637fe640 1 -- 172.21.15.115:0/2123685099 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbb6c075ff0 con 0x7fbb64003970 2023-12-01T01:54:42.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:42.742+0000 7fbb811b1640 1 -- 172.21.15.115:0/2123685099 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fbb7c0cf5d0 con 0x7fbb64003970 2023-12-01T01:54:43.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.604+0000 7fbb637fe640 1 -- 172.21.15.115:0/2123685099 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 62) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fbb6c0763c0 con 0x7fbb64003970 2023-12-01T01:54:43.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.609+0000 7fbb637fe640 1 -- 172.21.15.115:0/2123685099 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v62) v1 ==== 92+0+0 (secure 0 0 0) 0x7fbb6c07d480 con 0x7fbb64003970 2023-12-01T01:54:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 -- 172.21.15.115:0/2123685099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb6806a580 msgr2=0x7fbb6806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb6806a580 0x7fbb6806ca40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb700093d0 tx=0x7fbb70009300 comp rx=0 tx=0).stop 2023-12-01T01:54:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 -- 172.21.15.115:0/2123685099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 msgr2=0x7fbb7c0c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 0x7fbb7c0c9da0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c02c2d0 tx=0x7fbb6c004730 comp rx=0 tx=0).stop 2023-12-01T01:54:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 -- 172.21.15.115:0/2123685099 shutdown_connections 2023-12-01T01:54:43.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fbb6806a580 0x7fbb6806ca40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb7c0c8530 0x7fbb7c0c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c156aa0 0x7fbb7c156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 --2- 172.21.15.115:0/2123685099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb64003970 0x7fbb7c0c9da0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 -- 172.21.15.115:0/2123685099 >> 172.21.15.115:0/2123685099 conn(0x7fbb7c065960 msgr2=0x7fbb7c0cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 -- 172.21.15.115:0/2123685099 shutdown_connections 2023-12-01T01:54:43.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.612+0000 7fbb811b1640 1 -- 172.21.15.115:0/2123685099 wait complete. 2023-12-01T01:54:43.635 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T01:54:43.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.839+0000 7f68bc8b6640 1 Processor -- start 2023-12-01T01:54:43.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.839+0000 7f68bc8b6640 1 -- start start 2023-12-01T01:54:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.840+0000 7f68bc8b6640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f68b8153c30 0x7f68b80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.840+0000 7f68bc8b6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f68b8154010 0x7f68b80cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.840+0000 7f68bc8b6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b80cdcd0 con 0x7f68b80cf9b0 2023-12-01T01:54:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.840+0000 7f68bc8b6640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b80cfe20 con 0x7f68b8153c30 2023-12-01T01:54:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.840+0000 7f68bc8b6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b8161830 con 0x7f68b8154010 2023-12-01T01:54:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.841+0000 7f68b77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f68b8153c30 0x7f68b80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.841+0000 7f68b77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f68b8153c30 0x7f68b80ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49300/0 (socket says 172.21.15.115:49300) 2023-12-01T01:54:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.841+0000 7f68b77fe640 1 -- 172.21.15.115:0/67454786 learned_addr learned my addr 172.21.15.115:0/67454786 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:43.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.841+0000 7f68b77fe640 1 -- 172.21.15.115:0/67454786 >> v2:172.21.15.142:3300/0 conn(0x7f68b8154010 msgr2=0x7f68b80cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:43.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.841+0000 7f68b77fe640 1 --2- 172.21.15.115:0/67454786 >> v2:172.21.15.142:3300/0 conn(0x7f68b8154010 0x7f68b80cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:43.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.841+0000 7f68b77fe640 1 -- 172.21.15.115:0/67454786 >> v1:172.21.15.142:6789/0 conn(0x7f68b80cf9b0 legacy=0x7f68b8160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:43.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.841+0000 7f68b77fe640 1 -- 172.21.15.115:0/67454786 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b80cd3a0 con 0x7f68b8153c30 2023-12-01T01:54:43.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.842+0000 7f68b77fe640 1 --2- 172.21.15.115:0/67454786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8153c30 0x7f68b80ce160 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f68b80d3520 tx=0x7f68a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92c7e3ccb787fc4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:43.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.842+0000 7f68b6ffd640 1 -- 172.21.15.115:0/67454786 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8031870 con 0x7f68b8153c30 2023-12-01T01:54:43.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.842+0000 7f68b6ffd640 1 -- 172.21.15.115:0/67454786 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a80319d0 con 0x7f68b8153c30 2023-12-01T01:54:43.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.843+0000 7f68b6ffd640 1 -- 172.21.15.115:0/67454786 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8031d60 con 0x7f68b8153c30 2023-12-01T01:54:43.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.843+0000 7f68bc8b6640 1 -- 172.21.15.115:0/67454786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8153c30 msgr2=0x7f68b80ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:43.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.843+0000 7f68bc8b6640 1 --2- 172.21.15.115:0/67454786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8153c30 0x7f68b80ce160 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f68b80d3520 tx=0x7f68a802b910 comp rx=0 tx=0).stop 2023-12-01T01:54:43.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.843+0000 7f68bc8b6640 1 -- 172.21.15.115:0/67454786 shutdown_connections 2023-12-01T01:54:43.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.843+0000 7f68bc8b6640 1 --2- 172.21.15.115:0/67454786 >> v2:172.21.15.142:3300/0 conn(0x7f68b8154010 0x7f68b80cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:43.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.843+0000 7f68bc8b6640 1 --2- 172.21.15.115:0/67454786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8153c30 0x7f68b80ce160 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:43.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.843+0000 7f68bc8b6640 1 -- 172.21.15.115:0/67454786 >> 172.21.15.115:0/67454786 conn(0x7f68b806a0b0 msgr2=0x7f68b806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:43.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.843+0000 7f68bc8b6640 1 -- 172.21.15.115:0/67454786 shutdown_connections 2023-12-01T01:54:43.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.843+0000 7f68bc8b6640 1 -- 172.21.15.115:0/67454786 wait complete. 2023-12-01T01:54:43.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.845+0000 7f68bc8b6640 1 Processor -- start 2023-12-01T01:54:43.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.845+0000 7f68bc8b6640 1 -- start start 2023-12-01T01:54:43.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.846+0000 7f68bc8b6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68b80cf9b0 0x7f68b80c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:43.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.846+0000 7f68bc8b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68b8153c30 0x7f68b81f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.846+0000 7f68bc8b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8154010 0x7f68b80c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.846+0000 7f68bc8b6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b8158680 con 0x7f68b8154010 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.846+0000 7f68bc8b6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b8157ad0 con 0x7f68b80cf9b0 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.846+0000 7f68bc8b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b81f2750 con 0x7f68b8153c30 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.846+0000 7f68b7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8154010 0x7f68b80c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.846+0000 7f68b7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8154010 0x7f68b80c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49314/0 (socket says 172.21.15.115:49314) 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.846+0000 7f68b7fff640 1 -- 172.21.15.115:0/1617213668 learned_addr learned my addr 172.21.15.115:0/1617213668 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68b77fe640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68b80cf9b0 0x7f68b80c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68affff640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68b8153c30 0x7f68b81f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68b7fff640 1 -- 172.21.15.115:0/1617213668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68b8153c30 msgr2=0x7f68b81f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68b7fff640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68b8153c30 0x7f68b81f2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:43.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68b7fff640 1 -- 172.21.15.115:0/1617213668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68b80cf9b0 msgr2=0x7f68b80c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:43.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68b7fff640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68b80cf9b0 0x7f68b80c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:43.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68b7fff640 1 -- 172.21.15.115:0/1617213668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a80094d0 con 0x7f68b8154010 2023-12-01T01:54:43.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68b7fff640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8154010 0x7f68b80c6f60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c780 tx=0x7f68b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:43.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68b4ff9640 1 -- 172.21.15.115:0/1617213668 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0003d80 con 0x7f68b8154010 2023-12-01T01:54:43.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68b4ff9640 1 -- 172.21.15.115:0/1617213668 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b000ce90 con 0x7f68b8154010 2023-12-01T01:54:43.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.847+0000 7f68bc8b6640 1 -- 172.21.15.115:0/1617213668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b80c74a0 con 0x7f68b8154010 2023-12-01T01:54:43.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.848+0000 7f68b4ff9640 1 -- 172.21.15.115:0/1617213668 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b00052b0 con 0x7f68b8154010 2023-12-01T01:54:43.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.848+0000 7f68bc8b6640 1 -- 172.21.15.115:0/1617213668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b8157fc0 con 0x7f68b8154010 2023-12-01T01:54:43.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.849+0000 7f68bc8b6640 1 -- 172.21.15.115:0/1617213668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b81587c0 con 0x7f68b8154010 2023-12-01T01:54:43.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.850+0000 7f68b4ff9640 1 -- 172.21.15.115:0/1617213668 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 62) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f68b0005410 con 0x7f68b8154010 2023-12-01T01:54:43.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.850+0000 7f68b4ff9640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f689806a5a0 0x7f689806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:43.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.851+0000 7f68b77fe640 1 -- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f689806a5a0 msgr2=0x7f689806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:43.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.851+0000 7f68b4ff9640 1 -- 172.21.15.115:0/1617213668 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68b0088330 con 0x7f68b8154010 2023-12-01T01:54:43.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.851+0000 7f68b77fe640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f689806a5a0 0x7f689806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:43.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:43.855+0000 7f68b4ff9640 1 -- 172.21.15.115:0/1617213668 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f68b81587c0 con 0x7f68b8154010 2023-12-01T01:54:44.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.042+0000 7f68bc8b6640 1 -- 172.21.15.115:0/1617213668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f68b81587c0 con 0x7f68b8154010 2023-12-01T01:54:44.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.050+0000 7f68b4ff9640 1 -- 172.21.15.115:0/1617213668 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f68b81587c0 con 0x7f68b8154010 2023-12-01T01:54:44.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.052+0000 7f68b77fe640 1 -- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f689806a5a0 msgr2=0x7f689806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:44.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.052+0000 7f68b77fe640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f689806a5a0 0x7f689806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:54:44.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.053+0000 7f68bc8b6640 1 -- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f689806a5a0 msgr2=0x7f689806ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:44.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.053+0000 7f68bc8b6640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f689806a5a0 0x7f689806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.053+0000 7f68bc8b6640 1 -- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8154010 msgr2=0x7f68b80c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:44.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.053+0000 7f68bc8b6640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8154010 0x7f68b80c6f60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c780 tx=0x7f68b000cc40 comp rx=0 tx=0).stop 2023-12-01T01:54:44.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.054+0000 7f68bc8b6640 1 -- 172.21.15.115:0/1617213668 shutdown_connections 2023-12-01T01:54:44.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.054+0000 7f68bc8b6640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f689806a5a0 0x7f689806ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.054+0000 7f68bc8b6640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68b8153c30 0x7f68b81f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.054+0000 7f68bc8b6640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68b8154010 0x7f68b80c6f60 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.054+0000 7f68bc8b6640 1 --2- 172.21.15.115:0/1617213668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68b80cf9b0 0x7f68b80c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.054+0000 7f68bc8b6640 1 -- 172.21.15.115:0/1617213668 >> 172.21.15.115:0/1617213668 conn(0x7f68b806a0b0 msgr2=0x7f68b815e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:44.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.054+0000 7f68bc8b6640 1 -- 172.21.15.115:0/1617213668 shutdown_connections 2023-12-01T01:54:44.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.054+0000 7f68bc8b6640 1 -- 172.21.15.115:0/1617213668 wait complete. 2023-12-01T01:54:44.080 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T01:54:44.143 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:44.142+0000 7efddae32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:54:44.168 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:44.166+0000 7f0391e41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:54:44.199 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:44.197+0000 7efddae32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:54:44.310 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:44.224+0000 7f0391e41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:54:44.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.284+0000 7fc4ee009640 1 Processor -- start 2023-12-01T01:54:44.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.284+0000 7fc4ee009640 1 -- start start 2023-12-01T01:54:44.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.285+0000 7fc4ee009640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc4e815cf70 0x7fc4e80b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.285+0000 7fc4ee009640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d0260 0x7fc4e80b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.285+0000 7fc4ee009640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e80b7760 con 0x7fc4e80b18f0 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.285+0000 7fc4ee009640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e80b2170 con 0x7fc4e815cf70 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.285+0000 7fc4ee009640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80b8b70 con 0x7fc4e80d0260 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.286+0000 7fc4e6ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc4e80b18f0 0x7fc4e80cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48330/0 (socket says 172.21.15.115:48330) 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.286+0000 7fc4e6ffd640 1 -- 172.21.15.115:0/2519010783 learned_addr learned my addr 172.21.15.115:0/2519010783 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.286+0000 7fc4e77fe640 1 --2- 172.21.15.115:0/2519010783 >> v2:172.21.15.142:3300/0 conn(0x7fc4e815cf70 0x7fc4e80b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.286+0000 7fc4e7fff640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d0260 0x7fc4e80b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.286+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465640735 0 0) 0x7fc4e80b7760 con 0x7fc4e80b18f0 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.286+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4d0003520 con 0x7fc4e80b18f0 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1266258209 0 0) 0x7fc4d0003520 con 0x7fc4e80b18f0 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d0260 msgr2=0x7fc4e80b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d0260 0x7fc4e80b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 >> v2:172.21.15.142:3300/0 conn(0x7fc4e815cf70 msgr2=0x7fc4e80b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 --2- 172.21.15.115:0/2519010783 >> v2:172.21.15.142:3300/0 conn(0x7fc4e815cf70 0x7fc4e80b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80b6e30 con 0x7fc4e80b18f0 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc4dc001670 con 0x7fc4e80b18f0 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e77fe640 1 --2- 172.21.15.115:0/2519010783 >> v2:172.21.15.142:3300/0 conn(0x7fc4e815cf70 0x7fc4e80b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 >> v1:172.21.15.115:6789/0 conn(0x7fc4e80b18f0 legacy=0x7fc4e80cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d0260 0x7fc4d0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0003ef0 0x7fc4d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 0x7fc4d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80b2170 con 0x7fc4d0004d00 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80b8b70 con 0x7fc4d0003ef0 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d000b9b0 con 0x7fc4e80d0260 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e7fff640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 0x7fc4d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e77fe640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d0260 0x7fc4d0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e6ffd640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0003ef0 0x7fc4d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e7fff640 1 -- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d0260 msgr2=0x7fc4d0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e7fff640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e80d0260 0x7fc4d0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e7fff640 1 -- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0003ef0 msgr2=0x7fc4d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e7fff640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0003ef0 0x7fc4d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.287+0000 7fc4e7fff640 1 -- 172.21.15.115:0/2519010783 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d0003520 con 0x7fc4d0004d00 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.288+0000 7fc4e7fff640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 0x7fc4d0005180 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009fc0 tx=0x7fc4d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab064b9ca885ad5b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.289+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e002cc20 con 0x7fc4d0004d00 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.289+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e000f040 con 0x7fc4d0004d00 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.289+0000 7fc4e67fc640 1 -- 172.21.15.115:0/2519010783 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0034440 con 0x7fc4d0004d00 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.289+0000 7fc4ee009640 1 -- 172.21.15.115:0/2519010783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 msgr2=0x7fc4d0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.289+0000 7fc4ee009640 1 --2- 172.21.15.115:0/2519010783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 0x7fc4d0005180 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009fc0 tx=0x7fc4d00057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.289+0000 7fc4e7fff640 1 -- 172.21.15.115:0/2519010783 reap_dead start 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.289+0000 7fc4ee009640 1 -- 172.21.15.115:0/2519010783 shutdown_connections 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.289+0000 7fc4ee009640 1 -- 172.21.15.115:0/2519010783 >> 172.21.15.115:0/2519010783 conn(0x7fc4e806a0e0 msgr2=0x7fc4e806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:44.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.289+0000 7fc4ee009640 1 -- 172.21.15.115:0/2519010783 shutdown_connections 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.290+0000 7fc4ee009640 1 -- 172.21.15.115:0/2519010783 wait complete. 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.291+0000 7fc4ee009640 1 Processor -- start 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.291+0000 7fc4ee009640 1 -- start start 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.292+0000 7fc4ee009640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 0x7fc4e8153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.292+0000 7fc4ee009640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e8157730 0x7fc4e80ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.292+0000 7fc4ee009640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e8158d00 0x7fc4e81e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.292+0000 7fc4ee009640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80cb750 con 0x7fc4d0004d00 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.292+0000 7fc4ee009640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e81587d0 con 0x7fc4e8158d00 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.292+0000 7fc4ee009640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e8157bc0 con 0x7fc4e8157730 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.292+0000 7fc4e77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 0x7fc4e8153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.292+0000 7fc4e77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 0x7fc4e8153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49344/0 (socket says 172.21.15.115:49344) 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.292+0000 7fc4e77fe640 1 -- 172.21.15.115:0/3748271018 learned_addr learned my addr 172.21.15.115:0/3748271018 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4e7fff640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e8158d00 0x7fc4e81e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4e77fe640 1 -- 172.21.15.115:0/3748271018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e8157730 msgr2=0x7fc4e80ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:44.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4e77fe640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e8157730 0x7fc4e80ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4e77fe640 1 -- 172.21.15.115:0/3748271018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e8158d00 msgr2=0x7fc4e81e58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4e77fe640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e8158d00 0x7fc4e81e58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4e77fe640 1 -- 172.21.15.115:0/3748271018 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e8157ea0 con 0x7fc4d0004d00 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4e7fff640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e8158d00 0x7fc4e81e58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4e77fe640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 0x7fc4e8153c50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8011900 tx=0x7fc4d8011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4cffff640 1 -- 172.21.15.115:0/3748271018 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800fbd0 con 0x7fc4d0004d00 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4cffff640 1 -- 172.21.15.115:0/3748271018 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d800fd30 con 0x7fc4d0004d00 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.293+0000 7fc4cffff640 1 -- 172.21.15.115:0/3748271018 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8015660 con 0x7fc4d0004d00 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.294+0000 7fc4ee009640 1 -- 172.21.15.115:0/3748271018 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e8158180 con 0x7fc4d0004d00 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.294+0000 7fc4ee009640 1 -- 172.21.15.115:0/3748271018 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e81e5e00 con 0x7fc4d0004d00 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.295+0000 7fc4ee009640 1 -- 172.21.15.115:0/3748271018 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e804e890 con 0x7fc4d0004d00 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.296+0000 7fc4cffff640 1 -- 172.21.15.115:0/3748271018 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 62) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fc4d80157c0 con 0x7fc4d0004d00 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.296+0000 7fc4cffff640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc4d406a5a0 0x7fc4d406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.297+0000 7fc4cffff640 1 -- 172.21.15.115:0/3748271018 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4d8089200 con 0x7fc4d0004d00 2023-12-01T01:54:44.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.297+0000 7fc4e6ffd640 1 -- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc4d406a5a0 msgr2=0x7fc4d406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:44.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.297+0000 7fc4e6ffd640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc4d406a5a0 0x7fc4d406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:44.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.301+0000 7fc4cffff640 1 -- 172.21.15.115:0/3748271018 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc4d8054770 con 0x7fc4d0004d00 2023-12-01T01:54:44.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.498+0000 7fc4e6ffd640 1 -- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc4d406a5a0 msgr2=0x7fc4d406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:44.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.498+0000 7fc4e6ffd640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc4d406a5a0 0x7fc4d406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:54:44.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.505+0000 7fc4ee009640 1 -- 172.21.15.115:0/3748271018 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fc4e815c170 con 0x7fc4d0004d00 2023-12-01T01:54:44.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.512+0000 7fc4cffff640 1 -- 172.21.15.115:0/3748271018 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc4d800fea0 con 0x7fc4d0004d00 2023-12-01T01:54:44.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 -- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc4d406a5a0 msgr2=0x7fc4d406ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:44.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc4d406a5a0 0x7fc4d406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 -- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 msgr2=0x7fc4e8153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:44.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 0x7fc4e8153c50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8011900 tx=0x7fc4d8011dc0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 -- 172.21.15.115:0/3748271018 shutdown_connections 2023-12-01T01:54:44.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc4d406a5a0 0x7fc4d406ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4e8157730 0x7fc4e80ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e8158d00 0x7fc4e81e58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 --2- 172.21.15.115:0/3748271018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4d0004d00 0x7fc4e8153c50 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 -- 172.21.15.115:0/3748271018 >> 172.21.15.115:0/3748271018 conn(0x7fc4e806a0e0 msgr2=0x7fc4e80cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:44.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 -- 172.21.15.115:0/3748271018 shutdown_connections 2023-12-01T01:54:44.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.514+0000 7fc4ee009640 1 -- 172.21.15.115:0/3748271018 wait complete. 2023-12-01T01:54:44.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:44.534 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-01T01:54:44.587 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-01T01:54:44.677 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:44.675+0000 7f0391e41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:54:44.692 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:44.690+0000 7efddae32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:54:44.729 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:44.727+0000 7f0391e41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:54:44.745 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:44.743+0000 7efddae32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:54:44.779 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:44.777+0000 7f0391e41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:54:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.786+0000 7f5cc60b6640 1 Processor -- start 2023-12-01T01:54:44.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.787+0000 7f5cc60b6640 1 -- start start 2023-12-01T01:54:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.789+0000 7f5cc60b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc00aef60 0x7f5cc00b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.789+0000 7f5cc60b6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5cc00b3140 0x7f5cc00cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.789+0000 7f5cc60b6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc00b38d0 con 0x7f5cc015e3c0 2023-12-01T01:54:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.789+0000 7f5cc60b6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc00af340 con 0x7f5cc00b3140 2023-12-01T01:54:44.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.789+0000 7f5cc60b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00b35f0 con 0x7f5cc00aef60 2023-12-01T01:54:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.789+0000 7f5cb7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5cc015e3c0 0x7f5cc00cc3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:48338/0 (socket says 172.21.15.115:48338) 2023-12-01T01:54:44.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.789+0000 7f5cb7fff640 1 -- 172.21.15.115:0/616962123 learned_addr learned my addr 172.21.15.115:0/616962123 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.789+0000 7f5cbf7fe640 1 --2- 172.21.15.115:0/616962123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc00aef60 0x7f5cc00b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.789+0000 7f5cbffff640 1 --2- 172.21.15.115:0/616962123 >> v2:172.21.15.142:3300/0 conn(0x7f5cc00b3140 0x7f5cc00cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.790+0000 7f5cbeffd640 1 -- 172.21.15.115:0/616962123 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824797281 0 0) 0x7f5cc00b38d0 con 0x7f5cc015e3c0 2023-12-01T01:54:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.790+0000 7f5cbeffd640 1 -- 172.21.15.115:0/616962123 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cac003520 con 0x7f5cc015e3c0 2023-12-01T01:54:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.790+0000 7f5cbeffd640 1 -- 172.21.15.115:0/616962123 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1914580825 0 0) 0x7f5cac003520 con 0x7f5cc015e3c0 2023-12-01T01:54:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.790+0000 7f5cbffff640 1 -- 172.21.15.115:0/616962123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc00aef60 msgr2=0x7f5cc00b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.790+0000 7f5cbffff640 1 --2- 172.21.15.115:0/616962123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc00aef60 0x7f5cc00b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.790+0000 7f5cbffff640 1 -- 172.21.15.115:0/616962123 >> v1:172.21.15.115:6789/0 conn(0x7f5cc015e3c0 legacy=0x7f5cc00cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:44.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.790+0000 7f5cbffff640 1 -- 172.21.15.115:0/616962123 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0070570 con 0x7f5cc00b3140 2023-12-01T01:54:44.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.791+0000 7f5cbffff640 1 --2- 172.21.15.115:0/616962123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc00b3140 0x7f5cc00cdd60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009d90 tx=0x7f5cb802bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dc9a0aa49811796 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:44.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.792+0000 7f5cbeffd640 1 -- 172.21.15.115:0/616962123 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8001670 con 0x7f5cc00b3140 2023-12-01T01:54:44.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.792+0000 7f5cbeffd640 1 -- 172.21.15.115:0/616962123 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb802c8a0 con 0x7f5cc00b3140 2023-12-01T01:54:44.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.792+0000 7f5cbeffd640 1 -- 172.21.15.115:0/616962123 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb802cc10 con 0x7f5cc00b3140 2023-12-01T01:54:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.792+0000 7f5cc60b6640 1 -- 172.21.15.115:0/616962123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc00b3140 msgr2=0x7f5cc00cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.792+0000 7f5cc60b6640 1 --2- 172.21.15.115:0/616962123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc00b3140 0x7f5cc00cdd60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009d90 tx=0x7f5cb802bd70 comp rx=0 tx=0).stop 2023-12-01T01:54:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.792+0000 7f5cc60b6640 1 -- 172.21.15.115:0/616962123 shutdown_connections 2023-12-01T01:54:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.792+0000 7f5cc60b6640 1 --2- 172.21.15.115:0/616962123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc00b3140 0x7f5cc00cdd60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.792+0000 7f5cc60b6640 1 --2- 172.21.15.115:0/616962123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc00aef60 0x7f5cc00b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.792+0000 7f5cc60b6640 1 -- 172.21.15.115:0/616962123 >> 172.21.15.115:0/616962123 conn(0x7f5cc00b1720 msgr2=0x7f5cc00b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.792+0000 7f5cc60b6640 1 -- 172.21.15.115:0/616962123 shutdown_connections 2023-12-01T01:54:44.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.793+0000 7f5cc60b6640 1 -- 172.21.15.115:0/616962123 wait complete. 2023-12-01T01:54:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.794+0000 7f5cc60b6640 1 Processor -- start 2023-12-01T01:54:44.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.794+0000 7f5cc60b6640 1 -- start start 2023-12-01T01:54:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.795+0000 7f5cc60b6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc00aef60 0x7f5cc00b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.795+0000 7f5cc60b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc00b3140 0x7f5cc01f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.795+0000 7f5cc60b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc015e3c0 0x7f5cc01f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.795+0000 7f5cc60b6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00a8480 con 0x7f5cc015e3c0 2023-12-01T01:54:44.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.795+0000 7f5cc60b6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00a8690 con 0x7f5cc00aef60 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.795+0000 7f5cc60b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00a8970 con 0x7f5cc00b3140 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.795+0000 7f5cbffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc015e3c0 0x7f5cc01f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc015e3c0 0x7f5cc01f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49354/0 (socket says 172.21.15.115:49354) 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbffff640 1 -- 172.21.15.115:0/559108622 learned_addr learned my addr 172.21.15.115:0/559108622 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbf7fe640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc00aef60 0x7f5cc00b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbffff640 1 -- 172.21.15.115:0/559108622 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc00b3140 msgr2=0x7f5cc01f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbffff640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc00b3140 0x7f5cc01f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbffff640 1 -- 172.21.15.115:0/559108622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc00aef60 msgr2=0x7f5cc00b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbffff640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc00aef60 0x7f5cc00b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbffff640 1 -- 172.21.15.115:0/559108622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb8009a70 con 0x7f5cc015e3c0 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbf7fe640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc00aef60 0x7f5cc00b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:54:44.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbffff640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc015e3c0 0x7f5cc01f3f00 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009eb0 tx=0x7f5cb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.796+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8007000 con 0x7f5cc015e3c0 2023-12-01T01:54:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.797+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb8009450 con 0x7f5cc015e3c0 2023-12-01T01:54:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.797+0000 7f5cc60b6640 1 -- 172.21.15.115:0/559108622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc00a8c50 con 0x7f5cc015e3c0 2023-12-01T01:54:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.797+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8004470 con 0x7f5cc015e3c0 2023-12-01T01:54:44.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.797+0000 7f5cc60b6640 1 -- 172.21.15.115:0/559108622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc00a9450 con 0x7f5cc015e3c0 2023-12-01T01:54:44.800 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:44.797+0000 7efddae32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:54:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.798+0000 7f5cc60b6640 1 -- 172.21.15.115:0/559108622 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc00a9170 con 0x7f5cc015e3c0 2023-12-01T01:54:44.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.799+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 62) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f5cb803b070 con 0x7f5cc015e3c0 2023-12-01T01:54:44.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.800+0000 7f5cbcff9640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:44.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.800+0000 7f5cbf7fe640 1 -- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 msgr2=0x7f5ca006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:44.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.800+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cb80ab290 con 0x7f5cc015e3c0 2023-12-01T01:54:44.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.800+0000 7f5cbf7fe640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:54:44.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:44.808+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5cc00a9170 con 0x7f5cc015e3c0 2023-12-01T01:54:44.841 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:44.839+0000 7f0391e41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:54:44.866 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:44.864+0000 7efddae32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:54:44.890 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:44.889+0000 7f0391e41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:54:44.917 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:44.915+0000 7efddae32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:54:45.001 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:45.000+0000 7f0391e41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:54:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:45.001+0000 7f5cbf7fe640 1 -- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 msgr2=0x7f5ca006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:45.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:45.001+0000 7f5cbf7fe640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:54:45.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:45.015+0000 7f5cc60b6640 1 -- 172.21.15.115:0/559108622 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cc0070b30 con 0x7f5ca006a5a0 2023-12-01T01:54:45.031 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:45.029+0000 7efddae32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:54:45.065 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:45.064+0000 7f0391e41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:54:45.094 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:45.092+0000 7efddae32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:54:45.115 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:45.114+0000 7f0391e41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:54:45.143 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:45.141+0000 7efddae32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:54:45.172 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:45.170+0000 7f0391e41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:54:45.199 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:45.197+0000 7efddae32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:54:45.221 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:45.220+0000 7f0391e41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:54:45.249 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:45.248+0000 7efddae32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:54:45.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:45.402+0000 7f5cbf7fe640 1 -- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 msgr2=0x7f5ca006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:45.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:45.402+0000 7f5cbf7fe640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:54:45.624 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:45.623+0000 7f0391e41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:54:45.633 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:45.631+0000 7efddae32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:54:45.785 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:45.784+0000 7f0391e41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:54:45.793 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:45.791+0000 7efddae32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:54:45.893 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:45.892+0000 7f0391e41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:54:45.900 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:45.898+0000 7efddae32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:54:45.944 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:45.943+0000 7f0391e41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:54:45.950 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:45.949+0000 7efddae32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:54:46.051 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:46.051+0000 7f0391e41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:54:46.057 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:46.055+0000 7efddae32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:54:46.158 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:46.157+0000 7f0391e41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:54:46.162 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:46.161+0000 7efddae32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:54:46.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:46.203+0000 7f5cbf7fe640 1 -- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 msgr2=0x7f5ca006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:54:46.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:46.203+0000 7f5cbf7fe640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:54:46.314 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:46.313+0000 7f0391e41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:54:46.317 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:46.316+0000 7efddae32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:54:46.363 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:46.363+0000 7f0391e41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:54:46.367 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:46.366+0000 7efddae32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:54:46.483 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:46.482+0000 7f0391e41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:54:46.486 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:46.485+0000 7efddae32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:54:47.307 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.305+0000 7efddae32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:54:47.309 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:47.308+0000 7f0391e41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:54:47.602 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.600+0000 7efddae32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:54:47.605 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:47.604+0000 7f0391e41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:54:47.659 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.657+0000 7efddae32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:54:47.663 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:47.662+0000 7f0391e41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:54:47.774 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.773+0000 7efddae32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:54:47.777 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:54:47.775+0000 7f0391e41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:54:47.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:47.787+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 63) v1 ==== 88335+0+0 (secure 0 0 0) 0x7f5cb80a97b0 con 0x7f5cc015e3c0 2023-12-01T01:54:47.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:47.787+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 msgr2=0x7f5ca006ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:54:47.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:47.787+0000 7f5cbcff9640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:47.984 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.982+0000 7efd56282640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:47.984 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.982+0000 7efd56282640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:47.984 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.982+0000 7efd56282640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:47.984 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.982+0000 7efd56282640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:47.984 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.982+0000 7efd56282640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:47.984 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.982+0000 7efd5a28a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:47.984 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.982+0000 7efd5a28a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:47.985 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.982+0000 7efd5a28a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:47.985 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.982+0000 7efd5a28a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:47.985 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:54:47.982+0000 7efd5a28a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:54:48.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.803+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 64) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f5cb8076c60 con 0x7f5cc015e3c0 2023-12-01T01:54:48.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.804+0000 7f5cbcff9640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:48.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.804+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ca0027f80 con 0x7f5ca006a5a0 2023-12-01T01:54:48.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.810+0000 7f5cbf7fe640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:48.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.821+0000 7f5cbf7fe640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00093d0 tx=0x7f5cb0009190 comp rx=0 tx=0).ready entity=mgr.8151 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:48.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.988+0000 7f5cbcff9640 1 -- 172.21.15.115:0/559108622 <== mgr.8151 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5ca0027f80 con 0x7f5ca006a5a0 2023-12-01T01:54:48.990 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path snap_test_dir1 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 -- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 msgr2=0x7f5ca006ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5cb00093d0 tx=0x7f5cb0009190 comp rx=0 tx=0).stop 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 -- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc015e3c0 msgr2=0x7f5cc01f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc015e3c0 0x7f5cc01f3f00 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009eb0 tx=0x7f5cb8004a40 comp rx=0 tx=0).stop 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 -- 172.21.15.115:0/559108622 shutdown_connections 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5ca006a5a0 0x7f5ca006ca60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc00b3140 0x7f5cc01f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cc015e3c0 0x7f5cc01f3f00 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 --2- 172.21.15.115:0/559108622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc00aef60 0x7f5cc00b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 -- 172.21.15.115:0/559108622 >> 172.21.15.115:0/559108622 conn(0x7f5cc00b1720 msgr2=0x7f5cc015d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 -- 172.21.15.115:0/559108622 shutdown_connections 2023-12-01T01:54:48.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:48.992+0000 7f5cc60b6640 1 -- 172.21.15.115:0/559108622 wait complete. 2023-12-01T01:54:49.015 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-01T01:54:49.015 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-01-01_55_00 in ~11s... 2023-12-01T01:54:49.015 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-12-01T01:54:49.015 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-01T01:54:49.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.231+0000 7fa2ab722640 1 Processor -- start 2023-12-01T01:54:49.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.231+0000 7fa2ab722640 1 -- start start 2023-12-01T01:54:49.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.232+0000 7fa2ab722640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa2a40ce1b0 0x7fa2a40cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.232+0000 7fa2ab722640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a40cc9a0 con 0x7fa2a415ac80 2023-12-01T01:54:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.232+0000 7fa2ab722640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a40cbd90 con 0x7fa2a40b0e60 2023-12-01T01:54:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.232+0000 7fa2ab722640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a415b060 con 0x7fa2a40ce1b0 2023-12-01T01:54:49.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.233+0000 7fa2a94be640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa2a415ac80 0x7fa2a40cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56358/0 (socket says 172.21.15.115:56358) 2023-12-01T01:54:49.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.233+0000 7fa2a94be640 1 -- 172.21.15.115:0/3054580842 learned_addr learned my addr 172.21.15.115:0/3054580842 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:54:49.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.233+0000 7fa2a8cbd640 1 --2- 172.21.15.115:0/3054580842 >> v2:172.21.15.142:3300/0 conn(0x7fa2a40ce1b0 0x7fa2a40cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:49.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.233+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224187610 0 0) 0x7fa2a40cc9a0 con 0x7fa2a415ac80 2023-12-01T01:54:49.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.233+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa290003520 con 0x7fa2a415ac80 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.233+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4040769059 0 0) 0x7fa290003520 con 0x7fa2a415ac80 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.233+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 >> v2:172.21.15.142:3300/0 conn(0x7fa2a40ce1b0 msgr2=0x7fa2a40cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.233+0000 7fa297fff640 1 --2- 172.21.15.115:0/3054580842 >> v2:172.21.15.142:3300/0 conn(0x7fa2a40ce1b0 0x7fa2a40cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.233+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 >> v1:172.21.15.142:6789/0 conn(0x7fa2a40b0e60 legacy=0x7fa2a406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a40cc070 con 0x7fa2a415ac80 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa2a0001670 con 0x7fa2a415ac80 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa2a8cbd640 1 --2- 172.21.15.115:0/3054580842 >> v2:172.21.15.142:3300/0 conn(0x7fa2a40ce1b0 0x7fa2a40cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 >> v1:172.21.15.115:6789/0 conn(0x7fa2a415ac80 legacy=0x7fa2a40cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa297fff640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa290003970 0x7fa290003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa297fff640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa290004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa297fff640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa290005180 0x7fa29000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a415b060 con 0x7fa2900047c0 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29000c0b0 con 0x7fa290003970 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29000c270 con 0x7fa290005180 2023-12-01T01:54:49.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa2a8cbd640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa290004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa2a8cbd640 1 -- 172.21.15.115:0/3054580842 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa290005180 msgr2=0x7fa29000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa2a8cbd640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa290005180 0x7fa29000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa2a8cbd640 1 -- 172.21.15.115:0/3054580842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa290003970 msgr2=0x7fa290003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:54:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa2a94be640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa290003970 0x7fa290003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa2a8cbd640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa290003970 0x7fa290003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa2a8cbd640 1 -- 172.21.15.115:0/3054580842 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa290003520 con 0x7fa2900047c0 2023-12-01T01:54:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa2a9cbf640 1 -- 172.21.15.115:0/3054580842 reap_dead start 2023-12-01T01:54:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.234+0000 7fa2a94be640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa290003970 0x7fa290003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:54:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.235+0000 7fa2a8cbd640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa290004c40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fa298006fd0 tx=0x7fa2900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9284965a3345e5d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.236+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298004030 con 0x7fa2900047c0 2023-12-01T01:54:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.236+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa298004190 con 0x7fa2900047c0 2023-12-01T01:54:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.236+0000 7fa297fff640 1 -- 172.21.15.115:0/3054580842 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298003610 con 0x7fa2900047c0 2023-12-01T01:54:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.236+0000 7fa2ab722640 1 -- 172.21.15.115:0/3054580842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 msgr2=0x7fa290004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.236+0000 7fa2ab722640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa290004c40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fa298006fd0 tx=0x7fa2900057f0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.236+0000 7fa2ab722640 1 -- 172.21.15.115:0/3054580842 shutdown_connections 2023-12-01T01:54:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.236+0000 7fa2ab722640 1 --2- 172.21.15.115:0/3054580842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa290004c40 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.236+0000 7fa2ab722640 1 -- 172.21.15.115:0/3054580842 >> 172.21.15.115:0/3054580842 conn(0x7fa2a4065960 msgr2=0x7fa2a4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:49.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.237+0000 7fa2ab722640 1 -- 172.21.15.115:0/3054580842 shutdown_connections 2023-12-01T01:54:49.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.237+0000 7fa2ab722640 1 -- 172.21.15.115:0/3054580842 wait complete. 2023-12-01T01:54:49.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.238+0000 7fa2ab722640 1 Processor -- start 2023-12-01T01:54:49.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.238+0000 7fa2ab722640 1 -- start start 2023-12-01T01:54:49.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.239+0000 7fa2ab722640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa2a415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:49.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.239+0000 7fa2ab722640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2a415d560 0x7fa2a415bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:49.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.239+0000 7fa2ab722640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a4142b90 0x7fa2a4156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:49.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.239+0000 7fa2ab722640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a415d140 con 0x7fa2900047c0 2023-12-01T01:54:49.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.239+0000 7fa2ab722640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a415c290 con 0x7fa2a4142b90 2023-12-01T01:54:49.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.239+0000 7fa2ab722640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a415c4c0 con 0x7fa2a415d560 2023-12-01T01:54:49.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.240+0000 7fa2a94be640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa2a415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.240+0000 7fa2a94be640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa2a415a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34684/0 (socket says 172.21.15.115:34684) 2023-12-01T01:54:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.240+0000 7fa2a94be640 1 -- 172.21.15.115:0/2053222122 learned_addr learned my addr 172.21.15.115:0/2053222122 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:54:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.240+0000 7fa2a94be640 1 -- 172.21.15.115:0/2053222122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2a415d560 msgr2=0x7fa2a415bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.240+0000 7fa2a94be640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2a415d560 0x7fa2a415bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.240+0000 7fa2a94be640 1 -- 172.21.15.115:0/2053222122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a4142b90 msgr2=0x7fa2a4156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.240+0000 7fa2a94be640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a4142b90 0x7fa2a4156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.240+0000 7fa2a94be640 1 -- 172.21.15.115:0/2053222122 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a415c7a0 con 0x7fa2900047c0 2023-12-01T01:54:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.240+0000 7fa2a94be640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa2a415a060 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000d810 tx=0x7fa2a000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.240+0000 7fa295ffb640 1 -- 172.21.15.115:0/2053222122 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00074e0 con 0x7fa2900047c0 2023-12-01T01:54:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.241+0000 7fa295ffb640 1 -- 172.21.15.115:0/2053222122 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a0007640 con 0x7fa2900047c0 2023-12-01T01:54:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.241+0000 7fa2ab722640 1 -- 172.21.15.115:0/2053222122 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a415ca80 con 0x7fa2900047c0 2023-12-01T01:54:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.241+0000 7fa295ffb640 1 -- 172.21.15.115:0/2053222122 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0004a00 con 0x7fa2900047c0 2023-12-01T01:54:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.241+0000 7fa2ab722640 1 -- 172.21.15.115:0/2053222122 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a4143020 con 0x7fa2900047c0 2023-12-01T01:54:49.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.242+0000 7fa2ab722640 1 -- 172.21.15.115:0/2053222122 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a41431b0 con 0x7fa2900047c0 2023-12-01T01:54:49.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.243+0000 7fa295ffb640 1 -- 172.21.15.115:0/2053222122 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 64) v1 ==== 88462+0+0 (secure 0 0 0) 0x7fa2a0004b60 con 0x7fa2900047c0 2023-12-01T01:54:49.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.244+0000 7fa295ffb640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa28806a2d0 0x7fa28806c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:54:49.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.244+0000 7fa295ffb640 1 -- 172.21.15.115:0/2053222122 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fa2a0088410 con 0x7fa2900047c0 2023-12-01T01:54:49.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.244+0000 7fa2a8cbd640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa28806a2d0 0x7fa28806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:54:49.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.245+0000 7fa2a8cbd640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa28806a2d0 0x7fa28806c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa298035000 tx=0x7fa298030000 comp rx=0 tx=0).ready entity=mgr.8151 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:54:49.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.248+0000 7fa295ffb640 1 -- 172.21.15.115:0/2053222122 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa2a0053be0 con 0x7fa2900047c0 2023-12-01T01:54:49.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.459+0000 7fa2ab722640 1 -- 172.21.15.115:0/2053222122 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2a40cc630 con 0x7fa28806a2d0 2023-12-01T01:54:49.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.475+0000 7fa295ffb640 1 -- 172.21.15.115:0/2053222122 <== mgr.8151 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fa29803e040 con 0x7fa28806a2d0 2023-12-01T01:54:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:54:49.477 INFO:teuthology.orchestra.run.smithi115.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-12-01T01:54:49.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 -- 172.21.15.115:0/2053222122 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa28806a2d0 msgr2=0x7fa28806c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa28806a2d0 0x7fa28806c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa298035000 tx=0x7fa298030000 comp rx=0 tx=0).stop 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 -- 172.21.15.115:0/2053222122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 msgr2=0x7fa2a415a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa2a415a060 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000d810 tx=0x7fa2a000dcd0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 -- 172.21.15.115:0/2053222122 shutdown_connections 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa28806a2d0 0x7fa28806c790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2a415d560 0x7fa2a415bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a4142b90 0x7fa2a4156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 --2- 172.21.15.115:0/2053222122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2900047c0 0x7fa2a415a060 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 -- 172.21.15.115:0/2053222122 >> 172.21.15.115:0/2053222122 conn(0x7fa2a4065960 msgr2=0x7fa2a4070950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 -- 172.21.15.115:0/2053222122 shutdown_connections 2023-12-01T01:54:49.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:54:49.478+0000 7fa2ab722640 1 -- 172.21.15.115:0/2053222122 wait complete. 2023-12-01T01:54:49.500 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-12-01T01:54:49.500 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-12-01T01:54:49.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:49.500 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:54:49.555 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:54:49.555 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:54:49.555 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:54:50.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:50.557 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:54:50.619 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:54:50.619 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:54:50.619 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:54:51.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:51.620 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:54:51.684 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:54:51.684 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:54:51.684 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:54:52.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:52.685 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:54:52.745 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:54:52.745 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:54:52.745 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:54:53.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:53.746 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:54:53.808 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:54:53.808 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:54:53.808 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:54:54.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:54.810 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:54:54.872 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:54:54.873 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:54:54.873 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:54:55.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:55.874 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:54:55.936 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:54:55.936 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:54:55.937 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:54:56.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:56.938 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:54:57.001 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:54:57.001 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:54:57.001 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:54:58.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:58.002 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:54:58.064 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:54:58.064 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:54:58.064 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:54:59.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:54:59.065 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:54:59.128 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:54:59.128 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:54:59.128 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:55:00.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:00.130 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:55:00.195 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T01:55:00.196 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T01:55:00.196 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:55:00.196 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:55:00.202 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:55:01.012 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:01.010+0000 7efd49a69640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:01.012 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:01.010+0000 7efd49a69640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:01.012 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:01.010+0000 7efd49a69640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:01.012 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:01.010+0000 7efd49a69640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:01.012 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:01.010+0000 7efd49a69640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:01.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:01.196 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:55:01.258 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_55_01_UTC 2023-12-01T01:55:01.259 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-01_55_01_UTC'} 2023-12-01T01:55:01.259 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-01-01_55_01_UTC'} 2023-12-01T01:55:01.259 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T01:55:01.259 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-01-01_55_01_UTC'] 2023-12-01T01:55:01.259 DEBUG:tasks.cephfs.test_snap_schedules:exec=1701395689.0153794, now = 1701395701.2596216, timo = 11 2023-12-01T01:55:02.260 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-01T01:55:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.491+0000 7f6071955640 1 Processor -- start 2023-12-01T01:55:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.491+0000 7f6071955640 1 -- start start 2023-12-01T01:55:02.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.492+0000 7f6071955640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f606c0ceca0 0x7f606c0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:02.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.492+0000 7f6071955640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606c0cdbe0 con 0x7f606c0b3810 2023-12-01T01:55:02.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.492+0000 7f6071955640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606c0cff10 con 0x7f606c15abd0 2023-12-01T01:55:02.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.492+0000 7f6071955640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0d05d0 con 0x7f606c0ceca0 2023-12-01T01:55:02.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.493+0000 7f606a7fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f606c0b3810 0x7f606c0ce590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51166/0 (socket says 172.21.15.115:51166) 2023-12-01T01:55:02.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.493+0000 7f606a7fc640 1 -- 172.21.15.115:0/4161492488 learned_addr learned my addr 172.21.15.115:0/4161492488 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:02.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.493+0000 7f606b7fe640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f606c0ceca0 0x7f606c0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:02.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.493+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351978207 0 0) 0x7f606c0cdbe0 con 0x7f606c0b3810 2023-12-01T01:55:02.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.493+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6058003520 con 0x7f606c0b3810 2023-12-01T01:55:02.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1736980296 0 0) 0x7f6058003520 con 0x7f606c0b3810 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f606c0ceca0 msgr2=0x7f606c0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f606c0ceca0 0x7f606c0cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 >> v1:172.21.15.142:6789/0 conn(0x7f606c15abd0 legacy=0x7f606c0b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c0d01f0 con 0x7f606c0b3810 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6054001670 con 0x7f606c0b3810 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 >> v1:172.21.15.115:6789/0 conn(0x7f606c0b3810 legacy=0x7f606c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60580039a0 0x7f6058003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f606c0ceca0 0x7f60580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058004d00 0x7f6058005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0d05d0 con 0x7f60580039a0 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605800bb70 con 0x7f6058004d00 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605800bd30 con 0x7f606c0ceca0 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.494+0000 7f606affd640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60580039a0 0x7f6058003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:02.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.495+0000 7f606b7fe640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058004d00 0x7f6058005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:02.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.495+0000 7f606a7fc640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f606c0ceca0 0x7f60580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:02.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.495+0000 7f606affd640 1 -- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f606c0ceca0 msgr2=0x7f60580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:02.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.495+0000 7f606affd640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f606c0ceca0 0x7f60580047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:02.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.495+0000 7f606affd640 1 -- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058004d00 msgr2=0x7f6058005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:02.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.495+0000 7f606affd640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058004d00 0x7f6058005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:02.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.495+0000 7f606affd640 1 -- 172.21.15.115:0/4161492488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058003520 con 0x7f60580039a0 2023-12-01T01:55:02.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.495+0000 7f606b7fe640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6058004d00 0x7f6058005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.496+0000 7f606affd640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60580039a0 0x7f6058003dc0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f605c001e50 tx=0x7f60580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f108fc8ac5d7ea8f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.497+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c003b70 con 0x7f60580039a0 2023-12-01T01:55:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.497+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c003cd0 con 0x7f60580039a0 2023-12-01T01:55:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.497+0000 7f6069ffb640 1 -- 172.21.15.115:0/4161492488 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c005310 con 0x7f60580039a0 2023-12-01T01:55:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.497+0000 7f6071955640 1 -- 172.21.15.115:0/4161492488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60580039a0 msgr2=0x7f6058003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.497+0000 7f6071955640 1 --2- 172.21.15.115:0/4161492488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60580039a0 0x7f6058003dc0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f605c001e50 tx=0x7f60580057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.497+0000 7f606b7fe640 1 -- 172.21.15.115:0/4161492488 reap_dead start 2023-12-01T01:55:02.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.498+0000 7f6071955640 1 -- 172.21.15.115:0/4161492488 shutdown_connections 2023-12-01T01:55:02.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.498+0000 7f6071955640 1 -- 172.21.15.115:0/4161492488 >> 172.21.15.115:0/4161492488 conn(0x7f606c071430 msgr2=0x7f606c071840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:02.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.498+0000 7f6071955640 1 -- 172.21.15.115:0/4161492488 shutdown_connections 2023-12-01T01:55:02.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.498+0000 7f6071955640 1 -- 172.21.15.115:0/4161492488 wait complete. 2023-12-01T01:55:02.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.500+0000 7f6071955640 1 Processor -- start 2023-12-01T01:55:02.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.500+0000 7f6071955640 1 -- start start 2023-12-01T01:55:02.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.501+0000 7f6071955640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60580039a0 0x7f606c0c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:02.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.501+0000 7f6071955640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c156220 0x7f606c0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:02.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.501+0000 7f6071955640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c0c9f10 0x7f606c0ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:02.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.501+0000 7f6071955640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0ca8d0 con 0x7f606c0c9f10 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.501+0000 7f6071955640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0c7310 con 0x7f606c156220 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.501+0000 7f6071955640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0c6a70 con 0x7f60580039a0 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c0c9f10 0x7f606c0ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c0c9f10 0x7f606c0ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44026/0 (socket says 172.21.15.115:44026) 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606b7fe640 1 -- 172.21.15.115:0/3747258162 learned_addr learned my addr 172.21.15.115:0/3747258162 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606a7fc640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c156220 0x7f606c0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606affd640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60580039a0 0x7f606c0c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606b7fe640 1 -- 172.21.15.115:0/3747258162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60580039a0 msgr2=0x7f606c0c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606b7fe640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60580039a0 0x7f606c0c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606b7fe640 1 -- 172.21.15.115:0/3747258162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c156220 msgr2=0x7f606c0c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606b7fe640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c156220 0x7f606c0c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606b7fe640 1 -- 172.21.15.115:0/3747258162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c0c6d50 con 0x7f606c0c9f10 2023-12-01T01:55:02.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.502+0000 7f606affd640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60580039a0 0x7f606c0c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:02.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.503+0000 7f606b7fe640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c0c9f10 0x7f606c0ca390 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f606c070f10 tx=0x7f6064010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:02.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.503+0000 7f6063fff640 1 -- 172.21.15.115:0/3747258162 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400e8a0 con 0x7f606c0c9f10 2023-12-01T01:55:02.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.503+0000 7f6063fff640 1 -- 172.21.15.115:0/3747258162 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606400ea00 con 0x7f606c0c9f10 2023-12-01T01:55:02.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.503+0000 7f6063fff640 1 -- 172.21.15.115:0/3747258162 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400ed90 con 0x7f606c0c9f10 2023-12-01T01:55:02.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.503+0000 7f6071955640 1 -- 172.21.15.115:0/3747258162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606c0c7030 con 0x7f606c0c9f10 2023-12-01T01:55:02.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.503+0000 7f6071955640 1 -- 172.21.15.115:0/3747258162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c0c9270 con 0x7f606c0c9f10 2023-12-01T01:55:02.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.504+0000 7f6071955640 1 -- 172.21.15.115:0/3747258162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606c04e890 con 0x7f606c0c9f10 2023-12-01T01:55:02.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.505+0000 7f6063fff640 1 -- 172.21.15.115:0/3747258162 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 66) v1 ==== 88783+0+0 (secure 0 0 0) 0x7f6064021460 con 0x7f606c0c9f10 2023-12-01T01:55:02.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.506+0000 7f6063fff640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f604c06a630 0x7f604c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:02.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.507+0000 7f6063fff640 1 -- 172.21.15.115:0/3747258162 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f6064088970 con 0x7f606c0c9f10 2023-12-01T01:55:02.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.507+0000 7f606affd640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f604c06a630 0x7f604c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:02.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.507+0000 7f606affd640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f604c06a630 0x7f604c06caf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f605c034000 tx=0x7f605c009030 comp rx=0 tx=0).ready entity=mgr.8151 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:02.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.510+0000 7f6063fff640 1 -- 172.21.15.115:0/3747258162 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6064053af0 con 0x7f606c0c9f10 2023-12-01T01:55:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.608+0000 7f6063fff640 1 -- 172.21.15.115:0/3747258162 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 67) v1 ==== 88840+0+0 (secure 0 0 0) 0x7f6064053ec0 con 0x7f606c0c9f10 2023-12-01T01:55:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.729+0000 7f6071955640 1 -- 172.21.15.115:0/3747258162 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f606c0cd520 con 0x7f604c06a630 2023-12-01T01:55:02.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.894+0000 7f6063fff640 1 -- 172.21.15.115:0/3747258162 <== mgr.8151 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f605c0050d0 con 0x7f604c06a630 2023-12-01T01:55:02.898 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path snap_test_dir1 2023-12-01T01:55:02.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.897+0000 7f6071955640 1 -- 172.21.15.115:0/3747258162 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f604c06a630 msgr2=0x7f604c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.897+0000 7f6071955640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f604c06a630 0x7f604c06caf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f605c034000 tx=0x7f605c009030 comp rx=0 tx=0).stop 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.898+0000 7f6071955640 1 -- 172.21.15.115:0/3747258162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c0c9f10 msgr2=0x7f606c0ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.898+0000 7f6071955640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c0c9f10 0x7f606c0ca390 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f606c070f10 tx=0x7f6064010c20 comp rx=0 tx=0).stop 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.898+0000 7f6071955640 1 -- 172.21.15.115:0/3747258162 shutdown_connections 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.898+0000 7f6071955640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f604c06a630 0x7f604c06caf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.898+0000 7f6071955640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f606c0c9f10 0x7f606c0ca390 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.898+0000 7f6071955640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c156220 0x7f606c0c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.898+0000 7f6071955640 1 --2- 172.21.15.115:0/3747258162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60580039a0 0x7f606c0c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.898+0000 7f6071955640 1 -- 172.21.15.115:0/3747258162 >> 172.21.15.115:0/3747258162 conn(0x7f606c071430 msgr2=0x7f606c0d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.898+0000 7f6071955640 1 -- 172.21.15.115:0/3747258162 shutdown_connections 2023-12-01T01:55:02.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:02.898+0000 7f6071955640 1 -- 172.21.15.115:0/3747258162 wait complete. 2023-12-01T01:55:02.919 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-01T01:55:02.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:02.919 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T01:55:02.973 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_55_01_UTC 2023-12-01T01:55:02.974 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-01_55_01_UTC 2023-12-01T01:55:02.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:02.974 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-01_55_01_UTC) 2023-12-01T01:55:03.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:03.030 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-01T01:55:03.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T01:55:03.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.285+0000 7f9b71d8e640 1 Processor -- start 2023-12-01T01:55:03.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.285+0000 7f9b71d8e640 1 -- start start 2023-12-01T01:55:03.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.286+0000 7f9b71d8e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9b6c0addc0 0x7f9b6c15f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:03.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.286+0000 7f9b71d8e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0cb7e0 0x7f9b6c0b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:03.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.286+0000 7f9b71d8e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6c15b040 con 0x7f9b6c15bdf0 2023-12-01T01:55:03.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.286+0000 7f9b71d8e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6c15f5d0 con 0x7f9b6c0addc0 2023-12-01T01:55:03.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.286+0000 7f9b71d8e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0ca020 con 0x7f9b6c0cb7e0 2023-12-01T01:55:03.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.287+0000 7f9b70d8c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9b6c0addc0 0x7f9b6c15f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:03.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.287+0000 7f9b6bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9b6c15bdf0 0x7f9b6c0cb0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51172/0 (socket says 172.21.15.115:51172) 2023-12-01T01:55:03.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.287+0000 7f9b6bfff640 1 -- 172.21.15.115:0/2849993132 learned_addr learned my addr 172.21.15.115:0/2849993132 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:03.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.287+0000 7f9b7158d640 1 --2- 172.21.15.115:0/2849993132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0cb7e0 0x7f9b6c0b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:03.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.287+0000 7f9b70d8c640 1 -- 172.21.15.115:0/2849993132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0cb7e0 msgr2=0x7f9b6c0b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:03.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.287+0000 7f9b70d8c640 1 --2- 172.21.15.115:0/2849993132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0cb7e0 0x7f9b6c0b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.287+0000 7f9b70d8c640 1 -- 172.21.15.115:0/2849993132 >> v1:172.21.15.115:6789/0 conn(0x7f9b6c15bdf0 legacy=0x7f9b6c0cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:03.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.287+0000 7f9b70d8c640 1 -- 172.21.15.115:0/2849993132 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c0ae1a0 con 0x7f9b6c0addc0 2023-12-01T01:55:03.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.287+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/2849993132 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683125428 0 0) 0x7f9b6c15b040 con 0x7f9b6c15bdf0 2023-12-01T01:55:03.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.288+0000 7f9b70d8c640 1 --2- 172.21.15.115:0/2849993132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 0x7f9b6c15f090 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f9b600097a0 tx=0x7f9b6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1615a0cbf3893746 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:03.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/2849993132 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60031830 con 0x7f9b6c0addc0 2023-12-01T01:55:03.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/2849993132 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b60031990 con 0x7f9b6c0addc0 2023-12-01T01:55:03.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/2849993132 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60031d20 con 0x7f9b6c0addc0 2023-12-01T01:55:03.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b71d8e640 1 -- 172.21.15.115:0/2849993132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 msgr2=0x7f9b6c15f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:03.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b71d8e640 1 --2- 172.21.15.115:0/2849993132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 0x7f9b6c15f090 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f9b600097a0 tx=0x7f9b6002b8d0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b71d8e640 1 -- 172.21.15.115:0/2849993132 shutdown_connections 2023-12-01T01:55:03.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b71d8e640 1 --2- 172.21.15.115:0/2849993132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0cb7e0 0x7f9b6c0b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b71d8e640 1 --2- 172.21.15.115:0/2849993132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 0x7f9b6c15f090 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b71d8e640 1 -- 172.21.15.115:0/2849993132 >> 172.21.15.115:0/2849993132 conn(0x7f9b6c065960 msgr2=0x7f9b6c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:03.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b71d8e640 1 -- 172.21.15.115:0/2849993132 shutdown_connections 2023-12-01T01:55:03.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.289+0000 7f9b71d8e640 1 -- 172.21.15.115:0/2849993132 wait complete. 2023-12-01T01:55:03.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.291+0000 7f9b71d8e640 1 Processor -- start 2023-12-01T01:55:03.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.291+0000 7f9b71d8e640 1 -- start start 2023-12-01T01:55:03.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.292+0000 7f9b71d8e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 0x7f9b6c1f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:03.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.292+0000 7f9b71d8e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0cb7e0 0x7f9b6c0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.292+0000 7f9b71d8e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c15bdf0 0x7f9b6c0c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.292+0000 7f9b71d8e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0c3710 con 0x7f9b6c0addc0 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.292+0000 7f9b71d8e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0ad1f0 con 0x7f9b6c15bdf0 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.292+0000 7f9b71d8e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c0c5250 con 0x7f9b6c0cb7e0 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.292+0000 7f9b70d8c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 0x7f9b6c1f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.292+0000 7f9b70d8c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 0x7f9b6c1f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44048/0 (socket says 172.21.15.115:44048) 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.292+0000 7f9b70d8c640 1 -- 172.21.15.115:0/3583735124 learned_addr learned my addr 172.21.15.115:0/3583735124 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.293+0000 7f9b70d8c640 1 -- 172.21.15.115:0/3583735124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0cb7e0 msgr2=0x7f9b6c0accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.293+0000 7f9b70d8c640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0cb7e0 0x7f9b6c0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.293+0000 7f9b70d8c640 1 -- 172.21.15.115:0/3583735124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c15bdf0 msgr2=0x7f9b6c0c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.293+0000 7f9b70d8c640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c15bdf0 0x7f9b6c0c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.293+0000 7f9b70d8c640 1 -- 172.21.15.115:0/3583735124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60009400 con 0x7f9b6c0addc0 2023-12-01T01:55:03.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.293+0000 7f9b70d8c640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 0x7f9b6c1f5bf0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f9b600098c0 tx=0x7f9b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:03.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.293+0000 7f9b697fa640 1 -- 172.21.15.115:0/3583735124 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6003ba70 con 0x7f9b6c0addc0 2023-12-01T01:55:03.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.293+0000 7f9b697fa640 1 -- 172.21.15.115:0/3583735124 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6003bbf0 con 0x7f9b6c0addc0 2023-12-01T01:55:03.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.293+0000 7f9b697fa640 1 -- 172.21.15.115:0/3583735124 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b600453f0 con 0x7f9b6c0addc0 2023-12-01T01:55:03.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.294+0000 7f9b71d8e640 1 -- 172.21.15.115:0/3583735124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6c1f61d0 con 0x7f9b6c0addc0 2023-12-01T01:55:03.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.294+0000 7f9b71d8e640 1 -- 172.21.15.115:0/3583735124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c1f6790 con 0x7f9b6c0addc0 2023-12-01T01:55:03.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.295+0000 7f9b71d8e640 1 -- 172.21.15.115:0/3583735124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c0618d0 con 0x7f9b6c0addc0 2023-12-01T01:55:03.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.296+0000 7f9b697fa640 1 -- 172.21.15.115:0/3583735124 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 67) v1 ==== 88840+0+0 (secure 0 0 0) 0x7f9b6004e450 con 0x7f9b6c0addc0 2023-12-01T01:55:03.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.297+0000 7f9b697fa640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5006a6f0 0x7f9b5006cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:03.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.297+0000 7f9b697fa640 1 -- 172.21.15.115:0/3583735124 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f9b600ab340 con 0x7f9b6c0addc0 2023-12-01T01:55:03.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.297+0000 7f9b6bfff640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5006a6f0 0x7f9b5006cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:03.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.298+0000 7f9b6bfff640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5006a6f0 0x7f9b5006cbb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c15df80 tx=0x7f9b58007980 comp rx=0 tx=0).ready entity=mgr.8151 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:03.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.301+0000 7f9b697fa640 1 -- 172.21.15.115:0/3583735124 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9b600763e0 con 0x7f9b6c0addc0 2023-12-01T01:55:03.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.539+0000 7f9b71d8e640 1 -- 172.21.15.115:0/3583735124 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f9b6c0c9580 con 0x7f9b6c0addc0 2023-12-01T01:55:03.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.621+0000 7f9b697fa640 1 -- 172.21.15.115:0/3583735124 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 68) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f9b600767b0 con 0x7f9b6c0addc0 2023-12-01T01:55:03.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.621+0000 7f9b697fa640 1 -- 172.21.15.115:0/3583735124 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v68) v1 ==== 93+0+0 (secure 0 0 0) 0x7f9b60072890 con 0x7f9b6c0addc0 2023-12-01T01:55:03.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 -- 172.21.15.115:0/3583735124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5006a6f0 msgr2=0x7f9b5006cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5006a6f0 0x7f9b5006cbb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c15df80 tx=0x7f9b58007980 comp rx=0 tx=0).stop 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 -- 172.21.15.115:0/3583735124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 msgr2=0x7f9b6c1f5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 0x7f9b6c1f5bf0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f9b600098c0 tx=0x7f9b600046c0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 -- 172.21.15.115:0/3583735124 shutdown_connections 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5006a6f0 0x7f9b5006cbb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0cb7e0 0x7f9b6c0accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c15bdf0 0x7f9b6c0c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 --2- 172.21.15.115:0/3583735124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c0addc0 0x7f9b6c1f5bf0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 -- 172.21.15.115:0/3583735124 >> 172.21.15.115:0/3583735124 conn(0x7f9b6c065960 msgr2=0x7f9b6c0b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 -- 172.21.15.115:0/3583735124 shutdown_connections 2023-12-01T01:55:03.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:03.624+0000 7f9b71d8e640 1 -- 172.21.15.115:0/3583735124 wait complete. 2023-12-01T01:55:03.643 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:55:03.643 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:55:03.746 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:55:03.839 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:55:03.839 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:55:03.839 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:55:03.893 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:55:03.893 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:55:03.987 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:55:04.154 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:04.154+0000 7f3a6d640040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:55:04.182 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:04.182+0000 7f762883a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:55:04.209 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:04.209+0000 7f3a6d640040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:55:04.236 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:04.235+0000 7f762883a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:55:04.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.237+0000 7fa2ecf37640 1 Processor -- start 2023-12-01T01:55:04.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.237+0000 7fa2ecf37640 1 -- start start 2023-12-01T01:55:04.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.238+0000 7fa2ecf37640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e815cd20 0x7fa2e80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.238+0000 7fa2ecf37640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa2e80b5d80 0x7fa2e80b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.238+0000 7fa2ecf37640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e80ce2e0 con 0x7fa2e80d1230 2023-12-01T01:55:04.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.238+0000 7fa2ecf37640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e80b2160 con 0x7fa2e80b5d80 2023-12-01T01:55:04.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.238+0000 7fa2ecf37640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80d2a10 con 0x7fa2e815cd20 2023-12-01T01:55:04.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e6d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa2e80d1230 0x7fa2e80d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51186/0 (socket says 172.21.15.115:51186) 2023-12-01T01:55:04.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e6d9d640 1 -- 172.21.15.115:0/2752755238 learned_addr learned my addr 172.21.15.115:0/2752755238 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:04.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553532388 0 0) 0x7fa2e80ce2e0 con 0x7fa2e80d1230 2023-12-01T01:55:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e659c640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e815cd20 0x7fa2e80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e5d9b640 1 --2- 172.21.15.115:0/2752755238 >> v2:172.21.15.142:3300/0 conn(0x7fa2e80b5d80 0x7fa2e80b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d0003520 con 0x7fa2e80d1230 2023-12-01T01:55:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 924534450 0 0) 0x7fa2d0003520 con 0x7fa2e80d1230 2023-12-01T01:55:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e815cd20 msgr2=0x7fa2e80d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e559a640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e815cd20 0x7fa2e80d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 >> v2:172.21.15.142:3300/0 conn(0x7fa2e80b5d80 msgr2=0x7fa2e80b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e559a640 1 --2- 172.21.15.115:0/2752755238 >> v2:172.21.15.142:3300/0 conn(0x7fa2e80b5d80 0x7fa2e80b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80ce000 con 0x7fa2e80d1230 2023-12-01T01:55:04.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa2e00019f0 con 0x7fa2e80d1230 2023-12-01T01:55:04.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.239+0000 7fa2e5d9b640 1 --2- 172.21.15.115:0/2752755238 >> v2:172.21.15.142:3300/0 conn(0x7fa2e80b5d80 0x7fa2e80b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:55:04.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 >> v1:172.21.15.115:6789/0 conn(0x7fa2e80d1230 legacy=0x7fa2e80d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:04.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e559a640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 0x7fa2d0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e559a640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e815cd20 0x7fa2d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e559a640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d0004d00 0x7fa2d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80d2a10 con 0x7fa2d0003a10 2023-12-01T01:55:04.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d000ba80 con 0x7fa2d0004d00 2023-12-01T01:55:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d000bc40 con 0x7fa2e815cd20 2023-12-01T01:55:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e659c640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 0x7fa2d0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e5d9b640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e815cd20 0x7fa2d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e659c640 1 -- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e815cd20 msgr2=0x7fa2d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e659c640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e815cd20 0x7fa2d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e659c640 1 -- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d0004d00 msgr2=0x7fa2d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e659c640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d0004d00 0x7fa2d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e659c640 1 -- 172.21.15.115:0/2752755238 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0003520 con 0x7fa2d0003a10 2023-12-01T01:55:04.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.240+0000 7fa2e5d9b640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e815cd20 0x7fa2d00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:04.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.241+0000 7fa2e659c640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 0x7fa2d0003e30 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009ac0 tx=0x7fa2d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee75e65b42f8ac7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:04.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.241+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d802c6b0 con 0x7fa2d0003a10 2023-12-01T01:55:04.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.241+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d802c810 con 0x7fa2d0003a10 2023-12-01T01:55:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.241+0000 7fa2e559a640 1 -- 172.21.15.115:0/2752755238 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d802cba0 con 0x7fa2d0003a10 2023-12-01T01:55:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.242+0000 7fa2ecf37640 1 -- 172.21.15.115:0/2752755238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 msgr2=0x7fa2d0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.242+0000 7fa2ecf37640 1 --2- 172.21.15.115:0/2752755238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 0x7fa2d0003e30 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009ac0 tx=0x7fa2d00057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.242+0000 7fa2e6d9d640 1 -- 172.21.15.115:0/2752755238 reap_dead start 2023-12-01T01:55:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.242+0000 7fa2ecf37640 1 -- 172.21.15.115:0/2752755238 shutdown_connections 2023-12-01T01:55:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.242+0000 7fa2ecf37640 1 -- 172.21.15.115:0/2752755238 >> 172.21.15.115:0/2752755238 conn(0x7fa2e806a0e0 msgr2=0x7fa2e806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.242+0000 7fa2ecf37640 1 -- 172.21.15.115:0/2752755238 shutdown_connections 2023-12-01T01:55:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.242+0000 7fa2ecf37640 1 -- 172.21.15.115:0/2752755238 wait complete. 2023-12-01T01:55:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.244+0000 7fa2ecf37640 1 Processor -- start 2023-12-01T01:55:04.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.244+0000 7fa2ecf37640 1 -- start start 2023-12-01T01:55:04.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.245+0000 7fa2ecf37640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 0x7fa2e81f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.245+0000 7fa2ecf37640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e8154490 0x7fa2e80c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.245+0000 7fa2ecf37640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e80c87f0 0x7fa2e80c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.245+0000 7fa2ecf37640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80c91e0 con 0x7fa2d0003a10 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.245+0000 7fa2ecf37640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80c6dc0 con 0x7fa2e8154490 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.245+0000 7fa2ecf37640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80c7080 con 0x7fa2e80c87f0 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.245+0000 7fa2e659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 0x7fa2e81f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.245+0000 7fa2e659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 0x7fa2e81f24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44072/0 (socket says 172.21.15.115:44072) 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.245+0000 7fa2e659c640 1 -- 172.21.15.115:0/427296722 learned_addr learned my addr 172.21.15.115:0/427296722 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2e6d9d640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e80c87f0 0x7fa2e80c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2e5d9b640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e8154490 0x7fa2e80c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2e659c640 1 -- 172.21.15.115:0/427296722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e80c87f0 msgr2=0x7fa2e80c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2e659c640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e80c87f0 0x7fa2e80c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2e659c640 1 -- 172.21.15.115:0/427296722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e8154490 msgr2=0x7fa2e80c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2e659c640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e8154490 0x7fa2e80c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2e659c640 1 -- 172.21.15.115:0/427296722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80c7360 con 0x7fa2d0003a10 2023-12-01T01:55:04.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2e5d9b640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e8154490 0x7fa2e80c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2e659c640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 0x7fa2e81f24a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa2d802c250 tx=0x7fa2d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2d6ffd640 1 -- 172.21.15.115:0/427296722 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8003810 con 0x7fa2d0003a10 2023-12-01T01:55:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2d6ffd640 1 -- 172.21.15.115:0/427296722 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8003970 con 0x7fa2d0003a10 2023-12-01T01:55:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2d6ffd640 1 -- 172.21.15.115:0/427296722 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8003ce0 con 0x7fa2d0003a10 2023-12-01T01:55:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.246+0000 7fa2ecf37640 1 -- 172.21.15.115:0/427296722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e80c7640 con 0x7fa2d0003a10 2023-12-01T01:55:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.247+0000 7fa2ecf37640 1 -- 172.21.15.115:0/427296722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e80c7c00 con 0x7fa2d0003a10 2023-12-01T01:55:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.248+0000 7fa2ecf37640 1 -- 172.21.15.115:0/427296722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e80c7d90 con 0x7fa2d0003a10 2023-12-01T01:55:04.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.248+0000 7fa2d6ffd640 1 -- 172.21.15.115:0/427296722 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 68) v1 ==== 88823+0+0 (secure 0 0 0) 0x7fa2d8034440 con 0x7fa2d0003a10 2023-12-01T01:55:04.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.250+0000 7fa2d6ffd640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2c806a6a0 0x7fa2c806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.250+0000 7fa2e5d9b640 1 -- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2c806a6a0 msgr2=0x7fa2c806cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:04.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.250+0000 7fa2d6ffd640 1 -- 172.21.15.115:0/427296722 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fa2d80ab240 con 0x7fa2d0003a10 2023-12-01T01:55:04.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.250+0000 7fa2e5d9b640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2c806a6a0 0x7fa2c806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:04.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.254+0000 7fa2d6ffd640 1 -- 172.21.15.115:0/427296722 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa2d8076380 con 0x7fa2d0003a10 2023-12-01T01:55:04.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.447+0000 7fa2ecf37640 1 -- 172.21.15.115:0/427296722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2e804e890 con 0x7fa2d0003a10 2023-12-01T01:55:04.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.448+0000 7fa2d6ffd640 1 -- 172.21.15.115:0/427296722 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16595 (secure 0 0 0) 0x7fa2d80044e0 con 0x7fa2d0003a10 2023-12-01T01:55:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:04.449 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":126,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:54:47.777124+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:54:30.872492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-01T01:54:32.074780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:54:47.777090+0000","172.21.15.115:6822/88317":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/2560191707":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/31572672":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/1289974123":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/3424983653":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/3883676082":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/841702221":"2023-12-02T01:54:47.777090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 -- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2c806a6a0 msgr2=0x7fa2c806cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2e5d9b640 1 -- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2c806a6a0 msgr2=0x7fa2c806cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2e5d9b640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2c806a6a0 0x7fa2c806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2c806a6a0 0x7fa2c806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 -- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 msgr2=0x7fa2e81f24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 0x7fa2e81f24a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa2d802c250 tx=0x7fa2d8004730 comp rx=0 tx=0).stop 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 -- 172.21.15.115:0/427296722 shutdown_connections 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa2c806a6a0 0x7fa2c806cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e80c87f0 0x7fa2e80c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e8154490 0x7fa2e80c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 --2- 172.21.15.115:0/427296722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2d0003a10 0x7fa2e81f24a0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 -- 172.21.15.115:0/427296722 >> 172.21.15.115:0/427296722 conn(0x7fa2e806a0e0 msgr2=0x7fa2e8156260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:04.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 -- 172.21.15.115:0/427296722 shutdown_connections 2023-12-01T01:55:04.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.451+0000 7fa2ecf37640 1 -- 172.21.15.115:0/427296722 wait complete. 2023-12-01T01:55:04.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:54:30.872492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:54:32.074780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:55:04.471 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:55:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.671+0000 7f2efaf8e640 1 Processor -- start 2023-12-01T01:55:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.671+0000 7f2efaf8e640 1 -- start start 2023-12-01T01:55:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.672+0000 7f2efaf8e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef4146440 0x7f2ef40cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.672+0000 7f2efaf8e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2ef40ccf50 0x7f2ef40c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.672+0000 7f2efaf8e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef40c98a0 con 0x7f2ef40acd00 2023-12-01T01:55:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.672+0000 7f2efaf8e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef40cd3e0 con 0x7f2ef40ccf50 2023-12-01T01:55:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.672+0000 7f2efaf8e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40adbb0 con 0x7f2ef4146440 2023-12-01T01:55:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef9f8c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2ef40acd00 0x7f2ef40ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51194/0 (socket says 172.21.15.115:51194) 2023-12-01T01:55:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef9f8c640 1 -- 172.21.15.115:0/1574217811 learned_addr learned my addr 172.21.15.115:0/1574217811 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:04.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665019882 0 0) 0x7f2ef40c98a0 con 0x7f2ef40acd00 2023-12-01T01:55:04.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ee0003520 con 0x7f2ef40acd00 2023-12-01T01:55:04.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2efa78d640 1 --2- 172.21.15.115:0/1574217811 >> v2:172.21.15.142:3300/0 conn(0x7f2ef40ccf50 0x7f2ef40c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef978b640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef4146440 0x7f2ef40cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 767776662 0 0) 0x7f2ee0003520 con 0x7f2ef40acd00 2023-12-01T01:55:04.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef4146440 msgr2=0x7f2ef40cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef4146440 0x7f2ef40cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 >> v2:172.21.15.142:3300/0 conn(0x7f2ef40ccf50 msgr2=0x7f2ef40c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 --2- 172.21.15.115:0/1574217811 >> v2:172.21.15.142:3300/0 conn(0x7f2ef40ccf50 0x7f2ef40c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef40ade90 con 0x7f2ef40acd00 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2ef0001670 con 0x7f2ef40acd00 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 >> v1:172.21.15.115:6789/0 conn(0x7f2ef40acd00 legacy=0x7f2ef40ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef4146440 0x7f2ee0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.673+0000 7f2ef8f8a640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ee0003ff0 0x7f2ee00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2ef8f8a640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 0x7f2ee0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40cd3e0 con 0x7f2ee0004d00 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40adbb0 con 0x7f2ee0003ff0 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee000bae0 con 0x7f2ef4146440 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2efa78d640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 0x7f2ee0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2ef9f8c640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef4146440 0x7f2ee0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2ef978b640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ee0003ff0 0x7f2ee00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2efa78d640 1 -- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef4146440 msgr2=0x7f2ee0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:04.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2efa78d640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef4146440 0x7f2ee0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2efa78d640 1 -- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ee0003ff0 msgr2=0x7f2ee00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:04.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2efa78d640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ee0003ff0 0x7f2ee00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.674+0000 7f2efa78d640 1 -- 172.21.15.115:0/1574217811 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee0003520 con 0x7f2ee0004d00 2023-12-01T01:55:04.677 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:04.675+0000 7f3a6d640040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:55:04.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.675+0000 7f2efa78d640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 0x7f2ee0005180 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2eec007360 tx=0x7f2ee00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c60b1140e9e518 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:04.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.675+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec02ccf0 con 0x7f2ee0004d00 2023-12-01T01:55:04.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.675+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec02ce50 con 0x7f2ee0004d00 2023-12-01T01:55:04.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.675+0000 7f2ef8f8a640 1 -- 172.21.15.115:0/1574217811 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec034660 con 0x7f2ee0004d00 2023-12-01T01:55:04.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.676+0000 7f2efaf8e640 1 -- 172.21.15.115:0/1574217811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 msgr2=0x7f2ee0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:04.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.676+0000 7f2efaf8e640 1 --2- 172.21.15.115:0/1574217811 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 0x7f2ee0005180 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2eec007360 tx=0x7f2ee00057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.676+0000 7f2efa78d640 1 -- 172.21.15.115:0/1574217811 reap_dead start 2023-12-01T01:55:04.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.676+0000 7f2efaf8e640 1 -- 172.21.15.115:0/1574217811 shutdown_connections 2023-12-01T01:55:04.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.676+0000 7f2efaf8e640 1 -- 172.21.15.115:0/1574217811 >> 172.21.15.115:0/1574217811 conn(0x7f2ef40cee40 msgr2=0x7f2ef40cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:04.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.676+0000 7f2efaf8e640 1 -- 172.21.15.115:0/1574217811 shutdown_connections 2023-12-01T01:55:04.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.676+0000 7f2efaf8e640 1 -- 172.21.15.115:0/1574217811 wait complete. 2023-12-01T01:55:04.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.678+0000 7f2efaf8e640 1 Processor -- start 2023-12-01T01:55:04.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.678+0000 7f2efaf8e640 1 -- start start 2023-12-01T01:55:04.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2efaf8e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 0x7f2ef40c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2efaf8e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ef40af920 0x7f2ef40afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2efaf8e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef40c4af0 0x7f2ef40c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2efaf8e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40b02e0 con 0x7f2ee0004d00 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2efaf8e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef41f6960 con 0x7f2ef40af920 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2efaf8e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40c5890 con 0x7f2ef40c4af0 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2ef9f8c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 0x7f2ef40c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2ef9f8c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 0x7f2ef40c2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44080/0 (socket says 172.21.15.115:44080) 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2ef9f8c640 1 -- 172.21.15.115:0/4191319194 learned_addr learned my addr 172.21.15.115:0/4191319194 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2ef9f8c640 1 -- 172.21.15.115:0/4191319194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef40c4af0 msgr2=0x7f2ef40c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2ef9f8c640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef40c4af0 0x7f2ef40c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2ef9f8c640 1 -- 172.21.15.115:0/4191319194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ef40af920 msgr2=0x7f2ef40afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.679+0000 7f2ef9f8c640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ef40af920 0x7f2ef40afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.680+0000 7f2ef9f8c640 1 -- 172.21.15.115:0/4191319194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef41f35d0 con 0x7f2ee0004d00 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.680+0000 7f2ef9f8c640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 0x7f2ef40c2dd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000e820 tx=0x7f2ef000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.680+0000 7f2eea7fc640 1 -- 172.21.15.115:0/4191319194 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000c7f0 con 0x7f2ee0004d00 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.680+0000 7f2eea7fc640 1 -- 172.21.15.115:0/4191319194 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef000c950 con 0x7f2ee0004d00 2023-12-01T01:55:04.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.680+0000 7f2eea7fc640 1 -- 172.21.15.115:0/4191319194 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000cd00 con 0x7f2ee0004d00 2023-12-01T01:55:04.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.680+0000 7f2efaf8e640 1 -- 172.21.15.115:0/4191319194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef41f5ba0 con 0x7f2ee0004d00 2023-12-01T01:55:04.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.680+0000 7f2efaf8e640 1 -- 172.21.15.115:0/4191319194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef41f6160 con 0x7f2ee0004d00 2023-12-01T01:55:04.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.681+0000 7f2efaf8e640 1 -- 172.21.15.115:0/4191319194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef40618d0 con 0x7f2ee0004d00 2023-12-01T01:55:04.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.682+0000 7f2eea7fc640 1 -- 172.21.15.115:0/4191319194 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 68) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f2ef001d460 con 0x7f2ee0004d00 2023-12-01T01:55:04.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.683+0000 7f2eea7fc640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ed806a670 0x7f2ed806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:04.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.683+0000 7f2eea7fc640 1 -- 172.21.15.115:0/4191319194 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f2ef0088d70 con 0x7f2ee0004d00 2023-12-01T01:55:04.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.683+0000 7f2ef978b640 1 -- 172.21.15.115:0/4191319194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ed806a670 msgr2=0x7f2ed806cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:04.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.683+0000 7f2ef978b640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ed806a670 0x7f2ed806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:04.687 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:04.686+0000 7f762883a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:55:04.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.687+0000 7f2eea7fc640 1 -- 172.21.15.115:0/4191319194 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2ef0053f70 con 0x7f2ee0004d00 2023-12-01T01:55:04.732 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:04.731+0000 7f3a6d640040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:55:04.742 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:04.739+0000 7f762883a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:55:04.783 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:04.783+0000 7f3a6d640040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:55:04.790 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:04.789+0000 7f762883a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:55:04.847 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:04.845+0000 7f3a6d640040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:55:04.850 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:04.849+0000 7f762883a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:55:04.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.877+0000 7f2efaf8e640 1 -- 172.21.15.115:0/4191319194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ef415c740 con 0x7f2ee0004d00 2023-12-01T01:55:04.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.878+0000 7f2eea7fc640 1 -- 172.21.15.115:0/4191319194 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ef0057590 con 0x7f2ee0004d00 2023-12-01T01:55:04.879 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:55:04.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 -- 172.21.15.115:0/4191319194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ed806a670 msgr2=0x7f2ed806cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:04.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ed806a670 0x7f2ed806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 -- 172.21.15.115:0/4191319194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 msgr2=0x7f2ef40c2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:04.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 0x7f2ef40c2dd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000e820 tx=0x7f2ef000ece0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 -- 172.21.15.115:0/4191319194 shutdown_connections 2023-12-01T01:55:04.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2ed806a670 0x7f2ed806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ef40c4af0 0x7f2ef40c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ef40af920 0x7f2ef40afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 --2- 172.21.15.115:0/4191319194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ee0004d00 0x7f2ef40c2dd0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:04.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 -- 172.21.15.115:0/4191319194 >> 172.21.15.115:0/4191319194 conn(0x7f2ef40cee40 msgr2=0x7f2ef4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:04.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 -- 172.21.15.115:0/4191319194 shutdown_connections 2023-12-01T01:55:04.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:04.881+0000 7f2efaf8e640 1 -- 172.21.15.115:0/4191319194 wait complete. 2023-12-01T01:55:04.900 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:04.898+0000 7f3a6d640040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:55:04.901 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:04.898+0000 7f762883a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:55:04.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:55:05.012 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:05.009+0000 7f762883a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:55:05.015 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:05.014+0000 7f3a6d640040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:55:05.072 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:05.071+0000 7f762883a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:55:05.083 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:05.081+0000 7f3a6d640040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:55:05.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.125+0000 7f6753fff640 1 Processor -- start 2023-12-01T01:55:05.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.125+0000 7f6753fff640 1 -- start start 2023-12-01T01:55:05.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.126+0000 7f6753fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f675415cc90 0x7f67540d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.126+0000 7f6753fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f67540d0f70 0x7f67540cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.126+0000 7f6753fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67540cf900 0x7f67540b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.126+0000 7f6753fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67540cfd90 con 0x7f67540d0f70 2023-12-01T01:55:05.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.126+0000 7f6753fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675415d070 con 0x7f675415cc90 2023-12-01T01:55:05.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.126+0000 7f6753fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67540d15a0 con 0x7f67540cf900 2023-12-01T01:55:05.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.126+0000 7f674bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f67540d0f70 0x7f67540cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.126+0000 7f674bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f67540d0f70 0x7f67540cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44084/0 (socket says 172.21.15.115:44084) 2023-12-01T01:55:05.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.126+0000 7f674bfff640 1 -- 172.21.15.115:0/2180598688 learned_addr learned my addr 172.21.15.115:0/2180598688 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:05.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.127+0000 7f67537fe640 1 --2- 172.21.15.115:0/2180598688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67540cf900 0x7f67540b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.127+0000 7f6752ffd640 1 --2- 172.21.15.115:0/2180598688 >> v2:172.21.15.142:3300/0 conn(0x7f675415cc90 0x7f67540d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.127+0000 7f674bfff640 1 -- 172.21.15.115:0/2180598688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67540cf900 msgr2=0x7f67540b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:05.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.127+0000 7f674bfff640 1 --2- 172.21.15.115:0/2180598688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67540cf900 0x7f67540b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.127+0000 7f674bfff640 1 -- 172.21.15.115:0/2180598688 >> v2:172.21.15.142:3300/0 conn(0x7f675415cc90 msgr2=0x7f67540d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:05.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.127+0000 7f674bfff640 1 --2- 172.21.15.115:0/2180598688 >> v2:172.21.15.142:3300/0 conn(0x7f675415cc90 0x7f67540d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.127+0000 7f674bfff640 1 -- 172.21.15.115:0/2180598688 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67540d1880 con 0x7f67540d0f70 2023-12-01T01:55:05.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.128+0000 7f674bfff640 1 --2- 172.21.15.115:0/2180598688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 0x7f67540cf3c0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f67540b5530 tx=0x7f673c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8370b9c0b09253 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:05.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.128+0000 7f67527fc640 1 -- 172.21.15.115:0/2180598688 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c031870 con 0x7f67540d0f70 2023-12-01T01:55:05.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.128+0000 7f67527fc640 1 -- 172.21.15.115:0/2180598688 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c0319d0 con 0x7f67540d0f70 2023-12-01T01:55:05.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.128+0000 7f67527fc640 1 -- 172.21.15.115:0/2180598688 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c031d60 con 0x7f67540d0f70 2023-12-01T01:55:05.132 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:05.126+0000 7f762883a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:55:05.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.129+0000 7f6753fff640 1 -- 172.21.15.115:0/2180598688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 msgr2=0x7f67540cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.129+0000 7f6753fff640 1 --2- 172.21.15.115:0/2180598688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 0x7f67540cf3c0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f67540b5530 tx=0x7f673c02b910 comp rx=0 tx=0).stop 2023-12-01T01:55:05.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.129+0000 7f6753fff640 1 -- 172.21.15.115:0/2180598688 shutdown_connections 2023-12-01T01:55:05.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.129+0000 7f6753fff640 1 --2- 172.21.15.115:0/2180598688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67540cf900 0x7f67540b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.129+0000 7f6753fff640 1 --2- 172.21.15.115:0/2180598688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 0x7f67540cf3c0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.129+0000 7f6753fff640 1 --2- 172.21.15.115:0/2180598688 >> v2:172.21.15.142:3300/0 conn(0x7f675415cc90 0x7f67540d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.129+0000 7f6753fff640 1 -- 172.21.15.115:0/2180598688 >> 172.21.15.115:0/2180598688 conn(0x7f675406a0e0 msgr2=0x7f675406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:05.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.129+0000 7f6753fff640 1 -- 172.21.15.115:0/2180598688 shutdown_connections 2023-12-01T01:55:05.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.129+0000 7f6753fff640 1 -- 172.21.15.115:0/2180598688 wait complete. 2023-12-01T01:55:05.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.131+0000 7f6753fff640 1 Processor -- start 2023-12-01T01:55:05.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.131+0000 7f6753fff640 1 -- start start 2023-12-01T01:55:05.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f6753fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67540cf900 0x7f6754149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f6753fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 0x7f6754152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f6753fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f675415cc90 0x7f6754153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f6753fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540c6490 con 0x7f67540d0f70 2023-12-01T01:55:05.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f6753fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540c6770 con 0x7f675415cc90 2023-12-01T01:55:05.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f6753fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675414a360 con 0x7f67540cf900 2023-12-01T01:55:05.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f674bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 0x7f6754152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f674bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 0x7f6754152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44096/0 (socket says 172.21.15.115:44096) 2023-12-01T01:55:05.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f674bfff640 1 -- 172.21.15.115:0/2008046432 learned_addr learned my addr 172.21.15.115:0/2008046432 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:05.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f67537fe640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f675415cc90 0x7f6754153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.132+0000 7f6752ffd640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67540cf900 0x7f6754149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.133+0000 7f674bfff640 1 -- 172.21.15.115:0/2008046432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67540cf900 msgr2=0x7f6754149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.133+0000 7f674bfff640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67540cf900 0x7f6754149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.133+0000 7f674bfff640 1 -- 172.21.15.115:0/2008046432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f675415cc90 msgr2=0x7f6754153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.133+0000 7f674bfff640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f675415cc90 0x7f6754153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.133+0000 7f674bfff640 1 -- 172.21.15.115:0/2008046432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c0094d0 con 0x7f67540d0f70 2023-12-01T01:55:05.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.133+0000 7f674bfff640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 0x7f6754152b20 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f673c02b950 tx=0x7f673c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:05.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.133+0000 7f6750ff9640 1 -- 172.21.15.115:0/2008046432 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c03ad30 con 0x7f67540d0f70 2023-12-01T01:55:05.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.133+0000 7f6750ff9640 1 -- 172.21.15.115:0/2008046432 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c0443f0 con 0x7f67540d0f70 2023-12-01T01:55:05.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.133+0000 7f6753fff640 1 -- 172.21.15.115:0/2008046432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67540b3e70 con 0x7f67540d0f70 2023-12-01T01:55:05.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.133+0000 7f6750ff9640 1 -- 172.21.15.115:0/2008046432 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0447a0 con 0x7f67540d0f70 2023-12-01T01:55:05.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.134+0000 7f6753fff640 1 -- 172.21.15.115:0/2008046432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67540b4430 con 0x7f67540d0f70 2023-12-01T01:55:05.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.135+0000 7f6753fff640 1 -- 172.21.15.115:0/2008046432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67540618d0 con 0x7f67540d0f70 2023-12-01T01:55:05.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.136+0000 7f6750ff9640 1 -- 172.21.15.115:0/2008046432 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 68) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f673c04d450 con 0x7f67540d0f70 2023-12-01T01:55:05.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.136+0000 7f6750ff9640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f673406a670 0x7f673406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.137+0000 7f6750ff9640 1 -- 172.21.15.115:0/2008046432 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f673c02d040 con 0x7f67540d0f70 2023-12-01T01:55:05.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.137+0000 7f6752ffd640 1 -- 172.21.15.115:0/2008046432 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f673406a670 msgr2=0x7f673406cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:05.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.137+0000 7f6752ffd640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f673406a670 0x7f673406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:05.142 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:05.138+0000 7f3a6d640040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:55:05.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.141+0000 7f6750ff9640 1 -- 172.21.15.115:0/2008046432 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f673c076640 con 0x7f67540d0f70 2023-12-01T01:55:05.184 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:05.183+0000 7f762883a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:55:05.198 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:05.197+0000 7f3a6d640040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:55:05.234 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:05.234+0000 7f762883a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:55:05.250 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:05.249+0000 7f3a6d640040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:55:05.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.331+0000 7f6753fff640 1 -- 172.21.15.115:0/2008046432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67541585a0 con 0x7f67540d0f70 2023-12-01T01:55:05.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.332+0000 7f6750ff9640 1 -- 172.21.15.115:0/2008046432 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f673c079c60 con 0x7f67540d0f70 2023-12-01T01:55:05.332 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 -- 172.21.15.115:0/2008046432 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f673406a670 msgr2=0x7f673406cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f673406a670 0x7f673406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 -- 172.21.15.115:0/2008046432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 msgr2=0x7f6754152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 0x7f6754152b20 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f673c02b950 tx=0x7f673c0046c0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 -- 172.21.15.115:0/2008046432 shutdown_connections 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f673406a670 0x7f673406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f675415cc90 0x7f6754153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67540d0f70 0x7f6754152b20 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 --2- 172.21.15.115:0/2008046432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67540cf900 0x7f6754149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 -- 172.21.15.115:0/2008046432 >> 172.21.15.115:0/2008046432 conn(0x7f675406a0e0 msgr2=0x7f67540cd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:05.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 -- 172.21.15.115:0/2008046432 shutdown_connections 2023-12-01T01:55:05.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.334+0000 7f6753fff640 1 -- 172.21.15.115:0/2008046432 wait complete. 2023-12-01T01:55:05.353 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:55:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.547+0000 7f6c14c21640 1 Processor -- start 2023-12-01T01:55:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.547+0000 7f6c14c21640 1 -- start start 2023-12-01T01:55:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.548+0000 7f6c14c21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 0x7f6c100cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.548+0000 7f6c14c21640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6c100b5fa0 0x7f6c10161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.548+0000 7f6c14c21640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c100ce5e0 con 0x7f6c100cf650 2023-12-01T01:55:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.548+0000 7f6c14c21640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c100d0ec0 con 0x7f6c100b5fa0 2023-12-01T01:55:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.548+0000 7f6c14c21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10161950 con 0x7f6c1015cd90 2023-12-01T01:55:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.549+0000 7f6c0dd9b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6c100b5fa0 0x7f6c10161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.549+0000 7f6c0e59c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 0x7f6c100cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.549+0000 7f6c0dd9b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6c100b5fa0 0x7f6c10161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:42530/0 (socket says 172.21.15.115:42530) 2023-12-01T01:55:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.549+0000 7f6c0dd9b640 1 -- 172.21.15.115:0/1104594426 learned_addr learned my addr 172.21.15.115:0/1104594426 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.549+0000 7f6c0e59c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 0x7f6c100cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:41322/0 (socket says 172.21.15.115:41322) 2023-12-01T01:55:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.550+0000 7f6c0e59c640 1 -- 172.21.15.115:0/1104594426 >> v2:172.21.15.142:3300/0 conn(0x7f6c100b5fa0 msgr2=0x7f6c10161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.550+0000 7f6c0e59c640 1 --2- 172.21.15.115:0/1104594426 >> v2:172.21.15.142:3300/0 conn(0x7f6c100b5fa0 0x7f6c10161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.550+0000 7f6c0e59c640 1 -- 172.21.15.115:0/1104594426 >> v1:172.21.15.142:6789/0 conn(0x7f6c100cf650 legacy=0x7f6c100b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.550+0000 7f6c0e59c640 1 -- 172.21.15.115:0/1104594426 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c100ce300 con 0x7f6c1015cd90 2023-12-01T01:55:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.550+0000 7f6c0dd9b640 1 --2- 172.21.15.115:0/1104594426 >> v2:172.21.15.142:3300/0 conn(0x7f6c100b5fa0 0x7f6c10161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:55:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.551+0000 7f6c0e59c640 1 --2- 172.21.15.115:0/1104594426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 0x7f6c100cdd20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009d90 tx=0x7f6c0002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a62cd772fb3e7c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:05.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.552+0000 7f6c0d59a640 1 -- 172.21.15.115:0/1104594426 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0002cc60 con 0x7f6c1015cd90 2023-12-01T01:55:05.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.552+0000 7f6c0d59a640 1 -- 172.21.15.115:0/1104594426 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0002cdc0 con 0x7f6c1015cd90 2023-12-01T01:55:05.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.552+0000 7f6c0d59a640 1 -- 172.21.15.115:0/1104594426 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00034660 con 0x7f6c1015cd90 2023-12-01T01:55:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.552+0000 7f6c14c21640 1 -- 172.21.15.115:0/1104594426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 msgr2=0x7f6c100cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.552+0000 7f6c14c21640 1 --2- 172.21.15.115:0/1104594426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 0x7f6c100cdd20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009d90 tx=0x7f6c0002beb0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.553+0000 7f6c14c21640 1 -- 172.21.15.115:0/1104594426 shutdown_connections 2023-12-01T01:55:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.553+0000 7f6c14c21640 1 --2- 172.21.15.115:0/1104594426 >> v2:172.21.15.142:3300/0 conn(0x7f6c100b5fa0 0x7f6c10161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.553+0000 7f6c14c21640 1 --2- 172.21.15.115:0/1104594426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 0x7f6c100cdd20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.553+0000 7f6c14c21640 1 -- 172.21.15.115:0/1104594426 >> 172.21.15.115:0/1104594426 conn(0x7f6c10065960 msgr2=0x7f6c10065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.553+0000 7f6c14c21640 1 -- 172.21.15.115:0/1104594426 shutdown_connections 2023-12-01T01:55:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.553+0000 7f6c14c21640 1 -- 172.21.15.115:0/1104594426 wait complete. 2023-12-01T01:55:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.555+0000 7f6c14c21640 1 Processor -- start 2023-12-01T01:55:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.555+0000 7f6c14c21640 1 -- start start 2023-12-01T01:55:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c14c21640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c100b5fa0 0x7f6c100a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c14c21640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cf650 0x7f6c101f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c14c21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 0x7f6c1015f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c14c21640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100aa560 con 0x7f6c100cf650 2023-12-01T01:55:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c14c21640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100aa840 con 0x7f6c100b5fa0 2023-12-01T01:55:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c14c21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100aab20 con 0x7f6c1015cd90 2023-12-01T01:55:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c0dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cf650 0x7f6c101f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c0dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cf650 0x7f6c101f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44112/0 (socket says 172.21.15.115:44112) 2023-12-01T01:55:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c0dd9b640 1 -- 172.21.15.115:0/3428488937 learned_addr learned my addr 172.21.15.115:0/3428488937 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c0e59c640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c100b5fa0 0x7f6c100a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.556+0000 7f6c0ed9d640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 0x7f6c1015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.557+0000 7f6c0dd9b640 1 -- 172.21.15.115:0/3428488937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 msgr2=0x7f6c1015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.557+0000 7f6c0dd9b640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 0x7f6c1015f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.557+0000 7f6c0dd9b640 1 -- 172.21.15.115:0/3428488937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c100b5fa0 msgr2=0x7f6c100a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.557+0000 7f6c0dd9b640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c100b5fa0 0x7f6c100a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.557+0000 7f6c0dd9b640 1 -- 172.21.15.115:0/3428488937 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf80094d0 con 0x7f6c100cf650 2023-12-01T01:55:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.557+0000 7f6c0dd9b640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cf650 0x7f6c101f5030 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800e9d0 tx=0x7f6bf800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.557+0000 7f6c06ffd640 1 -- 172.21.15.115:0/3428488937 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800ca60 con 0x7f6c100cf650 2023-12-01T01:55:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.557+0000 7f6c06ffd640 1 -- 172.21.15.115:0/3428488937 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf800cbc0 con 0x7f6c100cf650 2023-12-01T01:55:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.557+0000 7f6c06ffd640 1 -- 172.21.15.115:0/3428488937 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8013470 con 0x7f6c100cf650 2023-12-01T01:55:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.557+0000 7f6c14c21640 1 -- 172.21.15.115:0/3428488937 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c00009a70 con 0x7f6c100cf650 2023-12-01T01:55:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.558+0000 7f6c14c21640 1 -- 172.21.15.115:0/3428488937 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1015f890 con 0x7f6c100cf650 2023-12-01T01:55:05.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.559+0000 7f6c14c21640 1 -- 172.21.15.115:0/3428488937 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c100ab320 con 0x7f6c100cf650 2023-12-01T01:55:05.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.560+0000 7f6c06ffd640 1 -- 172.21.15.115:0/3428488937 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 68) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f6bf80135d0 con 0x7f6c100cf650 2023-12-01T01:55:05.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.560+0000 7f6c06ffd640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a670 0x7f6bf006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.560+0000 7f6c06ffd640 1 -- 172.21.15.115:0/3428488937 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f6bf8089100 con 0x7f6c100cf650 2023-12-01T01:55:05.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.561+0000 7f6c0e59c640 1 -- 172.21.15.115:0/3428488937 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a670 msgr2=0x7f6bf006cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:05.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.561+0000 7f6c0e59c640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a670 0x7f6bf006cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:05.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.565+0000 7f6c06ffd640 1 -- 172.21.15.115:0/3428488937 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6bf80542d0 con 0x7f6c100cf650 2023-12-01T01:55:05.618 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:05.617+0000 7f762883a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:55:05.653 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:05.651+0000 7f3a6d640040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:55:05.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.754+0000 7f6c14c21640 1 -- 172.21.15.115:0/3428488937 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c100d1350 con 0x7f6c100cf650 2023-12-01T01:55:05.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.755+0000 7f6c06ffd640 1 -- 172.21.15.115:0/3428488937 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6bf80578f0 con 0x7f6c100cf650 2023-12-01T01:55:05.756 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:55:05.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 -- 172.21.15.115:0/3428488937 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a670 msgr2=0x7f6bf006cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:05.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a670 0x7f6bf006cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 -- 172.21.15.115:0/3428488937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cf650 msgr2=0x7f6c101f5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cf650 0x7f6c101f5030 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800e9d0 tx=0x7f6bf800ee90 comp rx=0 tx=0).stop 2023-12-01T01:55:05.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 -- 172.21.15.115:0/3428488937 shutdown_connections 2023-12-01T01:55:05.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6bf006a670 0x7f6bf006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c1015cd90 0x7f6c1015f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c100cf650 0x7f6c101f5030 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 --2- 172.21.15.115:0/3428488937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c100b5fa0 0x7f6c100a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 -- 172.21.15.115:0/3428488937 >> 172.21.15.115:0/3428488937 conn(0x7f6c10065960 msgr2=0x7f6c101597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:05.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 -- 172.21.15.115:0/3428488937 shutdown_connections 2023-12-01T01:55:05.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.758+0000 7f6c14c21640 1 -- 172.21.15.115:0/3428488937 wait complete. 2023-12-01T01:55:05.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:55:05.776 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:05.776+0000 7f762883a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:55:05.819 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:05.818+0000 7f3a6d640040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:55:05.885 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:05.884+0000 7f762883a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:55:05.932 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:05.932+0000 7f3a6d640040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:55:05.935 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:05.934+0000 7f762883a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:55:05.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.961+0000 7fc49fcdf640 1 Processor -- start 2023-12-01T01:55:05.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.962+0000 7fc49fcdf640 1 -- start start 2023-12-01T01:55:05.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.963+0000 7fc49fcdf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4980b5d80 0x7fc4980d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.963+0000 7fc49fcdf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc4980d1230 0x7fc4980d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.963+0000 7fc49fcdf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4980ce2e0 con 0x7fc49815cd20 2023-12-01T01:55:05.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.963+0000 7fc49fcdf640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4980b2160 con 0x7fc4980d1230 2023-12-01T01:55:05.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.963+0000 7fc49fcdf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980d1be0 con 0x7fc4980b5d80 2023-12-01T01:55:05.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.963+0000 7fc49e27c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc4980d1230 0x7fc4980d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.963+0000 7fc49da7b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc49815cd20 0x7fc4980b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51210/0 (socket says 172.21.15.115:51210) 2023-12-01T01:55:05.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.963+0000 7fc49da7b640 1 -- 172.21.15.115:0/1643299309 learned_addr learned my addr 172.21.15.115:0/1643299309 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:05.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.963+0000 7fc49d27a640 1 --2- 172.21.15.115:0/1643299309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4980b5d80 0x7fc4980d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.964+0000 7fc49e27c640 1 -- 172.21.15.115:0/1643299309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4980b5d80 msgr2=0x7fc4980d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:05.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.964+0000 7fc49e27c640 1 --2- 172.21.15.115:0/1643299309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4980b5d80 0x7fc4980d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.964+0000 7fc49e27c640 1 -- 172.21.15.115:0/1643299309 >> v1:172.21.15.115:6789/0 conn(0x7fc49815cd20 legacy=0x7fc4980b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.964+0000 7fc49e27c640 1 -- 172.21.15.115:0/1643299309 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980ce000 con 0x7fc4980d1230 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.965+0000 7fc49e27c640 1 --2- 172.21.15.115:0/1643299309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 0x7fc4980d16a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc490009d90 tx=0x7fc49002bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4054ae61ede0deb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.965+0000 7fc49ca79640 1 -- 172.21.15.115:0/1643299309 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49002c980 con 0x7fc4980d1230 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.965+0000 7fc49ca79640 1 -- 172.21.15.115:0/1643299309 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49002cae0 con 0x7fc4980d1230 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.965+0000 7fc49ca79640 1 -- 172.21.15.115:0/1643299309 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49002ce70 con 0x7fc4980d1230 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.966+0000 7fc49fcdf640 1 -- 172.21.15.115:0/1643299309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 msgr2=0x7fc4980d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.966+0000 7fc49fcdf640 1 --2- 172.21.15.115:0/1643299309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 0x7fc4980d16a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc490009d90 tx=0x7fc49002bc50 comp rx=0 tx=0).stop 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.966+0000 7fc49fcdf640 1 -- 172.21.15.115:0/1643299309 shutdown_connections 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.966+0000 7fc49fcdf640 1 --2- 172.21.15.115:0/1643299309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 0x7fc4980d16a0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.966+0000 7fc49fcdf640 1 --2- 172.21.15.115:0/1643299309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4980b5d80 0x7fc4980d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.966+0000 7fc49fcdf640 1 -- 172.21.15.115:0/1643299309 >> 172.21.15.115:0/1643299309 conn(0x7fc49806a0e0 msgr2=0x7fc49806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:05.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.966+0000 7fc49fcdf640 1 -- 172.21.15.115:0/1643299309 shutdown_connections 2023-12-01T01:55:05.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.966+0000 7fc49fcdf640 1 -- 172.21.15.115:0/1643299309 wait complete. 2023-12-01T01:55:05.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.968+0000 7fc49fcdf640 1 Processor -- start 2023-12-01T01:55:05.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.968+0000 7fc49fcdf640 1 -- start start 2023-12-01T01:55:05.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49fcdf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4980b5d80 0x7fc4980b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49fcdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 0x7fc498157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49fcdf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc49815cd20 0x7fc4980b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49fcdf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980b3550 con 0x7fc4980d1230 2023-12-01T01:55:05.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49fcdf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980b51f0 con 0x7fc4980b5d80 2023-12-01T01:55:05.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49fcdf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980b3f20 con 0x7fc49815cd20 2023-12-01T01:55:05.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49d27a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 0x7fc498157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49d27a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 0x7fc498157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44134/0 (socket says 172.21.15.115:44134) 2023-12-01T01:55:05.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49d27a640 1 -- 172.21.15.115:0/2853167874 learned_addr learned my addr 172.21.15.115:0/2853167874 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:05.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49da7b640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4980b5d80 0x7fc4980b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49e27c640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc49815cd20 0x7fc4980b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:05.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49d27a640 1 -- 172.21.15.115:0/2853167874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc49815cd20 msgr2=0x7fc4980b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.969+0000 7fc49d27a640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc49815cd20 0x7fc4980b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.970+0000 7fc49d27a640 1 -- 172.21.15.115:0/2853167874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4980b5d80 msgr2=0x7fc4980b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:05.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.970+0000 7fc49d27a640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4980b5d80 0x7fc4980b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:05.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.970+0000 7fc49d27a640 1 -- 172.21.15.115:0/2853167874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc490009a70 con 0x7fc4980d1230 2023-12-01T01:55:05.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.970+0000 7fc49d27a640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 0x7fc498157230 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc48800d5c0 tx=0x7fc48800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:05.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.970+0000 7fc48e7fc640 1 -- 172.21.15.115:0/2853167874 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4880125f0 con 0x7fc4980d1230 2023-12-01T01:55:05.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.970+0000 7fc48e7fc640 1 -- 172.21.15.115:0/2853167874 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc488012750 con 0x7fc4980d1230 2023-12-01T01:55:05.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.970+0000 7fc48e7fc640 1 -- 172.21.15.115:0/2853167874 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488012b00 con 0x7fc4980d1230 2023-12-01T01:55:05.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.970+0000 7fc49fcdf640 1 -- 172.21.15.115:0/2853167874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4980b2940 con 0x7fc4980d1230 2023-12-01T01:55:05.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.970+0000 7fc49fcdf640 1 -- 172.21.15.115:0/2853167874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4980b2f00 con 0x7fc4980d1230 2023-12-01T01:55:05.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.971+0000 7fc49fcdf640 1 -- 172.21.15.115:0/2853167874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4980b3700 con 0x7fc4980d1230 2023-12-01T01:55:05.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.973+0000 7fc48e7fc640 1 -- 172.21.15.115:0/2853167874 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 68) v1 ==== 88823+0+0 (secure 0 0 0) 0x7fc488012c60 con 0x7fc4980d1230 2023-12-01T01:55:05.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.974+0000 7fc48e7fc640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc47c06a6a0 0x7fc47c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:05.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.974+0000 7fc48e7fc640 1 -- 172.21.15.115:0/2853167874 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fc4880892a0 con 0x7fc4980d1230 2023-12-01T01:55:05.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.974+0000 7fc49da7b640 1 -- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc47c06a6a0 msgr2=0x7fc47c06cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:05.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.974+0000 7fc49da7b640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc47c06a6a0 0x7fc47c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:05.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:05.979+0000 7fc48e7fc640 1 -- 172.21.15.115:0/2853167874 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc4980b3700 con 0x7fc4980d1230 2023-12-01T01:55:05.987 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:05.987+0000 7f3a6d640040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:55:06.043 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:06.042+0000 7f762883a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:55:06.099 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:06.098+0000 7f3a6d640040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:55:06.149 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:06.148+0000 7f762883a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:55:06.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.175+0000 7fc49da7b640 1 -- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc47c06a6a0 msgr2=0x7fc47c06cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:06.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.175+0000 7fc49da7b640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc47c06a6a0 0x7fc47c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:55:06.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.179+0000 7fc49fcdf640 1 -- 172.21.15.115:0/2853167874 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc49804e890 con 0x7fc4980d1230 2023-12-01T01:55:06.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.180+0000 7fc48e7fc640 1 -- 172.21.15.115:0/2853167874 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v126) v1 ==== 80+0+465 (secure 0 0 0) 0x7fc4880543d0 con 0x7fc4980d1230 2023-12-01T01:55:06.181 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/88317 2023-12-02T01:54:47.777090+0000 2023-12-01T01:55:06.181 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/88317 2023-12-02T01:54:47.777090+0000 2023-12-01T01:55:06.181 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2560191707 2023-12-02T01:54:47.777090+0000 2023-12-01T01:55:06.181 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/31572672 2023-12-02T01:54:47.777090+0000 2023-12-01T01:55:06.181 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1289974123 2023-12-02T01:54:47.777090+0000 2023-12-01T01:55:06.181 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3424983653 2023-12-02T01:54:47.777090+0000 2023-12-01T01:55:06.181 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3883676082 2023-12-02T01:54:47.777090+0000 2023-12-01T01:55:06.181 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/841702221 2023-12-02T01:54:47.777090+0000 2023-12-01T01:55:06.181 INFO:teuthology.orchestra.run.smithi115.stderr:listed 8 entries 2023-12-01T01:55:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.182+0000 7fc49fcdf640 1 -- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc47c06a6a0 msgr2=0x7fc47c06cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.182+0000 7fc49fcdf640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc47c06a6a0 0x7fc47c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.182+0000 7fc49fcdf640 1 -- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 msgr2=0x7fc498157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.182+0000 7fc49fcdf640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 0x7fc498157230 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc48800d5c0 tx=0x7fc48800da80 comp rx=0 tx=0).stop 2023-12-01T01:55:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.183+0000 7fc49fcdf640 1 -- 172.21.15.115:0/2853167874 shutdown_connections 2023-12-01T01:55:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.183+0000 7fc49fcdf640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc47c06a6a0 0x7fc47c06cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.183+0000 7fc49fcdf640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc49815cd20 0x7fc4980b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.183+0000 7fc49fcdf640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4980d1230 0x7fc498157230 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.183+0000 7fc49fcdf640 1 --2- 172.21.15.115:0/2853167874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4980b5d80 0x7fc4980b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.183+0000 7fc49fcdf640 1 -- 172.21.15.115:0/2853167874 >> 172.21.15.115:0/2853167874 conn(0x7fc49806a0e0 msgr2=0x7fc4980b6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:06.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.183+0000 7fc49fcdf640 1 -- 172.21.15.115:0/2853167874 shutdown_connections 2023-12-01T01:55:06.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.183+0000 7fc49fcdf640 1 -- 172.21.15.115:0/2853167874 wait complete. 2023-12-01T01:55:06.203 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T01:55:06.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:06.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T01:55:06.211 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:06.211+0000 7f3a6d640040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:55:06.305 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T01:55:06.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:06.306 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T01:55:06.307 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:06.304+0000 7f762883a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:55:06.347 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:55:06.347 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:55:06.357 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:06.353+0000 7f762883a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:55:06.379 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:06.378+0000 7f3a6d640040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:55:06.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:55:06.403 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:55:06.403 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:55:06.429 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:06.429+0000 7f3a6d640040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:55:06.455 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:55:06.455 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:55:06.455 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:55:06.478 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:06.476+0000 7f762883a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:55:06.510 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:55:06.511 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:55:06.553 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:06.552+0000 7f3a6d640040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:55:06.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.712+0000 7f6460c38640 1 Processor -- start 2023-12-01T01:55:06.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.712+0000 7f6460c38640 1 -- start start 2023-12-01T01:55:06.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.713+0000 7f6460c38640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f645c15cd20 0x7f645c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:06.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.713+0000 7f6460c38640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f645c0b5d80 0x7f645c0b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:06.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.713+0000 7f6460c38640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c0ce2e0 con 0x7f645c0d1230 2023-12-01T01:55:06.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.713+0000 7f6460c38640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c0b2160 con 0x7f645c0b5d80 2023-12-01T01:55:06.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.713+0000 7f6460c38640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c0d2a10 con 0x7f645c15cd20 2023-12-01T01:55:06.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.713+0000 7f6453fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f645c0b5d80 0x7f645c0b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:06.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.713+0000 7f6453fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f645c0b5d80 0x7f645c0b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44148/0 (socket says 172.21.15.115:44148) 2023-12-01T01:55:06.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.713+0000 7f6453fff640 1 -- 172.21.15.115:0/3803754897 learned_addr learned my addr 172.21.15.115:0/3803754897 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:06.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.713+0000 7f645a59c640 1 --2- 172.21.15.115:0/3803754897 >> v2:172.21.15.142:3300/0 conn(0x7f645c15cd20 0x7f645c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:06.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.714+0000 7f6453fff640 1 -- 172.21.15.115:0/3803754897 >> v2:172.21.15.142:3300/0 conn(0x7f645c15cd20 msgr2=0x7f645c0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:06.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.714+0000 7f6453fff640 1 --2- 172.21.15.115:0/3803754897 >> v2:172.21.15.142:3300/0 conn(0x7f645c15cd20 0x7f645c0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.714+0000 7f6453fff640 1 -- 172.21.15.115:0/3803754897 >> v1:172.21.15.142:6789/0 conn(0x7f645c0d1230 legacy=0x7f645c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:06.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.714+0000 7f6453fff640 1 -- 172.21.15.115:0/3803754897 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0ce000 con 0x7f645c0b5d80 2023-12-01T01:55:06.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.715+0000 7f6453fff640 1 --2- 172.21.15.115:0/3803754897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0b5d80 0x7f645c0b8570 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f645c08bdb0 tx=0x7f644402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e838d04eb3082287 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:06.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.716+0000 7f6459d9b640 1 -- 172.21.15.115:0/3803754897 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64440317e0 con 0x7f645c0b5d80 2023-12-01T01:55:06.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.716+0000 7f6459d9b640 1 -- 172.21.15.115:0/3803754897 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444031940 con 0x7f645c0b5d80 2023-12-01T01:55:06.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.716+0000 7f6459d9b640 1 -- 172.21.15.115:0/3803754897 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444031cd0 con 0x7f645c0b5d80 2023-12-01T01:55:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.716+0000 7f6460c38640 1 -- 172.21.15.115:0/3803754897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0b5d80 msgr2=0x7f645c0b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.716+0000 7f6460c38640 1 --2- 172.21.15.115:0/3803754897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0b5d80 0x7f645c0b8570 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f645c08bdb0 tx=0x7f644402b880 comp rx=0 tx=0).stop 2023-12-01T01:55:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.716+0000 7f6460c38640 1 -- 172.21.15.115:0/3803754897 shutdown_connections 2023-12-01T01:55:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.716+0000 7f6460c38640 1 --2- 172.21.15.115:0/3803754897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0b5d80 0x7f645c0b8570 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.716+0000 7f6460c38640 1 --2- 172.21.15.115:0/3803754897 >> v2:172.21.15.142:3300/0 conn(0x7f645c15cd20 0x7f645c0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.716+0000 7f6460c38640 1 -- 172.21.15.115:0/3803754897 >> 172.21.15.115:0/3803754897 conn(0x7f645c06a0e0 msgr2=0x7f645c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:06.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.717+0000 7f6460c38640 1 -- 172.21.15.115:0/3803754897 shutdown_connections 2023-12-01T01:55:06.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.717+0000 7f6460c38640 1 -- 172.21.15.115:0/3803754897 wait complete. 2023-12-01T01:55:06.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.718+0000 7f6460c38640 1 Processor -- start 2023-12-01T01:55:06.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.718+0000 7f6460c38640 1 -- start start 2023-12-01T01:55:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.719+0000 7f6460c38640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645c0b5d80 0x7f645c0c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.719+0000 7f6460c38640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0d1230 0x7f645c154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.719+0000 7f6460c38640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f645c15cd20 0x7f645c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.719+0000 7f6460c38640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0c6c90 con 0x7f645c0d1230 2023-12-01T01:55:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.719+0000 7f6460c38640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c15d8f0 con 0x7f645c0b5d80 2023-12-01T01:55:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.719+0000 7f6460c38640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c154740 con 0x7f645c15cd20 2023-12-01T01:55:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f6453fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0d1230 0x7f645c154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:06.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f6453fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0d1230 0x7f645c154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44150/0 (socket says 172.21.15.115:44150) 2023-12-01T01:55:06.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f6453fff640 1 -- 172.21.15.115:0/2597130595 learned_addr learned my addr 172.21.15.115:0/2597130595 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:06.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f645a59c640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645c0b5d80 0x7f645c0c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:06.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f6453fff640 1 -- 172.21.15.115:0/2597130595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f645c15cd20 msgr2=0x7f645c155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:06.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f645ad9d640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f645c15cd20 0x7f645c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:06.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f6453fff640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f645c15cd20 0x7f645c155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f6453fff640 1 -- 172.21.15.115:0/2597130595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645c0b5d80 msgr2=0x7f645c0c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:06.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f6453fff640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645c0b5d80 0x7f645c0c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f6453fff640 1 -- 172.21.15.115:0/2597130595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64440094d0 con 0x7f645c0d1230 2023-12-01T01:55:06.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f645a59c640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645c0b5d80 0x7f645c0c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:55:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.720+0000 7f645ad9d640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f645c15cd20 0x7f645c155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.721+0000 7f6453fff640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0d1230 0x7f645c154200 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6444000c00 tx=0x7f64440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.721+0000 7f6452ffd640 1 -- 172.21.15.115:0/2597130595 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444032880 con 0x7f645c0d1230 2023-12-01T01:55:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.721+0000 7f6452ffd640 1 -- 172.21.15.115:0/2597130595 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444032a00 con 0x7f645c0d1230 2023-12-01T01:55:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.721+0000 7f6460c38640 1 -- 172.21.15.115:0/2597130595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c0b3800 con 0x7f645c0d1230 2023-12-01T01:55:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.721+0000 7f6452ffd640 1 -- 172.21.15.115:0/2597130595 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444032db0 con 0x7f645c0d1230 2023-12-01T01:55:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.721+0000 7f6460c38640 1 -- 172.21.15.115:0/2597130595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c0b3dc0 con 0x7f645c0d1230 2023-12-01T01:55:06.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.722+0000 7f6460c38640 1 -- 172.21.15.115:0/2597130595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c15daa0 con 0x7f645c0d1230 2023-12-01T01:55:06.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.724+0000 7f6452ffd640 1 -- 172.21.15.115:0/2597130595 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 68) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f644404d3f0 con 0x7f645c0d1230 2023-12-01T01:55:06.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.725+0000 7f6452ffd640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643c06a670 0x7f643c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:06.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.725+0000 7f6452ffd640 1 -- 172.21.15.115:0/2597130595 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f64440aac50 con 0x7f645c0d1230 2023-12-01T01:55:06.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.725+0000 7f645a59c640 1 -- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643c06a670 msgr2=0x7f643c06cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:06.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.725+0000 7f645a59c640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643c06a670 0x7f643c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:06.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.729+0000 7f6452ffd640 1 -- 172.21.15.115:0/2597130595 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f645c15daa0 con 0x7f645c0d1230 2023-12-01T01:55:06.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.926+0000 7f645a59c640 1 -- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643c06a670 msgr2=0x7f643c06cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:06.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.926+0000 7f645a59c640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643c06a670 0x7f643c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:55:06.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.957+0000 7f6460c38640 1 -- 172.21.15.115:0/2597130595 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f645c15daa0 con 0x7f645c0d1230 2023-12-01T01:55:06.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.958+0000 7f6452ffd640 1 -- 172.21.15.115:0/2597130595 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f645c15daa0 con 0x7f645c0d1230 2023-12-01T01:55:06.958 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:06.958 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/4122088547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":4122088547},{"type":"v1","addr":"172.21.15.142:6834","nonce":4122088547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/299645728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":299645728},{"type":"v1","addr":"172.21.15.142:6835","nonce":299645728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7983,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/150770274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":150770274},{"type":"v1","addr":"172.21.15.115:6836","nonce":150770274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:54:33.047892+0000","modified":"2023-12-01T01:54:36.558949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/2398273191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":2398273191},{"type":"v1","addr":"172.21.15.115:6837","nonce":2398273191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-01T01:55:06.959 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 56 2023-12-01T01:55:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.960+0000 7f6460c38640 1 -- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643c06a670 msgr2=0x7f643c06cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.960+0000 7f6460c38640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643c06a670 0x7f643c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.960+0000 7f6460c38640 1 -- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0d1230 msgr2=0x7f645c154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.960+0000 7f6460c38640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0d1230 0x7f645c154200 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6444000c00 tx=0x7f64440046c0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.961+0000 7f6460c38640 1 -- 172.21.15.115:0/2597130595 shutdown_connections 2023-12-01T01:55:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.961+0000 7f6460c38640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f643c06a670 0x7f643c06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.961+0000 7f6460c38640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f645c15cd20 0x7f645c155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.961+0000 7f6460c38640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f645c0d1230 0x7f645c154200 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.961+0000 7f6460c38640 1 --2- 172.21.15.115:0/2597130595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f645c0b5d80 0x7f645c0c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:06.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.961+0000 7f6460c38640 1 -- 172.21.15.115:0/2597130595 >> 172.21.15.115:0/2597130595 conn(0x7f645c06a0e0 msgr2=0x7f645c157500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:06.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.961+0000 7f6460c38640 1 -- 172.21.15.115:0/2597130595 shutdown_connections 2023-12-01T01:55:06.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:06.961+0000 7f6460c38640 1 -- 172.21.15.115:0/2597130595 wait complete. 2023-12-01T01:55:06.979 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:55:07.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.208+0000 7fcc091d5640 1 Processor -- start 2023-12-01T01:55:07.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.208+0000 7fcc091d5640 1 -- start start 2023-12-01T01:55:07.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.209+0000 7fcc091d5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcc0406d220 0x7fcc0406d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.209+0000 7fcc091d5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcc0406de30 0x7fcc04070220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.209+0000 7fcc091d5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0415d570 con 0x7fcc040ae4f0 2023-12-01T01:55:07.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.209+0000 7fcc091d5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc040cb510 con 0x7fcc0406de30 2023-12-01T01:55:07.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.209+0000 7fcc091d5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0406e240 con 0x7fcc0406d220 2023-12-01T01:55:07.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcc0359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcc0406de30 0x7fcc04070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcbfbfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fcc040ae4f0 0x7fcc0406fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51220/0 (socket says 172.21.15.115:51220) 2023-12-01T01:55:07.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcbfbfff640 1 -- 172.21.15.115:0/992031858 learned_addr learned my addr 172.21.15.115:0/992031858 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:07.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcc0359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcc0406de30 0x7fcc04070220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44166/0 (socket says 172.21.15.115:44166) 2023-12-01T01:55:07.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcc02d9d640 1 --2- 172.21.15.115:0/992031858 >> v2:172.21.15.142:3300/0 conn(0x7fcc0406d220 0x7fcc0406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcc0359e640 1 -- 172.21.15.115:0/992031858 >> v2:172.21.15.142:3300/0 conn(0x7fcc0406d220 msgr2=0x7fcc0406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:07.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcc0359e640 1 --2- 172.21.15.115:0/992031858 >> v2:172.21.15.142:3300/0 conn(0x7fcc0406d220 0x7fcc0406d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcc0359e640 1 -- 172.21.15.115:0/992031858 >> v1:172.21.15.115:6789/0 conn(0x7fcc040ae4f0 legacy=0x7fcc0406fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:07.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcc0359e640 1 -- 172.21.15.115:0/992031858 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040cb230 con 0x7fcc0406de30 2023-12-01T01:55:07.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcc02d9d640 1 --2- 172.21.15.115:0/992031858 >> v2:172.21.15.142:3300/0 conn(0x7fcc0406d220 0x7fcc0406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:07.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.210+0000 7fcc0259c640 1 -- 172.21.15.115:0/992031858 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622075083 0 0) 0x7fcc0415d570 con 0x7fcc040ae4f0 2023-12-01T01:55:07.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.211+0000 7fcc0359e640 1 --2- 172.21.15.115:0/992031858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc0406de30 0x7fcc04070220 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fcc0406f6a0 tx=0x7fcbfc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f8e80da0b50a00 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.211+0000 7fcc0259c640 1 -- 172.21.15.115:0/992031858 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc02cbf0 con 0x7fcc0406de30 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.212+0000 7fcc0259c640 1 -- 172.21.15.115:0/992031858 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc02cd50 con 0x7fcc0406de30 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.212+0000 7fcc0259c640 1 -- 172.21.15.115:0/992031858 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc034660 con 0x7fcc0406de30 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.212+0000 7fcc091d5640 1 -- 172.21.15.115:0/992031858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc0406de30 msgr2=0x7fcc04070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.212+0000 7fcc091d5640 1 --2- 172.21.15.115:0/992031858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc0406de30 0x7fcc04070220 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fcc0406f6a0 tx=0x7fcbfc02be40 comp rx=0 tx=0).stop 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.212+0000 7fcc091d5640 1 -- 172.21.15.115:0/992031858 shutdown_connections 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.212+0000 7fcc091d5640 1 --2- 172.21.15.115:0/992031858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc0406de30 0x7fcc04070220 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.212+0000 7fcc091d5640 1 --2- 172.21.15.115:0/992031858 >> v2:172.21.15.142:3300/0 conn(0x7fcc0406d220 0x7fcc0406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.212+0000 7fcc091d5640 1 -- 172.21.15.115:0/992031858 >> 172.21.15.115:0/992031858 conn(0x7fcc040cbb10 msgr2=0x7fcc040cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.212+0000 7fcc091d5640 1 -- 172.21.15.115:0/992031858 shutdown_connections 2023-12-01T01:55:07.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.212+0000 7fcc091d5640 1 -- 172.21.15.115:0/992031858 wait complete. 2023-12-01T01:55:07.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.214+0000 7fcc091d5640 1 Processor -- start 2023-12-01T01:55:07.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.214+0000 7fcc091d5640 1 -- start start 2023-12-01T01:55:07.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcc091d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0406d220 0x7fcc0415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcc091d5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0406de30 0x7fcc04153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcc091d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc040ae4f0 0x7fcc0415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcc091d5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0415b4e0 con 0x7fcc040ae4f0 2023-12-01T01:55:07.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcc091d5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc041586c0 con 0x7fcc0406de30 2023-12-01T01:55:07.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcc091d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc04153c40 con 0x7fcc0406d220 2023-12-01T01:55:07.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcc0359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc040ae4f0 0x7fcc0415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcc0359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc040ae4f0 0x7fcc0415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44168/0 (socket says 172.21.15.115:44168) 2023-12-01T01:55:07.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcc0359e640 1 -- 172.21.15.115:0/1962372374 learned_addr learned my addr 172.21.15.115:0/1962372374 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:07.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcc02d9d640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0406d220 0x7fcc0415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.215+0000 7fcbfbfff640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0406de30 0x7fcc04153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcc0359e640 1 -- 172.21.15.115:0/1962372374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0406d220 msgr2=0x7fcc0415afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:07.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcc0359e640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0406d220 0x7fcc0415afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcc0359e640 1 -- 172.21.15.115:0/1962372374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0406de30 msgr2=0x7fcc04153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:07.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcc0359e640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0406de30 0x7fcc04153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcc0359e640 1 -- 172.21.15.115:0/1962372374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc009a70 con 0x7fcc040ae4f0 2023-12-01T01:55:07.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcc0359e640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc040ae4f0 0x7fcc0415c280 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009e80 tx=0x7fcbfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:07.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcbfb7fe640 1 -- 172.21.15.115:0/1962372374 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc040650 con 0x7fcc040ae4f0 2023-12-01T01:55:07.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcbfb7fe640 1 -- 172.21.15.115:0/1962372374 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc0407d0 con 0x7fcc040ae4f0 2023-12-01T01:55:07.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcbfb7fe640 1 -- 172.21.15.115:0/1962372374 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc040b80 con 0x7fcc040ae4f0 2023-12-01T01:55:07.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcc091d5640 1 -- 172.21.15.115:0/1962372374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc040b48d0 con 0x7fcc040ae4f0 2023-12-01T01:55:07.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.216+0000 7fcc091d5640 1 -- 172.21.15.115:0/1962372374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc04158980 con 0x7fcc040ae4f0 2023-12-01T01:55:07.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.217+0000 7fcc091d5640 1 -- 172.21.15.115:0/1962372374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc040b4df0 con 0x7fcc040ae4f0 2023-12-01T01:55:07.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.219+0000 7fcbfb7fe640 1 -- 172.21.15.115:0/1962372374 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 68) v1 ==== 88823+0+0 (secure 0 0 0) 0x7fcbfc040ce0 con 0x7fcc040ae4f0 2023-12-01T01:55:07.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.219+0000 7fcbfb7fe640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a670 0x7fcbe406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.220+0000 7fcbfb7fe640 1 -- 172.21.15.115:0/1962372374 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fcbfc0abee0 con 0x7fcc040ae4f0 2023-12-01T01:55:07.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.220+0000 7fcc02d9d640 1 -- 172.21.15.115:0/1962372374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a670 msgr2=0x7fcbe406cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:07.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.220+0000 7fcc02d9d640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a670 0x7fcbe406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:07.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.224+0000 7fcbfb7fe640 1 -- 172.21.15.115:0/1962372374 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcbfc076f70 con 0x7fcc040ae4f0 2023-12-01T01:55:07.303 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:07.301+0000 7f762883a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:55:07.392 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.392+0000 7f3a6d640040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:55:07.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.414+0000 7fcc091d5640 1 -- 172.21.15.115:0/1962372374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc040aca10 con 0x7fcc040ae4f0 2023-12-01T01:55:07.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.416+0000 7fcbfb7fe640 1 -- 172.21.15.115:0/1962372374 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16595 (secure 0 0 0) 0x7fcbfc07a590 con 0x7fcc040ae4f0 2023-12-01T01:55:07.417 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:07.417 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":126,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:54:47.777124+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:54:30.872492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-01T01:54:32.074780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:54:47.777090+0000","172.21.15.115:6822/88317":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/2560191707":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/31572672":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/1289974123":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/3424983653":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/3883676082":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/841702221":"2023-12-02T01:54:47.777090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:55:07.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 -- 172.21.15.115:0/1962372374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a670 msgr2=0x7fcbe406cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a670 0x7fcbe406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 -- 172.21.15.115:0/1962372374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc040ae4f0 msgr2=0x7fcc0415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc040ae4f0 0x7fcc0415c280 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009e80 tx=0x7fcbfc004a40 comp rx=0 tx=0).stop 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 -- 172.21.15.115:0/1962372374 shutdown_connections 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcbe406a670 0x7fcbe406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc040ae4f0 0x7fcc0415c280 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc0406de30 0x7fcc04153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 --2- 172.21.15.115:0/1962372374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc0406d220 0x7fcc0415afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 -- 172.21.15.115:0/1962372374 >> 172.21.15.115:0/1962372374 conn(0x7fcc040cbb10 msgr2=0x7fcc040c96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 -- 172.21.15.115:0/1962372374 shutdown_connections 2023-12-01T01:55:07.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.419+0000 7fcc091d5640 1 -- 172.21.15.115:0/1962372374 wait complete. 2023-12-01T01:55:07.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:54:30.872492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:54:32.074780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:55:07.438 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:55:07.598 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:07.596+0000 7f762883a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:55:07.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.654+0000 7f57ad4af640 1 Processor -- start 2023-12-01T01:55:07.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.654+0000 7f57ad4af640 1 -- start start 2023-12-01T01:55:07.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.656+0000 7f57ad4af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a80cfd60 0x7f57a80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.656+0000 7f57ad4af640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f57a80b5fa0 0x7f57a8161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.656+0000 7f57ad4af640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a80ce5e0 con 0x7f57a815cd90 2023-12-01T01:55:07.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.656+0000 7f57ad4af640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a80d0ec0 con 0x7f57a80b5fa0 2023-12-01T01:55:07.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.656+0000 7f57ad4af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8161950 con 0x7f57a80cfd60 2023-12-01T01:55:07.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.656+0000 7f57a6ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f57a815cd90 0x7f57a80cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34922/0 (socket says 172.21.15.115:34922) 2023-12-01T01:55:07.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.656+0000 7f57a6ffd640 1 -- 172.21.15.115:0/1386461133 learned_addr learned my addr 172.21.15.115:0/1386461133 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.656+0000 7f579ffff640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a80cfd60 0x7f57a80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a77fe640 1 --2- 172.21.15.115:0/1386461133 >> v2:172.21.15.142:3300/0 conn(0x7f57a80b5fa0 0x7f57a8161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259613023 0 0) 0x7f5798001110 con 0x7f57a815cd90 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5794003520 con 0x7f57a815cd90 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2354284475 0 0) 0x7f5794003520 con 0x7f57a815cd90 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a80cfd60 msgr2=0x7f57a80cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a80cfd60 0x7f57a80cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 >> v2:172.21.15.142:3300/0 conn(0x7f57a80b5fa0 msgr2=0x7f57a8161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 --2- 172.21.15.115:0/1386461133 >> v2:172.21.15.142:3300/0 conn(0x7f57a80b5fa0 0x7f57a8161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80ce300 con 0x7f57a815cd90 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5798001800 con 0x7f57a815cd90 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 >> v1:172.21.15.115:6789/0 conn(0x7f57a815cd90 legacy=0x7f57a80cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:07.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a80cfd60 0x7f5794003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5794003ef0 0x7f57940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5794004d00 0x7f5794005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80d0ec0 con 0x7f5794003ef0 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8161950 con 0x7f5794004d00 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579400b9d0 con 0x7f57a80cfd60 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.657+0000 7f579ffff640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5794003ef0 0x7f57940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.658+0000 7f57a77fe640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5794004d00 0x7f5794005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.658+0000 7f57a6ffd640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a80cfd60 0x7f5794003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.658+0000 7f579ffff640 1 -- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a80cfd60 msgr2=0x7f5794003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.658+0000 7f579ffff640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a80cfd60 0x7f5794003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.658+0000 7f579ffff640 1 -- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5794004d00 msgr2=0x7f5794005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.658+0000 7f579ffff640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5794004d00 0x7f5794005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.658+0000 7f579ffff640 1 -- 172.21.15.115:0/1386461133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5794003520 con 0x7f5794003ef0 2023-12-01T01:55:07.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.658+0000 7f57a77fe640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5794004d00 0x7f5794005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:07.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.659+0000 7f579ffff640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5794003ef0 0x7f57940047c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5790009af0 tx=0x7f57940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c16c9d107e01e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:07.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.659+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579002c6b0 con 0x7f5794003ef0 2023-12-01T01:55:07.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.659+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579002c810 con 0x7f5794003ef0 2023-12-01T01:55:07.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.659+0000 7f57a67fc640 1 -- 172.21.15.115:0/1386461133 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579002cba0 con 0x7f5794003ef0 2023-12-01T01:55:07.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.660+0000 7f57ad4af640 1 -- 172.21.15.115:0/1386461133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5794003ef0 msgr2=0x7f57940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:07.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.660+0000 7f57ad4af640 1 --2- 172.21.15.115:0/1386461133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5794003ef0 0x7f57940047c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5790009af0 tx=0x7f57940057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.660+0000 7f57a77fe640 1 -- 172.21.15.115:0/1386461133 reap_dead start 2023-12-01T01:55:07.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.660+0000 7f57ad4af640 1 -- 172.21.15.115:0/1386461133 shutdown_connections 2023-12-01T01:55:07.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.660+0000 7f57ad4af640 1 -- 172.21.15.115:0/1386461133 >> 172.21.15.115:0/1386461133 conn(0x7f57a8065960 msgr2=0x7f57a8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:07.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.660+0000 7f57ad4af640 1 -- 172.21.15.115:0/1386461133 shutdown_connections 2023-12-01T01:55:07.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.660+0000 7f57ad4af640 1 -- 172.21.15.115:0/1386461133 wait complete. 2023-12-01T01:55:07.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.661+0000 7f57ad4af640 1 Processor -- start 2023-12-01T01:55:07.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.662+0000 7f57ad4af640 1 -- start start 2023-12-01T01:55:07.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.662+0000 7f57ad4af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5794003ef0 0x7f57a80c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57ad4af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a81f64f0 0x7f57a815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57ad4af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a81f58c0 0x7f57a81f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57ad4af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81f62b0 con 0x7f57a81f58c0 2023-12-01T01:55:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57ad4af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a815ac40 con 0x7f57a81f64f0 2023-12-01T01:55:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57ad4af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a815af20 con 0x7f5794003ef0 2023-12-01T01:55:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57a77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a81f58c0 0x7f57a81f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57a77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a81f58c0 0x7f57a81f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:59990/0 (socket says 172.21.15.115:59990) 2023-12-01T01:55:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57a77fe640 1 -- 172.21.15.115:0/1288801947 learned_addr learned my addr 172.21.15.115:0/1288801947 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f579ffff640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a81f64f0 0x7f57a815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57a6ffd640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5794003ef0 0x7f57a80c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:07.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57a77fe640 1 -- 172.21.15.115:0/1288801947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5794003ef0 msgr2=0x7f57a80c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57a77fe640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5794003ef0 0x7f57a80c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57a77fe640 1 -- 172.21.15.115:0/1288801947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a81f64f0 msgr2=0x7f57a815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57a77fe640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a81f64f0 0x7f57a815d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f57a77fe640 1 -- 172.21.15.115:0/1288801947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a815b200 con 0x7f57a81f58c0 2023-12-01T01:55:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.663+0000 7f579ffff640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a81f64f0 0x7f57a815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.664+0000 7f57a77fe640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a81f58c0 0x7f57a81f5d70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f57a8071ac0 tx=0x7f57a000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.664+0000 7f579f7fe640 1 -- 172.21.15.115:0/1288801947 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0017400 con 0x7f57a81f58c0 2023-12-01T01:55:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.664+0000 7f579f7fe640 1 -- 172.21.15.115:0/1288801947 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a0011040 con 0x7f57a81f58c0 2023-12-01T01:55:07.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.664+0000 7f57ad4af640 1 -- 172.21.15.115:0/1288801947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a815b4e0 con 0x7f57a81f58c0 2023-12-01T01:55:07.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.664+0000 7f579f7fe640 1 -- 172.21.15.115:0/1288801947 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0017780 con 0x7f57a81f58c0 2023-12-01T01:55:07.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.664+0000 7f57ad4af640 1 -- 172.21.15.115:0/1288801947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a815baa0 con 0x7f57a81f58c0 2023-12-01T01:55:07.668 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:07.665+0000 7f762883a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:55:07.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.665+0000 7f57ad4af640 1 -- 172.21.15.115:0/1288801947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a804e890 con 0x7f57a81f58c0 2023-12-01T01:55:07.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.666+0000 7f579f7fe640 1 -- 172.21.15.115:0/1288801947 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 68) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f57a00203b0 con 0x7f57a81f58c0 2023-12-01T01:55:07.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.667+0000 7f579f7fe640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f578806a670 0x7f578806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:07.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.668+0000 7f57a6ffd640 1 -- 172.21.15.115:0/1288801947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f578806a670 msgr2=0x7f578806cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:07.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.668+0000 7f579f7fe640 1 -- 172.21.15.115:0/1288801947 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f57a0089360 con 0x7f57a81f58c0 2023-12-01T01:55:07.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.668+0000 7f57a6ffd640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f578806a670 0x7f578806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:07.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.671+0000 7f579f7fe640 1 -- 172.21.15.115:0/1288801947 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f57a00542b0 con 0x7f57a81f58c0 2023-12-01T01:55:07.728 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.728+0000 7f3a6d640040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:55:07.782 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:07.780+0000 7f762883a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:55:07.787 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.787+0000 7f3a6d640040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:55:07.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.857+0000 7f57ad4af640 1 -- 172.21.15.115:0/1288801947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57a80d0a70 con 0x7f57a81f58c0 2023-12-01T01:55:07.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.858+0000 7f579f7fe640 1 -- 172.21.15.115:0/1288801947 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57a00578d0 con 0x7f57a81f58c0 2023-12-01T01:55:07.859 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:55:07.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.860+0000 7f57ad4af640 1 -- 172.21.15.115:0/1288801947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f578806a670 msgr2=0x7f578806cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:07.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.860+0000 7f57ad4af640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f578806a670 0x7f578806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.861+0000 7f57ad4af640 1 -- 172.21.15.115:0/1288801947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a81f58c0 msgr2=0x7f57a81f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:07.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.861+0000 7f57ad4af640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a81f58c0 0x7f57a81f5d70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f57a8071ac0 tx=0x7f57a000f4a0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.861+0000 7f57ad4af640 1 -- 172.21.15.115:0/1288801947 shutdown_connections 2023-12-01T01:55:07.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.861+0000 7f57ad4af640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f578806a670 0x7f578806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.861+0000 7f57ad4af640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a81f58c0 0x7f57a81f5d70 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.861+0000 7f57ad4af640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a81f64f0 0x7f57a815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.861+0000 7f57ad4af640 1 --2- 172.21.15.115:0/1288801947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5794003ef0 0x7f57a80c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:07.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.861+0000 7f57ad4af640 1 -- 172.21.15.115:0/1288801947 >> 172.21.15.115:0/1288801947 conn(0x7f57a8065960 msgr2=0x7f57a8158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:07.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.861+0000 7f57ad4af640 1 -- 172.21.15.115:0/1288801947 shutdown_connections 2023-12-01T01:55:07.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:07.861+0000 7f57ad4af640 1 -- 172.21.15.115:0/1288801947 wait complete. 2023-12-01T01:55:07.880 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:55:07.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.904+0000 7f3a6d640040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:55:07.986 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.986+0000 7f39f0134640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:07.986 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.986+0000 7f39f0134640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:07.986 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.986+0000 7f39f0134640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:07.987 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.986+0000 7f39f0134640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:07.987 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.986+0000 7f39f0134640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:07.987 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.986+0000 7f39f313a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:07.987 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.986+0000 7f39f313a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:07.987 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.986+0000 7f39f313a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:07.988 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.986+0000 7f39f313a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:07.988 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:07.986+0000 7f39f313a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.090+0000 7f018e99f640 1 Processor -- start 2023-12-01T01:55:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.090+0000 7f018e99f640 1 -- start start 2023-12-01T01:55:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.091+0000 7f018e99f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f01880ccca0 0x7f018806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.091+0000 7f018e99f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01880cb100 0x7f018806a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.091+0000 7f018e99f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f01880d0c40 0x7f01880d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.091+0000 7f018e99f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01880d07b0 con 0x7f01880d0c40 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.091+0000 7f018e99f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01880cb560 con 0x7f01880ccca0 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.091+0000 7f018e99f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018806af00 con 0x7f01880cb100 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.092+0000 7f018cf3c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f01880d0c40 0x7f01880d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.092+0000 7f018cf3c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f01880d0c40 0x7f01880d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60094/0 (socket says 172.21.15.115:60094) 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.092+0000 7f018cf3c640 1 -- 172.21.15.115:0/4034971064 learned_addr learned my addr 172.21.15.115:0/4034971064 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.092+0000 7f018cf3c640 1 -- 172.21.15.115:0/4034971064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01880cb100 msgr2=0x7f018806a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.092+0000 7f018cf3c640 1 --2- 172.21.15.115:0/4034971064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01880cb100 0x7f018806a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.092+0000 7f0187fff640 1 --2- 172.21.15.115:0/4034971064 >> v2:172.21.15.142:3300/0 conn(0x7f01880ccca0 0x7f018806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.092+0000 7f018cf3c640 1 -- 172.21.15.115:0/4034971064 >> v2:172.21.15.142:3300/0 conn(0x7f01880ccca0 msgr2=0x7f018806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.092+0000 7f018cf3c640 1 --2- 172.21.15.115:0/4034971064 >> v2:172.21.15.142:3300/0 conn(0x7f01880ccca0 0x7f018806b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.092+0000 7f018cf3c640 1 -- 172.21.15.115:0/4034971064 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01880d0160 con 0x7f01880d0c40 2023-12-01T01:55:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.093+0000 7f018cf3c640 1 --2- 172.21.15.115:0/4034971064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 0x7f01880d10d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f01880d1b40 tx=0x7f018002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43198d5d78e23d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.093+0000 7f01877fe640 1 -- 172.21.15.115:0/4034971064 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018002ca70 con 0x7f01880d0c40 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.093+0000 7f01877fe640 1 -- 172.21.15.115:0/4034971064 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018002cbd0 con 0x7f01880d0c40 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.093+0000 7f01877fe640 1 -- 172.21.15.115:0/4034971064 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01800343f0 con 0x7f01880d0c40 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.094+0000 7f018e99f640 1 -- 172.21.15.115:0/4034971064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 msgr2=0x7f01880d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.094+0000 7f018e99f640 1 --2- 172.21.15.115:0/4034971064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 0x7f01880d10d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f01880d1b40 tx=0x7f018002bed0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.094+0000 7f018e99f640 1 -- 172.21.15.115:0/4034971064 shutdown_connections 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.094+0000 7f018e99f640 1 --2- 172.21.15.115:0/4034971064 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 0x7f01880d10d0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.094+0000 7f018e99f640 1 --2- 172.21.15.115:0/4034971064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01880cb100 0x7f018806a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.094+0000 7f018e99f640 1 --2- 172.21.15.115:0/4034971064 >> v2:172.21.15.142:3300/0 conn(0x7f01880ccca0 0x7f018806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.094+0000 7f018e99f640 1 -- 172.21.15.115:0/4034971064 >> 172.21.15.115:0/4034971064 conn(0x7f01880713d0 msgr2=0x7f01880717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.094+0000 7f018e99f640 1 -- 172.21.15.115:0/4034971064 shutdown_connections 2023-12-01T01:55:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.094+0000 7f018e99f640 1 -- 172.21.15.115:0/4034971064 wait complete. 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.096+0000 7f018e99f640 1 Processor -- start 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.096+0000 7f018e99f640 1 -- start start 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.097+0000 7f018e99f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01880cb100 0x7f0188147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.097+0000 7f018e99f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01880ccca0 0x7f01881f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.097+0000 7f018e99f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 0x7f01881f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.097+0000 7f018e99f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01880c8a70 con 0x7f01880d0c40 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.097+0000 7f018e99f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01881f2660 con 0x7f01880cb100 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.097+0000 7f018e99f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01880c7b80 con 0x7f01880ccca0 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.097+0000 7f018cf3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 0x7f01881f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.097+0000 7f018cf3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 0x7f01881f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60098/0 (socket says 172.21.15.115:60098) 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.097+0000 7f018cf3c640 1 -- 172.21.15.115:0/1228615740 learned_addr learned my addr 172.21.15.115:0/1228615740 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f0187fff640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01880cb100 0x7f0188147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f018cf3c640 1 -- 172.21.15.115:0/1228615740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01880ccca0 msgr2=0x7f01881f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f017ffff640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01880ccca0 0x7f01881f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f018cf3c640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01880ccca0 0x7f01881f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f018cf3c640 1 -- 172.21.15.115:0/1228615740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01880cb100 msgr2=0x7f0188147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f018cf3c640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01880cb100 0x7f0188147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f018cf3c640 1 -- 172.21.15.115:0/1228615740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0180009b30 con 0x7f01880d0c40 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f0187fff640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01880cb100 0x7f0188147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f017ffff640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01880ccca0 0x7f01881f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f018cf3c640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 0x7f01881f2120 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f0180036040 tx=0x7f0180004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f01857fa640 1 -- 172.21.15.115:0/1228615740 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018003e450 con 0x7f01880d0c40 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f01857fa640 1 -- 172.21.15.115:0/1228615740 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018003e5d0 con 0x7f01880d0c40 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f01857fa640 1 -- 172.21.15.115:0/1228615740 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018003e980 con 0x7f01880d0c40 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f018e99f640 1 -- 172.21.15.115:0/1228615740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01880c7e60 con 0x7f01880d0c40 2023-12-01T01:55:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.098+0000 7f018e99f640 1 -- 172.21.15.115:0/1228615740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01880c8420 con 0x7f01880d0c40 2023-12-01T01:55:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.100+0000 7f018e99f640 1 -- 172.21.15.115:0/1228615740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01880c8c20 con 0x7f01880d0c40 2023-12-01T01:55:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.100+0000 7f01857fa640 1 -- 172.21.15.115:0/1228615740 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 69) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f018003eae0 con 0x7f01880d0c40 2023-12-01T01:55:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.101+0000 7f01857fa640 1 -- 172.21.15.115:0/1228615740 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f01800ab570 con 0x7f01880d0c40 2023-12-01T01:55:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.105+0000 7f01857fa640 1 -- 172.21.15.115:0/1228615740 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f01880c8c20 con 0x7f01880d0c40 2023-12-01T01:55:08.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.281+0000 7f018e99f640 1 -- 172.21.15.115:0/1228615740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f018806be50 con 0x7f01880d0c40 2023-12-01T01:55:08.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.282+0000 7f01857fa640 1 -- 172.21.15.115:0/1228615740 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0180076ce0 con 0x7f01880d0c40 2023-12-01T01:55:08.282 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T01:55:08.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.284+0000 7f018e99f640 1 -- 172.21.15.115:0/1228615740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 msgr2=0x7f01881f2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:08.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.284+0000 7f018e99f640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 0x7f01881f2120 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f0180036040 tx=0x7f0180004a40 comp rx=0 tx=0).stop 2023-12-01T01:55:08.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.284+0000 7f018e99f640 1 -- 172.21.15.115:0/1228615740 shutdown_connections 2023-12-01T01:55:08.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.284+0000 7f018e99f640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01880ccca0 0x7f01881f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.284+0000 7f018e99f640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01880d0c40 0x7f01881f2120 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.284+0000 7f018e99f640 1 --2- 172.21.15.115:0/1228615740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01880cb100 0x7f0188147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.284+0000 7f018e99f640 1 -- 172.21.15.115:0/1228615740 >> 172.21.15.115:0/1228615740 conn(0x7f01880713d0 msgr2=0x7f018806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:08.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.285+0000 7f018e99f640 1 -- 172.21.15.115:0/1228615740 shutdown_connections 2023-12-01T01:55:08.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.285+0000 7f018e99f640 1 -- 172.21.15.115:0/1228615740 wait complete. 2023-12-01T01:55:08.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:55:08.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.560+0000 7fee69192640 1 Processor -- start 2023-12-01T01:55:08.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.560+0000 7fee69192640 1 -- start start 2023-12-01T01:55:08.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.562+0000 7fee69192640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fee64143eb0 0x7fee6406b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.562+0000 7fee69192640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fee640cbc80 0x7fee64164c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.562+0000 7fee69192640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee640ce1b0 con 0x7fee6406b7a0 2023-12-01T01:55:08.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.562+0000 7fee69192640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee640ce490 con 0x7fee64143eb0 2023-12-01T01:55:08.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.562+0000 7fee69192640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee640caa90 con 0x7fee640cbc80 2023-12-01T01:55:08.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.562+0000 7fee62d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fee64143eb0 0x7fee6406b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:08.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.562+0000 7fee62d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fee64143eb0 0x7fee6406b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60112/0 (socket says 172.21.15.115:60112) 2023-12-01T01:55:08.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.562+0000 7fee62d9d640 1 -- 172.21.15.115:0/1685164580 learned_addr learned my addr 172.21.15.115:0/1685164580 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:08.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.562+0000 7fee6359e640 1 --2- 172.21.15.115:0/1685164580 >> v2:172.21.15.142:3300/0 conn(0x7fee640cbc80 0x7fee64164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:08.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.563+0000 7fee62d9d640 1 -- 172.21.15.115:0/1685164580 >> v2:172.21.15.142:3300/0 conn(0x7fee640cbc80 msgr2=0x7fee64164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:08.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.563+0000 7fee62d9d640 1 --2- 172.21.15.115:0/1685164580 >> v2:172.21.15.142:3300/0 conn(0x7fee640cbc80 0x7fee64164c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.563+0000 7fee62d9d640 1 -- 172.21.15.115:0/1685164580 >> v1:172.21.15.142:6789/0 conn(0x7fee6406b7a0 legacy=0x7fee640cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:08.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.563+0000 7fee62d9d640 1 -- 172.21.15.115:0/1685164580 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640cad70 con 0x7fee64143eb0 2023-12-01T01:55:08.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.563+0000 7fee6359e640 1 --2- 172.21.15.115:0/1685164580 >> v2:172.21.15.142:3300/0 conn(0x7fee640cbc80 0x7fee64164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:55:08.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.564+0000 7fee62d9d640 1 --2- 172.21.15.115:0/1685164580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 0x7fee6406b260 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fee640668d0 tx=0x7fee5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6164448df611e58b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:08.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.565+0000 7fee61d9b640 1 -- 172.21.15.115:0/1685164580 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54031870 con 0x7fee64143eb0 2023-12-01T01:55:08.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.565+0000 7fee61d9b640 1 -- 172.21.15.115:0/1685164580 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee540319d0 con 0x7fee64143eb0 2023-12-01T01:55:08.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.565+0000 7fee61d9b640 1 -- 172.21.15.115:0/1685164580 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54031d60 con 0x7fee64143eb0 2023-12-01T01:55:08.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.565+0000 7fee69192640 1 -- 172.21.15.115:0/1685164580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 msgr2=0x7fee6406b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:08.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.565+0000 7fee69192640 1 --2- 172.21.15.115:0/1685164580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 0x7fee6406b260 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fee640668d0 tx=0x7fee5402b910 comp rx=0 tx=0).stop 2023-12-01T01:55:08.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.565+0000 7fee69192640 1 -- 172.21.15.115:0/1685164580 shutdown_connections 2023-12-01T01:55:08.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.565+0000 7fee69192640 1 --2- 172.21.15.115:0/1685164580 >> v2:172.21.15.142:3300/0 conn(0x7fee640cbc80 0x7fee64164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.565+0000 7fee69192640 1 --2- 172.21.15.115:0/1685164580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 0x7fee6406b260 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.565+0000 7fee69192640 1 -- 172.21.15.115:0/1685164580 >> 172.21.15.115:0/1685164580 conn(0x7fee64065960 msgr2=0x7fee64065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:08.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.566+0000 7fee69192640 1 -- 172.21.15.115:0/1685164580 shutdown_connections 2023-12-01T01:55:08.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.566+0000 7fee69192640 1 -- 172.21.15.115:0/1685164580 wait complete. 2023-12-01T01:55:08.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.567+0000 7fee69192640 1 Processor -- start 2023-12-01T01:55:08.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.567+0000 7fee69192640 1 -- start start 2023-12-01T01:55:08.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.568+0000 7fee69192640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee6406b7a0 0x7fee641300c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.568+0000 7fee69192640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 0x7fee64163530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.568+0000 7fee69192640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee64154670 0x7fee64163a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:08.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.568+0000 7fee69192640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6413a310 con 0x7fee64143eb0 2023-12-01T01:55:08.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.568+0000 7fee69192640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641535d0 con 0x7fee64154670 2023-12-01T01:55:08.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.568+0000 7fee69192640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee641538b0 con 0x7fee6406b7a0 2023-12-01T01:55:08.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.568+0000 7fee6259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 0x7fee64163530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:08.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee6259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 0x7fee64163530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60122/0 (socket says 172.21.15.115:60122) 2023-12-01T01:55:08.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee6259c640 1 -- 172.21.15.115:0/1957103227 learned_addr learned my addr 172.21.15.115:0/1957103227 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:08.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee6259c640 1 -- 172.21.15.115:0/1957103227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee6406b7a0 msgr2=0x7fee641300c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:08.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee6259c640 1 --2- 172.21.15.115:0/1957103227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee6406b7a0 0x7fee641300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee6259c640 1 -- 172.21.15.115:0/1957103227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee64154670 msgr2=0x7fee64163a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:08.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee6259c640 1 --2- 172.21.15.115:0/1957103227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee64154670 0x7fee64163a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee6259c640 1 -- 172.21.15.115:0/1957103227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee540094d0 con 0x7fee64143eb0 2023-12-01T01:55:08.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee6259c640 1 --2- 172.21.15.115:0/1957103227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 0x7fee64163530 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fee5800c430 tx=0x7fee5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:08.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee4b7fe640 1 -- 172.21.15.115:0/1957103227 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58011790 con 0x7fee64143eb0 2023-12-01T01:55:08.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee4b7fe640 1 -- 172.21.15.115:0/1957103227 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee580118f0 con 0x7fee64143eb0 2023-12-01T01:55:08.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.569+0000 7fee4b7fe640 1 -- 172.21.15.115:0/1957103227 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58011ce0 con 0x7fee64143eb0 2023-12-01T01:55:08.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.570+0000 7fee69192640 1 -- 172.21.15.115:0/1957103227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee64153b90 con 0x7fee64143eb0 2023-12-01T01:55:08.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.570+0000 7fee69192640 1 -- 172.21.15.115:0/1957103227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee64154150 con 0x7fee64143eb0 2023-12-01T01:55:08.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.571+0000 7fee69192640 1 -- 172.21.15.115:0/1957103227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6404e890 con 0x7fee64143eb0 2023-12-01T01:55:08.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.571+0000 7fee4b7fe640 1 -- 172.21.15.115:0/1957103227 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 69) v1 ==== 88318+0+0 (secure 0 0 0) 0x7fee5801c460 con 0x7fee64143eb0 2023-12-01T01:55:08.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.572+0000 7fee4b7fe640 1 -- 172.21.15.115:0/1957103227 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fee58088660 con 0x7fee64143eb0 2023-12-01T01:55:08.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.576+0000 7fee4b7fe640 1 -- 172.21.15.115:0/1957103227 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fee58053d40 con 0x7fee64143eb0 2023-12-01T01:55:08.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.770+0000 7fee69192640 1 -- 172.21.15.115:0/1957103227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee64070140 con 0x7fee64143eb0 2023-12-01T01:55:08.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.771+0000 7fee4b7fe640 1 -- 172.21.15.115:0/1957103227 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee58057360 con 0x7fee64143eb0 2023-12-01T01:55:08.771 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T01:55:08.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.773+0000 7fee69192640 1 -- 172.21.15.115:0/1957103227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 msgr2=0x7fee64163530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:08.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.773+0000 7fee69192640 1 --2- 172.21.15.115:0/1957103227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 0x7fee64163530 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fee5800c430 tx=0x7fee5800c8f0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.774+0000 7fee69192640 1 -- 172.21.15.115:0/1957103227 shutdown_connections 2023-12-01T01:55:08.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.774+0000 7fee69192640 1 --2- 172.21.15.115:0/1957103227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee64154670 0x7fee64163a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.774+0000 7fee69192640 1 --2- 172.21.15.115:0/1957103227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee64143eb0 0x7fee64163530 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.774+0000 7fee69192640 1 --2- 172.21.15.115:0/1957103227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee6406b7a0 0x7fee641300c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:08.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.774+0000 7fee69192640 1 -- 172.21.15.115:0/1957103227 >> 172.21.15.115:0/1957103227 conn(0x7fee64065960 msgr2=0x7fee640cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:08.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.774+0000 7fee69192640 1 -- 172.21.15.115:0/1957103227 shutdown_connections 2023-12-01T01:55:08.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:08.774+0000 7fee69192640 1 -- 172.21.15.115:0/1957103227 wait complete. 2023-12-01T01:55:08.793 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:55:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.011+0000 7f949ffff640 1 Processor -- start 2023-12-01T01:55:09.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.011+0000 7f949ffff640 1 -- start start 2023-12-01T01:55:09.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f949ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f94a01584b0 0x7f94a00cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f949ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f94a015afe0 0x7f94a006b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f949ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94a006b7b0 0x7f94a015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f949ffff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a006bc40 con 0x7f94a015afe0 2023-12-01T01:55:09.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f949ffff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a00cc180 con 0x7f94a01584b0 2023-12-01T01:55:09.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f949ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a015b550 con 0x7f94a006b7b0 2023-12-01T01:55:09.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f94967fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f94a015afe0 0x7f94a006b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f94967fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f94a015afe0 0x7f94a006b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60136/0 (socket says 172.21.15.115:60136) 2023-12-01T01:55:09.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f94967fc640 1 -- 172.21.15.115:0/1663339783 learned_addr learned my addr 172.21.15.115:0/1663339783 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:09.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f949effd640 1 --2- 172.21.15.115:0/1663339783 >> v2:172.21.15.142:3300/0 conn(0x7f94a01584b0 0x7f94a00cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.013+0000 7f949f7fe640 1 --2- 172.21.15.115:0/1663339783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94a006b7b0 0x7f94a015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.014+0000 7f94967fc640 1 -- 172.21.15.115:0/1663339783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94a006b7b0 msgr2=0x7f94a015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:09.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.014+0000 7f94967fc640 1 --2- 172.21.15.115:0/1663339783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94a006b7b0 0x7f94a015e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.014+0000 7f94967fc640 1 -- 172.21.15.115:0/1663339783 >> v2:172.21.15.142:3300/0 conn(0x7f94a01584b0 msgr2=0x7f94a00cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:09.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.014+0000 7f94967fc640 1 --2- 172.21.15.115:0/1663339783 >> v2:172.21.15.142:3300/0 conn(0x7f94a01584b0 0x7f94a00cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.014+0000 7f94967fc640 1 -- 172.21.15.115:0/1663339783 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a015b830 con 0x7f94a015afe0 2023-12-01T01:55:09.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.015+0000 7f94967fc640 1 --2- 172.21.15.115:0/1663339783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 0x7f94a006b270 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f94a006bfb0 tx=0x7f948802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5dba4093689092 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:09.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.015+0000 7f949e7fc640 1 -- 172.21.15.115:0/1663339783 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488031870 con 0x7f94a015afe0 2023-12-01T01:55:09.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.015+0000 7f949e7fc640 1 -- 172.21.15.115:0/1663339783 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94880319d0 con 0x7f94a015afe0 2023-12-01T01:55:09.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.015+0000 7f949e7fc640 1 -- 172.21.15.115:0/1663339783 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488031d60 con 0x7f94a015afe0 2023-12-01T01:55:09.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.016+0000 7f949ffff640 1 -- 172.21.15.115:0/1663339783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 msgr2=0x7f94a006b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.016+0000 7f949ffff640 1 --2- 172.21.15.115:0/1663339783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 0x7f94a006b270 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f94a006bfb0 tx=0x7f948802b910 comp rx=0 tx=0).stop 2023-12-01T01:55:09.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.016+0000 7f949ffff640 1 -- 172.21.15.115:0/1663339783 shutdown_connections 2023-12-01T01:55:09.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.016+0000 7f949ffff640 1 --2- 172.21.15.115:0/1663339783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94a006b7b0 0x7f94a015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.016+0000 7f949ffff640 1 --2- 172.21.15.115:0/1663339783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 0x7f94a006b270 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.016+0000 7f949ffff640 1 --2- 172.21.15.115:0/1663339783 >> v2:172.21.15.142:3300/0 conn(0x7f94a01584b0 0x7f94a00cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.016+0000 7f949ffff640 1 -- 172.21.15.115:0/1663339783 >> 172.21.15.115:0/1663339783 conn(0x7f94a00ce640 msgr2=0x7f94a00cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:09.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.016+0000 7f949ffff640 1 -- 172.21.15.115:0/1663339783 shutdown_connections 2023-12-01T01:55:09.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.016+0000 7f949ffff640 1 -- 172.21.15.115:0/1663339783 wait complete. 2023-12-01T01:55:09.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.018+0000 7f949ffff640 1 Processor -- start 2023-12-01T01:55:09.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.018+0000 7f949ffff640 1 -- start start 2023-12-01T01:55:09.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.019+0000 7f949ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94a006b7b0 0x7f94a00ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.019+0000 7f949ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94a01584b0 0x7f94a00ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.019+0000 7f949ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 0x7f94a00acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.019+0000 7f949ffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0157cf0 con 0x7f94a015afe0 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.019+0000 7f949ffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0156f30 con 0x7f94a01584b0 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.019+0000 7f949ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0157210 con 0x7f94a006b7b0 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.019+0000 7f949f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 0x7f94a00acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.019+0000 7f949f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 0x7f94a00acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60150/0 (socket says 172.21.15.115:60150) 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.019+0000 7f949f7fe640 1 -- 172.21.15.115:0/1831983418 learned_addr learned my addr 172.21.15.115:0/1831983418 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f949effd640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94a006b7b0 0x7f94a00ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f94967fc640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94a01584b0 0x7f94a00ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f949f7fe640 1 -- 172.21.15.115:0/1831983418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94a006b7b0 msgr2=0x7f94a00ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f949f7fe640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94a006b7b0 0x7f94a00ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f949f7fe640 1 -- 172.21.15.115:0/1831983418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94a01584b0 msgr2=0x7f94a00ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f949f7fe640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94a01584b0 0x7f94a00ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f949f7fe640 1 -- 172.21.15.115:0/1831983418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94880094d0 con 0x7f94a015afe0 2023-12-01T01:55:09.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f949f7fe640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 0x7f94a00acdd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f9498007d20 tx=0x7f949800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:09.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f9497fff640 1 -- 172.21.15.115:0/1831983418 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498014420 con 0x7f94a015afe0 2023-12-01T01:55:09.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f9497fff640 1 -- 172.21.15.115:0/1831983418 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94980145a0 con 0x7f94a015afe0 2023-12-01T01:55:09.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f9497fff640 1 -- 172.21.15.115:0/1831983418 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498014950 con 0x7f94a015afe0 2023-12-01T01:55:09.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.020+0000 7f949ffff640 1 -- 172.21.15.115:0/1831983418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a01574f0 con 0x7f94a015afe0 2023-12-01T01:55:09.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.021+0000 7f949ffff640 1 -- 172.21.15.115:0/1831983418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a0156c30 con 0x7f94a015afe0 2023-12-01T01:55:09.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.022+0000 7f949ffff640 1 -- 172.21.15.115:0/1831983418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a00673a0 con 0x7f94a015afe0 2023-12-01T01:55:09.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.023+0000 7f9497fff640 1 -- 172.21.15.115:0/1831983418 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 70) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f9498014ab0 con 0x7f94a015afe0 2023-12-01T01:55:09.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.024+0000 7f9497fff640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f948006a2b0 0x7f948006c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.024+0000 7f9497fff640 1 -- 172.21.15.115:0/1831983418 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f9498089120 con 0x7f94a015afe0 2023-12-01T01:55:09.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.024+0000 7f949effd640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f948006a2b0 0x7f948006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.025+0000 7f949effd640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f948006a2b0 0x7f948006c770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9490006fd0 tx=0x7f9490008040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:09.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.028+0000 7f9497fff640 1 -- 172.21.15.115:0/1831983418 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f94980546b0 con 0x7f94a015afe0 2023-12-01T01:55:09.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.255+0000 7f949ffff640 1 -- 172.21.15.115:0/1831983418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94a00c78d0 con 0x7f94a015afe0 2023-12-01T01:55:09.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.256+0000 7f9497fff640 1 -- 172.21.15.115:0/1831983418 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f9498057cd0 con 0x7f94a015afe0 2023-12-01T01:55:09.257 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:09.257 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/4122088547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":4122088547},{"type":"v1","addr":"172.21.15.142:6834","nonce":4122088547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/299645728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":299645728},{"type":"v1","addr":"172.21.15.142:6835","nonce":299645728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7983,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/150770274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":150770274},{"type":"v1","addr":"172.21.15.115:6836","nonce":150770274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:54:33.047892+0000","modified":"2023-12-01T01:54:36.558949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/2398273191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":2398273191},{"type":"v1","addr":"172.21.15.115:6837","nonce":2398273191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-01T01:55:09.257 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 56 2023-12-01T01:55:09.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.258+0000 7f949ffff640 1 -- 172.21.15.115:0/1831983418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f948006a2b0 msgr2=0x7f948006c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.258+0000 7f949ffff640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f948006a2b0 0x7f948006c770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9490006fd0 tx=0x7f9490008040 comp rx=0 tx=0).stop 2023-12-01T01:55:09.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.259+0000 7f949ffff640 1 -- 172.21.15.115:0/1831983418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 msgr2=0x7f94a00acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.259+0000 7f949ffff640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 0x7f94a00acdd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f9498007d20 tx=0x7f949800e4b0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.259+0000 7f949ffff640 1 -- 172.21.15.115:0/1831983418 shutdown_connections 2023-12-01T01:55:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.259+0000 7f949ffff640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f948006a2b0 0x7f948006c770 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.259+0000 7f949ffff640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94a015afe0 0x7f94a00acdd0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.259+0000 7f949ffff640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94a01584b0 0x7f94a00ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.259+0000 7f949ffff640 1 --2- 172.21.15.115:0/1831983418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94a006b7b0 0x7f94a00ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.259+0000 7f949ffff640 1 -- 172.21.15.115:0/1831983418 >> 172.21.15.115:0/1831983418 conn(0x7f94a00ce640 msgr2=0x7f94a0154b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.259+0000 7f949ffff640 1 -- 172.21.15.115:0/1831983418 shutdown_connections 2023-12-01T01:55:09.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.259+0000 7f949ffff640 1 -- 172.21.15.115:0/1831983418 wait complete. 2023-12-01T01:55:09.276 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:55:09.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.485+0000 7fb8eac3e640 1 Processor -- start 2023-12-01T01:55:09.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.485+0000 7fb8eac3e640 1 -- start start 2023-12-01T01:55:09.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.487+0000 7fb8eac3e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb8e4160130 0x7fb8e40d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.487+0000 7fb8eac3e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cfb20 0x7fb8e40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.487+0000 7fb8eac3e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e4160510 con 0x7fb8e40d26e0 2023-12-01T01:55:09.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.487+0000 7fb8eac3e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e40d0510 con 0x7fb8e4160130 2023-12-01T01:55:09.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.487+0000 7fb8eac3e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40b6d20 con 0x7fb8e40cfb20 2023-12-01T01:55:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.487+0000 7fb8e9c3c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb8e4160130 0x7fb8e40d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.488+0000 7fb8e9c3c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb8e4160130 0x7fb8e40d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60152/0 (socket says 172.21.15.115:60152) 2023-12-01T01:55:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.488+0000 7fb8e9c3c640 1 -- 172.21.15.115:0/741120948 learned_addr learned my addr 172.21.15.115:0/741120948 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.488+0000 7fb8e9c3c640 1 -- 172.21.15.115:0/741120948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cfb20 msgr2=0x7fb8e40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.488+0000 7fb8ea43d640 1 --2- 172.21.15.115:0/741120948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cfb20 0x7fb8e40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.488+0000 7fb8e9c3c640 1 --2- 172.21.15.115:0/741120948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cfb20 0x7fb8e40cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.488+0000 7fb8e9c3c640 1 -- 172.21.15.115:0/741120948 >> v1:172.21.15.142:6789/0 conn(0x7fb8e40d26e0 legacy=0x7fb8e40b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.488+0000 7fb8e9c3c640 1 -- 172.21.15.115:0/741120948 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e40d1640 con 0x7fb8e4160130 2023-12-01T01:55:09.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.488+0000 7fb8ea43d640 1 --2- 172.21.15.115:0/741120948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cfb20 0x7fb8e40cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:09.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.489+0000 7fb8e9c3c640 1 --2- 172.21.15.115:0/741120948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e4160130 0x7fb8e40d0fb0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009860 tx=0x7fb8e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15d45f5ffc8f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:09.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.489+0000 7fb8e8c3a640 1 -- 172.21.15.115:0/741120948 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0030840 con 0x7fb8e4160130 2023-12-01T01:55:09.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.489+0000 7fb8e8c3a640 1 -- 172.21.15.115:0/741120948 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e00309a0 con 0x7fb8e4160130 2023-12-01T01:55:09.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.489+0000 7fb8e8c3a640 1 -- 172.21.15.115:0/741120948 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0030d30 con 0x7fb8e4160130 2023-12-01T01:55:09.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.490+0000 7fb8eac3e640 1 -- 172.21.15.115:0/741120948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e4160130 msgr2=0x7fb8e40d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.490+0000 7fb8eac3e640 1 --2- 172.21.15.115:0/741120948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e4160130 0x7fb8e40d0fb0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009860 tx=0x7fb8e002b980 comp rx=0 tx=0).stop 2023-12-01T01:55:09.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.490+0000 7fb8eac3e640 1 -- 172.21.15.115:0/741120948 shutdown_connections 2023-12-01T01:55:09.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.490+0000 7fb8eac3e640 1 --2- 172.21.15.115:0/741120948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40cfb20 0x7fb8e40cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.490+0000 7fb8eac3e640 1 --2- 172.21.15.115:0/741120948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e4160130 0x7fb8e40d0fb0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.490+0000 7fb8eac3e640 1 -- 172.21.15.115:0/741120948 >> 172.21.15.115:0/741120948 conn(0x7fb8e406a0b0 msgr2=0x7fb8e406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:09.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.490+0000 7fb8eac3e640 1 -- 172.21.15.115:0/741120948 shutdown_connections 2023-12-01T01:55:09.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.490+0000 7fb8eac3e640 1 -- 172.21.15.115:0/741120948 wait complete. 2023-12-01T01:55:09.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.492+0000 7fb8eac3e640 1 Processor -- start 2023-12-01T01:55:09.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.492+0000 7fb8eac3e640 1 -- start start 2023-12-01T01:55:09.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.493+0000 7fb8eac3e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e40cfb20 0x7fb8e41f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.493+0000 7fb8eac3e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40d26e0 0x7fb8e41f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.493+0000 7fb8eac3e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e4160130 0x7fb8e40cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.493+0000 7fb8eac3e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e41f32b0 con 0x7fb8e40cfb20 2023-12-01T01:55:09.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.493+0000 7fb8eac3e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40cb230 con 0x7fb8e4160130 2023-12-01T01:55:09.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.493+0000 7fb8eac3e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40c93f0 con 0x7fb8e40d26e0 2023-12-01T01:55:09.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8e9c3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e40cfb20 0x7fb8e41f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8e9c3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e40cfb20 0x7fb8e41f2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60154/0 (socket says 172.21.15.115:60154) 2023-12-01T01:55:09.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8e9c3c640 1 -- 172.21.15.115:0/3426324338 learned_addr learned my addr 172.21.15.115:0/3426324338 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8e943b640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40d26e0 0x7fb8e41f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8ea43d640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e4160130 0x7fb8e40cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8e9c3c640 1 -- 172.21.15.115:0/3426324338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40d26e0 msgr2=0x7fb8e41f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8e9c3c640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40d26e0 0x7fb8e41f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8e9c3c640 1 -- 172.21.15.115:0/3426324338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e4160130 msgr2=0x7fb8e40cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8e9c3c640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e4160130 0x7fb8e40cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8e9c3c640 1 -- 172.21.15.115:0/3426324338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0009470 con 0x7fb8e40cfb20 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.494+0000 7fb8e9c3c640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e40cfb20 0x7fb8e41f2d70 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb8e002b9c0 tx=0x7fb8e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.495+0000 7fb8da7fc640 1 -- 172.21.15.115:0/3426324338 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0030810 con 0x7fb8e40cfb20 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.495+0000 7fb8da7fc640 1 -- 172.21.15.115:0/3426324338 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e0039d90 con 0x7fb8e40cfb20 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.495+0000 7fb8da7fc640 1 -- 172.21.15.115:0/3426324338 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0043670 con 0x7fb8e40cfb20 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.495+0000 7fb8eac3e640 1 -- 172.21.15.115:0/3426324338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e40ca190 con 0x7fb8e40cfb20 2023-12-01T01:55:09.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.495+0000 7fb8eac3e640 1 -- 172.21.15.115:0/3426324338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e40ca750 con 0x7fb8e40cfb20 2023-12-01T01:55:09.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.496+0000 7fb8eac3e640 1 -- 172.21.15.115:0/3426324338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e40673a0 con 0x7fb8e40cfb20 2023-12-01T01:55:09.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.497+0000 7fb8da7fc640 1 -- 172.21.15.115:0/3426324338 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 70) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fb8e002f390 con 0x7fb8e40cfb20 2023-12-01T01:55:09.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.498+0000 7fb8da7fc640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8c806a280 0x7fb8c806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.498+0000 7fb8da7fc640 1 -- 172.21.15.115:0/3426324338 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fb8e00aac80 con 0x7fb8e40cfb20 2023-12-01T01:55:09.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.499+0000 7fb8e943b640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8c806a280 0x7fb8c806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.499+0000 7fb8e943b640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8c806a280 0x7fb8c806c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40c81c0 tx=0x7fb8d4009040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:09.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.503+0000 7fb8da7fc640 1 -- 172.21.15.115:0/3426324338 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb8e0076330 con 0x7fb8e40cfb20 2023-12-01T01:55:09.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.693+0000 7fb8eac3e640 1 -- 172.21.15.115:0/3426324338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8e40b7b90 con 0x7fb8e40cfb20 2023-12-01T01:55:09.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.695+0000 7fb8da7fc640 1 -- 172.21.15.115:0/3426324338 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+17164 (secure 0 0 0) 0x7fb8e0079950 con 0x7fb8e40cfb20 2023-12-01T01:55:09.695 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:09.695 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":128,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:55:07.906242+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:54:30.872492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-01T01:54:32.074780+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2985636090":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/3771486155":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/2297203451":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/3740383899":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/3964272962":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/558327360":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/337421348":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/2260205326":"2023-12-02T01:55:07.906155+0000","172.21.15.115:6824/88317":"2023-12-02T01:54:47.777090+0000","172.21.15.115:6822/88317":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/2560191707":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/31572672":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/1289974123":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/3424983653":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/3883676082":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/841702221":"2023-12-02T01:54:47.777090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:55:09.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 -- 172.21.15.115:0/3426324338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8c806a280 msgr2=0x7fb8c806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8c806a280 0x7fb8c806c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40c81c0 tx=0x7fb8d4009040 comp rx=0 tx=0).stop 2023-12-01T01:55:09.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 -- 172.21.15.115:0/3426324338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e40cfb20 msgr2=0x7fb8e41f2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e40cfb20 0x7fb8e41f2d70 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb8e002b9c0 tx=0x7fb8e0004730 comp rx=0 tx=0).stop 2023-12-01T01:55:09.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 -- 172.21.15.115:0/3426324338 shutdown_connections 2023-12-01T01:55:09.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8c806a280 0x7fb8c806c740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8e40d26e0 0x7fb8e41f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8e4160130 0x7fb8e40cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 --2- 172.21.15.115:0/3426324338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8e40cfb20 0x7fb8e41f2d70 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 -- 172.21.15.115:0/3426324338 >> 172.21.15.115:0/3426324338 conn(0x7fb8e406a0b0 msgr2=0x7fb8e40b7140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:09.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 -- 172.21.15.115:0/3426324338 shutdown_connections 2023-12-01T01:55:09.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.697+0000 7fb8eac3e640 1 -- 172.21.15.115:0/3426324338 wait complete. 2023-12-01T01:55:09.717 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T01:55:09.717 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:55:09.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.929+0000 7f55677ec640 1 Processor -- start 2023-12-01T01:55:09.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.929+0000 7f55677ec640 1 -- start start 2023-12-01T01:55:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.931+0000 7f55677ec640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f556015cd20 0x7f55600d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.931+0000 7f55677ec640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55600b5d80 0x7f55600b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.931+0000 7f55677ec640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55600ce2e0 con 0x7f55600d1230 2023-12-01T01:55:09.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.931+0000 7f55677ec640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55600b2160 con 0x7f556015cd20 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.931+0000 7f55677ec640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55600d2a10 con 0x7f55600b5d80 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.932+0000 7f5564d87640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55600b5d80 0x7f55600b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.932+0000 7f5565588640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f556015cd20 0x7f55600d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.932+0000 7f5564d87640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55600b5d80 0x7f55600b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:53820/0 (socket says 172.21.15.115:53820) 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.932+0000 7f5564d87640 1 -- 172.21.15.115:0/1520448930 learned_addr learned my addr 172.21.15.115:0/1520448930 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.933+0000 7f5565588640 1 -- 172.21.15.115:0/1520448930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55600b5d80 msgr2=0x7f55600b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.933+0000 7f5565588640 1 --2- 172.21.15.115:0/1520448930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55600b5d80 0x7f55600b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.933+0000 7f5565588640 1 -- 172.21.15.115:0/1520448930 >> v1:172.21.15.142:6789/0 conn(0x7f55600d1230 legacy=0x7f55600d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.933+0000 7f5565588640 1 -- 172.21.15.115:0/1520448930 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55600ce000 con 0x7f556015cd20 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.933+0000 7f555ffff640 1 -- 172.21.15.115:0/1520448930 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107017876 0 0) 0x7f55600ce2e0 con 0x7f55600d1230 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.934+0000 7f5565588640 1 --2- 172.21.15.115:0/1520448930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f556015cd20 0x7f55600d24d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5550009dc0 tx=0x7f555002be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f38399e4bbe2b96 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.935+0000 7f555ffff640 1 -- 172.21.15.115:0/1520448930 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555002cbb0 con 0x7f556015cd20 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.935+0000 7f555ffff640 1 -- 172.21.15.115:0/1520448930 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555002cd10 con 0x7f556015cd20 2023-12-01T01:55:09.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.935+0000 7f555ffff640 1 -- 172.21.15.115:0/1520448930 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550034660 con 0x7f556015cd20 2023-12-01T01:55:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.935+0000 7f55677ec640 1 -- 172.21.15.115:0/1520448930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f556015cd20 msgr2=0x7f55600d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.935+0000 7f55677ec640 1 --2- 172.21.15.115:0/1520448930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f556015cd20 0x7f55600d24d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5550009dc0 tx=0x7f555002be00 comp rx=0 tx=0).stop 2023-12-01T01:55:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.935+0000 7f55677ec640 1 -- 172.21.15.115:0/1520448930 shutdown_connections 2023-12-01T01:55:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.935+0000 7f55677ec640 1 --2- 172.21.15.115:0/1520448930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55600b5d80 0x7f55600b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.935+0000 7f55677ec640 1 --2- 172.21.15.115:0/1520448930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f556015cd20 0x7f55600d24d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.935+0000 7f55677ec640 1 -- 172.21.15.115:0/1520448930 >> 172.21.15.115:0/1520448930 conn(0x7f556006a0e0 msgr2=0x7f556006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.935+0000 7f55677ec640 1 -- 172.21.15.115:0/1520448930 shutdown_connections 2023-12-01T01:55:09.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.936+0000 7f55677ec640 1 -- 172.21.15.115:0/1520448930 wait complete. 2023-12-01T01:55:09.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.937+0000 7f55677ec640 1 Processor -- start 2023-12-01T01:55:09.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.937+0000 7f55677ec640 1 -- start start 2023-12-01T01:55:09.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.938+0000 7f55677ec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55600b5d80 0x7f55600c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.938+0000 7f55677ec640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55600d1230 0x7f556015f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.938+0000 7f55677ec640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f556015cd20 0x7f556015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.938+0000 7f55677ec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55600c6140 con 0x7f55600b5d80 2023-12-01T01:55:09.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.938+0000 7f55677ec640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55600c70b0 con 0x7f55600d1230 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.938+0000 7f55677ec640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5560160200 con 0x7f556015cd20 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5565588640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55600b5d80 0x7f55600c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5565588640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55600b5d80 0x7f55600c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60168/0 (socket says 172.21.15.115:60168) 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5565588640 1 -- 172.21.15.115:0/739821580 learned_addr learned my addr 172.21.15.115:0/739821580 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5564d87640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55600d1230 0x7f556015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5565588640 1 -- 172.21.15.115:0/739821580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f556015cd20 msgr2=0x7f556015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5565d89640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f556015cd20 0x7f556015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5565588640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f556015cd20 0x7f556015bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5565588640 1 -- 172.21.15.115:0/739821580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55600d1230 msgr2=0x7f556015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5565588640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55600d1230 0x7f556015f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5565588640 1 -- 172.21.15.115:0/739821580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5550009a70 con 0x7f55600b5d80 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5564d87640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55600d1230 0x7f556015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.939+0000 7f5565588640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55600b5d80 0x7f55600c5c00 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f5550000f80 tx=0x7f5550004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:09.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.940+0000 7f555dffb640 1 -- 172.21.15.115:0/739821580 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550040650 con 0x7f55600b5d80 2023-12-01T01:55:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.940+0000 7f555dffb640 1 -- 172.21.15.115:0/739821580 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55500407b0 con 0x7f55600b5d80 2023-12-01T01:55:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.940+0000 7f55677ec640 1 -- 172.21.15.115:0/739821580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55600c6780 con 0x7f55600b5d80 2023-12-01T01:55:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.940+0000 7f555dffb640 1 -- 172.21.15.115:0/739821580 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550040b60 con 0x7f55600b5d80 2023-12-01T01:55:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.940+0000 7f55677ec640 1 -- 172.21.15.115:0/739821580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55600c6d40 con 0x7f55600b5d80 2023-12-01T01:55:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.941+0000 7f55677ec640 1 -- 172.21.15.115:0/739821580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55600c6ed0 con 0x7f55600b5d80 2023-12-01T01:55:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.942+0000 7f555dffb640 1 -- 172.21.15.115:0/739821580 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 70) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f5550040cc0 con 0x7f55600b5d80 2023-12-01T01:55:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.943+0000 7f555dffb640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f554c06a280 0x7f554c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.944+0000 7f555dffb640 1 -- 172.21.15.115:0/739821580 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f55500ab6e0 con 0x7f55600b5d80 2023-12-01T01:55:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.944+0000 7f5564d87640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f554c06a280 0x7f554c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:09.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.944+0000 7f5564d87640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f554c06a280 0x7f554c06c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5560153460 tx=0x7f5554008040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:09.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:09.948+0000 7f555dffb640 1 -- 172.21.15.115:0/739821580 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5550076c70 con 0x7f55600b5d80 2023-12-01T01:55:10.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.189+0000 7f55677ec640 1 -- 172.21.15.115:0/739821580 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f556004e890 con 0x7f55600b5d80 2023-12-01T01:55:10.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.190+0000 7f555dffb640 1 -- 172.21.15.115:0/739821580 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f555007a290 con 0x7f55600b5d80 2023-12-01T01:55:10.191 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:10.191 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/4122088547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":4122088547},{"type":"v1","addr":"172.21.15.142:6834","nonce":4122088547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/299645728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":299645728},{"type":"v1","addr":"172.21.15.142:6835","nonce":299645728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7983,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/150770274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":150770274},{"type":"v1","addr":"172.21.15.115:6836","nonce":150770274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:54:33.047892+0000","modified":"2023-12-01T01:54:36.558949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/2398273191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":2398273191},{"type":"v1","addr":"172.21.15.115:6837","nonce":2398273191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-01T01:55:10.191 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 56 2023-12-01T01:55:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 -- 172.21.15.115:0/739821580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f554c06a280 msgr2=0x7f554c06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f554c06a280 0x7f554c06c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5560153460 tx=0x7f5554008040 comp rx=0 tx=0).stop 2023-12-01T01:55:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 -- 172.21.15.115:0/739821580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55600b5d80 msgr2=0x7f55600c5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55600b5d80 0x7f55600c5c00 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f5550000f80 tx=0x7f5550004a40 comp rx=0 tx=0).stop 2023-12-01T01:55:10.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 -- 172.21.15.115:0/739821580 shutdown_connections 2023-12-01T01:55:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f554c06a280 0x7f554c06c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f556015cd20 0x7f556015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55600d1230 0x7f556015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 --2- 172.21.15.115:0/739821580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55600b5d80 0x7f55600c5c00 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 -- 172.21.15.115:0/739821580 >> 172.21.15.115:0/739821580 conn(0x7f556006a0e0 msgr2=0x7f55600cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 -- 172.21.15.115:0/739821580 shutdown_connections 2023-12-01T01:55:10.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.193+0000 7f55677ec640 1 -- 172.21.15.115:0/739821580 wait complete. 2023-12-01T01:55:10.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:55:10.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.427+0000 7ff5b4241640 1 Processor -- start 2023-12-01T01:55:10.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.427+0000 7ff5b4241640 1 -- start start 2023-12-01T01:55:10.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.428+0000 7ff5b4241640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5ac0ccc80 0x7ff5ac06fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.428+0000 7ff5b4241640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ac0ad2b0 0x7ff5ac06fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.428+0000 7ff5b4241640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac15d600 con 0x7ff5ac15d220 2023-12-01T01:55:10.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.428+0000 7ff5b4241640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ac06cc10 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.428+0000 7ff5b4241640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac06c720 con 0x7ff5ac0ad2b0 2023-12-01T01:55:10.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.429+0000 7ff5b1fdd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff5ac15d220 0x7ff5ac0cc570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34924/0 (socket says 172.21.15.115:34924) 2023-12-01T01:55:10.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.429+0000 7ff5b1fdd640 1 -- 172.21.15.115:0/525657544 learned_addr learned my addr 172.21.15.115:0/525657544 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:10.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.429+0000 7ff5b17dc640 1 --2- 172.21.15.115:0/525657544 >> v2:172.21.15.115:3300/0 conn(0x7ff5ac0ccc80 0x7ff5ac06fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:10.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.429+0000 7ff5b27de640 1 --2- 172.21.15.115:0/525657544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ac0ad2b0 0x7ff5ac06fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:10.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.429+0000 7ff5b17dc640 1 -- 172.21.15.115:0/525657544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ac0ad2b0 msgr2=0x7ff5ac06fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:10.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.429+0000 7ff5b17dc640 1 --2- 172.21.15.115:0/525657544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ac0ad2b0 0x7ff5ac06fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.429+0000 7ff5b17dc640 1 -- 172.21.15.115:0/525657544 >> v1:172.21.15.115:6789/0 conn(0x7ff5ac15d220 legacy=0x7ff5ac0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.429+0000 7ff5b17dc640 1 -- 172.21.15.115:0/525657544 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac06c4b0 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.431+0000 7ff5b17dc640 1 --2- 172.21.15.115:0/525657544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 0x7ff5ac06fab0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0097a0 tx=0x7ff59c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29ce5884ad2abe5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:10.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.431+0000 7ff5b0fdb640 1 -- 172.21.15.115:0/525657544 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c031780 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.431+0000 7ff5b0fdb640 1 -- 172.21.15.115:0/525657544 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c0318e0 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.431+0000 7ff5b0fdb640 1 -- 172.21.15.115:0/525657544 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c031c70 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.431+0000 7ff5b4241640 1 -- 172.21.15.115:0/525657544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 msgr2=0x7ff5ac06fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.431+0000 7ff5b4241640 1 --2- 172.21.15.115:0/525657544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 0x7ff5ac06fab0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0097a0 tx=0x7ff59c02b820 comp rx=0 tx=0).stop 2023-12-01T01:55:10.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.432+0000 7ff5b4241640 1 -- 172.21.15.115:0/525657544 shutdown_connections 2023-12-01T01:55:10.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.432+0000 7ff5b4241640 1 --2- 172.21.15.115:0/525657544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ac0ad2b0 0x7ff5ac06fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.432+0000 7ff5b4241640 1 --2- 172.21.15.115:0/525657544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 0x7ff5ac06fab0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.432+0000 7ff5b4241640 1 -- 172.21.15.115:0/525657544 >> 172.21.15.115:0/525657544 conn(0x7ff5ac0ce880 msgr2=0x7ff5ac0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:10.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.432+0000 7ff5b4241640 1 -- 172.21.15.115:0/525657544 shutdown_connections 2023-12-01T01:55:10.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.432+0000 7ff5b4241640 1 -- 172.21.15.115:0/525657544 wait complete. 2023-12-01T01:55:10.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.433+0000 7ff5b4241640 1 Processor -- start 2023-12-01T01:55:10.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.433+0000 7ff5b4241640 1 -- start start 2023-12-01T01:55:10.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.434+0000 7ff5b4241640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac0ad2b0 0x7ff5ac0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.434+0000 7ff5b4241640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 0x7ff5ac0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.434+0000 7ff5b4241640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ac15d220 0x7ff5ac14f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.434+0000 7ff5b4241640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0b1a50 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.434+0000 7ff5b4241640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0b2ee0 con 0x7ff5ac0ad2b0 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.434+0000 7ff5b4241640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac1532a0 con 0x7ff5ac15d220 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.435+0000 7ff5b17dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 0x7ff5ac0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.435+0000 7ff5b17dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 0x7ff5ac0c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60222/0 (socket says 172.21.15.115:60222) 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.435+0000 7ff5b17dc640 1 -- 172.21.15.115:0/2978901732 learned_addr learned my addr 172.21.15.115:0/2978901732 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.435+0000 7ff5b17dc640 1 -- 172.21.15.115:0/2978901732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ac15d220 msgr2=0x7ff5ac14f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.435+0000 7ff5b17dc640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ac15d220 0x7ff5ac14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.435+0000 7ff5b17dc640 1 -- 172.21.15.115:0/2978901732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac0ad2b0 msgr2=0x7ff5ac0cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.435+0000 7ff5b1fdd640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac0ad2b0 0x7ff5ac0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:10.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.435+0000 7ff5b17dc640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac0ad2b0 0x7ff5ac0cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.435+0000 7ff5b17dc640 1 -- 172.21.15.115:0/2978901732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c009400 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.435+0000 7ff5b17dc640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 0x7ff5ac0c7bd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff59c034040 tx=0x7ff59c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:10.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.436+0000 7ff5a27fc640 1 -- 172.21.15.115:0/2978901732 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c032820 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.436+0000 7ff5a27fc640 1 -- 172.21.15.115:0/2978901732 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c032980 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.436+0000 7ff5b4241640 1 -- 172.21.15.115:0/2978901732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac0cbfd0 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.436+0000 7ff5a27fc640 1 -- 172.21.15.115:0/2978901732 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c032d30 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.436+0000 7ff5b4241640 1 -- 172.21.15.115:0/2978901732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac0b1770 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.437+0000 7ff5b4241640 1 -- 172.21.15.115:0/2978901732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ac04e890 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.438+0000 7ff5a27fc640 1 -- 172.21.15.115:0/2978901732 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 70) v1 ==== 88445+0+0 (secure 0 0 0) 0x7ff59c04e440 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.439+0000 7ff5a27fc640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a280 0x7ff59006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.439+0000 7ff5a27fc640 1 -- 172.21.15.115:0/2978901732 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6425+0+0 (secure 0 0 0) 0x7ff59c0aaa20 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.439+0000 7ff5b1fdd640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a280 0x7ff59006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:10.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.440+0000 7ff5b1fdd640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a280 0x7ff59006c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8005880 tx=0x7ff5a8009040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:10.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.443+0000 7ff5a27fc640 1 -- 172.21.15.115:0/2978901732 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff59c076060 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.662+0000 7ff5a27fc640 1 -- 172.21.15.115:0/2978901732 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 71) v1 ==== 88653+0+0 (secure 0 0 0) 0x7ff59c076430 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.673+0000 7ff5b4241640 1 -- 172.21.15.115:0/2978901732 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5ac0cc780 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.674+0000 7ff5a27fc640 1 -- 172.21.15.115:0/2978901732 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4656 (secure 0 0 0) 0x7ff59c08f480 con 0x7ff5ac0ccc80 2023-12-01T01:55:10.675 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:10.675 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/4122088547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":4122088547},{"type":"v1","addr":"172.21.15.142:6834","nonce":4122088547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/299645728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":299645728},{"type":"v1","addr":"172.21.15.142:6835","nonce":299645728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7983,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/150770274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":150770274},{"type":"v1","addr":"172.21.15.115:6836","nonce":150770274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:54:33.047892+0000","modified":"2023-12-01T01:54:36.558949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/2398273191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":2398273191},{"type":"v1","addr":"172.21.15.115:6837","nonce":2398273191}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-01T01:55:10.675 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 56 2023-12-01T01:55:10.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 -- 172.21.15.115:0/2978901732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a280 msgr2=0x7ff59006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a280 0x7ff59006c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8005880 tx=0x7ff5a8009040 comp rx=0 tx=0).stop 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 -- 172.21.15.115:0/2978901732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 msgr2=0x7ff5ac0c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 0x7ff5ac0c7bd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ff59c034040 tx=0x7ff59c0046c0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 -- 172.21.15.115:0/2978901732 shutdown_connections 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff59006a280 0x7ff59006c740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5ac15d220 0x7ff5ac14f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5ac0ccc80 0x7ff5ac0c7bd0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 --2- 172.21.15.115:0/2978901732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5ac0ad2b0 0x7ff5ac0cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 -- 172.21.15.115:0/2978901732 >> 172.21.15.115:0/2978901732 conn(0x7ff5ac0ce880 msgr2=0x7ff5ac0cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 -- 172.21.15.115:0/2978901732 shutdown_connections 2023-12-01T01:55:10.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.677+0000 7ff5b4241640 1 -- 172.21.15.115:0/2978901732 wait complete. 2023-12-01T01:55:10.694 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:55:10.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.893+0000 7f570b096640 1 Processor -- start 2023-12-01T01:55:10.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.893+0000 7f570b096640 1 -- start start 2023-12-01T01:55:10.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.894+0000 7f570b096640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 0x7f57040c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.894+0000 7f570b096640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f570406b6f0 0x7f57040c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.894+0000 7f570b096640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570406f8b0 con 0x7f57040ade80 2023-12-01T01:55:10.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.894+0000 7f570b096640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57040adcb0 con 0x7f570406b6f0 2023-12-01T01:55:10.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.894+0000 7f570b096640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57040ad9d0 con 0x7f5704159dd0 2023-12-01T01:55:10.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.895+0000 7f5703fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f570406b6f0 0x7f57040c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:10.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.895+0000 7f5703fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f570406b6f0 0x7f57040c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60238/0 (socket says 172.21.15.115:60238) 2023-12-01T01:55:10.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.895+0000 7f5703fff640 1 -- 172.21.15.115:0/117131755 learned_addr learned my addr 172.21.15.115:0/117131755 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:10.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.895+0000 7f5708e32640 1 --2- 172.21.15.115:0/117131755 >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 0x7f57040c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:10.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.895+0000 7f5703fff640 1 -- 172.21.15.115:0/117131755 >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 msgr2=0x7f57040c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:10.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.895+0000 7f5703fff640 1 --2- 172.21.15.115:0/117131755 >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 0x7f57040c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.895+0000 7f5703fff640 1 -- 172.21.15.115:0/117131755 >> v1:172.21.15.142:6789/0 conn(0x7f57040ade80 legacy=0x7f57040cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:10.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.895+0000 7f5703fff640 1 -- 172.21.15.115:0/117131755 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57040c9000 con 0x7f570406b6f0 2023-12-01T01:55:10.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.897+0000 7f5703fff640 1 --2- 172.21.15.115:0/117131755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570406b6f0 0x7f57040c9d20 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f57040cf4e0 tx=0x7f56f002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb3d66dd2b6a4e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:10.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.897+0000 7f57037fe640 1 -- 172.21.15.115:0/117131755 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0031750 con 0x7f570406b6f0 2023-12-01T01:55:10.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.897+0000 7f57037fe640 1 -- 172.21.15.115:0/117131755 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f00318b0 con 0x7f570406b6f0 2023-12-01T01:55:10.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.897+0000 7f57037fe640 1 -- 172.21.15.115:0/117131755 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0031c40 con 0x7f570406b6f0 2023-12-01T01:55:10.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.898+0000 7f570b096640 1 -- 172.21.15.115:0/117131755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570406b6f0 msgr2=0x7f57040c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.898+0000 7f570b096640 1 --2- 172.21.15.115:0/117131755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570406b6f0 0x7f57040c9d20 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f57040cf4e0 tx=0x7f56f002b7f0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.898+0000 7f570b096640 1 -- 172.21.15.115:0/117131755 shutdown_connections 2023-12-01T01:55:10.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.898+0000 7f570b096640 1 --2- 172.21.15.115:0/117131755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f570406b6f0 0x7f57040c9d20 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.898+0000 7f570b096640 1 --2- 172.21.15.115:0/117131755 >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 0x7f57040c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.898+0000 7f570b096640 1 -- 172.21.15.115:0/117131755 >> 172.21.15.115:0/117131755 conn(0x7f57040ce880 msgr2=0x7f57040cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:10.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.898+0000 7f570b096640 1 -- 172.21.15.115:0/117131755 shutdown_connections 2023-12-01T01:55:10.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.898+0000 7f570b096640 1 -- 172.21.15.115:0/117131755 wait complete. 2023-12-01T01:55:10.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.900+0000 7f570b096640 1 Processor -- start 2023-12-01T01:55:10.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.900+0000 7f570b096640 1 -- start start 2023-12-01T01:55:10.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f570b096640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570406b6f0 0x7f570415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f570b096640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57040ade80 0x7f570415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f570b096640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5704159dd0 0x7f570415ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f570b096640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57040c53b0 con 0x7f5704159dd0 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f570b096640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570415aae0 con 0x7f570406b6f0 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f570b096640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57040c2920 con 0x7f57040ade80 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f5709633640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5704159dd0 0x7f570415ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f5709633640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5704159dd0 0x7f570415ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60240/0 (socket says 172.21.15.115:60240) 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f5709633640 1 -- 172.21.15.115:0/1547131776 learned_addr learned my addr 172.21.15.115:0/1547131776 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f5709633640 1 -- 172.21.15.115:0/1547131776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57040ade80 msgr2=0x7f570415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f5709633640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57040ade80 0x7f570415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f5709633640 1 -- 172.21.15.115:0/1547131776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570406b6f0 msgr2=0x7f570415c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f5709633640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570406b6f0 0x7f570415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:10.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.901+0000 7f5709633640 1 -- 172.21.15.115:0/1547131776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00094d0 con 0x7f5704159dd0 2023-12-01T01:55:10.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.902+0000 7f5709633640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5704159dd0 0x7f570415ade0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c780 tx=0x7f56fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:10.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.902+0000 7f57017fa640 1 -- 172.21.15.115:0/1547131776 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc003d80 con 0x7f5704159dd0 2023-12-01T01:55:10.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.902+0000 7f57017fa640 1 -- 172.21.15.115:0/1547131776 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc00ce90 con 0x7f5704159dd0 2023-12-01T01:55:10.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.902+0000 7f57017fa640 1 -- 172.21.15.115:0/1547131776 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0052b0 con 0x7f5704159dd0 2023-12-01T01:55:10.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.902+0000 7f570b096640 1 -- 172.21.15.115:0/1547131776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57040c4a80 con 0x7f5704159dd0 2023-12-01T01:55:10.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.902+0000 7f570b096640 1 -- 172.21.15.115:0/1547131776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57040c5040 con 0x7f5704159dd0 2023-12-01T01:55:10.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.903+0000 7f570b096640 1 -- 172.21.15.115:0/1547131776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5704073910 con 0x7f5704159dd0 2023-12-01T01:55:10.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.904+0000 7f57017fa640 1 -- 172.21.15.115:0/1547131776 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 71) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f56fc005410 con 0x7f5704159dd0 2023-12-01T01:55:10.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.905+0000 7f57017fa640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f56e806a4c0 0x7f56e806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:10.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.905+0000 7f57017fa640 1 -- 172.21.15.115:0/1547131776 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f56fc088060 con 0x7f5704159dd0 2023-12-01T01:55:10.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.905+0000 7f5708e32640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f56e806a4c0 0x7f56e806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:10.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.906+0000 7f5708e32640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f56e806a4c0 0x7f56e806c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f56f8006fd0 tx=0x7f56f8008040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:10.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:10.909+0000 7f57017fa640 1 -- 172.21.15.115:0/1547131776 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f56fc053420 con 0x7f5704159dd0 2023-12-01T01:55:11.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.102+0000 7f570b096640 1 -- 172.21.15.115:0/1547131776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f570404e890 con 0x7f5704159dd0 2023-12-01T01:55:11.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.106+0000 7f57017fa640 1 -- 172.21.15.115:0/1547131776 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+17164 (secure 0 0 0) 0x7f56fc056a40 con 0x7f5704159dd0 2023-12-01T01:55:11.107 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:11.107 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":129,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:55:10.570167+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:54:30.872492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-01T01:54:32.074780+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":127,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2985636090":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/3771486155":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/2297203451":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/3740383899":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/3964272962":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/558327360":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/337421348":"2023-12-02T01:55:07.906155+0000","172.21.15.115:0/2260205326":"2023-12-02T01:55:07.906155+0000","172.21.15.115:6824/88317":"2023-12-02T01:54:47.777090+0000","172.21.15.115:6822/88317":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/2560191707":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/31572672":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/1289974123":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/3424983653":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/3883676082":"2023-12-02T01:54:47.777090+0000","172.21.15.115:0/841702221":"2023-12-02T01:54:47.777090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:55:11.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.108+0000 7f570b096640 1 -- 172.21.15.115:0/1547131776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f56e806a4c0 msgr2=0x7f56e806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.108+0000 7f570b096640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f56e806a4c0 0x7f56e806c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f56f8006fd0 tx=0x7f56f8008040 comp rx=0 tx=0).stop 2023-12-01T01:55:11.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.108+0000 7f570b096640 1 -- 172.21.15.115:0/1547131776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5704159dd0 msgr2=0x7f570415ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.108+0000 7f570b096640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5704159dd0 0x7f570415ade0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00c780 tx=0x7f56fc00cc40 comp rx=0 tx=0).stop 2023-12-01T01:55:11.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.108+0000 7f570b096640 1 -- 172.21.15.115:0/1547131776 shutdown_connections 2023-12-01T01:55:11.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.108+0000 7f570b096640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f56e806a4c0 0x7f56e806c980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.108+0000 7f570b096640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57040ade80 0x7f570415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.108+0000 7f570b096640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5704159dd0 0x7f570415ade0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.109+0000 7f570b096640 1 --2- 172.21.15.115:0/1547131776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570406b6f0 0x7f570415c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.109+0000 7f570b096640 1 -- 172.21.15.115:0/1547131776 >> 172.21.15.115:0/1547131776 conn(0x7f57040ce880 msgr2=0x7f57040cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:11.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.109+0000 7f570b096640 1 -- 172.21.15.115:0/1547131776 shutdown_connections 2023-12-01T01:55:11.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.109+0000 7f570b096640 1 -- 172.21.15.115:0/1547131776 wait complete. 2023-12-01T01:55:11.130 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T01:55:11.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.341+0000 7fa8b0adf640 1 Processor -- start 2023-12-01T01:55:11.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.342+0000 7fa8b0adf640 1 -- start start 2023-12-01T01:55:11.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.343+0000 7fa8b0adf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa8ac15cd20 0x7fa8ac0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.343+0000 7fa8b0adf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ac0ce2e0 con 0x7fa8ac0d1230 2023-12-01T01:55:11.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.343+0000 7fa8b0adf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ac0ce000 con 0x7fa8ac0b5d80 2023-12-01T01:55:11.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.343+0000 7fa8b0adf640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8ac0b2160 con 0x7fa8ac15cd20 2023-12-01T01:55:11.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.343+0000 7fa8aa59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa8ac15cd20 0x7fa8ac0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:11.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.343+0000 7fa8aad9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa8ac0d1230 0x7fa8ac0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34928/0 (socket says 172.21.15.115:34928) 2023-12-01T01:55:11.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.343+0000 7fa8aad9d640 1 -- 172.21.15.115:0/3756029000 learned_addr learned my addr 172.21.15.115:0/3756029000 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:11.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.344+0000 7fa8aa59c640 1 -- 172.21.15.115:0/3756029000 >> v1:172.21.15.142:6789/0 conn(0x7fa8ac0b5d80 legacy=0x7fa8ac0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:11.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.344+0000 7fa8aa59c640 1 -- 172.21.15.115:0/3756029000 >> v1:172.21.15.115:6789/0 conn(0x7fa8ac0d1230 legacy=0x7fa8ac0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.344+0000 7fa8aa59c640 1 -- 172.21.15.115:0/3756029000 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac0cd3d0 con 0x7fa8ac15cd20 2023-12-01T01:55:11.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.344+0000 7fa8a959a640 1 -- 172.21.15.115:0/3756029000 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133402438 0 0) 0x7fa8ac0ce2e0 con 0x7fa8ac0d1230 2023-12-01T01:55:11.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.345+0000 7fa8aa59c640 1 --2- 172.21.15.115:0/3756029000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 0x7fa8ac0d24d0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009d90 tx=0x7fa89c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0dfb86e3c387b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:11.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.345+0000 7fa8a959a640 1 -- 172.21.15.115:0/3756029000 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c02cbd0 con 0x7fa8ac15cd20 2023-12-01T01:55:11.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.346+0000 7fa8a959a640 1 -- 172.21.15.115:0/3756029000 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c02cd30 con 0x7fa8ac15cd20 2023-12-01T01:55:11.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.346+0000 7fa8a959a640 1 -- 172.21.15.115:0/3756029000 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c02cea0 con 0x7fa8ac15cd20 2023-12-01T01:55:11.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.346+0000 7fa8b0adf640 1 -- 172.21.15.115:0/3756029000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 msgr2=0x7fa8ac0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.346+0000 7fa8b0adf640 1 --2- 172.21.15.115:0/3756029000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 0x7fa8ac0d24d0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009d90 tx=0x7fa89c02be20 comp rx=0 tx=0).stop 2023-12-01T01:55:11.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.346+0000 7fa8b0adf640 1 -- 172.21.15.115:0/3756029000 shutdown_connections 2023-12-01T01:55:11.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.346+0000 7fa8b0adf640 1 --2- 172.21.15.115:0/3756029000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 0x7fa8ac0d24d0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.346+0000 7fa8b0adf640 1 -- 172.21.15.115:0/3756029000 >> 172.21.15.115:0/3756029000 conn(0x7fa8ac06a0e0 msgr2=0x7fa8ac06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:11.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.346+0000 7fa8b0adf640 1 -- 172.21.15.115:0/3756029000 shutdown_connections 2023-12-01T01:55:11.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.346+0000 7fa8b0adf640 1 -- 172.21.15.115:0/3756029000 wait complete. 2023-12-01T01:55:11.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.348+0000 7fa8b0adf640 1 Processor -- start 2023-12-01T01:55:11.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.348+0000 7fa8b0adf640 1 -- start start 2023-12-01T01:55:11.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.349+0000 7fa8b0adf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8ac0b5d80 0x7fa8ac154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.349+0000 7fa8b0adf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8ac0d1230 0x7fa8ac0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.349+0000 7fa8b0adf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 0x7fa8ac1e6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.349+0000 7fa8b0adf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0c7360 con 0x7fa8ac15cd20 2023-12-01T01:55:11.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.349+0000 7fa8b0adf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0c6750 con 0x7fa8ac0d1230 2023-12-01T01:55:11.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.349+0000 7fa8b0adf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0c6a30 con 0x7fa8ac0b5d80 2023-12-01T01:55:11.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8aad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 0x7fa8ac1e6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:11.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8aad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 0x7fa8ac1e6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60250/0 (socket says 172.21.15.115:60250) 2023-12-01T01:55:11.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8aad9d640 1 -- 172.21.15.115:0/1399353163 learned_addr learned my addr 172.21.15.115:0/1399353163 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8aa59c640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8ac0b5d80 0x7fa8ac154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8a9d9b640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8ac0d1230 0x7fa8ac0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8aad9d640 1 -- 172.21.15.115:0/1399353163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8ac0b5d80 msgr2=0x7fa8ac154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8aad9d640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8ac0b5d80 0x7fa8ac154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8aad9d640 1 -- 172.21.15.115:0/1399353163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8ac0d1230 msgr2=0x7fa8ac0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8aad9d640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8ac0d1230 0x7fa8ac0c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8aad9d640 1 -- 172.21.15.115:0/1399353163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c009a70 con 0x7fa8ac15cd20 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.350+0000 7fa8aad9d640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 0x7fa8ac1e6a80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c450 tx=0x7fa8a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.351+0000 7fa8a2ffd640 1 -- 172.21.15.115:0/1399353163 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40074e0 con 0x7fa8ac15cd20 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.351+0000 7fa8a2ffd640 1 -- 172.21.15.115:0/1399353163 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a4007660 con 0x7fa8ac15cd20 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.351+0000 7fa8a2ffd640 1 -- 172.21.15.115:0/1399353163 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4004a20 con 0x7fa8ac15cd20 2023-12-01T01:55:11.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.351+0000 7fa8b0adf640 1 -- 172.21.15.115:0/1399353163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac0c6d10 con 0x7fa8ac15cd20 2023-12-01T01:55:11.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.351+0000 7fa8b0adf640 1 -- 172.21.15.115:0/1399353163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac0c9190 con 0x7fa8ac15cd20 2023-12-01T01:55:11.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.352+0000 7fa8b0adf640 1 -- 172.21.15.115:0/1399353163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ac0c7510 con 0x7fa8ac15cd20 2023-12-01T01:55:11.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.353+0000 7fa8a2ffd640 1 -- 172.21.15.115:0/1399353163 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 71) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fa8a4004b80 con 0x7fa8ac15cd20 2023-12-01T01:55:11.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.354+0000 7fa8a2ffd640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa88c06a4c0 0x7fa88c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.354+0000 7fa8a2ffd640 1 -- 172.21.15.115:0/1399353163 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fa8a4087cf0 con 0x7fa8ac15cd20 2023-12-01T01:55:11.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.355+0000 7fa8aa59c640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa88c06a4c0 0x7fa88c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:11.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.355+0000 7fa8aa59c640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa88c06a4c0 0x7fa88c06c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa89c036000 tx=0x7fa89c030000 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:11.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.359+0000 7fa8a2ffd640 1 -- 172.21.15.115:0/1399353163 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa8ac0c7510 con 0x7fa8ac15cd20 2023-12-01T01:55:11.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.585+0000 7fa8b0adf640 1 -- 172.21.15.115:0/1399353163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa8ac04e890 con 0x7fa8ac15cd20 2023-12-01T01:55:11.588 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T01:55:11.587+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:55:11.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.606+0000 7fa8a2ffd640 1 -- 172.21.15.115:0/1399353163 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa8a4018020 con 0x7fa8ac15cd20 2023-12-01T01:55:11.607 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T01:55:11.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 -- 172.21.15.115:0/1399353163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa88c06a4c0 msgr2=0x7fa88c06c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa88c06a4c0 0x7fa88c06c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa89c036000 tx=0x7fa89c030000 comp rx=0 tx=0).stop 2023-12-01T01:55:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 -- 172.21.15.115:0/1399353163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 msgr2=0x7fa8ac1e6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 0x7fa8ac1e6a80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c450 tx=0x7fa8a400c910 comp rx=0 tx=0).stop 2023-12-01T01:55:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 -- 172.21.15.115:0/1399353163 shutdown_connections 2023-12-01T01:55:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa88c06a4c0 0x7fa88c06c980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8ac15cd20 0x7fa8ac1e6a80 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8ac0d1230 0x7fa8ac0c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 --2- 172.21.15.115:0/1399353163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8ac0b5d80 0x7fa8ac154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 -- 172.21.15.115:0/1399353163 >> 172.21.15.115:0/1399353163 conn(0x7fa8ac06a0e0 msgr2=0x7fa8ac0ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 -- 172.21.15.115:0/1399353163 shutdown_connections 2023-12-01T01:55:11.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.609+0000 7fa8b0adf640 1 -- 172.21.15.115:0/1399353163 wait complete. 2023-12-01T01:55:11.615 INFO:tasks.ceph.mds.c.smithi115.stderr: -6178> 2023-12-01T01:54:37.844+0000 7f1cf50d0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:55:11.628 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T01:55:11.672 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:55:11.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.820+0000 7f80f0d3c640 1 Processor -- start 2023-12-01T01:55:11.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.820+0000 7f80f0d3c640 1 -- start start 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.821+0000 7f80f0d3c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f80ec142870 0x7f80ec0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.821+0000 7f80f0d3c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f80ec0cd010 0x7f80ec0b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.821+0000 7f80f0d3c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ec0b2f30 con 0x7f80ec0b1c50 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.821+0000 7f80f0d3c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ec0b3ae0 con 0x7f80ec0cd010 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.821+0000 7f80f0d3c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ec0cd4a0 con 0x7f80ec142870 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.821+0000 7f80e9d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f80ec0cd010 0x7f80ec0b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.821+0000 7f80ead9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f80ec0b1c50 0x7f80ec15d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34944/0 (socket says 172.21.15.115:34944) 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.821+0000 7f80e9d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f80ec0cd010 0x7f80ec0b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60286/0 (socket says 172.21.15.115:60286) 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.821+0000 7f80ead9d640 1 -- 172.21.15.115:0/1935521507 learned_addr learned my addr 172.21.15.115:0/1935521507 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.822+0000 7f80ea59c640 1 --2- 172.21.15.115:0/1935521507 >> v2:172.21.15.142:3300/0 conn(0x7f80ec142870 0x7f80ec0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.822+0000 7f80e9d9b640 1 -- 172.21.15.115:0/1935521507 >> v2:172.21.15.142:3300/0 conn(0x7f80ec142870 msgr2=0x7f80ec0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.822+0000 7f80e9d9b640 1 --2- 172.21.15.115:0/1935521507 >> v2:172.21.15.142:3300/0 conn(0x7f80ec142870 0x7f80ec0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.822+0000 7f80e9d9b640 1 -- 172.21.15.115:0/1935521507 >> v1:172.21.15.115:6789/0 conn(0x7f80ec0b1c50 legacy=0x7f80ec15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.822+0000 7f80e9d9b640 1 -- 172.21.15.115:0/1935521507 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec0b20c0 con 0x7f80ec0cd010 2023-12-01T01:55:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.822+0000 7f80ea59c640 1 --2- 172.21.15.115:0/1935521507 >> v2:172.21.15.142:3300/0 conn(0x7f80ec142870 0x7f80ec0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.823+0000 7f80e9d9b640 1 --2- 172.21.15.115:0/1935521507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0cd010 0x7f80ec0b1710 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f80ec0b14d0 tx=0x7f80d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fd3664300957b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.823+0000 7f80e959a640 1 -- 172.21.15.115:0/1935521507 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4031830 con 0x7f80ec0cd010 2023-12-01T01:55:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.823+0000 7f80e959a640 1 -- 172.21.15.115:0/1935521507 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d4031990 con 0x7f80ec0cd010 2023-12-01T01:55:11.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.823+0000 7f80e959a640 1 -- 172.21.15.115:0/1935521507 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4031d20 con 0x7f80ec0cd010 2023-12-01T01:55:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.824+0000 7f80f0d3c640 1 -- 172.21.15.115:0/1935521507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0cd010 msgr2=0x7f80ec0b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.824+0000 7f80f0d3c640 1 --2- 172.21.15.115:0/1935521507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0cd010 0x7f80ec0b1710 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f80ec0b14d0 tx=0x7f80d402b8d0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.824+0000 7f80f0d3c640 1 -- 172.21.15.115:0/1935521507 shutdown_connections 2023-12-01T01:55:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.824+0000 7f80f0d3c640 1 --2- 172.21.15.115:0/1935521507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0cd010 0x7f80ec0b1710 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.824+0000 7f80f0d3c640 1 --2- 172.21.15.115:0/1935521507 >> v2:172.21.15.142:3300/0 conn(0x7f80ec142870 0x7f80ec0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.824+0000 7f80f0d3c640 1 -- 172.21.15.115:0/1935521507 >> 172.21.15.115:0/1935521507 conn(0x7f80ec065960 msgr2=0x7f80ec065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.824+0000 7f80f0d3c640 1 -- 172.21.15.115:0/1935521507 shutdown_connections 2023-12-01T01:55:11.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.824+0000 7f80f0d3c640 1 -- 172.21.15.115:0/1935521507 wait complete. 2023-12-01T01:55:11.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.826+0000 7f80f0d3c640 1 Processor -- start 2023-12-01T01:55:11.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.826+0000 7f80f0d3c640 1 -- start start 2023-12-01T01:55:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.827+0000 7f80f0d3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0b1c50 0x7f80ec0c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.827+0000 7f80f0d3c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80ec0cd010 0x7f80ec14d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.827+0000 7f80f0d3c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80ec142870 0x7f80ec14dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.827+0000 7f80f0d3c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec14e160 con 0x7f80ec0b1c50 2023-12-01T01:55:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.827+0000 7f80f0d3c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec0c3b20 con 0x7f80ec0cd010 2023-12-01T01:55:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.827+0000 7f80f0d3c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec0cb7e0 con 0x7f80ec142870 2023-12-01T01:55:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.827+0000 7f80ea59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0b1c50 0x7f80ec0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.828+0000 7f80ea59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0b1c50 0x7f80ec0c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60302/0 (socket says 172.21.15.115:60302) 2023-12-01T01:55:11.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.828+0000 7f80ea59c640 1 -- 172.21.15.115:0/3388882598 learned_addr learned my addr 172.21.15.115:0/3388882598 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:11.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.828+0000 7f80e9d9b640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80ec0cd010 0x7f80ec14d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:11.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.828+0000 7f80ea59c640 1 -- 172.21.15.115:0/3388882598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80ec142870 msgr2=0x7f80ec14dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.828+0000 7f80ea59c640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80ec142870 0x7f80ec14dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.828+0000 7f80ea59c640 1 -- 172.21.15.115:0/3388882598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80ec0cd010 msgr2=0x7f80ec14d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.828+0000 7f80ea59c640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80ec0cd010 0x7f80ec14d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.828+0000 7f80ea59c640 1 -- 172.21.15.115:0/3388882598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d4009400 con 0x7f80ec0b1c50 2023-12-01T01:55:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.828+0000 7f80ea59c640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0b1c50 0x7f80ec0c8a70 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00ce00 tx=0x7f80dc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.829+0000 7f80e2ffd640 1 -- 172.21.15.115:0/3388882598 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00f890 con 0x7f80ec0b1c50 2023-12-01T01:55:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.829+0000 7f80e2ffd640 1 -- 172.21.15.115:0/3388882598 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc00f9f0 con 0x7f80ec0b1c50 2023-12-01T01:55:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.829+0000 7f80f0d3c640 1 -- 172.21.15.115:0/3388882598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec0cbac0 con 0x7f80ec0b1c50 2023-12-01T01:55:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.829+0000 7f80e2ffd640 1 -- 172.21.15.115:0/3388882598 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00fdc0 con 0x7f80ec0b1c50 2023-12-01T01:55:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.829+0000 7f80f0d3c640 1 -- 172.21.15.115:0/3388882598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec0cc080 con 0x7f80ec0b1c50 2023-12-01T01:55:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.830+0000 7f80f0d3c640 1 -- 172.21.15.115:0/3388882598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec04e890 con 0x7f80ec0b1c50 2023-12-01T01:55:11.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.831+0000 7f80e2ffd640 1 -- 172.21.15.115:0/3388882598 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 71) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f80dc00b040 con 0x7f80ec0b1c50 2023-12-01T01:55:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.832+0000 7f80e2ffd640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f80cc06a4c0 0x7f80cc06c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.832+0000 7f80e2ffd640 1 -- 172.21.15.115:0/3388882598 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f80dc089730 con 0x7f80ec0b1c50 2023-12-01T01:55:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.833+0000 7f80e9d9b640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f80cc06a4c0 0x7f80cc06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:11.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.833+0000 7f80e9d9b640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f80cc06a4c0 0x7f80cc06c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f80d4036000 tx=0x7f80d4008030 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:11.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:11.837+0000 7f80e2ffd640 1 -- 172.21.15.115:0/3388882598 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f80dc054920 con 0x7f80ec0b1c50 2023-12-01T01:55:12.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.065+0000 7f80f0d3c640 1 -- 172.21.15.115:0/3388882598 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f80ec06b6d0 con 0x7f80ec0b1c50 2023-12-01T01:55:12.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.662+0000 7f80e2ffd640 1 -- 172.21.15.115:0/3388882598 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7f80dc057f40 con 0x7f80ec0b1c50 2023-12-01T01:55:12.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 -- 172.21.15.115:0/3388882598 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f80cc06a4c0 msgr2=0x7f80cc06c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f80cc06a4c0 0x7f80cc06c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f80d4036000 tx=0x7f80d4008030 comp rx=0 tx=0).stop 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 -- 172.21.15.115:0/3388882598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0b1c50 msgr2=0x7f80ec0c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0b1c50 0x7f80ec0c8a70 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00ce00 tx=0x7f80dc00a650 comp rx=0 tx=0).stop 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 -- 172.21.15.115:0/3388882598 shutdown_connections 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f80cc06a4c0 0x7f80cc06c980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80ec142870 0x7f80ec14dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80ec0cd010 0x7f80ec14d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 --2- 172.21.15.115:0/3388882598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80ec0b1c50 0x7f80ec0c8a70 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 -- 172.21.15.115:0/3388882598 >> 172.21.15.115:0/3388882598 conn(0x7f80ec065960 msgr2=0x7f80ec06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 -- 172.21.15.115:0/3388882598 shutdown_connections 2023-12-01T01:55:12.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.666+0000 7f80f0d3c640 1 -- 172.21.15.115:0/3388882598 wait complete. 2023-12-01T01:55:12.694 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T01:55:12.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.952+0000 7ff3af670640 1 Processor -- start 2023-12-01T01:55:12.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.952+0000 7ff3af670640 1 -- start start 2023-12-01T01:55:12.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.954+0000 7ff3af670640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff3a80b4070 0x7ff3a80cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:12.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.954+0000 7ff3af670640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a806c860 con 0x7ff3a815ac70 2023-12-01T01:55:12.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.954+0000 7ff3af670640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a815b050 con 0x7ff3a80cb8c0 2023-12-01T01:55:12.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.954+0000 7ff3af670640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a80ce0b0 con 0x7ff3a80b4070 2023-12-01T01:55:12.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.954+0000 7ff3ad40c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff3a815ac70 0x7ff3a80cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34952/0 (socket says 172.21.15.115:34952) 2023-12-01T01:55:12.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.954+0000 7ff3ad40c640 1 -- 172.21.15.115:0/1588029320 learned_addr learned my addr 172.21.15.115:0/1588029320 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:12.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff3adc0d640 1 --2- 172.21.15.115:0/1588029320 >> v2:172.21.15.142:3300/0 conn(0x7ff3a80b4070 0x7ff3a80cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:12.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604094939 0 0) 0x7ff3a806c860 con 0x7ff3a815ac70 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff394003520 con 0x7ff3a815ac70 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006626393 0 0) 0x7ff394003520 con 0x7ff3a815ac70 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 >> v2:172.21.15.142:3300/0 conn(0x7ff3a80b4070 msgr2=0x7ff3a80cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 --2- 172.21.15.115:0/1588029320 >> v2:172.21.15.142:3300/0 conn(0x7ff3a80b4070 0x7ff3a80cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 >> v1:172.21.15.142:6789/0 conn(0x7ff3a80cb8c0 legacy=0x7ff3a80cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80ce390 con 0x7ff3a815ac70 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7ff3a4001670 con 0x7ff3a815ac70 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 >> v1:172.21.15.115:6789/0 conn(0x7ff3a815ac70 legacy=0x7ff3a80cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 --2- 172.21.15.115:0/1588029320 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3940039c0 0x7ff394003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 --2- 172.21.15.115:0/1588029320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff394004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 --2- 172.21.15.115:0/1588029320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3940051d0 0x7ff39400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80ce0b0 con 0x7ff3940047c0 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39400c120 con 0x7ff3940051d0 2023-12-01T01:55:12.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.955+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39400c2e0 con 0x7ff3940039c0 2023-12-01T01:55:12.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.956+0000 7ff3acc0b640 1 --2- 172.21.15.115:0/1588029320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff394004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:12.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.956+0000 7ff3adc0d640 1 --2- 172.21.15.115:0/1588029320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3940051d0 0x7ff39400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:12.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.956+0000 7ff3acc0b640 1 -- 172.21.15.115:0/1588029320 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3940039c0 msgr2=0x7ff394003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:12.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.956+0000 7ff3acc0b640 1 --2- 172.21.15.115:0/1588029320 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3940039c0 0x7ff394003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.956+0000 7ff3acc0b640 1 -- 172.21.15.115:0/1588029320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3940051d0 msgr2=0x7ff39400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:12.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.956+0000 7ff3acc0b640 1 --2- 172.21.15.115:0/1588029320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3940051d0 0x7ff39400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.956+0000 7ff3acc0b640 1 -- 172.21.15.115:0/1588029320 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff394003520 con 0x7ff3940047c0 2023-12-01T01:55:12.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.956+0000 7ff3adc0d640 1 -- 172.21.15.115:0/1588029320 reap_dead start 2023-12-01T01:55:12.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.957+0000 7ff3acc0b640 1 --2- 172.21.15.115:0/1588029320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff394004c40 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7ff398001720 tx=0x7ff3940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6364612f19ad01 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:12.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.957+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3980052d0 con 0x7ff3940047c0 2023-12-01T01:55:12.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.957+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39802e050 con 0x7ff3940047c0 2023-12-01T01:55:12.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.957+0000 7ff39ffff640 1 -- 172.21.15.115:0/1588029320 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398003830 con 0x7ff3940047c0 2023-12-01T01:55:12.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.958+0000 7ff3af670640 1 -- 172.21.15.115:0/1588029320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 msgr2=0x7ff394004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:12.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.958+0000 7ff3af670640 1 --2- 172.21.15.115:0/1588029320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff394004c40 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7ff398001720 tx=0x7ff3940057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.958+0000 7ff3af670640 1 -- 172.21.15.115:0/1588029320 shutdown_connections 2023-12-01T01:55:12.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.958+0000 7ff3af670640 1 --2- 172.21.15.115:0/1588029320 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff394004c40 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.958+0000 7ff3af670640 1 -- 172.21.15.115:0/1588029320 >> 172.21.15.115:0/1588029320 conn(0x7ff3a80662c0 msgr2=0x7ff3a80b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:12.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.958+0000 7ff3af670640 1 -- 172.21.15.115:0/1588029320 shutdown_connections 2023-12-01T01:55:12.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.958+0000 7ff3af670640 1 -- 172.21.15.115:0/1588029320 wait complete. 2023-12-01T01:55:12.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.960+0000 7ff3af670640 1 Processor -- start 2023-12-01T01:55:12.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.960+0000 7ff3af670640 1 -- start start 2023-12-01T01:55:12.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.961+0000 7ff3af670640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff3a80c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:12.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.961+0000 7ff3af670640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3a81f56d0 0x7ff3a80c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:12.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.961+0000 7ff3af670640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3a80c7e30 0x7ff3a80ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:12.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.961+0000 7ff3af670640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80c82b0 con 0x7ff3940047c0 2023-12-01T01:55:12.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.961+0000 7ff3af670640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a8153170 con 0x7ff3a80c7e30 2023-12-01T01:55:12.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.961+0000 7ff3af670640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80c22d0 con 0x7ff3a81f56d0 2023-12-01T01:55:12.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.961+0000 7ff3ad40c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff3a80c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:12.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.961+0000 7ff3ad40c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff3a80c3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60306/0 (socket says 172.21.15.115:60306) 2023-12-01T01:55:12.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.961+0000 7ff3ad40c640 1 -- 172.21.15.115:0/1561099411 learned_addr learned my addr 172.21.15.115:0/1561099411 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:12.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3acc0b640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3a81f56d0 0x7ff3a80c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:12.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3adc0d640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3a80c7e30 0x7ff3a80ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:12.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3ad40c640 1 -- 172.21.15.115:0/1561099411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3a81f56d0 msgr2=0x7ff3a80c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:12.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3ad40c640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3a81f56d0 0x7ff3a80c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3ad40c640 1 -- 172.21.15.115:0/1561099411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3a80c7e30 msgr2=0x7ff3a80ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:12.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3ad40c640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3a80c7e30 0x7ff3a80ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:12.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3ad40c640 1 -- 172.21.15.115:0/1561099411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80c2b70 con 0x7ff3940047c0 2023-12-01T01:55:12.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3acc0b640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3a81f56d0 0x7ff3a80c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:12.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3ad40c640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff3a80c3000 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400c730 tx=0x7ff3a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:12.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff39dffb640 1 -- 172.21.15.115:0/1561099411 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40072d0 con 0x7ff3940047c0 2023-12-01T01:55:12.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff39dffb640 1 -- 172.21.15.115:0/1561099411 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a4007430 con 0x7ff3940047c0 2023-12-01T01:55:12.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff39dffb640 1 -- 172.21.15.115:0/1561099411 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40047e0 con 0x7ff3940047c0 2023-12-01T01:55:12.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3af670640 1 -- 172.21.15.115:0/1561099411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a80c84d0 con 0x7ff3940047c0 2023-12-01T01:55:12.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.962+0000 7ff3af670640 1 -- 172.21.15.115:0/1561099411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a80c2800 con 0x7ff3940047c0 2023-12-01T01:55:12.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.963+0000 7ff3af670640 1 -- 172.21.15.115:0/1561099411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a8074210 con 0x7ff3940047c0 2023-12-01T01:55:12.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.964+0000 7ff39dffb640 1 -- 172.21.15.115:0/1561099411 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff3a4004940 con 0x7ff3940047c0 2023-12-01T01:55:12.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.965+0000 7ff39dffb640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff38c06a580 0x7ff38c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:12.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.965+0000 7ff39dffb640 1 -- 172.21.15.115:0/1561099411 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6483+0+0 (secure 0 0 0) 0x7ff3a4088180 con 0x7ff3940047c0 2023-12-01T01:55:12.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.966+0000 7ff3acc0b640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff38c06a580 0x7ff38c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:12.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.966+0000 7ff3acc0b640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff38c06a580 0x7ff38c06ca40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff398033000 tx=0x7ff398009030 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:12.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:12.970+0000 7ff39dffb640 1 -- 172.21.15.115:0/1561099411 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff3a40532b0 con 0x7ff3940047c0 2023-12-01T01:55:13.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.167+0000 7ff3af670640 1 -- 172.21.15.115:0/1561099411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff3a80c20b0 con 0x7ff3940047c0 2023-12-01T01:55:13.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.683+0000 7ff39dffb640 1 -- 172.21.15.115:0/1561099411 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff3a40180d0 con 0x7ff3940047c0 2023-12-01T01:55:13.686 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T01:55:13.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.688+0000 7ff3af670640 1 -- 172.21.15.115:0/1561099411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff38c06a580 msgr2=0x7ff38c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:13.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.688+0000 7ff3af670640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff38c06a580 0x7ff38c06ca40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff398033000 tx=0x7ff398009030 comp rx=0 tx=0).stop 2023-12-01T01:55:13.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.688+0000 7ff3af670640 1 -- 172.21.15.115:0/1561099411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 msgr2=0x7ff3a80c3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:13.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.688+0000 7ff3af670640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff3a80c3000 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400c730 tx=0x7ff3a400cbf0 comp rx=0 tx=0).stop 2023-12-01T01:55:13.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.689+0000 7ff3af670640 1 -- 172.21.15.115:0/1561099411 shutdown_connections 2023-12-01T01:55:13.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.689+0000 7ff3af670640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff38c06a580 0x7ff38c06ca40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:13.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.689+0000 7ff3af670640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3a81f56d0 0x7ff3a80c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:13.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.689+0000 7ff3af670640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3a80c7e30 0x7ff3a80ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:13.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.689+0000 7ff3af670640 1 --2- 172.21.15.115:0/1561099411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3940047c0 0x7ff3a80c3000 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:13.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.689+0000 7ff3af670640 1 -- 172.21.15.115:0/1561099411 >> 172.21.15.115:0/1561099411 conn(0x7ff3a80662c0 msgr2=0x7ff3a80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:13.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.689+0000 7ff3af670640 1 -- 172.21.15.115:0/1561099411 shutdown_connections 2023-12-01T01:55:13.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.689+0000 7ff3af670640 1 -- 172.21.15.115:0/1561099411 wait complete. 2023-12-01T01:55:13.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T01:55:13.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.926+0000 7f648edcb640 1 Processor -- start 2023-12-01T01:55:13.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.926+0000 7f648edcb640 1 -- start start 2023-12-01T01:55:13.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.927+0000 7f648edcb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6488159bf0 0x7f648815e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:13.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.927+0000 7f648edcb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648815e570 con 0x7f64880ca7c0 2023-12-01T01:55:13.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.927+0000 7f648edcb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64880c9440 con 0x7f64880cb8c0 2023-12-01T01:55:13.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.927+0000 7f648edcb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6488159fd0 con 0x7f6488159bf0 2023-12-01T01:55:13.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.928+0000 7f648cb67640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6488159bf0 0x7f648815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:13.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.928+0000 7f6487fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f64880ca7c0 0x7f64880cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34954/0 (socket says 172.21.15.115:34954) 2023-12-01T01:55:13.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.928+0000 7f6487fff640 1 -- 172.21.15.115:0/3762006595 learned_addr learned my addr 172.21.15.115:0/3762006595 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:13.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.928+0000 7f648cb67640 1 -- 172.21.15.115:0/3762006595 >> v1:172.21.15.142:6789/0 conn(0x7f64880cb8c0 legacy=0x7f648806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:13.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.928+0000 7f648cb67640 1 -- 172.21.15.115:0/3762006595 >> v1:172.21.15.115:6789/0 conn(0x7f64880ca7c0 legacy=0x7f64880cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:13.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.928+0000 7f64877fe640 1 -- 172.21.15.115:0/3762006595 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772210647 0 0) 0x7f648815e570 con 0x7f64880ca7c0 2023-12-01T01:55:13.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.928+0000 7f648cb67640 1 -- 172.21.15.115:0/3762006595 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880c9720 con 0x7f6488159bf0 2023-12-01T01:55:13.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.930+0000 7f648cb67640 1 --2- 172.21.15.115:0/3762006595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6488159bf0 0x7f648815e030 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f64780097a0 tx=0x7f647802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24556a023624203 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:13.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.930+0000 7f64877fe640 1 -- 172.21.15.115:0/3762006595 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478031830 con 0x7f6488159bf0 2023-12-01T01:55:13.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.930+0000 7f64877fe640 1 -- 172.21.15.115:0/3762006595 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6478031990 con 0x7f6488159bf0 2023-12-01T01:55:13.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.930+0000 7f64877fe640 1 -- 172.21.15.115:0/3762006595 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478031d20 con 0x7f6488159bf0 2023-12-01T01:55:13.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.930+0000 7f648edcb640 1 -- 172.21.15.115:0/3762006595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6488159bf0 msgr2=0x7f648815e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:13.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.931+0000 7f648edcb640 1 --2- 172.21.15.115:0/3762006595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6488159bf0 0x7f648815e030 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f64780097a0 tx=0x7f647802b8d0 comp rx=0 tx=0).stop 2023-12-01T01:55:13.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.931+0000 7f648edcb640 1 -- 172.21.15.115:0/3762006595 shutdown_connections 2023-12-01T01:55:13.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.931+0000 7f648edcb640 1 --2- 172.21.15.115:0/3762006595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6488159bf0 0x7f648815e030 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:13.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.931+0000 7f648edcb640 1 -- 172.21.15.115:0/3762006595 >> 172.21.15.115:0/3762006595 conn(0x7f64880662c0 msgr2=0x7f64880b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:13.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.931+0000 7f648edcb640 1 -- 172.21.15.115:0/3762006595 shutdown_connections 2023-12-01T01:55:13.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.931+0000 7f648edcb640 1 -- 172.21.15.115:0/3762006595 wait complete. 2023-12-01T01:55:13.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.933+0000 7f648edcb640 1 Processor -- start 2023-12-01T01:55:13.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.933+0000 7f648edcb640 1 -- start start 2023-12-01T01:55:13.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.934+0000 7f648edcb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64880ca7c0 0x7f6488155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:13.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.934+0000 7f648edcb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64880cb8c0 0x7f64880abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:13.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.934+0000 7f648edcb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6488159bf0 0x7f648815b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:13.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.934+0000 7f648edcb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64880c6860 con 0x7f64880cb8c0 2023-12-01T01:55:13.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.934+0000 7f648edcb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64880ab130 con 0x7f64880ca7c0 2023-12-01T01:55:13.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.934+0000 7f648edcb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64880c6580 con 0x7f6488159bf0 2023-12-01T01:55:13.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.934+0000 7f6487fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64880cb8c0 0x7f64880abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:13.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.934+0000 7f6487fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64880cb8c0 0x7f64880abeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60322/0 (socket says 172.21.15.115:60322) 2023-12-01T01:55:13.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.934+0000 7f6487fff640 1 -- 172.21.15.115:0/627910463 learned_addr learned my addr 172.21.15.115:0/627910463 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:13.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.934+0000 7f648cb67640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64880ca7c0 0x7f6488155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:13.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f648d368640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6488159bf0 0x7f648815b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:13.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f6487fff640 1 -- 172.21.15.115:0/627910463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6488159bf0 msgr2=0x7f648815b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:13.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f6487fff640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6488159bf0 0x7f648815b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:13.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f6487fff640 1 -- 172.21.15.115:0/627910463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64880ca7c0 msgr2=0x7f6488155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:13.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f6487fff640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64880ca7c0 0x7f6488155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:13.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f6487fff640 1 -- 172.21.15.115:0/627910463 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6478009400 con 0x7f64880cb8c0 2023-12-01T01:55:13.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f648cb67640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64880ca7c0 0x7f6488155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:13.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f6487fff640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64880cb8c0 0x7f64880abeb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c430 tx=0x7f647c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:13.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f64857fa640 1 -- 172.21.15.115:0/627910463 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c0074e0 con 0x7f64880cb8c0 2023-12-01T01:55:13.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f64857fa640 1 -- 172.21.15.115:0/627910463 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c007660 con 0x7f64880cb8c0 2023-12-01T01:55:13.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f648edcb640 1 -- 172.21.15.115:0/627910463 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64880ab3d0 con 0x7f64880cb8c0 2023-12-01T01:55:13.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f64857fa640 1 -- 172.21.15.115:0/627910463 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c004a20 con 0x7f64880cb8c0 2023-12-01T01:55:13.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.935+0000 7f648edcb640 1 -- 172.21.15.115:0/627910463 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64880ab990 con 0x7f64880cb8c0 2023-12-01T01:55:13.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.936+0000 7f648edcb640 1 -- 172.21.15.115:0/627910463 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64880abb80 con 0x7f64880cb8c0 2023-12-01T01:55:13.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.938+0000 7f64857fa640 1 -- 172.21.15.115:0/627910463 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f647c004b80 con 0x7f64880cb8c0 2023-12-01T01:55:13.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.939+0000 7f64857fa640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f647406a580 0x7f647406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:13.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.939+0000 7f64857fa640 1 -- 172.21.15.115:0/627910463 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f647c088b00 con 0x7f64880cb8c0 2023-12-01T01:55:13.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.939+0000 7f648cb67640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f647406a580 0x7f647406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:13.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.940+0000 7f648cb67640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f647406a580 0x7f647406ca40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6478036000 tx=0x7f6478008030 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:13.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:13.944+0000 7f64857fa640 1 -- 172.21.15.115:0/627910463 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f647c08ae60 con 0x7f64880cb8c0 2023-12-01T01:55:14.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.132+0000 7f648edcb640 1 -- 172.21.15.115:0/627910463 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6488074210 con 0x7f64880cb8c0 2023-12-01T01:55:14.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.697+0000 7f64857fa640 1 -- 172.21.15.115:0/627910463 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) v1 ==== 169+0+0 (secure 0 0 0) 0x7f647c053d60 con 0x7f64880cb8c0 2023-12-01T01:55:14.699 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T01:55:14.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.702+0000 7f648edcb640 1 -- 172.21.15.115:0/627910463 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f647406a580 msgr2=0x7f647406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:14.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.702+0000 7f648edcb640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f647406a580 0x7f647406ca40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6478036000 tx=0x7f6478008030 comp rx=0 tx=0).stop 2023-12-01T01:55:14.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.702+0000 7f648edcb640 1 -- 172.21.15.115:0/627910463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64880cb8c0 msgr2=0x7f64880abeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:14.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.702+0000 7f648edcb640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64880cb8c0 0x7f64880abeb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c430 tx=0x7f647c00c8f0 comp rx=0 tx=0).stop 2023-12-01T01:55:14.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.702+0000 7f648edcb640 1 -- 172.21.15.115:0/627910463 shutdown_connections 2023-12-01T01:55:14.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.702+0000 7f648edcb640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f647406a580 0x7f647406ca40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:14.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.702+0000 7f648edcb640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6488159bf0 0x7f648815b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:14.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.702+0000 7f648edcb640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64880cb8c0 0x7f64880abeb0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:14.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.702+0000 7f648edcb640 1 --2- 172.21.15.115:0/627910463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64880ca7c0 0x7f6488155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:14.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.702+0000 7f648edcb640 1 -- 172.21.15.115:0/627910463 >> 172.21.15.115:0/627910463 conn(0x7f64880662c0 msgr2=0x7f6488070950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:14.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.703+0000 7f648edcb640 1 -- 172.21.15.115:0/627910463 shutdown_connections 2023-12-01T01:55:14.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.703+0000 7f648edcb640 1 -- 172.21.15.115:0/627910463 wait complete. 2023-12-01T01:55:14.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T01:55:14.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.926+0000 7f0df498a640 1 Processor -- start 2023-12-01T01:55:14.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.926+0000 7f0df498a640 1 -- start start 2023-12-01T01:55:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0df498a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0df015ac00 0x7f0df00ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0df498a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0df00cb800 0x7f0df006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0df498a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df015afe0 con 0x7f0df00aede0 2023-12-01T01:55:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0df498a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df00b12b0 con 0x7f0df015ac00 2023-12-01T01:55:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0df498a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df00b0a10 con 0x7f0df00cb800 2023-12-01T01:55:14.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0deed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0df015ac00 0x7f0df00ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0deed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0df015ac00 0x7f0df00ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60336/0 (socket says 172.21.15.115:60336) 2023-12-01T01:55:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0deed9d640 1 -- 172.21.15.115:0/3319659692 learned_addr learned my addr 172.21.15.115:0/3319659692 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0deed9d640 1 -- 172.21.15.115:0/3319659692 >> v2:172.21.15.142:3300/0 conn(0x7f0df00cb800 msgr2=0x7f0df006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0deed9d640 1 --2- 172.21.15.115:0/3319659692 >> v2:172.21.15.142:3300/0 conn(0x7f0df00cb800 0x7f0df006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0deed9d640 1 -- 172.21.15.115:0/3319659692 >> v1:172.21.15.142:6789/0 conn(0x7f0df00aede0 legacy=0x7f0df00cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:14.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.928+0000 7f0deed9d640 1 -- 172.21.15.115:0/3319659692 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df00b0cf0 con 0x7f0df015ac00 2023-12-01T01:55:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.929+0000 7f0deed9d640 1 --2- 172.21.15.115:0/3319659692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df015ac00 0x7f0df00ae8a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0df0066750 tx=0x7f0de002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37139654f1c4d18 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.930+0000 7f0dee59c640 1 -- 172.21.15.115:0/3319659692 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de00317e0 con 0x7f0df015ac00 2023-12-01T01:55:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.930+0000 7f0dee59c640 1 -- 172.21.15.115:0/3319659692 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0031940 con 0x7f0df015ac00 2023-12-01T01:55:14.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.930+0000 7f0dee59c640 1 -- 172.21.15.115:0/3319659692 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0031cd0 con 0x7f0df015ac00 2023-12-01T01:55:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.930+0000 7f0df498a640 1 -- 172.21.15.115:0/3319659692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df015ac00 msgr2=0x7f0df00ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.930+0000 7f0df498a640 1 --2- 172.21.15.115:0/3319659692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df015ac00 0x7f0df00ae8a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0df0066750 tx=0x7f0de002b880 comp rx=0 tx=0).stop 2023-12-01T01:55:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.931+0000 7f0df498a640 1 -- 172.21.15.115:0/3319659692 shutdown_connections 2023-12-01T01:55:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.931+0000 7f0df498a640 1 --2- 172.21.15.115:0/3319659692 >> v2:172.21.15.142:3300/0 conn(0x7f0df00cb800 0x7f0df006fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.931+0000 7f0df498a640 1 --2- 172.21.15.115:0/3319659692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df015ac00 0x7f0df00ae8a0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.931+0000 7f0df498a640 1 -- 172.21.15.115:0/3319659692 >> 172.21.15.115:0/3319659692 conn(0x7f0df0065960 msgr2=0x7f0df0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:14.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.931+0000 7f0df498a640 1 -- 172.21.15.115:0/3319659692 shutdown_connections 2023-12-01T01:55:14.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.931+0000 7f0df498a640 1 -- 172.21.15.115:0/3319659692 wait complete. 2023-12-01T01:55:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.933+0000 7f0df498a640 1 Processor -- start 2023-12-01T01:55:14.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.933+0000 7f0df498a640 1 -- start start 2023-12-01T01:55:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.934+0000 7f0df498a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df00aede0 0x7f0df00c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.934+0000 7f0df498a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00cb800 0x7f0df00c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:14.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.934+0000 7f0df498a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df015ac00 0x7f0df01f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.934+0000 7f0df498a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df00c3ff0 con 0x7f0df00aede0 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.934+0000 7f0df498a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df00c31b0 con 0x7f0df00cb800 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.934+0000 7f0df498a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df00c33e0 con 0x7f0df015ac00 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.935+0000 7f0deed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df00aede0 0x7f0df00c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.935+0000 7f0deed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df00aede0 0x7f0df00c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60340/0 (socket says 172.21.15.115:60340) 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.935+0000 7f0deed9d640 1 -- 172.21.15.115:0/3951082633 learned_addr learned my addr 172.21.15.115:0/3951082633 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.935+0000 7f0deed9d640 1 -- 172.21.15.115:0/3951082633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df015ac00 msgr2=0x7f0df01f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.935+0000 7f0deed9d640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df015ac00 0x7f0df01f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.935+0000 7f0deed9d640 1 -- 172.21.15.115:0/3951082633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00cb800 msgr2=0x7f0df00c2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.935+0000 7f0de659c640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00cb800 0x7f0df00c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.935+0000 7f0deed9d640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00cb800 0x7f0df00c2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.935+0000 7f0deed9d640 1 -- 172.21.15.115:0/3951082633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de00094d0 con 0x7f0df00aede0 2023-12-01T01:55:14.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.935+0000 7f0de659c640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00cb800 0x7f0df00c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:14.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.936+0000 7f0deed9d640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df00aede0 0x7f0df00c8360 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0de002b8c0 tx=0x7f0de00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.936+0000 7f0de7fff640 1 -- 172.21.15.115:0/3951082633 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0032940 con 0x7f0df00aede0 2023-12-01T01:55:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.936+0000 7f0de7fff640 1 -- 172.21.15.115:0/3951082633 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0032ac0 con 0x7f0df00aede0 2023-12-01T01:55:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.936+0000 7f0df498a640 1 -- 172.21.15.115:0/3951082633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df00c36c0 con 0x7f0df00aede0 2023-12-01T01:55:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.936+0000 7f0de7fff640 1 -- 172.21.15.115:0/3951082633 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0032e70 con 0x7f0df00aede0 2023-12-01T01:55:14.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.936+0000 7f0df498a640 1 -- 172.21.15.115:0/3951082633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df00c3c80 con 0x7f0df00aede0 2023-12-01T01:55:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.937+0000 7f0df498a640 1 -- 172.21.15.115:0/3951082633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df00618d0 con 0x7f0df00aede0 2023-12-01T01:55:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.938+0000 7f0de7fff640 1 -- 172.21.15.115:0/3951082633 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0de004e450 con 0x7f0df00aede0 2023-12-01T01:55:14.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.939+0000 7f0de7fff640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0dd006a580 0x7f0dd006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.939+0000 7f0de7fff640 1 -- 172.21.15.115:0/3951082633 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f0de00aad50 con 0x7f0df00aede0 2023-12-01T01:55:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.939+0000 7f0de659c640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0dd006a580 0x7f0dd006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:14.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.940+0000 7f0de659c640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0dd006a580 0x7f0dd006ca40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0df01f3480 tx=0x7f0dd8009040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:14.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:14.943+0000 7f0de7fff640 1 -- 172.21.15.115:0/3951082633 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0de0076160 con 0x7f0df00aede0 2023-12-01T01:55:15.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.136+0000 7f0df498a640 1 -- 172.21.15.115:0/3951082633 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0df01f35b0 con 0x7f0df00aede0 2023-12-01T01:55:15.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.142+0000 7f0de7fff640 1 -- 172.21.15.115:0/3951082633 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0de0079780 con 0x7f0df00aede0 2023-12-01T01:55:15.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.144+0000 7f0df498a640 1 -- 172.21.15.115:0/3951082633 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0dd006a580 msgr2=0x7f0dd006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.144+0000 7f0df498a640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0dd006a580 0x7f0dd006ca40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0df01f3480 tx=0x7f0dd8009040 comp rx=0 tx=0).stop 2023-12-01T01:55:15.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.144+0000 7f0df498a640 1 -- 172.21.15.115:0/3951082633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df00aede0 msgr2=0x7f0df00c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.144+0000 7f0df498a640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df00aede0 0x7f0df00c8360 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0de002b8c0 tx=0x7f0de00046c0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.144+0000 7f0df498a640 1 -- 172.21.15.115:0/3951082633 shutdown_connections 2023-12-01T01:55:15.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.144+0000 7f0df498a640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0dd006a580 0x7f0dd006ca40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.144+0000 7f0df498a640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df015ac00 0x7f0df01f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.144+0000 7f0df498a640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00cb800 0x7f0df00c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.144+0000 7f0df498a640 1 --2- 172.21.15.115:0/3951082633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df00aede0 0x7f0df00c8360 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.145+0000 7f0df498a640 1 -- 172.21.15.115:0/3951082633 >> 172.21.15.115:0/3951082633 conn(0x7f0df0065960 msgr2=0x7f0df006b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:15.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.145+0000 7f0df498a640 1 -- 172.21.15.115:0/3951082633 shutdown_connections 2023-12-01T01:55:15.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.145+0000 7f0df498a640 1 -- 172.21.15.115:0/3951082633 wait complete. 2023-12-01T01:55:15.164 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T01:55:15.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.367+0000 7f65937d2640 1 Processor -- start 2023-12-01T01:55:15.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.367+0000 7f65937d2640 1 -- start start 2023-12-01T01:55:15.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.369+0000 7f65937d2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f658c0ce1b0 0x7f658c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.369+0000 7f65937d2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658c0cc9a0 con 0x7f658c15ac80 2023-12-01T01:55:15.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.369+0000 7f65937d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658c0cbd90 con 0x7f658c0b0e60 2023-12-01T01:55:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.369+0000 7f65937d2640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658c15b060 con 0x7f658c0ce1b0 2023-12-01T01:55:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.369+0000 7f6590d6d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f658c0ce1b0 0x7f658c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.369+0000 7f6590d6d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f658c0ce1b0 0x7f658c0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60348/0 (socket says 172.21.15.115:60348) 2023-12-01T01:55:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.369+0000 7f6590d6d640 1 -- 172.21.15.115:0/587494683 learned_addr learned my addr 172.21.15.115:0/587494683 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:15.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.370+0000 7f6590d6d640 1 -- 172.21.15.115:0/587494683 >> v1:172.21.15.142:6789/0 conn(0x7f658c0b0e60 legacy=0x7f658c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.370+0000 7f6590d6d640 1 -- 172.21.15.115:0/587494683 >> v1:172.21.15.115:6789/0 conn(0x7f658c15ac80 legacy=0x7f658c0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.370+0000 7f6590d6d640 1 -- 172.21.15.115:0/587494683 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c0cc070 con 0x7f658c0ce1b0 2023-12-01T01:55:15.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.370+0000 7f6583fff640 1 -- 172.21.15.115:0/587494683 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503357104 0 0) 0x7f658c0cc9a0 con 0x7f658c15ac80 2023-12-01T01:55:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.371+0000 7f6590d6d640 1 --2- 172.21.15.115:0/587494683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0ce1b0 0x7f658c0cce30 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f657c009870 tx=0x7f657c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373ef65830433b0d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.371+0000 7f6583fff640 1 -- 172.21.15.115:0/587494683 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c030860 con 0x7f658c0ce1b0 2023-12-01T01:55:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.371+0000 7f6583fff640 1 -- 172.21.15.115:0/587494683 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657c0309c0 con 0x7f658c0ce1b0 2023-12-01T01:55:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.371+0000 7f6583fff640 1 -- 172.21.15.115:0/587494683 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c030d50 con 0x7f658c0ce1b0 2023-12-01T01:55:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.372+0000 7f65937d2640 1 -- 172.21.15.115:0/587494683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0ce1b0 msgr2=0x7f658c0cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.372+0000 7f65937d2640 1 --2- 172.21.15.115:0/587494683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0ce1b0 0x7f658c0cce30 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f657c009870 tx=0x7f657c02b9a0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.372+0000 7f65937d2640 1 -- 172.21.15.115:0/587494683 shutdown_connections 2023-12-01T01:55:15.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.372+0000 7f65937d2640 1 --2- 172.21.15.115:0/587494683 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0ce1b0 0x7f658c0cce30 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.372+0000 7f65937d2640 1 -- 172.21.15.115:0/587494683 >> 172.21.15.115:0/587494683 conn(0x7f658c065960 msgr2=0x7f658c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:15.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.372+0000 7f65937d2640 1 -- 172.21.15.115:0/587494683 shutdown_connections 2023-12-01T01:55:15.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.372+0000 7f65937d2640 1 -- 172.21.15.115:0/587494683 wait complete. 2023-12-01T01:55:15.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.373+0000 7f65937d2640 1 Processor -- start 2023-12-01T01:55:15.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.374+0000 7f65937d2640 1 -- start start 2023-12-01T01:55:15.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.374+0000 7f65937d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0b0e60 0x7f658c14a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f65937d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f658c0ce1b0 0x7f658c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f65937d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f658c15ac80 0x7f658c0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f65937d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c14baf0 con 0x7f658c0b0e60 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f65937d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c0aacd0 con 0x7f658c15ac80 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f65937d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c14ae70 con 0x7f658c0ce1b0 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f659156e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0b0e60 0x7f658c14a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f659156e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0b0e60 0x7f658c14a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60360/0 (socket says 172.21.15.115:60360) 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f659156e640 1 -- 172.21.15.115:0/1412205446 learned_addr learned my addr 172.21.15.115:0/1412205446 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f659156e640 1 -- 172.21.15.115:0/1412205446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f658c0ce1b0 msgr2=0x7f658c0b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f659156e640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f658c0ce1b0 0x7f658c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f659156e640 1 -- 172.21.15.115:0/1412205446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f658c15ac80 msgr2=0x7f658c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f659156e640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f658c15ac80 0x7f658c0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.375+0000 7f659156e640 1 -- 172.21.15.115:0/1412205446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0094d0 con 0x7f658c0b0e60 2023-12-01T01:55:15.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.376+0000 7f659156e640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0b0e60 0x7f658c14a5a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f658800c410 tx=0x7f658800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:15.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.376+0000 7f6581ffb640 1 -- 172.21.15.115:0/1412205446 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65880035f0 con 0x7f658c0b0e60 2023-12-01T01:55:15.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.376+0000 7f6581ffb640 1 -- 172.21.15.115:0/1412205446 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6588003770 con 0x7f658c0b0e60 2023-12-01T01:55:15.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.376+0000 7f65937d2640 1 -- 172.21.15.115:0/1412205446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c14b150 con 0x7f658c0b0e60 2023-12-01T01:55:15.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.376+0000 7f6581ffb640 1 -- 172.21.15.115:0/1412205446 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588003b20 con 0x7f658c0b0e60 2023-12-01T01:55:15.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.376+0000 7f65937d2640 1 -- 172.21.15.115:0/1412205446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658c14b710 con 0x7f658c0b0e60 2023-12-01T01:55:15.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.377+0000 7f65937d2640 1 -- 172.21.15.115:0/1412205446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658c04e890 con 0x7f658c0b0e60 2023-12-01T01:55:15.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.378+0000 7f6581ffb640 1 -- 172.21.15.115:0/1412205446 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6588003c80 con 0x7f658c0b0e60 2023-12-01T01:55:15.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.379+0000 7f6581ffb640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f657006a550 0x7f657006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.379+0000 7f6581ffb640 1 -- 172.21.15.115:0/1412205446 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f6588088090 con 0x7f658c0b0e60 2023-12-01T01:55:15.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.379+0000 7f6590d6d640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f657006a550 0x7f657006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:15.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.380+0000 7f6590d6d640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f657006a550 0x7f657006ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f657c036000 tx=0x7f657c03b040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:15.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.383+0000 7f6581ffb640 1 -- 172.21.15.115:0/1412205446 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f65880535b0 con 0x7f658c0b0e60 2023-12-01T01:55:15.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.574+0000 7f65937d2640 1 -- 172.21.15.115:0/1412205446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f658c04e040 con 0x7f658c0b0e60 2023-12-01T01:55:15.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.584+0000 7f6581ffb640 1 -- 172.21.15.115:0/1412205446 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v44) v1 ==== 120+0+0 (secure 0 0 0) 0x7f6588056bd0 con 0x7f658c0b0e60 2023-12-01T01:55:15.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.587+0000 7f65937d2640 1 -- 172.21.15.115:0/1412205446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f657006a550 msgr2=0x7f657006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.587+0000 7f65937d2640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f657006a550 0x7f657006ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f657c036000 tx=0x7f657c03b040 comp rx=0 tx=0).stop 2023-12-01T01:55:15.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.587+0000 7f65937d2640 1 -- 172.21.15.115:0/1412205446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0b0e60 msgr2=0x7f658c14a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.587+0000 7f65937d2640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0b0e60 0x7f658c14a5a0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f658800c410 tx=0x7f658800c8d0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.589+0000 7f65937d2640 1 -- 172.21.15.115:0/1412205446 shutdown_connections 2023-12-01T01:55:15.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.589+0000 7f65937d2640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f657006a550 0x7f657006ca10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.589+0000 7f65937d2640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f658c0ce1b0 0x7f658c0b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.589+0000 7f65937d2640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f658c15ac80 0x7f658c0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.589+0000 7f65937d2640 1 --2- 172.21.15.115:0/1412205446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f658c0b0e60 0x7f658c14a5a0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.589+0000 7f65937d2640 1 -- 172.21.15.115:0/1412205446 >> 172.21.15.115:0/1412205446 conn(0x7f658c065960 msgr2=0x7f658c06c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:15.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.590+0000 7f65937d2640 1 -- 172.21.15.115:0/1412205446 shutdown_connections 2023-12-01T01:55:15.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.590+0000 7f65937d2640 1 -- 172.21.15.115:0/1412205446 wait complete. 2023-12-01T01:55:15.613 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T01:55:15.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.800+0000 7febce9e3640 1 Processor -- start 2023-12-01T01:55:15.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.800+0000 7febce9e3640 1 -- start start 2023-12-01T01:55:15.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.801+0000 7febce9e3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febc815cd90 0x7febc80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febce9e3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7febc80b5fa0 0x7febc8161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febce9e3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7febc80cf650 0x7febc80cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febce9e3640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc80d0ec0 con 0x7febc80cf650 2023-12-01T01:55:15.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febce9e3640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc80ce5e0 con 0x7febc80b5fa0 2023-12-01T01:55:15.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febce9e3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc8161950 con 0x7febc815cd90 2023-12-01T01:55:15.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febccf80640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7febc80cf650 0x7febc80cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:15.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febccf80640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7febc80cf650 0x7febc80cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60366/0 (socket says 172.21.15.115:60366) 2023-12-01T01:55:15.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febccf80640 1 -- 172.21.15.115:0/3623829464 learned_addr learned my addr 172.21.15.115:0/3623829464 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febc77fe640 1 --2- 172.21.15.115:0/3623829464 >> v2:172.21.15.142:3300/0 conn(0x7febc80b5fa0 0x7febc8161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febc7fff640 1 --2- 172.21.15.115:0/3623829464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febc815cd90 0x7febc80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febccf80640 1 -- 172.21.15.115:0/3623829464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febc815cd90 msgr2=0x7febc80cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.802+0000 7febccf80640 1 --2- 172.21.15.115:0/3623829464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febc815cd90 0x7febc80cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.803+0000 7febccf80640 1 -- 172.21.15.115:0/3623829464 >> v2:172.21.15.142:3300/0 conn(0x7febc80b5fa0 msgr2=0x7febc8161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.803+0000 7febccf80640 1 --2- 172.21.15.115:0/3623829464 >> v2:172.21.15.142:3300/0 conn(0x7febc80b5fa0 0x7febc8161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.803+0000 7febccf80640 1 -- 172.21.15.115:0/3623829464 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80d0000 con 0x7febc80cf650 2023-12-01T01:55:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.803+0000 7febc77fe640 1 --2- 172.21.15.115:0/3623829464 >> v2:172.21.15.142:3300/0 conn(0x7febc80b5fa0 0x7febc8161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.803+0000 7febc7fff640 1 --2- 172.21.15.115:0/3623829464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febc815cd90 0x7febc80cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:15.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.804+0000 7febccf80640 1 --2- 172.21.15.115:0/3623829464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 0x7febc80cfac0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7febc80b5880 tx=0x7febc002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9461bcd0e9af5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:15.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.804+0000 7febc6ffd640 1 -- 172.21.15.115:0/3623829464 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc002ca80 con 0x7febc80cf650 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.804+0000 7febc6ffd640 1 -- 172.21.15.115:0/3623829464 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc002cbe0 con 0x7febc80cf650 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.804+0000 7febc6ffd640 1 -- 172.21.15.115:0/3623829464 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc00343f0 con 0x7febc80cf650 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.805+0000 7febce9e3640 1 -- 172.21.15.115:0/3623829464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 msgr2=0x7febc80cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.805+0000 7febce9e3640 1 --2- 172.21.15.115:0/3623829464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 0x7febc80cfac0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7febc80b5880 tx=0x7febc002bee0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.805+0000 7febce9e3640 1 -- 172.21.15.115:0/3623829464 shutdown_connections 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.805+0000 7febce9e3640 1 --2- 172.21.15.115:0/3623829464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 0x7febc80cfac0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.805+0000 7febce9e3640 1 --2- 172.21.15.115:0/3623829464 >> v2:172.21.15.142:3300/0 conn(0x7febc80b5fa0 0x7febc8161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.805+0000 7febce9e3640 1 --2- 172.21.15.115:0/3623829464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febc815cd90 0x7febc80cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.805+0000 7febce9e3640 1 -- 172.21.15.115:0/3623829464 >> 172.21.15.115:0/3623829464 conn(0x7febc8065960 msgr2=0x7febc8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.805+0000 7febce9e3640 1 -- 172.21.15.115:0/3623829464 shutdown_connections 2023-12-01T01:55:15.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.805+0000 7febce9e3640 1 -- 172.21.15.115:0/3623829464 wait complete. 2023-12-01T01:55:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.806+0000 7febce9e3640 1 Processor -- start 2023-12-01T01:55:15.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.807+0000 7febce9e3640 1 -- start start 2023-12-01T01:55:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febce9e3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febc80b5fa0 0x7febc81f9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febce9e3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 0x7febc81f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febce9e3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febc815cd90 0x7febc81f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febce9e3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc80cb890 con 0x7febc80cf650 2023-12-01T01:55:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febce9e3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc80caf60 con 0x7febc815cd90 2023-12-01T01:55:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febce9e3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc80cb240 con 0x7febc80b5fa0 2023-12-01T01:55:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febc77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 0x7febc81f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febc77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 0x7febc81f7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60382/0 (socket says 172.21.15.115:60382) 2023-12-01T01:55:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febc77fe640 1 -- 172.21.15.115:0/3084510895 learned_addr learned my addr 172.21.15.115:0/3084510895 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:15.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febc77fe640 1 -- 172.21.15.115:0/3084510895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febc80b5fa0 msgr2=0x7febc81f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febc77fe640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febc80b5fa0 0x7febc81f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febc77fe640 1 -- 172.21.15.115:0/3084510895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febc815cd90 msgr2=0x7febc81f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febc77fe640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febc815cd90 0x7febc81f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febc77fe640 1 -- 172.21.15.115:0/3084510895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0009b30 con 0x7febc80cf650 2023-12-01T01:55:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.808+0000 7febc77fe640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 0x7febc81f7b10 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c7f0 tx=0x7febbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.809+0000 7febc4ff9640 1 -- 172.21.15.115:0/3084510895 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc007820 con 0x7febc80cf650 2023-12-01T01:55:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.809+0000 7febc4ff9640 1 -- 172.21.15.115:0/3084510895 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc007980 con 0x7febc80cf650 2023-12-01T01:55:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.809+0000 7febc4ff9640 1 -- 172.21.15.115:0/3084510895 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc007d30 con 0x7febc80cf650 2023-12-01T01:55:15.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.809+0000 7febce9e3640 1 -- 172.21.15.115:0/3084510895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc80cb520 con 0x7febc80cf650 2023-12-01T01:55:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.809+0000 7febce9e3640 1 -- 172.21.15.115:0/3084510895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc81f8050 con 0x7febc80cf650 2023-12-01T01:55:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.810+0000 7febce9e3640 1 -- 172.21.15.115:0/3084510895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc804e890 con 0x7febc80cf650 2023-12-01T01:55:15.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.811+0000 7febc4ff9640 1 -- 172.21.15.115:0/3084510895 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7febbc0113f0 con 0x7febc80cf650 2023-12-01T01:55:15.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.812+0000 7febc4ff9640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7febb406a550 0x7febb406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:15.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.812+0000 7febc4ff9640 1 -- 172.21.15.115:0/3084510895 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5631+0+0 (secure 0 0 0) 0x7febbc088f40 con 0x7febc80cf650 2023-12-01T01:55:15.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.813+0000 7febc7fff640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7febb406a550 0x7febb406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:15.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.813+0000 7febc7fff640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7febb406a550 0x7febb406ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7febb8006fd0 tx=0x7febb8008040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:15.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:15.817+0000 7febc4ff9640 1 -- 172.21.15.115:0/3084510895 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7febbc054240 con 0x7febc80cf650 2023-12-01T01:55:16.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.003+0000 7febce9e3640 1 -- 172.21.15.115:0/3084510895 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7febc80cb9d0 con 0x7febc80cf650 2023-12-01T01:55:16.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.009+0000 7febc4ff9640 1 -- 172.21.15.115:0/3084510895 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v45) v1 ==== 115+0+0 (secure 0 0 0) 0x7febbc057860 con 0x7febc80cf650 2023-12-01T01:55:16.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.011+0000 7febce9e3640 1 -- 172.21.15.115:0/3084510895 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7febb406a550 msgr2=0x7febb406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:16.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.011+0000 7febce9e3640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7febb406a550 0x7febb406ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7febb8006fd0 tx=0x7febb8008040 comp rx=0 tx=0).stop 2023-12-01T01:55:16.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.011+0000 7febce9e3640 1 -- 172.21.15.115:0/3084510895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 msgr2=0x7febc81f7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:16.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.011+0000 7febce9e3640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 0x7febc81f7b10 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c7f0 tx=0x7febbc00ccb0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.012+0000 7febce9e3640 1 -- 172.21.15.115:0/3084510895 shutdown_connections 2023-12-01T01:55:16.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.012+0000 7febce9e3640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7febb406a550 0x7febb406ca10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.012+0000 7febce9e3640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febc815cd90 0x7febc81f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.012+0000 7febce9e3640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febc80cf650 0x7febc81f7b10 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.012+0000 7febce9e3640 1 --2- 172.21.15.115:0/3084510895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febc80b5fa0 0x7febc81f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.012+0000 7febce9e3640 1 -- 172.21.15.115:0/3084510895 >> 172.21.15.115:0/3084510895 conn(0x7febc8065960 msgr2=0x7febc815a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:16.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.012+0000 7febce9e3640 1 -- 172.21.15.115:0/3084510895 shutdown_connections 2023-12-01T01:55:16.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.012+0000 7febce9e3640 1 -- 172.21.15.115:0/3084510895 wait complete. 2023-12-01T01:55:16.032 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-12-01T01:55:16.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.235+0000 7f6699cd5640 1 Processor -- start 2023-12-01T01:55:16.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.235+0000 7f6699cd5640 1 -- start start 2023-12-01T01:55:16.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.237+0000 7f6699cd5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669415cd20 0x7f66940d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.237+0000 7f6699cd5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f66940b5d80 0x7f66940b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.237+0000 7f6699cd5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f66940d1230 0x7f66940d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.237+0000 7f6699cd5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66940b2160 con 0x7f66940b5d80 2023-12-01T01:55:16.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.237+0000 7f6699cd5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66940d1be0 con 0x7f66940d1230 2023-12-01T01:55:16.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.237+0000 7f6699cd5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66940d2a10 con 0x7f669415cd20 2023-12-01T01:55:16.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.237+0000 7f668bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f66940b5d80 0x7f66940b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.237+0000 7f668bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f66940b5d80 0x7f66940b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60386/0 (socket says 172.21.15.115:60386) 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.237+0000 7f668bfff640 1 -- 172.21.15.115:0/4161298487 learned_addr learned my addr 172.21.15.115:0/4161298487 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.238+0000 7f6693fff640 1 --2- 172.21.15.115:0/4161298487 >> v2:172.21.15.142:3300/0 conn(0x7f66940d1230 0x7f66940d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.238+0000 7f66937fe640 1 --2- 172.21.15.115:0/4161298487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669415cd20 0x7f66940d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.238+0000 7f668bfff640 1 -- 172.21.15.115:0/4161298487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669415cd20 msgr2=0x7f66940d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.238+0000 7f668bfff640 1 --2- 172.21.15.115:0/4161298487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669415cd20 0x7f66940d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.238+0000 7f668bfff640 1 -- 172.21.15.115:0/4161298487 >> v2:172.21.15.142:3300/0 conn(0x7f66940d1230 msgr2=0x7f66940d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.238+0000 7f668bfff640 1 --2- 172.21.15.115:0/4161298487 >> v2:172.21.15.142:3300/0 conn(0x7f66940d1230 0x7f66940d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.238+0000 7f668bfff640 1 -- 172.21.15.115:0/4161298487 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66940ce2e0 con 0x7f66940b5d80 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.238+0000 7f6693fff640 1 --2- 172.21.15.115:0/4161298487 >> v2:172.21.15.142:3300/0 conn(0x7f66940d1230 0x7f66940d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:16.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.238+0000 7f66937fe640 1 --2- 172.21.15.115:0/4161298487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669415cd20 0x7f66940d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:16.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.239+0000 7f668bfff640 1 --2- 172.21.15.115:0/4161298487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940b5d80 0x7f66940b8570 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f66940b5b50 tx=0x7f667c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4a434cd7eb3e53 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:16.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.239+0000 7f6692ffd640 1 -- 172.21.15.115:0/4161298487 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0317c0 con 0x7f66940b5d80 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.239+0000 7f6692ffd640 1 -- 172.21.15.115:0/4161298487 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c031920 con 0x7f66940b5d80 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.239+0000 7f6692ffd640 1 -- 172.21.15.115:0/4161298487 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c031cb0 con 0x7f66940b5d80 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.240+0000 7f6699cd5640 1 -- 172.21.15.115:0/4161298487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940b5d80 msgr2=0x7f66940b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.240+0000 7f6699cd5640 1 --2- 172.21.15.115:0/4161298487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940b5d80 0x7f66940b8570 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f66940b5b50 tx=0x7f667c02b860 comp rx=0 tx=0).stop 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.240+0000 7f6699cd5640 1 -- 172.21.15.115:0/4161298487 shutdown_connections 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.240+0000 7f6699cd5640 1 --2- 172.21.15.115:0/4161298487 >> v2:172.21.15.142:3300/0 conn(0x7f66940d1230 0x7f66940d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.240+0000 7f6699cd5640 1 --2- 172.21.15.115:0/4161298487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940b5d80 0x7f66940b8570 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.240+0000 7f6699cd5640 1 --2- 172.21.15.115:0/4161298487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f669415cd20 0x7f66940d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.240+0000 7f6699cd5640 1 -- 172.21.15.115:0/4161298487 >> 172.21.15.115:0/4161298487 conn(0x7f669406a0e0 msgr2=0x7f669406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.240+0000 7f6699cd5640 1 -- 172.21.15.115:0/4161298487 shutdown_connections 2023-12-01T01:55:16.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.240+0000 7f6699cd5640 1 -- 172.21.15.115:0/4161298487 wait complete. 2023-12-01T01:55:16.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.242+0000 7f6699cd5640 1 Processor -- start 2023-12-01T01:55:16.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.242+0000 7f6699cd5640 1 -- start start 2023-12-01T01:55:16.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.243+0000 7f6699cd5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b5d80 0x7f6694152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.243+0000 7f6699cd5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940d1230 0x7f66940c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.243+0000 7f6699cd5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f669415cd20 0x7f66940c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.243+0000 7f6699cd5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66940b4590 con 0x7f66940d1230 2023-12-01T01:55:16.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.243+0000 7f6699cd5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66940b47e0 con 0x7f669415cd20 2023-12-01T01:55:16.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.243+0000 7f6699cd5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66940b4ac0 con 0x7f66940b5d80 2023-12-01T01:55:16.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f668bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940d1230 0x7f66940c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f668bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940d1230 0x7f66940c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60396/0 (socket says 172.21.15.115:60396) 2023-12-01T01:55:16.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f668bfff640 1 -- 172.21.15.115:0/3097001661 learned_addr learned my addr 172.21.15.115:0/3097001661 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:16.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f6693fff640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f669415cd20 0x7f66940c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f66937fe640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b5d80 0x7f6694152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f668bfff640 1 -- 172.21.15.115:0/3097001661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b5d80 msgr2=0x7f6694152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:16.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f668bfff640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b5d80 0x7f6694152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f668bfff640 1 -- 172.21.15.115:0/3097001661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f669415cd20 msgr2=0x7f66940c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:16.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f668bfff640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f669415cd20 0x7f66940c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f668bfff640 1 -- 172.21.15.115:0/3097001661 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c0094d0 con 0x7f66940d1230 2023-12-01T01:55:16.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f6693fff640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f669415cd20 0x7f66940c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:16.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.244+0000 7f66937fe640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b5d80 0x7f6694152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:16.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.245+0000 7f668bfff640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940d1230 0x7f66940c8630 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f667c000c00 tx=0x7f667c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:16.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.245+0000 7f6690ff9640 1 -- 172.21.15.115:0/3097001661 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0328e0 con 0x7f66940d1230 2023-12-01T01:55:16.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.245+0000 7f6690ff9640 1 -- 172.21.15.115:0/3097001661 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c032a80 con 0x7f66940d1230 2023-12-01T01:55:16.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.245+0000 7f6699cd5640 1 -- 172.21.15.115:0/3097001661 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66940b4da0 con 0x7f66940d1230 2023-12-01T01:55:16.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.245+0000 7f6690ff9640 1 -- 172.21.15.115:0/3097001661 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c032e30 con 0x7f66940d1230 2023-12-01T01:55:16.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.245+0000 7f6699cd5640 1 -- 172.21.15.115:0/3097001661 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66940b5360 con 0x7f66940d1230 2023-12-01T01:55:16.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.246+0000 7f6699cd5640 1 -- 172.21.15.115:0/3097001661 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6694155140 con 0x7f66940d1230 2023-12-01T01:55:16.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.247+0000 7f6690ff9640 1 -- 172.21.15.115:0/3097001661 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f667c0453f0 con 0x7f66940d1230 2023-12-01T01:55:16.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.248+0000 7f6690ff9640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f667406a550 0x7f667406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.249+0000 7f6690ff9640 1 -- 172.21.15.115:0/3097001661 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f667c0aaff0 con 0x7f66940d1230 2023-12-01T01:55:16.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.249+0000 7f66937fe640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f667406a550 0x7f667406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.249+0000 7f66937fe640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f667406a550 0x7f667406ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6684006fd0 tx=0x7f6684008040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:16.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.254+0000 7f6690ff9640 1 -- 172.21.15.115:0/3097001661 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f667c0764e0 con 0x7f66940d1230 2023-12-01T01:55:16.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.497+0000 7f6699cd5640 1 -- 172.21.15.115:0/3097001661 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f669404e890 con 0x7f66940d1230 2023-12-01T01:55:16.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.706+0000 7f6690ff9640 1 -- 172.21.15.115:0/3097001661 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v736) v1 ==== 148+0+0 (secure 0 0 0) 0x7f667c079b00 con 0x7f66940d1230 2023-12-01T01:55:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 -- 172.21.15.115:0/3097001661 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f667406a550 msgr2=0x7f667406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f667406a550 0x7f667406ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6684006fd0 tx=0x7f6684008040 comp rx=0 tx=0).stop 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 -- 172.21.15.115:0/3097001661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940d1230 msgr2=0x7f66940c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940d1230 0x7f66940c8630 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f667c000c00 tx=0x7f667c0046c0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 -- 172.21.15.115:0/3097001661 shutdown_connections 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f667406a550 0x7f667406ca10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f669415cd20 0x7f66940c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66940d1230 0x7f66940c8630 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 --2- 172.21.15.115:0/3097001661 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b5d80 0x7f6694152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 -- 172.21.15.115:0/3097001661 >> 172.21.15.115:0/3097001661 conn(0x7f669406a0e0 msgr2=0x7f669406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 -- 172.21.15.115:0/3097001661 shutdown_connections 2023-12-01T01:55:16.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.710+0000 7f6699cd5640 1 -- 172.21.15.115:0/3097001661 wait complete. 2023-12-01T01:55:16.732 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-12-01T01:55:16.732 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-12-01T01:55:16.732 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-01T01:55:16.732 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-12-01T01:55:16.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.942+0000 7fe6c8fab640 1 Processor -- start 2023-12-01T01:55:16.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.942+0000 7fe6c8fab640 1 -- start start 2023-12-01T01:55:16.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.943+0000 7fe6c8fab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe6c415e2f0 0x7fe6c40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.943+0000 7fe6c8fab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6c406f2f0 0x7fe6c40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.943+0000 7fe6c8fab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe6c40adf10 0x7fe6c40c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.943+0000 7fe6c8fab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40add10 con 0x7fe6c415e2f0 2023-12-01T01:55:16.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.943+0000 7fe6c8fab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40c5760 con 0x7fe6c40adf10 2023-12-01T01:55:16.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.943+0000 7fe6c8fab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c415e6d0 con 0x7fe6c406f2f0 2023-12-01T01:55:16.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.944+0000 7fe6c259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe6c415e2f0 0x7fe6c40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.944+0000 7fe6c259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe6c415e2f0 0x7fe6c40ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60398/0 (socket says 172.21.15.115:60398) 2023-12-01T01:55:16.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.944+0000 7fe6c259c640 1 -- 172.21.15.115:0/1436147389 learned_addr learned my addr 172.21.15.115:0/1436147389 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:16.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.944+0000 7fe6c2d9d640 1 --2- 172.21.15.115:0/1436147389 >> v2:172.21.15.142:3300/0 conn(0x7fe6c40adf10 0x7fe6c40c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.944+0000 7fe6bbfff640 1 --2- 172.21.15.115:0/1436147389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6c406f2f0 0x7fe6c40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.944+0000 7fe6c259c640 1 -- 172.21.15.115:0/1436147389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6c406f2f0 msgr2=0x7fe6c40caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:16.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.944+0000 7fe6c259c640 1 --2- 172.21.15.115:0/1436147389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6c406f2f0 0x7fe6c40caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.944+0000 7fe6c259c640 1 -- 172.21.15.115:0/1436147389 >> v2:172.21.15.142:3300/0 conn(0x7fe6c40adf10 msgr2=0x7fe6c40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:16.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.944+0000 7fe6c259c640 1 --2- 172.21.15.115:0/1436147389 >> v2:172.21.15.142:3300/0 conn(0x7fe6c40adf10 0x7fe6c40c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.944+0000 7fe6c259c640 1 -- 172.21.15.115:0/1436147389 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40cf7b0 con 0x7fe6c415e2f0 2023-12-01T01:55:16.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.946+0000 7fe6c259c640 1 --2- 172.21.15.115:0/1436147389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 0x7fe6c40ca660 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40cb2b0 tx=0x7fe6b402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cddd4905be5c288 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:16.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.946+0000 7fe6c1d9b640 1 -- 172.21.15.115:0/1436147389 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4031750 con 0x7fe6c415e2f0 2023-12-01T01:55:16.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.946+0000 7fe6c1d9b640 1 -- 172.21.15.115:0/1436147389 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b40318b0 con 0x7fe6c415e2f0 2023-12-01T01:55:16.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.946+0000 7fe6c1d9b640 1 -- 172.21.15.115:0/1436147389 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4031c40 con 0x7fe6c415e2f0 2023-12-01T01:55:16.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.947+0000 7fe6c8fab640 1 -- 172.21.15.115:0/1436147389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 msgr2=0x7fe6c40ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:16.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.947+0000 7fe6c8fab640 1 --2- 172.21.15.115:0/1436147389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 0x7fe6c40ca660 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40cb2b0 tx=0x7fe6b402b7f0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.947+0000 7fe6c8fab640 1 -- 172.21.15.115:0/1436147389 shutdown_connections 2023-12-01T01:55:16.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.947+0000 7fe6c8fab640 1 --2- 172.21.15.115:0/1436147389 >> v2:172.21.15.142:3300/0 conn(0x7fe6c40adf10 0x7fe6c40c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.947+0000 7fe6c8fab640 1 --2- 172.21.15.115:0/1436147389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6c406f2f0 0x7fe6c40caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.947+0000 7fe6c8fab640 1 --2- 172.21.15.115:0/1436147389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 0x7fe6c40ca660 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.947+0000 7fe6c8fab640 1 -- 172.21.15.115:0/1436147389 >> 172.21.15.115:0/1436147389 conn(0x7fe6c40ce880 msgr2=0x7fe6c40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:16.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.947+0000 7fe6c8fab640 1 -- 172.21.15.115:0/1436147389 shutdown_connections 2023-12-01T01:55:16.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.947+0000 7fe6c8fab640 1 -- 172.21.15.115:0/1436147389 wait complete. 2023-12-01T01:55:16.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.949+0000 7fe6c8fab640 1 Processor -- start 2023-12-01T01:55:16.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.949+0000 7fe6c8fab640 1 -- start start 2023-12-01T01:55:16.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.950+0000 7fe6c8fab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c406f2f0 0x7fe6c40c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.950+0000 7fe6c8fab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6c40adf10 0x7fe6c4158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.950+0000 7fe6c8fab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 0x7fe6c40ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.950+0000 7fe6c8fab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40c6bd0 con 0x7fe6c415e2f0 2023-12-01T01:55:16.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.950+0000 7fe6c8fab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40c23b0 con 0x7fe6c406f2f0 2023-12-01T01:55:16.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.950+0000 7fe6c8fab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40c7fd0 con 0x7fe6c40adf10 2023-12-01T01:55:16.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.950+0000 7fe6c2d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 0x7fe6c40ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.950+0000 7fe6c2d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 0x7fe6c40ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60402/0 (socket says 172.21.15.115:60402) 2023-12-01T01:55:16.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.950+0000 7fe6c2d9d640 1 -- 172.21.15.115:0/342147430 learned_addr learned my addr 172.21.15.115:0/342147430 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:16.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.950+0000 7fe6bbfff640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6c40adf10 0x7fe6c4158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6c259c640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c406f2f0 0x7fe6c40c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6c2d9d640 1 -- 172.21.15.115:0/342147430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6c40adf10 msgr2=0x7fe6c4158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:16.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6c2d9d640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6c40adf10 0x7fe6c4158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6c2d9d640 1 -- 172.21.15.115:0/342147430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c406f2f0 msgr2=0x7fe6c40c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:16.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6c2d9d640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c406f2f0 0x7fe6c40c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:16.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6c2d9d640 1 -- 172.21.15.115:0/342147430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40094d0 con 0x7fe6c415e2f0 2023-12-01T01:55:16.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6c2d9d640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 0x7fe6c40ca5c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009fd0 tx=0x7fe6bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:16.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6baffd640 1 -- 172.21.15.115:0/342147430 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc014420 con 0x7fe6c415e2f0 2023-12-01T01:55:16.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6baffd640 1 -- 172.21.15.115:0/342147430 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc014580 con 0x7fe6c415e2f0 2023-12-01T01:55:16.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6baffd640 1 -- 172.21.15.115:0/342147430 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc014930 con 0x7fe6c415e2f0 2023-12-01T01:55:16.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6c8fab640 1 -- 172.21.15.115:0/342147430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c4158580 con 0x7fe6c415e2f0 2023-12-01T01:55:16.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.951+0000 7fe6c8fab640 1 -- 172.21.15.115:0/342147430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c40c8250 con 0x7fe6c415e2f0 2023-12-01T01:55:16.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.952+0000 7fe6c8fab640 1 -- 172.21.15.115:0/342147430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c406fda0 con 0x7fe6c415e2f0 2023-12-01T01:55:16.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.953+0000 7fe6baffd640 1 -- 172.21.15.115:0/342147430 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe6bc014a90 con 0x7fe6c415e2f0 2023-12-01T01:55:16.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.954+0000 7fe6baffd640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe6a406a550 0x7fe6a406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:16.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.954+0000 7fe6baffd640 1 -- 172.21.15.115:0/342147430 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fe6bc089370 con 0x7fe6c415e2f0 2023-12-01T01:55:16.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.955+0000 7fe6c259c640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe6a406a550 0x7fe6a406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:16.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.955+0000 7fe6c259c640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe6a406a550 0x7fe6a406ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4035000 tx=0x7fe6b4008030 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:16.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:16.958+0000 7fe6baffd640 1 -- 172.21.15.115:0/342147430 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe6bc054890 con 0x7fe6c415e2f0 2023-12-01T01:55:17.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.221+0000 7fe6c8fab640 1 -- 172.21.15.115:0/342147430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7fe6c404e890 con 0x7fe6c415e2f0 2023-12-01T01:55:17.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.711+0000 7fe6baffd640 1 -- 172.21.15.115:0/342147430 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v737) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe6bc057eb0 con 0x7fe6c415e2f0 2023-12-01T01:55:17.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.714+0000 7fe6c8fab640 1 -- 172.21.15.115:0/342147430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe6a406a550 msgr2=0x7fe6a406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:17.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.714+0000 7fe6c8fab640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe6a406a550 0x7fe6a406ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4035000 tx=0x7fe6b4008030 comp rx=0 tx=0).stop 2023-12-01T01:55:17.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.714+0000 7fe6c8fab640 1 -- 172.21.15.115:0/342147430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 msgr2=0x7fe6c40ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:17.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.714+0000 7fe6c8fab640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 0x7fe6c40ca5c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009fd0 tx=0x7fe6bc00e4b0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.715+0000 7fe6c8fab640 1 -- 172.21.15.115:0/342147430 shutdown_connections 2023-12-01T01:55:17.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.715+0000 7fe6c8fab640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe6a406a550 0x7fe6a406ca10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.715+0000 7fe6c8fab640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6c40adf10 0x7fe6c4158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.715+0000 7fe6c8fab640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c415e2f0 0x7fe6c40ca5c0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.715+0000 7fe6c8fab640 1 --2- 172.21.15.115:0/342147430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c406f2f0 0x7fe6c40c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.715+0000 7fe6c8fab640 1 -- 172.21.15.115:0/342147430 >> 172.21.15.115:0/342147430 conn(0x7fe6c40ce880 msgr2=0x7fe6c40c1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:17.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.715+0000 7fe6c8fab640 1 -- 172.21.15.115:0/342147430 shutdown_connections 2023-12-01T01:55:17.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.715+0000 7fe6c8fab640 1 -- 172.21.15.115:0/342147430 wait complete. 2023-12-01T01:55:17.738 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T01:55:17.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.954+0000 7f88f35f0640 1 Processor -- start 2023-12-01T01:55:17.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.955+0000 7f88f35f0640 1 -- start start 2023-12-01T01:55:17.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.956+0000 7f88f35f0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f88ec0d0db0 0x7f88ec0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:17.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.956+0000 7f88f35f0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec0d2a10 con 0x7f88ec0b4b90 2023-12-01T01:55:17.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.956+0000 7f88f35f0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec0ce470 con 0x7f88ec15cf70 2023-12-01T01:55:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.956+0000 7f88f35f0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ec15d350 con 0x7f88ec0d0db0 2023-12-01T01:55:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.956+0000 7f88f1b8d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f88ec0b4b90 0x7f88ec0b5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54670/0 (socket says 172.21.15.115:54670) 2023-12-01T01:55:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.956+0000 7f88f1b8d640 1 -- 172.21.15.115:0/1500421763 learned_addr learned my addr 172.21.15.115:0/1500421763 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.957+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247180308 0 0) 0x7f88ec0d2a10 con 0x7f88ec0b4b90 2023-12-01T01:55:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.957+0000 7f88f0b8b640 1 --2- 172.21.15.115:0/1500421763 >> v2:172.21.15.142:3300/0 conn(0x7f88ec0d0db0 0x7f88ec0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:17.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.957+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d8003520 con 0x7f88ec0b4b90 2023-12-01T01:55:17.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.957+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1504152306 0 0) 0x7f88d8003520 con 0x7f88ec0b4b90 2023-12-01T01:55:17.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.957+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 >> v2:172.21.15.142:3300/0 conn(0x7f88ec0d0db0 msgr2=0x7f88ec0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:17.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.957+0000 7f88e3fff640 1 --2- 172.21.15.115:0/1500421763 >> v2:172.21.15.142:3300/0 conn(0x7f88ec0d0db0 0x7f88ec0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.957+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 >> v1:172.21.15.142:6789/0 conn(0x7f88ec15cf70 legacy=0x7f88ec0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.957+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec0b4fe0 con 0x7f88ec0b4b90 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.957+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f88e40019f0 con 0x7f88ec0b4b90 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 >> v1:172.21.15.115:6789/0 conn(0x7f88ec0b4b90 legacy=0x7f88ec0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88e3fff640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d8003970 0x7f88d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88e3fff640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88d80047c0 0x7f88d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88e3fff640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88d8005180 0x7f88d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec15d350 con 0x7f88d80047c0 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d800c030 con 0x7f88d8005180 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d800c1f0 con 0x7f88d8003970 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88f0b8b640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88d80047c0 0x7f88d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88f0b8b640 1 -- 172.21.15.115:0/1500421763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d8003970 msgr2=0x7f88d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88f1b8d640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88d8005180 0x7f88d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88f138c640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d8003970 0x7f88d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88f0b8b640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d8003970 0x7f88d8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88f0b8b640 1 -- 172.21.15.115:0/1500421763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88d8005180 msgr2=0x7f88d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:17.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88f0b8b640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88d8005180 0x7f88d800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88f0b8b640 1 -- 172.21.15.115:0/1500421763 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d8003520 con 0x7f88d80047c0 2023-12-01T01:55:17.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88f1b8d640 1 -- 172.21.15.115:0/1500421763 reap_dead start 2023-12-01T01:55:17.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.958+0000 7f88f138c640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d8003970 0x7f88d8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:17.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.959+0000 7f88f0b8b640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88d80047c0 0x7f88d8004c40 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009ac0 tx=0x7f88d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a62455349e3dc7b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:17.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.960+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc02c6b0 con 0x7f88d80047c0 2023-12-01T01:55:17.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.960+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88dc02c810 con 0x7f88d80047c0 2023-12-01T01:55:17.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.960+0000 7f88e3fff640 1 -- 172.21.15.115:0/1500421763 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc02cba0 con 0x7f88d80047c0 2023-12-01T01:55:17.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.960+0000 7f88f35f0640 1 -- 172.21.15.115:0/1500421763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88d80047c0 msgr2=0x7f88d8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:17.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.960+0000 7f88f35f0640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88d80047c0 0x7f88d8004c40 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009ac0 tx=0x7f88d80057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.960+0000 7f88f35f0640 1 -- 172.21.15.115:0/1500421763 shutdown_connections 2023-12-01T01:55:17.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.960+0000 7f88f35f0640 1 --2- 172.21.15.115:0/1500421763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88d80047c0 0x7f88d8004c40 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.960+0000 7f88f35f0640 1 -- 172.21.15.115:0/1500421763 >> 172.21.15.115:0/1500421763 conn(0x7f88ec065d80 msgr2=0x7f88ec066190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:17.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.960+0000 7f88f35f0640 1 -- 172.21.15.115:0/1500421763 shutdown_connections 2023-12-01T01:55:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.961+0000 7f88f35f0640 1 -- 172.21.15.115:0/1500421763 wait complete. 2023-12-01T01:55:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.962+0000 7f88f35f0640 1 Processor -- start 2023-12-01T01:55:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.962+0000 7f88f35f0640 1 -- start start 2023-12-01T01:55:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.963+0000 7f88f35f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d80047c0 0x7f88ec152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.963+0000 7f88f35f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88ec151e40 0x7f88ec148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.963+0000 7f88f35f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88ec154960 0x7f88ec154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.963+0000 7f88f35f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec1534d0 con 0x7f88ec154960 2023-12-01T01:55:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.963+0000 7f88f35f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec1532a0 con 0x7f88ec151e40 2023-12-01T01:55:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.963+0000 7f88f35f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec1493f0 con 0x7f88d80047c0 2023-12-01T01:55:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f1b8d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88ec154960 0x7f88ec154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f1b8d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88ec154960 0x7f88ec154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32982/0 (socket says 172.21.15.115:32982) 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f1b8d640 1 -- 172.21.15.115:0/1036605139 learned_addr learned my addr 172.21.15.115:0/1036605139 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f138c640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d80047c0 0x7f88ec152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f0b8b640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88ec151e40 0x7f88ec148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f1b8d640 1 -- 172.21.15.115:0/1036605139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d80047c0 msgr2=0x7f88ec152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f1b8d640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d80047c0 0x7f88ec152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f1b8d640 1 -- 172.21.15.115:0/1036605139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88ec151e40 msgr2=0x7f88ec148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f1b8d640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88ec151e40 0x7f88ec148eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f1b8d640 1 -- 172.21.15.115:0/1036605139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec155730 con 0x7f88ec154960 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f138c640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d80047c0 0x7f88ec152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f0b8b640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88ec151e40 0x7f88ec148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.964+0000 7f88f1b8d640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88ec154960 0x7f88ec154de0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f88ec06bdd0 tx=0x7f88e400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.965+0000 7f88e1ffb640 1 -- 172.21.15.115:0/1036605139 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e400de70 con 0x7f88ec154960 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.965+0000 7f88e1ffb640 1 -- 172.21.15.115:0/1036605139 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e400bb80 con 0x7f88ec154960 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.965+0000 7f88f35f0640 1 -- 172.21.15.115:0/1036605139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ec155a10 con 0x7f88ec154960 2023-12-01T01:55:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.965+0000 7f88e1ffb640 1 -- 172.21.15.115:0/1036605139 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4007070 con 0x7f88ec154960 2023-12-01T01:55:17.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.965+0000 7f88f35f0640 1 -- 172.21.15.115:0/1036605139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec155fd0 con 0x7f88ec154960 2023-12-01T01:55:17.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.966+0000 7f88f35f0640 1 -- 172.21.15.115:0/1036605139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ec04e890 con 0x7f88ec154960 2023-12-01T01:55:17.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.967+0000 7f88e1ffb640 1 -- 172.21.15.115:0/1036605139 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f88e40071d0 con 0x7f88ec154960 2023-12-01T01:55:17.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.968+0000 7f88e1ffb640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f88d006a550 0x7f88d006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:17.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.968+0000 7f88e1ffb640 1 -- 172.21.15.115:0/1036605139 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f88e4088f00 con 0x7f88ec154960 2023-12-01T01:55:17.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.968+0000 7f88f138c640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f88d006a550 0x7f88d006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:17.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.969+0000 7f88f138c640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f88d006a550 0x7f88d006ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88ec04f940 tx=0x7f88e8009040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:17.972+0000 7f88e1ffb640 1 -- 172.21.15.115:0/1036605139 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f88e4054290 con 0x7f88ec154960 2023-12-01T01:55:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.162+0000 7f88f35f0640 1 -- 172.21.15.115:0/1036605139 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f88ec0b4fe0 con 0x7f88ec154960 2023-12-01T01:55:18.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.169+0000 7f88e1ffb640 1 -- 172.21.15.115:0/1036605139 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f88ec0b4fe0 con 0x7f88ec154960 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 -- 172.21.15.115:0/1036605139 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f88d006a550 msgr2=0x7f88d006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f88d006a550 0x7f88d006ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88ec04f940 tx=0x7f88e8009040 comp rx=0 tx=0).stop 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 -- 172.21.15.115:0/1036605139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88ec154960 msgr2=0x7f88ec154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88ec154960 0x7f88ec154de0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f88ec06bdd0 tx=0x7f88e400b4a0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 -- 172.21.15.115:0/1036605139 shutdown_connections 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f88d006a550 0x7f88d006ca10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88ec154960 0x7f88ec154de0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f88ec151e40 0x7f88ec148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 --2- 172.21.15.115:0/1036605139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88d80047c0 0x7f88ec152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 -- 172.21.15.115:0/1036605139 >> 172.21.15.115:0/1036605139 conn(0x7f88ec065d80 msgr2=0x7f88ec0b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:18.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 -- 172.21.15.115:0/1036605139 shutdown_connections 2023-12-01T01:55:18.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.172+0000 7f88f35f0640 1 -- 172.21.15.115:0/1036605139 wait complete. 2023-12-01T01:55:18.194 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T01:55:18.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:55:18.317 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T01:55:18.445 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T01:55:18.445 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:55:18.445 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:55:18.504 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:55:18.504 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:55:18.504 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:55:18.564 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:55:18.564 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:55:18.564 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:55:18.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T01:55:18.624 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T01:55:18.624 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:55:18.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T01:55:18.684 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T01:55:18.685 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:55:18.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.895+0000 7f2ad5f9c640 1 Processor -- start 2023-12-01T01:55:18.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.895+0000 7f2ad5f9c640 1 -- start start 2023-12-01T01:55:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.896+0000 7f2ad5f9c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0159dd0 0x7f2ad00c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.897+0000 7f2ad5f9c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2ad006b6f0 0x7f2ad00c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.897+0000 7f2ad5f9c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad006f8b0 con 0x7f2ad00ade80 2023-12-01T01:55:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.897+0000 7f2ad5f9c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad00adcb0 con 0x7f2ad006b6f0 2023-12-01T01:55:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.897+0000 7f2ad5f9c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00ad9d0 con 0x7f2ad0159dd0 2023-12-01T01:55:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.897+0000 7f2acffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2ad00ade80 0x7f2ad00cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54672/0 (socket says 172.21.15.115:54672) 2023-12-01T01:55:18.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.897+0000 7f2acffff640 1 -- 172.21.15.115:0/1397139780 learned_addr learned my addr 172.21.15.115:0/1397139780 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.897+0000 7f2acf7fe640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0159dd0 0x7f2ad00c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:18.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.897+0000 7f2aceffd640 1 --2- 172.21.15.115:0/1397139780 >> v2:172.21.15.142:3300/0 conn(0x7f2ad006b6f0 0x7f2ad00c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.897+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096162466 0 0) 0x7f2ad006f8b0 con 0x7f2ad00ade80 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.897+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ab8003520 con 0x7f2ad00ade80 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2590951913 0 0) 0x7f2ab8003520 con 0x7f2ad00ade80 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0159dd0 msgr2=0x7f2ad00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0159dd0 0x7f2ad00c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 >> v2:172.21.15.142:3300/0 conn(0x7f2ad006b6f0 msgr2=0x7f2ad00c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 --2- 172.21.15.115:0/1397139780 >> v2:172.21.15.142:3300/0 conn(0x7f2ad006b6f0 0x7f2ad00c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad00c9000 con 0x7f2ad00ade80 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2ac80019f0 con 0x7f2ad00ade80 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 >> v1:172.21.15.115:6789/0 conn(0x7f2ad00ade80 legacy=0x7f2ad00cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab80039e0 0x7f2ab8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0159dd0 0x7f2ab80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:18.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ab8004d00 0x7f2ab8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00adcb0 con 0x7f2ab8004d00 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00ad9d0 con 0x7f2ab80039e0 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab800bb30 con 0x7f2ad0159dd0 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2aceffd640 1 --2- 172.21.15.115:0/1397139780 >> v2:172.21.15.142:3300/0 conn(0x7f2ad006b6f0 0x7f2ad00c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.898+0000 7f2acffff640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ab8004d00 0x7f2ab8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.899+0000 7f2acffff640 1 -- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0159dd0 msgr2=0x7f2ab80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.899+0000 7f2aceffd640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0159dd0 0x7f2ab80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.899+0000 7f2acf7fe640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab80039e0 0x7f2ab8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.899+0000 7f2acffff640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0159dd0 0x7f2ab80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.899+0000 7f2acffff640 1 -- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab80039e0 msgr2=0x7f2ab8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.899+0000 7f2acffff640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab80039e0 0x7f2ab8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.899+0000 7f2acffff640 1 -- 172.21.15.115:0/1397139780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab8003520 con 0x7f2ab8004d00 2023-12-01T01:55:18.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.899+0000 7f2acf7fe640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab80039e0 0x7f2ab8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.900+0000 7f2acffff640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ab8004d00 0x7f2ab8005180 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800a860 tx=0x7f2ab80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cba5718e878f53 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.900+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8034630 con 0x7f2ab8004d00 2023-12-01T01:55:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.900+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac802f070 con 0x7f2ab8004d00 2023-12-01T01:55:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.900+0000 7f2ace7fc640 1 -- 172.21.15.115:0/1397139780 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80349f0 con 0x7f2ab8004d00 2023-12-01T01:55:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.901+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1397139780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ab8004d00 msgr2=0x7f2ab8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.901+0000 7f2ad5f9c640 1 --2- 172.21.15.115:0/1397139780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ab8004d00 0x7f2ab8005180 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800a860 tx=0x7f2ab80057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.901+0000 7f2acffff640 1 -- 172.21.15.115:0/1397139780 reap_dead start 2023-12-01T01:55:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.901+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1397139780 shutdown_connections 2023-12-01T01:55:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.901+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1397139780 >> 172.21.15.115:0/1397139780 conn(0x7f2ad00ce880 msgr2=0x7f2ad00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:18.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.901+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1397139780 shutdown_connections 2023-12-01T01:55:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.901+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1397139780 wait complete. 2023-12-01T01:55:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.902+0000 7f2ad5f9c640 1 Processor -- start 2023-12-01T01:55:18.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.903+0000 7f2ad5f9c640 1 -- start start 2023-12-01T01:55:18.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.903+0000 7f2ad5f9c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab8004d00 0x7f2ad015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2ad5f9c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad006ff90 0x7f2ad014ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2ad5f9c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0070410 0x7f2ad014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2ad5f9c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad015a940 con 0x7f2ad006ff90 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2ad5f9c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00712b0 con 0x7f2ab8004d00 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2ad5f9c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad015acf0 con 0x7f2ad0070410 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2aceffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad006ff90 0x7f2ad014ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2aceffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad006ff90 0x7f2ad014ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32998/0 (socket says 172.21.15.115:32998) 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2aceffd640 1 -- 172.21.15.115:0/1804683473 learned_addr learned my addr 172.21.15.115:0/1804683473 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2aceffd640 1 -- 172.21.15.115:0/1804683473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0070410 msgr2=0x7f2ad014f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2aceffd640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0070410 0x7f2ad014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2aceffd640 1 -- 172.21.15.115:0/1804683473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab8004d00 msgr2=0x7f2ad015bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2aceffd640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab8004d00 0x7f2ad015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2aceffd640 1 -- 172.21.15.115:0/1804683473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad015a610 con 0x7f2ad006ff90 2023-12-01T01:55:18.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.904+0000 7f2aceffd640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad006ff90 0x7f2ad014ee20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400f570 tx=0x7f2ac400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.905+0000 7f2ab7fff640 1 -- 172.21.15.115:0/1804683473 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4014890 con 0x7f2ad006ff90 2023-12-01T01:55:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.905+0000 7f2ab7fff640 1 -- 172.21.15.115:0/1804683473 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac400c030 con 0x7f2ad006ff90 2023-12-01T01:55:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.905+0000 7f2ab7fff640 1 -- 172.21.15.115:0/1804683473 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4014c30 con 0x7f2ad006ff90 2023-12-01T01:55:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.905+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1804683473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad015af90 con 0x7f2ad006ff90 2023-12-01T01:55:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.905+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1804683473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad00c5950 con 0x7f2ad006ff90 2023-12-01T01:55:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.906+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1804683473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad00cb170 con 0x7f2ad006ff90 2023-12-01T01:55:18.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.907+0000 7f2ab7fff640 1 -- 172.21.15.115:0/1804683473 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f2ac40133e0 con 0x7f2ad006ff90 2023-12-01T01:55:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.908+0000 7f2ab7fff640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2abc06a550 0x7f2abc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.908+0000 7f2ab7fff640 1 -- 172.21.15.115:0/1804683473 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f2ac4088ec0 con 0x7f2ad006ff90 2023-12-01T01:55:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.909+0000 7f2acf7fe640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2abc06a550 0x7f2abc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:18.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.909+0000 7f2acf7fe640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2abc06a550 0x7f2abc06ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2ad01481f0 tx=0x7f2ac0008040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:18.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:18.913+0000 7f2ab7fff640 1 -- 172.21.15.115:0/1804683473 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2ac4054370 con 0x7f2ad006ff90 2023-12-01T01:55:19.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.141+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1804683473 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ad004e890 con 0x7f2ad006ff90 2023-12-01T01:55:19.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.142+0000 7f2ab7fff640 1 -- 172.21.15.115:0/1804683473 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f2ac4022de0 con 0x7f2ad006ff90 2023-12-01T01:55:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:19.144 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/4122088547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":4122088547},{"type":"v1","addr":"172.21.15.142:6834","nonce":4122088547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6085,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/299645728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":299645728},{"type":"v1","addr":"172.21.15.142:6835","nonce":299645728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7983,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/150770274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":150770274},{"type":"v1","addr":"172.21.15.115:6836","nonce":150770274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":8397,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/4159972145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":4159972145},{"type":"v1","addr":"172.21.15.115:6837","nonce":4159972145}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[]} 2023-12-01T01:55:19.145 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 58 2023-12-01T01:55:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1804683473 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2abc06a550 msgr2=0x7f2abc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2abc06a550 0x7f2abc06ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2ad01481f0 tx=0x7f2ac0008040 comp rx=0 tx=0).stop 2023-12-01T01:55:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1804683473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad006ff90 msgr2=0x7f2ad014ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad006ff90 0x7f2ad014ee20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400f570 tx=0x7f2ac400fa30 comp rx=0 tx=0).stop 2023-12-01T01:55:19.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1804683473 shutdown_connections 2023-12-01T01:55:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2abc06a550 0x7f2abc06ca10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ad0070410 0x7f2ad014f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ad006ff90 0x7f2ad014ee20 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 --2- 172.21.15.115:0/1804683473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab8004d00 0x7f2ad015bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1804683473 >> 172.21.15.115:0/1804683473 conn(0x7f2ad00ce880 msgr2=0x7f2ad00ca2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:19.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1804683473 shutdown_connections 2023-12-01T01:55:19.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.145+0000 7f2ad5f9c640 1 -- 172.21.15.115:0/1804683473 wait complete. 2023-12-01T01:55:19.167 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T01:55:19.167 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T01:55:19.167 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T01:55:19.167 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:55:19.168 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T01:55:19.168 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T01:55:19.168 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T01:55:19.168 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:55:19.168 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T01:55:19.169 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T01:55:19.169 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T01:55:19.169 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:55:19.169 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T01:55:19.169 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T01:55:19.169 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T01:55:19.169 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T01:55:19.184 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T01:55:19.184 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T01:55:19.186 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T01:55:19.186 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T01:55:19.188 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T01:55:19.188 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T01:55:19.193 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T01:55:19.193 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T01:55:19.234 INFO:tasks.ceph.mds.a:Started 2023-12-01T01:55:19.234 DEBUG:teuthology.parallel:result is None 2023-12-01T01:55:19.241 INFO:tasks.ceph.mds.d:Started 2023-12-01T01:55:19.242 DEBUG:teuthology.parallel:result is None 2023-12-01T01:55:19.242 INFO:tasks.ceph.mds.c:Started 2023-12-01T01:55:19.242 DEBUG:teuthology.parallel:result is None 2023-12-01T01:55:19.249 INFO:tasks.ceph.mds.b:Started 2023-12-01T01:55:19.249 DEBUG:teuthology.parallel:result is None 2023-12-01T01:55:19.250 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T01:55:19.390 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T01:55:19.397 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T01:55:19.409 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T01:55:19.419 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T01:55:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.440+0000 7fde629ea640 1 Processor -- start 2023-12-01T01:55:19.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.441+0000 7fde629ea640 1 -- start start 2023-12-01T01:55:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde629ea640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fde5c15cd20 0x7fde5c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde629ea640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fde5c0b5d80 0x7fde5c0b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde629ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5c0d1230 0x7fde5c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde629ea640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5c0b2160 con 0x7fde5c15cd20 2023-12-01T01:55:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde629ea640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5c0d2a10 con 0x7fde5c0b5d80 2023-12-01T01:55:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde629ea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0d1b90 con 0x7fde5c0d1230 2023-12-01T01:55:19.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde5bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fde5c15cd20 0x7fde5c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde5bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fde5c15cd20 0x7fde5c0d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33022/0 (socket says 172.21.15.115:33022) 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde5bfff640 1 -- 172.21.15.115:0/3204840203 learned_addr learned my addr 172.21.15.115:0/3204840203 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde60f87640 1 --2- 172.21.15.115:0/3204840203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5c0d1230 0x7fde5c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde5b7fe640 1 --2- 172.21.15.115:0/3204840203 >> v2:172.21.15.142:3300/0 conn(0x7fde5c0b5d80 0x7fde5c0b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde5bfff640 1 -- 172.21.15.115:0/3204840203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5c0d1230 msgr2=0x7fde5c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde5bfff640 1 --2- 172.21.15.115:0/3204840203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5c0d1230 0x7fde5c0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde5bfff640 1 -- 172.21.15.115:0/3204840203 >> v2:172.21.15.142:3300/0 conn(0x7fde5c0b5d80 msgr2=0x7fde5c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.442+0000 7fde5bfff640 1 --2- 172.21.15.115:0/3204840203 >> v2:172.21.15.142:3300/0 conn(0x7fde5c0b5d80 0x7fde5c0b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.443+0000 7fde5bfff640 1 -- 172.21.15.115:0/3204840203 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0ce2e0 con 0x7fde5c15cd20 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde5bfff640 1 --2- 172.21.15.115:0/3204840203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 0x7fde5c0d24d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0cf630 tx=0x7fde4c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829320e6885fc79c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde5affd640 1 -- 172.21.15.115:0/3204840203 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0317e0 con 0x7fde5c15cd20 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde5affd640 1 -- 172.21.15.115:0/3204840203 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4c031940 con 0x7fde5c15cd20 2023-12-01T01:55:19.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde5affd640 1 -- 172.21.15.115:0/3204840203 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c031cd0 con 0x7fde5c15cd20 2023-12-01T01:55:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde629ea640 1 -- 172.21.15.115:0/3204840203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 msgr2=0x7fde5c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:19.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde629ea640 1 --2- 172.21.15.115:0/3204840203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 0x7fde5c0d24d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0cf630 tx=0x7fde4c02b880 comp rx=0 tx=0).stop 2023-12-01T01:55:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde629ea640 1 -- 172.21.15.115:0/3204840203 shutdown_connections 2023-12-01T01:55:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde629ea640 1 --2- 172.21.15.115:0/3204840203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5c0d1230 0x7fde5c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde629ea640 1 --2- 172.21.15.115:0/3204840203 >> v2:172.21.15.142:3300/0 conn(0x7fde5c0b5d80 0x7fde5c0b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde629ea640 1 --2- 172.21.15.115:0/3204840203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 0x7fde5c0d24d0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.444+0000 7fde629ea640 1 -- 172.21.15.115:0/3204840203 >> 172.21.15.115:0/3204840203 conn(0x7fde5c06a0e0 msgr2=0x7fde5c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.445+0000 7fde629ea640 1 -- 172.21.15.115:0/3204840203 shutdown_connections 2023-12-01T01:55:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.445+0000 7fde629ea640 1 -- 172.21.15.115:0/3204840203 wait complete. 2023-12-01T01:55:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.446+0000 7fde629ea640 1 Processor -- start 2023-12-01T01:55:19.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.446+0000 7fde629ea640 1 -- start start 2023-12-01T01:55:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.447+0000 7fde629ea640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde5c0b5d80 0x7fde5c1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.447+0000 7fde629ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5c0d1230 0x7fde5c154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:19.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.447+0000 7fde629ea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 0x7fde5c1f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.447+0000 7fde629ea640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c15eaf0 con 0x7fde5c15cd20 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.447+0000 7fde629ea640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c156c50 con 0x7fde5c0b5d80 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.447+0000 7fde629ea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c15ed30 con 0x7fde5c0d1230 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde60f87640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 0x7fde5c1f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde60f87640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 0x7fde5c1f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33036/0 (socket says 172.21.15.115:33036) 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde60f87640 1 -- 172.21.15.115:0/661976963 learned_addr learned my addr 172.21.15.115:0/661976963 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde5b7fe640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5c0d1230 0x7fde5c154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde5bfff640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde5c0b5d80 0x7fde5c1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde60f87640 1 -- 172.21.15.115:0/661976963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5c0d1230 msgr2=0x7fde5c154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde60f87640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5c0d1230 0x7fde5c154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde60f87640 1 -- 172.21.15.115:0/661976963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde5c0b5d80 msgr2=0x7fde5c1533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde60f87640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde5c0b5d80 0x7fde5c1533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde60f87640 1 -- 172.21.15.115:0/661976963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c0094d0 con 0x7fde5c15cd20 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde5bfff640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde5c0b5d80 0x7fde5c1533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:55:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.448+0000 7fde60f87640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 0x7fde5c1f2660 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fde54007e40 tx=0x7fde5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.449+0000 7fde58ff9640 1 -- 172.21.15.115:0/661976963 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54014620 con 0x7fde5c15cd20 2023-12-01T01:55:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.449+0000 7fde58ff9640 1 -- 172.21.15.115:0/661976963 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5400ee90 con 0x7fde5c15cd20 2023-12-01T01:55:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.449+0000 7fde58ff9640 1 -- 172.21.15.115:0/661976963 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540149e0 con 0x7fde5c15cd20 2023-12-01T01:55:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.449+0000 7fde629ea640 1 -- 172.21.15.115:0/661976963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c15f010 con 0x7fde5c15cd20 2023-12-01T01:55:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.449+0000 7fde629ea640 1 -- 172.21.15.115:0/661976963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c154e70 con 0x7fde5c15cd20 2023-12-01T01:55:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.450+0000 7fde629ea640 1 -- 172.21.15.115:0/661976963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5c155030 con 0x7fde5c15cd20 2023-12-01T01:55:19.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.451+0000 7fde58ff9640 1 -- 172.21.15.115:0/661976963 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fde54014b40 con 0x7fde5c15cd20 2023-12-01T01:55:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.452+0000 7fde58ff9640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde4806a550 0x7fde4806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.452+0000 7fde58ff9640 1 -- 172.21.15.115:0/661976963 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fde54089370 con 0x7fde5c15cd20 2023-12-01T01:55:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.452+0000 7fde5bfff640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde4806a550 0x7fde4806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.453+0000 7fde5bfff640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde4806a550 0x7fde4806ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fde4c036000 tx=0x7fde4c03d040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.456+0000 7fde58ff9640 1 -- 172.21.15.115:0/661976963 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fde54054800 con 0x7fde5c15cd20 2023-12-01T01:55:19.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:19.646+0000 7fde629ea640 1 -- 172.21.15.115:0/661976963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fde5c15f530 con 0x7fde5c15cd20 2023-12-01T01:55:20.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.445+0000 7fde58ff9640 1 -- 172.21.15.115:0/661976963 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v137) v1 ==== 97+0+0 (secure 0 0 0) 0x7fde54057e20 con 0x7fde5c15cd20 2023-12-01T01:55:20.447 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T01:55:20.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 -- 172.21.15.115:0/661976963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde4806a550 msgr2=0x7fde4806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde4806a550 0x7fde4806ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fde4c036000 tx=0x7fde4c03d040 comp rx=0 tx=0).stop 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 -- 172.21.15.115:0/661976963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 msgr2=0x7fde5c1f2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 0x7fde5c1f2660 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fde54007e40 tx=0x7fde5400e4b0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 -- 172.21.15.115:0/661976963 shutdown_connections 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde4806a550 0x7fde4806ca10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5c0d1230 0x7fde5c154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde5c15cd20 0x7fde5c1f2660 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 --2- 172.21.15.115:0/661976963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde5c0b5d80 0x7fde5c1533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 -- 172.21.15.115:0/661976963 >> 172.21.15.115:0/661976963 conn(0x7fde5c06a0e0 msgr2=0x7fde5c0d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 -- 172.21.15.115:0/661976963 shutdown_connections 2023-12-01T01:55:20.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.449+0000 7fde629ea640 1 -- 172.21.15.115:0/661976963 wait complete. 2023-12-01T01:55:20.471 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T01:55:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.691+0000 7eff6be0b640 1 Processor -- start 2023-12-01T01:55:20.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.692+0000 7eff6be0b640 1 -- start start 2023-12-01T01:55:20.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.693+0000 7eff6be0b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7eff6415cd20 0x7eff640d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:20.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.693+0000 7eff6be0b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff640b5d80 0x7eff640b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:20.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.693+0000 7eff6be0b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7eff640d1230 0x7eff640d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:20.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.693+0000 7eff6be0b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff640b2160 con 0x7eff6415cd20 2023-12-01T01:55:20.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.693+0000 7eff6be0b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff640d1be0 con 0x7eff640d1230 2023-12-01T01:55:20.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.693+0000 7eff6be0b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff640d2a10 con 0x7eff640b5d80 2023-12-01T01:55:20.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.693+0000 7eff69ba7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7eff6415cd20 0x7eff640d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:20.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.693+0000 7eff69ba7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7eff6415cd20 0x7eff640d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33042/0 (socket says 172.21.15.115:33042) 2023-12-01T01:55:20.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.693+0000 7eff69ba7640 1 -- 172.21.15.115:0/2130945031 learned_addr learned my addr 172.21.15.115:0/2130945031 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:20.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.694+0000 7eff69ba7640 1 -- 172.21.15.115:0/2130945031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff640b5d80 msgr2=0x7eff640b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:20.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.694+0000 7eff69ba7640 1 --2- 172.21.15.115:0/2130945031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff640b5d80 0x7eff640b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.694+0000 7eff69ba7640 1 -- 172.21.15.115:0/2130945031 >> v2:172.21.15.142:3300/0 conn(0x7eff640d1230 msgr2=0x7eff640d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:20.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.694+0000 7eff69ba7640 1 --2- 172.21.15.115:0/2130945031 >> v2:172.21.15.142:3300/0 conn(0x7eff640d1230 0x7eff640d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.694+0000 7eff69ba7640 1 -- 172.21.15.115:0/2130945031 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff640ce2e0 con 0x7eff6415cd20 2023-12-01T01:55:20.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.695+0000 7eff69ba7640 1 --2- 172.21.15.115:0/2130945031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff6415cd20 0x7eff640d24d0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7eff640b5b50 tx=0x7eff5402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af227fbb476e6bb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:20.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.695+0000 7eff68ba5640 1 -- 172.21.15.115:0/2130945031 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5402c9f0 con 0x7eff6415cd20 2023-12-01T01:55:20.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.695+0000 7eff68ba5640 1 -- 172.21.15.115:0/2130945031 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5402cb50 con 0x7eff6415cd20 2023-12-01T01:55:20.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.695+0000 7eff68ba5640 1 -- 172.21.15.115:0/2130945031 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff540343f0 con 0x7eff6415cd20 2023-12-01T01:55:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.696+0000 7eff6be0b640 1 -- 172.21.15.115:0/2130945031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff6415cd20 msgr2=0x7eff640d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.696+0000 7eff6be0b640 1 --2- 172.21.15.115:0/2130945031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff6415cd20 0x7eff640d24d0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7eff640b5b50 tx=0x7eff5402be50 comp rx=0 tx=0).stop 2023-12-01T01:55:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.696+0000 7eff6be0b640 1 -- 172.21.15.115:0/2130945031 shutdown_connections 2023-12-01T01:55:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.696+0000 7eff6be0b640 1 --2- 172.21.15.115:0/2130945031 >> v2:172.21.15.142:3300/0 conn(0x7eff640d1230 0x7eff640d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.696+0000 7eff6be0b640 1 --2- 172.21.15.115:0/2130945031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff640b5d80 0x7eff640b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.696+0000 7eff6be0b640 1 --2- 172.21.15.115:0/2130945031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff6415cd20 0x7eff640d24d0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.696+0000 7eff6be0b640 1 -- 172.21.15.115:0/2130945031 >> 172.21.15.115:0/2130945031 conn(0x7eff6406a0e0 msgr2=0x7eff6406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.696+0000 7eff6be0b640 1 -- 172.21.15.115:0/2130945031 shutdown_connections 2023-12-01T01:55:20.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.696+0000 7eff6be0b640 1 -- 172.21.15.115:0/2130945031 wait complete. 2023-12-01T01:55:20.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.698+0000 7eff6be0b640 1 Processor -- start 2023-12-01T01:55:20.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.698+0000 7eff6be0b640 1 -- start start 2023-12-01T01:55:20.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff6be0b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff640b5d80 0x7eff6415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:20.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff6be0b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff640d1230 0x7eff640c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:20.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff6be0b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff6415cd20 0x7eff6415d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff6be0b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6415bb20 con 0x7eff640d1230 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff6be0b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6415aea0 con 0x7eff640b5d80 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff6be0b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6415b180 con 0x7eff6415cd20 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff693a6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff640d1230 0x7eff640c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff693a6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff640d1230 0x7eff640c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33056/0 (socket says 172.21.15.115:33056) 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff693a6640 1 -- 172.21.15.115:0/418819740 learned_addr learned my addr 172.21.15.115:0/418819740 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff693a6640 1 -- 172.21.15.115:0/418819740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff6415cd20 msgr2=0x7eff6415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff693a6640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff6415cd20 0x7eff6415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff693a6640 1 -- 172.21.15.115:0/418819740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff640b5d80 msgr2=0x7eff6415bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff69ba7640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff640b5d80 0x7eff6415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff693a6640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff640b5d80 0x7eff6415bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.699+0000 7eff693a6640 1 -- 172.21.15.115:0/418819740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff54009b30 con 0x7eff640d1230 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.700+0000 7eff693a6640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff640d1230 0x7eff640c5e90 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7eff5800b450 tx=0x7eff5800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.700+0000 7eff627fc640 1 -- 172.21.15.115:0/418819740 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58011890 con 0x7eff640d1230 2023-12-01T01:55:20.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.700+0000 7eff627fc640 1 -- 172.21.15.115:0/418819740 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff580119f0 con 0x7eff640d1230 2023-12-01T01:55:20.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.700+0000 7eff6be0b640 1 -- 172.21.15.115:0/418819740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6415b460 con 0x7eff640d1230 2023-12-01T01:55:20.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.700+0000 7eff627fc640 1 -- 172.21.15.115:0/418819740 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58011da0 con 0x7eff640d1230 2023-12-01T01:55:20.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.700+0000 7eff6be0b640 1 -- 172.21.15.115:0/418819740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6415c480 con 0x7eff640d1230 2023-12-01T01:55:20.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.701+0000 7eff6be0b640 1 -- 172.21.15.115:0/418819740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6404e890 con 0x7eff640d1230 2023-12-01T01:55:20.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.702+0000 7eff627fc640 1 -- 172.21.15.115:0/418819740 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7eff5801a400 con 0x7eff640d1230 2023-12-01T01:55:20.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.703+0000 7eff627fc640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff5006a550 0x7eff5006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:20.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.703+0000 7eff627fc640 1 -- 172.21.15.115:0/418819740 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4857+0+0 (secure 0 0 0) 0x7eff58088c10 con 0x7eff640d1230 2023-12-01T01:55:20.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.703+0000 7eff69ba7640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff5006a550 0x7eff5006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:20.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.704+0000 7eff69ba7640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff5006a550 0x7eff5006ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7eff54035000 tx=0x7eff54030000 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:20.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.713+0000 7eff627fc640 1 -- 172.21.15.115:0/418819740 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7eff580543a0 con 0x7eff640d1230 2023-12-01T01:55:20.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.973+0000 7eff6be0b640 1 -- 172.21.15.115:0/418819740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7eff64067700 con 0x7eff640d1230 2023-12-01T01:55:20.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.974+0000 7eff627fc640 1 -- 172.21.15.115:0/418819740 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7eff58015090 con 0x7eff640d1230 2023-12-01T01:55:20.978 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:20.978 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T01:55:20.979 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T01:55:20.980 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.980 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:55:20.980 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:55:20.980 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:55:20.980 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.980 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T01:55:20.980 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.981 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:55:20.982 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:55:20.983 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.983 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T01:55:20.983 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T01:55:20.983 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.983 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:55:20.983 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:55:20.984 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.984 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T01:55:20.984 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T01:55:20.984 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.984 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:55:20.984 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:55:20.985 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.986 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T01:55:20.986 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T01:55:20.986 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.986 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:55:20.986 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:55:20.986 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.986 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.987 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T01:55:20.987 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T01:55:20.987 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.987 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:55:20.987 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:55:20.987 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.988 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T01:55:20.988 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T01:55:20.988 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.988 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:55:20.988 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:55:20.988 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T01:55:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T01:55:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:55:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:55:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.990 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T01:55:20.990 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T01:55:20.990 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.990 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T01:55:20.991 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T01:55:20.992 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T01:55:20.993 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.994 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:55:20.995 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T01:55:20.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:20.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T01:55:20.997 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:55:20.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 -- 172.21.15.115:0/418819740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff5006a550 msgr2=0x7eff5006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:20.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff5006a550 0x7eff5006ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7eff54035000 tx=0x7eff54030000 comp rx=0 tx=0).stop 2023-12-01T01:55:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 -- 172.21.15.115:0/418819740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff640d1230 msgr2=0x7eff640c5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff640d1230 0x7eff640c5e90 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7eff5800b450 tx=0x7eff5800b910 comp rx=0 tx=0).stop 2023-12-01T01:55:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 -- 172.21.15.115:0/418819740 shutdown_connections 2023-12-01T01:55:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff5006a550 0x7eff5006ca10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff6415cd20 0x7eff6415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff640d1230 0x7eff640c5e90 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 --2- 172.21.15.115:0/418819740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff640b5d80 0x7eff6415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 -- 172.21.15.115:0/418819740 >> 172.21.15.115:0/418819740 conn(0x7eff6406a0e0 msgr2=0x7eff640b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 -- 172.21.15.115:0/418819740 shutdown_connections 2023-12-01T01:55:20.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:20.977+0000 7eff6be0b640 1 -- 172.21.15.115:0/418819740 wait complete. 2023-12-01T01:55:21.000 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:55:21.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.215+0000 7f9944b77640 1 Processor -- start 2023-12-01T01:55:21.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.215+0000 7f9944b77640 1 -- start start 2023-12-01T01:55:21.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.216+0000 7f9944b77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940145620 0x7f99400ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.216+0000 7f9944b77640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f99400cdd80 0x7f994006b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.216+0000 7f9944b77640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99400ad7e0 con 0x7f994006b7b0 2023-12-01T01:55:21.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.217+0000 7f9944b77640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f994006bc20 con 0x7f99400cdd80 2023-12-01T01:55:21.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.217+0000 7f9944b77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99400cc8e0 con 0x7f9940145620 2023-12-01T01:55:21.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.217+0000 7f993ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f994006b7b0 0x7f9940160750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54686/0 (socket says 172.21.15.115:54686) 2023-12-01T01:55:21.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.217+0000 7f993ffff640 1 -- 172.21.15.115:0/956344930 learned_addr learned my addr 172.21.15.115:0/956344930 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:21.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.217+0000 7f9937fff640 1 --2- 172.21.15.115:0/956344930 >> v2:172.21.15.142:3300/0 conn(0x7f99400cdd80 0x7f994006b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.217+0000 7f993f7fe640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940145620 0x7f99400ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.217+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548402699 0 0) 0x7f99400ad7e0 con 0x7f994006b7b0 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.217+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f992c003520 con 0x7f994006b7b0 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2558238275 0 0) 0x7f992c003520 con 0x7f994006b7b0 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940145620 msgr2=0x7f99400ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940145620 0x7f99400ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 >> v2:172.21.15.142:3300/0 conn(0x7f99400cdd80 msgr2=0x7f994006b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 --2- 172.21.15.115:0/956344930 >> v2:172.21.15.142:3300/0 conn(0x7f99400cdd80 0x7f994006b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99400ce260 con 0x7f994006b7b0 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f99380019f0 con 0x7f994006b7b0 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 >> v1:172.21.15.115:6789/0 conn(0x7f994006b7b0 legacy=0x7f9940160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f992c0039e0 0x7f992c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f992c0047c0 0x7f992c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940145620 0x7f992c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994006bc20 con 0x7f992c0039e0 2023-12-01T01:55:21.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400cc8e0 con 0x7f992c0047c0 2023-12-01T01:55:21.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c00b980 con 0x7f9940145620 2023-12-01T01:55:21.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.218+0000 7f993f7fe640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f992c0039e0 0x7f992c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.219+0000 7f993f7fe640 1 -- 172.21.15.115:0/956344930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940145620 msgr2=0x7f992c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:21.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.219+0000 7f993f7fe640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940145620 0x7f992c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.219+0000 7f9937fff640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f992c0047c0 0x7f992c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.219+0000 7f993f7fe640 1 -- 172.21.15.115:0/956344930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f992c0047c0 msgr2=0x7f992c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:21.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.219+0000 7f993f7fe640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f992c0047c0 0x7f992c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.219+0000 7f993f7fe640 1 -- 172.21.15.115:0/956344930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c003520 con 0x7f992c0039e0 2023-12-01T01:55:21.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.220+0000 7f993f7fe640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f992c0039e0 0x7f992c003e00 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9930009af0 tx=0x7f992c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1809550d3938d8b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:21.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.220+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993002c710 con 0x7f992c0039e0 2023-12-01T01:55:21.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.220+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993002c870 con 0x7f992c0039e0 2023-12-01T01:55:21.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.220+0000 7f993effd640 1 -- 172.21.15.115:0/956344930 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993002cc00 con 0x7f992c0039e0 2023-12-01T01:55:21.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.220+0000 7f9944b77640 1 -- 172.21.15.115:0/956344930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f992c0039e0 msgr2=0x7f992c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.220+0000 7f9944b77640 1 --2- 172.21.15.115:0/956344930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f992c0039e0 0x7f992c003e00 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9930009af0 tx=0x7f992c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.221+0000 7f993ffff640 1 -- 172.21.15.115:0/956344930 reap_dead start 2023-12-01T01:55:21.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.221+0000 7f9944b77640 1 -- 172.21.15.115:0/956344930 shutdown_connections 2023-12-01T01:55:21.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.221+0000 7f9944b77640 1 -- 172.21.15.115:0/956344930 >> 172.21.15.115:0/956344930 conn(0x7f99400ce640 msgr2=0x7f99400cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:21.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.221+0000 7f9944b77640 1 -- 172.21.15.115:0/956344930 shutdown_connections 2023-12-01T01:55:21.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.221+0000 7f9944b77640 1 -- 172.21.15.115:0/956344930 wait complete. 2023-12-01T01:55:21.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.222+0000 7f9944b77640 1 Processor -- start 2023-12-01T01:55:21.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.223+0000 7f9944b77640 1 -- start start 2023-12-01T01:55:21.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.223+0000 7f9944b77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f992c0039e0 0x7f99401f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.223+0000 7f9944b77640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99401413d0 0x7f994015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.223+0000 7f9944b77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940141850 0x7f994015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9944b77640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994015c520 con 0x7f99401413d0 2023-12-01T01:55:21.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9944b77640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99401f69a0 con 0x7f992c0039e0 2023-12-01T01:55:21.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9944b77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99401f33d0 con 0x7f9940141850 2023-12-01T01:55:21.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9937fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99401413d0 0x7f994015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9937fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99401413d0 0x7f994015baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33066/0 (socket says 172.21.15.115:33066) 2023-12-01T01:55:21.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9937fff640 1 -- 172.21.15.115:0/347438371 learned_addr learned my addr 172.21.15.115:0/347438371 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:21.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f993f7fe640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f992c0039e0 0x7f99401f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9937fff640 1 -- 172.21.15.115:0/347438371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940141850 msgr2=0x7f994015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9937fff640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940141850 0x7f994015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9937fff640 1 -- 172.21.15.115:0/347438371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f992c0039e0 msgr2=0x7f99401f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9937fff640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f992c0039e0 0x7f99401f6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.224+0000 7f9937fff640 1 -- 172.21.15.115:0/347438371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99401f2850 con 0x7f99401413d0 2023-12-01T01:55:21.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.225+0000 7f9937fff640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99401413d0 0x7f994015baa0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f99280108e0 tx=0x7f9928010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:21.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.225+0000 7f993cff9640 1 -- 172.21.15.115:0/347438371 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992800eb60 con 0x7f99401413d0 2023-12-01T01:55:21.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.225+0000 7f993cff9640 1 -- 172.21.15.115:0/347438371 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992800ecc0 con 0x7f99401413d0 2023-12-01T01:55:21.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.225+0000 7f9944b77640 1 -- 172.21.15.115:0/347438371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99401f2b30 con 0x7f99401413d0 2023-12-01T01:55:21.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.225+0000 7f993cff9640 1 -- 172.21.15.115:0/347438371 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992801c400 con 0x7f99401413d0 2023-12-01T01:55:21.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.225+0000 7f9944b77640 1 -- 172.21.15.115:0/347438371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99401f30f0 con 0x7f99401413d0 2023-12-01T01:55:21.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.226+0000 7f9944b77640 1 -- 172.21.15.115:0/347438371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99400673a0 con 0x7f99401413d0 2023-12-01T01:55:21.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.227+0000 7f993cff9640 1 -- 172.21.15.115:0/347438371 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f992801c560 con 0x7f99401413d0 2023-12-01T01:55:21.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.228+0000 7f993cff9640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f992006a550 0x7f992006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.228+0000 7f993cff9640 1 -- 172.21.15.115:0/347438371 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9928088e00 con 0x7f99401413d0 2023-12-01T01:55:21.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.228+0000 7f993f7fe640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f992006a550 0x7f992006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.229+0000 7f993f7fe640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f992006a550 0x7f992006ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9930036000 tx=0x7f9930030000 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:21.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.232+0000 7f993cff9640 1 -- 172.21.15.115:0/347438371 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9928054640 con 0x7f99401413d0 2023-12-01T01:55:21.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.420+0000 7f9944b77640 1 -- 172.21.15.115:0/347438371 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99400c3cd0 con 0x7f99401413d0 2023-12-01T01:55:21.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.421+0000 7f993cff9640 1 -- 172.21.15.115:0/347438371 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+12447 (secure 0 0 0) 0x7f9928057c60 con 0x7f99401413d0 2023-12-01T01:55:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:21.427 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":137,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:55:20.436306+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:55:21.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.424+0000 7f9944b77640 1 -- 172.21.15.115:0/347438371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f992006a550 msgr2=0x7f992006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.424+0000 7f9944b77640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f992006a550 0x7f992006ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9930036000 tx=0x7f9930030000 comp rx=0 tx=0).stop 2023-12-01T01:55:21.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.424+0000 7f9944b77640 1 -- 172.21.15.115:0/347438371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99401413d0 msgr2=0x7f994015baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.424+0000 7f9944b77640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99401413d0 0x7f994015baa0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f99280108e0 tx=0x7f9928010da0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.424+0000 7f9944b77640 1 -- 172.21.15.115:0/347438371 shutdown_connections 2023-12-01T01:55:21.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.424+0000 7f9944b77640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f992006a550 0x7f992006ca10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.424+0000 7f9944b77640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9940141850 0x7f994015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.424+0000 7f9944b77640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99401413d0 0x7f994015baa0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.424+0000 7f9944b77640 1 --2- 172.21.15.115:0/347438371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f992c0039e0 0x7f99401f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.424+0000 7f9944b77640 1 -- 172.21.15.115:0/347438371 >> 172.21.15.115:0/347438371 conn(0x7f99400ce640 msgr2=0x7f9940140150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:21.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.425+0000 7f9944b77640 1 -- 172.21.15.115:0/347438371 shutdown_connections 2023-12-01T01:55:21.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.425+0000 7f9944b77640 1 -- 172.21.15.115:0/347438371 wait complete. 2023-12-01T01:55:21.445 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:55:21.445 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:55:21.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.656+0000 7f796b300640 1 Processor -- start 2023-12-01T01:55:21.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.656+0000 7f796b300640 1 -- start start 2023-12-01T01:55:21.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.658+0000 7f796b300640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f796415ac80 0x7f79640cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.658+0000 7f796b300640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79640cc9a0 con 0x7f79640b0e60 2023-12-01T01:55:21.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.658+0000 7f796b300640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79640cbd90 con 0x7f79640ce1b0 2023-12-01T01:55:21.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.658+0000 7f796b300640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796415b060 con 0x7f796415ac80 2023-12-01T01:55:21.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.658+0000 7f796909c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f796415ac80 0x7f79640cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.658+0000 7f796889b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f79640b0e60 0x7f79640cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54690/0 (socket says 172.21.15.115:54690) 2023-12-01T01:55:21.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.658+0000 7f796889b640 1 -- 172.21.15.115:0/1064213614 learned_addr learned my addr 172.21.15.115:0/1064213614 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:21.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.659+0000 7f796909c640 1 -- 172.21.15.115:0/1064213614 >> v1:172.21.15.142:6789/0 conn(0x7f79640ce1b0 legacy=0x7f796406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:21.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.659+0000 7f796909c640 1 -- 172.21.15.115:0/1064213614 >> v1:172.21.15.115:6789/0 conn(0x7f79640b0e60 legacy=0x7f79640cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.659+0000 7f795bfff640 1 -- 172.21.15.115:0/1064213614 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567409772 0 0) 0x7f79640cc9a0 con 0x7f79640b0e60 2023-12-01T01:55:21.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.659+0000 7f796909c640 1 -- 172.21.15.115:0/1064213614 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79640cc070 con 0x7f796415ac80 2023-12-01T01:55:21.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.661+0000 7f796909c640 1 --2- 172.21.15.115:0/1064213614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 0x7f79640cce30 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f79600097a0 tx=0x7f796002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cf30243d8a154b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:21.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.661+0000 7f795bfff640 1 -- 172.21.15.115:0/1064213614 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960030610 con 0x7f796415ac80 2023-12-01T01:55:21.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.661+0000 7f795bfff640 1 -- 172.21.15.115:0/1064213614 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7960030770 con 0x7f796415ac80 2023-12-01T01:55:21.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.661+0000 7f795bfff640 1 -- 172.21.15.115:0/1064213614 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960030b00 con 0x7f796415ac80 2023-12-01T01:55:21.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.662+0000 7f796b300640 1 -- 172.21.15.115:0/1064213614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 msgr2=0x7f79640cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.662+0000 7f796b300640 1 --2- 172.21.15.115:0/1064213614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 0x7f79640cce30 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f79600097a0 tx=0x7f796002b960 comp rx=0 tx=0).stop 2023-12-01T01:55:21.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.662+0000 7f796b300640 1 -- 172.21.15.115:0/1064213614 shutdown_connections 2023-12-01T01:55:21.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.662+0000 7f796b300640 1 --2- 172.21.15.115:0/1064213614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 0x7f79640cce30 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.662+0000 7f796b300640 1 -- 172.21.15.115:0/1064213614 >> 172.21.15.115:0/1064213614 conn(0x7f7964065960 msgr2=0x7f7964065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:21.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.662+0000 7f796b300640 1 -- 172.21.15.115:0/1064213614 shutdown_connections 2023-12-01T01:55:21.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.662+0000 7f796b300640 1 -- 172.21.15.115:0/1064213614 wait complete. 2023-12-01T01:55:21.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.664+0000 7f796b300640 1 Processor -- start 2023-12-01T01:55:21.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.665+0000 7f796b300640 1 -- start start 2023-12-01T01:55:21.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796b300640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79640b0e60 0x7f7964155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796b300640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79640ce1b0 0x7f7964158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796b300640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 0x7f79641590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796b300640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79640c59c0 con 0x7f796415ac80 2023-12-01T01:55:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796b300640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79640c4d80 con 0x7f79640ce1b0 2023-12-01T01:55:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796b300640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79640c5020 con 0x7f79640b0e60 2023-12-01T01:55:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796989d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 0x7f79641590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796989d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 0x7f79641590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33088/0 (socket says 172.21.15.115:33088) 2023-12-01T01:55:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796989d640 1 -- 172.21.15.115:0/1863890268 learned_addr learned my addr 172.21.15.115:0/1863890268 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796909c640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79640b0e60 0x7f7964155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.666+0000 7f796889b640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79640ce1b0 0x7f7964158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f796989d640 1 -- 172.21.15.115:0/1863890268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79640b0e60 msgr2=0x7f7964155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f796989d640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79640b0e60 0x7f7964155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f796989d640 1 -- 172.21.15.115:0/1863890268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79640ce1b0 msgr2=0x7f7964158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f796989d640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79640ce1b0 0x7f7964158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f796989d640 1 -- 172.21.15.115:0/1863890268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7960009400 con 0x7f796415ac80 2023-12-01T01:55:21.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f796989d640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 0x7f79641590e0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f795c00c730 tx=0x7f795c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:21.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f7959ffb640 1 -- 172.21.15.115:0/1863890268 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c003b70 con 0x7f796415ac80 2023-12-01T01:55:21.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f7959ffb640 1 -- 172.21.15.115:0/1863890268 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c003cd0 con 0x7f796415ac80 2023-12-01T01:55:21.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f7959ffb640 1 -- 172.21.15.115:0/1863890268 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c00aa90 con 0x7f796415ac80 2023-12-01T01:55:21.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f796b300640 1 -- 172.21.15.115:0/1863890268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79640c5300 con 0x7f796415ac80 2023-12-01T01:55:21.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.667+0000 7f796b300640 1 -- 172.21.15.115:0/1863890268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7964155950 con 0x7f796415ac80 2023-12-01T01:55:21.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.669+0000 7f796b300640 1 -- 172.21.15.115:0/1863890268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79640c5b00 con 0x7f796415ac80 2023-12-01T01:55:21.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.669+0000 7f7959ffb640 1 -- 172.21.15.115:0/1863890268 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f795c00abf0 con 0x7f796415ac80 2023-12-01T01:55:21.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.670+0000 7f7959ffb640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f794806a550 0x7f794806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:21.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.670+0000 7f7959ffb640 1 -- 172.21.15.115:0/1863890268 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f795c088560 con 0x7f796415ac80 2023-12-01T01:55:21.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.671+0000 7f796909c640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f794806a550 0x7f794806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:21.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.671+0000 7f796909c640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f794806a550 0x7f794806ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7960036000 tx=0x7f7960008030 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:21.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.675+0000 7f7959ffb640 1 -- 172.21.15.115:0/1863890268 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f79640c5b00 con 0x7f796415ac80 2023-12-01T01:55:21.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.862+0000 7f796b300640 1 -- 172.21.15.115:0/1863890268 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79640c5b00 con 0x7f796415ac80 2023-12-01T01:55:21.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.863+0000 7f7959ffb640 1 -- 172.21.15.115:0/1863890268 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79640c5b00 con 0x7f796415ac80 2023-12-01T01:55:21.864 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T01:55:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 -- 172.21.15.115:0/1863890268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f794806a550 msgr2=0x7f794806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f794806a550 0x7f794806ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7960036000 tx=0x7f7960008030 comp rx=0 tx=0).stop 2023-12-01T01:55:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 -- 172.21.15.115:0/1863890268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 msgr2=0x7f79641590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 0x7f79641590e0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f795c00c730 tx=0x7f795c00cbf0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 -- 172.21.15.115:0/1863890268 shutdown_connections 2023-12-01T01:55:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f794806a550 0x7f794806ca10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f796415ac80 0x7f79641590e0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79640ce1b0 0x7f7964158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 --2- 172.21.15.115:0/1863890268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79640b0e60 0x7f7964155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:21.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 -- 172.21.15.115:0/1863890268 >> 172.21.15.115:0/1863890268 conn(0x7f7964065960 msgr2=0x7f79640ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 -- 172.21.15.115:0/1863890268 shutdown_connections 2023-12-01T01:55:21.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:21.865+0000 7f796b300640 1 -- 172.21.15.115:0/1863890268 wait complete. 2023-12-01T01:55:21.886 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T01:55:22.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.103+0000 7f0a7d370640 1 Processor -- start 2023-12-01T01:55:22.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.103+0000 7f0a7d370640 1 -- start start 2023-12-01T01:55:22.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.104+0000 7f0a7d370640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a78159d90 0x7f0a780c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.104+0000 7f0a7d370640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a780707a0 con 0x7f0a7806b190 2023-12-01T01:55:22.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.104+0000 7f0a7d370640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a780b2f70 con 0x7f0a780710b0 2023-12-01T01:55:22.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.104+0000 7f0a7d370640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7815e2e0 con 0x7f0a78159d90 2023-12-01T01:55:22.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.105+0000 7f0a6ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0a7806b190 0x7f0a780709a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54704/0 (socket says 172.21.15.115:54704) 2023-12-01T01:55:22.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.105+0000 7f0a6ffff640 1 -- 172.21.15.115:0/2415215330 learned_addr learned my addr 172.21.15.115:0/2415215330 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.105+0000 7f0a77fff640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a78159d90 0x7f0a780c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.105+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402219223 0 0) 0x7f0a780707a0 con 0x7f0a7806b190 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.105+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a64005730 con 0x7f0a7806b190 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.105+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1675511538 0 0) 0x7f0a64005730 con 0x7f0a7806b190 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a78159d90 msgr2=0x7f0a780c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a78159d90 0x7f0a780c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 >> v1:172.21.15.142:6789/0 conn(0x7f0a780710b0 legacy=0x7f0a780b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7815e5c0 con 0x7f0a7806b190 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0a60001670 con 0x7f0a7806b190 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 >> v1:172.21.15.115:6789/0 conn(0x7f0a7806b190 legacy=0x7f0a780709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:22.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a64005c20 0x7f0a64006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 0x7f0a64006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a78159d90 0x7f0a64007440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7815e2e0 con 0x7f0a640069d0 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6400dd50 con 0x7f0a64005c20 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6400df10 con 0x7f0a78159d90 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a6ffff640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 0x7f0a64006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a77fff640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a64005c20 0x7f0a64006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a6ffff640 1 -- 172.21.15.115:0/2415215330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a78159d90 msgr2=0x7f0a64007440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a6ffff640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a78159d90 0x7f0a64007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a6ffff640 1 -- 172.21.15.115:0/2415215330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a64005c20 msgr2=0x7f0a64006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a6ffff640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a64005c20 0x7f0a64006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.106+0000 7f0a6ffff640 1 -- 172.21.15.115:0/2415215330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a64005730 con 0x7f0a640069d0 2023-12-01T01:55:22.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.108+0000 7f0a6ffff640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 0x7f0a64006e50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000c210 tx=0x7f0a64007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de518f1dffd32d34 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:22.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.108+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60005ad0 con 0x7f0a640069d0 2023-12-01T01:55:22.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.108+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a60005c30 con 0x7f0a640069d0 2023-12-01T01:55:22.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.108+0000 7f0a777fe640 1 -- 172.21.15.115:0/2415215330 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6002c890 con 0x7f0a640069d0 2023-12-01T01:55:22.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.108+0000 7f0a7d370640 1 -- 172.21.15.115:0/2415215330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 msgr2=0x7f0a64006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:22.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.108+0000 7f0a7d370640 1 --2- 172.21.15.115:0/2415215330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 0x7f0a64006e50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000c210 tx=0x7f0a64007a00 comp rx=0 tx=0).stop 2023-12-01T01:55:22.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.108+0000 7f0a7cb6f640 1 -- 172.21.15.115:0/2415215330 reap_dead start 2023-12-01T01:55:22.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.108+0000 7f0a7d370640 1 -- 172.21.15.115:0/2415215330 shutdown_connections 2023-12-01T01:55:22.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.108+0000 7f0a7d370640 1 -- 172.21.15.115:0/2415215330 >> 172.21.15.115:0/2415215330 conn(0x7f0a780ce620 msgr2=0x7f0a780cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:22.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.109+0000 7f0a7d370640 1 -- 172.21.15.115:0/2415215330 shutdown_connections 2023-12-01T01:55:22.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.109+0000 7f0a7d370640 1 -- 172.21.15.115:0/2415215330 wait complete. 2023-12-01T01:55:22.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.110+0000 7f0a7d370640 1 Processor -- start 2023-12-01T01:55:22.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.110+0000 7f0a7d370640 1 -- start start 2023-12-01T01:55:22.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.111+0000 7f0a7d370640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 0x7f0a780c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.111+0000 7f0a7d370640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a781536c0 0x7f0a7806b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.111+0000 7f0a7d370640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a7806bdd0 0x7f0a780c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.111+0000 7f0a7d370640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a780c43f0 con 0x7f0a640069d0 2023-12-01T01:55:22.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.111+0000 7f0a7d370640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7806c280 con 0x7f0a7806bdd0 2023-12-01T01:55:22.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.111+0000 7f0a7d370640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7815cdf0 con 0x7f0a781536c0 2023-12-01T01:55:22.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.112+0000 7f0a77fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 0x7f0a780c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.112+0000 7f0a77fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 0x7f0a780c4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33108/0 (socket says 172.21.15.115:33108) 2023-12-01T01:55:22.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.112+0000 7f0a77fff640 1 -- 172.21.15.115:0/2327491861 learned_addr learned my addr 172.21.15.115:0/2327491861 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.112+0000 7f0a77fff640 1 -- 172.21.15.115:0/2327491861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a781536c0 msgr2=0x7f0a7806b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.112+0000 7f0a77fff640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a781536c0 0x7f0a7806b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.112+0000 7f0a7cb6f640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a7806bdd0 0x7f0a780c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.112+0000 7f0a77fff640 1 -- 172.21.15.115:0/2327491861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a7806bdd0 msgr2=0x7f0a780c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.112+0000 7f0a77fff640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a7806bdd0 0x7f0a780c20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.112+0000 7f0a77fff640 1 -- 172.21.15.115:0/2327491861 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a780c2e40 con 0x7f0a640069d0 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.112+0000 7f0a77fff640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 0x7f0a780c4810 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f0a680115e0 tx=0x7f0a68011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.113+0000 7f0a757fa640 1 -- 172.21.15.115:0/2327491861 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68004220 con 0x7f0a640069d0 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.113+0000 7f0a757fa640 1 -- 172.21.15.115:0/2327491861 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a68004380 con 0x7f0a640069d0 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.113+0000 7f0a757fa640 1 -- 172.21.15.115:0/2327491861 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6800f8f0 con 0x7f0a640069d0 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.113+0000 7f0a7d370640 1 -- 172.21.15.115:0/2327491861 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a780c3d30 con 0x7f0a640069d0 2023-12-01T01:55:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.113+0000 7f0a7d370640 1 -- 172.21.15.115:0/2327491861 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a780c4d50 con 0x7f0a640069d0 2023-12-01T01:55:22.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.114+0000 7f0a7d370640 1 -- 172.21.15.115:0/2327491861 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a78067850 con 0x7f0a640069d0 2023-12-01T01:55:22.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.115+0000 7f0a757fa640 1 -- 172.21.15.115:0/2327491861 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0a6800fa50 con 0x7f0a640069d0 2023-12-01T01:55:22.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.116+0000 7f0a757fa640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a5806a550 0x7f0a5806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.116+0000 7f0a757fa640 1 -- 172.21.15.115:0/2327491861 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0a680889d0 con 0x7f0a640069d0 2023-12-01T01:55:22.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.116+0000 7f0a6ffff640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a5806a550 0x7f0a5806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.117+0000 7f0a6ffff640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a5806a550 0x7f0a5806ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a60034000 tx=0x7f0a60007000 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:22.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.120+0000 7f0a757fa640 1 -- 172.21.15.115:0/2327491861 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0a680541f0 con 0x7f0a640069d0 2023-12-01T01:55:22.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.311+0000 7f0a7d370640 1 -- 172.21.15.115:0/2327491861 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0a780c3b90 con 0x7f0a640069d0 2023-12-01T01:55:22.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.312+0000 7f0a757fa640 1 -- 172.21.15.115:0/2327491861 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v137) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0a780c3b90 con 0x7f0a640069d0 2023-12-01T01:55:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:22.313 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:55:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T01:55:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T01:55:22.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:22.313 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 -- 172.21.15.115:0/2327491861 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a5806a550 msgr2=0x7f0a5806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a5806a550 0x7f0a5806ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a60034000 tx=0x7f0a60007000 comp rx=0 tx=0).stop 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 -- 172.21.15.115:0/2327491861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 msgr2=0x7f0a780c4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 0x7f0a780c4810 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f0a680115e0 tx=0x7f0a68011aa0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 -- 172.21.15.115:0/2327491861 shutdown_connections 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0a5806a550 0x7f0a5806ca10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a781536c0 0x7f0a7806b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a7806bdd0 0x7f0a780c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 --2- 172.21.15.115:0/2327491861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a640069d0 0x7f0a780c4810 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 -- 172.21.15.115:0/2327491861 >> 172.21.15.115:0/2327491861 conn(0x7f0a780ce620 msgr2=0x7f0a780cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 -- 172.21.15.115:0/2327491861 shutdown_connections 2023-12-01T01:55:22.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.314+0000 7f0a7d370640 1 -- 172.21.15.115:0/2327491861 wait complete. 2023-12-01T01:55:22.335 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T01:55:22.335 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T01:55:22.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.548+0000 7f2007b39640 1 Processor -- start 2023-12-01T01:55:22.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.549+0000 7f2007b39640 1 -- start start 2023-12-01T01:55:22.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.550+0000 7f2007b39640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f200015ce40 0x7f20000b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.550+0000 7f2007b39640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20000d2730 0x7f20000d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.550+0000 7f2007b39640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20000d1460 con 0x7f20000b4640 2023-12-01T01:55:22.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.550+0000 7f2007b39640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200015d220 con 0x7f200015ce40 2023-12-01T01:55:22.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.550+0000 7f2007b39640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20000d1690 con 0x7f20000d2730 2023-12-01T01:55:22.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.550+0000 7f20058d5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f200015ce40 0x7f20000b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.550+0000 7f20058d5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f200015ce40 0x7f20000b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33124/0 (socket says 172.21.15.115:33124) 2023-12-01T01:55:22.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.550+0000 7f20058d5640 1 -- 172.21.15.115:0/501073372 learned_addr learned my addr 172.21.15.115:0/501073372 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:22.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.550+0000 7f20050d4640 1 --2- 172.21.15.115:0/501073372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20000d2730 0x7f20000d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.550+0000 7f20058d5640 1 -- 172.21.15.115:0/501073372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20000d2730 msgr2=0x7f20000d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:22.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.551+0000 7f20058d5640 1 --2- 172.21.15.115:0/501073372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20000d2730 0x7f20000d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.551+0000 7f20058d5640 1 -- 172.21.15.115:0/501073372 >> v1:172.21.15.142:6789/0 conn(0x7f20000b4640 legacy=0x7f2000069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:22.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.551+0000 7f20058d5640 1 -- 172.21.15.115:0/501073372 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000d1970 con 0x7f200015ce40 2023-12-01T01:55:22.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.551+0000 7f20050d4640 1 --2- 172.21.15.115:0/501073372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20000d2730 0x7f20000d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:22.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.552+0000 7f20058d5640 1 --2- 172.21.15.115:0/501073372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200015ce40 0x7f20000b8710 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009e50 tx=0x7f1ff002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7f8a878ac36c4a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:22.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.552+0000 7f20048d3640 1 -- 172.21.15.115:0/501073372 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff002c9e0 con 0x7f200015ce40 2023-12-01T01:55:22.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.552+0000 7f20048d3640 1 -- 172.21.15.115:0/501073372 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff002cb40 con 0x7f200015ce40 2023-12-01T01:55:22.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.552+0000 7f20048d3640 1 -- 172.21.15.115:0/501073372 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00343f0 con 0x7f200015ce40 2023-12-01T01:55:22.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.553+0000 7f2007b39640 1 -- 172.21.15.115:0/501073372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200015ce40 msgr2=0x7f20000b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:22.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.553+0000 7f2007b39640 1 --2- 172.21.15.115:0/501073372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200015ce40 0x7f20000b8710 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009e50 tx=0x7f1ff002be40 comp rx=0 tx=0).stop 2023-12-01T01:55:22.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.553+0000 7f2007b39640 1 -- 172.21.15.115:0/501073372 shutdown_connections 2023-12-01T01:55:22.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.553+0000 7f2007b39640 1 --2- 172.21.15.115:0/501073372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20000d2730 0x7f20000d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.553+0000 7f2007b39640 1 --2- 172.21.15.115:0/501073372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f200015ce40 0x7f20000b8710 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.553+0000 7f2007b39640 1 -- 172.21.15.115:0/501073372 >> 172.21.15.115:0/501073372 conn(0x7f2000065f90 msgr2=0x7f20000cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:22.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.553+0000 7f2007b39640 1 -- 172.21.15.115:0/501073372 shutdown_connections 2023-12-01T01:55:22.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.553+0000 7f2007b39640 1 -- 172.21.15.115:0/501073372 wait complete. 2023-12-01T01:55:22.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.555+0000 7f2007b39640 1 Processor -- start 2023-12-01T01:55:22.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.555+0000 7f2007b39640 1 -- start start 2023-12-01T01:55:22.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f2007b39640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20000b4640 0x7f20000d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f2007b39640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20000d2730 0x7f2000154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f2007b39640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200015ce40 0x7f2000152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f2007b39640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20001f3560 con 0x7f20000b4640 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f2007b39640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20001f28e0 con 0x7f200015ce40 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f2007b39640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20001f2bc0 con 0x7f20000d2730 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f20058d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20000b4640 0x7f20000d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f20058d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20000b4640 0x7f20000d2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33138/0 (socket says 172.21.15.115:33138) 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f20058d5640 1 -- 172.21.15.115:0/1869475910 learned_addr learned my addr 172.21.15.115:0/1869475910 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f20058d5640 1 -- 172.21.15.115:0/1869475910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20000d2730 msgr2=0x7f2000154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f20058d5640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20000d2730 0x7f2000154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f20058d5640 1 -- 172.21.15.115:0/1869475910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200015ce40 msgr2=0x7f2000152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f20060d6640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200015ce40 0x7f2000152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f20058d5640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200015ce40 0x7f2000152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:22.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.556+0000 7f20058d5640 1 -- 172.21.15.115:0/1869475910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff0009b30 con 0x7f20000b4640 2023-12-01T01:55:22.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.557+0000 7f20058d5640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20000b4640 0x7f20000d2e30 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0000f80 tx=0x7f1ff0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:22.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.557+0000 7f1ffe7fc640 1 -- 172.21.15.115:0/1869475910 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff003e450 con 0x7f20000b4640 2023-12-01T01:55:22.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.557+0000 7f1ffe7fc640 1 -- 172.21.15.115:0/1869475910 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff003e5f0 con 0x7f20000b4640 2023-12-01T01:55:22.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.557+0000 7f1ffe7fc640 1 -- 172.21.15.115:0/1869475910 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff003e9a0 con 0x7f20000b4640 2023-12-01T01:55:22.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.557+0000 7f2007b39640 1 -- 172.21.15.115:0/1869475910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20001f2ea0 con 0x7f20000b4640 2023-12-01T01:55:22.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.557+0000 7f2007b39640 1 -- 172.21.15.115:0/1869475910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20000d3370 con 0x7f20000b4640 2023-12-01T01:55:22.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.558+0000 7f2007b39640 1 -- 172.21.15.115:0/1869475910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20000d35c0 con 0x7f20000b4640 2023-12-01T01:55:22.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.559+0000 7f1ffe7fc640 1 -- 172.21.15.115:0/1869475910 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1ff003eb00 con 0x7f20000b4640 2023-12-01T01:55:22.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.560+0000 7f1ffe7fc640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fec06a550 0x7f1fec06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:22.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.560+0000 7f1ffe7fc640 1 -- 172.21.15.115:0/1869475910 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1ff00ab700 con 0x7f20000b4640 2023-12-01T01:55:22.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.560+0000 7f20050d4640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fec06a550 0x7f1fec06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:22.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.561+0000 7f20050d4640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fec06a550 0x7f1fec06ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4006fd0 tx=0x7f1ff4008040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:22.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.564+0000 7f1ffe7fc640 1 -- 172.21.15.115:0/1869475910 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1ff00770f0 con 0x7f20000b4640 2023-12-01T01:55:22.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:22.760+0000 7f2007b39640 1 -- 172.21.15.115:0/1869475910 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2000074210 con 0x7f20000b4640 2023-12-01T01:55:23.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.763+0000 7f1ffe7fc640 1 -- 172.21.15.115:0/1869475910 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v138) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1ff003edb0 con 0x7f20000b4640 2023-12-01T01:55:23.765 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T01:55:23.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.767+0000 7f2007b39640 1 -- 172.21.15.115:0/1869475910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fec06a550 msgr2=0x7f1fec06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:23.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fec06a550 0x7f1fec06ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4006fd0 tx=0x7f1ff4008040 comp rx=0 tx=0).stop 2023-12-01T01:55:23.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 -- 172.21.15.115:0/1869475910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20000b4640 msgr2=0x7f20000d2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:23.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20000b4640 0x7f20000d2e30 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0000f80 tx=0x7f1ff0004a40 comp rx=0 tx=0).stop 2023-12-01T01:55:23.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 -- 172.21.15.115:0/1869475910 shutdown_connections 2023-12-01T01:55:23.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1fec06a550 0x7f1fec06ca10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:23.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20000d2730 0x7f2000154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:23.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200015ce40 0x7f2000152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:23.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 --2- 172.21.15.115:0/1869475910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20000b4640 0x7f20000d2e30 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:23.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 -- 172.21.15.115:0/1869475910 >> 172.21.15.115:0/1869475910 conn(0x7f2000065f90 msgr2=0x7f200015b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:23.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 -- 172.21.15.115:0/1869475910 shutdown_connections 2023-12-01T01:55:23.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:23.768+0000 7f2007b39640 1 -- 172.21.15.115:0/1869475910 wait complete. 2023-12-01T01:55:23.794 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T01:55:24.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.010+0000 7fa338320640 1 Processor -- start 2023-12-01T01:55:24.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.011+0000 7fa338320640 1 -- start start 2023-12-01T01:55:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.012+0000 7fa338320640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330159db0 0x7fa330072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.012+0000 7fa338320640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3300cd030 con 0x7fa3300cd510 2023-12-01T01:55:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.012+0000 7fa338320640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa330071740 con 0x7fa3300ca110 2023-12-01T01:55:24.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.012+0000 7fa338320640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa330071a20 con 0x7fa330159db0 2023-12-01T01:55:24.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.013+0000 7fa3368bd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa3300cd510 0x7fa3300cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54714/0 (socket says 172.21.15.115:54714) 2023-12-01T01:55:24.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.013+0000 7fa3368bd640 1 -- 172.21.15.115:0/1496890605 learned_addr learned my addr 172.21.15.115:0/1496890605 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:24.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.013+0000 7fa3360bc640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330159db0 0x7fa330072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:24.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.013+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123643120 0 0) 0x7fa3300cd030 con 0x7fa3300cd510 2023-12-01T01:55:24.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.013+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa318005730 con 0x7fa3300cd510 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.013+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3027713534 0 0) 0x7fa318005730 con 0x7fa3300cd510 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.013+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330159db0 msgr2=0x7fa330072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.013+0000 7fa3350ba640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330159db0 0x7fa330072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 >> v1:172.21.15.142:6789/0 conn(0x7fa3300ca110 legacy=0x7fa3300ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa330071d00 con 0x7fa3300cd510 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa328001670 con 0x7fa3300cd510 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 >> v1:172.21.15.115:6789/0 conn(0x7fa3300cd510 legacy=0x7fa3300cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3350ba640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330159db0 0x7fa318005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3350ba640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318006130 0x7fa3180069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3350ba640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318006f10 0x7fa318007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa330071a20 con 0x7fa318006f10 2023-12-01T01:55:24.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31800dc70 con 0x7fa318006130 2023-12-01T01:55:24.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31800de30 con 0x7fa330159db0 2023-12-01T01:55:24.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3368bd640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318006f10 0x7fa318007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:24.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3360bc640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330159db0 0x7fa318005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:24.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3358bb640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318006130 0x7fa3180069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:24.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3368bd640 1 -- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330159db0 msgr2=0x7fa318005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:24.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3368bd640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330159db0 0x7fa318005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:24.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3368bd640 1 -- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318006130 msgr2=0x7fa3180069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:24.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3368bd640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318006130 0x7fa3180069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:24.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.014+0000 7fa3368bd640 1 -- 172.21.15.115:0/1496890605 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa318005730 con 0x7fa318006f10 2023-12-01T01:55:24.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.016+0000 7fa3368bd640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318006f10 0x7fa318007390 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c240 tx=0x7fa318007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb31c31d933d8de1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:24.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.016+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328005ad0 con 0x7fa318006f10 2023-12-01T01:55:24.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.016+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa328005c30 con 0x7fa318006f10 2023-12-01T01:55:24.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.016+0000 7fa3350ba640 1 -- 172.21.15.115:0/1496890605 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32802caf0 con 0x7fa318006f10 2023-12-01T01:55:24.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.016+0000 7fa338320640 1 -- 172.21.15.115:0/1496890605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318006f10 msgr2=0x7fa318007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:24.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.016+0000 7fa338320640 1 --2- 172.21.15.115:0/1496890605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318006f10 0x7fa318007390 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c240 tx=0x7fa318007a00 comp rx=0 tx=0).stop 2023-12-01T01:55:24.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.016+0000 7fa3368bd640 1 -- 172.21.15.115:0/1496890605 reap_dead start 2023-12-01T01:55:24.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.017+0000 7fa338320640 1 -- 172.21.15.115:0/1496890605 shutdown_connections 2023-12-01T01:55:24.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.017+0000 7fa338320640 1 -- 172.21.15.115:0/1496890605 >> 172.21.15.115:0/1496890605 conn(0x7fa3300cbb10 msgr2=0x7fa3300cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:24.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.017+0000 7fa338320640 1 -- 172.21.15.115:0/1496890605 shutdown_connections 2023-12-01T01:55:24.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.017+0000 7fa338320640 1 -- 172.21.15.115:0/1496890605 wait complete. 2023-12-01T01:55:24.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.018+0000 7fa338320640 1 Processor -- start 2023-12-01T01:55:24.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.019+0000 7fa338320640 1 -- start start 2023-12-01T01:55:24.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.019+0000 7fa338320640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318006f10 0x7fa3301559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:24.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.019+0000 7fa338320640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330152ef0 0x7fa3301f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:24.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.019+0000 7fa338320640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33015b200 0x7fa33015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:24.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.019+0000 7fa338320640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301f6910 con 0x7fa33015b200 2023-12-01T01:55:24.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.019+0000 7fa338320640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa330153370 con 0x7fa318006f10 2023-12-01T01:55:24.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.019+0000 7fa338320640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa330155fd0 con 0x7fa330152ef0 2023-12-01T01:55:24.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.020+0000 7fa3368bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33015b200 0x7fa33015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:24.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.020+0000 7fa3368bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33015b200 0x7fa33015c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33154/0 (socket says 172.21.15.115:33154) 2023-12-01T01:55:24.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.020+0000 7fa3368bd640 1 -- 172.21.15.115:0/1320546212 learned_addr learned my addr 172.21.15.115:0/1320546212 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.020+0000 7fa3358bb640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330152ef0 0x7fa3301f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.020+0000 7fa3360bc640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318006f10 0x7fa3301559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.020+0000 7fa3368bd640 1 -- 172.21.15.115:0/1320546212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330152ef0 msgr2=0x7fa3301f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.020+0000 7fa3368bd640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330152ef0 0x7fa3301f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.020+0000 7fa3368bd640 1 -- 172.21.15.115:0/1320546212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318006f10 msgr2=0x7fa3301559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.020+0000 7fa3368bd640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318006f10 0x7fa3301559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.020+0000 7fa3368bd640 1 -- 172.21.15.115:0/1320546212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33015a440 con 0x7fa33015b200 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.021+0000 7fa3368bd640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33015b200 0x7fa33015c490 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c360 tx=0x7fa32802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.021+0000 7fa32effd640 1 -- 172.21.15.115:0/1320546212 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32803c040 con 0x7fa33015b200 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.021+0000 7fa32effd640 1 -- 172.21.15.115:0/1320546212 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32802ccc0 con 0x7fa33015b200 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.021+0000 7fa32effd640 1 -- 172.21.15.115:0/1320546212 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328007270 con 0x7fa33015b200 2023-12-01T01:55:24.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.021+0000 7fa338320640 1 -- 172.21.15.115:0/1320546212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33015a720 con 0x7fa33015b200 2023-12-01T01:55:24.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.021+0000 7fa338320640 1 -- 172.21.15.115:0/1320546212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33015ace0 con 0x7fa33015b200 2023-12-01T01:55:24.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.022+0000 7fa338320640 1 -- 172.21.15.115:0/1320546212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3300cb7a0 con 0x7fa33015b200 2023-12-01T01:55:24.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.023+0000 7fa32effd640 1 -- 172.21.15.115:0/1320546212 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa328032070 con 0x7fa33015b200 2023-12-01T01:55:24.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.024+0000 7fa32effd640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa31c06a580 0x7fa31c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:24.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.024+0000 7fa32effd640 1 -- 172.21.15.115:0/1320546212 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fa3280a9360 con 0x7fa33015b200 2023-12-01T01:55:24.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.025+0000 7fa3360bc640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa31c06a580 0x7fa31c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:24.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.025+0000 7fa3360bc640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa31c06a580 0x7fa31c06ca40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa33015d070 tx=0x7fa32000b040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:24.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.029+0000 7fa32effd640 1 -- 172.21.15.115:0/1320546212 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa328074980 con 0x7fa33015b200 2023-12-01T01:55:24.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.220+0000 7fa338320640 1 -- 172.21.15.115:0/1320546212 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa33004e890 con 0x7fa33015b200 2023-12-01T01:55:24.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.784+0000 7fa32effd640 1 -- 172.21.15.115:0/1320546212 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v139) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa3280470d0 con 0x7fa33015b200 2023-12-01T01:55:24.788 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T01:55:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.795+0000 7fa338320640 1 -- 172.21.15.115:0/1320546212 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa31c06a580 msgr2=0x7fa31c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.795+0000 7fa338320640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa31c06a580 0x7fa31c06ca40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa33015d070 tx=0x7fa32000b040 comp rx=0 tx=0).stop 2023-12-01T01:55:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.795+0000 7fa338320640 1 -- 172.21.15.115:0/1320546212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33015b200 msgr2=0x7fa33015c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:24.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.795+0000 7fa338320640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33015b200 0x7fa33015c490 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c360 tx=0x7fa32802e070 comp rx=0 tx=0).stop 2023-12-01T01:55:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.797+0000 7fa338320640 1 -- 172.21.15.115:0/1320546212 shutdown_connections 2023-12-01T01:55:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.797+0000 7fa338320640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa31c06a580 0x7fa31c06ca40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.797+0000 7fa338320640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa330152ef0 0x7fa3301f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.797+0000 7fa338320640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33015b200 0x7fa33015c490 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.797+0000 7fa338320640 1 --2- 172.21.15.115:0/1320546212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318006f10 0x7fa3301559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.797+0000 7fa338320640 1 -- 172.21.15.115:0/1320546212 >> 172.21.15.115:0/1320546212 conn(0x7fa3300cbb10 msgr2=0x7fa3300cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:24.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.798+0000 7fa338320640 1 -- 172.21.15.115:0/1320546212 shutdown_connections 2023-12-01T01:55:24.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:24.801+0000 7fa338320640 1 -- 172.21.15.115:0/1320546212 wait complete. 2023-12-01T01:55:24.827 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T01:55:25.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.013+0000 7fe981f00640 1 Processor -- start 2023-12-01T01:55:25.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.013+0000 7fe981f00640 1 -- start start 2023-12-01T01:55:25.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.015+0000 7fe981f00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe97c15ac80 0x7fe97c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.015+0000 7fe981f00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe97c0b0e60 0x7fe97c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.015+0000 7fe981f00640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe97c0cdaa0 0x7fe97c0cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.015+0000 7fe981f00640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97c15b060 con 0x7fe97c0cdaa0 2023-12-01T01:55:25.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.015+0000 7fe981f00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97c0b0ce0 con 0x7fe97c0b0e60 2023-12-01T01:55:25.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.015+0000 7fe981f00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c0ce470 con 0x7fe97c15ac80 2023-12-01T01:55:25.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.015+0000 7fe97bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe97c0cdaa0 0x7fe97c0cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.015+0000 7fe97bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe97c0cdaa0 0x7fe97c0cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33164/0 (socket says 172.21.15.115:33164) 2023-12-01T01:55:25.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.015+0000 7fe97bfff640 1 -- 172.21.15.115:0/1706581284 learned_addr learned my addr 172.21.15.115:0/1706581284 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:25.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.016+0000 7fe97affd640 1 --2- 172.21.15.115:0/1706581284 >> v2:172.21.15.142:3300/0 conn(0x7fe97c0b0e60 0x7fe97c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.016+0000 7fe97b7fe640 1 --2- 172.21.15.115:0/1706581284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe97c15ac80 0x7fe97c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.016+0000 7fe97bfff640 1 -- 172.21.15.115:0/1706581284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe97c15ac80 msgr2=0x7fe97c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:25.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.016+0000 7fe97bfff640 1 --2- 172.21.15.115:0/1706581284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe97c15ac80 0x7fe97c0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.016+0000 7fe97bfff640 1 -- 172.21.15.115:0/1706581284 >> v2:172.21.15.142:3300/0 conn(0x7fe97c0b0e60 msgr2=0x7fe97c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:25.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.016+0000 7fe97bfff640 1 --2- 172.21.15.115:0/1706581284 >> v2:172.21.15.142:3300/0 conn(0x7fe97c0b0e60 0x7fe97c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.016+0000 7fe97bfff640 1 -- 172.21.15.115:0/1706581284 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c0cc9a0 con 0x7fe97c0cdaa0 2023-12-01T01:55:25.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.017+0000 7fe97bfff640 1 --2- 172.21.15.115:0/1706581284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c0cdaa0 0x7fe97c0cdf30 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe97c066990 tx=0x7fe97402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cc96b8b4459a0f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:25.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.017+0000 7fe97a7fc640 1 -- 172.21.15.115:0/1706581284 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974031870 con 0x7fe97c0cdaa0 2023-12-01T01:55:25.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.017+0000 7fe97a7fc640 1 -- 172.21.15.115:0/1706581284 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9740319d0 con 0x7fe97c0cdaa0 2023-12-01T01:55:25.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.017+0000 7fe97a7fc640 1 -- 172.21.15.115:0/1706581284 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974031d60 con 0x7fe97c0cdaa0 2023-12-01T01:55:25.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.018+0000 7fe981f00640 1 -- 172.21.15.115:0/1706581284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c0cdaa0 msgr2=0x7fe97c0cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.018+0000 7fe981f00640 1 --2- 172.21.15.115:0/1706581284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c0cdaa0 0x7fe97c0cdf30 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe97c066990 tx=0x7fe97402b910 comp rx=0 tx=0).stop 2023-12-01T01:55:25.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.018+0000 7fe981f00640 1 -- 172.21.15.115:0/1706581284 shutdown_connections 2023-12-01T01:55:25.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.018+0000 7fe981f00640 1 --2- 172.21.15.115:0/1706581284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c0cdaa0 0x7fe97c0cdf30 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.018+0000 7fe981f00640 1 --2- 172.21.15.115:0/1706581284 >> v2:172.21.15.142:3300/0 conn(0x7fe97c0b0e60 0x7fe97c0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.018+0000 7fe981f00640 1 --2- 172.21.15.115:0/1706581284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe97c15ac80 0x7fe97c0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.018+0000 7fe981f00640 1 -- 172.21.15.115:0/1706581284 >> 172.21.15.115:0/1706581284 conn(0x7fe97c065960 msgr2=0x7fe97c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:25.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.018+0000 7fe981f00640 1 -- 172.21.15.115:0/1706581284 shutdown_connections 2023-12-01T01:55:25.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.018+0000 7fe981f00640 1 -- 172.21.15.115:0/1706581284 wait complete. 2023-12-01T01:55:25.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.020+0000 7fe981f00640 1 Processor -- start 2023-12-01T01:55:25.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.020+0000 7fe981f00640 1 -- start start 2023-12-01T01:55:25.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.021+0000 7fe981f00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe97c0b0e60 0x7fe97c13bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.021+0000 7fe981f00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe97c0cdaa0 0x7fe97c13b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.021+0000 7fe981f00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c15ac80 0x7fe97c13d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.021+0000 7fe981f00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c13c2d0 con 0x7fe97c15ac80 2023-12-01T01:55:25.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.021+0000 7fe981f00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c13d0d0 con 0x7fe97c0cdaa0 2023-12-01T01:55:25.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.021+0000 7fe981f00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c13ba80 con 0x7fe97c0b0e60 2023-12-01T01:55:25.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.021+0000 7fe97bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c15ac80 0x7fe97c13d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.021+0000 7fe97bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c15ac80 0x7fe97c13d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33180/0 (socket says 172.21.15.115:33180) 2023-12-01T01:55:25.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.021+0000 7fe97bfff640 1 -- 172.21.15.115:0/65408301 learned_addr learned my addr 172.21.15.115:0/65408301 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:25.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe97affd640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe97c0cdaa0 0x7fe97c13b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe97b7fe640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe97c0b0e60 0x7fe97c13bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe97bfff640 1 -- 172.21.15.115:0/65408301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe97c0b0e60 msgr2=0x7fe97c13bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe97bfff640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe97c0b0e60 0x7fe97c13bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe97bfff640 1 -- 172.21.15.115:0/65408301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe97c0cdaa0 msgr2=0x7fe97c13b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe97bfff640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe97c0cdaa0 0x7fe97c13b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe97bfff640 1 -- 172.21.15.115:0/65408301 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9740094d0 con 0x7fe97c15ac80 2023-12-01T01:55:25.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe97bfff640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c15ac80 0x7fe97c13d560 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fe9740090b0 tx=0x7fe9740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:25.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe963fff640 1 -- 172.21.15.115:0/65408301 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97403ac60 con 0x7fe97c15ac80 2023-12-01T01:55:25.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe963fff640 1 -- 172.21.15.115:0/65408301 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97403adc0 con 0x7fe97c15ac80 2023-12-01T01:55:25.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe963fff640 1 -- 172.21.15.115:0/65408301 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974044630 con 0x7fe97c15ac80 2023-12-01T01:55:25.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.022+0000 7fe981f00640 1 -- 172.21.15.115:0/65408301 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97c13c4c0 con 0x7fe97c15ac80 2023-12-01T01:55:25.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.023+0000 7fe981f00640 1 -- 172.21.15.115:0/65408301 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c13ca80 con 0x7fe97c15ac80 2023-12-01T01:55:25.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.024+0000 7fe981f00640 1 -- 172.21.15.115:0/65408301 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97c04e890 con 0x7fe97c15ac80 2023-12-01T01:55:25.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.025+0000 7fe963fff640 1 -- 172.21.15.115:0/65408301 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe974044790 con 0x7fe97c15ac80 2023-12-01T01:55:25.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.026+0000 7fe963fff640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe96806a550 0x7fe96806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.026+0000 7fe963fff640 1 -- 172.21.15.115:0/65408301 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe97402d040 con 0x7fe97c15ac80 2023-12-01T01:55:25.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.026+0000 7fe97b7fe640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe96806a550 0x7fe96806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.027+0000 7fe97b7fe640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe96806a550 0x7fe96806ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0099c0 tx=0x7fe96c003990 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:25.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.030+0000 7fe963fff640 1 -- 172.21.15.115:0/65408301 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe974076e40 con 0x7fe97c15ac80 2023-12-01T01:55:25.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.251+0000 7fe981f00640 1 -- 172.21.15.115:0/65408301 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe97c0cefd0 con 0x7fe97c15ac80 2023-12-01T01:55:25.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.260+0000 7fe963fff640 1 -- 172.21.15.115:0/65408301 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v60) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe974052020 con 0x7fe97c15ac80 2023-12-01T01:55:25.262 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 16 and data pool 17 2023-12-01T01:55:25.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.265+0000 7fe981f00640 1 -- 172.21.15.115:0/65408301 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe96806a550 msgr2=0x7fe96806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.265+0000 7fe981f00640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe96806a550 0x7fe96806ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0099c0 tx=0x7fe96c003990 comp rx=0 tx=0).stop 2023-12-01T01:55:25.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.265+0000 7fe981f00640 1 -- 172.21.15.115:0/65408301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c15ac80 msgr2=0x7fe97c13d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.265+0000 7fe981f00640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c15ac80 0x7fe97c13d560 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fe9740090b0 tx=0x7fe9740046c0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.265+0000 7fe981f00640 1 -- 172.21.15.115:0/65408301 shutdown_connections 2023-12-01T01:55:25.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.265+0000 7fe981f00640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe96806a550 0x7fe96806ca10 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0099c0 tx=0x7fe96c003990 comp rx=0 tx=0).stop 2023-12-01T01:55:25.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.265+0000 7fe981f00640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe97c15ac80 0x7fe97c13d560 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.265+0000 7fe981f00640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe97c0cdaa0 0x7fe97c13b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.265+0000 7fe981f00640 1 --2- 172.21.15.115:0/65408301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe97c0b0e60 0x7fe97c13bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.265+0000 7fe981f00640 1 -- 172.21.15.115:0/65408301 >> 172.21.15.115:0/65408301 conn(0x7fe97c065960 msgr2=0x7fe97c15da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:25.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.266+0000 7fe981f00640 1 -- 172.21.15.115:0/65408301 shutdown_connections 2023-12-01T01:55:25.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.267+0000 7fe981f00640 1 -- 172.21.15.115:0/65408301 wait complete. 2023-12-01T01:55:25.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:55:25.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.472+0000 7f1ada777640 1 Processor -- start 2023-12-01T01:55:25.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.472+0000 7f1ada777640 1 -- start start 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.474+0000 7f1ada777640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1ad415e2f0 0x7f1ad40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.474+0000 7f1ada777640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40adf10 0x7f1ad40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.474+0000 7f1ada777640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad40add10 con 0x7f1ad406f2f0 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.474+0000 7f1ada777640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad40c5760 con 0x7f1ad415e2f0 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.474+0000 7f1ada777640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad415e6d0 con 0x7f1ad40adf10 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.474+0000 7f1ad37fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1ad406f2f0 0x7f1ad40c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54730/0 (socket says 172.21.15.115:54730) 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.474+0000 7f1ad37fe640 1 -- 172.21.15.115:0/1657139738 learned_addr learned my addr 172.21.15.115:0/1657139738 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.474+0000 7f1ad3fff640 1 --2- 172.21.15.115:0/1657139738 >> v2:172.21.15.142:3300/0 conn(0x7f1ad415e2f0 0x7f1ad40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.474+0000 7f1ad8d14640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40adf10 0x7f1ad40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487381250 0 0) 0x7f1ad40add10 con 0x7f1ad406f2f0 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1abc003520 con 0x7f1ad406f2f0 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1870709607 0 0) 0x7f1abc003520 con 0x7f1ad406f2f0 2023-12-01T01:55:25.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40adf10 msgr2=0x7f1ad40caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40adf10 0x7f1ad40caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 >> v2:172.21.15.142:3300/0 conn(0x7f1ad415e2f0 msgr2=0x7f1ad40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 --2- 172.21.15.115:0/1657139738 >> v2:172.21.15.142:3300/0 conn(0x7f1ad415e2f0 0x7f1ad40ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40cf7b0 con 0x7f1ad406f2f0 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f1ac8001a10 con 0x7f1ad406f2f0 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 >> v1:172.21.15.115:6789/0 conn(0x7f1ad406f2f0 legacy=0x7f1ad40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40adf10 0x7f1abc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1abc003fb0 0x7f1abc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1abc004d00 0x7f1abc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40c5760 con 0x7f1abc004d00 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad415e6d0 con 0x7f1abc003fb0 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc00bb40 con 0x7f1ad40adf10 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.475+0000 7f1ad8d14640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1abc004d00 0x7f1abc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.476+0000 7f1ad37fe640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1abc003fb0 0x7f1abc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.476+0000 7f1ad8d14640 1 -- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40adf10 msgr2=0x7f1abc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.476+0000 7f1ad3fff640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40adf10 0x7f1abc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.476+0000 7f1ad8d14640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40adf10 0x7f1abc0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.476+0000 7f1ad8d14640 1 -- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1abc003fb0 msgr2=0x7f1abc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.476+0000 7f1ad8d14640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1abc003fb0 0x7f1abc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.476+0000 7f1ad8d14640 1 -- 172.21.15.115:0/1657139738 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc003520 con 0x7f1abc004d00 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.476+0000 7f1ad37fe640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1abc003fb0 0x7f1abc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.477+0000 7f1ad8d14640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1abc004d00 0x7f1abc005180 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009ac0 tx=0x7f1abc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eeb37cf6d8e03e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.477+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc02c6b0 con 0x7f1abc004d00 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.477+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc02c810 con 0x7f1abc004d00 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.477+0000 7f1ad2ffd640 1 -- 172.21.15.115:0/1657139738 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc02cba0 con 0x7f1abc004d00 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.478+0000 7f1ada777640 1 -- 172.21.15.115:0/1657139738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1abc004d00 msgr2=0x7f1abc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.478+0000 7f1ada777640 1 --2- 172.21.15.115:0/1657139738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1abc004d00 0x7f1abc005180 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009ac0 tx=0x7f1abc0057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.478+0000 7f1ad8d14640 1 -- 172.21.15.115:0/1657139738 reap_dead start 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.478+0000 7f1ada777640 1 -- 172.21.15.115:0/1657139738 shutdown_connections 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.478+0000 7f1ada777640 1 -- 172.21.15.115:0/1657139738 >> 172.21.15.115:0/1657139738 conn(0x7f1ad40ce880 msgr2=0x7f1ad40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.478+0000 7f1ada777640 1 -- 172.21.15.115:0/1657139738 shutdown_connections 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.478+0000 7f1ada777640 1 -- 172.21.15.115:0/1657139738 wait complete. 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.480+0000 7f1ada777640 1 Processor -- start 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.480+0000 7f1ada777640 1 -- start start 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ada777640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ad4159ed0 0x7f1ad40c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ada777640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad406c2b0 0x7f1ad40c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ada777640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40c3b70 0x7f1ad40c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ada777640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40c6c60 con 0x7f1ad4159ed0 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ada777640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad415c1a0 con 0x7f1ad406c2b0 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ada777640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad415c480 con 0x7f1ad40c3b70 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ad3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ad4159ed0 0x7f1ad40c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ad3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ad4159ed0 0x7f1ad40c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33198/0 (socket says 172.21.15.115:33198) 2023-12-01T01:55:25.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ad3fff640 1 -- 172.21.15.115:0/2357141896 learned_addr learned my addr 172.21.15.115:0/2357141896 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ad37fe640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad406c2b0 0x7f1ad40c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ad3fff640 1 -- 172.21.15.115:0/2357141896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40c3b70 msgr2=0x7f1ad40c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ad3fff640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40c3b70 0x7f1ad40c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ad3fff640 1 -- 172.21.15.115:0/2357141896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad406c2b0 msgr2=0x7f1ad40c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.481+0000 7f1ad3fff640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad406c2b0 0x7f1ad40c3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.482+0000 7f1ad3fff640 1 -- 172.21.15.115:0/2357141896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad415c760 con 0x7f1ad4159ed0 2023-12-01T01:55:25.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.482+0000 7f1ad3fff640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ad4159ed0 0x7f1ad40c6360 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40108e0 tx=0x7f1ac4010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:25.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.482+0000 7f1ad0ff9640 1 -- 172.21.15.115:0/2357141896 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400eae0 con 0x7f1ad4159ed0 2023-12-01T01:55:25.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.482+0000 7f1ad0ff9640 1 -- 172.21.15.115:0/2357141896 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac400ec40 con 0x7f1ad4159ed0 2023-12-01T01:55:25.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.482+0000 7f1ad0ff9640 1 -- 172.21.15.115:0/2357141896 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac401c400 con 0x7f1ad4159ed0 2023-12-01T01:55:25.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.482+0000 7f1ada777640 1 -- 172.21.15.115:0/2357141896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad415ca40 con 0x7f1ad4159ed0 2023-12-01T01:55:25.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.482+0000 7f1ada777640 1 -- 172.21.15.115:0/2357141896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad40c68a0 con 0x7f1ad4159ed0 2023-12-01T01:55:25.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.483+0000 7f1ada777640 1 -- 172.21.15.115:0/2357141896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad406fda0 con 0x7f1ad4159ed0 2023-12-01T01:55:25.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.484+0000 7f1ad0ff9640 1 -- 172.21.15.115:0/2357141896 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1ac401c560 con 0x7f1ad4159ed0 2023-12-01T01:55:25.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.486+0000 7f1ad0ff9640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ac006a550 0x7f1ac006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.486+0000 7f1ad0ff9640 1 -- 172.21.15.115:0/2357141896 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ac4089100 con 0x7f1ad4159ed0 2023-12-01T01:55:25.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.486+0000 7f1ad37fe640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ac006a550 0x7f1ac006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.487+0000 7f1ad37fe640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ac006a550 0x7f1ac006ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8001460 tx=0x7f1ac8009040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:25.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.490+0000 7f1ad0ff9640 1 -- 172.21.15.115:0/2357141896 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1ac4054680 con 0x7f1ad4159ed0 2023-12-01T01:55:25.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.675+0000 7f1ada777640 1 -- 172.21.15.115:0/2357141896 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ad404e890 con 0x7f1ad4159ed0 2023-12-01T01:55:25.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.677+0000 7f1ad0ff9640 1 -- 172.21.15.115:0/2357141896 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16176 (secure 0 0 0) 0x7f1ac4057ca0 con 0x7f1ad4159ed0 2023-12-01T01:55:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:25.679 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":140,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:55:25.253044+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:55:22.818353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-01T01:55:24.272936+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:55:25.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 -- 172.21.15.115:0/2357141896 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ac006a550 msgr2=0x7f1ac006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ac006a550 0x7f1ac006ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8001460 tx=0x7f1ac8009040 comp rx=0 tx=0).stop 2023-12-01T01:55:25.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 -- 172.21.15.115:0/2357141896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ad4159ed0 msgr2=0x7f1ad40c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ad4159ed0 0x7f1ad40c6360 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40108e0 tx=0x7f1ac4010da0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 -- 172.21.15.115:0/2357141896 shutdown_connections 2023-12-01T01:55:25.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1ac006a550 0x7f1ac006ca10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ad40c3b70 0x7f1ad40c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad406c2b0 0x7f1ad40c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 --2- 172.21.15.115:0/2357141896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ad4159ed0 0x7f1ad40c6360 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 -- 172.21.15.115:0/2357141896 >> 172.21.15.115:0/2357141896 conn(0x7f1ad40ce880 msgr2=0x7f1ad40ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:25.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 -- 172.21.15.115:0/2357141896 shutdown_connections 2023-12-01T01:55:25.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.679+0000 7f1ada777640 1 -- 172.21.15.115:0/2357141896 wait complete. 2023-12-01T01:55:25.698 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:55:25.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.904+0000 7f7e01500640 1 Processor -- start 2023-12-01T01:55:25.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.905+0000 7f7e01500640 1 -- start start 2023-12-01T01:55:25.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.906+0000 7f7e01500640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7dfc15e2f0 0x7f7dfc0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.906+0000 7f7e01500640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dfc0add10 con 0x7f7dfc0adf10 2023-12-01T01:55:25.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.906+0000 7f7e01500640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dfc0c5760 con 0x7f7dfc06f2f0 2023-12-01T01:55:25.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.906+0000 7f7e01500640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dfc0c9cf0 con 0x7f7dfc15e2f0 2023-12-01T01:55:25.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.906+0000 7f7dfb7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7dfc0adf10 0x7f7dfc0c95e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54744/0 (socket says 172.21.15.115:54744) 2023-12-01T01:55:25.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.906+0000 7f7dfb7fe640 1 -- 172.21.15.115:0/3171043100 learned_addr learned my addr 172.21.15.115:0/3171043100 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:25.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfaffd640 1 --2- 172.21.15.115:0/3171043100 >> v2:172.21.15.142:3300/0 conn(0x7f7dfc15e2f0 0x7f7dfc0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214983410 0 0) 0x7f7dfc0add10 con 0x7f7dfc0adf10 2023-12-01T01:55:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de8003520 con 0x7f7dfc0adf10 2023-12-01T01:55:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1755908217 0 0) 0x7f7de8003520 con 0x7f7dfc0adf10 2023-12-01T01:55:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 >> v2:172.21.15.142:3300/0 conn(0x7f7dfc15e2f0 msgr2=0x7f7dfc0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 --2- 172.21.15.115:0/3171043100 >> v2:172.21.15.142:3300/0 conn(0x7f7dfc15e2f0 0x7f7dfc0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 >> v1:172.21.15.142:6789/0 conn(0x7f7dfc06f2f0 legacy=0x7f7dfc0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc15e6d0 con 0x7f7dfc0adf10 2023-12-01T01:55:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f7df40019f0 con 0x7f7dfc0adf10 2023-12-01T01:55:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 >> v1:172.21.15.115:6789/0 conn(0x7f7dfc0adf10 legacy=0x7f7dfc0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 --2- 172.21.15.115:0/3171043100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7de8003a30 0x7f7de8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 --2- 172.21.15.115:0/3171043100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7de8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 --2- 172.21.15.115:0/3171043100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7de8005360 0x7f7de800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0c9cf0 con 0x7f7de80047c0 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de800c2a0 con 0x7f7de8003a30 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.907+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de800c460 con 0x7f7de8005360 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.908+0000 7f7df3fff640 1 --2- 172.21.15.115:0/3171043100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7de8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.908+0000 7f7df3fff640 1 -- 172.21.15.115:0/3171043100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7de8005360 msgr2=0x7f7de800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.908+0000 7f7df3fff640 1 --2- 172.21.15.115:0/3171043100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7de8005360 0x7f7de800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.908+0000 7f7df3fff640 1 -- 172.21.15.115:0/3171043100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7de8003a30 msgr2=0x7f7de8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.908+0000 7f7dfaffd640 1 --2- 172.21.15.115:0/3171043100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7de8003a30 0x7f7de8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.908+0000 7f7df3fff640 1 --2- 172.21.15.115:0/3171043100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7de8003a30 0x7f7de8003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.908+0000 7f7dfb7fe640 1 -- 172.21.15.115:0/3171043100 reap_dead start 2023-12-01T01:55:25.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.908+0000 7f7df3fff640 1 -- 172.21.15.115:0/3171043100 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de8003520 con 0x7f7de80047c0 2023-12-01T01:55:25.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.909+0000 7f7df3fff640 1 --2- 172.21.15.115:0/3171043100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7de8004c40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f7de4003db0 tx=0x7f7de80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133a607c6e4178d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.909+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4032600 con 0x7f7de80047c0 2023-12-01T01:55:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.909+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de402e050 con 0x7f7de80047c0 2023-12-01T01:55:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.909+0000 7f7dfa7fc640 1 -- 172.21.15.115:0/3171043100 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de40329e0 con 0x7f7de80047c0 2023-12-01T01:55:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.910+0000 7f7e01500640 1 -- 172.21.15.115:0/3171043100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 msgr2=0x7f7de8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.910+0000 7f7e01500640 1 --2- 172.21.15.115:0/3171043100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7de8004c40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f7de4003db0 tx=0x7f7de80057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.910+0000 7f7e01500640 1 -- 172.21.15.115:0/3171043100 shutdown_connections 2023-12-01T01:55:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.910+0000 7f7e01500640 1 --2- 172.21.15.115:0/3171043100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7de8004c40 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.910+0000 7f7e01500640 1 -- 172.21.15.115:0/3171043100 >> 172.21.15.115:0/3171043100 conn(0x7f7dfc0ce880 msgr2=0x7f7dfc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.910+0000 7f7e01500640 1 -- 172.21.15.115:0/3171043100 shutdown_connections 2023-12-01T01:55:25.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.910+0000 7f7e01500640 1 -- 172.21.15.115:0/3171043100 wait complete. 2023-12-01T01:55:25.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.912+0000 7f7e01500640 1 Processor -- start 2023-12-01T01:55:25.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.912+0000 7f7e01500640 1 -- start start 2023-12-01T01:55:25.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.913+0000 7f7e01500640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7dfc0c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.913+0000 7f7e01500640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dfc1f1b10 0x7f7dfc0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.913+0000 7f7e01500640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dfc0ac8a0 0x7f7dfc0acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.913+0000 7f7e01500640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0cd440 con 0x7f7de80047c0 2023-12-01T01:55:25.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.913+0000 7f7e01500640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0ad260 con 0x7f7dfc1f1b10 2023-12-01T01:55:25.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.913+0000 7f7e01500640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0c3fb0 con 0x7f7dfc0ac8a0 2023-12-01T01:55:25.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.913+0000 7f7dfaffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7dfc0c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.913+0000 7f7dfaffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7dfc0c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33206/0 (socket says 172.21.15.115:33206) 2023-12-01T01:55:25.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.913+0000 7f7dfaffd640 1 -- 172.21.15.115:0/4157006799 learned_addr learned my addr 172.21.15.115:0/4157006799 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:25.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7df3fff640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dfc1f1b10 0x7f7dfc0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7dfb7fe640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dfc0ac8a0 0x7f7dfc0acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7dfaffd640 1 -- 172.21.15.115:0/4157006799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dfc0ac8a0 msgr2=0x7f7dfc0acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7dfaffd640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dfc0ac8a0 0x7f7dfc0acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7dfaffd640 1 -- 172.21.15.115:0/4157006799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dfc1f1b10 msgr2=0x7f7dfc0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7dfaffd640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dfc1f1b10 0x7f7dfc0cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7dfaffd640 1 -- 172.21.15.115:0/4157006799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc0cc8c0 con 0x7f7de80047c0 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7df3fff640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dfc1f1b10 0x7f7dfc0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7dfaffd640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7dfc0c5950 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00e540 tx=0x7f7dec00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7df37fe640 1 -- 172.21.15.115:0/4157006799 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec013610 con 0x7f7de80047c0 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7df37fe640 1 -- 172.21.15.115:0/4157006799 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec013770 con 0x7f7de80047c0 2023-12-01T01:55:25.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7e01500640 1 -- 172.21.15.115:0/4157006799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dfc0ccba0 con 0x7f7de80047c0 2023-12-01T01:55:25.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7df37fe640 1 -- 172.21.15.115:0/4157006799 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec013ae0 con 0x7f7de80047c0 2023-12-01T01:55:25.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.914+0000 7f7e01500640 1 -- 172.21.15.115:0/4157006799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dfc0cd160 con 0x7f7de80047c0 2023-12-01T01:55:25.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.916+0000 7f7e01500640 1 -- 172.21.15.115:0/4157006799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dfc06fda0 con 0x7f7de80047c0 2023-12-01T01:55:25.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.917+0000 7f7df37fe640 1 -- 172.21.15.115:0/4157006799 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f7dec013c40 con 0x7f7de80047c0 2023-12-01T01:55:25.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.917+0000 7f7df37fe640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 0x7f7ddc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:25.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.918+0000 7f7df37fe640 1 -- 172.21.15.115:0/4157006799 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dec088860 con 0x7f7de80047c0 2023-12-01T01:55:25.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.918+0000 7f7df3fff640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 0x7f7ddc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:25.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.918+0000 7f7df3fff640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 0x7f7ddc06ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7de4034000 tx=0x7f7de4009030 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:25.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:25.922+0000 7f7df37fe640 1 -- 172.21.15.115:0/4157006799 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7dec053c30 con 0x7f7de80047c0 2023-12-01T01:55:26.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.098+0000 7f7e01500640 1 -- 172.21.15.115:0/4157006799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dfc04e890 con 0x7f7de80047c0 2023-12-01T01:55:26.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.101+0000 7f7df37fe640 1 -- 172.21.15.115:0/4157006799 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16176 (secure 0 0 0) 0x7f7dec057250 con 0x7f7de80047c0 2023-12-01T01:55:26.103 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:26.103 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":140,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:55:25.253044+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:55:22.818353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-01T01:55:24.272936+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:55:26.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.103+0000 7f7e01500640 1 -- 172.21.15.115:0/4157006799 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 msgr2=0x7f7ddc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:26.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.103+0000 7f7e01500640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 0x7f7ddc06ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7de4034000 tx=0x7f7de4009030 comp rx=0 tx=0).stop 2023-12-01T01:55:26.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.103+0000 7f7e01500640 1 -- 172.21.15.115:0/4157006799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 msgr2=0x7f7dfc0c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:26.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.103+0000 7f7e01500640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7dfc0c5950 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00e540 tx=0x7f7dec00ea00 comp rx=0 tx=0).stop 2023-12-01T01:55:26.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.104+0000 7f7e01500640 1 -- 172.21.15.115:0/4157006799 shutdown_connections 2023-12-01T01:55:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.104+0000 7f7e01500640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ddc06a550 0x7f7ddc06ca10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.104+0000 7f7e01500640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dfc0ac8a0 0x7f7dfc0acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.104+0000 7f7e01500640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dfc1f1b10 0x7f7dfc0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.104+0000 7f7e01500640 1 --2- 172.21.15.115:0/4157006799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7de80047c0 0x7f7dfc0c5950 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.104+0000 7f7e01500640 1 -- 172.21.15.115:0/4157006799 >> 172.21.15.115:0/4157006799 conn(0x7f7dfc0ce880 msgr2=0x7f7dfc0aec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.104+0000 7f7e01500640 1 -- 172.21.15.115:0/4157006799 shutdown_connections 2023-12-01T01:55:26.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.104+0000 7f7e01500640 1 -- 172.21.15.115:0/4157006799 wait complete. 2023-12-01T01:55:26.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T01:55:26.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.339+0000 7f80bcfd9640 1 Processor -- start 2023-12-01T01:55:26.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.340+0000 7f80bcfd9640 1 -- start start 2023-12-01T01:55:26.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.341+0000 7f80bcfd9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f80b815cfa0 0x7f80b80b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:26.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.341+0000 7f80bcfd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80b80b7e70 0x7f80b80b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:26.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.341+0000 7f80bcfd9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b80b6890 con 0x7f80b80b6210 2023-12-01T01:55:26.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.341+0000 7f80bcfd9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b80b6b70 con 0x7f80b815cfa0 2023-12-01T01:55:26.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.341+0000 7f80bcfd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80b6e50 con 0x7f80b80b7e70 2023-12-01T01:55:26.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.341+0000 7f80b659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f80b815cfa0 0x7f80b80b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:26.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.341+0000 7f80b659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f80b815cfa0 0x7f80b80b7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33214/0 (socket says 172.21.15.115:33214) 2023-12-01T01:55:26.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.341+0000 7f80b659c640 1 -- 172.21.15.115:0/2994457008 learned_addr learned my addr 172.21.15.115:0/2994457008 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:26.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.342+0000 7f80b659c640 1 -- 172.21.15.115:0/2994457008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80b80b7e70 msgr2=0x7f80b80b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:26.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.342+0000 7f80b659c640 1 --2- 172.21.15.115:0/2994457008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80b80b7e70 0x7f80b80b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:26.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.342+0000 7f80b659c640 1 -- 172.21.15.115:0/2994457008 >> v1:172.21.15.115:6789/0 conn(0x7f80b80b6210 legacy=0x7f80b80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:26.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.342+0000 7f80b659c640 1 -- 172.21.15.115:0/2994457008 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80b7130 con 0x7f80b815cfa0 2023-12-01T01:55:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.343+0000 7f80b659c640 1 --2- 172.21.15.115:0/2994457008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b815cfa0 0x7f80b80b7930 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009870 tx=0x7f80a802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2985c98d781e7e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.343+0000 7f80b5d9b640 1 -- 172.21.15.115:0/2994457008 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8031640 con 0x7f80b815cfa0 2023-12-01T01:55:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.343+0000 7f80b5d9b640 1 -- 172.21.15.115:0/2994457008 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a80317a0 con 0x7f80b815cfa0 2023-12-01T01:55:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.343+0000 7f80b5d9b640 1 -- 172.21.15.115:0/2994457008 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8031b30 con 0x7f80b815cfa0 2023-12-01T01:55:26.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.344+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2994457008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b815cfa0 msgr2=0x7f80b80b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.344+0000 7f80bcfd9640 1 --2- 172.21.15.115:0/2994457008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b815cfa0 0x7f80b80b7930 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009870 tx=0x7f80a802b8f0 comp rx=0 tx=0).stop 2023-12-01T01:55:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.344+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2994457008 shutdown_connections 2023-12-01T01:55:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.344+0000 7f80bcfd9640 1 --2- 172.21.15.115:0/2994457008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80b80b7e70 0x7f80b80b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.344+0000 7f80bcfd9640 1 --2- 172.21.15.115:0/2994457008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b815cfa0 0x7f80b80b7930 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.344+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2994457008 >> 172.21.15.115:0/2994457008 conn(0x7f80b806a0b0 msgr2=0x7f80b806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.344+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2994457008 shutdown_connections 2023-12-01T01:55:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.344+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2994457008 wait complete. 2023-12-01T01:55:26.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.345+0000 7f80bcfd9640 1 Processor -- start 2023-12-01T01:55:26.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.346+0000 7f80bcfd9640 1 -- start start 2023-12-01T01:55:26.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.346+0000 7f80bcfd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b80b6210 0x7f80b8151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.346+0000 7f80bcfd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80b80b7e70 0x7f80b815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.346+0000 7f80bcfd9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80b815cfa0 0x7f80b80c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.346+0000 7f80bcfd9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b815fca0 con 0x7f80b80b6210 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.346+0000 7f80bcfd9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b815f090 con 0x7f80b815cfa0 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.346+0000 7f80bcfd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b815f370 con 0x7f80b80b7e70 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.347+0000 7f80b659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b80b6210 0x7f80b8151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.347+0000 7f80b659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b80b6210 0x7f80b8151e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33226/0 (socket says 172.21.15.115:33226) 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.347+0000 7f80b659c640 1 -- 172.21.15.115:0/2757739696 learned_addr learned my addr 172.21.15.115:0/2757739696 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.347+0000 7f80b659c640 1 -- 172.21.15.115:0/2757739696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80b80b7e70 msgr2=0x7f80b815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.347+0000 7f80b659c640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80b80b7e70 0x7f80b815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.347+0000 7f80b659c640 1 -- 172.21.15.115:0/2757739696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80b815cfa0 msgr2=0x7f80b80c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.347+0000 7f80b659c640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80b815cfa0 0x7f80b80c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.347+0000 7f80b659c640 1 -- 172.21.15.115:0/2757739696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80094d0 con 0x7f80b80b6210 2023-12-01T01:55:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.347+0000 7f80b659c640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b80b6210 0x7f80b8151e60 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f80a8000c00 tx=0x7f80a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:26.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.348+0000 7f80aeffd640 1 -- 172.21.15.115:0/2757739696 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a803ac60 con 0x7f80b80b6210 2023-12-01T01:55:26.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.348+0000 7f80aeffd640 1 -- 172.21.15.115:0/2757739696 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a803ade0 con 0x7f80b80b6210 2023-12-01T01:55:26.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.348+0000 7f80aeffd640 1 -- 172.21.15.115:0/2757739696 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8044630 con 0x7f80b80b6210 2023-12-01T01:55:26.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.348+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2757739696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b80b8c00 con 0x7f80b80b6210 2023-12-01T01:55:26.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.348+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2757739696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b815f930 con 0x7f80b80b6210 2023-12-01T01:55:26.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.349+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2757739696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b804e890 con 0x7f80b80b6210 2023-12-01T01:55:26.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.350+0000 7f80aeffd640 1 -- 172.21.15.115:0/2757739696 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f80a802f050 con 0x7f80b80b6210 2023-12-01T01:55:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.351+0000 7f80aeffd640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f809806a580 0x7f809806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.351+0000 7f80aeffd640 1 -- 172.21.15.115:0/2757739696 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80a80aac00 con 0x7f80b80b6210 2023-12-01T01:55:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.351+0000 7f80affff640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f809806a580 0x7f809806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:26.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.352+0000 7f80affff640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f809806a580 0x7f809806ca40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80b80b37e0 tx=0x7f80a0006bf0 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.355+0000 7f80aeffd640 1 -- 172.21.15.115:0/2757739696 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f80a80b1080 con 0x7f80b80b6210 2023-12-01T01:55:26.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:26.578+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2757739696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f80b80b5dd0 con 0x7f80b80b6210 2023-12-01T01:55:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.267+0000 7f80aeffd640 1 -- 172.21.15.115:0/2757739696 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) v1 ==== 118+0+0 (secure 0 0 0) 0x7f80a80350b0 con 0x7f80b80b6210 2023-12-01T01:55:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2757739696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f809806a580 msgr2=0x7f809806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f809806a580 0x7f809806ca40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80b80b37e0 tx=0x7f80a0006bf0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2757739696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b80b6210 msgr2=0x7f80b8151e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b80b6210 0x7f80b8151e60 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f80a8000c00 tx=0x7f80a80046c0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2757739696 shutdown_connections 2023-12-01T01:55:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f809806a580 0x7f809806ca40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80b80b7e70 0x7f80b815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80b815cfa0 0x7f80b80c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 --2- 172.21.15.115:0/2757739696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80b80b6210 0x7f80b8151e60 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2757739696 >> 172.21.15.115:0/2757739696 conn(0x7f80b806a0b0 msgr2=0x7f80b80b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2757739696 shutdown_connections 2023-12-01T01:55:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.271+0000 7f80bcfd9640 1 -- 172.21.15.115:0/2757739696 wait complete. 2023-12-01T01:55:27.293 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T01:55:27.293 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T01:55:27.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.516+0000 7fa828981640 1 Processor -- start 2023-12-01T01:55:27.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.517+0000 7fa828981640 1 -- start start 2023-12-01T01:55:27.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.518+0000 7fa828981640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415cd90 0x7fa8240cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:27.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.518+0000 7fa828981640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8240ce5e0 con 0x7fa8240cfd60 2023-12-01T01:55:27.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.518+0000 7fa828981640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8240ce300 con 0x7fa8240b5fa0 2023-12-01T01:55:27.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.518+0000 7fa828981640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8240d0ec0 con 0x7fa82415cd90 2023-12-01T01:55:27.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.518+0000 7fa822d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa8240cfd60 0x7fa8240d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54760/0 (socket says 172.21.15.115:54760) 2023-12-01T01:55:27.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.518+0000 7fa822d9d640 1 -- 172.21.15.115:0/1639011051 learned_addr learned my addr 172.21.15.115:0/1639011051 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:27.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82259c640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415cd90 0x7fa8240cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:27.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831834170 0 0) 0x7fa81c001490 con 0x7fa8240cfd60 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa810003520 con 0x7fa8240cfd60 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2891323618 0 0) 0x7fa810003520 con 0x7fa8240cfd60 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415cd90 msgr2=0x7fa8240cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415cd90 0x7fa8240cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 >> v1:172.21.15.142:6789/0 conn(0x7fa8240b5fa0 legacy=0x7fa8240cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8240d10a0 con 0x7fa8240cfd60 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa81c001b80 con 0x7fa8240cfd60 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 >> v1:172.21.15.115:6789/0 conn(0x7fa8240cfd60 legacy=0x7fa8240d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8100039a0 0x7fa810003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100047c0 0x7fa810004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415cd90 0x7fa810005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.519+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8240d0ec0 con 0x7fa8100047c0 2023-12-01T01:55:27.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.520+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81000ba60 con 0x7fa8100039a0 2023-12-01T01:55:27.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.520+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81000bc20 con 0x7fa82415cd90 2023-12-01T01:55:27.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.520+0000 7fa821d9b640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100047c0 0x7fa810004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:27.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.520+0000 7fa82259c640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8100039a0 0x7fa810003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:27.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.520+0000 7fa821d9b640 1 -- 172.21.15.115:0/1639011051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415cd90 msgr2=0x7fa810005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:27.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.520+0000 7fa821d9b640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415cd90 0x7fa810005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.520+0000 7fa821d9b640 1 -- 172.21.15.115:0/1639011051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8100039a0 msgr2=0x7fa810003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:27.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.520+0000 7fa821d9b640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8100039a0 0x7fa810003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.520+0000 7fa821d9b640 1 -- 172.21.15.115:0/1639011051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa810003520 con 0x7fa8100047c0 2023-12-01T01:55:27.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.521+0000 7fa821d9b640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100047c0 0x7fa810004c40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa80c001720 tx=0x7fa8100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0decb7da28bd3fc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:27.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.521+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0052d0 con 0x7fa8100047c0 2023-12-01T01:55:27.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.521+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c02e050 con 0x7fa8100047c0 2023-12-01T01:55:27.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.521+0000 7fa82159a640 1 -- 172.21.15.115:0/1639011051 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0035f0 con 0x7fa8100047c0 2023-12-01T01:55:27.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.522+0000 7fa828981640 1 -- 172.21.15.115:0/1639011051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100047c0 msgr2=0x7fa810004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:27.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.522+0000 7fa828981640 1 --2- 172.21.15.115:0/1639011051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100047c0 0x7fa810004c40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa80c001720 tx=0x7fa8100057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.522+0000 7fa822d9d640 1 -- 172.21.15.115:0/1639011051 reap_dead start 2023-12-01T01:55:27.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.522+0000 7fa828981640 1 -- 172.21.15.115:0/1639011051 shutdown_connections 2023-12-01T01:55:27.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.522+0000 7fa828981640 1 -- 172.21.15.115:0/1639011051 >> 172.21.15.115:0/1639011051 conn(0x7fa824065960 msgr2=0x7fa824065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:27.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.522+0000 7fa828981640 1 -- 172.21.15.115:0/1639011051 shutdown_connections 2023-12-01T01:55:27.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.522+0000 7fa828981640 1 -- 172.21.15.115:0/1639011051 wait complete. 2023-12-01T01:55:27.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.524+0000 7fa828981640 1 Processor -- start 2023-12-01T01:55:27.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.524+0000 7fa828981640 1 -- start start 2023-12-01T01:55:27.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.525+0000 7fa828981640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8100047c0 0x7fa82415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:27.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.525+0000 7fa828981640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8241f5bb0 0x7fa8240b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:27.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.525+0000 7fa828981640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415d4d0 0x7fa82415d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:27.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.525+0000 7fa828981640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8240ac830 con 0x7fa8241f5bb0 2023-12-01T01:55:27.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.525+0000 7fa828981640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8240b55e0 con 0x7fa8100047c0 2023-12-01T01:55:27.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.525+0000 7fa828981640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8241f6b80 con 0x7fa82415d4d0 2023-12-01T01:55:27.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.525+0000 7fa821d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8241f5bb0 0x7fa8240b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:27.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.525+0000 7fa821d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8241f5bb0 0x7fa8240b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33258/0 (socket says 172.21.15.115:33258) 2023-12-01T01:55:27.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.525+0000 7fa821d9b640 1 -- 172.21.15.115:0/1633943850 learned_addr learned my addr 172.21.15.115:0/1633943850 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:27.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.526+0000 7fa82259c640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8100047c0 0x7fa82415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:27.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.526+0000 7fa821d9b640 1 -- 172.21.15.115:0/1633943850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415d4d0 msgr2=0x7fa82415d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:27.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.526+0000 7fa822d9d640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415d4d0 0x7fa82415d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:27.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.526+0000 7fa821d9b640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415d4d0 0x7fa82415d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.526+0000 7fa821d9b640 1 -- 172.21.15.115:0/1633943850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8100047c0 msgr2=0x7fa82415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:27.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.526+0000 7fa821d9b640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8100047c0 0x7fa82415a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:27.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.526+0000 7fa821d9b640 1 -- 172.21.15.115:0/1633943850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82415dea0 con 0x7fa8241f5bb0 2023-12-01T01:55:27.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.526+0000 7fa82259c640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8100047c0 0x7fa82415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:27.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.526+0000 7fa821d9b640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8241f5bb0 0x7fa8240b50a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005040 tx=0x7fa80c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:27.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.526+0000 7fa81affd640 1 -- 172.21.15.115:0/1633943850 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0076c0 con 0x7fa8241f5bb0 2023-12-01T01:55:27.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.527+0000 7fa81affd640 1 -- 172.21.15.115:0/1633943850 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c0069e0 con 0x7fa8241f5bb0 2023-12-01T01:55:27.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.527+0000 7fa828981640 1 -- 172.21.15.115:0/1633943850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8241f6030 con 0x7fa8241f5bb0 2023-12-01T01:55:27.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.527+0000 7fa81affd640 1 -- 172.21.15.115:0/1633943850 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c006d50 con 0x7fa8241f5bb0 2023-12-01T01:55:27.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.527+0000 7fa828981640 1 -- 172.21.15.115:0/1633943850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8241f65c0 con 0x7fa8241f5bb0 2023-12-01T01:55:27.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.528+0000 7fa828981640 1 -- 172.21.15.115:0/1633943850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82404e890 con 0x7fa8241f5bb0 2023-12-01T01:55:27.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.529+0000 7fa81affd640 1 -- 172.21.15.115:0/1633943850 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa80c0035f0 con 0x7fa8241f5bb0 2023-12-01T01:55:27.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.530+0000 7fa81affd640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa80406a550 0x7fa80406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:27.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.530+0000 7fa81affd640 1 -- 172.21.15.115:0/1633943850 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa80c0aa8d0 con 0x7fa8241f5bb0 2023-12-01T01:55:27.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.530+0000 7fa82259c640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa80406a550 0x7fa80406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:27.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.531+0000 7fa82259c640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa80406a550 0x7fa80406ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa814009a00 tx=0x7fa814008040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:27.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.534+0000 7fa81affd640 1 -- 172.21.15.115:0/1633943850 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa80c075e50 con 0x7fa8241f5bb0 2023-12-01T01:55:27.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:27.757+0000 7fa828981640 1 -- 172.21.15.115:0/1633943850 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa8240d0a70 con 0x7fa8241f5bb0 2023-12-01T01:55:28.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.272+0000 7fa81affd640 1 -- 172.21.15.115:0/1633943850 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v63) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa80c04b0c0 con 0x7fa8241f5bb0 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.275+0000 7fa828981640 1 -- 172.21.15.115:0/1633943850 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa80406a550 msgr2=0x7fa80406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.275+0000 7fa828981640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa80406a550 0x7fa80406ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa814009a00 tx=0x7fa814008040 comp rx=0 tx=0).stop 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.275+0000 7fa828981640 1 -- 172.21.15.115:0/1633943850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8241f5bb0 msgr2=0x7fa8240b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.275+0000 7fa828981640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8241f5bb0 0x7fa8240b50a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005040 tx=0x7fa80c001120 comp rx=0 tx=0).stop 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.275+0000 7fa828981640 1 -- 172.21.15.115:0/1633943850 shutdown_connections 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.275+0000 7fa828981640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa80406a550 0x7fa80406ca10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.275+0000 7fa828981640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa82415d4d0 0x7fa82415d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.275+0000 7fa828981640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8241f5bb0 0x7fa8240b50a0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.275+0000 7fa828981640 1 --2- 172.21.15.115:0/1633943850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8100047c0 0x7fa82415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.275+0000 7fa828981640 1 -- 172.21.15.115:0/1633943850 >> 172.21.15.115:0/1633943850 conn(0x7fa824065960 msgr2=0x7fa8240b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:28.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.276+0000 7fa828981640 1 -- 172.21.15.115:0/1633943850 shutdown_connections 2023-12-01T01:55:28.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.276+0000 7fa828981640 1 -- 172.21.15.115:0/1633943850 wait complete. 2023-12-01T01:55:28.296 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T01:55:28.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:55:28.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.510+0000 7f716d815640 1 Processor -- start 2023-12-01T01:55:28.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.510+0000 7f716d815640 1 -- start start 2023-12-01T01:55:28.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.511+0000 7f716d815640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f716815e2f0 0x7f71680ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:28.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.511+0000 7f716d815640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 0x7f71680caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:28.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.511+0000 7f716d815640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f71680adf10 0x7f71680c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:28.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.511+0000 7f716d815640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71680add10 con 0x7f71680adf10 2023-12-01T01:55:28.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.511+0000 7f716d815640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71680c5760 con 0x7f716815e2f0 2023-12-01T01:55:28.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.511+0000 7f716d815640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716815e6d0 con 0x7f716806f2f0 2023-12-01T01:55:28.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f71677fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f71680adf10 0x7f71680c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:28.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f71677fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f71680adf10 0x7f71680c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55780/0 (socket says 172.21.15.115:55780) 2023-12-01T01:55:28.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f71677fe640 1 -- 172.21.15.115:0/2050230207 learned_addr learned my addr 172.21.15.115:0/2050230207 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:28.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f71667fc640 1 --2- 172.21.15.115:0/2050230207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 0x7f71680caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:28.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f7166ffd640 1 --2- 172.21.15.115:0/2050230207 >> v2:172.21.15.142:3300/0 conn(0x7f716815e2f0 0x7f71680ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:28.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f71677fe640 1 -- 172.21.15.115:0/2050230207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 msgr2=0x7f71680caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:28.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f71677fe640 1 --2- 172.21.15.115:0/2050230207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 0x7f71680caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f71677fe640 1 -- 172.21.15.115:0/2050230207 >> v2:172.21.15.142:3300/0 conn(0x7f716815e2f0 msgr2=0x7f71680ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:28.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f71677fe640 1 --2- 172.21.15.115:0/2050230207 >> v2:172.21.15.142:3300/0 conn(0x7f716815e2f0 0x7f71680ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f71677fe640 1 -- 172.21.15.115:0/2050230207 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71680cf7b0 con 0x7f71680adf10 2023-12-01T01:55:28.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f71667fc640 1 --2- 172.21.15.115:0/2050230207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 0x7f71680caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:28.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.512+0000 7f7166ffd640 1 --2- 172.21.15.115:0/2050230207 >> v2:172.21.15.142:3300/0 conn(0x7f716815e2f0 0x7f71680ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.513+0000 7f71677fe640 1 --2- 172.21.15.115:0/2050230207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 0x7f71680c8ed0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f716806ef60 tx=0x7f716002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1d56f679fbf647 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.514+0000 7f7165ffb640 1 -- 172.21.15.115:0/2050230207 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160031740 con 0x7f71680adf10 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.514+0000 7f7165ffb640 1 -- 172.21.15.115:0/2050230207 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71600318a0 con 0x7f71680adf10 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.514+0000 7f7165ffb640 1 -- 172.21.15.115:0/2050230207 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160031c30 con 0x7f71680adf10 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.514+0000 7f716d815640 1 -- 172.21.15.115:0/2050230207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 msgr2=0x7f71680c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.514+0000 7f716d815640 1 --2- 172.21.15.115:0/2050230207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 0x7f71680c8ed0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f716806ef60 tx=0x7f716002b7e0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.514+0000 7f716d815640 1 -- 172.21.15.115:0/2050230207 shutdown_connections 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.514+0000 7f716d815640 1 --2- 172.21.15.115:0/2050230207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 0x7f71680c8ed0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.514+0000 7f716d815640 1 --2- 172.21.15.115:0/2050230207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 0x7f71680caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.514+0000 7f716d815640 1 --2- 172.21.15.115:0/2050230207 >> v2:172.21.15.142:3300/0 conn(0x7f716815e2f0 0x7f71680ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.514+0000 7f716d815640 1 -- 172.21.15.115:0/2050230207 >> 172.21.15.115:0/2050230207 conn(0x7f71680ce880 msgr2=0x7f71680cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:28.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.515+0000 7f716d815640 1 -- 172.21.15.115:0/2050230207 shutdown_connections 2023-12-01T01:55:28.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.515+0000 7f716d815640 1 -- 172.21.15.115:0/2050230207 wait complete. 2023-12-01T01:55:28.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.517+0000 7f716d815640 1 Processor -- start 2023-12-01T01:55:28.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.517+0000 7f716d815640 1 -- start start 2023-12-01T01:55:28.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.518+0000 7f716d815640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 0x7f71680c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:28.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.518+0000 7f716d815640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 0x7f71681f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:28.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.518+0000 7f716d815640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716815e2f0 0x7f71680c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:28.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.518+0000 7f716d815640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71680c7b70 con 0x7f71680adf10 2023-12-01T01:55:28.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.518+0000 7f716d815640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71680c3d70 con 0x7f716815e2f0 2023-12-01T01:55:28.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.518+0000 7f716d815640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71680c4050 con 0x7f716806f2f0 2023-12-01T01:55:28.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.518+0000 7f71667fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 0x7f71681f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:28.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.518+0000 7f71667fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 0x7f71681f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55792/0 (socket says 172.21.15.115:55792) 2023-12-01T01:55:28.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.518+0000 7f71667fc640 1 -- 172.21.15.115:0/4203471071 learned_addr learned my addr 172.21.15.115:0/4203471071 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:28.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f71677fe640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716815e2f0 0x7f71680c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:28.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f7166ffd640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 0x7f71680c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:28.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f71667fc640 1 -- 172.21.15.115:0/4203471071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 msgr2=0x7f71680c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:28.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f71667fc640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 0x7f71680c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f71667fc640 1 -- 172.21.15.115:0/4203471071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716815e2f0 msgr2=0x7f71680c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:28.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f71667fc640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716815e2f0 0x7f71680c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f71667fc640 1 -- 172.21.15.115:0/4203471071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71600094d0 con 0x7f71680adf10 2023-12-01T01:55:28.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f71677fe640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716815e2f0 0x7f71680c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:28.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f7166ffd640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 0x7f71680c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:28.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f71667fc640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 0x7f71681f16a0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f715c00ce30 tx=0x7f715c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:28.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f714ffff640 1 -- 172.21.15.115:0/4203471071 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c00f680 con 0x7f71680adf10 2023-12-01T01:55:28.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f714ffff640 1 -- 172.21.15.115:0/4203471071 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c00f7e0 con 0x7f71680adf10 2023-12-01T01:55:28.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f716d815640 1 -- 172.21.15.115:0/4203471071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71680c4330 con 0x7f71680adf10 2023-12-01T01:55:28.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.519+0000 7f714ffff640 1 -- 172.21.15.115:0/4203471071 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c00fb90 con 0x7f71680adf10 2023-12-01T01:55:28.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.520+0000 7f716d815640 1 -- 172.21.15.115:0/4203471071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71680c3940 con 0x7f71680adf10 2023-12-01T01:55:28.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.521+0000 7f716d815640 1 -- 172.21.15.115:0/4203471071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716806fda0 con 0x7f71680adf10 2023-12-01T01:55:28.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.522+0000 7f714ffff640 1 -- 172.21.15.115:0/4203471071 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f715c00b040 con 0x7f71680adf10 2023-12-01T01:55:28.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.522+0000 7f714ffff640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f715406a550 0x7f715406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:28.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.523+0000 7f714ffff640 1 -- 172.21.15.115:0/4203471071 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f715c0895d0 con 0x7f71680adf10 2023-12-01T01:55:28.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.523+0000 7f7166ffd640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f715406a550 0x7f715406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:28.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.523+0000 7f7166ffd640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f715406a550 0x7f715406ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7158006fd0 tx=0x7f7158008040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:28.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.526+0000 7f714ffff640 1 -- 172.21.15.115:0/4203471071 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f715c0549c0 con 0x7f71680adf10 2023-12-01T01:55:28.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.753+0000 7f716d815640 1 -- 172.21.15.115:0/4203471071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71680c3b30 con 0x7f71680adf10 2023-12-01T01:55:28.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.754+0000 7f714ffff640 1 -- 172.21.15.115:0/4203471071 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f71680c3b30 con 0x7f71680adf10 2023-12-01T01:55:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:28.755 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3556763477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3556763477},{"type":"v1","addr":"172.21.15.142:6833","nonce":3556763477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3661095418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3661095418},{"type":"v1","addr":"172.21.15.142:6835","nonce":3661095418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3388585128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3388585128},{"type":"v1","addr":"172.21.15.115:6835","nonce":3388585128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:55:25.253838+0000","modified":"2023-12-01T01:55:28.264944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3894452047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3894452047},{"type":"v1","addr":"172.21.15.115:6837","nonce":3894452047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-01T01:55:28.755 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 63 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.756+0000 7f716d815640 1 -- 172.21.15.115:0/4203471071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f715406a550 msgr2=0x7f715406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.756+0000 7f716d815640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f715406a550 0x7f715406ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7158006fd0 tx=0x7f7158008040 comp rx=0 tx=0).stop 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.756+0000 7f716d815640 1 -- 172.21.15.115:0/4203471071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 msgr2=0x7f71681f16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.756+0000 7f716d815640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 0x7f71681f16a0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f715c00ce30 tx=0x7f715c00a650 comp rx=0 tx=0).stop 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.756+0000 7f716d815640 1 -- 172.21.15.115:0/4203471071 shutdown_connections 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.756+0000 7f716d815640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f715406a550 0x7f715406ca10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.756+0000 7f716d815640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716815e2f0 0x7f71680c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.756+0000 7f716d815640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71680adf10 0x7f71681f16a0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.756+0000 7f716d815640 1 --2- 172.21.15.115:0/4203471071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716806f2f0 0x7f71680c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.756+0000 7f716d815640 1 -- 172.21.15.115:0/4203471071 >> 172.21.15.115:0/4203471071 conn(0x7f71680ce880 msgr2=0x7f716815ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.757+0000 7f716d815640 1 -- 172.21.15.115:0/4203471071 shutdown_connections 2023-12-01T01:55:28.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.757+0000 7f716d815640 1 -- 172.21.15.115:0/4203471071 wait complete. 2023-12-01T01:55:28.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:55:28.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.990+0000 7f14ea42c640 1 Processor -- start 2023-12-01T01:55:28.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.991+0000 7f14ea42c640 1 -- start start 2023-12-01T01:55:28.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.992+0000 7f14ea42c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f14e40ad2b0 0x7f14e40701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:28.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.992+0000 7f14ea42c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e415d600 con 0x7f14e415d220 2023-12-01T01:55:28.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.992+0000 7f14ea42c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e406c4b0 con 0x7f14e40ccc80 2023-12-01T01:55:28.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.992+0000 7f14ea42c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e406cc10 con 0x7f14e40ad2b0 2023-12-01T01:55:28.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.992+0000 7f14e89c9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f14e40ad2b0 0x7f14e40701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:28.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.992+0000 7f14e3fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f14e415d220 0x7f14e40cc570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36422/0 (socket says 172.21.15.115:36422) 2023-12-01T01:55:28.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.992+0000 7f14e3fff640 1 -- 172.21.15.115:0/132133507 learned_addr learned my addr 172.21.15.115:0/132133507 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:28.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.993+0000 7f14e37fe640 1 -- 172.21.15.115:0/132133507 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897138988 0 0) 0x7f14d4001110 con 0x7f14e415d220 2023-12-01T01:55:28.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.993+0000 7f14e89c9640 1 -- 172.21.15.115:0/132133507 >> v1:172.21.15.142:6789/0 conn(0x7f14e40ccc80 legacy=0x7f14e406fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:28.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.993+0000 7f14e89c9640 1 -- 172.21.15.115:0/132133507 >> v1:172.21.15.115:6789/0 conn(0x7f14e415d220 legacy=0x7f14e40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:28.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.993+0000 7f14e89c9640 1 -- 172.21.15.115:0/132133507 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e406f2f0 con 0x7f14e40ad2b0 2023-12-01T01:55:28.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.994+0000 7f14e89c9640 1 --2- 172.21.15.115:0/132133507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ad2b0 0x7f14e40701c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009d90 tx=0x7f14dc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8e7c0c6aae164f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:28.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.994+0000 7f14e37fe640 1 -- 172.21.15.115:0/132133507 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc02cbd0 con 0x7f14e40ad2b0 2023-12-01T01:55:28.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.994+0000 7f14e37fe640 1 -- 172.21.15.115:0/132133507 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc02cd30 con 0x7f14e40ad2b0 2023-12-01T01:55:28.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.995+0000 7f14e37fe640 1 -- 172.21.15.115:0/132133507 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc02cea0 con 0x7f14e40ad2b0 2023-12-01T01:55:28.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.995+0000 7f14ea42c640 1 -- 172.21.15.115:0/132133507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ad2b0 msgr2=0x7f14e40701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:28.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.995+0000 7f14ea42c640 1 --2- 172.21.15.115:0/132133507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ad2b0 0x7f14e40701c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009d90 tx=0x7f14dc02be20 comp rx=0 tx=0).stop 2023-12-01T01:55:28.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.995+0000 7f14ea42c640 1 -- 172.21.15.115:0/132133507 shutdown_connections 2023-12-01T01:55:28.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.995+0000 7f14ea42c640 1 --2- 172.21.15.115:0/132133507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ad2b0 0x7f14e40701c0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:28.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.995+0000 7f14ea42c640 1 -- 172.21.15.115:0/132133507 >> 172.21.15.115:0/132133507 conn(0x7f14e40ce880 msgr2=0x7f14e40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:28.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.995+0000 7f14ea42c640 1 -- 172.21.15.115:0/132133507 shutdown_connections 2023-12-01T01:55:28.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.995+0000 7f14ea42c640 1 -- 172.21.15.115:0/132133507 wait complete. 2023-12-01T01:55:28.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.998+0000 7f14ea42c640 1 Processor -- start 2023-12-01T01:55:28.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.998+0000 7f14ea42c640 1 -- start start 2023-12-01T01:55:29.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.999+0000 7f14ea42c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14e40ad2b0 0x7f14e4157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.999+0000 7f14ea42c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ccc80 0x7f14e40b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.999+0000 7f14ea42c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14e415d220 0x7f14e40c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.999+0000 7f14ea42c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40b15b0 con 0x7f14e40ccc80 2023-12-01T01:55:29.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.999+0000 7f14ea42c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40b4530 con 0x7f14e40ad2b0 2023-12-01T01:55:29.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.999+0000 7f14ea42c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40b3c90 con 0x7f14e415d220 2023-12-01T01:55:29.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.999+0000 7f14dbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ccc80 0x7f14e40b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.999+0000 7f14dbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ccc80 0x7f14e40b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55816/0 (socket says 172.21.15.115:55816) 2023-12-01T01:55:29.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:28.999+0000 7f14dbfff640 1 -- 172.21.15.115:0/1231456164 learned_addr learned my addr 172.21.15.115:0/1231456164 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:29.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.000+0000 7f14e89c9640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14e415d220 0x7f14e40c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.000+0000 7f14e3fff640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14e40ad2b0 0x7f14e4157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.000+0000 7f14dbfff640 1 -- 172.21.15.115:0/1231456164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14e415d220 msgr2=0x7f14e40c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.000+0000 7f14dbfff640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14e415d220 0x7f14e40c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.000+0000 7f14dbfff640 1 -- 172.21.15.115:0/1231456164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14e40ad2b0 msgr2=0x7f14e4157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.000+0000 7f14dbfff640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14e40ad2b0 0x7f14e4157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.000+0000 7f14dbfff640 1 -- 172.21.15.115:0/1231456164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc009a70 con 0x7f14e40ccc80 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.000+0000 7f14e89c9640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14e415d220 0x7f14e40c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.000+0000 7f14dbfff640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ccc80 0x7f14e40b3320 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00b3f0 tx=0x7f14cc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.000+0000 7f14e17fa640 1 -- 172.21.15.115:0/1231456164 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc011830 con 0x7f14e40ccc80 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.001+0000 7f14e17fa640 1 -- 172.21.15.115:0/1231456164 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc011990 con 0x7f14e40ccc80 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.001+0000 7f14e17fa640 1 -- 172.21.15.115:0/1231456164 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc011d40 con 0x7f14e40ccc80 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.001+0000 7f14ea42c640 1 -- 172.21.15.115:0/1231456164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e40b3f70 con 0x7f14e40ccc80 2023-12-01T01:55:29.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.001+0000 7f14ea42c640 1 -- 172.21.15.115:0/1231456164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e41583f0 con 0x7f14e40ccc80 2023-12-01T01:55:29.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.002+0000 7f14ea42c640 1 -- 172.21.15.115:0/1231456164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e404e890 con 0x7f14e40ccc80 2023-12-01T01:55:29.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.003+0000 7f14e17fa640 1 -- 172.21.15.115:0/1231456164 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f14cc01a400 con 0x7f14e40ccc80 2023-12-01T01:55:29.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.004+0000 7f14e17fa640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f14c406a580 0x7f14c406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.004+0000 7f14e17fa640 1 -- 172.21.15.115:0/1231456164 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14cc088900 con 0x7f14e40ccc80 2023-12-01T01:55:29.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.004+0000 7f14e3fff640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f14c406a580 0x7f14c406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.005+0000 7f14e3fff640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f14c406a580 0x7f14c406ca40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f14d4000f60 tx=0x7f14d4009040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:29.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.008+0000 7f14e17fa640 1 -- 172.21.15.115:0/1231456164 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f14cc053dd0 con 0x7f14e40ccc80 2023-12-01T01:55:29.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.201+0000 7f14ea42c640 1 -- 172.21.15.115:0/1231456164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14e4068160 con 0x7f14e40ccc80 2023-12-01T01:55:29.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.203+0000 7f14e17fa640 1 -- 172.21.15.115:0/1231456164 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16163 (secure 0 0 0) 0x7f14cc015090 con 0x7f14e40ccc80 2023-12-01T01:55:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":141,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T01:55:26.257840+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:55:22.818353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-01T01:55:24.272936+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:55:29.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 -- 172.21.15.115:0/1231456164 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f14c406a580 msgr2=0x7f14c406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f14c406a580 0x7f14c406ca40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f14d4000f60 tx=0x7f14d4009040 comp rx=0 tx=0).stop 2023-12-01T01:55:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 -- 172.21.15.115:0/1231456164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ccc80 msgr2=0x7f14e40b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ccc80 0x7f14e40b3320 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00b3f0 tx=0x7f14cc00b8b0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 -- 172.21.15.115:0/1231456164 shutdown_connections 2023-12-01T01:55:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f14c406a580 0x7f14c406ca40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14e415d220 0x7f14e40c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14e40ccc80 0x7f14e40b3320 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 --2- 172.21.15.115:0/1231456164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14e40ad2b0 0x7f14e4157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 -- 172.21.15.115:0/1231456164 >> 172.21.15.115:0/1231456164 conn(0x7f14e40ce880 msgr2=0x7f14e40c4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 -- 172.21.15.115:0/1231456164 shutdown_connections 2023-12-01T01:55:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.206+0000 7f14ea42c640 1 -- 172.21.15.115:0/1231456164 wait complete. 2023-12-01T01:55:29.226 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T01:55:29.226 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:55:29.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.439+0000 7fc5f63e8640 1 Processor -- start 2023-12-01T01:55:29.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.439+0000 7fc5f63e8640 1 -- start start 2023-12-01T01:55:29.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.441+0000 7fc5f63e8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc5f006b270 0x7fc5f00ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.441+0000 7fc5f63e8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f00ce120 con 0x7fc5f01597d0 2023-12-01T01:55:29.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.441+0000 7fc5f63e8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f006b700 con 0x7fc5f00cdcc0 2023-12-01T01:55:29.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.441+0000 7fc5f63e8640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f006b9c0 con 0x7fc5f006b270 2023-12-01T01:55:29.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.441+0000 7fc5f53e6640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc5f01597d0 0x7fc5f015db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36438/0 (socket says 172.21.15.115:36438) 2023-12-01T01:55:29.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.441+0000 7fc5f53e6640 1 -- 172.21.15.115:0/3427444919 learned_addr learned my addr 172.21.15.115:0/3427444919 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:29.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.441+0000 7fc5f5be7640 1 --2- 172.21.15.115:0/3427444919 >> v2:172.21.15.115:3300/0 conn(0x7fc5f006b270 0x7fc5f00ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.442+0000 7fc5e7fff640 1 -- 172.21.15.115:0/3427444919 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782918366 0 0) 0x7fc5f00ce120 con 0x7fc5f01597d0 2023-12-01T01:55:29.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.442+0000 7fc5e7fff640 1 -- 172.21.15.115:0/3427444919 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5dc005730 con 0x7fc5f01597d0 2023-12-01T01:55:29.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.442+0000 7fc5f5be7640 1 -- 172.21.15.115:0/3427444919 >> v1:172.21.15.142:6789/0 conn(0x7fc5f00cdcc0 legacy=0x7fc5f00adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:29.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.442+0000 7fc5f5be7640 1 -- 172.21.15.115:0/3427444919 >> v1:172.21.15.115:6789/0 conn(0x7fc5f01597d0 legacy=0x7fc5f015db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.442+0000 7fc5f5be7640 1 -- 172.21.15.115:0/3427444919 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f006bca0 con 0x7fc5f006b270 2023-12-01T01:55:29.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.443+0000 7fc5f5be7640 1 --2- 172.21.15.115:0/3427444919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 0x7fc5f00ca350 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009d90 tx=0x7fc5e802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bff33cdaf47c1eb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:29.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.443+0000 7fc5e7fff640 1 -- 172.21.15.115:0/3427444919 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e802cbd0 con 0x7fc5f006b270 2023-12-01T01:55:29.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.443+0000 7fc5e7fff640 1 -- 172.21.15.115:0/3427444919 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e802cd30 con 0x7fc5f006b270 2023-12-01T01:55:29.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.443+0000 7fc5e7fff640 1 -- 172.21.15.115:0/3427444919 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e802cea0 con 0x7fc5f006b270 2023-12-01T01:55:29.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.444+0000 7fc5f63e8640 1 -- 172.21.15.115:0/3427444919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 msgr2=0x7fc5f00ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.444+0000 7fc5f63e8640 1 --2- 172.21.15.115:0/3427444919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 0x7fc5f00ca350 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009d90 tx=0x7fc5e802be20 comp rx=0 tx=0).stop 2023-12-01T01:55:29.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.444+0000 7fc5f63e8640 1 -- 172.21.15.115:0/3427444919 shutdown_connections 2023-12-01T01:55:29.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.444+0000 7fc5f63e8640 1 --2- 172.21.15.115:0/3427444919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 0x7fc5f00ca350 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.444+0000 7fc5f63e8640 1 -- 172.21.15.115:0/3427444919 >> 172.21.15.115:0/3427444919 conn(0x7fc5f00ce640 msgr2=0x7fc5f00cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:29.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.444+0000 7fc5f63e8640 1 -- 172.21.15.115:0/3427444919 shutdown_connections 2023-12-01T01:55:29.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.444+0000 7fc5f63e8640 1 -- 172.21.15.115:0/3427444919 wait complete. 2023-12-01T01:55:29.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.446+0000 7fc5f63e8640 1 Processor -- start 2023-12-01T01:55:29.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.446+0000 7fc5f63e8640 1 -- start start 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f63e8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 0x7fc5f00c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f63e8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5f00cdcc0 0x7fc5f01f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f63e8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5f01597d0 0x7fc5f01efab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f63e8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00c76a0 con 0x7fc5f006b270 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f63e8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00c6910 con 0x7fc5f01597d0 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f63e8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01f32a0 con 0x7fc5f00cdcc0 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f53e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 0x7fc5f00c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f53e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 0x7fc5f00c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55834/0 (socket says 172.21.15.115:55834) 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f53e6640 1 -- 172.21.15.115:0/1703153453 learned_addr learned my addr 172.21.15.115:0/1703153453 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f4be5640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5f00cdcc0 0x7fc5f01f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f5be7640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5f01597d0 0x7fc5f01efab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f53e6640 1 -- 172.21.15.115:0/1703153453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5f00cdcc0 msgr2=0x7fc5f01f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f53e6640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5f00cdcc0 0x7fc5f01f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f53e6640 1 -- 172.21.15.115:0/1703153453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5f01597d0 msgr2=0x7fc5f01efab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f53e6640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5f01597d0 0x7fc5f01efab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.447+0000 7fc5f53e6640 1 -- 172.21.15.115:0/1703153453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e8009a70 con 0x7fc5f006b270 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.448+0000 7fc5f4be5640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5f00cdcc0 0x7fc5f01f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.448+0000 7fc5f53e6640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 0x7fc5f00c63d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00c410 tx=0x7fc5ec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.448+0000 7fc5e5ffb640 1 -- 172.21.15.115:0/1703153453 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec005ce0 con 0x7fc5f006b270 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.448+0000 7fc5e5ffb640 1 -- 172.21.15.115:0/1703153453 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec005e40 con 0x7fc5f006b270 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.448+0000 7fc5e5ffb640 1 -- 172.21.15.115:0/1703153453 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0072f0 con 0x7fc5f006b270 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.448+0000 7fc5f63e8640 1 -- 172.21.15.115:0/1703153453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f01efff0 con 0x7fc5f006b270 2023-12-01T01:55:29.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.448+0000 7fc5f63e8640 1 -- 172.21.15.115:0/1703153453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f00c6d00 con 0x7fc5f006b270 2023-12-01T01:55:29.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.449+0000 7fc5f63e8640 1 -- 172.21.15.115:0/1703153453 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f00673a0 con 0x7fc5f006b270 2023-12-01T01:55:29.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.450+0000 7fc5e5ffb640 1 -- 172.21.15.115:0/1703153453 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc5ec007510 con 0x7fc5f006b270 2023-12-01T01:55:29.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.452+0000 7fc5e5ffb640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5d406a550 0x7fc5d406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.452+0000 7fc5e5ffb640 1 -- 172.21.15.115:0/1703153453 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5ec087f80 con 0x7fc5f006b270 2023-12-01T01:55:29.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.452+0000 7fc5f4be5640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5d406a550 0x7fc5d406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.453+0000 7fc5f4be5640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5d406a550 0x7fc5d406ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00ae350 tx=0x7fc5e0008040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:29.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.456+0000 7fc5e5ffb640 1 -- 172.21.15.115:0/1703153453 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc5ec053510 con 0x7fc5f006b270 2023-12-01T01:55:29.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.643+0000 7fc5f63e8640 1 -- 172.21.15.115:0/1703153453 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5f015d610 con 0x7fc5d406a550 2023-12-01T01:55:29.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.670+0000 7fc5e5ffb640 1 -- 172.21.15.115:0/1703153453 <== mgr.8247 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276327 (secure 0 0 0) 0x7fc5e000ad80 con 0x7fc5d406a550 2023-12-01T01:55:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:29.676 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:55:29.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 -- 172.21.15.115:0/1703153453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5d406a550 msgr2=0x7fc5d406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5d406a550 0x7fc5d406ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00ae350 tx=0x7fc5e0008040 comp rx=0 tx=0).stop 2023-12-01T01:55:29.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 -- 172.21.15.115:0/1703153453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 msgr2=0x7fc5f00c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 0x7fc5f00c63d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00c410 tx=0x7fc5ec00c8d0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 -- 172.21.15.115:0/1703153453 shutdown_connections 2023-12-01T01:55:29.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc5d406a550 0x7fc5d406ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5f00cdcc0 0x7fc5f01f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5f01597d0 0x7fc5f01efab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 --2- 172.21.15.115:0/1703153453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5f006b270 0x7fc5f00c63d0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 -- 172.21.15.115:0/1703153453 >> 172.21.15.115:0/1703153453 conn(0x7fc5f00ce640 msgr2=0x7fc5f00ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:29.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.675+0000 7fc5f63e8640 1 -- 172.21.15.115:0/1703153453 shutdown_connections 2023-12-01T01:55:29.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.676+0000 7fc5f63e8640 1 -- 172.21.15.115:0/1703153453 wait complete. 2023-12-01T01:55:29.697 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T01:55:27.936714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":331700,"kb_used_data":14452,"kb_used_omap":0,"kb_used_meta":317184,"kb_avail":754643020,"statfs":{"total":773094113280,"available":772754452480,"internally_reserved":0,"allocated":14798848,"data_stored":7683635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":324796416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.177327"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.840339+0000","last_change":"2023-12-01T01:55:23.840339+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.840339+0000","last_undegraded":"2023-12-01T01:55:23.840339+0000","last_fullsized":"2023-12-01T01:55:23.840339+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:49:21.973740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.765101+0000","last_change":"2023-12-01T01:55:23.765101+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.765101+0000","last_undegraded":"2023-12-01T01:55:23.765101+0000","last_fullsized":"2023-12-01T01:55:23.765101+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:31:34.839958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.768258+0000","last_change":"2023-12-01T01:55:23.768258+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.768258+0000","last_undegraded":"2023-12-01T01:55:23.768258+0000","last_fullsized":"2023-12-01T01:55:23.768258+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:51:24.042108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.768949+0000","last_change":"2023-12-01T01:55:23.768949+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.768949+0000","last_undegraded":"2023-12-01T01:55:23.768949+0000","last_fullsized":"2023-12-01T01:55:23.768949+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:21:25.546243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.770719+0000","last_change":"2023-12-01T01:55:23.770719+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.770719+0000","last_undegraded":"2023-12-01T01:55:23.770719+0000","last_fullsized":"2023-12-01T01:55:23.770719+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:04.786339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.777679+0000","last_change":"2023-12-01T01:55:23.777679+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.777679+0000","last_undegraded":"2023-12-01T01:55:23.777679+0000","last_fullsized":"2023-12-01T01:55:23.777679+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:49:13.404686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.766713+0000","last_change":"2023-12-01T01:55:23.766713+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.766713+0000","last_undegraded":"2023-12-01T01:55:23.766713+0000","last_fullsized":"2023-12-01T01:55:23.766713+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:29:19.806110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.773560+0000","last_change":"2023-12-01T01:55:23.773560+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.773560+0000","last_undegraded":"2023-12-01T01:55:23.773560+0000","last_fullsized":"2023-12-01T01:55:23.773560+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:02:49.343248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.770841+0000","last_change":"2023-12-01T01:55:23.770841+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.770841+0000","last_undegraded":"2023-12-01T01:55:23.770841+0000","last_fullsized":"2023-12-01T01:55:23.770841+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:14:12.270172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.775024+0000","last_change":"2023-12-01T01:55:23.775024+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.775024+0000","last_undegraded":"2023-12-01T01:55:23.775024+0000","last_fullsized":"2023-12-01T01:55:23.775024+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:54:42.635447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.773748+0000","last_change":"2023-12-01T01:55:23.773748+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.773748+0000","last_undegraded":"2023-12-01T01:55:23.773748+0000","last_fullsized":"2023-12-01T01:55:23.773748+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:26:29.479647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.767275+0000","last_change":"2023-12-01T01:55:23.767275+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.767275+0000","last_undegraded":"2023-12-01T01:55:23.767275+0000","last_fullsized":"2023-12-01T01:55:23.767275+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:31:34.839958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.775445+0000","last_change":"2023-12-01T01:55:23.775445+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.775445+0000","last_undegraded":"2023-12-01T01:55:23.775445+0000","last_fullsized":"2023-12-01T01:55:23.775445+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:38:25.116989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.770485+0000","last_change":"2023-12-01T01:55:23.770485+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.770485+0000","last_undegraded":"2023-12-01T01:55:23.770485+0000","last_fullsized":"2023-12-01T01:55:23.770485+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:31:31.035582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.775157+0000","last_change":"2023-12-01T01:55:23.775157+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.775157+0000","last_undegraded":"2023-12-01T01:55:23.775157+0000","last_fullsized":"2023-12-01T01:55:23.775157+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:13.267691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.778949+0000","last_change":"2023-12-01T01:55:23.778949+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.778949+0000","last_undegraded":"2023-12-01T01:55:23.778949+0000","last_fullsized":"2023-12-01T01:55:23.778949+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:00.680731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.773853+0000","last_change":"2023-12-01T01:55:23.773853+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.773853+0000","last_undegraded":"2023-12-01T01:55:23.773853+0000","last_fullsized":"2023-12-01T01:55:23.773853+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:41.929533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.840404+0000","last_change":"2023-12-01T01:55:23.840404+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.840404+0000","last_undegraded":"2023-12-01T01:55:23.840404+0000","last_fullsized":"2023-12-01T01:55:23.840404+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:14.405529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.840540+0000","last_change":"2023-12-01T01:55:23.840540+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.840540+0000","last_undegraded":"2023-12-01T01:55:23.840540+0000","last_fullsized":"2023-12-01T01:55:23.840540+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:50:06.161028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.774266+0000","last_change":"2023-12-01T01:55:23.774266+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.774266+0000","last_undegraded":"2023-12-01T01:55:23.774266+0000","last_fullsized":"2023-12-01T01:55:23.774266+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:00.939658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.776528+0000","last_change":"2023-12-01T01:55:23.776528+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.776528+0000","last_undegraded":"2023-12-01T01:55:23.776528+0000","last_fullsized":"2023-12-01T01:55:23.776528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:50.465362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.765852+0000","last_change":"2023-12-01T01:55:23.765852+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.765852+0000","last_undegraded":"2023-12-01T01:55:23.765852+0000","last_fullsized":"2023-12-01T01:55:23.765852+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:56:33.463990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.767190+0000","last_change":"2023-12-01T01:55:23.767190+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.767190+0000","last_undegraded":"2023-12-01T01:55:23.767190+0000","last_fullsized":"2023-12-01T01:55:23.767190+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:40.756808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.769841+0000","last_change":"2023-12-01T01:55:23.769841+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.769841+0000","last_undegraded":"2023-12-01T01:55:23.769841+0000","last_fullsized":"2023-12-01T01:55:23.769841+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:56.576935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.766693+0000","last_change":"2023-12-01T01:55:23.766693+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.766693+0000","last_undegraded":"2023-12-01T01:55:23.766693+0000","last_fullsized":"2023-12-01T01:55:23.766693+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:04.030142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.766196+0000","last_change":"2023-12-01T01:55:23.766196+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.766196+0000","last_undegraded":"2023-12-01T01:55:23.766196+0000","last_fullsized":"2023-12-01T01:55:23.766196+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:01:54.391568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.766038+0000","last_change":"2023-12-01T01:55:23.766038+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.766038+0000","last_undegraded":"2023-12-01T01:55:23.766038+0000","last_fullsized":"2023-12-01T01:55:23.766038+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:46:29.123760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.766579+0000","last_change":"2023-12-01T01:55:23.766579+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.766579+0000","last_undegraded":"2023-12-01T01:55:23.766579+0000","last_fullsized":"2023-12-01T01:55:23.766579+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:27.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.840379+0000","last_change":"2023-12-01T01:55:23.840379+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.840379+0000","last_undegraded":"2023-12-01T01:55:23.840379+0000","last_fullsized":"2023-12-01T01:55:23.840379+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:59.164113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.775476+0000","last_change":"2023-12-01T01:55:23.775476+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.775476+0000","last_undegraded":"2023-12-01T01:55:23.775476+0000","last_fullsized":"2023-12-01T01:55:23.775476+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:08.860150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.774749+0000","last_change":"2023-12-01T01:55:23.774749+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.774749+0000","last_undegraded":"2023-12-01T01:55:23.774749+0000","last_fullsized":"2023-12-01T01:55:23.774749+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:59:26.422380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T01:55:24.759522+0000","last_change":"2023-12-01T01:55:24.759522+0000","last_active":"2023-12-01T01:55:24.759522+0000","last_peered":"2023-12-01T01:55:24.759522+0000","last_clean":"2023-12-01T01:55:24.759522+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:24.759522+0000","last_undegraded":"2023-12-01T01:55:24.759522+0000","last_fullsized":"2023-12-01T01:55:24.759522+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":138,"state":"creating+peering","last_fresh":"2023-12-01T01:55:23.765101+0000","last_change":"2023-12-01T01:55:23.765101+0000","last_active":"2023-12-01T01:55:23.741967+0000","last_peered":"2023-12-01T01:55:23.741967+0000","last_clean":"2023-12-01T01:55:23.741967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T01:55:23.765101+0000","last_undegraded":"2023-12-01T01:55:23.765101+0000","last_fullsized":"2023-12-01T01:55:23.765101+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:14:12.270172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"128'227","reported_seq":662,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-01T01:55:23.170464+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:55:23.170464+0000","last_peered":"2023-12-01T01:55:23.170464+0000","last_clean":"2023-12-01T01:55:23.170464+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:55:23.170464+0000","last_undegraded":"2023-12-01T01:55:23.170464+0000","last_fullsized":"2023-12-01T01:55:23.170464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575080,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1699840,"data_stored":812119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0549999999999999,"15min":1.069},"min":{"1min":0.77600000000000002,"5min":0.54900000000000004,"15min":0.41299999999999998},"max":{"1min":1.6359999999999999,"5min":1.7430000000000001,"15min":1.8089999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.387,"5min":1.5289999999999999,"15min":2.2029999999999998},"last":1.099}]},{"osd":1,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1419999999999999,"15min":1.208},"min":{"1min":0.753,"5min":0.59799999999999998,"15min":0.57299999999999995},"max":{"1min":1.6000000000000001,"5min":1.802,"15min":1.8100000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2010000000000001,"15min":1.2509999999999999},"min":{"1min":0.84199999999999997,"5min":0.66800000000000004,"15min":0.59199999999999997},"max":{"1min":1.456,"5min":1.629,"15min":1.744},"last":1.2669999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1080000000000001,"15min":1.117},"min":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.50600000000000001},"max":{"1min":1.3740000000000001,"5min":1.607,"15min":1.728},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1910000000000001,"15min":1.21},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.36599999999999999},"max":{"1min":1.6080000000000001,"5min":1.681,"15min":1.9690000000000001},"last":1.46}]},{"osd":3,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1890000000000001,"15min":1.2310000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.496,"5min":1.637,"15min":2.0550000000000002},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.284,"15min":1.3280000000000001},"min":{"1min":0.90900000000000003,"5min":0.63700000000000001,"15min":0.44900000000000001},"max":{"1min":1.5580000000000001,"5min":1.7490000000000001,"15min":2.1880000000000002},"last":1.5720000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3,"15min":1.288},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.55500000000000005},"max":{"1min":1.663,"5min":1.8300000000000001,"15min":2.2130000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.385,"15min":1.399},"min":{"1min":0.93600000000000005,"5min":0.77500000000000002,"15min":0.63300000000000001},"max":{"1min":1.694,"5min":1.8380000000000001,"15min":2.0939999999999999},"last":1.6299999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.232,"15min":1.2450000000000001},"min":{"1min":0.72999999999999998,"5min":0.625,"15min":0.42799999999999999},"max":{"1min":1.5449999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"min":{"1min":1.111,"5min":0.69399999999999995,"15min":0.66900000000000004},"max":{"1min":1.6679999999999999,"5min":1.9379999999999999,"15min":2.2930000000000001},"last":1.712}]},{"osd":6,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3640000000000001,"15min":1.373},"min":{"1min":0.92800000000000005,"5min":0.753,"15min":0.51200000000000001},"max":{"1min":1.5309999999999999,"5min":2.294,"15min":2.3420000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.3540000000000001,"15min":1.3799999999999999},"min":{"1min":0.82899999999999996,"5min":0.82599999999999996,"15min":0.60899999999999999},"max":{"1min":1.5960000000000001,"5min":1.954,"15min":2.3500000000000001},"last":1.498}]}]},{"osd":6,"up_from":13,"seq":55834575081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36816,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":1712128,"data_stored":817936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.69799999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4139999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.085},{"interface":"front","average":{"1min":1.131,"5min":1.1080000000000001,"15min":1.1319999999999999},"min":{"1min":0.78700000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4570000000000001,"5min":1.579,"15min":2.5569999999999999},"last":1.234}]},{"osd":1,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.181,"15min":1.167},"min":{"1min":0.80500000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.5489999999999999,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":1.177},{"interface":"front","average":{"1min":1.256,"5min":1.198,"15min":1.208},"min":{"1min":0.90400000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5860000000000001,"5min":2.0720000000000001,"15min":2.0920000000000001},"last":1.383}]},{"osd":2,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1240000000000001,"15min":1.1659999999999999},"min":{"1min":0.84899999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.462,"5min":1.6359999999999999,"15min":2.3460000000000001},"last":1.129},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.216,"15min":1.238},"min":{"1min":0.82199999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.619,"5min":3.024,"15min":3.024},"last":0.82799999999999996}]},{"osd":3,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.23,"15min":1.2709999999999999},"min":{"1min":0.81799999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.556,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.8660000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4019999999999999,"15min":1.4039999999999999},"min":{"1min":0.95899999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.7450000000000001,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.6279999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.2949999999999999,"15min":1.278},"min":{"1min":0.92500000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":2.226},"last":1.675},{"interface":"front","average":{"1min":1.516,"5min":1.385,"15min":1.355},"min":{"1min":1.0600000000000001,"5min":0.93600000000000005,"15min":0.78100000000000003},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":2.2839999999999998},"last":1.7210000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.421,"15min":1.409},"min":{"1min":1.1699999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.9219999999999999},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.496,"15min":1.486},"min":{"1min":1.3180000000000001,"5min":0.96499999999999997,"15min":0.84099999999999997},"max":{"1min":1.9770000000000001,"5min":1.9790000000000001,"15min":2.8580000000000001},"last":1.2669999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":1.522,"15min":1.516},"min":{"1min":1.3759999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.958,"5min":1.978,"15min":2.762},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.748,"5min":1.577,"15min":1.5780000000000001},"min":{"1min":1.4159999999999999,"5min":1.038,"15min":1.038},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.8479999999999999},"last":1.52}]}]},{"osd":5,"up_from":13,"seq":55834575080,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2240,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2293760,"data_stored":1402487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.111,"15min":1.1080000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.522,"5min":1.8240000000000001,"15min":2.2010000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1060000000000001,"15min":1.1439999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5609999999999999,"5min":1.7130000000000001,"15min":2.153},"last":0.91100000000000003}]},{"osd":1,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.149,"15min":1.1499999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6040000000000001,"5min":1.77,"15min":1.77},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.139,"5min":1.145,"15min":1.157},"min":{"1min":0.754,"5min":0.72999999999999998,"15min":0.67800000000000005},"max":{"1min":1.4890000000000001,"5min":1.6519999999999999,"15min":1.7330000000000001},"last":1.3879999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.147,"15min":1.1879999999999999},"min":{"1min":0.71999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.532,"5min":1.702,"15min":1.702},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.226,"15min":1.2370000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.605,"5min":1.9239999999999999,"15min":2.0630000000000002},"last":1.0149999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2190000000000001,"15min":1.2629999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.651,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.365},{"interface":"front","average":{"1min":1.294,"5min":1.3109999999999999,"15min":1.304},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.292}]},{"osd":4,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.226,"15min":1.224},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.474,"5min":1.984,"15min":1.984},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.3009999999999999,"15min":1.3340000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.6299999999999999,"5min":1.927,"15min":1.927},"last":1.103}]},{"osd":6,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3300000000000001,"15min":1.325},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.694,"5min":1.885,"15min":1.9790000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3799999999999999,"15min":1.371},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.746,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.4359999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.45,"15min":1.4670000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.7210000000000001,"5min":2.0499999999999998,"15min":2.0920000000000001},"last":1.454},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.532,"15min":1.5229999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.861,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.403}]}]},{"osd":4,"up_from":13,"seq":55834575080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40912,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330928,"statfs":{"total":96636764160,"available":96594870272,"internally_reserved":0,"allocated":1712128,"data_stored":817936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.77000000000000002,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5509999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.995},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1599999999999999,"15min":1.1739999999999999},"min":{"1min":0.80600000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.603,"5min":1.778,"15min":2.1099999999999999},"last":1.2}]},{"osd":1,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.222,"15min":1.214},"min":{"1min":0.83699999999999997,"5min":0.72399999999999998,"15min":0.65900000000000003},"max":{"1min":1.8460000000000001,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.147,"15min":1.135},"min":{"1min":0.72999999999999998,"5min":0.67300000000000004,"15min":0.64200000000000002},"max":{"1min":1.6060000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.248}]},{"osd":2,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2150000000000001,"15min":1.202},"min":{"1min":0.91800000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4330000000000001,"5min":1.871,"15min":1.871},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.288,"15min":1.29},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.599,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.4510000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.246,"15min":1.244},"min":{"1min":0.93300000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5169999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.3180000000000001,"15min":1.3169999999999999},"min":{"1min":1.071,"5min":0.753,"15min":0.753},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.903},"last":1.2769999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.337,"15min":1.321},"min":{"1min":0.95999999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.7010000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.3080000000000001,"15min":1.292},"min":{"1min":1.0189999999999999,"5min":0.90300000000000002,"15min":0.85399999999999998},"max":{"1min":1.627,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.351}]},{"osd":6,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":1.3180000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.7649999999999999,"5min":1.8799999999999999,"15min":1.9950000000000001},"last":1.397},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.47,"15min":1.4790000000000001},"min":{"1min":1.2330000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.96,"5min":2.306,"15min":2.54},"last":1.54}]},{"osd":7,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.468,"15min":1.4630000000000001},"min":{"1min":1.2989999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.8799999999999999,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.5620000000000001,"15min":1.5660000000000001},"min":{"1min":1.1299999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":1.9830000000000001,"5min":2.04,"15min":2.157},"last":1.4950000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58948,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312892,"statfs":{"total":96636764160,"available":96576401408,"internally_reserved":0,"allocated":1699840,"data_stored":812119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.84699999999999998,"15min":0.85799999999999998},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.472,"5min":1.472,"15min":1.583},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.032,"5min":0.94299999999999995,"15min":0.98699999999999999},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.435},"max":{"1min":1.395,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.99199999999999999,"15min":1.0580000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.573,"5min":1.6200000000000001,"15min":1.6339999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.91300000000000003,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.48399999999999999,"15min":0.48199999999999998},"max":{"1min":1.3049999999999999,"5min":1.5129999999999999,"15min":1.6519999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.999,"15min":1.0429999999999999},"min":{"1min":0.59399999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3759999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1060000000000001,"15min":1.1799999999999999},"min":{"1min":0.495,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.575,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1930000000000001,"15min":1.232},"min":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.9319999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":0.85499999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.627,"5min":1.909,"15min":2.0419999999999998},"last":0.90600000000000003}]},{"osd":5,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2370000000000001,"15min":1.2709999999999999},"min":{"1min":1.071,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.706,"5min":1.7669999999999999,"15min":1.806},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.28,"15min":1.2989999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7549999999999999,"5min":1.831,"15min":2.1880000000000002},"last":0.81599999999999995}]},{"osd":6,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3500000000000001,"15min":1.3700000000000001},"min":{"1min":1.034,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.649,"5min":1.984,"15min":1.984},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.3839999999999999,"15min":1.411},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8109999999999999,"5min":1.9119999999999999,"15min":1.99},"last":1.228}]},{"osd":7,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.3660000000000001,"15min":1.411},"min":{"1min":0.873,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8799999999999999,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.177},{"interface":"front","average":{"1min":1.464,"5min":1.419,"15min":1.4430000000000001},"min":{"1min":1.121,"5min":0.87,"15min":0.87},"max":{"1min":1.8879999999999999,"5min":2.0430000000000001,"15min":2.1230000000000002},"last":1.1479999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1699840,"data_stored":812119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2729999999999999,"5min":1.411,"15min":1.5529999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.95299999999999996,"15min":0.99099999999999999},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.502,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.1890000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.94599999999999995,"15min":0.97899999999999998},"min":{"1min":0.434,"5min":0.375,"15min":0.373},"max":{"1min":1.5,"5min":2.5,"15min":2.5},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.036,"15min":1.075},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.5549999999999999,"5min":1.909,"15min":1.909},"last":0.92000000000000004}]},{"osd":3,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.91000000000000003,"15min":0.99099999999999999},"min":{"1min":0.48799999999999999,"5min":0.45800000000000002,"15min":0.45200000000000001},"max":{"1min":1.3500000000000001,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0649999999999999,"15min":1.1419999999999999},"min":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.571,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.379}]},{"osd":4,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2290000000000001,"15min":1.2909999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.571,"5min":1.869,"15min":1.9950000000000001},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.1870000000000001,"15min":1.2470000000000001},"min":{"1min":0.71099999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5069999999999999,"5min":1.6879999999999999,"15min":1.8080000000000001},"last":1.1399999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2669999999999999,"15min":1.3069999999999999},"min":{"1min":0.79200000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.6220000000000001,"5min":2.948,"15min":2.948},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.294,"15min":1.343},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7370000000000001,"5min":2.016,"15min":2.048},"last":1.5609999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.264,"15min":1.3120000000000001},"min":{"1min":0.77600000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.609,"5min":1.6439999999999999,"15min":1.988},"last":1.25},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.381,"15min":1.4139999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.2309999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.506}]},{"osd":7,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.379,"15min":1.4259999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.339},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3160000000000001,"15min":1.371},"min":{"1min":0.83299999999999996,"5min":0.67100000000000004,"15min":0.67000000000000004},"max":{"1min":1.7110000000000001,"5min":1.9259999999999999,"15min":2.101},"last":1.3129999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59516,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312324,"statfs":{"total":96636764160,"available":96575819776,"internally_reserved":0,"allocated":2281472,"data_stored":1396800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.77900000000000003,"15min":0.83899999999999997},"min":{"1min":0.48199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.5049999999999999},"last":1.042},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.95699999999999996,"15min":1.0069999999999999},"min":{"1min":0.52100000000000002,"5min":0.40500000000000003,"15min":0.39600000000000002},"max":{"1min":1.2749999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.0820000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.54400000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.54},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.026,"15min":1.0780000000000001},"min":{"1min":0.64700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.363,"5min":1.7250000000000001,"15min":1.804},"last":1.22}]},{"osd":3,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.165,"15min":1.21},"min":{"1min":0.73799999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.51,"5min":1.827,"15min":1.827},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.036,"5min":0.99299999999999999,"15min":1.0409999999999999},"min":{"1min":0.71099999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5069999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.284}]},{"osd":4,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.175,"15min":1.1839999999999999},"min":{"1min":0.89800000000000002,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.4850000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.319,"15min":1.335},"min":{"1min":0.90800000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8360000000000001,"5min":1.905,"15min":1.905},"last":1.46}]},{"osd":5,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2649999999999999,"15min":1.28},"min":{"1min":1.0489999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.706,"5min":1.988,"15min":1.988},"last":1.589},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.26,"15min":1.2769999999999999},"min":{"1min":0.95599999999999996,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6439999999999999,"5min":1.6799999999999999,"15min":1.825},"last":1.141}]},{"osd":6,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3089999999999999,"15min":1.333},"min":{"1min":1.087,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6100000000000001,"5min":1.839,"15min":1.861},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.321,"15min":1.3360000000000001},"min":{"1min":0.88100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.587,"5min":1.742,"15min":1.8109999999999999},"last":1.427}]},{"osd":7,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3899999999999999,"15min":1.4259999999999999},"min":{"1min":0.99099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.823,"5min":1.9099999999999999,"15min":2.0609999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.371,"15min":1.4039999999999999},"min":{"1min":0.93300000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7869999999999999,"5min":1.931,"15min":1.992},"last":1.6759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1699840,"data_stored":812119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.80700000000000005,"15min":0.83999999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.081,"5min":1.3520000000000001,"15min":1.8109999999999999},"last":1.161},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.93500000000000005,"15min":0.96499999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":1.5620000000000001,"5min":1.5860000000000001,"15min":1.6910000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95499999999999996,"15min":0.98799999999999999},"min":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.42699999999999999},"max":{"1min":1.3799999999999999,"5min":1.528,"15min":2.157},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.0089999999999999,"15min":1.054},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.389,"5min":1.3999999999999999,"15min":1.728},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97599999999999998,"15min":1.004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.331,"5min":1.4850000000000001,"15min":1.7509999999999999},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.137,"15min":1.155},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.46500000000000002},"max":{"1min":1.4830000000000001,"5min":1.7070000000000001,"15min":2.2610000000000001},"last":0.89600000000000002}]},{"osd":4,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.248,"15min":1.2949999999999999},"min":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6140000000000001,"5min":1.6240000000000001,"15min":2.3719999999999999},"last":1.421},{"interface":"front","average":{"1min":1.2,"5min":1.1930000000000001,"15min":1.248},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5800000000000001,"5min":1.7989999999999999,"15min":1.823},"last":1.05}]},{"osd":5,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3,"15min":1.333},"min":{"1min":0.88700000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.5600000000000001,"5min":1.841,"15min":2.351},"last":1.746},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.278,"15min":1.3220000000000001},"min":{"1min":0.89800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6890000000000001,"5min":1.7509999999999999,"15min":2.2909999999999999},"last":1.077}]},{"osd":6,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.272,"15min":1.3049999999999999},"min":{"1min":0.69299999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5700000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.111},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3600000000000001,"15min":1.381},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7290000000000001,"5min":1.9159999999999999,"15min":2.4380000000000002},"last":1.337}]},{"osd":7,"last update":"Fri Dec 1 01:54:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.353,"15min":1.3939999999999999},"min":{"1min":0.70699999999999996,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":2.4279999999999999},"last":1.234},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3500000000000001,"15min":1.383},"min":{"1min":0.82699999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.7669999999999999,"5min":1.9199999999999999,"15min":2.3039999999999998},"last":1.2949999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:55:29.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:55:29.732 INFO:tasks.ceph.mds.c.smithi115.stderr:2023-12-01T01:55:29.731+0000 7f55be235640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:55:29.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.950+0000 7fe198651640 1 Processor -- start 2023-12-01T01:55:29.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.950+0000 7fe198651640 1 -- start start 2023-12-01T01:55:29.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe198651640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1900aded0 0x7fe1900cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe198651640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe1900c8ed0 0x7fe19006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe198651640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19006f2f0 con 0x7fe19006c120 2023-12-01T01:55:29.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe198651640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19006c5b0 con 0x7fe1900c8ed0 2023-12-01T01:55:29.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe198651640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19006f5d0 con 0x7fe1900aded0 2023-12-01T01:55:29.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe195bec640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe1900c8ed0 0x7fe19006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe195bec640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe1900c8ed0 0x7fe19006bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55848/0 (socket says 172.21.15.115:55848) 2023-12-01T01:55:29.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe195bec640 1 -- 172.21.15.115:0/2846334614 learned_addr learned my addr 172.21.15.115:0/2846334614 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:29.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe1963ed640 1 --2- 172.21.15.115:0/2846334614 >> v2:172.21.15.142:3300/0 conn(0x7fe1900aded0 0x7fe1900cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe195bec640 1 -- 172.21.15.115:0/2846334614 >> v2:172.21.15.142:3300/0 conn(0x7fe1900aded0 msgr2=0x7fe1900cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:29.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe195bec640 1 --2- 172.21.15.115:0/2846334614 >> v2:172.21.15.142:3300/0 conn(0x7fe1900aded0 0x7fe1900cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.952+0000 7fe195bec640 1 -- 172.21.15.115:0/2846334614 >> v1:172.21.15.115:6789/0 conn(0x7fe19006c120 legacy=0x7fe1900cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.953+0000 7fe195bec640 1 -- 172.21.15.115:0/2846334614 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1900c9350 con 0x7fe1900c8ed0 2023-12-01T01:55:29.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.953+0000 7fe1953eb640 1 -- 172.21.15.115:0/2846334614 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358192864 0 0) 0x7fe19006f2f0 con 0x7fe19006c120 2023-12-01T01:55:29.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.954+0000 7fe195bec640 1 --2- 172.21.15.115:0/2846334614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1900c8ed0 0x7fe19006bbe0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe19006de50 tx=0x7fe18002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2b37fe4228dd65 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:29.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.954+0000 7fe1953eb640 1 -- 172.21.15.115:0/2846334614 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe180031870 con 0x7fe1900c8ed0 2023-12-01T01:55:29.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.954+0000 7fe1953eb640 1 -- 172.21.15.115:0/2846334614 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1800319d0 con 0x7fe1900c8ed0 2023-12-01T01:55:29.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.954+0000 7fe1953eb640 1 -- 172.21.15.115:0/2846334614 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe180031d60 con 0x7fe1900c8ed0 2023-12-01T01:55:29.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.954+0000 7fe198651640 1 -- 172.21.15.115:0/2846334614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1900c8ed0 msgr2=0x7fe19006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.955+0000 7fe198651640 1 --2- 172.21.15.115:0/2846334614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1900c8ed0 0x7fe19006bbe0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe19006de50 tx=0x7fe18002b910 comp rx=0 tx=0).stop 2023-12-01T01:55:29.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.955+0000 7fe198651640 1 -- 172.21.15.115:0/2846334614 shutdown_connections 2023-12-01T01:55:29.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.955+0000 7fe198651640 1 --2- 172.21.15.115:0/2846334614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1900c8ed0 0x7fe19006bbe0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.955+0000 7fe198651640 1 --2- 172.21.15.115:0/2846334614 >> v2:172.21.15.142:3300/0 conn(0x7fe1900aded0 0x7fe1900cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.955+0000 7fe198651640 1 -- 172.21.15.115:0/2846334614 >> 172.21.15.115:0/2846334614 conn(0x7fe1900ce880 msgr2=0x7fe1900cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:29.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.955+0000 7fe198651640 1 -- 172.21.15.115:0/2846334614 shutdown_connections 2023-12-01T01:55:29.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.955+0000 7fe198651640 1 -- 172.21.15.115:0/2846334614 wait complete. 2023-12-01T01:55:29.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.956+0000 7fe198651640 1 Processor -- start 2023-12-01T01:55:29.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.957+0000 7fe198651640 1 -- start start 2023-12-01T01:55:29.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.957+0000 7fe198651640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe19006c120 0x7fe190158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.957+0000 7fe198651640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1900aded0 0x7fe1901e4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe198651640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1900c8ed0 0x7fe1900c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe198651640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901e3d70 con 0x7fe19006c120 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe198651640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901593e0 con 0x7fe1900c8ed0 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe198651640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901e4740 con 0x7fe1900aded0 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe1963ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe19006c120 0x7fe190158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe1963ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe19006c120 0x7fe190158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55862/0 (socket says 172.21.15.115:55862) 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe1963ed640 1 -- 172.21.15.115:0/3907410002 learned_addr learned my addr 172.21.15.115:0/3907410002 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe1963ed640 1 -- 172.21.15.115:0/3907410002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1900aded0 msgr2=0x7fe1901e4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe195bec640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1900aded0 0x7fe1901e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe196bee640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1900c8ed0 0x7fe1900c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe1963ed640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1900aded0 0x7fe1901e4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe1963ed640 1 -- 172.21.15.115:0/3907410002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1900c8ed0 msgr2=0x7fe1900c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:29.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe1963ed640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1900c8ed0 0x7fe1900c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.958+0000 7fe1963ed640 1 -- 172.21.15.115:0/3907410002 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800094d0 con 0x7fe19006c120 2023-12-01T01:55:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.959+0000 7fe1963ed640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe19006c120 0x7fe190158ea0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00d830 tx=0x7fe18c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:29.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.959+0000 7fe186ffd640 1 -- 172.21.15.115:0/3907410002 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00b6c0 con 0x7fe19006c120 2023-12-01T01:55:29.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.959+0000 7fe186ffd640 1 -- 172.21.15.115:0/3907410002 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c00b820 con 0x7fe19006c120 2023-12-01T01:55:29.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.959+0000 7fe186ffd640 1 -- 172.21.15.115:0/3907410002 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00bbb0 con 0x7fe19006c120 2023-12-01T01:55:29.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.959+0000 7fe198651640 1 -- 172.21.15.115:0/3907410002 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1901e3160 con 0x7fe19006c120 2023-12-01T01:55:29.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.959+0000 7fe198651640 1 -- 172.21.15.115:0/3907410002 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1901e3720 con 0x7fe19006c120 2023-12-01T01:55:29.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.960+0000 7fe198651640 1 -- 172.21.15.115:0/3907410002 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19004e890 con 0x7fe19006c120 2023-12-01T01:55:29.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.961+0000 7fe186ffd640 1 -- 172.21.15.115:0/3907410002 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe18c01e460 con 0x7fe19006c120 2023-12-01T01:55:29.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.962+0000 7fe186ffd640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe17406a550 0x7fe17406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:29.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.962+0000 7fe186ffd640 1 -- 172.21.15.115:0/3907410002 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe18c089180 con 0x7fe19006c120 2023-12-01T01:55:29.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.962+0000 7fe195bec640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe17406a550 0x7fe17406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:29.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.963+0000 7fe195bec640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe17406a550 0x7fe17406ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe180036000 tx=0x7fe18003c040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:29.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:29.967+0000 7fe186ffd640 1 -- 172.21.15.115:0/3907410002 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe18c0546f0 con 0x7fe19006c120 2023-12-01T01:55:30.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.152+0000 7fe198651640 1 -- 172.21.15.115:0/3907410002 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1900c9910 con 0x7fe17406a550 2023-12-01T01:55:30.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.175+0000 7fe186ffd640 1 -- 172.21.15.115:0/3907410002 <== mgr.8247 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287220 (secure 0 0 0) 0x7fe18003a420 con 0x7fe17406a550 2023-12-01T01:55:30.181 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:30.183 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T01:55:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 -- 172.21.15.115:0/3907410002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe17406a550 msgr2=0x7fe17406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe17406a550 0x7fe17406ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe180036000 tx=0x7fe18003c040 comp rx=0 tx=0).stop 2023-12-01T01:55:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 -- 172.21.15.115:0/3907410002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe19006c120 msgr2=0x7fe190158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe19006c120 0x7fe190158ea0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00d830 tx=0x7fe18c00dcf0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 -- 172.21.15.115:0/3907410002 shutdown_connections 2023-12-01T01:55:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe17406a550 0x7fe17406ca10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1900aded0 0x7fe1901e4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1900c8ed0 0x7fe1900c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 --2- 172.21.15.115:0/3907410002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe19006c120 0x7fe190158ea0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 -- 172.21.15.115:0/3907410002 >> 172.21.15.115:0/3907410002 conn(0x7fe1900ce880 msgr2=0x7fe19006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:30.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 -- 172.21.15.115:0/3907410002 shutdown_connections 2023-12-01T01:55:30.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.181+0000 7fe198651640 1 -- 172.21.15.115:0/3907410002 wait complete. 2023-12-01T01:55:30.202 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-12-01T01:55:29.937338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":332136,"kb_used_data":14888,"kb_used_omap":0,"kb_used_meta":317184,"kb_avail":754642584,"statfs":{"total":773094113280,"available":772754006016,"internally_reserved":0,"allocated":15245312,"data_stored":7883748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":324796416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001685"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.237905+0000","last_change":"2023-12-01T01:55:25.297035+0000","last_active":"2023-12-01T01:55:27.237905+0000","last_peered":"2023-12-01T01:55:27.237905+0000","last_clean":"2023-12-01T01:55:27.237905+0000","last_became_active":"2023-12-01T01:55:25.296754+0000","last_became_peered":"2023-12-01T01:55:25.296754+0000","last_unstale":"2023-12-01T01:55:27.237905+0000","last_undegraded":"2023-12-01T01:55:27.237905+0000","last_fullsized":"2023-12-01T01:55:27.237905+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:52:33.717797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.945764+0000","last_change":"2023-12-01T01:55:25.290935+0000","last_active":"2023-12-01T01:55:27.945764+0000","last_peered":"2023-12-01T01:55:27.945764+0000","last_clean":"2023-12-01T01:55:27.945764+0000","last_became_active":"2023-12-01T01:55:25.290712+0000","last_became_peered":"2023-12-01T01:55:25.290712+0000","last_unstale":"2023-12-01T01:55:27.945764+0000","last_undegraded":"2023-12-01T01:55:27.945764+0000","last_fullsized":"2023-12-01T01:55:27.945764+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:13:27.640209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.945241+0000","last_change":"2023-12-01T01:55:25.283066+0000","last_active":"2023-12-01T01:55:27.945241+0000","last_peered":"2023-12-01T01:55:27.945241+0000","last_clean":"2023-12-01T01:55:27.945241+0000","last_became_active":"2023-12-01T01:55:25.282820+0000","last_became_peered":"2023-12-01T01:55:25.282820+0000","last_unstale":"2023-12-01T01:55:27.945241+0000","last_undegraded":"2023-12-01T01:55:27.945241+0000","last_fullsized":"2023-12-01T01:55:27.945241+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:15.423083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.271967+0000","last_change":"2023-12-01T01:55:25.291153+0000","last_active":"2023-12-01T01:55:26.271967+0000","last_peered":"2023-12-01T01:55:26.271967+0000","last_clean":"2023-12-01T01:55:26.271967+0000","last_became_active":"2023-12-01T01:55:25.290802+0000","last_became_peered":"2023-12-01T01:55:25.290802+0000","last_unstale":"2023-12-01T01:55:26.271967+0000","last_undegraded":"2023-12-01T01:55:26.271967+0000","last_fullsized":"2023-12-01T01:55:26.271967+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:10.460162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.272088+0000","last_change":"2023-12-01T01:55:25.301748+0000","last_active":"2023-12-01T01:55:26.272088+0000","last_peered":"2023-12-01T01:55:26.272088+0000","last_clean":"2023-12-01T01:55:26.272088+0000","last_became_active":"2023-12-01T01:55:25.301422+0000","last_became_peered":"2023-12-01T01:55:25.301422+0000","last_unstale":"2023-12-01T01:55:26.272088+0000","last_undegraded":"2023-12-01T01:55:26.272088+0000","last_fullsized":"2023-12-01T01:55:26.272088+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:57.594490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.268274+0000","last_change":"2023-12-01T01:55:25.290550+0000","last_active":"2023-12-01T01:55:26.268274+0000","last_peered":"2023-12-01T01:55:26.268274+0000","last_clean":"2023-12-01T01:55:26.268274+0000","last_became_active":"2023-12-01T01:55:25.290273+0000","last_became_peered":"2023-12-01T01:55:25.290273+0000","last_unstale":"2023-12-01T01:55:26.268274+0000","last_undegraded":"2023-12-01T01:55:26.268274+0000","last_fullsized":"2023-12-01T01:55:26.268274+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:12:28.380507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.943410+0000","last_change":"2023-12-01T01:55:25.271992+0000","last_active":"2023-12-01T01:55:27.943410+0000","last_peered":"2023-12-01T01:55:27.943410+0000","last_clean":"2023-12-01T01:55:27.943410+0000","last_became_active":"2023-12-01T01:55:25.271759+0000","last_became_peered":"2023-12-01T01:55:25.271759+0000","last_unstale":"2023-12-01T01:55:27.943410+0000","last_undegraded":"2023-12-01T01:55:27.943410+0000","last_fullsized":"2023-12-01T01:55:27.943410+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:31:26.401722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.944029+0000","last_change":"2023-12-01T01:55:25.288169+0000","last_active":"2023-12-01T01:55:27.944029+0000","last_peered":"2023-12-01T01:55:27.944029+0000","last_clean":"2023-12-01T01:55:27.944029+0000","last_became_active":"2023-12-01T01:55:25.286895+0000","last_became_peered":"2023-12-01T01:55:25.286895+0000","last_unstale":"2023-12-01T01:55:27.944029+0000","last_undegraded":"2023-12-01T01:55:27.944029+0000","last_fullsized":"2023-12-01T01:55:27.944029+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:20:44.399516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.237773+0000","last_change":"2023-12-01T01:55:25.303181+0000","last_active":"2023-12-01T01:55:27.237773+0000","last_peered":"2023-12-01T01:55:27.237773+0000","last_clean":"2023-12-01T01:55:27.237773+0000","last_became_active":"2023-12-01T01:55:25.302917+0000","last_became_peered":"2023-12-01T01:55:25.302917+0000","last_unstale":"2023-12-01T01:55:27.237773+0000","last_undegraded":"2023-12-01T01:55:27.237773+0000","last_fullsized":"2023-12-01T01:55:27.237773+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:33.213309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.271681+0000","last_change":"2023-12-01T01:55:25.296736+0000","last_active":"2023-12-01T01:55:26.271681+0000","last_peered":"2023-12-01T01:55:26.271681+0000","last_clean":"2023-12-01T01:55:26.271681+0000","last_became_active":"2023-12-01T01:55:25.296431+0000","last_became_peered":"2023-12-01T01:55:25.296431+0000","last_unstale":"2023-12-01T01:55:26.271681+0000","last_undegraded":"2023-12-01T01:55:26.271681+0000","last_fullsized":"2023-12-01T01:55:26.271681+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:46:30.141315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.945759+0000","last_change":"2023-12-01T01:55:25.271550+0000","last_active":"2023-12-01T01:55:27.945759+0000","last_peered":"2023-12-01T01:55:27.945759+0000","last_clean":"2023-12-01T01:55:27.945759+0000","last_became_active":"2023-12-01T01:55:25.271309+0000","last_became_peered":"2023-12-01T01:55:25.271309+0000","last_unstale":"2023-12-01T01:55:27.945759+0000","last_undegraded":"2023-12-01T01:55:27.945759+0000","last_fullsized":"2023-12-01T01:55:27.945759+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:51:04.612470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.272142+0000","last_change":"2023-12-01T01:55:25.288087+0000","last_active":"2023-12-01T01:55:26.272142+0000","last_peered":"2023-12-01T01:55:26.272142+0000","last_clean":"2023-12-01T01:55:26.272142+0000","last_became_active":"2023-12-01T01:55:25.287796+0000","last_became_peered":"2023-12-01T01:55:25.287796+0000","last_unstale":"2023-12-01T01:55:26.272142+0000","last_undegraded":"2023-12-01T01:55:26.272142+0000","last_fullsized":"2023-12-01T01:55:26.272142+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:21:26.563798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.946187+0000","last_change":"2023-12-01T01:55:25.298422+0000","last_active":"2023-12-01T01:55:27.946187+0000","last_peered":"2023-12-01T01:55:27.946187+0000","last_clean":"2023-12-01T01:55:27.946187+0000","last_became_active":"2023-12-01T01:55:25.298241+0000","last_became_peered":"2023-12-01T01:55:25.298241+0000","last_unstale":"2023-12-01T01:55:27.946187+0000","last_undegraded":"2023-12-01T01:55:27.946187+0000","last_fullsized":"2023-12-01T01:55:27.946187+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:18:08.684561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.947297+0000","last_change":"2023-12-01T01:55:25.271114+0000","last_active":"2023-12-01T01:55:27.947297+0000","last_peered":"2023-12-01T01:55:27.947297+0000","last_clean":"2023-12-01T01:55:27.947297+0000","last_became_active":"2023-12-01T01:55:25.270852+0000","last_became_peered":"2023-12-01T01:55:25.270852+0000","last_unstale":"2023-12-01T01:55:27.947297+0000","last_undegraded":"2023-12-01T01:55:27.947297+0000","last_fullsized":"2023-12-01T01:55:27.947297+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:33.315741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.237135+0000","last_change":"2023-12-01T01:55:25.299619+0000","last_active":"2023-12-01T01:55:27.237135+0000","last_peered":"2023-12-01T01:55:27.237135+0000","last_clean":"2023-12-01T01:55:27.237135+0000","last_became_active":"2023-12-01T01:55:25.299290+0000","last_became_peered":"2023-12-01T01:55:25.299290+0000","last_unstale":"2023-12-01T01:55:27.237135+0000","last_undegraded":"2023-12-01T01:55:27.237135+0000","last_fullsized":"2023-12-01T01:55:27.237135+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:44.114061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.269633+0000","last_change":"2023-12-01T01:55:25.282072+0000","last_active":"2023-12-01T01:55:26.269633+0000","last_peered":"2023-12-01T01:55:26.269633+0000","last_clean":"2023-12-01T01:55:26.269633+0000","last_became_active":"2023-12-01T01:55:25.281787+0000","last_became_peered":"2023-12-01T01:55:25.281787+0000","last_unstale":"2023-12-01T01:55:26.269633+0000","last_undegraded":"2023-12-01T01:55:26.269633+0000","last_fullsized":"2023-12-01T01:55:26.269633+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:56:34.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.943417+0000","last_change":"2023-12-01T01:55:25.273284+0000","last_active":"2023-12-01T01:55:27.943417+0000","last_peered":"2023-12-01T01:55:27.943417+0000","last_clean":"2023-12-01T01:55:27.943417+0000","last_became_active":"2023-12-01T01:55:25.273026+0000","last_became_peered":"2023-12-01T01:55:25.273026+0000","last_unstale":"2023-12-01T01:55:27.943417+0000","last_undegraded":"2023-12-01T01:55:27.943417+0000","last_fullsized":"2023-12-01T01:55:27.943417+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:46.937664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.945868+0000","last_change":"2023-12-01T01:55:25.301055+0000","last_active":"2023-12-01T01:55:27.945868+0000","last_peered":"2023-12-01T01:55:27.945868+0000","last_clean":"2023-12-01T01:55:27.945868+0000","last_became_active":"2023-12-01T01:55:25.300736+0000","last_became_peered":"2023-12-01T01:55:25.300736+0000","last_unstale":"2023-12-01T01:55:27.945868+0000","last_undegraded":"2023-12-01T01:55:27.945868+0000","last_fullsized":"2023-12-01T01:55:27.945868+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:55.675484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.945513+0000","last_change":"2023-12-01T01:55:25.291595+0000","last_active":"2023-12-01T01:55:27.945513+0000","last_peered":"2023-12-01T01:55:27.945513+0000","last_clean":"2023-12-01T01:55:27.945513+0000","last_became_active":"2023-12-01T01:55:25.291348+0000","last_became_peered":"2023-12-01T01:55:25.291348+0000","last_unstale":"2023-12-01T01:55:27.945513+0000","last_undegraded":"2023-12-01T01:55:27.945513+0000","last_fullsized":"2023-12-01T01:55:27.945513+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:01:16.787345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.943105+0000","last_change":"2023-12-01T01:55:25.292358+0000","last_active":"2023-12-01T01:55:27.943105+0000","last_peered":"2023-12-01T01:55:27.943105+0000","last_clean":"2023-12-01T01:55:27.943105+0000","last_became_active":"2023-12-01T01:55:25.292075+0000","last_became_peered":"2023-12-01T01:55:25.292075+0000","last_unstale":"2023-12-01T01:55:27.943105+0000","last_undegraded":"2023-12-01T01:55:27.943105+0000","last_fullsized":"2023-12-01T01:55:27.943105+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:50:07.178583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.944532+0000","last_change":"2023-12-01T01:55:25.290922+0000","last_active":"2023-12-01T01:55:27.944532+0000","last_peered":"2023-12-01T01:55:27.944532+0000","last_clean":"2023-12-01T01:55:27.944532+0000","last_became_active":"2023-12-01T01:55:25.290644+0000","last_became_peered":"2023-12-01T01:55:25.290644+0000","last_unstale":"2023-12-01T01:55:27.944532+0000","last_undegraded":"2023-12-01T01:55:27.944532+0000","last_fullsized":"2023-12-01T01:55:27.944532+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:08.030156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.268248+0000","last_change":"2023-12-01T01:55:25.281856+0000","last_active":"2023-12-01T01:55:26.268248+0000","last_peered":"2023-12-01T01:55:26.268248+0000","last_clean":"2023-12-01T01:55:26.268248+0000","last_became_active":"2023-12-01T01:55:25.274440+0000","last_became_peered":"2023-12-01T01:55:25.274440+0000","last_unstale":"2023-12-01T01:55:26.268248+0000","last_undegraded":"2023-12-01T01:55:26.268248+0000","last_fullsized":"2023-12-01T01:55:26.268248+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:45:54.033304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.946799+0000","last_change":"2023-12-01T01:55:25.283107+0000","last_active":"2023-12-01T01:55:27.946799+0000","last_peered":"2023-12-01T01:55:27.946799+0000","last_clean":"2023-12-01T01:55:27.946799+0000","last_became_active":"2023-12-01T01:55:25.282376+0000","last_became_peered":"2023-12-01T01:55:25.282376+0000","last_unstale":"2023-12-01T01:55:27.946799+0000","last_undegraded":"2023-12-01T01:55:27.946799+0000","last_fullsized":"2023-12-01T01:55:27.946799+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:14.285246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.269539+0000","last_change":"2023-12-01T01:55:25.296043+0000","last_active":"2023-12-01T01:55:26.269539+0000","last_peered":"2023-12-01T01:55:26.269539+0000","last_clean":"2023-12-01T01:55:26.269539+0000","last_became_active":"2023-12-01T01:55:25.295727+0000","last_became_peered":"2023-12-01T01:55:25.295727+0000","last_unstale":"2023-12-01T01:55:26.269539+0000","last_undegraded":"2023-12-01T01:55:26.269539+0000","last_fullsized":"2023-12-01T01:55:26.269539+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:03:09.670463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.945177+0000","last_change":"2023-12-01T01:55:25.283130+0000","last_active":"2023-12-01T01:55:27.945177+0000","last_peered":"2023-12-01T01:55:27.945177+0000","last_clean":"2023-12-01T01:55:27.945177+0000","last_became_active":"2023-12-01T01:55:25.282869+0000","last_became_peered":"2023-12-01T01:55:25.282869+0000","last_unstale":"2023-12-01T01:55:27.945177+0000","last_undegraded":"2023-12-01T01:55:27.945177+0000","last_fullsized":"2023-12-01T01:55:27.945177+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:21:11.003595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.268512+0000","last_change":"2023-12-01T01:55:25.284405+0000","last_active":"2023-12-01T01:55:26.268512+0000","last_peered":"2023-12-01T01:55:26.268512+0000","last_clean":"2023-12-01T01:55:26.268512+0000","last_became_active":"2023-12-01T01:55:25.283170+0000","last_became_peered":"2023-12-01T01:55:25.283170+0000","last_unstale":"2023-12-01T01:55:26.268512+0000","last_undegraded":"2023-12-01T01:55:26.268512+0000","last_fullsized":"2023-12-01T01:55:26.268512+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:48:34.017939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.271585+0000","last_change":"2023-12-01T01:55:25.300202+0000","last_active":"2023-12-01T01:55:26.271585+0000","last_peered":"2023-12-01T01:55:26.271585+0000","last_clean":"2023-12-01T01:55:26.271585+0000","last_became_active":"2023-12-01T01:55:25.297840+0000","last_became_peered":"2023-12-01T01:55:25.297840+0000","last_unstale":"2023-12-01T01:55:26.271585+0000","last_undegraded":"2023-12-01T01:55:26.271585+0000","last_fullsized":"2023-12-01T01:55:26.271585+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:42:36.853988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.269384+0000","last_change":"2023-12-01T01:55:25.293323+0000","last_active":"2023-12-01T01:55:26.269384+0000","last_peered":"2023-12-01T01:55:26.269384+0000","last_clean":"2023-12-01T01:55:26.269384+0000","last_became_active":"2023-12-01T01:55:25.293038+0000","last_became_peered":"2023-12-01T01:55:25.293038+0000","last_unstale":"2023-12-01T01:55:26.269384+0000","last_undegraded":"2023-12-01T01:55:26.269384+0000","last_fullsized":"2023-12-01T01:55:26.269384+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:36:59.415097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.943904+0000","last_change":"2023-12-01T01:55:25.271815+0000","last_active":"2023-12-01T01:55:27.943904+0000","last_peered":"2023-12-01T01:55:27.943904+0000","last_clean":"2023-12-01T01:55:27.943904+0000","last_became_active":"2023-12-01T01:55:25.271543+0000","last_became_peered":"2023-12-01T01:55:25.271543+0000","last_unstale":"2023-12-01T01:55:27.943904+0000","last_undegraded":"2023-12-01T01:55:27.943904+0000","last_fullsized":"2023-12-01T01:55:27.943904+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:35:11.252707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.943995+0000","last_change":"2023-12-01T01:55:25.284042+0000","last_active":"2023-12-01T01:55:27.943995+0000","last_peered":"2023-12-01T01:55:27.943995+0000","last_clean":"2023-12-01T01:55:27.943995+0000","last_became_active":"2023-12-01T01:55:25.283823+0000","last_became_peered":"2023-12-01T01:55:25.283823+0000","last_unstale":"2023-12-01T01:55:27.943995+0000","last_undegraded":"2023-12-01T01:55:27.943995+0000","last_fullsized":"2023-12-01T01:55:27.943995+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:22:53.603845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.237718+0000","last_change":"2023-12-01T01:55:25.285767+0000","last_active":"2023-12-01T01:55:27.237718+0000","last_peered":"2023-12-01T01:55:27.237718+0000","last_clean":"2023-12-01T01:55:27.237718+0000","last_became_active":"2023-12-01T01:55:25.285501+0000","last_became_peered":"2023-12-01T01:55:25.285501+0000","last_unstale":"2023-12-01T01:55:27.237718+0000","last_undegraded":"2023-12-01T01:55:27.237718+0000","last_fullsized":"2023-12-01T01:55:27.237718+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:45:54.033304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.269820+0000","last_change":"2023-12-01T01:55:25.282904+0000","last_active":"2023-12-01T01:55:26.269820+0000","last_peered":"2023-12-01T01:55:26.269820+0000","last_clean":"2023-12-01T01:55:26.269820+0000","last_became_active":"2023-12-01T01:55:25.282626+0000","last_became_peered":"2023-12-01T01:55:25.282626+0000","last_unstale":"2023-12-01T01:55:26.269820+0000","last_undegraded":"2023-12-01T01:55:26.269820+0000","last_fullsized":"2023-12-01T01:55:26.269820+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:51:25.059663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.237946+0000","last_change":"2023-12-01T01:55:25.301530+0000","last_active":"2023-12-01T01:55:27.237946+0000","last_peered":"2023-12-01T01:55:27.237946+0000","last_clean":"2023-12-01T01:55:27.237946+0000","last_became_active":"2023-12-01T01:55:25.301242+0000","last_became_peered":"2023-12-01T01:55:25.301242+0000","last_unstale":"2023-12-01T01:55:27.237946+0000","last_undegraded":"2023-12-01T01:55:27.237946+0000","last_fullsized":"2023-12-01T01:55:27.237946+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:57:33.963850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"140'4","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.279841+0000","last_change":"2023-12-01T01:55:24.815269+0000","last_active":"2023-12-01T01:55:25.279841+0000","last_peered":"2023-12-01T01:55:25.279841+0000","last_clean":"2023-12-01T01:55:25.279841+0000","last_became_active":"2023-12-01T01:55:24.814402+0000","last_became_peered":"2023-12-01T01:55:24.814402+0000","last_unstale":"2023-12-01T01:55:25.279841+0000","last_undegraded":"2023-12-01T01:55:25.279841+0000","last_fullsized":"2023-12-01T01:55:25.279841+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:49:21.973740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.947702+0000","last_change":"2023-12-01T01:55:25.285780+0000","last_active":"2023-12-01T01:55:27.947702+0000","last_peered":"2023-12-01T01:55:27.947702+0000","last_clean":"2023-12-01T01:55:27.947702+0000","last_became_active":"2023-12-01T01:55:25.284766+0000","last_became_peered":"2023-12-01T01:55:25.284766+0000","last_unstale":"2023-12-01T01:55:27.947702+0000","last_undegraded":"2023-12-01T01:55:27.947702+0000","last_fullsized":"2023-12-01T01:55:27.947702+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:09.877705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"140'1","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.285216+0000","last_change":"2023-12-01T01:55:24.811708+0000","last_active":"2023-12-01T01:55:25.285216+0000","last_peered":"2023-12-01T01:55:25.285216+0000","last_clean":"2023-12-01T01:55:25.285216+0000","last_became_active":"2023-12-01T01:55:24.811444+0000","last_became_peered":"2023-12-01T01:55:24.811444+0000","last_unstale":"2023-12-01T01:55:25.285216+0000","last_undegraded":"2023-12-01T01:55:25.285216+0000","last_fullsized":"2023-12-01T01:55:25.285216+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:31:34.839958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.270208+0000","last_change":"2023-12-01T01:55:25.298016+0000","last_active":"2023-12-01T01:55:26.270208+0000","last_peered":"2023-12-01T01:55:26.270208+0000","last_clean":"2023-12-01T01:55:26.270208+0000","last_became_active":"2023-12-01T01:55:25.297706+0000","last_became_peered":"2023-12-01T01:55:25.297706+0000","last_unstale":"2023-12-01T01:55:26.270208+0000","last_undegraded":"2023-12-01T01:55:26.270208+0000","last_fullsized":"2023-12-01T01:55:26.270208+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:25:15.521993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"140'2","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.298553+0000","last_change":"2023-12-01T01:55:24.814656+0000","last_active":"2023-12-01T01:55:25.298553+0000","last_peered":"2023-12-01T01:55:25.298553+0000","last_clean":"2023-12-01T01:55:25.298553+0000","last_became_active":"2023-12-01T01:55:24.814328+0000","last_became_peered":"2023-12-01T01:55:24.814328+0000","last_unstale":"2023-12-01T01:55:25.298553+0000","last_undegraded":"2023-12-01T01:55:25.298553+0000","last_fullsized":"2023-12-01T01:55:25.298553+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:51:24.042108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.945239+0000","last_change":"2023-12-01T01:55:25.297048+0000","last_active":"2023-12-01T01:55:27.945239+0000","last_peered":"2023-12-01T01:55:27.945239+0000","last_clean":"2023-12-01T01:55:27.945239+0000","last_became_active":"2023-12-01T01:55:25.296880+0000","last_became_peered":"2023-12-01T01:55:25.296880+0000","last_unstale":"2023-12-01T01:55:27.945239+0000","last_undegraded":"2023-12-01T01:55:27.945239+0000","last_fullsized":"2023-12-01T01:55:27.945239+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:12:15.345468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"140'2","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.303160+0000","last_change":"2023-12-01T01:55:24.814484+0000","last_active":"2023-12-01T01:55:25.303160+0000","last_peered":"2023-12-01T01:55:25.303160+0000","last_clean":"2023-12-01T01:55:25.303160+0000","last_became_active":"2023-12-01T01:55:24.814163+0000","last_became_peered":"2023-12-01T01:55:24.814163+0000","last_unstale":"2023-12-01T01:55:25.303160+0000","last_undegraded":"2023-12-01T01:55:25.303160+0000","last_fullsized":"2023-12-01T01:55:25.303160+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:21:25.546243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.271931+0000","last_change":"2023-12-01T01:55:25.296175+0000","last_active":"2023-12-01T01:55:26.271931+0000","last_peered":"2023-12-01T01:55:26.271931+0000","last_clean":"2023-12-01T01:55:26.271931+0000","last_became_active":"2023-12-01T01:55:25.295888+0000","last_became_peered":"2023-12-01T01:55:25.295888+0000","last_unstale":"2023-12-01T01:55:26.271931+0000","last_undegraded":"2023-12-01T01:55:26.271931+0000","last_fullsized":"2023-12-01T01:55:26.271931+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:59:27.439934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"140'1","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.285230+0000","last_change":"2023-12-01T01:55:24.805341+0000","last_active":"2023-12-01T01:55:25.285230+0000","last_peered":"2023-12-01T01:55:25.285230+0000","last_clean":"2023-12-01T01:55:25.285230+0000","last_became_active":"2023-12-01T01:55:24.805070+0000","last_became_peered":"2023-12-01T01:55:24.805070+0000","last_unstale":"2023-12-01T01:55:25.285230+0000","last_undegraded":"2023-12-01T01:55:25.285230+0000","last_fullsized":"2023-12-01T01:55:25.285230+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:04.786339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.944291+0000","last_change":"2023-12-01T01:55:25.289238+0000","last_active":"2023-12-01T01:55:27.944291+0000","last_peered":"2023-12-01T01:55:27.944291+0000","last_clean":"2023-12-01T01:55:27.944291+0000","last_became_active":"2023-12-01T01:55:25.288961+0000","last_became_peered":"2023-12-01T01:55:25.288961+0000","last_unstale":"2023-12-01T01:55:27.944291+0000","last_undegraded":"2023-12-01T01:55:27.944291+0000","last_fullsized":"2023-12-01T01:55:27.944291+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:40.391900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"140'1","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.293718+0000","last_change":"2023-12-01T01:55:24.810338+0000","last_active":"2023-12-01T01:55:25.293718+0000","last_peered":"2023-12-01T01:55:25.293718+0000","last_clean":"2023-12-01T01:55:25.293718+0000","last_became_active":"2023-12-01T01:55:24.810081+0000","last_became_peered":"2023-12-01T01:55:24.810081+0000","last_unstale":"2023-12-01T01:55:25.293718+0000","last_undegraded":"2023-12-01T01:55:25.293718+0000","last_fullsized":"2023-12-01T01:55:25.293718+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:49:13.404686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.270002+0000","last_change":"2023-12-01T01:55:25.292181+0000","last_active":"2023-12-01T01:55:26.270002+0000","last_peered":"2023-12-01T01:55:26.270002+0000","last_clean":"2023-12-01T01:55:26.270002+0000","last_became_active":"2023-12-01T01:55:25.291131+0000","last_became_peered":"2023-12-01T01:55:25.291131+0000","last_unstale":"2023-12-01T01:55:26.270002+0000","last_undegraded":"2023-12-01T01:55:26.270002+0000","last_fullsized":"2023-12-01T01:55:26.270002+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:14:13.287727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.263767+0000","last_change":"2023-12-01T01:55:24.809388+0000","last_active":"2023-12-01T01:55:25.263767+0000","last_peered":"2023-12-01T01:55:25.263767+0000","last_clean":"2023-12-01T01:55:25.263767+0000","last_became_active":"2023-12-01T01:55:24.809082+0000","last_became_peered":"2023-12-01T01:55:24.809082+0000","last_unstale":"2023-12-01T01:55:25.263767+0000","last_undegraded":"2023-12-01T01:55:25.263767+0000","last_fullsized":"2023-12-01T01:55:25.263767+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:29:19.806110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.943680+0000","last_change":"2023-12-01T01:55:25.285199+0000","last_active":"2023-12-01T01:55:27.943680+0000","last_peered":"2023-12-01T01:55:27.943680+0000","last_clean":"2023-12-01T01:55:27.943680+0000","last_became_active":"2023-12-01T01:55:25.284924+0000","last_became_peered":"2023-12-01T01:55:25.284924+0000","last_unstale":"2023-12-01T01:55:27.943680+0000","last_undegraded":"2023-12-01T01:55:27.943680+0000","last_fullsized":"2023-12-01T01:55:27.943680+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:17:08.225932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.276126+0000","last_change":"2023-12-01T01:55:24.817064+0000","last_active":"2023-12-01T01:55:25.276126+0000","last_peered":"2023-12-01T01:55:25.276126+0000","last_clean":"2023-12-01T01:55:25.276126+0000","last_became_active":"2023-12-01T01:55:24.816779+0000","last_became_peered":"2023-12-01T01:55:24.816779+0000","last_unstale":"2023-12-01T01:55:25.276126+0000","last_undegraded":"2023-12-01T01:55:25.276126+0000","last_fullsized":"2023-12-01T01:55:25.276126+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:02:49.343248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.945948+0000","last_change":"2023-12-01T01:55:25.301995+0000","last_active":"2023-12-01T01:55:27.945948+0000","last_peered":"2023-12-01T01:55:27.945948+0000","last_clean":"2023-12-01T01:55:27.945948+0000","last_became_active":"2023-12-01T01:55:25.301727+0000","last_became_peered":"2023-12-01T01:55:25.301727+0000","last_unstale":"2023-12-01T01:55:27.945948+0000","last_undegraded":"2023-12-01T01:55:27.945948+0000","last_fullsized":"2023-12-01T01:55:27.945948+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:20:24.665497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.286119+0000","last_change":"2023-12-01T01:55:24.810315+0000","last_active":"2023-12-01T01:55:25.286119+0000","last_peered":"2023-12-01T01:55:25.286119+0000","last_clean":"2023-12-01T01:55:25.286119+0000","last_became_active":"2023-12-01T01:55:24.808334+0000","last_became_peered":"2023-12-01T01:55:24.808334+0000","last_unstale":"2023-12-01T01:55:25.286119+0000","last_undegraded":"2023-12-01T01:55:25.286119+0000","last_fullsized":"2023-12-01T01:55:25.286119+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:14:12.270172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.946670+0000","last_change":"2023-12-01T01:55:25.298419+0000","last_active":"2023-12-01T01:55:27.946670+0000","last_peered":"2023-12-01T01:55:27.946670+0000","last_clean":"2023-12-01T01:55:27.946670+0000","last_became_active":"2023-12-01T01:55:25.298104+0000","last_became_peered":"2023-12-01T01:55:25.298104+0000","last_unstale":"2023-12-01T01:55:27.946670+0000","last_undegraded":"2023-12-01T01:55:27.946670+0000","last_fullsized":"2023-12-01T01:55:27.946670+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:14.285246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.280612+0000","last_change":"2023-12-01T01:55:24.815185+0000","last_active":"2023-12-01T01:55:25.280612+0000","last_peered":"2023-12-01T01:55:25.280612+0000","last_clean":"2023-12-01T01:55:25.280612+0000","last_became_active":"2023-12-01T01:55:24.814841+0000","last_became_peered":"2023-12-01T01:55:24.814841+0000","last_unstale":"2023-12-01T01:55:25.280612+0000","last_undegraded":"2023-12-01T01:55:25.280612+0000","last_fullsized":"2023-12-01T01:55:25.280612+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:54:42.635447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.269669+0000","last_change":"2023-12-01T01:55:25.286391+0000","last_active":"2023-12-01T01:55:26.269669+0000","last_peered":"2023-12-01T01:55:26.269669+0000","last_clean":"2023-12-01T01:55:26.269669+0000","last_became_active":"2023-12-01T01:55:25.286123+0000","last_became_peered":"2023-12-01T01:55:25.286123+0000","last_unstale":"2023-12-01T01:55:26.269669+0000","last_undegraded":"2023-12-01T01:55:26.269669+0000","last_fullsized":"2023-12-01T01:55:26.269669+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:23:35.768127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"140'3","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.299030+0000","last_change":"2023-12-01T01:55:24.816658+0000","last_active":"2023-12-01T01:55:25.299030+0000","last_peered":"2023-12-01T01:55:25.299030+0000","last_clean":"2023-12-01T01:55:25.299030+0000","last_became_active":"2023-12-01T01:55:24.816350+0000","last_became_peered":"2023-12-01T01:55:24.816350+0000","last_unstale":"2023-12-01T01:55:25.299030+0000","last_undegraded":"2023-12-01T01:55:25.299030+0000","last_fullsized":"2023-12-01T01:55:25.299030+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:26:29.479647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.946006+0000","last_change":"2023-12-01T01:55:25.271064+0000","last_active":"2023-12-01T01:55:27.946006+0000","last_peered":"2023-12-01T01:55:27.946006+0000","last_clean":"2023-12-01T01:55:27.946006+0000","last_became_active":"2023-12-01T01:55:25.270815+0000","last_became_peered":"2023-12-01T01:55:25.270815+0000","last_unstale":"2023-12-01T01:55:27.946006+0000","last_undegraded":"2023-12-01T01:55:27.946006+0000","last_fullsized":"2023-12-01T01:55:27.946006+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:03:59.772144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"140'6","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.306226+0000","last_change":"2023-12-01T01:55:24.811180+0000","last_active":"2023-12-01T01:55:25.306226+0000","last_peered":"2023-12-01T01:55:25.306226+0000","last_clean":"2023-12-01T01:55:25.306226+0000","last_became_active":"2023-12-01T01:55:24.808088+0000","last_became_peered":"2023-12-01T01:55:24.808088+0000","last_unstale":"2023-12-01T01:55:25.306226+0000","last_undegraded":"2023-12-01T01:55:25.306226+0000","last_fullsized":"2023-12-01T01:55:25.306226+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:31:34.839958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.271005+0000","last_change":"2023-12-01T01:55:25.282202+0000","last_active":"2023-12-01T01:55:26.271005+0000","last_peered":"2023-12-01T01:55:26.271005+0000","last_clean":"2023-12-01T01:55:26.271005+0000","last_became_active":"2023-12-01T01:55:25.281857+0000","last_became_peered":"2023-12-01T01:55:25.281857+0000","last_unstale":"2023-12-01T01:55:26.271005+0000","last_undegraded":"2023-12-01T01:55:26.271005+0000","last_fullsized":"2023-12-01T01:55:26.271005+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:01:55.409123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"140'2","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.297090+0000","last_change":"2023-12-01T01:55:24.815596+0000","last_active":"2023-12-01T01:55:25.297090+0000","last_peered":"2023-12-01T01:55:25.297090+0000","last_clean":"2023-12-01T01:55:25.297090+0000","last_became_active":"2023-12-01T01:55:24.815258+0000","last_became_peered":"2023-12-01T01:55:24.815258+0000","last_unstale":"2023-12-01T01:55:25.297090+0000","last_undegraded":"2023-12-01T01:55:25.297090+0000","last_fullsized":"2023-12-01T01:55:25.297090+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:38:25.116989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.271340+0000","last_change":"2023-12-01T01:55:25.289212+0000","last_active":"2023-12-01T01:55:26.271340+0000","last_peered":"2023-12-01T01:55:26.271340+0000","last_clean":"2023-12-01T01:55:26.271340+0000","last_became_active":"2023-12-01T01:55:25.285434+0000","last_became_peered":"2023-12-01T01:55:25.285434+0000","last_unstale":"2023-12-01T01:55:26.271340+0000","last_undegraded":"2023-12-01T01:55:26.271340+0000","last_fullsized":"2023-12-01T01:55:26.271340+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:01.698286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.262037+0000","last_change":"2023-12-01T01:55:24.801751+0000","last_active":"2023-12-01T01:55:25.262037+0000","last_peered":"2023-12-01T01:55:25.262037+0000","last_clean":"2023-12-01T01:55:25.262037+0000","last_became_active":"2023-12-01T01:55:24.801445+0000","last_became_peered":"2023-12-01T01:55:24.801445+0000","last_unstale":"2023-12-01T01:55:25.262037+0000","last_undegraded":"2023-12-01T01:55:25.262037+0000","last_fullsized":"2023-12-01T01:55:25.262037+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:31:31.035582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.270743+0000","last_change":"2023-12-01T01:55:25.284036+0000","last_active":"2023-12-01T01:55:26.270743+0000","last_peered":"2023-12-01T01:55:26.270743+0000","last_clean":"2023-12-01T01:55:26.270743+0000","last_became_active":"2023-12-01T01:55:25.283724+0000","last_became_peered":"2023-12-01T01:55:25.283724+0000","last_unstale":"2023-12-01T01:55:26.270743+0000","last_undegraded":"2023-12-01T01:55:26.270743+0000","last_fullsized":"2023-12-01T01:55:26.270743+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:33.315741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"140'8","reported_seq":19,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.304365+0000","last_change":"2023-12-01T01:55:24.811567+0000","last_active":"2023-12-01T01:55:25.304365+0000","last_peered":"2023-12-01T01:55:25.304365+0000","last_clean":"2023-12-01T01:55:25.304365+0000","last_became_active":"2023-12-01T01:55:24.811294+0000","last_became_peered":"2023-12-01T01:55:24.811294+0000","last_unstale":"2023-12-01T01:55:25.304365+0000","last_undegraded":"2023-12-01T01:55:25.304365+0000","last_fullsized":"2023-12-01T01:55:25.304365+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:13.267691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.238032+0000","last_change":"2023-12-01T01:55:25.291619+0000","last_active":"2023-12-01T01:55:27.238032+0000","last_peered":"2023-12-01T01:55:27.238032+0000","last_clean":"2023-12-01T01:55:27.238032+0000","last_became_active":"2023-12-01T01:55:25.290901+0000","last_became_peered":"2023-12-01T01:55:25.290901+0000","last_unstale":"2023-12-01T01:55:27.238032+0000","last_undegraded":"2023-12-01T01:55:27.238032+0000","last_fullsized":"2023-12-01T01:55:27.238032+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:00:01.128882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"140'1","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.291564+0000","last_change":"2023-12-01T01:55:24.810276+0000","last_active":"2023-12-01T01:55:25.291564+0000","last_peered":"2023-12-01T01:55:25.291564+0000","last_clean":"2023-12-01T01:55:25.291564+0000","last_became_active":"2023-12-01T01:55:24.809965+0000","last_became_peered":"2023-12-01T01:55:24.809965+0000","last_unstale":"2023-12-01T01:55:25.291564+0000","last_undegraded":"2023-12-01T01:55:25.291564+0000","last_fullsized":"2023-12-01T01:55:25.291564+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:09:00.680731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.238040+0000","last_change":"2023-12-01T01:55:25.303841+0000","last_active":"2023-12-01T01:55:27.238040+0000","last_peered":"2023-12-01T01:55:27.238040+0000","last_clean":"2023-12-01T01:55:27.238040+0000","last_became_active":"2023-12-01T01:55:25.303574+0000","last_became_peered":"2023-12-01T01:55:25.303574+0000","last_unstale":"2023-12-01T01:55:27.238040+0000","last_undegraded":"2023-12-01T01:55:27.238040+0000","last_fullsized":"2023-12-01T01:55:27.238040+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:27:14.539970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"140'2","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.296201+0000","last_change":"2023-12-01T01:55:24.817397+0000","last_active":"2023-12-01T01:55:25.296201+0000","last_peered":"2023-12-01T01:55:25.296201+0000","last_clean":"2023-12-01T01:55:25.296201+0000","last_became_active":"2023-12-01T01:55:24.816894+0000","last_became_peered":"2023-12-01T01:55:24.816894+0000","last_unstale":"2023-12-01T01:55:25.296201+0000","last_undegraded":"2023-12-01T01:55:25.296201+0000","last_fullsized":"2023-12-01T01:55:25.296201+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:41.929533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.945230+0000","last_change":"2023-12-01T01:55:25.290390+0000","last_active":"2023-12-01T01:55:27.945230+0000","last_peered":"2023-12-01T01:55:27.945230+0000","last_clean":"2023-12-01T01:55:27.945230+0000","last_became_active":"2023-12-01T01:55:25.289456+0000","last_became_peered":"2023-12-01T01:55:25.289456+0000","last_unstale":"2023-12-01T01:55:27.945230+0000","last_undegraded":"2023-12-01T01:55:27.945230+0000","last_fullsized":"2023-12-01T01:55:27.945230+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:31.433645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"140'1","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.285666+0000","last_change":"2023-12-01T01:55:24.814455+0000","last_active":"2023-12-01T01:55:25.285666+0000","last_peered":"2023-12-01T01:55:25.285666+0000","last_clean":"2023-12-01T01:55:25.285666+0000","last_became_active":"2023-12-01T01:55:24.813354+0000","last_became_peered":"2023-12-01T01:55:24.813354+0000","last_unstale":"2023-12-01T01:55:25.285666+0000","last_undegraded":"2023-12-01T01:55:25.285666+0000","last_fullsized":"2023-12-01T01:55:25.285666+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:14.405529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.946981+0000","last_change":"2023-12-01T01:55:25.272943+0000","last_active":"2023-12-01T01:55:27.946981+0000","last_peered":"2023-12-01T01:55:27.946981+0000","last_clean":"2023-12-01T01:55:27.946981+0000","last_became_active":"2023-12-01T01:55:25.272671+0000","last_became_peered":"2023-12-01T01:55:25.272671+0000","last_unstale":"2023-12-01T01:55:27.946981+0000","last_undegraded":"2023-12-01T01:55:27.946981+0000","last_fullsized":"2023-12-01T01:55:27.946981+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:18:08.684561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.266243+0000","last_change":"2023-12-01T01:55:24.803857+0000","last_active":"2023-12-01T01:55:25.266243+0000","last_peered":"2023-12-01T01:55:25.266243+0000","last_clean":"2023-12-01T01:55:25.266243+0000","last_became_active":"2023-12-01T01:55:24.803576+0000","last_became_peered":"2023-12-01T01:55:24.803576+0000","last_unstale":"2023-12-01T01:55:25.266243+0000","last_undegraded":"2023-12-01T01:55:25.266243+0000","last_fullsized":"2023-12-01T01:55:25.266243+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:50:06.161028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.944634+0000","last_change":"2023-12-01T01:55:25.279700+0000","last_active":"2023-12-01T01:55:27.944634+0000","last_peered":"2023-12-01T01:55:27.944634+0000","last_clean":"2023-12-01T01:55:27.944634+0000","last_became_active":"2023-12-01T01:55:25.279429+0000","last_became_peered":"2023-12-01T01:55:25.279429+0000","last_unstale":"2023-12-01T01:55:27.944634+0000","last_undegraded":"2023-12-01T01:55:27.944634+0000","last_fullsized":"2023-12-01T01:55:27.944634+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:49:22.991295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"140'2","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.297141+0000","last_change":"2023-12-01T01:55:24.814823+0000","last_active":"2023-12-01T01:55:25.297141+0000","last_peered":"2023-12-01T01:55:25.297141+0000","last_clean":"2023-12-01T01:55:25.297141+0000","last_became_active":"2023-12-01T01:55:24.814523+0000","last_became_peered":"2023-12-01T01:55:24.814523+0000","last_unstale":"2023-12-01T01:55:25.297141+0000","last_undegraded":"2023-12-01T01:55:25.297141+0000","last_fullsized":"2023-12-01T01:55:25.297141+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:00.939658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.237637+0000","last_change":"2023-12-01T01:55:25.289343+0000","last_active":"2023-12-01T01:55:27.237637+0000","last_peered":"2023-12-01T01:55:27.237637+0000","last_clean":"2023-12-01T01:55:27.237637+0000","last_became_active":"2023-12-01T01:55:25.288598+0000","last_became_peered":"2023-12-01T01:55:25.288598+0000","last_unstale":"2023-12-01T01:55:27.237637+0000","last_undegraded":"2023-12-01T01:55:27.237637+0000","last_fullsized":"2023-12-01T01:55:27.237637+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:33:45.189572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.282710+0000","last_change":"2023-12-01T01:55:24.807189+0000","last_active":"2023-12-01T01:55:25.282710+0000","last_peered":"2023-12-01T01:55:25.282710+0000","last_clean":"2023-12-01T01:55:25.282710+0000","last_became_active":"2023-12-01T01:55:24.804563+0000","last_became_peered":"2023-12-01T01:55:24.804563+0000","last_unstale":"2023-12-01T01:55:25.282710+0000","last_undegraded":"2023-12-01T01:55:25.282710+0000","last_fullsized":"2023-12-01T01:55:25.282710+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:50.465362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.943479+0000","last_change":"2023-12-01T01:55:25.294661+0000","last_active":"2023-12-01T01:55:27.943479+0000","last_peered":"2023-12-01T01:55:27.943479+0000","last_clean":"2023-12-01T01:55:27.943479+0000","last_became_active":"2023-12-01T01:55:25.294325+0000","last_became_peered":"2023-12-01T01:55:25.294325+0000","last_unstale":"2023-12-01T01:55:27.943479+0000","last_undegraded":"2023-12-01T01:55:27.943479+0000","last_fullsized":"2023-12-01T01:55:27.943479+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:51.482917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.283566+0000","last_change":"2023-12-01T01:55:24.808613+0000","last_active":"2023-12-01T01:55:25.283566+0000","last_peered":"2023-12-01T01:55:25.283566+0000","last_clean":"2023-12-01T01:55:25.283566+0000","last_became_active":"2023-12-01T01:55:24.808305+0000","last_became_peered":"2023-12-01T01:55:24.808305+0000","last_unstale":"2023-12-01T01:55:25.283566+0000","last_undegraded":"2023-12-01T01:55:25.283566+0000","last_fullsized":"2023-12-01T01:55:25.283566+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:56:33.463990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.271039+0000","last_change":"2023-12-01T01:55:25.290212+0000","last_active":"2023-12-01T01:55:26.271039+0000","last_peered":"2023-12-01T01:55:26.271039+0000","last_clean":"2023-12-01T01:55:26.271039+0000","last_became_active":"2023-12-01T01:55:25.289938+0000","last_became_peered":"2023-12-01T01:55:25.289938+0000","last_unstale":"2023-12-01T01:55:26.271039+0000","last_undegraded":"2023-12-01T01:55:26.271039+0000","last_fullsized":"2023-12-01T01:55:26.271039+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:31:35.857513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.283926+0000","last_change":"2023-12-01T01:55:24.804531+0000","last_active":"2023-12-01T01:55:25.283926+0000","last_peered":"2023-12-01T01:55:25.283926+0000","last_clean":"2023-12-01T01:55:25.283926+0000","last_became_active":"2023-12-01T01:55:24.804203+0000","last_became_peered":"2023-12-01T01:55:24.804203+0000","last_unstale":"2023-12-01T01:55:25.283926+0000","last_undegraded":"2023-12-01T01:55:25.283926+0000","last_fullsized":"2023-12-01T01:55:25.283926+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:40.756808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.947177+0000","last_change":"2023-12-01T01:55:25.296721+0000","last_active":"2023-12-01T01:55:27.947177+0000","last_peered":"2023-12-01T01:55:27.947177+0000","last_clean":"2023-12-01T01:55:27.947177+0000","last_became_active":"2023-12-01T01:55:25.296505+0000","last_became_peered":"2023-12-01T01:55:25.296505+0000","last_unstale":"2023-12-01T01:55:27.947177+0000","last_undegraded":"2023-12-01T01:55:27.947177+0000","last_fullsized":"2023-12-01T01:55:27.947177+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:49:14.422241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"140'2","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.297778+0000","last_change":"2023-12-01T01:55:24.808380+0000","last_active":"2023-12-01T01:55:25.297778+0000","last_peered":"2023-12-01T01:55:25.297778+0000","last_clean":"2023-12-01T01:55:25.297778+0000","last_became_active":"2023-12-01T01:55:24.807856+0000","last_became_peered":"2023-12-01T01:55:24.807856+0000","last_unstale":"2023-12-01T01:55:25.297778+0000","last_undegraded":"2023-12-01T01:55:25.297778+0000","last_fullsized":"2023-12-01T01:55:25.297778+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:56.576935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.268765+0000","last_change":"2023-12-01T01:55:25.287762+0000","last_active":"2023-12-01T01:55:26.268765+0000","last_peered":"2023-12-01T01:55:26.268765+0000","last_clean":"2023-12-01T01:55:26.268765+0000","last_became_active":"2023-12-01T01:55:25.287378+0000","last_became_peered":"2023-12-01T01:55:25.287378+0000","last_unstale":"2023-12-01T01:55:26.268765+0000","last_undegraded":"2023-12-01T01:55:26.268765+0000","last_fullsized":"2023-12-01T01:55:26.268765+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:52:33.717797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.264040+0000","last_change":"2023-12-01T01:55:24.807676+0000","last_active":"2023-12-01T01:55:25.264040+0000","last_peered":"2023-12-01T01:55:25.264040+0000","last_clean":"2023-12-01T01:55:25.264040+0000","last_became_active":"2023-12-01T01:55:24.807042+0000","last_became_peered":"2023-12-01T01:55:24.807042+0000","last_unstale":"2023-12-01T01:55:25.264040+0000","last_undegraded":"2023-12-01T01:55:25.264040+0000","last_fullsized":"2023-12-01T01:55:25.264040+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:32:04.030142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.237947+0000","last_change":"2023-12-01T01:55:25.301266+0000","last_active":"2023-12-01T01:55:27.237947+0000","last_peered":"2023-12-01T01:55:27.237947+0000","last_clean":"2023-12-01T01:55:27.237947+0000","last_became_active":"2023-12-01T01:55:25.300992+0000","last_became_peered":"2023-12-01T01:55:25.300992+0000","last_unstale":"2023-12-01T01:55:27.237947+0000","last_undegraded":"2023-12-01T01:55:27.237947+0000","last_fullsized":"2023-12-01T01:55:27.237947+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:09:42.935312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.286169+0000","last_change":"2023-12-01T01:55:24.807913+0000","last_active":"2023-12-01T01:55:25.286169+0000","last_peered":"2023-12-01T01:55:25.286169+0000","last_clean":"2023-12-01T01:55:25.286169+0000","last_became_active":"2023-12-01T01:55:24.806182+0000","last_became_peered":"2023-12-01T01:55:24.806182+0000","last_unstale":"2023-12-01T01:55:25.286169+0000","last_undegraded":"2023-12-01T01:55:25.286169+0000","last_fullsized":"2023-12-01T01:55:25.286169+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:01:54.391568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.946975+0000","last_change":"2023-12-01T01:55:25.288609+0000","last_active":"2023-12-01T01:55:27.946975+0000","last_peered":"2023-12-01T01:55:27.946975+0000","last_clean":"2023-12-01T01:55:27.946975+0000","last_became_active":"2023-12-01T01:55:25.288394+0000","last_became_peered":"2023-12-01T01:55:25.288394+0000","last_unstale":"2023-12-01T01:55:27.946975+0000","last_undegraded":"2023-12-01T01:55:27.946975+0000","last_fullsized":"2023-12-01T01:55:27.946975+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:43:40.866437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.284640+0000","last_change":"2023-12-01T01:55:24.812007+0000","last_active":"2023-12-01T01:55:25.284640+0000","last_peered":"2023-12-01T01:55:25.284640+0000","last_clean":"2023-12-01T01:55:25.284640+0000","last_became_active":"2023-12-01T01:55:24.811188+0000","last_became_peered":"2023-12-01T01:55:24.811188+0000","last_unstale":"2023-12-01T01:55:25.284640+0000","last_undegraded":"2023-12-01T01:55:25.284640+0000","last_fullsized":"2023-12-01T01:55:25.284640+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:46:29.123760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.944731+0000","last_change":"2023-12-01T01:55:25.289035+0000","last_active":"2023-12-01T01:55:27.944731+0000","last_peered":"2023-12-01T01:55:27.944731+0000","last_clean":"2023-12-01T01:55:27.944731+0000","last_became_active":"2023-12-01T01:55:25.288236+0000","last_became_peered":"2023-12-01T01:55:25.288236+0000","last_unstale":"2023-12-01T01:55:27.944731+0000","last_undegraded":"2023-12-01T01:55:27.944731+0000","last_fullsized":"2023-12-01T01:55:27.944731+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:20:00.181668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.263996+0000","last_change":"2023-12-01T01:55:24.810110+0000","last_active":"2023-12-01T01:55:25.263996+0000","last_peered":"2023-12-01T01:55:25.263996+0000","last_clean":"2023-12-01T01:55:25.263996+0000","last_became_active":"2023-12-01T01:55:24.809839+0000","last_became_peered":"2023-12-01T01:55:24.809839+0000","last_unstale":"2023-12-01T01:55:25.263996+0000","last_undegraded":"2023-12-01T01:55:25.263996+0000","last_fullsized":"2023-12-01T01:55:25.263996+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:27.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.944142+0000","last_change":"2023-12-01T01:55:25.294660+0000","last_active":"2023-12-01T01:55:27.944142+0000","last_peered":"2023-12-01T01:55:27.944142+0000","last_clean":"2023-12-01T01:55:27.944142+0000","last_became_active":"2023-12-01T01:55:25.294148+0000","last_became_peered":"2023-12-01T01:55:25.294148+0000","last_unstale":"2023-12-01T01:55:27.944142+0000","last_undegraded":"2023-12-01T01:55:27.944142+0000","last_fullsized":"2023-12-01T01:55:27.944142+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:20:24.665497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.266661+0000","last_change":"2023-12-01T01:55:24.810353+0000","last_active":"2023-12-01T01:55:25.266661+0000","last_peered":"2023-12-01T01:55:25.266661+0000","last_clean":"2023-12-01T01:55:25.266661+0000","last_became_active":"2023-12-01T01:55:24.810044+0000","last_became_peered":"2023-12-01T01:55:24.810044+0000","last_unstale":"2023-12-01T01:55:25.266661+0000","last_undegraded":"2023-12-01T01:55:25.266661+0000","last_fullsized":"2023-12-01T01:55:25.266661+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:59.164113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:27.943717+0000","last_change":"2023-12-01T01:55:25.279881+0000","last_active":"2023-12-01T01:55:27.943717+0000","last_peered":"2023-12-01T01:55:27.943717+0000","last_clean":"2023-12-01T01:55:27.943717+0000","last_became_active":"2023-12-01T01:55:25.279644+0000","last_became_peered":"2023-12-01T01:55:25.279644+0000","last_unstale":"2023-12-01T01:55:27.943717+0000","last_undegraded":"2023-12-01T01:55:27.943717+0000","last_fullsized":"2023-12-01T01:55:27.943717+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:44:17.371902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.284002+0000","last_change":"2023-12-01T01:55:24.814978+0000","last_active":"2023-12-01T01:55:25.284002+0000","last_peered":"2023-12-01T01:55:25.284002+0000","last_clean":"2023-12-01T01:55:25.284002+0000","last_became_active":"2023-12-01T01:55:24.813486+0000","last_became_peered":"2023-12-01T01:55:24.813486+0000","last_unstale":"2023-12-01T01:55:25.284002+0000","last_undegraded":"2023-12-01T01:55:25.284002+0000","last_fullsized":"2023-12-01T01:55:25.284002+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:08.860150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.267625+0000","last_change":"2023-12-01T01:55:25.295796+0000","last_active":"2023-12-01T01:55:26.267625+0000","last_peered":"2023-12-01T01:55:26.267625+0000","last_clean":"2023-12-01T01:55:26.267625+0000","last_became_active":"2023-12-01T01:55:25.295512+0000","last_became_peered":"2023-12-01T01:55:25.295512+0000","last_unstale":"2023-12-01T01:55:26.267625+0000","last_undegraded":"2023-12-01T01:55:26.267625+0000","last_fullsized":"2023-12-01T01:55:26.267625+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:06:36.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.273037+0000","last_change":"2023-12-01T01:55:24.809764+0000","last_active":"2023-12-01T01:55:25.273037+0000","last_peered":"2023-12-01T01:55:25.273037+0000","last_clean":"2023-12-01T01:55:25.273037+0000","last_became_active":"2023-12-01T01:55:24.809374+0000","last_became_peered":"2023-12-01T01:55:24.809374+0000","last_unstale":"2023-12-01T01:55:25.273037+0000","last_undegraded":"2023-12-01T01:55:25.273037+0000","last_fullsized":"2023-12-01T01:55:25.273037+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:59:26.422380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-12-01T01:55:26.269781+0000","last_change":"2023-12-01T01:55:25.284058+0000","last_active":"2023-12-01T01:55:26.269781+0000","last_peered":"2023-12-01T01:55:26.269781+0000","last_clean":"2023-12-01T01:55:26.269781+0000","last_became_active":"2023-12-01T01:55:25.283588+0000","last_became_peered":"2023-12-01T01:55:25.283588+0000","last_unstale":"2023-12-01T01:55:26.269781+0000","last_undegraded":"2023-12-01T01:55:26.269781+0000","last_fullsized":"2023-12-01T01:55:26.269781+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:24.759522+0000","last_clean_scrub_stamp":"2023-12-01T01:55:24.759522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:51.482917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.281048+0000","last_change":"2023-12-01T01:55:24.807417+0000","last_active":"2023-12-01T01:55:25.281048+0000","last_peered":"2023-12-01T01:55:25.281048+0000","last_clean":"2023-12-01T01:55:25.281048+0000","last_became_active":"2023-12-01T01:55:24.807063+0000","last_became_peered":"2023-12-01T01:55:24.807063+0000","last_unstale":"2023-12-01T01:55:25.281048+0000","last_undegraded":"2023-12-01T01:55:25.281048+0000","last_fullsized":"2023-12-01T01:55:25.281048+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:55:23.741967+0000","last_clean_scrub_stamp":"2023-12-01T01:55:23.741967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:14:12.270172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"128'227","reported_seq":665,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-12-01T01:55:25.262527+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T01:55:25.262527+0000","last_peered":"2023-12-01T01:55:25.262527+0000","last_clean":"2023-12-01T01:55:25.262527+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T01:55:25.262527+0000","last_undegraded":"2023-12-01T01:55:25.262527+0000","last_fullsized":"2023-12-01T01:55:25.262527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575081,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1769472,"data_stored":838974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.0549999999999999,"15min":1.069},"min":{"1min":0.77600000000000002,"5min":0.54900000000000004,"15min":0.41299999999999998},"max":{"1min":1.6359999999999999,"5min":1.7430000000000001,"15min":1.8089999999999999},"last":1.573},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.387,"5min":1.5289999999999999,"15min":2.2029999999999998},"last":0.78300000000000003}]},{"osd":1,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1419999999999999,"15min":1.208},"min":{"1min":0.753,"5min":0.59799999999999998,"15min":0.57299999999999995},"max":{"1min":1.6000000000000001,"5min":1.802,"15min":1.8100000000000001},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2010000000000001,"15min":1.2509999999999999},"min":{"1min":0.84199999999999997,"5min":0.66800000000000004,"15min":0.59199999999999997},"max":{"1min":1.456,"5min":1.629,"15min":1.744},"last":1.288}]},{"osd":2,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1080000000000001,"15min":1.117},"min":{"1min":0.65100000000000002,"5min":0.64100000000000001,"15min":0.50600000000000001},"max":{"1min":1.3740000000000001,"5min":1.607,"15min":1.728},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1910000000000001,"15min":1.21},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.36599999999999999},"max":{"1min":1.6080000000000001,"5min":1.681,"15min":1.9690000000000001},"last":1.627}]},{"osd":3,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.1890000000000001,"15min":1.2310000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.496,"5min":1.637,"15min":2.0550000000000002},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.284,"15min":1.3280000000000001},"min":{"1min":0.90900000000000003,"5min":0.63700000000000001,"15min":0.44900000000000001},"max":{"1min":1.5580000000000001,"5min":1.7490000000000001,"15min":2.1880000000000002},"last":1.6870000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3,"15min":1.288},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.55500000000000005},"max":{"1min":1.663,"5min":1.8300000000000001,"15min":2.2130000000000001},"last":1.649},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.385,"15min":1.399},"min":{"1min":0.93600000000000005,"5min":0.77500000000000002,"15min":0.63300000000000001},"max":{"1min":1.694,"5min":1.8380000000000001,"15min":2.0939999999999999},"last":1.5229999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.232,"15min":1.2450000000000001},"min":{"1min":0.72999999999999998,"5min":0.625,"15min":0.42799999999999999},"max":{"1min":1.5449999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.056},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4850000000000001,"15min":1.5129999999999999},"min":{"1min":1.111,"5min":0.69399999999999995,"15min":0.66900000000000004},"max":{"1min":1.6679999999999999,"5min":1.9379999999999999,"15min":2.2930000000000001},"last":1.73}]},{"osd":6,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3640000000000001,"15min":1.373},"min":{"1min":0.92800000000000005,"5min":0.753,"15min":0.51200000000000001},"max":{"1min":1.5309999999999999,"5min":2.294,"15min":2.3420000000000001},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.3540000000000001,"15min":1.3799999999999999},"min":{"1min":0.82899999999999996,"5min":0.82599999999999996,"15min":0.60899999999999999},"max":{"1min":1.5960000000000001,"5min":1.954,"15min":2.3500000000000001},"last":1.0249999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575082,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1748992,"data_stored":837286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.69799999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4139999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.131,"5min":1.1080000000000001,"15min":1.1319999999999999},"min":{"1min":0.78700000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.4570000000000001,"5min":1.579,"15min":2.5569999999999999},"last":1.115}]},{"osd":1,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.181,"15min":1.167},"min":{"1min":0.80500000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.5489999999999999,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":1.379},{"interface":"front","average":{"1min":1.256,"5min":1.198,"15min":1.208},"min":{"1min":0.90400000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5860000000000001,"5min":2.0720000000000001,"15min":2.0920000000000001},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1240000000000001,"15min":1.1659999999999999},"min":{"1min":0.84899999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.462,"5min":1.6359999999999999,"15min":2.3460000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.216,"15min":1.238},"min":{"1min":0.82199999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.619,"5min":3.024,"15min":3.024},"last":1.262}]},{"osd":3,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.23,"15min":1.2709999999999999},"min":{"1min":0.81799999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.556,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":1.238},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4019999999999999,"15min":1.4039999999999999},"min":{"1min":0.95899999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.7450000000000001,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.3260000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.2949999999999999,"15min":1.278},"min":{"1min":0.92500000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":2.226},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.385,"15min":1.355},"min":{"1min":1.0600000000000001,"5min":0.93600000000000005,"15min":0.78100000000000003},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":2.2839999999999998},"last":1.423}]},{"osd":5,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.421,"15min":1.409},"min":{"1min":1.1699999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.2250000000000001,"5min":2.2250000000000001,"15min":2.2250000000000001},"last":1.966},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.496,"15min":1.486},"min":{"1min":1.3180000000000001,"5min":0.96499999999999997,"15min":0.84099999999999997},"max":{"1min":1.9770000000000001,"5min":1.9790000000000001,"15min":2.8580000000000001},"last":2.1869999999999998}]},{"osd":7,"last update":"Fri Dec 1 01:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.6599999999999999,"5min":1.522,"15min":1.516},"min":{"1min":1.3759999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.958,"5min":1.978,"15min":2.762},"last":1.585},{"interface":"front","average":{"1min":1.748,"5min":1.577,"15min":1.5780000000000001},"min":{"1min":1.4159999999999999,"5min":1.038,"15min":1.038},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.8479999999999999},"last":1.911}]}]},{"osd":5,"up_from":13,"seq":55834575081,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33344,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338496,"statfs":{"total":96636764160,"available":96602619904,"internally_reserved":0,"allocated":2351104,"data_stored":1427746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.111,"15min":1.1080000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.522,"5min":1.8240000000000001,"15min":2.2010000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1060000000000001,"15min":1.1439999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5609999999999999,"5min":1.7130000000000001,"15min":2.153},"last":1.056}]},{"osd":1,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.149,"15min":1.1499999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6040000000000001,"5min":1.77,"15min":1.77},"last":1.492},{"interface":"front","average":{"1min":1.139,"5min":1.145,"15min":1.157},"min":{"1min":0.754,"5min":0.72999999999999998,"15min":0.67800000000000005},"max":{"1min":1.4890000000000001,"5min":1.6519999999999999,"15min":1.7330000000000001},"last":1.236}]},{"osd":2,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.147,"15min":1.1879999999999999},"min":{"1min":0.71999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.532,"5min":1.702,"15min":1.702},"last":1.256},{"interface":"front","average":{"1min":1.248,"5min":1.226,"15min":1.2370000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.605,"5min":1.9239999999999999,"15min":2.0630000000000002},"last":1.5920000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2190000000000001,"15min":1.2629999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.651,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.165},{"interface":"front","average":{"1min":1.294,"5min":1.3109999999999999,"15min":1.304},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.653}]},{"osd":4,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.226,"15min":1.224},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.474,"5min":1.984,"15min":1.984},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.3009999999999999,"15min":1.3340000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.6299999999999999,"5min":1.927,"15min":1.927},"last":1.355}]},{"osd":6,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3300000000000001,"15min":1.325},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.694,"5min":1.885,"15min":1.9790000000000001},"last":1.022},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3799999999999999,"15min":1.371},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.746,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.5349999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.45,"15min":1.4670000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.7210000000000001,"5min":2.0499999999999998,"15min":2.0920000000000001},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.532,"15min":1.5229999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.861,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.702}]}]},{"osd":4,"up_from":13,"seq":55834575081,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40964,"kb_used_data":1724,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330876,"statfs":{"total":96636764160,"available":96594817024,"internally_reserved":0,"allocated":1765376,"data_stored":838940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1100000000000001,"15min":1.107},"min":{"1min":0.77000000000000002,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5509999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1599999999999999,"15min":1.1739999999999999},"min":{"1min":0.80600000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.603,"5min":1.778,"15min":2.1099999999999999},"last":1.3200000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.222,"15min":1.214},"min":{"1min":0.83699999999999997,"5min":0.72399999999999998,"15min":0.65900000000000003},"max":{"1min":1.8460000000000001,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.147,"15min":1.135},"min":{"1min":0.72999999999999998,"5min":0.67300000000000004,"15min":0.64200000000000002},"max":{"1min":1.6060000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.252}]},{"osd":2,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.2150000000000001,"15min":1.202},"min":{"1min":0.91800000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4330000000000001,"5min":1.871,"15min":1.871},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.288,"15min":1.29},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.599,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.474}]},{"osd":3,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.246,"15min":1.244},"min":{"1min":0.93300000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5169999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.153},{"interface":"front","average":{"1min":1.383,"5min":1.3180000000000001,"15min":1.3169999999999999},"min":{"1min":1.071,"5min":0.753,"15min":0.753},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.903},"last":1.355}]},{"osd":5,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.337,"15min":1.321},"min":{"1min":0.95999999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.7010000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.3080000000000001,"15min":1.292},"min":{"1min":1.0189999999999999,"5min":0.90300000000000002,"15min":0.85399999999999998},"max":{"1min":1.627,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.5}]},{"osd":6,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":1.3180000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.7649999999999999,"5min":1.8799999999999999,"15min":1.9950000000000001},"last":1.113},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.47,"15min":1.4790000000000001},"min":{"1min":1.2330000000000001,"5min":0.879,"15min":0.879},"max":{"1min":1.96,"5min":2.306,"15min":2.54},"last":1.206}]},{"osd":7,"last update":"Fri Dec 1 01:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.468,"15min":1.4630000000000001},"min":{"1min":1.2989999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.8799999999999999,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.5620000000000001,"15min":1.5660000000000001},"min":{"1min":1.1299999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"max":{"1min":1.9830000000000001,"5min":2.04,"15min":2.157},"last":1.4430000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575080,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58996,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312844,"statfs":{"total":96636764160,"available":96576352256,"internally_reserved":0,"allocated":1748992,"data_stored":837286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.84699999999999998,"15min":0.85799999999999998},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.472,"5min":1.472,"15min":1.583},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.032,"5min":0.94299999999999995,"15min":0.98699999999999999},"min":{"1min":0.52500000000000002,"5min":0.48099999999999998,"15min":0.435},"max":{"1min":1.395,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.0269999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.99199999999999999,"15min":1.0580000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.573,"5min":1.6200000000000001,"15min":1.6339999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.91300000000000003,"15min":0.94999999999999996},"min":{"1min":0.60599999999999998,"5min":0.48399999999999999,"15min":0.48199999999999998},"max":{"1min":1.3049999999999999,"5min":1.5129999999999999,"15min":1.6519999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.999,"15min":1.0429999999999999},"min":{"1min":0.59399999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3759999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1060000000000001,"15min":1.1799999999999999},"min":{"1min":0.495,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.575,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.27}]},{"osd":4,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1930000000000001,"15min":1.232},"min":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.9319999999999999},"last":1.254},{"interface":"front","average":{"1min":1.228,"5min":1.2110000000000001,"15min":1.228},"min":{"1min":0.85499999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.627,"5min":1.909,"15min":2.0419999999999998},"last":1.363}]},{"osd":5,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2370000000000001,"15min":1.2709999999999999},"min":{"1min":1.071,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.706,"5min":1.7669999999999999,"15min":1.806},"last":1.589},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.28,"15min":1.2989999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7549999999999999,"5min":1.831,"15min":2.1880000000000002},"last":1.6499999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3500000000000001,"15min":1.3700000000000001},"min":{"1min":1.034,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.649,"5min":1.984,"15min":1.984},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.3839999999999999,"15min":1.411},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8109999999999999,"5min":1.9119999999999999,"15min":1.99},"last":1.5660000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:54:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.3660000000000001,"15min":1.411},"min":{"1min":0.873,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8799999999999999,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.615},{"interface":"front","average":{"1min":1.464,"5min":1.419,"15min":1.4430000000000001},"min":{"1min":1.121,"5min":0.87,"15min":0.87},"max":{"1min":1.8879999999999999,"5min":2.0430000000000001,"15min":2.1230000000000002},"last":1.667}]}]},{"osd":2,"up_from":13,"seq":55834575080,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36864,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334976,"statfs":{"total":96636764160,"available":96599015424,"internally_reserved":0,"allocated":1761280,"data_stored":837924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.33900000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2729999999999999,"5min":1.411,"15min":1.5529999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.95299999999999996,"15min":0.99099999999999999},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.502,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.94599999999999995,"15min":0.97899999999999998},"min":{"1min":0.434,"5min":0.375,"15min":0.373},"max":{"1min":1.5,"5min":2.5,"15min":2.5},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.036,"15min":1.075},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.5549999999999999,"5min":1.909,"15min":1.909},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.91000000000000003,"15min":0.99099999999999999},"min":{"1min":0.48799999999999999,"5min":0.45800000000000002,"15min":0.45200000000000001},"max":{"1min":1.3500000000000001,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0649999999999999,"15min":1.1419999999999999},"min":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.571,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.1839999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2290000000000001,"15min":1.2909999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.571,"5min":1.869,"15min":1.9950000000000001},"last":1.056},{"interface":"front","average":{"1min":1.202,"5min":1.1870000000000001,"15min":1.2470000000000001},"min":{"1min":0.71099999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5069999999999999,"5min":1.6879999999999999,"15min":1.8080000000000001},"last":0.91900000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2669999999999999,"15min":1.3069999999999999},"min":{"1min":0.79200000000000004,"5min":0.62,"15min":0.62},"max":{"1min":1.6220000000000001,"5min":2.948,"15min":2.948},"last":1.278},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.294,"15min":1.343},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7370000000000001,"5min":2.016,"15min":2.048},"last":1.1579999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.264,"15min":1.3120000000000001},"min":{"1min":0.77600000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.609,"5min":1.6439999999999999,"15min":1.988},"last":1.141},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.381,"15min":1.4139999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.2309999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.3680000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:54:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.379,"15min":1.4259999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.109},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3160000000000001,"15min":1.371},"min":{"1min":0.83299999999999996,"5min":0.67100000000000004,"15min":0.67000000000000004},"max":{"1min":1.7110000000000001,"5min":1.9259999999999999,"15min":2.101},"last":1.095}]}]},{"osd":1,"up_from":13,"seq":55834575079,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59576,"kb_used_data":2288,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312264,"statfs":{"total":96636764160,"available":96575758336,"internally_reserved":0,"allocated":2342912,"data_stored":1427654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.77900000000000003,"15min":0.83899999999999997},"min":{"1min":0.48199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.5049999999999999},"last":0.75},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.95699999999999996,"15min":1.0069999999999999},"min":{"1min":0.52100000000000002,"5min":0.40500000000000003,"15min":0.39600000000000002},"max":{"1min":1.2749999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.075}]},{"osd":2,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.54400000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.54},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.026,"15min":1.0780000000000001},"min":{"1min":0.64700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.363,"5min":1.7250000000000001,"15min":1.804},"last":0.90500000000000003}]},{"osd":3,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.165,"15min":1.21},"min":{"1min":0.73799999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.51,"5min":1.827,"15min":1.827},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.036,"5min":0.99299999999999999,"15min":1.0409999999999999},"min":{"1min":0.71099999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5069999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.1180000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.175,"15min":1.1839999999999999},"min":{"1min":0.89800000000000002,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.4850000000000001,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.319,"15min":1.335},"min":{"1min":0.90800000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8360000000000001,"5min":1.905,"15min":1.905},"last":1.1819999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2649999999999999,"15min":1.28},"min":{"1min":1.0489999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.706,"5min":1.988,"15min":1.988},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.26,"15min":1.2769999999999999},"min":{"1min":0.95599999999999996,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6439999999999999,"5min":1.6799999999999999,"15min":1.825},"last":1.3300000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3089999999999999,"15min":1.333},"min":{"1min":1.087,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6100000000000001,"5min":1.839,"15min":1.861},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.321,"15min":1.3360000000000001},"min":{"1min":0.88100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.587,"5min":1.742,"15min":1.8109999999999999},"last":1.387}]},{"osd":7,"last update":"Fri Dec 1 01:54:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3899999999999999,"15min":1.4259999999999999},"min":{"1min":0.99099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.823,"5min":1.9099999999999999,"15min":2.0609999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.371,"15min":1.4039999999999999},"min":{"1min":0.93300000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7869999999999999,"5min":1.931,"15min":1.992},"last":1.214}]}]},{"osd":0,"up_from":13,"seq":55834575081,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1757184,"data_stored":837938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:55:24 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.80400000000000005,"15min":0.84599999999999997},"min":{"1min":0.48799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2949999999999999,"5min":1.3520000000000001,"15min":1.8109999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89900000000000002,"15min":0.94999999999999996},"min":{"1min":0.51000000000000001,"5min":0.41799999999999998,"15min":0.36699999999999999},"max":{"1min":1.403,"5min":1.5860000000000001,"15min":1.6910000000000001},"last":1.1930000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:55:24 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.93400000000000005,"15min":0.98599999999999999},"min":{"1min":0.54900000000000004,"5min":0.51500000000000001,"15min":0.42699999999999999},"max":{"1min":1.3380000000000001,"5min":1.4690000000000001,"15min":2.157},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.038,"5min":1.006,"15min":1.044},"min":{"1min":0.59699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.728},"last":0.96399999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:55:24 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.95199999999999996,"15min":0.999},"min":{"1min":0.59499999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.458,"5min":1.4850000000000001,"15min":1.7509999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.105,"15min":1.1479999999999999},"min":{"1min":0.69699999999999995,"5min":0.48899999999999999,"15min":0.46500000000000002},"max":{"1min":1.3899999999999999,"5min":1.7070000000000001,"15min":2.2610000000000001},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:55:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.222,"15min":1.284},"min":{"1min":0.90700000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":2.3719999999999999},"last":1.456},{"interface":"front","average":{"1min":1.165,"5min":1.1699999999999999,"15min":1.234},"min":{"1min":0.76400000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.464,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:55:24 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2909999999999999,"15min":1.3280000000000001},"min":{"1min":0.85099999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.746,"5min":1.841,"15min":2.351},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2609999999999999,"15min":1.3160000000000001},"min":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":2.2909999999999999},"last":1.5600000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:55:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.232,"15min":1.2909999999999999},"min":{"1min":0.71699999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.827,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.341,"15min":1.369},"min":{"1min":0.81799999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7,"5min":1.9159999999999999,"15min":2.4380000000000002},"last":0.85199999999999998}]},{"osd":7,"last update":"Fri Dec 1 01:55:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.343,"15min":1.381},"min":{"1min":0.80400000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.4279999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.329,"15min":1.371},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.7609999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.3999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:55:30.208 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T01:55:30.209 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T01:55:30.339 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:55:30.347 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:55:30.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.470+0000 7f8019069640 1 Processor -- start 2023-12-01T01:55:30.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.470+0000 7f8019069640 1 -- start start 2023-12-01T01:55:30.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.471+0000 7f8019069640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801406d9c0 0x7f80140b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:30.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.471+0000 7f8019069640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80140ccf90 con 0x7f8014071d10 2023-12-01T01:55:30.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.471+0000 7f8019069640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80140cd550 con 0x7f801415ad00 2023-12-01T01:55:30.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.471+0000 7f8019069640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801406de50 con 0x7f801406d9c0 2023-12-01T01:55:30.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.472+0000 7f801259c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8014071d10 0x7f80140cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36450/0 (socket says 172.21.15.115:36450) 2023-12-01T01:55:30.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.472+0000 7f801259c640 1 -- 172.21.15.115:0/4102217805 learned_addr learned my addr 172.21.15.115:0/4102217805 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:30.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.472+0000 7f801359e640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801406d9c0 0x7f80140b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:30.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.472+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936097868 0 0) 0x7f80140ccf90 con 0x7f8014071d10 2023-12-01T01:55:30.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.472+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8000003520 con 0x7f8014071d10 2023-12-01T01:55:30.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2015463933 0 0) 0x7f8000003520 con 0x7f8014071d10 2023-12-01T01:55:30.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801406d9c0 msgr2=0x7f80140b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:30.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801406d9c0 0x7f80140b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 >> v1:172.21.15.142:6789/0 conn(0x7f801415ad00 legacy=0x7f8014071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801406fc40 con 0x7f8014071d10 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f7ffc001670 con 0x7f8014071d10 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 >> v1:172.21.15.115:6789/0 conn(0x7f8014071d10 legacy=0x7f80140cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80000039a0 0x7f8000003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801406d9c0 0x7f80000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 0x7f8000005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800000bb40 con 0x7f8000004d00 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800000bcb0 con 0x7f80000039a0 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800000be70 con 0x7f801406d9c0 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f801359e640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 0x7f8000005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f8012d9d640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80000039a0 0x7f8000003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:30.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.473+0000 7f801259c640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801406d9c0 0x7f80000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:30.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.474+0000 7f801359e640 1 -- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801406d9c0 msgr2=0x7f80000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:30.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.474+0000 7f801359e640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f801406d9c0 0x7f80000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.474+0000 7f801359e640 1 -- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80000039a0 msgr2=0x7f8000003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:30.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.474+0000 7f801359e640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80000039a0 0x7f8000003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.474+0000 7f801359e640 1 -- 172.21.15.115:0/4102217805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000003520 con 0x7f8000004d00 2023-12-01T01:55:30.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.475+0000 7f801359e640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 0x7f8000005180 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f800c009ac0 tx=0x7f80000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e32aff62ab26162 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:30.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.475+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c02c6b0 con 0x7f8000004d00 2023-12-01T01:55:30.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.475+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c02c810 con 0x7f8000004d00 2023-12-01T01:55:30.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.475+0000 7f8011d9b640 1 -- 172.21.15.115:0/4102217805 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c02cba0 con 0x7f8000004d00 2023-12-01T01:55:30.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.476+0000 7f8019069640 1 -- 172.21.15.115:0/4102217805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 msgr2=0x7f8000005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:30.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.476+0000 7f8019069640 1 --2- 172.21.15.115:0/4102217805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 0x7f8000005180 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f800c009ac0 tx=0x7f80000057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.476+0000 7f801359e640 1 -- 172.21.15.115:0/4102217805 reap_dead start 2023-12-01T01:55:30.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.476+0000 7f8019069640 1 -- 172.21.15.115:0/4102217805 shutdown_connections 2023-12-01T01:55:30.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.476+0000 7f8019069640 1 -- 172.21.15.115:0/4102217805 >> 172.21.15.115:0/4102217805 conn(0x7f80140cbb10 msgr2=0x7f80140cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:30.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.476+0000 7f8019069640 1 -- 172.21.15.115:0/4102217805 shutdown_connections 2023-12-01T01:55:30.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.476+0000 7f8019069640 1 -- 172.21.15.115:0/4102217805 wait complete. 2023-12-01T01:55:30.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.477+0000 7f8019069640 1 Processor -- start 2023-12-01T01:55:30.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.478+0000 7f8019069640 1 -- start start 2023-12-01T01:55:30.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.478+0000 7f8019069640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 0x7f80140b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:30.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8019069640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80140af670 0x7f8014159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:30.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8019069640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80140ae0a0 0x7f80140c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:30.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8019069640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80140ae5d0 con 0x7f8000004d00 2023-12-01T01:55:30.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8019069640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80140ae8b0 con 0x7f80140ae0a0 2023-12-01T01:55:30.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8019069640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80140aeb90 con 0x7f80140af670 2023-12-01T01:55:30.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8012d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 0x7f80140b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:30.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8012d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 0x7f80140b01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55874/0 (socket says 172.21.15.115:55874) 2023-12-01T01:55:30.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8012d9d640 1 -- 172.21.15.115:0/3669162446 learned_addr learned my addr 172.21.15.115:0/3669162446 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:30.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f801259c640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80140af670 0x7f8014159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:30.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f801359e640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80140ae0a0 0x7f80140c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:30.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8012d9d640 1 -- 172.21.15.115:0/3669162446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80140af670 msgr2=0x7f8014159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:30.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8012d9d640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80140af670 0x7f8014159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8012d9d640 1 -- 172.21.15.115:0/3669162446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80140ae0a0 msgr2=0x7f80140c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8012d9d640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80140ae0a0 0x7f80140c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.479+0000 7f8012d9d640 1 -- 172.21.15.115:0/3669162446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140aee70 con 0x7f8000004d00 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.480+0000 7f8012d9d640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 0x7f80140b01e0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f800400e8c0 tx=0x7f800400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.480+0000 7f800b7fe640 1 -- 172.21.15.115:0/3669162446 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004003b80 con 0x7f8000004d00 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.480+0000 7f800b7fe640 1 -- 172.21.15.115:0/3669162446 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8004003ce0 con 0x7f8000004d00 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.480+0000 7f800b7fe640 1 -- 172.21.15.115:0/3669162446 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040052f0 con 0x7f8000004d00 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.480+0000 7f8019069640 1 -- 172.21.15.115:0/3669162446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80140af150 con 0x7f8000004d00 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.480+0000 7f8019069640 1 -- 172.21.15.115:0/3669162446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8014159da0 con 0x7f8000004d00 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.481+0000 7f8019069640 1 -- 172.21.15.115:0/3669162446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801404e890 con 0x7f8000004d00 2023-12-01T01:55:30.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.482+0000 7f800b7fe640 1 -- 172.21.15.115:0/3669162446 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8004005450 con 0x7f8000004d00 2023-12-01T01:55:30.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.483+0000 7f800b7fe640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff406a550 0x7f7ff406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:30.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.483+0000 7f800b7fe640 1 -- 172.21.15.115:0/3669162446 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8004088c20 con 0x7f8000004d00 2023-12-01T01:55:30.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.483+0000 7f801259c640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff406a550 0x7f7ff406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:30.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.484+0000 7f801259c640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff406a550 0x7f7ff406ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f80140afa50 tx=0x7f7ffc009040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:30.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.487+0000 7f800b7fe640 1 -- 172.21.15.115:0/3669162446 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8004054100 con 0x7f8000004d00 2023-12-01T01:55:30.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.750+0000 7f8019069640 1 -- 172.21.15.115:0/3669162446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f801406e1d0 con 0x7f8000004d00 2023-12-01T01:55:30.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.759+0000 7f800b7fe640 1 -- 172.21.15.115:0/3669162446 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f800401a310 con 0x7f8000004d00 2023-12-01T01:55:30.760 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T01:55:30.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.761+0000 7f8019069640 1 -- 172.21.15.115:0/3669162446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff406a550 msgr2=0x7f7ff406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:30.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.761+0000 7f8019069640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff406a550 0x7f7ff406ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f80140afa50 tx=0x7f7ffc009040 comp rx=0 tx=0).stop 2023-12-01T01:55:30.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.761+0000 7f8019069640 1 -- 172.21.15.115:0/3669162446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 msgr2=0x7f80140b01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:30.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.761+0000 7f8019069640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 0x7f80140b01e0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f800400e8c0 tx=0x7f800400ed80 comp rx=0 tx=0).stop 2023-12-01T01:55:30.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.762+0000 7f8019069640 1 -- 172.21.15.115:0/3669162446 shutdown_connections 2023-12-01T01:55:30.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.762+0000 7f8019069640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7ff406a550 0x7f7ff406ca10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.762+0000 7f8019069640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80140af670 0x7f8014159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.762+0000 7f8019069640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80140ae0a0 0x7f80140c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.762+0000 7f8019069640 1 --2- 172.21.15.115:0/3669162446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8000004d00 0x7f80140b01e0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:30.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.762+0000 7f8019069640 1 -- 172.21.15.115:0/3669162446 >> 172.21.15.115:0/3669162446 conn(0x7f80140cbb10 msgr2=0x7f80140b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:30.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.762+0000 7f8019069640 1 -- 172.21.15.115:0/3669162446 shutdown_connections 2023-12-01T01:55:30.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:30.762+0000 7f8019069640 1 -- 172.21.15.115:0/3669162446 wait complete. 2023-12-01T01:55:30.783 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:55:31.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.041+0000 7f886e496640 1 Processor -- start 2023-12-01T01:55:31.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.041+0000 7f886e496640 1 -- start start 2023-12-01T01:55:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.042+0000 7f886e496640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8868159dd0 0x7f88680c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.042+0000 7f886e496640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f886806b6f0 0x7f88680c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.043+0000 7f886e496640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886806f8b0 con 0x7f88680ade80 2023-12-01T01:55:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.043+0000 7f886e496640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88680adcb0 con 0x7f886806b6f0 2023-12-01T01:55:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.043+0000 7f886e496640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88680ad9d0 con 0x7f8868159dd0 2023-12-01T01:55:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.043+0000 7f886ca33640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f88680ade80 0x7f88680cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36460/0 (socket says 172.21.15.115:36460) 2023-12-01T01:55:31.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.043+0000 7f886ca33640 1 -- 172.21.15.115:0/1105080948 learned_addr learned my addr 172.21.15.115:0/1105080948 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.043+0000 7f8867fff640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8868159dd0 0x7f88680c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.043+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204149812 0 0) 0x7f886806f8b0 con 0x7f88680ade80 2023-12-01T01:55:31.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.043+0000 7f885ffff640 1 --2- 172.21.15.115:0/1105080948 >> v2:172.21.15.142:3300/0 conn(0x7f886806b6f0 0x7f88680c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.043+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8854005730 con 0x7f88680ade80 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3304403949 0 0) 0x7f8854005730 con 0x7f88680ade80 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8868159dd0 msgr2=0x7f88680c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8868159dd0 0x7f88680c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 >> v2:172.21.15.142:3300/0 conn(0x7f886806b6f0 msgr2=0x7f88680c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 --2- 172.21.15.115:0/1105080948 >> v2:172.21.15.142:3300/0 conn(0x7f886806b6f0 0x7f88680c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88680c9000 con 0x7f88680ade80 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f88600019f0 con 0x7f88680ade80 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 >> v1:172.21.15.115:6789/0 conn(0x7f88680ade80 legacy=0x7f88680cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8854005bf0 0x7f8854006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 0x7f8854006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8868159dd0 0x7f8854007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88680adcb0 con 0x7f88540069d0 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88680ad9d0 con 0x7f8854005bf0 2023-12-01T01:55:31.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885400dcf0 con 0x7f8868159dd0 2023-12-01T01:55:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f885ffff640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 0x7f8854006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.044+0000 7f8867fff640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8854005bf0 0x7f8854006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.045+0000 7f886ca33640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8868159dd0 0x7f8854007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.045+0000 7f885ffff640 1 -- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8868159dd0 msgr2=0x7f8854007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.045+0000 7f885ffff640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8868159dd0 0x7f8854007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.045+0000 7f885ffff640 1 -- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8854005bf0 msgr2=0x7f8854006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.045+0000 7f885ffff640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8854005bf0 0x7f8854006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.045+0000 7f885ffff640 1 -- 172.21.15.115:0/1105080948 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8854005730 con 0x7f88540069d0 2023-12-01T01:55:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.046+0000 7f885ffff640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 0x7f8854006e50 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f8850009a90 tx=0x7f8854007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eaac87401f317b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.046+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885002c6b0 con 0x7f88540069d0 2023-12-01T01:55:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.046+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885002c810 con 0x7f88540069d0 2023-12-01T01:55:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.046+0000 7f88677fe640 1 -- 172.21.15.115:0/1105080948 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885002cba0 con 0x7f88540069d0 2023-12-01T01:55:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.047+0000 7f886e496640 1 -- 172.21.15.115:0/1105080948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 msgr2=0x7f8854006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.047+0000 7f886e496640 1 --2- 172.21.15.115:0/1105080948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 0x7f8854006e50 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f8850009a90 tx=0x7f8854007a00 comp rx=0 tx=0).stop 2023-12-01T01:55:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.047+0000 7f886ca33640 1 -- 172.21.15.115:0/1105080948 reap_dead start 2023-12-01T01:55:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.047+0000 7f886e496640 1 -- 172.21.15.115:0/1105080948 shutdown_connections 2023-12-01T01:55:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.047+0000 7f886e496640 1 -- 172.21.15.115:0/1105080948 >> 172.21.15.115:0/1105080948 conn(0x7f88680ce880 msgr2=0x7f88680cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.047+0000 7f886e496640 1 -- 172.21.15.115:0/1105080948 shutdown_connections 2023-12-01T01:55:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.047+0000 7f886e496640 1 -- 172.21.15.115:0/1105080948 wait complete. 2023-12-01T01:55:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.049+0000 7f886e496640 1 Processor -- start 2023-12-01T01:55:31.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.049+0000 7f886e496640 1 -- start start 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.050+0000 7f886e496640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 0x7f88681f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.050+0000 7f886e496640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88681f3410 0x7f88680c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.050+0000 7f886e496640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f886814db50 0x7f88680ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.050+0000 7f886e496640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88680c34e0 con 0x7f88540069d0 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.050+0000 7f886e496640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88681f2860 con 0x7f886814db50 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.050+0000 7f886e496640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88681f2b40 con 0x7f88681f3410 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.050+0000 7f8867fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 0x7f88681f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.050+0000 7f8867fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 0x7f88681f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55886/0 (socket says 172.21.15.115:55886) 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.050+0000 7f8867fff640 1 -- 172.21.15.115:0/2684384191 learned_addr learned my addr 172.21.15.115:0/2684384191 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f8867fff640 1 -- 172.21.15.115:0/2684384191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88681f3410 msgr2=0x7f88680c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f8867fff640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88681f3410 0x7f88680c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f8867fff640 1 -- 172.21.15.115:0/2684384191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f886814db50 msgr2=0x7f88680ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f886ca33640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f886814db50 0x7f88680ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f8867fff640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f886814db50 0x7f88680ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f8867fff640 1 -- 172.21.15.115:0/2684384191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88681f2e20 con 0x7f88540069d0 2023-12-01T01:55:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f8867fff640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 0x7f88681f2250 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f8858011600 tx=0x7f8858011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f88657fa640 1 -- 172.21.15.115:0/2684384191 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885800f920 con 0x7f88540069d0 2023-12-01T01:55:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f88657fa640 1 -- 172.21.15.115:0/2684384191 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885800fa80 con 0x7f88540069d0 2023-12-01T01:55:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f886e496640 1 -- 172.21.15.115:0/2684384191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88681f3100 con 0x7f88540069d0 2023-12-01T01:55:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f88657fa640 1 -- 172.21.15.115:0/2684384191 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885800fdf0 con 0x7f88540069d0 2023-12-01T01:55:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.051+0000 7f886e496640 1 -- 172.21.15.115:0/2684384191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886814dfd0 con 0x7f88540069d0 2023-12-01T01:55:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.052+0000 7f886e496640 1 -- 172.21.15.115:0/2684384191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886804e890 con 0x7f88540069d0 2023-12-01T01:55:31.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.053+0000 7f88657fa640 1 -- 172.21.15.115:0/2684384191 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8858015460 con 0x7f88540069d0 2023-12-01T01:55:31.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.054+0000 7f88657fa640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f884806a580 0x7f884806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.054+0000 7f88657fa640 1 -- 172.21.15.115:0/2684384191 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8858089240 con 0x7f88540069d0 2023-12-01T01:55:31.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.054+0000 7f885ffff640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f884806a580 0x7f884806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.055+0000 7f885ffff640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f884806a580 0x7f884806ca40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8850036000 tx=0x7f885002e000 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:31.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.059+0000 7f88657fa640 1 -- 172.21.15.115:0/2684384191 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8858054720 con 0x7f88540069d0 2023-12-01T01:55:31.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.284+0000 7f886e496640 1 -- 172.21.15.115:0/2684384191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f886806c510 con 0x7f88540069d0 2023-12-01T01:55:31.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.285+0000 7f88657fa640 1 -- 172.21.15.115:0/2684384191 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f8858057d40 con 0x7f88540069d0 2023-12-01T01:55:31.286 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:31.286 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3556763477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3556763477},{"type":"v1","addr":"172.21.15.142:6833","nonce":3556763477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3661095418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3661095418},{"type":"v1","addr":"172.21.15.142:6835","nonce":3661095418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3388585128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3388585128},{"type":"v1","addr":"172.21.15.115:6835","nonce":3388585128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:55:25.253838+0000","modified":"2023-12-01T01:55:28.264944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3894452047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3894452047},{"type":"v1","addr":"172.21.15.115:6837","nonce":3894452047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-01T01:55:31.287 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 63 2023-12-01T01:55:31.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.287+0000 7f886e496640 1 -- 172.21.15.115:0/2684384191 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f884806a580 msgr2=0x7f884806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.287+0000 7f886e496640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f884806a580 0x7f884806ca40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8850036000 tx=0x7f885002e000 comp rx=0 tx=0).stop 2023-12-01T01:55:31.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.287+0000 7f886e496640 1 -- 172.21.15.115:0/2684384191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 msgr2=0x7f88681f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.287+0000 7f886e496640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 0x7f88681f2250 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f8858011600 tx=0x7f8858011ac0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.288+0000 7f886e496640 1 -- 172.21.15.115:0/2684384191 shutdown_connections 2023-12-01T01:55:31.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.288+0000 7f886e496640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f884806a580 0x7f884806ca40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.288+0000 7f886e496640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f88681f3410 0x7f88680c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.288+0000 7f886e496640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f886814db50 0x7f88680ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.288+0000 7f886e496640 1 --2- 172.21.15.115:0/2684384191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88540069d0 0x7f88681f2250 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.288+0000 7f886e496640 1 -- 172.21.15.115:0/2684384191 >> 172.21.15.115:0/2684384191 conn(0x7f88680ce880 msgr2=0x7f88680c1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:31.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.288+0000 7f886e496640 1 -- 172.21.15.115:0/2684384191 shutdown_connections 2023-12-01T01:55:31.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.288+0000 7f886e496640 1 -- 172.21.15.115:0/2684384191 wait complete. 2023-12-01T01:55:31.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T01:55:25.253838+0000', 'modified': '2023-12-01T01:55:28.264944+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8475': {'gid': 8475, 'name': 'c', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6837/3894452047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 3894452047}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 3894452047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T01:55:31.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T01:55:31.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-01T01:55:31.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.937+0000 7fe1fe951640 1 Processor -- start 2023-12-01T01:55:31.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.937+0000 7fe1fe951640 1 -- start start 2023-12-01T01:55:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.938+0000 7fe1fe951640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f80ccc80 0x7fe1f806fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.938+0000 7fe1fe951640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f815d600 con 0x7fe1f80ad2b0 2023-12-01T01:55:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.938+0000 7fe1fe951640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f806c4b0 con 0x7fe1f815d220 2023-12-01T01:55:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.938+0000 7fe1fe951640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f806cc10 con 0x7fe1f80ccc80 2023-12-01T01:55:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1fceee640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe1f80ad2b0 0x7fe1f806fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36472/0 (socket says 172.21.15.115:36472) 2023-12-01T01:55:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1fceee640 1 -- 172.21.15.115:0/843312324 learned_addr learned my addr 172.21.15.115:0/843312324 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:31.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220432283 0 0) 0x7fe1f0001490 con 0x7fe1f80ad2b0 2023-12-01T01:55:31.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e0003520 con 0x7fe1f80ad2b0 2023-12-01T01:55:31.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f80ccc80 0x7fe1f806fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3397071029 0 0) 0x7fe1e0003520 con 0x7fe1f80ad2b0 2023-12-01T01:55:31.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f80ccc80 msgr2=0x7fe1f806fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1f6ffd640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f80ccc80 0x7fe1f806fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 >> v1:172.21.15.142:6789/0 conn(0x7fe1f815d220 legacy=0x7fe1f80cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f806f2f0 con 0x7fe1f80ad2b0 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.939+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fe1f0001b80 con 0x7fe1f80ad2b0 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 >> v1:172.21.15.115:6789/0 conn(0x7fe1f80ad2b0 legacy=0x7fe1f806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f6ffd640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0003a10 0x7fe1e0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f6ffd640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f80ccc80 0x7fe1e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f6ffd640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0004d70 0x7fe1e00051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f806cc10 con 0x7fe1e0003a10 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e000bc50 con 0x7fe1e0004d70 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e000be10 con 0x7fe1f80ccc80 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0003a10 0x7fe1e0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f80ccc80 0x7fe1e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f7fff640 1 -- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f80ccc80 msgr2=0x7fe1e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f80ccc80 0x7fe1e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1fceee640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0004d70 0x7fe1e00051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f7fff640 1 -- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0004d70 msgr2=0x7fe1e00051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0004d70 0x7fe1e00051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.940+0000 7fe1f7fff640 1 -- 172.21.15.115:0/843312324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e0003520 con 0x7fe1e0003a10 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.942+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0003a10 0x7fe1e0003e30 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8001750 tx=0x7fe1e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbc76bbdd3e519d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.942+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80052d0 con 0x7fe1e0003a10 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.942+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e802e050 con 0x7fe1e0003a10 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.942+0000 7fe1f6ffd640 1 -- 172.21.15.115:0/843312324 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80035f0 con 0x7fe1e0003a10 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.942+0000 7fe1fe951640 1 -- 172.21.15.115:0/843312324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0003a10 msgr2=0x7fe1e0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.942+0000 7fe1fe951640 1 --2- 172.21.15.115:0/843312324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0003a10 0x7fe1e0003e30 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8001750 tx=0x7fe1e00057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.942+0000 7fe1fceee640 1 -- 172.21.15.115:0/843312324 reap_dead start 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.942+0000 7fe1fe951640 1 -- 172.21.15.115:0/843312324 shutdown_connections 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.942+0000 7fe1fe951640 1 -- 172.21.15.115:0/843312324 >> 172.21.15.115:0/843312324 conn(0x7fe1f80ce880 msgr2=0x7fe1f80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.943+0000 7fe1fe951640 1 -- 172.21.15.115:0/843312324 shutdown_connections 2023-12-01T01:55:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.943+0000 7fe1fe951640 1 -- 172.21.15.115:0/843312324 wait complete. 2023-12-01T01:55:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.945+0000 7fe1fe951640 1 Processor -- start 2023-12-01T01:55:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.945+0000 7fe1fe951640 1 -- start start 2023-12-01T01:55:31.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.946+0000 7fe1fe951640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1e0003a10 0x7fe1f8157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1fe951640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f8157f80 0x7fe1f80c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1fe951640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1f80c3300 0x7fe1f80c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1fe951640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80c9f10 con 0x7fe1f8157f80 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1fe951640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80c9570 con 0x7fe1f80c3300 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1fe951640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80c9850 con 0x7fe1e0003a10 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1f77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f8157f80 0x7fe1f80c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1f77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f8157f80 0x7fe1f80c2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55900/0 (socket says 172.21.15.115:55900) 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1f77fe640 1 -- 172.21.15.115:0/1666128650 learned_addr learned my addr 172.21.15.115:0/1666128650 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1e0003a10 0x7fe1f8157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1fceee640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1f80c3300 0x7fe1f80c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1f77fe640 1 -- 172.21.15.115:0/1666128650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1e0003a10 msgr2=0x7fe1f8157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1e0003a10 0x7fe1f8157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1f77fe640 1 -- 172.21.15.115:0/1666128650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1f80c3300 msgr2=0x7fe1f80c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1f80c3300 0x7fe1f80c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.947+0000 7fe1f77fe640 1 -- 172.21.15.115:0/1666128650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f80c9b30 con 0x7fe1f8157f80 2023-12-01T01:55:31.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.948+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f8157f80 0x7fe1f80c2dc0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00f580 tx=0x7fe1ec00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:31.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.948+0000 7fe1f4ff9640 1 -- 172.21.15.115:0/1666128650 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec014870 con 0x7fe1f8157f80 2023-12-01T01:55:31.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.948+0000 7fe1f4ff9640 1 -- 172.21.15.115:0/1666128650 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec0149d0 con 0x7fe1f8157f80 2023-12-01T01:55:31.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.948+0000 7fe1f4ff9640 1 -- 172.21.15.115:0/1666128650 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec014d80 con 0x7fe1f8157f80 2023-12-01T01:55:31.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.948+0000 7fe1fe951640 1 -- 172.21.15.115:0/1666128650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f8157bc0 con 0x7fe1f8157f80 2023-12-01T01:55:31.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.948+0000 7fe1fe951640 1 -- 172.21.15.115:0/1666128650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f80c3d90 con 0x7fe1f8157f80 2023-12-01T01:55:31.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.951+0000 7fe1f4ff9640 1 -- 172.21.15.115:0/1666128650 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe1ec015410 con 0x7fe1f8157f80 2023-12-01T01:55:31.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.952+0000 7fe1f4ff9640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe1e406a550 0x7fe1e406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.952+0000 7fe1f4ff9640 1 -- 172.21.15.115:0/1666128650 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1ec089400 con 0x7fe1f8157f80 2023-12-01T01:55:31.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.953+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe1e406a550 0x7fe1e406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.953+0000 7fe20159b740 1 Processor -- start 2023-12-01T01:55:31.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.953+0000 7fe20159b740 1 -- start start 2023-12-01T01:55:31.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.953+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe1e406a550 0x7fe1e406ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8033000 tx=0x7fe1e8009030 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:31.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe20159b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df5436d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe20159b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df54068a0 0x558df5433b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe20159b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 0x558df54340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe20159b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df53e22f0 con 0x558df54068a0 2023-12-01T01:55:31.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe20159b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5376f80 con 0x558df5436920 2023-12-01T01:55:31.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe20159b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x558df5374710 con 0x558df540a4a0 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe1f77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df54068a0 0x558df5433b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe1f77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df54068a0 0x558df5433b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55916/0 (socket says 172.21.15.115:55916) 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe1f77fe640 1 -- 172.21.15.115:0/2639796192 learned_addr learned my addr 172.21.15.115:0/2639796192 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe1f77fe640 1 -- 172.21.15.115:0/2639796192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 msgr2=0x558df54340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/2639796192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 0x558df54340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe1f77fe640 1 -- 172.21.15.115:0/2639796192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 msgr2=0x558df5436d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/2639796192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df5436d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.954+0000 7fe1f77fe640 1 -- 172.21.15.115:0/2639796192 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558df5337690 con 0x558df54068a0 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.955+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/2639796192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df54068a0 0x558df5433b90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00f550 tx=0x7fe1ec058220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb2a3bc2b6e911e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.955+0000 7fe1ddffb640 1 -- 172.21.15.115:0/2639796192 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec05c3e0 con 0x558df54068a0 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.955+0000 7fe1ddffb640 1 -- 172.21.15.115:0/2639796192 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec060590 con 0x558df54068a0 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.955+0000 7fe1ddffb640 1 -- 172.21.15.115:0/2639796192 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec087920 con 0x558df54068a0 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.955+0000 7fe20159b740 1 -- 172.21.15.115:0/2639796192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df54068a0 msgr2=0x558df5433b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.955+0000 7fe20159b740 1 --2- 172.21.15.115:0/2639796192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df54068a0 0x558df5433b90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00f550 tx=0x7fe1ec058220 comp rx=0 tx=0).stop 2023-12-01T01:55:31.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.955+0000 7fe20159b740 1 -- 172.21.15.115:0/2639796192 shutdown_connections 2023-12-01T01:55:31.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.955+0000 7fe20159b740 1 --2- 172.21.15.115:0/2639796192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 0x558df54340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.955+0000 7fe20159b740 1 --2- 172.21.15.115:0/2639796192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df54068a0 0x558df5433b90 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.955+0000 7fe20159b740 1 --2- 172.21.15.115:0/2639796192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df5436d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.956+0000 7fe20159b740 1 -- 172.21.15.115:0/2639796192 >> 172.21.15.115:0/2639796192 conn(0x558df5408bb0 msgr2=0x558df5451730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:31.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.956+0000 7fe20159b740 1 -- 172.21.15.115:0/2639796192 shutdown_connections 2023-12-01T01:55:31.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.956+0000 7fe20159b740 1 -- 172.21.15.115:0/2639796192 wait complete. 2023-12-01T01:55:31.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.956+0000 7fe20159b740 1 -- start start 2023-12-01T01:55:31.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.956+0000 7fe20159b740 1 Processor -- start 2023-12-01T01:55:31.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.957+0000 7fe20159b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 0x558df541d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.957+0000 7fe20159b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df541a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.957+0000 7fe20159b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5451ed0 0x558df542f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.957+0000 7fe20159b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df5370000 con 0x558df5451ed0 2023-12-01T01:55:31.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.957+0000 7fe20159b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df53117d0 con 0x558df5436920 2023-12-01T01:55:31.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.957+0000 7fe20159b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x558df5311470 con 0x558df540a4a0 2023-12-01T01:55:31.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe20159b740 10 client.0 _ll_register_callbacks cb 0x558df53601b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1fceee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5451ed0 0x558df542f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1fceee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5451ed0 0x558df542f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55922/0 (socket says 172.21.15.115:55922) 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1fceee640 1 -- 172.21.15.115:0/4023790051 learned_addr learned my addr 172.21.15.115:0/4023790051 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1fceee640 1 -- 172.21.15.115:0/4023790051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 msgr2=0x558df541d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df541a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1fceee640 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 0x558df541d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1fceee640 1 -- 172.21.15.115:0/4023790051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 msgr2=0x558df541a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1fceee640 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df541a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1fceee640 1 -- 172.21.15.115:0/4023790051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f80c3d90 con 0x558df5451ed0 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1fceee640 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5451ed0 0x558df542f0a0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fe1f806d490 tx=0x7fe1f0007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a96651bc56f662 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.958+0000 7fe1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/4023790051 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f000f700 con 0x558df5451ed0 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/4023790051 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1f000f860 con 0x558df5451ed0 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe20159b740 1 -- 172.21.15.115:0/4023790051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558df539a550 con 0x558df5451ed0 2023-12-01T01:55:31.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/4023790051 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1f000fa20 con 0x558df5451ed0 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe20159b740 10 client.8619 fetch_fsmap learned FSMap version 63 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe20159b740 1 -- 172.21.15.115:0/4023790051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x558df536e680 con 0x558df5451ed0 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/4023790051 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558df539a550 con 0x558df5451ed0 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/4023790051 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 63) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe1f0012070 con 0x558df5451ed0 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe20159b740 10 client.8619 fetch_fsmap finished waiting for FSMap version 63 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe20159b740 10 client.8619 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe20159b740 20 client.8619 populate_metadata read hostname 'smithi115' 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe20159b740 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.115:6836/3894452047,v1:172.21.15.115:6837/3894452047] conn(0x558df542fb00 0x558df542fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe20159b740 4 client.8619 mds_command: new command op to 8475 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.959+0000 7fe20159b740 1 -- 172.21.15.115:0/4023790051 --> [v2:172.21.15.115:6836/3894452047,v1:172.21.15.115:6837/3894452047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558df5362b50 con 0x558df542fb00 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.960+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.115:6836/3894452047,v1:172.21.15.115:6837/3894452047] conn(0x558df542fb00 0x558df542fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.960+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.115:6836/3894452047,v1:172.21.15.115:6837/3894452047] conn(0x558df542fb00 0x558df542fee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d93aca72e09acd8e server_cookie=92d611067c99a18a in_seq=0 out_seq=0 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.960+0000 7fe1d6ffd640 10 client.8619 ms_handle_connect on v2:172.21.15.115:6836/3894452047 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/4023790051 <== mds.0 v2:172.21.15.115:6836/3894452047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558df5362b50 con 0x558df542fb00 2023-12-01T01:55:31.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe1d6ffd640 10 client.8619 handle_command_reply: tid=0 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 client.8619 shutdown 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 -- 172.21.15.115:0/4023790051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5451ed0 msgr2=0x558df542f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5451ed0 0x558df542f0a0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fe1f806d490 tx=0x7fe1f0007710 comp rx=0 tx=0).stop 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 -- 172.21.15.115:0/4023790051 shutdown_connections 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.115:6836/3894452047,v1:172.21.15.115:6837/3894452047] conn(0x558df542fb00 0x558df542fee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5451ed0 0x558df542f0a0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df541a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 --2- 172.21.15.115:0/4023790051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 0x558df541d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 -- 172.21.15.115:0/4023790051 >> 172.21.15.115:0/4023790051 conn(0x558df54068a0 msgr2=0x558df543a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe1d6ffd640 0 client.8619 ms_handle_reset on v2:172.21.15.115:6836/3894452047 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 -- 172.21.15.115:0/4023790051 shutdown_connections 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.961+0000 7fe20159b740 1 -- 172.21.15.115:0/4023790051 wait complete. 2023-12-01T01:55:31.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.962+0000 7fe20159b740 20 client.8619 trim_cache size 0 max 16384 2023-12-01T01:55:31.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.977+0000 7fe20159b740 1 Processor -- start 2023-12-01T01:55:31.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.977+0000 7fe20159b740 1 -- start start 2023-12-01T01:55:31.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.978+0000 7fe20159b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df540a4a0 0x558df542bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.978+0000 7fe20159b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df540ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.978+0000 7fe20159b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df5451ed0 0x558df5405920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.978+0000 7fe20159b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df50d7f20 con 0x558df540a4a0 2023-12-01T01:55:31.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.978+0000 7fe20159b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df542f080 con 0x558df5436920 2023-12-01T01:55:31.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.978+0000 7fe20159b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x558df542f360 con 0x558df5451ed0 2023-12-01T01:55:31.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.979+0000 7fe1f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df540a4a0 0x558df542bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.979+0000 7fe1f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df540a4a0 0x558df542bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55938/0 (socket says 172.21.15.115:55938) 2023-12-01T01:55:31.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.979+0000 7fe1f7fff640 1 -- 172.21.15.115:0/4002394802 learned_addr learned my addr 172.21.15.115:0/4002394802 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.979+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/4002394802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df540ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.979+0000 7fe1fceee640 1 --2- 172.21.15.115:0/4002394802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df5451ed0 0x558df5405920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.979+0000 7fe1f7fff640 1 -- 172.21.15.115:0/4002394802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df5451ed0 msgr2=0x558df5405920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.979+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/4002394802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df5451ed0 0x558df5405920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.979+0000 7fe1f7fff640 1 -- 172.21.15.115:0/4002394802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 msgr2=0x558df540ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.979+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/4002394802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df540ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.979+0000 7fe1f7fff640 1 -- 172.21.15.115:0/4002394802 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558df53109e0 con 0x558df540a4a0 2023-12-01T01:55:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.980+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/4002394802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df540a4a0 0x558df542bd30 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8005260 tx=0x7fe1e8001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b8dc11160fba9a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.980+0000 7fe1de7fc640 1 -- 172.21.15.115:0/4002394802 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e804e210 con 0x558df540a4a0 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.980+0000 7fe1de7fc640 1 -- 172.21.15.115:0/4002394802 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e804e370 con 0x558df540a4a0 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.980+0000 7fe1de7fc640 1 -- 172.21.15.115:0/4002394802 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8007230 con 0x558df540a4a0 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.980+0000 7fe20159b740 1 -- 172.21.15.115:0/4002394802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df540a4a0 msgr2=0x558df542bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.980+0000 7fe20159b740 1 --2- 172.21.15.115:0/4002394802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df540a4a0 0x558df542bd30 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8005260 tx=0x7fe1e8001190 comp rx=0 tx=0).stop 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.981+0000 7fe20159b740 1 -- 172.21.15.115:0/4002394802 shutdown_connections 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.981+0000 7fe20159b740 1 --2- 172.21.15.115:0/4002394802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df5451ed0 0x558df5405920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.981+0000 7fe20159b740 1 --2- 172.21.15.115:0/4002394802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5436920 0x558df540ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.981+0000 7fe20159b740 1 --2- 172.21.15.115:0/4002394802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df540a4a0 0x558df542bd30 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.981+0000 7fe20159b740 1 -- 172.21.15.115:0/4002394802 >> 172.21.15.115:0/4002394802 conn(0x558df54068a0 msgr2=0x558df541a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.981+0000 7fe20159b740 1 -- 172.21.15.115:0/4002394802 shutdown_connections 2023-12-01T01:55:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.981+0000 7fe20159b740 1 -- 172.21.15.115:0/4002394802 wait complete. 2023-12-01T01:55:31.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.981+0000 7fe20159b740 1 -- start start 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.982+0000 7fe20159b740 1 Processor -- start 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe20159b740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 0x558df5499e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe20159b740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5436920 0x558df549c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe20159b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5451ed0 0x558df54337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe20159b740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df545fe50 con 0x558df5436920 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe20159b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df540c560 con 0x558df5451ed0 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe20159b740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x558df5430a00 con 0x558df540a4a0 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe20159b740 10 client.0 _ll_register_callbacks cb 0x7fe1ec014ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe1f77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5436920 0x558df549c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe1f77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5436920 0x558df549c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55954/0 (socket says 172.21.15.115:55954) 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe1f77fe640 1 -- 172.21.15.115:0/1920956198 learned_addr learned my addr 172.21.15.115:0/1920956198 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe1f77fe640 1 -- 172.21.15.115:0/1920956198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 msgr2=0x558df5499e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 0x558df5499e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe1f77fe640 1 -- 172.21.15.115:0/1920956198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5451ed0 msgr2=0x558df54337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:31.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe1fceee640 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5451ed0 0x558df54337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5451ed0 0x558df54337b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.983+0000 7fe1f77fe640 1 -- 172.21.15.115:0/1920956198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e0003520 con 0x558df5436920 2023-12-01T01:55:31.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe1f77fe640 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5436920 0x558df549c680 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0584f0 tx=0x7fe1ec05a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589046d46df4985e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:31.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T01:55:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/1920956198 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec05c3e0 con 0x558df5436920 2023-12-01T01:55:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/1920956198 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1ec0878f0 con 0x558df5436920 2023-12-01T01:55:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe20159b740 1 -- 172.21.15.115:0/1920956198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1ec060590 con 0x558df5436920 2023-12-01T01:55:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/1920956198 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1ec087ad0 con 0x558df5436920 2023-12-01T01:55:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe20159b740 10 client.8625 fetch_fsmap learned FSMap version 63 2023-12-01T01:55:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe20159b740 1 -- 172.21.15.115:0/1920956198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x558df5335ef0 con 0x558df5436920 2023-12-01T01:55:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/1920956198 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec060590 con 0x558df5436920 2023-12-01T01:55:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/1920956198 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 63) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe1ec057aa0 con 0x558df5436920 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe20159b740 10 client.8625 fetch_fsmap finished waiting for FSMap version 63 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe20159b740 10 client.8625 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe20159b740 20 client.8625 populate_metadata read hostname 'smithi115' 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe20159b740 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.115:6836/3894452047,v1:172.21.15.115:6837/3894452047] conn(0x558df542fae0 0x558df542fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe20159b740 4 client.8625 mds_command: new command op to 8475 tid=0[{"prefix": "status"}] 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.984+0000 7fe20159b740 1 -- 172.21.15.115:0/1920956198 --> [v2:172.21.15.115:6836/3894452047,v1:172.21.15.115:6837/3894452047] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558df5369080 con 0x558df542fae0 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.985+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.115:6836/3894452047,v1:172.21.15.115:6837/3894452047] conn(0x558df542fae0 0x558df542fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.985+0000 7fe1f7fff640 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.115:6836/3894452047,v1:172.21.15.115:6837/3894452047] conn(0x558df542fae0 0x558df542fec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9073b3daf6aa4e8 server_cookie=e8006a9046aa3dbc in_seq=0 out_seq=0 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.985+0000 7fe1d6ffd640 10 client.8625 ms_handle_connect on v2:172.21.15.115:6836/3894452047 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.985+0000 7fe1d6ffd640 1 -- 172.21.15.115:0/1920956198 <== mds.0 v2:172.21.15.115:6836/3894452047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x558df5369080 con 0x558df542fae0 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.985+0000 7fe1d6ffd640 10 client.8625 handle_command_reply: tid=0 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.985+0000 7fe20159b740 1 client.8625 shutdown 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 1 -- 172.21.15.115:0/1920956198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5436920 msgr2=0x558df549c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5436920 0x558df549c680 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0584f0 tx=0x7fe1ec05a000 comp rx=0 tx=0).stop 2023-12-01T01:55:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 1 -- 172.21.15.115:0/1920956198 shutdown_connections 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.115:6836/3894452047,v1:172.21.15.115:6837/3894452047] conn(0x558df542fae0 0x558df542fec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558df5451ed0 0x558df54337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558df5436920 0x558df549c680 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 1 --2- 172.21.15.115:0/1920956198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558df540a4a0 0x558df5499e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe1d6ffd640 0 client.8625 ms_handle_reset on v2:172.21.15.115:6836/3894452047 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 1 -- 172.21.15.115:0/1920956198 >> 172.21.15.115:0/1920956198 conn(0x558df54068a0 msgr2=0x558df542bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 1 -- 172.21.15.115:0/1920956198 shutdown_connections 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 1 -- 172.21.15.115:0/1920956198 wait complete. 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe20159b740 20 client.8625 trim_cache size 0 max 16384 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T01:55:31.990 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 8475, 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 6.7275329270000004, 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 63, 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 141, 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 141, 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 12.582193978999999 2023-12-01T01:55:31.991 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T01:55:31.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe1fe951640 1 -- 172.21.15.115:0/1666128650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe1e406a550 msgr2=0x7fe1e406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe1fe951640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe1e406a550 0x7fe1e406ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8033000 tx=0x7fe1e8009030 comp rx=0 tx=0).stop 2023-12-01T01:55:31.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe1fe951640 1 -- 172.21.15.115:0/1666128650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f8157f80 msgr2=0x7fe1f80c2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:31.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.986+0000 7fe1fe951640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f8157f80 0x7fe1f80c2dc0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00f580 tx=0x7fe1ec00fa40 comp rx=0 tx=0).stop 2023-12-01T01:55:31.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.987+0000 7fe1fe951640 1 -- 172.21.15.115:0/1666128650 shutdown_connections 2023-12-01T01:55:31.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.987+0000 7fe1fe951640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe1e406a550 0x7fe1e406ca10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.987+0000 7fe1fe951640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1f80c3300 0x7fe1f80c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.987+0000 7fe1fe951640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f8157f80 0x7fe1f80c2dc0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.987+0000 7fe1fe951640 1 --2- 172.21.15.115:0/1666128650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1e0003a10 0x7fe1f8157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.987+0000 7fe1fe951640 1 -- 172.21.15.115:0/1666128650 >> 172.21.15.115:0/1666128650 conn(0x7fe1f80ce880 msgr2=0x7fe1f80b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.987+0000 7fe1fe951640 1 -- 172.21.15.115:0/1666128650 shutdown_connections 2023-12-01T01:55:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:31.987+0000 7fe1fe951640 1 -- 172.21.15.115:0/1666128650 wait complete. 2023-12-01T01:55:32.004 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T01:55:32.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.200+0000 7f8f51c96640 1 Processor -- start 2023-12-01T01:55:32.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.201+0000 7f8f51c96640 1 -- start start 2023-12-01T01:55:32.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.202+0000 7f8f51c96640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8f4c0c9ef0 0x7f8f4c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:32.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.202+0000 7f8f51c96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c06b6f0 0x7f8f4c070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:32.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.202+0000 7f8f51c96640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c06f8b0 con 0x7f8f4c159dd0 2023-12-01T01:55:32.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.202+0000 7f8f51c96640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4c0adcb0 con 0x7f8f4c0c9ef0 2023-12-01T01:55:32.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.202+0000 7f8f51c96640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0ad9d0 con 0x7f8f4c06b6f0 2023-12-01T01:55:32.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.202+0000 7f8f4b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8f4c159dd0 0x7f8f4c0c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36476/0 (socket says 172.21.15.115:36476) 2023-12-01T01:55:32.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.202+0000 7f8f4b7fe640 1 -- 172.21.15.115:0/2339303087 learned_addr learned my addr 172.21.15.115:0/2339303087 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:32.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512310810 0 0) 0x7f8f4c06f8b0 con 0x7f8f4c159dd0 2023-12-01T01:55:32.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f38003520 con 0x7f8f4c159dd0 2023-12-01T01:55:32.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4bfff640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c06b6f0 0x7f8f4c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:32.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f43fff640 1 --2- 172.21.15.115:0/2339303087 >> v2:172.21.15.142:3300/0 conn(0x7f8f4c0c9ef0 0x7f8f4c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:32.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1421906673 0 0) 0x7f8f38003520 con 0x7f8f4c159dd0 2023-12-01T01:55:32.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c06b6f0 msgr2=0x7f8f4c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:32.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c06b6f0 0x7f8f4c070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 >> v2:172.21.15.142:3300/0 conn(0x7f8f4c0c9ef0 msgr2=0x7f8f4c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:32.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 --2- 172.21.15.115:0/2339303087 >> v2:172.21.15.142:3300/0 conn(0x7f8f4c0c9ef0 0x7f8f4c06fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c0c9000 con 0x7f8f4c159dd0 2023-12-01T01:55:32.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f8f3c001670 con 0x7f8f4c159dd0 2023-12-01T01:55:32.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 >> v1:172.21.15.115:6789/0 conn(0x7f8f4c159dd0 legacy=0x7f8f4c0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:32.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f380039e0 0x7f8f38003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 0x7f8f38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c06b6f0 0x7f8f3800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0adcb0 con 0x7f8f380047c0 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0ad9d0 con 0x7f8f380039e0 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38005610 con 0x7f8f4c06b6f0 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.203+0000 7f8f43fff640 1 --2- 172.21.15.115:0/2339303087 >> v2:172.21.15.142:3300/0 conn(0x7f8f4c0c9ef0 0x7f8f4c06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.204+0000 7f8f43fff640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 0x7f8f38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.204+0000 7f8f4b7fe640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f380039e0 0x7f8f38003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.204+0000 7f8f4bfff640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c06b6f0 0x7f8f3800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.204+0000 7f8f43fff640 1 -- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c06b6f0 msgr2=0x7f8f3800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.204+0000 7f8f43fff640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c06b6f0 0x7f8f3800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.204+0000 7f8f43fff640 1 -- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f380039e0 msgr2=0x7f8f38003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.204+0000 7f8f43fff640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f380039e0 0x7f8f38003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.204+0000 7f8f43fff640 1 -- 172.21.15.115:0/2339303087 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f38003520 con 0x7f8f380047c0 2023-12-01T01:55:32.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.206+0000 7f8f43fff640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 0x7f8f38004c40 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400aa70 tx=0x7f8f380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0149599ee9aa11 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:32.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.206+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34004220 con 0x7f8f380047c0 2023-12-01T01:55:32.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.206+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f34004380 con 0x7f8f380047c0 2023-12-01T01:55:32.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.206+0000 7f8f4affd640 1 -- 172.21.15.115:0/2339303087 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34003830 con 0x7f8f380047c0 2023-12-01T01:55:32.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.207+0000 7f8f51c96640 1 -- 172.21.15.115:0/2339303087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 msgr2=0x7f8f38004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:32.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.207+0000 7f8f51c96640 1 --2- 172.21.15.115:0/2339303087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 0x7f8f38004c40 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400aa70 tx=0x7f8f380057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.207+0000 7f8f4bfff640 1 -- 172.21.15.115:0/2339303087 reap_dead start 2023-12-01T01:55:32.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.207+0000 7f8f51c96640 1 -- 172.21.15.115:0/2339303087 shutdown_connections 2023-12-01T01:55:32.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.207+0000 7f8f51c96640 1 -- 172.21.15.115:0/2339303087 >> 172.21.15.115:0/2339303087 conn(0x7f8f4c0ce880 msgr2=0x7f8f4c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:32.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.207+0000 7f8f51c96640 1 -- 172.21.15.115:0/2339303087 shutdown_connections 2023-12-01T01:55:32.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.207+0000 7f8f51c96640 1 -- 172.21.15.115:0/2339303087 wait complete. 2023-12-01T01:55:32.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.209+0000 7f8f51c96640 1 Processor -- start 2023-12-01T01:55:32.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.209+0000 7f8f51c96640 1 -- start start 2023-12-01T01:55:32.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.210+0000 7f8f51c96640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 0x7f8f4c14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:32.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.210+0000 7f8f51c96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c151680 0x7f8f4c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:32.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.210+0000 7f8f51c96640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f4c15a530 0x7f8f4c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:32.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.210+0000 7f8f51c96640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c153e90 con 0x7f8f380047c0 2023-12-01T01:55:32.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.210+0000 7f8f51c96640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0cdbe0 con 0x7f8f4c15a530 2023-12-01T01:55:32.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.210+0000 7f8f51c96640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c15d000 con 0x7f8f4c151680 2023-12-01T01:55:32.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.210+0000 7f8f4b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 0x7f8f4c14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.210+0000 7f8f4b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 0x7f8f4c14f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55976/0 (socket says 172.21.15.115:55976) 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.210+0000 7f8f4b7fe640 1 -- 172.21.15.115:0/399424096 learned_addr learned my addr 172.21.15.115:0/399424096 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.210+0000 7f8f4b7fe640 1 -- 172.21.15.115:0/399424096 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c151680 msgr2=0x7f8f4c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.211+0000 7f8f4b7fe640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c151680 0x7f8f4c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.211+0000 7f8f4b7fe640 1 -- 172.21.15.115:0/399424096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f4c15a530 msgr2=0x7f8f4c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.211+0000 7f8f4b7fe640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f4c15a530 0x7f8f4c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.211+0000 7f8f4b7fe640 1 -- 172.21.15.115:0/399424096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c154100 con 0x7f8f380047c0 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.211+0000 7f8f4b7fe640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 0x7f8f4c14f060 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c004ef0 tx=0x7f8f3c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.211+0000 7f8f48ff9640 1 -- 172.21.15.115:0/399424096 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00f730 con 0x7f8f380047c0 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.211+0000 7f8f48ff9640 1 -- 172.21.15.115:0/399424096 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c00f890 con 0x7f8f380047c0 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.211+0000 7f8f48ff9640 1 -- 172.21.15.115:0/399424096 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00fc00 con 0x7f8f380047c0 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.211+0000 7f8f51c96640 1 -- 172.21.15.115:0/399424096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c15d2e0 con 0x7f8f380047c0 2023-12-01T01:55:32.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.211+0000 7f8f51c96640 1 -- 172.21.15.115:0/399424096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c0cd800 con 0x7f8f380047c0 2023-12-01T01:55:32.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.212+0000 7f8f51c96640 1 -- 172.21.15.115:0/399424096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c154290 con 0x7f8f380047c0 2023-12-01T01:55:32.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.213+0000 7f8f48ff9640 1 -- 172.21.15.115:0/399424096 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8f3c00fd60 con 0x7f8f380047c0 2023-12-01T01:55:32.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.214+0000 7f8f48ff9640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f2c06a580 0x7f8f2c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:32.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.214+0000 7f8f48ff9640 1 -- 172.21.15.115:0/399424096 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f3c088970 con 0x7f8f380047c0 2023-12-01T01:55:32.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.214+0000 7f8f43fff640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f2c06a580 0x7f8f2c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:32.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.215+0000 7f8f43fff640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f2c06a580 0x7f8f2c06ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f34035000 tx=0x7f8f34030000 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:32.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.218+0000 7f8f48ff9640 1 -- 172.21.15.115:0/399424096 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8f3c053ee0 con 0x7f8f380047c0 2023-12-01T01:55:32.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.447+0000 7f8f51c96640 1 -- 172.21.15.115:0/399424096 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8f4c04e890 con 0x7f8f380047c0 2023-12-01T01:55:32.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.448+0000 7f8f48ff9640 1 -- 172.21.15.115:0/399424096 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8f3c057500 con 0x7f8f380047c0 2023-12-01T01:55:32.450 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:55:32.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.450+0000 7f8f51c96640 1 -- 172.21.15.115:0/399424096 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f2c06a580 msgr2=0x7f8f2c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:32.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f2c06a580 0x7f8f2c06ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f34035000 tx=0x7f8f34030000 comp rx=0 tx=0).stop 2023-12-01T01:55:32.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 -- 172.21.15.115:0/399424096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 msgr2=0x7f8f4c14f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:32.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 0x7f8f4c14f060 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c004ef0 tx=0x7f8f3c007430 comp rx=0 tx=0).stop 2023-12-01T01:55:32.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 -- 172.21.15.115:0/399424096 shutdown_connections 2023-12-01T01:55:32.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8f2c06a580 0x7f8f2c06ca40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4c151680 0x7f8f4c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f4c15a530 0x7f8f4c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 --2- 172.21.15.115:0/399424096 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f380047c0 0x7f8f4c14f060 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:32.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 -- 172.21.15.115:0/399424096 >> 172.21.15.115:0/399424096 conn(0x7f8f4c0ce880 msgr2=0x7f8f4c152500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:32.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 -- 172.21.15.115:0/399424096 shutdown_connections 2023-12-01T01:55:32.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:32.451+0000 7f8f51c96640 1 -- 172.21.15.115:0/399424096 wait complete. 2023-12-01T01:55:32.471 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T01:55:32.472 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:55:32.472 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T01:55:32.472 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T01:55:32.472 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:55:32.472 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T01:55:32.472 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T01:55:32.472 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:55:32.472 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:55:32.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:32.473 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T01:55:32.523 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:55:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:55:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:55:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:55:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T01:55:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:55:32.524 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:55:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:55:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T01:55:32.524 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:55:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:55:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T01:55:32.525 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:55:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T01:55:32.525 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:55:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T01:55:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:55:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 41225sec preferred_lft 41225sec 2023-12-01T01:55:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T01:55:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:55:32.526 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T01:55:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T01:55:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T01:55:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:55:32.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:32.527 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:55:32.618 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T01:55:32.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:32.621 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T01:55:32.674 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T01:55:32.674 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:55:32.674 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:55:32.674 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:55:32.674 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:55:32.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:32.675 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T01:55:32.675 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T01:55:32.675 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:55:32.675 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:55:32.675 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:55:32.675 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T01:55:32.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:32.863 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T01:55:32.915 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T01:55:32.915 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T01:55:32.915 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:55:32.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:32.916 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T01:55:33.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:55:33.009 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T01:55:33.063 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:55:33.063 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:55:33.063 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:55:33.111 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T01:55:33.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:33.111 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:55:33.303 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:55:33.303 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:55:33.303 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:55:33.323 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:55:33.323 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:55:33.323 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:55:33.323 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:55:33.323 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:55:33.352 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T01:55:33.352 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T01:55:33.352 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T01:55:33.352 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T01:55:33.352 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T01:55:33.352 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T01:55:33.352 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T01:55:33.352 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T01:55:33.353 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T01:55:33.353 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T01:55:33.353 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T01:55:33.353 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T01:55:33.353 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T01:55:33.359 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:55:33.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:33.359 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:55:33.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:33.445 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:55:33.509 INFO:teuthology.orchestra.run.smithi115.stdout:client6403 2023-12-01T01:55:33.509 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T01:55:33.510 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:55:33.560 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T01:55:33.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.823+0000 7f3a9e2cf640 1 Processor -- start 2023-12-01T01:55:33.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.823+0000 7f3a9e2cf640 1 -- start start 2023-12-01T01:55:33.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.824+0000 7f3a9e2cf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980cad70 0x7f3a9806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:33.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.825+0000 7f3a9e2cf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a980add10 con 0x7f3a9806f2f0 2023-12-01T01:55:33.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.825+0000 7f3a9e2cf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a980c5760 con 0x7f3a9815e2f0 2023-12-01T01:55:33.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.825+0000 7f3a9e2cf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9815e6d0 con 0x7f3a980cad70 2023-12-01T01:55:33.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.825+0000 7f3a9c86c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3a9806f2f0 0x7f3a9806bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36480/0 (socket says 172.21.15.115:36480) 2023-12-01T01:55:33.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.825+0000 7f3a9c86c640 1 -- 172.21.15.115:0/2601013721 learned_addr learned my addr 172.21.15.115:0/2601013721 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:33.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.825+0000 7f3a8ffff640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980cad70 0x7f3a9806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:33.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.825+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890792066 0 0) 0x7f3a980add10 con 0x7f3a9806f2f0 2023-12-01T01:55:33.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.825+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a84003520 con 0x7f3a9806f2f0 2023-12-01T01:55:33.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1789878094 0 0) 0x7f3a84003520 con 0x7f3a9806f2f0 2023-12-01T01:55:33.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980cad70 msgr2=0x7f3a9806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:33.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980cad70 0x7f3a9806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:33.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 >> v1:172.21.15.142:6789/0 conn(0x7f3a9815e2f0 legacy=0x7f3a980ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:33.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a980cf7b0 con 0x7f3a9806f2f0 2023-12-01T01:55:33.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3a900019f0 con 0x7f3a9806f2f0 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 >> v1:172.21.15.115:6789/0 conn(0x7f3a9806f2f0 legacy=0x7f3a9806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a84003a10 0x7f3a84003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980cad70 0x7f3a840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a84004d00 0x7f3a84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9815e6d0 con 0x7f3a84004d00 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8400bbb0 con 0x7f3a84003a10 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8400bd70 con 0x7f3a980cad70 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.826+0000 7f3a9c86c640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a84004d00 0x7f3a84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.827+0000 7f3a8ffff640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980cad70 0x7f3a840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.827+0000 7f3a97fff640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a84003a10 0x7f3a84003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.827+0000 7f3a9c86c640 1 -- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980cad70 msgr2=0x7f3a840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.827+0000 7f3a9c86c640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980cad70 0x7f3a840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.827+0000 7f3a9c86c640 1 -- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a84003a10 msgr2=0x7f3a84003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.827+0000 7f3a9c86c640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a84003a10 0x7f3a84003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:33.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.827+0000 7f3a9c86c640 1 -- 172.21.15.115:0/2601013721 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a84003520 con 0x7f3a84004d00 2023-12-01T01:55:33.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.828+0000 7f3a9c86c640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a84004d00 0x7f3a84005180 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000a830 tx=0x7f3a840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e80259cc45cbc0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:33.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.828+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900072d0 con 0x7f3a84004d00 2023-12-01T01:55:33.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.828+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9002f070 con 0x7f3a84004d00 2023-12-01T01:55:33.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.828+0000 7f3a977fe640 1 -- 172.21.15.115:0/2601013721 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900076b0 con 0x7f3a84004d00 2023-12-01T01:55:33.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.829+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/2601013721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a84004d00 msgr2=0x7f3a84005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:33.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.829+0000 7f3a9e2cf640 1 --2- 172.21.15.115:0/2601013721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a84004d00 0x7f3a84005180 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000a830 tx=0x7f3a840057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:33.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.829+0000 7f3a9c86c640 1 -- 172.21.15.115:0/2601013721 reap_dead start 2023-12-01T01:55:33.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.829+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/2601013721 shutdown_connections 2023-12-01T01:55:33.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.829+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/2601013721 >> 172.21.15.115:0/2601013721 conn(0x7f3a980ce880 msgr2=0x7f3a980cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:33.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.829+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/2601013721 shutdown_connections 2023-12-01T01:55:33.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.829+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/2601013721 wait complete. 2023-12-01T01:55:33.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.831+0000 7f3a9e2cf640 1 Processor -- start 2023-12-01T01:55:33.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.831+0000 7f3a9e2cf640 1 -- start start 2023-12-01T01:55:33.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a9e2cf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a84004d00 0x7f3a981516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a9e2cf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980ac8a0 0x7f3a980acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a9e2cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a980c4480 0x7f3a980c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a9e2cf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980ad260 con 0x7f3a980c4480 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a9e2cf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a981f3270 con 0x7f3a84004d00 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a9e2cf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9815c780 con 0x7f3a980ac8a0 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a9c86c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a980c4480 0x7f3a980c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a9c86c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a980c4480 0x7f3a980c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55990/0 (socket says 172.21.15.115:55990) 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a9c86c640 1 -- 172.21.15.115:0/1324585345 learned_addr learned my addr 172.21.15.115:0/1324585345 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a97fff640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a84004d00 0x7f3a981516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.832+0000 7f3a8ffff640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980ac8a0 0x7f3a980acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a9c86c640 1 -- 172.21.15.115:0/1324585345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980ac8a0 msgr2=0x7f3a980acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a9c86c640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980ac8a0 0x7f3a980acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a9c86c640 1 -- 172.21.15.115:0/1324585345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a84004d00 msgr2=0x7f3a981516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a9c86c640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a84004d00 0x7f3a981516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:33.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a9c86c640 1 -- 172.21.15.115:0/1324585345 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9815be50 con 0x7f3a980c4480 2023-12-01T01:55:33.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a9c86c640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a980c4480 0x7f3a980c4900 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f3a9002cb70 tx=0x7f3a90001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:33.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a957fa640 1 -- 172.21.15.115:0/1324585345 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a90004b60 con 0x7f3a980c4480 2023-12-01T01:55:33.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a957fa640 1 -- 172.21.15.115:0/1324585345 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a90004cc0 con 0x7f3a980c4480 2023-12-01T01:55:33.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a957fa640 1 -- 172.21.15.115:0/1324585345 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900072d0 con 0x7f3a980c4480 2023-12-01T01:55:33.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/1324585345 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9815c130 con 0x7f3a980c4480 2023-12-01T01:55:33.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.833+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/1324585345 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a980c4e40 con 0x7f3a980c4480 2023-12-01T01:55:33.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.834+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/1324585345 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9806fda0 con 0x7f3a980c4480 2023-12-01T01:55:33.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.835+0000 7f3a957fa640 1 -- 172.21.15.115:0/1324585345 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3a90031030 con 0x7f3a980c4480 2023-12-01T01:55:33.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.836+0000 7f3a957fa640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a7806a580 0x7f3a7806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:33.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.836+0000 7f3a957fa640 1 -- 172.21.15.115:0/1324585345 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a900aa700 con 0x7f3a980c4480 2023-12-01T01:55:33.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.836+0000 7f3a97fff640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a7806a580 0x7f3a7806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:33.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.837+0000 7f3a97fff640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a7806a580 0x7f3a7806ca40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007fd0 tx=0x7f3a88009040 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:33.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:33.841+0000 7f3a957fa640 1 -- 172.21.15.115:0/1324585345 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3a90075ad0 con 0x7f3a980c4480 2023-12-01T01:55:34.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.057+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/1324585345 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a980b43d0 con 0x7f3a7806a580 2023-12-01T01:55:34.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.060+0000 7f3a957fa640 1 -- 172.21.15.115:0/1324585345 <== mgr.8247 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3a880013b0 con 0x7f3a7806a580 2023-12-01T01:55:34.062 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T01:55:34.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T01:55:34.062 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T01:55:34.062 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T01:55:34.062 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T01:55:34.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/1324585345 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a7806a580 msgr2=0x7f3a7806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a7806a580 0x7f3a7806ca40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007fd0 tx=0x7f3a88009040 comp rx=0 tx=0).stop 2023-12-01T01:55:34.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/1324585345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a980c4480 msgr2=0x7f3a980c4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a980c4480 0x7f3a980c4900 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f3a9002cb70 tx=0x7f3a90001d80 comp rx=0 tx=0).stop 2023-12-01T01:55:34.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/1324585345 shutdown_connections 2023-12-01T01:55:34.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3a7806a580 0x7f3a7806ca40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a980ac8a0 0x7f3a980acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a980c4480 0x7f3a980c4900 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 --2- 172.21.15.115:0/1324585345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a84004d00 0x7f3a981516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/1324585345 >> 172.21.15.115:0/1324585345 conn(0x7f3a980ce880 msgr2=0x7f3a98158760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:34.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/1324585345 shutdown_connections 2023-12-01T01:55:34.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.063+0000 7f3a9e2cf640 1 -- 172.21.15.115:0/1324585345 wait complete. 2023-12-01T01:55:34.083 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T01:55:34.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.288+0000 7f2697fff640 1 Processor -- start 2023-12-01T01:55:34.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.288+0000 7f2697fff640 1 -- start start 2023-12-01T01:55:34.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.289+0000 7f2697fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f269815cb40 0x7f26980d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.289+0000 7f2697fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f26980b8400 0x7f26980cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.289+0000 7f2697fff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26980d1e00 con 0x7f26980cdc80 2023-12-01T01:55:34.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.289+0000 7f2697fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26980cd250 con 0x7f26980b8400 2023-12-01T01:55:34.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.289+0000 7f2697fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2698069cc0 con 0x7f269815cb40 2023-12-01T01:55:34.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.289+0000 7f26967fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f26980b8400 0x7f26980cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.289+0000 7f26967fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f26980b8400 0x7f26980cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55996/0 (socket says 172.21.15.115:55996) 2023-12-01T01:55:34.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.289+0000 7f26967fc640 1 -- 172.21.15.115:0/1769458336 learned_addr learned my addr 172.21.15.115:0/1769458336 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:34.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.290+0000 7f2696ffd640 1 --2- 172.21.15.115:0/1769458336 >> v2:172.21.15.142:3300/0 conn(0x7f269815cb40 0x7f26980d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.290+0000 7f26967fc640 1 -- 172.21.15.115:0/1769458336 >> v2:172.21.15.142:3300/0 conn(0x7f269815cb40 msgr2=0x7f26980d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:34.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.290+0000 7f26967fc640 1 --2- 172.21.15.115:0/1769458336 >> v2:172.21.15.142:3300/0 conn(0x7f269815cb40 0x7f26980d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.290+0000 7f26967fc640 1 -- 172.21.15.115:0/1769458336 >> v1:172.21.15.115:6789/0 conn(0x7f26980cdc80 legacy=0x7f26981607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.290+0000 7f26967fc640 1 -- 172.21.15.115:0/1769458336 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26980ce0e0 con 0x7f26980b8400 2023-12-01T01:55:34.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.290+0000 7f2696ffd640 1 --2- 172.21.15.115:0/1769458336 >> v2:172.21.15.142:3300/0 conn(0x7f269815cb40 0x7f26980d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:34.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.291+0000 7f26967fc640 1 --2- 172.21.15.115:0/1769458336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 0x7f26980cd740 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f26980d3b70 tx=0x7f268002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab867c336670fe85 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:34.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.291+0000 7f2695ffb640 1 -- 172.21.15.115:0/1769458336 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268002cb10 con 0x7f26980b8400 2023-12-01T01:55:34.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.291+0000 7f2695ffb640 1 -- 172.21.15.115:0/1769458336 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268002cc70 con 0x7f26980b8400 2023-12-01T01:55:34.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.291+0000 7f2695ffb640 1 -- 172.21.15.115:0/1769458336 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800343f0 con 0x7f26980b8400 2023-12-01T01:55:34.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.292+0000 7f2697fff640 1 -- 172.21.15.115:0/1769458336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 msgr2=0x7f26980cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.292+0000 7f2697fff640 1 --2- 172.21.15.115:0/1769458336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 0x7f26980cd740 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f26980d3b70 tx=0x7f268002bf70 comp rx=0 tx=0).stop 2023-12-01T01:55:34.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.292+0000 7f2697fff640 1 -- 172.21.15.115:0/1769458336 shutdown_connections 2023-12-01T01:55:34.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.292+0000 7f2697fff640 1 --2- 172.21.15.115:0/1769458336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 0x7f26980cd740 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.292+0000 7f2697fff640 1 --2- 172.21.15.115:0/1769458336 >> v2:172.21.15.142:3300/0 conn(0x7f269815cb40 0x7f26980d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.292+0000 7f2697fff640 1 -- 172.21.15.115:0/1769458336 >> 172.21.15.115:0/1769458336 conn(0x7f26980d2a70 msgr2=0x7f26980d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:34.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.292+0000 7f2697fff640 1 -- 172.21.15.115:0/1769458336 shutdown_connections 2023-12-01T01:55:34.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.292+0000 7f2697fff640 1 -- 172.21.15.115:0/1769458336 wait complete. 2023-12-01T01:55:34.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.294+0000 7f2697fff640 1 Processor -- start 2023-12-01T01:55:34.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.294+0000 7f2697fff640 1 -- start start 2023-12-01T01:55:34.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.295+0000 7f2697fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 0x7f2698156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.295+0000 7f2697fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26980cdc80 0x7f269815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.295+0000 7f2697fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f269815cb40 0x7f269806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.295+0000 7f2697fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26980c75f0 con 0x7f26980b8400 2023-12-01T01:55:34.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.295+0000 7f2697fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26981596e0 con 0x7f269815cb40 2023-12-01T01:55:34.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.295+0000 7f2697fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26981599c0 con 0x7f26980cdc80 2023-12-01T01:55:34.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.295+0000 7f2696ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 0x7f2698156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.295+0000 7f2696ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 0x7f2698156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56012/0 (socket says 172.21.15.115:56012) 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.295+0000 7f2696ffd640 1 -- 172.21.15.115:0/3281170637 learned_addr learned my addr 172.21.15.115:0/3281170637 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f26967fc640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26980cdc80 0x7f269815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f26977fe640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f269815cb40 0x7f269806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f2696ffd640 1 -- 172.21.15.115:0/3281170637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26980cdc80 msgr2=0x7f269815a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f2696ffd640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26980cdc80 0x7f269815a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f2696ffd640 1 -- 172.21.15.115:0/3281170637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f269815cb40 msgr2=0x7f269806a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f2696ffd640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f269815cb40 0x7f269806a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f2696ffd640 1 -- 172.21.15.115:0/3281170637 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2680009b30 con 0x7f26980b8400 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f26967fc640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26980cdc80 0x7f269815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f26977fe640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f269815cb40 0x7f269806a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f2696ffd640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 0x7f2698156040 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f268800c7f0 tx=0x7f268800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:34.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f268f7fe640 1 -- 172.21.15.115:0/3281170637 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688007a30 con 0x7f26980b8400 2023-12-01T01:55:34.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f268f7fe640 1 -- 172.21.15.115:0/3281170637 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2688007b90 con 0x7f26980b8400 2023-12-01T01:55:34.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f2697fff640 1 -- 172.21.15.115:0/3281170637 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2698159ca0 con 0x7f26980b8400 2023-12-01T01:55:34.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.296+0000 7f268f7fe640 1 -- 172.21.15.115:0/3281170637 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268800ce40 con 0x7f26980b8400 2023-12-01T01:55:34.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.297+0000 7f2697fff640 1 -- 172.21.15.115:0/3281170637 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2698156580 con 0x7f26980b8400 2023-12-01T01:55:34.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.298+0000 7f2697fff640 1 -- 172.21.15.115:0/3281170637 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26980c7a80 con 0x7f26980b8400 2023-12-01T01:55:34.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.298+0000 7f268f7fe640 1 -- 172.21.15.115:0/3281170637 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 72) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f26880113f0 con 0x7f26980b8400 2023-12-01T01:55:34.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.300+0000 7f268f7fe640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f267c06a550 0x7f267c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.301+0000 7f268f7fe640 1 -- 172.21.15.115:0/3281170637 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2688088ce0 con 0x7f26980b8400 2023-12-01T01:55:34.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.301+0000 7f26967fc640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f267c06a550 0x7f267c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.301+0000 7f26967fc640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f267c06a550 0x7f267c06ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2680035000 tx=0x7f2680030000 comp rx=0 tx=0).ready entity=mgr.8247 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:34.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.305+0000 7f268f7fe640 1 -- 172.21.15.115:0/3281170637 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f26980c7a80 con 0x7f26980b8400 2023-12-01T01:55:34.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.548+0000 7f2697fff640 1 -- 172.21.15.115:0/3281170637 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f26980c7a80 con 0x7f26980b8400 2023-12-01T01:55:34.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.623+0000 7f268f7fe640 1 -- 172.21.15.115:0/3281170637 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 73) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f2688087200 con 0x7f26980b8400 2023-12-01T01:55:34.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.624+0000 7f268f7fe640 1 -- 172.21.15.115:0/3281170637 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v73) v1 ==== 92+0+0 (secure 0 0 0) 0x7f268804fde0 con 0x7f26980b8400 2023-12-01T01:55:34.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 -- 172.21.15.115:0/3281170637 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f267c06a550 msgr2=0x7f267c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f267c06a550 0x7f267c06ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2680035000 tx=0x7f2680030000 comp rx=0 tx=0).stop 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 -- 172.21.15.115:0/3281170637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 msgr2=0x7f2698156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 0x7f2698156040 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f268800c7f0 tx=0x7f268800cbf0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 -- 172.21.15.115:0/3281170637 shutdown_connections 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f267c06a550 0x7f267c06ca10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26980cdc80 0x7f269815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f269815cb40 0x7f269806a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 --2- 172.21.15.115:0/3281170637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26980b8400 0x7f2698156040 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 -- 172.21.15.115:0/3281170637 >> 172.21.15.115:0/3281170637 conn(0x7f26980d2a70 msgr2=0x7f26980cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 -- 172.21.15.115:0/3281170637 shutdown_connections 2023-12-01T01:55:34.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.628+0000 7f2697fff640 1 -- 172.21.15.115:0/3281170637 wait complete. 2023-12-01T01:55:34.648 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T01:55:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.841+0000 7fb1a99d9640 1 Processor -- start 2023-12-01T01:55:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.842+0000 7fb1a99d9640 1 -- start start 2023-12-01T01:55:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.843+0000 7fb1a99d9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb1a40cfd60 0x7fb1a40cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.843+0000 7fb1a99d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40b5fa0 0x7fb1a4161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.843+0000 7fb1a99d9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40ce5e0 con 0x7fb1a415cd90 2023-12-01T01:55:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.843+0000 7fb1a99d9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a40d0ec0 con 0x7fb1a40cfd60 2023-12-01T01:55:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.843+0000 7fb1a99d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4161950 con 0x7fb1a40b5fa0 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.843+0000 7fb1a2ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb1a415cd90 0x7fb1a40cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36502/0 (socket says 172.21.15.115:36502) 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.843+0000 7fb1a2ffd640 1 -- 172.21.15.115:0/2395931726 learned_addr learned my addr 172.21.15.115:0/2395931726 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a27fc640 1 --2- 172.21.15.115:0/2395931726 >> v2:172.21.15.142:3300/0 conn(0x7fb1a40cfd60 0x7fb1a40cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a37fe640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40b5fa0 0x7fb1a4161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048222198 0 0) 0x7fb194001110 con 0x7fb1a415cd90 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb18c003520 con 0x7fb1a415cd90 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2842414006 0 0) 0x7fb18c003520 con 0x7fb1a415cd90 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40b5fa0 msgr2=0x7fb1a4161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40b5fa0 0x7fb1a4161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 >> v2:172.21.15.142:3300/0 conn(0x7fb1a40cfd60 msgr2=0x7fb1a40cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 --2- 172.21.15.115:0/2395931726 >> v2:172.21.15.142:3300/0 conn(0x7fb1a40cfd60 0x7fb1a40cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40ce300 con 0x7fb1a415cd90 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb194001800 con 0x7fb1a415cd90 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 >> v1:172.21.15.115:6789/0 conn(0x7fb1a415cd90 legacy=0x7fb1a40cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40b5fa0 0x7fb18c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c003ef0 0x7fb18c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18c004d00 0x7fb18c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40d0ec0 con 0x7fb18c004d00 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4161950 con 0x7fb18c003ef0 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.844+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c00b980 con 0x7fb1a40b5fa0 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.845+0000 7fb1a37fe640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18c004d00 0x7fb18c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.845+0000 7fb1a27fc640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c003ef0 0x7fb18c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.845+0000 7fb1a2ffd640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40b5fa0 0x7fb18c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.845+0000 7fb1a37fe640 1 -- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40b5fa0 msgr2=0x7fb18c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.845+0000 7fb1a37fe640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40b5fa0 0x7fb18c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.845+0000 7fb1a37fe640 1 -- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c003ef0 msgr2=0x7fb18c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.845+0000 7fb1a37fe640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c003ef0 0x7fb18c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.845+0000 7fb1a37fe640 1 -- 172.21.15.115:0/2395931726 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c003520 con 0x7fb18c004d00 2023-12-01T01:55:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.845+0000 7fb1a27fc640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c003ef0 0x7fb18c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:55:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.846+0000 7fb1a37fe640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18c004d00 0x7fb18c005180 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009fc0 tx=0x7fb18c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d977bfd1287d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.846+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c02cc00 con 0x7fb18c004d00 2023-12-01T01:55:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.847+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c00f040 con 0x7fb18c004d00 2023-12-01T01:55:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.847+0000 7fb1a1ffb640 1 -- 172.21.15.115:0/2395931726 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c034440 con 0x7fb18c004d00 2023-12-01T01:55:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.847+0000 7fb1a99d9640 1 -- 172.21.15.115:0/2395931726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18c004d00 msgr2=0x7fb18c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.847+0000 7fb1a99d9640 1 --2- 172.21.15.115:0/2395931726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18c004d00 0x7fb18c005180 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009fc0 tx=0x7fb18c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.847+0000 7fb1a37fe640 1 -- 172.21.15.115:0/2395931726 reap_dead start 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.847+0000 7fb1a99d9640 1 -- 172.21.15.115:0/2395931726 shutdown_connections 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.847+0000 7fb1a99d9640 1 -- 172.21.15.115:0/2395931726 >> 172.21.15.115:0/2395931726 conn(0x7fb1a4065960 msgr2=0x7fb1a4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.847+0000 7fb1a99d9640 1 -- 172.21.15.115:0/2395931726 shutdown_connections 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.847+0000 7fb1a99d9640 1 -- 172.21.15.115:0/2395931726 wait complete. 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.849+0000 7fb1a99d9640 1 Processor -- start 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.849+0000 7fb1a99d9640 1 -- start start 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.850+0000 7fb1a99d9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c004d00 0x7fb1a414fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.850+0000 7fb1a99d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40d2ba0 0x7fb1a40d3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.850+0000 7fb1a99d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a41516b0 0x7fb1a4151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.850+0000 7fb1a99d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40d3560 con 0x7fb1a41516b0 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.850+0000 7fb1a99d9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a41f3860 con 0x7fb18c004d00 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.850+0000 7fb1a99d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a414f900 con 0x7fb1a40d2ba0 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.850+0000 7fb1a37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a41516b0 0x7fb1a4151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.850+0000 7fb1a37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a41516b0 0x7fb1a4151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56046/0 (socket says 172.21.15.115:56046) 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.850+0000 7fb1a37fe640 1 -- 172.21.15.115:0/1009045720 learned_addr learned my addr 172.21.15.115:0/1009045720 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb1a2ffd640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c004d00 0x7fb1a414fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb1a37fe640 1 -- 172.21.15.115:0/1009045720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40d2ba0 msgr2=0x7fb1a40d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb1a27fc640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40d2ba0 0x7fb1a40d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb1a37fe640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40d2ba0 0x7fb1a40d3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb1a37fe640 1 -- 172.21.15.115:0/1009045720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c004d00 msgr2=0x7fb1a414fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb1a37fe640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c004d00 0x7fb1a414fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb1a37fe640 1 -- 172.21.15.115:0/1009045720 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a41f3580 con 0x7fb1a41516b0 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb1a2ffd640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c004d00 0x7fb1a414fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb1a37fe640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a41516b0 0x7fb1a4151b30 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fb19c02c6d0 tx=0x7fb19c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb18bfff640 1 -- 172.21.15.115:0/1009045720 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c034d50 con 0x7fb1a41516b0 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb18bfff640 1 -- 172.21.15.115:0/1009045720 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c003990 con 0x7fb1a41516b0 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.851+0000 7fb18bfff640 1 -- 172.21.15.115:0/1009045720 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c003d40 con 0x7fb1a41516b0 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.852+0000 7fb1a99d9640 1 -- 172.21.15.115:0/1009045720 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a414f2b0 con 0x7fb1a41516b0 2023-12-01T01:55:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.852+0000 7fb1a99d9640 1 -- 172.21.15.115:0/1009045720 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a4152070 con 0x7fb1a41516b0 2023-12-01T01:55:34.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.853+0000 7fb1a99d9640 1 -- 172.21.15.115:0/1009045720 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a414fa90 con 0x7fb1a41516b0 2023-12-01T01:55:34.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.854+0000 7fb18bfff640 1 -- 172.21.15.115:0/1009045720 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 73) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fb19c004030 con 0x7fb1a41516b0 2023-12-01T01:55:34.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.855+0000 7fb18bfff640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb19006a5d0 0x7fb19006ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:34.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.855+0000 7fb1a2ffd640 1 -- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb19006a5d0 msgr2=0x7fb19006ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:34.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.855+0000 7fb18bfff640 1 -- 172.21.15.115:0/1009045720 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb19c0aa580 con 0x7fb1a41516b0 2023-12-01T01:55:34.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.855+0000 7fb1a2ffd640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb19006a5d0 0x7fb19006ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:34.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:34.860+0000 7fb18bfff640 1 -- 172.21.15.115:0/1009045720 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb19c075ba0 con 0x7fb1a41516b0 2023-12-01T01:55:35.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.048+0000 7fb1a99d9640 1 -- 172.21.15.115:0/1009045720 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fb1a404e890 con 0x7fb1a41516b0 2023-12-01T01:55:35.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.056+0000 7fb1a2ffd640 1 -- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb19006a5d0 msgr2=0x7fb19006ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:35.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.056+0000 7fb1a2ffd640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb19006a5d0 0x7fb19006ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:55:35.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.057+0000 7fb18bfff640 1 -- 172.21.15.115:0/1009045720 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb19c0791c0 con 0x7fb1a41516b0 2023-12-01T01:55:35.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.059+0000 7fb1a99d9640 1 -- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb19006a5d0 msgr2=0x7fb19006ca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:35.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.059+0000 7fb1a99d9640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb19006a5d0 0x7fb19006ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.059+0000 7fb1a99d9640 1 -- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a41516b0 msgr2=0x7fb1a4151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.059+0000 7fb1a99d9640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a41516b0 0x7fb1a4151b30 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fb19c02c6d0 tx=0x7fb19c009750 comp rx=0 tx=0).stop 2023-12-01T01:55:35.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.059+0000 7fb1a99d9640 1 -- 172.21.15.115:0/1009045720 shutdown_connections 2023-12-01T01:55:35.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.059+0000 7fb1a99d9640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb19006a5d0 0x7fb19006ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.059+0000 7fb1a99d9640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a40d2ba0 0x7fb1a40d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.059+0000 7fb1a99d9640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a41516b0 0x7fb1a4151b30 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.059+0000 7fb1a99d9640 1 --2- 172.21.15.115:0/1009045720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c004d00 0x7fb1a414fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.059+0000 7fb1a99d9640 1 -- 172.21.15.115:0/1009045720 >> 172.21.15.115:0/1009045720 conn(0x7fb1a4065960 msgr2=0x7fb1a4158700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:35.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.060+0000 7fb1a99d9640 1 -- 172.21.15.115:0/1009045720 shutdown_connections 2023-12-01T01:55:35.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.060+0000 7fb1a99d9640 1 -- 172.21.15.115:0/1009045720 wait complete. 2023-12-01T01:55:35.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T01:55:35.131 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:35.129+0000 7f7c3766c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:55:35.149 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:35.147+0000 7fbb0642b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:55:35.183 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:35.181+0000 7f7c3766c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:55:35.204 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:35.203+0000 7fbb0642b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:55:35.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.271+0000 7f11511ab640 1 Processor -- start 2023-12-01T01:55:35.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.272+0000 7f11511ab640 1 -- start start 2023-12-01T01:55:35.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.273+0000 7f11511ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c15b200 0x7f114c0ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.273+0000 7f11511ab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114c15c1f0 con 0x7f114c071740 2023-12-01T01:55:35.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.273+0000 7f11511ab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114c0cba40 con 0x7f114c15b6b0 2023-12-01T01:55:35.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.273+0000 7f11511ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c1556f0 con 0x7f114c15b200 2023-12-01T01:55:35.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.273+0000 7f114ad9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f114c071740 0x7f114c070440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36508/0 (socket says 172.21.15.115:36508) 2023-12-01T01:55:35.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.273+0000 7f114ad9d640 1 -- 172.21.15.115:0/713404765 learned_addr learned my addr 172.21.15.115:0/713404765 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:35.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f1143fff640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c15b200 0x7f114c0ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114305882 0 0) 0x7f114c15c1f0 con 0x7f114c071740 2023-12-01T01:55:35.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1138003520 con 0x7f114c071740 2023-12-01T01:55:35.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4228175426 0 0) 0x7f1138003520 con 0x7f114c071740 2023-12-01T01:55:35.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c15b200 msgr2=0x7f114c0ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:35.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c15b200 0x7f114c0ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 >> v1:172.21.15.142:6789/0 conn(0x7f114c15b6b0 legacy=0x7f114c0cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c070b50 con 0x7f114c071740 2023-12-01T01:55:35.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f113c001670 con 0x7f114c071740 2023-12-01T01:55:35.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 >> v1:172.21.15.115:6789/0 conn(0x7f114c071740 legacy=0x7f114c070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c15b200 0x7f11380039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 0x7f11380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1138004d00 0x7f1138005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c1556f0 con 0x7f1138003f20 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113800bb10 con 0x7f1138004d00 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113800bcd0 con 0x7f114c15b200 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.274+0000 7f1143fff640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 0x7f11380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.275+0000 7f114ad9d640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c15b200 0x7f11380039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.275+0000 7f114b59e640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1138004d00 0x7f1138005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.275+0000 7f1143fff640 1 -- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c15b200 msgr2=0x7f11380039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.275+0000 7f1143fff640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c15b200 0x7f11380039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.275+0000 7f1143fff640 1 -- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1138004d00 msgr2=0x7f1138005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.275+0000 7f1143fff640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1138004d00 0x7f1138005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.275+0000 7f1143fff640 1 -- 172.21.15.115:0/713404765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138003520 con 0x7f1138003f20 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.275+0000 7f114ad9d640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c15b200 0x7f11380039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:35.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.275+0000 7f114b59e640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1138004d00 0x7f1138005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:35.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.276+0000 7f1143fff640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 0x7f11380047c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f1134009ac0 tx=0x7f11380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8a37e906942711 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.276+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113402c6b0 con 0x7f1138003f20 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.276+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113402c810 con 0x7f1138003f20 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.276+0000 7f114a59c640 1 -- 172.21.15.115:0/713404765 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113402cba0 con 0x7f1138003f20 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.277+0000 7f11511ab640 1 -- 172.21.15.115:0/713404765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 msgr2=0x7f11380047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.277+0000 7f11511ab640 1 --2- 172.21.15.115:0/713404765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 0x7f11380047c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f1134009ac0 tx=0x7f11380057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.277+0000 7f114b59e640 1 -- 172.21.15.115:0/713404765 reap_dead start 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.277+0000 7f11511ab640 1 -- 172.21.15.115:0/713404765 shutdown_connections 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.277+0000 7f11511ab640 1 -- 172.21.15.115:0/713404765 >> 172.21.15.115:0/713404765 conn(0x7f114c06bd00 msgr2=0x7f114c06c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.277+0000 7f11511ab640 1 -- 172.21.15.115:0/713404765 shutdown_connections 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.277+0000 7f11511ab640 1 -- 172.21.15.115:0/713404765 wait complete. 2023-12-01T01:55:35.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.279+0000 7f11511ab640 1 Processor -- start 2023-12-01T01:55:35.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.279+0000 7f11511ab640 1 -- start start 2023-12-01T01:55:35.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f11511ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 0x7f114c1e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f11511ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f114c1e3ea0 0x7f114c13c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f11511ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c14d600 0x7f114c14da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f11511ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c14b4c0 con 0x7f1138003f20 2023-12-01T01:55:35.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f11511ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c14dfc0 con 0x7f114c1e3ea0 2023-12-01T01:55:35.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f11511ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c1e4320 con 0x7f114c14d600 2023-12-01T01:55:35.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f114ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 0x7f114c1e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f114ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 0x7f114c1e52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56064/0 (socket says 172.21.15.115:56064) 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f114ad9d640 1 -- 172.21.15.115:0/1197633694 learned_addr learned my addr 172.21.15.115:0/1197633694 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f1143fff640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f114c1e3ea0 0x7f114c13c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f114b59e640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c14d600 0x7f114c14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f114ad9d640 1 -- 172.21.15.115:0/1197633694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c14d600 msgr2=0x7f114c14da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f114ad9d640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c14d600 0x7f114c14da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f114ad9d640 1 -- 172.21.15.115:0/1197633694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f114c1e3ea0 msgr2=0x7f114c13c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f114ad9d640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f114c1e3ea0 0x7f114c13c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f114ad9d640 1 -- 172.21.15.115:0/1197633694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c1e45d0 con 0x7f1138003f20 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.280+0000 7f114b59e640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c14d600 0x7f114c14da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.281+0000 7f114ad9d640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 0x7f114c1e52c0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f113c00d8e0 tx=0x7f113c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.281+0000 7f11437fe640 1 -- 172.21.15.115:0/1197633694 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c0074e0 con 0x7f1138003f20 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.281+0000 7f11437fe640 1 -- 172.21.15.115:0/1197633694 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c007640 con 0x7f1138003f20 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.281+0000 7f11437fe640 1 -- 172.21.15.115:0/1197633694 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c004a00 con 0x7f1138003f20 2023-12-01T01:55:35.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.281+0000 7f11511ab640 1 -- 172.21.15.115:0/1197633694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c1e48b0 con 0x7f1138003f20 2023-12-01T01:55:35.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.281+0000 7f11511ab640 1 -- 172.21.15.115:0/1197633694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c1e4e70 con 0x7f1138003f20 2023-12-01T01:55:35.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.282+0000 7f11511ab640 1 -- 172.21.15.115:0/1197633694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c073590 con 0x7f1138003f20 2023-12-01T01:55:35.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.283+0000 7f11437fe640 1 -- 172.21.15.115:0/1197633694 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 73) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f113c004b60 con 0x7f1138003f20 2023-12-01T01:55:35.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.284+0000 7f11437fe640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f112c06a5a0 0x7f112c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.284+0000 7f11437fe640 1 -- 172.21.15.115:0/1197633694 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f113c088650 con 0x7f1138003f20 2023-12-01T01:55:35.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.284+0000 7f1143fff640 1 -- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f112c06a5a0 msgr2=0x7f112c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:35.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.284+0000 7f1143fff640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f112c06a5a0 0x7f112c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:35.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.289+0000 7f11437fe640 1 -- 172.21.15.115:0/1197633694 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f113c053ab0 con 0x7f1138003f20 2023-12-01T01:55:35.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.478+0000 7f11511ab640 1 -- 172.21.15.115:0/1197633694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f114c0cf870 con 0x7f1138003f20 2023-12-01T01:55:35.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.483+0000 7f11437fe640 1 -- 172.21.15.115:0/1197633694 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7f113c0570d0 con 0x7f1138003f20 2023-12-01T01:55:35.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f1143fff640 1 -- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f112c06a5a0 msgr2=0x7f112c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:35.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f1143fff640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f112c06a5a0 0x7f112c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:55:35.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f11511ab640 1 -- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f112c06a5a0 msgr2=0x7f112c06ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:35.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f11511ab640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f112c06a5a0 0x7f112c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f11511ab640 1 -- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 msgr2=0x7f114c1e52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f11511ab640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 0x7f114c1e52c0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f113c00d8e0 tx=0x7f113c00dda0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f11511ab640 1 -- 172.21.15.115:0/1197633694 shutdown_connections 2023-12-01T01:55:35.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f11511ab640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f112c06a5a0 0x7f112c06ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f11511ab640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f114c14d600 0x7f114c14da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f11511ab640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f114c1e3ea0 0x7f114c13c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.485+0000 7f11511ab640 1 --2- 172.21.15.115:0/1197633694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1138003f20 0x7f114c1e52c0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.486+0000 7f11511ab640 1 -- 172.21.15.115:0/1197633694 >> 172.21.15.115:0/1197633694 conn(0x7f114c06bd00 msgr2=0x7f114c14aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.486+0000 7f11511ab640 1 -- 172.21.15.115:0/1197633694 shutdown_connections 2023-12-01T01:55:35.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.486+0000 7f11511ab640 1 -- 172.21.15.115:0/1197633694 wait complete. 2023-12-01T01:55:35.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:55:35.505 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-01T01:55:35.563 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1 1M 2023-12-01T01:55:35.638 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:35.636+0000 7f7c3766c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:55:35.679 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:35.678+0000 7fbb0642b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:55:35.691 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:35.690+0000 7f7c3766c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:55:35.734 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:35.732+0000 7fbb0642b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:55:35.742 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:35.740+0000 7f7c3766c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:55:35.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.769+0000 7f4c10d39640 1 Processor -- start 2023-12-01T01:55:35.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.769+0000 7f4c10d39640 1 -- start start 2023-12-01T01:55:35.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.770+0000 7f4c10d39640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c06af70 0x7f4c0c0ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.770+0000 7f4c10d39640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0c0cb290 con 0x7f4c0c0cdb30 2023-12-01T01:55:35.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.770+0000 7f4c10d39640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0c0cb570 con 0x7f4c0c15d090 2023-12-01T01:55:35.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.770+0000 7f4c10d39640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c0cb850 con 0x7f4c0c06af70 2023-12-01T01:55:35.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.771+0000 7f4c09d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4c0c0cdb30 0x7f4c0c0cbd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:36516/0 (socket says 172.21.15.115:36516) 2023-12-01T01:55:35.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.771+0000 7f4c09d9b640 1 -- 172.21.15.115:0/3572383022 learned_addr learned my addr 172.21.15.115:0/3572383022 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:35.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.771+0000 7f4c0ad9d640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c06af70 0x7f4c0c0ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.771+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695507471 0 0) 0x7f4c0c0cb290 con 0x7f4c0c0cdb30 2023-12-01T01:55:35.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.771+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bf4003520 con 0x7f4c0c0cdb30 2023-12-01T01:55:35.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1521210955 0 0) 0x7f4bf4003520 con 0x7f4c0c0cdb30 2023-12-01T01:55:35.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c06af70 msgr2=0x7f4c0c0ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c06af70 0x7f4c0c0ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 >> v1:172.21.15.142:6789/0 conn(0x7f4c0c15d090 legacy=0x7f4c0c0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c0cfec0 con 0x7f4c0c0cdb30 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4c00001670 con 0x7f4c0c0cdb30 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 >> v1:172.21.15.115:6789/0 conn(0x7f4c0c0cdb30 legacy=0x7f4c0c0cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf40039a0 0x7f4bf4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf40047c0 0x7f4bf4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c06af70 0x7f4bf4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c0cb850 con 0x7f4bf40039a0 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf400ba60 con 0x7f4bf40047c0 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf400bc20 con 0x7f4c0c06af70 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf40039a0 0x7f4bf4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c09d9b640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf40047c0 0x7f4bf4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0a59c640 1 -- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c06af70 msgr2=0x7f4bf4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:35.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c06af70 0x7f4bf4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0a59c640 1 -- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf40047c0 msgr2=0x7f4bf4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:35.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf40047c0 0x7f4bf4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c0a59c640 1 -- 172.21.15.115:0/3572383022 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4003520 con 0x7f4bf40039a0 2023-12-01T01:55:35.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.772+0000 7f4c09d9b640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf40047c0 0x7f4bf4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:35.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.773+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf40039a0 0x7f4bf4003dc0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc001720 tx=0x7f4bf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eadb8f663de4def server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:35.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.774+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0052d0 con 0x7f4bf40039a0 2023-12-01T01:55:35.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.774+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc02e050 con 0x7f4bf40039a0 2023-12-01T01:55:35.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.774+0000 7f4c0959a640 1 -- 172.21.15.115:0/3572383022 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0035f0 con 0x7f4bf40039a0 2023-12-01T01:55:35.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.775+0000 7f4c10d39640 1 -- 172.21.15.115:0/3572383022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf40039a0 msgr2=0x7f4bf4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.775+0000 7f4c10d39640 1 --2- 172.21.15.115:0/3572383022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bf40039a0 0x7f4bf4003dc0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc001720 tx=0x7f4bf40057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.775+0000 7f4c0ad9d640 1 -- 172.21.15.115:0/3572383022 reap_dead start 2023-12-01T01:55:35.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.775+0000 7f4c10d39640 1 -- 172.21.15.115:0/3572383022 shutdown_connections 2023-12-01T01:55:35.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.775+0000 7f4c10d39640 1 -- 172.21.15.115:0/3572383022 >> 172.21.15.115:0/3572383022 conn(0x7f4c0c0d0ba0 msgr2=0x7f4c0c0d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:35.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.775+0000 7f4c10d39640 1 -- 172.21.15.115:0/3572383022 shutdown_connections 2023-12-01T01:55:35.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.775+0000 7f4c10d39640 1 -- 172.21.15.115:0/3572383022 wait complete. 2023-12-01T01:55:35.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.777+0000 7f4c10d39640 1 Processor -- start 2023-12-01T01:55:35.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.777+0000 7f4c10d39640 1 -- start start 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c10d39640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf40039a0 0x7f4c0c149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c10d39640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c0ca750 0x7f4c0c1e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c10d39640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c0c8680 0x7f4c0c0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c10d39640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c0ae010 con 0x7f4c0c0c8680 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c10d39640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c0c9040 con 0x7f4bf40039a0 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c10d39640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c14a510 con 0x7f4c0c0ca750 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c0ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c0c8680 0x7f4c0c0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c0ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c0c8680 0x7f4c0c0c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:56078/0 (socket says 172.21.15.115:56078) 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c0ad9d640 1 -- 172.21.15.115:0/638949182 learned_addr learned my addr 172.21.15.115:0/638949182 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf40039a0 0x7f4c0c149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c09d9b640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c0ca750 0x7f4c0c1e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c0ad9d640 1 -- 172.21.15.115:0/638949182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c0ca750 msgr2=0x7f4c0c1e62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c0ad9d640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c0ca750 0x7f4c0c1e62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c0ad9d640 1 -- 172.21.15.115:0/638949182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf40039a0 msgr2=0x7f4c0c149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:35.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c0ad9d640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf40039a0 0x7f4c0c149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:35.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c0ad9d640 1 -- 172.21.15.115:0/638949182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c1e6ad0 con 0x7f4c0c0c8680 2023-12-01T01:55:35.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.778+0000 7f4c09d9b640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c0ca750 0x7f4c0c1e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:35.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.779+0000 7f4c0ad9d640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c0c8680 0x7f4c0c0c8b00 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0ce120 tx=0x7f4c0400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:35.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.779+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04010040 con 0x7f4c0c0c8680 2023-12-01T01:55:35.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.779+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0400ba80 con 0x7f4c0c0c8680 2023-12-01T01:55:35.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.779+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0400be10 con 0x7f4c0c0c8680 2023-12-01T01:55:35.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.779+0000 7f4c10d39640 1 -- 172.21.15.115:0/638949182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0c1e6db0 con 0x7f4c0c0c8680 2023-12-01T01:55:35.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.779+0000 7f4c10d39640 1 -- 172.21.15.115:0/638949182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c0adc30 con 0x7f4c0c0c8680 2023-12-01T01:55:35.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.780+0000 7f4c10d39640 1 -- 172.21.15.115:0/638949182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0c06be90 con 0x7f4c0c0c8680 2023-12-01T01:55:35.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.781+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 73) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f4c0401d400 con 0x7f4c0c0c8680 2023-12-01T01:55:35.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.783+0000 7f4bfaffd640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:35.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.783+0000 7f4c0a59c640 1 -- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 msgr2=0x7f4bec06ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:35.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.783+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c040891b0 con 0x7f4c0c0c8680 2023-12-01T01:55:35.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.783+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:55:35.788 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:35.787+0000 7fbb0642b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:55:35.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.787+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4c04054570 con 0x7f4c0c0c8680 2023-12-01T01:55:35.803 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:35.801+0000 7f7c3766c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:55:35.852 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:35.850+0000 7fbb0642b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:55:35.852 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:35.850+0000 7f7c3766c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:55:35.902 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:35.901+0000 7fbb0642b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:55:35.962 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:35.960+0000 7f7c3766c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:55:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.984+0000 7f4c0a59c640 1 -- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 msgr2=0x7f4bec06ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:35.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.984+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:55:35.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:35.997+0000 7f4c10d39640 1 -- 172.21.15.115:0/638949182 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c0c15d470 con 0x7f4bec06a5d0 2023-12-01T01:55:36.016 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:36.015+0000 7fbb0642b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:55:36.023 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:36.022+0000 7f7c3766c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:55:36.072 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:36.071+0000 7f7c3766c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:55:36.079 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:36.077+0000 7fbb0642b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:55:36.128 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:36.127+0000 7fbb0642b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:55:36.129 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:36.127+0000 7f7c3766c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:55:36.179 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:36.177+0000 7f7c3766c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:55:36.186 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:36.185+0000 7fbb0642b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:55:36.235 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:36.234+0000 7fbb0642b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:55:36.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:36.385+0000 7f4c0a59c640 1 -- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 msgr2=0x7f4bec06ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:36.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:36.385+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:55:36.557 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:36.555+0000 7f7c3766c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:55:36.610 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:36.609+0000 7fbb0642b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:55:36.715 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:36.713+0000 7f7c3766c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:55:36.768 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:36.766+0000 7fbb0642b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:55:36.823 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:36.821+0000 7f7c3766c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:55:36.872 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:36.871+0000 7f7c3766c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:55:36.876 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:36.874+0000 7fbb0642b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:55:36.926 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:36.925+0000 7fbb0642b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:55:36.980 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:36.978+0000 7f7c3766c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:55:37.033 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:37.032+0000 7fbb0642b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:55:37.088 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:37.086+0000 7f7c3766c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:55:37.140 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:37.139+0000 7fbb0642b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:55:37.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:37.186+0000 7f4c0a59c640 1 -- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 msgr2=0x7f4bec06ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T01:55:37.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:37.186+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:55:37.242 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:37.240+0000 7f7c3766c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:55:37.290 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:37.289+0000 7f7c3766c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:55:37.297 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:37.295+0000 7fbb0642b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:55:37.346 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:37.344+0000 7fbb0642b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:55:37.408 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:37.407+0000 7f7c3766c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:55:37.464 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:37.462+0000 7fbb0642b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:55:38.224 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:38.223+0000 7f7c3766c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:55:38.287 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.286+0000 7fbb0642b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:55:38.517 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:38.516+0000 7f7c3766c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:55:38.573 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:38.573+0000 7f7c3766c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:55:38.581 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.579+0000 7fbb0642b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:55:38.638 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.636+0000 7fbb0642b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:55:38.686 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T01:55:38.686+0000 7f7c3766c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:55:38.753 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.751+0000 7fbb0642b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:55:38.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:38.759+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 74) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f4c04054940 con 0x7f4c0c0c8680 2023-12-01T01:55:38.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:38.782+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 75) v1 ==== 88335+0+0 (secure 0 0 0) 0x7f4c0403acb0 con 0x7f4c0c0c8680 2023-12-01T01:55:38.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:38.783+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 msgr2=0x7f4bec06ca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:55:38.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:38.783+0000 7f4bfaffd640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:38.956 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.955+0000 7fba858e1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:38.956 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.955+0000 7fba858e1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:38.956 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.955+0000 7fba858e1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:38.956 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.955+0000 7fba858e1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:38.956 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.955+0000 7fba858e1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:38.957 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.956+0000 7fba828db640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:38.957 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.956+0000 7fba828db640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:38.957 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.956+0000 7fba828db640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:38.957 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.956+0000 7fba828db640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:38.957 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:55:38.956+0000 7fba828db640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:55:39.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.790+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 <== mon.0 v2:172.21.15.115:3300/0 9 ==== mgrmap(e 76) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f4c04055210 con 0x7f4c0c0c8680 2023-12-01T01:55:39.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.791+0000 7f4bfaffd640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:39.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.791+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bec027fa0 con 0x7f4bec06a5d0 2023-12-01T01:55:39.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.798+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:39.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.809+0000 7f4c0a59c640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc033000 tx=0x7f4bfc009030 comp rx=0 tx=0).ready entity=mgr.8658 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:39.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.967+0000 7f4bfaffd640 1 -- 172.21.15.115:0/638949182 <== mgr.8658 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f4bfc0054a0 con 0x7f4bec06a5d0 2023-12-01T01:55:39.969 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 -- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 msgr2=0x7f4bec06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc033000 tx=0x7f4bfc009030 comp rx=0 tx=0).stop 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 -- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c0c8680 msgr2=0x7f4c0c0c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c0c8680 0x7f4c0c0c8b00 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0ce120 tx=0x7f4c0400b480 comp rx=0 tx=0).stop 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 -- 172.21.15.115:0/638949182 shutdown_connections 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4bec06a5d0 0x7f4bec06ca10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c0c0ca750 0x7f4c0c1e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c0c0c8680 0x7f4c0c0c8b00 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 --2- 172.21.15.115:0/638949182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf40039a0 0x7f4c0c149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 -- 172.21.15.115:0/638949182 >> 172.21.15.115:0/638949182 conn(0x7f4c0c0d0ba0 msgr2=0x7f4c0c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 -- 172.21.15.115:0/638949182 shutdown_connections 2023-12-01T01:55:39.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:39.971+0000 7f4c10d39640 1 -- 172.21.15.115:0/638949182 wait complete. 2023-12-01T01:55:39.994 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-12-01T01:55:39.994 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add /snap_test_dir1 55n 2023-12-01T01:55:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.204+0000 7fda4624a640 1 Processor -- start 2023-12-01T01:55:40.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.204+0000 7fda4624a640 1 -- start start 2023-12-01T01:55:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.206+0000 7fda4624a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fda4015ac70 0x7fda400cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.206+0000 7fda4624a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4006c0b0 con 0x7fda400adca0 2023-12-01T01:55:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.206+0000 7fda4624a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4006c860 con 0x7fda400b4070 2023-12-01T01:55:40.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.206+0000 7fda4624a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4015b050 con 0x7fda4015ac70 2023-12-01T01:55:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.206+0000 7fda3ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fda400adca0 0x7fda4006b9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38100/0 (socket says 172.21.15.115:38100) 2023-12-01T01:55:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.206+0000 7fda3ffff640 1 -- 172.21.15.115:0/984572385 learned_addr learned my addr 172.21.15.115:0/984572385 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T01:55:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.206+0000 7fda3f7fe640 1 --2- 172.21.15.115:0/984572385 >> v2:172.21.15.142:3300/0 conn(0x7fda4015ac70 0x7fda400cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332488088 0 0) 0x7fda4006c0b0 con 0x7fda400adca0 2023-12-01T01:55:40.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda2c003520 con 0x7fda400adca0 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 969259014 0 0) 0x7fda2c003520 con 0x7fda400adca0 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 >> v2:172.21.15.142:3300/0 conn(0x7fda4015ac70 msgr2=0x7fda400cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 --2- 172.21.15.115:0/984572385 >> v2:172.21.15.142:3300/0 conn(0x7fda4015ac70 0x7fda400cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 >> v1:172.21.15.142:6789/0 conn(0x7fda400b4070 legacy=0x7fda4006b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400ce0b0 con 0x7fda400adca0 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fda380019f0 con 0x7fda400adca0 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 >> v1:172.21.15.115:6789/0 conn(0x7fda400adca0 legacy=0x7fda4006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 --2- 172.21.15.115:0/984572385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda2c003970 0x7fda2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 --2- 172.21.15.115:0/984572385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda2c0047c0 0x7fda2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 --2- 172.21.15.115:0/984572385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c005180 0x7fda2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4015b050 con 0x7fda2c0047c0 2023-12-01T01:55:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c00c120 con 0x7fda2c005180 2023-12-01T01:55:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.207+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c00c2e0 con 0x7fda2c003970 2023-12-01T01:55:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.208+0000 7fda37fff640 1 --2- 172.21.15.115:0/984572385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda2c0047c0 0x7fda2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.208+0000 7fda3ffff640 1 --2- 172.21.15.115:0/984572385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c005180 0x7fda2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.208+0000 7fda37fff640 1 -- 172.21.15.115:0/984572385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda2c003970 msgr2=0x7fda2c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.208+0000 7fda37fff640 1 --2- 172.21.15.115:0/984572385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda2c003970 0x7fda2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.208+0000 7fda37fff640 1 -- 172.21.15.115:0/984572385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c005180 msgr2=0x7fda2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.208+0000 7fda37fff640 1 --2- 172.21.15.115:0/984572385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda2c005180 0x7fda2c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.208+0000 7fda37fff640 1 -- 172.21.15.115:0/984572385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c003520 con 0x7fda2c0047c0 2023-12-01T01:55:40.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.208+0000 7fda3ffff640 1 -- 172.21.15.115:0/984572385 reap_dead start 2023-12-01T01:55:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.209+0000 7fda37fff640 1 --2- 172.21.15.115:0/984572385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda2c0047c0 0x7fda2c004c40 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fda28001720 tx=0x7fda2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940484c219400973 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.209+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280052d0 con 0x7fda2c0047c0 2023-12-01T01:55:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.209+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2802e050 con 0x7fda2c0047c0 2023-12-01T01:55:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.209+0000 7fda3effd640 1 -- 172.21.15.115:0/984572385 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28003830 con 0x7fda2c0047c0 2023-12-01T01:55:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.210+0000 7fda4624a640 1 -- 172.21.15.115:0/984572385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda2c0047c0 msgr2=0x7fda2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.210+0000 7fda4624a640 1 --2- 172.21.15.115:0/984572385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda2c0047c0 0x7fda2c004c40 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fda28001720 tx=0x7fda2c0057f0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.210+0000 7fda4624a640 1 -- 172.21.15.115:0/984572385 shutdown_connections 2023-12-01T01:55:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.210+0000 7fda4624a640 1 --2- 172.21.15.115:0/984572385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda2c0047c0 0x7fda2c004c40 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.210+0000 7fda4624a640 1 -- 172.21.15.115:0/984572385 >> 172.21.15.115:0/984572385 conn(0x7fda400662c0 msgr2=0x7fda400b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.210+0000 7fda4624a640 1 -- 172.21.15.115:0/984572385 shutdown_connections 2023-12-01T01:55:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.210+0000 7fda4624a640 1 -- 172.21.15.115:0/984572385 wait complete. 2023-12-01T01:55:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.212+0000 7fda4624a640 1 Processor -- start 2023-12-01T01:55:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.212+0000 7fda4624a640 1 -- start start 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.213+0000 7fda4624a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda2c0047c0 0x7fda40148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.213+0000 7fda4624a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda400ac8a0 0x7fda400acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.213+0000 7fda4624a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda400c6280 0x7fda400c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.213+0000 7fda4624a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda400ad260 con 0x7fda400ac8a0 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.213+0000 7fda4624a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda400c6fb0 con 0x7fda400c6280 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.213+0000 7fda4624a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda400c7290 con 0x7fda2c0047c0 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda37fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda400ac8a0 0x7fda400acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda37fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda400ac8a0 0x7fda400acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60816/0 (socket says 172.21.15.115:60816) 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda37fff640 1 -- 172.21.15.115:0/2988887930 learned_addr learned my addr 172.21.15.115:0/2988887930 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda3ffff640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda400c6280 0x7fda400c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda37fff640 1 -- 172.21.15.115:0/2988887930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda2c0047c0 msgr2=0x7fda40148b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda3f7fe640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda2c0047c0 0x7fda40148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda37fff640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda2c0047c0 0x7fda40148b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda37fff640 1 -- 172.21.15.115:0/2988887930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda400c6280 msgr2=0x7fda400c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda37fff640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda400c6280 0x7fda400c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda37fff640 1 -- 172.21.15.115:0/2988887930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400c7570 con 0x7fda400ac8a0 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda3ffff640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda400c6280 0x7fda400c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.214+0000 7fda37fff640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda400ac8a0 0x7fda400acd20 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fda2802bbc0 tx=0x7fda28001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.215+0000 7fda3cff9640 1 -- 172.21.15.115:0/2988887930 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280076c0 con 0x7fda400ac8a0 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.215+0000 7fda3cff9640 1 -- 172.21.15.115:0/2988887930 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda28007a30 con 0x7fda400ac8a0 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.215+0000 7fda3cff9640 1 -- 172.21.15.115:0/2988887930 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28006be0 con 0x7fda400ac8a0 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.215+0000 7fda4624a640 1 -- 172.21.15.115:0/2988887930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda400c7850 con 0x7fda400ac8a0 2023-12-01T01:55:40.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.215+0000 7fda4624a640 1 -- 172.21.15.115:0/2988887930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda401490b0 con 0x7fda400ac8a0 2023-12-01T01:55:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.216+0000 7fda4624a640 1 -- 172.21.15.115:0/2988887930 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda40074210 con 0x7fda400ac8a0 2023-12-01T01:55:40.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.217+0000 7fda3cff9640 1 -- 172.21.15.115:0/2988887930 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 76) v1 ==== 88462+0+0 (secure 0 0 0) 0x7fda28006d40 con 0x7fda400ac8a0 2023-12-01T01:55:40.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.218+0000 7fda3cff9640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda2006a300 0x7fda2006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.218+0000 7fda3cff9640 1 -- 172.21.15.115:0/2988887930 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fda280aa470 con 0x7fda400ac8a0 2023-12-01T01:55:40.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.218+0000 7fda3f7fe640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda2006a300 0x7fda2006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.219+0000 7fda3f7fe640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda2006a300 0x7fda2006c7c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fda40068bb0 tx=0x7fda30008040 comp rx=0 tx=0).ready entity=mgr.8658 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:40.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.222+0000 7fda3cff9640 1 -- 172.21.15.115:0/2988887930 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fda28075b70 con 0x7fda400ac8a0 2023-12-01T01:55:40.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.436+0000 7fda4624a640 1 -- 172.21.15.115:0/2988887930 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7fda40151240 con 0x7fda2006a300 2023-12-01T01:55:40.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.569+0000 7fda3cff9640 1 -- 172.21.15.115:0/2988887930 <== mgr.8658 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7fda300107c0 con 0x7fda2006a300 2023-12-01T01:55:40.571 INFO:teuthology.orchestra.run.smithi115.stdout:Retention added to path /snap_test_dir1 2023-12-01T01:55:40.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 -- 172.21.15.115:0/2988887930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda2006a300 msgr2=0x7fda2006c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda2006a300 0x7fda2006c7c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fda40068bb0 tx=0x7fda30008040 comp rx=0 tx=0).stop 2023-12-01T01:55:40.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 -- 172.21.15.115:0/2988887930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda400ac8a0 msgr2=0x7fda400acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda400ac8a0 0x7fda400acd20 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fda2802bbc0 tx=0x7fda28001120 comp rx=0 tx=0).stop 2023-12-01T01:55:40.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 -- 172.21.15.115:0/2988887930 shutdown_connections 2023-12-01T01:55:40.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fda2006a300 0x7fda2006c7c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda400c6280 0x7fda400c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda400ac8a0 0x7fda400acd20 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 --2- 172.21.15.115:0/2988887930 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda2c0047c0 0x7fda40148b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 -- 172.21.15.115:0/2988887930 >> 172.21.15.115:0/2988887930 conn(0x7fda400662c0 msgr2=0x7fda400b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:40.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 -- 172.21.15.115:0/2988887930 shutdown_connections 2023-12-01T01:55:40.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.572+0000 7fda4624a640 1 -- 172.21.15.115:0/2988887930 wait complete. 2023-12-01T01:55:40.593 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-12-01T01:55:40.593 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-12-01T01:55:40.593 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json /snap_test_dir1 2023-12-01T01:55:40.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.803+0000 7fa5fffff640 1 Processor -- start 2023-12-01T01:55:40.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.804+0000 7fa5fffff640 1 -- start start 2023-12-01T01:55:40.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.805+0000 7fa5fffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa60015cd80 0x7fa6001612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.805+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000b8400 0x7fa6000d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.805+0000 7fa5fffff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6000d0f20 con 0x7fa6000d2470 2023-12-01T01:55:40.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.805+0000 7fa5fffff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa600161820 con 0x7fa60015cd80 2023-12-01T01:55:40.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.805+0000 7fa5fffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60015e170 con 0x7fa6000b8400 2023-12-01T01:55:40.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.806+0000 7fa5feffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa60015cd80 0x7fa6001612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.806+0000 7fa5feffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa60015cd80 0x7fa6001612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60828/0 (socket says 172.21.15.115:60828) 2023-12-01T01:55:40.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.806+0000 7fa5feffd640 1 -- 172.21.15.115:0/2083780504 learned_addr learned my addr 172.21.15.115:0/2083780504 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:40.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.806+0000 7fa5feffd640 1 -- 172.21.15.115:0/2083780504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000b8400 msgr2=0x7fa6000d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:55:40.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.806+0000 7fa5f67fc640 1 --2- 172.21.15.115:0/2083780504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000b8400 0x7fa6000d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.806+0000 7fa5feffd640 1 --2- 172.21.15.115:0/2083780504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000b8400 0x7fa6000d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.806+0000 7fa5feffd640 1 -- 172.21.15.115:0/2083780504 >> v1:172.21.15.115:6789/0 conn(0x7fa6000d2470 legacy=0x7fa600069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.806+0000 7fa5feffd640 1 -- 172.21.15.115:0/2083780504 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6000d1170 con 0x7fa60015cd80 2023-12-01T01:55:40.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.806+0000 7fa5f67fc640 1 --2- 172.21.15.115:0/2083780504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000b8400 0x7fa6000d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:40.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.807+0000 7fa5feffd640 1 --2- 172.21.15.115:0/2083780504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 0x7fa6001612e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009e80 tx=0x7fa5f002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b86df31d94e136 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:40.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.808+0000 7fa5fe7fc640 1 -- 172.21.15.115:0/2083780504 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f002cae0 con 0x7fa60015cd80 2023-12-01T01:55:40.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.808+0000 7fa5fe7fc640 1 -- 172.21.15.115:0/2083780504 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f002cc40 con 0x7fa60015cd80 2023-12-01T01:55:40.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.808+0000 7fa5fe7fc640 1 -- 172.21.15.115:0/2083780504 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f00343f0 con 0x7fa60015cd80 2023-12-01T01:55:40.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.808+0000 7fa5fffff640 1 -- 172.21.15.115:0/2083780504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 msgr2=0x7fa6001612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.808+0000 7fa5fffff640 1 --2- 172.21.15.115:0/2083780504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 0x7fa6001612e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009e80 tx=0x7fa5f002bf40 comp rx=0 tx=0).stop 2023-12-01T01:55:40.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.808+0000 7fa5fffff640 1 -- 172.21.15.115:0/2083780504 shutdown_connections 2023-12-01T01:55:40.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.808+0000 7fa5fffff640 1 --2- 172.21.15.115:0/2083780504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000b8400 0x7fa6000d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.808+0000 7fa5fffff640 1 --2- 172.21.15.115:0/2083780504 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 0x7fa6001612e0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.808+0000 7fa5fffff640 1 -- 172.21.15.115:0/2083780504 >> 172.21.15.115:0/2083780504 conn(0x7fa6000d2a70 msgr2=0x7fa6000d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:40.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.809+0000 7fa5fffff640 1 -- 172.21.15.115:0/2083780504 shutdown_connections 2023-12-01T01:55:40.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.809+0000 7fa5fffff640 1 -- 172.21.15.115:0/2083780504 wait complete. 2023-12-01T01:55:40.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.811+0000 7fa5fffff640 1 Processor -- start 2023-12-01T01:55:40.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.811+0000 7fa5fffff640 1 -- start start 2023-12-01T01:55:40.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6000b8400 0x7fa60015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000d2470 0x7fa60015d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 0x7fa6000cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5fffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000aa830 con 0x7fa60015cd80 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5fffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000b1bb0 con 0x7fa6000b8400 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5fffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60015dd30 con 0x7fa6000d2470 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5ff7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 0x7fa6000cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5ff7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 0x7fa6000cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60844/0 (socket says 172.21.15.115:60844) 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5ff7fe640 1 -- 172.21.15.115:0/3859007034 learned_addr learned my addr 172.21.15.115:0/3859007034 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5f67fc640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000d2470 0x7fa60015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5ff7fe640 1 -- 172.21.15.115:0/3859007034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000d2470 msgr2=0x7fa60015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5ff7fe640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000d2470 0x7fa60015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5ff7fe640 1 -- 172.21.15.115:0/3859007034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6000b8400 msgr2=0x7fa60015c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:40.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5ff7fe640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6000b8400 0x7fa60015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:40.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.812+0000 7fa5ff7fe640 1 -- 172.21.15.115:0/3859007034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f0009b30 con 0x7fa60015cd80 2023-12-01T01:55:40.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.813+0000 7fa5f67fc640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000d2470 0x7fa60015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:55:40.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.813+0000 7fa5ff7fe640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 0x7fa6000cbd10 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800c390 tx=0x7fa5f800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:40.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.813+0000 7fa5f7fff640 1 -- 172.21.15.115:0/3859007034 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80052d0 con 0x7fa60015cd80 2023-12-01T01:55:40.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.813+0000 7fa5f7fff640 1 -- 172.21.15.115:0/3859007034 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f8005430 con 0x7fa60015cd80 2023-12-01T01:55:40.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.813+0000 7fa5fffff640 1 -- 172.21.15.115:0/3859007034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6000a9e90 con 0x7fa60015cd80 2023-12-01T01:55:40.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.813+0000 7fa5f7fff640 1 -- 172.21.15.115:0/3859007034 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8003810 con 0x7fa60015cd80 2023-12-01T01:55:40.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.813+0000 7fa5fffff640 1 -- 172.21.15.115:0/3859007034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6000aa450 con 0x7fa60015cd80 2023-12-01T01:55:40.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.815+0000 7fa5fffff640 1 -- 172.21.15.115:0/3859007034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6000aa970 con 0x7fa60015cd80 2023-12-01T01:55:40.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.816+0000 7fa5f7fff640 1 -- 172.21.15.115:0/3859007034 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 76) v1 ==== 88462+0+0 (secure 0 0 0) 0x7fa5f8003970 con 0x7fa60015cd80 2023-12-01T01:55:40.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.817+0000 7fa5f7fff640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e006a320 0x7fa5e006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:55:40.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.817+0000 7fa5f7fff640 1 -- 172.21.15.115:0/3859007034 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fa5f8087c40 con 0x7fa60015cd80 2023-12-01T01:55:40.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.817+0000 7fa5feffd640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e006a320 0x7fa5e006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:55:40.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.818+0000 7fa5feffd640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e006a320 0x7fa5e006c7e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0035000 tx=0x7fa5f002e000 comp rx=0 tx=0).ready entity=mgr.8658 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:55:40.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:40.822+0000 7fa5f7fff640 1 -- 172.21.15.115:0/3859007034 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa5f80533c0 con 0x7fa60015cd80 2023-12-01T01:55:41.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.039+0000 7fa5fffff640 1 -- 172.21.15.115:0/3859007034 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6000d1730 con 0x7fa5e006a320 2023-12-01T01:55:41.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.052+0000 7fa5f7fff640 1 -- 172.21.15.115:0/3859007034 <== mgr.8658 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7fa5f002ca80 con 0x7fa5e006a320 2023-12-01T01:55:41.054 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T01:55:41.054 INFO:teuthology.orchestra.run.smithi115.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-12-01T01:55:41.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 -- 172.21.15.115:0/3859007034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e006a320 msgr2=0x7fa5e006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:41.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e006a320 0x7fa5e006c7e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0035000 tx=0x7fa5f002e000 comp rx=0 tx=0).stop 2023-12-01T01:55:41.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 -- 172.21.15.115:0/3859007034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 msgr2=0x7fa6000cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:55:41.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 0x7fa6000cbd10 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800c390 tx=0x7fa5f800c850 comp rx=0 tx=0).stop 2023-12-01T01:55:41.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 -- 172.21.15.115:0/3859007034 shutdown_connections 2023-12-01T01:55:41.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5e006a320 0x7fa5e006c7e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:41.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000d2470 0x7fa60015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:41.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015cd80 0x7fa6000cbd10 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:41.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 --2- 172.21.15.115:0/3859007034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6000b8400 0x7fa60015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:55:41.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 -- 172.21.15.115:0/3859007034 >> 172.21.15.115:0/3859007034 conn(0x7fa6000d2a70 msgr2=0x7fa6000cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:55:41.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 -- 172.21.15.115:0/3859007034 shutdown_connections 2023-12-01T01:55:41.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T01:55:41.055+0000 7fa5fffff640 1 -- 172.21.15.115:0/3859007034 wait complete. 2023-12-01T01:55:41.076 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-12-01T01:55:41.076 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2023-12-01T01:56:00.487 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:56:00.498 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:56:00.586 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:56:00.585+0000 7fba748bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:56:00.586 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:56:00.585+0000 7fba748bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:56:00.586 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:56:00.585+0000 7fba748bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:56:00.586 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:56:00.585+0000 7fba748bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:56:00.586 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T01:56:00.585+0000 7fba748bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:56:30.649 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:56:30.659 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:57:00.836 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:57:00.846 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:57:30.983 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:57:30.993 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:58:01.138 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:58:01.148 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:58:31.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:58:31.296 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:59:01.475 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:59:01.484 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:59:31.637 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T01:59:31.647 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:00:01.826 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:00:01.842 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:00:32.001 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:00:32.010 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:01:02.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:01:02.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:01:32.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:01:32.324 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:02:02.462 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:02:02.472 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:02:32.629 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:02:32.638 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:03:02.780 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:03:02.790 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:03:32.926 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:03:32.936 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:04:03.372 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:04:03.382 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:04:33.518 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:04:33.528 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:05:03.692 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:05:03.702 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:05:33.847 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:05:33.857 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:06:04.017 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:06:04.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:06:34.174 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:06:34.183 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:07:04.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:07:04.339 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:07:34.484 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:07:34.493 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:08:04.796 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:08:04.806 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:08:34.943 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:08:34.955 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:09:05.136 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:09:05.146 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:09:35.313 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:09:35.322 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:10:05.509 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:10:05.520 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:10:35.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:10:35.705 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:11:05.862 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:11:05.872 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:11:36.042 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:11:36.052 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:12:06.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:12:06.207 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:12:36.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:12:36.353 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:13:06.490 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:13:06.499 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:13:36.646 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:13:36.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:14:06.801 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:14:06.811 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:14:36.964 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:14:36.974 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:15:07.169 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:15:07.181 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:15:37.340 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:15:37.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:16:07.513 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:16:07.523 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:16:37.692 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:16:37.702 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:17:07.839 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:17:07.850 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:17:37.985 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:17:37.995 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:18:08.140 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:18:08.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:18:38.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:18:38.297 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:19:08.475 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:19:08.485 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:19:38.702 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:19:38.712 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:20:08.843 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:20:08.853 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:20:39.082 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:20:39.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:21:09.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:21:09.313 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:21:39.492 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:21:39.502 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:22:09.713 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:22:09.723 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:22:39.877 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:22:39.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:23:10.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:23:10.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:23:40.220 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:23:40.230 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:24:10.409 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:24:10.418 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:24:40.928 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:24:40.937 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:25:11.072 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:25:11.081 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:25:41.298 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:25:41.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:26:11.460 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:26:11.470 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:26:41.624 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:26:41.633 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:27:11.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:27:11.839 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:27:41.979 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:27:41.988 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:28:12.199 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:28:12.209 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:28:42.396 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:28:42.406 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:29:12.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:29:12.575 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:29:42.771 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:29:42.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:30:12.942 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:30:12.953 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:30:43.190 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:30:43.199 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:31:13.428 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:31:13.437 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:31:43.608 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:31:43.617 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:32:13.779 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:32:13.789 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:32:44.120 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:32:44.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:33:14.340 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:33:14.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:33:44.552 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:33:44.562 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:34:14.708 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:34:14.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:34:44.920 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:34:44.933 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:35:15.092 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:35:15.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:35:45.313 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:35:45.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:36:15.511 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:36:15.521 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:36:45.652 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:36:45.662 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:37:15.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:37:15.897 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:37:46.096 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:37:46.105 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:38:16.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:38:16.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:38:46.570 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:38:46.580 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:39:16.767 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:39:16.777 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:39:46.997 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:39:47.008 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:40:17.160 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:40:17.170 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:40:47.408 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:40:47.420 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:41:17.571 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:41:17.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:41:47.812 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:41:47.822 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:42:17.957 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:42:17.967 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:42:48.111 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:42:48.120 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:43:18.325 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:43:18.334 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:43:48.546 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:43:48.556 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:44:18.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:44:18.785 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:44:49.007 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:44:49.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:45:19.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:45:19.157 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:45:49.365 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:45:49.375 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:46:19.586 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:46:19.595 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:46:49.732 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:46:49.742 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:47:19.972 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:47:19.982 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:47:50.122 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:47:50.131 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:48:20.358 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:48:20.368 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:48:50.562 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:48:50.572 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:49:20.800 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:49:20.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:49:51.030 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:49:51.039 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:50:21.244 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:50:21.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:50:51.583 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:50:51.593 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:51:16.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:16.077 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T02:51:16.145 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_57_00_UTC 2023-12-01T02:51:16.145 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_58_00_UTC 2023-12-01T02:51:16.145 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_59_00_UTC 2023-12-01T02:51:16.145 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_00_00_UTC 2023-12-01T02:51:16.145 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_01_00_UTC 2023-12-01T02:51:16.145 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_02_00_UTC 2023-12-01T02:51:16.145 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_03_00_UTC 2023-12-01T02:51:16.145 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_04_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_05_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_06_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_07_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_08_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_09_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_10_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_11_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_12_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_13_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_14_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_15_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_16_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_17_00_UTC 2023-12-01T02:51:16.146 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_18_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_19_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_20_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_21_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_22_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_23_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_24_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_25_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_26_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_27_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_28_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_29_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_30_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_31_00_UTC 2023-12-01T02:51:16.147 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_32_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_33_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_34_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_35_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_36_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_37_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_38_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_39_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_40_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_41_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_42_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_43_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_44_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_45_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_46_00_UTC 2023-12-01T02:51:16.148 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_47_00_UTC 2023-12-01T02:51:16.149 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_48_00_UTC 2023-12-01T02:51:16.149 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_49_00_UTC 2023-12-01T02:51:16.149 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_50_00_UTC 2023-12-01T02:51:16.149 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_51_00_UTC 2023-12-01T02:51:16.150 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-01_57_00_UTC', 'scheduled-2023-12-01-01_58_00_UTC', 'scheduled-2023-12-01-01_59_00_UTC', 'scheduled-2023-12-01-02_00_00_UTC', 'scheduled-2023-12-01-02_01_00_UTC', 'scheduled-2023-12-01-02_02_00_UTC', 'scheduled-2023-12-01-02_03_00_UTC', 'scheduled-2023-12-01-02_04_00_UTC', 'scheduled-2023-12-01-02_05_00_UTC', 'scheduled-2023-12-01-02_06_00_UTC', 'scheduled-2023-12-01-02_07_00_UTC', 'scheduled-2023-12-01-02_08_00_UTC', 'scheduled-2023-12-01-02_09_00_UTC', 'scheduled-2023-12-01-02_10_00_UTC', 'scheduled-2023-12-01-02_11_00_UTC', 'scheduled-2023-12-01-02_12_00_UTC', 'scheduled-2023-12-01-02_13_00_UTC', 'scheduled-2023-12-01-02_14_00_UTC', 'scheduled-2023-12-01-02_15_00_UTC', 'scheduled-2023-12-01-02_16_00_UTC', 'scheduled-2023-12-01-02_17_00_UTC', 'scheduled-2023-12-01-02_18_00_UTC', 'scheduled-2023-12-01-02_19_00_UTC', 'scheduled-2023-12-01-02_20_00_UTC', 'scheduled-2023-12-01-02_21_00_UTC', 'scheduled-2023-12-01-02_22_00_UTC', 'scheduled-2023-12-01-02_23_00_UTC', 'scheduled-2023-12-01-02_24_00_UTC', 'scheduled-2023-12-01-02_25_00_UTC', 'scheduled-2023-12-01-02_26_00_UTC', 'scheduled-2023-12-01-02_27_00_UTC', 'scheduled-2023-12-01-02_28_00_UTC', 'scheduled-2023-12-01-02_29_00_UTC', 'scheduled-2023-12-01-02_30_00_UTC', 'scheduled-2023-12-01-02_31_00_UTC', 'scheduled-2023-12-01-02_32_00_UTC', 'scheduled-2023-12-01-02_33_00_UTC', 'scheduled-2023-12-01-02_34_00_UTC', 'scheduled-2023-12-01-02_35_00_UTC', 'scheduled-2023-12-01-02_36_00_UTC', 'scheduled-2023-12-01-02_37_00_UTC', 'scheduled-2023-12-01-02_38_00_UTC', 'scheduled-2023-12-01-02_39_00_UTC', 'scheduled-2023-12-01-02_40_00_UTC', 'scheduled-2023-12-01-02_41_00_UTC', 'scheduled-2023-12-01-02_42_00_UTC', 'scheduled-2023-12-01-02_43_00_UTC', 'scheduled-2023-12-01-02_44_00_UTC', 'scheduled-2023-12-01-02_45_00_UTC', 'scheduled-2023-12-01-02_46_00_UTC', 'scheduled-2023-12-01-02_47_00_UTC', 'scheduled-2023-12-01-02_48_00_UTC', 'scheduled-2023-12-01-02_49_00_UTC', 'scheduled-2023-12-01-02_50_00_UTC', 'scheduled-2023-12-01-02_51_00_UTC'] 2023-12-01T02:51:16.150 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1 2023-12-01T02:51:16.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.362+0000 7f116359e640 1 Processor -- start 2023-12-01T02:51:16.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.362+0000 7f116359e640 1 -- start start 2023-12-01T02:51:16.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.364+0000 7f116359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f116415a760 0x7f1164160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.364+0000 7f116359e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11640cb150 0x7f11640cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.364+0000 7f116359e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116415d790 con 0x7f11640b2710 2023-12-01T02:51:16.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.364+0000 7f116359e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11640cbb00 con 0x7f116415a760 2023-12-01T02:51:16.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.364+0000 7f116359e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1164160e40 con 0x7f11640cb150 2023-12-01T02:51:16.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.364+0000 7f116259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f116415a760 0x7f1164160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:16.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.364+0000 7f116259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f116415a760 0x7f1164160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42636/0 (socket says 172.21.15.115:42636) 2023-12-01T02:51:16.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.364+0000 7f116259c640 1 -- 172.21.15.115:0/2238489862 learned_addr learned my addr 172.21.15.115:0/2238489862 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:16.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.364+0000 7f1159d9b640 1 --1- 172.21.15.115:0/2238489862 >> v1:172.21.15.115:6789/0 conn(0x7f11640b2710 0x7f116415dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41676/0 (socket says 172.21.15.115:41676) 2023-12-01T02:51:16.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.365+0000 7f116259c640 1 -- 172.21.15.115:0/2238489862 >> v2:172.21.15.142:3300/0 conn(0x7f11640cb150 msgr2=0x7f11640cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:16.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.365+0000 7f116259c640 1 --2- 172.21.15.115:0/2238489862 >> v2:172.21.15.142:3300/0 conn(0x7f11640cb150 0x7f11640cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.365+0000 7f116259c640 1 -- 172.21.15.115:0/2238489862 >> v1:172.21.15.115:6789/0 conn(0x7f11640b2710 legacy=0x7f116415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:16.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.365+0000 7f116259c640 1 -- 172.21.15.115:0/2238489862 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1164161110 con 0x7f116415a760 2023-12-01T02:51:16.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.366+0000 7f116259c640 1 --2- 172.21.15.115:0/2238489862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 0x7f1164160900 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f11640b16b0 tx=0x7f115402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1accd1448e788de server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:16.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.366+0000 7f1161d9b640 1 -- 172.21.15.115:0/2238489862 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154031890 con 0x7f116415a760 2023-12-01T02:51:16.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.366+0000 7f1161d9b640 1 -- 172.21.15.115:0/2238489862 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11540319f0 con 0x7f116415a760 2023-12-01T02:51:16.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.366+0000 7f1161d9b640 1 -- 172.21.15.115:0/2238489862 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154031d80 con 0x7f116415a760 2023-12-01T02:51:16.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.367+0000 7f116359e640 1 -- 172.21.15.115:0/2238489862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 msgr2=0x7f1164160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:16.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.367+0000 7f116359e640 1 --2- 172.21.15.115:0/2238489862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 0x7f1164160900 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f11640b16b0 tx=0x7f115402b930 comp rx=0 tx=0).stop 2023-12-01T02:51:16.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.367+0000 7f116359e640 1 -- 172.21.15.115:0/2238489862 shutdown_connections 2023-12-01T02:51:16.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.367+0000 7f116359e640 1 --2- 172.21.15.115:0/2238489862 >> v2:172.21.15.142:3300/0 conn(0x7f11640cb150 0x7f11640cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.367+0000 7f116359e640 1 --2- 172.21.15.115:0/2238489862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 0x7f1164160900 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.367+0000 7f116359e640 1 -- 172.21.15.115:0/2238489862 >> 172.21.15.115:0/2238489862 conn(0x7f1164065960 msgr2=0x7f1164065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:16.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.367+0000 7f116359e640 1 -- 172.21.15.115:0/2238489862 shutdown_connections 2023-12-01T02:51:16.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.367+0000 7f116359e640 1 -- 172.21.15.115:0/2238489862 wait complete. 2023-12-01T02:51:16.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.369+0000 7f116359e640 1 Processor -- start 2023-12-01T02:51:16.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.369+0000 7f116359e640 1 -- start start 2023-12-01T02:51:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.370+0000 7f116359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11640b2710 0x7f116414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.370+0000 7f116359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11640cb150 0x7f116414e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.370+0000 7f116359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 0x7f116414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.370+0000 7f116359e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1164071be0 con 0x7f116415a760 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.370+0000 7f116359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116414e760 con 0x7f11640b2710 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.370+0000 7f116359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11640ce340 con 0x7f11640cb150 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.370+0000 7f1162d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 0x7f116414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.370+0000 7f1162d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 0x7f116414eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42652/0 (socket says 172.21.15.115:42652) 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.370+0000 7f1162d9d640 1 -- 172.21.15.115:0/3926353780 learned_addr learned my addr 172.21.15.115:0/3926353780 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f116259c640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11640b2710 0x7f116414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f1162d9d640 1 -- 172.21.15.115:0/3926353780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11640cb150 msgr2=0x7f116414e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f1159d9b640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11640cb150 0x7f116414e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f1162d9d640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11640cb150 0x7f116414e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f1162d9d640 1 -- 172.21.15.115:0/3926353780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11640b2710 msgr2=0x7f116414e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f1162d9d640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11640b2710 0x7f116414e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f1162d9d640 1 -- 172.21.15.115:0/3926353780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11540094d0 con 0x7f116415a760 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f116259c640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11640b2710 0x7f116414e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:16.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f1162d9d640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 0x7f116414eea0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f115c00b970 tx=0x7f115c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:16.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f115b7fe640 1 -- 172.21.15.115:0/3926353780 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c00cb40 con 0x7f116415a760 2023-12-01T02:51:16.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f115b7fe640 1 -- 172.21.15.115:0/3926353780 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c00ccc0 con 0x7f116415a760 2023-12-01T02:51:16.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f115b7fe640 1 -- 172.21.15.115:0/3926353780 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c014680 con 0x7f116415a760 2023-12-01T02:51:16.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.371+0000 7f116359e640 1 -- 172.21.15.115:0/3926353780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1164071100 con 0x7f116415a760 2023-12-01T02:51:16.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.372+0000 7f116359e640 1 -- 172.21.15.115:0/3926353780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11640716c0 con 0x7f116415a760 2023-12-01T02:51:16.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.373+0000 7f116359e640 1 -- 172.21.15.115:0/3926353780 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11640618d0 con 0x7f116415a760 2023-12-01T02:51:16.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.374+0000 7f115b7fe640 1 -- 172.21.15.115:0/3926353780 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 80) v1 ==== 88738+0+0 (secure 0 0 0) 0x7f115c01d3b0 con 0x7f116415a760 2023-12-01T02:51:16.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.374+0000 7f115b7fe640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114406a5a0 0x7f114406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.375+0000 7f115b7fe640 1 -- 172.21.15.115:0/3926353780 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6199+0+0 (secure 0 0 0) 0x7f115c088bc0 con 0x7f116415a760 2023-12-01T02:51:16.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.375+0000 7f116259c640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114406a5a0 0x7f114406ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:16.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.375+0000 7f116259c640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114406a5a0 0x7f114406ca60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1154036000 tx=0x7f115403c040 comp rx=0 tx=0).ready entity=mgr.8658 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:16.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.379+0000 7f115b7fe640 1 -- 172.21.15.115:0/3926353780 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f115c053f40 con 0x7f116415a760 2023-12-01T02:51:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.592+0000 7f116359e640 1 -- 172.21.15.115:0/3926353780 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f116415d420 con 0x7f114406a5a0 2023-12-01T02:51:16.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.607+0000 7f115b7fe640 1 -- 172.21.15.115:0/3926353780 <== mgr.8658 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f1154031ce0 con 0x7f114406a5a0 2023-12-01T02:51:16.609 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:16.609 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:55:39", "first": "2023-12-01T01:56:00", "last": "2023-12-01T02:51:00", "last_pruned": "2023-12-01T02:51:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 -- 172.21.15.115:0/3926353780 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114406a5a0 msgr2=0x7f114406ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114406a5a0 0x7f114406ca60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1154036000 tx=0x7f115403c040 comp rx=0 tx=0).stop 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 -- 172.21.15.115:0/3926353780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 msgr2=0x7f116414eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 0x7f116414eea0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f115c00b970 tx=0x7f115c00be30 comp rx=0 tx=0).stop 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 -- 172.21.15.115:0/3926353780 shutdown_connections 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114406a5a0 0x7f114406ca60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11640cb150 0x7f116414e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f116415a760 0x7f116414eea0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 --2- 172.21.15.115:0/3926353780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11640b2710 0x7f116414e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 -- 172.21.15.115:0/3926353780 >> 172.21.15.115:0/3926353780 conn(0x7f1164065960 msgr2=0x7f1164157ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 -- 172.21.15.115:0/3926353780 shutdown_connections 2023-12-01T02:51:16.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.610+0000 7f116359e640 1 -- 172.21.15.115:0/3926353780 wait complete. 2023-12-01T02:51:16.631 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T01:55:39", "first": "2023-12-01T01:56:00", "last": "2023-12-01T02:51:00", "last_pruned": "2023-12-01T02:51:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-12-01T02:51:16.631 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T01:55:39', 'first': '2023-12-01T01:56:00', 'last': '2023-12-01T02:51:00', 'last_pruned': '2023-12-01T02:51:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-12-01T02:51:16.631 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-12-01T02:51:16.631 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-12-01T02:51:16.631 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1 2023-12-01T02:51:16.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.844+0000 7eff4c604640 1 Processor -- start 2023-12-01T02:51:16.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.844+0000 7eff4c604640 1 -- start start 2023-12-01T02:51:16.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.845+0000 7eff4c604640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7eff440b5cd0 0x7eff440b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.845+0000 7eff4c604640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff440b6890 con 0x7eff4415cfa0 2023-12-01T02:51:16.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.845+0000 7eff4c604640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff440b6b70 con 0x7eff440cfc10 2023-12-01T02:51:16.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.845+0000 7eff4c604640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff440b66a0 con 0x7eff440b5cd0 2023-12-01T02:51:16.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.846+0000 7eff4a3a0640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7eff4415cfa0 0x7eff440b7930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:41688/0 (socket says 172.21.15.115:41688) 2023-12-01T02:51:16.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.846+0000 7eff4a3a0640 1 -- 172.21.15.115:0/2843310432 learned_addr learned my addr 172.21.15.115:0/2843310432 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:16.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.846+0000 7eff49b9f640 1 --2- 172.21.15.115:0/2843310432 >> v2:172.21.15.115:3300/0 conn(0x7eff440b5cd0 0x7eff440b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:16.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.846+0000 7eff4939e640 1 -- 172.21.15.115:0/2843310432 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165073275 0 0) 0x7eff440b6890 con 0x7eff4415cfa0 2023-12-01T02:51:16.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.846+0000 7eff4939e640 1 -- 172.21.15.115:0/2843310432 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff30003520 con 0x7eff4415cfa0 2023-12-01T02:51:16.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.846+0000 7eff49b9f640 1 -- 172.21.15.115:0/2843310432 >> v1:172.21.15.142:6789/0 conn(0x7eff440cfc10 legacy=0x7eff440d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:16.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.846+0000 7eff49b9f640 1 -- 172.21.15.115:0/2843310432 >> v1:172.21.15.115:6789/0 conn(0x7eff4415cfa0 legacy=0x7eff440b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:16.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.846+0000 7eff49b9f640 1 -- 172.21.15.115:0/2843310432 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff440b6e50 con 0x7eff440b5cd0 2023-12-01T02:51:16.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.847+0000 7eff49b9f640 1 --2- 172.21.15.115:0/2843310432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440b5cd0 0x7eff440b6160 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7eff340097a0 tx=0x7eff3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78b49e7b5178c77 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:16.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.848+0000 7eff4939e640 1 -- 172.21.15.115:0/2843310432 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3402e070 con 0x7eff440b5cd0 2023-12-01T02:51:16.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.848+0000 7eff4939e640 1 -- 172.21.15.115:0/2843310432 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff340315a0 con 0x7eff440b5cd0 2023-12-01T02:51:16.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.848+0000 7eff4939e640 1 -- 172.21.15.115:0/2843310432 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34031930 con 0x7eff440b5cd0 2023-12-01T02:51:16.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.848+0000 7eff4c604640 1 -- 172.21.15.115:0/2843310432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440b5cd0 msgr2=0x7eff440b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:16.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.848+0000 7eff4c604640 1 --2- 172.21.15.115:0/2843310432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440b5cd0 0x7eff440b6160 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7eff340097a0 tx=0x7eff3402b840 comp rx=0 tx=0).stop 2023-12-01T02:51:16.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.848+0000 7eff4c604640 1 -- 172.21.15.115:0/2843310432 shutdown_connections 2023-12-01T02:51:16.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.848+0000 7eff4c604640 1 --2- 172.21.15.115:0/2843310432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440b5cd0 0x7eff440b6160 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.848+0000 7eff4c604640 1 -- 172.21.15.115:0/2843310432 >> 172.21.15.115:0/2843310432 conn(0x7eff4406a0b0 msgr2=0x7eff4406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:16.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.848+0000 7eff4c604640 1 -- 172.21.15.115:0/2843310432 shutdown_connections 2023-12-01T02:51:16.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.848+0000 7eff4c604640 1 -- 172.21.15.115:0/2843310432 wait complete. 2023-12-01T02:51:16.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.850+0000 7eff4c604640 1 Processor -- start 2023-12-01T02:51:16.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.850+0000 7eff4c604640 1 -- start start 2023-12-01T02:51:16.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.851+0000 7eff4c604640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff440b5cd0 0x7eff4414e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.851+0000 7eff4c604640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440cfc10 0x7eff4414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.851+0000 7eff4c604640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff4415cfa0 0x7eff441583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.851+0000 7eff4c604640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff440d2570 con 0x7eff440cfc10 2023-12-01T02:51:16.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.851+0000 7eff4c604640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff44141310 con 0x7eff4415cfa0 2023-12-01T02:51:16.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.851+0000 7eff4c604640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff44143740 con 0x7eff440b5cd0 2023-12-01T02:51:16.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.851+0000 7eff49b9f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440cfc10 0x7eff4414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:16.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.851+0000 7eff49b9f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440cfc10 0x7eff4414ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42660/0 (socket says 172.21.15.115:42660) 2023-12-01T02:51:16.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.851+0000 7eff49b9f640 1 -- 172.21.15.115:0/2365242394 learned_addr learned my addr 172.21.15.115:0/2365242394 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.852+0000 7eff4a3a0640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff440b5cd0 0x7eff4414e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.852+0000 7eff49b9f640 1 -- 172.21.15.115:0/2365242394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff440b5cd0 msgr2=0x7eff4414e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.852+0000 7eff4aba1640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff4415cfa0 0x7eff441583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.852+0000 7eff49b9f640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff440b5cd0 0x7eff4414e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.852+0000 7eff49b9f640 1 -- 172.21.15.115:0/2365242394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff4415cfa0 msgr2=0x7eff441583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.852+0000 7eff49b9f640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff4415cfa0 0x7eff441583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.852+0000 7eff49b9f640 1 -- 172.21.15.115:0/2365242394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff34009400 con 0x7eff440cfc10 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.852+0000 7eff4a3a0640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff440b5cd0 0x7eff4414e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.852+0000 7eff4aba1640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff4415cfa0 0x7eff441583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.852+0000 7eff49b9f640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440cfc10 0x7eff4414ee10 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7eff34034040 tx=0x7eff340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.853+0000 7eff3affd640 1 -- 172.21.15.115:0/2365242394 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34032840 con 0x7eff440cfc10 2023-12-01T02:51:16.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.853+0000 7eff3affd640 1 -- 172.21.15.115:0/2365242394 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff340329c0 con 0x7eff440cfc10 2023-12-01T02:51:16.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.853+0000 7eff3affd640 1 -- 172.21.15.115:0/2365242394 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34032d70 con 0x7eff440cfc10 2023-12-01T02:51:16.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.853+0000 7eff4c604640 1 -- 172.21.15.115:0/2365242394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff44140970 con 0x7eff440cfc10 2023-12-01T02:51:16.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.853+0000 7eff4c604640 1 -- 172.21.15.115:0/2365242394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff44140f30 con 0x7eff440cfc10 2023-12-01T02:51:16.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.854+0000 7eff4c604640 1 -- 172.21.15.115:0/2365242394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff44141450 con 0x7eff440cfc10 2023-12-01T02:51:16.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.855+0000 7eff3affd640 1 -- 172.21.15.115:0/2365242394 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 80) v1 ==== 88738+0+0 (secure 0 0 0) 0x7eff3404d3f0 con 0x7eff440cfc10 2023-12-01T02:51:16.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.856+0000 7eff3affd640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff2806a5d0 0x7eff2806ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:16.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.856+0000 7eff3affd640 1 -- 172.21.15.115:0/2365242394 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6199+0+0 (secure 0 0 0) 0x7eff340aa920 con 0x7eff440cfc10 2023-12-01T02:51:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.856+0000 7eff4a3a0640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff2806a5d0 0x7eff2806ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:16.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.857+0000 7eff4a3a0640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff2806a5d0 0x7eff2806ca90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff400011e0 tx=0x7eff40009040 comp rx=0 tx=0).ready entity=mgr.8658 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:16.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:16.861+0000 7eff3affd640 1 -- 172.21.15.115:0/2365242394 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7eff44141450 con 0x7eff440cfc10 2023-12-01T02:51:17.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.084+0000 7eff4c604640 1 -- 172.21.15.115:0/2365242394 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7eff440b7410 con 0x7eff2806a5d0 2023-12-01T02:51:17.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.257+0000 7eff3affd640 1 -- 172.21.15.115:0/2365242394 <== mgr.8658 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7eff400013f0 con 0x7eff2806a5d0 2023-12-01T02:51:17.259 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1 2023-12-01T02:51:17.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 -- 172.21.15.115:0/2365242394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff2806a5d0 msgr2=0x7eff2806ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:17.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff2806a5d0 0x7eff2806ca90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff400011e0 tx=0x7eff40009040 comp rx=0 tx=0).stop 2023-12-01T02:51:17.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 -- 172.21.15.115:0/2365242394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440cfc10 msgr2=0x7eff4414ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:17.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440cfc10 0x7eff4414ee10 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7eff34034040 tx=0x7eff340046c0 comp rx=0 tx=0).stop 2023-12-01T02:51:17.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 -- 172.21.15.115:0/2365242394 shutdown_connections 2023-12-01T02:51:17.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7eff2806a5d0 0x7eff2806ca90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:17.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff4415cfa0 0x7eff441583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:17.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff440cfc10 0x7eff4414ee10 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:17.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 --2- 172.21.15.115:0/2365242394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff440b5cd0 0x7eff4414e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:17.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 -- 172.21.15.115:0/2365242394 >> 172.21.15.115:0/2365242394 conn(0x7eff4406a0b0 msgr2=0x7eff44152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:17.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 -- 172.21.15.115:0/2365242394 shutdown_connections 2023-12-01T02:51:17.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:17.261+0000 7eff4c604640 1 -- 172.21.15.115:0/2365242394 wait complete. 2023-12-01T02:51:17.282 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-12-01T02:51:17.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.282 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T02:51:17.339 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_57_00_UTC 2023-12-01T02:51:17.339 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_58_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-01_59_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_00_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_01_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_02_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_03_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_04_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_05_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_06_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_07_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_08_00_UTC 2023-12-01T02:51:17.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_09_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_10_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_11_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_12_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_13_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_14_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_15_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_16_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_17_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_18_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_19_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_20_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_21_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_22_00_UTC 2023-12-01T02:51:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_23_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_24_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_25_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_26_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_27_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_28_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_29_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_30_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_31_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_32_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_33_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_34_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_35_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_36_00_UTC 2023-12-01T02:51:17.342 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_37_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_38_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_39_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_40_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_41_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_42_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_43_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_44_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_45_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_46_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_47_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_48_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_49_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_50_00_UTC 2023-12-01T02:51:17.343 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_51_00_UTC 2023-12-01T02:51:17.344 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-01_57_00_UTC 2023-12-01T02:51:17.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.345 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-01_57_00_UTC) 2023-12-01T02:51:17.413 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-01_58_00_UTC 2023-12-01T02:51:17.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.413 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-01_58_00_UTC) 2023-12-01T02:51:17.488 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-01_59_00_UTC 2023-12-01T02:51:17.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.489 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-01_59_00_UTC) 2023-12-01T02:51:17.557 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_00_00_UTC 2023-12-01T02:51:17.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.557 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_00_00_UTC) 2023-12-01T02:51:17.627 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_01_00_UTC 2023-12-01T02:51:17.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.627 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_01_00_UTC) 2023-12-01T02:51:17.696 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_02_00_UTC 2023-12-01T02:51:17.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.696 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_02_00_UTC) 2023-12-01T02:51:17.767 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_03_00_UTC 2023-12-01T02:51:17.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.768 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_03_00_UTC) 2023-12-01T02:51:17.838 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_04_00_UTC 2023-12-01T02:51:17.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.839 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_04_00_UTC) 2023-12-01T02:51:17.912 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_05_00_UTC 2023-12-01T02:51:17.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.913 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_05_00_UTC) 2023-12-01T02:51:17.984 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_06_00_UTC 2023-12-01T02:51:17.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:17.985 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_06_00_UTC) 2023-12-01T02:51:18.052 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_07_00_UTC 2023-12-01T02:51:18.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.053 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_07_00_UTC) 2023-12-01T02:51:18.121 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_08_00_UTC 2023-12-01T02:51:18.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.121 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_08_00_UTC) 2023-12-01T02:51:18.185 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_09_00_UTC 2023-12-01T02:51:18.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.185 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_09_00_UTC) 2023-12-01T02:51:18.258 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_10_00_UTC 2023-12-01T02:51:18.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.258 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_10_00_UTC) 2023-12-01T02:51:18.330 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_11_00_UTC 2023-12-01T02:51:18.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.331 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_11_00_UTC) 2023-12-01T02:51:18.401 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_12_00_UTC 2023-12-01T02:51:18.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.401 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_12_00_UTC) 2023-12-01T02:51:18.472 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_13_00_UTC 2023-12-01T02:51:18.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.472 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_13_00_UTC) 2023-12-01T02:51:18.545 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_14_00_UTC 2023-12-01T02:51:18.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.545 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_14_00_UTC) 2023-12-01T02:51:18.614 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_15_00_UTC 2023-12-01T02:51:18.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.615 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_15_00_UTC) 2023-12-01T02:51:18.683 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_16_00_UTC 2023-12-01T02:51:18.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.684 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_16_00_UTC) 2023-12-01T02:51:18.751 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_17_00_UTC 2023-12-01T02:51:18.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.752 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_17_00_UTC) 2023-12-01T02:51:18.825 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_18_00_UTC 2023-12-01T02:51:18.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.825 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_18_00_UTC) 2023-12-01T02:51:18.897 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_19_00_UTC 2023-12-01T02:51:18.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.897 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_19_00_UTC) 2023-12-01T02:51:18.966 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_20_00_UTC 2023-12-01T02:51:18.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:18.967 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_20_00_UTC) 2023-12-01T02:51:19.034 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_21_00_UTC 2023-12-01T02:51:19.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.034 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_21_00_UTC) 2023-12-01T02:51:19.106 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_22_00_UTC 2023-12-01T02:51:19.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.107 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_22_00_UTC) 2023-12-01T02:51:19.180 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_23_00_UTC 2023-12-01T02:51:19.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.181 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_23_00_UTC) 2023-12-01T02:51:19.250 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_24_00_UTC 2023-12-01T02:51:19.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.251 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_24_00_UTC) 2023-12-01T02:51:19.318 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_25_00_UTC 2023-12-01T02:51:19.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.319 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_25_00_UTC) 2023-12-01T02:51:19.389 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_26_00_UTC 2023-12-01T02:51:19.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.389 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_26_00_UTC) 2023-12-01T02:51:19.458 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_27_00_UTC 2023-12-01T02:51:19.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.459 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_27_00_UTC) 2023-12-01T02:51:19.525 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_28_00_UTC 2023-12-01T02:51:19.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.526 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_28_00_UTC) 2023-12-01T02:51:19.595 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_29_00_UTC 2023-12-01T02:51:19.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.595 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_29_00_UTC) 2023-12-01T02:51:19.662 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_30_00_UTC 2023-12-01T02:51:19.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.663 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_30_00_UTC) 2023-12-01T02:51:19.728 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_31_00_UTC 2023-12-01T02:51:19.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.728 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_31_00_UTC) 2023-12-01T02:51:19.796 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_32_00_UTC 2023-12-01T02:51:19.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.797 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_32_00_UTC) 2023-12-01T02:51:19.866 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_33_00_UTC 2023-12-01T02:51:19.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.866 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_33_00_UTC) 2023-12-01T02:51:19.936 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_34_00_UTC 2023-12-01T02:51:19.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:19.936 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_34_00_UTC) 2023-12-01T02:51:20.005 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_35_00_UTC 2023-12-01T02:51:20.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.005 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_35_00_UTC) 2023-12-01T02:51:20.072 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_36_00_UTC 2023-12-01T02:51:20.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.072 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_36_00_UTC) 2023-12-01T02:51:20.140 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_37_00_UTC 2023-12-01T02:51:20.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.141 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_37_00_UTC) 2023-12-01T02:51:20.236 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_38_00_UTC 2023-12-01T02:51:20.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.236 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_38_00_UTC) 2023-12-01T02:51:20.331 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_39_00_UTC 2023-12-01T02:51:20.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.331 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_39_00_UTC) 2023-12-01T02:51:20.402 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_40_00_UTC 2023-12-01T02:51:20.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.402 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_40_00_UTC) 2023-12-01T02:51:20.451 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_41_00_UTC 2023-12-01T02:51:20.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.451 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_41_00_UTC) 2023-12-01T02:51:20.497 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_42_00_UTC 2023-12-01T02:51:20.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.497 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_42_00_UTC) 2023-12-01T02:51:20.601 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_43_00_UTC 2023-12-01T02:51:20.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.601 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_43_00_UTC) 2023-12-01T02:51:20.661 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_44_00_UTC 2023-12-01T02:51:20.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.661 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_44_00_UTC) 2023-12-01T02:51:20.722 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_45_00_UTC 2023-12-01T02:51:20.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.722 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_45_00_UTC) 2023-12-01T02:51:20.810 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_46_00_UTC 2023-12-01T02:51:20.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.810 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_46_00_UTC) 2023-12-01T02:51:20.877 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_47_00_UTC 2023-12-01T02:51:20.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.877 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_47_00_UTC) 2023-12-01T02:51:20.925 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_48_00_UTC 2023-12-01T02:51:20.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.925 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_48_00_UTC) 2023-12-01T02:51:20.973 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_49_00_UTC 2023-12-01T02:51:20.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:20.973 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_49_00_UTC) 2023-12-01T02:51:21.027 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_50_00_UTC 2023-12-01T02:51:21.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:21.027 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_50_00_UTC) 2023-12-01T02:51:21.087 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-02_51_00_UTC 2023-12-01T02:51:21.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:21.087 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-02_51_00_UTC) 2023-12-01T02:51:21.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:21.146 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-01T02:51:21.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T02:51:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.419+0000 7fa201988640 1 Processor -- start 2023-12-01T02:51:21.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.419+0000 7fa201988640 1 -- start start 2023-12-01T02:51:21.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.420+0000 7fa201988640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa1fc15cb40 0x7fa1fc0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:21.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.420+0000 7fa201988640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc0cde50 0x7fa1fc06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:21.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.420+0000 7fa201988640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc0d1e00 con 0x7fa1fc0b8400 2023-12-01T02:51:21.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.420+0000 7fa201988640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc0cd250 con 0x7fa1fc15cb40 2023-12-01T02:51:21.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.420+0000 7fa201988640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc069cc0 con 0x7fa1fc0cde50 2023-12-01T02:51:21.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.421+0000 7fa200986640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa1fc15cb40 0x7fa1fc0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:21.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.421+0000 7fa1f3fff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fa1fc0b8400 0x7fa1fc0cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:57390/0 (socket says 172.21.15.115:57390) 2023-12-01T02:51:21.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.421+0000 7fa200986640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa1fc15cb40 0x7fa1fc0d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:46140/0 (socket says 172.21.15.115:46140) 2023-12-01T02:51:21.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.421+0000 7fa1f3fff640 1 -- 172.21.15.115:0/1806985314 learned_addr learned my addr 172.21.15.115:0/1806985314 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:21.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.421+0000 7fa201187640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc0cde50 0x7fa1fc06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:21.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.422+0000 7fa201187640 1 -- 172.21.15.115:0/1806985314 >> v2:172.21.15.142:3300/0 conn(0x7fa1fc15cb40 msgr2=0x7fa1fc0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:21.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.422+0000 7fa201187640 1 --2- 172.21.15.115:0/1806985314 >> v2:172.21.15.142:3300/0 conn(0x7fa1fc15cb40 0x7fa1fc0d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:21.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.422+0000 7fa201187640 1 -- 172.21.15.115:0/1806985314 >> v1:172.21.15.142:6789/0 conn(0x7fa1fc0b8400 legacy=0x7fa1fc0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:21.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.422+0000 7fa201187640 1 -- 172.21.15.115:0/1806985314 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc069f60 con 0x7fa1fc0cde50 2023-12-01T02:51:21.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.422+0000 7fa200986640 1 --2- 172.21.15.115:0/1806985314 >> v2:172.21.15.142:3300/0 conn(0x7fa1fc15cb40 0x7fa1fc0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T02:51:21.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.423+0000 7fa201187640 1 --2- 172.21.15.115:0/1806985314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc0cde50 0x7fa1fc06a3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40097a0 tx=0x7fa1f402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d54e8053d45f7f92 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:21.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.424+0000 7fa1f37fe640 1 -- 172.21.15.115:0/1806985314 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4031830 con 0x7fa1fc0cde50 2023-12-01T02:51:21.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.424+0000 7fa1f37fe640 1 -- 172.21.15.115:0/1806985314 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f4031990 con 0x7fa1fc0cde50 2023-12-01T02:51:21.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.424+0000 7fa1f37fe640 1 -- 172.21.15.115:0/1806985314 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4031d20 con 0x7fa1fc0cde50 2023-12-01T02:51:21.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.424+0000 7fa201988640 1 -- 172.21.15.115:0/1806985314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc0cde50 msgr2=0x7fa1fc06a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:21.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.424+0000 7fa201988640 1 --2- 172.21.15.115:0/1806985314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc0cde50 0x7fa1fc06a3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40097a0 tx=0x7fa1f402b8d0 comp rx=0 tx=0).stop 2023-12-01T02:51:21.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.425+0000 7fa201988640 1 -- 172.21.15.115:0/1806985314 shutdown_connections 2023-12-01T02:51:21.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.425+0000 7fa201988640 1 --2- 172.21.15.115:0/1806985314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc0cde50 0x7fa1fc06a3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:21.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.425+0000 7fa201988640 1 --2- 172.21.15.115:0/1806985314 >> v2:172.21.15.142:3300/0 conn(0x7fa1fc15cb40 0x7fa1fc0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:21.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.425+0000 7fa201988640 1 -- 172.21.15.115:0/1806985314 >> 172.21.15.115:0/1806985314 conn(0x7fa1fc0d2a70 msgr2=0x7fa1fc0d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:21.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.425+0000 7fa201988640 1 -- 172.21.15.115:0/1806985314 shutdown_connections 2023-12-01T02:51:21.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.425+0000 7fa201988640 1 -- 172.21.15.115:0/1806985314 wait complete. 2023-12-01T02:51:21.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.426+0000 7fa201988640 1 Processor -- start 2023-12-01T02:51:21.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.427+0000 7fa201988640 1 -- start start 2023-12-01T02:51:21.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.427+0000 7fa201988640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1fc0b8400 0x7fa1fc15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:21.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.427+0000 7fa201988640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1fc0cde50 0x7fa1fc150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:21.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.427+0000 7fa201988640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc15cb40 0x7fa1fc154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:21.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.427+0000 7fa201988640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc153b00 con 0x7fa1fc0b8400 2023-12-01T02:51:21.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.427+0000 7fa201988640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1526b0 con 0x7fa1fc0cde50 2023-12-01T02:51:21.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.427+0000 7fa201988640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc15c1e0 con 0x7fa1fc15cb40 2023-12-01T02:51:21.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa200986640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1fc0b8400 0x7fa1fc15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:21.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa200986640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1fc0b8400 0x7fa1fc15bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41816/0 (socket says 172.21.15.115:41816) 2023-12-01T02:51:21.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa200986640 1 -- 172.21.15.115:0/257370163 learned_addr learned my addr 172.21.15.115:0/257370163 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:21.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa1f3fff640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1fc0cde50 0x7fa1fc150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:21.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa201187640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc15cb40 0x7fa1fc154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:21.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa200986640 1 -- 172.21.15.115:0/257370163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc15cb40 msgr2=0x7fa1fc154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:21.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa200986640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc15cb40 0x7fa1fc154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:21.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa200986640 1 -- 172.21.15.115:0/257370163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1fc0cde50 msgr2=0x7fa1fc150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:21.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa200986640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1fc0cde50 0x7fa1fc150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:21.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa200986640 1 -- 172.21.15.115:0/257370163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f8009400 con 0x7fa1fc0b8400 2023-12-01T02:51:21.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.428+0000 7fa200986640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1fc0b8400 0x7fa1fc15bca0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800e990 tx=0x7fa1f800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:21.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.429+0000 7fa1f17fa640 1 -- 172.21.15.115:0/257370163 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800c9e0 con 0x7fa1fc0b8400 2023-12-01T02:51:21.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.429+0000 7fa1f17fa640 1 -- 172.21.15.115:0/257370163 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f800cb40 con 0x7fa1fc0b8400 2023-12-01T02:51:21.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.429+0000 7fa201988640 1 -- 172.21.15.115:0/257370163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f4009400 con 0x7fa1fc0b8400 2023-12-01T02:51:21.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.429+0000 7fa1f17fa640 1 -- 172.21.15.115:0/257370163 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800ced0 con 0x7fa1fc0b8400 2023-12-01T02:51:21.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.429+0000 7fa201988640 1 -- 172.21.15.115:0/257370163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc153020 con 0x7fa1fc0b8400 2023-12-01T02:51:21.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.430+0000 7fa201988640 1 -- 172.21.15.115:0/257370163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1fc0618d0 con 0x7fa1fc0b8400 2023-12-01T02:51:21.432 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.431+0000 7fa1f17fa640 1 -- 172.21.15.115:0/257370163 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 80) v1 ==== 88738+0+0 (secure 0 0 0) 0x7fa1f8013470 con 0x7fa1fc0b8400 2023-12-01T02:51:21.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.432+0000 7fa1f17fa640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1e806a5d0 0x7fa1e806ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:21.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.432+0000 7fa1f17fa640 1 -- 172.21.15.115:0/257370163 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6199+0+0 (secure 0 0 0) 0x7fa1f8088f20 con 0x7fa1fc0b8400 2023-12-01T02:51:21.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.432+0000 7fa1f3fff640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1e806a5d0 0x7fa1e806ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:21.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.433+0000 7fa1f3fff640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1e806a5d0 0x7fa1e806ca90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc1f5700 tx=0x7fa1ec009040 comp rx=0 tx=0).ready entity=mgr.8658 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:21.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.437+0000 7fa1f17fa640 1 -- 172.21.15.115:0/257370163 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa1f8054220 con 0x7fa1fc0b8400 2023-12-01T02:51:21.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:21.702+0000 7fa201988640 1 -- 172.21.15.115:0/257370163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa1fc0d0780 con 0x7fa1fc0b8400 2023-12-01T02:51:21.729 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:51:21.737 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:51:22.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.202+0000 7fa1f17fa640 1 -- 172.21.15.115:0/257370163 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 81) v1 ==== 88721+0+0 (secure 0 0 0) 0x7fa1f808c030 con 0x7fa1fc0b8400 2023-12-01T02:51:22.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.204+0000 7fa1f17fa640 1 -- 172.21.15.115:0/257370163 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v81) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa1f8086d00 con 0x7fa1fc0b8400 2023-12-01T02:51:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.208+0000 7fa201988640 1 -- 172.21.15.115:0/257370163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1e806a5d0 msgr2=0x7fa1e806ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.208+0000 7fa201988640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1e806a5d0 0x7fa1e806ca90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc1f5700 tx=0x7fa1ec009040 comp rx=0 tx=0).stop 2023-12-01T02:51:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.208+0000 7fa201988640 1 -- 172.21.15.115:0/257370163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1fc0b8400 msgr2=0x7fa1fc15bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.208+0000 7fa201988640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1fc0b8400 0x7fa1fc15bca0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800e990 tx=0x7fa1f800ee50 comp rx=0 tx=0).stop 2023-12-01T02:51:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.208+0000 7fa201988640 1 -- 172.21.15.115:0/257370163 shutdown_connections 2023-12-01T02:51:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.208+0000 7fa201988640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa1e806a5d0 0x7fa1e806ca90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.208+0000 7fa201988640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1fc15cb40 0x7fa1fc154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.208+0000 7fa201988640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1fc0cde50 0x7fa1fc150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.208+0000 7fa201988640 1 --2- 172.21.15.115:0/257370163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1fc0b8400 0x7fa1fc15bca0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:22.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.208+0000 7fa201988640 1 -- 172.21.15.115:0/257370163 >> 172.21.15.115:0/257370163 conn(0x7fa1fc0d2a70 msgr2=0x7fa1fc0b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:22.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.209+0000 7fa201988640 1 -- 172.21.15.115:0/257370163 shutdown_connections 2023-12-01T02:51:22.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.209+0000 7fa201988640 1 -- 172.21.15.115:0/257370163 wait complete. 2023-12-01T02:51:22.241 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T02:51:22.241 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T02:51:22.381 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T02:51:22.486 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T02:51:22.486 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:51:22.486 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:51:22.544 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:51:22.544 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:51:22.593 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:51:22.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.777+0000 7fd5e821e640 1 Processor -- start 2023-12-01T02:51:22.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.777+0000 7fd5e821e640 1 -- start start 2023-12-01T02:51:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.778+0000 7fd5e821e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd5e015cd20 0x7fd5e00d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.778+0000 7fd5e821e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd5e00d1230 0x7fd5e00d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.778+0000 7fd5e821e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e00d1b90 con 0x7fd5e00b5d80 2023-12-01T02:51:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.778+0000 7fd5e821e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00b2160 con 0x7fd5e015cd20 2023-12-01T02:51:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.778+0000 7fd5e821e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00b8c80 con 0x7fd5e00d1230 2023-12-01T02:51:22.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.778+0000 7fd5e5fba640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd5e015cd20 0x7fd5e00d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:22.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.779+0000 7fd5e5fba640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd5e015cd20 0x7fd5e00d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41836/0 (socket says 172.21.15.115:41836) 2023-12-01T02:51:22.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.779+0000 7fd5e5fba640 1 -- 172.21.15.115:0/1039328441 learned_addr learned my addr 172.21.15.115:0/1039328441 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:22.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.779+0000 7fd5e5fba640 1 -- 172.21.15.115:0/1039328441 >> v2:172.21.15.142:3300/0 conn(0x7fd5e00d1230 msgr2=0x7fd5e00d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:22.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.779+0000 7fd5e67bb640 1 --2- 172.21.15.115:0/1039328441 >> v2:172.21.15.142:3300/0 conn(0x7fd5e00d1230 0x7fd5e00d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:22.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.779+0000 7fd5e5fba640 1 --2- 172.21.15.115:0/1039328441 >> v2:172.21.15.142:3300/0 conn(0x7fd5e00d1230 0x7fd5e00d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:22.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.779+0000 7fd5e5fba640 1 -- 172.21.15.115:0/1039328441 >> v1:172.21.15.142:6789/0 conn(0x7fd5e00b5d80 legacy=0x7fd5e00b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:22.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.779+0000 7fd5e5fba640 1 -- 172.21.15.115:0/1039328441 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e00ce2e0 con 0x7fd5e015cd20 2023-12-01T02:51:22.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.780+0000 7fd5e5fba640 1 --2- 172.21.15.115:0/1039328441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e015cd20 0x7fd5e00d24d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00c6720 tx=0x7fd5dc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580d69e551126dc2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.780+0000 7fd5e4fb8640 1 -- 172.21.15.115:0/1039328441 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0317e0 con 0x7fd5e015cd20 2023-12-01T02:51:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.780+0000 7fd5e4fb8640 1 -- 172.21.15.115:0/1039328441 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc031940 con 0x7fd5e015cd20 2023-12-01T02:51:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.780+0000 7fd5e4fb8640 1 -- 172.21.15.115:0/1039328441 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc031cd0 con 0x7fd5e015cd20 2023-12-01T02:51:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.781+0000 7fd5e821e640 1 -- 172.21.15.115:0/1039328441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e015cd20 msgr2=0x7fd5e00d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.781+0000 7fd5e821e640 1 --2- 172.21.15.115:0/1039328441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e015cd20 0x7fd5e00d24d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00c6720 tx=0x7fd5dc02b880 comp rx=0 tx=0).stop 2023-12-01T02:51:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.781+0000 7fd5e821e640 1 -- 172.21.15.115:0/1039328441 shutdown_connections 2023-12-01T02:51:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.781+0000 7fd5e821e640 1 --2- 172.21.15.115:0/1039328441 >> v2:172.21.15.142:3300/0 conn(0x7fd5e00d1230 0x7fd5e00d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.781+0000 7fd5e821e640 1 --2- 172.21.15.115:0/1039328441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e015cd20 0x7fd5e00d24d0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.781+0000 7fd5e821e640 1 -- 172.21.15.115:0/1039328441 >> 172.21.15.115:0/1039328441 conn(0x7fd5e006a0e0 msgr2=0x7fd5e006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:22.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.781+0000 7fd5e821e640 1 -- 172.21.15.115:0/1039328441 shutdown_connections 2023-12-01T02:51:22.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.781+0000 7fd5e821e640 1 -- 172.21.15.115:0/1039328441 wait complete. 2023-12-01T02:51:22.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.784+0000 7fd5e821e640 1 Processor -- start 2023-12-01T02:51:22.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.784+0000 7fd5e821e640 1 -- start start 2023-12-01T02:51:22.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e821e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e00b5d80 0x7fd5e0150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:22.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e821e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e00d1230 0x7fd5e00cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:22.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e821e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e015cd20 0x7fd5e01f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e821e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e01f6730 con 0x7fd5e00d1230 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e821e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e01f6a10 con 0x7fd5e00b5d80 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e821e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e01517b0 con 0x7fd5e015cd20 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e57b9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e00d1230 0x7fd5e00cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e57b9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e00d1230 0x7fd5e00cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41842/0 (socket says 172.21.15.115:41842) 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e57b9640 1 -- 172.21.15.115:0/3662313021 learned_addr learned my addr 172.21.15.115:0/3662313021 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e5fba640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e00b5d80 0x7fd5e0150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e57b9640 1 -- 172.21.15.115:0/3662313021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e015cd20 msgr2=0x7fd5e01f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e57b9640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e015cd20 0x7fd5e01f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e57b9640 1 -- 172.21.15.115:0/3662313021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e00b5d80 msgr2=0x7fd5e0150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e57b9640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e00b5d80 0x7fd5e0150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.786+0000 7fd5e57b9640 1 -- 172.21.15.115:0/3662313021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc0094d0 con 0x7fd5e00d1230 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.787+0000 7fd5e57b9640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e00d1230 0x7fd5e00cae50 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000c410 tx=0x7fd5d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.787+0000 7fd5d67fc640 1 -- 172.21.15.115:0/3662313021 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00035f0 con 0x7fd5e00d1230 2023-12-01T02:51:22.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.787+0000 7fd5d67fc640 1 -- 172.21.15.115:0/3662313021 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d0003750 con 0x7fd5e00d1230 2023-12-01T02:51:22.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.787+0000 7fd5e821e640 1 -- 172.21.15.115:0/3662313021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e0151a90 con 0x7fd5e00d1230 2023-12-01T02:51:22.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.787+0000 7fd5d67fc640 1 -- 172.21.15.115:0/3662313021 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0003b40 con 0x7fd5e00d1230 2023-12-01T02:51:22.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.787+0000 7fd5e821e640 1 -- 172.21.15.115:0/3662313021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e00cb390 con 0x7fd5e00d1230 2023-12-01T02:51:22.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.788+0000 7fd5e821e640 1 -- 172.21.15.115:0/3662313021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e0151cd0 con 0x7fd5e00d1230 2023-12-01T02:51:22.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.789+0000 7fd5d67fc640 1 -- 172.21.15.115:0/3662313021 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 81) v1 ==== 88721+0+0 (secure 0 0 0) 0x7fd5d0003ca0 con 0x7fd5e00d1230 2023-12-01T02:51:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.790+0000 7fd5d67fc640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd5c406a550 0x7fd5c406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.791+0000 7fd5e5fba640 1 -- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd5c406a550 msgr2=0x7fd5c406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.791+0000 7fd5d67fc640 1 -- 172.21.15.115:0/3662313021 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6199+0+0 (secure 0 0 0) 0x7fd5d00882f0 con 0x7fd5e00d1230 2023-12-01T02:51:22.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.791+0000 7fd5e5fba640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd5c406a550 0x7fd5c406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:22.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.794+0000 7fd5d67fc640 1 -- 172.21.15.115:0/3662313021 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd5d0053670 con 0x7fd5e00d1230 2023-12-01T02:51:22.811 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:22.810+0000 7fbe99e10040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T02:51:22.825 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:22.823+0000 7f5858a14040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T02:51:22.865 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:22.863+0000 7fbe99e10040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T02:51:22.878 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:22.877+0000 7f5858a14040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T02:51:22.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.992+0000 7fd5e5fba640 1 -- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd5c406a550 msgr2=0x7fd5c406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:22.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:22.992+0000 7fd5e5fba640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd5c406a550 0x7fd5c406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:51:23.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.027+0000 7fd5e821e640 1 -- 172.21.15.115:0/3662313021 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5e01f6bd0 con 0x7fd5e00d1230 2023-12-01T02:51:23.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.028+0000 7fd5d67fc640 1 -- 172.21.15.115:0/3662313021 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16912 (secure 0 0 0) 0x7fd5d0056c90 con 0x7fd5e00d1230 2023-12-01T02:51:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:23.031 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":149,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:51:22.173247+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:55:22.818353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-01T01:55:24.272936+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":149,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:55:38.761001+0000","172.21.15.115:6822/88317":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3362083032":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3703392460":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3403958692":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/671134594":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3600416058":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/2599234561":"2023-12-02T01:55:38.761001+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":42,"length":15},{"begin":98,"length":15}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:51:23.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 -- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd5c406a550 msgr2=0x7fd5c406ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd5c406a550 0x7fd5c406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 -- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e00d1230 msgr2=0x7fd5e00cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e00d1230 0x7fd5e00cae50 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000c410 tx=0x7fd5d000c8d0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 -- 172.21.15.115:0/3662313021 shutdown_connections 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd5c406a550 0x7fd5c406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e015cd20 0x7fd5e01f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e00d1230 0x7fd5e00cae50 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 --2- 172.21.15.115:0/3662313021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e00b5d80 0x7fd5e0150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 -- 172.21.15.115:0/3662313021 >> 172.21.15.115:0/3662313021 conn(0x7fd5e006a0e0 msgr2=0x7fd5e01587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 -- 172.21.15.115:0/3662313021 shutdown_connections 2023-12-01T02:51:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.031+0000 7fd5e821e640 1 -- 172.21.15.115:0/3662313021 wait complete. 2023-12-01T02:51:23.051 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:55:22.818353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:55:24.272936+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 149, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T02:51:23.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T02:51:23.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.239+0000 7f7bca3ab640 1 Processor -- start 2023-12-01T02:51:23.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.239+0000 7f7bca3ab640 1 -- start start 2023-12-01T02:51:23.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.240+0000 7f7bca3ab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7bc415cd90 0x7f7bc40cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.240+0000 7f7bca3ab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7bc40b5fa0 0x7f7bc4161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.240+0000 7f7bca3ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc40cf650 0x7f7bc40cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.240+0000 7f7bca3ab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc40d0ec0 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bca3ab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc40ce5e0 con 0x7f7bc415cd90 2023-12-01T02:51:23.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bca3ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc4161950 con 0x7f7bc40cf650 2023-12-01T02:51:23.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bc37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7bc40b5fa0 0x7f7bc4161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bc37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7bc40b5fa0 0x7f7bc4161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41858/0 (socket says 172.21.15.115:41858) 2023-12-01T02:51:23.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bc37fe640 1 -- 172.21.15.115:0/3669685273 learned_addr learned my addr 172.21.15.115:0/3669685273 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:23.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bc8948640 1 --2- 172.21.15.115:0/3669685273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc40cf650 0x7f7bc40cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bc3fff640 1 --2- 172.21.15.115:0/3669685273 >> v2:172.21.15.142:3300/0 conn(0x7f7bc415cd90 0x7f7bc40cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bc37fe640 1 -- 172.21.15.115:0/3669685273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc40cf650 msgr2=0x7f7bc40cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:23.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bc37fe640 1 --2- 172.21.15.115:0/3669685273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc40cf650 0x7f7bc40cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bc37fe640 1 -- 172.21.15.115:0/3669685273 >> v2:172.21.15.142:3300/0 conn(0x7f7bc415cd90 msgr2=0x7f7bc40cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:23.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bc37fe640 1 --2- 172.21.15.115:0/3669685273 >> v2:172.21.15.142:3300/0 conn(0x7f7bc415cd90 0x7f7bc40cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.241+0000 7f7bc37fe640 1 -- 172.21.15.115:0/3669685273 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc40cffc0 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.242+0000 7f7bc37fe640 1 --2- 172.21.15.115:0/3669685273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 0x7f7bc4161410 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f7bc406bd10 tx=0x7f7bb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694dcb45bf8306d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:23.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.243+0000 7f7bc2ffd640 1 -- 172.21.15.115:0/3669685273 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8031870 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.243+0000 7f7bc2ffd640 1 -- 172.21.15.115:0/3669685273 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb80319d0 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.243+0000 7f7bc2ffd640 1 -- 172.21.15.115:0/3669685273 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8031d60 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.244+0000 7f7bca3ab640 1 -- 172.21.15.115:0/3669685273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 msgr2=0x7f7bc4161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:23.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.244+0000 7f7bca3ab640 1 --2- 172.21.15.115:0/3669685273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 0x7f7bc4161410 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f7bc406bd10 tx=0x7f7bb802b910 comp rx=0 tx=0).stop 2023-12-01T02:51:23.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.244+0000 7f7bca3ab640 1 -- 172.21.15.115:0/3669685273 shutdown_connections 2023-12-01T02:51:23.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.244+0000 7f7bca3ab640 1 --2- 172.21.15.115:0/3669685273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc40cf650 0x7f7bc40cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.244+0000 7f7bca3ab640 1 --2- 172.21.15.115:0/3669685273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 0x7f7bc4161410 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.244+0000 7f7bca3ab640 1 --2- 172.21.15.115:0/3669685273 >> v2:172.21.15.142:3300/0 conn(0x7f7bc415cd90 0x7f7bc40cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.244+0000 7f7bca3ab640 1 -- 172.21.15.115:0/3669685273 >> 172.21.15.115:0/3669685273 conn(0x7f7bc4065960 msgr2=0x7f7bc4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:23.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.244+0000 7f7bca3ab640 1 -- 172.21.15.115:0/3669685273 shutdown_connections 2023-12-01T02:51:23.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.244+0000 7f7bca3ab640 1 -- 172.21.15.115:0/3669685273 wait complete. 2023-12-01T02:51:23.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.246+0000 7f7bca3ab640 1 Processor -- start 2023-12-01T02:51:23.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.246+0000 7f7bca3ab640 1 -- start start 2023-12-01T02:51:23.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bca3ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 0x7f7bc40c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bca3ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc40cf650 0x7f7bc415e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bca3ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc415cd90 0x7f7bc41552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bca3ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40c96d0 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bca3ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc415fff0 con 0x7f7bc415cd90 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bca3ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc415ef50 con 0x7f7bc40cf650 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 0x7f7bc40c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 0x7f7bc40c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41870/0 (socket says 172.21.15.115:41870) 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc3fff640 1 -- 172.21.15.115:0/79697803 learned_addr learned my addr 172.21.15.115:0/79697803 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc37fe640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc40cf650 0x7f7bc415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc8948640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc415cd90 0x7f7bc41552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc3fff640 1 -- 172.21.15.115:0/79697803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc40cf650 msgr2=0x7f7bc415e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc3fff640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc40cf650 0x7f7bc415e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc3fff640 1 -- 172.21.15.115:0/79697803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc415cd90 msgr2=0x7f7bc41552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc3fff640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc415cd90 0x7f7bc41552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc3fff640 1 -- 172.21.15.115:0/79697803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80094d0 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.247+0000 7f7bc8948640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc415cd90 0x7f7bc41552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:23.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.248+0000 7f7bc3fff640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 0x7f7bc40c9190 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400d620 tx=0x7f7bb400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:23.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.248+0000 7f7bc0ff9640 1 -- 172.21.15.115:0/79697803 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400b620 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.248+0000 7f7bc0ff9640 1 -- 172.21.15.115:0/79697803 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb400b780 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.248+0000 7f7bc0ff9640 1 -- 172.21.15.115:0/79697803 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400bb30 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.248+0000 7f7bca3ab640 1 -- 172.21.15.115:0/79697803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc415f230 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.248+0000 7f7bca3ab640 1 -- 172.21.15.115:0/79697803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc415f7f0 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.250+0000 7f7bca3ab640 1 -- 172.21.15.115:0/79697803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc404e890 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.250+0000 7f7bc0ff9640 1 -- 172.21.15.115:0/79697803 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 81) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f7bb400bc90 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.251+0000 7f7bc0ff9640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bb006a550 0x7f7bb006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.251+0000 7f7bc0ff9640 1 -- 172.21.15.115:0/79697803 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6199+0+0 (secure 0 0 0) 0x7f7bb4089200 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.251+0000 7f7bc37fe640 1 -- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bb006a550 msgr2=0x7f7bb006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:23.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.251+0000 7f7bc37fe640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bb006a550 0x7f7bb006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.255+0000 7f7bc0ff9640 1 -- 172.21.15.115:0/79697803 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7bb4054450 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.345 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:23.343+0000 7f5858a14040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T02:51:23.379 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:23.377+0000 7fbe99e10040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T02:51:23.397 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:23.394+0000 7f5858a14040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T02:51:23.431 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:23.430+0000 7fbe99e10040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T02:51:23.446 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:23.444+0000 7f5858a14040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T02:51:23.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.452+0000 7f7bc37fe640 1 -- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bb006a550 msgr2=0x7f7bb006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:23.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.452+0000 7f7bc37fe640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bb006a550 0x7f7bb006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:51:23.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.462+0000 7f7bca3ab640 1 -- 172.21.15.115:0/79697803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bc4161190 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.463+0000 7f7bc0ff9640 1 -- 172.21.15.115:0/79697803 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bb4057a70 con 0x7f7bc40b5fa0 2023-12-01T02:51:23.465 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T02:51:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 -- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bb006a550 msgr2=0x7f7bb006ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bb006a550 0x7f7bb006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 -- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 msgr2=0x7f7bc40c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 0x7f7bc40c9190 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400d620 tx=0x7f7bb400dae0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 -- 172.21.15.115:0/79697803 shutdown_connections 2023-12-01T02:51:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7bb006a550 0x7f7bb006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bc40cf650 0x7f7bc415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bc415cd90 0x7f7bc41552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 --2- 172.21.15.115:0/79697803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bc40b5fa0 0x7f7bc40c9190 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 -- 172.21.15.115:0/79697803 >> 172.21.15.115:0/79697803 conn(0x7f7bc4065960 msgr2=0x7f7bc40b75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.465+0000 7f7bca3ab640 1 -- 172.21.15.115:0/79697803 shutdown_connections 2023-12-01T02:51:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.466+0000 7f7bca3ab640 1 -- 172.21.15.115:0/79697803 wait complete. 2023-12-01T02:51:23.484 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T02:51:23.484 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:23.483+0000 7fbe99e10040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T02:51:23.506 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:23.504+0000 7f5858a14040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T02:51:23.550 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:23.548+0000 7fbe99e10040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T02:51:23.554 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:23.552+0000 7f5858a14040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T02:51:23.602 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:23.600+0000 7fbe99e10040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T02:51:23.664 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:23.662+0000 7f5858a14040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T02:51:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.685+0000 7f78cbfff640 1 Processor -- start 2023-12-01T02:51:23.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.685+0000 7f78cbfff640 1 -- start start 2023-12-01T02:51:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.686+0000 7f78cbfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f78cc153c30 0x7f78cc0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.686+0000 7f78cbfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f78cc154010 0x7f78cc0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.686+0000 7f78cbfff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78cc0cdcd0 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.686+0000 7f78cbfff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78cc0cfe20 con 0x7f78cc153c30 2023-12-01T02:51:23.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.686+0000 7f78cbfff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78cc161830 con 0x7f78cc154010 2023-12-01T02:51:23.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.687+0000 7f78caffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f78cc153c30 0x7f78cc0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.687+0000 7f78caffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f78cc153c30 0x7f78cc0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41886/0 (socket says 172.21.15.115:41886) 2023-12-01T02:51:23.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.687+0000 7f78caffd640 1 -- 172.21.15.115:0/3259789815 learned_addr learned my addr 172.21.15.115:0/3259789815 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:23.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.687+0000 7f78ca7fc640 1 --2- 172.21.15.115:0/3259789815 >> v2:172.21.15.142:3300/0 conn(0x7f78cc154010 0x7f78cc0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.687+0000 7f78caffd640 1 -- 172.21.15.115:0/3259789815 >> v2:172.21.15.142:3300/0 conn(0x7f78cc154010 msgr2=0x7f78cc0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:23.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.687+0000 7f78caffd640 1 --2- 172.21.15.115:0/3259789815 >> v2:172.21.15.142:3300/0 conn(0x7f78cc154010 0x7f78cc0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.687+0000 7f78caffd640 1 -- 172.21.15.115:0/3259789815 >> v1:172.21.15.142:6789/0 conn(0x7f78cc0cf9b0 legacy=0x7f78cc160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:23.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.687+0000 7f78caffd640 1 -- 172.21.15.115:0/3259789815 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc0cd3a0 con 0x7f78cc153c30 2023-12-01T02:51:23.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.687+0000 7f78ca7fc640 1 --2- 172.21.15.115:0/3259789815 >> v2:172.21.15.142:3300/0 conn(0x7f78cc154010 0x7f78cc0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:23.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.688+0000 7f78caffd640 1 --2- 172.21.15.115:0/3259789815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc153c30 0x7f78cc0ce160 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0d3520 tx=0x7f78bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715935556a15e7ba server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:23.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78c9ffb640 1 -- 172.21.15.115:0/3259789815 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc031870 con 0x7f78cc153c30 2023-12-01T02:51:23.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78c9ffb640 1 -- 172.21.15.115:0/3259789815 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78bc0319d0 con 0x7f78cc153c30 2023-12-01T02:51:23.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78c9ffb640 1 -- 172.21.15.115:0/3259789815 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc031d60 con 0x7f78cc153c30 2023-12-01T02:51:23.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78cbfff640 1 -- 172.21.15.115:0/3259789815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc153c30 msgr2=0x7f78cc0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:23.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78cbfff640 1 --2- 172.21.15.115:0/3259789815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc153c30 0x7f78cc0ce160 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0d3520 tx=0x7f78bc02b910 comp rx=0 tx=0).stop 2023-12-01T02:51:23.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78cbfff640 1 -- 172.21.15.115:0/3259789815 shutdown_connections 2023-12-01T02:51:23.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78cbfff640 1 --2- 172.21.15.115:0/3259789815 >> v2:172.21.15.142:3300/0 conn(0x7f78cc154010 0x7f78cc0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78cbfff640 1 --2- 172.21.15.115:0/3259789815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc153c30 0x7f78cc0ce160 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78cbfff640 1 -- 172.21.15.115:0/3259789815 >> 172.21.15.115:0/3259789815 conn(0x7f78cc06a0b0 msgr2=0x7f78cc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:23.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78cbfff640 1 -- 172.21.15.115:0/3259789815 shutdown_connections 2023-12-01T02:51:23.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.689+0000 7f78cbfff640 1 -- 172.21.15.115:0/3259789815 wait complete. 2023-12-01T02:51:23.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.692+0000 7f78cbfff640 1 Processor -- start 2023-12-01T02:51:23.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.692+0000 7f78cbfff640 1 -- start start 2023-12-01T02:51:23.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.694+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc0cf9b0 0x7f78cc1f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.694+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc153c30 0x7f78cc0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.694+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc154010 0x7f78cc0c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.694+0000 7f78cbfff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1f2fd0 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.694+0000 7f78cbfff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc0cbbc0 con 0x7f78cc154010 2023-12-01T02:51:23.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.694+0000 7f78cbfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1f23e0 con 0x7f78cc153c30 2023-12-01T02:51:23.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.694+0000 7f78caffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc0cf9b0 0x7f78cc1f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.694+0000 7f78caffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc0cf9b0 0x7f78cc1f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41900/0 (socket says 172.21.15.115:41900) 2023-12-01T02:51:23.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.694+0000 7f78caffd640 1 -- 172.21.15.115:0/776267241 learned_addr learned my addr 172.21.15.115:0/776267241 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:23.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.694+0000 7f78cb7fe640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc154010 0x7f78cc0c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78ca7fc640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc153c30 0x7f78cc0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:23.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78caffd640 1 -- 172.21.15.115:0/776267241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc153c30 msgr2=0x7f78cc0cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:23.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78caffd640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc153c30 0x7f78cc0cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78caffd640 1 -- 172.21.15.115:0/776267241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc154010 msgr2=0x7f78cc0c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:23.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78caffd640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc154010 0x7f78cc0c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78caffd640 1 -- 172.21.15.115:0/776267241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc0094d0 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78caffd640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc0cf9b0 0x7f78cc1f6570 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f78bc000c00 tx=0x7f78bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:23.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78b37fe640 1 -- 172.21.15.115:0/776267241 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc031840 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78b37fe640 1 -- 172.21.15.115:0/776267241 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78bc0443f0 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78b37fe640 1 -- 172.21.15.115:0/776267241 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0447a0 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.695+0000 7f78cbfff640 1 -- 172.21.15.115:0/776267241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc1f26a0 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.696+0000 7f78cbfff640 1 -- 172.21.15.115:0/776267241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc1f2c60 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.697+0000 7f78cbfff640 1 -- 172.21.15.115:0/776267241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc0618d0 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.699+0000 7f78b37fe640 1 -- 172.21.15.115:0/776267241 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 81) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f78bc04d450 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.700+0000 7f78b37fe640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78b806a550 0x7f78b806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:23.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.700+0000 7f78b37fe640 1 -- 172.21.15.115:0/776267241 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6199+0+0 (secure 0 0 0) 0x7f78bc02d040 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.700+0000 7f78ca7fc640 1 -- 172.21.15.115:0/776267241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78b806a550 msgr2=0x7f78b806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:23.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.700+0000 7f78ca7fc640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78b806a550 0x7f78b806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:23.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.703+0000 7f78b37fe640 1 -- 172.21.15.115:0/776267241 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f78bc076c50 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.723 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:23.721+0000 7fbe99e10040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T02:51:23.727 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:23.725+0000 7f5858a14040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T02:51:23.776 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:23.774+0000 7f5858a14040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T02:51:23.788 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:23.786+0000 7fbe99e10040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T02:51:23.832 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:23.830+0000 7f5858a14040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T02:51:23.839 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:23.838+0000 7fbe99e10040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T02:51:23.882 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:23.880+0000 7f5858a14040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T02:51:23.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.894+0000 7f78cbfff640 1 -- 172.21.15.115:0/776267241 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78cc1611a0 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.894+0000 7f78b37fe640 1 -- 172.21.15.115:0/776267241 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v149) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78bc07a270 con 0x7f78cc0cf9b0 2023-12-01T02:51:23.896 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T02:51:23.898 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:23.896+0000 7fbe99e10040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T02:51:23.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 -- 172.21.15.115:0/776267241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78b806a550 msgr2=0x7f78b806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78b806a550 0x7f78b806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 -- 172.21.15.115:0/776267241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc0cf9b0 msgr2=0x7f78cc1f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc0cf9b0 0x7f78cc1f6570 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f78bc000c00 tx=0x7f78bc0046c0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 -- 172.21.15.115:0/776267241 shutdown_connections 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f78b806a550 0x7f78b806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc153c30 0x7f78cc0cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc154010 0x7f78cc0c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 --2- 172.21.15.115:0/776267241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78cc0cf9b0 0x7f78cc1f6570 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 -- 172.21.15.115:0/776267241 >> 172.21.15.115:0/776267241 conn(0x7f78cc06a0b0 msgr2=0x7f78cc151aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 -- 172.21.15.115:0/776267241 shutdown_connections 2023-12-01T02:51:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:23.897+0000 7f78cbfff640 1 -- 172.21.15.115:0/776267241 wait complete. 2023-12-01T02:51:23.917 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T02:51:23.954 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:23.952+0000 7fbe99e10040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T02:51:24.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.117+0000 7feda5548640 1 Processor -- start 2023-12-01T02:51:24.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.117+0000 7feda5548640 1 -- start start 2023-12-01T02:51:24.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.119+0000 7feda5548640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda00cd470 0x7feda00b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.119+0000 7feda5548640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda00cc2b0 con 0x7feda00b2040 2023-12-01T02:51:24.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.119+0000 7feda5548640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda00ce0b0 con 0x7feda015e720 2023-12-01T02:51:24.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.119+0000 7feda5548640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda00cddd0 con 0x7feda00cd470 2023-12-01T02:51:24.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.119+0000 7fed97fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7feda00b2040 0x7feda015e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40226/0 (socket says 172.21.15.115:40226) 2023-12-01T02:51:24.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.119+0000 7fed97fff640 1 -- 172.21.15.115:0/1686623148 learned_addr learned my addr 172.21.15.115:0/1686623148 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:24.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.119+0000 7fed9effd640 1 --2- 172.21.15.115:0/1686623148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda00cd470 0x7feda00b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:24.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524916067 0 0) 0x7feda00cc2b0 con 0x7feda00b2040 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed8c003520 con 0x7feda00b2040 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 549537760 0 0) 0x7fed8c003520 con 0x7feda00b2040 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda00cd470 msgr2=0x7feda00b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 --2- 172.21.15.115:0/1686623148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda00cd470 0x7feda00b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 >> v1:172.21.15.142:6789/0 conn(0x7feda015e720 legacy=0x7feda00c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda00d1780 con 0x7feda00b2040 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fed88001670 con 0x7feda00b2040 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 >> v1:172.21.15.115:6789/0 conn(0x7feda00b2040 legacy=0x7feda015e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 --2- 172.21.15.115:0/1686623148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed8c0039a0 0x7fed8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 --2- 172.21.15.115:0/1686623148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda00cd470 0x7fed8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 --2- 172.21.15.115:0/1686623148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c004d00 0x7fed8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda00cddd0 con 0x7fed8c0039a0 2023-12-01T02:51:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c00ba60 con 0x7fed8c004d00 2023-12-01T02:51:24.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c00bc20 con 0x7feda00cd470 2023-12-01T02:51:24.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.120+0000 7fed9effd640 1 --2- 172.21.15.115:0/1686623148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed8c0039a0 0x7fed8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:24.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.121+0000 7fed9effd640 1 -- 172.21.15.115:0/1686623148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda00cd470 msgr2=0x7fed8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:24.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.121+0000 7fed9effd640 1 --2- 172.21.15.115:0/1686623148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda00cd470 0x7fed8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.121+0000 7fed9effd640 1 -- 172.21.15.115:0/1686623148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c004d00 msgr2=0x7fed8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:24.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.121+0000 7fed9effd640 1 --2- 172.21.15.115:0/1686623148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c004d00 0x7fed8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.121+0000 7fed9effd640 1 -- 172.21.15.115:0/1686623148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c003520 con 0x7fed8c0039a0 2023-12-01T02:51:24.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.122+0000 7fed9effd640 1 --2- 172.21.15.115:0/1686623148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed8c0039a0 0x7fed8c003dc0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fed90009a90 tx=0x7fed8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9390962524a2b89a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:24.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.122+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9002c6b0 con 0x7fed8c0039a0 2023-12-01T02:51:24.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.122+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9002c810 con 0x7fed8c0039a0 2023-12-01T02:51:24.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.122+0000 7fed9e7fc640 1 -- 172.21.15.115:0/1686623148 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9002cba0 con 0x7fed8c0039a0 2023-12-01T02:51:24.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.123+0000 7feda5548640 1 -- 172.21.15.115:0/1686623148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed8c0039a0 msgr2=0x7fed8c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:24.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.123+0000 7feda5548640 1 --2- 172.21.15.115:0/1686623148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fed8c0039a0 0x7fed8c003dc0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fed90009a90 tx=0x7fed8c0057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.123+0000 7fed9f7fe640 1 -- 172.21.15.115:0/1686623148 reap_dead start 2023-12-01T02:51:24.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.123+0000 7feda5548640 1 -- 172.21.15.115:0/1686623148 shutdown_connections 2023-12-01T02:51:24.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.123+0000 7feda5548640 1 -- 172.21.15.115:0/1686623148 >> 172.21.15.115:0/1686623148 conn(0x7feda00d0ba0 msgr2=0x7feda00d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:24.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.123+0000 7feda5548640 1 -- 172.21.15.115:0/1686623148 shutdown_connections 2023-12-01T02:51:24.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.123+0000 7feda5548640 1 -- 172.21.15.115:0/1686623148 wait complete. 2023-12-01T02:51:24.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.125+0000 7feda5548640 1 Processor -- start 2023-12-01T02:51:24.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.125+0000 7feda5548640 1 -- start start 2023-12-01T02:51:24.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.125+0000 7feda5548640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c0039a0 0x7feda00b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7feda5548640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda006a720 0x7feda00c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7feda5548640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda006aba0 0x7feda00ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7feda5548640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda00c7820 con 0x7feda006aba0 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7feda5548640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda00b64e0 con 0x7fed8c0039a0 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7feda5548640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda0151d00 con 0x7feda006a720 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7fed9f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda006aba0 0x7feda00ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7fed9f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda006aba0 0x7feda00ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41926/0 (socket says 172.21.15.115:41926) 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7fed9f7fe640 1 -- 172.21.15.115:0/3553936341 learned_addr learned my addr 172.21.15.115:0/3553936341 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7fed9effd640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c0039a0 0x7feda00b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7fed9f7fe640 1 -- 172.21.15.115:0/3553936341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda006a720 msgr2=0x7feda00c9e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7fed9f7fe640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda006a720 0x7feda00c9e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7fed9f7fe640 1 -- 172.21.15.115:0/3553936341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c0039a0 msgr2=0x7feda00b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7fed9f7fe640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c0039a0 0x7feda00b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7fed9f7fe640 1 -- 172.21.15.115:0/3553936341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda00c6ba0 con 0x7feda006aba0 2023-12-01T02:51:24.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.126+0000 7fed9f7fe640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda006aba0 0x7feda00ca380 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7feda00b6c50 tx=0x7fed9800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:24.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.127+0000 7fed977fe640 1 -- 172.21.15.115:0/3553936341 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98005d40 con 0x7feda006aba0 2023-12-01T02:51:24.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.127+0000 7fed977fe640 1 -- 172.21.15.115:0/3553936341 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9800ced0 con 0x7feda006aba0 2023-12-01T02:51:24.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.127+0000 7fed977fe640 1 -- 172.21.15.115:0/3553936341 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9800a850 con 0x7feda006aba0 2023-12-01T02:51:24.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.127+0000 7feda5548640 1 -- 172.21.15.115:0/3553936341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda00c6e80 con 0x7feda006aba0 2023-12-01T02:51:24.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.127+0000 7feda5548640 1 -- 172.21.15.115:0/3553936341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda00c7440 con 0x7feda006aba0 2023-12-01T02:51:24.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.128+0000 7feda5548640 1 -- 172.21.15.115:0/3553936341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda0067850 con 0x7feda006aba0 2023-12-01T02:51:24.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.130+0000 7fed977fe640 1 -- 172.21.15.115:0/3553936341 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 81) v1 ==== 88721+0+0 (secure 0 0 0) 0x7fed9800a9b0 con 0x7feda006aba0 2023-12-01T02:51:24.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.131+0000 7fed977fe640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed8006a550 0x7fed8006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.131+0000 7fed977fe640 1 -- 172.21.15.115:0/3553936341 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6199+0+0 (secure 0 0 0) 0x7fed98087df0 con 0x7feda006aba0 2023-12-01T02:51:24.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.131+0000 7fed9effd640 1 -- 172.21.15.115:0/3553936341 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed8006a550 msgr2=0x7fed8006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:24.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.131+0000 7fed9effd640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed8006a550 0x7fed8006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:24.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.136+0000 7fed977fe640 1 -- 172.21.15.115:0/3553936341 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fed980530e0 con 0x7feda006aba0 2023-12-01T02:51:24.263 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:24.261+0000 7f5858a14040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T02:51:24.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.327+0000 7feda5548640 1 -- 172.21.15.115:0/3553936341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feda00c7960 con 0x7feda006aba0 2023-12-01T02:51:24.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.327+0000 7fed977fe640 1 -- 172.21.15.115:0/3553936341 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v149) v1 ==== 100+0+11 (secure 0 0 0) 0x7feda00c7960 con 0x7feda006aba0 2023-12-01T02:51:24.329 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T02:51:24.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 -- 172.21.15.115:0/3553936341 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed8006a550 msgr2=0x7fed8006ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:24.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed8006a550 0x7fed8006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 -- 172.21.15.115:0/3553936341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda006aba0 msgr2=0x7feda00ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:24.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda006aba0 0x7feda00ca380 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7feda00b6c50 tx=0x7fed9800cb90 comp rx=0 tx=0).stop 2023-12-01T02:51:24.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 -- 172.21.15.115:0/3553936341 shutdown_connections 2023-12-01T02:51:24.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fed8006a550 0x7fed8006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feda006a720 0x7feda00c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feda006aba0 0x7feda00ca380 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 --2- 172.21.15.115:0/3553936341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed8c0039a0 0x7feda00b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 -- 172.21.15.115:0/3553936341 >> 172.21.15.115:0/3553936341 conn(0x7feda00d0ba0 msgr2=0x7feda00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:24.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 -- 172.21.15.115:0/3553936341 shutdown_connections 2023-12-01T02:51:24.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.330+0000 7feda5548640 1 -- 172.21.15.115:0/3553936341 wait complete. 2023-12-01T02:51:24.351 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T02:51:24.353 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:24.352+0000 7fbe99e10040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T02:51:24.422 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:24.420+0000 7f5858a14040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T02:51:24.523 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:24.521+0000 7fbe99e10040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T02:51:24.530 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:24.528+0000 7f5858a14040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T02:51:24.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.554+0000 7fb85f277640 1 Processor -- start 2023-12-01T02:51:24.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.554+0000 7fb85f277640 1 -- start start 2023-12-01T02:51:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.555+0000 7fb85f277640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb85815e2f0 0x7fb8580ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.555+0000 7fb85f277640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb85806f2f0 0x7fb8580caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.555+0000 7fb85f277640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb8580adf10 0x7fb8580c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.555+0000 7fb85f277640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8580add10 con 0x7fb8580adf10 2023-12-01T02:51:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.555+0000 7fb85f277640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8580c5760 con 0x7fb85806f2f0 2023-12-01T02:51:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.555+0000 7fb85f277640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85815e6d0 con 0x7fb85815e2f0 2023-12-01T02:51:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.555+0000 7fb85d814640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb8580adf10 0x7fb8580c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:24.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.555+0000 7fb85d814640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb8580adf10 0x7fb8580c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41928/0 (socket says 172.21.15.115:41928) 2023-12-01T02:51:24.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.555+0000 7fb85d814640 1 -- 172.21.15.115:0/3751789481 learned_addr learned my addr 172.21.15.115:0/3751789481 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.556+0000 7fb85c812640 1 --2- 172.21.15.115:0/3751789481 >> v2:172.21.15.142:3300/0 conn(0x7fb85806f2f0 0x7fb8580caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.556+0000 7fb85d013640 1 --2- 172.21.15.115:0/3751789481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb85815e2f0 0x7fb8580ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.556+0000 7fb85d814640 1 -- 172.21.15.115:0/3751789481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb85815e2f0 msgr2=0x7fb8580ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.556+0000 7fb85d814640 1 --2- 172.21.15.115:0/3751789481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb85815e2f0 0x7fb8580ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.556+0000 7fb85d814640 1 -- 172.21.15.115:0/3751789481 >> v2:172.21.15.142:3300/0 conn(0x7fb85806f2f0 msgr2=0x7fb8580caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.556+0000 7fb85d814640 1 --2- 172.21.15.115:0/3751789481 >> v2:172.21.15.142:3300/0 conn(0x7fb85806f2f0 0x7fb8580caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.556+0000 7fb85d814640 1 -- 172.21.15.115:0/3751789481 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8580cf7b0 con 0x7fb8580adf10 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.557+0000 7fb85d814640 1 --2- 172.21.15.115:0/3751789481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8580adf10 0x7fb8580c8ed0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fb8580b1460 tx=0x7fb85002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88394280adc059b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.557+0000 7fb857fff640 1 -- 172.21.15.115:0/3751789481 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850031750 con 0x7fb8580adf10 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.557+0000 7fb857fff640 1 -- 172.21.15.115:0/3751789481 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8500318b0 con 0x7fb8580adf10 2023-12-01T02:51:24.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.557+0000 7fb857fff640 1 -- 172.21.15.115:0/3751789481 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850031c40 con 0x7fb8580adf10 2023-12-01T02:51:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.557+0000 7fb85f277640 1 -- 172.21.15.115:0/3751789481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8580adf10 msgr2=0x7fb8580c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.557+0000 7fb85f277640 1 --2- 172.21.15.115:0/3751789481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8580adf10 0x7fb8580c8ed0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fb8580b1460 tx=0x7fb85002b7f0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.558+0000 7fb85f277640 1 -- 172.21.15.115:0/3751789481 shutdown_connections 2023-12-01T02:51:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.558+0000 7fb85f277640 1 --2- 172.21.15.115:0/3751789481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8580adf10 0x7fb8580c8ed0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.558+0000 7fb85f277640 1 --2- 172.21.15.115:0/3751789481 >> v2:172.21.15.142:3300/0 conn(0x7fb85806f2f0 0x7fb8580caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.558+0000 7fb85f277640 1 --2- 172.21.15.115:0/3751789481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb85815e2f0 0x7fb8580ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.558+0000 7fb85f277640 1 -- 172.21.15.115:0/3751789481 >> 172.21.15.115:0/3751789481 conn(0x7fb8580ce880 msgr2=0x7fb8580cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.558+0000 7fb85f277640 1 -- 172.21.15.115:0/3751789481 shutdown_connections 2023-12-01T02:51:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.558+0000 7fb85f277640 1 -- 172.21.15.115:0/3751789481 wait complete. 2023-12-01T02:51:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.559+0000 7fb85f277640 1 Processor -- start 2023-12-01T02:51:24.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.559+0000 7fb85f277640 1 -- start start 2023-12-01T02:51:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.560+0000 7fb85f277640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85806f2f0 0x7fb858159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.560+0000 7fb85f277640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8580adf10 0x7fb8580cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.560+0000 7fb85f277640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85815e2f0 0x7fb8580c2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.560+0000 7fb85f277640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580c3610 con 0x7fb85806f2f0 2023-12-01T02:51:24.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.560+0000 7fb85f277640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580cda20 con 0x7fb85815e2f0 2023-12-01T02:51:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.560+0000 7fb85f277640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580c3050 con 0x7fb8580adf10 2023-12-01T02:51:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d013640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85806f2f0 0x7fb858159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d013640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85806f2f0 0x7fb858159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41942/0 (socket says 172.21.15.115:41942) 2023-12-01T02:51:24.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d013640 1 -- 172.21.15.115:0/3241103310 learned_addr learned my addr 172.21.15.115:0/3241103310 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d814640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85815e2f0 0x7fb8580c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85c812640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8580adf10 0x7fb8580cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d013640 1 -- 172.21.15.115:0/3241103310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8580adf10 msgr2=0x7fb8580cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d013640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8580adf10 0x7fb8580cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d013640 1 -- 172.21.15.115:0/3241103310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85815e2f0 msgr2=0x7fb8580c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d013640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85815e2f0 0x7fb8580c2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d013640 1 -- 172.21.15.115:0/3241103310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8500094d0 con 0x7fb85806f2f0 2023-12-01T02:51:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d814640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85815e2f0 0x7fb8580c2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.561+0000 7fb85d013640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85806f2f0 0x7fb858159ff0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fb84800ccd0 tx=0x7fb84800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.562+0000 7fb855ffb640 1 -- 172.21.15.115:0/3241103310 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848003990 con 0x7fb85806f2f0 2023-12-01T02:51:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.562+0000 7fb855ffb640 1 -- 172.21.15.115:0/3241103310 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb848003af0 con 0x7fb85806f2f0 2023-12-01T02:51:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.562+0000 7fb855ffb640 1 -- 172.21.15.115:0/3241103310 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848004020 con 0x7fb85806f2f0 2023-12-01T02:51:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.562+0000 7fb85f277640 1 -- 172.21.15.115:0/3241103310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8580c3330 con 0x7fb85806f2f0 2023-12-01T02:51:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.562+0000 7fb85f277640 1 -- 172.21.15.115:0/3241103310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8580ce3f0 con 0x7fb85806f2f0 2023-12-01T02:51:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.563+0000 7fb85f277640 1 -- 172.21.15.115:0/3241103310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85806fda0 con 0x7fb85806f2f0 2023-12-01T02:51:24.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.564+0000 7fb855ffb640 1 -- 172.21.15.115:0/3241103310 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 81) v1 ==== 88721+0+0 (secure 0 0 0) 0x7fb848004180 con 0x7fb85806f2f0 2023-12-01T02:51:24.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.565+0000 7fb855ffb640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 0x7fb84406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:24.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.565+0000 7fb85c812640 1 -- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 msgr2=0x7fb84406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:24.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.566+0000 7fb855ffb640 1 -- 172.21.15.115:0/3241103310 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6199+0+0 (secure 0 0 0) 0x7fb8480890f0 con 0x7fb85806f2f0 2023-12-01T02:51:24.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.566+0000 7fb85c812640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 0x7fb84406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:24.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.570+0000 7fb855ffb640 1 -- 172.21.15.115:0/3241103310 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb848054360 con 0x7fb85806f2f0 2023-12-01T02:51:24.580 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:24.579+0000 7f5858a14040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T02:51:24.639 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:24.637+0000 7fbe99e10040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T02:51:24.688 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:24.686+0000 7f5858a14040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:51:24.692 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:24.689+0000 7fbe99e10040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T02:51:24.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.767+0000 7fb85c812640 1 -- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 msgr2=0x7fb84406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:24.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.767+0000 7fb85c812640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 0x7fb84406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:51:24.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.773+0000 7fb85f277640 1 -- 172.21.15.115:0/3241103310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb8580cee90 con 0x7fb85806f2f0 2023-12-01T02:51:24.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.774+0000 7fb855ffb640 1 -- 172.21.15.115:0/3241103310 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v149) v1 ==== 80+0+467 (secure 0 0 0) 0x7fb848057980 con 0x7fb85806f2f0 2023-12-01T02:51:24.776 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/88317 2023-12-02T01:55:38.761001+0000 2023-12-01T02:51:24.776 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/88317 2023-12-02T01:55:38.761001+0000 2023-12-01T02:51:24.776 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3362083032 2023-12-02T01:55:38.761001+0000 2023-12-01T02:51:24.776 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3703392460 2023-12-02T01:55:38.761001+0000 2023-12-01T02:51:24.776 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3403958692 2023-12-02T01:55:38.761001+0000 2023-12-01T02:51:24.776 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/671134594 2023-12-02T01:55:38.761001+0000 2023-12-01T02:51:24.776 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3600416058 2023-12-02T01:55:38.761001+0000 2023-12-01T02:51:24.776 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2599234561 2023-12-02T01:55:38.761001+0000 2023-12-01T02:51:24.777 INFO:teuthology.orchestra.run.smithi115.stderr:listed 8 entries 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 -- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 msgr2=0x7fb84406ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 0x7fb84406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 -- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85806f2f0 msgr2=0x7fb858159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85806f2f0 0x7fb858159ff0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fb84800ccd0 tx=0x7fb84800a5c0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 -- 172.21.15.115:0/3241103310 shutdown_connections 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb84406a550 0x7fb84406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8580adf10 0x7fb8580cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb85815e2f0 0x7fb8580c2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 --2- 172.21.15.115:0/3241103310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb85806f2f0 0x7fb858159ff0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 -- 172.21.15.115:0/3241103310 >> 172.21.15.115:0/3241103310 conn(0x7fb8580ce880 msgr2=0x7fb85815a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 -- 172.21.15.115:0/3241103310 shutdown_connections 2023-12-01T02:51:24.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:24.777+0000 7fb85f277640 1 -- 172.21.15.115:0/3241103310 wait complete. 2023-12-01T02:51:24.794 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:24.791+0000 7f5858a14040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T02:51:24.796 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T02:51:24.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:24.796 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T02:51:24.804 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:24.802+0000 7fbe99e10040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:51:24.890 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T02:51:24.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:24.890 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T02:51:24.918 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:24.916+0000 7fbe99e10040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T02:51:24.949 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:24.946+0000 7f5858a14040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:51:24.964 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T02:51:24.964 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:51:25.001 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:25.000+0000 7f5858a14040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T02:51:25.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T02:51:25.019 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:51:25.020 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:51:25.072 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T02:51:25.072 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T02:51:25.072 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:51:25.082 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:25.081+0000 7fbe99e10040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:51:25.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T02:51:25.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:51:25.127 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:25.124+0000 7f5858a14040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T02:51:25.136 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:25.131+0000 7fbe99e10040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T02:51:25.259 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:25.257+0000 7fbe99e10040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T02:51:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.336+0000 7fb9eaecc640 1 Processor -- start 2023-12-01T02:51:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.337+0000 7fb9eaecc640 1 -- start start 2023-12-01T02:51:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.338+0000 7fb9eaecc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb9e415d7b0 0x7fb9e415db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.338+0000 7fb9eaecc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb9e4160670 0x7fb9e40cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.338+0000 7fb9eaecc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e40cd3f0 con 0x7fb9e40cddc0 2023-12-01T02:51:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.338+0000 7fb9eaecc640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e40cd0f0 con 0x7fb9e4160670 2023-12-01T02:51:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.338+0000 7fb9eaecc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e4160ec0 con 0x7fb9e415d7b0 2023-12-01T02:51:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.338+0000 7fb9e96c9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb9e4160670 0x7fb9e40cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:25.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.338+0000 7fb9ea6cb640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb9e40cddc0 0x7fb9e40d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40232/0 (socket says 172.21.15.115:40232) 2023-12-01T02:51:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.339+0000 7fb9ea6cb640 1 -- 172.21.15.115:0/3090005119 learned_addr learned my addr 172.21.15.115:0/3090005119 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.339+0000 7fb9e9eca640 1 --2- 172.21.15.115:0/3090005119 >> v2:172.21.15.142:3300/0 conn(0x7fb9e415d7b0 0x7fb9e415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.339+0000 7fb9e96c9640 1 -- 172.21.15.115:0/3090005119 >> v2:172.21.15.142:3300/0 conn(0x7fb9e415d7b0 msgr2=0x7fb9e415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.339+0000 7fb9e96c9640 1 --2- 172.21.15.115:0/3090005119 >> v2:172.21.15.142:3300/0 conn(0x7fb9e415d7b0 0x7fb9e415db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.339+0000 7fb9e96c9640 1 -- 172.21.15.115:0/3090005119 >> v1:172.21.15.115:6789/0 conn(0x7fb9e40cddc0 legacy=0x7fb9e40d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.339+0000 7fb9e96c9640 1 -- 172.21.15.115:0/3090005119 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e41611a0 con 0x7fb9e4160670 2023-12-01T02:51:25.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.339+0000 7fb9e8ec8640 1 -- 172.21.15.115:0/3090005119 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115977122 0 0) 0x7fb9e40cd3f0 con 0x7fb9e40cddc0 2023-12-01T02:51:25.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.341+0000 7fb9e96c9640 1 --2- 172.21.15.115:0/3090005119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e4160670 0x7fb9e40cd880 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40720e0 tx=0x7fb9d402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b784e0ae8b650760 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:25.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.341+0000 7fb9e8ec8640 1 -- 172.21.15.115:0/3090005119 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4030610 con 0x7fb9e4160670 2023-12-01T02:51:25.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.341+0000 7fb9e8ec8640 1 -- 172.21.15.115:0/3090005119 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4030770 con 0x7fb9e4160670 2023-12-01T02:51:25.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.341+0000 7fb9e8ec8640 1 -- 172.21.15.115:0/3090005119 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4030b00 con 0x7fb9e4160670 2023-12-01T02:51:25.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.342+0000 7fb9eaecc640 1 -- 172.21.15.115:0/3090005119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e4160670 msgr2=0x7fb9e40cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.342+0000 7fb9eaecc640 1 --2- 172.21.15.115:0/3090005119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e4160670 0x7fb9e40cd880 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40720e0 tx=0x7fb9d402b960 comp rx=0 tx=0).stop 2023-12-01T02:51:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.342+0000 7fb9eaecc640 1 -- 172.21.15.115:0/3090005119 shutdown_connections 2023-12-01T02:51:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.342+0000 7fb9eaecc640 1 --2- 172.21.15.115:0/3090005119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e4160670 0x7fb9e40cd880 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.342+0000 7fb9eaecc640 1 --2- 172.21.15.115:0/3090005119 >> v2:172.21.15.142:3300/0 conn(0x7fb9e415d7b0 0x7fb9e415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.342+0000 7fb9eaecc640 1 -- 172.21.15.115:0/3090005119 >> 172.21.15.115:0/3090005119 conn(0x7fb9e406a0e0 msgr2=0x7fb9e406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.342+0000 7fb9eaecc640 1 -- 172.21.15.115:0/3090005119 shutdown_connections 2023-12-01T02:51:25.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.342+0000 7fb9eaecc640 1 -- 172.21.15.115:0/3090005119 wait complete. 2023-12-01T02:51:25.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.344+0000 7fb9eaecc640 1 Processor -- start 2023-12-01T02:51:25.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.345+0000 7fb9eaecc640 1 -- start start 2023-12-01T02:51:25.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.346+0000 7fb9eaecc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e40cddc0 0x7fb9e41f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.346+0000 7fb9eaecc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e415d7b0 0x7fb9e40d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.346+0000 7fb9eaecc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9e4160670 0x7fb9e40d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.346+0000 7fb9eaecc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41f3640 con 0x7fb9e415d7b0 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.346+0000 7fb9eaecc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41f2d40 con 0x7fb9e40cddc0 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.346+0000 7fb9eaecc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e41f3020 con 0x7fb9e4160670 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.346+0000 7fb9e96c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e415d7b0 0x7fb9e40d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9e96c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e415d7b0 0x7fb9e40d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41974/0 (socket says 172.21.15.115:41974) 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9e96c9640 1 -- 172.21.15.115:0/2035433280 learned_addr learned my addr 172.21.15.115:0/2035433280 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9e9eca640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e40cddc0 0x7fb9e41f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9ea6cb640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9e4160670 0x7fb9e40d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9e96c9640 1 -- 172.21.15.115:0/2035433280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9e4160670 msgr2=0x7fb9e40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9e96c9640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9e4160670 0x7fb9e40d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9e96c9640 1 -- 172.21.15.115:0/2035433280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e40cddc0 msgr2=0x7fb9e41f2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9e96c9640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e40cddc0 0x7fb9e41f2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9e96c9640 1 -- 172.21.15.115:0/2035433280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4009400 con 0x7fb9e415d7b0 2023-12-01T02:51:25.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9ea6cb640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9e4160670 0x7fb9e40d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:25.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.347+0000 7fb9e96c9640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e415d7b0 0x7fb9e40d05d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009890 tx=0x7fb9d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:25.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.348+0000 7fb9da7fc640 1 -- 172.21.15.115:0/2035433280 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d403e070 con 0x7fb9e415d7b0 2023-12-01T02:51:25.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.348+0000 7fb9da7fc640 1 -- 172.21.15.115:0/2035433280 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4006e10 con 0x7fb9e415d7b0 2023-12-01T02:51:25.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.348+0000 7fb9da7fc640 1 -- 172.21.15.115:0/2035433280 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4039c10 con 0x7fb9e415d7b0 2023-12-01T02:51:25.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.348+0000 7fb9eaecc640 1 -- 172.21.15.115:0/2035433280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e41f3300 con 0x7fb9e415d7b0 2023-12-01T02:51:25.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.348+0000 7fb9eaecc640 1 -- 172.21.15.115:0/2035433280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e41f29c0 con 0x7fb9e415d7b0 2023-12-01T02:51:25.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.349+0000 7fb9eaecc640 1 -- 172.21.15.115:0/2035433280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e41f3b20 con 0x7fb9e415d7b0 2023-12-01T02:51:25.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.350+0000 7fb9da7fc640 1 -- 172.21.15.115:0/2035433280 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 81) v1 ==== 88721+0+0 (secure 0 0 0) 0x7fb9d40423f0 con 0x7fb9e415d7b0 2023-12-01T02:51:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.351+0000 7fb9da7fc640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9c806a550 0x7fb9c806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.351+0000 7fb9da7fc640 1 -- 172.21.15.115:0/2035433280 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6199+0+0 (secure 0 0 0) 0x7fb9d40aabd0 con 0x7fb9e415d7b0 2023-12-01T02:51:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.351+0000 7fb9e9eca640 1 -- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9c806a550 msgr2=0x7fb9c806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:25.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.351+0000 7fb9e9eca640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9c806a550 0x7fb9c806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:25.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.355+0000 7fb9da7fc640 1 -- 172.21.15.115:0/2035433280 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb9e41f3b20 con 0x7fb9e415d7b0 2023-12-01T02:51:25.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.552+0000 7fb9e9eca640 1 -- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9c806a550 msgr2=0x7fb9c806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:25.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.552+0000 7fb9e9eca640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9c806a550 0x7fb9c806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:51:25.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.595+0000 7fb9eaecc640 1 -- 172.21.15.115:0/2035433280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9e41f3b20 con 0x7fb9e415d7b0 2023-12-01T02:51:25.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.596+0000 7fb9da7fc640 1 -- 172.21.15.115:0/2035433280 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fb9e41f3b20 con 0x7fb9e415d7b0 2023-12-01T02:51:25.598 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:25.598 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3556763477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3556763477},{"type":"v1","addr":"172.21.15.142:6833","nonce":3556763477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3661095418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3661095418},{"type":"v1","addr":"172.21.15.142:6835","nonce":3661095418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3388585128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3388585128},{"type":"v1","addr":"172.21.15.115:6835","nonce":3388585128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:55:25.253838+0000","modified":"2023-12-01T01:55:28.264944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3894452047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3894452047},{"type":"v1","addr":"172.21.15.115:6837","nonce":3894452047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-01T02:51:25.598 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 63 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.598+0000 7fb9eaecc640 1 -- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9c806a550 msgr2=0x7fb9c806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.598+0000 7fb9eaecc640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9c806a550 0x7fb9c806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.598+0000 7fb9eaecc640 1 -- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e415d7b0 msgr2=0x7fb9e40d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.599+0000 7fb9eaecc640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e415d7b0 0x7fb9e40d05d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009890 tx=0x7fb9d40046c0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.599+0000 7fb9eaecc640 1 -- 172.21.15.115:0/2035433280 shutdown_connections 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.599+0000 7fb9eaecc640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9c806a550 0x7fb9c806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.599+0000 7fb9eaecc640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9e4160670 0x7fb9e40d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.599+0000 7fb9eaecc640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9e415d7b0 0x7fb9e40d05d0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.599+0000 7fb9eaecc640 1 --2- 172.21.15.115:0/2035433280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9e40cddc0 0x7fb9e41f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.599+0000 7fb9eaecc640 1 -- 172.21.15.115:0/2035433280 >> 172.21.15.115:0/2035433280 conn(0x7fb9e406a0e0 msgr2=0x7fb9e406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.599+0000 7fb9eaecc640 1 -- 172.21.15.115:0/2035433280 shutdown_connections 2023-12-01T02:51:25.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.599+0000 7fb9eaecc640 1 -- 172.21.15.115:0/2035433280 wait complete. 2023-12-01T02:51:25.617 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:51:25.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.808+0000 7f2da7432640 1 Processor -- start 2023-12-01T02:51:25.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.808+0000 7f2da7432640 1 -- start start 2023-12-01T02:51:25.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.810+0000 7f2da7432640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2da015dbf0 0x7f2da00cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.810+0000 7f2da7432640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da015dfd0 0x7f2da00cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.810+0000 7f2da7432640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da0160b20 con 0x7f2da00ce170 2023-12-01T02:51:25.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.810+0000 7f2da7432640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da0160dc0 con 0x7f2da015dbf0 2023-12-01T02:51:25.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.810+0000 7f2da7432640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01610a0 con 0x7f2da015dfd0 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.810+0000 7f2da59cf640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2da00ce170 0x7f2da00b7ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40236/0 (socket says 172.21.15.115:40236) 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.810+0000 7f2da59cf640 1 -- 172.21.15.115:0/933804688 learned_addr learned my addr 172.21.15.115:0/933804688 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.810+0000 7f2da51ce640 1 --2- 172.21.15.115:0/933804688 >> v2:172.21.15.142:3300/0 conn(0x7f2da015dbf0 0x7f2da00cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.810+0000 7f2da49cd640 1 --2- 172.21.15.115:0/933804688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da015dfd0 0x7f2da00cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.811+0000 7f2d97fff640 1 -- 172.21.15.115:0/933804688 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833694119 0 0) 0x7f2da0160b20 con 0x7f2da00ce170 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.811+0000 7f2d97fff640 1 -- 172.21.15.115:0/933804688 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d8c003520 con 0x7f2da00ce170 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.811+0000 7f2da51ce640 1 -- 172.21.15.115:0/933804688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da015dfd0 msgr2=0x7f2da00cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.811+0000 7f2d97fff640 1 -- 172.21.15.115:0/933804688 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3747373742 0 0) 0x7f2d8c003520 con 0x7f2da00ce170 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.811+0000 7f2da51ce640 1 --2- 172.21.15.115:0/933804688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da015dfd0 0x7f2da00cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.811+0000 7f2da51ce640 1 -- 172.21.15.115:0/933804688 >> v1:172.21.15.115:6789/0 conn(0x7f2da00ce170 legacy=0x7f2da00b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:25.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.811+0000 7f2da51ce640 1 -- 172.21.15.115:0/933804688 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da0161380 con 0x7f2da015dbf0 2023-12-01T02:51:25.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.812+0000 7f2da51ce640 1 --2- 172.21.15.115:0/933804688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015dbf0 0x7f2da00cdc30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0097a0 tx=0x7f2d9c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789f031b3c8fbc6c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:25.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.813+0000 7f2d97fff640 1 -- 172.21.15.115:0/933804688 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980019f0 con 0x7f2da015dbf0 2023-12-01T02:51:25.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.813+0000 7f2d97fff640 1 -- 172.21.15.115:0/933804688 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c02e050 con 0x7f2da015dbf0 2023-12-01T02:51:25.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.813+0000 7f2d97fff640 1 -- 172.21.15.115:0/933804688 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c030710 con 0x7f2da015dbf0 2023-12-01T02:51:25.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.813+0000 7f2da7432640 1 -- 172.21.15.115:0/933804688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015dbf0 msgr2=0x7f2da00cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:25.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.813+0000 7f2da7432640 1 --2- 172.21.15.115:0/933804688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015dbf0 0x7f2da00cdc30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0097a0 tx=0x7f2d9c02b790 comp rx=0 tx=0).stop 2023-12-01T02:51:25.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.813+0000 7f2da7432640 1 -- 172.21.15.115:0/933804688 shutdown_connections 2023-12-01T02:51:25.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.813+0000 7f2da7432640 1 --2- 172.21.15.115:0/933804688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da015dfd0 0x7f2da00cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.813+0000 7f2da7432640 1 --2- 172.21.15.115:0/933804688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015dbf0 0x7f2da00cdc30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.813+0000 7f2da7432640 1 -- 172.21.15.115:0/933804688 >> 172.21.15.115:0/933804688 conn(0x7f2da0065960 msgr2=0x7f2da0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:25.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.814+0000 7f2da7432640 1 -- 172.21.15.115:0/933804688 shutdown_connections 2023-12-01T02:51:25.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.814+0000 7f2da7432640 1 -- 172.21.15.115:0/933804688 wait complete. 2023-12-01T02:51:25.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.815+0000 7f2da7432640 1 Processor -- start 2023-12-01T02:51:25.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.815+0000 7f2da7432640 1 -- start start 2023-12-01T02:51:25.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.816+0000 7f2da7432640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00ce170 0x7f2da00c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.816+0000 7f2da7432640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015dbf0 0x7f2da00cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.816+0000 7f2da7432640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015dfd0 0x7f2da00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.816+0000 7f2da7432640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0158b90 con 0x7f2da015dfd0 2023-12-01T02:51:25.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.816+0000 7f2da7432640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0148260 con 0x7f2da015dbf0 2023-12-01T02:51:25.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.816+0000 7f2da7432640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00c9690 con 0x7f2da00ce170 2023-12-01T02:51:25.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.816+0000 7f2da59cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015dfd0 0x7f2da00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:25.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.817+0000 7f2da59cf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015dfd0 0x7f2da00cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41986/0 (socket says 172.21.15.115:41986) 2023-12-01T02:51:25.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.817+0000 7f2da59cf640 1 -- 172.21.15.115:0/784789033 learned_addr learned my addr 172.21.15.115:0/784789033 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.817+0000 7f2da49cd640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015dbf0 0x7f2da00cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.817+0000 7f2da51ce640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00ce170 0x7f2da00c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.817+0000 7f2da59cf640 1 -- 172.21.15.115:0/784789033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00ce170 msgr2=0x7f2da00c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.817+0000 7f2da59cf640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00ce170 0x7f2da00c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.817+0000 7f2da59cf640 1 -- 172.21.15.115:0/784789033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015dbf0 msgr2=0x7f2da00cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.817+0000 7f2da59cf640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015dbf0 0x7f2da00cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.817+0000 7f2da59cf640 1 -- 172.21.15.115:0/784789033 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c009400 con 0x7f2da015dfd0 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.817+0000 7f2da59cf640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015dfd0 0x7f2da00cc4a0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f2d98001f70 tx=0x7f2d9800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.818+0000 7f2d95ffb640 1 -- 172.21.15.115:0/784789033 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98012400 con 0x7f2da015dfd0 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.818+0000 7f2d95ffb640 1 -- 172.21.15.115:0/784789033 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98012560 con 0x7f2da015dfd0 2023-12-01T02:51:25.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.818+0000 7f2da7432640 1 -- 172.21.15.115:0/784789033 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da01588b0 con 0x7f2da015dfd0 2023-12-01T02:51:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.818+0000 7f2d95ffb640 1 -- 172.21.15.115:0/784789033 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98012910 con 0x7f2da015dfd0 2023-12-01T02:51:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.818+0000 7f2da7432640 1 -- 172.21.15.115:0/784789033 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da01451f0 con 0x7f2da015dfd0 2023-12-01T02:51:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.819+0000 7f2da7432640 1 -- 172.21.15.115:0/784789033 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da0144d70 con 0x7f2da015dfd0 2023-12-01T02:51:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.820+0000 7f2d95ffb640 1 -- 172.21.15.115:0/784789033 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 81) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f2d98012a70 con 0x7f2da015dfd0 2023-12-01T02:51:25.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.821+0000 7f2d95ffb640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a550 0x7f2d8406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:25.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.821+0000 7f2d95ffb640 1 -- 172.21.15.115:0/784789033 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6199+0+0 (secure 0 0 0) 0x7f2d980894a0 con 0x7f2da015dfd0 2023-12-01T02:51:25.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.821+0000 7f2da51ce640 1 -- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a550 msgr2=0x7f2d8406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:25.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.821+0000 7f2da51ce640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a550 0x7f2d8406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:25.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:25.829+0000 7f2d95ffb640 1 -- 172.21.15.115:0/784789033 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2da0144d70 con 0x7f2da015dfd0 2023-12-01T02:51:25.945 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:25.943+0000 7f5858a14040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T02:51:26.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.022+0000 7f2da51ce640 1 -- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a550 msgr2=0x7f2d8406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:26.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.022+0000 7f2da51ce640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a550 0x7f2d8406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:51:26.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.046+0000 7f2da7432640 1 -- 172.21.15.115:0/784789033 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2da0144d70 con 0x7f2da015dfd0 2023-12-01T02:51:26.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.049+0000 7f2d95ffb640 1 -- 172.21.15.115:0/784789033 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16912 (secure 0 0 0) 0x7f2da0144d70 con 0x7f2da015dfd0 2023-12-01T02:51:26.051 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:26.051 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":149,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:51:22.173247+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:55:22.818353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-01T01:55:24.272936+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":149,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T01:55:38.761001+0000","172.21.15.115:6822/88317":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3362083032":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3703392460":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3403958692":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/671134594":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3600416058":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/2599234561":"2023-12-02T01:55:38.761001+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":42,"length":15},{"begin":98,"length":15}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:51:26.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.051+0000 7f2da7432640 1 -- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a550 msgr2=0x7f2d8406ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.051+0000 7f2da7432640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a550 0x7f2d8406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.051+0000 7f2da7432640 1 -- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015dfd0 msgr2=0x7f2da00cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.051+0000 7f2da7432640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015dfd0 0x7f2da00cc4a0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f2d98001f70 tx=0x7f2d9800a430 comp rx=0 tx=0).stop 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.052+0000 7f2da7432640 1 -- 172.21.15.115:0/784789033 shutdown_connections 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.052+0000 7f2da7432640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a550 0x7f2d8406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.052+0000 7f2da7432640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015dfd0 0x7f2da00cc4a0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.052+0000 7f2da7432640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015dbf0 0x7f2da00cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.052+0000 7f2da7432640 1 --2- 172.21.15.115:0/784789033 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00ce170 0x7f2da00c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.052+0000 7f2da7432640 1 -- 172.21.15.115:0/784789033 >> 172.21.15.115:0/784789033 conn(0x7f2da0065960 msgr2=0x7f2da00b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.052+0000 7f2da7432640 1 -- 172.21.15.115:0/784789033 shutdown_connections 2023-12-01T02:51:26.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.052+0000 7f2da7432640 1 -- 172.21.15.115:0/784789033 wait complete. 2023-12-01T02:51:26.070 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:55:22.818353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:55:24.272936+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 149, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T02:51:26.071 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T02:51:26.120 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.119+0000 7fbe99e10040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T02:51:26.243 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:26.241+0000 7f5858a14040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T02:51:26.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.273+0000 7f7d33022640 1 Processor -- start 2023-12-01T02:51:26.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.273+0000 7f7d33022640 1 -- start start 2023-12-01T02:51:26.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.275+0000 7f7d33022640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7d2c153c30 0x7f7d2c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:26.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.275+0000 7f7d33022640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2c0cdcd0 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.275+0000 7f7d33022640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2c161830 con 0x7f7d2c154010 2023-12-01T02:51:26.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.275+0000 7f7d33022640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c0cd3a0 con 0x7f7d2c153c30 2023-12-01T02:51:26.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.275+0000 7f7d32020640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7d2c153c30 0x7f7d2c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:26.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.275+0000 7f7d32821640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7d2c0cfb80 0x7f7d2c160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40244/0 (socket says 172.21.15.115:40244) 2023-12-01T02:51:26.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.275+0000 7f7d32821640 1 -- 172.21.15.115:0/2451235953 learned_addr learned my addr 172.21.15.115:0/2451235953 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.275+0000 7f7d32020640 1 -- 172.21.15.115:0/2451235953 >> v1:172.21.15.142:6789/0 conn(0x7f7d2c154010 legacy=0x7f7d2c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.275+0000 7f7d32020640 1 -- 172.21.15.115:0/2451235953 >> v1:172.21.15.115:6789/0 conn(0x7f7d2c0cfb80 legacy=0x7f7d2c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.275+0000 7f7d32020640 1 -- 172.21.15.115:0/2451235953 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c0cd680 con 0x7f7d2c153c30 2023-12-01T02:51:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.277+0000 7f7d32020640 1 --2- 172.21.15.115:0/2451235953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c153c30 0x7f7d2c0ce160 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097a0 tx=0x7f7d2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12b2911ac818ca9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.277+0000 7f7d3101e640 1 -- 172.21.15.115:0/2451235953 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28031830 con 0x7f7d2c153c30 2023-12-01T02:51:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.277+0000 7f7d3101e640 1 -- 172.21.15.115:0/2451235953 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d28031990 con 0x7f7d2c153c30 2023-12-01T02:51:26.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.277+0000 7f7d3101e640 1 -- 172.21.15.115:0/2451235953 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28031d20 con 0x7f7d2c153c30 2023-12-01T02:51:26.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.277+0000 7f7d33022640 1 -- 172.21.15.115:0/2451235953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c153c30 msgr2=0x7f7d2c0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:26.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.277+0000 7f7d33022640 1 --2- 172.21.15.115:0/2451235953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c153c30 0x7f7d2c0ce160 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097a0 tx=0x7f7d2802b8d0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.278+0000 7f7d33022640 1 -- 172.21.15.115:0/2451235953 shutdown_connections 2023-12-01T02:51:26.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.278+0000 7f7d33022640 1 --2- 172.21.15.115:0/2451235953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c153c30 0x7f7d2c0ce160 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.278+0000 7f7d33022640 1 -- 172.21.15.115:0/2451235953 >> 172.21.15.115:0/2451235953 conn(0x7f7d2c06a0b0 msgr2=0x7f7d2c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:26.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.278+0000 7f7d33022640 1 -- 172.21.15.115:0/2451235953 shutdown_connections 2023-12-01T02:51:26.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.278+0000 7f7d33022640 1 -- 172.21.15.115:0/2451235953 wait complete. 2023-12-01T02:51:26.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.280+0000 7f7d33022640 1 Processor -- start 2023-12-01T02:51:26.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.280+0000 7f7d33022640 1 -- start start 2023-12-01T02:51:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.281+0000 7f7d33022640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0cfb80 0x7f7d2c0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.281+0000 7f7d33022640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d2c153c30 0x7f7d2c1f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.281+0000 7f7d33022640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d2c154010 0x7f7d2c0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.281+0000 7f7d33022640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c1464d0 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.281+0000 7f7d33022640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c157cc0 con 0x7f7d2c154010 2023-12-01T02:51:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.281+0000 7f7d33022640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c157fa0 con 0x7f7d2c153c30 2023-12-01T02:51:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.281+0000 7f7d32020640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0cfb80 0x7f7d2c0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.281+0000 7f7d32020640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0cfb80 0x7f7d2c0c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:41996/0 (socket says 172.21.15.115:41996) 2023-12-01T02:51:26.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.281+0000 7f7d32020640 1 -- 172.21.15.115:0/1521566247 learned_addr learned my addr 172.21.15.115:0/1521566247 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:26.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d32821640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d2c154010 0x7f7d2c0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d32020640 1 -- 172.21.15.115:0/1521566247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d2c153c30 msgr2=0x7f7d2c1f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d32020640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d2c153c30 0x7f7d2c1f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d32020640 1 -- 172.21.15.115:0/1521566247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d2c154010 msgr2=0x7f7d2c0d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d32020640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d2c154010 0x7f7d2c0d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d32020640 1 -- 172.21.15.115:0/1521566247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d28009400 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d32020640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0cfb80 0x7f7d2c0c8fe0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7d280098c0 tx=0x7f7d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d22ffd640 1 -- 172.21.15.115:0/1521566247 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2803ba70 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d22ffd640 1 -- 172.21.15.115:0/1521566247 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2803bbf0 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d33022640 1 -- 172.21.15.115:0/1521566247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c158280 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.282+0000 7f7d22ffd640 1 -- 172.21.15.115:0/1521566247 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d280453f0 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.283+0000 7f7d33022640 1 -- 172.21.15.115:0/1521566247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c158840 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.284+0000 7f7d33022640 1 -- 172.21.15.115:0/1521566247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c0618d0 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.285+0000 7f7d22ffd640 1 -- 172.21.15.115:0/1521566247 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 81) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f7d2804e450 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.286+0000 7f7d22ffd640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d1006a550 0x7f7d1006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:26.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.286+0000 7f7d22ffd640 1 -- 172.21.15.115:0/1521566247 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6199+0+0 (secure 0 0 0) 0x7f7d280ab160 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.286+0000 7f7d3181f640 1 -- 172.21.15.115:0/1521566247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d1006a550 msgr2=0x7f7d1006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:26.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.286+0000 7f7d3181f640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d1006a550 0x7f7d1006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:26.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.290+0000 7f7d22ffd640 1 -- 172.21.15.115:0/1521566247 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7d28076460 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.301 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:26.299+0000 7f5858a14040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T02:51:26.415 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:26.413+0000 7f5858a14040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T02:51:26.432 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.430+0000 7fbe99e10040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T02:51:26.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.479+0000 7f7d33022640 1 -- 172.21.15.115:0/1521566247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d2c15cea0 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.480+0000 7f7d22ffd640 1 -- 172.21.15.115:0/1521566247 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d28079a80 con 0x7f7d2c0cfb80 2023-12-01T02:51:26.483 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T02:51:26.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.482+0000 7f7d33022640 1 -- 172.21.15.115:0/1521566247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d1006a550 msgr2=0x7f7d1006ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:26.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.482+0000 7f7d33022640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d1006a550 0x7f7d1006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.482+0000 7f7d33022640 1 -- 172.21.15.115:0/1521566247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0cfb80 msgr2=0x7f7d2c0c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:26.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.482+0000 7f7d33022640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0cfb80 0x7f7d2c0c8fe0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7d280098c0 tx=0x7f7d280046c0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.482+0000 7f7d33022640 1 -- 172.21.15.115:0/1521566247 shutdown_connections 2023-12-01T02:51:26.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.482+0000 7f7d33022640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7d1006a550 0x7f7d1006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.482+0000 7f7d33022640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d2c153c30 0x7f7d2c1f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.482+0000 7f7d33022640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d2c154010 0x7f7d2c0d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.482+0000 7f7d33022640 1 --2- 172.21.15.115:0/1521566247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0cfb80 0x7f7d2c0c8fe0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.482+0000 7f7d33022640 1 -- 172.21.15.115:0/1521566247 >> 172.21.15.115:0/1521566247 conn(0x7f7d2c06a0b0 msgr2=0x7f7d2c15e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:26.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.483+0000 7f7d33022640 1 -- 172.21.15.115:0/1521566247 shutdown_connections 2023-12-01T02:51:26.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.483+0000 7f7d33022640 1 -- 172.21.15.115:0/1521566247 wait complete. 2023-12-01T02:51:26.492 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.490+0000 7fbe99e10040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T02:51:26.503 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T02:51:26.612 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.611+0000 7fbe99e10040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T02:51:26.700 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.698+0000 7fbe1bdb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:26.700 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.698+0000 7fbe1bdb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:26.700 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.698+0000 7fbe1bdb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:26.700 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.698+0000 7fbe1bdb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:26.700 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.698+0000 7fbe1bdb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:26.700 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.698+0000 7fbe1edfb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:26.700 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.698+0000 7fbe1edfb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:26.700 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.698+0000 7fbe1edfb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:26.700 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.698+0000 7fbe1edfb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:26.700 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:26.698+0000 7fbe1edfb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:26.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.708+0000 7f3ebcc43640 1 Processor -- start 2023-12-01T02:51:26.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.708+0000 7f3ebcc43640 1 -- start start 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3ebcc43640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3eb80cfd60 0x7f3eb80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3ebcc43640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb80b5fa0 0x7f3eb8161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3ebcc43640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb80ce5e0 con 0x7f3eb815cd90 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3ebcc43640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb80d0ec0 con 0x7f3eb80cfd60 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3ebcc43640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb8161950 con 0x7f3eb80b5fa0 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3eb5d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3eb80cfd60 0x7f3eb80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3eb5d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3eb80cfd60 0x7f3eb80cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42110/0 (socket says 172.21.15.115:42110) 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3eb5d9b640 1 -- 172.21.15.115:0/3359379281 learned_addr learned my addr 172.21.15.115:0/3359379281 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3eb5d9b640 1 -- 172.21.15.115:0/3359379281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb80b5fa0 msgr2=0x7f3eb8161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3eb5d9b640 1 --2- 172.21.15.115:0/3359379281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb80b5fa0 0x7f3eb8161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3eb5d9b640 1 -- 172.21.15.115:0/3359379281 >> v1:172.21.15.142:6789/0 conn(0x7f3eb815cd90 legacy=0x7f3eb80cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:26.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.710+0000 7f3eb5d9b640 1 -- 172.21.15.115:0/3359379281 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb80ce300 con 0x7f3eb80cfd60 2023-12-01T02:51:26.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.711+0000 7f3eb5d9b640 1 --2- 172.21.15.115:0/3359379281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 0x7f3eb80cdd20 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009870 tx=0x7f3eac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44775e76e00f5a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:26.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.712+0000 7f3eb559a640 1 -- 172.21.15.115:0/3359379281 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac031640 con 0x7f3eb80cfd60 2023-12-01T02:51:26.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.712+0000 7f3eb559a640 1 -- 172.21.15.115:0/3359379281 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac0317a0 con 0x7f3eb80cfd60 2023-12-01T02:51:26.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.712+0000 7f3eb559a640 1 -- 172.21.15.115:0/3359379281 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac031b30 con 0x7f3eb80cfd60 2023-12-01T02:51:26.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.712+0000 7f3ebcc43640 1 -- 172.21.15.115:0/3359379281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 msgr2=0x7f3eb80cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.712+0000 7f3ebcc43640 1 --2- 172.21.15.115:0/3359379281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 0x7f3eb80cdd20 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009870 tx=0x7f3eac02b8f0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.712+0000 7f3ebcc43640 1 -- 172.21.15.115:0/3359379281 shutdown_connections 2023-12-01T02:51:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.712+0000 7f3ebcc43640 1 --2- 172.21.15.115:0/3359379281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb80b5fa0 0x7f3eb8161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.712+0000 7f3ebcc43640 1 --2- 172.21.15.115:0/3359379281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 0x7f3eb80cdd20 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.712+0000 7f3ebcc43640 1 -- 172.21.15.115:0/3359379281 >> 172.21.15.115:0/3359379281 conn(0x7f3eb8065960 msgr2=0x7f3eb8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:26.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.713+0000 7f3ebcc43640 1 -- 172.21.15.115:0/3359379281 shutdown_connections 2023-12-01T02:51:26.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.713+0000 7f3ebcc43640 1 -- 172.21.15.115:0/3359379281 wait complete. 2023-12-01T02:51:26.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.714+0000 7f3ebcc43640 1 Processor -- start 2023-12-01T02:51:26.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.714+0000 7f3ebcc43640 1 -- start start 2023-12-01T02:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.715+0000 7f3ebcc43640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb80b5fa0 0x7f3eb80cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.715+0000 7f3ebcc43640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 0x7f3eb814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:26.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.715+0000 7f3ebcc43640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eb815cd90 0x7f3eb81f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.715+0000 7f3ebcc43640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb80cc100 con 0x7f3eb80cfd60 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.715+0000 7f3ebcc43640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb80cc610 con 0x7f3eb815cd90 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.715+0000 7f3ebcc43640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb80cc330 con 0x7f3eb80b5fa0 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.715+0000 7f3eb5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 0x7f3eb814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.715+0000 7f3eb5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 0x7f3eb814d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42114/0 (socket says 172.21.15.115:42114) 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.715+0000 7f3eb5d9b640 1 -- 172.21.15.115:0/1775926873 learned_addr learned my addr 172.21.15.115:0/1775926873 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.716+0000 7f3eb5d9b640 1 -- 172.21.15.115:0/1775926873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb80b5fa0 msgr2=0x7f3eb80cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.716+0000 7f3eb659c640 1 --2- 172.21.15.115:0/1775926873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb80b5fa0 0x7f3eb80cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.716+0000 7f3eb5d9b640 1 --2- 172.21.15.115:0/1775926873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb80b5fa0 0x7f3eb80cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.716+0000 7f3eb5d9b640 1 -- 172.21.15.115:0/1775926873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eb815cd90 msgr2=0x7f3eb81f9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.716+0000 7f3eb6d9d640 1 --2- 172.21.15.115:0/1775926873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eb815cd90 0x7f3eb81f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.716+0000 7f3eb5d9b640 1 --2- 172.21.15.115:0/1775926873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eb815cd90 0x7f3eb81f9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.716+0000 7f3eb5d9b640 1 -- 172.21.15.115:0/1775926873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac0094d0 con 0x7f3eb80cfd60 2023-12-01T02:51:26.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.716+0000 7f3eb5d9b640 1 --2- 172.21.15.115:0/1775926873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 0x7f3eb814d200 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009870 tx=0x7f3eac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.717+0000 7f3ea6ffd640 1 -- 172.21.15.115:0/1775926873 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac03ac60 con 0x7f3eb80cfd60 2023-12-01T02:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.717+0000 7f3ea6ffd640 1 -- 172.21.15.115:0/1775926873 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac03adc0 con 0x7f3eb80cfd60 2023-12-01T02:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.717+0000 7f3ea6ffd640 1 -- 172.21.15.115:0/1775926873 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac044630 con 0x7f3eb80cfd60 2023-12-01T02:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.717+0000 7f3ebcc43640 1 -- 172.21.15.115:0/1775926873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb8151450 con 0x7f3eb80cfd60 2023-12-01T02:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.717+0000 7f3ebcc43640 1 -- 172.21.15.115:0/1775926873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb8151a10 con 0x7f3eb80cfd60 2023-12-01T02:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.718+0000 7f3ebcc43640 1 -- 172.21.15.115:0/1775926873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb804e890 con 0x7f3eb80cfd60 2023-12-01T02:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.719+0000 7f3ea6ffd640 1 -- 172.21.15.115:0/1775926873 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 82) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f3eac02f050 con 0x7f3eb80cfd60 2023-12-01T02:51:26.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.720+0000 7f3ea6ffd640 1 -- 172.21.15.115:0/1775926873 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6413+0+0 (secure 0 0 0) 0x7f3eac0aa590 con 0x7f3eb80cfd60 2023-12-01T02:51:26.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.724+0000 7f3ea6ffd640 1 -- 172.21.15.115:0/1775926873 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3eac075c20 con 0x7f3eb80cfd60 2023-12-01T02:51:26.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.911+0000 7f3ebcc43640 1 -- 172.21.15.115:0/1775926873 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3eb80ce5e0 con 0x7f3eb80cfd60 2023-12-01T02:51:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.912+0000 7f3ea6ffd640 1 -- 172.21.15.115:0/1775926873 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3eb80ce5e0 con 0x7f3eb80cfd60 2023-12-01T02:51:26.914 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T02:51:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.915+0000 7f3ebcc43640 1 -- 172.21.15.115:0/1775926873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 msgr2=0x7f3eb814d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.915+0000 7f3ebcc43640 1 --2- 172.21.15.115:0/1775926873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 0x7f3eb814d200 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009870 tx=0x7f3eac0046c0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.915+0000 7f3ebcc43640 1 -- 172.21.15.115:0/1775926873 shutdown_connections 2023-12-01T02:51:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.915+0000 7f3ebcc43640 1 --2- 172.21.15.115:0/1775926873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eb815cd90 0x7f3eb81f9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.915+0000 7f3ebcc43640 1 --2- 172.21.15.115:0/1775926873 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3eb80cfd60 0x7f3eb814d200 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.915+0000 7f3ebcc43640 1 --2- 172.21.15.115:0/1775926873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb80b5fa0 0x7f3eb80cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.915+0000 7f3ebcc43640 1 -- 172.21.15.115:0/1775926873 >> 172.21.15.115:0/1775926873 conn(0x7f3eb8065960 msgr2=0x7f3eb8067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.915+0000 7f3ebcc43640 1 -- 172.21.15.115:0/1775926873 shutdown_connections 2023-12-01T02:51:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:26.915+0000 7f3ebcc43640 1 -- 172.21.15.115:0/1775926873 wait complete. 2023-12-01T02:51:26.935 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T02:51:27.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.137+0000 7fd6029e7640 1 Processor -- start 2023-12-01T02:51:27.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.137+0000 7fd6029e7640 1 -- start start 2023-12-01T02:51:27.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd6029e7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd5fc15ac80 0x7fd5fc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd6029e7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc0cc9a0 con 0x7fd5fc0b0e60 2023-12-01T02:51:27.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd6029e7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc0cbd90 con 0x7fd5fc0ce1b0 2023-12-01T02:51:27.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd6029e7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc15b060 con 0x7fd5fc15ac80 2023-12-01T02:51:27.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5fb7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd5fc0b0e60 0x7fd5fc0cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40260/0 (socket says 172.21.15.115:40260) 2023-12-01T02:51:27.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5fb7fe640 1 -- 172.21.15.115:0/3433253056 learned_addr learned my addr 172.21.15.115:0/3433253056 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5fbfff640 1 --2- 172.21.15.115:0/3433253056 >> v2:172.21.15.142:3300/0 conn(0x7fd5fc15ac80 0x7fd5fc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287305327 0 0) 0x7fd5fc0cc9a0 con 0x7fd5fc0b0e60 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e4003520 con 0x7fd5fc0b0e60 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2844536066 0 0) 0x7fd5e4003520 con 0x7fd5fc0b0e60 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 >> v2:172.21.15.142:3300/0 conn(0x7fd5fc15ac80 msgr2=0x7fd5fc0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 --2- 172.21.15.115:0/3433253056 >> v2:172.21.15.142:3300/0 conn(0x7fd5fc15ac80 0x7fd5fc0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 >> v1:172.21.15.142:6789/0 conn(0x7fd5fc0ce1b0 legacy=0x7fd5fc06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc0cc070 con 0x7fd5fc0b0e60 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd5f0001670 con 0x7fd5fc0b0e60 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 >> v1:172.21.15.115:6789/0 conn(0x7fd5fc0b0e60 legacy=0x7fd5fc0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 --2- 172.21.15.115:0/3433253056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e4003970 0x7fd5e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 --2- 172.21.15.115:0/3433253056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e40047c0 0x7fd5e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 --2- 172.21.15.115:0/3433253056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e4005180 0x7fd5e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc15b060 con 0x7fd5e4003970 2023-12-01T02:51:27.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e400c020 con 0x7fd5e40047c0 2023-12-01T02:51:27.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.140+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e400c1e0 con 0x7fd5e4005180 2023-12-01T02:51:27.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.141+0000 7fd5fbfff640 1 --2- 172.21.15.115:0/3433253056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e4003970 0x7fd5e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.141+0000 7fd5fbfff640 1 -- 172.21.15.115:0/3433253056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e4005180 msgr2=0x7fd5e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:27.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.141+0000 7fd5fbfff640 1 --2- 172.21.15.115:0/3433253056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5e4005180 0x7fd5e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.141+0000 7fd5fbfff640 1 -- 172.21.15.115:0/3433253056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e40047c0 msgr2=0x7fd5e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:27.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.141+0000 7fd5fbfff640 1 --2- 172.21.15.115:0/3433253056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e40047c0 0x7fd5e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.141+0000 7fd5fbfff640 1 -- 172.21.15.115:0/3433253056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4003520 con 0x7fd5e4003970 2023-12-01T02:51:27.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.141+0000 7fd600f84640 1 -- 172.21.15.115:0/3433253056 reap_dead start 2023-12-01T02:51:27.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.142+0000 7fd5fbfff640 1 --2- 172.21.15.115:0/3433253056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e4003970 0x7fd5e4003d90 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009ac0 tx=0x7fd5e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde5eb630bedcbd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:27.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.142+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec02c6d0 con 0x7fd5e4003970 2023-12-01T02:51:27.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.142+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec02c830 con 0x7fd5e4003970 2023-12-01T02:51:27.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.142+0000 7fd5faffd640 1 -- 172.21.15.115:0/3433253056 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec02cbc0 con 0x7fd5e4003970 2023-12-01T02:51:27.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.143+0000 7fd6029e7640 1 -- 172.21.15.115:0/3433253056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e4003970 msgr2=0x7fd5e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.143+0000 7fd6029e7640 1 --2- 172.21.15.115:0/3433253056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e4003970 0x7fd5e4003d90 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009ac0 tx=0x7fd5e40057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.143+0000 7fd6029e7640 1 -- 172.21.15.115:0/3433253056 shutdown_connections 2023-12-01T02:51:27.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.143+0000 7fd6029e7640 1 --2- 172.21.15.115:0/3433253056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5e4003970 0x7fd5e4003d90 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.143+0000 7fd6029e7640 1 -- 172.21.15.115:0/3433253056 >> 172.21.15.115:0/3433253056 conn(0x7fd5fc065960 msgr2=0x7fd5fc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:27.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.143+0000 7fd6029e7640 1 -- 172.21.15.115:0/3433253056 shutdown_connections 2023-12-01T02:51:27.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.143+0000 7fd6029e7640 1 -- 172.21.15.115:0/3433253056 wait complete. 2023-12-01T02:51:27.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.145+0000 7fd6029e7640 1 Processor -- start 2023-12-01T02:51:27.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.145+0000 7fd6029e7640 1 -- start start 2023-12-01T02:51:27.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd6029e7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e4003970 0x7fd5fc1f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd6029e7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5fc0b32c0 0x7fd5fc150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd6029e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5fc1522a0 0x7fd5fc152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd6029e7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc1f35a0 con 0x7fd5fc1522a0 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd6029e7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc1f2d00 con 0x7fd5e4003970 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd6029e7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc1f2fe0 con 0x7fd5fc0b32c0 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd600f84640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5fc1522a0 0x7fd5fc152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd600f84640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5fc1522a0 0x7fd5fc152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42128/0 (socket says 172.21.15.115:42128) 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd600f84640 1 -- 172.21.15.115:0/1145770932 learned_addr learned my addr 172.21.15.115:0/1145770932 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd600f84640 1 -- 172.21.15.115:0/1145770932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5fc0b32c0 msgr2=0x7fd5fc150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd600f84640 1 --2- 172.21.15.115:0/1145770932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5fc0b32c0 0x7fd5fc150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd600f84640 1 -- 172.21.15.115:0/1145770932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e4003970 msgr2=0x7fd5fc1f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd600f84640 1 --2- 172.21.15.115:0/1145770932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e4003970 0x7fd5fc1f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd600f84640 1 -- 172.21.15.115:0/1145770932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc1f32c0 con 0x7fd5fc1522a0 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.146+0000 7fd600f84640 1 --2- 172.21.15.115:0/1145770932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5fc1522a0 0x7fd5fc152720 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0ca270 tx=0x7fd5f400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.147+0000 7fd5f8ff9640 1 -- 172.21.15.115:0/1145770932 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4007070 con 0x7fd5fc1522a0 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.147+0000 7fd5f8ff9640 1 -- 172.21.15.115:0/1145770932 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f40071f0 con 0x7fd5fc1522a0 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.147+0000 7fd5f8ff9640 1 -- 172.21.15.115:0/1145770932 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40075a0 con 0x7fd5fc1522a0 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.147+0000 7fd6029e7640 1 -- 172.21.15.115:0/1145770932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc151d80 con 0x7fd5fc1522a0 2023-12-01T02:51:27.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.147+0000 7fd6029e7640 1 -- 172.21.15.115:0/1145770932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc0b3740 con 0x7fd5fc1522a0 2023-12-01T02:51:27.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.148+0000 7fd6029e7640 1 -- 172.21.15.115:0/1145770932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc151be0 con 0x7fd5fc1522a0 2023-12-01T02:51:27.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.149+0000 7fd5f8ff9640 1 -- 172.21.15.115:0/1145770932 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 82) v1 ==== 88318+0+0 (secure 0 0 0) 0x7fd5f4007700 con 0x7fd5fc1522a0 2023-12-01T02:51:27.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.150+0000 7fd5f8ff9640 1 -- 172.21.15.115:0/1145770932 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6413+0+0 (secure 0 0 0) 0x7fd5f40874c0 con 0x7fd5fc1522a0 2023-12-01T02:51:27.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.154+0000 7fd5f8ff9640 1 -- 172.21.15.115:0/1145770932 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd5fc151be0 con 0x7fd5fc1522a0 2023-12-01T02:51:27.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.330+0000 7fd6029e7640 1 -- 172.21.15.115:0/1145770932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5fc151be0 con 0x7fd5fc1522a0 2023-12-01T02:51:27.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.331+0000 7fd5f8ff9640 1 -- 172.21.15.115:0/1145770932 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5fc151be0 con 0x7fd5fc1522a0 2023-12-01T02:51:27.333 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T02:51:27.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.333+0000 7fd6029e7640 1 -- 172.21.15.115:0/1145770932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5fc1522a0 msgr2=0x7fd5fc152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.333+0000 7fd6029e7640 1 --2- 172.21.15.115:0/1145770932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5fc1522a0 0x7fd5fc152720 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0ca270 tx=0x7fd5f400c960 comp rx=0 tx=0).stop 2023-12-01T02:51:27.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.334+0000 7fd6029e7640 1 -- 172.21.15.115:0/1145770932 shutdown_connections 2023-12-01T02:51:27.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.334+0000 7fd6029e7640 1 --2- 172.21.15.115:0/1145770932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5fc0b32c0 0x7fd5fc150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.334+0000 7fd6029e7640 1 --2- 172.21.15.115:0/1145770932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5fc1522a0 0x7fd5fc152720 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.334+0000 7fd6029e7640 1 --2- 172.21.15.115:0/1145770932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5e4003970 0x7fd5fc1f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.334+0000 7fd6029e7640 1 -- 172.21.15.115:0/1145770932 >> 172.21.15.115:0/1145770932 conn(0x7fd5fc065960 msgr2=0x7fd5fc15b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:27.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.334+0000 7fd6029e7640 1 -- 172.21.15.115:0/1145770932 shutdown_connections 2023-12-01T02:51:27.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.334+0000 7fd6029e7640 1 -- 172.21.15.115:0/1145770932 wait complete. 2023-12-01T02:51:27.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:51:27.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.545+0000 7f220cf4f640 1 Processor -- start 2023-12-01T02:51:27.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.545+0000 7f220cf4f640 1 -- start start 2023-12-01T02:51:27.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f220cf4f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f22081584b0 0x7f22080cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f220cf4f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f220815afe0 0x7f220806b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f220cf4f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22080cc180 con 0x7f220806b7b0 2023-12-01T02:51:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f220cf4f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220806bc40 con 0x7f22081584b0 2023-12-01T02:51:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f220cf4f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220815b550 con 0x7f220815afe0 2023-12-01T02:51:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f2207fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f220806b7b0 0x7f220815e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40276/0 (socket says 172.21.15.115:40276) 2023-12-01T02:51:27.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f2207fff640 1 -- 172.21.15.115:0/3493599511 learned_addr learned my addr 172.21.15.115:0/3493599511 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956709808 0 0) 0x7f22080cc180 con 0x7f220806b7b0 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f22077fe640 1 --2- 172.21.15.115:0/3493599511 >> v2:172.21.15.142:3300/0 conn(0x7f22081584b0 0x7f22080cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f4003520 con 0x7f220806b7b0 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.546+0000 7f21fffff640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f220815afe0 0x7f220806b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3708498131 0 0) 0x7f21f4003520 con 0x7f220806b7b0 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f220815afe0 msgr2=0x7f220806b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f220815afe0 0x7f220806b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 >> v2:172.21.15.142:3300/0 conn(0x7f22081584b0 msgr2=0x7f22080cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 --2- 172.21.15.115:0/3493599511 >> v2:172.21.15.142:3300/0 conn(0x7f22081584b0 0x7f22080cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220815b830 con 0x7f220806b7b0 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f22000019f0 con 0x7f220806b7b0 2023-12-01T02:51:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 >> v1:172.21.15.115:6789/0 conn(0x7f220806b7b0 legacy=0x7f220815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21f40039e0 0x7f21f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f220815afe0 0x7f21f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4004d00 0x7f21f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220806bc40 con 0x7f21f4004d00 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220815b550 con 0x7f21f40039e0 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f400bab0 con 0x7f220815afe0 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.547+0000 7f2207fff640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4004d00 0x7f21f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.548+0000 7f22077fe640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21f40039e0 0x7f21f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.548+0000 7f21fffff640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f220815afe0 0x7f21f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.548+0000 7f2207fff640 1 -- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f220815afe0 msgr2=0x7f21f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.548+0000 7f2207fff640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f220815afe0 0x7f21f40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.548+0000 7f2207fff640 1 -- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21f40039e0 msgr2=0x7f21f4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.548+0000 7f2207fff640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21f40039e0 0x7f21f4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.548+0000 7f2207fff640 1 -- 172.21.15.115:0/3493599511 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f4003520 con 0x7f21f4004d00 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.549+0000 7f2207fff640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4004d00 0x7f21f4005180 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f220000a830 tx=0x7f21f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50027351e1b9a45f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.549+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200034630 con 0x7f21f4004d00 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.549+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220002f070 con 0x7f21f4004d00 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.549+0000 7f2206ffd640 1 -- 172.21.15.115:0/3493599511 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22000349f0 con 0x7f21f4004d00 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.549+0000 7f220cf4f640 1 -- 172.21.15.115:0/3493599511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4004d00 msgr2=0x7f21f4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.549+0000 7f220cf4f640 1 --2- 172.21.15.115:0/3493599511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21f4004d00 0x7f21f4005180 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f220000a830 tx=0x7f21f40057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.549+0000 7f2207fff640 1 -- 172.21.15.115:0/3493599511 reap_dead start 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.550+0000 7f220cf4f640 1 -- 172.21.15.115:0/3493599511 shutdown_connections 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.550+0000 7f220cf4f640 1 -- 172.21.15.115:0/3493599511 >> 172.21.15.115:0/3493599511 conn(0x7f22080ce640 msgr2=0x7f22080cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.550+0000 7f220cf4f640 1 -- 172.21.15.115:0/3493599511 shutdown_connections 2023-12-01T02:51:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.550+0000 7f220cf4f640 1 -- 172.21.15.115:0/3493599511 wait complete. 2023-12-01T02:51:27.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.551+0000 7f220cf4f640 1 Processor -- start 2023-12-01T02:51:27.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.551+0000 7f220cf4f640 1 -- start start 2023-12-01T02:51:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.552+0000 7f220cf4f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21f4004d00 0x7f22080ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.552+0000 7f220cf4f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f220814c470 0x7f220814b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.552+0000 7f220cf4f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f220814beb0 0x7f22080ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.552+0000 7f220cf4f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22080c9bf0 con 0x7f220814c470 2023-12-01T02:51:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.552+0000 7f220cf4f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22080c8fe0 con 0x7f220814beb0 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.552+0000 7f220cf4f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22080c92c0 con 0x7f21f4004d00 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f21fffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f220814c470 0x7f220814b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f21fffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f220814c470 0x7f220814b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:42142/0 (socket says 172.21.15.115:42142) 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f21fffff640 1 -- 172.21.15.115:0/1183458963 learned_addr learned my addr 172.21.15.115:0/1183458963 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f21fffff640 1 -- 172.21.15.115:0/1183458963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21f4004d00 msgr2=0x7f22080ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f22077fe640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21f4004d00 0x7f22080ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f21fffff640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21f4004d00 0x7f22080ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f21fffff640 1 -- 172.21.15.115:0/1183458963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f220814beb0 msgr2=0x7f22080ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f21fffff640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f220814beb0 0x7f22080ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f21fffff640 1 -- 172.21.15.115:0/1183458963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22080c95a0 con 0x7f220814c470 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f21fffff640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f220814c470 0x7f220814b5f0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f21f000e7e0 tx=0x7f21f000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f2204ff9640 1 -- 172.21.15.115:0/1183458963 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f000c650 con 0x7f220814c470 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f2204ff9640 1 -- 172.21.15.115:0/1183458963 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f000c7b0 con 0x7f220814c470 2023-12-01T02:51:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.553+0000 7f2204ff9640 1 -- 172.21.15.115:0/1183458963 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f000cb20 con 0x7f220814c470 2023-12-01T02:51:27.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.554+0000 7f220cf4f640 1 -- 172.21.15.115:0/1183458963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22080c9880 con 0x7f220814c470 2023-12-01T02:51:27.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.554+0000 7f220cf4f640 1 -- 172.21.15.115:0/1183458963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220814bb30 con 0x7f220814c470 2023-12-01T02:51:27.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.555+0000 7f220cf4f640 1 -- 172.21.15.115:0/1183458963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22080c9da0 con 0x7f220814c470 2023-12-01T02:51:27.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.556+0000 7f2204ff9640 1 -- 172.21.15.115:0/1183458963 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 82) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f21f000cc80 con 0x7f220814c470 2023-12-01T02:51:27.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.556+0000 7f2204ff9640 1 -- 172.21.15.115:0/1183458963 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6413+0+0 (secure 0 0 0) 0x7f21f00883b0 con 0x7f220814c470 2023-12-01T02:51:27.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.560+0000 7f2204ff9640 1 -- 172.21.15.115:0/1183458963 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f22080c9da0 con 0x7f220814c470 2023-12-01T02:51:27.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.661+0000 7f2204ff9640 1 -- 172.21.15.115:0/1183458963 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 83) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f21f00535e0 con 0x7f220814c470 2023-12-01T02:51:27.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.661+0000 7f2204ff9640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21e8035ce0 0x7f21e8071c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:27.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.663+0000 7f22077fe640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21e8035ce0 0x7f21e8071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:27.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.665+0000 7f22077fe640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21e8035ce0 0x7f21e8071c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f21f8006fd0 tx=0x7f21f8008040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:27.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.788+0000 7f220cf4f640 1 -- 172.21.15.115:0/1183458963 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22080673a0 con 0x7f220814c470 2023-12-01T02:51:27.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.789+0000 7f2204ff9640 1 -- 172.21.15.115:0/1183458963 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f21f0084360 con 0x7f220814c470 2023-12-01T02:51:27.791 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:27.791 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3556763477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3556763477},{"type":"v1","addr":"172.21.15.142:6833","nonce":3556763477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3661095418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3661095418},{"type":"v1","addr":"172.21.15.142:6835","nonce":3661095418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3388585128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3388585128},{"type":"v1","addr":"172.21.15.115:6835","nonce":3388585128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:55:25.253838+0000","modified":"2023-12-01T01:55:28.264944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3894452047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3894452047},{"type":"v1","addr":"172.21.15.115:6837","nonce":3894452047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-01T02:51:27.791 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 63 2023-12-01T02:51:27.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 -- 172.21.15.115:0/1183458963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21e8035ce0 msgr2=0x7f21e8071c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21e8035ce0 0x7f21e8071c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f21f8006fd0 tx=0x7f21f8008040 comp rx=0 tx=0).stop 2023-12-01T02:51:27.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 -- 172.21.15.115:0/1183458963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f220814c470 msgr2=0x7f220814b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:27.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f220814c470 0x7f220814b5f0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f21f000e7e0 tx=0x7f21f000eca0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 -- 172.21.15.115:0/1183458963 shutdown_connections 2023-12-01T02:51:27.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f21e8035ce0 0x7f21e8071c40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f220814beb0 0x7f22080ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f220814c470 0x7f220814b5f0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 --2- 172.21.15.115:0/1183458963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21f4004d00 0x7f22080ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:27.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 -- 172.21.15.115:0/1183458963 >> 172.21.15.115:0/1183458963 conn(0x7f22080ce640 msgr2=0x7f22080af590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:27.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 -- 172.21.15.115:0/1183458963 shutdown_connections 2023-12-01T02:51:27.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:27.792+0000 7f220cf4f640 1 -- 172.21.15.115:0/1183458963 wait complete. 2023-12-01T02:51:27.810 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:51:28.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.020+0000 7fdb5ae5f640 1 Processor -- start 2023-12-01T02:51:28.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.020+0000 7fdb5ae5f640 1 -- start start 2023-12-01T02:51:28.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.021+0000 7fdb5ae5f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdb54159dd0 0x7fdb540c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.021+0000 7fdb5ae5f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdb5406b6f0 0x7fdb540c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb5ae5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb540ade80 0x7fdb540cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb5ae5f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5406f8b0 con 0x7fdb54159dd0 2023-12-01T02:51:28.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb5ae5f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb540adcb0 con 0x7fdb5406b6f0 2023-12-01T02:51:28.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb5ae5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb540ad9d0 con 0x7fdb540ade80 2023-12-01T02:51:28.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb58bfb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdb54159dd0 0x7fdb540c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb58bfb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdb54159dd0 0x7fdb540c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34384/0 (socket says 172.21.15.115:34384) 2023-12-01T02:51:28.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb58bfb640 1 -- 172.21.15.115:0/1847153953 learned_addr learned my addr 172.21.15.115:0/1847153953 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:28.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb58bfb640 1 -- 172.21.15.115:0/1847153953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb540ade80 msgr2=0x7fdb540cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:28.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb58bfb640 1 --2- 172.21.15.115:0/1847153953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb540ade80 0x7fdb540cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb58bfb640 1 -- 172.21.15.115:0/1847153953 >> v2:172.21.15.142:3300/0 conn(0x7fdb5406b6f0 msgr2=0x7fdb540c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:28.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb58bfb640 1 --2- 172.21.15.115:0/1847153953 >> v2:172.21.15.142:3300/0 conn(0x7fdb5406b6f0 0x7fdb540c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.022+0000 7fdb58bfb640 1 -- 172.21.15.115:0/1847153953 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb540c9000 con 0x7fdb54159dd0 2023-12-01T02:51:28.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.024+0000 7fdb58bfb640 1 --2- 172.21.15.115:0/1847153953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb54159dd0 0x7fdb540c97e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fdb5406de80 tx=0x7fdb4802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f996fc8468e9d95 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:28.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.024+0000 7fdb537fe640 1 -- 172.21.15.115:0/1847153953 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48031740 con 0x7fdb54159dd0 2023-12-01T02:51:28.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.024+0000 7fdb537fe640 1 -- 172.21.15.115:0/1847153953 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb480318a0 con 0x7fdb54159dd0 2023-12-01T02:51:28.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.024+0000 7fdb537fe640 1 -- 172.21.15.115:0/1847153953 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48031c30 con 0x7fdb54159dd0 2023-12-01T02:51:28.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.024+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/1847153953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb54159dd0 msgr2=0x7fdb540c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.024+0000 7fdb5ae5f640 1 --2- 172.21.15.115:0/1847153953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb54159dd0 0x7fdb540c97e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fdb5406de80 tx=0x7fdb4802b7e0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.025+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/1847153953 shutdown_connections 2023-12-01T02:51:28.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.025+0000 7fdb5ae5f640 1 --2- 172.21.15.115:0/1847153953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb540ade80 0x7fdb540cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.025+0000 7fdb5ae5f640 1 --2- 172.21.15.115:0/1847153953 >> v2:172.21.15.142:3300/0 conn(0x7fdb5406b6f0 0x7fdb540c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.025+0000 7fdb5ae5f640 1 --2- 172.21.15.115:0/1847153953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb54159dd0 0x7fdb540c97e0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.025+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/1847153953 >> 172.21.15.115:0/1847153953 conn(0x7fdb540ce880 msgr2=0x7fdb540cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:28.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.025+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/1847153953 shutdown_connections 2023-12-01T02:51:28.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.025+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/1847153953 wait complete. 2023-12-01T02:51:28.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.028+0000 7fdb5ae5f640 1 Processor -- start 2023-12-01T02:51:28.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.028+0000 7fdb5ae5f640 1 -- start start 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb5ae5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb5406b6f0 0x7fdb541506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb5ae5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb540ade80 0x7fdb541e4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb5ae5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb54159dd0 0x7fdb540c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb5ae5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb540c2b20 con 0x7fdb540ade80 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb5ae5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb54150c00 con 0x7fdb5406b6f0 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb5ae5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb541520e0 con 0x7fdb54159dd0 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb53fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb540ade80 0x7fdb541e4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb53fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb540ade80 0x7fdb541e4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34398/0 (socket says 172.21.15.115:34398) 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb53fff640 1 -- 172.21.15.115:0/3197035612 learned_addr learned my addr 172.21.15.115:0/3197035612 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb58bfb640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb5406b6f0 0x7fdb541506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb593fc640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb54159dd0 0x7fdb540c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb53fff640 1 -- 172.21.15.115:0/3197035612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb54159dd0 msgr2=0x7fdb540c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb53fff640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb54159dd0 0x7fdb540c47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb53fff640 1 -- 172.21.15.115:0/3197035612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb5406b6f0 msgr2=0x7fdb541506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb53fff640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb5406b6f0 0x7fdb541506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb53fff640 1 -- 172.21.15.115:0/3197035612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480094d0 con 0x7fdb540ade80 2023-12-01T02:51:28.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb58bfb640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb5406b6f0 0x7fdb541506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:28.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.029+0000 7fdb53fff640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb540ade80 0x7fdb541e4640 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000b4d0 tx=0x7fdb4000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:28.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.031+0000 7fdb517fa640 1 -- 172.21.15.115:0/3197035612 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40010870 con 0x7fdb540ade80 2023-12-01T02:51:28.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.031+0000 7fdb517fa640 1 -- 172.21.15.115:0/3197035612 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb400109d0 con 0x7fdb540ade80 2023-12-01T02:51:28.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.031+0000 7fdb517fa640 1 -- 172.21.15.115:0/3197035612 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40010d80 con 0x7fdb540ade80 2023-12-01T02:51:28.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.031+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/3197035612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb541e4b80 con 0x7fdb540ade80 2023-12-01T02:51:28.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.031+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/3197035612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb541e7310 con 0x7fdb540ade80 2023-12-01T02:51:28.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.032+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/3197035612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb540c97e0 con 0x7fdb540ade80 2023-12-01T02:51:28.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.033+0000 7fdb517fa640 1 -- 172.21.15.115:0/3197035612 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 83) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fdb4000f3c0 con 0x7fdb540ade80 2023-12-01T02:51:28.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.033+0000 7fdb517fa640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdb3806a2b0 0x7fdb3806c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.033+0000 7fdb517fa640 1 -- 172.21.15.115:0/3197035612 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdb400885f0 con 0x7fdb540ade80 2023-12-01T02:51:28.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.033+0000 7fdb58bfb640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdb3806a2b0 0x7fdb3806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.034+0000 7fdb58bfb640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdb3806a2b0 0x7fdb3806c770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdb48035000 tx=0x7fdb48008030 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:28.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.037+0000 7fdb517fa640 1 -- 172.21.15.115:0/3197035612 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdb40053c30 con 0x7fdb540ade80 2023-12-01T02:51:28.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.233+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/3197035612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb541e4b80 con 0x7fdb540ade80 2023-12-01T02:51:28.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.235+0000 7fdb517fa640 1 -- 172.21.15.115:0/3197035612 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+17288 (secure 0 0 0) 0x7fdb541e4b80 con 0x7fdb540ade80 2023-12-01T02:51:28.236 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:28.236 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":151,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:51:27.619665+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:55:22.818353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-01T01:55:24.272936+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":151,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/3528600695":"2023-12-02T02:51:26.614456+0000","172.21.15.115:0/1781878170":"2023-12-02T02:51:26.614456+0000","172.21.15.115:0/1223115907":"2023-12-02T02:51:26.614456+0000","172.21.15.115:0/2452856887":"2023-12-02T02:51:26.614456+0000","172.21.15.115:0/813639547":"2023-12-02T02:51:26.614456+0000","172.21.15.115:0/4068542751":"2023-12-02T02:51:26.614456+0000","172.21.15.115:6824/88317":"2023-12-02T01:55:38.761001+0000","172.21.15.115:6822/88317":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3362083032":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3703392460":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3403958692":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/671134594":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3600416058":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/2599234561":"2023-12-02T01:55:38.761001+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:51:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/3197035612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdb3806a2b0 msgr2=0x7fdb3806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdb3806a2b0 0x7fdb3806c770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdb48035000 tx=0x7fdb48008030 comp rx=0 tx=0).stop 2023-12-01T02:51:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/3197035612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb540ade80 msgr2=0x7fdb541e4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb540ade80 0x7fdb541e4640 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000b4d0 tx=0x7fdb4000b990 comp rx=0 tx=0).stop 2023-12-01T02:51:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/3197035612 shutdown_connections 2023-12-01T02:51:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdb3806a2b0 0x7fdb3806c770 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb54159dd0 0x7fdb540c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb540ade80 0x7fdb541e4640 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 --2- 172.21.15.115:0/3197035612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb5406b6f0 0x7fdb541506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/3197035612 >> 172.21.15.115:0/3197035612 conn(0x7fdb540ce880 msgr2=0x7fdb540afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:28.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/3197035612 shutdown_connections 2023-12-01T02:51:28.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.238+0000 7fdb5ae5f640 1 -- 172.21.15.115:0/3197035612 wait complete. 2023-12-01T02:51:28.258 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T02:51:28.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:51:28.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.470+0000 7f8f1e88f640 1 Processor -- start 2023-12-01T02:51:28.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.470+0000 7f8f1e88f640 1 -- start start 2023-12-01T02:51:28.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.471+0000 7f8f1e88f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8f180cad70 0x7f8f1806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.471+0000 7f8f1e88f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f180add10 con 0x7f8f1806f2f0 2023-12-01T02:51:28.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.471+0000 7f8f1e88f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f180c5760 con 0x7f8f1815e2f0 2023-12-01T02:51:28.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.471+0000 7f8f1e88f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1815e6d0 con 0x7f8f180cad70 2023-12-01T02:51:28.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.472+0000 7f8f1ce2c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8f1806f2f0 0x7f8f1806bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39926/0 (socket says 172.21.15.115:39926) 2023-12-01T02:51:28.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.472+0000 7f8f1ce2c640 1 -- 172.21.15.115:0/1708031260 learned_addr learned my addr 172.21.15.115:0/1708031260 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:28.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.472+0000 7f8f0ffff640 1 --2- 172.21.15.115:0/1708031260 >> v2:172.21.15.115:3300/0 conn(0x7f8f180cad70 0x7f8f1806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.472+0000 7f8f177fe640 1 -- 172.21.15.115:0/1708031260 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307923163 0 0) 0x7f8f180add10 con 0x7f8f1806f2f0 2023-12-01T02:51:28.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.472+0000 7f8f177fe640 1 -- 172.21.15.115:0/1708031260 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f04003520 con 0x7f8f1806f2f0 2023-12-01T02:51:28.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.472+0000 7f8f0ffff640 1 -- 172.21.15.115:0/1708031260 >> v1:172.21.15.142:6789/0 conn(0x7f8f1815e2f0 legacy=0x7f8f180ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:28.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.473+0000 7f8f0ffff640 1 -- 172.21.15.115:0/1708031260 >> v1:172.21.15.115:6789/0 conn(0x7f8f1806f2f0 legacy=0x7f8f1806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.473+0000 7f8f0ffff640 1 -- 172.21.15.115:0/1708031260 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f180cf7b0 con 0x7f8f180cad70 2023-12-01T02:51:28.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.474+0000 7f8f0ffff640 1 --2- 172.21.15.115:0/1708031260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f180cad70 0x7f8f1806b5c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8f000097d0 tx=0x7f8f0002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7655fc923553a2e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:28.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.474+0000 7f8f177fe640 1 -- 172.21.15.115:0/1708031260 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0002f050 con 0x7f8f180cad70 2023-12-01T02:51:28.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.474+0000 7f8f177fe640 1 -- 172.21.15.115:0/1708031260 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f00031700 con 0x7f8f180cad70 2023-12-01T02:51:28.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.474+0000 7f8f177fe640 1 -- 172.21.15.115:0/1708031260 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00031a90 con 0x7f8f180cad70 2023-12-01T02:51:28.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.475+0000 7f8f1e88f640 1 -- 172.21.15.115:0/1708031260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f180cad70 msgr2=0x7f8f1806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.475+0000 7f8f1e88f640 1 --2- 172.21.15.115:0/1708031260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f180cad70 0x7f8f1806b5c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8f000097d0 tx=0x7f8f0002b7a0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.475+0000 7f8f1e88f640 1 -- 172.21.15.115:0/1708031260 shutdown_connections 2023-12-01T02:51:28.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.475+0000 7f8f1e88f640 1 --2- 172.21.15.115:0/1708031260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f180cad70 0x7f8f1806b5c0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.475+0000 7f8f1e88f640 1 -- 172.21.15.115:0/1708031260 >> 172.21.15.115:0/1708031260 conn(0x7f8f180ce880 msgr2=0x7f8f180cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:28.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.475+0000 7f8f1e88f640 1 -- 172.21.15.115:0/1708031260 shutdown_connections 2023-12-01T02:51:28.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.475+0000 7f8f1e88f640 1 -- 172.21.15.115:0/1708031260 wait complete. 2023-12-01T02:51:28.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.477+0000 7f8f1e88f640 1 Processor -- start 2023-12-01T02:51:28.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.477+0000 7f8f1e88f640 1 -- start start 2023-12-01T02:51:28.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.478+0000 7f8f1e88f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f1806f2f0 0x7f8f18153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.478+0000 7f8f1e88f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f180cad70 0x7f8f18153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.478+0000 7f8f1e88f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1815e2f0 0x7f8f180c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.478+0000 7f8f1e88f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180ae440 con 0x7f8f1815e2f0 2023-12-01T02:51:28.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.478+0000 7f8f1e88f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180ae720 con 0x7f8f180cad70 2023-12-01T02:51:28.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.478+0000 7f8f1e88f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180aea00 con 0x7f8f1806f2f0 2023-12-01T02:51:28.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.479+0000 7f8f1ce2c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1815e2f0 0x7f8f180c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.479+0000 7f8f1ce2c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1815e2f0 0x7f8f180c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34408/0 (socket says 172.21.15.115:34408) 2023-12-01T02:51:28.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.479+0000 7f8f1ce2c640 1 -- 172.21.15.115:0/43546521 learned_addr learned my addr 172.21.15.115:0/43546521 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:28.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.479+0000 7f8f1ce2c640 1 -- 172.21.15.115:0/43546521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f1806f2f0 msgr2=0x7f8f18153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.479+0000 7f8f1ce2c640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f1806f2f0 0x7f8f18153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.479+0000 7f8f1ce2c640 1 -- 172.21.15.115:0/43546521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f180cad70 msgr2=0x7f8f18153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.479+0000 7f8f1ce2c640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f180cad70 0x7f8f18153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.479+0000 7f8f1ce2c640 1 -- 172.21.15.115:0/43546521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f00009400 con 0x7f8f1815e2f0 2023-12-01T02:51:28.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.479+0000 7f8f1ce2c640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1815e2f0 0x7f8f180c23b0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000caa0 tx=0x7f8f1000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:28.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.480+0000 7f8f157fa640 1 -- 172.21.15.115:0/43546521 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1000aba0 con 0x7f8f1815e2f0 2023-12-01T02:51:28.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.480+0000 7f8f157fa640 1 -- 172.21.15.115:0/43546521 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1000ad00 con 0x7f8f1815e2f0 2023-12-01T02:51:28.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.480+0000 7f8f157fa640 1 -- 172.21.15.115:0/43546521 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10012600 con 0x7f8f1815e2f0 2023-12-01T02:51:28.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.480+0000 7f8f1e88f640 1 -- 172.21.15.115:0/43546521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f180aece0 con 0x7f8f1815e2f0 2023-12-01T02:51:28.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.480+0000 7f8f1e88f640 1 -- 172.21.15.115:0/43546521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f180c28f0 con 0x7f8f1815e2f0 2023-12-01T02:51:28.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.481+0000 7f8f1e88f640 1 -- 172.21.15.115:0/43546521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1806fda0 con 0x7f8f1815e2f0 2023-12-01T02:51:28.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.482+0000 7f8f157fa640 1 -- 172.21.15.115:0/43546521 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 83) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f8f10012760 con 0x7f8f1815e2f0 2023-12-01T02:51:28.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.483+0000 7f8f157fa640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ef806a280 0x7f8ef806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.483+0000 7f8f157fa640 1 -- 172.21.15.115:0/43546521 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8f100885c0 con 0x7f8f1815e2f0 2023-12-01T02:51:28.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.483+0000 7f8f17fff640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ef806a280 0x7f8ef806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.484+0000 7f8f17fff640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ef806a280 0x7f8ef806c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8f08003cc0 tx=0x7f8f080079b0 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:28.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.488+0000 7f8f157fa640 1 -- 172.21.15.115:0/43546521 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8f10053cb0 con 0x7f8f1815e2f0 2023-12-01T02:51:28.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.726+0000 7f8f1e88f640 1 -- 172.21.15.115:0/43546521 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f180af200 con 0x7f8f1815e2f0 2023-12-01T02:51:28.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.726+0000 7f8f157fa640 1 -- 172.21.15.115:0/43546521 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f8f180af200 con 0x7f8f1815e2f0 2023-12-01T02:51:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:28.727 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3556763477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3556763477},{"type":"v1","addr":"172.21.15.142:6833","nonce":3556763477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3661095418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3661095418},{"type":"v1","addr":"172.21.15.142:6835","nonce":3661095418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3388585128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3388585128},{"type":"v1","addr":"172.21.15.115:6835","nonce":3388585128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:55:25.253838+0000","modified":"2023-12-01T01:55:28.264944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3894452047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3894452047},{"type":"v1","addr":"172.21.15.115:6837","nonce":3894452047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-01T02:51:28.727 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 63 2023-12-01T02:51:28.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 -- 172.21.15.115:0/43546521 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ef806a280 msgr2=0x7f8ef806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ef806a280 0x7f8ef806c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8f08003cc0 tx=0x7f8f080079b0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 -- 172.21.15.115:0/43546521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1815e2f0 msgr2=0x7f8f180c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1815e2f0 0x7f8f180c23b0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000caa0 tx=0x7f8f1000cf60 comp rx=0 tx=0).stop 2023-12-01T02:51:28.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 -- 172.21.15.115:0/43546521 shutdown_connections 2023-12-01T02:51:28.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8ef806a280 0x7f8ef806c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1815e2f0 0x7f8f180c23b0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f180cad70 0x7f8f18153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 --2- 172.21.15.115:0/43546521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f1806f2f0 0x7f8f18153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 -- 172.21.15.115:0/43546521 >> 172.21.15.115:0/43546521 conn(0x7f8f180ce880 msgr2=0x7f8f180c1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:28.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.729+0000 7f8f1e88f640 1 -- 172.21.15.115:0/43546521 shutdown_connections 2023-12-01T02:51:28.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.730+0000 7f8f1e88f640 1 -- 172.21.15.115:0/43546521 wait complete. 2023-12-01T02:51:28.749 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:51:28.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.935+0000 7fb8fd3e1640 1 Processor -- start 2023-12-01T02:51:28.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.935+0000 7fb8fd3e1640 1 -- start start 2023-12-01T02:51:28.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.936+0000 7fb8fd3e1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb8f81584b0 0x7fb8f80cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.936+0000 7fb8fd3e1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f80cc180 con 0x7fb8f806b980 2023-12-01T02:51:28.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.936+0000 7fb8fd3e1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f815b550 con 0x7fb8f815afe0 2023-12-01T02:51:28.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.936+0000 7fb8fd3e1640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f815b830 con 0x7fb8f81584b0 2023-12-01T02:51:28.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.937+0000 7fb8f7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb8f81584b0 0x7fb8f80cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.937+0000 7fb8f7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb8f81584b0 0x7fb8f80cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34446/0 (socket says 172.21.15.115:34446) 2023-12-01T02:51:28.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.937+0000 7fb8f7fff640 1 -- 172.21.15.115:0/1952488720 learned_addr learned my addr 172.21.15.115:0/1952488720 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:28.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.937+0000 7fb8fcbe0640 1 --1- 172.21.15.115:0/1952488720 >> v1:172.21.15.115:6789/0 conn(0x7fb8f806b980 0x7fb8f80cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39936/0 (socket says 172.21.15.115:39936) 2023-12-01T02:51:28.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.937+0000 7fb8f7fff640 1 -- 172.21.15.115:0/1952488720 >> v1:172.21.15.142:6789/0 conn(0x7fb8f815afe0 legacy=0x7fb8f806b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:28.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.937+0000 7fb8f7fff640 1 -- 172.21.15.115:0/1952488720 >> v1:172.21.15.115:6789/0 conn(0x7fb8f806b980 legacy=0x7fb8f80cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.937+0000 7fb8f7fff640 1 -- 172.21.15.115:0/1952488720 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f815bb10 con 0x7fb8f81584b0 2023-12-01T02:51:28.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.938+0000 7fb8f7fff640 1 --2- 172.21.15.115:0/1952488720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 0x7fb8f80cdd90 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009870 tx=0x7fb8e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b966119ad291ff0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:28.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.938+0000 7fb8f77fe640 1 -- 172.21.15.115:0/1952488720 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8030860 con 0x7fb8f81584b0 2023-12-01T02:51:28.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.938+0000 7fb8f77fe640 1 -- 172.21.15.115:0/1952488720 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e80309c0 con 0x7fb8f81584b0 2023-12-01T02:51:28.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.939+0000 7fb8f77fe640 1 -- 172.21.15.115:0/1952488720 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8030d50 con 0x7fb8f81584b0 2023-12-01T02:51:28.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.939+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/1952488720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 msgr2=0x7fb8f80cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.939+0000 7fb8fd3e1640 1 --2- 172.21.15.115:0/1952488720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 0x7fb8f80cdd90 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009870 tx=0x7fb8e802b9a0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.939+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/1952488720 shutdown_connections 2023-12-01T02:51:28.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.939+0000 7fb8fd3e1640 1 --2- 172.21.15.115:0/1952488720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 0x7fb8f80cdd90 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.939+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/1952488720 >> 172.21.15.115:0/1952488720 conn(0x7fb8f80ce640 msgr2=0x7fb8f80cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:28.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.939+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/1952488720 shutdown_connections 2023-12-01T02:51:28.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.939+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/1952488720 wait complete. 2023-12-01T02:51:28.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.941+0000 7fb8fd3e1640 1 Processor -- start 2023-12-01T02:51:28.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.941+0000 7fb8fd3e1640 1 -- start start 2023-12-01T02:51:28.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.942+0000 7fb8fd3e1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f806b980 0x7fb8f81f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.942+0000 7fb8fd3e1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 0x7fb8f814eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.942+0000 7fb8fd3e1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8f815afe0 0x7fb8f8071850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.942+0000 7fb8fd3e1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f814ca10 con 0x7fb8f81584b0 2023-12-01T02:51:28.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.942+0000 7fb8fd3e1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f814eff0 con 0x7fb8f806b980 2023-12-01T02:51:28.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.942+0000 7fb8fd3e1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f81f4780 con 0x7fb8f815afe0 2023-12-01T02:51:28.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.942+0000 7fb8effff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 0x7fb8f814eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.942+0000 7fb8effff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 0x7fb8f814eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34448/0 (socket says 172.21.15.115:34448) 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.942+0000 7fb8effff640 1 -- 172.21.15.115:0/99019338 learned_addr learned my addr 172.21.15.115:0/99019338 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8effff640 1 -- 172.21.15.115:0/99019338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8f815afe0 msgr2=0x7fb8f8071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8effff640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8f815afe0 0x7fb8f8071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8f7fff640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f806b980 0x7fb8f81f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8effff640 1 -- 172.21.15.115:0/99019338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f806b980 msgr2=0x7fb8f81f11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8effff640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f806b980 0x7fb8f81f11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8effff640 1 -- 172.21.15.115:0/99019338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e80094d0 con 0x7fb8f81584b0 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8effff640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 0x7fb8f814eab0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c440 tx=0x7fb8e000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8f57fa640 1 -- 172.21.15.115:0/99019338 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0005480 con 0x7fb8f81584b0 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8f57fa640 1 -- 172.21.15.115:0/99019338 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e00035f0 con 0x7fb8f81584b0 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8f57fa640 1 -- 172.21.15.115:0/99019338 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00039c0 con 0x7fb8f81584b0 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.943+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/99019338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f81f06f0 con 0x7fb8f81584b0 2023-12-01T02:51:28.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.944+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/99019338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f81f0cb0 con 0x7fb8f81584b0 2023-12-01T02:51:28.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.945+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/99019338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f81f0ed0 con 0x7fb8f81584b0 2023-12-01T02:51:28.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.946+0000 7fb8f57fa640 1 -- 172.21.15.115:0/99019338 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 83) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fb8e0003b20 con 0x7fb8f81584b0 2023-12-01T02:51:28.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.947+0000 7fb8f57fa640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d806a280 0x7fb8d806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:28.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.947+0000 7fb8f57fa640 1 -- 172.21.15.115:0/99019338 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb8e0087930 con 0x7fb8f81584b0 2023-12-01T02:51:28.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.947+0000 7fb8f7fff640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d806a280 0x7fb8d806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:28.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.948+0000 7fb8f7fff640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d806a280 0x7fb8d806c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8036000 tx=0x7fb8e803b040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:28.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:28.951+0000 7fb8f57fa640 1 -- 172.21.15.115:0/99019338 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb8e0052f70 con 0x7fb8f81584b0 2023-12-01T02:51:29.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.177+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/99019338 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8f815a780 con 0x7fb8f81584b0 2023-12-01T02:51:29.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.177+0000 7fb8f57fa640 1 -- 172.21.15.115:0/99019338 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fb8e0056590 con 0x7fb8f81584b0 2023-12-01T02:51:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:29.178 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3556763477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3556763477},{"type":"v1","addr":"172.21.15.142:6833","nonce":3556763477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3661095418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3661095418},{"type":"v1","addr":"172.21.15.142:6835","nonce":3661095418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3388585128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3388585128},{"type":"v1","addr":"172.21.15.115:6835","nonce":3388585128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T01:55:25.253838+0000","modified":"2023-12-01T01:55:28.264944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8475":{"gid":8475,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3894452047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3894452047},{"type":"v1","addr":"172.21.15.115:6837","nonce":3894452047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-01T02:51:29.179 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 63 2023-12-01T02:51:29.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/99019338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d806a280 msgr2=0x7fb8d806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d806a280 0x7fb8d806c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8036000 tx=0x7fb8e803b040 comp rx=0 tx=0).stop 2023-12-01T02:51:29.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/99019338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 msgr2=0x7fb8f814eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 0x7fb8f814eab0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c440 tx=0x7fb8e000c900 comp rx=0 tx=0).stop 2023-12-01T02:51:29.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/99019338 shutdown_connections 2023-12-01T02:51:29.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8d806a280 0x7fb8d806c740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb8f815afe0 0x7fb8f8071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8f81584b0 0x7fb8f814eab0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 --2- 172.21.15.115:0/99019338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8f806b980 0x7fb8f81f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/99019338 >> 172.21.15.115:0/99019338 conn(0x7fb8f80ce640 msgr2=0x7fb8f80ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:29.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/99019338 shutdown_connections 2023-12-01T02:51:29.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.180+0000 7fb8fd3e1640 1 -- 172.21.15.115:0/99019338 wait complete. 2023-12-01T02:51:29.199 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:51:29.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.399+0000 7f0215c7a640 1 Processor -- start 2023-12-01T02:51:29.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.399+0000 7f0215c7a640 1 -- start start 2023-12-01T02:51:29.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.400+0000 7f0215c7a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0210159dd0 0x7f02100c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.400+0000 7f0215c7a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021006f8b0 con 0x7f021006b6f0 2023-12-01T02:51:29.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.400+0000 7f0215c7a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02100adcb0 con 0x7f0210070d30 2023-12-01T02:51:29.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.400+0000 7f0215c7a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02100cd270 con 0x7f0210159dd0 2023-12-01T02:51:29.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.401+0000 7f020f7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0210159dd0 0x7f02100c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:29.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.401+0000 7f020effd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f021006b6f0 0x7f02100cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39944/0 (socket says 172.21.15.115:39944) 2023-12-01T02:51:29.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.401+0000 7f020effd640 1 -- 172.21.15.115:0/658105851 learned_addr learned my addr 172.21.15.115:0/658105851 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:29.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.401+0000 7f020f7fe640 1 -- 172.21.15.115:0/658105851 >> v1:172.21.15.142:6789/0 conn(0x7f0210070d30 legacy=0x7f02100ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:29.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.401+0000 7f020f7fe640 1 -- 172.21.15.115:0/658105851 >> v1:172.21.15.115:6789/0 conn(0x7f021006b6f0 legacy=0x7f02100cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.401+0000 7f020e7fc640 1 -- 172.21.15.115:0/658105851 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603744461 0 0) 0x7f021006f8b0 con 0x7f021006b6f0 2023-12-01T02:51:29.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.401+0000 7f020f7fe640 1 -- 172.21.15.115:0/658105851 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02100ad9d0 con 0x7f0210159dd0 2023-12-01T02:51:29.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.402+0000 7f020f7fe640 1 --2- 172.21.15.115:0/658105851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 0x7f02100c97e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f02000097a0 tx=0x7f020002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d17947299eb95b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:29.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.402+0000 7f020e7fc640 1 -- 172.21.15.115:0/658105851 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020002f050 con 0x7f0210159dd0 2023-12-01T02:51:29.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.403+0000 7f020e7fc640 1 -- 172.21.15.115:0/658105851 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0200031700 con 0x7f0210159dd0 2023-12-01T02:51:29.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.403+0000 7f020e7fc640 1 -- 172.21.15.115:0/658105851 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200031a90 con 0x7f0210159dd0 2023-12-01T02:51:29.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.403+0000 7f0215c7a640 1 -- 172.21.15.115:0/658105851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 msgr2=0x7f02100c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.403+0000 7f0215c7a640 1 --2- 172.21.15.115:0/658105851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 0x7f02100c97e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f02000097a0 tx=0x7f020002b7a0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.403+0000 7f0215c7a640 1 -- 172.21.15.115:0/658105851 shutdown_connections 2023-12-01T02:51:29.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.403+0000 7f0215c7a640 1 --2- 172.21.15.115:0/658105851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 0x7f02100c97e0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.403+0000 7f0215c7a640 1 -- 172.21.15.115:0/658105851 >> 172.21.15.115:0/658105851 conn(0x7f02100ce880 msgr2=0x7f02100cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:29.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.403+0000 7f0215c7a640 1 -- 172.21.15.115:0/658105851 shutdown_connections 2023-12-01T02:51:29.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.403+0000 7f0215c7a640 1 -- 172.21.15.115:0/658105851 wait complete. 2023-12-01T02:51:29.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.405+0000 7f0215c7a640 1 Processor -- start 2023-12-01T02:51:29.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.405+0000 7f0215c7a640 1 -- start start 2023-12-01T02:51:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.406+0000 7f0215c7a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f021006b6f0 0x7f0210150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.406+0000 7f0215c7a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210070d30 0x7f021014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.406+0000 7f0215c7a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 0x7f021014f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.406+0000 7f0215c7a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021014ef30 con 0x7f0210159dd0 2023-12-01T02:51:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.406+0000 7f0215c7a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02101e2760 con 0x7f0210070d30 2023-12-01T02:51:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.406+0000 7f0215c7a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02101e2a00 con 0x7f021006b6f0 2023-12-01T02:51:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.406+0000 7f020ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 0x7f021014f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.406+0000 7f020ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 0x7f021014f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34464/0 (socket says 172.21.15.115:34464) 2023-12-01T02:51:29.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.406+0000 7f020ffff640 1 -- 172.21.15.115:0/1729293252 learned_addr learned my addr 172.21.15.115:0/1729293252 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:29.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f020ffff640 1 -- 172.21.15.115:0/1729293252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f021006b6f0 msgr2=0x7f0210150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f020ffff640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f021006b6f0 0x7f0210150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f020ffff640 1 -- 172.21.15.115:0/1729293252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210070d30 msgr2=0x7f021014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f020ffff640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210070d30 0x7f021014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f020ffff640 1 -- 172.21.15.115:0/1729293252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0200009400 con 0x7f0210159dd0 2023-12-01T02:51:29.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f020ffff640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 0x7f021014f0f0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f020800b970 tx=0x7f020800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:29.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f01f7fff640 1 -- 172.21.15.115:0/1729293252 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020800cb40 con 0x7f0210159dd0 2023-12-01T02:51:29.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f01f7fff640 1 -- 172.21.15.115:0/1729293252 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020800cca0 con 0x7f0210159dd0 2023-12-01T02:51:29.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f01f7fff640 1 -- 172.21.15.115:0/1729293252 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208014680 con 0x7f0210159dd0 2023-12-01T02:51:29.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f0215c7a640 1 -- 172.21.15.115:0/1729293252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02101e2ce0 con 0x7f0210159dd0 2023-12-01T02:51:29.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.407+0000 7f0215c7a640 1 -- 172.21.15.115:0/1729293252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02100c33c0 con 0x7f0210159dd0 2023-12-01T02:51:29.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.409+0000 7f0215c7a640 1 -- 172.21.15.115:0/1729293252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0210152500 con 0x7f0210159dd0 2023-12-01T02:51:29.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.410+0000 7f01f7fff640 1 -- 172.21.15.115:0/1729293252 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 84) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f020801d3b0 con 0x7f0210159dd0 2023-12-01T02:51:29.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.410+0000 7f01f7fff640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01fc06a4c0 0x7f01fc06c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.410+0000 7f01f7fff640 1 -- 172.21.15.115:0/1729293252 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f0208088a40 con 0x7f0210159dd0 2023-12-01T02:51:29.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.411+0000 7f020f7fe640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01fc06a4c0 0x7f01fc06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:29.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.411+0000 7f020f7fe640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01fc06a4c0 0x7f01fc06c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0200035000 tx=0x7f0200030000 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:29.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.415+0000 7f01f7fff640 1 -- 172.21.15.115:0/1729293252 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0208053de0 con 0x7f0210159dd0 2023-12-01T02:51:29.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.608+0000 7f0215c7a640 1 -- 172.21.15.115:0/1729293252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0210152920 con 0x7f0210159dd0 2023-12-01T02:51:29.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.610+0000 7f01f7fff640 1 -- 172.21.15.115:0/1729293252 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+17288 (secure 0 0 0) 0x7f0208080e60 con 0x7f0210159dd0 2023-12-01T02:51:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:29.611 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":151,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:51:27.619665+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:55:22.818353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-01T01:55:24.272936+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":151,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/3528600695":"2023-12-02T02:51:26.614456+0000","172.21.15.115:0/1781878170":"2023-12-02T02:51:26.614456+0000","172.21.15.115:0/1223115907":"2023-12-02T02:51:26.614456+0000","172.21.15.115:0/2452856887":"2023-12-02T02:51:26.614456+0000","172.21.15.115:0/813639547":"2023-12-02T02:51:26.614456+0000","172.21.15.115:0/4068542751":"2023-12-02T02:51:26.614456+0000","172.21.15.115:6824/88317":"2023-12-02T01:55:38.761001+0000","172.21.15.115:6822/88317":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3362083032":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3703392460":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3403958692":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/671134594":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/3600416058":"2023-12-02T01:55:38.761001+0000","172.21.15.115:0/2599234561":"2023-12-02T01:55:38.761001+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:51:29.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.612+0000 7f0215c7a640 1 -- 172.21.15.115:0/1729293252 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01fc06a4c0 msgr2=0x7f01fc06c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.612+0000 7f0215c7a640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01fc06a4c0 0x7f01fc06c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0200035000 tx=0x7f0200030000 comp rx=0 tx=0).stop 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.612+0000 7f0215c7a640 1 -- 172.21.15.115:0/1729293252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 msgr2=0x7f021014f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.612+0000 7f0215c7a640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 0x7f021014f0f0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f020800b970 tx=0x7f020800be30 comp rx=0 tx=0).stop 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.613+0000 7f0215c7a640 1 -- 172.21.15.115:0/1729293252 shutdown_connections 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.613+0000 7f0215c7a640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01fc06a4c0 0x7f01fc06c980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.613+0000 7f0215c7a640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0210159dd0 0x7f021014f0f0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.613+0000 7f0215c7a640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0210070d30 0x7f021014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.613+0000 7f0215c7a640 1 --2- 172.21.15.115:0/1729293252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f021006b6f0 0x7f0210150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.613+0000 7f0215c7a640 1 -- 172.21.15.115:0/1729293252 >> 172.21.15.115:0/1729293252 conn(0x7f02100ce880 msgr2=0x7f021015cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.613+0000 7f0215c7a640 1 -- 172.21.15.115:0/1729293252 shutdown_connections 2023-12-01T02:51:29.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.613+0000 7f0215c7a640 1 -- 172.21.15.115:0/1729293252 wait complete. 2023-12-01T02:51:29.632 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T02:51:29.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.849+0000 7f916d107640 1 Processor -- start 2023-12-01T02:51:29.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.849+0000 7f916d107640 1 -- start start 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.850+0000 7f916d107640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f916815a910 0x7f91680d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.850+0000 7f916d107640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f91680cc670 0x7f916815e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.850+0000 7f916d107640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91680b3670 con 0x7f916815acf0 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.850+0000 7f916d107640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91680cf110 con 0x7f91680cc670 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.850+0000 7f916d107640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91680cf3f0 con 0x7f916815a910 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f91677fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f91680cc670 0x7f916815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f91677fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f91680cc670 0x7f916815e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34478/0 (socket says 172.21.15.115:34478) 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f91677fe640 1 -- 172.21.15.115:0/3713948428 learned_addr learned my addr 172.21.15.115:0/3713948428 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f916c906640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f916815acf0 0x7f91680cb3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39952/0 (socket says 172.21.15.115:39952) 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f91677fe640 1 -- 172.21.15.115:0/3713948428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f916815a910 msgr2=0x7f91680d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f9167fff640 1 --2- 172.21.15.115:0/3713948428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f916815a910 0x7f91680d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f91677fe640 1 --2- 172.21.15.115:0/3713948428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f916815a910 0x7f91680d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f91677fe640 1 -- 172.21.15.115:0/3713948428 >> v1:172.21.15.115:6789/0 conn(0x7f916815acf0 legacy=0x7f91680cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f91677fe640 1 -- 172.21.15.115:0/3713948428 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91680cf6d0 con 0x7f91680cc670 2023-12-01T02:51:29.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f9167fff640 1 --2- 172.21.15.115:0/3713948428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f916815a910 0x7f91680d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:29.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.851+0000 7f9166ffd640 1 -- 172.21.15.115:0/3713948428 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654192401 0 0) 0x7f91680b3670 con 0x7f916815acf0 2023-12-01T02:51:29.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.852+0000 7f91677fe640 1 --2- 172.21.15.115:0/3713948428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91680cc670 0x7f916815e700 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f9150009870 tx=0x7f915002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414668e8e60a9642 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:29.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f9166ffd640 1 -- 172.21.15.115:0/3713948428 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150030840 con 0x7f91680cc670 2023-12-01T02:51:29.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f9166ffd640 1 -- 172.21.15.115:0/3713948428 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91500309a0 con 0x7f91680cc670 2023-12-01T02:51:29.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f9166ffd640 1 -- 172.21.15.115:0/3713948428 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150030d30 con 0x7f91680cc670 2023-12-01T02:51:29.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f916d107640 1 -- 172.21.15.115:0/3713948428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91680cc670 msgr2=0x7f916815e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f916d107640 1 --2- 172.21.15.115:0/3713948428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91680cc670 0x7f916815e700 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f9150009870 tx=0x7f915002b980 comp rx=0 tx=0).stop 2023-12-01T02:51:29.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f916d107640 1 -- 172.21.15.115:0/3713948428 shutdown_connections 2023-12-01T02:51:29.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f916d107640 1 --2- 172.21.15.115:0/3713948428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91680cc670 0x7f916815e700 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f916d107640 1 --2- 172.21.15.115:0/3713948428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f916815a910 0x7f91680d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f916d107640 1 -- 172.21.15.115:0/3713948428 >> 172.21.15.115:0/3713948428 conn(0x7f9168099a20 msgr2=0x7f91680d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f916d107640 1 -- 172.21.15.115:0/3713948428 shutdown_connections 2023-12-01T02:51:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.853+0000 7f916d107640 1 -- 172.21.15.115:0/3713948428 wait complete. 2023-12-01T02:51:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.855+0000 7f916d107640 1 Processor -- start 2023-12-01T02:51:29.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.855+0000 7f916d107640 1 -- start start 2023-12-01T02:51:29.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.856+0000 7f916d107640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91680cc670 0x7f916814ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.856+0000 7f916d107640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f916815a910 0x7f916806b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.856+0000 7f916d107640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f916815acf0 0x7f916806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.856+0000 7f916d107640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916815ca70 con 0x7f916815acf0 2023-12-01T02:51:29.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.856+0000 7f916d107640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916815d8d0 con 0x7f916815a910 2023-12-01T02:51:29.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.856+0000 7f916d107640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916814f230 con 0x7f91680cc670 2023-12-01T02:51:29.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.856+0000 7f916c906640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f916815acf0 0x7f916806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:29.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f916c906640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f916815acf0 0x7f916806b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34494/0 (socket says 172.21.15.115:34494) 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f916c906640 1 -- 172.21.15.115:0/3194195744 learned_addr learned my addr 172.21.15.115:0/3194195744 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f916c906640 1 -- 172.21.15.115:0/3194195744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91680cc670 msgr2=0x7f916814ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f916c906640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91680cc670 0x7f916814ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f916c906640 1 -- 172.21.15.115:0/3194195744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f916815a910 msgr2=0x7f916806b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f916c906640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f916815a910 0x7f916806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f916c906640 1 -- 172.21.15.115:0/3194195744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91500094d0 con 0x7f916815acf0 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f916c906640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f916815acf0 0x7f916806b8b0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f916000c440 tx=0x7f916000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f9164ff9640 1 -- 172.21.15.115:0/3194195744 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91600074e0 con 0x7f916815acf0 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f9164ff9640 1 -- 172.21.15.115:0/3194195744 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9160007660 con 0x7f916815acf0 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.857+0000 7f916d107640 1 -- 172.21.15.115:0/3194195744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916815ccc0 con 0x7f916815acf0 2023-12-01T02:51:29.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.858+0000 7f9164ff9640 1 -- 172.21.15.115:0/3194195744 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160004a20 con 0x7f916815acf0 2023-12-01T02:51:29.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.858+0000 7f916d107640 1 -- 172.21.15.115:0/3194195744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916815d280 con 0x7f916815acf0 2023-12-01T02:51:29.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.859+0000 7f916d107640 1 -- 172.21.15.115:0/3194195744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91680673a0 con 0x7f916815acf0 2023-12-01T02:51:29.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.860+0000 7f9164ff9640 1 -- 172.21.15.115:0/3194195744 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 84) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f9160004b80 con 0x7f916815acf0 2023-12-01T02:51:29.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.860+0000 7f9164ff9640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f914806a4c0 0x7f914806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:29.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.861+0000 7f9164ff9640 1 -- 172.21.15.115:0/3194195744 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9160088a00 con 0x7f916815acf0 2023-12-01T02:51:29.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.861+0000 7f9167fff640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f914806a4c0 0x7f914806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:29.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.862+0000 7f9167fff640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f914806a4c0 0x7f914806c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9158008ef0 tx=0x7f9158008420 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:29.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:29.865+0000 7f9164ff9640 1 -- 172.21.15.115:0/3194195744 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9160053c10 con 0x7f916815acf0 2023-12-01T02:51:30.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.091+0000 7f916d107640 1 -- 172.21.15.115:0/3194195744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f916815da10 con 0x7f916815acf0 2023-12-01T02:51:30.094 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T02:51:30.093+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T02:51:30.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.108+0000 7f9164ff9640 1 -- 172.21.15.115:0/3194195744 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7f916815da10 con 0x7f916815acf0 2023-12-01T02:51:30.109 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 -- 172.21.15.115:0/3194195744 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f914806a4c0 msgr2=0x7f914806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f914806a4c0 0x7f914806c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9158008ef0 tx=0x7f9158008420 comp rx=0 tx=0).stop 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 -- 172.21.15.115:0/3194195744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f916815acf0 msgr2=0x7f916806b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f916815acf0 0x7f916806b8b0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f916000c440 tx=0x7f916000c900 comp rx=0 tx=0).stop 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 -- 172.21.15.115:0/3194195744 shutdown_connections 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f914806a4c0 0x7f914806c980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f916815acf0 0x7f916806b8b0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f916815a910 0x7f916806b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 --2- 172.21.15.115:0/3194195744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91680cc670 0x7f916814ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 -- 172.21.15.115:0/3194195744 >> 172.21.15.115:0/3194195744 conn(0x7f9168099a20 msgr2=0x7f9168158eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 -- 172.21.15.115:0/3194195744 shutdown_connections 2023-12-01T02:51:30.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.111+0000 7f916d107640 1 -- 172.21.15.115:0/3194195744 wait complete. 2023-12-01T02:51:30.132 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T02:51:30.187 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T02:51:30.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.328+0000 7fdd7b449640 1 Processor -- start 2023-12-01T02:51:30.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.328+0000 7fdd7b449640 1 -- start start 2023-12-01T02:51:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.329+0000 7fdd7b449640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdd7415ac00 0x7fdd740ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.329+0000 7fdd7b449640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd740aede0 0x7fdd740cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.329+0000 7fdd7b449640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd740b04c0 0x7fdd740cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.329+0000 7fdd7b449640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd740b12b0 con 0x7fdd740b04c0 2023-12-01T02:51:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.329+0000 7fdd7b449640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7415afe0 con 0x7fdd7415ac00 2023-12-01T02:51:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.329+0000 7fdd7b449640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740b0a10 con 0x7fdd740aede0 2023-12-01T02:51:30.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.329+0000 7fdd7ac48640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd740b04c0 0x7fdd740cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:30.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.329+0000 7fdd7ac48640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd740b04c0 0x7fdd740cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34538/0 (socket says 172.21.15.115:34538) 2023-12-01T02:51:30.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.329+0000 7fdd7ac48640 1 -- 172.21.15.115:0/811799690 learned_addr learned my addr 172.21.15.115:0/811799690 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:30.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.330+0000 7fdd79c46640 1 --2- 172.21.15.115:0/811799690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd740aede0 0x7fdd740cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:30.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.330+0000 7fdd7ac48640 1 -- 172.21.15.115:0/811799690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd740aede0 msgr2=0x7fdd740cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:30.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.330+0000 7fdd7a447640 1 --2- 172.21.15.115:0/811799690 >> v2:172.21.15.142:3300/0 conn(0x7fdd7415ac00 0x7fdd740ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:30.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.330+0000 7fdd7ac48640 1 --2- 172.21.15.115:0/811799690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd740aede0 0x7fdd740cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.330+0000 7fdd7ac48640 1 -- 172.21.15.115:0/811799690 >> v2:172.21.15.142:3300/0 conn(0x7fdd7415ac00 msgr2=0x7fdd740ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:30.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.330+0000 7fdd7ac48640 1 --2- 172.21.15.115:0/811799690 >> v2:172.21.15.142:3300/0 conn(0x7fdd7415ac00 0x7fdd740ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.330+0000 7fdd7ac48640 1 -- 172.21.15.115:0/811799690 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd740b0cf0 con 0x7fdd740b04c0 2023-12-01T02:51:30.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.330+0000 7fdd79c46640 1 --2- 172.21.15.115:0/811799690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd740aede0 0x7fdd740cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:30.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.330+0000 7fdd7a447640 1 --2- 172.21.15.115:0/811799690 >> v2:172.21.15.142:3300/0 conn(0x7fdd7415ac00 0x7fdd740ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:30.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.331+0000 7fdd7ac48640 1 --2- 172.21.15.115:0/811799690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 0x7fdd740cb630 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fdd74066750 tx=0x7fdd6c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6268b21b0568f9fe server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:30.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.331+0000 7fdd79445640 1 -- 172.21.15.115:0/811799690 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c02c9f0 con 0x7fdd740b04c0 2023-12-01T02:51:30.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.331+0000 7fdd79445640 1 -- 172.21.15.115:0/811799690 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c02cb50 con 0x7fdd740b04c0 2023-12-01T02:51:30.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.331+0000 7fdd79445640 1 -- 172.21.15.115:0/811799690 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0343f0 con 0x7fdd740b04c0 2023-12-01T02:51:30.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.332+0000 7fdd7b449640 1 -- 172.21.15.115:0/811799690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 msgr2=0x7fdd740cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:30.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.332+0000 7fdd7b449640 1 --2- 172.21.15.115:0/811799690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 0x7fdd740cb630 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fdd74066750 tx=0x7fdd6c02be50 comp rx=0 tx=0).stop 2023-12-01T02:51:30.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.332+0000 7fdd7b449640 1 -- 172.21.15.115:0/811799690 shutdown_connections 2023-12-01T02:51:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.332+0000 7fdd7b449640 1 --2- 172.21.15.115:0/811799690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 0x7fdd740cb630 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.332+0000 7fdd7b449640 1 --2- 172.21.15.115:0/811799690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd740aede0 0x7fdd740cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.332+0000 7fdd7b449640 1 --2- 172.21.15.115:0/811799690 >> v2:172.21.15.142:3300/0 conn(0x7fdd7415ac00 0x7fdd740ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.332+0000 7fdd7b449640 1 -- 172.21.15.115:0/811799690 >> 172.21.15.115:0/811799690 conn(0x7fdd74065960 msgr2=0x7fdd74065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.332+0000 7fdd7b449640 1 -- 172.21.15.115:0/811799690 shutdown_connections 2023-12-01T02:51:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.332+0000 7fdd7b449640 1 -- 172.21.15.115:0/811799690 wait complete. 2023-12-01T02:51:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.334+0000 7fdd7b449640 1 Processor -- start 2023-12-01T02:51:30.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.334+0000 7fdd7b449640 1 -- start start 2023-12-01T02:51:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.335+0000 7fdd7b449640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd740aede0 0x7fdd740cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.335+0000 7fdd7b449640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 0x7fdd7414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.335+0000 7fdd7b449640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd7415ac00 0x7fdd7414ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.335+0000 7fdd7b449640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7414e550 con 0x7fdd740b04c0 2023-12-01T02:51:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.335+0000 7fdd7b449640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7414f3b0 con 0x7fdd7415ac00 2023-12-01T02:51:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.335+0000 7fdd7b449640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7414e300 con 0x7fdd740aede0 2023-12-01T02:51:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.335+0000 7fdd79c46640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 0x7fdd7414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.335+0000 7fdd79c46640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 0x7fdd7414f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34544/0 (socket says 172.21.15.115:34544) 2023-12-01T02:51:30.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.335+0000 7fdd79c46640 1 -- 172.21.15.115:0/3740494712 learned_addr learned my addr 172.21.15.115:0/3740494712 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd79c46640 1 -- 172.21.15.115:0/3740494712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd740aede0 msgr2=0x7fdd740cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:30.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd79c46640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd740aede0 0x7fdd740cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd79c46640 1 -- 172.21.15.115:0/3740494712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd7415ac00 msgr2=0x7fdd7414ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd79c46640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd7415ac00 0x7fdd7414ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd79c46640 1 -- 172.21.15.115:0/3740494712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c009b30 con 0x7fdd740b04c0 2023-12-01T02:51:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd79c46640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 0x7fdd7414f840 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c7f0 tx=0x7fdd6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd6affd640 1 -- 172.21.15.115:0/3740494712 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6400cea0 con 0x7fdd740b04c0 2023-12-01T02:51:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd6affd640 1 -- 172.21.15.115:0/3740494712 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd640079d0 con 0x7fdd740b04c0 2023-12-01T02:51:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd6affd640 1 -- 172.21.15.115:0/3740494712 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64007d60 con 0x7fdd740b04c0 2023-12-01T02:51:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd7b449640 1 -- 172.21.15.115:0/3740494712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7414e7a0 con 0x7fdd740b04c0 2023-12-01T02:51:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.336+0000 7fdd7b449640 1 -- 172.21.15.115:0/3740494712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7414ed60 con 0x7fdd740b04c0 2023-12-01T02:51:30.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.338+0000 7fdd7b449640 1 -- 172.21.15.115:0/3740494712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd740618d0 con 0x7fdd740b04c0 2023-12-01T02:51:30.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.339+0000 7fdd6affd640 1 -- 172.21.15.115:0/3740494712 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 84) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fdd640113f0 con 0x7fdd740b04c0 2023-12-01T02:51:30.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.339+0000 7fdd6affd640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4f0 0x7fdd5806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:30.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.340+0000 7fdd6affd640 1 -- 172.21.15.115:0/3740494712 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6499+0+0 (secure 0 0 0) 0x7fdd64088e10 con 0x7fdd740b04c0 2023-12-01T02:51:30.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.340+0000 7fdd7a447640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4f0 0x7fdd5806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:30.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.340+0000 7fdd7a447640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4f0 0x7fdd5806c9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdd70006fd0 tx=0x7fdd70008040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:30.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.344+0000 7fdd6affd640 1 -- 172.21.15.115:0/3740494712 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdd64053fe0 con 0x7fdd740b04c0 2023-12-01T02:51:30.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:30.575+0000 7fdd7b449640 1 -- 172.21.15.115:0/3740494712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdd740b42e0 con 0x7fdd740b04c0 2023-12-01T02:51:31.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.114+0000 7fdd6affd640 1 -- 172.21.15.115:0/3740494712 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd64057600 con 0x7fdd740b04c0 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 -- 172.21.15.115:0/3740494712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4f0 msgr2=0x7fdd5806c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4f0 0x7fdd5806c9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdd70006fd0 tx=0x7fdd70008040 comp rx=0 tx=0).stop 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 -- 172.21.15.115:0/3740494712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 msgr2=0x7fdd7414f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 0x7fdd7414f840 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c7f0 tx=0x7fdd6400ccb0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 -- 172.21.15.115:0/3740494712 shutdown_connections 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdd5806a4f0 0x7fdd5806c9b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd7415ac00 0x7fdd7414ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd740b04c0 0x7fdd7414f840 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 --2- 172.21.15.115:0/3740494712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd740aede0 0x7fdd740cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 -- 172.21.15.115:0/3740494712 >> 172.21.15.115:0/3740494712 conn(0x7fdd74065960 msgr2=0x7fdd740cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:31.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 -- 172.21.15.115:0/3740494712 shutdown_connections 2023-12-01T02:51:31.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.118+0000 7fdd7b449640 1 -- 172.21.15.115:0/3740494712 wait complete. 2023-12-01T02:51:31.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T02:51:31.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.354+0000 7f33de5d6640 1 Processor -- start 2023-12-01T02:51:31.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.355+0000 7f33de5d6640 1 -- start start 2023-12-01T02:51:31.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.356+0000 7f33de5d6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d80cb800 0x7f33d81656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:31.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.356+0000 7f33de5d6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d815afe0 con 0x7f33d815ac00 2023-12-01T02:51:31.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.356+0000 7f33de5d6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d80b0a10 con 0x7f33d80b04c0 2023-12-01T02:51:31.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.356+0000 7f33de5d6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80b12b0 con 0x7f33d80cb800 2023-12-01T02:51:31.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.356+0000 7f33dd5d4640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f33d815ac00 0x7f33d80ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39978/0 (socket says 172.21.15.115:39978) 2023-12-01T02:51:31.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.356+0000 7f33dd5d4640 1 -- 172.21.15.115:0/2788093053 learned_addr learned my addr 172.21.15.115:0/2788093053 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:31.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434121840 0 0) 0x7f33d815afe0 con 0x7f33d815ac00 2023-12-01T02:51:31.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33c4003520 con 0x7f33d815ac00 2023-12-01T02:51:31.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33dddd5640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d80cb800 0x7f33d81656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:31.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2580842802 0 0) 0x7f33c4003520 con 0x7f33d815ac00 2023-12-01T02:51:31.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d80cb800 msgr2=0x7f33d81656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d80cb800 0x7f33d81656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 >> v1:172.21.15.142:6789/0 conn(0x7f33d80b04c0 legacy=0x7f33d80cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d80b0cf0 con 0x7f33d815ac00 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f33d4001690 con 0x7f33d815ac00 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 >> v1:172.21.15.115:6789/0 conn(0x7f33d815ac00 legacy=0x7f33d80ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 0x7f33c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d80cb800 0x7f33c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33c4004d00 0x7f33c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80b12b0 con 0x7f33c40039a0 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c400bb60 con 0x7f33c4004d00 2023-12-01T02:51:31.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.357+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c400bd20 con 0x7f33d80cb800 2023-12-01T02:51:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.358+0000 7f33dd5d4640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 0x7f33c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.358+0000 7f33dd5d4640 1 -- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d80cb800 msgr2=0x7f33c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.358+0000 7f33dcdd3640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d80cb800 0x7f33c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.358+0000 7f33dddd5640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33c4004d00 0x7f33c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.358+0000 7f33dd5d4640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d80cb800 0x7f33c40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.358+0000 7f33dd5d4640 1 -- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33c4004d00 msgr2=0x7f33c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.358+0000 7f33dd5d4640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33c4004d00 0x7f33c4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.358+0000 7f33dd5d4640 1 -- 172.21.15.115:0/2788093053 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c4003520 con 0x7f33c40039a0 2023-12-01T02:51:31.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.358+0000 7f33dddd5640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33c4004d00 0x7f33c4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:31.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.359+0000 7f33dd5d4640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 0x7f33c4003dc0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f33d815b170 tx=0x7f33c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f96e1f5376685f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.359+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4005ad0 con 0x7f33c40039a0 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.359+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d4005c30 con 0x7f33c40039a0 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.359+0000 7f33cffff640 1 -- 172.21.15.115:0/2788093053 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d402c890 con 0x7f33c40039a0 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.360+0000 7f33de5d6640 1 -- 172.21.15.115:0/2788093053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 msgr2=0x7f33c4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.360+0000 7f33de5d6640 1 --2- 172.21.15.115:0/2788093053 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 0x7f33c4003dc0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f33d815b170 tx=0x7f33c40057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.360+0000 7f33dddd5640 1 -- 172.21.15.115:0/2788093053 reap_dead start 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.360+0000 7f33de5d6640 1 -- 172.21.15.115:0/2788093053 shutdown_connections 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.360+0000 7f33de5d6640 1 -- 172.21.15.115:0/2788093053 >> 172.21.15.115:0/2788093053 conn(0x7f33d8065960 msgr2=0x7f33d8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.360+0000 7f33de5d6640 1 -- 172.21.15.115:0/2788093053 shutdown_connections 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.360+0000 7f33de5d6640 1 -- 172.21.15.115:0/2788093053 wait complete. 2023-12-01T02:51:31.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.362+0000 7f33de5d6640 1 Processor -- start 2023-12-01T02:51:31.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.362+0000 7f33de5d6640 1 -- start start 2023-12-01T02:51:31.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.363+0000 7f33de5d6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 0x7f33d8158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:31.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.363+0000 7f33de5d6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d81f1ed0 0x7f33d815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:31.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.363+0000 7f33de5d6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33d81f2350 0x7f33d815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:31.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.363+0000 7f33de5d6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d815bce0 con 0x7f33c40039a0 2023-12-01T02:51:31.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.363+0000 7f33de5d6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c6400 con 0x7f33d81f2350 2023-12-01T02:51:31.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.363+0000 7f33de5d6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c66e0 con 0x7f33d81f1ed0 2023-12-01T02:51:31.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.363+0000 7f33dd5d4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 0x7f33d8158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.363+0000 7f33dd5d4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 0x7f33d8158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34560/0 (socket says 172.21.15.115:34560) 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.363+0000 7f33dd5d4640 1 -- 172.21.15.115:0/93038316 learned_addr learned my addr 172.21.15.115:0/93038316 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33dcdd3640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d81f1ed0 0x7f33d815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33dddd5640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33d81f2350 0x7f33d815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33dd5d4640 1 -- 172.21.15.115:0/93038316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d81f1ed0 msgr2=0x7f33d815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33dd5d4640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d81f1ed0 0x7f33d815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33dd5d4640 1 -- 172.21.15.115:0/93038316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33d81f2350 msgr2=0x7f33d815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33dd5d4640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33d81f2350 0x7f33d815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33dd5d4640 1 -- 172.21.15.115:0/93038316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d80c69c0 con 0x7f33c40039a0 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33dd5d4640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 0x7f33d8158140 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f33d4000c00 tx=0x7f33d4001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33cdffb640 1 -- 172.21.15.115:0/93038316 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4039070 con 0x7f33c40039a0 2023-12-01T02:51:31.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33cdffb640 1 -- 172.21.15.115:0/93038316 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d402cca0 con 0x7f33c40039a0 2023-12-01T02:51:31.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33cdffb640 1 -- 172.21.15.115:0/93038316 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d403c040 con 0x7f33c40039a0 2023-12-01T02:51:31.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33de5d6640 1 -- 172.21.15.115:0/93038316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d80c6ca0 con 0x7f33c40039a0 2023-12-01T02:51:31.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.364+0000 7f33de5d6640 1 -- 172.21.15.115:0/93038316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d806fbd0 con 0x7f33c40039a0 2023-12-01T02:51:31.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.365+0000 7f33de5d6640 1 -- 172.21.15.115:0/93038316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d80618d0 con 0x7f33c40039a0 2023-12-01T02:51:31.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.366+0000 7f33cdffb640 1 -- 172.21.15.115:0/93038316 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f33d4032070 con 0x7f33c40039a0 2023-12-01T02:51:31.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.367+0000 7f33cdffb640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f33bc06a550 0x7f33bc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:31.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.367+0000 7f33cdffb640 1 -- 172.21.15.115:0/93038316 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f33d40aa250 con 0x7f33c40039a0 2023-12-01T02:51:31.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.368+0000 7f33dcdd3640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f33bc06a550 0x7f33bc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:31.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.368+0000 7f33dcdd3640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f33bc06a550 0x7f33bc06ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f33d80c7f90 tx=0x7f33c8009040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.372+0000 7f33cdffb640 1 -- 172.21.15.115:0/93038316 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f33d4075400 con 0x7f33c40039a0 2023-12-01T02:51:31.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.566+0000 7f33de5d6640 1 -- 172.21.15.115:0/93038316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f33d80cf5c0 con 0x7f33c40039a0 2023-12-01T02:51:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.939+0000 7f33cdffb640 1 -- 172.21.15.115:0/93038316 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) v1 ==== 181+0+0 (secure 0 0 0) 0x7f33d4078a20 con 0x7f33c40039a0 2023-12-01T02:51:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.944+0000 7f33de5d6640 1 -- 172.21.15.115:0/93038316 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f33bc06a550 msgr2=0x7f33bc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.944+0000 7f33de5d6640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f33bc06a550 0x7f33bc06ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f33d80c7f90 tx=0x7f33c8009040 comp rx=0 tx=0).stop 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.944+0000 7f33de5d6640 1 -- 172.21.15.115:0/93038316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 msgr2=0x7f33d8158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.944+0000 7f33de5d6640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 0x7f33d8158140 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f33d4000c00 tx=0x7f33d4001a20 comp rx=0 tx=0).stop 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.944+0000 7f33de5d6640 1 -- 172.21.15.115:0/93038316 shutdown_connections 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.944+0000 7f33de5d6640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f33bc06a550 0x7f33bc06ca10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.944+0000 7f33de5d6640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33d81f1ed0 0x7f33d815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.944+0000 7f33de5d6640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33d81f2350 0x7f33d815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.945+0000 7f33de5d6640 1 --2- 172.21.15.115:0/93038316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c40039a0 0x7f33d8158140 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.945+0000 7f33de5d6640 1 -- 172.21.15.115:0/93038316 >> 172.21.15.115:0/93038316 conn(0x7f33d8065960 msgr2=0x7f33d80ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:31.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.945+0000 7f33de5d6640 1 -- 172.21.15.115:0/93038316 shutdown_connections 2023-12-01T02:51:31.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:31.945+0000 7f33de5d6640 1 -- 172.21.15.115:0/93038316 wait complete. 2023-12-01T02:51:31.968 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T02:51:32.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.165+0000 7f034759e640 1 Processor -- start 2023-12-01T02:51:32.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.166+0000 7f034759e640 1 -- start start 2023-12-01T02:51:32.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f034759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f034815cb60 0x7f03480caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:32.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f034759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f03480cb430 0x7f03480c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:32.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f034759e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03480ca100 0x7f0348160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:32.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f034759e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0348070000 con 0x7f03480ca100 2023-12-01T02:51:32.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f034759e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03480702e0 con 0x7f03480cb430 2023-12-01T02:51:32.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f034759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03480705c0 con 0x7f034815cb60 2023-12-01T02:51:32.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f0346d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03480ca100 0x7f0348160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:32.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f0346d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f03480ca100 0x7f0348160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34568/0 (socket says 172.21.15.115:34568) 2023-12-01T02:51:32.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f0346d9d640 1 -- 172.21.15.115:0/922913505 learned_addr learned my addr 172.21.15.115:0/922913505 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f033dd9b640 1 --2- 172.21.15.115:0/922913505 >> v2:172.21.15.142:3300/0 conn(0x7f03480cb430 0x7f03480c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f0346d9d640 1 -- 172.21.15.115:0/922913505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f034815cb60 msgr2=0x7f03480caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f0346d9d640 1 --2- 172.21.15.115:0/922913505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f034815cb60 0x7f03480caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f0346d9d640 1 -- 172.21.15.115:0/922913505 >> v2:172.21.15.142:3300/0 conn(0x7f03480cb430 msgr2=0x7f03480c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f0346d9d640 1 --2- 172.21.15.115:0/922913505 >> v2:172.21.15.142:3300/0 conn(0x7f03480cb430 0x7f03480c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:32.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.167+0000 7f0346d9d640 1 -- 172.21.15.115:0/922913505 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03480708a0 con 0x7f03480ca100 2023-12-01T02:51:32.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.168+0000 7f0346d9d640 1 --2- 172.21.15.115:0/922913505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 0x7f0348160880 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f03480667f0 tx=0x7f034002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3d415cf62df3c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:32.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f0345d9b640 1 -- 172.21.15.115:0/922913505 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034002ca70 con 0x7f03480ca100 2023-12-01T02:51:32.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f0345d9b640 1 -- 172.21.15.115:0/922913505 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034002cbd0 con 0x7f03480ca100 2023-12-01T02:51:32.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f0345d9b640 1 -- 172.21.15.115:0/922913505 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400343f0 con 0x7f03480ca100 2023-12-01T02:51:32.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f034759e640 1 -- 172.21.15.115:0/922913505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 msgr2=0x7f0348160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:32.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f034759e640 1 --2- 172.21.15.115:0/922913505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 0x7f0348160880 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f03480667f0 tx=0x7f034002bed0 comp rx=0 tx=0).stop 2023-12-01T02:51:32.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f034759e640 1 -- 172.21.15.115:0/922913505 shutdown_connections 2023-12-01T02:51:32.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f034759e640 1 --2- 172.21.15.115:0/922913505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 0x7f0348160880 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:32.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f034759e640 1 --2- 172.21.15.115:0/922913505 >> v2:172.21.15.142:3300/0 conn(0x7f03480cb430 0x7f03480c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:32.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f034759e640 1 --2- 172.21.15.115:0/922913505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f034815cb60 0x7f03480caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:32.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f034759e640 1 -- 172.21.15.115:0/922913505 >> 172.21.15.115:0/922913505 conn(0x7f0348065960 msgr2=0x7f0348065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:32.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.169+0000 7f034759e640 1 -- 172.21.15.115:0/922913505 shutdown_connections 2023-12-01T02:51:32.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.170+0000 7f034759e640 1 -- 172.21.15.115:0/922913505 wait complete. 2023-12-01T02:51:32.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.171+0000 7f034759e640 1 Processor -- start 2023-12-01T02:51:32.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.171+0000 7f034759e640 1 -- start start 2023-12-01T02:51:32.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.172+0000 7f034759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 0x7f034815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:32.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.172+0000 7f034759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03480cb430 0x7f03481431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.172+0000 7f034759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f034815cb60 0x7f03480c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.172+0000 7f034759e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481436e0 con 0x7f03480ca100 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.172+0000 7f034759e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03481600c0 con 0x7f034815cb60 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.172+0000 7f034759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034815f4b0 con 0x7f03480cb430 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.172+0000 7f034659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 0x7f034815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.172+0000 7f034659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 0x7f034815ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34584/0 (socket says 172.21.15.115:34584) 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.172+0000 7f034659c640 1 -- 172.21.15.115:0/3180000704 learned_addr learned my addr 172.21.15.115:0/3180000704 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f034659c640 1 -- 172.21.15.115:0/3180000704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03480cb430 msgr2=0x7f03481431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f034659c640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03480cb430 0x7f03481431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f034659c640 1 -- 172.21.15.115:0/3180000704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f034815cb60 msgr2=0x7f03480c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f034659c640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f034815cb60 0x7f03480c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f034659c640 1 -- 172.21.15.115:0/3180000704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0340009b30 con 0x7f03480ca100 2023-12-01T02:51:32.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f034659c640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 0x7f034815ee10 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f033800b410 tx=0x7f033800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:32.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f033f7fe640 1 -- 172.21.15.115:0/3180000704 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338011830 con 0x7f03480ca100 2023-12-01T02:51:32.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f033f7fe640 1 -- 172.21.15.115:0/3180000704 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03380119b0 con 0x7f03480ca100 2023-12-01T02:51:32.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f033f7fe640 1 -- 172.21.15.115:0/3180000704 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338011d60 con 0x7f03480ca100 2023-12-01T02:51:32.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f034759e640 1 -- 172.21.15.115:0/3180000704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034815f790 con 0x7f03480ca100 2023-12-01T02:51:32.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.173+0000 7f034759e640 1 -- 172.21.15.115:0/3180000704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034815fd50 con 0x7f03480ca100 2023-12-01T02:51:32.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.174+0000 7f034759e640 1 -- 172.21.15.115:0/3180000704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0348068090 con 0x7f03480ca100 2023-12-01T02:51:32.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.175+0000 7f033f7fe640 1 -- 172.21.15.115:0/3180000704 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f033801a400 con 0x7f03480ca100 2023-12-01T02:51:32.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.177+0000 7f033f7fe640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f032806a550 0x7f032806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:32.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.177+0000 7f033f7fe640 1 -- 172.21.15.115:0/3180000704 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6076+0+0 (secure 0 0 0) 0x7f0338088e00 con 0x7f03480ca100 2023-12-01T02:51:32.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.177+0000 7f033dd9b640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f032806a550 0x7f032806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:32.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.178+0000 7f033dd9b640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f032806a550 0x7f032806ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0330006fd0 tx=0x7f0330008040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.182+0000 7f033f7fe640 1 -- 172.21.15.115:0/3180000704 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0338054170 con 0x7f03480ca100 2023-12-01T02:51:32.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:32.378+0000 7f034759e640 1 -- 172.21.15.115:0/3180000704 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03480618d0 con 0x7f03480ca100 2023-12-01T02:51:33.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.252+0000 7f033f7fe640 1 -- 172.21.15.115:0/3180000704 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0338015020 con 0x7f03480ca100 2023-12-01T02:51:33.254 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T02:51:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 -- 172.21.15.115:0/3180000704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f032806a550 msgr2=0x7f032806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f032806a550 0x7f032806ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0330006fd0 tx=0x7f0330008040 comp rx=0 tx=0).stop 2023-12-01T02:51:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 -- 172.21.15.115:0/3180000704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 msgr2=0x7f034815ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 0x7f034815ee10 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f033800b410 tx=0x7f033800b8d0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 -- 172.21.15.115:0/3180000704 shutdown_connections 2023-12-01T02:51:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f032806a550 0x7f032806ca10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03480cb430 0x7f03481431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f034815cb60 0x7f03480c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 --2- 172.21.15.115:0/3180000704 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03480ca100 0x7f034815ee10 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 -- 172.21.15.115:0/3180000704 >> 172.21.15.115:0/3180000704 conn(0x7f0348065960 msgr2=0x7f034806bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 -- 172.21.15.115:0/3180000704 shutdown_connections 2023-12-01T02:51:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.256+0000 7f034759e640 1 -- 172.21.15.115:0/3180000704 wait complete. 2023-12-01T02:51:33.277 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T02:51:33.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.490+0000 7fdc397df640 1 Processor -- start 2023-12-01T02:51:33.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.490+0000 7fdc397df640 1 -- start start 2023-12-01T02:51:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.491+0000 7fdc397df640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc34159a20 0x7fdc340c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.491+0000 7fdc397df640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc340cc140 0x7fdc340cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.491+0000 7fdc397df640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdc340cc5a0 0x7fdc340cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.491+0000 7fdc397df640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc340b27d0 con 0x7fdc34159a20 2023-12-01T02:51:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.491+0000 7fdc397df640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc340b2a90 con 0x7fdc340cc5a0 2023-12-01T02:51:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.491+0000 7fdc397df640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340cd380 con 0x7fdc340cc140 2023-12-01T02:51:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc33fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc34159a20 0x7fdc340c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc33fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdc34159a20 0x7fdc340c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34590/0 (socket says 172.21.15.115:34590) 2023-12-01T02:51:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc33fff640 1 -- 172.21.15.115:0/2127363203 learned_addr learned my addr 172.21.15.115:0/2127363203 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:33.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc33fff640 1 -- 172.21.15.115:0/2127363203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc340cc140 msgr2=0x7fdc340cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc38fde640 1 --2- 172.21.15.115:0/2127363203 >> v2:172.21.15.142:3300/0 conn(0x7fdc340cc5a0 0x7fdc340cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc2b7fe640 1 --2- 172.21.15.115:0/2127363203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc340cc140 0x7fdc340cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc33fff640 1 --2- 172.21.15.115:0/2127363203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc340cc140 0x7fdc340cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc33fff640 1 -- 172.21.15.115:0/2127363203 >> v2:172.21.15.142:3300/0 conn(0x7fdc340cc5a0 msgr2=0x7fdc340cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc33fff640 1 --2- 172.21.15.115:0/2127363203 >> v2:172.21.15.142:3300/0 conn(0x7fdc340cc5a0 0x7fdc340cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc33fff640 1 -- 172.21.15.115:0/2127363203 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc340ccae0 con 0x7fdc34159a20 2023-12-01T02:51:33.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.492+0000 7fdc38fde640 1 --2- 172.21.15.115:0/2127363203 >> v2:172.21.15.142:3300/0 conn(0x7fdc340cc5a0 0x7fdc340cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.493+0000 7fdc33fff640 1 --2- 172.21.15.115:0/2127363203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc34159a20 0x7fdc340c8bc0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fdc340b4160 tx=0x7fdc2402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5cb3388c8269a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.494+0000 7fdc337fe640 1 -- 172.21.15.115:0/2127363203 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2402ca70 con 0x7fdc34159a20 2023-12-01T02:51:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.494+0000 7fdc337fe640 1 -- 172.21.15.115:0/2127363203 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2402cbd0 con 0x7fdc34159a20 2023-12-01T02:51:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.494+0000 7fdc337fe640 1 -- 172.21.15.115:0/2127363203 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc240343f0 con 0x7fdc34159a20 2023-12-01T02:51:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.494+0000 7fdc397df640 1 -- 172.21.15.115:0/2127363203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc34159a20 msgr2=0x7fdc340c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.494+0000 7fdc397df640 1 --2- 172.21.15.115:0/2127363203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc34159a20 0x7fdc340c8bc0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fdc340b4160 tx=0x7fdc2402bed0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.494+0000 7fdc397df640 1 -- 172.21.15.115:0/2127363203 shutdown_connections 2023-12-01T02:51:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.494+0000 7fdc397df640 1 --2- 172.21.15.115:0/2127363203 >> v2:172.21.15.142:3300/0 conn(0x7fdc340cc5a0 0x7fdc340cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.494+0000 7fdc397df640 1 --2- 172.21.15.115:0/2127363203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc340cc140 0x7fdc340cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.494+0000 7fdc397df640 1 --2- 172.21.15.115:0/2127363203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc34159a20 0x7fdc340c8bc0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.495+0000 7fdc397df640 1 -- 172.21.15.115:0/2127363203 >> 172.21.15.115:0/2127363203 conn(0x7fdc340660d0 msgr2=0x7fdc3406b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.495+0000 7fdc397df640 1 -- 172.21.15.115:0/2127363203 shutdown_connections 2023-12-01T02:51:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.495+0000 7fdc397df640 1 -- 172.21.15.115:0/2127363203 wait complete. 2023-12-01T02:51:33.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.496+0000 7fdc397df640 1 Processor -- start 2023-12-01T02:51:33.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.497+0000 7fdc397df640 1 -- start start 2023-12-01T02:51:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.497+0000 7fdc397df640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc340cc140 0x7fdc340a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.497+0000 7fdc397df640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc340cc5a0 0x7fdc340b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.497+0000 7fdc397df640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc34159a20 0x7fdc340c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc397df640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340c4890 con 0x7fdc340cc5a0 2023-12-01T02:51:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc397df640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340a9040 con 0x7fdc34159a20 2023-12-01T02:51:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc397df640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34070a40 con 0x7fdc340cc140 2023-12-01T02:51:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc2b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc340cc5a0 0x7fdc340b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc2b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc340cc5a0 0x7fdc340b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34592/0 (socket says 172.21.15.115:34592) 2023-12-01T02:51:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc2b7fe640 1 -- 172.21.15.115:0/4117718791 learned_addr learned my addr 172.21.15.115:0/4117718791 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:33.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc2b7fe640 1 -- 172.21.15.115:0/4117718791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc340cc140 msgr2=0x7fdc340a8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc33fff640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc340cc140 0x7fdc340a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc38fde640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc34159a20 0x7fdc340c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc2b7fe640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc340cc140 0x7fdc340a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc2b7fe640 1 -- 172.21.15.115:0/4117718791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc34159a20 msgr2=0x7fdc340c2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc2b7fe640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc34159a20 0x7fdc340c2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.498+0000 7fdc2b7fe640 1 -- 172.21.15.115:0/4117718791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009b30 con 0x7fdc340cc5a0 2023-12-01T02:51:33.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.499+0000 7fdc2b7fe640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc340cc5a0 0x7fdc340b3190 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00c7f0 tx=0x7fdc1c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:33.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.499+0000 7fdc317fa640 1 -- 172.21.15.115:0/4117718791 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c004220 con 0x7fdc340cc5a0 2023-12-01T02:51:33.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.499+0000 7fdc317fa640 1 -- 172.21.15.115:0/4117718791 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c004380 con 0x7fdc340cc5a0 2023-12-01T02:51:33.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.499+0000 7fdc397df640 1 -- 172.21.15.115:0/4117718791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc34166610 con 0x7fdc340cc5a0 2023-12-01T02:51:33.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.499+0000 7fdc317fa640 1 -- 172.21.15.115:0/4117718791 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c007860 con 0x7fdc340cc5a0 2023-12-01T02:51:33.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.499+0000 7fdc397df640 1 -- 172.21.15.115:0/4117718791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc34166330 con 0x7fdc340cc5a0 2023-12-01T02:51:33.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.500+0000 7fdc397df640 1 -- 172.21.15.115:0/4117718791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc34067850 con 0x7fdc340cc5a0 2023-12-01T02:51:33.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.501+0000 7fdc317fa640 1 -- 172.21.15.115:0/4117718791 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fdc1c007bd0 con 0x7fdc340cc5a0 2023-12-01T02:51:33.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.502+0000 7fdc317fa640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc1406a550 0x7fdc1406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.502+0000 7fdc317fa640 1 -- 172.21.15.115:0/4117718791 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fdc1c088b20 con 0x7fdc340cc5a0 2023-12-01T02:51:33.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.502+0000 7fdc33fff640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc1406a550 0x7fdc1406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.503+0000 7fdc33fff640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc1406a550 0x7fdc1406ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdc24035000 tx=0x7fdc24030000 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:33.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.506+0000 7fdc317fa640 1 -- 172.21.15.115:0/4117718791 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdc1c053ec0 con 0x7fdc340cc5a0 2023-12-01T02:51:33.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.714+0000 7fdc397df640 1 -- 172.21.15.115:0/4117718791 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdc340618d0 con 0x7fdc340cc5a0 2023-12-01T02:51:33.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.721+0000 7fdc317fa640 1 -- 172.21.15.115:0/4117718791 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdc1c0574e0 con 0x7fdc340cc5a0 2023-12-01T02:51:33.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 -- 172.21.15.115:0/4117718791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc1406a550 msgr2=0x7fdc1406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc1406a550 0x7fdc1406ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdc24035000 tx=0x7fdc24030000 comp rx=0 tx=0).stop 2023-12-01T02:51:33.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 -- 172.21.15.115:0/4117718791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc340cc5a0 msgr2=0x7fdc340b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc340cc5a0 0x7fdc340b3190 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00c7f0 tx=0x7fdc1c00ccb0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 -- 172.21.15.115:0/4117718791 shutdown_connections 2023-12-01T02:51:33.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc1406a550 0x7fdc1406ca10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc34159a20 0x7fdc340c2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc340cc5a0 0x7fdc340b3190 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 --2- 172.21.15.115:0/4117718791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc340cc140 0x7fdc340a8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 -- 172.21.15.115:0/4117718791 >> 172.21.15.115:0/4117718791 conn(0x7fdc340660d0 msgr2=0x7fdc340b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:33.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 -- 172.21.15.115:0/4117718791 shutdown_connections 2023-12-01T02:51:33.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.724+0000 7fdc397df640 1 -- 172.21.15.115:0/4117718791 wait complete. 2023-12-01T02:51:33.744 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T02:51:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.944+0000 7f14ac3d9640 1 Processor -- start 2023-12-01T02:51:33.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.944+0000 7f14ac3d9640 1 -- start start 2023-12-01T02:51:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.946+0000 7f14ac3d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a40b0e60 0x7f14a40cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.946+0000 7f14ac3d9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a40cc9a0 con 0x7f14a415ac80 2023-12-01T02:51:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.946+0000 7f14ac3d9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a40cbd90 con 0x7f14a40ce1b0 2023-12-01T02:51:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.946+0000 7f14ac3d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a415b060 con 0x7f14a40b0e60 2023-12-01T02:51:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.946+0000 7f14aa175640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f14a415ac80 0x7f14a40cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39988/0 (socket says 172.21.15.115:39988) 2023-12-01T02:51:33.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.946+0000 7f14aa175640 1 -- 172.21.15.115:0/1513897290 learned_addr learned my addr 172.21.15.115:0/1513897290 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14aa976640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a40b0e60 0x7f14a40cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495750367 0 0) 0x7f14a40cc9a0 con 0x7f14a415ac80 2023-12-01T02:51:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1490003520 con 0x7f14a415ac80 2023-12-01T02:51:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4180961670 0 0) 0x7f1490003520 con 0x7f14a415ac80 2023-12-01T02:51:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a40b0e60 msgr2=0x7f14a40cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a40b0e60 0x7f14a40cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 >> v1:172.21.15.142:6789/0 conn(0x7f14a40ce1b0 legacy=0x7f14a406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a40cc070 con 0x7f14a415ac80 2023-12-01T02:51:33.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f14a0001670 con 0x7f14a415ac80 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 >> v1:172.21.15.115:6789/0 conn(0x7f14a415ac80 legacy=0x7f14a40cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14900039a0 0x7f1490003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a40b0e60 0x7f14900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1490004d00 0x7f1490005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a415b060 con 0x7f1490004d00 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149000ba30 con 0x7f14900039a0 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.947+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149000bba0 con 0x7f14a40b0e60 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.948+0000 7f14aa976640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1490004d00 0x7f1490005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.948+0000 7f14aa175640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14900039a0 0x7f1490003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.948+0000 7f14a9974640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a40b0e60 0x7f14900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.948+0000 7f14aa976640 1 -- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a40b0e60 msgr2=0x7f14900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:33.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.948+0000 7f14aa976640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a40b0e60 0x7f14900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.948+0000 7f14aa976640 1 -- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14900039a0 msgr2=0x7f1490003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.948+0000 7f14aa976640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14900039a0 0x7f1490003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.948+0000 7f14aa976640 1 -- 172.21.15.115:0/1513897290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1490003520 con 0x7f1490004d00 2023-12-01T02:51:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.949+0000 7f14aa976640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1490004d00 0x7f1490005180 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f149c009fc0 tx=0x7f14900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33beddaf209d59f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.950+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c02cc20 con 0x7f1490004d00 2023-12-01T02:51:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.950+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c00f040 con 0x7f1490004d00 2023-12-01T02:51:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.950+0000 7f14a9173640 1 -- 172.21.15.115:0/1513897290 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c034440 con 0x7f1490004d00 2023-12-01T02:51:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.950+0000 7f14ac3d9640 1 -- 172.21.15.115:0/1513897290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1490004d00 msgr2=0x7f1490005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.950+0000 7f14ac3d9640 1 --2- 172.21.15.115:0/1513897290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1490004d00 0x7f1490005180 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f149c009fc0 tx=0x7f14900057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.950+0000 7f14aa976640 1 -- 172.21.15.115:0/1513897290 reap_dead start 2023-12-01T02:51:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.950+0000 7f14ac3d9640 1 -- 172.21.15.115:0/1513897290 shutdown_connections 2023-12-01T02:51:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.950+0000 7f14ac3d9640 1 -- 172.21.15.115:0/1513897290 >> 172.21.15.115:0/1513897290 conn(0x7f14a4065960 msgr2=0x7f14a4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:33.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.950+0000 7f14ac3d9640 1 -- 172.21.15.115:0/1513897290 shutdown_connections 2023-12-01T02:51:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.950+0000 7f14ac3d9640 1 -- 172.21.15.115:0/1513897290 wait complete. 2023-12-01T02:51:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.952+0000 7f14ac3d9640 1 Processor -- start 2023-12-01T02:51:33.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.952+0000 7f14ac3d9640 1 -- start start 2023-12-01T02:51:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.953+0000 7f14ac3d9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1490004d00 0x7f14a41e4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.953+0000 7f14ac3d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a41e48b0 0x7f14a406f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.953+0000 7f14ac3d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a406f760 0x7f14a413ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.953+0000 7f14ac3d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a406fbf0 con 0x7f14a406f760 2023-12-01T02:51:33.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.953+0000 7f14ac3d9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41e4d70 con 0x7f1490004d00 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.953+0000 7f14ac3d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41e5050 con 0x7f14a41e48b0 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.953+0000 7f14aa976640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a406f760 0x7f14a413ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f14aa976640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a406f760 0x7f14a413ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34600/0 (socket says 172.21.15.115:34600) 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f14aa976640 1 -- 172.21.15.115:0/2296577026 learned_addr learned my addr 172.21.15.115:0/2296577026 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f14a9974640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a41e48b0 0x7f14a406f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f14aa175640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1490004d00 0x7f14a41e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f14aa976640 1 -- 172.21.15.115:0/2296577026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a41e48b0 msgr2=0x7f14a406f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f14aa976640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a41e48b0 0x7f14a406f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f14aa976640 1 -- 172.21.15.115:0/2296577026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1490004d00 msgr2=0x7f14a41e4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f14aa976640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1490004d00 0x7f14a41e4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f14aa976640 1 -- 172.21.15.115:0/2296577026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a41e5330 con 0x7f14a406f760 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f14aa976640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a406f760 0x7f14a413ef80 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f149c037040 tx=0x7f149c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f1496ffd640 1 -- 172.21.15.115:0/2296577026 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c043070 con 0x7f14a406f760 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f1496ffd640 1 -- 172.21.15.115:0/2296577026 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c0349c0 con 0x7f14a406f760 2023-12-01T02:51:33.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.954+0000 7f1496ffd640 1 -- 172.21.15.115:0/2296577026 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c034d70 con 0x7f14a406f760 2023-12-01T02:51:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.955+0000 7f14ac3d9640 1 -- 172.21.15.115:0/2296577026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a41e5610 con 0x7f14a406f760 2023-12-01T02:51:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.955+0000 7f14ac3d9640 1 -- 172.21.15.115:0/2296577026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a413f4c0 con 0x7f14a406f760 2023-12-01T02:51:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.956+0000 7f14ac3d9640 1 -- 172.21.15.115:0/2296577026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a41e5b30 con 0x7f14a406f760 2023-12-01T02:51:33.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.957+0000 7f1496ffd640 1 -- 172.21.15.115:0/2296577026 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f149c03e440 con 0x7f14a406f760 2023-12-01T02:51:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.958+0000 7f1496ffd640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a580 0x7f148806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.958+0000 7f1496ffd640 1 -- 172.21.15.115:0/2296577026 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f149c0aa570 con 0x7f14a406f760 2023-12-01T02:51:33.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.958+0000 7f14aa175640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a580 0x7f148806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:33.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.959+0000 7f14aa175640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a580 0x7f148806ca40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f14a0001eb0 tx=0x7f14a0000ed0 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:33.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:33.962+0000 7f1496ffd640 1 -- 172.21.15.115:0/2296577026 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f14a41e5b30 con 0x7f14a406f760 2023-12-01T02:51:34.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.158+0000 7f14ac3d9640 1 -- 172.21.15.115:0/2296577026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f14a41e5b30 con 0x7f14a406f760 2023-12-01T02:51:34.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.165+0000 7f1496ffd640 1 -- 172.21.15.115:0/2296577026 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v50) v1 ==== 120+0+0 (secure 0 0 0) 0x7f14a41e5b30 con 0x7f14a406f760 2023-12-01T02:51:34.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 -- 172.21.15.115:0/2296577026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a580 msgr2=0x7f148806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a580 0x7f148806ca40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f14a0001eb0 tx=0x7f14a0000ed0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 -- 172.21.15.115:0/2296577026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a406f760 msgr2=0x7f14a413ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a406f760 0x7f14a413ef80 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f149c037040 tx=0x7f149c009750 comp rx=0 tx=0).stop 2023-12-01T02:51:34.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 -- 172.21.15.115:0/2296577026 shutdown_connections 2023-12-01T02:51:34.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148806a580 0x7f148806ca40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a41e48b0 0x7f14a406f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a406f760 0x7f14a413ef80 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 --2- 172.21.15.115:0/2296577026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1490004d00 0x7f14a41e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 -- 172.21.15.115:0/2296577026 >> 172.21.15.115:0/2296577026 conn(0x7f14a4065960 msgr2=0x7f14a40705f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:34.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 -- 172.21.15.115:0/2296577026 shutdown_connections 2023-12-01T02:51:34.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.168+0000 7f14ac3d9640 1 -- 172.21.15.115:0/2296577026 wait complete. 2023-12-01T02:51:34.187 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T02:51:34.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.392+0000 7fd1dcf30640 1 Processor -- start 2023-12-01T02:51:34.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.392+0000 7fd1dcf30640 1 -- start start 2023-12-01T02:51:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.393+0000 7fd1dcf30640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd1d815a760 0x7fd1d8160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.393+0000 7fd1dcf30640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd1d80cb150 0x7fd1d80cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.393+0000 7fd1dcf30640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d815d790 con 0x7fd1d80b2710 2023-12-01T02:51:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1dcf30640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d80cbb00 con 0x7fd1d815a760 2023-12-01T02:51:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1dcf30640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d8160e40 con 0x7fd1d80cb150 2023-12-01T02:51:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1d77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd1d815a760 0x7fd1d8160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1d77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd1d815a760 0x7fd1d8160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34612/0 (socket says 172.21.15.115:34612) 2023-12-01T02:51:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1d77fe640 1 -- 172.21.15.115:0/2366966634 learned_addr learned my addr 172.21.15.115:0/2366966634 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1d7fff640 1 --2- 172.21.15.115:0/2366966634 >> v2:172.21.15.142:3300/0 conn(0x7fd1d80cb150 0x7fd1d80cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1d77fe640 1 -- 172.21.15.115:0/2366966634 >> v2:172.21.15.142:3300/0 conn(0x7fd1d80cb150 msgr2=0x7fd1d80cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1d77fe640 1 --2- 172.21.15.115:0/2366966634 >> v2:172.21.15.142:3300/0 conn(0x7fd1d80cb150 0x7fd1d80cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1d77fe640 1 -- 172.21.15.115:0/2366966634 >> v1:172.21.15.142:6789/0 conn(0x7fd1d80b2710 legacy=0x7fd1d815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1d77fe640 1 -- 172.21.15.115:0/2366966634 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d8161110 con 0x7fd1d815a760 2023-12-01T02:51:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.394+0000 7fd1d7fff640 1 --2- 172.21.15.115:0/2366966634 >> v2:172.21.15.142:3300/0 conn(0x7fd1d80cb150 0x7fd1d80cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.395+0000 7fd1d77fe640 1 --2- 172.21.15.115:0/2366966634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d815a760 0x7fd1d8160900 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b16b0 tx=0x7fd1c802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1867d8bc31ed76ad server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:34.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.396+0000 7fd1d6ffd640 1 -- 172.21.15.115:0/2366966634 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8031800 con 0x7fd1d815a760 2023-12-01T02:51:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.396+0000 7fd1d6ffd640 1 -- 172.21.15.115:0/2366966634 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c8031960 con 0x7fd1d815a760 2023-12-01T02:51:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.396+0000 7fd1d6ffd640 1 -- 172.21.15.115:0/2366966634 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8031cf0 con 0x7fd1d815a760 2023-12-01T02:51:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.396+0000 7fd1dcf30640 1 -- 172.21.15.115:0/2366966634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d815a760 msgr2=0x7fd1d8160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.396+0000 7fd1dcf30640 1 --2- 172.21.15.115:0/2366966634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d815a760 0x7fd1d8160900 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b16b0 tx=0x7fd1c802b8a0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.397+0000 7fd1dcf30640 1 -- 172.21.15.115:0/2366966634 shutdown_connections 2023-12-01T02:51:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.397+0000 7fd1dcf30640 1 --2- 172.21.15.115:0/2366966634 >> v2:172.21.15.142:3300/0 conn(0x7fd1d80cb150 0x7fd1d80cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.397+0000 7fd1dcf30640 1 --2- 172.21.15.115:0/2366966634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d815a760 0x7fd1d8160900 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.397+0000 7fd1dcf30640 1 -- 172.21.15.115:0/2366966634 >> 172.21.15.115:0/2366966634 conn(0x7fd1d8065960 msgr2=0x7fd1d8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.397+0000 7fd1dcf30640 1 -- 172.21.15.115:0/2366966634 shutdown_connections 2023-12-01T02:51:34.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.397+0000 7fd1dcf30640 1 -- 172.21.15.115:0/2366966634 wait complete. 2023-12-01T02:51:34.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.398+0000 7fd1dcf30640 1 Processor -- start 2023-12-01T02:51:34.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.399+0000 7fd1dcf30640 1 -- start start 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.399+0000 7fd1dcf30640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d80b2710 0x7fd1d8149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.399+0000 7fd1dcf30640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1d80cb150 0x7fd1d80af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.399+0000 7fd1dcf30640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1d815a760 0x7fd1d80b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1dcf30640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d8149750 con 0x7fd1d80b2710 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1dcf30640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d80afec0 con 0x7fd1d815a760 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1dcf30640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d80b01a0 con 0x7fd1d80cb150 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1d77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d80b2710 0x7fd1d8149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1d77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d80b2710 0x7fd1d8149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34618/0 (socket says 172.21.15.115:34618) 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1d77fe640 1 -- 172.21.15.115:0/3285625897 learned_addr learned my addr 172.21.15.115:0/3285625897 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1d77fe640 1 -- 172.21.15.115:0/3285625897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1d80cb150 msgr2=0x7fd1d80af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1d77fe640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1d80cb150 0x7fd1d80af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1d77fe640 1 -- 172.21.15.115:0/3285625897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1d815a760 msgr2=0x7fd1d80b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1d77fe640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1d815a760 0x7fd1d80b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.400+0000 7fd1d77fe640 1 -- 172.21.15.115:0/3285625897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c80094d0 con 0x7fd1d80b2710 2023-12-01T02:51:34.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.401+0000 7fd1d77fe640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d80b2710 0x7fd1d8149210 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8000c00 tx=0x7fd1c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:34.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.401+0000 7fd1d4ff9640 1 -- 172.21.15.115:0/3285625897 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c803ad30 con 0x7fd1d80b2710 2023-12-01T02:51:34.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.401+0000 7fd1d4ff9640 1 -- 172.21.15.115:0/3285625897 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c80443f0 con 0x7fd1d80b2710 2023-12-01T02:51:34.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.401+0000 7fd1d4ff9640 1 -- 172.21.15.115:0/3285625897 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80447a0 con 0x7fd1d80b2710 2023-12-01T02:51:34.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.401+0000 7fd1dcf30640 1 -- 172.21.15.115:0/3285625897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d80b0480 con 0x7fd1d80b2710 2023-12-01T02:51:34.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.401+0000 7fd1dcf30640 1 -- 172.21.15.115:0/3285625897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d80b0a40 con 0x7fd1d80b2710 2023-12-01T02:51:34.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.402+0000 7fd1dcf30640 1 -- 172.21.15.115:0/3285625897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d80b0c30 con 0x7fd1d80b2710 2023-12-01T02:51:34.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.403+0000 7fd1d4ff9640 1 -- 172.21.15.115:0/3285625897 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd1c804d450 con 0x7fd1d80b2710 2023-12-01T02:51:34.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.404+0000 7fd1d4ff9640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd1bc06a550 0x7fd1bc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.404+0000 7fd1d4ff9640 1 -- 172.21.15.115:0/3285625897 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fd1c802e070 con 0x7fd1d80b2710 2023-12-01T02:51:34.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.405+0000 7fd1ceffd640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd1bc06a550 0x7fd1bc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.405+0000 7fd1ceffd640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd1bc06a550 0x7fd1bc06ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd1d81f5640 tx=0x7fd1c0007990 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:34.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.409+0000 7fd1d4ff9640 1 -- 172.21.15.115:0/3285625897 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd1c80764b0 con 0x7fd1d80b2710 2023-12-01T02:51:34.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.598+0000 7fd1dcf30640 1 -- 172.21.15.115:0/3285625897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fd1d81616a0 con 0x7fd1d80b2710 2023-12-01T02:51:34.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.608+0000 7fd1d4ff9640 1 -- 172.21.15.115:0/3285625897 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v51) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd1c80350b0 con 0x7fd1d80b2710 2023-12-01T02:51:34.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.611+0000 7fd1dcf30640 1 -- 172.21.15.115:0/3285625897 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd1bc06a550 msgr2=0x7fd1bc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.611+0000 7fd1dcf30640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd1bc06a550 0x7fd1bc06ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd1d81f5640 tx=0x7fd1c0007990 comp rx=0 tx=0).stop 2023-12-01T02:51:34.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.611+0000 7fd1dcf30640 1 -- 172.21.15.115:0/3285625897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d80b2710 msgr2=0x7fd1d8149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.611+0000 7fd1dcf30640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d80b2710 0x7fd1d8149210 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8000c00 tx=0x7fd1c80046c0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.611+0000 7fd1dcf30640 1 -- 172.21.15.115:0/3285625897 shutdown_connections 2023-12-01T02:51:34.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.611+0000 7fd1dcf30640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd1bc06a550 0x7fd1bc06ca10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.611+0000 7fd1dcf30640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1d80cb150 0x7fd1d80af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.611+0000 7fd1dcf30640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1d815a760 0x7fd1d80b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.611+0000 7fd1dcf30640 1 --2- 172.21.15.115:0/3285625897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1d80b2710 0x7fd1d8149210 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.611+0000 7fd1dcf30640 1 -- 172.21.15.115:0/3285625897 >> 172.21.15.115:0/3285625897 conn(0x7fd1d8065960 msgr2=0x7fd1d815c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:34.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.612+0000 7fd1dcf30640 1 -- 172.21.15.115:0/3285625897 shutdown_connections 2023-12-01T02:51:34.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.612+0000 7fd1dcf30640 1 -- 172.21.15.115:0/3285625897 wait complete. 2023-12-01T02:51:34.634 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-12-01T02:51:34.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.842+0000 7f61ecf31640 1 Processor -- start 2023-12-01T02:51:34.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.843+0000 7f61ecf31640 1 -- start start 2023-12-01T02:51:34.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.844+0000 7f61ecf31640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f61e80cad70 0x7f61e806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.844+0000 7f61ecf31640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61e806f2f0 0x7f61e806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.844+0000 7f61ecf31640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e80add10 con 0x7f61e815e2f0 2023-12-01T02:51:34.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.844+0000 7f61ecf31640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e80c5760 con 0x7f61e80cad70 2023-12-01T02:51:34.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.844+0000 7f61ecf31640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e815e6d0 con 0x7f61e806f2f0 2023-12-01T02:51:34.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e659c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f61e815e2f0 0x7f61e80ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:39992/0 (socket says 172.21.15.115:39992) 2023-12-01T02:51:34.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e659c640 1 -- 172.21.15.115:0/961639637 learned_addr learned my addr 172.21.15.115:0/961639637 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:34.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875900508 0 0) 0x7f61e80add10 con 0x7f61e815e2f0 2023-12-01T02:51:34.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e6d9d640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61e806f2f0 0x7f61e806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61dffff640 1 --2- 172.21.15.115:0/961639637 >> v2:172.21.15.142:3300/0 conn(0x7f61e80cad70 0x7f61e806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d4003520 con 0x7f61e815e2f0 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 189162246 0 0) 0x7f61d4003520 con 0x7f61e815e2f0 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61e806f2f0 msgr2=0x7f61e806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e5d9b640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61e806f2f0 0x7f61e806bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 >> v2:172.21.15.142:3300/0 conn(0x7f61e80cad70 msgr2=0x7f61e806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e5d9b640 1 --2- 172.21.15.115:0/961639637 >> v2:172.21.15.142:3300/0 conn(0x7f61e80cad70 0x7f61e806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80cf7b0 con 0x7f61e815e2f0 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.845+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f61d8001670 con 0x7f61e815e2f0 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 >> v1:172.21.15.115:6789/0 conn(0x7f61e815e2f0 legacy=0x7f61e80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e5d9b640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61d40039e0 0x7f61d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e5d9b640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d40047c0 0x7f61d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e5d9b640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61d4005180 0x7f61d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80c5760 con 0x7f61d4005180 2023-12-01T02:51:34.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e815e6d0 con 0x7f61d40047c0 2023-12-01T02:51:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d400c0d0 con 0x7f61d40039e0 2023-12-01T02:51:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e6d9d640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61e806f2f0 0x7f61e806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T02:51:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e6d9d640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61d4005180 0x7f61d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61dffff640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d40047c0 0x7f61d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e659c640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61d40039e0 0x7f61d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e6d9d640 1 -- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61d40039e0 msgr2=0x7f61d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e6d9d640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61d40039e0 0x7f61d4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e6d9d640 1 -- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d40047c0 msgr2=0x7f61d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.846+0000 7f61e6d9d640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61d40047c0 0x7f61d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.847+0000 7f61e6d9d640 1 -- 172.21.15.115:0/961639637 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4003520 con 0x7f61d4005180 2023-12-01T02:51:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.848+0000 7f61e6d9d640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61d4005180 0x7f61d400b980 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f61e806f6d0 tx=0x7f61d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f32c9b941e9003f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.848+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e002cc60 con 0x7f61d4005180 2023-12-01T02:51:34.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.848+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e002cdc0 con 0x7f61d4005180 2023-12-01T02:51:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.848+0000 7f61e5d9b640 1 -- 172.21.15.115:0/961639637 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0003bb0 con 0x7f61d4005180 2023-12-01T02:51:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.848+0000 7f61ecf31640 1 -- 172.21.15.115:0/961639637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61d4005180 msgr2=0x7f61d400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.848+0000 7f61ecf31640 1 --2- 172.21.15.115:0/961639637 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61d4005180 0x7f61d400b980 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f61e806f6d0 tx=0x7f61d40057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.848+0000 7f61e6d9d640 1 -- 172.21.15.115:0/961639637 reap_dead start 2023-12-01T02:51:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.849+0000 7f61ecf31640 1 -- 172.21.15.115:0/961639637 shutdown_connections 2023-12-01T02:51:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.849+0000 7f61ecf31640 1 -- 172.21.15.115:0/961639637 >> 172.21.15.115:0/961639637 conn(0x7f61e80ce880 msgr2=0x7f61e80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.849+0000 7f61ecf31640 1 -- 172.21.15.115:0/961639637 shutdown_connections 2023-12-01T02:51:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.849+0000 7f61ecf31640 1 -- 172.21.15.115:0/961639637 wait complete. 2023-12-01T02:51:34.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.851+0000 7f61ecf31640 1 Processor -- start 2023-12-01T02:51:34.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.851+0000 7f61ecf31640 1 -- start start 2023-12-01T02:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61ecf31640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61d4005180 0x7f61e8150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61ecf31640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80c60d0 0x7f61e80c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61ecf31640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e81e35b0 0x7f61e81e3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61ecf31640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80c6a90 con 0x7f61e81e35b0 2023-12-01T02:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61ecf31640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e81e3f70 con 0x7f61e80c60d0 2023-12-01T02:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61ecf31640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80c8670 con 0x7f61d4005180 2023-12-01T02:51:34.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61e6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e81e35b0 0x7f61e81e3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61e6d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e81e35b0 0x7f61e81e3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34624/0 (socket says 172.21.15.115:34624) 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61e6d9d640 1 -- 172.21.15.115:0/4188563563 learned_addr learned my addr 172.21.15.115:0/4188563563 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61e659c640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61d4005180 0x7f61e8150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61dffff640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80c60d0 0x7f61e80c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61e6d9d640 1 -- 172.21.15.115:0/4188563563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61d4005180 msgr2=0x7f61e8150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61e6d9d640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61d4005180 0x7f61e8150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61e6d9d640 1 -- 172.21.15.115:0/4188563563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80c60d0 msgr2=0x7f61e80c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61e6d9d640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80c60d0 0x7f61e80c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61e6d9d640 1 -- 172.21.15.115:0/4188563563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e81e5230 con 0x7f61e81e35b0 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.852+0000 7f61dffff640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80c60d0 0x7f61e80c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:34.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.853+0000 7f61e6d9d640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e81e35b0 0x7f61e81e3a30 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f61e002c6d0 tx=0x7f61e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.853+0000 7f61deffd640 1 -- 172.21.15.115:0/4188563563 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0004300 con 0x7f61e81e35b0 2023-12-01T02:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.853+0000 7f61deffd640 1 -- 172.21.15.115:0/4188563563 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e0004460 con 0x7f61e81e35b0 2023-12-01T02:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.853+0000 7f61deffd640 1 -- 172.21.15.115:0/4188563563 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e00047f0 con 0x7f61e81e35b0 2023-12-01T02:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.853+0000 7f61ecf31640 1 -- 172.21.15.115:0/4188563563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e81e4190 con 0x7f61e81e35b0 2023-12-01T02:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.853+0000 7f61ecf31640 1 -- 172.21.15.115:0/4188563563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e81e4750 con 0x7f61e81e35b0 2023-12-01T02:51:34.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.854+0000 7f61ecf31640 1 -- 172.21.15.115:0/4188563563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e806fda0 con 0x7f61e81e35b0 2023-12-01T02:51:34.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.855+0000 7f61deffd640 1 -- 172.21.15.115:0/4188563563 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f61e0037460 con 0x7f61e81e35b0 2023-12-01T02:51:34.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.856+0000 7f61deffd640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61c806a550 0x7f61c806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:34.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.856+0000 7f61deffd640 1 -- 172.21.15.115:0/4188563563 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f61e00a9ff0 con 0x7f61e81e35b0 2023-12-01T02:51:34.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.856+0000 7f61e659c640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61c806a550 0x7f61c806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:34.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.857+0000 7f61e659c640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61c806a550 0x7f61c806ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61d8001eb0 tx=0x7f61d8009040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:34.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:34.861+0000 7f61deffd640 1 -- 172.21.15.115:0/4188563563 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f61e0075570 con 0x7f61e81e35b0 2023-12-01T02:51:35.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.126+0000 7f61ecf31640 1 -- 172.21.15.115:0/4188563563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f61e804e890 con 0x7f61e81e35b0 2023-12-01T02:51:35.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.261+0000 7f61deffd640 1 -- 172.21.15.115:0/4188563563 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2445) v1 ==== 178+0+0 (secure 0 0 0) 0x7f61e0078b90 con 0x7f61e81e35b0 2023-12-01T02:51:35.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.265+0000 7f61ecf31640 1 -- 172.21.15.115:0/4188563563 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61c806a550 msgr2=0x7f61c806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:35.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.265+0000 7f61ecf31640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61c806a550 0x7f61c806ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61d8001eb0 tx=0x7f61d8009040 comp rx=0 tx=0).stop 2023-12-01T02:51:35.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.265+0000 7f61ecf31640 1 -- 172.21.15.115:0/4188563563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e81e35b0 msgr2=0x7f61e81e3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:35.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.265+0000 7f61ecf31640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e81e35b0 0x7f61e81e3a30 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f61e002c6d0 tx=0x7f61e0004ab0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.265+0000 7f61ecf31640 1 -- 172.21.15.115:0/4188563563 shutdown_connections 2023-12-01T02:51:35.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.265+0000 7f61ecf31640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f61c806a550 0x7f61c806ca10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.265+0000 7f61ecf31640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61e81e35b0 0x7f61e81e3a30 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.265+0000 7f61ecf31640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61e80c60d0 0x7f61e80c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.265+0000 7f61ecf31640 1 --2- 172.21.15.115:0/4188563563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61d4005180 0x7f61e8150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.266+0000 7f61ecf31640 1 -- 172.21.15.115:0/4188563563 >> 172.21.15.115:0/4188563563 conn(0x7f61e80ce880 msgr2=0x7f61e80c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:35.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.266+0000 7f61ecf31640 1 -- 172.21.15.115:0/4188563563 shutdown_connections 2023-12-01T02:51:35.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.266+0000 7f61ecf31640 1 -- 172.21.15.115:0/4188563563 wait complete. 2023-12-01T02:51:35.292 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-12-01T02:51:35.292 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-12-01T02:51:35.292 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-01T02:51:35.292 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-12-01T02:51:35.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.507+0000 7faabd028640 1 Processor -- start 2023-12-01T02:51:35.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.508+0000 7faabd028640 1 -- start start 2023-12-01T02:51:35.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.509+0000 7faabd028640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab815e2f0 0x7faab80ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:35.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.509+0000 7faabd028640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faab806f2f0 0x7faab80caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:35.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.509+0000 7faabd028640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab80add10 con 0x7faab80adf10 2023-12-01T02:51:35.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.509+0000 7faabd028640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab80c5760 con 0x7faab806f2f0 2023-12-01T02:51:35.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.509+0000 7faabd028640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab815e6d0 con 0x7faab815e2f0 2023-12-01T02:51:35.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.509+0000 7faaaffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faab806f2f0 0x7faab80caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:35.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.509+0000 7faab759e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7faab80adf10 0x7faab80c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40004/0 (socket says 172.21.15.115:40004) 2023-12-01T02:51:35.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.510+0000 7faab759e640 1 -- 172.21.15.115:0/1705040715 learned_addr learned my addr 172.21.15.115:0/1705040715 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:35.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.510+0000 7faab6d9d640 1 --2- 172.21.15.115:0/1705040715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab815e2f0 0x7faab80ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:35.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.510+0000 7faaaffff640 1 -- 172.21.15.115:0/1705040715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab815e2f0 msgr2=0x7faab80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:35.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.510+0000 7faaaffff640 1 --2- 172.21.15.115:0/1705040715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab815e2f0 0x7faab80ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.510+0000 7faaaffff640 1 -- 172.21.15.115:0/1705040715 >> v1:172.21.15.115:6789/0 conn(0x7faab80adf10 legacy=0x7faab80c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:35.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.510+0000 7faaaffff640 1 -- 172.21.15.115:0/1705040715 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab80cf7b0 con 0x7faab806f2f0 2023-12-01T02:51:35.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.510+0000 7faab659c640 1 -- 172.21.15.115:0/1705040715 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242222804 0 0) 0x7faab80add10 con 0x7faab80adf10 2023-12-01T02:51:35.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.511+0000 7faaaffff640 1 --2- 172.21.15.115:0/1705040715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab806f2f0 0x7faab80caba0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7faaa00097a0 tx=0x7faaa002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03f3719cb7afd7e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:35.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faab659c640 1 -- 172.21.15.115:0/1705040715 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa002f050 con 0x7faab806f2f0 2023-12-01T02:51:35.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faab659c640 1 -- 172.21.15.115:0/1705040715 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa0031700 con 0x7faab806f2f0 2023-12-01T02:51:35.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faab659c640 1 -- 172.21.15.115:0/1705040715 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0031a90 con 0x7faab806f2f0 2023-12-01T02:51:35.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faabd028640 1 -- 172.21.15.115:0/1705040715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab806f2f0 msgr2=0x7faab80caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:35.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faabd028640 1 --2- 172.21.15.115:0/1705040715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab806f2f0 0x7faab80caba0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7faaa00097a0 tx=0x7faaa002b7a0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faabd028640 1 -- 172.21.15.115:0/1705040715 shutdown_connections 2023-12-01T02:51:35.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faabd028640 1 --2- 172.21.15.115:0/1705040715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab806f2f0 0x7faab80caba0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faabd028640 1 --2- 172.21.15.115:0/1705040715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab815e2f0 0x7faab80ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faabd028640 1 -- 172.21.15.115:0/1705040715 >> 172.21.15.115:0/1705040715 conn(0x7faab80ce880 msgr2=0x7faab80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:35.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faabd028640 1 -- 172.21.15.115:0/1705040715 shutdown_connections 2023-12-01T02:51:35.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.512+0000 7faabd028640 1 -- 172.21.15.115:0/1705040715 wait complete. 2023-12-01T02:51:35.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.514+0000 7faabd028640 1 Processor -- start 2023-12-01T02:51:35.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.514+0000 7faabd028640 1 -- start start 2023-12-01T02:51:35.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.515+0000 7faabd028640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab806f2f0 0x7faab8151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:35.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.515+0000 7faabd028640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab80adf10 0x7faab80cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:35.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.515+0000 7faabd028640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab815e2f0 0x7faab80c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:35.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.515+0000 7faabd028640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab80c35f0 con 0x7faab815e2f0 2023-12-01T02:51:35.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.515+0000 7faabd028640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab80c2970 con 0x7faab806f2f0 2023-12-01T02:51:35.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.515+0000 7faabd028640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab80c2c50 con 0x7faab80adf10 2023-12-01T02:51:35.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faab759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab815e2f0 0x7faab80c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:35.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faab759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab815e2f0 0x7faab80c5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34642/0 (socket says 172.21.15.115:34642) 2023-12-01T02:51:35.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faab759e640 1 -- 172.21.15.115:0/2271220585 learned_addr learned my addr 172.21.15.115:0/2271220585 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:35.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faab6d9d640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab806f2f0 0x7faab8151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:35.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faaaffff640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab80adf10 0x7faab80cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:35.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faab759e640 1 -- 172.21.15.115:0/2271220585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab80adf10 msgr2=0x7faab80cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:35.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faab759e640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab80adf10 0x7faab80cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faab759e640 1 -- 172.21.15.115:0/2271220585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab806f2f0 msgr2=0x7faab8151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:35.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faab759e640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab806f2f0 0x7faab8151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:35.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faab759e640 1 -- 172.21.15.115:0/2271220585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0009400 con 0x7faab815e2f0 2023-12-01T02:51:35.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.516+0000 7faab759e640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab815e2f0 0x7faab80c5f10 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7faab000c6d0 tx=0x7faab000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:35.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.517+0000 7faaaf7fe640 1 -- 172.21.15.115:0/2271220585 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0004d60 con 0x7faab815e2f0 2023-12-01T02:51:35.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.517+0000 7faaaf7fe640 1 -- 172.21.15.115:0/2271220585 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab000a640 con 0x7faab815e2f0 2023-12-01T02:51:35.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.517+0000 7faaaf7fe640 1 -- 172.21.15.115:0/2271220585 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab000aa70 con 0x7faab815e2f0 2023-12-01T02:51:35.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.517+0000 7faabd028640 1 -- 172.21.15.115:0/2271220585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab80c2f30 con 0x7faab815e2f0 2023-12-01T02:51:35.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.517+0000 7faabd028640 1 -- 172.21.15.115:0/2271220585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab80cd9d0 con 0x7faab815e2f0 2023-12-01T02:51:35.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.518+0000 7faabd028640 1 -- 172.21.15.115:0/2271220585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab806fda0 con 0x7faab815e2f0 2023-12-01T02:51:35.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.519+0000 7faaaf7fe640 1 -- 172.21.15.115:0/2271220585 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7faab00070d0 con 0x7faab815e2f0 2023-12-01T02:51:35.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.520+0000 7faaaf7fe640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa9806a580 0x7faa9806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:35.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.520+0000 7faaaf7fe640 1 -- 172.21.15.115:0/2271220585 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5589+0+0 (secure 0 0 0) 0x7faab0087fe0 con 0x7faab815e2f0 2023-12-01T02:51:35.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.520+0000 7faab6d9d640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa9806a580 0x7faa9806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:35.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.521+0000 7faab6d9d640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa9806a580 0x7faa9806ca40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006fd0 tx=0x7faaa8008040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.524+0000 7faaaf7fe640 1 -- 172.21.15.115:0/2271220585 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faab0053560 con 0x7faab815e2f0 2023-12-01T02:51:35.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:35.777+0000 7faabd028640 1 -- 172.21.15.115:0/2271220585 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7faab80c3730 con 0x7faab815e2f0 2023-12-01T02:51:36.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.265+0000 7faaaf7fe640 1 -- 172.21.15.115:0/2271220585 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2446) v1 ==== 177+0+0 (secure 0 0 0) 0x7faab80c3730 con 0x7faab815e2f0 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 -- 172.21.15.115:0/2271220585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa9806a580 msgr2=0x7faa9806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa9806a580 0x7faa9806ca40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faaa8006fd0 tx=0x7faaa8008040 comp rx=0 tx=0).stop 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 -- 172.21.15.115:0/2271220585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab815e2f0 msgr2=0x7faab80c5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab815e2f0 0x7faab80c5f10 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7faab000c6d0 tx=0x7faab000cb90 comp rx=0 tx=0).stop 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 -- 172.21.15.115:0/2271220585 shutdown_connections 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faa9806a580 0x7faa9806ca40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab80adf10 0x7faab80cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab815e2f0 0x7faab80c5f10 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 --2- 172.21.15.115:0/2271220585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab806f2f0 0x7faab8151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 -- 172.21.15.115:0/2271220585 >> 172.21.15.115:0/2271220585 conn(0x7faab80ce880 msgr2=0x7faab80c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:36.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 -- 172.21.15.115:0/2271220585 shutdown_connections 2023-12-01T02:51:36.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.268+0000 7faabd028640 1 -- 172.21.15.115:0/2271220585 wait complete. 2023-12-01T02:51:36.289 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T02:51:36.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.505+0000 7f6e3dec7640 1 Processor -- start 2023-12-01T02:51:36.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.505+0000 7f6e3dec7640 1 -- start start 2023-12-01T02:51:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.507+0000 7f6e3dec7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e380b0f80 0x7f6e380b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.507+0000 7f6e3dec7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e380cb290 con 0x7f6e380cbd30 2023-12-01T02:51:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.507+0000 7f6e3dec7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e380b0ae0 con 0x7f6e38160200 2023-12-01T02:51:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.507+0000 7f6e3dec7640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e380b35f0 con 0x7f6e380b0f80 2023-12-01T02:51:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.507+0000 7f6e36ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e380b0f80 0x7f6e380b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.508+0000 7f6e37fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6e380cbd30 0x7f6e380cc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40010/0 (socket says 172.21.15.115:40010) 2023-12-01T02:51:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.508+0000 7f6e37fff640 1 -- 172.21.15.115:0/94580088 learned_addr learned my addr 172.21.15.115:0/94580088 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.508+0000 7f6e36ffd640 1 -- 172.21.15.115:0/94580088 >> v1:172.21.15.142:6789/0 conn(0x7f6e38160200 legacy=0x7f6e3806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.508+0000 7f6e36ffd640 1 -- 172.21.15.115:0/94580088 >> v1:172.21.15.115:6789/0 conn(0x7f6e380cbd30 legacy=0x7f6e380cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:36.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.508+0000 7f6e36ffd640 1 -- 172.21.15.115:0/94580088 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e380b3310 con 0x7f6e380b0f80 2023-12-01T02:51:36.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.509+0000 7f6e36ffd640 1 --2- 172.21.15.115:0/94580088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 0x7f6e380b2c80 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0097a0 tx=0x7f6e2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1247f9849d7d4234 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:36.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.509+0000 7f6e367fc640 1 -- 172.21.15.115:0/94580088 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c031830 con 0x7f6e380b0f80 2023-12-01T02:51:36.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.509+0000 7f6e367fc640 1 -- 172.21.15.115:0/94580088 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c031990 con 0x7f6e380b0f80 2023-12-01T02:51:36.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.509+0000 7f6e367fc640 1 -- 172.21.15.115:0/94580088 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c031d20 con 0x7f6e380b0f80 2023-12-01T02:51:36.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.510+0000 7f6e3dec7640 1 -- 172.21.15.115:0/94580088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 msgr2=0x7f6e380b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:36.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.510+0000 7f6e3dec7640 1 --2- 172.21.15.115:0/94580088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 0x7f6e380b2c80 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0097a0 tx=0x7f6e2c02b8d0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.510+0000 7f6e3dec7640 1 -- 172.21.15.115:0/94580088 shutdown_connections 2023-12-01T02:51:36.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.510+0000 7f6e3dec7640 1 --2- 172.21.15.115:0/94580088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 0x7f6e380b2c80 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.510+0000 7f6e3dec7640 1 -- 172.21.15.115:0/94580088 >> 172.21.15.115:0/94580088 conn(0x7f6e38065960 msgr2=0x7f6e38065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:36.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.510+0000 7f6e3dec7640 1 -- 172.21.15.115:0/94580088 shutdown_connections 2023-12-01T02:51:36.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.510+0000 7f6e3dec7640 1 -- 172.21.15.115:0/94580088 wait complete. 2023-12-01T02:51:36.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.512+0000 7f6e3dec7640 1 Processor -- start 2023-12-01T02:51:36.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.512+0000 7f6e3dec7640 1 -- start start 2023-12-01T02:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.513+0000 7f6e3dec7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 0x7f6e3815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.513+0000 7f6e3dec7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e380cbd30 0x7f6e381fa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.513+0000 7f6e3dec7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e38160200 0x7f6e38154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.513+0000 7f6e3dec7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e38149540 con 0x7f6e380b0f80 2023-12-01T02:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.513+0000 7f6e3dec7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3815b650 con 0x7f6e38160200 2023-12-01T02:51:36.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.513+0000 7f6e3dec7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3815b910 con 0x7f6e380cbd30 2023-12-01T02:51:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.513+0000 7f6e377fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 0x7f6e3815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.513+0000 7f6e377fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 0x7f6e3815c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34658/0 (socket says 172.21.15.115:34658) 2023-12-01T02:51:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.513+0000 7f6e377fe640 1 -- 172.21.15.115:0/618475479 learned_addr learned my addr 172.21.15.115:0/618475479 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:36.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e377fe640 1 -- 172.21.15.115:0/618475479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e380cbd30 msgr2=0x7f6e381fa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e377fe640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e380cbd30 0x7f6e381fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e377fe640 1 -- 172.21.15.115:0/618475479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e38160200 msgr2=0x7f6e38154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e377fe640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e38160200 0x7f6e38154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e377fe640 1 -- 172.21.15.115:0/618475479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c009400 con 0x7f6e380b0f80 2023-12-01T02:51:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e377fe640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 0x7f6e3815c6d0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c430 tx=0x7f6e2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:36.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e1ffff640 1 -- 172.21.15.115:0/618475479 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28011790 con 0x7f6e380b0f80 2023-12-01T02:51:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e1ffff640 1 -- 172.21.15.115:0/618475479 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e280118f0 con 0x7f6e380b0f80 2023-12-01T02:51:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e3dec7640 1 -- 172.21.15.115:0/618475479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3815bbf0 con 0x7f6e380b0f80 2023-12-01T02:51:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e1ffff640 1 -- 172.21.15.115:0/618475479 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28011ce0 con 0x7f6e380b0f80 2023-12-01T02:51:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.514+0000 7f6e3dec7640 1 -- 172.21.15.115:0/618475479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3815c1b0 con 0x7f6e380b0f80 2023-12-01T02:51:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.516+0000 7f6e3dec7640 1 -- 172.21.15.115:0/618475479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3804e890 con 0x7f6e380b0f80 2023-12-01T02:51:36.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.516+0000 7f6e1ffff640 1 -- 172.21.15.115:0/618475479 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6e2801c460 con 0x7f6e380b0f80 2023-12-01T02:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.518+0000 7f6e1ffff640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e2406a550 0x7f6e2406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.518+0000 7f6e1ffff640 1 -- 172.21.15.115:0/618475479 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f6e28088980 con 0x7f6e380b0f80 2023-12-01T02:51:36.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.518+0000 7f6e36ffd640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e2406a550 0x7f6e2406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:36.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.519+0000 7f6e36ffd640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e2406a550 0x7f6e2406ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c036000 tx=0x7f6e2c030000 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:36.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.522+0000 7f6e1ffff640 1 -- 172.21.15.115:0/618475479 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6e28053ed0 con 0x7f6e380b0f80 2023-12-01T02:51:36.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.711+0000 7f6e3dec7640 1 -- 172.21.15.115:0/618475479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6e380b1e20 con 0x7f6e380b0f80 2023-12-01T02:51:36.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.721+0000 7f6e1ffff640 1 -- 172.21.15.115:0/618475479 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6e280574f0 con 0x7f6e380b0f80 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 -- 172.21.15.115:0/618475479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e2406a550 msgr2=0x7f6e2406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e2406a550 0x7f6e2406ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c036000 tx=0x7f6e2c030000 comp rx=0 tx=0).stop 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 -- 172.21.15.115:0/618475479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 msgr2=0x7f6e3815c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 0x7f6e3815c6d0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c430 tx=0x7f6e2800c8f0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 -- 172.21.15.115:0/618475479 shutdown_connections 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6e2406a550 0x7f6e2406ca10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e380cbd30 0x7f6e381fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e38160200 0x7f6e38154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 --2- 172.21.15.115:0/618475479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e380b0f80 0x7f6e3815c6d0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 -- 172.21.15.115:0/618475479 >> 172.21.15.115:0/618475479 conn(0x7f6e38065960 msgr2=0x7f6e380ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:36.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 -- 172.21.15.115:0/618475479 shutdown_connections 2023-12-01T02:51:36.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:36.724+0000 7f6e3dec7640 1 -- 172.21.15.115:0/618475479 wait complete. 2023-12-01T02:51:36.745 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T02:51:36.745 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T02:51:36.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T02:51:36.989 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T02:51:36.989 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:51:36.989 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:51:37.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T02:51:37.050 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T02:51:37.050 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:51:37.108 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T02:51:37.108 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:51:37.108 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:51:37.168 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T02:51:37.168 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T02:51:37.168 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:51:37.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T02:51:37.224 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T02:51:37.224 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:51:37.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.449+0000 7f5a7759e640 1 Processor -- start 2023-12-01T02:51:37.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.450+0000 7f5a7759e640 1 -- start start 2023-12-01T02:51:37.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.451+0000 7f5a7759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5a780cdcc0 0x7f5a7806b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:37.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.451+0000 7f5a7759e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a780ce120 con 0x7f5a780adf00 2023-12-01T02:51:37.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.451+0000 7f5a7759e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7806b9c0 con 0x7f5a781597d0 2023-12-01T02:51:37.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.451+0000 7f5a7759e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7806b7b0 con 0x7f5a780cdcc0 2023-12-01T02:51:37.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.452+0000 7f5a76d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5a780adf00 0x7f5a780ca350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40024/0 (socket says 172.21.15.115:40024) 2023-12-01T02:51:37.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.452+0000 7f5a76d9d640 1 -- 172.21.15.115:0/1873977379 learned_addr learned my addr 172.21.15.115:0/1873977379 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.452+0000 7f5a6ffff640 1 --2- 172.21.15.115:0/1873977379 >> v2:172.21.15.142:3300/0 conn(0x7f5a780cdcc0 0x7f5a7806b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.452+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575422317 0 0) 0x7f5a780ce120 con 0x7f5a780adf00 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.452+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a64003520 con 0x7f5a780adf00 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.452+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3262840957 0 0) 0x7f5a64003520 con 0x7f5a780adf00 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.452+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 >> v2:172.21.15.142:3300/0 conn(0x7f5a780cdcc0 msgr2=0x7f5a7806b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.452+0000 7f5a75d9b640 1 --2- 172.21.15.115:0/1873977379 >> v2:172.21.15.142:3300/0 conn(0x7f5a780cdcc0 0x7f5a7806b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.452+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 >> v1:172.21.15.142:6789/0 conn(0x7f5a781597d0 legacy=0x7f5a7815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7806bca0 con 0x7f5a780adf00 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5a700019f0 con 0x7f5a780adf00 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 >> v1:172.21.15.115:6789/0 conn(0x7f5a780adf00 legacy=0x7f5a780ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a75d9b640 1 --2- 172.21.15.115:0/1873977379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a640039e0 0x7f5a64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a75d9b640 1 --2- 172.21.15.115:0/1873977379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a640047c0 0x7f5a64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:37.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a75d9b640 1 --2- 172.21.15.115:0/1873977379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a640051f0 0x7f5a6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:37.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7806b7b0 con 0x7f5a640051f0 2023-12-01T02:51:37.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6400c150 con 0x7f5a640039e0 2023-12-01T02:51:37.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6400e520 con 0x7f5a640047c0 2023-12-01T02:51:37.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a76d9d640 1 --2- 172.21.15.115:0/1873977379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a640051f0 0x7f5a6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:37.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a7659c640 1 --2- 172.21.15.115:0/1873977379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a640039e0 0x7f5a64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a76d9d640 1 -- 172.21.15.115:0/1873977379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a640047c0 msgr2=0x7f5a64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a76d9d640 1 --2- 172.21.15.115:0/1873977379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a640047c0 0x7f5a64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a76d9d640 1 -- 172.21.15.115:0/1873977379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a640039e0 msgr2=0x7f5a64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a76d9d640 1 --2- 172.21.15.115:0/1873977379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a640039e0 0x7f5a64003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.453+0000 7f5a76d9d640 1 -- 172.21.15.115:0/1873977379 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64003520 con 0x7f5a640051f0 2023-12-01T02:51:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.454+0000 7f5a76d9d640 1 -- 172.21.15.115:0/1873977379 reap_dead start 2023-12-01T02:51:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.455+0000 7f5a76d9d640 1 --2- 172.21.15.115:0/1873977379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a640051f0 0x7f5a6400b980 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000a830 tx=0x7f5a640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910f989ce99cd4b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:37.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.455+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a700072d0 con 0x7f5a640051f0 2023-12-01T02:51:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.455+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7002f070 con 0x7f5a640051f0 2023-12-01T02:51:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.455+0000 7f5a75d9b640 1 -- 172.21.15.115:0/1873977379 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a700076b0 con 0x7f5a640051f0 2023-12-01T02:51:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.455+0000 7f5a7759e640 1 -- 172.21.15.115:0/1873977379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a640051f0 msgr2=0x7f5a6400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.455+0000 7f5a7759e640 1 --2- 172.21.15.115:0/1873977379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a640051f0 0x7f5a6400b980 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000a830 tx=0x7f5a640057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.455+0000 7f5a7759e640 1 -- 172.21.15.115:0/1873977379 shutdown_connections 2023-12-01T02:51:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.455+0000 7f5a7759e640 1 --2- 172.21.15.115:0/1873977379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a640051f0 0x7f5a6400b980 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.455+0000 7f5a7759e640 1 -- 172.21.15.115:0/1873977379 >> 172.21.15.115:0/1873977379 conn(0x7f5a780ce640 msgr2=0x7f5a780cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.455+0000 7f5a7759e640 1 -- 172.21.15.115:0/1873977379 shutdown_connections 2023-12-01T02:51:37.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.456+0000 7f5a7759e640 1 -- 172.21.15.115:0/1873977379 wait complete. 2023-12-01T02:51:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.457+0000 7f5a7759e640 1 Processor -- start 2023-12-01T02:51:37.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.457+0000 7f5a7759e640 1 -- start start 2023-12-01T02:51:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.458+0000 7f5a7759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a640051f0 0x7f5a7815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.458+0000 7f5a7759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a7815a270 0x7f5a780ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:37.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.458+0000 7f5a7759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a780acdd0 0x7f5a78144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.458+0000 7f5a7759e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780ad250 con 0x7f5a7815a270 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.458+0000 7f5a7759e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7815c160 con 0x7f5a640051f0 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.458+0000 7f5a7759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7815c3e0 con 0x7f5a780acdd0 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a6ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a7815a270 0x7f5a780ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a6ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a7815a270 0x7f5a780ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34678/0 (socket says 172.21.15.115:34678) 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a6ffff640 1 -- 172.21.15.115:0/1432423966 learned_addr learned my addr 172.21.15.115:0/1432423966 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a7659c640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a640051f0 0x7f5a7815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a6ffff640 1 -- 172.21.15.115:0/1432423966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a780acdd0 msgr2=0x7f5a78144590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a6ffff640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a780acdd0 0x7f5a78144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a6ffff640 1 -- 172.21.15.115:0/1432423966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a640051f0 msgr2=0x7f5a7815bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a6ffff640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a640051f0 0x7f5a7815bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a6ffff640 1 -- 172.21.15.115:0/1432423966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7815c6c0 con 0x7f5a7815a270 2023-12-01T02:51:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a6ffff640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a7815a270 0x7f5a780ac890 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000e7a0 tx=0x7f5a6000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.459+0000 7f5a6effd640 1 -- 172.21.15.115:0/1432423966 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000eeb0 con 0x7f5a7815a270 2023-12-01T02:51:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.460+0000 7f5a6effd640 1 -- 172.21.15.115:0/1432423966 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6000c940 con 0x7f5a7815a270 2023-12-01T02:51:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.460+0000 7f5a7759e640 1 -- 172.21.15.115:0/1432423966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7815c9a0 con 0x7f5a7815a270 2023-12-01T02:51:37.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.460+0000 7f5a6effd640 1 -- 172.21.15.115:0/1432423966 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000cd10 con 0x7f5a7815a270 2023-12-01T02:51:37.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.460+0000 7f5a7759e640 1 -- 172.21.15.115:0/1432423966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a78141fd0 con 0x7f5a7815a270 2023-12-01T02:51:37.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.461+0000 7f5a7759e640 1 -- 172.21.15.115:0/1432423966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a78142190 con 0x7f5a7815a270 2023-12-01T02:51:37.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.462+0000 7f5a6effd640 1 -- 172.21.15.115:0/1432423966 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f5a60013470 con 0x7f5a7815a270 2023-12-01T02:51:37.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.463+0000 7f5a6effd640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a5806a550 0x7f5a5806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:37.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.463+0000 7f5a6effd640 1 -- 172.21.15.115:0/1432423966 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f5a60088c30 con 0x7f5a7815a270 2023-12-01T02:51:37.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.463+0000 7f5a7659c640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a5806a550 0x7f5a5806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:37.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.464+0000 7f5a7659c640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a5806a550 0x7f5a5806ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a68007fd0 tx=0x7f5a68009040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:37.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.467+0000 7f5a6effd640 1 -- 172.21.15.115:0/1432423966 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5a600540e0 con 0x7f5a7815a270 2023-12-01T02:51:37.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.693+0000 7f5a7759e640 1 -- 172.21.15.115:0/1432423966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a780673a0 con 0x7f5a7815a270 2023-12-01T02:51:37.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.694+0000 7f5a6effd640 1 -- 172.21.15.115:0/1432423966 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f5a60021df0 con 0x7f5a7815a270 2023-12-01T02:51:37.695 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:37.695 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3556763477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3556763477},{"type":"v1","addr":"172.21.15.142:6833","nonce":3556763477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5819,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3661095418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3661095418},{"type":"v1","addr":"172.21.15.142:6835","nonce":3661095418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":8472,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/3388585128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":3388585128},{"type":"v1","addr":"172.21.15.115:6835","nonce":3388585128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":8934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/4038415503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":4038415503},{"type":"v1","addr":"172.21.15.115:6837","nonce":4038415503}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[]} 2023-12-01T02:51:37.695 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 65 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 -- 172.21.15.115:0/1432423966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a5806a550 msgr2=0x7f5a5806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a5806a550 0x7f5a5806ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a68007fd0 tx=0x7f5a68009040 comp rx=0 tx=0).stop 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 -- 172.21.15.115:0/1432423966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a7815a270 msgr2=0x7f5a780ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a7815a270 0x7f5a780ac890 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000e7a0 tx=0x7f5a6000ec60 comp rx=0 tx=0).stop 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 -- 172.21.15.115:0/1432423966 shutdown_connections 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5a5806a550 0x7f5a5806ca10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a780acdd0 0x7f5a78144590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a7815a270 0x7f5a780ac890 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 --2- 172.21.15.115:0/1432423966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a640051f0 0x7f5a7815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 -- 172.21.15.115:0/1432423966 >> 172.21.15.115:0/1432423966 conn(0x7f5a780ce640 msgr2=0x7f5a781552d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 -- 172.21.15.115:0/1432423966 shutdown_connections 2023-12-01T02:51:37.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.697+0000 7f5a7759e640 1 -- 172.21.15.115:0/1432423966 wait complete. 2023-12-01T02:51:37.716 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T02:51:37.716 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T02:51:37.716 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T02:51:37.716 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T02:51:37.717 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T02:51:37.717 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T02:51:37.717 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T02:51:37.717 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T02:51:37.717 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T02:51:37.717 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T02:51:37.717 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T02:51:37.717 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T02:51:37.718 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T02:51:37.718 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T02:51:37.718 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T02:51:37.718 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T02:51:37.737 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T02:51:37.738 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T02:51:37.738 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T02:51:37.739 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T02:51:37.742 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T02:51:37.742 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T02:51:37.745 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T02:51:37.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T02:51:37.786 INFO:tasks.ceph.mds.c:Started 2023-12-01T02:51:37.787 DEBUG:teuthology.parallel:result is None 2023-12-01T02:51:37.794 INFO:tasks.ceph.mds.b:Started 2023-12-01T02:51:37.794 INFO:tasks.ceph.mds.a:Started 2023-12-01T02:51:37.794 DEBUG:teuthology.parallel:result is None 2023-12-01T02:51:37.794 DEBUG:teuthology.parallel:result is None 2023-12-01T02:51:37.802 INFO:tasks.ceph.mds.d:Started 2023-12-01T02:51:37.802 DEBUG:teuthology.parallel:result is None 2023-12-01T02:51:37.802 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T02:51:37.946 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T02:51:37.948 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T02:51:37.968 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T02:51:37.971 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T02:51:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.992+0000 7f115ffff640 1 Processor -- start 2023-12-01T02:51:37.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.993+0000 7f115ffff640 1 -- start start 2023-12-01T02:51:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.994+0000 7f115ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f116015c9c0 0x7f11600b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.994+0000 7f115ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11600b59d0 0x7f11600cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.994+0000 7f115ffff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11600b1f60 con 0x7f116015cda0 2023-12-01T02:51:37.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.994+0000 7f115ffff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11600d2860 con 0x7f116015c9c0 2023-12-01T02:51:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.994+0000 7f115ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11600d0ab0 con 0x7f11600b59d0 2023-12-01T02:51:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.994+0000 7f115f7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f116015cda0 0x7f11600b4470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38466/0 (socket says 172.21.15.115:38466) 2023-12-01T02:51:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.994+0000 7f115f7fe640 1 -- 172.21.15.115:0/661838245 learned_addr learned my addr 172.21.15.115:0/661838245 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.994+0000 7f115effd640 1 --2- 172.21.15.115:0/661838245 >> v2:172.21.15.142:3300/0 conn(0x7f116015c9c0 0x7f11600b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:37.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.994+0000 7f1157fff640 1 --2- 172.21.15.115:0/661838245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11600b59d0 0x7f11600cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.995+0000 7f115e7fc640 1 -- 172.21.15.115:0/661838245 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745640624 0 0) 0x7f11600b1f60 con 0x7f116015cda0 2023-12-01T02:51:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.995+0000 7f1157fff640 1 -- 172.21.15.115:0/661838245 >> v2:172.21.15.142:3300/0 conn(0x7f116015c9c0 msgr2=0x7f11600b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.995+0000 7f1157fff640 1 --2- 172.21.15.115:0/661838245 >> v2:172.21.15.142:3300/0 conn(0x7f116015c9c0 0x7f11600b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.995+0000 7f1157fff640 1 -- 172.21.15.115:0/661838245 >> v1:172.21.15.115:6789/0 conn(0x7f116015cda0 legacy=0x7f11600b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.995+0000 7f1157fff640 1 -- 172.21.15.115:0/661838245 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11600d0d90 con 0x7f11600b59d0 2023-12-01T02:51:37.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.996+0000 7f1157fff640 1 --2- 172.21.15.115:0/661838245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11600b59d0 0x7f11600cf3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f11480097a0 tx=0x7f114802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0cc2567122a4de8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.996+0000 7f115e7fc640 1 -- 172.21.15.115:0/661838245 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148030800 con 0x7f11600b59d0 2023-12-01T02:51:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.996+0000 7f115e7fc640 1 -- 172.21.15.115:0/661838245 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148030960 con 0x7f11600b59d0 2023-12-01T02:51:37.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.996+0000 7f115e7fc640 1 -- 172.21.15.115:0/661838245 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148030cf0 con 0x7f11600b59d0 2023-12-01T02:51:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.997+0000 7f115ffff640 1 -- 172.21.15.115:0/661838245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11600b59d0 msgr2=0x7f11600cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.997+0000 7f115ffff640 1 --2- 172.21.15.115:0/661838245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11600b59d0 0x7f11600cf3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f11480097a0 tx=0x7f114802b940 comp rx=0 tx=0).stop 2023-12-01T02:51:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.997+0000 7f115ffff640 1 -- 172.21.15.115:0/661838245 shutdown_connections 2023-12-01T02:51:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.997+0000 7f115ffff640 1 --2- 172.21.15.115:0/661838245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11600b59d0 0x7f11600cf3c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.997+0000 7f115ffff640 1 --2- 172.21.15.115:0/661838245 >> v2:172.21.15.142:3300/0 conn(0x7f116015c9c0 0x7f11600b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.997+0000 7f115ffff640 1 -- 172.21.15.115:0/661838245 >> 172.21.15.115:0/661838245 conn(0x7f1160065960 msgr2=0x7f1160065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.997+0000 7f115ffff640 1 -- 172.21.15.115:0/661838245 shutdown_connections 2023-12-01T02:51:37.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.997+0000 7f115ffff640 1 -- 172.21.15.115:0/661838245 wait complete. 2023-12-01T02:51:37.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.999+0000 7f115ffff640 1 Processor -- start 2023-12-01T02:51:37.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:37.999+0000 7f115ffff640 1 -- start start 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f115ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11600b59d0 0x7f11601f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f115ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f116015c9c0 0x7f116014d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f115ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f116015cda0 0x7f116015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f115ffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11601f68d0 con 0x7f11600b59d0 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f115ffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11601f6030 con 0x7f116015c9c0 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f115ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11601f6310 con 0x7f116015cda0 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f115effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11600b59d0 0x7f11601f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f115effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11600b59d0 0x7f11601f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39300/0 (socket says 172.21.15.115:39300) 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f115effd640 1 -- 172.21.15.115:0/4173907414 learned_addr learned my addr 172.21.15.115:0/4173907414 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f115f7fe640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f116015cda0 0x7f116015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:38.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.000+0000 7f1157fff640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f116015c9c0 0x7f116014d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.001+0000 7f115effd640 1 -- 172.21.15.115:0/4173907414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f116015cda0 msgr2=0x7f116015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:38.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.001+0000 7f115effd640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f116015cda0 0x7f116015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.001+0000 7f115effd640 1 -- 172.21.15.115:0/4173907414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f116015c9c0 msgr2=0x7f116014d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.001+0000 7f115effd640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f116015c9c0 0x7f116014d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.001+0000 7f115effd640 1 -- 172.21.15.115:0/4173907414 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1148009400 con 0x7f11600b59d0 2023-12-01T02:51:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.001+0000 7f115effd640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11600b59d0 0x7f11601f5a10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f115000e9b0 tx=0x7f115000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.001+0000 7f115cff9640 1 -- 172.21.15.115:0/4173907414 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150009670 con 0x7f11600b59d0 2023-12-01T02:51:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.001+0000 7f115cff9640 1 -- 172.21.15.115:0/4173907414 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115000c9a0 con 0x7f11600b59d0 2023-12-01T02:51:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.001+0000 7f115ffff640 1 -- 172.21.15.115:0/4173907414 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11601f65f0 con 0x7f11600b59d0 2023-12-01T02:51:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.002+0000 7f115cff9640 1 -- 172.21.15.115:0/4173907414 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115000cd50 con 0x7f11600b59d0 2023-12-01T02:51:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.002+0000 7f115ffff640 1 -- 172.21.15.115:0/4173907414 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116015e6e0 con 0x7f11600b59d0 2023-12-01T02:51:38.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.003+0000 7f115ffff640 1 -- 172.21.15.115:0/4173907414 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116015e8d0 con 0x7f11600b59d0 2023-12-01T02:51:38.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.004+0000 7f115cff9640 1 -- 172.21.15.115:0/4173907414 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1150013470 con 0x7f11600b59d0 2023-12-01T02:51:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.005+0000 7f115cff9640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114006a550 0x7f114006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.005+0000 7f115cff9640 1 -- 172.21.15.115:0/4173907414 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f1150088d00 con 0x7f11600b59d0 2023-12-01T02:51:38.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.005+0000 7f1157fff640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114006a550 0x7f114006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:38.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.006+0000 7f1157fff640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114006a550 0x7f114006ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1148036000 tx=0x7f1148008030 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:38.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.009+0000 7f115cff9640 1 -- 172.21.15.115:0/4173907414 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f11500541e0 con 0x7f11600b59d0 2023-12-01T02:51:38.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:38.213+0000 7f115ffff640 1 -- 172.21.15.115:0/4173907414 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f11600673a0 con 0x7f11600b59d0 2023-12-01T02:51:39.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.000+0000 7f115cff9640 1 -- 172.21.15.115:0/4173907414 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v159) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1150057800 con 0x7f11600b59d0 2023-12-01T02:51:39.002 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T02:51:39.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 -- 172.21.15.115:0/4173907414 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114006a550 msgr2=0x7f114006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114006a550 0x7f114006ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1148036000 tx=0x7f1148008030 comp rx=0 tx=0).stop 2023-12-01T02:51:39.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 -- 172.21.15.115:0/4173907414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11600b59d0 msgr2=0x7f11601f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11600b59d0 0x7f11601f5a10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f115000e9b0 tx=0x7f115000ee70 comp rx=0 tx=0).stop 2023-12-01T02:51:39.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 -- 172.21.15.115:0/4173907414 shutdown_connections 2023-12-01T02:51:39.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f114006a550 0x7f114006ca10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f116015cda0 0x7f116015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f116015c9c0 0x7f116014d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 --2- 172.21.15.115:0/4173907414 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11600b59d0 0x7f11601f5a10 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 -- 172.21.15.115:0/4173907414 >> 172.21.15.115:0/4173907414 conn(0x7f1160065960 msgr2=0x7f11600ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:39.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 -- 172.21.15.115:0/4173907414 shutdown_connections 2023-12-01T02:51:39.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.004+0000 7f115ffff640 1 -- 172.21.15.115:0/4173907414 wait complete. 2023-12-01T02:51:39.028 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T02:51:39.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.243+0000 7f0c98563640 1 Processor -- start 2023-12-01T02:51:39.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.243+0000 7f0c98563640 1 -- start start 2023-12-01T02:51:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.244+0000 7f0c98563640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0c90159dd0 0x7f0c900c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.244+0000 7f0c98563640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9006f8b0 con 0x7f0c90070d30 2023-12-01T02:51:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.244+0000 7f0c98563640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c900adcb0 con 0x7f0c9006b6f0 2023-12-01T02:51:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.244+0000 7f0c98563640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c900cd270 con 0x7f0c90159dd0 2023-12-01T02:51:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.245+0000 7f0c962ff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0c90159dd0 0x7f0c900c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.245+0000 7f0c962ff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0c90159dd0 0x7f0c900c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39304/0 (socket says 172.21.15.115:39304) 2023-12-01T02:51:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.245+0000 7f0c96b00640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0c90070d30 0x7f0c900ccb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38482/0 (socket says 172.21.15.115:38482) 2023-12-01T02:51:39.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.245+0000 7f0c962ff640 1 -- 172.21.15.115:0/313145040 learned_addr learned my addr 172.21.15.115:0/313145040 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:39.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.245+0000 7f0c962ff640 1 -- 172.21.15.115:0/313145040 >> v1:172.21.15.142:6789/0 conn(0x7f0c9006b6f0 legacy=0x7f0c900cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:39.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.245+0000 7f0c962ff640 1 -- 172.21.15.115:0/313145040 >> v1:172.21.15.115:6789/0 conn(0x7f0c90070d30 legacy=0x7f0c900ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.245+0000 7f0c962ff640 1 -- 172.21.15.115:0/313145040 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c900ad9d0 con 0x7f0c90159dd0 2023-12-01T02:51:39.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.245+0000 7f0c952fd640 1 -- 172.21.15.115:0/313145040 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081420482 0 0) 0x7f0c9006f8b0 con 0x7f0c90070d30 2023-12-01T02:51:39.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.246+0000 7f0c962ff640 1 --2- 172.21.15.115:0/313145040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 0x7f0c900c97e0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400bb10 tx=0x7f0c8402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3877dd8dab6a50 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:39.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.247+0000 7f0c952fd640 1 -- 172.21.15.115:0/313145040 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8402ee00 con 0x7f0c90159dd0 2023-12-01T02:51:39.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.247+0000 7f0c952fd640 1 -- 172.21.15.115:0/313145040 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c84036440 con 0x7f0c90159dd0 2023-12-01T02:51:39.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.247+0000 7f0c952fd640 1 -- 172.21.15.115:0/313145040 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c840367d0 con 0x7f0c90159dd0 2023-12-01T02:51:39.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.247+0000 7f0c98563640 1 -- 172.21.15.115:0/313145040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 msgr2=0x7f0c900c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.247+0000 7f0c98563640 1 --2- 172.21.15.115:0/313145040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 0x7f0c900c97e0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400bb10 tx=0x7f0c8402e050 comp rx=0 tx=0).stop 2023-12-01T02:51:39.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.247+0000 7f0c98563640 1 -- 172.21.15.115:0/313145040 shutdown_connections 2023-12-01T02:51:39.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.247+0000 7f0c98563640 1 --2- 172.21.15.115:0/313145040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 0x7f0c900c97e0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.247+0000 7f0c98563640 1 -- 172.21.15.115:0/313145040 >> 172.21.15.115:0/313145040 conn(0x7f0c900ce880 msgr2=0x7f0c900cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:39.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.247+0000 7f0c98563640 1 -- 172.21.15.115:0/313145040 shutdown_connections 2023-12-01T02:51:39.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.247+0000 7f0c98563640 1 -- 172.21.15.115:0/313145040 wait complete. 2023-12-01T02:51:39.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.249+0000 7f0c98563640 1 Processor -- start 2023-12-01T02:51:39.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.249+0000 7f0c98563640 1 -- start start 2023-12-01T02:51:39.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.250+0000 7f0c98563640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c9006b6f0 0x7f0c9015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.250+0000 7f0c98563640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c90070d30 0x7f0c9014da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.250+0000 7f0c98563640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 0x7f0c900c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.250+0000 7f0c98563640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900c35b0 con 0x7f0c90159dd0 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.250+0000 7f0c98563640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9014df90 con 0x7f0c90070d30 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.250+0000 7f0c98563640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c901ee710 con 0x7f0c9006b6f0 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.251+0000 7f0c96b00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 0x7f0c900c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.251+0000 7f0c96b00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 0x7f0c900c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39316/0 (socket says 172.21.15.115:39316) 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.251+0000 7f0c96b00640 1 -- 172.21.15.115:0/1727841198 learned_addr learned my addr 172.21.15.115:0/1727841198 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.251+0000 7f0c96b00640 1 -- 172.21.15.115:0/1727841198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c9006b6f0 msgr2=0x7f0c9015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.251+0000 7f0c96b00640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c9006b6f0 0x7f0c9015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.251+0000 7f0c96b00640 1 -- 172.21.15.115:0/1727841198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c90070d30 msgr2=0x7f0c9014da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.251+0000 7f0c96b00640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c90070d30 0x7f0c9014da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.251+0000 7f0c96b00640 1 -- 172.21.15.115:0/1727841198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8400b770 con 0x7f0c90159dd0 2023-12-01T02:51:39.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.251+0000 7f0c96b00640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 0x7f0c900c4c40 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800c410 tx=0x7f0c8800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:39.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.251+0000 7f0c82ffd640 1 -- 172.21.15.115:0/1727841198 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880074e0 con 0x7f0c90159dd0 2023-12-01T02:51:39.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.252+0000 7f0c82ffd640 1 -- 172.21.15.115:0/1727841198 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c88007640 con 0x7f0c90159dd0 2023-12-01T02:51:39.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.252+0000 7f0c98563640 1 -- 172.21.15.115:0/1727841198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c901ee9f0 con 0x7f0c90159dd0 2023-12-01T02:51:39.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.252+0000 7f0c82ffd640 1 -- 172.21.15.115:0/1727841198 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88004a20 con 0x7f0c90159dd0 2023-12-01T02:51:39.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.252+0000 7f0c98563640 1 -- 172.21.15.115:0/1727841198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c901eefb0 con 0x7f0c90159dd0 2023-12-01T02:51:39.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.253+0000 7f0c98563640 1 -- 172.21.15.115:0/1727841198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9004e890 con 0x7f0c90159dd0 2023-12-01T02:51:39.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.254+0000 7f0c82ffd640 1 -- 172.21.15.115:0/1727841198 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0c88004b80 con 0x7f0c90159dd0 2023-12-01T02:51:39.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.255+0000 7f0c82ffd640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0c7406a580 0x7f0c7406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.255+0000 7f0c82ffd640 1 -- 172.21.15.115:0/1727841198 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f0c88088550 con 0x7f0c90159dd0 2023-12-01T02:51:39.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.255+0000 7f0c962ff640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0c7406a580 0x7f0c7406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:39.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.256+0000 7f0c962ff640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0c7406a580 0x7f0c7406ca40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0c84038000 tx=0x7f0c84030000 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:39.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.260+0000 7f0c82ffd640 1 -- 172.21.15.115:0/1727841198 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0c88053cb0 con 0x7f0c90159dd0 2023-12-01T02:51:39.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.525+0000 7f0c98563640 1 -- 172.21.15.115:0/1727841198 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0c900c36f0 con 0x7f0c90159dd0 2023-12-01T02:51:39.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.526+0000 7f0c82ffd640 1 -- 172.21.15.115:0/1727841198 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v38) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0c88001060 con 0x7f0c90159dd0 2023-12-01T02:51:39.528 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:39.528 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T02:51:39.528 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T02:51:39.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.528 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T02:51:39.528 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T02:51:39.529 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.530 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:51:39.531 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.533 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.535 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T02:51:39.535 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T02:51:39.535 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.535 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:51:39.535 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:51:39.535 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.537 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T02:51:39.537 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T02:51:39.537 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.538 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:51:39.538 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:51:39.538 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.539 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T02:51:39.539 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T02:51:39.539 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.539 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:51:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:51:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.540 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T02:51:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T02:51:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:51:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T02:51:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T02:51:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T02:51:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T02:51:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:51:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T02:51:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T02:51:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T02:51:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T02:51:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T02:51:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.543 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:51:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T02:51:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T02:51:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T02:51:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T02:51:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T02:51:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:51:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T02:51:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T02:51:39.545 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T02:51:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T02:51:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T02:51:39.546 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T02:51:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T02:51:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T02:51:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.547 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T02:51:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T02:51:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T02:51:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T02:51:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:51:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T02:51:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T02:51:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:51:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T02:51:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T02:51:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:51:39.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:39.550 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T02:51:39.550 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T02:51:39.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 -- 172.21.15.115:0/1727841198 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0c7406a580 msgr2=0x7f0c7406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0c7406a580 0x7f0c7406ca40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0c84038000 tx=0x7f0c84030000 comp rx=0 tx=0).stop 2023-12-01T02:51:39.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 -- 172.21.15.115:0/1727841198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 msgr2=0x7f0c900c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 0x7f0c900c4c40 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800c410 tx=0x7f0c8800c8d0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 -- 172.21.15.115:0/1727841198 shutdown_connections 2023-12-01T02:51:39.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0c7406a580 0x7f0c7406ca40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c90159dd0 0x7f0c900c4c40 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c90070d30 0x7f0c9014da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 --2- 172.21.15.115:0/1727841198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c9006b6f0 0x7f0c9015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 -- 172.21.15.115:0/1727841198 >> 172.21.15.115:0/1727841198 conn(0x7f0c900ce880 msgr2=0x7f0c900cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 -- 172.21.15.115:0/1727841198 shutdown_connections 2023-12-01T02:51:39.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.530+0000 7f0c98563640 1 -- 172.21.15.115:0/1727841198 wait complete. 2023-12-01T02:51:39.554 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.763+0000 7fdcb9a0b640 1 Processor -- start 2023-12-01T02:51:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.763+0000 7fdcb9a0b640 1 -- start start 2023-12-01T02:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.765+0000 7fdcb9a0b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdcb415e2f0 0x7fdcb40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.765+0000 7fdcb9a0b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdcb406f2f0 0x7fdcb40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.765+0000 7fdcb9a0b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb40add10 con 0x7fdcb40adf10 2023-12-01T02:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.765+0000 7fdcb9a0b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb40c5760 con 0x7fdcb406f2f0 2023-12-01T02:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.765+0000 7fdcb9a0b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb415e6d0 con 0x7fdcb415e2f0 2023-12-01T02:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.765+0000 7fdcabfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdcb406f2f0 0x7fdcb40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.765+0000 7fdcb37fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fdcb40adf10 0x7fdcb40c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38498/0 (socket says 172.21.15.115:38498) 2023-12-01T02:51:39.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.765+0000 7fdcb37fe640 1 -- 172.21.15.115:0/763896041 learned_addr learned my addr 172.21.15.115:0/763896041 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:39.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.766+0000 7fdcb2ffd640 1 --2- 172.21.15.115:0/763896041 >> v2:172.21.15.142:3300/0 conn(0x7fdcb415e2f0 0x7fdcb40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:39.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.766+0000 7fdcabfff640 1 -- 172.21.15.115:0/763896041 >> v2:172.21.15.142:3300/0 conn(0x7fdcb415e2f0 msgr2=0x7fdcb40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:39.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.766+0000 7fdcabfff640 1 --2- 172.21.15.115:0/763896041 >> v2:172.21.15.142:3300/0 conn(0x7fdcb415e2f0 0x7fdcb40ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.766+0000 7fdcabfff640 1 -- 172.21.15.115:0/763896041 >> v1:172.21.15.115:6789/0 conn(0x7fdcb40adf10 legacy=0x7fdcb40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.766+0000 7fdcabfff640 1 -- 172.21.15.115:0/763896041 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb40cf7b0 con 0x7fdcb406f2f0 2023-12-01T02:51:39.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.767+0000 7fdcabfff640 1 --2- 172.21.15.115:0/763896041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb406f2f0 0x7fdcb40caba0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fdcb406ef60 tx=0x7fdc9c02b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb37d543904d912 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:39.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.767+0000 7fdcb27fc640 1 -- 172.21.15.115:0/763896041 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c031520 con 0x7fdcb406f2f0 2023-12-01T02:51:39.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.767+0000 7fdcb27fc640 1 -- 172.21.15.115:0/763896041 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c031680 con 0x7fdcb406f2f0 2023-12-01T02:51:39.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.767+0000 7fdcb27fc640 1 -- 172.21.15.115:0/763896041 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c031a10 con 0x7fdcb406f2f0 2023-12-01T02:51:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.768+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/763896041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb406f2f0 msgr2=0x7fdcb40caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.768+0000 7fdcb9a0b640 1 --2- 172.21.15.115:0/763896041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb406f2f0 0x7fdcb40caba0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fdcb406ef60 tx=0x7fdc9c02b7c0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.768+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/763896041 shutdown_connections 2023-12-01T02:51:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.768+0000 7fdcb9a0b640 1 --2- 172.21.15.115:0/763896041 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb406f2f0 0x7fdcb40caba0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.768+0000 7fdcb9a0b640 1 --2- 172.21.15.115:0/763896041 >> v2:172.21.15.142:3300/0 conn(0x7fdcb415e2f0 0x7fdcb40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.768+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/763896041 >> 172.21.15.115:0/763896041 conn(0x7fdcb40ce880 msgr2=0x7fdcb40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.768+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/763896041 shutdown_connections 2023-12-01T02:51:39.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.768+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/763896041 wait complete. 2023-12-01T02:51:39.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.770+0000 7fdcb9a0b640 1 Processor -- start 2023-12-01T02:51:39.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.770+0000 7fdcb9a0b640 1 -- start start 2023-12-01T02:51:39.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.771+0000 7fdcb9a0b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb406f2f0 0x7fdcb414a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.771+0000 7fdcb9a0b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb40adf10 0x7fdcb40c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.771+0000 7fdcb9a0b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb415e2f0 0x7fdcb40c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.771+0000 7fdcb9a0b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb414db10 con 0x7fdcb40adf10 2023-12-01T02:51:39.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.771+0000 7fdcb9a0b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb414cc20 con 0x7fdcb415e2f0 2023-12-01T02:51:39.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.771+0000 7fdcb9a0b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb414cf00 con 0x7fdcb406f2f0 2023-12-01T02:51:39.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.771+0000 7fdcabfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb40adf10 0x7fdcb40c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:39.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcabfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb40adf10 0x7fdcb40c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39328/0 (socket says 172.21.15.115:39328) 2023-12-01T02:51:39.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcabfff640 1 -- 172.21.15.115:0/3017834512 learned_addr learned my addr 172.21.15.115:0/3017834512 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:39.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcb37fe640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb415e2f0 0x7fdcb40c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:39.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcb2ffd640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb406f2f0 0x7fdcb414a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:39.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcabfff640 1 -- 172.21.15.115:0/3017834512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb406f2f0 msgr2=0x7fdcb414a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcabfff640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb406f2f0 0x7fdcb414a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcabfff640 1 -- 172.21.15.115:0/3017834512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb415e2f0 msgr2=0x7fdcb40c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcabfff640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb415e2f0 0x7fdcb40c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcabfff640 1 -- 172.21.15.115:0/3017834512 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c009400 con 0x7fdcb40adf10 2023-12-01T02:51:39.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcb37fe640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb415e2f0 0x7fdcb40c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:39.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.772+0000 7fdcabfff640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb40adf10 0x7fdcb40c7eb0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c000c00 tx=0x7fdc9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:39.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.773+0000 7fdcb0ff9640 1 -- 172.21.15.115:0/3017834512 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c033c50 con 0x7fdcb40adf10 2023-12-01T02:51:39.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.773+0000 7fdcb0ff9640 1 -- 172.21.15.115:0/3017834512 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c033dd0 con 0x7fdcb40adf10 2023-12-01T02:51:39.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.773+0000 7fdcb0ff9640 1 -- 172.21.15.115:0/3017834512 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c03c640 con 0x7fdcb40adf10 2023-12-01T02:51:39.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.773+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/3017834512 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb414d1e0 con 0x7fdcb40adf10 2023-12-01T02:51:39.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.773+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/3017834512 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb414d7a0 con 0x7fdcb40adf10 2023-12-01T02:51:39.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.774+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/3017834512 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb406fda0 con 0x7fdcb40adf10 2023-12-01T02:51:39.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.775+0000 7fdcb0ff9640 1 -- 172.21.15.115:0/3017834512 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fdc9c03a040 con 0x7fdcb40adf10 2023-12-01T02:51:39.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.776+0000 7fdcb0ff9640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc9406a580 0x7fdc9406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:39.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.776+0000 7fdcb0ff9640 1 -- 172.21.15.115:0/3017834512 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fdc9c0aa2f0 con 0x7fdcb40adf10 2023-12-01T02:51:39.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.776+0000 7fdcb2ffd640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc9406a580 0x7fdc9406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:39.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.777+0000 7fdcb2ffd640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc9406a580 0x7fdc9406ca40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdca4006fd0 tx=0x7fdca4008040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:39.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.782+0000 7fdcb0ff9640 1 -- 172.21.15.115:0/3017834512 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdc9c075a60 con 0x7fdcb40adf10 2023-12-01T02:51:39.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.976+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/3017834512 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcb404e890 con 0x7fdcb40adf10 2023-12-01T02:51:39.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.978+0000 7fdcb0ff9640 1 -- 172.21.15.115:0/3017834512 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+12517 (secure 0 0 0) 0x7fdc9c006e10 con 0x7fdcb40adf10 2023-12-01T02:51:39.980 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:39.980 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":159,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:51:38.992449+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:51:39.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.981+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/3017834512 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc9406a580 msgr2=0x7fdc9406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.981+0000 7fdcb9a0b640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc9406a580 0x7fdc9406ca40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdca4006fd0 tx=0x7fdca4008040 comp rx=0 tx=0).stop 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.981+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/3017834512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb40adf10 msgr2=0x7fdcb40c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.981+0000 7fdcb9a0b640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb40adf10 0x7fdcb40c7eb0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c000c00 tx=0x7fdc9c0046c0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.981+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/3017834512 shutdown_connections 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.981+0000 7fdcb9a0b640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdc9406a580 0x7fdc9406ca40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.982+0000 7fdcb9a0b640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdcb415e2f0 0x7fdcb40c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.982+0000 7fdcb9a0b640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdcb40adf10 0x7fdcb40c7eb0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.982+0000 7fdcb9a0b640 1 --2- 172.21.15.115:0/3017834512 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdcb406f2f0 0x7fdcb414a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.982+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/3017834512 >> 172.21.15.115:0/3017834512 conn(0x7fdcb40ce880 msgr2=0x7fdcb406b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.982+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/3017834512 shutdown_connections 2023-12-01T02:51:39.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:39.982+0000 7fdcb9a0b640 1 -- 172.21.15.115:0/3017834512 wait complete. 2023-12-01T02:51:40.002 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T02:51:40.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T02:51:40.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.217+0000 7f7c56fdf640 1 Processor -- start 2023-12-01T02:51:40.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.218+0000 7f7c56fdf640 1 -- start start 2023-12-01T02:51:40.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.219+0000 7f7c56fdf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7c500cfb80 0x7f7c500ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.219+0000 7f7c56fdf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c500cdcd0 con 0x7f7c50154010 2023-12-01T02:51:40.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.219+0000 7f7c56fdf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c50161830 con 0x7f7c50153c30 2023-12-01T02:51:40.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.219+0000 7f7c56fdf640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c500cd3a0 con 0x7f7c500cfb80 2023-12-01T02:51:40.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.219+0000 7f7c557dc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7c500cfb80 0x7f7c500ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:40.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.219+0000 7f7c557dc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7c500cfb80 0x7f7c500ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39332/0 (socket says 172.21.15.115:39332) 2023-12-01T02:51:40.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.219+0000 7f7c557dc640 1 -- 172.21.15.115:0/3658420269 learned_addr learned my addr 172.21.15.115:0/3658420269 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:40.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.219+0000 7f7c567de640 1 --1- 172.21.15.115:0/3658420269 >> v1:172.21.15.115:6789/0 conn(0x7f7c50154010 0x7f7c50160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38502/0 (socket says 172.21.15.115:38502) 2023-12-01T02:51:40.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.220+0000 7f7c557dc640 1 -- 172.21.15.115:0/3658420269 >> v1:172.21.15.142:6789/0 conn(0x7f7c50153c30 legacy=0x7f7c500cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:40.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.220+0000 7f7c557dc640 1 -- 172.21.15.115:0/3658420269 >> v1:172.21.15.115:6789/0 conn(0x7f7c50154010 legacy=0x7f7c50160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.220+0000 7f7c557dc640 1 -- 172.21.15.115:0/3658420269 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c500cd680 con 0x7f7c500cfb80 2023-12-01T02:51:40.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.221+0000 7f7c557dc640 1 --2- 172.21.15.115:0/3658420269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c500cfb80 0x7f7c500ce160 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c4402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9124a8bdb6f41fb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:40.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.221+0000 7f7c54fdb640 1 -- 172.21.15.115:0/3658420269 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c44030850 con 0x7f7c500cfb80 2023-12-01T02:51:40.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.221+0000 7f7c54fdb640 1 -- 172.21.15.115:0/3658420269 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c440309b0 con 0x7f7c500cfb80 2023-12-01T02:51:40.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.221+0000 7f7c54fdb640 1 -- 172.21.15.115:0/3658420269 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c44030d40 con 0x7f7c500cfb80 2023-12-01T02:51:40.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.222+0000 7f7c56fdf640 1 -- 172.21.15.115:0/3658420269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c500cfb80 msgr2=0x7f7c500ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.222+0000 7f7c56fdf640 1 --2- 172.21.15.115:0/3658420269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c500cfb80 0x7f7c500ce160 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c4402b990 comp rx=0 tx=0).stop 2023-12-01T02:51:40.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.222+0000 7f7c56fdf640 1 -- 172.21.15.115:0/3658420269 shutdown_connections 2023-12-01T02:51:40.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.222+0000 7f7c56fdf640 1 --2- 172.21.15.115:0/3658420269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c500cfb80 0x7f7c500ce160 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.222+0000 7f7c56fdf640 1 -- 172.21.15.115:0/3658420269 >> 172.21.15.115:0/3658420269 conn(0x7f7c5006a0b0 msgr2=0x7f7c5006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:40.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.222+0000 7f7c56fdf640 1 -- 172.21.15.115:0/3658420269 shutdown_connections 2023-12-01T02:51:40.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.222+0000 7f7c56fdf640 1 -- 172.21.15.115:0/3658420269 wait complete. 2023-12-01T02:51:40.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.224+0000 7f7c56fdf640 1 Processor -- start 2023-12-01T02:51:40.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.224+0000 7f7c56fdf640 1 -- start start 2023-12-01T02:51:40.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.225+0000 7f7c56fdf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c500cfb80 0x7f7c50155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.225+0000 7f7c56fdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c50153c30 0x7f7c501f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.225+0000 7f7c56fdf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c50154010 0x7f7c50154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.225+0000 7f7c56fdf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50156570 con 0x7f7c50153c30 2023-12-01T02:51:40.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.225+0000 7f7c56fdf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50155780 con 0x7f7c50154010 2023-12-01T02:51:40.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.225+0000 7f7c56fdf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c501f2ce0 con 0x7f7c500cfb80 2023-12-01T02:51:40.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.225+0000 7f7c557dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c50153c30 0x7f7c501f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:40.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.225+0000 7f7c557dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c50153c30 0x7f7c501f23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39334/0 (socket says 172.21.15.115:39334) 2023-12-01T02:51:40.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.225+0000 7f7c557dc640 1 -- 172.21.15.115:0/1703701596 learned_addr learned my addr 172.21.15.115:0/1703701596 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:40.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c557dc640 1 -- 172.21.15.115:0/1703701596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c500cfb80 msgr2=0x7f7c50155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c557dc640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c500cfb80 0x7f7c50155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c557dc640 1 -- 172.21.15.115:0/1703701596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c50154010 msgr2=0x7f7c50154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c567de640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c50154010 0x7f7c50154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:40.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c557dc640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c50154010 0x7f7c50154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c557dc640 1 -- 172.21.15.115:0/1703701596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440094d0 con 0x7f7c50153c30 2023-12-01T02:51:40.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c567de640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c50154010 0x7f7c50154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:40.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c557dc640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c50153c30 0x7f7c501f23e0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:40.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c4e7fc640 1 -- 172.21.15.115:0/1703701596 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c44030800 con 0x7f7c50153c30 2023-12-01T02:51:40.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c4e7fc640 1 -- 172.21.15.115:0/1703701596 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c44043400 con 0x7f7c50153c30 2023-12-01T02:51:40.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.226+0000 7f7c56fdf640 1 -- 172.21.15.115:0/1703701596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c501f2fc0 con 0x7f7c50153c30 2023-12-01T02:51:40.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.227+0000 7f7c4e7fc640 1 -- 172.21.15.115:0/1703701596 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c440437b0 con 0x7f7c50153c30 2023-12-01T02:51:40.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.227+0000 7f7c56fdf640 1 -- 172.21.15.115:0/1703701596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c501f32a0 con 0x7f7c50153c30 2023-12-01T02:51:40.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.228+0000 7f7c56fdf640 1 -- 172.21.15.115:0/1703701596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c500618d0 con 0x7f7c50153c30 2023-12-01T02:51:40.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.229+0000 7f7c4e7fc640 1 -- 172.21.15.115:0/1703701596 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f7c4402e070 con 0x7f7c50153c30 2023-12-01T02:51:40.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.230+0000 7f7c4e7fc640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7c3c06a550 0x7f7c3c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.230+0000 7f7c4e7fc640 1 -- 172.21.15.115:0/1703701596 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f7c4403e070 con 0x7f7c50153c30 2023-12-01T02:51:40.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.230+0000 7f7c55fdd640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7c3c06a550 0x7f7c3c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:40.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.231+0000 7f7c55fdd640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7c3c06a550 0x7f7c3c06ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c400043c0 tx=0x7f7c40007d30 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:40.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.234+0000 7f7c4e7fc640 1 -- 172.21.15.115:0/1703701596 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7c440770a0 con 0x7f7c50153c30 2023-12-01T02:51:40.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.428+0000 7f7c56fdf640 1 -- 172.21.15.115:0/1703701596 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c50074210 con 0x7f7c50153c30 2023-12-01T02:51:40.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.428+0000 7f7c4e7fc640 1 -- 172.21.15.115:0/1703701596 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c4407a6c0 con 0x7f7c50153c30 2023-12-01T02:51:40.431 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T02:51:40.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 -- 172.21.15.115:0/1703701596 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7c3c06a550 msgr2=0x7f7c3c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7c3c06a550 0x7f7c3c06ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c400043c0 tx=0x7f7c40007d30 comp rx=0 tx=0).stop 2023-12-01T02:51:40.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 -- 172.21.15.115:0/1703701596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c50153c30 msgr2=0x7f7c501f23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c50153c30 0x7f7c501f23e0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c44004730 comp rx=0 tx=0).stop 2023-12-01T02:51:40.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 -- 172.21.15.115:0/1703701596 shutdown_connections 2023-12-01T02:51:40.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7c3c06a550 0x7f7c3c06ca10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c50154010 0x7f7c50154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c50153c30 0x7f7c501f23e0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 --2- 172.21.15.115:0/1703701596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c500cfb80 0x7f7c50155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 -- 172.21.15.115:0/1703701596 >> 172.21.15.115:0/1703701596 conn(0x7f7c5006a0b0 msgr2=0x7f7c500ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:40.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 -- 172.21.15.115:0/1703701596 shutdown_connections 2023-12-01T02:51:40.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.431+0000 7f7c56fdf640 1 -- 172.21.15.115:0/1703701596 wait complete. 2023-12-01T02:51:40.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T02:51:40.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.664+0000 7efe77682640 1 Processor -- start 2023-12-01T02:51:40.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.664+0000 7efe77682640 1 -- start start 2023-12-01T02:51:40.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.666+0000 7efe77682640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efe700cad70 0x7efe7006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.666+0000 7efe77682640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efe7006f2f0 0x7efe7006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.666+0000 7efe77682640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe700add10 con 0x7efe7015e2f0 2023-12-01T02:51:40.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.666+0000 7efe77682640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe700c5760 con 0x7efe7006f2f0 2023-12-01T02:51:40.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.666+0000 7efe77682640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7015e6d0 con 0x7efe700cad70 2023-12-01T02:51:40.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.666+0000 7efe7541e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efe7015e2f0 0x7efe700ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38516/0 (socket says 172.21.15.115:38516) 2023-12-01T02:51:40.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.666+0000 7efe7541e640 1 -- 172.21.15.115:0/550924435 learned_addr learned my addr 172.21.15.115:0/550924435 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:40.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.666+0000 7efe75c1f640 1 --2- 172.21.15.115:0/550924435 >> v2:172.21.15.115:3300/0 conn(0x7efe7006f2f0 0x7efe7006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:40.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.666+0000 7efe74c1d640 1 --2- 172.21.15.115:0/550924435 >> v2:172.21.15.142:3300/0 conn(0x7efe700cad70 0x7efe7006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:40.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.667+0000 7efe67fff640 1 -- 172.21.15.115:0/550924435 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111479966 0 0) 0x7efe700add10 con 0x7efe7015e2f0 2023-12-01T02:51:40.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.667+0000 7efe67fff640 1 -- 172.21.15.115:0/550924435 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe5c003520 con 0x7efe7015e2f0 2023-12-01T02:51:40.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.667+0000 7efe75c1f640 1 -- 172.21.15.115:0/550924435 >> v2:172.21.15.142:3300/0 conn(0x7efe700cad70 msgr2=0x7efe7006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:40.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.667+0000 7efe75c1f640 1 --2- 172.21.15.115:0/550924435 >> v2:172.21.15.142:3300/0 conn(0x7efe700cad70 0x7efe7006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.667+0000 7efe75c1f640 1 -- 172.21.15.115:0/550924435 >> v1:172.21.15.115:6789/0 conn(0x7efe7015e2f0 legacy=0x7efe700ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.667+0000 7efe75c1f640 1 -- 172.21.15.115:0/550924435 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700cf7b0 con 0x7efe7006f2f0 2023-12-01T02:51:40.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.668+0000 7efe75c1f640 1 --2- 172.21.15.115:0/550924435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe7006f2f0 0x7efe7006bb00 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7efe68009d90 tx=0x7efe6802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d33523d73fd708 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:40.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.668+0000 7efe67fff640 1 -- 172.21.15.115:0/550924435 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6802cb40 con 0x7efe7006f2f0 2023-12-01T02:51:40.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.668+0000 7efe67fff640 1 -- 172.21.15.115:0/550924435 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6802cca0 con 0x7efe7006f2f0 2023-12-01T02:51:40.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.668+0000 7efe67fff640 1 -- 172.21.15.115:0/550924435 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68034660 con 0x7efe7006f2f0 2023-12-01T02:51:40.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.669+0000 7efe77682640 1 -- 172.21.15.115:0/550924435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe7006f2f0 msgr2=0x7efe7006bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.669+0000 7efe77682640 1 --2- 172.21.15.115:0/550924435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe7006f2f0 0x7efe7006bb00 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7efe68009d90 tx=0x7efe6802bd90 comp rx=0 tx=0).stop 2023-12-01T02:51:40.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.669+0000 7efe77682640 1 -- 172.21.15.115:0/550924435 shutdown_connections 2023-12-01T02:51:40.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.669+0000 7efe77682640 1 --2- 172.21.15.115:0/550924435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe7006f2f0 0x7efe7006bb00 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.669+0000 7efe77682640 1 --2- 172.21.15.115:0/550924435 >> v2:172.21.15.142:3300/0 conn(0x7efe700cad70 0x7efe7006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.669+0000 7efe77682640 1 -- 172.21.15.115:0/550924435 >> 172.21.15.115:0/550924435 conn(0x7efe700ce880 msgr2=0x7efe700cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:40.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.669+0000 7efe77682640 1 -- 172.21.15.115:0/550924435 shutdown_connections 2023-12-01T02:51:40.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.669+0000 7efe77682640 1 -- 172.21.15.115:0/550924435 wait complete. 2023-12-01T02:51:40.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.671+0000 7efe77682640 1 Processor -- start 2023-12-01T02:51:40.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.671+0000 7efe77682640 1 -- start start 2023-12-01T02:51:40.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe77682640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7006f2f0 0x7efe700cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe77682640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cad70 0x7efe70158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe77682640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7015e2f0 0x7efe700b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe77682640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe700a85a0 con 0x7efe700cad70 2023-12-01T02:51:40.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe77682640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe700ce480 con 0x7efe7015e2f0 2023-12-01T02:51:40.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe77682640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe701f2420 con 0x7efe7006f2f0 2023-12-01T02:51:40.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe74c1d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cad70 0x7efe70158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe74c1d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cad70 0x7efe70158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39352/0 (socket says 172.21.15.115:39352) 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe74c1d640 1 -- 172.21.15.115:0/1459558109 learned_addr learned my addr 172.21.15.115:0/1459558109 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe74c1d640 1 -- 172.21.15.115:0/1459558109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7006f2f0 msgr2=0x7efe700cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe7541e640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7006f2f0 0x7efe700cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe75c1f640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7015e2f0 0x7efe700b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe74c1d640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7006f2f0 0x7efe700cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe74c1d640 1 -- 172.21.15.115:0/1459558109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7015e2f0 msgr2=0x7efe700b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.672+0000 7efe74c1d640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7015e2f0 0x7efe700b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.673+0000 7efe74c1d640 1 -- 172.21.15.115:0/1459558109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe68009a70 con 0x7efe700cad70 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.673+0000 7efe75c1f640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7015e2f0 0x7efe700b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.673+0000 7efe74c1d640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cad70 0x7efe70158d30 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7efe6000d5f0 tx=0x7efe6000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.673+0000 7efe65ffb640 1 -- 172.21.15.115:0/1459558109 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe600125f0 con 0x7efe700cad70 2023-12-01T02:51:40.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.673+0000 7efe65ffb640 1 -- 172.21.15.115:0/1459558109 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe60012750 con 0x7efe700cad70 2023-12-01T02:51:40.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.673+0000 7efe65ffb640 1 -- 172.21.15.115:0/1459558109 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60012b00 con 0x7efe700cad70 2023-12-01T02:51:40.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.673+0000 7efe77682640 1 -- 172.21.15.115:0/1459558109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe700c23b0 con 0x7efe700cad70 2023-12-01T02:51:40.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.673+0000 7efe77682640 1 -- 172.21.15.115:0/1459558109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe700c2650 con 0x7efe700cad70 2023-12-01T02:51:40.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.674+0000 7efe77682640 1 -- 172.21.15.115:0/1459558109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7006fda0 con 0x7efe700cad70 2023-12-01T02:51:40.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.675+0000 7efe65ffb640 1 -- 172.21.15.115:0/1459558109 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7efe60012c60 con 0x7efe700cad70 2023-12-01T02:51:40.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.676+0000 7efe65ffb640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a580 0x7efe5406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:40.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.676+0000 7efe65ffb640 1 -- 172.21.15.115:0/1459558109 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4901+0+0 (secure 0 0 0) 0x7efe60088ba0 con 0x7efe700cad70 2023-12-01T02:51:40.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.677+0000 7efe7541e640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a580 0x7efe5406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:40.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.677+0000 7efe7541e640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a580 0x7efe5406ca40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efe6c000f60 tx=0x7efe6c009040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:40.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.681+0000 7efe65ffb640 1 -- 172.21.15.115:0/1459558109 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efe600543a0 con 0x7efe700cad70 2023-12-01T02:51:40.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.875+0000 7efe77682640 1 -- 172.21.15.115:0/1459558109 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efe7004e890 con 0x7efe700cad70 2023-12-01T02:51:40.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.875+0000 7efe65ffb640 1 -- 172.21.15.115:0/1459558109 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v159) v1 ==== 84+0+65 (secure 0 0 0) 0x7efe600579c0 con 0x7efe700cad70 2023-12-01T02:51:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:40.877 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T02:51:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T02:51:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T02:51:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:40.878 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T02:51:40.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.878+0000 7efe77682640 1 -- 172.21.15.115:0/1459558109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a580 msgr2=0x7efe5406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.878+0000 7efe77682640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a580 0x7efe5406ca40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efe6c000f60 tx=0x7efe6c009040 comp rx=0 tx=0).stop 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.878+0000 7efe77682640 1 -- 172.21.15.115:0/1459558109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cad70 msgr2=0x7efe70158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.878+0000 7efe77682640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cad70 0x7efe70158d30 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7efe6000d5f0 tx=0x7efe6000dab0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.878+0000 7efe77682640 1 -- 172.21.15.115:0/1459558109 shutdown_connections 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.879+0000 7efe77682640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a580 0x7efe5406ca40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.879+0000 7efe77682640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7015e2f0 0x7efe700b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.879+0000 7efe77682640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cad70 0x7efe70158d30 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.879+0000 7efe77682640 1 --2- 172.21.15.115:0/1459558109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7006f2f0 0x7efe700cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.879+0000 7efe77682640 1 -- 172.21.15.115:0/1459558109 >> 172.21.15.115:0/1459558109 conn(0x7efe700ce880 msgr2=0x7efe70159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.879+0000 7efe77682640 1 -- 172.21.15.115:0/1459558109 shutdown_connections 2023-12-01T02:51:40.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:40.879+0000 7efe77682640 1 -- 172.21.15.115:0/1459558109 wait complete. 2023-12-01T02:51:40.900 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T02:51:40.901 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T02:51:41.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.116+0000 7fc7bd062640 1 Processor -- start 2023-12-01T02:51:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.116+0000 7fc7bd062640 1 -- start start 2023-12-01T02:51:41.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.117+0000 7fc7bd062640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b815dcf0 0x7fc7b80d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:41.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.117+0000 7fc7bd062640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b80b2300 con 0x7fc7b815d910 2023-12-01T02:51:41.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.117+0000 7fc7bd062640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b80ce260 con 0x7fc7b80d2f10 2023-12-01T02:51:41.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.117+0000 7fc7bd062640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b81612b0 con 0x7fc7b815dcf0 2023-12-01T02:51:41.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.117+0000 7fc7b7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc7b815d910 0x7fc7b80cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38522/0 (socket says 172.21.15.115:38522) 2023-12-01T02:51:41.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.118+0000 7fc7b7fff640 1 -- 172.21.15.115:0/2717564351 learned_addr learned my addr 172.21.15.115:0/2717564351 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.118+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140407706 0 0) 0x7fc7a8001110 con 0x7fc7b815d910 2023-12-01T02:51:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.118+0000 7fc7affff640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b815dcf0 0x7fc7b80d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.118+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a4003520 con 0x7fc7b815d910 2023-12-01T02:51:41.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.118+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4169155258 0 0) 0x7fc7a4003520 con 0x7fc7b815d910 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.118+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b815dcf0 msgr2=0x7fc7b80d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.118+0000 7fc7b77fe640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b815dcf0 0x7fc7b80d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.118+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 >> v1:172.21.15.142:6789/0 conn(0x7fc7b80d2f10 legacy=0x7fc7b80d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.118+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b815e130 con 0x7fc7b815d910 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.118+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc7a8001800 con 0x7fc7b815d910 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 >> v1:172.21.15.115:6789/0 conn(0x7fc7b815d910 legacy=0x7fc7b80cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7b77fe640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7a40039a0 0x7fc7a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7b77fe640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 0x7fc7a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7b77fe640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a40052d0 0x7fc7a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b81612b0 con 0x7fc7a40047c0 2023-12-01T02:51:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a400c100 con 0x7fc7a40039a0 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a400c2c0 con 0x7fc7a40052d0 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7affff640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b815dcf0 0x7fc7b80d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7affff640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 0x7fc7a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7bc861640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a40052d0 0x7fc7a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7b7fff640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7a40039a0 0x7fc7a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7affff640 1 -- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a40052d0 msgr2=0x7fc7a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7affff640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7a40052d0 0x7fc7a400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7affff640 1 -- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7a40039a0 msgr2=0x7fc7a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7affff640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7a40039a0 0x7fc7a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7affff640 1 -- 172.21.15.115:0/2717564351 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4003520 con 0x7fc7a40047c0 2023-12-01T02:51:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.119+0000 7fc7b7fff640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7a40039a0 0x7fc7a4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.121+0000 7fc7affff640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 0x7fc7a4004c40 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0006e80 tx=0x7fc7a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaa411c1c8bed0a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.121+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0032600 con 0x7fc7a40047c0 2023-12-01T02:51:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.121+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a002e050 con 0x7fc7a40047c0 2023-12-01T02:51:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.121+0000 7fc7b77fe640 1 -- 172.21.15.115:0/2717564351 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00329c0 con 0x7fc7a40047c0 2023-12-01T02:51:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.121+0000 7fc7bd062640 1 -- 172.21.15.115:0/2717564351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 msgr2=0x7fc7a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.121+0000 7fc7bd062640 1 --2- 172.21.15.115:0/2717564351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 0x7fc7a4004c40 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0006e80 tx=0x7fc7a40057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.121+0000 7fc7bc861640 1 -- 172.21.15.115:0/2717564351 reap_dead start 2023-12-01T02:51:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.122+0000 7fc7bd062640 1 -- 172.21.15.115:0/2717564351 shutdown_connections 2023-12-01T02:51:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.122+0000 7fc7bd062640 1 -- 172.21.15.115:0/2717564351 >> 172.21.15.115:0/2717564351 conn(0x7fc7b806a0b0 msgr2=0x7fc7b806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.122+0000 7fc7bd062640 1 -- 172.21.15.115:0/2717564351 shutdown_connections 2023-12-01T02:51:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.122+0000 7fc7bd062640 1 -- 172.21.15.115:0/2717564351 wait complete. 2023-12-01T02:51:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.123+0000 7fc7bd062640 1 Processor -- start 2023-12-01T02:51:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.124+0000 7fc7bd062640 1 -- start start 2023-12-01T02:51:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7bd062640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 0x7fc7b80c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7bd062640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b81f2550 0x7fc7b815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7bd062640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b815aa80 0x7fc7b815c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7bd062640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b80c9b10 con 0x7fc7a40047c0 2023-12-01T02:51:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7bd062640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b80c9160 con 0x7fc7b815aa80 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7bd062640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b80c9d60 con 0x7fc7b81f2550 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7b7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 0x7fc7b80c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7b7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 0x7fc7b80c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39376/0 (socket says 172.21.15.115:39376) 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7b7fff640 1 -- 172.21.15.115:0/3246739133 learned_addr learned my addr 172.21.15.115:0/3246739133 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7bc861640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b815aa80 0x7fc7b815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7affff640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b81f2550 0x7fc7b815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7b7fff640 1 -- 172.21.15.115:0/3246739133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b81f2550 msgr2=0x7fc7b815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7b7fff640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b81f2550 0x7fc7b815c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7b7fff640 1 -- 172.21.15.115:0/3246739133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b815aa80 msgr2=0x7fc7b815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7b7fff640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b815aa80 0x7fc7b815c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7b7fff640 1 -- 172.21.15.115:0/3246739133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b80ca040 con 0x7fc7a40047c0 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.125+0000 7fc7bc861640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b815aa80 0x7fc7b815c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.126+0000 7fc7b7fff640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 0x7fc7b80c88e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800d860 tx=0x7fc7a800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.126+0000 7fc7b57fa640 1 -- 172.21.15.115:0/3246739133 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80074e0 con 0x7fc7a40047c0 2023-12-01T02:51:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.126+0000 7fc7b57fa640 1 -- 172.21.15.115:0/3246739133 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a8007640 con 0x7fc7a40047c0 2023-12-01T02:51:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.126+0000 7fc7b57fa640 1 -- 172.21.15.115:0/3246739133 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80049e0 con 0x7fc7a40047c0 2023-12-01T02:51:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.126+0000 7fc7bd062640 1 -- 172.21.15.115:0/3246739133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b80ca320 con 0x7fc7a40047c0 2023-12-01T02:51:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.126+0000 7fc7bd062640 1 -- 172.21.15.115:0/3246739133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b80b0da0 con 0x7fc7a40047c0 2023-12-01T02:51:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.127+0000 7fc7bd062640 1 -- 172.21.15.115:0/3246739133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b80618d0 con 0x7fc7a40047c0 2023-12-01T02:51:41.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.128+0000 7fc7b57fa640 1 -- 172.21.15.115:0/3246739133 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc7a8004b40 con 0x7fc7a40047c0 2023-12-01T02:51:41.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.129+0000 7fc7b57fa640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79806a550 0x7fc79806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:41.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.129+0000 7fc7b57fa640 1 -- 172.21.15.115:0/3246739133 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fc7a80880f0 con 0x7fc7a40047c0 2023-12-01T02:51:41.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.129+0000 7fc7affff640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79806a550 0x7fc79806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:41.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.130+0000 7fc7affff640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79806a550 0x7fc79806ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0035000 tx=0x7fc7a0030000 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:41.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.134+0000 7fc7b57fa640 1 -- 172.21.15.115:0/3246739133 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc7a8053740 con 0x7fc7a40047c0 2023-12-01T02:51:41.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:41.327+0000 7fc7bd062640 1 -- 172.21.15.115:0/3246739133 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc7b80b22d0 con 0x7fc7a40047c0 2023-12-01T02:51:42.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.326+0000 7fc7b57fa640 1 -- 172.21.15.115:0/3246739133 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v160) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc7a8056d60 con 0x7fc7a40047c0 2023-12-01T02:51:42.329 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T02:51:42.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.331+0000 7fc7bd062640 1 -- 172.21.15.115:0/3246739133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79806a550 msgr2=0x7fc79806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:42.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.331+0000 7fc7bd062640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79806a550 0x7fc79806ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0035000 tx=0x7fc7a0030000 comp rx=0 tx=0).stop 2023-12-01T02:51:42.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.331+0000 7fc7bd062640 1 -- 172.21.15.115:0/3246739133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 msgr2=0x7fc7b80c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:42.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.331+0000 7fc7bd062640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 0x7fc7b80c88e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800d860 tx=0x7fc7a800dd20 comp rx=0 tx=0).stop 2023-12-01T02:51:42.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.332+0000 7fc7bd062640 1 -- 172.21.15.115:0/3246739133 shutdown_connections 2023-12-01T02:51:42.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.332+0000 7fc7bd062640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc79806a550 0x7fc79806ca10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:42.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.332+0000 7fc7bd062640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b81f2550 0x7fc7b815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:42.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.332+0000 7fc7bd062640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b815aa80 0x7fc7b815c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:42.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.332+0000 7fc7bd062640 1 --2- 172.21.15.115:0/3246739133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7a40047c0 0x7fc7b80c88e0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:42.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.332+0000 7fc7bd062640 1 -- 172.21.15.115:0/3246739133 >> 172.21.15.115:0/3246739133 conn(0x7fc7b806a0b0 msgr2=0x7fc7b80b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:42.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.332+0000 7fc7bd062640 1 -- 172.21.15.115:0/3246739133 shutdown_connections 2023-12-01T02:51:42.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.332+0000 7fc7bd062640 1 -- 172.21.15.115:0/3246739133 wait complete. 2023-12-01T02:51:42.359 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T02:51:42.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.566+0000 7f98e2adc640 1 Processor -- start 2023-12-01T02:51:42.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.567+0000 7f98e2adc640 1 -- start start 2023-12-01T02:51:42.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.568+0000 7f98e2adc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdec0 0x7f98dc0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:42.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.568+0000 7f98e2adc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f98dc0b0620 0x7f98dc0acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:42.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.568+0000 7f98e2adc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc0ca080 con 0x7f98dc0cdae0 2023-12-01T02:51:42.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.568+0000 7f98e2adc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc0b0ab0 con 0x7f98dc0b0620 2023-12-01T02:51:42.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.568+0000 7f98e2adc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0ca360 con 0x7f98dc0cdec0 2023-12-01T02:51:42.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.569+0000 7f98e22db640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f98dc0b0620 0x7f98dc0acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:42.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.569+0000 7f98e12d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdec0 0x7f98dc0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:42.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.569+0000 7f98e12d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdec0 0x7f98dc0b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:46190/0 (socket says 172.21.15.115:46190) 2023-12-01T02:51:42.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.569+0000 7f98e12d9640 1 -- 172.21.15.115:0/1544202205 learned_addr learned my addr 172.21.15.115:0/1544202205 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:42.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.570+0000 7f98e12d9640 1 -- 172.21.15.115:0/1544202205 >> v2:172.21.15.142:3300/0 conn(0x7f98dc0b0620 msgr2=0x7f98dc0acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:42.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.570+0000 7f98e12d9640 1 --2- 172.21.15.115:0/1544202205 >> v2:172.21.15.142:3300/0 conn(0x7f98dc0b0620 0x7f98dc0acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:42.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.570+0000 7f98e12d9640 1 -- 172.21.15.115:0/1544202205 >> v1:172.21.15.142:6789/0 conn(0x7f98dc0cdae0 legacy=0x7f98dc15dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:42.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.570+0000 7f98e12d9640 1 -- 172.21.15.115:0/1544202205 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc0ca640 con 0x7f98dc0cdec0 2023-12-01T02:51:42.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.571+0000 7f98e12d9640 1 --2- 172.21.15.115:0/1544202205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdec0 0x7f98dc0b00e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009870 tx=0x7f98cc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78aaae2e3879e8f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:42.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.572+0000 7f98e0ad8640 1 -- 172.21.15.115:0/1544202205 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc031640 con 0x7f98dc0cdec0 2023-12-01T02:51:42.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.572+0000 7f98e0ad8640 1 -- 172.21.15.115:0/1544202205 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc0317a0 con 0x7f98dc0cdec0 2023-12-01T02:51:42.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.572+0000 7f98e0ad8640 1 -- 172.21.15.115:0/1544202205 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc031b30 con 0x7f98dc0cdec0 2023-12-01T02:51:42.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.572+0000 7f98e2adc640 1 -- 172.21.15.115:0/1544202205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdec0 msgr2=0x7f98dc0b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:42.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.572+0000 7f98e2adc640 1 --2- 172.21.15.115:0/1544202205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdec0 0x7f98dc0b00e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009870 tx=0x7f98cc02b8f0 comp rx=0 tx=0).stop 2023-12-01T02:51:42.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.572+0000 7f98e2adc640 1 -- 172.21.15.115:0/1544202205 shutdown_connections 2023-12-01T02:51:42.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.572+0000 7f98e2adc640 1 --2- 172.21.15.115:0/1544202205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdec0 0x7f98dc0b00e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:42.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.572+0000 7f98e2adc640 1 --2- 172.21.15.115:0/1544202205 >> v2:172.21.15.142:3300/0 conn(0x7f98dc0b0620 0x7f98dc0acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:42.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.572+0000 7f98e2adc640 1 -- 172.21.15.115:0/1544202205 >> 172.21.15.115:0/1544202205 conn(0x7f98dc066400 msgr2=0x7f98dc0cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:42.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.573+0000 7f98e2adc640 1 -- 172.21.15.115:0/1544202205 shutdown_connections 2023-12-01T02:51:42.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.573+0000 7f98e2adc640 1 -- 172.21.15.115:0/1544202205 wait complete. 2023-12-01T02:51:42.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.575+0000 7f98e2adc640 1 Processor -- start 2023-12-01T02:51:42.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.575+0000 7f98e2adc640 1 -- start start 2023-12-01T02:51:42.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.576+0000 7f98e2adc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc0b0620 0x7f98dc071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:42.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.576+0000 7f98e2adc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdae0 0x7f98dc1f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:42.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.576+0000 7f98e2adc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98dc0b3970 0x7f98dc0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:42.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.576+0000 7f98e2adc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc06f300 con 0x7f98dc0b0620 2023-12-01T02:51:42.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.576+0000 7f98e2adc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc071e60 con 0x7f98dc0b3970 2023-12-01T02:51:42.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.576+0000 7f98e2adc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc071490 con 0x7f98dc0cdae0 2023-12-01T02:51:42.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.576+0000 7f98e1ada640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc0b0620 0x7f98dc071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:42.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.576+0000 7f98e1ada640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc0b0620 0x7f98dc071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39390/0 (socket says 172.21.15.115:39390) 2023-12-01T02:51:42.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.576+0000 7f98e1ada640 1 -- 172.21.15.115:0/760356314 learned_addr learned my addr 172.21.15.115:0/760356314 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:42.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98e12d9640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdae0 0x7f98dc1f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:42.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98e22db640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98dc0b3970 0x7f98dc0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:42.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98e1ada640 1 -- 172.21.15.115:0/760356314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdae0 msgr2=0x7f98dc1f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:42.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98e1ada640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdae0 0x7f98dc1f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:42.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98e1ada640 1 -- 172.21.15.115:0/760356314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98dc0b3970 msgr2=0x7f98dc0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:42.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98e1ada640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98dc0b3970 0x7f98dc0b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:42.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98e1ada640 1 -- 172.21.15.115:0/760356314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0094d0 con 0x7f98dc0b0620 2023-12-01T02:51:42.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98e22db640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98dc0b3970 0x7f98dc0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:42.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98e1ada640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc0b0620 0x7f98dc071920 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c430 tx=0x7f98d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:42.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98d27fc640 1 -- 172.21.15.115:0/760356314 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8011790 con 0x7f98dc0b0620 2023-12-01T02:51:42.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98d27fc640 1 -- 172.21.15.115:0/760356314 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80118f0 con 0x7f98dc0b0620 2023-12-01T02:51:42.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.577+0000 7f98e2adc640 1 -- 172.21.15.115:0/760356314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc1f35a0 con 0x7f98dc0b0620 2023-12-01T02:51:42.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.578+0000 7f98d27fc640 1 -- 172.21.15.115:0/760356314 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8011ca0 con 0x7f98dc0b0620 2023-12-01T02:51:42.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.578+0000 7f98e2adc640 1 -- 172.21.15.115:0/760356314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc070b60 con 0x7f98dc0b0620 2023-12-01T02:51:42.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.579+0000 7f98e2adc640 1 -- 172.21.15.115:0/760356314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc0618d0 con 0x7f98dc0b0620 2023-12-01T02:51:42.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.580+0000 7f98d27fc640 1 -- 172.21.15.115:0/760356314 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f98d801c460 con 0x7f98dc0b0620 2023-12-01T02:51:42.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.581+0000 7f98d27fc640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f98c006a550 0x7f98c006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:42.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.581+0000 7f98d27fc640 1 -- 172.21.15.115:0/760356314 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5271+0+0 (secure 0 0 0) 0x7f98d8089300 con 0x7f98dc0b0620 2023-12-01T02:51:42.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.581+0000 7f98e12d9640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f98c006a550 0x7f98c006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:42.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.582+0000 7f98e12d9640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f98c006a550 0x7f98c006ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98cc036000 tx=0x7f98cc03c040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:42.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.585+0000 7f98d27fc640 1 -- 172.21.15.115:0/760356314 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f98d8054970 con 0x7f98dc0b0620 2023-12-01T02:51:42.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:42.776+0000 7f98e2adc640 1 -- 172.21.15.115:0/760356314 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f98dc0ac9e0 con 0x7f98dc0b0620 2023-12-01T02:51:43.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.348+0000 7f98d27fc640 1 -- 172.21.15.115:0/760356314 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v161) v1 ==== 170+0+0 (secure 0 0 0) 0x7f98d8057f90 con 0x7f98dc0b0620 2023-12-01T02:51:43.351 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T02:51:43.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.357+0000 7f98e2adc640 1 -- 172.21.15.115:0/760356314 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f98c006a550 msgr2=0x7f98c006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:43.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.357+0000 7f98e2adc640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f98c006a550 0x7f98c006ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98cc036000 tx=0x7f98cc03c040 comp rx=0 tx=0).stop 2023-12-01T02:51:43.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.357+0000 7f98e2adc640 1 -- 172.21.15.115:0/760356314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc0b0620 msgr2=0x7f98dc071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:43.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.357+0000 7f98e2adc640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc0b0620 0x7f98dc071920 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c430 tx=0x7f98d800c8f0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.357+0000 7f98e2adc640 1 -- 172.21.15.115:0/760356314 shutdown_connections 2023-12-01T02:51:43.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.357+0000 7f98e2adc640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f98c006a550 0x7f98c006ca10 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98cc036000 tx=0x7f98cc03c040 comp rx=0 tx=0).stop 2023-12-01T02:51:43.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.357+0000 7f98e2adc640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f98dc0cdae0 0x7f98dc1f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.357+0000 7f98e2adc640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98dc0b3970 0x7f98dc0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.357+0000 7f98e2adc640 1 --2- 172.21.15.115:0/760356314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98dc0b0620 0x7f98dc071920 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.357+0000 7f98e2adc640 1 -- 172.21.15.115:0/760356314 >> 172.21.15.115:0/760356314 conn(0x7f98dc066400 msgr2=0x7f98dc15ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:43.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.358+0000 7f98e2adc640 1 -- 172.21.15.115:0/760356314 shutdown_connections 2023-12-01T02:51:43.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.359+0000 7f98e2adc640 1 -- 172.21.15.115:0/760356314 wait complete. 2023-12-01T02:51:43.393 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T02:51:43.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.581+0000 7f074faab640 1 Processor -- start 2023-12-01T02:51:43.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.581+0000 7f074faab640 1 -- start start 2023-12-01T02:51:43.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.582+0000 7f074faab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f074815ac80 0x7f07480cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:43.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074faab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07480cc9a0 con 0x7f07480ce1b0 2023-12-01T02:51:43.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074faab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07480cbd90 con 0x7f07480b0e60 2023-12-01T02:51:43.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074faab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074815b060 con 0x7f074815ac80 2023-12-01T02:51:43.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074d847640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f074815ac80 0x7f07480cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:43.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074d847640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f074815ac80 0x7f07480cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39398/0 (socket says 172.21.15.115:39398) 2023-12-01T02:51:43.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074d847640 1 -- 172.21.15.115:0/3179815017 learned_addr learned my addr 172.21.15.115:0/3179815017 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:43.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074e048640 1 --1- 172.21.15.115:0/3179815017 >> v1:172.21.15.115:6789/0 conn(0x7f07480ce1b0 0x7f074806b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38538/0 (socket says 172.21.15.115:38538) 2023-12-01T02:51:43.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074d847640 1 -- 172.21.15.115:0/3179815017 >> v1:172.21.15.142:6789/0 conn(0x7f07480b0e60 legacy=0x7f07480cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:43.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074d847640 1 -- 172.21.15.115:0/3179815017 >> v1:172.21.15.115:6789/0 conn(0x7f07480ce1b0 legacy=0x7f074806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:43.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074d847640 1 -- 172.21.15.115:0/3179815017 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480cc070 con 0x7f074815ac80 2023-12-01T02:51:43.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.583+0000 7f074c845640 1 -- 172.21.15.115:0/3179815017 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441789682 0 0) 0x7f07480cc9a0 con 0x7f07480ce1b0 2023-12-01T02:51:43.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.584+0000 7f074d847640 1 --2- 172.21.15.115:0/3179815017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f074815ac80 0x7f07480cce30 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f0744009870 tx=0x7f074402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c683fcf8f06845 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:43.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.585+0000 7f074c845640 1 -- 172.21.15.115:0/3179815017 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744030860 con 0x7f074815ac80 2023-12-01T02:51:43.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.585+0000 7f074c845640 1 -- 172.21.15.115:0/3179815017 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07440309c0 con 0x7f074815ac80 2023-12-01T02:51:43.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.585+0000 7f074c845640 1 -- 172.21.15.115:0/3179815017 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744030d50 con 0x7f074815ac80 2023-12-01T02:51:43.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.585+0000 7f074faab640 1 -- 172.21.15.115:0/3179815017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f074815ac80 msgr2=0x7f07480cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:43.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.585+0000 7f074faab640 1 --2- 172.21.15.115:0/3179815017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f074815ac80 0x7f07480cce30 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f0744009870 tx=0x7f074402b9a0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.586+0000 7f074faab640 1 -- 172.21.15.115:0/3179815017 shutdown_connections 2023-12-01T02:51:43.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.586+0000 7f074faab640 1 --2- 172.21.15.115:0/3179815017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f074815ac80 0x7f07480cce30 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.586+0000 7f074faab640 1 -- 172.21.15.115:0/3179815017 >> 172.21.15.115:0/3179815017 conn(0x7f0748065960 msgr2=0x7f0748065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:43.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.586+0000 7f074faab640 1 -- 172.21.15.115:0/3179815017 shutdown_connections 2023-12-01T02:51:43.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.586+0000 7f074faab640 1 -- 172.21.15.115:0/3179815017 wait complete. 2023-12-01T02:51:43.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.587+0000 7f074faab640 1 Processor -- start 2023-12-01T02:51:43.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.587+0000 7f074faab640 1 -- start start 2023-12-01T02:51:43.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.588+0000 7f074faab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07480b0e60 0x7f074815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:43.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.588+0000 7f074faab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07480ce1b0 0x7f074815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:43.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.588+0000 7f074faab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f074815ac80 0x7f0748149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:43.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.588+0000 7f074faab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074815c540 con 0x7f07480ce1b0 2023-12-01T02:51:43.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.588+0000 7f074faab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074814c8c0 con 0x7f07480b0e60 2023-12-01T02:51:43.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.588+0000 7f074faab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074815bc50 con 0x7f074815ac80 2023-12-01T02:51:43.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d046640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07480ce1b0 0x7f074815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:43.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d046640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07480ce1b0 0x7f074815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39408/0 (socket says 172.21.15.115:39408) 2023-12-01T02:51:43.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d046640 1 -- 172.21.15.115:0/4095805646 learned_addr learned my addr 172.21.15.115:0/4095805646 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d847640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07480b0e60 0x7f074815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074e048640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f074815ac80 0x7f0748149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d046640 1 -- 172.21.15.115:0/4095805646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f074815ac80 msgr2=0x7f0748149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d046640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f074815ac80 0x7f0748149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d046640 1 -- 172.21.15.115:0/4095805646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07480b0e60 msgr2=0x7f074815c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d046640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07480b0e60 0x7f074815c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d046640 1 -- 172.21.15.115:0/4095805646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07440094d0 con 0x7f07480ce1b0 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d847640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07480b0e60 0x7f074815c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.589+0000 7f074d046640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07480ce1b0 0x7f074815d8e0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f073800c430 tx=0x7f073800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.590+0000 7f073e7fc640 1 -- 172.21.15.115:0/4095805646 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738011790 con 0x7f07480ce1b0 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.590+0000 7f073e7fc640 1 -- 172.21.15.115:0/4095805646 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07380118f0 con 0x7f07480ce1b0 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.590+0000 7f073e7fc640 1 -- 172.21.15.115:0/4095805646 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738011ce0 con 0x7f07480ce1b0 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.590+0000 7f074faab640 1 -- 172.21.15.115:0/4095805646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074815bef0 con 0x7f07480ce1b0 2023-12-01T02:51:43.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.590+0000 7f074faab640 1 -- 172.21.15.115:0/4095805646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074815df70 con 0x7f07480ce1b0 2023-12-01T02:51:43.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.591+0000 7f074faab640 1 -- 172.21.15.115:0/4095805646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074804e890 con 0x7f07480ce1b0 2023-12-01T02:51:43.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.592+0000 7f073e7fc640 1 -- 172.21.15.115:0/4095805646 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f073801c460 con 0x7f07480ce1b0 2023-12-01T02:51:43.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.593+0000 7f073e7fc640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f072c06a550 0x7f072c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:43.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.593+0000 7f073e7fc640 1 -- 172.21.15.115:0/4095805646 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5653+0+0 (secure 0 0 0) 0x7f07380889e0 con 0x7f07480ce1b0 2023-12-01T02:51:43.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.593+0000 7f074d847640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f072c06a550 0x7f072c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:43.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.594+0000 7f074d847640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f072c06a550 0x7f072c06ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0744036000 tx=0x7f074403b040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:43.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.598+0000 7f073e7fc640 1 -- 172.21.15.115:0/4095805646 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0738053ef0 con 0x7f07480ce1b0 2023-12-01T02:51:43.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.825+0000 7f074faab640 1 -- 172.21.15.115:0/4095805646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f07480c3a10 con 0x7f07480ce1b0 2023-12-01T02:51:43.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.836+0000 7f073e7fc640 1 -- 172.21.15.115:0/4095805646 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0738057510 con 0x7f07480ce1b0 2023-12-01T02:51:43.839 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 18 and data pool 19 2023-12-01T02:51:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.840+0000 7f074faab640 1 -- 172.21.15.115:0/4095805646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f072c06a550 msgr2=0x7f072c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.840+0000 7f074faab640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f072c06a550 0x7f072c06ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0744036000 tx=0x7f074403b040 comp rx=0 tx=0).stop 2023-12-01T02:51:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.840+0000 7f074faab640 1 -- 172.21.15.115:0/4095805646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07480ce1b0 msgr2=0x7f074815d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:43.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.840+0000 7f074faab640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07480ce1b0 0x7f074815d8e0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f073800c430 tx=0x7f073800c8f0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.841+0000 7f074faab640 1 -- 172.21.15.115:0/4095805646 shutdown_connections 2023-12-01T02:51:43.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.841+0000 7f074faab640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f072c06a550 0x7f072c06ca10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.841+0000 7f074faab640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f074815ac80 0x7f0748149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.841+0000 7f074faab640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07480ce1b0 0x7f074815d8e0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.841+0000 7f074faab640 1 --2- 172.21.15.115:0/4095805646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07480b0e60 0x7f074815c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:43.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.841+0000 7f074faab640 1 -- 172.21.15.115:0/4095805646 >> 172.21.15.115:0/4095805646 conn(0x7f0748065960 msgr2=0x7f07480ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:43.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.842+0000 7f074faab640 1 -- 172.21.15.115:0/4095805646 shutdown_connections 2023-12-01T02:51:43.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:43.843+0000 7f074faab640 1 -- 172.21.15.115:0/4095805646 wait complete. 2023-12-01T02:51:43.876 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:51:44.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.051+0000 7f4e539ce640 1 Processor -- start 2023-12-01T02:51:44.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.051+0000 7f4e539ce640 1 -- start start 2023-12-01T02:51:44.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.053+0000 7f4e539ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c15e2f0 0x7f4e4c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.053+0000 7f4e539ce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0add10 con 0x7f4e4c0adf10 2023-12-01T02:51:44.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.053+0000 7f4e539ce640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0c5760 con 0x7f4e4c06f2f0 2023-12-01T02:51:44.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.053+0000 7f4e539ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0c9cf0 con 0x7f4e4c15e2f0 2023-12-01T02:51:44.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.053+0000 7f4e51f6b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4e4c0adf10 0x7f4e4c0c95e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38548/0 (socket says 172.21.15.115:38548) 2023-12-01T02:51:44.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.053+0000 7f4e51f6b640 1 -- 172.21.15.115:0/4067905092 learned_addr learned my addr 172.21.15.115:0/4067905092 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:44.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.053+0000 7f4e5176a640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c15e2f0 0x7f4e4c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.053+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565700871 0 0) 0x7f4e4c0add10 con 0x7f4e4c0adf10 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.053+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e38003520 con 0x7f4e4c0adf10 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2951108203 0 0) 0x7f4e38003520 con 0x7f4e4c0adf10 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c15e2f0 msgr2=0x7f4e4c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c15e2f0 0x7f4e4c0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 >> v1:172.21.15.142:6789/0 conn(0x7f4e4c06f2f0 legacy=0x7f4e4c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c15e6d0 con 0x7f4e4c0adf10 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4e440019f0 con 0x7f4e4c0adf10 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 >> v1:172.21.15.115:6789/0 conn(0x7f4e4c0adf10 legacy=0x7f4e4c0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c15e2f0 0x7f4e38003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e38003fe0 0x7f4e380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e38004d50 0x7f4e380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0c9cf0 con 0x7f4e38004d50 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3800bc60 con 0x7f4e38003fe0 2023-12-01T02:51:44.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3800be20 con 0x7f4e4c15e2f0 2023-12-01T02:51:44.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.054+0000 7f4e51f6b640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e38004d50 0x7f4e380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.055+0000 7f4e5176a640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c15e2f0 0x7f4e38003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.055+0000 7f4e50f69640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e38003fe0 0x7f4e380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.055+0000 7f4e51f6b640 1 -- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c15e2f0 msgr2=0x7f4e38003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:44.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.055+0000 7f4e51f6b640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c15e2f0 0x7f4e38003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.055+0000 7f4e51f6b640 1 -- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e38003fe0 msgr2=0x7f4e380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:44.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.055+0000 7f4e51f6b640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e38003fe0 0x7f4e380047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.055+0000 7f4e51f6b640 1 -- 172.21.15.115:0/4067905092 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38003520 con 0x7f4e38004d50 2023-12-01T02:51:44.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.056+0000 7f4e51f6b640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e38004d50 0x7f4e380051d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0ae370 tx=0x7f4e380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d2dd95f45920a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:44.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.056+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44034630 con 0x7f4e38004d50 2023-12-01T02:51:44.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.056+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4402f070 con 0x7f4e38004d50 2023-12-01T02:51:44.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.056+0000 7f4e43fff640 1 -- 172.21.15.115:0/4067905092 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44034a50 con 0x7f4e38004d50 2023-12-01T02:51:44.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.057+0000 7f4e539ce640 1 -- 172.21.15.115:0/4067905092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e38004d50 msgr2=0x7f4e380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.057+0000 7f4e539ce640 1 --2- 172.21.15.115:0/4067905092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e38004d50 0x7f4e380051d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0ae370 tx=0x7f4e380057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.057+0000 7f4e51f6b640 1 -- 172.21.15.115:0/4067905092 reap_dead start 2023-12-01T02:51:44.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.057+0000 7f4e539ce640 1 -- 172.21.15.115:0/4067905092 shutdown_connections 2023-12-01T02:51:44.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.057+0000 7f4e539ce640 1 -- 172.21.15.115:0/4067905092 >> 172.21.15.115:0/4067905092 conn(0x7f4e4c0ce880 msgr2=0x7f4e4c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:44.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.057+0000 7f4e539ce640 1 -- 172.21.15.115:0/4067905092 shutdown_connections 2023-12-01T02:51:44.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.057+0000 7f4e539ce640 1 -- 172.21.15.115:0/4067905092 wait complete. 2023-12-01T02:51:44.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.059+0000 7f4e539ce640 1 Processor -- start 2023-12-01T02:51:44.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.059+0000 7f4e539ce640 1 -- start start 2023-12-01T02:51:44.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e539ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e38004d50 0x7f4e4c1f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e539ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c0c3070 0x7f4e4c0c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e539ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c0b1fd0 0x7f4e4c0b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e539ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0c3a30 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e539ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0b2990 con 0x7f4e38004d50 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e539ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c1f1670 con 0x7f4e4c0c3070 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e51f6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c0b1fd0 0x7f4e4c0b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e51f6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c0b1fd0 0x7f4e4c0b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39426/0 (socket says 172.21.15.115:39426) 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e51f6b640 1 -- 172.21.15.115:0/3076614613 learned_addr learned my addr 172.21.15.115:0/3076614613 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e5176a640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e38004d50 0x7f4e4c1f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.060+0000 7f4e50f69640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c0c3070 0x7f4e4c0c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e51f6b640 1 -- 172.21.15.115:0/3076614613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c0c3070 msgr2=0x7f4e4c0c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e51f6b640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c0c3070 0x7f4e4c0c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e51f6b640 1 -- 172.21.15.115:0/3076614613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e38004d50 msgr2=0x7f4e4c1f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e51f6b640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e38004d50 0x7f4e4c1f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e51f6b640 1 -- 172.21.15.115:0/3076614613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0c7790 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e51f6b640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c0b1fd0 0x7f4e4c0b2450 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400a850 tx=0x7f4e44001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:44.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e41ffb640 1 -- 172.21.15.115:0/3076614613 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44034840 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e41ffb640 1 -- 172.21.15.115:0/3076614613 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e440349a0 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e539ce640 1 -- 172.21.15.115:0/3076614613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c0c7a70 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e41ffb640 1 -- 172.21.15.115:0/3076614613 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44034d10 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.061+0000 7f4e539ce640 1 -- 172.21.15.115:0/3076614613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c0c8030 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.063+0000 7f4e539ce640 1 -- 172.21.15.115:0/3076614613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c06fda0 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.064+0000 7f4e41ffb640 1 -- 172.21.15.115:0/3076614613 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4e44031030 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.064+0000 7f4e41ffb640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e3006a580 0x7f4e3006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.065+0000 7f4e41ffb640 1 -- 172.21.15.115:0/3076614613 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f4e440aaf90 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.065+0000 7f4e5176a640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e3006a580 0x7f4e3006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.065+0000 7f4e5176a640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e3006a580 0x7f4e3006ca40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009a10 tx=0x7f4e48008040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:44.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.069+0000 7f4e41ffb640 1 -- 172.21.15.115:0/3076614613 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4e44076500 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.253+0000 7f4e539ce640 1 -- 172.21.15.115:0/3076614613 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e4c06fda0 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.255+0000 7f4e41ffb640 1 -- 172.21.15.115:0/3076614613 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16190 (secure 0 0 0) 0x7f4e44079b20 con 0x7f4e4c0b1fd0 2023-12-01T02:51:44.257 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:44.257 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":162,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:51:43.827010+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:51:41.383835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-01T02:51:42.832726+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:51:44.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.257+0000 7f4e539ce640 1 -- 172.21.15.115:0/3076614613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e3006a580 msgr2=0x7f4e3006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.257+0000 7f4e539ce640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e3006a580 0x7f4e3006ca40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009a10 tx=0x7f4e48008040 comp rx=0 tx=0).stop 2023-12-01T02:51:44.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.257+0000 7f4e539ce640 1 -- 172.21.15.115:0/3076614613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c0b1fd0 msgr2=0x7f4e4c0b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.257+0000 7f4e539ce640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c0b1fd0 0x7f4e4c0b2450 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400a850 tx=0x7f4e44001cb0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.257+0000 7f4e539ce640 1 -- 172.21.15.115:0/3076614613 shutdown_connections 2023-12-01T02:51:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.257+0000 7f4e539ce640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e3006a580 0x7f4e3006ca40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.257+0000 7f4e539ce640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e4c0c3070 0x7f4e4c0c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.257+0000 7f4e539ce640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c0b1fd0 0x7f4e4c0b2450 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.257+0000 7f4e539ce640 1 --2- 172.21.15.115:0/3076614613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e38004d50 0x7f4e4c1f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.258+0000 7f4e539ce640 1 -- 172.21.15.115:0/3076614613 >> 172.21.15.115:0/3076614613 conn(0x7f4e4c0ce880 msgr2=0x7f4e4c15b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.258+0000 7f4e539ce640 1 -- 172.21.15.115:0/3076614613 shutdown_connections 2023-12-01T02:51:44.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.258+0000 7f4e539ce640 1 -- 172.21.15.115:0/3076614613 wait complete. 2023-12-01T02:51:44.276 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:51:44.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.487+0000 7f05c1d27640 1 Processor -- start 2023-12-01T02:51:44.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.488+0000 7f05c1d27640 1 -- start start 2023-12-01T02:51:44.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.489+0000 7f05c1d27640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f05bc15e2f0 0x7f05bc0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.489+0000 7f05c1d27640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc0add10 con 0x7f05bc0adf10 2023-12-01T02:51:44.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.489+0000 7f05c1d27640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc0c5760 con 0x7f05bc06f2f0 2023-12-01T02:51:44.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.489+0000 7f05c1d27640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05bc0c9cf0 con 0x7f05bc15e2f0 2023-12-01T02:51:44.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.489+0000 7f05bb7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f05bc15e2f0 0x7f05bc0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.489+0000 7f05bbfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f05bc0adf10 0x7f05bc0c95e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38552/0 (socket says 172.21.15.115:38552) 2023-12-01T02:51:44.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.489+0000 7f05bbfff640 1 -- 172.21.15.115:0/1910858872 learned_addr learned my addr 172.21.15.115:0/1910858872 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:44.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.490+0000 7f05bb7fe640 1 -- 172.21.15.115:0/1910858872 >> v1:172.21.15.142:6789/0 conn(0x7f05bc06f2f0 legacy=0x7f05bc0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:44.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.490+0000 7f05bb7fe640 1 -- 172.21.15.115:0/1910858872 >> v1:172.21.15.115:6789/0 conn(0x7f05bc0adf10 legacy=0x7f05bc0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.490+0000 7f05bb7fe640 1 -- 172.21.15.115:0/1910858872 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc15e6d0 con 0x7f05bc15e2f0 2023-12-01T02:51:44.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.490+0000 7f05ba7fc640 1 -- 172.21.15.115:0/1910858872 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478300528 0 0) 0x7f05bc0add10 con 0x7f05bc0adf10 2023-12-01T02:51:44.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.491+0000 7f05bb7fe640 1 --2- 172.21.15.115:0/1910858872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc15e2f0 0x7f05bc0ca660 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0097a0 tx=0x7f05ac02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550b3262d0069773 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:44.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.491+0000 7f05ba7fc640 1 -- 172.21.15.115:0/1910858872 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac02f050 con 0x7f05bc15e2f0 2023-12-01T02:51:44.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.491+0000 7f05ba7fc640 1 -- 172.21.15.115:0/1910858872 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac031700 con 0x7f05bc15e2f0 2023-12-01T02:51:44.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.491+0000 7f05ba7fc640 1 -- 172.21.15.115:0/1910858872 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac031a90 con 0x7f05bc15e2f0 2023-12-01T02:51:44.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.492+0000 7f05c1d27640 1 -- 172.21.15.115:0/1910858872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc15e2f0 msgr2=0x7f05bc0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.492+0000 7f05c1d27640 1 --2- 172.21.15.115:0/1910858872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc15e2f0 0x7f05bc0ca660 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0097a0 tx=0x7f05ac02b7a0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.492+0000 7f05c1d27640 1 -- 172.21.15.115:0/1910858872 shutdown_connections 2023-12-01T02:51:44.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.492+0000 7f05c1d27640 1 --2- 172.21.15.115:0/1910858872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc15e2f0 0x7f05bc0ca660 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.492+0000 7f05c1d27640 1 -- 172.21.15.115:0/1910858872 >> 172.21.15.115:0/1910858872 conn(0x7f05bc0ce880 msgr2=0x7f05bc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:44.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.492+0000 7f05c1d27640 1 -- 172.21.15.115:0/1910858872 shutdown_connections 2023-12-01T02:51:44.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.492+0000 7f05c1d27640 1 -- 172.21.15.115:0/1910858872 wait complete. 2023-12-01T02:51:44.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.494+0000 7f05c1d27640 1 Processor -- start 2023-12-01T02:51:44.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.494+0000 7f05c1d27640 1 -- start start 2023-12-01T02:51:44.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05c1d27640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc06f2f0 0x7f05bc158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05c1d27640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05bc0adf10 0x7f05bc0c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05c1d27640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05bc15e2f0 0x7f05bc0c3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05c1d27640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0c5f10 con 0x7f05bc06f2f0 2023-12-01T02:51:44.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05c1d27640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0c3910 con 0x7f05bc15e2f0 2023-12-01T02:51:44.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05c1d27640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0c3bf0 con 0x7f05bc0adf10 2023-12-01T02:51:44.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc06f2f0 0x7f05bc158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc06f2f0 0x7f05bc158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39456/0 (socket says 172.21.15.115:39456) 2023-12-01T02:51:44.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05bb7fe640 1 -- 172.21.15.115:0/194872413 learned_addr learned my addr 172.21.15.115:0/194872413 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:44.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05bb7fe640 1 -- 172.21.15.115:0/194872413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05bc0adf10 msgr2=0x7f05bc0c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.495+0000 7f05bb7fe640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05bc0adf10 0x7f05bc0c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.496+0000 7f05bb7fe640 1 -- 172.21.15.115:0/194872413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05bc15e2f0 msgr2=0x7f05bc0c3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.496+0000 7f05bb7fe640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05bc15e2f0 0x7f05bc0c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.496+0000 7f05bb7fe640 1 -- 172.21.15.115:0/194872413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac009400 con 0x7f05bc06f2f0 2023-12-01T02:51:44.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.496+0000 7f05bb7fe640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc06f2f0 0x7f05bc158f50 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0097a0 tx=0x7f05ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:44.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.496+0000 7f05b3fff640 1 -- 172.21.15.115:0/194872413 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac033c30 con 0x7f05bc06f2f0 2023-12-01T02:51:44.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.496+0000 7f05b3fff640 1 -- 172.21.15.115:0/194872413 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac033d90 con 0x7f05bc06f2f0 2023-12-01T02:51:44.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.496+0000 7f05c1d27640 1 -- 172.21.15.115:0/194872413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc0c3ed0 con 0x7f05bc06f2f0 2023-12-01T02:51:44.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.496+0000 7f05b3fff640 1 -- 172.21.15.115:0/194872413 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac03d6b0 con 0x7f05bc06f2f0 2023-12-01T02:51:44.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.496+0000 7f05c1d27640 1 -- 172.21.15.115:0/194872413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc159490 con 0x7f05bc06f2f0 2023-12-01T02:51:44.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.497+0000 7f05c1d27640 1 -- 172.21.15.115:0/194872413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc06fda0 con 0x7f05bc06f2f0 2023-12-01T02:51:44.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.498+0000 7f05b3fff640 1 -- 172.21.15.115:0/194872413 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f05ac03a040 con 0x7f05bc06f2f0 2023-12-01T02:51:44.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.499+0000 7f05b3fff640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f059c06a550 0x7f059c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.499+0000 7f05b3fff640 1 -- 172.21.15.115:0/194872413 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f05ac0aa9d0 con 0x7f05bc06f2f0 2023-12-01T02:51:44.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.499+0000 7f05baffd640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f059c06a550 0x7f059c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.500+0000 7f05baffd640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f059c06a550 0x7f059c06ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0c5d00 tx=0x7f05a4009230 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:44.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.503+0000 7f05b3fff640 1 -- 172.21.15.115:0/194872413 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f05ac075f20 con 0x7f05bc06f2f0 2023-12-01T02:51:44.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.682+0000 7f05c1d27640 1 -- 172.21.15.115:0/194872413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05bc04e890 con 0x7f05bc06f2f0 2023-12-01T02:51:44.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.683+0000 7f05b3fff640 1 -- 172.21.15.115:0/194872413 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16190 (secure 0 0 0) 0x7f05ac044090 con 0x7f05bc06f2f0 2023-12-01T02:51:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:44.686 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":162,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:51:43.827010+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:51:41.383835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-01T02:51:42.832726+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:51:44.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.686+0000 7f05c1d27640 1 -- 172.21.15.115:0/194872413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f059c06a550 msgr2=0x7f059c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.686+0000 7f05c1d27640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f059c06a550 0x7f059c06ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0c5d00 tx=0x7f05a4009230 comp rx=0 tx=0).stop 2023-12-01T02:51:44.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.686+0000 7f05c1d27640 1 -- 172.21.15.115:0/194872413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc06f2f0 msgr2=0x7f05bc158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.686+0000 7f05c1d27640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc06f2f0 0x7f05bc158f50 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0097a0 tx=0x7f05ac0046c0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.687+0000 7f05c1d27640 1 -- 172.21.15.115:0/194872413 shutdown_connections 2023-12-01T02:51:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.687+0000 7f05c1d27640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f059c06a550 0x7f059c06ca10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.687+0000 7f05c1d27640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05bc0adf10 0x7f05bc0c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.687+0000 7f05c1d27640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05bc15e2f0 0x7f05bc0c3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.687+0000 7f05c1d27640 1 --2- 172.21.15.115:0/194872413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05bc06f2f0 0x7f05bc158f50 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.687+0000 7f05c1d27640 1 -- 172.21.15.115:0/194872413 >> 172.21.15.115:0/194872413 conn(0x7f05bc0ce880 msgr2=0x7f05bc0cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.687+0000 7f05c1d27640 1 -- 172.21.15.115:0/194872413 shutdown_connections 2023-12-01T02:51:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.687+0000 7f05c1d27640 1 -- 172.21.15.115:0/194872413 wait complete. 2023-12-01T02:51:44.708 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T02:51:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.923+0000 7f37d9769640 1 Processor -- start 2023-12-01T02:51:44.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.923+0000 7f37d9769640 1 -- start start 2023-12-01T02:51:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.925+0000 7f37d9769640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f37d415ac80 0x7f37d40cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.925+0000 7f37d9769640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f37d40ce1b0 0x7f37d40b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.925+0000 7f37d9769640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d40cc9a0 con 0x7f37d40b0e60 2023-12-01T02:51:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.925+0000 7f37d9769640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d415b060 con 0x7f37d40ce1b0 2023-12-01T02:51:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.925+0000 7f37d9769640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d40b0ce0 con 0x7f37d415ac80 2023-12-01T02:51:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.925+0000 7f37d37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f37d40ce1b0 0x7f37d40b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.925+0000 7f37d37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f37d40ce1b0 0x7f37d40b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39464/0 (socket says 172.21.15.115:39464) 2023-12-01T02:51:44.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.925+0000 7f37d37fe640 1 -- 172.21.15.115:0/1828106784 learned_addr learned my addr 172.21.15.115:0/1828106784 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:44.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.926+0000 7f37d2ffd640 1 --2- 172.21.15.115:0/1828106784 >> v2:172.21.15.142:3300/0 conn(0x7f37d415ac80 0x7f37d40cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.926+0000 7f37d37fe640 1 -- 172.21.15.115:0/1828106784 >> v2:172.21.15.142:3300/0 conn(0x7f37d415ac80 msgr2=0x7f37d40cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:44.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.926+0000 7f37d37fe640 1 --2- 172.21.15.115:0/1828106784 >> v2:172.21.15.142:3300/0 conn(0x7f37d415ac80 0x7f37d40cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.926+0000 7f37d37fe640 1 -- 172.21.15.115:0/1828106784 >> v1:172.21.15.142:6789/0 conn(0x7f37d40b0e60 legacy=0x7f37d40cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:44.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.926+0000 7f37d37fe640 1 -- 172.21.15.115:0/1828106784 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d40cbd90 con 0x7f37d40ce1b0 2023-12-01T02:51:44.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.927+0000 7f37d37fe640 1 --2- 172.21.15.115:0/1828106784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40ce1b0 0x7f37d40b07a0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f37d40b3cc0 tx=0x7f37cc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251af33c0442acb4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:44.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.927+0000 7f37d27fc640 1 -- 172.21.15.115:0/1828106784 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc02ca80 con 0x7f37d40ce1b0 2023-12-01T02:51:44.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.927+0000 7f37d27fc640 1 -- 172.21.15.115:0/1828106784 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37cc02cbe0 con 0x7f37d40ce1b0 2023-12-01T02:51:44.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.927+0000 7f37d27fc640 1 -- 172.21.15.115:0/1828106784 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0343f0 con 0x7f37d40ce1b0 2023-12-01T02:51:44.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.928+0000 7f37d9769640 1 -- 172.21.15.115:0/1828106784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40ce1b0 msgr2=0x7f37d40b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.928+0000 7f37d9769640 1 --2- 172.21.15.115:0/1828106784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40ce1b0 0x7f37d40b07a0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f37d40b3cc0 tx=0x7f37cc02bee0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.928+0000 7f37d9769640 1 -- 172.21.15.115:0/1828106784 shutdown_connections 2023-12-01T02:51:44.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.928+0000 7f37d9769640 1 --2- 172.21.15.115:0/1828106784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40ce1b0 0x7f37d40b07a0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.928+0000 7f37d9769640 1 --2- 172.21.15.115:0/1828106784 >> v2:172.21.15.142:3300/0 conn(0x7f37d415ac80 0x7f37d40cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.928+0000 7f37d9769640 1 -- 172.21.15.115:0/1828106784 >> 172.21.15.115:0/1828106784 conn(0x7f37d4065960 msgr2=0x7f37d4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:44.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.928+0000 7f37d9769640 1 -- 172.21.15.115:0/1828106784 shutdown_connections 2023-12-01T02:51:44.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.928+0000 7f37d9769640 1 -- 172.21.15.115:0/1828106784 wait complete. 2023-12-01T02:51:44.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.930+0000 7f37d9769640 1 Processor -- start 2023-12-01T02:51:44.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.930+0000 7f37d9769640 1 -- start start 2023-12-01T02:51:44.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.931+0000 7f37d9769640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40b0e60 0x7f37d41464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.931+0000 7f37d9769640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d40ce1b0 0x7f37d4151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.931+0000 7f37d9769640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37d415ac80 0x7f37d4070120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.931+0000 7f37d9769640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4152b60 con 0x7f37d40b0e60 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.931+0000 7f37d9769640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d41525a0 con 0x7f37d40ce1b0 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.931+0000 7f37d9769640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4152880 con 0x7f37d415ac80 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40b0e60 0x7f37d41464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40b0e60 0x7f37d41464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39478/0 (socket says 172.21.15.115:39478) 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d2ffd640 1 -- 172.21.15.115:0/1456966269 learned_addr learned my addr 172.21.15.115:0/1456966269 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d37fe640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37d415ac80 0x7f37d4070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37cbfff640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d40ce1b0 0x7f37d4151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d2ffd640 1 -- 172.21.15.115:0/1456966269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37d415ac80 msgr2=0x7f37d4070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d2ffd640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37d415ac80 0x7f37d4070120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d2ffd640 1 -- 172.21.15.115:0/1456966269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d40ce1b0 msgr2=0x7f37d4151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d2ffd640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d40ce1b0 0x7f37d4151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d2ffd640 1 -- 172.21.15.115:0/1456966269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc009b30 con 0x7f37d40b0e60 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d37fe640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37d415ac80 0x7f37d4070120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:44.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.932+0000 7f37d2ffd640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40b0e60 0x7f37d41464b0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f37c400c7f0 tx=0x7f37c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:44.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.933+0000 7f37cb7fe640 1 -- 172.21.15.115:0/1456966269 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4004220 con 0x7f37d40b0e60 2023-12-01T02:51:44.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.933+0000 7f37cb7fe640 1 -- 172.21.15.115:0/1456966269 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c4004380 con 0x7f37d40b0e60 2023-12-01T02:51:44.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.933+0000 7f37cb7fe640 1 -- 172.21.15.115:0/1456966269 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4007840 con 0x7f37d40b0e60 2023-12-01T02:51:44.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.933+0000 7f37d9769640 1 -- 172.21.15.115:0/1456966269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d4156c90 con 0x7f37d40b0e60 2023-12-01T02:51:44.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.933+0000 7f37d9769640 1 -- 172.21.15.115:0/1456966269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d41469f0 con 0x7f37d40b0e60 2023-12-01T02:51:44.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.934+0000 7f37d9769640 1 -- 172.21.15.115:0/1456966269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d41571b0 con 0x7f37d40b0e60 2023-12-01T02:51:44.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.935+0000 7f37cb7fe640 1 -- 172.21.15.115:0/1456966269 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f37c4007bb0 con 0x7f37d40b0e60 2023-12-01T02:51:44.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.936+0000 7f37cb7fe640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37b406a580 0x7f37b406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:44.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.937+0000 7f37cb7fe640 1 -- 172.21.15.115:0/1456966269 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f37c40889f0 con 0x7f37d40b0e60 2023-12-01T02:51:44.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.937+0000 7f37cbfff640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37b406a580 0x7f37b406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:44.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.937+0000 7f37cbfff640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37b406a580 0x7f37b406ca40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f37d4156810 tx=0x7f37bc009040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:44.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:44.941+0000 7f37cb7fe640 1 -- 172.21.15.115:0/1456966269 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f37d41571b0 con 0x7f37d40b0e60 2023-12-01T02:51:45.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.169+0000 7f37d9769640 1 -- 172.21.15.115:0/1456966269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f37d41571b0 con 0x7f37d40b0e60 2023-12-01T02:51:45.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.850+0000 7f37cb7fe640 1 -- 172.21.15.115:0/1456966269 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7f37d41571b0 con 0x7f37d40b0e60 2023-12-01T02:51:45.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 -- 172.21.15.115:0/1456966269 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37b406a580 msgr2=0x7f37b406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:45.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37b406a580 0x7f37b406ca40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f37d4156810 tx=0x7f37bc009040 comp rx=0 tx=0).stop 2023-12-01T02:51:45.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 -- 172.21.15.115:0/1456966269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40b0e60 msgr2=0x7f37d41464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:45.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40b0e60 0x7f37d41464b0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f37c400c7f0 tx=0x7f37c400ccb0 comp rx=0 tx=0).stop 2023-12-01T02:51:45.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 -- 172.21.15.115:0/1456966269 shutdown_connections 2023-12-01T02:51:45.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f37b406a580 0x7f37b406ca40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:45.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37d415ac80 0x7f37d4070120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:45.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d40ce1b0 0x7f37d4151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:45.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 --2- 172.21.15.115:0/1456966269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37d40b0e60 0x7f37d41464b0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:45.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 -- 172.21.15.115:0/1456966269 >> 172.21.15.115:0/1456966269 conn(0x7f37d4065960 msgr2=0x7f37d40ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:45.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 -- 172.21.15.115:0/1456966269 shutdown_connections 2023-12-01T02:51:45.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:45.854+0000 7f37d9769640 1 -- 172.21.15.115:0/1456966269 wait complete. 2023-12-01T02:51:45.877 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T02:51:45.877 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T02:51:46.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.096+0000 7fd15e08c640 1 Processor -- start 2023-12-01T02:51:46.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.097+0000 7fd15e08c640 1 -- start start 2023-12-01T02:51:46.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.098+0000 7fd15e08c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd158153c30 0x7fd1580ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:46.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.098+0000 7fd15e08c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd1580cfb80 0x7fd158160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:46.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.098+0000 7fd15e08c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1580cdcd0 con 0x7fd158154010 2023-12-01T02:51:46.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.098+0000 7fd15e08c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd158161830 con 0x7fd158153c30 2023-12-01T02:51:46.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.098+0000 7fd15e08c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580cd3a0 con 0x7fd1580cfb80 2023-12-01T02:51:46.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.099+0000 7fd15d08a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd158153c30 0x7fd1580ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:46.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.099+0000 7fd15d08a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd158153c30 0x7fd1580ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39494/0 (socket says 172.21.15.115:39494) 2023-12-01T02:51:46.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.099+0000 7fd15d08a640 1 -- 172.21.15.115:0/3206974314 learned_addr learned my addr 172.21.15.115:0/3206974314 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:46.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.099+0000 7fd15c889640 1 --1- 172.21.15.115:0/3206974314 >> v1:172.21.15.115:6789/0 conn(0x7fd158154010 0x7fd1580cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38568/0 (socket says 172.21.15.115:38568) 2023-12-01T02:51:46.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.099+0000 7fd15d88b640 1 --2- 172.21.15.115:0/3206974314 >> v2:172.21.15.142:3300/0 conn(0x7fd1580cfb80 0x7fd158160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:46.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.099+0000 7fd15d08a640 1 -- 172.21.15.115:0/3206974314 >> v2:172.21.15.142:3300/0 conn(0x7fd1580cfb80 msgr2=0x7fd158160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:46.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.099+0000 7fd15d08a640 1 --2- 172.21.15.115:0/3206974314 >> v2:172.21.15.142:3300/0 conn(0x7fd1580cfb80 0x7fd158160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:46.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.099+0000 7fd15d08a640 1 -- 172.21.15.115:0/3206974314 >> v1:172.21.15.115:6789/0 conn(0x7fd158154010 legacy=0x7fd1580cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:46.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.099+0000 7fd15d08a640 1 -- 172.21.15.115:0/3206974314 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1580cd680 con 0x7fd158153c30 2023-12-01T02:51:46.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.099+0000 7fd15d88b640 1 --2- 172.21.15.115:0/3206974314 >> v2:172.21.15.142:3300/0 conn(0x7fd1580cfb80 0x7fd158160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:46.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.100+0000 7fd15d08a640 1 --2- 172.21.15.115:0/3206974314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd158153c30 0x7fd1580ce160 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd15815afc0 tx=0x7fd14802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d35b3ddecbe5b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:46.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.100+0000 7fd157fff640 1 -- 172.21.15.115:0/3206974314 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14802cb10 con 0x7fd158153c30 2023-12-01T02:51:46.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.101+0000 7fd157fff640 1 -- 172.21.15.115:0/3206974314 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14802cc70 con 0x7fd158153c30 2023-12-01T02:51:46.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.101+0000 7fd157fff640 1 -- 172.21.15.115:0/3206974314 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1480343f0 con 0x7fd158153c30 2023-12-01T02:51:46.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.101+0000 7fd15e08c640 1 -- 172.21.15.115:0/3206974314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd158153c30 msgr2=0x7fd1580ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:46.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.101+0000 7fd15e08c640 1 --2- 172.21.15.115:0/3206974314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd158153c30 0x7fd1580ce160 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd15815afc0 tx=0x7fd14802bf70 comp rx=0 tx=0).stop 2023-12-01T02:51:46.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.101+0000 7fd15e08c640 1 -- 172.21.15.115:0/3206974314 shutdown_connections 2023-12-01T02:51:46.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.101+0000 7fd15e08c640 1 --2- 172.21.15.115:0/3206974314 >> v2:172.21.15.142:3300/0 conn(0x7fd1580cfb80 0x7fd158160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:46.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.101+0000 7fd15e08c640 1 --2- 172.21.15.115:0/3206974314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd158153c30 0x7fd1580ce160 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:46.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.101+0000 7fd15e08c640 1 -- 172.21.15.115:0/3206974314 >> 172.21.15.115:0/3206974314 conn(0x7fd15806a0b0 msgr2=0x7fd15806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:46.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.101+0000 7fd15e08c640 1 -- 172.21.15.115:0/3206974314 shutdown_connections 2023-12-01T02:51:46.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.101+0000 7fd15e08c640 1 -- 172.21.15.115:0/3206974314 wait complete. 2023-12-01T02:51:46.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.103+0000 7fd15e08c640 1 Processor -- start 2023-12-01T02:51:46.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.103+0000 7fd15e08c640 1 -- start start 2023-12-01T02:51:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.104+0000 7fd15e08c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1580cfb80 0x7fd1580c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.104+0000 7fd15e08c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd158153c30 0x7fd1580c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.104+0000 7fd15e08c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd158154010 0x7fd1580c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.104+0000 7fd15e08c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580ca060 con 0x7fd1580cfb80 2023-12-01T02:51:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.104+0000 7fd15e08c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd158154f30 con 0x7fd158154010 2023-12-01T02:51:46.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.104+0000 7fd15e08c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd158155210 con 0x7fd158153c30 2023-12-01T02:51:46.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.104+0000 7fd15d08a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1580cfb80 0x7fd1580c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:46.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.104+0000 7fd15d08a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1580cfb80 0x7fd1580c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39506/0 (socket says 172.21.15.115:39506) 2023-12-01T02:51:46.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.104+0000 7fd15d08a640 1 -- 172.21.15.115:0/622673660 learned_addr learned my addr 172.21.15.115:0/622673660 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:46.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.105+0000 7fd15d08a640 1 -- 172.21.15.115:0/622673660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd158153c30 msgr2=0x7fd1580c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:46.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.105+0000 7fd15d08a640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd158153c30 0x7fd1580c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:46.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.105+0000 7fd15d08a640 1 -- 172.21.15.115:0/622673660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd158154010 msgr2=0x7fd1580c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:46.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.105+0000 7fd15d08a640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd158154010 0x7fd1580c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:46.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.105+0000 7fd15d08a640 1 -- 172.21.15.115:0/622673660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd148009b30 con 0x7fd1580cfb80 2023-12-01T02:51:46.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.105+0000 7fd15d08a640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1580cfb80 0x7fd1580c5c70 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd148000f80 tx=0x7fd148004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:46.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.105+0000 7fd155ffb640 1 -- 172.21.15.115:0/622673660 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14803e450 con 0x7fd1580cfb80 2023-12-01T02:51:46.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.105+0000 7fd155ffb640 1 -- 172.21.15.115:0/622673660 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14803e5f0 con 0x7fd1580cfb80 2023-12-01T02:51:46.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.105+0000 7fd15e08c640 1 -- 172.21.15.115:0/622673660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1581554f0 con 0x7fd1580cfb80 2023-12-01T02:51:46.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.105+0000 7fd155ffb640 1 -- 172.21.15.115:0/622673660 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14803e9a0 con 0x7fd1580cfb80 2023-12-01T02:51:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.106+0000 7fd15e08c640 1 -- 172.21.15.115:0/622673660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1580c9320 con 0x7fd1580cfb80 2023-12-01T02:51:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.107+0000 7fd15e08c640 1 -- 172.21.15.115:0/622673660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1580c94b0 con 0x7fd1580cfb80 2023-12-01T02:51:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.108+0000 7fd155ffb640 1 -- 172.21.15.115:0/622673660 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd14803eb00 con 0x7fd1580cfb80 2023-12-01T02:51:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.109+0000 7fd155ffb640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd14406a550 0x7fd14406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.109+0000 7fd155ffb640 1 -- 172.21.15.115:0/622673660 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fd1480aba90 con 0x7fd1580cfb80 2023-12-01T02:51:46.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.109+0000 7fd15c889640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd14406a550 0x7fd14406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.110+0000 7fd15c889640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd14406a550 0x7fd14406ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd158151580 tx=0x7fd14c0079c0 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:46.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.113+0000 7fd155ffb640 1 -- 172.21.15.115:0/622673660 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd148076ea0 con 0x7fd1580cfb80 2023-12-01T02:51:46.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.341+0000 7fd15e08c640 1 -- 172.21.15.115:0/622673660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd1580618d0 con 0x7fd1580cfb80 2023-12-01T02:51:46.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.854+0000 7fd155ffb640 1 -- 172.21.15.115:0/622673660 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd14803edb0 con 0x7fd1580cfb80 2023-12-01T02:51:46.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 -- 172.21.15.115:0/622673660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd14406a550 msgr2=0x7fd14406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd14406a550 0x7fd14406ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd158151580 tx=0x7fd14c0079c0 comp rx=0 tx=0).stop 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 -- 172.21.15.115:0/622673660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1580cfb80 msgr2=0x7fd1580c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1580cfb80 0x7fd1580c5c70 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd148000f80 tx=0x7fd148004a40 comp rx=0 tx=0).stop 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 -- 172.21.15.115:0/622673660 shutdown_connections 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd14406a550 0x7fd14406ca10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd158153c30 0x7fd1580c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd158154010 0x7fd1580c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 --2- 172.21.15.115:0/622673660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1580cfb80 0x7fd1580c5c70 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 -- 172.21.15.115:0/622673660 >> 172.21.15.115:0/622673660 conn(0x7fd15806a0b0 msgr2=0x7fd15815d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 -- 172.21.15.115:0/622673660 shutdown_connections 2023-12-01T02:51:46.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:46.858+0000 7fd15e08c640 1 -- 172.21.15.115:0/622673660 wait complete. 2023-12-01T02:51:46.881 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T02:51:46.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:51:47.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.088+0000 7fa91634e640 1 Processor -- start 2023-12-01T02:51:47.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.088+0000 7fa91634e640 1 -- start start 2023-12-01T02:51:47.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.089+0000 7fa91634e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa91015e2f0 0x7fa9100ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.089+0000 7fa91634e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa91006f2f0 0x7fa9100caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.089+0000 7fa91634e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9100adf10 0x7fa9100c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.089+0000 7fa91634e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9100add10 con 0x7fa91006f2f0 2023-12-01T02:51:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.089+0000 7fa91634e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9100c5760 con 0x7fa9100adf10 2023-12-01T02:51:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.089+0000 7fa91634e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91015e6d0 con 0x7fa91015e2f0 2023-12-01T02:51:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa907fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa91006f2f0 0x7fa9100caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa907fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa91006f2f0 0x7fa9100caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39522/0 (socket says 172.21.15.115:39522) 2023-12-01T02:51:47.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa907fff640 1 -- 172.21.15.115:0/1681832139 learned_addr learned my addr 172.21.15.115:0/1681832139 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa90ffff640 1 --2- 172.21.15.115:0/1681832139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa91015e2f0 0x7fa9100ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa907fff640 1 -- 172.21.15.115:0/1681832139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa91015e2f0 msgr2=0x7fa9100ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa9148eb640 1 --2- 172.21.15.115:0/1681832139 >> v2:172.21.15.142:3300/0 conn(0x7fa9100adf10 0x7fa9100c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa907fff640 1 --2- 172.21.15.115:0/1681832139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa91015e2f0 0x7fa9100ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa907fff640 1 -- 172.21.15.115:0/1681832139 >> v2:172.21.15.142:3300/0 conn(0x7fa9100adf10 msgr2=0x7fa9100c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa907fff640 1 --2- 172.21.15.115:0/1681832139 >> v2:172.21.15.142:3300/0 conn(0x7fa9100adf10 0x7fa9100c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa907fff640 1 -- 172.21.15.115:0/1681832139 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100cf7b0 con 0x7fa91006f2f0 2023-12-01T02:51:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.090+0000 7fa9148eb640 1 --2- 172.21.15.115:0/1681832139 >> v2:172.21.15.142:3300/0 conn(0x7fa9100adf10 0x7fa9100c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:47.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.091+0000 7fa907fff640 1 --2- 172.21.15.115:0/1681832139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 0x7fa9100caba0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa9100cb2b0 tx=0x7fa8f802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b47fcae3949941 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:47.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.092+0000 7fa90f7fe640 1 -- 172.21.15.115:0/1681832139 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8031750 con 0x7fa91006f2f0 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.092+0000 7fa90f7fe640 1 -- 172.21.15.115:0/1681832139 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f80318b0 con 0x7fa91006f2f0 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.092+0000 7fa90f7fe640 1 -- 172.21.15.115:0/1681832139 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8031c40 con 0x7fa91006f2f0 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.092+0000 7fa91634e640 1 -- 172.21.15.115:0/1681832139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 msgr2=0x7fa9100caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.092+0000 7fa91634e640 1 --2- 172.21.15.115:0/1681832139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 0x7fa9100caba0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa9100cb2b0 tx=0x7fa8f802b7f0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.092+0000 7fa91634e640 1 -- 172.21.15.115:0/1681832139 shutdown_connections 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.092+0000 7fa91634e640 1 --2- 172.21.15.115:0/1681832139 >> v2:172.21.15.142:3300/0 conn(0x7fa9100adf10 0x7fa9100c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.092+0000 7fa91634e640 1 --2- 172.21.15.115:0/1681832139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 0x7fa9100caba0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.092+0000 7fa91634e640 1 --2- 172.21.15.115:0/1681832139 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa91015e2f0 0x7fa9100ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.092+0000 7fa91634e640 1 -- 172.21.15.115:0/1681832139 >> 172.21.15.115:0/1681832139 conn(0x7fa9100ce880 msgr2=0x7fa9100cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.093+0000 7fa91634e640 1 -- 172.21.15.115:0/1681832139 shutdown_connections 2023-12-01T02:51:47.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.093+0000 7fa91634e640 1 -- 172.21.15.115:0/1681832139 wait complete. 2023-12-01T02:51:47.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.094+0000 7fa91634e640 1 Processor -- start 2023-12-01T02:51:47.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.095+0000 7fa91634e640 1 -- start start 2023-12-01T02:51:47.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.095+0000 7fa91634e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 0x7fa910151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.095+0000 7fa91634e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9100adf10 0x7fa9100c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.095+0000 7fa91634e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa91015e2f0 0x7fa9100c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.095+0000 7fa91634e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c9dd0 con 0x7fa91006f2f0 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.095+0000 7fa91634e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c4980 con 0x7fa91015e2f0 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.095+0000 7fa91634e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c4c60 con 0x7fa9100adf10 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.096+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 0x7fa910151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.096+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 0x7fa910151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39524/0 (socket says 172.21.15.115:39524) 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.096+0000 7fa90ffff640 1 -- 172.21.15.115:0/4199186024 learned_addr learned my addr 172.21.15.115:0/4199186024 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.096+0000 7fa907fff640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9100adf10 0x7fa9100c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.096+0000 7fa90ffff640 1 -- 172.21.15.115:0/4199186024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9100adf10 msgr2=0x7fa9100c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.096+0000 7fa90ffff640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9100adf10 0x7fa9100c3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.096+0000 7fa9148eb640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa91015e2f0 0x7fa9100c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.096+0000 7fa90ffff640 1 -- 172.21.15.115:0/4199186024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa91015e2f0 msgr2=0x7fa9100c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.096+0000 7fa90ffff640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa91015e2f0 0x7fa9100c4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.096+0000 7fa90ffff640 1 -- 172.21.15.115:0/4199186024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f80094d0 con 0x7fa91006f2f0 2023-12-01T02:51:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.097+0000 7fa90ffff640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 0x7fa910151be0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c7f0 tx=0x7fa90000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.097+0000 7fa90d7fa640 1 -- 172.21.15.115:0/4199186024 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900004220 con 0x7fa91006f2f0 2023-12-01T02:51:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.097+0000 7fa90d7fa640 1 -- 172.21.15.115:0/4199186024 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa900004380 con 0x7fa91006f2f0 2023-12-01T02:51:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.097+0000 7fa90d7fa640 1 -- 172.21.15.115:0/4199186024 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900007840 con 0x7fa91006f2f0 2023-12-01T02:51:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.097+0000 7fa91634e640 1 -- 172.21.15.115:0/4199186024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9100ca040 con 0x7fa91006f2f0 2023-12-01T02:51:47.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.097+0000 7fa91634e640 1 -- 172.21.15.115:0/4199186024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9100c70e0 con 0x7fa91006f2f0 2023-12-01T02:51:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.098+0000 7fa91634e640 1 -- 172.21.15.115:0/4199186024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91006fda0 con 0x7fa91006f2f0 2023-12-01T02:51:47.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.099+0000 7fa90d7fa640 1 -- 172.21.15.115:0/4199186024 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa900007bb0 con 0x7fa91006f2f0 2023-12-01T02:51:47.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.100+0000 7fa90d7fa640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8f006a550 0x7fa8f006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.100+0000 7fa90d7fa640 1 -- 172.21.15.115:0/4199186024 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fa900088b20 con 0x7fa91006f2f0 2023-12-01T02:51:47.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.101+0000 7fa907fff640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8f006a550 0x7fa8f006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.101+0000 7fa907fff640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8f006a550 0x7fa8f006ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8035000 tx=0x7fa8f8008030 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:47.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.105+0000 7fa90d7fa640 1 -- 172.21.15.115:0/4199186024 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa900053eb0 con 0x7fa91006f2f0 2023-12-01T02:51:47.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.335+0000 7fa91634e640 1 -- 172.21.15.115:0/4199186024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa91006f850 con 0x7fa91006f2f0 2023-12-01T02:51:47.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.336+0000 7fa90d7fa640 1 -- 172.21.15.115:0/4199186024 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fa900007e60 con 0x7fa91006f2f0 2023-12-01T02:51:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:47.339 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1952979218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1952979218},{"type":"v1","addr":"172.21.15.142:6833","nonce":1952979218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3618529680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3618529680},{"type":"v1","addr":"172.21.15.142:6835","nonce":3618529680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":9009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1637880399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1637880399},{"type":"v1","addr":"172.21.15.115:6836","nonce":1637880399}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:51:43.827542+0000","modified":"2023-12-01T02:51:46.847110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3770885942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3770885942},{"type":"v1","addr":"172.21.15.115:6837","nonce":3770885942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-01T02:51:47.339 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 71 2023-12-01T02:51:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 -- 172.21.15.115:0/4199186024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8f006a550 msgr2=0x7fa8f006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8f006a550 0x7fa8f006ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8035000 tx=0x7fa8f8008030 comp rx=0 tx=0).stop 2023-12-01T02:51:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 -- 172.21.15.115:0/4199186024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 msgr2=0x7fa910151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 0x7fa910151be0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c7f0 tx=0x7fa90000ccb0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 -- 172.21.15.115:0/4199186024 shutdown_connections 2023-12-01T02:51:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa8f006a550 0x7fa8f006ca10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9100adf10 0x7fa9100c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa91015e2f0 0x7fa9100c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 --2- 172.21.15.115:0/4199186024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa91006f2f0 0x7fa910151be0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 -- 172.21.15.115:0/4199186024 >> 172.21.15.115:0/4199186024 conn(0x7fa9100ce880 msgr2=0x7fa91015c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:47.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 -- 172.21.15.115:0/4199186024 shutdown_connections 2023-12-01T02:51:47.342 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.339+0000 7fa91634e640 1 -- 172.21.15.115:0/4199186024 wait complete. 2023-12-01T02:51:47.361 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:51:47.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.574+0000 7f3891c28640 1 Processor -- start 2023-12-01T02:51:47.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.575+0000 7f3891c28640 1 -- start start 2023-12-01T02:51:47.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.576+0000 7f3891c28640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f388c15cd20 0x7f388c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.576+0000 7f3891c28640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388c0ce2e0 con 0x7f388c0d1230 2023-12-01T02:51:47.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.576+0000 7f3891c28640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388c0ce000 con 0x7f388c0b5d80 2023-12-01T02:51:47.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.576+0000 7f3891c28640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0b2160 con 0x7f388c15cd20 2023-12-01T02:51:47.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.576+0000 7f388bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f388c0d1230 0x7f388c0d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38570/0 (socket says 172.21.15.115:38570) 2023-12-01T02:51:47.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.576+0000 7f388bfff640 1 -- 172.21.15.115:0/1964930009 learned_addr learned my addr 172.21.15.115:0/1964930009 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:47.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388b7fe640 1 --2- 172.21.15.115:0/1964930009 >> v2:172.21.15.142:3300/0 conn(0x7f388c15cd20 0x7f388c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916115285 0 0) 0x7f388c0ce2e0 con 0x7f388c0d1230 2023-12-01T02:51:47.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3878003520 con 0x7f388c0d1230 2023-12-01T02:51:47.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2348428143 0 0) 0x7f3878003520 con 0x7f388c0d1230 2023-12-01T02:51:47.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 >> v2:172.21.15.142:3300/0 conn(0x7f388c15cd20 msgr2=0x7f388c0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:47.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 --2- 172.21.15.115:0/1964930009 >> v2:172.21.15.142:3300/0 conn(0x7f388c15cd20 0x7f388c0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 >> v1:172.21.15.142:6789/0 conn(0x7f388c0b5d80 legacy=0x7f388c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0cd3d0 con 0x7f388c0d1230 2023-12-01T02:51:47.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3884001670 con 0x7f388c0d1230 2023-12-01T02:51:47.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 >> v1:172.21.15.115:6789/0 conn(0x7f388c0d1230 legacy=0x7f388c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 --2- 172.21.15.115:0/1964930009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3878003a30 0x7f3878003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 --2- 172.21.15.115:0/1964930009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38780047c0 0x7f3878004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 --2- 172.21.15.115:0/1964930009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38780052a0 0x7f387800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387800c140 con 0x7f38780047c0 2023-12-01T02:51:47.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387800c2b0 con 0x7f38780052a0 2023-12-01T02:51:47.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.577+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387800c470 con 0x7f3878003a30 2023-12-01T02:51:47.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.578+0000 7f388affd640 1 --2- 172.21.15.115:0/1964930009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38780047c0 0x7f3878004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.578+0000 7f388b7fe640 1 --2- 172.21.15.115:0/1964930009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3878003a30 0x7f3878003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.578+0000 7f388affd640 1 -- 172.21.15.115:0/1964930009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3878003a30 msgr2=0x7f3878003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:47.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.578+0000 7f388affd640 1 --2- 172.21.15.115:0/1964930009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3878003a30 0x7f3878003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.578+0000 7f388affd640 1 -- 172.21.15.115:0/1964930009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38780052a0 msgr2=0x7f387800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:47.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.578+0000 7f388affd640 1 --2- 172.21.15.115:0/1964930009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38780052a0 0x7f387800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.578+0000 7f388affd640 1 -- 172.21.15.115:0/1964930009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3878003520 con 0x7f38780047c0 2023-12-01T02:51:47.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.578+0000 7f388bfff640 1 -- 172.21.15.115:0/1964930009 reap_dead start 2023-12-01T02:51:47.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.579+0000 7f388affd640 1 --2- 172.21.15.115:0/1964930009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38780047c0 0x7f3878004c40 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3874001750 tx=0x7f38780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8beb8394766b617e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:47.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.580+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38740052d0 con 0x7f38780047c0 2023-12-01T02:51:47.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.580+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387402e050 con 0x7f38780047c0 2023-12-01T02:51:47.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.580+0000 7f388a7fc640 1 -- 172.21.15.115:0/1964930009 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874003830 con 0x7f38780047c0 2023-12-01T02:51:47.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.580+0000 7f3891c28640 1 -- 172.21.15.115:0/1964930009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38780047c0 msgr2=0x7f3878004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.580+0000 7f3891c28640 1 --2- 172.21.15.115:0/1964930009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38780047c0 0x7f3878004c40 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3874001750 tx=0x7f38780057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.580+0000 7f3891c28640 1 -- 172.21.15.115:0/1964930009 shutdown_connections 2023-12-01T02:51:47.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.580+0000 7f3891c28640 1 --2- 172.21.15.115:0/1964930009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38780047c0 0x7f3878004c40 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.580+0000 7f3891c28640 1 -- 172.21.15.115:0/1964930009 >> 172.21.15.115:0/1964930009 conn(0x7f388c06a0e0 msgr2=0x7f388c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:47.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.580+0000 7f3891c28640 1 -- 172.21.15.115:0/1964930009 shutdown_connections 2023-12-01T02:51:47.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.580+0000 7f3891c28640 1 -- 172.21.15.115:0/1964930009 wait complete. 2023-12-01T02:51:47.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.582+0000 7f3891c28640 1 Processor -- start 2023-12-01T02:51:47.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.582+0000 7f3891c28640 1 -- start start 2023-12-01T02:51:47.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.583+0000 7f3891c28640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38780047c0 0x7f388c147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.583+0000 7f3891c28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388c14a400 0x7f388c0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.583+0000 7f3891c28640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f388c0c8c30 0x7f388c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.583+0000 7f3891c28640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0b54e0 con 0x7f388c14a400 2023-12-01T02:51:47.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.583+0000 7f3891c28640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0b21e0 con 0x7f388c0c8c30 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.583+0000 7f3891c28640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0b24c0 con 0x7f38780047c0 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.584+0000 7f388affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388c14a400 0x7f388c0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.584+0000 7f388affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388c14a400 0x7f388c0b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39548/0 (socket says 172.21.15.115:39548) 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.584+0000 7f388affd640 1 -- 172.21.15.115:0/64261083 learned_addr learned my addr 172.21.15.115:0/64261083 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.584+0000 7f388affd640 1 -- 172.21.15.115:0/64261083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38780047c0 msgr2=0x7f388c147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.584+0000 7f388b7fe640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38780047c0 0x7f388c147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.584+0000 7f388affd640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38780047c0 0x7f388c147850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.584+0000 7f388affd640 1 -- 172.21.15.115:0/64261083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f388c0c8c30 msgr2=0x7f388c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.584+0000 7f388affd640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f388c0c8c30 0x7f388c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.584+0000 7f388affd640 1 -- 172.21.15.115:0/64261083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0b27a0 con 0x7f388c14a400 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.584+0000 7f388affd640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388c14a400 0x7f388c0b4fa0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3874005040 tx=0x7f3874001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.585+0000 7f3883fff640 1 -- 172.21.15.115:0/64261083 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38740076c0 con 0x7f388c14a400 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.585+0000 7f3883fff640 1 -- 172.21.15.115:0/64261083 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38740069e0 con 0x7f388c14a400 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.585+0000 7f3883fff640 1 -- 172.21.15.115:0/64261083 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874006d90 con 0x7f388c14a400 2023-12-01T02:51:47.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.585+0000 7f3891c28640 1 -- 172.21.15.115:0/64261083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c155160 con 0x7f388c14a400 2023-12-01T02:51:47.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.585+0000 7f3891c28640 1 -- 172.21.15.115:0/64261083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c155720 con 0x7f388c14a400 2023-12-01T02:51:47.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.586+0000 7f3891c28640 1 -- 172.21.15.115:0/64261083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388c0b54e0 con 0x7f388c14a400 2023-12-01T02:51:47.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.587+0000 7f3883fff640 1 -- 172.21.15.115:0/64261083 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f38740035f0 con 0x7f388c14a400 2023-12-01T02:51:47.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.588+0000 7f3883fff640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f386c06a550 0x7f386c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:47.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.588+0000 7f3883fff640 1 -- 172.21.15.115:0/64261083 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f38740aa900 con 0x7f388c14a400 2023-12-01T02:51:47.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.588+0000 7f388b7fe640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f386c06a550 0x7f386c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:47.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.589+0000 7f388b7fe640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f386c06a550 0x7f386c06ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f388c04f940 tx=0x7f387c00b040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:47.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.592+0000 7f3883fff640 1 -- 172.21.15.115:0/64261083 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3874075ca0 con 0x7f388c14a400 2023-12-01T02:51:47.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.786+0000 7f3891c28640 1 -- 172.21.15.115:0/64261083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f388c1558b0 con 0x7f388c14a400 2023-12-01T02:51:47.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.789+0000 7f3883fff640 1 -- 172.21.15.115:0/64261083 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+16177 (secure 0 0 0) 0x7f387404b020 con 0x7f388c14a400 2023-12-01T02:51:47.791 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:47.791 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":163,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:51:44.831803+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:51:41.383835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-01T02:51:42.832726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:51:47.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.791+0000 7f3891c28640 1 -- 172.21.15.115:0/64261083 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f386c06a550 msgr2=0x7f386c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.791+0000 7f3891c28640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f386c06a550 0x7f386c06ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f388c04f940 tx=0x7f387c00b040 comp rx=0 tx=0).stop 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.791+0000 7f3891c28640 1 -- 172.21.15.115:0/64261083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388c14a400 msgr2=0x7f388c0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.791+0000 7f3891c28640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388c14a400 0x7f388c0b4fa0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3874005040 tx=0x7f3874001120 comp rx=0 tx=0).stop 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.792+0000 7f3891c28640 1 -- 172.21.15.115:0/64261083 shutdown_connections 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.792+0000 7f3891c28640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f386c06a550 0x7f386c06ca10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.792+0000 7f3891c28640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f388c0c8c30 0x7f388c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.792+0000 7f3891c28640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f388c14a400 0x7f388c0b4fa0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.792+0000 7f3891c28640 1 --2- 172.21.15.115:0/64261083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38780047c0 0x7f388c147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.792+0000 7f3891c28640 1 -- 172.21.15.115:0/64261083 >> 172.21.15.115:0/64261083 conn(0x7f388c06a0e0 msgr2=0x7f388c15eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.792+0000 7f3891c28640 1 -- 172.21.15.115:0/64261083 shutdown_connections 2023-12-01T02:51:47.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:47.792+0000 7f3891c28640 1 -- 172.21.15.115:0/64261083 wait complete. 2023-12-01T02:51:47.812 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T02:51:47.812 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T02:51:48.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.026+0000 7f79ab0ee640 1 Processor -- start 2023-12-01T02:51:48.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.026+0000 7f79ab0ee640 1 -- start start 2023-12-01T02:51:48.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.027+0000 7f79ab0ee640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f79a415dfd0 0x7f79a40cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79ab0ee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a40cf9a0 0x7f79a40b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79ab0ee640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a4160b20 con 0x7f79a415dbf0 2023-12-01T02:51:48.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79ab0ee640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a4160dc0 con 0x7f79a415dfd0 2023-12-01T02:51:48.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79ab0ee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41610a0 con 0x7f79a40cf9a0 2023-12-01T02:51:48.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79a3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f79a415dfd0 0x7f79a40cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79a3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f79a415dfd0 0x7f79a40cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45542/0 (socket says 172.21.15.115:45542) 2023-12-01T02:51:48.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79a3fff640 1 -- 172.21.15.115:0/2933911982 learned_addr learned my addr 172.21.15.115:0/2933911982 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79a968b640 1 --2- 172.21.15.115:0/2933911982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a40cf9a0 0x7f79a40b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79a3fff640 1 -- 172.21.15.115:0/2933911982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a40cf9a0 msgr2=0x7f79a40b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79a3fff640 1 --2- 172.21.15.115:0/2933911982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a40cf9a0 0x7f79a40b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79a3fff640 1 -- 172.21.15.115:0/2933911982 >> v1:172.21.15.142:6789/0 conn(0x7f79a415dbf0 legacy=0x7f79a40cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.028+0000 7f79a3fff640 1 -- 172.21.15.115:0/2933911982 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a4161380 con 0x7f79a415dfd0 2023-12-01T02:51:48.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.030+0000 7f79a3fff640 1 --2- 172.21.15.115:0/2933911982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dfd0 0x7f79a40cf460 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f798c009870 tx=0x7f798c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137e9ff0ee72e450 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:48.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.030+0000 7f79a37fe640 1 -- 172.21.15.115:0/2933911982 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c031640 con 0x7f79a415dfd0 2023-12-01T02:51:48.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.030+0000 7f79a37fe640 1 -- 172.21.15.115:0/2933911982 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c0317a0 con 0x7f79a415dfd0 2023-12-01T02:51:48.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.030+0000 7f79a37fe640 1 -- 172.21.15.115:0/2933911982 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c031b30 con 0x7f79a415dfd0 2023-12-01T02:51:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.030+0000 7f79ab0ee640 1 -- 172.21.15.115:0/2933911982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dfd0 msgr2=0x7f79a40cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.030+0000 7f79ab0ee640 1 --2- 172.21.15.115:0/2933911982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dfd0 0x7f79a40cf460 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f798c009870 tx=0x7f798c02b8f0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.031+0000 7f79ab0ee640 1 -- 172.21.15.115:0/2933911982 shutdown_connections 2023-12-01T02:51:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.031+0000 7f79ab0ee640 1 --2- 172.21.15.115:0/2933911982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a40cf9a0 0x7f79a40b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.031+0000 7f79ab0ee640 1 --2- 172.21.15.115:0/2933911982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dfd0 0x7f79a40cf460 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.031+0000 7f79ab0ee640 1 -- 172.21.15.115:0/2933911982 >> 172.21.15.115:0/2933911982 conn(0x7f79a4065960 msgr2=0x7f79a4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.031+0000 7f79ab0ee640 1 -- 172.21.15.115:0/2933911982 shutdown_connections 2023-12-01T02:51:48.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.031+0000 7f79ab0ee640 1 -- 172.21.15.115:0/2933911982 wait complete. 2023-12-01T02:51:48.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.032+0000 7f79ab0ee640 1 Processor -- start 2023-12-01T02:51:48.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.033+0000 7f79ab0ee640 1 -- start start 2023-12-01T02:51:48.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.033+0000 7f79ab0ee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40cf9a0 0x7f79a40c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.033+0000 7f79ab0ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dbf0 0x7f79a4154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.033+0000 7f79ab0ee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415dfd0 0x7f79a414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.033+0000 7f79ab0ee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4154cf0 con 0x7f79a415dbf0 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.033+0000 7f79ab0ee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41510e0 con 0x7f79a40cf9a0 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79ab0ee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4154a10 con 0x7f79a415dfd0 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dbf0 0x7f79a4154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dbf0 0x7f79a4154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45558/0 (socket says 172.21.15.115:45558) 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a3fff640 1 -- 172.21.15.115:0/4117470466 learned_addr learned my addr 172.21.15.115:0/4117470466 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a8e8a640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40cf9a0 0x7f79a40c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a3fff640 1 -- 172.21.15.115:0/4117470466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415dfd0 msgr2=0x7f79a414eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a968b640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415dfd0 0x7f79a414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a3fff640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415dfd0 0x7f79a414eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a3fff640 1 -- 172.21.15.115:0/4117470466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40cf9a0 msgr2=0x7f79a40c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a3fff640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40cf9a0 0x7f79a40c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a3fff640 1 -- 172.21.15.115:0/4117470466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c0094d0 con 0x7f79a415dbf0 2023-12-01T02:51:48.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.034+0000 7f79a8e8a640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40cf9a0 0x7f79a40c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:48.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.035+0000 7f79a3fff640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dbf0 0x7f79a4154110 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f798c009870 tx=0x7f798c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:48.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.035+0000 7f79a17fa640 1 -- 172.21.15.115:0/4117470466 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c03ac60 con 0x7f79a415dbf0 2023-12-01T02:51:48.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.035+0000 7f79a17fa640 1 -- 172.21.15.115:0/4117470466 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c03ade0 con 0x7f79a415dbf0 2023-12-01T02:51:48.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.035+0000 7f79a17fa640 1 -- 172.21.15.115:0/4117470466 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c044630 con 0x7f79a415dbf0 2023-12-01T02:51:48.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.035+0000 7f79ab0ee640 1 -- 172.21.15.115:0/4117470466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a4151380 con 0x7f79a415dbf0 2023-12-01T02:51:48.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.035+0000 7f79ab0ee640 1 -- 172.21.15.115:0/4117470466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a4154650 con 0x7f79a415dbf0 2023-12-01T02:51:48.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.036+0000 7f79ab0ee640 1 -- 172.21.15.115:0/4117470466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a404e890 con 0x7f79a415dbf0 2023-12-01T02:51:48.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.037+0000 7f79a17fa640 1 -- 172.21.15.115:0/4117470466 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f798c02f050 con 0x7f79a415dbf0 2023-12-01T02:51:48.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.038+0000 7f79a17fa640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 0x7f798406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.038+0000 7f79a17fa640 1 -- 172.21.15.115:0/4117470466 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f798c0aadb0 con 0x7f79a415dbf0 2023-12-01T02:51:48.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.038+0000 7f79a8e8a640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 0x7f798406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.039+0000 7f79a8e8a640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 0x7f798406ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7994004520 tx=0x7f7994009040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:48.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.042+0000 7f79a17fa640 1 -- 172.21.15.115:0/4117470466 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f798c0af070 con 0x7f79a415dbf0 2023-12-01T02:51:48.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.227+0000 7f79ab0ee640 1 -- 172.21.15.115:0/4117470466 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79a40cfdf0 con 0x7f798406a550 2023-12-01T02:51:48.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.240+0000 7f79a17fa640 1 -- 172.21.15.115:0/4117470466 <== mgr.8769 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276046 (secure 0 0 0) 0x7f799400d8d0 con 0x7f798406a550 2023-12-01T02:51:48.246 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:48.247 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.246+0000 7f79ab0ee640 1 -- 172.21.15.115:0/4117470466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 msgr2=0x7f798406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.246+0000 7f79ab0ee640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 0x7f798406ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7994004520 tx=0x7f7994009040 comp rx=0 tx=0).stop 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.246+0000 7f79ab0ee640 1 -- 172.21.15.115:0/4117470466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dbf0 msgr2=0x7f79a4154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.246+0000 7f79ab0ee640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dbf0 0x7f79a4154110 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f798c009870 tx=0x7f798c0046c0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.246+0000 7f79ab0ee640 1 -- 172.21.15.115:0/4117470466 shutdown_connections 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.246+0000 7f79ab0ee640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f798406a550 0x7f798406ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.246+0000 7f79ab0ee640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a415dfd0 0x7f79a414eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.246+0000 7f79ab0ee640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79a415dbf0 0x7f79a4154110 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.246+0000 7f79ab0ee640 1 --2- 172.21.15.115:0/4117470466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a40cf9a0 0x7f79a40c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.247+0000 7f79ab0ee640 1 -- 172.21.15.115:0/4117470466 >> 172.21.15.115:0/4117470466 conn(0x7f79a4065960 msgr2=0x7f79a40c90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:48.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.247+0000 7f79ab0ee640 1 -- 172.21.15.115:0/4117470466 shutdown_connections 2023-12-01T02:51:48.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.247+0000 7f79ab0ee640 1 -- 172.21.15.115:0/4117470466 wait complete. 2023-12-01T02:51:48.268 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T02:51:46.648360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":383948,"kb_used_data":16652,"kb_used_omap":0,"kb_used_meta":367232,"kb_avail":754590772,"statfs":{"total":773094113280,"available":772700950528,"internally_reserved":0,"allocated":17051648,"data_stored":8822127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":376045568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.324006"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.336029+0000","last_change":"2023-12-01T02:51:42.336029+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.336029+0000","last_undegraded":"2023-12-01T02:51:42.336029+0000","last_fullsized":"2023-12-01T02:51:42.336029+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:28:31.216397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.330249+0000","last_change":"2023-12-01T02:51:42.330249+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.330249+0000","last_undegraded":"2023-12-01T02:51:42.330249+0000","last_fullsized":"2023-12-01T02:51:42.330249+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:17:14.987724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.339343+0000","last_change":"2023-12-01T02:51:42.339343+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.339343+0000","last_undegraded":"2023-12-01T02:51:42.339343+0000","last_fullsized":"2023-12-01T02:51:42.339343+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:56.639523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.334397+0000","last_change":"2023-12-01T02:51:42.334397+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.334397+0000","last_undegraded":"2023-12-01T02:51:42.334397+0000","last_fullsized":"2023-12-01T02:51:42.334397+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:23:23.540300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.334240+0000","last_change":"2023-12-01T02:51:42.334240+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.334240+0000","last_undegraded":"2023-12-01T02:51:42.334240+0000","last_fullsized":"2023-12-01T02:51:42.334240+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:30.270995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.331552+0000","last_change":"2023-12-01T02:51:42.331552+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.331552+0000","last_undegraded":"2023-12-01T02:51:42.331552+0000","last_fullsized":"2023-12-01T02:51:42.331552+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:46:16.844083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.330265+0000","last_change":"2023-12-01T02:51:42.330265+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.330265+0000","last_undegraded":"2023-12-01T02:51:42.330265+0000","last_fullsized":"2023-12-01T02:51:42.330265+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:42:36.272402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.332158+0000","last_change":"2023-12-01T02:51:42.332158+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.332158+0000","last_undegraded":"2023-12-01T02:51:42.332158+0000","last_fullsized":"2023-12-01T02:51:42.332158+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:00:11.086798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.340024+0000","last_change":"2023-12-01T02:51:42.340024+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.340024+0000","last_undegraded":"2023-12-01T02:51:42.340024+0000","last_fullsized":"2023-12-01T02:51:42.340024+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:11:28.528185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.331392+0000","last_change":"2023-12-01T02:51:42.331392+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.331392+0000","last_undegraded":"2023-12-01T02:51:42.331392+0000","last_fullsized":"2023-12-01T02:51:42.331392+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:41.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.337193+0000","last_change":"2023-12-01T02:51:42.337193+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.337193+0000","last_undegraded":"2023-12-01T02:51:42.337193+0000","last_fullsized":"2023-12-01T02:51:42.337193+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:12:22.519981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.334476+0000","last_change":"2023-12-01T02:51:42.334476+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.334476+0000","last_undegraded":"2023-12-01T02:51:42.334476+0000","last_fullsized":"2023-12-01T02:51:42.334476+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:03.506649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.437510+0000","last_change":"2023-12-01T02:51:42.437510+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.437510+0000","last_undegraded":"2023-12-01T02:51:42.437510+0000","last_fullsized":"2023-12-01T02:51:42.437510+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:02:25.926506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.340106+0000","last_change":"2023-12-01T02:51:42.340106+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.340106+0000","last_undegraded":"2023-12-01T02:51:42.340106+0000","last_fullsized":"2023-12-01T02:51:42.340106+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:42:36.272402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.337038+0000","last_change":"2023-12-01T02:51:42.337038+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.337038+0000","last_undegraded":"2023-12-01T02:51:42.337038+0000","last_fullsized":"2023-12-01T02:51:42.337038+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:58.168046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.338923+0000","last_change":"2023-12-01T02:51:42.338923+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.338923+0000","last_undegraded":"2023-12-01T02:51:42.338923+0000","last_fullsized":"2023-12-01T02:51:42.338923+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:06.631985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.339197+0000","last_change":"2023-12-01T02:51:42.339197+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.339197+0000","last_undegraded":"2023-12-01T02:51:42.339197+0000","last_fullsized":"2023-12-01T02:51:42.339197+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:44.731298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.329770+0000","last_change":"2023-12-01T02:51:42.329770+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.329770+0000","last_undegraded":"2023-12-01T02:51:42.329770+0000","last_fullsized":"2023-12-01T02:51:42.329770+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:19.891780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.335880+0000","last_change":"2023-12-01T02:51:42.335880+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.335880+0000","last_undegraded":"2023-12-01T02:51:42.335880+0000","last_fullsized":"2023-12-01T02:51:42.335880+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:56:15.834365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.330141+0000","last_change":"2023-12-01T02:51:42.330141+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.330141+0000","last_undegraded":"2023-12-01T02:51:42.330141+0000","last_fullsized":"2023-12-01T02:51:42.330141+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:44.731298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.330103+0000","last_change":"2023-12-01T02:51:42.330103+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.330103+0000","last_undegraded":"2023-12-01T02:51:42.330103+0000","last_fullsized":"2023-12-01T02:51:42.330103+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:15.600950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.437376+0000","last_change":"2023-12-01T02:51:42.437376+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.437376+0000","last_undegraded":"2023-12-01T02:51:42.437376+0000","last_fullsized":"2023-12-01T02:51:42.437376+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:48:48.689940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.338603+0000","last_change":"2023-12-01T02:51:42.338603+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.338603+0000","last_undegraded":"2023-12-01T02:51:42.338603+0000","last_fullsized":"2023-12-01T02:51:42.338603+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:10:55.039229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.331272+0000","last_change":"2023-12-01T02:51:42.331272+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.331272+0000","last_undegraded":"2023-12-01T02:51:42.331272+0000","last_fullsized":"2023-12-01T02:51:42.331272+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:42.251726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.336774+0000","last_change":"2023-12-01T02:51:42.336774+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.336774+0000","last_undegraded":"2023-12-01T02:51:42.336774+0000","last_fullsized":"2023-12-01T02:51:42.336774+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:08:07.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.330453+0000","last_change":"2023-12-01T02:51:42.330453+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.330453+0000","last_undegraded":"2023-12-01T02:51:42.330453+0000","last_fullsized":"2023-12-01T02:51:42.330453+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:58.168046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.437370+0000","last_change":"2023-12-01T02:51:42.437370+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.437370+0000","last_undegraded":"2023-12-01T02:51:42.437370+0000","last_fullsized":"2023-12-01T02:51:42.437370+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:08.095677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.332840+0000","last_change":"2023-12-01T02:51:42.332840+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.332840+0000","last_undegraded":"2023-12-01T02:51:42.332840+0000","last_fullsized":"2023-12-01T02:51:42.332840+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:05:25.494760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.336838+0000","last_change":"2023-12-01T02:51:42.336838+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.336838+0000","last_undegraded":"2023-12-01T02:51:42.336838+0000","last_fullsized":"2023-12-01T02:51:42.336838+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:42.251726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.338757+0000","last_change":"2023-12-01T02:51:42.338757+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.338757+0000","last_undegraded":"2023-12-01T02:51:42.338757+0000","last_fullsized":"2023-12-01T02:51:42.338757+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:08.604890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.437074+0000","last_change":"2023-12-01T02:51:42.437074+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.437074+0000","last_undegraded":"2023-12-01T02:51:42.437074+0000","last_fullsized":"2023-12-01T02:51:42.437074+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:30.398882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:51:43.324525+0000","last_change":"2023-12-01T02:51:43.324525+0000","last_active":"2023-12-01T02:51:43.324525+0000","last_peered":"2023-12-01T02:51:43.324525+0000","last_clean":"2023-12-01T02:51:43.324525+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:43.324525+0000","last_undegraded":"2023-12-01T02:51:43.324525+0000","last_fullsized":"2023-12-01T02:51:43.324525+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":160,"state":"creating+peering","last_fresh":"2023-12-01T02:51:42.337656+0000","last_change":"2023-12-01T02:51:42.337656+0000","last_active":"2023-12-01T02:51:42.307107+0000","last_peered":"2023-12-01T02:51:42.307107+0000","last_clean":"2023-12-01T02:51:42.307107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:51:42.337656+0000","last_undegraded":"2023-12-01T02:51:42.337656+0000","last_fullsized":"2023-12-01T02:51:42.337656+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:41.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"151'249","reported_seq":788,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-12-01T02:51:39.550150+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T02:51:39.550150+0000","last_peered":"2023-12-01T02:51:39.550150+0000","last_clean":"2023-12-01T02:51:39.550150+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T02:51:39.550150+0000","last_undegraded":"2023-12-01T02:51:39.550150+0000","last_fullsized":"2023-12-01T02:51:39.550150+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575755,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55136,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316704,"statfs":{"total":96636764160,"available":96580304896,"internally_reserved":0,"allocated":1990656,"data_stored":958837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3799999999999999,"5min":1.7829999999999999,"15min":1.893},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1539999999999999,"15min":1.1100000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.514,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.96799999999999997}]},{"osd":1,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.266,"15min":1.2709999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.02},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85399999999999998},"max":{"1min":1.585,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":1.387}]},{"osd":2,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.2010000000000001},"min":{"1min":0.82499999999999996,"5min":0.80600000000000005,"15min":0.78500000000000003},"max":{"1min":1.554,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.252,"5min":1.28,"15min":1.25},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8440000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.996}]},{"osd":3,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.343,"15min":1.3220000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.742,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.224},{"interface":"front","average":{"1min":1.415,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.048,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.1719999999999999}]},{"osd":4,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.851,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.506,"5min":1.4930000000000001,"15min":1.4750000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.157,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.4450000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3149999999999999,"15min":1.284},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.0699999999999998,"5min":2.976,"15min":2.976},"last":1.288},{"interface":"front","average":{"1min":1.597,"5min":1.595,"15min":1.569},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.2160000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.569}]},{"osd":6,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4590000000000001,"15min":1.4179999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.1499999999999999,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4870000000000001,"15min":1.4470000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.966,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.357}]}]},{"osd":6,"up_from":13,"seq":55834575756,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1932,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1978368,"data_stored":952976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.161,"15min":1.165},"min":{"1min":0.84299999999999997,"5min":0.78100000000000003,"15min":0.75},"max":{"1min":1.6080000000000001,"5min":1.73,"15min":2.2759999999999998},"last":1.016},{"interface":"front","average":{"1min":1.083,"5min":1.1439999999999999,"15min":1.1619999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.56799999999999995},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.966},"last":1.1699999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1459999999999999,"15min":1.1859999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.78800000000000003},"max":{"1min":1.6459999999999999,"5min":1.772,"15min":2.0409999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6679999999999999,"5min":1.7370000000000001,"15min":2.093},"last":1.4339999999999999}]},{"osd":2,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1679999999999999,"15min":1.1879999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.80600000000000005},"max":{"1min":1.5369999999999999,"5min":1.736,"15min":1.946},"last":1.639},{"interface":"front","average":{"1min":1.274,"5min":1.272,"15min":1.284},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.637,"5min":1.7190000000000001,"15min":2.1230000000000002},"last":1.696}]},{"osd":3,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":1.0620000000000001,"5min":1.006,"15min":0.90500000000000003},"max":{"1min":1.6140000000000001,"5min":1.819,"15min":1.819},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.431,"15min":1.4319999999999999},"min":{"1min":1.042,"5min":0.98699999999999999,"15min":0.92800000000000005},"max":{"1min":1.738,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.756}]},{"osd":4,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.262},"min":{"1min":0.93000000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.585},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3500000000000001,"15min":1.3360000000000001},"min":{"1min":0.96699999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.98,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.875}]},{"osd":5,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.4419999999999999,"15min":1.4279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":2.0830000000000002},{"interface":"front","average":{"1min":1.603,"5min":1.5,"15min":1.4970000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.7799999999999998},"last":2.29}]},{"osd":7,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.5149999999999999,"15min":1.524},"min":{"1min":1.0980000000000001,"5min":1.034,"15min":1.034},"max":{"1min":1.9419999999999999,"5min":2.1549999999999998,"15min":2.5659999999999998},"last":2.1960000000000002},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6040000000000001,"15min":1.5940000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.0499999999999998,"5min":2.226,"15min":2.3809999999999998},"last":2.161}]}]},{"osd":5,"up_from":13,"seq":55834575755,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59812,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312028,"statfs":{"total":96636764160,"available":96575516672,"internally_reserved":0,"allocated":2584576,"data_stored":1549205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1499999999999999,"15min":1.1419999999999999},"min":{"1min":0.68700000000000006,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4810000000000001,"5min":1.73,"15min":1.744},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.3140000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.137,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.56200000000000006},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.704},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.226,"15min":1.2330000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.016},"last":1.143}]},{"osd":2,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2270000000000001,"15min":1.24},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5900000000000001,"5min":1.7430000000000001,"15min":1.746},"last":1.474},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2609999999999999,"15min":1.2809999999999999},"min":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.627},"max":{"1min":1.655,"5min":1.671,"15min":1.764},"last":1.0900000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.321,"15min":1.319},"min":{"1min":0.98399999999999999,"5min":0.872,"15min":0.84899999999999998},"max":{"1min":1.546,"5min":1.7729999999999999,"15min":1.871},"last":1.411},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3460000000000001,"15min":1.3620000000000001},"min":{"1min":1.0269999999999999,"5min":0.91500000000000004,"15min":0.879},"max":{"1min":1.7250000000000001,"5min":1.7470000000000001,"15min":1.962},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.254,"15min":1.2569999999999999},"min":{"1min":0.91300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.696,"5min":1.847,"15min":1.978},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.361,"15min":1.3620000000000001},"min":{"1min":1.0740000000000001,"5min":0.91600000000000004,"15min":0.88600000000000001},"max":{"1min":1.911,"5min":1.911,"15min":2.0350000000000001},"last":1.583}]},{"osd":6,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3520000000000001,"15min":1.375},"min":{"1min":0.89300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":2.4470000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.391,"15min":1.419},"min":{"1min":1.077,"5min":0.876,"15min":0.876},"max":{"1min":1.798,"5min":1.946,"15min":2.7210000000000001},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.5449999999999999,"15min":1.5369999999999999},"min":{"1min":1.1200000000000001,"5min":1.099,"15min":0.92100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":3.0379999999999998},"last":1.532},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5780000000000001,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":3.5270000000000001},"last":1.46}]}]},{"osd":4,"up_from":13,"seq":55834575755,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46932,"kb_used_data":1932,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324908,"statfs":{"total":96636764160,"available":96588705792,"internally_reserved":0,"allocated":1978368,"data_stored":952976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.163,"15min":1.1519999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70699999999999996},"max":{"1min":1.444,"5min":1.5269999999999999,"15min":1.6830000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.749},"max":{"1min":1.466,"5min":1.581,"15min":1.724},"last":1.5880000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.272,"15min":1.2330000000000001},"min":{"1min":0.93000000000000005,"5min":0.86299999999999999,"15min":0.81399999999999995},"max":{"1min":1.5800000000000001,"5min":1.724,"15min":1.7709999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1950000000000001,"15min":1.1970000000000001},"min":{"1min":0.80700000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.605,"5min":1.6100000000000001,"15min":1.8640000000000001},"last":1.381}]},{"osd":2,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.89400000000000002,"5min":0.83899999999999997,"15min":0.82799999999999996},"max":{"1min":1.5229999999999999,"5min":1.6419999999999999,"15min":2.1499999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.335,"15min":1.321},"min":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":0.85999999999999999},"max":{"1min":1.5760000000000001,"5min":1.802,"15min":1.802},"last":1.466}]},{"osd":3,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2769999999999999,"15min":1.2689999999999999},"min":{"1min":1,"5min":0.95299999999999996,"15min":0.85399999999999998},"max":{"1min":1.536,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3460000000000001,"15min":1.333},"min":{"1min":1.046,"5min":1.0009999999999999,"15min":0.88100000000000001},"max":{"1min":1.6180000000000001,"5min":1.7050000000000001,"15min":2.2440000000000002},"last":1.3999999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.3149999999999999},"min":{"1min":0.96199999999999997,"5min":0.86499999999999999,"15min":0.84499999999999997},"max":{"1min":1.677,"5min":1.804,"15min":1.8839999999999999},"last":1.736},{"interface":"front","average":{"1min":1.29,"5min":1.321,"15min":1.2829999999999999},"min":{"1min":0.871,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.75,"5min":1.875,"15min":1.875},"last":1.4370000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.373,"15min":1.325},"min":{"1min":0.89100000000000001,"5min":0.81299999999999994,"15min":0.78500000000000003},"max":{"1min":1.7490000000000001,"5min":1.853,"15min":2.4489999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.4610000000000001,"15min":1.4450000000000001},"min":{"1min":1.0509999999999999,"5min":0.97999999999999998,"15min":0.97899999999999998},"max":{"1min":1.7290000000000001,"5min":1.8939999999999999,"15min":2.4870000000000001},"last":1.6759999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4670000000000001,"15min":1.4470000000000001},"min":{"1min":1.204,"5min":1.1020000000000001,"15min":0.92900000000000005},"max":{"1min":1.8540000000000001,"5min":2.044,"15min":2.5529999999999999},"last":1.526},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.5600000000000001,"15min":1.5469999999999999},"min":{"1min":1.244,"5min":1.0600000000000001,"15min":0.99099999999999999},"max":{"1min":1.8899999999999999,"5min":2.0670000000000002,"15min":2.5760000000000001},"last":1.6180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51028,"kb_used_data":1932,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320812,"statfs":{"total":96636764160,"available":96584511488,"internally_reserved":0,"allocated":1978368,"data_stored":952976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.90900000000000003,"15min":0.88700000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3160000000000001,"5min":1.4970000000000001,"15min":1.5009999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.071,"15min":1.077},"min":{"1min":0.751,"5min":0.65000000000000002,"15min":0.60299999999999998},"max":{"1min":1.444,"5min":1.5449999999999999,"15min":1.5669999999999999},"last":0.91400000000000003}]},{"osd":1,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1839999999999999,"15min":1.159},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.755},"max":{"1min":1.625,"5min":1.708,"15min":1.708},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0089999999999999,"15min":1.018},"min":{"1min":0.77900000000000003,"5min":0.72999999999999998,"15min":0.52700000000000002},"max":{"1min":1.5609999999999999,"5min":1.5980000000000001,"15min":1.744},"last":0.65100000000000002}]},{"osd":2,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.081,"15min":1.099},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.7549999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.2589999999999999,"15min":1.264},"min":{"1min":0.93899999999999995,"5min":0.93000000000000005,"15min":0.53400000000000003},"max":{"1min":1.4870000000000001,"5min":1.6639999999999999,"15min":1.728},"last":1.1779999999999999}]},{"osd":4,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.294,"15min":1.29},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.89200000000000002},"max":{"1min":1.611,"5min":1.6479999999999999,"15min":1.7050000000000001},"last":1.198},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3140000000000001,"15min":1.2929999999999999},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":0.67100000000000004},"max":{"1min":1.6719999999999999,"5min":1.7789999999999999,"15min":1.927},"last":1.161}]},{"osd":5,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3180000000000001,"15min":1.3160000000000001},"min":{"1min":1.022,"5min":0.92300000000000004,"15min":0.92000000000000004},"max":{"1min":1.589,"5min":1.788,"15min":1.8260000000000001},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.4059999999999999,"15min":1.3859999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":0.94299999999999995},"max":{"1min":1.8,"5min":1.865,"15min":1.865},"last":1.113}]},{"osd":6,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.4630000000000001,"15min":1.4399999999999999},"min":{"1min":1.1040000000000001,"5min":1.069,"15min":0.67900000000000005},"max":{"1min":1.8440000000000001,"5min":2.0230000000000001,"15min":2.1160000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5349999999999999,"15min":1.496},"min":{"1min":1.3440000000000001,"5min":1.093,"15min":0.85499999999999998},"max":{"1min":1.867,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.2629999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5269999999999999,"15min":1.5109999999999999},"min":{"1min":1.228,"5min":1.177,"15min":0.80600000000000005},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.1549999999999998},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.506,"5min":1.528,"15min":1.516},"min":{"1min":1.3320000000000001,"5min":1.21,"15min":0.73699999999999999},"max":{"1min":1.8420000000000001,"5min":2.0150000000000001,"15min":2.1259999999999999},"last":1.526}]}]},{"osd":2,"up_from":13,"seq":55834575754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1932,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1978368,"data_stored":952976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.89700000000000002,"15min":0.872},"min":{"1min":0.72899999999999998,"5min":0.57299999999999995,"15min":0.49299999999999999},"max":{"1min":1.4059999999999999,"5min":1.532,"15min":1.5569999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.077,"15min":1.0660000000000001},"min":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.59599999999999997},"max":{"1min":1.4139999999999999,"5min":1.728,"15min":1.728},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.101,"15min":1.077},"min":{"1min":0.83899999999999997,"5min":0.80300000000000005,"15min":0.67400000000000004},"max":{"1min":1.502,"5min":1.7070000000000001,"15min":1.8029999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.113,"15min":1.1140000000000001},"min":{"1min":0.88700000000000001,"5min":0.77000000000000002,"15min":0.53300000000000003},"max":{"1min":1.383,"5min":1.6259999999999999,"15min":1.6599999999999999},"last":1.3720000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0780000000000001,"15min":1.056},"min":{"1min":0.81799999999999995,"5min":0.72799999999999998,"15min":0.55500000000000005},"max":{"1min":1.4239999999999999,"5min":1.492,"15min":1.7070000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2549999999999999,"15min":1.2170000000000001},"min":{"1min":0.98099999999999998,"5min":0.90100000000000002,"15min":0.70599999999999996},"max":{"1min":1.5329999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.3340000000000001}]},{"osd":4,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3500000000000001,"15min":1.319},"min":{"1min":1.123,"5min":0.89700000000000002,"15min":0.71099999999999997},"max":{"1min":1.7230000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.141},{"interface":"front","average":{"1min":1.248,"5min":1.3049999999999999,"15min":1.3009999999999999},"min":{"1min":0.96599999999999997,"5min":0.90700000000000003,"15min":0.78200000000000003},"max":{"1min":1.518,"5min":1.78,"15min":1.8520000000000001},"last":1.3999999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3819999999999999,"15min":1.3640000000000001},"min":{"1min":1.1060000000000001,"5min":1.095,"15min":0.873},"max":{"1min":1.6479999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.115},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4339999999999999,"15min":1.399},"min":{"1min":1.1339999999999999,"5min":0.84699999999999998,"15min":0.79200000000000004},"max":{"1min":1.9450000000000001,"5min":1.978,"15min":2.0790000000000002},"last":1.4510000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3999999999999999,"15min":1.385},"min":{"1min":1.1559999999999999,"5min":0.98399999999999999,"15min":0.73999999999999999},"max":{"1min":1.6619999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5349999999999999,"15min":1.496},"min":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":0.82699999999999996},"max":{"1min":1.9450000000000001,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":1.1830000000000001}]},{"osd":7,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5089999999999999,"15min":1.478},"min":{"1min":1.208,"5min":0.95799999999999996,"15min":0.83899999999999997},"max":{"1min":1.8500000000000001,"5min":1.9019999999999999,"15min":2.048},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4990000000000001,"15min":1.46},"min":{"1min":1.111,"5min":1.0840000000000001,"15min":0.64500000000000002},"max":{"1min":1.73,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.5329999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575753,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59800,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312040,"statfs":{"total":96636764160,"available":96575528960,"internally_reserved":0,"allocated":2572288,"data_stored":1543344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.137,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.135},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.095,"15min":1.0740000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.722,"15min":1.722},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93200000000000005,"15min":0.92500000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.466,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1439999999999999,"15min":1.1479999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5880000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.876}]},{"osd":3,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5189999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1339999999999999,"15min":1.1279999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.444,"5min":1.6759999999999999,"15min":1.9219999999999999},"last":1.0489999999999999}]},{"osd":4,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.24,"15min":1.226},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.409},{"interface":"front","average":{"1min":1.413,"5min":1.4219999999999999,"15min":1.391},"min":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.032},"max":{"1min":1.8140000000000001,"5min":1.9730000000000001,"15min":2.1280000000000001},"last":1.198}]},{"osd":5,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.349},"min":{"1min":1.0620000000000001,"5min":0.98899999999999999,"15min":0.97899999999999998},"max":{"1min":1.49,"5min":1.9930000000000001,"15min":2.0449999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.3660000000000001,"15min":1.357},"min":{"1min":1.0349999999999999,"5min":1.0229999999999999,"15min":1.004},"max":{"1min":1.804,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.635}]},{"osd":6,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.454,"15min":1.4399999999999999},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.873,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.4139999999999999,"15min":1.4119999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.7250000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.7410000000000001}]},{"osd":7,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5289999999999999,"15min":1.518},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.9339999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.516,"15min":1.514},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8029999999999999,"5min":2.1749999999999998,"15min":2.282},"last":1.365}]}]},{"osd":0,"up_from":13,"seq":55834575755,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37088,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334752,"statfs":{"total":96636764160,"available":96598786048,"internally_reserved":0,"allocated":1990656,"data_stored":958837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.90600000000000003,"15min":0.90700000000000003},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.63400000000000001},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.766},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.119,"5min":1.087,"15min":1.075},"min":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.64500000000000002},"max":{"1min":1.5029999999999999,"5min":1.5980000000000001,"15min":1.7889999999999999},"last":1.345}]},{"osd":2,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.05,"15min":1.069},"min":{"1min":0.76800000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3120000000000001,"5min":1.4470000000000001,"15min":1.8540000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1160000000000001,"15min":1.1339999999999999},"min":{"1min":0.84299999999999997,"5min":0.76600000000000001,"15min":0.76100000000000001},"max":{"1min":1.4199999999999999,"5min":1.595,"15min":1.728},"last":1.157}]},{"osd":3,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.0880000000000001,"15min":1.0700000000000001},"min":{"1min":0.82499999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.421,"5min":1.5509999999999999,"15min":1.8100000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2729999999999999,"15min":1.2609999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.81599999999999995},"max":{"1min":1.498,"5min":1.798,"15min":1.9470000000000001},"last":1.107}]},{"osd":4,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.365,"15min":1.3560000000000001},"min":{"1min":1.048,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.0979999999999999},"last":1.488},{"interface":"front","average":{"1min":1.276,"5min":1.286,"15min":1.2889999999999999},"min":{"1min":0.98299999999999998,"5min":0.90200000000000002,"15min":0.88200000000000001},"max":{"1min":1.55,"5min":1.6579999999999999,"15min":2.0739999999999998},"last":1.0700000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.429,"15min":1.409},"min":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0109999999999999},"max":{"1min":1.946,"5min":2.2370000000000001,"15min":2.6110000000000002},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.4330000000000001,"15min":1.415},"min":{"1min":1.2490000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.831,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.649}]},{"osd":6,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.391,"15min":1.3779999999999999},"min":{"1min":1.083,"5min":0.745,"15min":0.745},"max":{"1min":1.6100000000000001,"5min":2.0939999999999999,"15min":2.1680000000000001},"last":1.224},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.5129999999999999,"15min":1.5069999999999999},"min":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.107},"max":{"1min":1.863,"5min":2.798,"15min":2.798},"last":1.419}]},{"osd":7,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.552,"15min":1.516},"min":{"1min":1.3320000000000001,"5min":1.085,"15min":1.085},"max":{"1min":1.923,"5min":2.2919999999999998,"15min":2.5329999999999999},"last":1.573},{"interface":"front","average":{"1min":1.486,"5min":1.492,"15min":1.4870000000000001},"min":{"1min":1.2110000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.909,"5min":2.097,"15min":3.0590000000000002},"last":1.2829999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T02:51:48.273 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T02:51:48.290 INFO:tasks.ceph.mds.a.smithi115.stderr:2023-12-01T02:51:48.288+0000 7fc87f2d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T02:51:48.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.506+0000 7fa606bdc640 1 Processor -- start 2023-12-01T02:51:48.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.507+0000 7fa606bdc640 1 -- start start 2023-12-01T02:51:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.508+0000 7fa606bdc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa6000ad2b0 0x7fa6000701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.508+0000 7fa606bdc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60015d600 con 0x7fa60015d220 2023-12-01T02:51:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.508+0000 7fa606bdc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60006c4b0 con 0x7fa6000ccc80 2023-12-01T02:51:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.508+0000 7fa606bdc640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60006cc10 con 0x7fa6000ad2b0 2023-12-01T02:51:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.508+0000 7fa604978640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa60015d220 0x7fa6000cc570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40276/0 (socket says 172.21.15.115:40276) 2023-12-01T02:51:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.508+0000 7fa604978640 1 -- 172.21.15.115:0/1055836909 learned_addr learned my addr 172.21.15.115:0/1055836909 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:48.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.508+0000 7fa605179640 1 --2- 172.21.15.115:0/1055836909 >> v2:172.21.15.115:3300/0 conn(0x7fa6000ad2b0 0x7fa6000701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.509+0000 7fa605179640 1 -- 172.21.15.115:0/1055836909 >> v1:172.21.15.142:6789/0 conn(0x7fa6000ccc80 legacy=0x7fa60006fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:48.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.509+0000 7fa605179640 1 -- 172.21.15.115:0/1055836909 >> v1:172.21.15.115:6789/0 conn(0x7fa60015d220 legacy=0x7fa6000cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.509+0000 7fa5ff7fe640 1 -- 172.21.15.115:0/1055836909 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777846532 0 0) 0x7fa5f4001110 con 0x7fa60015d220 2023-12-01T02:51:48.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.509+0000 7fa605179640 1 -- 172.21.15.115:0/1055836909 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60006f2f0 con 0x7fa6000ad2b0 2023-12-01T02:51:48.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.510+0000 7fa605179640 1 --2- 172.21.15.115:0/1055836909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6000ad2b0 0x7fa6000701c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009d90 tx=0x7fa5f802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883d65b6ccce218b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:48.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.510+0000 7fa5ff7fe640 1 -- 172.21.15.115:0/1055836909 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f802cbd0 con 0x7fa6000ad2b0 2023-12-01T02:51:48.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.510+0000 7fa5ff7fe640 1 -- 172.21.15.115:0/1055836909 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f802cd30 con 0x7fa6000ad2b0 2023-12-01T02:51:48.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.510+0000 7fa5ff7fe640 1 -- 172.21.15.115:0/1055836909 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f802cea0 con 0x7fa6000ad2b0 2023-12-01T02:51:48.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.511+0000 7fa606bdc640 1 -- 172.21.15.115:0/1055836909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6000ad2b0 msgr2=0x7fa6000701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.511+0000 7fa606bdc640 1 --2- 172.21.15.115:0/1055836909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6000ad2b0 0x7fa6000701c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009d90 tx=0x7fa5f802be20 comp rx=0 tx=0).stop 2023-12-01T02:51:48.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.511+0000 7fa606bdc640 1 -- 172.21.15.115:0/1055836909 shutdown_connections 2023-12-01T02:51:48.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.511+0000 7fa606bdc640 1 --2- 172.21.15.115:0/1055836909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6000ad2b0 0x7fa6000701c0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.511+0000 7fa606bdc640 1 -- 172.21.15.115:0/1055836909 >> 172.21.15.115:0/1055836909 conn(0x7fa6000ce880 msgr2=0x7fa6000cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:48.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.511+0000 7fa606bdc640 1 -- 172.21.15.115:0/1055836909 shutdown_connections 2023-12-01T02:51:48.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.511+0000 7fa606bdc640 1 -- 172.21.15.115:0/1055836909 wait complete. 2023-12-01T02:51:48.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.513+0000 7fa606bdc640 1 Processor -- start 2023-12-01T02:51:48.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.513+0000 7fa606bdc640 1 -- start start 2023-12-01T02:51:48.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.514+0000 7fa606bdc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000ad2b0 0x7fa600151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.514+0000 7fa606bdc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6000ccc80 0x7fa6000703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.514+0000 7fa606bdc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015d220 0x7fa600070930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.514+0000 7fa606bdc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001f3630 con 0x7fa60015d220 2023-12-01T02:51:48.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.514+0000 7fa606bdc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001f70f0 con 0x7fa6000ccc80 2023-12-01T02:51:48.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.514+0000 7fa606bdc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa600152200 con 0x7fa6000ad2b0 2023-12-01T02:51:48.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.514+0000 7fa605179640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015d220 0x7fa600070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.514+0000 7fa605179640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015d220 0x7fa600070930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45576/0 (socket says 172.21.15.115:45576) 2023-12-01T02:51:48.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.514+0000 7fa605179640 1 -- 172.21.15.115:0/497955217 learned_addr learned my addr 172.21.15.115:0/497955217 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa604978640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000ad2b0 0x7fa600151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa5fffff640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6000ccc80 0x7fa6000703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa605179640 1 -- 172.21.15.115:0/497955217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000ad2b0 msgr2=0x7fa600151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa605179640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000ad2b0 0x7fa600151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa605179640 1 -- 172.21.15.115:0/497955217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6000ccc80 msgr2=0x7fa6000703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa605179640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6000ccc80 0x7fa6000703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa605179640 1 -- 172.21.15.115:0/497955217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8009a70 con 0x7fa60015d220 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa5fffff640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6000ccc80 0x7fa6000703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa605179640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015d220 0x7fa600070930 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009d90 tx=0x7fa5f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa5fd7fa640 1 -- 172.21.15.115:0/497955217 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f802cea0 con 0x7fa60015d220 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa5fd7fa640 1 -- 172.21.15.115:0/497955217 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f803e630 con 0x7fa60015d220 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa5fd7fa640 1 -- 172.21.15.115:0/497955217 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f803e9e0 con 0x7fa60015d220 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.515+0000 7fa606bdc640 1 -- 172.21.15.115:0/497955217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6001f2d00 con 0x7fa60015d220 2023-12-01T02:51:48.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.516+0000 7fa606bdc640 1 -- 172.21.15.115:0/497955217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6001f32c0 con 0x7fa60015d220 2023-12-01T02:51:48.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.517+0000 7fa606bdc640 1 -- 172.21.15.115:0/497955217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6001f37a0 con 0x7fa60015d220 2023-12-01T02:51:48.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.517+0000 7fa5fd7fa640 1 -- 172.21.15.115:0/497955217 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa5f803b050 con 0x7fa60015d220 2023-12-01T02:51:48.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.518+0000 7fa5fd7fa640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5ec06a550 0x7fa5ec06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:48.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.518+0000 7fa5fd7fa640 1 -- 172.21.15.115:0/497955217 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fa5f80ab600 con 0x7fa60015d220 2023-12-01T02:51:48.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.519+0000 7fa604978640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5ec06a550 0x7fa5ec06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:48.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.519+0000 7fa604978640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5ec06a550 0x7fa5ec06ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4000f60 tx=0x7fa5f4007c00 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:48.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.525+0000 7fa5fd7fa640 1 -- 172.21.15.115:0/497955217 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa5f8076b30 con 0x7fa60015d220 2023-12-01T02:51:48.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.710+0000 7fa606bdc640 1 -- 172.21.15.115:0/497955217 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa60006c930 con 0x7fa5ec06a550 2023-12-01T02:51:48.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.723+0000 7fa5fd7fa640 1 -- 172.21.15.115:0/497955217 <== mgr.8769 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287124 (secure 0 0 0) 0x7fa5f4007270 con 0x7fa5ec06a550 2023-12-01T02:51:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:48.730 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 -- 172.21.15.115:0/497955217 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5ec06a550 msgr2=0x7fa5ec06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5ec06a550 0x7fa5ec06ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4000f60 tx=0x7fa5f4007c00 comp rx=0 tx=0).stop 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 -- 172.21.15.115:0/497955217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015d220 msgr2=0x7fa600070930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015d220 0x7fa600070930 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009d90 tx=0x7fa5f8004a40 comp rx=0 tx=0).stop 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 -- 172.21.15.115:0/497955217 shutdown_connections 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa5ec06a550 0x7fa5ec06ca10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa60015d220 0x7fa600070930 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6000ccc80 0x7fa6000703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 --2- 172.21.15.115:0/497955217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6000ad2b0 0x7fa600151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 -- 172.21.15.115:0/497955217 >> 172.21.15.115:0/497955217 conn(0x7fa6000ce880 msgr2=0x7fa60006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 -- 172.21.15.115:0/497955217 shutdown_connections 2023-12-01T02:51:48.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:48.730+0000 7fa606bdc640 1 -- 172.21.15.115:0/497955217 wait complete. 2023-12-01T02:51:48.751 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-12-01T02:51:48.649180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388432,"kb_used_data":17040,"kb_used_omap":0,"kb_used_meta":371328,"kb_avail":754586288,"statfs":{"total":773094113280,"available":772696358912,"internally_reserved":0,"allocated":17448960,"data_stored":8999324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":380239872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001841"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.852114+0000","last_change":"2023-12-01T02:51:43.860564+0000","last_active":"2023-12-01T02:51:44.852114+0000","last_peered":"2023-12-01T02:51:44.852114+0000","last_clean":"2023-12-01T02:51:44.852114+0000","last_became_active":"2023-12-01T02:51:43.859903+0000","last_became_peered":"2023-12-01T02:51:43.859903+0000","last_unstale":"2023-12-01T02:51:44.852114+0000","last_undegraded":"2023-12-01T02:51:44.852114+0000","last_fullsized":"2023-12-01T02:51:44.852114+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:57.656941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.852389+0000","last_change":"2023-12-01T02:51:43.868516+0000","last_active":"2023-12-01T02:51:44.852389+0000","last_peered":"2023-12-01T02:51:44.852389+0000","last_clean":"2023-12-01T02:51:44.852389+0000","last_became_active":"2023-12-01T02:51:43.866239+0000","last_became_peered":"2023-12-01T02:51:43.866239+0000","last_unstale":"2023-12-01T02:51:44.852389+0000","last_undegraded":"2023-12-01T02:51:44.852389+0000","last_fullsized":"2023-12-01T02:51:44.852389+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:05:10.657919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.846738+0000","last_change":"2023-12-01T02:51:43.857906+0000","last_active":"2023-12-01T02:51:44.846738+0000","last_peered":"2023-12-01T02:51:44.846738+0000","last_clean":"2023-12-01T02:51:44.846738+0000","last_became_active":"2023-12-01T02:51:43.857649+0000","last_became_peered":"2023-12-01T02:51:43.857649+0000","last_unstale":"2023-12-01T02:51:44.846738+0000","last_undegraded":"2023-12-01T02:51:44.846738+0000","last_fullsized":"2023-12-01T02:51:44.846738+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:02:39.647380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:46.145996+0000","last_change":"2023-12-01T02:51:43.867211+0000","last_active":"2023-12-01T02:51:46.145996+0000","last_peered":"2023-12-01T02:51:46.145996+0000","last_clean":"2023-12-01T02:51:46.145996+0000","last_became_active":"2023-12-01T02:51:43.862260+0000","last_became_peered":"2023-12-01T02:51:43.862260+0000","last_unstale":"2023-12-01T02:51:46.145996+0000","last_undegraded":"2023-12-01T02:51:46.145996+0000","last_fullsized":"2023-12-01T02:51:46.145996+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:40:14.146606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.209630+0000","last_change":"2023-12-01T02:51:43.862397+0000","last_active":"2023-12-01T02:51:47.209630+0000","last_peered":"2023-12-01T02:51:47.209630+0000","last_clean":"2023-12-01T02:51:47.209630+0000","last_became_active":"2023-12-01T02:51:43.860797+0000","last_became_peered":"2023-12-01T02:51:43.860797+0000","last_unstale":"2023-12-01T02:51:47.209630+0000","last_undegraded":"2023-12-01T02:51:47.209630+0000","last_fullsized":"2023-12-01T02:51:47.209630+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:11:33.974545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.339389+0000","last_change":"2023-12-01T02:51:43.855786+0000","last_active":"2023-12-01T02:51:45.339389+0000","last_peered":"2023-12-01T02:51:45.339389+0000","last_clean":"2023-12-01T02:51:45.339389+0000","last_became_active":"2023-12-01T02:51:43.855522+0000","last_became_peered":"2023-12-01T02:51:43.855522+0000","last_unstale":"2023-12-01T02:51:45.339389+0000","last_undegraded":"2023-12-01T02:51:45.339389+0000","last_fullsized":"2023-12-01T02:51:45.339389+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:07:04.965148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.209464+0000","last_change":"2023-12-01T02:51:43.866172+0000","last_active":"2023-12-01T02:51:47.209464+0000","last_peered":"2023-12-01T02:51:47.209464+0000","last_clean":"2023-12-01T02:51:47.209464+0000","last_became_active":"2023-12-01T02:51:43.865609+0000","last_became_peered":"2023-12-01T02:51:43.865609+0000","last_unstale":"2023-12-01T02:51:47.209464+0000","last_undegraded":"2023-12-01T02:51:47.209464+0000","last_fullsized":"2023-12-01T02:51:47.209464+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:09.113095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.209435+0000","last_change":"2023-12-01T02:51:43.867643+0000","last_active":"2023-12-01T02:51:47.209435+0000","last_peered":"2023-12-01T02:51:47.209435+0000","last_clean":"2023-12-01T02:51:47.209435+0000","last_became_active":"2023-12-01T02:51:43.867154+0000","last_became_peered":"2023-12-01T02:51:43.867154+0000","last_unstale":"2023-12-01T02:51:47.209435+0000","last_undegraded":"2023-12-01T02:51:47.209435+0000","last_fullsized":"2023-12-01T02:51:47.209435+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:02:26.943924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.846891+0000","last_change":"2023-12-01T02:51:43.862055+0000","last_active":"2023-12-01T02:51:44.846891+0000","last_peered":"2023-12-01T02:51:44.846891+0000","last_clean":"2023-12-01T02:51:44.846891+0000","last_became_active":"2023-12-01T02:51:43.861782+0000","last_became_peered":"2023-12-01T02:51:43.861782+0000","last_unstale":"2023-12-01T02:51:44.846891+0000","last_undegraded":"2023-12-01T02:51:44.846891+0000","last_fullsized":"2023-12-01T02:51:44.846891+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:00.098930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.852558+0000","last_change":"2023-12-01T02:51:43.860814+0000","last_active":"2023-12-01T02:51:44.852558+0000","last_peered":"2023-12-01T02:51:44.852558+0000","last_clean":"2023-12-01T02:51:44.852558+0000","last_became_active":"2023-12-01T02:51:43.860530+0000","last_became_peered":"2023-12-01T02:51:43.860530+0000","last_unstale":"2023-12-01T02:51:44.852558+0000","last_undegraded":"2023-12-01T02:51:44.852558+0000","last_fullsized":"2023-12-01T02:51:44.852558+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:10:56.056647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:46.146258+0000","last_change":"2023-12-01T02:51:43.872137+0000","last_active":"2023-12-01T02:51:46.146258+0000","last_peered":"2023-12-01T02:51:46.146258+0000","last_clean":"2023-12-01T02:51:46.146258+0000","last_became_active":"2023-12-01T02:51:43.871829+0000","last_became_peered":"2023-12-01T02:51:43.871829+0000","last_unstale":"2023-12-01T02:51:46.146258+0000","last_undegraded":"2023-12-01T02:51:46.146258+0000","last_fullsized":"2023-12-01T02:51:46.146258+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:18:28.979535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.339699+0000","last_change":"2023-12-01T02:51:43.854855+0000","last_active":"2023-12-01T02:51:45.339699+0000","last_peered":"2023-12-01T02:51:45.339699+0000","last_clean":"2023-12-01T02:51:45.339699+0000","last_became_active":"2023-12-01T02:51:43.854446+0000","last_became_peered":"2023-12-01T02:51:43.854446+0000","last_unstale":"2023-12-01T02:51:45.339699+0000","last_undegraded":"2023-12-01T02:51:45.339699+0000","last_fullsized":"2023-12-01T02:51:45.339699+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:06.242457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:46.145898+0000","last_change":"2023-12-01T02:51:43.865019+0000","last_active":"2023-12-01T02:51:46.145898+0000","last_peered":"2023-12-01T02:51:46.145898+0000","last_clean":"2023-12-01T02:51:46.145898+0000","last_became_active":"2023-12-01T02:51:43.864743+0000","last_became_peered":"2023-12-01T02:51:43.864743+0000","last_unstale":"2023-12-01T02:51:46.145898+0000","last_undegraded":"2023-12-01T02:51:46.145898+0000","last_fullsized":"2023-12-01T02:51:46.145898+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:44:34.455197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210136+0000","last_change":"2023-12-01T02:51:43.868268+0000","last_active":"2023-12-01T02:51:47.210136+0000","last_peered":"2023-12-01T02:51:47.210136+0000","last_clean":"2023-12-01T02:51:47.210136+0000","last_became_active":"2023-12-01T02:51:43.867943+0000","last_became_peered":"2023-12-01T02:51:43.867943+0000","last_unstale":"2023-12-01T02:51:47.210136+0000","last_undegraded":"2023-12-01T02:51:47.210136+0000","last_fullsized":"2023-12-01T02:51:47.210136+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:09.731884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.851059+0000","last_change":"2023-12-01T02:51:43.864479+0000","last_active":"2023-12-01T02:51:44.851059+0000","last_peered":"2023-12-01T02:51:44.851059+0000","last_clean":"2023-12-01T02:51:44.851059+0000","last_became_active":"2023-12-01T02:51:43.862676+0000","last_became_peered":"2023-12-01T02:51:43.862676+0000","last_unstale":"2023-12-01T02:51:44.851059+0000","last_undegraded":"2023-12-01T02:51:44.851059+0000","last_fullsized":"2023-12-01T02:51:44.851059+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:05:26.512178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210275+0000","last_change":"2023-12-01T02:51:43.869562+0000","last_active":"2023-12-01T02:51:47.210275+0000","last_peered":"2023-12-01T02:51:47.210275+0000","last_clean":"2023-12-01T02:51:47.210275+0000","last_became_active":"2023-12-01T02:51:43.867107+0000","last_became_peered":"2023-12-01T02:51:43.867107+0000","last_unstale":"2023-12-01T02:51:47.210275+0000","last_undegraded":"2023-12-01T02:51:47.210275+0000","last_fullsized":"2023-12-01T02:51:47.210275+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:05:10.657919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.847188+0000","last_change":"2023-12-01T02:51:43.875792+0000","last_active":"2023-12-01T02:51:44.847188+0000","last_peered":"2023-12-01T02:51:44.847188+0000","last_clean":"2023-12-01T02:51:44.847188+0000","last_became_active":"2023-12-01T02:51:43.875504+0000","last_became_peered":"2023-12-01T02:51:43.875504+0000","last_unstale":"2023-12-01T02:51:44.847188+0000","last_undegraded":"2023-12-01T02:51:44.847188+0000","last_fullsized":"2023-12-01T02:51:44.847188+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:12.727366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.852701+0000","last_change":"2023-12-01T02:51:43.866997+0000","last_active":"2023-12-01T02:51:44.852701+0000","last_peered":"2023-12-01T02:51:44.852701+0000","last_clean":"2023-12-01T02:51:44.852701+0000","last_became_active":"2023-12-01T02:51:43.866431+0000","last_became_peered":"2023-12-01T02:51:43.866431+0000","last_unstale":"2023-12-01T02:51:44.852701+0000","last_undegraded":"2023-12-01T02:51:44.852701+0000","last_fullsized":"2023-12-01T02:51:44.852701+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:57:42.020340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.852256+0000","last_change":"2023-12-01T02:51:43.872419+0000","last_active":"2023-12-01T02:51:44.852256+0000","last_peered":"2023-12-01T02:51:44.852256+0000","last_clean":"2023-12-01T02:51:44.852256+0000","last_became_active":"2023-12-01T02:51:43.868251+0000","last_became_peered":"2023-12-01T02:51:43.868251+0000","last_unstale":"2023-12-01T02:51:44.852256+0000","last_undegraded":"2023-12-01T02:51:44.852256+0000","last_fullsized":"2023-12-01T02:51:44.852256+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:26:37.832484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.852160+0000","last_change":"2023-12-01T02:51:43.863467+0000","last_active":"2023-12-01T02:51:44.852160+0000","last_peered":"2023-12-01T02:51:44.852160+0000","last_clean":"2023-12-01T02:51:44.852160+0000","last_became_active":"2023-12-01T02:51:43.863159+0000","last_became_peered":"2023-12-01T02:51:43.863159+0000","last_unstale":"2023-12-01T02:51:44.852160+0000","last_undegraded":"2023-12-01T02:51:44.852160+0000","last_fullsized":"2023-12-01T02:51:44.852160+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:23:24.557718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.339664+0000","last_change":"2023-12-01T02:51:43.865974+0000","last_active":"2023-12-01T02:51:45.339664+0000","last_peered":"2023-12-01T02:51:45.339664+0000","last_clean":"2023-12-01T02:51:45.339664+0000","last_became_active":"2023-12-01T02:51:43.864984+0000","last_became_peered":"2023-12-01T02:51:43.864984+0000","last_unstale":"2023-12-01T02:51:45.339664+0000","last_undegraded":"2023-12-01T02:51:45.339664+0000","last_fullsized":"2023-12-01T02:51:45.339664+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:21.573823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.851233+0000","last_change":"2023-12-01T02:51:43.859852+0000","last_active":"2023-12-01T02:51:44.851233+0000","last_peered":"2023-12-01T02:51:44.851233+0000","last_clean":"2023-12-01T02:51:44.851233+0000","last_became_active":"2023-12-01T02:51:43.859563+0000","last_became_peered":"2023-12-01T02:51:43.859563+0000","last_unstale":"2023-12-01T02:51:44.851233+0000","last_undegraded":"2023-12-01T02:51:44.851233+0000","last_fullsized":"2023-12-01T02:51:44.851233+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:02:27.276957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:46.783468+0000","last_change":"2023-12-01T02:51:43.855429+0000","last_active":"2023-12-01T02:51:46.783468+0000","last_peered":"2023-12-01T02:51:46.783468+0000","last_clean":"2023-12-01T02:51:46.783468+0000","last_became_active":"2023-12-01T02:51:43.855154+0000","last_became_peered":"2023-12-01T02:51:43.855154+0000","last_unstale":"2023-12-01T02:51:46.783468+0000","last_undegraded":"2023-12-01T02:51:46.783468+0000","last_fullsized":"2023-12-01T02:51:46.783468+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:35:59.266065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.850262+0000","last_change":"2023-12-01T02:51:43.876433+0000","last_active":"2023-12-01T02:51:44.850262+0000","last_peered":"2023-12-01T02:51:44.850262+0000","last_clean":"2023-12-01T02:51:44.850262+0000","last_became_active":"2023-12-01T02:51:43.876135+0000","last_became_peered":"2023-12-01T02:51:43.876135+0000","last_unstale":"2023-12-01T02:51:44.850262+0000","last_undegraded":"2023-12-01T02:51:44.850262+0000","last_fullsized":"2023-12-01T02:51:44.850262+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:48:49.707358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:46.146812+0000","last_change":"2023-12-01T02:51:43.874559+0000","last_active":"2023-12-01T02:51:46.146812+0000","last_peered":"2023-12-01T02:51:46.146812+0000","last_clean":"2023-12-01T02:51:46.146812+0000","last_became_active":"2023-12-01T02:51:43.871910+0000","last_became_peered":"2023-12-01T02:51:43.871910+0000","last_unstale":"2023-12-01T02:51:46.146812+0000","last_undegraded":"2023-12-01T02:51:46.146812+0000","last_fullsized":"2023-12-01T02:51:46.146812+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:23.072312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.852657+0000","last_change":"2023-12-01T02:51:43.868037+0000","last_active":"2023-12-01T02:51:44.852657+0000","last_peered":"2023-12-01T02:51:44.852657+0000","last_clean":"2023-12-01T02:51:44.852657+0000","last_became_active":"2023-12-01T02:51:43.867512+0000","last_became_peered":"2023-12-01T02:51:43.867512+0000","last_unstale":"2023-12-01T02:51:44.852657+0000","last_undegraded":"2023-12-01T02:51:44.852657+0000","last_fullsized":"2023-12-01T02:51:44.852657+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:02.157362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210978+0000","last_change":"2023-12-01T02:51:43.869338+0000","last_active":"2023-12-01T02:51:47.210978+0000","last_peered":"2023-12-01T02:51:47.210978+0000","last_clean":"2023-12-01T02:51:47.210978+0000","last_became_active":"2023-12-01T02:51:43.869060+0000","last_became_peered":"2023-12-01T02:51:43.869060+0000","last_unstale":"2023-12-01T02:51:47.210978+0000","last_undegraded":"2023-12-01T02:51:47.210978+0000","last_fullsized":"2023-12-01T02:51:47.210978+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:45:55.952941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:46.783798+0000","last_change":"2023-12-01T02:51:43.870274+0000","last_active":"2023-12-01T02:51:46.783798+0000","last_peered":"2023-12-01T02:51:46.783798+0000","last_clean":"2023-12-01T02:51:46.783798+0000","last_became_active":"2023-12-01T02:51:43.869977+0000","last_became_peered":"2023-12-01T02:51:43.869977+0000","last_unstale":"2023-12-01T02:51:46.783798+0000","last_undegraded":"2023-12-01T02:51:46.783798+0000","last_fullsized":"2023-12-01T02:51:46.783798+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:54.860697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.851570+0000","last_change":"2023-12-01T02:51:43.875430+0000","last_active":"2023-12-01T02:51:44.851570+0000","last_peered":"2023-12-01T02:51:44.851570+0000","last_clean":"2023-12-01T02:51:44.851570+0000","last_became_active":"2023-12-01T02:51:43.874797+0000","last_became_peered":"2023-12-01T02:51:43.874797+0000","last_unstale":"2023-12-01T02:51:44.851570+0000","last_undegraded":"2023-12-01T02:51:44.851570+0000","last_fullsized":"2023-12-01T02:51:44.851570+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:06:26.434676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.851532+0000","last_change":"2023-12-01T02:51:43.852540+0000","last_active":"2023-12-01T02:51:44.851532+0000","last_peered":"2023-12-01T02:51:44.851532+0000","last_clean":"2023-12-01T02:51:44.851532+0000","last_became_active":"2023-12-01T02:51:43.851831+0000","last_became_peered":"2023-12-01T02:51:43.851831+0000","last_unstale":"2023-12-01T02:51:44.851532+0000","last_undegraded":"2023-12-01T02:51:44.851532+0000","last_fullsized":"2023-12-01T02:51:44.851532+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:45:55.952941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:46.784033+0000","last_change":"2023-12-01T02:51:43.862777+0000","last_active":"2023-12-01T02:51:46.784033+0000","last_peered":"2023-12-01T02:51:46.784033+0000","last_clean":"2023-12-01T02:51:46.784033+0000","last_became_active":"2023-12-01T02:51:43.862486+0000","last_became_peered":"2023-12-01T02:51:43.862486+0000","last_unstale":"2023-12-01T02:51:46.784033+0000","last_undegraded":"2023-12-01T02:51:46.784033+0000","last_fullsized":"2023-12-01T02:51:46.784033+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:30:45.710751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.847536+0000","last_change":"2023-12-01T02:51:43.860493+0000","last_active":"2023-12-01T02:51:44.847536+0000","last_peered":"2023-12-01T02:51:44.847536+0000","last_clean":"2023-12-01T02:51:44.847536+0000","last_became_active":"2023-12-01T02:51:43.859632+0000","last_became_peered":"2023-12-01T02:51:43.859632+0000","last_unstale":"2023-12-01T02:51:44.847536+0000","last_undegraded":"2023-12-01T02:51:44.847536+0000","last_fullsized":"2023-12-01T02:51:44.847536+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:34.155339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210873+0000","last_change":"2023-12-01T02:51:43.870047+0000","last_active":"2023-12-01T02:51:47.210873+0000","last_peered":"2023-12-01T02:51:47.210873+0000","last_clean":"2023-12-01T02:51:47.210873+0000","last_became_active":"2023-12-01T02:51:43.869768+0000","last_became_peered":"2023-12-01T02:51:43.869768+0000","last_unstale":"2023-12-01T02:51:47.210873+0000","last_undegraded":"2023-12-01T02:51:47.210873+0000","last_fullsized":"2023-12-01T02:51:47.210873+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:02.157362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"162'4","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.878196+0000","last_change":"2023-12-01T02:51:43.373357+0000","last_active":"2023-12-01T02:51:43.878196+0000","last_peered":"2023-12-01T02:51:43.878196+0000","last_clean":"2023-12-01T02:51:43.878196+0000","last_became_active":"2023-12-01T02:51:43.372980+0000","last_became_peered":"2023-12-01T02:51:43.372980+0000","last_unstale":"2023-12-01T02:51:43.878196+0000","last_undegraded":"2023-12-01T02:51:43.878196+0000","last_fullsized":"2023-12-01T02:51:43.878196+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:28:31.216397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.847106+0000","last_change":"2023-12-01T02:51:43.876701+0000","last_active":"2023-12-01T02:51:44.847106+0000","last_peered":"2023-12-01T02:51:44.847106+0000","last_clean":"2023-12-01T02:51:44.847106+0000","last_became_active":"2023-12-01T02:51:43.876392+0000","last_became_peered":"2023-12-01T02:51:43.876392+0000","last_unstale":"2023-12-01T02:51:44.847106+0000","last_undegraded":"2023-12-01T02:51:44.847106+0000","last_fullsized":"2023-12-01T02:51:44.847106+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:59:29.887954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"162'1","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.853427+0000","last_change":"2023-12-01T02:51:43.363963+0000","last_active":"2023-12-01T02:51:43.853427+0000","last_peered":"2023-12-01T02:51:43.853427+0000","last_clean":"2023-12-01T02:51:43.853427+0000","last_became_active":"2023-12-01T02:51:43.363643+0000","last_became_peered":"2023-12-01T02:51:43.363643+0000","last_unstale":"2023-12-01T02:51:43.853427+0000","last_undegraded":"2023-12-01T02:51:43.853427+0000","last_fullsized":"2023-12-01T02:51:43.853427+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:17:14.987724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.846761+0000","last_change":"2023-12-01T02:51:43.845083+0000","last_active":"2023-12-01T02:51:44.846761+0000","last_peered":"2023-12-01T02:51:44.846761+0000","last_clean":"2023-12-01T02:51:44.846761+0000","last_became_active":"2023-12-01T02:51:43.844301+0000","last_became_peered":"2023-12-01T02:51:43.844301+0000","last_unstale":"2023-12-01T02:51:44.846761+0000","last_undegraded":"2023-12-01T02:51:44.846761+0000","last_fullsized":"2023-12-01T02:51:44.846761+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:51.082105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"162'2","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.877924+0000","last_change":"2023-12-01T02:51:43.373662+0000","last_active":"2023-12-01T02:51:43.877924+0000","last_peered":"2023-12-01T02:51:43.877924+0000","last_clean":"2023-12-01T02:51:43.877924+0000","last_became_active":"2023-12-01T02:51:43.372886+0000","last_became_peered":"2023-12-01T02:51:43.372886+0000","last_unstale":"2023-12-01T02:51:43.877924+0000","last_undegraded":"2023-12-01T02:51:43.877924+0000","last_fullsized":"2023-12-01T02:51:43.877924+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:56.639523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.339860+0000","last_change":"2023-12-01T02:51:43.868946+0000","last_active":"2023-12-01T02:51:45.339860+0000","last_peered":"2023-12-01T02:51:45.339860+0000","last_clean":"2023-12-01T02:51:45.339860+0000","last_became_active":"2023-12-01T02:51:43.867940+0000","last_became_peered":"2023-12-01T02:51:43.867940+0000","last_unstale":"2023-12-01T02:51:45.339860+0000","last_undegraded":"2023-12-01T02:51:45.339860+0000","last_fullsized":"2023-12-01T02:51:45.339860+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:05:26.512178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"162'2","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.873658+0000","last_change":"2023-12-01T02:51:43.373636+0000","last_active":"2023-12-01T02:51:43.873658+0000","last_peered":"2023-12-01T02:51:43.873658+0000","last_clean":"2023-12-01T02:51:43.873658+0000","last_became_active":"2023-12-01T02:51:43.372141+0000","last_became_peered":"2023-12-01T02:51:43.372141+0000","last_unstale":"2023-12-01T02:51:43.873658+0000","last_undegraded":"2023-12-01T02:51:43.873658+0000","last_fullsized":"2023-12-01T02:51:43.873658+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:23:23.540300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.851518+0000","last_change":"2023-12-01T02:51:43.863391+0000","last_active":"2023-12-01T02:51:44.851518+0000","last_peered":"2023-12-01T02:51:44.851518+0000","last_clean":"2023-12-01T02:51:44.851518+0000","last_became_active":"2023-12-01T02:51:43.861335+0000","last_became_peered":"2023-12-01T02:51:43.861335+0000","last_unstale":"2023-12-01T02:51:44.851518+0000","last_undegraded":"2023-12-01T02:51:44.851518+0000","last_fullsized":"2023-12-01T02:51:44.851518+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:31.288413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"162'1","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.871141+0000","last_change":"2023-12-01T02:51:43.367265+0000","last_active":"2023-12-01T02:51:43.871141+0000","last_peered":"2023-12-01T02:51:43.871141+0000","last_clean":"2023-12-01T02:51:43.871141+0000","last_became_active":"2023-12-01T02:51:43.366502+0000","last_became_peered":"2023-12-01T02:51:43.366502+0000","last_unstale":"2023-12-01T02:51:43.871141+0000","last_undegraded":"2023-12-01T02:51:43.871141+0000","last_fullsized":"2023-12-01T02:51:43.871141+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:30.270995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.339926+0000","last_change":"2023-12-01T02:51:43.872194+0000","last_active":"2023-12-01T02:51:45.339926+0000","last_peered":"2023-12-01T02:51:45.339926+0000","last_clean":"2023-12-01T02:51:45.339926+0000","last_became_active":"2023-12-01T02:51:43.871339+0000","last_became_peered":"2023-12-01T02:51:43.871339+0000","last_unstale":"2023-12-01T02:51:45.339926+0000","last_undegraded":"2023-12-01T02:51:45.339926+0000","last_fullsized":"2023-12-01T02:51:45.339926+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:31.416300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"162'1","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.872954+0000","last_change":"2023-12-01T02:51:43.370477+0000","last_active":"2023-12-01T02:51:43.872954+0000","last_peered":"2023-12-01T02:51:43.872954+0000","last_clean":"2023-12-01T02:51:43.872954+0000","last_became_active":"2023-12-01T02:51:43.369810+0000","last_became_peered":"2023-12-01T02:51:43.369810+0000","last_unstale":"2023-12-01T02:51:43.872954+0000","last_undegraded":"2023-12-01T02:51:43.872954+0000","last_fullsized":"2023-12-01T02:51:43.872954+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:46:16.844083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.339183+0000","last_change":"2023-12-01T02:51:43.848333+0000","last_active":"2023-12-01T02:51:45.339183+0000","last_peered":"2023-12-01T02:51:45.339183+0000","last_clean":"2023-12-01T02:51:45.339183+0000","last_became_active":"2023-12-01T02:51:43.848009+0000","last_became_peered":"2023-12-01T02:51:43.848009+0000","last_unstale":"2023-12-01T02:51:45.339183+0000","last_undegraded":"2023-12-01T02:51:45.339183+0000","last_fullsized":"2023-12-01T02:51:45.339183+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:33:05.648898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.836040+0000","last_change":"2023-12-01T02:51:43.369672+0000","last_active":"2023-12-01T02:51:43.836040+0000","last_peered":"2023-12-01T02:51:43.836040+0000","last_clean":"2023-12-01T02:51:43.836040+0000","last_became_active":"2023-12-01T02:51:43.369181+0000","last_became_peered":"2023-12-01T02:51:43.369181+0000","last_unstale":"2023-12-01T02:51:43.836040+0000","last_undegraded":"2023-12-01T02:51:43.836040+0000","last_fullsized":"2023-12-01T02:51:43.836040+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:42:36.272402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.850494+0000","last_change":"2023-12-01T02:51:43.856193+0000","last_active":"2023-12-01T02:51:44.850494+0000","last_peered":"2023-12-01T02:51:44.850494+0000","last_clean":"2023-12-01T02:51:44.850494+0000","last_became_active":"2023-12-01T02:51:43.855876+0000","last_became_peered":"2023-12-01T02:51:43.855876+0000","last_unstale":"2023-12-01T02:51:44.850494+0000","last_undegraded":"2023-12-01T02:51:44.850494+0000","last_fullsized":"2023-12-01T02:51:44.850494+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:50:29.480200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.848804+0000","last_change":"2023-12-01T02:51:43.358078+0000","last_active":"2023-12-01T02:51:43.848804+0000","last_peered":"2023-12-01T02:51:43.848804+0000","last_clean":"2023-12-01T02:51:43.848804+0000","last_became_active":"2023-12-01T02:51:43.357368+0000","last_became_peered":"2023-12-01T02:51:43.357368+0000","last_unstale":"2023-12-01T02:51:43.848804+0000","last_undegraded":"2023-12-01T02:51:43.848804+0000","last_fullsized":"2023-12-01T02:51:43.848804+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:00:11.086798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.852770+0000","last_change":"2023-12-01T02:51:43.859343+0000","last_active":"2023-12-01T02:51:44.852770+0000","last_peered":"2023-12-01T02:51:44.852770+0000","last_clean":"2023-12-01T02:51:44.852770+0000","last_became_active":"2023-12-01T02:51:43.855954+0000","last_became_peered":"2023-12-01T02:51:43.855954+0000","last_unstale":"2023-12-01T02:51:44.852770+0000","last_undegraded":"2023-12-01T02:51:44.852770+0000","last_fullsized":"2023-12-01T02:51:44.852770+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:11:09.693053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.848194+0000","last_change":"2023-12-01T02:51:43.380323+0000","last_active":"2023-12-01T02:51:43.848194+0000","last_peered":"2023-12-01T02:51:43.848194+0000","last_clean":"2023-12-01T02:51:43.848194+0000","last_became_active":"2023-12-01T02:51:43.380019+0000","last_became_peered":"2023-12-01T02:51:43.380019+0000","last_unstale":"2023-12-01T02:51:43.848194+0000","last_undegraded":"2023-12-01T02:51:43.848194+0000","last_fullsized":"2023-12-01T02:51:43.848194+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:11:28.528185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210080+0000","last_change":"2023-12-01T02:51:43.867183+0000","last_active":"2023-12-01T02:51:47.210080+0000","last_peered":"2023-12-01T02:51:47.210080+0000","last_clean":"2023-12-01T02:51:47.210080+0000","last_became_active":"2023-12-01T02:51:43.866871+0000","last_became_peered":"2023-12-01T02:51:43.866871+0000","last_unstale":"2023-12-01T02:51:47.210080+0000","last_undegraded":"2023-12-01T02:51:47.210080+0000","last_fullsized":"2023-12-01T02:51:47.210080+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:06.242457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.857167+0000","last_change":"2023-12-01T02:51:43.370190+0000","last_active":"2023-12-01T02:51:43.857167+0000","last_peered":"2023-12-01T02:51:43.857167+0000","last_clean":"2023-12-01T02:51:43.857167+0000","last_became_active":"2023-12-01T02:51:43.367830+0000","last_became_peered":"2023-12-01T02:51:43.367830+0000","last_unstale":"2023-12-01T02:51:43.857167+0000","last_undegraded":"2023-12-01T02:51:43.857167+0000","last_fullsized":"2023-12-01T02:51:43.857167+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:41.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.339759+0000","last_change":"2023-12-01T02:51:43.861993+0000","last_active":"2023-12-01T02:51:45.339759+0000","last_peered":"2023-12-01T02:51:45.339759+0000","last_clean":"2023-12-01T02:51:45.339759+0000","last_became_active":"2023-12-01T02:51:43.861689+0000","last_became_peered":"2023-12-01T02:51:43.861689+0000","last_unstale":"2023-12-01T02:51:45.339759+0000","last_undegraded":"2023-12-01T02:51:45.339759+0000","last_fullsized":"2023-12-01T02:51:45.339759+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:38.480096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"162'3","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.876275+0000","last_change":"2023-12-01T02:51:43.371788+0000","last_active":"2023-12-01T02:51:43.876275+0000","last_peered":"2023-12-01T02:51:43.876275+0000","last_clean":"2023-12-01T02:51:43.876275+0000","last_became_active":"2023-12-01T02:51:43.371503+0000","last_became_peered":"2023-12-01T02:51:43.371503+0000","last_unstale":"2023-12-01T02:51:43.876275+0000","last_undegraded":"2023-12-01T02:51:43.876275+0000","last_fullsized":"2023-12-01T02:51:43.876275+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:12:22.519981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:46.783553+0000","last_change":"2023-12-01T02:51:43.857037+0000","last_active":"2023-12-01T02:51:46.783553+0000","last_peered":"2023-12-01T02:51:46.783553+0000","last_clean":"2023-12-01T02:51:46.783553+0000","last_became_active":"2023-12-01T02:51:43.856451+0000","last_became_peered":"2023-12-01T02:51:43.856451+0000","last_unstale":"2023-12-01T02:51:46.783553+0000","last_undegraded":"2023-12-01T02:51:46.783553+0000","last_fullsized":"2023-12-01T02:51:46.783553+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:40.453888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"162'6","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.875731+0000","last_change":"2023-12-01T02:51:43.375698+0000","last_active":"2023-12-01T02:51:43.875731+0000","last_peered":"2023-12-01T02:51:43.875731+0000","last_clean":"2023-12-01T02:51:43.875731+0000","last_became_active":"2023-12-01T02:51:43.371796+0000","last_became_peered":"2023-12-01T02:51:43.371796+0000","last_unstale":"2023-12-01T02:51:43.875731+0000","last_undegraded":"2023-12-01T02:51:43.875731+0000","last_fullsized":"2023-12-01T02:51:43.875731+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:03.506649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210552+0000","last_change":"2023-12-01T02:51:43.875883+0000","last_active":"2023-12-01T02:51:47.210552+0000","last_peered":"2023-12-01T02:51:47.210552+0000","last_clean":"2023-12-01T02:51:47.210552+0000","last_became_active":"2023-12-01T02:51:43.875368+0000","last_became_peered":"2023-12-01T02:51:43.875368+0000","last_unstale":"2023-12-01T02:51:47.210552+0000","last_undegraded":"2023-12-01T02:51:47.210552+0000","last_fullsized":"2023-12-01T02:51:47.210552+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:38.367687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.857276+0000","last_change":"2023-12-01T02:51:43.376222+0000","last_active":"2023-12-01T02:51:43.857276+0000","last_peered":"2023-12-01T02:51:43.857276+0000","last_clean":"2023-12-01T02:51:43.857276+0000","last_became_active":"2023-12-01T02:51:43.375870+0000","last_became_peered":"2023-12-01T02:51:43.375870+0000","last_unstale":"2023-12-01T02:51:43.857276+0000","last_undegraded":"2023-12-01T02:51:43.857276+0000","last_fullsized":"2023-12-01T02:51:43.857276+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:02:25.926506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210179+0000","last_change":"2023-12-01T02:51:43.866924+0000","last_active":"2023-12-01T02:51:47.210179+0000","last_peered":"2023-12-01T02:51:47.210179+0000","last_clean":"2023-12-01T02:51:47.210179+0000","last_became_active":"2023-12-01T02:51:43.866633+0000","last_became_peered":"2023-12-01T02:51:43.866633+0000","last_unstale":"2023-12-01T02:51:47.210179+0000","last_undegraded":"2023-12-01T02:51:47.210179+0000","last_fullsized":"2023-12-01T02:51:47.210179+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:26:37.832484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.864093+0000","last_change":"2023-12-01T02:51:43.379947+0000","last_active":"2023-12-01T02:51:43.864093+0000","last_peered":"2023-12-01T02:51:43.864093+0000","last_clean":"2023-12-01T02:51:43.864093+0000","last_became_active":"2023-12-01T02:51:43.379675+0000","last_became_peered":"2023-12-01T02:51:43.379675+0000","last_unstale":"2023-12-01T02:51:43.864093+0000","last_undegraded":"2023-12-01T02:51:43.864093+0000","last_fullsized":"2023-12-01T02:51:43.864093+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:42:36.272402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.339372+0000","last_change":"2023-12-01T02:51:43.849185+0000","last_active":"2023-12-01T02:51:45.339372+0000","last_peered":"2023-12-01T02:51:45.339372+0000","last_clean":"2023-12-01T02:51:45.339372+0000","last_became_active":"2023-12-01T02:51:43.848095+0000","last_became_peered":"2023-12-01T02:51:43.848095+0000","last_unstale":"2023-12-01T02:51:45.339372+0000","last_undegraded":"2023-12-01T02:51:45.339372+0000","last_fullsized":"2023-12-01T02:51:45.339372+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:11:33.974545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"162'8","reported_seq":19,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.873830+0000","last_change":"2023-12-01T02:51:43.375308+0000","last_active":"2023-12-01T02:51:43.873830+0000","last_peered":"2023-12-01T02:51:43.873830+0000","last_clean":"2023-12-01T02:51:43.873830+0000","last_became_active":"2023-12-01T02:51:43.375027+0000","last_became_peered":"2023-12-01T02:51:43.375027+0000","last_unstale":"2023-12-01T02:51:43.873830+0000","last_undegraded":"2023-12-01T02:51:43.873830+0000","last_fullsized":"2023-12-01T02:51:43.873830+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:58.168046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210076+0000","last_change":"2023-12-01T02:51:43.871364+0000","last_active":"2023-12-01T02:51:47.210076+0000","last_peered":"2023-12-01T02:51:47.210076+0000","last_clean":"2023-12-01T02:51:47.210076+0000","last_became_active":"2023-12-01T02:51:43.870653+0000","last_became_peered":"2023-12-01T02:51:43.870653+0000","last_unstale":"2023-12-01T02:51:47.210076+0000","last_undegraded":"2023-12-01T02:51:47.210076+0000","last_fullsized":"2023-12-01T02:51:47.210076+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:38.480096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"162'1","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.864038+0000","last_change":"2023-12-01T02:51:43.377762+0000","last_active":"2023-12-01T02:51:43.864038+0000","last_peered":"2023-12-01T02:51:43.864038+0000","last_clean":"2023-12-01T02:51:43.864038+0000","last_became_active":"2023-12-01T02:51:43.376893+0000","last_became_peered":"2023-12-01T02:51:43.376893+0000","last_unstale":"2023-12-01T02:51:43.864038+0000","last_undegraded":"2023-12-01T02:51:43.864038+0000","last_fullsized":"2023-12-01T02:51:43.864038+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:06.631985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210181+0000","last_change":"2023-12-01T02:51:43.875559+0000","last_active":"2023-12-01T02:51:47.210181+0000","last_peered":"2023-12-01T02:51:47.210181+0000","last_clean":"2023-12-01T02:51:47.210181+0000","last_became_active":"2023-12-01T02:51:43.875037+0000","last_became_peered":"2023-12-01T02:51:43.875037+0000","last_unstale":"2023-12-01T02:51:47.210181+0000","last_undegraded":"2023-12-01T02:51:47.210181+0000","last_fullsized":"2023-12-01T02:51:47.210181+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:33:05.648898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"162'2","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.876977+0000","last_change":"2023-12-01T02:51:43.372446+0000","last_active":"2023-12-01T02:51:43.876977+0000","last_peered":"2023-12-01T02:51:43.876977+0000","last_clean":"2023-12-01T02:51:43.876977+0000","last_became_active":"2023-12-01T02:51:43.372146+0000","last_became_peered":"2023-12-01T02:51:43.372146+0000","last_unstale":"2023-12-01T02:51:43.876977+0000","last_undegraded":"2023-12-01T02:51:43.876977+0000","last_fullsized":"2023-12-01T02:51:43.876977+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:44.731298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.853177+0000","last_change":"2023-12-01T02:51:43.866656+0000","last_active":"2023-12-01T02:51:44.853177+0000","last_peered":"2023-12-01T02:51:44.853177+0000","last_clean":"2023-12-01T02:51:44.853177+0000","last_became_active":"2023-12-01T02:51:43.865215+0000","last_became_peered":"2023-12-01T02:51:43.865215+0000","last_unstale":"2023-12-01T02:51:44.853177+0000","last_undegraded":"2023-12-01T02:51:44.853177+0000","last_fullsized":"2023-12-01T02:51:44.853177+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:09.731884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"162'1","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.876599+0000","last_change":"2023-12-01T02:51:43.372595+0000","last_active":"2023-12-01T02:51:43.876599+0000","last_peered":"2023-12-01T02:51:43.876599+0000","last_clean":"2023-12-01T02:51:43.876599+0000","last_became_active":"2023-12-01T02:51:43.372309+0000","last_became_peered":"2023-12-01T02:51:43.372309+0000","last_unstale":"2023-12-01T02:51:43.876599+0000","last_undegraded":"2023-12-01T02:51:43.876599+0000","last_fullsized":"2023-12-01T02:51:43.876599+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:19.891780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.851463+0000","last_change":"2023-12-01T02:51:43.876975+0000","last_active":"2023-12-01T02:51:44.851463+0000","last_peered":"2023-12-01T02:51:44.851463+0000","last_clean":"2023-12-01T02:51:44.851463+0000","last_became_active":"2023-12-01T02:51:43.876680+0000","last_became_peered":"2023-12-01T02:51:43.876680+0000","last_unstale":"2023-12-01T02:51:44.851463+0000","last_undegraded":"2023-12-01T02:51:44.851463+0000","last_fullsized":"2023-12-01T02:51:44.851463+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:00:27.738861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.855768+0000","last_change":"2023-12-01T02:51:43.374134+0000","last_active":"2023-12-01T02:51:43.855768+0000","last_peered":"2023-12-01T02:51:43.855768+0000","last_clean":"2023-12-01T02:51:43.855768+0000","last_became_active":"2023-12-01T02:51:43.373881+0000","last_became_peered":"2023-12-01T02:51:43.373881+0000","last_unstale":"2023-12-01T02:51:43.855768+0000","last_undegraded":"2023-12-01T02:51:43.855768+0000","last_fullsized":"2023-12-01T02:51:43.855768+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:56:15.834365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.851793+0000","last_change":"2023-12-01T02:51:43.853982+0000","last_active":"2023-12-01T02:51:44.851793+0000","last_peered":"2023-12-01T02:51:44.851793+0000","last_clean":"2023-12-01T02:51:44.851793+0000","last_became_active":"2023-12-01T02:51:43.853515+0000","last_became_peered":"2023-12-01T02:51:43.853515+0000","last_unstale":"2023-12-01T02:51:44.851793+0000","last_undegraded":"2023-12-01T02:51:44.851793+0000","last_fullsized":"2023-12-01T02:51:44.851793+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:54.796980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"162'2","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.865315+0000","last_change":"2023-12-01T02:51:43.372625+0000","last_active":"2023-12-01T02:51:43.865315+0000","last_peered":"2023-12-01T02:51:43.865315+0000","last_clean":"2023-12-01T02:51:43.865315+0000","last_became_active":"2023-12-01T02:51:43.372344+0000","last_became_peered":"2023-12-01T02:51:43.372344+0000","last_unstale":"2023-12-01T02:51:43.865315+0000","last_undegraded":"2023-12-01T02:51:43.865315+0000","last_fullsized":"2023-12-01T02:51:43.865315+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:44.731298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:46.783682+0000","last_change":"2023-12-01T02:51:43.843738+0000","last_active":"2023-12-01T02:51:46.783682+0000","last_peered":"2023-12-01T02:51:46.783682+0000","last_clean":"2023-12-01T02:51:46.783682+0000","last_became_active":"2023-12-01T02:51:43.843489+0000","last_became_peered":"2023-12-01T02:51:43.843489+0000","last_unstale":"2023-12-01T02:51:46.783682+0000","last_undegraded":"2023-12-01T02:51:46.783682+0000","last_fullsized":"2023-12-01T02:51:46.783682+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:27:21.021300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.854010+0000","last_change":"2023-12-01T02:51:43.367372+0000","last_active":"2023-12-01T02:51:43.854010+0000","last_peered":"2023-12-01T02:51:43.854010+0000","last_clean":"2023-12-01T02:51:43.854010+0000","last_became_active":"2023-12-01T02:51:43.365888+0000","last_became_peered":"2023-12-01T02:51:43.365888+0000","last_unstale":"2023-12-01T02:51:43.854010+0000","last_undegraded":"2023-12-01T02:51:43.854010+0000","last_fullsized":"2023-12-01T02:51:43.854010+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:15.600950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210491+0000","last_change":"2023-12-01T02:51:43.864118+0000","last_active":"2023-12-01T02:51:47.210491+0000","last_peered":"2023-12-01T02:51:47.210491+0000","last_clean":"2023-12-01T02:51:47.210491+0000","last_became_active":"2023-12-01T02:51:43.863746+0000","last_became_peered":"2023-12-01T02:51:43.863746+0000","last_unstale":"2023-12-01T02:51:47.210491+0000","last_undegraded":"2023-12-01T02:51:47.210491+0000","last_fullsized":"2023-12-01T02:51:47.210491+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:21.573823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.858952+0000","last_change":"2023-12-01T02:51:43.375067+0000","last_active":"2023-12-01T02:51:43.858952+0000","last_peered":"2023-12-01T02:51:43.858952+0000","last_clean":"2023-12-01T02:51:43.858952+0000","last_became_active":"2023-12-01T02:51:43.374382+0000","last_became_peered":"2023-12-01T02:51:43.374382+0000","last_unstale":"2023-12-01T02:51:43.858952+0000","last_undegraded":"2023-12-01T02:51:43.858952+0000","last_fullsized":"2023-12-01T02:51:43.858952+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:48:48.689940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.340183+0000","last_change":"2023-12-01T02:51:43.862053+0000","last_active":"2023-12-01T02:51:45.340183+0000","last_peered":"2023-12-01T02:51:45.340183+0000","last_clean":"2023-12-01T02:51:45.340183+0000","last_became_active":"2023-12-01T02:51:43.861800+0000","last_became_peered":"2023-12-01T02:51:43.861800+0000","last_unstale":"2023-12-01T02:51:45.340183+0000","last_undegraded":"2023-12-01T02:51:45.340183+0000","last_fullsized":"2023-12-01T02:51:45.340183+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:20:50.332069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.863086+0000","last_change":"2023-12-01T02:51:43.374129+0000","last_active":"2023-12-01T02:51:43.863086+0000","last_peered":"2023-12-01T02:51:43.863086+0000","last_clean":"2023-12-01T02:51:43.863086+0000","last_became_active":"2023-12-01T02:51:43.373869+0000","last_became_peered":"2023-12-01T02:51:43.373869+0000","last_unstale":"2023-12-01T02:51:43.863086+0000","last_undegraded":"2023-12-01T02:51:43.863086+0000","last_fullsized":"2023-12-01T02:51:43.863086+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:10:55.039229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:46.783828+0000","last_change":"2023-12-01T02:51:43.856801+0000","last_active":"2023-12-01T02:51:46.783828+0000","last_peered":"2023-12-01T02:51:46.783828+0000","last_clean":"2023-12-01T02:51:46.783828+0000","last_became_active":"2023-12-01T02:51:43.856548+0000","last_became_peered":"2023-12-01T02:51:43.856548+0000","last_unstale":"2023-12-01T02:51:46.783828+0000","last_undegraded":"2023-12-01T02:51:46.783828+0000","last_fullsized":"2023-12-01T02:51:46.783828+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:30:11.650172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"162'2","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.867962+0000","last_change":"2023-12-01T02:51:43.363234+0000","last_active":"2023-12-01T02:51:43.867962+0000","last_peered":"2023-12-01T02:51:43.867962+0000","last_clean":"2023-12-01T02:51:43.867962+0000","last_became_active":"2023-12-01T02:51:43.362804+0000","last_became_peered":"2023-12-01T02:51:43.362804+0000","last_unstale":"2023-12-01T02:51:43.867962+0000","last_undegraded":"2023-12-01T02:51:43.867962+0000","last_fullsized":"2023-12-01T02:51:43.867962+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:42.251726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.852246+0000","last_change":"2023-12-01T02:51:43.871727+0000","last_active":"2023-12-01T02:51:44.852246+0000","last_peered":"2023-12-01T02:51:44.852246+0000","last_clean":"2023-12-01T02:51:44.852246+0000","last_became_active":"2023-12-01T02:51:43.870069+0000","last_became_peered":"2023-12-01T02:51:43.870069+0000","last_unstale":"2023-12-01T02:51:44.852246+0000","last_undegraded":"2023-12-01T02:51:44.852246+0000","last_fullsized":"2023-12-01T02:51:44.852246+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:38.367687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.854989+0000","last_change":"2023-12-01T02:51:43.379848+0000","last_active":"2023-12-01T02:51:43.854989+0000","last_peered":"2023-12-01T02:51:43.854989+0000","last_clean":"2023-12-01T02:51:43.854989+0000","last_became_active":"2023-12-01T02:51:43.379529+0000","last_became_peered":"2023-12-01T02:51:43.379529+0000","last_unstale":"2023-12-01T02:51:43.854989+0000","last_undegraded":"2023-12-01T02:51:43.854989+0000","last_fullsized":"2023-12-01T02:51:43.854989+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:08:07.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.211114+0000","last_change":"2023-12-01T02:51:43.874455+0000","last_active":"2023-12-01T02:51:47.211114+0000","last_peered":"2023-12-01T02:51:47.211114+0000","last_clean":"2023-12-01T02:51:47.211114+0000","last_became_active":"2023-12-01T02:51:43.874190+0000","last_became_peered":"2023-12-01T02:51:43.874190+0000","last_unstale":"2023-12-01T02:51:47.211114+0000","last_undegraded":"2023-12-01T02:51:47.211114+0000","last_fullsized":"2023-12-01T02:51:47.211114+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:50:29.480200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.859328+0000","last_change":"2023-12-01T02:51:43.374535+0000","last_active":"2023-12-01T02:51:43.859328+0000","last_peered":"2023-12-01T02:51:43.859328+0000","last_clean":"2023-12-01T02:51:43.859328+0000","last_became_active":"2023-12-01T02:51:43.373732+0000","last_became_peered":"2023-12-01T02:51:43.373732+0000","last_unstale":"2023-12-01T02:51:43.859328+0000","last_undegraded":"2023-12-01T02:51:43.859328+0000","last_fullsized":"2023-12-01T02:51:43.859328+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:58.168046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.847070+0000","last_change":"2023-12-01T02:51:43.857890+0000","last_active":"2023-12-01T02:51:44.847070+0000","last_peered":"2023-12-01T02:51:44.847070+0000","last_clean":"2023-12-01T02:51:44.847070+0000","last_became_active":"2023-12-01T02:51:43.857238+0000","last_became_peered":"2023-12-01T02:51:43.857238+0000","last_unstale":"2023-12-01T02:51:44.847070+0000","last_undegraded":"2023-12-01T02:51:44.847070+0000","last_fullsized":"2023-12-01T02:51:44.847070+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:33.691592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.854030+0000","last_change":"2023-12-01T02:51:43.372524+0000","last_active":"2023-12-01T02:51:43.854030+0000","last_peered":"2023-12-01T02:51:43.854030+0000","last_clean":"2023-12-01T02:51:43.854030+0000","last_became_active":"2023-12-01T02:51:43.372250+0000","last_became_peered":"2023-12-01T02:51:43.372250+0000","last_unstale":"2023-12-01T02:51:43.854030+0000","last_undegraded":"2023-12-01T02:51:43.854030+0000","last_fullsized":"2023-12-01T02:51:43.854030+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:08.095677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.339380+0000","last_change":"2023-12-01T02:51:43.859670+0000","last_active":"2023-12-01T02:51:45.339380+0000","last_peered":"2023-12-01T02:51:45.339380+0000","last_clean":"2023-12-01T02:51:45.339380+0000","last_became_active":"2023-12-01T02:51:43.859416+0000","last_became_peered":"2023-12-01T02:51:43.859416+0000","last_unstale":"2023-12-01T02:51:45.339380+0000","last_undegraded":"2023-12-01T02:51:45.339380+0000","last_fullsized":"2023-12-01T02:51:45.339380+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:47:23.015954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.845357+0000","last_change":"2023-12-01T02:51:43.373696+0000","last_active":"2023-12-01T02:51:43.845357+0000","last_peered":"2023-12-01T02:51:43.845357+0000","last_clean":"2023-12-01T02:51:43.845357+0000","last_became_active":"2023-12-01T02:51:43.373372+0000","last_became_peered":"2023-12-01T02:51:43.373372+0000","last_unstale":"2023-12-01T02:51:43.845357+0000","last_undegraded":"2023-12-01T02:51:43.845357+0000","last_fullsized":"2023-12-01T02:51:43.845357+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:05:25.494760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:45.340121+0000","last_change":"2023-12-01T02:51:43.865853+0000","last_active":"2023-12-01T02:51:45.340121+0000","last_peered":"2023-12-01T02:51:45.340121+0000","last_clean":"2023-12-01T02:51:45.340121+0000","last_became_active":"2023-12-01T02:51:43.864994+0000","last_became_peered":"2023-12-01T02:51:43.864994+0000","last_unstale":"2023-12-01T02:51:45.340121+0000","last_undegraded":"2023-12-01T02:51:45.340121+0000","last_fullsized":"2023-12-01T02:51:45.340121+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:57:20.260024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.847449+0000","last_change":"2023-12-01T02:51:43.377946+0000","last_active":"2023-12-01T02:51:43.847449+0000","last_peered":"2023-12-01T02:51:43.847449+0000","last_clean":"2023-12-01T02:51:43.847449+0000","last_became_active":"2023-12-01T02:51:43.377680+0000","last_became_peered":"2023-12-01T02:51:43.377680+0000","last_unstale":"2023-12-01T02:51:43.847449+0000","last_undegraded":"2023-12-01T02:51:43.847449+0000","last_fullsized":"2023-12-01T02:51:43.847449+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:42.251726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.850472+0000","last_change":"2023-12-01T02:51:43.874403+0000","last_active":"2023-12-01T02:51:44.850472+0000","last_peered":"2023-12-01T02:51:44.850472+0000","last_clean":"2023-12-01T02:51:44.850472+0000","last_became_active":"2023-12-01T02:51:43.874123+0000","last_became_peered":"2023-12-01T02:51:43.874123+0000","last_unstale":"2023-12-01T02:51:44.850472+0000","last_undegraded":"2023-12-01T02:51:44.850472+0000","last_fullsized":"2023-12-01T02:51:44.850472+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:31.416300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.864433+0000","last_change":"2023-12-01T02:51:43.379902+0000","last_active":"2023-12-01T02:51:43.864433+0000","last_peered":"2023-12-01T02:51:43.864433+0000","last_clean":"2023-12-01T02:51:43.864433+0000","last_became_active":"2023-12-01T02:51:43.379590+0000","last_became_peered":"2023-12-01T02:51:43.379590+0000","last_unstale":"2023-12-01T02:51:43.864433+0000","last_undegraded":"2023-12-01T02:51:43.864433+0000","last_fullsized":"2023-12-01T02:51:43.864433+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:08.604890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:47.210156+0000","last_change":"2023-12-01T02:51:43.870972+0000","last_active":"2023-12-01T02:51:47.210156+0000","last_peered":"2023-12-01T02:51:47.210156+0000","last_clean":"2023-12-01T02:51:47.210156+0000","last_became_active":"2023-12-01T02:51:43.870670+0000","last_became_peered":"2023-12-01T02:51:43.870670+0000","last_unstale":"2023-12-01T02:51:47.210156+0000","last_undegraded":"2023-12-01T02:51:47.210156+0000","last_fullsized":"2023-12-01T02:51:47.210156+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:54.796980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.854154+0000","last_change":"2023-12-01T02:51:43.371742+0000","last_active":"2023-12-01T02:51:43.854154+0000","last_peered":"2023-12-01T02:51:43.854154+0000","last_clean":"2023-12-01T02:51:43.854154+0000","last_became_active":"2023-12-01T02:51:43.371228+0000","last_became_peered":"2023-12-01T02:51:43.371228+0000","last_unstale":"2023-12-01T02:51:43.854154+0000","last_undegraded":"2023-12-01T02:51:43.854154+0000","last_fullsized":"2023-12-01T02:51:43.854154+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:30.398882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-12-01T02:51:44.847044+0000","last_change":"2023-12-01T02:51:43.857477+0000","last_active":"2023-12-01T02:51:44.847044+0000","last_peered":"2023-12-01T02:51:44.847044+0000","last_clean":"2023-12-01T02:51:44.847044+0000","last_became_active":"2023-12-01T02:51:43.857212+0000","last_became_peered":"2023-12-01T02:51:43.857212+0000","last_unstale":"2023-12-01T02:51:44.847044+0000","last_undegraded":"2023-12-01T02:51:44.847044+0000","last_fullsized":"2023-12-01T02:51:44.847044+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:43.324525+0000","last_clean_scrub_stamp":"2023-12-01T02:51:43.324525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:02.554560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"162'2","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.863865+0000","last_change":"2023-12-01T02:51:43.377875+0000","last_active":"2023-12-01T02:51:43.863865+0000","last_peered":"2023-12-01T02:51:43.863865+0000","last_clean":"2023-12-01T02:51:43.863865+0000","last_became_active":"2023-12-01T02:51:43.377573+0000","last_became_peered":"2023-12-01T02:51:43.377573+0000","last_unstale":"2023-12-01T02:51:43.863865+0000","last_undegraded":"2023-12-01T02:51:43.863865+0000","last_fullsized":"2023-12-01T02:51:43.863865+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:51:42.307107+0000","last_clean_scrub_stamp":"2023-12-01T02:51:42.307107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:41.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"151'249","reported_seq":791,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-01T02:51:43.841135+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T02:51:43.841135+0000","last_peered":"2023-12-01T02:51:43.841135+0000","last_clean":"2023-12-01T02:51:43.841135+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T02:51:43.841135+0000","last_undegraded":"2023-12-01T02:51:43.841135+0000","last_fullsized":"2023-12-01T02:51:43.841135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575756,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1984,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":2031616,"data_stored":978353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3799999999999999,"5min":1.7829999999999999,"15min":1.893},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1539999999999999,"15min":1.1100000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.514,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.2589999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.266,"15min":1.2709999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.02},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85399999999999998},"max":{"1min":1.585,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":1.391}]},{"osd":2,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.2010000000000001},"min":{"1min":0.82499999999999996,"5min":0.80600000000000005,"15min":0.78500000000000003},"max":{"1min":1.554,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.28,"15min":1.25},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8440000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":2.4780000000000002}]},{"osd":3,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.343,"15min":1.3220000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.742,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":2.5139999999999998},{"interface":"front","average":{"1min":1.415,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.048,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":2.6160000000000001}]},{"osd":4,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.851,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":2.298},{"interface":"front","average":{"1min":1.506,"5min":1.4930000000000001,"15min":1.4750000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.157,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":2.665}]},{"osd":5,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3149999999999999,"15min":1.284},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.0699999999999998,"5min":2.976,"15min":2.976},"last":2.1920000000000002},{"interface":"front","average":{"1min":1.597,"5min":1.595,"15min":1.569},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.2160000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":2.5609999999999999}]},{"osd":6,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4590000000000001,"15min":1.4179999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.1499999999999999,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":2.448},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4870000000000001,"15min":1.4470000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.966,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":2.4180000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575757,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37112,"kb_used_data":1968,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334728,"statfs":{"total":96636764160,"available":96598761472,"internally_reserved":0,"allocated":2015232,"data_stored":971975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.161,"15min":1.165},"min":{"1min":0.84299999999999997,"5min":0.78100000000000003,"15min":0.75},"max":{"1min":1.6080000000000001,"5min":1.73,"15min":2.2759999999999998},"last":1.016},{"interface":"front","average":{"1min":1.083,"5min":1.1439999999999999,"15min":1.1619999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.56799999999999995},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.966},"last":1.1699999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1459999999999999,"15min":1.1859999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.78800000000000003},"max":{"1min":1.6459999999999999,"5min":1.772,"15min":2.0409999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6679999999999999,"5min":1.7370000000000001,"15min":2.093},"last":1.4339999999999999}]},{"osd":2,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1679999999999999,"15min":1.1879999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.80600000000000005},"max":{"1min":1.5369999999999999,"5min":1.736,"15min":1.946},"last":1.639},{"interface":"front","average":{"1min":1.274,"5min":1.272,"15min":1.284},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.637,"5min":1.7190000000000001,"15min":2.1230000000000002},"last":1.696}]},{"osd":3,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":1.0620000000000001,"5min":1.006,"15min":0.90500000000000003},"max":{"1min":1.6140000000000001,"5min":1.819,"15min":1.819},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.431,"15min":1.4319999999999999},"min":{"1min":1.042,"5min":0.98699999999999999,"15min":0.92800000000000005},"max":{"1min":1.738,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.756}]},{"osd":4,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.262},"min":{"1min":0.93000000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.585},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3500000000000001,"15min":1.3360000000000001},"min":{"1min":0.96699999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.98,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.875}]},{"osd":5,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.4419999999999999,"15min":1.4279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":2.0830000000000002},{"interface":"front","average":{"1min":1.603,"5min":1.5,"15min":1.4970000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.7799999999999998},"last":2.29}]},{"osd":7,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.5149999999999999,"15min":1.524},"min":{"1min":1.0980000000000001,"5min":1.034,"15min":1.034},"max":{"1min":1.9419999999999999,"5min":2.1549999999999998,"15min":2.5659999999999998},"last":2.1960000000000002},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6040000000000001,"15min":1.5940000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.0499999999999998,"5min":2.226,"15min":2.3809999999999998},"last":2.161}]}]},{"osd":5,"up_from":13,"seq":55834575756,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59860,"kb_used_data":2572,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311980,"statfs":{"total":96636764160,"available":96575467520,"internally_reserved":0,"allocated":2633728,"data_stored":1569429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1499999999999999,"15min":1.1419999999999999},"min":{"1min":0.68700000000000006,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4810000000000001,"5min":1.73,"15min":1.744},"last":1.411},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.0680000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.137,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.56200000000000006},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.704},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.226,"15min":1.2330000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.016},"last":0.81399999999999995}]},{"osd":2,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2270000000000001,"15min":1.24},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5900000000000001,"5min":1.7430000000000001,"15min":1.746},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2609999999999999,"15min":1.2809999999999999},"min":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.627},"max":{"1min":1.655,"5min":1.671,"15min":1.764},"last":1.238}]},{"osd":3,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.321,"15min":1.319},"min":{"1min":0.98399999999999999,"5min":0.872,"15min":0.84899999999999998},"max":{"1min":1.546,"5min":1.7729999999999999,"15min":1.871},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3460000000000001,"15min":1.3620000000000001},"min":{"1min":1.0269999999999999,"5min":0.91500000000000004,"15min":0.879},"max":{"1min":1.7250000000000001,"5min":1.7470000000000001,"15min":1.962},"last":1.006}]},{"osd":4,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.254,"15min":1.2569999999999999},"min":{"1min":0.91300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.696,"5min":1.847,"15min":1.978},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.361,"15min":1.3620000000000001},"min":{"1min":1.0740000000000001,"5min":0.91600000000000004,"15min":0.88600000000000001},"max":{"1min":1.911,"5min":1.911,"15min":2.0350000000000001},"last":1.4870000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3520000000000001,"15min":1.375},"min":{"1min":0.89300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":2.4470000000000001},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.391,"15min":1.419},"min":{"1min":1.077,"5min":0.876,"15min":0.876},"max":{"1min":1.798,"5min":1.946,"15min":2.7210000000000001},"last":1.736}]},{"osd":7,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.5449999999999999,"15min":1.5369999999999999},"min":{"1min":1.1200000000000001,"5min":1.099,"15min":0.92100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":3.0379999999999998},"last":1.323},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5780000000000001,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":3.5270000000000001},"last":1.7969999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575756,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46980,"kb_used_data":1980,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324860,"statfs":{"total":96636764160,"available":96588656640,"internally_reserved":0,"allocated":2027520,"data_stored":978319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.163,"15min":1.1519999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70699999999999996},"max":{"1min":1.444,"5min":1.5269999999999999,"15min":1.6830000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.157,"15min":1.1759999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.749},"max":{"1min":1.466,"5min":1.581,"15min":1.724},"last":1.5880000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.272,"15min":1.2330000000000001},"min":{"1min":0.93000000000000005,"5min":0.86299999999999999,"15min":0.81399999999999995},"max":{"1min":1.5800000000000001,"5min":1.724,"15min":1.7709999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1950000000000001,"15min":1.1970000000000001},"min":{"1min":0.80700000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.605,"5min":1.6100000000000001,"15min":1.8640000000000001},"last":1.381}]},{"osd":2,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.89400000000000002,"5min":0.83899999999999997,"15min":0.82799999999999996},"max":{"1min":1.5229999999999999,"5min":1.6419999999999999,"15min":2.1499999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.335,"15min":1.321},"min":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":0.85999999999999999},"max":{"1min":1.5760000000000001,"5min":1.802,"15min":1.802},"last":1.466}]},{"osd":3,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2769999999999999,"15min":1.2689999999999999},"min":{"1min":1,"5min":0.95299999999999996,"15min":0.85399999999999998},"max":{"1min":1.536,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3460000000000001,"15min":1.333},"min":{"1min":1.046,"5min":1.0009999999999999,"15min":0.88100000000000001},"max":{"1min":1.6180000000000001,"5min":1.7050000000000001,"15min":2.2440000000000002},"last":1.3999999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.363,"15min":1.3149999999999999},"min":{"1min":0.96199999999999997,"5min":0.86499999999999999,"15min":0.84499999999999997},"max":{"1min":1.677,"5min":1.804,"15min":1.8839999999999999},"last":1.736},{"interface":"front","average":{"1min":1.29,"5min":1.321,"15min":1.2829999999999999},"min":{"1min":0.871,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.75,"5min":1.875,"15min":1.875},"last":1.4370000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.373,"15min":1.325},"min":{"1min":0.89100000000000001,"5min":0.81299999999999994,"15min":0.78500000000000003},"max":{"1min":1.7490000000000001,"5min":1.853,"15min":2.4489999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.4610000000000001,"15min":1.4450000000000001},"min":{"1min":1.0509999999999999,"5min":0.97999999999999998,"15min":0.97899999999999998},"max":{"1min":1.7290000000000001,"5min":1.8939999999999999,"15min":2.4870000000000001},"last":1.6759999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4670000000000001,"15min":1.4470000000000001},"min":{"1min":1.204,"5min":1.1020000000000001,"15min":0.92900000000000005},"max":{"1min":1.8540000000000001,"5min":2.044,"15min":2.5529999999999999},"last":1.526},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.5600000000000001,"15min":1.5469999999999999},"min":{"1min":1.244,"5min":1.0600000000000001,"15min":0.99099999999999999},"max":{"1min":1.8899999999999999,"5min":2.0670000000000002,"15min":2.5760000000000001},"last":1.6180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575755,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51080,"kb_used_data":1984,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320760,"statfs":{"total":96636764160,"available":96584458240,"internally_reserved":0,"allocated":2031616,"data_stored":978389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.90900000000000003,"15min":0.88700000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3160000000000001,"5min":1.4970000000000001,"15min":1.5009999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.071,"15min":1.077},"min":{"1min":0.751,"5min":0.65000000000000002,"15min":0.60299999999999998},"max":{"1min":1.444,"5min":1.5449999999999999,"15min":1.5669999999999999},"last":0.92800000000000005}]},{"osd":1,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1839999999999999,"15min":1.159},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.755},"max":{"1min":1.625,"5min":1.708,"15min":1.708},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0089999999999999,"15min":1.018},"min":{"1min":0.77900000000000003,"5min":0.72999999999999998,"15min":0.52700000000000002},"max":{"1min":1.5609999999999999,"5min":1.5980000000000001,"15min":1.744},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.081,"15min":1.099},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.7549999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.2589999999999999,"15min":1.264},"min":{"1min":0.93899999999999995,"5min":0.93000000000000005,"15min":0.53400000000000003},"max":{"1min":1.4870000000000001,"5min":1.6639999999999999,"15min":1.728},"last":1.296}]},{"osd":4,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.294,"15min":1.29},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.89200000000000002},"max":{"1min":1.611,"5min":1.6479999999999999,"15min":1.7050000000000001},"last":1.329},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3140000000000001,"15min":1.2929999999999999},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":0.67100000000000004},"max":{"1min":1.6719999999999999,"5min":1.7789999999999999,"15min":1.927},"last":1.2370000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3180000000000001,"15min":1.3160000000000001},"min":{"1min":1.022,"5min":0.92300000000000004,"15min":0.92000000000000004},"max":{"1min":1.589,"5min":1.788,"15min":1.8260000000000001},"last":1.357},{"interface":"front","average":{"1min":1.397,"5min":1.4059999999999999,"15min":1.3859999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":0.94299999999999995},"max":{"1min":1.8,"5min":1.865,"15min":1.865},"last":1.282}]},{"osd":6,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.4630000000000001,"15min":1.4399999999999999},"min":{"1min":1.1040000000000001,"5min":1.069,"15min":0.67900000000000005},"max":{"1min":1.8440000000000001,"5min":2.0230000000000001,"15min":2.1160000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5349999999999999,"15min":1.496},"min":{"1min":1.3440000000000001,"5min":1.093,"15min":0.85499999999999998},"max":{"1min":1.867,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.3819999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:51:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5269999999999999,"15min":1.5109999999999999},"min":{"1min":1.228,"5min":1.177,"15min":0.80600000000000005},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.1549999999999998},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.506,"5min":1.528,"15min":1.516},"min":{"1min":1.3320000000000001,"5min":1.21,"15min":0.73699999999999999},"max":{"1min":1.8420000000000001,"5min":2.0150000000000001,"15min":2.1259999999999999},"last":1.5669999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37120,"kb_used_data":1976,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334720,"statfs":{"total":96636764160,"available":96598753280,"internally_reserved":0,"allocated":2023424,"data_stored":972647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.89700000000000002,"15min":0.872},"min":{"1min":0.72899999999999998,"5min":0.57299999999999995,"15min":0.49299999999999999},"max":{"1min":1.4059999999999999,"5min":1.532,"15min":1.5569999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.077,"15min":1.0660000000000001},"min":{"1min":0.68700000000000006,"5min":0.66300000000000003,"15min":0.59599999999999997},"max":{"1min":1.4139999999999999,"5min":1.728,"15min":1.728},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.101,"15min":1.077},"min":{"1min":0.83899999999999997,"5min":0.80300000000000005,"15min":0.67400000000000004},"max":{"1min":1.502,"5min":1.7070000000000001,"15min":1.8029999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.113,"15min":1.1140000000000001},"min":{"1min":0.88700000000000001,"5min":0.77000000000000002,"15min":0.53300000000000003},"max":{"1min":1.383,"5min":1.6259999999999999,"15min":1.6599999999999999},"last":1.1879999999999999}]},{"osd":3,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0780000000000001,"15min":1.056},"min":{"1min":0.81799999999999995,"5min":0.72799999999999998,"15min":0.55500000000000005},"max":{"1min":1.4239999999999999,"5min":1.492,"15min":1.7070000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2549999999999999,"15min":1.2170000000000001},"min":{"1min":0.98099999999999998,"5min":0.90100000000000002,"15min":0.70599999999999996},"max":{"1min":1.5329999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.98399999999999999}]},{"osd":4,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3500000000000001,"15min":1.319},"min":{"1min":1.123,"5min":0.89700000000000002,"15min":0.71099999999999997},"max":{"1min":1.7230000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.3049999999999999,"15min":1.3009999999999999},"min":{"1min":0.96599999999999997,"5min":0.90700000000000003,"15min":0.78200000000000003},"max":{"1min":1.518,"5min":1.78,"15min":1.8520000000000001},"last":1.385}]},{"osd":5,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3819999999999999,"15min":1.3640000000000001},"min":{"1min":1.1060000000000001,"5min":1.095,"15min":0.873},"max":{"1min":1.6479999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.145},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4339999999999999,"15min":1.399},"min":{"1min":1.1339999999999999,"5min":0.84699999999999998,"15min":0.79200000000000004},"max":{"1min":1.9450000000000001,"5min":1.978,"15min":2.0790000000000002},"last":1.4610000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3999999999999999,"15min":1.385},"min":{"1min":1.1559999999999999,"5min":0.98399999999999999,"15min":0.73999999999999999},"max":{"1min":1.6619999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5349999999999999,"15min":1.496},"min":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":0.82699999999999996},"max":{"1min":1.9450000000000001,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":1.661}]},{"osd":7,"last update":"Fri Dec 1 02:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5089999999999999,"15min":1.478},"min":{"1min":1.208,"5min":0.95799999999999996,"15min":0.83899999999999997},"max":{"1min":1.8500000000000001,"5min":1.9019999999999999,"15min":2.048},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4990000000000001,"15min":1.46},"min":{"1min":1.111,"5min":1.0840000000000001,"15min":0.64500000000000002},"max":{"1min":1.73,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.446}]}]},{"osd":1,"up_from":13,"seq":55834575754,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63960,"kb_used_data":2576,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307880,"statfs":{"total":96636764160,"available":96571269120,"internally_reserved":0,"allocated":2637824,"data_stored":1570273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.137,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.095,"15min":1.0740000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.722,"15min":1.722},"last":1.1519999999999999}]},{"osd":2,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93200000000000005,"15min":0.92500000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.466,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1439999999999999,"15min":1.1479999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5880000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.371}]},{"osd":3,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5189999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.274},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1339999999999999,"15min":1.1279999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.444,"5min":1.6759999999999999,"15min":1.9219999999999999},"last":1.4710000000000001}]},{"osd":4,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.24,"15min":1.226},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.504},{"interface":"front","average":{"1min":1.413,"5min":1.4219999999999999,"15min":1.391},"min":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.032},"max":{"1min":1.8140000000000001,"5min":1.9730000000000001,"15min":2.1280000000000001},"last":1.5669999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.349},"min":{"1min":1.0620000000000001,"5min":0.98899999999999999,"15min":0.97899999999999998},"max":{"1min":1.49,"5min":1.9930000000000001,"15min":2.0449999999999999},"last":1.306},{"interface":"front","average":{"1min":1.375,"5min":1.3660000000000001,"15min":1.357},"min":{"1min":1.0349999999999999,"5min":1.0229999999999999,"15min":1.004},"max":{"1min":1.804,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.9710000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.454,"15min":1.4399999999999999},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.873,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.667},{"interface":"front","average":{"1min":1.341,"5min":1.4139999999999999,"15min":1.4119999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.7250000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.7789999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5289999999999999,"15min":1.518},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.9339999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.696},{"interface":"front","average":{"1min":1.431,"5min":1.516,"15min":1.514},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8029999999999999,"5min":2.1749999999999998,"15min":2.282},"last":1.754}]}]},{"osd":0,"up_from":13,"seq":55834575756,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37144,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334696,"statfs":{"total":96636764160,"available":96598728704,"internally_reserved":0,"allocated":2048000,"data_stored":979939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.90600000000000003,"15min":0.90700000000000003},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.63400000000000001},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.766},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.119,"5min":1.087,"15min":1.075},"min":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.64500000000000002},"max":{"1min":1.5029999999999999,"5min":1.5980000000000001,"15min":1.7889999999999999},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.05,"15min":1.069},"min":{"1min":0.76800000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.3120000000000001,"5min":1.4470000000000001,"15min":1.8540000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1160000000000001,"15min":1.1339999999999999},"min":{"1min":0.84299999999999997,"5min":0.76600000000000001,"15min":0.76100000000000001},"max":{"1min":1.4199999999999999,"5min":1.595,"15min":1.728},"last":0.98999999999999999}]},{"osd":3,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.0880000000000001,"15min":1.0700000000000001},"min":{"1min":0.82499999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.421,"5min":1.5509999999999999,"15min":1.8100000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2729999999999999,"15min":1.2609999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.81599999999999995},"max":{"1min":1.498,"5min":1.798,"15min":1.9470000000000001},"last":1.2509999999999999}]},{"osd":4,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.365,"15min":1.3560000000000001},"min":{"1min":1.048,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.0979999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.286,"15min":1.2889999999999999},"min":{"1min":0.98299999999999998,"5min":0.90200000000000002,"15min":0.88200000000000001},"max":{"1min":1.55,"5min":1.6579999999999999,"15min":2.0739999999999998},"last":1.1890000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.429,"15min":1.409},"min":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0109999999999999},"max":{"1min":1.946,"5min":2.2370000000000001,"15min":2.6110000000000002},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.4330000000000001,"15min":1.415},"min":{"1min":1.2490000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.831,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.149}]},{"osd":6,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.391,"15min":1.3779999999999999},"min":{"1min":1.083,"5min":0.745,"15min":0.745},"max":{"1min":1.6100000000000001,"5min":2.0939999999999999,"15min":2.1680000000000001},"last":1.03},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.5129999999999999,"15min":1.5069999999999999},"min":{"1min":1.1870000000000001,"5min":1.1259999999999999,"15min":1.107},"max":{"1min":1.863,"5min":2.798,"15min":2.798},"last":1.3240000000000001}]},{"osd":7,"last update":"Fri Dec 1 02:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.552,"15min":1.516},"min":{"1min":1.3320000000000001,"5min":1.085,"15min":1.085},"max":{"1min":1.923,"5min":2.2919999999999998,"15min":2.5329999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.492,"15min":1.4870000000000001},"min":{"1min":1.2110000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.909,"5min":2.097,"15min":3.0590000000000002},"last":0.96799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T02:51:48.756 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T02:51:48.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T02:51:49.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.022+0000 7fafaddb1640 1 Processor -- start 2023-12-01T02:51:49.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.022+0000 7fafaddb1640 1 -- start start 2023-12-01T02:51:49.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.024+0000 7fafaddb1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa806c800 0x7fafa80ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.024+0000 7fafaddb1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa80b2b00 con 0x7fafa8143240 2023-12-01T02:51:49.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.024+0000 7fafaddb1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa80b2de0 con 0x7fafa806c390 2023-12-01T02:51:49.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.024+0000 7fafaddb1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80b30c0 con 0x7fafa806c800 2023-12-01T02:51:49.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.024+0000 7fafa77fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fafa8143240 0x7fafa80b20a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40282/0 (socket says 172.21.15.115:40282) 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.024+0000 7fafa77fe640 1 -- 172.21.15.115:0/548265784 learned_addr learned my addr 172.21.15.115:0/548265784 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa7fff640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa806c800 0x7fafa80ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899254845 0 0) 0x7fafa80b2b00 con 0x7fafa8143240 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf94003520 con 0x7fafa8143240 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3034024584 0 0) 0x7faf94003520 con 0x7fafa8143240 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa806c800 msgr2=0x7fafa80ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa806c800 0x7fafa80ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 >> v1:172.21.15.142:6789/0 conn(0x7fafa806c390 legacy=0x7fafa80ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa80b33a0 con 0x7fafa8143240 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7faf98001670 con 0x7fafa8143240 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 >> v1:172.21.15.115:6789/0 conn(0x7fafa8143240 legacy=0x7fafa80b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 0x7faf94003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf940047c0 0x7faf94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa806c800 0x7faf94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80b30c0 con 0x7faf940039a0 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9400ba30 con 0x7faf940047c0 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.025+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9400de00 con 0x7fafa806c800 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.026+0000 7fafa77fe640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 0x7faf94003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.026+0000 7faf9effd640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf940047c0 0x7faf94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.026+0000 7fafa77fe640 1 -- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa806c800 msgr2=0x7faf94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.026+0000 7fafa7fff640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa806c800 0x7faf94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.026+0000 7fafa77fe640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa806c800 0x7faf94005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.026+0000 7fafa77fe640 1 -- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf940047c0 msgr2=0x7faf94004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.026+0000 7fafa77fe640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf940047c0 0x7faf94004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.026+0000 7fafa77fe640 1 -- 172.21.15.115:0/548265784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf94003520 con 0x7faf940039a0 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.026+0000 7faf9effd640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf940047c0 0x7faf94004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.027+0000 7fafa77fe640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 0x7faf94003dc0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7faf9800c210 tx=0x7faf940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8e538b35e7055e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.027+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98005ad0 con 0x7faf940039a0 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.027+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf98005c30 con 0x7faf940039a0 2023-12-01T02:51:49.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.027+0000 7fafa6ffd640 1 -- 172.21.15.115:0/548265784 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9802c870 con 0x7faf940039a0 2023-12-01T02:51:49.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.028+0000 7fafaddb1640 1 -- 172.21.15.115:0/548265784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 msgr2=0x7faf94003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.028+0000 7fafaddb1640 1 --2- 172.21.15.115:0/548265784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 0x7faf94003dc0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7faf9800c210 tx=0x7faf940057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.028+0000 7fafa7fff640 1 -- 172.21.15.115:0/548265784 reap_dead start 2023-12-01T02:51:49.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.028+0000 7fafaddb1640 1 -- 172.21.15.115:0/548265784 shutdown_connections 2023-12-01T02:51:49.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.028+0000 7fafaddb1640 1 -- 172.21.15.115:0/548265784 >> 172.21.15.115:0/548265784 conn(0x7fafa8065960 msgr2=0x7fafa8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:49.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.028+0000 7fafaddb1640 1 -- 172.21.15.115:0/548265784 shutdown_connections 2023-12-01T02:51:49.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.028+0000 7fafaddb1640 1 -- 172.21.15.115:0/548265784 wait complete. 2023-12-01T02:51:49.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.030+0000 7fafaddb1640 1 Processor -- start 2023-12-01T02:51:49.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.030+0000 7fafaddb1640 1 -- start start 2023-12-01T02:51:49.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.031+0000 7fafaddb1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 0x7fafa813fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.031+0000 7fafaddb1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa813acf0 0x7fafa81ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.031+0000 7fafaddb1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa8153400 0x7fafa8153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.031+0000 7fafaddb1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa8153dc0 con 0x7faf940039a0 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.031+0000 7fafaddb1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa81f11c0 con 0x7fafa8153400 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.031+0000 7fafaddb1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa813c7e0 con 0x7fafa813acf0 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.031+0000 7fafa77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 0x7fafa813fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.031+0000 7fafa77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 0x7fafa813fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45592/0 (socket says 172.21.15.115:45592) 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.031+0000 7fafa77fe640 1 -- 172.21.15.115:0/1170299798 learned_addr learned my addr 172.21.15.115:0/1170299798 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7faf9effd640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa813acf0 0x7fafa81ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7fafa77fe640 1 -- 172.21.15.115:0/1170299798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa813acf0 msgr2=0x7fafa81ed8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7fafa7fff640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa8153400 0x7fafa8153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7fafa77fe640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa813acf0 0x7fafa81ed8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7fafa77fe640 1 -- 172.21.15.115:0/1170299798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa8153400 msgr2=0x7fafa8153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7fafa77fe640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa8153400 0x7fafa8153880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7fafa77fe640 1 -- 172.21.15.115:0/1170299798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa813b170 con 0x7faf940039a0 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7faf9effd640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa813acf0 0x7fafa81ed8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:49.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7fafa77fe640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 0x7fafa813fd10 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7faf9800c210 tx=0x7faf98001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7fafa4ff9640 1 -- 172.21.15.115:0/1170299798 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98039070 con 0x7faf940039a0 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7fafa4ff9640 1 -- 172.21.15.115:0/1170299798 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9802cc80 con 0x7faf940039a0 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.032+0000 7fafa4ff9640 1 -- 172.21.15.115:0/1170299798 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9803c040 con 0x7faf940039a0 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.033+0000 7fafaddb1640 1 -- 172.21.15.115:0/1170299798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa813b3f0 con 0x7faf940039a0 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.033+0000 7fafaddb1640 1 -- 172.21.15.115:0/1170299798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa813b9b0 con 0x7faf940039a0 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.034+0000 7fafaddb1640 1 -- 172.21.15.115:0/1170299798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa804e890 con 0x7faf940039a0 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.035+0000 7fafa4ff9640 1 -- 172.21.15.115:0/1170299798 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7faf98032070 con 0x7faf940039a0 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.035+0000 7fafa4ff9640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf8806a550 0x7faf8806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.036+0000 7fafa4ff9640 1 -- 172.21.15.115:0/1170299798 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7faf980a9e90 con 0x7faf940039a0 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.036+0000 7faf9effd640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf8806a550 0x7faf8806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.036+0000 7faf9effd640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf8806a550 0x7faf8806ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fafa8139c50 tx=0x7faf90009040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:49.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.040+0000 7fafa4ff9640 1 -- 172.21.15.115:0/1170299798 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faf980752a0 con 0x7faf940039a0 2023-12-01T02:51:49.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.302+0000 7fafaddb1640 1 -- 172.21.15.115:0/1170299798 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fafa815a250 con 0x7faf940039a0 2023-12-01T02:51:49.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.311+0000 7fafa4ff9640 1 -- 172.21.15.115:0/1170299798 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v40) v1 ==== 190+0+0 (secure 0 0 0) 0x7faf980ab020 con 0x7faf940039a0 2023-12-01T02:51:49.312 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T02:51:49.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.313+0000 7fafaddb1640 1 -- 172.21.15.115:0/1170299798 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf8806a550 msgr2=0x7faf8806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.313+0000 7fafaddb1640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf8806a550 0x7faf8806ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fafa8139c50 tx=0x7faf90009040 comp rx=0 tx=0).stop 2023-12-01T02:51:49.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.313+0000 7fafaddb1640 1 -- 172.21.15.115:0/1170299798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 msgr2=0x7fafa813fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.313+0000 7fafaddb1640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 0x7fafa813fd10 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7faf9800c210 tx=0x7faf98001a00 comp rx=0 tx=0).stop 2023-12-01T02:51:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.313+0000 7fafaddb1640 1 -- 172.21.15.115:0/1170299798 shutdown_connections 2023-12-01T02:51:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.313+0000 7fafaddb1640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7faf8806a550 0x7faf8806ca10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.313+0000 7fafaddb1640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fafa813acf0 0x7fafa81ed8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.313+0000 7fafaddb1640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fafa8153400 0x7fafa8153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.313+0000 7fafaddb1640 1 --2- 172.21.15.115:0/1170299798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf940039a0 0x7fafa813fd10 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.313+0000 7fafaddb1640 1 -- 172.21.15.115:0/1170299798 >> 172.21.15.115:0/1170299798 conn(0x7fafa8065960 msgr2=0x7fafa80b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.314+0000 7fafaddb1640 1 -- 172.21.15.115:0/1170299798 shutdown_connections 2023-12-01T02:51:49.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.314+0000 7fafaddb1640 1 -- 172.21.15.115:0/1170299798 wait complete. 2023-12-01T02:51:49.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:51:49.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.543+0000 7f553744a640 1 Processor -- start 2023-12-01T02:51:49.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.543+0000 7f553744a640 1 -- start start 2023-12-01T02:51:49.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.544+0000 7f553744a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f553006f2f0 0x7f553006bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.545+0000 7f553744a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55300add10 con 0x7f55300cad70 2023-12-01T02:51:49.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.545+0000 7f553744a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55300c5760 con 0x7f553015e2f0 2023-12-01T02:51:49.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.545+0000 7f553744a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f553015e6d0 con 0x7f553006f2f0 2023-12-01T02:51:49.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.545+0000 7f55349e5640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f55300cad70 0x7f553006b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40294/0 (socket says 172.21.15.115:40294) 2023-12-01T02:51:49.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.545+0000 7f55349e5640 1 -- 172.21.15.115:0/236413217 learned_addr learned my addr 172.21.15.115:0/236413217 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:49.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.545+0000 7f55359e7640 1 --2- 172.21.15.115:0/236413217 >> v2:172.21.15.142:3300/0 conn(0x7f553006f2f0 0x7f553006bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.545+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506341516 0 0) 0x7f55300add10 con 0x7f55300cad70 2023-12-01T02:51:49.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.545+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f551c003520 con 0x7f55300cad70 2023-12-01T02:51:49.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 273197163 0 0) 0x7f551c003520 con 0x7f55300cad70 2023-12-01T02:51:49.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 >> v2:172.21.15.142:3300/0 conn(0x7f553006f2f0 msgr2=0x7f553006bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:49.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 --2- 172.21.15.115:0/236413217 >> v2:172.21.15.142:3300/0 conn(0x7f553006f2f0 0x7f553006bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 >> v1:172.21.15.142:6789/0 conn(0x7f553015e2f0 legacy=0x7f55300ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55300cf7b0 con 0x7f55300cad70 2023-12-01T02:51:49.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5520001690 con 0x7f55300cad70 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 >> v1:172.21.15.115:6789/0 conn(0x7f55300cad70 legacy=0x7f553006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 --2- 172.21.15.115:0/236413217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f551c003a10 0x7f551c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 --2- 172.21.15.115:0/236413217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f551c0047c0 0x7f551c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 --2- 172.21.15.115:0/236413217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f551c005380 0x7f551c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553015e6d0 con 0x7f551c0047c0 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c00c1f0 con 0x7f551c003a10 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c00c3b0 con 0x7f551c005380 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f55359e7640 1 --2- 172.21.15.115:0/236413217 >> v2:172.21.15.142:3300/0 conn(0x7f553006f2f0 0x7f553006bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.546+0000 7f55349e5640 1 --2- 172.21.15.115:0/236413217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f551c0047c0 0x7f551c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.547+0000 7f55349e5640 1 -- 172.21.15.115:0/236413217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f551c005380 msgr2=0x7f551c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.547+0000 7f55349e5640 1 --2- 172.21.15.115:0/236413217 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f551c005380 0x7f551c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.547+0000 7f55349e5640 1 -- 172.21.15.115:0/236413217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f551c003a10 msgr2=0x7f551c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.547+0000 7f55349e5640 1 --2- 172.21.15.115:0/236413217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f551c003a10 0x7f551c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.547+0000 7f55349e5640 1 -- 172.21.15.115:0/236413217 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c003520 con 0x7f551c0047c0 2023-12-01T02:51:49.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.547+0000 7f55359e7640 1 -- 172.21.15.115:0/236413217 reap_dead start 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.548+0000 7f55349e5640 1 --2- 172.21.15.115:0/236413217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f551c0047c0 0x7f551c004c40 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f55300adea0 tx=0x7f551c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b26f12dc6ac9ecd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.548+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520005ad0 con 0x7f551c0047c0 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.548+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520005c30 con 0x7f551c0047c0 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.548+0000 7f5527fff640 1 -- 172.21.15.115:0/236413217 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552002c850 con 0x7f551c0047c0 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.549+0000 7f553744a640 1 -- 172.21.15.115:0/236413217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f551c0047c0 msgr2=0x7f551c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.549+0000 7f553744a640 1 --2- 172.21.15.115:0/236413217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f551c0047c0 0x7f551c004c40 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f55300adea0 tx=0x7f551c0057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.549+0000 7f553744a640 1 -- 172.21.15.115:0/236413217 shutdown_connections 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.549+0000 7f553744a640 1 --2- 172.21.15.115:0/236413217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f551c0047c0 0x7f551c004c40 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.549+0000 7f553744a640 1 -- 172.21.15.115:0/236413217 >> 172.21.15.115:0/236413217 conn(0x7f55300ce880 msgr2=0x7f55300cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.549+0000 7f553744a640 1 -- 172.21.15.115:0/236413217 shutdown_connections 2023-12-01T02:51:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.549+0000 7f553744a640 1 -- 172.21.15.115:0/236413217 wait complete. 2023-12-01T02:51:49.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.551+0000 7f553744a640 1 Processor -- start 2023-12-01T02:51:49.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.551+0000 7f553744a640 1 -- start start 2023-12-01T02:51:49.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.552+0000 7f553744a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f551c0047c0 0x7f55301f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.552+0000 7f553744a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55300c59c0 0x7f55300c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.552+0000 7f553744a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300c7030 0x7f55301f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.552+0000 7f553744a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55300c88b0 con 0x7f55300c7030 2023-12-01T02:51:49.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.552+0000 7f553744a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553015a680 con 0x7f55300c59c0 2023-12-01T02:51:49.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.552+0000 7f553744a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55301f1550 con 0x7f551c0047c0 2023-12-01T02:51:49.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.552+0000 7f55359e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300c7030 0x7f55301f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.552+0000 7f55359e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300c7030 0x7f55301f17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45616/0 (socket says 172.21.15.115:45616) 2023-12-01T02:51:49.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.552+0000 7f55359e7640 1 -- 172.21.15.115:0/530093430 learned_addr learned my addr 172.21.15.115:0/530093430 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:49.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f55349e5640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55300c59c0 0x7f55300c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f55351e6640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f551c0047c0 0x7f55301f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f55359e7640 1 -- 172.21.15.115:0/530093430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f551c0047c0 msgr2=0x7f55301f1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f55359e7640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f551c0047c0 0x7f55301f1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f55359e7640 1 -- 172.21.15.115:0/530093430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55300c59c0 msgr2=0x7f55300c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f55359e7640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55300c59c0 0x7f55300c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f55359e7640 1 -- 172.21.15.115:0/530093430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55300c82f0 con 0x7f55300c7030 2023-12-01T02:51:49.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f55349e5640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55300c59c0 0x7f55300c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:49.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f55359e7640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300c7030 0x7f55301f17a0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f553006ef30 tx=0x7f552800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:49.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f5525ffb640 1 -- 172.21.15.115:0/530093430 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528010040 con 0x7f55300c7030 2023-12-01T02:51:49.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f5525ffb640 1 -- 172.21.15.115:0/530093430 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552800bcb0 con 0x7f55300c7030 2023-12-01T02:51:49.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.553+0000 7f553744a640 1 -- 172.21.15.115:0/530093430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55300c85d0 con 0x7f55300c7030 2023-12-01T02:51:49.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.554+0000 7f5525ffb640 1 -- 172.21.15.115:0/530093430 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528014640 con 0x7f55300c7030 2023-12-01T02:51:49.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.554+0000 7f553744a640 1 -- 172.21.15.115:0/530093430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55300c65d0 con 0x7f55300c7030 2023-12-01T02:51:49.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.555+0000 7f553744a640 1 -- 172.21.15.115:0/530093430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f553006fda0 con 0x7f55300c7030 2023-12-01T02:51:49.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.556+0000 7f5525ffb640 1 -- 172.21.15.115:0/530093430 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f55280147a0 con 0x7f55300c7030 2023-12-01T02:51:49.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.556+0000 7f5525ffb640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f551406a550 0x7f551406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:49.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.557+0000 7f5525ffb640 1 -- 172.21.15.115:0/530093430 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f5528089040 con 0x7f55300c7030 2023-12-01T02:51:49.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.557+0000 7f55351e6640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f551406a550 0x7f551406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:49.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.557+0000 7f55351e6640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f551406a550 0x7f551406ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f552c001590 tx=0x7f552c009040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:49.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.561+0000 7f5525ffb640 1 -- 172.21.15.115:0/530093430 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5528057e70 con 0x7f55300c7030 2023-12-01T02:51:49.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.788+0000 7f553744a640 1 -- 172.21.15.115:0/530093430 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f553004e890 con 0x7f55300c7030 2023-12-01T02:51:49.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.789+0000 7f5525ffb640 1 -- 172.21.15.115:0/530093430 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f5528057a00 con 0x7f55300c7030 2023-12-01T02:51:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:51:49.791 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1952979218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1952979218},{"type":"v1","addr":"172.21.15.142:6833","nonce":1952979218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3618529680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3618529680},{"type":"v1","addr":"172.21.15.142:6835","nonce":3618529680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":9009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1637880399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1637880399},{"type":"v1","addr":"172.21.15.115:6836","nonce":1637880399}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:51:43.827542+0000","modified":"2023-12-01T02:51:46.847110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3770885942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3770885942},{"type":"v1","addr":"172.21.15.115:6837","nonce":3770885942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-01T02:51:49.791 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 71 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.791+0000 7f553744a640 1 -- 172.21.15.115:0/530093430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f551406a550 msgr2=0x7f551406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.791+0000 7f553744a640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f551406a550 0x7f551406ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f552c001590 tx=0x7f552c009040 comp rx=0 tx=0).stop 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.791+0000 7f553744a640 1 -- 172.21.15.115:0/530093430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300c7030 msgr2=0x7f55301f17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.791+0000 7f553744a640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300c7030 0x7f55301f17a0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f553006ef30 tx=0x7f552800b480 comp rx=0 tx=0).stop 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.792+0000 7f553744a640 1 -- 172.21.15.115:0/530093430 shutdown_connections 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.792+0000 7f553744a640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f551406a550 0x7f551406ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.792+0000 7f553744a640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55300c7030 0x7f55301f17a0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.792+0000 7f553744a640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55300c59c0 0x7f55300c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.792+0000 7f553744a640 1 --2- 172.21.15.115:0/530093430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f551c0047c0 0x7f55301f1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.792+0000 7f553744a640 1 -- 172.21.15.115:0/530093430 >> 172.21.15.115:0/530093430 conn(0x7f55300ce880 msgr2=0x7f553015ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.792+0000 7f553744a640 1 -- 172.21.15.115:0/530093430 shutdown_connections 2023-12-01T02:51:49.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:49.792+0000 7f553744a640 1 -- 172.21.15.115:0/530093430 wait complete. 2023-12-01T02:51:50.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T02:51:43.827542+0000', 'modified': '2023-12-01T02:51:46.847110+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9012}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9012': {'gid': 9012, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6837/3770885942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6835', 'nonce': 3770885942}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 3770885942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T02:51:50.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T02:51:50.215 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-01T02:51:50.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.438+0000 7f9097188640 1 Processor -- start 2023-12-01T02:51:50.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.439+0000 7f9097188640 1 -- start start 2023-12-01T02:51:50.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.440+0000 7f9097188640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f90900b5d80 0x7f90900d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.440+0000 7f9097188640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90900ce2e0 con 0x7f90900d1230 2023-12-01T02:51:50.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.440+0000 7f9097188640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90900ce000 con 0x7f909015cd20 2023-12-01T02:51:50.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.440+0000 7f9097188640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90900b2160 con 0x7f90900b5d80 2023-12-01T02:51:50.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.440+0000 7f908ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f90900b5d80 0x7f90900d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.440+0000 7f9095725640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f90900d1230 0x7f90900d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40310/0 (socket says 172.21.15.115:40310) 2023-12-01T02:51:50.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.441+0000 7f9095725640 1 -- 172.21.15.115:0/895048323 learned_addr learned my addr 172.21.15.115:0/895048323 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:50.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.441+0000 7f908ffff640 1 -- 172.21.15.115:0/895048323 >> v1:172.21.15.142:6789/0 conn(0x7f909015cd20 legacy=0x7f90900b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.441+0000 7f908ffff640 1 -- 172.21.15.115:0/895048323 >> v1:172.21.15.115:6789/0 conn(0x7f90900d1230 legacy=0x7f90900d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.441+0000 7f908ffff640 1 -- 172.21.15.115:0/895048323 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900cd3d0 con 0x7f90900b5d80 2023-12-01T02:51:50.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.442+0000 7f908ffff640 1 --2- 172.21.15.115:0/895048323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90900b5d80 0x7f90900d24d0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f90780097a0 tx=0x7f907802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db740ad5d12e17 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:50.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.442+0000 7f908f7fe640 1 -- 172.21.15.115:0/895048323 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907802e070 con 0x7f90900b5d80 2023-12-01T02:51:50.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.442+0000 7f908f7fe640 1 -- 172.21.15.115:0/895048323 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90780315a0 con 0x7f90900b5d80 2023-12-01T02:51:50.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.442+0000 7f908f7fe640 1 -- 172.21.15.115:0/895048323 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9078031930 con 0x7f90900b5d80 2023-12-01T02:51:50.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.443+0000 7f9097188640 1 -- 172.21.15.115:0/895048323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90900b5d80 msgr2=0x7f90900d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.443+0000 7f9097188640 1 --2- 172.21.15.115:0/895048323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90900b5d80 0x7f90900d24d0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f90780097a0 tx=0x7f907802b840 comp rx=0 tx=0).stop 2023-12-01T02:51:50.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.443+0000 7f9097188640 1 -- 172.21.15.115:0/895048323 shutdown_connections 2023-12-01T02:51:50.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.443+0000 7f9097188640 1 --2- 172.21.15.115:0/895048323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90900b5d80 0x7f90900d24d0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.443+0000 7f9097188640 1 -- 172.21.15.115:0/895048323 >> 172.21.15.115:0/895048323 conn(0x7f909006a0e0 msgr2=0x7f909006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:50.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.443+0000 7f9097188640 1 -- 172.21.15.115:0/895048323 shutdown_connections 2023-12-01T02:51:50.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.443+0000 7f9097188640 1 -- 172.21.15.115:0/895048323 wait complete. 2023-12-01T02:51:50.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.445+0000 7f9097188640 1 Processor -- start 2023-12-01T02:51:50.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.445+0000 7f9097188640 1 -- start start 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9097188640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90900b5d80 0x7f90900c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9097188640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90900d1230 0x7f90901423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9097188640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f909015cd20 0x7f90900d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9097188640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90900c6560 con 0x7f909015cd20 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9097188640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901428e0 con 0x7f90900d1230 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9097188640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90900c8f20 con 0x7f90900b5d80 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9095725640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f909015cd20 0x7f90900d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9095725640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f909015cd20 0x7f90900d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45620/0 (socket says 172.21.15.115:45620) 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9095725640 1 -- 172.21.15.115:0/2010825522 learned_addr learned my addr 172.21.15.115:0/2010825522 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9094f24640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90900b5d80 0x7f90900c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9095725640 1 -- 172.21.15.115:0/2010825522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90900b5d80 msgr2=0x7f90900c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9095725640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90900b5d80 0x7f90900c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9095725640 1 -- 172.21.15.115:0/2010825522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90900d1230 msgr2=0x7f90901423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9095725640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90900d1230 0x7f90901423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.446+0000 7f9095725640 1 -- 172.21.15.115:0/2010825522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9078009400 con 0x7f909015cd20 2023-12-01T02:51:50.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.447+0000 7f9095725640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f909015cd20 0x7f90900d1650 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f908800c420 tx=0x7f908800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:50.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.447+0000 7f908d7fa640 1 -- 172.21.15.115:0/2010825522 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088005ce0 con 0x7f909015cd20 2023-12-01T02:51:50.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.447+0000 7f908d7fa640 1 -- 172.21.15.115:0/2010825522 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9088005e40 con 0x7f909015cd20 2023-12-01T02:51:50.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.447+0000 7f908d7fa640 1 -- 172.21.15.115:0/2010825522 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90880072d0 con 0x7f909015cd20 2023-12-01T02:51:50.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.447+0000 7f9097188640 1 -- 172.21.15.115:0/2010825522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90900c9200 con 0x7f909015cd20 2023-12-01T02:51:50.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.447+0000 7f9097188640 1 -- 172.21.15.115:0/2010825522 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90900b40d0 con 0x7f909015cd20 2023-12-01T02:51:50.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.450+0000 7f908d7fa640 1 -- 172.21.15.115:0/2010825522 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f90880074f0 con 0x7f909015cd20 2023-12-01T02:51:50.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.451+0000 7f908d7fa640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f907006a550 0x7f907006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.451+0000 7f908d7fa640 1 -- 172.21.15.115:0/2010825522 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f9088087f50 con 0x7f909015cd20 2023-12-01T02:51:50.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.451+0000 7f9094f24640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f907006a550 0x7f907006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.452+0000 7f9099dcf740 1 Processor -- start 2023-12-01T02:51:50.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.452+0000 7f9099dcf740 1 -- start start 2023-12-01T02:51:50.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.452+0000 7f9094f24640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f907006a550 0x7f907006ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f90901f5850 tx=0x7f90800072f0 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:50.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 0x558001e02d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001dd28a0 0x558001dffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001e000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9099dcf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001dae2f0 con 0x558001dd64a0 2023-12-01T02:51:50.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9099dcf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001d42f80 con 0x558001e02920 2023-12-01T02:51:50.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9099dcf740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x558001d40710 con 0x558001dd28a0 2023-12-01T02:51:50.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9095725640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001e000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9095725640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001e000d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45630/0 (socket says 172.21.15.115:45630) 2023-12-01T02:51:50.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9095725640 1 -- 172.21.15.115:0/3605542911 learned_addr learned my addr 172.21.15.115:0/3605542911 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:50.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9094f24640 1 --2- 172.21.15.115:0/3605542911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 0x558001e02d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9095725640 1 -- 172.21.15.115:0/3605542911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001dd28a0 msgr2=0x558001dffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9095725640 1 --2- 172.21.15.115:0/3605542911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001dd28a0 0x558001dffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9095725640 1 -- 172.21.15.115:0/3605542911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 msgr2=0x558001e02d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9095725640 1 --2- 172.21.15.115:0/3605542911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 0x558001e02d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9095725640 1 -- 172.21.15.115:0/3605542911 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558001d03690 con 0x558001dd64a0 2023-12-01T02:51:50.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.453+0000 7f9094f24640 1 --2- 172.21.15.115:0/3605542911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 0x558001e02d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:50.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.454+0000 7f9095725640 1 --2- 172.21.15.115:0/3605542911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001e000d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f9088053660 tx=0x7f9088057220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f30ef2cee9b6eb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:50.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.454+0000 7f90867fc640 1 -- 172.21.15.115:0/3605542911 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90880863a0 con 0x558001dd64a0 2023-12-01T02:51:50.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.454+0000 7f90867fc640 1 -- 172.21.15.115:0/3605542911 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9088086500 con 0x558001dd64a0 2023-12-01T02:51:50.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.454+0000 7f90867fc640 1 -- 172.21.15.115:0/3605542911 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908805ec00 con 0x558001dd64a0 2023-12-01T02:51:50.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.455+0000 7f9099dcf740 1 -- 172.21.15.115:0/3605542911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 msgr2=0x558001e000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.455+0000 7f9099dcf740 1 --2- 172.21.15.115:0/3605542911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001e000d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f9088053660 tx=0x7f9088057220 comp rx=0 tx=0).stop 2023-12-01T02:51:50.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.455+0000 7f9099dcf740 1 -- 172.21.15.115:0/3605542911 shutdown_connections 2023-12-01T02:51:50.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.455+0000 7f9099dcf740 1 --2- 172.21.15.115:0/3605542911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001dd28a0 0x558001dffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.455+0000 7f9099dcf740 1 --2- 172.21.15.115:0/3605542911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001e000d0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.455+0000 7f9099dcf740 1 --2- 172.21.15.115:0/3605542911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 0x558001e02d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.455+0000 7f9099dcf740 1 -- 172.21.15.115:0/3605542911 >> 172.21.15.115:0/3605542911 conn(0x558001dd4bb0 msgr2=0x558001e1d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:50.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.455+0000 7f9099dcf740 1 -- 172.21.15.115:0/3605542911 shutdown_connections 2023-12-01T02:51:50.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.455+0000 7f9099dcf740 1 -- 172.21.15.115:0/3605542911 wait complete. 2023-12-01T02:51:50.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.456+0000 7f9099dcf740 1 -- start start 2023-12-01T02:51:50.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.456+0000 7f9099dcf740 1 Processor -- start 2023-12-01T02:51:50.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001dd64a0 0x558001df8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001e02920 0x558001e1cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dec860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9099dcf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001d3c000 con 0x558001e1d120 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9099dcf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001cdd7d0 con 0x558001dd64a0 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9099dcf740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x558001cdd470 con 0x558001e02920 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9099dcf740 10 client.0 _ll_register_callbacks cb 0x558001d2c1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9095725640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9095725640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dec860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45634/0 (socket says 172.21.15.115:45634) 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9095725640 1 -- 172.21.15.115:0/4100255900 learned_addr learned my addr 172.21.15.115:0/4100255900 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9094f24640 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001dd64a0 0x558001df8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9095725640 1 -- 172.21.15.115:0/4100255900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001e02920 msgr2=0x558001e1cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9095725640 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001e02920 0x558001e1cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9095725640 1 -- 172.21.15.115:0/4100255900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001dd64a0 msgr2=0x558001df8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9095725640 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001dd64a0 0x558001df8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9095725640 1 -- 172.21.15.115:0/4100255900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90900b40d0 con 0x558001e1d120 2023-12-01T02:51:50.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.457+0000 7f9094f24640 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001dd64a0 0x558001df8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f9095725640 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dec860 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f90880554b0 tx=0x7f908800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27349d5d41bcbf65 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f906f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f906f7fe640 1 -- 172.21.15.115:0/4100255900 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088086070 con 0x558001e1d120 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f906f7fe640 1 -- 172.21.15.115:0/4100255900 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f908806aae0 con 0x558001e1d120 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f9099dcf740 1 -- 172.21.15.115:0/4100255900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558001d66550 con 0x558001e1d120 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f906f7fe640 1 -- 172.21.15.115:0/4100255900 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f908806aca0 con 0x558001e1d120 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f9099dcf740 10 client.9168 fetch_fsmap learned FSMap version 71 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f9099dcf740 1 -- 172.21.15.115:0/4100255900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x558001d3a680 con 0x558001e1d120 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f906f7fe640 1 -- 172.21.15.115:0/4100255900 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558001d66550 con 0x558001e1d120 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f906f7fe640 1 -- 172.21.15.115:0/4100255900 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 71) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9088052f50 con 0x558001e1d120 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.458+0000 7f9099dcf740 10 client.9168 fetch_fsmap finished waiting for FSMap version 71 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.459+0000 7f9099dcf740 10 client.9168 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.459+0000 7f9099dcf740 20 client.9168 populate_metadata read hostname 'smithi115' 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.459+0000 7f9099dcf740 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.115:6835/3770885942,v1:172.21.15.115:6837/3770885942] conn(0x558001dfd430 0x558001dfabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.459+0000 7f9099dcf740 4 client.9168 mds_command: new command op to 9012 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T02:51:50.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.459+0000 7f9099dcf740 1 -- 172.21.15.115:0/4100255900 --> [v2:172.21.15.115:6835/3770885942,v1:172.21.15.115:6837/3770885942] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558001d2eb50 con 0x558001dfd430 2023-12-01T02:51:50.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.459+0000 7f9094f24640 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.115:6835/3770885942,v1:172.21.15.115:6837/3770885942] conn(0x558001dfd430 0x558001dfabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.459+0000 7f9094f24640 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.115:6835/3770885942,v1:172.21.15.115:6837/3770885942] conn(0x558001dfd430 0x558001dfabb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71fcf0416210b49a server_cookie=e5db317b1bb1fc77 in_seq=0 out_seq=0 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.459+0000 7f906f7fe640 10 client.9168 ms_handle_connect on v2:172.21.15.115:6835/3770885942 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f906f7fe640 1 -- 172.21.15.115:0/4100255900 <== mds.0 v2:172.21.15.115:6835/3770885942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558001d2eb50 con 0x558001dfd430 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f906f7fe640 10 client.9168 handle_command_reply: tid=0 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f9099dcf740 1 client.9168 shutdown 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f9099dcf740 1 -- 172.21.15.115:0/4100255900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 msgr2=0x558001dec860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f9099dcf740 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dec860 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f90880554b0 tx=0x7f908800cf90 comp rx=0 tx=0).stop 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f9099dcf740 1 -- 172.21.15.115:0/4100255900 shutdown_connections 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f9099dcf740 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.115:6835/3770885942,v1:172.21.15.115:6837/3770885942] conn(0x558001dfd430 0x558001dfabb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f9099dcf740 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001e02920 0x558001e1cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f9099dcf740 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dec860 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f9099dcf740 1 --2- 172.21.15.115:0/4100255900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001dd64a0 0x558001df8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f906f7fe640 0 client.9168 ms_handle_reset on v2:172.21.15.115:6835/3770885942 2023-12-01T02:51:50.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.460+0000 7f9099dcf740 1 -- 172.21.15.115:0/4100255900 >> 172.21.15.115:0/4100255900 conn(0x558001dd28a0 msgr2=0x558001dd6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:50.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.461+0000 7f9099dcf740 1 -- 172.21.15.115:0/4100255900 shutdown_connections 2023-12-01T02:51:50.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.461+0000 7f9099dcf740 1 -- 172.21.15.115:0/4100255900 wait complete. 2023-12-01T02:51:50.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.461+0000 7f9099dcf740 20 client.9168 trim_cache size 0 max 16384 2023-12-01T02:51:50.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.477+0000 7f9099dcf740 1 Processor -- start 2023-12-01T02:51:50.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.477+0000 7f9099dcf740 1 -- start start 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001dd1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001e02920 0x558001e36cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e1d120 0x558001e371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9099dcf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001aa3f20 con 0x558001dd64a0 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9099dcf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001dfc580 con 0x558001e1d120 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9099dcf740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x558001dd43b0 con 0x558001e02920 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9094f24640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001dd1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9094f24640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001dd1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45650/0 (socket says 172.21.15.115:45650) 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9094f24640 1 -- 172.21.15.115:0/2817285489 learned_addr learned my addr 172.21.15.115:0/2817285489 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9094f24640 1 -- 172.21.15.115:0/2817285489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001e02920 msgr2=0x558001e36cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9094f24640 1 --2- 172.21.15.115:0/2817285489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001e02920 0x558001e36cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9094f24640 1 -- 172.21.15.115:0/2817285489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e1d120 msgr2=0x558001e371f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9094f24640 1 --2- 172.21.15.115:0/2817285489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e1d120 0x558001e371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.478+0000 7f9094f24640 1 -- 172.21.15.115:0/2817285489 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558001e1bc70 con 0x558001dd64a0 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.479+0000 7f9094f24640 1 --2- 172.21.15.115:0/2817285489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001dd1b10 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f908000c9f0 tx=0x7f9080012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9eb051b818fdbb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.479+0000 7f9086ffd640 1 -- 172.21.15.115:0/2817285489 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080018040 con 0x558001dd64a0 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.479+0000 7f9086ffd640 1 -- 172.21.15.115:0/2817285489 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9080019040 con 0x558001dd64a0 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.479+0000 7f9086ffd640 1 -- 172.21.15.115:0/2817285489 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90800139e0 con 0x558001dd64a0 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.479+0000 7f9099dcf740 1 -- 172.21.15.115:0/2817285489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 msgr2=0x558001dd1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.480+0000 7f9099dcf740 1 --2- 172.21.15.115:0/2817285489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001dd1b10 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f908000c9f0 tx=0x7f9080012000 comp rx=0 tx=0).stop 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.480+0000 7f9099dcf740 1 -- 172.21.15.115:0/2817285489 shutdown_connections 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.480+0000 7f9099dcf740 1 --2- 172.21.15.115:0/2817285489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001e02920 0x558001e36cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.480+0000 7f9099dcf740 1 --2- 172.21.15.115:0/2817285489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e1d120 0x558001e371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.480+0000 7f9099dcf740 1 --2- 172.21.15.115:0/2817285489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001dd64a0 0x558001dd1b10 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.480+0000 7f9099dcf740 1 -- 172.21.15.115:0/2817285489 >> 172.21.15.115:0/2817285489 conn(0x558001dd28a0 msgr2=0x558001dce040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:50.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.480+0000 7f9099dcf740 1 -- 172.21.15.115:0/2817285489 shutdown_connections 2023-12-01T02:51:50.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.480+0000 7f9099dcf740 1 -- 172.21.15.115:0/2817285489 wait complete. 2023-12-01T02:51:50.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.480+0000 7f9099dcf740 1 -- start start 2023-12-01T02:51:50.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.481+0000 7f9099dcf740 1 Processor -- start 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.481+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001dd64a0 0x558001dfd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.481+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 0x558001dd1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9099dcf740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dd2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9099dcf740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001e1fbc0 con 0x558001e1d120 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9099dcf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001e1fea0 con 0x558001e02920 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9099dcf740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x558001e20180 con 0x558001dd64a0 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9099dcf740 10 client.0 _ll_register_callbacks cb 0x7f90880534a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9095725640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dd2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9095725640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dd2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45664/0 (socket says 172.21.15.115:45664) 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9095725640 1 -- 172.21.15.115:0/2357088373 learned_addr learned my addr 172.21.15.115:0/2357088373 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f908ffff640 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 0x558001dd1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9094f24640 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001dd64a0 0x558001dfd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9095725640 1 -- 172.21.15.115:0/2357088373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001dd64a0 msgr2=0x558001dfd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9095725640 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001dd64a0 0x558001dfd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9095725640 1 -- 172.21.15.115:0/2357088373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 msgr2=0x558001dd1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9095725640 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 0x558001dd1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.482+0000 7f9095725640 1 -- 172.21.15.115:0/2357088373 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90800079f0 con 0x558001e1d120 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f9095725640 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dd2050 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f90880570f0 tx=0x7f9088000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe6994f591fc7b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f906f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f906f7fe640 1 -- 172.21.15.115:0/2357088373 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90880532d0 con 0x558001e1d120 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f906f7fe640 1 -- 172.21.15.115:0/2357088373 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f908805aee0 con 0x558001e1d120 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f9099dcf740 1 -- 172.21.15.115:0/2357088373 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9088086500 con 0x558001e1d120 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f906f7fe640 1 -- 172.21.15.115:0/2357088373 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9088056ac0 con 0x558001e1d120 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f9099dcf740 10 client.9174 fetch_fsmap learned FSMap version 71 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f9099dcf740 1 -- 172.21.15.115:0/2357088373 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x558001cdc9e0 con 0x558001e1d120 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f906f7fe640 1 -- 172.21.15.115:0/2357088373 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9088086500 con 0x558001e1d120 2023-12-01T02:51:50.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f906f7fe640 1 -- 172.21.15.115:0/2357088373 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 71) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f908807d020 con 0x558001e1d120 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f9099dcf740 10 client.9174 fetch_fsmap finished waiting for FSMap version 71 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f9099dcf740 10 client.9174 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f9099dcf740 20 client.9174 populate_metadata read hostname 'smithi115' 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f9099dcf740 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.115:6835/3770885942,v1:172.21.15.115:6837/3770885942] conn(0x558001dfac70 0x558001dfb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f9099dcf740 4 client.9174 mds_command: new command op to 9012 tid=0[{"prefix": "status"}] 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.483+0000 7f9099dcf740 1 -- 172.21.15.115:0/2357088373 --> [v2:172.21.15.115:6835/3770885942,v1:172.21.15.115:6837/3770885942] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558001d35080 con 0x558001dfac70 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.484+0000 7f9094f24640 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.115:6835/3770885942,v1:172.21.15.115:6837/3770885942] conn(0x558001dfac70 0x558001dfb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.484+0000 7f9094f24640 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.115:6835/3770885942,v1:172.21.15.115:6837/3770885942] conn(0x558001dfac70 0x558001dfb050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f265fbafc1499ba7 server_cookie=5c9906d012ba0f5a in_seq=0 out_seq=0 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.484+0000 7f906f7fe640 10 client.9174 ms_handle_connect on v2:172.21.15.115:6835/3770885942 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.484+0000 7f906f7fe640 1 -- 172.21.15.115:0/2357088373 <== mds.0 v2:172.21.15.115:6835/3770885942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x558001d35080 con 0x558001dfac70 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.484+0000 7f906f7fe640 10 client.9174 handle_command_reply: tid=0 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.484+0000 7f9099dcf740 1 client.9174 shutdown 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.484+0000 7f9099dcf740 1 -- 172.21.15.115:0/2357088373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 msgr2=0x558001dd2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.484+0000 7f9099dcf740 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dd2050 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f90880570f0 tx=0x7f9088000ed0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9099dcf740 1 -- 172.21.15.115:0/2357088373 shutdown_connections 2023-12-01T02:51:50.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9099dcf740 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.115:6835/3770885942,v1:172.21.15.115:6837/3770885942] conn(0x558001dfac70 0x558001dfb050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9099dcf740 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x558001e1d120 0x558001dd2050 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9099dcf740 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558001e02920 0x558001dd1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9099dcf740 1 --2- 172.21.15.115:0/2357088373 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558001dd64a0 0x558001dfd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9099dcf740 1 -- 172.21.15.115:0/2357088373 >> 172.21.15.115:0/2357088373 conn(0x558001dd28a0 msgr2=0x558001dd7770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f906f7fe640 0 client.9174 ms_handle_reset on v2:172.21.15.115:6835/3770885942 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9099dcf740 1 -- 172.21.15.115:0/2357088373 shutdown_connections 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9099dcf740 1 -- 172.21.15.115:0/2357088373 wait complete. 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9099dcf740 20 client.9174 trim_cache size 0 max 16384 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 9012, 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T02:51:50.490 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T02:51:50.491 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T02:51:50.491 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 6.6533539429999999, 2023-12-01T02:51:50.491 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 71, 2023-12-01T02:51:50.491 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 163, 2023-12-01T02:51:50.491 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 163, 2023-12-01T02:51:50.491 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 12.532255621999999 2023-12-01T02:51:50.491 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9097188640 1 -- 172.21.15.115:0/2010825522 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f907006a550 msgr2=0x7f907006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9097188640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f907006a550 0x7f907006ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f90901f5850 tx=0x7f90800072f0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9097188640 1 -- 172.21.15.115:0/2010825522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f909015cd20 msgr2=0x7f90900d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.485+0000 7f9097188640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f909015cd20 0x7f90900d1650 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f908800c420 tx=0x7f908800c8e0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.486+0000 7f9097188640 1 -- 172.21.15.115:0/2010825522 shutdown_connections 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.486+0000 7f9097188640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f907006a550 0x7f907006ca10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.486+0000 7f9097188640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f909015cd20 0x7f90900d1650 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.486+0000 7f9097188640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90900d1230 0x7f90901423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.486+0000 7f9097188640 1 --2- 172.21.15.115:0/2010825522 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90900b5d80 0x7f90900c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.486+0000 7f9097188640 1 -- 172.21.15.115:0/2010825522 >> 172.21.15.115:0/2010825522 conn(0x7f909006a0e0 msgr2=0x7f909015ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.486+0000 7f9097188640 1 -- 172.21.15.115:0/2010825522 shutdown_connections 2023-12-01T02:51:50.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.486+0000 7f9097188640 1 -- 172.21.15.115:0/2010825522 wait complete. 2023-12-01T02:51:50.504 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T02:51:50.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.710+0000 7f96cffff640 1 Processor -- start 2023-12-01T02:51:50.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.711+0000 7f96cffff640 1 -- start start 2023-12-01T02:51:50.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.713+0000 7f96cffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96d006b4e0 0x7f96d0161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.713+0000 7f96cffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f96d006b8c0 0x7f96d00cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.713+0000 7f96cffff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d0162530 con 0x7f96d015b0d0 2023-12-01T02:51:50.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.713+0000 7f96cffff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d00caf40 con 0x7f96d006b8c0 2023-12-01T02:51:50.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.713+0000 7f96cffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00cd7f0 con 0x7f96d006b4e0 2023-12-01T02:51:50.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.713+0000 7f96cf7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f96d015b0d0 0x7f96d015b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40314/0 (socket says 172.21.15.115:40314) 2023-12-01T02:51:50.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.713+0000 7f96cf7fe640 1 -- 172.21.15.115:0/219909995 learned_addr learned my addr 172.21.15.115:0/219909995 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:50.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96c67fc640 1 --2- 172.21.15.115:0/219909995 >> v2:172.21.15.142:3300/0 conn(0x7f96d006b8c0 0x7f96d00cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ceffd640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96d006b4e0 0x7f96d0161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908485787 0 0) 0x7f96d0162530 con 0x7f96d015b0d0 2023-12-01T02:51:50.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96bc003520 con 0x7f96d015b0d0 2023-12-01T02:51:50.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1798556665 0 0) 0x7f96bc003520 con 0x7f96d015b0d0 2023-12-01T02:51:50.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96d006b4e0 msgr2=0x7f96d0161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96d006b4e0 0x7f96d0161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 >> v2:172.21.15.142:3300/0 conn(0x7f96d006b8c0 msgr2=0x7f96d00cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 --2- 172.21.15.115:0/219909995 >> v2:172.21.15.142:3300/0 conn(0x7f96d006b8c0 0x7f96d00cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00ca3c0 con 0x7f96d015b0d0 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f96c80019f0 con 0x7f96d015b0d0 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 >> v1:172.21.15.115:6789/0 conn(0x7f96d015b0d0 legacy=0x7f96d015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96d006b4e0 0x7f96bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96bc003f60 0x7f96bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc004d00 0x7f96bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00caf40 con 0x7f96bc004d00 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00cd7f0 con 0x7f96bc003f60 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.714+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc00b9b0 con 0x7f96d006b4e0 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.715+0000 7f96cf7fe640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc004d00 0x7f96bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.715+0000 7f96ceffd640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96d006b4e0 0x7f96bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.715+0000 7f96c67fc640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96bc003f60 0x7f96bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.715+0000 7f96cf7fe640 1 -- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96d006b4e0 msgr2=0x7f96bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.715+0000 7f96cf7fe640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96d006b4e0 0x7f96bc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.715+0000 7f96cf7fe640 1 -- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96bc003f60 msgr2=0x7f96bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:50.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.715+0000 7f96cf7fe640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96bc003f60 0x7f96bc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.715+0000 7f96cf7fe640 1 -- 172.21.15.115:0/219909995 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc003520 con 0x7f96bc004d00 2023-12-01T02:51:50.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.715+0000 7f96ceffd640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96d006b4e0 0x7f96bc0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.716+0000 7f96cf7fe640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc004d00 0x7f96bc005180 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f96c800a860 tx=0x7f96bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e22b4d4d8a3c9d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.716+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8034630 con 0x7f96bc004d00 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.716+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c802f070 con 0x7f96bc004d00 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.716+0000 7f96ce7fc640 1 -- 172.21.15.115:0/219909995 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8034a30 con 0x7f96bc004d00 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.717+0000 7f96cffff640 1 -- 172.21.15.115:0/219909995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc004d00 msgr2=0x7f96bc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.717+0000 7f96cffff640 1 --2- 172.21.15.115:0/219909995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc004d00 0x7f96bc005180 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f96c800a860 tx=0x7f96bc0057f0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.717+0000 7f96cf7fe640 1 -- 172.21.15.115:0/219909995 reap_dead start 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.717+0000 7f96cffff640 1 -- 172.21.15.115:0/219909995 shutdown_connections 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.717+0000 7f96cffff640 1 -- 172.21.15.115:0/219909995 >> 172.21.15.115:0/219909995 conn(0x7f96d00ce640 msgr2=0x7f96d00cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.717+0000 7f96cffff640 1 -- 172.21.15.115:0/219909995 shutdown_connections 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.717+0000 7f96cffff640 1 -- 172.21.15.115:0/219909995 wait complete. 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.719+0000 7f96cffff640 1 Processor -- start 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.719+0000 7f96cffff640 1 -- start start 2023-12-01T02:51:50.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.720+0000 7f96cffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96bc004d00 0x7f96d01f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.720+0000 7f96cffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96d01f7760 0x7f96d00af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.720+0000 7f96cffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d00afa40 0x7f96d00c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.720+0000 7f96cffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d01f7c50 con 0x7f96d01f7760 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.720+0000 7f96cffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d01f7ef0 con 0x7f96d00afa40 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.720+0000 7f96cffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d01f81d0 con 0x7f96bc004d00 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.720+0000 7f96c67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96d01f7760 0x7f96d00af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.720+0000 7f96c67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96d01f7760 0x7f96d00af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45670/0 (socket says 172.21.15.115:45670) 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.720+0000 7f96c67fc640 1 -- 172.21.15.115:0/149958629 learned_addr learned my addr 172.21.15.115:0/149958629 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96cf7fe640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d00afa40 0x7f96d00c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96ceffd640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96bc004d00 0x7f96d01f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96c67fc640 1 -- 172.21.15.115:0/149958629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96bc004d00 msgr2=0x7f96d01f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96c67fc640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96bc004d00 0x7f96d01f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96c67fc640 1 -- 172.21.15.115:0/149958629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d00afa40 msgr2=0x7f96d00c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96c67fc640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d00afa40 0x7f96d00c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96c67fc640 1 -- 172.21.15.115:0/149958629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d01f84b0 con 0x7f96d01f7760 2023-12-01T02:51:50.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96ceffd640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96bc004d00 0x7f96d01f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:50.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96cf7fe640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d00afa40 0x7f96d00c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:50.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96c67fc640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96d01f7760 0x7f96d00af500 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f96b8011600 tx=0x7f96b8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:50.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96c7fff640 1 -- 172.21.15.115:0/149958629 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800f8c0 con 0x7f96d01f7760 2023-12-01T02:51:50.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96c7fff640 1 -- 172.21.15.115:0/149958629 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b800fa20 con 0x7f96d01f7760 2023-12-01T02:51:50.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96c7fff640 1 -- 172.21.15.115:0/149958629 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800fd90 con 0x7f96d01f7760 2023-12-01T02:51:50.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.721+0000 7f96cffff640 1 -- 172.21.15.115:0/149958629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d00aff80 con 0x7f96d01f7760 2023-12-01T02:51:50.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.722+0000 7f96cffff640 1 -- 172.21.15.115:0/149958629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d00c40e0 con 0x7f96d01f7760 2023-12-01T02:51:50.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.723+0000 7f96cffff640 1 -- 172.21.15.115:0/149958629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d00c4360 con 0x7f96d01f7760 2023-12-01T02:51:50.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.724+0000 7f96c7fff640 1 -- 172.21.15.115:0/149958629 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f96b8015460 con 0x7f96d01f7760 2023-12-01T02:51:50.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.725+0000 7f96c7fff640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f96b006a550 0x7f96b006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:50.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.725+0000 7f96c7fff640 1 -- 172.21.15.115:0/149958629 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f96b8088fb0 con 0x7f96d01f7760 2023-12-01T02:51:50.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.725+0000 7f96ceffd640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f96b006a550 0x7f96b006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:50.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.726+0000 7f96ceffd640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f96b006a550 0x7f96b006ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96d00ad710 tx=0x7f96c0008040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:50.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.730+0000 7f96c7fff640 1 -- 172.21.15.115:0/149958629 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f96b80bbc80 con 0x7f96d01f7760 2023-12-01T02:51:50.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.958+0000 7f96cffff640 1 -- 172.21.15.115:0/149958629 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96d01f8640 con 0x7f96d01f7760 2023-12-01T02:51:50.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.959+0000 7f96c7fff640 1 -- 172.21.15.115:0/149958629 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f96b8054460 con 0x7f96d01f7760 2023-12-01T02:51:50.961 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T02:51:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 -- 172.21.15.115:0/149958629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f96b006a550 msgr2=0x7f96b006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f96b006a550 0x7f96b006ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96d00ad710 tx=0x7f96c0008040 comp rx=0 tx=0).stop 2023-12-01T02:51:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 -- 172.21.15.115:0/149958629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96d01f7760 msgr2=0x7f96d00af500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96d01f7760 0x7f96d00af500 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f96b8011600 tx=0x7f96b8011ac0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 -- 172.21.15.115:0/149958629 shutdown_connections 2023-12-01T02:51:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f96b006a550 0x7f96b006ca10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96d00afa40 0x7f96d00c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96d01f7760 0x7f96d00af500 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 --2- 172.21.15.115:0/149958629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96bc004d00 0x7f96d01f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 -- 172.21.15.115:0/149958629 >> 172.21.15.115:0/149958629 conn(0x7f96d00ce640 msgr2=0x7f96d0159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 -- 172.21.15.115:0/149958629 shutdown_connections 2023-12-01T02:51:50.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:50.962+0000 7f96cffff640 1 -- 172.21.15.115:0/149958629 wait complete. 2023-12-01T02:51:50.983 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T02:51:50.983 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T02:51:50.983 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T02:51:50.983 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T02:51:50.983 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T02:51:50.983 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T02:51:50.983 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T02:51:50.983 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T02:51:50.983 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T02:51:50.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:50.984 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T02:51:51.032 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T02:51:51.032 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T02:51:51.032 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T02:51:51.032 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:51:51.032 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 37846sec preferred_lft 37846sec 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T02:51:51.033 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T02:51:51.034 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:51:51.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:51.036 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T02:51:51.116 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T02:51:51.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:51.119 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T02:51:51.171 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T02:51:51.171 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T02:51:51.171 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T02:51:51.171 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T02:51:51.171 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T02:51:51.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:51.171 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T02:51:51.172 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T02:51:51.172 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T02:51:51.172 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T02:51:51.172 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T02:51:51.172 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T02:51:51.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:51.347 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T02:51:51.399 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T02:51:51.399 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T02:51:51.400 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T02:51:51.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:51.400 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T02:51:51.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T02:51:51.492 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T02:51:51.546 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T02:51:51.546 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T02:51:51.546 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T02:51:51.598 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T02:51:51.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:51.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T02:51:51.730 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T02:51:51.730 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T02:51:51.730 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T02:51:51.753 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T02:51:51.753 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T02:51:51.753 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T02:51:51.753 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T02:51:51.753 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T02:51:51.776 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T02:51:51.777 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T02:51:51.777 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T02:51:51.777 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T02:51:51.777 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T02:51:51.777 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T02:51:51.777 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T02:51:51.777 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T02:51:51.779 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T02:51:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T02:51:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T02:51:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T02:51:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T02:51:51.783 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T02:51:51.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:51.783 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T02:51:51.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:51.872 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T02:51:51.938 INFO:teuthology.orchestra.run.smithi115.stdout:client9183 2023-12-01T02:51:51.938 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:51:51.938 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:51:51.988 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T02:51:52.167 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:51:52.176 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:51:52.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.242+0000 7f27b0696640 1 Processor -- start 2023-12-01T02:51:52.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.242+0000 7f27b0696640 1 -- start start 2023-12-01T02:51:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.244+0000 7f27b0696640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f27a815dfd0 0x7f27a80cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.244+0000 7f27b0696640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f27a80cf9a0 0x7f27a80b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.244+0000 7f27b0696640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a8160b20 con 0x7f27a815dbf0 2023-12-01T02:51:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.244+0000 7f27b0696640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a8160dc0 con 0x7f27a80cf9a0 2023-12-01T02:51:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.244+0000 7f27b0696640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a81610a0 con 0x7f27a815dfd0 2023-12-01T02:51:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.244+0000 7f27aec33640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f27a80cf9a0 0x7f27a80b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.244+0000 7f27aec33640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f27a80cf9a0 0x7f27a80b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45682/0 (socket says 172.21.15.115:45682) 2023-12-01T02:51:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.244+0000 7f27aec33640 1 -- 172.21.15.115:0/837868089 learned_addr learned my addr 172.21.15.115:0/837868089 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:52.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.244+0000 7f27adc31640 1 --2- 172.21.15.115:0/837868089 >> v2:172.21.15.142:3300/0 conn(0x7f27a815dfd0 0x7f27a80cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.245+0000 7f27aec33640 1 -- 172.21.15.115:0/837868089 >> v2:172.21.15.142:3300/0 conn(0x7f27a815dfd0 msgr2=0x7f27a80cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.245+0000 7f27aec33640 1 --2- 172.21.15.115:0/837868089 >> v2:172.21.15.142:3300/0 conn(0x7f27a815dfd0 0x7f27a80cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.245+0000 7f27aec33640 1 -- 172.21.15.115:0/837868089 >> v1:172.21.15.142:6789/0 conn(0x7f27a815dbf0 legacy=0x7f27a80cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.245+0000 7f27aec33640 1 -- 172.21.15.115:0/837868089 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a8161380 con 0x7f27a80cf9a0 2023-12-01T02:51:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.246+0000 7f27aec33640 1 --2- 172.21.15.115:0/837868089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a80cf9a0 0x7f27a80b7ef0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f27a80b21c0 tx=0x7f27a002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d46f5fa26815f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:52.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.246+0000 7f27ad430640 1 -- 172.21.15.115:0/837868089 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a002cb20 con 0x7f27a80cf9a0 2023-12-01T02:51:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.246+0000 7f27ad430640 1 -- 172.21.15.115:0/837868089 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a002cc80 con 0x7f27a80cf9a0 2023-12-01T02:51:52.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.246+0000 7f27ad430640 1 -- 172.21.15.115:0/837868089 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a00343f0 con 0x7f27a80cf9a0 2023-12-01T02:51:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.247+0000 7f27b0696640 1 -- 172.21.15.115:0/837868089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a80cf9a0 msgr2=0x7f27a80b7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.247+0000 7f27b0696640 1 --2- 172.21.15.115:0/837868089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a80cf9a0 0x7f27a80b7ef0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f27a80b21c0 tx=0x7f27a002bf80 comp rx=0 tx=0).stop 2023-12-01T02:51:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.247+0000 7f27b0696640 1 -- 172.21.15.115:0/837868089 shutdown_connections 2023-12-01T02:51:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.247+0000 7f27b0696640 1 --2- 172.21.15.115:0/837868089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a80cf9a0 0x7f27a80b7ef0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.247+0000 7f27b0696640 1 --2- 172.21.15.115:0/837868089 >> v2:172.21.15.142:3300/0 conn(0x7f27a815dfd0 0x7f27a80cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.247+0000 7f27b0696640 1 -- 172.21.15.115:0/837868089 >> 172.21.15.115:0/837868089 conn(0x7f27a8065960 msgr2=0x7f27a8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.247+0000 7f27b0696640 1 -- 172.21.15.115:0/837868089 shutdown_connections 2023-12-01T02:51:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.247+0000 7f27b0696640 1 -- 172.21.15.115:0/837868089 wait complete. 2023-12-01T02:51:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.249+0000 7f27b0696640 1 Processor -- start 2023-12-01T02:51:52.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.249+0000 7f27b0696640 1 -- start start 2023-12-01T02:51:52.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.250+0000 7f27b0696640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27a80cf9a0 0x7f27a814f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.250+0000 7f27b0696640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27a815dbf0 0x7f27a8153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.250+0000 7f27b0696640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a815dfd0 0x7f27a814db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.250+0000 7f27b0696640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a81542f0 con 0x7f27a815dfd0 2023-12-01T02:51:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.250+0000 7f27b0696640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a8154d80 con 0x7f27a80cf9a0 2023-12-01T02:51:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.250+0000 7f27b0696640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a8154520 con 0x7f27a815dbf0 2023-12-01T02:51:52.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27aec33640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a815dfd0 0x7f27a814db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27aec33640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a815dfd0 0x7f27a814db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45694/0 (socket says 172.21.15.115:45694) 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27aec33640 1 -- 172.21.15.115:0/1744078515 learned_addr learned my addr 172.21.15.115:0/1744078515 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27ae432640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27a80cf9a0 0x7f27a814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27adc31640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27a815dbf0 0x7f27a8153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27aec33640 1 -- 172.21.15.115:0/1744078515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27a815dbf0 msgr2=0x7f27a8153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27aec33640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27a815dbf0 0x7f27a8153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27aec33640 1 -- 172.21.15.115:0/1744078515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27a80cf9a0 msgr2=0x7f27a814f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27aec33640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27a80cf9a0 0x7f27a814f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27aec33640 1 -- 172.21.15.115:0/1744078515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a0009b30 con 0x7f27a815dfd0 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.251+0000 7f27aec33640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a815dfd0 0x7f27a814db90 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f27a0036040 tx=0x7f27a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.252+0000 7f279effd640 1 -- 172.21.15.115:0/1744078515 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0009510 con 0x7f27a815dfd0 2023-12-01T02:51:52.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.252+0000 7f279effd640 1 -- 172.21.15.115:0/1744078515 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a002c920 con 0x7f27a815dfd0 2023-12-01T02:51:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.252+0000 7f27b0696640 1 -- 172.21.15.115:0/1744078515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a8154fd0 con 0x7f27a815dfd0 2023-12-01T02:51:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.252+0000 7f279effd640 1 -- 172.21.15.115:0/1744078515 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a003e690 con 0x7f27a815dfd0 2023-12-01T02:51:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.252+0000 7f27b0696640 1 -- 172.21.15.115:0/1744078515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a8155590 con 0x7f27a815dfd0 2023-12-01T02:51:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.253+0000 7f27b0696640 1 -- 172.21.15.115:0/1744078515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a81557b0 con 0x7f27a815dfd0 2023-12-01T02:51:52.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.254+0000 7f279effd640 1 -- 172.21.15.115:0/1744078515 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f27a003e7f0 con 0x7f27a815dfd0 2023-12-01T02:51:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.255+0000 7f279effd640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f278c06a550 0x7f278c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.255+0000 7f279effd640 1 -- 172.21.15.115:0/1744078515 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f27a00ab5c0 con 0x7f27a815dfd0 2023-12-01T02:51:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.255+0000 7f27ae432640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f278c06a550 0x7f278c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:52.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.256+0000 7f27ae432640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f278c06a550 0x7f278c06ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27a4000f60 tx=0x7f27a4009040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:52.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.260+0000 7f279effd640 1 -- 172.21.15.115:0/1744078515 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f27a0076a60 con 0x7f27a815dfd0 2023-12-01T02:51:52.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.474+0000 7f27b0696640 1 -- 172.21.15.115:0/1744078515 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f27a80cfdb0 con 0x7f278c06a550 2023-12-01T02:51:52.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.477+0000 7f279effd640 1 -- 172.21.15.115:0/1744078515 <== mgr.8769 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f27a4001c70 con 0x7f278c06a550 2023-12-01T02:51:52.479 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T02:51:52.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:51:52.479 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T02:51:52.479 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:51:52.479 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T02:51:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 -- 172.21.15.115:0/1744078515 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f278c06a550 msgr2=0x7f278c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f278c06a550 0x7f278c06ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27a4000f60 tx=0x7f27a4009040 comp rx=0 tx=0).stop 2023-12-01T02:51:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 -- 172.21.15.115:0/1744078515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a815dfd0 msgr2=0x7f27a814db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a815dfd0 0x7f27a814db90 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f27a0036040 tx=0x7f27a0004a40 comp rx=0 tx=0).stop 2023-12-01T02:51:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 -- 172.21.15.115:0/1744078515 shutdown_connections 2023-12-01T02:51:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f278c06a550 0x7f278c06ca10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27a815dbf0 0x7f27a8153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27a815dfd0 0x7f27a814db90 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 --2- 172.21.15.115:0/1744078515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27a80cf9a0 0x7f27a814f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 -- 172.21.15.115:0/1744078515 >> 172.21.15.115:0/1744078515 conn(0x7f27a8065960 msgr2=0x7f27a80d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:52.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 -- 172.21.15.115:0/1744078515 shutdown_connections 2023-12-01T02:51:52.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.480+0000 7f27b0696640 1 -- 172.21.15.115:0/1744078515 wait complete. 2023-12-01T02:51:52.499 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T02:51:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.707+0000 7fdf145c9640 1 Processor -- start 2023-12-01T02:51:52.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.708+0000 7fdf145c9640 1 -- start start 2023-12-01T02:51:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.709+0000 7fdf145c9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdf0c0cfd60 0x7fdf0c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.709+0000 7fdf145c9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0ce5e0 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.709+0000 7fdf145c9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0ce300 con 0x7fdf0c15cd90 2023-12-01T02:51:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.709+0000 7fdf145c9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c0d0ec0 con 0x7fdf0c0cfd60 2023-12-01T02:51:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.709+0000 7fdf11b64640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdf0c0cfd60 0x7fdf0c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.709+0000 7fdf11b64640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdf0c0cfd60 0x7fdf0c0cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45702/0 (socket says 172.21.15.115:45702) 2023-12-01T02:51:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.709+0000 7fdf11b64640 1 -- 172.21.15.115:0/1629722763 learned_addr learned my addr 172.21.15.115:0/1629722763 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:52.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.709+0000 7fdf12b66640 1 --1- 172.21.15.115:0/1629722763 >> v1:172.21.15.115:6789/0 conn(0x7fdf0c0b5fa0 0x7fdf0c0d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40324/0 (socket says 172.21.15.115:40324) 2023-12-01T02:51:52.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.710+0000 7fdf11b64640 1 -- 172.21.15.115:0/1629722763 >> v1:172.21.15.142:6789/0 conn(0x7fdf0c15cd90 legacy=0x7fdf0c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:52.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.710+0000 7fdf11b64640 1 -- 172.21.15.115:0/1629722763 >> v1:172.21.15.115:6789/0 conn(0x7fdf0c0b5fa0 legacy=0x7fdf0c0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:52.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.710+0000 7fdf11b64640 1 -- 172.21.15.115:0/1629722763 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0d10a0 con 0x7fdf0c0cfd60 2023-12-01T02:51:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.711+0000 7fdf11b64640 1 --2- 172.21.15.115:0/1629722763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0cfd60 0x7fdf0c0cdd20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009870 tx=0x7fdefc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9741fac1fc6628c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.711+0000 7fdf11363640 1 -- 172.21.15.115:0/1629722763 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc030860 con 0x7fdf0c0cfd60 2023-12-01T02:51:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.711+0000 7fdf11363640 1 -- 172.21.15.115:0/1629722763 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc0309c0 con 0x7fdf0c0cfd60 2023-12-01T02:51:52.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.711+0000 7fdf11363640 1 -- 172.21.15.115:0/1629722763 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc030d50 con 0x7fdf0c0cfd60 2023-12-01T02:51:52.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.712+0000 7fdf145c9640 1 -- 172.21.15.115:0/1629722763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0cfd60 msgr2=0x7fdf0c0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:52.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.712+0000 7fdf145c9640 1 --2- 172.21.15.115:0/1629722763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0cfd60 0x7fdf0c0cdd20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009870 tx=0x7fdefc02b9a0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.712+0000 7fdf145c9640 1 -- 172.21.15.115:0/1629722763 shutdown_connections 2023-12-01T02:51:52.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.712+0000 7fdf145c9640 1 --2- 172.21.15.115:0/1629722763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0cfd60 0x7fdf0c0cdd20 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.712+0000 7fdf145c9640 1 -- 172.21.15.115:0/1629722763 >> 172.21.15.115:0/1629722763 conn(0x7fdf0c065960 msgr2=0x7fdf0c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:52.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.712+0000 7fdf145c9640 1 -- 172.21.15.115:0/1629722763 shutdown_connections 2023-12-01T02:51:52.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.712+0000 7fdf145c9640 1 -- 172.21.15.115:0/1629722763 wait complete. 2023-12-01T02:51:52.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.714+0000 7fdf145c9640 1 Processor -- start 2023-12-01T02:51:52.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.714+0000 7fdf145c9640 1 -- start start 2023-12-01T02:51:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.715+0000 7fdf145c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0b5fa0 0x7fdf0c1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.715+0000 7fdf145c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c0cfd60 0x7fdf0c15ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.715+0000 7fdf145c9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c15cd90 0x7fdf0c15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.715+0000 7fdf145c9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1e6f60 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.715+0000 7fdf145c9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1e6930 con 0x7fdf0c15cd90 2023-12-01T02:51:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.715+0000 7fdf145c9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1e6120 con 0x7fdf0c0cfd60 2023-12-01T02:51:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.715+0000 7fdf12365640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0b5fa0 0x7fdf0c1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.715+0000 7fdf12365640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0b5fa0 0x7fdf0c1554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45714/0 (socket says 172.21.15.115:45714) 2023-12-01T02:51:52.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.715+0000 7fdf12365640 1 -- 172.21.15.115:0/2857866478 learned_addr learned my addr 172.21.15.115:0/2857866478 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:52.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf12365640 1 -- 172.21.15.115:0/2857866478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c0cfd60 msgr2=0x7fdf0c15ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf12365640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c0cfd60 0x7fdf0c15ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf12b66640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c15cd90 0x7fdf0c15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf12365640 1 -- 172.21.15.115:0/2857866478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c15cd90 msgr2=0x7fdf0c15efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf12365640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c15cd90 0x7fdf0c15efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf12365640 1 -- 172.21.15.115:0/2857866478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0094d0 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf12b66640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c15cd90 0x7fdf0c15efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf12365640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0b5fa0 0x7fdf0c1554f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800ba80 tx=0x7fdf0800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf02ffd640 1 -- 172.21.15.115:0/2857866478 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800cc90 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf02ffd640 1 -- 172.21.15.115:0/2857866478 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0800cdf0 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf145c9640 1 -- 172.21.15.115:0/2857866478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c1e6400 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf02ffd640 1 -- 172.21.15.115:0/2857866478 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08014680 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.716+0000 7fdf145c9640 1 -- 172.21.15.115:0/2857866478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c160070 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.717+0000 7fdf145c9640 1 -- 172.21.15.115:0/2857866478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c1e70a0 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.718+0000 7fdf02ffd640 1 -- 172.21.15.115:0/2857866478 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 85) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fdf0801d3b0 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.719+0000 7fdf02ffd640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdef006a550 0x7fdef006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:52.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.719+0000 7fdf02ffd640 1 -- 172.21.15.115:0/2857866478 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fdf08088b30 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.720+0000 7fdf11b64640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdef006a550 0x7fdef006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:52.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.720+0000 7fdf11b64640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdef006a550 0x7fdef006ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdefc036000 tx=0x7fdefc03b040 comp rx=0 tx=0).ready entity=mgr.8769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:52.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.723+0000 7fdf02ffd640 1 -- 172.21.15.115:0/2857866478 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdf08053fd0 con 0x7fdf0c0b5fa0 2023-12-01T02:51:52.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:52.964+0000 7fdf145c9640 1 -- 172.21.15.115:0/2857866478 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fdf0c160260 con 0x7fdf0c0b5fa0 2023-12-01T02:51:53.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.979+0000 7fdf02ffd640 1 -- 172.21.15.115:0/2857866478 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 86) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fdf080543a0 con 0x7fdf0c0b5fa0 2023-12-01T02:51:53.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.982+0000 7fdf02ffd640 1 -- 172.21.15.115:0/2857866478 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v86) v1 ==== 92+0+0 (secure 0 0 0) 0x7fdf080726e0 con 0x7fdf0c0b5fa0 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 -- 172.21.15.115:0/2857866478 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdef006a550 msgr2=0x7fdef006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdef006a550 0x7fdef006ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdefc036000 tx=0x7fdefc03b040 comp rx=0 tx=0).stop 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 -- 172.21.15.115:0/2857866478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0b5fa0 msgr2=0x7fdf0c1554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0b5fa0 0x7fdf0c1554f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800ba80 tx=0x7fdf0800bf40 comp rx=0 tx=0).stop 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 -- 172.21.15.115:0/2857866478 shutdown_connections 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fdef006a550 0x7fdef006ca10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c0cfd60 0x7fdf0c15ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf0c15cd90 0x7fdf0c15efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 --2- 172.21.15.115:0/2857866478 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf0c0b5fa0 0x7fdf0c1554f0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 -- 172.21.15.115:0/2857866478 >> 172.21.15.115:0/2857866478 conn(0x7fdf0c065960 msgr2=0x7fdf0c0b7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 -- 172.21.15.115:0/2857866478 shutdown_connections 2023-12-01T02:51:53.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:53.986+0000 7fdf145c9640 1 -- 172.21.15.115:0/2857866478 wait complete. 2023-12-01T02:51:54.008 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T02:51:54.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.201+0000 7fe5d259f640 1 Processor -- start 2023-12-01T02:51:54.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.201+0000 7fe5d259f640 1 -- start start 2023-12-01T02:51:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5d259f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe5cc15cfa0 0x7fe5cc0b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5d259f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe5cc0b7e70 0x7fe5cc0b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5d259f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5cc0b6660 con 0x7fe5cc0b6210 2023-12-01T02:51:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5d259f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5cc0b6890 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5d259f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5cc0b6b70 con 0x7fe5cc0b7e70 2023-12-01T02:51:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5cbfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe5cc15cfa0 0x7fe5cc0b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5d0b3c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe5cc0b6210 0x7fe5cc0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40330/0 (socket says 172.21.15.115:40330) 2023-12-01T02:51:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5d0b3c640 1 -- 172.21.15.115:0/2089077770 learned_addr learned my addr 172.21.15.115:0/2089077770 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5cbfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe5cc15cfa0 0x7fe5cc0b7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45740/0 (socket says 172.21.15.115:45740) 2023-12-01T02:51:54.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5c3fff640 1 --2- 172.21.15.115:0/2089077770 >> v2:172.21.15.142:3300/0 conn(0x7fe5cc0b7e70 0x7fe5cc0b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:54.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5cbfff640 1 -- 172.21.15.115:0/2089077770 >> v2:172.21.15.142:3300/0 conn(0x7fe5cc0b7e70 msgr2=0x7fe5cc0b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:54.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5cbfff640 1 --2- 172.21.15.115:0/2089077770 >> v2:172.21.15.142:3300/0 conn(0x7fe5cc0b7e70 0x7fe5cc0b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.203+0000 7fe5cbfff640 1 -- 172.21.15.115:0/2089077770 >> v1:172.21.15.115:6789/0 conn(0x7fe5cc0b6210 legacy=0x7fe5cc0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:54.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.204+0000 7fe5cbfff640 1 -- 172.21.15.115:0/2089077770 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc0b6e50 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.204+0000 7fe5cb7fe640 1 -- 172.21.15.115:0/2089077770 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102877736 0 0) 0x7fe5cc0b6660 con 0x7fe5cc0b6210 2023-12-01T02:51:54.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.205+0000 7fe5cbfff640 1 --2- 172.21.15.115:0/2089077770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 0x7fe5cc0b7930 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0b4930 tx=0x7fe5bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8234c074a31532 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:54.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.205+0000 7fe5cb7fe640 1 -- 172.21.15.115:0/2089077770 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc02e070 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.205+0000 7fe5cb7fe640 1 -- 172.21.15.115:0/2089077770 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0315a0 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.205+0000 7fe5cb7fe640 1 -- 172.21.15.115:0/2089077770 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc031930 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.206+0000 7fe5d259f640 1 -- 172.21.15.115:0/2089077770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 msgr2=0x7fe5cc0b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:54.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.206+0000 7fe5d259f640 1 --2- 172.21.15.115:0/2089077770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 0x7fe5cc0b7930 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0b4930 tx=0x7fe5bc02b840 comp rx=0 tx=0).stop 2023-12-01T02:51:54.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.206+0000 7fe5d259f640 1 -- 172.21.15.115:0/2089077770 shutdown_connections 2023-12-01T02:51:54.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.206+0000 7fe5d259f640 1 --2- 172.21.15.115:0/2089077770 >> v2:172.21.15.142:3300/0 conn(0x7fe5cc0b7e70 0x7fe5cc0b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.206+0000 7fe5d259f640 1 --2- 172.21.15.115:0/2089077770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 0x7fe5cc0b7930 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.206+0000 7fe5d259f640 1 -- 172.21.15.115:0/2089077770 >> 172.21.15.115:0/2089077770 conn(0x7fe5cc06a0b0 msgr2=0x7fe5cc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:54.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.206+0000 7fe5d259f640 1 -- 172.21.15.115:0/2089077770 shutdown_connections 2023-12-01T02:51:54.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.206+0000 7fe5d259f640 1 -- 172.21.15.115:0/2089077770 wait complete. 2023-12-01T02:51:54.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.207+0000 7fe5d259f640 1 Processor -- start 2023-12-01T02:51:54.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.208+0000 7fe5d259f640 1 -- start start 2023-12-01T02:51:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.208+0000 7fe5d259f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5cc0b6210 0x7fe5cc0d2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.208+0000 7fe5d259f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc0b7e70 0x7fe5cc1f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.208+0000 7fe5d259f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 0x7fe5cc1f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.208+0000 7fe5d259f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc1f5da0 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.208+0000 7fe5d259f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc1f2c10 con 0x7fe5cc0b7e70 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.208+0000 7fe5d259f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc1f2ef0 con 0x7fe5cc0b6210 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.209+0000 7fe5d0b3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 0x7fe5cc1f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.209+0000 7fe5d0b3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 0x7fe5cc1f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45748/0 (socket says 172.21.15.115:45748) 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.209+0000 7fe5d0b3c640 1 -- 172.21.15.115:0/2008494851 learned_addr learned my addr 172.21.15.115:0/2008494851 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.209+0000 7fe5c3fff640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc0b7e70 0x7fe5cc1f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.209+0000 7fe5cbfff640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5cc0b6210 0x7fe5cc0d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.209+0000 7fe5d0b3c640 1 -- 172.21.15.115:0/2008494851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5cc0b6210 msgr2=0x7fe5cc0d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.209+0000 7fe5d0b3c640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5cc0b6210 0x7fe5cc0d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.209+0000 7fe5d0b3c640 1 -- 172.21.15.115:0/2008494851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc0b7e70 msgr2=0x7fe5cc1f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.209+0000 7fe5d0b3c640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc0b7e70 0x7fe5cc1f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.209+0000 7fe5d0b3c640 1 -- 172.21.15.115:0/2008494851 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc009400 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.210+0000 7fe5d0b3c640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 0x7fe5cc1f2350 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c780 tx=0x7fe5c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.210+0000 7fe5c97fa640 1 -- 172.21.15.115:0/2008494851 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400ce30 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.210+0000 7fe5c97fa640 1 -- 172.21.15.115:0/2008494851 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c400a640 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.210+0000 7fe5c97fa640 1 -- 172.21.15.115:0/2008494851 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400aa70 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.210+0000 7fe5d259f640 1 -- 172.21.15.115:0/2008494851 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5cc1f59c0 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.210+0000 7fe5d259f640 1 -- 172.21.15.115:0/2008494851 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc1f6700 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.211+0000 7fe5d259f640 1 -- 172.21.15.115:0/2008494851 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5cc1f5ee0 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.213+0000 7fe5c97fa640 1 -- 172.21.15.115:0/2008494851 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 86) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fe5c400abd0 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.214+0000 7fe5c97fa640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5ac06a5a0 0x7fe5ac06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.214+0000 7fe5c97fa640 1 -- 172.21.15.115:0/2008494851 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fe5c4088060 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.214+0000 7fe5cbfff640 1 -- 172.21.15.115:0/2008494851 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5ac06a5a0 msgr2=0x7fe5ac06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:54.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.214+0000 7fe5cbfff640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5ac06a5a0 0x7fe5ac06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:54.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.219+0000 7fe5c97fa640 1 -- 172.21.15.115:0/2008494851 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe5c40bac80 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.401+0000 7fe5d259f640 1 -- 172.21.15.115:0/2008494851 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe5cc073950 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.408+0000 7fe5c97fa640 1 -- 172.21.15.115:0/2008494851 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe5c4018020 con 0x7fe5cc15cfa0 2023-12-01T02:51:54.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 -- 172.21.15.115:0/2008494851 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5ac06a5a0 msgr2=0x7fe5ac06ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:54.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5ac06a5a0 0x7fe5ac06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 -- 172.21.15.115:0/2008494851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 msgr2=0x7fe5cc1f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:54.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 0x7fe5cc1f2350 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c780 tx=0x7fe5c400cc40 comp rx=0 tx=0).stop 2023-12-01T02:51:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 -- 172.21.15.115:0/2008494851 shutdown_connections 2023-12-01T02:51:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe5ac06a5a0 0x7fe5ac06ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5cc15cfa0 0x7fe5cc1f2350 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5cc0b7e70 0x7fe5cc1f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 --2- 172.21.15.115:0/2008494851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5cc0b6210 0x7fe5cc0d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 -- 172.21.15.115:0/2008494851 >> 172.21.15.115:0/2008494851 conn(0x7fe5cc06a0b0 msgr2=0x7fe5cc0c9030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 -- 172.21.15.115:0/2008494851 shutdown_connections 2023-12-01T02:51:54.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.411+0000 7fe5d259f640 1 -- 172.21.15.115:0/2008494851 wait complete. 2023-12-01T02:51:54.435 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T02:51:54.496 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:54.495+0000 7f8a9b2da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T02:51:54.505 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:54.504+0000 7f042aa2e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T02:51:54.550 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:54.549+0000 7f8a9b2da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T02:51:54.561 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:54.560+0000 7f042aa2e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T02:51:54.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.623+0000 7f5db6d00640 1 Processor -- start 2023-12-01T02:51:54.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.623+0000 7f5db6d00640 1 -- start start 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.624+0000 7f5db6d00640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5db00cfd60 0x7f5db00cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.624+0000 7f5db6d00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5db00b5fa0 0x7f5db0161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.624+0000 7f5db6d00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db00ce5e0 con 0x7f5db015cd90 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.624+0000 7f5db6d00640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db00d0ec0 con 0x7f5db00cfd60 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.624+0000 7f5db6d00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db0161950 con 0x7f5db00b5fa0 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.625+0000 7f5daffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5db00cfd60 0x7f5db00cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.625+0000 7f5daffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5db00cfd60 0x7f5db00cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45756/0 (socket says 172.21.15.115:45756) 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.625+0000 7f5daffff640 1 -- 172.21.15.115:0/1088723782 learned_addr learned my addr 172.21.15.115:0/1088723782 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.625+0000 7f5db529d640 1 --2- 172.21.15.115:0/1088723782 >> v2:172.21.15.142:3300/0 conn(0x7f5db00b5fa0 0x7f5db0161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.625+0000 7f5daffff640 1 -- 172.21.15.115:0/1088723782 >> v2:172.21.15.142:3300/0 conn(0x7f5db00b5fa0 msgr2=0x7f5db0161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.625+0000 7f5daffff640 1 --2- 172.21.15.115:0/1088723782 >> v2:172.21.15.142:3300/0 conn(0x7f5db00b5fa0 0x7f5db0161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.625+0000 7f5daffff640 1 -- 172.21.15.115:0/1088723782 >> v1:172.21.15.142:6789/0 conn(0x7f5db015cd90 legacy=0x7f5db00cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.625+0000 7f5daffff640 1 -- 172.21.15.115:0/1088723782 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db00ce300 con 0x7f5db00cfd60 2023-12-01T02:51:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.626+0000 7f5daffff640 1 --2- 172.21.15.115:0/1088723782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db00cfd60 0x7f5db00cdd20 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5db0160940 tx=0x7f5da402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0624753271cca3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5daf7fe640 1 -- 172.21.15.115:0/1088723782 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4031870 con 0x7f5db00cfd60 2023-12-01T02:51:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5daf7fe640 1 -- 172.21.15.115:0/1088723782 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da40319d0 con 0x7f5db00cfd60 2023-12-01T02:51:54.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5daf7fe640 1 -- 172.21.15.115:0/1088723782 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4031d60 con 0x7f5db00cfd60 2023-12-01T02:51:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5db6d00640 1 -- 172.21.15.115:0/1088723782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db00cfd60 msgr2=0x7f5db00cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5db6d00640 1 --2- 172.21.15.115:0/1088723782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db00cfd60 0x7f5db00cdd20 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5db0160940 tx=0x7f5da402b910 comp rx=0 tx=0).stop 2023-12-01T02:51:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5db6d00640 1 -- 172.21.15.115:0/1088723782 shutdown_connections 2023-12-01T02:51:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5db6d00640 1 --2- 172.21.15.115:0/1088723782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db00cfd60 0x7f5db00cdd20 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5db6d00640 1 --2- 172.21.15.115:0/1088723782 >> v2:172.21.15.142:3300/0 conn(0x7f5db00b5fa0 0x7f5db0161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5db6d00640 1 -- 172.21.15.115:0/1088723782 >> 172.21.15.115:0/1088723782 conn(0x7f5db0065960 msgr2=0x7f5db0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5db6d00640 1 -- 172.21.15.115:0/1088723782 shutdown_connections 2023-12-01T02:51:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.627+0000 7f5db6d00640 1 -- 172.21.15.115:0/1088723782 wait complete. 2023-12-01T02:51:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.629+0000 7f5db6d00640 1 Processor -- start 2023-12-01T02:51:54.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.629+0000 7f5db6d00640 1 -- start start 2023-12-01T02:51:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.630+0000 7f5db6d00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db00b5fa0 0x7f5db0158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.630+0000 7f5db6d00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5db00cfd60 0x7f5db0152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.630+0000 7f5db6d00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db015cd90 0x7f5db0157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.630+0000 7f5db6d00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db0160140 con 0x7f5db015cd90 2023-12-01T02:51:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.630+0000 7f5db6d00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db0159070 con 0x7f5db00cfd60 2023-12-01T02:51:54.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.630+0000 7f5db6d00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db015df40 con 0x7f5db00b5fa0 2023-12-01T02:51:54.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.630+0000 7f5db529d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db015cd90 0x7f5db0157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:54.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.630+0000 7f5db529d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db015cd90 0x7f5db0157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45762/0 (socket says 172.21.15.115:45762) 2023-12-01T02:51:54.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.630+0000 7f5db529d640 1 -- 172.21.15.115:0/3683137783 learned_addr learned my addr 172.21.15.115:0/3683137783 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:54.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.630+0000 7f5daffff640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5db00cfd60 0x7f5db0152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:54.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5db4a9c640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db00b5fa0 0x7f5db0158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:54.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5db529d640 1 -- 172.21.15.115:0/3683137783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db00b5fa0 msgr2=0x7f5db0158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5db529d640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db00b5fa0 0x7f5db0158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5db529d640 1 -- 172.21.15.115:0/3683137783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5db00cfd60 msgr2=0x7f5db0152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5db529d640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5db00cfd60 0x7f5db0152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5db529d640 1 -- 172.21.15.115:0/3683137783 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da40094d0 con 0x7f5db015cd90 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5db529d640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db015cd90 0x7f5db0157e50 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f5da8007e40 tx=0x7f5da800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5dad7fa640 1 -- 172.21.15.115:0/3683137783 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8014620 con 0x7f5db015cd90 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5dad7fa640 1 -- 172.21.15.115:0/3683137783 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da800ee90 con 0x7f5db015cd90 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5db6d00640 1 -- 172.21.15.115:0/3683137783 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db015f530 con 0x7f5db015cd90 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5dad7fa640 1 -- 172.21.15.115:0/3683137783 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da80149e0 con 0x7f5db015cd90 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.631+0000 7f5db6d00640 1 -- 172.21.15.115:0/3683137783 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db015faf0 con 0x7f5db015cd90 2023-12-01T02:51:54.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.633+0000 7f5db6d00640 1 -- 172.21.15.115:0/3683137783 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db004e890 con 0x7f5db015cd90 2023-12-01T02:51:54.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.633+0000 7f5dad7fa640 1 -- 172.21.15.115:0/3683137783 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 86) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f5da8014b40 con 0x7f5db015cd90 2023-12-01T02:51:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.635+0000 7f5dad7fa640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5d9c06a5a0 0x7f5d9c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.635+0000 7f5dad7fa640 1 -- 172.21.15.115:0/3683137783 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f5da80894d0 con 0x7f5db015cd90 2023-12-01T02:51:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.635+0000 7f5db4a9c640 1 -- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5d9c06a5a0 msgr2=0x7f5d9c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:54.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.635+0000 7f5db4a9c640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5d9c06a5a0 0x7f5d9c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:54.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.639+0000 7f5dad7fa640 1 -- 172.21.15.115:0/3683137783 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5da8054980 con 0x7f5db015cd90 2023-12-01T02:51:54.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.828+0000 7f5db6d00640 1 -- 172.21.15.115:0/3683137783 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f5db00d0a70 con 0x7f5db015cd90 2023-12-01T02:51:54.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.835+0000 7f5dad7fa640 1 -- 172.21.15.115:0/3683137783 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5da8057fa0 con 0x7f5db015cd90 2023-12-01T02:51:54.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.836+0000 7f5db4a9c640 1 -- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5d9c06a5a0 msgr2=0x7f5d9c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:54.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.836+0000 7f5db4a9c640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5d9c06a5a0 0x7f5d9c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:51:54.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.837+0000 7f5db6d00640 1 -- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5d9c06a5a0 msgr2=0x7f5d9c06ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:54.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.837+0000 7f5db6d00640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5d9c06a5a0 0x7f5d9c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.837+0000 7f5db6d00640 1 -- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db015cd90 msgr2=0x7f5db0157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:54.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.837+0000 7f5db6d00640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db015cd90 0x7f5db0157e50 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f5da8007e40 tx=0x7f5da800e4b0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.837+0000 7f5db6d00640 1 -- 172.21.15.115:0/3683137783 shutdown_connections 2023-12-01T02:51:54.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.837+0000 7f5db6d00640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5d9c06a5a0 0x7f5d9c06ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.837+0000 7f5db6d00640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db015cd90 0x7f5db0157e50 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.837+0000 7f5db6d00640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5db00cfd60 0x7f5db0152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.837+0000 7f5db6d00640 1 --2- 172.21.15.115:0/3683137783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db00b5fa0 0x7f5db0158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:54.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.837+0000 7f5db6d00640 1 -- 172.21.15.115:0/3683137783 >> 172.21.15.115:0/3683137783 conn(0x7f5db0065960 msgr2=0x7f5db00b6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:54.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.838+0000 7f5db6d00640 1 -- 172.21.15.115:0/3683137783 shutdown_connections 2023-12-01T02:51:54.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:54.838+0000 7f5db6d00640 1 -- 172.21.15.115:0/3683137783 wait complete. 2023-12-01T02:51:54.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:51:54.857 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-01T02:51:54.911 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1h 2023-12-01T02:51:54.997 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:54.996+0000 7f8a9b2da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T02:51:55.038 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.036+0000 7f042aa2e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T02:51:55.048 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:55.047+0000 7f8a9b2da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T02:51:55.092 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.091+0000 7f042aa2e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T02:51:55.097 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:55.096+0000 7f8a9b2da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T02:51:55.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.112+0000 7f3378b9b640 1 Processor -- start 2023-12-01T02:51:55.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.112+0000 7f3378b9b640 1 -- start start 2023-12-01T02:51:55.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.114+0000 7f3378b9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 0x7f33740ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:55.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.114+0000 7f3378b9b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33740cdcd0 con 0x7f3374154010 2023-12-01T02:51:55.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.114+0000 7f3378b9b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3374161830 con 0x7f33740cfb80 2023-12-01T02:51:55.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.114+0000 7f3378b9b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33740cd3a0 con 0x7f3374153c30 2023-12-01T02:51:55.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.114+0000 7f336bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3374154010 0x7f33740cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40332/0 (socket says 172.21.15.115:40332) 2023-12-01T02:51:55.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.114+0000 7f336bfff640 1 -- 172.21.15.115:0/128797492 learned_addr learned my addr 172.21.15.115:0/128797492 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:51:55.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.114+0000 7f33737fe640 1 --2- 172.21.15.115:0/128797492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 0x7f33740ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:55.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.115+0000 7f3372ffd640 1 -- 172.21.15.115:0/128797492 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342982275 0 0) 0x7f33740cdcd0 con 0x7f3374154010 2023-12-01T02:51:55.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.115+0000 7f3372ffd640 1 -- 172.21.15.115:0/128797492 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3360003520 con 0x7f3374154010 2023-12-01T02:51:55.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.115+0000 7f33737fe640 1 -- 172.21.15.115:0/128797492 >> v1:172.21.15.142:6789/0 conn(0x7f33740cfb80 legacy=0x7f3374160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:55.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.115+0000 7f3372ffd640 1 -- 172.21.15.115:0/128797492 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1139129162 0 0) 0x7f3360003520 con 0x7f3374154010 2023-12-01T02:51:55.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.115+0000 7f33737fe640 1 -- 172.21.15.115:0/128797492 >> v1:172.21.15.115:6789/0 conn(0x7f3374154010 legacy=0x7f33740cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:55.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.115+0000 7f33737fe640 1 -- 172.21.15.115:0/128797492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33740cd680 con 0x7f3374153c30 2023-12-01T02:51:55.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.116+0000 7f33737fe640 1 --2- 172.21.15.115:0/128797492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 0x7f33740ce160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f33640097d0 tx=0x7f336402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cce692e5b7d54981 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:55.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.116+0000 7f3372ffd640 1 -- 172.21.15.115:0/128797492 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c001670 con 0x7f3374153c30 2023-12-01T02:51:55.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.117+0000 7f3372ffd640 1 -- 172.21.15.115:0/128797492 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336402e050 con 0x7f3374153c30 2023-12-01T02:51:55.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.117+0000 7f3372ffd640 1 -- 172.21.15.115:0/128797492 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364030710 con 0x7f3374153c30 2023-12-01T02:51:55.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.117+0000 7f3378b9b640 1 -- 172.21.15.115:0/128797492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 msgr2=0x7f33740ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:55.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.117+0000 7f3378b9b640 1 --2- 172.21.15.115:0/128797492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 0x7f33740ce160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f33640097d0 tx=0x7f336402b800 comp rx=0 tx=0).stop 2023-12-01T02:51:55.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.117+0000 7f3378b9b640 1 -- 172.21.15.115:0/128797492 shutdown_connections 2023-12-01T02:51:55.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.117+0000 7f3378b9b640 1 --2- 172.21.15.115:0/128797492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 0x7f33740ce160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:55.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.117+0000 7f3378b9b640 1 -- 172.21.15.115:0/128797492 >> 172.21.15.115:0/128797492 conn(0x7f337406a0b0 msgr2=0x7f337406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:55.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.117+0000 7f3378b9b640 1 -- 172.21.15.115:0/128797492 shutdown_connections 2023-12-01T02:51:55.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.117+0000 7f3378b9b640 1 -- 172.21.15.115:0/128797492 wait complete. 2023-12-01T02:51:55.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.119+0000 7f3378b9b640 1 Processor -- start 2023-12-01T02:51:55.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.119+0000 7f3378b9b640 1 -- start start 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.120+0000 7f3378b9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33740cfb80 0x7f33740c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.120+0000 7f3378b9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 0x7f33741439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.120+0000 7f3378b9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3374154010 0x7f33740d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.120+0000 7f3378b9b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33740c7320 con 0x7f3374154010 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.120+0000 7f3378b9b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33741468a0 con 0x7f33740cfb80 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.120+0000 7f3378b9b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33740c7570 con 0x7f3374153c30 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.120+0000 7f3373fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3374154010 0x7f33740d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.120+0000 7f3373fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3374154010 0x7f33740d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:45772/0 (socket says 172.21.15.115:45772) 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f3373fff640 1 -- 172.21.15.115:0/1422805859 learned_addr learned my addr 172.21.15.115:0/1422805859 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f336bfff640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 0x7f33741439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f33737fe640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33740cfb80 0x7f33740c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f3373fff640 1 -- 172.21.15.115:0/1422805859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 msgr2=0x7f33741439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f3373fff640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 0x7f33741439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:55.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f3373fff640 1 -- 172.21.15.115:0/1422805859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33740cfb80 msgr2=0x7f33740c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:55.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f3373fff640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33740cfb80 0x7f33740c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:55.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f3373fff640 1 -- 172.21.15.115:0/1422805859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3364009400 con 0x7f3374154010 2023-12-01T02:51:55.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f33737fe640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33740cfb80 0x7f33740c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:55.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f3373fff640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3374154010 0x7f33740d11a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f336c00c7a0 tx=0x7f336c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:55.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.121+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c003b70 con 0x7f3374154010 2023-12-01T02:51:55.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.122+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336c003cf0 con 0x7f3374154010 2023-12-01T02:51:55.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.122+0000 7f3378b9b640 1 -- 172.21.15.115:0/1422805859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33740c7850 con 0x7f3374154010 2023-12-01T02:51:55.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.122+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c003ea0 con 0x7f3374154010 2023-12-01T02:51:55.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.122+0000 7f3378b9b640 1 -- 172.21.15.115:0/1422805859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33740c7e10 con 0x7f3374154010 2023-12-01T02:51:55.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.123+0000 7f3378b9b640 1 -- 172.21.15.115:0/1422805859 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33740c8000 con 0x7f3374154010 2023-12-01T02:51:55.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.124+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 86) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f336c00aa90 con 0x7f3374154010 2023-12-01T02:51:55.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.125+0000 7f3370ff9640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:55.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.125+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f336c088b40 con 0x7f3374154010 2023-12-01T02:51:55.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.125+0000 7f33737fe640 1 -- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 msgr2=0x7f335406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:55.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.125+0000 7f33737fe640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:51:55.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.133+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f336c053e40 con 0x7f3374154010 2023-12-01T02:51:55.150 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.149+0000 7f042aa2e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T02:51:55.159 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:55.158+0000 7f8a9b2da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T02:51:55.207 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:55.207+0000 7f8a9b2da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T02:51:55.211 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.210+0000 7f042aa2e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T02:51:55.260 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.259+0000 7f042aa2e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T02:51:55.317 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:55.316+0000 7f8a9b2da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T02:51:55.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.326+0000 7f33737fe640 1 -- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 msgr2=0x7f335406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:55.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.326+0000 7f33737fe640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:51:55.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.346+0000 7f3378b9b640 1 -- 172.21.15.115:0/1422805859 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f33740cd960 con 0x7f335406a5a0 2023-12-01T02:51:55.371 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.370+0000 7f042aa2e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T02:51:55.378 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:55.378+0000 7f8a9b2da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T02:51:55.427 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:55.426+0000 7f8a9b2da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T02:51:55.433 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.431+0000 7f042aa2e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T02:51:55.483 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.481+0000 7f042aa2e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T02:51:55.484 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:55.483+0000 7f8a9b2da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T02:51:55.534 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:55.533+0000 7f8a9b2da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T02:51:55.540 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.538+0000 7f042aa2e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T02:51:55.589 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.587+0000 7f042aa2e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T02:51:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.727+0000 7f33737fe640 1 -- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 msgr2=0x7f335406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:55.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:55.727+0000 7f33737fe640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T02:51:55.913 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:55.912+0000 7f8a9b2da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T02:51:55.970 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:55.968+0000 7f042aa2e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T02:51:56.070 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:56.069+0000 7f8a9b2da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T02:51:56.129 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:56.128+0000 7f042aa2e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T02:51:56.178 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:56.177+0000 7f8a9b2da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T02:51:56.228 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:56.227+0000 7f8a9b2da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T02:51:56.237 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:56.235+0000 7f042aa2e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T02:51:56.287 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:56.285+0000 7f042aa2e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T02:51:56.335 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:56.334+0000 7f8a9b2da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:51:56.394 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:56.392+0000 7f042aa2e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:51:56.441 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:56.440+0000 7f8a9b2da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T02:51:56.499 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:56.497+0000 7f042aa2e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T02:51:56.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:56.528+0000 7f33737fe640 1 -- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 msgr2=0x7f335406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:56.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:56.528+0000 7f33737fe640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T02:51:56.595 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:56.593+0000 7f8a9b2da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:51:56.643 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:56.642+0000 7f8a9b2da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T02:51:56.657 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:56.655+0000 7f042aa2e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:51:56.706 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:56.704+0000 7f042aa2e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T02:51:56.760 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:56.759+0000 7f8a9b2da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T02:51:56.825 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:56.823+0000 7f042aa2e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T02:51:57.582 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:57.581+0000 7f8a9b2da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T02:51:57.649 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:57.646+0000 7f042aa2e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T02:51:57.876 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:57.875+0000 7f8a9b2da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T02:51:57.933 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:57.932+0000 7f8a9b2da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T02:51:57.943 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:57.941+0000 7f042aa2e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T02:51:58.002 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.000+0000 7f042aa2e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T02:51:58.046 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:51:58.045+0000 7f8a9b2da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T02:51:58.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:58.114+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 87) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f336c00ad40 con 0x7f3374154010 2023-12-01T02:51:58.119 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.117+0000 7f042aa2e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T02:51:58.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:58.130+0000 7f33737fe640 1 -- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 msgr2=0x7f335406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:51:58.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:58.130+0000 7f33737fe640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-01T02:51:58.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:58.133+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 88) v1 ==== 88335+0+0 (secure 0 0 0) 0x7f336c054210 con 0x7f3374154010 2023-12-01T02:51:58.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:58.134+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 msgr2=0x7f335406ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:51:58.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:58.134+0000 7f3370ff9640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:58.274 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.273+0000 7f03a6e64640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:58.275 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.273+0000 7f03a6e64640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:58.275 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.273+0000 7f03a6e64640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:58.275 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.273+0000 7f03a6e64640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:58.275 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.273+0000 7f03a6e64640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:58.275 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.273+0000 7f03a9e6a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:58.275 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.273+0000 7f03a9e6a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:58.275 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.273+0000 7f03a9e6a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:58.275 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.273+0000 7f03a9e6a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:58.275 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:51:58.273+0000 7f03a9e6a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:51:59.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.150+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 <== mon.0 v2:172.21.15.115:3300/0 9 ==== mgrmap(e 89) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f336c054ae0 con 0x7f3374154010 2023-12-01T02:51:59.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.151+0000 7f3370ff9640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:59.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.151+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f3354027760 con 0x7f335406a5a0 2023-12-01T02:51:59.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.159+0000 7f33737fe640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:59.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.166+0000 7f33737fe640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406c9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3364034000 tx=0x7f3364008030 comp rx=0 tx=0).ready entity=mgr.9213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:59.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.326+0000 7f3370ff9640 1 -- 172.21.15.115:0/1422805859 <== mgr.9213 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3354027760 con 0x7f335406a5a0 2023-12-01T02:51:59.329 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path snap_test_dir1 2023-12-01T02:51:59.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.329+0000 7f3378b9b640 1 -- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 msgr2=0x7f335406c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.329+0000 7f3378b9b640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406c9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3364034000 tx=0x7f3364008030 comp rx=0 tx=0).stop 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.330+0000 7f3378b9b640 1 -- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3374154010 msgr2=0x7f33740d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.330+0000 7f3378b9b640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3374154010 0x7f33740d11a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f336c00c7a0 tx=0x7f336c00cc60 comp rx=0 tx=0).stop 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.330+0000 7f3378b9b640 1 -- 172.21.15.115:0/1422805859 shutdown_connections 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.330+0000 7f3378b9b640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f335406a5a0 0x7f335406c9e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.330+0000 7f3378b9b640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3374153c30 0x7f33741439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.330+0000 7f3378b9b640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3374154010 0x7f33740d11a0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.330+0000 7f3378b9b640 1 --2- 172.21.15.115:0/1422805859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33740cfb80 0x7f33740c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.330+0000 7f3378b9b640 1 -- 172.21.15.115:0/1422805859 >> 172.21.15.115:0/1422805859 conn(0x7f337406a0b0 msgr2=0x7f337406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.330+0000 7f3378b9b640 1 -- 172.21.15.115:0/1422805859 shutdown_connections 2023-12-01T02:51:59.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.330+0000 7f3378b9b640 1 -- 172.21.15.115:0/1422805859 wait complete. 2023-12-01T02:51:59.354 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-01T02:51:59.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-01T02:51:59.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.562+0000 7f092bc28640 1 Processor -- start 2023-12-01T02:51:59.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.563+0000 7f092bc28640 1 -- start start 2023-12-01T02:51:59.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.564+0000 7f092bc28640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092415cd90 0x7f09240cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:59.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.564+0000 7f092bc28640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f09240b5fa0 0x7f0924161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:59.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.564+0000 7f092bc28640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09240ce5e0 con 0x7f09240cf650 2023-12-01T02:51:59.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.564+0000 7f092bc28640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09240d0ec0 con 0x7f09240b5fa0 2023-12-01T02:51:59.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.564+0000 7f092bc28640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0924161950 con 0x7f092415cd90 2023-12-01T02:51:59.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.564+0000 7f09291c3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f09240b5fa0 0x7f0924161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:59.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.565+0000 7f09291c3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f09240b5fa0 0x7f0924161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39794/0 (socket says 172.21.15.115:39794) 2023-12-01T02:51:59.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.565+0000 7f09291c3640 1 -- 172.21.15.115:0/995069899 learned_addr learned my addr 172.21.15.115:0/995069899 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:59.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.565+0000 7f09299c4640 1 --2- 172.21.15.115:0/995069899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092415cd90 0x7f09240cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:59.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.565+0000 7f09291c3640 1 -- 172.21.15.115:0/995069899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092415cd90 msgr2=0x7f09240cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:59.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.565+0000 7f09291c3640 1 --2- 172.21.15.115:0/995069899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092415cd90 0x7f09240cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.565+0000 7f09291c3640 1 -- 172.21.15.115:0/995069899 >> v1:172.21.15.142:6789/0 conn(0x7f09240cf650 legacy=0x7f09240b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:51:59.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.565+0000 7f09291c3640 1 -- 172.21.15.115:0/995069899 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09240ce300 con 0x7f09240b5fa0 2023-12-01T02:51:59.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.565+0000 7f09299c4640 1 --2- 172.21.15.115:0/995069899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092415cd90 0x7f09240cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.566+0000 7f09291c3640 1 --2- 172.21.15.115:0/995069899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 0x7f0924161410 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0918009870 tx=0x7f091802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73258d90804d9ba0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.566+0000 7f09289c2640 1 -- 172.21.15.115:0/995069899 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918031640 con 0x7f09240b5fa0 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.566+0000 7f09289c2640 1 -- 172.21.15.115:0/995069899 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09180317a0 con 0x7f09240b5fa0 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.566+0000 7f09289c2640 1 -- 172.21.15.115:0/995069899 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918031b30 con 0x7f09240b5fa0 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.567+0000 7f092bc28640 1 -- 172.21.15.115:0/995069899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 msgr2=0x7f0924161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.567+0000 7f092bc28640 1 --2- 172.21.15.115:0/995069899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 0x7f0924161410 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0918009870 tx=0x7f091802b8f0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.567+0000 7f092bc28640 1 -- 172.21.15.115:0/995069899 shutdown_connections 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.567+0000 7f092bc28640 1 --2- 172.21.15.115:0/995069899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 0x7f0924161410 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.567+0000 7f092bc28640 1 --2- 172.21.15.115:0/995069899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092415cd90 0x7f09240cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.567+0000 7f092bc28640 1 -- 172.21.15.115:0/995069899 >> 172.21.15.115:0/995069899 conn(0x7f0924065960 msgr2=0x7f0924065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:59.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.567+0000 7f092bc28640 1 -- 172.21.15.115:0/995069899 shutdown_connections 2023-12-01T02:51:59.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.567+0000 7f092bc28640 1 -- 172.21.15.115:0/995069899 wait complete. 2023-12-01T02:51:59.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.569+0000 7f092bc28640 1 Processor -- start 2023-12-01T02:51:59.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.569+0000 7f092bc28640 1 -- start start 2023-12-01T02:51:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.570+0000 7f092bc28640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 0x7f0924153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.570+0000 7f092bc28640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09240cf650 0x7f092415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.570+0000 7f092bc28640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f092415cd90 0x7f09241f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.570+0000 7f092bc28640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240cbf20 con 0x7f09240b5fa0 2023-12-01T02:51:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.570+0000 7f092bc28640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09241537c0 con 0x7f092415cd90 2023-12-01T02:51:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.570+0000 7f092bc28640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09240cb310 con 0x7f09240cf650 2023-12-01T02:51:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.570+0000 7f09299c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 0x7f0924153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.570+0000 7f09299c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 0x7f0924153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39802/0 (socket says 172.21.15.115:39802) 2023-12-01T02:51:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.570+0000 7f09299c4640 1 -- 172.21.15.115:0/2625888847 learned_addr learned my addr 172.21.15.115:0/2625888847 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:51:59.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f09299c4640 1 -- 172.21.15.115:0/2625888847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09240cf650 msgr2=0x7f092415e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f09299c4640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09240cf650 0x7f092415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f09299c4640 1 -- 172.21.15.115:0/2625888847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f092415cd90 msgr2=0x7f09241f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f09299c4640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f092415cd90 0x7f09241f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f09299c4640 1 -- 172.21.15.115:0/2625888847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09180094d0 con 0x7f09240b5fa0 2023-12-01T02:51:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f09299c4640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 0x7f0924153280 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f091400cf60 tx=0x7f091400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f09227fc640 1 -- 172.21.15.115:0/2625888847 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091400d840 con 0x7f09240b5fa0 2023-12-01T02:51:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f09227fc640 1 -- 172.21.15.115:0/2625888847 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091400d9a0 con 0x7f09240b5fa0 2023-12-01T02:51:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f09227fc640 1 -- 172.21.15.115:0/2625888847 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091400dd30 con 0x7f09240b5fa0 2023-12-01T02:51:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f092bc28640 1 -- 172.21.15.115:0/2625888847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09240cb5f0 con 0x7f09240b5fa0 2023-12-01T02:51:59.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.571+0000 7f092bc28640 1 -- 172.21.15.115:0/2625888847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09240cbbb0 con 0x7f09240b5fa0 2023-12-01T02:51:59.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.573+0000 7f092bc28640 1 -- 172.21.15.115:0/2625888847 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092404e890 con 0x7f09240b5fa0 2023-12-01T02:51:59.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.574+0000 7f09227fc640 1 -- 172.21.15.115:0/2625888847 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 89) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f0914015440 con 0x7f09240b5fa0 2023-12-01T02:51:59.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.575+0000 7f09227fc640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f091006a2d0 0x7f091006c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:51:59.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.575+0000 7f09227fc640 1 -- 172.21.15.115:0/2625888847 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f09140891b0 con 0x7f09240b5fa0 2023-12-01T02:51:59.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.575+0000 7f09291c3640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f091006a2d0 0x7f091006c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:51:59.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.576+0000 7f09291c3640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f091006a2d0 0x7f091006c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0918036000 tx=0x7f0918030000 comp rx=0 tx=0).ready entity=mgr.9213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:51:59.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.580+0000 7f09227fc640 1 -- 172.21.15.115:0/2625888847 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0914054930 con 0x7f09240b5fa0 2023-12-01T02:51:59.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.792+0000 7f092bc28640 1 -- 172.21.15.115:0/2625888847 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f09240d1380 con 0x7f091006a2d0 2023-12-01T02:51:59.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.923+0000 7f09227fc640 1 -- 172.21.15.115:0/2625888847 <== mgr.9213 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0918031a10 con 0x7f091006a2d0 2023-12-01T02:51:59.925 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path snap_test_dir1 2023-12-01T02:51:59.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 -- 172.21.15.115:0/2625888847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f091006a2d0 msgr2=0x7f091006c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:59.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f091006a2d0 0x7f091006c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0918036000 tx=0x7f0918030000 comp rx=0 tx=0).stop 2023-12-01T02:51:59.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 -- 172.21.15.115:0/2625888847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 msgr2=0x7f0924153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:51:59.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 0x7f0924153280 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f091400cf60 tx=0x7f091400cf90 comp rx=0 tx=0).stop 2023-12-01T02:51:59.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 -- 172.21.15.115:0/2625888847 shutdown_connections 2023-12-01T02:51:59.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f091006a2d0 0x7f091006c790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f09240cf650 0x7f092415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f092415cd90 0x7f09241f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 --2- 172.21.15.115:0/2625888847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09240b5fa0 0x7f0924153280 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:51:59.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 -- 172.21.15.115:0/2625888847 >> 172.21.15.115:0/2625888847 conn(0x7f0924065960 msgr2=0x7f092415b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:51:59.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 -- 172.21.15.115:0/2625888847 shutdown_connections 2023-12-01T02:51:59.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:51:59.927+0000 7f092bc28640 1 -- 172.21.15.115:0/2625888847 wait complete. 2023-12-01T02:51:59.951 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-01T02:51:59.951 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-12-01T02:52:00.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.165+0000 7f9f1a3d9640 1 Processor -- start 2023-12-01T02:52:00.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.165+0000 7f9f1a3d9640 1 -- start start 2023-12-01T02:52:00.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.166+0000 7f9f1a3d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1415cd90 0x7f9f140cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.166+0000 7f9f1a3d9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9f140cfd60 0x7f9f14161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.166+0000 7f9f1a3d9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f140ce5e0 con 0x7f9f140b5fa0 2023-12-01T02:52:00.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.166+0000 7f9f1a3d9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f140d0ec0 con 0x7f9f140cfd60 2023-12-01T02:52:00.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.166+0000 7f9f1a3d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f14161950 con 0x7f9f1415cd90 2023-12-01T02:52:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.167+0000 7f9f13fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1415cd90 0x7f9f140cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.167+0000 7f9f18976640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9f140cfd60 0x7f9f14161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.167+0000 7f9f13fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1415cd90 0x7f9f140cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:53410/0 (socket says 172.21.15.115:53410) 2023-12-01T02:52:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.167+0000 7f9f13fff640 1 -- 172.21.15.115:0/3349740732 learned_addr learned my addr 172.21.15.115:0/3349740732 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:00.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.167+0000 7f9f18976640 1 --2- 172.21.15.115:0/3349740732 >> v2:172.21.15.142:3300/0 conn(0x7f9f140cfd60 0x7f9f14161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:55062/0 (socket says 172.21.15.115:55062) 2023-12-01T02:52:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.168+0000 7f9f18976640 1 -- 172.21.15.115:0/3349740732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1415cd90 msgr2=0x7f9f140cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.168+0000 7f9f18976640 1 --2- 172.21.15.115:0/3349740732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1415cd90 0x7f9f140cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.168+0000 7f9f18976640 1 -- 172.21.15.115:0/3349740732 >> v1:172.21.15.142:6789/0 conn(0x7f9f140b5fa0 legacy=0x7f9f140cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.168+0000 7f9f18976640 1 -- 172.21.15.115:0/3349740732 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f140ce300 con 0x7f9f140cfd60 2023-12-01T02:52:00.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.168+0000 7f9f13fff640 1 --2- 172.21.15.115:0/3349740732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1415cd90 0x7f9f140cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T02:52:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.169+0000 7f9f18976640 1 --2- 172.21.15.115:0/3349740732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140cfd60 0x7f9f14161410 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009d90 tx=0x7f9f0c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14305be595015733 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:00.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.170+0000 7f9f12ffd640 1 -- 172.21.15.115:0/3349740732 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c02cc50 con 0x7f9f140cfd60 2023-12-01T02:52:00.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.170+0000 7f9f12ffd640 1 -- 172.21.15.115:0/3349740732 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c02cdb0 con 0x7f9f140cfd60 2023-12-01T02:52:00.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.170+0000 7f9f12ffd640 1 -- 172.21.15.115:0/3349740732 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c034660 con 0x7f9f140cfd60 2023-12-01T02:52:00.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.171+0000 7f9f1a3d9640 1 -- 172.21.15.115:0/3349740732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140cfd60 msgr2=0x7f9f14161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:00.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.171+0000 7f9f1a3d9640 1 --2- 172.21.15.115:0/3349740732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140cfd60 0x7f9f14161410 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009d90 tx=0x7f9f0c02bea0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.171+0000 7f9f1a3d9640 1 -- 172.21.15.115:0/3349740732 shutdown_connections 2023-12-01T02:52:00.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.171+0000 7f9f1a3d9640 1 --2- 172.21.15.115:0/3349740732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140cfd60 0x7f9f14161410 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.171+0000 7f9f1a3d9640 1 --2- 172.21.15.115:0/3349740732 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f1415cd90 0x7f9f140cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.171+0000 7f9f1a3d9640 1 -- 172.21.15.115:0/3349740732 >> 172.21.15.115:0/3349740732 conn(0x7f9f14065960 msgr2=0x7f9f14065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:00.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.171+0000 7f9f1a3d9640 1 -- 172.21.15.115:0/3349740732 shutdown_connections 2023-12-01T02:52:00.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.171+0000 7f9f1a3d9640 1 -- 172.21.15.115:0/3349740732 wait complete. 2023-12-01T02:52:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.173+0000 7f9f1a3d9640 1 Processor -- start 2023-12-01T02:52:00.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.174+0000 7f9f1a3d9640 1 -- start start 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.175+0000 7f9f1a3d9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f140b5fa0 0x7f9f1415fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.175+0000 7f9f1a3d9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140cfd60 0x7f9f140c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.175+0000 7f9f1a3d9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f1415cd90 0x7f9f141f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.175+0000 7f9f1a3d9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f140c9d90 con 0x7f9f1415cd90 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.175+0000 7f9f1a3d9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f140c94a0 con 0x7f9f140cfd60 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.175+0000 7f9f1a3d9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f141f2f40 con 0x7f9f140b5fa0 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.175+0000 7f9f18976640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f1415cd90 0x7f9f141f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.176+0000 7f9f18976640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f1415cd90 0x7f9f141f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39812/0 (socket says 172.21.15.115:39812) 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.176+0000 7f9f18976640 1 -- 172.21.15.115:0/3733629292 learned_addr learned my addr 172.21.15.115:0/3733629292 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.176+0000 7f9f13fff640 1 --2- 172.21.15.115:0/3733629292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f140b5fa0 0x7f9f1415fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.176+0000 7f9f137fe640 1 --2- 172.21.15.115:0/3733629292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140cfd60 0x7f9f140c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.176+0000 7f9f18976640 1 -- 172.21.15.115:0/3733629292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f140b5fa0 msgr2=0x7f9f1415fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:00.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.176+0000 7f9f18976640 1 --2- 172.21.15.115:0/3733629292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9f140b5fa0 0x7f9f1415fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.176+0000 7f9f18976640 1 -- 172.21.15.115:0/3733629292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140cfd60 msgr2=0x7f9f140c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:00.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.176+0000 7f9f18976640 1 --2- 172.21.15.115:0/3733629292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f140cfd60 0x7f9f140c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.176+0000 7f9f18976640 1 -- 172.21.15.115:0/3733629292 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f040094d0 con 0x7f9f1415cd90 2023-12-01T02:52:00.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.176+0000 7f9f18976640 1 --2- 172.21.15.115:0/3733629292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f1415cd90 0x7f9f141f2a00 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009eb0 tx=0x7f9f0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:00.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.177+0000 7f9f10ff9640 1 -- 172.21.15.115:0/3733629292 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c040650 con 0x7f9f1415cd90 2023-12-01T02:52:00.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.177+0000 7f9f10ff9640 1 -- 172.21.15.115:0/3733629292 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c0407b0 con 0x7f9f1415cd90 2023-12-01T02:52:00.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.177+0000 7f9f10ff9640 1 -- 172.21.15.115:0/3733629292 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c040b60 con 0x7f9f1415cd90 2023-12-01T02:52:00.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.177+0000 7f9f1a3d9640 1 -- 172.21.15.115:0/3733629292 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0c009a70 con 0x7f9f1415cd90 2023-12-01T02:52:00.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.177+0000 7f9f1a3d9640 1 -- 172.21.15.115:0/3733629292 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f141f31a0 con 0x7f9f1415cd90 2023-12-01T02:52:00.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.178+0000 7f9f1a3d9640 1 -- 172.21.15.115:0/3733629292 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1404e890 con 0x7f9f1415cd90 2023-12-01T02:52:00.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.179+0000 7f9f10ff9640 1 -- 172.21.15.115:0/3733629292 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 90) v1 ==== 88727+0+0 (secure 0 0 0) 0x7f9f0c040cc0 con 0x7f9f1415cd90 2023-12-01T02:52:00.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.180+0000 7f9f10ff9640 1 --2- 172.21.15.115:0/3733629292 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9f0006a5d0 0x7f9f0006ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.181+0000 7f9f10ff9640 1 -- 172.21.15.115:0/3733629292 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f9f0c0ab900 con 0x7f9f1415cd90 2023-12-01T02:52:00.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.181+0000 7f9f13fff640 1 --2- 172.21.15.115:0/3733629292 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9f0006a5d0 0x7f9f0006ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.181+0000 7f9f13fff640 1 --2- 172.21.15.115:0/3733629292 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9f0006a5d0 0x7f9f0006ca90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009c50 tx=0x7f9f04008040 comp rx=0 tx=0).ready entity=mgr.9213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:00.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.185+0000 7f9f10ff9640 1 -- 172.21.15.115:0/3733629292 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9f0c076ce0 con 0x7f9f1415cd90 2023-12-01T02:52:00.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.404+0000 7f9f1a3d9640 1 -- 172.21.15.115:0/3733629292 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f140d1380 con 0x7f9f0006a5d0 2023-12-01T02:52:00.421 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:00.418+0000 7f03b0eb8640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-01T02:52:00.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.418+0000 7f9f10ff9640 1 -- 172.21.15.115:0/3733629292 <== mgr.9213 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f9f040098a0 con 0x7f9f0006a5d0 2023-12-01T02:52:00.422 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-01T02:52:00.434 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-01T02:52:00.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:00.435 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-01T02:52:00.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T02:52:00.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.695+0000 7fc0b2920640 1 Processor -- start 2023-12-01T02:52:00.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.695+0000 7fc0b2920640 1 -- start start 2023-12-01T02:52:00.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.696+0000 7fc0b2920640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc0ac15cd90 0x7fc0ac0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.696+0000 7fc0b2920640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc0ac0b5fa0 0x7fc0ac161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.696+0000 7fc0b2920640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0ac0cf650 0x7fc0ac0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.696+0000 7fc0b2920640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ac0d0ec0 con 0x7fc0ac15cd90 2023-12-01T02:52:00.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.696+0000 7fc0b2920640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ac0ce5e0 con 0x7fc0ac0b5fa0 2023-12-01T02:52:00.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.696+0000 7fc0b2920640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac161950 con 0x7fc0ac0cf650 2023-12-01T02:52:00.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.696+0000 7fc0abfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc0ac15cd90 0x7fc0ac0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.697+0000 7fc0abfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc0ac15cd90 0x7fc0ac0cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39842/0 (socket says 172.21.15.115:39842) 2023-12-01T02:52:00.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.697+0000 7fc0abfff640 1 -- 172.21.15.115:0/2368180551 learned_addr learned my addr 172.21.15.115:0/2368180551 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:00.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.697+0000 7fc0abfff640 1 -- 172.21.15.115:0/2368180551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0ac0cf650 msgr2=0x7fc0ac0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:00.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.697+0000 7fc0abfff640 1 --2- 172.21.15.115:0/2368180551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0ac0cf650 0x7fc0ac0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.697+0000 7fc0abfff640 1 -- 172.21.15.115:0/2368180551 >> v2:172.21.15.142:3300/0 conn(0x7fc0ac0b5fa0 msgr2=0x7fc0ac161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:00.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.697+0000 7fc0abfff640 1 --2- 172.21.15.115:0/2368180551 >> v2:172.21.15.142:3300/0 conn(0x7fc0ac0b5fa0 0x7fc0ac161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.697+0000 7fc0abfff640 1 -- 172.21.15.115:0/2368180551 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac0cffc0 con 0x7fc0ac15cd90 2023-12-01T02:52:00.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.698+0000 7fc0abfff640 1 --2- 172.21.15.115:0/2368180551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 0x7fc0ac0cdd20 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0d0760 tx=0x7fc09c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63a1172a3ba0ed4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:00.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.698+0000 7fc0aaffd640 1 -- 172.21.15.115:0/2368180551 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c031870 con 0x7fc0ac15cd90 2023-12-01T02:52:00.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.698+0000 7fc0aaffd640 1 -- 172.21.15.115:0/2368180551 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c0319d0 con 0x7fc0ac15cd90 2023-12-01T02:52:00.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.698+0000 7fc0aaffd640 1 -- 172.21.15.115:0/2368180551 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c031d60 con 0x7fc0ac15cd90 2023-12-01T02:52:00.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.699+0000 7fc0b2920640 1 -- 172.21.15.115:0/2368180551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 msgr2=0x7fc0ac0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:00.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.699+0000 7fc0b2920640 1 --2- 172.21.15.115:0/2368180551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 0x7fc0ac0cdd20 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0d0760 tx=0x7fc09c02b910 comp rx=0 tx=0).stop 2023-12-01T02:52:00.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.699+0000 7fc0b2920640 1 -- 172.21.15.115:0/2368180551 shutdown_connections 2023-12-01T02:52:00.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.699+0000 7fc0b2920640 1 --2- 172.21.15.115:0/2368180551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0ac0cf650 0x7fc0ac0cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.699+0000 7fc0b2920640 1 --2- 172.21.15.115:0/2368180551 >> v2:172.21.15.142:3300/0 conn(0x7fc0ac0b5fa0 0x7fc0ac161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.699+0000 7fc0b2920640 1 --2- 172.21.15.115:0/2368180551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 0x7fc0ac0cdd20 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.699+0000 7fc0b2920640 1 -- 172.21.15.115:0/2368180551 >> 172.21.15.115:0/2368180551 conn(0x7fc0ac065960 msgr2=0x7fc0ac065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:00.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.699+0000 7fc0b2920640 1 -- 172.21.15.115:0/2368180551 shutdown_connections 2023-12-01T02:52:00.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.699+0000 7fc0b2920640 1 -- 172.21.15.115:0/2368180551 wait complete. 2023-12-01T02:52:00.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.701+0000 7fc0b2920640 1 Processor -- start 2023-12-01T02:52:00.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.701+0000 7fc0b2920640 1 -- start start 2023-12-01T02:52:00.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.702+0000 7fc0b2920640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0ac0b5fa0 0x7fc0ac0ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.702+0000 7fc0b2920640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0ac0cf650 0x7fc0ac1f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.702+0000 7fc0b2920640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 0x7fc0ac1f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.702+0000 7fc0b2920640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cafd0 con 0x7fc0ac15cd90 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.702+0000 7fc0b2920640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cb2b0 con 0x7fc0ac0b5fa0 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.702+0000 7fc0b2920640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cb590 con 0x7fc0ac0cf650 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0b0ebd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 0x7fc0ac1f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0b0ebd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 0x7fc0ac1f2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39846/0 (socket says 172.21.15.115:39846) 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0b0ebd640 1 -- 172.21.15.115:0/1499615956 learned_addr learned my addr 172.21.15.115:0/1499615956 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0abfff640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0ac0b5fa0 0x7fc0ac0ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0b0ebd640 1 -- 172.21.15.115:0/1499615956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0ac0cf650 msgr2=0x7fc0ac1f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0ab7fe640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0ac0cf650 0x7fc0ac1f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0b0ebd640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0ac0cf650 0x7fc0ac1f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0b0ebd640 1 -- 172.21.15.115:0/1499615956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0ac0b5fa0 msgr2=0x7fc0ac0ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0b0ebd640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0ac0b5fa0 0x7fc0ac0ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0b0ebd640 1 -- 172.21.15.115:0/1499615956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c0094d0 con 0x7fc0ac15cd90 2023-12-01T02:52:00.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0abfff640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0ac0b5fa0 0x7fc0ac0ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:00.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.703+0000 7fc0b0ebd640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 0x7fc0ac1f2870 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009fd0 tx=0x7fc0a400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:00.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.704+0000 7fc0a8ff9640 1 -- 172.21.15.115:0/1499615956 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4011830 con 0x7fc0ac15cd90 2023-12-01T02:52:00.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.704+0000 7fc0a8ff9640 1 -- 172.21.15.115:0/1499615956 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a4011990 con 0x7fc0ac15cd90 2023-12-01T02:52:00.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.704+0000 7fc0a8ff9640 1 -- 172.21.15.115:0/1499615956 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4011d20 con 0x7fc0ac15cd90 2023-12-01T02:52:00.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.704+0000 7fc0b2920640 1 -- 172.21.15.115:0/1499615956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac0a9570 con 0x7fc0ac15cd90 2023-12-01T02:52:00.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.704+0000 7fc0b2920640 1 -- 172.21.15.115:0/1499615956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac0cac90 con 0x7fc0ac15cd90 2023-12-01T02:52:00.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.705+0000 7fc0b2920640 1 -- 172.21.15.115:0/1499615956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ac0a9a90 con 0x7fc0ac15cd90 2023-12-01T02:52:00.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.706+0000 7fc0a8ff9640 1 -- 172.21.15.115:0/1499615956 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 90) v1 ==== 88727+0+0 (secure 0 0 0) 0x7fc0a401b460 con 0x7fc0ac15cd90 2023-12-01T02:52:00.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.707+0000 7fc0a8ff9640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc08c06a5a0 0x7fc08c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:00.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.707+0000 7fc0a8ff9640 1 -- 172.21.15.115:0/1499615956 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fc0a4088a90 con 0x7fc0ac15cd90 2023-12-01T02:52:00.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.707+0000 7fc0abfff640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc08c06a5a0 0x7fc08c06ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:00.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.708+0000 7fc0abfff640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc08c06a5a0 0x7fc08c06ca60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc09c036000 tx=0x7fc09c03c040 comp rx=0 tx=0).ready entity=mgr.9213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:00.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.711+0000 7fc0a8ff9640 1 -- 172.21.15.115:0/1499615956 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc0ac0a9a90 con 0x7fc0ac15cd90 2023-12-01T02:52:00.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:00.949+0000 7fc0b2920640 1 -- 172.21.15.115:0/1499615956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fc0ac0a9a90 con 0x7fc0ac15cd90 2023-12-01T02:52:01.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.177+0000 7fc0a8ff9640 1 -- 172.21.15.115:0/1499615956 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 91) v1 ==== 88710+0+0 (secure 0 0 0) 0x7fc0a4086e20 con 0x7fc0ac15cd90 2023-12-01T02:52:01.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.179+0000 7fc0a8ff9640 1 -- 172.21.15.115:0/1499615956 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v91) v1 ==== 93+0+0 (secure 0 0 0) 0x7fc0a4072470 con 0x7fc0ac15cd90 2023-12-01T02:52:01.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 -- 172.21.15.115:0/1499615956 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc08c06a5a0 msgr2=0x7fc08c06ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:01.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc08c06a5a0 0x7fc08c06ca60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc09c036000 tx=0x7fc09c03c040 comp rx=0 tx=0).stop 2023-12-01T02:52:01.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 -- 172.21.15.115:0/1499615956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 msgr2=0x7fc0ac1f2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:01.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 0x7fc0ac1f2870 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009fd0 tx=0x7fc0a400b8e0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 -- 172.21.15.115:0/1499615956 shutdown_connections 2023-12-01T02:52:01.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc08c06a5a0 0x7fc08c06ca60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0ac0cf650 0x7fc0ac1f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac15cd90 0x7fc0ac1f2870 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 --2- 172.21.15.115:0/1499615956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0ac0b5fa0 0x7fc0ac0ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 -- 172.21.15.115:0/1499615956 >> 172.21.15.115:0/1499615956 conn(0x7fc0ac065960 msgr2=0x7fc0ac1590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:01.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 -- 172.21.15.115:0/1499615956 shutdown_connections 2023-12-01T02:52:01.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.182+0000 7fc0b2920640 1 -- 172.21.15.115:0/1499615956 wait complete. 2023-12-01T02:52:01.203 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T02:52:01.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T02:52:01.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T02:52:01.391 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T02:52:01.391 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:52:01.391 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:52:01.442 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:52:01.442 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:52:01.537 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:52:01.707 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:01.705+0000 7f50e5e58040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T02:52:01.723 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:01.722+0000 7fa03f956040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T02:52:01.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.735+0000 7f1708bf0640 1 Processor -- start 2023-12-01T02:52:01.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.735+0000 7f1708bf0640 1 -- start start 2023-12-01T02:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.736+0000 7f1708bf0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f170415cd20 0x7f17040d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.736+0000 7f1708bf0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f17040d1230 0x7f17040d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.736+0000 7f1708bf0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17040ce2e0 con 0x7f17040b5d80 2023-12-01T02:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.736+0000 7f1708bf0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17040b2160 con 0x7f17040d1230 2023-12-01T02:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.736+0000 7f1708bf0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17040d1be0 con 0x7f170415cd20 2023-12-01T02:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.737+0000 7f1702d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f17040d1230 0x7f17040d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.737+0000 7f1702d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f17040d1230 0x7f17040d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39864/0 (socket says 172.21.15.115:39864) 2023-12-01T02:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.737+0000 7f1702d9d640 1 -- 172.21.15.115:0/925208361 learned_addr learned my addr 172.21.15.115:0/925208361 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.737+0000 7f170259c640 1 --2- 172.21.15.115:0/925208361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f170415cd20 0x7f17040d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:01.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.737+0000 7f1702d9d640 1 -- 172.21.15.115:0/925208361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f170415cd20 msgr2=0x7f17040d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:01.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.737+0000 7f1702d9d640 1 --2- 172.21.15.115:0/925208361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f170415cd20 0x7f17040d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.737+0000 7f1702d9d640 1 -- 172.21.15.115:0/925208361 >> v1:172.21.15.142:6789/0 conn(0x7f17040b5d80 legacy=0x7f17040b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:01.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.737+0000 7f1702d9d640 1 -- 172.21.15.115:0/925208361 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17040ce000 con 0x7f17040d1230 2023-12-01T02:52:01.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.738+0000 7f1702d9d640 1 --2- 172.21.15.115:0/925208361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040d1230 0x7f17040d16a0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009e50 tx=0x7f16fc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc79f2e182c524f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:01.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.738+0000 7f1701d9b640 1 -- 172.21.15.115:0/925208361 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc02c9e0 con 0x7f17040d1230 2023-12-01T02:52:01.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.738+0000 7f1701d9b640 1 -- 172.21.15.115:0/925208361 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc02cb40 con 0x7f17040d1230 2023-12-01T02:52:01.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.738+0000 7f1701d9b640 1 -- 172.21.15.115:0/925208361 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0343f0 con 0x7f17040d1230 2023-12-01T02:52:01.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.739+0000 7f1708bf0640 1 -- 172.21.15.115:0/925208361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040d1230 msgr2=0x7f17040d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:01.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.739+0000 7f1708bf0640 1 --2- 172.21.15.115:0/925208361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040d1230 0x7f17040d16a0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009e50 tx=0x7f16fc02be40 comp rx=0 tx=0).stop 2023-12-01T02:52:01.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.739+0000 7f1708bf0640 1 -- 172.21.15.115:0/925208361 shutdown_connections 2023-12-01T02:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.739+0000 7f1708bf0640 1 --2- 172.21.15.115:0/925208361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040d1230 0x7f17040d16a0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.739+0000 7f1708bf0640 1 --2- 172.21.15.115:0/925208361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f170415cd20 0x7f17040d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.739+0000 7f1708bf0640 1 -- 172.21.15.115:0/925208361 >> 172.21.15.115:0/925208361 conn(0x7f170406a0e0 msgr2=0x7f170406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.739+0000 7f1708bf0640 1 -- 172.21.15.115:0/925208361 shutdown_connections 2023-12-01T02:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.739+0000 7f1708bf0640 1 -- 172.21.15.115:0/925208361 wait complete. 2023-12-01T02:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.741+0000 7f1708bf0640 1 Processor -- start 2023-12-01T02:52:01.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.741+0000 7f1708bf0640 1 -- start start 2023-12-01T02:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.742+0000 7f1708bf0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040b5d80 0x7f17040b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.742+0000 7f1708bf0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17040d1230 0x7f17040c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.742+0000 7f1708bf0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f170415cd20 0x7f1704156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.742+0000 7f1708bf0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1704156100 con 0x7f17040b5d80 2023-12-01T02:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.742+0000 7f1708bf0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17040b3550 con 0x7f17040d1230 2023-12-01T02:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.742+0000 7f1708bf0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17041554f0 con 0x7f170415cd20 2023-12-01T02:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.742+0000 7f170259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040b5d80 0x7f17040b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.742+0000 7f170259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040b5d80 0x7f17040b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39876/0 (socket says 172.21.15.115:39876) 2023-12-01T02:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.742+0000 7f170259c640 1 -- 172.21.15.115:0/332624586 learned_addr learned my addr 172.21.15.115:0/332624586 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f1702d9d640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f170415cd20 0x7f1704156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f170259c640 1 -- 172.21.15.115:0/332624586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f170415cd20 msgr2=0x7f1704156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f16fbfff640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17040d1230 0x7f17040c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f170259c640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f170415cd20 0x7f1704156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f170259c640 1 -- 172.21.15.115:0/332624586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17040d1230 msgr2=0x7f17040c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f170259c640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17040d1230 0x7f17040c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f170259c640 1 -- 172.21.15.115:0/332624586 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc009b30 con 0x7f17040b5d80 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f16fbfff640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17040d1230 0x7f17040c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f170259c640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040b5d80 0x7f17040b3010 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f16f400d5f0 tx=0x7f16f400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f16faffd640 1 -- 172.21.15.115:0/332624586 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f40125f0 con 0x7f17040b5d80 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f16faffd640 1 -- 172.21.15.115:0/332624586 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f4012750 con 0x7f17040b5d80 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f16faffd640 1 -- 172.21.15.115:0/332624586 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4012b00 con 0x7f17040b5d80 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f1708bf0640 1 -- 172.21.15.115:0/332624586 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1704155760 con 0x7f17040b5d80 2023-12-01T02:52:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.743+0000 7f1708bf0640 1 -- 172.21.15.115:0/332624586 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1704155d20 con 0x7f17040b5d80 2023-12-01T02:52:01.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.745+0000 7f1708bf0640 1 -- 172.21.15.115:0/332624586 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1704156370 con 0x7f17040b5d80 2023-12-01T02:52:01.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.745+0000 7f16faffd640 1 -- 172.21.15.115:0/332624586 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 91) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f16f4012c60 con 0x7f17040b5d80 2023-12-01T02:52:01.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.746+0000 7f16faffd640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16e406a550 0x7f16e406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:01.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.746+0000 7f16faffd640 1 -- 172.21.15.115:0/332624586 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f16f4088fe0 con 0x7f17040b5d80 2023-12-01T02:52:01.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.746+0000 7f16fbfff640 1 -- 172.21.15.115:0/332624586 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16e406a550 msgr2=0x7f16e406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:01.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.746+0000 7f16fbfff640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16e406a550 0x7f16e406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:01.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.750+0000 7f16faffd640 1 -- 172.21.15.115:0/332624586 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f16f40543c0 con 0x7f17040b5d80 2023-12-01T02:52:01.764 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:01.762+0000 7f50e5e58040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T02:52:01.781 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:01.780+0000 7fa03f956040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T02:52:01.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.941+0000 7f1708bf0640 1 -- 172.21.15.115:0/332624586 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f170404e890 con 0x7f17040b5d80 2023-12-01T02:52:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.943+0000 7f16faffd640 1 -- 172.21.15.115:0/332624586 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+16675 (secure 0 0 0) 0x7f16f40579e0 con 0x7f17040b5d80 2023-12-01T02:52:01.945 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:01.946 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":164,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:51:58.123244+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:51:41.383835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-01T02:51:42.832726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/2871788797":"2023-12-02T02:51:58.123217+0000","172.21.15.115:6822/88317":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/2571267301":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/1473681116":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/3088218027":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/500113028":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/1225510452":"2023-12-02T02:51:58.123217+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:52:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 -- 172.21.15.115:0/332624586 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16e406a550 msgr2=0x7f16e406ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16e406a550 0x7f16e406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 -- 172.21.15.115:0/332624586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040b5d80 msgr2=0x7f17040b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040b5d80 0x7f17040b3010 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f16f400d5f0 tx=0x7f16f400dab0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 -- 172.21.15.115:0/332624586 shutdown_connections 2023-12-01T02:52:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f16e406a550 0x7f16e406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f170415cd20 0x7f1704156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17040d1230 0x7f17040c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 --2- 172.21.15.115:0/332624586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17040b5d80 0x7f17040b3010 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 -- 172.21.15.115:0/332624586 >> 172.21.15.115:0/332624586 conn(0x7f170406a0e0 msgr2=0x7f170415b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 -- 172.21.15.115:0/332624586 shutdown_connections 2023-12-01T02:52:01.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:01.946+0000 7f1708bf0640 1 -- 172.21.15.115:0/332624586 wait complete. 2023-12-01T02:52:01.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T02:51:41.383835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T02:51:42.832726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T02:52:01.968 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T02:52:02.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.173+0000 7f65cbfff640 1 Processor -- start 2023-12-01T02:52:02.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.173+0000 7f65cbfff640 1 -- start start 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65cbfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f65cc153c30 0x7f65cc0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65cbfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65cc154010 0x7f65cc0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65cbfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f65cc0cf9b0 0x7f65cc160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65cbfff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65cc0cdcd0 con 0x7f65cc0cf9b0 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65cbfff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65cc0cfe20 con 0x7f65cc153c30 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65cbfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc161830 con 0x7f65cc154010 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65cb7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f65cc0cf9b0 0x7f65cc160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65cb7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f65cc0cf9b0 0x7f65cc160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39890/0 (socket says 172.21.15.115:39890) 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65cb7fe640 1 -- 172.21.15.115:0/4059995718 learned_addr learned my addr 172.21.15.115:0/4059995718 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65caffd640 1 --2- 172.21.15.115:0/4059995718 >> v2:172.21.15.142:3300/0 conn(0x7f65cc153c30 0x7f65cc0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.175+0000 7f65ca7fc640 1 --2- 172.21.15.115:0/4059995718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65cc154010 0x7f65cc0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:02.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.176+0000 7f65cb7fe640 1 -- 172.21.15.115:0/4059995718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65cc154010 msgr2=0x7f65cc0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:02.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.176+0000 7f65cb7fe640 1 --2- 172.21.15.115:0/4059995718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65cc154010 0x7f65cc0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.176+0000 7f65cb7fe640 1 -- 172.21.15.115:0/4059995718 >> v2:172.21.15.142:3300/0 conn(0x7f65cc153c30 msgr2=0x7f65cc0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:02.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.176+0000 7f65cb7fe640 1 --2- 172.21.15.115:0/4059995718 >> v2:172.21.15.142:3300/0 conn(0x7f65cc153c30 0x7f65cc0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.176+0000 7f65cb7fe640 1 -- 172.21.15.115:0/4059995718 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc0cd3a0 con 0x7f65cc0cf9b0 2023-12-01T02:52:02.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.176+0000 7f65caffd640 1 --2- 172.21.15.115:0/4059995718 >> v2:172.21.15.142:3300/0 conn(0x7f65cc153c30 0x7f65cc0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T02:52:02.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.177+0000 7f65cb7fe640 1 --2- 172.21.15.115:0/4059995718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc0cf9b0 0x7f65cc160ac0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0d3800 tx=0x7f65c402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0e8527d657067a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:02.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.177+0000 7f65c9ffb640 1 -- 172.21.15.115:0/4059995718 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c402ca70 con 0x7f65cc0cf9b0 2023-12-01T02:52:02.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.177+0000 7f65c9ffb640 1 -- 172.21.15.115:0/4059995718 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c402cbd0 con 0x7f65cc0cf9b0 2023-12-01T02:52:02.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.177+0000 7f65c9ffb640 1 -- 172.21.15.115:0/4059995718 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c40343f0 con 0x7f65cc0cf9b0 2023-12-01T02:52:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.178+0000 7f65cbfff640 1 -- 172.21.15.115:0/4059995718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc0cf9b0 msgr2=0x7f65cc160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.178+0000 7f65cbfff640 1 --2- 172.21.15.115:0/4059995718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc0cf9b0 0x7f65cc160ac0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0d3800 tx=0x7f65c402bed0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.178+0000 7f65cbfff640 1 -- 172.21.15.115:0/4059995718 shutdown_connections 2023-12-01T02:52:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.178+0000 7f65cbfff640 1 --2- 172.21.15.115:0/4059995718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc0cf9b0 0x7f65cc160ac0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.178+0000 7f65cbfff640 1 --2- 172.21.15.115:0/4059995718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65cc154010 0x7f65cc0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.178+0000 7f65cbfff640 1 --2- 172.21.15.115:0/4059995718 >> v2:172.21.15.142:3300/0 conn(0x7f65cc153c30 0x7f65cc0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.178+0000 7f65cbfff640 1 -- 172.21.15.115:0/4059995718 >> 172.21.15.115:0/4059995718 conn(0x7f65cc06a0b0 msgr2=0x7f65cc06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.178+0000 7f65cbfff640 1 -- 172.21.15.115:0/4059995718 shutdown_connections 2023-12-01T02:52:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.178+0000 7f65cbfff640 1 -- 172.21.15.115:0/4059995718 wait complete. 2023-12-01T02:52:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.180+0000 7f65cbfff640 1 Processor -- start 2023-12-01T02:52:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.180+0000 7f65cbfff640 1 -- start start 2023-12-01T02:52:02.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.181+0000 7f65cbfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65cc0cf9b0 0x7f65cc0c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.181+0000 7f65cbfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc153c30 0x7f65cc0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.181+0000 7f65cbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc154010 0x7f65cc157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.181+0000 7f65cbfff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc0cac50 con 0x7f65cc154010 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.181+0000 7f65cbfff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc0ca620 con 0x7f65cc153c30 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.181+0000 7f65cbfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc06a5c0 con 0x7f65cc0cf9b0 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.181+0000 7f65cb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc154010 0x7f65cc157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.181+0000 7f65cb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc154010 0x7f65cc157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39904/0 (socket says 172.21.15.115:39904) 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.181+0000 7f65cb7fe640 1 -- 172.21.15.115:0/688500465 learned_addr learned my addr 172.21.15.115:0/688500465 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.181+0000 7f65caffd640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65cc0cf9b0 0x7f65cc0c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65ca7fc640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc153c30 0x7f65cc0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65cb7fe640 1 -- 172.21.15.115:0/688500465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65cc0cf9b0 msgr2=0x7f65cc0c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65cb7fe640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65cc0cf9b0 0x7f65cc0c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65cb7fe640 1 -- 172.21.15.115:0/688500465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc153c30 msgr2=0x7f65cc0cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65cb7fe640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc153c30 0x7f65cc0cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65cb7fe640 1 -- 172.21.15.115:0/688500465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c4009b30 con 0x7f65cc154010 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65ca7fc640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc153c30 0x7f65cc0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65cb7fe640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc154010 0x7f65cc157b20 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f65c4036040 tx=0x7f65c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:02.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65b3fff640 1 -- 172.21.15.115:0/688500465 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4009510 con 0x7f65cc154010 2023-12-01T02:52:02.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65b3fff640 1 -- 172.21.15.115:0/688500465 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c403e450 con 0x7f65cc154010 2023-12-01T02:52:02.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65cbfff640 1 -- 172.21.15.115:0/688500465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc06a8a0 con 0x7f65cc154010 2023-12-01T02:52:02.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.182+0000 7f65b3fff640 1 -- 172.21.15.115:0/688500465 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c403e800 con 0x7f65cc154010 2023-12-01T02:52:02.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.183+0000 7f65cbfff640 1 -- 172.21.15.115:0/688500465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc0d1810 con 0x7f65cc154010 2023-12-01T02:52:02.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.184+0000 7f65cbfff640 1 -- 172.21.15.115:0/688500465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65cc0cae00 con 0x7f65cc154010 2023-12-01T02:52:02.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.185+0000 7f65b3fff640 1 -- 172.21.15.115:0/688500465 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 91) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f65c403e960 con 0x7f65cc154010 2023-12-01T02:52:02.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.186+0000 7f65b3fff640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f65b806a550 0x7f65b806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.186+0000 7f65b3fff640 1 -- 172.21.15.115:0/688500465 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f65c40ab9c0 con 0x7f65cc154010 2023-12-01T02:52:02.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.186+0000 7f65caffd640 1 -- 172.21.15.115:0/688500465 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f65b806a550 msgr2=0x7f65b806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:02.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.186+0000 7f65caffd640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f65b806a550 0x7f65b806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:02.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.191+0000 7f65b3fff640 1 -- 172.21.15.115:0/688500465 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f65cc0cae00 con 0x7f65cc154010 2023-12-01T02:52:02.233 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:02.231+0000 7f50e5e58040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T02:52:02.233 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:02.232+0000 7fa03f956040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T02:52:02.284 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:02.283+0000 7fa03f956040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T02:52:02.286 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:02.285+0000 7f50e5e58040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T02:52:02.335 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:02.334+0000 7fa03f956040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T02:52:02.339 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:02.337+0000 7f50e5e58040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T02:52:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.380+0000 7f65cbfff640 1 -- 172.21.15.115:0/688500465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65cc0618d0 con 0x7f65cc154010 2023-12-01T02:52:02.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.381+0000 7f65b3fff640 1 -- 172.21.15.115:0/688500465 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v164) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65c4043020 con 0x7f65cc154010 2023-12-01T02:52:02.383 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T02:52:02.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 -- 172.21.15.115:0/688500465 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f65b806a550 msgr2=0x7f65b806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:02.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f65b806a550 0x7f65b806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 -- 172.21.15.115:0/688500465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc154010 msgr2=0x7f65cc157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:02.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc154010 0x7f65cc157b20 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f65c4036040 tx=0x7f65c4004a40 comp rx=0 tx=0).stop 2023-12-01T02:52:02.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 -- 172.21.15.115:0/688500465 shutdown_connections 2023-12-01T02:52:02.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f65b806a550 0x7f65b806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65cc154010 0x7f65cc157b20 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65cc153c30 0x7f65cc0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 --2- 172.21.15.115:0/688500465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65cc0cf9b0 0x7f65cc0c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 -- 172.21.15.115:0/688500465 >> 172.21.15.115:0/688500465 conn(0x7f65cc06a0b0 msgr2=0x7f65cc15d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:02.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 -- 172.21.15.115:0/688500465 shutdown_connections 2023-12-01T02:52:02.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.384+0000 7f65cbfff640 1 -- 172.21.15.115:0/688500465 wait complete. 2023-12-01T02:52:02.395 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:02.394+0000 7fa03f956040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T02:52:02.403 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T02:52:02.404 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:02.401+0000 7f50e5e58040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T02:52:02.444 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:02.443+0000 7fa03f956040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T02:52:02.454 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:02.453+0000 7f50e5e58040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T02:52:02.554 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:02.553+0000 7fa03f956040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T02:52:02.569 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:02.567+0000 7f50e5e58040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T02:52:02.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.604+0000 7fab2bfff640 1 Processor -- start 2023-12-01T02:52:02.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.604+0000 7fab2bfff640 1 -- start start 2023-12-01T02:52:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.606+0000 7fab2bfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fab2c0acd00 0x7fab2c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.606+0000 7fab2bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fab2c0cd120 0x7fab2c0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.606+0000 7fab2bfff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c0c98a0 con 0x7fab2c146440 2023-12-01T02:52:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.606+0000 7fab2bfff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c0adbb0 con 0x7fab2c0cd120 2023-12-01T02:52:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.606+0000 7fab2bfff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c0ade90 con 0x7fab2c0acd00 2023-12-01T02:52:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.606+0000 7fab2b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fab2c0cd120 0x7fab2c0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.606+0000 7fab2b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fab2c0cd120 0x7fab2c0c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39906/0 (socket says 172.21.15.115:39906) 2023-12-01T02:52:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.606+0000 7fab2b7fe640 1 -- 172.21.15.115:0/1507434825 learned_addr learned my addr 172.21.15.115:0/1507434825 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:02.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.606+0000 7fab2affd640 1 --2- 172.21.15.115:0/1507434825 >> v2:172.21.15.142:3300/0 conn(0x7fab2c0acd00 0x7fab2c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.607+0000 7fab2b7fe640 1 -- 172.21.15.115:0/1507434825 >> v2:172.21.15.142:3300/0 conn(0x7fab2c0acd00 msgr2=0x7fab2c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.607+0000 7fab2b7fe640 1 --2- 172.21.15.115:0/1507434825 >> v2:172.21.15.142:3300/0 conn(0x7fab2c0acd00 0x7fab2c0ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.607+0000 7fab2b7fe640 1 -- 172.21.15.115:0/1507434825 >> v1:172.21.15.142:6789/0 conn(0x7fab2c146440 legacy=0x7fab2c0cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.607+0000 7fab2b7fe640 1 -- 172.21.15.115:0/1507434825 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c0ae170 con 0x7fab2c0cd120 2023-12-01T02:52:02.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.608+0000 7fab2b7fe640 1 --2- 172.21.15.115:0/1507434825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0cd120 0x7fab2c0c9d30 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0ae450 tx=0x7fab2402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27776d4785a7f4d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:02.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.608+0000 7fab29ffb640 1 -- 172.21.15.115:0/1507434825 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2402ca80 con 0x7fab2c0cd120 2023-12-01T02:52:02.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.608+0000 7fab29ffb640 1 -- 172.21.15.115:0/1507434825 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2402cbe0 con 0x7fab2c0cd120 2023-12-01T02:52:02.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.608+0000 7fab29ffb640 1 -- 172.21.15.115:0/1507434825 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240343f0 con 0x7fab2c0cd120 2023-12-01T02:52:02.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.609+0000 7fab2bfff640 1 -- 172.21.15.115:0/1507434825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0cd120 msgr2=0x7fab2c0c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.609+0000 7fab2bfff640 1 --2- 172.21.15.115:0/1507434825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0cd120 0x7fab2c0c9d30 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0ae450 tx=0x7fab2402bee0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.609+0000 7fab2bfff640 1 -- 172.21.15.115:0/1507434825 shutdown_connections 2023-12-01T02:52:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.609+0000 7fab2bfff640 1 --2- 172.21.15.115:0/1507434825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0cd120 0x7fab2c0c9d30 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.609+0000 7fab2bfff640 1 --2- 172.21.15.115:0/1507434825 >> v2:172.21.15.142:3300/0 conn(0x7fab2c0acd00 0x7fab2c0ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.609+0000 7fab2bfff640 1 -- 172.21.15.115:0/1507434825 >> 172.21.15.115:0/1507434825 conn(0x7fab2c0cee40 msgr2=0x7fab2c0cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.609+0000 7fab2bfff640 1 -- 172.21.15.115:0/1507434825 shutdown_connections 2023-12-01T02:52:02.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.609+0000 7fab2bfff640 1 -- 172.21.15.115:0/1507434825 wait complete. 2023-12-01T02:52:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.611+0000 7fab2bfff640 1 Processor -- start 2023-12-01T02:52:02.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.611+0000 7fab2bfff640 1 -- start start 2023-12-01T02:52:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0acd00 0x7fab2c1efd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab2c0cd120 0x7fab2c0b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab2c146440 0x7fab2c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2bfff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1f0260 con 0x7fab2c0acd00 2023-12-01T02:52:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2bfff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1f5f50 con 0x7fab2c146440 2023-12-01T02:52:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2bfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1f6230 con 0x7fab2c0cd120 2023-12-01T02:52:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0acd00 0x7fab2c1efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0acd00 0x7fab2c1efd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39918/0 (socket says 172.21.15.115:39918) 2023-12-01T02:52:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2affd640 1 -- 172.21.15.115:0/2407174304 learned_addr learned my addr 172.21.15.115:0/2407174304 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:02.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2affd640 1 -- 172.21.15.115:0/2407174304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab2c0cd120 msgr2=0x7fab2c0b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:02.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2affd640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab2c0cd120 0x7fab2c0b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2affd640 1 -- 172.21.15.115:0/2407174304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab2c146440 msgr2=0x7fab2c070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:02.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2affd640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab2c146440 0x7fab2c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.612+0000 7fab2affd640 1 -- 172.21.15.115:0/2407174304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24009b30 con 0x7fab2c0acd00 2023-12-01T02:52:02.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.613+0000 7fab2affd640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0acd00 0x7fab2c1efd20 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00d5f0 tx=0x7fab1c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:02.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.613+0000 7fab137fe640 1 -- 172.21.15.115:0/2407174304 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0125f0 con 0x7fab2c0acd00 2023-12-01T02:52:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.613+0000 7fab137fe640 1 -- 172.21.15.115:0/2407174304 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c012750 con 0x7fab2c0acd00 2023-12-01T02:52:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.613+0000 7fab137fe640 1 -- 172.21.15.115:0/2407174304 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c012b00 con 0x7fab2c0acd00 2023-12-01T02:52:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.613+0000 7fab2bfff640 1 -- 172.21.15.115:0/2407174304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c1f6510 con 0x7fab2c0acd00 2023-12-01T02:52:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.613+0000 7fab2bfff640 1 -- 172.21.15.115:0/2407174304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c1f6ad0 con 0x7fab2c0acd00 2023-12-01T02:52:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.614+0000 7fab2bfff640 1 -- 172.21.15.115:0/2407174304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c1f6c90 con 0x7fab2c0acd00 2023-12-01T02:52:02.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.615+0000 7fab137fe640 1 -- 172.21.15.115:0/2407174304 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 91) v1 ==== 88710+0+0 (secure 0 0 0) 0x7fab1c012c60 con 0x7fab2c0acd00 2023-12-01T02:52:02.617 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:02.616+0000 7fa03f956040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T02:52:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.616+0000 7fab137fe640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fab1806a550 0x7fab1806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.616+0000 7fab2a7fc640 1 -- 172.21.15.115:0/2407174304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fab1806a550 msgr2=0x7fab1806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.616+0000 7fab137fe640 1 -- 172.21.15.115:0/2407174304 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fab1c0890a0 con 0x7fab2c0acd00 2023-12-01T02:52:02.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.616+0000 7fab2a7fc640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fab1806a550 0x7fab1806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:02.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.621+0000 7fab137fe640 1 -- 172.21.15.115:0/2407174304 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fab1c0543e0 con 0x7fab2c0acd00 2023-12-01T02:52:02.635 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:02.633+0000 7f50e5e58040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T02:52:02.667 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:02.666+0000 7fa03f956040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T02:52:02.687 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:02.685+0000 7f50e5e58040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T02:52:02.724 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:02.723+0000 7fa03f956040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T02:52:02.746 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:02.744+0000 7f50e5e58040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T02:52:02.774 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:02.773+0000 7fa03f956040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T02:52:02.797 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:02.795+0000 7f50e5e58040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T02:52:02.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.810+0000 7fab2bfff640 1 -- 172.21.15.115:0/2407174304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab2c0618d0 con 0x7fab2c0acd00 2023-12-01T02:52:02.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.810+0000 7fab137fe640 1 -- 172.21.15.115:0/2407174304 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v164) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab1c057a00 con 0x7fab2c0acd00 2023-12-01T02:52:02.812 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 -- 172.21.15.115:0/2407174304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fab1806a550 msgr2=0x7fab1806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fab1806a550 0x7fab1806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 -- 172.21.15.115:0/2407174304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0acd00 msgr2=0x7fab2c1efd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0acd00 0x7fab2c1efd20 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00d5f0 tx=0x7fab1c00dab0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 -- 172.21.15.115:0/2407174304 shutdown_connections 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fab1806a550 0x7fab1806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fab2c0cd120 0x7fab2c0b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab2c146440 0x7fab2c070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 --2- 172.21.15.115:0/2407174304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab2c0acd00 0x7fab2c1efd20 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 -- 172.21.15.115:0/2407174304 >> 172.21.15.115:0/2407174304 conn(0x7fab2c0cee40 msgr2=0x7fab2c06f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 -- 172.21.15.115:0/2407174304 shutdown_connections 2023-12-01T02:52:02.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:02.813+0000 7fab2bfff640 1 -- 172.21.15.115:0/2407174304 wait complete. 2023-12-01T02:52:02.833 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T02:52:03.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.033+0000 7fe3926c4640 1 Processor -- start 2023-12-01T02:52:03.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.033+0000 7fe3926c4640 1 -- start start 2023-12-01T02:52:03.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.035+0000 7fe3926c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe38c0d0260 0x7fe38c0b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.035+0000 7fe3926c4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe38c0b18f0 0x7fe38c0b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.035+0000 7fe3926c4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38c0b7760 con 0x7fe38c15cf70 2023-12-01T02:52:03.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.035+0000 7fe3926c4640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38c0b2170 con 0x7fe38c0d0260 2023-12-01T02:52:03.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.035+0000 7fe3926c4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38c0b8b70 con 0x7fe38c0b18f0 2023-12-01T02:52:03.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.035+0000 7fe38bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe38c15cf70 0x7fe38c0cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:47756/0 (socket says 172.21.15.115:47756) 2023-12-01T02:52:03.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.035+0000 7fe38bfff640 1 -- 172.21.15.115:0/2089052428 learned_addr learned my addr 172.21.15.115:0/2089052428 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:03.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.035+0000 7fe38b7fe640 1 --2- 172.21.15.115:0/2089052428 >> v2:172.21.15.115:3300/0 conn(0x7fe38c0d0260 0x7fe38c0b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:03.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.035+0000 7fe390c61640 1 --2- 172.21.15.115:0/2089052428 >> v2:172.21.15.142:3300/0 conn(0x7fe38c0b18f0 0x7fe38c0b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:03.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.035+0000 7fe38b7fe640 1 -- 172.21.15.115:0/2089052428 >> v2:172.21.15.142:3300/0 conn(0x7fe38c0b18f0 msgr2=0x7fe38c0b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:03.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.036+0000 7fe38b7fe640 1 --2- 172.21.15.115:0/2089052428 >> v2:172.21.15.142:3300/0 conn(0x7fe38c0b18f0 0x7fe38c0b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.036+0000 7fe38b7fe640 1 -- 172.21.15.115:0/2089052428 >> v1:172.21.15.115:6789/0 conn(0x7fe38c15cf70 legacy=0x7fe38c0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.036+0000 7fe38b7fe640 1 -- 172.21.15.115:0/2089052428 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c0b6e30 con 0x7fe38c0d0260 2023-12-01T02:52:03.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.037+0000 7fe38b7fe640 1 --2- 172.21.15.115:0/2089052428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 0x7fe38c0b8470 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0d3bb0 tx=0x7fe38002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd8db8d0d0afcda server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:03.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.037+0000 7fe38affd640 1 -- 172.21.15.115:0/2089052428 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380031830 con 0x7fe38c0d0260 2023-12-01T02:52:03.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.037+0000 7fe38affd640 1 -- 172.21.15.115:0/2089052428 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe380031990 con 0x7fe38c0d0260 2023-12-01T02:52:03.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.037+0000 7fe38affd640 1 -- 172.21.15.115:0/2089052428 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380031d20 con 0x7fe38c0d0260 2023-12-01T02:52:03.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.038+0000 7fe3926c4640 1 -- 172.21.15.115:0/2089052428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 msgr2=0x7fe38c0b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.038+0000 7fe3926c4640 1 --2- 172.21.15.115:0/2089052428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 0x7fe38c0b8470 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0d3bb0 tx=0x7fe38002b8d0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.038+0000 7fe3926c4640 1 -- 172.21.15.115:0/2089052428 shutdown_connections 2023-12-01T02:52:03.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.038+0000 7fe3926c4640 1 --2- 172.21.15.115:0/2089052428 >> v2:172.21.15.142:3300/0 conn(0x7fe38c0b18f0 0x7fe38c0b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.038+0000 7fe3926c4640 1 --2- 172.21.15.115:0/2089052428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 0x7fe38c0b8470 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.038+0000 7fe3926c4640 1 -- 172.21.15.115:0/2089052428 >> 172.21.15.115:0/2089052428 conn(0x7fe38c06a0e0 msgr2=0x7fe38c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:03.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.038+0000 7fe3926c4640 1 -- 172.21.15.115:0/2089052428 shutdown_connections 2023-12-01T02:52:03.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.038+0000 7fe3926c4640 1 -- 172.21.15.115:0/2089052428 wait complete. 2023-12-01T02:52:03.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.040+0000 7fe3926c4640 1 Processor -- start 2023-12-01T02:52:03.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.040+0000 7fe3926c4640 1 -- start start 2023-12-01T02:52:03.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe3926c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe38c0b18f0 0x7fe38c15e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe3926c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 0x7fe38c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe3926c4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe38c15cf70 0x7fe38c15da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe3926c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0cc030 con 0x7fe38c0d0260 2023-12-01T02:52:03.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe3926c4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c15dfd0 con 0x7fe38c15cf70 2023-12-01T02:52:03.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe3926c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c15e830 con 0x7fe38c0b18f0 2023-12-01T02:52:03.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe38b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 0x7fe38c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:03.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe38b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 0x7fe38c0cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39944/0 (socket says 172.21.15.115:39944) 2023-12-01T02:52:03.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe38b7fe640 1 -- 172.21.15.115:0/2459671703 learned_addr learned my addr 172.21.15.115:0/2459671703 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:03.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe38bfff640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe38c0b18f0 0x7fe38c15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:03.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.041+0000 7fe390c61640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe38c15cf70 0x7fe38c15da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:03.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe38b7fe640 1 -- 172.21.15.115:0/2459671703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe38c0b18f0 msgr2=0x7fe38c15e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe38b7fe640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe38c0b18f0 0x7fe38c15e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe38b7fe640 1 -- 172.21.15.115:0/2459671703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe38c15cf70 msgr2=0x7fe38c15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe38b7fe640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe38c15cf70 0x7fe38c15da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe38b7fe640 1 -- 172.21.15.115:0/2459671703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380009400 con 0x7fe38c0d0260 2023-12-01T02:52:03.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe38b7fe640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 0x7fe38c0cbaf0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fe380009890 tx=0x7fe3800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:03.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe388ff9640 1 -- 172.21.15.115:0/2459671703 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38003ba70 con 0x7fe38c0d0260 2023-12-01T02:52:03.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe388ff9640 1 -- 172.21.15.115:0/2459671703 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38003bbf0 con 0x7fe38c0d0260 2023-12-01T02:52:03.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe388ff9640 1 -- 172.21.15.115:0/2459671703 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380045400 con 0x7fe38c0d0260 2023-12-01T02:52:03.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe3926c4640 1 -- 172.21.15.115:0/2459671703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c15eb10 con 0x7fe38c0d0260 2023-12-01T02:52:03.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.042+0000 7fe3926c4640 1 -- 172.21.15.115:0/2459671703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c15f0d0 con 0x7fe38c0d0260 2023-12-01T02:52:03.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.043+0000 7fe3926c4640 1 -- 172.21.15.115:0/2459671703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38c04e890 con 0x7fe38c0d0260 2023-12-01T02:52:03.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.044+0000 7fe388ff9640 1 -- 172.21.15.115:0/2459671703 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 91) v1 ==== 88710+0+0 (secure 0 0 0) 0x7fe38004e440 con 0x7fe38c0d0260 2023-12-01T02:52:03.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.045+0000 7fe388ff9640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe37806a550 0x7fe37806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.045+0000 7fe38bfff640 1 -- 172.21.15.115:0/2459671703 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe37806a550 msgr2=0x7fe37806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:03.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.045+0000 7fe388ff9640 1 -- 172.21.15.115:0/2459671703 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe3800aaed0 con 0x7fe38c0d0260 2023-12-01T02:52:03.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.045+0000 7fe38bfff640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe37806a550 0x7fe37806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:03.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.049+0000 7fe388ff9640 1 -- 172.21.15.115:0/2459671703 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe3800761a0 con 0x7fe38c0d0260 2023-12-01T02:52:03.153 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:03.152+0000 7fa03f956040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T02:52:03.197 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:03.195+0000 7f50e5e58040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T02:52:03.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.239+0000 7fe3926c4640 1 -- 172.21.15.115:0/2459671703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe38c0b16d0 con 0x7fe38c0d0260 2023-12-01T02:52:03.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.240+0000 7fe388ff9640 1 -- 172.21.15.115:0/2459671703 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v164) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3800350a0 con 0x7fe38c0d0260 2023-12-01T02:52:03.242 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T02:52:03.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.242+0000 7fe3926c4640 1 -- 172.21.15.115:0/2459671703 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe37806a550 msgr2=0x7fe37806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:03.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.242+0000 7fe3926c4640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe37806a550 0x7fe37806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.243+0000 7fe3926c4640 1 -- 172.21.15.115:0/2459671703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 msgr2=0x7fe38c0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.243+0000 7fe3926c4640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 0x7fe38c0cbaf0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fe380009890 tx=0x7fe3800046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.243+0000 7fe3926c4640 1 -- 172.21.15.115:0/2459671703 shutdown_connections 2023-12-01T02:52:03.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.243+0000 7fe3926c4640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe37806a550 0x7fe37806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.243+0000 7fe3926c4640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe38c15cf70 0x7fe38c15da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.243+0000 7fe3926c4640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe38c0d0260 0x7fe38c0cbaf0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.243+0000 7fe3926c4640 1 --2- 172.21.15.115:0/2459671703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe38c0b18f0 0x7fe38c15e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.243+0000 7fe3926c4640 1 -- 172.21.15.115:0/2459671703 >> 172.21.15.115:0/2459671703 conn(0x7fe38c06a0e0 msgr2=0x7fe38c0d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:03.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.243+0000 7fe3926c4640 1 -- 172.21.15.115:0/2459671703 shutdown_connections 2023-12-01T02:52:03.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.243+0000 7fe3926c4640 1 -- 172.21.15.115:0/2459671703 wait complete. 2023-12-01T02:52:03.263 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T02:52:03.313 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:03.313+0000 7fa03f956040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T02:52:03.362 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:03.360+0000 7f50e5e58040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T02:52:03.423 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:03.422+0000 7fa03f956040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T02:52:03.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.465+0000 7fb920f3c640 1 Processor -- start 2023-12-01T02:52:03.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.465+0000 7fb920f3c640 1 -- start start 2023-12-01T02:52:03.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.467+0000 7fb920f3c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb91c06d9c0 0x7fb91c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.467+0000 7fb920f3c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91c0ccf90 con 0x7fb91c071d10 2023-12-01T02:52:03.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.467+0000 7fb920f3c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91c0cd550 con 0x7fb91c15ad00 2023-12-01T02:52:03.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.467+0000 7fb920f3c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91c06de50 con 0x7fb91c06d9c0 2023-12-01T02:52:03.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.467+0000 7fb91ad9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb91c06d9c0 0x7fb91c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:03.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.467+0000 7fb91ad9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb91c06d9c0 0x7fb91c0b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39956/0 (socket says 172.21.15.115:39956) 2023-12-01T02:52:03.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.467+0000 7fb91ad9d640 1 -- 172.21.15.115:0/3772795255 learned_addr learned my addr 172.21.15.115:0/3772795255 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:03.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.467+0000 7fb91ad9d640 1 -- 172.21.15.115:0/3772795255 >> v1:172.21.15.142:6789/0 conn(0x7fb91c15ad00 legacy=0x7fb91c071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.467+0000 7fb91ad9d640 1 -- 172.21.15.115:0/3772795255 >> v1:172.21.15.115:6789/0 conn(0x7fb91c071d10 legacy=0x7fb91c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.467+0000 7fb91ad9d640 1 -- 172.21.15.115:0/3772795255 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c06fc40 con 0x7fb91c06d9c0 2023-12-01T02:52:03.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.469+0000 7fb91ad9d640 1 --2- 172.21.15.115:0/3772795255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c06d9c0 0x7fb91c0b4be0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fb914009e50 tx=0x7fb91402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c0fd8181865920 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:03.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.469+0000 7fb919d9b640 1 -- 172.21.15.115:0/3772795255 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91402ca80 con 0x7fb91c06d9c0 2023-12-01T02:52:03.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.469+0000 7fb919d9b640 1 -- 172.21.15.115:0/3772795255 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91402cbe0 con 0x7fb91c06d9c0 2023-12-01T02:52:03.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.469+0000 7fb919d9b640 1 -- 172.21.15.115:0/3772795255 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9140343f0 con 0x7fb91c06d9c0 2023-12-01T02:52:03.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.469+0000 7fb920f3c640 1 -- 172.21.15.115:0/3772795255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c06d9c0 msgr2=0x7fb91c0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.469+0000 7fb920f3c640 1 --2- 172.21.15.115:0/3772795255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c06d9c0 0x7fb91c0b4be0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fb914009e50 tx=0x7fb91402bee0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.470+0000 7fb920f3c640 1 -- 172.21.15.115:0/3772795255 shutdown_connections 2023-12-01T02:52:03.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.470+0000 7fb920f3c640 1 --2- 172.21.15.115:0/3772795255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c06d9c0 0x7fb91c0b4be0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.470+0000 7fb920f3c640 1 -- 172.21.15.115:0/3772795255 >> 172.21.15.115:0/3772795255 conn(0x7fb91c0cbb10 msgr2=0x7fb91c0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:03.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.470+0000 7fb920f3c640 1 -- 172.21.15.115:0/3772795255 shutdown_connections 2023-12-01T02:52:03.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.470+0000 7fb920f3c640 1 -- 172.21.15.115:0/3772795255 wait complete. 2023-12-01T02:52:03.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.471+0000 7fb920f3c640 1 Processor -- start 2023-12-01T02:52:03.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.471+0000 7fb920f3c640 1 -- start start 2023-12-01T02:52:03.473 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:03.472+0000 7fa03f956040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.472+0000 7fb920f3c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb91c06d9c0 0x7fb91c0ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.472+0000 7fb920f3c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb91c071d10 0x7fb91c1f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.472+0000 7fb920f3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c15ad00 0x7fb91c1f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.472+0000 7fb920f3c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0c4bd0 con 0x7fb91c15ad00 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.472+0000 7fb920f3c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0c68c0 con 0x7fb91c071d10 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.472+0000 7fb920f3c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0af9d0 con 0x7fb91c06d9c0 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c15ad00 0x7fb91c1f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c15ad00 0x7fb91c1f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39962/0 (socket says 172.21.15.115:39962) 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91ad9d640 1 -- 172.21.15.115:0/446131696 learned_addr learned my addr 172.21.15.115:0/446131696 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb913fff640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb91c071d10 0x7fb91c1f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91a59c640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb91c06d9c0 0x7fb91c0ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:03.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91ad9d640 1 -- 172.21.15.115:0/446131696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb91c06d9c0 msgr2=0x7fb91c0ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91ad9d640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb91c06d9c0 0x7fb91c0ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91ad9d640 1 -- 172.21.15.115:0/446131696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb91c071d10 msgr2=0x7fb91c1f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91ad9d640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb91c071d10 0x7fb91c1f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91ad9d640 1 -- 172.21.15.115:0/446131696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb914009b30 con 0x7fb91c15ad00 2023-12-01T02:52:03.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91a59c640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb91c06d9c0 0x7fb91c0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:03.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb913fff640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb91c071d10 0x7fb91c1f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:03.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.473+0000 7fb91ad9d640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c15ad00 0x7fb91c1f4330 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fb914036040 tx=0x7fb914004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:03.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.474+0000 7fb912ffd640 1 -- 172.21.15.115:0/446131696 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb914009510 con 0x7fb91c15ad00 2023-12-01T02:52:03.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.474+0000 7fb912ffd640 1 -- 172.21.15.115:0/446131696 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91403e450 con 0x7fb91c15ad00 2023-12-01T02:52:03.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.474+0000 7fb912ffd640 1 -- 172.21.15.115:0/446131696 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91403e800 con 0x7fb91c15ad00 2023-12-01T02:52:03.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.474+0000 7fb920f3c640 1 -- 172.21.15.115:0/446131696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91c0af6f0 con 0x7fb91c15ad00 2023-12-01T02:52:03.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.474+0000 7fb920f3c640 1 -- 172.21.15.115:0/446131696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91c0c6550 con 0x7fb91c15ad00 2023-12-01T02:52:03.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.475+0000 7fb920f3c640 1 -- 172.21.15.115:0/446131696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91c0c6a50 con 0x7fb91c15ad00 2023-12-01T02:52:03.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.476+0000 7fb912ffd640 1 -- 172.21.15.115:0/446131696 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 91) v1 ==== 88710+0+0 (secure 0 0 0) 0x7fb91402cd50 con 0x7fb91c15ad00 2023-12-01T02:52:03.478 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:03.476+0000 7f50e5e58040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T02:52:03.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.477+0000 7fb912ffd640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8fc06a550 0x7fb8fc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:03.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.477+0000 7fb912ffd640 1 -- 172.21.15.115:0/446131696 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fb9140ab820 con 0x7fb91c15ad00 2023-12-01T02:52:03.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.477+0000 7fb91a59c640 1 -- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8fc06a550 msgr2=0x7fb8fc06ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:03.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.477+0000 7fb91a59c640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8fc06a550 0x7fb8fc06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:03.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.485+0000 7fb912ffd640 1 -- 172.21.15.115:0/446131696 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb914076b80 con 0x7fb91c15ad00 2023-12-01T02:52:03.533 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:03.531+0000 7f50e5e58040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T02:52:03.582 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:03.581+0000 7fa03f956040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:52:03.646 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:03.644+0000 7f50e5e58040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:52:03.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.678+0000 7fb91a59c640 1 -- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8fc06a550 msgr2=0x7fb8fc06ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:03.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.678+0000 7fb91a59c640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8fc06a550 0x7fb8fc06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:52:03.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.680+0000 7fb920f3c640 1 -- 172.21.15.115:0/446131696 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb91c04e890 con 0x7fb91c15ad00 2023-12-01T02:52:03.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.681+0000 7fb912ffd640 1 -- 172.21.15.115:0/446131696 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v164) v1 ==== 80+0+467 (secure 0 0 0) 0x7fb91407a1a0 con 0x7fb91c15ad00 2023-12-01T02:52:03.683 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/88317 2023-12-02T02:51:58.123217+0000 2023-12-01T02:52:03.683 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2871788797 2023-12-02T02:51:58.123217+0000 2023-12-01T02:52:03.683 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/88317 2023-12-02T02:51:58.123217+0000 2023-12-01T02:52:03.683 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2571267301 2023-12-02T02:51:58.123217+0000 2023-12-01T02:52:03.683 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1473681116 2023-12-02T02:51:58.123217+0000 2023-12-01T02:52:03.683 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3088218027 2023-12-02T02:51:58.123217+0000 2023-12-01T02:52:03.683 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/500113028 2023-12-02T02:51:58.123217+0000 2023-12-01T02:52:03.683 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1225510452 2023-12-02T02:51:58.123217+0000 2023-12-01T02:52:03.683 INFO:teuthology.orchestra.run.smithi115.stderr:listed 8 entries 2023-12-01T02:52:03.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.683+0000 7fb920f3c640 1 -- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8fc06a550 msgr2=0x7fb8fc06ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:03.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.683+0000 7fb920f3c640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8fc06a550 0x7fb8fc06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.683+0000 7fb920f3c640 1 -- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c15ad00 msgr2=0x7fb91c1f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:03.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.683+0000 7fb920f3c640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c15ad00 0x7fb91c1f4330 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fb914036040 tx=0x7fb914004a40 comp rx=0 tx=0).stop 2023-12-01T02:52:03.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.684+0000 7fb920f3c640 1 -- 172.21.15.115:0/446131696 shutdown_connections 2023-12-01T02:52:03.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.684+0000 7fb920f3c640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb8fc06a550 0x7fb8fc06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.684+0000 7fb920f3c640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb91c15ad00 0x7fb91c1f4330 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.684+0000 7fb920f3c640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb91c071d10 0x7fb91c1f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.684+0000 7fb920f3c640 1 --2- 172.21.15.115:0/446131696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb91c06d9c0 0x7fb91c0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:03.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.684+0000 7fb920f3c640 1 -- 172.21.15.115:0/446131696 >> 172.21.15.115:0/446131696 conn(0x7fb91c0cbb10 msgr2=0x7fb91c0b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:03.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.684+0000 7fb920f3c640 1 -- 172.21.15.115:0/446131696 shutdown_connections 2023-12-01T02:52:03.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:03.684+0000 7fb920f3c640 1 -- 172.21.15.115:0/446131696 wait complete. 2023-12-01T02:52:03.687 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:03.686+0000 7fa03f956040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T02:52:03.703 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T02:52:03.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:03.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T02:52:03.759 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:03.757+0000 7f50e5e58040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T02:52:03.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T02:52:03.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:03.792 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T02:52:03.834 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T02:52:03.834 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:52:03.845 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:03.844+0000 7fa03f956040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:52:03.884 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T02:52:03.884 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:52:03.884 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:52:03.898 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:03.897+0000 7fa03f956040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T02:52:03.926 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:03.924+0000 7f50e5e58040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:52:03.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T02:52:03.936 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T02:52:03.936 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:52:03.978 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:03.976+0000 7f50e5e58040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T02:52:03.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T02:52:03.991 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:52:04.026 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:04.025+0000 7fa03f956040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T02:52:04.103 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:04.101+0000 7f50e5e58040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T02:52:04.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.198+0000 7f0ba86b6640 1 Processor -- start 2023-12-01T02:52:04.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.198+0000 7f0ba86b6640 1 -- start start 2023-12-01T02:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.200+0000 7f0ba86b6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0ba015ad00 0x7f0ba006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.200+0000 7f0ba86b6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba00b5070 con 0x7f0ba0071d10 2023-12-01T02:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.200+0000 7f0ba86b6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba00ccf90 con 0x7f0ba00b4be0 2023-12-01T02:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.200+0000 7f0ba86b6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba00cd550 con 0x7f0ba015ad00 2023-12-01T02:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.200+0000 7f0ba6c53640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0ba0071d10 0x7f0ba00cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:47782/0 (socket says 172.21.15.115:47782) 2023-12-01T02:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.200+0000 7f0ba6c53640 1 -- 172.21.15.115:0/2549857982 learned_addr learned my addr 172.21.15.115:0/2549857982 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.200+0000 7f0ba6452640 1 --2- 172.21.15.115:0/2549857982 >> v2:172.21.15.142:3300/0 conn(0x7f0ba015ad00 0x7f0ba006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391540854 0 0) 0x7f0ba00b5070 con 0x7f0ba0071d10 2023-12-01T02:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b8c003520 con 0x7f0ba0071d10 2023-12-01T02:52:04.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 211183179 0 0) 0x7f0b8c003520 con 0x7f0ba0071d10 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 >> v2:172.21.15.142:3300/0 conn(0x7f0ba015ad00 msgr2=0x7f0ba006d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 --2- 172.21.15.115:0/2549857982 >> v2:172.21.15.142:3300/0 conn(0x7f0ba015ad00 0x7f0ba006d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 >> v1:172.21.15.142:6789/0 conn(0x7f0ba00b4be0 legacy=0x7f0ba0071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba006fc40 con 0x7f0ba0071d10 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0b980019f0 con 0x7f0ba0071d10 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 >> v1:172.21.15.115:6789/0 conn(0x7f0ba0071d10 legacy=0x7f0ba00cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b8c0039e0 0x7f0b8c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0b8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b8c005280 0x7f0b8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00cd550 con 0x7f0b8c0047c0 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c00c1c0 con 0x7f0b8c0039e0 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c00c380 con 0x7f0b8c005280 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.201+0000 7f0ba5c51640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0b8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.202+0000 7f0ba6c53640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b8c005280 0x7f0b8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.202+0000 7f0ba6452640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b8c0039e0 0x7f0b8c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.202+0000 7f0ba5c51640 1 -- 172.21.15.115:0/2549857982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b8c005280 msgr2=0x7f0b8c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.202+0000 7f0ba5c51640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b8c005280 0x7f0b8c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.202+0000 7f0ba5c51640 1 -- 172.21.15.115:0/2549857982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b8c0039e0 msgr2=0x7f0b8c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.202+0000 7f0ba5c51640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b8c0039e0 0x7f0b8c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.202+0000 7f0ba5c51640 1 -- 172.21.15.115:0/2549857982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c003520 con 0x7f0b8c0047c0 2023-12-01T02:52:04.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.202+0000 7f0ba6c53640 1 -- 172.21.15.115:0/2549857982 reap_dead start 2023-12-01T02:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.204+0000 7f0ba5c51640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0b8c004c40 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0b90001750 tx=0x7f0b8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80f1af16f2a284c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.204+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b900052d0 con 0x7f0b8c0047c0 2023-12-01T02:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.204+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9002e050 con 0x7f0b8c0047c0 2023-12-01T02:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.204+0000 7f0ba5450640 1 -- 172.21.15.115:0/2549857982 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90003830 con 0x7f0b8c0047c0 2023-12-01T02:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.204+0000 7f0ba86b6640 1 -- 172.21.15.115:0/2549857982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 msgr2=0x7f0b8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.204+0000 7f0ba86b6640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0b8c004c40 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0b90001750 tx=0x7f0b8c0057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.205+0000 7f0ba86b6640 1 -- 172.21.15.115:0/2549857982 shutdown_connections 2023-12-01T02:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.205+0000 7f0ba86b6640 1 --2- 172.21.15.115:0/2549857982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0b8c004c40 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.205+0000 7f0ba86b6640 1 -- 172.21.15.115:0/2549857982 >> 172.21.15.115:0/2549857982 conn(0x7f0ba00cbb10 msgr2=0x7f0ba00cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:04.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.205+0000 7f0ba86b6640 1 -- 172.21.15.115:0/2549857982 shutdown_connections 2023-12-01T02:52:04.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.205+0000 7f0ba86b6640 1 -- 172.21.15.115:0/2549857982 wait complete. 2023-12-01T02:52:04.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.206+0000 7f0ba86b6640 1 Processor -- start 2023-12-01T02:52:04.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.207+0000 7f0ba86b6640 1 -- start start 2023-12-01T02:52:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.207+0000 7f0ba86b6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0ba015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba86b6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba01e6c90 0x7f0ba01e7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba86b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba01e46e0 0x7f0ba01e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba86b6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba01e50a0 con 0x7f0b8c0047c0 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba86b6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba0158e70 con 0x7f0ba01e6c90 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba86b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00ca3d0 con 0x7f0ba01e46e0 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba6452640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0ba015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba6452640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0ba015d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39984/0 (socket says 172.21.15.115:39984) 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba6452640 1 -- 172.21.15.115:0/3664911503 learned_addr learned my addr 172.21.15.115:0/3664911503 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba6c53640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba01e46e0 0x7f0ba01e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba6452640 1 -- 172.21.15.115:0/3664911503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba01e46e0 msgr2=0x7f0ba01e4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba6452640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba01e46e0 0x7f0ba01e4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba6452640 1 -- 172.21.15.115:0/3664911503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba01e6c90 msgr2=0x7f0ba01e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba5c51640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba01e6c90 0x7f0ba01e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba6452640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba01e6c90 0x7f0ba01e7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba6452640 1 -- 172.21.15.115:0/3664911503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba00ca6b0 con 0x7f0b8c0047c0 2023-12-01T02:52:04.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.208+0000 7f0ba6452640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0ba015d200 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0114e0 tx=0x7f0b9c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:04.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.209+0000 7f0b96ffd640 1 -- 172.21.15.115:0/3664911503 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c015870 con 0x7f0b8c0047c0 2023-12-01T02:52:04.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.209+0000 7f0b96ffd640 1 -- 172.21.15.115:0/3664911503 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c0159d0 con 0x7f0b8c0047c0 2023-12-01T02:52:04.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.209+0000 7f0ba86b6640 1 -- 172.21.15.115:0/3664911503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba00ca990 con 0x7f0b8c0047c0 2023-12-01T02:52:04.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.209+0000 7f0b96ffd640 1 -- 172.21.15.115:0/3664911503 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c015d40 con 0x7f0b8c0047c0 2023-12-01T02:52:04.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.209+0000 7f0ba86b6640 1 -- 172.21.15.115:0/3664911503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba00caf50 con 0x7f0b8c0047c0 2023-12-01T02:52:04.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.210+0000 7f0ba86b6640 1 -- 172.21.15.115:0/3664911503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba004e890 con 0x7f0b8c0047c0 2023-12-01T02:52:04.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.211+0000 7f0b96ffd640 1 -- 172.21.15.115:0/3664911503 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 91) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f0b9c013400 con 0x7f0b8c0047c0 2023-12-01T02:52:04.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.212+0000 7f0b96ffd640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8406a550 0x7f0b8406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.212+0000 7f0b96ffd640 1 -- 172.21.15.115:0/3664911503 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f0b9c089470 con 0x7f0b8c0047c0 2023-12-01T02:52:04.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.212+0000 7f0ba5c51640 1 -- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8406a550 msgr2=0x7f0b8406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:04.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.212+0000 7f0ba5c51640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8406a550 0x7f0b8406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:04.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.216+0000 7f0b96ffd640 1 -- 172.21.15.115:0/3664911503 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0b9c054720 con 0x7f0b8c0047c0 2023-12-01T02:52:04.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.413+0000 7f0ba5c51640 1 -- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8406a550 msgr2=0x7f0b8406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:04.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.413+0000 7f0ba5c51640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8406a550 0x7f0b8406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:52:04.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.433+0000 7f0ba86b6640 1 -- 172.21.15.115:0/3664911503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ba006cb20 con 0x7f0b8c0047c0 2023-12-01T02:52:04.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.434+0000 7f0b96ffd640 1 -- 172.21.15.115:0/3664911503 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f0b9c057d40 con 0x7f0b8c0047c0 2023-12-01T02:52:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:04.436 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1952979218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1952979218},{"type":"v1","addr":"172.21.15.142:6833","nonce":1952979218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3618529680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3618529680},{"type":"v1","addr":"172.21.15.142:6835","nonce":3618529680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":9009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1637880399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1637880399},{"type":"v1","addr":"172.21.15.115:6836","nonce":1637880399}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:51:43.827542+0000","modified":"2023-12-01T02:51:46.847110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3770885942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3770885942},{"type":"v1","addr":"172.21.15.115:6837","nonce":3770885942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-01T02:52:04.437 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 71 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 -- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8406a550 msgr2=0x7f0b8406ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8406a550 0x7f0b8406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 -- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 msgr2=0x7f0ba015d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0ba015d200 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0114e0 tx=0x7f0b9c0119a0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 -- 172.21.15.115:0/3664911503 shutdown_connections 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f0b8406a550 0x7f0b8406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ba01e46e0 0x7f0ba01e4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ba01e6c90 0x7f0ba01e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 --2- 172.21.15.115:0/3664911503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b8c0047c0 0x7f0ba015d200 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 -- 172.21.15.115:0/3664911503 >> 172.21.15.115:0/3664911503 conn(0x7f0ba00cbb10 msgr2=0x7f0ba00b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 -- 172.21.15.115:0/3664911503 shutdown_connections 2023-12-01T02:52:04.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.437+0000 7f0ba86b6640 1 -- 172.21.15.115:0/3664911503 wait complete. 2023-12-01T02:52:04.457 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.661+0000 7ff2c6ea1640 1 Processor -- start 2023-12-01T02:52:04.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.661+0000 7ff2c6ea1640 1 -- start start 2023-12-01T02:52:04.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c6ea1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff2c015cd20 0x7ff2c00d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c6ea1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff2c00b5d80 0x7ff2c00b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c6ea1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c00ce2e0 con 0x7ff2c00d1230 2023-12-01T02:52:04.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c6ea1640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c00b2160 con 0x7ff2c015cd20 2023-12-01T02:52:04.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c6ea1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c00d2a10 con 0x7ff2c00b5d80 2023-12-01T02:52:04.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c4c3d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff2c015cd20 0x7ff2c00d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c543e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff2c00d1230 0x7ff2c00d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:47790/0 (socket says 172.21.15.115:47790) 2023-12-01T02:52:04.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c4c3d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff2c015cd20 0x7ff2c00d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39986/0 (socket says 172.21.15.115:39986) 2023-12-01T02:52:04.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c543e640 1 -- 172.21.15.115:0/1731861206 learned_addr learned my addr 172.21.15.115:0/1731861206 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:04.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2b7fff640 1 --2- 172.21.15.115:0/1731861206 >> v2:172.21.15.142:3300/0 conn(0x7ff2c00b5d80 0x7ff2c00b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c4c3d640 1 -- 172.21.15.115:0/1731861206 >> v2:172.21.15.142:3300/0 conn(0x7ff2c00b5d80 msgr2=0x7ff2c00b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:04.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c4c3d640 1 --2- 172.21.15.115:0/1731861206 >> v2:172.21.15.142:3300/0 conn(0x7ff2c00b5d80 0x7ff2c00b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c4c3d640 1 -- 172.21.15.115:0/1731861206 >> v1:172.21.15.115:6789/0 conn(0x7ff2c00d1230 legacy=0x7ff2c00d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.663+0000 7ff2c4c3d640 1 -- 172.21.15.115:0/1731861206 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c00ce000 con 0x7ff2c015cd20 2023-12-01T02:52:04.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.664+0000 7ff2c4c3d640 1 --2- 172.21.15.115:0/1731861206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c015cd20 0x7ff2c00d24d0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00cf630 tx=0x7ff2bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617ee4539d8666e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:04.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.665+0000 7ff2b77fe640 1 -- 172.21.15.115:0/1731861206 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc02e070 con 0x7ff2c015cd20 2023-12-01T02:52:04.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.665+0000 7ff2b77fe640 1 -- 172.21.15.115:0/1731861206 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc0315a0 con 0x7ff2c015cd20 2023-12-01T02:52:04.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.665+0000 7ff2b77fe640 1 -- 172.21.15.115:0/1731861206 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc031930 con 0x7ff2c015cd20 2023-12-01T02:52:04.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.665+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/1731861206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c015cd20 msgr2=0x7ff2c00d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.665+0000 7ff2c6ea1640 1 --2- 172.21.15.115:0/1731861206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c015cd20 0x7ff2c00d24d0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00cf630 tx=0x7ff2bc02b840 comp rx=0 tx=0).stop 2023-12-01T02:52:04.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.666+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/1731861206 shutdown_connections 2023-12-01T02:52:04.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.666+0000 7ff2c6ea1640 1 --2- 172.21.15.115:0/1731861206 >> v2:172.21.15.142:3300/0 conn(0x7ff2c00b5d80 0x7ff2c00b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.666+0000 7ff2c6ea1640 1 --2- 172.21.15.115:0/1731861206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c015cd20 0x7ff2c00d24d0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.666+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/1731861206 >> 172.21.15.115:0/1731861206 conn(0x7ff2c006a0e0 msgr2=0x7ff2c006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:04.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.666+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/1731861206 shutdown_connections 2023-12-01T02:52:04.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.666+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/1731861206 wait complete. 2023-12-01T02:52:04.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.667+0000 7ff2c6ea1640 1 Processor -- start 2023-12-01T02:52:04.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.667+0000 7ff2c6ea1640 1 -- start start 2023-12-01T02:52:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.668+0000 7ff2c6ea1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c00b5d80 0x7ff2c01510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.668+0000 7ff2c6ea1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c00d1230 0x7ff2c00b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.668+0000 7ff2c6ea1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2c015cd20 0x7ff2c015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.668+0000 7ff2c6ea1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c00c7620 con 0x7ff2c00b5d80 2023-12-01T02:52:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.668+0000 7ff2c6ea1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c00c6750 con 0x7ff2c00d1230 2023-12-01T02:52:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.668+0000 7ff2c6ea1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c0160080 con 0x7ff2c015cd20 2023-12-01T02:52:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2c4c3d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c00b5d80 0x7ff2c01510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2c4c3d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c00b5d80 0x7ff2c01510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39996/0 (socket says 172.21.15.115:39996) 2023-12-01T02:52:04.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2c4c3d640 1 -- 172.21.15.115:0/622540146 learned_addr learned my addr 172.21.15.115:0/622540146 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:04.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2b7fff640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c00d1230 0x7ff2c00b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2c543e640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2c015cd20 0x7ff2c015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:04.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2c4c3d640 1 -- 172.21.15.115:0/622540146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2c015cd20 msgr2=0x7ff2c015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2c4c3d640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2c015cd20 0x7ff2c015adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2c4c3d640 1 -- 172.21.15.115:0/622540146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c00d1230 msgr2=0x7ff2c00b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2c4c3d640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c00d1230 0x7ff2c00b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2c4c3d640 1 -- 172.21.15.115:0/622540146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc009400 con 0x7ff2c00b5d80 2023-12-01T02:52:04.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.669+0000 7ff2c4c3d640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c00b5d80 0x7ff2c01510d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc034040 tx=0x7ff2bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.670+0000 7ff2b57fa640 1 -- 172.21.15.115:0/622540146 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc032900 con 0x7ff2c00b5d80 2023-12-01T02:52:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.670+0000 7ff2b57fa640 1 -- 172.21.15.115:0/622540146 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc032a60 con 0x7ff2c00b5d80 2023-12-01T02:52:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.670+0000 7ff2b57fa640 1 -- 172.21.15.115:0/622540146 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc032e10 con 0x7ff2c00b5d80 2023-12-01T02:52:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.670+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/622540146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c00c69a0 con 0x7ff2c00b5d80 2023-12-01T02:52:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.670+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/622540146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c00c6f60 con 0x7ff2c00b5d80 2023-12-01T02:52:04.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.671+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/622540146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c004e890 con 0x7ff2c00b5d80 2023-12-01T02:52:04.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.672+0000 7ff2b57fa640 1 -- 172.21.15.115:0/622540146 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 91) v1 ==== 88710+0+0 (secure 0 0 0) 0x7ff2bc04e430 con 0x7ff2c00b5d80 2023-12-01T02:52:04.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.673+0000 7ff2b57fa640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff2ac06a550 0x7ff2ac06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:04.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.673+0000 7ff2b57fa640 1 -- 172.21.15.115:0/622540146 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff2bc0aaee0 con 0x7ff2c00b5d80 2023-12-01T02:52:04.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.673+0000 7ff2b7fff640 1 -- 172.21.15.115:0/622540146 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff2ac06a550 msgr2=0x7ff2ac06ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:04.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.673+0000 7ff2b7fff640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff2ac06a550 0x7ff2ac06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:04.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.677+0000 7ff2b57fa640 1 -- 172.21.15.115:0/622540146 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff2bc076110 con 0x7ff2c00b5d80 2023-12-01T02:52:04.858 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:04.857+0000 7fa03f956040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T02:52:04.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.869+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/622540146 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2c006a560 con 0x7ff2c00b5d80 2023-12-01T02:52:04.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.870+0000 7ff2b57fa640 1 -- 172.21.15.115:0/622540146 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+16675 (secure 0 0 0) 0x7ff2bc079730 con 0x7ff2c00b5d80 2023-12-01T02:52:04.872 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:04.873 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":164,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:51:58.123244+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:51:41.383835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-01T02:51:42.832726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/88317":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/2871788797":"2023-12-02T02:51:58.123217+0000","172.21.15.115:6822/88317":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/2571267301":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/1473681116":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/3088218027":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/500113028":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/1225510452":"2023-12-02T02:51:58.123217+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.873+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/622540146 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff2ac06a550 msgr2=0x7ff2ac06ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.873+0000 7ff2c6ea1640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff2ac06a550 0x7ff2ac06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.874+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/622540146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c00b5d80 msgr2=0x7ff2c01510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.874+0000 7ff2c6ea1640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c00b5d80 0x7ff2c01510d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc034040 tx=0x7ff2bc0046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.874+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/622540146 shutdown_connections 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.874+0000 7ff2c6ea1640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff2ac06a550 0x7ff2ac06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.874+0000 7ff2c6ea1640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2c015cd20 0x7ff2c015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.874+0000 7ff2c6ea1640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c00d1230 0x7ff2c00b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.874+0000 7ff2c6ea1640 1 --2- 172.21.15.115:0/622540146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2c00b5d80 0x7ff2c01510d0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.874+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/622540146 >> 172.21.15.115:0/622540146 conn(0x7ff2c006a0e0 msgr2=0x7ff2c00d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.874+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/622540146 shutdown_connections 2023-12-01T02:52:04.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:04.874+0000 7ff2c6ea1640 1 -- 172.21.15.115:0/622540146 wait complete. 2023-12-01T02:52:04.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T02:51:41.383835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T02:51:42.832726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T02:52:04.892 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T02:52:04.943 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:04.942+0000 7f50e5e58040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T02:52:05.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.101+0000 7f448373e640 1 Processor -- start 2023-12-01T02:52:05.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.101+0000 7f448373e640 1 -- start start 2023-12-01T02:52:05.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f448373e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f447c15ab40 0x7f447c0ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f448373e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f447c0ccf90 0x7f447c0cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f448373e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f447c0cd410 0x7f447c0cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f448373e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c15e4c0 con 0x7f447c15ab40 2023-12-01T02:52:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f448373e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c15af20 con 0x7f447c0cd410 2023-12-01T02:52:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f448373e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c15d8b0 con 0x7f447c0ccf90 2023-12-01T02:52:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f44814da640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f447c15ab40 0x7f447c0ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f44814da640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f447c15ab40 0x7f447c0ad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40000/0 (socket says 172.21.15.115:40000) 2023-12-01T02:52:05.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f44814da640 1 -- 172.21.15.115:0/3247415993 learned_addr learned my addr 172.21.15.115:0/3247415993 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f4481cdb640 1 --2- 172.21.15.115:0/3247415993 >> v2:172.21.15.142:3300/0 conn(0x7f447c0cd410 0x7f447c0cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f4480cd9640 1 --2- 172.21.15.115:0/3247415993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f447c0ccf90 0x7f447c0cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f44814da640 1 -- 172.21.15.115:0/3247415993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f447c0ccf90 msgr2=0x7f447c0cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f44814da640 1 --2- 172.21.15.115:0/3247415993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f447c0ccf90 0x7f447c0cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f44814da640 1 -- 172.21.15.115:0/3247415993 >> v2:172.21.15.142:3300/0 conn(0x7f447c0cd410 msgr2=0x7f447c0cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f44814da640 1 --2- 172.21.15.115:0/3247415993 >> v2:172.21.15.142:3300/0 conn(0x7f447c0cd410 0x7f447c0cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.103+0000 7f44814da640 1 -- 172.21.15.115:0/3247415993 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c15db90 con 0x7f447c15ab40 2023-12-01T02:52:05.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.104+0000 7f44814da640 1 --2- 172.21.15.115:0/3247415993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c15ab40 0x7f447c0ad3f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f447c06e6a0 tx=0x7f446c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1f732def0d957 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:05.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.105+0000 7f4473fff640 1 -- 172.21.15.115:0/3247415993 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0317c0 con 0x7f447c15ab40 2023-12-01T02:52:05.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.105+0000 7f4473fff640 1 -- 172.21.15.115:0/3247415993 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c031920 con 0x7f447c15ab40 2023-12-01T02:52:05.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.105+0000 7f4473fff640 1 -- 172.21.15.115:0/3247415993 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c031cb0 con 0x7f447c15ab40 2023-12-01T02:52:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.105+0000 7f448373e640 1 -- 172.21.15.115:0/3247415993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c15ab40 msgr2=0x7f447c0ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.105+0000 7f448373e640 1 --2- 172.21.15.115:0/3247415993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c15ab40 0x7f447c0ad3f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f447c06e6a0 tx=0x7f446c02b860 comp rx=0 tx=0).stop 2023-12-01T02:52:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.106+0000 7f448373e640 1 -- 172.21.15.115:0/3247415993 shutdown_connections 2023-12-01T02:52:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.106+0000 7f448373e640 1 --2- 172.21.15.115:0/3247415993 >> v2:172.21.15.142:3300/0 conn(0x7f447c0cd410 0x7f447c0cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.106+0000 7f448373e640 1 --2- 172.21.15.115:0/3247415993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f447c0ccf90 0x7f447c0cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.106+0000 7f448373e640 1 --2- 172.21.15.115:0/3247415993 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c15ab40 0x7f447c0ad3f0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.106+0000 7f448373e640 1 -- 172.21.15.115:0/3247415993 >> 172.21.15.115:0/3247415993 conn(0x7f447c06bd00 msgr2=0x7f447c06c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.106+0000 7f448373e640 1 -- 172.21.15.115:0/3247415993 shutdown_connections 2023-12-01T02:52:05.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.106+0000 7f448373e640 1 -- 172.21.15.115:0/3247415993 wait complete. 2023-12-01T02:52:05.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.108+0000 7f448373e640 1 Processor -- start 2023-12-01T02:52:05.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.108+0000 7f448373e640 1 -- start start 2023-12-01T02:52:05.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.109+0000 7f448373e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f447c0ccf90 0x7f447c143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.109+0000 7f448373e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c0cd410 0x7f447c1e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.109+0000 7f448373e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f447c15ab40 0x7f447c157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.109+0000 7f448373e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1e7b70 con 0x7f447c0cd410 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.109+0000 7f448373e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c15b1d0 con 0x7f447c15ab40 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.109+0000 7f448373e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c15b4b0 con 0x7f447c0ccf90 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.110+0000 7f4480cd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c0cd410 0x7f447c1e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.110+0000 7f4480cd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c0cd410 0x7f447c1e70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40006/0 (socket says 172.21.15.115:40006) 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.110+0000 7f4480cd9640 1 -- 172.21.15.115:0/1389907669 learned_addr learned my addr 172.21.15.115:0/1389907669 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.110+0000 7f44814da640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f447c0ccf90 0x7f447c143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.110+0000 7f4481cdb640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f447c15ab40 0x7f447c157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.110+0000 7f4480cd9640 1 -- 172.21.15.115:0/1389907669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f447c0ccf90 msgr2=0x7f447c143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.110+0000 7f4480cd9640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f447c0ccf90 0x7f447c143a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.110+0000 7f4480cd9640 1 -- 172.21.15.115:0/1389907669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f447c15ab40 msgr2=0x7f447c157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.110+0000 7f4480cd9640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f447c15ab40 0x7f447c157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.110+0000 7f4480cd9640 1 -- 172.21.15.115:0/1389907669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0094d0 con 0x7f447c0cd410 2023-12-01T02:52:05.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.111+0000 7f4480cd9640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c0cd410 0x7f447c1e70b0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f447800c7f0 tx=0x7f447800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:05.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.111+0000 7f4471ffb640 1 -- 172.21.15.115:0/1389907669 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478004220 con 0x7f447c0cd410 2023-12-01T02:52:05.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.111+0000 7f4471ffb640 1 -- 172.21.15.115:0/1389907669 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4478004380 con 0x7f447c0cd410 2023-12-01T02:52:05.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.111+0000 7f4471ffb640 1 -- 172.21.15.115:0/1389907669 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478007860 con 0x7f447c0cd410 2023-12-01T02:52:05.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.111+0000 7f448373e640 1 -- 172.21.15.115:0/1389907669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c15b790 con 0x7f447c0cd410 2023-12-01T02:52:05.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.111+0000 7f448373e640 1 -- 172.21.15.115:0/1389907669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c143fd0 con 0x7f447c0cd410 2023-12-01T02:52:05.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.113+0000 7f448373e640 1 -- 172.21.15.115:0/1389907669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c073590 con 0x7f447c0cd410 2023-12-01T02:52:05.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.113+0000 7f4471ffb640 1 -- 172.21.15.115:0/1389907669 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 91) v1 ==== 88710+0+0 (secure 0 0 0) 0x7f4478007bd0 con 0x7f447c0cd410 2023-12-01T02:52:05.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.114+0000 7f4471ffb640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446006a550 0x7f446006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.114+0000 7f4471ffb640 1 -- 172.21.15.115:0/1389907669 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f4478088cc0 con 0x7f447c0cd410 2023-12-01T02:52:05.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.114+0000 7f44814da640 1 -- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446006a550 msgr2=0x7f446006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:05.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.114+0000 7f44814da640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446006a550 0x7f446006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:05.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.118+0000 7f4471ffb640 1 -- 172.21.15.115:0/1389907669 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4478053e70 con 0x7f447c0cd410 2023-12-01T02:52:05.152 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:05.151+0000 7fa03f956040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T02:52:05.213 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:05.212+0000 7fa03f956040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T02:52:05.269 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.267+0000 7f50e5e58040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T02:52:05.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.314+0000 7f448373e640 1 -- 172.21.15.115:0/1389907669 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f447c04e890 con 0x7f447c0cd410 2023-12-01T02:52:05.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.315+0000 7f4471ffb640 1 -- 172.21.15.115:0/1389907669 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v164) v1 ==== 93+0+10 (secure 0 0 0) 0x7f447801e070 con 0x7f447c0cd410 2023-12-01T02:52:05.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.315+0000 7f44814da640 1 -- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446006a550 msgr2=0x7f446006ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:05.317 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T02:52:05.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.315+0000 7f44814da640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446006a550 0x7f446006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:52:05.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 -- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446006a550 msgr2=0x7f446006ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:05.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446006a550 0x7f446006ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 -- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c0cd410 msgr2=0x7f447c1e70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c0cd410 0x7f447c1e70b0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f447800c7f0 tx=0x7f447800ccb0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 -- 172.21.15.115:0/1389907669 shutdown_connections 2023-12-01T02:52:05.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f446006a550 0x7f446006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f447c15ab40 0x7f447c157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f447c0cd410 0x7f447c1e70b0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 --2- 172.21.15.115:0/1389907669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f447c0ccf90 0x7f447c143a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 -- 172.21.15.115:0/1389907669 >> 172.21.15.115:0/1389907669 conn(0x7f447c06bd00 msgr2=0x7f447c0ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:05.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 -- 172.21.15.115:0/1389907669 shutdown_connections 2023-12-01T02:52:05.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.317+0000 7f448373e640 1 -- 172.21.15.115:0/1389907669 wait complete. 2023-12-01T02:52:05.326 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:05.325+0000 7fa03f956040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T02:52:05.330 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.329+0000 7f50e5e58040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T02:52:05.336 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T02:52:05.452 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.451+0000 7f50e5e58040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T02:52:05.542 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.540+0000 7f50682f4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:05.542 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.540+0000 7f50682f4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:05.542 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.540+0000 7f50682f4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:05.542 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.540+0000 7f50682f4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:05.542 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.540+0000 7f50682f4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:05.542 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.540+0000 7f506b2fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:05.542 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.540+0000 7f506b2fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:05.542 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.540+0000 7f506b2fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:05.542 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.540+0000 7f506b2fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:05.542 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:05.540+0000 7f506b2fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:05.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.543+0000 7fe9b475c640 1 Processor -- start 2023-12-01T02:52:05.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.543+0000 7fe9b475c640 1 -- start start 2023-12-01T02:52:05.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.545+0000 7fe9b475c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9ac15cd90 0x7fe9ac0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.545+0000 7fe9b475c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe9ac0b5fa0 0x7fe9ac161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.545+0000 7fe9b475c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe9ac0cf650 0x7fe9ac0cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.545+0000 7fe9b475c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ac0d0ec0 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.545+0000 7fe9b475c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ac0ce5e0 con 0x7fe9ac0cf650 2023-12-01T02:52:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.545+0000 7fe9b475c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac161950 con 0x7fe9ac15cd90 2023-12-01T02:52:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.545+0000 7fe9b1cf7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe9ac0b5fa0 0x7fe9ac161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.545+0000 7fe9b1cf7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe9ac0b5fa0 0x7fe9ac161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40102/0 (socket says 172.21.15.115:40102) 2023-12-01T02:52:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.545+0000 7fe9b1cf7640 1 -- 172.21.15.115:0/898743505 learned_addr learned my addr 172.21.15.115:0/898743505 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.545+0000 7fe9b2cf9640 1 --2- 172.21.15.115:0/898743505 >> v2:172.21.15.142:3300/0 conn(0x7fe9ac0cf650 0x7fe9ac0cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.546+0000 7fe9b24f8640 1 --2- 172.21.15.115:0/898743505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9ac15cd90 0x7fe9ac0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.546+0000 7fe9b2cf9640 1 -- 172.21.15.115:0/898743505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9ac15cd90 msgr2=0x7fe9ac0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.546+0000 7fe9b2cf9640 1 --2- 172.21.15.115:0/898743505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9ac15cd90 0x7fe9ac0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.546+0000 7fe9b2cf9640 1 -- 172.21.15.115:0/898743505 >> v2:172.21.15.115:3300/0 conn(0x7fe9ac0b5fa0 msgr2=0x7fe9ac161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.546+0000 7fe9b2cf9640 1 --2- 172.21.15.115:0/898743505 >> v2:172.21.15.115:3300/0 conn(0x7fe9ac0b5fa0 0x7fe9ac161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.546+0000 7fe9b2cf9640 1 -- 172.21.15.115:0/898743505 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0d0000 con 0x7fe9ac0cf650 2023-12-01T02:52:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.548+0000 7fe9b2cf9640 1 --2- 172.21.15.115:0/898743505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ac0cf650 0x7fe9ac0cfac0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0cd3a0 tx=0x7fe9a402be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de86c92ccbeba2c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.548+0000 7fe9b14f6640 1 -- 172.21.15.115:0/898743505 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a402cbd0 con 0x7fe9ac0cf650 2023-12-01T02:52:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.548+0000 7fe9b14f6640 1 -- 172.21.15.115:0/898743505 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a402cd30 con 0x7fe9ac0cf650 2023-12-01T02:52:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.548+0000 7fe9b14f6640 1 -- 172.21.15.115:0/898743505 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a402cea0 con 0x7fe9ac0cf650 2023-12-01T02:52:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.549+0000 7fe9b475c640 1 -- 172.21.15.115:0/898743505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ac0cf650 msgr2=0x7fe9ac0cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.549+0000 7fe9b475c640 1 --2- 172.21.15.115:0/898743505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ac0cf650 0x7fe9ac0cfac0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0cd3a0 tx=0x7fe9a402be20 comp rx=0 tx=0).stop 2023-12-01T02:52:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.549+0000 7fe9b475c640 1 -- 172.21.15.115:0/898743505 shutdown_connections 2023-12-01T02:52:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.549+0000 7fe9b475c640 1 --2- 172.21.15.115:0/898743505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ac0cf650 0x7fe9ac0cfac0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.549+0000 7fe9b475c640 1 --2- 172.21.15.115:0/898743505 >> v2:172.21.15.115:3300/0 conn(0x7fe9ac0b5fa0 0x7fe9ac161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.549+0000 7fe9b475c640 1 --2- 172.21.15.115:0/898743505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9ac15cd90 0x7fe9ac0cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.549+0000 7fe9b475c640 1 -- 172.21.15.115:0/898743505 >> 172.21.15.115:0/898743505 conn(0x7fe9ac065960 msgr2=0x7fe9ac065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.549+0000 7fe9b475c640 1 -- 172.21.15.115:0/898743505 shutdown_connections 2023-12-01T02:52:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.549+0000 7fe9b475c640 1 -- 172.21.15.115:0/898743505 wait complete. 2023-12-01T02:52:05.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.551+0000 7fe9b475c640 1 Processor -- start 2023-12-01T02:52:05.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.551+0000 7fe9b475c640 1 -- start start 2023-12-01T02:52:05.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.552+0000 7fe9b475c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9ac0b5fa0 0x7fe9ac0d2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.552+0000 7fe9b475c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ac0cf650 0x7fe9ac0d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.552+0000 7fe9b475c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9ac15cd90 0x7fe9ac06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.552+0000 7fe9b475c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac06dd00 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.552+0000 7fe9b475c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac06d080 con 0x7fe9ac0cf650 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.552+0000 7fe9b475c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac06d360 con 0x7fe9ac15cd90 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.552+0000 7fe9b24f8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9ac0b5fa0 0x7fe9ac0d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9b24f8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9ac0b5fa0 0x7fe9ac0d2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40106/0 (socket says 172.21.15.115:40106) 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9b24f8640 1 -- 172.21.15.115:0/2335912337 learned_addr learned my addr 172.21.15.115:0/2335912337 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9b24f8640 1 -- 172.21.15.115:0/2335912337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9ac15cd90 msgr2=0x7fe9ac06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9b24f8640 1 --2- 172.21.15.115:0/2335912337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9ac15cd90 0x7fe9ac06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9b24f8640 1 -- 172.21.15.115:0/2335912337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ac0cf650 msgr2=0x7fe9ac0d2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9b24f8640 1 --2- 172.21.15.115:0/2335912337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ac0cf650 0x7fe9ac0d2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9b24f8640 1 -- 172.21.15.115:0/2335912337 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a4009a70 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9b24f8640 1 --2- 172.21.15.115:0/2335912337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9ac0b5fa0 0x7fe9ac0d2980 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800d5f0 tx=0x7fe9a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9a2ffd640 1 -- 172.21.15.115:0/2335912337 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80125f0 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9a2ffd640 1 -- 172.21.15.115:0/2335912337 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8012750 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.553+0000 7fe9a2ffd640 1 -- 172.21.15.115:0/2335912337 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8012b00 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.554+0000 7fe9b475c640 1 -- 172.21.15.115:0/2335912337 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ac06d640 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.554+0000 7fe9b475c640 1 -- 172.21.15.115:0/2335912337 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac06e120 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.555+0000 7fe9b475c640 1 -- 172.21.15.115:0/2335912337 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ac04e890 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.556+0000 7fe9a2ffd640 1 -- 172.21.15.115:0/2335912337 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 92) v1 ==== 88318+0+0 (secure 0 0 0) 0x7fe9a8012c60 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.557+0000 7fe9a2ffd640 1 -- 172.21.15.115:0/2335912337 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fe9a8088ae0 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.560+0000 7fe9a2ffd640 1 -- 172.21.15.115:0/2335912337 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe9a8054290 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.748+0000 7fe9b475c640 1 -- 172.21.15.115:0/2335912337 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9ac0d0a70 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.749+0000 7fe9a2ffd640 1 -- 172.21.15.115:0/2335912337 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v165) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9a80578b0 con 0x7fe9ac0b5fa0 2023-12-01T02:52:05.751 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T02:52:05.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.751+0000 7fe9b475c640 1 -- 172.21.15.115:0/2335912337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9ac0b5fa0 msgr2=0x7fe9ac0d2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.751+0000 7fe9b475c640 1 --2- 172.21.15.115:0/2335912337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9ac0b5fa0 0x7fe9ac0d2980 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800d5f0 tx=0x7fe9a800dab0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.752+0000 7fe9b475c640 1 -- 172.21.15.115:0/2335912337 shutdown_connections 2023-12-01T02:52:05.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.752+0000 7fe9b475c640 1 --2- 172.21.15.115:0/2335912337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9ac15cd90 0x7fe9ac06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.752+0000 7fe9b475c640 1 --2- 172.21.15.115:0/2335912337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9ac0cf650 0x7fe9ac0d2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.752+0000 7fe9b475c640 1 --2- 172.21.15.115:0/2335912337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9ac0b5fa0 0x7fe9ac0d2980 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.752+0000 7fe9b475c640 1 -- 172.21.15.115:0/2335912337 >> 172.21.15.115:0/2335912337 conn(0x7fe9ac065960 msgr2=0x7fe9ac15fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:05.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.752+0000 7fe9b475c640 1 -- 172.21.15.115:0/2335912337 shutdown_connections 2023-12-01T02:52:05.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.752+0000 7fe9b475c640 1 -- 172.21.15.115:0/2335912337 wait complete. 2023-12-01T02:52:05.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T02:52:05.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.976+0000 7fee3a669640 1 Processor -- start 2023-12-01T02:52:05.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.977+0000 7fee3a669640 1 -- start start 2023-12-01T02:52:05.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.978+0000 7fee3a669640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fee340cb570 0x7fee340cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.978+0000 7fee3a669640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fee34164c60 0x7fee34165060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.978+0000 7fee3a669640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee341655a0 con 0x7fee34143eb0 2023-12-01T02:52:05.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.978+0000 7fee3a669640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee340cbf20 con 0x7fee34164c60 2023-12-01T02:52:05.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.978+0000 7fee3a669640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee340ce1b0 con 0x7fee340cb570 2023-12-01T02:52:05.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.979+0000 7fee38c06640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fee34164c60 0x7fee34165060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.979+0000 7fee38c06640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fee34164c60 0x7fee34165060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40112/0 (socket says 172.21.15.115:40112) 2023-12-01T02:52:05.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.979+0000 7fee38c06640 1 -- 172.21.15.115:0/794091155 learned_addr learned my addr 172.21.15.115:0/794091155 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:05.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.979+0000 7fee2bfff640 1 --2- 172.21.15.115:0/794091155 >> v2:172.21.15.142:3300/0 conn(0x7fee340cb570 0x7fee340cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.979+0000 7fee38c06640 1 -- 172.21.15.115:0/794091155 >> v2:172.21.15.142:3300/0 conn(0x7fee340cb570 msgr2=0x7fee340cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:05.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.979+0000 7fee38c06640 1 --2- 172.21.15.115:0/794091155 >> v2:172.21.15.142:3300/0 conn(0x7fee340cb570 0x7fee340cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.979+0000 7fee38c06640 1 -- 172.21.15.115:0/794091155 >> v1:172.21.15.142:6789/0 conn(0x7fee34143eb0 legacy=0x7fee3406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.979+0000 7fee38c06640 1 -- 172.21.15.115:0/794091155 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee340ce490 con 0x7fee34164c60 2023-12-01T02:52:05.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.980+0000 7fee38c06640 1 --2- 172.21.15.115:0/794091155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 0x7fee34165060 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fee3406d6f0 tx=0x7fee2c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47014c8c308d13e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:05.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee337fe640 1 -- 172.21.15.115:0/794091155 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c02cb20 con 0x7fee34164c60 2023-12-01T02:52:05.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee337fe640 1 -- 172.21.15.115:0/794091155 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c02cc80 con 0x7fee34164c60 2023-12-01T02:52:05.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee337fe640 1 -- 172.21.15.115:0/794091155 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0343f0 con 0x7fee34164c60 2023-12-01T02:52:05.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee3a669640 1 -- 172.21.15.115:0/794091155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 msgr2=0x7fee34165060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee3a669640 1 --2- 172.21.15.115:0/794091155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 0x7fee34165060 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fee3406d6f0 tx=0x7fee2c02bf80 comp rx=0 tx=0).stop 2023-12-01T02:52:05.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee3a669640 1 -- 172.21.15.115:0/794091155 shutdown_connections 2023-12-01T02:52:05.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee3a669640 1 --2- 172.21.15.115:0/794091155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 0x7fee34165060 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee3a669640 1 --2- 172.21.15.115:0/794091155 >> v2:172.21.15.142:3300/0 conn(0x7fee340cb570 0x7fee340cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee3a669640 1 -- 172.21.15.115:0/794091155 >> 172.21.15.115:0/794091155 conn(0x7fee34065960 msgr2=0x7fee34065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:05.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee3a669640 1 -- 172.21.15.115:0/794091155 shutdown_connections 2023-12-01T02:52:05.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.981+0000 7fee3a669640 1 -- 172.21.15.115:0/794091155 wait complete. 2023-12-01T02:52:05.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.983+0000 7fee3a669640 1 Processor -- start 2023-12-01T02:52:05.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.983+0000 7fee3a669640 1 -- start start 2023-12-01T02:52:05.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.984+0000 7fee3a669640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee340cb570 0x7fee341f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.984+0000 7fee3a669640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee34143eb0 0x7fee341f7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.984+0000 7fee3a669640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 0x7fee3413a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:05.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.984+0000 7fee3a669640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee341f76a0 con 0x7fee34164c60 2023-12-01T02:52:05.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.984+0000 7fee3a669640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3413bff0 con 0x7fee340cb570 2023-12-01T02:52:05.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.984+0000 7fee3a669640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee34142df0 con 0x7fee34143eb0 2023-12-01T02:52:05.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee38c06640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 0x7fee3413a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee38c06640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 0x7fee3413a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40124/0 (socket says 172.21.15.115:40124) 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee38c06640 1 -- 172.21.15.115:0/580670454 learned_addr learned my addr 172.21.15.115:0/580670454 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee33fff640 1 --2- 172.21.15.115:0/580670454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee340cb570 0x7fee341f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee2bfff640 1 --2- 172.21.15.115:0/580670454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee34143eb0 0x7fee341f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee38c06640 1 -- 172.21.15.115:0/580670454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee34143eb0 msgr2=0x7fee341f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee38c06640 1 --2- 172.21.15.115:0/580670454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee34143eb0 0x7fee341f7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee38c06640 1 -- 172.21.15.115:0/580670454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee340cb570 msgr2=0x7fee341f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee38c06640 1 --2- 172.21.15.115:0/580670454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee340cb570 0x7fee341f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee38c06640 1 -- 172.21.15.115:0/580670454 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c009b30 con 0x7fee34164c60 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee2bfff640 1 --2- 172.21.15.115:0/580670454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee34143eb0 0x7fee341f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.985+0000 7fee38c06640 1 --2- 172.21.15.115:0/580670454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 0x7fee3413a110 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fee2c036040 tx=0x7fee2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.986+0000 7fee317fa640 1 -- 172.21.15.115:0/580670454 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c009510 con 0x7fee34164c60 2023-12-01T02:52:05.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.986+0000 7fee317fa640 1 -- 172.21.15.115:0/580670454 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c02c920 con 0x7fee34164c60 2023-12-01T02:52:05.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.986+0000 7fee317fa640 1 -- 172.21.15.115:0/580670454 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c03e690 con 0x7fee34164c60 2023-12-01T02:52:05.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.986+0000 7fee3a669640 1 -- 172.21.15.115:0/580670454 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee341430d0 con 0x7fee34164c60 2023-12-01T02:52:05.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.986+0000 7fee3a669640 1 -- 172.21.15.115:0/580670454 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3413c570 con 0x7fee34164c60 2023-12-01T02:52:05.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.987+0000 7fee3a669640 1 -- 172.21.15.115:0/580670454 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee34143310 con 0x7fee34164c60 2023-12-01T02:52:05.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.988+0000 7fee317fa640 1 -- 172.21.15.115:0/580670454 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 92) v1 ==== 88318+0+0 (secure 0 0 0) 0x7fee2c03e7f0 con 0x7fee34164c60 2023-12-01T02:52:05.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.989+0000 7fee317fa640 1 -- 172.21.15.115:0/580670454 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fee2c0ab0d0 con 0x7fee34164c60 2023-12-01T02:52:05.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:05.993+0000 7fee317fa640 1 -- 172.21.15.115:0/580670454 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fee2c076880 con 0x7fee34164c60 2023-12-01T02:52:06.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.167+0000 7fee3a669640 1 -- 172.21.15.115:0/580670454 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee340cced0 con 0x7fee34164c60 2023-12-01T02:52:06.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.168+0000 7fee317fa640 1 -- 172.21.15.115:0/580670454 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v165) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee2c079ea0 con 0x7fee34164c60 2023-12-01T02:52:06.170 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T02:52:06.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.170+0000 7fee3a669640 1 -- 172.21.15.115:0/580670454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 msgr2=0x7fee3413a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.170+0000 7fee3a669640 1 --2- 172.21.15.115:0/580670454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 0x7fee3413a110 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fee2c036040 tx=0x7fee2c004a40 comp rx=0 tx=0).stop 2023-12-01T02:52:06.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.170+0000 7fee3a669640 1 -- 172.21.15.115:0/580670454 shutdown_connections 2023-12-01T02:52:06.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.170+0000 7fee3a669640 1 --2- 172.21.15.115:0/580670454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee34143eb0 0x7fee341f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.170+0000 7fee3a669640 1 --2- 172.21.15.115:0/580670454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee34164c60 0x7fee3413a110 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.170+0000 7fee3a669640 1 --2- 172.21.15.115:0/580670454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee340cb570 0x7fee341f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.170+0000 7fee3a669640 1 -- 172.21.15.115:0/580670454 >> 172.21.15.115:0/580670454 conn(0x7fee34065960 msgr2=0x7fee3406f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:06.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.171+0000 7fee3a669640 1 -- 172.21.15.115:0/580670454 shutdown_connections 2023-12-01T02:52:06.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.171+0000 7fee3a669640 1 -- 172.21.15.115:0/580670454 wait complete. 2023-12-01T02:52:06.191 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:52:06.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.393+0000 7f7043c84640 1 Processor -- start 2023-12-01T02:52:06.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.393+0000 7f7043c84640 1 -- start start 2023-12-01T02:52:06.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.395+0000 7f7043c84640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f703c15e2f0 0x7f703c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.395+0000 7f7043c84640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f703c0adf10 0x7f703c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.395+0000 7f7043c84640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703c0add10 con 0x7f703c06f2f0 2023-12-01T02:52:06.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.395+0000 7f7043c84640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703c0c5760 con 0x7f703c0adf10 2023-12-01T02:52:06.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.395+0000 7f7043c84640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c15e6d0 con 0x7f703c15e2f0 2023-12-01T02:52:06.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.395+0000 7f7042221640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f703c0adf10 0x7f703c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.395+0000 7f7042221640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f703c0adf10 0x7f703c0caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40130/0 (socket says 172.21.15.115:40130) 2023-12-01T02:52:06.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.395+0000 7f7042221640 1 -- 172.21.15.115:0/3624013816 learned_addr learned my addr 172.21.15.115:0/3624013816 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:06.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.395+0000 7f7041a20640 1 --2- 172.21.15.115:0/3624013816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f703c15e2f0 0x7f703c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.396+0000 7f7042221640 1 -- 172.21.15.115:0/3624013816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f703c15e2f0 msgr2=0x7f703c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:06.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.396+0000 7f7042221640 1 --2- 172.21.15.115:0/3624013816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f703c15e2f0 0x7f703c0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.396+0000 7f7042221640 1 -- 172.21.15.115:0/3624013816 >> v1:172.21.15.142:6789/0 conn(0x7f703c06f2f0 legacy=0x7f703c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.396+0000 7f7042221640 1 -- 172.21.15.115:0/3624013816 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c0cf7b0 con 0x7f703c0adf10 2023-12-01T02:52:06.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.397+0000 7f7042221640 1 --2- 172.21.15.115:0/3624013816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c0adf10 0x7f703c0caba0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f7034009e50 tx=0x7f703402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d20c8e05245a3fc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:06.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.397+0000 7f7040a1e640 1 -- 172.21.15.115:0/3624013816 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703402c9e0 con 0x7f703c0adf10 2023-12-01T02:52:06.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.397+0000 7f7040a1e640 1 -- 172.21.15.115:0/3624013816 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703402cb40 con 0x7f703c0adf10 2023-12-01T02:52:06.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.397+0000 7f7040a1e640 1 -- 172.21.15.115:0/3624013816 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70340343f0 con 0x7f703c0adf10 2023-12-01T02:52:06.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.398+0000 7f7043c84640 1 -- 172.21.15.115:0/3624013816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c0adf10 msgr2=0x7f703c0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.398+0000 7f7043c84640 1 --2- 172.21.15.115:0/3624013816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c0adf10 0x7f703c0caba0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f7034009e50 tx=0x7f703402be40 comp rx=0 tx=0).stop 2023-12-01T02:52:06.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.398+0000 7f7043c84640 1 -- 172.21.15.115:0/3624013816 shutdown_connections 2023-12-01T02:52:06.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.398+0000 7f7043c84640 1 --2- 172.21.15.115:0/3624013816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c0adf10 0x7f703c0caba0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.398+0000 7f7043c84640 1 --2- 172.21.15.115:0/3624013816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f703c15e2f0 0x7f703c0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.398+0000 7f7043c84640 1 -- 172.21.15.115:0/3624013816 >> 172.21.15.115:0/3624013816 conn(0x7f703c0ce880 msgr2=0x7f703c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:06.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.398+0000 7f7043c84640 1 -- 172.21.15.115:0/3624013816 shutdown_connections 2023-12-01T02:52:06.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.398+0000 7f7043c84640 1 -- 172.21.15.115:0/3624013816 wait complete. 2023-12-01T02:52:06.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.400+0000 7f7043c84640 1 Processor -- start 2023-12-01T02:52:06.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.400+0000 7f7043c84640 1 -- start start 2023-12-01T02:52:06.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7043c84640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c06f2f0 0x7f703c14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7043c84640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f703c0adf10 0x7f703c0c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7043c84640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f703c15e2f0 0x7f703c0c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7043c84640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c14f1d0 con 0x7f703c06f2f0 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7043c84640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0c2d70 con 0x7f703c0adf10 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7043c84640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0c3050 con 0x7f703c15e2f0 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7041a20640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c06f2f0 0x7f703c14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7041a20640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c06f2f0 0x7f703c14ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40140/0 (socket says 172.21.15.115:40140) 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7041a20640 1 -- 172.21.15.115:0/1072576507 learned_addr learned my addr 172.21.15.115:0/1072576507 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f704121f640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f703c0adf10 0x7f703c0c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7041a20640 1 -- 172.21.15.115:0/1072576507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f703c15e2f0 msgr2=0x7f703c0c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7041a20640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f703c15e2f0 0x7f703c0c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7041a20640 1 -- 172.21.15.115:0/1072576507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f703c0adf10 msgr2=0x7f703c0c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7041a20640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f703c0adf10 0x7f703c0c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.401+0000 7f7041a20640 1 -- 172.21.15.115:0/1072576507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7034009b30 con 0x7f703c06f2f0 2023-12-01T02:52:06.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.402+0000 7f7041a20640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c06f2f0 0x7f703c14ec90 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f703800d5f0 tx=0x7f703800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:06.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.402+0000 7f70327fc640 1 -- 172.21.15.115:0/1072576507 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70380125f0 con 0x7f703c06f2f0 2023-12-01T02:52:06.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.402+0000 7f70327fc640 1 -- 172.21.15.115:0/1072576507 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7038012750 con 0x7f703c06f2f0 2023-12-01T02:52:06.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.402+0000 7f7043c84640 1 -- 172.21.15.115:0/1072576507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703c0c6290 con 0x7f703c06f2f0 2023-12-01T02:52:06.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.402+0000 7f70327fc640 1 -- 172.21.15.115:0/1072576507 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038012b00 con 0x7f703c06f2f0 2023-12-01T02:52:06.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.402+0000 7f7043c84640 1 -- 172.21.15.115:0/1072576507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703c0c6850 con 0x7f703c06f2f0 2023-12-01T02:52:06.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.403+0000 7f7043c84640 1 -- 172.21.15.115:0/1072576507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703c0c6a10 con 0x7f703c06f2f0 2023-12-01T02:52:06.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.404+0000 7f70327fc640 1 -- 172.21.15.115:0/1072576507 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 92) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f7038012c60 con 0x7f703c06f2f0 2023-12-01T02:52:06.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.405+0000 7f70327fc640 1 -- 172.21.15.115:0/1072576507 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f7038088ac0 con 0x7f703c06f2f0 2023-12-01T02:52:06.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.410+0000 7f70327fc640 1 -- 172.21.15.115:0/1072576507 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7038054270 con 0x7f703c06f2f0 2023-12-01T02:52:06.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.483+0000 7f70327fc640 1 -- 172.21.15.115:0/1072576507 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 93) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f7038053d20 con 0x7f703c06f2f0 2023-12-01T02:52:06.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.484+0000 7f70327fc640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7020035d20 0x7f7020071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.487+0000 7f704121f640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7020035d20 0x7f7020071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.493+0000 7f704121f640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7020035d20 0x7f7020071b50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f703c1f3e40 tx=0x7f702c009040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:06.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.649+0000 7f7043c84640 1 -- 172.21.15.115:0/1072576507 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f703c06fda0 con 0x7f703c06f2f0 2023-12-01T02:52:06.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.651+0000 7f70327fc640 1 -- 172.21.15.115:0/1072576507 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f703806fe60 con 0x7f703c06f2f0 2023-12-01T02:52:06.653 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:06.653 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1952979218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1952979218},{"type":"v1","addr":"172.21.15.142:6833","nonce":1952979218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3618529680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3618529680},{"type":"v1","addr":"172.21.15.142:6835","nonce":3618529680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":9009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1637880399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1637880399},{"type":"v1","addr":"172.21.15.115:6836","nonce":1637880399}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:51:43.827542+0000","modified":"2023-12-01T02:51:46.847110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3770885942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3770885942},{"type":"v1","addr":"172.21.15.115:6837","nonce":3770885942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-01T02:52:06.653 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 71 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 -- 172.21.15.115:0/1072576507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7020035d20 msgr2=0x7f7020071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7020035d20 0x7f7020071b50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f703c1f3e40 tx=0x7f702c009040 comp rx=0 tx=0).stop 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 -- 172.21.15.115:0/1072576507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c06f2f0 msgr2=0x7f703c14ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c06f2f0 0x7f703c14ec90 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f703800d5f0 tx=0x7f703800dab0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 -- 172.21.15.115:0/1072576507 shutdown_connections 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f7020035d20 0x7f7020071b50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f703c15e2f0 0x7f703c0c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f703c0adf10 0x7f703c0c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 --2- 172.21.15.115:0/1072576507 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f703c06f2f0 0x7f703c14ec90 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 -- 172.21.15.115:0/1072576507 >> 172.21.15.115:0/1072576507 conn(0x7f703c0ce880 msgr2=0x7f703c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:06.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 -- 172.21.15.115:0/1072576507 shutdown_connections 2023-12-01T02:52:06.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.654+0000 7f7043c84640 1 -- 172.21.15.115:0/1072576507 wait complete. 2023-12-01T02:52:06.675 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:52:06.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.883+0000 7f0214e51640 1 Processor -- start 2023-12-01T02:52:06.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.883+0000 7f0214e51640 1 -- start start 2023-12-01T02:52:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.885+0000 7f0214e51640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f02100ccc80 0x7f021006fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.885+0000 7f0214e51640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f02100ad2b0 0x7f021006fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.885+0000 7f0214e51640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021015d600 con 0x7f021015d220 2023-12-01T02:52:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.885+0000 7f0214e51640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021006cc10 con 0x7f02100ad2b0 2023-12-01T02:52:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.885+0000 7f0214e51640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021006c720 con 0x7f02100ccc80 2023-12-01T02:52:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.885+0000 7f020ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f02100ad2b0 0x7f021006fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.885+0000 7f020ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f02100ad2b0 0x7f021006fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40152/0 (socket says 172.21.15.115:40152) 2023-12-01T02:52:06.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.885+0000 7f020ed9d640 1 -- 172.21.15.115:0/3886989337 learned_addr learned my addr 172.21.15.115:0/3886989337 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:06.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.886+0000 7f0207fff640 1 --2- 172.21.15.115:0/3886989337 >> v2:172.21.15.142:3300/0 conn(0x7f02100ccc80 0x7f021006fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.886+0000 7f020ed9d640 1 -- 172.21.15.115:0/3886989337 >> v2:172.21.15.142:3300/0 conn(0x7f02100ccc80 msgr2=0x7f021006fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:06.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.886+0000 7f020ed9d640 1 --2- 172.21.15.115:0/3886989337 >> v2:172.21.15.142:3300/0 conn(0x7f02100ccc80 0x7f021006fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.886+0000 7f020ed9d640 1 -- 172.21.15.115:0/3886989337 >> v1:172.21.15.115:6789/0 conn(0x7f021015d220 legacy=0x7f02100cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.886+0000 7f020ed9d640 1 -- 172.21.15.115:0/3886989337 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021006c4b0 con 0x7f02100ad2b0 2023-12-01T02:52:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.887+0000 7f020ed9d640 1 --2- 172.21.15.115:0/3886989337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02100ad2b0 0x7f021006fff0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f02100cc4e0 tx=0x7f020802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca229b3a23736ff server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.887+0000 7f020dd9b640 1 -- 172.21.15.115:0/3886989337 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020802ca80 con 0x7f02100ad2b0 2023-12-01T02:52:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.887+0000 7f020dd9b640 1 -- 172.21.15.115:0/3886989337 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020802cbe0 con 0x7f02100ad2b0 2023-12-01T02:52:06.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.887+0000 7f020dd9b640 1 -- 172.21.15.115:0/3886989337 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02080343f0 con 0x7f02100ad2b0 2023-12-01T02:52:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.888+0000 7f0214e51640 1 -- 172.21.15.115:0/3886989337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02100ad2b0 msgr2=0x7f021006fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.888+0000 7f0214e51640 1 --2- 172.21.15.115:0/3886989337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02100ad2b0 0x7f021006fff0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f02100cc4e0 tx=0x7f020802bee0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.888+0000 7f0214e51640 1 -- 172.21.15.115:0/3886989337 shutdown_connections 2023-12-01T02:52:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.888+0000 7f0214e51640 1 --2- 172.21.15.115:0/3886989337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02100ad2b0 0x7f021006fff0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.888+0000 7f0214e51640 1 --2- 172.21.15.115:0/3886989337 >> v2:172.21.15.142:3300/0 conn(0x7f02100ccc80 0x7f021006fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.888+0000 7f0214e51640 1 -- 172.21.15.115:0/3886989337 >> 172.21.15.115:0/3886989337 conn(0x7f02100ce880 msgr2=0x7f02100cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.888+0000 7f0214e51640 1 -- 172.21.15.115:0/3886989337 shutdown_connections 2023-12-01T02:52:06.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.888+0000 7f0214e51640 1 -- 172.21.15.115:0/3886989337 wait complete. 2023-12-01T02:52:06.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.890+0000 7f0214e51640 1 Processor -- start 2023-12-01T02:52:06.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.890+0000 7f0214e51640 1 -- start start 2023-12-01T02:52:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f0214e51640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02100ad2b0 0x7f021014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f0214e51640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02100ccc80 0x7f02101e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f0214e51640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f021015d220 0x7f02101e3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f0214e51640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02101e6460 con 0x7f021015d220 2023-12-01T02:52:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f0214e51640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02101e6700 con 0x7f02100ad2b0 2023-12-01T02:52:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f0214e51640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02101e69e0 con 0x7f02100ccc80 2023-12-01T02:52:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f020ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f021015d220 0x7f02101e3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f020ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f021015d220 0x7f02101e3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40156/0 (socket says 172.21.15.115:40156) 2023-12-01T02:52:06.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f020ed9d640 1 -- 172.21.15.115:0/2187134266 learned_addr learned my addr 172.21.15.115:0/2187134266 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:06.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f020e59c640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02100ad2b0 0x7f021014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f0207fff640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02100ccc80 0x7f02101e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f020ed9d640 1 -- 172.21.15.115:0/2187134266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02100ccc80 msgr2=0x7f02101e5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f020ed9d640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02100ccc80 0x7f02101e5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f020ed9d640 1 -- 172.21.15.115:0/2187134266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02100ad2b0 msgr2=0x7f021014db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:06.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f020ed9d640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02100ad2b0 0x7f021014db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:06.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.891+0000 7f020ed9d640 1 -- 172.21.15.115:0/2187134266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0208009b30 con 0x7f021015d220 2023-12-01T02:52:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.892+0000 7f020ed9d640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f021015d220 0x7f02101e3b40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f0208036040 tx=0x7f0208004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.892+0000 7f0206ffd640 1 -- 172.21.15.115:0/2187134266 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208009510 con 0x7f021015d220 2023-12-01T02:52:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.892+0000 7f0206ffd640 1 -- 172.21.15.115:0/2187134266 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020803e450 con 0x7f021015d220 2023-12-01T02:52:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.892+0000 7f0206ffd640 1 -- 172.21.15.115:0/2187134266 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020803e800 con 0x7f021015d220 2023-12-01T02:52:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.892+0000 7f0214e51640 1 -- 172.21.15.115:0/2187134266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02101e6cc0 con 0x7f021015d220 2023-12-01T02:52:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.892+0000 7f0214e51640 1 -- 172.21.15.115:0/2187134266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021014e090 con 0x7f021015d220 2023-12-01T02:52:06.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.893+0000 7f0214e51640 1 -- 172.21.15.115:0/2187134266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021004e890 con 0x7f021015d220 2023-12-01T02:52:06.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.894+0000 7f0206ffd640 1 -- 172.21.15.115:0/2187134266 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 93) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f020802cd50 con 0x7f021015d220 2023-12-01T02:52:06.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.895+0000 7f0206ffd640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01f006a280 0x7f01f006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:06.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.895+0000 7f0206ffd640 1 -- 172.21.15.115:0/2187134266 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f02080ab390 con 0x7f021015d220 2023-12-01T02:52:06.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.896+0000 7f020e59c640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01f006a280 0x7f01f006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:06.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.896+0000 7f020e59c640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01f006a280 0x7f01f006c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0200000f60 tx=0x7f0200009040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:06.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:06.900+0000 7f0206ffd640 1 -- 172.21.15.115:0/2187134266 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0208076960 con 0x7f021015d220 2023-12-01T02:52:07.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.088+0000 7f0214e51640 1 -- 172.21.15.115:0/2187134266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02101e7180 con 0x7f021015d220 2023-12-01T02:52:07.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.090+0000 7f0206ffd640 1 -- 172.21.15.115:0/2187134266 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+17051 (secure 0 0 0) 0x7f0208079f80 con 0x7f021015d220 2023-12-01T02:52:07.096 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:07.096 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":165,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:52:05.454646+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:51:41.383835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-01T02:51:42.832726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2293895421":"2023-12-02T02:52:05.454600+0000","172.21.15.115:0/3717000617":"2023-12-02T02:52:05.454600+0000","172.21.15.115:0/3148712762":"2023-12-02T02:52:05.454600+0000","172.21.15.115:0/671507930":"2023-12-02T02:52:05.454600+0000","172.21.15.115:0/137452291":"2023-12-02T02:52:05.454600+0000","172.21.15.115:0/3435146247":"2023-12-02T02:52:05.454600+0000","172.21.15.115:6824/88317":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/2871788797":"2023-12-02T02:51:58.123217+0000","172.21.15.115:6822/88317":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/2571267301":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/1473681116":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/3088218027":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/500113028":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/1225510452":"2023-12-02T02:51:58.123217+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.092+0000 7f0214e51640 1 -- 172.21.15.115:0/2187134266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01f006a280 msgr2=0x7f01f006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.092+0000 7f0214e51640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01f006a280 0x7f01f006c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0200000f60 tx=0x7f0200009040 comp rx=0 tx=0).stop 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.092+0000 7f0214e51640 1 -- 172.21.15.115:0/2187134266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f021015d220 msgr2=0x7f02101e3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.092+0000 7f0214e51640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f021015d220 0x7f02101e3b40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f0208036040 tx=0x7f0208004a40 comp rx=0 tx=0).stop 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.093+0000 7f0214e51640 1 -- 172.21.15.115:0/2187134266 shutdown_connections 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.093+0000 7f0214e51640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f01f006a280 0x7f01f006c740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.093+0000 7f0214e51640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02100ccc80 0x7f02101e5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.093+0000 7f0214e51640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f021015d220 0x7f02101e3b40 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.093+0000 7f0214e51640 1 --2- 172.21.15.115:0/2187134266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02100ad2b0 0x7f021014db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.093+0000 7f0214e51640 1 -- 172.21.15.115:0/2187134266 >> 172.21.15.115:0/2187134266 conn(0x7f02100ce880 msgr2=0x7f021006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.093+0000 7f0214e51640 1 -- 172.21.15.115:0/2187134266 shutdown_connections 2023-12-01T02:52:07.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.093+0000 7f0214e51640 1 -- 172.21.15.115:0/2187134266 wait complete. 2023-12-01T02:52:07.112 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T02:52:07.113 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:52:07.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.319+0000 7fcfb900b640 1 Processor -- start 2023-12-01T02:52:07.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.319+0000 7fcfb900b640 1 -- start start 2023-12-01T02:52:07.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.320+0000 7fcfb900b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcfb40cad70 0x7fcfb406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb900b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcfb406f2f0 0x7fcfb406bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb900b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb40add10 con 0x7fcfb415e2f0 2023-12-01T02:52:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb900b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb40c5760 con 0x7fcfb406f2f0 2023-12-01T02:52:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb900b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb415e6d0 con 0x7fcfb40cad70 2023-12-01T02:52:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb2d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fcfb415e2f0 0x7fcfb40ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:47802/0 (socket says 172.21.15.115:47802) 2023-12-01T02:52:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb2d9d640 1 -- 172.21.15.115:0/22906979 learned_addr learned my addr 172.21.15.115:0/22906979 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:07.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb359e640 1 --2- 172.21.15.115:0/22906979 >> v2:172.21.15.115:3300/0 conn(0x7fcfb406f2f0 0x7fcfb406bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb259c640 1 --2- 172.21.15.115:0/22906979 >> v2:172.21.15.142:3300/0 conn(0x7fcfb40cad70 0x7fcfb406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb359e640 1 -- 172.21.15.115:0/22906979 >> v2:172.21.15.142:3300/0 conn(0x7fcfb40cad70 msgr2=0x7fcfb406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb1d9b640 1 -- 172.21.15.115:0/22906979 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608160700 0 0) 0x7fcfb40add10 con 0x7fcfb415e2f0 2023-12-01T02:52:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb359e640 1 --2- 172.21.15.115:0/22906979 >> v2:172.21.15.142:3300/0 conn(0x7fcfb40cad70 0x7fcfb406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb359e640 1 -- 172.21.15.115:0/22906979 >> v1:172.21.15.115:6789/0 conn(0x7fcfb415e2f0 legacy=0x7fcfb40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb359e640 1 -- 172.21.15.115:0/22906979 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40cf7b0 con 0x7fcfb406f2f0 2023-12-01T02:52:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.321+0000 7fcfb259c640 1 --2- 172.21.15.115:0/22906979 >> v2:172.21.15.142:3300/0 conn(0x7fcfb40cad70 0x7fcfb406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:07.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.322+0000 7fcfb359e640 1 --2- 172.21.15.115:0/22906979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb406f2f0 0x7fcfb406bb00 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009d90 tx=0x7fcfac02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c7fb1af550f58 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb1d9b640 1 -- 172.21.15.115:0/22906979 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac02cb40 con 0x7fcfb406f2f0 2023-12-01T02:52:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb1d9b640 1 -- 172.21.15.115:0/22906979 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac02cca0 con 0x7fcfb406f2f0 2023-12-01T02:52:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb1d9b640 1 -- 172.21.15.115:0/22906979 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac034660 con 0x7fcfb406f2f0 2023-12-01T02:52:07.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb900b640 1 -- 172.21.15.115:0/22906979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb406f2f0 msgr2=0x7fcfb406bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb900b640 1 --2- 172.21.15.115:0/22906979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb406f2f0 0x7fcfb406bb00 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009d90 tx=0x7fcfac02bd90 comp rx=0 tx=0).stop 2023-12-01T02:52:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb900b640 1 -- 172.21.15.115:0/22906979 shutdown_connections 2023-12-01T02:52:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb900b640 1 --2- 172.21.15.115:0/22906979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb406f2f0 0x7fcfb406bb00 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb900b640 1 --2- 172.21.15.115:0/22906979 >> v2:172.21.15.142:3300/0 conn(0x7fcfb40cad70 0x7fcfb406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb900b640 1 -- 172.21.15.115:0/22906979 >> 172.21.15.115:0/22906979 conn(0x7fcfb40ce880 msgr2=0x7fcfb40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb900b640 1 -- 172.21.15.115:0/22906979 shutdown_connections 2023-12-01T02:52:07.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.323+0000 7fcfb900b640 1 -- 172.21.15.115:0/22906979 wait complete. 2023-12-01T02:52:07.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.325+0000 7fcfb900b640 1 Processor -- start 2023-12-01T02:52:07.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.325+0000 7fcfb900b640 1 -- start start 2023-12-01T02:52:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.326+0000 7fcfb900b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb406f2f0 0x7fcfb41f6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.326+0000 7fcfb900b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcfb40cad70 0x7fcfb40cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.326+0000 7fcfb900b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb415e2f0 0x7fcfb4157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.326+0000 7fcfb900b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40cd810 con 0x7fcfb415e2f0 2023-12-01T02:52:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.326+0000 7fcfb900b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb41f3170 con 0x7fcfb406f2f0 2023-12-01T02:52:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.326+0000 7fcfb900b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40cda50 con 0x7fcfb40cad70 2023-12-01T02:52:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.326+0000 7fcfb359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb415e2f0 0x7fcfb4157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.326+0000 7fcfb359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb415e2f0 0x7fcfb4157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:40172/0 (socket says 172.21.15.115:40172) 2023-12-01T02:52:07.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.326+0000 7fcfb359e640 1 -- 172.21.15.115:0/1437386719 learned_addr learned my addr 172.21.15.115:0/1437386719 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcfb259c640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcfb40cad70 0x7fcfb40cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcfb359e640 1 -- 172.21.15.115:0/1437386719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcfb40cad70 msgr2=0x7fcfb40cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcfb359e640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcfb40cad70 0x7fcfb40cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcfb359e640 1 -- 172.21.15.115:0/1437386719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb406f2f0 msgr2=0x7fcfb41f6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcfb2d9d640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb406f2f0 0x7fcfb41f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcfb359e640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb406f2f0 0x7fcfb41f6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcfb359e640 1 -- 172.21.15.115:0/1437386719 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac009a70 con 0x7fcfb415e2f0 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcfb359e640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb415e2f0 0x7fcfb4157b80 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009d90 tx=0x7fcfac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcf9b7fe640 1 -- 172.21.15.115:0/1437386719 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac040650 con 0x7fcfb415e2f0 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcf9b7fe640 1 -- 172.21.15.115:0/1437386719 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac0407d0 con 0x7fcfb415e2f0 2023-12-01T02:52:07.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcfb900b640 1 -- 172.21.15.115:0/1437386719 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb41f2840 con 0x7fcfb415e2f0 2023-12-01T02:52:07.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.327+0000 7fcf9b7fe640 1 -- 172.21.15.115:0/1437386719 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac040b80 con 0x7fcfb415e2f0 2023-12-01T02:52:07.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.328+0000 7fcfb900b640 1 -- 172.21.15.115:0/1437386719 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb41f2e00 con 0x7fcfb415e2f0 2023-12-01T02:52:07.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.329+0000 7fcfb900b640 1 -- 172.21.15.115:0/1437386719 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb406fda0 con 0x7fcfb415e2f0 2023-12-01T02:52:07.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.329+0000 7fcf9b7fe640 1 -- 172.21.15.115:0/1437386719 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 93) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fcfac040ce0 con 0x7fcfb415e2f0 2023-12-01T02:52:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.330+0000 7fcf9b7fe640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcfa006a280 0x7fcfa006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.330+0000 7fcf9b7fe640 1 -- 172.21.15.115:0/1437386719 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fcfac0ab820 con 0x7fcfb415e2f0 2023-12-01T02:52:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.330+0000 7fcfb2d9d640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcfa006a280 0x7fcfa006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:07.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.331+0000 7fcfb2d9d640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcfa006a280 0x7fcfa006c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4000f60 tx=0x7fcfa4009040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:07.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.335+0000 7fcf9b7fe640 1 -- 172.21.15.115:0/1437386719 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcfac076df0 con 0x7fcfb415e2f0 2023-12-01T02:52:07.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.561+0000 7fcfb900b640 1 -- 172.21.15.115:0/1437386719 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfb41f3300 con 0x7fcfb415e2f0 2023-12-01T02:52:07.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.561+0000 7fcf9b7fe640 1 -- 172.21.15.115:0/1437386719 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4662 (secure 0 0 0) 0x7fcfac0a9e60 con 0x7fcfb415e2f0 2023-12-01T02:52:07.563 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:07.564 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1952979218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1952979218},{"type":"v1","addr":"172.21.15.142:6833","nonce":1952979218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3618529680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3618529680},{"type":"v1","addr":"172.21.15.142:6835","nonce":3618529680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":9009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1637880399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1637880399},{"type":"v1","addr":"172.21.15.115:6836","nonce":1637880399}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:51:43.827542+0000","modified":"2023-12-01T02:51:46.847110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3770885942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3770885942},{"type":"v1","addr":"172.21.15.115:6837","nonce":3770885942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-01T02:52:07.564 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 71 2023-12-01T02:52:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.564+0000 7fcfb900b640 1 -- 172.21.15.115:0/1437386719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcfa006a280 msgr2=0x7fcfa006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.564+0000 7fcfb900b640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcfa006a280 0x7fcfa006c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4000f60 tx=0x7fcfa4009040 comp rx=0 tx=0).stop 2023-12-01T02:52:07.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.564+0000 7fcfb900b640 1 -- 172.21.15.115:0/1437386719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb415e2f0 msgr2=0x7fcfb4157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.564+0000 7fcfb900b640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb415e2f0 0x7fcfb4157b80 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009d90 tx=0x7fcfac004a40 comp rx=0 tx=0).stop 2023-12-01T02:52:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.564+0000 7fcfb900b640 1 -- 172.21.15.115:0/1437386719 shutdown_connections 2023-12-01T02:52:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.564+0000 7fcfb900b640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcfa006a280 0x7fcfa006c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.564+0000 7fcfb900b640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcfb40cad70 0x7fcfb40cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.564+0000 7fcfb900b640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfb415e2f0 0x7fcfb4157b80 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.565+0000 7fcfb900b640 1 --2- 172.21.15.115:0/1437386719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfb406f2f0 0x7fcfb41f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.565+0000 7fcfb900b640 1 -- 172.21.15.115:0/1437386719 >> 172.21.15.115:0/1437386719 conn(0x7fcfb40ce880 msgr2=0x7fcfb415c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.565+0000 7fcfb900b640 1 -- 172.21.15.115:0/1437386719 shutdown_connections 2023-12-01T02:52:07.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.565+0000 7fcfb900b640 1 -- 172.21.15.115:0/1437386719 wait complete. 2023-12-01T02:52:07.584 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:52:07.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.764+0000 7f8c23e5f640 1 Processor -- start 2023-12-01T02:52:07.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.764+0000 7f8c23e5f640 1 -- start start 2023-12-01T02:52:07.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.766+0000 7f8c23e5f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8c1c15cd20 0x7f8c1c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.766+0000 7f8c23e5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c1c0d1230 0x7f8c1c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.766+0000 7f8c23e5f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c0d1b90 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.766+0000 7f8c23e5f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c0b2160 con 0x7f8c1c15cd20 2023-12-01T02:52:07.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.766+0000 7f8c23e5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0b8c80 con 0x7f8c1c0d1230 2023-12-01T02:52:07.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.766+0000 7f8c21bfb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8c1c15cd20 0x7f8c1c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:07.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.766+0000 7f8c213fa640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8c1c0b5d80 0x7f8c1c0b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60112/0 (socket says 172.21.15.115:60112) 2023-12-01T02:52:07.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.766+0000 7f8c213fa640 1 -- 172.21.15.115:0/550515215 learned_addr learned my addr 172.21.15.115:0/550515215 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:07.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.767+0000 7f8c223fc640 1 --2- 172.21.15.115:0/550515215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c1c0d1230 0x7f8c1c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:07.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.767+0000 7f8c21bfb640 1 -- 172.21.15.115:0/550515215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c1c0d1230 msgr2=0x7f8c1c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:07.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.767+0000 7f8c21bfb640 1 --2- 172.21.15.115:0/550515215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c1c0d1230 0x7f8c1c0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.767+0000 7f8c21bfb640 1 -- 172.21.15.115:0/550515215 >> v1:172.21.15.115:6789/0 conn(0x7f8c1c0b5d80 legacy=0x7f8c1c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.767+0000 7f8c21bfb640 1 -- 172.21.15.115:0/550515215 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0ce2e0 con 0x7f8c1c15cd20 2023-12-01T02:52:07.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.767+0000 7f8c223fc640 1 --2- 172.21.15.115:0/550515215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c1c0d1230 0x7f8c1c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:07.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.768+0000 7f8c21bfb640 1 --2- 172.21.15.115:0/550515215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c15cd20 0x7f8c1c0d24d0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f8c180097a0 tx=0x7f8c1802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17e6a973f19bdac server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.768+0000 7f8c20bf9640 1 -- 172.21.15.115:0/550515215 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18031780 con 0x7f8c1c15cd20 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.768+0000 7f8c20bf9640 1 -- 172.21.15.115:0/550515215 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c180318e0 con 0x7f8c1c15cd20 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.768+0000 7f8c20bf9640 1 -- 172.21.15.115:0/550515215 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18031c70 con 0x7f8c1c15cd20 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.769+0000 7f8c23e5f640 1 -- 172.21.15.115:0/550515215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c15cd20 msgr2=0x7f8c1c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.769+0000 7f8c23e5f640 1 --2- 172.21.15.115:0/550515215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c15cd20 0x7f8c1c0d24d0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f8c180097a0 tx=0x7f8c1802b820 comp rx=0 tx=0).stop 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.769+0000 7f8c23e5f640 1 -- 172.21.15.115:0/550515215 shutdown_connections 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.769+0000 7f8c23e5f640 1 --2- 172.21.15.115:0/550515215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c1c0d1230 0x7f8c1c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.769+0000 7f8c23e5f640 1 --2- 172.21.15.115:0/550515215 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c15cd20 0x7f8c1c0d24d0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.769+0000 7f8c23e5f640 1 -- 172.21.15.115:0/550515215 >> 172.21.15.115:0/550515215 conn(0x7f8c1c06a0e0 msgr2=0x7f8c1c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.769+0000 7f8c23e5f640 1 -- 172.21.15.115:0/550515215 shutdown_connections 2023-12-01T02:52:07.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.769+0000 7f8c23e5f640 1 -- 172.21.15.115:0/550515215 wait complete. 2023-12-01T02:52:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.771+0000 7f8c23e5f640 1 Processor -- start 2023-12-01T02:52:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.771+0000 7f8c23e5f640 1 -- start start 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.772+0000 7f8c23e5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0b5d80 0x7f8c1c157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.772+0000 7f8c23e5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c1c0d1230 0x7f8c1c1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.772+0000 7f8c23e5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c1c15cd20 0x7f8c1c0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.772+0000 7f8c23e5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c15ace0 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.772+0000 7f8c23e5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c15dff0 con 0x7f8c1c15cd20 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.772+0000 7f8c23e5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c15d100 con 0x7f8c1c0d1230 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.773+0000 7f8c21bfb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0b5d80 0x7f8c1c157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.773+0000 7f8c21bfb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0b5d80 0x7f8c1c157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43210/0 (socket says 172.21.15.115:43210) 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.773+0000 7f8c21bfb640 1 -- 172.21.15.115:0/1246558003 learned_addr learned my addr 172.21.15.115:0/1246558003 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.773+0000 7f8c21bfb640 1 -- 172.21.15.115:0/1246558003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c1c0d1230 msgr2=0x7f8c1c1582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.773+0000 7f8c21bfb640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c1c0d1230 0x7f8c1c1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.773+0000 7f8c21bfb640 1 -- 172.21.15.115:0/1246558003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c1c15cd20 msgr2=0x7f8c1c0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:07.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.773+0000 7f8c21bfb640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c1c15cd20 0x7f8c1c0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:07.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.773+0000 7f8c21bfb640 1 -- 172.21.15.115:0/1246558003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c18009400 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.773+0000 7f8c21bfb640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0b5d80 0x7f8c1c157d80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f8c18034040 tx=0x7f8c180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:07.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.774+0000 7f8c127fc640 1 -- 172.21.15.115:0/1246558003 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18032960 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.774+0000 7f8c127fc640 1 -- 172.21.15.115:0/1246558003 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c18032ac0 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.774+0000 7f8c127fc640 1 -- 172.21.15.115:0/1246558003 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18032e70 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.774+0000 7f8c23e5f640 1 -- 172.21.15.115:0/1246558003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c15d3e0 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.774+0000 7f8c23e5f640 1 -- 172.21.15.115:0/1246558003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c15d9a0 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.775+0000 7f8c23e5f640 1 -- 172.21.15.115:0/1246558003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c0d2990 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.776+0000 7f8c127fc640 1 -- 172.21.15.115:0/1246558003 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 93) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f8c1803b3f0 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.777+0000 7f8c127fc640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8c0006a280 0x7f8c0006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.777+0000 7f8c127fc640 1 -- 172.21.15.115:0/1246558003 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f8c180aad40 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.778+0000 7f8c213fa640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8c0006a280 0x7f8c0006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:07.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.778+0000 7f8c213fa640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8c0006a280 0x7f8c0006c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0cddf0 tx=0x7f8c0c007a40 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:07.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.782+0000 7f8c127fc640 1 -- 172.21.15.115:0/1246558003 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8c18076310 con 0x7f8c1c0b5d80 2023-12-01T02:52:07.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:07.972+0000 7f8c127fc640 1 -- 172.21.15.115:0/1246558003 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 94) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f8c180766e0 con 0x7f8c1c0b5d80 2023-12-01T02:52:08.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.021+0000 7f8c23e5f640 1 -- 172.21.15.115:0/1246558003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c1c04e890 con 0x7f8c1c0b5d80 2023-12-01T02:52:08.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.022+0000 7f8c127fc640 1 -- 172.21.15.115:0/1246558003 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4662 (secure 0 0 0) 0x7f8c1808ee60 con 0x7f8c1c0b5d80 2023-12-01T02:52:08.024 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:08.024 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1952979218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1952979218},{"type":"v1","addr":"172.21.15.142:6833","nonce":1952979218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3618529680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3618529680},{"type":"v1","addr":"172.21.15.142:6835","nonce":3618529680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":9009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1637880399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1637880399},{"type":"v1","addr":"172.21.15.115:6836","nonce":1637880399}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:51:43.827542+0000","modified":"2023-12-01T02:51:46.847110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/3770885942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":3770885942},{"type":"v1","addr":"172.21.15.115:6837","nonce":3770885942}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-01T02:52:08.025 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 71 2023-12-01T02:52:08.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 -- 172.21.15.115:0/1246558003 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8c0006a280 msgr2=0x7f8c0006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8c0006a280 0x7f8c0006c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0cddf0 tx=0x7f8c0c007a40 comp rx=0 tx=0).stop 2023-12-01T02:52:08.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 -- 172.21.15.115:0/1246558003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0b5d80 msgr2=0x7f8c1c157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0b5d80 0x7f8c1c157d80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f8c18034040 tx=0x7f8c180046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 -- 172.21.15.115:0/1246558003 shutdown_connections 2023-12-01T02:52:08.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8c0006a280 0x7f8c0006c740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c1c0d1230 0x7f8c1c1582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c1c15cd20 0x7f8c1c0b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 --2- 172.21.15.115:0/1246558003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0b5d80 0x7f8c1c157d80 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 -- 172.21.15.115:0/1246558003 >> 172.21.15.115:0/1246558003 conn(0x7f8c1c06a0e0 msgr2=0x7f8c1c0b7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:08.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 -- 172.21.15.115:0/1246558003 shutdown_connections 2023-12-01T02:52:08.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.025+0000 7f8c23e5f640 1 -- 172.21.15.115:0/1246558003 wait complete. 2023-12-01T02:52:08.045 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:52:08.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.247+0000 7fcd27869640 1 Processor -- start 2023-12-01T02:52:08.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.248+0000 7fcd27869640 1 -- start start 2023-12-01T02:52:08.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.249+0000 7fcd27869640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd2015a8f0 0x7fcd200ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.249+0000 7fcd27869640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd2015acd0 0x7fcd2015d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.249+0000 7fcd27869640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015da50 0x7fcd200cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.249+0000 7fcd27869640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2015dee0 con 0x7fcd2015a8f0 2023-12-01T02:52:08.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.249+0000 7fcd27869640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd200cdc10 con 0x7fcd2015acd0 2023-12-01T02:52:08.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.249+0000 7fcd27869640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2015e0d0 con 0x7fcd2015da50 2023-12-01T02:52:08.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.249+0000 7fcd25605640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd2015a8f0 0x7fcd200ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.249+0000 7fcd25605640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd2015a8f0 0x7fcd200ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43220/0 (socket says 172.21.15.115:43220) 2023-12-01T02:52:08.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.249+0000 7fcd25605640 1 -- 172.21.15.115:0/861868446 learned_addr learned my addr 172.21.15.115:0/861868446 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:08.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.250+0000 7fcd24e04640 1 --2- 172.21.15.115:0/861868446 >> v2:172.21.15.142:3300/0 conn(0x7fcd2015acd0 0x7fcd2015d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.250+0000 7fcd25e06640 1 --2- 172.21.15.115:0/861868446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015da50 0x7fcd200cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.250+0000 7fcd25605640 1 -- 172.21.15.115:0/861868446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015da50 msgr2=0x7fcd200cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:08.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.250+0000 7fcd25605640 1 --2- 172.21.15.115:0/861868446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015da50 0x7fcd200cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.250+0000 7fcd25605640 1 -- 172.21.15.115:0/861868446 >> v2:172.21.15.142:3300/0 conn(0x7fcd2015acd0 msgr2=0x7fcd2015d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:08.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.250+0000 7fcd25605640 1 --2- 172.21.15.115:0/861868446 >> v2:172.21.15.142:3300/0 conn(0x7fcd2015acd0 0x7fcd2015d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.250+0000 7fcd25605640 1 -- 172.21.15.115:0/861868446 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2015e3b0 con 0x7fcd2015a8f0 2023-12-01T02:52:08.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.250+0000 7fcd24e04640 1 --2- 172.21.15.115:0/861868446 >> v2:172.21.15.142:3300/0 conn(0x7fcd2015acd0 0x7fcd2015d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:08.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.250+0000 7fcd25e06640 1 --2- 172.21.15.115:0/861868446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015da50 0x7fcd200cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:08.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.251+0000 7fcd25605640 1 --2- 172.21.15.115:0/861868446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015a8f0 0x7fcd200ce0a0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fcd20071210 tx=0x7fcd1402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02c6ca28872468e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:08.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.252+0000 7fcd1ffff640 1 -- 172.21.15.115:0/861868446 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1402ca80 con 0x7fcd2015a8f0 2023-12-01T02:52:08.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.252+0000 7fcd1ffff640 1 -- 172.21.15.115:0/861868446 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1402cbe0 con 0x7fcd2015a8f0 2023-12-01T02:52:08.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.252+0000 7fcd1ffff640 1 -- 172.21.15.115:0/861868446 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd140343f0 con 0x7fcd2015a8f0 2023-12-01T02:52:08.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.253+0000 7fcd27869640 1 -- 172.21.15.115:0/861868446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015a8f0 msgr2=0x7fcd200ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.253+0000 7fcd27869640 1 --2- 172.21.15.115:0/861868446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015a8f0 0x7fcd200ce0a0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fcd20071210 tx=0x7fcd1402bee0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.253+0000 7fcd27869640 1 -- 172.21.15.115:0/861868446 shutdown_connections 2023-12-01T02:52:08.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.253+0000 7fcd27869640 1 --2- 172.21.15.115:0/861868446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015da50 0x7fcd200cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.253+0000 7fcd27869640 1 --2- 172.21.15.115:0/861868446 >> v2:172.21.15.142:3300/0 conn(0x7fcd2015acd0 0x7fcd2015d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.253+0000 7fcd27869640 1 --2- 172.21.15.115:0/861868446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015a8f0 0x7fcd200ce0a0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.253+0000 7fcd27869640 1 -- 172.21.15.115:0/861868446 >> 172.21.15.115:0/861868446 conn(0x7fcd20065960 msgr2=0x7fcd20065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:08.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.253+0000 7fcd27869640 1 -- 172.21.15.115:0/861868446 shutdown_connections 2023-12-01T02:52:08.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.253+0000 7fcd27869640 1 -- 172.21.15.115:0/861868446 wait complete. 2023-12-01T02:52:08.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.254+0000 7fcd27869640 1 Processor -- start 2023-12-01T02:52:08.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.255+0000 7fcd27869640 1 -- start start 2023-12-01T02:52:08.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.255+0000 7fcd27869640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd2015a8f0 0x7fcd20148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd27869640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015acd0 0x7fcd2015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd27869640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015da50 0x7fcd2015fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd27869640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201488a0 con 0x7fcd2015da50 2023-12-01T02:52:08.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd27869640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201f60b0 con 0x7fcd2015a8f0 2023-12-01T02:52:08.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd27869640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201f6390 con 0x7fcd2015acd0 2023-12-01T02:52:08.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd25e06640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015da50 0x7fcd2015fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd25e06640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015da50 0x7fcd2015fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43222/0 (socket says 172.21.15.115:43222) 2023-12-01T02:52:08.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd25e06640 1 -- 172.21.15.115:0/2782989915 learned_addr learned my addr 172.21.15.115:0/2782989915 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:08.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd24e04640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015acd0 0x7fcd2015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd25605640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd2015a8f0 0x7fcd20148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd25e06640 1 -- 172.21.15.115:0/2782989915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015acd0 msgr2=0x7fcd2015f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd25e06640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015acd0 0x7fcd2015f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd25e06640 1 -- 172.21.15.115:0/2782989915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd2015a8f0 msgr2=0x7fcd20148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd25e06640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd2015a8f0 0x7fcd20148360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd25e06640 1 -- 172.21.15.115:0/2782989915 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd14009b30 con 0x7fcd2015da50 2023-12-01T02:52:08.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.256+0000 7fcd24e04640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015acd0 0x7fcd2015f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:08.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.257+0000 7fcd25e06640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015da50 0x7fcd2015fd10 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800c5e0 tx=0x7fcd1800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:08.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.257+0000 7fcd1dffb640 1 -- 172.21.15.115:0/2782989915 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18011800 con 0x7fcd2015da50 2023-12-01T02:52:08.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.257+0000 7fcd1dffb640 1 -- 172.21.15.115:0/2782989915 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd180119a0 con 0x7fcd2015da50 2023-12-01T02:52:08.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.257+0000 7fcd1dffb640 1 -- 172.21.15.115:0/2782989915 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18011d50 con 0x7fcd2015da50 2023-12-01T02:52:08.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.257+0000 7fcd27869640 1 -- 172.21.15.115:0/2782989915 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd201f6670 con 0x7fcd2015da50 2023-12-01T02:52:08.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.257+0000 7fcd27869640 1 -- 172.21.15.115:0/2782989915 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd201f6c30 con 0x7fcd2015da50 2023-12-01T02:52:08.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.258+0000 7fcd27869640 1 -- 172.21.15.115:0/2782989915 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2004e890 con 0x7fcd2015da50 2023-12-01T02:52:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.259+0000 7fcd1dffb640 1 -- 172.21.15.115:0/2782989915 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 94) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fcd18022350 con 0x7fcd2015da50 2023-12-01T02:52:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.260+0000 7fcd1dffb640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd0406a4c0 0x7fcd0406c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.260+0000 7fcd1dffb640 1 -- 172.21.15.115:0/2782989915 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fcd18088cd0 con 0x7fcd2015da50 2023-12-01T02:52:08.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.260+0000 7fcd25605640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd0406a4c0 0x7fcd0406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.261+0000 7fcd25605640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd0406a4c0 0x7fcd0406c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcd14035000 tx=0x7fcd14030000 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:08.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.264+0000 7fcd1dffb640 1 -- 172.21.15.115:0/2782989915 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcd18053fc0 con 0x7fcd2015da50 2023-12-01T02:52:08.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.458+0000 7fcd27869640 1 -- 172.21.15.115:0/2782989915 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd200cd080 con 0x7fcd2015da50 2023-12-01T02:52:08.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.460+0000 7fcd1dffb640 1 -- 172.21.15.115:0/2782989915 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+17051 (secure 0 0 0) 0x7fcd1801f090 con 0x7fcd2015da50 2023-12-01T02:52:08.463 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:08.463 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":165,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:52:05.454646+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:51:41.383835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-01T02:51:42.832726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2293895421":"2023-12-02T02:52:05.454600+0000","172.21.15.115:0/3717000617":"2023-12-02T02:52:05.454600+0000","172.21.15.115:0/3148712762":"2023-12-02T02:52:05.454600+0000","172.21.15.115:0/671507930":"2023-12-02T02:52:05.454600+0000","172.21.15.115:0/137452291":"2023-12-02T02:52:05.454600+0000","172.21.15.115:0/3435146247":"2023-12-02T02:52:05.454600+0000","172.21.15.115:6824/88317":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/2871788797":"2023-12-02T02:51:58.123217+0000","172.21.15.115:6822/88317":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/2571267301":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/1473681116":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/3088218027":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/500113028":"2023-12-02T02:51:58.123217+0000","172.21.15.115:0/1225510452":"2023-12-02T02:51:58.123217+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:52:08.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.463+0000 7fcd27869640 1 -- 172.21.15.115:0/2782989915 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd0406a4c0 msgr2=0x7fcd0406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.463+0000 7fcd27869640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd0406a4c0 0x7fcd0406c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcd14035000 tx=0x7fcd14030000 comp rx=0 tx=0).stop 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.463+0000 7fcd27869640 1 -- 172.21.15.115:0/2782989915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015da50 msgr2=0x7fcd2015fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.463+0000 7fcd27869640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015da50 0x7fcd2015fd10 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800c5e0 tx=0x7fcd1800caa0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.464+0000 7fcd27869640 1 -- 172.21.15.115:0/2782989915 shutdown_connections 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.464+0000 7fcd27869640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fcd0406a4c0 0x7fcd0406c980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.464+0000 7fcd27869640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd2015acd0 0x7fcd2015f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.464+0000 7fcd27869640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2015da50 0x7fcd2015fd10 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.464+0000 7fcd27869640 1 --2- 172.21.15.115:0/2782989915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd2015a8f0 0x7fcd20148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.464+0000 7fcd27869640 1 -- 172.21.15.115:0/2782989915 >> 172.21.15.115:0/2782989915 conn(0x7fcd20065960 msgr2=0x7fcd20159400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.464+0000 7fcd27869640 1 -- 172.21.15.115:0/2782989915 shutdown_connections 2023-12-01T02:52:08.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.464+0000 7fcd27869640 1 -- 172.21.15.115:0/2782989915 wait complete. 2023-12-01T02:52:08.486 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T02:52:08.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.693+0000 7fa29bd77640 1 Processor -- start 2023-12-01T02:52:08.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.693+0000 7fa29bd77640 1 -- start start 2023-12-01T02:52:08.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.694+0000 7fa29bd77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa29406b6f0 0x7fa294070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.695+0000 7fa29bd77640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa29406f8b0 con 0x7fa294159dd0 2023-12-01T02:52:08.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.695+0000 7fa29bd77640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2940adcb0 con 0x7fa2940c9ef0 2023-12-01T02:52:08.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.695+0000 7fa29bd77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940ad9d0 con 0x7fa29406b6f0 2023-12-01T02:52:08.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.695+0000 7fa299b13640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa294159dd0 0x7fa2940c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60116/0 (socket says 172.21.15.115:60116) 2023-12-01T02:52:08.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.695+0000 7fa299b13640 1 -- 172.21.15.115:0/2631063051 learned_addr learned my addr 172.21.15.115:0/2631063051 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:08.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.695+0000 7fa29a314640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa29406b6f0 0x7fa294070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.695+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605053221 0 0) 0x7fa29406f8b0 con 0x7fa294159dd0 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.695+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa280003520 con 0x7fa294159dd0 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1013765895 0 0) 0x7fa280003520 con 0x7fa294159dd0 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa29406b6f0 msgr2=0x7fa294070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa29406b6f0 0x7fa294070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 >> v1:172.21.15.142:6789/0 conn(0x7fa2940c9ef0 legacy=0x7fa29406fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2940c9000 con 0x7fa294159dd0 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa290001670 con 0x7fa294159dd0 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 >> v1:172.21.15.115:6789/0 conn(0x7fa294159dd0 legacy=0x7fa2940c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa280003a10 0x7fa280003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa29406b6f0 0x7fa2800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa280004d50 0x7fa2800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940ad9d0 con 0x7fa280004d50 2023-12-01T02:52:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28000bc00 con 0x7fa280003a10 2023-12-01T02:52:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28000bdc0 con 0x7fa29406b6f0 2023-12-01T02:52:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.696+0000 7fa29a314640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa280004d50 0x7fa2800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.697+0000 7fa299312640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa29406b6f0 0x7fa2800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.697+0000 7fa299b13640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa280003a10 0x7fa280003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.697+0000 7fa29a314640 1 -- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa29406b6f0 msgr2=0x7fa2800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.697+0000 7fa29a314640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa29406b6f0 0x7fa2800047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.697+0000 7fa29a314640 1 -- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa280003a10 msgr2=0x7fa280003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.697+0000 7fa29a314640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa280003a10 0x7fa280003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.697+0000 7fa29a314640 1 -- 172.21.15.115:0/2631063051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa280003520 con 0x7fa280004d50 2023-12-01T02:52:08.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.698+0000 7fa29a314640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa280004d50 0x7fa2800051d0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009fc0 tx=0x7fa2800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0fc83a4409866 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:08.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.698+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c02cc20 con 0x7fa280004d50 2023-12-01T02:52:08.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.698+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c00f040 con 0x7fa280004d50 2023-12-01T02:52:08.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.698+0000 7fa298b11640 1 -- 172.21.15.115:0/2631063051 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c034440 con 0x7fa280004d50 2023-12-01T02:52:08.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.699+0000 7fa29bd77640 1 -- 172.21.15.115:0/2631063051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa280004d50 msgr2=0x7fa2800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.699+0000 7fa29bd77640 1 --2- 172.21.15.115:0/2631063051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa280004d50 0x7fa2800051d0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009fc0 tx=0x7fa2800057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.699+0000 7fa29a314640 1 -- 172.21.15.115:0/2631063051 reap_dead start 2023-12-01T02:52:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.699+0000 7fa29bd77640 1 -- 172.21.15.115:0/2631063051 shutdown_connections 2023-12-01T02:52:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.699+0000 7fa29bd77640 1 -- 172.21.15.115:0/2631063051 >> 172.21.15.115:0/2631063051 conn(0x7fa2940ce880 msgr2=0x7fa2940cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.699+0000 7fa29bd77640 1 -- 172.21.15.115:0/2631063051 shutdown_connections 2023-12-01T02:52:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.699+0000 7fa29bd77640 1 -- 172.21.15.115:0/2631063051 wait complete. 2023-12-01T02:52:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.700+0000 7fa29bd77640 1 Processor -- start 2023-12-01T02:52:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.701+0000 7fa29bd77640 1 -- start start 2023-12-01T02:52:08.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.701+0000 7fa29bd77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa280004d50 0x7fa294146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.702+0000 7fa29bd77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2940c5ae0 0x7fa2940ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.702+0000 7fa29bd77640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2940acde0 0x7fa2940ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.702+0000 7fa29bd77640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940ad260 con 0x7fa2940acde0 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.702+0000 7fa29bd77640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa294147130 con 0x7fa280004d50 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.702+0000 7fa29bd77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa294149410 con 0x7fa2940c5ae0 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.702+0000 7fa29a314640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2940acde0 0x7fa2940ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.702+0000 7fa29a314640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2940acde0 0x7fa2940ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43246/0 (socket says 172.21.15.115:43246) 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.702+0000 7fa29a314640 1 -- 172.21.15.115:0/69196883 learned_addr learned my addr 172.21.15.115:0/69196883 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.702+0000 7fa299b13640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa280004d50 0x7fa294146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa299312640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2940c5ae0 0x7fa2940ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa29a314640 1 -- 172.21.15.115:0/69196883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2940c5ae0 msgr2=0x7fa2940ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa29a314640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2940c5ae0 0x7fa2940ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa29a314640 1 -- 172.21.15.115:0/69196883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa280004d50 msgr2=0x7fa294146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa29a314640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa280004d50 0x7fa294146850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa29a314640 1 -- 172.21.15.115:0/69196883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2941496f0 con 0x7fa2940acde0 2023-12-01T02:52:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa29a314640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2940acde0 0x7fa2940ae040 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fa28c002230 tx=0x7fa28c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:08.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa28a7fc640 1 -- 172.21.15.115:0/69196883 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c043070 con 0x7fa2940acde0 2023-12-01T02:52:08.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa28a7fc640 1 -- 172.21.15.115:0/69196883 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c034ad0 con 0x7fa2940acde0 2023-12-01T02:52:08.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa28a7fc640 1 -- 172.21.15.115:0/69196883 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c034e80 con 0x7fa2940acde0 2023-12-01T02:52:08.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.703+0000 7fa29bd77640 1 -- 172.21.15.115:0/69196883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2941499d0 con 0x7fa2940acde0 2023-12-01T02:52:08.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.704+0000 7fa29bd77640 1 -- 172.21.15.115:0/69196883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa294146d90 con 0x7fa2940acde0 2023-12-01T02:52:08.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.705+0000 7fa29bd77640 1 -- 172.21.15.115:0/69196883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29404e890 con 0x7fa2940acde0 2023-12-01T02:52:08.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.706+0000 7fa28a7fc640 1 -- 172.21.15.115:0/69196883 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 94) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fa28c03e440 con 0x7fa2940acde0 2023-12-01T02:52:08.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.706+0000 7fa28a7fc640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa27806a4c0 0x7fa27806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:08.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.707+0000 7fa28a7fc640 1 -- 172.21.15.115:0/69196883 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fa28c0aa780 con 0x7fa2940acde0 2023-12-01T02:52:08.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.707+0000 7fa299b13640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa27806a4c0 0x7fa27806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:08.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.707+0000 7fa299b13640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa27806a4c0 0x7fa27806c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa290001eb0 tx=0x7fa290009040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:08.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.711+0000 7fa28a7fc640 1 -- 172.21.15.115:0/69196883 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa28c075b80 con 0x7fa2940acde0 2023-12-01T02:52:08.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.939+0000 7fa29bd77640 1 -- 172.21.15.115:0/69196883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa294071b00 con 0x7fa2940acde0 2023-12-01T02:52:08.942 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T02:52:08.941+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T02:52:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.957+0000 7fa28a7fc640 1 -- 172.21.15.115:0/69196883 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa28c0791a0 con 0x7fa2940acde0 2023-12-01T02:52:08.958 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T02:52:08.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.959+0000 7fa29bd77640 1 -- 172.21.15.115:0/69196883 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa27806a4c0 msgr2=0x7fa27806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.959+0000 7fa29bd77640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa27806a4c0 0x7fa27806c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa290001eb0 tx=0x7fa290009040 comp rx=0 tx=0).stop 2023-12-01T02:52:08.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.959+0000 7fa29bd77640 1 -- 172.21.15.115:0/69196883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2940acde0 msgr2=0x7fa2940ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:08.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.959+0000 7fa29bd77640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2940acde0 0x7fa2940ae040 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fa28c002230 tx=0x7fa28c009750 comp rx=0 tx=0).stop 2023-12-01T02:52:08.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.960+0000 7fa29bd77640 1 -- 172.21.15.115:0/69196883 shutdown_connections 2023-12-01T02:52:08.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.960+0000 7fa29bd77640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa27806a4c0 0x7fa27806c980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.960+0000 7fa29bd77640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2940c5ae0 0x7fa2940ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.960+0000 7fa29bd77640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2940acde0 0x7fa2940ae040 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.960+0000 7fa29bd77640 1 --2- 172.21.15.115:0/69196883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa280004d50 0x7fa294146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:08.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.960+0000 7fa29bd77640 1 -- 172.21.15.115:0/69196883 >> 172.21.15.115:0/69196883 conn(0x7fa2940ce880 msgr2=0x7fa294070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:08.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.960+0000 7fa29bd77640 1 -- 172.21.15.115:0/69196883 shutdown_connections 2023-12-01T02:52:08.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:08.960+0000 7fa29bd77640 1 -- 172.21.15.115:0/69196883 wait complete. 2023-12-01T02:52:08.971 INFO:tasks.ceph.mds.a.smithi115.stderr: -3758> 2023-12-01T02:51:48.288+0000 7fc87f2d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T02:52:08.980 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T02:52:09.035 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T02:52:09.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.182+0000 7f2da5da9640 1 Processor -- start 2023-12-01T02:52:09.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.182+0000 7f2da5da9640 1 -- start start 2023-12-01T02:52:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.183+0000 7f2da5da9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2da015ac00 0x7f2da00ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.183+0000 7f2da5da9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2da00aede0 0x7f2da00cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.183+0000 7f2da5da9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 0x7f2da00cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.183+0000 7f2da5da9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da00b12b0 con 0x7f2da015ac00 2023-12-01T02:52:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.183+0000 7f2da5da9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da015afe0 con 0x7f2da00aede0 2023-12-01T02:52:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.183+0000 7f2da5da9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00b0a10 con 0x7f2da00b04c0 2023-12-01T02:52:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.183+0000 7f2da4da7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2da015ac00 0x7f2da00ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2da4da7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2da015ac00 0x7f2da00ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43278/0 (socket says 172.21.15.115:43278) 2023-12-01T02:52:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2da4da7640 1 -- 172.21.15.115:0/3778049596 learned_addr learned my addr 172.21.15.115:0/3778049596 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2d9ffff640 1 --2- 172.21.15.115:0/3778049596 >> v2:172.21.15.142:3300/0 conn(0x7f2da00aede0 0x7f2da00cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2da55a8640 1 --2- 172.21.15.115:0/3778049596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 0x7f2da00cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2da4da7640 1 -- 172.21.15.115:0/3778049596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 msgr2=0x7f2da00cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2da4da7640 1 --2- 172.21.15.115:0/3778049596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 0x7f2da00cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2da4da7640 1 -- 172.21.15.115:0/3778049596 >> v2:172.21.15.142:3300/0 conn(0x7f2da00aede0 msgr2=0x7f2da00cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2da4da7640 1 --2- 172.21.15.115:0/3778049596 >> v2:172.21.15.142:3300/0 conn(0x7f2da00aede0 0x7f2da00cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2da4da7640 1 -- 172.21.15.115:0/3778049596 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da00b0cf0 con 0x7f2da015ac00 2023-12-01T02:52:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2da55a8640 1 --2- 172.21.15.115:0/3778049596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 0x7f2da00cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.184+0000 7f2d9ffff640 1 --2- 172.21.15.115:0/3778049596 >> v2:172.21.15.142:3300/0 conn(0x7f2da00aede0 0x7f2da00cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.185+0000 7f2da4da7640 1 --2- 172.21.15.115:0/3778049596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015ac00 0x7f2da00ae8a0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f2da00aff50 tx=0x7f2d9402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5066db92e0190a90 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.185+0000 7f2d9f7fe640 1 -- 172.21.15.115:0/3778049596 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d940317e0 con 0x7f2da015ac00 2023-12-01T02:52:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.185+0000 7f2d9f7fe640 1 -- 172.21.15.115:0/3778049596 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94031940 con 0x7f2da015ac00 2023-12-01T02:52:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.185+0000 7f2d9f7fe640 1 -- 172.21.15.115:0/3778049596 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94031cd0 con 0x7f2da015ac00 2023-12-01T02:52:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.186+0000 7f2da5da9640 1 -- 172.21.15.115:0/3778049596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015ac00 msgr2=0x7f2da00ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.186+0000 7f2da5da9640 1 --2- 172.21.15.115:0/3778049596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015ac00 0x7f2da00ae8a0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f2da00aff50 tx=0x7f2d9402b880 comp rx=0 tx=0).stop 2023-12-01T02:52:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.186+0000 7f2da5da9640 1 -- 172.21.15.115:0/3778049596 shutdown_connections 2023-12-01T02:52:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.186+0000 7f2da5da9640 1 --2- 172.21.15.115:0/3778049596 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 0x7f2da00cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.186+0000 7f2da5da9640 1 --2- 172.21.15.115:0/3778049596 >> v2:172.21.15.142:3300/0 conn(0x7f2da00aede0 0x7f2da00cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.186+0000 7f2da5da9640 1 --2- 172.21.15.115:0/3778049596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da015ac00 0x7f2da00ae8a0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.186+0000 7f2da5da9640 1 -- 172.21.15.115:0/3778049596 >> 172.21.15.115:0/3778049596 conn(0x7f2da0065960 msgr2=0x7f2da0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:09.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.186+0000 7f2da5da9640 1 -- 172.21.15.115:0/3778049596 shutdown_connections 2023-12-01T02:52:09.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.186+0000 7f2da5da9640 1 -- 172.21.15.115:0/3778049596 wait complete. 2023-12-01T02:52:09.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.188+0000 7f2da5da9640 1 Processor -- start 2023-12-01T02:52:09.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.188+0000 7f2da5da9640 1 -- start start 2023-12-01T02:52:09.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da5da9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da00aede0 0x7f2da014f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:09.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da5da9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 0x7f2da015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:09.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da5da9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015ac00 0x7f2da00c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:09.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da5da9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da014d720 con 0x7f2da00aede0 2023-12-01T02:52:09.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da5da9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da015c960 con 0x7f2da015ac00 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da5da9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da015bd50 con 0x7f2da00b04c0 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da4da7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da00aede0 0x7f2da014f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da4da7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da00aede0 0x7f2da014f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43288/0 (socket says 172.21.15.115:43288) 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da4da7640 1 -- 172.21.15.115:0/2072660735 learned_addr learned my addr 172.21.15.115:0/2072660735 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2d9ffff640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 0x7f2da015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da4da7640 1 -- 172.21.15.115:0/2072660735 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 msgr2=0x7f2da015d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da55a8640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015ac00 0x7f2da00c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.189+0000 7f2da4da7640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 0x7f2da015d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.190+0000 7f2da4da7640 1 -- 172.21.15.115:0/2072660735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015ac00 msgr2=0x7f2da00c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.190+0000 7f2da4da7640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015ac00 0x7f2da00c5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.190+0000 7f2da4da7640 1 -- 172.21.15.115:0/2072660735 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d940094d0 con 0x7f2da00aede0 2023-12-01T02:52:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.190+0000 7f2da4da7640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da00aede0 0x7f2da014f800 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:09.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.190+0000 7f2d9d7fa640 1 -- 172.21.15.115:0/2072660735 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94032900 con 0x7f2da00aede0 2023-12-01T02:52:09.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.190+0000 7f2d9d7fa640 1 -- 172.21.15.115:0/2072660735 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94032a60 con 0x7f2da00aede0 2023-12-01T02:52:09.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.190+0000 7f2d9d7fa640 1 -- 172.21.15.115:0/2072660735 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94032e10 con 0x7f2da00aede0 2023-12-01T02:52:09.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.190+0000 7f2da5da9640 1 -- 172.21.15.115:0/2072660735 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da015c030 con 0x7f2da00aede0 2023-12-01T02:52:09.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.190+0000 7f2da5da9640 1 -- 172.21.15.115:0/2072660735 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da015c5f0 con 0x7f2da00aede0 2023-12-01T02:52:09.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.191+0000 7f2da5da9640 1 -- 172.21.15.115:0/2072660735 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da00618d0 con 0x7f2da00aede0 2023-12-01T02:52:09.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.192+0000 7f2d9d7fa640 1 -- 172.21.15.115:0/2072660735 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 94) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f2d9404e450 con 0x7f2da00aede0 2023-12-01T02:52:09.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.194+0000 7f2d9d7fa640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a4c0 0x7f2d8406c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:09.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.194+0000 7f2d9d7fa640 1 -- 172.21.15.115:0/2072660735 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f2d940ab050 con 0x7f2da00aede0 2023-12-01T02:52:09.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.194+0000 7f2d9ffff640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a4c0 0x7f2d8406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:09.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.195+0000 7f2d9ffff640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a4c0 0x7f2d8406c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c006fd0 tx=0x7f2d8c008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:09.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.198+0000 7f2d9d7fa640 1 -- 172.21.15.115:0/2072660735 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2d94076310 con 0x7f2da00aede0 2023-12-01T02:52:09.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.425+0000 7f2da5da9640 1 -- 172.21.15.115:0/2072660735 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2da00afae0 con 0x7f2da00aede0 2023-12-01T02:52:09.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.979+0000 7f2d9d7fa640 1 -- 172.21.15.115:0/2072660735 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f2d940766e0 con 0x7f2da00aede0 2023-12-01T02:52:09.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.982+0000 7f2d9d7fa640 1 -- 172.21.15.115:0/2072660735 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2d9407dae0 con 0x7f2da00aede0 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 -- 172.21.15.115:0/2072660735 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a4c0 msgr2=0x7f2d8406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a4c0 0x7f2d8406c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c006fd0 tx=0x7f2d8c008040 comp rx=0 tx=0).stop 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 -- 172.21.15.115:0/2072660735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da00aede0 msgr2=0x7f2da014f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da00aede0 0x7f2da014f800 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d940046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 -- 172.21.15.115:0/2072660735 shutdown_connections 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2d8406a4c0 0x7f2d8406c980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2da00b04c0 0x7f2da015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2da015ac00 0x7f2da00c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 --2- 172.21.15.115:0/2072660735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2da00aede0 0x7f2da014f800 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 -- 172.21.15.115:0/2072660735 >> 172.21.15.115:0/2072660735 conn(0x7f2da0065960 msgr2=0x7f2da00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 -- 172.21.15.115:0/2072660735 shutdown_connections 2023-12-01T02:52:09.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:09.986+0000 7f2da5da9640 1 -- 172.21.15.115:0/2072660735 wait complete. 2023-12-01T02:52:10.009 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T02:52:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.220+0000 7f8ea8cab640 1 Processor -- start 2023-12-01T02:52:10.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.220+0000 7f8ea8cab640 1 -- start start 2023-12-01T02:52:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.222+0000 7f8ea8cab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8ea40b4d50 0x7f8ea40b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:10.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.222+0000 7f8ea8cab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea40d1690 con 0x7f8ea40d2730 2023-12-01T02:52:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.222+0000 7f8ea8cab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea40d1970 con 0x7f8ea415ce40 2023-12-01T02:52:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.222+0000 7f8ea8cab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea415d220 con 0x7f8ea40b4d50 2023-12-01T02:52:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.222+0000 7f8ea1d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8ea40b4d50 0x7f8ea40b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.222+0000 7f8ea2d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8ea40d2730 0x7f8ea40b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60126/0 (socket says 172.21.15.115:60126) 2023-12-01T02:52:10.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.222+0000 7f8ea2d9d640 1 -- 172.21.15.115:0/2901298252 learned_addr learned my addr 172.21.15.115:0/2901298252 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:10.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.223+0000 7f8ea1d9b640 1 -- 172.21.15.115:0/2901298252 >> v1:172.21.15.142:6789/0 conn(0x7f8ea415ce40 legacy=0x7f8ea40b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:10.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.223+0000 7f8ea1d9b640 1 -- 172.21.15.115:0/2901298252 >> v1:172.21.15.115:6789/0 conn(0x7f8ea40d2730 legacy=0x7f8ea40b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:10.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.223+0000 7f8ea1d9b640 1 -- 172.21.15.115:0/2901298252 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea40d1c50 con 0x7f8ea40b4d50 2023-12-01T02:52:10.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.223+0000 7f8ea159a640 1 -- 172.21.15.115:0/2901298252 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631299921 0 0) 0x7f8ea40d1690 con 0x7f8ea40d2730 2023-12-01T02:52:10.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.224+0000 7f8ea1d9b640 1 --2- 172.21.15.115:0/2901298252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 0x7f8ea40b8710 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8e980097a0 tx=0x7f8e9802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f40c047cb1e272d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.224+0000 7f8ea159a640 1 -- 172.21.15.115:0/2901298252 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9802e070 con 0x7f8ea40b4d50 2023-12-01T02:52:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.224+0000 7f8ea159a640 1 -- 172.21.15.115:0/2901298252 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e980315a0 con 0x7f8ea40b4d50 2023-12-01T02:52:10.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.224+0000 7f8ea159a640 1 -- 172.21.15.115:0/2901298252 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98031930 con 0x7f8ea40b4d50 2023-12-01T02:52:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.225+0000 7f8ea8cab640 1 -- 172.21.15.115:0/2901298252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 msgr2=0x7f8ea40b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.225+0000 7f8ea8cab640 1 --2- 172.21.15.115:0/2901298252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 0x7f8ea40b8710 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8e980097a0 tx=0x7f8e9802b840 comp rx=0 tx=0).stop 2023-12-01T02:52:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.225+0000 7f8ea8cab640 1 -- 172.21.15.115:0/2901298252 shutdown_connections 2023-12-01T02:52:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.225+0000 7f8ea8cab640 1 --2- 172.21.15.115:0/2901298252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 0x7f8ea40b8710 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.225+0000 7f8ea8cab640 1 -- 172.21.15.115:0/2901298252 >> 172.21.15.115:0/2901298252 conn(0x7f8ea4065f90 msgr2=0x7f8ea40cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.225+0000 7f8ea8cab640 1 -- 172.21.15.115:0/2901298252 shutdown_connections 2023-12-01T02:52:10.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.225+0000 7f8ea8cab640 1 -- 172.21.15.115:0/2901298252 wait complete. 2023-12-01T02:52:10.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.227+0000 7f8ea8cab640 1 Processor -- start 2023-12-01T02:52:10.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.227+0000 7f8ea8cab640 1 -- start start 2023-12-01T02:52:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea8cab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 0x7f8ea415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea8cab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ea40d2730 0x7f8ea415ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea8cab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ea415ce40 0x7f8ea4157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea8cab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4154cf0 con 0x7f8ea40b4d50 2023-12-01T02:52:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea8cab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea415ee90 con 0x7f8ea40d2730 2023-12-01T02:52:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea8cab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea415f170 con 0x7f8ea415ce40 2023-12-01T02:52:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 0x7f8ea415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:10.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 0x7f8ea415e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43304/0 (socket says 172.21.15.115:43304) 2023-12-01T02:52:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea259c640 1 -- 172.21.15.115:0/996272247 learned_addr learned my addr 172.21.15.115:0/996272247 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea259c640 1 -- 172.21.15.115:0/996272247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ea415ce40 msgr2=0x7f8ea4157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea259c640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ea415ce40 0x7f8ea4157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea259c640 1 -- 172.21.15.115:0/996272247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ea40d2730 msgr2=0x7f8ea415ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea259c640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ea40d2730 0x7f8ea415ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.229+0000 7f8ea259c640 1 -- 172.21.15.115:0/996272247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e98009400 con 0x7f8ea40b4d50 2023-12-01T02:52:10.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.230+0000 7f8ea259c640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 0x7f8ea415e8b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400b3f0 tx=0x7f8e9400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.230+0000 7f8e92ffd640 1 -- 172.21.15.115:0/996272247 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94011830 con 0x7f8ea40b4d50 2023-12-01T02:52:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.230+0000 7f8e92ffd640 1 -- 172.21.15.115:0/996272247 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e94011990 con 0x7f8ea40b4d50 2023-12-01T02:52:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.230+0000 7f8e92ffd640 1 -- 172.21.15.115:0/996272247 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94011d40 con 0x7f8ea40b4d50 2023-12-01T02:52:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.230+0000 7f8ea8cab640 1 -- 172.21.15.115:0/996272247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea415f450 con 0x7f8ea40b4d50 2023-12-01T02:52:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.230+0000 7f8ea8cab640 1 -- 172.21.15.115:0/996272247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea415fa10 con 0x7f8ea40b4d50 2023-12-01T02:52:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.231+0000 7f8ea8cab640 1 -- 172.21.15.115:0/996272247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea40c6060 con 0x7f8ea40b4d50 2023-12-01T02:52:10.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.232+0000 7f8e92ffd640 1 -- 172.21.15.115:0/996272247 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8e9401a400 con 0x7f8ea40b4d50 2023-12-01T02:52:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.233+0000 7f8e92ffd640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e8406a550 0x7f8e8406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.233+0000 7f8e92ffd640 1 -- 172.21.15.115:0/996272247 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8e94088c40 con 0x7f8ea40b4d50 2023-12-01T02:52:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.234+0000 7f8ea1d9b640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e8406a550 0x7f8e8406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.234+0000 7f8ea1d9b640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e8406a550 0x7f8e8406ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8e98036000 tx=0x7f8e98030000 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.238+0000 7f8e92ffd640 1 -- 172.21.15.115:0/996272247 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8e94053e60 con 0x7f8ea40b4d50 2023-12-01T02:52:10.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.427+0000 7f8ea8cab640 1 -- 172.21.15.115:0/996272247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ea415fc00 con 0x7f8ea40b4d50 2023-12-01T02:52:10.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.978+0000 7f8e92ffd640 1 -- 172.21.15.115:0/996272247 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v167) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8e94015020 con 0x7f8ea40b4d50 2023-12-01T02:52:10.980 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T02:52:10.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.981+0000 7f8ea8cab640 1 -- 172.21.15.115:0/996272247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e8406a550 msgr2=0x7f8e8406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:10.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.981+0000 7f8ea8cab640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e8406a550 0x7f8e8406ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8e98036000 tx=0x7f8e98030000 comp rx=0 tx=0).stop 2023-12-01T02:52:10.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.981+0000 7f8ea8cab640 1 -- 172.21.15.115:0/996272247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 msgr2=0x7f8ea415e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:10.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.981+0000 7f8ea8cab640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 0x7f8ea415e8b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400b3f0 tx=0x7f8e9400b8b0 comp rx=0 tx=0).stop 2023-12-01T02:52:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.981+0000 7f8ea8cab640 1 -- 172.21.15.115:0/996272247 shutdown_connections 2023-12-01T02:52:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.981+0000 7f8ea8cab640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f8e8406a550 0x7f8e8406ca10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.981+0000 7f8ea8cab640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ea415ce40 0x7f8ea4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.981+0000 7f8ea8cab640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ea40d2730 0x7f8ea415ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.981+0000 7f8ea8cab640 1 --2- 172.21.15.115:0/996272247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ea40b4d50 0x7f8ea415e8b0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.981+0000 7f8ea8cab640 1 -- 172.21.15.115:0/996272247 >> 172.21.15.115:0/996272247 conn(0x7f8ea4065f90 msgr2=0x7f8ea40cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.982+0000 7f8ea8cab640 1 -- 172.21.15.115:0/996272247 shutdown_connections 2023-12-01T02:52:10.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:10.982+0000 7f8ea8cab640 1 -- 172.21.15.115:0/996272247 wait complete. 2023-12-01T02:52:11.004 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T02:52:11.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.224+0000 7f8705496640 1 Processor -- start 2023-12-01T02:52:11.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.224+0000 7f8705496640 1 -- start start 2023-12-01T02:52:11.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.226+0000 7f8705496640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000ca7c0 0x7f87000cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:11.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.226+0000 7f8705496640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87000c9440 con 0x7f87000cb6f0 2023-12-01T02:52:11.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.226+0000 7f8705496640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87000c9720 con 0x7f8700159bf0 2023-12-01T02:52:11.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.226+0000 7f8705496640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8700159fd0 con 0x7f87000ca7c0 2023-12-01T02:52:11.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.226+0000 7f86ff7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f87000cb6f0 0x7f870006f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60132/0 (socket says 172.21.15.115:60132) 2023-12-01T02:52:11.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.226+0000 7f86ff7fe640 1 -- 172.21.15.115:0/4183734009 learned_addr learned my addr 172.21.15.115:0/4183734009 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:11.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716245915 0 0) 0x7f87000c9440 con 0x7f87000cb6f0 2023-12-01T02:52:11.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86f67fc640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000ca7c0 0x7f87000cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:11.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86ec003520 con 0x7f87000cb6f0 2023-12-01T02:52:11.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3550562537 0 0) 0x7f86ec003520 con 0x7f87000cb6f0 2023-12-01T02:52:11.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000ca7c0 msgr2=0x7f87000cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:11.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000ca7c0 0x7f87000cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 >> v1:172.21.15.142:6789/0 conn(0x7f8700159bf0 legacy=0x7f870015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87000c9a00 con 0x7f87000cb6f0 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f86f80019f0 con 0x7f87000cb6f0 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 >> v1:172.21.15.115:6789/0 conn(0x7f87000cb6f0 legacy=0x7f870006f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86ec0039a0 0x7f86ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 0x7f86ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000ca7c0 0x7f86ec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec00dd20 con 0x7f86ec0047c0 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec00de60 con 0x7f86ec0039a0 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec00e020 con 0x7f87000ca7c0 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.227+0000 7f86f67fc640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 0x7f86ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.228+0000 7f86ff7fe640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000ca7c0 0x7f86ec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.228+0000 7f86feffd640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86ec0039a0 0x7f86ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:11.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.228+0000 7f86f67fc640 1 -- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000ca7c0 msgr2=0x7f86ec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:11.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.228+0000 7f86f67fc640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000ca7c0 0x7f86ec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:11.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.228+0000 7f86f67fc640 1 -- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86ec0039a0 msgr2=0x7f86ec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:11.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.228+0000 7f86f67fc640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86ec0039a0 0x7f86ec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:11.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.228+0000 7f86f67fc640 1 -- 172.21.15.115:0/4183734009 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec003520 con 0x7f86ec0047c0 2023-12-01T02:52:11.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.228+0000 7f86feffd640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86ec0039a0 0x7f86ec003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:11.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.228+0000 7f86ff7fe640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000ca7c0 0x7f86ec005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:11.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.229+0000 7f86f67fc640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 0x7f86ec004c40 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f86e800ba30 tx=0x7f86ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278bef5a09eee9e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:11.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.229+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8032630 con 0x7f86ec0047c0 2023-12-01T02:52:11.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.229+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e8032790 con 0x7f86ec0047c0 2023-12-01T02:52:11.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.229+0000 7f86fe7fc640 1 -- 172.21.15.115:0/4183734009 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8032b20 con 0x7f86ec0047c0 2023-12-01T02:52:11.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.230+0000 7f8705496640 1 -- 172.21.15.115:0/4183734009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 msgr2=0x7f86ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:11.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.230+0000 7f8705496640 1 --2- 172.21.15.115:0/4183734009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 0x7f86ec004c40 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f86e800ba30 tx=0x7f86ec0057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:11.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.230+0000 7f86ff7fe640 1 -- 172.21.15.115:0/4183734009 reap_dead start 2023-12-01T02:52:11.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.230+0000 7f8705496640 1 -- 172.21.15.115:0/4183734009 shutdown_connections 2023-12-01T02:52:11.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.230+0000 7f8705496640 1 -- 172.21.15.115:0/4183734009 >> 172.21.15.115:0/4183734009 conn(0x7f87000662c0 msgr2=0x7f87000b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:11.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.230+0000 7f8705496640 1 -- 172.21.15.115:0/4183734009 shutdown_connections 2023-12-01T02:52:11.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.230+0000 7f8705496640 1 -- 172.21.15.115:0/4183734009 wait complete. 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.232+0000 7f8705496640 1 Processor -- start 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.232+0000 7f8705496640 1 -- start start 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.233+0000 7f8705496640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 0x7f8700144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.233+0000 7f8705496640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000b1460 0x7f87000b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.233+0000 7f8705496640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87000b36c0 0x7f87000b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.233+0000 7f8705496640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000b2200 con 0x7f86ec0047c0 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.233+0000 7f8705496640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87001450e0 con 0x7f87000b36c0 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.233+0000 7f8705496640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87000b1f20 con 0x7f87000b1460 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.233+0000 7f86feffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 0x7f8700144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.233+0000 7f86feffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 0x7f8700144ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43316/0 (socket says 172.21.15.115:43316) 2023-12-01T02:52:11.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.233+0000 7f86feffd640 1 -- 172.21.15.115:0/1105273784 learned_addr learned my addr 172.21.15.115:0/1105273784 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:11.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f86feffd640 1 -- 172.21.15.115:0/1105273784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000b1460 msgr2=0x7f87000b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f86feffd640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000b1460 0x7f87000b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f86feffd640 1 -- 172.21.15.115:0/1105273784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87000b36c0 msgr2=0x7f87000b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f86feffd640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87000b36c0 0x7f87000b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f86feffd640 1 -- 172.21.15.115:0/1105273784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87000c2d70 con 0x7f86ec0047c0 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f86feffd640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 0x7f8700144ba0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f86f000d770 tx=0x7f86f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f86f7fff640 1 -- 172.21.15.115:0/1105273784 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0005270 con 0x7f86ec0047c0 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f86f7fff640 1 -- 172.21.15.115:0/1105273784 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f00053d0 con 0x7f86ec0047c0 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f8705496640 1 -- 172.21.15.115:0/1105273784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87000c3050 con 0x7f86ec0047c0 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f86f7fff640 1 -- 172.21.15.115:0/1105273784 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f000b8f0 con 0x7f86ec0047c0 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.234+0000 7f8705496640 1 -- 172.21.15.115:0/1105273784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87000b4080 con 0x7f86ec0047c0 2023-12-01T02:52:11.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.235+0000 7f8705496640 1 -- 172.21.15.115:0/1105273784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87000c2bd0 con 0x7f86ec0047c0 2023-12-01T02:52:11.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.236+0000 7f86f7fff640 1 -- 172.21.15.115:0/1105273784 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f86f000ba50 con 0x7f86ec0047c0 2023-12-01T02:52:11.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.237+0000 7f86f7fff640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a580 0x7f86e006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:11.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.237+0000 7f86f7fff640 1 -- 172.21.15.115:0/1105273784 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6003+0+0 (secure 0 0 0) 0x7f86f0088bb0 con 0x7f86ec0047c0 2023-12-01T02:52:11.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.238+0000 7f86f67fc640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a580 0x7f86e006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:11.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.238+0000 7f86f67fc640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a580 0x7f86e006ca40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f86e8037000 tx=0x7f86e8032000 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:11.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.241+0000 7f86f7fff640 1 -- 172.21.15.115:0/1105273784 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f87000c2bd0 con 0x7f86ec0047c0 2023-12-01T02:52:11.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.431+0000 7f8705496640 1 -- 172.21.15.115:0/1105273784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87000c2bd0 con 0x7f86ec0047c0 2023-12-01T02:52:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.990+0000 7f86f7fff640 1 -- 172.21.15.115:0/1105273784 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v168) v1 ==== 169+0+0 (secure 0 0 0) 0x7f87000c2bd0 con 0x7f86ec0047c0 2023-12-01T02:52:11.992 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T02:52:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 -- 172.21.15.115:0/1105273784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a580 msgr2=0x7f86e006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a580 0x7f86e006ca40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f86e8037000 tx=0x7f86e8032000 comp rx=0 tx=0).stop 2023-12-01T02:52:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 -- 172.21.15.115:0/1105273784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 msgr2=0x7f8700144ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 0x7f8700144ba0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f86f000d770 tx=0x7f86f000dc30 comp rx=0 tx=0).stop 2023-12-01T02:52:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 -- 172.21.15.115:0/1105273784 shutdown_connections 2023-12-01T02:52:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f86e006a580 0x7f86e006ca40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87000b1460 0x7f87000b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87000b36c0 0x7f87000b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 --2- 172.21.15.115:0/1105273784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86ec0047c0 0x7f8700144ba0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:11.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 -- 172.21.15.115:0/1105273784 >> 172.21.15.115:0/1105273784 conn(0x7f87000662c0 msgr2=0x7f87000cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 -- 172.21.15.115:0/1105273784 shutdown_connections 2023-12-01T02:52:11.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:11.994+0000 7f8705496640 1 -- 172.21.15.115:0/1105273784 wait complete. 2023-12-01T02:52:12.019 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T02:52:12.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.235+0000 7f5f956c4640 1 Processor -- start 2023-12-01T02:52:12.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.236+0000 7f5f956c4640 1 -- start start 2023-12-01T02:52:12.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.237+0000 7f5f956c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5f900b1710 0x7f5f9006baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.237+0000 7f5f956c4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5f900b1ba0 0x7f5f9006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.237+0000 7f5f956c4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f900cac30 con 0x7f5f901590b0 2023-12-01T02:52:12.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.237+0000 7f5f956c4640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900b2140 con 0x7f5f900b1710 2023-12-01T02:52:12.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.237+0000 7f5f956c4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900b2420 con 0x7f5f900b1ba0 2023-12-01T02:52:12.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.238+0000 7f5f867fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5f900b1710 0x7f5f9006baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.238+0000 7f5f867fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5f900b1710 0x7f5f9006baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43328/0 (socket says 172.21.15.115:43328) 2023-12-01T02:52:12.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.238+0000 7f5f867fc640 1 -- 172.21.15.115:0/411856434 learned_addr learned my addr 172.21.15.115:0/411856434 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:12.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.238+0000 7f5f8f7fe640 1 --2- 172.21.15.115:0/411856434 >> v2:172.21.15.142:3300/0 conn(0x7f5f900b1ba0 0x7f5f9006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.238+0000 7f5f867fc640 1 -- 172.21.15.115:0/411856434 >> v2:172.21.15.142:3300/0 conn(0x7f5f900b1ba0 msgr2=0x7f5f9006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:12.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.238+0000 7f5f867fc640 1 --2- 172.21.15.115:0/411856434 >> v2:172.21.15.142:3300/0 conn(0x7f5f900b1ba0 0x7f5f9006c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.238+0000 7f5f867fc640 1 -- 172.21.15.115:0/411856434 >> v1:172.21.15.142:6789/0 conn(0x7f5f901590b0 legacy=0x7f5f90159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:12.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.238+0000 7f5f867fc640 1 -- 172.21.15.115:0/411856434 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900caed0 con 0x7f5f900b1710 2023-12-01T02:52:12.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.239+0000 7f5f867fc640 1 --2- 172.21.15.115:0/411856434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1710 0x7f5f9006baf0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f5f900ce740 tx=0x7f5f7802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f43e8cda9c3cbce server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:12.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.240+0000 7f5f8e7fc640 1 -- 172.21.15.115:0/411856434 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78031870 con 0x7f5f900b1710 2023-12-01T02:52:12.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.240+0000 7f5f8e7fc640 1 -- 172.21.15.115:0/411856434 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f780319d0 con 0x7f5f900b1710 2023-12-01T02:52:12.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.240+0000 7f5f8e7fc640 1 -- 172.21.15.115:0/411856434 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78031d60 con 0x7f5f900b1710 2023-12-01T02:52:12.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.240+0000 7f5f956c4640 1 -- 172.21.15.115:0/411856434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1710 msgr2=0x7f5f9006baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.240+0000 7f5f956c4640 1 --2- 172.21.15.115:0/411856434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1710 0x7f5f9006baf0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f5f900ce740 tx=0x7f5f7802b910 comp rx=0 tx=0).stop 2023-12-01T02:52:12.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.241+0000 7f5f956c4640 1 -- 172.21.15.115:0/411856434 shutdown_connections 2023-12-01T02:52:12.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.241+0000 7f5f956c4640 1 --2- 172.21.15.115:0/411856434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1710 0x7f5f9006baf0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.241+0000 7f5f956c4640 1 --2- 172.21.15.115:0/411856434 >> v2:172.21.15.142:3300/0 conn(0x7f5f900b1ba0 0x7f5f9006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.241+0000 7f5f956c4640 1 -- 172.21.15.115:0/411856434 >> 172.21.15.115:0/411856434 conn(0x7f5f90065960 msgr2=0x7f5f90065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:12.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.241+0000 7f5f956c4640 1 -- 172.21.15.115:0/411856434 shutdown_connections 2023-12-01T02:52:12.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.241+0000 7f5f956c4640 1 -- 172.21.15.115:0/411856434 wait complete. 2023-12-01T02:52:12.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.242+0000 7f5f956c4640 1 Processor -- start 2023-12-01T02:52:12.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.243+0000 7f5f956c4640 1 -- start start 2023-12-01T02:52:12.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f956c4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f900b1710 0x7f5f9015b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f956c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1ba0 0x7f5f900cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f956c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f901590b0 0x7f5f900cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f956c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90145ab0 con 0x7f5f900b1ba0 2023-12-01T02:52:12.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f956c4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9013f700 con 0x7f5f900b1710 2023-12-01T02:52:12.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f956c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f901537a0 con 0x7f5f901590b0 2023-12-01T02:52:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f867fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1ba0 0x7f5f900cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f867fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1ba0 0x7f5f900cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43340/0 (socket says 172.21.15.115:43340) 2023-12-01T02:52:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f867fc640 1 -- 172.21.15.115:0/483044311 learned_addr learned my addr 172.21.15.115:0/483044311 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f8effd640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f900b1710 0x7f5f9015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f8f7fe640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f901590b0 0x7f5f900cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f867fc640 1 -- 172.21.15.115:0/483044311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f901590b0 msgr2=0x7f5f900cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f867fc640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f901590b0 0x7f5f900cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f867fc640 1 -- 172.21.15.115:0/483044311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f900b1710 msgr2=0x7f5f9015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f867fc640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f900b1710 0x7f5f9015b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.244+0000 7f5f867fc640 1 -- 172.21.15.115:0/483044311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f780094d0 con 0x7f5f900b1ba0 2023-12-01T02:52:12.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.245+0000 7f5f867fc640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1ba0 0x7f5f900cebc0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f5f78000c00 tx=0x7f5f780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:12.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.245+0000 7f5f87fff640 1 -- 172.21.15.115:0/483044311 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78031840 con 0x7f5f900b1ba0 2023-12-01T02:52:12.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.245+0000 7f5f87fff640 1 -- 172.21.15.115:0/483044311 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f780443f0 con 0x7f5f900b1ba0 2023-12-01T02:52:12.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.245+0000 7f5f87fff640 1 -- 172.21.15.115:0/483044311 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780447a0 con 0x7f5f900b1ba0 2023-12-01T02:52:12.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.245+0000 7f5f956c4640 1 -- 172.21.15.115:0/483044311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9013f9a0 con 0x7f5f900b1ba0 2023-12-01T02:52:12.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.245+0000 7f5f956c4640 1 -- 172.21.15.115:0/483044311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f90153a80 con 0x7f5f900b1ba0 2023-12-01T02:52:12.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.246+0000 7f5f956c4640 1 -- 172.21.15.115:0/483044311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9013fec0 con 0x7f5f900b1ba0 2023-12-01T02:52:12.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.247+0000 7f5f87fff640 1 -- 172.21.15.115:0/483044311 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f5f78044900 con 0x7f5f900b1ba0 2023-12-01T02:52:12.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.248+0000 7f5f87fff640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f7406a550 0x7f5f7406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.248+0000 7f5f87fff640 1 -- 172.21.15.115:0/483044311 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f5f7802d040 con 0x7f5f900b1ba0 2023-12-01T02:52:12.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.248+0000 7f5f8effd640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f7406a550 0x7f5f7406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.249+0000 7f5f8effd640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f7406a550 0x7f5f7406ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5f80006fd0 tx=0x7f5f80008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:12.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.253+0000 7f5f87fff640 1 -- 172.21.15.115:0/483044311 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5f9013fec0 con 0x7f5f900b1ba0 2023-12-01T02:52:12.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.442+0000 7f5f956c4640 1 -- 172.21.15.115:0/483044311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5f9013fec0 con 0x7f5f900b1ba0 2023-12-01T02:52:12.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.452+0000 7f5f87fff640 1 -- 172.21.15.115:0/483044311 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5f9013fec0 con 0x7f5f900b1ba0 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.454+0000 7f5f956c4640 1 -- 172.21.15.115:0/483044311 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f7406a550 msgr2=0x7f5f7406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.454+0000 7f5f956c4640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f7406a550 0x7f5f7406ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5f80006fd0 tx=0x7f5f80008040 comp rx=0 tx=0).stop 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.454+0000 7f5f956c4640 1 -- 172.21.15.115:0/483044311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1ba0 msgr2=0x7f5f900cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.454+0000 7f5f956c4640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1ba0 0x7f5f900cebc0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f5f78000c00 tx=0x7f5f780046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.455+0000 7f5f956c4640 1 -- 172.21.15.115:0/483044311 shutdown_connections 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.455+0000 7f5f956c4640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f5f7406a550 0x7f5f7406ca10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.455+0000 7f5f956c4640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f901590b0 0x7f5f900cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.455+0000 7f5f956c4640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900b1ba0 0x7f5f900cebc0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.455+0000 7f5f956c4640 1 --2- 172.21.15.115:0/483044311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f900b1710 0x7f5f9015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.455+0000 7f5f956c4640 1 -- 172.21.15.115:0/483044311 >> 172.21.15.115:0/483044311 conn(0x7f5f90065960 msgr2=0x7f5f900b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:12.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.455+0000 7f5f956c4640 1 -- 172.21.15.115:0/483044311 shutdown_connections 2023-12-01T02:52:12.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.455+0000 7f5f956c4640 1 -- 172.21.15.115:0/483044311 wait complete. 2023-12-01T02:52:12.475 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T02:52:12.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.684+0000 7f6d0d08b640 1 Processor -- start 2023-12-01T02:52:12.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.684+0000 7f6d0d08b640 1 -- start start 2023-12-01T02:52:12.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.685+0000 7f6d0d08b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6d0815cf70 0x7f6d080b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.685+0000 7f6d0d08b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080d0260 0x7f6d080b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.685+0000 7f6d0d08b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d080b7760 con 0x7f6d080b18f0 2023-12-01T02:52:12.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.685+0000 7f6d0d08b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d080b2170 con 0x7f6d0815cf70 2023-12-01T02:52:12.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.685+0000 7f6d0d08b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080b8b70 con 0x7f6d080d0260 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6cfffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6d080b18f0 0x7f6d080cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60136/0 (socket says 172.21.15.115:60136) 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6cfffff640 1 -- 172.21.15.115:0/3910437946 learned_addr learned my addr 172.21.15.115:0/3910437946 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336933259 0 0) 0x7f6d080b7760 con 0x7f6d080b18f0 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d06d9d640 1 --2- 172.21.15.115:0/3910437946 >> v2:172.21.15.142:3300/0 conn(0x7f6d0815cf70 0x7f6d080b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d0759e640 1 --2- 172.21.15.115:0/3910437946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080d0260 0x7f6d080b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf4003520 con 0x7f6d080b18f0 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1059526949 0 0) 0x7f6cf4003520 con 0x7f6d080b18f0 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080d0260 msgr2=0x7f6d080b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d0659c640 1 --2- 172.21.15.115:0/3910437946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080d0260 0x7f6d080b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 >> v2:172.21.15.142:3300/0 conn(0x7f6d0815cf70 msgr2=0x7f6d080b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d0659c640 1 --2- 172.21.15.115:0/3910437946 >> v2:172.21.15.142:3300/0 conn(0x7f6d0815cf70 0x7f6d080b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080b6e30 con 0x7f6d080b18f0 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.686+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6cf0001670 con 0x7f6d080b18f0 2023-12-01T02:52:12.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 >> v1:172.21.15.115:6789/0 conn(0x7f6d080b18f0 legacy=0x7f6d080cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6d0659c640 1 --2- 172.21.15.115:0/3910437946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4003970 0x7f6cf4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6d0659c640 1 --2- 172.21.15.115:0/3910437946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cf40047c0 0x7f6cf4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6d0659c640 1 --2- 172.21.15.115:0/3910437946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080d0260 0x7f6cf4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080b2170 con 0x7f6cf40047c0 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080b8b70 con 0x7f6cf4003970 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf400b980 con 0x7f6d080d0260 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6cfffff640 1 --2- 172.21.15.115:0/3910437946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cf40047c0 0x7f6cf4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6cfffff640 1 -- 172.21.15.115:0/3910437946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080d0260 msgr2=0x7f6cf4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6cfffff640 1 --2- 172.21.15.115:0/3910437946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080d0260 0x7f6cf4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6cfffff640 1 -- 172.21.15.115:0/3910437946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4003970 msgr2=0x7f6cf4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6cfffff640 1 --2- 172.21.15.115:0/3910437946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf4003970 0x7f6cf4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.687+0000 7f6cfffff640 1 -- 172.21.15.115:0/3910437946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf4003520 con 0x7f6cf40047c0 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.688+0000 7f6cfffff640 1 --2- 172.21.15.115:0/3910437946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cf40047c0 0x7f6cf4004c40 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000c290 tx=0x7f6cf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cad3a8c0f0b0d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.689+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0005ad0 con 0x7f6cf40047c0 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.689+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf0005c30 con 0x7f6cf40047c0 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.689+0000 7f6d0659c640 1 -- 172.21.15.115:0/3910437946 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf002c8d0 con 0x7f6cf40047c0 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.689+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3910437946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cf40047c0 msgr2=0x7f6cf4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.689+0000 7f6d0d08b640 1 --2- 172.21.15.115:0/3910437946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cf40047c0 0x7f6cf4004c40 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000c290 tx=0x7f6cf40057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.689+0000 7f6d0759e640 1 -- 172.21.15.115:0/3910437946 reap_dead start 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.689+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3910437946 shutdown_connections 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.689+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3910437946 >> 172.21.15.115:0/3910437946 conn(0x7f6d0806a0e0 msgr2=0x7f6d0806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.689+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3910437946 shutdown_connections 2023-12-01T02:52:12.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.689+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3910437946 wait complete. 2023-12-01T02:52:12.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.691+0000 7f6d0d08b640 1 Processor -- start 2023-12-01T02:52:12.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.691+0000 7f6d0d08b640 1 -- start start 2023-12-01T02:52:12.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.692+0000 7f6d0d08b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf40047c0 0x7f6d080caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.692+0000 7f6d0d08b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080cc4d0 0x7f6d080cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.692+0000 7f6d0d08b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d0815e3a0 0x7f6d0815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.692+0000 7f6d0d08b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d081f5780 con 0x7f6d0815e3a0 2023-12-01T02:52:12.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.692+0000 7f6d0d08b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d081f59d0 con 0x7f6cf40047c0 2023-12-01T02:52:12.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.692+0000 7f6d0d08b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d081f5cb0 con 0x7f6d080cc4d0 2023-12-01T02:52:12.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6d0759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d0815e3a0 0x7f6d0815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6d0759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d0815e3a0 0x7f6d0815e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43360/0 (socket says 172.21.15.115:43360) 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6d0759e640 1 -- 172.21.15.115:0/3912235663 learned_addr learned my addr 172.21.15.115:0/3912235663 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6d06d9d640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf40047c0 0x7f6d080caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6cfffff640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080cc4d0 0x7f6d080cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6d0759e640 1 -- 172.21.15.115:0/3912235663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080cc4d0 msgr2=0x7f6d080cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6d0759e640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080cc4d0 0x7f6d080cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6d0759e640 1 -- 172.21.15.115:0/3912235663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf40047c0 msgr2=0x7f6d080caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6d0759e640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf40047c0 0x7f6d080caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6d0759e640 1 -- 172.21.15.115:0/3912235663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d081f5f90 con 0x7f6d0815e3a0 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.693+0000 7f6d0759e640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d0815e3a0 0x7f6d0815e800 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f6d080cf4d0 tx=0x7f6d0000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.694+0000 7f6cff7fe640 1 -- 172.21.15.115:0/3912235663 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00010040 con 0x7f6d0815e3a0 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.694+0000 7f6cff7fe640 1 -- 172.21.15.115:0/3912235663 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0000bae0 con 0x7f6d0815e3a0 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.694+0000 7f6cff7fe640 1 -- 172.21.15.115:0/3912235663 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0000be70 con 0x7f6d0815e3a0 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.694+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3912235663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d081f6270 con 0x7f6d0815e3a0 2023-12-01T02:52:12.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.694+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3912235663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d080cafb0 con 0x7f6d0815e3a0 2023-12-01T02:52:12.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.695+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3912235663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0804e890 con 0x7f6d0815e3a0 2023-12-01T02:52:12.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.696+0000 7f6cff7fe640 1 -- 172.21.15.115:0/3912235663 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6d0001e410 con 0x7f6d0815e3a0 2023-12-01T02:52:12.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.697+0000 7f6cff7fe640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ce806a550 0x7f6ce806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:12.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.697+0000 7f6cff7fe640 1 -- 172.21.15.115:0/3912235663 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f6d00088d90 con 0x7f6d0815e3a0 2023-12-01T02:52:12.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.697+0000 7f6d06d9d640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ce806a550 0x7f6ce806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:12.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.698+0000 7f6d06d9d640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ce806a550 0x7f6ce806ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8004770 tx=0x7f6cf8008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:12.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.701+0000 7f6cff7fe640 1 -- 172.21.15.115:0/3912235663 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6d00057c60 con 0x7f6d0815e3a0 2023-12-01T02:52:12.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.891+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3912235663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f6d08153620 con 0x7f6d0815e3a0 2023-12-01T02:52:12.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.900+0000 7f6cff7fe640 1 -- 172.21.15.115:0/3912235663 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v56) v1 ==== 120+0+0 (secure 0 0 0) 0x7f6d000577f0 con 0x7f6d0815e3a0 2023-12-01T02:52:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3912235663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ce806a550 msgr2=0x7f6ce806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ce806a550 0x7f6ce806ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8004770 tx=0x7f6cf8008040 comp rx=0 tx=0).stop 2023-12-01T02:52:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3912235663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d0815e3a0 msgr2=0x7f6d0815e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d0815e3a0 0x7f6d0815e800 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f6d080cf4d0 tx=0x7f6d0000b480 comp rx=0 tx=0).stop 2023-12-01T02:52:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3912235663 shutdown_connections 2023-12-01T02:52:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f6ce806a550 0x7f6ce806ca10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d080cc4d0 0x7f6d080cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d0815e3a0 0x7f6d0815e800 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 --2- 172.21.15.115:0/3912235663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cf40047c0 0x7f6d080caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:12.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3912235663 >> 172.21.15.115:0/3912235663 conn(0x7f6d0806a0e0 msgr2=0x7f6d080cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:12.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3912235663 shutdown_connections 2023-12-01T02:52:12.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:12.903+0000 7f6d0d08b640 1 -- 172.21.15.115:0/3912235663 wait complete. 2023-12-01T02:52:12.923 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T02:52:13.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.136+0000 7fe80239f640 1 Processor -- start 2023-12-01T02:52:13.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.136+0000 7fe80239f640 1 -- start start 2023-12-01T02:52:13.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.138+0000 7fe80239f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc0ce1b0 0x7fe7fc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.138+0000 7fe80239f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc0cc9a0 con 0x7fe7fc0b0e60 2023-12-01T02:52:13.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.138+0000 7fe80239f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc0cbd90 con 0x7fe7fc15ac80 2023-12-01T02:52:13.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.138+0000 7fe80239f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc15b060 con 0x7fe7fc0ce1b0 2023-12-01T02:52:13.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.138+0000 7fe80093c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe7fc0b0e60 0x7fe7fc06b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60142/0 (socket says 172.21.15.115:60142) 2023-12-01T02:52:13.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.138+0000 7fe80093c640 1 -- 172.21.15.115:0/3579131027 learned_addr learned my addr 172.21.15.115:0/3579131027 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.138+0000 7fe7f3fff640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc0ce1b0 0x7fe7fc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755904397 0 0) 0x7fe7fc0cc9a0 con 0x7fe7fc0b0e60 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e8003520 con 0x7fe7fc0b0e60 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3930203097 0 0) 0x7fe7e8003520 con 0x7fe7fc0b0e60 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc0ce1b0 msgr2=0x7fe7fc0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc0ce1b0 0x7fe7fc0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 >> v1:172.21.15.142:6789/0 conn(0x7fe7fc15ac80 legacy=0x7fe7fc0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc0cc070 con 0x7fe7fc0b0e60 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fe7f40019f0 con 0x7fe7fc0b0e60 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 >> v1:172.21.15.115:6789/0 conn(0x7fe7fc0b0e60 legacy=0x7fe7fc06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc0ce1b0 0x7fe7e80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e8003f20 0x7fe7e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7e8004d00 0x7fe7e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc15b060 con 0x7fe7e8004d00 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e800ba60 con 0x7fe7e8003f20 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e800bc20 con 0x7fe7fc0ce1b0 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.139+0000 7fe80093c640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7e8004d00 0x7fe7e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.140+0000 7fe7fbfff640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc0ce1b0 0x7fe7e80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.140+0000 7fe80093c640 1 -- 172.21.15.115:0/3579131027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc0ce1b0 msgr2=0x7fe7e80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.140+0000 7fe80093c640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc0ce1b0 0x7fe7e80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.140+0000 7fe80093c640 1 -- 172.21.15.115:0/3579131027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e8003f20 msgr2=0x7fe7e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.140+0000 7fe80093c640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7e8003f20 0x7fe7e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.140+0000 7fe80093c640 1 -- 172.21.15.115:0/3579131027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8003520 con 0x7fe7e8004d00 2023-12-01T02:52:13.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.141+0000 7fe80093c640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7e8004d00 0x7fe7e8005180 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400a820 tx=0x7fe7e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f68d6261a0ae6b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:13.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.141+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40072d0 con 0x7fe7e8004d00 2023-12-01T02:52:13.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.141+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f402f070 con 0x7fe7e8004d00 2023-12-01T02:52:13.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.141+0000 7fe7fb7fe640 1 -- 172.21.15.115:0/3579131027 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40076f0 con 0x7fe7e8004d00 2023-12-01T02:52:13.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.142+0000 7fe80239f640 1 -- 172.21.15.115:0/3579131027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7e8004d00 msgr2=0x7fe7e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:13.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.142+0000 7fe80239f640 1 --2- 172.21.15.115:0/3579131027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7e8004d00 0x7fe7e8005180 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400a820 tx=0x7fe7e80057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.142+0000 7fe80093c640 1 -- 172.21.15.115:0/3579131027 reap_dead start 2023-12-01T02:52:13.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.142+0000 7fe80239f640 1 -- 172.21.15.115:0/3579131027 shutdown_connections 2023-12-01T02:52:13.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.142+0000 7fe80239f640 1 -- 172.21.15.115:0/3579131027 >> 172.21.15.115:0/3579131027 conn(0x7fe7fc065960 msgr2=0x7fe7fc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:13.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.142+0000 7fe80239f640 1 -- 172.21.15.115:0/3579131027 shutdown_connections 2023-12-01T02:52:13.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.142+0000 7fe80239f640 1 -- 172.21.15.115:0/3579131027 wait complete. 2023-12-01T02:52:13.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.144+0000 7fe80239f640 1 Processor -- start 2023-12-01T02:52:13.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.144+0000 7fe80239f640 1 -- start start 2023-12-01T02:52:13.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.145+0000 7fe80239f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc152f30 0x7fe7fc1f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.145+0000 7fe80239f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc1f6240 0x7fe7fc153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.145+0000 7fe80239f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7fc153850 0x7fe7fc1f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.145+0000 7fe80239f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc1f6680 con 0x7fe7fc153850 2023-12-01T02:52:13.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.145+0000 7fe80239f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc154aa0 con 0x7fe7fc152f30 2023-12-01T02:52:13.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.145+0000 7fe80239f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc1f6b80 con 0x7fe7fc1f6240 2023-12-01T02:52:13.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.145+0000 7fe80093c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7fc153850 0x7fe7fc1f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.145+0000 7fe80093c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7fc153850 0x7fe7fc1f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43368/0 (socket says 172.21.15.115:43368) 2023-12-01T02:52:13.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.145+0000 7fe80093c640 1 -- 172.21.15.115:0/2841903678 learned_addr learned my addr 172.21.15.115:0/2841903678 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:13.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.146+0000 7fe7f3fff640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc1f6240 0x7fe7fc153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.146+0000 7fe7fbfff640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc152f30 0x7fe7fc1f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.146+0000 7fe80093c640 1 -- 172.21.15.115:0/2841903678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc1f6240 msgr2=0x7fe7fc153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.146+0000 7fe80093c640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc1f6240 0x7fe7fc153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.146+0000 7fe80093c640 1 -- 172.21.15.115:0/2841903678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc152f30 msgr2=0x7fe7fc1f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.146+0000 7fe80093c640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc152f30 0x7fe7fc1f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.146+0000 7fe80093c640 1 -- 172.21.15.115:0/2841903678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc154220 con 0x7fe7fc153850 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.146+0000 7fe80093c640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7fc153850 0x7fe7fc1f2710 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400a820 tx=0x7fe7fc0cc910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.146+0000 7fe7f97fa640 1 -- 172.21.15.115:0/2841903678 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4044070 con 0x7fe7fc153850 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.147+0000 7fe7f97fa640 1 -- 172.21.15.115:0/2841903678 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f4001250 con 0x7fe7fc153850 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.147+0000 7fe7f97fa640 1 -- 172.21.15.115:0/2841903678 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4007520 con 0x7fe7fc153850 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.147+0000 7fe80239f640 1 -- 172.21.15.115:0/2841903678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc1544e0 con 0x7fe7fc153850 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.147+0000 7fe80239f640 1 -- 172.21.15.115:0/2841903678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc1f39c0 con 0x7fe7fc153850 2023-12-01T02:52:13.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.148+0000 7fe80239f640 1 -- 172.21.15.115:0/2841903678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc1f3820 con 0x7fe7fc153850 2023-12-01T02:52:13.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.149+0000 7fe7f97fa640 1 -- 172.21.15.115:0/2841903678 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe7f4031030 con 0x7fe7fc153850 2023-12-01T02:52:13.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.150+0000 7fe7f97fa640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7dc06a580 0x7fe7dc06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.150+0000 7fe7f97fa640 1 -- 172.21.15.115:0/2841903678 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fe7f40aa2a0 con 0x7fe7fc153850 2023-12-01T02:52:13.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.150+0000 7fe7fbfff640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7dc06a580 0x7fe7dc06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.151+0000 7fe7fbfff640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7dc06a580 0x7fe7dc06ca40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec001710 tx=0x7fe7ec009040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:13.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.155+0000 7fe7f97fa640 1 -- 172.21.15.115:0/2841903678 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe7fc1f3820 con 0x7fe7fc153850 2023-12-01T02:52:13.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.346+0000 7fe80239f640 1 -- 172.21.15.115:0/2841903678 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fe7fc1f3820 con 0x7fe7fc153850 2023-12-01T02:52:13.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.352+0000 7fe7f97fa640 1 -- 172.21.15.115:0/2841903678 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v57) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe7fc1f3820 con 0x7fe7fc153850 2023-12-01T02:52:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.355+0000 7fe80239f640 1 -- 172.21.15.115:0/2841903678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7dc06a580 msgr2=0x7fe7dc06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.355+0000 7fe80239f640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7dc06a580 0x7fe7dc06ca40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec001710 tx=0x7fe7ec009040 comp rx=0 tx=0).stop 2023-12-01T02:52:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.355+0000 7fe80239f640 1 -- 172.21.15.115:0/2841903678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7fc153850 msgr2=0x7fe7fc1f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:13.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.355+0000 7fe80239f640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7fc153850 0x7fe7fc1f2710 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400a820 tx=0x7fe7fc0cc910 comp rx=0 tx=0).stop 2023-12-01T02:52:13.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.355+0000 7fe80239f640 1 -- 172.21.15.115:0/2841903678 shutdown_connections 2023-12-01T02:52:13.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.355+0000 7fe80239f640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe7dc06a580 0x7fe7dc06ca40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.356+0000 7fe80239f640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7fc1f6240 0x7fe7fc153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.356+0000 7fe80239f640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7fc153850 0x7fe7fc1f2710 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.356+0000 7fe80239f640 1 --2- 172.21.15.115:0/2841903678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7fc152f30 0x7fe7fc1f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.356+0000 7fe80239f640 1 -- 172.21.15.115:0/2841903678 >> 172.21.15.115:0/2841903678 conn(0x7fe7fc065960 msgr2=0x7fe7fc0c90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:13.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.356+0000 7fe80239f640 1 -- 172.21.15.115:0/2841903678 shutdown_connections 2023-12-01T02:52:13.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.356+0000 7fe80239f640 1 -- 172.21.15.115:0/2841903678 wait complete. 2023-12-01T02:52:13.377 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-12-01T02:52:13.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.573+0000 7fc8afce0640 1 Processor -- start 2023-12-01T02:52:13.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.573+0000 7fc8afce0640 1 -- start start 2023-12-01T02:52:13.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.574+0000 7fc8afce0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc8a8159dd0 0x7fc8a80c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.574+0000 7fc8afce0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc8a806b6f0 0x7fc8a80c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.574+0000 7fc8afce0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a806f8b0 con 0x7fc8a80ade80 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.574+0000 7fc8afce0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a80adcb0 con 0x7fc8a806b6f0 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.574+0000 7fc8afce0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a80ad9d0 con 0x7fc8a8159dd0 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.575+0000 7fc8ad27b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc8a806b6f0 0x7fc8a80c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.575+0000 7fc8ad27b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc8a806b6f0 0x7fc8a80c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43382/0 (socket says 172.21.15.115:43382) 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.575+0000 7fc8ad27b640 1 -- 172.21.15.115:0/3075969997 learned_addr learned my addr 172.21.15.115:0/3075969997 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.575+0000 7fc8ada7c640 1 --2- 172.21.15.115:0/3075969997 >> v2:172.21.15.142:3300/0 conn(0x7fc8a8159dd0 0x7fc8a80c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.575+0000 7fc8ad27b640 1 -- 172.21.15.115:0/3075969997 >> v2:172.21.15.142:3300/0 conn(0x7fc8a8159dd0 msgr2=0x7fc8a80c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.575+0000 7fc8ad27b640 1 --2- 172.21.15.115:0/3075969997 >> v2:172.21.15.142:3300/0 conn(0x7fc8a8159dd0 0x7fc8a80c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.575+0000 7fc8ad27b640 1 -- 172.21.15.115:0/3075969997 >> v1:172.21.15.142:6789/0 conn(0x7fc8a80ade80 legacy=0x7fc8a80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.575+0000 7fc8ad27b640 1 -- 172.21.15.115:0/3075969997 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80c9000 con 0x7fc8a806b6f0 2023-12-01T02:52:13.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.575+0000 7fc8ada7c640 1 --2- 172.21.15.115:0/3075969997 >> v2:172.21.15.142:3300/0 conn(0x7fc8a8159dd0 0x7fc8a80c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.576+0000 7fc8ad27b640 1 --2- 172.21.15.115:0/3075969997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a806b6f0 0x7fc8a80c9d20 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80cf4e0 tx=0x7fc89c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485f914ed2fc7d06 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.576+0000 7fc8aca7a640 1 -- 172.21.15.115:0/3075969997 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c031750 con 0x7fc8a806b6f0 2023-12-01T02:52:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.577+0000 7fc8aca7a640 1 -- 172.21.15.115:0/3075969997 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c0318b0 con 0x7fc8a806b6f0 2023-12-01T02:52:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.577+0000 7fc8aca7a640 1 -- 172.21.15.115:0/3075969997 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c031c40 con 0x7fc8a806b6f0 2023-12-01T02:52:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.577+0000 7fc8afce0640 1 -- 172.21.15.115:0/3075969997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a806b6f0 msgr2=0x7fc8a80c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.577+0000 7fc8afce0640 1 --2- 172.21.15.115:0/3075969997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a806b6f0 0x7fc8a80c9d20 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80cf4e0 tx=0x7fc89c02b7f0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.577+0000 7fc8afce0640 1 -- 172.21.15.115:0/3075969997 shutdown_connections 2023-12-01T02:52:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.577+0000 7fc8afce0640 1 --2- 172.21.15.115:0/3075969997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a806b6f0 0x7fc8a80c9d20 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.577+0000 7fc8afce0640 1 --2- 172.21.15.115:0/3075969997 >> v2:172.21.15.142:3300/0 conn(0x7fc8a8159dd0 0x7fc8a80c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.577+0000 7fc8afce0640 1 -- 172.21.15.115:0/3075969997 >> 172.21.15.115:0/3075969997 conn(0x7fc8a80ce880 msgr2=0x7fc8a80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:13.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.577+0000 7fc8afce0640 1 -- 172.21.15.115:0/3075969997 shutdown_connections 2023-12-01T02:52:13.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.577+0000 7fc8afce0640 1 -- 172.21.15.115:0/3075969997 wait complete. 2023-12-01T02:52:13.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.579+0000 7fc8afce0640 1 Processor -- start 2023-12-01T02:52:13.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.579+0000 7fc8afce0640 1 -- start start 2023-12-01T02:52:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8afce0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8a806b6f0 0x7fc8a8165460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8afce0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a80ade80 0x7fc8a8149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8afce0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a8159dd0 0x7fc8a80c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8afce0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8166450 con 0x7fc8a80ade80 2023-12-01T02:52:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8afce0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81659a0 con 0x7fc8a8159dd0 2023-12-01T02:52:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8afce0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80c1d90 con 0x7fc8a806b6f0 2023-12-01T02:52:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8ad27b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a80ade80 0x7fc8a8149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8ad27b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a80ade80 0x7fc8a8149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43398/0 (socket says 172.21.15.115:43398) 2023-12-01T02:52:13.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8ad27b640 1 -- 172.21.15.115:0/1726509929 learned_addr learned my addr 172.21.15.115:0/1726509929 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:13.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8ada7c640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8a806b6f0 0x7fc8a8165460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.580+0000 7fc8ad27b640 1 -- 172.21.15.115:0/1726509929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8a806b6f0 msgr2=0x7fc8a8165460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:13.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc8ae27d640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a8159dd0 0x7fc8a80c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc8ad27b640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8a806b6f0 0x7fc8a8165460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc8ad27b640 1 -- 172.21.15.115:0/1726509929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a8159dd0 msgr2=0x7fc8a80c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc8ad27b640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a8159dd0 0x7fc8a80c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc8ad27b640 1 -- 172.21.15.115:0/1726509929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0094d0 con 0x7fc8a80ade80 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc8ada7c640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8a806b6f0 0x7fc8a8165460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc8ad27b640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a80ade80 0x7fc8a8149e60 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fc89c02b830 tx=0x7fc89c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc89a7fc640 1 -- 172.21.15.115:0/1726509929 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c033c80 con 0x7fc8a80ade80 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc89a7fc640 1 -- 172.21.15.115:0/1726509929 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c033e00 con 0x7fc8a80ade80 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc8afce0640 1 -- 172.21.15.115:0/1726509929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a8165bb0 con 0x7fc8a80ade80 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc89a7fc640 1 -- 172.21.15.115:0/1726509929 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c03c640 con 0x7fc8a80ade80 2023-12-01T02:52:13.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.581+0000 7fc8afce0640 1 -- 172.21.15.115:0/1726509929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a8166170 con 0x7fc8a80ade80 2023-12-01T02:52:13.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.582+0000 7fc8afce0640 1 -- 172.21.15.115:0/1726509929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a804e890 con 0x7fc8a80ade80 2023-12-01T02:52:13.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.583+0000 7fc89a7fc640 1 -- 172.21.15.115:0/1726509929 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc89c03a040 con 0x7fc8a80ade80 2023-12-01T02:52:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.585+0000 7fc89a7fc640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc88c06a550 0x7fc88c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.585+0000 7fc89a7fc640 1 -- 172.21.15.115:0/1726509929 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fc89c0aa8d0 con 0x7fc8a80ade80 2023-12-01T02:52:13.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.585+0000 7fc8ada7c640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc88c06a550 0x7fc88c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:13.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.585+0000 7fc8ada7c640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc88c06a550 0x7fc88c06ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4006fd0 tx=0x7fc8a4008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:13.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.589+0000 7fc89a7fc640 1 -- 172.21.15.115:0/1726509929 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc89c075d80 con 0x7fc8a80ade80 2023-12-01T02:52:13.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.826+0000 7fc8afce0640 1 -- 172.21.15.115:0/1726509929 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7fc8a8150790 con 0x7fc8a80ade80 2023-12-01T02:52:13.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:13.997+0000 7fc89a7fc640 1 -- 172.21.15.115:0/1726509929 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2478) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc89c038020 con 0x7fc8a80ade80 2023-12-01T02:52:14.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.001+0000 7fc8afce0640 1 -- 172.21.15.115:0/1726509929 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc88c06a550 msgr2=0x7fc88c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:14.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.001+0000 7fc8afce0640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc88c06a550 0x7fc88c06ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4006fd0 tx=0x7fc8a4008040 comp rx=0 tx=0).stop 2023-12-01T02:52:14.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.001+0000 7fc8afce0640 1 -- 172.21.15.115:0/1726509929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a80ade80 msgr2=0x7fc8a8149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:14.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.001+0000 7fc8afce0640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a80ade80 0x7fc8a8149e60 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fc89c02b830 tx=0x7fc89c0046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:14.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.002+0000 7fc8afce0640 1 -- 172.21.15.115:0/1726509929 shutdown_connections 2023-12-01T02:52:14.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.002+0000 7fc8afce0640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc88c06a550 0x7fc88c06ca10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:14.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.002+0000 7fc8afce0640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a8159dd0 0x7fc8a80c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:14.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.002+0000 7fc8afce0640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8a80ade80 0x7fc8a8149e60 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:14.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.002+0000 7fc8afce0640 1 --2- 172.21.15.115:0/1726509929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8a806b6f0 0x7fc8a8165460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:14.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.002+0000 7fc8afce0640 1 -- 172.21.15.115:0/1726509929 >> 172.21.15.115:0/1726509929 conn(0x7fc8a80ce880 msgr2=0x7fc8a8070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:14.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.002+0000 7fc8afce0640 1 -- 172.21.15.115:0/1726509929 shutdown_connections 2023-12-01T02:52:14.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.002+0000 7fc8afce0640 1 -- 172.21.15.115:0/1726509929 wait complete. 2023-12-01T02:52:14.022 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-12-01T02:52:14.022 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-12-01T02:52:14.022 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-01T02:52:14.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-12-01T02:52:14.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.226+0000 7f860d399640 1 Processor -- start 2023-12-01T02:52:14.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.227+0000 7f860d399640 1 -- start start 2023-12-01T02:52:14.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.228+0000 7f860d399640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f860815cd20 0x7f86080d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:14.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.228+0000 7f860d399640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86080b5d80 0x7f86080b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:14.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.228+0000 7f860d399640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86080ce2e0 con 0x7f86080d1230 2023-12-01T02:52:14.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.229+0000 7f860d399640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86080b2160 con 0x7f86080b5d80 2023-12-01T02:52:14.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.229+0000 7f860d399640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86080d2a10 con 0x7f860815cd20 2023-12-01T02:52:14.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.229+0000 7f86067fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86080b5d80 0x7f86080b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:14.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.229+0000 7f86067fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86080b5d80 0x7f86080b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43414/0 (socket says 172.21.15.115:43414) 2023-12-01T02:52:14.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.229+0000 7f86067fc640 1 -- 172.21.15.115:0/464097865 learned_addr learned my addr 172.21.15.115:0/464097865 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:14.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.229+0000 7f8606ffd640 1 --2- 172.21.15.115:0/464097865 >> v2:172.21.15.142:3300/0 conn(0x7f860815cd20 0x7f86080d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:14.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.229+0000 7f86067fc640 1 -- 172.21.15.115:0/464097865 >> v2:172.21.15.142:3300/0 conn(0x7f860815cd20 msgr2=0x7f86080d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:14.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.229+0000 7f86067fc640 1 --2- 172.21.15.115:0/464097865 >> v2:172.21.15.142:3300/0 conn(0x7f860815cd20 0x7f86080d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:14.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.229+0000 7f86067fc640 1 -- 172.21.15.115:0/464097865 >> v1:172.21.15.142:6789/0 conn(0x7f86080d1230 legacy=0x7f86080d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:14.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.229+0000 7f86067fc640 1 -- 172.21.15.115:0/464097865 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86080ce000 con 0x7f86080b5d80 2023-12-01T02:52:14.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.230+0000 7f86067fc640 1 --2- 172.21.15.115:0/464097865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080b5d80 0x7f86080b8570 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f860808bdb0 tx=0x7f85f002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653c6d78af7dfe6b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f8605ffb640 1 -- 172.21.15.115:0/464097865 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00317e0 con 0x7f86080b5d80 2023-12-01T02:52:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f8605ffb640 1 -- 172.21.15.115:0/464097865 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f0031940 con 0x7f86080b5d80 2023-12-01T02:52:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f8605ffb640 1 -- 172.21.15.115:0/464097865 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0031cd0 con 0x7f86080b5d80 2023-12-01T02:52:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f860d399640 1 -- 172.21.15.115:0/464097865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080b5d80 msgr2=0x7f86080b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f860d399640 1 --2- 172.21.15.115:0/464097865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080b5d80 0x7f86080b8570 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f860808bdb0 tx=0x7f85f002b880 comp rx=0 tx=0).stop 2023-12-01T02:52:14.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f860d399640 1 -- 172.21.15.115:0/464097865 shutdown_connections 2023-12-01T02:52:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f860d399640 1 --2- 172.21.15.115:0/464097865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080b5d80 0x7f86080b8570 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f860d399640 1 --2- 172.21.15.115:0/464097865 >> v2:172.21.15.142:3300/0 conn(0x7f860815cd20 0x7f86080d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f860d399640 1 -- 172.21.15.115:0/464097865 >> 172.21.15.115:0/464097865 conn(0x7f860806a0e0 msgr2=0x7f860806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f860d399640 1 -- 172.21.15.115:0/464097865 shutdown_connections 2023-12-01T02:52:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.231+0000 7f860d399640 1 -- 172.21.15.115:0/464097865 wait complete. 2023-12-01T02:52:14.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.233+0000 7f860d399640 1 Processor -- start 2023-12-01T02:52:14.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.233+0000 7f860d399640 1 -- start start 2023-12-01T02:52:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.234+0000 7f860d399640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86080b5d80 0x7f8608151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.234+0000 7f860d399640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080d1230 0x7f8608158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.234+0000 7f860d399640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f860815cd20 0x7f86081588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.234+0000 7f860d399640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c8900 con 0x7f86080d1230 2023-12-01T02:52:14.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.234+0000 7f860d399640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c8ba0 con 0x7f86080b5d80 2023-12-01T02:52:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.234+0000 7f860d399640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c8e80 con 0x7f860815cd20 2023-12-01T02:52:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.234+0000 7f86067fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080d1230 0x7f8608158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.234+0000 7f86067fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080d1230 0x7f8608158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43416/0 (socket says 172.21.15.115:43416) 2023-12-01T02:52:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.234+0000 7f86067fc640 1 -- 172.21.15.115:0/2223344418 learned_addr learned my addr 172.21.15.115:0/2223344418 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.235+0000 7f86067fc640 1 -- 172.21.15.115:0/2223344418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f860815cd20 msgr2=0x7f86081588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.235+0000 7f86067fc640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f860815cd20 0x7f86081588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.235+0000 7f86067fc640 1 -- 172.21.15.115:0/2223344418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86080b5d80 msgr2=0x7f8608151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.235+0000 7f86067fc640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86080b5d80 0x7f8608151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:14.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.235+0000 7f86067fc640 1 -- 172.21.15.115:0/2223344418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00094d0 con 0x7f86080d1230 2023-12-01T02:52:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.235+0000 7f86067fc640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080d1230 0x7f8608158390 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f85f0000c00 tx=0x7f85f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.235+0000 7f85ff7fe640 1 -- 172.21.15.115:0/2223344418 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00329c0 con 0x7f86080d1230 2023-12-01T02:52:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.235+0000 7f85ff7fe640 1 -- 172.21.15.115:0/2223344418 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f0032b40 con 0x7f86080d1230 2023-12-01T02:52:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.235+0000 7f860d399640 1 -- 172.21.15.115:0/2223344418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86080c9160 con 0x7f86080d1230 2023-12-01T02:52:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.236+0000 7f85ff7fe640 1 -- 172.21.15.115:0/2223344418 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00453f0 con 0x7f86080d1230 2023-12-01T02:52:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.236+0000 7f860d399640 1 -- 172.21.15.115:0/2223344418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86081518a0 con 0x7f86080d1230 2023-12-01T02:52:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.237+0000 7f860d399640 1 -- 172.21.15.115:0/2223344418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86080c9680 con 0x7f86080d1230 2023-12-01T02:52:14.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.238+0000 7f85ff7fe640 1 -- 172.21.15.115:0/2223344418 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f85f004e450 con 0x7f86080d1230 2023-12-01T02:52:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.238+0000 7f85ff7fe640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85e806a550 0x7f85e806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.239+0000 7f85ff7fe640 1 -- 172.21.15.115:0/2223344418 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f85f00aae70 con 0x7f86080d1230 2023-12-01T02:52:14.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.239+0000 7f8606ffd640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85e806a550 0x7f85e806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:14.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.239+0000 7f8606ffd640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85e806a550 0x7f85e806ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f85f8007db0 tx=0x7f85f80072e0 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:14.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.243+0000 7f85ff7fe640 1 -- 172.21.15.115:0/2223344418 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f86080c9680 con 0x7f86080d1230 2023-12-01T02:52:14.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.501+0000 7f860d399640 1 -- 172.21.15.115:0/2223344418 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f86080c9680 con 0x7f86080d1230 2023-12-01T02:52:15.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:14.999+0000 7f85ff7fe640 1 -- 172.21.15.115:0/2223344418 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2479) v1 ==== 168+0+0 (secure 0 0 0) 0x7f86080c9680 con 0x7f86080d1230 2023-12-01T02:52:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 -- 172.21.15.115:0/2223344418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85e806a550 msgr2=0x7f85e806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85e806a550 0x7f85e806ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f85f8007db0 tx=0x7f85f80072e0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 -- 172.21.15.115:0/2223344418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080d1230 msgr2=0x7f8608158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080d1230 0x7f8608158390 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f85f0000c00 tx=0x7f85f00046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 -- 172.21.15.115:0/2223344418 shutdown_connections 2023-12-01T02:52:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f85e806a550 0x7f85e806ca10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f860815cd20 0x7f86081588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86080d1230 0x7f8608158390 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 --2- 172.21.15.115:0/2223344418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86080b5d80 0x7f8608151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 -- 172.21.15.115:0/2223344418 >> 172.21.15.115:0/2223344418 conn(0x7f860806a0e0 msgr2=0x7f86081597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:15.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 -- 172.21.15.115:0/2223344418 shutdown_connections 2023-12-01T02:52:15.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.003+0000 7f860d399640 1 -- 172.21.15.115:0/2223344418 wait complete. 2023-12-01T02:52:15.027 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T02:52:15.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.238+0000 7fc48b739640 1 Processor -- start 2023-12-01T02:52:15.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.238+0000 7fc48b739640 1 -- start start 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.239+0000 7fc48b739640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc484160200 0x7fc4840b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.239+0000 7fc48b739640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4840cbd30 0x7fc4840cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.239+0000 7fc48b739640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4840cc6e0 con 0x7fc4840b0f80 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.239+0000 7fc48b739640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4840b35f0 con 0x7fc484160200 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.239+0000 7fc48b739640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840cb290 con 0x7fc4840cbd30 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.240+0000 7fc4894d5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc484160200 0x7fc4840b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.240+0000 7fc4894d5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc484160200 0x7fc4840b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43430/0 (socket says 172.21.15.115:43430) 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.240+0000 7fc4894d5640 1 -- 172.21.15.115:0/1631509687 learned_addr learned my addr 172.21.15.115:0/1631509687 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.240+0000 7fc4894d5640 1 -- 172.21.15.115:0/1631509687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4840cbd30 msgr2=0x7fc4840cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.240+0000 7fc4894d5640 1 --2- 172.21.15.115:0/1631509687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4840cbd30 0x7fc4840cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.240+0000 7fc4894d5640 1 -- 172.21.15.115:0/1631509687 >> v1:172.21.15.142:6789/0 conn(0x7fc4840b0f80 legacy=0x7fc48406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:15.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.240+0000 7fc4894d5640 1 -- 172.21.15.115:0/1631509687 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4840b0ae0 con 0x7fc484160200 2023-12-01T02:52:15.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.241+0000 7fc4894d5640 1 --2- 172.21.15.115:0/1631509687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc484160200 0x7fc4840b2c80 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc480009870 tx=0x7fc48002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c76951448c3aa5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:15.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.241+0000 7fc47bfff640 1 -- 172.21.15.115:0/1631509687 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480031640 con 0x7fc484160200 2023-12-01T02:52:15.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.241+0000 7fc47bfff640 1 -- 172.21.15.115:0/1631509687 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4800317a0 con 0x7fc484160200 2023-12-01T02:52:15.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.241+0000 7fc47bfff640 1 -- 172.21.15.115:0/1631509687 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480031b30 con 0x7fc484160200 2023-12-01T02:52:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.242+0000 7fc48b739640 1 -- 172.21.15.115:0/1631509687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc484160200 msgr2=0x7fc4840b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.242+0000 7fc48b739640 1 --2- 172.21.15.115:0/1631509687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc484160200 0x7fc4840b2c80 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc480009870 tx=0x7fc48002b8f0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.242+0000 7fc48b739640 1 -- 172.21.15.115:0/1631509687 shutdown_connections 2023-12-01T02:52:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.242+0000 7fc48b739640 1 --2- 172.21.15.115:0/1631509687 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc4840cbd30 0x7fc4840cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.242+0000 7fc48b739640 1 --2- 172.21.15.115:0/1631509687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc484160200 0x7fc4840b2c80 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.242+0000 7fc48b739640 1 -- 172.21.15.115:0/1631509687 >> 172.21.15.115:0/1631509687 conn(0x7fc484065960 msgr2=0x7fc484065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.242+0000 7fc48b739640 1 -- 172.21.15.115:0/1631509687 shutdown_connections 2023-12-01T02:52:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.242+0000 7fc48b739640 1 -- 172.21.15.115:0/1631509687 wait complete. 2023-12-01T02:52:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.244+0000 7fc48b739640 1 Processor -- start 2023-12-01T02:52:15.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.244+0000 7fc48b739640 1 -- start start 2023-12-01T02:52:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc48b739640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4840b0f80 0x7fc4841510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc48b739640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4840cbd30 0x7fc4840c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc48b739640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc484160200 0x7fc4841f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc48b739640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48414fc90 con 0x7fc4840cbd30 2023-12-01T02:52:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc48b739640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840c8910 con 0x7fc4840b0f80 2023-12-01T02:52:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc48b739640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840c8bb0 con 0x7fc484160200 2023-12-01T02:52:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc488cd4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4840cbd30 0x7fc4840c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc488cd4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4840cbd30 0x7fc4840c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43444/0 (socket says 172.21.15.115:43444) 2023-12-01T02:52:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc488cd4640 1 -- 172.21.15.115:0/2429831159 learned_addr learned my addr 172.21.15.115:0/2429831159 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:15.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc4894d5640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4840b0f80 0x7fc4841510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc489cd6640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc484160200 0x7fc4841f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc488cd4640 1 -- 172.21.15.115:0/2429831159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc484160200 msgr2=0x7fc4841f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc488cd4640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc484160200 0x7fc4841f1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc488cd4640 1 -- 172.21.15.115:0/2429831159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4840b0f80 msgr2=0x7fc4841510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc488cd4640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4840b0f80 0x7fc4841510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.245+0000 7fc488cd4640 1 -- 172.21.15.115:0/2429831159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800094d0 con 0x7fc4840cbd30 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.246+0000 7fc488cd4640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4840cbd30 0x7fc4840c83d0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fc47400b3e0 tx=0x7fc47400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.246+0000 7fc479ffb640 1 -- 172.21.15.115:0/2429831159 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474011830 con 0x7fc4840cbd30 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.246+0000 7fc479ffb640 1 -- 172.21.15.115:0/2429831159 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4740119b0 con 0x7fc4840cbd30 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.246+0000 7fc479ffb640 1 -- 172.21.15.115:0/2429831159 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474011d60 con 0x7fc4840cbd30 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.246+0000 7fc48b739640 1 -- 172.21.15.115:0/2429831159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4840c8e90 con 0x7fc4840cbd30 2023-12-01T02:52:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.246+0000 7fc48b739640 1 -- 172.21.15.115:0/2429831159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4840c9450 con 0x7fc4840cbd30 2023-12-01T02:52:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.247+0000 7fc48b739640 1 -- 172.21.15.115:0/2429831159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48404e890 con 0x7fc4840cbd30 2023-12-01T02:52:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.248+0000 7fc479ffb640 1 -- 172.21.15.115:0/2429831159 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc47401a400 con 0x7fc4840cbd30 2023-12-01T02:52:15.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.249+0000 7fc479ffb640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc46806a550 0x7fc46806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:15.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.250+0000 7fc479ffb640 1 -- 172.21.15.115:0/2429831159 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fc474088c50 con 0x7fc4840cbd30 2023-12-01T02:52:15.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.250+0000 7fc4894d5640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc46806a550 0x7fc46806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.250+0000 7fc4894d5640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc46806a550 0x7fc46806ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc480036000 tx=0x7fc48003c040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:15.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.253+0000 7fc479ffb640 1 -- 172.21.15.115:0/2429831159 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc474054100 con 0x7fc4840cbd30 2023-12-01T02:52:15.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.429+0000 7fc48b739640 1 -- 172.21.15.115:0/2429831159 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc4840b1e20 con 0x7fc4840cbd30 2023-12-01T02:52:15.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.436+0000 7fc479ffb640 1 -- 172.21.15.115:0/2429831159 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc474015020 con 0x7fc4840cbd30 2023-12-01T02:52:15.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 -- 172.21.15.115:0/2429831159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc46806a550 msgr2=0x7fc46806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:15.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc46806a550 0x7fc46806ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc480036000 tx=0x7fc48003c040 comp rx=0 tx=0).stop 2023-12-01T02:52:15.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 -- 172.21.15.115:0/2429831159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4840cbd30 msgr2=0x7fc4840c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:15.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4840cbd30 0x7fc4840c83d0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fc47400b3e0 tx=0x7fc47400b8a0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 -- 172.21.15.115:0/2429831159 shutdown_connections 2023-12-01T02:52:15.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc46806a550 0x7fc46806ca10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc484160200 0x7fc4841f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4840cbd30 0x7fc4840c83d0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 --2- 172.21.15.115:0/2429831159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4840b0f80 0x7fc4841510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:15.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 -- 172.21.15.115:0/2429831159 >> 172.21.15.115:0/2429831159 conn(0x7fc484065960 msgr2=0x7fc4840709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:15.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 -- 172.21.15.115:0/2429831159 shutdown_connections 2023-12-01T02:52:15.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:15.439+0000 7fc48b739640 1 -- 172.21.15.115:0/2429831159 wait complete. 2023-12-01T02:52:15.457 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T02:52:15.457 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T02:52:15.571 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T02:52:15.696 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T02:52:15.696 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:52:15.696 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:52:15.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T02:52:15.758 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T02:52:15.758 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:52:15.817 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T02:52:15.818 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:52:15.818 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:52:15.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T02:52:15.880 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T02:52:15.880 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:52:15.937 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T02:52:15.938 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T02:52:15.938 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:52:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.164+0000 7f4e64c15640 1 Processor -- start 2023-12-01T02:52:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.164+0000 7f4e64c15640 1 -- start start 2023-12-01T02:52:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.166+0000 7f4e64c15640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4e6015e2f0 0x7f4e600ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.166+0000 7f4e64c15640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600adf10 0x7f4e600caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.166+0000 7f4e64c15640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e600add10 con 0x7f4e6006f2f0 2023-12-01T02:52:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.166+0000 7f4e64c15640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e600c5760 con 0x7f4e6015e2f0 2023-12-01T02:52:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.166+0000 7f4e64c15640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6015e6d0 con 0x7f4e600adf10 2023-12-01T02:52:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.166+0000 7f4e57fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4e6006f2f0 0x7f4e600c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60154/0 (socket says 172.21.15.115:60154) 2023-12-01T02:52:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.166+0000 7f4e57fff640 1 -- 172.21.15.115:0/4192537386 learned_addr learned my addr 172.21.15.115:0/4192537386 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.166+0000 7f4e5e59c640 1 --2- 172.21.15.115:0/4192537386 >> v2:172.21.15.142:3300/0 conn(0x7f4e6015e2f0 0x7f4e600ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5ed9d640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600adf10 0x7f4e600caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789045179 0 0) 0x7f4e600add10 con 0x7f4e6006f2f0 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e4c003520 con 0x7f4e6006f2f0 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3430037463 0 0) 0x7f4e4c003520 con 0x7f4e6006f2f0 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600adf10 msgr2=0x7f4e600caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600adf10 0x7f4e600caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 >> v2:172.21.15.142:3300/0 conn(0x7f4e6015e2f0 msgr2=0x7f4e600ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 --2- 172.21.15.115:0/4192537386 >> v2:172.21.15.142:3300/0 conn(0x7f4e6015e2f0 0x7f4e600ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600cf7b0 con 0x7f4e6006f2f0 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4e48001690 con 0x7f4e6006f2f0 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 >> v1:172.21.15.115:6789/0 conn(0x7f4e6006f2f0 legacy=0x7f4e600c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:16.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600adf10 0x7f4e4c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 0x7f4e4c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4c004d00 0x7f4e4c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600c5760 con 0x7f4e4c003f60 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6015e6d0 con 0x7f4e4c004d00 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c00bae0 con 0x7f4e600adf10 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.167+0000 7f4e5e59c640 1 --2- 172.21.15.115:0/4192537386 >> v2:172.21.15.142:3300/0 conn(0x7f4e6015e2f0 0x7f4e600ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.168+0000 7f4e57fff640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 0x7f4e4c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.168+0000 7f4e57fff640 1 -- 172.21.15.115:0/4192537386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600adf10 msgr2=0x7f4e4c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.168+0000 7f4e57fff640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600adf10 0x7f4e4c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.168+0000 7f4e5ed9d640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4c004d00 0x7f4e4c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.168+0000 7f4e57fff640 1 -- 172.21.15.115:0/4192537386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4c004d00 msgr2=0x7f4e4c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.168+0000 7f4e57fff640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e4c004d00 0x7f4e4c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.168+0000 7f4e57fff640 1 -- 172.21.15.115:0/4192537386 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c003520 con 0x7f4e4c003f60 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.169+0000 7f4e57fff640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 0x7f4e4c0047c0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800c210 tx=0x7f4e4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142755bcf43ed5f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:16.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.169+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48005ad0 con 0x7f4e4c003f60 2023-12-01T02:52:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.169+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e48005c30 con 0x7f4e4c003f60 2023-12-01T02:52:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.169+0000 7f4e5dd9b640 1 -- 172.21.15.115:0/4192537386 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4802c8b0 con 0x7f4e4c003f60 2023-12-01T02:52:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.170+0000 7f4e64c15640 1 -- 172.21.15.115:0/4192537386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 msgr2=0x7f4e4c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.170+0000 7f4e64c15640 1 --2- 172.21.15.115:0/4192537386 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 0x7f4e4c0047c0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800c210 tx=0x7f4e4c0057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.170+0000 7f4e5ed9d640 1 -- 172.21.15.115:0/4192537386 reap_dead start 2023-12-01T02:52:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.170+0000 7f4e64c15640 1 -- 172.21.15.115:0/4192537386 shutdown_connections 2023-12-01T02:52:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.170+0000 7f4e64c15640 1 -- 172.21.15.115:0/4192537386 >> 172.21.15.115:0/4192537386 conn(0x7f4e600ce880 msgr2=0x7f4e600cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.170+0000 7f4e64c15640 1 -- 172.21.15.115:0/4192537386 shutdown_connections 2023-12-01T02:52:16.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.170+0000 7f4e64c15640 1 -- 172.21.15.115:0/4192537386 wait complete. 2023-12-01T02:52:16.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.172+0000 7f4e64c15640 1 Processor -- start 2023-12-01T02:52:16.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.172+0000 7f4e64c15640 1 -- start start 2023-12-01T02:52:16.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.173+0000 7f4e64c15640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 0x7f4e60157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.173+0000 7f4e64c15640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e600cb2f0 0x7f4e600c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.173+0000 7f4e64c15640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600cd0f0 0x7f4e600c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.173+0000 7f4e64c15640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600cccd0 con 0x7f4e4c003f60 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.173+0000 7f4e64c15640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600c5000 con 0x7f4e600cb2f0 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.173+0000 7f4e64c15640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e600cc610 con 0x7f4e600cd0f0 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.173+0000 7f4e5e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 0x7f4e60157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.173+0000 7f4e5e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 0x7f4e60157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43456/0 (socket says 172.21.15.115:43456) 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.173+0000 7f4e5e59c640 1 -- 172.21.15.115:0/110796947 learned_addr learned my addr 172.21.15.115:0/110796947 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e57fff640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e600cb2f0 0x7f4e600c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e5e59c640 1 -- 172.21.15.115:0/110796947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600cd0f0 msgr2=0x7f4e600c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e5ed9d640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600cd0f0 0x7f4e600c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e5e59c640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600cd0f0 0x7f4e600c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e5e59c640 1 -- 172.21.15.115:0/110796947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e600cb2f0 msgr2=0x7f4e600c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e5e59c640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e600cb2f0 0x7f4e600c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e5e59c640 1 -- 172.21.15.115:0/110796947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600cc8f0 con 0x7f4e4c003f60 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e57fff640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e600cb2f0 0x7f4e600c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e5e59c640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 0x7f4e60157d40 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000eb60 tx=0x7f4e5000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:16.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e56ffd640 1 -- 172.21.15.115:0/110796947 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50009750 con 0x7f4e4c003f60 2023-12-01T02:52:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e56ffd640 1 -- 172.21.15.115:0/110796947 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5000ce40 con 0x7f4e4c003f60 2023-12-01T02:52:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e56ffd640 1 -- 172.21.15.115:0/110796947 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000ee90 con 0x7f4e4c003f60 2023-12-01T02:52:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.174+0000 7f4e64c15640 1 -- 172.21.15.115:0/110796947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e60158280 con 0x7f4e4c003f60 2023-12-01T02:52:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.175+0000 7f4e64c15640 1 -- 172.21.15.115:0/110796947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e600c4dd0 con 0x7f4e4c003f60 2023-12-01T02:52:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.176+0000 7f4e64c15640 1 -- 172.21.15.115:0/110796947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6006fda0 con 0x7f4e4c003f60 2023-12-01T02:52:16.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.177+0000 7f4e56ffd640 1 -- 172.21.15.115:0/110796947 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4e50013470 con 0x7f4e4c003f60 2023-12-01T02:52:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.178+0000 7f4e56ffd640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e4006a550 0x7f4e4006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.178+0000 7f4e56ffd640 1 -- 172.21.15.115:0/110796947 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f4e50089150 con 0x7f4e4c003f60 2023-12-01T02:52:16.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.178+0000 7f4e57fff640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e4006a550 0x7f4e4006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.179+0000 7f4e57fff640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e4006a550 0x7f4e4006ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e48034000 tx=0x7f4e48037040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:16.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.182+0000 7f4e56ffd640 1 -- 172.21.15.115:0/110796947 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4e50054690 con 0x7f4e4c003f60 2023-12-01T02:52:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.408+0000 7f4e64c15640 1 -- 172.21.15.115:0/110796947 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e600cc450 con 0x7f4e4c003f60 2023-12-01T02:52:16.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.409+0000 7f4e56ffd640 1 -- 172.21.15.115:0/110796947 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f4e50017df0 con 0x7f4e4c003f60 2023-12-01T02:52:16.411 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:16.411 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/1952979218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1952979218},{"type":"v1","addr":"172.21.15.142:6833","nonce":1952979218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":5993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/3618529680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":3618529680},{"type":"v1","addr":"172.21.15.142:6835","nonce":3618529680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":9009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/1637880399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1637880399},{"type":"v1","addr":"172.21.15.115:6836","nonce":1637880399}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":9459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2363778158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":2363778158},{"type":"v1","addr":"172.21.15.115:6837","nonce":2363778158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[]} 2023-12-01T02:52:16.411 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 73 2023-12-01T02:52:16.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 -- 172.21.15.115:0/110796947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e4006a550 msgr2=0x7f4e4006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:16.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e4006a550 0x7f4e4006ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e48034000 tx=0x7f4e48037040 comp rx=0 tx=0).stop 2023-12-01T02:52:16.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 -- 172.21.15.115:0/110796947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 msgr2=0x7f4e60157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:16.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 0x7f4e60157d40 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000eb60 tx=0x7f4e5000c520 comp rx=0 tx=0).stop 2023-12-01T02:52:16.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 -- 172.21.15.115:0/110796947 shutdown_connections 2023-12-01T02:52:16.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f4e4006a550 0x7f4e4006ca10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e600cd0f0 0x7f4e600c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e600cb2f0 0x7f4e600c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 --2- 172.21.15.115:0/110796947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e4c003f60 0x7f4e60157d40 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 -- 172.21.15.115:0/110796947 >> 172.21.15.115:0/110796947 conn(0x7f4e600ce880 msgr2=0x7f4e6015a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:16.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.412+0000 7f4e64c15640 1 -- 172.21.15.115:0/110796947 shutdown_connections 2023-12-01T02:52:16.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.413+0000 7f4e64c15640 1 -- 172.21.15.115:0/110796947 wait complete. 2023-12-01T02:52:16.434 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T02:52:16.435 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T02:52:16.435 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T02:52:16.435 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T02:52:16.435 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T02:52:16.436 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T02:52:16.436 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T02:52:16.436 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T02:52:16.436 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T02:52:16.437 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T02:52:16.437 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T02:52:16.437 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T02:52:16.437 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T02:52:16.438 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T02:52:16.438 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T02:52:16.438 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T02:52:16.453 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T02:52:16.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T02:52:16.456 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T02:52:16.457 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T02:52:16.459 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T02:52:16.459 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T02:52:16.461 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T02:52:16.461 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T02:52:16.506 INFO:tasks.ceph.mds.a:Started 2023-12-01T02:52:16.507 DEBUG:teuthology.parallel:result is None 2023-12-01T02:52:16.510 INFO:tasks.ceph.mds.d:Started 2023-12-01T02:52:16.511 DEBUG:teuthology.parallel:result is None 2023-12-01T02:52:16.514 INFO:tasks.ceph.mds.c:Started 2023-12-01T02:52:16.514 DEBUG:teuthology.parallel:result is None 2023-12-01T02:52:16.518 INFO:tasks.ceph.mds.b:Started 2023-12-01T02:52:16.518 DEBUG:teuthology.parallel:result is None 2023-12-01T02:52:16.518 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T02:52:16.668 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T02:52:16.673 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T02:52:16.674 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T02:52:16.675 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T02:52:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.710+0000 7f02905fa640 1 Processor -- start 2023-12-01T02:52:16.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.710+0000 7f02905fa640 1 -- start start 2023-12-01T02:52:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.711+0000 7f02905fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 0x7f02880ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.711+0000 7f02905fa640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f028806f2f0 0x7f02880caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.711+0000 7f02905fa640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02880add10 con 0x7f02880adf10 2023-12-01T02:52:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.711+0000 7f02905fa640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02880c5760 con 0x7f028806f2f0 2023-12-01T02:52:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.711+0000 7f02905fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028815e6d0 con 0x7f028815e2f0 2023-12-01T02:52:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.711+0000 7f028db95640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f028806f2f0 0x7f02880caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.711+0000 7f028db95640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f028806f2f0 0x7f02880caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43508/0 (socket says 172.21.15.115:43508) 2023-12-01T02:52:16.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.711+0000 7f028e396640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 0x7f02880ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.711+0000 7f028db95640 1 -- 172.21.15.115:0/929152123 learned_addr learned my addr 172.21.15.115:0/929152123 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.711+0000 7f028eb97640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f02880adf10 0x7f02880c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60166/0 (socket says 172.21.15.115:60166) 2023-12-01T02:52:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.712+0000 7f028db95640 1 -- 172.21.15.115:0/929152123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 msgr2=0x7f02880ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.712+0000 7f028db95640 1 --2- 172.21.15.115:0/929152123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 0x7f02880ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.712+0000 7f028db95640 1 -- 172.21.15.115:0/929152123 >> v1:172.21.15.115:6789/0 conn(0x7f02880adf10 legacy=0x7f02880c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.712+0000 7f028db95640 1 -- 172.21.15.115:0/929152123 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02880cf7b0 con 0x7f028806f2f0 2023-12-01T02:52:16.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.713+0000 7f028db95640 1 --2- 172.21.15.115:0/929152123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 0x7f02880caba0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f0278009870 tx=0x7f027802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e57bdb3ab88af7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.713+0000 7f028d394640 1 -- 172.21.15.115:0/929152123 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02780317c0 con 0x7f028806f2f0 2023-12-01T02:52:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.713+0000 7f028d394640 1 -- 172.21.15.115:0/929152123 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0278031920 con 0x7f028806f2f0 2023-12-01T02:52:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.713+0000 7f028d394640 1 -- 172.21.15.115:0/929152123 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278031cb0 con 0x7f028806f2f0 2023-12-01T02:52:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.714+0000 7f02905fa640 1 -- 172.21.15.115:0/929152123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 msgr2=0x7f02880caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.714+0000 7f02905fa640 1 --2- 172.21.15.115:0/929152123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 0x7f02880caba0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f0278009870 tx=0x7f027802b860 comp rx=0 tx=0).stop 2023-12-01T02:52:16.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.714+0000 7f02905fa640 1 -- 172.21.15.115:0/929152123 shutdown_connections 2023-12-01T02:52:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.714+0000 7f02905fa640 1 --2- 172.21.15.115:0/929152123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 0x7f02880caba0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.714+0000 7f02905fa640 1 --2- 172.21.15.115:0/929152123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 0x7f02880ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.714+0000 7f02905fa640 1 -- 172.21.15.115:0/929152123 >> 172.21.15.115:0/929152123 conn(0x7f02880ce880 msgr2=0x7f02880cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.714+0000 7f02905fa640 1 -- 172.21.15.115:0/929152123 shutdown_connections 2023-12-01T02:52:16.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.714+0000 7f02905fa640 1 -- 172.21.15.115:0/929152123 wait complete. 2023-12-01T02:52:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.716+0000 7f02905fa640 1 Processor -- start 2023-12-01T02:52:16.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.716+0000 7f02905fa640 1 -- start start 2023-12-01T02:52:16.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f02905fa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 0x7f02880c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f02905fa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02880adf10 0x7f02880cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f02905fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 0x7f028814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f02905fa640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02880cd200 con 0x7f028806f2f0 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f02905fa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02880cc5f0 con 0x7f02880adf10 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f02905fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02880cc8d0 con 0x7f028815e2f0 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028e396640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 0x7f02880c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028e396640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 0x7f02880c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:43512/0 (socket says 172.21.15.115:43512) 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028e396640 1 -- 172.21.15.115:0/1479916093 learned_addr learned my addr 172.21.15.115:0/1479916093 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028eb97640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 0x7f028814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028db95640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02880adf10 0x7f02880cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028e396640 1 -- 172.21.15.115:0/1479916093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 msgr2=0x7f028814cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028e396640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 0x7f028814cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028e396640 1 -- 172.21.15.115:0/1479916093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02880adf10 msgr2=0x7f02880cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:16.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028e396640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02880adf10 0x7f02880cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028e396640 1 -- 172.21.15.115:0/1479916093 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780094d0 con 0x7f028806f2f0 2023-12-01T02:52:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028db95640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02880adf10 0x7f02880cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.717+0000 7f028eb97640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 0x7f028814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.718+0000 7f028e396640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 0x7f02880c8140 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f028400e870 tx=0x7f028400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.718+0000 7f027effd640 1 -- 172.21.15.115:0/1479916093 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02840094f0 con 0x7f028806f2f0 2023-12-01T02:52:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.718+0000 7f027effd640 1 -- 172.21.15.115:0/1479916093 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028400c940 con 0x7f028806f2f0 2023-12-01T02:52:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.718+0000 7f027effd640 1 -- 172.21.15.115:0/1479916093 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028400ccf0 con 0x7f028806f2f0 2023-12-01T02:52:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.718+0000 7f02905fa640 1 -- 172.21.15.115:0/1479916093 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02880ccbb0 con 0x7f028806f2f0 2023-12-01T02:52:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.718+0000 7f02905fa640 1 -- 172.21.15.115:0/1479916093 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02880c8680 con 0x7f028806f2f0 2023-12-01T02:52:16.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.719+0000 7f02905fa640 1 -- 172.21.15.115:0/1479916093 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028806fda0 con 0x7f028806f2f0 2023-12-01T02:52:16.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.720+0000 7f027effd640 1 -- 172.21.15.115:0/1479916093 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0284013470 con 0x7f028806f2f0 2023-12-01T02:52:16.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.721+0000 7f027effd640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f026c06a550 0x7f026c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:16.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.721+0000 7f027effd640 1 -- 172.21.15.115:0/1479916093 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f0284088b50 con 0x7f028806f2f0 2023-12-01T02:52:16.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.721+0000 7f028db95640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f026c06a550 0x7f026c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:16.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.722+0000 7f028db95640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f026c06a550 0x7f026c06ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0278036000 tx=0x7f027803d040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:16.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.725+0000 7f027effd640 1 -- 172.21.15.115:0/1479916093 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f02840540a0 con 0x7f028806f2f0 2023-12-01T02:52:16.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:16.928+0000 7f02905fa640 1 -- 172.21.15.115:0/1479916093 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f02880b43d0 con 0x7f028806f2f0 2023-12-01T02:52:17.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.720+0000 7f027effd640 1 -- 172.21.15.115:0/1479916093 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v173) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0284021080 con 0x7f028806f2f0 2023-12-01T02:52:17.722 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T02:52:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 -- 172.21.15.115:0/1479916093 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f026c06a550 msgr2=0x7f026c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f026c06a550 0x7f026c06ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0278036000 tx=0x7f027803d040 comp rx=0 tx=0).stop 2023-12-01T02:52:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 -- 172.21.15.115:0/1479916093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 msgr2=0x7f02880c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 0x7f02880c8140 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f028400e870 tx=0x7f028400ed30 comp rx=0 tx=0).stop 2023-12-01T02:52:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 -- 172.21.15.115:0/1479916093 shutdown_connections 2023-12-01T02:52:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f026c06a550 0x7f026c06ca10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f028815e2f0 0x7f028814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02880adf10 0x7f02880cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:17.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 --2- 172.21.15.115:0/1479916093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f028806f2f0 0x7f02880c8140 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:17.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 -- 172.21.15.115:0/1479916093 >> 172.21.15.115:0/1479916093 conn(0x7f02880ce880 msgr2=0x7f028806b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:17.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 -- 172.21.15.115:0/1479916093 shutdown_connections 2023-12-01T02:52:17.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.724+0000 7f02905fa640 1 -- 172.21.15.115:0/1479916093 wait complete. 2023-12-01T02:52:17.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T02:52:17.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.960+0000 7f332c7b9640 1 Processor -- start 2023-12-01T02:52:17.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.960+0000 7f332c7b9640 1 -- start start 2023-12-01T02:52:17.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.961+0000 7f332c7b9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240cad70 0x7f332406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:17.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.962+0000 7f332c7b9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33240add10 con 0x7f332415e2f0 2023-12-01T02:52:17.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.962+0000 7f332c7b9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33240c5760 con 0x7f332406f2f0 2023-12-01T02:52:17.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.962+0000 7f332c7b9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332415e6d0 con 0x7f33240cad70 2023-12-01T02:52:17.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.962+0000 7f332a555640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f332415e2f0 0x7f33240ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54014/0 (socket says 172.21.15.115:54014) 2023-12-01T02:52:17.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.962+0000 7f332a555640 1 -- 172.21.15.115:0/3063267481 learned_addr learned my addr 172.21.15.115:0/3063267481 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.962+0000 7f3329d54640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240cad70 0x7f332406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.962+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865777527 0 0) 0x7f33240add10 con 0x7f332415e2f0 2023-12-01T02:52:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.962+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3310003520 con 0x7f332415e2f0 2023-12-01T02:52:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1855598642 0 0) 0x7f3310003520 con 0x7f332415e2f0 2023-12-01T02:52:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240cad70 msgr2=0x7f332406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240cad70 0x7f332406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 >> v1:172.21.15.142:6789/0 conn(0x7f332406f2f0 legacy=0x7f332406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:17.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33240cf7b0 con 0x7f332415e2f0 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3320001670 con 0x7f332415e2f0 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 >> v1:172.21.15.115:6789/0 conn(0x7f332415e2f0 legacy=0x7f33240ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3310003a10 0x7f3310003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 0x7f3310004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240cad70 0x7f331000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332415e6d0 con 0x7f33100047c0 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3310005500 con 0x7f3310003a10 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331000bec0 con 0x7f33240cad70 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f3329d54640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 0x7f3310004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f332a555640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3310003a10 0x7f3310003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.963+0000 7f332ad56640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240cad70 0x7f331000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.964+0000 7f3329d54640 1 -- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240cad70 msgr2=0x7f331000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.964+0000 7f3329d54640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240cad70 0x7f331000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.964+0000 7f3329d54640 1 -- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3310003a10 msgr2=0x7f3310003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:17.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.964+0000 7f3329d54640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3310003a10 0x7f3310003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.964+0000 7f3329d54640 1 -- 172.21.15.115:0/3063267481 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3310003520 con 0x7f33100047c0 2023-12-01T02:52:17.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.964+0000 7f332ad56640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240cad70 0x7f331000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.965+0000 7f3329d54640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 0x7f3310004c40 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f3314009ac0 tx=0x7f33100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69670ec5345ac84e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.965+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331402c6b0 con 0x7f33100047c0 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.965+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331402c810 con 0x7f33100047c0 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.965+0000 7f3329553640 1 -- 172.21.15.115:0/3063267481 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331402cba0 con 0x7f33100047c0 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.965+0000 7f332c7b9640 1 -- 172.21.15.115:0/3063267481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 msgr2=0x7f3310004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.965+0000 7f332c7b9640 1 --2- 172.21.15.115:0/3063267481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 0x7f3310004c40 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f3314009ac0 tx=0x7f33100057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.966+0000 7f332ad56640 1 -- 172.21.15.115:0/3063267481 reap_dead start 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.966+0000 7f332c7b9640 1 -- 172.21.15.115:0/3063267481 shutdown_connections 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.966+0000 7f332c7b9640 1 -- 172.21.15.115:0/3063267481 >> 172.21.15.115:0/3063267481 conn(0x7f33240ce880 msgr2=0x7f33240cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.966+0000 7f332c7b9640 1 -- 172.21.15.115:0/3063267481 shutdown_connections 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.966+0000 7f332c7b9640 1 -- 172.21.15.115:0/3063267481 wait complete. 2023-12-01T02:52:17.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.967+0000 7f332c7b9640 1 Processor -- start 2023-12-01T02:52:17.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.968+0000 7f332c7b9640 1 -- start start 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.968+0000 7f332c7b9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 0x7f33240cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.968+0000 7f332c7b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33241f6840 0x7f33240c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.968+0000 7f332c7b9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240c28f0 0x7f332415a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332c7b9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33240c3200 con 0x7f33100047c0 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332c7b9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33241f63b0 con 0x7f33241f6840 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332c7b9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33240c6530 con 0x7f33240c28f0 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332a555640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 0x7f33240cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332a555640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 0x7f33240cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55568/0 (socket says 172.21.15.115:55568) 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332a555640 1 -- 172.21.15.115:0/3567690982 learned_addr learned my addr 172.21.15.115:0/3567690982 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332a555640 1 -- 172.21.15.115:0/3567690982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240c28f0 msgr2=0x7f332415a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332a555640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240c28f0 0x7f332415a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332a555640 1 -- 172.21.15.115:0/3567690982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33241f6840 msgr2=0x7f33240c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:17.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332a555640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33241f6840 0x7f33240c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332a555640 1 -- 172.21.15.115:0/3567690982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33240c2f20 con 0x7f33100047c0 2023-12-01T02:52:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.969+0000 7f332a555640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 0x7f33240cdb90 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f332000e7b0 tx=0x7f332000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.970+0000 7f331affd640 1 -- 172.21.15.115:0/3567690982 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332000eec0 con 0x7f33100047c0 2023-12-01T02:52:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.970+0000 7f331affd640 1 -- 172.21.15.115:0/3567690982 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332000c870 con 0x7f33100047c0 2023-12-01T02:52:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.970+0000 7f331affd640 1 -- 172.21.15.115:0/3567690982 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332000cc00 con 0x7f33100047c0 2023-12-01T02:52:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.970+0000 7f332c7b9640 1 -- 172.21.15.115:0/3567690982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33241f5d60 con 0x7f33100047c0 2023-12-01T02:52:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.970+0000 7f332c7b9640 1 -- 172.21.15.115:0/3567690982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33240ce0d0 con 0x7f33100047c0 2023-12-01T02:52:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.971+0000 7f332c7b9640 1 -- 172.21.15.115:0/3567690982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332406fda0 con 0x7f33100047c0 2023-12-01T02:52:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.972+0000 7f331affd640 1 -- 172.21.15.115:0/3567690982 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f332000cd60 con 0x7f33100047c0 2023-12-01T02:52:17.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.973+0000 7f331affd640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f330806a580 0x7f330806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:17.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.973+0000 7f331affd640 1 -- 172.21.15.115:0/3567690982 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f3320088650 con 0x7f33100047c0 2023-12-01T02:52:17.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.973+0000 7f3329d54640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f330806a580 0x7f330806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:17.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.974+0000 7f3329d54640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f330806a580 0x7f330806ca40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3314036000 tx=0x7f331402e000 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:17.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:17.977+0000 7f331affd640 1 -- 172.21.15.115:0/3567690982 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3320053ee0 con 0x7f33100047c0 2023-12-01T02:52:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.235+0000 7f332c7b9640 1 -- 172.21.15.115:0/3567690982 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f33240ce2f0 con 0x7f33100047c0 2023-12-01T02:52:18.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.237+0000 7f331affd640 1 -- 172.21.15.115:0/3567690982 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v40) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3320057500 con 0x7f33100047c0 2023-12-01T02:52:18.238 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:18.238 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T02:52:18.239 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T02:52:18.240 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:52:18.242 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T02:52:18.243 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:52:18.244 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.245 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T02:52:18.246 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.248 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T02:52:18.250 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:18.251 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T02:52:18.252 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T02:52:18.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.239+0000 7f332c7b9640 1 -- 172.21.15.115:0/3567690982 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f330806a580 msgr2=0x7f330806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.239+0000 7f332c7b9640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f330806a580 0x7f330806ca40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3314036000 tx=0x7f331402e000 comp rx=0 tx=0).stop 2023-12-01T02:52:18.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.239+0000 7f332c7b9640 1 -- 172.21.15.115:0/3567690982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 msgr2=0x7f33240cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.239+0000 7f332c7b9640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 0x7f33240cdb90 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f332000e7b0 tx=0x7f332000ec70 comp rx=0 tx=0).stop 2023-12-01T02:52:18.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.239+0000 7f332c7b9640 1 -- 172.21.15.115:0/3567690982 shutdown_connections 2023-12-01T02:52:18.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.239+0000 7f332c7b9640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f330806a580 0x7f330806ca40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.239+0000 7f332c7b9640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33240c28f0 0x7f332415a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.239+0000 7f332c7b9640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f33241f6840 0x7f33240c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.239+0000 7f332c7b9640 1 --2- 172.21.15.115:0/3567690982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33100047c0 0x7f33240cdb90 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.239+0000 7f332c7b9640 1 -- 172.21.15.115:0/3567690982 >> 172.21.15.115:0/3567690982 conn(0x7f33240ce880 msgr2=0x7f332415cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:18.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.240+0000 7f332c7b9640 1 -- 172.21.15.115:0/3567690982 shutdown_connections 2023-12-01T02:52:18.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.240+0000 7f332c7b9640 1 -- 172.21.15.115:0/3567690982 wait complete. 2023-12-01T02:52:18.261 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:52:18.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.473+0000 7f830b59e640 1 Processor -- start 2023-12-01T02:52:18.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.473+0000 7f830b59e640 1 -- start start 2023-12-01T02:52:18.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.475+0000 7f830b59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c145fa0 0x7f830c065960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.475+0000 7f830b59e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c1609a0 con 0x7f830c071e30 2023-12-01T02:52:18.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.475+0000 7f830b59e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c160c80 con 0x7f830c065ea0 2023-12-01T02:52:18.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.475+0000 7f830b59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c161620 con 0x7f830c145fa0 2023-12-01T02:52:18.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.475+0000 7f830a59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f830c071e30 0x7f830c0b2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54026/0 (socket says 172.21.15.115:54026) 2023-12-01T02:52:18.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.475+0000 7f830a59c640 1 -- 172.21.15.115:0/1502774310 learned_addr learned my addr 172.21.15.115:0/1502774310 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.475+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197950179 0 0) 0x7f830c1609a0 con 0x7f830c071e30 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.475+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82f8003520 con 0x7f830c071e30 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8301d9b640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c145fa0 0x7f830c065960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2435696275 0 0) 0x7f82f8003520 con 0x7f830c071e30 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c145fa0 msgr2=0x7f830c065960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c145fa0 0x7f830c065960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 >> v1:172.21.15.142:6789/0 conn(0x7f830c065ea0 legacy=0x7f830c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c160f60 con 0x7f830c071e30 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f82fc0019f0 con 0x7f830c071e30 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8301d9b640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c145fa0 0x7f830c065960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T02:52:18.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 >> v1:172.21.15.115:6789/0 conn(0x7f830c071e30 legacy=0x7f830c0b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f8003a10 0x7f82f8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f80047c0 0x7f82f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c145fa0 0x7f82f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c161620 con 0x7f82f80047c0 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80054a0 con 0x7f82f8003a10 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f8005660 con 0x7f830c145fa0 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.476+0000 7f8301d9b640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f80047c0 0x7f82f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.477+0000 7f830a59c640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f8003a10 0x7f82f8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.477+0000 7f8301d9b640 1 -- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c145fa0 msgr2=0x7f82f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.477+0000 7f830ad9d640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c145fa0 0x7f82f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.477+0000 7f8301d9b640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c145fa0 0x7f82f800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.477+0000 7f8301d9b640 1 -- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f8003a10 msgr2=0x7f82f8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.477+0000 7f8301d9b640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f8003a10 0x7f82f8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.477+0000 7f8301d9b640 1 -- 172.21.15.115:0/1502774310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f8003520 con 0x7f82f80047c0 2023-12-01T02:52:18.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.477+0000 7f830a59c640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82f8003a10 0x7f82f8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:18.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.477+0000 7f830ad9d640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f830c145fa0 0x7f82f800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.478+0000 7f8301d9b640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f80047c0 0x7f82f8004c40 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f830c0d0640 tx=0x7f82f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb42ab4f3f2cba4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.478+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4030400 con 0x7f82f80047c0 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.478+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4030560 con 0x7f82f80047c0 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.478+0000 7f8309d9b640 1 -- 172.21.15.115:0/1502774310 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40308d0 con 0x7f82f80047c0 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.479+0000 7f830b59e640 1 -- 172.21.15.115:0/1502774310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f80047c0 msgr2=0x7f82f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.479+0000 7f830b59e640 1 --2- 172.21.15.115:0/1502774310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82f80047c0 0x7f82f8004c40 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f830c0d0640 tx=0x7f82f80057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.479+0000 7f830ad9d640 1 -- 172.21.15.115:0/1502774310 reap_dead start 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.479+0000 7f830b59e640 1 -- 172.21.15.115:0/1502774310 shutdown_connections 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.479+0000 7f830b59e640 1 -- 172.21.15.115:0/1502774310 >> 172.21.15.115:0/1502774310 conn(0x7f830c073330 msgr2=0x7f830c069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.479+0000 7f830b59e640 1 -- 172.21.15.115:0/1502774310 shutdown_connections 2023-12-01T02:52:18.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.479+0000 7f830b59e640 1 -- 172.21.15.115:0/1502774310 wait complete. 2023-12-01T02:52:18.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.480+0000 7f830b59e640 1 Processor -- start 2023-12-01T02:52:18.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.481+0000 7f830b59e640 1 -- start start 2023-12-01T02:52:18.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.481+0000 7f830b59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82f80047c0 0x7f830c13e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.481+0000 7f830b59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c13ef00 0x7f830c15d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830b59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f830c15d8b0 0x7f830c15dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830b59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c066350 con 0x7f830c15d8b0 2023-12-01T02:52:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830b59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c1e66d0 con 0x7f830c13ef00 2023-12-01T02:52:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830b59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c13ed30 con 0x7f82f80047c0 2023-12-01T02:52:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f830c15d8b0 0x7f830c15dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f830c15d8b0 0x7f830c15dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55586/0 (socket says 172.21.15.115:55586) 2023-12-01T02:52:18.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830ad9d640 1 -- 172.21.15.115:0/1291003229 learned_addr learned my addr 172.21.15.115:0/1291003229 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830ad9d640 1 -- 172.21.15.115:0/1291003229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82f80047c0 msgr2=0x7f830c13e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830ad9d640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82f80047c0 0x7f830c13e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830ad9d640 1 -- 172.21.15.115:0/1291003229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c13ef00 msgr2=0x7f830c15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830ad9d640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c13ef00 0x7f830c15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830ad9d640 1 -- 172.21.15.115:0/1291003229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c0b1b10 con 0x7f830c15d8b0 2023-12-01T02:52:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.482+0000 7f830ad9d640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f830c15d8b0 0x7f830c15dd30 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f830c073800 tx=0x7f830400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:18.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.483+0000 7f83037fe640 1 -- 172.21.15.115:0/1291003229 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83040052d0 con 0x7f830c15d8b0 2023-12-01T02:52:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.483+0000 7f83037fe640 1 -- 172.21.15.115:0/1291003229 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8304005430 con 0x7f830c15d8b0 2023-12-01T02:52:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.483+0000 7f83037fe640 1 -- 172.21.15.115:0/1291003229 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304012840 con 0x7f830c15d8b0 2023-12-01T02:52:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.483+0000 7f830b59e640 1 -- 172.21.15.115:0/1291003229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c0b1df0 con 0x7f830c15d8b0 2023-12-01T02:52:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.483+0000 7f830b59e640 1 -- 172.21.15.115:0/1291003229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c1e6920 con 0x7f830c15d8b0 2023-12-01T02:52:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.484+0000 7f830b59e640 1 -- 172.21.15.115:0/1291003229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830c1e6b40 con 0x7f830c15d8b0 2023-12-01T02:52:18.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.485+0000 7f83037fe640 1 -- 172.21.15.115:0/1291003229 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f83040129a0 con 0x7f830c15d8b0 2023-12-01T02:52:18.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.486+0000 7f83037fe640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82ec06a580 0x7f82ec06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.486+0000 7f83037fe640 1 -- 172.21.15.115:0/1291003229 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f83040880e0 con 0x7f830c15d8b0 2023-12-01T02:52:18.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.486+0000 7f830a59c640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82ec06a580 0x7f82ec06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.487+0000 7f830a59c640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82ec06a580 0x7f82ec06ca40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f82fc001b20 tx=0x7f82fc009040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:18.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.490+0000 7f83037fe640 1 -- 172.21.15.115:0/1291003229 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f83040537c0 con 0x7f830c15d8b0 2023-12-01T02:52:18.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.679+0000 7f830b59e640 1 -- 172.21.15.115:0/1291003229 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f830c0673a0 con 0x7f830c15d8b0 2023-12-01T02:52:18.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.681+0000 7f83037fe640 1 -- 172.21.15.115:0/1291003229 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+12517 (secure 0 0 0) 0x7f8304056de0 con 0x7f830c15d8b0 2023-12-01T02:52:18.683 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:18.683 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":173,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:52:17.712049+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:52:18.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.683+0000 7f830b59e640 1 -- 172.21.15.115:0/1291003229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82ec06a580 msgr2=0x7f82ec06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.683+0000 7f830b59e640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82ec06a580 0x7f82ec06ca40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f82fc001b20 tx=0x7f82fc009040 comp rx=0 tx=0).stop 2023-12-01T02:52:18.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.683+0000 7f830b59e640 1 -- 172.21.15.115:0/1291003229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f830c15d8b0 msgr2=0x7f830c15dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.683+0000 7f830b59e640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f830c15d8b0 0x7f830c15dd30 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f830c073800 tx=0x7f830400d890 comp rx=0 tx=0).stop 2023-12-01T02:52:18.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.684+0000 7f830b59e640 1 -- 172.21.15.115:0/1291003229 shutdown_connections 2023-12-01T02:52:18.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.684+0000 7f830b59e640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f82ec06a580 0x7f82ec06ca40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.684+0000 7f830b59e640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f830c15d8b0 0x7f830c15dd30 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.684+0000 7f830b59e640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f830c13ef00 0x7f830c15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.684+0000 7f830b59e640 1 --2- 172.21.15.115:0/1291003229 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82f80047c0 0x7f830c13e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.684+0000 7f830b59e640 1 -- 172.21.15.115:0/1291003229 >> 172.21.15.115:0/1291003229 conn(0x7f830c073330 msgr2=0x7f830c0cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:18.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.684+0000 7f830b59e640 1 -- 172.21.15.115:0/1291003229 shutdown_connections 2023-12-01T02:52:18.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.684+0000 7f830b59e640 1 -- 172.21.15.115:0/1291003229 wait complete. 2023-12-01T02:52:18.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T02:52:18.705 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T02:52:18.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.911+0000 7efe7782c640 1 Processor -- start 2023-12-01T02:52:18.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.911+0000 7efe7782c640 1 -- start start 2023-12-01T02:52:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.913+0000 7efe7782c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7015ac80 0x7efe700cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.913+0000 7efe7782c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efe700b0e60 0x7efe700b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.913+0000 7efe7782c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efe700cdaa0 0x7efe700cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.913+0000 7efe7782c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7015b060 con 0x7efe700b0e60 2023-12-01T02:52:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.913+0000 7efe7782c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe700b0ce0 con 0x7efe700cdaa0 2023-12-01T02:52:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.913+0000 7efe7782c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe700ce470 con 0x7efe7015ac80 2023-12-01T02:52:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.913+0000 7efe74dc7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efe700b0e60 0x7efe700b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.913+0000 7efe74dc7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efe700b0e60 0x7efe700b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55592/0 (socket says 172.21.15.115:55592) 2023-12-01T02:52:18.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.914+0000 7efe74dc7640 1 -- 172.21.15.115:0/2126325658 learned_addr learned my addr 172.21.15.115:0/2126325658 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:18.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.914+0000 7efe74dc7640 1 -- 172.21.15.115:0/2126325658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7015ac80 msgr2=0x7efe700cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:18.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.914+0000 7efe755c8640 1 --2- 172.21.15.115:0/2126325658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7015ac80 0x7efe700cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.914+0000 7efe75dc9640 1 --2- 172.21.15.115:0/2126325658 >> v2:172.21.15.142:3300/0 conn(0x7efe700cdaa0 0x7efe700cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.914+0000 7efe74dc7640 1 --2- 172.21.15.115:0/2126325658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7015ac80 0x7efe700cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.914+0000 7efe74dc7640 1 -- 172.21.15.115:0/2126325658 >> v2:172.21.15.142:3300/0 conn(0x7efe700cdaa0 msgr2=0x7efe700cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.914+0000 7efe74dc7640 1 --2- 172.21.15.115:0/2126325658 >> v2:172.21.15.142:3300/0 conn(0x7efe700cdaa0 0x7efe700cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.914+0000 7efe74dc7640 1 -- 172.21.15.115:0/2126325658 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700cc9a0 con 0x7efe700b0e60 2023-12-01T02:52:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.914+0000 7efe755c8640 1 --2- 172.21.15.115:0/2126325658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7015ac80 0x7efe700cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.914+0000 7efe75dc9640 1 --2- 172.21.15.115:0/2126325658 >> v2:172.21.15.142:3300/0 conn(0x7efe700cdaa0 0x7efe700cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.915+0000 7efe74dc7640 1 --2- 172.21.15.115:0/2126325658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700b0e60 0x7efe700b07a0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7efe7006d710 tx=0x7efe6002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5486bca0e17a46d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:18.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.915+0000 7efe67fff640 1 -- 172.21.15.115:0/2126325658 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60031880 con 0x7efe700b0e60 2023-12-01T02:52:18.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.915+0000 7efe67fff640 1 -- 172.21.15.115:0/2126325658 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe600319e0 con 0x7efe700b0e60 2023-12-01T02:52:18.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.915+0000 7efe67fff640 1 -- 172.21.15.115:0/2126325658 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60031d70 con 0x7efe700b0e60 2023-12-01T02:52:18.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.916+0000 7efe7782c640 1 -- 172.21.15.115:0/2126325658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700b0e60 msgr2=0x7efe700b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.916+0000 7efe7782c640 1 --2- 172.21.15.115:0/2126325658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700b0e60 0x7efe700b07a0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7efe7006d710 tx=0x7efe6002b920 comp rx=0 tx=0).stop 2023-12-01T02:52:18.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.916+0000 7efe7782c640 1 -- 172.21.15.115:0/2126325658 shutdown_connections 2023-12-01T02:52:18.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.916+0000 7efe7782c640 1 --2- 172.21.15.115:0/2126325658 >> v2:172.21.15.142:3300/0 conn(0x7efe700cdaa0 0x7efe700cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.916+0000 7efe7782c640 1 --2- 172.21.15.115:0/2126325658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700b0e60 0x7efe700b07a0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.916+0000 7efe7782c640 1 --2- 172.21.15.115:0/2126325658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7015ac80 0x7efe700cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.916+0000 7efe7782c640 1 -- 172.21.15.115:0/2126325658 >> 172.21.15.115:0/2126325658 conn(0x7efe70065960 msgr2=0x7efe70065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:18.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.916+0000 7efe7782c640 1 -- 172.21.15.115:0/2126325658 shutdown_connections 2023-12-01T02:52:18.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.916+0000 7efe7782c640 1 -- 172.21.15.115:0/2126325658 wait complete. 2023-12-01T02:52:18.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.918+0000 7efe7782c640 1 Processor -- start 2023-12-01T02:52:18.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.918+0000 7efe7782c640 1 -- start start 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.919+0000 7efe7782c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe700b0e60 0x7efe70140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.919+0000 7efe7782c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cdaa0 0x7efe70153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.919+0000 7efe7782c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7015ac80 0x7efe701594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.919+0000 7efe7782c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe70140790 con 0x7efe700cdaa0 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.919+0000 7efe7782c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe701571f0 con 0x7efe700b0e60 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.919+0000 7efe7782c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe701574d0 con 0x7efe7015ac80 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.919+0000 7efe74dc7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cdaa0 0x7efe70153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.919+0000 7efe74dc7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cdaa0 0x7efe70153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55608/0 (socket says 172.21.15.115:55608) 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.919+0000 7efe74dc7640 1 -- 172.21.15.115:0/1273452413 learned_addr learned my addr 172.21.15.115:0/1273452413 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.920+0000 7efe755c8640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe700b0e60 0x7efe70140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.920+0000 7efe74dc7640 1 -- 172.21.15.115:0/1273452413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7015ac80 msgr2=0x7efe701594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.920+0000 7efe74dc7640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7015ac80 0x7efe701594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.920+0000 7efe74dc7640 1 -- 172.21.15.115:0/1273452413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe700b0e60 msgr2=0x7efe70140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.920+0000 7efe74dc7640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe700b0e60 0x7efe70140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.920+0000 7efe74dc7640 1 -- 172.21.15.115:0/1273452413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe600094d0 con 0x7efe700cdaa0 2023-12-01T02:52:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.920+0000 7efe755c8640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe700b0e60 0x7efe70140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.920+0000 7efe74dc7640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cdaa0 0x7efe70153250 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7efe6002b960 tx=0x7efe600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.920+0000 7efe65ffb640 1 -- 172.21.15.115:0/1273452413 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60031870 con 0x7efe700cdaa0 2023-12-01T02:52:18.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.921+0000 7efe65ffb640 1 -- 172.21.15.115:0/1273452413 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe600443f0 con 0x7efe700cdaa0 2023-12-01T02:52:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.921+0000 7efe65ffb640 1 -- 172.21.15.115:0/1273452413 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe600447a0 con 0x7efe700cdaa0 2023-12-01T02:52:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.921+0000 7efe7782c640 1 -- 172.21.15.115:0/1273452413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe701577b0 con 0x7efe700cdaa0 2023-12-01T02:52:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.921+0000 7efe7782c640 1 -- 172.21.15.115:0/1273452413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe70157d70 con 0x7efe700cdaa0 2023-12-01T02:52:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.922+0000 7efe7782c640 1 -- 172.21.15.115:0/1273452413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7004e890 con 0x7efe700cdaa0 2023-12-01T02:52:18.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.923+0000 7efe65ffb640 1 -- 172.21.15.115:0/1273452413 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7efe6004d450 con 0x7efe700cdaa0 2023-12-01T02:52:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.924+0000 7efe65ffb640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a550 0x7efe5406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:18.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.924+0000 7efe65ffb640 1 -- 172.21.15.115:0/1273452413 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4901+0+0 (secure 0 0 0) 0x7efe600ab160 con 0x7efe700cdaa0 2023-12-01T02:52:18.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.924+0000 7efe755c8640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a550 0x7efe5406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:18.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.924+0000 7efe755c8640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a550 0x7efe5406ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efe6c006fd0 tx=0x7efe6c008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:18.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:18.928+0000 7efe65ffb640 1 -- 172.21.15.115:0/1273452413 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efe600767b0 con 0x7efe700cdaa0 2023-12-01T02:52:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.115+0000 7efe7782c640 1 -- 172.21.15.115:0/1273452413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe7006fdf0 con 0x7efe700cdaa0 2023-12-01T02:52:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.116+0000 7efe65ffb640 1 -- 172.21.15.115:0/1273452413 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe60035020 con 0x7efe700cdaa0 2023-12-01T02:52:19.118 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T02:52:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.118+0000 7efe7782c640 1 -- 172.21.15.115:0/1273452413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a550 msgr2=0x7efe5406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.118+0000 7efe7782c640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a550 0x7efe5406ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efe6c006fd0 tx=0x7efe6c008040 comp rx=0 tx=0).stop 2023-12-01T02:52:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.118+0000 7efe7782c640 1 -- 172.21.15.115:0/1273452413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cdaa0 msgr2=0x7efe70153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.118+0000 7efe7782c640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cdaa0 0x7efe70153250 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7efe6002b960 tx=0x7efe600046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.119+0000 7efe7782c640 1 -- 172.21.15.115:0/1273452413 shutdown_connections 2023-12-01T02:52:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.119+0000 7efe7782c640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7efe5406a550 0x7efe5406ca10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.119+0000 7efe7782c640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7015ac80 0x7efe701594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.119+0000 7efe7782c640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe700cdaa0 0x7efe70153250 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.119+0000 7efe7782c640 1 --2- 172.21.15.115:0/1273452413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe700b0e60 0x7efe70140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.119+0000 7efe7782c640 1 -- 172.21.15.115:0/1273452413 >> 172.21.15.115:0/1273452413 conn(0x7efe70065960 msgr2=0x7efe70070310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.119+0000 7efe7782c640 1 -- 172.21.15.115:0/1273452413 shutdown_connections 2023-12-01T02:52:19.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.119+0000 7efe7782c640 1 -- 172.21.15.115:0/1273452413 wait complete. 2023-12-01T02:52:19.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T02:52:19.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.342+0000 7f17d6311640 1 Processor -- start 2023-12-01T02:52:19.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.342+0000 7f17d6311640 1 -- start start 2023-12-01T02:52:19.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.343+0000 7f17d6311640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f17d00cc670 0x7f17d00d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.343+0000 7f17d6311640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00b3670 con 0x7f17d015a910 2023-12-01T02:52:19.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.344+0000 7f17d6311640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00cf110 con 0x7f17d015ee10 2023-12-01T02:52:19.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.344+0000 7f17d6311640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d00cf3f0 con 0x7f17d00cc670 2023-12-01T02:52:19.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.344+0000 7f17d530f640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f17d015a910 0x7f17d015e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54040/0 (socket says 172.21.15.115:54040) 2023-12-01T02:52:19.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.344+0000 7f17d530f640 1 -- 172.21.15.115:0/3278723985 learned_addr learned my addr 172.21.15.115:0/3278723985 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:19.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.344+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926337826 0 0) 0x7f17d00b3670 con 0x7f17d015a910 2023-12-01T02:52:19.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.344+0000 7f17d5b10640 1 --2- 172.21.15.115:0/3278723985 >> v2:172.21.15.142:3300/0 conn(0x7f17d00cc670 0x7f17d00d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.344+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17bc003520 con 0x7f17d015a910 2023-12-01T02:52:19.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.344+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3383621692 0 0) 0x7f17bc003520 con 0x7f17d015a910 2023-12-01T02:52:19.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 >> v2:172.21.15.142:3300/0 conn(0x7f17d00cc670 msgr2=0x7f17d00d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:19.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 --2- 172.21.15.115:0/3278723985 >> v2:172.21.15.142:3300/0 conn(0x7f17d00cc670 0x7f17d00d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 >> v1:172.21.15.142:6789/0 conn(0x7f17d015ee10 legacy=0x7f17d00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:19.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00cf6d0 con 0x7f17d015a910 2023-12-01T02:52:19.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f17cc0017f0 con 0x7f17d015a910 2023-12-01T02:52:19.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 >> v1:172.21.15.115:6789/0 conn(0x7f17d015a910 legacy=0x7f17d015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17bc003a30 0x7f17bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17bc005180 0x7f17bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00cf3f0 con 0x7f17bc0047c0 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc00c120 con 0x7f17bc003a30 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc00c2e0 con 0x7f17bc005180 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17d4b0e640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17d530f640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17bc003a30 0x7f17bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17d5b10640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17bc005180 0x7f17bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17d4b0e640 1 -- 172.21.15.115:0/3278723985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17bc005180 msgr2=0x7f17bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17d4b0e640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17bc005180 0x7f17bc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17d4b0e640 1 -- 172.21.15.115:0/3278723985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17bc003a30 msgr2=0x7f17bc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:19.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.345+0000 7f17d4b0e640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17bc003a30 0x7f17bc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.346+0000 7f17d4b0e640 1 -- 172.21.15.115:0/3278723985 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc003520 con 0x7f17bc0047c0 2023-12-01T02:52:19.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.346+0000 7f17d530f640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17bc003a30 0x7f17bc003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:19.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.346+0000 7f17d5b10640 1 -- 172.21.15.115:0/3278723985 reap_dead start 2023-12-01T02:52:19.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.347+0000 7f17d4b0e640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17bc004c40 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f17c0003db0 tx=0x7f17bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1038757d97e6bc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:19.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.347+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0032600 con 0x7f17bc0047c0 2023-12-01T02:52:19.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.347+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c002e050 con 0x7f17bc0047c0 2023-12-01T02:52:19.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.347+0000 7f17c7fff640 1 -- 172.21.15.115:0/3278723985 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00329e0 con 0x7f17bc0047c0 2023-12-01T02:52:19.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.347+0000 7f17d6311640 1 -- 172.21.15.115:0/3278723985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 msgr2=0x7f17bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.347+0000 7f17d6311640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17bc004c40 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f17c0003db0 tx=0x7f17bc0057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.348+0000 7f17d6311640 1 -- 172.21.15.115:0/3278723985 shutdown_connections 2023-12-01T02:52:19.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.348+0000 7f17d6311640 1 --2- 172.21.15.115:0/3278723985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17bc004c40 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.348+0000 7f17d6311640 1 -- 172.21.15.115:0/3278723985 >> 172.21.15.115:0/3278723985 conn(0x7f17d0099a20 msgr2=0x7f17d00d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:19.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.348+0000 7f17d6311640 1 -- 172.21.15.115:0/3278723985 shutdown_connections 2023-12-01T02:52:19.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.348+0000 7f17d6311640 1 -- 172.21.15.115:0/3278723985 wait complete. 2023-12-01T02:52:19.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.349+0000 7f17d6311640 1 Processor -- start 2023-12-01T02:52:19.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.349+0000 7f17d6311640 1 -- start start 2023-12-01T02:52:19.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.350+0000 7f17d6311640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17d014eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.350+0000 7f17d6311640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d015c740 0x7f17d015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.350+0000 7f17d6311640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17d0157420 0x7f17d01578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.350+0000 7f17d6311640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d014f0b0 con 0x7f17bc0047c0 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.350+0000 7f17d6311640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d0157de0 con 0x7f17d0157420 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.350+0000 7f17d6311640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d015d7e0 con 0x7f17d015c740 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.351+0000 7f17d530f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17d014eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.351+0000 7f17d530f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17d014eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55624/0 (socket says 172.21.15.115:55624) 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.351+0000 7f17d530f640 1 -- 172.21.15.115:0/1638207360 learned_addr learned my addr 172.21.15.115:0/1638207360 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.351+0000 7f17d530f640 1 -- 172.21.15.115:0/1638207360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d015c740 msgr2=0x7f17d015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.351+0000 7f17d530f640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d015c740 0x7f17d015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.351+0000 7f17d530f640 1 -- 172.21.15.115:0/1638207360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17d0157420 msgr2=0x7f17d01578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.351+0000 7f17d530f640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17d0157420 0x7f17d01578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.351+0000 7f17d530f640 1 -- 172.21.15.115:0/1638207360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d015cbd0 con 0x7f17bc0047c0 2023-12-01T02:52:19.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.351+0000 7f17d530f640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17d014eb70 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00d4a0 tx=0x7f17cc00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:19.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.351+0000 7f17c5ffb640 1 -- 172.21.15.115:0/1638207360 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00b070 con 0x7f17bc0047c0 2023-12-01T02:52:19.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.352+0000 7f17c5ffb640 1 -- 172.21.15.115:0/1638207360 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc0074e0 con 0x7f17bc0047c0 2023-12-01T02:52:19.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.352+0000 7f17c5ffb640 1 -- 172.21.15.115:0/1638207360 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0059a0 con 0x7f17bc0047c0 2023-12-01T02:52:19.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.352+0000 7f17d6311640 1 -- 172.21.15.115:0/1638207360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d015ceb0 con 0x7f17bc0047c0 2023-12-01T02:52:19.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.352+0000 7f17d6311640 1 -- 172.21.15.115:0/1638207360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d015d470 con 0x7f17bc0047c0 2023-12-01T02:52:19.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.353+0000 7f17d6311640 1 -- 172.21.15.115:0/1638207360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d015d600 con 0x7f17bc0047c0 2023-12-01T02:52:19.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.354+0000 7f17c5ffb640 1 -- 172.21.15.115:0/1638207360 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f17cc005b00 con 0x7f17bc0047c0 2023-12-01T02:52:19.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.355+0000 7f17c5ffb640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f17b406a580 0x7f17b406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.355+0000 7f17c5ffb640 1 -- 172.21.15.115:0/1638207360 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f17cc088160 con 0x7f17bc0047c0 2023-12-01T02:52:19.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.355+0000 7f17d4b0e640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f17b406a580 0x7f17b406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.356+0000 7f17d4b0e640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f17b406a580 0x7f17b406ca40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f17c0034000 tx=0x7f17c002f000 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:19.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.359+0000 7f17c5ffb640 1 -- 172.21.15.115:0/1638207360 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f17d015d600 con 0x7f17bc0047c0 2023-12-01T02:52:19.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.551+0000 7f17d6311640 1 -- 172.21.15.115:0/1638207360 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f17d015d600 con 0x7f17bc0047c0 2023-12-01T02:52:19.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.552+0000 7f17c5ffb640 1 -- 172.21.15.115:0/1638207360 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v173) v1 ==== 84+0+65 (secure 0 0 0) 0x7f17cc018060 con 0x7f17bc0047c0 2023-12-01T02:52:19.554 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:19.554 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T02:52:19.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:19.554 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T02:52:19.554 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T02:52:19.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:19.555 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T02:52:19.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 -- 172.21.15.115:0/1638207360 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f17b406a580 msgr2=0x7f17b406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f17b406a580 0x7f17b406ca40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f17c0034000 tx=0x7f17c002f000 comp rx=0 tx=0).stop 2023-12-01T02:52:19.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 -- 172.21.15.115:0/1638207360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 msgr2=0x7f17d014eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17d014eb70 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00d4a0 tx=0x7f17cc00d960 comp rx=0 tx=0).stop 2023-12-01T02:52:19.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 -- 172.21.15.115:0/1638207360 shutdown_connections 2023-12-01T02:52:19.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f17b406a580 0x7f17b406ca40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d015c740 0x7f17d015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17d0157420 0x7f17d01578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 --2- 172.21.15.115:0/1638207360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17bc0047c0 0x7f17d014eb70 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 -- 172.21.15.115:0/1638207360 >> 172.21.15.115:0/1638207360 conn(0x7f17d0099a20 msgr2=0x7f17d00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:19.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 -- 172.21.15.115:0/1638207360 shutdown_connections 2023-12-01T02:52:19.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.555+0000 7f17d6311640 1 -- 172.21.15.115:0/1638207360 wait complete. 2023-12-01T02:52:19.575 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T02:52:19.575 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T02:52:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.784+0000 7f8190ab4640 1 Processor -- start 2023-12-01T02:52:19.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.785+0000 7f8190ab4640 1 -- start start 2023-12-01T02:52:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.786+0000 7f8190ab4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f818c153c30 0x7f818c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.786+0000 7f8190ab4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f818c0cfb80 0x7f818c160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.786+0000 7f8190ab4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c0cdcd0 con 0x7f818c154010 2023-12-01T02:52:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.786+0000 7f8190ab4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c161830 con 0x7f818c0cfb80 2023-12-01T02:52:19.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.786+0000 7f8190ab4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0cd3a0 con 0x7f818c153c30 2023-12-01T02:52:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.787+0000 7f818bfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f818c0cfb80 0x7f818c160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.787+0000 7f818bfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f818c0cfb80 0x7f818c160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:48228/0 (socket says 172.21.15.115:48228) 2023-12-01T02:52:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.787+0000 7f818bfff640 1 -- 172.21.15.115:0/105214881 learned_addr learned my addr 172.21.15.115:0/105214881 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.787+0000 7f818b7fe640 1 --2- 172.21.15.115:0/105214881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f818c153c30 0x7f818c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.788+0000 7f818bfff640 1 -- 172.21.15.115:0/105214881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f818c153c30 msgr2=0x7f818c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.788+0000 7f818bfff640 1 --2- 172.21.15.115:0/105214881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f818c153c30 0x7f818c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.788+0000 7f818bfff640 1 -- 172.21.15.115:0/105214881 >> v1:172.21.15.142:6789/0 conn(0x7f818c154010 legacy=0x7f818c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.788+0000 7f818bfff640 1 -- 172.21.15.115:0/105214881 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0cd680 con 0x7f818c0cfb80 2023-12-01T02:52:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.789+0000 7f818bfff640 1 --2- 172.21.15.115:0/105214881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f818c0cfb80 0x7f818c160ac0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8184009e50 tx=0x7f818402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20d25b03219bf4b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f818affd640 1 -- 172.21.15.115:0/105214881 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818402cb00 con 0x7f818c0cfb80 2023-12-01T02:52:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f818affd640 1 -- 172.21.15.115:0/105214881 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818402cc60 con 0x7f818c0cfb80 2023-12-01T02:52:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f818affd640 1 -- 172.21.15.115:0/105214881 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840343f0 con 0x7f818c0cfb80 2023-12-01T02:52:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f8190ab4640 1 -- 172.21.15.115:0/105214881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f818c0cfb80 msgr2=0x7f818c160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f8190ab4640 1 --2- 172.21.15.115:0/105214881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f818c0cfb80 0x7f818c160ac0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8184009e50 tx=0x7f818402bf60 comp rx=0 tx=0).stop 2023-12-01T02:52:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f8190ab4640 1 -- 172.21.15.115:0/105214881 shutdown_connections 2023-12-01T02:52:19.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f8190ab4640 1 --2- 172.21.15.115:0/105214881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f818c0cfb80 0x7f818c160ac0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f8190ab4640 1 --2- 172.21.15.115:0/105214881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f818c153c30 0x7f818c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f8190ab4640 1 -- 172.21.15.115:0/105214881 >> 172.21.15.115:0/105214881 conn(0x7f818c06a0b0 msgr2=0x7f818c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:19.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f8190ab4640 1 -- 172.21.15.115:0/105214881 shutdown_connections 2023-12-01T02:52:19.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.790+0000 7f8190ab4640 1 -- 172.21.15.115:0/105214881 wait complete. 2023-12-01T02:52:19.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.792+0000 7f8190ab4640 1 Processor -- start 2023-12-01T02:52:19.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.792+0000 7f8190ab4640 1 -- start start 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.793+0000 7f8190ab4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f818c0cfb80 0x7f818c152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.793+0000 7f8190ab4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f818c153c30 0x7f818c0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.793+0000 7f8190ab4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f818c154010 0x7f818c1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.793+0000 7f8190ab4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0c6c40 con 0x7f818c0cfb80 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.793+0000 7f8190ab4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c152ad0 con 0x7f818c154010 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.793+0000 7f8190ab4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c155f10 con 0x7f818c153c30 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.793+0000 7f818b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f818c0cfb80 0x7f818c152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.793+0000 7f818b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f818c0cfb80 0x7f818c152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55626/0 (socket says 172.21.15.115:55626) 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.793+0000 7f818b7fe640 1 -- 172.21.15.115:0/1029165593 learned_addr learned my addr 172.21.15.115:0/1029165593 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f818b7fe640 1 -- 172.21.15.115:0/1029165593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f818c153c30 msgr2=0x7f818c0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f818bfff640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f818c154010 0x7f818c1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f8183fff640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f818c153c30 0x7f818c0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f818b7fe640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f818c153c30 0x7f818c0cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f818b7fe640 1 -- 172.21.15.115:0/1029165593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f818c154010 msgr2=0x7f818c1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f818b7fe640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f818c154010 0x7f818c1554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f818b7fe640 1 -- 172.21.15.115:0/1029165593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8184009b30 con 0x7f818c0cfb80 2023-12-01T02:52:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f818b7fe640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f818c0cfb80 0x7f818c152590 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f817c00e9b0 tx=0x7f817c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f8188ff9640 1 -- 172.21.15.115:0/1029165593 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c009670 con 0x7f818c0cfb80 2023-12-01T02:52:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f8188ff9640 1 -- 172.21.15.115:0/1029165593 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c00c9a0 con 0x7f818c0cfb80 2023-12-01T02:52:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f8188ff9640 1 -- 172.21.15.115:0/1029165593 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00cd50 con 0x7f818c0cfb80 2023-12-01T02:52:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f8190ab4640 1 -- 172.21.15.115:0/1029165593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c1561f0 con 0x7f818c0cfb80 2023-12-01T02:52:19.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.794+0000 7f8190ab4640 1 -- 172.21.15.115:0/1029165593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c0c6e90 con 0x7f818c0cfb80 2023-12-01T02:52:19.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.795+0000 7f8190ab4640 1 -- 172.21.15.115:0/1029165593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c0c7050 con 0x7f818c0cfb80 2023-12-01T02:52:19.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.796+0000 7f8188ff9640 1 -- 172.21.15.115:0/1029165593 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f817c013470 con 0x7f818c0cfb80 2023-12-01T02:52:19.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.797+0000 7f8188ff9640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f816c06a550 0x7f816c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:19.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.798+0000 7f8188ff9640 1 -- 172.21.15.115:0/1029165593 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f817c088a00 con 0x7f818c0cfb80 2023-12-01T02:52:19.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.798+0000 7f8183fff640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f816c06a550 0x7f816c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:19.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.798+0000 7f8183fff640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f816c06a550 0x7f816c06ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f818c156a20 tx=0x7f8174009040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:19.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.802+0000 7f8188ff9640 1 -- 172.21.15.115:0/1029165593 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f817c054200 con 0x7f818c0cfb80 2023-12-01T02:52:19.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:19.992+0000 7f8190ab4640 1 -- 172.21.15.115:0/1029165593 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f818c0618d0 con 0x7f818c0cfb80 2023-12-01T02:52:20.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.477+0000 7f8188ff9640 1 -- 172.21.15.115:0/1029165593 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v174) v1 ==== 138+0+0 (secure 0 0 0) 0x7f817c0170b0 con 0x7f818c0cfb80 2023-12-01T02:52:20.479 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T02:52:20.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.482+0000 7f8190ab4640 1 -- 172.21.15.115:0/1029165593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f816c06a550 msgr2=0x7f816c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:20.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.482+0000 7f8190ab4640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f816c06a550 0x7f816c06ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f818c156a20 tx=0x7f8174009040 comp rx=0 tx=0).stop 2023-12-01T02:52:20.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.482+0000 7f8190ab4640 1 -- 172.21.15.115:0/1029165593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f818c0cfb80 msgr2=0x7f818c152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:20.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.482+0000 7f8190ab4640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f818c0cfb80 0x7f818c152590 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f817c00e9b0 tx=0x7f817c00ee70 comp rx=0 tx=0).stop 2023-12-01T02:52:20.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.483+0000 7f8190ab4640 1 -- 172.21.15.115:0/1029165593 shutdown_connections 2023-12-01T02:52:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.483+0000 7f8190ab4640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f816c06a550 0x7f816c06ca10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.483+0000 7f8190ab4640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f818c153c30 0x7f818c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.483+0000 7f8190ab4640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f818c154010 0x7f818c1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.483+0000 7f8190ab4640 1 --2- 172.21.15.115:0/1029165593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f818c0cfb80 0x7f818c152590 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.483+0000 7f8190ab4640 1 -- 172.21.15.115:0/1029165593 >> 172.21.15.115:0/1029165593 conn(0x7f818c06a0b0 msgr2=0x7f818c15c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.483+0000 7f8190ab4640 1 -- 172.21.15.115:0/1029165593 shutdown_connections 2023-12-01T02:52:20.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.483+0000 7f8190ab4640 1 -- 172.21.15.115:0/1029165593 wait complete. 2023-12-01T02:52:20.508 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T02:52:20.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.712+0000 7f8214a42640 1 Processor -- start 2023-12-01T02:52:20.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.712+0000 7f8214a42640 1 -- start start 2023-12-01T02:52:20.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.714+0000 7f8214a42640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f821015cc70 0x7f82100b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:20.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.714+0000 7f8214a42640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82100cd490 0x7f82100cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:20.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.714+0000 7f8214a42640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82100ce510 con 0x7f82100b4620 2023-12-01T02:52:20.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.714+0000 7f8214a42640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82100b6380 con 0x7f82100cd490 2023-12-01T02:52:20.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.714+0000 7f8214a42640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82100d10e0 con 0x7f821015cc70 2023-12-01T02:52:20.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.714+0000 7f820e59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82100cd490 0x7f82100cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:20.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.714+0000 7f820e59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82100cd490 0x7f82100cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55634/0 (socket says 172.21.15.115:55634) 2023-12-01T02:52:20.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.714+0000 7f820e59c640 1 -- 172.21.15.115:0/2779268100 learned_addr learned my addr 172.21.15.115:0/2779268100 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:20.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.714+0000 7f820ed9d640 1 --2- 172.21.15.115:0/2779268100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f821015cc70 0x7f82100b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:20.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.715+0000 7f820e59c640 1 -- 172.21.15.115:0/2779268100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f821015cc70 msgr2=0x7f82100b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:20.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.715+0000 7f820e59c640 1 --2- 172.21.15.115:0/2779268100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f821015cc70 0x7f82100b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:20.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.715+0000 7f820e59c640 1 -- 172.21.15.115:0/2779268100 >> v1:172.21.15.115:6789/0 conn(0x7f82100b4620 legacy=0x7f82100d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:20.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.715+0000 7f820e59c640 1 -- 172.21.15.115:0/2779268100 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82100d16a0 con 0x7f82100cd490 2023-12-01T02:52:20.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.716+0000 7f820e59c640 1 --2- 172.21.15.115:0/2779268100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82100cd490 0x7f82100cd910 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009870 tx=0x7f81f802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43892545b3b8a276 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:20.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.716+0000 7f820dd9b640 1 -- 172.21.15.115:0/2779268100 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8030840 con 0x7f82100cd490 2023-12-01T02:52:20.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.716+0000 7f820dd9b640 1 -- 172.21.15.115:0/2779268100 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f80309a0 con 0x7f82100cd490 2023-12-01T02:52:20.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.716+0000 7f820dd9b640 1 -- 172.21.15.115:0/2779268100 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8030d30 con 0x7f82100cd490 2023-12-01T02:52:20.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.717+0000 7f8214a42640 1 -- 172.21.15.115:0/2779268100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82100cd490 msgr2=0x7f82100cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:20.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.717+0000 7f8214a42640 1 --2- 172.21.15.115:0/2779268100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82100cd490 0x7f82100cd910 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009870 tx=0x7f81f802b980 comp rx=0 tx=0).stop 2023-12-01T02:52:20.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.717+0000 7f8214a42640 1 -- 172.21.15.115:0/2779268100 shutdown_connections 2023-12-01T02:52:20.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.717+0000 7f8214a42640 1 --2- 172.21.15.115:0/2779268100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82100cd490 0x7f82100cd910 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:20.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.717+0000 7f8214a42640 1 --2- 172.21.15.115:0/2779268100 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f821015cc70 0x7f82100b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:20.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.717+0000 7f8214a42640 1 -- 172.21.15.115:0/2779268100 >> 172.21.15.115:0/2779268100 conn(0x7f82100cf620 msgr2=0x7f82100cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:20.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.717+0000 7f8214a42640 1 -- 172.21.15.115:0/2779268100 shutdown_connections 2023-12-01T02:52:20.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.717+0000 7f8214a42640 1 -- 172.21.15.115:0/2779268100 wait complete. 2023-12-01T02:52:20.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.719+0000 7f8214a42640 1 Processor -- start 2023-12-01T02:52:20.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.719+0000 7f8214a42640 1 -- start start 2023-12-01T02:52:20.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.720+0000 7f8214a42640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82100b4620 0x7f82100ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:20.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.720+0000 7f8214a42640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82100cd490 0x7f82101f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:20.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.720+0000 7f8214a42640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821015cc70 0x7f8210149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:20.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.720+0000 7f8214a42640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82101f63c0 con 0x7f821015cc70 2023-12-01T02:52:20.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.720+0000 7f8214a42640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82100cac40 con 0x7f82100b4620 2023-12-01T02:52:20.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.720+0000 7f8214a42640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82100caf20 con 0x7f82100cd490 2023-12-01T02:52:20.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.720+0000 7f820f59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821015cc70 0x7f8210149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:20.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.720+0000 7f820f59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821015cc70 0x7f8210149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55636/0 (socket says 172.21.15.115:55636) 2023-12-01T02:52:20.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.720+0000 7f820f59e640 1 -- 172.21.15.115:0/3555518207 learned_addr learned my addr 172.21.15.115:0/3555518207 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:20.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f820e59c640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82100cd490 0x7f82101f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:20.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f820ed9d640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82100b4620 0x7f82100ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:20.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f820f59e640 1 -- 172.21.15.115:0/3555518207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82100cd490 msgr2=0x7f82101f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:20.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f820f59e640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82100cd490 0x7f82101f5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:20.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f820f59e640 1 -- 172.21.15.115:0/3555518207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82100b4620 msgr2=0x7f82100ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:20.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f820f59e640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82100b4620 0x7f82100ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:20.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f820f59e640 1 -- 172.21.15.115:0/3555518207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f80094d0 con 0x7f821015cc70 2023-12-01T02:52:20.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f820f59e640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821015cc70 0x7f8210149e60 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f820800c390 tx=0x7f820800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:20.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f82077fe640 1 -- 172.21.15.115:0/3555518207 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82080052d0 con 0x7f821015cc70 2023-12-01T02:52:20.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f82077fe640 1 -- 172.21.15.115:0/3555518207 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8208005430 con 0x7f821015cc70 2023-12-01T02:52:20.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f82077fe640 1 -- 172.21.15.115:0/3555518207 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208003810 con 0x7f821015cc70 2023-12-01T02:52:20.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f8214a42640 1 -- 172.21.15.115:0/3555518207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82101e7390 con 0x7f821015cc70 2023-12-01T02:52:20.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.721+0000 7f8214a42640 1 -- 172.21.15.115:0/3555518207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821014a3a0 con 0x7f821015cc70 2023-12-01T02:52:20.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.723+0000 7f8214a42640 1 -- 172.21.15.115:0/3555518207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82101e78b0 con 0x7f821015cc70 2023-12-01T02:52:20.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.724+0000 7f82077fe640 1 -- 172.21.15.115:0/3555518207 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8208003970 con 0x7f821015cc70 2023-12-01T02:52:20.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.726+0000 7f82077fe640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a550 0x7f81f006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:20.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.726+0000 7f82077fe640 1 -- 172.21.15.115:0/3555518207 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5271+0+0 (secure 0 0 0) 0x7f8208087f90 con 0x7f821015cc70 2023-12-01T02:52:20.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.726+0000 7f820ed9d640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a550 0x7f81f006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:20.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.727+0000 7f820ed9d640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a550 0x7f81f006ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8200009f30 tx=0x7f820000b040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:20.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.730+0000 7f82077fe640 1 -- 172.21.15.115:0/3555518207 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8208053790 con 0x7f821015cc70 2023-12-01T02:52:20.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:20.920+0000 7f8214a42640 1 -- 172.21.15.115:0/3555518207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f82101e78b0 con 0x7f821015cc70 2023-12-01T02:52:21.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.491+0000 7f82077fe640 1 -- 172.21.15.115:0/3555518207 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v175) v1 ==== 170+0+0 (secure 0 0 0) 0x7f82101e78b0 con 0x7f821015cc70 2023-12-01T02:52:21.493 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T02:52:21.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.497+0000 7f8214a42640 1 -- 172.21.15.115:0/3555518207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a550 msgr2=0x7f81f006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:21.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.497+0000 7f8214a42640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a550 0x7f81f006ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8200009f30 tx=0x7f820000b040 comp rx=0 tx=0).stop 2023-12-01T02:52:21.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.497+0000 7f8214a42640 1 -- 172.21.15.115:0/3555518207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821015cc70 msgr2=0x7f8210149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:21.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.497+0000 7f8214a42640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821015cc70 0x7f8210149e60 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f820800c390 tx=0x7f820800c850 comp rx=0 tx=0).stop 2023-12-01T02:52:21.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.498+0000 7f8214a42640 1 -- 172.21.15.115:0/3555518207 shutdown_connections 2023-12-01T02:52:21.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.498+0000 7f8214a42640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f81f006a550 0x7f81f006ca10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.498+0000 7f8214a42640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82100cd490 0x7f82101f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.498+0000 7f8214a42640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821015cc70 0x7f8210149e60 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.498+0000 7f8214a42640 1 --2- 172.21.15.115:0/3555518207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82100b4620 0x7f82100ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.498+0000 7f8214a42640 1 -- 172.21.15.115:0/3555518207 >> 172.21.15.115:0/3555518207 conn(0x7f82100cf620 msgr2=0x7f82100b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.499+0000 7f8214a42640 1 -- 172.21.15.115:0/3555518207 shutdown_connections 2023-12-01T02:52:21.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.500+0000 7f8214a42640 1 -- 172.21.15.115:0/3555518207 wait complete. 2023-12-01T02:52:21.533 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T02:52:21.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.713+0000 7fa127ca1640 1 Processor -- start 2023-12-01T02:52:21.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.713+0000 7fa127ca1640 1 -- start start 2023-12-01T02:52:21.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.714+0000 7fa127ca1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa1200cfd60 0x7fa1200cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:21.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.715+0000 7fa127ca1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1200b5fa0 0x7fa120161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:21.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.715+0000 7fa127ca1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1200ce5e0 con 0x7fa12015cd90 2023-12-01T02:52:21.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.715+0000 7fa127ca1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1200d0ec0 con 0x7fa1200cfd60 2023-12-01T02:52:21.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.715+0000 7fa127ca1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa120161950 con 0x7fa1200b5fa0 2023-12-01T02:52:21.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.715+0000 7fa12523c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa1200cfd60 0x7fa1200cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:21.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.715+0000 7fa12623e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1200b5fa0 0x7fa120161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:21.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.715+0000 7fa12523c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa1200cfd60 0x7fa1200cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:48250/0 (socket says 172.21.15.115:48250) 2023-12-01T02:52:21.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.715+0000 7fa12523c640 1 -- 172.21.15.115:0/3840341965 learned_addr learned my addr 172.21.15.115:0/3840341965 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:21.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.716+0000 7fa12523c640 1 -- 172.21.15.115:0/3840341965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1200b5fa0 msgr2=0x7fa120161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:21.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.716+0000 7fa12523c640 1 --2- 172.21.15.115:0/3840341965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1200b5fa0 0x7fa120161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.716+0000 7fa12523c640 1 -- 172.21.15.115:0/3840341965 >> v1:172.21.15.142:6789/0 conn(0x7fa12015cd90 legacy=0x7fa1200cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:21.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.716+0000 7fa12523c640 1 -- 172.21.15.115:0/3840341965 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1200ce300 con 0x7fa1200cfd60 2023-12-01T02:52:21.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.717+0000 7fa12523c640 1 --2- 172.21.15.115:0/3840341965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1200cfd60 0x7fa1200cdd20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa110009870 tx=0x7fa11002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2fdd83d430b1122 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:21.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.718+0000 7fa124a3b640 1 -- 172.21.15.115:0/3840341965 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110031640 con 0x7fa1200cfd60 2023-12-01T02:52:21.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.718+0000 7fa124a3b640 1 -- 172.21.15.115:0/3840341965 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1100317a0 con 0x7fa1200cfd60 2023-12-01T02:52:21.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.718+0000 7fa124a3b640 1 -- 172.21.15.115:0/3840341965 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110031b30 con 0x7fa1200cfd60 2023-12-01T02:52:21.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.719+0000 7fa127ca1640 1 -- 172.21.15.115:0/3840341965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1200cfd60 msgr2=0x7fa1200cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:21.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.719+0000 7fa127ca1640 1 --2- 172.21.15.115:0/3840341965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1200cfd60 0x7fa1200cdd20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa110009870 tx=0x7fa11002b8f0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.719+0000 7fa127ca1640 1 -- 172.21.15.115:0/3840341965 shutdown_connections 2023-12-01T02:52:21.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.719+0000 7fa127ca1640 1 --2- 172.21.15.115:0/3840341965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1200b5fa0 0x7fa120161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.719+0000 7fa127ca1640 1 --2- 172.21.15.115:0/3840341965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1200cfd60 0x7fa1200cdd20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.719+0000 7fa127ca1640 1 -- 172.21.15.115:0/3840341965 >> 172.21.15.115:0/3840341965 conn(0x7fa120065960 msgr2=0x7fa120065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:21.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.719+0000 7fa127ca1640 1 -- 172.21.15.115:0/3840341965 shutdown_connections 2023-12-01T02:52:21.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.719+0000 7fa127ca1640 1 -- 172.21.15.115:0/3840341965 wait complete. 2023-12-01T02:52:21.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.721+0000 7fa127ca1640 1 Processor -- start 2023-12-01T02:52:21.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.721+0000 7fa127ca1640 1 -- start start 2023-12-01T02:52:21.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.722+0000 7fa127ca1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1200b5fa0 0x7fa1201f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:21.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.722+0000 7fa127ca1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1200cfd60 0x7fa120153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:21.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.722+0000 7fa127ca1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa12015cd90 0x7fa120160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:21.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.722+0000 7fa127ca1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200c9ec0 con 0x7fa1200cfd60 2023-12-01T02:52:21.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.722+0000 7fa127ca1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1201538c0 con 0x7fa12015cd90 2023-12-01T02:52:21.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.722+0000 7fa127ca1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1201f2de0 con 0x7fa1200b5fa0 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa12523c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1200cfd60 0x7fa120153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa12523c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1200cfd60 0x7fa120153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55642/0 (socket says 172.21.15.115:55642) 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa12523c640 1 -- 172.21.15.115:0/3775468333 learned_addr learned my addr 172.21.15.115:0/3775468333 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa125a3d640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1200b5fa0 0x7fa1201f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa12523c640 1 -- 172.21.15.115:0/3775468333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1200b5fa0 msgr2=0x7fa1201f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa12623e640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa12015cd90 0x7fa120160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa12523c640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1200b5fa0 0x7fa1201f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa12523c640 1 -- 172.21.15.115:0/3775468333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa12015cd90 msgr2=0x7fa120160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa12523c640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa12015cd90 0x7fa120160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa12523c640 1 -- 172.21.15.115:0/3775468333 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1100094d0 con 0x7fa1200cfd60 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.723+0000 7fa12523c640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1200cfd60 0x7fa120153380 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fa1100090b0 tx=0x7fa1100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.724+0000 7fa1167fc640 1 -- 172.21.15.115:0/3775468333 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11003a9f0 con 0x7fa1200cfd60 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.724+0000 7fa1167fc640 1 -- 172.21.15.115:0/3775468333 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11003ab70 con 0x7fa1200cfd60 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.724+0000 7fa1167fc640 1 -- 172.21.15.115:0/3775468333 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1100443f0 con 0x7fa1200cfd60 2023-12-01T02:52:21.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.724+0000 7fa127ca1640 1 -- 172.21.15.115:0/3775468333 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1201f30c0 con 0x7fa1200cfd60 2023-12-01T02:52:21.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.724+0000 7fa127ca1640 1 -- 172.21.15.115:0/3775468333 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1201606a0 con 0x7fa1200cfd60 2023-12-01T02:52:21.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.725+0000 7fa127ca1640 1 -- 172.21.15.115:0/3775468333 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12004e890 con 0x7fa1200cfd60 2023-12-01T02:52:21.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.726+0000 7fa1167fc640 1 -- 172.21.15.115:0/3775468333 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa11002f050 con 0x7fa1200cfd60 2023-12-01T02:52:21.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.727+0000 7fa1167fc640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa10406a550 0x7fa10406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:21.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.727+0000 7fa1167fc640 1 -- 172.21.15.115:0/3775468333 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5653+0+0 (secure 0 0 0) 0x7fa1100aa990 con 0x7fa1200cfd60 2023-12-01T02:52:21.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.727+0000 7fa125a3d640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa10406a550 0x7fa10406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:21.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.728+0000 7fa125a3d640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa10406a550 0x7fa10406ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006fd0 tx=0x7fa11c008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:21.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.732+0000 7fa1167fc640 1 -- 172.21.15.115:0/3775468333 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa110075e30 con 0x7fa1200cfd60 2023-12-01T02:52:21.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.958+0000 7fa127ca1640 1 -- 172.21.15.115:0/3775468333 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa1200ce5e0 con 0x7fa1200cfd60 2023-12-01T02:52:21.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.974+0000 7fa1167fc640 1 -- 172.21.15.115:0/3775468333 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa1200ce5e0 con 0x7fa1200cfd60 2023-12-01T02:52:21.976 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 20 and data pool 21 2023-12-01T02:52:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.978+0000 7fa127ca1640 1 -- 172.21.15.115:0/3775468333 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa10406a550 msgr2=0x7fa10406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.978+0000 7fa127ca1640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa10406a550 0x7fa10406ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006fd0 tx=0x7fa11c008040 comp rx=0 tx=0).stop 2023-12-01T02:52:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.978+0000 7fa127ca1640 1 -- 172.21.15.115:0/3775468333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1200cfd60 msgr2=0x7fa120153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.978+0000 7fa127ca1640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1200cfd60 0x7fa120153380 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fa1100090b0 tx=0x7fa1100046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.981+0000 7fa127ca1640 1 -- 172.21.15.115:0/3775468333 shutdown_connections 2023-12-01T02:52:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.981+0000 7fa127ca1640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa10406a550 0x7fa10406ca10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.981+0000 7fa127ca1640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa12015cd90 0x7fa120160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.981+0000 7fa127ca1640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1200cfd60 0x7fa120153380 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.981+0000 7fa127ca1640 1 --2- 172.21.15.115:0/3775468333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa1200b5fa0 0x7fa1201f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.981+0000 7fa127ca1640 1 -- 172.21.15.115:0/3775468333 >> 172.21.15.115:0/3775468333 conn(0x7fa120065960 msgr2=0x7fa1200b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.981+0000 7fa127ca1640 1 -- 172.21.15.115:0/3775468333 shutdown_connections 2023-12-01T02:52:21.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:21.983+0000 7fa127ca1640 1 -- 172.21.15.115:0/3775468333 wait complete. 2023-12-01T02:52:22.013 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:52:22.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.191+0000 7fa04144a640 1 Processor -- start 2023-12-01T02:52:22.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.191+0000 7fa04144a640 1 -- start start 2023-12-01T02:52:22.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.192+0000 7fa04144a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa03c159dd0 0x7fa03c0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.192+0000 7fa04144a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03c06f8b0 con 0x7fa03c070d30 2023-12-01T02:52:22.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.192+0000 7fa04144a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03c0adcb0 con 0x7fa03c06b6f0 2023-12-01T02:52:22.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.192+0000 7fa04144a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa03c0cd270 con 0x7fa03c159dd0 2023-12-01T02:52:22.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.193+0000 7fa03affd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa03c159dd0 0x7fa03c0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:22.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.193+0000 7fa03affd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa03c159dd0 0x7fa03c0c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55644/0 (socket says 172.21.15.115:55644) 2023-12-01T02:52:22.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.193+0000 7fa03affd640 1 -- 172.21.15.115:0/3010023661 learned_addr learned my addr 172.21.15.115:0/3010023661 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:22.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.193+0000 7fa03b7fe640 1 --1- 172.21.15.115:0/3010023661 >> v1:172.21.15.115:6789/0 conn(0x7fa03c070d30 0x7fa03c0ccb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54054/0 (socket says 172.21.15.115:54054) 2023-12-01T02:52:22.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.193+0000 7fa03affd640 1 -- 172.21.15.115:0/3010023661 >> v1:172.21.15.142:6789/0 conn(0x7fa03c06b6f0 legacy=0x7fa03c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:22.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.193+0000 7fa03affd640 1 -- 172.21.15.115:0/3010023661 >> v1:172.21.15.115:6789/0 conn(0x7fa03c070d30 legacy=0x7fa03c0ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.193+0000 7fa03affd640 1 -- 172.21.15.115:0/3010023661 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c0ad9d0 con 0x7fa03c159dd0 2023-12-01T02:52:22.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.194+0000 7fa03affd640 1 --2- 172.21.15.115:0/3010023661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c159dd0 0x7fa03c0c97e0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009870 tx=0x7fa02c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36864fd67daed47 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:22.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.195+0000 7fa03a7fc640 1 -- 172.21.15.115:0/3010023661 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c0317d0 con 0x7fa03c159dd0 2023-12-01T02:52:22.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.195+0000 7fa03a7fc640 1 -- 172.21.15.115:0/3010023661 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c031930 con 0x7fa03c159dd0 2023-12-01T02:52:22.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.195+0000 7fa03a7fc640 1 -- 172.21.15.115:0/3010023661 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c031cc0 con 0x7fa03c159dd0 2023-12-01T02:52:22.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.195+0000 7fa04144a640 1 -- 172.21.15.115:0/3010023661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c159dd0 msgr2=0x7fa03c0c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.195+0000 7fa04144a640 1 --2- 172.21.15.115:0/3010023661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c159dd0 0x7fa03c0c97e0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009870 tx=0x7fa02c02b870 comp rx=0 tx=0).stop 2023-12-01T02:52:22.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.195+0000 7fa04144a640 1 -- 172.21.15.115:0/3010023661 shutdown_connections 2023-12-01T02:52:22.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.195+0000 7fa04144a640 1 --2- 172.21.15.115:0/3010023661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c159dd0 0x7fa03c0c97e0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.195+0000 7fa04144a640 1 -- 172.21.15.115:0/3010023661 >> 172.21.15.115:0/3010023661 conn(0x7fa03c0ce880 msgr2=0x7fa03c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:22.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.195+0000 7fa04144a640 1 -- 172.21.15.115:0/3010023661 shutdown_connections 2023-12-01T02:52:22.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.195+0000 7fa04144a640 1 -- 172.21.15.115:0/3010023661 wait complete. 2023-12-01T02:52:22.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.197+0000 7fa04144a640 1 Processor -- start 2023-12-01T02:52:22.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.197+0000 7fa04144a640 1 -- start start 2023-12-01T02:52:22.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.198+0000 7fa04144a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa03c06b6f0 0x7fa03c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.198+0000 7fa04144a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c070d30 0x7fa03c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.198+0000 7fa04144a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa03c159dd0 0x7fa03c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.198+0000 7fa04144a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c1573e0 con 0x7fa03c070d30 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.198+0000 7fa04144a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c0ad640 con 0x7fa03c159dd0 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.198+0000 7fa04144a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c0ad360 con 0x7fa03c06b6f0 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.198+0000 7fa0327fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c070d30 0x7fa03c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.198+0000 7fa0327fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c070d30 0x7fa03c0ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55646/0 (socket says 172.21.15.115:55646) 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.199+0000 7fa0327fc640 1 -- 172.21.15.115:0/1397897055 learned_addr learned my addr 172.21.15.115:0/1397897055 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.199+0000 7fa03affd640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa03c06b6f0 0x7fa03c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.199+0000 7fa0327fc640 1 -- 172.21.15.115:0/1397897055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa03c06b6f0 msgr2=0x7fa03c156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.199+0000 7fa0327fc640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa03c06b6f0 0x7fa03c156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.199+0000 7fa0327fc640 1 -- 172.21.15.115:0/1397897055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa03c159dd0 msgr2=0x7fa03c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.199+0000 7fa0327fc640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa03c159dd0 0x7fa03c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.199+0000 7fa0327fc640 1 -- 172.21.15.115:0/1397897055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c0094d0 con 0x7fa03c070d30 2023-12-01T02:52:22.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.199+0000 7fa0327fc640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c070d30 0x7fa03c0ac8a0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fa02400b3f0 tx=0x7fa02400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:22.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.199+0000 7fa033fff640 1 -- 172.21.15.115:0/1397897055 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024011830 con 0x7fa03c070d30 2023-12-01T02:52:22.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.199+0000 7fa033fff640 1 -- 172.21.15.115:0/1397897055 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa024011990 con 0x7fa03c070d30 2023-12-01T02:52:22.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.200+0000 7fa033fff640 1 -- 172.21.15.115:0/1397897055 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024011d40 con 0x7fa03c070d30 2023-12-01T02:52:22.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.200+0000 7fa04144a640 1 -- 172.21.15.115:0/1397897055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03c0c2510 con 0x7fa03c070d30 2023-12-01T02:52:22.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.200+0000 7fa04144a640 1 -- 172.21.15.115:0/1397897055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03c0c2ad0 con 0x7fa03c070d30 2023-12-01T02:52:22.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.201+0000 7fa04144a640 1 -- 172.21.15.115:0/1397897055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03c0c2370 con 0x7fa03c070d30 2023-12-01T02:52:22.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.202+0000 7fa033fff640 1 -- 172.21.15.115:0/1397897055 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa02401a400 con 0x7fa03c070d30 2023-12-01T02:52:22.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.203+0000 7fa033fff640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa01c06a550 0x7fa01c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.203+0000 7fa033fff640 1 -- 172.21.15.115:0/1397897055 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fa024088990 con 0x7fa03c070d30 2023-12-01T02:52:22.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.203+0000 7fa03affd640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa01c06a550 0x7fa01c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:22.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.204+0000 7fa03affd640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa01c06a550 0x7fa01c06ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa02c036000 tx=0x7fa02c03d040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:22.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.207+0000 7fa033fff640 1 -- 172.21.15.115:0/1397897055 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa03c0c2370 con 0x7fa03c070d30 2023-12-01T02:52:22.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.396+0000 7fa04144a640 1 -- 172.21.15.115:0/1397897055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa03c0c2370 con 0x7fa03c070d30 2023-12-01T02:52:22.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.399+0000 7fa033fff640 1 -- 172.21.15.115:0/1397897055 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16242 (secure 0 0 0) 0x7fa03c0c2370 con 0x7fa03c070d30 2023-12-01T02:52:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:22.400 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":176,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:52:21.959293+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:52:20.047003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-01T02:52:20.974031+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.401+0000 7fa04144a640 1 -- 172.21.15.115:0/1397897055 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa01c06a550 msgr2=0x7fa01c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.401+0000 7fa04144a640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa01c06a550 0x7fa01c06ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa02c036000 tx=0x7fa02c03d040 comp rx=0 tx=0).stop 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.401+0000 7fa04144a640 1 -- 172.21.15.115:0/1397897055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c070d30 msgr2=0x7fa03c0ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.401+0000 7fa04144a640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c070d30 0x7fa03c0ac8a0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fa02400b3f0 tx=0x7fa02400b8b0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.402+0000 7fa04144a640 1 -- 172.21.15.115:0/1397897055 shutdown_connections 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.402+0000 7fa04144a640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa01c06a550 0x7fa01c06ca10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.402+0000 7fa04144a640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa03c159dd0 0x7fa03c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.402+0000 7fa04144a640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa03c070d30 0x7fa03c0ac8a0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.402+0000 7fa04144a640 1 --2- 172.21.15.115:0/1397897055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa03c06b6f0 0x7fa03c156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.402+0000 7fa04144a640 1 -- 172.21.15.115:0/1397897055 >> 172.21.15.115:0/1397897055 conn(0x7fa03c0ce880 msgr2=0x7fa03c06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.402+0000 7fa04144a640 1 -- 172.21.15.115:0/1397897055 shutdown_connections 2023-12-01T02:52:22.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.402+0000 7fa04144a640 1 -- 172.21.15.115:0/1397897055 wait complete. 2023-12-01T02:52:22.406 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:52:22.414 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:52:22.421 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:52:22.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.649+0000 7fa028873640 1 Processor -- start 2023-12-01T02:52:22.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.649+0000 7fa028873640 1 -- start start 2023-12-01T02:52:22.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.651+0000 7fa028873640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa0241584b0 0x7fa0240cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.651+0000 7fa028873640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 0x7fa02406b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.651+0000 7fa028873640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240cc180 con 0x7fa02406b7b0 2023-12-01T02:52:22.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.651+0000 7fa028873640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02406bc40 con 0x7fa0241584b0 2023-12-01T02:52:22.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.651+0000 7fa028873640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02415b550 con 0x7fa02415afe0 2023-12-01T02:52:22.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.651+0000 7fa01bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 0x7fa02406b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:22.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.652+0000 7fa0237fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa0241584b0 0x7fa0240cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:22.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.652+0000 7fa01bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 0x7fa02406b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:36362/0 (socket says 172.21.15.115:36362) 2023-12-01T02:52:22.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.652+0000 7fa01bfff640 1 -- 172.21.15.115:0/1619989354 learned_addr learned my addr 172.21.15.115:0/1619989354 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:22.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.652+0000 7fa01bfff640 1 -- 172.21.15.115:0/1619989354 >> v2:172.21.15.142:3300/0 conn(0x7fa0241584b0 msgr2=0x7fa0240cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:22.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.652+0000 7fa01bfff640 1 --2- 172.21.15.115:0/1619989354 >> v2:172.21.15.142:3300/0 conn(0x7fa0241584b0 0x7fa0240cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.652+0000 7fa01bfff640 1 -- 172.21.15.115:0/1619989354 >> v1:172.21.15.142:6789/0 conn(0x7fa02406b7b0 legacy=0x7fa02415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.652+0000 7fa01bfff640 1 -- 172.21.15.115:0/1619989354 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02415b830 con 0x7fa02415afe0 2023-12-01T02:52:22.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.652+0000 7fa0237fe640 1 --2- 172.21.15.115:0/1619989354 >> v2:172.21.15.142:3300/0 conn(0x7fa0241584b0 0x7fa0240cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T02:52:22.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.654+0000 7fa01bfff640 1 --2- 172.21.15.115:0/1619989354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 0x7fa02406b270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009870 tx=0x7fa00c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a77d203a71abf967 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:22.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.654+0000 7fa022ffd640 1 -- 172.21.15.115:0/1619989354 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c030840 con 0x7fa02415afe0 2023-12-01T02:52:22.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.655+0000 7fa022ffd640 1 -- 172.21.15.115:0/1619989354 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c0309a0 con 0x7fa02415afe0 2023-12-01T02:52:22.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.655+0000 7fa022ffd640 1 -- 172.21.15.115:0/1619989354 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c030d30 con 0x7fa02415afe0 2023-12-01T02:52:22.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.655+0000 7fa028873640 1 -- 172.21.15.115:0/1619989354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 msgr2=0x7fa02406b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.655+0000 7fa028873640 1 --2- 172.21.15.115:0/1619989354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 0x7fa02406b270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009870 tx=0x7fa00c02b980 comp rx=0 tx=0).stop 2023-12-01T02:52:22.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.655+0000 7fa028873640 1 -- 172.21.15.115:0/1619989354 shutdown_connections 2023-12-01T02:52:22.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.655+0000 7fa028873640 1 --2- 172.21.15.115:0/1619989354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 0x7fa02406b270 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.655+0000 7fa028873640 1 --2- 172.21.15.115:0/1619989354 >> v2:172.21.15.142:3300/0 conn(0x7fa0241584b0 0x7fa0240cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.655+0000 7fa028873640 1 -- 172.21.15.115:0/1619989354 >> 172.21.15.115:0/1619989354 conn(0x7fa0240ce640 msgr2=0x7fa0240cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:22.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.655+0000 7fa028873640 1 -- 172.21.15.115:0/1619989354 shutdown_connections 2023-12-01T02:52:22.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.655+0000 7fa028873640 1 -- 172.21.15.115:0/1619989354 wait complete. 2023-12-01T02:52:22.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.657+0000 7fa028873640 1 Processor -- start 2023-12-01T02:52:22.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.657+0000 7fa028873640 1 -- start start 2023-12-01T02:52:22.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.658+0000 7fa028873640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa02406b7b0 0x7fa024143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.658+0000 7fa028873640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0241584b0 0x7fa0240b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.658+0000 7fa028873640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 0x7fa0240b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.658+0000 7fa028873640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa024143660 con 0x7fa02406b7b0 2023-12-01T02:52:22.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.658+0000 7fa028873640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240c7fe0 con 0x7fa0241584b0 2023-12-01T02:52:22.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.658+0000 7fa028873640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240b31d0 con 0x7fa02415afe0 2023-12-01T02:52:22.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.658+0000 7fa0237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa02406b7b0 0x7fa024143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:22.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.658+0000 7fa0237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa02406b7b0 0x7fa024143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55662/0 (socket says 172.21.15.115:55662) 2023-12-01T02:52:22.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.658+0000 7fa0237fe640 1 -- 172.21.15.115:0/3156876558 learned_addr learned my addr 172.21.15.115:0/3156876558 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:22.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa023fff640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 0x7fa0240b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:22.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa01bfff640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0241584b0 0x7fa0240b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:22.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa0237fe640 1 -- 172.21.15.115:0/3156876558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 msgr2=0x7fa0240b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa0237fe640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 0x7fa0240b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa0237fe640 1 -- 172.21.15.115:0/3156876558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0241584b0 msgr2=0x7fa0240b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa0237fe640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0241584b0 0x7fa0240b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa0237fe640 1 -- 172.21.15.115:0/3156876558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa014009400 con 0x7fa02406b7b0 2023-12-01T02:52:22.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa0237fe640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa02406b7b0 0x7fa024143120 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fa01400ebe0 tx=0x7fa01400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:22.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa020ff9640 1 -- 172.21.15.115:0/3156876558 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014009760 con 0x7fa02406b7b0 2023-12-01T02:52:22.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa020ff9640 1 -- 172.21.15.115:0/3156876558 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01400cc30 con 0x7fa02406b7b0 2023-12-01T02:52:22.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa020ff9640 1 -- 172.21.15.115:0/3156876558 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01400eee0 con 0x7fa02406b7b0 2023-12-01T02:52:22.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.659+0000 7fa028873640 1 -- 172.21.15.115:0/3156876558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00c0094d0 con 0x7fa02406b7b0 2023-12-01T02:52:22.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.660+0000 7fa028873640 1 -- 172.21.15.115:0/3156876558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0240b28a0 con 0x7fa02406b7b0 2023-12-01T02:52:22.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.661+0000 7fa028873640 1 -- 172.21.15.115:0/3156876558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0240673a0 con 0x7fa02406b7b0 2023-12-01T02:52:22.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.661+0000 7fa020ff9640 1 -- 172.21.15.115:0/3156876558 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa014013470 con 0x7fa02406b7b0 2023-12-01T02:52:22.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.662+0000 7fa020ff9640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa00406a550 0x7fa00406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:22.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.662+0000 7fa020ff9640 1 -- 172.21.15.115:0/3156876558 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fa014088df0 con 0x7fa02406b7b0 2023-12-01T02:52:22.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.663+0000 7fa01bfff640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa00406a550 0x7fa00406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:22.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.663+0000 7fa01bfff640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa00406a550 0x7fa00406ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa00c036000 tx=0x7fa00c03b040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:22.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.667+0000 7fa020ff9640 1 -- 172.21.15.115:0/3156876558 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa0140542a0 con 0x7fa02406b7b0 2023-12-01T02:52:22.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.843+0000 7fa028873640 1 -- 172.21.15.115:0/3156876558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa024157ac0 con 0x7fa02406b7b0 2023-12-01T02:52:22.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.845+0000 7fa020ff9640 1 -- 172.21.15.115:0/3156876558 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16242 (secure 0 0 0) 0x7fa0140578c0 con 0x7fa02406b7b0 2023-12-01T02:52:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:22.846 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":176,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:52:21.959293+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:52:20.047003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-01T02:52:20.974031+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:52:22.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.847+0000 7fa028873640 1 -- 172.21.15.115:0/3156876558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa00406a550 msgr2=0x7fa00406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.847+0000 7fa028873640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa00406a550 0x7fa00406ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa00c036000 tx=0x7fa00c03b040 comp rx=0 tx=0).stop 2023-12-01T02:52:22.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.847+0000 7fa028873640 1 -- 172.21.15.115:0/3156876558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa02406b7b0 msgr2=0x7fa024143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:22.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.847+0000 7fa028873640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa02406b7b0 0x7fa024143120 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fa01400ebe0 tx=0x7fa01400c520 comp rx=0 tx=0).stop 2023-12-01T02:52:22.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.847+0000 7fa028873640 1 -- 172.21.15.115:0/3156876558 shutdown_connections 2023-12-01T02:52:22.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.847+0000 7fa028873640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fa00406a550 0x7fa00406ca10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.847+0000 7fa028873640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa02415afe0 0x7fa0240b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.848+0000 7fa028873640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0241584b0 0x7fa0240b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.848+0000 7fa028873640 1 --2- 172.21.15.115:0/3156876558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa02406b7b0 0x7fa024143120 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:22.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.848+0000 7fa028873640 1 -- 172.21.15.115:0/3156876558 >> 172.21.15.115:0/3156876558 conn(0x7fa0240ce640 msgr2=0x7fa0240acaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:22.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.848+0000 7fa028873640 1 -- 172.21.15.115:0/3156876558 shutdown_connections 2023-12-01T02:52:22.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:22.848+0000 7fa028873640 1 -- 172.21.15.115:0/3156876558 wait complete. 2023-12-01T02:52:22.866 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T02:52:23.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.080+0000 7fd94e65f640 1 Processor -- start 2023-12-01T02:52:23.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.080+0000 7fd94e65f640 1 -- start start 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.081+0000 7fd94e65f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd948142870 0x7fd9480ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.081+0000 7fd94e65f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd9480cd010 0x7fd9480b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.081+0000 7fd94e65f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9480b2f30 con 0x7fd9480b1c50 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.082+0000 7fd94e65f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9480b3ae0 con 0x7fd9480cd010 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.082+0000 7fd94e65f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9480cd4a0 con 0x7fd948142870 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.082+0000 7fd9477fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd9480cd010 0x7fd9480b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.082+0000 7fd9477fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd9480cd010 0x7fd9480b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55672/0 (socket says 172.21.15.115:55672) 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.082+0000 7fd9477fe640 1 -- 172.21.15.115:0/4171356887 learned_addr learned my addr 172.21.15.115:0/4171356887 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.082+0000 7fd947fff640 1 --2- 172.21.15.115:0/4171356887 >> v2:172.21.15.142:3300/0 conn(0x7fd948142870 0x7fd9480ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.082+0000 7fd9477fe640 1 -- 172.21.15.115:0/4171356887 >> v2:172.21.15.142:3300/0 conn(0x7fd948142870 msgr2=0x7fd9480ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.082+0000 7fd9477fe640 1 --2- 172.21.15.115:0/4171356887 >> v2:172.21.15.142:3300/0 conn(0x7fd948142870 0x7fd9480ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.082+0000 7fd9477fe640 1 -- 172.21.15.115:0/4171356887 >> v1:172.21.15.142:6789/0 conn(0x7fd9480b1c50 legacy=0x7fd94815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.082+0000 7fd9477fe640 1 -- 172.21.15.115:0/4171356887 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9480b20c0 con 0x7fd9480cd010 2023-12-01T02:52:23.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.083+0000 7fd9477fe640 1 --2- 172.21.15.115:0/4171356887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 0x7fd9480b1710 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fd9480b14d0 tx=0x7fd93c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad753961205a132 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:23.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.084+0000 7fd946ffd640 1 -- 172.21.15.115:0/4171356887 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c031870 con 0x7fd9480cd010 2023-12-01T02:52:23.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.084+0000 7fd946ffd640 1 -- 172.21.15.115:0/4171356887 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c0319d0 con 0x7fd9480cd010 2023-12-01T02:52:23.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.084+0000 7fd946ffd640 1 -- 172.21.15.115:0/4171356887 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c031d60 con 0x7fd9480cd010 2023-12-01T02:52:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.084+0000 7fd94e65f640 1 -- 172.21.15.115:0/4171356887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 msgr2=0x7fd9480b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.084+0000 7fd94e65f640 1 --2- 172.21.15.115:0/4171356887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 0x7fd9480b1710 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fd9480b14d0 tx=0x7fd93c02b910 comp rx=0 tx=0).stop 2023-12-01T02:52:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.084+0000 7fd94e65f640 1 -- 172.21.15.115:0/4171356887 shutdown_connections 2023-12-01T02:52:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.084+0000 7fd94e65f640 1 --2- 172.21.15.115:0/4171356887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 0x7fd9480b1710 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.084+0000 7fd94e65f640 1 --2- 172.21.15.115:0/4171356887 >> v2:172.21.15.142:3300/0 conn(0x7fd948142870 0x7fd9480ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.084+0000 7fd94e65f640 1 -- 172.21.15.115:0/4171356887 >> 172.21.15.115:0/4171356887 conn(0x7fd948065960 msgr2=0x7fd948065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.085+0000 7fd94e65f640 1 -- 172.21.15.115:0/4171356887 shutdown_connections 2023-12-01T02:52:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.085+0000 7fd94e65f640 1 -- 172.21.15.115:0/4171356887 wait complete. 2023-12-01T02:52:23.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.086+0000 7fd94e65f640 1 Processor -- start 2023-12-01T02:52:23.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.086+0000 7fd94e65f640 1 -- start start 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.087+0000 7fd94e65f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9480b1c50 0x7fd9480cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.087+0000 7fd94e65f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 0x7fd9480c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.087+0000 7fd94e65f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd948142870 0x7fd94814d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.087+0000 7fd94e65f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9480aed00 con 0x7fd9480cd010 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.087+0000 7fd94e65f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd948159250 con 0x7fd948142870 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.087+0000 7fd94e65f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94814e8d0 con 0x7fd9480b1c50 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd9477fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 0x7fd9480c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd9477fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 0x7fd9480c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55686/0 (socket says 172.21.15.115:55686) 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd9477fe640 1 -- 172.21.15.115:0/2052366390 learned_addr learned my addr 172.21.15.115:0/2052366390 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd947fff640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9480b1c50 0x7fd9480cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd9477fe640 1 -- 172.21.15.115:0/2052366390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9480b1c50 msgr2=0x7fd9480cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:23.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd9477fe640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9480b1c50 0x7fd9480cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:23.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd9477fe640 1 -- 172.21.15.115:0/2052366390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd948142870 msgr2=0x7fd94814d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:23.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd9477fe640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd948142870 0x7fd94814d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:23.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd9477fe640 1 -- 172.21.15.115:0/2052366390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c0094d0 con 0x7fd9480cd010 2023-12-01T02:52:23.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd947fff640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9480b1c50 0x7fd9480cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:23.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.088+0000 7fd9477fe640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 0x7fd9480c4530 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fd93c02b950 tx=0x7fd93c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:23.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.089+0000 7fd944ff9640 1 -- 172.21.15.115:0/2052366390 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c031860 con 0x7fd9480cd010 2023-12-01T02:52:23.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.089+0000 7fd944ff9640 1 -- 172.21.15.115:0/2052366390 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c0443f0 con 0x7fd9480cd010 2023-12-01T02:52:23.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.089+0000 7fd94e65f640 1 -- 172.21.15.115:0/2052366390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94814ddf0 con 0x7fd9480cd010 2023-12-01T02:52:23.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.089+0000 7fd944ff9640 1 -- 172.21.15.115:0/2052366390 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0447a0 con 0x7fd9480cd010 2023-12-01T02:52:23.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.089+0000 7fd94e65f640 1 -- 172.21.15.115:0/2052366390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94814e310 con 0x7fd9480cd010 2023-12-01T02:52:23.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.090+0000 7fd94e65f640 1 -- 172.21.15.115:0/2052366390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9480aea60 con 0x7fd9480cd010 2023-12-01T02:52:23.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.091+0000 7fd944ff9640 1 -- 172.21.15.115:0/2052366390 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd93c04d450 con 0x7fd9480cd010 2023-12-01T02:52:23.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.092+0000 7fd944ff9640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd93406a550 0x7fd93406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:23.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.092+0000 7fd944ff9640 1 -- 172.21.15.115:0/2052366390 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fd93c02d040 con 0x7fd9480cd010 2023-12-01T02:52:23.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.093+0000 7fd947fff640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd93406a550 0x7fd93406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:23.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.093+0000 7fd947fff640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd93406a550 0x7fd93406ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd94806fe30 tx=0x7fd938008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:23.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.097+0000 7fd944ff9640 1 -- 172.21.15.115:0/2052366390 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd93c0ddc80 con 0x7fd9480cd010 2023-12-01T02:52:23.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.332+0000 7fd94e65f640 1 -- 172.21.15.115:0/2052366390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd9480aee40 con 0x7fd9480cd010 2023-12-01T02:52:23.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.985+0000 7fd944ff9640 1 -- 172.21.15.115:0/2052366390 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd9480aee40 con 0x7fd9480cd010 2023-12-01T02:52:23.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.989+0000 7fd94e65f640 1 -- 172.21.15.115:0/2052366390 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd93406a550 msgr2=0x7fd93406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.989+0000 7fd94e65f640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd93406a550 0x7fd93406ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd94806fe30 tx=0x7fd938008040 comp rx=0 tx=0).stop 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.989+0000 7fd94e65f640 1 -- 172.21.15.115:0/2052366390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 msgr2=0x7fd9480c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.989+0000 7fd94e65f640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 0x7fd9480c4530 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fd93c02b950 tx=0x7fd93c0046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.990+0000 7fd94e65f640 1 -- 172.21.15.115:0/2052366390 shutdown_connections 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.990+0000 7fd94e65f640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd93406a550 0x7fd93406ca10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.990+0000 7fd94e65f640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd948142870 0x7fd94814d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.990+0000 7fd94e65f640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9480cd010 0x7fd9480c4530 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.990+0000 7fd94e65f640 1 --2- 172.21.15.115:0/2052366390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9480b1c50 0x7fd9480cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.990+0000 7fd94e65f640 1 -- 172.21.15.115:0/2052366390 >> 172.21.15.115:0/2052366390 conn(0x7fd948065960 msgr2=0x7fd9480b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.990+0000 7fd94e65f640 1 -- 172.21.15.115:0/2052366390 shutdown_connections 2023-12-01T02:52:23.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:23.990+0000 7fd94e65f640 1 -- 172.21.15.115:0/2052366390 wait complete. 2023-12-01T02:52:24.012 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T02:52:24.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T02:52:24.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.230+0000 7f04b68e9640 1 Processor -- start 2023-12-01T02:52:24.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.230+0000 7f04b68e9640 1 -- start start 2023-12-01T02:52:24.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.231+0000 7f04b68e9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0142870 0x7f04b00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:24.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.231+0000 7f04b68e9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b00cd470 con 0x7f04b00cd010 2023-12-01T02:52:24.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.231+0000 7f04b68e9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b00b2f30 con 0x7f04b00b1e20 2023-12-01T02:52:24.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.231+0000 7f04b68e9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00b3ae0 con 0x7f04b0142870 2023-12-01T02:52:24.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.232+0000 7f04af7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f04b00cd010 0x7f04b00b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54070/0 (socket says 172.21.15.115:54070) 2023-12-01T02:52:24.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.232+0000 7f04af7fe640 1 -- 172.21.15.115:0/1703954682 learned_addr learned my addr 172.21.15.115:0/1703954682 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:24.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.232+0000 7f04affff640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0142870 0x7f04b00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:24.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.232+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930589877 0 0) 0x7f04b00cd470 con 0x7f04b00cd010 2023-12-01T02:52:24.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.232+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0498003520 con 0x7f04b00cd010 2023-12-01T02:52:24.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3851775674 0 0) 0x7f0498003520 con 0x7f04b00cd010 2023-12-01T02:52:24.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0142870 msgr2=0x7f04b00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:24.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0142870 0x7f04b00ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:24.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 >> v1:172.21.15.142:6789/0 conn(0x7f04b00b1e20 legacy=0x7f04b015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:24.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b00b2320 con 0x7f04b00cd010 2023-12-01T02:52:24.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f04a4001670 con 0x7f04b00cd010 2023-12-01T02:52:24.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 >> v1:172.21.15.115:6789/0 conn(0x7f04b00cd010 legacy=0x7f04b00b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04980039a0 0x7f0498003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0142870 0x7f04980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0498004d00 0x7f0498005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00b3ae0 con 0x7f0498004d00 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049800ba30 con 0x7f04980039a0 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049800de00 con 0x7f04b0142870 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04b4e86640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0498004d00 0x7f0498005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04af7fe640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0142870 0x7f04980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04b4e86640 1 -- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0142870 msgr2=0x7f04980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04b4e86640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0142870 0x7f04980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04b4e86640 1 -- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04980039a0 msgr2=0x7f0498003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04b4e86640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04980039a0 0x7f0498003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.233+0000 7f04b4e86640 1 -- 172.21.15.115:0/1703954682 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0498003520 con 0x7f0498004d00 2023-12-01T02:52:24.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.234+0000 7f04af7fe640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0142870 0x7f04980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:24.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.235+0000 7f04b4e86640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0498004d00 0x7f0498005180 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f04b00b2250 tx=0x7f04980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa76802be9da4cc3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:24.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.235+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8003b70 con 0x7f0498004d00 2023-12-01T02:52:24.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.235+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a8003cd0 con 0x7f0498004d00 2023-12-01T02:52:24.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.235+0000 7f04aeffd640 1 -- 172.21.15.115:0/1703954682 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8005310 con 0x7f0498004d00 2023-12-01T02:52:24.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.235+0000 7f04b68e9640 1 -- 172.21.15.115:0/1703954682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0498004d00 msgr2=0x7f0498005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:24.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.235+0000 7f04b68e9640 1 --2- 172.21.15.115:0/1703954682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0498004d00 0x7f0498005180 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f04b00b2250 tx=0x7f04980057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:24.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.235+0000 7f04b4e86640 1 -- 172.21.15.115:0/1703954682 reap_dead start 2023-12-01T02:52:24.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.235+0000 7f04b68e9640 1 -- 172.21.15.115:0/1703954682 shutdown_connections 2023-12-01T02:52:24.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.235+0000 7f04b68e9640 1 -- 172.21.15.115:0/1703954682 >> 172.21.15.115:0/1703954682 conn(0x7f04b0065960 msgr2=0x7f04b0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:24.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.236+0000 7f04b68e9640 1 -- 172.21.15.115:0/1703954682 shutdown_connections 2023-12-01T02:52:24.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.236+0000 7f04b68e9640 1 -- 172.21.15.115:0/1703954682 wait complete. 2023-12-01T02:52:24.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.237+0000 7f04b68e9640 1 Processor -- start 2023-12-01T02:52:24.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.238+0000 7f04b68e9640 1 -- start start 2023-12-01T02:52:24.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.238+0000 7f04b68e9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0498004d00 0x7f04b013d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:24.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04b68e9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04b00aeb00 0x7f04b01418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:24.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04b68e9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0141e20 0x7f04b00ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:24.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04b68e9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b01422a0 con 0x7f04b00aeb00 2023-12-01T02:52:24.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04b68e9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b0159630 con 0x7f0498004d00 2023-12-01T02:52:24.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04b68e9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00ada60 con 0x7f04b0141e20 2023-12-01T02:52:24.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04af7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04b00aeb00 0x7f04b01418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04af7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04b00aeb00 0x7f04b01418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55694/0 (socket says 172.21.15.115:55694) 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04af7fe640 1 -- 172.21.15.115:0/2023557024 learned_addr learned my addr 172.21.15.115:0/2023557024 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04b4e86640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0141e20 0x7f04b00ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04affff640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0498004d00 0x7f04b013d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04af7fe640 1 -- 172.21.15.115:0/2023557024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0141e20 msgr2=0x7f04b00ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04af7fe640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0141e20 0x7f04b00ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04af7fe640 1 -- 172.21.15.115:0/2023557024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0498004d00 msgr2=0x7f04b013d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04af7fe640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0498004d00 0x7f04b013d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04af7fe640 1 -- 172.21.15.115:0/2023557024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b00add40 con 0x7f04b00aeb00 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.239+0000 7f04b4e86640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0141e20 0x7f04b00ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.240+0000 7f04af7fe640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04b00aeb00 0x7f04b01418e0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f04a400d8e0 tx=0x7f04a400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.240+0000 7f04acff9640 1 -- 172.21.15.115:0/2023557024 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a40074e0 con 0x7f04b00aeb00 2023-12-01T02:52:24.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.240+0000 7f04acff9640 1 -- 172.21.15.115:0/2023557024 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a4007640 con 0x7f04b00aeb00 2023-12-01T02:52:24.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.240+0000 7f04acff9640 1 -- 172.21.15.115:0/2023557024 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a40049e0 con 0x7f04b00aeb00 2023-12-01T02:52:24.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.240+0000 7f04b68e9640 1 -- 172.21.15.115:0/2023557024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b00ae020 con 0x7f04b00aeb00 2023-12-01T02:52:24.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.240+0000 7f04b68e9640 1 -- 172.21.15.115:0/2023557024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b00ae5e0 con 0x7f04b00aeb00 2023-12-01T02:52:24.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.241+0000 7f04b68e9640 1 -- 172.21.15.115:0/2023557024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b00ae7a0 con 0x7f04b00aeb00 2023-12-01T02:52:24.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.242+0000 7f04acff9640 1 -- 172.21.15.115:0/2023557024 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f04a4004b40 con 0x7f04b00aeb00 2023-12-01T02:52:24.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.243+0000 7f04acff9640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f049c06a550 0x7f049c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:24.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.243+0000 7f04acff9640 1 -- 172.21.15.115:0/2023557024 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f04a40885b0 con 0x7f04b00aeb00 2023-12-01T02:52:24.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.243+0000 7f04affff640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f049c06a550 0x7f049c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:24.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.244+0000 7f04affff640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f049c06a550 0x7f049c06ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f04a0004770 tx=0x7f04a0008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:24.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.247+0000 7f04acff9640 1 -- 172.21.15.115:0/2023557024 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f04a4057490 con 0x7f04b00aeb00 2023-12-01T02:52:24.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.473+0000 7f04b68e9640 1 -- 172.21.15.115:0/2023557024 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f04b00cc940 con 0x7f04b00aeb00 2023-12-01T02:52:25.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:24.999+0000 7f04acff9640 1 -- 172.21.15.115:0/2023557024 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v79) v1 ==== 122+0+0 (secure 0 0 0) 0x7f04a4057020 con 0x7f04b00aeb00 2023-12-01T02:52:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 -- 172.21.15.115:0/2023557024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f049c06a550 msgr2=0x7f049c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f049c06a550 0x7f049c06ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f04a0004770 tx=0x7f04a0008040 comp rx=0 tx=0).stop 2023-12-01T02:52:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 -- 172.21.15.115:0/2023557024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04b00aeb00 msgr2=0x7f04b01418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04b00aeb00 0x7f04b01418e0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f04a400d8e0 tx=0x7f04a400dda0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 -- 172.21.15.115:0/2023557024 shutdown_connections 2023-12-01T02:52:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f049c06a550 0x7f049c06ca10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04b0141e20 0x7f04b00ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04b00aeb00 0x7f04b01418e0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 --2- 172.21.15.115:0/2023557024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0498004d00 0x7f04b013d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 -- 172.21.15.115:0/2023557024 >> 172.21.15.115:0/2023557024 conn(0x7f04b0065960 msgr2=0x7f04b013eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:25.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.002+0000 7f04b68e9640 1 -- 172.21.15.115:0/2023557024 shutdown_connections 2023-12-01T02:52:25.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.003+0000 7f04b68e9640 1 -- 172.21.15.115:0/2023557024 wait complete. 2023-12-01T02:52:25.023 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T02:52:25.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:52:25.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.226+0000 7fca19084640 1 Processor -- start 2023-12-01T02:52:25.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.227+0000 7fca19084640 1 -- start start 2023-12-01T02:52:25.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.228+0000 7fca19084640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14159dd0 0x7fca140c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.228+0000 7fca19084640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fca14070d30 0x7fca140c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.228+0000 7fca19084640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1406f8b0 con 0x7fca1406b6f0 2023-12-01T02:52:25.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.228+0000 7fca19084640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca140adcb0 con 0x7fca14070d30 2023-12-01T02:52:25.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.228+0000 7fca19084640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca140ad9d0 con 0x7fca14159dd0 2023-12-01T02:52:25.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.229+0000 7fca12d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14159dd0 0x7fca140c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.229+0000 7fca1359e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fca14070d30 0x7fca140c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.229+0000 7fca12d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14159dd0 0x7fca140c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:36436/0 (socket says 172.21.15.115:36436) 2023-12-01T02:52:25.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.229+0000 7fca12d9d640 1 -- 172.21.15.115:0/2277745122 learned_addr learned my addr 172.21.15.115:0/2277745122 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:25.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.230+0000 7fca12d9d640 1 -- 172.21.15.115:0/2277745122 >> v2:172.21.15.142:3300/0 conn(0x7fca14070d30 msgr2=0x7fca140c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:25.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.230+0000 7fca12d9d640 1 --2- 172.21.15.115:0/2277745122 >> v2:172.21.15.142:3300/0 conn(0x7fca14070d30 0x7fca140c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.230+0000 7fca12d9d640 1 -- 172.21.15.115:0/2277745122 >> v1:172.21.15.142:6789/0 conn(0x7fca1406b6f0 legacy=0x7fca140cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.230+0000 7fca12d9d640 1 -- 172.21.15.115:0/2277745122 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca140c9000 con 0x7fca14159dd0 2023-12-01T02:52:25.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.230+0000 7fca1359e640 1 --2- 172.21.15.115:0/2277745122 >> v2:172.21.15.142:3300/0 conn(0x7fca14070d30 0x7fca140c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T02:52:25.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.231+0000 7fca12d9d640 1 --2- 172.21.15.115:0/2277745122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14159dd0 0x7fca140c97e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca04009870 tx=0x7fca0402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e441a6f5973faf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:25.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.231+0000 7fca1259c640 1 -- 172.21.15.115:0/2277745122 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca040317c0 con 0x7fca14159dd0 2023-12-01T02:52:25.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.231+0000 7fca1259c640 1 -- 172.21.15.115:0/2277745122 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca04031920 con 0x7fca14159dd0 2023-12-01T02:52:25.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.231+0000 7fca1259c640 1 -- 172.21.15.115:0/2277745122 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04031cb0 con 0x7fca14159dd0 2023-12-01T02:52:25.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.232+0000 7fca19084640 1 -- 172.21.15.115:0/2277745122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14159dd0 msgr2=0x7fca140c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.232+0000 7fca19084640 1 --2- 172.21.15.115:0/2277745122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14159dd0 0x7fca140c97e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca04009870 tx=0x7fca0402b860 comp rx=0 tx=0).stop 2023-12-01T02:52:25.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.232+0000 7fca19084640 1 -- 172.21.15.115:0/2277745122 shutdown_connections 2023-12-01T02:52:25.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.232+0000 7fca19084640 1 --2- 172.21.15.115:0/2277745122 >> v2:172.21.15.142:3300/0 conn(0x7fca14070d30 0x7fca140c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.232+0000 7fca19084640 1 --2- 172.21.15.115:0/2277745122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14159dd0 0x7fca140c97e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.232+0000 7fca19084640 1 -- 172.21.15.115:0/2277745122 >> 172.21.15.115:0/2277745122 conn(0x7fca140ce880 msgr2=0x7fca140cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:25.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.232+0000 7fca19084640 1 -- 172.21.15.115:0/2277745122 shutdown_connections 2023-12-01T02:52:25.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.232+0000 7fca19084640 1 -- 172.21.15.115:0/2277745122 wait complete. 2023-12-01T02:52:25.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.234+0000 7fca19084640 1 Processor -- start 2023-12-01T02:52:25.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.235+0000 7fca19084640 1 -- start start 2023-12-01T02:52:25.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.236+0000 7fca19084640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1406b6f0 0x7fca140c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.236+0000 7fca19084640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14070d30 0x7fca1414cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.236+0000 7fca19084640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca14159dd0 0x7fca140ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.236+0000 7fca19084640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca14155720 con 0x7fca14159dd0 2023-12-01T02:52:25.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.236+0000 7fca19084640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1414d460 con 0x7fca1406b6f0 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.236+0000 7fca19084640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca14154aa0 con 0x7fca14070d30 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.236+0000 7fca1359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca14159dd0 0x7fca140ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.236+0000 7fca1359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca14159dd0 0x7fca140ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55696/0 (socket says 172.21.15.115:55696) 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.236+0000 7fca1359e640 1 -- 172.21.15.115:0/3120069194 learned_addr learned my addr 172.21.15.115:0/3120069194 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.236+0000 7fca12d9d640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1406b6f0 0x7fca140c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca0bfff640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14070d30 0x7fca1414cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca1359e640 1 -- 172.21.15.115:0/3120069194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14070d30 msgr2=0x7fca1414cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca1359e640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14070d30 0x7fca1414cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca1359e640 1 -- 172.21.15.115:0/3120069194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1406b6f0 msgr2=0x7fca140c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca1359e640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1406b6f0 0x7fca140c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca1359e640 1 -- 172.21.15.115:0/3120069194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c009a70 con 0x7fca14159dd0 2023-12-01T02:52:25.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca1359e640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca14159dd0 0x7fca140ac8a0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00ef90 tx=0x7fca0c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:25.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca0b7fe640 1 -- 172.21.15.115:0/3120069194 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c009d70 con 0x7fca14159dd0 2023-12-01T02:52:25.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca0b7fe640 1 -- 172.21.15.115:0/3120069194 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c00f040 con 0x7fca14159dd0 2023-12-01T02:52:25.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca0b7fe640 1 -- 172.21.15.115:0/3120069194 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0146c0 con 0x7fca14159dd0 2023-12-01T02:52:25.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.237+0000 7fca19084640 1 -- 172.21.15.115:0/3120069194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca040094d0 con 0x7fca14159dd0 2023-12-01T02:52:25.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.238+0000 7fca19084640 1 -- 172.21.15.115:0/3120069194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca14155060 con 0x7fca14159dd0 2023-12-01T02:52:25.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.239+0000 7fca19084640 1 -- 172.21.15.115:0/3120069194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca14155860 con 0x7fca14159dd0 2023-12-01T02:52:25.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.240+0000 7fca0b7fe640 1 -- 172.21.15.115:0/3120069194 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fca0c014820 con 0x7fca14159dd0 2023-12-01T02:52:25.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.241+0000 7fca0b7fe640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9f406a550 0x7fc9f406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.241+0000 7fca0b7fe640 1 -- 172.21.15.115:0/3120069194 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fca0c089b00 con 0x7fca14159dd0 2023-12-01T02:52:25.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.241+0000 7fca12d9d640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9f406a550 0x7fc9f406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.242+0000 7fca12d9d640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9f406a550 0x7fc9f406ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca04036000 tx=0x7fca0403d040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:25.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.245+0000 7fca0b7fe640 1 -- 172.21.15.115:0/3120069194 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fca14155860 con 0x7fca14159dd0 2023-12-01T02:52:25.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.472+0000 7fca19084640 1 -- 172.21.15.115:0/3120069194 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca14155860 con 0x7fca14159dd0 2023-12-01T02:52:25.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.473+0000 7fca0b7fe640 1 -- 172.21.15.115:0/3120069194 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4661 (secure 0 0 0) 0x7fca14155860 con 0x7fca14159dd0 2023-12-01T02:52:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:25.474 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1616304732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1616304732},{"type":"v1","addr":"172.21.15.115:6835","nonce":1616304732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6766,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3109897397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3109897397},{"type":"v1","addr":"172.21.15.142:6834","nonce":3109897397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/446557816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":446557816},{"type":"v1","addr":"172.21.15.142:6835","nonce":446557816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:52:21.959726+0000","modified":"2023-12-01T02:52:24.991376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9549":{"gid":9549,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1064849689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1064849689},{"type":"v1","addr":"172.21.15.115:6837","nonce":1064849689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-01T02:52:25.475 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 79 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 -- 172.21.15.115:0/3120069194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9f406a550 msgr2=0x7fc9f406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9f406a550 0x7fc9f406ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca04036000 tx=0x7fca0403d040 comp rx=0 tx=0).stop 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 -- 172.21.15.115:0/3120069194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca14159dd0 msgr2=0x7fca140ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca14159dd0 0x7fca140ac8a0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00ef90 tx=0x7fca0c009f90 comp rx=0 tx=0).stop 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 -- 172.21.15.115:0/3120069194 shutdown_connections 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc9f406a550 0x7fc9f406ca10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca14070d30 0x7fca1414cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca14159dd0 0x7fca140ac8a0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 --2- 172.21.15.115:0/3120069194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1406b6f0 0x7fca140c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 -- 172.21.15.115:0/3120069194 >> 172.21.15.115:0/3120069194 conn(0x7fca140ce880 msgr2=0x7fca14157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 -- 172.21.15.115:0/3120069194 shutdown_connections 2023-12-01T02:52:25.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.476+0000 7fca19084640 1 -- 172.21.15.115:0/3120069194 wait complete. 2023-12-01T02:52:25.496 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T02:52:25.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.698+0000 7fb9f175c640 1 Processor -- start 2023-12-01T02:52:25.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.698+0000 7fb9f175c640 1 -- start start 2023-12-01T02:52:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.700+0000 7fb9f175c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb9ec15e2f0 0x7fb9ec0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.700+0000 7fb9f175c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9ec06f2f0 0x7fb9ec0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.700+0000 7fb9f175c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb9ec0adf10 0x7fb9ec0c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.700+0000 7fb9f175c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ec0add10 con 0x7fb9ec0adf10 2023-12-01T02:52:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.700+0000 7fb9f175c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ec0c5760 con 0x7fb9ec15e2f0 2023-12-01T02:52:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.700+0000 7fb9f175c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec15e6d0 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.700+0000 7fb9eb7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb9ec0adf10 0x7fb9ec0c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9eb7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb9ec0adf10 0x7fb9ec0c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55704/0 (socket says 172.21.15.115:55704) 2023-12-01T02:52:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/872661310 learned_addr learned my addr 172.21.15.115:0/872661310 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9e27fc640 1 --2- 172.21.15.115:0/872661310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9ec06f2f0 0x7fb9ec0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/872661310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9ec06f2f0 msgr2=0x7fb9ec0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9eaffd640 1 --2- 172.21.15.115:0/872661310 >> v2:172.21.15.142:3300/0 conn(0x7fb9ec15e2f0 0x7fb9ec0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9eb7fe640 1 --2- 172.21.15.115:0/872661310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9ec06f2f0 0x7fb9ec0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/872661310 >> v2:172.21.15.142:3300/0 conn(0x7fb9ec15e2f0 msgr2=0x7fb9ec0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9eb7fe640 1 --2- 172.21.15.115:0/872661310 >> v2:172.21.15.142:3300/0 conn(0x7fb9ec15e2f0 0x7fb9ec0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9eb7fe640 1 -- 172.21.15.115:0/872661310 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec0cf7b0 con 0x7fb9ec0adf10 2023-12-01T02:52:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9e27fc640 1 --2- 172.21.15.115:0/872661310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9ec06f2f0 0x7fb9ec0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:25.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.701+0000 7fb9eaffd640 1 --2- 172.21.15.115:0/872661310 >> v2:172.21.15.142:3300/0 conn(0x7fb9ec15e2f0 0x7fb9ec0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:25.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.702+0000 7fb9eb7fe640 1 --2- 172.21.15.115:0/872661310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec0adf10 0x7fb9ec0c8ed0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec06ef60 tx=0x7fb9e402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a75540970265e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:25.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.703+0000 7fb9ea7fc640 1 -- 172.21.15.115:0/872661310 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4031740 con 0x7fb9ec0adf10 2023-12-01T02:52:25.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.703+0000 7fb9ea7fc640 1 -- 172.21.15.115:0/872661310 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e40318a0 con 0x7fb9ec0adf10 2023-12-01T02:52:25.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.703+0000 7fb9ea7fc640 1 -- 172.21.15.115:0/872661310 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4031c30 con 0x7fb9ec0adf10 2023-12-01T02:52:25.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.703+0000 7fb9f175c640 1 -- 172.21.15.115:0/872661310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec0adf10 msgr2=0x7fb9ec0c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.703+0000 7fb9f175c640 1 --2- 172.21.15.115:0/872661310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec0adf10 0x7fb9ec0c8ed0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec06ef60 tx=0x7fb9e402b7e0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.703+0000 7fb9f175c640 1 -- 172.21.15.115:0/872661310 shutdown_connections 2023-12-01T02:52:25.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.703+0000 7fb9f175c640 1 --2- 172.21.15.115:0/872661310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec0adf10 0x7fb9ec0c8ed0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.703+0000 7fb9f175c640 1 --2- 172.21.15.115:0/872661310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9ec06f2f0 0x7fb9ec0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.703+0000 7fb9f175c640 1 --2- 172.21.15.115:0/872661310 >> v2:172.21.15.142:3300/0 conn(0x7fb9ec15e2f0 0x7fb9ec0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.703+0000 7fb9f175c640 1 -- 172.21.15.115:0/872661310 >> 172.21.15.115:0/872661310 conn(0x7fb9ec0ce880 msgr2=0x7fb9ec0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:25.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.704+0000 7fb9f175c640 1 -- 172.21.15.115:0/872661310 shutdown_connections 2023-12-01T02:52:25.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.704+0000 7fb9f175c640 1 -- 172.21.15.115:0/872661310 wait complete. 2023-12-01T02:52:25.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.705+0000 7fb9f175c640 1 Processor -- start 2023-12-01T02:52:25.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.705+0000 7fb9f175c640 1 -- start start 2023-12-01T02:52:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.706+0000 7fb9f175c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec06f2f0 0x7fb9ec0c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.706+0000 7fb9f175c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9ec0adf10 0x7fb9ec159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.706+0000 7fb9f175c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9ec15e2f0 0x7fb9ec0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.706+0000 7fb9f175c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0c8100 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.706+0000 7fb9f175c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0c83e0 con 0x7fb9ec15e2f0 2023-12-01T02:52:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.706+0000 7fb9f175c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0c34a0 con 0x7fb9ec0adf10 2023-12-01T02:52:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.707+0000 7fb9eaffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec06f2f0 0x7fb9ec0c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.707+0000 7fb9eaffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec06f2f0 0x7fb9ec0c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55706/0 (socket says 172.21.15.115:55706) 2023-12-01T02:52:25.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.707+0000 7fb9eaffd640 1 -- 172.21.15.115:0/3241072254 learned_addr learned my addr 172.21.15.115:0/3241072254 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.707+0000 7fb9eb7fe640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9ec15e2f0 0x7fb9ec0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.707+0000 7fb9eaffd640 1 -- 172.21.15.115:0/3241072254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9ec0adf10 msgr2=0x7fb9ec159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.707+0000 7fb9eaffd640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9ec0adf10 0x7fb9ec159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.707+0000 7fb9eaffd640 1 -- 172.21.15.115:0/3241072254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9ec15e2f0 msgr2=0x7fb9ec0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.707+0000 7fb9eaffd640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9ec15e2f0 0x7fb9ec0cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.707+0000 7fb9eaffd640 1 -- 172.21.15.115:0/3241072254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e40094d0 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.707+0000 7fb9eaffd640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec06f2f0 0x7fb9ec0c7b20 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00c7f0 tx=0x7fb9dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.708+0000 7fb9e3fff640 1 -- 172.21.15.115:0/3241072254 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc007820 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.708+0000 7fb9e3fff640 1 -- 172.21.15.115:0/3241072254 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc007980 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.708+0000 7fb9e3fff640 1 -- 172.21.15.115:0/3241072254 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc007cf0 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.708+0000 7fb9f175c640 1 -- 172.21.15.115:0/3241072254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec0c3780 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.708+0000 7fb9f175c640 1 -- 172.21.15.115:0/3241072254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec15a510 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.709+0000 7fb9f175c640 1 -- 172.21.15.115:0/3241072254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec06fda0 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.710+0000 7fb9e3fff640 1 -- 172.21.15.115:0/3241072254 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb9dc0113f0 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.711+0000 7fb9e3fff640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9cc06a550 0x7fb9cc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:25.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.712+0000 7fb9e3fff640 1 -- 172.21.15.115:0/3241072254 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fb9dc088e70 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.712+0000 7fb9e27fc640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9cc06a550 0x7fb9cc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:25.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.712+0000 7fb9e27fc640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9cc06a550 0x7fb9cc06ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4007130 tx=0x7fb9d4007300 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:25.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.716+0000 7fb9e3fff640 1 -- 172.21.15.115:0/3241072254 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb9dc0542a0 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.903+0000 7fb9f175c640 1 -- 172.21.15.115:0/3241072254 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9ec04e890 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.906+0000 7fb9e3fff640 1 -- 172.21.15.115:0/3241072254 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+16229 (secure 0 0 0) 0x7fb9dc01f090 con 0x7fb9ec06f2f0 2023-12-01T02:52:25.907 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:25.907 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":177,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T02:52:22.969012+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:52:20.047003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-01T02:52:20.974031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T02:52:25.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.908+0000 7fb9f175c640 1 -- 172.21.15.115:0/3241072254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9cc06a550 msgr2=0x7fb9cc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.908+0000 7fb9f175c640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9cc06a550 0x7fb9cc06ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4007130 tx=0x7fb9d4007300 comp rx=0 tx=0).stop 2023-12-01T02:52:25.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.908+0000 7fb9f175c640 1 -- 172.21.15.115:0/3241072254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec06f2f0 msgr2=0x7fb9ec0c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:25.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.908+0000 7fb9f175c640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec06f2f0 0x7fb9ec0c7b20 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00c7f0 tx=0x7fb9dc00ccb0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.908+0000 7fb9f175c640 1 -- 172.21.15.115:0/3241072254 shutdown_connections 2023-12-01T02:52:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.908+0000 7fb9f175c640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fb9cc06a550 0x7fb9cc06ca10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.908+0000 7fb9f175c640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9ec0adf10 0x7fb9ec159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.908+0000 7fb9f175c640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9ec15e2f0 0x7fb9ec0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.908+0000 7fb9f175c640 1 --2- 172.21.15.115:0/3241072254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb9ec06f2f0 0x7fb9ec0c7b20 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.908+0000 7fb9f175c640 1 -- 172.21.15.115:0/3241072254 >> 172.21.15.115:0/3241072254 conn(0x7fb9ec0ce880 msgr2=0x7fb9ec15ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.909+0000 7fb9f175c640 1 -- 172.21.15.115:0/3241072254 shutdown_connections 2023-12-01T02:52:25.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:25.909+0000 7fb9f175c640 1 -- 172.21.15.115:0/3241072254 wait complete. 2023-12-01T02:52:25.926 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T02:52:25.927 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T02:52:26.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.129+0000 7f1e14992640 1 Processor -- start 2023-12-01T02:52:26.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.129+0000 7f1e14992640 1 -- start start 2023-12-01T02:52:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.130+0000 7f1e14992640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1e1015e2f0 0x7f1e100ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.130+0000 7f1e14992640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e100add10 con 0x7f1e100adf10 2023-12-01T02:52:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.130+0000 7f1e14992640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e100c5760 con 0x7f1e1006f2f0 2023-12-01T02:52:26.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.131+0000 7f1e14992640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e100c9cf0 con 0x7f1e1015e2f0 2023-12-01T02:52:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.131+0000 7f1e0ed9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1e100adf10 0x7f1e100c95e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54078/0 (socket says 172.21.15.115:54078) 2023-12-01T02:52:26.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.131+0000 7f1e0ed9d640 1 -- 172.21.15.115:0/200899977 learned_addr learned my addr 172.21.15.115:0/200899977 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:26.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.131+0000 7f1e0e59c640 1 --2- 172.21.15.115:0/200899977 >> v2:172.21.15.142:3300/0 conn(0x7f1e1015e2f0 0x7f1e100ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.131+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025916109 0 0) 0x7f1e100add10 con 0x7f1e100adf10 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.131+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dfc003520 con 0x7f1e100adf10 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1008660151 0 0) 0x7f1dfc003520 con 0x7f1e100adf10 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 >> v2:172.21.15.142:3300/0 conn(0x7f1e1015e2f0 msgr2=0x7f1e100ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 --2- 172.21.15.115:0/200899977 >> v2:172.21.15.142:3300/0 conn(0x7f1e1015e2f0 0x7f1e100ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 >> v1:172.21.15.142:6789/0 conn(0x7f1e1006f2f0 legacy=0x7f1e100c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1015e6d0 con 0x7f1e100adf10 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f1e080019f0 con 0x7f1e100adf10 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 >> v1:172.21.15.115:6789/0 conn(0x7f1e100adf10 legacy=0x7f1e100c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 --2- 172.21.15.115:0/200899977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dfc0039e0 0x7f1dfc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 --2- 172.21.15.115:0/200899977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dfc0047c0 0x7f1dfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 --2- 172.21.15.115:0/200899977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dfc005280 0x7f1dfc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100c9cf0 con 0x7f1dfc0047c0 2023-12-01T02:52:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc00c1c0 con 0x7f1dfc0039e0 2023-12-01T02:52:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc00c380 con 0x7f1dfc005280 2023-12-01T02:52:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e07fff640 1 --2- 172.21.15.115:0/200899977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dfc0047c0 0x7f1dfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e0e59c640 1 --2- 172.21.15.115:0/200899977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dfc0039e0 0x7f1dfc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e07fff640 1 -- 172.21.15.115:0/200899977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dfc005280 msgr2=0x7f1dfc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e07fff640 1 --2- 172.21.15.115:0/200899977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dfc005280 0x7f1dfc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e07fff640 1 -- 172.21.15.115:0/200899977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dfc0039e0 msgr2=0x7f1dfc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e07fff640 1 --2- 172.21.15.115:0/200899977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dfc0039e0 0x7f1dfc003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.132+0000 7f1e07fff640 1 -- 172.21.15.115:0/200899977 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc003520 con 0x7f1dfc0047c0 2023-12-01T02:52:26.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.133+0000 7f1e0ed9d640 1 -- 172.21.15.115:0/200899977 reap_dead start 2023-12-01T02:52:26.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.134+0000 7f1e07fff640 1 --2- 172.21.15.115:0/200899977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dfc0047c0 0x7f1dfc004c40 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f1df8001750 tx=0x7f1dfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3f172d731c6e0f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:26.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.134+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df80052d0 con 0x7f1dfc0047c0 2023-12-01T02:52:26.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.134+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df802e050 con 0x7f1dfc0047c0 2023-12-01T02:52:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.134+0000 7f1e0dd9b640 1 -- 172.21.15.115:0/200899977 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8003830 con 0x7f1dfc0047c0 2023-12-01T02:52:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.134+0000 7f1e14992640 1 -- 172.21.15.115:0/200899977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dfc0047c0 msgr2=0x7f1dfc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.134+0000 7f1e14992640 1 --2- 172.21.15.115:0/200899977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dfc0047c0 0x7f1dfc004c40 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f1df8001750 tx=0x7f1dfc0057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.135+0000 7f1e14992640 1 -- 172.21.15.115:0/200899977 shutdown_connections 2023-12-01T02:52:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.135+0000 7f1e14992640 1 --2- 172.21.15.115:0/200899977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1dfc0047c0 0x7f1dfc004c40 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.135+0000 7f1e14992640 1 -- 172.21.15.115:0/200899977 >> 172.21.15.115:0/200899977 conn(0x7f1e100ce880 msgr2=0x7f1e100cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.135+0000 7f1e14992640 1 -- 172.21.15.115:0/200899977 shutdown_connections 2023-12-01T02:52:26.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.135+0000 7f1e14992640 1 -- 172.21.15.115:0/200899977 wait complete. 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.136+0000 7f1e14992640 1 Processor -- start 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.136+0000 7f1e14992640 1 -- start start 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.137+0000 7f1e14992640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dfc0047c0 0x7f1e101f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.137+0000 7f1e14992640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e10155c60 0x7f1e100cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.137+0000 7f1e14992640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e100cdb10 0x7f1e100c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.137+0000 7f1e14992640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100cdf90 con 0x7f1e100cdb10 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.137+0000 7f1e14992640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e101f2ca0 con 0x7f1e10155c60 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.137+0000 7f1e14992640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100c7b20 con 0x7f1dfc0047c0 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.137+0000 7f1e0ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e100cdb10 0x7f1e100c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e0ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e100cdb10 0x7f1e100c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55730/0 (socket says 172.21.15.115:55730) 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e0ed9d640 1 -- 172.21.15.115:0/2756761302 learned_addr learned my addr 172.21.15.115:0/2756761302 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e07fff640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e10155c60 0x7f1e100cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e0e59c640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dfc0047c0 0x7f1e101f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e0ed9d640 1 -- 172.21.15.115:0/2756761302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dfc0047c0 msgr2=0x7f1e101f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e0ed9d640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dfc0047c0 0x7f1e101f2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e0ed9d640 1 -- 172.21.15.115:0/2756761302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e10155c60 msgr2=0x7f1e100cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e0ed9d640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e10155c60 0x7f1e100cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e0ed9d640 1 -- 172.21.15.115:0/2756761302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100c24e0 con 0x7f1e100cdb10 2023-12-01T02:52:26.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e0ed9d640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e100cdb10 0x7f1e100c6af0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f1e100c8e50 tx=0x7f1e0800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:26.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e06ffd640 1 -- 172.21.15.115:0/2756761302 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800cee0 con 0x7f1e100cdb10 2023-12-01T02:52:26.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e06ffd640 1 -- 172.21.15.115:0/2756761302 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0800a610 con 0x7f1e100cdb10 2023-12-01T02:52:26.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e14992640 1 -- 172.21.15.115:0/2756761302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e100c7840 con 0x7f1e100cdb10 2023-12-01T02:52:26.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.138+0000 7f1e06ffd640 1 -- 172.21.15.115:0/2756761302 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800a9c0 con 0x7f1e100cdb10 2023-12-01T02:52:26.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.139+0000 7f1e14992640 1 -- 172.21.15.115:0/2756761302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e100c2a60 con 0x7f1e100cdb10 2023-12-01T02:52:26.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.140+0000 7f1e14992640 1 -- 172.21.15.115:0/2756761302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1006fda0 con 0x7f1e100cdb10 2023-12-01T02:52:26.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.140+0000 7f1e06ffd640 1 -- 172.21.15.115:0/2756761302 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1e0800ab20 con 0x7f1e100cdb10 2023-12-01T02:52:26.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.141+0000 7f1e06ffd640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1df006a550 0x7f1df006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.141+0000 7f1e06ffd640 1 -- 172.21.15.115:0/2756761302 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f1e08087e70 con 0x7f1e100cdb10 2023-12-01T02:52:26.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.142+0000 7f1e0e59c640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1df006a550 0x7f1df006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.142+0000 7f1e0e59c640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1df006a550 0x7f1df006ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009910 tx=0x7f1e00008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:26.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.145+0000 7f1e06ffd640 1 -- 172.21.15.115:0/2756761302 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1e08056e40 con 0x7f1e100cdb10 2023-12-01T02:52:26.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.331+0000 7f1e14992640 1 -- 172.21.15.115:0/2756761302 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e100cf4d0 con 0x7f1df006a550 2023-12-01T02:52:26.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.348+0000 7f1e06ffd640 1 -- 172.21.15.115:0/2756761302 <== mgr.9297 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280455 (secure 0 0 0) 0x7f1e000118e0 con 0x7f1df006a550 2023-12-01T02:52:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:26.355 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 -- 172.21.15.115:0/2756761302 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1df006a550 msgr2=0x7f1df006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1df006a550 0x7f1df006ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009910 tx=0x7f1e00008040 comp rx=0 tx=0).stop 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 -- 172.21.15.115:0/2756761302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e100cdb10 msgr2=0x7f1e100c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e100cdb10 0x7f1e100c6af0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f1e100c8e50 tx=0x7f1e0800cc30 comp rx=0 tx=0).stop 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 -- 172.21.15.115:0/2756761302 shutdown_connections 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1df006a550 0x7f1df006ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e100cdb10 0x7f1e100c6af0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e10155c60 0x7f1e100cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 --2- 172.21.15.115:0/2756761302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dfc0047c0 0x7f1e101f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 -- 172.21.15.115:0/2756761302 >> 172.21.15.115:0/2756761302 conn(0x7f1e100ce880 msgr2=0x7f1e100ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:26.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 -- 172.21.15.115:0/2756761302 shutdown_connections 2023-12-01T02:52:26.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.356+0000 7f1e14992640 1 -- 172.21.15.115:0/2756761302 wait complete. 2023-12-01T02:52:26.377 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T02:52:25.491335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":393608,"kb_used_data":18120,"kb_used_omap":0,"kb_used_meta":375424,"kb_avail":754581112,"statfs":{"total":773094113280,"available":772691058688,"internally_reserved":0,"allocated":18554880,"data_stored":9550186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":384434176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.018808"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.491054+0000","last_change":"2023-12-01T02:52:21.491054+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.491054+0000","last_undegraded":"2023-12-01T02:52:21.491054+0000","last_fullsized":"2023-12-01T02:52:21.491054+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:31.174876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.490218+0000","last_change":"2023-12-01T02:52:21.490218+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.490218+0000","last_undegraded":"2023-12-01T02:52:21.490218+0000","last_fullsized":"2023-12-01T02:52:21.490218+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:10.320275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.500819+0000","last_change":"2023-12-01T02:52:21.500819+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.500819+0000","last_undegraded":"2023-12-01T02:52:21.500819+0000","last_fullsized":"2023-12-01T02:52:21.500819+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:10.320275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.485590+0000","last_change":"2023-12-01T02:52:21.485590+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.485590+0000","last_undegraded":"2023-12-01T02:52:21.485590+0000","last_fullsized":"2023-12-01T02:52:21.485590+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:58.071226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.505503+0000","last_change":"2023-12-01T02:52:21.505503+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.505503+0000","last_undegraded":"2023-12-01T02:52:21.505503+0000","last_fullsized":"2023-12-01T02:52:21.505503+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:54.211416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487545+0000","last_change":"2023-12-01T02:52:21.487545+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487545+0000","last_undegraded":"2023-12-01T02:52:21.487545+0000","last_fullsized":"2023-12-01T02:52:21.487545+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:16:32.491266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.488522+0000","last_change":"2023-12-01T02:52:21.488522+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488522+0000","last_undegraded":"2023-12-01T02:52:21.488522+0000","last_fullsized":"2023-12-01T02:52:21.488522+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:47:40.922670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.504618+0000","last_change":"2023-12-01T02:52:21.504618+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.504618+0000","last_undegraded":"2023-12-01T02:52:21.504618+0000","last_fullsized":"2023-12-01T02:52:21.504618+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:37.307890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.496757+0000","last_change":"2023-12-01T02:52:21.496757+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.496757+0000","last_undegraded":"2023-12-01T02:52:21.496757+0000","last_fullsized":"2023-12-01T02:52:21.496757+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:59.055845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.493670+0000","last_change":"2023-12-01T02:52:21.493670+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.493670+0000","last_undegraded":"2023-12-01T02:52:21.493670+0000","last_fullsized":"2023-12-01T02:52:21.493670+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:43.800296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.492259+0000","last_change":"2023-12-01T02:52:21.492259+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.492259+0000","last_undegraded":"2023-12-01T02:52:21.492259+0000","last_fullsized":"2023-12-01T02:52:21.492259+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:16.098248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.490155+0000","last_change":"2023-12-01T02:52:21.490155+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.490155+0000","last_undegraded":"2023-12-01T02:52:21.490155+0000","last_fullsized":"2023-12-01T02:52:21.490155+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:04:48.367542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487045+0000","last_change":"2023-12-01T02:52:21.487045+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487045+0000","last_undegraded":"2023-12-01T02:52:21.487045+0000","last_fullsized":"2023-12-01T02:52:21.487045+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.486225+0000","last_change":"2023-12-01T02:52:21.486225+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.486225+0000","last_undegraded":"2023-12-01T02:52:21.486225+0000","last_fullsized":"2023-12-01T02:52:21.486225+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:24.096134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.492521+0000","last_change":"2023-12-01T02:52:21.492521+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.492521+0000","last_undegraded":"2023-12-01T02:52:21.492521+0000","last_fullsized":"2023-12-01T02:52:21.492521+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:38.297728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.488879+0000","last_change":"2023-12-01T02:52:21.488879+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488879+0000","last_undegraded":"2023-12-01T02:52:21.488879+0000","last_fullsized":"2023-12-01T02:52:21.488879+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:52:02.202035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.503971+0000","last_change":"2023-12-01T02:52:21.503971+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.503971+0000","last_undegraded":"2023-12-01T02:52:21.503971+0000","last_fullsized":"2023-12-01T02:52:21.503971+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:19:07.126182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481702+0000","last_change":"2023-12-01T02:52:20.481702+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481702+0000","last_undegraded":"2023-12-01T02:52:20.481702+0000","last_fullsized":"2023-12-01T02:52:20.481702+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:51.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486539+0000","last_change":"2023-12-01T02:52:20.486539+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486539+0000","last_undegraded":"2023-12-01T02:52:20.486539+0000","last_fullsized":"2023-12-01T02:52:20.486539+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481432+0000","last_change":"2023-12-01T02:52:20.481432+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481432+0000","last_undegraded":"2023-12-01T02:52:20.481432+0000","last_fullsized":"2023-12-01T02:52:20.481432+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:18.775753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.499930+0000","last_change":"2023-12-01T02:52:21.499930+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.499930+0000","last_undegraded":"2023-12-01T02:52:21.499930+0000","last_fullsized":"2023-12-01T02:52:21.499930+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:39.498194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481197+0000","last_change":"2023-12-01T02:52:20.481197+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481197+0000","last_undegraded":"2023-12-01T02:52:20.481197+0000","last_fullsized":"2023-12-01T02:52:20.481197+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:49.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.489389+0000","last_change":"2023-12-01T02:52:21.489389+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.489389+0000","last_undegraded":"2023-12-01T02:52:21.489389+0000","last_fullsized":"2023-12-01T02:52:21.489389+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:57.637709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.485885+0000","last_change":"2023-12-01T02:52:21.485885+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.485885+0000","last_undegraded":"2023-12-01T02:52:21.485885+0000","last_fullsized":"2023-12-01T02:52:21.485885+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:40.990842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.502490+0000","last_change":"2023-12-01T02:52:21.502490+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502490+0000","last_undegraded":"2023-12-01T02:52:21.502490+0000","last_fullsized":"2023-12-01T02:52:21.502490+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:42.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.498065+0000","last_change":"2023-12-01T02:52:21.498065+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.498065+0000","last_undegraded":"2023-12-01T02:52:21.498065+0000","last_fullsized":"2023-12-01T02:52:21.498065+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:34.788423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487675+0000","last_change":"2023-12-01T02:52:21.487675+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487675+0000","last_undegraded":"2023-12-01T02:52:21.487675+0000","last_fullsized":"2023-12-01T02:52:21.487675+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:04.052509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.485774+0000","last_change":"2023-12-01T02:52:20.485774+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.485774+0000","last_undegraded":"2023-12-01T02:52:20.485774+0000","last_fullsized":"2023-12-01T02:52:20.485774+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:22.856154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.501546+0000","last_change":"2023-12-01T02:52:21.501546+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.501546+0000","last_undegraded":"2023-12-01T02:52:21.501546+0000","last_fullsized":"2023-12-01T02:52:21.501546+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.482235+0000","last_change":"2023-12-01T02:52:20.482235+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.482235+0000","last_undegraded":"2023-12-01T02:52:20.482235+0000","last_fullsized":"2023-12-01T02:52:20.482235+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:48:39.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.505781+0000","last_change":"2023-12-01T02:52:21.505781+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.505781+0000","last_undegraded":"2023-12-01T02:52:21.505781+0000","last_fullsized":"2023-12-01T02:52:21.505781+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:32.342375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486360+0000","last_change":"2023-12-01T02:52:20.486360+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486360+0000","last_undegraded":"2023-12-01T02:52:20.486360+0000","last_fullsized":"2023-12-01T02:52:20.486360+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:08.328341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.504430+0000","last_change":"2023-12-01T02:52:21.504430+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.504430+0000","last_undegraded":"2023-12-01T02:52:21.504430+0000","last_fullsized":"2023-12-01T02:52:21.504430+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:46.753790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.487600+0000","last_change":"2023-12-01T02:52:20.487600+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.487600+0000","last_undegraded":"2023-12-01T02:52:20.487600+0000","last_fullsized":"2023-12-01T02:52:20.487600+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:57.056061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.504912+0000","last_change":"2023-12-01T02:52:21.504912+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.504912+0000","last_undegraded":"2023-12-01T02:52:21.504912+0000","last_fullsized":"2023-12-01T02:52:21.504912+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:41.511561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.480601+0000","last_change":"2023-12-01T02:52:20.480601+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.480601+0000","last_undegraded":"2023-12-01T02:52:20.480601+0000","last_fullsized":"2023-12-01T02:52:20.480601+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:16.375606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487348+0000","last_change":"2023-12-01T02:52:21.487348+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487348+0000","last_undegraded":"2023-12-01T02:52:21.487348+0000","last_fullsized":"2023-12-01T02:52:21.487348+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:34.006926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.484366+0000","last_change":"2023-12-01T02:52:21.484366+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.484366+0000","last_undegraded":"2023-12-01T02:52:21.484366+0000","last_fullsized":"2023-12-01T02:52:21.484366+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:37.282563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.506646+0000","last_change":"2023-12-01T02:52:21.506646+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.506646+0000","last_undegraded":"2023-12-01T02:52:21.506646+0000","last_fullsized":"2023-12-01T02:52:21.506646+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:38.746327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.493462+0000","last_change":"2023-12-01T02:52:21.493462+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.493462+0000","last_undegraded":"2023-12-01T02:52:21.493462+0000","last_fullsized":"2023-12-01T02:52:21.493462+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:11:02.383031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.499995+0000","last_change":"2023-12-01T02:52:21.499995+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.499995+0000","last_undegraded":"2023-12-01T02:52:21.499995+0000","last_fullsized":"2023-12-01T02:52:21.499995+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:38.297728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.485211+0000","last_change":"2023-12-01T02:52:21.485211+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.485211+0000","last_undegraded":"2023-12-01T02:52:21.485211+0000","last_fullsized":"2023-12-01T02:52:21.485211+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:24:04.462724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.503442+0000","last_change":"2023-12-01T02:52:21.503442+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.503442+0000","last_undegraded":"2023-12-01T02:52:21.503442+0000","last_fullsized":"2023-12-01T02:52:21.503442+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:17:54.151789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.502652+0000","last_change":"2023-12-01T02:52:21.502652+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502652+0000","last_undegraded":"2023-12-01T02:52:21.502652+0000","last_fullsized":"2023-12-01T02:52:21.502652+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:56.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.492178+0000","last_change":"2023-12-01T02:52:21.492178+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.492178+0000","last_undegraded":"2023-12-01T02:52:21.492178+0000","last_fullsized":"2023-12-01T02:52:21.492178+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:39.498194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.488796+0000","last_change":"2023-12-01T02:52:21.488796+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488796+0000","last_undegraded":"2023-12-01T02:52:21.488796+0000","last_fullsized":"2023-12-01T02:52:21.488796+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.502542+0000","last_change":"2023-12-01T02:52:21.502542+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502542+0000","last_undegraded":"2023-12-01T02:52:21.502542+0000","last_fullsized":"2023-12-01T02:52:21.502542+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:49.683912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.488038+0000","last_change":"2023-12-01T02:52:20.488038+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.488038+0000","last_undegraded":"2023-12-01T02:52:20.488038+0000","last_fullsized":"2023-12-01T02:52:20.488038+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:23.080969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.488165+0000","last_change":"2023-12-01T02:52:21.488165+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488165+0000","last_undegraded":"2023-12-01T02:52:21.488165+0000","last_fullsized":"2023-12-01T02:52:21.488165+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:44:46.015030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.495589+0000","last_change":"2023-12-01T02:52:21.495589+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.495589+0000","last_undegraded":"2023-12-01T02:52:21.495589+0000","last_fullsized":"2023-12-01T02:52:21.495589+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:18.077875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.485633+0000","last_change":"2023-12-01T02:52:20.485633+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.485633+0000","last_undegraded":"2023-12-01T02:52:20.485633+0000","last_fullsized":"2023-12-01T02:52:20.485633+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:36.292725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487161+0000","last_change":"2023-12-01T02:52:21.487161+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487161+0000","last_undegraded":"2023-12-01T02:52:21.487161+0000","last_fullsized":"2023-12-01T02:52:21.487161+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:24.957113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.499966+0000","last_change":"2023-12-01T02:52:21.499966+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.499966+0000","last_undegraded":"2023-12-01T02:52:21.499966+0000","last_fullsized":"2023-12-01T02:52:21.499966+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:30.297120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.494693+0000","last_change":"2023-12-01T02:52:21.494693+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.494693+0000","last_undegraded":"2023-12-01T02:52:21.494693+0000","last_fullsized":"2023-12-01T02:52:21.494693+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.500840+0000","last_change":"2023-12-01T02:52:21.500840+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.500840+0000","last_undegraded":"2023-12-01T02:52:21.500840+0000","last_fullsized":"2023-12-01T02:52:21.500840+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:52.712427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.489829+0000","last_change":"2023-12-01T02:52:21.489829+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.489829+0000","last_undegraded":"2023-12-01T02:52:21.489829+0000","last_fullsized":"2023-12-01T02:52:21.489829+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:31.312285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.508458+0000","last_change":"2023-12-01T02:52:21.508458+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.508458+0000","last_undegraded":"2023-12-01T02:52:21.508458+0000","last_fullsized":"2023-12-01T02:52:21.508458+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:04:47.352377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.505348+0000","last_change":"2023-12-01T02:52:21.505348+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.505348+0000","last_undegraded":"2023-12-01T02:52:21.505348+0000","last_fullsized":"2023-12-01T02:52:21.505348+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:09.343506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.501801+0000","last_change":"2023-12-01T02:52:21.501801+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.501801+0000","last_undegraded":"2023-12-01T02:52:21.501801+0000","last_fullsized":"2023-12-01T02:52:21.501801+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:19.139617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.502268+0000","last_change":"2023-12-01T02:52:21.502268+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502268+0000","last_undegraded":"2023-12-01T02:52:21.502268+0000","last_fullsized":"2023-12-01T02:52:21.502268+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.491172+0000","last_change":"2023-12-01T02:52:21.491172+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.491172+0000","last_undegraded":"2023-12-01T02:52:21.491172+0000","last_fullsized":"2023-12-01T02:52:21.491172+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:36.014503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487296+0000","last_change":"2023-12-01T02:52:21.487296+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487296+0000","last_undegraded":"2023-12-01T02:52:21.487296+0000","last_fullsized":"2023-12-01T02:52:21.487296+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:51.425590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486216+0000","last_change":"2023-12-01T02:52:20.486216+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486216+0000","last_undegraded":"2023-12-01T02:52:20.486216+0000","last_fullsized":"2023-12-01T02:52:20.486216+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:50.410425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.488770+0000","last_change":"2023-12-01T02:52:21.488770+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488770+0000","last_undegraded":"2023-12-01T02:52:21.488770+0000","last_fullsized":"2023-12-01T02:52:21.488770+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:30:41.584477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486389+0000","last_change":"2023-12-01T02:52:20.486389+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486389+0000","last_undegraded":"2023-12-01T02:52:20.486389+0000","last_fullsized":"2023-12-01T02:52:20.486389+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:15:51.909423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":6,"reported_epoch":175,"state":"active","last_fresh":"2023-12-01T02:52:21.499662+0000","last_change":"2023-12-01T02:52:21.499540+0000","last_active":"2023-12-01T02:52:21.499662+0000","last_peered":"2023-12-01T02:52:21.499662+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"2023-12-01T02:52:21.499540+0000","last_became_peered":"2023-12-01T02:52:21.499540+0000","last_unstale":"2023-12-01T02:52:21.499662+0000","last_undegraded":"2023-12-01T02:52:21.499662+0000","last_fullsized":"2023-12-01T02:52:21.499662+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:14.533994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.500456+0000","last_change":"2023-12-01T02:52:21.500456+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.500456+0000","last_undegraded":"2023-12-01T02:52:21.500456+0000","last_fullsized":"2023-12-01T02:52:21.500456+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:43.800296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481543+0000","last_change":"2023-12-01T02:52:20.481543+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481543+0000","last_undegraded":"2023-12-01T02:52:20.481543+0000","last_fullsized":"2023-12-01T02:52:20.481543+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:39.608571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.503299+0000","last_change":"2023-12-01T02:52:21.503299+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.503299+0000","last_undegraded":"2023-12-01T02:52:21.503299+0000","last_fullsized":"2023-12-01T02:52:21.503299+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:52.569081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.484343+0000","last_change":"2023-12-01T02:52:21.484343+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.484343+0000","last_undegraded":"2023-12-01T02:52:21.484343+0000","last_fullsized":"2023-12-01T02:52:21.484343+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:09.305110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.485770+0000","last_change":"2023-12-01T02:52:20.485770+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.485770+0000","last_undegraded":"2023-12-01T02:52:20.485770+0000","last_fullsized":"2023-12-01T02:52:20.485770+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:31.327211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.484334+0000","last_change":"2023-12-01T02:52:21.484334+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.484334+0000","last_undegraded":"2023-12-01T02:52:21.484334+0000","last_fullsized":"2023-12-01T02:52:21.484334+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:42.785131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"165'271","reported_seq":879,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-01T02:52:20.487518+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T02:52:20.487518+0000","last_peered":"2023-12-01T02:52:20.487518+0000","last_clean":"2023-12-01T02:52:20.487518+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T02:52:20.487518+0000","last_undegraded":"2023-12-01T02:52:20.487518+0000","last_fullsized":"2023-12-01T02:52:20.487518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":80,"acting":80,"num_store_stats":5},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575762,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3799999999999999,"5min":1.7829999999999999,"15min":1.893},"last":1.171},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1539999999999999,"15min":1.1100000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.514,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.91900000000000004}]},{"osd":1,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.266,"15min":1.2709999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.02},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.26,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85399999999999998},"max":{"1min":1.585,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":1.095}]},{"osd":2,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.2010000000000001},"min":{"1min":0.82499999999999996,"5min":0.80600000000000005,"15min":0.78500000000000003},"max":{"1min":1.554,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.252,"5min":1.28,"15min":1.25},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8440000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.343,"15min":1.3220000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.742,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.048,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.038}]},{"osd":4,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.851,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.506,"5min":1.4930000000000001,"15min":1.4750000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.157,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.276}]},{"osd":5,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3149999999999999,"15min":1.284},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.0699999999999998,"5min":2.976,"15min":2.976},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.595,"15min":1.569},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.2160000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.363}]},{"osd":6,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4590000000000001,"15min":1.4179999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.1499999999999999,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.21},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4870000000000001,"15min":1.4470000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.966,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.244}]}]},{"osd":6,"up_from":13,"seq":55834575763,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.161,"15min":1.165},"min":{"1min":0.84299999999999997,"5min":0.78100000000000003,"15min":0.75},"max":{"1min":1.6080000000000001,"5min":1.73,"15min":2.2759999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.1439999999999999,"15min":1.1619999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.56799999999999995},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.966},"last":1.4099999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1459999999999999,"15min":1.1859999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.78800000000000003},"max":{"1min":1.6459999999999999,"5min":1.772,"15min":2.0409999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.2,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6679999999999999,"5min":1.7370000000000001,"15min":2.093},"last":1.3460000000000001}]},{"osd":2,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1679999999999999,"15min":1.1879999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.80600000000000005},"max":{"1min":1.5369999999999999,"5min":1.736,"15min":1.946},"last":1.593},{"interface":"front","average":{"1min":1.274,"5min":1.272,"15min":1.284},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.637,"5min":1.7190000000000001,"15min":2.1230000000000002},"last":1.4810000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":1.0620000000000001,"5min":1.006,"15min":0.90500000000000003},"max":{"1min":1.6140000000000001,"5min":1.819,"15min":1.819},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.431,"15min":1.4319999999999999},"min":{"1min":1.042,"5min":0.98699999999999999,"15min":0.92800000000000005},"max":{"1min":1.738,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.556}]},{"osd":4,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.262},"min":{"1min":0.93000000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.698},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3500000000000001,"15min":1.3360000000000001},"min":{"1min":0.96699999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.98,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.4330000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.4419999999999999,"15min":1.4279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.5,"15min":1.4970000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.7799999999999998},"last":1.25}]},{"osd":7,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.5149999999999999,"15min":1.524},"min":{"1min":1.0980000000000001,"5min":1.034,"15min":1.034},"max":{"1min":1.9419999999999999,"5min":2.1549999999999998,"15min":2.5659999999999998},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6040000000000001,"15min":1.5940000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.0499999999999998,"5min":2.226,"15min":2.3809999999999998},"last":1.6559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575762,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59976,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311864,"statfs":{"total":96636764160,"available":96575348736,"internally_reserved":0,"allocated":2752512,"data_stored":1630631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1499999999999999,"15min":1.1419999999999999},"min":{"1min":0.68700000000000006,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4810000000000001,"5min":1.73,"15min":1.744},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1519999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.137,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.56200000000000006},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.704},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.226,"15min":1.2330000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.016},"last":1.548}]},{"osd":2,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2270000000000001,"15min":1.24},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5900000000000001,"5min":1.7430000000000001,"15min":1.746},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2609999999999999,"15min":1.2809999999999999},"min":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.627},"max":{"1min":1.655,"5min":1.671,"15min":1.764},"last":1.5940000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.321,"15min":1.319},"min":{"1min":0.98399999999999999,"5min":0.872,"15min":0.84899999999999998},"max":{"1min":1.546,"5min":1.7729999999999999,"15min":1.871},"last":0.996},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3460000000000001,"15min":1.3620000000000001},"min":{"1min":1.0269999999999999,"5min":0.91500000000000004,"15min":0.879},"max":{"1min":1.7250000000000001,"5min":1.7470000000000001,"15min":1.962},"last":1.417}]},{"osd":4,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.254,"15min":1.2569999999999999},"min":{"1min":0.91300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.696,"5min":1.847,"15min":1.978},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.391,"5min":1.361,"15min":1.3620000000000001},"min":{"1min":1.0740000000000001,"5min":0.91600000000000004,"15min":0.88600000000000001},"max":{"1min":1.911,"5min":1.911,"15min":2.0350000000000001},"last":1.7270000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3520000000000001,"15min":1.375},"min":{"1min":0.89300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":2.4470000000000001},"last":1.494},{"interface":"front","average":{"1min":1.411,"5min":1.391,"15min":1.419},"min":{"1min":1.077,"5min":0.876,"15min":0.876},"max":{"1min":1.798,"5min":1.946,"15min":2.7210000000000001},"last":1.325}]},{"osd":7,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.5449999999999999,"15min":1.5369999999999999},"min":{"1min":1.1200000000000001,"5min":1.099,"15min":0.92100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":3.0379999999999998},"last":1.464},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5780000000000001,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":3.5270000000000001},"last":1.7729999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575762,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47120,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324720,"statfs":{"total":96636764160,"available":96588513280,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.113,"15min":1.137},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.6180000000000001},"last":1.153},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1599999999999999,"15min":1.1699999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.3899999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.234,"15min":1.2270000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.48,"5min":1.706,"15min":1.7709999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.139,"15min":1.1830000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.381,"5min":1.6100000000000001,"15min":1.8640000000000001},"last":1.296}]},{"osd":2,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1910000000000001,"15min":1.218},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":2.1499999999999999},"last":1.472},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3240000000000001,"15min":1.3169999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.85999999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.1220000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2470000000000001,"15min":1.2609999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.454,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.016},{"interface":"front","average":{"1min":1.298,"5min":1.3260000000000001,"15min":1.329},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":2.2440000000000002},"last":1.343}]},{"osd":5,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.367,"15min":1.3140000000000001},"min":{"1min":0.92300000000000004,"5min":0.86499999999999999,"15min":0.84499999999999997},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3069999999999999,"15min":1.284},"min":{"1min":0.96699999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.712,"5min":1.875,"15min":1.875},"last":1.4430000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3580000000000001,"15min":1.325},"min":{"1min":0.82599999999999996,"5min":0.81299999999999994,"15min":0.78500000000000003},"max":{"1min":1.996,"5min":1.996,"15min":2.4489999999999998},"last":1.609},{"interface":"front","average":{"1min":1.554,"5min":1.474,"15min":1.448},"min":{"1min":1.097,"5min":1.014,"15min":0.97899999999999998},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":1.708}]},{"osd":7,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.444,"15min":1.4379999999999999},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.92900000000000005},"max":{"1min":1.905,"5min":2.044,"15min":2.044},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.552,"15min":1.544},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.8859999999999999,"5min":2.0670000000000002,"15min":2.5760000000000001},"last":1.671}]}]},{"osd":3,"up_from":13,"seq":55834575761,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51216,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320624,"statfs":{"total":96636764160,"available":96584318976,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.90600000000000003,"15min":0.89300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.0309999999999999,"15min":1.0660000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3680000000000001,"5min":1.5449999999999999,"15min":1.5669999999999999},"last":1.081}]},{"osd":1,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.1459999999999999,"15min":1.145},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2709999999999999,"5min":1.708,"15min":1.708},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.97799999999999998,"15min":1.004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.329,"5min":1.5980000000000001,"15min":1.744},"last":1.3879999999999999}]},{"osd":2,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0800000000000001,"15min":1.097},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.171},{"interface":"front","average":{"1min":1.028,"5min":1.2050000000000001,"15min":1.2490000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4550000000000001,"5min":1.5740000000000001,"15min":1.728},"last":1.1970000000000001}]},{"osd":4,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.302,"15min":1.294},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.306,"15min":1.2949999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.67100000000000004},"max":{"1min":1.802,"5min":1.802,"15min":1.927},"last":1.127}]},{"osd":5,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.294},{"interface":"front","average":{"1min":1.377,"5min":1.3979999999999999,"15min":1.387},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.9769999999999999,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.482}]},{"osd":6,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.448,"15min":1.4379999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.67900000000000005},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.524,"15min":1.4970000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.554}]},{"osd":7,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.919,"5min":2.0790000000000002,"15min":2.1549999999999998},"last":1.673},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.5109999999999999,"15min":1.51},"min":{"1min":0.747,"5min":0.747,"15min":0.73699999999999999},"max":{"1min":1.827,"5min":2.0150000000000001,"15min":2.1259999999999999},"last":1.645}]}]},{"osd":2,"up_from":13,"seq":55834575761,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41360,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330480,"statfs":{"total":96636764160,"available":96594411520,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":0.93200000000000005,"15min":0.88900000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.49299999999999999},"max":{"1min":5.2080000000000002,"5min":5.2080000000000002,"15min":5.2080000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.127,"5min":1.0860000000000001,"15min":1.073},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.3959999999999999,"5min":5.3959999999999999,"15min":5.3959999999999999},"last":0.88400000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1160000000000001,"15min":1.085},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":5.1109999999999998,"5min":5.1109999999999998,"15min":5.1109999999999998},"last":1.071},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.1619999999999999,"15min":1.1259999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":5.4800000000000004,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.105}]},{"osd":3,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.073,"15min":1.0549999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.0540000000000003,"5min":4.0540000000000003,"15min":4.0540000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.2869999999999999,"15min":1.2230000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":5.7690000000000001,"5min":5.7690000000000001,"15min":5.7690000000000001},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3779999999999999,"15min":1.327},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":5.5590000000000002,"5min":5.5590000000000002,"15min":5.5590000000000002},"last":1.599},{"interface":"front","average":{"1min":1.198,"5min":1.29,"15min":1.2949999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.2170000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.4119999999999999,"15min":1.379},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":5.8520000000000003,"5min":5.8520000000000003,"15min":5.8520000000000003},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.4610000000000001,"15min":1.4079999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":5.7279999999999998,"5min":5.7279999999999998,"15min":5.7279999999999998},"last":1.4470000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.397,"15min":1.3819999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5669999999999999,"15min":1.51},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":0.82699999999999996},"max":{"1min":5.8159999999999998,"5min":5.8159999999999998,"15min":5.8159999999999998},"last":1.637}]},{"osd":7,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.548,"15min":1.492},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4890000000000001,"15min":1.458},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.2689999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575760,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64084,"kb_used_data":2700,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307756,"statfs":{"total":96636764160,"available":96571142144,"internally_reserved":0,"allocated":2764800,"data_stored":1636492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.137,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.095,"15min":1.0740000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.722,"15min":1.722},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93200000000000005,"15min":0.92500000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.466,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1439999999999999,"15min":1.1479999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5880000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.177}]},{"osd":3,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5189999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.256},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1339999999999999,"15min":1.1279999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.444,"5min":1.6759999999999999,"15min":1.9219999999999999},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.24,"15min":1.226},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.4219999999999999,"15min":1.391},"min":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.032},"max":{"1min":1.8140000000000001,"5min":1.9730000000000001,"15min":2.1280000000000001},"last":1.4399999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.349},"min":{"1min":1.0620000000000001,"5min":0.98899999999999999,"15min":0.97899999999999998},"max":{"1min":1.49,"5min":1.9930000000000001,"15min":2.0449999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.3660000000000001,"15min":1.357},"min":{"1min":1.0349999999999999,"5min":1.0229999999999999,"15min":1.004},"max":{"1min":1.804,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.2210000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.454,"15min":1.4399999999999999},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.873,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.536},{"interface":"front","average":{"1min":1.341,"5min":1.4139999999999999,"15min":1.4119999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.7250000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.389}]},{"osd":7,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5289999999999999,"15min":1.518},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.9339999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.516,"15min":1.514},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8029999999999999,"5min":2.1749999999999998,"15min":2.282},"last":1.298}]}]},{"osd":0,"up_from":13,"seq":55834575762,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37276,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334564,"statfs":{"total":96636764160,"available":96598593536,"internally_reserved":0,"allocated":2183168,"data_stored":1052443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.88600000000000001,"15min":0.89700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2470000000000001,"5min":1.5309999999999999,"15min":1.766},"last":1.028},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.345,"5min":1.5980000000000001,"15min":1.605},"last":0.95999999999999996}]},{"osd":2,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.034,"15min":1.056},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2410000000000001,"5min":1.4470000000000001,"15min":1.8540000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.089,"15min":1.1220000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.464,"5min":1.595,"15min":1.7030000000000001},"last":1.073}]},{"osd":3,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":1.036,"15min":1.054},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1890000000000001,"5min":1.5509999999999999,"15min":1.8100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.2270000000000001,"15min":1.244},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7230000000000001,"5min":1.798,"15min":1.9470000000000001},"last":1.45}]},{"osd":4,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.327,"15min":1.345},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.591,"5min":1.6910000000000001,"15min":2.0979999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.258,"15min":1.282},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.651,"5min":1.6579999999999999,"15min":2.0739999999999998},"last":1.282}]},{"osd":5,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.407,"15min":1.3999999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6930000000000001,"5min":2.2370000000000001,"15min":2.6110000000000002},"last":1.2},{"interface":"front","average":{"1min":1.306,"5min":1.4079999999999999,"15min":1.4099999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.71}]},{"osd":6,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.377,"15min":1.367},"min":{"1min":0.77700000000000002,"5min":0.745,"15min":0.745},"max":{"1min":1.643,"5min":2.0939999999999999,"15min":2.1680000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.5009999999999999,"15min":1.4970000000000001},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":1.748,"5min":2.798,"15min":2.798},"last":1.6399999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.548,"15min":1.514},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.458,"15min":1.472},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.7689999999999999,"5min":2.0830000000000002,"15min":3.0590000000000002},"last":1.3560000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T02:52:26.382 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T02:52:26.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.631+0000 7f2a9a9ab640 1 Processor -- start 2023-12-01T02:52:26.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.631+0000 7f2a9a9ab640 1 -- start start 2023-12-01T02:52:26.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.632+0000 7f2a9a9ab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2a940cad70 0x7f2a9406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.632+0000 7f2a9a9ab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a9406f2f0 0x7f2a9406bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.632+0000 7f2a9a9ab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a940add10 con 0x7f2a9415e2f0 2023-12-01T02:52:26.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.632+0000 7f2a9a9ab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a940c5760 con 0x7f2a940cad70 2023-12-01T02:52:26.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.632+0000 7f2a9a9ab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9415e6d0 con 0x7f2a9406f2f0 2023-12-01T02:52:26.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.633+0000 7f2a93fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2a9415e2f0 0x7f2a940ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54092/0 (socket says 172.21.15.115:54092) 2023-12-01T02:52:26.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.633+0000 7f2a93fff640 1 -- 172.21.15.115:0/266071121 learned_addr learned my addr 172.21.15.115:0/266071121 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:26.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.633+0000 7f2a8bfff640 1 --2- 172.21.15.115:0/266071121 >> v2:172.21.15.115:3300/0 conn(0x7f2a940cad70 0x7f2a9406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.633+0000 7f2a98f48640 1 --2- 172.21.15.115:0/266071121 >> v2:172.21.15.142:3300/0 conn(0x7f2a9406f2f0 0x7f2a9406bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.633+0000 7f2a8bfff640 1 -- 172.21.15.115:0/266071121 >> v2:172.21.15.142:3300/0 conn(0x7f2a9406f2f0 msgr2=0x7f2a9406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:26.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.633+0000 7f2a8bfff640 1 --2- 172.21.15.115:0/266071121 >> v2:172.21.15.142:3300/0 conn(0x7f2a9406f2f0 0x7f2a9406bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.633+0000 7f2a8bfff640 1 -- 172.21.15.115:0/266071121 >> v1:172.21.15.115:6789/0 conn(0x7f2a9415e2f0 legacy=0x7f2a940ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.633+0000 7f2a8bfff640 1 -- 172.21.15.115:0/266071121 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a940cf7b0 con 0x7f2a940cad70 2023-12-01T02:52:26.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.635+0000 7f2a8bfff640 1 --2- 172.21.15.115:0/266071121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 0x7f2a9406b5c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0097a0 tx=0x7f2a7c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c66e70a1dfaf063 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:26.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.635+0000 7f2a937fe640 1 -- 172.21.15.115:0/266071121 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c02f050 con 0x7f2a940cad70 2023-12-01T02:52:26.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.635+0000 7f2a937fe640 1 -- 172.21.15.115:0/266071121 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c031700 con 0x7f2a940cad70 2023-12-01T02:52:26.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.635+0000 7f2a937fe640 1 -- 172.21.15.115:0/266071121 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c031a90 con 0x7f2a940cad70 2023-12-01T02:52:26.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.635+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/266071121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 msgr2=0x7f2a9406b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.635+0000 7f2a9a9ab640 1 --2- 172.21.15.115:0/266071121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 0x7f2a9406b5c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0097a0 tx=0x7f2a7c02b7a0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.635+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/266071121 shutdown_connections 2023-12-01T02:52:26.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.635+0000 7f2a9a9ab640 1 --2- 172.21.15.115:0/266071121 >> v2:172.21.15.142:3300/0 conn(0x7f2a9406f2f0 0x7f2a9406bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.635+0000 7f2a9a9ab640 1 --2- 172.21.15.115:0/266071121 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 0x7f2a9406b5c0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.635+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/266071121 >> 172.21.15.115:0/266071121 conn(0x7f2a940ce880 msgr2=0x7f2a940cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:26.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.636+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/266071121 shutdown_connections 2023-12-01T02:52:26.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.636+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/266071121 wait complete. 2023-12-01T02:52:26.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.637+0000 7f2a9a9ab640 1 Processor -- start 2023-12-01T02:52:26.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.637+0000 7f2a9a9ab640 1 -- start start 2023-12-01T02:52:26.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.638+0000 7f2a9a9ab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9406f2f0 0x7f2a940c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.638+0000 7f2a9a9ab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 0x7f2a94154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.638+0000 7f2a9a9ab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9415e2f0 0x7f2a940ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.638+0000 7f2a9a9ab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940c4660 con 0x7f2a940cad70 2023-12-01T02:52:26.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.638+0000 7f2a9a9ab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940c4940 con 0x7f2a9415e2f0 2023-12-01T02:52:26.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.638+0000 7f2a9a9ab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940c4c20 con 0x7f2a9406f2f0 2023-12-01T02:52:26.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.638+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 0x7f2a94154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.639+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 0x7f2a94154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55746/0 (socket says 172.21.15.115:55746) 2023-12-01T02:52:26.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.639+0000 7f2a8bfff640 1 -- 172.21.15.115:0/3385221795 learned_addr learned my addr 172.21.15.115:0/3385221795 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.639+0000 7f2a8bfff640 1 -- 172.21.15.115:0/3385221795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9406f2f0 msgr2=0x7f2a940c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.639+0000 7f2a8bfff640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9406f2f0 0x7f2a940c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.639+0000 7f2a8bfff640 1 -- 172.21.15.115:0/3385221795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9415e2f0 msgr2=0x7f2a940ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.639+0000 7f2a8bfff640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9415e2f0 0x7f2a940ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.639+0000 7f2a8bfff640 1 -- 172.21.15.115:0/3385221795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c009400 con 0x7f2a940cad70 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.639+0000 7f2a8bfff640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 0x7f2a94154e60 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c036040 tx=0x7f2a7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.639+0000 7f2a917fa640 1 -- 172.21.15.115:0/3385221795 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c033a30 con 0x7f2a940cad70 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.639+0000 7f2a917fa640 1 -- 172.21.15.115:0/3385221795 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c033bb0 con 0x7f2a940cad70 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.640+0000 7f2a917fa640 1 -- 172.21.15.115:0/3385221795 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c03c400 con 0x7f2a940cad70 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.640+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/3385221795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a940c4f00 con 0x7f2a940cad70 2023-12-01T02:52:26.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.640+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/3385221795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a941553a0 con 0x7f2a940cad70 2023-12-01T02:52:26.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.641+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/3385221795 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9406fda0 con 0x7f2a940cad70 2023-12-01T02:52:26.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.642+0000 7f2a917fa640 1 -- 172.21.15.115:0/3385221795 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f2a7c03a040 con 0x7f2a940cad70 2023-12-01T02:52:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.642+0000 7f2a917fa640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a7406a550 0x7f2a7406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.642+0000 7f2a917fa640 1 -- 172.21.15.115:0/3385221795 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f2a7c0aa0b0 con 0x7f2a940cad70 2023-12-01T02:52:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.643+0000 7f2a93fff640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a7406a550 0x7f2a7406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:26.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.643+0000 7f2a93fff640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a7406a550 0x7f2a7406ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a84005880 tx=0x7f2a84007a00 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:26.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.647+0000 7f2a917fa640 1 -- 172.21.15.115:0/3385221795 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2a7c0754d0 con 0x7f2a940cad70 2023-12-01T02:52:26.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.833+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/3385221795 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a940b43d0 con 0x7f2a7406a550 2023-12-01T02:52:26.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.845+0000 7f2a917fa640 1 -- 172.21.15.115:0/3385221795 <== mgr.9297 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280455 (secure 0 0 0) 0x7f2a840058b0 con 0x7f2a7406a550 2023-12-01T02:52:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:26.851 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T02:52:26.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.851+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/3385221795 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a7406a550 msgr2=0x7f2a7406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.851+0000 7f2a9a9ab640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a7406a550 0x7f2a7406ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a84005880 tx=0x7f2a84007a00 comp rx=0 tx=0).stop 2023-12-01T02:52:26.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.851+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/3385221795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 msgr2=0x7f2a94154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:26.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.851+0000 7f2a9a9ab640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 0x7f2a94154e60 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c036040 tx=0x7f2a7c0046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.851+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/3385221795 shutdown_connections 2023-12-01T02:52:26.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.851+0000 7f2a9a9ab640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f2a7406a550 0x7f2a7406ca10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.851+0000 7f2a9a9ab640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9415e2f0 0x7f2a940ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.851+0000 7f2a9a9ab640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a940cad70 0x7f2a94154e60 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.851+0000 7f2a9a9ab640 1 --2- 172.21.15.115:0/3385221795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9406f2f0 0x7f2a940c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:26.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.851+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/3385221795 >> 172.21.15.115:0/3385221795 conn(0x7f2a940ce880 msgr2=0x7f2a940b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:26.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.852+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/3385221795 shutdown_connections 2023-12-01T02:52:26.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:26.852+0000 7f2a9a9ab640 1 -- 172.21.15.115:0/3385221795 wait complete. 2023-12-01T02:52:26.872 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T02:52:25.491335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":393608,"kb_used_data":18120,"kb_used_omap":0,"kb_used_meta":375424,"kb_avail":754581112,"statfs":{"total":773094113280,"available":772691058688,"internally_reserved":0,"allocated":18554880,"data_stored":9550186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":384434176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.018808"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.491054+0000","last_change":"2023-12-01T02:52:21.491054+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.491054+0000","last_undegraded":"2023-12-01T02:52:21.491054+0000","last_fullsized":"2023-12-01T02:52:21.491054+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:31.174876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.490218+0000","last_change":"2023-12-01T02:52:21.490218+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.490218+0000","last_undegraded":"2023-12-01T02:52:21.490218+0000","last_fullsized":"2023-12-01T02:52:21.490218+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:10.320275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.500819+0000","last_change":"2023-12-01T02:52:21.500819+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.500819+0000","last_undegraded":"2023-12-01T02:52:21.500819+0000","last_fullsized":"2023-12-01T02:52:21.500819+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:10.320275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.485590+0000","last_change":"2023-12-01T02:52:21.485590+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.485590+0000","last_undegraded":"2023-12-01T02:52:21.485590+0000","last_fullsized":"2023-12-01T02:52:21.485590+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:58.071226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.505503+0000","last_change":"2023-12-01T02:52:21.505503+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.505503+0000","last_undegraded":"2023-12-01T02:52:21.505503+0000","last_fullsized":"2023-12-01T02:52:21.505503+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:54.211416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487545+0000","last_change":"2023-12-01T02:52:21.487545+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487545+0000","last_undegraded":"2023-12-01T02:52:21.487545+0000","last_fullsized":"2023-12-01T02:52:21.487545+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:16:32.491266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.488522+0000","last_change":"2023-12-01T02:52:21.488522+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488522+0000","last_undegraded":"2023-12-01T02:52:21.488522+0000","last_fullsized":"2023-12-01T02:52:21.488522+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:47:40.922670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.504618+0000","last_change":"2023-12-01T02:52:21.504618+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.504618+0000","last_undegraded":"2023-12-01T02:52:21.504618+0000","last_fullsized":"2023-12-01T02:52:21.504618+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:37.307890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.496757+0000","last_change":"2023-12-01T02:52:21.496757+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.496757+0000","last_undegraded":"2023-12-01T02:52:21.496757+0000","last_fullsized":"2023-12-01T02:52:21.496757+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:59.055845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.493670+0000","last_change":"2023-12-01T02:52:21.493670+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.493670+0000","last_undegraded":"2023-12-01T02:52:21.493670+0000","last_fullsized":"2023-12-01T02:52:21.493670+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:43.800296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.492259+0000","last_change":"2023-12-01T02:52:21.492259+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.492259+0000","last_undegraded":"2023-12-01T02:52:21.492259+0000","last_fullsized":"2023-12-01T02:52:21.492259+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:16.098248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.490155+0000","last_change":"2023-12-01T02:52:21.490155+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.490155+0000","last_undegraded":"2023-12-01T02:52:21.490155+0000","last_fullsized":"2023-12-01T02:52:21.490155+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:04:48.367542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487045+0000","last_change":"2023-12-01T02:52:21.487045+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487045+0000","last_undegraded":"2023-12-01T02:52:21.487045+0000","last_fullsized":"2023-12-01T02:52:21.487045+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.486225+0000","last_change":"2023-12-01T02:52:21.486225+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.486225+0000","last_undegraded":"2023-12-01T02:52:21.486225+0000","last_fullsized":"2023-12-01T02:52:21.486225+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:24.096134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.492521+0000","last_change":"2023-12-01T02:52:21.492521+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.492521+0000","last_undegraded":"2023-12-01T02:52:21.492521+0000","last_fullsized":"2023-12-01T02:52:21.492521+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:38.297728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.488879+0000","last_change":"2023-12-01T02:52:21.488879+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488879+0000","last_undegraded":"2023-12-01T02:52:21.488879+0000","last_fullsized":"2023-12-01T02:52:21.488879+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:52:02.202035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.503971+0000","last_change":"2023-12-01T02:52:21.503971+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.503971+0000","last_undegraded":"2023-12-01T02:52:21.503971+0000","last_fullsized":"2023-12-01T02:52:21.503971+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:19:07.126182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481702+0000","last_change":"2023-12-01T02:52:20.481702+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481702+0000","last_undegraded":"2023-12-01T02:52:20.481702+0000","last_fullsized":"2023-12-01T02:52:20.481702+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:51.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486539+0000","last_change":"2023-12-01T02:52:20.486539+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486539+0000","last_undegraded":"2023-12-01T02:52:20.486539+0000","last_fullsized":"2023-12-01T02:52:20.486539+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481432+0000","last_change":"2023-12-01T02:52:20.481432+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481432+0000","last_undegraded":"2023-12-01T02:52:20.481432+0000","last_fullsized":"2023-12-01T02:52:20.481432+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:18.775753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.499930+0000","last_change":"2023-12-01T02:52:21.499930+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.499930+0000","last_undegraded":"2023-12-01T02:52:21.499930+0000","last_fullsized":"2023-12-01T02:52:21.499930+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:39.498194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481197+0000","last_change":"2023-12-01T02:52:20.481197+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481197+0000","last_undegraded":"2023-12-01T02:52:20.481197+0000","last_fullsized":"2023-12-01T02:52:20.481197+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:49.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.489389+0000","last_change":"2023-12-01T02:52:21.489389+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.489389+0000","last_undegraded":"2023-12-01T02:52:21.489389+0000","last_fullsized":"2023-12-01T02:52:21.489389+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:57.637709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.485885+0000","last_change":"2023-12-01T02:52:21.485885+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.485885+0000","last_undegraded":"2023-12-01T02:52:21.485885+0000","last_fullsized":"2023-12-01T02:52:21.485885+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:40.990842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.502490+0000","last_change":"2023-12-01T02:52:21.502490+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502490+0000","last_undegraded":"2023-12-01T02:52:21.502490+0000","last_fullsized":"2023-12-01T02:52:21.502490+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:42.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.498065+0000","last_change":"2023-12-01T02:52:21.498065+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.498065+0000","last_undegraded":"2023-12-01T02:52:21.498065+0000","last_fullsized":"2023-12-01T02:52:21.498065+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:34.788423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487675+0000","last_change":"2023-12-01T02:52:21.487675+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487675+0000","last_undegraded":"2023-12-01T02:52:21.487675+0000","last_fullsized":"2023-12-01T02:52:21.487675+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:04.052509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.485774+0000","last_change":"2023-12-01T02:52:20.485774+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.485774+0000","last_undegraded":"2023-12-01T02:52:20.485774+0000","last_fullsized":"2023-12-01T02:52:20.485774+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:22.856154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.501546+0000","last_change":"2023-12-01T02:52:21.501546+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.501546+0000","last_undegraded":"2023-12-01T02:52:21.501546+0000","last_fullsized":"2023-12-01T02:52:21.501546+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.482235+0000","last_change":"2023-12-01T02:52:20.482235+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.482235+0000","last_undegraded":"2023-12-01T02:52:20.482235+0000","last_fullsized":"2023-12-01T02:52:20.482235+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:48:39.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.505781+0000","last_change":"2023-12-01T02:52:21.505781+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.505781+0000","last_undegraded":"2023-12-01T02:52:21.505781+0000","last_fullsized":"2023-12-01T02:52:21.505781+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:32.342375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486360+0000","last_change":"2023-12-01T02:52:20.486360+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486360+0000","last_undegraded":"2023-12-01T02:52:20.486360+0000","last_fullsized":"2023-12-01T02:52:20.486360+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:08.328341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.504430+0000","last_change":"2023-12-01T02:52:21.504430+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.504430+0000","last_undegraded":"2023-12-01T02:52:21.504430+0000","last_fullsized":"2023-12-01T02:52:21.504430+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:46.753790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.487600+0000","last_change":"2023-12-01T02:52:20.487600+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.487600+0000","last_undegraded":"2023-12-01T02:52:20.487600+0000","last_fullsized":"2023-12-01T02:52:20.487600+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:57.056061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.504912+0000","last_change":"2023-12-01T02:52:21.504912+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.504912+0000","last_undegraded":"2023-12-01T02:52:21.504912+0000","last_fullsized":"2023-12-01T02:52:21.504912+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:41.511561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.480601+0000","last_change":"2023-12-01T02:52:20.480601+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.480601+0000","last_undegraded":"2023-12-01T02:52:20.480601+0000","last_fullsized":"2023-12-01T02:52:20.480601+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:16.375606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487348+0000","last_change":"2023-12-01T02:52:21.487348+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487348+0000","last_undegraded":"2023-12-01T02:52:21.487348+0000","last_fullsized":"2023-12-01T02:52:21.487348+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:34.006926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.484366+0000","last_change":"2023-12-01T02:52:21.484366+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.484366+0000","last_undegraded":"2023-12-01T02:52:21.484366+0000","last_fullsized":"2023-12-01T02:52:21.484366+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:37.282563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.506646+0000","last_change":"2023-12-01T02:52:21.506646+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.506646+0000","last_undegraded":"2023-12-01T02:52:21.506646+0000","last_fullsized":"2023-12-01T02:52:21.506646+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:38.746327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.493462+0000","last_change":"2023-12-01T02:52:21.493462+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.493462+0000","last_undegraded":"2023-12-01T02:52:21.493462+0000","last_fullsized":"2023-12-01T02:52:21.493462+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:11:02.383031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.499995+0000","last_change":"2023-12-01T02:52:21.499995+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.499995+0000","last_undegraded":"2023-12-01T02:52:21.499995+0000","last_fullsized":"2023-12-01T02:52:21.499995+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:38.297728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.485211+0000","last_change":"2023-12-01T02:52:21.485211+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.485211+0000","last_undegraded":"2023-12-01T02:52:21.485211+0000","last_fullsized":"2023-12-01T02:52:21.485211+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:24:04.462724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.503442+0000","last_change":"2023-12-01T02:52:21.503442+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.503442+0000","last_undegraded":"2023-12-01T02:52:21.503442+0000","last_fullsized":"2023-12-01T02:52:21.503442+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:17:54.151789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.502652+0000","last_change":"2023-12-01T02:52:21.502652+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502652+0000","last_undegraded":"2023-12-01T02:52:21.502652+0000","last_fullsized":"2023-12-01T02:52:21.502652+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:56.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.492178+0000","last_change":"2023-12-01T02:52:21.492178+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.492178+0000","last_undegraded":"2023-12-01T02:52:21.492178+0000","last_fullsized":"2023-12-01T02:52:21.492178+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:39.498194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.488796+0000","last_change":"2023-12-01T02:52:21.488796+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488796+0000","last_undegraded":"2023-12-01T02:52:21.488796+0000","last_fullsized":"2023-12-01T02:52:21.488796+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.502542+0000","last_change":"2023-12-01T02:52:21.502542+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502542+0000","last_undegraded":"2023-12-01T02:52:21.502542+0000","last_fullsized":"2023-12-01T02:52:21.502542+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:49.683912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.488038+0000","last_change":"2023-12-01T02:52:20.488038+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.488038+0000","last_undegraded":"2023-12-01T02:52:20.488038+0000","last_fullsized":"2023-12-01T02:52:20.488038+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:23.080969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.488165+0000","last_change":"2023-12-01T02:52:21.488165+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488165+0000","last_undegraded":"2023-12-01T02:52:21.488165+0000","last_fullsized":"2023-12-01T02:52:21.488165+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:44:46.015030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.495589+0000","last_change":"2023-12-01T02:52:21.495589+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.495589+0000","last_undegraded":"2023-12-01T02:52:21.495589+0000","last_fullsized":"2023-12-01T02:52:21.495589+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:18.077875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.485633+0000","last_change":"2023-12-01T02:52:20.485633+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.485633+0000","last_undegraded":"2023-12-01T02:52:20.485633+0000","last_fullsized":"2023-12-01T02:52:20.485633+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:36.292725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487161+0000","last_change":"2023-12-01T02:52:21.487161+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487161+0000","last_undegraded":"2023-12-01T02:52:21.487161+0000","last_fullsized":"2023-12-01T02:52:21.487161+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:24.957113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.499966+0000","last_change":"2023-12-01T02:52:21.499966+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.499966+0000","last_undegraded":"2023-12-01T02:52:21.499966+0000","last_fullsized":"2023-12-01T02:52:21.499966+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:30.297120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.494693+0000","last_change":"2023-12-01T02:52:21.494693+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.494693+0000","last_undegraded":"2023-12-01T02:52:21.494693+0000","last_fullsized":"2023-12-01T02:52:21.494693+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.500840+0000","last_change":"2023-12-01T02:52:21.500840+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.500840+0000","last_undegraded":"2023-12-01T02:52:21.500840+0000","last_fullsized":"2023-12-01T02:52:21.500840+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:52.712427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.489829+0000","last_change":"2023-12-01T02:52:21.489829+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.489829+0000","last_undegraded":"2023-12-01T02:52:21.489829+0000","last_fullsized":"2023-12-01T02:52:21.489829+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:31.312285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.508458+0000","last_change":"2023-12-01T02:52:21.508458+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.508458+0000","last_undegraded":"2023-12-01T02:52:21.508458+0000","last_fullsized":"2023-12-01T02:52:21.508458+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:04:47.352377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.505348+0000","last_change":"2023-12-01T02:52:21.505348+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.505348+0000","last_undegraded":"2023-12-01T02:52:21.505348+0000","last_fullsized":"2023-12-01T02:52:21.505348+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:09.343506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.501801+0000","last_change":"2023-12-01T02:52:21.501801+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.501801+0000","last_undegraded":"2023-12-01T02:52:21.501801+0000","last_fullsized":"2023-12-01T02:52:21.501801+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:19.139617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.502268+0000","last_change":"2023-12-01T02:52:21.502268+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502268+0000","last_undegraded":"2023-12-01T02:52:21.502268+0000","last_fullsized":"2023-12-01T02:52:21.502268+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.491172+0000","last_change":"2023-12-01T02:52:21.491172+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.491172+0000","last_undegraded":"2023-12-01T02:52:21.491172+0000","last_fullsized":"2023-12-01T02:52:21.491172+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:36.014503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487296+0000","last_change":"2023-12-01T02:52:21.487296+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487296+0000","last_undegraded":"2023-12-01T02:52:21.487296+0000","last_fullsized":"2023-12-01T02:52:21.487296+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:51.425590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486216+0000","last_change":"2023-12-01T02:52:20.486216+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486216+0000","last_undegraded":"2023-12-01T02:52:20.486216+0000","last_fullsized":"2023-12-01T02:52:20.486216+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:50.410425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.488770+0000","last_change":"2023-12-01T02:52:21.488770+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488770+0000","last_undegraded":"2023-12-01T02:52:21.488770+0000","last_fullsized":"2023-12-01T02:52:21.488770+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:30:41.584477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486389+0000","last_change":"2023-12-01T02:52:20.486389+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486389+0000","last_undegraded":"2023-12-01T02:52:20.486389+0000","last_fullsized":"2023-12-01T02:52:20.486389+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:15:51.909423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":6,"reported_epoch":175,"state":"active","last_fresh":"2023-12-01T02:52:21.499662+0000","last_change":"2023-12-01T02:52:21.499540+0000","last_active":"2023-12-01T02:52:21.499662+0000","last_peered":"2023-12-01T02:52:21.499662+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"2023-12-01T02:52:21.499540+0000","last_became_peered":"2023-12-01T02:52:21.499540+0000","last_unstale":"2023-12-01T02:52:21.499662+0000","last_undegraded":"2023-12-01T02:52:21.499662+0000","last_fullsized":"2023-12-01T02:52:21.499662+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:14.533994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.500456+0000","last_change":"2023-12-01T02:52:21.500456+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.500456+0000","last_undegraded":"2023-12-01T02:52:21.500456+0000","last_fullsized":"2023-12-01T02:52:21.500456+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:43.800296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481543+0000","last_change":"2023-12-01T02:52:20.481543+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481543+0000","last_undegraded":"2023-12-01T02:52:20.481543+0000","last_fullsized":"2023-12-01T02:52:20.481543+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:39.608571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.503299+0000","last_change":"2023-12-01T02:52:21.503299+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.503299+0000","last_undegraded":"2023-12-01T02:52:21.503299+0000","last_fullsized":"2023-12-01T02:52:21.503299+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:52.569081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.484343+0000","last_change":"2023-12-01T02:52:21.484343+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.484343+0000","last_undegraded":"2023-12-01T02:52:21.484343+0000","last_fullsized":"2023-12-01T02:52:21.484343+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:09.305110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.485770+0000","last_change":"2023-12-01T02:52:20.485770+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.485770+0000","last_undegraded":"2023-12-01T02:52:20.485770+0000","last_fullsized":"2023-12-01T02:52:20.485770+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:31.327211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.484334+0000","last_change":"2023-12-01T02:52:21.484334+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.484334+0000","last_undegraded":"2023-12-01T02:52:21.484334+0000","last_fullsized":"2023-12-01T02:52:21.484334+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:42.785131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"165'271","reported_seq":879,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-01T02:52:20.487518+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T02:52:20.487518+0000","last_peered":"2023-12-01T02:52:20.487518+0000","last_clean":"2023-12-01T02:52:20.487518+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T02:52:20.487518+0000","last_undegraded":"2023-12-01T02:52:20.487518+0000","last_fullsized":"2023-12-01T02:52:20.487518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":80,"acting":80,"num_store_stats":5},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575762,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3799999999999999,"5min":1.7829999999999999,"15min":1.893},"last":1.171},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1539999999999999,"15min":1.1100000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.514,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.91900000000000004}]},{"osd":1,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.266,"15min":1.2709999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.02},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.26,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85399999999999998},"max":{"1min":1.585,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":1.095}]},{"osd":2,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.2010000000000001},"min":{"1min":0.82499999999999996,"5min":0.80600000000000005,"15min":0.78500000000000003},"max":{"1min":1.554,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.252,"5min":1.28,"15min":1.25},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8440000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.343,"15min":1.3220000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.742,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.048,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.038}]},{"osd":4,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.851,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.506,"5min":1.4930000000000001,"15min":1.4750000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.157,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.276}]},{"osd":5,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3149999999999999,"15min":1.284},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.0699999999999998,"5min":2.976,"15min":2.976},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.595,"15min":1.569},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.2160000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.363}]},{"osd":6,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4590000000000001,"15min":1.4179999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.1499999999999999,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.21},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4870000000000001,"15min":1.4470000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.966,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.244}]}]},{"osd":6,"up_from":13,"seq":55834575763,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.161,"15min":1.165},"min":{"1min":0.84299999999999997,"5min":0.78100000000000003,"15min":0.75},"max":{"1min":1.6080000000000001,"5min":1.73,"15min":2.2759999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.1439999999999999,"15min":1.1619999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.56799999999999995},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.966},"last":1.4099999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1459999999999999,"15min":1.1859999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.78800000000000003},"max":{"1min":1.6459999999999999,"5min":1.772,"15min":2.0409999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.2,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6679999999999999,"5min":1.7370000000000001,"15min":2.093},"last":1.3460000000000001}]},{"osd":2,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1679999999999999,"15min":1.1879999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.80600000000000005},"max":{"1min":1.5369999999999999,"5min":1.736,"15min":1.946},"last":1.593},{"interface":"front","average":{"1min":1.274,"5min":1.272,"15min":1.284},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.637,"5min":1.7190000000000001,"15min":2.1230000000000002},"last":1.4810000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":1.0620000000000001,"5min":1.006,"15min":0.90500000000000003},"max":{"1min":1.6140000000000001,"5min":1.819,"15min":1.819},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.431,"15min":1.4319999999999999},"min":{"1min":1.042,"5min":0.98699999999999999,"15min":0.92800000000000005},"max":{"1min":1.738,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.556}]},{"osd":4,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.262},"min":{"1min":0.93000000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.698},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3500000000000001,"15min":1.3360000000000001},"min":{"1min":0.96699999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.98,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.4330000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.4419999999999999,"15min":1.4279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.5,"15min":1.4970000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.7799999999999998},"last":1.25}]},{"osd":7,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.5149999999999999,"15min":1.524},"min":{"1min":1.0980000000000001,"5min":1.034,"15min":1.034},"max":{"1min":1.9419999999999999,"5min":2.1549999999999998,"15min":2.5659999999999998},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6040000000000001,"15min":1.5940000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.0499999999999998,"5min":2.226,"15min":2.3809999999999998},"last":1.6559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575762,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59976,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311864,"statfs":{"total":96636764160,"available":96575348736,"internally_reserved":0,"allocated":2752512,"data_stored":1630631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1499999999999999,"15min":1.1419999999999999},"min":{"1min":0.68700000000000006,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4810000000000001,"5min":1.73,"15min":1.744},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1519999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.137,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.56200000000000006},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.704},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.226,"15min":1.2330000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.016},"last":1.548}]},{"osd":2,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2270000000000001,"15min":1.24},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5900000000000001,"5min":1.7430000000000001,"15min":1.746},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2609999999999999,"15min":1.2809999999999999},"min":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.627},"max":{"1min":1.655,"5min":1.671,"15min":1.764},"last":1.5940000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.321,"15min":1.319},"min":{"1min":0.98399999999999999,"5min":0.872,"15min":0.84899999999999998},"max":{"1min":1.546,"5min":1.7729999999999999,"15min":1.871},"last":0.996},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3460000000000001,"15min":1.3620000000000001},"min":{"1min":1.0269999999999999,"5min":0.91500000000000004,"15min":0.879},"max":{"1min":1.7250000000000001,"5min":1.7470000000000001,"15min":1.962},"last":1.417}]},{"osd":4,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.254,"15min":1.2569999999999999},"min":{"1min":0.91300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.696,"5min":1.847,"15min":1.978},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.391,"5min":1.361,"15min":1.3620000000000001},"min":{"1min":1.0740000000000001,"5min":0.91600000000000004,"15min":0.88600000000000001},"max":{"1min":1.911,"5min":1.911,"15min":2.0350000000000001},"last":1.7270000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3520000000000001,"15min":1.375},"min":{"1min":0.89300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":2.4470000000000001},"last":1.494},{"interface":"front","average":{"1min":1.411,"5min":1.391,"15min":1.419},"min":{"1min":1.077,"5min":0.876,"15min":0.876},"max":{"1min":1.798,"5min":1.946,"15min":2.7210000000000001},"last":1.325}]},{"osd":7,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.5449999999999999,"15min":1.5369999999999999},"min":{"1min":1.1200000000000001,"5min":1.099,"15min":0.92100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":3.0379999999999998},"last":1.464},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5780000000000001,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":3.5270000000000001},"last":1.7729999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575762,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47120,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324720,"statfs":{"total":96636764160,"available":96588513280,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.113,"15min":1.137},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.6180000000000001},"last":1.153},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1599999999999999,"15min":1.1699999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.3899999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.234,"15min":1.2270000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.48,"5min":1.706,"15min":1.7709999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.139,"15min":1.1830000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.381,"5min":1.6100000000000001,"15min":1.8640000000000001},"last":1.296}]},{"osd":2,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1910000000000001,"15min":1.218},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":2.1499999999999999},"last":1.472},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3240000000000001,"15min":1.3169999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.85999999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.1220000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2470000000000001,"15min":1.2609999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.454,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.016},{"interface":"front","average":{"1min":1.298,"5min":1.3260000000000001,"15min":1.329},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":2.2440000000000002},"last":1.343}]},{"osd":5,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.367,"15min":1.3140000000000001},"min":{"1min":0.92300000000000004,"5min":0.86499999999999999,"15min":0.84499999999999997},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3069999999999999,"15min":1.284},"min":{"1min":0.96699999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.712,"5min":1.875,"15min":1.875},"last":1.4430000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3580000000000001,"15min":1.325},"min":{"1min":0.82599999999999996,"5min":0.81299999999999994,"15min":0.78500000000000003},"max":{"1min":1.996,"5min":1.996,"15min":2.4489999999999998},"last":1.609},{"interface":"front","average":{"1min":1.554,"5min":1.474,"15min":1.448},"min":{"1min":1.097,"5min":1.014,"15min":0.97899999999999998},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":1.708}]},{"osd":7,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.444,"15min":1.4379999999999999},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.92900000000000005},"max":{"1min":1.905,"5min":2.044,"15min":2.044},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.552,"15min":1.544},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.8859999999999999,"5min":2.0670000000000002,"15min":2.5760000000000001},"last":1.671}]}]},{"osd":3,"up_from":13,"seq":55834575761,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51216,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320624,"statfs":{"total":96636764160,"available":96584318976,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.90600000000000003,"15min":0.89300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.0309999999999999,"15min":1.0660000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3680000000000001,"5min":1.5449999999999999,"15min":1.5669999999999999},"last":1.081}]},{"osd":1,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.1459999999999999,"15min":1.145},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2709999999999999,"5min":1.708,"15min":1.708},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.97799999999999998,"15min":1.004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.329,"5min":1.5980000000000001,"15min":1.744},"last":1.3879999999999999}]},{"osd":2,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0800000000000001,"15min":1.097},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.171},{"interface":"front","average":{"1min":1.028,"5min":1.2050000000000001,"15min":1.2490000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4550000000000001,"5min":1.5740000000000001,"15min":1.728},"last":1.1970000000000001}]},{"osd":4,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.302,"15min":1.294},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.306,"15min":1.2949999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.67100000000000004},"max":{"1min":1.802,"5min":1.802,"15min":1.927},"last":1.127}]},{"osd":5,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.294},{"interface":"front","average":{"1min":1.377,"5min":1.3979999999999999,"15min":1.387},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.9769999999999999,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.482}]},{"osd":6,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.448,"15min":1.4379999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.67900000000000005},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.524,"15min":1.4970000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.554}]},{"osd":7,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.919,"5min":2.0790000000000002,"15min":2.1549999999999998},"last":1.673},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.5109999999999999,"15min":1.51},"min":{"1min":0.747,"5min":0.747,"15min":0.73699999999999999},"max":{"1min":1.827,"5min":2.0150000000000001,"15min":2.1259999999999999},"last":1.645}]}]},{"osd":2,"up_from":13,"seq":55834575761,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41360,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330480,"statfs":{"total":96636764160,"available":96594411520,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":0.93200000000000005,"15min":0.88900000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.49299999999999999},"max":{"1min":5.2080000000000002,"5min":5.2080000000000002,"15min":5.2080000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.127,"5min":1.0860000000000001,"15min":1.073},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.3959999999999999,"5min":5.3959999999999999,"15min":5.3959999999999999},"last":0.88400000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1160000000000001,"15min":1.085},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":5.1109999999999998,"5min":5.1109999999999998,"15min":5.1109999999999998},"last":1.071},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.1619999999999999,"15min":1.1259999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":5.4800000000000004,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.105}]},{"osd":3,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.073,"15min":1.0549999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.0540000000000003,"5min":4.0540000000000003,"15min":4.0540000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.2869999999999999,"15min":1.2230000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":5.7690000000000001,"5min":5.7690000000000001,"15min":5.7690000000000001},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3779999999999999,"15min":1.327},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":5.5590000000000002,"5min":5.5590000000000002,"15min":5.5590000000000002},"last":1.599},{"interface":"front","average":{"1min":1.198,"5min":1.29,"15min":1.2949999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.2170000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.4119999999999999,"15min":1.379},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":5.8520000000000003,"5min":5.8520000000000003,"15min":5.8520000000000003},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.4610000000000001,"15min":1.4079999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":5.7279999999999998,"5min":5.7279999999999998,"15min":5.7279999999999998},"last":1.4470000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.397,"15min":1.3819999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5669999999999999,"15min":1.51},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":0.82699999999999996},"max":{"1min":5.8159999999999998,"5min":5.8159999999999998,"15min":5.8159999999999998},"last":1.637}]},{"osd":7,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.548,"15min":1.492},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4890000000000001,"15min":1.458},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.2689999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575760,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64084,"kb_used_data":2700,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307756,"statfs":{"total":96636764160,"available":96571142144,"internally_reserved":0,"allocated":2764800,"data_stored":1636492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.137,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.095,"15min":1.0740000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.722,"15min":1.722},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93200000000000005,"15min":0.92500000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.466,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1439999999999999,"15min":1.1479999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5880000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.177}]},{"osd":3,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5189999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.256},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1339999999999999,"15min":1.1279999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.444,"5min":1.6759999999999999,"15min":1.9219999999999999},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.24,"15min":1.226},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.4219999999999999,"15min":1.391},"min":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.032},"max":{"1min":1.8140000000000001,"5min":1.9730000000000001,"15min":2.1280000000000001},"last":1.4399999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.349},"min":{"1min":1.0620000000000001,"5min":0.98899999999999999,"15min":0.97899999999999998},"max":{"1min":1.49,"5min":1.9930000000000001,"15min":2.0449999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.3660000000000001,"15min":1.357},"min":{"1min":1.0349999999999999,"5min":1.0229999999999999,"15min":1.004},"max":{"1min":1.804,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.2210000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.454,"15min":1.4399999999999999},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.873,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.536},{"interface":"front","average":{"1min":1.341,"5min":1.4139999999999999,"15min":1.4119999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.7250000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.389}]},{"osd":7,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5289999999999999,"15min":1.518},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.9339999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.516,"15min":1.514},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8029999999999999,"5min":2.1749999999999998,"15min":2.282},"last":1.298}]}]},{"osd":0,"up_from":13,"seq":55834575762,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37276,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334564,"statfs":{"total":96636764160,"available":96598593536,"internally_reserved":0,"allocated":2183168,"data_stored":1052443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.88600000000000001,"15min":0.89700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2470000000000001,"5min":1.5309999999999999,"15min":1.766},"last":1.028},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.345,"5min":1.5980000000000001,"15min":1.605},"last":0.95999999999999996}]},{"osd":2,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.034,"15min":1.056},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2410000000000001,"5min":1.4470000000000001,"15min":1.8540000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.089,"15min":1.1220000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.464,"5min":1.595,"15min":1.7030000000000001},"last":1.073}]},{"osd":3,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":1.036,"15min":1.054},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1890000000000001,"5min":1.5509999999999999,"15min":1.8100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.2270000000000001,"15min":1.244},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7230000000000001,"5min":1.798,"15min":1.9470000000000001},"last":1.45}]},{"osd":4,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.327,"15min":1.345},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.591,"5min":1.6910000000000001,"15min":2.0979999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.258,"15min":1.282},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.651,"5min":1.6579999999999999,"15min":2.0739999999999998},"last":1.282}]},{"osd":5,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.407,"15min":1.3999999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6930000000000001,"5min":2.2370000000000001,"15min":2.6110000000000002},"last":1.2},{"interface":"front","average":{"1min":1.306,"5min":1.4079999999999999,"15min":1.4099999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.71}]},{"osd":6,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.377,"15min":1.367},"min":{"1min":0.77700000000000002,"5min":0.745,"15min":0.745},"max":{"1min":1.643,"5min":2.0939999999999999,"15min":2.1680000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.5009999999999999,"15min":1.4970000000000001},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":1.748,"5min":2.798,"15min":2.798},"last":1.6399999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.548,"15min":1.514},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.458,"15min":1.472},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.7689999999999999,"5min":2.0830000000000002,"15min":3.0590000000000002},"last":1.3560000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T02:52:26.876 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T02:52:27.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.131+0000 7fe85d2e5640 1 Processor -- start 2023-12-01T02:52:27.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.131+0000 7fe85d2e5640 1 -- start start 2023-12-01T02:52:27.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.132+0000 7fe85d2e5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe858159dd0 0x7fe8580c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.132+0000 7fe85d2e5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85806f8b0 con 0x7fe85806b6f0 2023-12-01T02:52:27.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.132+0000 7fe85d2e5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8580adcb0 con 0x7fe858070d30 2023-12-01T02:52:27.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.132+0000 7fe85d2e5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580cd270 con 0x7fe858159dd0 2023-12-01T02:52:27.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe84ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe85806b6f0 0x7fe8580cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:54106/0 (socket says 172.21.15.115:54106) 2023-12-01T02:52:27.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe84ffff640 1 -- 172.21.15.115:0/3058822366 learned_addr learned my addr 172.21.15.115:0/3058822366 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2870361234 0 0) 0x7fe85806f8b0 con 0x7fe85806b6f0 2023-12-01T02:52:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe844003520 con 0x7fe85806b6f0 2023-12-01T02:52:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe856ffd640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe858159dd0 0x7fe8580c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207663819 0 0) 0x7fe844003520 con 0x7fe85806b6f0 2023-12-01T02:52:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe858159dd0 msgr2=0x7fe8580c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:27.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe858159dd0 0x7fe8580c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 >> v1:172.21.15.142:6789/0 conn(0x7fe858070d30 legacy=0x7fe8580ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580ad9d0 con 0x7fe85806b6f0 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fe840001670 con 0x7fe85806b6f0 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 >> v1:172.21.15.115:6789/0 conn(0x7fe85806b6f0 legacy=0x7fe8580cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe844003a10 0x7fe844003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.133+0000 7fe8567fc640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8440047c0 0x7fe844004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.134+0000 7fe8567fc640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe844005380 0x7fe84400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.134+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580cd270 con 0x7fe844003a10 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.134+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84400c1f0 con 0x7fe8440047c0 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.134+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84400e5c0 con 0x7fe844005380 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.134+0000 7fe856ffd640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe858159dd0 0x7fe8580c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.134+0000 7fe856ffd640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe844003a10 0x7fe844003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.134+0000 7fe84ffff640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8440047c0 0x7fe844004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.134+0000 7fe8577fe640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe844005380 0x7fe84400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.134+0000 7fe856ffd640 1 -- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe844005380 msgr2=0x7fe84400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.135+0000 7fe856ffd640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe844005380 0x7fe84400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.135+0000 7fe856ffd640 1 -- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8440047c0 msgr2=0x7fe844004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.135+0000 7fe856ffd640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8440047c0 0x7fe844004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.135+0000 7fe856ffd640 1 -- 172.21.15.115:0/3058822366 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844003520 con 0x7fe844003a10 2023-12-01T02:52:27.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.136+0000 7fe856ffd640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe844003a10 0x7fe844003e30 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fe848006fd0 tx=0x7fe8440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa8dcc40fcb0f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:27.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.136+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848033600 con 0x7fe844003a10 2023-12-01T02:52:27.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.136+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84802f050 con 0x7fe844003a10 2023-12-01T02:52:27.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.136+0000 7fe8567fc640 1 -- 172.21.15.115:0/3058822366 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848033980 con 0x7fe844003a10 2023-12-01T02:52:27.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.136+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3058822366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe844003a10 msgr2=0x7fe844003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.136+0000 7fe85d2e5640 1 --2- 172.21.15.115:0/3058822366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe844003a10 0x7fe844003e30 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fe848006fd0 tx=0x7fe8440057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.137+0000 7fe8577fe640 1 -- 172.21.15.115:0/3058822366 reap_dead start 2023-12-01T02:52:27.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.137+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3058822366 shutdown_connections 2023-12-01T02:52:27.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.137+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3058822366 >> 172.21.15.115:0/3058822366 conn(0x7fe8580ce880 msgr2=0x7fe8580cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:27.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.137+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3058822366 shutdown_connections 2023-12-01T02:52:27.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.137+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3058822366 wait complete. 2023-12-01T02:52:27.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.138+0000 7fe85d2e5640 1 Processor -- start 2023-12-01T02:52:27.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.139+0000 7fe85d2e5640 1 -- start start 2023-12-01T02:52:27.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.139+0000 7fe85d2e5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe844003a10 0x7fe8580c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.139+0000 7fe85d2e5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8580c18d0 0x7fe8581536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.139+0000 7fe85d2e5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe85815ac90 0x7fe8580cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe85d2e5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858153c30 con 0x7fe8580c18d0 2023-12-01T02:52:27.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe85d2e5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85815a800 con 0x7fe844003a10 2023-12-01T02:52:27.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe85d2e5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580c1d50 con 0x7fe85815ac90 2023-12-01T02:52:27.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe84ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8580c18d0 0x7fe8581536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe84ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8580c18d0 0x7fe8581536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55764/0 (socket says 172.21.15.115:55764) 2023-12-01T02:52:27.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe84ffff640 1 -- 172.21.15.115:0/3909127495 learned_addr learned my addr 172.21.15.115:0/3909127495 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe856ffd640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe844003a10 0x7fe8580c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe8577fe640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe85815ac90 0x7fe8580cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe84ffff640 1 -- 172.21.15.115:0/3909127495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe85815ac90 msgr2=0x7fe8580cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe84ffff640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe85815ac90 0x7fe8580cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe84ffff640 1 -- 172.21.15.115:0/3909127495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe844003a10 msgr2=0x7fe8580c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe84ffff640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe844003a10 0x7fe8580c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe84ffff640 1 -- 172.21.15.115:0/3909127495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580c40e0 con 0x7fe8580c18d0 2023-12-01T02:52:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.140+0000 7fe856ffd640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe844003a10 0x7fe8580c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:27.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.141+0000 7fe84ffff640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8580c18d0 0x7fe8581536f0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e700 tx=0x7fe84000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.141+0000 7fe84f7fe640 1 -- 172.21.15.115:0/3909127495 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84000c6c0 con 0x7fe8580c18d0 2023-12-01T02:52:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.141+0000 7fe84f7fe640 1 -- 172.21.15.115:0/3909127495 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84000c820 con 0x7fe8580c18d0 2023-12-01T02:52:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.141+0000 7fe84f7fe640 1 -- 172.21.15.115:0/3909127495 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84000cbd0 con 0x7fe8580c18d0 2023-12-01T02:52:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.141+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3909127495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8580c43c0 con 0x7fe8580c18d0 2023-12-01T02:52:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.141+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3909127495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85815a490 con 0x7fe8580c18d0 2023-12-01T02:52:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.142+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3909127495 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8580c4600 con 0x7fe8580c18d0 2023-12-01T02:52:27.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.143+0000 7fe84f7fe640 1 -- 172.21.15.115:0/3909127495 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe84000cd30 con 0x7fe8580c18d0 2023-12-01T02:52:27.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.144+0000 7fe84f7fe640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe83806a580 0x7fe83806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.144+0000 7fe84f7fe640 1 -- 172.21.15.115:0/3909127495 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fe840088690 con 0x7fe8580c18d0 2023-12-01T02:52:27.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.144+0000 7fe856ffd640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe83806a580 0x7fe83806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.145+0000 7fe856ffd640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe83806a580 0x7fe83806ca40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe848036000 tx=0x7fe848031000 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:27.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.149+0000 7fe84f7fe640 1 -- 172.21.15.115:0/3909127495 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe840053aa0 con 0x7fe8580c18d0 2023-12-01T02:52:27.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.334+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3909127495 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8580c92e0 con 0x7fe83806a580 2023-12-01T02:52:27.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.354+0000 7fe84f7fe640 1 -- 172.21.15.115:0/3909127495 <== mgr.9297 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280455 (secure 0 0 0) 0x7fe848041070 con 0x7fe83806a580 2023-12-01T02:52:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:27.361 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T02:52:27.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.361+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3909127495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe83806a580 msgr2=0x7fe83806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.361+0000 7fe85d2e5640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe83806a580 0x7fe83806ca40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe848036000 tx=0x7fe848031000 comp rx=0 tx=0).stop 2023-12-01T02:52:27.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.361+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3909127495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8580c18d0 msgr2=0x7fe8581536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.361+0000 7fe85d2e5640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8580c18d0 0x7fe8581536f0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fe84000e700 tx=0x7fe84000ebc0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.361+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3909127495 shutdown_connections 2023-12-01T02:52:27.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.361+0000 7fe85d2e5640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fe83806a580 0x7fe83806ca40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.361+0000 7fe85d2e5640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe85815ac90 0x7fe8580cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.361+0000 7fe85d2e5640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8580c18d0 0x7fe8581536f0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.361+0000 7fe85d2e5640 1 --2- 172.21.15.115:0/3909127495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe844003a10 0x7fe8580c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.361+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3909127495 >> 172.21.15.115:0/3909127495 conn(0x7fe8580ce880 msgr2=0x7fe85806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:27.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.362+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3909127495 shutdown_connections 2023-12-01T02:52:27.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.362+0000 7fe85d2e5640 1 -- 172.21.15.115:0/3909127495 wait complete. 2023-12-01T02:52:27.382 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T02:52:25.491335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":393608,"kb_used_data":18120,"kb_used_omap":0,"kb_used_meta":375424,"kb_avail":754581112,"statfs":{"total":773094113280,"available":772691058688,"internally_reserved":0,"allocated":18554880,"data_stored":9550186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":384434176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.018808"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.491054+0000","last_change":"2023-12-01T02:52:21.491054+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.491054+0000","last_undegraded":"2023-12-01T02:52:21.491054+0000","last_fullsized":"2023-12-01T02:52:21.491054+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:31.174876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.490218+0000","last_change":"2023-12-01T02:52:21.490218+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.490218+0000","last_undegraded":"2023-12-01T02:52:21.490218+0000","last_fullsized":"2023-12-01T02:52:21.490218+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:10.320275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.500819+0000","last_change":"2023-12-01T02:52:21.500819+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.500819+0000","last_undegraded":"2023-12-01T02:52:21.500819+0000","last_fullsized":"2023-12-01T02:52:21.500819+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:10.320275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.485590+0000","last_change":"2023-12-01T02:52:21.485590+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.485590+0000","last_undegraded":"2023-12-01T02:52:21.485590+0000","last_fullsized":"2023-12-01T02:52:21.485590+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:58.071226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.505503+0000","last_change":"2023-12-01T02:52:21.505503+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.505503+0000","last_undegraded":"2023-12-01T02:52:21.505503+0000","last_fullsized":"2023-12-01T02:52:21.505503+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:54.211416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487545+0000","last_change":"2023-12-01T02:52:21.487545+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487545+0000","last_undegraded":"2023-12-01T02:52:21.487545+0000","last_fullsized":"2023-12-01T02:52:21.487545+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:16:32.491266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.488522+0000","last_change":"2023-12-01T02:52:21.488522+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488522+0000","last_undegraded":"2023-12-01T02:52:21.488522+0000","last_fullsized":"2023-12-01T02:52:21.488522+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:47:40.922670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.504618+0000","last_change":"2023-12-01T02:52:21.504618+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.504618+0000","last_undegraded":"2023-12-01T02:52:21.504618+0000","last_fullsized":"2023-12-01T02:52:21.504618+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:37.307890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.496757+0000","last_change":"2023-12-01T02:52:21.496757+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.496757+0000","last_undegraded":"2023-12-01T02:52:21.496757+0000","last_fullsized":"2023-12-01T02:52:21.496757+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:59.055845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.493670+0000","last_change":"2023-12-01T02:52:21.493670+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.493670+0000","last_undegraded":"2023-12-01T02:52:21.493670+0000","last_fullsized":"2023-12-01T02:52:21.493670+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:43.800296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.492259+0000","last_change":"2023-12-01T02:52:21.492259+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.492259+0000","last_undegraded":"2023-12-01T02:52:21.492259+0000","last_fullsized":"2023-12-01T02:52:21.492259+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:16.098248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.490155+0000","last_change":"2023-12-01T02:52:21.490155+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.490155+0000","last_undegraded":"2023-12-01T02:52:21.490155+0000","last_fullsized":"2023-12-01T02:52:21.490155+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:04:48.367542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487045+0000","last_change":"2023-12-01T02:52:21.487045+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487045+0000","last_undegraded":"2023-12-01T02:52:21.487045+0000","last_fullsized":"2023-12-01T02:52:21.487045+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.486225+0000","last_change":"2023-12-01T02:52:21.486225+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.486225+0000","last_undegraded":"2023-12-01T02:52:21.486225+0000","last_fullsized":"2023-12-01T02:52:21.486225+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:24.096134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.492521+0000","last_change":"2023-12-01T02:52:21.492521+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.492521+0000","last_undegraded":"2023-12-01T02:52:21.492521+0000","last_fullsized":"2023-12-01T02:52:21.492521+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:38.297728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.488879+0000","last_change":"2023-12-01T02:52:21.488879+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488879+0000","last_undegraded":"2023-12-01T02:52:21.488879+0000","last_fullsized":"2023-12-01T02:52:21.488879+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:52:02.202035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.503971+0000","last_change":"2023-12-01T02:52:21.503971+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.503971+0000","last_undegraded":"2023-12-01T02:52:21.503971+0000","last_fullsized":"2023-12-01T02:52:21.503971+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:19:07.126182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481702+0000","last_change":"2023-12-01T02:52:20.481702+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481702+0000","last_undegraded":"2023-12-01T02:52:20.481702+0000","last_fullsized":"2023-12-01T02:52:20.481702+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:51.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486539+0000","last_change":"2023-12-01T02:52:20.486539+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486539+0000","last_undegraded":"2023-12-01T02:52:20.486539+0000","last_fullsized":"2023-12-01T02:52:20.486539+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481432+0000","last_change":"2023-12-01T02:52:20.481432+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481432+0000","last_undegraded":"2023-12-01T02:52:20.481432+0000","last_fullsized":"2023-12-01T02:52:20.481432+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:18.775753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.499930+0000","last_change":"2023-12-01T02:52:21.499930+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.499930+0000","last_undegraded":"2023-12-01T02:52:21.499930+0000","last_fullsized":"2023-12-01T02:52:21.499930+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:39.498194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481197+0000","last_change":"2023-12-01T02:52:20.481197+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481197+0000","last_undegraded":"2023-12-01T02:52:20.481197+0000","last_fullsized":"2023-12-01T02:52:20.481197+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:49.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.489389+0000","last_change":"2023-12-01T02:52:21.489389+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.489389+0000","last_undegraded":"2023-12-01T02:52:21.489389+0000","last_fullsized":"2023-12-01T02:52:21.489389+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:57.637709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.485885+0000","last_change":"2023-12-01T02:52:21.485885+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.485885+0000","last_undegraded":"2023-12-01T02:52:21.485885+0000","last_fullsized":"2023-12-01T02:52:21.485885+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:40.990842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.502490+0000","last_change":"2023-12-01T02:52:21.502490+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502490+0000","last_undegraded":"2023-12-01T02:52:21.502490+0000","last_fullsized":"2023-12-01T02:52:21.502490+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:42.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.498065+0000","last_change":"2023-12-01T02:52:21.498065+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.498065+0000","last_undegraded":"2023-12-01T02:52:21.498065+0000","last_fullsized":"2023-12-01T02:52:21.498065+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:34.788423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487675+0000","last_change":"2023-12-01T02:52:21.487675+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487675+0000","last_undegraded":"2023-12-01T02:52:21.487675+0000","last_fullsized":"2023-12-01T02:52:21.487675+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:04.052509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.485774+0000","last_change":"2023-12-01T02:52:20.485774+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.485774+0000","last_undegraded":"2023-12-01T02:52:20.485774+0000","last_fullsized":"2023-12-01T02:52:20.485774+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:22.856154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.501546+0000","last_change":"2023-12-01T02:52:21.501546+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.501546+0000","last_undegraded":"2023-12-01T02:52:21.501546+0000","last_fullsized":"2023-12-01T02:52:21.501546+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.482235+0000","last_change":"2023-12-01T02:52:20.482235+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.482235+0000","last_undegraded":"2023-12-01T02:52:20.482235+0000","last_fullsized":"2023-12-01T02:52:20.482235+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:48:39.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.505781+0000","last_change":"2023-12-01T02:52:21.505781+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.505781+0000","last_undegraded":"2023-12-01T02:52:21.505781+0000","last_fullsized":"2023-12-01T02:52:21.505781+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:32.342375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486360+0000","last_change":"2023-12-01T02:52:20.486360+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486360+0000","last_undegraded":"2023-12-01T02:52:20.486360+0000","last_fullsized":"2023-12-01T02:52:20.486360+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:08.328341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.504430+0000","last_change":"2023-12-01T02:52:21.504430+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.504430+0000","last_undegraded":"2023-12-01T02:52:21.504430+0000","last_fullsized":"2023-12-01T02:52:21.504430+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:46.753790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.487600+0000","last_change":"2023-12-01T02:52:20.487600+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.487600+0000","last_undegraded":"2023-12-01T02:52:20.487600+0000","last_fullsized":"2023-12-01T02:52:20.487600+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:57.056061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.504912+0000","last_change":"2023-12-01T02:52:21.504912+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.504912+0000","last_undegraded":"2023-12-01T02:52:21.504912+0000","last_fullsized":"2023-12-01T02:52:21.504912+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:41.511561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.480601+0000","last_change":"2023-12-01T02:52:20.480601+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.480601+0000","last_undegraded":"2023-12-01T02:52:20.480601+0000","last_fullsized":"2023-12-01T02:52:20.480601+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:16.375606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487348+0000","last_change":"2023-12-01T02:52:21.487348+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487348+0000","last_undegraded":"2023-12-01T02:52:21.487348+0000","last_fullsized":"2023-12-01T02:52:21.487348+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:34.006926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.484366+0000","last_change":"2023-12-01T02:52:21.484366+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.484366+0000","last_undegraded":"2023-12-01T02:52:21.484366+0000","last_fullsized":"2023-12-01T02:52:21.484366+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:37.282563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.506646+0000","last_change":"2023-12-01T02:52:21.506646+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.506646+0000","last_undegraded":"2023-12-01T02:52:21.506646+0000","last_fullsized":"2023-12-01T02:52:21.506646+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:38.746327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.493462+0000","last_change":"2023-12-01T02:52:21.493462+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.493462+0000","last_undegraded":"2023-12-01T02:52:21.493462+0000","last_fullsized":"2023-12-01T02:52:21.493462+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:11:02.383031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.499995+0000","last_change":"2023-12-01T02:52:21.499995+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.499995+0000","last_undegraded":"2023-12-01T02:52:21.499995+0000","last_fullsized":"2023-12-01T02:52:21.499995+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:38.297728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.485211+0000","last_change":"2023-12-01T02:52:21.485211+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.485211+0000","last_undegraded":"2023-12-01T02:52:21.485211+0000","last_fullsized":"2023-12-01T02:52:21.485211+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:24:04.462724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.503442+0000","last_change":"2023-12-01T02:52:21.503442+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.503442+0000","last_undegraded":"2023-12-01T02:52:21.503442+0000","last_fullsized":"2023-12-01T02:52:21.503442+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:17:54.151789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.502652+0000","last_change":"2023-12-01T02:52:21.502652+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502652+0000","last_undegraded":"2023-12-01T02:52:21.502652+0000","last_fullsized":"2023-12-01T02:52:21.502652+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:56.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.492178+0000","last_change":"2023-12-01T02:52:21.492178+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.492178+0000","last_undegraded":"2023-12-01T02:52:21.492178+0000","last_fullsized":"2023-12-01T02:52:21.492178+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:57:39.498194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.488796+0000","last_change":"2023-12-01T02:52:21.488796+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488796+0000","last_undegraded":"2023-12-01T02:52:21.488796+0000","last_fullsized":"2023-12-01T02:52:21.488796+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.502542+0000","last_change":"2023-12-01T02:52:21.502542+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502542+0000","last_undegraded":"2023-12-01T02:52:21.502542+0000","last_fullsized":"2023-12-01T02:52:21.502542+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:49.683912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.488038+0000","last_change":"2023-12-01T02:52:20.488038+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.488038+0000","last_undegraded":"2023-12-01T02:52:20.488038+0000","last_fullsized":"2023-12-01T02:52:20.488038+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:23.080969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.488165+0000","last_change":"2023-12-01T02:52:21.488165+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488165+0000","last_undegraded":"2023-12-01T02:52:21.488165+0000","last_fullsized":"2023-12-01T02:52:21.488165+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:44:46.015030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.495589+0000","last_change":"2023-12-01T02:52:21.495589+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.495589+0000","last_undegraded":"2023-12-01T02:52:21.495589+0000","last_fullsized":"2023-12-01T02:52:21.495589+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:18.077875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.485633+0000","last_change":"2023-12-01T02:52:20.485633+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.485633+0000","last_undegraded":"2023-12-01T02:52:20.485633+0000","last_fullsized":"2023-12-01T02:52:20.485633+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:36.292725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487161+0000","last_change":"2023-12-01T02:52:21.487161+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487161+0000","last_undegraded":"2023-12-01T02:52:21.487161+0000","last_fullsized":"2023-12-01T02:52:21.487161+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:24.957113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.499966+0000","last_change":"2023-12-01T02:52:21.499966+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.499966+0000","last_undegraded":"2023-12-01T02:52:21.499966+0000","last_fullsized":"2023-12-01T02:52:21.499966+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:30.297120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.494693+0000","last_change":"2023-12-01T02:52:21.494693+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.494693+0000","last_undegraded":"2023-12-01T02:52:21.494693+0000","last_fullsized":"2023-12-01T02:52:21.494693+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.500840+0000","last_change":"2023-12-01T02:52:21.500840+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.500840+0000","last_undegraded":"2023-12-01T02:52:21.500840+0000","last_fullsized":"2023-12-01T02:52:21.500840+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:52.712427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.489829+0000","last_change":"2023-12-01T02:52:21.489829+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.489829+0000","last_undegraded":"2023-12-01T02:52:21.489829+0000","last_fullsized":"2023-12-01T02:52:21.489829+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:31.312285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.508458+0000","last_change":"2023-12-01T02:52:21.508458+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.508458+0000","last_undegraded":"2023-12-01T02:52:21.508458+0000","last_fullsized":"2023-12-01T02:52:21.508458+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:04:47.352377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.505348+0000","last_change":"2023-12-01T02:52:21.505348+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.505348+0000","last_undegraded":"2023-12-01T02:52:21.505348+0000","last_fullsized":"2023-12-01T02:52:21.505348+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:09.343506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.501801+0000","last_change":"2023-12-01T02:52:21.501801+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.501801+0000","last_undegraded":"2023-12-01T02:52:21.501801+0000","last_fullsized":"2023-12-01T02:52:21.501801+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:19.139617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.502268+0000","last_change":"2023-12-01T02:52:21.502268+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.502268+0000","last_undegraded":"2023-12-01T02:52:21.502268+0000","last_fullsized":"2023-12-01T02:52:21.502268+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.491172+0000","last_change":"2023-12-01T02:52:21.491172+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.491172+0000","last_undegraded":"2023-12-01T02:52:21.491172+0000","last_fullsized":"2023-12-01T02:52:21.491172+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:36.014503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.487296+0000","last_change":"2023-12-01T02:52:21.487296+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.487296+0000","last_undegraded":"2023-12-01T02:52:21.487296+0000","last_fullsized":"2023-12-01T02:52:21.487296+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:51.425590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486216+0000","last_change":"2023-12-01T02:52:20.486216+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486216+0000","last_undegraded":"2023-12-01T02:52:20.486216+0000","last_fullsized":"2023-12-01T02:52:20.486216+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:50.410425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.488770+0000","last_change":"2023-12-01T02:52:21.488770+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.488770+0000","last_undegraded":"2023-12-01T02:52:21.488770+0000","last_fullsized":"2023-12-01T02:52:21.488770+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:30:41.584477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.486389+0000","last_change":"2023-12-01T02:52:20.486389+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.486389+0000","last_undegraded":"2023-12-01T02:52:20.486389+0000","last_fullsized":"2023-12-01T02:52:20.486389+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:15:51.909423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":6,"reported_epoch":175,"state":"active","last_fresh":"2023-12-01T02:52:21.499662+0000","last_change":"2023-12-01T02:52:21.499540+0000","last_active":"2023-12-01T02:52:21.499662+0000","last_peered":"2023-12-01T02:52:21.499662+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"2023-12-01T02:52:21.499540+0000","last_became_peered":"2023-12-01T02:52:21.499540+0000","last_unstale":"2023-12-01T02:52:21.499662+0000","last_undegraded":"2023-12-01T02:52:21.499662+0000","last_fullsized":"2023-12-01T02:52:21.499662+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:14.533994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.500456+0000","last_change":"2023-12-01T02:52:21.500456+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.500456+0000","last_undegraded":"2023-12-01T02:52:21.500456+0000","last_fullsized":"2023-12-01T02:52:21.500456+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:43.800296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.481543+0000","last_change":"2023-12-01T02:52:20.481543+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.481543+0000","last_undegraded":"2023-12-01T02:52:20.481543+0000","last_fullsized":"2023-12-01T02:52:20.481543+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:39.608571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":175,"state":"creating+peering","last_fresh":"2023-12-01T02:52:21.503299+0000","last_change":"2023-12-01T02:52:21.503299+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.503299+0000","last_undegraded":"2023-12-01T02:52:21.503299+0000","last_fullsized":"2023-12-01T02:52:21.503299+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:52.569081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.484343+0000","last_change":"2023-12-01T02:52:21.484343+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.484343+0000","last_undegraded":"2023-12-01T02:52:21.484343+0000","last_fullsized":"2023-12-01T02:52:21.484343+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:09.305110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":174,"state":"creating+peering","last_fresh":"2023-12-01T02:52:20.485770+0000","last_change":"2023-12-01T02:52:20.485770+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:20.485770+0000","last_undegraded":"2023-12-01T02:52:20.485770+0000","last_fullsized":"2023-12-01T02:52:20.485770+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:31.327211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T02:52:21.471172+0000","last_change":"2023-12-01T02:52:21.471172+0000","last_active":"2023-12-01T02:52:21.471172+0000","last_peered":"2023-12-01T02:52:21.471172+0000","last_clean":"2023-12-01T02:52:21.471172+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.471172+0000","last_undegraded":"2023-12-01T02:52:21.471172+0000","last_fullsized":"2023-12-01T02:52:21.471172+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":3,"reported_epoch":175,"state":"creating+activating","last_fresh":"2023-12-01T02:52:21.484334+0000","last_change":"2023-12-01T02:52:21.484334+0000","last_active":"2023-12-01T02:52:20.456007+0000","last_peered":"2023-12-01T02:52:20.456007+0000","last_clean":"2023-12-01T02:52:20.456007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T02:52:21.484334+0000","last_undegraded":"2023-12-01T02:52:21.484334+0000","last_fullsized":"2023-12-01T02:52:21.484334+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:42.785131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"165'271","reported_seq":879,"reported_epoch":174,"state":"active+clean","last_fresh":"2023-12-01T02:52:20.487518+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T02:52:20.487518+0000","last_peered":"2023-12-01T02:52:20.487518+0000","last_clean":"2023-12-01T02:52:20.487518+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T02:52:20.487518+0000","last_undegraded":"2023-12-01T02:52:20.487518+0000","last_fullsized":"2023-12-01T02:52:20.487518+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":80,"acting":80,"num_store_stats":5},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575762,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3799999999999999,"5min":1.7829999999999999,"15min":1.893},"last":1.171},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1539999999999999,"15min":1.1100000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.514,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.91900000000000004}]},{"osd":1,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.266,"15min":1.2709999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.02},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.26,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85399999999999998},"max":{"1min":1.585,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":1.095}]},{"osd":2,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.2010000000000001},"min":{"1min":0.82499999999999996,"5min":0.80600000000000005,"15min":0.78500000000000003},"max":{"1min":1.554,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.252,"5min":1.28,"15min":1.25},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8440000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.343,"15min":1.3220000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.742,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.048,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.038}]},{"osd":4,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.851,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.506,"5min":1.4930000000000001,"15min":1.4750000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.157,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.276}]},{"osd":5,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3149999999999999,"15min":1.284},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.0699999999999998,"5min":2.976,"15min":2.976},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.595,"15min":1.569},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.2160000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.363}]},{"osd":6,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4590000000000001,"15min":1.4179999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.1499999999999999,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.21},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4870000000000001,"15min":1.4470000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.966,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.244}]}]},{"osd":6,"up_from":13,"seq":55834575763,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.161,"15min":1.165},"min":{"1min":0.84299999999999997,"5min":0.78100000000000003,"15min":0.75},"max":{"1min":1.6080000000000001,"5min":1.73,"15min":2.2759999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.1439999999999999,"15min":1.1619999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.56799999999999995},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.966},"last":1.4099999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1459999999999999,"15min":1.1859999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.78800000000000003},"max":{"1min":1.6459999999999999,"5min":1.772,"15min":2.0409999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.2,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6679999999999999,"5min":1.7370000000000001,"15min":2.093},"last":1.3460000000000001}]},{"osd":2,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1679999999999999,"15min":1.1879999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.80600000000000005},"max":{"1min":1.5369999999999999,"5min":1.736,"15min":1.946},"last":1.593},{"interface":"front","average":{"1min":1.274,"5min":1.272,"15min":1.284},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.637,"5min":1.7190000000000001,"15min":2.1230000000000002},"last":1.4810000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":1.0620000000000001,"5min":1.006,"15min":0.90500000000000003},"max":{"1min":1.6140000000000001,"5min":1.819,"15min":1.819},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.431,"15min":1.4319999999999999},"min":{"1min":1.042,"5min":0.98699999999999999,"15min":0.92800000000000005},"max":{"1min":1.738,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.556}]},{"osd":4,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.262},"min":{"1min":0.93000000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.698},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3500000000000001,"15min":1.3360000000000001},"min":{"1min":0.96699999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.98,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.4330000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.4419999999999999,"15min":1.4279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.5,"15min":1.4970000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.7799999999999998},"last":1.25}]},{"osd":7,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.5149999999999999,"15min":1.524},"min":{"1min":1.0980000000000001,"5min":1.034,"15min":1.034},"max":{"1min":1.9419999999999999,"5min":2.1549999999999998,"15min":2.5659999999999998},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6040000000000001,"15min":1.5940000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.0499999999999998,"5min":2.226,"15min":2.3809999999999998},"last":1.6559999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575762,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59976,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311864,"statfs":{"total":96636764160,"available":96575348736,"internally_reserved":0,"allocated":2752512,"data_stored":1630631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1499999999999999,"15min":1.1419999999999999},"min":{"1min":0.68700000000000006,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4810000000000001,"5min":1.73,"15min":1.744},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1519999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.137,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.56200000000000006},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.704},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.226,"15min":1.2330000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.016},"last":1.548}]},{"osd":2,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2270000000000001,"15min":1.24},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5900000000000001,"5min":1.7430000000000001,"15min":1.746},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2609999999999999,"15min":1.2809999999999999},"min":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.627},"max":{"1min":1.655,"5min":1.671,"15min":1.764},"last":1.5940000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.321,"15min":1.319},"min":{"1min":0.98399999999999999,"5min":0.872,"15min":0.84899999999999998},"max":{"1min":1.546,"5min":1.7729999999999999,"15min":1.871},"last":0.996},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3460000000000001,"15min":1.3620000000000001},"min":{"1min":1.0269999999999999,"5min":0.91500000000000004,"15min":0.879},"max":{"1min":1.7250000000000001,"5min":1.7470000000000001,"15min":1.962},"last":1.417}]},{"osd":4,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.254,"15min":1.2569999999999999},"min":{"1min":0.91300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.696,"5min":1.847,"15min":1.978},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.391,"5min":1.361,"15min":1.3620000000000001},"min":{"1min":1.0740000000000001,"5min":0.91600000000000004,"15min":0.88600000000000001},"max":{"1min":1.911,"5min":1.911,"15min":2.0350000000000001},"last":1.7270000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3520000000000001,"15min":1.375},"min":{"1min":0.89300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":2.4470000000000001},"last":1.494},{"interface":"front","average":{"1min":1.411,"5min":1.391,"15min":1.419},"min":{"1min":1.077,"5min":0.876,"15min":0.876},"max":{"1min":1.798,"5min":1.946,"15min":2.7210000000000001},"last":1.325}]},{"osd":7,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.5449999999999999,"15min":1.5369999999999999},"min":{"1min":1.1200000000000001,"5min":1.099,"15min":0.92100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":3.0379999999999998},"last":1.464},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5780000000000001,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":3.5270000000000001},"last":1.7729999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575762,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47120,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324720,"statfs":{"total":96636764160,"available":96588513280,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.113,"15min":1.137},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.6180000000000001},"last":1.153},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1599999999999999,"15min":1.1699999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.3899999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.234,"15min":1.2270000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.48,"5min":1.706,"15min":1.7709999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.139,"15min":1.1830000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.381,"5min":1.6100000000000001,"15min":1.8640000000000001},"last":1.296}]},{"osd":2,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1910000000000001,"15min":1.218},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":2.1499999999999999},"last":1.472},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3240000000000001,"15min":1.3169999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.85999999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.1220000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2470000000000001,"15min":1.2609999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.454,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.016},{"interface":"front","average":{"1min":1.298,"5min":1.3260000000000001,"15min":1.329},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":2.2440000000000002},"last":1.343}]},{"osd":5,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.367,"15min":1.3140000000000001},"min":{"1min":0.92300000000000004,"5min":0.86499999999999999,"15min":0.84499999999999997},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3069999999999999,"15min":1.284},"min":{"1min":0.96699999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.712,"5min":1.875,"15min":1.875},"last":1.4430000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3580000000000001,"15min":1.325},"min":{"1min":0.82599999999999996,"5min":0.81299999999999994,"15min":0.78500000000000003},"max":{"1min":1.996,"5min":1.996,"15min":2.4489999999999998},"last":1.609},{"interface":"front","average":{"1min":1.554,"5min":1.474,"15min":1.448},"min":{"1min":1.097,"5min":1.014,"15min":0.97899999999999998},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":1.708}]},{"osd":7,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.444,"15min":1.4379999999999999},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.92900000000000005},"max":{"1min":1.905,"5min":2.044,"15min":2.044},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.552,"15min":1.544},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.8859999999999999,"5min":2.0670000000000002,"15min":2.5760000000000001},"last":1.671}]}]},{"osd":3,"up_from":13,"seq":55834575761,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51216,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320624,"statfs":{"total":96636764160,"available":96584318976,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.90600000000000003,"15min":0.89300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.0309999999999999,"15min":1.0660000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3680000000000001,"5min":1.5449999999999999,"15min":1.5669999999999999},"last":1.081}]},{"osd":1,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.1459999999999999,"15min":1.145},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2709999999999999,"5min":1.708,"15min":1.708},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.97799999999999998,"15min":1.004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.329,"5min":1.5980000000000001,"15min":1.744},"last":1.3879999999999999}]},{"osd":2,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0800000000000001,"15min":1.097},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.171},{"interface":"front","average":{"1min":1.028,"5min":1.2050000000000001,"15min":1.2490000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4550000000000001,"5min":1.5740000000000001,"15min":1.728},"last":1.1970000000000001}]},{"osd":4,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.302,"15min":1.294},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.306,"15min":1.2949999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.67100000000000004},"max":{"1min":1.802,"5min":1.802,"15min":1.927},"last":1.127}]},{"osd":5,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.294},{"interface":"front","average":{"1min":1.377,"5min":1.3979999999999999,"15min":1.387},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.9769999999999999,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.482}]},{"osd":6,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.448,"15min":1.4379999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.67900000000000005},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.524,"15min":1.4970000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.554}]},{"osd":7,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.919,"5min":2.0790000000000002,"15min":2.1549999999999998},"last":1.673},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.5109999999999999,"15min":1.51},"min":{"1min":0.747,"5min":0.747,"15min":0.73699999999999999},"max":{"1min":1.827,"5min":2.0150000000000001,"15min":2.1259999999999999},"last":1.645}]}]},{"osd":2,"up_from":13,"seq":55834575761,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41360,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330480,"statfs":{"total":96636764160,"available":96594411520,"internally_reserved":0,"allocated":2170880,"data_stored":1046124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":0.93200000000000005,"15min":0.88900000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.49299999999999999},"max":{"1min":5.2080000000000002,"5min":5.2080000000000002,"15min":5.2080000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.127,"5min":1.0860000000000001,"15min":1.073},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.3959999999999999,"5min":5.3959999999999999,"15min":5.3959999999999999},"last":0.88400000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1160000000000001,"15min":1.085},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":5.1109999999999998,"5min":5.1109999999999998,"15min":5.1109999999999998},"last":1.071},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.1619999999999999,"15min":1.1259999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":5.4800000000000004,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.105}]},{"osd":3,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.073,"15min":1.0549999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.0540000000000003,"5min":4.0540000000000003,"15min":4.0540000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.2869999999999999,"15min":1.2230000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":5.7690000000000001,"5min":5.7690000000000001,"15min":5.7690000000000001},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3779999999999999,"15min":1.327},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":5.5590000000000002,"5min":5.5590000000000002,"15min":5.5590000000000002},"last":1.599},{"interface":"front","average":{"1min":1.198,"5min":1.29,"15min":1.2949999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.2170000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.4119999999999999,"15min":1.379},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":5.8520000000000003,"5min":5.8520000000000003,"15min":5.8520000000000003},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.4610000000000001,"15min":1.4079999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":5.7279999999999998,"5min":5.7279999999999998,"15min":5.7279999999999998},"last":1.4470000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.397,"15min":1.3819999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5669999999999999,"15min":1.51},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":0.82699999999999996},"max":{"1min":5.8159999999999998,"5min":5.8159999999999998,"15min":5.8159999999999998},"last":1.637}]},{"osd":7,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.548,"15min":1.492},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4890000000000001,"15min":1.458},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.2689999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575760,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64084,"kb_used_data":2700,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307756,"statfs":{"total":96636764160,"available":96571142144,"internally_reserved":0,"allocated":2764800,"data_stored":1636492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.137,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.095,"15min":1.0740000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.722,"15min":1.722},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93200000000000005,"15min":0.92500000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.466,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1439999999999999,"15min":1.1479999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5880000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.177}]},{"osd":3,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5189999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.256},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1339999999999999,"15min":1.1279999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.444,"5min":1.6759999999999999,"15min":1.9219999999999999},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.24,"15min":1.226},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.4219999999999999,"15min":1.391},"min":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.032},"max":{"1min":1.8140000000000001,"5min":1.9730000000000001,"15min":2.1280000000000001},"last":1.4399999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.349},"min":{"1min":1.0620000000000001,"5min":0.98899999999999999,"15min":0.97899999999999998},"max":{"1min":1.49,"5min":1.9930000000000001,"15min":2.0449999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.3660000000000001,"15min":1.357},"min":{"1min":1.0349999999999999,"5min":1.0229999999999999,"15min":1.004},"max":{"1min":1.804,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.2210000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.454,"15min":1.4399999999999999},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.873,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.536},{"interface":"front","average":{"1min":1.341,"5min":1.4139999999999999,"15min":1.4119999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.7250000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.389}]},{"osd":7,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5289999999999999,"15min":1.518},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.9339999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.516,"15min":1.514},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8029999999999999,"5min":2.1749999999999998,"15min":2.282},"last":1.298}]}]},{"osd":0,"up_from":13,"seq":55834575762,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37276,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334564,"statfs":{"total":96636764160,"available":96598593536,"internally_reserved":0,"allocated":2183168,"data_stored":1052443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.88600000000000001,"15min":0.89700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2470000000000001,"5min":1.5309999999999999,"15min":1.766},"last":1.028},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.345,"5min":1.5980000000000001,"15min":1.605},"last":0.95999999999999996}]},{"osd":2,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.034,"15min":1.056},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2410000000000001,"5min":1.4470000000000001,"15min":1.8540000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.089,"15min":1.1220000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.464,"5min":1.595,"15min":1.7030000000000001},"last":1.073}]},{"osd":3,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":1.036,"15min":1.054},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1890000000000001,"5min":1.5509999999999999,"15min":1.8100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.2270000000000001,"15min":1.244},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7230000000000001,"5min":1.798,"15min":1.9470000000000001},"last":1.45}]},{"osd":4,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.327,"15min":1.345},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.591,"5min":1.6910000000000001,"15min":2.0979999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.258,"15min":1.282},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.651,"5min":1.6579999999999999,"15min":2.0739999999999998},"last":1.282}]},{"osd":5,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.407,"15min":1.3999999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6930000000000001,"5min":2.2370000000000001,"15min":2.6110000000000002},"last":1.2},{"interface":"front","average":{"1min":1.306,"5min":1.4079999999999999,"15min":1.4099999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.71}]},{"osd":6,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.377,"15min":1.367},"min":{"1min":0.77700000000000002,"5min":0.745,"15min":0.745},"max":{"1min":1.643,"5min":2.0939999999999999,"15min":2.1680000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.5009999999999999,"15min":1.4970000000000001},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":1.748,"5min":2.798,"15min":2.798},"last":1.6399999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.548,"15min":1.514},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.458,"15min":1.472},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.7689999999999999,"5min":2.0830000000000002,"15min":3.0590000000000002},"last":1.3560000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T02:52:27.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3f is not active+clean 2023-12-01T02:52:27.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.491054+0000', 'last_change': '2023-12-01T02:52:21.491054+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.491054+0000', 'last_undegraded': '2023-12-01T02:52:21.491054+0000', 'last_fullsized': '2023-12-01T02:52:21.491054+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:10:31.174876+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3e is not active+clean 2023-12-01T02:52:27.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.490218+0000', 'last_change': '2023-12-01T02:52:21.490218+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.490218+0000', 'last_undegraded': '2023-12-01T02:52:21.490218+0000', 'last_fullsized': '2023-12-01T02:52:21.490218+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:46:10.320275+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3d is not active+clean 2023-12-01T02:52:27.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3c is not active+clean 2023-12-01T02:52:27.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.500819+0000', 'last_change': '2023-12-01T02:52:21.500819+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.500819+0000', 'last_undegraded': '2023-12-01T02:52:21.500819+0000', 'last_fullsized': '2023-12-01T02:52:21.500819+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:46:10.320275+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3b is not active+clean 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3a is not active+clean 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.39 is not active+clean 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.38 is not active+clean 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.485590+0000', 'last_change': '2023-12-01T02:52:21.485590+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.485590+0000', 'last_undegraded': '2023-12-01T02:52:21.485590+0000', 'last_fullsized': '2023-12-01T02:52:21.485590+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:22:58.071226+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.37 is not active+clean 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.36 is not active+clean 2023-12-01T02:52:27.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.35 is not active+clean 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.34 is not active+clean 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.33 is not active+clean 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.32 is not active+clean 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.31 is not active+clean 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.30 is not active+clean 2023-12-01T02:52:27.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2f is not active+clean 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2e is not active+clean 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.505503+0000', 'last_change': '2023-12-01T02:52:21.505503+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.505503+0000', 'last_undegraded': '2023-12-01T02:52:21.505503+0000', 'last_fullsized': '2023-12-01T02:52:21.505503+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:17:54.211416+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2d is not active+clean 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.487545+0000', 'last_change': '2023-12-01T02:52:21.487545+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.487545+0000', 'last_undegraded': '2023-12-01T02:52:21.487545+0000', 'last_fullsized': '2023-12-01T02:52:21.487545+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:16:32.491266+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2c is not active+clean 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.488522+0000', 'last_change': '2023-12-01T02:52:21.488522+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.488522+0000', 'last_undegraded': '2023-12-01T02:52:21.488522+0000', 'last_fullsized': '2023-12-01T02:52:21.488522+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:47:40.922670+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2b is not active+clean 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2a is not active+clean 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.29 is not active+clean 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.504618+0000', 'last_change': '2023-12-01T02:52:21.504618+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.504618+0000', 'last_undegraded': '2023-12-01T02:52:21.504618+0000', 'last_fullsized': '2023-12-01T02:52:21.504618+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:34:37.307890+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.28 is not active+clean 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.496757+0000', 'last_change': '2023-12-01T02:52:21.496757+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.496757+0000', 'last_undegraded': '2023-12-01T02:52:21.496757+0000', 'last_fullsized': '2023-12-01T02:52:21.496757+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:53:59.055845+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.27 is not active+clean 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.493670+0000', 'last_change': '2023-12-01T02:52:21.493670+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.493670+0000', 'last_undegraded': '2023-12-01T02:52:21.493670+0000', 'last_fullsized': '2023-12-01T02:52:21.493670+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:16:43.800296+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.26 is not active+clean 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.492259+0000', 'last_change': '2023-12-01T02:52:21.492259+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.492259+0000', 'last_undegraded': '2023-12-01T02:52:21.492259+0000', 'last_fullsized': '2023-12-01T02:52:21.492259+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:28:16.098248+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.25 is not active+clean 2023-12-01T02:52:27.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.490155+0000', 'last_change': '2023-12-01T02:52:21.490155+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.490155+0000', 'last_undegraded': '2023-12-01T02:52:21.490155+0000', 'last_fullsized': '2023-12-01T02:52:21.490155+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:04:48.367542+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.24 is not active+clean 2023-12-01T02:52:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.487045+0000', 'last_change': '2023-12-01T02:52:21.487045+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.487045+0000', 'last_undegraded': '2023-12-01T02:52:21.487045+0000', 'last_fullsized': '2023-12-01T02:52:21.487045+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.23 is not active+clean 2023-12-01T02:52:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.486225+0000', 'last_change': '2023-12-01T02:52:21.486225+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.486225+0000', 'last_undegraded': '2023-12-01T02:52:21.486225+0000', 'last_fullsized': '2023-12-01T02:52:21.486225+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:18:24.096134+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.22 is not active+clean 2023-12-01T02:52:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.21 is not active+clean 2023-12-01T02:52:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.492521+0000', 'last_change': '2023-12-01T02:52:21.492521+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.492521+0000', 'last_undegraded': '2023-12-01T02:52:21.492521+0000', 'last_fullsized': '2023-12-01T02:52:21.492521+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:21:38.297728+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.20 is not active+clean 2023-12-01T02:52:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.488879+0000', 'last_change': '2023-12-01T02:52:21.488879+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.488879+0000', 'last_undegraded': '2023-12-01T02:52:21.488879+0000', 'last_fullsized': '2023-12-01T02:52:21.488879+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:52:02.202035+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1e is not active+clean 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.503971+0000', 'last_change': '2023-12-01T02:52:21.503971+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.503971+0000', 'last_undegraded': '2023-12-01T02:52:21.503971+0000', 'last_fullsized': '2023-12-01T02:52:21.503971+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:19:07.126182+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1f is not active+clean 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.481702+0000', 'last_change': '2023-12-01T02:52:20.481702+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.481702+0000', 'last_undegraded': '2023-12-01T02:52:20.481702+0000', 'last_fullsized': '2023-12-01T02:52:20.481702+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:18:51.736257+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1f is not active+clean 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1e is not active+clean 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.486539+0000', 'last_change': '2023-12-01T02:52:20.486539+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.486539+0000', 'last_undegraded': '2023-12-01T02:52:20.486539+0000', 'last_fullsized': '2023-12-01T02:52:20.486539+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1c is not active+clean 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1d is not active+clean 2023-12-01T02:52:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.481432+0000', 'last_change': '2023-12-01T02:52:20.481432+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.481432+0000', 'last_undegraded': '2023-12-01T02:52:20.481432+0000', 'last_fullsized': '2023-12-01T02:52:20.481432+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:42:18.775753+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T02:52:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1d is not active+clean 2023-12-01T02:52:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.499930+0000', 'last_change': '2023-12-01T02:52:21.499930+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.499930+0000', 'last_undegraded': '2023-12-01T02:52:21.499930+0000', 'last_fullsized': '2023-12-01T02:52:21.499930+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:57:39.498194+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1c is not active+clean 2023-12-01T02:52:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.481197+0000', 'last_change': '2023-12-01T02:52:20.481197+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.481197+0000', 'last_undegraded': '2023-12-01T02:52:20.481197+0000', 'last_fullsized': '2023-12-01T02:52:20.481197+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:19:49.949971+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T02:52:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1a is not active+clean 2023-12-01T02:52:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.489389+0000', 'last_change': '2023-12-01T02:52:21.489389+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.489389+0000', 'last_undegraded': '2023-12-01T02:52:21.489389+0000', 'last_fullsized': '2023-12-01T02:52:21.489389+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:47:57.637709+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1b is not active+clean 2023-12-01T02:52:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.485885+0000', 'last_change': '2023-12-01T02:52:21.485885+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.485885+0000', 'last_undegraded': '2023-12-01T02:52:21.485885+0000', 'last_fullsized': '2023-12-01T02:52:21.485885+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:01:40.990842+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': ['4', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1b is not active+clean 2023-12-01T02:52:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.502490+0000', 'last_change': '2023-12-01T02:52:21.502490+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.502490+0000', 'last_undegraded': '2023-12-01T02:52:21.502490+0000', 'last_fullsized': '2023-12-01T02:52:21.502490+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:01:42.006007+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1a is not active+clean 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.498065+0000', 'last_change': '2023-12-01T02:52:21.498065+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.498065+0000', 'last_undegraded': '2023-12-01T02:52:21.498065+0000', 'last_fullsized': '2023-12-01T02:52:21.498065+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:01:34.788423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': ['1', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.18 is not active+clean 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.487675+0000', 'last_change': '2023-12-01T02:52:21.487675+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.487675+0000', 'last_undegraded': '2023-12-01T02:52:21.487675+0000', 'last_fullsized': '2023-12-01T02:52:21.487675+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:22:04.052509+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.19 is not active+clean 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.485774+0000', 'last_change': '2023-12-01T02:52:20.485774+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.485774+0000', 'last_undegraded': '2023-12-01T02:52:20.485774+0000', 'last_fullsized': '2023-12-01T02:52:20.485774+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:42:22.856154+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.19 is not active+clean 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.501546+0000', 'last_change': '2023-12-01T02:52:21.501546+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.501546+0000', 'last_undegraded': '2023-12-01T02:52:21.501546+0000', 'last_fullsized': '2023-12-01T02:52:21.501546+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:42:23.871319+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.18 is not active+clean 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.482235+0000', 'last_change': '2023-12-01T02:52:20.482235+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.482235+0000', 'last_undegraded': '2023-12-01T02:52:20.482235+0000', 'last_fullsized': '2023-12-01T02:52:20.482235+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:48:39.078788+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.16 is not active+clean 2023-12-01T02:52:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.505781+0000', 'last_change': '2023-12-01T02:52:21.505781+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.505781+0000', 'last_undegraded': '2023-12-01T02:52:21.505781+0000', 'last_fullsized': '2023-12-01T02:52:21.505781+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:23:32.342375+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.17 is not active+clean 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.486360+0000', 'last_change': '2023-12-01T02:52:20.486360+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.486360+0000', 'last_undegraded': '2023-12-01T02:52:20.486360+0000', 'last_fullsized': '2023-12-01T02:52:20.486360+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:30:08.328341+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.17 is not active+clean 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.16 is not active+clean 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.16', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.504430+0000', 'last_change': '2023-12-01T02:52:21.504430+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.504430+0000', 'last_undegraded': '2023-12-01T02:52:21.504430+0000', 'last_fullsized': '2023-12-01T02:52:21.504430+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:20:46.753790+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.15 is not active+clean 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.14 is not active+clean 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.487600+0000', 'last_change': '2023-12-01T02:52:20.487600+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.487600+0000', 'last_undegraded': '2023-12-01T02:52:20.487600+0000', 'last_fullsized': '2023-12-01T02:52:20.487600+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:22:57.056061+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.12 is not active+clean 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.504912+0000', 'last_change': '2023-12-01T02:52:21.504912+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.504912+0000', 'last_undegraded': '2023-12-01T02:52:21.504912+0000', 'last_fullsized': '2023-12-01T02:52:21.504912+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:37:41.511561+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.13 is not active+clean 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.480601+0000', 'last_change': '2023-12-01T02:52:20.480601+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.480601+0000', 'last_undegraded': '2023-12-01T02:52:20.480601+0000', 'last_fullsized': '2023-12-01T02:52:20.480601+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:52:16.375606+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.13 is not active+clean 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.487348+0000', 'last_change': '2023-12-01T02:52:21.487348+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.487348+0000', 'last_undegraded': '2023-12-01T02:52:21.487348+0000', 'last_fullsized': '2023-12-01T02:52:21.487348+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:04:34.006926+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.12 is not active+clean 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.12', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.484366+0000', 'last_change': '2023-12-01T02:52:21.484366+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.484366+0000', 'last_undegraded': '2023-12-01T02:52:21.484366+0000', 'last_fullsized': '2023-12-01T02:52:21.484366+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:21:37.282563+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': ['0', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.10 is not active+clean 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.506646+0000', 'last_change': '2023-12-01T02:52:21.506646+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.506646+0000', 'last_undegraded': '2023-12-01T02:52:21.506646+0000', 'last_fullsized': '2023-12-01T02:52:21.506646+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:30:38.746327+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.11 is not active+clean 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.11', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.493462+0000', 'last_change': '2023-12-01T02:52:21.493462+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.493462+0000', 'last_undegraded': '2023-12-01T02:52:21.493462+0000', 'last_fullsized': '2023-12-01T02:52:21.493462+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:11:02.383031+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.11 is not active+clean 2023-12-01T02:52:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.499995+0000', 'last_change': '2023-12-01T02:52:21.499995+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.499995+0000', 'last_undegraded': '2023-12-01T02:52:21.499995+0000', 'last_fullsized': '2023-12-01T02:52:21.499995+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:21:38.297728+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.10 is not active+clean 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.10', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.485211+0000', 'last_change': '2023-12-01T02:52:21.485211+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.485211+0000', 'last_undegraded': '2023-12-01T02:52:21.485211+0000', 'last_fullsized': '2023-12-01T02:52:21.485211+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:24:04.462724+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.e is not active+clean 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.503442+0000', 'last_change': '2023-12-01T02:52:21.503442+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.503442+0000', 'last_undegraded': '2023-12-01T02:52:21.503442+0000', 'last_fullsized': '2023-12-01T02:52:21.503442+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:17:54.151789+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.f is not active+clean 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.502652+0000', 'last_change': '2023-12-01T02:52:21.502652+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.502652+0000', 'last_undegraded': '2023-12-01T02:52:21.502652+0000', 'last_fullsized': '2023-12-01T02:52:21.502652+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:47:56.622544+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.f is not active+clean 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.492178+0000', 'last_change': '2023-12-01T02:52:21.492178+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.492178+0000', 'last_undegraded': '2023-12-01T02:52:21.492178+0000', 'last_fullsized': '2023-12-01T02:52:21.492178+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T02:57:39.498194+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.e is not active+clean 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.488796+0000', 'last_change': '2023-12-01T02:52:21.488796+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.488796+0000', 'last_undegraded': '2023-12-01T02:52:21.488796+0000', 'last_fullsized': '2023-12-01T02:52:21.488796+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': ['4', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.c is not active+clean 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.502542+0000', 'last_change': '2023-12-01T02:52:21.502542+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.502542+0000', 'last_undegraded': '2023-12-01T02:52:21.502542+0000', 'last_fullsized': '2023-12-01T02:52:21.502542+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:10:49.683912+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.d is not active+clean 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.488038+0000', 'last_change': '2023-12-01T02:52:20.488038+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.488038+0000', 'last_undegraded': '2023-12-01T02:52:20.488038+0000', 'last_fullsized': '2023-12-01T02:52:20.488038+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:18:23.080969+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.d is not active+clean 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.c is not active+clean 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.488165+0000', 'last_change': '2023-12-01T02:52:21.488165+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.488165+0000', 'last_undegraded': '2023-12-01T02:52:21.488165+0000', 'last_fullsized': '2023-12-01T02:52:21.488165+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:44:46.015030+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': ['7', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.a is not active+clean 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.495589+0000', 'last_change': '2023-12-01T02:52:21.495589+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.495589+0000', 'last_undegraded': '2023-12-01T02:52:21.495589+0000', 'last_fullsized': '2023-12-01T02:52:21.495589+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:40:18.077875+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.b is not active+clean 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.485633+0000', 'last_change': '2023-12-01T02:52:20.485633+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.485633+0000', 'last_undegraded': '2023-12-01T02:52:20.485633+0000', 'last_fullsized': '2023-12-01T02:52:20.485633+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:34:36.292725+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.b is not active+clean 2023-12-01T02:52:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.487161+0000', 'last_change': '2023-12-01T02:52:21.487161+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.487161+0000', 'last_undegraded': '2023-12-01T02:52:21.487161+0000', 'last_fullsized': '2023-12-01T02:52:21.487161+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:22:24.957113+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.a is not active+clean 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.499966+0000', 'last_change': '2023-12-01T02:52:21.499966+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.499966+0000', 'last_undegraded': '2023-12-01T02:52:21.499966+0000', 'last_fullsized': '2023-12-01T02:52:21.499966+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:56:30.297120+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': ['2', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.8 is not active+clean 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.494693+0000', 'last_change': '2023-12-01T02:52:21.494693+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.494693+0000', 'last_undegraded': '2023-12-01T02:52:21.494693+0000', 'last_fullsized': '2023-12-01T02:52:21.494693+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.9 is not active+clean 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.9', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.500840+0000', 'last_change': '2023-12-01T02:52:21.500840+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.500840+0000', 'last_undegraded': '2023-12-01T02:52:21.500840+0000', 'last_fullsized': '2023-12-01T02:52:21.500840+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:54:52.712427+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.9 is not active+clean 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.489829+0000', 'last_change': '2023-12-01T02:52:21.489829+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.489829+0000', 'last_undegraded': '2023-12-01T02:52:21.489829+0000', 'last_fullsized': '2023-12-01T02:52:21.489829+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:56:31.312285+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.8 is not active+clean 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.8', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.508458+0000', 'last_change': '2023-12-01T02:52:21.508458+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.508458+0000', 'last_undegraded': '2023-12-01T02:52:21.508458+0000', 'last_fullsized': '2023-12-01T02:52:21.508458+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:04:47.352377+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': ['2', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.6 is not active+clean 2023-12-01T02:52:27.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.505348+0000', 'last_change': '2023-12-01T02:52:21.505348+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.505348+0000', 'last_undegraded': '2023-12-01T02:52:21.505348+0000', 'last_fullsized': '2023-12-01T02:52:21.505348+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:30:09.343506+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.7 is not active+clean 2023-12-01T02:52:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.501801+0000', 'last_change': '2023-12-01T02:52:21.501801+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.501801+0000', 'last_undegraded': '2023-12-01T02:52:21.501801+0000', 'last_fullsized': '2023-12-01T02:52:21.501801+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:46:19.139617+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.7 is not active+clean 2023-12-01T02:52:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.502268+0000', 'last_change': '2023-12-01T02:52:21.502268+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.502268+0000', 'last_undegraded': '2023-12-01T02:52:21.502268+0000', 'last_fullsized': '2023-12-01T02:52:21.502268+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.6 is not active+clean 2023-12-01T02:52:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.6', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.491172+0000', 'last_change': '2023-12-01T02:52:21.491172+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.491172+0000', 'last_undegraded': '2023-12-01T02:52:21.491172+0000', 'last_fullsized': '2023-12-01T02:52:21.491172+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:51:36.014503+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.4 is not active+clean 2023-12-01T02:52:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.487296+0000', 'last_change': '2023-12-01T02:52:21.487296+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.487296+0000', 'last_undegraded': '2023-12-01T02:52:21.487296+0000', 'last_fullsized': '2023-12-01T02:52:21.487296+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:34:51.425590+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.5 is not active+clean 2023-12-01T02:52:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.486216+0000', 'last_change': '2023-12-01T02:52:20.486216+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.486216+0000', 'last_undegraded': '2023-12-01T02:52:20.486216+0000', 'last_fullsized': '2023-12-01T02:52:20.486216+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:34:50.410425+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.5 is not active+clean 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.488770+0000', 'last_change': '2023-12-01T02:52:21.488770+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.488770+0000', 'last_undegraded': '2023-12-01T02:52:21.488770+0000', 'last_fullsized': '2023-12-01T02:52:21.488770+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:30:41.584477+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.4 is not active+clean 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.486389+0000', 'last_change': '2023-12-01T02:52:20.486389+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.486389+0000', 'last_undegraded': '2023-12-01T02:52:20.486389+0000', 'last_fullsized': '2023-12-01T02:52:20.486389+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:15:51.909423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2 is not active+clean 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3 is not active+clean 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3', 'version': "0'0", 'reported_seq': 6, 'reported_epoch': 175, 'state': 'active', 'last_fresh': '2023-12-01T02:52:21.499662+0000', 'last_change': '2023-12-01T02:52:21.499540+0000', 'last_active': '2023-12-01T02:52:21.499662+0000', 'last_peered': '2023-12-01T02:52:21.499662+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '2023-12-01T02:52:21.499540+0000', 'last_became_peered': '2023-12-01T02:52:21.499540+0000', 'last_unstale': '2023-12-01T02:52:21.499662+0000', 'last_undegraded': '2023-12-01T02:52:21.499662+0000', 'last_fullsized': '2023-12-01T02:52:21.499662+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:39:14.533994+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': ['7', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3 is not active+clean 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.500456+0000', 'last_change': '2023-12-01T02:52:21.500456+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.500456+0000', 'last_undegraded': '2023-12-01T02:52:21.500456+0000', 'last_fullsized': '2023-12-01T02:52:21.500456+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:16:43.800296+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2 is not active+clean 2023-12-01T02:52:27.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.481543+0000', 'last_change': '2023-12-01T02:52:20.481543+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.481543+0000', 'last_undegraded': '2023-12-01T02:52:20.481543+0000', 'last_fullsized': '2023-12-01T02:52:20.481543+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:56:39.608571+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T02:52:27.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.0 is not active+clean 2023-12-01T02:52:27.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 175, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:21.503299+0000', 'last_change': '2023-12-01T02:52:21.503299+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.503299+0000', 'last_undegraded': '2023-12-01T02:52:21.503299+0000', 'last_fullsized': '2023-12-01T02:52:21.503299+0000', 'mapping_epoch': 175, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 175, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:23:52.569081+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T02:52:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1 is not active+clean 2023-12-01T02:52:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.484343+0000', 'last_change': '2023-12-01T02:52:21.484343+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.484343+0000', 'last_undegraded': '2023-12-01T02:52:21.484343+0000', 'last_fullsized': '2023-12-01T02:52:21.484343+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:46:09.305110+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': ['0', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T02:52:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1 is not active+clean 2023-12-01T02:52:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.0 is not active+clean 2023-12-01T02:52:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 174, 'state': 'creating+peering', 'last_fresh': '2023-12-01T02:52:20.485770+0000', 'last_change': '2023-12-01T02:52:20.485770+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:20.485770+0000', 'last_undegraded': '2023-12-01T02:52:20.485770+0000', 'last_fullsized': '2023-12-01T02:52:20.485770+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:23:31.327211+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T02:52:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.14 is not active+clean 2023-12-01T02:52:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-01T02:52:21.471172+0000', 'last_change': '2023-12-01T02:52:21.471172+0000', 'last_active': '2023-12-01T02:52:21.471172+0000', 'last_peered': '2023-12-01T02:52:21.471172+0000', 'last_clean': '2023-12-01T02:52:21.471172+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.471172+0000', 'last_undegraded': '2023-12-01T02:52:21.471172+0000', 'last_fullsized': '2023-12-01T02:52:21.471172+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:21.471172+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-01T02:52:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.15 is not active+clean 2023-12-01T02:52:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.15', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 175, 'state': 'creating+activating', 'last_fresh': '2023-12-01T02:52:21.484334+0000', 'last_change': '2023-12-01T02:52:21.484334+0000', 'last_active': '2023-12-01T02:52:20.456007+0000', 'last_peered': '2023-12-01T02:52:20.456007+0000', 'last_clean': '2023-12-01T02:52:20.456007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T02:52:21.484334+0000', 'last_undegraded': '2023-12-01T02:52:21.484334+0000', 'last_fullsized': '2023-12-01T02:52:21.484334+0000', 'mapping_epoch': 174, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 174, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'last_clean_scrub_stamp': '2023-12-01T02:52:20.456007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:16:42.785131+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': ['0', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T02:52:27.406 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-01T02:52:27.459 INFO:tasks.ceph.mds.c.smithi115.stderr:2023-12-01T02:52:27.458+0000 7f5474477640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T02:52:27.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.638+0000 7f40ea8d5640 1 Processor -- start 2023-12-01T02:52:27.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.639+0000 7f40ea8d5640 1 -- start start 2023-12-01T02:52:27.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.640+0000 7f40ea8d5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f40e415e670 0x7f40e40cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.640+0000 7f40ea8d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e40ce750 0x7f40e40cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.640+0000 7f40ea8d5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e40b3850 con 0x7f40e415e290 2023-12-01T02:52:27.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.640+0000 7f40ea8d5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e40cc1c0 con 0x7f40e415e670 2023-12-01T02:52:27.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.640+0000 7f40ea8d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40cc4a0 con 0x7f40e40ce750 2023-12-01T02:52:27.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.640+0000 7f40e37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f40e415e670 0x7f40e40cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.640+0000 7f40e37fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f40e415e670 0x7f40e40cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34566/0 (socket says 172.21.15.115:34566) 2023-12-01T02:52:27.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.640+0000 7f40e37fe640 1 -- 172.21.15.115:0/3170228769 learned_addr learned my addr 172.21.15.115:0/3170228769 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:27.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.641+0000 7f40e37fe640 1 -- 172.21.15.115:0/3170228769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e40ce750 msgr2=0x7f40e40cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:27.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.641+0000 7f40e8e72640 1 --2- 172.21.15.115:0/3170228769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e40ce750 0x7f40e40cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.641+0000 7f40e37fe640 1 --2- 172.21.15.115:0/3170228769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e40ce750 0x7f40e40cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.641+0000 7f40e37fe640 1 -- 172.21.15.115:0/3170228769 >> v1:172.21.15.115:6789/0 conn(0x7f40e415e290 legacy=0x7f40e40b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.641+0000 7f40e37fe640 1 -- 172.21.15.115:0/3170228769 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e40cc780 con 0x7f40e415e670 2023-12-01T02:52:27.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.641+0000 7f40e8e72640 1 --2- 172.21.15.115:0/3170228769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e40ce750 0x7f40e40cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:27.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.642+0000 7f40e37fe640 1 --2- 172.21.15.115:0/3170228769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e415e670 0x7f40e40cef50 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009870 tx=0x7f40cc02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee57a434be601b4a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:27.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.643+0000 7f40e2ffd640 1 -- 172.21.15.115:0/3170228769 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc02f830 con 0x7f40e415e670 2023-12-01T02:52:27.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.643+0000 7f40e2ffd640 1 -- 172.21.15.115:0/3170228769 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc02f990 con 0x7f40e415e670 2023-12-01T02:52:27.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.643+0000 7f40e2ffd640 1 -- 172.21.15.115:0/3170228769 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc02fd20 con 0x7f40e415e670 2023-12-01T02:52:27.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.643+0000 7f40ea8d5640 1 -- 172.21.15.115:0/3170228769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e415e670 msgr2=0x7f40e40cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.643+0000 7f40ea8d5640 1 --2- 172.21.15.115:0/3170228769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e415e670 0x7f40e40cef50 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009870 tx=0x7f40cc02ba10 comp rx=0 tx=0).stop 2023-12-01T02:52:27.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.643+0000 7f40ea8d5640 1 -- 172.21.15.115:0/3170228769 shutdown_connections 2023-12-01T02:52:27.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.643+0000 7f40ea8d5640 1 --2- 172.21.15.115:0/3170228769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e40ce750 0x7f40e40cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.643+0000 7f40ea8d5640 1 --2- 172.21.15.115:0/3170228769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e415e670 0x7f40e40cef50 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.643+0000 7f40ea8d5640 1 -- 172.21.15.115:0/3170228769 >> 172.21.15.115:0/3170228769 conn(0x7f40e40d0a00 msgr2=0x7f40e40d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:27.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.644+0000 7f40ea8d5640 1 -- 172.21.15.115:0/3170228769 shutdown_connections 2023-12-01T02:52:27.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.644+0000 7f40ea8d5640 1 -- 172.21.15.115:0/3170228769 wait complete. 2023-12-01T02:52:27.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.645+0000 7f40ea8d5640 1 Processor -- start 2023-12-01T02:52:27.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.645+0000 7f40ea8d5640 1 -- start start 2023-12-01T02:52:27.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.646+0000 7f40ea8d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e40ce750 0x7f40e40c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.646+0000 7f40ea8d5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e415e290 0x7f40e40c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.646+0000 7f40ea8d5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e415e670 0x7f40e4157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.646+0000 7f40ea8d5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40c94e0 con 0x7f40e40ce750 2023-12-01T02:52:27.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.646+0000 7f40ea8d5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40c5ce0 con 0x7f40e415e290 2023-12-01T02:52:27.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.646+0000 7f40ea8d5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40c5fc0 con 0x7f40e415e670 2023-12-01T02:52:27.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e40ce750 0x7f40e40c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e40ce750 0x7f40e40c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34578/0 (socket says 172.21.15.115:34578) 2023-12-01T02:52:27.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e3fff640 1 -- 172.21.15.115:0/2454512205 learned_addr learned my addr 172.21.15.115:0/2454512205 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:27.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e37fe640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e415e290 0x7f40e40c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e8e72640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e415e670 0x7f40e4157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e3fff640 1 -- 172.21.15.115:0/2454512205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e415e670 msgr2=0x7f40e4157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e3fff640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e415e670 0x7f40e4157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e3fff640 1 -- 172.21.15.115:0/2454512205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e415e290 msgr2=0x7f40e40c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e3fff640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e415e290 0x7f40e40c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e3fff640 1 -- 172.21.15.115:0/2454512205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc0094d0 con 0x7f40e40ce750 2023-12-01T02:52:27.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e37fe640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e415e290 0x7f40e40c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:27.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e8e72640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e415e670 0x7f40e4157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:27.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.647+0000 7f40e3fff640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e40ce750 0x7f40e40c6d80 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c7c0 tx=0x7f40d400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:27.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.648+0000 7f40e0ff9640 1 -- 172.21.15.115:0/2454512205 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4003da0 con 0x7f40e40ce750 2023-12-01T02:52:27.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.648+0000 7f40e0ff9640 1 -- 172.21.15.115:0/2454512205 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4005070 con 0x7f40e40ce750 2023-12-01T02:52:27.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.648+0000 7f40e0ff9640 1 -- 172.21.15.115:0/2454512205 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d4005400 con 0x7f40e40ce750 2023-12-01T02:52:27.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.648+0000 7f40ea8d5640 1 -- 172.21.15.115:0/2454512205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e40c62a0 con 0x7f40e40ce750 2023-12-01T02:52:27.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.648+0000 7f40ea8d5640 1 -- 172.21.15.115:0/2454512205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e40c72c0 con 0x7f40e40ce750 2023-12-01T02:52:27.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.649+0000 7f40ea8d5640 1 -- 172.21.15.115:0/2454512205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e40c66c0 con 0x7f40e40ce750 2023-12-01T02:52:27.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.650+0000 7f40e0ff9640 1 -- 172.21.15.115:0/2454512205 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f40d4005560 con 0x7f40e40ce750 2023-12-01T02:52:27.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.651+0000 7f40e0ff9640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40c406a550 0x7f40c406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:27.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.651+0000 7f40e0ff9640 1 -- 172.21.15.115:0/2454512205 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f40d40882c0 con 0x7f40e40ce750 2023-12-01T02:52:27.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.651+0000 7f40e37fe640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40c406a550 0x7f40c406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:27.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.652+0000 7f40e37fe640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40c406a550 0x7f40c406ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40cc036000 tx=0x7f40cc039040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:27.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.655+0000 7f40e0ff9640 1 -- 172.21.15.115:0/2454512205 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f40e40c66c0 con 0x7f40e40ce750 2023-12-01T02:52:27.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.913+0000 7f40ea8d5640 1 -- 172.21.15.115:0/2454512205 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f40e40c66c0 con 0x7f40e40ce750 2023-12-01T02:52:27.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.914+0000 7f40e0ff9640 1 -- 172.21.15.115:0/2454512205 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3390 (secure 0 0 0) 0x7f40d4017d50 con 0x7f40e40ce750 2023-12-01T02:52:27.915 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:27.915 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4602,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":177,"num_osds":8,"num_up_osds":8,"osd_up_since":1701394558,"num_in_osds":8,"osd_in_since":1701394549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":55},{"state_name":"unknown","count":24},{"state_name":"creating+activating","count":16},{"state_name":"active+clean","count":1},{"state_name":"active","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":403054592,"bytes_avail":772691058688,"bytes_total":773094113280,"unknown_pgs_ratio":0.24742268025875092,"inactive_pgs_ratio":0.73195874691009521},"fsmap":{"epoch":79,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9549}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2023-12-01T02:52:09.485926+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-01T02:52:27.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 -- 172.21.15.115:0/2454512205 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40c406a550 msgr2=0x7f40c406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40c406a550 0x7f40c406ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40cc036000 tx=0x7f40cc039040 comp rx=0 tx=0).stop 2023-12-01T02:52:27.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 -- 172.21.15.115:0/2454512205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e40ce750 msgr2=0x7f40e40c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:27.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e40ce750 0x7f40e40c6d80 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c7c0 tx=0x7f40d400cc80 comp rx=0 tx=0).stop 2023-12-01T02:52:27.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 -- 172.21.15.115:0/2454512205 shutdown_connections 2023-12-01T02:52:27.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f40c406a550 0x7f40c406ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e415e670 0x7f40e4157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e415e290 0x7f40e40c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 --2- 172.21.15.115:0/2454512205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40e40ce750 0x7f40e40c6d80 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:27.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 -- 172.21.15.115:0/2454512205 >> 172.21.15.115:0/2454512205 conn(0x7f40e40d0a00 msgr2=0x7f40e40d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:27.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 -- 172.21.15.115:0/2454512205 shutdown_connections 2023-12-01T02:52:27.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:27.917+0000 7f40ea8d5640 1 -- 172.21.15.115:0/2454512205 wait complete. 2023-12-01T02:52:27.938 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-01T02:52:27.938 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T02:52:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.187+0000 7f3d8331f640 1 Processor -- start 2023-12-01T02:52:28.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.187+0000 7f3d8331f640 1 -- start start 2023-12-01T02:52:28.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.189+0000 7f3d8331f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3d7c15db90 0x7f3d7c0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:28.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.189+0000 7f3d8331f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c15e020 con 0x7f3d7c0cddc0 2023-12-01T02:52:28.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.189+0000 7f3d8331f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0cd3f0 con 0x7f3d7c15d7b0 2023-12-01T02:52:28.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.189+0000 7f3d8331f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7c0cd0f0 con 0x7f3d7c15db90 2023-12-01T02:52:28.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.189+0000 7f3d82b1e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3d7c0cddc0 0x7f3d7c0d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59194/0 (socket says 172.21.15.115:59194) 2023-12-01T02:52:28.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.189+0000 7f3d82b1e640 1 -- 172.21.15.115:0/2879389106 learned_addr learned my addr 172.21.15.115:0/2879389106 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:28.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d81b1c640 1 --2- 172.21.15.115:0/2879389106 >> v2:172.21.15.142:3300/0 conn(0x7f3d7c15db90 0x7f3d7c0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:28.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537939031 0 0) 0x7f3d7c15e020 con 0x7f3d7c0cddc0 2023-12-01T02:52:28.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d68003520 con 0x7f3d7c0cddc0 2023-12-01T02:52:28.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1266415568 0 0) 0x7f3d68003520 con 0x7f3d7c0cddc0 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 >> v2:172.21.15.142:3300/0 conn(0x7f3d7c15db90 msgr2=0x7f3d7c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 --2- 172.21.15.115:0/2879389106 >> v2:172.21.15.142:3300/0 conn(0x7f3d7c15db90 0x7f3d7c0cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 >> v1:172.21.15.142:6789/0 conn(0x7f3d7c15d7b0 legacy=0x7f3d7c160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c160ec0 con 0x7f3d7c0cddc0 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3d740019f0 con 0x7f3d7c0cddc0 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 >> v1:172.21.15.115:6789/0 conn(0x7f3d7c0cddc0 legacy=0x7f3d7c0d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 --2- 172.21.15.115:0/2879389106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680039e0 0x7f3d68003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 --2- 172.21.15.115:0/2879389106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680047c0 0x7f3d68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 --2- 172.21.15.115:0/2879389106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0cd0f0 con 0x7f3d68005180 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6800c090 con 0x7f3d680047c0 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.190+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6800c250 con 0x7f3d680039e0 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.191+0000 7f3d82b1e640 1 --2- 172.21.15.115:0/2879389106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d6800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:28.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.191+0000 7f3d81b1c640 1 --2- 172.21.15.115:0/2879389106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680047c0 0x7f3d68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:28.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.191+0000 7f3d82b1e640 1 -- 172.21.15.115:0/2879389106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680039e0 msgr2=0x7f3d68003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:28.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.191+0000 7f3d82b1e640 1 --2- 172.21.15.115:0/2879389106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d680039e0 0x7f3d68003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.191+0000 7f3d82b1e640 1 -- 172.21.15.115:0/2879389106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680047c0 msgr2=0x7f3d68004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:28.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.191+0000 7f3d82b1e640 1 --2- 172.21.15.115:0/2879389106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d680047c0 0x7f3d68004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.191+0000 7f3d82b1e640 1 -- 172.21.15.115:0/2879389106 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68003520 con 0x7f3d68005180 2023-12-01T02:52:28.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.192+0000 7f3d82b1e640 1 -- 172.21.15.115:0/2879389106 reap_dead start 2023-12-01T02:52:28.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.192+0000 7f3d82b1e640 1 --2- 172.21.15.115:0/2879389106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d6800b980 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400a820 tx=0x7f3d680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c479094cbb3d89 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:28.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.192+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d740072d0 con 0x7f3d68005180 2023-12-01T02:52:28.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.192+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7402f070 con 0x7f3d68005180 2023-12-01T02:52:28.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.192+0000 7f3d8131b640 1 -- 172.21.15.115:0/2879389106 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d740076b0 con 0x7f3d68005180 2023-12-01T02:52:28.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.193+0000 7f3d8331f640 1 -- 172.21.15.115:0/2879389106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 msgr2=0x7f3d6800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:28.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.193+0000 7f3d8331f640 1 --2- 172.21.15.115:0/2879389106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d6800b980 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400a820 tx=0x7f3d680057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.193+0000 7f3d8331f640 1 -- 172.21.15.115:0/2879389106 shutdown_connections 2023-12-01T02:52:28.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.193+0000 7f3d8331f640 1 --2- 172.21.15.115:0/2879389106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d6800b980 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.193+0000 7f3d8331f640 1 -- 172.21.15.115:0/2879389106 >> 172.21.15.115:0/2879389106 conn(0x7f3d7c06a0e0 msgr2=0x7f3d7c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:28.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.193+0000 7f3d8331f640 1 -- 172.21.15.115:0/2879389106 shutdown_connections 2023-12-01T02:52:28.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.193+0000 7f3d8331f640 1 -- 172.21.15.115:0/2879389106 wait complete. 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.195+0000 7f3d8331f640 1 Processor -- start 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.195+0000 7f3d8331f640 1 -- start start 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.196+0000 7f3d8331f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d7c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.196+0000 7f3d8331f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c14dd20 0x7f3d7c0c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.196+0000 7f3d8331f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d7c0c8950 0x7f3d7c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.196+0000 7f3d8331f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0b2dc0 con 0x7f3d68005180 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.196+0000 7f3d8331f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c8dd0 con 0x7f3d7c0c8950 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.196+0000 7f3d8331f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0b3d60 con 0x7f3d7c14dd20 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.196+0000 7f3d8231d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d7c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.196+0000 7f3d8231d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d7c0b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34596/0 (socket says 172.21.15.115:34596) 2023-12-01T02:52:28.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.196+0000 7f3d8231d640 1 -- 172.21.15.115:0/4074168863 learned_addr learned my addr 172.21.15.115:0/4074168863 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:28.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d82b1e640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d7c0c8950 0x7f3d7c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d8231d640 1 -- 172.21.15.115:0/4074168863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c14dd20 msgr2=0x7f3d7c0c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d8231d640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c14dd20 0x7f3d7c0c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d8231d640 1 -- 172.21.15.115:0/4074168863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d7c0c8950 msgr2=0x7f3d7c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d8231d640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d7c0c8950 0x7f3d7c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d8231d640 1 -- 172.21.15.115:0/4074168863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0c7cb0 con 0x7f3d68005180 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d8231d640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d7c0b5000 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800c530 tx=0x7f3d7800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d72ffd640 1 -- 172.21.15.115:0/4074168863 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d780052d0 con 0x7f3d68005180 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d72ffd640 1 -- 172.21.15.115:0/4074168863 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d78005430 con 0x7f3d68005180 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d72ffd640 1 -- 172.21.15.115:0/4074168863 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78012870 con 0x7f3d68005180 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.197+0000 7f3d8331f640 1 -- 172.21.15.115:0/4074168863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c0b2fa0 con 0x7f3d68005180 2023-12-01T02:52:28.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.198+0000 7f3d8331f640 1 -- 172.21.15.115:0/4074168863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c0b3560 con 0x7f3d68005180 2023-12-01T02:52:28.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.199+0000 7f3d8331f640 1 -- 172.21.15.115:0/4074168863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c0673a0 con 0x7f3d68005180 2023-12-01T02:52:28.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.200+0000 7f3d72ffd640 1 -- 172.21.15.115:0/4074168863 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3d780129d0 con 0x7f3d68005180 2023-12-01T02:52:28.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.200+0000 7f3d72ffd640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d6006a550 0x7f3d6006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:28.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.200+0000 7f3d72ffd640 1 -- 172.21.15.115:0/4074168863 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f3d78088520 con 0x7f3d68005180 2023-12-01T02:52:28.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.201+0000 7f3d81b1c640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d6006a550 0x7f3d6006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:28.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.201+0000 7f3d81b1c640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d6006a550 0x7f3d6006ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c1602c0 tx=0x7f3d6c008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:28.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.205+0000 7f3d72ffd640 1 -- 172.21.15.115:0/4074168863 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3d78057460 con 0x7f3d68005180 2023-12-01T02:52:28.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.394+0000 7f3d8331f640 1 -- 172.21.15.115:0/4074168863 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d7c161480 con 0x7f3d6006a550 2023-12-01T02:52:28.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.415+0000 7f3d72ffd640 1 -- 172.21.15.115:0/4074168863 <== mgr.9297 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287114 (secure 0 0 0) 0x7f3d6c009e40 con 0x7f3d6006a550 2023-12-01T02:52:28.420 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T02:52:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 -- 172.21.15.115:0/4074168863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d6006a550 msgr2=0x7f3d6006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d6006a550 0x7f3d6006ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c1602c0 tx=0x7f3d6c008040 comp rx=0 tx=0).stop 2023-12-01T02:52:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 -- 172.21.15.115:0/4074168863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 msgr2=0x7f3d7c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d7c0b5000 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800c530 tx=0x7f3d7800c9f0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 -- 172.21.15.115:0/4074168863 shutdown_connections 2023-12-01T02:52:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f3d6006a550 0x7f3d6006ca10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d7c14dd20 0x7f3d7c0c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d7c0c8950 0x7f3d7c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 --2- 172.21.15.115:0/4074168863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d68005180 0x7f3d7c0b5000 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 -- 172.21.15.115:0/4074168863 >> 172.21.15.115:0/4074168863 conn(0x7f3d7c06a0e0 msgr2=0x7f3d7c0d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 -- 172.21.15.115:0/4074168863 shutdown_connections 2023-12-01T02:52:28.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:28.422+0000 7f3d8331f640 1 -- 172.21.15.115:0/4074168863 wait complete. 2023-12-01T02:52:28.441 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T02:52:27.491893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402152,"kb_used_data":18472,"kb_used_omap":0,"kb_used_meta":383616,"kb_avail":754572568,"statfs":{"total":773094113280,"available":772682309632,"internally_reserved":0,"allocated":18915328,"data_stored":9710308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":392822784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.011472"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.996509+0000","last_change":"2023-12-01T02:52:21.994463+0000","last_active":"2023-12-01T02:52:21.996509+0000","last_peered":"2023-12-01T02:52:21.996509+0000","last_clean":"2023-12-01T02:52:21.996509+0000","last_became_active":"2023-12-01T02:52:21.993031+0000","last_became_peered":"2023-12-01T02:52:21.993031+0000","last_unstale":"2023-12-01T02:52:21.996509+0000","last_undegraded":"2023-12-01T02:52:21.996509+0000","last_fullsized":"2023-12-01T02:52:21.996509+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:56.336050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.013167+0000","last_change":"2023-12-01T02:52:22.011643+0000","last_active":"2023-12-01T02:52:22.013167+0000","last_peered":"2023-12-01T02:52:22.013167+0000","last_clean":"2023-12-01T02:52:22.013167+0000","last_became_active":"2023-12-01T02:52:22.011264+0000","last_became_peered":"2023-12-01T02:52:22.011264+0000","last_unstale":"2023-12-01T02:52:22.013167+0000","last_undegraded":"2023-12-01T02:52:22.013167+0000","last_fullsized":"2023-12-01T02:52:22.013167+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:10.320275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.981345+0000","last_change":"2023-12-01T02:52:21.999113+0000","last_active":"2023-12-01T02:52:22.981345+0000","last_peered":"2023-12-01T02:52:22.981345+0000","last_clean":"2023-12-01T02:52:22.981345+0000","last_became_active":"2023-12-01T02:52:21.998878+0000","last_became_peered":"2023-12-01T02:52:21.998878+0000","last_unstale":"2023-12-01T02:52:22.981345+0000","last_undegraded":"2023-12-01T02:52:22.981345+0000","last_fullsized":"2023-12-01T02:52:22.981345+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:52:21.333565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.003834+0000","last_change":"2023-12-01T02:52:21.995052+0000","last_active":"2023-12-01T02:52:22.003834+0000","last_peered":"2023-12-01T02:52:22.003834+0000","last_clean":"2023-12-01T02:52:22.003834+0000","last_became_active":"2023-12-01T02:52:21.994794+0000","last_became_peered":"2023-12-01T02:52:21.994794+0000","last_unstale":"2023-12-01T02:52:22.003834+0000","last_undegraded":"2023-12-01T02:52:22.003834+0000","last_fullsized":"2023-12-01T02:52:22.003834+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:31.036932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.893198+0000","last_change":"2023-12-01T02:52:21.991368+0000","last_active":"2023-12-01T02:52:23.893198+0000","last_peered":"2023-12-01T02:52:23.893198+0000","last_clean":"2023-12-01T02:52:23.893198+0000","last_became_active":"2023-12-01T02:52:21.990342+0000","last_became_peered":"2023-12-01T02:52:21.990342+0000","last_unstale":"2023-12-01T02:52:23.893198+0000","last_undegraded":"2023-12-01T02:52:23.893198+0000","last_fullsized":"2023-12-01T02:52:23.893198+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:38.245577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.147907+0000","last_change":"2023-12-01T02:52:21.991765+0000","last_active":"2023-12-01T02:52:23.147907+0000","last_peered":"2023-12-01T02:52:23.147907+0000","last_clean":"2023-12-01T02:52:23.147907+0000","last_became_active":"2023-12-01T02:52:21.991137+0000","last_became_peered":"2023-12-01T02:52:21.991137+0000","last_unstale":"2023-12-01T02:52:23.147907+0000","last_undegraded":"2023-12-01T02:52:23.147907+0000","last_fullsized":"2023-12-01T02:52:23.147907+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:30:41.584477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.950977+0000","last_change":"2023-12-01T02:52:22.004656+0000","last_active":"2023-12-01T02:52:23.950977+0000","last_peered":"2023-12-01T02:52:23.950977+0000","last_clean":"2023-12-01T02:52:23.950977+0000","last_became_active":"2023-12-01T02:52:22.004287+0000","last_became_peered":"2023-12-01T02:52:22.004287+0000","last_unstale":"2023-12-01T02:52:23.950977+0000","last_undegraded":"2023-12-01T02:52:23.950977+0000","last_fullsized":"2023-12-01T02:52:23.950977+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:50.965136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.014096+0000","last_change":"2023-12-01T02:52:22.009385+0000","last_active":"2023-12-01T02:52:22.014096+0000","last_peered":"2023-12-01T02:52:22.014096+0000","last_clean":"2023-12-01T02:52:22.014096+0000","last_became_active":"2023-12-01T02:52:22.009055+0000","last_became_peered":"2023-12-01T02:52:22.009055+0000","last_unstale":"2023-12-01T02:52:22.014096+0000","last_undegraded":"2023-12-01T02:52:22.014096+0000","last_fullsized":"2023-12-01T02:52:22.014096+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:58.071226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.148370+0000","last_change":"2023-12-01T02:52:22.000217+0000","last_active":"2023-12-01T02:52:23.148370+0000","last_peered":"2023-12-01T02:52:23.148370+0000","last_clean":"2023-12-01T02:52:23.148370+0000","last_became_active":"2023-12-01T02:52:21.999925+0000","last_became_peered":"2023-12-01T02:52:21.999925+0000","last_unstale":"2023-12-01T02:52:23.148370+0000","last_undegraded":"2023-12-01T02:52:23.148370+0000","last_fullsized":"2023-12-01T02:52:23.148370+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:09.328057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.981954+0000","last_change":"2023-12-01T02:52:21.990495+0000","last_active":"2023-12-01T02:52:22.981954+0000","last_peered":"2023-12-01T02:52:22.981954+0000","last_clean":"2023-12-01T02:52:22.981954+0000","last_became_active":"2023-12-01T02:52:21.987764+0000","last_became_peered":"2023-12-01T02:52:21.987764+0000","last_unstale":"2023-12-01T02:52:22.981954+0000","last_undegraded":"2023-12-01T02:52:22.981954+0000","last_fullsized":"2023-12-01T02:52:22.981954+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:51:13.200609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.148260+0000","last_change":"2023-12-01T02:52:21.995402+0000","last_active":"2023-12-01T02:52:23.148260+0000","last_peered":"2023-12-01T02:52:23.148260+0000","last_clean":"2023-12-01T02:52:23.148260+0000","last_became_active":"2023-12-01T02:52:21.995032+0000","last_became_peered":"2023-12-01T02:52:21.995032+0000","last_unstale":"2023-12-01T02:52:23.148260+0000","last_undegraded":"2023-12-01T02:52:23.148260+0000","last_fullsized":"2023-12-01T02:52:23.148260+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:20:16.758051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.952460+0000","last_change":"2023-12-01T02:52:21.987421+0000","last_active":"2023-12-01T02:52:23.952460+0000","last_peered":"2023-12-01T02:52:23.952460+0000","last_clean":"2023-12-01T02:52:23.952460+0000","last_became_active":"2023-12-01T02:52:21.986795+0000","last_became_peered":"2023-12-01T02:52:21.986795+0000","last_unstale":"2023-12-01T02:52:23.952460+0000","last_undegraded":"2023-12-01T02:52:23.952460+0000","last_fullsized":"2023-12-01T02:52:23.952460+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:38.746327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.893102+0000","last_change":"2023-12-01T02:52:21.989676+0000","last_active":"2023-12-01T02:52:23.893102+0000","last_peered":"2023-12-01T02:52:23.893102+0000","last_clean":"2023-12-01T02:52:23.893102+0000","last_became_active":"2023-12-01T02:52:21.989406+0000","last_became_peered":"2023-12-01T02:52:21.989406+0000","last_unstale":"2023-12-01T02:52:23.893102+0000","last_undegraded":"2023-12-01T02:52:23.893102+0000","last_fullsized":"2023-12-01T02:52:23.893102+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:17.794027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.952336+0000","last_change":"2023-12-01T02:52:21.997158+0000","last_active":"2023-12-01T02:52:23.952336+0000","last_peered":"2023-12-01T02:52:23.952336+0000","last_clean":"2023-12-01T02:52:23.952336+0000","last_became_active":"2023-12-01T02:52:21.996919+0000","last_became_peered":"2023-12-01T02:52:21.996919+0000","last_unstale":"2023-12-01T02:52:23.952336+0000","last_undegraded":"2023-12-01T02:52:23.952336+0000","last_fullsized":"2023-12-01T02:52:23.952336+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:48:36.083993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.892954+0000","last_change":"2023-12-01T02:52:21.997730+0000","last_active":"2023-12-01T02:52:23.892954+0000","last_peered":"2023-12-01T02:52:23.892954+0000","last_clean":"2023-12-01T02:52:23.892954+0000","last_became_active":"2023-12-01T02:52:21.997483+0000","last_became_peered":"2023-12-01T02:52:21.997483+0000","last_unstale":"2023-12-01T02:52:23.892954+0000","last_undegraded":"2023-12-01T02:52:23.892954+0000","last_fullsized":"2023-12-01T02:52:23.892954+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:01.218959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.950974+0000","last_change":"2023-12-01T02:52:22.000724+0000","last_active":"2023-12-01T02:52:23.950974+0000","last_peered":"2023-12-01T02:52:23.950974+0000","last_clean":"2023-12-01T02:52:23.950974+0000","last_became_active":"2023-12-01T02:52:21.999040+0000","last_became_peered":"2023-12-01T02:52:21.999040+0000","last_unstale":"2023-12-01T02:52:23.950974+0000","last_undegraded":"2023-12-01T02:52:23.950974+0000","last_fullsized":"2023-12-01T02:52:23.950974+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:47:21.653533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.981137+0000","last_change":"2023-12-01T02:52:21.991169+0000","last_active":"2023-12-01T02:52:22.981137+0000","last_peered":"2023-12-01T02:52:22.981137+0000","last_clean":"2023-12-01T02:52:22.981137+0000","last_became_active":"2023-12-01T02:52:21.990909+0000","last_became_peered":"2023-12-01T02:52:21.990909+0000","last_unstale":"2023-12-01T02:52:22.981137+0000","last_undegraded":"2023-12-01T02:52:22.981137+0000","last_fullsized":"2023-12-01T02:52:22.981137+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:14:01.220260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.998742+0000","last_change":"2023-12-01T02:52:21.995718+0000","last_active":"2023-12-01T02:52:21.998742+0000","last_peered":"2023-12-01T02:52:21.998742+0000","last_clean":"2023-12-01T02:52:21.998742+0000","last_became_active":"2023-12-01T02:52:21.995473+0000","last_became_peered":"2023-12-01T02:52:21.995473+0000","last_unstale":"2023-12-01T02:52:21.998742+0000","last_undegraded":"2023-12-01T02:52:21.998742+0000","last_fullsized":"2023-12-01T02:52:21.998742+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:40.701207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.000904+0000","last_change":"2023-12-01T02:52:21.995642+0000","last_active":"2023-12-01T02:52:22.000904+0000","last_peered":"2023-12-01T02:52:22.000904+0000","last_clean":"2023-12-01T02:52:22.000904+0000","last_became_active":"2023-12-01T02:52:21.995246+0000","last_became_peered":"2023-12-01T02:52:21.995246+0000","last_unstale":"2023-12-01T02:52:22.000904+0000","last_undegraded":"2023-12-01T02:52:22.000904+0000","last_fullsized":"2023-12-01T02:52:22.000904+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:47:21.653533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.009627+0000","last_change":"2023-12-01T02:52:22.004740+0000","last_active":"2023-12-01T02:52:22.009627+0000","last_peered":"2023-12-01T02:52:22.009627+0000","last_clean":"2023-12-01T02:52:22.009627+0000","last_became_active":"2023-12-01T02:52:22.003845+0000","last_became_peered":"2023-12-01T02:52:22.003845+0000","last_unstale":"2023-12-01T02:52:22.009627+0000","last_undegraded":"2023-12-01T02:52:22.009627+0000","last_fullsized":"2023-12-01T02:52:22.009627+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:47:40.922670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.981060+0000","last_change":"2023-12-01T02:52:21.991147+0000","last_active":"2023-12-01T02:52:22.981060+0000","last_peered":"2023-12-01T02:52:22.981060+0000","last_clean":"2023-12-01T02:52:22.981060+0000","last_became_active":"2023-12-01T02:52:21.990919+0000","last_became_peered":"2023-12-01T02:52:21.990919+0000","last_unstale":"2023-12-01T02:52:22.981060+0000","last_undegraded":"2023-12-01T02:52:22.981060+0000","last_fullsized":"2023-12-01T02:52:22.981060+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:23:17.913730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.952445+0000","last_change":"2023-12-01T02:52:21.996650+0000","last_active":"2023-12-01T02:52:23.952445+0000","last_peered":"2023-12-01T02:52:23.952445+0000","last_clean":"2023-12-01T02:52:23.952445+0000","last_became_active":"2023-12-01T02:52:21.996251+0000","last_became_peered":"2023-12-01T02:52:21.996251+0000","last_unstale":"2023-12-01T02:52:23.952445+0000","last_undegraded":"2023-12-01T02:52:23.952445+0000","last_fullsized":"2023-12-01T02:52:23.952445+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:31.036932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.996982+0000","last_change":"2023-12-01T02:52:21.992890+0000","last_active":"2023-12-01T02:52:21.996982+0000","last_peered":"2023-12-01T02:52:21.996982+0000","last_clean":"2023-12-01T02:52:21.996982+0000","last_became_active":"2023-12-01T02:52:21.992631+0000","last_became_peered":"2023-12-01T02:52:21.992631+0000","last_unstale":"2023-12-01T02:52:21.996982+0000","last_undegraded":"2023-12-01T02:52:21.996982+0000","last_fullsized":"2023-12-01T02:52:21.996982+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:31.174876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.986074+0000","last_change":"2023-12-01T02:52:21.985298+0000","last_active":"2023-12-01T02:52:21.986074+0000","last_peered":"2023-12-01T02:52:21.986074+0000","last_clean":"2023-12-01T02:52:21.986074+0000","last_became_active":"2023-12-01T02:52:21.985037+0000","last_became_peered":"2023-12-01T02:52:21.985037+0000","last_unstale":"2023-12-01T02:52:21.986074+0000","last_undegraded":"2023-12-01T02:52:21.986074+0000","last_fullsized":"2023-12-01T02:52:21.986074+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:02:22.859562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.012135+0000","last_change":"2023-12-01T02:52:22.009228+0000","last_active":"2023-12-01T02:52:22.012135+0000","last_peered":"2023-12-01T02:52:22.012135+0000","last_clean":"2023-12-01T02:52:22.012135+0000","last_became_active":"2023-12-01T02:52:22.008603+0000","last_became_peered":"2023-12-01T02:52:22.008603+0000","last_unstale":"2023-12-01T02:52:22.012135+0000","last_undegraded":"2023-12-01T02:52:22.012135+0000","last_fullsized":"2023-12-01T02:52:22.012135+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:52.569081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010574+0000","last_change":"2023-12-01T02:52:22.006541+0000","last_active":"2023-12-01T02:52:22.010574+0000","last_peered":"2023-12-01T02:52:22.010574+0000","last_clean":"2023-12-01T02:52:22.010574+0000","last_became_active":"2023-12-01T02:52:22.003660+0000","last_became_peered":"2023-12-01T02:52:22.003660+0000","last_unstale":"2023-12-01T02:52:22.010574+0000","last_undegraded":"2023-12-01T02:52:22.010574+0000","last_fullsized":"2023-12-01T02:52:22.010574+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:16.098248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.013788+0000","last_change":"2023-12-01T02:52:22.012721+0000","last_active":"2023-12-01T02:52:22.013788+0000","last_peered":"2023-12-01T02:52:22.013788+0000","last_clean":"2023-12-01T02:52:22.013788+0000","last_became_active":"2023-12-01T02:52:22.009654+0000","last_became_peered":"2023-12-01T02:52:22.009654+0000","last_unstale":"2023-12-01T02:52:22.013788+0000","last_undegraded":"2023-12-01T02:52:22.013788+0000","last_fullsized":"2023-12-01T02:52:22.013788+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:18.077875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.013533+0000","last_change":"2023-12-01T02:52:22.012258+0000","last_active":"2023-12-01T02:52:22.013533+0000","last_peered":"2023-12-01T02:52:22.013533+0000","last_clean":"2023-12-01T02:52:22.013533+0000","last_became_active":"2023-12-01T02:52:22.011788+0000","last_became_peered":"2023-12-01T02:52:22.011788+0000","last_unstale":"2023-12-01T02:52:22.013533+0000","last_undegraded":"2023-12-01T02:52:22.013533+0000","last_fullsized":"2023-12-01T02:52:22.013533+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010091+0000","last_change":"2023-12-01T02:52:22.005635+0000","last_active":"2023-12-01T02:52:22.010091+0000","last_peered":"2023-12-01T02:52:22.010091+0000","last_clean":"2023-12-01T02:52:22.010091+0000","last_became_active":"2023-12-01T02:52:22.005358+0000","last_became_peered":"2023-12-01T02:52:22.005358+0000","last_unstale":"2023-12-01T02:52:22.010091+0000","last_undegraded":"2023-12-01T02:52:22.010091+0000","last_fullsized":"2023-12-01T02:52:22.010091+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:24.096134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.952500+0000","last_change":"2023-12-01T02:52:21.996489+0000","last_active":"2023-12-01T02:52:23.952500+0000","last_peered":"2023-12-01T02:52:23.952500+0000","last_clean":"2023-12-01T02:52:23.952500+0000","last_became_active":"2023-12-01T02:52:21.993148+0000","last_became_peered":"2023-12-01T02:52:21.993148+0000","last_unstale":"2023-12-01T02:52:23.952500+0000","last_undegraded":"2023-12-01T02:52:23.952500+0000","last_fullsized":"2023-12-01T02:52:23.952500+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:24:05.477889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.008522+0000","last_change":"2023-12-01T02:52:22.001823+0000","last_active":"2023-12-01T02:52:22.008522+0000","last_peered":"2023-12-01T02:52:22.008522+0000","last_clean":"2023-12-01T02:52:22.008522+0000","last_became_active":"2023-12-01T02:52:22.000229+0000","last_became_peered":"2023-12-01T02:52:22.000229+0000","last_unstale":"2023-12-01T02:52:22.008522+0000","last_undegraded":"2023-12-01T02:52:22.008522+0000","last_fullsized":"2023-12-01T02:52:22.008522+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:09.343506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.012676+0000","last_change":"2023-12-01T02:52:22.010609+0000","last_active":"2023-12-01T02:52:22.012676+0000","last_peered":"2023-12-01T02:52:22.012676+0000","last_clean":"2023-12-01T02:52:22.012676+0000","last_became_active":"2023-12-01T02:52:22.008695+0000","last_became_peered":"2023-12-01T02:52:22.008695+0000","last_unstale":"2023-12-01T02:52:22.012676+0000","last_undegraded":"2023-12-01T02:52:22.012676+0000","last_fullsized":"2023-12-01T02:52:22.012676+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:52:02.202035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.991893+0000","last_change":"2023-12-01T02:52:21.987390+0000","last_active":"2023-12-01T02:52:21.991893+0000","last_peered":"2023-12-01T02:52:21.991893+0000","last_clean":"2023-12-01T02:52:21.991893+0000","last_became_active":"2023-12-01T02:52:21.987129+0000","last_became_peered":"2023-12-01T02:52:21.987129+0000","last_unstale":"2023-12-01T02:52:21.991893+0000","last_undegraded":"2023-12-01T02:52:21.991893+0000","last_fullsized":"2023-12-01T02:52:21.991893+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:19:07.126182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"176'4","reported_seq":15,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010863+0000","last_change":"2023-12-01T02:52:21.516667+0000","last_active":"2023-12-01T02:52:22.010863+0000","last_peered":"2023-12-01T02:52:22.010863+0000","last_clean":"2023-12-01T02:52:22.010863+0000","last_became_active":"2023-12-01T02:52:21.516412+0000","last_became_peered":"2023-12-01T02:52:21.516412+0000","last_unstale":"2023-12-01T02:52:22.010863+0000","last_undegraded":"2023-12-01T02:52:22.010863+0000","last_fullsized":"2023-12-01T02:52:22.010863+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:51.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.951854+0000","last_change":"2023-12-01T02:52:21.996199+0000","last_active":"2023-12-01T02:52:23.951854+0000","last_peered":"2023-12-01T02:52:23.951854+0000","last_clean":"2023-12-01T02:52:23.951854+0000","last_became_active":"2023-12-01T02:52:21.995890+0000","last_became_peered":"2023-12-01T02:52:21.995890+0000","last_unstale":"2023-12-01T02:52:23.951854+0000","last_undegraded":"2023-12-01T02:52:23.951854+0000","last_fullsized":"2023-12-01T02:52:23.951854+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:01.554053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"176'1","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.007387+0000","last_change":"2023-12-01T02:52:21.521815+0000","last_active":"2023-12-01T02:52:22.007387+0000","last_peered":"2023-12-01T02:52:22.007387+0000","last_clean":"2023-12-01T02:52:22.007387+0000","last_became_active":"2023-12-01T02:52:21.521432+0000","last_became_peered":"2023-12-01T02:52:21.521432+0000","last_unstale":"2023-12-01T02:52:22.007387+0000","last_undegraded":"2023-12-01T02:52:22.007387+0000","last_fullsized":"2023-12-01T02:52:22.007387+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.148114+0000","last_change":"2023-12-01T02:52:21.994370+0000","last_active":"2023-12-01T02:52:23.148114+0000","last_peered":"2023-12-01T02:52:23.148114+0000","last_clean":"2023-12-01T02:52:23.148114+0000","last_became_active":"2023-12-01T02:52:21.993797+0000","last_became_peered":"2023-12-01T02:52:21.993797+0000","last_unstale":"2023-12-01T02:52:23.148114+0000","last_undegraded":"2023-12-01T02:52:23.148114+0000","last_fullsized":"2023-12-01T02:52:23.148114+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:16:32.491266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010903+0000","last_change":"2023-12-01T02:52:21.518182+0000","last_active":"2023-12-01T02:52:22.010903+0000","last_peered":"2023-12-01T02:52:22.010903+0000","last_clean":"2023-12-01T02:52:22.010903+0000","last_became_active":"2023-12-01T02:52:21.517878+0000","last_became_peered":"2023-12-01T02:52:21.517878+0000","last_unstale":"2023-12-01T02:52:22.010903+0000","last_undegraded":"2023-12-01T02:52:22.010903+0000","last_fullsized":"2023-12-01T02:52:22.010903+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:18.775753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.998570+0000","last_change":"2023-12-01T02:52:21.991684+0000","last_active":"2023-12-01T02:52:21.998570+0000","last_peered":"2023-12-01T02:52:21.998570+0000","last_clean":"2023-12-01T02:52:21.998570+0000","last_became_active":"2023-12-01T02:52:21.991423+0000","last_became_peered":"2023-12-01T02:52:21.991423+0000","last_unstale":"2023-12-01T02:52:21.998570+0000","last_undegraded":"2023-12-01T02:52:21.998570+0000","last_fullsized":"2023-12-01T02:52:21.998570+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:48:36.083993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.011036+0000","last_change":"2023-12-01T02:52:21.521668+0000","last_active":"2023-12-01T02:52:22.011036+0000","last_peered":"2023-12-01T02:52:22.011036+0000","last_clean":"2023-12-01T02:52:22.011036+0000","last_became_active":"2023-12-01T02:52:21.521385+0000","last_became_peered":"2023-12-01T02:52:21.521385+0000","last_unstale":"2023-12-01T02:52:22.011036+0000","last_undegraded":"2023-12-01T02:52:22.011036+0000","last_fullsized":"2023-12-01T02:52:22.011036+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:49.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.012254+0000","last_change":"2023-12-01T02:52:22.008213+0000","last_active":"2023-12-01T02:52:22.012254+0000","last_peered":"2023-12-01T02:52:22.012254+0000","last_clean":"2023-12-01T02:52:22.012254+0000","last_became_active":"2023-12-01T02:52:22.006615+0000","last_became_peered":"2023-12-01T02:52:22.006615+0000","last_unstale":"2023-12-01T02:52:22.012254+0000","last_undegraded":"2023-12-01T02:52:22.012254+0000","last_fullsized":"2023-12-01T02:52:22.012254+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:41.511561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"176'1","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010136+0000","last_change":"2023-12-01T02:52:21.519085+0000","last_active":"2023-12-01T02:52:22.010136+0000","last_peered":"2023-12-01T02:52:22.010136+0000","last_clean":"2023-12-01T02:52:22.010136+0000","last_became_active":"2023-12-01T02:52:21.518796+0000","last_became_peered":"2023-12-01T02:52:21.518796+0000","last_unstale":"2023-12-01T02:52:22.010136+0000","last_undegraded":"2023-12-01T02:52:22.010136+0000","last_fullsized":"2023-12-01T02:52:22.010136+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:40.990842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.998537+0000","last_change":"2023-12-01T02:52:21.989289+0000","last_active":"2023-12-01T02:52:21.998537+0000","last_peered":"2023-12-01T02:52:21.998537+0000","last_clean":"2023-12-01T02:52:21.998537+0000","last_became_active":"2023-12-01T02:52:21.989010+0000","last_became_peered":"2023-12-01T02:52:21.989010+0000","last_unstale":"2023-12-01T02:52:21.998537+0000","last_undegraded":"2023-12-01T02:52:21.998537+0000","last_fullsized":"2023-12-01T02:52:21.998537+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:42.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"176'1","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.002482+0000","last_change":"2023-12-01T02:52:21.522775+0000","last_active":"2023-12-01T02:52:22.002482+0000","last_peered":"2023-12-01T02:52:22.002482+0000","last_clean":"2023-12-01T02:52:22.002482+0000","last_became_active":"2023-12-01T02:52:21.522180+0000","last_became_peered":"2023-12-01T02:52:21.522180+0000","last_unstale":"2023-12-01T02:52:22.002482+0000","last_undegraded":"2023-12-01T02:52:22.002482+0000","last_fullsized":"2023-12-01T02:52:22.002482+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:34.788423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.013572+0000","last_change":"2023-12-01T02:52:22.011979+0000","last_active":"2023-12-01T02:52:22.013572+0000","last_peered":"2023-12-01T02:52:22.013572+0000","last_clean":"2023-12-01T02:52:22.013572+0000","last_became_active":"2023-12-01T02:52:22.011003+0000","last_became_peered":"2023-12-01T02:52:22.011003+0000","last_unstale":"2023-12-01T02:52:22.013572+0000","last_undegraded":"2023-12-01T02:52:22.013572+0000","last_fullsized":"2023-12-01T02:52:22.013572+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:04.052509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.983010+0000","last_change":"2023-12-01T02:52:21.516907+0000","last_active":"2023-12-01T02:52:21.983010+0000","last_peered":"2023-12-01T02:52:21.983010+0000","last_clean":"2023-12-01T02:52:21.983010+0000","last_became_active":"2023-12-01T02:52:21.516334+0000","last_became_peered":"2023-12-01T02:52:21.516334+0000","last_unstale":"2023-12-01T02:52:21.983010+0000","last_undegraded":"2023-12-01T02:52:21.983010+0000","last_fullsized":"2023-12-01T02:52:21.983010+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:22.856154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.002820+0000","last_change":"2023-12-01T02:52:21.993975+0000","last_active":"2023-12-01T02:52:22.002820+0000","last_peered":"2023-12-01T02:52:22.002820+0000","last_clean":"2023-12-01T02:52:22.002820+0000","last_became_active":"2023-12-01T02:52:21.993703+0000","last_became_peered":"2023-12-01T02:52:21.993703+0000","last_unstale":"2023-12-01T02:52:22.002820+0000","last_undegraded":"2023-12-01T02:52:22.002820+0000","last_fullsized":"2023-12-01T02:52:22.002820+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.976159+0000","last_change":"2023-12-01T02:52:21.517632+0000","last_active":"2023-12-01T02:52:21.976159+0000","last_peered":"2023-12-01T02:52:21.976159+0000","last_clean":"2023-12-01T02:52:21.976159+0000","last_became_active":"2023-12-01T02:52:21.517198+0000","last_became_peered":"2023-12-01T02:52:21.517198+0000","last_unstale":"2023-12-01T02:52:21.976159+0000","last_undegraded":"2023-12-01T02:52:21.976159+0000","last_fullsized":"2023-12-01T02:52:21.976159+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:48:39.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.995069+0000","last_change":"2023-12-01T02:52:21.991298+0000","last_active":"2023-12-01T02:52:21.995069+0000","last_peered":"2023-12-01T02:52:21.995069+0000","last_clean":"2023-12-01T02:52:21.995069+0000","last_became_active":"2023-12-01T02:52:21.991036+0000","last_became_peered":"2023-12-01T02:52:21.991036+0000","last_unstale":"2023-12-01T02:52:21.995069+0000","last_undegraded":"2023-12-01T02:52:21.995069+0000","last_fullsized":"2023-12-01T02:52:21.995069+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:09.328057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.988861+0000","last_change":"2023-12-01T02:52:21.516838+0000","last_active":"2023-12-01T02:52:21.988861+0000","last_peered":"2023-12-01T02:52:21.988861+0000","last_clean":"2023-12-01T02:52:21.988861+0000","last_became_active":"2023-12-01T02:52:21.516232+0000","last_became_peered":"2023-12-01T02:52:21.516232+0000","last_unstale":"2023-12-01T02:52:21.988861+0000","last_undegraded":"2023-12-01T02:52:21.988861+0000","last_fullsized":"2023-12-01T02:52:21.988861+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:08.328341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.952362+0000","last_change":"2023-12-01T02:52:21.998409+0000","last_active":"2023-12-01T02:52:23.952362+0000","last_peered":"2023-12-01T02:52:23.952362+0000","last_clean":"2023-12-01T02:52:23.952362+0000","last_became_active":"2023-12-01T02:52:21.998162+0000","last_became_peered":"2023-12-01T02:52:21.998162+0000","last_unstale":"2023-12-01T02:52:23.952362+0000","last_undegraded":"2023-12-01T02:52:23.952362+0000","last_fullsized":"2023-12-01T02:52:23.952362+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:54.304823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.985052+0000","last_change":"2023-12-01T02:52:21.518112+0000","last_active":"2023-12-01T02:52:21.985052+0000","last_peered":"2023-12-01T02:52:21.985052+0000","last_clean":"2023-12-01T02:52:21.985052+0000","last_became_active":"2023-12-01T02:52:21.517851+0000","last_became_peered":"2023-12-01T02:52:21.517851+0000","last_unstale":"2023-12-01T02:52:21.985052+0000","last_undegraded":"2023-12-01T02:52:21.985052+0000","last_fullsized":"2023-12-01T02:52:21.985052+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:46.753790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.951483+0000","last_change":"2023-12-01T02:52:22.002998+0000","last_active":"2023-12-01T02:52:23.951483+0000","last_peered":"2023-12-01T02:52:23.951483+0000","last_clean":"2023-12-01T02:52:23.951483+0000","last_became_active":"2023-12-01T02:52:22.002649+0000","last_became_peered":"2023-12-01T02:52:22.002649+0000","last_unstale":"2023-12-01T02:52:23.951483+0000","last_undegraded":"2023-12-01T02:52:23.951483+0000","last_fullsized":"2023-12-01T02:52:23.951483+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:40.623736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"176'6","reported_seq":17,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.015126+0000","last_change":"2023-12-01T02:52:21.514385+0000","last_active":"2023-12-01T02:52:22.015126+0000","last_peered":"2023-12-01T02:52:22.015126+0000","last_clean":"2023-12-01T02:52:22.015126+0000","last_became_active":"2023-12-01T02:52:21.513977+0000","last_became_peered":"2023-12-01T02:52:21.513977+0000","last_unstale":"2023-12-01T02:52:22.015126+0000","last_undegraded":"2023-12-01T02:52:22.015126+0000","last_fullsized":"2023-12-01T02:52:22.015126+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:57.056061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.988752+0000","last_change":"2023-12-01T02:52:21.987166+0000","last_active":"2023-12-01T02:52:21.988752+0000","last_peered":"2023-12-01T02:52:21.988752+0000","last_clean":"2023-12-01T02:52:21.988752+0000","last_became_active":"2023-12-01T02:52:21.986924+0000","last_became_peered":"2023-12-01T02:52:21.986924+0000","last_unstale":"2023-12-01T02:52:21.988752+0000","last_undegraded":"2023-12-01T02:52:21.988752+0000","last_fullsized":"2023-12-01T02:52:21.988752+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:20:16.758051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.011353+0000","last_change":"2023-12-01T02:52:21.518964+0000","last_active":"2023-12-01T02:52:22.011353+0000","last_peered":"2023-12-01T02:52:22.011353+0000","last_clean":"2023-12-01T02:52:22.011353+0000","last_became_active":"2023-12-01T02:52:21.518401+0000","last_became_peered":"2023-12-01T02:52:21.518401+0000","last_unstale":"2023-12-01T02:52:22.011353+0000","last_undegraded":"2023-12-01T02:52:22.011353+0000","last_fullsized":"2023-12-01T02:52:22.011353+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:16.375606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010113+0000","last_change":"2023-12-01T02:52:22.006473+0000","last_active":"2023-12-01T02:52:22.010113+0000","last_peered":"2023-12-01T02:52:22.010113+0000","last_clean":"2023-12-01T02:52:22.010113+0000","last_became_active":"2023-12-01T02:52:22.004904+0000","last_became_peered":"2023-12-01T02:52:22.004904+0000","last_unstale":"2023-12-01T02:52:22.010113+0000","last_undegraded":"2023-12-01T02:52:22.010113+0000","last_fullsized":"2023-12-01T02:52:22.010113+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:34.006926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.980070+0000","last_change":"2023-12-01T02:52:21.517770+0000","last_active":"2023-12-01T02:52:21.980070+0000","last_peered":"2023-12-01T02:52:21.980070+0000","last_clean":"2023-12-01T02:52:21.980070+0000","last_became_active":"2023-12-01T02:52:21.517465+0000","last_became_peered":"2023-12-01T02:52:21.517465+0000","last_unstale":"2023-12-01T02:52:21.980070+0000","last_undegraded":"2023-12-01T02:52:21.980070+0000","last_fullsized":"2023-12-01T02:52:21.980070+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:37.282563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.004069+0000","last_change":"2023-12-01T02:52:21.997830+0000","last_active":"2023-12-01T02:52:22.004069+0000","last_peered":"2023-12-01T02:52:22.004069+0000","last_clean":"2023-12-01T02:52:22.004069+0000","last_became_active":"2023-12-01T02:52:21.997562+0000","last_became_peered":"2023-12-01T02:52:21.997562+0000","last_unstale":"2023-12-01T02:52:22.004069+0000","last_undegraded":"2023-12-01T02:52:22.004069+0000","last_fullsized":"2023-12-01T02:52:22.004069+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:13.272131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.996870+0000","last_change":"2023-12-01T02:52:21.517568+0000","last_active":"2023-12-01T02:52:21.996870+0000","last_peered":"2023-12-01T02:52:21.996870+0000","last_clean":"2023-12-01T02:52:21.996870+0000","last_became_active":"2023-12-01T02:52:21.517259+0000","last_became_peered":"2023-12-01T02:52:21.517259+0000","last_unstale":"2023-12-01T02:52:21.996870+0000","last_undegraded":"2023-12-01T02:52:21.996870+0000","last_fullsized":"2023-12-01T02:52:21.996870+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:11:02.383031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.997002+0000","last_change":"2023-12-01T02:52:21.992889+0000","last_active":"2023-12-01T02:52:21.997002+0000","last_peered":"2023-12-01T02:52:21.997002+0000","last_clean":"2023-12-01T02:52:21.997002+0000","last_became_active":"2023-12-01T02:52:21.992598+0000","last_became_peered":"2023-12-01T02:52:21.992598+0000","last_unstale":"2023-12-01T02:52:21.997002+0000","last_undegraded":"2023-12-01T02:52:21.997002+0000","last_fullsized":"2023-12-01T02:52:21.997002+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:16:32.491266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"176'8","reported_seq":19,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.015653+0000","last_change":"2023-12-01T02:52:21.515675+0000","last_active":"2023-12-01T02:52:22.015653+0000","last_peered":"2023-12-01T02:52:22.015653+0000","last_clean":"2023-12-01T02:52:22.015653+0000","last_became_active":"2023-12-01T02:52:21.515354+0000","last_became_peered":"2023-12-01T02:52:21.515354+0000","last_unstale":"2023-12-01T02:52:22.015653+0000","last_undegraded":"2023-12-01T02:52:22.015653+0000","last_fullsized":"2023-12-01T02:52:22.015653+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:24:04.462724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.998760+0000","last_change":"2023-12-01T02:52:21.996419+0000","last_active":"2023-12-01T02:52:21.998760+0000","last_peered":"2023-12-01T02:52:21.998760+0000","last_clean":"2023-12-01T02:52:21.998760+0000","last_became_active":"2023-12-01T02:52:21.992392+0000","last_became_peered":"2023-12-01T02:52:21.992392+0000","last_unstale":"2023-12-01T02:52:21.998760+0000","last_undegraded":"2023-12-01T02:52:21.998760+0000","last_fullsized":"2023-12-01T02:52:21.998760+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:46:29.228752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"176'1","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.011955+0000","last_change":"2023-12-01T02:52:21.515254+0000","last_active":"2023-12-01T02:52:22.011955+0000","last_peered":"2023-12-01T02:52:22.011955+0000","last_clean":"2023-12-01T02:52:22.011955+0000","last_became_active":"2023-12-01T02:52:21.514932+0000","last_became_peered":"2023-12-01T02:52:21.514932+0000","last_unstale":"2023-12-01T02:52:22.011955+0000","last_undegraded":"2023-12-01T02:52:22.011955+0000","last_fullsized":"2023-12-01T02:52:22.011955+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:56.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010976+0000","last_change":"2023-12-01T02:52:22.004977+0000","last_active":"2023-12-01T02:52:22.010976+0000","last_peered":"2023-12-01T02:52:22.010976+0000","last_clean":"2023-12-01T02:52:22.010976+0000","last_became_active":"2023-12-01T02:52:22.004594+0000","last_became_peered":"2023-12-01T02:52:22.004594+0000","last_unstale":"2023-12-01T02:52:22.010976+0000","last_undegraded":"2023-12-01T02:52:22.010976+0000","last_fullsized":"2023-12-01T02:52:22.010976+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:32.342375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.014084+0000","last_change":"2023-12-01T02:52:21.516963+0000","last_active":"2023-12-01T02:52:22.014084+0000","last_peered":"2023-12-01T02:52:22.014084+0000","last_clean":"2023-12-01T02:52:22.014084+0000","last_became_active":"2023-12-01T02:52:21.516690+0000","last_became_peered":"2023-12-01T02:52:21.516690+0000","last_unstale":"2023-12-01T02:52:22.014084+0000","last_undegraded":"2023-12-01T02:52:22.014084+0000","last_fullsized":"2023-12-01T02:52:22.014084+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.001179+0000","last_change":"2023-12-01T02:52:21.999699+0000","last_active":"2023-12-01T02:52:22.001179+0000","last_peered":"2023-12-01T02:52:22.001179+0000","last_clean":"2023-12-01T02:52:22.001179+0000","last_became_active":"2023-12-01T02:52:21.999434+0000","last_became_peered":"2023-12-01T02:52:21.999434+0000","last_unstale":"2023-12-01T02:52:22.001179+0000","last_undegraded":"2023-12-01T02:52:22.001179+0000","last_fullsized":"2023-12-01T02:52:22.001179+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:24:11.838238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"176'1","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.007470+0000","last_change":"2023-12-01T02:52:21.518990+0000","last_active":"2023-12-01T02:52:22.007470+0000","last_peered":"2023-12-01T02:52:22.007470+0000","last_clean":"2023-12-01T02:52:22.007470+0000","last_became_active":"2023-12-01T02:52:21.518653+0000","last_became_peered":"2023-12-01T02:52:21.518653+0000","last_unstale":"2023-12-01T02:52:22.007470+0000","last_undegraded":"2023-12-01T02:52:22.007470+0000","last_fullsized":"2023-12-01T02:52:22.007470+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:23.080969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.148377+0000","last_change":"2023-12-01T02:52:21.991178+0000","last_active":"2023-12-01T02:52:23.148377+0000","last_peered":"2023-12-01T02:52:23.148377+0000","last_clean":"2023-12-01T02:52:23.148377+0000","last_became_active":"2023-12-01T02:52:21.989781+0000","last_became_peered":"2023-12-01T02:52:21.989781+0000","last_unstale":"2023-12-01T02:52:23.148377+0000","last_undegraded":"2023-12-01T02:52:23.148377+0000","last_fullsized":"2023-12-01T02:52:23.148377+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:13.272131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.999491+0000","last_change":"2023-12-01T02:52:21.509641+0000","last_active":"2023-12-01T02:52:21.999491+0000","last_peered":"2023-12-01T02:52:21.999491+0000","last_clean":"2023-12-01T02:52:21.999491+0000","last_became_active":"2023-12-01T02:52:21.508511+0000","last_became_peered":"2023-12-01T02:52:21.508511+0000","last_unstale":"2023-12-01T02:52:21.999491+0000","last_undegraded":"2023-12-01T02:52:21.999491+0000","last_fullsized":"2023-12-01T02:52:21.999491+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:44:46.015030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.007550+0000","last_change":"2023-12-01T02:52:22.001107+0000","last_active":"2023-12-01T02:52:22.007550+0000","last_peered":"2023-12-01T02:52:22.007550+0000","last_clean":"2023-12-01T02:52:22.007550+0000","last_became_active":"2023-12-01T02:52:21.999794+0000","last_became_peered":"2023-12-01T02:52:21.999794+0000","last_unstale":"2023-12-01T02:52:22.007550+0000","last_undegraded":"2023-12-01T02:52:22.007550+0000","last_fullsized":"2023-12-01T02:52:22.007550+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.008400+0000","last_change":"2023-12-01T02:52:21.521786+0000","last_active":"2023-12-01T02:52:22.008400+0000","last_peered":"2023-12-01T02:52:22.008400+0000","last_clean":"2023-12-01T02:52:22.008400+0000","last_became_active":"2023-12-01T02:52:21.521463+0000","last_became_peered":"2023-12-01T02:52:21.521463+0000","last_unstale":"2023-12-01T02:52:22.008400+0000","last_undegraded":"2023-12-01T02:52:22.008400+0000","last_fullsized":"2023-12-01T02:52:22.008400+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:36.292725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.011636+0000","last_change":"2023-12-01T02:52:22.005777+0000","last_active":"2023-12-01T02:52:22.011636+0000","last_peered":"2023-12-01T02:52:22.011636+0000","last_clean":"2023-12-01T02:52:22.011636+0000","last_became_active":"2023-12-01T02:52:22.003932+0000","last_became_peered":"2023-12-01T02:52:22.003932+0000","last_unstale":"2023-12-01T02:52:22.011636+0000","last_undegraded":"2023-12-01T02:52:22.011636+0000","last_fullsized":"2023-12-01T02:52:22.011636+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.977144+0000","last_change":"2023-12-01T02:52:21.516256+0000","last_active":"2023-12-01T02:52:21.977144+0000","last_peered":"2023-12-01T02:52:21.977144+0000","last_clean":"2023-12-01T02:52:21.977144+0000","last_became_active":"2023-12-01T02:52:21.515650+0000","last_became_peered":"2023-12-01T02:52:21.515650+0000","last_unstale":"2023-12-01T02:52:21.977144+0000","last_undegraded":"2023-12-01T02:52:21.977144+0000","last_fullsized":"2023-12-01T02:52:21.977144+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:30.297120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.007643+0000","last_change":"2023-12-01T02:52:22.000613+0000","last_active":"2023-12-01T02:52:22.007643+0000","last_peered":"2023-12-01T02:52:22.007643+0000","last_clean":"2023-12-01T02:52:22.007643+0000","last_became_active":"2023-12-01T02:52:22.000378+0000","last_became_peered":"2023-12-01T02:52:22.000378+0000","last_unstale":"2023-12-01T02:52:22.007643+0000","last_undegraded":"2023-12-01T02:52:22.007643+0000","last_fullsized":"2023-12-01T02:52:22.007643+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:42.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.975540+0000","last_change":"2023-12-01T02:52:21.513604+0000","last_active":"2023-12-01T02:52:21.975540+0000","last_peered":"2023-12-01T02:52:21.975540+0000","last_clean":"2023-12-01T02:52:21.975540+0000","last_became_active":"2023-12-01T02:52:21.512902+0000","last_became_peered":"2023-12-01T02:52:21.512902+0000","last_unstale":"2023-12-01T02:52:21.975540+0000","last_undegraded":"2023-12-01T02:52:21.975540+0000","last_fullsized":"2023-12-01T02:52:21.975540+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:52.712427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.008544+0000","last_change":"2023-12-01T02:52:22.003588+0000","last_active":"2023-12-01T02:52:22.008544+0000","last_peered":"2023-12-01T02:52:22.008544+0000","last_clean":"2023-12-01T02:52:22.008544+0000","last_became_active":"2023-12-01T02:52:22.002988+0000","last_became_peered":"2023-12-01T02:52:22.002988+0000","last_unstale":"2023-12-01T02:52:22.008544+0000","last_undegraded":"2023-12-01T02:52:22.008544+0000","last_fullsized":"2023-12-01T02:52:22.008544+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:37.307890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.974887+0000","last_change":"2023-12-01T02:52:21.519031+0000","last_active":"2023-12-01T02:52:21.974887+0000","last_peered":"2023-12-01T02:52:21.974887+0000","last_clean":"2023-12-01T02:52:21.974887+0000","last_became_active":"2023-12-01T02:52:21.518624+0000","last_became_peered":"2023-12-01T02:52:21.518624+0000","last_unstale":"2023-12-01T02:52:21.974887+0000","last_undegraded":"2023-12-01T02:52:21.974887+0000","last_fullsized":"2023-12-01T02:52:21.974887+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:04:47.352377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.001915+0000","last_change":"2023-12-01T02:52:21.994986+0000","last_active":"2023-12-01T02:52:22.001915+0000","last_peered":"2023-12-01T02:52:22.001915+0000","last_clean":"2023-12-01T02:52:22.001915+0000","last_became_active":"2023-12-01T02:52:21.994737+0000","last_became_peered":"2023-12-01T02:52:21.994737+0000","last_unstale":"2023-12-01T02:52:22.001915+0000","last_undegraded":"2023-12-01T02:52:22.001915+0000","last_fullsized":"2023-12-01T02:52:22.001915+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:48:36.231827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.008744+0000","last_change":"2023-12-01T02:52:21.516514+0000","last_active":"2023-12-01T02:52:22.008744+0000","last_peered":"2023-12-01T02:52:22.008744+0000","last_clean":"2023-12-01T02:52:22.008744+0000","last_became_active":"2023-12-01T02:52:21.516229+0000","last_became_peered":"2023-12-01T02:52:21.516229+0000","last_unstale":"2023-12-01T02:52:22.008744+0000","last_undegraded":"2023-12-01T02:52:22.008744+0000","last_fullsized":"2023-12-01T02:52:22.008744+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:19.139617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.004002+0000","last_change":"2023-12-01T02:52:21.995168+0000","last_active":"2023-12-01T02:52:22.004002+0000","last_peered":"2023-12-01T02:52:22.004002+0000","last_clean":"2023-12-01T02:52:22.004002+0000","last_became_active":"2023-12-01T02:52:21.994888+0000","last_became_peered":"2023-12-01T02:52:21.994888+0000","last_unstale":"2023-12-01T02:52:22.004002+0000","last_undegraded":"2023-12-01T02:52:22.004002+0000","last_fullsized":"2023-12-01T02:52:22.004002+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:54.304823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.998557+0000","last_change":"2023-12-01T02:52:21.510073+0000","last_active":"2023-12-01T02:52:21.998557+0000","last_peered":"2023-12-01T02:52:21.998557+0000","last_clean":"2023-12-01T02:52:21.998557+0000","last_became_active":"2023-12-01T02:52:21.509784+0000","last_became_peered":"2023-12-01T02:52:21.509784+0000","last_unstale":"2023-12-01T02:52:21.998557+0000","last_undegraded":"2023-12-01T02:52:21.998557+0000","last_fullsized":"2023-12-01T02:52:21.998557+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:36.014503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.009649+0000","last_change":"2023-12-01T02:52:22.005229+0000","last_active":"2023-12-01T02:52:22.009649+0000","last_peered":"2023-12-01T02:52:22.009649+0000","last_clean":"2023-12-01T02:52:22.009649+0000","last_became_active":"2023-12-01T02:52:22.004891+0000","last_became_peered":"2023-12-01T02:52:22.004891+0000","last_unstale":"2023-12-01T02:52:22.009649+0000","last_undegraded":"2023-12-01T02:52:22.009649+0000","last_fullsized":"2023-12-01T02:52:22.009649+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:51.425590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.988153+0000","last_change":"2023-12-01T02:52:21.512850+0000","last_active":"2023-12-01T02:52:21.988153+0000","last_peered":"2023-12-01T02:52:21.988153+0000","last_clean":"2023-12-01T02:52:21.988153+0000","last_became_active":"2023-12-01T02:52:21.509201+0000","last_became_peered":"2023-12-01T02:52:21.509201+0000","last_unstale":"2023-12-01T02:52:21.988153+0000","last_undegraded":"2023-12-01T02:52:21.988153+0000","last_fullsized":"2023-12-01T02:52:21.988153+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:50.410425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.995510+0000","last_change":"2023-12-01T02:52:21.991927+0000","last_active":"2023-12-01T02:52:21.995510+0000","last_peered":"2023-12-01T02:52:21.995510+0000","last_clean":"2023-12-01T02:52:21.995510+0000","last_became_active":"2023-12-01T02:52:21.991483+0000","last_became_peered":"2023-12-01T02:52:21.991483+0000","last_unstale":"2023-12-01T02:52:21.995510+0000","last_undegraded":"2023-12-01T02:52:21.995510+0000","last_fullsized":"2023-12-01T02:52:21.995510+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:22:04.947371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.985750+0000","last_change":"2023-12-01T02:52:21.514300+0000","last_active":"2023-12-01T02:52:21.985750+0000","last_peered":"2023-12-01T02:52:21.985750+0000","last_clean":"2023-12-01T02:52:21.985750+0000","last_became_active":"2023-12-01T02:52:21.513915+0000","last_became_peered":"2023-12-01T02:52:21.513915+0000","last_unstale":"2023-12-01T02:52:21.985750+0000","last_undegraded":"2023-12-01T02:52:21.985750+0000","last_fullsized":"2023-12-01T02:52:21.985750+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:15:51.909423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.951309+0000","last_change":"2023-12-01T02:52:21.991107+0000","last_active":"2023-12-01T02:52:23.951309+0000","last_peered":"2023-12-01T02:52:23.951309+0000","last_clean":"2023-12-01T02:52:23.951309+0000","last_became_active":"2023-12-01T02:52:21.990671+0000","last_became_peered":"2023-12-01T02:52:21.990671+0000","last_unstale":"2023-12-01T02:52:23.951309+0000","last_undegraded":"2023-12-01T02:52:23.951309+0000","last_fullsized":"2023-12-01T02:52:23.951309+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:48:40.093953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.001018+0000","last_change":"2023-12-01T02:52:21.500628+0000","last_active":"2023-12-01T02:52:22.001018+0000","last_peered":"2023-12-01T02:52:22.001018+0000","last_clean":"2023-12-01T02:52:22.001018+0000","last_became_active":"2023-12-01T02:52:21.499540+0000","last_became_peered":"2023-12-01T02:52:21.499540+0000","last_unstale":"2023-12-01T02:52:22.001018+0000","last_undegraded":"2023-12-01T02:52:22.001018+0000","last_fullsized":"2023-12-01T02:52:22.001018+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:14.533994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.003977+0000","last_change":"2023-12-01T02:52:21.995598+0000","last_active":"2023-12-01T02:52:22.003977+0000","last_peered":"2023-12-01T02:52:22.003977+0000","last_clean":"2023-12-01T02:52:22.003977+0000","last_became_active":"2023-12-01T02:52:21.995297+0000","last_became_peered":"2023-12-01T02:52:21.995297+0000","last_unstale":"2023-12-01T02:52:22.003977+0000","last_undegraded":"2023-12-01T02:52:22.003977+0000","last_fullsized":"2023-12-01T02:52:22.003977+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:30:41.584477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.978908+0000","last_change":"2023-12-01T02:52:21.521596+0000","last_active":"2023-12-01T02:52:21.978908+0000","last_peered":"2023-12-01T02:52:21.978908+0000","last_clean":"2023-12-01T02:52:21.978908+0000","last_became_active":"2023-12-01T02:52:21.520352+0000","last_became_peered":"2023-12-01T02:52:21.520352+0000","last_unstale":"2023-12-01T02:52:21.978908+0000","last_undegraded":"2023-12-01T02:52:21.978908+0000","last_fullsized":"2023-12-01T02:52:21.978908+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:39.608571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.008054+0000","last_change":"2023-12-01T02:52:22.001332+0000","last_active":"2023-12-01T02:52:22.008054+0000","last_peered":"2023-12-01T02:52:22.008054+0000","last_clean":"2023-12-01T02:52:22.008054+0000","last_became_active":"2023-12-01T02:52:22.001081+0000","last_became_peered":"2023-12-01T02:52:22.001081+0000","last_unstale":"2023-12-01T02:52:22.008054+0000","last_undegraded":"2023-12-01T02:52:22.008054+0000","last_fullsized":"2023-12-01T02:52:22.008054+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:24:05.477889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.980630+0000","last_change":"2023-12-01T02:52:21.512432+0000","last_active":"2023-12-01T02:52:21.980630+0000","last_peered":"2023-12-01T02:52:21.980630+0000","last_clean":"2023-12-01T02:52:21.980630+0000","last_became_active":"2023-12-01T02:52:21.512157+0000","last_became_peered":"2023-12-01T02:52:21.512157+0000","last_unstale":"2023-12-01T02:52:21.980630+0000","last_undegraded":"2023-12-01T02:52:21.980630+0000","last_fullsized":"2023-12-01T02:52:21.980630+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:09.305110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.980846+0000","last_change":"2023-12-01T02:52:21.995816+0000","last_active":"2023-12-01T02:52:22.980846+0000","last_peered":"2023-12-01T02:52:22.980846+0000","last_clean":"2023-12-01T02:52:22.980846+0000","last_became_active":"2023-12-01T02:52:21.995554+0000","last_became_peered":"2023-12-01T02:52:21.995554+0000","last_unstale":"2023-12-01T02:52:22.980846+0000","last_undegraded":"2023-12-01T02:52:22.980846+0000","last_fullsized":"2023-12-01T02:52:22.980846+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:32:56.513251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.982797+0000","last_change":"2023-12-01T02:52:21.518362+0000","last_active":"2023-12-01T02:52:21.982797+0000","last_peered":"2023-12-01T02:52:21.982797+0000","last_clean":"2023-12-01T02:52:21.982797+0000","last_became_active":"2023-12-01T02:52:21.518064+0000","last_became_peered":"2023-12-01T02:52:21.518064+0000","last_unstale":"2023-12-01T02:52:21.982797+0000","last_undegraded":"2023-12-01T02:52:21.982797+0000","last_fullsized":"2023-12-01T02:52:21.982797+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:31.327211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.951405+0000","last_change":"2023-12-01T02:52:21.995129+0000","last_active":"2023-12-01T02:52:23.951405+0000","last_peered":"2023-12-01T02:52:23.951405+0000","last_clean":"2023-12-01T02:52:23.951405+0000","last_became_active":"2023-12-01T02:52:21.994473+0000","last_became_peered":"2023-12-01T02:52:21.994473+0000","last_unstale":"2023-12-01T02:52:23.951405+0000","last_undegraded":"2023-12-01T02:52:23.951405+0000","last_fullsized":"2023-12-01T02:52:23.951405+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:17.390771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"176'3","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.999416+0000","last_change":"2023-12-01T02:52:21.511311+0000","last_active":"2023-12-01T02:52:21.999416+0000","last_peered":"2023-12-01T02:52:21.999416+0000","last_clean":"2023-12-01T02:52:21.999416+0000","last_became_active":"2023-12-01T02:52:21.511021+0000","last_became_peered":"2023-12-01T02:52:21.511021+0000","last_unstale":"2023-12-01T02:52:21.999416+0000","last_undegraded":"2023-12-01T02:52:21.999416+0000","last_fullsized":"2023-12-01T02:52:21.999416+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:42.785131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"165'271","reported_seq":882,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.892693+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T02:52:23.892693+0000","last_peered":"2023-12-01T02:52:23.892693+0000","last_clean":"2023-12-01T02:52:23.892693+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T02:52:23.892693+0000","last_undegraded":"2023-12-01T02:52:23.892693+0000","last_fullsized":"2023-12-01T02:52:23.892693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575763,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55348,"kb_used_data":2156,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316492,"statfs":{"total":96636764160,"available":96580087808,"internally_reserved":0,"allocated":2207744,"data_stored":1065606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3799999999999999,"5min":1.7829999999999999,"15min":1.893},"last":1.26},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1539999999999999,"15min":1.1100000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.514,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.266,"15min":1.2709999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.02},"last":1.647},{"interface":"front","average":{"1min":1.26,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85399999999999998},"max":{"1min":1.585,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":1.5880000000000001}]},{"osd":2,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.2010000000000001},"min":{"1min":0.82499999999999996,"5min":0.80600000000000005,"15min":0.78500000000000003},"max":{"1min":1.554,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.28,"15min":1.25},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8440000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.534}]},{"osd":3,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.343,"15min":1.3220000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.742,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.048,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.7509999999999999}]},{"osd":4,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.851,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.506,"5min":1.4930000000000001,"15min":1.4750000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.157,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.9179999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3149999999999999,"15min":1.284},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.0699999999999998,"5min":2.976,"15min":2.976},"last":1.337},{"interface":"front","average":{"1min":1.597,"5min":1.595,"15min":1.569},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.2160000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.857}]},{"osd":6,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4590000000000001,"15min":1.4179999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.1499999999999999,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4870000000000001,"15min":1.4470000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.966,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.4350000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575764,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37300,"kb_used_data":2156,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334540,"statfs":{"total":96636764160,"available":96598568960,"internally_reserved":0,"allocated":2207744,"data_stored":1065606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.161,"15min":1.165},"min":{"1min":0.84299999999999997,"5min":0.78100000000000003,"15min":0.75},"max":{"1min":1.6080000000000001,"5min":1.73,"15min":2.2759999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.083,"5min":1.1439999999999999,"15min":1.1619999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.56799999999999995},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.966},"last":0.98699999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1459999999999999,"15min":1.1859999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.78800000000000003},"max":{"1min":1.6459999999999999,"5min":1.772,"15min":2.0409999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.2,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6679999999999999,"5min":1.7370000000000001,"15min":2.093},"last":0.94599999999999995}]},{"osd":2,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1679999999999999,"15min":1.1879999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.80600000000000005},"max":{"1min":1.5369999999999999,"5min":1.736,"15min":1.946},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.272,"15min":1.284},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.637,"5min":1.7190000000000001,"15min":2.1230000000000002},"last":1.343}]},{"osd":3,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":1.0620000000000001,"5min":1.006,"15min":0.90500000000000003},"max":{"1min":1.6140000000000001,"5min":1.819,"15min":1.819},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.431,"15min":1.4319999999999999},"min":{"1min":1.042,"5min":0.98699999999999999,"15min":0.92800000000000005},"max":{"1min":1.738,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.4890000000000001}]},{"osd":4,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.262},"min":{"1min":0.93000000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.698},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3500000000000001,"15min":1.3360000000000001},"min":{"1min":0.96699999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.98,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.847}]},{"osd":5,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.4419999999999999,"15min":1.4279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.8120000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.5,"15min":1.4970000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.7799999999999998},"last":1.6459999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.5149999999999999,"15min":1.524},"min":{"1min":1.0980000000000001,"5min":1.034,"15min":1.034},"max":{"1min":1.9419999999999999,"5min":2.1549999999999998,"15min":2.5659999999999998},"last":1.891},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6040000000000001,"15min":1.5940000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.0499999999999998,"5min":2.226,"15min":2.3809999999999998},"last":1.7909999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575763,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64144,"kb_used_data":2760,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307696,"statfs":{"total":96636764160,"available":96571080704,"internally_reserved":0,"allocated":2826240,"data_stored":1657732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1499999999999999,"15min":1.1419999999999999},"min":{"1min":0.68700000000000006,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4810000000000001,"5min":1.73,"15min":1.744},"last":0.625},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1000000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.137,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.56200000000000006},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.704},"last":1.159},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.226,"15min":1.2330000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.016},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2270000000000001,"15min":1.24},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5900000000000001,"5min":1.7430000000000001,"15min":1.746},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2609999999999999,"15min":1.2809999999999999},"min":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.627},"max":{"1min":1.655,"5min":1.671,"15min":1.764},"last":1.282}]},{"osd":3,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.321,"15min":1.319},"min":{"1min":0.98399999999999999,"5min":0.872,"15min":0.84899999999999998},"max":{"1min":1.546,"5min":1.7729999999999999,"15min":1.871},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3460000000000001,"15min":1.3620000000000001},"min":{"1min":1.0269999999999999,"5min":0.91500000000000004,"15min":0.879},"max":{"1min":1.7250000000000001,"5min":1.7470000000000001,"15min":1.962},"last":1.395}]},{"osd":4,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.254,"15min":1.2569999999999999},"min":{"1min":0.91300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.696,"5min":1.847,"15min":1.978},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.391,"5min":1.361,"15min":1.3620000000000001},"min":{"1min":1.0740000000000001,"5min":0.91600000000000004,"15min":0.88600000000000001},"max":{"1min":1.911,"5min":1.911,"15min":2.0350000000000001},"last":1.3129999999999999}]},{"osd":6,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3520000000000001,"15min":1.375},"min":{"1min":0.89300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":2.4470000000000001},"last":1.526},{"interface":"front","average":{"1min":1.411,"5min":1.391,"15min":1.419},"min":{"1min":1.077,"5min":0.876,"15min":0.876},"max":{"1min":1.798,"5min":1.946,"15min":2.7210000000000001},"last":1.0509999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.5449999999999999,"15min":1.5369999999999999},"min":{"1min":1.1200000000000001,"5min":1.099,"15min":0.92100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":3.0379999999999998},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5780000000000001,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":3.5270000000000001},"last":1.472}]}]},{"osd":4,"up_from":13,"seq":55834575763,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":2176,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":2228224,"data_stored":1067774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.113,"15min":1.137},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.6180000000000001},"last":1.153},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1599999999999999,"15min":1.1699999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.3899999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.234,"15min":1.2270000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.48,"5min":1.706,"15min":1.7709999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.139,"15min":1.1830000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.381,"5min":1.6100000000000001,"15min":1.8640000000000001},"last":1.296}]},{"osd":2,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1910000000000001,"15min":1.218},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":2.1499999999999999},"last":1.472},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3240000000000001,"15min":1.3169999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.85999999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.1220000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2470000000000001,"15min":1.2609999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.454,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.016},{"interface":"front","average":{"1min":1.298,"5min":1.3260000000000001,"15min":1.329},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":2.2440000000000002},"last":1.343}]},{"osd":5,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.367,"15min":1.3140000000000001},"min":{"1min":0.92300000000000004,"5min":0.86499999999999999,"15min":0.84499999999999997},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3069999999999999,"15min":1.284},"min":{"1min":0.96699999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.712,"5min":1.875,"15min":1.875},"last":1.4430000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3580000000000001,"15min":1.325},"min":{"1min":0.82599999999999996,"5min":0.81299999999999994,"15min":0.78500000000000003},"max":{"1min":1.996,"5min":1.996,"15min":2.4489999999999998},"last":1.609},{"interface":"front","average":{"1min":1.554,"5min":1.474,"15min":1.448},"min":{"1min":1.097,"5min":1.014,"15min":0.97899999999999998},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":1.708}]},{"osd":7,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.444,"15min":1.4379999999999999},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.92900000000000005},"max":{"1min":1.905,"5min":2.044,"15min":2.044},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.552,"15min":1.544},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.8859999999999999,"5min":2.0670000000000002,"15min":2.5760000000000001},"last":1.671}]}]},{"osd":3,"up_from":13,"seq":55834575762,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51260,"kb_used_data":2164,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320580,"statfs":{"total":96636764160,"available":96584273920,"internally_reserved":0,"allocated":2215936,"data_stored":1066222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.90600000000000003,"15min":0.89300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.0309999999999999,"15min":1.0660000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3680000000000001,"5min":1.5449999999999999,"15min":1.5669999999999999},"last":1.081}]},{"osd":1,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.1459999999999999,"15min":1.145},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2709999999999999,"5min":1.708,"15min":1.708},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.97799999999999998,"15min":1.004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.329,"5min":1.5980000000000001,"15min":1.744},"last":1.3879999999999999}]},{"osd":2,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0800000000000001,"15min":1.097},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.171},{"interface":"front","average":{"1min":1.028,"5min":1.2050000000000001,"15min":1.2490000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4550000000000001,"5min":1.5740000000000001,"15min":1.728},"last":1.1970000000000001}]},{"osd":4,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.302,"15min":1.294},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.306,"15min":1.2949999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.67100000000000004},"max":{"1min":1.802,"5min":1.802,"15min":1.927},"last":1.127}]},{"osd":5,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.294},{"interface":"front","average":{"1min":1.377,"5min":1.3979999999999999,"15min":1.387},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.9769999999999999,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.482}]},{"osd":6,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.448,"15min":1.4379999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.67900000000000005},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.524,"15min":1.4970000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.554}]},{"osd":7,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.919,"5min":2.0790000000000002,"15min":2.1549999999999998},"last":1.673},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.5109999999999999,"15min":1.51},"min":{"1min":0.747,"5min":0.747,"15min":0.73699999999999999},"max":{"1min":1.827,"5min":2.0150000000000001,"15min":2.1259999999999999},"last":1.645}]}]},{"osd":2,"up_from":13,"seq":55834575762,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41400,"kb_used_data":2160,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330440,"statfs":{"total":96636764160,"available":96594370560,"internally_reserved":0,"allocated":2211840,"data_stored":1065628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":0.93200000000000005,"15min":0.88900000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.49299999999999999},"max":{"1min":5.2080000000000002,"5min":5.2080000000000002,"15min":5.2080000000000002},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.127,"5min":1.0860000000000001,"15min":1.073},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.3959999999999999,"5min":5.3959999999999999,"15min":5.3959999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1160000000000001,"15min":1.085},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":5.1109999999999998,"5min":5.1109999999999998,"15min":5.1109999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.1619999999999999,"15min":1.1259999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":5.4800000000000004,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.073,"15min":1.0549999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.0540000000000003,"5min":4.0540000000000003,"15min":4.0540000000000003},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.2869999999999999,"15min":1.2230000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":5.7690000000000001,"5min":5.7690000000000001,"15min":5.7690000000000001},"last":1}]},{"osd":4,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3779999999999999,"15min":1.327},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":5.5590000000000002,"5min":5.5590000000000002,"15min":5.5590000000000002},"last":1.579},{"interface":"front","average":{"1min":1.198,"5min":1.29,"15min":1.2949999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.5009999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.4119999999999999,"15min":1.379},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":5.8520000000000003,"5min":5.8520000000000003,"15min":5.8520000000000003},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.4610000000000001,"15min":1.4079999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":5.7279999999999998,"5min":5.7279999999999998,"15min":5.7279999999999998},"last":1.228}]},{"osd":6,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.397,"15min":1.3819999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5669999999999999,"15min":1.51},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":0.82699999999999996},"max":{"1min":5.8159999999999998,"5min":5.8159999999999998,"15min":5.8159999999999998},"last":1.274}]},{"osd":7,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.548,"15min":1.492},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4890000000000001,"15min":1.458},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.4339999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575761,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64124,"kb_used_data":2740,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307716,"statfs":{"total":96636764160,"available":96571101184,"internally_reserved":0,"allocated":2805760,"data_stored":1656044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.137,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.133},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.095,"15min":1.0740000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.722,"15min":1.722},"last":1.085}]},{"osd":2,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93200000000000005,"15min":0.92500000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.466,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.038},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1439999999999999,"15min":1.1479999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5880000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.389}]},{"osd":3,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5189999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.712},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1339999999999999,"15min":1.1279999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.444,"5min":1.6759999999999999,"15min":1.9219999999999999},"last":1.6599999999999999}]},{"osd":4,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.24,"15min":1.226},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.76},{"interface":"front","average":{"1min":1.413,"5min":1.4219999999999999,"15min":1.391},"min":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.032},"max":{"1min":1.8140000000000001,"5min":1.9730000000000001,"15min":2.1280000000000001},"last":1.6100000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.349},"min":{"1min":1.0620000000000001,"5min":0.98899999999999999,"15min":0.97899999999999998},"max":{"1min":1.49,"5min":1.9930000000000001,"15min":2.0449999999999999},"last":1.9590000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.3660000000000001,"15min":1.357},"min":{"1min":1.0349999999999999,"5min":1.0229999999999999,"15min":1.004},"max":{"1min":1.804,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.4359999999999999}]},{"osd":6,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.454,"15min":1.4399999999999999},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.873,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":2.1230000000000002},{"interface":"front","average":{"1min":1.341,"5min":1.4139999999999999,"15min":1.4119999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.7250000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.869}]},{"osd":7,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5289999999999999,"15min":1.518},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.9339999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.8220000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.516,"15min":1.514},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8029999999999999,"5min":2.1749999999999998,"15min":2.282},"last":2.448}]}]},{"osd":0,"up_from":13,"seq":55834575763,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41400,"kb_used_data":2160,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330440,"statfs":{"total":96636764160,"available":96594370560,"internally_reserved":0,"allocated":2211840,"data_stored":1065696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.88600000000000001,"15min":0.89700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2470000000000001,"5min":1.5309999999999999,"15min":1.766},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.345,"5min":1.5980000000000001,"15min":1.605},"last":0.98499999999999999}]},{"osd":2,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.034,"15min":1.056},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2410000000000001,"5min":1.4470000000000001,"15min":1.8540000000000001},"last":1.262},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.089,"15min":1.1220000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.464,"5min":1.595,"15min":1.7030000000000001},"last":1.4630000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":1.036,"15min":1.054},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1890000000000001,"5min":1.5509999999999999,"15min":1.8100000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.2270000000000001,"15min":1.244},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7230000000000001,"5min":1.798,"15min":1.9470000000000001},"last":1.401}]},{"osd":4,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.327,"15min":1.345},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.591,"5min":1.6910000000000001,"15min":2.0979999999999999},"last":1.341},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.258,"15min":1.282},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.651,"5min":1.6579999999999999,"15min":2.0739999999999998},"last":1.151}]},{"osd":5,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.407,"15min":1.3999999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6930000000000001,"5min":2.2370000000000001,"15min":2.6110000000000002},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.4079999999999999,"15min":1.4099999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.206}]},{"osd":6,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.377,"15min":1.367},"min":{"1min":0.77700000000000002,"5min":0.745,"15min":0.745},"max":{"1min":1.643,"5min":2.0939999999999999,"15min":2.1680000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.5009999999999999,"15min":1.4970000000000001},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":1.748,"5min":2.798,"15min":2.798},"last":1.9099999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.548,"15min":1.514},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.458,"15min":1.472},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.7689999999999999,"5min":2.0830000000000002,"15min":3.0590000000000002},"last":1.3819999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T02:52:31.447 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T02:52:31.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.663+0000 7fdeb4043640 1 Processor -- start 2023-12-01T02:52:31.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.663+0000 7fdeb4043640 1 -- start start 2023-12-01T02:52:31.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.664+0000 7fdeb4043640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdeac0cad70 0x7fdeac06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:31.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.664+0000 7fdeb4043640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac0add10 con 0x7fdeac15e2f0 2023-12-01T02:52:31.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.664+0000 7fdeb4043640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac0c5760 con 0x7fdeac06f2f0 2023-12-01T02:52:31.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.664+0000 7fdeb4043640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac15e6d0 con 0x7fdeac0cad70 2023-12-01T02:52:31.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.665+0000 7fdeb1ddf640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fdeac15e2f0 0x7fdeac0ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59208/0 (socket says 172.21.15.115:59208) 2023-12-01T02:52:31.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.665+0000 7fdeb1ddf640 1 -- 172.21.15.115:0/425069313 learned_addr learned my addr 172.21.15.115:0/425069313 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.665+0000 7fdeb15de640 1 --2- 172.21.15.115:0/425069313 >> v2:172.21.15.142:3300/0 conn(0x7fdeac0cad70 0x7fdeac06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.665+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645460509 0 0) 0x7fdeac0add10 con 0x7fdeac15e2f0 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.665+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde98003520 con 0x7fdeac15e2f0 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.665+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2825336314 0 0) 0x7fde98003520 con 0x7fdeac15e2f0 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 >> v2:172.21.15.142:3300/0 conn(0x7fdeac0cad70 msgr2=0x7fdeac06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 --2- 172.21.15.115:0/425069313 >> v2:172.21.15.142:3300/0 conn(0x7fdeac0cad70 0x7fdeac06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 >> v1:172.21.15.142:6789/0 conn(0x7fdeac06f2f0 legacy=0x7fdeac06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0cf7b0 con 0x7fdeac15e2f0 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fdea8001690 con 0x7fdeac15e2f0 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb15de640 1 --2- 172.21.15.115:0/425069313 >> v2:172.21.15.142:3300/0 conn(0x7fdeac0cad70 0x7fdeac06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 >> v1:172.21.15.115:6789/0 conn(0x7fdeac15e2f0 legacy=0x7fdeac0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:31.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fde98003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde980047c0 0x7fde98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde98005380 0x7fde9800b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac15e6d0 con 0x7fde98003a30 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9800c2c0 con 0x7fde980047c0 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9800c480 con 0x7fde98005380 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.666+0000 7fdeb1ddf640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fde98003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.667+0000 7fdeb25e0640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde98005380 0x7fde9800b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.667+0000 7fdeb15de640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde980047c0 0x7fde98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.667+0000 7fdeb1ddf640 1 -- 172.21.15.115:0/425069313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde98005380 msgr2=0x7fde9800b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.667+0000 7fdeb1ddf640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde98005380 0x7fde9800b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.667+0000 7fdeb1ddf640 1 -- 172.21.15.115:0/425069313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde980047c0 msgr2=0x7fde98004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.667+0000 7fdeb1ddf640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde980047c0 0x7fde98004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.667+0000 7fdeb1ddf640 1 -- 172.21.15.115:0/425069313 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98003520 con 0x7fde98003a30 2023-12-01T02:52:31.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.667+0000 7fdeb25e0640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde98005380 0x7fde9800b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:31.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.667+0000 7fdeb15de640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde980047c0 0x7fde98004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:31.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.667+0000 7fdeb25e0640 1 -- 172.21.15.115:0/425069313 reap_dead start 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.668+0000 7fdeb1ddf640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fde98003e50 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fdea8007fb0 tx=0x7fde980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30df67d472b6765b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.668+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8005ad0 con 0x7fde98003a30 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.668+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea8005c30 con 0x7fde98003a30 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.668+0000 7fdeb0ddd640 1 -- 172.21.15.115:0/425069313 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea802c790 con 0x7fde98003a30 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.669+0000 7fdeb4043640 1 -- 172.21.15.115:0/425069313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 msgr2=0x7fde98003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.669+0000 7fdeb4043640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fde98003e50 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fdea8007fb0 tx=0x7fde980057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.669+0000 7fdeb4043640 1 -- 172.21.15.115:0/425069313 shutdown_connections 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.669+0000 7fdeb4043640 1 --2- 172.21.15.115:0/425069313 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fde98003e50 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.669+0000 7fdeb4043640 1 -- 172.21.15.115:0/425069313 >> 172.21.15.115:0/425069313 conn(0x7fdeac0ce880 msgr2=0x7fdeac0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.669+0000 7fdeb4043640 1 -- 172.21.15.115:0/425069313 shutdown_connections 2023-12-01T02:52:31.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.669+0000 7fdeb4043640 1 -- 172.21.15.115:0/425069313 wait complete. 2023-12-01T02:52:31.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.670+0000 7fdeb4043640 1 Processor -- start 2023-12-01T02:52:31.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.671+0000 7fdeb4043640 1 -- start start 2023-12-01T02:52:31.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.671+0000 7fdeb4043640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fdeac158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:31.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.671+0000 7fdeb4043640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeac0ac8a0 0x7fdeac0acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:31.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb4043640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac0cc5a0 0x7fdeac0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:31.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb4043640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0ad260 con 0x7fde98003a30 2023-12-01T02:52:31.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb4043640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac1f3ae0 con 0x7fdeac0cc5a0 2023-12-01T02:52:31.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb4043640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0cd310 con 0x7fdeac0ac8a0 2023-12-01T02:52:31.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb1ddf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fdeac158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:31.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb1ddf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fdeac158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34610/0 (socket says 172.21.15.115:34610) 2023-12-01T02:52:31.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb1ddf640 1 -- 172.21.15.115:0/2013846740 learned_addr learned my addr 172.21.15.115:0/2013846740 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:31.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb25e0640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac0cc5a0 0x7fdeac0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:31.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb1ddf640 1 -- 172.21.15.115:0/2013846740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeac0ac8a0 msgr2=0x7fdeac0acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:31.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb1ddf640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeac0ac8a0 0x7fdeac0acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb1ddf640 1 -- 172.21.15.115:0/2013846740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac0cc5a0 msgr2=0x7fdeac0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:31.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb1ddf640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac0cc5a0 0x7fdeac0cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.672+0000 7fdeb1ddf640 1 -- 172.21.15.115:0/2013846740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0cd5f0 con 0x7fde98003a30 2023-12-01T02:52:31.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.673+0000 7fdeb1ddf640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fdeac158ea0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fdea8007fb0 tx=0x7fdea8001900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:31.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.673+0000 7fdea27fc640 1 -- 172.21.15.115:0/2013846740 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea803c040 con 0x7fde98003a30 2023-12-01T02:52:31.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.673+0000 7fdea27fc640 1 -- 172.21.15.115:0/2013846740 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea802cc20 con 0x7fde98003a30 2023-12-01T02:52:31.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.673+0000 7fdea27fc640 1 -- 172.21.15.115:0/2013846740 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8007070 con 0x7fde98003a30 2023-12-01T02:52:31.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.673+0000 7fdeb4043640 1 -- 172.21.15.115:0/2013846740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac0cd8d0 con 0x7fde98003a30 2023-12-01T02:52:31.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.673+0000 7fdeb4043640 1 -- 172.21.15.115:0/2013846740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac0ccf60 con 0x7fde98003a30 2023-12-01T02:52:31.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.674+0000 7fdeb4043640 1 -- 172.21.15.115:0/2013846740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac06fda0 con 0x7fde98003a30 2023-12-01T02:52:31.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.675+0000 7fdea27fc640 1 -- 172.21.15.115:0/2013846740 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fdea8032070 con 0x7fde98003a30 2023-12-01T02:52:31.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.676+0000 7fdea27fc640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde9006a580 0x7fde9006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:31.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.676+0000 7fdea27fc640 1 -- 172.21.15.115:0/2013846740 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fdea80a9c50 con 0x7fde98003a30 2023-12-01T02:52:31.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.676+0000 7fdeb15de640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde9006a580 0x7fde9006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:31.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.677+0000 7fdeb15de640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde9006a580 0x7fde9006ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fde9c006e80 tx=0x7fde9c006e10 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:31.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.680+0000 7fdea27fc640 1 -- 172.21.15.115:0/2013846740 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdea80750f0 con 0x7fde98003a30 2023-12-01T02:52:31.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.872+0000 7fdeb4043640 1 -- 172.21.15.115:0/2013846740 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdeac0b43d0 con 0x7fde9006a580 2023-12-01T02:52:31.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.883+0000 7fdea27fc640 1 -- 172.21.15.115:0/2013846740 <== mgr.9297 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287114 (secure 0 0 0) 0x7fde9c00d4c0 con 0x7fde9006a580 2023-12-01T02:52:31.888 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:31.890 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T02:52:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 -- 172.21.15.115:0/2013846740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde9006a580 msgr2=0x7fde9006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde9006a580 0x7fde9006ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fde9c006e80 tx=0x7fde9c006e10 comp rx=0 tx=0).stop 2023-12-01T02:52:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 -- 172.21.15.115:0/2013846740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 msgr2=0x7fdeac158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fdeac158ea0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fdea8007fb0 tx=0x7fdea8001900 comp rx=0 tx=0).stop 2023-12-01T02:52:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 -- 172.21.15.115:0/2013846740 shutdown_connections 2023-12-01T02:52:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fde9006a580 0x7fde9006ca40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeac0ac8a0 0x7fdeac0acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac0cc5a0 0x7fdeac0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 --2- 172.21.15.115:0/2013846740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde98003a30 0x7fdeac158ea0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 -- 172.21.15.115:0/2013846740 >> 172.21.15.115:0/2013846740 conn(0x7fdeac0ce880 msgr2=0x7fdeac0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 -- 172.21.15.115:0/2013846740 shutdown_connections 2023-12-01T02:52:31.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:31.888+0000 7fdeb4043640 1 -- 172.21.15.115:0/2013846740 wait complete. 2023-12-01T02:52:31.909 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-01T02:52:31.493027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402152,"kb_used_data":18472,"kb_used_omap":0,"kb_used_meta":383616,"kb_avail":754572568,"statfs":{"total":773094113280,"available":772682309632,"internally_reserved":0,"allocated":18915328,"data_stored":9710308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":392822784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.521955"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.996509+0000","last_change":"2023-12-01T02:52:21.994463+0000","last_active":"2023-12-01T02:52:21.996509+0000","last_peered":"2023-12-01T02:52:21.996509+0000","last_clean":"2023-12-01T02:52:21.996509+0000","last_became_active":"2023-12-01T02:52:21.993031+0000","last_became_peered":"2023-12-01T02:52:21.993031+0000","last_unstale":"2023-12-01T02:52:21.996509+0000","last_undegraded":"2023-12-01T02:52:21.996509+0000","last_fullsized":"2023-12-01T02:52:21.996509+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:56.336050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.013167+0000","last_change":"2023-12-01T02:52:22.011643+0000","last_active":"2023-12-01T02:52:22.013167+0000","last_peered":"2023-12-01T02:52:22.013167+0000","last_clean":"2023-12-01T02:52:22.013167+0000","last_became_active":"2023-12-01T02:52:22.011264+0000","last_became_peered":"2023-12-01T02:52:22.011264+0000","last_unstale":"2023-12-01T02:52:22.013167+0000","last_undegraded":"2023-12-01T02:52:22.013167+0000","last_fullsized":"2023-12-01T02:52:22.013167+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:10.320275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.981345+0000","last_change":"2023-12-01T02:52:21.999113+0000","last_active":"2023-12-01T02:52:22.981345+0000","last_peered":"2023-12-01T02:52:22.981345+0000","last_clean":"2023-12-01T02:52:22.981345+0000","last_became_active":"2023-12-01T02:52:21.998878+0000","last_became_peered":"2023-12-01T02:52:21.998878+0000","last_unstale":"2023-12-01T02:52:22.981345+0000","last_undegraded":"2023-12-01T02:52:22.981345+0000","last_fullsized":"2023-12-01T02:52:22.981345+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:52:21.333565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.003834+0000","last_change":"2023-12-01T02:52:21.995052+0000","last_active":"2023-12-01T02:52:22.003834+0000","last_peered":"2023-12-01T02:52:22.003834+0000","last_clean":"2023-12-01T02:52:22.003834+0000","last_became_active":"2023-12-01T02:52:21.994794+0000","last_became_peered":"2023-12-01T02:52:21.994794+0000","last_unstale":"2023-12-01T02:52:22.003834+0000","last_undegraded":"2023-12-01T02:52:22.003834+0000","last_fullsized":"2023-12-01T02:52:22.003834+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:31.036932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.893198+0000","last_change":"2023-12-01T02:52:21.991368+0000","last_active":"2023-12-01T02:52:23.893198+0000","last_peered":"2023-12-01T02:52:23.893198+0000","last_clean":"2023-12-01T02:52:23.893198+0000","last_became_active":"2023-12-01T02:52:21.990342+0000","last_became_peered":"2023-12-01T02:52:21.990342+0000","last_unstale":"2023-12-01T02:52:23.893198+0000","last_undegraded":"2023-12-01T02:52:23.893198+0000","last_fullsized":"2023-12-01T02:52:23.893198+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:58:38.245577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.147907+0000","last_change":"2023-12-01T02:52:21.991765+0000","last_active":"2023-12-01T02:52:23.147907+0000","last_peered":"2023-12-01T02:52:23.147907+0000","last_clean":"2023-12-01T02:52:23.147907+0000","last_became_active":"2023-12-01T02:52:21.991137+0000","last_became_peered":"2023-12-01T02:52:21.991137+0000","last_unstale":"2023-12-01T02:52:23.147907+0000","last_undegraded":"2023-12-01T02:52:23.147907+0000","last_fullsized":"2023-12-01T02:52:23.147907+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:30:41.584477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.950977+0000","last_change":"2023-12-01T02:52:22.004656+0000","last_active":"2023-12-01T02:52:23.950977+0000","last_peered":"2023-12-01T02:52:23.950977+0000","last_clean":"2023-12-01T02:52:23.950977+0000","last_became_active":"2023-12-01T02:52:22.004287+0000","last_became_peered":"2023-12-01T02:52:22.004287+0000","last_unstale":"2023-12-01T02:52:23.950977+0000","last_undegraded":"2023-12-01T02:52:23.950977+0000","last_fullsized":"2023-12-01T02:52:23.950977+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:50.965136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.014096+0000","last_change":"2023-12-01T02:52:22.009385+0000","last_active":"2023-12-01T02:52:22.014096+0000","last_peered":"2023-12-01T02:52:22.014096+0000","last_clean":"2023-12-01T02:52:22.014096+0000","last_became_active":"2023-12-01T02:52:22.009055+0000","last_became_peered":"2023-12-01T02:52:22.009055+0000","last_unstale":"2023-12-01T02:52:22.014096+0000","last_undegraded":"2023-12-01T02:52:22.014096+0000","last_fullsized":"2023-12-01T02:52:22.014096+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:58.071226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.148370+0000","last_change":"2023-12-01T02:52:22.000217+0000","last_active":"2023-12-01T02:52:23.148370+0000","last_peered":"2023-12-01T02:52:23.148370+0000","last_clean":"2023-12-01T02:52:23.148370+0000","last_became_active":"2023-12-01T02:52:21.999925+0000","last_became_peered":"2023-12-01T02:52:21.999925+0000","last_unstale":"2023-12-01T02:52:23.148370+0000","last_undegraded":"2023-12-01T02:52:23.148370+0000","last_fullsized":"2023-12-01T02:52:23.148370+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:09.328057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.981954+0000","last_change":"2023-12-01T02:52:21.990495+0000","last_active":"2023-12-01T02:52:22.981954+0000","last_peered":"2023-12-01T02:52:22.981954+0000","last_clean":"2023-12-01T02:52:22.981954+0000","last_became_active":"2023-12-01T02:52:21.987764+0000","last_became_peered":"2023-12-01T02:52:21.987764+0000","last_unstale":"2023-12-01T02:52:22.981954+0000","last_undegraded":"2023-12-01T02:52:22.981954+0000","last_fullsized":"2023-12-01T02:52:22.981954+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:51:13.200609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.148260+0000","last_change":"2023-12-01T02:52:21.995402+0000","last_active":"2023-12-01T02:52:23.148260+0000","last_peered":"2023-12-01T02:52:23.148260+0000","last_clean":"2023-12-01T02:52:23.148260+0000","last_became_active":"2023-12-01T02:52:21.995032+0000","last_became_peered":"2023-12-01T02:52:21.995032+0000","last_unstale":"2023-12-01T02:52:23.148260+0000","last_undegraded":"2023-12-01T02:52:23.148260+0000","last_fullsized":"2023-12-01T02:52:23.148260+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:20:16.758051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.952460+0000","last_change":"2023-12-01T02:52:21.987421+0000","last_active":"2023-12-01T02:52:23.952460+0000","last_peered":"2023-12-01T02:52:23.952460+0000","last_clean":"2023-12-01T02:52:23.952460+0000","last_became_active":"2023-12-01T02:52:21.986795+0000","last_became_peered":"2023-12-01T02:52:21.986795+0000","last_unstale":"2023-12-01T02:52:23.952460+0000","last_undegraded":"2023-12-01T02:52:23.952460+0000","last_fullsized":"2023-12-01T02:52:23.952460+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:38.746327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.893102+0000","last_change":"2023-12-01T02:52:21.989676+0000","last_active":"2023-12-01T02:52:23.893102+0000","last_peered":"2023-12-01T02:52:23.893102+0000","last_clean":"2023-12-01T02:52:23.893102+0000","last_became_active":"2023-12-01T02:52:21.989406+0000","last_became_peered":"2023-12-01T02:52:21.989406+0000","last_unstale":"2023-12-01T02:52:23.893102+0000","last_undegraded":"2023-12-01T02:52:23.893102+0000","last_fullsized":"2023-12-01T02:52:23.893102+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:03:17.794027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.952336+0000","last_change":"2023-12-01T02:52:21.997158+0000","last_active":"2023-12-01T02:52:23.952336+0000","last_peered":"2023-12-01T02:52:23.952336+0000","last_clean":"2023-12-01T02:52:23.952336+0000","last_became_active":"2023-12-01T02:52:21.996919+0000","last_became_peered":"2023-12-01T02:52:21.996919+0000","last_unstale":"2023-12-01T02:52:23.952336+0000","last_undegraded":"2023-12-01T02:52:23.952336+0000","last_fullsized":"2023-12-01T02:52:23.952336+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:48:36.083993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.892954+0000","last_change":"2023-12-01T02:52:21.997730+0000","last_active":"2023-12-01T02:52:23.892954+0000","last_peered":"2023-12-01T02:52:23.892954+0000","last_clean":"2023-12-01T02:52:23.892954+0000","last_became_active":"2023-12-01T02:52:21.997483+0000","last_became_peered":"2023-12-01T02:52:21.997483+0000","last_unstale":"2023-12-01T02:52:23.892954+0000","last_undegraded":"2023-12-01T02:52:23.892954+0000","last_fullsized":"2023-12-01T02:52:23.892954+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:01.218959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.950974+0000","last_change":"2023-12-01T02:52:22.000724+0000","last_active":"2023-12-01T02:52:23.950974+0000","last_peered":"2023-12-01T02:52:23.950974+0000","last_clean":"2023-12-01T02:52:23.950974+0000","last_became_active":"2023-12-01T02:52:21.999040+0000","last_became_peered":"2023-12-01T02:52:21.999040+0000","last_unstale":"2023-12-01T02:52:23.950974+0000","last_undegraded":"2023-12-01T02:52:23.950974+0000","last_fullsized":"2023-12-01T02:52:23.950974+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:47:21.653533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.981137+0000","last_change":"2023-12-01T02:52:21.991169+0000","last_active":"2023-12-01T02:52:22.981137+0000","last_peered":"2023-12-01T02:52:22.981137+0000","last_clean":"2023-12-01T02:52:22.981137+0000","last_became_active":"2023-12-01T02:52:21.990909+0000","last_became_peered":"2023-12-01T02:52:21.990909+0000","last_unstale":"2023-12-01T02:52:22.981137+0000","last_undegraded":"2023-12-01T02:52:22.981137+0000","last_fullsized":"2023-12-01T02:52:22.981137+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:14:01.220260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.998742+0000","last_change":"2023-12-01T02:52:21.995718+0000","last_active":"2023-12-01T02:52:21.998742+0000","last_peered":"2023-12-01T02:52:21.998742+0000","last_clean":"2023-12-01T02:52:21.998742+0000","last_became_active":"2023-12-01T02:52:21.995473+0000","last_became_peered":"2023-12-01T02:52:21.995473+0000","last_unstale":"2023-12-01T02:52:21.998742+0000","last_undegraded":"2023-12-01T02:52:21.998742+0000","last_fullsized":"2023-12-01T02:52:21.998742+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:40.701207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.000904+0000","last_change":"2023-12-01T02:52:21.995642+0000","last_active":"2023-12-01T02:52:22.000904+0000","last_peered":"2023-12-01T02:52:22.000904+0000","last_clean":"2023-12-01T02:52:22.000904+0000","last_became_active":"2023-12-01T02:52:21.995246+0000","last_became_peered":"2023-12-01T02:52:21.995246+0000","last_unstale":"2023-12-01T02:52:22.000904+0000","last_undegraded":"2023-12-01T02:52:22.000904+0000","last_fullsized":"2023-12-01T02:52:22.000904+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:47:21.653533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.009627+0000","last_change":"2023-12-01T02:52:22.004740+0000","last_active":"2023-12-01T02:52:22.009627+0000","last_peered":"2023-12-01T02:52:22.009627+0000","last_clean":"2023-12-01T02:52:22.009627+0000","last_became_active":"2023-12-01T02:52:22.003845+0000","last_became_peered":"2023-12-01T02:52:22.003845+0000","last_unstale":"2023-12-01T02:52:22.009627+0000","last_undegraded":"2023-12-01T02:52:22.009627+0000","last_fullsized":"2023-12-01T02:52:22.009627+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:47:40.922670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.981060+0000","last_change":"2023-12-01T02:52:21.991147+0000","last_active":"2023-12-01T02:52:22.981060+0000","last_peered":"2023-12-01T02:52:22.981060+0000","last_clean":"2023-12-01T02:52:22.981060+0000","last_became_active":"2023-12-01T02:52:21.990919+0000","last_became_peered":"2023-12-01T02:52:21.990919+0000","last_unstale":"2023-12-01T02:52:22.981060+0000","last_undegraded":"2023-12-01T02:52:22.981060+0000","last_fullsized":"2023-12-01T02:52:22.981060+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:23:17.913730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.952445+0000","last_change":"2023-12-01T02:52:21.996650+0000","last_active":"2023-12-01T02:52:23.952445+0000","last_peered":"2023-12-01T02:52:23.952445+0000","last_clean":"2023-12-01T02:52:23.952445+0000","last_became_active":"2023-12-01T02:52:21.996251+0000","last_became_peered":"2023-12-01T02:52:21.996251+0000","last_unstale":"2023-12-01T02:52:23.952445+0000","last_undegraded":"2023-12-01T02:52:23.952445+0000","last_fullsized":"2023-12-01T02:52:23.952445+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:31.036932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.996982+0000","last_change":"2023-12-01T02:52:21.992890+0000","last_active":"2023-12-01T02:52:21.996982+0000","last_peered":"2023-12-01T02:52:21.996982+0000","last_clean":"2023-12-01T02:52:21.996982+0000","last_became_active":"2023-12-01T02:52:21.992631+0000","last_became_peered":"2023-12-01T02:52:21.992631+0000","last_unstale":"2023-12-01T02:52:21.996982+0000","last_undegraded":"2023-12-01T02:52:21.996982+0000","last_fullsized":"2023-12-01T02:52:21.996982+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:31.174876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.986074+0000","last_change":"2023-12-01T02:52:21.985298+0000","last_active":"2023-12-01T02:52:21.986074+0000","last_peered":"2023-12-01T02:52:21.986074+0000","last_clean":"2023-12-01T02:52:21.986074+0000","last_became_active":"2023-12-01T02:52:21.985037+0000","last_became_peered":"2023-12-01T02:52:21.985037+0000","last_unstale":"2023-12-01T02:52:21.986074+0000","last_undegraded":"2023-12-01T02:52:21.986074+0000","last_fullsized":"2023-12-01T02:52:21.986074+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:02:22.859562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.012135+0000","last_change":"2023-12-01T02:52:22.009228+0000","last_active":"2023-12-01T02:52:22.012135+0000","last_peered":"2023-12-01T02:52:22.012135+0000","last_clean":"2023-12-01T02:52:22.012135+0000","last_became_active":"2023-12-01T02:52:22.008603+0000","last_became_peered":"2023-12-01T02:52:22.008603+0000","last_unstale":"2023-12-01T02:52:22.012135+0000","last_undegraded":"2023-12-01T02:52:22.012135+0000","last_fullsized":"2023-12-01T02:52:22.012135+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:52.569081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010574+0000","last_change":"2023-12-01T02:52:22.006541+0000","last_active":"2023-12-01T02:52:22.010574+0000","last_peered":"2023-12-01T02:52:22.010574+0000","last_clean":"2023-12-01T02:52:22.010574+0000","last_became_active":"2023-12-01T02:52:22.003660+0000","last_became_peered":"2023-12-01T02:52:22.003660+0000","last_unstale":"2023-12-01T02:52:22.010574+0000","last_undegraded":"2023-12-01T02:52:22.010574+0000","last_fullsized":"2023-12-01T02:52:22.010574+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:16.098248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.013788+0000","last_change":"2023-12-01T02:52:22.012721+0000","last_active":"2023-12-01T02:52:22.013788+0000","last_peered":"2023-12-01T02:52:22.013788+0000","last_clean":"2023-12-01T02:52:22.013788+0000","last_became_active":"2023-12-01T02:52:22.009654+0000","last_became_peered":"2023-12-01T02:52:22.009654+0000","last_unstale":"2023-12-01T02:52:22.013788+0000","last_undegraded":"2023-12-01T02:52:22.013788+0000","last_fullsized":"2023-12-01T02:52:22.013788+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:40:18.077875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.013533+0000","last_change":"2023-12-01T02:52:22.012258+0000","last_active":"2023-12-01T02:52:22.013533+0000","last_peered":"2023-12-01T02:52:22.013533+0000","last_clean":"2023-12-01T02:52:22.013533+0000","last_became_active":"2023-12-01T02:52:22.011788+0000","last_became_peered":"2023-12-01T02:52:22.011788+0000","last_unstale":"2023-12-01T02:52:22.013533+0000","last_undegraded":"2023-12-01T02:52:22.013533+0000","last_fullsized":"2023-12-01T02:52:22.013533+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:26:45.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010091+0000","last_change":"2023-12-01T02:52:22.005635+0000","last_active":"2023-12-01T02:52:22.010091+0000","last_peered":"2023-12-01T02:52:22.010091+0000","last_clean":"2023-12-01T02:52:22.010091+0000","last_became_active":"2023-12-01T02:52:22.005358+0000","last_became_peered":"2023-12-01T02:52:22.005358+0000","last_unstale":"2023-12-01T02:52:22.010091+0000","last_undegraded":"2023-12-01T02:52:22.010091+0000","last_fullsized":"2023-12-01T02:52:22.010091+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:24.096134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.952500+0000","last_change":"2023-12-01T02:52:21.996489+0000","last_active":"2023-12-01T02:52:23.952500+0000","last_peered":"2023-12-01T02:52:23.952500+0000","last_clean":"2023-12-01T02:52:23.952500+0000","last_became_active":"2023-12-01T02:52:21.993148+0000","last_became_peered":"2023-12-01T02:52:21.993148+0000","last_unstale":"2023-12-01T02:52:23.952500+0000","last_undegraded":"2023-12-01T02:52:23.952500+0000","last_fullsized":"2023-12-01T02:52:23.952500+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:24:05.477889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.008522+0000","last_change":"2023-12-01T02:52:22.001823+0000","last_active":"2023-12-01T02:52:22.008522+0000","last_peered":"2023-12-01T02:52:22.008522+0000","last_clean":"2023-12-01T02:52:22.008522+0000","last_became_active":"2023-12-01T02:52:22.000229+0000","last_became_peered":"2023-12-01T02:52:22.000229+0000","last_unstale":"2023-12-01T02:52:22.008522+0000","last_undegraded":"2023-12-01T02:52:22.008522+0000","last_fullsized":"2023-12-01T02:52:22.008522+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:09.343506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.012676+0000","last_change":"2023-12-01T02:52:22.010609+0000","last_active":"2023-12-01T02:52:22.012676+0000","last_peered":"2023-12-01T02:52:22.012676+0000","last_clean":"2023-12-01T02:52:22.012676+0000","last_became_active":"2023-12-01T02:52:22.008695+0000","last_became_peered":"2023-12-01T02:52:22.008695+0000","last_unstale":"2023-12-01T02:52:22.012676+0000","last_undegraded":"2023-12-01T02:52:22.012676+0000","last_fullsized":"2023-12-01T02:52:22.012676+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:52:02.202035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.991893+0000","last_change":"2023-12-01T02:52:21.987390+0000","last_active":"2023-12-01T02:52:21.991893+0000","last_peered":"2023-12-01T02:52:21.991893+0000","last_clean":"2023-12-01T02:52:21.991893+0000","last_became_active":"2023-12-01T02:52:21.987129+0000","last_became_peered":"2023-12-01T02:52:21.987129+0000","last_unstale":"2023-12-01T02:52:21.991893+0000","last_undegraded":"2023-12-01T02:52:21.991893+0000","last_fullsized":"2023-12-01T02:52:21.991893+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:19:07.126182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"176'4","reported_seq":15,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010863+0000","last_change":"2023-12-01T02:52:21.516667+0000","last_active":"2023-12-01T02:52:22.010863+0000","last_peered":"2023-12-01T02:52:22.010863+0000","last_clean":"2023-12-01T02:52:22.010863+0000","last_became_active":"2023-12-01T02:52:21.516412+0000","last_became_peered":"2023-12-01T02:52:21.516412+0000","last_unstale":"2023-12-01T02:52:22.010863+0000","last_undegraded":"2023-12-01T02:52:22.010863+0000","last_fullsized":"2023-12-01T02:52:22.010863+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:51.736257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.951854+0000","last_change":"2023-12-01T02:52:21.996199+0000","last_active":"2023-12-01T02:52:23.951854+0000","last_peered":"2023-12-01T02:52:23.951854+0000","last_clean":"2023-12-01T02:52:23.951854+0000","last_became_active":"2023-12-01T02:52:21.995890+0000","last_became_peered":"2023-12-01T02:52:21.995890+0000","last_unstale":"2023-12-01T02:52:23.951854+0000","last_undegraded":"2023-12-01T02:52:23.951854+0000","last_fullsized":"2023-12-01T02:52:23.951854+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:01.554053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"176'1","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.007387+0000","last_change":"2023-12-01T02:52:21.521815+0000","last_active":"2023-12-01T02:52:22.007387+0000","last_peered":"2023-12-01T02:52:22.007387+0000","last_clean":"2023-12-01T02:52:22.007387+0000","last_became_active":"2023-12-01T02:52:21.521432+0000","last_became_peered":"2023-12-01T02:52:21.521432+0000","last_unstale":"2023-12-01T02:52:22.007387+0000","last_undegraded":"2023-12-01T02:52:22.007387+0000","last_fullsized":"2023-12-01T02:52:22.007387+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.148114+0000","last_change":"2023-12-01T02:52:21.994370+0000","last_active":"2023-12-01T02:52:23.148114+0000","last_peered":"2023-12-01T02:52:23.148114+0000","last_clean":"2023-12-01T02:52:23.148114+0000","last_became_active":"2023-12-01T02:52:21.993797+0000","last_became_peered":"2023-12-01T02:52:21.993797+0000","last_unstale":"2023-12-01T02:52:23.148114+0000","last_undegraded":"2023-12-01T02:52:23.148114+0000","last_fullsized":"2023-12-01T02:52:23.148114+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:16:32.491266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010903+0000","last_change":"2023-12-01T02:52:21.518182+0000","last_active":"2023-12-01T02:52:22.010903+0000","last_peered":"2023-12-01T02:52:22.010903+0000","last_clean":"2023-12-01T02:52:22.010903+0000","last_became_active":"2023-12-01T02:52:21.517878+0000","last_became_peered":"2023-12-01T02:52:21.517878+0000","last_unstale":"2023-12-01T02:52:22.010903+0000","last_undegraded":"2023-12-01T02:52:22.010903+0000","last_fullsized":"2023-12-01T02:52:22.010903+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:18.775753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.998570+0000","last_change":"2023-12-01T02:52:21.991684+0000","last_active":"2023-12-01T02:52:21.998570+0000","last_peered":"2023-12-01T02:52:21.998570+0000","last_clean":"2023-12-01T02:52:21.998570+0000","last_became_active":"2023-12-01T02:52:21.991423+0000","last_became_peered":"2023-12-01T02:52:21.991423+0000","last_unstale":"2023-12-01T02:52:21.998570+0000","last_undegraded":"2023-12-01T02:52:21.998570+0000","last_fullsized":"2023-12-01T02:52:21.998570+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:48:36.083993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.011036+0000","last_change":"2023-12-01T02:52:21.521668+0000","last_active":"2023-12-01T02:52:22.011036+0000","last_peered":"2023-12-01T02:52:22.011036+0000","last_clean":"2023-12-01T02:52:22.011036+0000","last_became_active":"2023-12-01T02:52:21.521385+0000","last_became_peered":"2023-12-01T02:52:21.521385+0000","last_unstale":"2023-12-01T02:52:22.011036+0000","last_undegraded":"2023-12-01T02:52:22.011036+0000","last_fullsized":"2023-12-01T02:52:22.011036+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:49.949971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.012254+0000","last_change":"2023-12-01T02:52:22.008213+0000","last_active":"2023-12-01T02:52:22.012254+0000","last_peered":"2023-12-01T02:52:22.012254+0000","last_clean":"2023-12-01T02:52:22.012254+0000","last_became_active":"2023-12-01T02:52:22.006615+0000","last_became_peered":"2023-12-01T02:52:22.006615+0000","last_unstale":"2023-12-01T02:52:22.012254+0000","last_undegraded":"2023-12-01T02:52:22.012254+0000","last_fullsized":"2023-12-01T02:52:22.012254+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:41.511561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"176'1","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010136+0000","last_change":"2023-12-01T02:52:21.519085+0000","last_active":"2023-12-01T02:52:22.010136+0000","last_peered":"2023-12-01T02:52:22.010136+0000","last_clean":"2023-12-01T02:52:22.010136+0000","last_became_active":"2023-12-01T02:52:21.518796+0000","last_became_peered":"2023-12-01T02:52:21.518796+0000","last_unstale":"2023-12-01T02:52:22.010136+0000","last_undegraded":"2023-12-01T02:52:22.010136+0000","last_fullsized":"2023-12-01T02:52:22.010136+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:40.990842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.998537+0000","last_change":"2023-12-01T02:52:21.989289+0000","last_active":"2023-12-01T02:52:21.998537+0000","last_peered":"2023-12-01T02:52:21.998537+0000","last_clean":"2023-12-01T02:52:21.998537+0000","last_became_active":"2023-12-01T02:52:21.989010+0000","last_became_peered":"2023-12-01T02:52:21.989010+0000","last_unstale":"2023-12-01T02:52:21.998537+0000","last_undegraded":"2023-12-01T02:52:21.998537+0000","last_fullsized":"2023-12-01T02:52:21.998537+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:42.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"176'1","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.002482+0000","last_change":"2023-12-01T02:52:21.522775+0000","last_active":"2023-12-01T02:52:22.002482+0000","last_peered":"2023-12-01T02:52:22.002482+0000","last_clean":"2023-12-01T02:52:22.002482+0000","last_became_active":"2023-12-01T02:52:21.522180+0000","last_became_peered":"2023-12-01T02:52:21.522180+0000","last_unstale":"2023-12-01T02:52:22.002482+0000","last_undegraded":"2023-12-01T02:52:22.002482+0000","last_fullsized":"2023-12-01T02:52:22.002482+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:34.788423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.013572+0000","last_change":"2023-12-01T02:52:22.011979+0000","last_active":"2023-12-01T02:52:22.013572+0000","last_peered":"2023-12-01T02:52:22.013572+0000","last_clean":"2023-12-01T02:52:22.013572+0000","last_became_active":"2023-12-01T02:52:22.011003+0000","last_became_peered":"2023-12-01T02:52:22.011003+0000","last_unstale":"2023-12-01T02:52:22.013572+0000","last_undegraded":"2023-12-01T02:52:22.013572+0000","last_fullsized":"2023-12-01T02:52:22.013572+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:04.052509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.983010+0000","last_change":"2023-12-01T02:52:21.516907+0000","last_active":"2023-12-01T02:52:21.983010+0000","last_peered":"2023-12-01T02:52:21.983010+0000","last_clean":"2023-12-01T02:52:21.983010+0000","last_became_active":"2023-12-01T02:52:21.516334+0000","last_became_peered":"2023-12-01T02:52:21.516334+0000","last_unstale":"2023-12-01T02:52:21.983010+0000","last_undegraded":"2023-12-01T02:52:21.983010+0000","last_fullsized":"2023-12-01T02:52:21.983010+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:22.856154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.002820+0000","last_change":"2023-12-01T02:52:21.993975+0000","last_active":"2023-12-01T02:52:22.002820+0000","last_peered":"2023-12-01T02:52:22.002820+0000","last_clean":"2023-12-01T02:52:22.002820+0000","last_became_active":"2023-12-01T02:52:21.993703+0000","last_became_peered":"2023-12-01T02:52:21.993703+0000","last_unstale":"2023-12-01T02:52:22.002820+0000","last_undegraded":"2023-12-01T02:52:22.002820+0000","last_fullsized":"2023-12-01T02:52:22.002820+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.976159+0000","last_change":"2023-12-01T02:52:21.517632+0000","last_active":"2023-12-01T02:52:21.976159+0000","last_peered":"2023-12-01T02:52:21.976159+0000","last_clean":"2023-12-01T02:52:21.976159+0000","last_became_active":"2023-12-01T02:52:21.517198+0000","last_became_peered":"2023-12-01T02:52:21.517198+0000","last_unstale":"2023-12-01T02:52:21.976159+0000","last_undegraded":"2023-12-01T02:52:21.976159+0000","last_fullsized":"2023-12-01T02:52:21.976159+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:48:39.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.995069+0000","last_change":"2023-12-01T02:52:21.991298+0000","last_active":"2023-12-01T02:52:21.995069+0000","last_peered":"2023-12-01T02:52:21.995069+0000","last_clean":"2023-12-01T02:52:21.995069+0000","last_became_active":"2023-12-01T02:52:21.991036+0000","last_became_peered":"2023-12-01T02:52:21.991036+0000","last_unstale":"2023-12-01T02:52:21.995069+0000","last_undegraded":"2023-12-01T02:52:21.995069+0000","last_fullsized":"2023-12-01T02:52:21.995069+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:09.328057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.988861+0000","last_change":"2023-12-01T02:52:21.516838+0000","last_active":"2023-12-01T02:52:21.988861+0000","last_peered":"2023-12-01T02:52:21.988861+0000","last_clean":"2023-12-01T02:52:21.988861+0000","last_became_active":"2023-12-01T02:52:21.516232+0000","last_became_peered":"2023-12-01T02:52:21.516232+0000","last_unstale":"2023-12-01T02:52:21.988861+0000","last_undegraded":"2023-12-01T02:52:21.988861+0000","last_fullsized":"2023-12-01T02:52:21.988861+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:30:08.328341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.952362+0000","last_change":"2023-12-01T02:52:21.998409+0000","last_active":"2023-12-01T02:52:23.952362+0000","last_peered":"2023-12-01T02:52:23.952362+0000","last_clean":"2023-12-01T02:52:23.952362+0000","last_became_active":"2023-12-01T02:52:21.998162+0000","last_became_peered":"2023-12-01T02:52:21.998162+0000","last_unstale":"2023-12-01T02:52:23.952362+0000","last_undegraded":"2023-12-01T02:52:23.952362+0000","last_fullsized":"2023-12-01T02:52:23.952362+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:54.304823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.985052+0000","last_change":"2023-12-01T02:52:21.518112+0000","last_active":"2023-12-01T02:52:21.985052+0000","last_peered":"2023-12-01T02:52:21.985052+0000","last_clean":"2023-12-01T02:52:21.985052+0000","last_became_active":"2023-12-01T02:52:21.517851+0000","last_became_peered":"2023-12-01T02:52:21.517851+0000","last_unstale":"2023-12-01T02:52:21.985052+0000","last_undegraded":"2023-12-01T02:52:21.985052+0000","last_fullsized":"2023-12-01T02:52:21.985052+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:20:46.753790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.951483+0000","last_change":"2023-12-01T02:52:22.002998+0000","last_active":"2023-12-01T02:52:23.951483+0000","last_peered":"2023-12-01T02:52:23.951483+0000","last_clean":"2023-12-01T02:52:23.951483+0000","last_became_active":"2023-12-01T02:52:22.002649+0000","last_became_peered":"2023-12-01T02:52:22.002649+0000","last_unstale":"2023-12-01T02:52:23.951483+0000","last_undegraded":"2023-12-01T02:52:23.951483+0000","last_fullsized":"2023-12-01T02:52:23.951483+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:40.623736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"176'6","reported_seq":17,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.015126+0000","last_change":"2023-12-01T02:52:21.514385+0000","last_active":"2023-12-01T02:52:22.015126+0000","last_peered":"2023-12-01T02:52:22.015126+0000","last_clean":"2023-12-01T02:52:22.015126+0000","last_became_active":"2023-12-01T02:52:21.513977+0000","last_became_peered":"2023-12-01T02:52:21.513977+0000","last_unstale":"2023-12-01T02:52:22.015126+0000","last_undegraded":"2023-12-01T02:52:22.015126+0000","last_fullsized":"2023-12-01T02:52:22.015126+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:57.056061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.988752+0000","last_change":"2023-12-01T02:52:21.987166+0000","last_active":"2023-12-01T02:52:21.988752+0000","last_peered":"2023-12-01T02:52:21.988752+0000","last_clean":"2023-12-01T02:52:21.988752+0000","last_became_active":"2023-12-01T02:52:21.986924+0000","last_became_peered":"2023-12-01T02:52:21.986924+0000","last_unstale":"2023-12-01T02:52:21.988752+0000","last_undegraded":"2023-12-01T02:52:21.988752+0000","last_fullsized":"2023-12-01T02:52:21.988752+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:20:16.758051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.011353+0000","last_change":"2023-12-01T02:52:21.518964+0000","last_active":"2023-12-01T02:52:22.011353+0000","last_peered":"2023-12-01T02:52:22.011353+0000","last_clean":"2023-12-01T02:52:22.011353+0000","last_became_active":"2023-12-01T02:52:21.518401+0000","last_became_peered":"2023-12-01T02:52:21.518401+0000","last_unstale":"2023-12-01T02:52:22.011353+0000","last_undegraded":"2023-12-01T02:52:22.011353+0000","last_fullsized":"2023-12-01T02:52:22.011353+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:16.375606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010113+0000","last_change":"2023-12-01T02:52:22.006473+0000","last_active":"2023-12-01T02:52:22.010113+0000","last_peered":"2023-12-01T02:52:22.010113+0000","last_clean":"2023-12-01T02:52:22.010113+0000","last_became_active":"2023-12-01T02:52:22.004904+0000","last_became_peered":"2023-12-01T02:52:22.004904+0000","last_unstale":"2023-12-01T02:52:22.010113+0000","last_undegraded":"2023-12-01T02:52:22.010113+0000","last_fullsized":"2023-12-01T02:52:22.010113+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:34.006926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.980070+0000","last_change":"2023-12-01T02:52:21.517770+0000","last_active":"2023-12-01T02:52:21.980070+0000","last_peered":"2023-12-01T02:52:21.980070+0000","last_clean":"2023-12-01T02:52:21.980070+0000","last_became_active":"2023-12-01T02:52:21.517465+0000","last_became_peered":"2023-12-01T02:52:21.517465+0000","last_unstale":"2023-12-01T02:52:21.980070+0000","last_undegraded":"2023-12-01T02:52:21.980070+0000","last_fullsized":"2023-12-01T02:52:21.980070+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:21:37.282563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.004069+0000","last_change":"2023-12-01T02:52:21.997830+0000","last_active":"2023-12-01T02:52:22.004069+0000","last_peered":"2023-12-01T02:52:22.004069+0000","last_clean":"2023-12-01T02:52:22.004069+0000","last_became_active":"2023-12-01T02:52:21.997562+0000","last_became_peered":"2023-12-01T02:52:21.997562+0000","last_unstale":"2023-12-01T02:52:22.004069+0000","last_undegraded":"2023-12-01T02:52:22.004069+0000","last_fullsized":"2023-12-01T02:52:22.004069+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:13.272131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.996870+0000","last_change":"2023-12-01T02:52:21.517568+0000","last_active":"2023-12-01T02:52:21.996870+0000","last_peered":"2023-12-01T02:52:21.996870+0000","last_clean":"2023-12-01T02:52:21.996870+0000","last_became_active":"2023-12-01T02:52:21.517259+0000","last_became_peered":"2023-12-01T02:52:21.517259+0000","last_unstale":"2023-12-01T02:52:21.996870+0000","last_undegraded":"2023-12-01T02:52:21.996870+0000","last_fullsized":"2023-12-01T02:52:21.996870+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:11:02.383031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.997002+0000","last_change":"2023-12-01T02:52:21.992889+0000","last_active":"2023-12-01T02:52:21.997002+0000","last_peered":"2023-12-01T02:52:21.997002+0000","last_clean":"2023-12-01T02:52:21.997002+0000","last_became_active":"2023-12-01T02:52:21.992598+0000","last_became_peered":"2023-12-01T02:52:21.992598+0000","last_unstale":"2023-12-01T02:52:21.997002+0000","last_undegraded":"2023-12-01T02:52:21.997002+0000","last_fullsized":"2023-12-01T02:52:21.997002+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:16:32.491266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"176'8","reported_seq":19,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.015653+0000","last_change":"2023-12-01T02:52:21.515675+0000","last_active":"2023-12-01T02:52:22.015653+0000","last_peered":"2023-12-01T02:52:22.015653+0000","last_clean":"2023-12-01T02:52:22.015653+0000","last_became_active":"2023-12-01T02:52:21.515354+0000","last_became_peered":"2023-12-01T02:52:21.515354+0000","last_unstale":"2023-12-01T02:52:22.015653+0000","last_undegraded":"2023-12-01T02:52:22.015653+0000","last_fullsized":"2023-12-01T02:52:22.015653+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:24:04.462724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.998760+0000","last_change":"2023-12-01T02:52:21.996419+0000","last_active":"2023-12-01T02:52:21.998760+0000","last_peered":"2023-12-01T02:52:21.998760+0000","last_clean":"2023-12-01T02:52:21.998760+0000","last_became_active":"2023-12-01T02:52:21.992392+0000","last_became_peered":"2023-12-01T02:52:21.992392+0000","last_unstale":"2023-12-01T02:52:21.998760+0000","last_undegraded":"2023-12-01T02:52:21.998760+0000","last_fullsized":"2023-12-01T02:52:21.998760+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:46:29.228752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"176'1","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.011955+0000","last_change":"2023-12-01T02:52:21.515254+0000","last_active":"2023-12-01T02:52:22.011955+0000","last_peered":"2023-12-01T02:52:22.011955+0000","last_clean":"2023-12-01T02:52:22.011955+0000","last_became_active":"2023-12-01T02:52:21.514932+0000","last_became_peered":"2023-12-01T02:52:21.514932+0000","last_unstale":"2023-12-01T02:52:22.011955+0000","last_undegraded":"2023-12-01T02:52:22.011955+0000","last_fullsized":"2023-12-01T02:52:22.011955+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:56.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.010976+0000","last_change":"2023-12-01T02:52:22.004977+0000","last_active":"2023-12-01T02:52:22.010976+0000","last_peered":"2023-12-01T02:52:22.010976+0000","last_clean":"2023-12-01T02:52:22.010976+0000","last_became_active":"2023-12-01T02:52:22.004594+0000","last_became_peered":"2023-12-01T02:52:22.004594+0000","last_unstale":"2023-12-01T02:52:22.010976+0000","last_undegraded":"2023-12-01T02:52:22.010976+0000","last_fullsized":"2023-12-01T02:52:22.010976+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:32.342375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.014084+0000","last_change":"2023-12-01T02:52:21.516963+0000","last_active":"2023-12-01T02:52:22.014084+0000","last_peered":"2023-12-01T02:52:22.014084+0000","last_clean":"2023-12-01T02:52:22.014084+0000","last_became_active":"2023-12-01T02:52:21.516690+0000","last_became_peered":"2023-12-01T02:52:21.516690+0000","last_unstale":"2023-12-01T02:52:22.014084+0000","last_undegraded":"2023-12-01T02:52:22.014084+0000","last_fullsized":"2023-12-01T02:52:22.014084+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:30:37.731163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.001179+0000","last_change":"2023-12-01T02:52:21.999699+0000","last_active":"2023-12-01T02:52:22.001179+0000","last_peered":"2023-12-01T02:52:22.001179+0000","last_clean":"2023-12-01T02:52:22.001179+0000","last_became_active":"2023-12-01T02:52:21.999434+0000","last_became_peered":"2023-12-01T02:52:21.999434+0000","last_unstale":"2023-12-01T02:52:22.001179+0000","last_undegraded":"2023-12-01T02:52:22.001179+0000","last_fullsized":"2023-12-01T02:52:22.001179+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:24:11.838238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"176'1","reported_seq":12,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.007470+0000","last_change":"2023-12-01T02:52:21.518990+0000","last_active":"2023-12-01T02:52:22.007470+0000","last_peered":"2023-12-01T02:52:22.007470+0000","last_clean":"2023-12-01T02:52:22.007470+0000","last_became_active":"2023-12-01T02:52:21.518653+0000","last_became_peered":"2023-12-01T02:52:21.518653+0000","last_unstale":"2023-12-01T02:52:22.007470+0000","last_undegraded":"2023-12-01T02:52:22.007470+0000","last_fullsized":"2023-12-01T02:52:22.007470+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:18:23.080969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.148377+0000","last_change":"2023-12-01T02:52:21.991178+0000","last_active":"2023-12-01T02:52:23.148377+0000","last_peered":"2023-12-01T02:52:23.148377+0000","last_clean":"2023-12-01T02:52:23.148377+0000","last_became_active":"2023-12-01T02:52:21.989781+0000","last_became_peered":"2023-12-01T02:52:21.989781+0000","last_unstale":"2023-12-01T02:52:23.148377+0000","last_undegraded":"2023-12-01T02:52:23.148377+0000","last_fullsized":"2023-12-01T02:52:23.148377+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:13.272131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.999491+0000","last_change":"2023-12-01T02:52:21.509641+0000","last_active":"2023-12-01T02:52:21.999491+0000","last_peered":"2023-12-01T02:52:21.999491+0000","last_clean":"2023-12-01T02:52:21.999491+0000","last_became_active":"2023-12-01T02:52:21.508511+0000","last_became_peered":"2023-12-01T02:52:21.508511+0000","last_unstale":"2023-12-01T02:52:21.999491+0000","last_undegraded":"2023-12-01T02:52:21.999491+0000","last_fullsized":"2023-12-01T02:52:21.999491+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:44:46.015030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.007550+0000","last_change":"2023-12-01T02:52:22.001107+0000","last_active":"2023-12-01T02:52:22.007550+0000","last_peered":"2023-12-01T02:52:22.007550+0000","last_clean":"2023-12-01T02:52:22.007550+0000","last_became_active":"2023-12-01T02:52:21.999794+0000","last_became_peered":"2023-12-01T02:52:21.999794+0000","last_unstale":"2023-12-01T02:52:22.007550+0000","last_undegraded":"2023-12-01T02:52:22.007550+0000","last_fullsized":"2023-12-01T02:52:22.007550+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.008400+0000","last_change":"2023-12-01T02:52:21.521786+0000","last_active":"2023-12-01T02:52:22.008400+0000","last_peered":"2023-12-01T02:52:22.008400+0000","last_clean":"2023-12-01T02:52:22.008400+0000","last_became_active":"2023-12-01T02:52:21.521463+0000","last_became_peered":"2023-12-01T02:52:21.521463+0000","last_unstale":"2023-12-01T02:52:22.008400+0000","last_undegraded":"2023-12-01T02:52:22.008400+0000","last_fullsized":"2023-12-01T02:52:22.008400+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:36.292725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.011636+0000","last_change":"2023-12-01T02:52:22.005777+0000","last_active":"2023-12-01T02:52:22.011636+0000","last_peered":"2023-12-01T02:52:22.011636+0000","last_clean":"2023-12-01T02:52:22.011636+0000","last_became_active":"2023-12-01T02:52:22.003932+0000","last_became_peered":"2023-12-01T02:52:22.003932+0000","last_unstale":"2023-12-01T02:52:22.011636+0000","last_undegraded":"2023-12-01T02:52:22.011636+0000","last_fullsized":"2023-12-01T02:52:22.011636+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.977144+0000","last_change":"2023-12-01T02:52:21.516256+0000","last_active":"2023-12-01T02:52:21.977144+0000","last_peered":"2023-12-01T02:52:21.977144+0000","last_clean":"2023-12-01T02:52:21.977144+0000","last_became_active":"2023-12-01T02:52:21.515650+0000","last_became_peered":"2023-12-01T02:52:21.515650+0000","last_unstale":"2023-12-01T02:52:21.977144+0000","last_undegraded":"2023-12-01T02:52:21.977144+0000","last_fullsized":"2023-12-01T02:52:21.977144+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:30.297120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.007643+0000","last_change":"2023-12-01T02:52:22.000613+0000","last_active":"2023-12-01T02:52:22.007643+0000","last_peered":"2023-12-01T02:52:22.007643+0000","last_clean":"2023-12-01T02:52:22.007643+0000","last_became_active":"2023-12-01T02:52:22.000378+0000","last_became_peered":"2023-12-01T02:52:22.000378+0000","last_unstale":"2023-12-01T02:52:22.007643+0000","last_undegraded":"2023-12-01T02:52:22.007643+0000","last_fullsized":"2023-12-01T02:52:22.007643+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:42.006007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.975540+0000","last_change":"2023-12-01T02:52:21.513604+0000","last_active":"2023-12-01T02:52:21.975540+0000","last_peered":"2023-12-01T02:52:21.975540+0000","last_clean":"2023-12-01T02:52:21.975540+0000","last_became_active":"2023-12-01T02:52:21.512902+0000","last_became_peered":"2023-12-01T02:52:21.512902+0000","last_unstale":"2023-12-01T02:52:21.975540+0000","last_undegraded":"2023-12-01T02:52:21.975540+0000","last_fullsized":"2023-12-01T02:52:21.975540+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:52.712427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.008544+0000","last_change":"2023-12-01T02:52:22.003588+0000","last_active":"2023-12-01T02:52:22.008544+0000","last_peered":"2023-12-01T02:52:22.008544+0000","last_clean":"2023-12-01T02:52:22.008544+0000","last_became_active":"2023-12-01T02:52:22.002988+0000","last_became_peered":"2023-12-01T02:52:22.002988+0000","last_unstale":"2023-12-01T02:52:22.008544+0000","last_undegraded":"2023-12-01T02:52:22.008544+0000","last_fullsized":"2023-12-01T02:52:22.008544+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:37.307890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.974887+0000","last_change":"2023-12-01T02:52:21.519031+0000","last_active":"2023-12-01T02:52:21.974887+0000","last_peered":"2023-12-01T02:52:21.974887+0000","last_clean":"2023-12-01T02:52:21.974887+0000","last_became_active":"2023-12-01T02:52:21.518624+0000","last_became_peered":"2023-12-01T02:52:21.518624+0000","last_unstale":"2023-12-01T02:52:21.974887+0000","last_undegraded":"2023-12-01T02:52:21.974887+0000","last_fullsized":"2023-12-01T02:52:21.974887+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:04:47.352377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.001915+0000","last_change":"2023-12-01T02:52:21.994986+0000","last_active":"2023-12-01T02:52:22.001915+0000","last_peered":"2023-12-01T02:52:22.001915+0000","last_clean":"2023-12-01T02:52:22.001915+0000","last_became_active":"2023-12-01T02:52:21.994737+0000","last_became_peered":"2023-12-01T02:52:21.994737+0000","last_unstale":"2023-12-01T02:52:22.001915+0000","last_undegraded":"2023-12-01T02:52:22.001915+0000","last_fullsized":"2023-12-01T02:52:22.001915+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:48:36.231827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"176'2","reported_seq":13,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.008744+0000","last_change":"2023-12-01T02:52:21.516514+0000","last_active":"2023-12-01T02:52:22.008744+0000","last_peered":"2023-12-01T02:52:22.008744+0000","last_clean":"2023-12-01T02:52:22.008744+0000","last_became_active":"2023-12-01T02:52:21.516229+0000","last_became_peered":"2023-12-01T02:52:21.516229+0000","last_unstale":"2023-12-01T02:52:22.008744+0000","last_undegraded":"2023-12-01T02:52:22.008744+0000","last_fullsized":"2023-12-01T02:52:22.008744+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:19.139617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.004002+0000","last_change":"2023-12-01T02:52:21.995168+0000","last_active":"2023-12-01T02:52:22.004002+0000","last_peered":"2023-12-01T02:52:22.004002+0000","last_clean":"2023-12-01T02:52:22.004002+0000","last_became_active":"2023-12-01T02:52:21.994888+0000","last_became_peered":"2023-12-01T02:52:21.994888+0000","last_unstale":"2023-12-01T02:52:22.004002+0000","last_undegraded":"2023-12-01T02:52:22.004002+0000","last_fullsized":"2023-12-01T02:52:22.004002+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:54.304823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.998557+0000","last_change":"2023-12-01T02:52:21.510073+0000","last_active":"2023-12-01T02:52:21.998557+0000","last_peered":"2023-12-01T02:52:21.998557+0000","last_clean":"2023-12-01T02:52:21.998557+0000","last_became_active":"2023-12-01T02:52:21.509784+0000","last_became_peered":"2023-12-01T02:52:21.509784+0000","last_unstale":"2023-12-01T02:52:21.998557+0000","last_undegraded":"2023-12-01T02:52:21.998557+0000","last_fullsized":"2023-12-01T02:52:21.998557+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:36.014503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.009649+0000","last_change":"2023-12-01T02:52:22.005229+0000","last_active":"2023-12-01T02:52:22.009649+0000","last_peered":"2023-12-01T02:52:22.009649+0000","last_clean":"2023-12-01T02:52:22.009649+0000","last_became_active":"2023-12-01T02:52:22.004891+0000","last_became_peered":"2023-12-01T02:52:22.004891+0000","last_unstale":"2023-12-01T02:52:22.009649+0000","last_undegraded":"2023-12-01T02:52:22.009649+0000","last_fullsized":"2023-12-01T02:52:22.009649+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:51.425590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.988153+0000","last_change":"2023-12-01T02:52:21.512850+0000","last_active":"2023-12-01T02:52:21.988153+0000","last_peered":"2023-12-01T02:52:21.988153+0000","last_clean":"2023-12-01T02:52:21.988153+0000","last_became_active":"2023-12-01T02:52:21.509201+0000","last_became_peered":"2023-12-01T02:52:21.509201+0000","last_unstale":"2023-12-01T02:52:21.988153+0000","last_undegraded":"2023-12-01T02:52:21.988153+0000","last_fullsized":"2023-12-01T02:52:21.988153+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:50.410425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.995510+0000","last_change":"2023-12-01T02:52:21.991927+0000","last_active":"2023-12-01T02:52:21.995510+0000","last_peered":"2023-12-01T02:52:21.995510+0000","last_clean":"2023-12-01T02:52:21.995510+0000","last_became_active":"2023-12-01T02:52:21.991483+0000","last_became_peered":"2023-12-01T02:52:21.991483+0000","last_unstale":"2023-12-01T02:52:21.995510+0000","last_undegraded":"2023-12-01T02:52:21.995510+0000","last_fullsized":"2023-12-01T02:52:21.995510+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:22:04.947371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.985750+0000","last_change":"2023-12-01T02:52:21.514300+0000","last_active":"2023-12-01T02:52:21.985750+0000","last_peered":"2023-12-01T02:52:21.985750+0000","last_clean":"2023-12-01T02:52:21.985750+0000","last_became_active":"2023-12-01T02:52:21.513915+0000","last_became_peered":"2023-12-01T02:52:21.513915+0000","last_unstale":"2023-12-01T02:52:21.985750+0000","last_undegraded":"2023-12-01T02:52:21.985750+0000","last_fullsized":"2023-12-01T02:52:21.985750+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:15:51.909423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.951309+0000","last_change":"2023-12-01T02:52:21.991107+0000","last_active":"2023-12-01T02:52:23.951309+0000","last_peered":"2023-12-01T02:52:23.951309+0000","last_clean":"2023-12-01T02:52:23.951309+0000","last_became_active":"2023-12-01T02:52:21.990671+0000","last_became_peered":"2023-12-01T02:52:21.990671+0000","last_unstale":"2023-12-01T02:52:23.951309+0000","last_undegraded":"2023-12-01T02:52:23.951309+0000","last_fullsized":"2023-12-01T02:52:23.951309+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:48:40.093953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.001018+0000","last_change":"2023-12-01T02:52:21.500628+0000","last_active":"2023-12-01T02:52:22.001018+0000","last_peered":"2023-12-01T02:52:22.001018+0000","last_clean":"2023-12-01T02:52:22.001018+0000","last_became_active":"2023-12-01T02:52:21.499540+0000","last_became_peered":"2023-12-01T02:52:21.499540+0000","last_unstale":"2023-12-01T02:52:22.001018+0000","last_undegraded":"2023-12-01T02:52:22.001018+0000","last_fullsized":"2023-12-01T02:52:22.001018+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:14.533994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.003977+0000","last_change":"2023-12-01T02:52:21.995598+0000","last_active":"2023-12-01T02:52:22.003977+0000","last_peered":"2023-12-01T02:52:22.003977+0000","last_clean":"2023-12-01T02:52:22.003977+0000","last_became_active":"2023-12-01T02:52:21.995297+0000","last_became_peered":"2023-12-01T02:52:21.995297+0000","last_unstale":"2023-12-01T02:52:22.003977+0000","last_undegraded":"2023-12-01T02:52:22.003977+0000","last_fullsized":"2023-12-01T02:52:22.003977+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:30:41.584477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.978908+0000","last_change":"2023-12-01T02:52:21.521596+0000","last_active":"2023-12-01T02:52:21.978908+0000","last_peered":"2023-12-01T02:52:21.978908+0000","last_clean":"2023-12-01T02:52:21.978908+0000","last_became_active":"2023-12-01T02:52:21.520352+0000","last_became_peered":"2023-12-01T02:52:21.520352+0000","last_unstale":"2023-12-01T02:52:21.978908+0000","last_undegraded":"2023-12-01T02:52:21.978908+0000","last_fullsized":"2023-12-01T02:52:21.978908+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:56:39.608571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.008054+0000","last_change":"2023-12-01T02:52:22.001332+0000","last_active":"2023-12-01T02:52:22.008054+0000","last_peered":"2023-12-01T02:52:22.008054+0000","last_clean":"2023-12-01T02:52:22.008054+0000","last_became_active":"2023-12-01T02:52:22.001081+0000","last_became_peered":"2023-12-01T02:52:22.001081+0000","last_unstale":"2023-12-01T02:52:22.008054+0000","last_undegraded":"2023-12-01T02:52:22.008054+0000","last_fullsized":"2023-12-01T02:52:22.008054+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:24:05.477889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.980630+0000","last_change":"2023-12-01T02:52:21.512432+0000","last_active":"2023-12-01T02:52:21.980630+0000","last_peered":"2023-12-01T02:52:21.980630+0000","last_clean":"2023-12-01T02:52:21.980630+0000","last_became_active":"2023-12-01T02:52:21.512157+0000","last_became_peered":"2023-12-01T02:52:21.512157+0000","last_unstale":"2023-12-01T02:52:21.980630+0000","last_undegraded":"2023-12-01T02:52:21.980630+0000","last_fullsized":"2023-12-01T02:52:21.980630+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:09.305110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:22.980846+0000","last_change":"2023-12-01T02:52:21.995816+0000","last_active":"2023-12-01T02:52:22.980846+0000","last_peered":"2023-12-01T02:52:22.980846+0000","last_clean":"2023-12-01T02:52:22.980846+0000","last_became_active":"2023-12-01T02:52:21.995554+0000","last_became_peered":"2023-12-01T02:52:21.995554+0000","last_unstale":"2023-12-01T02:52:22.980846+0000","last_undegraded":"2023-12-01T02:52:22.980846+0000","last_fullsized":"2023-12-01T02:52:22.980846+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:32:56.513251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.982797+0000","last_change":"2023-12-01T02:52:21.518362+0000","last_active":"2023-12-01T02:52:21.982797+0000","last_peered":"2023-12-01T02:52:21.982797+0000","last_clean":"2023-12-01T02:52:21.982797+0000","last_became_active":"2023-12-01T02:52:21.518064+0000","last_became_peered":"2023-12-01T02:52:21.518064+0000","last_unstale":"2023-12-01T02:52:21.982797+0000","last_undegraded":"2023-12-01T02:52:21.982797+0000","last_fullsized":"2023-12-01T02:52:21.982797+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:23:31.327211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.951405+0000","last_change":"2023-12-01T02:52:21.995129+0000","last_active":"2023-12-01T02:52:23.951405+0000","last_peered":"2023-12-01T02:52:23.951405+0000","last_clean":"2023-12-01T02:52:23.951405+0000","last_became_active":"2023-12-01T02:52:21.994473+0000","last_became_peered":"2023-12-01T02:52:21.994473+0000","last_unstale":"2023-12-01T02:52:23.951405+0000","last_undegraded":"2023-12-01T02:52:23.951405+0000","last_fullsized":"2023-12-01T02:52:23.951405+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:21.471172+0000","last_clean_scrub_stamp":"2023-12-01T02:52:21.471172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:52:17.390771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"176'3","reported_seq":14,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-01T02:52:21.999416+0000","last_change":"2023-12-01T02:52:21.511311+0000","last_active":"2023-12-01T02:52:21.999416+0000","last_peered":"2023-12-01T02:52:21.999416+0000","last_clean":"2023-12-01T02:52:21.999416+0000","last_became_active":"2023-12-01T02:52:21.511021+0000","last_became_peered":"2023-12-01T02:52:21.511021+0000","last_unstale":"2023-12-01T02:52:21.999416+0000","last_undegraded":"2023-12-01T02:52:21.999416+0000","last_fullsized":"2023-12-01T02:52:21.999416+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T02:52:20.456007+0000","last_clean_scrub_stamp":"2023-12-01T02:52:20.456007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:16:42.785131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"165'271","reported_seq":882,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-01T02:52:23.892693+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T02:52:23.892693+0000","last_peered":"2023-12-01T02:52:23.892693+0000","last_clean":"2023-12-01T02:52:23.892693+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T02:52:23.892693+0000","last_undegraded":"2023-12-01T02:52:23.892693+0000","last_fullsized":"2023-12-01T02:52:23.892693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575763,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55348,"kb_used_data":2156,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316492,"statfs":{"total":96636764160,"available":96580087808,"internally_reserved":0,"allocated":2207744,"data_stored":1065606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.083,"15min":1.0720000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3799999999999999,"5min":1.7829999999999999,"15min":1.893},"last":1.26},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1539999999999999,"15min":1.1100000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.514,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.266,"15min":1.2709999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.02},"last":1.647},{"interface":"front","average":{"1min":1.26,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85399999999999998},"max":{"1min":1.585,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":1.5880000000000001}]},{"osd":2,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2350000000000001,"15min":1.2010000000000001},"min":{"1min":0.82499999999999996,"5min":0.80600000000000005,"15min":0.78500000000000003},"max":{"1min":1.554,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.28,"15min":1.25},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.8440000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.534}]},{"osd":3,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.343,"15min":1.3220000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.742,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.048,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.7509999999999999}]},{"osd":4,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3340000000000001,"15min":1.3280000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.851,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.506,"5min":1.4930000000000001,"15min":1.4750000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.157,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.9179999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3149999999999999,"15min":1.284},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":2.0699999999999998,"5min":2.976,"15min":2.976},"last":1.337},{"interface":"front","average":{"1min":1.597,"5min":1.595,"15min":1.569},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.2160000000000002,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.857}]},{"osd":6,"last update":"Fri Dec 1 02:51:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4590000000000001,"15min":1.4179999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.1499999999999999,"5min":3.3380000000000001,"15min":3.3380000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4870000000000001,"15min":1.4470000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.966,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.4350000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575764,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37300,"kb_used_data":2156,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334540,"statfs":{"total":96636764160,"available":96598568960,"internally_reserved":0,"allocated":2207744,"data_stored":1065606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.161,"15min":1.165},"min":{"1min":0.84299999999999997,"5min":0.78100000000000003,"15min":0.75},"max":{"1min":1.6080000000000001,"5min":1.73,"15min":2.2759999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.083,"5min":1.1439999999999999,"15min":1.1619999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.56799999999999995},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.966},"last":0.98699999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1459999999999999,"15min":1.1859999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.78800000000000003},"max":{"1min":1.6459999999999999,"5min":1.772,"15min":2.0409999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.2,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6679999999999999,"5min":1.7370000000000001,"15min":2.093},"last":0.94599999999999995}]},{"osd":2,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1679999999999999,"15min":1.1879999999999999},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.80600000000000005},"max":{"1min":1.5369999999999999,"5min":1.736,"15min":1.946},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.272,"15min":1.284},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.637,"5min":1.7190000000000001,"15min":2.1230000000000002},"last":1.343}]},{"osd":3,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3080000000000001,"15min":1.3160000000000001},"min":{"1min":1.0620000000000001,"5min":1.006,"15min":0.90500000000000003},"max":{"1min":1.6140000000000001,"5min":1.819,"15min":1.819},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.431,"15min":1.4319999999999999},"min":{"1min":1.042,"5min":0.98699999999999999,"15min":0.92800000000000005},"max":{"1min":1.738,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.4890000000000001}]},{"osd":4,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.262},"min":{"1min":0.93000000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.698},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.3500000000000001,"15min":1.3360000000000001},"min":{"1min":0.96699999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.98,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.847}]},{"osd":5,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.4419999999999999,"15min":1.4279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.8120000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.5,"15min":1.4970000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.7799999999999998},"last":1.6459999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:51:41 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.5149999999999999,"15min":1.524},"min":{"1min":1.0980000000000001,"5min":1.034,"15min":1.034},"max":{"1min":1.9419999999999999,"5min":2.1549999999999998,"15min":2.5659999999999998},"last":1.891},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6040000000000001,"15min":1.5940000000000001},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"max":{"1min":2.0499999999999998,"5min":2.226,"15min":2.3809999999999998},"last":1.7909999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575763,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64144,"kb_used_data":2760,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307696,"statfs":{"total":96636764160,"available":96571080704,"internally_reserved":0,"allocated":2826240,"data_stored":1657732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1499999999999999,"15min":1.1419999999999999},"min":{"1min":0.68700000000000006,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4810000000000001,"5min":1.73,"15min":1.744},"last":0.625},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1000000000000001}]},{"osd":1,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.137,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.56200000000000006},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.704},"last":1.159},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.226,"15min":1.2330000000000001},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.016},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2270000000000001,"15min":1.24},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5900000000000001,"5min":1.7430000000000001,"15min":1.746},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2609999999999999,"15min":1.2809999999999999},"min":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.627},"max":{"1min":1.655,"5min":1.671,"15min":1.764},"last":1.282}]},{"osd":3,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.321,"15min":1.319},"min":{"1min":0.98399999999999999,"5min":0.872,"15min":0.84899999999999998},"max":{"1min":1.546,"5min":1.7729999999999999,"15min":1.871},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3460000000000001,"15min":1.3620000000000001},"min":{"1min":1.0269999999999999,"5min":0.91500000000000004,"15min":0.879},"max":{"1min":1.7250000000000001,"5min":1.7470000000000001,"15min":1.962},"last":1.395}]},{"osd":4,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.254,"15min":1.2569999999999999},"min":{"1min":0.91300000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.696,"5min":1.847,"15min":1.978},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.391,"5min":1.361,"15min":1.3620000000000001},"min":{"1min":1.0740000000000001,"5min":0.91600000000000004,"15min":0.88600000000000001},"max":{"1min":1.911,"5min":1.911,"15min":2.0350000000000001},"last":1.3129999999999999}]},{"osd":6,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3520000000000001,"15min":1.375},"min":{"1min":0.89300000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":2.4470000000000001},"last":1.526},{"interface":"front","average":{"1min":1.411,"5min":1.391,"15min":1.419},"min":{"1min":1.077,"5min":0.876,"15min":0.876},"max":{"1min":1.798,"5min":1.946,"15min":2.7210000000000001},"last":1.0509999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:51:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.5449999999999999,"15min":1.5369999999999999},"min":{"1min":1.1200000000000001,"5min":1.099,"15min":0.92100000000000004},"max":{"1min":2.157,"5min":2.157,"15min":3.0379999999999998},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.5780000000000001,"15min":1.5860000000000001},"min":{"1min":1.2290000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":3.5270000000000001},"last":1.472}]}]},{"osd":4,"up_from":13,"seq":55834575763,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":2176,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":2228224,"data_stored":1067774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.113,"15min":1.137},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.6180000000000001},"last":1.153},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1599999999999999,"15min":1.1699999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.3899999999999999}]},{"osd":1,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.234,"15min":1.2270000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.48,"5min":1.706,"15min":1.7709999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.139,"15min":1.1830000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.381,"5min":1.6100000000000001,"15min":1.8640000000000001},"last":1.296}]},{"osd":2,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1910000000000001,"15min":1.218},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":2.1499999999999999},"last":1.472},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3240000000000001,"15min":1.3169999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.85999999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":1.1220000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.2470000000000001,"15min":1.2609999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.454,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.016},{"interface":"front","average":{"1min":1.298,"5min":1.3260000000000001,"15min":1.329},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":2.2440000000000002},"last":1.343}]},{"osd":5,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.367,"15min":1.3140000000000001},"min":{"1min":0.92300000000000004,"5min":0.86499999999999999,"15min":0.84499999999999997},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3069999999999999,"15min":1.284},"min":{"1min":0.96699999999999997,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.712,"5min":1.875,"15min":1.875},"last":1.4430000000000001}]},{"osd":6,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3580000000000001,"15min":1.325},"min":{"1min":0.82599999999999996,"5min":0.81299999999999994,"15min":0.78500000000000003},"max":{"1min":1.996,"5min":1.996,"15min":2.4489999999999998},"last":1.609},{"interface":"front","average":{"1min":1.554,"5min":1.474,"15min":1.448},"min":{"1min":1.097,"5min":1.014,"15min":0.97899999999999998},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":1.708}]},{"osd":7,"last update":"Fri Dec 1 02:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.444,"15min":1.4379999999999999},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.92900000000000005},"max":{"1min":1.905,"5min":2.044,"15min":2.044},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.552,"15min":1.544},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.8859999999999999,"5min":2.0670000000000002,"15min":2.5760000000000001},"last":1.671}]}]},{"osd":3,"up_from":13,"seq":55834575762,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51260,"kb_used_data":2164,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320580,"statfs":{"total":96636764160,"available":96584273920,"internally_reserved":0,"allocated":2215936,"data_stored":1066222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.90600000000000003,"15min":0.89300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.0309999999999999,"15min":1.0660000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3680000000000001,"5min":1.5449999999999999,"15min":1.5669999999999999},"last":1.081}]},{"osd":1,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.1459999999999999,"15min":1.145},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2709999999999999,"5min":1.708,"15min":1.708},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.97799999999999998,"15min":1.004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.329,"5min":1.5980000000000001,"15min":1.744},"last":1.3879999999999999}]},{"osd":2,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0800000000000001,"15min":1.097},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.9140000000000001,"5min":2.9140000000000001,"15min":2.9140000000000001},"last":1.171},{"interface":"front","average":{"1min":1.028,"5min":1.2050000000000001,"15min":1.2490000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4550000000000001,"5min":1.5740000000000001,"15min":1.728},"last":1.1970000000000001}]},{"osd":4,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.302,"15min":1.294},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.306,"15min":1.2949999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.67100000000000004},"max":{"1min":1.802,"5min":1.802,"15min":1.927},"last":1.127}]},{"osd":5,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3160000000000001,"15min":1.3140000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.294},{"interface":"front","average":{"1min":1.377,"5min":1.3979999999999999,"15min":1.387},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.9769999999999999,"5min":2.9769999999999999,"15min":2.9769999999999999},"last":1.482}]},{"osd":6,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.448,"15min":1.4379999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.67900000000000005},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.524,"15min":1.4970000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.554}]},{"osd":7,"last update":"Fri Dec 1 02:52:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.919,"5min":2.0790000000000002,"15min":2.1549999999999998},"last":1.673},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.5109999999999999,"15min":1.51},"min":{"1min":0.747,"5min":0.747,"15min":0.73699999999999999},"max":{"1min":1.827,"5min":2.0150000000000001,"15min":2.1259999999999999},"last":1.645}]}]},{"osd":2,"up_from":13,"seq":55834575762,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41400,"kb_used_data":2160,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330440,"statfs":{"total":96636764160,"available":96594370560,"internally_reserved":0,"allocated":2211840,"data_stored":1065628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":0.93200000000000005,"15min":0.88900000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.49299999999999999},"max":{"1min":5.2080000000000002,"5min":5.2080000000000002,"15min":5.2080000000000002},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.127,"5min":1.0860000000000001,"15min":1.073},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.3959999999999999,"5min":5.3959999999999999,"15min":5.3959999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1160000000000001,"15min":1.085},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":5.1109999999999998,"5min":5.1109999999999998,"15min":5.1109999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.1619999999999999,"15min":1.1259999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":5.4800000000000004,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.073,"15min":1.0549999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":4.0540000000000003,"5min":4.0540000000000003,"15min":4.0540000000000003},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.2869999999999999,"15min":1.2230000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":5.7690000000000001,"5min":5.7690000000000001,"15min":5.7690000000000001},"last":1}]},{"osd":4,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3779999999999999,"15min":1.327},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":5.5590000000000002,"5min":5.5590000000000002,"15min":5.5590000000000002},"last":1.579},{"interface":"front","average":{"1min":1.198,"5min":1.29,"15min":1.2949999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.5009999999999999}]},{"osd":5,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.4119999999999999,"15min":1.379},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":5.8520000000000003,"5min":5.8520000000000003,"15min":5.8520000000000003},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.4610000000000001,"15min":1.4079999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":5.7279999999999998,"5min":5.7279999999999998,"15min":5.7279999999999998},"last":1.228}]},{"osd":6,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.397,"15min":1.3819999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5669999999999999,"15min":1.51},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":0.82699999999999996},"max":{"1min":5.8159999999999998,"5min":5.8159999999999998,"15min":5.8159999999999998},"last":1.274}]},{"osd":7,"last update":"Fri Dec 1 02:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.548,"15min":1.492},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.4890000000000001,"15min":1.458},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.4339999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575761,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64124,"kb_used_data":2740,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307716,"statfs":{"total":96636764160,"available":96571101184,"internally_reserved":0,"allocated":2805760,"data_stored":1656044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.92300000000000004,"15min":0.90500000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.137,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.133},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.095,"15min":1.0740000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.722,"15min":1.722},"last":1.085}]},{"osd":2,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93200000000000005,"15min":0.92500000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.466,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.038},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.1439999999999999,"15min":1.1479999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5880000000000001,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.389}]},{"osd":3,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.5189999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":1.712},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1339999999999999,"15min":1.1279999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.444,"5min":1.6759999999999999,"15min":1.9219999999999999},"last":1.6599999999999999}]},{"osd":4,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.24,"15min":1.226},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.4910000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.76},{"interface":"front","average":{"1min":1.413,"5min":1.4219999999999999,"15min":1.391},"min":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.032},"max":{"1min":1.8140000000000001,"5min":1.9730000000000001,"15min":2.1280000000000001},"last":1.6100000000000001}]},{"osd":5,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.349},"min":{"1min":1.0620000000000001,"5min":0.98899999999999999,"15min":0.97899999999999998},"max":{"1min":1.49,"5min":1.9930000000000001,"15min":2.0449999999999999},"last":1.9590000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.3660000000000001,"15min":1.357},"min":{"1min":1.0349999999999999,"5min":1.0229999999999999,"15min":1.004},"max":{"1min":1.804,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.4359999999999999}]},{"osd":6,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.454,"15min":1.4399999999999999},"min":{"1min":0.996,"5min":0.996,"15min":0.996},"max":{"1min":1.873,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":2.1230000000000002},{"interface":"front","average":{"1min":1.341,"5min":1.4139999999999999,"15min":1.4119999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.7250000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.869}]},{"osd":7,"last update":"Fri Dec 1 02:51:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5289999999999999,"15min":1.518},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.9339999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.8220000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.516,"15min":1.514},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8029999999999999,"5min":2.1749999999999998,"15min":2.282},"last":2.448}]}]},{"osd":0,"up_from":13,"seq":55834575763,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41400,"kb_used_data":2160,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330440,"statfs":{"total":96636764160,"available":96594370560,"internally_reserved":0,"allocated":2211840,"data_stored":1065696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.88600000000000001,"15min":0.89700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2470000000000001,"5min":1.5309999999999999,"15min":1.766},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.0489999999999999,"15min":1.0609999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.345,"5min":1.5980000000000001,"15min":1.605},"last":0.98499999999999999}]},{"osd":2,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.034,"15min":1.056},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2410000000000001,"5min":1.4470000000000001,"15min":1.8540000000000001},"last":1.262},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.089,"15min":1.1220000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.464,"5min":1.595,"15min":1.7030000000000001},"last":1.4630000000000001}]},{"osd":3,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":1.036,"15min":1.054},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1890000000000001,"5min":1.5509999999999999,"15min":1.8100000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.2270000000000001,"15min":1.244},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7230000000000001,"5min":1.798,"15min":1.9470000000000001},"last":1.401}]},{"osd":4,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.327,"15min":1.345},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.591,"5min":1.6910000000000001,"15min":2.0979999999999999},"last":1.341},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.258,"15min":1.282},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.651,"5min":1.6579999999999999,"15min":2.0739999999999998},"last":1.151}]},{"osd":5,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.407,"15min":1.3999999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6930000000000001,"5min":2.2370000000000001,"15min":2.6110000000000002},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.4079999999999999,"15min":1.4099999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.206}]},{"osd":6,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.377,"15min":1.367},"min":{"1min":0.77700000000000002,"5min":0.745,"15min":0.745},"max":{"1min":1.643,"5min":2.0939999999999999,"15min":2.1680000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.5009999999999999,"15min":1.4970000000000001},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":1.748,"5min":2.798,"15min":2.798},"last":1.9099999999999999}]},{"osd":7,"last update":"Fri Dec 1 02:52:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.548,"15min":1.514},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":3.0369999999999999,"5min":3.0369999999999999,"15min":3.0369999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.458,"15min":1.472},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.7689999999999999,"5min":2.0830000000000002,"15min":3.0590000000000002},"last":1.3819999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T02:52:31.914 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T02:52:31.914 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T02:52:32.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.175+0000 7f9b723d2640 1 Processor -- start 2023-12-01T02:52:32.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.175+0000 7f9b723d2640 1 -- start start 2023-12-01T02:52:32.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.177+0000 7f9b723d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0afcd0 0x7f9b6c0b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.177+0000 7f9b723d2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6c06b280 con 0x7f9b6c157250 2023-12-01T02:52:32.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.177+0000 7f9b723d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6c0b0120 con 0x7f9b6c0af820 2023-12-01T02:52:32.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.177+0000 7f9b723d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c06b520 con 0x7f9b6c0afcd0 2023-12-01T02:52:32.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.177+0000 7f9b6bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9b6c157250 0x7f9b6c15dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59210/0 (socket says 172.21.15.115:59210) 2023-12-01T02:52:32.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.177+0000 7f9b6bfff640 1 -- 172.21.15.115:0/2546193372 learned_addr learned my addr 172.21.15.115:0/2546193372 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.177+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422733829 0 0) 0x7f9b6c06b280 con 0x7f9b6c157250 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b7096f640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0afcd0 0x7f9b6c0b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b54003520 con 0x7f9b6c157250 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3584427005 0 0) 0x7f9b54003520 con 0x7f9b6c157250 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0afcd0 msgr2=0x7f9b6c0b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0afcd0 0x7f9b6c0b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 >> v1:172.21.15.142:6789/0 conn(0x7f9b6c0af820 legacy=0x7f9b6c0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c06b800 con 0x7f9b6c157250 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f9b5c001690 con 0x7f9b6c157250 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 >> v1:172.21.15.115:6789/0 conn(0x7f9b6c157250 legacy=0x7f9b6c15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0afcd0 0x7f9b540039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b54003f20 0x7f9b540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b54004d00 0x7f9b54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c06b520 con 0x7f9b54003f20 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5400ba60 con 0x7f9b54004d00 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.178+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5400bc20 con 0x7f9b6c0afcd0 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.179+0000 7f9b6b7fe640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b54003f20 0x7f9b540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.179+0000 7f9b6bfff640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0afcd0 0x7f9b540039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.179+0000 7f9b7096f640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b54004d00 0x7f9b54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.179+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0afcd0 msgr2=0x7f9b540039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.179+0000 7f9b6b7fe640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0afcd0 0x7f9b540039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.179+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b54004d00 msgr2=0x7f9b54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.179+0000 7f9b6b7fe640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b54004d00 0x7f9b54005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.179+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/2546193372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b54003520 con 0x7f9b54003f20 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.179+0000 7f9b6bfff640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b6c0afcd0 0x7f9b540039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:32.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.179+0000 7f9b7096f640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b54004d00 0x7f9b54005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:32.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.180+0000 7f9b6b7fe640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b54003f20 0x7f9b540047c0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f9b60001750 tx=0x7f9b540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f50e0c0d07ef94 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:32.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.180+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b600052d0 con 0x7f9b54003f20 2023-12-01T02:52:32.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.180+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6002e050 con 0x7f9b54003f20 2023-12-01T02:52:32.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.180+0000 7f9b6affd640 1 -- 172.21.15.115:0/2546193372 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60003830 con 0x7f9b54003f20 2023-12-01T02:52:32.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.181+0000 7f9b723d2640 1 -- 172.21.15.115:0/2546193372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b54003f20 msgr2=0x7f9b540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.181+0000 7f9b723d2640 1 --2- 172.21.15.115:0/2546193372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b54003f20 0x7f9b540047c0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f9b60001750 tx=0x7f9b540057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.181+0000 7f9b7096f640 1 -- 172.21.15.115:0/2546193372 reap_dead start 2023-12-01T02:52:32.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.181+0000 7f9b723d2640 1 -- 172.21.15.115:0/2546193372 shutdown_connections 2023-12-01T02:52:32.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.181+0000 7f9b723d2640 1 -- 172.21.15.115:0/2546193372 >> 172.21.15.115:0/2546193372 conn(0x7f9b6c065960 msgr2=0x7f9b6c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:32.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.181+0000 7f9b723d2640 1 -- 172.21.15.115:0/2546193372 shutdown_connections 2023-12-01T02:52:32.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.181+0000 7f9b723d2640 1 -- 172.21.15.115:0/2546193372 wait complete. 2023-12-01T02:52:32.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.183+0000 7f9b723d2640 1 Processor -- start 2023-12-01T02:52:32.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.183+0000 7f9b723d2640 1 -- start start 2023-12-01T02:52:32.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.184+0000 7f9b723d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54003f20 0x7f9b6c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.184+0000 7f9b723d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c143080 0x7f9b6c144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.184+0000 7f9b723d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c1f4d50 0x7f9b6c145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.184+0000 7f9b723d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c1434e0 con 0x7f9b6c143080 2023-12-01T02:52:32.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.184+0000 7f9b723d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c1437b0 con 0x7f9b6c1f4d50 2023-12-01T02:52:32.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.184+0000 7f9b723d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c143a90 con 0x7f9b54003f20 2023-12-01T02:52:32.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.184+0000 7f9b6b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c143080 0x7f9b6c144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b6b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c143080 0x7f9b6c144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34632/0 (socket says 172.21.15.115:34632) 2023-12-01T02:52:32.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/3575270091 learned_addr learned my addr 172.21.15.115:0/3575270091 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b7096f640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c1f4d50 0x7f9b6c145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b6bfff640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54003f20 0x7f9b6c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/3575270091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54003f20 msgr2=0x7f9b6c15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b6b7fe640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54003f20 0x7f9b6c15a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/3575270091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c1f4d50 msgr2=0x7f9b6c145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b6b7fe640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c1f4d50 0x7f9b6c145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b6b7fe640 1 -- 172.21.15.115:0/3575270091 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c143d70 con 0x7f9b6c143080 2023-12-01T02:52:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b7096f640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c1f4d50 0x7f9b6c145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b6bfff640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54003f20 0x7f9b6c15a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:32.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.185+0000 7f9b6b7fe640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c143080 0x7f9b6c144850 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f9b60008fd0 tx=0x7f9b60001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:32.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.186+0000 7f9b68ff9640 1 -- 172.21.15.115:0/3575270091 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b600076c0 con 0x7f9b6c143080 2023-12-01T02:52:32.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.186+0000 7f9b68ff9640 1 -- 172.21.15.115:0/3575270091 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b60007a30 con 0x7f9b6c143080 2023-12-01T02:52:32.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.186+0000 7f9b723d2640 1 -- 172.21.15.115:0/3575270091 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6c144050 con 0x7f9b6c143080 2023-12-01T02:52:32.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.186+0000 7f9b68ff9640 1 -- 172.21.15.115:0/3575270091 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60006be0 con 0x7f9b6c143080 2023-12-01T02:52:32.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.186+0000 7f9b723d2640 1 -- 172.21.15.115:0/3575270091 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c144d90 con 0x7f9b6c143080 2023-12-01T02:52:32.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.187+0000 7f9b723d2640 1 -- 172.21.15.115:0/3575270091 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c04e890 con 0x7f9b6c143080 2023-12-01T02:52:32.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.188+0000 7f9b68ff9640 1 -- 172.21.15.115:0/3575270091 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9b60006d40 con 0x7f9b6c143080 2023-12-01T02:52:32.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.189+0000 7f9b68ff9640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5806a580 0x7f9b5806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.190+0000 7f9b68ff9640 1 -- 172.21.15.115:0/3575270091 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f9b600aaca0 con 0x7f9b6c143080 2023-12-01T02:52:32.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.190+0000 7f9b6bfff640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5806a580 0x7f9b5806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.190+0000 7f9b6bfff640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5806a580 0x7f9b5806ca40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c001eb0 tx=0x7f9b5c009040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:32.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.194+0000 7f9b68ff9640 1 -- 172.21.15.115:0/3575270091 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9b60075fb0 con 0x7f9b6c143080 2023-12-01T02:52:32.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.453+0000 7f9b723d2640 1 -- 172.21.15.115:0/3575270091 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9b6c1434e0 con 0x7f9b6c143080 2023-12-01T02:52:32.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.462+0000 7f9b68ff9640 1 -- 172.21.15.115:0/3575270091 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v41) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9b6004b020 con 0x7f9b6c143080 2023-12-01T02:52:32.464 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T02:52:32.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.464+0000 7f9b723d2640 1 -- 172.21.15.115:0/3575270091 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5806a580 msgr2=0x7f9b5806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.464+0000 7f9b723d2640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5806a580 0x7f9b5806ca40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c001eb0 tx=0x7f9b5c009040 comp rx=0 tx=0).stop 2023-12-01T02:52:32.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.464+0000 7f9b723d2640 1 -- 172.21.15.115:0/3575270091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c143080 msgr2=0x7f9b6c144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.464+0000 7f9b723d2640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c143080 0x7f9b6c144850 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f9b60008fd0 tx=0x7f9b60001120 comp rx=0 tx=0).stop 2023-12-01T02:52:32.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.465+0000 7f9b723d2640 1 -- 172.21.15.115:0/3575270091 shutdown_connections 2023-12-01T02:52:32.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.465+0000 7f9b723d2640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f9b5806a580 0x7f9b5806ca40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.465+0000 7f9b723d2640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b6c1f4d50 0x7f9b6c145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.465+0000 7f9b723d2640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b6c143080 0x7f9b6c144850 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.465+0000 7f9b723d2640 1 --2- 172.21.15.115:0/3575270091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b54003f20 0x7f9b6c15a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.465+0000 7f9b723d2640 1 -- 172.21.15.115:0/3575270091 >> 172.21.15.115:0/3575270091 conn(0x7f9b6c065960 msgr2=0x7f9b6c15c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:32.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.465+0000 7f9b723d2640 1 -- 172.21.15.115:0/3575270091 shutdown_connections 2023-12-01T02:52:32.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.465+0000 7f9b723d2640 1 -- 172.21.15.115:0/3575270091 wait complete. 2023-12-01T02:52:32.484 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T02:52:32.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.698+0000 7fabdb221640 1 Processor -- start 2023-12-01T02:52:32.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.698+0000 7fabdb221640 1 -- start start 2023-12-01T02:52:32.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.699+0000 7fabdb221640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fabd415d220 0x7fabd40cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.699+0000 7fabdb221640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fabd40ad2b0 0x7fabd40ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.699+0000 7fabdb221640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabd40ade80 0x7fabd406b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.699+0000 7fabdb221640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd406cc10 con 0x7fabd40ad2b0 2023-12-01T02:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.699+0000 7fabdb221640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd406c720 con 0x7fabd415d220 2023-12-01T02:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.699+0000 7fabdb221640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd415d600 con 0x7fabd40ade80 2023-12-01T02:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.700+0000 7fabd3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fabd40ad2b0 0x7fabd40ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.700+0000 7fabd3fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fabd40ad2b0 0x7fabd40ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34646/0 (socket says 172.21.15.115:34646) 2023-12-01T02:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.700+0000 7fabd3fff640 1 -- 172.21.15.115:0/3107576859 learned_addr learned my addr 172.21.15.115:0/3107576859 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:32.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.700+0000 7fabd3fff640 1 -- 172.21.15.115:0/3107576859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabd40ade80 msgr2=0x7fabd406b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.700+0000 7fabd3fff640 1 --2- 172.21.15.115:0/3107576859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabd40ade80 0x7fabd406b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.700+0000 7fabd8fbd640 1 --2- 172.21.15.115:0/3107576859 >> v2:172.21.15.142:3300/0 conn(0x7fabd415d220 0x7fabd40cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.700+0000 7fabd3fff640 1 -- 172.21.15.115:0/3107576859 >> v2:172.21.15.142:3300/0 conn(0x7fabd415d220 msgr2=0x7fabd40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.700+0000 7fabd3fff640 1 --2- 172.21.15.115:0/3107576859 >> v2:172.21.15.142:3300/0 conn(0x7fabd415d220 0x7fabd40cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.700+0000 7fabd3fff640 1 -- 172.21.15.115:0/3107576859 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd406c4b0 con 0x7fabd40ad2b0 2023-12-01T02:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.701+0000 7fabd3fff640 1 --2- 172.21.15.115:0/3107576859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd40ad2b0 0x7fabd40ccab0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fabd40724a0 tx=0x7fabc402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6791ba120258b1af server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabd37fe640 1 -- 172.21.15.115:0/3107576859 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc40317d0 con 0x7fabd40ad2b0 2023-12-01T02:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabd37fe640 1 -- 172.21.15.115:0/3107576859 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc4031930 con 0x7fabd40ad2b0 2023-12-01T02:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabd37fe640 1 -- 172.21.15.115:0/3107576859 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4031cc0 con 0x7fabd40ad2b0 2023-12-01T02:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabdb221640 1 -- 172.21.15.115:0/3107576859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd40ad2b0 msgr2=0x7fabd40ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabdb221640 1 --2- 172.21.15.115:0/3107576859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd40ad2b0 0x7fabd40ccab0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fabd40724a0 tx=0x7fabc402b870 comp rx=0 tx=0).stop 2023-12-01T02:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabdb221640 1 -- 172.21.15.115:0/3107576859 shutdown_connections 2023-12-01T02:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabdb221640 1 --2- 172.21.15.115:0/3107576859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabd40ade80 0x7fabd406b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabdb221640 1 --2- 172.21.15.115:0/3107576859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd40ad2b0 0x7fabd40ccab0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabdb221640 1 --2- 172.21.15.115:0/3107576859 >> v2:172.21.15.142:3300/0 conn(0x7fabd415d220 0x7fabd40cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabdb221640 1 -- 172.21.15.115:0/3107576859 >> 172.21.15.115:0/3107576859 conn(0x7fabd40ce880 msgr2=0x7fabd40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:32.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabdb221640 1 -- 172.21.15.115:0/3107576859 shutdown_connections 2023-12-01T02:52:32.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.702+0000 7fabdb221640 1 -- 172.21.15.115:0/3107576859 wait complete. 2023-12-01T02:52:32.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.704+0000 7fabdb221640 1 Processor -- start 2023-12-01T02:52:32.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.704+0000 7fabdb221640 1 -- start start 2023-12-01T02:52:32.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.705+0000 7fabdb221640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabd40ad2b0 0x7fabd40c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.705+0000 7fabdb221640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabd40ade80 0x7fabd41e24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.705+0000 7fabdb221640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd415d220 0x7fabd41e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.705+0000 7fabdb221640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40c5530 con 0x7fabd415d220 2023-12-01T02:52:32.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.705+0000 7fabdb221640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd41e29f0 con 0x7fabd40ad2b0 2023-12-01T02:52:32.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.705+0000 7fabdb221640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40c3a30 con 0x7fabd40ade80 2023-12-01T02:52:32.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.705+0000 7fabd97be640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd415d220 0x7fabd41e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd97be640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd415d220 0x7fabd41e4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34652/0 (socket says 172.21.15.115:34652) 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd97be640 1 -- 172.21.15.115:0/1601339409 learned_addr learned my addr 172.21.15.115:0/1601339409 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd8fbd640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabd40ad2b0 0x7fabd40c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd3fff640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabd40ade80 0x7fabd41e24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd97be640 1 -- 172.21.15.115:0/1601339409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabd40ade80 msgr2=0x7fabd41e24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd97be640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabd40ade80 0x7fabd41e24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd97be640 1 -- 172.21.15.115:0/1601339409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabd40ad2b0 msgr2=0x7fabd40c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd97be640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabd40ad2b0 0x7fabd40c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd97be640 1 -- 172.21.15.115:0/1601339409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40094d0 con 0x7fabd415d220 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd3fff640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabd40ade80 0x7fabd41e24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd97be640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd415d220 0x7fabd41e4d80 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00b970 tx=0x7fabcc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.706+0000 7fabd17fa640 1 -- 172.21.15.115:0/1601339409 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00cb40 con 0x7fabd415d220 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.707+0000 7fabd17fa640 1 -- 172.21.15.115:0/1601339409 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc00ccc0 con 0x7fabd415d220 2023-12-01T02:52:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.707+0000 7fabd17fa640 1 -- 172.21.15.115:0/1601339409 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc014680 con 0x7fabd415d220 2023-12-01T02:52:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.707+0000 7fabdb221640 1 -- 172.21.15.115:0/1601339409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd40c3d10 con 0x7fabd415d220 2023-12-01T02:52:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.707+0000 7fabdb221640 1 -- 172.21.15.115:0/1601339409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd40c42d0 con 0x7fabd415d220 2023-12-01T02:52:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.708+0000 7fabdb221640 1 -- 172.21.15.115:0/1601339409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd404e890 con 0x7fabd415d220 2023-12-01T02:52:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.709+0000 7fabd17fa640 1 -- 172.21.15.115:0/1601339409 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fabcc01d3b0 con 0x7fabd415d220 2023-12-01T02:52:32.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.710+0000 7fabd17fa640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fabc006a580 0x7fabc006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:32.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.710+0000 7fabd17fa640 1 -- 172.21.15.115:0/1601339409 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fabcc088980 con 0x7fabd415d220 2023-12-01T02:52:32.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.710+0000 7fabd8fbd640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fabc006a580 0x7fabc006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.710+0000 7fabd8fbd640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fabc006a580 0x7fabc006ca40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fabc800b7c0 tx=0x7fabc8008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.714+0000 7fabd17fa640 1 -- 172.21.15.115:0/1601339409 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fabcc053ec0 con 0x7fabd415d220 2023-12-01T02:52:32.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.937+0000 7fabdb221640 1 -- 172.21.15.115:0/1601339409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabd40acb50 con 0x7fabd415d220 2023-12-01T02:52:32.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.938+0000 7fabd17fa640 1 -- 172.21.15.115:0/1601339409 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4661 (secure 0 0 0) 0x7fabcc019020 con 0x7fabd415d220 2023-12-01T02:52:32.940 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T02:52:32.940 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1616304732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1616304732},{"type":"v1","addr":"172.21.15.115:6835","nonce":1616304732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6766,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3109897397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3109897397},{"type":"v1","addr":"172.21.15.142:6834","nonce":3109897397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/446557816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":446557816},{"type":"v1","addr":"172.21.15.142:6835","nonce":446557816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:52:21.959726+0000","modified":"2023-12-01T02:52:24.991376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9549":{"gid":9549,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1064849689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1064849689},{"type":"v1","addr":"172.21.15.115:6837","nonce":1064849689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-01T02:52:32.941 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 79 2023-12-01T02:52:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 -- 172.21.15.115:0/1601339409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fabc006a580 msgr2=0x7fabc006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fabc006a580 0x7fabc006ca40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fabc800b7c0 tx=0x7fabc8008040 comp rx=0 tx=0).stop 2023-12-01T02:52:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 -- 172.21.15.115:0/1601339409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd415d220 msgr2=0x7fabd41e4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd415d220 0x7fabd41e4d80 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00b970 tx=0x7fabcc00be30 comp rx=0 tx=0).stop 2023-12-01T02:52:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 -- 172.21.15.115:0/1601339409 shutdown_connections 2023-12-01T02:52:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fabc006a580 0x7fabc006ca40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabd40ade80 0x7fabd41e24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabd415d220 0x7fabd41e4d80 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 --2- 172.21.15.115:0/1601339409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabd40ad2b0 0x7fabd40c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:32.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 -- 172.21.15.115:0/1601339409 >> 172.21.15.115:0/1601339409 conn(0x7fabd40ce880 msgr2=0x7fabd40af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:32.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 -- 172.21.15.115:0/1601339409 shutdown_connections 2023-12-01T02:52:32.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:32.941+0000 7fabdb221640 1 -- 172.21.15.115:0/1601339409 wait complete. 2023-12-01T02:52:33.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T02:52:21.959726+0000', 'modified': '2023-12-01T02:52:24.991376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9549}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9549': {'gid': 9549, 'name': 'c', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.115:6837/1064849689', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6836', 'nonce': 1064849689}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 1064849689}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T02:52:33.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T02:52:33.365 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-01T02:52:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.590+0000 7f14a49c3640 1 Processor -- start 2023-12-01T02:52:33.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.591+0000 7f14a49c3640 1 -- start start 2023-12-01T02:52:33.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.592+0000 7f14a49c3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a015cf90 0x7f14a00d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.592+0000 7f14a49c3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f14a00b8610 0x7f14a00cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.592+0000 7f14a49c3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a015d370 con 0x7f14a00cdf00 2023-12-01T02:52:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.592+0000 7f14a49c3640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a00ce390 con 0x7f14a00b8610 2023-12-01T02:52:33.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.592+0000 7f14a49c3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a00d19a0 con 0x7f14a015cf90 2023-12-01T02:52:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.592+0000 7f149dd9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f14a00b8610 0x7f14a00cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.593+0000 7f149dd9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f14a00b8610 0x7f14a00cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34666/0 (socket says 172.21.15.115:34666) 2023-12-01T02:52:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.593+0000 7f149dd9b640 1 -- 172.21.15.115:0/1875967609 learned_addr learned my addr 172.21.15.115:0/1875967609 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.593+0000 7f149e59c640 1 --2- 172.21.15.115:0/1875967609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a015cf90 0x7f14a00d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.593+0000 7f149dd9b640 1 -- 172.21.15.115:0/1875967609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a015cf90 msgr2=0x7f14a00d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.593+0000 7f149dd9b640 1 --2- 172.21.15.115:0/1875967609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a015cf90 0x7f14a00d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.593+0000 7f149dd9b640 1 -- 172.21.15.115:0/1875967609 >> v1:172.21.15.142:6789/0 conn(0x7f14a00cdf00 legacy=0x7f14a00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.593+0000 7f149dd9b640 1 -- 172.21.15.115:0/1875967609 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a00d1c80 con 0x7f14a00b8610 2023-12-01T02:52:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.594+0000 7f149dd9b640 1 --2- 172.21.15.115:0/1875967609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 0x7f14a00cd9c0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f1488009870 tx=0x7f148802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed88031dc2ecc4cf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f149d59a640 1 -- 172.21.15.115:0/1875967609 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488031640 con 0x7f14a00b8610 2023-12-01T02:52:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f149d59a640 1 -- 172.21.15.115:0/1875967609 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14880317a0 con 0x7f14a00b8610 2023-12-01T02:52:33.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f149d59a640 1 -- 172.21.15.115:0/1875967609 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488031b30 con 0x7f14a00b8610 2023-12-01T02:52:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f14a49c3640 1 -- 172.21.15.115:0/1875967609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 msgr2=0x7f14a00cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f14a49c3640 1 --2- 172.21.15.115:0/1875967609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 0x7f14a00cd9c0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f1488009870 tx=0x7f148802b8f0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f14a49c3640 1 -- 172.21.15.115:0/1875967609 shutdown_connections 2023-12-01T02:52:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f14a49c3640 1 --2- 172.21.15.115:0/1875967609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 0x7f14a00cd9c0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f14a49c3640 1 --2- 172.21.15.115:0/1875967609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a015cf90 0x7f14a00d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f14a49c3640 1 -- 172.21.15.115:0/1875967609 >> 172.21.15.115:0/1875967609 conn(0x7f14a0065960 msgr2=0x7f14a0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f14a49c3640 1 -- 172.21.15.115:0/1875967609 shutdown_connections 2023-12-01T02:52:33.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.595+0000 7f14a49c3640 1 -- 172.21.15.115:0/1875967609 wait complete. 2023-12-01T02:52:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.597+0000 7f14a49c3640 1 Processor -- start 2023-12-01T02:52:33.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.597+0000 7f14a49c3640 1 -- start start 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.598+0000 7f14a49c3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 0x7f14a014e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.598+0000 7f14a49c3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a00cdf00 0x7f14a0151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.598+0000 7f14a49c3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a015cf90 0x7f14a00b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.598+0000 7f14a49c3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a00b4600 con 0x7f14a00b8610 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.598+0000 7f14a49c3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a015d560 con 0x7f14a015cf90 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.598+0000 7f14a49c3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a015e3c0 con 0x7f14a00cdf00 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 0x7f14a014e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 0x7f14a014e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34672/0 (socket says 172.21.15.115:34672) 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149e59c640 1 -- 172.21.15.115:0/320334881 learned_addr learned my addr 172.21.15.115:0/320334881 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149e59c640 1 -- 172.21.15.115:0/320334881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a00cdf00 msgr2=0x7f14a0151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149dd9b640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a00cdf00 0x7f14a0151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149e59c640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a00cdf00 0x7f14a0151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149e59c640 1 -- 172.21.15.115:0/320334881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a015cf90 msgr2=0x7f14a00b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149e59c640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a015cf90 0x7f14a00b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149e59c640 1 -- 172.21.15.115:0/320334881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880094d0 con 0x7f14a00b8610 2023-12-01T02:52:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149dd9b640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a00cdf00 0x7f14a0151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.599+0000 7f149e59c640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 0x7f14a014e860 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f149000d620 tx=0x7f149000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:33.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.600+0000 7f1496ffd640 1 -- 172.21.15.115:0/320334881 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149000b620 con 0x7f14a00b8610 2023-12-01T02:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.600+0000 7f1496ffd640 1 -- 172.21.15.115:0/320334881 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149000b780 con 0x7f14a00b8610 2023-12-01T02:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.600+0000 7f14a49c3640 1 -- 172.21.15.115:0/320334881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a015d7b0 con 0x7f14a00b8610 2023-12-01T02:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.600+0000 7f1496ffd640 1 -- 172.21.15.115:0/320334881 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149000bb30 con 0x7f14a00b8610 2023-12-01T02:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.600+0000 7f14a49c3640 1 -- 172.21.15.115:0/320334881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a015dd70 con 0x7f14a00b8610 2023-12-01T02:52:33.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.602+0000 7f1496ffd640 1 -- 172.21.15.115:0/320334881 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f149000bc90 con 0x7f14a00b8610 2023-12-01T02:52:33.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.603+0000 7f1496ffd640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148006a580 0x7f148006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.603+0000 7f1496ffd640 1 -- 172.21.15.115:0/320334881 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f1490088fe0 con 0x7f14a00b8610 2023-12-01T02:52:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.603+0000 7f149dd9b640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148006a580 0x7f148006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.603+0000 7f14a760c740 1 Processor -- start 2023-12-01T02:52:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.604+0000 7f14a760c740 1 -- start start 2023-12-01T02:52:33.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.604+0000 7f149dd9b640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148006a580 0x7f148006ca40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1488036000 tx=0x7f148803c040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f14a760c740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x560624481d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f14a760c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244518a0 0x56062447eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f14a760c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062447f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f14a760c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062442d2f0 con 0x5606244518a0 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f14a760c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606243c1f80 con 0x5606244554a0 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f14a760c740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606243bf710 con 0x560624481920 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244518a0 0x56062447eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244518a0 0x56062447eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34680/0 (socket says 172.21.15.115:34680) 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149dd9b640 1 -- 172.21.15.115:0/1862330812 learned_addr learned my addr 172.21.15.115:0/1862330812 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149e59c640 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x560624481d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149ed9d640 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062447f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149dd9b640 1 -- 172.21.15.115:0/1862330812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 msgr2=0x560624481d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149dd9b640 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x560624481d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149dd9b640 1 -- 172.21.15.115:0/1862330812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 msgr2=0x56062447f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149dd9b640 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062447f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149dd9b640 1 -- 172.21.15.115:0/1862330812 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560624382690 con 0x5606244518a0 2023-12-01T02:52:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149e59c640 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x560624481d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.605+0000 7f149ed9d640 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062447f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.606+0000 7f149dd9b640 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244518a0 0x56062447eb90 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f1488009960 tx=0x7f1488030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269ec1b5e704cb86 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.606+0000 7f147ffff640 1 -- 172.21.15.115:0/1862330812 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148803a6a0 con 0x5606244518a0 2023-12-01T02:52:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.606+0000 7f147ffff640 1 -- 172.21.15.115:0/1862330812 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148803a800 con 0x5606244518a0 2023-12-01T02:52:33.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.606+0000 7f147ffff640 1 -- 172.21.15.115:0/1862330812 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148803ab70 con 0x5606244518a0 2023-12-01T02:52:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.607+0000 7f14a760c740 1 -- 172.21.15.115:0/1862330812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244518a0 msgr2=0x56062447eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.607+0000 7f14a760c740 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244518a0 0x56062447eb90 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f1488009960 tx=0x7f1488030000 comp rx=0 tx=0).stop 2023-12-01T02:52:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.607+0000 7f14a760c740 1 -- 172.21.15.115:0/1862330812 shutdown_connections 2023-12-01T02:52:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.607+0000 7f14a760c740 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062447f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.607+0000 7f14a760c740 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244518a0 0x56062447eb90 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.607+0000 7f14a760c740 1 --2- 172.21.15.115:0/1862330812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x560624481d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.607+0000 7f14a760c740 1 -- 172.21.15.115:0/1862330812 >> 172.21.15.115:0/1862330812 conn(0x560624453bb0 msgr2=0x56062449c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.607+0000 7f14a760c740 1 -- 172.21.15.115:0/1862330812 shutdown_connections 2023-12-01T02:52:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.607+0000 7f14a760c740 1 -- 172.21.15.115:0/1862330812 wait complete. 2023-12-01T02:52:33.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.607+0000 7f14a760c740 1 -- start start 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.608+0000 7f14a760c740 1 Processor -- start 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f14a760c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062447b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f14a760c740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x56062447b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f14a760c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624467940 0x560624467dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f14a760c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606243bb000 con 0x560624467940 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f14a760c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062435c7d0 con 0x5606244554a0 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f14a760c740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x56062435c470 con 0x560624481920 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f14a760c740 10 client.0 _ll_register_callbacks cb 0x5606243ab1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f149ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624467940 0x560624467dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f149ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624467940 0x560624467dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34690/0 (socket says 172.21.15.115:34690) 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f149ed9d640 1 -- 172.21.15.115:0/1554321196 learned_addr learned my addr 172.21.15.115:0/1554321196 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f149e59c640 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062447b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f149dd9b640 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x56062447b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f149ed9d640 1 -- 172.21.15.115:0/1554321196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 msgr2=0x56062447b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f149ed9d640 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x56062447b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f149ed9d640 1 -- 172.21.15.115:0/1554321196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 msgr2=0x56062447b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f149ed9d640 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062447b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.609+0000 7f149ed9d640 1 -- 172.21.15.115:0/1554321196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a00d1c80 con 0x560624467940 2023-12-01T02:52:33.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.610+0000 7f149ed9d640 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624467940 0x560624467dc0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f149800d4b0 tx=0x7f149800d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ea956136407a34 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.610+0000 7f147d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.610+0000 7f147d7fa640 1 -- 172.21.15.115:0/1554321196 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498012800 con 0x560624467940 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.610+0000 7f147d7fa640 1 -- 172.21.15.115:0/1554321196 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f1498012960 con 0x560624467940 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.610+0000 7f14a760c740 1 -- 172.21.15.115:0/1554321196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606243e5550 con 0x560624467940 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.610+0000 7f147d7fa640 1 -- 172.21.15.115:0/1554321196 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1498006b50 con 0x560624467940 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.610+0000 7f14a760c740 10 client.9690 fetch_fsmap learned FSMap version 79 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.610+0000 7f14a760c740 1 -- 172.21.15.115:0/1554321196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5606243b9680 con 0x560624467940 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.610+0000 7f147d7fa640 1 -- 172.21.15.115:0/1554321196 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5606243e5550 con 0x560624467940 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.610+0000 7f147d7fa640 1 -- 172.21.15.115:0/1554321196 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f1498006cb0 con 0x560624467940 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.611+0000 7f14a760c740 10 client.9690 fetch_fsmap finished waiting for FSMap version 79 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.611+0000 7f14a760c740 10 client.9690 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.611+0000 7f14a760c740 20 client.9690 populate_metadata read hostname 'smithi115' 2023-12-01T02:52:33.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.611+0000 7f14a760c740 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.115:6836/1064849689,v1:172.21.15.115:6837/1064849689] conn(0x5606244b5550 0x5606244b9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.611+0000 7f14a760c740 4 client.9690 mds_command: new command op to 9549 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.611+0000 7f14a760c740 1 -- 172.21.15.115:0/1554321196 --> [v2:172.21.15.115:6836/1064849689,v1:172.21.15.115:6837/1064849689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606243adb50 con 0x5606244b5550 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.611+0000 7f149e59c640 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.115:6836/1064849689,v1:172.21.15.115:6837/1064849689] conn(0x5606244b5550 0x5606244b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.611+0000 7f149e59c640 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.115:6836/1064849689,v1:172.21.15.115:6837/1064849689] conn(0x5606244b5550 0x5606244b9440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866c908f695d8d69 server_cookie=f700c0e6c8f1c746 in_seq=0 out_seq=0 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.611+0000 7f147d7fa640 10 client.9690 ms_handle_connect on v2:172.21.15.115:6836/1064849689 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f147d7fa640 1 -- 172.21.15.115:0/1554321196 <== mds.0 v2:172.21.15.115:6836/1064849689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5606243adb50 con 0x5606244b5550 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f147d7fa640 10 client.9690 handle_command_reply: tid=0 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f14a760c740 1 client.9690 shutdown 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f14a760c740 1 -- 172.21.15.115:0/1554321196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624467940 msgr2=0x560624467dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f14a760c740 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624467940 0x560624467dc0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f149800d4b0 tx=0x7f149800d970 comp rx=0 tx=0).stop 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f14a760c740 1 -- 172.21.15.115:0/1554321196 shutdown_connections 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f14a760c740 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.115:6836/1064849689,v1:172.21.15.115:6837/1064849689] conn(0x5606244b5550 0x5606244b9440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f14a760c740 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x56062447b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f14a760c740 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624467940 0x560624467dc0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f14a760c740 1 --2- 172.21.15.115:0/1554321196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062447b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f147d7fa640 0 client.9690 ms_handle_reset on v2:172.21.15.115:6836/1064849689 2023-12-01T02:52:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.612+0000 7f14a760c740 1 -- 172.21.15.115:0/1554321196 >> 172.21.15.115:0/1554321196 conn(0x5606244518a0 msgr2=0x56062449fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.613+0000 7f14a760c740 1 -- 172.21.15.115:0/1554321196 shutdown_connections 2023-12-01T02:52:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.613+0000 7f14a760c740 1 -- 172.21.15.115:0/1554321196 wait complete. 2023-12-01T02:52:33.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.613+0000 7f14a760c740 20 client.9690 trim_cache size 0 max 16384 2023-12-01T02:52:33.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.628+0000 7f14a760c740 1 Processor -- start 2023-12-01T02:52:33.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.629+0000 7f14a760c740 1 -- start start 2023-12-01T02:52:33.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.629+0000 7f14a760c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244554a0 0x560624475e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.629+0000 7f14a760c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560624467940 0x56062449ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.629+0000 7f14a760c740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x56062449f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.629+0000 7f14a760c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606244520a0 con 0x5606244554a0 2023-12-01T02:52:33.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.629+0000 7f14a760c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560624122f20 con 0x560624467940 2023-12-01T02:52:33.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.629+0000 7f14a760c740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x56062435b7b0 con 0x560624481920 2023-12-01T02:52:33.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f149e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244554a0 0x560624475e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f149e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244554a0 0x560624475e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34704/0 (socket says 172.21.15.115:34704) 2023-12-01T02:52:33.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f149e59c640 1 -- 172.21.15.115:0/1785274945 learned_addr learned my addr 172.21.15.115:0/1785274945 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:33.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f149e59c640 1 -- 172.21.15.115:0/1785274945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 msgr2=0x56062449f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f149e59c640 1 --2- 172.21.15.115:0/1785274945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x56062449f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f149e59c640 1 -- 172.21.15.115:0/1785274945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560624467940 msgr2=0x56062449ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f149e59c640 1 --2- 172.21.15.115:0/1785274945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560624467940 0x56062449ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f149e59c640 1 -- 172.21.15.115:0/1785274945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56062435b9e0 con 0x5606244554a0 2023-12-01T02:52:33.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f149e59c640 1 --2- 172.21.15.115:0/1785274945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244554a0 0x560624475e00 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f149000b320 tx=0x7f1490057cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccf134daabe98e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:33.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f1494ff9640 1 -- 172.21.15.115:0/1785274945 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149005b040 con 0x5606244554a0 2023-12-01T02:52:33.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f1494ff9640 1 -- 172.21.15.115:0/1785274945 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1490060090 con 0x5606244554a0 2023-12-01T02:52:33.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.630+0000 7f1494ff9640 1 -- 172.21.15.115:0/1785274945 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14900547c0 con 0x5606244554a0 2023-12-01T02:52:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.631+0000 7f14a760c740 1 -- 172.21.15.115:0/1785274945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244554a0 msgr2=0x560624475e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.631+0000 7f14a760c740 1 --2- 172.21.15.115:0/1785274945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244554a0 0x560624475e00 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f149000b320 tx=0x7f1490057cb0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.631+0000 7f14a760c740 1 -- 172.21.15.115:0/1785274945 shutdown_connections 2023-12-01T02:52:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.631+0000 7f14a760c740 1 --2- 172.21.15.115:0/1785274945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624481920 0x56062449f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.631+0000 7f14a760c740 1 --2- 172.21.15.115:0/1785274945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560624467940 0x56062449ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.631+0000 7f14a760c740 1 --2- 172.21.15.115:0/1785274945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5606244554a0 0x560624475e00 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.631+0000 7f14a760c740 1 -- 172.21.15.115:0/1785274945 >> 172.21.15.115:0/1785274945 conn(0x5606244518a0 msgr2=0x560624450920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.631+0000 7f14a760c740 1 -- 172.21.15.115:0/1785274945 shutdown_connections 2023-12-01T02:52:33.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.631+0000 7f14a760c740 1 -- 172.21.15.115:0/1785274945 wait complete. 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.632+0000 7f14a760c740 1 -- start start 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.632+0000 7f14a760c740 1 Processor -- start 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f14a760c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062449cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f14a760c740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624467940 0x5606244aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f14a760c740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624481920 0x5606244ab390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f14a760c740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606244abd00 con 0x560624481920 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f14a760c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560624506150 con 0x5606244554a0 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f14a760c740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x560624484410 con 0x560624467940 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f14a760c740 10 client.0 _ll_register_callbacks cb 0x7f14880096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f149ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624481920 0x5606244ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f149ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624481920 0x5606244ab390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34706/0 (socket says 172.21.15.115:34706) 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f149ed9d640 1 -- 172.21.15.115:0/3676033917 learned_addr learned my addr 172.21.15.115:0/3676033917 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:33.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f149dd9b640 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624467940 0x5606244aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.633+0000 7f149e59c640 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062449cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f149ed9d640 1 -- 172.21.15.115:0/3676033917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624467940 msgr2=0x5606244aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f149ed9d640 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624467940 0x5606244aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f149ed9d640 1 -- 172.21.15.115:0/3676033917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 msgr2=0x56062449cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f149ed9d640 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062449cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f149ed9d640 1 -- 172.21.15.115:0/3676033917 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a015dd70 con 0x560624481920 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f149dd9b640 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624467940 0x5606244aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f149e59c640 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062449cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f149ed9d640 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624481920 0x5606244ab390 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f1498000f80 tx=0x7f1498003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adb73510f7432cd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f147d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f147d7fa640 1 -- 172.21.15.115:0/3676033917 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498007df0 con 0x560624481920 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f147d7fa640 1 -- 172.21.15.115:0/3676033917 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f149800e000 con 0x560624481920 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f14a760c740 1 -- 172.21.15.115:0/3676033917 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f148803a800 con 0x560624481920 2023-12-01T02:52:33.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f147d7fa640 1 -- 172.21.15.115:0/3676033917 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1498012c60 con 0x560624481920 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f14a760c740 10 client.9696 fetch_fsmap learned FSMap version 79 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.634+0000 7f14a760c740 1 -- 172.21.15.115:0/3676033917 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x560624380ef0 con 0x560624481920 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.635+0000 7f147d7fa640 1 -- 172.21.15.115:0/3676033917 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148803a800 con 0x560624481920 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.635+0000 7f147d7fa640 1 -- 172.21.15.115:0/3676033917 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 79) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f1498007480 con 0x560624481920 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.635+0000 7f14a760c740 10 client.9696 fetch_fsmap finished waiting for FSMap version 79 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.635+0000 7f14a760c740 10 client.9696 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.635+0000 7f14a760c740 20 client.9696 populate_metadata read hostname 'smithi115' 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.635+0000 7f14a760c740 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.115:6836/1064849689,v1:172.21.15.115:6837/1064849689] conn(0x560624483990 0x56062449de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.635+0000 7f14a760c740 4 client.9696 mds_command: new command op to 9549 tid=0[{"prefix": "status"}] 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.635+0000 7f14a760c740 1 -- 172.21.15.115:0/3676033917 --> [v2:172.21.15.115:6836/1064849689,v1:172.21.15.115:6837/1064849689] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5606243b4080 con 0x560624483990 2023-12-01T02:52:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.635+0000 7f149e59c640 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.115:6836/1064849689,v1:172.21.15.115:6837/1064849689] conn(0x560624483990 0x56062449de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f149e59c640 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.115:6836/1064849689,v1:172.21.15.115:6837/1064849689] conn(0x560624483990 0x56062449de80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ce28584c3d4b1a server_cookie=b4f4efc5a5822665 in_seq=0 out_seq=0 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f147d7fa640 10 client.9696 ms_handle_connect on v2:172.21.15.115:6836/1064849689 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f147d7fa640 1 -- 172.21.15.115:0/3676033917 <== mds.0 v2:172.21.15.115:6836/1064849689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5606243b4080 con 0x560624483990 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f147d7fa640 10 client.9696 handle_command_reply: tid=0 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 client.9696 shutdown 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 -- 172.21.15.115:0/3676033917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624481920 msgr2=0x5606244ab390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624481920 0x5606244ab390 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f1498000f80 tx=0x7f1498003b50 comp rx=0 tx=0).stop 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 -- 172.21.15.115:0/3676033917 shutdown_connections 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.115:6836/1064849689,v1:172.21.15.115:6837/1064849689] conn(0x560624483990 0x56062449de80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x560624467940 0x5606244aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x560624481920 0x5606244ab390 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 --2- 172.21.15.115:0/3676033917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606244554a0 0x56062449cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 -- 172.21.15.115:0/3676033917 >> 172.21.15.115:0/3676033917 conn(0x5606244518a0 msgr2=0x56062449ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f147d7fa640 0 client.9696 ms_handle_reset on v2:172.21.15.115:6836/1064849689 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 -- 172.21.15.115:0/3676033917 shutdown_connections 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.636+0000 7f14a760c740 1 -- 172.21.15.115:0/3676033917 wait complete. 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a760c740 20 client.9696 trim_cache size 0 max 16384 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 9549, 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T02:52:33.642 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T02:52:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 11.667591994, 2023-12-01T02:52:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 79, 2023-12-01T02:52:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 177, 2023-12-01T02:52:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 177, 2023-12-01T02:52:33.643 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 16.957609808000001 2023-12-01T02:52:33.643 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 -- 172.21.15.115:0/320334881 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148006a580 msgr2=0x7f148006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148006a580 0x7f148006ca40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1488036000 tx=0x7f148803c040 comp rx=0 tx=0).stop 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 -- 172.21.15.115:0/320334881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 msgr2=0x7f14a014e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 0x7f14a014e860 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f149000d620 tx=0x7f149000dae0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 -- 172.21.15.115:0/320334881 shutdown_connections 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f148006a580 0x7f148006ca40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14a00cdf00 0x7f14a0151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14a015cf90 0x7f14a00b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 --2- 172.21.15.115:0/320334881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14a00b8610 0x7f14a014e860 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 -- 172.21.15.115:0/320334881 >> 172.21.15.115:0/320334881 conn(0x7f14a0065960 msgr2=0x7f14a00cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 -- 172.21.15.115:0/320334881 shutdown_connections 2023-12-01T02:52:33.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.637+0000 7f14a49c3640 1 -- 172.21.15.115:0/320334881 wait complete. 2023-12-01T02:52:33.655 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T02:52:33.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.859+0000 7fd87bd20640 1 Processor -- start 2023-12-01T02:52:33.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.859+0000 7fd87bd20640 1 -- start start 2023-12-01T02:52:33.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.860+0000 7fd87bd20640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd87415e2f0 0x7fd8740ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.860+0000 7fd87bd20640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd87406f2f0 0x7fd8740caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.860+0000 7fd87bd20640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8740add10 con 0x7fd8740adf10 2023-12-01T02:52:33.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.860+0000 7fd87bd20640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8740c5760 con 0x7fd87406f2f0 2023-12-01T02:52:33.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.860+0000 7fd87bd20640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87415e6d0 con 0x7fd87415e2f0 2023-12-01T02:52:33.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.860+0000 7fd87a2bd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd8740adf10 0x7fd8740c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59220/0 (socket says 172.21.15.115:59220) 2023-12-01T02:52:33.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.860+0000 7fd87a2bd640 1 -- 172.21.15.115:0/4109448372 learned_addr learned my addr 172.21.15.115:0/4109448372 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:33.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd879abc640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd87415e2f0 0x7fd8740ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527693021 0 0) 0x7fd8740add10 con 0x7fd8740adf10 2023-12-01T02:52:33.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd864003520 con 0x7fd8740adf10 2023-12-01T02:52:33.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd8792bb640 1 --2- 172.21.15.115:0/4109448372 >> v2:172.21.15.142:3300/0 conn(0x7fd87406f2f0 0x7fd8740caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 755500722 0 0) 0x7fd864003520 con 0x7fd8740adf10 2023-12-01T02:52:33.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd87415e2f0 msgr2=0x7fd8740ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd87415e2f0 0x7fd8740ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 >> v2:172.21.15.142:3300/0 conn(0x7fd87406f2f0 msgr2=0x7fd8740caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 --2- 172.21.15.115:0/4109448372 >> v2:172.21.15.142:3300/0 conn(0x7fd87406f2f0 0x7fd8740caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8740cf7b0 con 0x7fd8740adf10 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd86c001670 con 0x7fd8740adf10 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 >> v1:172.21.15.115:6789/0 conn(0x7fd8740adf10 legacy=0x7fd8740c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd87415e2f0 0x7fd8640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd864004010 0x7fd8640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 0x7fd864005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8740c5760 con 0x7fd864004d00 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87415e6d0 con 0x7fd864004010 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86400bc40 con 0x7fd87415e2f0 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.861+0000 7fd87a2bd640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 0x7fd864005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.862+0000 7fd879abc640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd87415e2f0 0x7fd8640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.862+0000 7fd87a2bd640 1 -- 172.21.15.115:0/4109448372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd87415e2f0 msgr2=0x7fd8640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.862+0000 7fd87a2bd640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd87415e2f0 0x7fd8640039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.862+0000 7fd87a2bd640 1 -- 172.21.15.115:0/4109448372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd864004010 msgr2=0x7fd8640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:33.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.862+0000 7fd87a2bd640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd864004010 0x7fd8640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.862+0000 7fd87a2bd640 1 -- 172.21.15.115:0/4109448372 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd864003520 con 0x7fd864004d00 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.863+0000 7fd87a2bd640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 0x7fd864005180 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fd86c007cd0 tx=0x7fd8640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a058563d8dd2470 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.863+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0058d0 con 0x7fd864004d00 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.863+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86c005a30 con 0x7fd864004d00 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.863+0000 7fd878aba640 1 -- 172.21.15.115:0/4109448372 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c005e40 con 0x7fd864004d00 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.864+0000 7fd87bd20640 1 -- 172.21.15.115:0/4109448372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 msgr2=0x7fd864005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.864+0000 7fd87bd20640 1 --2- 172.21.15.115:0/4109448372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 0x7fd864005180 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fd86c007cd0 tx=0x7fd8640057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.864+0000 7fd87a2bd640 1 -- 172.21.15.115:0/4109448372 reap_dead start 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.864+0000 7fd87bd20640 1 -- 172.21.15.115:0/4109448372 shutdown_connections 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.864+0000 7fd87bd20640 1 -- 172.21.15.115:0/4109448372 >> 172.21.15.115:0/4109448372 conn(0x7fd8740ce880 msgr2=0x7fd8740cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.864+0000 7fd87bd20640 1 -- 172.21.15.115:0/4109448372 shutdown_connections 2023-12-01T02:52:33.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.864+0000 7fd87bd20640 1 -- 172.21.15.115:0/4109448372 wait complete. 2023-12-01T02:52:33.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.866+0000 7fd87bd20640 1 Processor -- start 2023-12-01T02:52:33.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.866+0000 7fd87bd20640 1 -- start start 2023-12-01T02:52:33.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.867+0000 7fd87bd20640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 0x7fd8740c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.867+0000 7fd87bd20640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8740c4320 0x7fd87415be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.867+0000 7fd87bd20640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8740c47a0 0x7fd87415c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.867+0000 7fd87bd20640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8740c87c0 con 0x7fd864004d00 2023-12-01T02:52:33.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.867+0000 7fd87bd20640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8740a85a0 con 0x7fd8740c47a0 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.867+0000 7fd87bd20640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8740c78b0 con 0x7fd8740c4320 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.867+0000 7fd879abc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 0x7fd8740c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.867+0000 7fd879abc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 0x7fd8740c7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34726/0 (socket says 172.21.15.115:34726) 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.867+0000 7fd879abc640 1 -- 172.21.15.115:0/3388593231 learned_addr learned my addr 172.21.15.115:0/3388593231 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.868+0000 7fd879abc640 1 -- 172.21.15.115:0/3388593231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8740c4320 msgr2=0x7fd87415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.868+0000 7fd879abc640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8740c4320 0x7fd87415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.868+0000 7fd879abc640 1 -- 172.21.15.115:0/3388593231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8740c47a0 msgr2=0x7fd87415c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.868+0000 7fd879abc640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8740c47a0 0x7fd87415c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.868+0000 7fd879abc640 1 -- 172.21.15.115:0/3388593231 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8740c5950 con 0x7fd864004d00 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.868+0000 7fd879abc640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 0x7fd8740c7370 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fd868007e60 tx=0x7fd86800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.868+0000 7fd8727fc640 1 -- 172.21.15.115:0/3388593231 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680143f0 con 0x7fd864004d00 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.868+0000 7fd8727fc640 1 -- 172.21.15.115:0/3388593231 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd868014550 con 0x7fd864004d00 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.869+0000 7fd87bd20640 1 -- 172.21.15.115:0/3388593231 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8740c5c10 con 0x7fd864004d00 2023-12-01T02:52:33.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.869+0000 7fd8727fc640 1 -- 172.21.15.115:0/3388593231 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680148c0 con 0x7fd864004d00 2023-12-01T02:52:33.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.869+0000 7fd87bd20640 1 -- 172.21.15.115:0/3388593231 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8740c61d0 con 0x7fd864004d00 2023-12-01T02:52:33.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.870+0000 7fd87bd20640 1 -- 172.21.15.115:0/3388593231 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87406fda0 con 0x7fd864004d00 2023-12-01T02:52:33.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.871+0000 7fd8727fc640 1 -- 172.21.15.115:0/3388593231 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd868014a20 con 0x7fd864004d00 2023-12-01T02:52:33.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.872+0000 7fd8727fc640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd85806a550 0x7fd85806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:33.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.872+0000 7fd8727fc640 1 -- 172.21.15.115:0/3388593231 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fd8680899c0 con 0x7fd864004d00 2023-12-01T02:52:33.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.872+0000 7fd8792bb640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd85806a550 0x7fd85806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:33.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.873+0000 7fd8792bb640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd85806a550 0x7fd85806ca10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8740c53d0 tx=0x7fd860008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:33.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:33.876+0000 7fd8727fc640 1 -- 172.21.15.115:0/3388593231 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd868054cc0 con 0x7fd864004d00 2023-12-01T02:52:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.106+0000 7fd87bd20640 1 -- 172.21.15.115:0/3388593231 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd87404e890 con 0x7fd864004d00 2023-12-01T02:52:34.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.107+0000 7fd8727fc640 1 -- 172.21.15.115:0/3388593231 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd8680582e0 con 0x7fd864004d00 2023-12-01T02:52:34.109 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.109+0000 7fd87bd20640 1 -- 172.21.15.115:0/3388593231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd85806a550 msgr2=0x7fd85806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.109+0000 7fd87bd20640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd85806a550 0x7fd85806ca10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8740c53d0 tx=0x7fd860008040 comp rx=0 tx=0).stop 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.110+0000 7fd87bd20640 1 -- 172.21.15.115:0/3388593231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 msgr2=0x7fd8740c7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.110+0000 7fd87bd20640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 0x7fd8740c7370 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fd868007e60 tx=0x7fd86800f520 comp rx=0 tx=0).stop 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.110+0000 7fd87bd20640 1 -- 172.21.15.115:0/3388593231 shutdown_connections 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.110+0000 7fd87bd20640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fd85806a550 0x7fd85806ca10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.110+0000 7fd87bd20640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8740c4320 0x7fd87415be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.110+0000 7fd87bd20640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8740c47a0 0x7fd87415c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.110+0000 7fd87bd20640 1 --2- 172.21.15.115:0/3388593231 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd864004d00 0x7fd8740c7370 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.110+0000 7fd87bd20640 1 -- 172.21.15.115:0/3388593231 >> 172.21.15.115:0/3388593231 conn(0x7fd8740ce880 msgr2=0x7fd87415d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.110+0000 7fd87bd20640 1 -- 172.21.15.115:0/3388593231 shutdown_connections 2023-12-01T02:52:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:34.110+0000 7fd87bd20640 1 -- 172.21.15.115:0/3388593231 wait complete. 2023-12-01T02:52:34.131 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T02:52:34.131 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T02:52:34.131 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T02:52:34.131 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T02:52:34.131 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T02:52:34.131 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T02:52:34.131 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T02:52:34.131 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T02:52:34.131 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T02:52:34.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:34.132 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T02:52:34.186 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T02:52:34.186 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T02:52:34.186 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T02:52:34.186 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:52:34.186 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 37803sec preferred_lft 37803sec 2023-12-01T02:52:34.187 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T02:52:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T02:52:34.189 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T02:52:34.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:34.190 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T02:52:34.280 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T02:52:34.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:34.285 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T02:52:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T02:52:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T02:52:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T02:52:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T02:52:34.338 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T02:52:34.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:34.338 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T02:52:34.338 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T02:52:34.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T02:52:34.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T02:52:34.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T02:52:34.338 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T02:52:34.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:34.524 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T02:52:34.575 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T02:52:34.575 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T02:52:34.575 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T02:52:34.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:34.575 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T02:52:34.670 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T02:52:34.670 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T02:52:34.726 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T02:52:34.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T02:52:34.726 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T02:52:34.777 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T02:52:34.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:34.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T02:52:34.920 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T02:52:34.920 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T02:52:34.921 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T02:52:34.942 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T02:52:34.942 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T02:52:34.942 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T02:52:34.942 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T02:52:34.942 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T02:52:34.969 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T02:52:34.969 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T02:52:34.969 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T02:52:34.969 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T02:52:34.969 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T02:52:34.969 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T02:52:34.969 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T02:52:34.970 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T02:52:34.971 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T02:52:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T02:52:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T02:52:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T02:52:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T02:52:34.975 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T02:52:34.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:34.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T02:52:35.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:35.060 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T02:52:35.119 INFO:teuthology.orchestra.run.smithi115.stdout:client6832 2023-12-01T02:52:35.119 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T02:52:35.119 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T02:52:35.175 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T02:52:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.381+0000 7f9292518640 1 Processor -- start 2023-12-01T02:52:35.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.382+0000 7f9292518640 1 -- start start 2023-12-01T02:52:35.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.383+0000 7f9292518640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f928c15e2f0 0x7f928c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.383+0000 7f9292518640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f928c0adf10 0x7f928c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.383+0000 7f9292518640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928c0add10 con 0x7f928c06f2f0 2023-12-01T02:52:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.383+0000 7f9292518640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928c0c5760 con 0x7f928c0adf10 2023-12-01T02:52:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.383+0000 7f9292518640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928c15e6d0 con 0x7f928c15e2f0 2023-12-01T02:52:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.383+0000 7f928b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f928c06f2f0 0x7f928c0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59224/0 (socket says 172.21.15.115:59224) 2023-12-01T02:52:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.383+0000 7f928b7fe640 1 -- 172.21.15.115:0/1212974322 learned_addr learned my addr 172.21.15.115:0/1212974322 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.383+0000 7f9290ab5640 1 --2- 172.21.15.115:0/1212974322 >> v2:172.21.15.115:3300/0 conn(0x7f928c0adf10 0x7f928c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.383+0000 7f928bfff640 1 --2- 172.21.15.115:0/1212974322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f928c15e2f0 0x7f928c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.384+0000 7f9290ab5640 1 -- 172.21.15.115:0/1212974322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f928c15e2f0 msgr2=0x7f928c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.384+0000 7f9290ab5640 1 --2- 172.21.15.115:0/1212974322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f928c15e2f0 0x7f928c0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.384+0000 7f9290ab5640 1 -- 172.21.15.115:0/1212974322 >> v1:172.21.15.115:6789/0 conn(0x7f928c06f2f0 legacy=0x7f928c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.384+0000 7f9290ab5640 1 -- 172.21.15.115:0/1212974322 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c0cf7b0 con 0x7f928c0adf10 2023-12-01T02:52:35.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.385+0000 7f9290ab5640 1 --2- 172.21.15.115:0/1212974322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c0adf10 0x7f928c0caba0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f92840097a0 tx=0x7f928402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3960accd0e609f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.385+0000 7f928affd640 1 -- 172.21.15.115:0/1212974322 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92840316f0 con 0x7f928c0adf10 2023-12-01T02:52:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.385+0000 7f928affd640 1 -- 172.21.15.115:0/1212974322 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9284031850 con 0x7f928c0adf10 2023-12-01T02:52:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.385+0000 7f928affd640 1 -- 172.21.15.115:0/1212974322 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284031be0 con 0x7f928c0adf10 2023-12-01T02:52:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.386+0000 7f9292518640 1 -- 172.21.15.115:0/1212974322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c0adf10 msgr2=0x7f928c0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.386+0000 7f9292518640 1 --2- 172.21.15.115:0/1212974322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c0adf10 0x7f928c0caba0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f92840097a0 tx=0x7f928402b790 comp rx=0 tx=0).stop 2023-12-01T02:52:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.386+0000 7f9292518640 1 -- 172.21.15.115:0/1212974322 shutdown_connections 2023-12-01T02:52:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.386+0000 7f9292518640 1 --2- 172.21.15.115:0/1212974322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c0adf10 0x7f928c0caba0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.386+0000 7f9292518640 1 --2- 172.21.15.115:0/1212974322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f928c15e2f0 0x7f928c0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.386+0000 7f9292518640 1 -- 172.21.15.115:0/1212974322 >> 172.21.15.115:0/1212974322 conn(0x7f928c0ce880 msgr2=0x7f928c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.386+0000 7f9292518640 1 -- 172.21.15.115:0/1212974322 shutdown_connections 2023-12-01T02:52:35.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.386+0000 7f9292518640 1 -- 172.21.15.115:0/1212974322 wait complete. 2023-12-01T02:52:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.387+0000 7f9292518640 1 Processor -- start 2023-12-01T02:52:35.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.388+0000 7f9292518640 1 -- start start 2023-12-01T02:52:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.388+0000 7f9292518640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f928c06f2f0 0x7f928c06c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.388+0000 7f9292518640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c0adf10 0x7f928c14e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.388+0000 7f9292518640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c15e2f0 0x7f928c0cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.388+0000 7f9292518640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c14e970 con 0x7f928c15e2f0 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.388+0000 7f9292518640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c14f420 con 0x7f928c0adf10 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.388+0000 7f9292518640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928c14eb80 con 0x7f928c06f2f0 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.389+0000 7f9290ab5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c15e2f0 0x7f928c0cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.389+0000 7f9290ab5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c15e2f0 0x7f928c0cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34744/0 (socket says 172.21.15.115:34744) 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.389+0000 7f9290ab5640 1 -- 172.21.15.115:0/1740830716 learned_addr learned my addr 172.21.15.115:0/1740830716 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.389+0000 7f928bfff640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f928c06f2f0 0x7f928c06c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.389+0000 7f928b7fe640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c0adf10 0x7f928c14e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.389+0000 7f9290ab5640 1 -- 172.21.15.115:0/1740830716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f928c06f2f0 msgr2=0x7f928c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.389+0000 7f9290ab5640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f928c06f2f0 0x7f928c06c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.389+0000 7f9290ab5640 1 -- 172.21.15.115:0/1740830716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c0adf10 msgr2=0x7f928c14e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.389+0000 7f9290ab5640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c0adf10 0x7f928c14e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.389+0000 7f9290ab5640 1 -- 172.21.15.115:0/1740830716 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9284009400 con 0x7f928c15e2f0 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.390+0000 7f9290ab5640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c15e2f0 0x7f928c0cbac0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f92840097a0 tx=0x7f92840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.390+0000 7f9288ff9640 1 -- 172.21.15.115:0/1740830716 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284034680 con 0x7f928c15e2f0 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.390+0000 7f9288ff9640 1 -- 172.21.15.115:0/1740830716 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92840347e0 con 0x7f928c15e2f0 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.390+0000 7f9288ff9640 1 -- 172.21.15.115:0/1740830716 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284034b70 con 0x7f928c15e2f0 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.390+0000 7f9292518640 1 -- 172.21.15.115:0/1740830716 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928c14ee60 con 0x7f928c15e2f0 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.390+0000 7f9292518640 1 -- 172.21.15.115:0/1740830716 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928c0c24e0 con 0x7f928c15e2f0 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.391+0000 7f9292518640 1 -- 172.21.15.115:0/1740830716 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928c06fda0 con 0x7f928c15e2f0 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.392+0000 7f9288ff9640 1 -- 172.21.15.115:0/1740830716 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f928403b040 con 0x7f928c15e2f0 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.393+0000 7f9288ff9640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f926c06a550 0x7f926c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.393+0000 7f9288ff9640 1 -- 172.21.15.115:0/1740830716 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f92840aa180 con 0x7f928c15e2f0 2023-12-01T02:52:35.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.393+0000 7f928bfff640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f926c06a550 0x7f926c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.394+0000 7f928bfff640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f926c06a550 0x7f926c06ca10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f927c0099c0 tx=0x7f927c003990 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:35.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.398+0000 7f9288ff9640 1 -- 172.21.15.115:0/1740830716 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9284075510 con 0x7f928c15e2f0 2023-12-01T02:52:35.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.612+0000 7f9292518640 1 -- 172.21.15.115:0/1740830716 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f928c0b43d0 con 0x7f926c06a550 2023-12-01T02:52:35.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.614+0000 7f9288ff9640 1 -- 172.21.15.115:0/1740830716 <== mgr.9297 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f927c009650 con 0x7f926c06a550 2023-12-01T02:52:35.616 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T02:52:35.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T02:52:35.616 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T02:52:35.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T02:52:35.616 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T02:52:35.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.616+0000 7f9292518640 1 -- 172.21.15.115:0/1740830716 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f926c06a550 msgr2=0x7f926c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.616+0000 7f9292518640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f926c06a550 0x7f926c06ca10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f927c0099c0 tx=0x7f927c003990 comp rx=0 tx=0).stop 2023-12-01T02:52:35.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.616+0000 7f9292518640 1 -- 172.21.15.115:0/1740830716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c15e2f0 msgr2=0x7f928c0cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.616+0000 7f9292518640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c15e2f0 0x7f928c0cbac0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f92840097a0 tx=0x7f92840046c0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.617+0000 7f9292518640 1 -- 172.21.15.115:0/1740830716 shutdown_connections 2023-12-01T02:52:35.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.617+0000 7f9292518640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f926c06a550 0x7f926c06ca10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.617+0000 7f9292518640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f928c15e2f0 0x7f928c0cbac0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.617+0000 7f9292518640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c0adf10 0x7f928c14e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.617+0000 7f9292518640 1 --2- 172.21.15.115:0/1740830716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f928c06f2f0 0x7f928c06c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.617+0000 7f9292518640 1 -- 172.21.15.115:0/1740830716 >> 172.21.15.115:0/1740830716 conn(0x7f928c0ce880 msgr2=0x7f928c15d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:35.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.617+0000 7f9292518640 1 -- 172.21.15.115:0/1740830716 shutdown_connections 2023-12-01T02:52:35.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.617+0000 7f9292518640 1 -- 172.21.15.115:0/1740830716 wait complete. 2023-12-01T02:52:35.637 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T02:52:35.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.850+0000 7f8088db1640 1 Processor -- start 2023-12-01T02:52:35.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.850+0000 7f8088db1640 1 -- start start 2023-12-01T02:52:35.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.851+0000 7f8088db1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80840ce1b0 0x7f80840cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.852+0000 7f8088db1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840cc9a0 con 0x7f80840b0e60 2023-12-01T02:52:35.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.852+0000 7f8088db1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840cbd90 con 0x7f808415ac80 2023-12-01T02:52:35.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.852+0000 7f8088db1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808415b060 con 0x7f80840ce1b0 2023-12-01T02:52:35.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.852+0000 7f8082d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f80840b0e60 0x7f808406b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59234/0 (socket says 172.21.15.115:59234) 2023-12-01T02:52:35.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.852+0000 7f8082d9d640 1 -- 172.21.15.115:0/3440778132 learned_addr learned my addr 172.21.15.115:0/3440778132 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:35.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.852+0000 7f8081d9b640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80840ce1b0 0x7f80840cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.852+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101375165 0 0) 0x7f80840cc9a0 con 0x7f80840b0e60 2023-12-01T02:52:35.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.852+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f806c003520 con 0x7f80840b0e60 2023-12-01T02:52:35.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2755034068 0 0) 0x7f806c003520 con 0x7f80840b0e60 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80840ce1b0 msgr2=0x7f80840cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80840ce1b0 0x7f80840cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 >> v1:172.21.15.142:6789/0 conn(0x7f808415ac80 legacy=0x7f80840cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840cc070 con 0x7f80840b0e60 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f807c0019f0 con 0x7f80840b0e60 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 >> v1:172.21.15.115:6789/0 conn(0x7f80840b0e60 legacy=0x7f808406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 0x7f806c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80840ce1b0 0x7f806c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f806c004d00 0x7f806c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808415b060 con 0x7f806c0039a0 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c00bb60 con 0x7f806c004d00 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c00bd20 con 0x7f80840ce1b0 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f808259c640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 0x7f806c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.853+0000 7f8081d9b640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80840ce1b0 0x7f806c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.854+0000 7f808259c640 1 -- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80840ce1b0 msgr2=0x7f806c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:35.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.854+0000 7f808259c640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80840ce1b0 0x7f806c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.854+0000 7f8082d9d640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f806c004d00 0x7f806c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.854+0000 7f808259c640 1 -- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f806c004d00 msgr2=0x7f806c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:35.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.854+0000 7f808259c640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f806c004d00 0x7f806c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.854+0000 7f808259c640 1 -- 172.21.15.115:0/3440778132 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c003520 con 0x7f806c0039a0 2023-12-01T02:52:35.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.854+0000 7f8081d9b640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80840ce1b0 0x7f806c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:35.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.855+0000 7f808259c640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 0x7f806c003dc0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f8074001750 tx=0x7f806c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d5456c662de725 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:35.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.855+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740052d0 con 0x7f806c0039a0 2023-12-01T02:52:35.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.855+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807402e050 con 0x7f806c0039a0 2023-12-01T02:52:35.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.855+0000 7f808159a640 1 -- 172.21.15.115:0/3440778132 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740035f0 con 0x7f806c0039a0 2023-12-01T02:52:35.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.855+0000 7f8088db1640 1 -- 172.21.15.115:0/3440778132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 msgr2=0x7f806c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.855+0000 7f8088db1640 1 --2- 172.21.15.115:0/3440778132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 0x7f806c003dc0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f8074001750 tx=0x7f806c0057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.855+0000 7f8082d9d640 1 -- 172.21.15.115:0/3440778132 reap_dead start 2023-12-01T02:52:35.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.856+0000 7f8088db1640 1 -- 172.21.15.115:0/3440778132 shutdown_connections 2023-12-01T02:52:35.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.856+0000 7f8088db1640 1 -- 172.21.15.115:0/3440778132 >> 172.21.15.115:0/3440778132 conn(0x7f8084065960 msgr2=0x7f8084065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:35.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.856+0000 7f8088db1640 1 -- 172.21.15.115:0/3440778132 shutdown_connections 2023-12-01T02:52:35.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.856+0000 7f8088db1640 1 -- 172.21.15.115:0/3440778132 wait complete. 2023-12-01T02:52:35.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.857+0000 7f8088db1640 1 Processor -- start 2023-12-01T02:52:35.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.858+0000 7f8088db1640 1 -- start start 2023-12-01T02:52:35.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.858+0000 7f8088db1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 0x7f8084152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.858+0000 7f8088db1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8084151340 0x7f808414dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.858+0000 7f8088db1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808415d370 0x7f808415d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.858+0000 7f8088db1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808415dd40 con 0x7f806c0039a0 2023-12-01T02:52:35.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.858+0000 7f8088db1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084152450 con 0x7f808415d370 2023-12-01T02:52:35.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.858+0000 7f8088db1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8084151840 con 0x7f8084151340 2023-12-01T02:52:35.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.859+0000 7f808259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 0x7f8084152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.859+0000 7f808259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 0x7f8084152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34766/0 (socket says 172.21.15.115:34766) 2023-12-01T02:52:35.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.859+0000 7f808259c640 1 -- 172.21.15.115:0/972831695 learned_addr learned my addr 172.21.15.115:0/972831695 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:35.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.859+0000 7f8082d9d640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808415d370 0x7f808415d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.859+0000 7f8081d9b640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8084151340 0x7f808414dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.859+0000 7f808259c640 1 -- 172.21.15.115:0/972831695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8084151340 msgr2=0x7f808414dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.859+0000 7f808259c640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8084151340 0x7f808414dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.859+0000 7f808259c640 1 -- 172.21.15.115:0/972831695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808415d370 msgr2=0x7f808415d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:35.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.859+0000 7f808259c640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808415d370 0x7f808415d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:35.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.859+0000 7f808259c640 1 -- 172.21.15.115:0/972831695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8084151b20 con 0x7f806c0039a0 2023-12-01T02:52:35.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.860+0000 7f808259c640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 0x7f8084152d00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f8074008fd0 tx=0x7f8074001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:35.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.860+0000 7f8072ffd640 1 -- 172.21.15.115:0/972831695 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740076c0 con 0x7f806c0039a0 2023-12-01T02:52:35.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.860+0000 7f8072ffd640 1 -- 172.21.15.115:0/972831695 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8074007a30 con 0x7f806c0039a0 2023-12-01T02:52:35.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.860+0000 7f8072ffd640 1 -- 172.21.15.115:0/972831695 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074006be0 con 0x7f806c0039a0 2023-12-01T02:52:35.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.860+0000 7f8088db1640 1 -- 172.21.15.115:0/972831695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8084151e00 con 0x7f806c0039a0 2023-12-01T02:52:35.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.860+0000 7f8088db1640 1 -- 172.21.15.115:0/972831695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808414e320 con 0x7f806c0039a0 2023-12-01T02:52:35.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.861+0000 7f8088db1640 1 -- 172.21.15.115:0/972831695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808404e890 con 0x7f806c0039a0 2023-12-01T02:52:35.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.862+0000 7f8072ffd640 1 -- 172.21.15.115:0/972831695 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 95) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8074006d40 con 0x7f806c0039a0 2023-12-01T02:52:35.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.863+0000 7f8072ffd640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f806406a550 0x7f806406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:35.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.863+0000 7f8072ffd640 1 -- 172.21.15.115:0/972831695 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f80740aac60 con 0x7f806c0039a0 2023-12-01T02:52:35.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.863+0000 7f8081d9b640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f806406a550 0x7f806406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:35.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.864+0000 7f8081d9b640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f806406a550 0x7f806406ca10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f808406d470 tx=0x7f8078008040 comp rx=0 tx=0).ready entity=mgr.9297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:35.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:35.867+0000 7f8072ffd640 1 -- 172.21.15.115:0/972831695 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8074076090 con 0x7f806c0039a0 2023-12-01T02:52:36.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:36.109+0000 7f8088db1640 1 -- 172.21.15.115:0/972831695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f808406f750 con 0x7f806c0039a0 2023-12-01T02:52:37.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.073+0000 7f8072ffd640 1 -- 172.21.15.115:0/972831695 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 96) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f80740ddc80 con 0x7f806c0039a0 2023-12-01T02:52:37.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.075+0000 7f8072ffd640 1 -- 172.21.15.115:0/972831695 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v96) v1 ==== 92+0+0 (secure 0 0 0) 0x7f807408d780 con 0x7f806c0039a0 2023-12-01T02:52:37.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 -- 172.21.15.115:0/972831695 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f806406a550 msgr2=0x7f806406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f806406a550 0x7f806406ca10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f808406d470 tx=0x7f8078008040 comp rx=0 tx=0).stop 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 -- 172.21.15.115:0/972831695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 msgr2=0x7f8084152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 0x7f8084152d00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f8074008fd0 tx=0x7f8074001120 comp rx=0 tx=0).stop 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 -- 172.21.15.115:0/972831695 shutdown_connections 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f806406a550 0x7f806406ca10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8084151340 0x7f808414dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f808415d370 0x7f808415d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 --2- 172.21.15.115:0/972831695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f806c0039a0 0x7f8084152d00 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 -- 172.21.15.115:0/972831695 >> 172.21.15.115:0/972831695 conn(0x7f8084065960 msgr2=0x7f80840cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 -- 172.21.15.115:0/972831695 shutdown_connections 2023-12-01T02:52:37.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.079+0000 7f8088db1640 1 -- 172.21.15.115:0/972831695 wait complete. 2023-12-01T02:52:37.102 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T02:52:37.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.307+0000 7f1f30c43640 1 Processor -- start 2023-12-01T02:52:37.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.307+0000 7f1f30c43640 1 -- start start 2023-12-01T02:52:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.308+0000 7f1f30c43640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1f2c0b5fa0 0x7f1f2c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.309+0000 7f1f30c43640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2c0ce5e0 con 0x7f1f2c0cfd60 2023-12-01T02:52:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.309+0000 7f1f30c43640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2c0ce300 con 0x7f1f2c15cd90 2023-12-01T02:52:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.309+0000 7f1f30c43640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2c0d0ec0 con 0x7f1f2c0b5fa0 2023-12-01T02:52:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.309+0000 7f1f29d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1f2c0cfd60 0x7f1f2c0d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59250/0 (socket says 172.21.15.115:59250) 2023-12-01T02:52:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.309+0000 7f1f29d9b640 1 -- 172.21.15.115:0/2980150390 learned_addr learned my addr 172.21.15.115:0/2980150390 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.309+0000 7f1f2ad9d640 1 --2- 172.21.15.115:0/2980150390 >> v2:172.21.15.142:3300/0 conn(0x7f1f2c0b5fa0 0x7f1f2c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.309+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289453695 0 0) 0x7f1f20001110 con 0x7f1f2c0cfd60 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.309+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f14003520 con 0x7f1f2c0cfd60 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4283936969 0 0) 0x7f1f14003520 con 0x7f1f2c0cfd60 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 >> v2:172.21.15.142:3300/0 conn(0x7f1f2c0b5fa0 msgr2=0x7f1f2c0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 --2- 172.21.15.115:0/2980150390 >> v2:172.21.15.142:3300/0 conn(0x7f1f2c0b5fa0 0x7f1f2c0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 >> v1:172.21.15.142:6789/0 conn(0x7f1f2c15cd90 legacy=0x7f1f2c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0d10a0 con 0x7f1f2c0cfd60 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f1f20001800 con 0x7f1f2c0cfd60 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 >> v1:172.21.15.115:6789/0 conn(0x7f1f2c0cfd60 legacy=0x7f1f2c0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 --2- 172.21.15.115:0/2980150390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f14003970 0x7f1f14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 --2- 172.21.15.115:0/2980150390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f140047c0 0x7f1f14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 --2- 172.21.15.115:0/2980150390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0d0ec0 con 0x7f1f14005180 2023-12-01T02:52:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1400c020 con 0x7f1f140047c0 2023-12-01T02:52:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1400c1e0 con 0x7f1f14003970 2023-12-01T02:52:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.310+0000 7f1f2ad9d640 1 --2- 172.21.15.115:0/2980150390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.311+0000 7f1f29d9b640 1 --2- 172.21.15.115:0/2980150390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f140047c0 0x7f1f14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.311+0000 7f1f2ad9d640 1 -- 172.21.15.115:0/2980150390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f14003970 msgr2=0x7f1f14003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.311+0000 7f1f2ad9d640 1 --2- 172.21.15.115:0/2980150390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f14003970 0x7f1f14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.311+0000 7f1f2ad9d640 1 -- 172.21.15.115:0/2980150390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f140047c0 msgr2=0x7f1f14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.311+0000 7f1f2ad9d640 1 --2- 172.21.15.115:0/2980150390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f140047c0 0x7f1f14004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.311+0000 7f1f2ad9d640 1 -- 172.21.15.115:0/2980150390 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f14003520 con 0x7f1f14005180 2023-12-01T02:52:37.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.312+0000 7f1f2ad9d640 1 -- 172.21.15.115:0/2980150390 reap_dead start 2023-12-01T02:52:37.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.312+0000 7f1f2ad9d640 1 --2- 172.21.15.115:0/2980150390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f1400b980 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009fc0 tx=0x7f1f140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0f30c6825d7127 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:37.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.312+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2402cc60 con 0x7f1f14005180 2023-12-01T02:52:37.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.312+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2400f040 con 0x7f1f14005180 2023-12-01T02:52:37.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.312+0000 7f1f2959a640 1 -- 172.21.15.115:0/2980150390 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24003990 con 0x7f1f14005180 2023-12-01T02:52:37.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.313+0000 7f1f30c43640 1 -- 172.21.15.115:0/2980150390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 msgr2=0x7f1f1400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.313+0000 7f1f30c43640 1 --2- 172.21.15.115:0/2980150390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f1400b980 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009fc0 tx=0x7f1f140057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.313+0000 7f1f30c43640 1 -- 172.21.15.115:0/2980150390 shutdown_connections 2023-12-01T02:52:37.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.313+0000 7f1f30c43640 1 --2- 172.21.15.115:0/2980150390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f1400b980 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.313+0000 7f1f30c43640 1 -- 172.21.15.115:0/2980150390 >> 172.21.15.115:0/2980150390 conn(0x7f1f2c065960 msgr2=0x7f1f2c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:37.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.313+0000 7f1f30c43640 1 -- 172.21.15.115:0/2980150390 shutdown_connections 2023-12-01T02:52:37.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.313+0000 7f1f30c43640 1 -- 172.21.15.115:0/2980150390 wait complete. 2023-12-01T02:52:37.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.315+0000 7f1f30c43640 1 Processor -- start 2023-12-01T02:52:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.315+0000 7f1f30c43640 1 -- start start 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f30c43640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f2c0ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f30c43640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f2c152630 0x7f1f2c0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f30c43640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c15f270 0x7f1f2c15f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f30c43640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c15fc30 con 0x7f1f14005180 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f30c43640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0cccc0 con 0x7f1f2c15f270 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f30c43640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c0cae10 con 0x7f1f2c152630 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f2a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f2c0ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f2a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f2c0ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34792/0 (socket says 172.21.15.115:34792) 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f2a59c640 1 -- 172.21.15.115:0/2773526876 learned_addr learned my addr 172.21.15.115:0/2773526876 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f2ad9d640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c15f270 0x7f1f2c15f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.316+0000 7f1f29d9b640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f2c152630 0x7f1f2c0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f2a59c640 1 -- 172.21.15.115:0/2773526876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f2c152630 msgr2=0x7f1f2c0cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f2a59c640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f2c152630 0x7f1f2c0cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f2a59c640 1 -- 172.21.15.115:0/2773526876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c15f270 msgr2=0x7f1f2c15f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f2a59c640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c15f270 0x7f1f2c15f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f2a59c640 1 -- 172.21.15.115:0/2773526876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c0cb0f0 con 0x7f1f14005180 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f29d9b640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f2c152630 0x7f1f2c0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f2ad9d640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c15f270 0x7f1f2c15f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f2a59c640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f2c0ca770 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00c450 tx=0x7f1f1c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f1affd640 1 -- 172.21.15.115:0/2773526876 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c0052d0 con 0x7f1f14005180 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f1affd640 1 -- 172.21.15.115:0/2773526876 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c005430 con 0x7f1f14005180 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f1affd640 1 -- 172.21.15.115:0/2773526876 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c012870 con 0x7f1f14005180 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f30c43640 1 -- 172.21.15.115:0/2773526876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c0cb3d0 con 0x7f1f14005180 2023-12-01T02:52:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.317+0000 7f1f30c43640 1 -- 172.21.15.115:0/2773526876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c1e70b0 con 0x7f1f14005180 2023-12-01T02:52:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.319+0000 7f1f30c43640 1 -- 172.21.15.115:0/2773526876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2c1e7270 con 0x7f1f14005180 2023-12-01T02:52:37.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.320+0000 7f1f1affd640 1 -- 172.21.15.115:0/2773526876 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 96) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f1f1c0129d0 con 0x7f1f14005180 2023-12-01T02:52:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.320+0000 7f1f1affd640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f0c06a5a0 0x7f1f0c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.321+0000 7f1f1affd640 1 -- 172.21.15.115:0/2773526876 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f1f1c0886b0 con 0x7f1f14005180 2023-12-01T02:52:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.321+0000 7f1f29d9b640 1 -- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f0c06a5a0 msgr2=0x7f1f0c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.321+0000 7f1f29d9b640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f0c06a5a0 0x7f1f0c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:37.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.326+0000 7f1f1affd640 1 -- 172.21.15.115:0/2773526876 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1f1c053ad0 con 0x7f1f14005180 2023-12-01T02:52:37.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.522+0000 7f1f29d9b640 1 -- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f0c06a5a0 msgr2=0x7f1f0c06ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:37.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.522+0000 7f1f29d9b640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f0c06a5a0 0x7f1f0c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:52:37.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.531+0000 7f1f30c43640 1 -- 172.21.15.115:0/2773526876 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f1f2c1e6f10 con 0x7f1f14005180 2023-12-01T02:52:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.539+0000 7f1f1affd640 1 -- 172.21.15.115:0/2773526876 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1f2c1e6f10 con 0x7f1f14005180 2023-12-01T02:52:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.541+0000 7f1f30c43640 1 -- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f0c06a5a0 msgr2=0x7f1f0c06ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:37.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.541+0000 7f1f30c43640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f0c06a5a0 0x7f1f0c06ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.541+0000 7f1f30c43640 1 -- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 msgr2=0x7f1f2c0ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.541+0000 7f1f30c43640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f2c0ca770 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00c450 tx=0x7f1f1c00c910 comp rx=0 tx=0).stop 2023-12-01T02:52:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.541+0000 7f1f30c43640 1 -- 172.21.15.115:0/2773526876 shutdown_connections 2023-12-01T02:52:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.542+0000 7f1f30c43640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1f0c06a5a0 0x7f1f0c06ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.542+0000 7f1f30c43640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f2c152630 0x7f1f2c0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.542+0000 7f1f30c43640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f2c15f270 0x7f1f2c15f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.542+0000 7f1f30c43640 1 --2- 172.21.15.115:0/2773526876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f14005180 0x7f1f2c0ca770 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.542+0000 7f1f30c43640 1 -- 172.21.15.115:0/2773526876 >> 172.21.15.115:0/2773526876 conn(0x7f1f2c065960 msgr2=0x7f1f2c0d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.542+0000 7f1f30c43640 1 -- 172.21.15.115:0/2773526876 shutdown_connections 2023-12-01T02:52:37.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.542+0000 7f1f30c43640 1 -- 172.21.15.115:0/2773526876 wait complete. 2023-12-01T02:52:37.562 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T02:52:37.583 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:37.581+0000 7f3d73e20040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T02:52:37.632 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:37.630+0000 7f01fe617040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T02:52:37.635 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:37.633+0000 7f3d73e20040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T02:52:37.688 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:37.686+0000 7f01fe617040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T02:52:37.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.761+0000 7fc202007640 1 Processor -- start 2023-12-01T02:52:37.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.762+0000 7fc202007640 1 -- start start 2023-12-01T02:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.763+0000 7fc202007640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1fc0cfd60 0x7fc1fc0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.763+0000 7fc202007640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b5fa0 0x7fc1fc161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.763+0000 7fc202007640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc0ce5e0 con 0x7fc1fc15cd90 2023-12-01T02:52:37.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.763+0000 7fc202007640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc0d0ec0 con 0x7fc1fc0cfd60 2023-12-01T02:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.763+0000 7fc202007640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc161950 con 0x7fc1fc0b5fa0 2023-12-01T02:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.763+0000 7fc1faffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1fc0cfd60 0x7fc1fc0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.763+0000 7fc1faffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1fc0cfd60 0x7fc1fc0cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32932/0 (socket says 172.21.15.115:32932) 2023-12-01T02:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.763+0000 7fc1faffd640 1 -- 172.21.15.115:0/4208711425 learned_addr learned my addr 172.21.15.115:0/4208711425 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.764+0000 7fc1fbfff640 1 --2- 172.21.15.115:0/4208711425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b5fa0 0x7fc1fc161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.764+0000 7fc1faffd640 1 -- 172.21.15.115:0/4208711425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b5fa0 msgr2=0x7fc1fc161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.764+0000 7fc1faffd640 1 --2- 172.21.15.115:0/4208711425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b5fa0 0x7fc1fc161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.764+0000 7fc1faffd640 1 -- 172.21.15.115:0/4208711425 >> v1:172.21.15.142:6789/0 conn(0x7fc1fc15cd90 legacy=0x7fc1fc0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.764+0000 7fc1faffd640 1 -- 172.21.15.115:0/4208711425 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc0ce300 con 0x7fc1fc0cfd60 2023-12-01T02:52:37.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.766+0000 7fc1faffd640 1 --2- 172.21.15.115:0/4208711425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cfd60 0x7fc1fc0cdd20 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009870 tx=0x7fc1f002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d4f38a342e9b88 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:37.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.766+0000 7fc1fa7fc640 1 -- 172.21.15.115:0/4208711425 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0030840 con 0x7fc1fc0cfd60 2023-12-01T02:52:37.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.766+0000 7fc1fa7fc640 1 -- 172.21.15.115:0/4208711425 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f00309a0 con 0x7fc1fc0cfd60 2023-12-01T02:52:37.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.766+0000 7fc1fa7fc640 1 -- 172.21.15.115:0/4208711425 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0030d30 con 0x7fc1fc0cfd60 2023-12-01T02:52:37.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.766+0000 7fc202007640 1 -- 172.21.15.115:0/4208711425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cfd60 msgr2=0x7fc1fc0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.766+0000 7fc202007640 1 --2- 172.21.15.115:0/4208711425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cfd60 0x7fc1fc0cdd20 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009870 tx=0x7fc1f002b980 comp rx=0 tx=0).stop 2023-12-01T02:52:37.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.767+0000 7fc202007640 1 -- 172.21.15.115:0/4208711425 shutdown_connections 2023-12-01T02:52:37.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.767+0000 7fc202007640 1 --2- 172.21.15.115:0/4208711425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b5fa0 0x7fc1fc161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.767+0000 7fc202007640 1 --2- 172.21.15.115:0/4208711425 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc0cfd60 0x7fc1fc0cdd20 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.767+0000 7fc202007640 1 -- 172.21.15.115:0/4208711425 >> 172.21.15.115:0/4208711425 conn(0x7fc1fc065960 msgr2=0x7fc1fc065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:37.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.767+0000 7fc202007640 1 -- 172.21.15.115:0/4208711425 shutdown_connections 2023-12-01T02:52:37.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.767+0000 7fc202007640 1 -- 172.21.15.115:0/4208711425 wait complete. 2023-12-01T02:52:37.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.768+0000 7fc202007640 1 Processor -- start 2023-12-01T02:52:37.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.768+0000 7fc202007640 1 -- start start 2023-12-01T02:52:37.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.769+0000 7fc202007640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b5fa0 0x7fc1fc15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.769+0000 7fc202007640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1fc0cfd60 0x7fc1fc151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.769+0000 7fc202007640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc15cd90 0x7fc1fc1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.769+0000 7fc202007640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc1f23f0 con 0x7fc1fc15cd90 2023-12-01T02:52:37.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.769+0000 7fc202007640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc15bc40 con 0x7fc1fc0cfd60 2023-12-01T02:52:37.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.769+0000 7fc202007640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc152010 con 0x7fc1fc0b5fa0 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.770+0000 7fc1fbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc15cd90 0x7fc1fc1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.770+0000 7fc1faffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1fc0cfd60 0x7fc1fc151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.770+0000 7fc1faffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1fc0cfd60 0x7fc1fc151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:52498/0 (socket says 172.21.15.115:52498) 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.770+0000 7fc1fb7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b5fa0 0x7fc1fc15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.770+0000 7fc1faffd640 1 -- 172.21.15.115:0/2246754209 learned_addr learned my addr 172.21.15.115:0/2246754209 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.770+0000 7fc1fbfff640 1 -- 172.21.15.115:0/2246754209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b5fa0 msgr2=0x7fc1fc15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.770+0000 7fc1fbfff640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b5fa0 0x7fc1fc15c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.770+0000 7fc1fbfff640 1 -- 172.21.15.115:0/2246754209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1fc0cfd60 msgr2=0x7fc1fc151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.770+0000 7fc1fbfff640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1fc0cfd60 0x7fc1fc151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.770+0000 7fc1fbfff640 1 -- 172.21.15.115:0/2246754209 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00094d0 con 0x7fc1fc15cd90 2023-12-01T02:52:37.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.771+0000 7fc1fbfff640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc15cd90 0x7fc1fc1589a0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4007dc0 tx=0x7fc1f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:37.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.771+0000 7fc1e3fff640 1 -- 172.21.15.115:0/2246754209 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4018070 con 0x7fc1fc15cd90 2023-12-01T02:52:37.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.771+0000 7fc1e3fff640 1 -- 172.21.15.115:0/2246754209 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f400ed70 con 0x7fc1fc15cd90 2023-12-01T02:52:37.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.771+0000 7fc1e3fff640 1 -- 172.21.15.115:0/2246754209 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400eee0 con 0x7fc1fc15cd90 2023-12-01T02:52:37.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.771+0000 7fc202007640 1 -- 172.21.15.115:0/2246754209 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc15b310 con 0x7fc1fc15cd90 2023-12-01T02:52:37.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.771+0000 7fc202007640 1 -- 172.21.15.115:0/2246754209 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc15b8d0 con 0x7fc1fc15cd90 2023-12-01T02:52:37.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.772+0000 7fc202007640 1 -- 172.21.15.115:0/2246754209 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1fc15e590 con 0x7fc1fc15cd90 2023-12-01T02:52:37.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.773+0000 7fc1e3fff640 1 -- 172.21.15.115:0/2246754209 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 96) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fc1f4003bb0 con 0x7fc1fc15cd90 2023-12-01T02:52:37.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.775+0000 7fc1e3fff640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1e806a5d0 0x7fc1e806ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:37.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.775+0000 7fc1fb7fe640 1 -- 172.21.15.115:0/2246754209 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1e806a5d0 msgr2=0x7fc1e806ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:37.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.775+0000 7fc1e3fff640 1 -- 172.21.15.115:0/2246754209 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fc1f4088c20 con 0x7fc1fc15cd90 2023-12-01T02:52:37.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.775+0000 7fc1fb7fe640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1e806a5d0 0x7fc1e806ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:37.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.779+0000 7fc1e3fff640 1 -- 172.21.15.115:0/2246754209 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc1fc15e590 con 0x7fc1fc15cd90 2023-12-01T02:52:37.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.966+0000 7fc202007640 1 -- 172.21.15.115:0/2246754209 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fc1fc15e590 con 0x7fc1fc15cd90 2023-12-01T02:52:37.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.973+0000 7fc1e3fff640 1 -- 172.21.15.115:0/2246754209 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc1fc15e590 con 0x7fc1fc15cd90 2023-12-01T02:52:37.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.975+0000 7fc202007640 1 -- 172.21.15.115:0/2246754209 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1e806a5d0 msgr2=0x7fc1e806ca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:37.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.975+0000 7fc202007640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1e806a5d0 0x7fc1e806ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.975+0000 7fc202007640 1 -- 172.21.15.115:0/2246754209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc15cd90 msgr2=0x7fc1fc1589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:37.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.975+0000 7fc202007640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc15cd90 0x7fc1fc1589a0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4007dc0 tx=0x7fc1f400e4b0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.975+0000 7fc202007640 1 -- 172.21.15.115:0/2246754209 shutdown_connections 2023-12-01T02:52:37.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.975+0000 7fc202007640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7fc1e806a5d0 0x7fc1e806ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.975+0000 7fc202007640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1fc15cd90 0x7fc1fc1589a0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.975+0000 7fc202007640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1fc0cfd60 0x7fc1fc151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.975+0000 7fc202007640 1 --2- 172.21.15.115:0/2246754209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1fc0b5fa0 0x7fc1fc15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:37.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.975+0000 7fc202007640 1 -- 172.21.15.115:0/2246754209 >> 172.21.15.115:0/2246754209 conn(0x7fc1fc065960 msgr2=0x7fc1fc0b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:37.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.976+0000 7fc202007640 1 -- 172.21.15.115:0/2246754209 shutdown_connections 2023-12-01T02:52:37.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:37.976+0000 7fc202007640 1 -- 172.21.15.115:0/2246754209 wait complete. 2023-12-01T02:52:37.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:37.999 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-01T02:52:38.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T02:52:38.051 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-12-01T02:52:38.089 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:38.087+0000 7f3d73e20040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T02:52:38.096 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/test_restart 1M 2023-12-01T02:52:38.140 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:38.138+0000 7f3d73e20040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T02:52:38.158 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:38.157+0000 7f01fe617040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T02:52:38.190 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:38.188+0000 7f3d73e20040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T02:52:38.213 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:38.211+0000 7f01fe617040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T02:52:38.251 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:38.249+0000 7f3d73e20040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T02:52:38.266 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:38.264+0000 7f01fe617040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T02:52:38.300 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:38.298+0000 7f3d73e20040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T02:52:38.328 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:38.326+0000 7f01fe617040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T02:52:38.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.342+0000 7f1e99acf640 1 Processor -- start 2023-12-01T02:52:38.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.342+0000 7f1e99acf640 1 -- start start 2023-12-01T02:52:38.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.343+0000 7f1e99acf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1e9415cf70 0x7f1e940b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:38.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.343+0000 7f1e99acf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1e940b18f0 0x7f1e940b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:38.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.343+0000 7f1e99acf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e940cfb50 0x7f1e940cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:38.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.343+0000 7f1e99acf640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e940b2170 con 0x7f1e9415cf70 2023-12-01T02:52:38.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.343+0000 7f1e99acf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e940b7760 con 0x7f1e940b18f0 2023-12-01T02:52:38.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.343+0000 7f1e99acf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940d04e0 con 0x7f1e940cfb50 2023-12-01T02:52:38.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.344+0000 7f1e937fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1e9415cf70 0x7f1e940b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:38.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.344+0000 7f1e937fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1e9415cf70 0x7f1e940b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32952/0 (socket says 172.21.15.115:32952) 2023-12-01T02:52:38.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.344+0000 7f1e937fe640 1 -- 172.21.15.115:0/2184621822 learned_addr learned my addr 172.21.15.115:0/2184621822 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:38.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.344+0000 7f1e92ffd640 1 --2- 172.21.15.115:0/2184621822 >> v2:172.21.15.142:3300/0 conn(0x7f1e940b18f0 0x7f1e940b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:38.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.344+0000 7f1e93fff640 1 --2- 172.21.15.115:0/2184621822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e940cfb50 0x7f1e940cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:38.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.344+0000 7f1e937fe640 1 -- 172.21.15.115:0/2184621822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e940cfb50 msgr2=0x7f1e940cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:38.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.344+0000 7f1e937fe640 1 --2- 172.21.15.115:0/2184621822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e940cfb50 0x7f1e940cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:38.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.344+0000 7f1e937fe640 1 -- 172.21.15.115:0/2184621822 >> v2:172.21.15.142:3300/0 conn(0x7f1e940b18f0 msgr2=0x7f1e940b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:38.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.344+0000 7f1e937fe640 1 --2- 172.21.15.115:0/2184621822 >> v2:172.21.15.142:3300/0 conn(0x7f1e940b18f0 0x7f1e940b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:38.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.344+0000 7f1e937fe640 1 -- 172.21.15.115:0/2184621822 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940b8b70 con 0x7f1e9415cf70 2023-12-01T02:52:38.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.345+0000 7f1e937fe640 1 --2- 172.21.15.115:0/2184621822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 0x7f1e940b8470 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f1e940b5580 tx=0x7f1e8402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea0a6eacfde5840 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:38.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.346+0000 7f1e927fc640 1 -- 172.21.15.115:0/2184621822 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84030860 con 0x7f1e9415cf70 2023-12-01T02:52:38.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.346+0000 7f1e927fc640 1 -- 172.21.15.115:0/2184621822 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e840309c0 con 0x7f1e9415cf70 2023-12-01T02:52:38.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.346+0000 7f1e927fc640 1 -- 172.21.15.115:0/2184621822 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84030d50 con 0x7f1e9415cf70 2023-12-01T02:52:38.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.346+0000 7f1e99acf640 1 -- 172.21.15.115:0/2184621822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 msgr2=0x7f1e940b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:38.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.346+0000 7f1e99acf640 1 --2- 172.21.15.115:0/2184621822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 0x7f1e940b8470 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f1e940b5580 tx=0x7f1e8402b9a0 comp rx=0 tx=0).stop 2023-12-01T02:52:38.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.346+0000 7f1e99acf640 1 -- 172.21.15.115:0/2184621822 shutdown_connections 2023-12-01T02:52:38.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.346+0000 7f1e99acf640 1 --2- 172.21.15.115:0/2184621822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e940cfb50 0x7f1e940cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:38.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.346+0000 7f1e99acf640 1 --2- 172.21.15.115:0/2184621822 >> v2:172.21.15.142:3300/0 conn(0x7f1e940b18f0 0x7f1e940b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:38.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.346+0000 7f1e99acf640 1 --2- 172.21.15.115:0/2184621822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 0x7f1e940b8470 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:38.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.346+0000 7f1e99acf640 1 -- 172.21.15.115:0/2184621822 >> 172.21.15.115:0/2184621822 conn(0x7f1e9406a0e0 msgr2=0x7f1e9406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:38.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.347+0000 7f1e99acf640 1 -- 172.21.15.115:0/2184621822 shutdown_connections 2023-12-01T02:52:38.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.347+0000 7f1e99acf640 1 -- 172.21.15.115:0/2184621822 wait complete. 2023-12-01T02:52:38.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.348+0000 7f1e99acf640 1 Processor -- start 2023-12-01T02:52:38.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.348+0000 7f1e99acf640 1 -- start start 2023-12-01T02:52:38.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.349+0000 7f1e99acf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e940b18f0 0x7f1e94152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:38.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.349+0000 7f1e99acf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e940cfb50 0x7f1e940d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:38.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.349+0000 7f1e99acf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 0x7f1e9415e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:38.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.349+0000 7f1e99acf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940d1cc0 con 0x7f1e9415cf70 2023-12-01T02:52:38.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.349+0000 7f1e99acf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940d1420 con 0x7f1e940cfb50 2023-12-01T02:52:38.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.349+0000 7f1e99acf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940d1700 con 0x7f1e940b18f0 2023-12-01T02:52:38.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.350+0000 7f1e93fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 0x7f1e9415e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:38.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.350+0000 7f1e93fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 0x7f1e9415e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32966/0 (socket says 172.21.15.115:32966) 2023-12-01T02:52:38.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.350+0000 7f1e93fff640 1 -- 172.21.15.115:0/3933642905 learned_addr learned my addr 172.21.15.115:0/3933642905 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:38.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.350+0000 7f1e92ffd640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e940cfb50 0x7f1e940d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:38.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.350+0000 7f1e937fe640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e940b18f0 0x7f1e94152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:38.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.350+0000 7f1e93fff640 1 -- 172.21.15.115:0/3933642905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e940b18f0 msgr2=0x7f1e94152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:38.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.350+0000 7f1e93fff640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e940b18f0 0x7f1e94152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:38.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.350+0000 7f1e93fff640 1 -- 172.21.15.115:0/3933642905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e940cfb50 msgr2=0x7f1e940d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:38.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.350+0000 7f1e93fff640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e940cfb50 0x7f1e940d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:38.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.350+0000 7f1e93fff640 1 -- 172.21.15.115:0/3933642905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e840094d0 con 0x7f1e9415cf70 2023-12-01T02:52:38.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.351+0000 7f1e93fff640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 0x7f1e9415e3a0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c007e60 tx=0x7f1e8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:38.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.351+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c014420 con 0x7f1e9415cf70 2023-12-01T02:52:38.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.351+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c0145a0 con 0x7f1e9415cf70 2023-12-01T02:52:38.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.351+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c014950 con 0x7f1e9415cf70 2023-12-01T02:52:38.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.351+0000 7f1e99acf640 1 -- 172.21.15.115:0/3933642905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e940d19e0 con 0x7f1e9415cf70 2023-12-01T02:52:38.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.351+0000 7f1e99acf640 1 -- 172.21.15.115:0/3933642905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e941533f0 con 0x7f1e9415cf70 2023-12-01T02:52:38.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.352+0000 7f1e99acf640 1 -- 172.21.15.115:0/3933642905 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e940d1fe0 con 0x7f1e9415cf70 2023-12-01T02:52:38.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.353+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 96) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f1e8c014ab0 con 0x7f1e9415cf70 2023-12-01T02:52:38.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.354+0000 7f1e7bfff640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:38.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.354+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f1e8c0893c0 con 0x7f1e9415cf70 2023-12-01T02:52:38.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.354+0000 7f1e937fe640 1 -- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 msgr2=0x7f1e8006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:38.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.354+0000 7f1e937fe640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T02:52:38.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.358+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1e940d1fe0 con 0x7f1e9415cf70 2023-12-01T02:52:38.381 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:38.379+0000 7f01fe617040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T02:52:38.410 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:38.408+0000 7f3d73e20040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T02:52:38.471 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:38.469+0000 7f3d73e20040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T02:52:38.495 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:38.493+0000 7f01fe617040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T02:52:38.520 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:38.518+0000 7f3d73e20040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T02:52:38.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.555+0000 7f1e937fe640 1 -- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 msgr2=0x7f1e8006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:38.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.555+0000 7f1e937fe640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T02:52:38.558 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:38.557+0000 7f01fe617040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T02:52:38.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.571+0000 7f1e99acf640 1 -- 172.21.15.115:0/3933642905 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e940b7110 con 0x7f1e8006a5a0 2023-12-01T02:52:38.577 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:38.575+0000 7f3d73e20040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T02:52:38.607 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:38.606+0000 7f01fe617040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T02:52:38.627 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:38.625+0000 7f3d73e20040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T02:52:38.665 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:38.661+0000 7f01fe617040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T02:52:38.713 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:38.711+0000 7f01fe617040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T02:52:38.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.956+0000 7f1e937fe640 1 -- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 msgr2=0x7f1e8006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:38.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:38.956+0000 7f1e937fe640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T02:52:39.007 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:39.005+0000 7f3d73e20040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T02:52:39.089 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:39.087+0000 7f01fe617040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T02:52:39.163 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:39.162+0000 7f3d73e20040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T02:52:39.246 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:39.244+0000 7f01fe617040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T02:52:39.271 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:39.269+0000 7f3d73e20040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T02:52:39.320 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:39.318+0000 7f3d73e20040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T02:52:39.352 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:39.350+0000 7f01fe617040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T02:52:39.402 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:39.400+0000 7f01fe617040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T02:52:39.426 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:39.425+0000 7f3d73e20040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:52:39.507 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:39.506+0000 7f01fe617040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:52:39.532 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:39.530+0000 7f3d73e20040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T02:52:39.612 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:39.610+0000 7f01fe617040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T02:52:39.688 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:39.685+0000 7f3d73e20040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:52:39.735 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:39.733+0000 7f3d73e20040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T02:52:39.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:39.757+0000 7f1e937fe640 1 -- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 msgr2=0x7f1e8006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/88317 2023-12-01T02:52:39.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:39.757+0000 7f1e937fe640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T02:52:39.768 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:39.766+0000 7f01fe617040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T02:52:39.816 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:39.814+0000 7f01fe617040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T02:52:39.853 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:39.850+0000 7f3d73e20040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T02:52:39.935 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:39.933+0000 7f01fe617040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T02:52:40.683 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:40.680+0000 7f3d73e20040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T02:52:40.758 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:40.756+0000 7f01fe617040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T02:52:40.974 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:40.972+0000 7f3d73e20040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T02:52:41.031 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:41.029+0000 7f3d73e20040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T02:52:41.049 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.047+0000 7f01fe617040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T02:52:41.105 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.103+0000 7f01fe617040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T02:52:41.143 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T02:52:41.141+0000 7f3d73e20040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T02:52:41.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:41.208+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 97) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f1e8c054c30 con 0x7f1e9415cf70 2023-12-01T02:52:41.221 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.220+0000 7f01fe617040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T02:52:41.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:41.240+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 98) v1 ==== 88335+0+0 (secure 0 0 0) 0x7f1e8c0506c0 con 0x7f1e9415cf70 2023-12-01T02:52:41.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:41.240+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 msgr2=0x7f1e8006ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T02:52:41.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:41.240+0000 7f1e7bfff640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:41.441 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.439+0000 7f017dde9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:41.441 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.439+0000 7f017dde9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:41.441 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.439+0000 7f017dde9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:41.441 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.439+0000 7f017dde9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:41.441 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.439+0000 7f017dde9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:41.442 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.439+0000 7f017ade3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:41.442 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.439+0000 7f017ade3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:41.442 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.439+0000 7f017ade3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:41.442 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.439+0000 7f017ade3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:41.442 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:52:41.439+0000 7f017ade3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:52:42.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.251+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 <== mon.0 v2:172.21.15.115:3300/0 9 ==== mgrmap(e 99) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f1e8c055500 con 0x7f1e9415cf70 2023-12-01T02:52:42.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.252+0000 7f1e7bfff640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:42.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.252+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e80036460 con 0x7f1e8006a5a0 2023-12-01T02:52:42.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.259+0000 7f1e937fe640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:42.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.267+0000 7f1e937fe640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006c9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1e84036000 tx=0x7f1e84008030 comp rx=0 tx=0).ready entity=mgr.6194 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:42.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.429+0000 7f1e7bfff640 1 -- 172.21.15.115:0/3933642905 <== mgr.6194 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1e84030cb0 con 0x7f1e8006a5a0 2023-12-01T02:52:42.432 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-12-01T02:52:42.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.433+0000 7f1e99acf640 1 -- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 msgr2=0x7f1e8006c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:42.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.433+0000 7f1e99acf640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006c9e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1e84036000 tx=0x7f1e84008030 comp rx=0 tx=0).stop 2023-12-01T02:52:42.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.433+0000 7f1e99acf640 1 -- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 msgr2=0x7f1e9415e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:42.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.433+0000 7f1e99acf640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 0x7f1e9415e3a0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c007e60 tx=0x7f1e8c00e4b0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.433+0000 7f1e99acf640 1 -- 172.21.15.115:0/3933642905 shutdown_connections 2023-12-01T02:52:42.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.433+0000 7f1e99acf640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f1e8006a5a0 0x7f1e8006c9e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.434+0000 7f1e99acf640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e9415cf70 0x7f1e9415e3a0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.434+0000 7f1e99acf640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e940cfb50 0x7f1e940d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.434+0000 7f1e99acf640 1 --2- 172.21.15.115:0/3933642905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e940b18f0 0x7f1e94152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.434+0000 7f1e99acf640 1 -- 172.21.15.115:0/3933642905 >> 172.21.15.115:0/3933642905 conn(0x7f1e9406a0e0 msgr2=0x7f1e940cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:42.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.434+0000 7f1e99acf640 1 -- 172.21.15.115:0/3933642905 shutdown_connections 2023-12-01T02:52:42.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.434+0000 7f1e99acf640 1 -- 172.21.15.115:0/3933642905 wait complete. 2023-12-01T02:52:42.455 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-12-01T02:52:42.456 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/test_restart 1M 2023-12-01T02:52:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.707+0000 7f806c284640 1 Processor -- start 2023-12-01T02:52:42.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.707+0000 7f806c284640 1 -- start start 2023-12-01T02:52:42.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.708+0000 7f806c284640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f806415ac80 0x7f80640cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:42.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.708+0000 7f806c284640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80640b0e60 0x7f80640b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:42.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.708+0000 7f806c284640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80640cc9a0 con 0x7f80640cdaa0 2023-12-01T02:52:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.709+0000 7f806c284640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f806415b060 con 0x7f806415ac80 2023-12-01T02:52:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.709+0000 7f806c284640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80640b0ce0 con 0x7f80640b0e60 2023-12-01T02:52:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.709+0000 7f806a821640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f80640cdaa0 0x7f806406b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46364/0 (socket says 172.21.15.115:46364) 2023-12-01T02:52:42.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.709+0000 7f806a821640 1 -- 172.21.15.115:0/1779427076 learned_addr learned my addr 172.21.15.115:0/1779427076 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T02:52:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.709+0000 7f806981f640 1 --2- 172.21.15.115:0/1779427076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80640b0e60 0x7f80640b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.709+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659352328 0 0) 0x7f80640cc9a0 con 0x7f80640cdaa0 2023-12-01T02:52:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.709+0000 7f806a020640 1 --2- 172.21.15.115:0/1779427076 >> v2:172.21.15.142:3300/0 conn(0x7f806415ac80 0x7f80640cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.709+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8050003520 con 0x7f80640cdaa0 2023-12-01T02:52:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1058168301 0 0) 0x7f8050003520 con 0x7f80640cdaa0 2023-12-01T02:52:42.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80640b0e60 msgr2=0x7f80640b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:42.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 --2- 172.21.15.115:0/1779427076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80640b0e60 0x7f80640b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 >> v2:172.21.15.142:3300/0 conn(0x7f806415ac80 msgr2=0x7f80640cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:42.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 --2- 172.21.15.115:0/1779427076 >> v2:172.21.15.142:3300/0 conn(0x7f806415ac80 0x7f80640cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80640cbd90 con 0x7f80640cdaa0 2023-12-01T02:52:42.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f805c0019f0 con 0x7f80640cdaa0 2023-12-01T02:52:42.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 >> v1:172.21.15.115:6789/0 conn(0x7f80640cdaa0 legacy=0x7f806406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:42.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 --2- 172.21.15.115:0/1779427076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8050003970 0x7f8050003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 --2- 172.21.15.115:0/1779427076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 0x7f8050004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 --2- 172.21.15.115:0/1779427076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80640b0e60 0x7f8050005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806415b060 con 0x7f80500047c0 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80640b0ce0 con 0x7f8050003970 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805000b9d0 con 0x7f80640b0e60 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806a020640 1 --2- 172.21.15.115:0/1779427076 >> v2:172.21.15.142:3300/0 conn(0x7f806415ac80 0x7f80640cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806981f640 1 --2- 172.21.15.115:0/1779427076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 0x7f8050004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806981f640 1 -- 172.21.15.115:0/1779427076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80640b0e60 msgr2=0x7f8050005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806981f640 1 --2- 172.21.15.115:0/1779427076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80640b0e60 0x7f8050005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806981f640 1 -- 172.21.15.115:0/1779427076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8050003970 msgr2=0x7f8050003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806981f640 1 --2- 172.21.15.115:0/1779427076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8050003970 0x7f8050003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.710+0000 7f806981f640 1 -- 172.21.15.115:0/1779427076 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8050003520 con 0x7f80500047c0 2023-12-01T02:52:42.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.712+0000 7f806981f640 1 --2- 172.21.15.115:0/1779427076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 0x7f8050004c40 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f8054009ac0 tx=0x7f80500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915b638cc85e4e67 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:42.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.712+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805402c710 con 0x7f80500047c0 2023-12-01T02:52:42.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.712+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805402c870 con 0x7f80500047c0 2023-12-01T02:52:42.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.712+0000 7f806901e640 1 -- 172.21.15.115:0/1779427076 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805402cc00 con 0x7f80500047c0 2023-12-01T02:52:42.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.712+0000 7f806c284640 1 -- 172.21.15.115:0/1779427076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 msgr2=0x7f8050004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:42.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.712+0000 7f806c284640 1 --2- 172.21.15.115:0/1779427076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 0x7f8050004c40 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f8054009ac0 tx=0x7f80500057f0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.712+0000 7f806a821640 1 -- 172.21.15.115:0/1779427076 reap_dead start 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.712+0000 7f806c284640 1 -- 172.21.15.115:0/1779427076 shutdown_connections 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.712+0000 7f806c284640 1 -- 172.21.15.115:0/1779427076 >> 172.21.15.115:0/1779427076 conn(0x7f8064065960 msgr2=0x7f8064065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.713+0000 7f806c284640 1 -- 172.21.15.115:0/1779427076 shutdown_connections 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.713+0000 7f806c284640 1 -- 172.21.15.115:0/1779427076 wait complete. 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.714+0000 7f806c284640 1 Processor -- start 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.714+0000 7f806c284640 1 -- start start 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.715+0000 7f806c284640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 0x7f80641f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.715+0000 7f806c284640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8064153630 0x7f8064158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.715+0000 7f806c284640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8064159480 0x7f8064157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.715+0000 7f806c284640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8064159900 con 0x7f80500047c0 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.715+0000 7f806c284640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806415b340 con 0x7f8064153630 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.715+0000 7f806c284640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806415b620 con 0x7f8064159480 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.716+0000 7f806a020640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 0x7f80641f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:42.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.716+0000 7f806a020640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 0x7f80641f2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33084/0 (socket says 172.21.15.115:33084) 2023-12-01T02:52:42.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.716+0000 7f806a020640 1 -- 172.21.15.115:0/1084632506 learned_addr learned my addr 172.21.15.115:0/1084632506 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.716+0000 7f806a020640 1 -- 172.21.15.115:0/1084632506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8064159480 msgr2=0x7f8064157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.716+0000 7f806a020640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8064159480 0x7f8064157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.716+0000 7f806a020640 1 -- 172.21.15.115:0/1084632506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8064153630 msgr2=0x7f8064158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.716+0000 7f806a020640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8064153630 0x7f8064158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.716+0000 7f806a020640 1 -- 172.21.15.115:0/1084632506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806415b900 con 0x7f80500047c0 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.716+0000 7f806a020640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 0x7f80641f2610 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f806000e7e0 tx=0x7f806000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.716+0000 7f805affd640 1 -- 172.21.15.115:0/1084632506 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060004230 con 0x7f80500047c0 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.717+0000 7f805affd640 1 -- 172.21.15.115:0/1084632506 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806000ee90 con 0x7f80500047c0 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.717+0000 7f806c284640 1 -- 172.21.15.115:0/1084632506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806415bbe0 con 0x7f80500047c0 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.717+0000 7f805affd640 1 -- 172.21.15.115:0/1084632506 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806000c6d0 con 0x7f80500047c0 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.717+0000 7f806c284640 1 -- 172.21.15.115:0/1084632506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80641586b0 con 0x7f80500047c0 2023-12-01T02:52:42.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.718+0000 7f806c284640 1 -- 172.21.15.115:0/1084632506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806404e890 con 0x7f80500047c0 2023-12-01T02:52:42.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.719+0000 7f805affd640 1 -- 172.21.15.115:0/1084632506 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 99) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f806000c830 con 0x7f80500047c0 2023-12-01T02:52:42.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.721+0000 7f805affd640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f804806a300 0x7f804806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T02:52:42.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.721+0000 7f805affd640 1 -- 172.21.15.115:0/1084632506 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f80600884b0 con 0x7f80500047c0 2023-12-01T02:52:42.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.721+0000 7f806981f640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f804806a300 0x7f804806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T02:52:42.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.722+0000 7f806981f640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f804806a300 0x7f804806c7c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8054036000 tx=0x7f805402e000 comp rx=0 tx=0).ready entity=mgr.6194 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T02:52:42.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.726+0000 7f805affd640 1 -- 172.21.15.115:0/1084632506 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8060053b80 con 0x7f80500047c0 2023-12-01T02:52:42.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:42.940+0000 7f806c284640 1 -- 172.21.15.115:0/1084632506 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f80640cc350 con 0x7f804806a300 2023-12-01T02:52:43.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.052+0000 7f805affd640 1 -- 172.21.15.115:0/1084632506 <== mgr.6194 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f805402c9e0 con 0x7f804806a300 2023-12-01T02:52:43.055 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-12-01T02:52:43.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.056+0000 7f806c284640 1 -- 172.21.15.115:0/1084632506 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f804806a300 msgr2=0x7f804806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:43.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.056+0000 7f806c284640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f804806a300 0x7f804806c7c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8054036000 tx=0x7f805402e000 comp rx=0 tx=0).stop 2023-12-01T02:52:43.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.056+0000 7f806c284640 1 -- 172.21.15.115:0/1084632506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 msgr2=0x7f80641f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T02:52:43.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.056+0000 7f806c284640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 0x7f80641f2610 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f806000e7e0 tx=0x7f806000eca0 comp rx=0 tx=0).stop 2023-12-01T02:52:43.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.056+0000 7f806c284640 1 -- 172.21.15.115:0/1084632506 shutdown_connections 2023-12-01T02:52:43.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.056+0000 7f806c284640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f804806a300 0x7f804806c7c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:43.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.056+0000 7f806c284640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8064159480 0x7f8064157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:43.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.056+0000 7f806c284640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8064153630 0x7f8064158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:43.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.056+0000 7f806c284640 1 --2- 172.21.15.115:0/1084632506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80500047c0 0x7f80641f2610 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T02:52:43.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.056+0000 7f806c284640 1 -- 172.21.15.115:0/1084632506 >> 172.21.15.115:0/1084632506 conn(0x7f8064065960 msgr2=0x7f806406f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T02:52:43.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.057+0000 7f806c284640 1 -- 172.21.15.115:0/1084632506 shutdown_connections 2023-12-01T02:52:43.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T02:52:43.057+0000 7f806c284640 1 -- 172.21.15.115:0/1084632506 wait complete. 2023-12-01T02:52:43.087 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-12-01T02:52:52.561 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:52:52.571 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:53:00.081 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:53:00.079+0000 7f016cdc7640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:53:00.081 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:53:00.079+0000 7f016cdc7640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:53:00.081 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:53:00.079+0000 7f016cdc7640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:53:00.081 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:53:00.079+0000 7f016cdc7640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:53:00.081 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T02:53:00.079+0000 7f016cdc7640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T02:53:22.767 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:53:22.776 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:53:52.980 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:53:52.990 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:54:23.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:54:23.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:54:53.638 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:54:53.648 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:55:23.853 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:55:23.862 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:55:54.109 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:55:54.118 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:56:24.271 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:56:24.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:56:54.417 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:56:54.426 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:57:24.623 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:57:24.632 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:57:54.837 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:57:54.847 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:58:25.250 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:58:25.260 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:58:55.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:58:55.421 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:59:25.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:59:25.660 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:59:55.863 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T02:59:55.873 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:00:26.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:00:26.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:00:56.282 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:00:56.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:01:26.438 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:01:26.448 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:01:56.580 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:01:56.589 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:02:26.805 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:02:26.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:02:57.044 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:02:57.054 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:03:16.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:03:16.088 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-01T03:03:16.156 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_53_00_UTC 2023-12-01T03:03:16.156 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_54_00_UTC 2023-12-01T03:03:16.156 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_55_00_UTC 2023-12-01T03:03:16.156 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_56_00_UTC 2023-12-01T03:03:16.157 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_57_00_UTC 2023-12-01T03:03:16.157 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_58_00_UTC 2023-12-01T03:03:16.157 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_59_00_UTC 2023-12-01T03:03:16.157 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_00_00_UTC 2023-12-01T03:03:16.157 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_01_00_UTC 2023-12-01T03:03:16.157 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_02_00_UTC 2023-12-01T03:03:16.157 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_03_00_UTC 2023-12-01T03:03:16.159 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-02_53_00_UTC', 'scheduled-2023-12-01-02_54_00_UTC', 'scheduled-2023-12-01-02_55_00_UTC', 'scheduled-2023-12-01-02_56_00_UTC', 'scheduled-2023-12-01-02_57_00_UTC', 'scheduled-2023-12-01-02_58_00_UTC', 'scheduled-2023-12-01-02_59_00_UTC', 'scheduled-2023-12-01-03_00_00_UTC', 'scheduled-2023-12-01-03_01_00_UTC', 'scheduled-2023-12-01-03_02_00_UTC', 'scheduled-2023-12-01-03_03_00_UTC'] 2023-12-01T03:03:16.159 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-12-01T03:03:16.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.384+0000 7ff058bb9640 1 Processor -- start 2023-12-01T03:03:16.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.384+0000 7ff058bb9640 1 -- start start 2023-12-01T03:03:16.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.385+0000 7ff058bb9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff05415ac80 0x7ff0540cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.385+0000 7ff058bb9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff0540ce1b0 0x7ff0540b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.386+0000 7ff058bb9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0540cc9a0 con 0x7ff0540b0e60 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.386+0000 7ff058bb9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05415b060 con 0x7ff05415ac80 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.386+0000 7ff058bb9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0540b0ce0 con 0x7ff0540ce1b0 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.386+0000 7ff05259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff05415ac80 0x7ff0540cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.386+0000 7ff05259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff05415ac80 0x7ff0540cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44582/0 (socket says 172.21.15.115:44582) 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.386+0000 7ff05259c640 1 -- 172.21.15.115:0/984374453 learned_addr learned my addr 172.21.15.115:0/984374453 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.386+0000 7ff052d9d640 1 --2- 172.21.15.115:0/984374453 >> v2:172.21.15.142:3300/0 conn(0x7ff0540ce1b0 0x7ff0540b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.387+0000 7ff05259c640 1 -- 172.21.15.115:0/984374453 >> v2:172.21.15.142:3300/0 conn(0x7ff0540ce1b0 msgr2=0x7ff0540b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.387+0000 7ff05259c640 1 --2- 172.21.15.115:0/984374453 >> v2:172.21.15.142:3300/0 conn(0x7ff0540ce1b0 0x7ff0540b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.387+0000 7ff05259c640 1 -- 172.21.15.115:0/984374453 >> v1:172.21.15.142:6789/0 conn(0x7ff0540b0e60 legacy=0x7ff0540cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:03:16.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.387+0000 7ff05259c640 1 -- 172.21.15.115:0/984374453 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540cbd90 con 0x7ff05415ac80 2023-12-01T03:03:16.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.388+0000 7ff05259c640 1 --2- 172.21.15.115:0/984374453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05415ac80 0x7ff0540cce30 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7ff05406f1b0 tx=0x7ff04402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8541835f8b2c2c56 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:03:16.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.388+0000 7ff051d9b640 1 -- 172.21.15.115:0/984374453 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044031870 con 0x7ff05415ac80 2023-12-01T03:03:16.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.388+0000 7ff051d9b640 1 -- 172.21.15.115:0/984374453 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0440319d0 con 0x7ff05415ac80 2023-12-01T03:03:16.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.388+0000 7ff051d9b640 1 -- 172.21.15.115:0/984374453 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044031d60 con 0x7ff05415ac80 2023-12-01T03:03:16.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.389+0000 7ff058bb9640 1 -- 172.21.15.115:0/984374453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05415ac80 msgr2=0x7ff0540cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:16.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.389+0000 7ff058bb9640 1 --2- 172.21.15.115:0/984374453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05415ac80 0x7ff0540cce30 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7ff05406f1b0 tx=0x7ff04402b910 comp rx=0 tx=0).stop 2023-12-01T03:03:16.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.389+0000 7ff058bb9640 1 -- 172.21.15.115:0/984374453 shutdown_connections 2023-12-01T03:03:16.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.389+0000 7ff058bb9640 1 --2- 172.21.15.115:0/984374453 >> v2:172.21.15.142:3300/0 conn(0x7ff0540ce1b0 0x7ff0540b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.389+0000 7ff058bb9640 1 --2- 172.21.15.115:0/984374453 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff05415ac80 0x7ff0540cce30 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.389+0000 7ff058bb9640 1 -- 172.21.15.115:0/984374453 >> 172.21.15.115:0/984374453 conn(0x7ff054065960 msgr2=0x7ff054065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:03:16.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.389+0000 7ff058bb9640 1 -- 172.21.15.115:0/984374453 shutdown_connections 2023-12-01T03:03:16.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.389+0000 7ff058bb9640 1 -- 172.21.15.115:0/984374453 wait complete. 2023-12-01T03:03:16.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.391+0000 7ff058bb9640 1 Processor -- start 2023-12-01T03:03:16.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.391+0000 7ff058bb9640 1 -- start start 2023-12-01T03:03:16.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.392+0000 7ff058bb9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0540b0e60 0x7ff054158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.392+0000 7ff058bb9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0540ce1b0 0x7ff05414c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.392+0000 7ff058bb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff05415ac80 0x7ff0541f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.392+0000 7ff058bb9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0541f69e0 con 0x7ff0540b0e60 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.392+0000 7ff058bb9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff054158d00 con 0x7ff0540ce1b0 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.392+0000 7ff058bb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff054158fe0 con 0x7ff05415ac80 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.392+0000 7ff05259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0540b0e60 0x7ff054158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.393+0000 7ff05259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0540b0e60 0x7ff054158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44594/0 (socket says 172.21.15.115:44594) 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.393+0000 7ff05259c640 1 -- 172.21.15.115:0/3119882768 learned_addr learned my addr 172.21.15.115:0/3119882768 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.393+0000 7ff05259c640 1 -- 172.21.15.115:0/3119882768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff05415ac80 msgr2=0x7ff0541f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.393+0000 7ff05259c640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff05415ac80 0x7ff0541f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.393+0000 7ff05259c640 1 -- 172.21.15.115:0/3119882768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0540ce1b0 msgr2=0x7ff05414c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.393+0000 7ff05259c640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0540ce1b0 0x7ff05414c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.393+0000 7ff05259c640 1 -- 172.21.15.115:0/3119882768 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0440094d0 con 0x7ff0540b0e60 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.393+0000 7ff05259c640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0540b0e60 0x7ff054158720 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7ff044000c00 tx=0x7ff0440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.394+0000 7ff04affd640 1 -- 172.21.15.115:0/3119882768 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04403abe0 con 0x7ff0540b0e60 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.394+0000 7ff04affd640 1 -- 172.21.15.115:0/3119882768 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04403ad40 con 0x7ff0540b0e60 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.394+0000 7ff04affd640 1 -- 172.21.15.115:0/3119882768 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044044610 con 0x7ff0540b0e60 2023-12-01T03:03:16.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.394+0000 7ff058bb9640 1 -- 172.21.15.115:0/3119882768 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0541592c0 con 0x7ff0540b0e60 2023-12-01T03:03:16.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.394+0000 7ff058bb9640 1 -- 172.21.15.115:0/3119882768 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0540b1810 con 0x7ff0540b0e60 2023-12-01T03:03:16.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.395+0000 7ff058bb9640 1 -- 172.21.15.115:0/3119882768 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05404e890 con 0x7ff0540b0e60 2023-12-01T03:03:16.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.396+0000 7ff04affd640 1 -- 172.21.15.115:0/3119882768 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 103) v1 ==== 88738+0+0 (secure 0 0 0) 0x7ff04404d450 con 0x7ff0540b0e60 2023-12-01T03:03:16.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.396+0000 7ff04affd640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff03406a5d0 0x7ff03406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.397+0000 7ff04affd640 1 -- 172.21.15.115:0/3119882768 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6155+0+0 (secure 0 0 0) 0x7ff04402d040 con 0x7ff0540b0e60 2023-12-01T03:03:16.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.397+0000 7ff04bfff640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff03406a5d0 0x7ff03406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.397+0000 7ff04bfff640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff03406a5d0 0x7ff03406ca90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff054150960 tx=0x7ff03c0079b0 comp rx=0 tx=0).ready entity=mgr.6194 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:03:16.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.401+0000 7ff04affd640 1 -- 172.21.15.115:0/3119882768 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff044076570 con 0x7ff0540b0e60 2023-12-01T03:03:16.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.612+0000 7ff058bb9640 1 -- 172.21.15.115:0/3119882768 --> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff054159150 con 0x7ff03406a5d0 2023-12-01T03:03:16.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.626+0000 7ff04affd640 1 -- 172.21.15.115:0/3119882768 <== mgr.6194 v2:172.21.15.115:6822/88317 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7ff054159150 con 0x7ff03406a5d0 2023-12-01T03:03:16.629 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:03:16.629 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T02:52:42", "first": "2023-12-01T02:53:00", "last": "2023-12-01T03:03:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-12-01T03:03:16.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.629+0000 7ff058bb9640 1 -- 172.21.15.115:0/3119882768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff03406a5d0 msgr2=0x7ff03406ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:16.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.629+0000 7ff058bb9640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff03406a5d0 0x7ff03406ca90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff054150960 tx=0x7ff03c0079b0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.629+0000 7ff058bb9640 1 -- 172.21.15.115:0/3119882768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0540b0e60 msgr2=0x7ff054158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:16.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.629+0000 7ff058bb9640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0540b0e60 0x7ff054158720 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7ff044000c00 tx=0x7ff0440046c0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.629+0000 7ff058bb9640 1 -- 172.21.15.115:0/3119882768 shutdown_connections 2023-12-01T03:03:16.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.629+0000 7ff058bb9640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7ff03406a5d0 0x7ff03406ca90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.629+0000 7ff058bb9640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff05415ac80 0x7ff0541f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.629+0000 7ff058bb9640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0540ce1b0 0x7ff05414c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.629+0000 7ff058bb9640 1 --2- 172.21.15.115:0/3119882768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0540b0e60 0x7ff054158720 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.629+0000 7ff058bb9640 1 -- 172.21.15.115:0/3119882768 >> 172.21.15.115:0/3119882768 conn(0x7ff054065960 msgr2=0x7ff05415d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:03:16.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.630+0000 7ff058bb9640 1 -- 172.21.15.115:0/3119882768 shutdown_connections 2023-12-01T03:03:16.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.630+0000 7ff058bb9640 1 -- 172.21.15.115:0/3119882768 wait complete. 2023-12-01T03:03:16.650 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T02:52:42", "first": "2023-12-01T02:53:00", "last": "2023-12-01T03:03:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-12-01T03:03:16.651 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T02:52:42', 'first': '2023-12-01T02:53:00', 'last': '2023-12-01T03:03:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-12-01T03:03:16.651 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-01T03:03:16.651 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-01T03:03:16.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-01T03:03:16.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.863+0000 7f84afa4a640 1 Processor -- start 2023-12-01T03:03:16.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.863+0000 7f84afa4a640 1 -- start start 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.864+0000 7f84afa4a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f84a80cad70 0x7f84a806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.864+0000 7f84afa4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a806f2f0 0x7f84a806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.865+0000 7f84afa4a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a80add10 con 0x7f84a815e2f0 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.865+0000 7f84afa4a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a80c5760 con 0x7f84a80cad70 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.865+0000 7f84afa4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a815e6d0 con 0x7f84a806f2f0 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.865+0000 7f84ad7e6640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f84a815e2f0 0x7f84a80ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:49240/0 (socket says 172.21.15.115:49240) 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.865+0000 7f84ad7e6640 1 -- 172.21.15.115:0/1759878816 learned_addr learned my addr 172.21.15.115:0/1759878816 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.865+0000 7f84adfe7640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a806f2f0 0x7f84a806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.865+0000 7f84acfe5640 1 --2- 172.21.15.115:0/1759878816 >> v2:172.21.15.142:3300/0 conn(0x7f84a80cad70 0x7f84a806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.865+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262195461 0 0) 0x7f84a80add10 con 0x7f84a815e2f0 2023-12-01T03:03:16.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.865+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8490003520 con 0x7f84a815e2f0 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3467599340 0 0) 0x7f8490003520 con 0x7f84a815e2f0 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a806f2f0 msgr2=0x7f84a806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a806f2f0 0x7f84a806bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 >> v2:172.21.15.142:3300/0 conn(0x7f84a80cad70 msgr2=0x7f84a806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 --2- 172.21.15.115:0/1759878816 >> v2:172.21.15.142:3300/0 conn(0x7f84a80cad70 0x7f84a806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80cf7b0 con 0x7f84a815e2f0 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f84a0001670 con 0x7f84a815e2f0 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 >> v1:172.21.15.115:6789/0 conn(0x7f84a815e2f0 legacy=0x7f84a80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84900039e0 0x7f8490003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a806f2f0 0x7f84900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8490004d00 0x7f8490005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a80c5760 con 0x7f8490004d00 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a815e6d0 con 0x7f84900039e0 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849000bb30 con 0x7f84a806f2f0 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84acfe5640 1 --2- 172.21.15.115:0/1759878816 >> v2:172.21.15.142:3300/0 conn(0x7f84a80cad70 0x7f84a806b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:03:16.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.866+0000 7f84adfe7640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8490004d00 0x7f8490005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.867+0000 7f84adfe7640 1 -- 172.21.15.115:0/1759878816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a806f2f0 msgr2=0x7f84900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.867+0000 7f84adfe7640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a806f2f0 0x7f84900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.867+0000 7f84adfe7640 1 -- 172.21.15.115:0/1759878816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84900039e0 msgr2=0x7f8490003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.867+0000 7f84ad7e6640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84900039e0 0x7f8490003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.867+0000 7f84adfe7640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84900039e0 0x7f8490003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.867+0000 7f84adfe7640 1 -- 172.21.15.115:0/1759878816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8490003520 con 0x7f8490004d00 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.868+0000 7f84adfe7640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8490004d00 0x7f8490005180 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f8498009a90 tx=0x7f84900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0bb383783f831d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.868+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849802c6b0 con 0x7f8490004d00 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.868+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849802c810 con 0x7f8490004d00 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.868+0000 7f84a7fff640 1 -- 172.21.15.115:0/1759878816 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849802cba0 con 0x7f8490004d00 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.868+0000 7f84afa4a640 1 -- 172.21.15.115:0/1759878816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8490004d00 msgr2=0x7f8490005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.868+0000 7f84afa4a640 1 --2- 172.21.15.115:0/1759878816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8490004d00 0x7f8490005180 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f8498009a90 tx=0x7f84900057f0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.868+0000 7f84adfe7640 1 -- 172.21.15.115:0/1759878816 reap_dead start 2023-12-01T03:03:16.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.869+0000 7f84afa4a640 1 -- 172.21.15.115:0/1759878816 shutdown_connections 2023-12-01T03:03:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.869+0000 7f84afa4a640 1 -- 172.21.15.115:0/1759878816 >> 172.21.15.115:0/1759878816 conn(0x7f84a80ce880 msgr2=0x7f84a80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:03:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.869+0000 7f84afa4a640 1 -- 172.21.15.115:0/1759878816 shutdown_connections 2023-12-01T03:03:16.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.869+0000 7f84afa4a640 1 -- 172.21.15.115:0/1759878816 wait complete. 2023-12-01T03:03:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.870+0000 7f84afa4a640 1 Processor -- start 2023-12-01T03:03:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.871+0000 7f84afa4a640 1 -- start start 2023-12-01T03:03:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.871+0000 7f84afa4a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8490004d00 0x7f84a80c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84afa4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a80b1520 0x7f84a80b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84afa4a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a80c80b0 0x7f84a80c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84afa4a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a80c7c20 con 0x7f84a80c80b0 2023-12-01T03:03:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84afa4a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a80c7010 con 0x7f8490004d00 2023-12-01T03:03:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84afa4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a80c72f0 con 0x7f84a80b1520 2023-12-01T03:03:16.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84adfe7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a80c80b0 0x7f84a80c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84adfe7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a80c80b0 0x7f84a80c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:44610/0 (socket says 172.21.15.115:44610) 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84adfe7640 1 -- 172.21.15.115:0/3267993272 learned_addr learned my addr 172.21.15.115:0/3267993272 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84ad7e6640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8490004d00 0x7f84a80c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84adfe7640 1 -- 172.21.15.115:0/3267993272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a80b1520 msgr2=0x7f84a80b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84adfe7640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a80b1520 0x7f84a80b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84adfe7640 1 -- 172.21.15.115:0/3267993272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8490004d00 msgr2=0x7f84a80c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84adfe7640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8490004d00 0x7f84a80c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.872+0000 7f84adfe7640 1 -- 172.21.15.115:0/3267993272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80c75d0 con 0x7f84a80c80b0 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.873+0000 7f84adfe7640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a80c80b0 0x7f84a80c8530 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f8498037040 tx=0x7f8498004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.873+0000 7f84a5ffb640 1 -- 172.21.15.115:0/3267993272 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84980044e0 con 0x7f84a80c80b0 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.873+0000 7f84a5ffb640 1 -- 172.21.15.115:0/3267993272 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849802de50 con 0x7f84a80c80b0 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.873+0000 7f84a5ffb640 1 -- 172.21.15.115:0/3267993272 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498003830 con 0x7f84a80c80b0 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.873+0000 7f84afa4a640 1 -- 172.21.15.115:0/3267993272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a80c78b0 con 0x7f84a80c80b0 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.873+0000 7f84afa4a640 1 -- 172.21.15.115:0/3267993272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a80c51a0 con 0x7f84a80c80b0 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.874+0000 7f84afa4a640 1 -- 172.21.15.115:0/3267993272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a806fda0 con 0x7f84a80c80b0 2023-12-01T03:03:16.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.875+0000 7f84a5ffb640 1 -- 172.21.15.115:0/3267993272 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 103) v1 ==== 88738+0+0 (secure 0 0 0) 0x7f8498003990 con 0x7f84a80c80b0 2023-12-01T03:03:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.876+0000 7f84a5ffb640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a5a0 0x7f849406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:03:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.876+0000 7f84a5ffb640 1 -- 172.21.15.115:0/3267993272 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f84980aa270 con 0x7f84a80c80b0 2023-12-01T03:03:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.876+0000 7f84ad7e6640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a5a0 0x7f849406ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:03:16.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.877+0000 7f84ad7e6640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a5a0 0x7f849406ca60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84a0000f60 tx=0x7f84a0009040 comp rx=0 tx=0).ready entity=mgr.6194 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:03:16.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:16.880+0000 7f84a5ffb640 1 -- 172.21.15.115:0/3267993272 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8498075580 con 0x7f84a80c80b0 2023-12-01T03:03:17.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.122+0000 7f84afa4a640 1 -- 172.21.15.115:0/3267993272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f84a80c51a0 con 0x7f84a80c80b0 2023-12-01T03:03:17.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.127+0000 7f84a5ffb640 1 -- 172.21.15.115:0/3267993272 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v103) v1 ==== 74+0+170177 (secure 0 0 0) 0x7f84a80c51a0 con 0x7f84a80c80b0 2023-12-01T03:03:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 -- 172.21.15.115:0/3267993272 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a5a0 msgr2=0x7f849406ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a5a0 0x7f849406ca60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84a0000f60 tx=0x7f84a0009040 comp rx=0 tx=0).stop 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 -- 172.21.15.115:0/3267993272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a80c80b0 msgr2=0x7f84a80c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a80c80b0 0x7f84a80c8530 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f8498037040 tx=0x7f8498004730 comp rx=0 tx=0).stop 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 -- 172.21.15.115:0/3267993272 shutdown_connections 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.115:6822/88317,v1:172.21.15.115:6824/88317] conn(0x7f849406a5a0 0x7f849406ca60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84a80b1520 0x7f84a80b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a80c80b0 0x7f84a80c8530 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 --2- 172.21.15.115:0/3267993272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8490004d00 0x7f84a80c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 -- 172.21.15.115:0/3267993272 >> 172.21.15.115:0/3267993272 conn(0x7f84a80ce880 msgr2=0x7f84a80c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:03:17.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 -- 172.21.15.115:0/3267993272 shutdown_connections 2023-12-01T03:03:17.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:03:17.131+0000 7f84afa4a640 1 -- 172.21.15.115:0/3267993272 wait complete. 2023-12-01T03:03:17.150 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":103,"active_gid":6194,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":88317},{"type":"v1","addr":"172.21.15.115:6824","nonce":88317}]},"active_addr":"172.21.15.115:6824/88317","active_change":"2023-12-01T02:52:41.228503+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":6191,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":178,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":1462551426}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":2970056045}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":718367345}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":3219939439}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":363147954}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":858082591}]}]} 2023-12-01T03:03:17.219 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: y 2023-12-01T03:03:17.219 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-01T03:03:17.219 INFO:tasks.ceph.mgr.y:Stopping old one... 2023-12-01T03:03:17.219 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-01T03:03:17.219 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:03:17.294 INFO:tasks.ceph.mgr.y:Stopped 2023-12-01T03:03:17.294 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-01T03:03:17.300 INFO:tasks.ceph.mgr.y:Started 2023-12-01T03:03:17.890 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:17.888+0000 7f29d6c25040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:03:17.943 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:17.941+0000 7f29d6c25040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:03:18.392 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:18.390+0000 7f29d6c25040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:03:18.444 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:18.442+0000 7f29d6c25040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:03:18.494 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:18.492+0000 7f29d6c25040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:03:18.554 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:18.552+0000 7f29d6c25040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:03:18.602 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:18.601+0000 7f29d6c25040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:03:18.713 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:18.711+0000 7f29d6c25040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:03:18.775 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:18.773+0000 7f29d6c25040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:03:18.824 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:18.822+0000 7f29d6c25040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:03:18.880 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:18.878+0000 7f29d6c25040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:03:18.930 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:18.928+0000 7f29d6c25040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:03:19.309 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:19.307+0000 7f29d6c25040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:03:19.469 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:19.467+0000 7f29d6c25040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:03:19.578 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:19.575+0000 7f29d6c25040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:03:19.626 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:19.624+0000 7f29d6c25040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:03:19.734 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:19.732+0000 7f29d6c25040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:03:19.840 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:19.839+0000 7f29d6c25040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:03:19.996 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:19.994+0000 7f29d6c25040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:03:20.046 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:20.044+0000 7f29d6c25040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:03:20.163 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:20.161+0000 7f29d6c25040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:03:20.988 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:20.986+0000 7f29d6c25040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:03:21.283 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.281+0000 7f29d6c25040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:03:21.340 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.337+0000 7f29d6c25040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:03:21.453 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.451+0000 7f29d6c25040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:03:21.761 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.758+0000 7f2954cc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:03:21.761 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.758+0000 7f2954cc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:03:21.762 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.758+0000 7f2954cc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:03:21.762 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.758+0000 7f2954cc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:03:21.762 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.758+0000 7f2954cc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:03:21.762 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.759+0000 7f2952cc2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:03:21.762 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.759+0000 7f2952cc2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:03:21.762 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.759+0000 7f2952cc2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:03:21.762 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.759+0000 7f2952cc2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:03:21.763 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:03:21.759+0000 7f2952cc2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:03:27.250 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:03:27.260 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:03:57.388 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:03:57.398 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:04:00.753 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:04:00.751+0000 7f295dd18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:04:00.754 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:04:00.751+0000 7f295dd18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:04:00.754 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:04:00.751+0000 7f295dd18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:04:00.754 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:04:00.751+0000 7f295dd18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:04:00.754 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:04:00.751+0000 7f295dd18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:04:27.619 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:04:27.629 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:04:57.975 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:04:57.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:05:28.219 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:05:28.228 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:05:58.440 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:05:58.450 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:06:28.594 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:06:28.603 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:06:58.792 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:06:58.802 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:07:29.021 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:07:29.031 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:07:59.244 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:07:59.253 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:08:17.301 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/test_restart 1M 2023-12-01T03:08:17.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.527+0000 7f35bc884640 1 Processor -- start 2023-12-01T03:08:17.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.527+0000 7f35bc884640 1 -- start start 2023-12-01T03:08:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.528+0000 7f35bc884640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f35b806b900 0x7f35b80cb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.528+0000 7f35bc884640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f35b80d0c40 0x7f35b80d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.528+0000 7f35bc884640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b80d07b0 con 0x7f35b80ccca0 2023-12-01T03:08:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.528+0000 7f35bc884640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b806af00 con 0x7f35b80d0c40 2023-12-01T03:08:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.528+0000 7f35bc884640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b80d0160 con 0x7f35b806b900 2023-12-01T03:08:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.529+0000 7f35b6d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f35b80d0c40 0x7f35b80d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.529+0000 7f35b6d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f35b80d0c40 0x7f35b80d10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36670/0 (socket says 172.21.15.115:36670) 2023-12-01T03:08:17.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.529+0000 7f35b6d9d640 1 -- 172.21.15.115:0/1063478410 learned_addr learned my addr 172.21.15.115:0/1063478410 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:17.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.529+0000 7f35affff640 1 --2- 172.21.15.115:0/1063478410 >> v2:172.21.15.142:3300/0 conn(0x7f35b806b900 0x7f35b80cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:17.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.529+0000 7f35b6d9d640 1 -- 172.21.15.115:0/1063478410 >> v2:172.21.15.142:3300/0 conn(0x7f35b806b900 msgr2=0x7f35b80cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:17.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.529+0000 7f35b6d9d640 1 --2- 172.21.15.115:0/1063478410 >> v2:172.21.15.142:3300/0 conn(0x7f35b806b900 0x7f35b80cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.529+0000 7f35b6d9d640 1 -- 172.21.15.115:0/1063478410 >> v1:172.21.15.142:6789/0 conn(0x7f35b80ccca0 legacy=0x7f35b806a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:17.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.529+0000 7f35b6d9d640 1 -- 172.21.15.115:0/1063478410 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b80d0440 con 0x7f35b80d0c40 2023-12-01T03:08:17.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.531+0000 7f35b6d9d640 1 --2- 172.21.15.115:0/1063478410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b80d0c40 0x7f35b80d10b0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f35b80d1af0 tx=0x7f35b002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42f8cc46aca0451 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:17.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.531+0000 7f35b5d9b640 1 -- 172.21.15.115:0/1063478410 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b002ca80 con 0x7f35b80d0c40 2023-12-01T03:08:17.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.531+0000 7f35b5d9b640 1 -- 172.21.15.115:0/1063478410 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b002cbe0 con 0x7f35b80d0c40 2023-12-01T03:08:17.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.531+0000 7f35b5d9b640 1 -- 172.21.15.115:0/1063478410 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00343f0 con 0x7f35b80d0c40 2023-12-01T03:08:17.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.532+0000 7f35bc884640 1 -- 172.21.15.115:0/1063478410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b80d0c40 msgr2=0x7f35b80d10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:17.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.532+0000 7f35bc884640 1 --2- 172.21.15.115:0/1063478410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b80d0c40 0x7f35b80d10b0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f35b80d1af0 tx=0x7f35b002bee0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.532+0000 7f35bc884640 1 -- 172.21.15.115:0/1063478410 shutdown_connections 2023-12-01T03:08:17.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.532+0000 7f35bc884640 1 --2- 172.21.15.115:0/1063478410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b80d0c40 0x7f35b80d10b0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.532+0000 7f35bc884640 1 --2- 172.21.15.115:0/1063478410 >> v2:172.21.15.142:3300/0 conn(0x7f35b806b900 0x7f35b80cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.532+0000 7f35bc884640 1 -- 172.21.15.115:0/1063478410 >> 172.21.15.115:0/1063478410 conn(0x7f35b80713d0 msgr2=0x7f35b80717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:17.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.532+0000 7f35bc884640 1 -- 172.21.15.115:0/1063478410 shutdown_connections 2023-12-01T03:08:17.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.532+0000 7f35bc884640 1 -- 172.21.15.115:0/1063478410 wait complete. 2023-12-01T03:08:17.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.534+0000 7f35bc884640 1 Processor -- start 2023-12-01T03:08:17.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.534+0000 7f35bc884640 1 -- start start 2023-12-01T03:08:17.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.535+0000 7f35bc884640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b806b900 0x7f35b8150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:17.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.535+0000 7f35bc884640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35b80ccca0 0x7f35b80ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:17.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.535+0000 7f35bc884640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b80d0c40 0x7f35b80aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:17.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.535+0000 7f35bc884640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80adf70 con 0x7f35b806b900 2023-12-01T03:08:17.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.535+0000 7f35bc884640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80b3540 con 0x7f35b80ccca0 2023-12-01T03:08:17.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.535+0000 7f35bc884640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80b3820 con 0x7f35b80d0c40 2023-12-01T03:08:17.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.535+0000 7f35b659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b806b900 0x7f35b8150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:17.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.535+0000 7f35b659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b806b900 0x7f35b8150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:36684/0 (socket says 172.21.15.115:36684) 2023-12-01T03:08:17.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.535+0000 7f35b659c640 1 -- 172.21.15.115:0/2939251601 learned_addr learned my addr 172.21.15.115:0/2939251601 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:17.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35affff640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35b80ccca0 0x7f35b80ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:17.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35b6d9d640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b80d0c40 0x7f35b80aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:17.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35b659c640 1 -- 172.21.15.115:0/2939251601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b80d0c40 msgr2=0x7f35b80aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:17.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35b659c640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b80d0c40 0x7f35b80aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35b659c640 1 -- 172.21.15.115:0/2939251601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35b80ccca0 msgr2=0x7f35b80ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:17.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35b659c640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35b80ccca0 0x7f35b80ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35b659c640 1 -- 172.21.15.115:0/2939251601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b0009b30 con 0x7f35b806b900 2023-12-01T03:08:17.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35b659c640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b806b900 0x7f35b8150e20 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c430 tx=0x7f35a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:17.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35aeffd640 1 -- 172.21.15.115:0/2939251601 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8011790 con 0x7f35b806b900 2023-12-01T03:08:17.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35aeffd640 1 -- 172.21.15.115:0/2939251601 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a80118f0 con 0x7f35b806b900 2023-12-01T03:08:17.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35aeffd640 1 -- 172.21.15.115:0/2939251601 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8011ce0 con 0x7f35b806b900 2023-12-01T03:08:17.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.536+0000 7f35bc884640 1 -- 172.21.15.115:0/2939251601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b80b3b00 con 0x7f35b806b900 2023-12-01T03:08:17.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.537+0000 7f35bc884640 1 -- 172.21.15.115:0/2939251601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b8151360 con 0x7f35b806b900 2023-12-01T03:08:17.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.538+0000 7f35bc884640 1 -- 172.21.15.115:0/2939251601 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b806be50 con 0x7f35b806b900 2023-12-01T03:08:17.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.539+0000 7f35aeffd640 1 -- 172.21.15.115:0/2939251601 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 109) v1 ==== 88738+0+0 (secure 0 0 0) 0x7f35a801c460 con 0x7f35b806b900 2023-12-01T03:08:17.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.540+0000 7f35aeffd640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f359806a5d0 0x7f359806ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:17.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.540+0000 7f35aeffd640 1 -- 172.21.15.115:0/2939251601 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6413+0+0 (secure 0 0 0) 0x7f35a8088da0 con 0x7f35b806b900 2023-12-01T03:08:17.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.540+0000 7f35affff640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f359806a5d0 0x7f359806ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:17.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.541+0000 7f35affff640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f359806a5d0 0x7f359806ca90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f35b80d1440 tx=0x7f35a0008040 comp rx=0 tx=0).ready entity=mgr.9819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:17.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.545+0000 7f35aeffd640 1 -- 172.21.15.115:0/2939251601 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f35a8053f20 con 0x7f35b806b900 2023-12-01T03:08:17.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.762+0000 7f35bc884640 1 -- 172.21.15.115:0/2939251601 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f35b80b36b0 con 0x7f359806a5d0 2023-12-01T03:08:17.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.909+0000 7f35aeffd640 1 -- 172.21.15.115:0/2939251601 <== mgr.9819 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f35b80b36b0 con 0x7f359806a5d0 2023-12-01T03:08:17.911 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-12-01T03:08:17.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 -- 172.21.15.115:0/2939251601 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f359806a5d0 msgr2=0x7f359806ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:17.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f359806a5d0 0x7f359806ca90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f35b80d1440 tx=0x7f35a0008040 comp rx=0 tx=0).stop 2023-12-01T03:08:17.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 -- 172.21.15.115:0/2939251601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b806b900 msgr2=0x7f35b8150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:17.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b806b900 0x7f35b8150e20 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c430 tx=0x7f35a800c8f0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 -- 172.21.15.115:0/2939251601 shutdown_connections 2023-12-01T03:08:17.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f359806a5d0 0x7f359806ca90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35b80d0c40 0x7f35b80aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35b80ccca0 0x7f35b80ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 --2- 172.21.15.115:0/2939251601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35b806b900 0x7f35b8150e20 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 -- 172.21.15.115:0/2939251601 >> 172.21.15.115:0/2939251601 conn(0x7f35b80713d0 msgr2=0x7f35b80b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.912+0000 7f35bc884640 1 -- 172.21.15.115:0/2939251601 shutdown_connections 2023-12-01T03:08:17.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:17.913+0000 7f35bc884640 1 -- 172.21.15.115:0/2939251601 wait complete. 2023-12-01T03:08:17.935 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-12-01T03:08:17.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:17.935 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-01T03:08:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_53_00_UTC 2023-12-01T03:08:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_54_00_UTC 2023-12-01T03:08:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_55_00_UTC 2023-12-01T03:08:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_56_00_UTC 2023-12-01T03:08:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_57_00_UTC 2023-12-01T03:08:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_58_00_UTC 2023-12-01T03:08:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_59_00_UTC 2023-12-01T03:08:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_00_00_UTC 2023-12-01T03:08:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_01_00_UTC 2023-12-01T03:08:17.988 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_02_00_UTC 2023-12-01T03:08:17.988 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_03_00_UTC 2023-12-01T03:08:17.988 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_04_00_UTC 2023-12-01T03:08:17.988 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_05_00_UTC 2023-12-01T03:08:17.988 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_06_00_UTC 2023-12-01T03:08:17.988 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_07_00_UTC 2023-12-01T03:08:17.988 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_08_00_UTC 2023-12-01T03:08:17.989 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-01-02_53_00_UTC', 'scheduled-2023-12-01-02_54_00_UTC', 'scheduled-2023-12-01-02_55_00_UTC', 'scheduled-2023-12-01-02_56_00_UTC', 'scheduled-2023-12-01-02_57_00_UTC', 'scheduled-2023-12-01-02_58_00_UTC', 'scheduled-2023-12-01-02_59_00_UTC', 'scheduled-2023-12-01-03_00_00_UTC', 'scheduled-2023-12-01-03_01_00_UTC', 'scheduled-2023-12-01-03_02_00_UTC', 'scheduled-2023-12-01-03_03_00_UTC', 'scheduled-2023-12-01-03_04_00_UTC', 'scheduled-2023-12-01-03_05_00_UTC', 'scheduled-2023-12-01-03_06_00_UTC', 'scheduled-2023-12-01-03_07_00_UTC', 'scheduled-2023-12-01-03_08_00_UTC'] 2023-12-01T03:08:17.989 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-12-01T03:08:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.202+0000 7f30dae43640 1 Processor -- start 2023-12-01T03:08:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.202+0000 7f30dae43640 1 -- start start 2023-12-01T03:08:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.203+0000 7f30dae43640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f30d4142870 0x7f30d40ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.203+0000 7f30dae43640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f30d40b1e20 0x7f30d415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.203+0000 7f30dae43640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d40b2f30 con 0x7f30d40cd010 2023-12-01T03:08:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.203+0000 7f30dae43640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d40b3ae0 con 0x7f30d4142870 2023-12-01T03:08:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.203+0000 7f30dae43640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d40cd4a0 con 0x7f30d40b1e20 2023-12-01T03:08:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.204+0000 7f30d8bdf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f30d4142870 0x7f30d40ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.204+0000 7f30d8bdf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f30d4142870 0x7f30d40ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35308/0 (socket says 172.21.15.115:35308) 2023-12-01T03:08:18.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.204+0000 7f30d3fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f30d40cd010 0x7f30d40b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57060/0 (socket says 172.21.15.115:57060) 2023-12-01T03:08:18.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.204+0000 7f30d8bdf640 1 -- 172.21.15.115:0/1521467178 learned_addr learned my addr 172.21.15.115:0/1521467178 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:18.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.204+0000 7f30d93e0640 1 --2- 172.21.15.115:0/1521467178 >> v2:172.21.15.142:3300/0 conn(0x7f30d40b1e20 0x7f30d415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.204+0000 7f30d8bdf640 1 -- 172.21.15.115:0/1521467178 >> v2:172.21.15.142:3300/0 conn(0x7f30d40b1e20 msgr2=0x7f30d415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:18.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.204+0000 7f30d8bdf640 1 --2- 172.21.15.115:0/1521467178 >> v2:172.21.15.142:3300/0 conn(0x7f30d40b1e20 0x7f30d415a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.204+0000 7f30d8bdf640 1 -- 172.21.15.115:0/1521467178 >> v1:172.21.15.115:6789/0 conn(0x7f30d40cd010 legacy=0x7f30d40b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.204+0000 7f30d8bdf640 1 -- 172.21.15.115:0/1521467178 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d40b2320 con 0x7f30d4142870 2023-12-01T03:08:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.204+0000 7f30d37fe640 1 -- 172.21.15.115:0/1521467178 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496798623 0 0) 0x7f30d40b2f30 con 0x7f30d40cd010 2023-12-01T03:08:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.205+0000 7f30d8bdf640 1 --2- 172.21.15.115:0/1521467178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d4142870 0x7f30d40ccad0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f30d406efe0 tx=0x7f30c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2566a890588767 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:18.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30d37fe640 1 -- 172.21.15.115:0/1521467178 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8030860 con 0x7f30d4142870 2023-12-01T03:08:18.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30d37fe640 1 -- 172.21.15.115:0/1521467178 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c80309c0 con 0x7f30d4142870 2023-12-01T03:08:18.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30d37fe640 1 -- 172.21.15.115:0/1521467178 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8030d50 con 0x7f30d4142870 2023-12-01T03:08:18.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30dae43640 1 -- 172.21.15.115:0/1521467178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d4142870 msgr2=0x7f30d40ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:18.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30dae43640 1 --2- 172.21.15.115:0/1521467178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d4142870 0x7f30d40ccad0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f30d406efe0 tx=0x7f30c802b9a0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30dae43640 1 -- 172.21.15.115:0/1521467178 shutdown_connections 2023-12-01T03:08:18.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30dae43640 1 --2- 172.21.15.115:0/1521467178 >> v2:172.21.15.142:3300/0 conn(0x7f30d40b1e20 0x7f30d415a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30dae43640 1 --2- 172.21.15.115:0/1521467178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d4142870 0x7f30d40ccad0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30dae43640 1 -- 172.21.15.115:0/1521467178 >> 172.21.15.115:0/1521467178 conn(0x7f30d4065960 msgr2=0x7f30d4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:18.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30dae43640 1 -- 172.21.15.115:0/1521467178 shutdown_connections 2023-12-01T03:08:18.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.206+0000 7f30dae43640 1 -- 172.21.15.115:0/1521467178 wait complete. 2023-12-01T03:08:18.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.208+0000 7f30dae43640 1 Processor -- start 2023-12-01T03:08:18.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.208+0000 7f30dae43640 1 -- start start 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.209+0000 7f30dae43640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d40b1e20 0x7f30d4158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.209+0000 7f30dae43640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d40cd010 0x7f30d415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.209+0000 7f30dae43640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d4142870 0x7f30d415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.209+0000 7f30dae43640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4070af0 con 0x7f30d40b1e20 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.209+0000 7f30dae43640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d41f19c0 con 0x7f30d4142870 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.209+0000 7f30dae43640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d415e2e0 con 0x7f30d40cd010 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.209+0000 7f30d8bdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d40b1e20 0x7f30d4158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.210+0000 7f30d8bdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d40b1e20 0x7f30d4158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35314/0 (socket says 172.21.15.115:35314) 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.210+0000 7f30d8bdf640 1 -- 172.21.15.115:0/4096922359 learned_addr learned my addr 172.21.15.115:0/4096922359 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.210+0000 7f30d8bdf640 1 -- 172.21.15.115:0/4096922359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d40cd010 msgr2=0x7f30d415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:18.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.210+0000 7f30d8bdf640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d40cd010 0x7f30d415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.210+0000 7f30d93e0640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d4142870 0x7f30d415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.210+0000 7f30d8bdf640 1 -- 172.21.15.115:0/4096922359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d4142870 msgr2=0x7f30d415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:18.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.210+0000 7f30d8bdf640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d4142870 0x7f30d415dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.210+0000 7f30d8bdf640 1 -- 172.21.15.115:0/4096922359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c80094d0 con 0x7f30d40b1e20 2023-12-01T03:08:18.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.210+0000 7f30d8bdf640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d40b1e20 0x7f30d4158650 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f30c802b9e0 tx=0x7f30c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:18.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.210+0000 7f30d17fa640 1 -- 172.21.15.115:0/4096922359 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8039b60 con 0x7f30d40b1e20 2023-12-01T03:08:18.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.211+0000 7f30d17fa640 1 -- 172.21.15.115:0/4096922359 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c8039ce0 con 0x7f30d40b1e20 2023-12-01T03:08:18.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.211+0000 7f30d17fa640 1 -- 172.21.15.115:0/4096922359 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8043650 con 0x7f30d40b1e20 2023-12-01T03:08:18.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.211+0000 7f30dae43640 1 -- 172.21.15.115:0/4096922359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d41f0db0 con 0x7f30d40b1e20 2023-12-01T03:08:18.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.211+0000 7f30dae43640 1 -- 172.21.15.115:0/4096922359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d41f1370 con 0x7f30d40b1e20 2023-12-01T03:08:18.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.212+0000 7f30dae43640 1 -- 172.21.15.115:0/4096922359 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d404e890 con 0x7f30d40b1e20 2023-12-01T03:08:18.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.213+0000 7f30d17fa640 1 -- 172.21.15.115:0/4096922359 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 109) v1 ==== 88738+0+0 (secure 0 0 0) 0x7f30c802f3b0 con 0x7f30d40b1e20 2023-12-01T03:08:18.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.214+0000 7f30d17fa640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f30b806a5a0 0x7f30b806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.214+0000 7f30d17fa640 1 -- 172.21.15.115:0/4096922359 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6413+0+0 (secure 0 0 0) 0x7f30c80ab1b0 con 0x7f30d40b1e20 2023-12-01T03:08:18.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.214+0000 7f30d3fff640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f30b806a5a0 0x7f30b806ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.215+0000 7f30d3fff640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f30b806a5a0 0x7f30b806ca60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f30c0007fd0 tx=0x7f30c0007920 comp rx=0 tx=0).ready entity=mgr.9819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:18.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.219+0000 7f30d17fa640 1 -- 172.21.15.115:0/4096922359 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f30c80762c0 con 0x7f30d40b1e20 2023-12-01T03:08:18.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.431+0000 7f30dae43640 1 -- 172.21.15.115:0/4096922359 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30d40b2600 con 0x7f30b806a5a0 2023-12-01T03:08:18.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.447+0000 7f30d17fa640 1 -- 172.21.15.115:0/4096922359 <== mgr.9819 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f30c00070d0 con 0x7f30b806a5a0 2023-12-01T03:08:18.449 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:18.449 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T02:52:42", "first": "2023-12-01T02:53:00", "last": "2023-12-01T03:08:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-12-01T03:08:18.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.449+0000 7f30dae43640 1 -- 172.21.15.115:0/4096922359 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f30b806a5a0 msgr2=0x7f30b806ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:18.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.449+0000 7f30dae43640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f30b806a5a0 0x7f30b806ca60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f30c0007fd0 tx=0x7f30c0007920 comp rx=0 tx=0).stop 2023-12-01T03:08:18.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.449+0000 7f30dae43640 1 -- 172.21.15.115:0/4096922359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d40b1e20 msgr2=0x7f30d4158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:18.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.449+0000 7f30dae43640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d40b1e20 0x7f30d4158650 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f30c802b9e0 tx=0x7f30c8004730 comp rx=0 tx=0).stop 2023-12-01T03:08:18.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.450+0000 7f30dae43640 1 -- 172.21.15.115:0/4096922359 shutdown_connections 2023-12-01T03:08:18.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.450+0000 7f30dae43640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f30b806a5a0 0x7f30b806ca60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.450+0000 7f30dae43640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f30d40cd010 0x7f30d415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.450+0000 7f30dae43640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30d4142870 0x7f30d415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.450+0000 7f30dae43640 1 --2- 172.21.15.115:0/4096922359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30d40b1e20 0x7f30d4158650 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.450+0000 7f30dae43640 1 -- 172.21.15.115:0/4096922359 >> 172.21.15.115:0/4096922359 conn(0x7f30d4065960 msgr2=0x7f30d415a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:18.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.450+0000 7f30dae43640 1 -- 172.21.15.115:0/4096922359 shutdown_connections 2023-12-01T03:08:18.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.450+0000 7f30dae43640 1 -- 172.21.15.115:0/4096922359 wait complete. 2023-12-01T03:08:18.469 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T02:52:42", "first": "2023-12-01T02:53:00", "last": "2023-12-01T03:08:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-12-01T03:08:18.470 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-12-01T00:00:00', 'created': '2023-12-01T02:52:42', 'first': '2023-12-01T02:53:00', 'last': '2023-12-01T03:08:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-12-01T03:08:18.470 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-12-01T03:08:18.470 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-12-01T03:08:18.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/test_restart 1M 2023-12-01T03:08:18.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.686+0000 7f6407316640 1 Processor -- start 2023-12-01T03:08:18.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.686+0000 7f6407316640 1 -- start start 2023-12-01T03:08:18.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.688+0000 7f6407316640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f640015ac80 0x7f64000cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.688+0000 7f6407316640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f64000b0e60 0x7f64000b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.688+0000 7f6407316640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64000cc9a0 con 0x7f64000cdaa0 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.688+0000 7f6407316640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640015b060 con 0x7f64000b0e60 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.688+0000 7f6407316640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64000b0ce0 con 0x7f640015ac80 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.688+0000 7f64048b1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f64000b0e60 0x7f64000b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.688+0000 7f64048b1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f64000b0e60 0x7f64000b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35320/0 (socket says 172.21.15.115:35320) 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.688+0000 7f64048b1640 1 -- 172.21.15.115:0/2824883870 learned_addr learned my addr 172.21.15.115:0/2824883870 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.689+0000 7f64048b1640 1 -- 172.21.15.115:0/2824883870 >> v2:172.21.15.142:3300/0 conn(0x7f640015ac80 msgr2=0x7f64000cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.688+0000 7f64050b2640 1 --2- 172.21.15.115:0/2824883870 >> v2:172.21.15.142:3300/0 conn(0x7f640015ac80 0x7f64000cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.689+0000 7f64048b1640 1 --2- 172.21.15.115:0/2824883870 >> v2:172.21.15.142:3300/0 conn(0x7f640015ac80 0x7f64000cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.689+0000 7f64048b1640 1 -- 172.21.15.115:0/2824883870 >> v1:172.21.15.142:6789/0 conn(0x7f64000cdaa0 legacy=0x7f640006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.689+0000 7f64048b1640 1 -- 172.21.15.115:0/2824883870 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64000cbd90 con 0x7f64000b0e60 2023-12-01T03:08:18.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.689+0000 7f64050b2640 1 --2- 172.21.15.115:0/2824883870 >> v2:172.21.15.142:3300/0 conn(0x7f640015ac80 0x7f64000cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:18.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.690+0000 7f64048b1640 1 --2- 172.21.15.115:0/2824883870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64000b0e60 0x7f64000b07a0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f64000b3cc0 tx=0x7f63f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9479efc8d0f58ef2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:18.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.690+0000 7f63f7fff640 1 -- 172.21.15.115:0/2824883870 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0031870 con 0x7f64000b0e60 2023-12-01T03:08:18.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.690+0000 7f63f7fff640 1 -- 172.21.15.115:0/2824883870 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f00319d0 con 0x7f64000b0e60 2023-12-01T03:08:18.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.690+0000 7f63f7fff640 1 -- 172.21.15.115:0/2824883870 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0031d60 con 0x7f64000b0e60 2023-12-01T03:08:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.691+0000 7f6407316640 1 -- 172.21.15.115:0/2824883870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64000b0e60 msgr2=0x7f64000b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.691+0000 7f6407316640 1 --2- 172.21.15.115:0/2824883870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64000b0e60 0x7f64000b07a0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f64000b3cc0 tx=0x7f63f002b910 comp rx=0 tx=0).stop 2023-12-01T03:08:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.691+0000 7f6407316640 1 -- 172.21.15.115:0/2824883870 shutdown_connections 2023-12-01T03:08:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.691+0000 7f6407316640 1 --2- 172.21.15.115:0/2824883870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64000b0e60 0x7f64000b07a0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.691+0000 7f6407316640 1 --2- 172.21.15.115:0/2824883870 >> v2:172.21.15.142:3300/0 conn(0x7f640015ac80 0x7f64000cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.691+0000 7f6407316640 1 -- 172.21.15.115:0/2824883870 >> 172.21.15.115:0/2824883870 conn(0x7f6400065960 msgr2=0x7f6400065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.691+0000 7f6407316640 1 -- 172.21.15.115:0/2824883870 shutdown_connections 2023-12-01T03:08:18.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.691+0000 7f6407316640 1 -- 172.21.15.115:0/2824883870 wait complete. 2023-12-01T03:08:18.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.693+0000 7f6407316640 1 Processor -- start 2023-12-01T03:08:18.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.693+0000 7f6407316640 1 -- start start 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.694+0000 7f6407316640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64000b0e60 0x7f64001f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.694+0000 7f6407316640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64000cdaa0 0x7f640014f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.694+0000 7f6407316640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f640015ac80 0x7f6400155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.694+0000 7f6407316640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64001f32a0 con 0x7f640015ac80 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.694+0000 7f6407316640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64001547d0 con 0x7f64000b0e60 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.694+0000 7f6407316640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6400153730 con 0x7f64000cdaa0 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.694+0000 7f64058b3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f640015ac80 0x7f6400155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64058b3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f640015ac80 0x7f6400155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35324/0 (socket says 172.21.15.115:35324) 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64058b3640 1 -- 172.21.15.115:0/427025272 learned_addr learned my addr 172.21.15.115:0/427025272 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64048b1640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64000cdaa0 0x7f640014f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64050b2640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64000b0e60 0x7f64001f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64058b3640 1 -- 172.21.15.115:0/427025272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64000cdaa0 msgr2=0x7f640014f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:18.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64058b3640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64000cdaa0 0x7f640014f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64058b3640 1 -- 172.21.15.115:0/427025272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64000b0e60 msgr2=0x7f64001f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:18.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64058b3640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64000b0e60 0x7f64001f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:18.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64058b3640 1 -- 172.21.15.115:0/427025272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00094d0 con 0x7f640015ac80 2023-12-01T03:08:18.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64050b2640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64000b0e60 0x7f64001f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:18.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.695+0000 7f64058b3640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f640015ac80 0x7f6400155140 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f63f800c780 tx=0x7f63f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:18.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.696+0000 7f63f5ffb640 1 -- 172.21.15.115:0/427025272 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8004b60 con 0x7f640015ac80 2023-12-01T03:08:18.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.696+0000 7f63f5ffb640 1 -- 172.21.15.115:0/427025272 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f800ce90 con 0x7f640015ac80 2023-12-01T03:08:18.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.696+0000 7f6407316640 1 -- 172.21.15.115:0/427025272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6400153a10 con 0x7f640015ac80 2023-12-01T03:08:18.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.696+0000 7f63f5ffb640 1 -- 172.21.15.115:0/427025272 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f800a850 con 0x7f640015ac80 2023-12-01T03:08:18.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.696+0000 7f6407316640 1 -- 172.21.15.115:0/427025272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6400153fd0 con 0x7f640015ac80 2023-12-01T03:08:18.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.697+0000 7f6407316640 1 -- 172.21.15.115:0/427025272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640004e890 con 0x7f640015ac80 2023-12-01T03:08:18.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.698+0000 7f63f5ffb640 1 -- 172.21.15.115:0/427025272 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 109) v1 ==== 88738+0+0 (secure 0 0 0) 0x7f63f800a9b0 con 0x7f640015ac80 2023-12-01T03:08:18.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.699+0000 7f63f5ffb640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f63e406a5a0 0x7f63e406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:18.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.700+0000 7f63f5ffb640 1 -- 172.21.15.115:0/427025272 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6413+0+0 (secure 0 0 0) 0x7f63f8088f60 con 0x7f640015ac80 2023-12-01T03:08:18.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.700+0000 7f64050b2640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f63e406a5a0 0x7f63e406ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:18.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.700+0000 7f64050b2640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f63e406a5a0 0x7f63e406ca60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63fc006fd0 tx=0x7f63fc008040 comp rx=0 tx=0).ready entity=mgr.9819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:18.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.704+0000 7f63f5ffb640 1 -- 172.21.15.115:0/427025272 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f63f8054210 con 0x7f640015ac80 2023-12-01T03:08:18.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:18.916+0000 7f6407316640 1 -- 172.21.15.115:0/427025272 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f64000cc350 con 0x7f63e406a5a0 2023-12-01T03:08:19.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.083+0000 7f63f5ffb640 1 -- 172.21.15.115:0/427025272 <== mgr.9819 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f63fc00c040 con 0x7f63e406a5a0 2023-12-01T03:08:19.086 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-12-01T03:08:19.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 -- 172.21.15.115:0/427025272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f63e406a5a0 msgr2=0x7f63e406ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:19.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f63e406a5a0 0x7f63e406ca60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f63fc006fd0 tx=0x7f63fc008040 comp rx=0 tx=0).stop 2023-12-01T03:08:19.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 -- 172.21.15.115:0/427025272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f640015ac80 msgr2=0x7f6400155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:19.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f640015ac80 0x7f6400155140 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f63f800c780 tx=0x7f63f800cc40 comp rx=0 tx=0).stop 2023-12-01T03:08:19.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 -- 172.21.15.115:0/427025272 shutdown_connections 2023-12-01T03:08:19.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f63e406a5a0 0x7f63e406ca60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:19.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64000cdaa0 0x7f640014f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:19.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f640015ac80 0x7f6400155140 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:19.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 --2- 172.21.15.115:0/427025272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64000b0e60 0x7f64001f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:19.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 -- 172.21.15.115:0/427025272 >> 172.21.15.115:0/427025272 conn(0x7f6400065960 msgr2=0x7f640006f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:19.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 -- 172.21.15.115:0/427025272 shutdown_connections 2023-12-01T03:08:19.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:19.087+0000 7f6407316640 1 -- 172.21.15.115:0/427025272 wait complete. 2023-12-01T03:08:19.108 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-12-01T03:08:19.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.109 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-01T03:08:19.166 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_53_00_UTC 2023-12-01T03:08:19.166 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_54_00_UTC 2023-12-01T03:08:19.166 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_55_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_56_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_57_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_58_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-02_59_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_00_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_01_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_02_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_03_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_04_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_05_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_06_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_07_00_UTC 2023-12-01T03:08:19.167 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_08_00_UTC 2023-12-01T03:08:19.168 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_53_00_UTC 2023-12-01T03:08:19.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.169 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_53_00_UTC) 2023-12-01T03:08:19.235 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_54_00_UTC 2023-12-01T03:08:19.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.236 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_54_00_UTC) 2023-12-01T03:08:19.306 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_55_00_UTC 2023-12-01T03:08:19.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.306 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_55_00_UTC) 2023-12-01T03:08:19.375 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_56_00_UTC 2023-12-01T03:08:19.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.375 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_56_00_UTC) 2023-12-01T03:08:19.442 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_57_00_UTC 2023-12-01T03:08:19.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.442 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_57_00_UTC) 2023-12-01T03:08:19.508 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_58_00_UTC 2023-12-01T03:08:19.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.508 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_58_00_UTC) 2023-12-01T03:08:19.578 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_59_00_UTC 2023-12-01T03:08:19.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.579 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-02_59_00_UTC) 2023-12-01T03:08:19.644 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_00_00_UTC 2023-12-01T03:08:19.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.645 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_00_00_UTC) 2023-12-01T03:08:19.712 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_01_00_UTC 2023-12-01T03:08:19.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.712 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_01_00_UTC) 2023-12-01T03:08:19.781 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_02_00_UTC 2023-12-01T03:08:19.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.782 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_02_00_UTC) 2023-12-01T03:08:19.852 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_03_00_UTC 2023-12-01T03:08:19.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.852 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_03_00_UTC) 2023-12-01T03:08:19.920 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_04_00_UTC 2023-12-01T03:08:19.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.920 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_04_00_UTC) 2023-12-01T03:08:19.986 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_05_00_UTC 2023-12-01T03:08:19.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:19.986 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_05_00_UTC) 2023-12-01T03:08:20.056 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_06_00_UTC 2023-12-01T03:08:20.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:20.057 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_06_00_UTC) 2023-12-01T03:08:20.125 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_07_00_UTC 2023-12-01T03:08:20.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:20.125 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_07_00_UTC) 2023-12-01T03:08:20.198 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_08_00_UTC 2023-12-01T03:08:20.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:20.198 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-01-03_08_00_UTC) 2023-12-01T03:08:20.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:20.271 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-12-01T03:08:20.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T03:08:20.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.553+0000 7f0881f47640 1 Processor -- start 2023-12-01T03:08:20.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.553+0000 7f0881f47640 1 -- start start 2023-12-01T03:08:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.555+0000 7f0881f47640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f087c15de10 0x7f087c0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.555+0000 7f0881f47640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c15e1f0 0x7f087c0adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.555+0000 7f0881f47640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c0ca670 con 0x7f087c0c9e50 2023-12-01T03:08:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.555+0000 7f0881f47640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c0ca390 con 0x7f087c15de10 2023-12-01T03:08:20.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.555+0000 7f0881f47640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c160c20 con 0x7f087c15e1f0 2023-12-01T03:08:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.555+0000 7f087bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f087c0c9e50 0x7f087c0acd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57068/0 (socket says 172.21.15.115:57068) 2023-12-01T03:08:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.555+0000 7f087bfff640 1 -- 172.21.15.115:0/3392977599 learned_addr learned my addr 172.21.15.115:0/3392977599 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f0873fff640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c15e1f0 0x7f087c0adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087b7fe640 1 --2- 172.21.15.115:0/3392977599 >> v2:172.21.15.142:3300/0 conn(0x7f087c15de10 0x7f087c0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278100144 0 0) 0x7f087c0ca670 con 0x7f087c0c9e50 2023-12-01T03:08:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0868003520 con 0x7f087c0c9e50 2023-12-01T03:08:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 975796356 0 0) 0x7f0868003520 con 0x7f087c0c9e50 2023-12-01T03:08:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c15e1f0 msgr2=0x7f087c0adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:20.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c15e1f0 0x7f087c0adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 >> v2:172.21.15.142:3300/0 conn(0x7f087c15de10 msgr2=0x7f087c0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 --2- 172.21.15.115:0/3392977599 >> v2:172.21.15.142:3300/0 conn(0x7f087c15de10 0x7f087c0c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c160f00 con 0x7f087c0c9e50 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f08740019f0 con 0x7f087c0c9e50 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 >> v1:172.21.15.115:6789/0 conn(0x7f087c0c9e50 legacy=0x7f087c0acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0868003970 0x7f0868003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.556+0000 7f087affd640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c15e1f0 0x7f08680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087affd640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 0x7f0868005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0ca390 con 0x7f0868004d00 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c160c20 con 0x7f0868003970 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086800b980 con 0x7f087c15e1f0 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087bfff640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 0x7f0868005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:20.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f0873fff640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c15e1f0 0x7f08680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:20.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087b7fe640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0868003970 0x7f0868003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:20.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087bfff640 1 -- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c15e1f0 msgr2=0x7f08680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:20.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087bfff640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c15e1f0 0x7f08680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:20.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087bfff640 1 -- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0868003970 msgr2=0x7f0868003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:20.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087bfff640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0868003970 0x7f0868003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:20.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087bfff640 1 -- 172.21.15.115:0/3392977599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868003520 con 0x7f0868004d00 2023-12-01T03:08:20.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.557+0000 7f087b7fe640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0868003970 0x7f0868003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:20.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.558+0000 7f087bfff640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 0x7f0868005180 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f087400a780 tx=0x7f08680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730ba93038f477c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:20.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.559+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874034630 con 0x7f0868004d00 2023-12-01T03:08:20.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.559+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087402f070 con 0x7f0868004d00 2023-12-01T03:08:20.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.559+0000 7f087affd640 1 -- 172.21.15.115:0/3392977599 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08740349d0 con 0x7f0868004d00 2023-12-01T03:08:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.559+0000 7f0881f47640 1 -- 172.21.15.115:0/3392977599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 msgr2=0x7f0868005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.559+0000 7f0881f47640 1 --2- 172.21.15.115:0/3392977599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 0x7f0868005180 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f087400a780 tx=0x7f08680057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.559+0000 7f087bfff640 1 -- 172.21.15.115:0/3392977599 reap_dead start 2023-12-01T03:08:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.559+0000 7f0881f47640 1 -- 172.21.15.115:0/3392977599 shutdown_connections 2023-12-01T03:08:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.559+0000 7f0881f47640 1 -- 172.21.15.115:0/3392977599 >> 172.21.15.115:0/3392977599 conn(0x7f087c0cf0b0 msgr2=0x7f087c0cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.559+0000 7f0881f47640 1 -- 172.21.15.115:0/3392977599 shutdown_connections 2023-12-01T03:08:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.560+0000 7f0881f47640 1 -- 172.21.15.115:0/3392977599 wait complete. 2023-12-01T03:08:20.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.561+0000 7f0881f47640 1 Processor -- start 2023-12-01T03:08:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.561+0000 7f0881f47640 1 -- start start 2023-12-01T03:08:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.562+0000 7f0881f47640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 0x7f087c0c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.562+0000 7f0881f47640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f087c0c3080 0x7f087c0c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.562+0000 7f0881f47640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c150e50 0x7f087c1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.562+0000 7f0881f47640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c14e6d0 con 0x7f0868004d00 2023-12-01T03:08:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.562+0000 7f0881f47640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0c7710 con 0x7f087c0c3080 2023-12-01T03:08:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.562+0000 7f0881f47640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0c6b00 con 0x7f087c150e50 2023-12-01T03:08:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.563+0000 7f087b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 0x7f087c0c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.563+0000 7f087b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 0x7f087c0c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35340/0 (socket says 172.21.15.115:35340) 2023-12-01T03:08:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.563+0000 7f087b7fe640 1 -- 172.21.15.115:0/2870910831 learned_addr learned my addr 172.21.15.115:0/2870910831 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.563+0000 7f087b7fe640 1 -- 172.21.15.115:0/2870910831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c150e50 msgr2=0x7f087c1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.563+0000 7f087b7fe640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c150e50 0x7f087c1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.563+0000 7f087b7fe640 1 -- 172.21.15.115:0/2870910831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f087c0c3080 msgr2=0x7f087c0c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.563+0000 7f087b7fe640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f087c0c3080 0x7f087c0c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.563+0000 7f087b7fe640 1 -- 172.21.15.115:0/2870910831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c0c6de0 con 0x7f0868004d00 2023-12-01T03:08:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.563+0000 7f087b7fe640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 0x7f087c0c7ba0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f086c010950 tx=0x7f086c010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.564+0000 7f0878ff9640 1 -- 172.21.15.115:0/2870910831 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00eb60 con 0x7f0868004d00 2023-12-01T03:08:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.564+0000 7f0878ff9640 1 -- 172.21.15.115:0/2870910831 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c00ecc0 con 0x7f0868004d00 2023-12-01T03:08:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.564+0000 7f0878ff9640 1 -- 172.21.15.115:0/2870910831 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c01c400 con 0x7f0868004d00 2023-12-01T03:08:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.564+0000 7f0881f47640 1 -- 172.21.15.115:0/2870910831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c0c70c0 con 0x7f0868004d00 2023-12-01T03:08:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.564+0000 7f0881f47640 1 -- 172.21.15.115:0/2870910831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c0c80e0 con 0x7f0868004d00 2023-12-01T03:08:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.565+0000 7f0881f47640 1 -- 172.21.15.115:0/2870910831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c04e890 con 0x7f0868004d00 2023-12-01T03:08:20.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.566+0000 7f0878ff9640 1 -- 172.21.15.115:0/2870910831 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 109) v1 ==== 88738+0+0 (secure 0 0 0) 0x7f086c01c560 con 0x7f0868004d00 2023-12-01T03:08:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.567+0000 7f0878ff9640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f085c06a5a0 0x7f085c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.567+0000 7f0878ff9640 1 -- 172.21.15.115:0/2870910831 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6413+0+0 (secure 0 0 0) 0x7f086c0893a0 con 0x7f0868004d00 2023-12-01T03:08:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.567+0000 7f0873fff640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f085c06a5a0 0x7f085c06ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:20.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.568+0000 7f0873fff640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f085c06a5a0 0x7f085c06ca60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f087c0c9180 tx=0x7f0864008040 comp rx=0 tx=0).ready entity=mgr.9819 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:20.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.572+0000 7f0878ff9640 1 -- 172.21.15.115:0/2870910831 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f086c054670 con 0x7f0868004d00 2023-12-01T03:08:20.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:20.811+0000 7f0881f47640 1 -- 172.21.15.115:0/2870910831 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f087c073170 con 0x7f0868004d00 2023-12-01T03:08:21.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.407+0000 7f0878ff9640 1 -- 172.21.15.115:0/2870910831 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 110) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f086c087600 con 0x7f0868004d00 2023-12-01T03:08:21.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.410+0000 7f0878ff9640 1 -- 172.21.15.115:0/2870910831 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v110) v1 ==== 93+0+0 (secure 0 0 0) 0x7f086c0858b0 con 0x7f0868004d00 2023-12-01T03:08:21.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.414+0000 7f0881f47640 1 -- 172.21.15.115:0/2870910831 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f085c06a5a0 msgr2=0x7f085c06ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:21.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.414+0000 7f0881f47640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f085c06a5a0 0x7f085c06ca60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f087c0c9180 tx=0x7f0864008040 comp rx=0 tx=0).stop 2023-12-01T03:08:21.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.414+0000 7f0881f47640 1 -- 172.21.15.115:0/2870910831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 msgr2=0x7f087c0c7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:21.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.414+0000 7f0881f47640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 0x7f087c0c7ba0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f086c010950 tx=0x7f086c010e10 comp rx=0 tx=0).stop 2023-12-01T03:08:21.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.415+0000 7f0881f47640 1 -- 172.21.15.115:0/2870910831 shutdown_connections 2023-12-01T03:08:21.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.415+0000 7f0881f47640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f085c06a5a0 0x7f085c06ca60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:21.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.415+0000 7f0881f47640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087c150e50 0x7f087c1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:21.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.415+0000 7f0881f47640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f087c0c3080 0x7f087c0c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:21.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.415+0000 7f0881f47640 1 --2- 172.21.15.115:0/2870910831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0868004d00 0x7f087c0c7ba0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.415+0000 7f0881f47640 1 -- 172.21.15.115:0/2870910831 >> 172.21.15.115:0/2870910831 conn(0x7f087c0cf0b0 msgr2=0x7f087c15f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.415+0000 7f0881f47640 1 -- 172.21.15.115:0/2870910831 shutdown_connections 2023-12-01T03:08:21.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:21.415+0000 7f0881f47640 1 -- 172.21.15.115:0/2870910831 wait complete. 2023-12-01T03:08:21.440 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T03:08:21.441 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:08:21.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:08:21.647 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T03:08:21.647 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:08:21.647 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:08:21.698 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:08:21.698 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:08:21.790 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:08:21.949 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:21.947+0000 7fdca5eda040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:08:21.961 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:21.960+0000 7f49b6054040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:08:22.003 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:22.001+0000 7fdca5eda040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:08:22.013 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:22.012+0000 7f49b6054040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:08:22.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.036+0000 7fdeb082d640 1 Processor -- start 2023-12-01T03:08:22.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.036+0000 7fdeb082d640 1 -- start start 2023-12-01T03:08:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.038+0000 7fdeb082d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdeac160fe0 0x7fdeac0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.038+0000 7fdeb082d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac0ae270 con 0x7fdeac0adde0 2023-12-01T03:08:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.038+0000 7fdeb082d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac0ae550 con 0x7fdeac0c3f00 2023-12-01T03:08:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.038+0000 7fdeb082d640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac0ae830 con 0x7fdeac160fe0 2023-12-01T03:08:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.038+0000 7fdeab7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdeac160fe0 0x7fdeac0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.038+0000 7fdeab7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdeac160fe0 0x7fdeac0ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35360/0 (socket says 172.21.15.115:35360) 2023-12-01T03:08:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.038+0000 7fdeab7fe640 1 -- 172.21.15.115:0/3474785058 learned_addr learned my addr 172.21.15.115:0/3474785058 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:22.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.038+0000 7fdeaaffd640 1 --1- 172.21.15.115:0/3474785058 >> v1:172.21.15.115:6789/0 conn(0x7fdeac0adde0 0x7fdeac0c37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57078/0 (socket says 172.21.15.115:57078) 2023-12-01T03:08:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.039+0000 7fdeab7fe640 1 -- 172.21.15.115:0/3474785058 >> v1:172.21.15.142:6789/0 conn(0x7fdeac0c3f00 legacy=0x7fdeac0b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.039+0000 7fdeab7fe640 1 -- 172.21.15.115:0/3474785058 >> v1:172.21.15.115:6789/0 conn(0x7fdeac0adde0 legacy=0x7fdeac0c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.039+0000 7fdeaa7fc640 1 -- 172.21.15.115:0/3474785058 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354475294 0 0) 0x7fdeac0ae270 con 0x7fdeac0adde0 2023-12-01T03:08:22.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.039+0000 7fdeab7fe640 1 -- 172.21.15.115:0/3474785058 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0aeb10 con 0x7fdeac160fe0 2023-12-01T03:08:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.040+0000 7fdeab7fe640 1 --2- 172.21.15.115:0/3474785058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac160fe0 0x7fdeac0ce700 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009850 tx=0x7fde9c02ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6878c5e4002f585 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.041+0000 7fdeaa7fc640 1 -- 172.21.15.115:0/3474785058 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c004230 con 0x7fdeac160fe0 2023-12-01T03:08:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.041+0000 7fdeaa7fc640 1 -- 172.21.15.115:0/3474785058 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c004390 con 0x7fdeac160fe0 2023-12-01T03:08:22.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.041+0000 7fdeaa7fc640 1 -- 172.21.15.115:0/3474785058 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c02f820 con 0x7fdeac160fe0 2023-12-01T03:08:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.041+0000 7fdeb082d640 1 -- 172.21.15.115:0/3474785058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac160fe0 msgr2=0x7fdeac0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.041+0000 7fdeb082d640 1 --2- 172.21.15.115:0/3474785058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac160fe0 0x7fdeac0ce700 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009850 tx=0x7fde9c02ba00 comp rx=0 tx=0).stop 2023-12-01T03:08:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.041+0000 7fdeb082d640 1 -- 172.21.15.115:0/3474785058 shutdown_connections 2023-12-01T03:08:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.041+0000 7fdeb082d640 1 --2- 172.21.15.115:0/3474785058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac160fe0 0x7fdeac0ce700 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.041+0000 7fdeb082d640 1 -- 172.21.15.115:0/3474785058 >> 172.21.15.115:0/3474785058 conn(0x7fdeac0d0820 msgr2=0x7fdeac0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.042+0000 7fdeb082d640 1 -- 172.21.15.115:0/3474785058 shutdown_connections 2023-12-01T03:08:22.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.042+0000 7fdeb082d640 1 -- 172.21.15.115:0/3474785058 wait complete. 2023-12-01T03:08:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.043+0000 7fdeb082d640 1 Processor -- start 2023-12-01T03:08:22.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.044+0000 7fdeb082d640 1 -- start start 2023-12-01T03:08:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeb082d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeac0c3f00 0x7fdeac0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeb082d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac160fe0 0x7fdeac162da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeb082d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac0b56b0 0x7fdeac15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeb082d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0b33d0 con 0x7fdeac0b56b0 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeb082d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0b60c0 con 0x7fdeac160fe0 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeb082d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0b3620 con 0x7fdeac0c3f00 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeabfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac0b56b0 0x7fdeac15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeabfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac0b56b0 0x7fdeac15d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35372/0 (socket says 172.21.15.115:35372) 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeabfff640 1 -- 172.21.15.115:0/1712331686 learned_addr learned my addr 172.21.15.115:0/1712331686 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeaaffd640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac160fe0 0x7fdeac162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeabfff640 1 -- 172.21.15.115:0/1712331686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeac0c3f00 msgr2=0x7fdeac0b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeabfff640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeac0c3f00 0x7fdeac0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeabfff640 1 -- 172.21.15.115:0/1712331686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac160fe0 msgr2=0x7fdeac162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeabfff640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac160fe0 0x7fdeac162da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeabfff640 1 -- 172.21.15.115:0/1712331686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c009460 con 0x7fdeac0b56b0 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.045+0000 7fdeaaffd640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac160fe0 0x7fdeac162da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:22.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.046+0000 7fdeabfff640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac0b56b0 0x7fdeac15d880 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fdea400c780 tx=0x7fdea400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.046+0000 7fde93fff640 1 -- 172.21.15.115:0/1712331686 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4004b60 con 0x7fdeac0b56b0 2023-12-01T03:08:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.046+0000 7fde93fff640 1 -- 172.21.15.115:0/1712331686 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea400ce90 con 0x7fdeac0b56b0 2023-12-01T03:08:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.046+0000 7fde93fff640 1 -- 172.21.15.115:0/1712331686 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400a850 con 0x7fdeac0b56b0 2023-12-01T03:08:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.046+0000 7fdeb082d640 1 -- 172.21.15.115:0/1712331686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac0b3900 con 0x7fdeac0b56b0 2023-12-01T03:08:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.046+0000 7fdeb082d640 1 -- 172.21.15.115:0/1712331686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac1face0 con 0x7fdeac0b56b0 2023-12-01T03:08:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.047+0000 7fdeb082d640 1 -- 172.21.15.115:0/1712331686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac0673a0 con 0x7fdeac0b56b0 2023-12-01T03:08:22.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.048+0000 7fde93fff640 1 -- 172.21.15.115:0/1712331686 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 110) v1 ==== 88721+0+0 (secure 0 0 0) 0x7fdea400a9b0 con 0x7fdeac0b56b0 2023-12-01T03:08:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.049+0000 7fde93fff640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde9806a550 0x7fde9806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.049+0000 7fde93fff640 1 -- 172.21.15.115:0/1712331686 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6413+0+0 (secure 0 0 0) 0x7fdea4088f80 con 0x7fdeac0b56b0 2023-12-01T03:08:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.049+0000 7fdeab7fe640 1 -- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde9806a550 msgr2=0x7fde9806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.049+0000 7fdeab7fe640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde9806a550 0x7fde9806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.053+0000 7fde93fff640 1 -- 172.21.15.115:0/1712331686 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdea4054190 con 0x7fdeac0b56b0 2023-12-01T03:08:22.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.245+0000 7fdeb082d640 1 -- 172.21.15.115:0/1712331686 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdeac0b67a0 con 0x7fdeac0b56b0 2023-12-01T03:08:22.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.247+0000 7fde93fff640 1 -- 172.21.15.115:0/1712331686 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+17199 (secure 0 0 0) 0x7fdea40577b0 con 0x7fdeac0b56b0 2023-12-01T03:08:22.250 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:22.250 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":181,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:03:21.455303+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:52:20.047003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-01T02:52:20.974031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2f","mappings":[{"from":2,"to":5}]},{"pgid":"21.3b","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/363147954":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/1462551426":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/718367345":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/2970056045":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/858082591":"2023-12-02T03:03:21.455245+0000","172.21.15.115:6824/88317":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/3219939439":"2023-12-02T03:03:21.455245+0000","172.21.15.115:6822/88317":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/673199316":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/2404622555":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/1684885368":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/4185429564":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/682039355":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/2564902155":"2023-12-02T02:52:41.228363+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:08:22.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeab7fe640 1 -- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde9806a550 msgr2=0x7fde9806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:22.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeab7fe640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde9806a550 0x7fde9806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:08:22.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 -- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde9806a550 msgr2=0x7fde9806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:22.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde9806a550 0x7fde9806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 -- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac0b56b0 msgr2=0x7fdeac15d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac0b56b0 0x7fdeac15d880 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fdea400c780 tx=0x7fdea400cc40 comp rx=0 tx=0).stop 2023-12-01T03:08:22.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 -- 172.21.15.115:0/1712331686 shutdown_connections 2023-12-01T03:08:22.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde9806a550 0x7fde9806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdeac0b56b0 0x7fdeac15d880 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeac160fe0 0x7fdeac162da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 --2- 172.21.15.115:0/1712331686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdeac0c3f00 0x7fdeac0b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 -- 172.21.15.115:0/1712331686 >> 172.21.15.115:0/1712331686 conn(0x7fdeac0d0820 msgr2=0x7fdeac15fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:22.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 -- 172.21.15.115:0/1712331686 shutdown_connections 2023-12-01T03:08:22.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.250+0000 7fdeb082d640 1 -- 172.21.15.115:0/1712331686 wait complete. 2023-12-01T03:08:22.272 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T02:52:20.047003+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T02:52:20.974031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '177', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:08:22.272 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:08:22.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.496+0000 7f7ffcfb1640 1 Processor -- start 2023-12-01T03:08:22.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.496+0000 7f7ffcfb1640 1 -- start start 2023-12-01T03:08:22.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.497+0000 7f7ffcfb1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff8143eb0 0x7f7ff806b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.497+0000 7f7ffcfb1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7ff806b7a0 0x7f7ff80cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.498+0000 7f7ffcfb1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7ff80cbab0 0x7f7ff8164c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.498+0000 7f7ffcfb1640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff80cbf20 con 0x7f7ff806b7a0 2023-12-01T03:08:22.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.498+0000 7f7ffcfb1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff80ce1b0 con 0x7f7ff80cbab0 2023-12-01T03:08:22.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.498+0000 7f7ffcfb1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80ce490 con 0x7f7ff8143eb0 2023-12-01T03:08:22.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.498+0000 7f7ff5d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7ff806b7a0 0x7f7ff80cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.498+0000 7f7ff5d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7ff806b7a0 0x7f7ff80cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35386/0 (socket says 172.21.15.115:35386) 2023-12-01T03:08:22.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.498+0000 7f7ff5d9b640 1 -- 172.21.15.115:0/3390534190 learned_addr learned my addr 172.21.15.115:0/3390534190 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:22.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.498+0000 7f7ff659c640 1 --2- 172.21.15.115:0/3390534190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff8143eb0 0x7f7ff806b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.498+0000 7f7ff6d9d640 1 --2- 172.21.15.115:0/3390534190 >> v2:172.21.15.142:3300/0 conn(0x7f7ff80cbab0 0x7f7ff8164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.499+0000 7f7ff5d9b640 1 -- 172.21.15.115:0/3390534190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff8143eb0 msgr2=0x7f7ff806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:22.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.499+0000 7f7ff5d9b640 1 --2- 172.21.15.115:0/3390534190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff8143eb0 0x7f7ff806b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.499+0000 7f7ff5d9b640 1 -- 172.21.15.115:0/3390534190 >> v2:172.21.15.142:3300/0 conn(0x7f7ff80cbab0 msgr2=0x7f7ff8164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:22.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.499+0000 7f7ff5d9b640 1 --2- 172.21.15.115:0/3390534190 >> v2:172.21.15.142:3300/0 conn(0x7f7ff80cbab0 0x7f7ff8164c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.499+0000 7f7ff5d9b640 1 -- 172.21.15.115:0/3390534190 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80caa90 con 0x7f7ff806b7a0 2023-12-01T03:08:22.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.500+0000 7f7ff5d9b640 1 --2- 172.21.15.115:0/3390534190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 0x7f7ff80cb570 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80b0fb0 tx=0x7f7fec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421782252265d816 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:22.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.500+0000 7f7ff559a640 1 -- 172.21.15.115:0/3390534190 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec031870 con 0x7f7ff806b7a0 2023-12-01T03:08:22.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.500+0000 7f7ff559a640 1 -- 172.21.15.115:0/3390534190 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec0319d0 con 0x7f7ff806b7a0 2023-12-01T03:08:22.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.500+0000 7f7ff559a640 1 -- 172.21.15.115:0/3390534190 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec031d60 con 0x7f7ff806b7a0 2023-12-01T03:08:22.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.501+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3390534190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 msgr2=0x7f7ff80cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.501+0000 7f7ffcfb1640 1 --2- 172.21.15.115:0/3390534190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 0x7f7ff80cb570 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80b0fb0 tx=0x7f7fec02b910 comp rx=0 tx=0).stop 2023-12-01T03:08:22.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.501+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3390534190 shutdown_connections 2023-12-01T03:08:22.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.501+0000 7f7ffcfb1640 1 --2- 172.21.15.115:0/3390534190 >> v2:172.21.15.142:3300/0 conn(0x7f7ff80cbab0 0x7f7ff8164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.501+0000 7f7ffcfb1640 1 --2- 172.21.15.115:0/3390534190 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 0x7f7ff80cb570 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.501+0000 7f7ffcfb1640 1 --2- 172.21.15.115:0/3390534190 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff8143eb0 0x7f7ff806b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.501+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3390534190 >> 172.21.15.115:0/3390534190 conn(0x7f7ff8065960 msgr2=0x7f7ff8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:22.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.501+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3390534190 shutdown_connections 2023-12-01T03:08:22.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.501+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3390534190 wait complete. 2023-12-01T03:08:22.506 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:22.503+0000 7fdca5eda040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:08:22.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.503+0000 7f7ffcfb1640 1 Processor -- start 2023-12-01T03:08:22.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.503+0000 7f7ffcfb1640 1 -- start start 2023-12-01T03:08:22.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ffcfb1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 0x7f7ff813cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ffcfb1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff80cbab0 0x7f7ff815a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ffcfb1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff8143eb0 0x7f7ff8163880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ffcfb1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8140b30 con 0x7f7ff806b7a0 2023-12-01T03:08:22.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ffcfb1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff815ac30 con 0x7f7ff80cbab0 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ffcfb1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8140190 con 0x7f7ff8143eb0 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ff659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 0x7f7ff813cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ff659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 0x7f7ff813cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35390/0 (socket says 172.21.15.115:35390) 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ff659c640 1 -- 172.21.15.115:0/3300715364 learned_addr learned my addr 172.21.15.115:0/3300715364 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ff5d9b640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff80cbab0 0x7f7ff815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ff659c640 1 -- 172.21.15.115:0/3300715364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff8143eb0 msgr2=0x7f7ff8163880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ff659c640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff8143eb0 0x7f7ff8163880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ff659c640 1 -- 172.21.15.115:0/3300715364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff80cbab0 msgr2=0x7f7ff815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ff659c640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff80cbab0 0x7f7ff815a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.505+0000 7f7ff659c640 1 -- 172.21.15.115:0/3300715364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec0094d0 con 0x7f7ff806b7a0 2023-12-01T03:08:22.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.506+0000 7f7ff659c640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 0x7f7ff813cb40 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800e870 tx=0x7f7fe800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:22.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.506+0000 7f7fe6ffd640 1 -- 172.21.15.115:0/3300715364 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80094f0 con 0x7f7ff806b7a0 2023-12-01T03:08:22.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.506+0000 7f7fe6ffd640 1 -- 172.21.15.115:0/3300715364 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe800c940 con 0x7f7ff806b7a0 2023-12-01T03:08:22.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.506+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3300715364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff8140470 con 0x7f7ff806b7a0 2023-12-01T03:08:22.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.506+0000 7f7fe6ffd640 1 -- 172.21.15.115:0/3300715364 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800ccf0 con 0x7f7ff806b7a0 2023-12-01T03:08:22.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.507+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3300715364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff813d5f0 con 0x7f7ff806b7a0 2023-12-01T03:08:22.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.508+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3300715364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff804e890 con 0x7f7ff806b7a0 2023-12-01T03:08:22.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.509+0000 7f7fe6ffd640 1 -- 172.21.15.115:0/3300715364 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 110) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f7fe8013470 con 0x7f7ff806b7a0 2023-12-01T03:08:22.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.511+0000 7f7fe6ffd640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7fd806a550 0x7f7fd806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.511+0000 7f7fe6ffd640 1 -- 172.21.15.115:0/3300715364 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6501+0+0 (secure 0 0 0) 0x7f7fe8088ed0 con 0x7f7ff806b7a0 2023-12-01T03:08:22.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.511+0000 7f7ff5d9b640 1 -- 172.21.15.115:0/3300715364 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7fd806a550 msgr2=0x7f7fd806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:22.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.511+0000 7f7ff5d9b640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7fd806a550 0x7f7fd806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:22.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.516+0000 7f7fe6ffd640 1 -- 172.21.15.115:0/3300715364 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7fe8053ff0 con 0x7f7ff806b7a0 2023-12-01T03:08:22.522 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:22.521+0000 7f49b6054040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:08:22.562 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:22.561+0000 7fdca5eda040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:08:22.574 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:22.573+0000 7f49b6054040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:08:22.617 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:22.615+0000 7fdca5eda040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:08:22.623 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:22.623+0000 7f49b6054040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:08:22.683 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:22.681+0000 7fdca5eda040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:08:22.684 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:22.683+0000 7f49b6054040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:08:22.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.706+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3300715364 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ff80b12a0 con 0x7f7ff806b7a0 2023-12-01T03:08:22.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.707+0000 7f7fe6ffd640 1 -- 172.21.15.115:0/3300715364 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fe80210a0 con 0x7f7ff806b7a0 2023-12-01T03:08:22.709 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:08:22.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3300715364 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7fd806a550 msgr2=0x7f7fd806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:22.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7fd806a550 0x7f7fd806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3300715364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 msgr2=0x7f7ff813cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 0x7f7ff813cb40 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800e870 tx=0x7f7fe800ed30 comp rx=0 tx=0).stop 2023-12-01T03:08:22.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3300715364 shutdown_connections 2023-12-01T03:08:22.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7fd806a550 0x7f7fd806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ff8143eb0 0x7f7ff8163880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ff80cbab0 0x7f7ff815a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 --2- 172.21.15.115:0/3300715364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff806b7a0 0x7f7ff813cb40 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3300715364 >> 172.21.15.115:0/3300715364 conn(0x7f7ff8065960 msgr2=0x7f7ff806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:22.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3300715364 shutdown_connections 2023-12-01T03:08:22.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.710+0000 7f7ffcfb1640 1 -- 172.21.15.115:0/3300715364 wait complete. 2023-12-01T03:08:22.737 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:22.736+0000 7f49b6054040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:08:22.739 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:08:22.755 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:22.753+0000 7fdca5eda040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:08:22.890 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:22.889+0000 7f49b6054040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:08:22.937 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:22.935+0000 7fdca5eda040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:08:22.952 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:22.951+0000 7f49b6054040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:08:22.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.982+0000 7f82d8dd9640 1 Processor -- start 2023-12-01T03:08:22.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.982+0000 7f82d8dd9640 1 -- start start 2023-12-01T03:08:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.983+0000 7f82d8dd9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82d415cd90 0x7f82d40cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.983+0000 7f82d8dd9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d40ce5e0 con 0x7f82d40cfd60 2023-12-01T03:08:22.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.983+0000 7f82d8dd9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d40ce300 con 0x7f82d40b5fa0 2023-12-01T03:08:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.983+0000 7f82d8dd9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d40d0ec0 con 0x7f82d415cd90 2023-12-01T03:08:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.983+0000 7f82d2d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f82d40cfd60 0x7f82d40d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57090/0 (socket says 172.21.15.115:57090) 2023-12-01T03:08:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.983+0000 7f82d2d9d640 1 -- 172.21.15.115:0/4281283317 learned_addr learned my addr 172.21.15.115:0/4281283317 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.983+0000 7f82d259c640 1 --2- 172.21.15.115:0/4281283317 >> v2:172.21.15.142:3300/0 conn(0x7f82d415cd90 0x7f82d40cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069824403 0 0) 0x7f82cc001490 con 0x7f82d40cfd60 2023-12-01T03:08:22.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c0003520 con 0x7f82d40cfd60 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1961164153 0 0) 0x7f82c0003520 con 0x7f82d40cfd60 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 >> v2:172.21.15.142:3300/0 conn(0x7f82d415cd90 msgr2=0x7f82d40cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 --2- 172.21.15.115:0/4281283317 >> v2:172.21.15.142:3300/0 conn(0x7f82d415cd90 0x7f82d40cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 >> v1:172.21.15.142:6789/0 conn(0x7f82d40b5fa0 legacy=0x7f82d40cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d40d10a0 con 0x7f82d40cfd60 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f82cc001b80 con 0x7f82d40cfd60 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 >> v1:172.21.15.115:6789/0 conn(0x7f82d40cfd60 legacy=0x7f82d40d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82c00039c0 0x7f82c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82c00051d0 0x7f82c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40d0ec0 con 0x7f82c00047c0 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c000c120 con 0x7f82c00051d0 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c000c2e0 con 0x7f82c00039c0 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.984+0000 7f82cbfff640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.985+0000 7f82d259c640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82c00039c0 0x7f82c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.985+0000 7f82d2d9d640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82c00051d0 0x7f82c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.985+0000 7f82cbfff640 1 -- 172.21.15.115:0/4281283317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82c00039c0 msgr2=0x7f82c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.985+0000 7f82cbfff640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82c00039c0 0x7f82c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.985+0000 7f82cbfff640 1 -- 172.21.15.115:0/4281283317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82c00051d0 msgr2=0x7f82c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.985+0000 7f82cbfff640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82c00051d0 0x7f82c000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.985+0000 7f82cbfff640 1 -- 172.21.15.115:0/4281283317 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0003520 con 0x7f82c00047c0 2023-12-01T03:08:22.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.985+0000 7f82d2d9d640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82c00051d0 0x7f82c000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:22.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.985+0000 7f82d2d9d640 1 -- 172.21.15.115:0/4281283317 reap_dead start 2023-12-01T03:08:22.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.987+0000 7f82cbfff640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f82bc001720 tx=0x7f82c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382da65f19557162 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:22.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.987+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0052d0 con 0x7f82c00047c0 2023-12-01T03:08:22.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.987+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc02e050 con 0x7f82c00047c0 2023-12-01T03:08:22.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.987+0000 7f82d1d9b640 1 -- 172.21.15.115:0/4281283317 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc003830 con 0x7f82c00047c0 2023-12-01T03:08:22.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.988+0000 7f82d8dd9640 1 -- 172.21.15.115:0/4281283317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 msgr2=0x7f82c0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.988+0000 7f82d8dd9640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f82bc001720 tx=0x7f82c00057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.988+0000 7f82d8dd9640 1 -- 172.21.15.115:0/4281283317 shutdown_connections 2023-12-01T03:08:22.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.988+0000 7f82d8dd9640 1 --2- 172.21.15.115:0/4281283317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.988+0000 7f82d8dd9640 1 -- 172.21.15.115:0/4281283317 >> 172.21.15.115:0/4281283317 conn(0x7f82d4065960 msgr2=0x7f82d4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:22.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.988+0000 7f82d8dd9640 1 -- 172.21.15.115:0/4281283317 shutdown_connections 2023-12-01T03:08:22.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.988+0000 7f82d8dd9640 1 -- 172.21.15.115:0/4281283317 wait complete. 2023-12-01T03:08:22.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.990+0000 7f82d8dd9640 1 Processor -- start 2023-12-01T03:08:22.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.990+0000 7f82d8dd9640 1 -- start start 2023-12-01T03:08:22.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d8dd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82d4152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d8dd9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82d40caa10 0x7f82d4159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d8dd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82d4159b40 0x7f82d41f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d8dd9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d41f3720 con 0x7f82c00047c0 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d8dd9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d4159fc0 con 0x7f82d40caa10 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d8dd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40ca580 con 0x7f82d4159b40 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82d4152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82d4152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35404/0 (socket says 172.21.15.115:35404) 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d259c640 1 -- 172.21.15.115:0/129537118 learned_addr learned my addr 172.21.15.115:0/129537118 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d259c640 1 -- 172.21.15.115:0/129537118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82d4159b40 msgr2=0x7f82d41f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d259c640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82d4159b40 0x7f82d41f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d259c640 1 -- 172.21.15.115:0/129537118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82d40caa10 msgr2=0x7f82d4159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d259c640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82d40caa10 0x7f82d4159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.991+0000 7f82d259c640 1 -- 172.21.15.115:0/129537118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d4152790 con 0x7f82c00047c0 2023-12-01T03:08:22.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.992+0000 7f82d259c640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82d4152250 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f82c400f630 tx=0x7f82c400faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:22.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.992+0000 7f82caffd640 1 -- 172.21.15.115:0/129537118 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4014870 con 0x7f82c00047c0 2023-12-01T03:08:22.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.992+0000 7f82caffd640 1 -- 172.21.15.115:0/129537118 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c40149d0 con 0x7f82c00047c0 2023-12-01T03:08:22.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.992+0000 7f82caffd640 1 -- 172.21.15.115:0/129537118 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4014d40 con 0x7f82c00047c0 2023-12-01T03:08:22.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.992+0000 7f82d8dd9640 1 -- 172.21.15.115:0/129537118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d41f3440 con 0x7f82c00047c0 2023-12-01T03:08:22.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.992+0000 7f82d8dd9640 1 -- 172.21.15.115:0/129537118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d40ca210 con 0x7f82c00047c0 2023-12-01T03:08:22.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.993+0000 7f82d8dd9640 1 -- 172.21.15.115:0/129537118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d40c9d70 con 0x7f82c00047c0 2023-12-01T03:08:22.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.994+0000 7f82caffd640 1 -- 172.21.15.115:0/129537118 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 110) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f82c4015410 con 0x7f82c00047c0 2023-12-01T03:08:22.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.995+0000 7f82caffd640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f82b406a580 0x7f82b406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:22.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.995+0000 7f82caffd640 1 -- 172.21.15.115:0/129537118 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6501+0+0 (secure 0 0 0) 0x7f82c4089730 con 0x7f82c00047c0 2023-12-01T03:08:22.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.995+0000 7f82cbfff640 1 -- 172.21.15.115:0/129537118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f82b406a580 msgr2=0x7f82b406ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:22.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.995+0000 7f82cbfff640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f82b406a580 0x7f82b406ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:23.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:22.999+0000 7f82caffd640 1 -- 172.21.15.115:0/129537118 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f82c4054980 con 0x7f82c00047c0 2023-12-01T03:08:23.002 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:23.001+0000 7f49b6054040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:08:23.008 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:23.007+0000 7fdca5eda040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:08:23.058 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:23.057+0000 7f49b6054040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:08:23.063 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:23.061+0000 7fdca5eda040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:08:23.108 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:23.107+0000 7f49b6054040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:08:23.126 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:23.124+0000 7fdca5eda040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:08:23.178 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:23.177+0000 7fdca5eda040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:08:23.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.190+0000 7f82d8dd9640 1 -- 172.21.15.115:0/129537118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82d404e890 con 0x7f82c00047c0 2023-12-01T03:08:23.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.191+0000 7f82caffd640 1 -- 172.21.15.115:0/129537118 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v182) v1 ==== 104+0+11 (secure 0 0 0) 0x7f82c4057fa0 con 0x7f82c00047c0 2023-12-01T03:08:23.193 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 -- 172.21.15.115:0/129537118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f82b406a580 msgr2=0x7f82b406ca40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f82b406a580 0x7f82b406ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 -- 172.21.15.115:0/129537118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 msgr2=0x7f82d4152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82d4152250 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f82c400f630 tx=0x7f82c400faf0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 -- 172.21.15.115:0/129537118 shutdown_connections 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f82b406a580 0x7f82b406ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82d4159b40 0x7f82d41f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82d40caa10 0x7f82d4159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 --2- 172.21.15.115:0/129537118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82c00047c0 0x7f82d4152250 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 -- 172.21.15.115:0/129537118 >> 172.21.15.115:0/129537118 conn(0x7f82d4065960 msgr2=0x7f82d40b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:23.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 -- 172.21.15.115:0/129537118 shutdown_connections 2023-12-01T03:08:23.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.194+0000 7f82d8dd9640 1 -- 172.21.15.115:0/129537118 wait complete. 2023-12-01T03:08:23.216 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:08:23.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.463+0000 7f01b0173640 1 Processor -- start 2023-12-01T03:08:23.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.464+0000 7f01b0173640 1 -- start start 2023-12-01T03:08:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.465+0000 7f01b0173640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a815cd90 0x7f01a80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.466+0000 7f01b0173640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a80ce5e0 con 0x7f01a80b5fa0 2023-12-01T03:08:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.466+0000 7f01b0173640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a80ce300 con 0x7f01a80cfd60 2023-12-01T03:08:23.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.466+0000 7f01b0173640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80d0ec0 con 0x7f01a815cd90 2023-12-01T03:08:23.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.466+0000 7f01ad70e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f01a80b5fa0 0x7f01a80cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57096/0 (socket says 172.21.15.115:57096) 2023-12-01T03:08:23.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.466+0000 7f01ad70e640 1 -- 172.21.15.115:0/1113563728 learned_addr learned my addr 172.21.15.115:0/1113563728 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:23.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.466+0000 7f01adf0f640 1 --2- 172.21.15.115:0/1113563728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a815cd90 0x7f01a80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.467+0000 7f01acf0d640 1 -- 172.21.15.115:0/1113563728 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129819160 0 0) 0x7f0198001110 con 0x7f01a80b5fa0 2023-12-01T03:08:23.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.467+0000 7f01acf0d640 1 -- 172.21.15.115:0/1113563728 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0194003520 con 0x7f01a80b5fa0 2023-12-01T03:08:23.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.467+0000 7f01acf0d640 1 -- 172.21.15.115:0/1113563728 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270181479 0 0) 0x7f01a80ce300 con 0x7f01a80cfd60 2023-12-01T03:08:23.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.467+0000 7f01acf0d640 1 -- 172.21.15.115:0/1113563728 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0198001110 con 0x7f01a80cfd60 2023-12-01T03:08:23.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.467+0000 7f01adf0f640 1 -- 172.21.15.115:0/1113563728 >> v1:172.21.15.142:6789/0 conn(0x7f01a80cfd60 legacy=0x7f01a80d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.467+0000 7f01adf0f640 1 -- 172.21.15.115:0/1113563728 >> v1:172.21.15.115:6789/0 conn(0x7f01a80b5fa0 legacy=0x7f01a80cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.467+0000 7f01adf0f640 1 -- 172.21.15.115:0/1113563728 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80d10a0 con 0x7f01a815cd90 2023-12-01T03:08:23.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.469+0000 7f01adf0f640 1 --2- 172.21.15.115:0/1113563728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a815cd90 0x7f01a80cdd20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f01a40097d0 tx=0x7f01a402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cd096507137fa29 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:23.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.470+0000 7f01acf0d640 1 -- 172.21.15.115:0/1113563728 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4030800 con 0x7f01a815cd90 2023-12-01T03:08:23.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.470+0000 7f01acf0d640 1 -- 172.21.15.115:0/1113563728 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4030960 con 0x7f01a815cd90 2023-12-01T03:08:23.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.470+0000 7f01acf0d640 1 -- 172.21.15.115:0/1113563728 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4030cf0 con 0x7f01a815cd90 2023-12-01T03:08:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.470+0000 7f01b0173640 1 -- 172.21.15.115:0/1113563728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a815cd90 msgr2=0x7f01a80cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.470+0000 7f01b0173640 1 --2- 172.21.15.115:0/1113563728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a815cd90 0x7f01a80cdd20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f01a40097d0 tx=0x7f01a402b940 comp rx=0 tx=0).stop 2023-12-01T03:08:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.470+0000 7f01b0173640 1 -- 172.21.15.115:0/1113563728 shutdown_connections 2023-12-01T03:08:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.470+0000 7f01b0173640 1 --2- 172.21.15.115:0/1113563728 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a815cd90 0x7f01a80cdd20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.470+0000 7f01b0173640 1 -- 172.21.15.115:0/1113563728 >> 172.21.15.115:0/1113563728 conn(0x7f01a8065960 msgr2=0x7f01a8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.471+0000 7f01b0173640 1 -- 172.21.15.115:0/1113563728 shutdown_connections 2023-12-01T03:08:23.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.471+0000 7f01b0173640 1 -- 172.21.15.115:0/1113563728 wait complete. 2023-12-01T03:08:23.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.472+0000 7f01b0173640 1 Processor -- start 2023-12-01T03:08:23.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.472+0000 7f01b0173640 1 -- start start 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.473+0000 7f01b0173640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a80b5fa0 0x7f01a8156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.473+0000 7f01b0173640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a80cfd60 0x7f01a81f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.473+0000 7f01b0173640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a815cd90 0x7f01a81564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.473+0000 7f01b0173640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81e7280 con 0x7f01a80cfd60 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.473+0000 7f01b0173640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81f2ac0 con 0x7f01a815cd90 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.473+0000 7f01b0173640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81e6670 con 0x7f01a80b5fa0 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ad70e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a80cfd60 0x7f01a81f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ad70e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a80cfd60 0x7f01a81f2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35416/0 (socket says 172.21.15.115:35416) 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ad70e640 1 -- 172.21.15.115:0/4259365792 learned_addr learned my addr 172.21.15.115:0/4259365792 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01adf0f640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a80b5fa0 0x7f01a8156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ae710640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a815cd90 0x7f01a81564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ad70e640 1 -- 172.21.15.115:0/4259365792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a80b5fa0 msgr2=0x7f01a8156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ad70e640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a80b5fa0 0x7f01a8156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ad70e640 1 -- 172.21.15.115:0/4259365792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a815cd90 msgr2=0x7f01a81564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ad70e640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a815cd90 0x7f01a81564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ad70e640 1 -- 172.21.15.115:0/4259365792 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a4009400 con 0x7f01a80cfd60 2023-12-01T03:08:23.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ae710640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a815cd90 0x7f01a81564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:23.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.474+0000 7f01ad70e640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a80cfd60 0x7f01a81f2580 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f019800c760 tx=0x7f019800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:23.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.475+0000 7f019e7fc640 1 -- 172.21.15.115:0/4259365792 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198005ce0 con 0x7f01a80cfd60 2023-12-01T03:08:23.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.475+0000 7f019e7fc640 1 -- 172.21.15.115:0/4259365792 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0198005e40 con 0x7f01a80cfd60 2023-12-01T03:08:23.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.475+0000 7f01b0173640 1 -- 172.21.15.115:0/4259365792 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a81e6950 con 0x7f01a80cfd60 2023-12-01T03:08:23.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.475+0000 7f019e7fc640 1 -- 172.21.15.115:0/4259365792 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198007270 con 0x7f01a80cfd60 2023-12-01T03:08:23.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.475+0000 7f01b0173640 1 -- 172.21.15.115:0/4259365792 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a81e6f10 con 0x7f01a80cfd60 2023-12-01T03:08:23.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.476+0000 7f01b0173640 1 -- 172.21.15.115:0/4259365792 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a81e73f0 con 0x7f01a80cfd60 2023-12-01T03:08:23.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.477+0000 7f019e7fc640 1 -- 172.21.15.115:0/4259365792 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 110) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f01980073d0 con 0x7f01a80cfd60 2023-12-01T03:08:23.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.478+0000 7f019e7fc640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f018c06a580 0x7f018c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.478+0000 7f019e7fc640 1 -- 172.21.15.115:0/4259365792 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6485+0+0 (secure 0 0 0) 0x7f0198088260 con 0x7f01a80cfd60 2023-12-01T03:08:23.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.478+0000 7f01adf0f640 1 -- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f018c06a580 msgr2=0x7f018c06ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:23.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.478+0000 7f01adf0f640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f018c06a580 0x7f018c06ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:23.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.482+0000 7f019e7fc640 1 -- 172.21.15.115:0/4259365792 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f01980534e0 con 0x7f01a80cfd60 2023-12-01T03:08:23.501 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:23.500+0000 7f49b6054040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:08:23.588 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:23.586+0000 7fdca5eda040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:08:23.665 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:23.664+0000 7f49b6054040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:08:23.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.678+0000 7f01b0173640 1 -- 172.21.15.115:0/4259365792 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01a80ce5e0 con 0x7f01a80cfd60 2023-12-01T03:08:23.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.679+0000 7f01adf0f640 1 -- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f018c06a580 msgr2=0x7f018c06ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:23.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.679+0000 7f01adf0f640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f018c06a580 0x7f018c06ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:08:23.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.679+0000 7f019e7fc640 1 -- 172.21.15.115:0/4259365792 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) v1 ==== 100+0+11 (secure 0 0 0) 0x7f01a80ce5e0 con 0x7f01a80cfd60 2023-12-01T03:08:23.681 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:08:23.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.681+0000 7f01b0173640 1 -- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f018c06a580 msgr2=0x7f018c06ca40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:23.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.681+0000 7f01b0173640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f018c06a580 0x7f018c06ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.681+0000 7f01b0173640 1 -- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a80cfd60 msgr2=0x7f01a81f2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.681+0000 7f01b0173640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a80cfd60 0x7f01a81f2580 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f019800c760 tx=0x7f019800cc20 comp rx=0 tx=0).stop 2023-12-01T03:08:23.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.681+0000 7f01b0173640 1 -- 172.21.15.115:0/4259365792 shutdown_connections 2023-12-01T03:08:23.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.681+0000 7f01b0173640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f018c06a580 0x7f018c06ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.681+0000 7f01b0173640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01a815cd90 0x7f01a81564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.681+0000 7f01b0173640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a80cfd60 0x7f01a81f2580 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.681+0000 7f01b0173640 1 --2- 172.21.15.115:0/4259365792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01a80b5fa0 0x7f01a8156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.681+0000 7f01b0173640 1 -- 172.21.15.115:0/4259365792 >> 172.21.15.115:0/4259365792 conn(0x7f01a8065960 msgr2=0x7f01a80d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:23.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.682+0000 7f01b0173640 1 -- 172.21.15.115:0/4259365792 shutdown_connections 2023-12-01T03:08:23.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.682+0000 7f01b0173640 1 -- 172.21.15.115:0/4259365792 wait complete. 2023-12-01T03:08:23.702 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T03:08:23.752 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:23.750+0000 7fdca5eda040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:08:23.773 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:23.772+0000 7f49b6054040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:08:23.822 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:23.821+0000 7f49b6054040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:08:23.865 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:23.863+0000 7fdca5eda040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:08:23.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.899+0000 7f33155a9640 1 Processor -- start 2023-12-01T03:08:23.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.899+0000 7f33155a9640 1 -- start start 2023-12-01T03:08:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.900+0000 7f33155a9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f331015cd20 0x7f33100d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.901+0000 7f33155a9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100ce2e0 con 0x7f33100d1230 2023-12-01T03:08:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.901+0000 7f33155a9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100ce000 con 0x7f33100b5d80 2023-12-01T03:08:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.901+0000 7f33155a9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33100b2160 con 0x7f331015cd20 2023-12-01T03:08:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.901+0000 7f330f7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f33100d1230 0x7f33100d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57104/0 (socket says 172.21.15.115:57104) 2023-12-01T03:08:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.901+0000 7f330f7fe640 1 -- 172.21.15.115:0/1061344066 learned_addr learned my addr 172.21.15.115:0/1061344066 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.901+0000 7f330effd640 1 --2- 172.21.15.115:0/1061344066 >> v2:172.21.15.142:3300/0 conn(0x7f331015cd20 0x7f33100d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.901+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550967779 0 0) 0x7f33100ce2e0 con 0x7f33100d1230 2023-12-01T03:08:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f8003520 con 0x7f33100d1230 2023-12-01T03:08:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 974908118 0 0) 0x7f32f8003520 con 0x7f33100d1230 2023-12-01T03:08:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 >> v2:172.21.15.142:3300/0 conn(0x7f331015cd20 msgr2=0x7f33100d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 --2- 172.21.15.115:0/1061344066 >> v2:172.21.15.142:3300/0 conn(0x7f331015cd20 0x7f33100d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 >> v1:172.21.15.142:6789/0 conn(0x7f33100b5d80 legacy=0x7f33100b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33100cd3d0 con 0x7f33100d1230 2023-12-01T03:08:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f33080019f0 con 0x7f33100d1230 2023-12-01T03:08:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 >> v1:172.21.15.115:6789/0 conn(0x7f33100d1230 legacy=0x7f33100d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f80039e0 0x7f32f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32f80047c0 0x7f32f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f80051f0 0x7f32f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f800c120 con 0x7f32f80047c0 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f800c290 con 0x7f32f80039e0 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f800c450 con 0x7f32f80051f0 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.902+0000 7f330e7fc640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32f80047c0 0x7f32f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.903+0000 7f330effd640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f80039e0 0x7f32f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.903+0000 7f330f7fe640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f80051f0 0x7f32f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.903+0000 7f330e7fc640 1 -- 172.21.15.115:0/1061344066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f80051f0 msgr2=0x7f32f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.903+0000 7f330e7fc640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32f80051f0 0x7f32f800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.903+0000 7f330e7fc640 1 -- 172.21.15.115:0/1061344066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f80039e0 msgr2=0x7f32f8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.903+0000 7f330e7fc640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f80039e0 0x7f32f8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.903+0000 7f330e7fc640 1 -- 172.21.15.115:0/1061344066 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f8003520 con 0x7f32f80047c0 2023-12-01T03:08:23.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.903+0000 7f330f7fe640 1 -- 172.21.15.115:0/1061344066 reap_dead start 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.904+0000 7f330e7fc640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32f80047c0 0x7f32f8004c40 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f3304001750 tx=0x7f32f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f4ffe365c3c5b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.904+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33040052d0 con 0x7f32f80047c0 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.904+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330402e050 con 0x7f32f80047c0 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.904+0000 7f330dffb640 1 -- 172.21.15.115:0/1061344066 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304003830 con 0x7f32f80047c0 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.905+0000 7f33155a9640 1 -- 172.21.15.115:0/1061344066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32f80047c0 msgr2=0x7f32f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.905+0000 7f33155a9640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32f80047c0 0x7f32f8004c40 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f3304001750 tx=0x7f32f80057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.905+0000 7f33155a9640 1 -- 172.21.15.115:0/1061344066 shutdown_connections 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.905+0000 7f33155a9640 1 --2- 172.21.15.115:0/1061344066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32f80047c0 0x7f32f8004c40 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.905+0000 7f33155a9640 1 -- 172.21.15.115:0/1061344066 >> 172.21.15.115:0/1061344066 conn(0x7f331006a0e0 msgr2=0x7f331006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.905+0000 7f33155a9640 1 -- 172.21.15.115:0/1061344066 shutdown_connections 2023-12-01T03:08:23.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.905+0000 7f33155a9640 1 -- 172.21.15.115:0/1061344066 wait complete. 2023-12-01T03:08:23.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.907+0000 7f33155a9640 1 Processor -- start 2023-12-01T03:08:23.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.907+0000 7f33155a9640 1 -- start start 2023-12-01T03:08:23.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.908+0000 7f33155a9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f80047c0 0x7f33101f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.908+0000 7f33155a9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f331015fd20 0x7f3310152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.908+0000 7f33155a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f331015dd80 0x7f3310152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.908+0000 7f33155a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331015e340 con 0x7f331015dd80 2023-12-01T03:08:23.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.908+0000 7f33155a9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331015e620 con 0x7f32f80047c0 2023-12-01T03:08:23.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.908+0000 7f33155a9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331015e900 con 0x7f331015fd20 2023-12-01T03:08:23.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.908+0000 7f330f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f331015dd80 0x7f3310152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.908+0000 7f330f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f331015dd80 0x7f3310152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35430/0 (socket says 172.21.15.115:35430) 2023-12-01T03:08:23.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.908+0000 7f330f7fe640 1 -- 172.21.15.115:0/3089168049 learned_addr learned my addr 172.21.15.115:0/3089168049 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f330e7fc640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f331015fd20 0x7f3310152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f330effd640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f80047c0 0x7f33101f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f330f7fe640 1 -- 172.21.15.115:0/3089168049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f331015fd20 msgr2=0x7f3310152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f330f7fe640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f331015fd20 0x7f3310152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f330f7fe640 1 -- 172.21.15.115:0/3089168049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f80047c0 msgr2=0x7f33101f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f330f7fe640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f80047c0 0x7f33101f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f330f7fe640 1 -- 172.21.15.115:0/3089168049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331015ebe0 con 0x7f331015dd80 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f330f7fe640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f331015dd80 0x7f3310152750 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f33100ce4f0 tx=0x7f330800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f32f77fe640 1 -- 172.21.15.115:0/3089168049 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330800ec10 con 0x7f331015dd80 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f32f77fe640 1 -- 172.21.15.115:0/3089168049 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330800ed70 con 0x7f331015dd80 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f33155a9640 1 -- 172.21.15.115:0/3089168049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3310154750 con 0x7f331015dd80 2023-12-01T03:08:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f32f77fe640 1 -- 172.21.15.115:0/3089168049 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330800eee0 con 0x7f331015dd80 2023-12-01T03:08:23.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.909+0000 7f33155a9640 1 -- 172.21.15.115:0/3089168049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33101f1fe0 con 0x7f331015dd80 2023-12-01T03:08:23.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.911+0000 7f33155a9640 1 -- 172.21.15.115:0/3089168049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33100738e0 con 0x7f331015dd80 2023-12-01T03:08:23.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.912+0000 7f32f77fe640 1 -- 172.21.15.115:0/3089168049 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 110) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f3308013660 con 0x7f331015dd80 2023-12-01T03:08:23.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.912+0000 7f32f77fe640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f32fc06a580 0x7f32fc06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:23.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.913+0000 7f32f77fe640 1 -- 172.21.15.115:0/3089168049 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6485+0+0 (secure 0 0 0) 0x7f33080891a0 con 0x7f331015dd80 2023-12-01T03:08:23.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.913+0000 7f330effd640 1 -- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f32fc06a580 msgr2=0x7f32fc06ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:23.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.913+0000 7f330effd640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f32fc06a580 0x7f32fc06ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:23.918 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:23.916+0000 7fdca5eda040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:08:23.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:23.917+0000 7f32f77fe640 1 -- 172.21.15.115:0/3089168049 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3308054370 con 0x7f331015dd80 2023-12-01T03:08:23.930 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:23.929+0000 7f49b6054040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:08:24.028 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:24.026+0000 7fdca5eda040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:08:24.036 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:24.035+0000 7f49b6054040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:08:24.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.114+0000 7f330effd640 1 -- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f32fc06a580 msgr2=0x7f32fc06ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:24.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.114+0000 7f330effd640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f32fc06a580 0x7f32fc06ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:08:24.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.116+0000 7f33155a9640 1 -- 172.21.15.115:0/3089168049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3310154940 con 0x7f331015dd80 2023-12-01T03:08:24.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.117+0000 7f32f77fe640 1 -- 172.21.15.115:0/3089168049 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 14 entries v183) v1 ==== 81+0+817 (secure 0 0 0) 0x7f3308057990 con 0x7f331015dd80 2023-12-01T03:08:24.119 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/363147954 2023-12-02T03:03:21.455245+0000 2023-12-01T03:08:24.119 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1462551426 2023-12-02T03:03:21.455245+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/718367345 2023-12-02T03:03:21.455245+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2970056045 2023-12-02T03:03:21.455245+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/858082591 2023-12-02T03:03:21.455245+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/88317 2023-12-02T02:52:41.228363+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3219939439 2023-12-02T03:03:21.455245+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/88317 2023-12-02T02:52:41.228363+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/673199316 2023-12-02T02:52:41.228363+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2404622555 2023-12-02T02:52:41.228363+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1684885368 2023-12-02T02:52:41.228363+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/4185429564 2023-12-02T02:52:41.228363+0000 2023-12-01T03:08:24.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/682039355 2023-12-02T02:52:41.228363+0000 2023-12-01T03:08:24.121 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2564902155 2023-12-02T02:52:41.228363+0000 2023-12-01T03:08:24.121 INFO:teuthology.orchestra.run.smithi115.stderr:listed 14 entries 2023-12-01T03:08:24.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 -- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f32fc06a580 msgr2=0x7f32fc06ca40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:24.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f32fc06a580 0x7f32fc06ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 -- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f331015dd80 msgr2=0x7f3310152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:24.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f331015dd80 0x7f3310152750 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f33100ce4f0 tx=0x7f330800e530 comp rx=0 tx=0).stop 2023-12-01T03:08:24.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 -- 172.21.15.115:0/3089168049 shutdown_connections 2023-12-01T03:08:24.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f32fc06a580 0x7f32fc06ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f331015fd20 0x7f3310152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f331015dd80 0x7f3310152750 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 --2- 172.21.15.115:0/3089168049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32f80047c0 0x7f33101f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 -- 172.21.15.115:0/3089168049 >> 172.21.15.115:0/3089168049 conn(0x7f331006a0e0 msgr2=0x7f331015b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 -- 172.21.15.115:0/3089168049 shutdown_connections 2023-12-01T03:08:24.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.120+0000 7f33155a9640 1 -- 172.21.15.115:0/3089168049 wait complete. 2023-12-01T03:08:24.137 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:24.136+0000 7fdca5eda040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:08:24.139 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T03:08:24.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:24.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T03:08:24.192 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:24.191+0000 7f49b6054040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:08:24.230 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T03:08:24.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:24.231 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T03:08:24.242 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:24.241+0000 7f49b6054040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:08:24.278 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:08:24.279 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:08:24.304 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:24.302+0000 7fdca5eda040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:08:24.334 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:08:24.334 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:08:24.334 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:08:24.354 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:24.352+0000 7fdca5eda040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:08:24.364 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:24.363+0000 7f49b6054040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:08:24.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:08:24.388 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:08:24.388 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:08:24.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:08:24.443 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:08:24.479 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:24.477+0000 7fdca5eda040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:08:24.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.645+0000 7f791924d640 1 Processor -- start 2023-12-01T03:08:24.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.645+0000 7f791924d640 1 -- start start 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.646+0000 7f791924d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f791415d7b0 0x7f791415db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.646+0000 7f791924d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7914160670 0x7f79140cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.646+0000 7f791924d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79140cddc0 0x7f79140d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.646+0000 7f791924d640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79140cd3f0 con 0x7f7914160670 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.646+0000 7f791924d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79140cd0f0 con 0x7f791415d7b0 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.646+0000 7f791924d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7914160ec0 con 0x7f79140cddc0 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f790bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7914160670 0x7f79140cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f790bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7914160670 0x7f79140cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35438/0 (socket says 172.21.15.115:35438) 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f790bfff640 1 -- 172.21.15.115:0/1131785757 learned_addr learned my addr 172.21.15.115:0/1131785757 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f7918a4c640 1 --2- 172.21.15.115:0/1131785757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79140cddc0 0x7f79140d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f7913fff640 1 --2- 172.21.15.115:0/1131785757 >> v2:172.21.15.142:3300/0 conn(0x7f791415d7b0 0x7f791415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:24.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f790bfff640 1 -- 172.21.15.115:0/1131785757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79140cddc0 msgr2=0x7f79140d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:24.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f790bfff640 1 --2- 172.21.15.115:0/1131785757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79140cddc0 0x7f79140d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f790bfff640 1 -- 172.21.15.115:0/1131785757 >> v2:172.21.15.142:3300/0 conn(0x7f791415d7b0 msgr2=0x7f791415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:24.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f790bfff640 1 --2- 172.21.15.115:0/1131785757 >> v2:172.21.15.142:3300/0 conn(0x7f791415d7b0 0x7f791415db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f790bfff640 1 -- 172.21.15.115:0/1131785757 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79141611a0 con 0x7f7914160670 2023-12-01T03:08:24.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.647+0000 7f7918a4c640 1 --2- 172.21.15.115:0/1131785757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79140cddc0 0x7f79140d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:24.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.648+0000 7f790bfff640 1 --2- 172.21.15.115:0/1131785757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914160670 0x7f79140cd880 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f79140720e0 tx=0x7f78fc02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13678abd9824ccf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:24.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f79137fe640 1 -- 172.21.15.115:0/1131785757 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc030850 con 0x7f7914160670 2023-12-01T03:08:24.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f79137fe640 1 -- 172.21.15.115:0/1131785757 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc0309b0 con 0x7f7914160670 2023-12-01T03:08:24.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f79137fe640 1 -- 172.21.15.115:0/1131785757 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc030d40 con 0x7f7914160670 2023-12-01T03:08:24.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f791924d640 1 -- 172.21.15.115:0/1131785757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914160670 msgr2=0x7f79140cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:24.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f791924d640 1 --2- 172.21.15.115:0/1131785757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914160670 0x7f79140cd880 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f79140720e0 tx=0x7f78fc02b990 comp rx=0 tx=0).stop 2023-12-01T03:08:24.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f791924d640 1 -- 172.21.15.115:0/1131785757 shutdown_connections 2023-12-01T03:08:24.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f791924d640 1 --2- 172.21.15.115:0/1131785757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79140cddc0 0x7f79140d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f791924d640 1 --2- 172.21.15.115:0/1131785757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7914160670 0x7f79140cd880 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f791924d640 1 --2- 172.21.15.115:0/1131785757 >> v2:172.21.15.142:3300/0 conn(0x7f791415d7b0 0x7f791415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f791924d640 1 -- 172.21.15.115:0/1131785757 >> 172.21.15.115:0/1131785757 conn(0x7f791406a0e0 msgr2=0x7f791406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:24.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.649+0000 7f791924d640 1 -- 172.21.15.115:0/1131785757 shutdown_connections 2023-12-01T03:08:24.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.650+0000 7f791924d640 1 -- 172.21.15.115:0/1131785757 wait complete. 2023-12-01T03:08:24.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.651+0000 7f791924d640 1 Processor -- start 2023-12-01T03:08:24.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.651+0000 7f791924d640 1 -- start start 2023-12-01T03:08:24.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.652+0000 7f791924d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79140cddc0 0x7f79140b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:24.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.652+0000 7f791924d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791415d7b0 0x7f79141f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:24.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.652+0000 7f791924d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7914160670 0x7f79140cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:24.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.652+0000 7f791924d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79140b4390 con 0x7f79140cddc0 2023-12-01T03:08:24.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.652+0000 7f791924d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79140c9ad0 con 0x7f791415d7b0 2023-12-01T03:08:24.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.652+0000 7f791924d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79140b45a0 con 0x7f7914160670 2023-12-01T03:08:24.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.652+0000 7f7913fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79140cddc0 0x7f79140b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:24.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f7913fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79140cddc0 0x7f79140b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35440/0 (socket says 172.21.15.115:35440) 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f7913fff640 1 -- 172.21.15.115:0/2710287660 learned_addr learned my addr 172.21.15.115:0/2710287660 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f7918a4c640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7914160670 0x7f79140cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f790bfff640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791415d7b0 0x7f79141f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f7913fff640 1 -- 172.21.15.115:0/2710287660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7914160670 msgr2=0x7f79140cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f7913fff640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7914160670 0x7f79140cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f7913fff640 1 -- 172.21.15.115:0/2710287660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791415d7b0 msgr2=0x7f79141f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f7913fff640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791415d7b0 0x7f79141f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f7913fff640 1 -- 172.21.15.115:0/2710287660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc0094d0 con 0x7f79140cddc0 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f7918a4c640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7914160670 0x7f79140cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f7913fff640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79140cddc0 0x7f79140b3e50 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f790400c7f0 tx=0x7f790400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.653+0000 7f79117fa640 1 -- 172.21.15.115:0/2710287660 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904004220 con 0x7f79140cddc0 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.654+0000 7f79117fa640 1 -- 172.21.15.115:0/2710287660 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79040043a0 con 0x7f79140cddc0 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.654+0000 7f79117fa640 1 -- 172.21.15.115:0/2710287660 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904007860 con 0x7f79140cddc0 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.654+0000 7f791924d640 1 -- 172.21.15.115:0/2710287660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79140b4880 con 0x7f79140cddc0 2023-12-01T03:08:24.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.654+0000 7f791924d640 1 -- 172.21.15.115:0/2710287660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79140b4e40 con 0x7f79140cddc0 2023-12-01T03:08:24.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.655+0000 7f791924d640 1 -- 172.21.15.115:0/2710287660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79140673a0 con 0x7f79140cddc0 2023-12-01T03:08:24.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.656+0000 7f79117fa640 1 -- 172.21.15.115:0/2710287660 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 110) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f7904007bd0 con 0x7f79140cddc0 2023-12-01T03:08:24.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.656+0000 7f79117fa640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f78f406a550 0x7f78f406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:24.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.657+0000 7f79117fa640 1 -- 172.21.15.115:0/2710287660 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6485+0+0 (secure 0 0 0) 0x7f7904088e80 con 0x7f79140cddc0 2023-12-01T03:08:24.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.657+0000 7f790bfff640 1 -- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f78f406a550 msgr2=0x7f78f406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:24.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.657+0000 7f790bfff640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f78f406a550 0x7f78f406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:24.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.661+0000 7f79117fa640 1 -- 172.21.15.115:0/2710287660 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7904053f40 con 0x7f79140cddc0 2023-12-01T03:08:24.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.858+0000 7f790bfff640 1 -- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f78f406a550 msgr2=0x7f78f406ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:24.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.858+0000 7f790bfff640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f78f406a550 0x7f78f406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:08:24.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.887+0000 7f791924d640 1 -- 172.21.15.115:0/2710287660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79140d3100 con 0x7f79140cddc0 2023-12-01T03:08:24.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.888+0000 7f79117fa640 1 -- 172.21.15.115:0/2710287660 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f790401e020 con 0x7f79140cddc0 2023-12-01T03:08:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:24.890 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1616304732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1616304732},{"type":"v1","addr":"172.21.15.115:6835","nonce":1616304732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6766,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3109897397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3109897397},{"type":"v1","addr":"172.21.15.142:6834","nonce":3109897397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/446557816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":446557816},{"type":"v1","addr":"172.21.15.142:6835","nonce":446557816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:52:21.959726+0000","modified":"2023-12-01T02:52:24.991376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9549":{"gid":9549,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1064849689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1064849689},{"type":"v1","addr":"172.21.15.115:6837","nonce":1064849689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-01T03:08:24.890 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 79 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 -- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f78f406a550 msgr2=0x7f78f406ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f78f406a550 0x7f78f406ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 -- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79140cddc0 msgr2=0x7f79140b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79140cddc0 0x7f79140b3e50 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f790400c7f0 tx=0x7f790400ccb0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 -- 172.21.15.115:0/2710287660 shutdown_connections 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f78f406a550 0x7f78f406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7914160670 0x7f79140cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791415d7b0 0x7f79141f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 --2- 172.21.15.115:0/2710287660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79140cddc0 0x7f79140b3e50 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 -- 172.21.15.115:0/2710287660 >> 172.21.15.115:0/2710287660 conn(0x7f791406a0e0 msgr2=0x7f791415fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 -- 172.21.15.115:0/2710287660 shutdown_connections 2023-12-01T03:08:24.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:24.891+0000 7f791924d640 1 -- 172.21.15.115:0/2710287660 wait complete. 2023-12-01T03:08:24.912 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:08:25.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.130+0000 7f2026843640 1 Processor -- start 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.130+0000 7f2026843640 1 -- start start 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f2026843640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f202015cd20 0x7f20200d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f2026843640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200d1230 0x7f20200d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f2026843640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20200d1b90 con 0x7f20200b5d80 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f2026843640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20200b2160 con 0x7f202015cd20 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f2026843640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20200b8c80 con 0x7f20200d1230 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f201ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f202015cd20 0x7f20200d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f201ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f202015cd20 0x7f20200d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35454/0 (socket says 172.21.15.115:35454) 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f201ffff640 1 -- 172.21.15.115:0/522451042 learned_addr learned my addr 172.21.15.115:0/522451042 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f2024de0640 1 --2- 172.21.15.115:0/522451042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200d1230 0x7f20200d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f201ffff640 1 -- 172.21.15.115:0/522451042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200d1230 msgr2=0x7f20200d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:25.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f201ffff640 1 --2- 172.21.15.115:0/522451042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200d1230 0x7f20200d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f201ffff640 1 -- 172.21.15.115:0/522451042 >> v1:172.21.15.142:6789/0 conn(0x7f20200b5d80 legacy=0x7f20200b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:25.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.132+0000 7f201ffff640 1 -- 172.21.15.115:0/522451042 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20200ce2e0 con 0x7f202015cd20 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.133+0000 7f201ffff640 1 --2- 172.21.15.115:0/522451042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f202015cd20 0x7f20200d24d0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f2010009870 tx=0x7f201002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd3ac8d9c7f58e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.134+0000 7f201effd640 1 -- 172.21.15.115:0/522451042 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20100317c0 con 0x7f202015cd20 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.134+0000 7f201effd640 1 -- 172.21.15.115:0/522451042 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2010031920 con 0x7f202015cd20 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.134+0000 7f201effd640 1 -- 172.21.15.115:0/522451042 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010031cb0 con 0x7f202015cd20 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.134+0000 7f2026843640 1 -- 172.21.15.115:0/522451042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f202015cd20 msgr2=0x7f20200d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.134+0000 7f2026843640 1 --2- 172.21.15.115:0/522451042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f202015cd20 0x7f20200d24d0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f2010009870 tx=0x7f201002b860 comp rx=0 tx=0).stop 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.135+0000 7f2026843640 1 -- 172.21.15.115:0/522451042 shutdown_connections 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.135+0000 7f2026843640 1 --2- 172.21.15.115:0/522451042 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200d1230 0x7f20200d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.135+0000 7f2026843640 1 --2- 172.21.15.115:0/522451042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f202015cd20 0x7f20200d24d0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.135+0000 7f2026843640 1 -- 172.21.15.115:0/522451042 >> 172.21.15.115:0/522451042 conn(0x7f202006a0e0 msgr2=0x7f202006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.135+0000 7f2026843640 1 -- 172.21.15.115:0/522451042 shutdown_connections 2023-12-01T03:08:25.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.135+0000 7f2026843640 1 -- 172.21.15.115:0/522451042 wait complete. 2023-12-01T03:08:25.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.136+0000 7f2026843640 1 Processor -- start 2023-12-01T03:08:25.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.137+0000 7f2026843640 1 -- start start 2023-12-01T03:08:25.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.137+0000 7f2026843640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200b5d80 0x7f20201510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.137+0000 7f2026843640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200d1230 0x7f202015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.137+0000 7f2026843640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f202015cd20 0x7f202015f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.137+0000 7f2026843640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20200c9d20 con 0x7f20200d1230 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f2026843640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20200b48b0 con 0x7f202015cd20 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f2026843640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20200b4050 con 0x7f20200b5d80 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200d1230 0x7f202015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200d1230 0x7f202015ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35456/0 (socket says 172.21.15.115:35456) 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201f7fe640 1 -- 172.21.15.115:0/3028661040 learned_addr learned my addr 172.21.15.115:0/3028661040 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201ffff640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200b5d80 0x7f20201510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201f7fe640 1 -- 172.21.15.115:0/3028661040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200b5d80 msgr2=0x7f20201510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f2024de0640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f202015cd20 0x7f202015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201f7fe640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200b5d80 0x7f20201510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201f7fe640 1 -- 172.21.15.115:0/3028661040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f202015cd20 msgr2=0x7f202015f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201f7fe640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f202015cd20 0x7f202015f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201f7fe640 1 -- 172.21.15.115:0/3028661040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100094d0 con 0x7f20200d1230 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201ffff640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200b5d80 0x7f20201510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:25.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.138+0000 7f201f7fe640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200d1230 0x7f202015ed00 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f201400c430 tx=0x7f201400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:25.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.139+0000 7f201cff9640 1 -- 172.21.15.115:0/3028661040 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014011790 con 0x7f20200d1230 2023-12-01T03:08:25.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.139+0000 7f201cff9640 1 -- 172.21.15.115:0/3028661040 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20140118f0 con 0x7f20200d1230 2023-12-01T03:08:25.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.139+0000 7f2026843640 1 -- 172.21.15.115:0/3028661040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20200b42f0 con 0x7f20200d1230 2023-12-01T03:08:25.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.139+0000 7f201cff9640 1 -- 172.21.15.115:0/3028661040 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014011ce0 con 0x7f20200d1230 2023-12-01T03:08:25.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.139+0000 7f2026843640 1 -- 172.21.15.115:0/3028661040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202015ea60 con 0x7f20200d1230 2023-12-01T03:08:25.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.140+0000 7f2026843640 1 -- 172.21.15.115:0/3028661040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202004e890 con 0x7f20200d1230 2023-12-01T03:08:25.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.141+0000 7f201cff9640 1 -- 172.21.15.115:0/3028661040 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 110) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f201401c460 con 0x7f20200d1230 2023-12-01T03:08:25.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.142+0000 7f201cff9640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f200c06a550 0x7f200c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.142+0000 7f201cff9640 1 -- 172.21.15.115:0/3028661040 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6485+0+0 (secure 0 0 0) 0x7f2014088c70 con 0x7f20200d1230 2023-12-01T03:08:25.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.142+0000 7f201ffff640 1 -- 172.21.15.115:0/3028661040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f200c06a550 msgr2=0x7f200c06ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:25.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.142+0000 7f201ffff640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f200c06a550 0x7f200c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:25.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.146+0000 7f201cff9640 1 -- 172.21.15.115:0/3028661040 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2014053ed0 con 0x7f20200d1230 2023-12-01T03:08:25.189 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:25.188+0000 7f49b6054040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:08:25.322 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.320+0000 7fdca5eda040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:08:25.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.328+0000 7f2026843640 1 -- 172.21.15.115:0/3028661040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20200d0560 con 0x7f20200d1230 2023-12-01T03:08:25.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.330+0000 7f201cff9640 1 -- 172.21.15.115:0/3028661040 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+17338 (secure 0 0 0) 0x7f20140574f0 con 0x7f20200d1230 2023-12-01T03:08:25.332 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:25.333 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":183,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:08:23.399100+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:52:20.047003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-01T02:52:20.974031+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":183,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2f","mappings":[{"from":2,"to":5}]},{"pgid":"21.3b","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/363147954":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/1462551426":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/718367345":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/2970056045":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/858082591":"2023-12-02T03:03:21.455245+0000","172.21.15.115:6824/88317":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/3219939439":"2023-12-02T03:03:21.455245+0000","172.21.15.115:6822/88317":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/673199316":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/2404622555":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/1684885368":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/4185429564":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/682039355":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/2564902155":"2023-12-02T02:52:41.228363+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:08:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 -- 172.21.15.115:0/3028661040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f200c06a550 msgr2=0x7f200c06ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f200c06a550 0x7f200c06ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 -- 172.21.15.115:0/3028661040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200d1230 msgr2=0x7f202015ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200d1230 0x7f202015ed00 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f201400c430 tx=0x7f201400c8f0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 -- 172.21.15.115:0/3028661040 shutdown_connections 2023-12-01T03:08:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f200c06a550 0x7f200c06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f202015cd20 0x7f202015f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200d1230 0x7f202015ed00 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 --2- 172.21.15.115:0/3028661040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20200b5d80 0x7f20201510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 -- 172.21.15.115:0/3028661040 >> 172.21.15.115:0/3028661040 conn(0x7f202006a0e0 msgr2=0x7f202015b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 -- 172.21.15.115:0/3028661040 shutdown_connections 2023-12-01T03:08:25.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.333+0000 7f2026843640 1 -- 172.21.15.115:0/3028661040 wait complete. 2023-12-01T03:08:25.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T02:52:20.047003+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T02:52:20.974031+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 183, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:08:25.354 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:08:25.484 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:25.483+0000 7f49b6054040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:08:25.540 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:25.539+0000 7f49b6054040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:08:25.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.563+0000 7f3c7ce22640 1 Processor -- start 2023-12-01T03:08:25.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.563+0000 7f3c7ce22640 1 -- start start 2023-12-01T03:08:25.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.564+0000 7f3c7ce22640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78153c30 0x7f3c780ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.564+0000 7f3c7ce22640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3c780cfb80 0x7f3c78160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.564+0000 7f3c7ce22640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c780cdcd0 con 0x7f3c78154010 2023-12-01T03:08:25.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.564+0000 7f3c7ce22640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c78161830 con 0x7f3c780cfb80 2023-12-01T03:08:25.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.564+0000 7f3c7ce22640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780cd3a0 con 0x7f3c78153c30 2023-12-01T03:08:25.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.565+0000 7f3c777fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78153c30 0x7f3c780ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.565+0000 7f3c777fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78153c30 0x7f3c780ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:40276/0 (socket says 172.21.15.115:40276) 2023-12-01T03:08:25.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.565+0000 7f3c777fe640 1 -- 172.21.15.115:0/271235402 learned_addr learned my addr 172.21.15.115:0/271235402 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:25.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.565+0000 7f3c77fff640 1 --2- 172.21.15.115:0/271235402 >> v2:172.21.15.142:3300/0 conn(0x7f3c780cfb80 0x7f3c78160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.566+0000 7f3c777fe640 1 -- 172.21.15.115:0/271235402 >> v2:172.21.15.142:3300/0 conn(0x7f3c780cfb80 msgr2=0x7f3c78160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:25.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.566+0000 7f3c777fe640 1 --2- 172.21.15.115:0/271235402 >> v2:172.21.15.142:3300/0 conn(0x7f3c780cfb80 0x7f3c78160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.566+0000 7f3c777fe640 1 -- 172.21.15.115:0/271235402 >> v1:172.21.15.142:6789/0 conn(0x7f3c78154010 legacy=0x7f3c780cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.566+0000 7f3c777fe640 1 -- 172.21.15.115:0/271235402 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c780cd680 con 0x7f3c78153c30 2023-12-01T03:08:25.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.567+0000 7f3c777fe640 1 --2- 172.21.15.115:0/271235402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78153c30 0x7f3c780ce160 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009e50 tx=0x7f3c6802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eabe6214473c58b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:25.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.567+0000 7f3c76ffd640 1 -- 172.21.15.115:0/271235402 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6802cb00 con 0x7f3c78153c30 2023-12-01T03:08:25.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.567+0000 7f3c76ffd640 1 -- 172.21.15.115:0/271235402 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6802cc60 con 0x7f3c78153c30 2023-12-01T03:08:25.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.567+0000 7f3c76ffd640 1 -- 172.21.15.115:0/271235402 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680343f0 con 0x7f3c78153c30 2023-12-01T03:08:25.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.568+0000 7f3c7ce22640 1 -- 172.21.15.115:0/271235402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78153c30 msgr2=0x7f3c780ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.568+0000 7f3c7ce22640 1 --2- 172.21.15.115:0/271235402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78153c30 0x7f3c780ce160 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009e50 tx=0x7f3c6802bf60 comp rx=0 tx=0).stop 2023-12-01T03:08:25.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.568+0000 7f3c7ce22640 1 -- 172.21.15.115:0/271235402 shutdown_connections 2023-12-01T03:08:25.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.568+0000 7f3c7ce22640 1 --2- 172.21.15.115:0/271235402 >> v2:172.21.15.142:3300/0 conn(0x7f3c780cfb80 0x7f3c78160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.568+0000 7f3c7ce22640 1 --2- 172.21.15.115:0/271235402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78153c30 0x7f3c780ce160 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.568+0000 7f3c7ce22640 1 -- 172.21.15.115:0/271235402 >> 172.21.15.115:0/271235402 conn(0x7f3c7806a0b0 msgr2=0x7f3c7806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:25.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.568+0000 7f3c7ce22640 1 -- 172.21.15.115:0/271235402 shutdown_connections 2023-12-01T03:08:25.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.568+0000 7f3c7ce22640 1 -- 172.21.15.115:0/271235402 wait complete. 2023-12-01T03:08:25.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.570+0000 7f3c7ce22640 1 Processor -- start 2023-12-01T03:08:25.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.570+0000 7f3c7ce22640 1 -- start start 2023-12-01T03:08:25.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.572+0000 7f3c7ce22640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c780cfb80 0x7f3c78150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.572+0000 7f3c7ce22640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c78153c30 0x7f3c780c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.572+0000 7f3c7ce22640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78154010 0x7f3c78157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.572+0000 7f3c7ce22640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780cb070 con 0x7f3c780cfb80 2023-12-01T03:08:25.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.572+0000 7f3c7ce22640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780ca460 con 0x7f3c78153c30 2023-12-01T03:08:25.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.572+0000 7f3c7ce22640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780ca740 con 0x7f3c78154010 2023-12-01T03:08:25.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.572+0000 7f3c777fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c780cfb80 0x7f3c78150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.572+0000 7f3c777fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c780cfb80 0x7f3c78150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35468/0 (socket says 172.21.15.115:35468) 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.572+0000 7f3c777fe640 1 -- 172.21.15.115:0/1209686222 learned_addr learned my addr 172.21.15.115:0/1209686222 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.573+0000 7f3c77fff640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78154010 0x7f3c78157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.573+0000 7f3c6effd640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c78153c30 0x7f3c780c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.573+0000 7f3c777fe640 1 -- 172.21.15.115:0/1209686222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78154010 msgr2=0x7f3c78157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.573+0000 7f3c777fe640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78154010 0x7f3c78157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.573+0000 7f3c777fe640 1 -- 172.21.15.115:0/1209686222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c78153c30 msgr2=0x7f3c780c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.573+0000 7f3c777fe640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c78153c30 0x7f3c780c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.573+0000 7f3c777fe640 1 -- 172.21.15.115:0/1209686222 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68009b30 con 0x7f3c780cfb80 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.573+0000 7f3c777fe640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c780cfb80 0x7f3c78150720 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f3c68000f80 tx=0x7f3c68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.574+0000 7f3c74ff9640 1 -- 172.21.15.115:0/1209686222 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68009510 con 0x7f3c780cfb80 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.574+0000 7f3c74ff9640 1 -- 172.21.15.115:0/1209686222 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6803e450 con 0x7f3c780cfb80 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.574+0000 7f3c74ff9640 1 -- 172.21.15.115:0/1209686222 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6803e800 con 0x7f3c780cfb80 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.574+0000 7f3c7ce22640 1 -- 172.21.15.115:0/1209686222 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c780caa20 con 0x7f3c780cfb80 2023-12-01T03:08:25.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.574+0000 7f3c7ce22640 1 -- 172.21.15.115:0/1209686222 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c78150c60 con 0x7f3c780cfb80 2023-12-01T03:08:25.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.575+0000 7f3c7ce22640 1 -- 172.21.15.115:0/1209686222 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c780618d0 con 0x7f3c780cfb80 2023-12-01T03:08:25.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.576+0000 7f3c74ff9640 1 -- 172.21.15.115:0/1209686222 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 110) v1 ==== 88721+0+0 (secure 0 0 0) 0x7f3c6803e960 con 0x7f3c780cfb80 2023-12-01T03:08:25.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.577+0000 7f3c74ff9640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3c5806a550 0x7f3c5806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.577+0000 7f3c74ff9640 1 -- 172.21.15.115:0/1209686222 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6485+0+0 (secure 0 0 0) 0x7f3c680aba30 con 0x7f3c780cfb80 2023-12-01T03:08:25.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.577+0000 7f3c6effd640 1 -- 172.21.15.115:0/1209686222 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3c5806a550 msgr2=0x7f3c5806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:25.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.577+0000 7f3c6effd640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3c5806a550 0x7f3c5806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:25.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.581+0000 7f3c74ff9640 1 -- 172.21.15.115:0/1209686222 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3c68076c10 con 0x7f3c780cfb80 2023-12-01T03:08:25.646 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.644+0000 7fdca5eda040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:08:25.654 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:25.653+0000 7f49b6054040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:08:25.705 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.703+0000 7fdca5eda040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:08:25.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.771+0000 7f3c7ce22640 1 -- 172.21.15.115:0/1209686222 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c78160540 con 0x7f3c780cfb80 2023-12-01T03:08:25.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.771+0000 7f3c74ff9640 1 -- 172.21.15.115:0/1209686222 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c6807a230 con 0x7f3c780cfb80 2023-12-01T03:08:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 -- 172.21.15.115:0/1209686222 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3c5806a550 msgr2=0x7f3c5806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3c5806a550 0x7f3c5806ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 -- 172.21.15.115:0/1209686222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c780cfb80 msgr2=0x7f3c78150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c780cfb80 0x7f3c78150720 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f3c68000f80 tx=0x7f3c68004a40 comp rx=0 tx=0).stop 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 -- 172.21.15.115:0/1209686222 shutdown_connections 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3c5806a550 0x7f3c5806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c78154010 0x7f3c78157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c78153c30 0x7f3c780c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 --2- 172.21.15.115:0/1209686222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c780cfb80 0x7f3c78150720 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 -- 172.21.15.115:0/1209686222 >> 172.21.15.115:0/1209686222 conn(0x7f3c7806a0b0 msgr2=0x7f3c78151aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:25.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 -- 172.21.15.115:0/1209686222 shutdown_connections 2023-12-01T03:08:25.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.774+0000 7f3c7ce22640 1 -- 172.21.15.115:0/1209686222 wait complete. 2023-12-01T03:08:25.794 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:08:25.824 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.822+0000 7fdca5eda040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:08:25.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.902+0000 7fdc279c5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:25.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.902+0000 7fdc279c5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:25.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.902+0000 7fdc279c5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:25.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.902+0000 7fdc279c5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:25.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.902+0000 7fdc279c5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:25.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.902+0000 7fdc2a9cb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:25.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.902+0000 7fdc2a9cb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:25.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.902+0000 7fdc2a9cb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:25.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.902+0000 7fdc2a9cb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:25.904 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:25.902+0000 7fdc2a9cb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:25.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.990+0000 7f2254ffb640 1 Processor -- start 2023-12-01T03:08:25.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.990+0000 7f2254ffb640 1 -- start start 2023-12-01T03:08:25.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.991+0000 7f2254ffb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22500b0e60 0x7f22500cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.991+0000 7f2254ffb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500cc9a0 con 0x7f225015ac80 2023-12-01T03:08:25.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.991+0000 7f2254ffb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500cbd90 con 0x7f22500ce1b0 2023-12-01T03:08:25.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.991+0000 7f2254ffb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225015b060 con 0x7f22500b0e60 2023-12-01T03:08:25.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224e59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f225015ac80 0x7f22500cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57106/0 (socket says 172.21.15.115:57106) 2023-12-01T03:08:25.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224e59c640 1 -- 172.21.15.115:0/958736027 learned_addr learned my addr 172.21.15.115:0/958736027 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:25.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224ed9d640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22500b0e60 0x7f22500cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857079678 0 0) 0x7f22500cc9a0 con 0x7f225015ac80 2023-12-01T03:08:25.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2238003520 con 0x7f225015ac80 2023-12-01T03:08:25.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1344457396 0 0) 0x7f2238003520 con 0x7f225015ac80 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22500b0e60 msgr2=0x7f22500cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224d59a640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22500b0e60 0x7f22500cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 >> v1:172.21.15.142:6789/0 conn(0x7f22500ce1b0 legacy=0x7f225006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22500cc070 con 0x7f225015ac80 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2240001670 con 0x7f225015ac80 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.992+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 >> v1:172.21.15.115:6789/0 conn(0x7f225015ac80 legacy=0x7f22500cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224d59a640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22380039a0 0x7f2238003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224d59a640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22380047c0 0x7f2238004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224d59a640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2238005180 0x7f223800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225015b060 con 0x7f22380039a0 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223800c020 con 0x7f22380047c0 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223800c1e0 con 0x7f2238005180 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224ed9d640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22500b0e60 0x7f22500cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224e59c640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22380039a0 0x7f2238003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224e59c640 1 -- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2238005180 msgr2=0x7f223800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:25.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224dd9b640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22380047c0 0x7f2238004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:25.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224e59c640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2238005180 0x7f223800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224e59c640 1 -- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22380047c0 msgr2=0x7f2238004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:25.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224e59c640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22380047c0 0x7f2238004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.993+0000 7f224e59c640 1 -- 172.21.15.115:0/958736027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2238003520 con 0x7f22380039a0 2023-12-01T03:08:25.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.994+0000 7f224e59c640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22380039a0 0x7f2238003dc0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f224000c240 tx=0x7f22380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41edc117b88a2184 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:25.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.995+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240005ad0 con 0x7f22380039a0 2023-12-01T03:08:25.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.995+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2240005c30 con 0x7f22380039a0 2023-12-01T03:08:25.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.995+0000 7f224d59a640 1 -- 172.21.15.115:0/958736027 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224002c8b0 con 0x7f22380039a0 2023-12-01T03:08:25.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.995+0000 7f2254ffb640 1 -- 172.21.15.115:0/958736027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22380039a0 msgr2=0x7f2238003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:25.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.995+0000 7f2254ffb640 1 --2- 172.21.15.115:0/958736027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22380039a0 0x7f2238003dc0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f224000c240 tx=0x7f22380057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:25.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.995+0000 7f224ed9d640 1 -- 172.21.15.115:0/958736027 reap_dead start 2023-12-01T03:08:25.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.995+0000 7f2254ffb640 1 -- 172.21.15.115:0/958736027 shutdown_connections 2023-12-01T03:08:25.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.995+0000 7f2254ffb640 1 -- 172.21.15.115:0/958736027 >> 172.21.15.115:0/958736027 conn(0x7f2250065960 msgr2=0x7f2250065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:25.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.995+0000 7f2254ffb640 1 -- 172.21.15.115:0/958736027 shutdown_connections 2023-12-01T03:08:25.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.995+0000 7f2254ffb640 1 -- 172.21.15.115:0/958736027 wait complete. 2023-12-01T03:08:26.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.997+0000 7f2254ffb640 1 Processor -- start 2023-12-01T03:08:26.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.997+0000 7f2254ffb640 1 -- start start 2023-12-01T03:08:26.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.998+0000 7f2254ffb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22380039a0 0x7f22501f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.998+0000 7f2254ffb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22501591a0 0x7f22501472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.998+0000 7f2254ffb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f225015be90 0x7f225015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.998+0000 7f2254ffb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225015c880 con 0x7f22501591a0 2023-12-01T03:08:26.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.998+0000 7f2254ffb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22501f7c60 con 0x7f22380039a0 2023-12-01T03:08:26.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.998+0000 7f2254ffb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22501f7f40 con 0x7f225015be90 2023-12-01T03:08:26.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.998+0000 7f224dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22501591a0 0x7f22501472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.998+0000 7f224dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22501591a0 0x7f22501472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35578/0 (socket says 172.21.15.115:35578) 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.998+0000 7f224dd9b640 1 -- 172.21.15.115:0/675394591 learned_addr learned my addr 172.21.15.115:0/675394591 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.998+0000 7f224e59c640 1 --2- 172.21.15.115:0/675394591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22380039a0 0x7f22501f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f224ed9d640 1 --2- 172.21.15.115:0/675394591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f225015be90 0x7f225015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f224dd9b640 1 -- 172.21.15.115:0/675394591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f225015be90 msgr2=0x7f225015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f224dd9b640 1 --2- 172.21.15.115:0/675394591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f225015be90 0x7f225015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f224dd9b640 1 -- 172.21.15.115:0/675394591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22380039a0 msgr2=0x7f22501f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f224dd9b640 1 --2- 172.21.15.115:0/675394591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22380039a0 0x7f22501f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f224dd9b640 1 -- 172.21.15.115:0/675394591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22501f8220 con 0x7f22501591a0 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f224ed9d640 1 --2- 172.21.15.115:0/675394591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f225015be90 0x7f225015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f224dd9b640 1 --2- 172.21.15.115:0/675394591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22501591a0 0x7f22501472e0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f224400d570 tx=0x7f224400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f223effd640 1 -- 172.21.15.115:0/675394591 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244005480 con 0x7f22501591a0 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f223effd640 1 -- 172.21.15.115:0/675394591 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2244003610 con 0x7f22501591a0 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f223effd640 1 -- 172.21.15.115:0/675394591 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244003980 con 0x7f22501591a0 2023-12-01T03:08:26.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f2254ffb640 1 -- 172.21.15.115:0/675394591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22501f8500 con 0x7f22501591a0 2023-12-01T03:08:26.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:25.999+0000 7f2254ffb640 1 -- 172.21.15.115:0/675394591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2250159620 con 0x7f22501591a0 2023-12-01T03:08:26.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.000+0000 7f2254ffb640 1 -- 172.21.15.115:0/675394591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f225004e890 con 0x7f22501591a0 2023-12-01T03:08:26.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.001+0000 7f223effd640 1 -- 172.21.15.115:0/675394591 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 111) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f2244003ae0 con 0x7f22501591a0 2023-12-01T03:08:26.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.002+0000 7f223effd640 1 -- 172.21.15.115:0/675394591 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f22440887e0 con 0x7f22501591a0 2023-12-01T03:08:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.006+0000 7f223effd640 1 -- 172.21.15.115:0/675394591 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2244053cf0 con 0x7f22501591a0 2023-12-01T03:08:26.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.183+0000 7f2254ffb640 1 -- 172.21.15.115:0/675394591 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22500cd230 con 0x7f22501591a0 2023-12-01T03:08:26.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.184+0000 7f223effd640 1 -- 172.21.15.115:0/675394591 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v184) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2244001230 con 0x7f22501591a0 2023-12-01T03:08:26.186 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:08:26.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.186+0000 7f2254ffb640 1 -- 172.21.15.115:0/675394591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22501591a0 msgr2=0x7f22501472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.186+0000 7f2254ffb640 1 --2- 172.21.15.115:0/675394591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22501591a0 0x7f22501472e0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f224400d570 tx=0x7f224400da30 comp rx=0 tx=0).stop 2023-12-01T03:08:26.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.186+0000 7f2254ffb640 1 -- 172.21.15.115:0/675394591 shutdown_connections 2023-12-01T03:08:26.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.186+0000 7f2254ffb640 1 --2- 172.21.15.115:0/675394591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f225015be90 0x7f225015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.186+0000 7f2254ffb640 1 --2- 172.21.15.115:0/675394591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22501591a0 0x7f22501472e0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.187+0000 7f2254ffb640 1 --2- 172.21.15.115:0/675394591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22380039a0 0x7f22501f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.187+0000 7f2254ffb640 1 -- 172.21.15.115:0/675394591 >> 172.21.15.115:0/675394591 conn(0x7f2250065960 msgr2=0x7f2250070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:26.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.187+0000 7f2254ffb640 1 -- 172.21.15.115:0/675394591 shutdown_connections 2023-12-01T03:08:26.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.187+0000 7f2254ffb640 1 -- 172.21.15.115:0/675394591 wait complete. 2023-12-01T03:08:26.205 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:08:26.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.467+0000 7f2bd993e640 1 Processor -- start 2023-12-01T03:08:26.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.467+0000 7f2bd993e640 1 -- start start 2023-12-01T03:08:26.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.468+0000 7f2bd993e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd4142870 0x7f2bd40ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.468+0000 7f2bd993e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2bd40cd010 0x7f2bd40b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.469+0000 7f2bd993e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd40cd470 con 0x7f2bd40b1c50 2023-12-01T03:08:26.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.469+0000 7f2bd993e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd40b3ae0 con 0x7f2bd40cd010 2023-12-01T03:08:26.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.469+0000 7f2bd993e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd4142c50 con 0x7f2bd4142870 2023-12-01T03:08:26.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.469+0000 7f2bd37fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2bd40b1c50 0x7f2bd415d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57120/0 (socket says 172.21.15.115:57120) 2023-12-01T03:08:26.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.469+0000 7f2bd37fe640 1 -- 172.21.15.115:0/19716068 learned_addr learned my addr 172.21.15.115:0/19716068 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:26.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.469+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107813129 0 0) 0x7f2bd40cd470 con 0x7f2bd40b1c50 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.469+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bbc003520 con 0x7f2bd40b1c50 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.469+0000 7f2bd2ffd640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd4142870 0x7f2bd40ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.469+0000 7f2bd27fc640 1 --2- 172.21.15.115:0/19716068 >> v2:172.21.15.142:3300/0 conn(0x7f2bd40cd010 0x7f2bd40b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1235933049 0 0) 0x7f2bbc003520 con 0x7f2bd40b1c50 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd4142870 msgr2=0x7f2bd40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd4142870 0x7f2bd40ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 >> v2:172.21.15.142:3300/0 conn(0x7f2bd40cd010 msgr2=0x7f2bd40b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 --2- 172.21.15.115:0/19716068 >> v2:172.21.15.142:3300/0 conn(0x7f2bd40cd010 0x7f2bd40b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd40b20e0 con 0x7f2bd40b1c50 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2bcc0019f0 con 0x7f2bd40b1c50 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 >> v1:172.21.15.115:6789/0 conn(0x7f2bd40b1c50 legacy=0x7f2bd415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bbc0039a0 0x7f2bbc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 0x7f2bbc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd4142870 0x7f2bbc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40b3ae0 con 0x7f2bbc0047c0 2023-12-01T03:08:26.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd4142c50 con 0x7f2bbc0039a0 2023-12-01T03:08:26.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc00bab0 con 0x7f2bd4142870 2023-12-01T03:08:26.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd27fc640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 0x7f2bbc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.470+0000 7f2bd2ffd640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bbc0039a0 0x7f2bbc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.471+0000 7f2bd27fc640 1 -- 172.21.15.115:0/19716068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd4142870 msgr2=0x7f2bbc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:26.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.471+0000 7f2bd27fc640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd4142870 0x7f2bbc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.471+0000 7f2bd27fc640 1 -- 172.21.15.115:0/19716068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bbc0039a0 msgr2=0x7f2bbc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:26.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.471+0000 7f2bd27fc640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bbc0039a0 0x7f2bbc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.471+0000 7f2bd27fc640 1 -- 172.21.15.115:0/19716068 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc003520 con 0x7f2bbc0047c0 2023-12-01T03:08:26.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.472+0000 7f2bd27fc640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 0x7f2bbc004c40 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800ba00 tx=0x7f2bbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd84dcad058d1ba server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:26.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.472+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8032630 con 0x7f2bbc0047c0 2023-12-01T03:08:26.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.472+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc8032790 con 0x7f2bbc0047c0 2023-12-01T03:08:26.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.472+0000 7f2bd1ffb640 1 -- 172.21.15.115:0/19716068 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8032b20 con 0x7f2bbc0047c0 2023-12-01T03:08:26.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.473+0000 7f2bd993e640 1 -- 172.21.15.115:0/19716068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 msgr2=0x7f2bbc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.473+0000 7f2bd993e640 1 --2- 172.21.15.115:0/19716068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 0x7f2bbc004c40 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800ba00 tx=0x7f2bbc0057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.473+0000 7f2bd37fe640 1 -- 172.21.15.115:0/19716068 reap_dead start 2023-12-01T03:08:26.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.473+0000 7f2bd993e640 1 -- 172.21.15.115:0/19716068 shutdown_connections 2023-12-01T03:08:26.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.473+0000 7f2bd993e640 1 -- 172.21.15.115:0/19716068 >> 172.21.15.115:0/19716068 conn(0x7f2bd4065960 msgr2=0x7f2bd4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:26.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.473+0000 7f2bd993e640 1 -- 172.21.15.115:0/19716068 shutdown_connections 2023-12-01T03:08:26.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.473+0000 7f2bd993e640 1 -- 172.21.15.115:0/19716068 wait complete. 2023-12-01T03:08:26.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.475+0000 7f2bd993e640 1 Processor -- start 2023-12-01T03:08:26.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.475+0000 7f2bd993e640 1 -- start start 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd993e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 0x7f2bd4157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd993e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd4158040 0x7f2bd41e3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd993e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd40a92e0 0x7f2bd41e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd993e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd413c1e0 con 0x7f2bbc0047c0 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd993e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd413c430 con 0x7f2bd4158040 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd993e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd413c710 con 0x7f2bd40a92e0 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 0x7f2bd4157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd2ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 0x7f2bd4157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35600/0 (socket says 172.21.15.115:35600) 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd2ffd640 1 -- 172.21.15.115:0/242904644 learned_addr learned my addr 172.21.15.115:0/242904644 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd2ffd640 1 -- 172.21.15.115:0/242904644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd40a92e0 msgr2=0x7f2bd41e36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd27fc640 1 --2- 172.21.15.115:0/242904644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd4158040 0x7f2bd41e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd2ffd640 1 --2- 172.21.15.115:0/242904644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd40a92e0 0x7f2bd41e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd2ffd640 1 -- 172.21.15.115:0/242904644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd4158040 msgr2=0x7f2bd41e3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd2ffd640 1 --2- 172.21.15.115:0/242904644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd4158040 0x7f2bd41e3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.476+0000 7f2bd2ffd640 1 -- 172.21.15.115:0/242904644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd413c9f0 con 0x7f2bbc0047c0 2023-12-01T03:08:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.477+0000 7f2bd2ffd640 1 --2- 172.21.15.115:0/242904644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 0x7f2bd4157890 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400f580 tx=0x7f2bc400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.477+0000 7f2bbbfff640 1 -- 172.21.15.115:0/242904644 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4014870 con 0x7f2bbc0047c0 2023-12-01T03:08:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.477+0000 7f2bbbfff640 1 -- 172.21.15.115:0/242904644 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc40149d0 con 0x7f2bbc0047c0 2023-12-01T03:08:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.477+0000 7f2bbbfff640 1 -- 172.21.15.115:0/242904644 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4014d80 con 0x7f2bbc0047c0 2023-12-01T03:08:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.477+0000 7f2bd993e640 1 -- 172.21.15.115:0/242904644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd413ccd0 con 0x7f2bbc0047c0 2023-12-01T03:08:26.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.477+0000 7f2bd993e640 1 -- 172.21.15.115:0/242904644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd4158800 con 0x7f2bbc0047c0 2023-12-01T03:08:26.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.478+0000 7f2bd993e640 1 -- 172.21.15.115:0/242904644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd404e890 con 0x7f2bbc0047c0 2023-12-01T03:08:26.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.479+0000 7f2bbbfff640 1 -- 172.21.15.115:0/242904644 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 111) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f2bc4015410 con 0x7f2bbc0047c0 2023-12-01T03:08:26.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.480+0000 7f2bbbfff640 1 -- 172.21.15.115:0/242904644 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f2bc40891b0 con 0x7f2bbc0047c0 2023-12-01T03:08:26.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.484+0000 7f2bbbfff640 1 -- 172.21.15.115:0/242904644 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2bc4054730 con 0x7f2bbc0047c0 2023-12-01T03:08:26.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.671+0000 7f2bd993e640 1 -- 172.21.15.115:0/242904644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bd406b6d0 con 0x7f2bbc0047c0 2023-12-01T03:08:26.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.672+0000 7f2bbbfff640 1 -- 172.21.15.115:0/242904644 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v184) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bc4057d50 con 0x7f2bbc0047c0 2023-12-01T03:08:26.674 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:08:26.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.675+0000 7f2bd993e640 1 -- 172.21.15.115:0/242904644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 msgr2=0x7f2bd4157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.675+0000 7f2bd993e640 1 --2- 172.21.15.115:0/242904644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 0x7f2bd4157890 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400f580 tx=0x7f2bc400fa40 comp rx=0 tx=0).stop 2023-12-01T03:08:26.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.675+0000 7f2bd993e640 1 -- 172.21.15.115:0/242904644 shutdown_connections 2023-12-01T03:08:26.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.675+0000 7f2bd993e640 1 --2- 172.21.15.115:0/242904644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2bd40a92e0 0x7f2bd41e36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.675+0000 7f2bd993e640 1 --2- 172.21.15.115:0/242904644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd4158040 0x7f2bd41e3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.675+0000 7f2bd993e640 1 --2- 172.21.15.115:0/242904644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bbc0047c0 0x7f2bd4157890 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.675+0000 7f2bd993e640 1 -- 172.21.15.115:0/242904644 >> 172.21.15.115:0/242904644 conn(0x7f2bd4065960 msgr2=0x7f2bd406bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:26.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.675+0000 7f2bd993e640 1 -- 172.21.15.115:0/242904644 shutdown_connections 2023-12-01T03:08:26.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.675+0000 7f2bd993e640 1 -- 172.21.15.115:0/242904644 wait complete. 2023-12-01T03:08:26.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:08:26.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.909+0000 7f27bc44a640 1 Processor -- start 2023-12-01T03:08:26.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.909+0000 7f27bc44a640 1 -- start start 2023-12-01T03:08:26.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.910+0000 7f27bc44a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f27b415e2f0 0x7f27b40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.910+0000 7f27bc44a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f27b406f2f0 0x7f27b40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.910+0000 7f27bc44a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40adf10 0x7f27b40c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.910+0000 7f27bc44a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b40add10 con 0x7f27b406f2f0 2023-12-01T03:08:26.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.910+0000 7f27bc44a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b40c5760 con 0x7f27b415e2f0 2023-12-01T03:08:26.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.910+0000 7f27bc44a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b415e6d0 con 0x7f27b40adf10 2023-12-01T03:08:26.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27b99e5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f27b406f2f0 0x7f27b40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27b99e5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f27b406f2f0 0x7f27b40caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35610/0 (socket says 172.21.15.115:35610) 2023-12-01T03:08:26.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27b99e5640 1 -- 172.21.15.115:0/72021439 learned_addr learned my addr 172.21.15.115:0/72021439 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27ba9e7640 1 --2- 172.21.15.115:0/72021439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40adf10 0x7f27b40c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27b99e5640 1 -- 172.21.15.115:0/72021439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40adf10 msgr2=0x7f27b40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27ba1e6640 1 --2- 172.21.15.115:0/72021439 >> v2:172.21.15.142:3300/0 conn(0x7f27b415e2f0 0x7f27b40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27b99e5640 1 --2- 172.21.15.115:0/72021439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40adf10 0x7f27b40c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27b99e5640 1 -- 172.21.15.115:0/72021439 >> v2:172.21.15.142:3300/0 conn(0x7f27b415e2f0 msgr2=0x7f27b40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27b99e5640 1 --2- 172.21.15.115:0/72021439 >> v2:172.21.15.142:3300/0 conn(0x7f27b415e2f0 0x7f27b40ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27b99e5640 1 -- 172.21.15.115:0/72021439 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b40cf7b0 con 0x7f27b406f2f0 2023-12-01T03:08:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27ba1e6640 1 --2- 172.21.15.115:0/72021439 >> v2:172.21.15.142:3300/0 conn(0x7f27b415e2f0 0x7f27b40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.911+0000 7f27ba9e7640 1 --2- 172.21.15.115:0/72021439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40adf10 0x7f27b40c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:26.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.912+0000 7f27b99e5640 1 --2- 172.21.15.115:0/72021439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b406f2f0 0x7f27b40caba0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f27b406ef60 tx=0x7f27a402b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7f16a74355fd88 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27b91e4640 1 -- 172.21.15.115:0/72021439 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a4031760 con 0x7f27b406f2f0 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27b91e4640 1 -- 172.21.15.115:0/72021439 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a40318c0 con 0x7f27b406f2f0 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27b91e4640 1 -- 172.21.15.115:0/72021439 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a4031c50 con 0x7f27b406f2f0 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27bc44a640 1 -- 172.21.15.115:0/72021439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b406f2f0 msgr2=0x7f27b40caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27bc44a640 1 --2- 172.21.15.115:0/72021439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b406f2f0 0x7f27b40caba0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f27b406ef60 tx=0x7f27a402b800 comp rx=0 tx=0).stop 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27bc44a640 1 -- 172.21.15.115:0/72021439 shutdown_connections 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27bc44a640 1 --2- 172.21.15.115:0/72021439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b40adf10 0x7f27b40c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27bc44a640 1 --2- 172.21.15.115:0/72021439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b406f2f0 0x7f27b40caba0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27bc44a640 1 --2- 172.21.15.115:0/72021439 >> v2:172.21.15.142:3300/0 conn(0x7f27b415e2f0 0x7f27b40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27bc44a640 1 -- 172.21.15.115:0/72021439 >> 172.21.15.115:0/72021439 conn(0x7f27b40ce880 msgr2=0x7f27b40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27bc44a640 1 -- 172.21.15.115:0/72021439 shutdown_connections 2023-12-01T03:08:26.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.913+0000 7f27bc44a640 1 -- 172.21.15.115:0/72021439 wait complete. 2023-12-01T03:08:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.915+0000 7f27bc44a640 1 Processor -- start 2023-12-01T03:08:26.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.915+0000 7f27bc44a640 1 -- start start 2023-12-01T03:08:26.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.916+0000 7f27bc44a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b406f2f0 0x7f27b40c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.916+0000 7f27bc44a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27b40adf10 0x7f27b41f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.916+0000 7f27bc44a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b415e2f0 0x7f27b40c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.916+0000 7f27bc44a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40c28f0 con 0x7f27b415e2f0 2023-12-01T03:08:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.916+0000 7f27bc44a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40c5230 con 0x7f27b40adf10 2023-12-01T03:08:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.916+0000 7f27bc44a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b415aca0 con 0x7f27b406f2f0 2023-12-01T03:08:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba9e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b415e2f0 0x7f27b40c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba9e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b415e2f0 0x7f27b40c2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35622/0 (socket says 172.21.15.115:35622) 2023-12-01T03:08:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba9e7640 1 -- 172.21.15.115:0/2237829191 learned_addr learned my addr 172.21.15.115:0/2237829191 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:26.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba1e6640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b406f2f0 0x7f27b40c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba9e7640 1 -- 172.21.15.115:0/2237829191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b406f2f0 msgr2=0x7f27b40c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba9e7640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b406f2f0 0x7f27b40c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27b99e5640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27b40adf10 0x7f27b41f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba9e7640 1 -- 172.21.15.115:0/2237829191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27b40adf10 msgr2=0x7f27b41f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba9e7640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27b40adf10 0x7f27b41f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba9e7640 1 -- 172.21.15.115:0/2237829191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a40094d0 con 0x7f27b415e2f0 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba1e6640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b406f2f0 0x7f27b40c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.917+0000 7f27ba9e7640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b415e2f0 0x7f27b40c2c30 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00cd10 tx=0x7f27ac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.918+0000 7f27aaffd640 1 -- 172.21.15.115:0/2237829191 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00f680 con 0x7f27b415e2f0 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.918+0000 7f27aaffd640 1 -- 172.21.15.115:0/2237829191 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac00f800 con 0x7f27b415e2f0 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.918+0000 7f27aaffd640 1 -- 172.21.15.115:0/2237829191 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00fbb0 con 0x7f27b415e2f0 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.918+0000 7f27bc44a640 1 -- 172.21.15.115:0/2237829191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b415af80 con 0x7f27b415e2f0 2023-12-01T03:08:26.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.918+0000 7f27bc44a640 1 -- 172.21.15.115:0/2237829191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b40b18a0 con 0x7f27b415e2f0 2023-12-01T03:08:26.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.919+0000 7f27bc44a640 1 -- 172.21.15.115:0/2237829191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b406fda0 con 0x7f27b415e2f0 2023-12-01T03:08:26.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.920+0000 7f27aaffd640 1 -- 172.21.15.115:0/2237829191 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 112) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f27ac00b040 con 0x7f27b415e2f0 2023-12-01T03:08:26.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.921+0000 7f27aaffd640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f279806a280 0x7f279806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:26.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.921+0000 7f27aaffd640 1 -- 172.21.15.115:0/2237829191 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f27ac0893e0 con 0x7f27b415e2f0 2023-12-01T03:08:26.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.921+0000 7f27ba1e6640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f279806a280 0x7f279806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:26.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.922+0000 7f27ba1e6640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f279806a280 0x7f279806c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27b0006fd0 tx=0x7f27b0008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:26.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:26.926+0000 7f27aaffd640 1 -- 172.21.15.115:0/2237829191 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f27ac054800 con 0x7f27b415e2f0 2023-12-01T03:08:27.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.152+0000 7f27bc44a640 1 -- 172.21.15.115:0/2237829191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27b404e890 con 0x7f27b415e2f0 2023-12-01T03:08:27.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.153+0000 7f27aaffd640 1 -- 172.21.15.115:0/2237829191 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f27ac01d090 con 0x7f27b415e2f0 2023-12-01T03:08:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:27.156 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1616304732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1616304732},{"type":"v1","addr":"172.21.15.115:6835","nonce":1616304732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6766,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3109897397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3109897397},{"type":"v1","addr":"172.21.15.142:6834","nonce":3109897397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/446557816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":446557816},{"type":"v1","addr":"172.21.15.142:6835","nonce":446557816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:52:21.959726+0000","modified":"2023-12-01T02:52:24.991376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9549":{"gid":9549,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1064849689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1064849689},{"type":"v1","addr":"172.21.15.115:6837","nonce":1064849689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-01T03:08:27.156 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 79 2023-12-01T03:08:27.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.156+0000 7f27bc44a640 1 -- 172.21.15.115:0/2237829191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f279806a280 msgr2=0x7f279806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:27.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.156+0000 7f27bc44a640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f279806a280 0x7f279806c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27b0006fd0 tx=0x7f27b0008040 comp rx=0 tx=0).stop 2023-12-01T03:08:27.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.156+0000 7f27bc44a640 1 -- 172.21.15.115:0/2237829191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b415e2f0 msgr2=0x7f27b40c2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:27.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.156+0000 7f27bc44a640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b415e2f0 0x7f27b40c2c30 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00cd10 tx=0x7f27ac00a650 comp rx=0 tx=0).stop 2023-12-01T03:08:27.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.156+0000 7f27bc44a640 1 -- 172.21.15.115:0/2237829191 shutdown_connections 2023-12-01T03:08:27.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.156+0000 7f27bc44a640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f279806a280 0x7f279806c740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.156+0000 7f27bc44a640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27b415e2f0 0x7f27b40c2c30 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.156+0000 7f27bc44a640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27b40adf10 0x7f27b41f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.156+0000 7f27bc44a640 1 --2- 172.21.15.115:0/2237829191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27b406f2f0 0x7f27b40c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.156+0000 7f27bc44a640 1 -- 172.21.15.115:0/2237829191 >> 172.21.15.115:0/2237829191 conn(0x7f27b40ce880 msgr2=0x7f27b415d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:27.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.157+0000 7f27bc44a640 1 -- 172.21.15.115:0/2237829191 shutdown_connections 2023-12-01T03:08:27.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.157+0000 7f27bc44a640 1 -- 172.21.15.115:0/2237829191 wait complete. 2023-12-01T03:08:27.177 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:08:27.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.388+0000 7f8e33842640 1 Processor -- start 2023-12-01T03:08:27.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.388+0000 7f8e33842640 1 -- start start 2023-12-01T03:08:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.390+0000 7f8e33842640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8e2c0cad70 0x7f8e2c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.390+0000 7f8e33842640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8e2c06f2f0 0x7f8e2c06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.390+0000 7f8e33842640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2c0add10 con 0x7f8e2c15e2f0 2023-12-01T03:08:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.390+0000 7f8e33842640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2c0c5760 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.390+0000 7f8e33842640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2c15e6d0 con 0x7f8e2c0cad70 2023-12-01T03:08:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.390+0000 7f8e31ddf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8e2c06f2f0 0x7f8e2c06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.390+0000 7f8e31ddf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8e2c06f2f0 0x7f8e2c06bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35630/0 (socket says 172.21.15.115:35630) 2023-12-01T03:08:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.390+0000 7f8e31ddf640 1 -- 172.21.15.115:0/1321107942 learned_addr learned my addr 172.21.15.115:0/1321107942 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:27.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.390+0000 7f8e30ddd640 1 --2- 172.21.15.115:0/1321107942 >> v2:172.21.15.142:3300/0 conn(0x7f8e2c0cad70 0x7f8e2c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.390+0000 7f8e31ddf640 1 -- 172.21.15.115:0/1321107942 >> v2:172.21.15.142:3300/0 conn(0x7f8e2c0cad70 msgr2=0x7f8e2c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.391+0000 7f8e31ddf640 1 --2- 172.21.15.115:0/1321107942 >> v2:172.21.15.142:3300/0 conn(0x7f8e2c0cad70 0x7f8e2c06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.391+0000 7f8e31ddf640 1 -- 172.21.15.115:0/1321107942 >> v1:172.21.15.142:6789/0 conn(0x7f8e2c15e2f0 legacy=0x7f8e2c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.391+0000 7f8e31ddf640 1 -- 172.21.15.115:0/1321107942 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c0cf7b0 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.392+0000 7f8e31ddf640 1 --2- 172.21.15.115:0/1321107942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 0x7f8e2c06bb00 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c06de80 tx=0x7f8e2402bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd63f0b303128fe9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.392+0000 7f8e23fff640 1 -- 172.21.15.115:0/1321107942 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2402c940 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.392+0000 7f8e23fff640 1 -- 172.21.15.115:0/1321107942 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2402caa0 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.392+0000 7f8e23fff640 1 -- 172.21.15.115:0/1321107942 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2402ce30 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.393+0000 7f8e33842640 1 -- 172.21.15.115:0/1321107942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 msgr2=0x7f8e2c06bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.393+0000 7f8e33842640 1 --2- 172.21.15.115:0/1321107942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 0x7f8e2c06bb00 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c06de80 tx=0x7f8e2402bda0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.393+0000 7f8e33842640 1 -- 172.21.15.115:0/1321107942 shutdown_connections 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.393+0000 7f8e33842640 1 --2- 172.21.15.115:0/1321107942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 0x7f8e2c06bb00 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.393+0000 7f8e33842640 1 --2- 172.21.15.115:0/1321107942 >> v2:172.21.15.142:3300/0 conn(0x7f8e2c0cad70 0x7f8e2c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.393+0000 7f8e33842640 1 -- 172.21.15.115:0/1321107942 >> 172.21.15.115:0/1321107942 conn(0x7f8e2c0ce880 msgr2=0x7f8e2c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:27.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.393+0000 7f8e33842640 1 -- 172.21.15.115:0/1321107942 shutdown_connections 2023-12-01T03:08:27.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.393+0000 7f8e33842640 1 -- 172.21.15.115:0/1321107942 wait complete. 2023-12-01T03:08:27.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.395+0000 7f8e33842640 1 Processor -- start 2023-12-01T03:08:27.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.395+0000 7f8e33842640 1 -- start start 2023-12-01T03:08:27.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e33842640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 0x7f8e2c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e33842640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e2c0cad70 0x7f8e2c0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e33842640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c15e2f0 0x7f8e2c0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e33842640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c14fa90 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e33842640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c0b2920 con 0x7f8e2c0cad70 2023-12-01T03:08:27.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e33842640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c0c4d70 con 0x7f8e2c15e2f0 2023-12-01T03:08:27.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e315de640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 0x7f8e2c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e315de640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 0x7f8e2c06bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35640/0 (socket says 172.21.15.115:35640) 2023-12-01T03:08:27.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e315de640 1 -- 172.21.15.115:0/1841829564 learned_addr learned my addr 172.21.15.115:0/1841829564 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:27.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e30ddd640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e2c0cad70 0x7f8e2c0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e315de640 1 -- 172.21.15.115:0/1841829564 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c15e2f0 msgr2=0x7f8e2c0ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:27.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e31ddf640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c15e2f0 0x7f8e2c0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e315de640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c15e2f0 0x7f8e2c0ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e315de640 1 -- 172.21.15.115:0/1841829564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e2c0cad70 msgr2=0x7f8e2c0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.396+0000 7f8e315de640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e2c0cad70 0x7f8e2c0adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.397+0000 7f8e315de640 1 -- 172.21.15.115:0/1841829564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e24009b30 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.397+0000 7f8e30ddd640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e2c0cad70 0x7f8e2c0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.397+0000 7f8e315de640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 0x7f8e2c06bdf0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800b3e0 tx=0x7f8e2800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.397+0000 7f8e21ffb640 1 -- 172.21.15.115:0/1841829564 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28011830 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.397+0000 7f8e21ffb640 1 -- 172.21.15.115:0/1841829564 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e280119b0 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.397+0000 7f8e21ffb640 1 -- 172.21.15.115:0/1841829564 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28011d60 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.397+0000 7f8e33842640 1 -- 172.21.15.115:0/1841829564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2c0c5050 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.397+0000 7f8e33842640 1 -- 172.21.15.115:0/1841829564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2c14ff80 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.398+0000 7f8e33842640 1 -- 172.21.15.115:0/1841829564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2c06fda0 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.399+0000 7f8e21ffb640 1 -- 172.21.15.115:0/1841829564 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 112) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f8e2801a400 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.400+0000 7f8e21ffb640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8e1006a280 0x7f8e1006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.400+0000 7f8e21ffb640 1 -- 172.21.15.115:0/1841829564 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f8e28088c00 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.401+0000 7f8e30ddd640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8e1006a280 0x7f8e1006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.401+0000 7f8e30ddd640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8e1006a280 0x7f8e1006c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c15b770 tx=0x7f8e1c008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:27.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.405+0000 7f8e21ffb640 1 -- 172.21.15.115:0/1841829564 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8e28054020 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.593+0000 7f8e33842640 1 -- 172.21.15.115:0/1841829564 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e2c073910 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.595+0000 7f8e21ffb640 1 -- 172.21.15.115:0/1841829564 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+17771 (secure 0 0 0) 0x7f8e28015020 con 0x7f8e2c06f2f0 2023-12-01T03:08:27.598 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:27.599 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":184,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:08:25.825688+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:52:20.047003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-01T02:52:20.974031+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":183,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2f","mappings":[{"from":2,"to":5}]},{"pgid":"21.3b","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2876702189":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/2712149733":"2023-12-02T03:08:25.825656+0000","172.21.15.115:6824/138932":"2023-12-02T03:08:25.825656+0000","172.21.15.115:6822/138932":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/3057742638":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/1510611733":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/2942324483":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/363147954":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/1462551426":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/718367345":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/2970056045":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/858082591":"2023-12-02T03:03:21.455245+0000","172.21.15.115:6824/88317":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/3219939439":"2023-12-02T03:03:21.455245+0000","172.21.15.115:6822/88317":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/673199316":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/3225301105":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/2404622555":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/1684885368":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/4185429564":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/682039355":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/2564902155":"2023-12-02T02:52:41.228363+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:08:27.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 -- 172.21.15.115:0/1841829564 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8e1006a280 msgr2=0x7f8e1006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:27.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8e1006a280 0x7f8e1006c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c15b770 tx=0x7f8e1c008040 comp rx=0 tx=0).stop 2023-12-01T03:08:27.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 -- 172.21.15.115:0/1841829564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 msgr2=0x7f8e2c06bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:27.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 0x7f8e2c06bdf0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800b3e0 tx=0x7f8e2800b8a0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 -- 172.21.15.115:0/1841829564 shutdown_connections 2023-12-01T03:08:27.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8e1006a280 0x7f8e1006c740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8e2c15e2f0 0x7f8e2c0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e2c0cad70 0x7f8e2c0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 --2- 172.21.15.115:0/1841829564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e2c06f2f0 0x7f8e2c06bdf0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 -- 172.21.15.115:0/1841829564 >> 172.21.15.115:0/1841829564 conn(0x7f8e2c0ce880 msgr2=0x7f8e2c0c1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:27.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.598+0000 7f8e33842640 1 -- 172.21.15.115:0/1841829564 shutdown_connections 2023-12-01T03:08:27.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.599+0000 7f8e33842640 1 -- 172.21.15.115:0/1841829564 wait complete. 2023-12-01T03:08:27.617 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T03:08:27.617 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:08:27.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.821+0000 7f4edbf5c640 1 Processor -- start 2023-12-01T03:08:27.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.821+0000 7f4edbf5c640 1 -- start start 2023-12-01T03:08:27.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.822+0000 7f4edbf5c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 0x7f4ed406fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.822+0000 7f4edbf5c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4ed40ad2b0 0x7f4ed406fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.822+0000 7f4edbf5c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed415d600 con 0x7f4ed415d220 2023-12-01T03:08:27.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.822+0000 7f4edbf5c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed406cc10 con 0x7f4ed40ad2b0 2023-12-01T03:08:27.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.822+0000 7f4edbf5c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed406c720 con 0x7f4ed40ccc80 2023-12-01T03:08:27.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.823+0000 7f4eda4f9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4ed40ad2b0 0x7f4ed406fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.823+0000 7f4eda4f9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4ed40ad2b0 0x7f4ed406fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49476/0 (socket says 172.21.15.115:49476) 2023-12-01T03:08:27.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.823+0000 7f4eda4f9640 1 -- 172.21.15.115:0/3637509484 learned_addr learned my addr 172.21.15.115:0/3637509484 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:27.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.823+0000 7f4ed94f7640 1 --2- 172.21.15.115:0/3637509484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 0x7f4ed406fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.823+0000 7f4eda4f9640 1 -- 172.21.15.115:0/3637509484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 msgr2=0x7f4ed406fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:27.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.823+0000 7f4eda4f9640 1 --2- 172.21.15.115:0/3637509484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 0x7f4ed406fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.823+0000 7f4eda4f9640 1 -- 172.21.15.115:0/3637509484 >> v1:172.21.15.142:6789/0 conn(0x7f4ed415d220 legacy=0x7f4ed40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.823+0000 7f4eda4f9640 1 -- 172.21.15.115:0/3637509484 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed406c4b0 con 0x7f4ed40ad2b0 2023-12-01T03:08:27.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.825+0000 7f4eda4f9640 1 --2- 172.21.15.115:0/3637509484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ad2b0 0x7f4ed406fff0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009e50 tx=0x7f4ecc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e48f9d0d8eff332 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:27.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.825+0000 7f4ed8cf6640 1 -- 172.21.15.115:0/3637509484 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc02c9e0 con 0x7f4ed40ad2b0 2023-12-01T03:08:27.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.825+0000 7f4ed8cf6640 1 -- 172.21.15.115:0/3637509484 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc02cb40 con 0x7f4ed40ad2b0 2023-12-01T03:08:27.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.825+0000 7f4ed8cf6640 1 -- 172.21.15.115:0/3637509484 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0343f0 con 0x7f4ed40ad2b0 2023-12-01T03:08:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.826+0000 7f4edbf5c640 1 -- 172.21.15.115:0/3637509484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ad2b0 msgr2=0x7f4ed406fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.826+0000 7f4edbf5c640 1 --2- 172.21.15.115:0/3637509484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ad2b0 0x7f4ed406fff0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009e50 tx=0x7f4ecc02be40 comp rx=0 tx=0).stop 2023-12-01T03:08:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.826+0000 7f4edbf5c640 1 -- 172.21.15.115:0/3637509484 shutdown_connections 2023-12-01T03:08:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.826+0000 7f4edbf5c640 1 --2- 172.21.15.115:0/3637509484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ad2b0 0x7f4ed406fff0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.826+0000 7f4edbf5c640 1 --2- 172.21.15.115:0/3637509484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 0x7f4ed406fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.826+0000 7f4edbf5c640 1 -- 172.21.15.115:0/3637509484 >> 172.21.15.115:0/3637509484 conn(0x7f4ed40ce880 msgr2=0x7f4ed40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.826+0000 7f4edbf5c640 1 -- 172.21.15.115:0/3637509484 shutdown_connections 2023-12-01T03:08:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.826+0000 7f4edbf5c640 1 -- 172.21.15.115:0/3637509484 wait complete. 2023-12-01T03:08:27.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.828+0000 7f4edbf5c640 1 Processor -- start 2023-12-01T03:08:27.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.828+0000 7f4edbf5c640 1 -- start start 2023-12-01T03:08:27.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4edbf5c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40ad2b0 0x7f4ed41581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4edbf5c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 0x7f4ed40c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4edbf5c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415d220 0x7f4ed40caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4edbf5c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40c46c0 con 0x7f4ed415d220 2023-12-01T03:08:27.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4edbf5c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed4158710 con 0x7f4ed40ad2b0 2023-12-01T03:08:27.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4edbf5c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40c4ae0 con 0x7f4ed40ccc80 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4eda4f9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415d220 0x7f4ed40caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4eda4f9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415d220 0x7f4ed40caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49480/0 (socket says 172.21.15.115:49480) 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4eda4f9640 1 -- 172.21.15.115:0/2990379225 learned_addr learned my addr 172.21.15.115:0/2990379225 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4ed9cf8640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40ad2b0 0x7f4ed41581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4ed94f7640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 0x7f4ed40c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4eda4f9640 1 -- 172.21.15.115:0/2990379225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 msgr2=0x7f4ed40c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4eda4f9640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 0x7f4ed40c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4eda4f9640 1 -- 172.21.15.115:0/2990379225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40ad2b0 msgr2=0x7f4ed41581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4eda4f9640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40ad2b0 0x7f4ed41581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4eda4f9640 1 -- 172.21.15.115:0/2990379225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc009b30 con 0x7f4ed415d220 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.829+0000 7f4ed94f7640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 0x7f4ed40c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:27.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.830+0000 7f4eda4f9640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415d220 0x7f4ed40caec0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc036040 tx=0x7f4ecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:27.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.830+0000 7f4eca7fc640 1 -- 172.21.15.115:0/2990379225 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc009510 con 0x7f4ed415d220 2023-12-01T03:08:27.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.830+0000 7f4eca7fc640 1 -- 172.21.15.115:0/2990379225 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc03e450 con 0x7f4ed415d220 2023-12-01T03:08:27.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.830+0000 7f4eca7fc640 1 -- 172.21.15.115:0/2990379225 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc03e7e0 con 0x7f4ed415d220 2023-12-01T03:08:27.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.830+0000 7f4edbf5c640 1 -- 172.21.15.115:0/2990379225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed40c3760 con 0x7f4ed415d220 2023-12-01T03:08:27.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.830+0000 7f4edbf5c640 1 -- 172.21.15.115:0/2990379225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed40c3d20 con 0x7f4ed415d220 2023-12-01T03:08:27.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.831+0000 7f4edbf5c640 1 -- 172.21.15.115:0/2990379225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed40c4800 con 0x7f4ed415d220 2023-12-01T03:08:27.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.832+0000 7f4eca7fc640 1 -- 172.21.15.115:0/2990379225 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 112) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f4ecc03e940 con 0x7f4ed415d220 2023-12-01T03:08:27.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.833+0000 7f4eca7fc640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a280 0x7f4eb806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:27.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.833+0000 7f4eca7fc640 1 -- 172.21.15.115:0/2990379225 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f4ecc0ab930 con 0x7f4ed415d220 2023-12-01T03:08:27.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.833+0000 7f4ed9cf8640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a280 0x7f4eb806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:27.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.834+0000 7f4ed9cf8640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a280 0x7f4eb806c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006fd0 tx=0x7f4ed0008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:27.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:27.837+0000 7f4eca7fc640 1 -- 172.21.15.115:0/2990379225 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4ed40c4800 con 0x7f4ed415d220 2023-12-01T03:08:28.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.064+0000 7f4edbf5c640 1 -- 172.21.15.115:0/2990379225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ed415d600 con 0x7f4ed415d220 2023-12-01T03:08:28.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.065+0000 7f4eca7fc640 1 -- 172.21.15.115:0/2990379225 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f4ed415d600 con 0x7f4ed415d220 2023-12-01T03:08:28.067 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:28.067 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1616304732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1616304732},{"type":"v1","addr":"172.21.15.115:6835","nonce":1616304732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6766,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3109897397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3109897397},{"type":"v1","addr":"172.21.15.142:6834","nonce":3109897397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/446557816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":446557816},{"type":"v1","addr":"172.21.15.142:6835","nonce":446557816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:52:21.959726+0000","modified":"2023-12-01T02:52:24.991376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9549":{"gid":9549,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1064849689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1064849689},{"type":"v1","addr":"172.21.15.115:6837","nonce":1064849689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-01T03:08:28.067 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 79 2023-12-01T03:08:28.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 -- 172.21.15.115:0/2990379225 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a280 msgr2=0x7f4eb806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a280 0x7f4eb806c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006fd0 tx=0x7f4ed0008040 comp rx=0 tx=0).stop 2023-12-01T03:08:28.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 -- 172.21.15.115:0/2990379225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415d220 msgr2=0x7f4ed40caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415d220 0x7f4ed40caec0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc036040 tx=0x7f4ecc004a40 comp rx=0 tx=0).stop 2023-12-01T03:08:28.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 -- 172.21.15.115:0/2990379225 shutdown_connections 2023-12-01T03:08:28.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a280 0x7f4eb806c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40ccc80 0x7f4ed40c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415d220 0x7f4ed40caec0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 --2- 172.21.15.115:0/2990379225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40ad2b0 0x7f4ed41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 -- 172.21.15.115:0/2990379225 >> 172.21.15.115:0/2990379225 conn(0x7f4ed40ce880 msgr2=0x7f4ed4159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:28.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 -- 172.21.15.115:0/2990379225 shutdown_connections 2023-12-01T03:08:28.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.067+0000 7f4edbf5c640 1 -- 172.21.15.115:0/2990379225 wait complete. 2023-12-01T03:08:28.087 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:08:28.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.285+0000 7fd298db9640 1 Processor -- start 2023-12-01T03:08:28.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.285+0000 7fd298db9640 1 -- start start 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.286+0000 7fd298db9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd2940cad70 0x7fd29406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.286+0000 7fd298db9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2940add10 con 0x7fd29406f2f0 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.287+0000 7fd298db9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2940c5760 con 0x7fd29415e2f0 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.287+0000 7fd298db9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29415e6d0 con 0x7fd2940cad70 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.287+0000 7fd291d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd2940cad70 0x7fd29406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.287+0000 7fd292d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd29406f2f0 0x7fd29406bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51304/0 (socket says 172.21.15.115:51304) 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.287+0000 7fd291d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd2940cad70 0x7fd29406b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49510/0 (socket says 172.21.15.115:49510) 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.287+0000 7fd292d9d640 1 -- 172.21.15.115:0/243083603 learned_addr learned my addr 172.21.15.115:0/243083603 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.287+0000 7fd291d9b640 1 -- 172.21.15.115:0/243083603 >> v1:172.21.15.142:6789/0 conn(0x7fd29415e2f0 legacy=0x7fd2940ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.287+0000 7fd291d9b640 1 -- 172.21.15.115:0/243083603 >> v1:172.21.15.115:6789/0 conn(0x7fd29406f2f0 legacy=0x7fd29406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.287+0000 7fd291d9b640 1 -- 172.21.15.115:0/243083603 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940cf7b0 con 0x7fd2940cad70 2023-12-01T03:08:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.288+0000 7fd291d9b640 1 --2- 172.21.15.115:0/243083603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2940cad70 0x7fd29406b5c0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fd2880097d0 tx=0x7fd28802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffafa100c84d7db server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.289+0000 7fd29159a640 1 -- 172.21.15.115:0/243083603 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28802f050 con 0x7fd2940cad70 2023-12-01T03:08:28.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.289+0000 7fd29159a640 1 -- 172.21.15.115:0/243083603 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd288031700 con 0x7fd2940cad70 2023-12-01T03:08:28.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.289+0000 7fd29159a640 1 -- 172.21.15.115:0/243083603 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288031a90 con 0x7fd2940cad70 2023-12-01T03:08:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.289+0000 7fd298db9640 1 -- 172.21.15.115:0/243083603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2940cad70 msgr2=0x7fd29406b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.289+0000 7fd298db9640 1 --2- 172.21.15.115:0/243083603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2940cad70 0x7fd29406b5c0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fd2880097d0 tx=0x7fd28802b7a0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.289+0000 7fd298db9640 1 -- 172.21.15.115:0/243083603 shutdown_connections 2023-12-01T03:08:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.289+0000 7fd298db9640 1 --2- 172.21.15.115:0/243083603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2940cad70 0x7fd29406b5c0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.289+0000 7fd298db9640 1 -- 172.21.15.115:0/243083603 >> 172.21.15.115:0/243083603 conn(0x7fd2940ce880 msgr2=0x7fd2940cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.289+0000 7fd298db9640 1 -- 172.21.15.115:0/243083603 shutdown_connections 2023-12-01T03:08:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.290+0000 7fd298db9640 1 -- 172.21.15.115:0/243083603 wait complete. 2023-12-01T03:08:28.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.291+0000 7fd298db9640 1 Processor -- start 2023-12-01T03:08:28.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.291+0000 7fd298db9640 1 -- start start 2023-12-01T03:08:28.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.292+0000 7fd298db9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29406f2f0 0x7fd294157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.292+0000 7fd298db9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940cad70 0x7fd294159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.292+0000 7fd298db9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29415e2f0 0x7fd2941f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.292+0000 7fd298db9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294159ae0 con 0x7fd29406f2f0 2023-12-01T03:08:28.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.292+0000 7fd298db9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940c3550 con 0x7fd2940cad70 2023-12-01T03:08:28.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.292+0000 7fd298db9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940af380 con 0x7fd29415e2f0 2023-12-01T03:08:28.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.292+0000 7fd29259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29406f2f0 0x7fd294157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:28.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.292+0000 7fd29259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29406f2f0 0x7fd294157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49524/0 (socket says 172.21.15.115:49524) 2023-12-01T03:08:28.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.292+0000 7fd29259c640 1 -- 172.21.15.115:0/3503646657 learned_addr learned my addr 172.21.15.115:0/3503646657 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:28.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd29259c640 1 -- 172.21.15.115:0/3503646657 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29415e2f0 msgr2=0x7fd2941f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd29259c640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29415e2f0 0x7fd2941f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd29259c640 1 -- 172.21.15.115:0/3503646657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940cad70 msgr2=0x7fd294159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd29259c640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940cad70 0x7fd294159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd29259c640 1 -- 172.21.15.115:0/3503646657 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd288009400 con 0x7fd29406f2f0 2023-12-01T03:08:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd29259c640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29406f2f0 0x7fd294157e90 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fd28400c460 tx=0x7fd28400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd282ffd640 1 -- 172.21.15.115:0/3503646657 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284012830 con 0x7fd29406f2f0 2023-12-01T03:08:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd282ffd640 1 -- 172.21.15.115:0/3503646657 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2840129d0 con 0x7fd29406f2f0 2023-12-01T03:08:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd282ffd640 1 -- 172.21.15.115:0/3503646657 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284012d80 con 0x7fd29406f2f0 2023-12-01T03:08:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd298db9640 1 -- 172.21.15.115:0/3503646657 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2941583d0 con 0x7fd29406f2f0 2023-12-01T03:08:28.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.293+0000 7fd298db9640 1 -- 172.21.15.115:0/3503646657 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2940c3270 con 0x7fd29406f2f0 2023-12-01T03:08:28.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.294+0000 7fd298db9640 1 -- 172.21.15.115:0/3503646657 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29406fda0 con 0x7fd29406f2f0 2023-12-01T03:08:28.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.295+0000 7fd282ffd640 1 -- 172.21.15.115:0/3503646657 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 112) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fd28401a3b0 con 0x7fd29406f2f0 2023-12-01T03:08:28.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.296+0000 7fd282ffd640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27406a280 0x7fd27406c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.296+0000 7fd282ffd640 1 -- 172.21.15.115:0/3503646657 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fd284088960 con 0x7fd29406f2f0 2023-12-01T03:08:28.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.296+0000 7fd291d9b640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27406a280 0x7fd27406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:28.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.297+0000 7fd291d9b640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27406a280 0x7fd27406c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd288035000 tx=0x7fd288030000 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:28.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.300+0000 7fd282ffd640 1 -- 172.21.15.115:0/3503646657 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd284053d80 con 0x7fd29406f2f0 2023-12-01T03:08:28.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.446+0000 7fd282ffd640 1 -- 172.21.15.115:0/3503646657 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 113) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fd284054150 con 0x7fd29406f2f0 2023-12-01T03:08:28.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.529+0000 7fd298db9640 1 -- 172.21.15.115:0/3503646657 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd29404e890 con 0x7fd29406f2f0 2023-12-01T03:08:28.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.530+0000 7fd282ffd640 1 -- 172.21.15.115:0/3503646657 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4661 (secure 0 0 0) 0x7fd284070390 con 0x7fd29406f2f0 2023-12-01T03:08:28.532 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:28.532 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1616304732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1616304732},{"type":"v1","addr":"172.21.15.115:6835","nonce":1616304732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6766,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3109897397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3109897397},{"type":"v1","addr":"172.21.15.142:6834","nonce":3109897397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/446557816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":446557816},{"type":"v1","addr":"172.21.15.142:6835","nonce":446557816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T02:52:21.959726+0000","modified":"2023-12-01T02:52:24.991376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9549":{"gid":9549,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6837/1064849689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":1064849689},{"type":"v1","addr":"172.21.15.115:6837","nonce":1064849689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-01T03:08:28.533 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 79 2023-12-01T03:08:28.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 -- 172.21.15.115:0/3503646657 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27406a280 msgr2=0x7fd27406c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27406a280 0x7fd27406c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd288035000 tx=0x7fd288030000 comp rx=0 tx=0).stop 2023-12-01T03:08:28.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 -- 172.21.15.115:0/3503646657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29406f2f0 msgr2=0x7fd294157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29406f2f0 0x7fd294157e90 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fd28400c460 tx=0x7fd28400c920 comp rx=0 tx=0).stop 2023-12-01T03:08:28.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 -- 172.21.15.115:0/3503646657 shutdown_connections 2023-12-01T03:08:28.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27406a280 0x7fd27406c740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29415e2f0 0x7fd2941f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2940cad70 0x7fd294159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 --2- 172.21.15.115:0/3503646657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29406f2f0 0x7fd294157e90 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 -- 172.21.15.115:0/3503646657 >> 172.21.15.115:0/3503646657 conn(0x7fd2940ce880 msgr2=0x7fd2940cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:28.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 -- 172.21.15.115:0/3503646657 shutdown_connections 2023-12-01T03:08:28.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.533+0000 7fd298db9640 1 -- 172.21.15.115:0/3503646657 wait complete. 2023-12-01T03:08:28.553 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:08:28.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.757+0000 7fa28e891640 1 Processor -- start 2023-12-01T03:08:28.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.758+0000 7fa28e891640 1 -- start start 2023-12-01T03:08:28.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.759+0000 7fa28e891640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa2881584b0 0x7fa2880cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.759+0000 7fa28e891640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa28806b980 0x7fa2880cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.759+0000 7fa28e891640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2880cc180 con 0x7fa28815afe0 2023-12-01T03:08:28.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.759+0000 7fa28e891640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28815b550 con 0x7fa28806b980 2023-12-01T03:08:28.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.759+0000 7fa28e891640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28815b830 con 0x7fa2881584b0 2023-12-01T03:08:28.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.759+0000 7fa28e090640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa28806b980 0x7fa2880cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:28.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.759+0000 7fa28d08e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa28815afe0 0x7fa28806b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51308/0 (socket says 172.21.15.115:51308) 2023-12-01T03:08:28.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.760+0000 7fa28e090640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa28806b980 0x7fa2880cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49538/0 (socket says 172.21.15.115:49538) 2023-12-01T03:08:28.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.760+0000 7fa28d08e640 1 -- 172.21.15.115:0/4136738690 learned_addr learned my addr 172.21.15.115:0/4136738690 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:28.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.760+0000 7fa28d88f640 1 --2- 172.21.15.115:0/4136738690 >> v2:172.21.15.142:3300/0 conn(0x7fa2881584b0 0x7fa2880cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:28.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.760+0000 7fa28e090640 1 -- 172.21.15.115:0/4136738690 >> v2:172.21.15.142:3300/0 conn(0x7fa2881584b0 msgr2=0x7fa2880cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:28.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.760+0000 7fa28e090640 1 --2- 172.21.15.115:0/4136738690 >> v2:172.21.15.142:3300/0 conn(0x7fa2881584b0 0x7fa2880cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.760+0000 7fa28e090640 1 -- 172.21.15.115:0/4136738690 >> v1:172.21.15.115:6789/0 conn(0x7fa28815afe0 legacy=0x7fa28806b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.760+0000 7fa28e090640 1 -- 172.21.15.115:0/4136738690 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28815bb10 con 0x7fa28806b980 2023-12-01T03:08:28.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.760+0000 7fa28c88d640 1 -- 172.21.15.115:0/4136738690 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176192059 0 0) 0x7fa2880cc180 con 0x7fa28815afe0 2023-12-01T03:08:28.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.760+0000 7fa28d88f640 1 --2- 172.21.15.115:0/4136738690 >> v2:172.21.15.142:3300/0 conn(0x7fa2881584b0 0x7fa2880cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:28.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.761+0000 7fa28e090640 1 --2- 172.21.15.115:0/4136738690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa28806b980 0x7fa2880cc680 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fa28806bfb0 tx=0x7fa28002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bec9aa1258a533 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:28.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.761+0000 7fa28c88d640 1 -- 172.21.15.115:0/4136738690 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28002cc70 con 0x7fa28806b980 2023-12-01T03:08:28.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.761+0000 7fa28c88d640 1 -- 172.21.15.115:0/4136738690 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28002cdd0 con 0x7fa28806b980 2023-12-01T03:08:28.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.761+0000 7fa28c88d640 1 -- 172.21.15.115:0/4136738690 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280034660 con 0x7fa28806b980 2023-12-01T03:08:28.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.762+0000 7fa28e891640 1 -- 172.21.15.115:0/4136738690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa28806b980 msgr2=0x7fa2880cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.762+0000 7fa28e891640 1 --2- 172.21.15.115:0/4136738690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa28806b980 0x7fa2880cc680 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fa28806bfb0 tx=0x7fa28002bec0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.762+0000 7fa28e891640 1 -- 172.21.15.115:0/4136738690 shutdown_connections 2023-12-01T03:08:28.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.762+0000 7fa28e891640 1 --2- 172.21.15.115:0/4136738690 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa28806b980 0x7fa2880cc680 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.762+0000 7fa28e891640 1 --2- 172.21.15.115:0/4136738690 >> v2:172.21.15.142:3300/0 conn(0x7fa2881584b0 0x7fa2880cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.762+0000 7fa28e891640 1 -- 172.21.15.115:0/4136738690 >> 172.21.15.115:0/4136738690 conn(0x7fa2880ce640 msgr2=0x7fa2880cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:28.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.762+0000 7fa28e891640 1 -- 172.21.15.115:0/4136738690 shutdown_connections 2023-12-01T03:08:28.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.762+0000 7fa28e891640 1 -- 172.21.15.115:0/4136738690 wait complete. 2023-12-01T03:08:28.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.764+0000 7fa28e891640 1 Processor -- start 2023-12-01T03:08:28.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.764+0000 7fa28e891640 1 -- start start 2023-12-01T03:08:28.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.765+0000 7fa28e891640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28806b980 0x7fa2881563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.765+0000 7fa28e891640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2881584b0 0x7fa2881f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.765+0000 7fa28e891640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa28815afe0 0x7fa2880ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.765+0000 7fa28e891640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2881f2870 con 0x7fa2881584b0 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.765+0000 7fa28e891640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2881f5f20 con 0x7fa28815afe0 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.765+0000 7fa28e891640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880cac20 con 0x7fa28806b980 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.765+0000 7fa28d08e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2881584b0 0x7fa2881f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.765+0000 7fa28d08e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2881584b0 0x7fa2881f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49552/0 (socket says 172.21.15.115:49552) 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.765+0000 7fa28d08e640 1 -- 172.21.15.115:0/803436016 learned_addr learned my addr 172.21.15.115:0/803436016 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa28e090640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa28815afe0 0x7fa2880ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa28d88f640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28806b980 0x7fa2881563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa28d08e640 1 -- 172.21.15.115:0/803436016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28806b980 msgr2=0x7fa2881563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa28d08e640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28806b980 0x7fa2881563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa28d08e640 1 -- 172.21.15.115:0/803436016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa28815afe0 msgr2=0x7fa2880ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa28d08e640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa28815afe0 0x7fa2880ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa28d08e640 1 -- 172.21.15.115:0/803436016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa280009a70 con 0x7fa2881584b0 2023-12-01T03:08:28.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa28d08e640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2881584b0 0x7fa2881f59e0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fa27800c4d0 tx=0x7fa27800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:28.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa27e7fc640 1 -- 172.21.15.115:0/803436016 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2780074e0 con 0x7fa2881584b0 2023-12-01T03:08:28.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa27e7fc640 1 -- 172.21.15.115:0/803436016 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa278007660 con 0x7fa2881584b0 2023-12-01T03:08:28.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa28e891640 1 -- 172.21.15.115:0/803436016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2881f1bf0 con 0x7fa2881584b0 2023-12-01T03:08:28.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa27e7fc640 1 -- 172.21.15.115:0/803436016 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278004a20 con 0x7fa2881584b0 2023-12-01T03:08:28.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.766+0000 7fa28e891640 1 -- 172.21.15.115:0/803436016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2881f21b0 con 0x7fa2881584b0 2023-12-01T03:08:28.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.767+0000 7fa28e891640 1 -- 172.21.15.115:0/803436016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2881f29b0 con 0x7fa2881584b0 2023-12-01T03:08:28.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.768+0000 7fa27e7fc640 1 -- 172.21.15.115:0/803436016 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 113) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fa278004b80 con 0x7fa2881584b0 2023-12-01T03:08:28.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.769+0000 7fa27e7fc640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa26c06a4f0 0x7fa26c06c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:28.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.769+0000 7fa27e7fc640 1 -- 172.21.15.115:0/803436016 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fa278088410 con 0x7fa2881584b0 2023-12-01T03:08:28.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.770+0000 7fa28d88f640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa26c06a4f0 0x7fa26c06c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:28.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.770+0000 7fa28d88f640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa26c06a4f0 0x7fa26c06c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa284008850 tx=0x7fa284008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:28.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.773+0000 7fa27e7fc640 1 -- 172.21.15.115:0/803436016 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa2881f29b0 con 0x7fa2881584b0 2023-12-01T03:08:28.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.964+0000 7fa28e891640 1 -- 172.21.15.115:0/803436016 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2881f29b0 con 0x7fa2881584b0 2023-12-01T03:08:28.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.966+0000 7fa27e7fc640 1 -- 172.21.15.115:0/803436016 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+17771 (secure 0 0 0) 0x7fa2881f29b0 con 0x7fa2881584b0 2023-12-01T03:08:28.969 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:28.969 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":184,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:08:25.825688+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-01T02:52:20.047003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-01T02:52:20.974031+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":183,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2f","mappings":[{"from":2,"to":5}]},{"pgid":"21.3b","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2876702189":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/2712149733":"2023-12-02T03:08:25.825656+0000","172.21.15.115:6824/138932":"2023-12-02T03:08:25.825656+0000","172.21.15.115:6822/138932":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/3057742638":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/1510611733":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/2942324483":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/363147954":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/1462551426":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/718367345":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/2970056045":"2023-12-02T03:03:21.455245+0000","172.21.15.115:0/858082591":"2023-12-02T03:03:21.455245+0000","172.21.15.115:6824/88317":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/3219939439":"2023-12-02T03:03:21.455245+0000","172.21.15.115:6822/88317":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/673199316":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/3225301105":"2023-12-02T03:08:25.825656+0000","172.21.15.115:0/2404622555":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/1684885368":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/4185429564":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/682039355":"2023-12-02T02:52:41.228363+0000","172.21.15.115:0/2564902155":"2023-12-02T02:52:41.228363+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:08:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 -- 172.21.15.115:0/803436016 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa26c06a4f0 msgr2=0x7fa26c06c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa26c06a4f0 0x7fa26c06c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa284008850 tx=0x7fa284008040 comp rx=0 tx=0).stop 2023-12-01T03:08:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 -- 172.21.15.115:0/803436016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2881584b0 msgr2=0x7fa2881f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2881584b0 0x7fa2881f59e0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fa27800c4d0 tx=0x7fa27800c990 comp rx=0 tx=0).stop 2023-12-01T03:08:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 -- 172.21.15.115:0/803436016 shutdown_connections 2023-12-01T03:08:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa26c06a4f0 0x7fa26c06c9b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa28815afe0 0x7fa2880ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2881584b0 0x7fa2881f59e0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 --2- 172.21.15.115:0/803436016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28806b980 0x7fa2881563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:28.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 -- 172.21.15.115:0/803436016 >> 172.21.15.115:0/803436016 conn(0x7fa2880ce640 msgr2=0x7fa2880ccf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:28.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 -- 172.21.15.115:0/803436016 shutdown_connections 2023-12-01T03:08:28.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:28.969+0000 7fa28e891640 1 -- 172.21.15.115:0/803436016 wait complete. 2023-12-01T03:08:28.990 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T03:08:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.206+0000 7fb10ca70640 1 Processor -- start 2023-12-01T03:08:29.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.207+0000 7fb10ca70640 1 -- start start 2023-12-01T03:08:29.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.208+0000 7fb10ca70640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb10806f2f0 0x7fb10806bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:29.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.208+0000 7fb10ca70640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1080add10 con 0x7fb1080cad70 2023-12-01T03:08:29.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.208+0000 7fb10ca70640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1080c5760 con 0x7fb10815e2f0 2023-12-01T03:08:29.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.208+0000 7fb10ca70640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10815e6d0 con 0x7fb10806f2f0 2023-12-01T03:08:29.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.209+0000 7fb106d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb10806f2f0 0x7fb10806bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:29.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.209+0000 7fb106d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb10806f2f0 0x7fb10806bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49562/0 (socket says 172.21.15.115:49562) 2023-12-01T03:08:29.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.209+0000 7fb106d9d640 1 -- 172.21.15.115:0/529827969 learned_addr learned my addr 172.21.15.115:0/529827969 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:29.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.209+0000 7fb0fffff640 1 --1- 172.21.15.115:0/529827969 >> v1:172.21.15.115:6789/0 conn(0x7fb1080cad70 0x7fb10806b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51312/0 (socket says 172.21.15.115:51312) 2023-12-01T03:08:29.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.209+0000 7fb106d9d640 1 -- 172.21.15.115:0/529827969 >> v1:172.21.15.142:6789/0 conn(0x7fb10815e2f0 legacy=0x7fb1080ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:29.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.209+0000 7fb106d9d640 1 -- 172.21.15.115:0/529827969 >> v1:172.21.15.115:6789/0 conn(0x7fb1080cad70 legacy=0x7fb10806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:29.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.209+0000 7fb106d9d640 1 -- 172.21.15.115:0/529827969 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080cf7b0 con 0x7fb10806f2f0 2023-12-01T03:08:29.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.209+0000 7fb105d9b640 1 -- 172.21.15.115:0/529827969 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985222477 0 0) 0x7fb1080add10 con 0x7fb1080cad70 2023-12-01T03:08:29.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.210+0000 7fb106d9d640 1 --2- 172.21.15.115:0/529827969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10806f2f0 0x7fb10806bcd0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fb100009e80 tx=0x7fb10002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31bf99e36d55a4b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:29.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.210+0000 7fb105d9b640 1 -- 172.21.15.115:0/529827969 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10002c9e0 con 0x7fb10806f2f0 2023-12-01T03:08:29.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.210+0000 7fb105d9b640 1 -- 172.21.15.115:0/529827969 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10002cb40 con 0x7fb10806f2f0 2023-12-01T03:08:29.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.210+0000 7fb105d9b640 1 -- 172.21.15.115:0/529827969 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000343f0 con 0x7fb10806f2f0 2023-12-01T03:08:29.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.211+0000 7fb10ca70640 1 -- 172.21.15.115:0/529827969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10806f2f0 msgr2=0x7fb10806bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:29.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.211+0000 7fb10ca70640 1 --2- 172.21.15.115:0/529827969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10806f2f0 0x7fb10806bcd0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fb100009e80 tx=0x7fb10002be40 comp rx=0 tx=0).stop 2023-12-01T03:08:29.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.211+0000 7fb10ca70640 1 -- 172.21.15.115:0/529827969 shutdown_connections 2023-12-01T03:08:29.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.211+0000 7fb10ca70640 1 --2- 172.21.15.115:0/529827969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10806f2f0 0x7fb10806bcd0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:29.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.211+0000 7fb10ca70640 1 -- 172.21.15.115:0/529827969 >> 172.21.15.115:0/529827969 conn(0x7fb1080ce880 msgr2=0x7fb1080cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:29.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.211+0000 7fb10ca70640 1 -- 172.21.15.115:0/529827969 shutdown_connections 2023-12-01T03:08:29.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.211+0000 7fb10ca70640 1 -- 172.21.15.115:0/529827969 wait complete. 2023-12-01T03:08:29.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.213+0000 7fb10ca70640 1 Processor -- start 2023-12-01T03:08:29.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.213+0000 7fb10ca70640 1 -- start start 2023-12-01T03:08:29.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.214+0000 7fb10ca70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10806f2f0 0x7fb1080ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:29.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.214+0000 7fb10ca70640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1080cad70 0x7fb108156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:29.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.214+0000 7fb10ca70640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10815e2f0 0x7fb10814ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:29.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.214+0000 7fb10ca70640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1081589d0 con 0x7fb10815e2f0 2023-12-01T03:08:29.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.214+0000 7fb10ca70640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb108158c20 con 0x7fb1080cad70 2023-12-01T03:08:29.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.214+0000 7fb10ca70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb108158f00 con 0x7fb10806f2f0 2023-12-01T03:08:29.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.214+0000 7fb106d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10815e2f0 0x7fb10814ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:29.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.214+0000 7fb106d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10815e2f0 0x7fb10814ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49564/0 (socket says 172.21.15.115:49564) 2023-12-01T03:08:29.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.214+0000 7fb106d9d640 1 -- 172.21.15.115:0/3630435236 learned_addr learned my addr 172.21.15.115:0/3630435236 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb0fffff640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1080cad70 0x7fb108156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb10659c640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10806f2f0 0x7fb1080ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb106d9d640 1 -- 172.21.15.115:0/3630435236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10806f2f0 msgr2=0x7fb1080ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb106d9d640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10806f2f0 0x7fb1080ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb106d9d640 1 -- 172.21.15.115:0/3630435236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1080cad70 msgr2=0x7fb108156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb106d9d640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1080cad70 0x7fb108156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb106d9d640 1 -- 172.21.15.115:0/3630435236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb100009b30 con 0x7fb10815e2f0 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb0fffff640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1080cad70 0x7fb108156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb10659c640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10806f2f0 0x7fb1080ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb106d9d640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10815e2f0 0x7fb10814ff30 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fb100036040 tx=0x7fb100004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.215+0000 7fb0feffd640 1 -- 172.21.15.115:0/3630435236 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10003e450 con 0x7fb10815e2f0 2023-12-01T03:08:29.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.216+0000 7fb0feffd640 1 -- 172.21.15.115:0/3630435236 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10003e5b0 con 0x7fb10815e2f0 2023-12-01T03:08:29.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.216+0000 7fb10ca70640 1 -- 172.21.15.115:0/3630435236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1081591e0 con 0x7fb10815e2f0 2023-12-01T03:08:29.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.216+0000 7fb0feffd640 1 -- 172.21.15.115:0/3630435236 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10003e740 con 0x7fb10815e2f0 2023-12-01T03:08:29.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.216+0000 7fb10ca70640 1 -- 172.21.15.115:0/3630435236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1081f2950 con 0x7fb10815e2f0 2023-12-01T03:08:29.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.217+0000 7fb10ca70640 1 -- 172.21.15.115:0/3630435236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10806fda0 con 0x7fb10815e2f0 2023-12-01T03:08:29.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.218+0000 7fb0feffd640 1 -- 172.21.15.115:0/3630435236 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 113) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fb10003e8a0 con 0x7fb10815e2f0 2023-12-01T03:08:29.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.219+0000 7fb0feffd640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb0e806a4f0 0x7fb0e806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:29.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.219+0000 7fb0feffd640 1 -- 172.21.15.115:0/3630435236 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fb1000ab7b0 con 0x7fb10815e2f0 2023-12-01T03:08:29.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.219+0000 7fb10659c640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb0e806a4f0 0x7fb0e806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:29.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.220+0000 7fb10659c640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb0e806a4f0 0x7fb0e806c9b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb1080c4390 tx=0x7fb0f8008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:29.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.224+0000 7fb0feffd640 1 -- 172.21.15.115:0/3630435236 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb100076a00 con 0x7fb10815e2f0 2023-12-01T03:08:29.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.454+0000 7fb10ca70640 1 -- 172.21.15.115:0/3630435236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb10804e890 con 0x7fb10815e2f0 2023-12-01T03:08:29.461 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T03:08:29.459+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T03:08:29.473 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:08:29.482 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:08:29.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.477+0000 7fb0feffd640 1 -- 172.21.15.115:0/3630435236 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb10007a020 con 0x7fb10815e2f0 2023-12-01T03:08:29.483 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T03:08:29.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.480+0000 7fb10ca70640 1 -- 172.21.15.115:0/3630435236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb0e806a4f0 msgr2=0x7fb0e806c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:29.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.480+0000 7fb10ca70640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb0e806a4f0 0x7fb0e806c9b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb1080c4390 tx=0x7fb0f8008040 comp rx=0 tx=0).stop 2023-12-01T03:08:29.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.480+0000 7fb10ca70640 1 -- 172.21.15.115:0/3630435236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10815e2f0 msgr2=0x7fb10814ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:29.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.480+0000 7fb10ca70640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10815e2f0 0x7fb10814ff30 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fb100036040 tx=0x7fb100004a40 comp rx=0 tx=0).stop 2023-12-01T03:08:29.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.480+0000 7fb10ca70640 1 -- 172.21.15.115:0/3630435236 shutdown_connections 2023-12-01T03:08:29.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.480+0000 7fb10ca70640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb0e806a4f0 0x7fb0e806c9b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:29.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.480+0000 7fb10ca70640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb10815e2f0 0x7fb10814ff30 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:29.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.480+0000 7fb10ca70640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1080cad70 0x7fb108156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:29.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.480+0000 7fb10ca70640 1 --2- 172.21.15.115:0/3630435236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb10806f2f0 0x7fb1080ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:29.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.480+0000 7fb10ca70640 1 -- 172.21.15.115:0/3630435236 >> 172.21.15.115:0/3630435236 conn(0x7fb1080ce880 msgr2=0x7fb1080cbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:29.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.481+0000 7fb10ca70640 1 -- 172.21.15.115:0/3630435236 shutdown_connections 2023-12-01T03:08:29.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.481+0000 7fb10ca70640 1 -- 172.21.15.115:0/3630435236 wait complete. 2023-12-01T03:08:29.504 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T03:08:29.566 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T03:08:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.733+0000 7f4ed999c640 1 Processor -- start 2023-12-01T03:08:29.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.734+0000 7f4ed999c640 1 -- start start 2023-12-01T03:08:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.735+0000 7f4ed999c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4ed415cfa0 0x7f4ed40b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.735+0000 7f4ed999c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed40b6890 con 0x7f4ed40b63e0 2023-12-01T03:08:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.735+0000 7f4ed999c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed40b6b70 con 0x7f4ed40b7e70 2023-12-01T03:08:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.735+0000 7f4ed999c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed40b6e50 con 0x7f4ed415cfa0 2023-12-01T03:08:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.736+0000 7f4ed2ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4ed415cfa0 0x7f4ed40b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.736+0000 7f4ed37fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4ed40b63e0 0x7f4ed40cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51318/0 (socket says 172.21.15.115:51318) 2023-12-01T03:08:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.736+0000 7f4ed37fe640 1 -- 172.21.15.115:0/65961694 learned_addr learned my addr 172.21.15.115:0/65961694 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.736+0000 7f4ed2ffd640 1 -- 172.21.15.115:0/65961694 >> v1:172.21.15.142:6789/0 conn(0x7f4ed40b7e70 legacy=0x7f4ed40b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:29.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.736+0000 7f4ed2ffd640 1 -- 172.21.15.115:0/65961694 >> v1:172.21.15.115:6789/0 conn(0x7f4ed40b63e0 legacy=0x7f4ed40cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:29.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.736+0000 7f4ed2ffd640 1 -- 172.21.15.115:0/65961694 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40b7130 con 0x7f4ed415cfa0 2023-12-01T03:08:29.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.736+0000 7f4ed27fc640 1 -- 172.21.15.115:0/65961694 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204652655 0 0) 0x7f4ed40b6890 con 0x7f4ed40b63e0 2023-12-01T03:08:29.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.737+0000 7f4ed2ffd640 1 --2- 172.21.15.115:0/65961694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415cfa0 0x7f4ed40b7930 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40097a0 tx=0x7f4ec402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5381b995857881f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:29.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.738+0000 7f4ed27fc640 1 -- 172.21.15.115:0/65961694 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec402e070 con 0x7f4ed415cfa0 2023-12-01T03:08:29.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.738+0000 7f4ed27fc640 1 -- 172.21.15.115:0/65961694 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec40315a0 con 0x7f4ed415cfa0 2023-12-01T03:08:29.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.738+0000 7f4ed27fc640 1 -- 172.21.15.115:0/65961694 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4031930 con 0x7f4ed415cfa0 2023-12-01T03:08:29.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.738+0000 7f4ed999c640 1 -- 172.21.15.115:0/65961694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415cfa0 msgr2=0x7f4ed40b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:29.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.738+0000 7f4ed999c640 1 --2- 172.21.15.115:0/65961694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415cfa0 0x7f4ed40b7930 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40097a0 tx=0x7f4ec402b840 comp rx=0 tx=0).stop 2023-12-01T03:08:29.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.739+0000 7f4ed999c640 1 -- 172.21.15.115:0/65961694 shutdown_connections 2023-12-01T03:08:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.739+0000 7f4ed999c640 1 --2- 172.21.15.115:0/65961694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed415cfa0 0x7f4ed40b7930 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.739+0000 7f4ed999c640 1 -- 172.21.15.115:0/65961694 >> 172.21.15.115:0/65961694 conn(0x7f4ed406a0b0 msgr2=0x7f4ed406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.739+0000 7f4ed999c640 1 -- 172.21.15.115:0/65961694 shutdown_connections 2023-12-01T03:08:29.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.739+0000 7f4ed999c640 1 -- 172.21.15.115:0/65961694 wait complete. 2023-12-01T03:08:29.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.740+0000 7f4ed999c640 1 Processor -- start 2023-12-01T03:08:29.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.741+0000 7f4ed999c640 1 -- start start 2023-12-01T03:08:29.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.741+0000 7f4ed999c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40b63e0 0x7f4ed414e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:29.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.741+0000 7f4ed999c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40b7e70 0x7f4ed40cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.741+0000 7f4ed999c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed415cfa0 0x7f4ed415d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.741+0000 7f4ed999c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41f49b0 con 0x7f4ed40b7e70 2023-12-01T03:08:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.741+0000 7f4ed999c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41f3d30 con 0x7f4ed40b63e0 2023-12-01T03:08:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.741+0000 7f4ed999c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41f4010 con 0x7f4ed415cfa0 2023-12-01T03:08:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.742+0000 7f4eca7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40b7e70 0x7f4ed40cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.742+0000 7f4eca7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40b7e70 0x7f4ed40cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49590/0 (socket says 172.21.15.115:49590) 2023-12-01T03:08:29.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.742+0000 7f4eca7fc640 1 -- 172.21.15.115:0/1151458288 learned_addr learned my addr 172.21.15.115:0/1151458288 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.742+0000 7f4eca7fc640 1 -- 172.21.15.115:0/1151458288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed415cfa0 msgr2=0x7f4ed415d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.742+0000 7f4eca7fc640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed415cfa0 0x7f4ed415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.742+0000 7f4eca7fc640 1 -- 172.21.15.115:0/1151458288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40b63e0 msgr2=0x7f4ed414e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.742+0000 7f4eca7fc640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40b63e0 0x7f4ed414e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.742+0000 7f4eca7fc640 1 -- 172.21.15.115:0/1151458288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4009400 con 0x7f4ed40b7e70 2023-12-01T03:08:29.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.743+0000 7f4eca7fc640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40b7e70 0x7f4ed40cc420 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00c400 tx=0x7f4ebc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.743+0000 7f4ed0ff9640 1 -- 172.21.15.115:0/1151458288 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc011790 con 0x7f4ed40b7e70 2023-12-01T03:08:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.743+0000 7f4ed0ff9640 1 -- 172.21.15.115:0/1151458288 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc011930 con 0x7f4ed40b7e70 2023-12-01T03:08:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.743+0000 7f4ed0ff9640 1 -- 172.21.15.115:0/1151458288 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc011ce0 con 0x7f4ed40b7e70 2023-12-01T03:08:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.743+0000 7f4ed999c640 1 -- 172.21.15.115:0/1151458288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed40b8c00 con 0x7f4ed40b7e70 2023-12-01T03:08:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.743+0000 7f4ed999c640 1 -- 172.21.15.115:0/1151458288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed41f45d0 con 0x7f4ed40b7e70 2023-12-01T03:08:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.744+0000 7f4ed999c640 1 -- 172.21.15.115:0/1151458288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed404e890 con 0x7f4ed40b7e70 2023-12-01T03:08:29.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.745+0000 7f4ed0ff9640 1 -- 172.21.15.115:0/1151458288 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 113) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f4ebc01c460 con 0x7f4ed40b7e70 2023-12-01T03:08:29.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.746+0000 7f4ed0ff9640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb406a4c0 0x7f4eb406c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:29.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.747+0000 7f4ed0ff9640 1 -- 172.21.15.115:0/1151458288 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4ebc088dc0 con 0x7f4ed40b7e70 2023-12-01T03:08:29.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.747+0000 7f4ed2ffd640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb406a4c0 0x7f4eb406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:29.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.747+0000 7f4ed2ffd640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb406a4c0 0x7f4eb406c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4036000 tx=0x7f4ec4030000 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:29.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.751+0000 7f4ed0ff9640 1 -- 172.21.15.115:0/1151458288 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4ebc053ec0 con 0x7f4ed40b7e70 2023-12-01T03:08:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:29.978+0000 7f4ed999c640 1 -- 172.21.15.115:0/1151458288 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4ed40d3570 con 0x7f4ed40b7e70 2023-12-01T03:08:30.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.482+0000 7f4ed0ff9640 1 -- 172.21.15.115:0/1151458288 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4ebc0574e0 con 0x7f4ed40b7e70 2023-12-01T03:08:30.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 -- 172.21.15.115:0/1151458288 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb406a4c0 msgr2=0x7f4eb406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:30.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb406a4c0 0x7f4eb406c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4036000 tx=0x7f4ec4030000 comp rx=0 tx=0).stop 2023-12-01T03:08:30.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 -- 172.21.15.115:0/1151458288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40b7e70 msgr2=0x7f4ed40cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:30.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40b7e70 0x7f4ed40cc420 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00c400 tx=0x7f4ebc00c8c0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 -- 172.21.15.115:0/1151458288 shutdown_connections 2023-12-01T03:08:30.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb406a4c0 0x7f4eb406c980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed415cfa0 0x7f4ed415d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40b7e70 0x7f4ed40cc420 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 --2- 172.21.15.115:0/1151458288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40b63e0 0x7f4ed414e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 -- 172.21.15.115:0/1151458288 >> 172.21.15.115:0/1151458288 conn(0x7f4ed406a0b0 msgr2=0x7f4ed4161210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:30.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 -- 172.21.15.115:0/1151458288 shutdown_connections 2023-12-01T03:08:30.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.486+0000 7f4ed999c640 1 -- 172.21.15.115:0/1151458288 wait complete. 2023-12-01T03:08:30.514 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T03:08:30.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.718+0000 7fe05a821640 1 Processor -- start 2023-12-01T03:08:30.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.718+0000 7fe05a821640 1 -- start start 2023-12-01T03:08:30.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.719+0000 7fe05a821640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0540cb8c0 0x7fe0540cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:30.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.719+0000 7fe05a821640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe0540b4070 0x7fe0540cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:30.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.719+0000 7fe05a821640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05406c860 con 0x7fe05415ac70 2023-12-01T03:08:30.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.719+0000 7fe05a821640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05415b050 con 0x7fe0540b4070 2023-12-01T03:08:30.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.719+0000 7fe05a821640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540ce0b0 con 0x7fe0540cb8c0 2023-12-01T03:08:30.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.720+0000 7fe058dbe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe0540b4070 0x7fe0540cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:30.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.720+0000 7fe058dbe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe0540b4070 0x7fe0540cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49602/0 (socket says 172.21.15.115:49602) 2023-12-01T03:08:30.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.720+0000 7fe058dbe640 1 -- 172.21.15.115:0/2507684893 learned_addr learned my addr 172.21.15.115:0/2507684893 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:30.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.720+0000 7fe04bfff640 1 --2- 172.21.15.115:0/2507684893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0540cb8c0 0x7fe0540cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:30.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.720+0000 7fe058dbe640 1 -- 172.21.15.115:0/2507684893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0540cb8c0 msgr2=0x7fe0540cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:30.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.720+0000 7fe058dbe640 1 --2- 172.21.15.115:0/2507684893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0540cb8c0 0x7fe0540cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.720+0000 7fe058dbe640 1 -- 172.21.15.115:0/2507684893 >> v1:172.21.15.142:6789/0 conn(0x7fe05415ac70 legacy=0x7fe0540cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:30.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.720+0000 7fe058dbe640 1 -- 172.21.15.115:0/2507684893 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0540ce390 con 0x7fe0540b4070 2023-12-01T03:08:30.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.721+0000 7fe058dbe640 1 --2- 172.21.15.115:0/2507684893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540b4070 0x7fe0540cc8b0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009e50 tx=0x7fe04c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4194c45a163eff server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:30.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.722+0000 7fe0537fe640 1 -- 172.21.15.115:0/2507684893 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c02ca70 con 0x7fe0540b4070 2023-12-01T03:08:30.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.722+0000 7fe0537fe640 1 -- 172.21.15.115:0/2507684893 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c02cbd0 con 0x7fe0540b4070 2023-12-01T03:08:30.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.722+0000 7fe0537fe640 1 -- 172.21.15.115:0/2507684893 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0343f0 con 0x7fe0540b4070 2023-12-01T03:08:30.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.722+0000 7fe05a821640 1 -- 172.21.15.115:0/2507684893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540b4070 msgr2=0x7fe0540cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:30.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.722+0000 7fe05a821640 1 --2- 172.21.15.115:0/2507684893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540b4070 0x7fe0540cc8b0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009e50 tx=0x7fe04c02bed0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.722+0000 7fe05a821640 1 -- 172.21.15.115:0/2507684893 shutdown_connections 2023-12-01T03:08:30.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.722+0000 7fe05a821640 1 --2- 172.21.15.115:0/2507684893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540b4070 0x7fe0540cc8b0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.722+0000 7fe05a821640 1 --2- 172.21.15.115:0/2507684893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0540cb8c0 0x7fe0540cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.722+0000 7fe05a821640 1 -- 172.21.15.115:0/2507684893 >> 172.21.15.115:0/2507684893 conn(0x7fe0540662c0 msgr2=0x7fe0540b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:30.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.722+0000 7fe05a821640 1 -- 172.21.15.115:0/2507684893 shutdown_connections 2023-12-01T03:08:30.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.723+0000 7fe05a821640 1 -- 172.21.15.115:0/2507684893 wait complete. 2023-12-01T03:08:30.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.724+0000 7fe05a821640 1 Processor -- start 2023-12-01T03:08:30.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.724+0000 7fe05a821640 1 -- start start 2023-12-01T03:08:30.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.725+0000 7fe05a821640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0540b4070 0x7fe054152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:30.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.725+0000 7fe05a821640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540cb8c0 0x7fe0540ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:30.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.725+0000 7fe05a821640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe05415ac70 0x7fe0540ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:30.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.725+0000 7fe05a821640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05415b2c0 con 0x7fe0540cb8c0 2023-12-01T03:08:30.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.725+0000 7fe05a821640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540af820 con 0x7fe0540b4070 2023-12-01T03:08:30.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.725+0000 7fe05a821640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540aea80 con 0x7fe05415ac70 2023-12-01T03:08:30.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.726+0000 7fe04bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540cb8c0 0x7fe0540ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:30.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.726+0000 7fe04bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540cb8c0 0x7fe0540ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49612/0 (socket says 172.21.15.115:49612) 2023-12-01T03:08:30.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.726+0000 7fe04bfff640 1 -- 172.21.15.115:0/59424698 learned_addr learned my addr 172.21.15.115:0/59424698 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:30.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.726+0000 7fe053fff640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0540b4070 0x7fe054152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:30.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.726+0000 7fe058dbe640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe05415ac70 0x7fe0540ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:30.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.726+0000 7fe04bfff640 1 -- 172.21.15.115:0/59424698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe05415ac70 msgr2=0x7fe0540ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:30.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.726+0000 7fe04bfff640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe05415ac70 0x7fe0540ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.726+0000 7fe04bfff640 1 -- 172.21.15.115:0/59424698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0540b4070 msgr2=0x7fe054152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:30.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.726+0000 7fe04bfff640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0540b4070 0x7fe054152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:30.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.726+0000 7fe04bfff640 1 -- 172.21.15.115:0/59424698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c009b30 con 0x7fe0540cb8c0 2023-12-01T03:08:30.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.727+0000 7fe04bfff640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540cb8c0 0x7fe0540ad2b0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00c7f0 tx=0x7fe03c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:30.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.727+0000 7fe0517fa640 1 -- 172.21.15.115:0/59424698 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c004220 con 0x7fe0540cb8c0 2023-12-01T03:08:30.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.727+0000 7fe0517fa640 1 -- 172.21.15.115:0/59424698 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c0043a0 con 0x7fe0540cb8c0 2023-12-01T03:08:30.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.727+0000 7fe05a821640 1 -- 172.21.15.115:0/59424698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0541f2030 con 0x7fe0540cb8c0 2023-12-01T03:08:30.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.727+0000 7fe0517fa640 1 -- 172.21.15.115:0/59424698 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c007860 con 0x7fe0540cb8c0 2023-12-01T03:08:30.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.727+0000 7fe05a821640 1 -- 172.21.15.115:0/59424698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0540ae150 con 0x7fe0540cb8c0 2023-12-01T03:08:30.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.728+0000 7fe05a821640 1 -- 172.21.15.115:0/59424698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe054074210 con 0x7fe0540cb8c0 2023-12-01T03:08:30.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.729+0000 7fe0517fa640 1 -- 172.21.15.115:0/59424698 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe03c007bd0 con 0x7fe0540cb8c0 2023-12-01T03:08:30.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.730+0000 7fe0517fa640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe03406a550 0x7fe03406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:30.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.730+0000 7fe0517fa640 1 -- 172.21.15.115:0/59424698 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe03c088fe0 con 0x7fe0540cb8c0 2023-12-01T03:08:30.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.731+0000 7fe053fff640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe03406a550 0x7fe03406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:30.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.731+0000 7fe053fff640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe03406a550 0x7fe03406ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe044006fd0 tx=0x7fe044008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:30.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.735+0000 7fe0517fa640 1 -- 172.21.15.115:0/59424698 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe03c053ec0 con 0x7fe0540cb8c0 2023-12-01T03:08:30.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:30.924+0000 7fe05a821640 1 -- 172.21.15.115:0/59424698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe0540cd8f0 con 0x7fe0540cb8c0 2023-12-01T03:08:31.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.497+0000 7fe0517fa640 1 -- 172.21.15.115:0/59424698 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v186) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe03c01e020 con 0x7fe0540cb8c0 2023-12-01T03:08:31.500 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T03:08:31.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 -- 172.21.15.115:0/59424698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe03406a550 msgr2=0x7fe03406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:31.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe03406a550 0x7fe03406ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe044006fd0 tx=0x7fe044008040 comp rx=0 tx=0).stop 2023-12-01T03:08:31.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 -- 172.21.15.115:0/59424698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540cb8c0 msgr2=0x7fe0540ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:31.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540cb8c0 0x7fe0540ad2b0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00c7f0 tx=0x7fe03c00ccb0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 -- 172.21.15.115:0/59424698 shutdown_connections 2023-12-01T03:08:31.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe03406a550 0x7fe03406ca10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe05415ac70 0x7fe0540ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0540cb8c0 0x7fe0540ad2b0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 --2- 172.21.15.115:0/59424698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0540b4070 0x7fe054152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 -- 172.21.15.115:0/59424698 >> 172.21.15.115:0/59424698 conn(0x7fe0540662c0 msgr2=0x7fe0540cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:31.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 -- 172.21.15.115:0/59424698 shutdown_connections 2023-12-01T03:08:31.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.502+0000 7fe05a821640 1 -- 172.21.15.115:0/59424698 wait complete. 2023-12-01T03:08:31.525 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T03:08:31.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.742+0000 7f38f3f99640 1 Processor -- start 2023-12-01T03:08:31.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.742+0000 7f38f3f99640 1 -- start start 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.743+0000 7f38f3f99640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f38ec15ce40 0x7f38ec0b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.743+0000 7f38f3f99640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38ec0d2730 0x7f38ec0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.743+0000 7f38f3f99640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f38ec0b4640 0x7f38ec0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.743+0000 7f38f3f99640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec0b5030 con 0x7f38ec0b4640 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.743+0000 7f38f3f99640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec15d220 con 0x7f38ec15ce40 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.743+0000 7f38f3f99640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0d1460 con 0x7f38ec0d2730 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f2536640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f38ec0b4640 0x7f38ec0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f2536640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f38ec0b4640 0x7f38ec0b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49618/0 (socket says 172.21.15.115:49618) 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f2536640 1 -- 172.21.15.115:0/1899538566 learned_addr learned my addr 172.21.15.115:0/1899538566 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f1534640 1 --2- 172.21.15.115:0/1899538566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38ec0d2730 0x7f38ec0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f1d35640 1 --2- 172.21.15.115:0/1899538566 >> v2:172.21.15.142:3300/0 conn(0x7f38ec15ce40 0x7f38ec0b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f2536640 1 -- 172.21.15.115:0/1899538566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38ec0d2730 msgr2=0x7f38ec0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:31.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f2536640 1 --2- 172.21.15.115:0/1899538566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38ec0d2730 0x7f38ec0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f2536640 1 -- 172.21.15.115:0/1899538566 >> v2:172.21.15.142:3300/0 conn(0x7f38ec15ce40 msgr2=0x7f38ec0b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f2536640 1 --2- 172.21.15.115:0/1899538566 >> v2:172.21.15.142:3300/0 conn(0x7f38ec15ce40 0x7f38ec0b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f2536640 1 -- 172.21.15.115:0/1899538566 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0d1690 con 0x7f38ec0b4640 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.744+0000 7f38f1d35640 1 --2- 172.21.15.115:0/1899538566 >> v2:172.21.15.142:3300/0 conn(0x7f38ec15ce40 0x7f38ec0b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.745+0000 7f38f2536640 1 --2- 172.21.15.115:0/1899538566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0b4640 0x7f38ec0b4af0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0cf5a0 tx=0x7f38e402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecd7f05ef68212a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f0d33640 1 -- 172.21.15.115:0/1899538566 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e402c9f0 con 0x7f38ec0b4640 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f0d33640 1 -- 172.21.15.115:0/1899538566 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e402cb50 con 0x7f38ec0b4640 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f0d33640 1 -- 172.21.15.115:0/1899538566 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e40343f0 con 0x7f38ec0b4640 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f3f99640 1 -- 172.21.15.115:0/1899538566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0b4640 msgr2=0x7f38ec0b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f3f99640 1 --2- 172.21.15.115:0/1899538566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0b4640 0x7f38ec0b4af0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0cf5a0 tx=0x7f38e402be50 comp rx=0 tx=0).stop 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f3f99640 1 -- 172.21.15.115:0/1899538566 shutdown_connections 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f3f99640 1 --2- 172.21.15.115:0/1899538566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0b4640 0x7f38ec0b4af0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f3f99640 1 --2- 172.21.15.115:0/1899538566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38ec0d2730 0x7f38ec0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f3f99640 1 --2- 172.21.15.115:0/1899538566 >> v2:172.21.15.142:3300/0 conn(0x7f38ec15ce40 0x7f38ec0b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f3f99640 1 -- 172.21.15.115:0/1899538566 >> 172.21.15.115:0/1899538566 conn(0x7f38ec065f90 msgr2=0x7f38ec0cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f3f99640 1 -- 172.21.15.115:0/1899538566 shutdown_connections 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.746+0000 7f38f3f99640 1 -- 172.21.15.115:0/1899538566 wait complete. 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.748+0000 7f38f3f99640 1 Processor -- start 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.748+0000 7f38f3f99640 1 -- start start 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.749+0000 7f38f3f99640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38ec0b4640 0x7f38ec1491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.749+0000 7f38f3f99640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0d2730 0x7f38ec0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.749+0000 7f38f3f99640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38ec15ce40 0x7f38ec149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.749+0000 7f38f3f99640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec1561c0 con 0x7f38ec0d2730 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.749+0000 7f38f3f99640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0cb1a0 con 0x7f38ec0b4640 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.749+0000 7f38f3f99640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0cb480 con 0x7f38ec15ce40 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.750+0000 7f38f1534640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0d2730 0x7f38ec0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.750+0000 7f38f1534640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0d2730 0x7f38ec0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49624/0 (socket says 172.21.15.115:49624) 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.750+0000 7f38f1534640 1 -- 172.21.15.115:0/3496679269 learned_addr learned my addr 172.21.15.115:0/3496679269 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:31.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.750+0000 7f38f1534640 1 -- 172.21.15.115:0/3496679269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38ec15ce40 msgr2=0x7f38ec149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.750+0000 7f38f1534640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38ec15ce40 0x7f38ec149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.750+0000 7f38f1534640 1 -- 172.21.15.115:0/3496679269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38ec0b4640 msgr2=0x7f38ec1491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.750+0000 7f38f1534640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38ec0b4640 0x7f38ec1491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.750+0000 7f38f1534640 1 -- 172.21.15.115:0/3496679269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4009b30 con 0x7f38ec0d2730 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.750+0000 7f38f1534640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0d2730 0x7f38ec0c66f0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d5f0 tx=0x7f38dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.751+0000 7f38e27fc640 1 -- 172.21.15.115:0/3496679269 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc018070 con 0x7f38ec0d2730 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.751+0000 7f38e27fc640 1 -- 172.21.15.115:0/3496679269 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc004240 con 0x7f38ec0d2730 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.751+0000 7f38f3f99640 1 -- 172.21.15.115:0/3496679269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec0cb760 con 0x7f38ec0d2730 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.751+0000 7f38e27fc640 1 -- 172.21.15.115:0/3496679269 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0043f0 con 0x7f38ec0d2730 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.751+0000 7f38f3f99640 1 -- 172.21.15.115:0/3496679269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec0cbd20 con 0x7f38ec0d2730 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.752+0000 7f38f3f99640 1 -- 172.21.15.115:0/3496679269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec0cbee0 con 0x7f38ec0d2730 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.753+0000 7f38e27fc640 1 -- 172.21.15.115:0/3496679269 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f38dc003610 con 0x7f38ec0d2730 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.754+0000 7f38e27fc640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f38d006a5a0 0x7f38d006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.754+0000 7f38e27fc640 1 -- 172.21.15.115:0/3496679269 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f38dc088b90 con 0x7f38ec0d2730 2023-12-01T03:08:31.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.754+0000 7f38f1d35640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f38d006a5a0 0x7f38d006ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:31.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.755+0000 7f38f1d35640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f38d006a5a0 0x7f38d006ca60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f38e8006fd0 tx=0x7f38e8008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:31.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.758+0000 7f38e27fc640 1 -- 172.21.15.115:0/3496679269 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f38dc053d70 con 0x7f38ec0d2730 2023-12-01T03:08:31.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:31.953+0000 7f38f3f99640 1 -- 172.21.15.115:0/3496679269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38ec0d1f00 con 0x7f38ec0d2730 2023-12-01T03:08:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T03:08:32.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.359+0000 7f38e27fc640 1 -- 172.21.15.115:0/3496679269 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v187) v1 ==== 169+0+0 (secure 0 0 0) 0x7f38dc057390 con 0x7f38ec0d2730 2023-12-01T03:08:32.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.364+0000 7f38f3f99640 1 -- 172.21.15.115:0/3496679269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f38d006a5a0 msgr2=0x7f38d006ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:32.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.364+0000 7f38f3f99640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f38d006a5a0 0x7f38d006ca60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f38e8006fd0 tx=0x7f38e8008040 comp rx=0 tx=0).stop 2023-12-01T03:08:32.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.364+0000 7f38f3f99640 1 -- 172.21.15.115:0/3496679269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0d2730 msgr2=0x7f38ec0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:32.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.364+0000 7f38f3f99640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0d2730 0x7f38ec0c66f0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d5f0 tx=0x7f38dc00dab0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.365+0000 7f38f3f99640 1 -- 172.21.15.115:0/3496679269 shutdown_connections 2023-12-01T03:08:32.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.365+0000 7f38f3f99640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f38d006a5a0 0x7f38d006ca60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.365+0000 7f38f3f99640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38ec15ce40 0x7f38ec149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.365+0000 7f38f3f99640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38ec0d2730 0x7f38ec0c66f0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.365+0000 7f38f3f99640 1 --2- 172.21.15.115:0/3496679269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38ec0b4640 0x7f38ec1491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.365+0000 7f38f3f99640 1 -- 172.21.15.115:0/3496679269 >> 172.21.15.115:0/3496679269 conn(0x7f38ec065f90 msgr2=0x7f38ec15c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:32.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.365+0000 7f38f3f99640 1 -- 172.21.15.115:0/3496679269 shutdown_connections 2023-12-01T03:08:32.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.365+0000 7f38f3f99640 1 -- 172.21.15.115:0/3496679269 wait complete. 2023-12-01T03:08:32.393 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T03:08:32.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.606+0000 7f941670a640 1 Processor -- start 2023-12-01T03:08:32.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.606+0000 7f941670a640 1 -- start start 2023-12-01T03:08:32.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.608+0000 7f941670a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f941015ac80 0x7f94100cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:32.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.608+0000 7f941670a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100cc9a0 con 0x7f94100ce1b0 2023-12-01T03:08:32.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.608+0000 7f941670a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100cbd90 con 0x7f94100b0e60 2023-12-01T03:08:32.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.608+0000 7f941670a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941015b060 con 0x7f941015ac80 2023-12-01T03:08:32.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.608+0000 7f9414ca7640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f94100ce1b0 0x7f941006b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51320/0 (socket says 172.21.15.115:51320) 2023-12-01T03:08:32.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.608+0000 7f9414ca7640 1 -- 172.21.15.115:0/4229450739 learned_addr learned my addr 172.21.15.115:0/4229450739 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:32.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940ffff640 1 --2- 172.21.15.115:0/4229450739 >> v2:172.21.15.142:3300/0 conn(0x7f941015ac80 0x7f94100cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:32.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322575481 0 0) 0x7f94100cc9a0 con 0x7f94100ce1b0 2023-12-01T03:08:32.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93fc003520 con 0x7f94100ce1b0 2023-12-01T03:08:32.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3494918530 0 0) 0x7f93fc003520 con 0x7f94100ce1b0 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 >> v2:172.21.15.142:3300/0 conn(0x7f941015ac80 msgr2=0x7f94100cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 --2- 172.21.15.115:0/4229450739 >> v2:172.21.15.142:3300/0 conn(0x7f941015ac80 0x7f94100cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 >> v1:172.21.15.142:6789/0 conn(0x7f94100b0e60 legacy=0x7f94100cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94100cc070 con 0x7f94100ce1b0 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f94080019f0 con 0x7f94100ce1b0 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940ffff640 1 --2- 172.21.15.115:0/4229450739 >> v2:172.21.15.142:3300/0 conn(0x7f941015ac80 0x7f94100cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 >> v1:172.21.15.115:6789/0 conn(0x7f94100ce1b0 legacy=0x7f941006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 --2- 172.21.15.115:0/4229450739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93fc003970 0x7f93fc003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 --2- 172.21.15.115:0/4229450739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93fc0047c0 0x7f93fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 --2- 172.21.15.115:0/4229450739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc005180 0x7f93fc00fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941015b060 con 0x7f93fc003970 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0104c0 con 0x7f93fc0047c0 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.609+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc010680 con 0x7f93fc005180 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.610+0000 7f940ffff640 1 --2- 172.21.15.115:0/4229450739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93fc003970 0x7f93fc003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:32.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.610+0000 7f940ffff640 1 -- 172.21.15.115:0/4229450739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc005180 msgr2=0x7f93fc00fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:32.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.610+0000 7f940ffff640 1 --2- 172.21.15.115:0/4229450739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc005180 0x7f93fc00fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.610+0000 7f940ffff640 1 -- 172.21.15.115:0/4229450739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93fc0047c0 msgr2=0x7f93fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:32.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.610+0000 7f9407fff640 1 --2- 172.21.15.115:0/4229450739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93fc0047c0 0x7f93fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:32.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.610+0000 7f940ffff640 1 --2- 172.21.15.115:0/4229450739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93fc0047c0 0x7f93fc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.610+0000 7f9414ca7640 1 -- 172.21.15.115:0/4229450739 reap_dead start 2023-12-01T03:08:32.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.610+0000 7f940ffff640 1 -- 172.21.15.115:0/4229450739 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc003520 con 0x7f93fc003970 2023-12-01T03:08:32.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.611+0000 7f940ffff640 1 --2- 172.21.15.115:0/4229450739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93fc003970 0x7f93fc003d70 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f9400006fd0 tx=0x7f93fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2453be7f5aed3b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:32.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.611+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400004030 con 0x7f93fc003970 2023-12-01T03:08:32.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.611+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9400004190 con 0x7f93fc003970 2023-12-01T03:08:32.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.611+0000 7f940f7fe640 1 -- 172.21.15.115:0/4229450739 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400003610 con 0x7f93fc003970 2023-12-01T03:08:32.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.612+0000 7f941670a640 1 -- 172.21.15.115:0/4229450739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93fc003970 msgr2=0x7f93fc003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:32.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.612+0000 7f941670a640 1 --2- 172.21.15.115:0/4229450739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93fc003970 0x7f93fc003d70 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f9400006fd0 tx=0x7f93fc0057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.612+0000 7f941670a640 1 -- 172.21.15.115:0/4229450739 shutdown_connections 2023-12-01T03:08:32.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.612+0000 7f941670a640 1 --2- 172.21.15.115:0/4229450739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93fc003970 0x7f93fc003d70 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.612+0000 7f941670a640 1 -- 172.21.15.115:0/4229450739 >> 172.21.15.115:0/4229450739 conn(0x7f9410065960 msgr2=0x7f9410065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:32.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.612+0000 7f941670a640 1 -- 172.21.15.115:0/4229450739 shutdown_connections 2023-12-01T03:08:32.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.612+0000 7f941670a640 1 -- 172.21.15.115:0/4229450739 wait complete. 2023-12-01T03:08:32.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.614+0000 7f941670a640 1 Processor -- start 2023-12-01T03:08:32.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.614+0000 7f941670a640 1 -- start start 2023-12-01T03:08:32.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f941670a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc003970 0x7f9410156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:32.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f941670a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f941015db00 0x7f941013f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:32.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f941670a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f941006b160 0x7f941006b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:32.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f941670a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941013f810 con 0x7f941006b160 2023-12-01T03:08:32.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f941670a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941006bb20 con 0x7f941015db00 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f941670a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941015d670 con 0x7f93fc003970 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f9414ca7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f941006b160 0x7f941006b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f9414ca7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f941006b160 0x7f941006b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49642/0 (socket says 172.21.15.115:49642) 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f9414ca7640 1 -- 172.21.15.115:0/2803054166 learned_addr learned my addr 172.21.15.115:0/2803054166 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f9407fff640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f941015db00 0x7f941013f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f940ffff640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc003970 0x7f9410156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f9414ca7640 1 -- 172.21.15.115:0/2803054166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc003970 msgr2=0x7f9410156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f9414ca7640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc003970 0x7f9410156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f9414ca7640 1 -- 172.21.15.115:0/2803054166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f941015db00 msgr2=0x7f941013f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f9414ca7640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f941015db00 0x7f941013f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f9414ca7640 1 -- 172.21.15.115:0/2803054166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941015df80 con 0x7f941006b160 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.616+0000 7f940ffff640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc003970 0x7f9410156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.617+0000 7f9414ca7640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f941006b160 0x7f941006b5e0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f94100cbb40 tx=0x7f940800ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:32.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.617+0000 7f940d7fa640 1 -- 172.21.15.115:0/2803054166 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940800a590 con 0x7f941006b160 2023-12-01T03:08:32.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.617+0000 7f940d7fa640 1 -- 172.21.15.115:0/2803054166 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940800a710 con 0x7f941006b160 2023-12-01T03:08:32.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.617+0000 7f940d7fa640 1 -- 172.21.15.115:0/2803054166 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940800aac0 con 0x7f941006b160 2023-12-01T03:08:32.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.617+0000 7f941670a640 1 -- 172.21.15.115:0/2803054166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941015c960 con 0x7f941006b160 2023-12-01T03:08:32.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.617+0000 7f941670a640 1 -- 172.21.15.115:0/2803054166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941015d300 con 0x7f941006b160 2023-12-01T03:08:32.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.618+0000 7f941670a640 1 -- 172.21.15.115:0/2803054166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941015d7e0 con 0x7f941006b160 2023-12-01T03:08:32.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.619+0000 7f940d7fa640 1 -- 172.21.15.115:0/2803054166 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f940800ac20 con 0x7f941006b160 2023-12-01T03:08:32.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.620+0000 7f940d7fa640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f93f006a580 0x7f93f006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:32.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.620+0000 7f940d7fa640 1 -- 172.21.15.115:0/2803054166 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f9408088840 con 0x7f941006b160 2023-12-01T03:08:32.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.620+0000 7f940ffff640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f93f006a580 0x7f93f006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:32.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.621+0000 7f940ffff640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f93f006a580 0x7f93f006ca40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9400035000 tx=0x7f940003a040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:32.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.626+0000 7f940d7fa640 1 -- 172.21.15.115:0/2803054166 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9408053b80 con 0x7f941006b160 2023-12-01T03:08:32.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.835+0000 7f941670a640 1 -- 172.21.15.115:0/2803054166 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f941004e890 con 0x7f941006b160 2023-12-01T03:08:32.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.841+0000 7f940d7fa640 1 -- 172.21.15.115:0/2803054166 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f94080571a0 con 0x7f941006b160 2023-12-01T03:08:32.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 -- 172.21.15.115:0/2803054166 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f93f006a580 msgr2=0x7f93f006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:32.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f93f006a580 0x7f93f006ca40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9400035000 tx=0x7f940003a040 comp rx=0 tx=0).stop 2023-12-01T03:08:32.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 -- 172.21.15.115:0/2803054166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f941006b160 msgr2=0x7f941006b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:32.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f941006b160 0x7f941006b5e0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f94100cbb40 tx=0x7f940800ce80 comp rx=0 tx=0).stop 2023-12-01T03:08:32.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 -- 172.21.15.115:0/2803054166 shutdown_connections 2023-12-01T03:08:32.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f93f006a580 0x7f93f006ca40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f941006b160 0x7f941006b5e0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f941015db00 0x7f941013f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 --2- 172.21.15.115:0/2803054166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc003970 0x7f9410156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:32.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 -- 172.21.15.115:0/2803054166 >> 172.21.15.115:0/2803054166 conn(0x7f9410065960 msgr2=0x7f94100b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:32.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 -- 172.21.15.115:0/2803054166 shutdown_connections 2023-12-01T03:08:32.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:32.844+0000 7f941670a640 1 -- 172.21.15.115:0/2803054166 wait complete. 2023-12-01T03:08:32.863 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T03:08:33.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.064+0000 7f3413589640 1 Processor -- start 2023-12-01T03:08:33.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.064+0000 7f3413589640 1 -- start start 2023-12-01T03:08:33.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.066+0000 7f3413589640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f340c15cd90 0x7f340c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.066+0000 7f3413589640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f340c0cfd60 0x7f340c161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.066+0000 7f3413589640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c0ce5e0 con 0x7f340c0b5fa0 2023-12-01T03:08:33.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.066+0000 7f3413589640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340c0d0ec0 con 0x7f340c15cd90 2023-12-01T03:08:33.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.066+0000 7f3413589640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c161950 con 0x7f340c0cfd60 2023-12-01T03:08:33.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.066+0000 7f3410b24640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f340c0b5fa0 0x7f340c0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51322/0 (socket says 172.21.15.115:51322) 2023-12-01T03:08:33.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.066+0000 7f3410b24640 1 -- 172.21.15.115:0/2065586658 learned_addr learned my addr 172.21.15.115:0/2065586658 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:33.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.066+0000 7f3411325640 1 --2- 172.21.15.115:0/2065586658 >> v2:172.21.15.115:3300/0 conn(0x7f340c15cd90 0x7f340c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.066+0000 7f3411b26640 1 --2- 172.21.15.115:0/2065586658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f340c0cfd60 0x7f340c161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.067+0000 7f340bfff640 1 -- 172.21.15.115:0/2065586658 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349437875 0 0) 0x7f3400001490 con 0x7f340c0b5fa0 2023-12-01T03:08:33.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.067+0000 7f340bfff640 1 -- 172.21.15.115:0/2065586658 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33f4003520 con 0x7f340c0b5fa0 2023-12-01T03:08:33.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.067+0000 7f3411325640 1 -- 172.21.15.115:0/2065586658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f340c0cfd60 msgr2=0x7f340c161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:33.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.067+0000 7f3411325640 1 --2- 172.21.15.115:0/2065586658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f340c0cfd60 0x7f340c161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.067+0000 7f3411325640 1 -- 172.21.15.115:0/2065586658 >> v1:172.21.15.115:6789/0 conn(0x7f340c0b5fa0 legacy=0x7f340c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.067+0000 7f3411325640 1 -- 172.21.15.115:0/2065586658 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0ce300 con 0x7f340c15cd90 2023-12-01T03:08:33.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.068+0000 7f3411325640 1 --2- 172.21.15.115:0/2065586658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c15cd90 0x7f340c0cdd20 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0097a0 tx=0x7f33fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917f1be71a07a68f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:33.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f340bfff640 1 -- 172.21.15.115:0/2065586658 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc031830 con 0x7f340c15cd90 2023-12-01T03:08:33.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f340bfff640 1 -- 172.21.15.115:0/2065586658 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc031990 con 0x7f340c15cd90 2023-12-01T03:08:33.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f340bfff640 1 -- 172.21.15.115:0/2065586658 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc031d20 con 0x7f340c15cd90 2023-12-01T03:08:33.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f3413589640 1 -- 172.21.15.115:0/2065586658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c15cd90 msgr2=0x7f340c0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f3413589640 1 --2- 172.21.15.115:0/2065586658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c15cd90 0x7f340c0cdd20 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0097a0 tx=0x7f33fc02b8d0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f3413589640 1 -- 172.21.15.115:0/2065586658 shutdown_connections 2023-12-01T03:08:33.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f3413589640 1 --2- 172.21.15.115:0/2065586658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f340c0cfd60 0x7f340c161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f3413589640 1 --2- 172.21.15.115:0/2065586658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c15cd90 0x7f340c0cdd20 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f3413589640 1 -- 172.21.15.115:0/2065586658 >> 172.21.15.115:0/2065586658 conn(0x7f340c065960 msgr2=0x7f340c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:33.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f3413589640 1 -- 172.21.15.115:0/2065586658 shutdown_connections 2023-12-01T03:08:33.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.069+0000 7f3413589640 1 -- 172.21.15.115:0/2065586658 wait complete. 2023-12-01T03:08:33.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.071+0000 7f3413589640 1 Processor -- start 2023-12-01T03:08:33.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.071+0000 7f3413589640 1 -- start start 2023-12-01T03:08:33.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.072+0000 7f3413589640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f340c0b5fa0 0x7f340c0aa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.072+0000 7f3413589640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c0cfd60 0x7f340c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.072+0000 7f3413589640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f340c15cd90 0x7f340c15b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.072+0000 7f3413589640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0ab330 con 0x7f340c0cfd60 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.072+0000 7f3413589640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0ab610 con 0x7f340c0b5fa0 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.072+0000 7f3413589640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0ab8f0 con 0x7f340c15cd90 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.072+0000 7f3410b24640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c0cfd60 0x7f340c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.072+0000 7f3410b24640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c0cfd60 0x7f340c15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49670/0 (socket says 172.21.15.115:49670) 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.072+0000 7f3410b24640 1 -- 172.21.15.115:0/3712228860 learned_addr learned my addr 172.21.15.115:0/3712228860 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.073+0000 7f3410b24640 1 -- 172.21.15.115:0/3712228860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f340c15cd90 msgr2=0x7f340c15b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.073+0000 7f3410b24640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f340c15cd90 0x7f340c15b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.073+0000 7f3410b24640 1 -- 172.21.15.115:0/3712228860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f340c0b5fa0 msgr2=0x7f340c0aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.073+0000 7f3410b24640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f340c0b5fa0 0x7f340c0aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.073+0000 7f3410b24640 1 -- 172.21.15.115:0/3712228860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc009400 con 0x7f340c0cfd60 2023-12-01T03:08:33.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.073+0000 7f3410b24640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c0cfd60 0x7f340c15aae0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f340000cb90 tx=0x7f340000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:33.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.073+0000 7f3409ffb640 1 -- 172.21.15.115:0/3712228860 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340000ad40 con 0x7f340c0cfd60 2023-12-01T03:08:33.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.073+0000 7f3409ffb640 1 -- 172.21.15.115:0/3712228860 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3400012400 con 0x7f340c0cfd60 2023-12-01T03:08:33.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.073+0000 7f3409ffb640 1 -- 172.21.15.115:0/3712228860 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340000aea0 con 0x7f340c0cfd60 2023-12-01T03:08:33.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.073+0000 7f3413589640 1 -- 172.21.15.115:0/3712228860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c0abbd0 con 0x7f340c0cfd60 2023-12-01T03:08:33.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.074+0000 7f3413589640 1 -- 172.21.15.115:0/3712228860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c0aafc0 con 0x7f340c0cfd60 2023-12-01T03:08:33.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.074+0000 7f3413589640 1 -- 172.21.15.115:0/3712228860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340c0ac0f0 con 0x7f340c0cfd60 2023-12-01T03:08:33.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.076+0000 7f3409ffb640 1 -- 172.21.15.115:0/3712228860 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3400007070 con 0x7f340c0cfd60 2023-12-01T03:08:33.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.076+0000 7f3409ffb640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f33f806a550 0x7f33f806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.077+0000 7f3409ffb640 1 -- 172.21.15.115:0/3712228860 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f3400088ac0 con 0x7f340c0cfd60 2023-12-01T03:08:33.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.077+0000 7f3411325640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f33f806a550 0x7f33f806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.077+0000 7f3411325640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f33f806a550 0x7f33f806ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f33fc036000 tx=0x7f33fc030000 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:33.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.081+0000 7f3409ffb640 1 -- 172.21.15.115:0/3712228860 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f340c0ac0f0 con 0x7f340c0cfd60 2023-12-01T03:08:33.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.272+0000 7f3413589640 1 -- 172.21.15.115:0/3712228860 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f340c0ce5e0 con 0x7f340c0cfd60 2023-12-01T03:08:33.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.281+0000 7f3409ffb640 1 -- 172.21.15.115:0/3712228860 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v62) v1 ==== 120+0+0 (secure 0 0 0) 0x7f340c0ce5e0 con 0x7f340c0cfd60 2023-12-01T03:08:33.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.284+0000 7f3413589640 1 -- 172.21.15.115:0/3712228860 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f33f806a550 msgr2=0x7f33f806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.284+0000 7f3413589640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f33f806a550 0x7f33f806ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f33fc036000 tx=0x7f33fc030000 comp rx=0 tx=0).stop 2023-12-01T03:08:33.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.284+0000 7f3413589640 1 -- 172.21.15.115:0/3712228860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c0cfd60 msgr2=0x7f340c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.284+0000 7f3413589640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c0cfd60 0x7f340c15aae0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f340000cb90 tx=0x7f340000a430 comp rx=0 tx=0).stop 2023-12-01T03:08:33.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.284+0000 7f3413589640 1 -- 172.21.15.115:0/3712228860 shutdown_connections 2023-12-01T03:08:33.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.284+0000 7f3413589640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f33f806a550 0x7f33f806ca10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.284+0000 7f3413589640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f340c15cd90 0x7f340c15b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.284+0000 7f3413589640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f340c0cfd60 0x7f340c15aae0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.284+0000 7f3413589640 1 --2- 172.21.15.115:0/3712228860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f340c0b5fa0 0x7f340c0aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.284+0000 7f3413589640 1 -- 172.21.15.115:0/3712228860 >> 172.21.15.115:0/3712228860 conn(0x7f340c065960 msgr2=0x7f340c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:33.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.285+0000 7f3413589640 1 -- 172.21.15.115:0/3712228860 shutdown_connections 2023-12-01T03:08:33.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.285+0000 7f3413589640 1 -- 172.21.15.115:0/3712228860 wait complete. 2023-12-01T03:08:33.307 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T03:08:33.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.520+0000 7ff050edc640 1 Processor -- start 2023-12-01T03:08:33.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.520+0000 7ff050edc640 1 -- start start 2023-12-01T03:08:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.521+0000 7ff050edc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff04c06d180 0x7ff04c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.521+0000 7ff050edc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04c15e390 con 0x7ff04c0cdc40 2023-12-01T03:08:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.521+0000 7ff050edc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04c0cce80 con 0x7ff04c150940 2023-12-01T03:08:33.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.521+0000 7ff050edc640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04c0cd820 con 0x7ff04c06d180 2023-12-01T03:08:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.522+0000 7ff04bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff04c0cdc40 0x7ff04c15db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51330/0 (socket says 172.21.15.115:51330) 2023-12-01T03:08:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.522+0000 7ff04affd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff04c06d180 0x7ff04c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.522+0000 7ff04bfff640 1 -- 172.21.15.115:0/2765347789 learned_addr learned my addr 172.21.15.115:0/2765347789 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.522+0000 7ff04affd640 1 -- 172.21.15.115:0/2765347789 >> v1:172.21.15.142:6789/0 conn(0x7ff04c150940 legacy=0x7ff04c06ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.522+0000 7ff04affd640 1 -- 172.21.15.115:0/2765347789 >> v1:172.21.15.115:6789/0 conn(0x7ff04c0cdc40 legacy=0x7ff04c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.522+0000 7ff04a7fc640 1 -- 172.21.15.115:0/2765347789 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922816989 0 0) 0x7ff04c15e390 con 0x7ff04c0cdc40 2023-12-01T03:08:33.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.522+0000 7ff04affd640 1 -- 172.21.15.115:0/2765347789 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0cd160 con 0x7ff04c06d180 2023-12-01T03:08:33.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.524+0000 7ff04affd640 1 --2- 172.21.15.115:0/2765347789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c06d180 0x7ff04c15d5d0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7ff0340097a0 tx=0x7ff03402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf39a78367661b90 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.524+0000 7ff04a7fc640 1 -- 172.21.15.115:0/2765347789 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03402e070 con 0x7ff04c06d180 2023-12-01T03:08:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.524+0000 7ff04a7fc640 1 -- 172.21.15.115:0/2765347789 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0340315a0 con 0x7ff04c06d180 2023-12-01T03:08:33.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.524+0000 7ff04a7fc640 1 -- 172.21.15.115:0/2765347789 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034031930 con 0x7ff04c06d180 2023-12-01T03:08:33.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.524+0000 7ff050edc640 1 -- 172.21.15.115:0/2765347789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c06d180 msgr2=0x7ff04c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.524+0000 7ff050edc640 1 --2- 172.21.15.115:0/2765347789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c06d180 0x7ff04c15d5d0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7ff0340097a0 tx=0x7ff03402b840 comp rx=0 tx=0).stop 2023-12-01T03:08:33.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.524+0000 7ff050edc640 1 -- 172.21.15.115:0/2765347789 shutdown_connections 2023-12-01T03:08:33.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.525+0000 7ff050edc640 1 --2- 172.21.15.115:0/2765347789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c06d180 0x7ff04c15d5d0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.525+0000 7ff050edc640 1 -- 172.21.15.115:0/2765347789 >> 172.21.15.115:0/2765347789 conn(0x7ff04c0cee70 msgr2=0x7ff04c0cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:33.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.525+0000 7ff050edc640 1 -- 172.21.15.115:0/2765347789 shutdown_connections 2023-12-01T03:08:33.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.525+0000 7ff050edc640 1 -- 172.21.15.115:0/2765347789 wait complete. 2023-12-01T03:08:33.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.526+0000 7ff050edc640 1 Processor -- start 2023-12-01T03:08:33.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.526+0000 7ff050edc640 1 -- start start 2023-12-01T03:08:33.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.527+0000 7ff050edc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff04c06d180 0x7ff04c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.527+0000 7ff050edc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff04c0cdc40 0x7ff04c14d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.527+0000 7ff050edc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c150940 0x7ff04c1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.527+0000 7ff050edc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c070970 con 0x7ff04c150940 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.527+0000 7ff050edc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c070c50 con 0x7ff04c06d180 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.527+0000 7ff050edc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c06f8e0 con 0x7ff04c0cdc40 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c150940 0x7ff04c1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c150940 0x7ff04c1521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49674/0 (socket says 172.21.15.115:49674) 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04bfff640 1 -- 172.21.15.115:0/3291264664 learned_addr learned my addr 172.21.15.115:0/3291264664 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04b7fe640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff04c06d180 0x7ff04c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04bfff640 1 -- 172.21.15.115:0/3291264664 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff04c0cdc40 msgr2=0x7ff04c14d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04affd640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff04c0cdc40 0x7ff04c14d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04bfff640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff04c0cdc40 0x7ff04c14d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04bfff640 1 -- 172.21.15.115:0/3291264664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff04c06d180 msgr2=0x7ff04c0703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04bfff640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff04c06d180 0x7ff04c0703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04bfff640 1 -- 172.21.15.115:0/3291264664 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff034009400 con 0x7ff04c150940 2023-12-01T03:08:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04b7fe640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff04c06d180 0x7ff04c0703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.528+0000 7ff04affd640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff04c0cdc40 0x7ff04c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.529+0000 7ff04bfff640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c150940 0x7ff04c1521e0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ff04400c450 tx=0x7ff04400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.529+0000 7ff043fff640 1 -- 172.21.15.115:0/3291264664 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0440074e0 con 0x7ff04c150940 2023-12-01T03:08:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.529+0000 7ff043fff640 1 -- 172.21.15.115:0/3291264664 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff044007660 con 0x7ff04c150940 2023-12-01T03:08:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.529+0000 7ff043fff640 1 -- 172.21.15.115:0/3291264664 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044004a20 con 0x7ff04c150940 2023-12-01T03:08:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.529+0000 7ff050edc640 1 -- 172.21.15.115:0/3291264664 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04c06fbc0 con 0x7ff04c150940 2023-12-01T03:08:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.529+0000 7ff050edc640 1 -- 172.21.15.115:0/3291264664 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04c070f30 con 0x7ff04c150940 2023-12-01T03:08:33.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.530+0000 7ff050edc640 1 -- 172.21.15.115:0/3291264664 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04c0700e0 con 0x7ff04c150940 2023-12-01T03:08:33.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.531+0000 7ff043fff640 1 -- 172.21.15.115:0/3291264664 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff044004b80 con 0x7ff04c150940 2023-12-01T03:08:33.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.532+0000 7ff043fff640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff02c06a550 0x7ff02c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.532+0000 7ff043fff640 1 -- 172.21.15.115:0/3291264664 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5961+0+0 (secure 0 0 0) 0x7ff044087de0 con 0x7ff04c150940 2023-12-01T03:08:33.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.532+0000 7ff04b7fe640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff02c06a550 0x7ff02c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.533+0000 7ff04b7fe640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff02c06a550 0x7ff02c06ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0012e0 tx=0x7ff03c009040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:33.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.536+0000 7ff043fff640 1 -- 172.21.15.115:0/3291264664 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff04c0700e0 con 0x7ff04c150940 2023-12-01T03:08:33.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.731+0000 7ff050edc640 1 -- 172.21.15.115:0/3291264664 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ff04c0700e0 con 0x7ff04c150940 2023-12-01T03:08:33.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.740+0000 7ff043fff640 1 -- 172.21.15.115:0/3291264664 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v63) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff04c0700e0 con 0x7ff04c150940 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.743+0000 7ff050edc640 1 -- 172.21.15.115:0/3291264664 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff02c06a550 msgr2=0x7ff02c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.743+0000 7ff050edc640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff02c06a550 0x7ff02c06ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0012e0 tx=0x7ff03c009040 comp rx=0 tx=0).stop 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.743+0000 7ff050edc640 1 -- 172.21.15.115:0/3291264664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c150940 msgr2=0x7ff04c1521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.743+0000 7ff050edc640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c150940 0x7ff04c1521e0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ff04400c450 tx=0x7ff04400c910 comp rx=0 tx=0).stop 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.744+0000 7ff050edc640 1 -- 172.21.15.115:0/3291264664 shutdown_connections 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.744+0000 7ff050edc640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff02c06a550 0x7ff02c06ca10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.744+0000 7ff050edc640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff04c0cdc40 0x7ff04c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.744+0000 7ff050edc640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff04c150940 0x7ff04c1521e0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.744+0000 7ff050edc640 1 --2- 172.21.15.115:0/3291264664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff04c06d180 0x7ff04c0703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.744+0000 7ff050edc640 1 -- 172.21.15.115:0/3291264664 >> 172.21.15.115:0/3291264664 conn(0x7ff04c0cee70 msgr2=0x7ff04c14fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.744+0000 7ff050edc640 1 -- 172.21.15.115:0/3291264664 shutdown_connections 2023-12-01T03:08:33.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.744+0000 7ff050edc640 1 -- 172.21.15.115:0/3291264664 wait complete. 2023-12-01T03:08:33.764 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-12-01T03:08:33.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.970+0000 7f74b2cf7640 1 Processor -- start 2023-12-01T03:08:33.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.971+0000 7f74b2cf7640 1 -- start start 2023-12-01T03:08:33.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.972+0000 7f74b2cf7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f74ac15cf90 0x7f74ac0d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.972+0000 7f74b2cf7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac0b8610 0x7f74ac0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.972+0000 7f74b2cf7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f74ac0cdf00 0x7f74ac0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.972+0000 7f74b2cf7640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ac0ce390 con 0x7f74ac0cdf00 2023-12-01T03:08:33.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.972+0000 7f74b2cf7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ac15d370 con 0x7f74ac15cf90 2023-12-01T03:08:33.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.972+0000 7f74b2cf7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac0d19a0 con 0x7f74ac0b8610 2023-12-01T03:08:33.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74b1294640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f74ac0cdf00 0x7f74ac0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74b1294640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f74ac0cdf00 0x7f74ac0b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49678/0 (socket says 172.21.15.115:49678) 2023-12-01T03:08:33.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74b1294640 1 -- 172.21.15.115:0/227381273 learned_addr learned my addr 172.21.15.115:0/227381273 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:33.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74abfff640 1 --2- 172.21.15.115:0/227381273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac0b8610 0x7f74ac0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74b0a93640 1 --2- 172.21.15.115:0/227381273 >> v2:172.21.15.142:3300/0 conn(0x7f74ac15cf90 0x7f74ac0d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74b1294640 1 -- 172.21.15.115:0/227381273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac0b8610 msgr2=0x7f74ac0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:33.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74b1294640 1 --2- 172.21.15.115:0/227381273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac0b8610 0x7f74ac0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74b1294640 1 -- 172.21.15.115:0/227381273 >> v2:172.21.15.142:3300/0 conn(0x7f74ac15cf90 msgr2=0x7f74ac0d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:33.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74b1294640 1 --2- 172.21.15.115:0/227381273 >> v2:172.21.15.142:3300/0 conn(0x7f74ac15cf90 0x7f74ac0d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74b1294640 1 -- 172.21.15.115:0/227381273 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac0d1c80 con 0x7f74ac0cdf00 2023-12-01T03:08:33.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74abfff640 1 --2- 172.21.15.115:0/227381273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac0b8610 0x7f74ac0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:33.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.973+0000 7f74b0a93640 1 --2- 172.21.15.115:0/227381273 >> v2:172.21.15.142:3300/0 conn(0x7f74ac15cf90 0x7f74ac0d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:33.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.974+0000 7f74b1294640 1 --2- 172.21.15.115:0/227381273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 0x7f74ac0b4670 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f74ac066fc0 tx=0x7f74a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84283818c85713af server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.975+0000 7f74ab7fe640 1 -- 172.21.15.115:0/227381273 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4031870 con 0x7f74ac0cdf00 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.975+0000 7f74ab7fe640 1 -- 172.21.15.115:0/227381273 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a40319d0 con 0x7f74ac0cdf00 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.975+0000 7f74ab7fe640 1 -- 172.21.15.115:0/227381273 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4031d60 con 0x7f74ac0cdf00 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.975+0000 7f74b2cf7640 1 -- 172.21.15.115:0/227381273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 msgr2=0x7f74ac0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.975+0000 7f74b2cf7640 1 --2- 172.21.15.115:0/227381273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 0x7f74ac0b4670 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f74ac066fc0 tx=0x7f74a402b910 comp rx=0 tx=0).stop 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.976+0000 7f74b2cf7640 1 -- 172.21.15.115:0/227381273 shutdown_connections 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.976+0000 7f74b2cf7640 1 --2- 172.21.15.115:0/227381273 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 0x7f74ac0b4670 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.976+0000 7f74b2cf7640 1 --2- 172.21.15.115:0/227381273 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac0b8610 0x7f74ac0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.976+0000 7f74b2cf7640 1 --2- 172.21.15.115:0/227381273 >> v2:172.21.15.142:3300/0 conn(0x7f74ac15cf90 0x7f74ac0d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.976+0000 7f74b2cf7640 1 -- 172.21.15.115:0/227381273 >> 172.21.15.115:0/227381273 conn(0x7f74ac065960 msgr2=0x7f74ac065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.976+0000 7f74b2cf7640 1 -- 172.21.15.115:0/227381273 shutdown_connections 2023-12-01T03:08:33.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.976+0000 7f74b2cf7640 1 -- 172.21.15.115:0/227381273 wait complete. 2023-12-01T03:08:33.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.977+0000 7f74b2cf7640 1 Processor -- start 2023-12-01T03:08:33.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.978+0000 7f74b2cf7640 1 -- start start 2023-12-01T03:08:33.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.978+0000 7f74b2cf7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0b8610 0x7f74ac159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.978+0000 7f74b2cf7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 0x7f74ac0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.978+0000 7f74b2cf7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac15cf90 0x7f74ac0c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74b2cf7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac1f3aa0 con 0x7f74ac0cdf00 2023-12-01T03:08:33.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74b2cf7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac1f3d80 con 0x7f74ac0b8610 2023-12-01T03:08:33.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74b2cf7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac1f4060 con 0x7f74ac15cf90 2023-12-01T03:08:33.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74abfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 0x7f74ac0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74abfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 0x7f74ac0ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49686/0 (socket says 172.21.15.115:49686) 2023-12-01T03:08:33.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74abfff640 1 -- 172.21.15.115:0/985413329 learned_addr learned my addr 172.21.15.115:0/985413329 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:33.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74b0a93640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0b8610 0x7f74ac159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74abfff640 1 -- 172.21.15.115:0/985413329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac15cf90 msgr2=0x7f74ac0c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74abfff640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac15cf90 0x7f74ac0c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74abfff640 1 -- 172.21.15.115:0/985413329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0b8610 msgr2=0x7f74ac159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:33.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74abfff640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0b8610 0x7f74ac159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:33.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74abfff640 1 -- 172.21.15.115:0/985413329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a40094d0 con 0x7f74ac0cdf00 2023-12-01T03:08:33.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.979+0000 7f74b0a93640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0b8610 0x7f74ac159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:33.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.980+0000 7f74abfff640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 0x7f74ac0ca650 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f749400c7c0 tx=0x7f749400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:33.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.980+0000 7f74a97fa640 1 -- 172.21.15.115:0/985413329 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494007820 con 0x7f74ac0cdf00 2023-12-01T03:08:33.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.980+0000 7f74a97fa640 1 -- 172.21.15.115:0/985413329 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7494007980 con 0x7f74ac0cdf00 2023-12-01T03:08:33.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.980+0000 7f74a97fa640 1 -- 172.21.15.115:0/985413329 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494007d10 con 0x7f74ac0cdf00 2023-12-01T03:08:33.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.980+0000 7f74b2cf7640 1 -- 172.21.15.115:0/985413329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac1f4340 con 0x7f74ac0cdf00 2023-12-01T03:08:33.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.980+0000 7f74b2cf7640 1 -- 172.21.15.115:0/985413329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac0c9ee0 con 0x7f74ac0cdf00 2023-12-01T03:08:33.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.981+0000 7f74b2cf7640 1 -- 172.21.15.115:0/985413329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac1f4860 con 0x7f74ac0cdf00 2023-12-01T03:08:33.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.982+0000 7f74a97fa640 1 -- 172.21.15.115:0/985413329 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f74940113f0 con 0x7f74ac0cdf00 2023-12-01T03:08:33.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.983+0000 7f74a97fa640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f748c06a580 0x7f748c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:33.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.983+0000 7f74a97fa640 1 -- 172.21.15.115:0/985413329 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f7494088df0 con 0x7f74ac0cdf00 2023-12-01T03:08:33.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.983+0000 7f74b0a93640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f748c06a580 0x7f748c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:33.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.984+0000 7f74b0a93640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f748c06a580 0x7f748c06ca40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f749c007f40 tx=0x7f749c007490 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:33.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:33.987+0000 7f74a97fa640 1 -- 172.21.15.115:0/985413329 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f74ac1f4860 con 0x7f74ac0cdf00 2023-12-01T03:08:34.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.247+0000 7f74b2cf7640 1 -- 172.21.15.115:0/985413329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f74ac1f4860 con 0x7f74ac0cdf00 2023-12-01T03:08:34.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.516+0000 7f74a97fa640 1 -- 172.21.15.115:0/985413329 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2987) v1 ==== 165+0+0 (secure 0 0 0) 0x7f74ac1f4860 con 0x7f74ac0cdf00 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 -- 172.21.15.115:0/985413329 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f748c06a580 msgr2=0x7f748c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f748c06a580 0x7f748c06ca40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f749c007f40 tx=0x7f749c007490 comp rx=0 tx=0).stop 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 -- 172.21.15.115:0/985413329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 msgr2=0x7f74ac0ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 0x7f74ac0ca650 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f749400c7c0 tx=0x7f749400cc80 comp rx=0 tx=0).stop 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 -- 172.21.15.115:0/985413329 shutdown_connections 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f748c06a580 0x7f748c06ca40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac15cf90 0x7f74ac0c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac0cdf00 0x7f74ac0ca650 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 --2- 172.21.15.115:0/985413329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0b8610 0x7f74ac159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 -- 172.21.15.115:0/985413329 >> 172.21.15.115:0/985413329 conn(0x7f74ac065960 msgr2=0x7f74ac0c6720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 -- 172.21.15.115:0/985413329 shutdown_connections 2023-12-01T03:08:34.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.520+0000 7f74b2cf7640 1 -- 172.21.15.115:0/985413329 wait complete. 2023-12-01T03:08:34.543 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-12-01T03:08:34.543 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-12-01T03:08:34.543 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-01T03:08:34.543 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-12-01T03:08:34.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.750+0000 7f7b3c938640 1 Processor -- start 2023-12-01T03:08:34.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.750+0000 7f7b3c938640 1 -- start start 2023-12-01T03:08:34.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.751+0000 7f7b3c938640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7b381584b0 0x7f7b380cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:34.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.751+0000 7f7b3c938640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b380cc180 con 0x7f7b3815afe0 2023-12-01T03:08:34.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.751+0000 7f7b3c938640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3815b550 con 0x7f7b3806b980 2023-12-01T03:08:34.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.751+0000 7f7b3c938640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3815b830 con 0x7f7b381584b0 2023-12-01T03:08:34.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.752+0000 7f7b36d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7b381584b0 0x7f7b380cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:34.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.752+0000 7f7b2ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7b3815afe0 0x7f7b3806b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51346/0 (socket says 172.21.15.115:51346) 2023-12-01T03:08:34.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.752+0000 7f7b2ffff640 1 -- 172.21.15.115:0/884207371 learned_addr learned my addr 172.21.15.115:0/884207371 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:34.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.752+0000 7f7b36d9d640 1 -- 172.21.15.115:0/884207371 >> v1:172.21.15.142:6789/0 conn(0x7f7b3806b980 legacy=0x7f7b380cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:34.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.752+0000 7f7b36d9d640 1 -- 172.21.15.115:0/884207371 >> v1:172.21.15.115:6789/0 conn(0x7f7b3815afe0 legacy=0x7f7b3806b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:34.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.752+0000 7f7b36d9d640 1 -- 172.21.15.115:0/884207371 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3815bb10 con 0x7f7b381584b0 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.753+0000 7f7b36d9d640 1 --2- 172.21.15.115:0/884207371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b381584b0 0x7f7b380cdd90 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f7b280097a0 tx=0x7f7b2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab24ea7b5cd8142 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.754+0000 7f7b3659c640 1 -- 172.21.15.115:0/884207371 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28031830 con 0x7f7b381584b0 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.754+0000 7f7b3659c640 1 -- 172.21.15.115:0/884207371 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b28031990 con 0x7f7b381584b0 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.754+0000 7f7b3659c640 1 -- 172.21.15.115:0/884207371 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28031d20 con 0x7f7b381584b0 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.754+0000 7f7b3c938640 1 -- 172.21.15.115:0/884207371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b381584b0 msgr2=0x7f7b380cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.754+0000 7f7b3c938640 1 --2- 172.21.15.115:0/884207371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b381584b0 0x7f7b380cdd90 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f7b280097a0 tx=0x7f7b2802b8d0 comp rx=0 tx=0).stop 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.754+0000 7f7b3c938640 1 -- 172.21.15.115:0/884207371 shutdown_connections 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.754+0000 7f7b3c938640 1 --2- 172.21.15.115:0/884207371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b381584b0 0x7f7b380cdd90 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.754+0000 7f7b3c938640 1 -- 172.21.15.115:0/884207371 >> 172.21.15.115:0/884207371 conn(0x7f7b380ce640 msgr2=0x7f7b380cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.754+0000 7f7b3c938640 1 -- 172.21.15.115:0/884207371 shutdown_connections 2023-12-01T03:08:34.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.755+0000 7f7b3c938640 1 -- 172.21.15.115:0/884207371 wait complete. 2023-12-01T03:08:34.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.756+0000 7f7b3c938640 1 Processor -- start 2023-12-01T03:08:34.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.756+0000 7f7b3c938640 1 -- start start 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.757+0000 7f7b3c938640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3806b980 0x7f7b381e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.757+0000 7f7b3c938640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b381584b0 0x7f7b381e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.757+0000 7f7b3c938640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b3815afe0 0x7f7b38149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b3c938640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381e3120 con 0x7f7b3806b980 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b3c938640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b38158ce0 con 0x7f7b3815afe0 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b3c938640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b38158fc0 con 0x7f7b381584b0 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b36d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3806b980 0x7f7b381e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b36d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3806b980 0x7f7b381e28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49700/0 (socket says 172.21.15.115:49700) 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b36d9d640 1 -- 172.21.15.115:0/1640871046 learned_addr learned my addr 172.21.15.115:0/1640871046 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b36d9d640 1 -- 172.21.15.115:0/1640871046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b381584b0 msgr2=0x7f7b381e4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:34.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b36d9d640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b381584b0 0x7f7b381e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:34.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b36d9d640 1 -- 172.21.15.115:0/1640871046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b3815afe0 msgr2=0x7f7b38149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:34.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b36d9d640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b3815afe0 0x7f7b38149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:34.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.758+0000 7f7b36d9d640 1 -- 172.21.15.115:0/1640871046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b28009400 con 0x7f7b3806b980 2023-12-01T03:08:34.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.759+0000 7f7b36d9d640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3806b980 0x7f7b381e28b0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f7b280098c0 tx=0x7f7b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:34.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.759+0000 7f7b2f7fe640 1 -- 172.21.15.115:0/1640871046 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2803ba70 con 0x7f7b3806b980 2023-12-01T03:08:34.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.759+0000 7f7b2f7fe640 1 -- 172.21.15.115:0/1640871046 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2803bbf0 con 0x7f7b3806b980 2023-12-01T03:08:34.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.759+0000 7f7b3c938640 1 -- 172.21.15.115:0/1640871046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b381592a0 con 0x7f7b3806b980 2023-12-01T03:08:34.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.759+0000 7f7b2f7fe640 1 -- 172.21.15.115:0/1640871046 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b280453f0 con 0x7f7b3806b980 2023-12-01T03:08:34.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.759+0000 7f7b3c938640 1 -- 172.21.15.115:0/1640871046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b381e2df0 con 0x7f7b3806b980 2023-12-01T03:08:34.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.760+0000 7f7b3c938640 1 -- 172.21.15.115:0/1640871046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b380673a0 con 0x7f7b3806b980 2023-12-01T03:08:34.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.761+0000 7f7b2f7fe640 1 -- 172.21.15.115:0/1640871046 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f7b2804e450 con 0x7f7b3806b980 2023-12-01T03:08:34.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.762+0000 7f7b2f7fe640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b1806a550 0x7f7b1806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:34.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.763+0000 7f7b2f7fe640 1 -- 172.21.15.115:0/1640871046 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f7b280ab000 con 0x7f7b3806b980 2023-12-01T03:08:34.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.763+0000 7f7b2ffff640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b1806a550 0x7f7b1806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:34.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.763+0000 7f7b2ffff640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b1806a550 0x7f7b1806ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b381e5420 tx=0x7f7b20007a40 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:34.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:34.766+0000 7f7b2f7fe640 1 -- 172.21.15.115:0/1640871046 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7b28076350 con 0x7f7b3806b980 2023-12-01T03:08:35.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.009+0000 7f7b3c938640 1 -- 172.21.15.115:0/1640871046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f7b381e86e0 con 0x7f7b3806b980 2023-12-01T03:08:35.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.519+0000 7f7b2f7fe640 1 -- 172.21.15.115:0/1640871046 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2988) v1 ==== 166+0+0 (secure 0 0 0) 0x7f7b28079970 con 0x7f7b3806b980 2023-12-01T03:08:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.522+0000 7f7b3c938640 1 -- 172.21.15.115:0/1640871046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b1806a550 msgr2=0x7f7b1806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b1806a550 0x7f7b1806ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b381e5420 tx=0x7f7b20007a40 comp rx=0 tx=0).stop 2023-12-01T03:08:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 -- 172.21.15.115:0/1640871046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3806b980 msgr2=0x7f7b381e28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3806b980 0x7f7b381e28b0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f7b280098c0 tx=0x7f7b280046c0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 -- 172.21.15.115:0/1640871046 shutdown_connections 2023-12-01T03:08:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b1806a550 0x7f7b1806ca10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b381584b0 0x7f7b381e4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b3815afe0 0x7f7b38149d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 --2- 172.21.15.115:0/1640871046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b3806b980 0x7f7b381e28b0 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 -- 172.21.15.115:0/1640871046 >> 172.21.15.115:0/1640871046 conn(0x7f7b380ce640 msgr2=0x7f7b381e0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:35.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 -- 172.21.15.115:0/1640871046 shutdown_connections 2023-12-01T03:08:35.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.523+0000 7f7b3c938640 1 -- 172.21.15.115:0/1640871046 wait complete. 2023-12-01T03:08:35.545 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T03:08:35.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.746+0000 7fd146599640 1 Processor -- start 2023-12-01T03:08:35.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.747+0000 7fd146599640 1 -- start start 2023-12-01T03:08:35.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.748+0000 7fd146599640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd1400cfd60 0x7fd1400cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:35.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.748+0000 7fd146599640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1400ce5e0 con 0x7fd1400b5fa0 2023-12-01T03:08:35.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.748+0000 7fd146599640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1400ce300 con 0x7fd14015cd90 2023-12-01T03:08:35.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.748+0000 7fd146599640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1400d0ec0 con 0x7fd1400cfd60 2023-12-01T03:08:35.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.748+0000 7fd144b36640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd1400b5fa0 0x7fd1400d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51356/0 (socket says 172.21.15.115:51356) 2023-12-01T03:08:35.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.748+0000 7fd144b36640 1 -- 172.21.15.115:0/249572237 learned_addr learned my addr 172.21.15.115:0/249572237 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.748+0000 7fd13f7fe640 1 --2- 172.21.15.115:0/249572237 >> v2:172.21.15.142:3300/0 conn(0x7fd1400cfd60 0x7fd1400cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.748+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092820139 0 0) 0x7fd138001110 con 0x7fd1400b5fa0 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.748+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd12c003520 con 0x7fd1400b5fa0 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1693969826 0 0) 0x7fd12c003520 con 0x7fd1400b5fa0 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 >> v2:172.21.15.142:3300/0 conn(0x7fd1400cfd60 msgr2=0x7fd1400cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 --2- 172.21.15.115:0/249572237 >> v2:172.21.15.142:3300/0 conn(0x7fd1400cfd60 0x7fd1400cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 >> v1:172.21.15.142:6789/0 conn(0x7fd14015cd90 legacy=0x7fd1400cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1400d10a0 con 0x7fd1400b5fa0 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd138001800 con 0x7fd1400b5fa0 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 >> v1:172.21.15.115:6789/0 conn(0x7fd1400b5fa0 legacy=0x7fd1400d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:35.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 --2- 172.21.15.115:0/249572237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd12c0039c0 0x7fd12c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 --2- 172.21.15.115:0/249572237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd12c0047c0 0x7fd12c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 --2- 172.21.15.115:0/249572237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd12c0051d0 0x7fd12c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400d0ec0 con 0x7fd12c0051d0 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c00c100 con 0x7fd12c0039c0 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c00c2c0 con 0x7fd12c0047c0 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd144b36640 1 --2- 172.21.15.115:0/249572237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd12c0051d0 0x7fd12c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.749+0000 7fd13ffff640 1 --2- 172.21.15.115:0/249572237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd12c0039c0 0x7fd12c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.750+0000 7fd144b36640 1 -- 172.21.15.115:0/249572237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd12c0047c0 msgr2=0x7fd12c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.750+0000 7fd144b36640 1 --2- 172.21.15.115:0/249572237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd12c0047c0 0x7fd12c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.750+0000 7fd144b36640 1 -- 172.21.15.115:0/249572237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd12c0039c0 msgr2=0x7fd12c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.750+0000 7fd144b36640 1 --2- 172.21.15.115:0/249572237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd12c0039c0 0x7fd12c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.750+0000 7fd144b36640 1 -- 172.21.15.115:0/249572237 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c003520 con 0x7fd12c0051d0 2023-12-01T03:08:35.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.750+0000 7fd144b36640 1 -- 172.21.15.115:0/249572237 reap_dead start 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.751+0000 7fd144b36640 1 --2- 172.21.15.115:0/249572237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd12c0051d0 0x7fd12c00b980 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fd13800c0e0 tx=0x7fd12c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcf0151cb862797 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.751+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138005ad0 con 0x7fd12c0051d0 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.751+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd138005c30 con 0x7fd12c0051d0 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.751+0000 7fd13effd640 1 -- 172.21.15.115:0/249572237 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13802c7c0 con 0x7fd12c0051d0 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.751+0000 7fd146599640 1 -- 172.21.15.115:0/249572237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd12c0051d0 msgr2=0x7fd12c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.751+0000 7fd146599640 1 --2- 172.21.15.115:0/249572237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd12c0051d0 0x7fd12c00b980 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fd13800c0e0 tx=0x7fd12c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.751+0000 7fd146599640 1 -- 172.21.15.115:0/249572237 shutdown_connections 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.751+0000 7fd146599640 1 --2- 172.21.15.115:0/249572237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd12c0051d0 0x7fd12c00b980 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.751+0000 7fd146599640 1 -- 172.21.15.115:0/249572237 >> 172.21.15.115:0/249572237 conn(0x7fd140065960 msgr2=0x7fd140065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.751+0000 7fd146599640 1 -- 172.21.15.115:0/249572237 shutdown_connections 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.752+0000 7fd146599640 1 -- 172.21.15.115:0/249572237 wait complete. 2023-12-01T03:08:35.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.753+0000 7fd146599640 1 Processor -- start 2023-12-01T03:08:35.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.753+0000 7fd146599640 1 -- start start 2023-12-01T03:08:35.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.754+0000 7fd146599640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd12c0051d0 0x7fd1401541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.754+0000 7fd146599640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1400cca60 0x7fd140153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.754+0000 7fd146599640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14015e640 0x7fd14015f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.754+0000 7fd146599640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400cb790 con 0x7fd14015e640 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.754+0000 7fd146599640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd140154030 con 0x7fd1400cca60 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.754+0000 7fd146599640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400ccee0 con 0x7fd12c0051d0 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd144b36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14015e640 0x7fd14015f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd144b36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14015e640 0x7fd14015f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49732/0 (socket says 172.21.15.115:49732) 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd144b36640 1 -- 172.21.15.115:0/2133024583 learned_addr learned my addr 172.21.15.115:0/2133024583 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd13ffff640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd12c0051d0 0x7fd1401541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd13f7fe640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1400cca60 0x7fd140153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd144b36640 1 -- 172.21.15.115:0/2133024583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd12c0051d0 msgr2=0x7fd1401541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd144b36640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd12c0051d0 0x7fd1401541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd144b36640 1 -- 172.21.15.115:0/2133024583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1400cca60 msgr2=0x7fd140153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:35.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd144b36640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1400cca60 0x7fd140153af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd144b36640 1 -- 172.21.15.115:0/2133024583 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1401549a0 con 0x7fd14015e640 2023-12-01T03:08:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd144b36640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14015e640 0x7fd14015f790 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fd13802c430 tx=0x7fd138001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.755+0000 7fd13cff9640 1 -- 172.21.15.115:0/2133024583 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13802c7c0 con 0x7fd14015e640 2023-12-01T03:08:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.756+0000 7fd13cff9640 1 -- 172.21.15.115:0/2133024583 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13802c940 con 0x7fd14015e640 2023-12-01T03:08:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.756+0000 7fd13cff9640 1 -- 172.21.15.115:0/2133024583 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13802ccf0 con 0x7fd14015e640 2023-12-01T03:08:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.756+0000 7fd146599640 1 -- 172.21.15.115:0/2133024583 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14015d5a0 con 0x7fd14015e640 2023-12-01T03:08:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.756+0000 7fd146599640 1 -- 172.21.15.115:0/2133024583 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14015db60 con 0x7fd14015e640 2023-12-01T03:08:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.757+0000 7fd146599640 1 -- 172.21.15.115:0/2133024583 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14004e890 con 0x7fd14015e640 2023-12-01T03:08:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.758+0000 7fd13cff9640 1 -- 172.21.15.115:0/2133024583 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd138032070 con 0x7fd14015e640 2023-12-01T03:08:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.758+0000 7fd13cff9640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd12006a580 0x7fd12006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.759+0000 7fd13cff9640 1 -- 172.21.15.115:0/2133024583 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5961+0+0 (secure 0 0 0) 0x7fd13803c070 con 0x7fd14015e640 2023-12-01T03:08:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.759+0000 7fd13ffff640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd12006a580 0x7fd12006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:35.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.759+0000 7fd13ffff640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd12006a580 0x7fd12006ca40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd130007fd0 tx=0x7fd130009040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:35.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.763+0000 7fd13cff9640 1 -- 172.21.15.115:0/2133024583 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd138074880 con 0x7fd14015e640 2023-12-01T03:08:35.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.953+0000 7fd146599640 1 -- 172.21.15.115:0/2133024583 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd1400ce5e0 con 0x7fd14015e640 2023-12-01T03:08:35.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.958+0000 7fd13cff9640 1 -- 172.21.15.115:0/2133024583 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1400ce5e0 con 0x7fd14015e640 2023-12-01T03:08:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 -- 172.21.15.115:0/2133024583 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd12006a580 msgr2=0x7fd12006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd12006a580 0x7fd12006ca40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd130007fd0 tx=0x7fd130009040 comp rx=0 tx=0).stop 2023-12-01T03:08:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 -- 172.21.15.115:0/2133024583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14015e640 msgr2=0x7fd14015f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14015e640 0x7fd14015f790 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fd13802c430 tx=0x7fd138001ac0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 -- 172.21.15.115:0/2133024583 shutdown_connections 2023-12-01T03:08:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd12006a580 0x7fd12006ca40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14015e640 0x7fd14015f790 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1400cca60 0x7fd140153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 --2- 172.21.15.115:0/2133024583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd12c0051d0 0x7fd1401541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:35.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 -- 172.21.15.115:0/2133024583 >> 172.21.15.115:0/2133024583 conn(0x7fd140065960 msgr2=0x7fd1400d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 -- 172.21.15.115:0/2133024583 shutdown_connections 2023-12-01T03:08:35.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:35.961+0000 7fd146599640 1 -- 172.21.15.115:0/2133024583 wait complete. 2023-12-01T03:08:35.981 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T03:08:35.981 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:08:36.099 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:08:36.229 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T03:08:36.229 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:08:36.230 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:08:36.294 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:08:36.294 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:08:36.295 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:08:36.356 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:08:36.356 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:08:36.356 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:08:36.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:08:36.416 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:08:36.416 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:08:36.476 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:08:36.476 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T03:08:36.476 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:08:36.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.693+0000 7f9645209640 1 Processor -- start 2023-12-01T03:08:36.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.693+0000 7f9645209640 1 -- start start 2023-12-01T03:08:36.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.695+0000 7f9645209640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f964015cd20 0x7f96400d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:36.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.695+0000 7f9645209640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f96400d1230 0x7f96400d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:36.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.695+0000 7f9645209640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96400ce2e0 con 0x7f96400b5d80 2023-12-01T03:08:36.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.695+0000 7f9645209640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400b2160 con 0x7f96400d1230 2023-12-01T03:08:36.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.695+0000 7f9645209640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96400d1be0 con 0x7f964015cd20 2023-12-01T03:08:36.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.695+0000 7f9637fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f96400b5d80 0x7f96400b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51370/0 (socket says 172.21.15.115:51370) 2023-12-01T03:08:36.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.695+0000 7f9637fff640 1 -- 172.21.15.115:0/1914213080 learned_addr learned my addr 172.21.15.115:0/1914213080 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499862338 0 0) 0x7f96400ce2e0 con 0x7f96400b5d80 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f962c003520 con 0x7f96400b5d80 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963ed9d640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f964015cd20 0x7f96400d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963f59e640 1 --2- 172.21.15.115:0/1914213080 >> v2:172.21.15.142:3300/0 conn(0x7f96400d1230 0x7f96400d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2312046108 0 0) 0x7f962c003520 con 0x7f96400b5d80 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f964015cd20 msgr2=0x7f96400d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963e59c640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f964015cd20 0x7f96400d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 >> v2:172.21.15.142:3300/0 conn(0x7f96400d1230 msgr2=0x7f96400d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963e59c640 1 --2- 172.21.15.115:0/1914213080 >> v2:172.21.15.142:3300/0 conn(0x7f96400d1230 0x7f96400d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400ce000 con 0x7f96400b5d80 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f9628001670 con 0x7f96400b5d80 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963f59e640 1 --2- 172.21.15.115:0/1914213080 >> v2:172.21.15.142:3300/0 conn(0x7f96400d1230 0x7f96400d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.696+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 >> v1:172.21.15.115:6789/0 conn(0x7f96400b5d80 legacy=0x7f96400b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:36.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963e59c640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c0039e0 0x7f962c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963e59c640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f964015cd20 0x7f962c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963e59c640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f962c004d00 0x7f962c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c00ba30 con 0x7f962c0039e0 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c00bb70 con 0x7f962c004d00 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c00bd00 con 0x7f964015cd20 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963ed9d640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c0039e0 0x7f962c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f9637fff640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f964015cd20 0x7f962c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963ed9d640 1 -- 172.21.15.115:0/1914213080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f964015cd20 msgr2=0x7f962c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963ed9d640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f964015cd20 0x7f962c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963ed9d640 1 -- 172.21.15.115:0/1914213080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f962c004d00 msgr2=0x7f962c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:36.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963ed9d640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f962c004d00 0x7f962c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.697+0000 7f963ed9d640 1 -- 172.21.15.115:0/1914213080 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c003520 con 0x7f962c0039e0 2023-12-01T03:08:36.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.699+0000 7f963ed9d640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c0039e0 0x7f962c003e00 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f9630009ac0 tx=0x7f962c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476ebcc1d09ae44d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:36.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.699+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963002c6b0 con 0x7f962c0039e0 2023-12-01T03:08:36.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.699+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963002c810 con 0x7f962c0039e0 2023-12-01T03:08:36.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.699+0000 7f963e59c640 1 -- 172.21.15.115:0/1914213080 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963002cba0 con 0x7f962c0039e0 2023-12-01T03:08:36.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.700+0000 7f9645209640 1 -- 172.21.15.115:0/1914213080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c0039e0 msgr2=0x7f962c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:36.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.700+0000 7f9645209640 1 --2- 172.21.15.115:0/1914213080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f962c0039e0 0x7f962c003e00 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f9630009ac0 tx=0x7f962c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.700+0000 7f963f59e640 1 -- 172.21.15.115:0/1914213080 reap_dead start 2023-12-01T03:08:36.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.700+0000 7f9645209640 1 -- 172.21.15.115:0/1914213080 shutdown_connections 2023-12-01T03:08:36.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.700+0000 7f9645209640 1 -- 172.21.15.115:0/1914213080 >> 172.21.15.115:0/1914213080 conn(0x7f964006a0e0 msgr2=0x7f964006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:36.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.700+0000 7f9645209640 1 -- 172.21.15.115:0/1914213080 shutdown_connections 2023-12-01T03:08:36.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.700+0000 7f9645209640 1 -- 172.21.15.115:0/1914213080 wait complete. 2023-12-01T03:08:36.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.702+0000 7f9645209640 1 Processor -- start 2023-12-01T03:08:36.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.702+0000 7f9645209640 1 -- start start 2023-12-01T03:08:36.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.703+0000 7f9645209640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f962c0039e0 0x7f96400b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:36.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.703+0000 7f9645209640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96401f6040 0x7f96401f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:36.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.703+0000 7f9645209640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9640145730 0x7f9640145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:36.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.703+0000 7f9645209640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96401f6a00 con 0x7f96401f6040 2023-12-01T03:08:36.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.703+0000 7f9645209640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964015fac0 con 0x7f9640145730 2023-12-01T03:08:36.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.703+0000 7f9645209640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96400c9940 con 0x7f962c0039e0 2023-12-01T03:08:36.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.703+0000 7f9637fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96401f6040 0x7f96401f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:36.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.703+0000 7f9637fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96401f6040 0x7f96401f64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49740/0 (socket says 172.21.15.115:49740) 2023-12-01T03:08:36.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.703+0000 7f9637fff640 1 -- 172.21.15.115:0/3188739216 learned_addr learned my addr 172.21.15.115:0/3188739216 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:36.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f963f59e640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9640145730 0x7f9640145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f963ed9d640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f962c0039e0 0x7f96400b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f9637fff640 1 -- 172.21.15.115:0/3188739216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f962c0039e0 msgr2=0x7f96400b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f9637fff640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f962c0039e0 0x7f96400b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f9637fff640 1 -- 172.21.15.115:0/3188739216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9640145730 msgr2=0x7f9640145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f9637fff640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9640145730 0x7f9640145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f9637fff640 1 -- 172.21.15.115:0/3188739216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964015f630 con 0x7f96401f6040 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f9637fff640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96401f6040 0x7f96401f64c0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f962800d7e0 tx=0x7f962800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f96377fe640 1 -- 172.21.15.115:0/3188739216 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96280074e0 con 0x7f96401f6040 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f96377fe640 1 -- 172.21.15.115:0/3188739216 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9628007640 con 0x7f96401f6040 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f96377fe640 1 -- 172.21.15.115:0/3188739216 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96280049e0 con 0x7f96401f6040 2023-12-01T03:08:36.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.704+0000 7f9645209640 1 -- 172.21.15.115:0/3188739216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96400b2840 con 0x7f96401f6040 2023-12-01T03:08:36.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.705+0000 7f9645209640 1 -- 172.21.15.115:0/3188739216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964015efe0 con 0x7f96401f6040 2023-12-01T03:08:36.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.706+0000 7f9645209640 1 -- 172.21.15.115:0/3188739216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964015f7c0 con 0x7f96401f6040 2023-12-01T03:08:36.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.707+0000 7f96377fe640 1 -- 172.21.15.115:0/3188739216 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9628004b40 con 0x7f96401f6040 2023-12-01T03:08:36.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.708+0000 7f96377fe640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f962006a550 0x7f962006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:36.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.708+0000 7f96377fe640 1 -- 172.21.15.115:0/3188739216 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f96280884b0 con 0x7f96401f6040 2023-12-01T03:08:36.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.708+0000 7f963ed9d640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f962006a550 0x7f962006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:36.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.709+0000 7f963ed9d640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f962006a550 0x7f962006ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9630036000 tx=0x7f9630030000 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:36.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.712+0000 7f96377fe640 1 -- 172.21.15.115:0/3188739216 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9628053780 con 0x7f96401f6040 2023-12-01T03:08:36.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.940+0000 7f9645209640 1 -- 172.21.15.115:0/3188739216 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f964004e890 con 0x7f96401f6040 2023-12-01T03:08:36.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.941+0000 7f96377fe640 1 -- 172.21.15.115:0/3188739216 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+3539 (secure 0 0 0) 0x7f9628019df0 con 0x7f96401f6040 2023-12-01T03:08:36.943 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:36.944 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":81,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1616304732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1616304732},{"type":"v1","addr":"172.21.15.115:6835","nonce":1616304732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6766,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/3109897397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3109897397},{"type":"v1","addr":"172.21.15.142:6834","nonce":3109897397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":6769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/446557816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":446557816},{"type":"v1","addr":"172.21.15.142:6835","nonce":446557816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":10053,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/2134519917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":2134519917},{"type":"v1","addr":"172.21.15.115:6837","nonce":2134519917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[]} 2023-12-01T03:08:36.944 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 81 2023-12-01T03:08:36.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.944+0000 7f9645209640 1 -- 172.21.15.115:0/3188739216 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f962006a550 msgr2=0x7f962006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:36.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.944+0000 7f9645209640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f962006a550 0x7f962006ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9630036000 tx=0x7f9630030000 comp rx=0 tx=0).stop 2023-12-01T03:08:36.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.944+0000 7f9645209640 1 -- 172.21.15.115:0/3188739216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96401f6040 msgr2=0x7f96401f64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:36.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.944+0000 7f9645209640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96401f6040 0x7f96401f64c0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f962800d7e0 tx=0x7f962800dca0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.944+0000 7f9645209640 1 -- 172.21.15.115:0/3188739216 shutdown_connections 2023-12-01T03:08:36.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.944+0000 7f9645209640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f962006a550 0x7f962006ca10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.944+0000 7f9645209640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9640145730 0x7f9640145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.944+0000 7f9645209640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96401f6040 0x7f96401f64c0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.944+0000 7f9645209640 1 --2- 172.21.15.115:0/3188739216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f962c0039e0 0x7f96400b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:36.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.944+0000 7f9645209640 1 -- 172.21.15.115:0/3188739216 >> 172.21.15.115:0/3188739216 conn(0x7f964006a0e0 msgr2=0x7f96400cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:36.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.945+0000 7f9645209640 1 -- 172.21.15.115:0/3188739216 shutdown_connections 2023-12-01T03:08:36.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:36.945+0000 7f9645209640 1 -- 172.21.15.115:0/3188739216 wait complete. 2023-12-01T03:08:36.966 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T03:08:36.967 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T03:08:36.967 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T03:08:36.967 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:08:36.967 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T03:08:36.967 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T03:08:36.967 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T03:08:36.968 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:08:36.968 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T03:08:36.968 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T03:08:36.968 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T03:08:36.968 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:08:36.968 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T03:08:36.968 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T03:08:36.968 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T03:08:36.969 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:08:36.987 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T03:08:36.987 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T03:08:36.988 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T03:08:36.988 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T03:08:36.989 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T03:08:36.989 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T03:08:36.990 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T03:08:36.990 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T03:08:37.035 INFO:tasks.ceph.mds.b:Started 2023-12-01T03:08:37.036 DEBUG:teuthology.parallel:result is None 2023-12-01T03:08:37.036 INFO:tasks.ceph.mds.a:Started 2023-12-01T03:08:37.036 DEBUG:teuthology.parallel:result is None 2023-12-01T03:08:37.042 INFO:tasks.ceph.mds.d:Started 2023-12-01T03:08:37.042 DEBUG:teuthology.parallel:result is None 2023-12-01T03:08:37.043 INFO:tasks.ceph.mds.c:Started 2023-12-01T03:08:37.043 DEBUG:teuthology.parallel:result is None 2023-12-01T03:08:37.043 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T03:08:37.197 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T03:08:37.204 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T03:08:37.210 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T03:08:37.211 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T03:08:37.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.240+0000 7fc5238ce640 1 Processor -- start 2023-12-01T03:08:37.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.240+0000 7fc5238ce640 1 -- start start 2023-12-01T03:08:37.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc5238ce640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc51c15e320 0x7fc51c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc5238ce640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc51c0c5180 0x7fc51c0c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc5238ce640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51c06c440 con 0x7fc51c159890 2023-12-01T03:08:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc5238ce640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51c0c5b30 con 0x7fc51c15e320 2023-12-01T03:08:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc5238ce640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51c06ccd0 con 0x7fc51c0c5180 2023-12-01T03:08:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc520e69640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc51c15e320 0x7fc51c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc520e69640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc51c15e320 0x7fc51c06d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49784/0 (socket says 172.21.15.115:49784) 2023-12-01T03:08:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc520e69640 1 -- 172.21.15.115:0/3314721439 learned_addr learned my addr 172.21.15.115:0/3314721439 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:37.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc521e6b640 1 --2- 172.21.15.115:0/3314721439 >> v2:172.21.15.142:3300/0 conn(0x7fc51c0c5180 0x7fc51c0c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:37.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc520e69640 1 -- 172.21.15.115:0/3314721439 >> v2:172.21.15.142:3300/0 conn(0x7fc51c0c5180 msgr2=0x7fc51c0c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:37.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc520e69640 1 --2- 172.21.15.115:0/3314721439 >> v2:172.21.15.142:3300/0 conn(0x7fc51c0c5180 0x7fc51c0c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:37.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc520e69640 1 -- 172.21.15.115:0/3314721439 >> v1:172.21.15.142:6789/0 conn(0x7fc51c159890 legacy=0x7fc51c0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:37.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.242+0000 7fc520e69640 1 -- 172.21.15.115:0/3314721439 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c06d6a0 con 0x7fc51c15e320 2023-12-01T03:08:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.243+0000 7fc520e69640 1 --2- 172.21.15.115:0/3314721439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c15e320 0x7fc51c06d160 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0710e0 tx=0x7fc50c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a49dfb006e6683 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.244+0000 7fc513fff640 1 -- 172.21.15.115:0/3314721439 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c030860 con 0x7fc51c15e320 2023-12-01T03:08:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.244+0000 7fc513fff640 1 -- 172.21.15.115:0/3314721439 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c0309c0 con 0x7fc51c15e320 2023-12-01T03:08:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.244+0000 7fc513fff640 1 -- 172.21.15.115:0/3314721439 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c030d50 con 0x7fc51c15e320 2023-12-01T03:08:37.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.244+0000 7fc5238ce640 1 -- 172.21.15.115:0/3314721439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c15e320 msgr2=0x7fc51c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:37.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.244+0000 7fc5238ce640 1 --2- 172.21.15.115:0/3314721439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c15e320 0x7fc51c06d160 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0710e0 tx=0x7fc50c02b9a0 comp rx=0 tx=0).stop 2023-12-01T03:08:37.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.244+0000 7fc5238ce640 1 -- 172.21.15.115:0/3314721439 shutdown_connections 2023-12-01T03:08:37.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.244+0000 7fc5238ce640 1 --2- 172.21.15.115:0/3314721439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c15e320 0x7fc51c06d160 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:37.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.244+0000 7fc5238ce640 1 --2- 172.21.15.115:0/3314721439 >> v2:172.21.15.142:3300/0 conn(0x7fc51c0c5180 0x7fc51c0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:37.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.244+0000 7fc5238ce640 1 -- 172.21.15.115:0/3314721439 >> 172.21.15.115:0/3314721439 conn(0x7fc51c069cc0 msgr2=0x7fc51c06a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:37.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.245+0000 7fc5238ce640 1 -- 172.21.15.115:0/3314721439 shutdown_connections 2023-12-01T03:08:37.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.245+0000 7fc5238ce640 1 -- 172.21.15.115:0/3314721439 wait complete. 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.246+0000 7fc5238ce640 1 Processor -- start 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.246+0000 7fc5238ce640 1 -- start start 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.247+0000 7fc5238ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c0c5180 0x7fc51c1f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.247+0000 7fc5238ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc51c159890 0x7fc51c1f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.247+0000 7fc5238ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c0b2600 0x7fc51c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.247+0000 7fc5238ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c0b2fd0 con 0x7fc51c0c5180 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.247+0000 7fc5238ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c15acc0 con 0x7fc51c0b2600 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.247+0000 7fc5238ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c1f3b70 con 0x7fc51c159890 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc52166a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c0c5180 0x7fc51c1f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc52166a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c0c5180 0x7fc51c1f3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:49786/0 (socket says 172.21.15.115:49786) 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc52166a640 1 -- 172.21.15.115:0/3391579455 learned_addr learned my addr 172.21.15.115:0/3391579455 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc520e69640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc51c159890 0x7fc51c1f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc521e6b640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c0b2600 0x7fc51c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc52166a640 1 -- 172.21.15.115:0/3391579455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc51c159890 msgr2=0x7fc51c1f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc52166a640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc51c159890 0x7fc51c1f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc52166a640 1 -- 172.21.15.115:0/3391579455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c0b2600 msgr2=0x7fc51c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc52166a640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c0b2600 0x7fc51c0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc52166a640 1 -- 172.21.15.115:0/3391579455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c0094d0 con 0x7fc51c0c5180 2023-12-01T03:08:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.248+0000 7fc52166a640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c0c5180 0x7fc51c1f3630 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fc51800c450 tx=0x7fc51800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.249+0000 7fc511ffb640 1 -- 172.21.15.115:0/3391579455 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5180052d0 con 0x7fc51c0c5180 2023-12-01T03:08:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.249+0000 7fc511ffb640 1 -- 172.21.15.115:0/3391579455 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc518005430 con 0x7fc51c0c5180 2023-12-01T03:08:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.249+0000 7fc511ffb640 1 -- 172.21.15.115:0/3391579455 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518012870 con 0x7fc51c0c5180 2023-12-01T03:08:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.249+0000 7fc5238ce640 1 -- 172.21.15.115:0/3391579455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c0b5630 con 0x7fc51c0c5180 2023-12-01T03:08:37.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.249+0000 7fc5238ce640 1 -- 172.21.15.115:0/3391579455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c15b1f0 con 0x7fc51c0c5180 2023-12-01T03:08:37.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.250+0000 7fc5238ce640 1 -- 172.21.15.115:0/3391579455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c04e890 con 0x7fc51c0c5180 2023-12-01T03:08:37.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.251+0000 7fc511ffb640 1 -- 172.21.15.115:0/3391579455 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc5180129d0 con 0x7fc51c0c5180 2023-12-01T03:08:37.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.252+0000 7fc511ffb640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc50006a5a0 0x7fc50006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:37.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.252+0000 7fc511ffb640 1 -- 172.21.15.115:0/3391579455 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5933+0+0 (secure 0 0 0) 0x7fc518088580 con 0x7fc51c0c5180 2023-12-01T03:08:37.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.252+0000 7fc520e69640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc50006a5a0 0x7fc50006ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:37.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.253+0000 7fc520e69640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc50006a5a0 0x7fc50006ca60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc50c036000 tx=0x7fc50c03b040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:37.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.256+0000 7fc511ffb640 1 -- 172.21.15.115:0/3391579455 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc518053960 con 0x7fc51c0c5180 2023-12-01T03:08:37.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:37.460+0000 7fc5238ce640 1 -- 172.21.15.115:0/3391579455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc51c06c440 con 0x7fc51c0c5180 2023-12-01T03:08:38.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.365+0000 7fc511ffb640 1 -- 172.21.15.115:0/3391579455 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v192) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc51c06c440 con 0x7fc51c0c5180 2023-12-01T03:08:38.367 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.368+0000 7fc5238ce640 1 -- 172.21.15.115:0/3391579455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc50006a5a0 msgr2=0x7fc50006ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc50006a5a0 0x7fc50006ca60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc50c036000 tx=0x7fc50c03b040 comp rx=0 tx=0).stop 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 -- 172.21.15.115:0/3391579455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c0c5180 msgr2=0x7fc51c1f3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c0c5180 0x7fc51c1f3630 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fc51800c450 tx=0x7fc51800c910 comp rx=0 tx=0).stop 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 -- 172.21.15.115:0/3391579455 shutdown_connections 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc50006a5a0 0x7fc50006ca60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc51c159890 0x7fc51c1f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc51c0b2600 0x7fc51c0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 --2- 172.21.15.115:0/3391579455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc51c0c5180 0x7fc51c1f3630 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 -- 172.21.15.115:0/3391579455 >> 172.21.15.115:0/3391579455 conn(0x7fc51c069cc0 msgr2=0x7fc51c0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 -- 172.21.15.115:0/3391579455 shutdown_connections 2023-12-01T03:08:38.371 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.369+0000 7fc5238ce640 1 -- 172.21.15.115:0/3391579455 wait complete. 2023-12-01T03:08:38.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T03:08:38.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.596+0000 7fa8d7900640 1 Processor -- start 2023-12-01T03:08:38.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.596+0000 7fa8d7900640 1 -- start start 2023-12-01T03:08:38.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.598+0000 7fa8d7900640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa8d0159dd0 0x7fa8d00c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:38.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.598+0000 7fa8d7900640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa8d0070d30 0x7fa8d00c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:38.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.598+0000 7fa8d7900640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d006f8b0 con 0x7fa8d006b6f0 2023-12-01T03:08:38.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.598+0000 7fa8d7900640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d00adcb0 con 0x7fa8d0070d30 2023-12-01T03:08:38.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.598+0000 7fa8d7900640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d00ad9d0 con 0x7fa8d0159dd0 2023-12-01T03:08:38.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.598+0000 7fa8d5e9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa8d0070d30 0x7fa8d00c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:38.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.598+0000 7fa8d5e9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa8d0070d30 0x7fa8d00c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51474/0 (socket says 172.21.15.115:51474) 2023-12-01T03:08:38.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.598+0000 7fa8d5e9d640 1 -- 172.21.15.115:0/3755343194 learned_addr learned my addr 172.21.15.115:0/3755343194 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:38.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.599+0000 7fa8d569c640 1 --2- 172.21.15.115:0/3755343194 >> v2:172.21.15.142:3300/0 conn(0x7fa8d0159dd0 0x7fa8d00c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.599+0000 7fa8d5e9d640 1 -- 172.21.15.115:0/3755343194 >> v2:172.21.15.142:3300/0 conn(0x7fa8d0159dd0 msgr2=0x7fa8d00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.599+0000 7fa8d5e9d640 1 --2- 172.21.15.115:0/3755343194 >> v2:172.21.15.142:3300/0 conn(0x7fa8d0159dd0 0x7fa8d00c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.599+0000 7fa8d5e9d640 1 -- 172.21.15.115:0/3755343194 >> v1:172.21.15.142:6789/0 conn(0x7fa8d006b6f0 legacy=0x7fa8d00cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.599+0000 7fa8d5e9d640 1 -- 172.21.15.115:0/3755343194 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00c9000 con 0x7fa8d0070d30 2023-12-01T03:08:38.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.600+0000 7fa8d5e9d640 1 --2- 172.21.15.115:0/3755343194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d0070d30 0x7fa8d00c9d20 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00cf4e0 tx=0x7fa8c802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114cf4a3e16ef6ce server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.600+0000 7fa8c7fff640 1 -- 172.21.15.115:0/3755343194 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c802c960 con 0x7fa8d0070d30 2023-12-01T03:08:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.600+0000 7fa8c7fff640 1 -- 172.21.15.115:0/3755343194 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c802cac0 con 0x7fa8d0070d30 2023-12-01T03:08:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.600+0000 7fa8c7fff640 1 -- 172.21.15.115:0/3755343194 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c802ce50 con 0x7fa8d0070d30 2023-12-01T03:08:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.601+0000 7fa8d7900640 1 -- 172.21.15.115:0/3755343194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d0070d30 msgr2=0x7fa8d00c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.601+0000 7fa8d7900640 1 --2- 172.21.15.115:0/3755343194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d0070d30 0x7fa8d00c9d20 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00cf4e0 tx=0x7fa8c802bdc0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.601+0000 7fa8d7900640 1 -- 172.21.15.115:0/3755343194 shutdown_connections 2023-12-01T03:08:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.601+0000 7fa8d7900640 1 --2- 172.21.15.115:0/3755343194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d0070d30 0x7fa8d00c9d20 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.601+0000 7fa8d7900640 1 --2- 172.21.15.115:0/3755343194 >> v2:172.21.15.142:3300/0 conn(0x7fa8d0159dd0 0x7fa8d00c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.601+0000 7fa8d7900640 1 -- 172.21.15.115:0/3755343194 >> 172.21.15.115:0/3755343194 conn(0x7fa8d00ce880 msgr2=0x7fa8d00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.601+0000 7fa8d7900640 1 -- 172.21.15.115:0/3755343194 shutdown_connections 2023-12-01T03:08:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.601+0000 7fa8d7900640 1 -- 172.21.15.115:0/3755343194 wait complete. 2023-12-01T03:08:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.603+0000 7fa8d7900640 1 Processor -- start 2023-12-01T03:08:38.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.603+0000 7fa8d7900640 1 -- start start 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.604+0000 7fa8d7900640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d006b6f0 0x7fa8d00c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.604+0000 7fa8d7900640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8d0070d30 0x7fa8d00c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.604+0000 7fa8d7900640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8d0159dd0 0x7fa8d00ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.604+0000 7fa8d7900640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d01f46c0 con 0x7fa8d006b6f0 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.604+0000 7fa8d7900640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d01f3a40 con 0x7fa8d0070d30 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.604+0000 7fa8d7900640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d01f3d20 con 0x7fa8d0159dd0 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d569c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d006b6f0 0x7fa8d00c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d569c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d006b6f0 0x7fa8d00c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51488/0 (socket says 172.21.15.115:51488) 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d569c640 1 -- 172.21.15.115:0/70596128 learned_addr learned my addr 172.21.15.115:0/70596128 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d4e9b640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8d0070d30 0x7fa8d00c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:38.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d5e9d640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8d0159dd0 0x7fa8d00ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:38.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d569c640 1 -- 172.21.15.115:0/70596128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8d0159dd0 msgr2=0x7fa8d00ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:38.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d569c640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8d0159dd0 0x7fa8d00ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d569c640 1 -- 172.21.15.115:0/70596128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8d0070d30 msgr2=0x7fa8d00c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:38.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d569c640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8d0070d30 0x7fa8d00c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d569c640 1 -- 172.21.15.115:0/70596128 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8009b30 con 0x7fa8d006b6f0 2023-12-01T03:08:38.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.605+0000 7fa8d569c640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d006b6f0 0x7fa8d00c4df0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00d5f0 tx=0x7fa8cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:38.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.606+0000 7fa8c5ffb640 1 -- 172.21.15.115:0/70596128 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc0125f0 con 0x7fa8d006b6f0 2023-12-01T03:08:38.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.606+0000 7fa8c5ffb640 1 -- 172.21.15.115:0/70596128 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc012750 con 0x7fa8d006b6f0 2023-12-01T03:08:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.606+0000 7fa8d7900640 1 -- 172.21.15.115:0/70596128 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d01f4000 con 0x7fa8d006b6f0 2023-12-01T03:08:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.606+0000 7fa8c5ffb640 1 -- 172.21.15.115:0/70596128 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc012b00 con 0x7fa8d006b6f0 2023-12-01T03:08:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.606+0000 7fa8d7900640 1 -- 172.21.15.115:0/70596128 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d00c5330 con 0x7fa8d006b6f0 2023-12-01T03:08:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.607+0000 7fa8d7900640 1 -- 172.21.15.115:0/70596128 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d004e890 con 0x7fa8d006b6f0 2023-12-01T03:08:38.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.608+0000 7fa8c5ffb640 1 -- 172.21.15.115:0/70596128 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa8cc012c60 con 0x7fa8d006b6f0 2023-12-01T03:08:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.609+0000 7fa8c5ffb640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa8b406a550 0x7fa8b406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.609+0000 7fa8c5ffb640 1 -- 172.21.15.115:0/70596128 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fa8cc088c60 con 0x7fa8d006b6f0 2023-12-01T03:08:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.609+0000 7fa8d4e9b640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa8b406a550 0x7fa8b406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:38.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.610+0000 7fa8d4e9b640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa8b406a550 0x7fa8b406ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa8d01f3900 tx=0x7fa8c0009040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:38.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.613+0000 7fa8c5ffb640 1 -- 172.21.15.115:0/70596128 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa8cc0543c0 con 0x7fa8d006b6f0 2023-12-01T03:08:38.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.874+0000 7fa8d7900640 1 -- 172.21.15.115:0/70596128 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa8d01f4930 con 0x7fa8d006b6f0 2023-12-01T03:08:38.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.876+0000 7fa8c5ffb640 1 -- 172.21.15.115:0/70596128 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v46) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa8d01f4930 con 0x7fa8d006b6f0 2023-12-01T03:08:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:38.878 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:08:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T03:08:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T03:08:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T03:08:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:08:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:08:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:08:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T03:08:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T03:08:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:08:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T03:08:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T03:08:38.891 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:08:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 -- 172.21.15.115:0/70596128 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa8b406a550 msgr2=0x7fa8b406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:38.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa8b406a550 0x7fa8b406ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa8d01f3900 tx=0x7fa8c0009040 comp rx=0 tx=0).stop 2023-12-01T03:08:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 -- 172.21.15.115:0/70596128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d006b6f0 msgr2=0x7fa8d00c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d006b6f0 0x7fa8d00c4df0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00d5f0 tx=0x7fa8cc00dab0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 -- 172.21.15.115:0/70596128 shutdown_connections 2023-12-01T03:08:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa8b406a550 0x7fa8b406ca10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa8d0159dd0 0x7fa8d00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8d0070d30 0x7fa8d00c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 --2- 172.21.15.115:0/70596128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8d006b6f0 0x7fa8d00c4df0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 -- 172.21.15.115:0/70596128 >> 172.21.15.115:0/70596128 conn(0x7fa8d00ce880 msgr2=0x7fa8d015a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.878+0000 7fa8d7900640 1 -- 172.21.15.115:0/70596128 shutdown_connections 2023-12-01T03:08:38.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:38.879+0000 7fa8d7900640 1 -- 172.21.15.115:0/70596128 wait complete. 2023-12-01T03:08:38.899 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:08:39.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.107+0000 7fb7ec92e640 1 Processor -- start 2023-12-01T03:08:39.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.107+0000 7fb7ec92e640 1 -- start start 2023-12-01T03:08:39.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.109+0000 7fb7ec92e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb7e806b6f0 0x7fb7e8070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.109+0000 7fb7ec92e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e806f8b0 con 0x7fb7e8159dd0 2023-12-01T03:08:39.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.109+0000 7fb7ec92e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e80adcb0 con 0x7fb7e80c9ef0 2023-12-01T03:08:39.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.109+0000 7fb7ec92e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e80ad9d0 con 0x7fb7e806b6f0 2023-12-01T03:08:39.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.109+0000 7fb7e659c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb7e8159dd0 0x7fb7e80c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38210/0 (socket says 172.21.15.115:38210) 2023-12-01T03:08:39.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.109+0000 7fb7e659c640 1 -- 172.21.15.115:0/1557179049 learned_addr learned my addr 172.21.15.115:0/1557179049 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:39.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.109+0000 7fb7e6d9d640 1 --2- 172.21.15.115:0/1557179049 >> v2:172.21.15.142:3300/0 conn(0x7fb7e806b6f0 0x7fb7e8070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.109+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593261014 0 0) 0x7fb7e806f8b0 con 0x7fb7e8159dd0 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.109+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d0003520 con 0x7fb7e8159dd0 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 850794400 0 0) 0x7fb7d0003520 con 0x7fb7e8159dd0 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 >> v2:172.21.15.142:3300/0 conn(0x7fb7e806b6f0 msgr2=0x7fb7e8070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 --2- 172.21.15.115:0/1557179049 >> v2:172.21.15.142:3300/0 conn(0x7fb7e806b6f0 0x7fb7e8070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 >> v1:172.21.15.142:6789/0 conn(0x7fb7e80c9ef0 legacy=0x7fb7e806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e80c9000 con 0x7fb7e8159dd0 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb7d8001670 con 0x7fb7e8159dd0 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 >> v1:172.21.15.115:6789/0 conn(0x7fb7e8159dd0 legacy=0x7fb7e80c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7d00039e0 0x7fb7d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7d00047c0 0x7fb7d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80ad9d0 con 0x7fb7d0005380 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d000c260 con 0x7fb7d00047c0 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d000e630 con 0x7fb7d00039e0 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e6d9d640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e5d9b640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7d00047c0 0x7fb7d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.110+0000 7fb7e659c640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7d00039e0 0x7fb7d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.111+0000 7fb7e6d9d640 1 -- 172.21.15.115:0/1557179049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7d00039e0 msgr2=0x7fb7d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.111+0000 7fb7e6d9d640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7d00039e0 0x7fb7d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.111+0000 7fb7e6d9d640 1 -- 172.21.15.115:0/1557179049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7d00047c0 msgr2=0x7fb7d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.111+0000 7fb7e6d9d640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7d00047c0 0x7fb7d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.111+0000 7fb7e6d9d640 1 -- 172.21.15.115:0/1557179049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d0003520 con 0x7fb7d0005380 2023-12-01T03:08:39.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.111+0000 7fb7e659c640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7d00039e0 0x7fb7d0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:39.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.112+0000 7fb7e6d9d640 1 -- 172.21.15.115:0/1557179049 reap_dead start 2023-12-01T03:08:39.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.112+0000 7fb7e6d9d640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7d000b980 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000ba30 tx=0x7fb7d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d220e103935631 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:39.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.112+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0004020 con 0x7fb7d0005380 2023-12-01T03:08:39.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.112+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e00041c0 con 0x7fb7d0005380 2023-12-01T03:08:39.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.112+0000 7fb7e559a640 1 -- 172.21.15.115:0/1557179049 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0003610 con 0x7fb7d0005380 2023-12-01T03:08:39.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.113+0000 7fb7ec92e640 1 -- 172.21.15.115:0/1557179049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 msgr2=0x7fb7d000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.113+0000 7fb7ec92e640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7d000b980 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000ba30 tx=0x7fb7d00057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.113+0000 7fb7ec92e640 1 -- 172.21.15.115:0/1557179049 shutdown_connections 2023-12-01T03:08:39.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.113+0000 7fb7ec92e640 1 --2- 172.21.15.115:0/1557179049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7d000b980 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.113+0000 7fb7ec92e640 1 -- 172.21.15.115:0/1557179049 >> 172.21.15.115:0/1557179049 conn(0x7fb7e80ce880 msgr2=0x7fb7e80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:39.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.113+0000 7fb7ec92e640 1 -- 172.21.15.115:0/1557179049 shutdown_connections 2023-12-01T03:08:39.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.113+0000 7fb7ec92e640 1 -- 172.21.15.115:0/1557179049 wait complete. 2023-12-01T03:08:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.115+0000 7fb7ec92e640 1 Processor -- start 2023-12-01T03:08:39.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.115+0000 7fb7ec92e640 1 -- start start 2023-12-01T03:08:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7ec92e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7e815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7ec92e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7e814cd00 0x7fb7e8070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7ec92e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7e80711b0 0x7fb7e80c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7ec92e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8071630 con 0x7fb7d0005380 2023-12-01T03:08:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7ec92e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e815ae80 con 0x7fb7e80711b0 2023-12-01T03:08:39.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7ec92e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e815b850 con 0x7fb7e814cd00 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7e659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7e815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7e659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7e815b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51498/0 (socket says 172.21.15.115:51498) 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7e659c640 1 -- 172.21.15.115:0/573372396 learned_addr learned my addr 172.21.15.115:0/573372396 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7e5d9b640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7e814cd00 0x7fb7e8070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7e659c640 1 -- 172.21.15.115:0/573372396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7e814cd00 msgr2=0x7fb7e8070c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7e659c640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7e814cd00 0x7fb7e8070c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7e6d9d640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7e80711b0 0x7fb7e80c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7e659c640 1 -- 172.21.15.115:0/573372396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7e80711b0 msgr2=0x7fb7e80c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.116+0000 7fb7e659c640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7e80711b0 0x7fb7e80c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.117+0000 7fb7e659c640 1 -- 172.21.15.115:0/573372396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e80c5190 con 0x7fb7d0005380 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.117+0000 7fb7e5d9b640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7e814cd00 0x7fb7e8070c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.117+0000 7fb7e659c640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7e815b310 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800d7c0 tx=0x7fb7d800dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:39.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.117+0000 7fb7d6ffd640 1 -- 172.21.15.115:0/573372396 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80074e0 con 0x7fb7d0005380 2023-12-01T03:08:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.117+0000 7fb7d6ffd640 1 -- 172.21.15.115:0/573372396 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d8007640 con 0x7fb7d0005380 2023-12-01T03:08:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.117+0000 7fb7ec92e640 1 -- 172.21.15.115:0/573372396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e815a550 con 0x7fb7d0005380 2023-12-01T03:08:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.117+0000 7fb7d6ffd640 1 -- 172.21.15.115:0/573372396 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8004a00 con 0x7fb7d0005380 2023-12-01T03:08:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.117+0000 7fb7ec92e640 1 -- 172.21.15.115:0/573372396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e815ab10 con 0x7fb7d0005380 2023-12-01T03:08:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.119+0000 7fb7ec92e640 1 -- 172.21.15.115:0/573372396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e804e890 con 0x7fb7d0005380 2023-12-01T03:08:39.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.119+0000 7fb7d6ffd640 1 -- 172.21.15.115:0/573372396 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb7d8004b60 con 0x7fb7d0005380 2023-12-01T03:08:39.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.120+0000 7fb7d6ffd640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb7c806a550 0x7fb7c806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.120+0000 7fb7d6ffd640 1 -- 172.21.15.115:0/573372396 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fb7d8088500 con 0x7fb7d0005380 2023-12-01T03:08:39.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.121+0000 7fb7e5d9b640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb7c806a550 0x7fb7c806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.121+0000 7fb7e5d9b640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb7c806a550 0x7fb7c806ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc007fd0 tx=0x7fb7dc009040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:39.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.125+0000 7fb7d6ffd640 1 -- 172.21.15.115:0/573372396 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb7d8053bf0 con 0x7fb7d0005380 2023-12-01T03:08:39.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.313+0000 7fb7ec92e640 1 -- 172.21.15.115:0/573372396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7e80ae8f0 con 0x7fb7d0005380 2023-12-01T03:08:39.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.316+0000 7fb7d6ffd640 1 -- 172.21.15.115:0/573372396 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+12517 (secure 0 0 0) 0x7fb7d8019090 con 0x7fb7d0005380 2023-12-01T03:08:39.318 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:39.318 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":192,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:08:38.356532+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:08:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.318+0000 7fb7ec92e640 1 -- 172.21.15.115:0/573372396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb7c806a550 msgr2=0x7fb7c806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.318+0000 7fb7ec92e640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb7c806a550 0x7fb7c806ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc007fd0 tx=0x7fb7dc009040 comp rx=0 tx=0).stop 2023-12-01T03:08:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.318+0000 7fb7ec92e640 1 -- 172.21.15.115:0/573372396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 msgr2=0x7fb7e815b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.318+0000 7fb7ec92e640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7e815b310 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800d7c0 tx=0x7fb7d800dc80 comp rx=0 tx=0).stop 2023-12-01T03:08:39.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.318+0000 7fb7ec92e640 1 -- 172.21.15.115:0/573372396 shutdown_connections 2023-12-01T03:08:39.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.318+0000 7fb7ec92e640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb7c806a550 0x7fb7c806ca10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.318+0000 7fb7ec92e640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7e814cd00 0x7fb7e8070c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.318+0000 7fb7ec92e640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7e80711b0 0x7fb7e80c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.318+0000 7fb7ec92e640 1 --2- 172.21.15.115:0/573372396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7d0005380 0x7fb7e815b310 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.318+0000 7fb7ec92e640 1 -- 172.21.15.115:0/573372396 >> 172.21.15.115:0/573372396 conn(0x7fb7e80ce880 msgr2=0x7fb7e8158de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:39.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.319+0000 7fb7ec92e640 1 -- 172.21.15.115:0/573372396 shutdown_connections 2023-12-01T03:08:39.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.319+0000 7fb7ec92e640 1 -- 172.21.15.115:0/573372396 wait complete. 2023-12-01T03:08:39.340 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:08:39.340 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:08:39.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.555+0000 7fb0bcd73640 1 Processor -- start 2023-12-01T03:08:39.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.555+0000 7fb0bcd73640 1 -- start start 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.556+0000 7fb0bcd73640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0b815cf70 0x7fb0b80b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.556+0000 7fb0bcd73640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb0b80d0260 0x7fb0b80b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.556+0000 7fb0bcd73640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b80b7760 con 0x7fb0b80b18f0 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.556+0000 7fb0bcd73640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b80b2170 con 0x7fb0b80d0260 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.556+0000 7fb0bcd73640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80b8b70 con 0x7fb0b815cf70 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.557+0000 7fb0add9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb0b80b18f0 0x7fb0b80cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38220/0 (socket says 172.21.15.115:38220) 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.557+0000 7fb0add9b640 1 -- 172.21.15.115:0/44585336 learned_addr learned my addr 172.21.15.115:0/44585336 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.557+0000 7fb0b6d9d640 1 --2- 172.21.15.115:0/44585336 >> v2:172.21.15.115:3300/0 conn(0x7fb0b80d0260 0x7fb0b80b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.557+0000 7fb0b659c640 1 --2- 172.21.15.115:0/44585336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0b815cf70 0x7fb0b80b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.557+0000 7fb0b5d9b640 1 -- 172.21.15.115:0/44585336 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941814134 0 0) 0x7fb0b80b7760 con 0x7fb0b80b18f0 2023-12-01T03:08:39.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.557+0000 7fb0b5d9b640 1 -- 172.21.15.115:0/44585336 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a4003520 con 0x7fb0b80b18f0 2023-12-01T03:08:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.557+0000 7fb0b6d9d640 1 -- 172.21.15.115:0/44585336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0b815cf70 msgr2=0x7fb0b80b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.557+0000 7fb0b6d9d640 1 --2- 172.21.15.115:0/44585336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0b815cf70 0x7fb0b80b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.557+0000 7fb0b6d9d640 1 -- 172.21.15.115:0/44585336 >> v1:172.21.15.115:6789/0 conn(0x7fb0b80b18f0 legacy=0x7fb0b80cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.557+0000 7fb0b6d9d640 1 -- 172.21.15.115:0/44585336 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80b6e30 con 0x7fb0b80d0260 2023-12-01T03:08:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.558+0000 7fb0b6d9d640 1 --2- 172.21.15.115:0/44585336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 0x7fb0b80b7bf0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00097a0 tx=0x7fb0b002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb3bed2869ba1e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0b5d9b640 1 -- 172.21.15.115:0/44585336 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0030800 con 0x7fb0b80d0260 2023-12-01T03:08:39.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0b5d9b640 1 -- 172.21.15.115:0/44585336 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b0030960 con 0x7fb0b80d0260 2023-12-01T03:08:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0b5d9b640 1 -- 172.21.15.115:0/44585336 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0030cf0 con 0x7fb0b80d0260 2023-12-01T03:08:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0bcd73640 1 -- 172.21.15.115:0/44585336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 msgr2=0x7fb0b80b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0bcd73640 1 --2- 172.21.15.115:0/44585336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 0x7fb0b80b7bf0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00097a0 tx=0x7fb0b002b940 comp rx=0 tx=0).stop 2023-12-01T03:08:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0bcd73640 1 -- 172.21.15.115:0/44585336 shutdown_connections 2023-12-01T03:08:39.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0bcd73640 1 --2- 172.21.15.115:0/44585336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 0x7fb0b80b7bf0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0bcd73640 1 --2- 172.21.15.115:0/44585336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0b815cf70 0x7fb0b80b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0bcd73640 1 -- 172.21.15.115:0/44585336 >> 172.21.15.115:0/44585336 conn(0x7fb0b806a0e0 msgr2=0x7fb0b806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:39.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0bcd73640 1 -- 172.21.15.115:0/44585336 shutdown_connections 2023-12-01T03:08:39.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.559+0000 7fb0bcd73640 1 -- 172.21.15.115:0/44585336 wait complete. 2023-12-01T03:08:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.561+0000 7fb0bcd73640 1 Processor -- start 2023-12-01T03:08:39.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.561+0000 7fb0bcd73640 1 -- start start 2023-12-01T03:08:39.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.562+0000 7fb0bcd73640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0b80b18f0 0x7fb0b815a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.562+0000 7fb0bcd73640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 0x7fb0b81567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.562+0000 7fb0bcd73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b815cf70 0x7fb0b815e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.562+0000 7fb0bcd73640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80cb210 con 0x7fb0b80d0260 2023-12-01T03:08:39.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.562+0000 7fb0bcd73640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b815f7f0 con 0x7fb0b815cf70 2023-12-01T03:08:39.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.562+0000 7fb0bcd73640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b815acb0 con 0x7fb0b80b18f0 2023-12-01T03:08:39.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.562+0000 7fb0add9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 0x7fb0b81567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0add9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 0x7fb0b81567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51518/0 (socket says 172.21.15.115:51518) 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0add9b640 1 -- 172.21.15.115:0/2948507756 learned_addr learned my addr 172.21.15.115:0/2948507756 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0add9b640 1 -- 172.21.15.115:0/2948507756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0b80b18f0 msgr2=0x7fb0b815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0add9b640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0b80b18f0 0x7fb0b815a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0add9b640 1 -- 172.21.15.115:0/2948507756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b815cf70 msgr2=0x7fb0b815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0add9b640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b815cf70 0x7fb0b815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0add9b640 1 -- 172.21.15.115:0/2948507756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b0009400 con 0x7fb0b80d0260 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0add9b640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 0x7fb0b81567e0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000c500 tx=0x7fb0a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0af7fe640 1 -- 172.21.15.115:0/2948507756 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0005ce0 con 0x7fb0b80d0260 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0af7fe640 1 -- 172.21.15.115:0/2948507756 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a0005e40 con 0x7fb0b80d0260 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0af7fe640 1 -- 172.21.15.115:0/2948507756 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0007330 con 0x7fb0b80d0260 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.563+0000 7fb0bcd73640 1 -- 172.21.15.115:0/2948507756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b80ca870 con 0x7fb0b80d0260 2023-12-01T03:08:39.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.564+0000 7fb0bcd73640 1 -- 172.21.15.115:0/2948507756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b80cae30 con 0x7fb0b80d0260 2023-12-01T03:08:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.565+0000 7fb0bcd73640 1 -- 172.21.15.115:0/2948507756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b80cb350 con 0x7fb0b80d0260 2023-12-01T03:08:39.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.565+0000 7fb0af7fe640 1 -- 172.21.15.115:0/2948507756 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb0a0007550 con 0x7fb0b80d0260 2023-12-01T03:08:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.566+0000 7fb0af7fe640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb09806a550 0x7fb09806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.566+0000 7fb0af7fe640 1 -- 172.21.15.115:0/2948507756 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fb0a0087de0 con 0x7fb0b80d0260 2023-12-01T03:08:39.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.567+0000 7fb0b659c640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb09806a550 0x7fb09806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.567+0000 7fb0b659c640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb09806a550 0x7fb09806ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80070b0 tx=0x7fb0a8007040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:39.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.571+0000 7fb0af7fe640 1 -- 172.21.15.115:0/2948507756 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb0b80cb350 con 0x7fb0b80d0260 2023-12-01T03:08:39.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.759+0000 7fb0bcd73640 1 -- 172.21.15.115:0/2948507756 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0b80cb350 con 0x7fb0b80d0260 2023-12-01T03:08:39.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.760+0000 7fb0af7fe640 1 -- 172.21.15.115:0/2948507756 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v192) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0b80cb350 con 0x7fb0b80d0260 2023-12-01T03:08:39.762 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:08:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.762+0000 7fb0bcd73640 1 -- 172.21.15.115:0/2948507756 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb09806a550 msgr2=0x7fb09806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.762+0000 7fb0bcd73640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb09806a550 0x7fb09806ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80070b0 tx=0x7fb0a8007040 comp rx=0 tx=0).stop 2023-12-01T03:08:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.762+0000 7fb0bcd73640 1 -- 172.21.15.115:0/2948507756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 msgr2=0x7fb0b81567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.762+0000 7fb0bcd73640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 0x7fb0b81567e0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000c500 tx=0x7fb0a000c9c0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.763+0000 7fb0bcd73640 1 -- 172.21.15.115:0/2948507756 shutdown_connections 2023-12-01T03:08:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.763+0000 7fb0bcd73640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb09806a550 0x7fb09806ca10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.763+0000 7fb0bcd73640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0b815cf70 0x7fb0b815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.763+0000 7fb0bcd73640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0b80d0260 0x7fb0b81567e0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.763+0000 7fb0bcd73640 1 --2- 172.21.15.115:0/2948507756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0b80b18f0 0x7fb0b815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.763+0000 7fb0bcd73640 1 -- 172.21.15.115:0/2948507756 >> 172.21.15.115:0/2948507756 conn(0x7fb0b806a0e0 msgr2=0x7fb0b80d3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:39.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.763+0000 7fb0bcd73640 1 -- 172.21.15.115:0/2948507756 shutdown_connections 2023-12-01T03:08:39.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.763+0000 7fb0bcd73640 1 -- 172.21.15.115:0/2948507756 wait complete. 2023-12-01T03:08:39.784 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T03:08:39.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.990+0000 7fde70ee9640 1 Processor -- start 2023-12-01T03:08:39.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.990+0000 7fde70ee9640 1 -- start start 2023-12-01T03:08:39.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.991+0000 7fde70ee9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fde6c0cad70 0x7fde6c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.991+0000 7fde70ee9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 0x7fde6c06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:39.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.991+0000 7fde70ee9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6c0add10 con 0x7fde6c15e2f0 2023-12-01T03:08:39.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.991+0000 7fde70ee9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6c0c5760 con 0x7fde6c0cad70 2023-12-01T03:08:39.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.991+0000 7fde70ee9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c15e6d0 con 0x7fde6c06f2f0 2023-12-01T03:08:39.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.992+0000 7fde6ad9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 0x7fde6c06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.992+0000 7fde61d9b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fde6c0cad70 0x7fde6c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:39.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.992+0000 7fde6ad9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 0x7fde6c06bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:42706/0 (socket says 172.21.15.115:42706) 2023-12-01T03:08:39.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.992+0000 7fde6ad9d640 1 -- 172.21.15.115:0/391277288 learned_addr learned my addr 172.21.15.115:0/391277288 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:39.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.993+0000 7fde6ad9d640 1 -- 172.21.15.115:0/391277288 >> v2:172.21.15.142:3300/0 conn(0x7fde6c0cad70 msgr2=0x7fde6c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:39.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.993+0000 7fde6ad9d640 1 --2- 172.21.15.115:0/391277288 >> v2:172.21.15.142:3300/0 conn(0x7fde6c0cad70 0x7fde6c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.993+0000 7fde6ad9d640 1 -- 172.21.15.115:0/391277288 >> v1:172.21.15.142:6789/0 conn(0x7fde6c15e2f0 legacy=0x7fde6c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.993+0000 7fde6ad9d640 1 -- 172.21.15.115:0/391277288 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c0cf7b0 con 0x7fde6c06f2f0 2023-12-01T03:08:39.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.994+0000 7fde6ad9d640 1 --2- 172.21.15.115:0/391277288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 0x7fde6c06bb00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde64009870 tx=0x7fde6402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408eacc3aba88c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:39.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.995+0000 7fde69d9b640 1 -- 172.21.15.115:0/391277288 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde640317c0 con 0x7fde6c06f2f0 2023-12-01T03:08:39.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.995+0000 7fde69d9b640 1 -- 172.21.15.115:0/391277288 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde64031920 con 0x7fde6c06f2f0 2023-12-01T03:08:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.995+0000 7fde69d9b640 1 -- 172.21.15.115:0/391277288 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64031cb0 con 0x7fde6c06f2f0 2023-12-01T03:08:39.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.996+0000 7fde70ee9640 1 -- 172.21.15.115:0/391277288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 msgr2=0x7fde6c06bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.996+0000 7fde70ee9640 1 --2- 172.21.15.115:0/391277288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 0x7fde6c06bb00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde64009870 tx=0x7fde6402b860 comp rx=0 tx=0).stop 2023-12-01T03:08:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.996+0000 7fde70ee9640 1 -- 172.21.15.115:0/391277288 shutdown_connections 2023-12-01T03:08:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.996+0000 7fde70ee9640 1 --2- 172.21.15.115:0/391277288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 0x7fde6c06bb00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.996+0000 7fde70ee9640 1 --2- 172.21.15.115:0/391277288 >> v2:172.21.15.142:3300/0 conn(0x7fde6c0cad70 0x7fde6c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.996+0000 7fde70ee9640 1 -- 172.21.15.115:0/391277288 >> 172.21.15.115:0/391277288 conn(0x7fde6c0ce880 msgr2=0x7fde6c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.996+0000 7fde70ee9640 1 -- 172.21.15.115:0/391277288 shutdown_connections 2023-12-01T03:08:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.996+0000 7fde70ee9640 1 -- 172.21.15.115:0/391277288 wait complete. 2023-12-01T03:08:39.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.998+0000 7fde70ee9640 1 Processor -- start 2023-12-01T03:08:39.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.998+0000 7fde70ee9640 1 -- start start 2023-12-01T03:08:40.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde70ee9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 0x7fde6c158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde70ee9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde6c0cad70 0x7fde6c0ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde70ee9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde6c15e2f0 0x7fde6c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde70ee9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c0cc4f0 con 0x7fde6c15e2f0 2023-12-01T03:08:40.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde70ee9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1f2c30 con 0x7fde6c0cad70 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde70ee9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1f2f10 con 0x7fde6c06f2f0 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde6ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde6c15e2f0 0x7fde6c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde6ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde6c15e2f0 0x7fde6c0ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51526/0 (socket says 172.21.15.115:51526) 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde6ad9d640 1 -- 172.21.15.115:0/1314650575 learned_addr learned my addr 172.21.15.115:0/1314650575 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde6a59c640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 0x7fde6c158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde61d9b640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde6c0cad70 0x7fde6c0ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde6ad9d640 1 -- 172.21.15.115:0/1314650575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 msgr2=0x7fde6c158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde6ad9d640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 0x7fde6c158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde6ad9d640 1 -- 172.21.15.115:0/1314650575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde6c0cad70 msgr2=0x7fde6c0ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde6ad9d640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde6c0cad70 0x7fde6c0ae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:39.999+0000 7fde6ad9d640 1 -- 172.21.15.115:0/1314650575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde640094d0 con 0x7fde6c15e2f0 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.000+0000 7fde6ad9d640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde6c15e2f0 0x7fde6c0ac8a0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fde640090b0 tx=0x7fde640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.000+0000 7fde637fe640 1 -- 172.21.15.115:0/1314650575 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64032860 con 0x7fde6c15e2f0 2023-12-01T03:08:40.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.000+0000 7fde637fe640 1 -- 172.21.15.115:0/1314650575 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde64031f00 con 0x7fde6c15e2f0 2023-12-01T03:08:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.000+0000 7fde637fe640 1 -- 172.21.15.115:0/1314650575 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64032c00 con 0x7fde6c15e2f0 2023-12-01T03:08:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.000+0000 7fde70ee9640 1 -- 172.21.15.115:0/1314650575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c1f31f0 con 0x7fde6c15e2f0 2023-12-01T03:08:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.000+0000 7fde70ee9640 1 -- 172.21.15.115:0/1314650575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c1f34d0 con 0x7fde6c15e2f0 2023-12-01T03:08:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.001+0000 7fde70ee9640 1 -- 172.21.15.115:0/1314650575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c06fda0 con 0x7fde6c15e2f0 2023-12-01T03:08:40.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.002+0000 7fde637fe640 1 -- 172.21.15.115:0/1314650575 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fde6404e450 con 0x7fde6c15e2f0 2023-12-01T03:08:40.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.006+0000 7fde637fe640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde4c06a550 0x7fde4c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.006+0000 7fde637fe640 1 -- 172.21.15.115:0/1314650575 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fde640aa8f0 con 0x7fde6c15e2f0 2023-12-01T03:08:40.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.006+0000 7fde6a59c640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde4c06a550 0x7fde4c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.007+0000 7fde6a59c640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde4c06a550 0x7fde4c06ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0ad4b0 tx=0x7fde5c009040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:40.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.013+0000 7fde637fe640 1 -- 172.21.15.115:0/1314650575 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fde64076100 con 0x7fde6c15e2f0 2023-12-01T03:08:40.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.208+0000 7fde70ee9640 1 -- 172.21.15.115:0/1314650575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fde6c04e890 con 0x7fde6c15e2f0 2023-12-01T03:08:40.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.208+0000 7fde637fe640 1 -- 172.21.15.115:0/1314650575 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v192) v1 ==== 84+0+65 (secure 0 0 0) 0x7fde64079720 con 0x7fde6c15e2f0 2023-12-01T03:08:40.210 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:40.210 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T03:08:40.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:40.211 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T03:08:40.211 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T03:08:40.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:40.211 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T03:08:40.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 -- 172.21.15.115:0/1314650575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde4c06a550 msgr2=0x7fde4c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde4c06a550 0x7fde4c06ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0ad4b0 tx=0x7fde5c009040 comp rx=0 tx=0).stop 2023-12-01T03:08:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 -- 172.21.15.115:0/1314650575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde6c15e2f0 msgr2=0x7fde6c0ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde6c15e2f0 0x7fde6c0ac8a0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fde640090b0 tx=0x7fde640046c0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 -- 172.21.15.115:0/1314650575 shutdown_connections 2023-12-01T03:08:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde4c06a550 0x7fde4c06ca10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde6c15e2f0 0x7fde6c0ac8a0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde6c0cad70 0x7fde6c0ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 --2- 172.21.15.115:0/1314650575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde6c06f2f0 0x7fde6c158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 -- 172.21.15.115:0/1314650575 >> 172.21.15.115:0/1314650575 conn(0x7fde6c0ce880 msgr2=0x7fde6c0c1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:40.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.211+0000 7fde70ee9640 1 -- 172.21.15.115:0/1314650575 shutdown_connections 2023-12-01T03:08:40.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.212+0000 7fde70ee9640 1 -- 172.21.15.115:0/1314650575 wait complete. 2023-12-01T03:08:40.233 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T03:08:40.233 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T03:08:40.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.443+0000 7ff14160e640 1 Processor -- start 2023-12-01T03:08:40.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.443+0000 7ff14160e640 1 -- start start 2023-12-01T03:08:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.444+0000 7ff14160e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c0b04c0 0x7ff13c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.444+0000 7ff14160e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c0cbac0 con 0x7ff13c15ac00 2023-12-01T03:08:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.444+0000 7ff14160e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13c15afe0 con 0x7ff13c0cb630 2023-12-01T03:08:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.444+0000 7ff14160e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0b12b0 con 0x7ff13c0b04c0 2023-12-01T03:08:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.445+0000 7ff13bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff13c15ac00 0x7ff13c0ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38228/0 (socket says 172.21.15.115:38228) 2023-12-01T03:08:40.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.445+0000 7ff13bfff640 1 -- 172.21.15.115:0/806525651 learned_addr learned my addr 172.21.15.115:0/806525651 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.445+0000 7ff133fff640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c0b04c0 0x7ff13c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.445+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533900973 0 0) 0x7ff13c0cbac0 con 0x7ff13c15ac00 2023-12-01T03:08:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.445+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff128003520 con 0x7ff13c15ac00 2023-12-01T03:08:40.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.445+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3216876306 0 0) 0x7ff128003520 con 0x7ff13c15ac00 2023-12-01T03:08:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.445+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c0b04c0 msgr2=0x7ff13c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.445+0000 7ff13b7fe640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c0b04c0 0x7ff13c0cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.445+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 >> v1:172.21.15.142:6789/0 conn(0x7ff13c0cb630 legacy=0x7ff13c06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.445+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c0b0a10 con 0x7ff13c15ac00 2023-12-01T03:08:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7ff12c001670 con 0x7ff13c15ac00 2023-12-01T03:08:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 >> v1:172.21.15.115:6789/0 conn(0x7ff13c15ac00 legacy=0x7ff13c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13b7fe640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 0x7ff128003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13b7fe640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1280047c0 0x7ff128004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13b7fe640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1280051d0 0x7ff12800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0b12b0 con 0x7ff1280039a0 2023-12-01T03:08:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12800c0d0 con 0x7ff1280051d0 2023-12-01T03:08:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12800c290 con 0x7ff1280047c0 2023-12-01T03:08:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff133fff640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c0b04c0 0x7ff13c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:08:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13bfff640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 0x7ff128003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff140e0d640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1280051d0 0x7ff12800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff133fff640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1280047c0 0x7ff128004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13bfff640 1 -- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1280047c0 msgr2=0x7ff128004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13bfff640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1280047c0 0x7ff128004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13bfff640 1 -- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1280051d0 msgr2=0x7ff12800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13bfff640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1280051d0 0x7ff12800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.446+0000 7ff13bfff640 1 -- 172.21.15.115:0/806525651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128003520 con 0x7ff1280039a0 2023-12-01T03:08:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.447+0000 7ff13bfff640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 0x7ff128003dc0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00c240 tx=0x7ff1280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff09075d5764d144 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.448+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c005ad0 con 0x7ff1280039a0 2023-12-01T03:08:40.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.448+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c005c30 con 0x7ff1280039a0 2023-12-01T03:08:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.448+0000 7ff13b7fe640 1 -- 172.21.15.115:0/806525651 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c02c8b0 con 0x7ff1280039a0 2023-12-01T03:08:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.448+0000 7ff14160e640 1 -- 172.21.15.115:0/806525651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 msgr2=0x7ff128003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.448+0000 7ff14160e640 1 --2- 172.21.15.115:0/806525651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 0x7ff128003dc0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00c240 tx=0x7ff1280057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.448+0000 7ff140e0d640 1 -- 172.21.15.115:0/806525651 reap_dead start 2023-12-01T03:08:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.449+0000 7ff14160e640 1 -- 172.21.15.115:0/806525651 shutdown_connections 2023-12-01T03:08:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.449+0000 7ff14160e640 1 -- 172.21.15.115:0/806525651 >> 172.21.15.115:0/806525651 conn(0x7ff13c065960 msgr2=0x7ff13c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.449+0000 7ff14160e640 1 -- 172.21.15.115:0/806525651 shutdown_connections 2023-12-01T03:08:40.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.449+0000 7ff14160e640 1 -- 172.21.15.115:0/806525651 wait complete. 2023-12-01T03:08:40.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.450+0000 7ff14160e640 1 Processor -- start 2023-12-01T03:08:40.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.451+0000 7ff14160e640 1 -- start start 2023-12-01T03:08:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff14160e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 0x7ff13c1edb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff14160e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0c1a70 0x7ff13c0ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff14160e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c0c3740 0x7ff13c0c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff14160e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0c4100 con 0x7ff1280039a0 2023-12-01T03:08:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff14160e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0c3010 con 0x7ff13c0c1a70 2023-12-01T03:08:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff14160e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c0c34a0 con 0x7ff13c0c3740 2023-12-01T03:08:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff13bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 0x7ff13c1edb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff13bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 0x7ff13c1edb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51552/0 (socket says 172.21.15.115:51552) 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff13bfff640 1 -- 172.21.15.115:0/1249762518 learned_addr learned my addr 172.21.15.115:0/1249762518 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff133fff640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0c1a70 0x7ff13c0ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff140e0d640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c0c3740 0x7ff13c0c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff13bfff640 1 -- 172.21.15.115:0/1249762518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c0c3740 msgr2=0x7ff13c0c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff13bfff640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c0c3740 0x7ff13c0c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff13bfff640 1 -- 172.21.15.115:0/1249762518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0c1a70 msgr2=0x7ff13c0ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff13bfff640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0c1a70 0x7ff13c0ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.452+0000 7ff13bfff640 1 -- 172.21.15.115:0/1249762518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c0c2400 con 0x7ff1280039a0 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.453+0000 7ff13bfff640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 0x7ff13c1edb40 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7ff12c02c450 tx=0x7ff12c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.453+0000 7ff1397fa640 1 -- 172.21.15.115:0/1249762518 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c039070 con 0x7ff1280039a0 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.453+0000 7ff1397fa640 1 -- 172.21.15.115:0/1249762518 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c02ccc0 con 0x7ff1280039a0 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.453+0000 7ff14160e640 1 -- 172.21.15.115:0/1249762518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c0c26e0 con 0x7ff1280039a0 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.453+0000 7ff1397fa640 1 -- 172.21.15.115:0/1249762518 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c03c040 con 0x7ff1280039a0 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.453+0000 7ff14160e640 1 -- 172.21.15.115:0/1249762518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c0c2ca0 con 0x7ff1280039a0 2023-12-01T03:08:40.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.454+0000 7ff14160e640 1 -- 172.21.15.115:0/1249762518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c0618d0 con 0x7ff1280039a0 2023-12-01T03:08:40.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.455+0000 7ff1397fa640 1 -- 172.21.15.115:0/1249762518 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff12c032070 con 0x7ff1280039a0 2023-12-01T03:08:40.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.456+0000 7ff1397fa640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff11c06a550 0x7ff11c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:40.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.456+0000 7ff1397fa640 1 -- 172.21.15.115:0/1249762518 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4901+0+0 (secure 0 0 0) 0x7ff12c0a9a60 con 0x7ff1280039a0 2023-12-01T03:08:40.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.456+0000 7ff133fff640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff11c06a550 0x7ff11c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:40.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.457+0000 7ff133fff640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff11c06a550 0x7ff11c06ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff13c151af0 tx=0x7ff124008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:40.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.461+0000 7ff1397fa640 1 -- 172.21.15.115:0/1249762518 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff12c075260 con 0x7ff1280039a0 2023-12-01T03:08:40.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:40.652+0000 7ff14160e640 1 -- 172.21.15.115:0/1249762518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff13c157e70 con 0x7ff1280039a0 2023-12-01T03:08:41.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.582+0000 7ff1397fa640 1 -- 172.21.15.115:0/1249762518 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v193) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff12c078880 con 0x7ff1280039a0 2023-12-01T03:08:41.584 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T03:08:41.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.586+0000 7ff14160e640 1 -- 172.21.15.115:0/1249762518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff11c06a550 msgr2=0x7ff11c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:41.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.586+0000 7ff14160e640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff11c06a550 0x7ff11c06ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff13c151af0 tx=0x7ff124008040 comp rx=0 tx=0).stop 2023-12-01T03:08:41.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.586+0000 7ff14160e640 1 -- 172.21.15.115:0/1249762518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 msgr2=0x7ff13c1edb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:41.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.586+0000 7ff14160e640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 0x7ff13c1edb40 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7ff12c02c450 tx=0x7ff12c001a00 comp rx=0 tx=0).stop 2023-12-01T03:08:41.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.587+0000 7ff14160e640 1 -- 172.21.15.115:0/1249762518 shutdown_connections 2023-12-01T03:08:41.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.587+0000 7ff14160e640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff11c06a550 0x7ff11c06ca10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.587+0000 7ff14160e640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c0c3740 0x7ff13c0c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.587+0000 7ff14160e640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0c1a70 0x7ff13c0ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.587+0000 7ff14160e640 1 --2- 172.21.15.115:0/1249762518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1280039a0 0x7ff13c1edb40 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.587+0000 7ff14160e640 1 -- 172.21.15.115:0/1249762518 >> 172.21.15.115:0/1249762518 conn(0x7ff13c065960 msgr2=0x7ff13c06afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:41.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.587+0000 7ff14160e640 1 -- 172.21.15.115:0/1249762518 shutdown_connections 2023-12-01T03:08:41.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.587+0000 7ff14160e640 1 -- 172.21.15.115:0/1249762518 wait complete. 2023-12-01T03:08:41.614 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T03:08:41.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.806+0000 7f1b64d92640 1 Processor -- start 2023-12-01T03:08:41.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.806+0000 7f1b64d92640 1 -- start start 2023-12-01T03:08:41.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.808+0000 7f1b64d92640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b601552b0 0x7f1b600ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:41.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.808+0000 7f1b64d92640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6006b400 con 0x7f1b600afe10 2023-12-01T03:08:41.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.808+0000 7f1b64d92640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6006b6e0 con 0x7f1b6006c1c0 2023-12-01T03:08:41.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.808+0000 7f1b64d92640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b60155690 con 0x7f1b601552b0 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.809+0000 7f1b5ed9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1b600afe10 0x7f1b60164730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38244/0 (socket says 172.21.15.115:38244) 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.809+0000 7f1b5ed9d640 1 -- 172.21.15.115:0/1251370800 learned_addr learned my addr 172.21.15.115:0/1251370800 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.809+0000 7f1b5e59c640 1 --2- 172.21.15.115:0/1251370800 >> v2:172.21.15.142:3300/0 conn(0x7f1b601552b0 0x7f1b600ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.809+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171740981 0 0) 0x7f1b6006b400 con 0x7f1b600afe10 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.809+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b4c003520 con 0x7f1b600afe10 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3627237452 0 0) 0x7f1b4c003520 con 0x7f1b600afe10 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 >> v2:172.21.15.142:3300/0 conn(0x7f1b601552b0 msgr2=0x7f1b600ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 --2- 172.21.15.115:0/1251370800 >> v2:172.21.15.142:3300/0 conn(0x7f1b601552b0 0x7f1b600ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 >> v1:172.21.15.142:6789/0 conn(0x7f1b6006c1c0 legacy=0x7f1b600af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6006b9c0 con 0x7f1b600afe10 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f1b580019f0 con 0x7f1b600afe10 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 >> v1:172.21.15.115:6789/0 conn(0x7f1b600afe10 legacy=0x7f1b60164730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:41.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c003970 0x7f1b4c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b4c0047c0 0x7f1b4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b4c005180 0x7f1b4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60155690 con 0x7f1b4c0047c0 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c00c120 con 0x7f1b4c003970 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c00c2e0 con 0x7f1b4c005180 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.810+0000 7f1b57fff640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b4c0047c0 0x7f1b4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.811+0000 7f1b5e59c640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c003970 0x7f1b4c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.811+0000 7f1b5ed9d640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b4c005180 0x7f1b4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.811+0000 7f1b57fff640 1 -- 172.21.15.115:0/1251370800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b4c005180 msgr2=0x7f1b4c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.811+0000 7f1b57fff640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b4c005180 0x7f1b4c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.811+0000 7f1b57fff640 1 -- 172.21.15.115:0/1251370800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c003970 msgr2=0x7f1b4c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.811+0000 7f1b57fff640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b4c003970 0x7f1b4c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.811+0000 7f1b57fff640 1 -- 172.21.15.115:0/1251370800 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c003520 con 0x7f1b4c0047c0 2023-12-01T03:08:41.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.811+0000 7f1b5ed9d640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b4c005180 0x7f1b4c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:41.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.811+0000 7f1b5ed9d640 1 -- 172.21.15.115:0/1251370800 reap_dead start 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.812+0000 7f1b57fff640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b4c0047c0 0x7f1b4c004c40 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f1b48001750 tx=0x7f1b4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141e0c0d30dabab1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.812+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b480052d0 con 0x7f1b4c0047c0 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.812+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4802e050 con 0x7f1b4c0047c0 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.812+0000 7f1b5dd9b640 1 -- 172.21.15.115:0/1251370800 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48003830 con 0x7f1b4c0047c0 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.813+0000 7f1b64d92640 1 -- 172.21.15.115:0/1251370800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b4c0047c0 msgr2=0x7f1b4c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.813+0000 7f1b64d92640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b4c0047c0 0x7f1b4c004c40 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f1b48001750 tx=0x7f1b4c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.813+0000 7f1b64d92640 1 -- 172.21.15.115:0/1251370800 shutdown_connections 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.813+0000 7f1b64d92640 1 --2- 172.21.15.115:0/1251370800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b4c0047c0 0x7f1b4c004c40 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.813+0000 7f1b64d92640 1 -- 172.21.15.115:0/1251370800 >> 172.21.15.115:0/1251370800 conn(0x7f1b60065960 msgr2=0x7f1b60065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.813+0000 7f1b64d92640 1 -- 172.21.15.115:0/1251370800 shutdown_connections 2023-12-01T03:08:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.813+0000 7f1b64d92640 1 -- 172.21.15.115:0/1251370800 wait complete. 2023-12-01T03:08:41.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.815+0000 7f1b64d92640 1 Processor -- start 2023-12-01T03:08:41.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.815+0000 7f1b64d92640 1 -- start start 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b64d92640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b4c0047c0 0x7f1b60142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b64d92640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b60141cd0 0x7f1b600a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b64d92640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b60156820 0x7f1b60156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b64d92640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6006bda0 con 0x7f1b60141cd0 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b64d92640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b600cd300 con 0x7f1b60156820 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b64d92640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b600cd5e0 con 0x7f1b4c0047c0 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b60141cd0 0x7f1b600a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b57fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b60141cd0 0x7f1b600a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51568/0 (socket says 172.21.15.115:51568) 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b57fff640 1 -- 172.21.15.115:0/1592208541 learned_addr learned my addr 172.21.15.115:0/1592208541 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b5ed9d640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b60156820 0x7f1b60156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b5e59c640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b4c0047c0 0x7f1b60142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b57fff640 1 -- 172.21.15.115:0/1592208541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b4c0047c0 msgr2=0x7f1b60142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:41.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b57fff640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b4c0047c0 0x7f1b60142200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b57fff640 1 -- 172.21.15.115:0/1592208541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b60156820 msgr2=0x7f1b60156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:41.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b57fff640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b60156820 0x7f1b60156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:41.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.816+0000 7f1b57fff640 1 -- 172.21.15.115:0/1592208541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b600cd8c0 con 0x7f1b60141cd0 2023-12-01T03:08:41.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.817+0000 7f1b57fff640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b60141cd0 0x7f1b600a8480 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f1b48005040 tx=0x7f1b48001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:41.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.817+0000 7f1b56ffd640 1 -- 172.21.15.115:0/1592208541 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b480076c0 con 0x7f1b60141cd0 2023-12-01T03:08:41.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.817+0000 7f1b56ffd640 1 -- 172.21.15.115:0/1592208541 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b48007a30 con 0x7f1b60141cd0 2023-12-01T03:08:41.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.817+0000 7f1b56ffd640 1 -- 172.21.15.115:0/1592208541 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48006be0 con 0x7f1b60141cd0 2023-12-01T03:08:41.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.817+0000 7f1b64d92640 1 -- 172.21.15.115:0/1592208541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b60157b10 con 0x7f1b60141cd0 2023-12-01T03:08:41.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.817+0000 7f1b64d92640 1 -- 172.21.15.115:0/1592208541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b60158030 con 0x7f1b60141cd0 2023-12-01T03:08:41.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.818+0000 7f1b64d92640 1 -- 172.21.15.115:0/1592208541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b600cd720 con 0x7f1b60141cd0 2023-12-01T03:08:41.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.820+0000 7f1b56ffd640 1 -- 172.21.15.115:0/1592208541 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1b48006d40 con 0x7f1b60141cd0 2023-12-01T03:08:41.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.821+0000 7f1b56ffd640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1b4006a550 0x7f1b4006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:41.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.821+0000 7f1b56ffd640 1 -- 172.21.15.115:0/1592208541 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5271+0+0 (secure 0 0 0) 0x7f1b480aa9d0 con 0x7f1b60141cd0 2023-12-01T03:08:41.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.821+0000 7f1b5e59c640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1b4006a550 0x7f1b4006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:41.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.822+0000 7f1b5e59c640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1b4006a550 0x7f1b4006ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009990 tx=0x7f1b50008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:41.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:41.825+0000 7f1b56ffd640 1 -- 172.21.15.115:0/1592208541 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1b48076030 con 0x7f1b60141cd0 2023-12-01T03:08:42.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.015+0000 7f1b64d92640 1 -- 172.21.15.115:0/1592208541 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1b600cd720 con 0x7f1b60141cd0 2023-12-01T03:08:42.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.598+0000 7f1b56ffd640 1 -- 172.21.15.115:0/1592208541 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v194) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1b600cd720 con 0x7f1b60141cd0 2023-12-01T03:08:42.601 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T03:08:42.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.607+0000 7f1b64d92640 1 -- 172.21.15.115:0/1592208541 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1b4006a550 msgr2=0x7f1b4006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:42.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.607+0000 7f1b64d92640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1b4006a550 0x7f1b4006ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009990 tx=0x7f1b50008040 comp rx=0 tx=0).stop 2023-12-01T03:08:42.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.607+0000 7f1b64d92640 1 -- 172.21.15.115:0/1592208541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b60141cd0 msgr2=0x7f1b600a8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:42.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.607+0000 7f1b64d92640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b60141cd0 0x7f1b600a8480 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f1b48005040 tx=0x7f1b48001120 comp rx=0 tx=0).stop 2023-12-01T03:08:42.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.608+0000 7f1b64d92640 1 -- 172.21.15.115:0/1592208541 shutdown_connections 2023-12-01T03:08:42.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.608+0000 7f1b64d92640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1b4006a550 0x7f1b4006ca10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:42.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.608+0000 7f1b64d92640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b60156820 0x7f1b60156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:42.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.608+0000 7f1b64d92640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b60141cd0 0x7f1b600a8480 secure :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f1b48005040 tx=0x7f1b48001120 comp rx=0 tx=0).stop 2023-12-01T03:08:42.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.608+0000 7f1b64d92640 1 --2- 172.21.15.115:0/1592208541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b4c0047c0 0x7f1b60142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:42.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.608+0000 7f1b64d92640 1 -- 172.21.15.115:0/1592208541 >> 172.21.15.115:0/1592208541 conn(0x7f1b60065960 msgr2=0x7f1b600b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:42.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.611+0000 7f1b64d92640 1 -- 172.21.15.115:0/1592208541 shutdown_connections 2023-12-01T03:08:42.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.612+0000 7f1b64d92640 1 -- 172.21.15.115:0/1592208541 wait complete. 2023-12-01T03:08:42.643 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T03:08:42.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.827+0000 7f6586e41640 1 Processor -- start 2023-12-01T03:08:42.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.828+0000 7f6586e41640 1 -- start start 2023-12-01T03:08:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.829+0000 7f6586e41640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f658015ac80 0x7f65800cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.829+0000 7f6586e41640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65800cc9a0 con 0x7f65800b0e60 2023-12-01T03:08:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.829+0000 7f6586e41640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65800cbd90 con 0x7f65800ce1b0 2023-12-01T03:08:42.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.829+0000 7f6586e41640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658015b060 con 0x7f658015ac80 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.830+0000 7f657ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f65800b0e60 0x7f65800cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38260/0 (socket says 172.21.15.115:38260) 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.830+0000 7f657ffff640 1 -- 172.21.15.115:0/3545342197 learned_addr learned my addr 172.21.15.115:0/3545342197 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.830+0000 7f6584bdd640 1 --2- 172.21.15.115:0/3545342197 >> v2:172.21.15.142:3300/0 conn(0x7f658015ac80 0x7f65800cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.830+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867782552 0 0) 0x7f65800cc9a0 con 0x7f65800b0e60 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.830+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6570003520 con 0x7f65800b0e60 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1217742046 0 0) 0x7f6570003520 con 0x7f65800b0e60 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 >> v2:172.21.15.142:3300/0 conn(0x7f658015ac80 msgr2=0x7f65800cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 --2- 172.21.15.115:0/3545342197 >> v2:172.21.15.142:3300/0 conn(0x7f658015ac80 0x7f65800cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 >> v1:172.21.15.142:6789/0 conn(0x7f65800ce1b0 legacy=0x7f658006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65800cc070 con 0x7f65800b0e60 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f656c001670 con 0x7f65800b0e60 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 >> v1:172.21.15.115:6789/0 conn(0x7f65800b0e60 legacy=0x7f65800cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6570003970 0x7f6570003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:42.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65700047c0 0x7f6570004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:42.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6570005180 0x7f657000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:42.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658015b060 con 0x7f6570003970 2023-12-01T03:08:42.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657000c020 con 0x7f6570005180 2023-12-01T03:08:42.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657000c1e0 con 0x7f65700047c0 2023-12-01T03:08:42.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.831+0000 7f6584bdd640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6570003970 0x7f6570003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.832+0000 7f657ffff640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65700047c0 0x7f6570004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.832+0000 7f65853de640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6570005180 0x7f657000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:42.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.832+0000 7f6584bdd640 1 -- 172.21.15.115:0/3545342197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65700047c0 msgr2=0x7f6570004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.832+0000 7f6584bdd640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65700047c0 0x7f6570004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.832+0000 7f6584bdd640 1 -- 172.21.15.115:0/3545342197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6570005180 msgr2=0x7f657000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.832+0000 7f6584bdd640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6570005180 0x7f657000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.832+0000 7f6584bdd640 1 -- 172.21.15.115:0/3545342197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570003520 con 0x7f6570003970 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.832+0000 7f65853de640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6570005180 0x7f657000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.832+0000 7f65853de640 1 -- 172.21.15.115:0/3545342197 reap_dead start 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.833+0000 7f6584bdd640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6570003970 0x7f6570003d90 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f6574009ac0 tx=0x7f65700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea2418b0271c650 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.833+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657402c6b0 con 0x7f6570003970 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.833+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657402c810 con 0x7f6570003970 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.833+0000 7f657f7fe640 1 -- 172.21.15.115:0/3545342197 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657402cba0 con 0x7f6570003970 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.833+0000 7f6586e41640 1 -- 172.21.15.115:0/3545342197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6570003970 msgr2=0x7f6570003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.833+0000 7f6586e41640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6570003970 0x7f6570003d90 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f6574009ac0 tx=0x7f65700057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.834+0000 7f6586e41640 1 -- 172.21.15.115:0/3545342197 shutdown_connections 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.834+0000 7f6586e41640 1 --2- 172.21.15.115:0/3545342197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6570003970 0x7f6570003d90 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:42.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.834+0000 7f6586e41640 1 -- 172.21.15.115:0/3545342197 >> 172.21.15.115:0/3545342197 conn(0x7f6580065960 msgr2=0x7f6580065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:42.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.834+0000 7f6586e41640 1 -- 172.21.15.115:0/3545342197 shutdown_connections 2023-12-01T03:08:42.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.834+0000 7f6586e41640 1 -- 172.21.15.115:0/3545342197 wait complete. 2023-12-01T03:08:42.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.835+0000 7f6586e41640 1 Processor -- start 2023-12-01T03:08:42.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.835+0000 7f6586e41640 1 -- start start 2023-12-01T03:08:42.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.836+0000 7f6586e41640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6570003970 0x7f6580143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:42.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.836+0000 7f6586e41640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6580157550 0x7f6580159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:42.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.836+0000 7f6586e41640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65801579b0 0x7f6580159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:42.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.836+0000 7f6586e41640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6580152300 con 0x7f6580157550 2023-12-01T03:08:42.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.836+0000 7f6586e41640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6580150640 con 0x7f6570003970 2023-12-01T03:08:42.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.836+0000 7f6586e41640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6580151c60 con 0x7f65801579b0 2023-12-01T03:08:42.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.837+0000 7f657ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6580157550 0x7f6580159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:42.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.837+0000 7f657ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6580157550 0x7f6580159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51590/0 (socket says 172.21.15.115:51590) 2023-12-01T03:08:42.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.837+0000 7f657ffff640 1 -- 172.21.15.115:0/1549950913 learned_addr learned my addr 172.21.15.115:0/1549950913 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.837+0000 7f657ffff640 1 -- 172.21.15.115:0/1549950913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65801579b0 msgr2=0x7f6580159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.837+0000 7f657ffff640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65801579b0 0x7f6580159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.837+0000 7f657ffff640 1 -- 172.21.15.115:0/1549950913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6570003970 msgr2=0x7f6580143210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.837+0000 7f657ffff640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6570003970 0x7f6580143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.837+0000 7f657ffff640 1 -- 172.21.15.115:0/1549950913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658015a0e0 con 0x7f6580157550 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.837+0000 7f657ffff640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6580157550 0x7f6580159140 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f656c00d770 tx=0x7f656c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.838+0000 7f657d7fa640 1 -- 172.21.15.115:0/1549950913 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c0074e0 con 0x7f6580157550 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.838+0000 7f657d7fa640 1 -- 172.21.15.115:0/1549950913 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c007640 con 0x7f6580157550 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.838+0000 7f657d7fa640 1 -- 172.21.15.115:0/1549950913 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c0049e0 con 0x7f6580157550 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.838+0000 7f6586e41640 1 -- 172.21.15.115:0/1549950913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658015a3c0 con 0x7f6580157550 2023-12-01T03:08:42.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.838+0000 7f6586e41640 1 -- 172.21.15.115:0/1549950913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6580159bc0 con 0x7f6580157550 2023-12-01T03:08:42.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.839+0000 7f6586e41640 1 -- 172.21.15.115:0/1549950913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658004e890 con 0x7f6580157550 2023-12-01T03:08:42.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.841+0000 7f657d7fa640 1 -- 172.21.15.115:0/1549950913 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f656c00b650 con 0x7f6580157550 2023-12-01T03:08:42.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.842+0000 7f657d7fa640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f656406a550 0x7f656406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:42.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.842+0000 7f657d7fa640 1 -- 172.21.15.115:0/1549950913 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5653+0+0 (secure 0 0 0) 0x7f656c088710 con 0x7f6580157550 2023-12-01T03:08:42.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.843+0000 7f6584bdd640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f656406a550 0x7f656406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:42.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.843+0000 7f6584bdd640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f656406a550 0x7f656406ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6574036000 tx=0x7f657402e000 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:42.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:42.847+0000 7f657d7fa640 1 -- 172.21.15.115:0/1549950913 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f656c053d30 con 0x7f6580157550 2023-12-01T03:08:43.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.075+0000 7f6586e41640 1 -- 172.21.15.115:0/1549950913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f658004e040 con 0x7f6580157550 2023-12-01T03:08:43.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.085+0000 7f657d7fa640 1 -- 172.21.15.115:0/1549950913 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v83) v1 ==== 174+0+0 (secure 0 0 0) 0x7f656c057350 con 0x7f6580157550 2023-12-01T03:08:43.088 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 22 and data pool 23 2023-12-01T03:08:43.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.090+0000 7f6586e41640 1 -- 172.21.15.115:0/1549950913 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f656406a550 msgr2=0x7f656406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.090+0000 7f6586e41640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f656406a550 0x7f656406ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6574036000 tx=0x7f657402e000 comp rx=0 tx=0).stop 2023-12-01T03:08:43.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.090+0000 7f6586e41640 1 -- 172.21.15.115:0/1549950913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6580157550 msgr2=0x7f6580159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.090+0000 7f6586e41640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6580157550 0x7f6580159140 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f656c00d770 tx=0x7f656c00dc30 comp rx=0 tx=0).stop 2023-12-01T03:08:43.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.092+0000 7f6586e41640 1 -- 172.21.15.115:0/1549950913 shutdown_connections 2023-12-01T03:08:43.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.092+0000 7f6586e41640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f656406a550 0x7f656406ca10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.092+0000 7f6586e41640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65801579b0 0x7f6580159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.092+0000 7f6586e41640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6580157550 0x7f6580159140 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.092+0000 7f6586e41640 1 --2- 172.21.15.115:0/1549950913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6570003970 0x7f6580143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.092+0000 7f6586e41640 1 -- 172.21.15.115:0/1549950913 >> 172.21.15.115:0/1549950913 conn(0x7f6580065960 msgr2=0x7f658015b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:43.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.093+0000 7f6586e41640 1 -- 172.21.15.115:0/1549950913 shutdown_connections 2023-12-01T03:08:43.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.093+0000 7f6586e41640 1 -- 172.21.15.115:0/1549950913 wait complete. 2023-12-01T03:08:43.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:08:43.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.293+0000 7ffb42820640 1 Processor -- start 2023-12-01T03:08:43.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.293+0000 7ffb42820640 1 -- start start 2023-12-01T03:08:43.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.295+0000 7ffb42820640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb3c15afe0 0x7ffb3c06b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.295+0000 7ffb42820640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffb3c06b7b0 0x7ffb3c0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.295+0000 7ffb42820640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3c0ce4a0 con 0x7ffb3c1584b0 2023-12-01T03:08:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.295+0000 7ffb42820640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3c06bc40 con 0x7ffb3c06b7b0 2023-12-01T03:08:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.295+0000 7ffb42820640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0cc180 con 0x7ffb3c15afe0 2023-12-01T03:08:43.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.295+0000 7ffb4201f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffb3c06b7b0 0x7ffb3c0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.295+0000 7ffb4201f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffb3c06b7b0 0x7ffb3c0cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51600/0 (socket says 172.21.15.115:51600) 2023-12-01T03:08:43.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.295+0000 7ffb4201f640 1 -- 172.21.15.115:0/2052456240 learned_addr learned my addr 172.21.15.115:0/2052456240 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:43.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.295+0000 7ffb4101d640 1 --2- 172.21.15.115:0/2052456240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb3c15afe0 0x7ffb3c06b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.296+0000 7ffb4201f640 1 -- 172.21.15.115:0/2052456240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb3c15afe0 msgr2=0x7ffb3c06b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:43.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.296+0000 7ffb4201f640 1 --2- 172.21.15.115:0/2052456240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb3c15afe0 0x7ffb3c06b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.296+0000 7ffb4201f640 1 -- 172.21.15.115:0/2052456240 >> v1:172.21.15.142:6789/0 conn(0x7ffb3c1584b0 legacy=0x7ffb3c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:43.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.296+0000 7ffb4201f640 1 -- 172.21.15.115:0/2052456240 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c15b550 con 0x7ffb3c06b7b0 2023-12-01T03:08:43.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.296+0000 7ffb4101d640 1 --2- 172.21.15.115:0/2052456240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb3c15afe0 0x7ffb3c06b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:43.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.297+0000 7ffb4201f640 1 --2- 172.21.15.115:0/2052456240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c06b7b0 0x7ffb3c0cc680 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009e50 tx=0x7ffb3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55db02a3e3e278c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:43.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.297+0000 7ffb4081c640 1 -- 172.21.15.115:0/2052456240 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3402ca70 con 0x7ffb3c06b7b0 2023-12-01T03:08:43.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.297+0000 7ffb4081c640 1 -- 172.21.15.115:0/2052456240 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3402cbd0 con 0x7ffb3c06b7b0 2023-12-01T03:08:43.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.297+0000 7ffb4081c640 1 -- 172.21.15.115:0/2052456240 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb340343f0 con 0x7ffb3c06b7b0 2023-12-01T03:08:43.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.297+0000 7ffb42820640 1 -- 172.21.15.115:0/2052456240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c06b7b0 msgr2=0x7ffb3c0cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.297+0000 7ffb42820640 1 --2- 172.21.15.115:0/2052456240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c06b7b0 0x7ffb3c0cc680 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009e50 tx=0x7ffb3402bed0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.298+0000 7ffb42820640 1 -- 172.21.15.115:0/2052456240 shutdown_connections 2023-12-01T03:08:43.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.298+0000 7ffb42820640 1 --2- 172.21.15.115:0/2052456240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c06b7b0 0x7ffb3c0cc680 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.298+0000 7ffb42820640 1 --2- 172.21.15.115:0/2052456240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb3c15afe0 0x7ffb3c06b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.298+0000 7ffb42820640 1 -- 172.21.15.115:0/2052456240 >> 172.21.15.115:0/2052456240 conn(0x7ffb3c0ce640 msgr2=0x7ffb3c0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:43.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.298+0000 7ffb42820640 1 -- 172.21.15.115:0/2052456240 shutdown_connections 2023-12-01T03:08:43.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.298+0000 7ffb42820640 1 -- 172.21.15.115:0/2052456240 wait complete. 2023-12-01T03:08:43.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.299+0000 7ffb42820640 1 Processor -- start 2023-12-01T03:08:43.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.299+0000 7ffb42820640 1 -- start start 2023-12-01T03:08:43.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.300+0000 7ffb42820640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb3c06b7b0 0x7ffb3c06f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.300+0000 7ffb42820640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c1584b0 0x7ffb3c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.300+0000 7ffb42820640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb3c15afe0 0x7ffb3c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.300+0000 7ffb42820640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c06f960 con 0x7ffb3c1584b0 2023-12-01T03:08:43.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.300+0000 7ffb42820640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0c5680 con 0x7ffb3c06b7b0 2023-12-01T03:08:43.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.300+0000 7ffb42820640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0c5960 con 0x7ffb3c15afe0 2023-12-01T03:08:43.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb4101d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c1584b0 0x7ffb3c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb4101d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c1584b0 0x7ffb3c0b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51610/0 (socket says 172.21.15.115:51610) 2023-12-01T03:08:43.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb4101d640 1 -- 172.21.15.115:0/487842127 learned_addr learned my addr 172.21.15.115:0/487842127 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb4101d640 1 -- 172.21.15.115:0/487842127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb3c15afe0 msgr2=0x7ffb3c157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb4101d640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb3c15afe0 0x7ffb3c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb4101d640 1 -- 172.21.15.115:0/487842127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb3c06b7b0 msgr2=0x7ffb3c06f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb4101d640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb3c06b7b0 0x7ffb3c06f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb4101d640 1 -- 172.21.15.115:0/487842127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb34009b30 con 0x7ffb3c1584b0 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb4101d640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c1584b0 0x7ffb3c0b23f0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00c7f0 tx=0x7ffb2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb327fc640 1 -- 172.21.15.115:0/487842127 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c007820 con 0x7ffb3c1584b0 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb327fc640 1 -- 172.21.15.115:0/487842127 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c007980 con 0x7ffb3c1584b0 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb327fc640 1 -- 172.21.15.115:0/487842127 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c007cf0 con 0x7ffb3c1584b0 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.301+0000 7ffb42820640 1 -- 172.21.15.115:0/487842127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c0c5c40 con 0x7ffb3c1584b0 2023-12-01T03:08:43.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.302+0000 7ffb42820640 1 -- 172.21.15.115:0/487842127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c0c6200 con 0x7ffb3c1584b0 2023-12-01T03:08:43.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.303+0000 7ffb42820640 1 -- 172.21.15.115:0/487842127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c0673a0 con 0x7ffb3c1584b0 2023-12-01T03:08:43.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.304+0000 7ffb327fc640 1 -- 172.21.15.115:0/487842127 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ffb2c0113f0 con 0x7ffb3c1584b0 2023-12-01T03:08:43.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.305+0000 7ffb327fc640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffb2006a550 0x7ffb2006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.305+0000 7ffb327fc640 1 -- 172.21.15.115:0/487842127 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5753+0+0 (secure 0 0 0) 0x7ffb2c088fa0 con 0x7ffb3c1584b0 2023-12-01T03:08:43.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.305+0000 7ffb4181e640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffb2006a550 0x7ffb2006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.306+0000 7ffb4181e640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffb2006a550 0x7ffb2006ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffb38003cc0 tx=0x7ffb380079b0 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:43.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.309+0000 7ffb327fc640 1 -- 172.21.15.115:0/487842127 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ffb2c0542a0 con 0x7ffb3c1584b0 2023-12-01T03:08:43.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.490+0000 7ffb42820640 1 -- 172.21.15.115:0/487842127 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb3c06a2f0 con 0x7ffb3c1584b0 2023-12-01T03:08:43.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.493+0000 7ffb327fc640 1 -- 172.21.15.115:0/487842127 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16242 (secure 0 0 0) 0x7ffb2c01f090 con 0x7ffb3c1584b0 2023-12-01T03:08:43.495 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:43.495 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":195,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:08:43.077501+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:08:40.709195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-01T03:08:42.071749+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:08:43.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.495+0000 7ffb42820640 1 -- 172.21.15.115:0/487842127 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffb2006a550 msgr2=0x7ffb2006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.495+0000 7ffb42820640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffb2006a550 0x7ffb2006ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffb38003cc0 tx=0x7ffb380079b0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.495+0000 7ffb42820640 1 -- 172.21.15.115:0/487842127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c1584b0 msgr2=0x7ffb3c0b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.495+0000 7ffb42820640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c1584b0 0x7ffb3c0b23f0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00c7f0 tx=0x7ffb2c00ccb0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.495+0000 7ffb42820640 1 -- 172.21.15.115:0/487842127 shutdown_connections 2023-12-01T03:08:43.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.495+0000 7ffb42820640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffb2006a550 0x7ffb2006ca10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.495+0000 7ffb42820640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb3c15afe0 0x7ffb3c157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.495+0000 7ffb42820640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c1584b0 0x7ffb3c0b23f0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.495+0000 7ffb42820640 1 --2- 172.21.15.115:0/487842127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb3c06b7b0 0x7ffb3c06f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.495+0000 7ffb42820640 1 -- 172.21.15.115:0/487842127 >> 172.21.15.115:0/487842127 conn(0x7ffb3c0ce640 msgr2=0x7ffb3c0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:43.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.496+0000 7ffb42820640 1 -- 172.21.15.115:0/487842127 shutdown_connections 2023-12-01T03:08:43.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.496+0000 7ffb42820640 1 -- 172.21.15.115:0/487842127 wait complete. 2023-12-01T03:08:43.515 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:08:43.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.721+0000 7f9a05188640 1 Processor -- start 2023-12-01T03:08:43.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.721+0000 7f9a05188640 1 -- start start 2023-12-01T03:08:43.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.722+0000 7f9a05188640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a000c9ef0 0x7f9a0006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.722+0000 7f9a05188640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9a0006b6f0 0x7f9a00070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.722+0000 7f9a05188640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0006f8b0 con 0x7f9a00159dd0 2023-12-01T03:08:43.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.722+0000 7f9a05188640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a000adcb0 con 0x7f9a0006b6f0 2023-12-01T03:08:43.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.722+0000 7f9a05188640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000ad9d0 con 0x7f9a000c9ef0 2023-12-01T03:08:43.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fed9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9a00159dd0 0x7f9a000c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38268/0 (socket says 172.21.15.115:38268) 2023-12-01T03:08:43.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fed9d640 1 -- 172.21.15.115:0/4140633663 learned_addr learned my addr 172.21.15.115:0/4140633663 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:43.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383244298 0 0) 0x7f9a0006f8b0 con 0x7f9a00159dd0 2023-12-01T03:08:43.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99e8003520 con 0x7f9a00159dd0 2023-12-01T03:08:43.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99ff59e640 1 --2- 172.21.15.115:0/4140633663 >> v2:172.21.15.142:3300/0 conn(0x7f9a0006b6f0 0x7f9a00070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fe59c640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a000c9ef0 0x7f9a0006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1757091125 0 0) 0x7f99e8003520 con 0x7f9a00159dd0 2023-12-01T03:08:43.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a000c9ef0 msgr2=0x7f9a0006fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:43.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fdd9b640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a000c9ef0 0x7f9a0006fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 >> v2:172.21.15.142:3300/0 conn(0x7f9a0006b6f0 msgr2=0x7f9a00070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fdd9b640 1 --2- 172.21.15.115:0/4140633663 >> v2:172.21.15.142:3300/0 conn(0x7f9a0006b6f0 0x7f9a00070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000c9000 con 0x7f9a00159dd0 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.723+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f99f0001670 con 0x7f9a00159dd0 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 >> v1:172.21.15.115:6789/0 conn(0x7f9a00159dd0 legacy=0x7f9a000c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fdd9b640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99e80039e0 0x7f99e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fdd9b640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a000c9ef0 0x7f99e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fdd9b640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99e8004d00 0x7f99e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000adcb0 con 0x7f99e80039e0 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000ad9d0 con 0x7f99e8004d00 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e800bbe0 con 0x7f9a000c9ef0 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fed9d640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99e80039e0 0x7f99e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fe59c640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a000c9ef0 0x7f99e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99ff59e640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99e8004d00 0x7f99e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fed9d640 1 -- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a000c9ef0 msgr2=0x7f99e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:43.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fed9d640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a000c9ef0 0x7f99e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fed9d640 1 -- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99e8004d00 msgr2=0x7f99e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:43.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fed9d640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99e8004d00 0x7f99e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99fed9d640 1 -- 172.21.15.115:0/4140633663 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8003520 con 0x7f99e80039e0 2023-12-01T03:08:43.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.724+0000 7f99ff59e640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99e8004d00 0x7f99e8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:43.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.725+0000 7f99fed9d640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99e80039e0 0x7f99e8003e00 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c240 tx=0x7f99e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969b6d2091c792ec server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:43.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.726+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f002cb00 con 0x7f99e80039e0 2023-12-01T03:08:43.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.726+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f002cc60 con 0x7f99e80039e0 2023-12-01T03:08:43.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.726+0000 7f99fdd9b640 1 -- 172.21.15.115:0/4140633663 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0005ad0 con 0x7f99e80039e0 2023-12-01T03:08:43.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.726+0000 7f9a05188640 1 -- 172.21.15.115:0/4140633663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99e80039e0 msgr2=0x7f99e8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.726+0000 7f9a05188640 1 --2- 172.21.15.115:0/4140633663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99e80039e0 0x7f99e8003e00 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c240 tx=0x7f99e80057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.726+0000 7f99ff59e640 1 -- 172.21.15.115:0/4140633663 reap_dead start 2023-12-01T03:08:43.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.726+0000 7f9a05188640 1 -- 172.21.15.115:0/4140633663 shutdown_connections 2023-12-01T03:08:43.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.726+0000 7f9a05188640 1 -- 172.21.15.115:0/4140633663 >> 172.21.15.115:0/4140633663 conn(0x7f9a000ce880 msgr2=0x7f9a000cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:43.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.726+0000 7f9a05188640 1 -- 172.21.15.115:0/4140633663 shutdown_connections 2023-12-01T03:08:43.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.726+0000 7f9a05188640 1 -- 172.21.15.115:0/4140633663 wait complete. 2023-12-01T03:08:43.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.728+0000 7f9a05188640 1 Processor -- start 2023-12-01T03:08:43.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.728+0000 7f9a05188640 1 -- start start 2023-12-01T03:08:43.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.729+0000 7f9a05188640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99e80039e0 0x7f9a001500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.729+0000 7f9a05188640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a0015a4b0 0x7f9a0015a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.729+0000 7f9a05188640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a0015ae50 0x7f9a000c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.729+0000 7f9a05188640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000ad340 con 0x7f9a0015a4b0 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.729+0000 7f9a05188640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0015c220 con 0x7f99e80039e0 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.729+0000 7f9a05188640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0015b2d0 con 0x7f9a0015ae50 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.729+0000 7f99fe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a0015a4b0 0x7f9a0015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.730+0000 7f99fe59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a0015a4b0 0x7f9a0015a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51626/0 (socket says 172.21.15.115:51626) 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.730+0000 7f99fe59c640 1 -- 172.21.15.115:0/3722616470 learned_addr learned my addr 172.21.15.115:0/3722616470 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.730+0000 7f99ff59e640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a0015ae50 0x7f9a000c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.730+0000 7f99fed9d640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99e80039e0 0x7f9a001500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.730+0000 7f99fe59c640 1 -- 172.21.15.115:0/3722616470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a0015ae50 msgr2=0x7f9a000c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.730+0000 7f99fe59c640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a0015ae50 0x7f9a000c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.730+0000 7f99fe59c640 1 -- 172.21.15.115:0/3722616470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99e80039e0 msgr2=0x7f9a001500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.730+0000 7f99fe59c640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99e80039e0 0x7f9a001500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.730+0000 7f99fe59c640 1 -- 172.21.15.115:0/3722616470 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00151fd0 con 0x7f9a0015a4b0 2023-12-01T03:08:43.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.730+0000 7f99fe59c640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a0015a4b0 0x7f9a0015a910 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f99f400f580 tx=0x7f99f400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:43.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.731+0000 7f99e77fe640 1 -- 172.21.15.115:0/3722616470 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4014870 con 0x7f9a0015a4b0 2023-12-01T03:08:43.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.731+0000 7f99e77fe640 1 -- 172.21.15.115:0/3722616470 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f40149d0 con 0x7f9a0015a4b0 2023-12-01T03:08:43.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.731+0000 7f99e77fe640 1 -- 172.21.15.115:0/3722616470 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4014d80 con 0x7f9a0015a4b0 2023-12-01T03:08:43.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.731+0000 7f9a05188640 1 -- 172.21.15.115:0/3722616470 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a000c24e0 con 0x7f9a0015a4b0 2023-12-01T03:08:43.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.731+0000 7f9a05188640 1 -- 172.21.15.115:0/3722616470 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a000c2aa0 con 0x7f9a0015a4b0 2023-12-01T03:08:43.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.732+0000 7f9a05188640 1 -- 172.21.15.115:0/3722616470 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0004e890 con 0x7f9a0015a4b0 2023-12-01T03:08:43.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.733+0000 7f99e77fe640 1 -- 172.21.15.115:0/3722616470 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f99f4015410 con 0x7f9a0015a4b0 2023-12-01T03:08:43.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.734+0000 7f99e77fe640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f99ec06a580 0x7f99ec06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:43.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.734+0000 7f99e77fe640 1 -- 172.21.15.115:0/3722616470 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f99f40894a0 con 0x7f9a0015a4b0 2023-12-01T03:08:43.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.734+0000 7f99fed9d640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f99ec06a580 0x7f99ec06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:43.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.735+0000 7f99fed9d640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f99ec06a580 0x7f99ec06ca40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f99f0034000 tx=0x7f99f0037040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:43.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.738+0000 7f99e77fe640 1 -- 172.21.15.115:0/3722616470 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f99f4054960 con 0x7f9a0015a4b0 2023-12-01T03:08:43.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.929+0000 7f9a05188640 1 -- 172.21.15.115:0/3722616470 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a00150610 con 0x7f9a0015a4b0 2023-12-01T03:08:43.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.931+0000 7f99e77fe640 1 -- 172.21.15.115:0/3722616470 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16242 (secure 0 0 0) 0x7f99f4057f80 con 0x7f9a0015a4b0 2023-12-01T03:08:43.933 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:43.933 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":195,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:08:43.077501+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:08:40.709195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-01T03:08:42.071749+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:08:43.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 -- 172.21.15.115:0/3722616470 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f99ec06a580 msgr2=0x7f99ec06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f99ec06a580 0x7f99ec06ca40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f99f0034000 tx=0x7f99f0037040 comp rx=0 tx=0).stop 2023-12-01T03:08:43.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 -- 172.21.15.115:0/3722616470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a0015a4b0 msgr2=0x7f9a0015a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:43.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a0015a4b0 0x7f9a0015a910 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f99f400f580 tx=0x7f99f400fa40 comp rx=0 tx=0).stop 2023-12-01T03:08:43.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 -- 172.21.15.115:0/3722616470 shutdown_connections 2023-12-01T03:08:43.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f99ec06a580 0x7f99ec06ca40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a0015ae50 0x7f9a000c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a0015a4b0 0x7f9a0015a910 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 --2- 172.21.15.115:0/3722616470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99e80039e0 0x7f9a001500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:43.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 -- 172.21.15.115:0/3722616470 >> 172.21.15.115:0/3722616470 conn(0x7f9a000ce880 msgr2=0x7f9a0006fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:43.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 -- 172.21.15.115:0/3722616470 shutdown_connections 2023-12-01T03:08:43.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:43.934+0000 7f9a05188640 1 -- 172.21.15.115:0/3722616470 wait complete. 2023-12-01T03:08:43.954 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T03:08:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.165+0000 7f0117851640 1 Processor -- start 2023-12-01T03:08:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.165+0000 7f0117851640 1 -- start start 2023-12-01T03:08:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.166+0000 7f0117851640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f011015b0e0 0x7f0110164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.167+0000 7f0117851640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01100b2f30 con 0x7f0110142870 2023-12-01T03:08:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.167+0000 7f0117851640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01100b2320 con 0x7f01100cd1e0 2023-12-01T03:08:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.167+0000 7f0117851640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01100b3ae0 con 0x7f011015b0e0 2023-12-01T03:08:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.167+0000 7f01155ed640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0110142870 0x7f01100ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38284/0 (socket says 172.21.15.115:38284) 2023-12-01T03:08:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.167+0000 7f01155ed640 1 -- 172.21.15.115:0/1844485168 learned_addr learned my addr 172.21.15.115:0/1844485168 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.167+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908204154 0 0) 0x7f01100b2f30 con 0x7f0110142870 2023-12-01T03:08:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.167+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00fc005730 con 0x7f0110142870 2023-12-01T03:08:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.167+0000 7f0115dee640 1 --2- 172.21.15.115:0/1844485168 >> v2:172.21.15.142:3300/0 conn(0x7f011015b0e0 0x7f0110164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:44.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3312517329 0 0) 0x7f00fc005730 con 0x7f0110142870 2023-12-01T03:08:44.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 >> v2:172.21.15.142:3300/0 conn(0x7f011015b0e0 msgr2=0x7f0110164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:44.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 --2- 172.21.15.115:0/1844485168 >> v2:172.21.15.142:3300/0 conn(0x7f011015b0e0 0x7f0110164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:44.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 >> v1:172.21.15.142:6789/0 conn(0x7f01100cd1e0 legacy=0x7f01100b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01100b2600 con 0x7f0110142870 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f010c001670 con 0x7f0110142870 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 >> v1:172.21.15.115:6789/0 conn(0x7f0110142870 legacy=0x7f01100ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00fc005bd0 0x7f00fc005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00fc0069d0 0x7f00fc006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00fc007390 0x7f00fc00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01100b3ae0 con 0x7f00fc005bd0 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc00e290 con 0x7f00fc0069d0 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc00e450 con 0x7f00fc007390 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f01155ed640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00fc005bd0 0x7f00fc005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f0114dec640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00fc0069d0 0x7f00fc006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f01155ed640 1 -- 172.21.15.115:0/1844485168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00fc007390 msgr2=0x7f00fc00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f01155ed640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00fc007390 0x7f00fc00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f01155ed640 1 -- 172.21.15.115:0/1844485168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00fc0069d0 msgr2=0x7f00fc006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.168+0000 7f01155ed640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00fc0069d0 0x7f00fc006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.169+0000 7f01155ed640 1 -- 172.21.15.115:0/1844485168 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc005730 con 0x7f00fc005bd0 2023-12-01T03:08:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.169+0000 7f0115dee640 1 -- 172.21.15.115:0/1844485168 reap_dead start 2023-12-01T03:08:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.169+0000 7f0114dec640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00fc0069d0 0x7f00fc006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.170+0000 7f01155ed640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00fc005bd0 0x7f00fc005ff0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f010c00c0e0 tx=0x7f00fc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f93160944e0527 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.170+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c005ad0 con 0x7f00fc005bd0 2023-12-01T03:08:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.170+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c005c50 con 0x7f00fc005bd0 2023-12-01T03:08:44.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.170+0000 7f0107fff640 1 -- 172.21.15.115:0/1844485168 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c02c790 con 0x7f00fc005bd0 2023-12-01T03:08:44.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.170+0000 7f0117851640 1 -- 172.21.15.115:0/1844485168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00fc005bd0 msgr2=0x7f00fc005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:44.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.170+0000 7f0117851640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00fc005bd0 0x7f00fc005ff0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f010c00c0e0 tx=0x7f00fc007a00 comp rx=0 tx=0).stop 2023-12-01T03:08:44.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.170+0000 7f0117851640 1 -- 172.21.15.115:0/1844485168 shutdown_connections 2023-12-01T03:08:44.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.170+0000 7f0117851640 1 --2- 172.21.15.115:0/1844485168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00fc005bd0 0x7f00fc005ff0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:44.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.170+0000 7f0117851640 1 -- 172.21.15.115:0/1844485168 >> 172.21.15.115:0/1844485168 conn(0x7f0110065960 msgr2=0x7f0110065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:44.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.170+0000 7f0117851640 1 -- 172.21.15.115:0/1844485168 shutdown_connections 2023-12-01T03:08:44.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.171+0000 7f0117851640 1 -- 172.21.15.115:0/1844485168 wait complete. 2023-12-01T03:08:44.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.172+0000 7f0117851640 1 Processor -- start 2023-12-01T03:08:44.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.172+0000 7f0117851640 1 -- start start 2023-12-01T03:08:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.173+0000 7f0117851640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00fc005bd0 0x7f01100ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.173+0000 7f0117851640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01100b03a0 0x7f0110140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.173+0000 7f0117851640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01101f22f0 0x7f011013d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.173+0000 7f0117851640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01100c4710 con 0x7f01101f22f0 2023-12-01T03:08:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.173+0000 7f0117851640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01100add30 con 0x7f00fc005bd0 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.173+0000 7f0117851640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01100ad120 con 0x7f01100b03a0 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.173+0000 7f0115dee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01101f22f0 0x7f011013d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.173+0000 7f0115dee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01101f22f0 0x7f011013d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51646/0 (socket says 172.21.15.115:51646) 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.173+0000 7f0115dee640 1 -- 172.21.15.115:0/251186997 learned_addr learned my addr 172.21.15.115:0/251186997 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f0114dec640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01100b03a0 0x7f0110140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f0115dee640 1 -- 172.21.15.115:0/251186997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01100b03a0 msgr2=0x7f0110140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f01155ed640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00fc005bd0 0x7f01100ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f0115dee640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01100b03a0 0x7f0110140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f0115dee640 1 -- 172.21.15.115:0/251186997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00fc005bd0 msgr2=0x7f01100ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f0115dee640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00fc005bd0 0x7f01100ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f0115dee640 1 -- 172.21.15.115:0/251186997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01100ad400 con 0x7f01101f22f0 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f0115dee640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01101f22f0 0x7f011013d040 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f01100b34f0 tx=0x7f010800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f0105ffb640 1 -- 172.21.15.115:0/251186997 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108010040 con 0x7f01101f22f0 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f0105ffb640 1 -- 172.21.15.115:0/251186997 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010800bc80 con 0x7f01101f22f0 2023-12-01T03:08:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.174+0000 7f0117851640 1 -- 172.21.15.115:0/251186997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01100ad6e0 con 0x7f01101f22f0 2023-12-01T03:08:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.175+0000 7f0105ffb640 1 -- 172.21.15.115:0/251186997 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108014400 con 0x7f01101f22f0 2023-12-01T03:08:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.175+0000 7f0117851640 1 -- 172.21.15.115:0/251186997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01100acdd0 con 0x7f01101f22f0 2023-12-01T03:08:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.176+0000 7f0117851640 1 -- 172.21.15.115:0/251186997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01100adee0 con 0x7f01101f22f0 2023-12-01T03:08:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.176+0000 7f0105ffb640 1 -- 172.21.15.115:0/251186997 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0108014560 con 0x7f01101f22f0 2023-12-01T03:08:44.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.178+0000 7f0105ffb640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f00f406a580 0x7f00f406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:44.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.178+0000 7f0105ffb640 1 -- 172.21.15.115:0/251186997 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f0108089080 con 0x7f01101f22f0 2023-12-01T03:08:44.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.178+0000 7f01155ed640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f00f406a580 0x7f00f406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:44.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.179+0000 7f01155ed640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f00f406a580 0x7f00f406ca40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f010c034000 tx=0x7f010c009030 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:44.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.182+0000 7f0105ffb640 1 -- 172.21.15.115:0/251186997 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f01100adee0 con 0x7f01101f22f0 2023-12-01T03:08:44.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:44.413+0000 7f0117851640 1 -- 172.21.15.115:0/251186997 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f01100adea0 con 0x7f01101f22f0 2023-12-01T03:08:45.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.100+0000 7f0105ffb640 1 -- 172.21.15.115:0/251186997 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0108054380 con 0x7f01101f22f0 2023-12-01T03:08:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 -- 172.21.15.115:0/251186997 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f00f406a580 msgr2=0x7f00f406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f00f406a580 0x7f00f406ca40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f010c034000 tx=0x7f010c009030 comp rx=0 tx=0).stop 2023-12-01T03:08:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 -- 172.21.15.115:0/251186997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01101f22f0 msgr2=0x7f011013d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01101f22f0 0x7f011013d040 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f01100b34f0 tx=0x7f010800b480 comp rx=0 tx=0).stop 2023-12-01T03:08:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 -- 172.21.15.115:0/251186997 shutdown_connections 2023-12-01T03:08:45.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f00f406a580 0x7f00f406ca40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:45.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01100b03a0 0x7f0110140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:45.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01101f22f0 0x7f011013d040 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:45.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 --2- 172.21.15.115:0/251186997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00fc005bd0 0x7f01100ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:45.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 -- 172.21.15.115:0/251186997 >> 172.21.15.115:0/251186997 conn(0x7f0110065960 msgr2=0x7f01100b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:45.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.104+0000 7f0117851640 1 -- 172.21.15.115:0/251186997 shutdown_connections 2023-12-01T03:08:45.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.105+0000 7f0117851640 1 -- 172.21.15.115:0/251186997 wait complete. 2023-12-01T03:08:45.129 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T03:08:45.129 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T03:08:45.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.341+0000 7f21e8496640 1 Processor -- start 2023-12-01T03:08:45.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.341+0000 7f21e8496640 1 -- start start 2023-12-01T03:08:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.343+0000 7f21e8496640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f21e0160200 0x7f21e00b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.343+0000 7f21e8496640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f21e00cbd30 0x7f21e00cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.343+0000 7f21e8496640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e00cb290 con 0x7f21e00b0f80 2023-12-01T03:08:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.343+0000 7f21e8496640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e00b35f0 con 0x7f21e00cbd30 2023-12-01T03:08:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.343+0000 7f21e8496640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e00cc6a0 con 0x7f21e0160200 2023-12-01T03:08:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.343+0000 7f21e6a33640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f21e00cbd30 0x7f21e00cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.343+0000 7f21e6a33640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f21e00cbd30 0x7f21e00cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51652/0 (socket says 172.21.15.115:51652) 2023-12-01T03:08:45.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.343+0000 7f21e6a33640 1 -- 172.21.15.115:0/1350195124 learned_addr learned my addr 172.21.15.115:0/1350195124 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.344+0000 7f21e6232640 1 --2- 172.21.15.115:0/1350195124 >> v2:172.21.15.142:3300/0 conn(0x7f21e0160200 0x7f21e00b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.344+0000 7f21e6a33640 1 -- 172.21.15.115:0/1350195124 >> v2:172.21.15.142:3300/0 conn(0x7f21e0160200 msgr2=0x7f21e00b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.344+0000 7f21e6a33640 1 --2- 172.21.15.115:0/1350195124 >> v2:172.21.15.142:3300/0 conn(0x7f21e0160200 0x7f21e00b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.344+0000 7f21e6a33640 1 -- 172.21.15.115:0/1350195124 >> v1:172.21.15.142:6789/0 conn(0x7f21e00b0f80 legacy=0x7f21e006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:45.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.344+0000 7f21e6a33640 1 -- 172.21.15.115:0/1350195124 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e00b0ae0 con 0x7f21e00cbd30 2023-12-01T03:08:45.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.345+0000 7f21e6a33640 1 --2- 172.21.15.115:0/1350195124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e00cbd30 0x7f21e00cc160 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f21e006ce00 tx=0x7f21d802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a295b5999b4870a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:45.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.345+0000 7f21e5230640 1 -- 172.21.15.115:0/1350195124 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d802ca80 con 0x7f21e00cbd30 2023-12-01T03:08:45.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.345+0000 7f21e5230640 1 -- 172.21.15.115:0/1350195124 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d802cbe0 con 0x7f21e00cbd30 2023-12-01T03:08:45.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.345+0000 7f21e5230640 1 -- 172.21.15.115:0/1350195124 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d80343f0 con 0x7f21e00cbd30 2023-12-01T03:08:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.346+0000 7f21e8496640 1 -- 172.21.15.115:0/1350195124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e00cbd30 msgr2=0x7f21e00cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.346+0000 7f21e8496640 1 --2- 172.21.15.115:0/1350195124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e00cbd30 0x7f21e00cc160 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f21e006ce00 tx=0x7f21d802bee0 comp rx=0 tx=0).stop 2023-12-01T03:08:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.346+0000 7f21e8496640 1 -- 172.21.15.115:0/1350195124 shutdown_connections 2023-12-01T03:08:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.346+0000 7f21e8496640 1 --2- 172.21.15.115:0/1350195124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e00cbd30 0x7f21e00cc160 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.346+0000 7f21e8496640 1 --2- 172.21.15.115:0/1350195124 >> v2:172.21.15.142:3300/0 conn(0x7f21e0160200 0x7f21e00b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.346+0000 7f21e8496640 1 -- 172.21.15.115:0/1350195124 >> 172.21.15.115:0/1350195124 conn(0x7f21e0065960 msgr2=0x7f21e0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.346+0000 7f21e8496640 1 -- 172.21.15.115:0/1350195124 shutdown_connections 2023-12-01T03:08:45.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.346+0000 7f21e8496640 1 -- 172.21.15.115:0/1350195124 wait complete. 2023-12-01T03:08:45.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.348+0000 7f21e8496640 1 Processor -- start 2023-12-01T03:08:45.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.348+0000 7f21e8496640 1 -- start start 2023-12-01T03:08:45.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.349+0000 7f21e8496640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e00b0f80 0x7f21e01f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.349+0000 7f21e8496640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21e00cbd30 0x7f21e015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.349+0000 7f21e8496640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e0160200 0x7f21e015d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.349+0000 7f21e8496640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c98c0 con 0x7f21e0160200 2023-12-01T03:08:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.349+0000 7f21e8496640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c9ba0 con 0x7f21e00cbd30 2023-12-01T03:08:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.349+0000 7f21e8496640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c9e80 con 0x7f21e00b0f80 2023-12-01T03:08:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.349+0000 7f21e6a33640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e0160200 0x7f21e015d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.349+0000 7f21e6a33640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e0160200 0x7f21e015d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51668/0 (socket says 172.21.15.115:51668) 2023-12-01T03:08:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.349+0000 7f21e6a33640 1 -- 172.21.15.115:0/1110302899 learned_addr learned my addr 172.21.15.115:0/1110302899 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:45.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.349+0000 7f21e5a31640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21e00cbd30 0x7f21e015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21e6232640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e00b0f80 0x7f21e01f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21e6a33640 1 -- 172.21.15.115:0/1110302899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e00b0f80 msgr2=0x7f21e01f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21e6a33640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e00b0f80 0x7f21e01f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21e6a33640 1 -- 172.21.15.115:0/1110302899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21e00cbd30 msgr2=0x7f21e015cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21e6a33640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21e00cbd30 0x7f21e015cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21e6a33640 1 -- 172.21.15.115:0/1110302899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8009b30 con 0x7f21e0160200 2023-12-01T03:08:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21e6a33640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e0160200 0x7f21e015d0c0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f21d8036040 tx=0x7f21d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:45.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21d2ffd640 1 -- 172.21.15.115:0/1110302899 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8009510 con 0x7f21e0160200 2023-12-01T03:08:45.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21d2ffd640 1 -- 172.21.15.115:0/1110302899 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d803e450 con 0x7f21e0160200 2023-12-01T03:08:45.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21d2ffd640 1 -- 172.21.15.115:0/1110302899 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d803e7e0 con 0x7f21e0160200 2023-12-01T03:08:45.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.350+0000 7f21e8496640 1 -- 172.21.15.115:0/1110302899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e00ca160 con 0x7f21e0160200 2023-12-01T03:08:45.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.351+0000 7f21e8496640 1 -- 172.21.15.115:0/1110302899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e00c8b50 con 0x7f21e0160200 2023-12-01T03:08:45.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.352+0000 7f21e8496640 1 -- 172.21.15.115:0/1110302899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e015a680 con 0x7f21e0160200 2023-12-01T03:08:45.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.353+0000 7f21d2ffd640 1 -- 172.21.15.115:0/1110302899 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f21d802cd50 con 0x7f21e0160200 2023-12-01T03:08:45.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.354+0000 7f21d2ffd640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f21c406a580 0x7f21c406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:45.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.354+0000 7f21d2ffd640 1 -- 172.21.15.115:0/1110302899 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f21d80ab340 con 0x7f21e0160200 2023-12-01T03:08:45.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.354+0000 7f21e6232640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f21c406a580 0x7f21c406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:45.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.355+0000 7f21e6232640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f21c406a580 0x7f21c406ca40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21dc006fd0 tx=0x7f21dc008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:45.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.358+0000 7f21d2ffd640 1 -- 172.21.15.115:0/1110302899 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f21e015a680 con 0x7f21e0160200 2023-12-01T03:08:45.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:45.587+0000 7f21e8496640 1 -- 172.21.15.115:0/1110302899 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f21e015a680 con 0x7f21e0160200 2023-12-01T03:08:46.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.103+0000 7f21d2ffd640 1 -- 172.21.15.115:0/1110302899 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) v1 ==== 122+0+0 (secure 0 0 0) 0x7f21e015a680 con 0x7f21e0160200 2023-12-01T03:08:46.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 -- 172.21.15.115:0/1110302899 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f21c406a580 msgr2=0x7f21c406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f21c406a580 0x7f21c406ca40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21dc006fd0 tx=0x7f21dc008040 comp rx=0 tx=0).stop 2023-12-01T03:08:46.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 -- 172.21.15.115:0/1110302899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e0160200 msgr2=0x7f21e015d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e0160200 0x7f21e015d0c0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f21d8036040 tx=0x7f21d8004a40 comp rx=0 tx=0).stop 2023-12-01T03:08:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 -- 172.21.15.115:0/1110302899 shutdown_connections 2023-12-01T03:08:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f21c406a580 0x7f21c406ca40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e0160200 0x7f21e015d0c0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21e00cbd30 0x7f21e015cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 --2- 172.21.15.115:0/1110302899 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e00b0f80 0x7f21e01f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 -- 172.21.15.115:0/1110302899 >> 172.21.15.115:0/1110302899 conn(0x7f21e0065960 msgr2=0x7f21e00709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.106+0000 7f21e8496640 1 -- 172.21.15.115:0/1110302899 shutdown_connections 2023-12-01T03:08:46.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.107+0000 7f21e8496640 1 -- 172.21.15.115:0/1110302899 wait complete. 2023-12-01T03:08:46.128 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T03:08:46.129 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:08:46.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.339+0000 7f79affff640 1 Processor -- start 2023-12-01T03:08:46.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.339+0000 7f79affff640 1 -- start start 2023-12-01T03:08:46.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.341+0000 7f79affff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b00cdd80 0x7f79b006b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.341+0000 7f79affff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79b006b7b0 0x7f79b00c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.341+0000 7f79affff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b006bbf0 con 0x7f79b0145620 2023-12-01T03:08:46.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.341+0000 7f79affff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b00cc8e0 con 0x7f79b006b7b0 2023-12-01T03:08:46.343 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.341+0000 7f79affff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00ce260 con 0x7f79b00cdd80 2023-12-01T03:08:46.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.342+0000 7f79af7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79b006b7b0 0x7f79b00c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.342+0000 7f79a67fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b00cdd80 0x7f79b006b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.342+0000 7f79af7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79b006b7b0 0x7f79b00c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:52242/0 (socket says 172.21.15.115:52242) 2023-12-01T03:08:46.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.342+0000 7f79af7fe640 1 -- 172.21.15.115:0/2875410883 learned_addr learned my addr 172.21.15.115:0/2875410883 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:46.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.342+0000 7f79af7fe640 1 -- 172.21.15.115:0/2875410883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b00cdd80 msgr2=0x7f79b006b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:46.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.342+0000 7f79af7fe640 1 --2- 172.21.15.115:0/2875410883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b00cdd80 0x7f79b006b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.342+0000 7f79af7fe640 1 -- 172.21.15.115:0/2875410883 >> v1:172.21.15.142:6789/0 conn(0x7f79b0145620 legacy=0x7f79b00ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.343+0000 7f79af7fe640 1 -- 172.21.15.115:0/2875410883 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b00cc320 con 0x7f79b006b7b0 2023-12-01T03:08:46.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.343+0000 7f79a67fc640 1 --2- 172.21.15.115:0/2875410883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b00cdd80 0x7f79b006b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:08:46.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.344+0000 7f79af7fe640 1 --2- 172.21.15.115:0/2875410883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b006b7b0 0x7f79b00c8fa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009e50 tx=0x7f79a802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1055421459aed7d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:46.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.344+0000 7f79ae7fc640 1 -- 172.21.15.115:0/2875410883 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a802cb00 con 0x7f79b006b7b0 2023-12-01T03:08:46.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.344+0000 7f79ae7fc640 1 -- 172.21.15.115:0/2875410883 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a802cc60 con 0x7f79b006b7b0 2023-12-01T03:08:46.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.344+0000 7f79ae7fc640 1 -- 172.21.15.115:0/2875410883 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a80343f0 con 0x7f79b006b7b0 2023-12-01T03:08:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.345+0000 7f79affff640 1 -- 172.21.15.115:0/2875410883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b006b7b0 msgr2=0x7f79b00c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.345+0000 7f79affff640 1 --2- 172.21.15.115:0/2875410883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b006b7b0 0x7f79b00c8fa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009e50 tx=0x7f79a802bf60 comp rx=0 tx=0).stop 2023-12-01T03:08:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.345+0000 7f79affff640 1 -- 172.21.15.115:0/2875410883 shutdown_connections 2023-12-01T03:08:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.345+0000 7f79affff640 1 --2- 172.21.15.115:0/2875410883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b00cdd80 0x7f79b006b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.345+0000 7f79affff640 1 --2- 172.21.15.115:0/2875410883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b006b7b0 0x7f79b00c8fa0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.345+0000 7f79affff640 1 -- 172.21.15.115:0/2875410883 >> 172.21.15.115:0/2875410883 conn(0x7f79b00ce640 msgr2=0x7f79b00cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.345+0000 7f79affff640 1 -- 172.21.15.115:0/2875410883 shutdown_connections 2023-12-01T03:08:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.345+0000 7f79affff640 1 -- 172.21.15.115:0/2875410883 wait complete. 2023-12-01T03:08:46.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.347+0000 7f79affff640 1 Processor -- start 2023-12-01T03:08:46.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.347+0000 7f79affff640 1 -- start start 2023-12-01T03:08:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79affff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79b006b7b0 0x7f79b00c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79affff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b00cdd80 0x7f79b00c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79affff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b0145620 0x7f79b015be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79affff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00714c0 con 0x7f79b006b7b0 2023-12-01T03:08:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79affff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00717a0 con 0x7f79b00cdd80 2023-12-01T03:08:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79affff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0071a80 con 0x7f79b0145620 2023-12-01T03:08:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79aeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79b006b7b0 0x7f79b00c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79aeffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79b006b7b0 0x7f79b00c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51676/0 (socket says 172.21.15.115:51676) 2023-12-01T03:08:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79aeffd640 1 -- 172.21.15.115:0/3586080161 learned_addr learned my addr 172.21.15.115:0/3586080161 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79af7fe640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b0145620 0x7f79b015be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79a67fc640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b00cdd80 0x7f79b00c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79aeffd640 1 -- 172.21.15.115:0/3586080161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b0145620 msgr2=0x7f79b015be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79aeffd640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b0145620 0x7f79b015be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79aeffd640 1 -- 172.21.15.115:0/3586080161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b00cdd80 msgr2=0x7f79b00c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.348+0000 7f79aeffd640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b00cdd80 0x7f79b00c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.349+0000 7f79aeffd640 1 -- 172.21.15.115:0/3586080161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998009400 con 0x7f79b006b7b0 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.349+0000 7f79af7fe640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b0145620 0x7f79b015be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.349+0000 7f79aeffd640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79b006b7b0 0x7f79b00c6ee0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c400 tx=0x7f79a000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.349+0000 7f79a7fff640 1 -- 172.21.15.115:0/3586080161 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00052d0 con 0x7f79b006b7b0 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.349+0000 7f79a7fff640 1 -- 172.21.15.115:0/3586080161 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a0005430 con 0x7f79b006b7b0 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.349+0000 7f79a7fff640 1 -- 172.21.15.115:0/3586080161 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0003830 con 0x7f79b006b7b0 2023-12-01T03:08:46.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.349+0000 7f79affff640 1 -- 172.21.15.115:0/3586080161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a8009b30 con 0x7f79b006b7b0 2023-12-01T03:08:46.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.349+0000 7f79affff640 1 -- 172.21.15.115:0/3586080161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b0071d60 con 0x7f79b006b7b0 2023-12-01T03:08:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.350+0000 7f79affff640 1 -- 172.21.15.115:0/3586080161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b00673a0 con 0x7f79b006b7b0 2023-12-01T03:08:46.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.351+0000 7f79a7fff640 1 -- 172.21.15.115:0/3586080161 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f79a0003990 con 0x7f79b006b7b0 2023-12-01T03:08:46.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.352+0000 7f79a7fff640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f799006a580 0x7f799006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.352+0000 7f79a7fff640 1 -- 172.21.15.115:0/3586080161 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f79a0088100 con 0x7f79b006b7b0 2023-12-01T03:08:46.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.353+0000 7f79a67fc640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f799006a580 0x7f799006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.353+0000 7f79a67fc640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f799006a580 0x7f799006ca40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f79b00c7b10 tx=0x7f7998008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:46.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.357+0000 7f79a7fff640 1 -- 172.21.15.115:0/3586080161 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f79a0053510 con 0x7f79b006b7b0 2023-12-01T03:08:46.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.584+0000 7f79affff640 1 -- 172.21.15.115:0/3586080161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79b00ad9e0 con 0x7f79b006b7b0 2023-12-01T03:08:46.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.585+0000 7f79a7fff640 1 -- 172.21.15.115:0/3586080161 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f79a0056b30 con 0x7f79b006b7b0 2023-12-01T03:08:46.587 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:46.587 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/974148243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":974148243},{"type":"v1","addr":"172.21.15.115:6837","nonce":974148243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/476969842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":476969842},{"type":"v1","addr":"172.21.15.142:6834","nonce":476969842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1103499235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1103499235},{"type":"v1","addr":"172.21.15.115:6835","nonce":1103499235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:08:43.078263+0000","modified":"2023-12-01T03:08:46.096728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10149":{"gid":10149,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/3801423208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3801423208},{"type":"v1","addr":"172.21.15.142:6835","nonce":3801423208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-01T03:08:46.587 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 87 2023-12-01T03:08:46.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.587+0000 7f79affff640 1 -- 172.21.15.115:0/3586080161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f799006a580 msgr2=0x7f799006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.587+0000 7f79affff640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f799006a580 0x7f799006ca40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f79b00c7b10 tx=0x7f7998008040 comp rx=0 tx=0).stop 2023-12-01T03:08:46.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.587+0000 7f79affff640 1 -- 172.21.15.115:0/3586080161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79b006b7b0 msgr2=0x7f79b00c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.587+0000 7f79affff640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79b006b7b0 0x7f79b00c6ee0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c400 tx=0x7f79a000c8c0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.587+0000 7f79affff640 1 -- 172.21.15.115:0/3586080161 shutdown_connections 2023-12-01T03:08:46.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.587+0000 7f79affff640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f799006a580 0x7f799006ca40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.587+0000 7f79affff640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79b0145620 0x7f79b015be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.587+0000 7f79affff640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79b00cdd80 0x7f79b00c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.587+0000 7f79affff640 1 --2- 172.21.15.115:0/3586080161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79b006b7b0 0x7f79b00c6ee0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.588+0000 7f79affff640 1 -- 172.21.15.115:0/3586080161 >> 172.21.15.115:0/3586080161 conn(0x7f79b00ce640 msgr2=0x7f79b00c4390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:46.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.588+0000 7f79affff640 1 -- 172.21.15.115:0/3586080161 shutdown_connections 2023-12-01T03:08:46.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.588+0000 7f79affff640 1 -- 172.21.15.115:0/3586080161 wait complete. 2023-12-01T03:08:46.608 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:08:46.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.819+0000 7f01e7afb640 1 Processor -- start 2023-12-01T03:08:46.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.819+0000 7f01e7afb640 1 -- start start 2023-12-01T03:08:46.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.820+0000 7f01e7afb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f01e00ccc80 0x7f01e006fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.821+0000 7f01e7afb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f01e00ad2b0 0x7f01e006fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.821+0000 7f01e7afb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e015d600 con 0x7f01e015d220 2023-12-01T03:08:46.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.821+0000 7f01e7afb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e006cc10 con 0x7f01e00ad2b0 2023-12-01T03:08:46.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.821+0000 7f01e7afb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e006c720 con 0x7f01e00ccc80 2023-12-01T03:08:46.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.821+0000 7f01e6098640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f01e00ad2b0 0x7f01e006fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.821+0000 7f01e6098640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f01e00ad2b0 0x7f01e006fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51688/0 (socket says 172.21.15.115:51688) 2023-12-01T03:08:46.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.821+0000 7f01e6098640 1 -- 172.21.15.115:0/327212659 learned_addr learned my addr 172.21.15.115:0/327212659 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:46.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.821+0000 7f01e5096640 1 --2- 172.21.15.115:0/327212659 >> v2:172.21.15.142:3300/0 conn(0x7f01e00ccc80 0x7f01e006fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.822+0000 7f01e6098640 1 -- 172.21.15.115:0/327212659 >> v2:172.21.15.142:3300/0 conn(0x7f01e00ccc80 msgr2=0x7f01e006fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:46.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.822+0000 7f01e6098640 1 --2- 172.21.15.115:0/327212659 >> v2:172.21.15.142:3300/0 conn(0x7f01e00ccc80 0x7f01e006fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.822+0000 7f01e6098640 1 -- 172.21.15.115:0/327212659 >> v1:172.21.15.142:6789/0 conn(0x7f01e015d220 legacy=0x7f01e00cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:46.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.822+0000 7f01e6098640 1 -- 172.21.15.115:0/327212659 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e006c4b0 con 0x7f01e00ad2b0 2023-12-01T03:08:46.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.823+0000 7f01e6098640 1 --2- 172.21.15.115:0/327212659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e00ad2b0 0x7f01e006fff0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f01e00cc4e0 tx=0x7f01d802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8732eefa2505587d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:46.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.824+0000 7f01e4895640 1 -- 172.21.15.115:0/327212659 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80317e0 con 0x7f01e00ad2b0 2023-12-01T03:08:46.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.824+0000 7f01e4895640 1 -- 172.21.15.115:0/327212659 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8031940 con 0x7f01e00ad2b0 2023-12-01T03:08:46.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.824+0000 7f01e4895640 1 -- 172.21.15.115:0/327212659 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8031cd0 con 0x7f01e00ad2b0 2023-12-01T03:08:46.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.824+0000 7f01e7afb640 1 -- 172.21.15.115:0/327212659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e00ad2b0 msgr2=0x7f01e006fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.824+0000 7f01e7afb640 1 --2- 172.21.15.115:0/327212659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e00ad2b0 0x7f01e006fff0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f01e00cc4e0 tx=0x7f01d802b880 comp rx=0 tx=0).stop 2023-12-01T03:08:46.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.825+0000 7f01e7afb640 1 -- 172.21.15.115:0/327212659 shutdown_connections 2023-12-01T03:08:46.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.825+0000 7f01e7afb640 1 --2- 172.21.15.115:0/327212659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e00ad2b0 0x7f01e006fff0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.825+0000 7f01e7afb640 1 --2- 172.21.15.115:0/327212659 >> v2:172.21.15.142:3300/0 conn(0x7f01e00ccc80 0x7f01e006fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.825+0000 7f01e7afb640 1 -- 172.21.15.115:0/327212659 >> 172.21.15.115:0/327212659 conn(0x7f01e00ce880 msgr2=0x7f01e00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:46.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.825+0000 7f01e7afb640 1 -- 172.21.15.115:0/327212659 shutdown_connections 2023-12-01T03:08:46.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.825+0000 7f01e7afb640 1 -- 172.21.15.115:0/327212659 wait complete. 2023-12-01T03:08:46.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.826+0000 7f01e7afb640 1 Processor -- start 2023-12-01T03:08:46.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.827+0000 7f01e7afb640 1 -- start start 2023-12-01T03:08:46.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.827+0000 7f01e7afb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01e00ad2b0 0x7f01e014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.827+0000 7f01e7afb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01e00ccc80 0x7f01e00cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.827+0000 7f01e7afb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e015d220 0x7f01e00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.827+0000 7f01e7afb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0152dd0 con 0x7f01e015d220 2023-12-01T03:08:46.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.827+0000 7f01e7afb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00b3100 con 0x7f01e00ccc80 2023-12-01T03:08:46.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.827+0000 7f01e7afb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00b24f0 con 0x7f01e00ad2b0 2023-12-01T03:08:46.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.828+0000 7f01e6098640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e015d220 0x7f01e00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.828+0000 7f01e6098640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e015d220 0x7f01e00b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51702/0 (socket says 172.21.15.115:51702) 2023-12-01T03:08:46.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.828+0000 7f01e6098640 1 -- 172.21.15.115:0/779907161 learned_addr learned my addr 172.21.15.115:0/779907161 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:46.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.828+0000 7f01e5096640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01e00ccc80 0x7f01e00cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.828+0000 7f01e5897640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01e00ad2b0 0x7f01e014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.828+0000 7f01e6098640 1 -- 172.21.15.115:0/779907161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01e00ad2b0 msgr2=0x7f01e014c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.828+0000 7f01e6098640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01e00ad2b0 0x7f01e014c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.828+0000 7f01e6098640 1 -- 172.21.15.115:0/779907161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01e00ccc80 msgr2=0x7f01e00cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:46.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.828+0000 7f01e6098640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01e00ccc80 0x7f01e00cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:46.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.828+0000 7f01e6098640 1 -- 172.21.15.115:0/779907161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80094d0 con 0x7f01e015d220 2023-12-01T03:08:46.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.829+0000 7f01e6098640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e015d220 0x7f01e00b3590 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f01d80090b0 tx=0x7f01d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:46.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.829+0000 7f01d27fc640 1 -- 172.21.15.115:0/779907161 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8032880 con 0x7f01e015d220 2023-12-01T03:08:46.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.829+0000 7f01d27fc640 1 -- 172.21.15.115:0/779907161 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8032a00 con 0x7f01e015d220 2023-12-01T03:08:46.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.829+0000 7f01d27fc640 1 -- 172.21.15.115:0/779907161 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8032db0 con 0x7f01e015d220 2023-12-01T03:08:46.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.829+0000 7f01e7afb640 1 -- 172.21.15.115:0/779907161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e00b27d0 con 0x7f01e015d220 2023-12-01T03:08:46.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.829+0000 7f01e7afb640 1 -- 172.21.15.115:0/779907161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e00b2d90 con 0x7f01e015d220 2023-12-01T03:08:46.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.830+0000 7f01e7afb640 1 -- 172.21.15.115:0/779907161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e00b3270 con 0x7f01e015d220 2023-12-01T03:08:46.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.831+0000 7f01d27fc640 1 -- 172.21.15.115:0/779907161 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f01d80453f0 con 0x7f01e015d220 2023-12-01T03:08:46.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.833+0000 7f01d27fc640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01c406a550 0x7f01c406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:46.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.833+0000 7f01d27fc640 1 -- 172.21.15.115:0/779907161 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f01d80aac20 con 0x7f01e015d220 2023-12-01T03:08:46.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.833+0000 7f01e5897640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01c406a550 0x7f01c406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:46.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.834+0000 7f01e5897640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01c406a550 0x7f01c406ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f01d4006fd0 tx=0x7f01d4008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:46.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:46.837+0000 7f01d27fc640 1 -- 172.21.15.115:0/779907161 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f01d8076170 con 0x7f01e015d220 2023-12-01T03:08:47.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.027+0000 7f01e7afb640 1 -- 172.21.15.115:0/779907161 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01e015d600 con 0x7f01e015d220 2023-12-01T03:08:47.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.029+0000 7f01d27fc640 1 -- 172.21.15.115:0/779907161 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+16229 (secure 0 0 0) 0x7f01e015d600 con 0x7f01e015d220 2023-12-01T03:08:47.032 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:47.032 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":196,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:08:44.083122+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:08:40.709195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-01T03:08:42.071749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:08:47.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 -- 172.21.15.115:0/779907161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01c406a550 msgr2=0x7f01c406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01c406a550 0x7f01c406ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f01d4006fd0 tx=0x7f01d4008040 comp rx=0 tx=0).stop 2023-12-01T03:08:47.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 -- 172.21.15.115:0/779907161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e015d220 msgr2=0x7f01e00b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e015d220 0x7f01e00b3590 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f01d80090b0 tx=0x7f01d80046c0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 -- 172.21.15.115:0/779907161 shutdown_connections 2023-12-01T03:08:47.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01c406a550 0x7f01c406ca10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01e015d220 0x7f01e00b3590 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01e00ccc80 0x7f01e00cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 --2- 172.21.15.115:0/779907161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01e00ad2b0 0x7f01e014c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 -- 172.21.15.115:0/779907161 >> 172.21.15.115:0/779907161 conn(0x7f01e00ce880 msgr2=0x7f01e00ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:47.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 -- 172.21.15.115:0/779907161 shutdown_connections 2023-12-01T03:08:47.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.032+0000 7f01e7afb640 1 -- 172.21.15.115:0/779907161 wait complete. 2023-12-01T03:08:47.051 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T03:08:47.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:08:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.254+0000 7fb634539640 1 Processor -- start 2023-12-01T03:08:47.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.254+0000 7fb634539640 1 -- start start 2023-12-01T03:08:47.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.255+0000 7fb634539640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb62c15e2f0 0x7fb62c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.255+0000 7fb634539640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c0add10 con 0x7fb62c06f2f0 2023-12-01T03:08:47.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.255+0000 7fb634539640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c0c5760 con 0x7fb62c0adf10 2023-12-01T03:08:47.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.255+0000 7fb634539640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0c9cf0 con 0x7fb62c15e2f0 2023-12-01T03:08:47.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb631ad4640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb62c06f2f0 0x7fb62c0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:38286/0 (socket says 172.21.15.115:38286) 2023-12-01T03:08:47.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb631ad4640 1 -- 172.21.15.115:0/1068333774 learned_addr learned my addr 172.21.15.115:0/1068333774 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:47.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb6322d5640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb62c15e2f0 0x7fb62c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673770444 0 0) 0x7fb62c0add10 con 0x7fb62c06f2f0 2023-12-01T03:08:47.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb618003520 con 0x7fb62c06f2f0 2023-12-01T03:08:47.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1565828823 0 0) 0x7fb618003520 con 0x7fb62c06f2f0 2023-12-01T03:08:47.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb62c15e2f0 msgr2=0x7fb62c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:47.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb6312d3640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb62c15e2f0 0x7fb62c0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 >> v1:172.21.15.142:6789/0 conn(0x7fb62c0adf10 legacy=0x7fb62c0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c15e6d0 con 0x7fb62c06f2f0 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb61c001670 con 0x7fb62c06f2f0 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.256+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 >> v1:172.21.15.115:6789/0 conn(0x7fb62c06f2f0 legacy=0x7fb62c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb6312d3640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb62c15e2f0 0x7fb618003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb6312d3640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb618003f90 0x7fb6180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb6312d3640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb618004d00 0x7fb618005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0c9cf0 con 0x7fb618003f90 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61800bc10 con 0x7fb618004d00 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61800bdd0 con 0x7fb62c15e2f0 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb631ad4640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb618003f90 0x7fb6180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb631ad4640 1 -- 172.21.15.115:0/1068333774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb62c15e2f0 msgr2=0x7fb618003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb6322d5640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb62c15e2f0 0x7fb618003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb631ad4640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb62c15e2f0 0x7fb618003a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb631ad4640 1 -- 172.21.15.115:0/1068333774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb618004d00 msgr2=0x7fb618005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:47.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb631ad4640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb618004d00 0x7fb618005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.257+0000 7fb631ad4640 1 -- 172.21.15.115:0/1068333774 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb618003520 con 0x7fb618003f90 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.258+0000 7fb631ad4640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb618003f90 0x7fb6180047c0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00c240 tx=0x7fb6180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51613532447d5ee server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.259+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c005ad0 con 0x7fb618003f90 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.259+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c005c30 con 0x7fb618003f90 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.259+0000 7fb6312d3640 1 -- 172.21.15.115:0/1068333774 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c02caf0 con 0x7fb618003f90 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.259+0000 7fb634539640 1 -- 172.21.15.115:0/1068333774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb618003f90 msgr2=0x7fb6180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.259+0000 7fb634539640 1 --2- 172.21.15.115:0/1068333774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb618003f90 0x7fb6180047c0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00c240 tx=0x7fb6180057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.259+0000 7fb632ad6640 1 -- 172.21.15.115:0/1068333774 reap_dead start 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.259+0000 7fb634539640 1 -- 172.21.15.115:0/1068333774 shutdown_connections 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.259+0000 7fb634539640 1 -- 172.21.15.115:0/1068333774 >> 172.21.15.115:0/1068333774 conn(0x7fb62c0ce880 msgr2=0x7fb62c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.259+0000 7fb634539640 1 -- 172.21.15.115:0/1068333774 shutdown_connections 2023-12-01T03:08:47.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.259+0000 7fb634539640 1 -- 172.21.15.115:0/1068333774 wait complete. 2023-12-01T03:08:47.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.261+0000 7fb634539640 1 Processor -- start 2023-12-01T03:08:47.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.261+0000 7fb634539640 1 -- start start 2023-12-01T03:08:47.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.262+0000 7fb634539640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb618003f90 0x7fb62c0b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.262+0000 7fb634539640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb62c0c4a60 0x7fb62c0c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.262+0000 7fb634539640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb62c1508d0 0x7fb62c1f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.262+0000 7fb634539640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b1a60 con 0x7fb62c0c4a60 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.262+0000 7fb634539640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c1f3440 con 0x7fb62c1508d0 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.262+0000 7fb634539640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0c5200 con 0x7fb618003f90 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.262+0000 7fb631ad4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb62c0c4a60 0x7fb62c0c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.262+0000 7fb631ad4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb62c0c4a60 0x7fb62c0c2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51720/0 (socket says 172.21.15.115:51720) 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.262+0000 7fb631ad4640 1 -- 172.21.15.115:0/2584383330 learned_addr learned my addr 172.21.15.115:0/2584383330 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb6322d5640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb618003f90 0x7fb62c0b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb631ad4640 1 -- 172.21.15.115:0/2584383330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb618003f90 msgr2=0x7fb62c0b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb631ad4640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb618003f90 0x7fb62c0b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb632ad6640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb62c1508d0 0x7fb62c1f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb631ad4640 1 -- 172.21.15.115:0/2584383330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb62c1508d0 msgr2=0x7fb62c1f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb631ad4640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb62c1508d0 0x7fb62c1f2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb631ad4640 1 -- 172.21.15.115:0/2584383330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0c4f20 con 0x7fb62c0c4a60 2023-12-01T03:08:47.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb631ad4640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb62c0c4a60 0x7fb62c0c2fd0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00c240 tx=0x7fb61c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:47.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb622ffd640 1 -- 172.21.15.115:0/2584383330 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c039070 con 0x7fb62c0c4a60 2023-12-01T03:08:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb622ffd640 1 -- 172.21.15.115:0/2584383330 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c02ccc0 con 0x7fb62c0c4a60 2023-12-01T03:08:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb622ffd640 1 -- 172.21.15.115:0/2584383330 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c03c040 con 0x7fb62c0c4a60 2023-12-01T03:08:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.263+0000 7fb634539640 1 -- 172.21.15.115:0/2584383330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c1e3810 con 0x7fb62c0c4a60 2023-12-01T03:08:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.264+0000 7fb634539640 1 -- 172.21.15.115:0/2584383330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c1e3dd0 con 0x7fb62c0c4a60 2023-12-01T03:08:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.265+0000 7fb634539640 1 -- 172.21.15.115:0/2584383330 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c06fda0 con 0x7fb62c0c4a60 2023-12-01T03:08:47.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.265+0000 7fb622ffd640 1 -- 172.21.15.115:0/2584383330 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb61c032070 con 0x7fb62c0c4a60 2023-12-01T03:08:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.266+0000 7fb622ffd640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb61006a550 0x7fb61006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.266+0000 7fb622ffd640 1 -- 172.21.15.115:0/2584383330 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fb61c0a9ed0 con 0x7fb62c0c4a60 2023-12-01T03:08:47.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.267+0000 7fb6322d5640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb61006a550 0x7fb61006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.267+0000 7fb6322d5640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb61006a550 0x7fb61006ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb628009910 tx=0x7fb628008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:47.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.271+0000 7fb622ffd640 1 -- 172.21.15.115:0/2584383330 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb61c075380 con 0x7fb62c0c4a60 2023-12-01T03:08:47.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.456+0000 7fb634539640 1 -- 172.21.15.115:0/2584383330 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb62c0cf4d0 con 0x7fb61006a550 2023-12-01T03:08:47.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.483+0000 7fb622ffd640 1 -- 172.21.15.115:0/2584383330 <== mgr.9903 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275795 (secure 0 0 0) 0x7fb6280118e0 con 0x7fb61006a550 2023-12-01T03:08:47.487 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:47.490 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:08:47.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 -- 172.21.15.115:0/2584383330 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb61006a550 msgr2=0x7fb61006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb61006a550 0x7fb61006ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb628009910 tx=0x7fb628008040 comp rx=0 tx=0).stop 2023-12-01T03:08:47.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 -- 172.21.15.115:0/2584383330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb62c0c4a60 msgr2=0x7fb62c0c2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb62c0c4a60 0x7fb62c0c2fd0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00c240 tx=0x7fb61c001a00 comp rx=0 tx=0).stop 2023-12-01T03:08:47.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 -- 172.21.15.115:0/2584383330 shutdown_connections 2023-12-01T03:08:47.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb61006a550 0x7fb61006ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb62c1508d0 0x7fb62c1f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb62c0c4a60 0x7fb62c0c2fd0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 --2- 172.21.15.115:0/2584383330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb618003f90 0x7fb62c0b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 -- 172.21.15.115:0/2584383330 >> 172.21.15.115:0/2584383330 conn(0x7fb62c0ce880 msgr2=0x7fb62c0c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:47.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 -- 172.21.15.115:0/2584383330 shutdown_connections 2023-12-01T03:08:47.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.490+0000 7fb634539640 1 -- 172.21.15.115:0/2584383330 wait complete. 2023-12-01T03:08:47.511 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-01T03:08:45.851739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":370860,"kb_used_data":19948,"kb_used_omap":0,"kb_used_meta":350848,"kb_avail":754603860,"statfs":{"total":773094113280,"available":772714352640,"internally_reserved":0,"allocated":20426752,"data_stored":10517735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":359268352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.271945"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.592231+0000","last_change":"2023-12-01T03:08:41.592231+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.592231+0000","last_undegraded":"2023-12-01T03:08:41.592231+0000","last_fullsized":"2023-12-01T03:08:41.592231+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:07.847258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.598468+0000","last_change":"2023-12-01T03:08:41.598468+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.598468+0000","last_undegraded":"2023-12-01T03:08:41.598468+0000","last_fullsized":"2023-12-01T03:08:41.598468+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:42:50.607707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.588627+0000","last_change":"2023-12-01T03:08:41.588627+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.588627+0000","last_undegraded":"2023-12-01T03:08:41.588627+0000","last_fullsized":"2023-12-01T03:08:41.588627+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:09:31.296149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.587644+0000","last_change":"2023-12-01T03:08:41.587644+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.587644+0000","last_undegraded":"2023-12-01T03:08:41.587644+0000","last_fullsized":"2023-12-01T03:08:41.587644+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:35:23.311979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.591440+0000","last_change":"2023-12-01T03:08:41.591440+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.591440+0000","last_undegraded":"2023-12-01T03:08:41.591440+0000","last_fullsized":"2023-12-01T03:08:41.591440+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:48.817809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.585750+0000","last_change":"2023-12-01T03:08:41.585750+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.585750+0000","last_undegraded":"2023-12-01T03:08:41.585750+0000","last_fullsized":"2023-12-01T03:08:41.585750+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:07.847258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.592020+0000","last_change":"2023-12-01T03:08:41.592020+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.592020+0000","last_undegraded":"2023-12-01T03:08:41.592020+0000","last_fullsized":"2023-12-01T03:08:41.592020+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:40.615426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.590148+0000","last_change":"2023-12-01T03:08:41.590148+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.590148+0000","last_undegraded":"2023-12-01T03:08:41.590148+0000","last_fullsized":"2023-12-01T03:08:41.590148+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:12.741584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.588830+0000","last_change":"2023-12-01T03:08:41.588830+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.588830+0000","last_undegraded":"2023-12-01T03:08:41.588830+0000","last_fullsized":"2023-12-01T03:08:41.588830+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:03:24.457743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.588785+0000","last_change":"2023-12-01T03:08:41.588785+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.588785+0000","last_undegraded":"2023-12-01T03:08:41.588785+0000","last_fullsized":"2023-12-01T03:08:41.588785+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:11.213037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.592254+0000","last_change":"2023-12-01T03:08:41.592254+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.592254+0000","last_undegraded":"2023-12-01T03:08:41.592254+0000","last_fullsized":"2023-12-01T03:08:41.592254+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:02:47.336331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.585967+0000","last_change":"2023-12-01T03:08:41.585967+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.585967+0000","last_undegraded":"2023-12-01T03:08:41.585967+0000","last_fullsized":"2023-12-01T03:08:41.585967+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:38:56.058423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.589841+0000","last_change":"2023-12-01T03:08:41.589841+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.589841+0000","last_undegraded":"2023-12-01T03:08:41.589841+0000","last_fullsized":"2023-12-01T03:08:41.589841+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:31:47.243234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.593399+0000","last_change":"2023-12-01T03:08:41.593399+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.593399+0000","last_undegraded":"2023-12-01T03:08:41.593399+0000","last_fullsized":"2023-12-01T03:08:41.593399+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:42:50.607707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.599147+0000","last_change":"2023-12-01T03:08:41.599147+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.599147+0000","last_undegraded":"2023-12-01T03:08:41.599147+0000","last_fullsized":"2023-12-01T03:08:41.599147+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:28:30.128979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.593256+0000","last_change":"2023-12-01T03:08:41.593256+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.593256+0000","last_undegraded":"2023-12-01T03:08:41.593256+0000","last_fullsized":"2023-12-01T03:08:41.593256+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:42:24.909711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.596896+0000","last_change":"2023-12-01T03:08:41.596896+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.596896+0000","last_undegraded":"2023-12-01T03:08:41.596896+0000","last_fullsized":"2023-12-01T03:08:41.596896+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:49.756181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.593930+0000","last_change":"2023-12-01T03:08:41.593930+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.593930+0000","last_undegraded":"2023-12-01T03:08:41.593930+0000","last_fullsized":"2023-12-01T03:08:41.593930+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:38.313581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.595664+0000","last_change":"2023-12-01T03:08:41.595664+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.595664+0000","last_undegraded":"2023-12-01T03:08:41.595664+0000","last_fullsized":"2023-12-01T03:08:41.595664+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:07:58.560943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.588499+0000","last_change":"2023-12-01T03:08:41.588499+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.588499+0000","last_undegraded":"2023-12-01T03:08:41.588499+0000","last_fullsized":"2023-12-01T03:08:41.588499+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:31.484313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.594943+0000","last_change":"2023-12-01T03:08:41.594943+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.594943+0000","last_undegraded":"2023-12-01T03:08:41.594943+0000","last_fullsized":"2023-12-01T03:08:41.594943+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:25:35.364206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.586832+0000","last_change":"2023-12-01T03:08:41.586832+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.586832+0000","last_undegraded":"2023-12-01T03:08:41.586832+0000","last_fullsized":"2023-12-01T03:08:41.586832+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:40.615426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.595128+0000","last_change":"2023-12-01T03:08:41.595128+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.595128+0000","last_undegraded":"2023-12-01T03:08:41.595128+0000","last_fullsized":"2023-12-01T03:08:41.595128+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:20:14.369832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.589016+0000","last_change":"2023-12-01T03:08:41.589016+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.589016+0000","last_undegraded":"2023-12-01T03:08:41.589016+0000","last_fullsized":"2023-12-01T03:08:41.589016+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:26:59.122192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.598059+0000","last_change":"2023-12-01T03:08:41.598059+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.598059+0000","last_undegraded":"2023-12-01T03:08:41.598059+0000","last_fullsized":"2023-12-01T03:08:41.598059+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:05.817369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.592376+0000","last_change":"2023-12-01T03:08:41.592376+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.592376+0000","last_undegraded":"2023-12-01T03:08:41.592376+0000","last_fullsized":"2023-12-01T03:08:41.592376+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:31.232706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.586311+0000","last_change":"2023-12-01T03:08:41.586311+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.586311+0000","last_undegraded":"2023-12-01T03:08:41.586311+0000","last_fullsized":"2023-12-01T03:08:41.586311+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:51:45.640915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.594228+0000","last_change":"2023-12-01T03:08:41.594228+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.594228+0000","last_undegraded":"2023-12-01T03:08:41.594228+0000","last_fullsized":"2023-12-01T03:08:41.594228+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:27:39.871916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.592515+0000","last_change":"2023-12-01T03:08:41.592515+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.592515+0000","last_undegraded":"2023-12-01T03:08:41.592515+0000","last_fullsized":"2023-12-01T03:08:41.592515+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:49.756181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.591436+0000","last_change":"2023-12-01T03:08:41.591436+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.591436+0000","last_undegraded":"2023-12-01T03:08:41.591436+0000","last_fullsized":"2023-12-01T03:08:41.591436+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:35.903418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.592766+0000","last_change":"2023-12-01T03:08:41.592766+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.592766+0000","last_undegraded":"2023-12-01T03:08:41.592766+0000","last_fullsized":"2023-12-01T03:08:41.592766+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:38:56.058423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:08:42.577501+0000","last_change":"2023-12-01T03:08:42.577501+0000","last_active":"2023-12-01T03:08:42.577501+0000","last_peered":"2023-12-01T03:08:42.577501+0000","last_clean":"2023-12-01T03:08:42.577501+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:42.577501+0000","last_undegraded":"2023-12-01T03:08:42.577501+0000","last_fullsized":"2023-12-01T03:08:42.577501+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":193,"state":"creating+peering","last_fresh":"2023-12-01T03:08:41.601060+0000","last_change":"2023-12-01T03:08:41.601060+0000","last_active":"2023-12-01T03:08:41.561384+0000","last_peered":"2023-12-01T03:08:41.561384+0000","last_clean":"2023-12-01T03:08:41.561384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:08:41.601060+0000","last_undegraded":"2023-12-01T03:08:41.601060+0000","last_fullsized":"2023-12-01T03:08:41.601060+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:02:14.580168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"184'304","reported_seq":1021,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-01T03:08:38.972500+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:08:38.972500+0000","last_peered":"2023-12-01T03:08:38.972500+0000","last_clean":"2023-12-01T03:08:38.972500+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:08:38.972500+0000","last_undegraded":"2023-12-01T03:08:38.972500+0000","last_fullsized":"2023-12-01T03:08:38.972500+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55536,"kb_used_data":2344,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316304,"statfs":{"total":96636764160,"available":96579895296,"internally_reserved":0,"allocated":2400256,"data_stored":1164927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0900000000000001,"15min":1.0860000000000001},"min":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.71299999999999997},"max":{"1min":1.6000000000000001,"5min":1.7390000000000001,"15min":1.98},"last":1.179},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.131,"15min":1.125},"min":{"1min":0.85499999999999998,"5min":0.80300000000000005,"15min":0.754},"max":{"1min":1.6699999999999999,"5min":1.7,"15min":1.7},"last":0.85699999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3420000000000001,"15min":1.306},"min":{"1min":1.018,"5min":0.90700000000000003,"15min":0.73099999999999998},"max":{"1min":1.673,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.331,"15min":1.3400000000000001},"min":{"1min":1.1220000000000001,"5min":0.871,"15min":0.871},"max":{"1min":1.5429999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.1559999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1970000000000001,"15min":1.1739999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.76700000000000002},"max":{"1min":1.6020000000000001,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.262,"15min":1.2509999999999999},"min":{"1min":0.95699999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.526,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.89800000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.292,"15min":1.3089999999999999},"min":{"1min":1.087,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.613,"5min":2.431,"15min":2.431},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4470000000000001,"15min":1.4279999999999999},"min":{"1min":1.1719999999999999,"5min":1.032,"15min":0.92900000000000005},"max":{"1min":1.7909999999999999,"5min":2.198,"15min":2.198},"last":1.0109999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3220000000000001,"15min":1.3320000000000001},"min":{"1min":1.0549999999999999,"5min":0.82599999999999996,"15min":0.81200000000000006},"max":{"1min":1.6859999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":1.222},{"interface":"front","average":{"1min":1.571,"5min":1.5189999999999999,"15min":1.4990000000000001},"min":{"1min":1.3009999999999999,"5min":0.92800000000000005,"15min":0.79500000000000004},"max":{"1min":1.853,"5min":2.569,"15min":2.569},"last":1.3959999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3220000000000001,"15min":1.3049999999999999},"min":{"1min":0.94699999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.6739999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.609,"15min":1.595},"min":{"1min":1.401,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.9079999999999999,"5min":2.484,"15min":2.484},"last":1.446}]},{"osd":6,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.4370000000000001,"15min":1.4430000000000001},"min":{"1min":1.2030000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7829999999999999,"5min":2.532,"15min":2.532},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4419999999999999,"15min":1.46},"min":{"1min":1.091,"5min":1.0189999999999999,"15min":0.97999999999999998},"max":{"1min":1.8819999999999999,"5min":2.2749999999999999,"15min":2.762},"last":1.3580000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37500,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334340,"statfs":{"total":96636764160,"available":96598364160,"internally_reserved":0,"allocated":2412544,"data_stored":1170788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1080000000000001,"15min":1.127},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.361,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.1359999999999999,"15min":1.1759999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5720000000000001,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.085}]},{"osd":1,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1890000000000001,"15min":1.2230000000000001},"min":{"1min":0.88,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5740000000000001,"5min":1.7789999999999999,"15min":2.0449999999999999},"last":1.242},{"interface":"front","average":{"1min":1.212,"5min":1.244,"15min":1.228},"min":{"1min":0.85599999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.548,"5min":1.835,"15min":1.835},"last":1.3460000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.216,"15min":1.2210000000000001},"min":{"1min":0.92900000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5249999999999999,"5min":1.976,"15min":2.1469999999999998},"last":1.298},{"interface":"front","average":{"1min":1.264,"5min":1.2949999999999999,"15min":1.29},"min":{"1min":1.0029999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.625,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.9339999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.331,"15min":1.3300000000000001},"min":{"1min":1.052,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.637,"5min":1.8480000000000001,"15min":2.2669999999999999},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.423,"15min":1.4279999999999999},"min":{"1min":0.997,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.613,"5min":1.7949999999999999,"15min":1.923},"last":1.5920000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2310000000000001,"15min":1.2729999999999999},"min":{"1min":0.90000000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.476,"5min":1.944,"15min":2.0819999999999999},"last":1.633},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.339,"15min":1.349},"min":{"1min":1.042,"5min":0.89100000000000001,"15min":0.82699999999999996},"max":{"1min":1.6619999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":2.0409999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3859999999999999,"15min":1.423},"min":{"1min":1.036,"5min":0.88,"15min":0.873},"max":{"1min":1.7050000000000001,"5min":2.1360000000000001,"15min":2.1739999999999999},"last":1.53},{"interface":"front","average":{"1min":1.423,"5min":1.4810000000000001,"15min":1.488},"min":{"1min":1.1299999999999999,"5min":0.95599999999999996,"15min":0.93999999999999995},"max":{"1min":1.7110000000000001,"5min":1.9059999999999999,"15min":2.0379999999999998},"last":1.7050000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.508,"15min":1.536},"min":{"1min":1.161,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":1.6870000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":2.2200000000000002},{"interface":"front","average":{"1min":1.498,"5min":1.6000000000000001,"15min":1.589},"min":{"1min":1.2230000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.669,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.845}]}]},{"osd":5,"up_from":13,"seq":55834575956,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47924,"kb_used_data":2924,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323916,"statfs":{"total":96636764160,"available":96587689984,"internally_reserved":0,"allocated":2994176,"data_stored":1755295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.115,"15min":1.131},"min":{"1min":0.72499999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.218,"5min":1.6559999999999999,"15min":1.8899999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.181,"5min":1.135,"15min":1.1739999999999999},"min":{"1min":0.96199999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.415,"5min":1.752,"15min":1.8009999999999999},"last":1.0920000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.165,"15min":1.1599999999999999},"min":{"1min":0.81699999999999995,"5min":0.81200000000000006,"15min":0.76500000000000001},"max":{"1min":1.613,"5min":1.696,"15min":1.7849999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.234,"15min":1.2490000000000001},"min":{"1min":0.95299999999999996,"5min":0.80200000000000005,"15min":0.60899999999999999},"max":{"1min":1.454,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.155}]},{"osd":2,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.5700000000000001,"5min":1.575,"15min":1.8129999999999999},"last":1.264},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.262,"15min":1.2709999999999999},"min":{"1min":0.92200000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.635,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.014}]},{"osd":3,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.296,"15min":1.3220000000000001},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.93100000000000005},"max":{"1min":1.5920000000000001,"5min":1.8979999999999999,"15min":1.994},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.3700000000000001,"15min":1.371},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":0.90000000000000002},"max":{"1min":1.7709999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.1279999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.2569999999999999,"15min":1.254},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.74099999999999999},"max":{"1min":1.5409999999999999,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.381,"15min":1.3919999999999999},"min":{"1min":0.94599999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.6919999999999999,"5min":1.8680000000000001,"15min":1.931},"last":1.3169999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.383,"15min":1.371},"min":{"1min":0.93200000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.718,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.464},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.4059999999999999,"15min":1.411},"min":{"1min":0.95199999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.6379999999999999,"5min":2.577,"15min":2.577},"last":1.23}]},{"osd":7,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5409999999999999,"15min":1.54},"min":{"1min":1.282,"5min":0.92800000000000005,"15min":0.91700000000000004},"max":{"1min":1.663,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.593,"15min":1.5740000000000001},"min":{"1min":1.2330000000000001,"5min":1.03,"15min":1.014},"max":{"1min":1.893,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.401}]}]},{"osd":4,"up_from":13,"seq":55834575956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47356,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324484,"statfs":{"total":96636764160,"available":96588271616,"internally_reserved":0,"allocated":2412544,"data_stored":1170788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.125,"15min":1.1379999999999999},"min":{"1min":0.77000000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.421,"5min":1.607,"15min":1.6759999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1559999999999999,"15min":1.171},"min":{"1min":0.85499999999999998,"5min":0.78700000000000003,"15min":0.77500000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.704},"last":1.226}]},{"osd":1,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.2430000000000001,"15min":1.2629999999999999},"min":{"1min":0.97599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.581,"5min":1.629,"15min":1.8640000000000001},"last":1.506},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.198,"15min":1.1879999999999999},"min":{"1min":0.85299999999999998,"5min":0.84099999999999997,"15min":0.72899999999999998},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.002}]},{"osd":2,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2190000000000001,"15min":1.2090000000000001},"min":{"1min":0.81699999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.613,"5min":1.613,"15min":1.6739999999999999},"last":1.446},{"interface":"front","average":{"1min":1.389,"5min":1.363,"15min":1.343},"min":{"1min":1.145,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.637,"5min":1.786,"15min":1.9450000000000001},"last":1.399}]},{"osd":3,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2869999999999999,"15min":1.282},"min":{"1min":0.97899999999999998,"5min":0.88300000000000001,"15min":0.879},"max":{"1min":1.77,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.3220000000000001,"15min":1.329},"min":{"1min":1.0209999999999999,"5min":0.88,"15min":0.88},"max":{"1min":1.609,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.2649999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3220000000000001,"15min":1.3069999999999999},"min":{"1min":1.0309999999999999,"5min":0.89600000000000002,"15min":0.755},"max":{"1min":1.7529999999999999,"5min":1.7789999999999999,"15min":1.841},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.331,"15min":1.294},"min":{"1min":1.0069999999999999,"5min":0.91400000000000003,"15min":0.86699999999999999},"max":{"1min":1.716,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.48}]},{"osd":6,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3580000000000001,"15min":1.367},"min":{"1min":1.1299999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5269999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.542},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4570000000000001,"15min":1.466},"min":{"1min":1.2150000000000001,"5min":1.071,"15min":0.98699999999999999},"max":{"1min":1.7689999999999999,"5min":1.853,"15min":2.0089999999999999},"last":1.675}]},{"osd":7,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.49,"15min":1.476},"min":{"1min":1.212,"5min":1.0389999999999999,"15min":0.93799999999999994},"max":{"1min":1.909,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5529999999999999,"15min":1.5660000000000001},"min":{"1min":1.327,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":1.9019999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.629}]}]},{"osd":3,"up_from":13,"seq":55834575955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51440,"kb_used_data":2344,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320400,"statfs":{"total":96636764160,"available":96584089600,"internally_reserved":0,"allocated":2400256,"data_stored":1164927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.92300000000000004,"15min":0.91100000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.423,"5min":1.4650000000000001,"15min":1.476},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.02,"5min":1.1160000000000001,"15min":1.0880000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.542,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.3440000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.169,"15min":1.173},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4670000000000001,"5min":1.625,"15min":1.667},"last":1.655},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1,"15min":0.998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.41099999999999998},"max":{"1min":1.4410000000000001,"5min":1.899,"15min":1.899},"last":0.999}]},{"osd":2,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0920000000000001,"15min":1.105},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.379,"5min":1.647,"15min":1.8200000000000001},"last":1.258},{"interface":"front","average":{"1min":1.218,"5min":1.216,"15min":1.252},"min":{"1min":0.747,"5min":0.747,"15min":0.58599999999999997},"max":{"1min":1.5529999999999999,"5min":1.611,"15min":1.9219999999999999},"last":1.107}]},{"osd":4,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.294,"15min":1.2949999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.5820000000000001,"5min":1.615,"15min":1.6599999999999999},"last":1.179},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.268,"15min":1.292},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.476,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.2929999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.306,"15min":1.327},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.55,"5min":1.7829999999999999,"15min":1.8979999999999999},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.351,"15min":1.371},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":2.1779999999999999},"last":1.54}]},{"osd":6,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.4379999999999999,"15min":1.448},"min":{"1min":1.046,"5min":1.03,"15min":0.78400000000000003},"max":{"1min":1.5229999999999999,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.464},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.5029999999999999,"15min":1.5129999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.8859999999999999,"5min":2.0819999999999999,"15min":2.3660000000000001},"last":1.486}]},{"osd":7,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.514,"15min":1.528},"min":{"1min":1.1259999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":1.9370000000000001,"5min":1.9870000000000001,"15min":2.3109999999999999},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.53,"15min":1.5269999999999999},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.878},"max":{"1min":1.738,"5min":2.984,"15min":2.984},"last":1.7}]}]},{"osd":2,"up_from":13,"seq":55834575955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41596,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330244,"statfs":{"total":96636764160,"available":96594169856,"internally_reserved":0,"allocated":2412544,"data_stored":1170788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91400000000000003,"15min":0.89500000000000002},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.60399999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.4730000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.93300000000000005,"5min":1.0649999999999999,"15min":1.0589999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.329,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.208}]},{"osd":1,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0620000000000001,"15min":1.0720000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3959999999999999,"5min":1.698,"15min":1.698},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.071,"5min":1.147,"15min":1.1579999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5800000000000001,"5min":1.635,"15min":1.71},"last":1.5800000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2130000000000001,"15min":1.23},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.615,"5min":1.6879999999999999,"15min":1.923},"last":1.4890000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.347,"15min":1.3420000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.252,"5min":1.2989999999999999,"15min":1.3},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.617,"5min":1.702,"15min":1.835},"last":1.4339999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.405,"15min":1.397},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7989999999999999,"5min":1.976,"15min":2.2040000000000002},"last":1.625},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.421,"15min":1.4179999999999999},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":2.016},"last":1.8260000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3759999999999999,"15min":1.3819999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":2.0289999999999999},"last":1.8819999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.514,"15min":1.5089999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":2.3010000000000002},"last":1.9590000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.504,"15min":1.494},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":2.3170000000000002},{"interface":"front","average":{"1min":1.379,"5min":1.427,"15min":1.466},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":2.0670000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575954,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47924,"kb_used_data":2924,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323916,"statfs":{"total":96636764160,"available":96587689984,"internally_reserved":0,"allocated":2994176,"data_stored":1755295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.89400000000000002,"15min":0.91800000000000004},"min":{"1min":0.66100000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.0429999999999999,"15min":1.0580000000000001},"min":{"1min":0.77700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.4530000000000001,"5min":1.617,"15min":1.776},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.96999999999999997,"15min":0.96699999999999997},"min":{"1min":0.85599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.157,"5min":1.159,"15min":1.159},"min":{"1min":0.78700000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6080000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.93600000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3300000000000001,"15min":1.329},"min":{"1min":0.90700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.587,"5min":1.788,"15min":1.9159999999999999},"last":1.069},{"interface":"front","average":{"1min":1.147,"5min":1.145,"15min":1.151},"min":{"1min":0.79500000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.8049999999999999},"last":1.177}]},{"osd":4,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2110000000000001,"15min":1.2250000000000001},"min":{"1min":1.0049999999999999,"5min":0.92300000000000004,"15min":0.84999999999999998},"max":{"1min":1.478,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.425,"15min":1.419},"min":{"1min":1.085,"5min":1.085,"15min":1.042},"max":{"1min":1.8620000000000001,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":1.143}]},{"osd":5,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.353,"15min":1.351},"min":{"1min":0.95699999999999996,"5min":0.92800000000000005,"15min":0.89000000000000001},"max":{"1min":1.665,"5min":2.0379999999999998,"15min":2.1160000000000001},"last":1.325},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3560000000000001,"15min":1.3640000000000001},"min":{"1min":1.1120000000000001,"5min":1.014,"15min":0.94399999999999995},"max":{"1min":1.927,"5min":1.927,"15min":2.1659999999999999},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.466,"15min":1.4490000000000001},"min":{"1min":1.1279999999999999,"5min":1.036,"15min":0.93000000000000005},"max":{"1min":1.6399999999999999,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.409,"15min":1.4279999999999999},"min":{"1min":1.2230000000000001,"5min":1.0149999999999999,"15min":0.90200000000000002},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0209999999999999},"last":1.4690000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.552,"15min":1.5429999999999999},"min":{"1min":1.3180000000000001,"5min":1.1419999999999999,"15min":1.1140000000000001},"max":{"1min":1.9970000000000001,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.228},{"interface":"front","average":{"1min":1.5,"5min":1.5189999999999999,"15min":1.5269999999999999},"min":{"1min":1.282,"5min":1.103,"15min":0.79000000000000004},"max":{"1min":1.7709999999999999,"5min":2.2069999999999999,"15min":2.3090000000000002},"last":1.383}]}]},{"osd":0,"up_from":13,"seq":55834575956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41584,"kb_used_data":2344,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330256,"statfs":{"total":96636764160,"available":96594182144,"internally_reserved":0,"allocated":2400256,"data_stored":1164927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.91100000000000003,"15min":0.90400000000000003},"min":{"1min":0.68400000000000005,"5min":0.435,"15min":0.38300000000000001},"max":{"1min":1.262,"5min":1.994,"15min":1.994},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0660000000000001},"min":{"1min":0.72399999999999998,"5min":0.46400000000000002,"15min":0.439},"max":{"1min":1.383,"5min":1.6379999999999999,"15min":1.7050000000000001},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0669999999999999,"15min":1.0740000000000001},"min":{"1min":0.84599999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.331,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.86599999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.675,"5min":1.792,"15min":1.792},"last":1.0249999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.0920000000000001,"15min":1.083},"min":{"1min":0.749,"5min":0.65600000000000003,"15min":0.52900000000000003},"max":{"1min":1.4239999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.24,"5min":1.2450000000000001,"15min":1.244},"min":{"1min":0.89000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6379999999999999,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":0.83499999999999996}]},{"osd":4,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.369,"15min":1.3660000000000001},"min":{"1min":0.998,"5min":0.998,"15min":0.74099999999999999},"max":{"1min":1.5960000000000001,"5min":2.085,"15min":2.1400000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.3440000000000001,"15min":1.3140000000000001},"min":{"1min":1.0720000000000001,"5min":0.84699999999999998,"15min":0.626},"max":{"1min":1.74,"5min":2.3210000000000002,"15min":2.3610000000000002},"last":1.1160000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4330000000000001,"15min":1.4159999999999999},"min":{"1min":1.228,"5min":0.88400000000000001,"15min":0.82599999999999996},"max":{"1min":1.8859999999999999,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.385},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.45,"15min":1.444},"min":{"1min":1.0900000000000001,"5min":0.79600000000000004,"15min":0.69599999999999995},"max":{"1min":1.9690000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.159}]},{"osd":6,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4039999999999999,"15min":1.3879999999999999},"min":{"1min":1.141,"5min":0.88300000000000001,"15min":0.76800000000000002},"max":{"1min":1.8169999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.4990000000000001,"15min":1.4950000000000001},"min":{"1min":1.1639999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.9419999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.296}]},{"osd":7,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.546,"15min":1.532},"min":{"1min":1.1799999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.0390000000000001,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.431},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.498,"15min":1.4850000000000001},"min":{"1min":1.28,"5min":1.032,"15min":0.72599999999999998},"max":{"1min":1.9059999999999999,"5min":2.431,"15min":2.431},"last":1.226}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:08:47.516 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:08:47.546 INFO:tasks.ceph.mds.d.smithi142.stderr:2023-12-01T03:08:47.544+0000 7f2892215640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:08:47.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.762+0000 7f90bffff640 1 Processor -- start 2023-12-01T03:08:47.764 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.763+0000 7f90bffff640 1 -- start start 2023-12-01T03:08:47.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.764+0000 7f90bffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f90c015db90 0x7f90c00cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.764+0000 7f90bffff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c015e020 con 0x7f90c015d7b0 2023-12-01T03:08:47.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.764+0000 7f90bffff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c00cd3f0 con 0x7f90c00cddc0 2023-12-01T03:08:47.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.764+0000 7f90bffff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c00cd0f0 con 0x7f90c015db90 2023-12-01T03:08:47.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.765+0000 7f90b7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f90c015db90 0x7f90c00cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.765+0000 7f90beffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f90c015d7b0 0x7f90c0160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51510/0 (socket says 172.21.15.115:51510) 2023-12-01T03:08:47.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.765+0000 7f90b7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f90c015db90 0x7f90c00cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58206/0 (socket says 172.21.15.115:58206) 2023-12-01T03:08:47.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.765+0000 7f90beffd640 1 -- 172.21.15.115:0/1303315271 learned_addr learned my addr 172.21.15.115:0/1303315271 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:47.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.765+0000 7f90b7fff640 1 -- 172.21.15.115:0/1303315271 >> v1:172.21.15.142:6789/0 conn(0x7f90c00cddc0 legacy=0x7f90c00d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:47.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.765+0000 7f90b7fff640 1 -- 172.21.15.115:0/1303315271 >> v1:172.21.15.115:6789/0 conn(0x7f90c015d7b0 legacy=0x7f90c0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.765+0000 7f90b7fff640 1 -- 172.21.15.115:0/1303315271 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c0160ec0 con 0x7f90c015db90 2023-12-01T03:08:47.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.766+0000 7f90b7fff640 1 --2- 172.21.15.115:0/1303315271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 0x7f90c00cd880 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f90a80097a0 tx=0x7f90a802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bd8e019e7679eb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:47.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.766+0000 7f90be7fc640 1 -- 172.21.15.115:0/1303315271 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a8030610 con 0x7f90c015db90 2023-12-01T03:08:47.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.766+0000 7f90be7fc640 1 -- 172.21.15.115:0/1303315271 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a8030770 con 0x7f90c015db90 2023-12-01T03:08:47.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.766+0000 7f90be7fc640 1 -- 172.21.15.115:0/1303315271 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a8030b00 con 0x7f90c015db90 2023-12-01T03:08:47.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.767+0000 7f90bffff640 1 -- 172.21.15.115:0/1303315271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 msgr2=0x7f90c00cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.767+0000 7f90bffff640 1 --2- 172.21.15.115:0/1303315271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 0x7f90c00cd880 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f90a80097a0 tx=0x7f90a802b960 comp rx=0 tx=0).stop 2023-12-01T03:08:47.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.767+0000 7f90bffff640 1 -- 172.21.15.115:0/1303315271 shutdown_connections 2023-12-01T03:08:47.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.767+0000 7f90bffff640 1 --2- 172.21.15.115:0/1303315271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 0x7f90c00cd880 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.767+0000 7f90bffff640 1 -- 172.21.15.115:0/1303315271 >> 172.21.15.115:0/1303315271 conn(0x7f90c006a0e0 msgr2=0x7f90c006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:47.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.767+0000 7f90bffff640 1 -- 172.21.15.115:0/1303315271 shutdown_connections 2023-12-01T03:08:47.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.767+0000 7f90bffff640 1 -- 172.21.15.115:0/1303315271 wait complete. 2023-12-01T03:08:47.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.769+0000 7f90bffff640 1 Processor -- start 2023-12-01T03:08:47.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.769+0000 7f90bffff640 1 -- start start 2023-12-01T03:08:47.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.770+0000 7f90bffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90c00cddc0 0x7f90c015c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.770+0000 7f90bffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c015d7b0 0x7f90c0156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.770+0000 7f90bffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 0x7f90c01f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.770+0000 7f90bffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c0156ff0 con 0x7f90c015db90 2023-12-01T03:08:47.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.770+0000 7f90bffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c01565d0 con 0x7f90c015d7b0 2023-12-01T03:08:47.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.770+0000 7f90bffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c01f2f20 con 0x7f90c00cddc0 2023-12-01T03:08:47.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.770+0000 7f90bf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 0x7f90c01f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.770+0000 7f90bf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 0x7f90c01f29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58222/0 (socket says 172.21.15.115:58222) 2023-12-01T03:08:47.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.770+0000 7f90bf7fe640 1 -- 172.21.15.115:0/999039784 learned_addr learned my addr 172.21.15.115:0/999039784 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:47.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90b7fff640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c015d7b0 0x7f90c0156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90beffd640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90c00cddc0 0x7f90c015c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90bf7fe640 1 -- 172.21.15.115:0/999039784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90c00cddc0 msgr2=0x7f90c015c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90bf7fe640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90c00cddc0 0x7f90c015c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90bf7fe640 1 -- 172.21.15.115:0/999039784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c015d7b0 msgr2=0x7f90c0156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90bf7fe640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c015d7b0 0x7f90c0156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90bf7fe640 1 -- 172.21.15.115:0/999039784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a8009400 con 0x7f90c015db90 2023-12-01T03:08:47.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90beffd640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90c00cddc0 0x7f90c015c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:47.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90bf7fe640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 0x7f90c01f29e0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f90b800c780 tx=0x7f90b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:47.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90b77fe640 1 -- 172.21.15.115:0/999039784 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8004b60 con 0x7f90c015db90 2023-12-01T03:08:47.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90b77fe640 1 -- 172.21.15.115:0/999039784 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b800ce90 con 0x7f90c015db90 2023-12-01T03:08:47.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.771+0000 7f90b77fe640 1 -- 172.21.15.115:0/999039784 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b800a850 con 0x7f90c015db90 2023-12-01T03:08:47.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.772+0000 7f90bffff640 1 -- 172.21.15.115:0/999039784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c00c9c70 con 0x7f90c015db90 2023-12-01T03:08:47.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.772+0000 7f90bffff640 1 -- 172.21.15.115:0/999039784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c00ca230 con 0x7f90c015db90 2023-12-01T03:08:47.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.773+0000 7f90bffff640 1 -- 172.21.15.115:0/999039784 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c00673a0 con 0x7f90c015db90 2023-12-01T03:08:47.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.774+0000 7f90b77fe640 1 -- 172.21.15.115:0/999039784 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f90b800a9b0 con 0x7f90c015db90 2023-12-01T03:08:47.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.775+0000 7f90b77fe640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90a006a550 0x7f90a006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:47.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.775+0000 7f90b77fe640 1 -- 172.21.15.115:0/999039784 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f90b8088d60 con 0x7f90c015db90 2023-12-01T03:08:47.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.775+0000 7f90beffd640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90a006a550 0x7f90a006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:47.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.776+0000 7f90beffd640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90a006a550 0x7f90a006ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f90b0005880 tx=0x7f90b0009040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:47.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.780+0000 7f90b77fe640 1 -- 172.21.15.115:0/999039784 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f90b8054170 con 0x7f90c015db90 2023-12-01T03:08:47.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.954+0000 7f90bffff640 1 -- 172.21.15.115:0/999039784 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90c0161480 con 0x7f90a006a550 2023-12-01T03:08:47.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.977+0000 7f90b77fe640 1 -- 172.21.15.115:0/999039784 <== mgr.9903 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286714 (secure 0 0 0) 0x7f90b0001ce0 con 0x7f90a006a550 2023-12-01T03:08:47.981 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:47.984 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:08:47.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 -- 172.21.15.115:0/999039784 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90a006a550 msgr2=0x7f90a006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90a006a550 0x7f90a006ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f90b0005880 tx=0x7f90b0009040 comp rx=0 tx=0).stop 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 -- 172.21.15.115:0/999039784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 msgr2=0x7f90c01f29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 0x7f90c01f29e0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f90b800c780 tx=0x7f90b800cc40 comp rx=0 tx=0).stop 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 -- 172.21.15.115:0/999039784 shutdown_connections 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90a006a550 0x7f90a006ca10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90c015db90 0x7f90c01f29e0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c015d7b0 0x7f90c0156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 --2- 172.21.15.115:0/999039784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90c00cddc0 0x7f90c015c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 -- 172.21.15.115:0/999039784 >> 172.21.15.115:0/999039784 conn(0x7f90c006a0e0 msgr2=0x7f90c006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 -- 172.21.15.115:0/999039784 shutdown_connections 2023-12-01T03:08:47.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:47.984+0000 7f90bffff640 1 -- 172.21.15.115:0/999039784 wait complete. 2023-12-01T03:08:48.005 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T03:08:47.852420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":543,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":375368,"kb_used_data":20360,"kb_used_omap":0,"kb_used_meta":354944,"kb_avail":754599352,"statfs":{"total":773094113280,"available":772709736448,"internally_reserved":0,"allocated":20848640,"data_stored":10707620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":363462656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001786"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.325913+0000","last_change":"2023-12-01T03:08:43.117913+0000","last_active":"2023-12-01T03:08:44.325913+0000","last_peered":"2023-12-01T03:08:44.325913+0000","last_clean":"2023-12-01T03:08:44.325913+0000","last_became_active":"2023-12-01T03:08:43.117644+0000","last_became_peered":"2023-12-01T03:08:43.117644+0000","last_unstale":"2023-12-01T03:08:44.325913+0000","last_undegraded":"2023-12-01T03:08:44.325913+0000","last_fullsized":"2023-12-01T03:08:44.325913+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:51:46.657032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.325830+0000","last_change":"2023-12-01T03:08:43.112634+0000","last_active":"2023-12-01T03:08:44.325830+0000","last_peered":"2023-12-01T03:08:44.325830+0000","last_clean":"2023-12-01T03:08:44.325830+0000","last_became_active":"2023-12-01T03:08:43.112401+0000","last_became_peered":"2023-12-01T03:08:43.112401+0000","last_unstale":"2023-12-01T03:08:44.325830+0000","last_undegraded":"2023-12-01T03:08:44.325830+0000","last_fullsized":"2023-12-01T03:08:44.325830+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:44:00.522998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.325573+0000","last_change":"2023-12-01T03:08:43.119923+0000","last_active":"2023-12-01T03:08:44.325573+0000","last_peered":"2023-12-01T03:08:44.325573+0000","last_clean":"2023-12-01T03:08:44.325573+0000","last_became_active":"2023-12-01T03:08:43.119479+0000","last_became_peered":"2023-12-01T03:08:43.119479+0000","last_unstale":"2023-12-01T03:08:44.325573+0000","last_undegraded":"2023-12-01T03:08:44.325573+0000","last_fullsized":"2023-12-01T03:08:44.325573+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:12.229154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.097937+0000","last_change":"2023-12-01T03:08:43.107643+0000","last_active":"2023-12-01T03:08:44.097937+0000","last_peered":"2023-12-01T03:08:44.097937+0000","last_clean":"2023-12-01T03:08:44.097937+0000","last_became_active":"2023-12-01T03:08:43.107027+0000","last_became_peered":"2023-12-01T03:08:43.107027+0000","last_unstale":"2023-12-01T03:08:44.097937+0000","last_undegraded":"2023-12-01T03:08:44.097937+0000","last_fullsized":"2023-12-01T03:08:44.097937+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:35:24.328096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.677638+0000","last_change":"2023-12-01T03:08:43.122644+0000","last_active":"2023-12-01T03:08:44.677638+0000","last_peered":"2023-12-01T03:08:44.677638+0000","last_clean":"2023-12-01T03:08:44.677638+0000","last_became_active":"2023-12-01T03:08:43.121307+0000","last_became_peered":"2023-12-01T03:08:43.121307+0000","last_unstale":"2023-12-01T03:08:44.677638+0000","last_undegraded":"2023-12-01T03:08:44.677638+0000","last_fullsized":"2023-12-01T03:08:44.677638+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:38.381441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.098858+0000","last_change":"2023-12-01T03:08:43.107951+0000","last_active":"2023-12-01T03:08:44.098858+0000","last_peered":"2023-12-01T03:08:44.098858+0000","last_clean":"2023-12-01T03:08:44.098858+0000","last_became_active":"2023-12-01T03:08:43.106585+0000","last_became_peered":"2023-12-01T03:08:43.106585+0000","last_unstale":"2023-12-01T03:08:44.098858+0000","last_undegraded":"2023-12-01T03:08:44.098858+0000","last_fullsized":"2023-12-01T03:08:44.098858+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:08.863375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.600023+0000","last_change":"2023-12-01T03:08:43.114233+0000","last_active":"2023-12-01T03:08:44.600023+0000","last_peered":"2023-12-01T03:08:44.600023+0000","last_clean":"2023-12-01T03:08:44.600023+0000","last_became_active":"2023-12-01T03:08:43.113344+0000","last_became_peered":"2023-12-01T03:08:43.113344+0000","last_unstale":"2023-12-01T03:08:44.600023+0000","last_undegraded":"2023-12-01T03:08:44.600023+0000","last_fullsized":"2023-12-01T03:08:44.600023+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:13.757701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.677747+0000","last_change":"2023-12-01T03:08:43.100768+0000","last_active":"2023-12-01T03:08:44.677747+0000","last_peered":"2023-12-01T03:08:44.677747+0000","last_clean":"2023-12-01T03:08:44.677747+0000","last_became_active":"2023-12-01T03:08:43.100492+0000","last_became_peered":"2023-12-01T03:08:43.100492+0000","last_unstale":"2023-12-01T03:08:44.677747+0000","last_undegraded":"2023-12-01T03:08:44.677747+0000","last_fullsized":"2023-12-01T03:08:44.677747+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:36:24.022936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.679684+0000","last_change":"2023-12-01T03:08:43.114830+0000","last_active":"2023-12-01T03:08:44.679684+0000","last_peered":"2023-12-01T03:08:44.679684+0000","last_clean":"2023-12-01T03:08:44.679684+0000","last_became_active":"2023-12-01T03:08:43.114564+0000","last_became_peered":"2023-12-01T03:08:43.114564+0000","last_unstale":"2023-12-01T03:08:44.679684+0000","last_undegraded":"2023-12-01T03:08:44.679684+0000","last_fullsized":"2023-12-01T03:08:44.679684+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:42.898084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.679273+0000","last_change":"2023-12-01T03:08:43.110016+0000","last_active":"2023-12-01T03:08:44.679273+0000","last_peered":"2023-12-01T03:08:44.679273+0000","last_clean":"2023-12-01T03:08:44.679273+0000","last_became_active":"2023-12-01T03:08:43.109770+0000","last_became_peered":"2023-12-01T03:08:43.109770+0000","last_unstale":"2023-12-01T03:08:44.679273+0000","last_undegraded":"2023-12-01T03:08:44.679273+0000","last_fullsized":"2023-12-01T03:08:44.679273+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:03:17.032410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.678548+0000","last_change":"2023-12-01T03:08:43.112442+0000","last_active":"2023-12-01T03:08:44.678548+0000","last_peered":"2023-12-01T03:08:44.678548+0000","last_clean":"2023-12-01T03:08:44.678548+0000","last_became_active":"2023-12-01T03:08:43.112211+0000","last_became_peered":"2023-12-01T03:08:43.112211+0000","last_unstale":"2023-12-01T03:08:44.678548+0000","last_undegraded":"2023-12-01T03:08:44.678548+0000","last_fullsized":"2023-12-01T03:08:44.678548+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:55:54.996890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.099553+0000","last_change":"2023-12-01T03:08:43.099429+0000","last_active":"2023-12-01T03:08:44.099553+0000","last_peered":"2023-12-01T03:08:44.099553+0000","last_clean":"2023-12-01T03:08:44.099553+0000","last_became_active":"2023-12-01T03:08:43.099188+0000","last_became_peered":"2023-12-01T03:08:43.099188+0000","last_unstale":"2023-12-01T03:08:44.099553+0000","last_undegraded":"2023-12-01T03:08:44.099553+0000","last_fullsized":"2023-12-01T03:08:44.099553+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:59:41.989034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.099412+0000","last_change":"2023-12-01T03:08:43.104096+0000","last_active":"2023-12-01T03:08:44.099412+0000","last_peered":"2023-12-01T03:08:44.099412+0000","last_clean":"2023-12-01T03:08:44.099412+0000","last_became_active":"2023-12-01T03:08:43.102422+0000","last_became_peered":"2023-12-01T03:08:43.102422+0000","last_unstale":"2023-12-01T03:08:44.099412+0000","last_undegraded":"2023-12-01T03:08:44.099412+0000","last_fullsized":"2023-12-01T03:08:44.099412+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:34:41.646374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.678522+0000","last_change":"2023-12-01T03:08:43.118417+0000","last_active":"2023-12-01T03:08:44.678522+0000","last_peered":"2023-12-01T03:08:44.678522+0000","last_clean":"2023-12-01T03:08:44.678522+0000","last_became_active":"2023-12-01T03:08:43.118186+0000","last_became_peered":"2023-12-01T03:08:43.118186+0000","last_unstale":"2023-12-01T03:08:44.678522+0000","last_undegraded":"2023-12-01T03:08:44.678522+0000","last_fullsized":"2023-12-01T03:08:44.678522+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:41.126577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.677837+0000","last_change":"2023-12-01T03:08:43.111122+0000","last_active":"2023-12-01T03:08:44.677837+0000","last_peered":"2023-12-01T03:08:44.677837+0000","last_clean":"2023-12-01T03:08:44.677837+0000","last_became_active":"2023-12-01T03:08:43.108068+0000","last_became_peered":"2023-12-01T03:08:43.108068+0000","last_unstale":"2023-12-01T03:08:44.677837+0000","last_undegraded":"2023-12-01T03:08:44.677837+0000","last_fullsized":"2023-12-01T03:08:44.677837+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:49.833926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.599650+0000","last_change":"2023-12-01T03:08:43.098818+0000","last_active":"2023-12-01T03:08:44.599650+0000","last_peered":"2023-12-01T03:08:44.599650+0000","last_clean":"2023-12-01T03:08:44.599650+0000","last_became_active":"2023-12-01T03:08:43.097119+0000","last_became_peered":"2023-12-01T03:08:43.097119+0000","last_unstale":"2023-12-01T03:08:44.599650+0000","last_undegraded":"2023-12-01T03:08:44.599650+0000","last_fullsized":"2023-12-01T03:08:44.599650+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:32.500430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.100612+0000","last_change":"2023-12-01T03:08:43.108000+0000","last_active":"2023-12-01T03:08:44.100612+0000","last_peered":"2023-12-01T03:08:44.100612+0000","last_clean":"2023-12-01T03:08:44.100612+0000","last_became_active":"2023-12-01T03:08:43.107657+0000","last_became_peered":"2023-12-01T03:08:43.107657+0000","last_unstale":"2023-12-01T03:08:44.100612+0000","last_undegraded":"2023-12-01T03:08:44.100612+0000","last_fullsized":"2023-12-01T03:08:44.100612+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:26:58.980087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.098667+0000","last_change":"2023-12-01T03:08:43.113568+0000","last_active":"2023-12-01T03:08:44.098667+0000","last_peered":"2023-12-01T03:08:44.098667+0000","last_clean":"2023-12-01T03:08:44.098667+0000","last_became_active":"2023-12-01T03:08:43.109712+0000","last_became_peered":"2023-12-01T03:08:43.109712+0000","last_unstale":"2023-12-01T03:08:44.098667+0000","last_undegraded":"2023-12-01T03:08:44.098667+0000","last_fullsized":"2023-12-01T03:08:44.098667+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:59.504069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.600932+0000","last_change":"2023-12-01T03:08:43.107538+0000","last_active":"2023-12-01T03:08:44.600932+0000","last_peered":"2023-12-01T03:08:44.600932+0000","last_clean":"2023-12-01T03:08:44.600932+0000","last_became_active":"2023-12-01T03:08:43.107220+0000","last_became_peered":"2023-12-01T03:08:43.107220+0000","last_unstale":"2023-12-01T03:08:44.600932+0000","last_undegraded":"2023-12-01T03:08:44.600932+0000","last_fullsized":"2023-12-01T03:08:44.600932+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:51:46.657032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.099580+0000","last_change":"2023-12-01T03:08:43.098946+0000","last_active":"2023-12-01T03:08:44.099580+0000","last_peered":"2023-12-01T03:08:44.099580+0000","last_clean":"2023-12-01T03:08:44.099580+0000","last_became_active":"2023-12-01T03:08:43.098698+0000","last_became_peered":"2023-12-01T03:08:43.098698+0000","last_unstale":"2023-12-01T03:08:44.099580+0000","last_undegraded":"2023-12-01T03:08:44.099580+0000","last_fullsized":"2023-12-01T03:08:44.099580+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:53:26.372187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.099895+0000","last_change":"2023-12-01T03:08:43.109056+0000","last_active":"2023-12-01T03:08:44.099895+0000","last_peered":"2023-12-01T03:08:44.099895+0000","last_clean":"2023-12-01T03:08:44.099895+0000","last_became_active":"2023-12-01T03:08:43.106866+0000","last_became_peered":"2023-12-01T03:08:43.106866+0000","last_unstale":"2023-12-01T03:08:44.099895+0000","last_undegraded":"2023-12-01T03:08:44.099895+0000","last_fullsized":"2023-12-01T03:08:44.099895+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:00.147728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.677819+0000","last_change":"2023-12-01T03:08:43.111695+0000","last_active":"2023-12-01T03:08:44.677819+0000","last_peered":"2023-12-01T03:08:44.677819+0000","last_clean":"2023-12-01T03:08:44.677819+0000","last_became_active":"2023-12-01T03:08:43.111458+0000","last_became_peered":"2023-12-01T03:08:43.111458+0000","last_unstale":"2023-12-01T03:08:44.677819+0000","last_undegraded":"2023-12-01T03:08:44.677819+0000","last_fullsized":"2023-12-01T03:08:44.677819+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:52:17.751784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.677608+0000","last_change":"2023-12-01T03:08:43.102268+0000","last_active":"2023-12-01T03:08:44.677608+0000","last_peered":"2023-12-01T03:08:44.677608+0000","last_clean":"2023-12-01T03:08:44.677608+0000","last_became_active":"2023-12-01T03:08:43.101960+0000","last_became_peered":"2023-12-01T03:08:43.101960+0000","last_unstale":"2023-12-01T03:08:44.677608+0000","last_undegraded":"2023-12-01T03:08:44.677608+0000","last_fullsized":"2023-12-01T03:08:44.677608+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:06.833485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.677012+0000","last_change":"2023-12-01T03:08:43.119788+0000","last_active":"2023-12-01T03:08:44.677012+0000","last_peered":"2023-12-01T03:08:44.677012+0000","last_clean":"2023-12-01T03:08:44.677012+0000","last_became_active":"2023-12-01T03:08:43.119182+0000","last_became_peered":"2023-12-01T03:08:43.119182+0000","last_unstale":"2023-12-01T03:08:44.677012+0000","last_undegraded":"2023-12-01T03:08:44.677012+0000","last_fullsized":"2023-12-01T03:08:44.677012+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:53.226762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.678715+0000","last_change":"2023-12-01T03:08:43.114945+0000","last_active":"2023-12-01T03:08:44.678715+0000","last_peered":"2023-12-01T03:08:44.678715+0000","last_clean":"2023-12-01T03:08:44.678715+0000","last_became_active":"2023-12-01T03:08:43.114548+0000","last_became_peered":"2023-12-01T03:08:43.114548+0000","last_unstale":"2023-12-01T03:08:44.678715+0000","last_undegraded":"2023-12-01T03:08:44.678715+0000","last_fullsized":"2023-12-01T03:08:44.678715+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:01:10.633660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.678450+0000","last_change":"2023-12-01T03:08:43.111956+0000","last_active":"2023-12-01T03:08:44.678450+0000","last_peered":"2023-12-01T03:08:44.678450+0000","last_clean":"2023-12-01T03:08:44.678450+0000","last_became_active":"2023-12-01T03:08:43.111288+0000","last_became_peered":"2023-12-01T03:08:43.111288+0000","last_unstale":"2023-12-01T03:08:44.678450+0000","last_undegraded":"2023-12-01T03:08:44.678450+0000","last_fullsized":"2023-12-01T03:08:44.678450+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:20:15.385949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.679064+0000","last_change":"2023-12-01T03:08:43.099521+0000","last_active":"2023-12-01T03:08:44.679064+0000","last_peered":"2023-12-01T03:08:44.679064+0000","last_clean":"2023-12-01T03:08:44.679064+0000","last_became_active":"2023-12-01T03:08:43.099163+0000","last_became_peered":"2023-12-01T03:08:43.099163+0000","last_unstale":"2023-12-01T03:08:44.679064+0000","last_undegraded":"2023-12-01T03:08:44.679064+0000","last_fullsized":"2023-12-01T03:08:44.679064+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:21.604782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.600541+0000","last_change":"2023-12-01T03:08:43.112088+0000","last_active":"2023-12-01T03:08:44.600541+0000","last_peered":"2023-12-01T03:08:44.600541+0000","last_clean":"2023-12-01T03:08:44.600541+0000","last_became_active":"2023-12-01T03:08:43.111829+0000","last_became_peered":"2023-12-01T03:08:43.111829+0000","last_unstale":"2023-12-01T03:08:44.600541+0000","last_undegraded":"2023-12-01T03:08:44.600541+0000","last_fullsized":"2023-12-01T03:08:44.600541+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:12.229154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.325348+0000","last_change":"2023-12-01T03:08:43.117388+0000","last_active":"2023-12-01T03:08:44.325348+0000","last_peered":"2023-12-01T03:08:44.325348+0000","last_clean":"2023-12-01T03:08:44.325348+0000","last_became_active":"2023-12-01T03:08:43.116532+0000","last_became_peered":"2023-12-01T03:08:43.116532+0000","last_unstale":"2023-12-01T03:08:44.325348+0000","last_undegraded":"2023-12-01T03:08:44.325348+0000","last_fullsized":"2023-12-01T03:08:44.325348+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:11.917428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.678611+0000","last_change":"2023-12-01T03:08:43.119765+0000","last_active":"2023-12-01T03:08:44.678611+0000","last_peered":"2023-12-01T03:08:44.678611+0000","last_clean":"2023-12-01T03:08:44.678611+0000","last_became_active":"2023-12-01T03:08:43.119524+0000","last_became_peered":"2023-12-01T03:08:43.119524+0000","last_unstale":"2023-12-01T03:08:44.678611+0000","last_undegraded":"2023-12-01T03:08:44.678611+0000","last_fullsized":"2023-12-01T03:08:44.678611+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:58.057235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.325180+0000","last_change":"2023-12-01T03:08:43.109877+0000","last_active":"2023-12-01T03:08:44.325180+0000","last_peered":"2023-12-01T03:08:44.325180+0000","last_clean":"2023-12-01T03:08:44.325180+0000","last_became_active":"2023-12-01T03:08:43.109601+0000","last_became_peered":"2023-12-01T03:08:43.109601+0000","last_unstale":"2023-12-01T03:08:44.325180+0000","last_undegraded":"2023-12-01T03:08:44.325180+0000","last_fullsized":"2023-12-01T03:08:44.325180+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:17:59.504069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.098857+0000","last_change":"2023-12-01T03:08:43.111545+0000","last_active":"2023-12-01T03:08:44.098857+0000","last_peered":"2023-12-01T03:08:44.098857+0000","last_clean":"2023-12-01T03:08:44.098857+0000","last_became_active":"2023-12-01T03:08:43.110014+0000","last_became_peered":"2023-12-01T03:08:43.110014+0000","last_unstale":"2023-12-01T03:08:44.098857+0000","last_undegraded":"2023-12-01T03:08:44.098857+0000","last_fullsized":"2023-12-01T03:08:44.098857+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:48:23.978730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.098727+0000","last_change":"2023-12-01T03:08:43.117463+0000","last_active":"2023-12-01T03:08:44.098727+0000","last_peered":"2023-12-01T03:08:44.098727+0000","last_clean":"2023-12-01T03:08:44.098727+0000","last_became_active":"2023-12-01T03:08:43.117170+0000","last_became_peered":"2023-12-01T03:08:43.117170+0000","last_unstale":"2023-12-01T03:08:44.098727+0000","last_undegraded":"2023-12-01T03:08:44.098727+0000","last_fullsized":"2023-12-01T03:08:44.098727+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:35.955665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"195'4","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.118143+0000","last_change":"2023-12-01T03:08:42.625851+0000","last_active":"2023-12-01T03:08:43.118143+0000","last_peered":"2023-12-01T03:08:43.118143+0000","last_clean":"2023-12-01T03:08:43.118143+0000","last_became_active":"2023-12-01T03:08:42.625468+0000","last_became_peered":"2023-12-01T03:08:42.625468+0000","last_unstale":"2023-12-01T03:08:43.118143+0000","last_undegraded":"2023-12-01T03:08:43.118143+0000","last_fullsized":"2023-12-01T03:08:43.118143+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:07.847258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.325991+0000","last_change":"2023-12-01T03:08:43.113487+0000","last_active":"2023-12-01T03:08:44.325991+0000","last_peered":"2023-12-01T03:08:44.325991+0000","last_clean":"2023-12-01T03:08:44.325991+0000","last_became_active":"2023-12-01T03:08:43.113242+0000","last_became_peered":"2023-12-01T03:08:43.113242+0000","last_unstale":"2023-12-01T03:08:44.325991+0000","last_undegraded":"2023-12-01T03:08:44.325991+0000","last_fullsized":"2023-12-01T03:08:44.325991+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:35.955665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.116316+0000","last_change":"2023-12-01T03:08:42.616289+0000","last_active":"2023-12-01T03:08:43.116316+0000","last_peered":"2023-12-01T03:08:43.116316+0000","last_clean":"2023-12-01T03:08:43.116316+0000","last_became_active":"2023-12-01T03:08:42.615868+0000","last_became_peered":"2023-12-01T03:08:42.615868+0000","last_unstale":"2023-12-01T03:08:43.116316+0000","last_undegraded":"2023-12-01T03:08:43.116316+0000","last_fullsized":"2023-12-01T03:08:43.116316+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:42:50.607707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.100172+0000","last_change":"2023-12-01T03:08:43.111912+0000","last_active":"2023-12-01T03:08:44.100172+0000","last_peered":"2023-12-01T03:08:44.100172+0000","last_clean":"2023-12-01T03:08:44.100172+0000","last_became_active":"2023-12-01T03:08:43.111627+0000","last_became_peered":"2023-12-01T03:08:43.111627+0000","last_unstale":"2023-12-01T03:08:44.100172+0000","last_undegraded":"2023-12-01T03:08:44.100172+0000","last_fullsized":"2023-12-01T03:08:44.100172+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:41.631543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.119356+0000","last_change":"2023-12-01T03:08:42.626787+0000","last_active":"2023-12-01T03:08:43.119356+0000","last_peered":"2023-12-01T03:08:43.119356+0000","last_clean":"2023-12-01T03:08:43.119356+0000","last_became_active":"2023-12-01T03:08:42.626514+0000","last_became_peered":"2023-12-01T03:08:42.626514+0000","last_unstale":"2023-12-01T03:08:43.119356+0000","last_undegraded":"2023-12-01T03:08:43.119356+0000","last_fullsized":"2023-12-01T03:08:43.119356+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:09:31.296149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.679246+0000","last_change":"2023-12-01T03:08:43.122109+0000","last_active":"2023-12-01T03:08:44.679246+0000","last_peered":"2023-12-01T03:08:44.679246+0000","last_clean":"2023-12-01T03:08:44.679246+0000","last_became_active":"2023-12-01T03:08:43.120666+0000","last_became_peered":"2023-12-01T03:08:43.120666+0000","last_unstale":"2023-12-01T03:08:44.679246+0000","last_undegraded":"2023-12-01T03:08:44.679246+0000","last_fullsized":"2023-12-01T03:08:44.679246+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:21.422507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.119273+0000","last_change":"2023-12-01T03:08:42.622116+0000","last_active":"2023-12-01T03:08:43.119273+0000","last_peered":"2023-12-01T03:08:43.119273+0000","last_clean":"2023-12-01T03:08:43.119273+0000","last_became_active":"2023-12-01T03:08:42.621806+0000","last_became_peered":"2023-12-01T03:08:42.621806+0000","last_unstale":"2023-12-01T03:08:43.119273+0000","last_undegraded":"2023-12-01T03:08:43.119273+0000","last_fullsized":"2023-12-01T03:08:43.119273+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:35:23.311979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.098288+0000","last_change":"2023-12-01T03:08:43.108808+0000","last_active":"2023-12-01T03:08:44.098288+0000","last_peered":"2023-12-01T03:08:44.098288+0000","last_clean":"2023-12-01T03:08:44.098288+0000","last_became_active":"2023-12-01T03:08:43.108327+0000","last_became_peered":"2023-12-01T03:08:43.108327+0000","last_unstale":"2023-12-01T03:08:44.098288+0000","last_undegraded":"2023-12-01T03:08:44.098288+0000","last_fullsized":"2023-12-01T03:08:44.098288+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:12.229154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.114570+0000","last_change":"2023-12-01T03:08:42.629825+0000","last_active":"2023-12-01T03:08:43.114570+0000","last_peered":"2023-12-01T03:08:43.114570+0000","last_clean":"2023-12-01T03:08:43.114570+0000","last_became_active":"2023-12-01T03:08:42.628647+0000","last_became_peered":"2023-12-01T03:08:42.628647+0000","last_unstale":"2023-12-01T03:08:43.114570+0000","last_undegraded":"2023-12-01T03:08:43.114570+0000","last_fullsized":"2023-12-01T03:08:43.114570+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:41:48.817809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.099896+0000","last_change":"2023-12-01T03:08:43.115616+0000","last_active":"2023-12-01T03:08:44.099896+0000","last_peered":"2023-12-01T03:08:44.099896+0000","last_clean":"2023-12-01T03:08:44.099896+0000","last_became_active":"2023-12-01T03:08:43.115324+0000","last_became_peered":"2023-12-01T03:08:43.115324+0000","last_unstale":"2023-12-01T03:08:44.099896+0000","last_undegraded":"2023-12-01T03:08:44.099896+0000","last_fullsized":"2023-12-01T03:08:44.099896+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:38:57.074540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.115773+0000","last_change":"2023-12-01T03:08:42.629675+0000","last_active":"2023-12-01T03:08:43.115773+0000","last_peered":"2023-12-01T03:08:43.115773+0000","last_clean":"2023-12-01T03:08:43.115773+0000","last_became_active":"2023-12-01T03:08:42.628813+0000","last_became_peered":"2023-12-01T03:08:42.628813+0000","last_unstale":"2023-12-01T03:08:43.115773+0000","last_undegraded":"2023-12-01T03:08:43.115773+0000","last_fullsized":"2023-12-01T03:08:43.115773+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:07.847258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.677216+0000","last_change":"2023-12-01T03:08:43.102716+0000","last_active":"2023-12-01T03:08:44.677216+0000","last_peered":"2023-12-01T03:08:44.677216+0000","last_clean":"2023-12-01T03:08:44.677216+0000","last_became_active":"2023-12-01T03:08:43.102362+0000","last_became_peered":"2023-12-01T03:08:43.102362+0000","last_unstale":"2023-12-01T03:08:44.677216+0000","last_undegraded":"2023-12-01T03:08:44.677216+0000","last_fullsized":"2023-12-01T03:08:44.677216+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:59:09.728003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.098337+0000","last_change":"2023-12-01T03:08:42.627653+0000","last_active":"2023-12-01T03:08:43.098337+0000","last_peered":"2023-12-01T03:08:43.098337+0000","last_clean":"2023-12-01T03:08:43.098337+0000","last_became_active":"2023-12-01T03:08:42.627042+0000","last_became_peered":"2023-12-01T03:08:42.627042+0000","last_unstale":"2023-12-01T03:08:43.098337+0000","last_undegraded":"2023-12-01T03:08:43.098337+0000","last_fullsized":"2023-12-01T03:08:43.098337+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:40.615426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.098203+0000","last_change":"2023-12-01T03:08:43.105977+0000","last_active":"2023-12-01T03:08:44.098203+0000","last_peered":"2023-12-01T03:08:44.098203+0000","last_clean":"2023-12-01T03:08:44.098203+0000","last_became_active":"2023-12-01T03:08:43.105525+0000","last_became_peered":"2023-12-01T03:08:43.105525+0000","last_unstale":"2023-12-01T03:08:44.098203+0000","last_undegraded":"2023-12-01T03:08:44.098203+0000","last_fullsized":"2023-12-01T03:08:44.098203+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:32.500430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.088945+0000","last_change":"2023-12-01T03:08:42.625307+0000","last_active":"2023-12-01T03:08:43.088945+0000","last_peered":"2023-12-01T03:08:43.088945+0000","last_clean":"2023-12-01T03:08:43.088945+0000","last_became_active":"2023-12-01T03:08:42.625012+0000","last_became_peered":"2023-12-01T03:08:42.625012+0000","last_unstale":"2023-12-01T03:08:43.088945+0000","last_undegraded":"2023-12-01T03:08:43.088945+0000","last_fullsized":"2023-12-01T03:08:43.088945+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:12.741584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.601263+0000","last_change":"2023-12-01T03:08:43.099782+0000","last_active":"2023-12-01T03:08:44.601263+0000","last_peered":"2023-12-01T03:08:44.601263+0000","last_clean":"2023-12-01T03:08:44.601263+0000","last_became_active":"2023-12-01T03:08:43.099521+0000","last_became_peered":"2023-12-01T03:08:43.099521+0000","last_unstale":"2023-12-01T03:08:44.601263+0000","last_undegraded":"2023-12-01T03:08:44.601263+0000","last_fullsized":"2023-12-01T03:08:44.601263+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:26:22.714239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.094206+0000","last_change":"2023-12-01T03:08:42.627728+0000","last_active":"2023-12-01T03:08:43.094206+0000","last_peered":"2023-12-01T03:08:43.094206+0000","last_clean":"2023-12-01T03:08:43.094206+0000","last_became_active":"2023-12-01T03:08:42.627423+0000","last_became_peered":"2023-12-01T03:08:42.627423+0000","last_unstale":"2023-12-01T03:08:43.094206+0000","last_undegraded":"2023-12-01T03:08:43.094206+0000","last_fullsized":"2023-12-01T03:08:43.094206+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:03:24.457743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.600664+0000","last_change":"2023-12-01T03:08:43.102470+0000","last_active":"2023-12-01T03:08:44.600664+0000","last_peered":"2023-12-01T03:08:44.600664+0000","last_clean":"2023-12-01T03:08:44.600664+0000","last_became_active":"2023-12-01T03:08:43.102199+0000","last_became_peered":"2023-12-01T03:08:43.102199+0000","last_unstale":"2023-12-01T03:08:44.600664+0000","last_undegraded":"2023-12-01T03:08:44.600664+0000","last_fullsized":"2023-12-01T03:08:44.600664+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:03:17.032410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"195'3","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.123796+0000","last_change":"2023-12-01T03:08:42.628648+0000","last_active":"2023-12-01T03:08:43.123796+0000","last_peered":"2023-12-01T03:08:43.123796+0000","last_clean":"2023-12-01T03:08:43.123796+0000","last_became_active":"2023-12-01T03:08:42.628305+0000","last_became_peered":"2023-12-01T03:08:42.628305+0000","last_unstale":"2023-12-01T03:08:43.123796+0000","last_undegraded":"2023-12-01T03:08:43.123796+0000","last_fullsized":"2023-12-01T03:08:43.123796+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:11.213037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.325778+0000","last_change":"2023-12-01T03:08:43.119486+0000","last_active":"2023-12-01T03:08:44.325778+0000","last_peered":"2023-12-01T03:08:44.325778+0000","last_clean":"2023-12-01T03:08:44.325778+0000","last_became_active":"2023-12-01T03:08:43.119121+0000","last_became_peered":"2023-12-01T03:08:43.119121+0000","last_unstale":"2023-12-01T03:08:44.325778+0000","last_undegraded":"2023-12-01T03:08:44.325778+0000","last_fullsized":"2023-12-01T03:08:44.325778+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:13.757701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"195'6","reported_seq":17,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.124297+0000","last_change":"2023-12-01T03:08:42.616691+0000","last_active":"2023-12-01T03:08:43.124297+0000","last_peered":"2023-12-01T03:08:43.124297+0000","last_clean":"2023-12-01T03:08:43.124297+0000","last_became_active":"2023-12-01T03:08:42.616039+0000","last_became_peered":"2023-12-01T03:08:42.616039+0000","last_unstale":"2023-12-01T03:08:43.124297+0000","last_undegraded":"2023-12-01T03:08:43.124297+0000","last_fullsized":"2023-12-01T03:08:43.124297+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:02:47.336331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.600093+0000","last_change":"2023-12-01T03:08:43.098581+0000","last_active":"2023-12-01T03:08:44.600093+0000","last_peered":"2023-12-01T03:08:44.600093+0000","last_clean":"2023-12-01T03:08:44.600093+0000","last_became_active":"2023-12-01T03:08:43.098322+0000","last_became_peered":"2023-12-01T03:08:43.098322+0000","last_unstale":"2023-12-01T03:08:44.600093+0000","last_undegraded":"2023-12-01T03:08:44.600093+0000","last_fullsized":"2023-12-01T03:08:44.600093+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:58.057235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.120275+0000","last_change":"2023-12-01T03:08:42.620988+0000","last_active":"2023-12-01T03:08:43.120275+0000","last_peered":"2023-12-01T03:08:43.120275+0000","last_clean":"2023-12-01T03:08:43.120275+0000","last_became_active":"2023-12-01T03:08:42.620436+0000","last_became_peered":"2023-12-01T03:08:42.620436+0000","last_unstale":"2023-12-01T03:08:43.120275+0000","last_undegraded":"2023-12-01T03:08:43.120275+0000","last_fullsized":"2023-12-01T03:08:43.120275+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:38:56.058423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.677767+0000","last_change":"2023-12-01T03:08:43.108111+0000","last_active":"2023-12-01T03:08:44.677767+0000","last_peered":"2023-12-01T03:08:44.677767+0000","last_clean":"2023-12-01T03:08:44.677767+0000","last_became_active":"2023-12-01T03:08:43.100552+0000","last_became_peered":"2023-12-01T03:08:43.100552+0000","last_unstale":"2023-12-01T03:08:44.677767+0000","last_undegraded":"2023-12-01T03:08:44.677767+0000","last_fullsized":"2023-12-01T03:08:44.677767+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:25:36.380322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.096405+0000","last_change":"2023-12-01T03:08:42.626045+0000","last_active":"2023-12-01T03:08:43.096405+0000","last_peered":"2023-12-01T03:08:43.096405+0000","last_clean":"2023-12-01T03:08:43.096405+0000","last_became_active":"2023-12-01T03:08:42.625686+0000","last_became_peered":"2023-12-01T03:08:42.625686+0000","last_unstale":"2023-12-01T03:08:43.096405+0000","last_undegraded":"2023-12-01T03:08:43.096405+0000","last_fullsized":"2023-12-01T03:08:43.096405+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:31:47.243234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.099105+0000","last_change":"2023-12-01T03:08:43.120471+0000","last_active":"2023-12-01T03:08:44.099105+0000","last_peered":"2023-12-01T03:08:44.099105+0000","last_clean":"2023-12-01T03:08:44.099105+0000","last_became_active":"2023-12-01T03:08:43.119318+0000","last_became_peered":"2023-12-01T03:08:43.119318+0000","last_unstale":"2023-12-01T03:08:44.099105+0000","last_undegraded":"2023-12-01T03:08:44.099105+0000","last_fullsized":"2023-12-01T03:08:44.099105+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:42:51.623824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.088846+0000","last_change":"2023-12-01T03:08:42.620141+0000","last_active":"2023-12-01T03:08:43.088846+0000","last_peered":"2023-12-01T03:08:43.088846+0000","last_clean":"2023-12-01T03:08:43.088846+0000","last_became_active":"2023-12-01T03:08:42.619812+0000","last_became_peered":"2023-12-01T03:08:42.619812+0000","last_unstale":"2023-12-01T03:08:43.088846+0000","last_undegraded":"2023-12-01T03:08:43.088846+0000","last_fullsized":"2023-12-01T03:08:43.088846+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:42:50.607707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.678266+0000","last_change":"2023-12-01T03:08:43.106623+0000","last_active":"2023-12-01T03:08:44.678266+0000","last_peered":"2023-12-01T03:08:44.678266+0000","last_clean":"2023-12-01T03:08:44.678266+0000","last_became_active":"2023-12-01T03:08:43.106183+0000","last_became_peered":"2023-12-01T03:08:43.106183+0000","last_unstale":"2023-12-01T03:08:44.678266+0000","last_undegraded":"2023-12-01T03:08:44.678266+0000","last_fullsized":"2023-12-01T03:08:44.678266+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:29:02.253316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"195'8","reported_seq":19,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.122007+0000","last_change":"2023-12-01T03:08:42.617298+0000","last_active":"2023-12-01T03:08:43.122007+0000","last_peered":"2023-12-01T03:08:43.122007+0000","last_clean":"2023-12-01T03:08:43.122007+0000","last_became_active":"2023-12-01T03:08:42.616998+0000","last_became_peered":"2023-12-01T03:08:42.616998+0000","last_unstale":"2023-12-01T03:08:43.122007+0000","last_undegraded":"2023-12-01T03:08:43.122007+0000","last_fullsized":"2023-12-01T03:08:43.122007+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:28:30.128979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.601144+0000","last_change":"2023-12-01T03:08:43.104964+0000","last_active":"2023-12-01T03:08:44.601144+0000","last_peered":"2023-12-01T03:08:44.601144+0000","last_clean":"2023-12-01T03:08:44.601144+0000","last_became_active":"2023-12-01T03:08:43.104658+0000","last_became_peered":"2023-12-01T03:08:43.104658+0000","last_unstale":"2023-12-01T03:08:44.601144+0000","last_undegraded":"2023-12-01T03:08:44.601144+0000","last_fullsized":"2023-12-01T03:08:44.601144+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:18:21.422507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.114793+0000","last_change":"2023-12-01T03:08:42.617702+0000","last_active":"2023-12-01T03:08:43.114793+0000","last_peered":"2023-12-01T03:08:43.114793+0000","last_clean":"2023-12-01T03:08:43.114793+0000","last_became_active":"2023-12-01T03:08:42.617433+0000","last_became_peered":"2023-12-01T03:08:42.617433+0000","last_unstale":"2023-12-01T03:08:43.114793+0000","last_undegraded":"2023-12-01T03:08:43.114793+0000","last_fullsized":"2023-12-01T03:08:43.114793+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:42:24.909711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.099474+0000","last_change":"2023-12-01T03:08:43.119713+0000","last_active":"2023-12-01T03:08:44.099474+0000","last_peered":"2023-12-01T03:08:44.099474+0000","last_clean":"2023-12-01T03:08:44.099474+0000","last_became_active":"2023-12-01T03:08:43.119035+0000","last_became_peered":"2023-12-01T03:08:43.119035+0000","last_unstale":"2023-12-01T03:08:44.099474+0000","last_undegraded":"2023-12-01T03:08:44.099474+0000","last_fullsized":"2023-12-01T03:08:44.099474+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:02:56.434374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.119341+0000","last_change":"2023-12-01T03:08:42.622534+0000","last_active":"2023-12-01T03:08:43.119341+0000","last_peered":"2023-12-01T03:08:43.119341+0000","last_clean":"2023-12-01T03:08:43.119341+0000","last_became_active":"2023-12-01T03:08:42.622080+0000","last_became_peered":"2023-12-01T03:08:42.622080+0000","last_unstale":"2023-12-01T03:08:43.119341+0000","last_undegraded":"2023-12-01T03:08:43.119341+0000","last_fullsized":"2023-12-01T03:08:43.119341+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:49.756181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.098102+0000","last_change":"2023-12-01T03:08:43.107290+0000","last_active":"2023-12-01T03:08:44.098102+0000","last_peered":"2023-12-01T03:08:44.098102+0000","last_clean":"2023-12-01T03:08:44.098102+0000","last_became_active":"2023-12-01T03:08:43.106672+0000","last_became_peered":"2023-12-01T03:08:43.106672+0000","last_unstale":"2023-12-01T03:08:44.098102+0000","last_undegraded":"2023-12-01T03:08:44.098102+0000","last_fullsized":"2023-12-01T03:08:44.098102+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:51:46.657032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.121346+0000","last_change":"2023-12-01T03:08:42.626861+0000","last_active":"2023-12-01T03:08:43.121346+0000","last_peered":"2023-12-01T03:08:43.121346+0000","last_clean":"2023-12-01T03:08:43.121346+0000","last_became_active":"2023-12-01T03:08:42.625517+0000","last_became_peered":"2023-12-01T03:08:42.625517+0000","last_unstale":"2023-12-01T03:08:43.121346+0000","last_undegraded":"2023-12-01T03:08:43.121346+0000","last_fullsized":"2023-12-01T03:08:43.121346+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:38.313581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.678457+0000","last_change":"2023-12-01T03:08:43.115168+0000","last_active":"2023-12-01T03:08:44.678457+0000","last_peered":"2023-12-01T03:08:44.678457+0000","last_clean":"2023-12-01T03:08:44.678457+0000","last_became_active":"2023-12-01T03:08:43.114856+0000","last_became_peered":"2023-12-01T03:08:43.114856+0000","last_unstale":"2023-12-01T03:08:44.678457+0000","last_undegraded":"2023-12-01T03:08:44.678457+0000","last_fullsized":"2023-12-01T03:08:44.678457+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:07:59.577060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.091667+0000","last_change":"2023-12-01T03:08:42.621953+0000","last_active":"2023-12-01T03:08:43.091667+0000","last_peered":"2023-12-01T03:08:43.091667+0000","last_clean":"2023-12-01T03:08:43.091667+0000","last_became_active":"2023-12-01T03:08:42.621695+0000","last_became_peered":"2023-12-01T03:08:42.621695+0000","last_unstale":"2023-12-01T03:08:43.091667+0000","last_undegraded":"2023-12-01T03:08:43.091667+0000","last_fullsized":"2023-12-01T03:08:43.091667+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:07:58.560943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.324987+0000","last_change":"2023-12-01T03:08:43.112433+0000","last_active":"2023-12-01T03:08:44.324987+0000","last_peered":"2023-12-01T03:08:44.324987+0000","last_clean":"2023-12-01T03:08:44.324987+0000","last_became_active":"2023-12-01T03:08:43.111405+0000","last_became_peered":"2023-12-01T03:08:43.111405+0000","last_unstale":"2023-12-01T03:08:44.324987+0000","last_undegraded":"2023-12-01T03:08:44.324987+0000","last_fullsized":"2023-12-01T03:08:44.324987+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:02:56.434374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.120483+0000","last_change":"2023-12-01T03:08:42.627814+0000","last_active":"2023-12-01T03:08:43.120483+0000","last_peered":"2023-12-01T03:08:43.120483+0000","last_clean":"2023-12-01T03:08:43.120483+0000","last_became_active":"2023-12-01T03:08:42.627507+0000","last_became_peered":"2023-12-01T03:08:42.627507+0000","last_unstale":"2023-12-01T03:08:43.120483+0000","last_undegraded":"2023-12-01T03:08:43.120483+0000","last_fullsized":"2023-12-01T03:08:43.120483+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:31.484313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.678008+0000","last_change":"2023-12-01T03:08:43.116772+0000","last_active":"2023-12-01T03:08:44.678008+0000","last_peered":"2023-12-01T03:08:44.678008+0000","last_clean":"2023-12-01T03:08:44.678008+0000","last_became_active":"2023-12-01T03:08:43.116510+0000","last_became_peered":"2023-12-01T03:08:43.116510+0000","last_unstale":"2023-12-01T03:08:44.678008+0000","last_undegraded":"2023-12-01T03:08:44.678008+0000","last_fullsized":"2023-12-01T03:08:44.678008+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:33:49.100075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.088521+0000","last_change":"2023-12-01T03:08:42.620163+0000","last_active":"2023-12-01T03:08:43.088521+0000","last_peered":"2023-12-01T03:08:43.088521+0000","last_clean":"2023-12-01T03:08:43.088521+0000","last_became_active":"2023-12-01T03:08:42.619818+0000","last_became_peered":"2023-12-01T03:08:42.619818+0000","last_unstale":"2023-12-01T03:08:43.088521+0000","last_undegraded":"2023-12-01T03:08:43.088521+0000","last_fullsized":"2023-12-01T03:08:43.088521+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:25:35.364206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.324779+0000","last_change":"2023-12-01T03:08:43.108491+0000","last_active":"2023-12-01T03:08:44.324779+0000","last_peered":"2023-12-01T03:08:44.324779+0000","last_clean":"2023-12-01T03:08:44.324779+0000","last_became_active":"2023-12-01T03:08:43.106122+0000","last_became_peered":"2023-12-01T03:08:43.106122+0000","last_unstale":"2023-12-01T03:08:44.324779+0000","last_undegraded":"2023-12-01T03:08:44.324779+0000","last_fullsized":"2023-12-01T03:08:44.324779+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:07.698865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.105838+0000","last_change":"2023-12-01T03:08:42.629520+0000","last_active":"2023-12-01T03:08:43.105838+0000","last_peered":"2023-12-01T03:08:43.105838+0000","last_clean":"2023-12-01T03:08:43.105838+0000","last_became_active":"2023-12-01T03:08:42.629186+0000","last_became_peered":"2023-12-01T03:08:42.629186+0000","last_unstale":"2023-12-01T03:08:43.105838+0000","last_undegraded":"2023-12-01T03:08:43.105838+0000","last_fullsized":"2023-12-01T03:08:43.105838+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:40.615426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.600119+0000","last_change":"2023-12-01T03:08:43.098539+0000","last_active":"2023-12-01T03:08:44.600119+0000","last_peered":"2023-12-01T03:08:44.600119+0000","last_clean":"2023-12-01T03:08:44.600119+0000","last_became_active":"2023-12-01T03:08:43.098240+0000","last_became_peered":"2023-12-01T03:08:43.098240+0000","last_unstale":"2023-12-01T03:08:44.600119+0000","last_undegraded":"2023-12-01T03:08:44.600119+0000","last_fullsized":"2023-12-01T03:08:44.600119+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:36:24.022936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.088479+0000","last_change":"2023-12-01T03:08:42.615902+0000","last_active":"2023-12-01T03:08:43.088479+0000","last_peered":"2023-12-01T03:08:43.088479+0000","last_clean":"2023-12-01T03:08:43.088479+0000","last_became_active":"2023-12-01T03:08:42.615555+0000","last_became_peered":"2023-12-01T03:08:42.615555+0000","last_unstale":"2023-12-01T03:08:43.088479+0000","last_undegraded":"2023-12-01T03:08:43.088479+0000","last_fullsized":"2023-12-01T03:08:43.088479+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:20:14.369832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.678506+0000","last_change":"2023-12-01T03:08:43.112136+0000","last_active":"2023-12-01T03:08:44.678506+0000","last_peered":"2023-12-01T03:08:44.678506+0000","last_clean":"2023-12-01T03:08:44.678506+0000","last_became_active":"2023-12-01T03:08:43.111682+0000","last_became_peered":"2023-12-01T03:08:43.111682+0000","last_unstale":"2023-12-01T03:08:44.678506+0000","last_undegraded":"2023-12-01T03:08:44.678506+0000","last_fullsized":"2023-12-01T03:08:44.678506+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:21.883283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.117607+0000","last_change":"2023-12-01T03:08:42.623315+0000","last_active":"2023-12-01T03:08:43.117607+0000","last_peered":"2023-12-01T03:08:43.117607+0000","last_clean":"2023-12-01T03:08:43.117607+0000","last_became_active":"2023-12-01T03:08:42.622940+0000","last_became_peered":"2023-12-01T03:08:42.622940+0000","last_unstale":"2023-12-01T03:08:43.117607+0000","last_undegraded":"2023-12-01T03:08:43.117607+0000","last_fullsized":"2023-12-01T03:08:43.117607+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:26:59.122192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.099311+0000","last_change":"2023-12-01T03:08:43.109548+0000","last_active":"2023-12-01T03:08:44.099311+0000","last_peered":"2023-12-01T03:08:44.099311+0000","last_clean":"2023-12-01T03:08:44.099311+0000","last_became_active":"2023-12-01T03:08:43.109027+0000","last_became_peered":"2023-12-01T03:08:43.109027+0000","last_unstale":"2023-12-01T03:08:44.099311+0000","last_undegraded":"2023-12-01T03:08:44.099311+0000","last_fullsized":"2023-12-01T03:08:44.099311+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:26:22.714239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.090409+0000","last_change":"2023-12-01T03:08:42.630469+0000","last_active":"2023-12-01T03:08:43.090409+0000","last_peered":"2023-12-01T03:08:43.090409+0000","last_clean":"2023-12-01T03:08:43.090409+0000","last_became_active":"2023-12-01T03:08:42.629721+0000","last_became_peered":"2023-12-01T03:08:42.629721+0000","last_unstale":"2023-12-01T03:08:43.090409+0000","last_undegraded":"2023-12-01T03:08:43.090409+0000","last_fullsized":"2023-12-01T03:08:43.090409+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:05.817369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.098510+0000","last_change":"2023-12-01T03:08:43.107820+0000","last_active":"2023-12-01T03:08:44.098510+0000","last_peered":"2023-12-01T03:08:44.098510+0000","last_clean":"2023-12-01T03:08:44.098510+0000","last_became_active":"2023-12-01T03:08:43.107239+0000","last_became_peered":"2023-12-01T03:08:43.107239+0000","last_unstale":"2023-12-01T03:08:44.098510+0000","last_undegraded":"2023-12-01T03:08:44.098510+0000","last_fullsized":"2023-12-01T03:08:44.098510+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:13.757701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.101019+0000","last_change":"2023-12-01T03:08:42.620892+0000","last_active":"2023-12-01T03:08:43.101019+0000","last_peered":"2023-12-01T03:08:43.101019+0000","last_clean":"2023-12-01T03:08:43.101019+0000","last_became_active":"2023-12-01T03:08:42.620610+0000","last_became_peered":"2023-12-01T03:08:42.620610+0000","last_unstale":"2023-12-01T03:08:43.101019+0000","last_undegraded":"2023-12-01T03:08:43.101019+0000","last_fullsized":"2023-12-01T03:08:43.101019+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:31.232706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.098493+0000","last_change":"2023-12-01T03:08:43.114782+0000","last_active":"2023-12-01T03:08:44.098493+0000","last_peered":"2023-12-01T03:08:44.098493+0000","last_clean":"2023-12-01T03:08:44.098493+0000","last_became_active":"2023-12-01T03:08:43.113666+0000","last_became_peered":"2023-12-01T03:08:43.113666+0000","last_unstale":"2023-12-01T03:08:44.098493+0000","last_undegraded":"2023-12-01T03:08:44.098493+0000","last_fullsized":"2023-12-01T03:08:44.098493+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:36:38.983295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.092632+0000","last_change":"2023-12-01T03:08:42.620021+0000","last_active":"2023-12-01T03:08:43.092632+0000","last_peered":"2023-12-01T03:08:43.092632+0000","last_clean":"2023-12-01T03:08:43.092632+0000","last_became_active":"2023-12-01T03:08:42.619693+0000","last_became_peered":"2023-12-01T03:08:42.619693+0000","last_unstale":"2023-12-01T03:08:43.092632+0000","last_undegraded":"2023-12-01T03:08:43.092632+0000","last_fullsized":"2023-12-01T03:08:43.092632+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:51:45.640915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.099120+0000","last_change":"2023-12-01T03:08:43.112766+0000","last_active":"2023-12-01T03:08:44.099120+0000","last_peered":"2023-12-01T03:08:44.099120+0000","last_clean":"2023-12-01T03:08:44.099120+0000","last_became_active":"2023-12-01T03:08:43.112396+0000","last_became_peered":"2023-12-01T03:08:43.112396+0000","last_unstale":"2023-12-01T03:08:44.099120+0000","last_undegraded":"2023-12-01T03:08:44.099120+0000","last_fullsized":"2023-12-01T03:08:44.099120+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:50.772298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.092520+0000","last_change":"2023-12-01T03:08:42.620478+0000","last_active":"2023-12-01T03:08:43.092520+0000","last_peered":"2023-12-01T03:08:43.092520+0000","last_clean":"2023-12-01T03:08:43.092520+0000","last_became_active":"2023-12-01T03:08:42.619147+0000","last_became_peered":"2023-12-01T03:08:42.619147+0000","last_unstale":"2023-12-01T03:08:43.092520+0000","last_undegraded":"2023-12-01T03:08:43.092520+0000","last_fullsized":"2023-12-01T03:08:43.092520+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:27:39.871916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.325280+0000","last_change":"2023-12-01T03:08:43.113698+0000","last_active":"2023-12-01T03:08:44.325280+0000","last_peered":"2023-12-01T03:08:44.325280+0000","last_clean":"2023-12-01T03:08:44.325280+0000","last_became_active":"2023-12-01T03:08:43.113457+0000","last_became_peered":"2023-12-01T03:08:43.113457+0000","last_unstale":"2023-12-01T03:08:44.325280+0000","last_undegraded":"2023-12-01T03:08:44.325280+0000","last_fullsized":"2023-12-01T03:08:44.325280+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:37:05.480677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.090058+0000","last_change":"2023-12-01T03:08:42.618895+0000","last_active":"2023-12-01T03:08:43.090058+0000","last_peered":"2023-12-01T03:08:43.090058+0000","last_clean":"2023-12-01T03:08:43.090058+0000","last_became_active":"2023-12-01T03:08:42.618622+0000","last_became_peered":"2023-12-01T03:08:42.618622+0000","last_unstale":"2023-12-01T03:08:43.090058+0000","last_undegraded":"2023-12-01T03:08:43.090058+0000","last_fullsized":"2023-12-01T03:08:43.090058+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:49.756181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.599787+0000","last_change":"2023-12-01T03:08:43.107577+0000","last_active":"2023-12-01T03:08:44.599787+0000","last_peered":"2023-12-01T03:08:44.599787+0000","last_clean":"2023-12-01T03:08:44.599787+0000","last_became_active":"2023-12-01T03:08:43.107275+0000","last_became_peered":"2023-12-01T03:08:43.107275+0000","last_unstale":"2023-12-01T03:08:44.599787+0000","last_undegraded":"2023-12-01T03:08:44.599787+0000","last_fullsized":"2023-12-01T03:08:44.599787+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:32.248822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.098436+0000","last_change":"2023-12-01T03:08:42.627769+0000","last_active":"2023-12-01T03:08:43.098436+0000","last_peered":"2023-12-01T03:08:43.098436+0000","last_clean":"2023-12-01T03:08:43.098436+0000","last_became_active":"2023-12-01T03:08:42.627414+0000","last_became_peered":"2023-12-01T03:08:42.627414+0000","last_unstale":"2023-12-01T03:08:43.098436+0000","last_undegraded":"2023-12-01T03:08:43.098436+0000","last_fullsized":"2023-12-01T03:08:43.098436+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:35.903418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.325029+0000","last_change":"2023-12-01T03:08:43.109170+0000","last_active":"2023-12-01T03:08:44.325029+0000","last_peered":"2023-12-01T03:08:44.325029+0000","last_clean":"2023-12-01T03:08:44.325029+0000","last_became_active":"2023-12-01T03:08:43.108901+0000","last_became_peered":"2023-12-01T03:08:43.108901+0000","last_unstale":"2023-12-01T03:08:44.325029+0000","last_undegraded":"2023-12-01T03:08:44.325029+0000","last_fullsized":"2023-12-01T03:08:44.325029+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:42.952791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.099908+0000","last_change":"2023-12-01T03:08:42.627087+0000","last_active":"2023-12-01T03:08:43.099908+0000","last_peered":"2023-12-01T03:08:43.099908+0000","last_clean":"2023-12-01T03:08:43.099908+0000","last_became_active":"2023-12-01T03:08:42.625906+0000","last_became_peered":"2023-12-01T03:08:42.625906+0000","last_unstale":"2023-12-01T03:08:43.099908+0000","last_undegraded":"2023-12-01T03:08:43.099908+0000","last_fullsized":"2023-12-01T03:08:43.099908+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:38:56.058423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-01T03:08:44.601038+0000","last_change":"2023-12-01T03:08:43.112836+0000","last_active":"2023-12-01T03:08:44.601038+0000","last_peered":"2023-12-01T03:08:44.601038+0000","last_clean":"2023-12-01T03:08:44.601038+0000","last_became_active":"2023-12-01T03:08:43.112578+0000","last_became_peered":"2023-12-01T03:08:43.112578+0000","last_unstale":"2023-12-01T03:08:44.601038+0000","last_undegraded":"2023-12-01T03:08:44.601038+0000","last_fullsized":"2023-12-01T03:08:44.601038+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:42.577501+0000","last_clean_scrub_stamp":"2023-12-01T03:08:42.577501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:36:38.983295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.093298+0000","last_change":"2023-12-01T03:08:42.622692+0000","last_active":"2023-12-01T03:08:43.093298+0000","last_peered":"2023-12-01T03:08:43.093298+0000","last_clean":"2023-12-01T03:08:43.093298+0000","last_became_active":"2023-12-01T03:08:42.622444+0000","last_became_peered":"2023-12-01T03:08:42.622444+0000","last_unstale":"2023-12-01T03:08:43.093298+0000","last_undegraded":"2023-12-01T03:08:43.093298+0000","last_fullsized":"2023-12-01T03:08:43.093298+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:08:41.561384+0000","last_clean_scrub_stamp":"2023-12-01T03:08:41.561384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:02:14.580168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"184'304","reported_seq":1024,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-01T03:08:43.091763+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:08:43.091763+0000","last_peered":"2023-12-01T03:08:43.091763+0000","last_clean":"2023-12-01T03:08:43.091763+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:08:43.091763+0000","last_undegraded":"2023-12-01T03:08:43.091763+0000","last_fullsized":"2023-12-01T03:08:43.091763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1748,"num_read_kb":3602,"num_write":499,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575957,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55584,"kb_used_data":2392,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316256,"statfs":{"total":96636764160,"available":96579846144,"internally_reserved":0,"allocated":2449408,"data_stored":1190270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0900000000000001,"15min":1.0860000000000001},"min":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.71299999999999997},"max":{"1min":1.6000000000000001,"5min":1.7390000000000001,"15min":1.98},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.131,"15min":1.125},"min":{"1min":0.85499999999999998,"5min":0.80300000000000005,"15min":0.754},"max":{"1min":1.6699999999999999,"5min":1.7,"15min":1.7},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3420000000000001,"15min":1.306},"min":{"1min":1.018,"5min":0.90700000000000003,"15min":0.73099999999999998},"max":{"1min":1.673,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.365,"5min":1.331,"15min":1.3400000000000001},"min":{"1min":1.1220000000000001,"5min":0.871,"15min":0.871},"max":{"1min":1.5429999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.147}]},{"osd":2,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1970000000000001,"15min":1.1739999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.76700000000000002},"max":{"1min":1.6020000000000001,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.262,"15min":1.2509999999999999},"min":{"1min":0.95699999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.526,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.359}]},{"osd":3,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.292,"15min":1.3089999999999999},"min":{"1min":1.087,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.613,"5min":2.431,"15min":2.431},"last":1.282},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4470000000000001,"15min":1.4279999999999999},"min":{"1min":1.1719999999999999,"5min":1.032,"15min":0.92900000000000005},"max":{"1min":1.7909999999999999,"5min":2.198,"15min":2.198},"last":1.474}]},{"osd":4,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3220000000000001,"15min":1.3320000000000001},"min":{"1min":1.0549999999999999,"5min":0.82599999999999996,"15min":0.81200000000000006},"max":{"1min":1.6859999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":1.403},{"interface":"front","average":{"1min":1.571,"5min":1.5189999999999999,"15min":1.4990000000000001},"min":{"1min":1.3009999999999999,"5min":0.92800000000000005,"15min":0.79500000000000004},"max":{"1min":1.853,"5min":2.569,"15min":2.569},"last":1.552}]},{"osd":5,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3220000000000001,"15min":1.3049999999999999},"min":{"1min":0.94699999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.6739999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":1.444},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.609,"15min":1.595},"min":{"1min":1.401,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.9079999999999999,"5min":2.484,"15min":2.484},"last":1.609}]},{"osd":6,"last update":"Fri Dec 1 03:08:07 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.4370000000000001,"15min":1.4430000000000001},"min":{"1min":1.2030000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7829999999999999,"5min":2.532,"15min":2.532},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4419999999999999,"15min":1.46},"min":{"1min":1.091,"5min":1.0189999999999999,"15min":0.97999999999999998},"max":{"1min":1.8819999999999999,"5min":2.2749999999999999,"15min":2.762},"last":1.516}]}]},{"osd":6,"up_from":13,"seq":55834575958,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41636,"kb_used_data":2396,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330204,"statfs":{"total":96636764160,"available":96594128896,"internally_reserved":0,"allocated":2453504,"data_stored":1190304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1080000000000001,"15min":1.127},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.361,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.306},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.1359999999999999,"15min":1.1759999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5720000000000001,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.1579999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1890000000000001,"15min":1.2230000000000001},"min":{"1min":0.88,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5740000000000001,"5min":1.7789999999999999,"15min":2.0449999999999999},"last":1.8420000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.244,"15min":1.228},"min":{"1min":0.85599999999999998,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.548,"5min":1.835,"15min":1.835},"last":1.125}]},{"osd":2,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.216,"15min":1.2210000000000001},"min":{"1min":0.92900000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5249999999999999,"5min":1.976,"15min":2.1469999999999998},"last":1.274},{"interface":"front","average":{"1min":1.264,"5min":1.2949999999999999,"15min":1.29},"min":{"1min":1.0029999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.625,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.5740000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.331,"15min":1.3300000000000001},"min":{"1min":1.052,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.637,"5min":1.8480000000000001,"15min":2.2669999999999999},"last":1.927},{"interface":"front","average":{"1min":1.345,"5min":1.423,"15min":1.4279999999999999},"min":{"1min":0.997,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.613,"5min":1.7949999999999999,"15min":1.923},"last":2.032}]},{"osd":4,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.2310000000000001,"15min":1.2729999999999999},"min":{"1min":0.90000000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.476,"5min":1.944,"15min":2.0819999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.339,"15min":1.349},"min":{"1min":1.042,"5min":0.89100000000000001,"15min":0.82699999999999996},"max":{"1min":1.6619999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":2.1440000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3859999999999999,"15min":1.423},"min":{"1min":1.036,"5min":0.88,"15min":0.873},"max":{"1min":1.7050000000000001,"5min":2.1360000000000001,"15min":2.1739999999999999},"last":1.482},{"interface":"front","average":{"1min":1.423,"5min":1.4810000000000001,"15min":1.488},"min":{"1min":1.1299999999999999,"5min":0.95599999999999996,"15min":0.93999999999999995},"max":{"1min":1.7110000000000001,"5min":1.9059999999999999,"15min":2.0379999999999998},"last":2.2429999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.508,"15min":1.536},"min":{"1min":1.161,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":1.6870000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.6000000000000001,"15min":1.589},"min":{"1min":1.2230000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.669,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.6639999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575957,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47980,"kb_used_data":2980,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323860,"statfs":{"total":96636764160,"available":96587632640,"internally_reserved":0,"allocated":3051520,"data_stored":1781310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.115,"15min":1.131},"min":{"1min":0.72499999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.218,"5min":1.6559999999999999,"15min":1.8899999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.181,"5min":1.135,"15min":1.1739999999999999},"min":{"1min":0.96199999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.415,"5min":1.752,"15min":1.8009999999999999},"last":1.135}]},{"osd":1,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.165,"15min":1.1599999999999999},"min":{"1min":0.81699999999999995,"5min":0.81200000000000006,"15min":0.76500000000000001},"max":{"1min":1.613,"5min":1.696,"15min":1.7849999999999999},"last":1.099},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.234,"15min":1.2490000000000001},"min":{"1min":0.95299999999999996,"5min":0.80200000000000005,"15min":0.60899999999999999},"max":{"1min":1.454,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.95599999999999996}]},{"osd":2,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2330000000000001,"15min":1.2490000000000001},"min":{"1min":0.999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.5700000000000001,"5min":1.575,"15min":1.8129999999999999},"last":1.03},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.262,"15min":1.2709999999999999},"min":{"1min":0.92200000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.635,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1699999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.296,"15min":1.3220000000000001},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.93100000000000005},"max":{"1min":1.5920000000000001,"5min":1.8979999999999999,"15min":1.994},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.3700000000000001,"15min":1.371},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":0.90000000000000002},"max":{"1min":1.7709999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":1.3839999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.2569999999999999,"15min":1.254},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.74099999999999999},"max":{"1min":1.5409999999999999,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.381,"15min":1.3919999999999999},"min":{"1min":0.94599999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.6919999999999999,"5min":1.8680000000000001,"15min":1.931},"last":1.2649999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.383,"15min":1.371},"min":{"1min":0.93200000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.718,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":1.504},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.4059999999999999,"15min":1.411},"min":{"1min":0.95199999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.6379999999999999,"5min":2.577,"15min":2.577},"last":1.458}]},{"osd":7,"last update":"Fri Dec 1 03:07:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5409999999999999,"15min":1.54},"min":{"1min":1.282,"5min":0.92800000000000005,"15min":0.91700000000000004},"max":{"1min":1.663,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.429},{"interface":"front","average":{"1min":1.498,"5min":1.593,"15min":1.5740000000000001},"min":{"1min":1.2330000000000001,"5min":1.03,"15min":1.014},"max":{"1min":1.893,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.5609999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47396,"kb_used_data":2396,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324444,"statfs":{"total":96636764160,"available":96588230656,"internally_reserved":0,"allocated":2453504,"data_stored":1190304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.125,"15min":1.1379999999999999},"min":{"1min":0.77000000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.421,"5min":1.607,"15min":1.6759999999999999},"last":0.873},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1559999999999999,"15min":1.171},"min":{"1min":0.85499999999999998,"5min":0.78700000000000003,"15min":0.77500000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.704},"last":0.92700000000000005}]},{"osd":1,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.2430000000000001,"15min":1.2629999999999999},"min":{"1min":0.97599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.581,"5min":1.629,"15min":1.8640000000000001},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.198,"15min":1.1879999999999999},"min":{"1min":0.85299999999999998,"5min":0.84099999999999997,"15min":0.72899999999999998},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.296}]},{"osd":2,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2190000000000001,"15min":1.2090000000000001},"min":{"1min":0.81699999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.613,"5min":1.613,"15min":1.6739999999999999},"last":1.357},{"interface":"front","average":{"1min":1.389,"5min":1.363,"15min":1.343},"min":{"1min":1.145,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.637,"5min":1.786,"15min":1.9450000000000001},"last":1.056}]},{"osd":3,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2869999999999999,"15min":1.282},"min":{"1min":0.97899999999999998,"5min":0.88300000000000001,"15min":0.879},"max":{"1min":1.77,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.548},{"interface":"front","average":{"1min":1.339,"5min":1.3220000000000001,"15min":1.329},"min":{"1min":1.0209999999999999,"5min":0.88,"15min":0.88},"max":{"1min":1.609,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.619}]},{"osd":5,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3220000000000001,"15min":1.3069999999999999},"min":{"1min":1.0309999999999999,"5min":0.89600000000000002,"15min":0.755},"max":{"1min":1.7529999999999999,"5min":1.7789999999999999,"15min":1.841},"last":1.427},{"interface":"front","average":{"1min":1.327,"5min":1.331,"15min":1.294},"min":{"1min":1.0069999999999999,"5min":0.91400000000000003,"15min":0.86699999999999999},"max":{"1min":1.716,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.667}]},{"osd":6,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3580000000000001,"15min":1.367},"min":{"1min":1.1299999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5269999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4570000000000001,"15min":1.466},"min":{"1min":1.2150000000000001,"5min":1.071,"15min":0.98699999999999999},"max":{"1min":1.7689999999999999,"5min":1.853,"15min":2.0089999999999999},"last":1.593}]},{"osd":7,"last update":"Fri Dec 1 03:07:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.49,"15min":1.476},"min":{"1min":1.212,"5min":1.0389999999999999,"15min":0.93799999999999994},"max":{"1min":1.909,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.75},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5529999999999999,"15min":1.5660000000000001},"min":{"1min":1.327,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":1.9019999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.7130000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575956,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":2408,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":2465792,"data_stored":1191856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.92300000000000004,"15min":0.91100000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.423,"5min":1.4650000000000001,"15min":1.476},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.02,"5min":1.1160000000000001,"15min":1.0880000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.542,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.1299999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.169,"15min":1.173},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4670000000000001,"5min":1.625,"15min":1.667},"last":1.2490000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1,"15min":0.998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.41099999999999998},"max":{"1min":1.4410000000000001,"5min":1.899,"15min":1.899},"last":0.88800000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0920000000000001,"15min":1.105},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.379,"5min":1.647,"15min":1.8200000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.218,"5min":1.216,"15min":1.252},"min":{"1min":0.747,"5min":0.747,"15min":0.58599999999999997},"max":{"1min":1.5529999999999999,"5min":1.611,"15min":1.9219999999999999},"last":1.49}]},{"osd":4,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.294,"15min":1.2949999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.5820000000000001,"5min":1.615,"15min":1.6599999999999999},"last":1.321},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.268,"15min":1.292},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.476,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.3759999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.306,"15min":1.327},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.55,"5min":1.7829999999999999,"15min":1.8979999999999999},"last":1.288},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.351,"15min":1.371},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9690000000000001,"5min":1.9690000000000001,"15min":2.1779999999999999},"last":1.1930000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.4379999999999999,"15min":1.448},"min":{"1min":1.046,"5min":1.03,"15min":0.78400000000000003},"max":{"1min":1.5229999999999999,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.456},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.5029999999999999,"15min":1.5129999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.8859999999999999,"5min":2.0819999999999999,"15min":2.3660000000000001},"last":1.399}]},{"osd":7,"last update":"Fri Dec 1 03:08:38 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.514,"15min":1.528},"min":{"1min":1.1259999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":1.9370000000000001,"5min":1.9870000000000001,"15min":2.3109999999999999},"last":1.861},{"interface":"front","average":{"1min":1.389,"5min":1.53,"15min":1.5269999999999999},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.878},"max":{"1min":1.738,"5min":2.984,"15min":2.984},"last":1.6000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41636,"kb_used_data":2396,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330204,"statfs":{"total":96636764160,"available":96594128896,"internally_reserved":0,"allocated":2453504,"data_stored":1190852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91400000000000003,"15min":0.89500000000000002},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.60399999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.4730000000000001},"last":0.877},{"interface":"front","average":{"1min":0.93300000000000005,"5min":1.0649999999999999,"15min":1.0589999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.329,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.377}]},{"osd":1,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0620000000000001,"15min":1.0720000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.3959999999999999,"5min":1.698,"15min":1.698},"last":1},{"interface":"front","average":{"1min":1.071,"5min":1.147,"15min":1.1579999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5800000000000001,"5min":1.635,"15min":1.71},"last":1.474}]},{"osd":3,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2130000000000001,"15min":1.23},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.615,"5min":1.6879999999999999,"15min":1.923},"last":1.256}]},{"osd":4,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.347,"15min":1.3420000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.651},{"interface":"front","average":{"1min":1.252,"5min":1.2989999999999999,"15min":1.3},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.617,"5min":1.702,"15min":1.835},"last":1.1140000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.405,"15min":1.397},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7989999999999999,"5min":1.976,"15min":2.2040000000000002},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.421,"15min":1.4179999999999999},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":2.016},"last":1.569}]},{"osd":6,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3759999999999999,"15min":1.3819999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":2.0289999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.514,"15min":1.5089999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":2.3010000000000002},"last":1.329}]},{"osd":7,"last update":"Fri Dec 1 03:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.504,"15min":1.494},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.7350000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.427,"15min":1.466},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.226}]}]},{"osd":1,"up_from":13,"seq":55834575955,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47996,"kb_used_data":2996,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323844,"statfs":{"total":96636764160,"available":96587616256,"internally_reserved":0,"allocated":3067904,"data_stored":1782384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.89400000000000002,"15min":0.91800000000000004},"min":{"1min":0.66100000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.0429999999999999,"15min":1.0580000000000001},"min":{"1min":0.77700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.4530000000000001,"5min":1.617,"15min":1.776},"last":0.37}]},{"osd":2,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.96999999999999997,"15min":0.96699999999999997},"min":{"1min":0.85599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.157,"5min":1.159,"15min":1.159},"min":{"1min":0.78700000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6080000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.47799999999999998}]},{"osd":3,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3300000000000001,"15min":1.329},"min":{"1min":0.90700000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.587,"5min":1.788,"15min":1.9159999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.145,"15min":1.151},"min":{"1min":0.79500000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.8049999999999999},"last":0.497}]},{"osd":4,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2110000000000001,"15min":1.2250000000000001},"min":{"1min":1.0049999999999999,"5min":0.92300000000000004,"15min":0.84999999999999998},"max":{"1min":1.478,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.403,"5min":1.425,"15min":1.419},"min":{"1min":1.085,"5min":1.085,"15min":1.042},"max":{"1min":1.8620000000000001,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.353,"15min":1.351},"min":{"1min":0.95699999999999996,"5min":0.92800000000000005,"15min":0.89000000000000001},"max":{"1min":1.665,"5min":2.0379999999999998,"15min":2.1160000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3560000000000001,"15min":1.3640000000000001},"min":{"1min":1.1120000000000001,"5min":1.014,"15min":0.94399999999999995},"max":{"1min":1.927,"5min":1.927,"15min":2.1659999999999999},"last":0.88500000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.466,"15min":1.4490000000000001},"min":{"1min":1.1279999999999999,"5min":1.036,"15min":0.93000000000000005},"max":{"1min":1.6399999999999999,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.409,"15min":1.4279999999999999},"min":{"1min":1.2230000000000001,"5min":1.0149999999999999,"15min":0.90200000000000002},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0209999999999999},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:07:52 2023","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.552,"15min":1.5429999999999999},"min":{"1min":1.3180000000000001,"5min":1.1419999999999999,"15min":1.1140000000000001},"max":{"1min":1.9970000000000001,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.872},{"interface":"front","average":{"1min":1.5,"5min":1.5189999999999999,"15min":1.5269999999999999},"min":{"1min":1.282,"5min":1.103,"15min":0.79000000000000004},"max":{"1min":1.7709999999999999,"5min":2.2069999999999999,"15min":2.3090000000000002},"last":1.0329999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575957,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41636,"kb_used_data":2396,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330204,"statfs":{"total":96636764160,"available":96594128896,"internally_reserved":0,"allocated":2453504,"data_stored":1190340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.91100000000000003,"15min":0.90400000000000003},"min":{"1min":0.68400000000000005,"5min":0.435,"15min":0.38300000000000001},"max":{"1min":1.262,"5min":1.994,"15min":1.994},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0660000000000001},"min":{"1min":0.72399999999999998,"5min":0.46400000000000002,"15min":0.439},"max":{"1min":1.383,"5min":1.6379999999999999,"15min":1.7050000000000001},"last":0.495}]},{"osd":2,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0669999999999999,"15min":1.0740000000000001},"min":{"1min":0.84599999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.331,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.133,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.86599999999999999,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.675,"5min":1.792,"15min":1.792},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.0920000000000001,"15min":1.083},"min":{"1min":0.749,"5min":0.65600000000000003,"15min":0.52900000000000003},"max":{"1min":1.4239999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.24,"5min":1.2450000000000001,"15min":1.244},"min":{"1min":0.89000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6379999999999999,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":0.88400000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.369,"15min":1.3660000000000001},"min":{"1min":0.998,"5min":0.998,"15min":0.74099999999999999},"max":{"1min":1.5960000000000001,"5min":2.085,"15min":2.1400000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.347,"5min":1.3440000000000001,"15min":1.3140000000000001},"min":{"1min":1.0720000000000001,"5min":0.84699999999999998,"15min":0.626},"max":{"1min":1.74,"5min":2.3210000000000002,"15min":2.3610000000000002},"last":0.997}]},{"osd":5,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4330000000000001,"15min":1.4159999999999999},"min":{"1min":1.228,"5min":0.88400000000000001,"15min":0.82599999999999996},"max":{"1min":1.8859999999999999,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.45,"15min":1.444},"min":{"1min":1.0900000000000001,"5min":0.79600000000000004,"15min":0.69599999999999995},"max":{"1min":1.9690000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":1.044}]},{"osd":6,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4039999999999999,"15min":1.3879999999999999},"min":{"1min":1.141,"5min":0.88300000000000001,"15min":0.76800000000000002},"max":{"1min":1.8169999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.4990000000000001,"15min":1.4950000000000001},"min":{"1min":1.1639999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.9419999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.3049999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.546,"15min":1.532},"min":{"1min":1.1799999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.0390000000000001,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.498,"15min":1.4850000000000001},"min":{"1min":1.28,"5min":1.032,"15min":0.72599999999999998},"max":{"1min":1.9059999999999999,"5min":2.431,"15min":2.431},"last":1.25}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:08:48.010 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T03:08:48.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T03:08:48.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.271+0000 7ff3c1589640 1 Processor -- start 2023-12-01T03:08:48.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.271+0000 7ff3c1589640 1 -- start start 2023-12-01T03:08:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.273+0000 7ff3c1589640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3bc159db0 0x7ff3bc072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.273+0000 7ff3c1589640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff3bc0cd510 0x7ff3bc0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.273+0000 7ff3c1589640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3bc0cd030 con 0x7ff3bc0ca110 2023-12-01T03:08:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.273+0000 7ff3c1589640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc071740 con 0x7ff3bc159db0 2023-12-01T03:08:48.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.273+0000 7ff3c1589640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc071a20 con 0x7ff3bc0cd510 2023-12-01T03:08:48.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.273+0000 7ff3baffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3bc159db0 0x7ff3bc072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.274+0000 7ff3baffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3bc159db0 0x7ff3bc072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58238/0 (socket says 172.21.15.115:58238) 2023-12-01T03:08:48.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.274+0000 7ff3baffd640 1 -- 172.21.15.115:0/1958082441 learned_addr learned my addr 172.21.15.115:0/1958082441 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:48.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.274+0000 7ff3bb7fe640 1 --2- 172.21.15.115:0/1958082441 >> v2:172.21.15.142:3300/0 conn(0x7ff3bc0cd510 0x7ff3bc0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.274+0000 7ff3baffd640 1 -- 172.21.15.115:0/1958082441 >> v2:172.21.15.142:3300/0 conn(0x7ff3bc0cd510 msgr2=0x7ff3bc0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:48.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.274+0000 7ff3baffd640 1 --2- 172.21.15.115:0/1958082441 >> v2:172.21.15.142:3300/0 conn(0x7ff3bc0cd510 0x7ff3bc0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.274+0000 7ff3baffd640 1 -- 172.21.15.115:0/1958082441 >> v1:172.21.15.142:6789/0 conn(0x7ff3bc0ca110 legacy=0x7ff3bc0ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:48.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.274+0000 7ff3baffd640 1 -- 172.21.15.115:0/1958082441 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc071d00 con 0x7ff3bc159db0 2023-12-01T03:08:48.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.274+0000 7ff3bb7fe640 1 --2- 172.21.15.115:0/1958082441 >> v2:172.21.15.142:3300/0 conn(0x7ff3bc0cd510 0x7ff3bc0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:48.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.275+0000 7ff3baffd640 1 --2- 172.21.15.115:0/1958082441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc159db0 0x7ff3bc072d00 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0cf030 tx=0x7ff3ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5662bb817ea8209 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:48.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.275+0000 7ff3ba7fc640 1 -- 172.21.15.115:0/1958082441 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac031870 con 0x7ff3bc159db0 2023-12-01T03:08:48.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.276+0000 7ff3ba7fc640 1 -- 172.21.15.115:0/1958082441 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac0319d0 con 0x7ff3bc159db0 2023-12-01T03:08:48.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.276+0000 7ff3ba7fc640 1 -- 172.21.15.115:0/1958082441 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac031d60 con 0x7ff3bc159db0 2023-12-01T03:08:48.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.276+0000 7ff3c1589640 1 -- 172.21.15.115:0/1958082441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc159db0 msgr2=0x7ff3bc072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:48.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.276+0000 7ff3c1589640 1 --2- 172.21.15.115:0/1958082441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc159db0 0x7ff3bc072d00 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0cf030 tx=0x7ff3ac02b910 comp rx=0 tx=0).stop 2023-12-01T03:08:48.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.276+0000 7ff3c1589640 1 -- 172.21.15.115:0/1958082441 shutdown_connections 2023-12-01T03:08:48.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.276+0000 7ff3c1589640 1 --2- 172.21.15.115:0/1958082441 >> v2:172.21.15.142:3300/0 conn(0x7ff3bc0cd510 0x7ff3bc0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.276+0000 7ff3c1589640 1 --2- 172.21.15.115:0/1958082441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc159db0 0x7ff3bc072d00 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.276+0000 7ff3c1589640 1 -- 172.21.15.115:0/1958082441 >> 172.21.15.115:0/1958082441 conn(0x7ff3bc0cbb10 msgr2=0x7ff3bc0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.276+0000 7ff3c1589640 1 -- 172.21.15.115:0/1958082441 shutdown_connections 2023-12-01T03:08:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.276+0000 7ff3c1589640 1 -- 172.21.15.115:0/1958082441 wait complete. 2023-12-01T03:08:48.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.278+0000 7ff3c1589640 1 Processor -- start 2023-12-01T03:08:48.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.278+0000 7ff3c1589640 1 -- start start 2023-12-01T03:08:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.279+0000 7ff3c1589640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3bc0ca110 0x7ff3bc152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.279+0000 7ff3c1589640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc0cd510 0x7ff3bc14ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.279+0000 7ff3c1589640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3bc159db0 0x7ff3bc151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.279+0000 7ff3c1589640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc152a70 con 0x7ff3bc0cd510 2023-12-01T03:08:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.279+0000 7ff3c1589640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc151a70 con 0x7ff3bc159db0 2023-12-01T03:08:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.279+0000 7ff3c1589640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0b1df0 con 0x7ff3bc0ca110 2023-12-01T03:08:48.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.279+0000 7ff3b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc0cd510 0x7ff3bc14ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.279+0000 7ff3b3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc0cd510 0x7ff3bc14ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58254/0 (socket says 172.21.15.115:58254) 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.279+0000 7ff3b3fff640 1 -- 172.21.15.115:0/2443337388 learned_addr learned my addr 172.21.15.115:0/2443337388 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3baffd640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3bc0ca110 0x7ff3bc152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3bb7fe640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3bc159db0 0x7ff3bc151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3b3fff640 1 -- 172.21.15.115:0/2443337388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3bc0ca110 msgr2=0x7ff3bc152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3b3fff640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3bc0ca110 0x7ff3bc152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3b3fff640 1 -- 172.21.15.115:0/2443337388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3bc159db0 msgr2=0x7ff3bc151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3b3fff640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3bc159db0 0x7ff3bc151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3b3fff640 1 -- 172.21.15.115:0/2443337388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0094d0 con 0x7ff3bc0cd510 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3baffd640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3bc0ca110 0x7ff3bc152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3b3fff640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc0cd510 0x7ff3bc14ca30 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400b3e0 tx=0x7ff3a400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3b37fe640 1 -- 172.21.15.115:0/2443337388 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011830 con 0x7ff3bc0cd510 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3b37fe640 1 -- 172.21.15.115:0/2443337388 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a40119b0 con 0x7ff3bc0cd510 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3b37fe640 1 -- 172.21.15.115:0/2443337388 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011d60 con 0x7ff3bc0cd510 2023-12-01T03:08:48.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.280+0000 7ff3c1589640 1 -- 172.21.15.115:0/2443337388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3bc0b20d0 con 0x7ff3bc0cd510 2023-12-01T03:08:48.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.281+0000 7ff3c1589640 1 -- 172.21.15.115:0/2443337388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3bc0b2690 con 0x7ff3bc0cd510 2023-12-01T03:08:48.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.282+0000 7ff3c1589640 1 -- 172.21.15.115:0/2443337388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3bc0cb7a0 con 0x7ff3bc0cd510 2023-12-01T03:08:48.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.283+0000 7ff3b37fe640 1 -- 172.21.15.115:0/2443337388 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff3a401a400 con 0x7ff3bc0cd510 2023-12-01T03:08:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.284+0000 7ff3b37fe640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff39c06a550 0x7ff39c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.284+0000 7ff3b37fe640 1 -- 172.21.15.115:0/2443337388 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7ff3a4088c30 con 0x7ff3bc0cd510 2023-12-01T03:08:48.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.284+0000 7ff3baffd640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff39c06a550 0x7ff39c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.285+0000 7ff3baffd640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff39c06a550 0x7ff39c06ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac036000 tx=0x7ff3ac03c040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:48.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.288+0000 7ff3b37fe640 1 -- 172.21.15.115:0/2443337388 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff3a40540e0 con 0x7ff3bc0cd510 2023-12-01T03:08:48.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.554+0000 7ff3c1589640 1 -- 172.21.15.115:0/2443337388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff3bc0b28b0 con 0x7ff3bc0cd510 2023-12-01T03:08:48.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.562+0000 7ff3b37fe640 1 -- 172.21.15.115:0/2443337388 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v47) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff3a4015020 con 0x7ff3bc0cd510 2023-12-01T03:08:48.564 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T03:08:48.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.564+0000 7ff3c1589640 1 -- 172.21.15.115:0/2443337388 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff39c06a550 msgr2=0x7ff39c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:48.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.564+0000 7ff3c1589640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff39c06a550 0x7ff39c06ca10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac036000 tx=0x7ff3ac03c040 comp rx=0 tx=0).stop 2023-12-01T03:08:48.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.564+0000 7ff3c1589640 1 -- 172.21.15.115:0/2443337388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc0cd510 msgr2=0x7ff3bc14ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:48.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.564+0000 7ff3c1589640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc0cd510 0x7ff3bc14ca30 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400b3e0 tx=0x7ff3a400b8a0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.564+0000 7ff3c1589640 1 -- 172.21.15.115:0/2443337388 shutdown_connections 2023-12-01T03:08:48.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.564+0000 7ff3c1589640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff39c06a550 0x7ff39c06ca10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.564+0000 7ff3c1589640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3bc159db0 0x7ff3bc151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.564+0000 7ff3c1589640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3bc0cd510 0x7ff3bc14ca30 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.564+0000 7ff3c1589640 1 --2- 172.21.15.115:0/2443337388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3bc0ca110 0x7ff3bc152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.564+0000 7ff3c1589640 1 -- 172.21.15.115:0/2443337388 >> 172.21.15.115:0/2443337388 conn(0x7ff3bc0cbb10 msgr2=0x7ff3bc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:48.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.565+0000 7ff3c1589640 1 -- 172.21.15.115:0/2443337388 shutdown_connections 2023-12-01T03:08:48.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.565+0000 7ff3c1589640 1 -- 172.21.15.115:0/2443337388 wait complete. 2023-12-01T03:08:48.586 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:08:48.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.790+0000 7f910ff39640 1 Processor -- start 2023-12-01T03:08:48.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.790+0000 7f910ff39640 1 -- start start 2023-12-01T03:08:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.791+0000 7f910ff39640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9108159dd0 0x7f91080c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.791+0000 7f910ff39640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9108070d30 0x7f91080c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.791+0000 7f910ff39640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910806f8b0 con 0x7f910806b6f0 2023-12-01T03:08:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.791+0000 7f910ff39640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91080adcb0 con 0x7f9108159dd0 2023-12-01T03:08:48.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.791+0000 7f910ff39640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91080ad9d0 con 0x7f9108070d30 2023-12-01T03:08:48.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910d4d4640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f910806b6f0 0x7f91080cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51524/0 (socket says 172.21.15.115:51524) 2023-12-01T03:08:48.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910d4d4640 1 -- 172.21.15.115:0/2350579342 learned_addr learned my addr 172.21.15.115:0/2350579342 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:48.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910e4d6640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9108070d30 0x7f91080c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910dcd5640 1 --2- 172.21.15.115:0/2350579342 >> v2:172.21.15.142:3300/0 conn(0x7f9108159dd0 0x7f91080c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408539258 0 0) 0x7f910806f8b0 con 0x7f910806b6f0 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90f8003520 con 0x7f910806b6f0 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2858975452 0 0) 0x7f90f8003520 con 0x7f910806b6f0 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9108070d30 msgr2=0x7f91080c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910ccd3640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9108070d30 0x7f91080c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 >> v2:172.21.15.142:3300/0 conn(0x7f9108159dd0 msgr2=0x7f91080c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910ccd3640 1 --2- 172.21.15.115:0/2350579342 >> v2:172.21.15.142:3300/0 conn(0x7f9108159dd0 0x7f91080c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91080c9000 con 0x7f910806b6f0 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f90f4001670 con 0x7f910806b6f0 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.792+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 >> v1:172.21.15.115:6789/0 conn(0x7f910806b6f0 legacy=0x7f91080cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910ccd3640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f80039e0 0x7f90f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910ccd3640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f80047c0 0x7f90f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910ccd3640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9108070d30 0x7f90f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91080adcb0 con 0x7f90f80039e0 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91080ad9d0 con 0x7f90f80047c0 2023-12-01T03:08:48.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f800dcf0 con 0x7f9108070d30 2023-12-01T03:08:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910dcd5640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f80039e0 0x7f90f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910d4d4640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f80047c0 0x7f90f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910e4d6640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9108070d30 0x7f90f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910dcd5640 1 -- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9108070d30 msgr2=0x7f90f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910dcd5640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9108070d30 0x7f90f8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910dcd5640 1 -- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f80047c0 msgr2=0x7f90f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910dcd5640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f80047c0 0x7f90f8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.793+0000 7f910dcd5640 1 -- 172.21.15.115:0/2350579342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f8003520 con 0x7f90f80039e0 2023-12-01T03:08:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.794+0000 7f910dcd5640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f80039e0 0x7f90f8003e00 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009f90 tx=0x7f90f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f573d57739bba8f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.795+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc02cbf0 con 0x7f90f80039e0 2023-12-01T03:08:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.795+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc00f040 con 0x7f90f80039e0 2023-12-01T03:08:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.795+0000 7f910ccd3640 1 -- 172.21.15.115:0/2350579342 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc034440 con 0x7f90f80039e0 2023-12-01T03:08:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.795+0000 7f910ff39640 1 -- 172.21.15.115:0/2350579342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f80039e0 msgr2=0x7f90f8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.795+0000 7f910ff39640 1 --2- 172.21.15.115:0/2350579342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90f80039e0 0x7f90f8003e00 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009f90 tx=0x7f90f80057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.795+0000 7f910e4d6640 1 -- 172.21.15.115:0/2350579342 reap_dead start 2023-12-01T03:08:48.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.795+0000 7f910ff39640 1 -- 172.21.15.115:0/2350579342 shutdown_connections 2023-12-01T03:08:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.795+0000 7f910ff39640 1 -- 172.21.15.115:0/2350579342 >> 172.21.15.115:0/2350579342 conn(0x7f91080ce880 msgr2=0x7f91080cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.796+0000 7f910ff39640 1 -- 172.21.15.115:0/2350579342 shutdown_connections 2023-12-01T03:08:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.796+0000 7f910ff39640 1 -- 172.21.15.115:0/2350579342 wait complete. 2023-12-01T03:08:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.797+0000 7f910ff39640 1 Processor -- start 2023-12-01T03:08:48.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.797+0000 7f910ff39640 1 -- start start 2023-12-01T03:08:48.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.798+0000 7f910ff39640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90f80039e0 0x7f910814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.798+0000 7f910ff39640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91080b16a0 0x7f910814c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.798+0000 7f910ff39640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080aeed0 0x7f91080af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.798+0000 7f910ff39640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91080b28e0 con 0x7f91080aeed0 2023-12-01T03:08:48.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.798+0000 7f910ff39640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91080af890 con 0x7f91080b16a0 2023-12-01T03:08:48.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.798+0000 7f910ff39640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910814dfa0 con 0x7f90f80039e0 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910e4d6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080aeed0 0x7f91080af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910e4d6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080aeed0 0x7f91080af350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58270/0 (socket says 172.21.15.115:58270) 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910e4d6640 1 -- 172.21.15.115:0/3319340897 learned_addr learned my addr 172.21.15.115:0/3319340897 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910d4d4640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91080b16a0 0x7f910814c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910dcd5640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90f80039e0 0x7f910814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910e4d6640 1 -- 172.21.15.115:0/3319340897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90f80039e0 msgr2=0x7f910814da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910e4d6640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90f80039e0 0x7f910814da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910e4d6640 1 -- 172.21.15.115:0/3319340897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91080b16a0 msgr2=0x7f910814c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910e4d6640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91080b16a0 0x7f910814c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910e4d6640 1 -- 172.21.15.115:0/3319340897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910814cf30 con 0x7f91080aeed0 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910d4d4640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91080b16a0 0x7f910814c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.799+0000 7f910e4d6640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080aeed0 0x7f91080af350 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f90f8004740 tx=0x7f9100010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:48.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.800+0000 7f91067fc640 1 -- 172.21.15.115:0/3319340897 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910000e8a0 con 0x7f91080aeed0 2023-12-01T03:08:48.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.800+0000 7f91067fc640 1 -- 172.21.15.115:0/3319340897 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910000ea00 con 0x7f91080aeed0 2023-12-01T03:08:48.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.800+0000 7f91067fc640 1 -- 172.21.15.115:0/3319340897 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910000edb0 con 0x7f91080aeed0 2023-12-01T03:08:48.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.800+0000 7f910ff39640 1 -- 172.21.15.115:0/3319340897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91080b1c60 con 0x7f91080aeed0 2023-12-01T03:08:48.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.800+0000 7f910ff39640 1 -- 172.21.15.115:0/3319340897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91080b2220 con 0x7f91080aeed0 2023-12-01T03:08:48.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.801+0000 7f910ff39640 1 -- 172.21.15.115:0/3319340897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91080b2a20 con 0x7f91080aeed0 2023-12-01T03:08:48.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.802+0000 7f91067fc640 1 -- 172.21.15.115:0/3319340897 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9100020410 con 0x7f91080aeed0 2023-12-01T03:08:48.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.803+0000 7f91067fc640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90ec06a550 0x7f90ec06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:48.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.803+0000 7f91067fc640 1 -- 172.21.15.115:0/3319340897 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f9100088580 con 0x7f91080aeed0 2023-12-01T03:08:48.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.803+0000 7f910dcd5640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90ec06a550 0x7f90ec06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:48.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.804+0000 7f910dcd5640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90ec06a550 0x7f90ec06ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f90fc036000 tx=0x7f90fc02e000 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:48.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:48.807+0000 7f91067fc640 1 -- 172.21.15.115:0/3319340897 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f91080b2a20 con 0x7f91080aeed0 2023-12-01T03:08:49.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.033+0000 7f910ff39640 1 -- 172.21.15.115:0/3319340897 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91080b2a20 con 0x7f91080aeed0 2023-12-01T03:08:49.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.034+0000 7f91067fc640 1 -- 172.21.15.115:0/3319340897 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f91080b2a20 con 0x7f91080aeed0 2023-12-01T03:08:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:49.036 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/974148243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":974148243},{"type":"v1","addr":"172.21.15.115:6837","nonce":974148243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/476969842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":476969842},{"type":"v1","addr":"172.21.15.142:6834","nonce":476969842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1103499235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1103499235},{"type":"v1","addr":"172.21.15.115:6835","nonce":1103499235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:08:43.078263+0000","modified":"2023-12-01T03:08:46.096728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10149":{"gid":10149,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/3801423208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3801423208},{"type":"v1","addr":"172.21.15.142:6835","nonce":3801423208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-01T03:08:49.037 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 87 2023-12-01T03:08:49.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.036+0000 7f910ff39640 1 -- 172.21.15.115:0/3319340897 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90ec06a550 msgr2=0x7f90ec06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90ec06a550 0x7f90ec06ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f90fc036000 tx=0x7f90fc02e000 comp rx=0 tx=0).stop 2023-12-01T03:08:49.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 -- 172.21.15.115:0/3319340897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080aeed0 msgr2=0x7f91080af350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080aeed0 0x7f91080af350 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f90f8004740 tx=0x7f9100010c20 comp rx=0 tx=0).stop 2023-12-01T03:08:49.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 -- 172.21.15.115:0/3319340897 shutdown_connections 2023-12-01T03:08:49.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f90ec06a550 0x7f90ec06ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91080aeed0 0x7f91080af350 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91080b16a0 0x7f910814c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 --2- 172.21.15.115:0/3319340897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90f80039e0 0x7f910814da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 -- 172.21.15.115:0/3319340897 >> 172.21.15.115:0/3319340897 conn(0x7f91080ce880 msgr2=0x7f9108152020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:49.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 -- 172.21.15.115:0/3319340897 shutdown_connections 2023-12-01T03:08:49.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.037+0000 7f910ff39640 1 -- 172.21.15.115:0/3319340897 wait complete. 2023-12-01T03:08:49.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T03:08:43.078263+0000', 'modified': '2023-12-01T03:08:46.096728+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10149}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10149': {'gid': 10149, 'name': 'd', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6835/3801423208', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6833', 'nonce': 3801423208}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 3801423208}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T03:08:49.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T03:08:49.460 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-01T03:08:49.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.685+0000 7f4f845ae640 1 Processor -- start 2023-12-01T03:08:49.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.685+0000 7f4f845ae640 1 -- start start 2023-12-01T03:08:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.686+0000 7f4f845ae640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4f7c159dd0 0x7f4f7c0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.686+0000 7f4f845ae640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4f7c06b6f0 0x7f4f7c0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.686+0000 7f4f845ae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7c0ade80 0x7f4f7c0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.686+0000 7f4f845ae640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f7c06f8b0 con 0x7f4f7c159dd0 2023-12-01T03:08:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.686+0000 7f4f845ae640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f7c0adcb0 con 0x7f4f7c06b6f0 2023-12-01T03:08:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.686+0000 7f4f845ae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c0ad9d0 con 0x7f4f7c0ade80 2023-12-01T03:08:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.686+0000 7f4f8234a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4f7c159dd0 0x7f4f7c0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.686+0000 7f4f8234a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4f7c159dd0 0x7f4f7c0c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58276/0 (socket says 172.21.15.115:58276) 2023-12-01T03:08:49.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.686+0000 7f4f8234a640 1 -- 172.21.15.115:0/2017035832 learned_addr learned my addr 172.21.15.115:0/2017035832 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:49.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.687+0000 7f4f8234a640 1 -- 172.21.15.115:0/2017035832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7c0ade80 msgr2=0x7f4f7c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.687+0000 7f4f8234a640 1 --2- 172.21.15.115:0/2017035832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7c0ade80 0x7f4f7c0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.687+0000 7f4f8234a640 1 -- 172.21.15.115:0/2017035832 >> v2:172.21.15.142:3300/0 conn(0x7f4f7c06b6f0 msgr2=0x7f4f7c0c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.687+0000 7f4f8234a640 1 --2- 172.21.15.115:0/2017035832 >> v2:172.21.15.142:3300/0 conn(0x7f4f7c06b6f0 0x7f4f7c0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.687+0000 7f4f8234a640 1 -- 172.21.15.115:0/2017035832 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c0c9000 con 0x7f4f7c159dd0 2023-12-01T03:08:49.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.688+0000 7f4f8234a640 1 --2- 172.21.15.115:0/2017035832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c159dd0 0x7f4f7c0c97e0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c06de80 tx=0x7f4f7802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a6e786c2babed9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:49.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.688+0000 7f4f81348640 1 -- 172.21.15.115:0/2017035832 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78031750 con 0x7f4f7c159dd0 2023-12-01T03:08:49.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.688+0000 7f4f81348640 1 -- 172.21.15.115:0/2017035832 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f780318b0 con 0x7f4f7c159dd0 2023-12-01T03:08:49.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.688+0000 7f4f81348640 1 -- 172.21.15.115:0/2017035832 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78031c40 con 0x7f4f7c159dd0 2023-12-01T03:08:49.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.689+0000 7f4f845ae640 1 -- 172.21.15.115:0/2017035832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c159dd0 msgr2=0x7f4f7c0c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.689+0000 7f4f845ae640 1 --2- 172.21.15.115:0/2017035832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c159dd0 0x7f4f7c0c97e0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c06de80 tx=0x7f4f7802b7f0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.689+0000 7f4f845ae640 1 -- 172.21.15.115:0/2017035832 shutdown_connections 2023-12-01T03:08:49.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.689+0000 7f4f845ae640 1 --2- 172.21.15.115:0/2017035832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7c0ade80 0x7f4f7c0cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.689+0000 7f4f845ae640 1 --2- 172.21.15.115:0/2017035832 >> v2:172.21.15.142:3300/0 conn(0x7f4f7c06b6f0 0x7f4f7c0c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.689+0000 7f4f845ae640 1 --2- 172.21.15.115:0/2017035832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c159dd0 0x7f4f7c0c97e0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.689+0000 7f4f845ae640 1 -- 172.21.15.115:0/2017035832 >> 172.21.15.115:0/2017035832 conn(0x7f4f7c0ce880 msgr2=0x7f4f7c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.689+0000 7f4f845ae640 1 -- 172.21.15.115:0/2017035832 shutdown_connections 2023-12-01T03:08:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.689+0000 7f4f845ae640 1 -- 172.21.15.115:0/2017035832 wait complete. 2023-12-01T03:08:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.691+0000 7f4f845ae640 1 Processor -- start 2023-12-01T03:08:49.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.691+0000 7f4f845ae640 1 -- start start 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.692+0000 7f4f845ae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c06b6f0 0x7f4f7c14fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.692+0000 7f4f845ae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f7c0ade80 0x7f4f7c0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.692+0000 7f4f845ae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7c159dd0 0x7f4f7c0c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.692+0000 7f4f845ae640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c0c5d80 con 0x7f4f7c06b6f0 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.692+0000 7f4f845ae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c1e4720 con 0x7f4f7c0ade80 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.692+0000 7f4f845ae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c1e4a00 con 0x7f4f7c159dd0 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.692+0000 7f4f8234a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c06b6f0 0x7f4f7c14fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.692+0000 7f4f8234a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c06b6f0 0x7f4f7c14fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58292/0 (socket says 172.21.15.115:58292) 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.692+0000 7f4f8234a640 1 -- 172.21.15.115:0/2970414592 learned_addr learned my addr 172.21.15.115:0/2970414592 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f8234a640 1 -- 172.21.15.115:0/2970414592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7c159dd0 msgr2=0x7f4f7c0c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f8234a640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7c159dd0 0x7f4f7c0c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f8234a640 1 -- 172.21.15.115:0/2970414592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f7c0ade80 msgr2=0x7f4f7c0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f81b49640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f7c0ade80 0x7f4f7c0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f8234a640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f7c0ade80 0x7f4f7c0c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f8234a640 1 -- 172.21.15.115:0/2970414592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f780094d0 con 0x7f4f7c06b6f0 2023-12-01T03:08:49.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f8234a640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c06b6f0 0x7f4f7c14fec0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f4f78000c00 tx=0x7f4f780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:49.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f72ffd640 1 -- 172.21.15.115:0/2970414592 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78033c60 con 0x7f4f7c06b6f0 2023-12-01T03:08:49.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f72ffd640 1 -- 172.21.15.115:0/2970414592 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f78033de0 con 0x7f4f7c06b6f0 2023-12-01T03:08:49.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f72ffd640 1 -- 172.21.15.115:0/2970414592 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7803d6b0 con 0x7f4f7c06b6f0 2023-12-01T03:08:49.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.693+0000 7f4f845ae640 1 -- 172.21.15.115:0/2970414592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7c1e4ce0 con 0x7f4f7c06b6f0 2023-12-01T03:08:49.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.694+0000 7f4f845ae640 1 -- 172.21.15.115:0/2970414592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7c0c5950 con 0x7f4f7c06b6f0 2023-12-01T03:08:49.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.696+0000 7f4f72ffd640 1 -- 172.21.15.115:0/2970414592 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4f7803a040 con 0x7f4f7c06b6f0 2023-12-01T03:08:49.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.697+0000 7f4f72ffd640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f6006a550 0x7f4f6006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.697+0000 7f4f72ffd640 1 -- 172.21.15.115:0/2970414592 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f4f780aaba0 con 0x7f4f7c06b6f0 2023-12-01T03:08:49.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.697+0000 7f4f81b49640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f6006a550 0x7f4f6006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.698+0000 7f4f871f3740 1 Processor -- start 2023-12-01T03:08:49.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.698+0000 7f4f81b49640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f6006a550 0x7f4f6006ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c150850 tx=0x7f4f6c008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:49.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.698+0000 7f4f871f3740 1 -- start start 2023-12-01T03:08:49.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.699+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb2650 0x55dd5bc94730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.699+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb0170 0x55dd5bc94c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.699+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bc91d60 0x55dd5bc684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.699+0000 7f4f871f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bc402f0 con 0x55dd5bcb0170 2023-12-01T03:08:49.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.699+0000 7f4f871f3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bbd4f80 con 0x55dd5bc91d60 2023-12-01T03:08:49.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.699+0000 7f4f871f3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bbd2710 con 0x55dd5bcb2650 2023-12-01T03:08:49.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.699+0000 7f4f81b49640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb0170 0x55dd5bc94c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.699+0000 7f4f81b49640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb0170 0x55dd5bc94c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58306/0 (socket says 172.21.15.115:58306) 2023-12-01T03:08:49.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.699+0000 7f4f81b49640 1 -- 172.21.15.115:0/399640477 learned_addr learned my addr 172.21.15.115:0/399640477 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:49.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f8234a640 1 --2- 172.21.15.115:0/399640477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb2650 0x55dd5bc94730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f82b4b640 1 --2- 172.21.15.115:0/399640477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bc91d60 0x55dd5bc684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f81b49640 1 -- 172.21.15.115:0/399640477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb2650 msgr2=0x55dd5bc94730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f81b49640 1 --2- 172.21.15.115:0/399640477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb2650 0x55dd5bc94730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f81b49640 1 -- 172.21.15.115:0/399640477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bc91d60 msgr2=0x55dd5bc684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f81b49640 1 --2- 172.21.15.115:0/399640477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bc91d60 0x55dd5bc684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f81b49640 1 -- 172.21.15.115:0/399640477 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd5bb95690 con 0x55dd5bcb0170 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f81b49640 1 --2- 172.21.15.115:0/399640477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb0170 0x55dd5bc94c70 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00c9e0 tx=0x7f4f6c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37d4633bd265075 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f5ffff640 1 -- 172.21.15.115:0/399640477 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c013db0 con 0x55dd5bcb0170 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f5ffff640 1 -- 172.21.15.115:0/399640477 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c01e400 con 0x55dd5bcb0170 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.700+0000 7f4f5ffff640 1 -- 172.21.15.115:0/399640477 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c01e770 con 0x55dd5bcb0170 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.701+0000 7f4f871f3740 1 -- 172.21.15.115:0/399640477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb0170 msgr2=0x55dd5bc94c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.701+0000 7f4f871f3740 1 --2- 172.21.15.115:0/399640477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb0170 0x55dd5bc94c70 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00c9e0 tx=0x7f4f6c00f000 comp rx=0 tx=0).stop 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.701+0000 7f4f871f3740 1 -- 172.21.15.115:0/399640477 shutdown_connections 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.701+0000 7f4f871f3740 1 --2- 172.21.15.115:0/399640477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bc91d60 0x55dd5bc684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.701+0000 7f4f871f3740 1 --2- 172.21.15.115:0/399640477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb0170 0x55dd5bc94c70 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.701+0000 7f4f871f3740 1 --2- 172.21.15.115:0/399640477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb2650 0x55dd5bc94730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.701+0000 7f4f871f3740 1 -- 172.21.15.115:0/399640477 >> 172.21.15.115:0/399640477 conn(0x55dd5bc66bb0 msgr2=0x55dd5bcaf730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:49.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.701+0000 7f4f871f3740 1 -- 172.21.15.115:0/399640477 shutdown_connections 2023-12-01T03:08:49.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.701+0000 7f4f871f3740 1 -- 172.21.15.115:0/399640477 wait complete. 2023-12-01T03:08:49.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.702+0000 7f4f871f3740 1 -- start start 2023-12-01T03:08:49.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.702+0000 7f4f871f3740 1 Processor -- start 2023-12-01T03:08:49.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.703+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb0170 0x55dd5bcb18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.703+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.703+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb11f0 0x55dd5bc8f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.703+0000 7f4f871f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bbce000 con 0x55dd5bcb2650 2023-12-01T03:08:49.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.703+0000 7f4f871f3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bb6f7d0 con 0x55dd5bcb11f0 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.703+0000 7f4f871f3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bb6f470 con 0x55dd5bcb0170 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.703+0000 7f4f871f3740 10 client.0 _ll_register_callbacks cb 0x55dd5bbbe1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.703+0000 7f4f81b49640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.703+0000 7f4f81b49640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58320/0 (socket says 172.21.15.115:58320) 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.703+0000 7f4f81b49640 1 -- 172.21.15.115:0/1206239815 learned_addr learned my addr 172.21.15.115:0/1206239815 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.704+0000 7f4f81b49640 1 -- 172.21.15.115:0/1206239815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb0170 msgr2=0x55dd5bcb18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.704+0000 7f4f81b49640 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb0170 0x55dd5bcb18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.704+0000 7f4f81b49640 1 -- 172.21.15.115:0/1206239815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb11f0 msgr2=0x55dd5bc8f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.704+0000 7f4f81b49640 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb11f0 0x55dd5bc8f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.704+0000 7f4f81b49640 1 -- 172.21.15.115:0/1206239815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c00c7d0 con 0x55dd5bcb2650 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.704+0000 7f4f81b49640 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb1de0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0137f0 tx=0x7f4f6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45347595d988498 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.704+0000 7f4f5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.704+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/1206239815 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c01e980 con 0x55dd5bcb2650 2023-12-01T03:08:49.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.704+0000 7f4f871f3740 1 -- 172.21.15.115:0/1206239815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd5bbf8550 con 0x55dd5bcb2650 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.704+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/1206239815 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f4f6c030ae0 con 0x55dd5bcb2650 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/1206239815 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f6c030ca0 con 0x55dd5bcb2650 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f871f3740 10 client.10269 fetch_fsmap learned FSMap version 87 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f871f3740 1 -- 172.21.15.115:0/1206239815 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55dd5bbcc680 con 0x55dd5bcb2650 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/1206239815 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dd5bbf8550 con 0x55dd5bcb2650 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/1206239815 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 87) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4f6c02d020 con 0x55dd5bcb2650 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f871f3740 10 client.10269 fetch_fsmap finished waiting for FSMap version 87 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f871f3740 10 client.10269 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f871f3740 20 client.10269 populate_metadata read hostname 'smithi115' 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f871f3740 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.142:6833/3801423208,v1:172.21.15.142:6835/3801423208] conn(0x55dd5bcb9f70 0x55dd5bcba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f871f3740 4 client.10269 mds_command: new command op to 10149 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.705+0000 7f4f871f3740 1 -- 172.21.15.115:0/1206239815 --> [v2:172.21.15.142:6833/3801423208,v1:172.21.15.142:6835/3801423208] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd5bbc0b50 con 0x55dd5bcb9f70 2023-12-01T03:08:49.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.706+0000 7f4f8234a640 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.142:6833/3801423208,v1:172.21.15.142:6835/3801423208] conn(0x55dd5bcb9f70 0x55dd5bcba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.707+0000 7f4f8234a640 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.142:6833/3801423208,v1:172.21.15.142:6835/3801423208] conn(0x55dd5bcb9f70 0x55dd5bcba370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbab93218be579e0 server_cookie=247ebc3d66f6495d in_seq=0 out_seq=0 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.707+0000 7f4f5d7fa640 10 client.10269 ms_handle_connect on v2:172.21.15.142:6833/3801423208 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.708+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/1206239815 <== mds.0 v2:172.21.15.142:6833/3801423208 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55dd5bbc0b50 con 0x55dd5bcb9f70 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.708+0000 7f4f5d7fa640 10 client.10269 handle_command_reply: tid=0 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 client.10269 shutdown 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 -- 172.21.15.115:0/1206239815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 msgr2=0x55dd5bcb1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb1de0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0137f0 tx=0x7f4f6c0046c0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 -- 172.21.15.115:0/1206239815 shutdown_connections 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.142:6833/3801423208,v1:172.21.15.142:6835/3801423208] conn(0x55dd5bcb9f70 0x55dd5bcba370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb11f0 0x55dd5bc8f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb1de0 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 --2- 172.21.15.115:0/1206239815 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb0170 0x55dd5bcb18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f5d7fa640 0 client.10269 ms_handle_reset on v2:172.21.15.142:6833/3801423208 2023-12-01T03:08:49.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 -- 172.21.15.115:0/1206239815 >> 172.21.15.115:0/1206239815 conn(0x55dd5bc91d60 msgr2=0x55dd5bcb0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:49.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 -- 172.21.15.115:0/1206239815 shutdown_connections 2023-12-01T03:08:49.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 1 -- 172.21.15.115:0/1206239815 wait complete. 2023-12-01T03:08:49.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.709+0000 7f4f871f3740 20 client.10269 trim_cache size 0 max 16384 2023-12-01T03:08:49.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.725+0000 7f4f871f3740 1 Processor -- start 2023-12-01T03:08:49.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.725+0000 7f4f871f3740 1 -- start start 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.726+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb0170 0x55dd5bc6ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.726+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb11f0 0x55dd5bc6a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.726+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb2650 0x55dd5bcbb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.726+0000 7f4f871f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5b935f20 con 0x55dd5bcb11f0 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.726+0000 7f4f871f3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bcc37b0 con 0x55dd5bcb0170 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.726+0000 7f4f871f3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bc6b8d0 con 0x55dd5bcb2650 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.726+0000 7f4f81b49640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb11f0 0x55dd5bc6a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f81b49640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb11f0 0x55dd5bc6a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58326/0 (socket says 172.21.15.115:58326) 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f81b49640 1 -- 172.21.15.115:0/1028811677 learned_addr learned my addr 172.21.15.115:0/1028811677 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f81b49640 1 -- 172.21.15.115:0/1028811677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb2650 msgr2=0x55dd5bcbb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f82b4b640 1 --2- 172.21.15.115:0/1028811677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb2650 0x55dd5bcbb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f8234a640 1 --2- 172.21.15.115:0/1028811677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb0170 0x55dd5bc6ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f81b49640 1 --2- 172.21.15.115:0/1028811677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb2650 0x55dd5bcbb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f81b49640 1 -- 172.21.15.115:0/1028811677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb0170 msgr2=0x55dd5bc6ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f81b49640 1 --2- 172.21.15.115:0/1028811677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb0170 0x55dd5bc6ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f81b49640 1 -- 172.21.15.115:0/1028811677 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd5bb6e9e0 con 0x55dd5bcb11f0 2023-12-01T03:08:49.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f81b49640 1 --2- 172.21.15.115:0/1028811677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb11f0 0x55dd5bc6a7d0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c006eb0 tx=0x7f4f6c026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b068883c7bf6ff1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.727+0000 7f4f70ff9640 1 -- 172.21.15.115:0/1028811677 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c01e980 con 0x55dd5bcb11f0 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f70ff9640 1 -- 172.21.15.115:0/1028811677 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c01eb80 con 0x55dd5bcb11f0 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f70ff9640 1 -- 172.21.15.115:0/1028811677 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c042470 con 0x55dd5bcb11f0 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f871f3740 1 -- 172.21.15.115:0/1028811677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb11f0 msgr2=0x55dd5bc6a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f871f3740 1 --2- 172.21.15.115:0/1028811677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb11f0 0x55dd5bc6a7d0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c006eb0 tx=0x7f4f6c026000 comp rx=0 tx=0).stop 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f871f3740 1 -- 172.21.15.115:0/1028811677 shutdown_connections 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f871f3740 1 --2- 172.21.15.115:0/1028811677 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb2650 0x55dd5bcbb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f871f3740 1 --2- 172.21.15.115:0/1028811677 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb11f0 0x55dd5bc6a7d0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f871f3740 1 --2- 172.21.15.115:0/1028811677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb0170 0x55dd5bc6ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f871f3740 1 -- 172.21.15.115:0/1028811677 >> 172.21.15.115:0/1028811677 conn(0x55dd5bc91d60 msgr2=0x55dd5bc60040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:49.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f871f3740 1 -- 172.21.15.115:0/1028811677 shutdown_connections 2023-12-01T03:08:49.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.728+0000 7f4f871f3740 1 -- 172.21.15.115:0/1028811677 wait complete. 2023-12-01T03:08:49.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.729+0000 7f4f871f3740 1 -- start start 2023-12-01T03:08:49.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.729+0000 7f4f871f3740 1 Processor -- start 2023-12-01T03:08:49.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.730+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb0170 0x55dd5bcd61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.730+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb11f0 0x55dd5bd18940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.730+0000 7f4f871f3740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.730+0000 7f4f871f3740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bccd060 con 0x55dd5bcb2650 2023-12-01T03:08:49.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.730+0000 7f4f871f3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bcad690 con 0x55dd5bcb0170 2023-12-01T03:08:49.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.730+0000 7f4f871f3740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd5bd18e80 con 0x55dd5bcb11f0 2023-12-01T03:08:49.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.730+0000 7f4f871f3740 10 client.0 _ll_register_callbacks cb 0x7f4f6c00adc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:08:49.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.730+0000 7f4f82b4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.730+0000 7f4f82b4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58332/0 (socket says 172.21.15.115:58332) 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f82b4b640 1 -- 172.21.15.115:0/2885182135 learned_addr learned my addr 172.21.15.115:0/2885182135 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f8234a640 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb0170 0x55dd5bcd61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f81b49640 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb11f0 0x55dd5bd18940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f82b4b640 1 -- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb11f0 msgr2=0x55dd5bd18940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f82b4b640 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb11f0 0x55dd5bd18940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f82b4b640 1 -- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb0170 msgr2=0x55dd5bcd61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f82b4b640 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb0170 0x55dd5bcd61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f82b4b640 1 -- 172.21.15.115:0/2885182135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd5bbcc680 con 0x55dd5bcb2650 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f81b49640 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb11f0 0x55dd5bd18940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f82b4b640 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb0550 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f4f740070b0 tx=0x7f4f7400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43828634ce68f485 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/2885182135 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74011070 con 0x55dd5bcb2650 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/2885182135 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f4f7400c0d0 con 0x55dd5bcb2650 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.731+0000 7f4f871f3740 1 -- 172.21.15.115:0/2885182135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f6c01e400 con 0x55dd5bcb2650 2023-12-01T03:08:49.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/2885182135 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f74009790 con 0x55dd5bcb2650 2023-12-01T03:08:49.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f871f3740 10 client.10275 fetch_fsmap learned FSMap version 87 2023-12-01T03:08:49.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f871f3740 1 -- 172.21.15.115:0/2885182135 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55dd5bb93ef0 con 0x55dd5bcb2650 2023-12-01T03:08:49.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/2885182135 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c01e400 con 0x55dd5bcb2650 2023-12-01T03:08:49.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/2885182135 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 87) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4f74008690 con 0x55dd5bcb2650 2023-12-01T03:08:49.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f871f3740 10 client.10275 fetch_fsmap finished waiting for FSMap version 87 2023-12-01T03:08:49.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f871f3740 10 client.10275 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-01T03:08:49.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f871f3740 20 client.10275 populate_metadata read hostname 'smithi115' 2023-12-01T03:08:49.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f871f3740 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:6833/3801423208,v1:172.21.15.142:6835/3801423208] conn(0x55dd5bcb2d30 0x55dd5bcb3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f871f3740 4 client.10275 mds_command: new command op to 10149 tid=0[{"prefix": "status"}] 2023-12-01T03:08:49.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.732+0000 7f4f871f3740 1 -- 172.21.15.115:0/2885182135 --> [v2:172.21.15.142:6833/3801423208,v1:172.21.15.142:6835/3801423208] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dd5bbc7080 con 0x55dd5bcb2d30 2023-12-01T03:08:49.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.733+0000 7f4f8234a640 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:6833/3801423208,v1:172.21.15.142:6835/3801423208] conn(0x55dd5bcb2d30 0x55dd5bcb3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.734+0000 7f4f5d7fa640 10 client.10275 ms_handle_connect on v2:172.21.15.142:6833/3801423208 2023-12-01T03:08:49.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.734+0000 7f4f8234a640 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:6833/3801423208,v1:172.21.15.142:6835/3801423208] conn(0x55dd5bcb2d30 0x55dd5bcb3110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf2a5a632eb37b9 server_cookie=41cca433bd2f1b7a in_seq=0 out_seq=0 2023-12-01T03:08:49.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.734+0000 7f4f5d7fa640 1 -- 172.21.15.115:0/2885182135 <== mds.0 v2:172.21.15.142:6833/3801423208 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55dd5bbc7080 con 0x55dd5bcb2d30 2023-12-01T03:08:49.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.734+0000 7f4f5d7fa640 10 client.10275 handle_command_reply: tid=0 2023-12-01T03:08:49.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.734+0000 7f4f871f3740 1 client.10275 shutdown 2023-12-01T03:08:49.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 1 -- 172.21.15.115:0/2885182135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 msgr2=0x55dd5bcb0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb0550 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f4f740070b0 tx=0x7f4f7400cb30 comp rx=0 tx=0).stop 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 1 -- 172.21.15.115:0/2885182135 shutdown_connections 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:6833/3801423208,v1:172.21.15.142:6835/3801423208] conn(0x55dd5bcb2d30 0x55dd5bcb3110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55dd5bcb11f0 0x55dd5bd18940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55dd5bcb2650 0x55dd5bcb0550 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 1 --2- 172.21.15.115:0/2885182135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dd5bcb0170 0x55dd5bcd61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f5d7fa640 0 client.10275 ms_handle_reset on v2:172.21.15.142:6833/3801423208 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 1 -- 172.21.15.115:0/2885182135 >> 172.21.15.115:0/2885182135 conn(0x55dd5bc91d60 msgr2=0x55dd5bcaf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 1 -- 172.21.15.115:0/2885182135 shutdown_connections 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 1 -- 172.21.15.115:0/2885182135 wait complete. 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.735+0000 7f4f871f3740 20 client.10275 trim_cache size 0 max 16384 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:08:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 10149, 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 6.6512222059999999, 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 87, 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 196, 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 196, 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 12.519705516 2023-12-01T03:08:49.740 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:08:49.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 -- 172.21.15.115:0/2970414592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f6006a550 msgr2=0x7f4f6006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f6006a550 0x7f4f6006ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c150850 tx=0x7f4f6c008040 comp rx=0 tx=0).stop 2023-12-01T03:08:49.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 -- 172.21.15.115:0/2970414592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c06b6f0 msgr2=0x7f4f7c14fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c06b6f0 0x7f4f7c14fec0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f4f78000c00 tx=0x7f4f780046c0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 -- 172.21.15.115:0/2970414592 shutdown_connections 2023-12-01T03:08:49.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f6006a550 0x7f4f6006ca10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7c159dd0 0x7f4f7c0c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f7c0ade80 0x7f4f7c0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 --2- 172.21.15.115:0/2970414592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7c06b6f0 0x7f4f7c14fec0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 -- 172.21.15.115:0/2970414592 >> 172.21.15.115:0/2970414592 conn(0x7f4f7c0ce880 msgr2=0x7f4f7c0c1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:49.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 -- 172.21.15.115:0/2970414592 shutdown_connections 2023-12-01T03:08:49.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.736+0000 7f4f845ae640 1 -- 172.21.15.115:0/2970414592 wait complete. 2023-12-01T03:08:49.754 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T03:08:49.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.950+0000 7f8982ac5640 1 Processor -- start 2023-12-01T03:08:49.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.950+0000 7f8982ac5640 1 -- start start 2023-12-01T03:08:49.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.951+0000 7f8982ac5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f897c06b270 0x7f897c0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.951+0000 7f8982ac5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c0ce120 con 0x7f897c0cdcc0 2023-12-01T03:08:49.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.951+0000 7f8982ac5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c06b700 con 0x7f897c1597d0 2023-12-01T03:08:49.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.951+0000 7f8982ac5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c06b9c0 con 0x7f897c06b270 2023-12-01T03:08:49.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.952+0000 7f89812c2640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f897c0cdcc0 0x7f897c0adf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51528/0 (socket says 172.21.15.115:51528) 2023-12-01T03:08:49.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.952+0000 7f89812c2640 1 -- 172.21.15.115:0/999608858 learned_addr learned my addr 172.21.15.115:0/999608858 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:49.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.952+0000 7f89822c4640 1 --2- 172.21.15.115:0/999608858 >> v2:172.21.15.115:3300/0 conn(0x7f897c06b270 0x7f897c0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.952+0000 7f8980ac1640 1 -- 172.21.15.115:0/999608858 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171869083 0 0) 0x7f897c0ce120 con 0x7f897c0cdcc0 2023-12-01T03:08:49.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.952+0000 7f8980ac1640 1 -- 172.21.15.115:0/999608858 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8968005730 con 0x7f897c0cdcc0 2023-12-01T03:08:49.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.952+0000 7f89822c4640 1 -- 172.21.15.115:0/999608858 >> v1:172.21.15.142:6789/0 conn(0x7f897c1597d0 legacy=0x7f897c15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:49.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.952+0000 7f89822c4640 1 -- 172.21.15.115:0/999608858 >> v1:172.21.15.115:6789/0 conn(0x7f897c0cdcc0 legacy=0x7f897c0adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.952+0000 7f89822c4640 1 -- 172.21.15.115:0/999608858 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c06bca0 con 0x7f897c06b270 2023-12-01T03:08:49.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.953+0000 7f89822c4640 1 --2- 172.21.15.115:0/999608858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c06b270 0x7f897c0ca350 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f8974009d90 tx=0x7f897402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc57575c42f6b6af server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:49.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.954+0000 7f8980ac1640 1 -- 172.21.15.115:0/999608858 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897402cbd0 con 0x7f897c06b270 2023-12-01T03:08:49.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.954+0000 7f8980ac1640 1 -- 172.21.15.115:0/999608858 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897402cd30 con 0x7f897c06b270 2023-12-01T03:08:49.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.954+0000 7f8980ac1640 1 -- 172.21.15.115:0/999608858 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897402cea0 con 0x7f897c06b270 2023-12-01T03:08:49.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.954+0000 7f8982ac5640 1 -- 172.21.15.115:0/999608858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c06b270 msgr2=0x7f897c0ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.954+0000 7f8982ac5640 1 --2- 172.21.15.115:0/999608858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c06b270 0x7f897c0ca350 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f8974009d90 tx=0x7f897402be20 comp rx=0 tx=0).stop 2023-12-01T03:08:49.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.954+0000 7f8982ac5640 1 -- 172.21.15.115:0/999608858 shutdown_connections 2023-12-01T03:08:49.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.954+0000 7f8982ac5640 1 --2- 172.21.15.115:0/999608858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c06b270 0x7f897c0ca350 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.954+0000 7f8982ac5640 1 -- 172.21.15.115:0/999608858 >> 172.21.15.115:0/999608858 conn(0x7f897c0ce640 msgr2=0x7f897c0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:49.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.954+0000 7f8982ac5640 1 -- 172.21.15.115:0/999608858 shutdown_connections 2023-12-01T03:08:49.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.955+0000 7f8982ac5640 1 -- 172.21.15.115:0/999608858 wait complete. 2023-12-01T03:08:49.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.956+0000 7f8982ac5640 1 Processor -- start 2023-12-01T03:08:49.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.956+0000 7f8982ac5640 1 -- start start 2023-12-01T03:08:49.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.957+0000 7f8982ac5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897c06b270 0x7f897c070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.957+0000 7f8982ac5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897c0cdcc0 0x7f897c1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.957+0000 7f8982ac5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c1597d0 0x7f897c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.957+0000 7f8982ac5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c06ef10 con 0x7f897c1597d0 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.957+0000 7f8982ac5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c06f1b0 con 0x7f897c06b270 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.957+0000 7f8982ac5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c06f490 con 0x7f897c0cdcc0 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.957+0000 7f89822c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c1597d0 0x7f897c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.958+0000 7f89822c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c1597d0 0x7f897c15c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58336/0 (socket says 172.21.15.115:58336) 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.958+0000 7f89822c4640 1 -- 172.21.15.115:0/1898417182 learned_addr learned my addr 172.21.15.115:0/1898417182 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.958+0000 7f89822c4640 1 -- 172.21.15.115:0/1898417182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897c0cdcc0 msgr2=0x7f897c1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.958+0000 7f89822c4640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897c0cdcc0 0x7f897c1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.958+0000 7f89822c4640 1 -- 172.21.15.115:0/1898417182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897c06b270 msgr2=0x7f897c070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.958+0000 7f89822c4640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897c06b270 0x7f897c070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:49.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.958+0000 7f89822c4640 1 -- 172.21.15.115:0/1898417182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8974009a70 con 0x7f897c1597d0 2023-12-01T03:08:49.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.958+0000 7f89822c4640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c1597d0 0x7f897c15c120 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f8974009d90 tx=0x7f8974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:49.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.958+0000 7f89727fc640 1 -- 172.21.15.115:0/1898417182 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897402cea0 con 0x7f897c1597d0 2023-12-01T03:08:49.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.958+0000 7f89727fc640 1 -- 172.21.15.115:0/1898417182 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8974040650 con 0x7f897c1597d0 2023-12-01T03:08:49.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.959+0000 7f89727fc640 1 -- 172.21.15.115:0/1898417182 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974040a00 con 0x7f897c1597d0 2023-12-01T03:08:49.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.959+0000 7f8982ac5640 1 -- 172.21.15.115:0/1898417182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c06f770 con 0x7f897c1597d0 2023-12-01T03:08:49.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.959+0000 7f8982ac5640 1 -- 172.21.15.115:0/1898417182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c06fd30 con 0x7f897c1597d0 2023-12-01T03:08:49.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.960+0000 7f8982ac5640 1 -- 172.21.15.115:0/1898417182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c0673a0 con 0x7f897c1597d0 2023-12-01T03:08:49.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.961+0000 7f89727fc640 1 -- 172.21.15.115:0/1898417182 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8974040b60 con 0x7f897c1597d0 2023-12-01T03:08:49.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.962+0000 7f89727fc640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f896006a550 0x7f896006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:49.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.962+0000 7f89727fc640 1 -- 172.21.15.115:0/1898417182 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f89740ab760 con 0x7f897c1597d0 2023-12-01T03:08:49.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.962+0000 7f8981ac3640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f896006a550 0x7f896006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:49.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.963+0000 7f8981ac3640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f896006a550 0x7f896006ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8978006fd0 tx=0x7f8978006bc0 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:49.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:49.966+0000 7f89727fc640 1 -- 172.21.15.115:0/1898417182 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8974076cb0 con 0x7f897c1597d0 2023-12-01T03:08:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.197+0000 7f8982ac5640 1 -- 172.21.15.115:0/1898417182 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f897c074260 con 0x7f897c1597d0 2023-12-01T03:08:50.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.198+0000 7f89727fc640 1 -- 172.21.15.115:0/1898417182 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8974009450 con 0x7f897c1597d0 2023-12-01T03:08:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T03:08:50.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 -- 172.21.15.115:0/1898417182 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f896006a550 msgr2=0x7f896006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:50.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f896006a550 0x7f896006ca10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8978006fd0 tx=0x7f8978006bc0 comp rx=0 tx=0).stop 2023-12-01T03:08:50.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 -- 172.21.15.115:0/1898417182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c1597d0 msgr2=0x7f897c15c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:50.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c1597d0 0x7f897c15c120 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f8974009d90 tx=0x7f8974004a40 comp rx=0 tx=0).stop 2023-12-01T03:08:50.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 -- 172.21.15.115:0/1898417182 shutdown_connections 2023-12-01T03:08:50.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f896006a550 0x7f896006ca10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:50.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f897c0cdcc0 0x7f897c1586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:50.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f897c1597d0 0x7f897c15c120 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:50.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 --2- 172.21.15.115:0/1898417182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f897c06b270 0x7f897c070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:50.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 -- 172.21.15.115:0/1898417182 >> 172.21.15.115:0/1898417182 conn(0x7f897c0ce640 msgr2=0x7f897c067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:50.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 -- 172.21.15.115:0/1898417182 shutdown_connections 2023-12-01T03:08:50.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:50.201+0000 7f8982ac5640 1 -- 172.21.15.115:0/1898417182 wait complete. 2023-12-01T03:08:50.220 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T03:08:50.220 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T03:08:50.220 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T03:08:50.221 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T03:08:50.221 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T03:08:50.221 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T03:08:50.221 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T03:08:50.221 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T03:08:50.221 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:08:50.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:50.221 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T03:08:50.270 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T03:08:50.271 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 36827sec preferred_lft 36827sec 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T03:08:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:08:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T03:08:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:08:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T03:08:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T03:08:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T03:08:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:08:50.273 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T03:08:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T03:08:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T03:08:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:08:50.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:50.275 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T03:08:50.357 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T03:08:50.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:50.361 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T03:08:50.415 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T03:08:50.416 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T03:08:50.416 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T03:08:50.416 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T03:08:50.416 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T03:08:50.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:50.416 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T03:08:50.416 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T03:08:50.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T03:08:50.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T03:08:50.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T03:08:50.416 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T03:08:50.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:50.593 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T03:08:50.643 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T03:08:50.643 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T03:08:50.643 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:08:50.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:50.644 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T03:08:50.725 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T03:08:50.726 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T03:08:50.778 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:08:50.778 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T03:08:50.778 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T03:08:50.829 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T03:08:50.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:50.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:08:50.964 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:08:50.964 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:08:50.964 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:08:50.987 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:08:50.987 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:08:50.987 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:08:50.987 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T03:08:50.987 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T03:08:51.011 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:08:51.012 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:08:51.012 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:08:51.012 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T03:08:51.012 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T03:08:51.012 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T03:08:51.012 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T03:08:51.012 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T03:08:51.014 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T03:08:51.014 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T03:08:51.014 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T03:08:51.014 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T03:08:51.014 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T03:08:51.018 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T03:08:51.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:51.019 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T03:08:51.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:51.101 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T03:08:51.172 INFO:teuthology.orchestra.run.smithi115.stdout:client6413 2023-12-01T03:08:51.172 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:08:51.172 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:08:51.227 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T03:08:51.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.443+0000 7ff9a5989640 1 Processor -- start 2023-12-01T03:08:51.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.444+0000 7ff9a5989640 1 -- start start 2023-12-01T03:08:51.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.445+0000 7ff9a5989640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff9a00aded0 0x7ff9a00cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.445+0000 7ff9a5989640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a006f2f0 con 0x7ff9a00ccb60 2023-12-01T03:08:51.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.445+0000 7ff9a5989640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a006f5d0 con 0x7ff9a00c8ed0 2023-12-01T03:08:51.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.445+0000 7ff9a5989640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a00c9350 con 0x7ff9a00aded0 2023-12-01T03:08:51.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.446+0000 7ff99f7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff9a00ccb60 0x7ff9a0163f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:51544/0 (socket says 172.21.15.115:51544) 2023-12-01T03:08:51.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.446+0000 7ff99f7fe640 1 -- 172.21.15.115:0/2020567120 learned_addr learned my addr 172.21.15.115:0/2020567120 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:51.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.446+0000 7ff9967fc640 1 --2- 172.21.15.115:0/2020567120 >> v2:172.21.15.142:3300/0 conn(0x7ff9a00aded0 0x7ff9a00cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.446+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727077955 0 0) 0x7ff9a006f2f0 con 0x7ff9a00ccb60 2023-12-01T03:08:51.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff98c005730 con 0x7ff9a00ccb60 2023-12-01T03:08:51.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4136263763 0 0) 0x7ff98c005730 con 0x7ff9a00ccb60 2023-12-01T03:08:51.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 >> v2:172.21.15.142:3300/0 conn(0x7ff9a00aded0 msgr2=0x7ff9a00cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:51.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 --2- 172.21.15.115:0/2020567120 >> v2:172.21.15.142:3300/0 conn(0x7ff9a00aded0 0x7ff9a00cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 >> v1:172.21.15.142:6789/0 conn(0x7ff9a00c8ed0 legacy=0x7ff9a00cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a00c9630 con 0x7ff9a00ccb60 2023-12-01T03:08:51.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7ff9980019f0 con 0x7ff9a00ccb60 2023-12-01T03:08:51.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 >> v1:172.21.15.115:6789/0 conn(0x7ff9a00ccb60 legacy=0x7ff9a0163f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c005c40 0x7ff98c006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98c0069d0 0x7ff98c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff98c0074b0 0x7ff98c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00c9350 con 0x7ff98c0074b0 2023-12-01T03:08:51.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c00e380 con 0x7ff98c005c40 2023-12-01T03:08:51.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c00e540 con 0x7ff98c0069d0 2023-12-01T03:08:51.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.447+0000 7ff99f7fe640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff98c0074b0 0x7ff98c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.448+0000 7ff99effd640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c005c40 0x7ff98c006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.448+0000 7ff9967fc640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98c0069d0 0x7ff98c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.448+0000 7ff99f7fe640 1 -- 172.21.15.115:0/2020567120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98c0069d0 msgr2=0x7ff98c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:51.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.448+0000 7ff99f7fe640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98c0069d0 0x7ff98c006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.448+0000 7ff99f7fe640 1 -- 172.21.15.115:0/2020567120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c005c40 msgr2=0x7ff98c006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:51.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.448+0000 7ff99f7fe640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff98c005c40 0x7ff98c006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.448+0000 7ff99f7fe640 1 -- 172.21.15.115:0/2020567120 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c005730 con 0x7ff98c0074b0 2023-12-01T03:08:51.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.448+0000 7ff9967fc640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98c0069d0 0x7ff98c006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:51.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.449+0000 7ff99f7fe640 1 -- 172.21.15.115:0/2020567120 reap_dead start 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.449+0000 7ff99f7fe640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff98c0074b0 0x7ff98c00db90 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7ff99800a760 tx=0x7ff98c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915afc8693883305 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.449+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998034630 con 0x7ff98c0074b0 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.449+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99802f070 con 0x7ff98c0074b0 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.449+0000 7ff99e7fc640 1 -- 172.21.15.115:0/2020567120 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998034a10 con 0x7ff98c0074b0 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.450+0000 7ff9a5989640 1 -- 172.21.15.115:0/2020567120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff98c0074b0 msgr2=0x7ff98c00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.450+0000 7ff9a5989640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff98c0074b0 0x7ff98c00db90 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7ff99800a760 tx=0x7ff98c007a00 comp rx=0 tx=0).stop 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.450+0000 7ff9a5989640 1 -- 172.21.15.115:0/2020567120 shutdown_connections 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.450+0000 7ff9a5989640 1 --2- 172.21.15.115:0/2020567120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff98c0074b0 0x7ff98c00db90 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.450+0000 7ff9a5989640 1 -- 172.21.15.115:0/2020567120 >> 172.21.15.115:0/2020567120 conn(0x7ff9a00ce880 msgr2=0x7ff9a00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.450+0000 7ff9a5989640 1 -- 172.21.15.115:0/2020567120 shutdown_connections 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.450+0000 7ff9a5989640 1 -- 172.21.15.115:0/2020567120 wait complete. 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.452+0000 7ff9a5989640 1 Processor -- start 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.452+0000 7ff9a5989640 1 -- start start 2023-12-01T03:08:51.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff9a5989640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98c0074b0 0x7ff9a00b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff9a5989640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a01f60d0 0x7ff9a006ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff9a5989640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a014e880 0x7ff9a014ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff9a5989640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00b3760 con 0x7ff9a014e880 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff9a5989640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a014f220 con 0x7ff9a01f60d0 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff9a5989640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00b2b50 con 0x7ff98c0074b0 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff99f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a014e880 0x7ff9a014ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff99f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a014e880 0x7ff9a014ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58360/0 (socket says 172.21.15.115:58360) 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff99f7fe640 1 -- 172.21.15.115:0/1997076701 learned_addr learned my addr 172.21.15.115:0/1997076701 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff99effd640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98c0074b0 0x7ff9a00b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.453+0000 7ff9967fc640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a01f60d0 0x7ff9a006ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff99f7fe640 1 -- 172.21.15.115:0/1997076701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98c0074b0 msgr2=0x7ff9a00b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff99f7fe640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98c0074b0 0x7ff9a00b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff99f7fe640 1 -- 172.21.15.115:0/1997076701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a01f60d0 msgr2=0x7ff9a006ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff99f7fe640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a01f60d0 0x7ff9a006ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff99f7fe640 1 -- 172.21.15.115:0/1997076701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a00b2e30 con 0x7ff9a014e880 2023-12-01T03:08:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff99f7fe640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a014e880 0x7ff9a014ece0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7ff998002230 tx=0x7ff998001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff99cff9640 1 -- 172.21.15.115:0/1997076701 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980017f0 con 0x7ff9a014e880 2023-12-01T03:08:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff99cff9640 1 -- 172.21.15.115:0/1997076701 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff998001950 con 0x7ff9a014e880 2023-12-01T03:08:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff99cff9640 1 -- 172.21.15.115:0/1997076701 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998004d80 con 0x7ff9a014e880 2023-12-01T03:08:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff9a5989640 1 -- 172.21.15.115:0/1997076701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a00b3110 con 0x7ff9a014e880 2023-12-01T03:08:51.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.454+0000 7ff9a5989640 1 -- 172.21.15.115:0/1997076701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a01f6550 con 0x7ff9a014e880 2023-12-01T03:08:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.456+0000 7ff9a5989640 1 -- 172.21.15.115:0/1997076701 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a00c9ed0 con 0x7ff9a014e880 2023-12-01T03:08:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.457+0000 7ff99cff9640 1 -- 172.21.15.115:0/1997076701 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff998031030 con 0x7ff9a014e880 2023-12-01T03:08:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.457+0000 7ff99cff9640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff98006a550 0x7ff98006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.458+0000 7ff99cff9640 1 -- 172.21.15.115:0/1997076701 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7ff9980a9fe0 con 0x7ff9a014e880 2023-12-01T03:08:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.458+0000 7ff99effd640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff98006a550 0x7ff98006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.458+0000 7ff99effd640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff98006a550 0x7ff98006ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff990005410 tx=0x7ff990009040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:51.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.462+0000 7ff99cff9640 1 -- 172.21.15.115:0/1997076701 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff998075380 con 0x7ff9a014e880 2023-12-01T03:08:51.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.676+0000 7ff9a5989640 1 -- 172.21.15.115:0/1997076701 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9a006d590 con 0x7ff98006a550 2023-12-01T03:08:51.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.679+0000 7ff99cff9640 1 -- 172.21.15.115:0/1997076701 <== mgr.9903 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff9900050f0 con 0x7ff98006a550 2023-12-01T03:08:51.681 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T03:08:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:08:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T03:08:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:08:51.681 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T03:08:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 -- 172.21.15.115:0/1997076701 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff98006a550 msgr2=0x7ff98006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff98006a550 0x7ff98006ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff990005410 tx=0x7ff990009040 comp rx=0 tx=0).stop 2023-12-01T03:08:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 -- 172.21.15.115:0/1997076701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a014e880 msgr2=0x7ff9a014ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a014e880 0x7ff9a014ece0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7ff998002230 tx=0x7ff998001cb0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 -- 172.21.15.115:0/1997076701 shutdown_connections 2023-12-01T03:08:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff98006a550 0x7ff98006ca10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a014e880 0x7ff9a014ece0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9a01f60d0 0x7ff9a006ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 --2- 172.21.15.115:0/1997076701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff98c0074b0 0x7ff9a00b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 -- 172.21.15.115:0/1997076701 >> 172.21.15.115:0/1997076701 conn(0x7ff9a00ce880 msgr2=0x7ff9a00af480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 -- 172.21.15.115:0/1997076701 shutdown_connections 2023-12-01T03:08:51.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.682+0000 7ff9a5989640 1 -- 172.21.15.115:0/1997076701 wait complete. 2023-12-01T03:08:51.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T03:08:51.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.915+0000 7f4c296d8640 1 Processor -- start 2023-12-01T03:08:51.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.915+0000 7f4c296d8640 1 -- start start 2023-12-01T03:08:51.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.917+0000 7f4c296d8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4c24160200 0x7f4c240b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.917+0000 7f4c296d8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c240cbd30 0x7f4c240cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.917+0000 7f4c296d8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c240cc6e0 con 0x7f4c240b0f80 2023-12-01T03:08:51.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.917+0000 7f4c296d8640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c240b35f0 con 0x7f4c24160200 2023-12-01T03:08:51.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.917+0000 7f4c296d8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c240cb290 con 0x7f4c240cbd30 2023-12-01T03:08:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.918+0000 7f4c22ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4c24160200 0x7f4c240b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.918+0000 7f4c237fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c240cbd30 0x7f4c240cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.918+0000 7f4c22ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4c24160200 0x7f4c240b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:46734/0 (socket says 172.21.15.115:46734) 2023-12-01T03:08:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.918+0000 7f4c237fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c240cbd30 0x7f4c240cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:36378/0 (socket says 172.21.15.115:36378) 2023-12-01T03:08:51.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.918+0000 7f4c22ffd640 1 -- 172.21.15.115:0/3559107711 learned_addr learned my addr 172.21.15.115:0/3559107711 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.919+0000 7f4c237fe640 1 -- 172.21.15.115:0/3559107711 >> v2:172.21.15.142:3300/0 conn(0x7f4c24160200 msgr2=0x7f4c240b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.919+0000 7f4c237fe640 1 --2- 172.21.15.115:0/3559107711 >> v2:172.21.15.142:3300/0 conn(0x7f4c24160200 0x7f4c240b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.919+0000 7f4c237fe640 1 -- 172.21.15.115:0/3559107711 >> v1:172.21.15.142:6789/0 conn(0x7f4c240b0f80 legacy=0x7f4c2406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.919+0000 7f4c237fe640 1 -- 172.21.15.115:0/3559107711 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240b0ae0 con 0x7f4c240cbd30 2023-12-01T03:08:51.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.919+0000 7f4c22ffd640 1 --2- 172.21.15.115:0/3559107711 >> v2:172.21.15.142:3300/0 conn(0x7f4c24160200 0x7f4c240b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:08:51.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.921+0000 7f4c237fe640 1 --2- 172.21.15.115:0/3559107711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c240cbd30 0x7f4c240cc1a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009d90 tx=0x7f4c1c02beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31722d63383b7da2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.921+0000 7f4c21ffb640 1 -- 172.21.15.115:0/3559107711 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c02cc60 con 0x7f4c240cbd30 2023-12-01T03:08:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.921+0000 7f4c21ffb640 1 -- 172.21.15.115:0/3559107711 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c02cdc0 con 0x7f4c240cbd30 2023-12-01T03:08:51.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.921+0000 7f4c21ffb640 1 -- 172.21.15.115:0/3559107711 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c034660 con 0x7f4c240cbd30 2023-12-01T03:08:51.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.922+0000 7f4c296d8640 1 -- 172.21.15.115:0/3559107711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c240cbd30 msgr2=0x7f4c240cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.922+0000 7f4c296d8640 1 --2- 172.21.15.115:0/3559107711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c240cbd30 0x7f4c240cc1a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009d90 tx=0x7f4c1c02beb0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.922+0000 7f4c296d8640 1 -- 172.21.15.115:0/3559107711 shutdown_connections 2023-12-01T03:08:51.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.922+0000 7f4c296d8640 1 --2- 172.21.15.115:0/3559107711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c240cbd30 0x7f4c240cc1a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.922+0000 7f4c296d8640 1 --2- 172.21.15.115:0/3559107711 >> v2:172.21.15.142:3300/0 conn(0x7f4c24160200 0x7f4c240b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.922+0000 7f4c296d8640 1 -- 172.21.15.115:0/3559107711 >> 172.21.15.115:0/3559107711 conn(0x7f4c24065960 msgr2=0x7f4c24065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:51.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.922+0000 7f4c296d8640 1 -- 172.21.15.115:0/3559107711 shutdown_connections 2023-12-01T03:08:51.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.922+0000 7f4c296d8640 1 -- 172.21.15.115:0/3559107711 wait complete. 2023-12-01T03:08:51.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.924+0000 7f4c296d8640 1 Processor -- start 2023-12-01T03:08:51.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.924+0000 7f4c296d8640 1 -- start start 2023-12-01T03:08:51.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.925+0000 7f4c296d8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c240b0f80 0x7f4c241f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.925+0000 7f4c296d8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c240cbd30 0x7f4c241503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.925+0000 7f4c296d8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c24160200 0x7f4c24158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.925+0000 7f4c296d8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2415db90 con 0x7f4c240cbd30 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.925+0000 7f4c296d8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2415d7f0 con 0x7f4c240b0f80 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.925+0000 7f4c296d8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241f2a50 con 0x7f4c24160200 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.925+0000 7f4c227fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c240cbd30 0x7f4c241503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.925+0000 7f4c227fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c240cbd30 0x7f4c241503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58370/0 (socket says 172.21.15.115:58370) 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.925+0000 7f4c227fc640 1 -- 172.21.15.115:0/3451644137 learned_addr learned my addr 172.21.15.115:0/3451644137 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c22ffd640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c240b0f80 0x7f4c241f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c237fe640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c24160200 0x7f4c24158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c227fc640 1 -- 172.21.15.115:0/3451644137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c24160200 msgr2=0x7f4c24158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c227fc640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c24160200 0x7f4c24158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c227fc640 1 -- 172.21.15.115:0/3451644137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c240b0f80 msgr2=0x7f4c241f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c227fc640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c240b0f80 0x7f4c241f2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c227fc640 1 -- 172.21.15.115:0/3451644137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c140094d0 con 0x7f4c240cbd30 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c227fc640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c240cbd30 0x7f4c241503e0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800c450 tx=0x7f4c1800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:51.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c0b7fe640 1 -- 172.21.15.115:0/3451644137 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c180052d0 con 0x7f4c240cbd30 2023-12-01T03:08:51.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c0b7fe640 1 -- 172.21.15.115:0/3451644137 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c18005430 con 0x7f4c240cbd30 2023-12-01T03:08:51.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c296d8640 1 -- 172.21.15.115:0/3451644137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c009a70 con 0x7f4c240cbd30 2023-12-01T03:08:51.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c0b7fe640 1 -- 172.21.15.115:0/3451644137 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18012870 con 0x7f4c240cbd30 2023-12-01T03:08:51.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.926+0000 7f4c296d8640 1 -- 172.21.15.115:0/3451644137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c241f3010 con 0x7f4c240cbd30 2023-12-01T03:08:51.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.928+0000 7f4c296d8640 1 -- 172.21.15.115:0/3451644137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2415dcd0 con 0x7f4c240cbd30 2023-12-01T03:08:51.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.929+0000 7f4c0b7fe640 1 -- 172.21.15.115:0/3451644137 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 114) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4c180129d0 con 0x7f4c240cbd30 2023-12-01T03:08:51.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.929+0000 7f4c0b7fe640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4c1006a550 0x7f4c1006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:51.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.930+0000 7f4c0b7fe640 1 -- 172.21.15.115:0/3451644137 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f4c180886d0 con 0x7f4c240cbd30 2023-12-01T03:08:51.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.930+0000 7f4c22ffd640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4c1006a550 0x7f4c1006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:51.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.930+0000 7f4c22ffd640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4c1006a550 0x7f4c1006ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009b70 tx=0x7f4c14008040 comp rx=0 tx=0).ready entity=mgr.9903 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:51.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:51.934+0000 7f4c0b7fe640 1 -- 172.21.15.115:0/3451644137 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4c2415dcd0 con 0x7f4c240cbd30 2023-12-01T03:08:52.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.169+0000 7f4c296d8640 1 -- 172.21.15.115:0/3451644137 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f4c2415dcd0 con 0x7f4c240cbd30 2023-12-01T03:08:52.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.414+0000 7f4c0b7fe640 1 -- 172.21.15.115:0/3451644137 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 115) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f4c18053eb0 con 0x7f4c240cbd30 2023-12-01T03:08:52.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.415+0000 7f4c0b7fe640 1 -- 172.21.15.115:0/3451644137 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v115) v1 ==== 92+0+0 (secure 0 0 0) 0x7f4c18070650 con 0x7f4c240cbd30 2023-12-01T03:08:52.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 -- 172.21.15.115:0/3451644137 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4c1006a550 msgr2=0x7f4c1006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:52.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4c1006a550 0x7f4c1006ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009b70 tx=0x7f4c14008040 comp rx=0 tx=0).stop 2023-12-01T03:08:52.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 -- 172.21.15.115:0/3451644137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c240cbd30 msgr2=0x7f4c241503e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:52.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c240cbd30 0x7f4c241503e0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800c450 tx=0x7f4c1800c910 comp rx=0 tx=0).stop 2023-12-01T03:08:52.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 -- 172.21.15.115:0/3451644137 shutdown_connections 2023-12-01T03:08:52.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4c1006a550 0x7f4c1006ca10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c24160200 0x7f4c24158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c240cbd30 0x7f4c241503e0 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 --2- 172.21.15.115:0/3451644137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c240b0f80 0x7f4c241f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 -- 172.21.15.115:0/3451644137 >> 172.21.15.115:0/3451644137 conn(0x7f4c24065960 msgr2=0x7f4c24065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:52.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 -- 172.21.15.115:0/3451644137 shutdown_connections 2023-12-01T03:08:52.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.418+0000 7f4c296d8640 1 -- 172.21.15.115:0/3451644137 wait complete. 2023-12-01T03:08:52.440 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T03:08:52.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.632+0000 7ff8897ce640 1 Processor -- start 2023-12-01T03:08:52.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.632+0000 7ff8897ce640 1 -- start start 2023-12-01T03:08:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.633+0000 7ff8897ce640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff88415afe0 0x7ff884070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.633+0000 7ff8897ce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8840660d0 con 0x7ff8840d0420 2023-12-01T03:08:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.633+0000 7ff8897ce640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff884070400 con 0x7ff88406cb40 2023-12-01T03:08:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.633+0000 7ff8897ce640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8840d08b0 con 0x7ff88415afe0 2023-12-01T03:08:52.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.634+0000 7ff882ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff88415afe0 0x7ff884070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:52.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.634+0000 7ff882ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff88415afe0 0x7ff884070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58402/0 (socket says 172.21.15.115:58402) 2023-12-01T03:08:52.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.634+0000 7ff882ffd640 1 -- 172.21.15.115:0/974428706 learned_addr learned my addr 172.21.15.115:0/974428706 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:52.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.634+0000 7ff882ffd640 1 -- 172.21.15.115:0/974428706 >> v1:172.21.15.142:6789/0 conn(0x7ff88406cb40 legacy=0x7ff8840ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:52.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.634+0000 7ff882ffd640 1 -- 172.21.15.115:0/974428706 >> v1:172.21.15.115:6789/0 conn(0x7ff8840d0420 legacy=0x7ff88406c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:52.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.634+0000 7ff882ffd640 1 -- 172.21.15.115:0/974428706 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8840706e0 con 0x7ff88415afe0 2023-12-01T03:08:52.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.634+0000 7ff881ffb640 1 -- 172.21.15.115:0/974428706 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437741885 0 0) 0x7ff8840660d0 con 0x7ff8840d0420 2023-12-01T03:08:52.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.635+0000 7ff882ffd640 1 --2- 172.21.15.115:0/974428706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88415afe0 0x7ff884070ee0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7ff874009870 tx=0x7ff87402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e198ea49f61b0f49 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:52.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.636+0000 7ff881ffb640 1 -- 172.21.15.115:0/974428706 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87402f840 con 0x7ff88415afe0 2023-12-01T03:08:52.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.636+0000 7ff881ffb640 1 -- 172.21.15.115:0/974428706 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87402f9a0 con 0x7ff88415afe0 2023-12-01T03:08:52.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.636+0000 7ff881ffb640 1 -- 172.21.15.115:0/974428706 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87402fd30 con 0x7ff88415afe0 2023-12-01T03:08:52.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.636+0000 7ff8897ce640 1 -- 172.21.15.115:0/974428706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88415afe0 msgr2=0x7ff884070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:52.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.636+0000 7ff8897ce640 1 --2- 172.21.15.115:0/974428706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88415afe0 0x7ff884070ee0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7ff874009870 tx=0x7ff87402ba20 comp rx=0 tx=0).stop 2023-12-01T03:08:52.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.636+0000 7ff8897ce640 1 -- 172.21.15.115:0/974428706 shutdown_connections 2023-12-01T03:08:52.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.636+0000 7ff8897ce640 1 --2- 172.21.15.115:0/974428706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff88415afe0 0x7ff884070ee0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.636+0000 7ff8897ce640 1 -- 172.21.15.115:0/974428706 >> 172.21.15.115:0/974428706 conn(0x7ff8840cf900 msgr2=0x7ff8840cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:52.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.636+0000 7ff8897ce640 1 -- 172.21.15.115:0/974428706 shutdown_connections 2023-12-01T03:08:52.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.636+0000 7ff8897ce640 1 -- 172.21.15.115:0/974428706 wait complete. 2023-12-01T03:08:52.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.638+0000 7ff8897ce640 1 Processor -- start 2023-12-01T03:08:52.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.638+0000 7ff8897ce640 1 -- start start 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.639+0000 7ff8897ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88406cb40 0x7ff88414b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.639+0000 7ff8897ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8840d0420 0x7ff8840b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.639+0000 7ff8897ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88415afe0 0x7ff8840ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.639+0000 7ff8897ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b36d0 con 0x7ff8840d0420 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.639+0000 7ff8897ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b4530 con 0x7ff88406cb40 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.639+0000 7ff8897ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b38b0 con 0x7ff88415afe0 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.639+0000 7ff8827fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8840d0420 0x7ff8840b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.639+0000 7ff8827fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8840d0420 0x7ff8840b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58406/0 (socket says 172.21.15.115:58406) 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.639+0000 7ff8827fc640 1 -- 172.21.15.115:0/3955683492 learned_addr learned my addr 172.21.15.115:0/3955683492 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff882ffd640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88406cb40 0x7ff88414b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff8827fc640 1 -- 172.21.15.115:0/3955683492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88415afe0 msgr2=0x7ff8840ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff8827fc640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88415afe0 0x7ff8840ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff8827fc640 1 -- 172.21.15.115:0/3955683492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88406cb40 msgr2=0x7ff88414b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff8827fc640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88406cb40 0x7ff88414b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff8827fc640 1 -- 172.21.15.115:0/3955683492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740094d0 con 0x7ff8840d0420 2023-12-01T03:08:52.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff8827fc640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8840d0420 0x7ff8840b5000 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c460 tx=0x7ff87800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:52.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff86b7fe640 1 -- 172.21.15.115:0/3955683492 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878005480 con 0x7ff8840d0420 2023-12-01T03:08:52.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff86b7fe640 1 -- 172.21.15.115:0/3955683492 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8780035f0 con 0x7ff8840d0420 2023-12-01T03:08:52.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff86b7fe640 1 -- 172.21.15.115:0/3955683492 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8780039a0 con 0x7ff8840d0420 2023-12-01T03:08:52.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.640+0000 7ff8897ce640 1 -- 172.21.15.115:0/3955683492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8840b3b90 con 0x7ff8840d0420 2023-12-01T03:08:52.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.641+0000 7ff8897ce640 1 -- 172.21.15.115:0/3955683492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8840b4150 con 0x7ff8840d0420 2023-12-01T03:08:52.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.642+0000 7ff8897ce640 1 -- 172.21.15.115:0/3955683492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8840b4670 con 0x7ff8840d0420 2023-12-01T03:08:52.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.643+0000 7ff86b7fe640 1 -- 172.21.15.115:0/3955683492 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 115) v1 ==== 88726+0+0 (secure 0 0 0) 0x7ff878003b00 con 0x7ff8840d0420 2023-12-01T03:08:52.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.644+0000 7ff86b7fe640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff87006a5a0 0x7ff87006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:52.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.644+0000 7ff86b7fe640 1 -- 172.21.15.115:0/3955683492 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7ff878088660 con 0x7ff8840d0420 2023-12-01T03:08:52.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.644+0000 7ff882ffd640 1 -- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff87006a5a0 msgr2=0x7ff87006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:52.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.644+0000 7ff882ffd640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff87006a5a0 0x7ff87006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:52.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.648+0000 7ff86b7fe640 1 -- 172.21.15.115:0/3955683492 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff8840b4670 con 0x7ff8840d0420 2023-12-01T03:08:52.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.839+0000 7ff8897ce640 1 -- 172.21.15.115:0/3955683492 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff8840b4670 con 0x7ff8840d0420 2023-12-01T03:08:52.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.845+0000 7ff882ffd640 1 -- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff87006a5a0 msgr2=0x7ff87006ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:52.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.845+0000 7ff882ffd640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff87006a5a0 0x7ff87006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:08:52.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.845+0000 7ff86b7fe640 1 -- 172.21.15.115:0/3955683492 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff8840b4670 con 0x7ff8840d0420 2023-12-01T03:08:52.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 -- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff87006a5a0 msgr2=0x7ff87006ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:52.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff87006a5a0 0x7ff87006ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 -- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8840d0420 msgr2=0x7ff8840b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:52.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8840d0420 0x7ff8840b5000 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c460 tx=0x7ff87800c920 comp rx=0 tx=0).stop 2023-12-01T03:08:52.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 -- 172.21.15.115:0/3955683492 shutdown_connections 2023-12-01T03:08:52.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff87006a5a0 0x7ff87006ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff88415afe0 0x7ff8840ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8840d0420 0x7ff8840b5000 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 --2- 172.21.15.115:0/3955683492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff88406cb40 0x7ff88414b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:52.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 -- 172.21.15.115:0/3955683492 >> 172.21.15.115:0/3955683492 conn(0x7ff8840cf900 msgr2=0x7ff8840b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:52.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 -- 172.21.15.115:0/3955683492 shutdown_connections 2023-12-01T03:08:52.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:52.848+0000 7ff8897ce640 1 -- 172.21.15.115:0/3955683492 wait complete. 2023-12-01T03:08:52.868 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T03:08:52.929 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:52.928+0000 7f0e36c10040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:08:52.967 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:52.965+0000 7fbf3a62b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:08:52.985 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:52.983+0000 7f0e36c10040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:08:53.019 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:53.017+0000 7fbf3a62b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:08:53.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.071+0000 7fa707199640 1 Processor -- start 2023-12-01T03:08:53.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.071+0000 7fa707199640 1 -- start start 2023-12-01T03:08:53.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.072+0000 7fa707199640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa70015cd90 0x7fa7000cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.072+0000 7fa707199640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 0x7fa700161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.072+0000 7fa707199640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7000ce5e0 con 0x7fa7000cf650 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.072+0000 7fa707199640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7000d0ec0 con 0x7fa70015cd90 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.072+0000 7fa707199640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa700161950 con 0x7fa7000b5fa0 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.073+0000 7fa704f35640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa70015cd90 0x7fa7000cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.073+0000 7fa704f35640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa70015cd90 0x7fa7000cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58414/0 (socket says 172.21.15.115:58414) 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.073+0000 7fa704f35640 1 -- 172.21.15.115:0/3322869200 learned_addr learned my addr 172.21.15.115:0/3322869200 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.073+0000 7fa6fffff640 1 --2- 172.21.15.115:0/3322869200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 0x7fa700161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.073+0000 7fa704f35640 1 -- 172.21.15.115:0/3322869200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 msgr2=0x7fa700161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.073+0000 7fa704f35640 1 --2- 172.21.15.115:0/3322869200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 0x7fa700161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.073+0000 7fa704f35640 1 -- 172.21.15.115:0/3322869200 >> v1:172.21.15.142:6789/0 conn(0x7fa7000cf650 legacy=0x7fa7000b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:53.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.073+0000 7fa704f35640 1 -- 172.21.15.115:0/3322869200 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7000ce300 con 0x7fa70015cd90 2023-12-01T03:08:53.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.073+0000 7fa6fffff640 1 --2- 172.21.15.115:0/3322869200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 0x7fa700161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:08:53.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.074+0000 7fa704f35640 1 --2- 172.21.15.115:0/3322869200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa70015cd90 0x7fa7000cdd20 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009870 tx=0x7fa6f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6881db26aba960 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.075+0000 7fa6ff7fe640 1 -- 172.21.15.115:0/3322869200 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4031640 con 0x7fa70015cd90 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.075+0000 7fa6ff7fe640 1 -- 172.21.15.115:0/3322869200 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40317a0 con 0x7fa70015cd90 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.075+0000 7fa6ff7fe640 1 -- 172.21.15.115:0/3322869200 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4031b30 con 0x7fa70015cd90 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.075+0000 7fa707199640 1 -- 172.21.15.115:0/3322869200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa70015cd90 msgr2=0x7fa7000cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.075+0000 7fa707199640 1 --2- 172.21.15.115:0/3322869200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa70015cd90 0x7fa7000cdd20 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009870 tx=0x7fa6f402b8f0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.075+0000 7fa707199640 1 -- 172.21.15.115:0/3322869200 shutdown_connections 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.076+0000 7fa707199640 1 --2- 172.21.15.115:0/3322869200 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 0x7fa700161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.076+0000 7fa707199640 1 --2- 172.21.15.115:0/3322869200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa70015cd90 0x7fa7000cdd20 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.076+0000 7fa707199640 1 -- 172.21.15.115:0/3322869200 >> 172.21.15.115:0/3322869200 conn(0x7fa700065960 msgr2=0x7fa700065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.076+0000 7fa707199640 1 -- 172.21.15.115:0/3322869200 shutdown_connections 2023-12-01T03:08:53.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.076+0000 7fa707199640 1 -- 172.21.15.115:0/3322869200 wait complete. 2023-12-01T03:08:53.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.077+0000 7fa707199640 1 Processor -- start 2023-12-01T03:08:53.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.077+0000 7fa707199640 1 -- start start 2023-12-01T03:08:53.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.078+0000 7fa707199640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 0x7fa7001efe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.078+0000 7fa707199640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7000cf650 0x7fa7000a9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.078+0000 7fa707199640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa70015cd90 0x7fa7000abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.078+0000 7fa707199640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000aab20 con 0x7fa7000cf650 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.078+0000 7fa707199640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000a9f10 con 0x7fa70015cd90 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.078+0000 7fa707199640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000aa1f0 con 0x7fa7000b5fa0 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.078+0000 7fa6fffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7000cf650 0x7fa7000a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa6fffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7000cf650 0x7fa7000a9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58422/0 (socket says 172.21.15.115:58422) 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa6fffff640 1 -- 172.21.15.115:0/833797354 learned_addr learned my addr 172.21.15.115:0/833797354 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa704f35640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 0x7fa7001efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa705736640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa70015cd90 0x7fa7000abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:53.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa6fffff640 1 -- 172.21.15.115:0/833797354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 msgr2=0x7fa7001efe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa6fffff640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 0x7fa7001efe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa6fffff640 1 -- 172.21.15.115:0/833797354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa70015cd90 msgr2=0x7fa7000abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa6fffff640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa70015cd90 0x7fa7000abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa6fffff640 1 -- 172.21.15.115:0/833797354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f40094d0 con 0x7fa7000cf650 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa705736640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa70015cd90 0x7fa7000abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa6fffff640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7000cf650 0x7fa7000a9690 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00e7e0 tx=0x7fa6ec00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.079+0000 7fa6fd7fa640 1 -- 172.21.15.115:0/833797354 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec004030 con 0x7fa7000cf650 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.080+0000 7fa6fd7fa640 1 -- 172.21.15.115:0/833797354 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec004190 con 0x7fa7000cf650 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.080+0000 7fa707199640 1 -- 172.21.15.115:0/833797354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7000aa4d0 con 0x7fa7000cf650 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.080+0000 7fa6fd7fa640 1 -- 172.21.15.115:0/833797354 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec004520 con 0x7fa7000cf650 2023-12-01T03:08:53.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.080+0000 7fa707199640 1 -- 172.21.15.115:0/833797354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7000a9bd0 con 0x7fa7000cf650 2023-12-01T03:08:53.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.081+0000 7fa707199640 1 -- 172.21.15.115:0/833797354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7000aac90 con 0x7fa7000cf650 2023-12-01T03:08:53.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.082+0000 7fa6fd7fa640 1 -- 172.21.15.115:0/833797354 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 115) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fa6ec00c6d0 con 0x7fa7000cf650 2023-12-01T03:08:53.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.083+0000 7fa6fd7fa640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6e406a5a0 0x7fa6e406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.083+0000 7fa6fd7fa640 1 -- 172.21.15.115:0/833797354 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fa6ec088970 con 0x7fa7000cf650 2023-12-01T03:08:53.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.083+0000 7fa704f35640 1 -- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6e406a5a0 msgr2=0x7fa6e406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:53.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.083+0000 7fa704f35640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6e406a5a0 0x7fa6e406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:53.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.087+0000 7fa6fd7fa640 1 -- 172.21.15.115:0/833797354 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa6ec053c70 con 0x7fa7000cf650 2023-12-01T03:08:53.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.275+0000 7fa707199640 1 -- 172.21.15.115:0/833797354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fa70004e890 con 0x7fa7000cf650 2023-12-01T03:08:53.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.283+0000 7fa6fd7fa640 1 -- 172.21.15.115:0/833797354 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa6ec0200c0 con 0x7fa7000cf650 2023-12-01T03:08:53.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.284+0000 7fa704f35640 1 -- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6e406a5a0 msgr2=0x7fa6e406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:53.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.284+0000 7fa704f35640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6e406a5a0 0x7fa6e406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:08:53.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 -- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6e406a5a0 msgr2=0x7fa6e406ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:53.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6e406a5a0 0x7fa6e406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 -- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7000cf650 msgr2=0x7fa7000a9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:53.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7000cf650 0x7fa7000a9690 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00e7e0 tx=0x7fa6ec00eca0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 -- 172.21.15.115:0/833797354 shutdown_connections 2023-12-01T03:08:53.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6e406a5a0 0x7fa6e406ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa70015cd90 0x7fa7000abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7000cf650 0x7fa7000a9690 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 --2- 172.21.15.115:0/833797354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7000b5fa0 0x7fa7001efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 -- 172.21.15.115:0/833797354 >> 172.21.15.115:0/833797354 conn(0x7fa700065960 msgr2=0x7fa7000cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:53.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 -- 172.21.15.115:0/833797354 shutdown_connections 2023-12-01T03:08:53.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.285+0000 7fa707199640 1 -- 172.21.15.115:0/833797354 wait complete. 2023-12-01T03:08:53.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:53.307 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-01T03:08:53.360 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-01T03:08:53.456 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:53.454+0000 7f0e36c10040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:08:53.464 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:53.462+0000 7fbf3a62b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:08:53.510 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:53.508+0000 7f0e36c10040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:08:53.516 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:53.514+0000 7fbf3a62b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:08:53.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.551+0000 7fec2bd89640 1 Processor -- start 2023-12-01T03:08:53.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.551+0000 7fec2bd89640 1 -- start start 2023-12-01T03:08:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.553+0000 7fec2bd89640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec2415cd90 0x7fec240cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.553+0000 7fec2bd89640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fec240cfd60 0x7fec24161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.553+0000 7fec2bd89640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec240ce5e0 con 0x7fec240b5fa0 2023-12-01T03:08:53.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.553+0000 7fec2bd89640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240d0ec0 con 0x7fec240cfd60 2023-12-01T03:08:53.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.553+0000 7fec2bd89640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec24161950 con 0x7fec2415cd90 2023-12-01T03:08:53.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.553+0000 7fec2a326640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fec240cfd60 0x7fec24161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:53.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.553+0000 7fec29b25640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec2415cd90 0x7fec240cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:53.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.553+0000 7fec2a326640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fec240cfd60 0x7fec24161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:46784/0 (socket says 172.21.15.115:46784) 2023-12-01T03:08:53.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.553+0000 7fec2a326640 1 -- 172.21.15.115:0/3929979949 learned_addr learned my addr 172.21.15.115:0/3929979949 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:53.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.554+0000 7fec2a326640 1 -- 172.21.15.115:0/3929979949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec2415cd90 msgr2=0x7fec240cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:53.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.554+0000 7fec2a326640 1 --2- 172.21.15.115:0/3929979949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec2415cd90 0x7fec240cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.554+0000 7fec2a326640 1 -- 172.21.15.115:0/3929979949 >> v1:172.21.15.142:6789/0 conn(0x7fec240b5fa0 legacy=0x7fec240cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:53.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.554+0000 7fec2a326640 1 -- 172.21.15.115:0/3929979949 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240ce300 con 0x7fec240cfd60 2023-12-01T03:08:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.555+0000 7fec2a326640 1 --2- 172.21.15.115:0/3929979949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec240cfd60 0x7fec24161410 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fec1c009e50 tx=0x7fec1c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd3af0929c2836e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.556+0000 7fec28b23640 1 -- 172.21.15.115:0/3929979949 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c02cb00 con 0x7fec240cfd60 2023-12-01T03:08:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.556+0000 7fec28b23640 1 -- 172.21.15.115:0/3929979949 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c02cc60 con 0x7fec240cfd60 2023-12-01T03:08:53.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.556+0000 7fec28b23640 1 -- 172.21.15.115:0/3929979949 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c0343f0 con 0x7fec240cfd60 2023-12-01T03:08:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.556+0000 7fec2bd89640 1 -- 172.21.15.115:0/3929979949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec240cfd60 msgr2=0x7fec24161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.556+0000 7fec2bd89640 1 --2- 172.21.15.115:0/3929979949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec240cfd60 0x7fec24161410 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fec1c009e50 tx=0x7fec1c02bf60 comp rx=0 tx=0).stop 2023-12-01T03:08:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.556+0000 7fec2bd89640 1 -- 172.21.15.115:0/3929979949 shutdown_connections 2023-12-01T03:08:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.556+0000 7fec2bd89640 1 --2- 172.21.15.115:0/3929979949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec240cfd60 0x7fec24161410 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.556+0000 7fec2bd89640 1 --2- 172.21.15.115:0/3929979949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec2415cd90 0x7fec240cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.556+0000 7fec2bd89640 1 -- 172.21.15.115:0/3929979949 >> 172.21.15.115:0/3929979949 conn(0x7fec24065960 msgr2=0x7fec24065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:53.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.557+0000 7fec2bd89640 1 -- 172.21.15.115:0/3929979949 shutdown_connections 2023-12-01T03:08:53.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.557+0000 7fec2bd89640 1 -- 172.21.15.115:0/3929979949 wait complete. 2023-12-01T03:08:53.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.558+0000 7fec2bd89640 1 Processor -- start 2023-12-01T03:08:53.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.558+0000 7fec2bd89640 1 -- start start 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.559+0000 7fec2bd89640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec240b5fa0 0x7fec24144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.559+0000 7fec2bd89640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec240cfd60 0x7fec2415da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.559+0000 7fec2bd89640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec2415cd90 0x7fec241467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.559+0000 7fec2bd89640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec24145460 con 0x7fec240b5fa0 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.559+0000 7fec2bd89640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec24145740 con 0x7fec240cfd60 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.559+0000 7fec2bd89640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec24145a20 con 0x7fec2415cd90 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.560+0000 7fec29b25640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec240b5fa0 0x7fec24144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.560+0000 7fec29b25640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec240b5fa0 0x7fec24144d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:58438/0 (socket says 172.21.15.115:58438) 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.560+0000 7fec29b25640 1 -- 172.21.15.115:0/259288974 learned_addr learned my addr 172.21.15.115:0/259288974 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.560+0000 7fec29b25640 1 -- 172.21.15.115:0/259288974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec2415cd90 msgr2=0x7fec241467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.560+0000 7fec29b25640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec2415cd90 0x7fec241467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.560+0000 7fec29b25640 1 -- 172.21.15.115:0/259288974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec240cfd60 msgr2=0x7fec2415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.560+0000 7fec29b25640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec240cfd60 0x7fec2415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.560+0000 7fec29b25640 1 -- 172.21.15.115:0/259288974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c009b30 con 0x7fec240b5fa0 2023-12-01T03:08:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.560+0000 7fec29b25640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec240b5fa0 0x7fec24144d30 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fec2000e9b0 tx=0x7fec2000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.561+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20009670 con 0x7fec240b5fa0 2023-12-01T03:08:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.561+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2000c9a0 con 0x7fec240b5fa0 2023-12-01T03:08:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.561+0000 7fec2bd89640 1 -- 172.21.15.115:0/259288974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec24145d00 con 0x7fec240b5fa0 2023-12-01T03:08:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.561+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2000cd50 con 0x7fec240b5fa0 2023-12-01T03:08:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.561+0000 7fec2bd89640 1 -- 172.21.15.115:0/259288974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec241462c0 con 0x7fec240b5fa0 2023-12-01T03:08:53.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.562+0000 7fec2bd89640 1 -- 172.21.15.115:0/259288974 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2404e890 con 0x7fec240b5fa0 2023-12-01T03:08:53.565 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:53.562+0000 7f0e36c10040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:08:53.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.563+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 115) v1 ==== 88726+0+0 (secure 0 0 0) 0x7fec20013470 con 0x7fec240b5fa0 2023-12-01T03:08:53.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.564+0000 7fec1a7fc640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:53.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.564+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fec20088df0 con 0x7fec240b5fa0 2023-12-01T03:08:53.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.564+0000 7fec29324640 1 -- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 msgr2=0x7fec0806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:53.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.564+0000 7fec29324640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:08:53.567 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:53.565+0000 7fbf3a62b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:08:53.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.568+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fec200542a0 con 0x7fec240b5fa0 2023-12-01T03:08:53.627 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:53.626+0000 7f0e36c10040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:08:53.629 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:53.627+0000 7fbf3a62b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:08:53.677 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:53.676+0000 7f0e36c10040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:08:53.678 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:53.676+0000 7fbf3a62b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:08:53.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.765+0000 7fec29324640 1 -- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 msgr2=0x7fec0806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:53.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.765+0000 7fec29324640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:08:53.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:53.781+0000 7fec2bd89640 1 -- 172.21.15.115:0/259288974 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fec240d1380 con 0x7fec0806a5a0 2023-12-01T03:08:53.788 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:53.786+0000 7fbf3a62b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:08:53.789 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:53.787+0000 7f0e36c10040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:08:53.849 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:53.847+0000 7fbf3a62b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:08:53.851 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:53.849+0000 7f0e36c10040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:08:53.899 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:53.897+0000 7fbf3a62b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:08:53.900 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:53.899+0000 7f0e36c10040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:08:53.956 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:53.953+0000 7fbf3a62b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:08:53.956 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:53.954+0000 7f0e36c10040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:08:54.005 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:54.003+0000 7fbf3a62b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:08:54.006 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:54.004+0000 7f0e36c10040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:08:54.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:54.166+0000 7fec29324640 1 -- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 msgr2=0x7fec0806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:54.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:54.166+0000 7fec29324640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T03:08:54.381 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:54.379+0000 7fbf3a62b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:08:54.383 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:54.382+0000 7f0e36c10040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:08:54.540 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:54.538+0000 7fbf3a62b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:08:54.540 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:54.538+0000 7f0e36c10040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:08:54.646 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:54.644+0000 7fbf3a62b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:08:54.646 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:54.645+0000 7f0e36c10040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:08:54.696 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:54.694+0000 7fbf3a62b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:08:54.697 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:54.696+0000 7f0e36c10040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:08:54.802 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:54.800+0000 7fbf3a62b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:08:54.803 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:54.801+0000 7f0e36c10040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:08:54.909 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:54.905+0000 7fbf3a62b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:08:54.910 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:54.908+0000 7f0e36c10040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:08:54.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:54.967+0000 7fec29324640 1 -- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 msgr2=0x7fec0806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:08:54.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:54.967+0000 7fec29324640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T03:08:55.061 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:55.058+0000 7fbf3a62b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:08:55.064 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:55.062+0000 7f0e36c10040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:08:55.109 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:55.107+0000 7fbf3a62b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:08:55.113 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:55.111+0000 7f0e36c10040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:08:55.226 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:55.224+0000 7fbf3a62b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:08:55.230 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:55.228+0000 7f0e36c10040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:08:56.042 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:56.040+0000 7fbf3a62b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:08:56.050 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.049+0000 7f0e36c10040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:08:56.341 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:56.339+0000 7fbf3a62b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:08:56.346 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.345+0000 7f0e36c10040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:08:56.401 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:56.399+0000 7fbf3a62b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:08:56.402 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.401+0000 7f0e36c10040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:08:56.514 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.514+0000 7f0e36c10040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:08:56.526 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:08:56.524+0000 7fbf3a62b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:08:56.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:56.529+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 116) v1 ==== 88335+0+0 (secure 0 0 0) 0x7fec200872e0 con 0x7fec240b5fa0 2023-12-01T03:08:56.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:56.529+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 msgr2=0x7fec0806ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:08:56.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:56.529+0000 7fec1a7fc640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:56.657 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.656+0000 7f0db6442640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:56.658 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.656+0000 7f0db6442640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:56.658 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.656+0000 7f0db6442640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:56.658 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.656+0000 7f0db6442640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:56.658 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.656+0000 7f0db6442640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:56.658 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.656+0000 7f0db343c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:56.658 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.656+0000 7f0db343c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:56.658 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.657+0000 7f0db343c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:56.658 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.657+0000 7f0db343c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:56.658 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:08:56.657+0000 7f0db343c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:08:57.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.541+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 117) v1 ==== 88462+0+0 (secure 0 0 0) 0x7fec20054670 con 0x7fec240b5fa0 2023-12-01T03:08:57.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.542+0000 7fec1a7fc640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:57.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.542+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fec08027f80 con 0x7fec0806a5a0 2023-12-01T03:08:57.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.549+0000 7fec29324640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:57.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.554+0000 7fec29324640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fec14001030 tx=0x7fec14007a30 comp rx=0 tx=0).ready entity=mgr.10299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:57.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.705+0000 7fec1a7fc640 1 -- 172.21.15.115:0/259288974 <== mgr.10299 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fec08027f80 con 0x7fec0806a5a0 2023-12-01T03:08:57.706 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path snap_test_dir1 2023-12-01T03:08:57.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 -- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 msgr2=0x7fec0806ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:57.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fec14001030 tx=0x7fec14007a30 comp rx=0 tx=0).stop 2023-12-01T03:08:57.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 -- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec240b5fa0 msgr2=0x7fec24144d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:57.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec240b5fa0 0x7fec24144d30 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fec2000e9b0 tx=0x7fec2000ee70 comp rx=0 tx=0).stop 2023-12-01T03:08:57.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 -- 172.21.15.115:0/259288974 shutdown_connections 2023-12-01T03:08:57.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec0806a5a0 0x7fec0806ca60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:57.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec2415cd90 0x7fec241467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:57.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec240cfd60 0x7fec2415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:57.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 --2- 172.21.15.115:0/259288974 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec240b5fa0 0x7fec24144d30 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:57.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 -- 172.21.15.115:0/259288974 >> 172.21.15.115:0/259288974 conn(0x7fec24065960 msgr2=0x7fec24160970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:57.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 -- 172.21.15.115:0/259288974 shutdown_connections 2023-12-01T03:08:57.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.708+0000 7fec2bd89640 1 -- 172.21.15.115:0/259288974 wait complete. 2023-12-01T03:08:57.730 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-01T03:08:57.730 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1M 2023-12-01T03:08:57.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.943+0000 7fa6f0c70640 1 Processor -- start 2023-12-01T03:08:57.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.943+0000 7fa6f0c70640 1 -- start start 2023-12-01T03:08:57.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.944+0000 7fa6f0c70640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa6ec0cb8c0 0x7fa6ec0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:57.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.944+0000 7fa6f0c70640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa6ec0b4070 0x7fa6ec0cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:57.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.944+0000 7fa6f0c70640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ec06c860 con 0x7fa6ec15ac70 2023-12-01T03:08:57.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.944+0000 7fa6f0c70640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec15b050 con 0x7fa6ec0b4070 2023-12-01T03:08:57.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.944+0000 7fa6f0c70640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec0ce0b0 con 0x7fa6ec0cb8c0 2023-12-01T03:08:57.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.945+0000 7fa6ead9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa6ec0b4070 0x7fa6ec0cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:57.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.945+0000 7fa6ead9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa6ec0b4070 0x7fa6ec0cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39856/0 (socket says 172.21.15.115:39856) 2023-12-01T03:08:57.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.945+0000 7fa6ead9d640 1 -- 172.21.15.115:0/3450988837 learned_addr learned my addr 172.21.15.115:0/3450988837 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:57.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.945+0000 7fa6e9d9b640 1 --2- 172.21.15.115:0/3450988837 >> v2:172.21.15.142:3300/0 conn(0x7fa6ec0cb8c0 0x7fa6ec0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:57.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.945+0000 7fa6ead9d640 1 -- 172.21.15.115:0/3450988837 >> v2:172.21.15.142:3300/0 conn(0x7fa6ec0cb8c0 msgr2=0x7fa6ec0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:57.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.945+0000 7fa6ead9d640 1 --2- 172.21.15.115:0/3450988837 >> v2:172.21.15.142:3300/0 conn(0x7fa6ec0cb8c0 0x7fa6ec0cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:57.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.945+0000 7fa6ead9d640 1 -- 172.21.15.115:0/3450988837 >> v1:172.21.15.142:6789/0 conn(0x7fa6ec15ac70 legacy=0x7fa6ec0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:57.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.945+0000 7fa6ead9d640 1 -- 172.21.15.115:0/3450988837 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec0ce390 con 0x7fa6ec0b4070 2023-12-01T03:08:57.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.945+0000 7fa6e9d9b640 1 --2- 172.21.15.115:0/3450988837 >> v2:172.21.15.142:3300/0 conn(0x7fa6ec0cb8c0 0x7fa6ec0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:08:57.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.946+0000 7fa6ead9d640 1 --2- 172.21.15.115:0/3450988837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec0b4070 0x7fa6ec0cc8b0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0cbe60 tx=0x7fa6e402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695cd6c176bcccc1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:57.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.946+0000 7fa6e959a640 1 -- 172.21.15.115:0/3450988837 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40317e0 con 0x7fa6ec0b4070 2023-12-01T03:08:57.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.947+0000 7fa6e959a640 1 -- 172.21.15.115:0/3450988837 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e4031940 con 0x7fa6ec0b4070 2023-12-01T03:08:57.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.947+0000 7fa6e959a640 1 -- 172.21.15.115:0/3450988837 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4031cd0 con 0x7fa6ec0b4070 2023-12-01T03:08:57.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.947+0000 7fa6f0c70640 1 -- 172.21.15.115:0/3450988837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec0b4070 msgr2=0x7fa6ec0cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:57.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.947+0000 7fa6f0c70640 1 --2- 172.21.15.115:0/3450988837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec0b4070 0x7fa6ec0cc8b0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0cbe60 tx=0x7fa6e402b880 comp rx=0 tx=0).stop 2023-12-01T03:08:57.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.947+0000 7fa6f0c70640 1 -- 172.21.15.115:0/3450988837 shutdown_connections 2023-12-01T03:08:57.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.947+0000 7fa6f0c70640 1 --2- 172.21.15.115:0/3450988837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec0b4070 0x7fa6ec0cc8b0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:57.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.947+0000 7fa6f0c70640 1 --2- 172.21.15.115:0/3450988837 >> v2:172.21.15.142:3300/0 conn(0x7fa6ec0cb8c0 0x7fa6ec0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:57.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.947+0000 7fa6f0c70640 1 -- 172.21.15.115:0/3450988837 >> 172.21.15.115:0/3450988837 conn(0x7fa6ec0662c0 msgr2=0x7fa6ec0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:57.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.947+0000 7fa6f0c70640 1 -- 172.21.15.115:0/3450988837 shutdown_connections 2023-12-01T03:08:57.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.947+0000 7fa6f0c70640 1 -- 172.21.15.115:0/3450988837 wait complete. 2023-12-01T03:08:57.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.949+0000 7fa6f0c70640 1 Processor -- start 2023-12-01T03:08:57.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.949+0000 7fa6f0c70640 1 -- start start 2023-12-01T03:08:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.950+0000 7fa6f0c70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6ec0b4070 0x7fa6ec15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.950+0000 7fa6f0c70640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6ec0cb8c0 0x7fa6ec151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.950+0000 7fa6f0c70640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec15ac70 0x7fa6ec15da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.950+0000 7fa6f0c70640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec151f80 con 0x7fa6ec15ac70 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.950+0000 7fa6f0c70640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0ca0e0 con 0x7fa6ec0cb8c0 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.950+0000 7fa6f0c70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec15dfb0 con 0x7fa6ec0b4070 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.950+0000 7fa6ead9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec15ac70 0x7fa6ec15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.950+0000 7fa6ead9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec15ac70 0x7fa6ec15da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39864/0 (socket says 172.21.15.115:39864) 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.950+0000 7fa6ead9d640 1 -- 172.21.15.115:0/1634641931 learned_addr learned my addr 172.21.15.115:0/1634641931 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.951+0000 7fa6ead9d640 1 -- 172.21.15.115:0/1634641931 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6ec0b4070 msgr2=0x7fa6ec15b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.951+0000 7fa6ead9d640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6ec0b4070 0x7fa6ec15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.951+0000 7fa6ead9d640 1 -- 172.21.15.115:0/1634641931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6ec0cb8c0 msgr2=0x7fa6ec151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.951+0000 7fa6e9d9b640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6ec0cb8c0 0x7fa6ec151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.951+0000 7fa6ead9d640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6ec0cb8c0 0x7fa6ec151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.951+0000 7fa6ead9d640 1 -- 172.21.15.115:0/1634641931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e40094d0 con 0x7fa6ec15ac70 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.951+0000 7fa6ead9d640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec15ac70 0x7fa6ec15da70 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40090b0 tx=0x7fa6e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.951+0000 7fa6e2ffd640 1 -- 172.21.15.115:0/1634641931 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40329c0 con 0x7fa6ec15ac70 2023-12-01T03:08:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.951+0000 7fa6e2ffd640 1 -- 172.21.15.115:0/1634641931 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e4032b40 con 0x7fa6ec15ac70 2023-12-01T03:08:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.952+0000 7fa6f0c70640 1 -- 172.21.15.115:0/1634641931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec15bdf0 con 0x7fa6ec15ac70 2023-12-01T03:08:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.952+0000 7fa6e2ffd640 1 -- 172.21.15.115:0/1634641931 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40453f0 con 0x7fa6ec15ac70 2023-12-01T03:08:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.952+0000 7fa6f0c70640 1 -- 172.21.15.115:0/1634641931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec15c3b0 con 0x7fa6ec15ac70 2023-12-01T03:08:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.953+0000 7fa6f0c70640 1 -- 172.21.15.115:0/1634641931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec074210 con 0x7fa6ec15ac70 2023-12-01T03:08:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.954+0000 7fa6e2ffd640 1 -- 172.21.15.115:0/1634641931 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 117) v1 ==== 88462+0+0 (secure 0 0 0) 0x7fa6e404d3f0 con 0x7fa6ec15ac70 2023-12-01T03:08:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.954+0000 7fa6e2ffd640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6cc06a2d0 0x7fa6cc06c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.955+0000 7fa6e2ffd640 1 -- 172.21.15.115:0/1634641931 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fa6e40aa6d0 con 0x7fa6ec15ac70 2023-12-01T03:08:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.955+0000 7fa6ea59c640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6cc06a2d0 0x7fa6cc06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.955+0000 7fa6ea59c640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6cc06a2d0 0x7fa6cc06c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0043c0 tx=0x7fa6dc007d30 comp rx=0 tx=0).ready entity=mgr.10299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:57.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:57.959+0000 7fa6e2ffd640 1 -- 172.21.15.115:0/1634641931 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa6e4076000 con 0x7fa6ec15ac70 2023-12-01T03:08:58.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.172+0000 7fa6f0c70640 1 -- 172.21.15.115:0/1634641931 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6ec06cde0 con 0x7fa6cc06a2d0 2023-12-01T03:08:58.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.285+0000 7fa6e2ffd640 1 -- 172.21.15.115:0/1634641931 <== mgr.10299 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7fa6dc00c760 con 0x7fa6cc06a2d0 2023-12-01T03:08:58.286 INFO:teuthology.orchestra.run.smithi115.stdout:Retention added to path snap_test_dir1 2023-12-01T03:08:58.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.289+0000 7fa6f0c70640 1 -- 172.21.15.115:0/1634641931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6cc06a2d0 msgr2=0x7fa6cc06c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:58.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.289+0000 7fa6f0c70640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6cc06a2d0 0x7fa6cc06c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0043c0 tx=0x7fa6dc007d30 comp rx=0 tx=0).stop 2023-12-01T03:08:58.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.289+0000 7fa6f0c70640 1 -- 172.21.15.115:0/1634641931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec15ac70 msgr2=0x7fa6ec15da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:58.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.289+0000 7fa6f0c70640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec15ac70 0x7fa6ec15da70 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40090b0 tx=0x7fa6e40046c0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.289+0000 7fa6f0c70640 1 -- 172.21.15.115:0/1634641931 shutdown_connections 2023-12-01T03:08:58.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.289+0000 7fa6f0c70640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa6cc06a2d0 0x7fa6cc06c790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.289+0000 7fa6f0c70640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6ec15ac70 0x7fa6ec15da70 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.289+0000 7fa6f0c70640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6ec0cb8c0 0x7fa6ec151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.289+0000 7fa6f0c70640 1 --2- 172.21.15.115:0/1634641931 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa6ec0b4070 0x7fa6ec15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.289+0000 7fa6f0c70640 1 -- 172.21.15.115:0/1634641931 >> 172.21.15.115:0/1634641931 conn(0x7fa6ec0662c0 msgr2=0x7fa6ec0ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:58.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.290+0000 7fa6f0c70640 1 -- 172.21.15.115:0/1634641931 shutdown_connections 2023-12-01T03:08:58.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.290+0000 7fa6f0c70640 1 -- 172.21.15.115:0/1634641931 wait complete. 2023-12-01T03:08:58.312 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-12-01T03:08:58.312 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-01-03_09_00 in ~2s... 2023-12-01T03:08:58.312 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-12-01T03:08:58.313 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-01T03:08:58.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.511+0000 7f7369bc9640 1 Processor -- start 2023-12-01T03:08:58.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.512+0000 7f7369bc9640 1 -- start start 2023-12-01T03:08:58.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.513+0000 7f7369bc9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f736415d0b0 0x7f73640cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:58.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.513+0000 7f7369bc9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73640cd810 0x7f736406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:58.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.513+0000 7f7369bc9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73640b33e0 con 0x7f73640b29d0 2023-12-01T03:08:58.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.513+0000 7f7369bc9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736415d490 con 0x7f736415d0b0 2023-12-01T03:08:58.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.513+0000 7f7369bc9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7364069df0 con 0x7f73640cd810 2023-12-01T03:08:58.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.513+0000 7f7362ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f73640b29d0 0x7f73640cd100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34654/0 (socket says 172.21.15.115:34654) 2023-12-01T03:08:58.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.513+0000 7f7362ffd640 1 -- 172.21.15.115:0/1159852913 learned_addr learned my addr 172.21.15.115:0/1159852913 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:08:58.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73637fe640 1 --2- 172.21.15.115:0/1159852913 >> v2:172.21.15.142:3300/0 conn(0x7f736415d0b0 0x7f73640cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:58.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635244868 0 0) 0x7f73640b33e0 con 0x7f73640b29d0 2023-12-01T03:08:58.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7350003520 con 0x7f73640b29d0 2023-12-01T03:08:58.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f7363fff640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73640cd810 0x7f736406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:58.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 278613204 0 0) 0x7f7350003520 con 0x7f73640b29d0 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73640cd810 msgr2=0x7f736406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73640cd810 0x7f736406a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 >> v2:172.21.15.142:3300/0 conn(0x7f736415d0b0 msgr2=0x7f73640cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 --2- 172.21.15.115:0/1159852913 >> v2:172.21.15.142:3300/0 conn(0x7f736415d0b0 0x7f73640cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73640cce90 con 0x7f73640b29d0 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f734c001670 con 0x7f73640b29d0 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 >> v1:172.21.15.115:6789/0 conn(0x7f73640b29d0 legacy=0x7f73640cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73640cd810 0x7f7350003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7350003f40 0x7f73500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 0x7f7350005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736415d490 con 0x7f7350004d00 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7364069df0 con 0x7f7350003f40 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.514+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735000ba30 con 0x7f73640cd810 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.515+0000 7f7363fff640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 0x7f7350005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:58.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.515+0000 7f73637fe640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73640cd810 0x7f7350003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:58.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.515+0000 7f7362ffd640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7350003f40 0x7f73500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:58.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.515+0000 7f7363fff640 1 -- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73640cd810 msgr2=0x7f7350003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:58.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.515+0000 7f7363fff640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73640cd810 0x7f7350003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.515+0000 7f7363fff640 1 -- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7350003f40 msgr2=0x7f73500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:08:58.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.515+0000 7f7363fff640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7350003f40 0x7f73500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.515+0000 7f7363fff640 1 -- 172.21.15.115:0/1159852913 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7350003520 con 0x7f7350004d00 2023-12-01T03:08:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.516+0000 7f7363fff640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 0x7f7350005180 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f736406e8e0 tx=0x7f73500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c498b38e78092cd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.517+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c0325f0 con 0x7f7350004d00 2023-12-01T03:08:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.517+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c032750 con 0x7f7350004d00 2023-12-01T03:08:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.517+0000 7f73627fc640 1 -- 172.21.15.115:0/1159852913 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c032ae0 con 0x7f7350004d00 2023-12-01T03:08:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.517+0000 7f7369bc9640 1 -- 172.21.15.115:0/1159852913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 msgr2=0x7f7350005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.517+0000 7f7369bc9640 1 --2- 172.21.15.115:0/1159852913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 0x7f7350005180 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f736406e8e0 tx=0x7f73500057f0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.517+0000 7f7363fff640 1 -- 172.21.15.115:0/1159852913 reap_dead start 2023-12-01T03:08:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.517+0000 7f7369bc9640 1 -- 172.21.15.115:0/1159852913 shutdown_connections 2023-12-01T03:08:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.517+0000 7f7369bc9640 1 -- 172.21.15.115:0/1159852913 >> 172.21.15.115:0/1159852913 conn(0x7f7364071430 msgr2=0x7f7364071840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.517+0000 7f7369bc9640 1 -- 172.21.15.115:0/1159852913 shutdown_connections 2023-12-01T03:08:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.517+0000 7f7369bc9640 1 -- 172.21.15.115:0/1159852913 wait complete. 2023-12-01T03:08:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.519+0000 7f7369bc9640 1 Processor -- start 2023-12-01T03:08:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.519+0000 7f7369bc9640 1 -- start start 2023-12-01T03:08:58.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.520+0000 7f7369bc9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 0x7f73640c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:58.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.520+0000 7f7369bc9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73641f22f0 0x7f73640ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:58.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.520+0000 7f7369bc9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73640c9390 0x7f73640aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:58.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.520+0000 7f7369bc9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73640c87b0 con 0x7f7350004d00 2023-12-01T03:08:58.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.520+0000 7f7369bc9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73640c7b30 con 0x7f73640c9390 2023-12-01T03:08:58.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.520+0000 7f7369bc9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73640c7e10 con 0x7f73641f22f0 2023-12-01T03:08:58.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f73637fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 0x7f73640c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:58.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f73637fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 0x7f73640c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:39868/0 (socket says 172.21.15.115:39868) 2023-12-01T03:08:58.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f73637fe640 1 -- 172.21.15.115:0/3157143755 learned_addr learned my addr 172.21.15.115:0/3157143755 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:08:58.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f7362ffd640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73641f22f0 0x7f73640ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:58.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f7363fff640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73640c9390 0x7f73640aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:58.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f73637fe640 1 -- 172.21.15.115:0/3157143755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73641f22f0 msgr2=0x7f73640ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:58.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f73637fe640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73641f22f0 0x7f73640ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f73637fe640 1 -- 172.21.15.115:0/3157143755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73640c9390 msgr2=0x7f73640aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:58.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f73637fe640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73640c9390 0x7f73640aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f73637fe640 1 -- 172.21.15.115:0/3157143755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73640c80f0 con 0x7f7350004d00 2023-12-01T03:08:58.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.521+0000 7f73637fe640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 0x7f73640c6c90 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f7354007f30 tx=0x7f7354011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:58.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.522+0000 7f735bfff640 1 -- 172.21.15.115:0/3157143755 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735400e030 con 0x7f7350004d00 2023-12-01T03:08:58.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.522+0000 7f735bfff640 1 -- 172.21.15.115:0/3157143755 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7354016470 con 0x7f7350004d00 2023-12-01T03:08:58.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.522+0000 7f735bfff640 1 -- 172.21.15.115:0/3157143755 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354016820 con 0x7f7350004d00 2023-12-01T03:08:58.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.522+0000 7f7369bc9640 1 -- 172.21.15.115:0/3157143755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73640c83d0 con 0x7f7350004d00 2023-12-01T03:08:58.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.522+0000 7f7369bc9640 1 -- 172.21.15.115:0/3157143755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73640c5540 con 0x7f7350004d00 2023-12-01T03:08:58.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.523+0000 7f7369bc9640 1 -- 172.21.15.115:0/3157143755 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736404e890 con 0x7f7350004d00 2023-12-01T03:08:58.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.524+0000 7f735bfff640 1 -- 172.21.15.115:0/3157143755 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 117) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f7354016980 con 0x7f7350004d00 2023-12-01T03:08:58.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.525+0000 7f735bfff640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f734406a2d0 0x7f734406c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:08:58.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.525+0000 7f735bfff640 1 -- 172.21.15.115:0/3157143755 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f735400f070 con 0x7f7350004d00 2023-12-01T03:08:58.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.525+0000 7f7362ffd640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f734406a2d0 0x7f734406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:08:58.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.526+0000 7f7362ffd640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f734406a2d0 0x7f734406c790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f734c000c00 tx=0x7f734c001150 comp rx=0 tx=0).ready entity=mgr.10299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:08:58.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.529+0000 7f735bfff640 1 -- 172.21.15.115:0/3157143755 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7354054f20 con 0x7f7350004d00 2023-12-01T03:08:58.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.599+0000 7f735bfff640 1 -- 172.21.15.115:0/3157143755 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 118) v1 ==== 88727+0+0 (secure 0 0 0) 0x7f73540552f0 con 0x7f7350004d00 2023-12-01T03:08:58.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.760+0000 7f7369bc9640 1 -- 172.21.15.115:0/3157143755 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73640ccbb0 con 0x7f734406a2d0 2023-12-01T03:08:58.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.772+0000 7f735bfff640 1 -- 172.21.15.115:0/3157143755 <== mgr.10299 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f734c001530 con 0x7f734406a2d0 2023-12-01T03:08:58.774 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:08:58.774 INFO:teuthology.orchestra.run.smithi115.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-01T03:08:58.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.774+0000 7f7369bc9640 1 -- 172.21.15.115:0/3157143755 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f734406a2d0 msgr2=0x7f734406c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:58.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.774+0000 7f7369bc9640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f734406a2d0 0x7f734406c790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f734c000c00 tx=0x7f734c001150 comp rx=0 tx=0).stop 2023-12-01T03:08:58.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.774+0000 7f7369bc9640 1 -- 172.21.15.115:0/3157143755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 msgr2=0x7f73640c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:08:58.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.774+0000 7f7369bc9640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 0x7f73640c6c90 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f7354007f30 tx=0x7f7354011660 comp rx=0 tx=0).stop 2023-12-01T03:08:58.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.775+0000 7f7369bc9640 1 -- 172.21.15.115:0/3157143755 shutdown_connections 2023-12-01T03:08:58.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.775+0000 7f7369bc9640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f734406a2d0 0x7f734406c790 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.775+0000 7f7369bc9640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73641f22f0 0x7f73640ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.775+0000 7f7369bc9640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73640c9390 0x7f73640aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.775+0000 7f7369bc9640 1 --2- 172.21.15.115:0/3157143755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7350004d00 0x7f73640c6c90 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:08:58.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.775+0000 7f7369bc9640 1 -- 172.21.15.115:0/3157143755 >> 172.21.15.115:0/3157143755 conn(0x7f7364071430 msgr2=0x7f73640b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:08:58.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.775+0000 7f7369bc9640 1 -- 172.21.15.115:0/3157143755 shutdown_connections 2023-12-01T03:08:58.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:08:58.775+0000 7f7369bc9640 1 -- 172.21.15.115:0/3157143755 wait complete. 2023-12-01T03:08:58.794 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-01T03:08:58.794 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-12-01T03:08:58.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:58.794 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:08:58.850 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T03:08:58.850 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:08:58.850 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:08:59.636 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:08:59.647 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:08:59.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:08:59.851 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:08:59.915 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-01T03:08:59.915 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:08:59.915 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:00.308 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:09:00.307+0000 7f0da5420640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:09:00.308 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:09:00.307+0000 7f0da5420640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:09:00.308 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:09:00.307+0000 7f0da5420640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:09:00.308 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:09:00.307+0000 7f0da5420640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:09:00.308 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:09:00.307+0000 7f0da5420640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:09:00.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:00.916 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:00.978 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:00.979 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:00.979 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:00.979 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:00.980 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-01-03_09_00_UTC'] 2023-12-01T03:09:00.980 DEBUG:tasks.cephfs.test_snap_schedules:exec=1701400138.3127363, now = 1701400140.980142, timo = 2 2023-12-01T03:09:01.980 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-12-01-03_09_00 in ~62s... 2023-12-01T03:09:01.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:01.981 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:02.043 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:02.044 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:02.044 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:02.044 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:03.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:03.045 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:03.108 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:03.109 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:03.109 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:03.109 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:04.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:04.110 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:04.171 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:04.173 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:04.173 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:04.173 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:05.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:05.174 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:05.237 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:05.238 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:05.238 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:05.239 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:06.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:06.239 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:06.303 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:06.304 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:06.305 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:06.305 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:07.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:07.306 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:07.368 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:07.369 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:07.369 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:07.369 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:08.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:08.370 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:08.430 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:08.432 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:08.432 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:08.432 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:09.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:09.432 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:09.495 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:09.495 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:09.495 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:10.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:10.496 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:10.557 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:10.558 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:10.558 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:10.558 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:11.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:11.559 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:11.623 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:11.624 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:11.624 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:11.625 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:12.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:12.625 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:12.687 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:12.688 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:12.688 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:12.688 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:13.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:13.690 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:13.751 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:13.752 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:13.752 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:13.752 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:14.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:14.754 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:14.815 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:14.817 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:14.817 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:14.817 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:15.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:15.818 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:15.880 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:15.881 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:15.881 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:15.881 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:16.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:16.882 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:16.946 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:16.947 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:16.947 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:16.947 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:17.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:17.949 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:18.010 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:18.012 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:18.012 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:18.012 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:19.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:19.013 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:19.075 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:19.077 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:19.077 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:19.077 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:20.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:20.078 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:20.139 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:20.141 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:20.141 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:20.141 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:21.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:21.142 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:21.203 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:21.205 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:21.205 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:21.205 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:22.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:22.206 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:22.269 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:22.271 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:22.271 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:22.271 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:23.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:23.272 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:23.333 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:23.335 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:23.335 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:23.335 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:24.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:24.336 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:24.397 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:24.399 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:24.399 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:24.399 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:25.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:25.399 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:25.461 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:25.463 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:25.463 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:25.463 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:26.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:26.464 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:26.526 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:26.528 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:26.528 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:26.528 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:27.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:27.530 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:27.590 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:27.592 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:27.592 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:27.592 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:28.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:28.593 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:28.656 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:28.657 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:28.657 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:28.658 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:29.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:29.659 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:29.722 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:29.724 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:29.724 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:29.724 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:29.867 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:09:29.876 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:09:30.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:30.725 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:30.788 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:30.790 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:30.790 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:30.790 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:31.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:31.792 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:31.854 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:31.856 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:31.856 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:31.856 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:32.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:32.857 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:32.920 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:32.921 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:32.921 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:32.921 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:33.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:33.922 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:33.984 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:33.986 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:33.986 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:33.986 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:34.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:34.987 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:35.049 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:35.050 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:35.051 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:35.051 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:36.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:36.052 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:36.112 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:36.113 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:36.114 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:36.114 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:37.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:37.114 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:37.177 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:37.178 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:37.178 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:37.178 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:38.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:38.180 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:38.242 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:38.244 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:38.244 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:38.244 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:39.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:39.244 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:39.305 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:39.307 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:39.307 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:39.307 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:40.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:40.308 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:40.371 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:40.372 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:40.372 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:40.373 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:41.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:41.374 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:41.437 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:41.438 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:41.438 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:41.438 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:42.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:42.440 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:42.499 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:42.500 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:42.500 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:42.500 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:43.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:43.502 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:43.564 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:43.565 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:43.565 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:43.566 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:44.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:44.567 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:44.629 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:44.631 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:44.631 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:44.631 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:45.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:45.632 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:45.692 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:45.694 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:45.694 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:45.694 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:46.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:46.695 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:46.756 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:46.758 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:46.758 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:46.758 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:47.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:47.759 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:47.820 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:47.821 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:47.821 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:47.821 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:48.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:48.823 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:48.885 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:48.886 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:48.886 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:48.886 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:49.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:49.887 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:49.950 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:49.951 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:49.952 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:49.952 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:50.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:50.953 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:51.013 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:51.015 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:51.015 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:51.015 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:52.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:52.016 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:52.080 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:52.081 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:52.081 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:52.081 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:53.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:53.082 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:53.144 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:53.146 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:53.146 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:53.146 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:54.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:54.148 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:54.209 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:54.211 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:54.211 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:54.211 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:55.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:55.212 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:55.274 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:55.276 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:55.276 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:55.276 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:56.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:56.277 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:56.340 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:56.342 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:56.342 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:56.342 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:57.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:57.343 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:57.405 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:57.407 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:57.407 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:57.407 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:58.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:58.408 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:58.469 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:58.470 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:58.470 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:58.470 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:09:59.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:09:59.471 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:09:59.531 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_09_00_UTC 2023-12-01T03:09:59.532 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:09:59.532 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-01T03:09:59.532 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-01T03:10:00.072 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:10:00.081 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:10:00.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:00.534 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:10:00.592 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_10_00_UTC 2023-12-01T03:10:00.593 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-01-03_10_00_UTC'} 2023-12-01T03:10:00.593 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-01-03_10_00_UTC'} 2023-12-01T03:10:00.593 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-12-01-03_09_00_UTC'} 2023-12-01T03:10:00.593 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-12-01-03_09_00_UTC'] 2023-12-01T03:10:00.593 DEBUG:tasks.cephfs.test_snap_schedules:exec=1701400138.3127363, now = 1701400200.593941, timo = 62 2023-12-01T03:10:01.595 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-01T03:10:01.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.820+0000 7f3f9d5bb640 1 Processor -- start 2023-12-01T03:10:01.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.820+0000 7f3f9d5bb640 1 -- start start 2023-12-01T03:10:01.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f9d5bb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3f980ce1b0 0x7f3f980cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:01.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f9d5bb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3f980b0e60 0x7f3f980b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:01.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f9d5bb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f980cc9a0 con 0x7f3f9815ac80 2023-12-01T03:10:01.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f9d5bb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f9815b060 con 0x7f3f980b0e60 2023-12-01T03:10:01.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f9d5bb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f980b0ce0 con 0x7f3f980ce1b0 2023-12-01T03:10:01.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f977fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3f980b0e60 0x7f3f980b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:01.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f977fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3f980b0e60 0x7f3f980b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35356/0 (socket says 172.21.15.115:35356) 2023-12-01T03:10:01.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f977fe640 1 -- 172.21.15.115:0/962037348 learned_addr learned my addr 172.21.15.115:0/962037348 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:01.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f8ffff640 1 --2- 172.21.15.115:0/962037348 >> v2:172.21.15.142:3300/0 conn(0x7f3f980ce1b0 0x7f3f980cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:01.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f977fe640 1 -- 172.21.15.115:0/962037348 >> v2:172.21.15.142:3300/0 conn(0x7f3f980ce1b0 msgr2=0x7f3f980cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:01.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f977fe640 1 --2- 172.21.15.115:0/962037348 >> v2:172.21.15.142:3300/0 conn(0x7f3f980ce1b0 0x7f3f980cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:01.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f977fe640 1 -- 172.21.15.115:0/962037348 >> v1:172.21.15.142:6789/0 conn(0x7f3f9815ac80 legacy=0x7f3f980cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:01.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.822+0000 7f3f977fe640 1 -- 172.21.15.115:0/962037348 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980cbd90 con 0x7f3f980b0e60 2023-12-01T03:10:01.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.823+0000 7f3f977fe640 1 --2- 172.21.15.115:0/962037348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f980b0e60 0x7f3f980b07a0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f3f9815e680 tx=0x7f3f9002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371677a5ebb943ca server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.824+0000 7f3f967fc640 1 -- 172.21.15.115:0/962037348 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9002ca80 con 0x7f3f980b0e60 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.824+0000 7f3f967fc640 1 -- 172.21.15.115:0/962037348 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9002cbe0 con 0x7f3f980b0e60 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.824+0000 7f3f967fc640 1 -- 172.21.15.115:0/962037348 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f900343f0 con 0x7f3f980b0e60 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.824+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/962037348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f980b0e60 msgr2=0x7f3f980b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.824+0000 7f3f9d5bb640 1 --2- 172.21.15.115:0/962037348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f980b0e60 0x7f3f980b07a0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f3f9815e680 tx=0x7f3f9002bee0 comp rx=0 tx=0).stop 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.825+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/962037348 shutdown_connections 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.825+0000 7f3f9d5bb640 1 --2- 172.21.15.115:0/962037348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f980b0e60 0x7f3f980b07a0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.825+0000 7f3f9d5bb640 1 --2- 172.21.15.115:0/962037348 >> v2:172.21.15.142:3300/0 conn(0x7f3f980ce1b0 0x7f3f980cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.825+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/962037348 >> 172.21.15.115:0/962037348 conn(0x7f3f98065960 msgr2=0x7f3f98065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.825+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/962037348 shutdown_connections 2023-12-01T03:10:01.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.825+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/962037348 wait complete. 2023-12-01T03:10:01.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.826+0000 7f3f9d5bb640 1 Processor -- start 2023-12-01T03:10:01.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.827+0000 7f3f9d5bb640 1 -- start start 2023-12-01T03:10:01.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f9d5bb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f980b0e60 0x7f3f9814d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:01.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f9d5bb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f980ce1b0 0x7f3f981587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:01.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f9d5bb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f9815ac80 0x7f3f98158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:01.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f9d5bb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9814da00 con 0x7f3f9815ac80 2023-12-01T03:10:01.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f9d5bb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f98159f50 con 0x7f3f980ce1b0 2023-12-01T03:10:01.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f9d5bb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f98159340 con 0x7f3f980b0e60 2023-12-01T03:10:01.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f977fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f9815ac80 0x7f3f98158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:01.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f977fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f9815ac80 0x7f3f98158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35360/0 (socket says 172.21.15.115:35360) 2023-12-01T03:10:01.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f977fe640 1 -- 172.21.15.115:0/3791518483 learned_addr learned my addr 172.21.15.115:0/3791518483 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:01.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f8ffff640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f980ce1b0 0x7f3f981587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:01.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f96ffd640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f980b0e60 0x7f3f9814d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f977fe640 1 -- 172.21.15.115:0/3791518483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f980b0e60 msgr2=0x7f3f9814d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f977fe640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f980b0e60 0x7f3f9814d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f977fe640 1 -- 172.21.15.115:0/3791518483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f980ce1b0 msgr2=0x7f3f981587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f977fe640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f980ce1b0 0x7f3f981587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f977fe640 1 -- 172.21.15.115:0/3791518483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f90009b30 con 0x7f3f9815ac80 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.828+0000 7f3f8ffff640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f980ce1b0 0x7f3f981587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.829+0000 7f3f977fe640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f9815ac80 0x7f3f98158d30 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f3f90036040 tx=0x7f3f90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.829+0000 7f3f8f7fe640 1 -- 172.21.15.115:0/3791518483 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9003e450 con 0x7f3f9815ac80 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.829+0000 7f3f8f7fe640 1 -- 172.21.15.115:0/3791518483 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9003e5d0 con 0x7f3f9815ac80 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.829+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/3791518483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f98159620 con 0x7f3f9815ac80 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.829+0000 7f3f8f7fe640 1 -- 172.21.15.115:0/3791518483 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9003e980 con 0x7f3f9815ac80 2023-12-01T03:10:01.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.829+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/3791518483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f98159be0 con 0x7f3f9815ac80 2023-12-01T03:10:01.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.830+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/3791518483 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9804e890 con 0x7f3f9815ac80 2023-12-01T03:10:01.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.831+0000 7f3f8f7fe640 1 -- 172.21.15.115:0/3791518483 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 119) v1 ==== 88840+0+0 (secure 0 0 0) 0x7f3f9002cd50 con 0x7f3f9815ac80 2023-12-01T03:10:01.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.832+0000 7f3f8f7fe640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7806a6f0 0x7f3f7806cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:01.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.832+0000 7f3f8f7fe640 1 -- 172.21.15.115:0/3791518483 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f3f900ab940 con 0x7f3f9815ac80 2023-12-01T03:10:01.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.832+0000 7f3f96ffd640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7806a6f0 0x7f3f7806cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:01.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.833+0000 7f3f96ffd640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7806a6f0 0x7f3f7806cbb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3f88006fd0 tx=0x7f3f88008040 comp rx=0 tx=0).ready entity=mgr.10299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:01.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:01.837+0000 7f3f8f7fe640 1 -- 172.21.15.115:0/3791518483 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3f90076970 con 0x7f3f9815ac80 2023-12-01T03:10:02.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.046+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/3791518483 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f980cc350 con 0x7f3f7806a6f0 2023-12-01T03:10:02.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.175+0000 7f3f8f7fe640 1 -- 172.21.15.115:0/3791518483 <== mgr.10299 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3f8800be50 con 0x7f3f7806a6f0 2023-12-01T03:10:02.177 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path snap_test_dir1 2023-12-01T03:10:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.178+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/3791518483 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7806a6f0 msgr2=0x7f3f7806cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.178+0000 7f3f9d5bb640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7806a6f0 0x7f3f7806cbb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3f88006fd0 tx=0x7f3f88008040 comp rx=0 tx=0).stop 2023-12-01T03:10:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.178+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/3791518483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f9815ac80 msgr2=0x7f3f98158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.178+0000 7f3f9d5bb640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f9815ac80 0x7f3f98158d30 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f3f90036040 tx=0x7f3f90004a40 comp rx=0 tx=0).stop 2023-12-01T03:10:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.178+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/3791518483 shutdown_connections 2023-12-01T03:10:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.178+0000 7f3f9d5bb640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7806a6f0 0x7f3f7806cbb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.178+0000 7f3f9d5bb640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f9815ac80 0x7f3f98158d30 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.178+0000 7f3f9d5bb640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f980ce1b0 0x7f3f981587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.178+0000 7f3f9d5bb640 1 --2- 172.21.15.115:0/3791518483 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f980b0e60 0x7f3f9814d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.178+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/3791518483 >> 172.21.15.115:0/3791518483 conn(0x7f3f98065960 msgr2=0x7f3f9806c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.179+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/3791518483 shutdown_connections 2023-12-01T03:10:02.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.179+0000 7f3f9d5bb640 1 -- 172.21.15.115:0/3791518483 wait complete. 2023-12-01T03:10:02.201 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-01T03:10:02.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:02.201 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:10:02.251 INFO:teuthology.orchestra.run.smithi115.stdout:scheduled-2023-12-01-03_10_00_UTC 2023-12-01T03:10:02.252 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-01-03_10_00_UTC 2023-12-01T03:10:02.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:02.252 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-01-03_10_00_UTC) 2023-12-01T03:10:02.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:02.306 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-01T03:10:02.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T03:10:02.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.581+0000 7fda80f9b640 1 Processor -- start 2023-12-01T03:10:02.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.581+0000 7fda80f9b640 1 -- start start 2023-12-01T03:10:02.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.582+0000 7fda80f9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda7c066070 0x7fda7c15b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:02.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.583+0000 7fda80f9b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7c155220 con 0x7fda7c154e40 2023-12-01T03:10:02.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.583+0000 7fda80f9b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7c0ccef0 con 0x7fda7c0c9d60 2023-12-01T03:10:02.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.583+0000 7fda80f9b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c071190 con 0x7fda7c066070 2023-12-01T03:10:02.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.583+0000 7fda7b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fda7c154e40 0x7fda7c0cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52540/0 (socket says 172.21.15.115:52540) 2023-12-01T03:10:02.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.583+0000 7fda7b7fe640 1 -- 172.21.15.115:0/1096495117 learned_addr learned my addr 172.21.15.115:0/1096495117 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.583+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572288755 0 0) 0x7fda7c155220 con 0x7fda7c154e40 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.583+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda68003520 con 0x7fda7c154e40 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.583+0000 7fda7bfff640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda7c066070 0x7fda7c15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2171857555 0 0) 0x7fda68003520 con 0x7fda7c154e40 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda7c066070 msgr2=0x7fda7c15b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda7c066070 0x7fda7c15b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 >> v1:172.21.15.142:6789/0 conn(0x7fda7c0c9d60 legacy=0x7fda7c065960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c0ca340 con 0x7fda7c154e40 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fda6c0019f0 con 0x7fda7c154e40 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 >> v1:172.21.15.115:6789/0 conn(0x7fda7c154e40 legacy=0x7fda7c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:02.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda7c066070 0x7fda680039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda68003f70 0x7fda680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda68004d00 0x7fda68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6800ba90 con 0x7fda68004d00 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6800bc00 con 0x7fda68003f70 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6800bdc0 con 0x7fda7c066070 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7bfff640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda68004d00 0x7fda68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7b7fe640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda7c066070 0x7fda680039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.584+0000 7fda7affd640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda68003f70 0x7fda680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.585+0000 7fda7bfff640 1 -- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda7c066070 msgr2=0x7fda680039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.585+0000 7fda7bfff640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda7c066070 0x7fda680039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.585+0000 7fda7bfff640 1 -- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda68003f70 msgr2=0x7fda680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.585+0000 7fda7bfff640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda68003f70 0x7fda680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:02.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.585+0000 7fda7bfff640 1 -- 172.21.15.115:0/1096495117 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda68003520 con 0x7fda68004d00 2023-12-01T03:10:02.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.586+0000 7fda7bfff640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda68004d00 0x7fda68005180 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fda7c15e360 tx=0x7fda680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb66cf0ba7ae7f6c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.586+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7402c660 con 0x7fda68004d00 2023-12-01T03:10:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.586+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7402c7c0 con 0x7fda68004d00 2023-12-01T03:10:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.586+0000 7fda7a7fc640 1 -- 172.21.15.115:0/1096495117 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7402cb70 con 0x7fda68004d00 2023-12-01T03:10:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.587+0000 7fda80f9b640 1 -- 172.21.15.115:0/1096495117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda68004d00 msgr2=0x7fda68005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.587+0000 7fda80f9b640 1 --2- 172.21.15.115:0/1096495117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda68004d00 0x7fda68005180 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fda7c15e360 tx=0x7fda680057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.587+0000 7fda7bfff640 1 -- 172.21.15.115:0/1096495117 reap_dead start 2023-12-01T03:10:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.587+0000 7fda80f9b640 1 -- 172.21.15.115:0/1096495117 shutdown_connections 2023-12-01T03:10:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.587+0000 7fda80f9b640 1 -- 172.21.15.115:0/1096495117 >> 172.21.15.115:0/1096495117 conn(0x7fda7c0b1380 msgr2=0x7fda7c0b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.587+0000 7fda80f9b640 1 -- 172.21.15.115:0/1096495117 shutdown_connections 2023-12-01T03:10:02.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.587+0000 7fda80f9b640 1 -- 172.21.15.115:0/1096495117 wait complete. 2023-12-01T03:10:02.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.589+0000 7fda80f9b640 1 Processor -- start 2023-12-01T03:10:02.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.589+0000 7fda80f9b640 1 -- start start 2023-12-01T03:10:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda80f9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda68004d00 0x7fda7c14b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda80f9b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda7c159850 0x7fda7c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda80f9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda7c0cd5f0 0x7fda7c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:02.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda80f9b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c15a1f0 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda80f9b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0cec00 con 0x7fda7c159850 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda80f9b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0ce360 con 0x7fda68004d00 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda7c0cd5f0 0x7fda7c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda7c0cd5f0 0x7fda7c0cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35386/0 (socket says 172.21.15.115:35386) 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7bfff640 1 -- 172.21.15.115:0/1549885247 learned_addr learned my addr 172.21.15.115:0/1549885247 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7affd640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda7c159850 0x7fda7c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7b7fe640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda68004d00 0x7fda7c14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7bfff640 1 -- 172.21.15.115:0/1549885247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda68004d00 msgr2=0x7fda7c14b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7bfff640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda68004d00 0x7fda7c14b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7bfff640 1 -- 172.21.15.115:0/1549885247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda7c159850 msgr2=0x7fda7c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7bfff640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda7c159850 0x7fda7c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7bfff640 1 -- 172.21.15.115:0/1549885247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c0ce640 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.590+0000 7fda7b7fe640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda68004d00 0x7fda7c14b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.591+0000 7fda7bfff640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda7c0cd5f0 0x7fda7c0cda70 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fda7402c200 tx=0x7fda74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:02.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.591+0000 7fda73fff640 1 -- 172.21.15.115:0/1549885247 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7403e850 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.591+0000 7fda73fff640 1 -- 172.21.15.115:0/1549885247 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7403e9d0 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.591+0000 7fda73fff640 1 -- 172.21.15.115:0/1549885247 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7403ed80 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.591+0000 7fda80f9b640 1 -- 172.21.15.115:0/1549885247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c0ce920 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.591+0000 7fda80f9b640 1 -- 172.21.15.115:0/1549885247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c14b6a0 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.592+0000 7fda80f9b640 1 -- 172.21.15.115:0/1549885247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c0c6b50 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.593+0000 7fda73fff640 1 -- 172.21.15.115:0/1549885247 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 119) v1 ==== 88840+0+0 (secure 0 0 0) 0x7fda74034440 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.594+0000 7fda73fff640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fda5c06a6c0 0x7fda5c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.594+0000 7fda73fff640 1 -- 172.21.15.115:0/1549885247 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fda74000e40 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.594+0000 7fda7b7fe640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fda5c06a6c0 0x7fda5c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:02.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.595+0000 7fda7b7fe640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fda5c06a6c0 0x7fda5c06cb80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda6c002230 tx=0x7fda6c001310 comp rx=0 tx=0).ready entity=mgr.10299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:02.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.599+0000 7fda73fff640 1 -- 172.21.15.115:0/1549885247 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fda7c0c6b50 con 0x7fda7c0cd5f0 2023-12-01T03:10:02.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:02.839+0000 7fda80f9b640 1 -- 172.21.15.115:0/1549885247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fda7c0c6b50 con 0x7fda7c0cd5f0 2023-12-01T03:10:03.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.473+0000 7fda73fff640 1 -- 172.21.15.115:0/1549885247 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 120) v1 ==== 88823+0+0 (secure 0 0 0) 0x7fda74075d60 con 0x7fda7c0cd5f0 2023-12-01T03:10:03.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.475+0000 7fda73fff640 1 -- 172.21.15.115:0/1549885247 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v120) v1 ==== 93+0+0 (secure 0 0 0) 0x7fda7407d1d0 con 0x7fda7c0cd5f0 2023-12-01T03:10:03.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.477+0000 7fda80f9b640 1 -- 172.21.15.115:0/1549885247 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fda5c06a6c0 msgr2=0x7fda5c06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.477+0000 7fda80f9b640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fda5c06a6c0 0x7fda5c06cb80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda6c002230 tx=0x7fda6c001310 comp rx=0 tx=0).stop 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.477+0000 7fda80f9b640 1 -- 172.21.15.115:0/1549885247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda7c0cd5f0 msgr2=0x7fda7c0cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.477+0000 7fda80f9b640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda7c0cd5f0 0x7fda7c0cda70 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fda7402c200 tx=0x7fda74004730 comp rx=0 tx=0).stop 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.478+0000 7fda80f9b640 1 -- 172.21.15.115:0/1549885247 shutdown_connections 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.478+0000 7fda80f9b640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fda5c06a6c0 0x7fda5c06cb80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.478+0000 7fda80f9b640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda7c0cd5f0 0x7fda7c0cda70 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.478+0000 7fda80f9b640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda7c159850 0x7fda7c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.478+0000 7fda80f9b640 1 --2- 172.21.15.115:0/1549885247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda68004d00 0x7fda7c14b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.478+0000 7fda80f9b640 1 -- 172.21.15.115:0/1549885247 >> 172.21.15.115:0/1549885247 conn(0x7fda7c0b1380 msgr2=0x7fda7c0b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.478+0000 7fda80f9b640 1 -- 172.21.15.115:0/1549885247 shutdown_connections 2023-12-01T03:10:03.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:03.478+0000 7fda80f9b640 1 -- 172.21.15.115:0/1549885247 wait complete. 2023-12-01T03:10:03.499 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T03:10:03.499 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:10:03.598 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:10:03.690 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T03:10:03.690 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:10:03.690 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:10:03.744 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:10:03.744 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:10:03.835 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:03.978 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:03.977+0000 7f4ac4c33040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:10:04.005 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:04.003+0000 7feeb6a2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:10:04.032 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:04.031+0000 7f4ac4c33040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:10:04.061 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:04.059+0000 7feeb6a2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:10:04.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.082+0000 7f8a7ffff640 1 Processor -- start 2023-12-01T03:10:04.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.082+0000 7f8a7ffff640 1 -- start start 2023-12-01T03:10:04.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.083+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a8015d7b0 0x7f8a8015db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.083+0000 7f8a7ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8a80160670 0x7f8a800cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.083+0000 7f8a7ffff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a800cd3f0 con 0x7f8a800cddc0 2023-12-01T03:10:04.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.083+0000 7f8a7ffff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a800cd0f0 con 0x7f8a80160670 2023-12-01T03:10:04.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.083+0000 7f8a7ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80160ec0 con 0x7f8a8015d7b0 2023-12-01T03:10:04.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.084+0000 7f8a77fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8a80160670 0x7f8a800cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.084+0000 7f8a77fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8a80160670 0x7f8a800cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35412/0 (socket says 172.21.15.115:35412) 2023-12-01T03:10:04.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.084+0000 7f8a77fff640 1 -- 172.21.15.115:0/2645439438 learned_addr learned my addr 172.21.15.115:0/2645439438 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:04.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.084+0000 7f8a7effd640 1 --2- 172.21.15.115:0/2645439438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a8015d7b0 0x7f8a8015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.084+0000 7f8a77fff640 1 -- 172.21.15.115:0/2645439438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a8015d7b0 msgr2=0x7f8a8015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:04.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.084+0000 7f8a77fff640 1 --2- 172.21.15.115:0/2645439438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a8015d7b0 0x7f8a8015db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.084+0000 7f8a77fff640 1 -- 172.21.15.115:0/2645439438 >> v1:172.21.15.115:6789/0 conn(0x7f8a800cddc0 legacy=0x7f8a800d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.084+0000 7f8a7e7fc640 1 -- 172.21.15.115:0/2645439438 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293189210 0 0) 0x7f8a800cd3f0 con 0x7f8a800cddc0 2023-12-01T03:10:04.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.084+0000 7f8a77fff640 1 -- 172.21.15.115:0/2645439438 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a801611a0 con 0x7f8a80160670 2023-12-01T03:10:04.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.084+0000 7f8a7effd640 1 --2- 172.21.15.115:0/2645439438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a8015d7b0 0x7f8a8015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:04.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.086+0000 7f8a77fff640 1 --2- 172.21.15.115:0/2645439438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 0x7f8a800cd880 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009870 tx=0x7f8a6802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89aebd55b767d3fc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:04.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.086+0000 7f8a7e7fc640 1 -- 172.21.15.115:0/2645439438 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6802f840 con 0x7f8a80160670 2023-12-01T03:10:04.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.086+0000 7f8a7e7fc640 1 -- 172.21.15.115:0/2645439438 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6802f9a0 con 0x7f8a80160670 2023-12-01T03:10:04.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.086+0000 7f8a7e7fc640 1 -- 172.21.15.115:0/2645439438 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6802fd30 con 0x7f8a80160670 2023-12-01T03:10:04.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.087+0000 7f8a7ffff640 1 -- 172.21.15.115:0/2645439438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 msgr2=0x7f8a800cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.087+0000 7f8a7ffff640 1 --2- 172.21.15.115:0/2645439438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 0x7f8a800cd880 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009870 tx=0x7f8a6802ba20 comp rx=0 tx=0).stop 2023-12-01T03:10:04.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.087+0000 7f8a7ffff640 1 -- 172.21.15.115:0/2645439438 shutdown_connections 2023-12-01T03:10:04.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.087+0000 7f8a7ffff640 1 --2- 172.21.15.115:0/2645439438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 0x7f8a800cd880 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.087+0000 7f8a7ffff640 1 --2- 172.21.15.115:0/2645439438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a8015d7b0 0x7f8a8015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.087+0000 7f8a7ffff640 1 -- 172.21.15.115:0/2645439438 >> 172.21.15.115:0/2645439438 conn(0x7f8a8006a0e0 msgr2=0x7f8a8006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:04.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.087+0000 7f8a7ffff640 1 -- 172.21.15.115:0/2645439438 shutdown_connections 2023-12-01T03:10:04.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.087+0000 7f8a7ffff640 1 -- 172.21.15.115:0/2645439438 wait complete. 2023-12-01T03:10:04.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.088+0000 7f8a7ffff640 1 Processor -- start 2023-12-01T03:10:04.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.089+0000 7f8a7ffff640 1 -- start start 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.089+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a800cddc0 0x7f8a800c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.089+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a8015d7b0 0x7f8a801576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 0x7f8a80158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7ffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a800c9010 con 0x7f8a80160670 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7ffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80158ba0 con 0x7f8a8015d7b0 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80158020 con 0x7f8a800cddc0 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 0x7f8a80158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 0x7f8a80158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35422/0 (socket says 172.21.15.115:35422) 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7f7fe640 1 -- 172.21.15.115:0/566982731 learned_addr learned my addr 172.21.15.115:0/566982731 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7effd640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a800cddc0 0x7f8a800c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7f7fe640 1 -- 172.21.15.115:0/566982731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a800cddc0 msgr2=0x7f8a800c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7f7fe640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a800cddc0 0x7f8a800c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7f7fe640 1 -- 172.21.15.115:0/566982731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a8015d7b0 msgr2=0x7f8a801576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7f7fe640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a8015d7b0 0x7f8a801576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.090+0000 7f8a7f7fe640 1 -- 172.21.15.115:0/566982731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a680094d0 con 0x7f8a80160670 2023-12-01T03:10:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.091+0000 7f8a7f7fe640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 0x7f8a80158e00 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f8a78001560 tx=0x7f8a7800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.091+0000 7f8a777fe640 1 -- 172.21.15.115:0/566982731 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78003da0 con 0x7f8a80160670 2023-12-01T03:10:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.091+0000 7f8a777fe640 1 -- 172.21.15.115:0/566982731 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a78005070 con 0x7f8a80160670 2023-12-01T03:10:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.091+0000 7f8a7ffff640 1 -- 172.21.15.115:0/566982731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a80158300 con 0x7f8a80160670 2023-12-01T03:10:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.091+0000 7f8a777fe640 1 -- 172.21.15.115:0/566982731 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78005420 con 0x7f8a80160670 2023-12-01T03:10:04.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.091+0000 7f8a7ffff640 1 -- 172.21.15.115:0/566982731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a801588c0 con 0x7f8a80160670 2023-12-01T03:10:04.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.092+0000 7f8a7ffff640 1 -- 172.21.15.115:0/566982731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a800673a0 con 0x7f8a80160670 2023-12-01T03:10:04.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.093+0000 7f8a777fe640 1 -- 172.21.15.115:0/566982731 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 120) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f8a78005580 con 0x7f8a80160670 2023-12-01T03:10:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.094+0000 7f8a777fe640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8a6006a670 0x7f8a6006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.095+0000 7f8a7effd640 1 -- 172.21.15.115:0/566982731 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8a6006a670 msgr2=0x7f8a6006cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.095+0000 7f8a777fe640 1 -- 172.21.15.115:0/566982731 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f8a78088850 con 0x7f8a80160670 2023-12-01T03:10:04.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.095+0000 7f8a7effd640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8a6006a670 0x7f8a6006cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:04.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.099+0000 7f8a777fe640 1 -- 172.21.15.115:0/566982731 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8a78053880 con 0x7f8a80160670 2023-12-01T03:10:04.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.290+0000 7f8a7ffff640 1 -- 172.21.15.115:0/566982731 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a800d2600 con 0x7f8a80160670 2023-12-01T03:10:04.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.292+0000 7f8a777fe640 1 -- 172.21.15.115:0/566982731 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+16827 (secure 0 0 0) 0x7f8a78017090 con 0x7f8a80160670 2023-12-01T03:10:04.294 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:04.295 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":199,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:09:58.429023+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:08:40.709195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-01T03:08:42.071749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.18","mappings":[{"from":5,"to":3}]},{"pgid":"23.19","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/361869044":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/4037780703":"2023-12-02T03:08:56.516548+0000","172.21.15.115:6824/138932":"2023-12-02T03:08:56.516548+0000","172.21.15.115:6822/138932":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/3414835041":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/1495322174":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/2764969208":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/1016167991":"2023-12-02T03:08:56.516548+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 -- 172.21.15.115:0/566982731 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8a6006a670 msgr2=0x7f8a6006cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8a6006a670 0x7f8a6006cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 -- 172.21.15.115:0/566982731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 msgr2=0x7f8a80158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 0x7f8a80158e00 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f8a78001560 tx=0x7f8a7800cc70 comp rx=0 tx=0).stop 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 -- 172.21.15.115:0/566982731 shutdown_connections 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8a6006a670 0x7f8a6006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a80160670 0x7f8a80158e00 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a8015d7b0 0x7f8a801576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 --2- 172.21.15.115:0/566982731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a800cddc0 0x7f8a800c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 -- 172.21.15.115:0/566982731 >> 172.21.15.115:0/566982731 conn(0x7f8a8006a0e0 msgr2=0x7f8a8006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:04.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 -- 172.21.15.115:0/566982731 shutdown_connections 2023-12-01T03:10:04.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.295+0000 7f8a7ffff640 1 -- 172.21.15.115:0/566982731 wait complete. 2023-12-01T03:10:04.315 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T03:08:40.709195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T03:08:42.071749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:10:04.315 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:10:04.479 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:04.478+0000 7f4ac4c33040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:10:04.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.518+0000 7fdd7ca50640 1 Processor -- start 2023-12-01T03:10:04.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.518+0000 7fdd7ca50640 1 -- start start 2023-12-01T03:10:04.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.519+0000 7fdd7ca50640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdd781597a0 0x7fdd7806d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.519+0000 7fdd7ca50640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd78072d00 0x7fdd7815df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.519+0000 7fdd7ca50640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7806cfa0 con 0x7fdd780cdb20 2023-12-01T03:10:04.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.519+0000 7fdd7ca50640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7815e440 con 0x7fdd78072d00 2023-12-01T03:10:04.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.519+0000 7fdd7ca50640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd78073160 con 0x7fdd781597a0 2023-12-01T03:10:04.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.520+0000 7fdd7659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd78072d00 0x7fdd7815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.520+0000 7fdd7659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd78072d00 0x7fdd7815df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35432/0 (socket says 172.21.15.115:35432) 2023-12-01T03:10:04.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.520+0000 7fdd7659c640 1 -- 172.21.15.115:0/3398171668 learned_addr learned my addr 172.21.15.115:0/3398171668 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:04.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.520+0000 7fdd76d9d640 1 --2- 172.21.15.115:0/3398171668 >> v2:172.21.15.142:3300/0 conn(0x7fdd781597a0 0x7fdd7806d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.520+0000 7fdd7659c640 1 -- 172.21.15.115:0/3398171668 >> v2:172.21.15.142:3300/0 conn(0x7fdd781597a0 msgr2=0x7fdd7806d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:04.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.520+0000 7fdd7659c640 1 --2- 172.21.15.115:0/3398171668 >> v2:172.21.15.142:3300/0 conn(0x7fdd781597a0 0x7fdd7806d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.520+0000 7fdd7659c640 1 -- 172.21.15.115:0/3398171668 >> v1:172.21.15.115:6789/0 conn(0x7fdd780cdb20 legacy=0x7fdd780c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.520+0000 7fdd7659c640 1 -- 172.21.15.115:0/3398171668 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7806c3d0 con 0x7fdd78072d00 2023-12-01T03:10:04.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.520+0000 7fdd76d9d640 1 --2- 172.21.15.115:0/3398171668 >> v2:172.21.15.142:3300/0 conn(0x7fdd781597a0 0x7fdd7806d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:04.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.521+0000 7fdd7659c640 1 --2- 172.21.15.115:0/3398171668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd78072d00 0x7fdd7815df00 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b1970 tx=0x7fdd6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fbfcbc2f2bf9f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:04.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.521+0000 7fdd75d9b640 1 -- 172.21.15.115:0/3398171668 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c031870 con 0x7fdd78072d00 2023-12-01T03:10:04.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.522+0000 7fdd75d9b640 1 -- 172.21.15.115:0/3398171668 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c0319d0 con 0x7fdd78072d00 2023-12-01T03:10:04.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.522+0000 7fdd75d9b640 1 -- 172.21.15.115:0/3398171668 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c031d60 con 0x7fdd78072d00 2023-12-01T03:10:04.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.522+0000 7fdd7ca50640 1 -- 172.21.15.115:0/3398171668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd78072d00 msgr2=0x7fdd7815df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.522+0000 7fdd7ca50640 1 --2- 172.21.15.115:0/3398171668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd78072d00 0x7fdd7815df00 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b1970 tx=0x7fdd6c02b910 comp rx=0 tx=0).stop 2023-12-01T03:10:04.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.522+0000 7fdd7ca50640 1 -- 172.21.15.115:0/3398171668 shutdown_connections 2023-12-01T03:10:04.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.522+0000 7fdd7ca50640 1 --2- 172.21.15.115:0/3398171668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd78072d00 0x7fdd7815df00 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.522+0000 7fdd7ca50640 1 --2- 172.21.15.115:0/3398171668 >> v2:172.21.15.142:3300/0 conn(0x7fdd781597a0 0x7fdd7806d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.522+0000 7fdd7ca50640 1 -- 172.21.15.115:0/3398171668 >> 172.21.15.115:0/3398171668 conn(0x7fdd780cee40 msgr2=0x7fdd780cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:04.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.522+0000 7fdd7ca50640 1 -- 172.21.15.115:0/3398171668 shutdown_connections 2023-12-01T03:10:04.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.522+0000 7fdd7ca50640 1 -- 172.21.15.115:0/3398171668 wait complete. 2023-12-01T03:10:04.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.524+0000 7fdd7ca50640 1 Processor -- start 2023-12-01T03:10:04.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.524+0000 7fdd7ca50640 1 -- start start 2023-12-01T03:10:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.525+0000 7fdd7ca50640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd78072d00 0x7fdd781500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.525+0000 7fdd7ca50640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780cdb20 0x7fdd780ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.525+0000 7fdd7ca50640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd781597a0 0x7fdd7814f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.525+0000 7fdd7ca50640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780ad030 con 0x7fdd781597a0 2023-12-01T03:10:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.525+0000 7fdd7ca50640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780ad310 con 0x7fdd78072d00 2023-12-01T03:10:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.525+0000 7fdd7ca50640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780ad5f0 con 0x7fdd780cdb20 2023-12-01T03:10:04.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.525+0000 7fdd7759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd781597a0 0x7fdd7814f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.525+0000 7fdd7759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd781597a0 0x7fdd7814f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35436/0 (socket says 172.21.15.115:35436) 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd7759e640 1 -- 172.21.15.115:0/178869123 learned_addr learned my addr 172.21.15.115:0/178869123 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd7659c640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780cdb20 0x7fdd780ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd76d9d640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd78072d00 0x7fdd781500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd7759e640 1 -- 172.21.15.115:0/178869123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780cdb20 msgr2=0x7fdd780ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd7759e640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780cdb20 0x7fdd780ac9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd7759e640 1 -- 172.21.15.115:0/178869123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd78072d00 msgr2=0x7fdd781500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd7759e640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd78072d00 0x7fdd781500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd7759e640 1 -- 172.21.15.115:0/178869123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0094d0 con 0x7fdd781597a0 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd7759e640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd781597a0 0x7fdd7814f7a0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c7a0 tx=0x7fdd7000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd5f7fe640 1 -- 172.21.15.115:0/178869123 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70003b70 con 0x7fdd781597a0 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd5f7fe640 1 -- 172.21.15.115:0/178869123 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70003cf0 con 0x7fdd781597a0 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd5f7fe640 1 -- 172.21.15.115:0/178869123 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70003ea0 con 0x7fdd781597a0 2023-12-01T03:10:04.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.526+0000 7fdd7ca50640 1 -- 172.21.15.115:0/178869123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd780ad8d0 con 0x7fdd781597a0 2023-12-01T03:10:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.527+0000 7fdd7ca50640 1 -- 172.21.15.115:0/178869123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd78150900 con 0x7fdd781597a0 2023-12-01T03:10:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.528+0000 7fdd7ca50640 1 -- 172.21.15.115:0/178869123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd780addf0 con 0x7fdd781597a0 2023-12-01T03:10:04.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.528+0000 7fdd5f7fe640 1 -- 172.21.15.115:0/178869123 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 120) v1 ==== 88823+0+0 (secure 0 0 0) 0x7fdd7000aa90 con 0x7fdd781597a0 2023-12-01T03:10:04.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.529+0000 7fdd5f7fe640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdd6406a670 0x7fdd6406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.529+0000 7fdd5f7fe640 1 -- 172.21.15.115:0/178869123 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fdd70088cd0 con 0x7fdd781597a0 2023-12-01T03:10:04.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.529+0000 7fdd76d9d640 1 -- 172.21.15.115:0/178869123 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdd6406a670 msgr2=0x7fdd6406cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:04.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.529+0000 7fdd76d9d640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdd6406a670 0x7fdd6406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:04.531 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:04.529+0000 7feeb6a2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:10:04.533 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:04.532+0000 7f4ac4c33040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:10:04.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.534+0000 7fdd5f7fe640 1 -- 172.21.15.115:0/178869123 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdd780addf0 con 0x7fdd781597a0 2023-12-01T03:10:04.584 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:04.583+0000 7f4ac4c33040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:10:04.585 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:04.583+0000 7feeb6a2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:10:04.636 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:04.635+0000 7feeb6a2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:10:04.645 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:04.644+0000 7f4ac4c33040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:10:04.694 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:04.693+0000 7f4ac4c33040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:10:04.698 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:04.696+0000 7feeb6a2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:10:04.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.722+0000 7fdd7ca50640 1 -- 172.21.15.115:0/178869123 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd780addf0 con 0x7fdd781597a0 2023-12-01T03:10:04.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.723+0000 7fdd5f7fe640 1 -- 172.21.15.115:0/178869123 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v199) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd780addf0 con 0x7fdd781597a0 2023-12-01T03:10:04.725 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:10:04.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 -- 172.21.15.115:0/178869123 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdd6406a670 msgr2=0x7fdd6406cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:04.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdd6406a670 0x7fdd6406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 -- 172.21.15.115:0/178869123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd781597a0 msgr2=0x7fdd7814f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd781597a0 0x7fdd7814f7a0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c7a0 tx=0x7fdd7000cc60 comp rx=0 tx=0).stop 2023-12-01T03:10:04.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 -- 172.21.15.115:0/178869123 shutdown_connections 2023-12-01T03:10:04.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdd6406a670 0x7fdd6406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780cdb20 0x7fdd780ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd781597a0 0x7fdd7814f7a0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 --2- 172.21.15.115:0/178869123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd78072d00 0x7fdd781500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 -- 172.21.15.115:0/178869123 >> 172.21.15.115:0/178869123 conn(0x7fdd780cee40 msgr2=0x7fdd780afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:04.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 -- 172.21.15.115:0/178869123 shutdown_connections 2023-12-01T03:10:04.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.726+0000 7fdd7ca50640 1 -- 172.21.15.115:0/178869123 wait complete. 2023-12-01T03:10:04.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:10:04.752 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:04.748+0000 7feeb6a2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:10:04.803 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:04.802+0000 7f4ac4c33040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:10:04.866 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:04.864+0000 7f4ac4c33040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:10:04.866 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:04.864+0000 7feeb6a2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:10:04.915 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:04.914+0000 7f4ac4c33040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:10:04.932 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:04.930+0000 7feeb6a2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:10:04.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.953+0000 7f4836c36640 1 Processor -- start 2023-12-01T03:10:04.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.953+0000 7f4836c36640 1 -- start start 2023-12-01T03:10:04.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.954+0000 7f4836c36640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f483015cf70 0x7f48300b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.954+0000 7f4836c36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48300b18f0 0x7f48300b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f4836c36640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48300b7760 con 0x7f48300cfb50 2023-12-01T03:10:04.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f4836c36640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48300b2170 con 0x7f483015cf70 2023-12-01T03:10:04.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f4836c36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48300b8b70 con 0x7f48300b18f0 2023-12-01T03:10:04.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f48349d2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f483015cf70 0x7f48300b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f48349d2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f483015cf70 0x7f48300b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35444/0 (socket says 172.21.15.115:35444) 2023-12-01T03:10:04.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f48349d2640 1 -- 172.21.15.115:0/1723990481 learned_addr learned my addr 172.21.15.115:0/1723990481 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:04.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f4827fff640 1 --2- 172.21.15.115:0/1723990481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48300b18f0 0x7f48300b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f48349d2640 1 -- 172.21.15.115:0/1723990481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48300b18f0 msgr2=0x7f48300b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f48349d2640 1 --2- 172.21.15.115:0/1723990481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48300b18f0 0x7f48300b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f48349d2640 1 -- 172.21.15.115:0/1723990481 >> v1:172.21.15.142:6789/0 conn(0x7f48300cfb50 legacy=0x7f48300d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.955+0000 7f48349d2640 1 -- 172.21.15.115:0/1723990481 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48300b6e30 con 0x7f483015cf70 2023-12-01T03:10:04.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.956+0000 7f48349d2640 1 --2- 172.21.15.115:0/1723990481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f483015cf70 0x7f48300b8470 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f482c009870 tx=0x7f482c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ae25dfd5cde38 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.957+0000 7f48277fe640 1 -- 172.21.15.115:0/1723990481 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c030840 con 0x7f483015cf70 2023-12-01T03:10:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.957+0000 7f48277fe640 1 -- 172.21.15.115:0/1723990481 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c0309a0 con 0x7f483015cf70 2023-12-01T03:10:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.957+0000 7f48277fe640 1 -- 172.21.15.115:0/1723990481 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c030d30 con 0x7f483015cf70 2023-12-01T03:10:04.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.957+0000 7f4836c36640 1 -- 172.21.15.115:0/1723990481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f483015cf70 msgr2=0x7f48300b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.957+0000 7f4836c36640 1 --2- 172.21.15.115:0/1723990481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f483015cf70 0x7f48300b8470 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f482c009870 tx=0x7f482c02b980 comp rx=0 tx=0).stop 2023-12-01T03:10:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.958+0000 7f4836c36640 1 -- 172.21.15.115:0/1723990481 shutdown_connections 2023-12-01T03:10:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.958+0000 7f4836c36640 1 --2- 172.21.15.115:0/1723990481 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48300b18f0 0x7f48300b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.958+0000 7f4836c36640 1 --2- 172.21.15.115:0/1723990481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f483015cf70 0x7f48300b8470 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.958+0000 7f4836c36640 1 -- 172.21.15.115:0/1723990481 >> 172.21.15.115:0/1723990481 conn(0x7f483006a0e0 msgr2=0x7f483006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.958+0000 7f4836c36640 1 -- 172.21.15.115:0/1723990481 shutdown_connections 2023-12-01T03:10:04.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.958+0000 7f4836c36640 1 -- 172.21.15.115:0/1723990481 wait complete. 2023-12-01T03:10:04.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.960+0000 7f4836c36640 1 Processor -- start 2023-12-01T03:10:04.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.960+0000 7f4836c36640 1 -- start start 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.961+0000 7f4836c36640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48300b18f0 0x7f483014a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.961+0000 7f4836c36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48300cfb50 0x7f48300ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.961+0000 7f4836c36640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483015cf70 0x7f48300b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.961+0000 7f4836c36640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483014a760 con 0x7f48300cfb50 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.961+0000 7f4836c36640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4830150aa0 con 0x7f48300b18f0 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.961+0000 7f4836c36640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4830151620 con 0x7f483015cf70 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.961+0000 7f4827fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48300cfb50 0x7f48300ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.961+0000 7f4827fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48300cfb50 0x7f48300ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35446/0 (socket says 172.21.15.115:35446) 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.961+0000 7f4827fff640 1 -- 172.21.15.115:0/2794138413 learned_addr learned my addr 172.21.15.115:0/2794138413 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.961+0000 7f48349d2640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48300b18f0 0x7f483014a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.962+0000 7f48351d3640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483015cf70 0x7f48300b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.962+0000 7f4827fff640 1 -- 172.21.15.115:0/2794138413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483015cf70 msgr2=0x7f48300b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.962+0000 7f4827fff640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483015cf70 0x7f48300b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.962+0000 7f4827fff640 1 -- 172.21.15.115:0/2794138413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48300b18f0 msgr2=0x7f483014a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:04.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.962+0000 7f4827fff640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48300b18f0 0x7f483014a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:04.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.962+0000 7f4827fff640 1 -- 172.21.15.115:0/2794138413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0094d0 con 0x7f48300cfb50 2023-12-01T03:10:04.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.962+0000 7f4827fff640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48300cfb50 0x7f48300ca6d0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f482000c820 tx=0x7f482000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:04.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.962+0000 7f48257fa640 1 -- 172.21.15.115:0/2794138413 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820004220 con 0x7f48300cfb50 2023-12-01T03:10:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.962+0000 7f48257fa640 1 -- 172.21.15.115:0/2794138413 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48200043a0 con 0x7f48300cfb50 2023-12-01T03:10:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.962+0000 7f4836c36640 1 -- 172.21.15.115:0/2794138413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4830150cf0 con 0x7f48300cfb50 2023-12-01T03:10:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.963+0000 7f48257fa640 1 -- 172.21.15.115:0/2794138413 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820007a70 con 0x7f48300cfb50 2023-12-01T03:10:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.963+0000 7f4836c36640 1 -- 172.21.15.115:0/2794138413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48301512b0 con 0x7f48300cfb50 2023-12-01T03:10:04.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.964+0000 7f4836c36640 1 -- 172.21.15.115:0/2794138413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483004e890 con 0x7f48300cfb50 2023-12-01T03:10:04.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.966+0000 7f48257fa640 1 -- 172.21.15.115:0/2794138413 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 120) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f4820007bd0 con 0x7f48300cfb50 2023-12-01T03:10:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.967+0000 7f48257fa640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f481c06a6a0 0x7f481c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.967+0000 7f48257fa640 1 -- 172.21.15.115:0/2794138413 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f4820088db0 con 0x7f48300cfb50 2023-12-01T03:10:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.967+0000 7f48349d2640 1 -- 172.21.15.115:0/2794138413 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f481c06a6a0 msgr2=0x7f481c06cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:04.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.967+0000 7f48349d2640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f481c06a6a0 0x7f481c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:04.973 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:04.972+0000 7f4ac4c33040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:10:04.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:04.972+0000 7f48257fa640 1 -- 172.21.15.115:0/2794138413 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4820053e00 con 0x7f48300cfb50 2023-12-01T03:10:04.987 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:04.985+0000 7feeb6a2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:10:05.023 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:05.022+0000 7f4ac4c33040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:10:05.046 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:05.044+0000 7feeb6a2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:10:05.098 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:05.096+0000 7feeb6a2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:10:05.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.160+0000 7f4836c36640 1 -- 172.21.15.115:0/2794138413 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4830151760 con 0x7f48300cfb50 2023-12-01T03:10:05.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.161+0000 7f48257fa640 1 -- 172.21.15.115:0/2794138413 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v199) v1 ==== 104+0+11 (secure 0 0 0) 0x7f482001e0a0 con 0x7f48300cfb50 2023-12-01T03:10:05.164 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:10:05.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 -- 172.21.15.115:0/2794138413 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f481c06a6a0 msgr2=0x7f481c06cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:05.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f481c06a6a0 0x7f481c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 -- 172.21.15.115:0/2794138413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48300cfb50 msgr2=0x7f48300ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:05.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48300cfb50 0x7f48300ca6d0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f482000c820 tx=0x7f482000cce0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 -- 172.21.15.115:0/2794138413 shutdown_connections 2023-12-01T03:10:05.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f481c06a6a0 0x7f481c06cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f483015cf70 0x7f48300b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f48300cfb50 0x7f48300ca6d0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 --2- 172.21.15.115:0/2794138413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48300b18f0 0x7f483014a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 -- 172.21.15.115:0/2794138413 >> 172.21.15.115:0/2794138413 conn(0x7f483006a0e0 msgr2=0x7f4830152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:05.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 -- 172.21.15.115:0/2794138413 shutdown_connections 2023-12-01T03:10:05.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.164+0000 7f4836c36640 1 -- 172.21.15.115:0/2794138413 wait complete. 2023-12-01T03:10:05.184 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:10:05.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.379+0000 7f266759e640 1 Processor -- start 2023-12-01T03:10:05.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.380+0000 7f266759e640 1 -- start start 2023-12-01T03:10:05.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.381+0000 7f266759e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f26680cfb80 0x7f26680ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.381+0000 7f266759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2668154010 0x7f2668160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.381+0000 7f266759e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26680cdcd0 con 0x7f2668153c30 2023-12-01T03:10:05.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.381+0000 7f266759e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2668161830 con 0x7f26680cfb80 2023-12-01T03:10:05.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.381+0000 7f266759e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26680cd3a0 con 0x7f2668154010 2023-12-01T03:10:05.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.381+0000 7f265ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f26680cfb80 0x7f26680ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:05.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.381+0000 7f265ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f26680cfb80 0x7f26680ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35448/0 (socket says 172.21.15.115:35448) 2023-12-01T03:10:05.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.381+0000 7f265ffff640 1 -- 172.21.15.115:0/1390729825 learned_addr learned my addr 172.21.15.115:0/1390729825 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:05.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.381+0000 7f266659c640 1 --1- 172.21.15.115:0/1390729825 >> v1:172.21.15.115:6789/0 conn(0x7f2668153c30 0x7f26680cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52574/0 (socket says 172.21.15.115:52574) 2023-12-01T03:10:05.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.382+0000 7f2666d9d640 1 --2- 172.21.15.115:0/1390729825 >> v2:172.21.15.142:3300/0 conn(0x7f2668154010 0x7f2668160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:05.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.382+0000 7f265ffff640 1 -- 172.21.15.115:0/1390729825 >> v2:172.21.15.142:3300/0 conn(0x7f2668154010 msgr2=0x7f2668160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:05.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.382+0000 7f265ffff640 1 --2- 172.21.15.115:0/1390729825 >> v2:172.21.15.142:3300/0 conn(0x7f2668154010 0x7f2668160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.382+0000 7f265ffff640 1 -- 172.21.15.115:0/1390729825 >> v1:172.21.15.115:6789/0 conn(0x7f2668153c30 legacy=0x7f26680cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:05.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.382+0000 7f265ffff640 1 -- 172.21.15.115:0/1390729825 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26680cd680 con 0x7f26680cfb80 2023-12-01T03:10:05.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.383+0000 7f265ffff640 1 --2- 172.21.15.115:0/1390729825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26680cfb80 0x7f26680ce160 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f266815af90 tx=0x7f265002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb301ce15ab31e98 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:05.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.383+0000 7f2665d9b640 1 -- 172.21.15.115:0/1390729825 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650030860 con 0x7f26680cfb80 2023-12-01T03:10:05.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.383+0000 7f2665d9b640 1 -- 172.21.15.115:0/1390729825 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26500309c0 con 0x7f26680cfb80 2023-12-01T03:10:05.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.383+0000 7f2665d9b640 1 -- 172.21.15.115:0/1390729825 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650030d50 con 0x7f26680cfb80 2023-12-01T03:10:05.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.384+0000 7f266759e640 1 -- 172.21.15.115:0/1390729825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26680cfb80 msgr2=0x7f26680ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:05.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.384+0000 7f266759e640 1 --2- 172.21.15.115:0/1390729825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26680cfb80 0x7f26680ce160 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f266815af90 tx=0x7f265002b9a0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.384+0000 7f266759e640 1 -- 172.21.15.115:0/1390729825 shutdown_connections 2023-12-01T03:10:05.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.384+0000 7f266759e640 1 --2- 172.21.15.115:0/1390729825 >> v2:172.21.15.142:3300/0 conn(0x7f2668154010 0x7f2668160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.384+0000 7f266759e640 1 --2- 172.21.15.115:0/1390729825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26680cfb80 0x7f26680ce160 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.384+0000 7f266759e640 1 -- 172.21.15.115:0/1390729825 >> 172.21.15.115:0/1390729825 conn(0x7f266806a0b0 msgr2=0x7f266806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:05.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.384+0000 7f266759e640 1 -- 172.21.15.115:0/1390729825 shutdown_connections 2023-12-01T03:10:05.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.384+0000 7f266759e640 1 -- 172.21.15.115:0/1390729825 wait complete. 2023-12-01T03:10:05.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.386+0000 7f266759e640 1 Processor -- start 2023-12-01T03:10:05.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.386+0000 7f266759e640 1 -- start start 2023-12-01T03:10:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.387+0000 7f266759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26680cfb80 0x7f2668155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.387+0000 7f266759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2668153c30 0x7f266814fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.387+0000 7f266759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2668154010 0x7f2668158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.387+0000 7f266759e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266814f890 con 0x7f2668154010 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.387+0000 7f266759e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266814e5e0 con 0x7f2668153c30 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.387+0000 7f266759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266814ec80 con 0x7f26680cfb80 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.387+0000 7f2666d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2668154010 0x7f2668158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.387+0000 7f2666d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2668154010 0x7f2668158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35462/0 (socket says 172.21.15.115:35462) 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.387+0000 7f2666d9d640 1 -- 172.21.15.115:0/464135479 learned_addr learned my addr 172.21.15.115:0/464135479 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.387+0000 7f265ffff640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2668153c30 0x7f266814fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f266659c640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26680cfb80 0x7f2668155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f2666d9d640 1 -- 172.21.15.115:0/464135479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26680cfb80 msgr2=0x7f2668155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:05.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f2666d9d640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26680cfb80 0x7f2668155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f2666d9d640 1 -- 172.21.15.115:0/464135479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2668153c30 msgr2=0x7f266814fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f2666d9d640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2668153c30 0x7f266814fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f2666d9d640 1 -- 172.21.15.115:0/464135479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26500094d0 con 0x7f2668154010 2023-12-01T03:10:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f2666d9d640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2668154010 0x7f2668158ed0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f266000cc60 tx=0x7f266000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:05.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f265effd640 1 -- 172.21.15.115:0/464135479 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266000ce70 con 0x7f2668154010 2023-12-01T03:10:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f265effd640 1 -- 172.21.15.115:0/464135479 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2660003990 con 0x7f2668154010 2023-12-01T03:10:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f266759e640 1 -- 172.21.15.115:0/464135479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266814ef60 con 0x7f2668154010 2023-12-01T03:10:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f265effd640 1 -- 172.21.15.115:0/464135479 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660003d20 con 0x7f2668154010 2023-12-01T03:10:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.388+0000 7f266759e640 1 -- 172.21.15.115:0/464135479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266814f520 con 0x7f2668154010 2023-12-01T03:10:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.390+0000 7f266759e640 1 -- 172.21.15.115:0/464135479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266814fa40 con 0x7f2668154010 2023-12-01T03:10:05.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.391+0000 7f265effd640 1 -- 172.21.15.115:0/464135479 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 120) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f2660003e80 con 0x7f2668154010 2023-12-01T03:10:05.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.392+0000 7f265effd640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f264806a6a0 0x7f264806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.392+0000 7f265effd640 1 -- 172.21.15.115:0/464135479 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f2660089380 con 0x7f2668154010 2023-12-01T03:10:05.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.392+0000 7f266659c640 1 -- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f264806a6a0 msgr2=0x7f264806cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:05.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.392+0000 7f266659c640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f264806a6a0 0x7f264806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:05.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.396+0000 7f265effd640 1 -- 172.21.15.115:0/464135479 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f266814fa40 con 0x7f2668154010 2023-12-01T03:10:05.407 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:05.406+0000 7f4ac4c33040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:10:05.498 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:05.496+0000 7feeb6a2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:10:05.568 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:05.567+0000 7f4ac4c33040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:10:05.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.590+0000 7f266759e640 1 -- 172.21.15.115:0/464135479 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f266814fa40 con 0x7f2668154010 2023-12-01T03:10:05.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.590+0000 7f265effd640 1 -- 172.21.15.115:0/464135479 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v199) v1 ==== 100+0+11 (secure 0 0 0) 0x7f266814fa40 con 0x7f2668154010 2023-12-01T03:10:05.592 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:10:05.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266659c640 1 -- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f264806a6a0 msgr2=0x7f264806cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:05.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266659c640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f264806a6a0 0x7f264806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:10:05.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266759e640 1 -- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f264806a6a0 msgr2=0x7f264806cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:05.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266759e640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f264806a6a0 0x7f264806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266759e640 1 -- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2668154010 msgr2=0x7f2668158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:05.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266759e640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2668154010 0x7f2668158ed0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f266000cc60 tx=0x7f266000a5c0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266759e640 1 -- 172.21.15.115:0/464135479 shutdown_connections 2023-12-01T03:10:05.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266759e640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f264806a6a0 0x7f264806cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266759e640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2668154010 0x7f2668158ed0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266759e640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2668153c30 0x7f266814fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266759e640 1 --2- 172.21.15.115:0/464135479 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26680cfb80 0x7f2668155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.593+0000 7f266759e640 1 -- 172.21.15.115:0/464135479 >> 172.21.15.115:0/464135479 conn(0x7f266806a0b0 msgr2=0x7f266815c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:05.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.594+0000 7f266759e640 1 -- 172.21.15.115:0/464135479 shutdown_connections 2023-12-01T03:10:05.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.594+0000 7f266759e640 1 -- 172.21.15.115:0/464135479 wait complete. 2023-12-01T03:10:05.615 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T03:10:05.666 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:05.664+0000 7feeb6a2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:10:05.675 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:05.674+0000 7f4ac4c33040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:10:05.725 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:05.724+0000 7f4ac4c33040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:10:05.780 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:05.778+0000 7feeb6a2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:10:05.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.815+0000 7f6ca6a42640 1 Processor -- start 2023-12-01T03:10:05.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.815+0000 7f6ca6a42640 1 -- start start 2023-12-01T03:10:05.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.816+0000 7f6ca6a42640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.817+0000 7f6ca6a42640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6ca00cfb70 0x7f6ca00b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.817+0000 7f6ca6a42640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca0160b20 con 0x7f6ca015dfd0 2023-12-01T03:10:05.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.817+0000 7f6ca6a42640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca0160dc0 con 0x7f6ca00cfb70 2023-12-01T03:10:05.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.817+0000 7f6ca6a42640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01610a0 con 0x7f6ca015dbf0 2023-12-01T03:10:05.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.817+0000 7f6ca4fdf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6ca00cfb70 0x7f6ca00b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.817+0000 7f6c9ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.817+0000 7f6c9ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:46702/0 (socket says 172.21.15.115:46702) 2023-12-01T03:10:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.817+0000 7f6ca4fdf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6ca00cfb70 0x7f6ca00b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:46116/0 (socket says 172.21.15.115:46116) 2023-12-01T03:10:05.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.817+0000 7f6c9ffff640 1 -- 172.21.15.115:0/1703893669 learned_addr learned my addr 172.21.15.115:0/1703893669 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:05.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.818+0000 7f6c9ffff640 1 -- 172.21.15.115:0/1703893669 >> v2:172.21.15.142:3300/0 conn(0x7f6ca00cfb70 msgr2=0x7f6ca00b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:05.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.818+0000 7f6c9ffff640 1 --2- 172.21.15.115:0/1703893669 >> v2:172.21.15.142:3300/0 conn(0x7f6ca00cfb70 0x7f6ca00b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.818+0000 7f6c9ffff640 1 -- 172.21.15.115:0/1703893669 >> v1:172.21.15.142:6789/0 conn(0x7f6ca015dfd0 legacy=0x7f6ca00cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.818+0000 7f6c9ffff640 1 -- 172.21.15.115:0/1703893669 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca0161380 con 0x7f6ca015dbf0 2023-12-01T03:10:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.818+0000 7f6ca4fdf640 1 --2- 172.21.15.115:0/1703893669 >> v2:172.21.15.142:3300/0 conn(0x7f6ca00cfb70 0x7f6ca00b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:10:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.819+0000 7f6c9ffff640 1 --2- 172.21.15.115:0/1703893669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00cdc30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009870 tx=0x7f6c9002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a912ec4f8e87c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.820+0000 7f6c9effd640 1 -- 172.21.15.115:0/1703893669 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90031640 con 0x7f6ca015dbf0 2023-12-01T03:10:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.820+0000 7f6c9effd640 1 -- 172.21.15.115:0/1703893669 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c900317a0 con 0x7f6ca015dbf0 2023-12-01T03:10:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.820+0000 7f6c9effd640 1 -- 172.21.15.115:0/1703893669 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90031b30 con 0x7f6ca015dbf0 2023-12-01T03:10:05.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.820+0000 7f6ca6a42640 1 -- 172.21.15.115:0/1703893669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 msgr2=0x7f6ca00cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:05.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.820+0000 7f6ca6a42640 1 --2- 172.21.15.115:0/1703893669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00cdc30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009870 tx=0x7f6c9002b8f0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.821+0000 7f6ca6a42640 1 -- 172.21.15.115:0/1703893669 shutdown_connections 2023-12-01T03:10:05.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.821+0000 7f6ca6a42640 1 --2- 172.21.15.115:0/1703893669 >> v2:172.21.15.142:3300/0 conn(0x7f6ca00cfb70 0x7f6ca00b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.821+0000 7f6ca6a42640 1 --2- 172.21.15.115:0/1703893669 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00cdc30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.821+0000 7f6ca6a42640 1 -- 172.21.15.115:0/1703893669 >> 172.21.15.115:0/1703893669 conn(0x7f6ca0065960 msgr2=0x7f6ca0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.821+0000 7f6ca6a42640 1 -- 172.21.15.115:0/1703893669 shutdown_connections 2023-12-01T03:10:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.821+0000 7f6ca6a42640 1 -- 172.21.15.115:0/1703893669 wait complete. 2023-12-01T03:10:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.823+0000 7f6ca6a42640 1 Processor -- start 2023-12-01T03:10:05.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.823+0000 7f6ca6a42640 1 -- start start 2023-12-01T03:10:05.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.824+0000 7f6ca6a42640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca00cfb70 0x7f6ca0157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.824+0000 7f6ca6a42640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00d1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.824+0000 7f6ca6a42640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ca015dfd0 0x7f6ca00cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.824+0000 7f6ca6a42640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00c9110 con 0x7f6ca015dfd0 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.824+0000 7f6ca6a42640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0158380 con 0x7f6ca00cfb70 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.824+0000 7f6ca6a42640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca00d1ad0 con 0x7f6ca015dbf0 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.824+0000 7f6ca4fdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ca015dfd0 0x7f6ca00cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.824+0000 7f6ca4fdf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ca015dfd0 0x7f6ca00cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35472/0 (socket says 172.21.15.115:35472) 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.824+0000 7f6ca4fdf640 1 -- 172.21.15.115:0/2040913484 learned_addr learned my addr 172.21.15.115:0/2040913484 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6c9f7fe640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6c9ffff640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca00cfb70 0x7f6ca0157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6ca4fdf640 1 -- 172.21.15.115:0/2040913484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 msgr2=0x7f6ca00d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6ca4fdf640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6ca4fdf640 1 -- 172.21.15.115:0/2040913484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca00cfb70 msgr2=0x7f6ca0157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6ca4fdf640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca00cfb70 0x7f6ca0157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:05.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6ca4fdf640 1 -- 172.21.15.115:0/2040913484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c98009a70 con 0x7f6ca015dfd0 2023-12-01T03:10:05.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6c9ffff640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca00cfb70 0x7f6ca0157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:05.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6c9f7fe640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:05.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6ca4fdf640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ca015dfd0 0x7f6ca00cc840 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009e60 tx=0x7f6c9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:05.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6c9cff9640 1 -- 172.21.15.115:0/2040913484 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98009cd0 con 0x7f6ca015dfd0 2023-12-01T03:10:05.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6c9cff9640 1 -- 172.21.15.115:0/2040913484 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9800f040 con 0x7f6ca015dfd0 2023-12-01T03:10:05.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6c9cff9640 1 -- 172.21.15.115:0/2040913484 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98003990 con 0x7f6ca015dfd0 2023-12-01T03:10:05.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.825+0000 7f6ca6a42640 1 -- 172.21.15.115:0/2040913484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c900094d0 con 0x7f6ca015dfd0 2023-12-01T03:10:05.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.826+0000 7f6ca6a42640 1 -- 172.21.15.115:0/2040913484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca01f3980 con 0x7f6ca015dfd0 2023-12-01T03:10:05.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.827+0000 7f6ca6a42640 1 -- 172.21.15.115:0/2040913484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca01f3b10 con 0x7f6ca015dfd0 2023-12-01T03:10:05.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.828+0000 7f6c9cff9640 1 -- 172.21.15.115:0/2040913484 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 120) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f6c98014480 con 0x7f6ca015dfd0 2023-12-01T03:10:05.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.829+0000 7f6c9cff9640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6c8c06a670 0x7f6c8c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:05.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.829+0000 7f6c9ffff640 1 -- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6c8c06a670 msgr2=0x7f6c8c06cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:05.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.829+0000 7f6c9cff9640 1 -- 172.21.15.115:0/2040913484 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f6c98089c30 con 0x7f6ca015dfd0 2023-12-01T03:10:05.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.829+0000 7f6c9ffff640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6c8c06a670 0x7f6c8c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:05.833 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:05.832+0000 7f4ac4c33040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:10:05.833 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:05.832+0000 7feeb6a2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:10:05.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:05.833+0000 7f6c9cff9640 1 -- 172.21.15.115:0/2040913484 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6c98054b50 con 0x7f6ca015dfd0 2023-12-01T03:10:05.940 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:05.939+0000 7f4ac4c33040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:10:05.945 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:05.943+0000 7feeb6a2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:10:06.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.030+0000 7f6c9ffff640 1 -- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6c8c06a670 msgr2=0x7f6c8c06cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:06.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.030+0000 7f6c9ffff640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6c8c06a670 0x7f6c8c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:10:06.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.037+0000 7f6ca6a42640 1 -- 172.21.15.115:0/2040913484 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6ca004e890 con 0x7f6ca015dfd0 2023-12-01T03:10:06.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.037+0000 7f6c9cff9640 1 -- 172.21.15.115:0/2040913484 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v199) v1 ==== 80+0+469 (secure 0 0 0) 0x7f6c98018080 con 0x7f6ca015dfd0 2023-12-01T03:10:06.039 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/361869044 2023-12-02T03:08:56.516548+0000 2023-12-01T03:10:06.039 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/4037780703 2023-12-02T03:08:56.516548+0000 2023-12-01T03:10:06.039 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/138932 2023-12-02T03:08:56.516548+0000 2023-12-01T03:10:06.039 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/138932 2023-12-02T03:08:56.516548+0000 2023-12-01T03:10:06.039 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3414835041 2023-12-02T03:08:56.516548+0000 2023-12-01T03:10:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1495322174 2023-12-02T03:08:56.516548+0000 2023-12-01T03:10:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2764969208 2023-12-02T03:08:56.516548+0000 2023-12-01T03:10:06.040 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1016167991 2023-12-02T03:08:56.516548+0000 2023-12-01T03:10:06.040 INFO:teuthology.orchestra.run.smithi115.stderr:listed 8 entries 2023-12-01T03:10:06.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 -- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6c8c06a670 msgr2=0x7f6c8c06cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:06.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6c8c06a670 0x7f6c8c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 -- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ca015dfd0 msgr2=0x7f6ca00cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:06.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ca015dfd0 0x7f6ca00cc840 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009e60 tx=0x7f6c9800c490 comp rx=0 tx=0).stop 2023-12-01T03:10:06.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 -- 172.21.15.115:0/2040913484 shutdown_connections 2023-12-01T03:10:06.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6c8c06a670 0x7f6c8c06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ca015dbf0 0x7f6ca00d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ca015dfd0 0x7f6ca00cc840 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 --2- 172.21.15.115:0/2040913484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ca00cfb70 0x7f6ca0157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 -- 172.21.15.115:0/2040913484 >> 172.21.15.115:0/2040913484 conn(0x7f6ca0065960 msgr2=0x7f6ca00b6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 -- 172.21.15.115:0/2040913484 shutdown_connections 2023-12-01T03:10:06.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.040+0000 7f6ca6a42640 1 -- 172.21.15.115:0/2040913484 wait complete. 2023-12-01T03:10:06.058 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:06.056+0000 7feeb6a2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:10:06.060 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T03:10:06.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:06.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T03:10:06.099 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:06.095+0000 7f4ac4c33040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:10:06.145 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:06.144+0000 7f4ac4c33040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:10:06.164 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T03:10:06.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:06.164 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T03:10:06.215 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:10:06.216 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:10:06.232 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:06.230+0000 7feeb6a2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:10:06.269 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:06.268+0000 7f4ac4c33040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:10:06.273 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:10:06.274 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:10:06.274 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:10:06.284 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:06.280+0000 7feeb6a2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:10:06.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:10:06.328 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:10:06.329 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:10:06.384 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:10:06.384 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:10:06.405 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:06.403+0000 7feeb6a2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:10:06.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.590+0000 7f84c17fb640 1 Processor -- start 2023-12-01T03:10:06.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.591+0000 7f84c17fb640 1 -- start start 2023-12-01T03:10:06.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.592+0000 7f84c17fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f84bc15cd20 0x7f84bc0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:06.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.592+0000 7f84c17fb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84bc0ce2e0 con 0x7f84bc0d1230 2023-12-01T03:10:06.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.592+0000 7f84c17fb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84bc0ce000 con 0x7f84bc0b5d80 2023-12-01T03:10:06.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.592+0000 7f84c17fb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84bc0b2160 con 0x7f84bc15cd20 2023-12-01T03:10:06.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.592+0000 7f84baffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f84bc15cd20 0x7f84bc0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:06.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.593+0000 7f84baffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f84bc15cd20 0x7f84bc0d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35476/0 (socket says 172.21.15.115:35476) 2023-12-01T03:10:06.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.593+0000 7f84baffd640 1 -- 172.21.15.115:0/1041924913 learned_addr learned my addr 172.21.15.115:0/1041924913 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:06.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.593+0000 7f84baffd640 1 -- 172.21.15.115:0/1041924913 >> v1:172.21.15.142:6789/0 conn(0x7f84bc0b5d80 legacy=0x7f84bc0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:06.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.593+0000 7f84baffd640 1 -- 172.21.15.115:0/1041924913 >> v1:172.21.15.115:6789/0 conn(0x7f84bc0d1230 legacy=0x7f84bc0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:06.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.593+0000 7f84baffd640 1 -- 172.21.15.115:0/1041924913 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc0cd3d0 con 0x7f84bc15cd20 2023-12-01T03:10:06.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.594+0000 7f84baffd640 1 --2- 172.21.15.115:0/1041924913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 0x7f84bc0d24d0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009870 tx=0x7f84ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c220ed1f76debf93 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:06.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.594+0000 7f84ba7fc640 1 -- 172.21.15.115:0/1041924913 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac031870 con 0x7f84bc15cd20 2023-12-01T03:10:06.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.594+0000 7f84ba7fc640 1 -- 172.21.15.115:0/1041924913 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac0319d0 con 0x7f84bc15cd20 2023-12-01T03:10:06.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.594+0000 7f84ba7fc640 1 -- 172.21.15.115:0/1041924913 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac031d60 con 0x7f84bc15cd20 2023-12-01T03:10:06.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.595+0000 7f84c17fb640 1 -- 172.21.15.115:0/1041924913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 msgr2=0x7f84bc0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:06.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.595+0000 7f84c17fb640 1 --2- 172.21.15.115:0/1041924913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 0x7f84bc0d24d0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009870 tx=0x7f84ac02b910 comp rx=0 tx=0).stop 2023-12-01T03:10:06.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.595+0000 7f84c17fb640 1 -- 172.21.15.115:0/1041924913 shutdown_connections 2023-12-01T03:10:06.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.595+0000 7f84c17fb640 1 --2- 172.21.15.115:0/1041924913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 0x7f84bc0d24d0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.595+0000 7f84c17fb640 1 -- 172.21.15.115:0/1041924913 >> 172.21.15.115:0/1041924913 conn(0x7f84bc06a0e0 msgr2=0x7f84bc06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:06.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.595+0000 7f84c17fb640 1 -- 172.21.15.115:0/1041924913 shutdown_connections 2023-12-01T03:10:06.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.595+0000 7f84c17fb640 1 -- 172.21.15.115:0/1041924913 wait complete. 2023-12-01T03:10:06.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.597+0000 7f84c17fb640 1 Processor -- start 2023-12-01T03:10:06.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.597+0000 7f84c17fb640 1 -- start start 2023-12-01T03:10:06.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84c17fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc0b5d80 0x7f84bc1e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:06.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84c17fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84bc0d1230 0x7f84bc0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:06.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84c17fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 0x7f84bc0c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:06.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84c17fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc0c8260 con 0x7f84bc15cd20 2023-12-01T03:10:06.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84c17fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc1ea330 con 0x7f84bc0b5d80 2023-12-01T03:10:06.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84c17fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc0c7300 con 0x7f84bc0d1230 2023-12-01T03:10:06.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 0x7f84bc0c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 0x7f84bc0c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35490/0 (socket says 172.21.15.115:35490) 2023-12-01T03:10:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84bb7fe640 1 -- 172.21.15.115:0/1308573950 learned_addr learned my addr 172.21.15.115:0/1308573950 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84baffd640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc0b5d80 0x7f84bc1e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84b3fff640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84bc0d1230 0x7f84bc0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.598+0000 7f84bb7fe640 1 -- 172.21.15.115:0/1308573950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84bc0d1230 msgr2=0x7f84bc0b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84bb7fe640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84bc0d1230 0x7f84bc0b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84bb7fe640 1 -- 172.21.15.115:0/1308573950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc0b5d80 msgr2=0x7f84bc1e9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84bb7fe640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc0b5d80 0x7f84bc1e9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84bb7fe640 1 -- 172.21.15.115:0/1308573950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac0094d0 con 0x7f84bc15cd20 2023-12-01T03:10:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84b3fff640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84bc0d1230 0x7f84bc0b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84baffd640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc0b5d80 0x7f84bc1e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:10:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84bb7fe640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 0x7f84bc0c8680 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f84b400c7a0 tx=0x7f84b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84b37fe640 1 -- 172.21.15.115:0/1308573950 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4003b70 con 0x7f84bc15cd20 2023-12-01T03:10:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84b37fe640 1 -- 172.21.15.115:0/1308573950 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b4003cf0 con 0x7f84bc15cd20 2023-12-01T03:10:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84b37fe640 1 -- 172.21.15.115:0/1308573950 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b400aa90 con 0x7f84bc15cd20 2023-12-01T03:10:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84c17fb640 1 -- 172.21.15.115:0/1308573950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc0c75e0 con 0x7f84bc15cd20 2023-12-01T03:10:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.599+0000 7f84c17fb640 1 -- 172.21.15.115:0/1308573950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc0c7ba0 con 0x7f84bc15cd20 2023-12-01T03:10:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.600+0000 7f84c17fb640 1 -- 172.21.15.115:0/1308573950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc0c83a0 con 0x7f84bc15cd20 2023-12-01T03:10:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.601+0000 7f84b37fe640 1 -- 172.21.15.115:0/1308573950 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 120) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f84b400abf0 con 0x7f84bc15cd20 2023-12-01T03:10:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.603+0000 7f84b37fe640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f849c06a670 0x7f849c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.603+0000 7f84baffd640 1 -- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f849c06a670 msgr2=0x7f849c06cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.603+0000 7f84b37fe640 1 -- 172.21.15.115:0/1308573950 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f84b4088df0 con 0x7f84bc15cd20 2023-12-01T03:10:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.603+0000 7f84baffd640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f849c06a670 0x7f849c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:06.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.606+0000 7f84b37fe640 1 -- 172.21.15.115:0/1308573950 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f84bc0c83a0 con 0x7f84bc15cd20 2023-12-01T03:10:06.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.804+0000 7f84baffd640 1 -- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f849c06a670 msgr2=0x7f849c06cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:06.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.804+0000 7f84baffd640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f849c06a670 0x7f849c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:10:06.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.833+0000 7f84c17fb640 1 -- 172.21.15.115:0/1308573950 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84bc0c83a0 con 0x7f84bc15cd20 2023-12-01T03:10:06.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.835+0000 7f84b37fe640 1 -- 172.21.15.115:0/1308573950 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f84bc0c83a0 con 0x7f84bc15cd20 2023-12-01T03:10:06.837 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:06.837 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/974148243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":974148243},{"type":"v1","addr":"172.21.15.115:6837","nonce":974148243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/476969842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":476969842},{"type":"v1","addr":"172.21.15.142:6834","nonce":476969842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1103499235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1103499235},{"type":"v1","addr":"172.21.15.115:6835","nonce":1103499235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:08:43.078263+0000","modified":"2023-12-01T03:08:46.096728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10149":{"gid":10149,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/3801423208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3801423208},{"type":"v1","addr":"172.21.15.142:6835","nonce":3801423208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-01T03:10:06.837 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 87 2023-12-01T03:10:06.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.837+0000 7f84c17fb640 1 -- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f849c06a670 msgr2=0x7f849c06cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:06.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.837+0000 7f84c17fb640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f849c06a670 0x7f849c06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.837+0000 7f84c17fb640 1 -- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 msgr2=0x7f84bc0c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:06.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.837+0000 7f84c17fb640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 0x7f84bc0c8680 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f84b400c7a0 tx=0x7f84b400cc60 comp rx=0 tx=0).stop 2023-12-01T03:10:06.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.837+0000 7f84c17fb640 1 -- 172.21.15.115:0/1308573950 shutdown_connections 2023-12-01T03:10:06.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.837+0000 7f84c17fb640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f849c06a670 0x7f849c06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.837+0000 7f84c17fb640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84bc0d1230 0x7f84bc0b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.837+0000 7f84c17fb640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84bc15cd20 0x7f84bc0c8680 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.837+0000 7f84c17fb640 1 --2- 172.21.15.115:0/1308573950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84bc0b5d80 0x7f84bc1e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:06.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.837+0000 7f84c17fb640 1 -- 172.21.15.115:0/1308573950 >> 172.21.15.115:0/1308573950 conn(0x7f84bc06a0e0 msgr2=0x7f84bc1597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:06.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.838+0000 7f84c17fb640 1 -- 172.21.15.115:0/1308573950 shutdown_connections 2023-12-01T03:10:06.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:06.838+0000 7f84c17fb640 1 -- 172.21.15.115:0/1308573950 wait complete. 2023-12-01T03:10:06.859 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:07.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.090+0000 7f798a527640 1 Processor -- start 2023-12-01T03:10:07.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.090+0000 7f798a527640 1 -- start start 2023-12-01T03:10:07.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.092+0000 7f798a527640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 0x7f798415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.092+0000 7f798a527640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79840ca8f0 0x7f79840cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.092+0000 7f798a527640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798415dbb0 con 0x7f798415d600 2023-12-01T03:10:07.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.092+0000 7f798a527640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798415de90 con 0x7f79840ca8f0 2023-12-01T03:10:07.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.092+0000 7f798a527640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79840ca0f0 con 0x7f798415a8e0 2023-12-01T03:10:07.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.093+0000 7f7989525640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 0x7f798415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.093+0000 7f7989d26640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79840ca8f0 0x7f79840cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.093+0000 7f7989525640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 0x7f798415d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:46732/0 (socket says 172.21.15.115:46732) 2023-12-01T03:10:07.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.093+0000 7f7989d26640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79840ca8f0 0x7f79840cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:46136/0 (socket says 172.21.15.115:46136) 2023-12-01T03:10:07.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.093+0000 7f7989525640 1 -- 172.21.15.115:0/855502978 learned_addr learned my addr 172.21.15.115:0/855502978 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:07.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.093+0000 7f7989525640 1 -- 172.21.15.115:0/855502978 >> v2:172.21.15.142:3300/0 conn(0x7f79840ca8f0 msgr2=0x7f79840cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:07.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.093+0000 7f7989525640 1 --2- 172.21.15.115:0/855502978 >> v2:172.21.15.142:3300/0 conn(0x7f79840ca8f0 0x7f79840cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.093+0000 7f7989525640 1 -- 172.21.15.115:0/855502978 >> v1:172.21.15.142:6789/0 conn(0x7f798415d600 legacy=0x7f7984070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.093+0000 7f7989525640 1 -- 172.21.15.115:0/855502978 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79840ca3d0 con 0x7f798415a8e0 2023-12-01T03:10:07.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.095+0000 7f7989525640 1 --2- 172.21.15.115:0/855502978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 0x7f798415d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7980009870 tx=0x7f798002ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9076fa87a49a2165 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.095+0000 7f797bfff640 1 -- 172.21.15.115:0/855502978 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798002f840 con 0x7f798415a8e0 2023-12-01T03:10:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.095+0000 7f797bfff640 1 -- 172.21.15.115:0/855502978 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798002f9a0 con 0x7f798415a8e0 2023-12-01T03:10:07.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.095+0000 7f797bfff640 1 -- 172.21.15.115:0/855502978 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798002fd30 con 0x7f798415a8e0 2023-12-01T03:10:07.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.096+0000 7f798a527640 1 -- 172.21.15.115:0/855502978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 msgr2=0x7f798415d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.096+0000 7f798a527640 1 --2- 172.21.15.115:0/855502978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 0x7f798415d0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7980009870 tx=0x7f798002ba20 comp rx=0 tx=0).stop 2023-12-01T03:10:07.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.096+0000 7f798a527640 1 -- 172.21.15.115:0/855502978 shutdown_connections 2023-12-01T03:10:07.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.096+0000 7f798a527640 1 --2- 172.21.15.115:0/855502978 >> v2:172.21.15.142:3300/0 conn(0x7f79840ca8f0 0x7f79840cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.096+0000 7f798a527640 1 --2- 172.21.15.115:0/855502978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 0x7f798415d0c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.096+0000 7f798a527640 1 -- 172.21.15.115:0/855502978 >> 172.21.15.115:0/855502978 conn(0x7f798406a0e0 msgr2=0x7f798406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:07.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.096+0000 7f798a527640 1 -- 172.21.15.115:0/855502978 shutdown_connections 2023-12-01T03:10:07.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.096+0000 7f798a527640 1 -- 172.21.15.115:0/855502978 wait complete. 2023-12-01T03:10:07.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.098+0000 7f798a527640 1 Processor -- start 2023-12-01T03:10:07.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.098+0000 7f798a527640 1 -- start start 2023-12-01T03:10:07.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f798a527640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79840ca8f0 0x7f7984157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f798a527640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 0x7f7984147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f798a527640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f798415d600 0x7f7984159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f798a527640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79841585c0 con 0x7f798415d600 2023-12-01T03:10:07.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f798a527640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79841588a0 con 0x7f79840ca8f0 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f798a527640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7984158b80 con 0x7f798415a8e0 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f7989d26640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f798415d600 0x7f7984159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f7989d26640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f798415d600 0x7f7984159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35494/0 (socket says 172.21.15.115:35494) 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f7989d26640 1 -- 172.21.15.115:0/378538531 learned_addr learned my addr 172.21.15.115:0/378538531 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f7989525640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79840ca8f0 0x7f7984157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.099+0000 7f7988d24640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 0x7f7984147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f7989d26640 1 -- 172.21.15.115:0/378538531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 msgr2=0x7f7984147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f7989d26640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 0x7f7984147fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f7989d26640 1 -- 172.21.15.115:0/378538531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79840ca8f0 msgr2=0x7f7984157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f7989d26640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79840ca8f0 0x7f7984157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f7989d26640 1 -- 172.21.15.115:0/378538531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79800094d0 con 0x7f798415d600 2023-12-01T03:10:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f7989d26640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f798415d600 0x7f7984159660 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f797c00efc0 tx=0x7f797c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f7979ffb640 1 -- 172.21.15.115:0/378538531 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c009b10 con 0x7f798415d600 2023-12-01T03:10:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f7979ffb640 1 -- 172.21.15.115:0/378538531 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c00f040 con 0x7f798415d600 2023-12-01T03:10:07.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f7979ffb640 1 -- 172.21.15.115:0/378538531 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c0146c0 con 0x7f798415d600 2023-12-01T03:10:07.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f798a527640 1 -- 172.21.15.115:0/378538531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7984158e60 con 0x7f798415d600 2023-12-01T03:10:07.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.100+0000 7f798a527640 1 -- 172.21.15.115:0/378538531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7984158200 con 0x7f798415d600 2023-12-01T03:10:07.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.102+0000 7f798a527640 1 -- 172.21.15.115:0/378538531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79840722d0 con 0x7f798415d600 2023-12-01T03:10:07.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.102+0000 7f7979ffb640 1 -- 172.21.15.115:0/378538531 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 120) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f797c014820 con 0x7f798415d600 2023-12-01T03:10:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.103+0000 7f7979ffb640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f796806a670 0x7f796806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.103+0000 7f7979ffb640 1 -- 172.21.15.115:0/378538531 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f797c089bc0 con 0x7f798415d600 2023-12-01T03:10:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.103+0000 7f7989525640 1 -- 172.21.15.115:0/378538531 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f796806a670 msgr2=0x7f796806cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:07.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.103+0000 7f7989525640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f796806a670 0x7f796806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:07.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.107+0000 7f7979ffb640 1 -- 172.21.15.115:0/378538531 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f797c054af0 con 0x7f798415d600 2023-12-01T03:10:07.111 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:07.110+0000 7f4ac4c33040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:10:07.256 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.254+0000 7feeb6a2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:10:07.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.291+0000 7f798a527640 1 -- 172.21.15.115:0/378538531 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f798406ac50 con 0x7f798415d600 2023-12-01T03:10:07.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.295+0000 7f7979ffb640 1 -- 172.21.15.115:0/378538531 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+16827 (secure 0 0 0) 0x7f797c018070 con 0x7f798415d600 2023-12-01T03:10:07.297 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:07.297 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":199,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:09:58.429023+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:08:40.709195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-01T03:08:42.071749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.18","mappings":[{"from":5,"to":3}]},{"pgid":"23.19","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/361869044":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/4037780703":"2023-12-02T03:08:56.516548+0000","172.21.15.115:6824/138932":"2023-12-02T03:08:56.516548+0000","172.21.15.115:6822/138932":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/3414835041":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/1495322174":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/2764969208":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/1016167991":"2023-12-02T03:08:56.516548+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:07.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 -- 172.21.15.115:0/378538531 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f796806a670 msgr2=0x7f796806cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:07.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f796806a670 0x7f796806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 -- 172.21.15.115:0/378538531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f798415d600 msgr2=0x7f7984159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f798415d600 0x7f7984159660 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f797c00efc0 tx=0x7f797c00c490 comp rx=0 tx=0).stop 2023-12-01T03:10:07.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 -- 172.21.15.115:0/378538531 shutdown_connections 2023-12-01T03:10:07.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f796806a670 0x7f796806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f798415a8e0 0x7f7984147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f798415d600 0x7f7984159660 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 --2- 172.21.15.115:0/378538531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79840ca8f0 0x7f7984157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 -- 172.21.15.115:0/378538531 >> 172.21.15.115:0/378538531 conn(0x7f798406a0e0 msgr2=0x7f798415ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:07.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 -- 172.21.15.115:0/378538531 shutdown_connections 2023-12-01T03:10:07.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.297+0000 7f798a527640 1 -- 172.21.15.115:0/378538531 wait complete. 2023-12-01T03:10:07.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T03:08:40.709195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T03:08:42.071749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:10:07.318 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:10:07.406 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:07.405+0000 7f4ac4c33040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:10:07.464 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:07.463+0000 7f4ac4c33040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:10:07.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.531+0000 7f389685b640 1 Processor -- start 2023-12-01T03:10:07.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.532+0000 7f389685b640 1 -- start start 2023-12-01T03:10:07.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.533+0000 7f389685b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3890144120 0x7f389015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.533+0000 7f389685b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3890070b10 0x7f3890070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.533+0000 7f389685b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389015bf00 con 0x7f3890065960 2023-12-01T03:10:07.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.533+0000 7f389685b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389015eab0 con 0x7f3890070b10 2023-12-01T03:10:07.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.533+0000 7f389685b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389015b2f0 con 0x7f3890144120 2023-12-01T03:10:07.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.533+0000 7f3895058640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3890070b10 0x7f3890070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.533+0000 7f389605a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3890065960 0x7f389006c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:52596/0 (socket says 172.21.15.115:52596) 2023-12-01T03:10:07.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.533+0000 7f3895058640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3890070b10 0x7f3890070f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35500/0 (socket says 172.21.15.115:35500) 2023-12-01T03:10:07.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.533+0000 7f389605a640 1 -- 172.21.15.115:0/1110589394 learned_addr learned my addr 172.21.15.115:0/1110589394 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:07.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.534+0000 7f3895859640 1 --2- 172.21.15.115:0/1110589394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3890144120 0x7f389015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.534+0000 7f3895058640 1 -- 172.21.15.115:0/1110589394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3890144120 msgr2=0x7f389015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:07.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.534+0000 7f3895058640 1 --2- 172.21.15.115:0/1110589394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3890144120 0x7f389015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.534+0000 7f3895058640 1 -- 172.21.15.115:0/1110589394 >> v1:172.21.15.115:6789/0 conn(0x7f3890065960 legacy=0x7f389006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.534+0000 7f3895058640 1 -- 172.21.15.115:0/1110589394 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389015b5d0 con 0x7f3890070b10 2023-12-01T03:10:07.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.534+0000 7f3894857640 1 -- 172.21.15.115:0/1110589394 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486532942 0 0) 0x7f389015bf00 con 0x7f3890065960 2023-12-01T03:10:07.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.535+0000 7f3895058640 1 --2- 172.21.15.115:0/1110589394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 0x7f3890070f70 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f38800097a0 tx=0x7f388002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d24766515218ffb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:07.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.535+0000 7f3894857640 1 -- 172.21.15.115:0/1110589394 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388002d040 con 0x7f3890070b10 2023-12-01T03:10:07.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.535+0000 7f3894857640 1 -- 172.21.15.115:0/1110589394 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3880031810 con 0x7f3890070b10 2023-12-01T03:10:07.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.535+0000 7f3894857640 1 -- 172.21.15.115:0/1110589394 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880031ba0 con 0x7f3890070b10 2023-12-01T03:10:07.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.536+0000 7f389685b640 1 -- 172.21.15.115:0/1110589394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 msgr2=0x7f3890070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.536+0000 7f389685b640 1 --2- 172.21.15.115:0/1110589394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 0x7f3890070f70 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f38800097a0 tx=0x7f388002b8b0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.536+0000 7f389685b640 1 -- 172.21.15.115:0/1110589394 shutdown_connections 2023-12-01T03:10:07.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.536+0000 7f389685b640 1 --2- 172.21.15.115:0/1110589394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 0x7f3890070f70 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.536+0000 7f389685b640 1 --2- 172.21.15.115:0/1110589394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3890144120 0x7f389015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.536+0000 7f389685b640 1 -- 172.21.15.115:0/1110589394 >> 172.21.15.115:0/1110589394 conn(0x7f3890073330 msgr2=0x7f38900cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:07.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.536+0000 7f389685b640 1 -- 172.21.15.115:0/1110589394 shutdown_connections 2023-12-01T03:10:07.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.536+0000 7f389685b640 1 -- 172.21.15.115:0/1110589394 wait complete. 2023-12-01T03:10:07.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.538+0000 7f389685b640 1 Processor -- start 2023-12-01T03:10:07.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.538+0000 7f389685b640 1 -- start start 2023-12-01T03:10:07.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f389685b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3890065960 0x7f38901f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f389685b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 0x7f38900ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f389685b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3890144120 0x7f38900ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f389685b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38900a9920 con 0x7f3890070b10 2023-12-01T03:10:07.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f389685b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38900a8a50 con 0x7f3890144120 2023-12-01T03:10:07.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f389685b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38900a8ca0 con 0x7f3890065960 2023-12-01T03:10:07.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f3895058640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 0x7f38900ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f3895058640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 0x7f38900ab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35508/0 (socket says 172.21.15.115:35508) 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f3895058640 1 -- 172.21.15.115:0/3463763855 learned_addr learned my addr 172.21.15.115:0/3463763855 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f3895859640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3890065960 0x7f38901f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f389605a640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3890144120 0x7f38900ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f3895058640 1 -- 172.21.15.115:0/3463763855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3890065960 msgr2=0x7f38901f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f3895058640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3890065960 0x7f38901f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f3895058640 1 -- 172.21.15.115:0/3463763855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3890144120 msgr2=0x7f38900ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f3895058640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3890144120 0x7f38900ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f3895058640 1 -- 172.21.15.115:0/3463763855 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880009400 con 0x7f3890070b10 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.539+0000 7f389605a640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3890144120 0x7f38900ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.540+0000 7f3895058640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 0x7f38900ab400 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f3880000c00 tx=0x7f38800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.540+0000 7f38867fc640 1 -- 172.21.15.115:0/3463763855 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388003ba60 con 0x7f3890070b10 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.540+0000 7f38867fc640 1 -- 172.21.15.115:0/3463763855 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388003bbc0 con 0x7f3890070b10 2023-12-01T03:10:07.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.540+0000 7f38867fc640 1 -- 172.21.15.115:0/3463763855 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880004030 con 0x7f3890070b10 2023-12-01T03:10:07.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.540+0000 7f389685b640 1 -- 172.21.15.115:0/3463763855 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38900a8f80 con 0x7f3890070b10 2023-12-01T03:10:07.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.540+0000 7f389685b640 1 -- 172.21.15.115:0/3463763855 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38900a9540 con 0x7f3890070b10 2023-12-01T03:10:07.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.541+0000 7f389685b640 1 -- 172.21.15.115:0/3463763855 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38900a9a60 con 0x7f3890070b10 2023-12-01T03:10:07.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.542+0000 7f38867fc640 1 -- 172.21.15.115:0/3463763855 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 120) v1 ==== 88823+0+0 (secure 0 0 0) 0x7f3880043070 con 0x7f3890070b10 2023-12-01T03:10:07.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.543+0000 7f38867fc640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f387406a670 0x7f387406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.543+0000 7f38867fc640 1 -- 172.21.15.115:0/3463763855 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6211+0+0 (secure 0 0 0) 0x7f38800aade0 con 0x7f3890070b10 2023-12-01T03:10:07.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.543+0000 7f3895859640 1 -- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f387406a670 msgr2=0x7f387406cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:07.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.543+0000 7f3895859640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f387406a670 0x7f387406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:07.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.547+0000 7f38867fc640 1 -- 172.21.15.115:0/3463763855 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f38900a9a60 con 0x7f3890070b10 2023-12-01T03:10:07.580 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.578+0000 7feeb6a2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:10:07.590 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:07.589+0000 7f4ac4c33040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:10:07.642 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.640+0000 7feeb6a2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:10:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.742+0000 7f389685b640 1 -- 172.21.15.115:0/3463763855 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38900a9a60 con 0x7f3890070b10 2023-12-01T03:10:07.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.743+0000 7f38867fc640 1 -- 172.21.15.115:0/3463763855 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v200) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38900a9a60 con 0x7f3890070b10 2023-12-01T03:10:07.745 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:10:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.744+0000 7f3895859640 1 -- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f387406a670 msgr2=0x7f387406cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:07.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.744+0000 7f3895859640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f387406a670 0x7f387406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:10:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 -- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f387406a670 msgr2=0x7f387406cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f387406a670 0x7f387406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 -- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 msgr2=0x7f38900ab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 0x7f38900ab400 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f3880000c00 tx=0x7f38800046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 -- 172.21.15.115:0/3463763855 shutdown_connections 2023-12-01T03:10:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f387406a670 0x7f387406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3890144120 0x7f38900ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3890070b10 0x7f38900ab400 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 --2- 172.21.15.115:0/3463763855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3890065960 0x7f38901f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 -- 172.21.15.115:0/3463763855 >> 172.21.15.115:0/3463763855 conn(0x7f3890073330 msgr2=0x7f389006d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:07.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 -- 172.21.15.115:0/3463763855 shutdown_connections 2023-12-01T03:10:07.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.745+0000 7f389685b640 1 -- 172.21.15.115:0/3463763855 wait complete. 2023-12-01T03:10:07.757 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.756+0000 7feeb6a2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:10:07.764 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:10:07.844 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.841+0000 7fee38543640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:07.844 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.841+0000 7fee38543640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:07.844 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.841+0000 7fee38543640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:07.844 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.841+0000 7fee38543640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:07.845 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.841+0000 7fee38543640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:07.845 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.841+0000 7fee3b549640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:07.845 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.841+0000 7fee3b549640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:07.845 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.841+0000 7fee3b549640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:07.845 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.841+0000 7fee3b549640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:07.845 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:07.841+0000 7fee3b549640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:07.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.952+0000 7f7564599640 1 Processor -- start 2023-12-01T03:10:07.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.952+0000 7f7564599640 1 -- start start 2023-12-01T03:10:07.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.953+0000 7f7564599640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f755c142870 0x7f755c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.953+0000 7f7564599640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f755c0cd010 0x7f755c0b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.953+0000 7f7564599640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755c0cd470 con 0x7f755c0b1c50 2023-12-01T03:10:07.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.953+0000 7f7564599640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755c0b3ae0 con 0x7f755c142870 2023-12-01T03:10:07.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.953+0000 7f7564599640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c142c50 con 0x7f755c0cd010 2023-12-01T03:10:07.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.954+0000 7f7562335640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f755c142870 0x7f755c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.954+0000 7f7562335640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f755c142870 0x7f755c0ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60484/0 (socket says 172.21.15.115:60484) 2023-12-01T03:10:07.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.954+0000 7f7562335640 1 -- 172.21.15.115:0/2300746836 learned_addr learned my addr 172.21.15.115:0/2300746836 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:07.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.954+0000 7f7562335640 1 -- 172.21.15.115:0/2300746836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f755c0cd010 msgr2=0x7f755c0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:07.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.954+0000 7f7562335640 1 --2- 172.21.15.115:0/2300746836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f755c0cd010 0x7f755c0b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.954+0000 7f7562335640 1 -- 172.21.15.115:0/2300746836 >> v1:172.21.15.142:6789/0 conn(0x7f755c0b1c50 legacy=0x7f755c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:07.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.954+0000 7f7562335640 1 -- 172.21.15.115:0/2300746836 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c0b20e0 con 0x7f755c142870 2023-12-01T03:10:07.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.955+0000 7f7562335640 1 --2- 172.21.15.115:0/2300746836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 0x7f755c0ccad0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f75580098a0 tx=0x7f755802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f520d8442a2d1d3b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:07.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.955+0000 7f7561333640 1 -- 172.21.15.115:0/2300746836 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558031640 con 0x7f755c142870 2023-12-01T03:10:07.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.956+0000 7f7561333640 1 -- 172.21.15.115:0/2300746836 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75580317a0 con 0x7f755c142870 2023-12-01T03:10:07.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.956+0000 7f7561333640 1 -- 172.21.15.115:0/2300746836 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558031b30 con 0x7f755c142870 2023-12-01T03:10:07.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.956+0000 7f7564599640 1 -- 172.21.15.115:0/2300746836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 msgr2=0x7f755c0ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.956+0000 7f7564599640 1 --2- 172.21.15.115:0/2300746836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 0x7f755c0ccad0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f75580098a0 tx=0x7f755802b8f0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.956+0000 7f7564599640 1 -- 172.21.15.115:0/2300746836 shutdown_connections 2023-12-01T03:10:07.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.956+0000 7f7564599640 1 --2- 172.21.15.115:0/2300746836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f755c0cd010 0x7f755c0b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.956+0000 7f7564599640 1 --2- 172.21.15.115:0/2300746836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 0x7f755c0ccad0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.956+0000 7f7564599640 1 -- 172.21.15.115:0/2300746836 >> 172.21.15.115:0/2300746836 conn(0x7f755c065960 msgr2=0x7f755c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:07.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.956+0000 7f7564599640 1 -- 172.21.15.115:0/2300746836 shutdown_connections 2023-12-01T03:10:07.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.956+0000 7f7564599640 1 -- 172.21.15.115:0/2300746836 wait complete. 2023-12-01T03:10:07.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.958+0000 7f7564599640 1 Processor -- start 2023-12-01T03:10:07.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.958+0000 7f7564599640 1 -- start start 2023-12-01T03:10:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.959+0000 7f7564599640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f755c0b1c50 0x7f755c1f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.959+0000 7f7564599640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c0cd010 0x7f755c0ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.959+0000 7f7564599640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 0x7f755c0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.959+0000 7f7564599640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0ca070 con 0x7f755c142870 2023-12-01T03:10:07.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.959+0000 7f7564599640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0cb0a0 con 0x7f755c0cd010 2023-12-01T03:10:07.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.959+0000 7f7564599640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0cb380 con 0x7f755c0b1c50 2023-12-01T03:10:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.960+0000 7f7562b36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 0x7f755c0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.960+0000 7f7562b36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 0x7f755c0caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60486/0 (socket says 172.21.15.115:60486) 2023-12-01T03:10:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.960+0000 7f7562b36640 1 -- 172.21.15.115:0/3852097401 learned_addr learned my addr 172.21.15.115:0/3852097401 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.960+0000 7f7562335640 1 --2- 172.21.15.115:0/3852097401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f755c0b1c50 0x7f755c1f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.960+0000 7f7561b34640 1 --2- 172.21.15.115:0/3852097401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c0cd010 0x7f755c0ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.960+0000 7f7562b36640 1 -- 172.21.15.115:0/3852097401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f755c0b1c50 msgr2=0x7f755c1f0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.960+0000 7f7562b36640 1 --2- 172.21.15.115:0/3852097401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f755c0b1c50 0x7f755c1f0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.960+0000 7f7562b36640 1 -- 172.21.15.115:0/3852097401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c0cd010 msgr2=0x7f755c0ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:07.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.960+0000 7f7562b36640 1 --2- 172.21.15.115:0/3852097401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c0cd010 0x7f755c0ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.960+0000 7f7562b36640 1 -- 172.21.15.115:0/3852097401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580094d0 con 0x7f755c142870 2023-12-01T03:10:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.961+0000 7f7562b36640 1 --2- 172.21.15.115:0/3852097401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 0x7f755c0caab0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f755400c780 tx=0x7f755400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.962+0000 7f754effd640 1 -- 172.21.15.115:0/3852097401 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554004b60 con 0x7f755c142870 2023-12-01T03:10:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.962+0000 7f754effd640 1 -- 172.21.15.115:0/3852097401 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7554004cc0 con 0x7f755c142870 2023-12-01T03:10:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.962+0000 7f754effd640 1 -- 172.21.15.115:0/3852097401 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554005270 con 0x7f755c142870 2023-12-01T03:10:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.962+0000 7f7564599640 1 -- 172.21.15.115:0/3852097401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c0cb660 con 0x7f755c142870 2023-12-01T03:10:07.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.962+0000 7f7564599640 1 -- 172.21.15.115:0/3852097401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c1f2ba0 con 0x7f755c142870 2023-12-01T03:10:07.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.963+0000 7f7564599640 1 -- 172.21.15.115:0/3852097401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c04e890 con 0x7f755c142870 2023-12-01T03:10:07.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.964+0000 7f754effd640 1 -- 172.21.15.115:0/3852097401 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 121) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f75540053d0 con 0x7f755c142870 2023-12-01T03:10:07.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.965+0000 7f754effd640 1 -- 172.21.15.115:0/3852097401 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f7554088860 con 0x7f755c142870 2023-12-01T03:10:07.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:07.969+0000 7f754effd640 1 -- 172.21.15.115:0/3852097401 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7554053f00 con 0x7f755c142870 2023-12-01T03:10:08.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.144+0000 7f7564599640 1 -- 172.21.15.115:0/3852097401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f755c06b6d0 con 0x7f755c142870 2023-12-01T03:10:08.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.144+0000 7f754effd640 1 -- 172.21.15.115:0/3852097401 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v201) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7554005680 con 0x7f755c142870 2023-12-01T03:10:08.146 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:10:08.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.147+0000 7f7564599640 1 -- 172.21.15.115:0/3852097401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 msgr2=0x7f755c0caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.147+0000 7f7564599640 1 --2- 172.21.15.115:0/3852097401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 0x7f755c0caab0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f755400c780 tx=0x7f755400cb80 comp rx=0 tx=0).stop 2023-12-01T03:10:08.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.147+0000 7f7564599640 1 -- 172.21.15.115:0/3852097401 shutdown_connections 2023-12-01T03:10:08.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.147+0000 7f7564599640 1 --2- 172.21.15.115:0/3852097401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f755c142870 0x7f755c0caab0 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.147+0000 7f7564599640 1 --2- 172.21.15.115:0/3852097401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f755c0cd010 0x7f755c0ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.147+0000 7f7564599640 1 --2- 172.21.15.115:0/3852097401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f755c0b1c50 0x7f755c1f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.147+0000 7f7564599640 1 -- 172.21.15.115:0/3852097401 >> 172.21.15.115:0/3852097401 conn(0x7f755c065960 msgr2=0x7f755c06f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:08.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.147+0000 7f7564599640 1 -- 172.21.15.115:0/3852097401 shutdown_connections 2023-12-01T03:10:08.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.147+0000 7f7564599640 1 -- 172.21.15.115:0/3852097401 wait complete. 2023-12-01T03:10:08.167 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:10:08.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.375+0000 7f2f8f8ce640 1 Processor -- start 2023-12-01T03:10:08.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.376+0000 7f2f8f8ce640 1 -- start start 2023-12-01T03:10:08.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.377+0000 7f2f8f8ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f880cc440 0x7f2f880b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.377+0000 7f2f8f8ce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f880cb290 con 0x7f2f88160200 2023-12-01T03:10:08.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.377+0000 7f2f8f8ce640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f880b0ae0 con 0x7f2f880b0f80 2023-12-01T03:10:08.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.377+0000 7f2f8f8ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f880b35f0 con 0x7f2f880cc440 2023-12-01T03:10:08.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.378+0000 7f2f8d66a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2f88160200 0x7f2f8806b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42552/0 (socket says 172.21.15.115:42552) 2023-12-01T03:10:08.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.378+0000 7f2f8d66a640 1 -- 172.21.15.115:0/4263302740 learned_addr learned my addr 172.21.15.115:0/4263302740 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:08.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.378+0000 7f2f8de6b640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f880cc440 0x7f2f880b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:08.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.378+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557417971 0 0) 0x7f2f880cb290 con 0x7f2f88160200 2023-12-01T03:10:08.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.378+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f74003520 con 0x7f2f88160200 2023-12-01T03:10:08.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2157628498 0 0) 0x7f2f74003520 con 0x7f2f88160200 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f880cc440 msgr2=0x7f2f880b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f880cc440 0x7f2f880b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 >> v1:172.21.15.142:6789/0 conn(0x7f2f880b0f80 legacy=0x7f2f880cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f880b3310 con 0x7f2f88160200 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2f84001670 con 0x7f2f88160200 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 >> v1:172.21.15.115:6789/0 conn(0x7f2f88160200 legacy=0x7f2f8806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f880cc440 0x7f2f740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 0x7f2f740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f74004d00 0x7f2f74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f880b35f0 con 0x7f2f74003f70 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7400bac0 con 0x7f2f74004d00 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7400bc80 con 0x7f2f880cc440 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f8ce69640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 0x7f2f740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f8de6b640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f74004d00 0x7f2f74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:08.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.379+0000 7f2f8d66a640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f880cc440 0x7f2f740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.380+0000 7f2f8ce69640 1 -- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f880cc440 msgr2=0x7f2f740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.380+0000 7f2f8ce69640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f880cc440 0x7f2f740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.380+0000 7f2f8ce69640 1 -- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f74004d00 msgr2=0x7f2f74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.380+0000 7f2f8ce69640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f74004d00 0x7f2f74005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.380+0000 7f2f8ce69640 1 -- 172.21.15.115:0/4263302740 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f74003520 con 0x7f2f74003f70 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.380+0000 7f2f8de6b640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f74004d00 0x7f2f74005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.381+0000 7f2f8ce69640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 0x7f2f740047c0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f2f78001750 tx=0x7f2f740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ceb9c898d49895 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.381+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f780052d0 con 0x7f2f74003f70 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.381+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7802e050 con 0x7f2f74003f70 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.381+0000 7f2f7ffff640 1 -- 172.21.15.115:0/4263302740 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78003830 con 0x7f2f74003f70 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.381+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/4263302740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 msgr2=0x7f2f740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.381+0000 7f2f8f8ce640 1 --2- 172.21.15.115:0/4263302740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 0x7f2f740047c0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f2f78001750 tx=0x7f2f740057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.381+0000 7f2f8de6b640 1 -- 172.21.15.115:0/4263302740 reap_dead start 2023-12-01T03:10:08.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.382+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/4263302740 shutdown_connections 2023-12-01T03:10:08.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.382+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/4263302740 >> 172.21.15.115:0/4263302740 conn(0x7f2f88065960 msgr2=0x7f2f88065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:08.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.382+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/4263302740 shutdown_connections 2023-12-01T03:10:08.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.382+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/4263302740 wait complete. 2023-12-01T03:10:08.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.383+0000 7f2f8f8ce640 1 Processor -- start 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.384+0000 7f2f8f8ce640 1 -- start start 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.384+0000 7f2f8f8ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 0x7f2f8814ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8f8ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f8814efd0 0x7f2f8814cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8f8ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f8814d4e0 0x7f2f88150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8f8ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8814d9f0 con 0x7f2f74003f70 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8f8ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8814dcd0 con 0x7f2f8814efd0 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8f8ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8814dfb0 con 0x7f2f8814d4e0 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8d66a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 0x7f2f8814ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8d66a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 0x7f2f8814ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60502/0 (socket says 172.21.15.115:60502) 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8d66a640 1 -- 172.21.15.115:0/442063816 learned_addr learned my addr 172.21.15.115:0/442063816 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8d66a640 1 -- 172.21.15.115:0/442063816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f8814d4e0 msgr2=0x7f2f88150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8d66a640 1 --2- 172.21.15.115:0/442063816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f8814d4e0 0x7f2f88150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8d66a640 1 -- 172.21.15.115:0/442063816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f8814efd0 msgr2=0x7f2f8814cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8ce69640 1 --2- 172.21.15.115:0/442063816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f8814efd0 0x7f2f8814cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:08.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8d66a640 1 --2- 172.21.15.115:0/442063816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f8814efd0 0x7f2f8814cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.385+0000 7f2f8d66a640 1 -- 172.21.15.115:0/442063816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8814e290 con 0x7f2f74003f70 2023-12-01T03:10:08.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.386+0000 7f2f8d66a640 1 --2- 172.21.15.115:0/442063816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 0x7f2f8814ea90 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f2f84001050 tx=0x7f2f8400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:08.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.386+0000 7f2f7dffb640 1 -- 172.21.15.115:0/442063816 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400ee50 con 0x7f2f74003f70 2023-12-01T03:10:08.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.386+0000 7f2f7dffb640 1 -- 172.21.15.115:0/442063816 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8400c870 con 0x7f2f74003f70 2023-12-01T03:10:08.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.386+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/442063816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8814e570 con 0x7f2f74003f70 2023-12-01T03:10:08.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.386+0000 7f2f7dffb640 1 -- 172.21.15.115:0/442063816 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400cc20 con 0x7f2f74003f70 2023-12-01T03:10:08.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.386+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/442063816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f880c9ac0 con 0x7f2f74003f70 2023-12-01T03:10:08.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.387+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/442063816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8804e890 con 0x7f2f74003f70 2023-12-01T03:10:08.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.388+0000 7f2f7dffb640 1 -- 172.21.15.115:0/442063816 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 121) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f2f8401d460 con 0x7f2f74003f70 2023-12-01T03:10:08.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.389+0000 7f2f7dffb640 1 -- 172.21.15.115:0/442063816 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f2f84088710 con 0x7f2f74003f70 2023-12-01T03:10:08.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.393+0000 7f2f7dffb640 1 -- 172.21.15.115:0/442063816 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2f84053d30 con 0x7f2f74003f70 2023-12-01T03:10:08.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.586+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/442063816 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f8814e760 con 0x7f2f74003f70 2023-12-01T03:10:08.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.587+0000 7f2f7dffb640 1 -- 172.21.15.115:0/442063816 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v201) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f84057350 con 0x7f2f74003f70 2023-12-01T03:10:08.589 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:10:08.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.589+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/442063816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 msgr2=0x7f2f8814ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.589+0000 7f2f8f8ce640 1 --2- 172.21.15.115:0/442063816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 0x7f2f8814ea90 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f2f84001050 tx=0x7f2f8400ec00 comp rx=0 tx=0).stop 2023-12-01T03:10:08.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.589+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/442063816 shutdown_connections 2023-12-01T03:10:08.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.589+0000 7f2f8f8ce640 1 --2- 172.21.15.115:0/442063816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2f8814d4e0 0x7f2f88150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.589+0000 7f2f8f8ce640 1 --2- 172.21.15.115:0/442063816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f8814efd0 0x7f2f8814cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.589+0000 7f2f8f8ce640 1 --2- 172.21.15.115:0/442063816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f74003f70 0x7f2f8814ea90 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.589+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/442063816 >> 172.21.15.115:0/442063816 conn(0x7f2f88065960 msgr2=0x7f2f88070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:08.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.589+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/442063816 shutdown_connections 2023-12-01T03:10:08.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.589+0000 7f2f8f8ce640 1 -- 172.21.15.115:0/442063816 wait complete. 2023-12-01T03:10:08.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:10:08.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.826+0000 7f70a0ccf640 1 Processor -- start 2023-12-01T03:10:08.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.826+0000 7f70a0ccf640 1 -- start start 2023-12-01T03:10:08.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.827+0000 7f70a0ccf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f709c15d080 0x7f709c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.827+0000 7f70a0ccf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f709c0cf7a0 0x7f709c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.827+0000 7f70a0ccf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709c0cb0e0 con 0x7f709c06a950 2023-12-01T03:10:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.827+0000 7f70a0ccf640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709c06ad80 con 0x7f709c15d080 2023-12-01T03:10:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.827+0000 7f70a0ccf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709c15d460 con 0x7f709c0cf7a0 2023-12-01T03:10:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.828+0000 7f709a59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f709c15d080 0x7f709c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.828+0000 7f709a59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f709c15d080 0x7f709c069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60504/0 (socket says 172.21.15.115:60504) 2023-12-01T03:10:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.828+0000 7f709a59c640 1 -- 172.21.15.115:0/2460486496 learned_addr learned my addr 172.21.15.115:0/2460486496 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:08.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.828+0000 7f709ad9d640 1 --1- 172.21.15.115:0/2460486496 >> v1:172.21.15.115:6789/0 conn(0x7f709c06a950 0x7f709c0cd310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42566/0 (socket says 172.21.15.115:42566) 2023-12-01T03:10:08.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.828+0000 7f7099d9b640 1 --2- 172.21.15.115:0/2460486496 >> v2:172.21.15.142:3300/0 conn(0x7f709c0cf7a0 0x7f709c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:08.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.828+0000 7f709a59c640 1 -- 172.21.15.115:0/2460486496 >> v2:172.21.15.142:3300/0 conn(0x7f709c0cf7a0 msgr2=0x7f709c06a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:08.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.828+0000 7f709a59c640 1 --2- 172.21.15.115:0/2460486496 >> v2:172.21.15.142:3300/0 conn(0x7f709c0cf7a0 0x7f709c06a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.828+0000 7f709a59c640 1 -- 172.21.15.115:0/2460486496 >> v1:172.21.15.115:6789/0 conn(0x7f709c06a950 legacy=0x7f709c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.828+0000 7f709a59c640 1 -- 172.21.15.115:0/2460486496 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0d1860 con 0x7f709c15d080 2023-12-01T03:10:08.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.828+0000 7f709959a640 1 -- 172.21.15.115:0/2460486496 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237151024 0 0) 0x7f709c0cb0e0 con 0x7f709c06a950 2023-12-01T03:10:08.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.829+0000 7f709a59c640 1 --2- 172.21.15.115:0/2460486496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 0x7f709c069ed0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f709c06fcb0 tx=0x7f708c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee7f0277ef306a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:08.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f709959a640 1 -- 172.21.15.115:0/2460486496 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c031870 con 0x7f709c15d080 2023-12-01T03:10:08.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f709959a640 1 -- 172.21.15.115:0/2460486496 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c0319d0 con 0x7f709c15d080 2023-12-01T03:10:08.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f709959a640 1 -- 172.21.15.115:0/2460486496 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c031d60 con 0x7f709c15d080 2023-12-01T03:10:08.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2460486496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 msgr2=0x7f709c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f70a0ccf640 1 --2- 172.21.15.115:0/2460486496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 0x7f709c069ed0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f709c06fcb0 tx=0x7f708c02b910 comp rx=0 tx=0).stop 2023-12-01T03:10:08.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2460486496 shutdown_connections 2023-12-01T03:10:08.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f70a0ccf640 1 --2- 172.21.15.115:0/2460486496 >> v2:172.21.15.142:3300/0 conn(0x7f709c0cf7a0 0x7f709c06a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f70a0ccf640 1 --2- 172.21.15.115:0/2460486496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 0x7f709c069ed0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2460486496 >> 172.21.15.115:0/2460486496 conn(0x7f709c0d0a90 msgr2=0x7f709c0d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:08.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2460486496 shutdown_connections 2023-12-01T03:10:08.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.830+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2460486496 wait complete. 2023-12-01T03:10:08.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.832+0000 7f70a0ccf640 1 Processor -- start 2023-12-01T03:10:08.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.832+0000 7f70a0ccf640 1 -- start start 2023-12-01T03:10:08.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.833+0000 7f70a0ccf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f709c06a950 0x7f709c0b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.833+0000 7f70a0ccf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 0x7f709c158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.833+0000 7f70a0ccf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f709c1601c0 0x7f709c0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.833+0000 7f70a0ccf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0c5340 con 0x7f709c15d080 2023-12-01T03:10:08.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.833+0000 7f70a0ccf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0b3860 con 0x7f709c1601c0 2023-12-01T03:10:08.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.833+0000 7f70a0ccf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0b5f30 con 0x7f709c06a950 2023-12-01T03:10:08.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.833+0000 7f7099d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 0x7f709c158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:08.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.833+0000 7f7099d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 0x7f709c158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60520/0 (socket says 172.21.15.115:60520) 2023-12-01T03:10:08.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.833+0000 7f7099d9b640 1 -- 172.21.15.115:0/2170150592 learned_addr learned my addr 172.21.15.115:0/2170150592 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:08.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.834+0000 7f7099d9b640 1 -- 172.21.15.115:0/2170150592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f709c06a950 msgr2=0x7f709c0b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.834+0000 7f7099d9b640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f709c06a950 0x7f709c0b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.834+0000 7f7099d9b640 1 -- 172.21.15.115:0/2170150592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f709c1601c0 msgr2=0x7f709c0ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:08.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.834+0000 7f7099d9b640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f709c1601c0 0x7f709c0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:08.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.834+0000 7f7099d9b640 1 -- 172.21.15.115:0/2170150592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0094d0 con 0x7f709c15d080 2023-12-01T03:10:08.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.834+0000 7f7099d9b640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 0x7f709c158d80 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f709000d830 tx=0x7f709000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:08.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.834+0000 7f708affd640 1 -- 172.21.15.115:0/2170150592 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000b6c0 con 0x7f709c15d080 2023-12-01T03:10:08.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.834+0000 7f708affd640 1 -- 172.21.15.115:0/2170150592 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709000b820 con 0x7f709c15d080 2023-12-01T03:10:08.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.834+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2170150592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c0b61f0 con 0x7f709c15d080 2023-12-01T03:10:08.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.834+0000 7f708affd640 1 -- 172.21.15.115:0/2170150592 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000bbb0 con 0x7f709c15d080 2023-12-01T03:10:08.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.835+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2170150592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c0c5550 con 0x7f709c15d080 2023-12-01T03:10:08.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.835+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2170150592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c04e890 con 0x7f709c15d080 2023-12-01T03:10:08.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.836+0000 7f708affd640 1 -- 172.21.15.115:0/2170150592 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 122) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f709001e460 con 0x7f709c15d080 2023-12-01T03:10:08.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.837+0000 7f708affd640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f707c06a280 0x7f707c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:08.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.837+0000 7f708affd640 1 -- 172.21.15.115:0/2170150592 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f7090089060 con 0x7f709c15d080 2023-12-01T03:10:08.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.838+0000 7f709a59c640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f707c06a280 0x7f707c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:08.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.838+0000 7f709a59c640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f707c06a280 0x7f707c06c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f708c036000 tx=0x7f708c030000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:08.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:08.842+0000 7f708affd640 1 -- 172.21.15.115:0/2170150592 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7090054620 con 0x7f709c15d080 2023-12-01T03:10:09.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.067+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2170150592 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f709c0d1560 con 0x7f709c15d080 2023-12-01T03:10:09.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.068+0000 7f708affd640 1 -- 172.21.15.115:0/2170150592 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f7090057c40 con 0x7f709c15d080 2023-12-01T03:10:09.071 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:09.071 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/974148243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":974148243},{"type":"v1","addr":"172.21.15.115:6837","nonce":974148243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/476969842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":476969842},{"type":"v1","addr":"172.21.15.142:6834","nonce":476969842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1103499235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1103499235},{"type":"v1","addr":"172.21.15.115:6835","nonce":1103499235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:08:43.078263+0000","modified":"2023-12-01T03:08:46.096728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10149":{"gid":10149,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/3801423208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3801423208},{"type":"v1","addr":"172.21.15.142:6835","nonce":3801423208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-01T03:10:09.072 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 87 2023-12-01T03:10:09.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2170150592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f707c06a280 msgr2=0x7f707c06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f707c06a280 0x7f707c06c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f708c036000 tx=0x7f708c030000 comp rx=0 tx=0).stop 2023-12-01T03:10:09.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2170150592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 msgr2=0x7f709c158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 0x7f709c158d80 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f709000d830 tx=0x7f709000dcf0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2170150592 shutdown_connections 2023-12-01T03:10:09.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f707c06a280 0x7f707c06c740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f709c1601c0 0x7f709c0ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f709c15d080 0x7f709c158d80 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 --2- 172.21.15.115:0/2170150592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f709c06a950 0x7f709c0b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2170150592 >> 172.21.15.115:0/2170150592 conn(0x7f709c0d0a90 msgr2=0x7f709c15b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:09.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2170150592 shutdown_connections 2023-12-01T03:10:09.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.071+0000 7f70a0ccf640 1 -- 172.21.15.115:0/2170150592 wait complete. 2023-12-01T03:10:09.093 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:09.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.308+0000 7fdd21684640 1 Processor -- start 2023-12-01T03:10:09.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.309+0000 7fdd21684640 1 -- start start 2023-12-01T03:10:09.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.310+0000 7fdd21684640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd1c0b5d80 0x7fdd1c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.310+0000 7fdd21684640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdd1c0d1230 0x7fdd1c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.310+0000 7fdd21684640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1c0d1b90 con 0x7fdd1c15cd20 2023-12-01T03:10:09.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.310+0000 7fdd21684640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1c0b2160 con 0x7fdd1c0b5d80 2023-12-01T03:10:09.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.310+0000 7fdd21684640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1c0b8c80 con 0x7fdd1c0d1230 2023-12-01T03:10:09.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.310+0000 7fdd1a7fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd1c0b5d80 0x7fdd1c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.311+0000 7fdd1a7fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdd1c0b5d80 0x7fdd1c0d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60534/0 (socket says 172.21.15.115:60534) 2023-12-01T03:10:09.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.311+0000 7fdd1a7fc640 1 -- 172.21.15.115:0/1892346589 learned_addr learned my addr 172.21.15.115:0/1892346589 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:09.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.311+0000 7fdd1b7fe640 1 --2- 172.21.15.115:0/1892346589 >> v2:172.21.15.142:3300/0 conn(0x7fdd1c0d1230 0x7fdd1c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.311+0000 7fdd1a7fc640 1 -- 172.21.15.115:0/1892346589 >> v2:172.21.15.142:3300/0 conn(0x7fdd1c0d1230 msgr2=0x7fdd1c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:09.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.311+0000 7fdd1a7fc640 1 --2- 172.21.15.115:0/1892346589 >> v2:172.21.15.142:3300/0 conn(0x7fdd1c0d1230 0x7fdd1c0d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.311+0000 7fdd1a7fc640 1 -- 172.21.15.115:0/1892346589 >> v1:172.21.15.142:6789/0 conn(0x7fdd1c15cd20 legacy=0x7fdd1c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:09.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.311+0000 7fdd1a7fc640 1 -- 172.21.15.115:0/1892346589 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c0ce2e0 con 0x7fdd1c0b5d80 2023-12-01T03:10:09.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.312+0000 7fdd1a7fc640 1 --2- 172.21.15.115:0/1892346589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0b5d80 0x7fdd1c0d24d0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0c6720 tx=0x7fdd0402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593111ade43c887 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:09.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd19ffb640 1 -- 172.21.15.115:0/1892346589 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040317e0 con 0x7fdd1c0b5d80 2023-12-01T03:10:09.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd19ffb640 1 -- 172.21.15.115:0/1892346589 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd04031940 con 0x7fdd1c0b5d80 2023-12-01T03:10:09.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd19ffb640 1 -- 172.21.15.115:0/1892346589 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04031cd0 con 0x7fdd1c0b5d80 2023-12-01T03:10:09.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd21684640 1 -- 172.21.15.115:0/1892346589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0b5d80 msgr2=0x7fdd1c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd21684640 1 --2- 172.21.15.115:0/1892346589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0b5d80 0x7fdd1c0d24d0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0c6720 tx=0x7fdd0402b880 comp rx=0 tx=0).stop 2023-12-01T03:10:09.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd21684640 1 -- 172.21.15.115:0/1892346589 shutdown_connections 2023-12-01T03:10:09.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd21684640 1 --2- 172.21.15.115:0/1892346589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0b5d80 0x7fdd1c0d24d0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd21684640 1 --2- 172.21.15.115:0/1892346589 >> v2:172.21.15.142:3300/0 conn(0x7fdd1c0d1230 0x7fdd1c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd21684640 1 -- 172.21.15.115:0/1892346589 >> 172.21.15.115:0/1892346589 conn(0x7fdd1c06a0e0 msgr2=0x7fdd1c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:09.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd21684640 1 -- 172.21.15.115:0/1892346589 shutdown_connections 2023-12-01T03:10:09.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.313+0000 7fdd21684640 1 -- 172.21.15.115:0/1892346589 wait complete. 2023-12-01T03:10:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.315+0000 7fdd21684640 1 Processor -- start 2023-12-01T03:10:09.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.315+0000 7fdd21684640 1 -- start start 2023-12-01T03:10:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.316+0000 7fdd21684640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd1c0b5d80 0x7fdd1c1f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.316+0000 7fdd21684640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0d1230 0x7fdd1c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.316+0000 7fdd21684640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd1c15cd20 0x7fdd1c1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.316+0000 7fdd21684640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c1559c0 con 0x7fdd1c0d1230 2023-12-01T03:10:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.316+0000 7fdd21684640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0cad70 con 0x7fdd1c15cd20 2023-12-01T03:10:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.316+0000 7fdd21684640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0cafc0 con 0x7fdd1c0b5d80 2023-12-01T03:10:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.316+0000 7fdd1a7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0d1230 0x7fdd1c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1a7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0d1230 0x7fdd1c15f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60550/0 (socket says 172.21.15.115:60550) 2023-12-01T03:10:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1a7fc640 1 -- 172.21.15.115:0/1711071733 learned_addr learned my addr 172.21.15.115:0/1711071733 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1a7fc640 1 -- 172.21.15.115:0/1711071733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd1c0b5d80 msgr2=0x7fdd1c1f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1affd640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd1c0b5d80 0x7fdd1c1f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1a7fc640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd1c0b5d80 0x7fdd1c1f2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1b7fe640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd1c15cd20 0x7fdd1c1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1a7fc640 1 -- 172.21.15.115:0/1711071733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd1c15cd20 msgr2=0x7fdd1c1586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1a7fc640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd1c15cd20 0x7fdd1c1586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1a7fc640 1 -- 172.21.15.115:0/1711071733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd040094d0 con 0x7fdd1c0d1230 2023-12-01T03:10:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1affd640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd1c0b5d80 0x7fdd1c1f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:09.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.317+0000 7fdd1a7fc640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0d1230 0x7fdd1c15f710 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fdd0402b8c0 tx=0x7fdd040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:09.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.318+0000 7fdd137fe640 1 -- 172.21.15.115:0/1711071733 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040329c0 con 0x7fdd1c0d1230 2023-12-01T03:10:09.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.318+0000 7fdd137fe640 1 -- 172.21.15.115:0/1711071733 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd04032b40 con 0x7fdd1c0d1230 2023-12-01T03:10:09.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.318+0000 7fdd137fe640 1 -- 172.21.15.115:0/1711071733 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040453f0 con 0x7fdd1c0d1230 2023-12-01T03:10:09.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.318+0000 7fdd21684640 1 -- 172.21.15.115:0/1711071733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c0cb2a0 con 0x7fdd1c0d1230 2023-12-01T03:10:09.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.318+0000 7fdd21684640 1 -- 172.21.15.115:0/1711071733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c0cb860 con 0x7fdd1c0d1230 2023-12-01T03:10:09.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.319+0000 7fdd21684640 1 -- 172.21.15.115:0/1711071733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c0cba50 con 0x7fdd1c0d1230 2023-12-01T03:10:09.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.320+0000 7fdd137fe640 1 -- 172.21.15.115:0/1711071733 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 122) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fdd04045550 con 0x7fdd1c0d1230 2023-12-01T03:10:09.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.321+0000 7fdd137fe640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdcfc06a280 0x7fdcfc06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.321+0000 7fdd137fe640 1 -- 172.21.15.115:0/1711071733 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6527+0+0 (secure 0 0 0) 0x7fdd040aab10 con 0x7fdd1c0d1230 2023-12-01T03:10:09.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.321+0000 7fdd1affd640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdcfc06a280 0x7fdcfc06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.322+0000 7fdd1affd640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdcfc06a280 0x7fdcfc06c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c007eb0 tx=0x7fdd0c0073c0 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:09.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.325+0000 7fdd137fe640 1 -- 172.21.15.115:0/1711071733 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdd04076110 con 0x7fdd1c0d1230 2023-12-01T03:10:09.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.518+0000 7fdd21684640 1 -- 172.21.15.115:0/1711071733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd1c04e890 con 0x7fdd1c0d1230 2023-12-01T03:10:09.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.520+0000 7fdd137fe640 1 -- 172.21.15.115:0/1711071733 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+17396 (secure 0 0 0) 0x7fdd04079730 con 0x7fdd1c0d1230 2023-12-01T03:10:09.523 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:09.523 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":201,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:07.759323+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:08:40.709195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-01T03:08:42.071749+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":200,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.18","mappings":[{"from":5,"to":3}]},{"pgid":"23.19","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/606861280":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/1640166514":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/2718416441":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/443657929":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/3570546488":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/3032781514":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/2184838711":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/288376877":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/361869044":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/4037780703":"2023-12-02T03:08:56.516548+0000","172.21.15.115:6824/138932":"2023-12-02T03:08:56.516548+0000","172.21.15.115:6822/138932":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/3414835041":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/1495322174":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/2764969208":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/1016167991":"2023-12-02T03:08:56.516548+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.523+0000 7fdd21684640 1 -- 172.21.15.115:0/1711071733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdcfc06a280 msgr2=0x7fdcfc06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.523+0000 7fdd21684640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdcfc06a280 0x7fdcfc06c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c007eb0 tx=0x7fdd0c0073c0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.523+0000 7fdd21684640 1 -- 172.21.15.115:0/1711071733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0d1230 msgr2=0x7fdd1c15f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.523+0000 7fdd21684640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0d1230 0x7fdd1c15f710 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fdd0402b8c0 tx=0x7fdd040046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.523+0000 7fdd21684640 1 -- 172.21.15.115:0/1711071733 shutdown_connections 2023-12-01T03:10:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.523+0000 7fdd21684640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdcfc06a280 0x7fdcfc06c740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.523+0000 7fdd21684640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd1c15cd20 0x7fdd1c1586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.523+0000 7fdd21684640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd1c0d1230 0x7fdd1c15f710 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.523+0000 7fdd21684640 1 --2- 172.21.15.115:0/1711071733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd1c0b5d80 0x7fdd1c1f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.523+0000 7fdd21684640 1 -- 172.21.15.115:0/1711071733 >> 172.21.15.115:0/1711071733 conn(0x7fdd1c06a0e0 msgr2=0x7fdd1c1598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.524+0000 7fdd21684640 1 -- 172.21.15.115:0/1711071733 shutdown_connections 2023-12-01T03:10:09.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.524+0000 7fdd21684640 1 -- 172.21.15.115:0/1711071733 wait complete. 2023-12-01T03:10:09.544 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T03:10:09.544 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:10:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.754+0000 7f1e304fb640 1 Processor -- start 2023-12-01T03:10:09.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.754+0000 7f1e304fb640 1 -- start start 2023-12-01T03:10:09.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.755+0000 7f1e304fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1e28159dd0 0x7f1e280c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.755+0000 7f1e304fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28070d30 0x7f1e280c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.755+0000 7f1e304fb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2806f8b0 con 0x7f1e2806b6f0 2023-12-01T03:10:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.755+0000 7f1e304fb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e280adcb0 con 0x7f1e28159dd0 2023-12-01T03:10:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.755+0000 7f1e304fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e280ad9d0 con 0x7f1e28070d30 2023-12-01T03:10:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.756+0000 7f1e2e297640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1e28159dd0 0x7f1e280c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.756+0000 7f1e2e297640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1e28159dd0 0x7f1e280c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60566/0 (socket says 172.21.15.115:60566) 2023-12-01T03:10:09.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.756+0000 7f1e2e297640 1 -- 172.21.15.115:0/1742179252 learned_addr learned my addr 172.21.15.115:0/1742179252 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.756+0000 7f1e2ea98640 1 --2- 172.21.15.115:0/1742179252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28070d30 0x7f1e280c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.756+0000 7f1e2e297640 1 -- 172.21.15.115:0/1742179252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28070d30 msgr2=0x7f1e280c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.756+0000 7f1e2e297640 1 --2- 172.21.15.115:0/1742179252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28070d30 0x7f1e280c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.756+0000 7f1e2e297640 1 -- 172.21.15.115:0/1742179252 >> v1:172.21.15.115:6789/0 conn(0x7f1e2806b6f0 legacy=0x7f1e280cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.756+0000 7f1e2d295640 1 -- 172.21.15.115:0/1742179252 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497055955 0 0) 0x7f1e2806f8b0 con 0x7f1e2806b6f0 2023-12-01T03:10:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.756+0000 7f1e2e297640 1 -- 172.21.15.115:0/1742179252 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e280c9000 con 0x7f1e28159dd0 2023-12-01T03:10:09.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.756+0000 7f1e2ea98640 1 --2- 172.21.15.115:0/1742179252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28070d30 0x7f1e280c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.757+0000 7f1e2e297640 1 --2- 172.21.15.115:0/1742179252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28159dd0 0x7f1e280c97e0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009870 tx=0x7f1e2402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a801d6f184f7c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.758+0000 7f1e2d295640 1 -- 172.21.15.115:0/1742179252 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e240317c0 con 0x7f1e28159dd0 2023-12-01T03:10:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.758+0000 7f1e2d295640 1 -- 172.21.15.115:0/1742179252 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e24031920 con 0x7f1e28159dd0 2023-12-01T03:10:09.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.758+0000 7f1e2d295640 1 -- 172.21.15.115:0/1742179252 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e24031cb0 con 0x7f1e28159dd0 2023-12-01T03:10:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.759+0000 7f1e304fb640 1 -- 172.21.15.115:0/1742179252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28159dd0 msgr2=0x7f1e280c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.759+0000 7f1e304fb640 1 --2- 172.21.15.115:0/1742179252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28159dd0 0x7f1e280c97e0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009870 tx=0x7f1e2402b860 comp rx=0 tx=0).stop 2023-12-01T03:10:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.759+0000 7f1e304fb640 1 -- 172.21.15.115:0/1742179252 shutdown_connections 2023-12-01T03:10:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.759+0000 7f1e304fb640 1 --2- 172.21.15.115:0/1742179252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28070d30 0x7f1e280c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.759+0000 7f1e304fb640 1 --2- 172.21.15.115:0/1742179252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28159dd0 0x7f1e280c97e0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.759+0000 7f1e304fb640 1 -- 172.21.15.115:0/1742179252 >> 172.21.15.115:0/1742179252 conn(0x7f1e280ce880 msgr2=0x7f1e280cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.759+0000 7f1e304fb640 1 -- 172.21.15.115:0/1742179252 shutdown_connections 2023-12-01T03:10:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.759+0000 7f1e304fb640 1 -- 172.21.15.115:0/1742179252 wait complete. 2023-12-01T03:10:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.761+0000 7f1e304fb640 1 Processor -- start 2023-12-01T03:10:09.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.761+0000 7f1e304fb640 1 -- start start 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e304fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e2806b6f0 0x7f1e28150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e304fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28070d30 0x7f1e2815bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e304fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28159dd0 0x7f1e2815a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e304fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e280c3180 con 0x7f1e28070d30 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e304fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e280c3700 con 0x7f1e2806b6f0 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e304fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2815c8b0 con 0x7f1e28159dd0 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2da96640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28070d30 0x7f1e2815bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2da96640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28070d30 0x7f1e2815bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60574/0 (socket says 172.21.15.115:60574) 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2da96640 1 -- 172.21.15.115:0/3211196846 learned_addr learned my addr 172.21.15.115:0/3211196846 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2ea98640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28159dd0 0x7f1e2815a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2e297640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e2806b6f0 0x7f1e28150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2da96640 1 -- 172.21.15.115:0/3211196846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28159dd0 msgr2=0x7f1e2815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2da96640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28159dd0 0x7f1e2815a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2da96640 1 -- 172.21.15.115:0/3211196846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e2806b6f0 msgr2=0x7f1e28150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2da96640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e2806b6f0 0x7f1e28150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2da96640 1 -- 172.21.15.115:0/3211196846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240094d0 con 0x7f1e28070d30 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.762+0000 7f1e2e297640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e2806b6f0 0x7f1e28150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.763+0000 7f1e2da96640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28070d30 0x7f1e2815bf60 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00c4b0 tx=0x7f1e1c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.763+0000 7f1e1affd640 1 -- 172.21.15.115:0/3211196846 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c005480 con 0x7f1e28070d30 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.763+0000 7f1e1affd640 1 -- 172.21.15.115:0/3211196846 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c0035f0 con 0x7f1e28070d30 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.763+0000 7f1e1affd640 1 -- 172.21.15.115:0/3211196846 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0039e0 con 0x7f1e28070d30 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.763+0000 7f1e304fb640 1 -- 172.21.15.115:0/3211196846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2815cb90 con 0x7f1e28070d30 2023-12-01T03:10:09.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.763+0000 7f1e304fb640 1 -- 172.21.15.115:0/3211196846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2815ce70 con 0x7f1e28070d30 2023-12-01T03:10:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.764+0000 7f1e304fb640 1 -- 172.21.15.115:0/3211196846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2804e890 con 0x7f1e28070d30 2023-12-01T03:10:09.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.765+0000 7f1e1affd640 1 -- 172.21.15.115:0/3211196846 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 122) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f1e1c003b40 con 0x7f1e28070d30 2023-12-01T03:10:09.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.766+0000 7f1e1affd640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1e0c06a280 0x7f1e0c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:09.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.766+0000 7f1e1affd640 1 -- 172.21.15.115:0/3211196846 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f1e1c087f60 con 0x7f1e28070d30 2023-12-01T03:10:09.768 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.766+0000 7f1e2e297640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1e0c06a280 0x7f1e0c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:09.769 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.767+0000 7f1e2e297640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1e0c06a280 0x7f1e0c06c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1e24036000 tx=0x7f1e2403d040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:09.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:09.770+0000 7f1e1affd640 1 -- 172.21.15.115:0/3211196846 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1e1c053490 con 0x7f1e28070d30 2023-12-01T03:10:10.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.012+0000 7f1e304fb640 1 -- 172.21.15.115:0/3211196846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e280c8cb0 con 0x7f1e28070d30 2023-12-01T03:10:10.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.013+0000 7f1e1affd640 1 -- 172.21.15.115:0/3211196846 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f1e1c017020 con 0x7f1e28070d30 2023-12-01T03:10:10.015 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:10.015 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/974148243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":974148243},{"type":"v1","addr":"172.21.15.115:6837","nonce":974148243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/476969842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":476969842},{"type":"v1","addr":"172.21.15.142:6834","nonce":476969842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1103499235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1103499235},{"type":"v1","addr":"172.21.15.115:6835","nonce":1103499235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:08:43.078263+0000","modified":"2023-12-01T03:08:46.096728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10149":{"gid":10149,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/3801423208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3801423208},{"type":"v1","addr":"172.21.15.142:6835","nonce":3801423208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-01T03:10:10.015 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 87 2023-12-01T03:10:10.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 -- 172.21.15.115:0/3211196846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1e0c06a280 msgr2=0x7f1e0c06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1e0c06a280 0x7f1e0c06c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1e24036000 tx=0x7f1e2403d040 comp rx=0 tx=0).stop 2023-12-01T03:10:10.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 -- 172.21.15.115:0/3211196846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28070d30 msgr2=0x7f1e2815bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28070d30 0x7f1e2815bf60 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00c4b0 tx=0x7f1e1c00c970 comp rx=0 tx=0).stop 2023-12-01T03:10:10.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 -- 172.21.15.115:0/3211196846 shutdown_connections 2023-12-01T03:10:10.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1e0c06a280 0x7f1e0c06c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e28159dd0 0x7f1e2815a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e28070d30 0x7f1e2815bf60 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 --2- 172.21.15.115:0/3211196846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e2806b6f0 0x7f1e28150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 -- 172.21.15.115:0/3211196846 >> 172.21.15.115:0/3211196846 conn(0x7f1e280ce880 msgr2=0x7f1e2806c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 -- 172.21.15.115:0/3211196846 shutdown_connections 2023-12-01T03:10:10.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.015+0000 7f1e304fb640 1 -- 172.21.15.115:0/3211196846 wait complete. 2023-12-01T03:10:10.034 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:10:10.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.234+0000 7f631c499640 1 Processor -- start 2023-12-01T03:10:10.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.234+0000 7f631c499640 1 -- start start 2023-12-01T03:10:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.235+0000 7f631c499640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f631415cd20 0x7f63140d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.235+0000 7f631c499640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f63140b5d80 0x7f63140b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.235+0000 7f631c499640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f63140d1230 0x7f63140d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.235+0000 7f631c499640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63140b2160 con 0x7f63140d1230 2023-12-01T03:10:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.235+0000 7f631c499640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63140d1be0 con 0x7f63140b5d80 2023-12-01T03:10:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.235+0000 7f631c499640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63140d2a10 con 0x7f631415cd20 2023-12-01T03:10:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.236+0000 7f631aa36640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f63140d1230 0x7f63140d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.236+0000 7f631aa36640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f63140d1230 0x7f63140d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60606/0 (socket says 172.21.15.115:60606) 2023-12-01T03:10:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.236+0000 7f631aa36640 1 -- 172.21.15.115:0/619021535 learned_addr learned my addr 172.21.15.115:0/619021535 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.236+0000 7f631a235640 1 --2- 172.21.15.115:0/619021535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f631415cd20 0x7f63140d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.236+0000 7f6319a34640 1 --2- 172.21.15.115:0/619021535 >> v2:172.21.15.142:3300/0 conn(0x7f63140b5d80 0x7f63140b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.236+0000 7f631aa36640 1 -- 172.21.15.115:0/619021535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f631415cd20 msgr2=0x7f63140d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.236+0000 7f631aa36640 1 --2- 172.21.15.115:0/619021535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f631415cd20 0x7f63140d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.236+0000 7f631aa36640 1 -- 172.21.15.115:0/619021535 >> v2:172.21.15.142:3300/0 conn(0x7f63140b5d80 msgr2=0x7f63140b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.236+0000 7f631aa36640 1 --2- 172.21.15.115:0/619021535 >> v2:172.21.15.142:3300/0 conn(0x7f63140b5d80 0x7f63140b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.236+0000 7f631aa36640 1 -- 172.21.15.115:0/619021535 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63140ce2e0 con 0x7f63140d1230 2023-12-01T03:10:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.237+0000 7f631aa36640 1 --2- 172.21.15.115:0/619021535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63140d1230 0x7f63140d16a0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f631408bcc0 tx=0x7f630c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9f4ebf575970e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.238+0000 7f6319233640 1 -- 172.21.15.115:0/619021535 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c02c9d0 con 0x7f63140d1230 2023-12-01T03:10:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.238+0000 7f6319233640 1 -- 172.21.15.115:0/619021535 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c02cb30 con 0x7f63140d1230 2023-12-01T03:10:10.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.238+0000 7f6319233640 1 -- 172.21.15.115:0/619021535 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c0343f0 con 0x7f63140d1230 2023-12-01T03:10:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.238+0000 7f631c499640 1 -- 172.21.15.115:0/619021535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63140d1230 msgr2=0x7f63140d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.238+0000 7f631c499640 1 --2- 172.21.15.115:0/619021535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63140d1230 0x7f63140d16a0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f631408bcc0 tx=0x7f630c02be30 comp rx=0 tx=0).stop 2023-12-01T03:10:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.238+0000 7f631c499640 1 -- 172.21.15.115:0/619021535 shutdown_connections 2023-12-01T03:10:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.238+0000 7f631c499640 1 --2- 172.21.15.115:0/619021535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63140d1230 0x7f63140d16a0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.238+0000 7f631c499640 1 --2- 172.21.15.115:0/619021535 >> v2:172.21.15.142:3300/0 conn(0x7f63140b5d80 0x7f63140b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.238+0000 7f631c499640 1 --2- 172.21.15.115:0/619021535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f631415cd20 0x7f63140d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.238+0000 7f631c499640 1 -- 172.21.15.115:0/619021535 >> 172.21.15.115:0/619021535 conn(0x7f631406a0e0 msgr2=0x7f631406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.239+0000 7f631c499640 1 -- 172.21.15.115:0/619021535 shutdown_connections 2023-12-01T03:10:10.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.239+0000 7f631c499640 1 -- 172.21.15.115:0/619021535 wait complete. 2023-12-01T03:10:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.241+0000 7f631c499640 1 Processor -- start 2023-12-01T03:10:10.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.241+0000 7f631c499640 1 -- start start 2023-12-01T03:10:10.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.242+0000 7f631c499640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63140b5d80 0x7f6314151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631c499640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63140d1230 0x7f631415db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631c499640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f631415cd20 0x7f631415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631c499640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631415f5c0 con 0x7f631415cd20 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631c499640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631415fa50 con 0x7f63140d1230 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631c499640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63141516a0 con 0x7f63140b5d80 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631aa36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f631415cd20 0x7f631415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631aa36640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f631415cd20 0x7f631415a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60616/0 (socket says 172.21.15.115:60616) 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631aa36640 1 -- 172.21.15.115:0/284591129 learned_addr learned my addr 172.21.15.115:0/284591129 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631a235640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63140b5d80 0x7f6314151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f6319a34640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63140d1230 0x7f631415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631aa36640 1 -- 172.21.15.115:0/284591129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63140b5d80 msgr2=0x7f6314151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631aa36640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63140b5d80 0x7f6314151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631aa36640 1 -- 172.21.15.115:0/284591129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63140d1230 msgr2=0x7f631415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631aa36640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63140d1230 0x7f631415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f631aa36640 1 -- 172.21.15.115:0/284591129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c009b30 con 0x7f631415cd20 2023-12-01T03:10:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.243+0000 7f6319a34640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63140d1230 0x7f631415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.244+0000 7f631aa36640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f631415cd20 0x7f631415a650 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f630c036040 tx=0x7f630c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.244+0000 7f630affd640 1 -- 172.21.15.115:0/284591129 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c009510 con 0x7f631415cd20 2023-12-01T03:10:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.244+0000 7f630affd640 1 -- 172.21.15.115:0/284591129 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c03e450 con 0x7f631415cd20 2023-12-01T03:10:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.244+0000 7f630affd640 1 -- 172.21.15.115:0/284591129 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c03e800 con 0x7f631415cd20 2023-12-01T03:10:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.244+0000 7f631c499640 1 -- 172.21.15.115:0/284591129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631415e9b0 con 0x7f631415cd20 2023-12-01T03:10:10.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.244+0000 7f631c499640 1 -- 172.21.15.115:0/284591129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631415ef70 con 0x7f631415cd20 2023-12-01T03:10:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.245+0000 7f631c499640 1 -- 172.21.15.115:0/284591129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631415f770 con 0x7f631415cd20 2023-12-01T03:10:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.246+0000 7f630affd640 1 -- 172.21.15.115:0/284591129 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 122) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f630c03e960 con 0x7f631415cd20 2023-12-01T03:10:10.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.247+0000 7f630affd640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f62f806a2b0 0x7f62f806c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.247+0000 7f630affd640 1 -- 172.21.15.115:0/284591129 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f630c0ab6a0 con 0x7f631415cd20 2023-12-01T03:10:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.247+0000 7f631a235640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f62f806a2b0 0x7f62f806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.248+0000 7f631a235640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f62f806a2b0 0x7f62f806c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6310006fd0 tx=0x7f6310008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:10.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.251+0000 7f630affd640 1 -- 172.21.15.115:0/284591129 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f631415f770 con 0x7f631415cd20 2023-12-01T03:10:10.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.481+0000 7f631c499640 1 -- 172.21.15.115:0/284591129 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f631404e890 con 0x7f631415cd20 2023-12-01T03:10:10.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.482+0000 7f630affd640 1 -- 172.21.15.115:0/284591129 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4661 (secure 0 0 0) 0x7f630c076bd0 con 0x7f631415cd20 2023-12-01T03:10:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:10.484 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/974148243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":974148243},{"type":"v1","addr":"172.21.15.115:6837","nonce":974148243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/476969842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":476969842},{"type":"v1","addr":"172.21.15.142:6834","nonce":476969842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1103499235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1103499235},{"type":"v1","addr":"172.21.15.115:6835","nonce":1103499235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:08:43.078263+0000","modified":"2023-12-01T03:08:46.096728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10149":{"gid":10149,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/3801423208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":3801423208},{"type":"v1","addr":"172.21.15.142:6835","nonce":3801423208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-01T03:10:10.484 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 87 2023-12-01T03:10:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 -- 172.21.15.115:0/284591129 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f62f806a2b0 msgr2=0x7f62f806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f62f806a2b0 0x7f62f806c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6310006fd0 tx=0x7f6310008040 comp rx=0 tx=0).stop 2023-12-01T03:10:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 -- 172.21.15.115:0/284591129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f631415cd20 msgr2=0x7f631415a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f631415cd20 0x7f631415a650 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f630c036040 tx=0x7f630c004a40 comp rx=0 tx=0).stop 2023-12-01T03:10:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 -- 172.21.15.115:0/284591129 shutdown_connections 2023-12-01T03:10:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f62f806a2b0 0x7f62f806c770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f631415cd20 0x7f631415a650 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63140d1230 0x7f631415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 --2- 172.21.15.115:0/284591129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f63140b5d80 0x7f6314151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 -- 172.21.15.115:0/284591129 >> 172.21.15.115:0/284591129 conn(0x7f631406a0e0 msgr2=0x7f6314157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:10.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 -- 172.21.15.115:0/284591129 shutdown_connections 2023-12-01T03:10:10.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.485+0000 7f631c499640 1 -- 172.21.15.115:0/284591129 wait complete. 2023-12-01T03:10:10.506 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:10.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.703+0000 7f1869d88640 1 Processor -- start 2023-12-01T03:10:10.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.703+0000 7f1869d88640 1 -- start start 2023-12-01T03:10:10.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.704+0000 7f1869d88640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f186415e2f0 0x7f18640ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.704+0000 7f1869d88640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 0x7f18640caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.704+0000 7f1869d88640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f18640adf10 0x7f18640c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.704+0000 7f1869d88640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18640add10 con 0x7f186415e2f0 2023-12-01T03:10:10.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.704+0000 7f1869d88640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18640c5760 con 0x7f18640adf10 2023-12-01T03:10:10.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.704+0000 7f1869d88640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186415e6d0 con 0x7f186406f2f0 2023-12-01T03:10:10.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.704+0000 7f18637fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f186415e2f0 0x7f18640ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.705+0000 7f18637fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f186415e2f0 0x7f18640ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60624/0 (socket says 172.21.15.115:60624) 2023-12-01T03:10:10.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.705+0000 7f18637fe640 1 -- 172.21.15.115:0/2696848284 learned_addr learned my addr 172.21.15.115:0/2696848284 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:10.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.705+0000 7f18637fe640 1 -- 172.21.15.115:0/2696848284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 msgr2=0x7f18640caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:10.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.705+0000 7f185bfff640 1 --2- 172.21.15.115:0/2696848284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 0x7f18640caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.705+0000 7f1863fff640 1 --2- 172.21.15.115:0/2696848284 >> v2:172.21.15.142:3300/0 conn(0x7f18640adf10 0x7f18640c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.705+0000 7f18637fe640 1 --2- 172.21.15.115:0/2696848284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 0x7f18640caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.705+0000 7f18637fe640 1 -- 172.21.15.115:0/2696848284 >> v2:172.21.15.142:3300/0 conn(0x7f18640adf10 msgr2=0x7f18640c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:10.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.705+0000 7f18637fe640 1 --2- 172.21.15.115:0/2696848284 >> v2:172.21.15.142:3300/0 conn(0x7f18640adf10 0x7f18640c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.705+0000 7f18637fe640 1 -- 172.21.15.115:0/2696848284 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18640cf7b0 con 0x7f186415e2f0 2023-12-01T03:10:10.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.705+0000 7f1863fff640 1 --2- 172.21.15.115:0/2696848284 >> v2:172.21.15.142:3300/0 conn(0x7f18640adf10 0x7f18640c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:10.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.706+0000 7f18637fe640 1 --2- 172.21.15.115:0/2696848284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f186415e2f0 0x7f18640ca660 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f18640cb2b0 tx=0x7f185402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81474ce5daf7fafd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:10.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1862ffd640 1 -- 172.21.15.115:0/2696848284 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854031750 con 0x7f186415e2f0 2023-12-01T03:10:10.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1862ffd640 1 -- 172.21.15.115:0/2696848284 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18540318b0 con 0x7f186415e2f0 2023-12-01T03:10:10.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1862ffd640 1 -- 172.21.15.115:0/2696848284 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854031c40 con 0x7f186415e2f0 2023-12-01T03:10:10.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1869d88640 1 -- 172.21.15.115:0/2696848284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f186415e2f0 msgr2=0x7f18640ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1869d88640 1 --2- 172.21.15.115:0/2696848284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f186415e2f0 0x7f18640ca660 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f18640cb2b0 tx=0x7f185402b7f0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1869d88640 1 -- 172.21.15.115:0/2696848284 shutdown_connections 2023-12-01T03:10:10.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1869d88640 1 --2- 172.21.15.115:0/2696848284 >> v2:172.21.15.142:3300/0 conn(0x7f18640adf10 0x7f18640c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1869d88640 1 --2- 172.21.15.115:0/2696848284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 0x7f18640caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1869d88640 1 --2- 172.21.15.115:0/2696848284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f186415e2f0 0x7f18640ca660 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1869d88640 1 -- 172.21.15.115:0/2696848284 >> 172.21.15.115:0/2696848284 conn(0x7f18640ce880 msgr2=0x7f18640cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:10.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.707+0000 7f1869d88640 1 -- 172.21.15.115:0/2696848284 shutdown_connections 2023-12-01T03:10:10.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.708+0000 7f1869d88640 1 -- 172.21.15.115:0/2696848284 wait complete. 2023-12-01T03:10:10.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.710+0000 7f1869d88640 1 Processor -- start 2023-12-01T03:10:10.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.710+0000 7f1869d88640 1 -- start start 2023-12-01T03:10:10.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.711+0000 7f1869d88640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 0x7f186415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.711+0000 7f1869d88640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18640adf10 0x7f186415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.711+0000 7f1869d88640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f186415e2f0 0x7f186415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.711+0000 7f1869d88640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18640c23b0 con 0x7f18640adf10 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.711+0000 7f1869d88640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186415d160 con 0x7f186415e2f0 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.711+0000 7f1869d88640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18640c2620 con 0x7f186406f2f0 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.711+0000 7f185bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18640adf10 0x7f186415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.711+0000 7f185bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18640adf10 0x7f186415c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60636/0 (socket says 172.21.15.115:60636) 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.711+0000 7f185bfff640 1 -- 172.21.15.115:0/496678466 learned_addr learned my addr 172.21.15.115:0/496678466 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f18637fe640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 0x7f186415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f1863fff640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f186415e2f0 0x7f186415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f185bfff640 1 -- 172.21.15.115:0/496678466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 msgr2=0x7f186415a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f185bfff640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 0x7f186415a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f185bfff640 1 -- 172.21.15.115:0/496678466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f186415e2f0 msgr2=0x7f186415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f185bfff640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f186415e2f0 0x7f186415cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f185bfff640 1 -- 172.21.15.115:0/496678466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540094d0 con 0x7f18640adf10 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f18637fe640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 0x7f186415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f1863fff640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f186415e2f0 0x7f186415cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f185bfff640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18640adf10 0x7f186415c6e0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f184c00c5a0 tx=0x7f184c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f1860ff9640 1 -- 172.21.15.115:0/496678466 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c004220 con 0x7f18640adf10 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f1860ff9640 1 -- 172.21.15.115:0/496678466 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c004380 con 0x7f18640adf10 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f1860ff9640 1 -- 172.21.15.115:0/496678466 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c003850 con 0x7f18640adf10 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f1869d88640 1 -- 172.21.15.115:0/496678466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186415d390 con 0x7f18640adf10 2023-12-01T03:10:10.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.712+0000 7f1869d88640 1 -- 172.21.15.115:0/496678466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186415d950 con 0x7f18640adf10 2023-12-01T03:10:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.713+0000 7f1869d88640 1 -- 172.21.15.115:0/496678466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186406fda0 con 0x7f18640adf10 2023-12-01T03:10:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.714+0000 7f1860ff9640 1 -- 172.21.15.115:0/496678466 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 123) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f184c0039b0 con 0x7f18640adf10 2023-12-01T03:10:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.715+0000 7f1860ff9640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f184406a4f0 0x7f184406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.715+0000 7f1860ff9640 1 -- 172.21.15.115:0/496678466 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f184c088a50 con 0x7f18640adf10 2023-12-01T03:10:10.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.716+0000 7f18637fe640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f184406a4f0 0x7f184406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:10.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.716+0000 7f18637fe640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f184406a4f0 0x7f184406c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1854035000 tx=0x7f1854008030 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:10.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.719+0000 7f1860ff9640 1 -- 172.21.15.115:0/496678466 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f184c053b10 con 0x7f18640adf10 2023-12-01T03:10:10.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.900+0000 7f1869d88640 1 -- 172.21.15.115:0/496678466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f186404e890 con 0x7f18640adf10 2023-12-01T03:10:10.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.902+0000 7f1860ff9640 1 -- 172.21.15.115:0/496678466 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+17396 (secure 0 0 0) 0x7f184c01e090 con 0x7f18640adf10 2023-12-01T03:10:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:10.904 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":201,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:07.759323+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:08:40.709195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-01T03:08:42.071749+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":200,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.18","mappings":[{"from":5,"to":3}]},{"pgid":"23.19","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/606861280":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/1640166514":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/2718416441":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/443657929":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/3570546488":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/3032781514":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/2184838711":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/288376877":"2023-12-02T03:10:07.759287+0000","172.21.15.115:0/361869044":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/4037780703":"2023-12-02T03:08:56.516548+0000","172.21.15.115:6824/138932":"2023-12-02T03:08:56.516548+0000","172.21.15.115:6822/138932":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/3414835041":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/1495322174":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/2764969208":"2023-12-02T03:08:56.516548+0000","172.21.15.115:0/1016167991":"2023-12-02T03:08:56.516548+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:10.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 -- 172.21.15.115:0/496678466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f184406a4f0 msgr2=0x7f184406c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f184406a4f0 0x7f184406c9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1854035000 tx=0x7f1854008030 comp rx=0 tx=0).stop 2023-12-01T03:10:10.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 -- 172.21.15.115:0/496678466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18640adf10 msgr2=0x7f186415c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:10.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18640adf10 0x7f186415c6e0 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f184c00c5a0 tx=0x7f184c00ca60 comp rx=0 tx=0).stop 2023-12-01T03:10:10.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 -- 172.21.15.115:0/496678466 shutdown_connections 2023-12-01T03:10:10.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f184406a4f0 0x7f184406c9b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f186415e2f0 0x7f186415cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18640adf10 0x7f186415c6e0 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 --2- 172.21.15.115:0/496678466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f186406f2f0 0x7f186415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:10.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 -- 172.21.15.115:0/496678466 >> 172.21.15.115:0/496678466 conn(0x7f18640ce880 msgr2=0x7f18640c98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:10.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.905+0000 7f1869d88640 1 -- 172.21.15.115:0/496678466 shutdown_connections 2023-12-01T03:10:10.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:10.906+0000 7f1869d88640 1 -- 172.21.15.115:0/496678466 wait complete. 2023-12-01T03:10:10.925 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T03:10:11.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.131+0000 7f72d73a9640 1 Processor -- start 2023-12-01T03:10:11.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.131+0000 7f72d73a9640 1 -- start start 2023-12-01T03:10:11.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.133+0000 7f72d73a9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72d00b5d80 0x7f72d00d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.133+0000 7f72d73a9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00d1230 0x7f72d00d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.133+0000 7f72d73a9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d00d1b90 con 0x7f72d015cd20 2023-12-01T03:10:11.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.133+0000 7f72d73a9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d00b2160 con 0x7f72d00b5d80 2023-12-01T03:10:11.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.133+0000 7f72d73a9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00b8c80 con 0x7f72d00d1230 2023-12-01T03:10:11.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.133+0000 7f72d5145640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f72d015cd20 0x7f72d00b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42594/0 (socket says 172.21.15.115:42594) 2023-12-01T03:10:11.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.133+0000 7f72d5145640 1 -- 172.21.15.115:0/2770418475 learned_addr learned my addr 172.21.15.115:0/2770418475 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634668161 0 0) 0x7f72c0001490 con 0x7f72d015cd20 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b8003520 con 0x7f72d015cd20 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72d4944640 1 --2- 172.21.15.115:0/2770418475 >> v2:172.21.15.142:3300/0 conn(0x7f72d00b5d80 0x7f72d00d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72d5946640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00d1230 0x7f72d00d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1099373591 0 0) 0x7f72b8003520 con 0x7f72d015cd20 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00d1230 msgr2=0x7f72d00d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00d1230 0x7f72d00d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 >> v2:172.21.15.142:3300/0 conn(0x7f72d00b5d80 msgr2=0x7f72d00d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 --2- 172.21.15.115:0/2770418475 >> v2:172.21.15.142:3300/0 conn(0x7f72d00b5d80 0x7f72d00d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d00ce2e0 con 0x7f72d015cd20 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f72c0001b80 con 0x7f72d015cd20 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 >> v1:172.21.15.115:6789/0 conn(0x7f72d015cd20 legacy=0x7f72d00b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b80039e0 0x7f72b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00d1230 0x7f72b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b8004d00 0x7f72b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00b8c80 con 0x7f72b80039e0 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b800bb40 con 0x7f72b8004d00 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b800bd00 con 0x7f72d00d1230 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.134+0000 7f72d5145640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b80039e0 0x7f72b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.135+0000 7f72d4944640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00d1230 0x7f72b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.135+0000 7f72d5946640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b8004d00 0x7f72b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.135+0000 7f72d5145640 1 -- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00d1230 msgr2=0x7f72b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.135+0000 7f72d5145640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00d1230 0x7f72b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.135+0000 7f72d5145640 1 -- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b8004d00 msgr2=0x7f72b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.135+0000 7f72d5145640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b8004d00 0x7f72b8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.135+0000 7f72d5145640 1 -- 172.21.15.115:0/2770418475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8003520 con 0x7f72b80039e0 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.136+0000 7f72d5145640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b80039e0 0x7f72b8003e00 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f72c000a780 tx=0x7f72b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb0bf37b2f2b54f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.136+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0034630 con 0x7f72b80039e0 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.136+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c002f070 con 0x7f72b80039e0 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.136+0000 7f72cffff640 1 -- 172.21.15.115:0/2770418475 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0034990 con 0x7f72b80039e0 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.137+0000 7f72d73a9640 1 -- 172.21.15.115:0/2770418475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b80039e0 msgr2=0x7f72b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.137+0000 7f72d73a9640 1 --2- 172.21.15.115:0/2770418475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72b80039e0 0x7f72b8003e00 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f72c000a780 tx=0x7f72b80057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.137+0000 7f72d5946640 1 -- 172.21.15.115:0/2770418475 reap_dead start 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.137+0000 7f72d73a9640 1 -- 172.21.15.115:0/2770418475 shutdown_connections 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.137+0000 7f72d73a9640 1 -- 172.21.15.115:0/2770418475 >> 172.21.15.115:0/2770418475 conn(0x7f72d006a0e0 msgr2=0x7f72d006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.137+0000 7f72d73a9640 1 -- 172.21.15.115:0/2770418475 shutdown_connections 2023-12-01T03:10:11.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.137+0000 7f72d73a9640 1 -- 172.21.15.115:0/2770418475 wait complete. 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.138+0000 7f72d73a9640 1 Processor -- start 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.139+0000 7f72d73a9640 1 -- start start 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.139+0000 7f72d73a9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b80039e0 0x7f72d0151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.139+0000 7f72d73a9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00b4e30 0x7f72d015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d73a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72d00cc210 0x7f72d00cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d73a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00ccbd0 con 0x7f72d00cc210 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d73a9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015da70 con 0x7f72b80039e0 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d73a9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015dd50 con 0x7f72d00b4e30 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d5946640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72d00cc210 0x7f72d00cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d5946640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72d00cc210 0x7f72d00cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60656/0 (socket says 172.21.15.115:60656) 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d5946640 1 -- 172.21.15.115:0/2024311461 learned_addr learned my addr 172.21.15.115:0/2024311461 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d4944640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00b4e30 0x7f72d015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d5145640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b80039e0 0x7f72d0151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d5946640 1 -- 172.21.15.115:0/2024311461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00b4e30 msgr2=0x7f72d015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d5946640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00b4e30 0x7f72d015d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d5946640 1 -- 172.21.15.115:0/2024311461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b80039e0 msgr2=0x7f72d0151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d5946640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b80039e0 0x7f72d0151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d5946640 1 -- 172.21.15.115:0/2024311461 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d015e030 con 0x7f72d00cc210 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.140+0000 7f72d4944640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00b4e30 0x7f72d015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.141+0000 7f72d5946640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72d00cc210 0x7f72d00cc690 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f72d00d37c0 tx=0x7f72c800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.141+0000 7f72cdffb640 1 -- 172.21.15.115:0/2024311461 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8004440 con 0x7f72d00cc210 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.141+0000 7f72cdffb640 1 -- 172.21.15.115:0/2024311461 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c800fe70 con 0x7f72d00cc210 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.141+0000 7f72cdffb640 1 -- 172.21.15.115:0/2024311461 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8003850 con 0x7f72d00cc210 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.141+0000 7f72d73a9640 1 -- 172.21.15.115:0/2024311461 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d015e310 con 0x7f72d00cc210 2023-12-01T03:10:11.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.141+0000 7f72d73a9640 1 -- 172.21.15.115:0/2024311461 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d0151680 con 0x7f72d00cc210 2023-12-01T03:10:11.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.142+0000 7f72d73a9640 1 -- 172.21.15.115:0/2024311461 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d00b3110 con 0x7f72d00cc210 2023-12-01T03:10:11.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.143+0000 7f72cdffb640 1 -- 172.21.15.115:0/2024311461 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 123) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f72c80039b0 con 0x7f72d00cc210 2023-12-01T03:10:11.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.144+0000 7f72cdffb640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f72bc06a4f0 0x7f72bc06c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.144+0000 7f72cdffb640 1 -- 172.21.15.115:0/2024311461 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f72c8088b50 con 0x7f72d00cc210 2023-12-01T03:10:11.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.145+0000 7f72d5145640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f72bc06a4f0 0x7f72bc06c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.145+0000 7f72d5145640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f72bc06a4f0 0x7f72bc06c9b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f72c0034000 tx=0x7f72c0009030 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:11.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.149+0000 7f72cdffb640 1 -- 172.21.15.115:0/2024311461 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f72d00b3110 con 0x7f72d00cc210 2023-12-01T03:10:11.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.378+0000 7f72d73a9640 1 -- 172.21.15.115:0/2024311461 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f72d00b34f0 con 0x7f72d00cc210 2023-12-01T03:10:11.380 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T03:10:11.379+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T03:10:11.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.396+0000 7f72cdffb640 1 -- 172.21.15.115:0/2024311461 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7f72d00b34f0 con 0x7f72d00cc210 2023-12-01T03:10:11.398 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 -- 172.21.15.115:0/2024311461 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f72bc06a4f0 msgr2=0x7f72bc06c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f72bc06a4f0 0x7f72bc06c9b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f72c0034000 tx=0x7f72c0009030 comp rx=0 tx=0).stop 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 -- 172.21.15.115:0/2024311461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72d00cc210 msgr2=0x7f72d00cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72d00cc210 0x7f72d00cc690 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f72d00d37c0 tx=0x7f72c800f670 comp rx=0 tx=0).stop 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 -- 172.21.15.115:0/2024311461 shutdown_connections 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f72bc06a4f0 0x7f72bc06c9b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72d00b4e30 0x7f72d015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72d00cc210 0x7f72d00cc690 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 --2- 172.21.15.115:0/2024311461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72b80039e0 0x7f72d0151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 -- 172.21.15.115:0/2024311461 >> 172.21.15.115:0/2024311461 conn(0x7f72d006a0e0 msgr2=0x7f72d00b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 -- 172.21.15.115:0/2024311461 shutdown_connections 2023-12-01T03:10:11.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.399+0000 7f72d73a9640 1 -- 172.21.15.115:0/2024311461 wait complete. 2023-12-01T03:10:11.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T03:10:11.520 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T03:10:11.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.628+0000 7fde5359e640 1 Processor -- start 2023-12-01T03:10:11.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.628+0000 7fde5359e640 1 -- start start 2023-12-01T03:10:11.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.629+0000 7fde5359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5415bdf0 0x7fde5415f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.629+0000 7fde5359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fde540cb7e0 0x7fde540b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.629+0000 7fde5359e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5415b040 con 0x7fde540addc0 2023-12-01T03:10:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.629+0000 7fde5359e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5415f5d0 con 0x7fde540cb7e0 2023-12-01T03:10:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.629+0000 7fde5359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde540ca020 con 0x7fde5415bdf0 2023-12-01T03:10:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.630+0000 7fde52d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fde540cb7e0 0x7fde540b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.630+0000 7fde52d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fde540cb7e0 0x7fde540b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60672/0 (socket says 172.21.15.115:60672) 2023-12-01T03:10:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.630+0000 7fde52d9d640 1 -- 172.21.15.115:0/3812568957 learned_addr learned my addr 172.21.15.115:0/3812568957 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.630+0000 7fde52d9d640 1 -- 172.21.15.115:0/3812568957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5415bdf0 msgr2=0x7fde5415f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:11.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.630+0000 7fde52d9d640 1 --2- 172.21.15.115:0/3812568957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5415bdf0 0x7fde5415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.630+0000 7fde52d9d640 1 -- 172.21.15.115:0/3812568957 >> v1:172.21.15.142:6789/0 conn(0x7fde540addc0 legacy=0x7fde540cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.630+0000 7fde52d9d640 1 -- 172.21.15.115:0/3812568957 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde540ae1a0 con 0x7fde540cb7e0 2023-12-01T03:10:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.631+0000 7fde52d9d640 1 --2- 172.21.15.115:0/3812568957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 0x7fde540b2bd0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009870 tx=0x7fde4c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9c9941a7b4e855 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5159a640 1 -- 172.21.15.115:0/3812568957 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c031640 con 0x7fde540cb7e0 2023-12-01T03:10:11.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5159a640 1 -- 172.21.15.115:0/3812568957 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4c0317a0 con 0x7fde540cb7e0 2023-12-01T03:10:11.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5159a640 1 -- 172.21.15.115:0/3812568957 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c031b30 con 0x7fde540cb7e0 2023-12-01T03:10:11.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5359e640 1 -- 172.21.15.115:0/3812568957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 msgr2=0x7fde540b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:11.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5359e640 1 --2- 172.21.15.115:0/3812568957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 0x7fde540b2bd0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009870 tx=0x7fde4c02b8f0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5359e640 1 -- 172.21.15.115:0/3812568957 shutdown_connections 2023-12-01T03:10:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5359e640 1 --2- 172.21.15.115:0/3812568957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 0x7fde540b2bd0 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5359e640 1 --2- 172.21.15.115:0/3812568957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5415bdf0 0x7fde5415f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5359e640 1 -- 172.21.15.115:0/3812568957 >> 172.21.15.115:0/3812568957 conn(0x7fde54065960 msgr2=0x7fde54065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5359e640 1 -- 172.21.15.115:0/3812568957 shutdown_connections 2023-12-01T03:10:11.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.632+0000 7fde5359e640 1 -- 172.21.15.115:0/3812568957 wait complete. 2023-12-01T03:10:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.634+0000 7fde5359e640 1 Processor -- start 2023-12-01T03:10:11.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.634+0000 7fde5359e640 1 -- start start 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.635+0000 7fde5359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde540addc0 0x7fde541f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.635+0000 7fde5359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 0x7fde5406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.635+0000 7fde5359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5415bdf0 0x7fde54070e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.635+0000 7fde5359e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde541f2440 con 0x7fde540cb7e0 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.635+0000 7fde5359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde541f29c0 con 0x7fde540addc0 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.635+0000 7fde5359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde540719c0 con 0x7fde5415bdf0 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.636+0000 7fde51d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 0x7fde5406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.636+0000 7fde51d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 0x7fde5406f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60688/0 (socket says 172.21.15.115:60688) 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.636+0000 7fde51d9b640 1 -- 172.21.15.115:0/2802240916 learned_addr learned my addr 172.21.15.115:0/2802240916 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.636+0000 7fde5259c640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde540addc0 0x7fde541f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.636+0000 7fde51d9b640 1 -- 172.21.15.115:0/2802240916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5415bdf0 msgr2=0x7fde54070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:11.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.636+0000 7fde51d9b640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5415bdf0 0x7fde54070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.636+0000 7fde51d9b640 1 -- 172.21.15.115:0/2802240916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde540addc0 msgr2=0x7fde541f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:11.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.636+0000 7fde51d9b640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde540addc0 0x7fde541f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:11.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.636+0000 7fde51d9b640 1 -- 172.21.15.115:0/2802240916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c0094d0 con 0x7fde540cb7e0 2023-12-01T03:10:11.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.636+0000 7fde51d9b640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 0x7fde5406f250 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fde4800b410 tx=0x7fde4800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:11.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.637+0000 7fde42ffd640 1 -- 172.21.15.115:0/2802240916 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48011830 con 0x7fde540cb7e0 2023-12-01T03:10:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.637+0000 7fde42ffd640 1 -- 172.21.15.115:0/2802240916 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde480119b0 con 0x7fde540cb7e0 2023-12-01T03:10:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.637+0000 7fde42ffd640 1 -- 172.21.15.115:0/2802240916 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48011d60 con 0x7fde540cb7e0 2023-12-01T03:10:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.637+0000 7fde5359e640 1 -- 172.21.15.115:0/2802240916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde54071ca0 con 0x7fde540cb7e0 2023-12-01T03:10:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.637+0000 7fde5359e640 1 -- 172.21.15.115:0/2802240916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde541f3390 con 0x7fde540cb7e0 2023-12-01T03:10:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.638+0000 7fde5359e640 1 -- 172.21.15.115:0/2802240916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde541f2b70 con 0x7fde540cb7e0 2023-12-01T03:10:11.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.639+0000 7fde42ffd640 1 -- 172.21.15.115:0/2802240916 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 123) v1 ==== 88653+0+0 (secure 0 0 0) 0x7fde4801a400 con 0x7fde540cb7e0 2023-12-01T03:10:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.640+0000 7fde42ffd640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde3406a4c0 0x7fde3406c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:11.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.640+0000 7fde42ffd640 1 -- 172.21.15.115:0/2802240916 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6613+0+0 (secure 0 0 0) 0x7fde48088ec0 con 0x7fde540cb7e0 2023-12-01T03:10:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.640+0000 7fde5259c640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde3406a4c0 0x7fde3406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:11.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.641+0000 7fde5259c640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde3406a4c0 0x7fde3406c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde44007fd0 tx=0x7fde44009040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:11.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.644+0000 7fde42ffd640 1 -- 172.21.15.115:0/2802240916 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fde541f2b70 con 0x7fde540cb7e0 2023-12-01T03:10:11.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:11.870+0000 7fde5359e640 1 -- 172.21.15.115:0/2802240916 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fde541f2b30 con 0x7fde540cb7e0 2023-12-01T03:10:12.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.545+0000 7fde42ffd640 1 -- 172.21.15.115:0/2802240916 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fde48054500 con 0x7fde540cb7e0 2023-12-01T03:10:12.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.548+0000 7fde42ffd640 1 -- 172.21.15.115:0/2802240916 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) v1 ==== 104+0+0 (secure 0 0 0) 0x7fde480713d0 con 0x7fde540cb7e0 2023-12-01T03:10:12.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.551+0000 7fde5359e640 1 -- 172.21.15.115:0/2802240916 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde3406a4c0 msgr2=0x7fde3406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.551+0000 7fde5359e640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde3406a4c0 0x7fde3406c980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde44007fd0 tx=0x7fde44009040 comp rx=0 tx=0).stop 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.551+0000 7fde5359e640 1 -- 172.21.15.115:0/2802240916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 msgr2=0x7fde5406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.551+0000 7fde5359e640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 0x7fde5406f250 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fde4800b410 tx=0x7fde4800b8d0 comp rx=0 tx=0).stop 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.551+0000 7fde5359e640 1 -- 172.21.15.115:0/2802240916 shutdown_connections 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.551+0000 7fde5359e640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde3406a4c0 0x7fde3406c980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.551+0000 7fde5359e640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde5415bdf0 0x7fde54070e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.551+0000 7fde5359e640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde540cb7e0 0x7fde5406f250 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.551+0000 7fde5359e640 1 --2- 172.21.15.115:0/2802240916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde540addc0 0x7fde541f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.552+0000 7fde5359e640 1 -- 172.21.15.115:0/2802240916 >> 172.21.15.115:0/2802240916 conn(0x7fde54065960 msgr2=0x7fde54065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.552+0000 7fde5359e640 1 -- 172.21.15.115:0/2802240916 shutdown_connections 2023-12-01T03:10:12.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.552+0000 7fde5359e640 1 -- 172.21.15.115:0/2802240916 wait complete. 2023-12-01T03:10:12.572 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T03:10:12.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.788+0000 7f8c8b7b1640 1 Processor -- start 2023-12-01T03:10:12.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.788+0000 7f8c8b7b1640 1 -- start start 2023-12-01T03:10:12.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.790+0000 7f8c8b7b1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8c840ca7c0 0x7f8c840cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.790+0000 7f8c8b7b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c840cb6f0 0x7f8c8406f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.790+0000 7f8c8b7b1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c840c9440 con 0x7f8c84159bf0 2023-12-01T03:10:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.790+0000 7f8c8b7b1640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c84159fd0 con 0x7f8c840ca7c0 2023-12-01T03:10:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.790+0000 7f8c8b7b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840cbb80 con 0x7f8c840cb6f0 2023-12-01T03:10:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.790+0000 7f8c88d4c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8c840ca7c0 0x7f8c840cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.790+0000 7f8c88d4c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8c840ca7c0 0x7f8c840cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60694/0 (socket says 172.21.15.115:60694) 2023-12-01T03:10:12.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.790+0000 7f8c88d4c640 1 -- 172.21.15.115:0/3656793705 learned_addr learned my addr 172.21.15.115:0/3656793705 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.790+0000 7f8c89d4e640 1 --2- 172.21.15.115:0/3656793705 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c840cb6f0 0x7f8c8406f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.791+0000 7f8c88d4c640 1 -- 172.21.15.115:0/3656793705 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c840cb6f0 msgr2=0x7f8c8406f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.791+0000 7f8c88d4c640 1 --2- 172.21.15.115:0/3656793705 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c840cb6f0 0x7f8c8406f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.791+0000 7f8c88d4c640 1 -- 172.21.15.115:0/3656793705 >> v1:172.21.15.142:6789/0 conn(0x7f8c84159bf0 legacy=0x7f8c8415e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:12.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.791+0000 7f8c88d4c640 1 -- 172.21.15.115:0/3656793705 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840c9720 con 0x7f8c840ca7c0 2023-12-01T03:10:12.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.792+0000 7f8c88d4c640 1 --2- 172.21.15.115:0/3656793705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 0x7f8c840cb1b0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400bb10 tx=0x7f8c7402e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6f77db8e90517e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:12.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.792+0000 7f8c7bfff640 1 -- 172.21.15.115:0/3656793705 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74036440 con 0x7f8c840ca7c0 2023-12-01T03:10:12.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.793+0000 7f8c7bfff640 1 -- 172.21.15.115:0/3656793705 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c740365a0 con 0x7f8c840ca7c0 2023-12-01T03:10:12.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.793+0000 7f8c7bfff640 1 -- 172.21.15.115:0/3656793705 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74036930 con 0x7f8c840ca7c0 2023-12-01T03:10:12.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.793+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/3656793705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 msgr2=0x7f8c840cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:12.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.793+0000 7f8c8b7b1640 1 --2- 172.21.15.115:0/3656793705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 0x7f8c840cb1b0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400bb10 tx=0x7f8c7402e150 comp rx=0 tx=0).stop 2023-12-01T03:10:12.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.793+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/3656793705 shutdown_connections 2023-12-01T03:10:12.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.793+0000 7f8c8b7b1640 1 --2- 172.21.15.115:0/3656793705 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c840cb6f0 0x7f8c8406f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:12.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.793+0000 7f8c8b7b1640 1 --2- 172.21.15.115:0/3656793705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 0x7f8c840cb1b0 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:12.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.793+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/3656793705 >> 172.21.15.115:0/3656793705 conn(0x7f8c840662c0 msgr2=0x7f8c840b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:12.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.793+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/3656793705 shutdown_connections 2023-12-01T03:10:12.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.793+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/3656793705 wait complete. 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.795+0000 7f8c8b7b1640 1 Processor -- start 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.795+0000 7f8c8b7b1640 1 -- start start 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.796+0000 7f8c8b7b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 0x7f8c8415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.796+0000 7f8c8b7b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c840cb6f0 0x7f8c840c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.796+0000 7f8c8b7b1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c84159bf0 0x7f8c840b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.796+0000 7f8c8b7b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c841f1fb0 con 0x7f8c840ca7c0 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.796+0000 7f8c8b7b1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c841ef720 con 0x7f8c84159bf0 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.796+0000 7f8c8b7b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c841f21c0 con 0x7f8c840cb6f0 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.796+0000 7f8c8954d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 0x7f8c8415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.796+0000 7f8c8954d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 0x7f8c8415a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60710/0 (socket says 172.21.15.115:60710) 2023-12-01T03:10:12.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.796+0000 7f8c8954d640 1 -- 172.21.15.115:0/763721952 learned_addr learned my addr 172.21.15.115:0/763721952 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c88d4c640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c840cb6f0 0x7f8c840c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c89d4e640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c84159bf0 0x7f8c840b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c8954d640 1 -- 172.21.15.115:0/763721952 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c840cb6f0 msgr2=0x7f8c840c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c8954d640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c840cb6f0 0x7f8c840c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c8954d640 1 -- 172.21.15.115:0/763721952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c84159bf0 msgr2=0x7f8c840b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c8954d640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c84159bf0 0x7f8c840b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c8954d640 1 -- 172.21.15.115:0/763721952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7400b770 con 0x7f8c840ca7c0 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c8954d640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 0x7f8c8415a120 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c400 tx=0x7f8c8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c79ffb640 1 -- 172.21.15.115:0/763721952 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c800052d0 con 0x7f8c840ca7c0 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c79ffb640 1 -- 172.21.15.115:0/763721952 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80005430 con 0x7f8c840ca7c0 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.797+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/763721952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c841f24a0 con 0x7f8c840ca7c0 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.798+0000 7f8c79ffb640 1 -- 172.21.15.115:0/763721952 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80003830 con 0x7f8c840ca7c0 2023-12-01T03:10:12.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.798+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/763721952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c841ef3d0 con 0x7f8c840ca7c0 2023-12-01T03:10:12.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.799+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/763721952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c841f2960 con 0x7f8c840ca7c0 2023-12-01T03:10:12.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.800+0000 7f8c79ffb640 1 -- 172.21.15.115:0/763721952 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8c80003990 con 0x7f8c840ca7c0 2023-12-01T03:10:12.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.800+0000 7f8c79ffb640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8c6806a580 0x7f8c6806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:12.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.801+0000 7f8c79ffb640 1 -- 172.21.15.115:0/763721952 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6613+0+0 (secure 0 0 0) 0x7f8c800883c0 con 0x7f8c840ca7c0 2023-12-01T03:10:12.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.801+0000 7f8c88d4c640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8c6806a580 0x7f8c6806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:12.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.802+0000 7f8c88d4c640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8c6806a580 0x7f8c6806ca40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c74038000 tx=0x7f8c74032000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:12.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.805+0000 7f8c79ffb640 1 -- 172.21.15.115:0/763721952 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8c80053510 con 0x7f8c840ca7c0 2023-12-01T03:10:13.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:12.999+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/763721952 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c84074210 con 0x7f8c840ca7c0 2023-12-01T03:10:13.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.549+0000 7f8c79ffb640 1 -- 172.21.15.115:0/763721952 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v204) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8c80056b30 con 0x7f8c840ca7c0 2023-12-01T03:10:13.552 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T03:10:13.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.554+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/763721952 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8c6806a580 msgr2=0x7f8c6806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.554+0000 7f8c8b7b1640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8c6806a580 0x7f8c6806ca40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c74038000 tx=0x7f8c74032000 comp rx=0 tx=0).stop 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.554+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/763721952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 msgr2=0x7f8c8415a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.554+0000 7f8c8b7b1640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 0x7f8c8415a120 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c400 tx=0x7f8c8000c8c0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.555+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/763721952 shutdown_connections 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.555+0000 7f8c8b7b1640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8c6806a580 0x7f8c6806ca40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.555+0000 7f8c8b7b1640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c840cb6f0 0x7f8c840c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.555+0000 7f8c8b7b1640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c84159bf0 0x7f8c840b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.555+0000 7f8c8b7b1640 1 --2- 172.21.15.115:0/763721952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c840ca7c0 0x7f8c8415a120 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.555+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/763721952 >> 172.21.15.115:0/763721952 conn(0x7f8c840662c0 msgr2=0x7f8c84156820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.555+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/763721952 shutdown_connections 2023-12-01T03:10:13.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.555+0000 7f8c8b7b1640 1 -- 172.21.15.115:0/763721952 wait complete. 2023-12-01T03:10:13.579 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T03:10:13.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.796+0000 7fd8e7bf0640 1 Processor -- start 2023-12-01T03:10:13.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.797+0000 7fd8e7bf0640 1 -- start start 2023-12-01T03:10:13.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.798+0000 7fd8e7bf0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd8e015ce40 0x7fd8e00b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:13.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.798+0000 7fd8e7bf0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2730 0x7fd8e00d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:13.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.798+0000 7fd8e7bf0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e00d1460 con 0x7fd8e00b4640 2023-12-01T03:10:13.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.798+0000 7fd8e7bf0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e015d220 con 0x7fd8e015ce40 2023-12-01T03:10:13.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.798+0000 7fd8e7bf0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00d1690 con 0x7fd8e00d2730 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e618d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd8e00b4640 0x7fd8e0069cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42606/0 (socket says 172.21.15.115:42606) 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e618d640 1 -- 172.21.15.115:0/3263257188 learned_addr learned my addr 172.21.15.115:0/3263257188 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e598c640 1 --2- 172.21.15.115:0/3263257188 >> v2:172.21.15.142:3300/0 conn(0x7fd8e015ce40 0x7fd8e00b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e518b640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2730 0x7fd8e00d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300861336 0 0) 0x7fd8e00d1460 con 0x7fd8e00b4640 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8cc003520 con 0x7fd8e00b4640 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4284095191 0 0) 0x7fd8cc003520 con 0x7fd8e00b4640 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2730 msgr2=0x7fd8e00d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e498a640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2730 0x7fd8e00d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 >> v2:172.21.15.142:3300/0 conn(0x7fd8e015ce40 msgr2=0x7fd8e00b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e498a640 1 --2- 172.21.15.115:0/3263257188 >> v2:172.21.15.142:3300/0 conn(0x7fd8e015ce40 0x7fd8e00b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e00d1970 con 0x7fd8e00b4640 2023-12-01T03:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.799+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd8d80019f0 con 0x7fd8e00b4640 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 >> v1:172.21.15.115:6789/0 conn(0x7fd8e00b4640 legacy=0x7fd8e0069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e498a640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8cc003970 0x7fd8cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e498a640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8cc0047c0 0x7fd8cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e498a640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2730 0x7fd8cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e015d220 con 0x7fd8cc0047c0 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00d1690 con 0x7fd8cc003970 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc00ba30 con 0x7fd8e00d2730 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e518b640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8cc0047c0 0x7fd8cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e598c640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8cc003970 0x7fd8cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e618d640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2730 0x7fd8cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e518b640 1 -- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2730 msgr2=0x7fd8cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e518b640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2730 0x7fd8cc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e518b640 1 -- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8cc003970 msgr2=0x7fd8cc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:13.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e518b640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8cc003970 0x7fd8cc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e518b640 1 -- 172.21.15.115:0/3263257188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc003520 con 0x7fd8cc0047c0 2023-12-01T03:10:13.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.800+0000 7fd8e618d640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8e00d2730 0x7fd8cc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:13.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.802+0000 7fd8e518b640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8cc0047c0 0x7fd8cc004c40 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00d2b90 tx=0x7fd8cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f2d5a52786df76 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:13.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.802+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0034610 con 0x7fd8cc0047c0 2023-12-01T03:10:13.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.802+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d002e070 con 0x7fd8cc0047c0 2023-12-01T03:10:13.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.802+0000 7fd8e498a640 1 -- 172.21.15.115:0/3263257188 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0034990 con 0x7fd8cc0047c0 2023-12-01T03:10:13.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.803+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3263257188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8cc0047c0 msgr2=0x7fd8cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.803+0000 7fd8e7bf0640 1 --2- 172.21.15.115:0/3263257188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8cc0047c0 0x7fd8cc004c40 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00d2b90 tx=0x7fd8cc0057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.803+0000 7fd8e618d640 1 -- 172.21.15.115:0/3263257188 reap_dead start 2023-12-01T03:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.803+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3263257188 shutdown_connections 2023-12-01T03:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.803+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3263257188 >> 172.21.15.115:0/3263257188 conn(0x7fd8e0065f90 msgr2=0x7fd8e00cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.803+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3263257188 shutdown_connections 2023-12-01T03:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.803+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3263257188 wait complete. 2023-12-01T03:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.805+0000 7fd8e7bf0640 1 Processor -- start 2023-12-01T03:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.805+0000 7fd8e7bf0640 1 -- start start 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e7bf0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8cc0047c0 0x7fd8e015d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e7bf0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015f2d0 0x7fd8e015f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e7bf0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e01f1530 0x7fd8e01f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e7bf0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e01603e0 con 0x7fd8e01f1530 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e7bf0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e015fe20 con 0x7fd8e015f2d0 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e7bf0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0160100 con 0x7fd8cc0047c0 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e618d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e01f1530 0x7fd8e01f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e618d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e01f1530 0x7fd8e01f19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60728/0 (socket says 172.21.15.115:60728) 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e618d640 1 -- 172.21.15.115:0/3553571893 learned_addr learned my addr 172.21.15.115:0/3553571893 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e518b640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015f2d0 0x7fd8e015f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e598c640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8cc0047c0 0x7fd8e015d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e618d640 1 -- 172.21.15.115:0/3553571893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8cc0047c0 msgr2=0x7fd8e015d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e618d640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8cc0047c0 0x7fd8e015d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e618d640 1 -- 172.21.15.115:0/3553571893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015f2d0 msgr2=0x7fd8e015f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.806+0000 7fd8e618d640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015f2d0 0x7fd8e015f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.807+0000 7fd8e618d640 1 -- 172.21.15.115:0/3553571893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e00c78e0 con 0x7fd8e01f1530 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.807+0000 7fd8e618d640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e01f1530 0x7fd8e01f19b0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00d3ca0 tx=0x7fd8d800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.807+0000 7fd8d67fc640 1 -- 172.21.15.115:0/3553571893 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8001990 con 0x7fd8e01f1530 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.807+0000 7fd8d67fc640 1 -- 172.21.15.115:0/3553571893 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d800bb80 con 0x7fd8e01f1530 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.807+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3553571893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e00c7bc0 con 0x7fd8e01f1530 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.807+0000 7fd8d67fc640 1 -- 172.21.15.115:0/3553571893 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8007070 con 0x7fd8e01f1530 2023-12-01T03:10:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.807+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3553571893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e0146200 con 0x7fd8e01f1530 2023-12-01T03:10:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.808+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3553571893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e00c7740 con 0x7fd8e01f1530 2023-12-01T03:10:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.809+0000 7fd8d67fc640 1 -- 172.21.15.115:0/3553571893 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd8d80071d0 con 0x7fd8e01f1530 2023-12-01T03:10:13.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.810+0000 7fd8d67fc640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd8c406a550 0x7fd8c406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:13.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.810+0000 7fd8d67fc640 1 -- 172.21.15.115:0/3553571893 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6147+0+0 (secure 0 0 0) 0x7fd8d8089200 con 0x7fd8e01f1530 2023-12-01T03:10:13.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.810+0000 7fd8e598c640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd8c406a550 0x7fd8c406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:13.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.811+0000 7fd8e598c640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd8c406a550 0x7fd8c406ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0099d0 tx=0x7fd8dc008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:13.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:13.814+0000 7fd8d67fc640 1 -- 172.21.15.115:0/3553571893 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd8e00c7740 con 0x7fd8e01f1530 2023-12-01T03:10:14.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.003+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3553571893 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8e00c7740 con 0x7fd8e01f1530 2023-12-01T03:10:14.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.562+0000 7fd8d67fc640 1 -- 172.21.15.115:0/3553571893 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v205) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd8e00c7740 con 0x7fd8e01f1530 2023-12-01T03:10:14.567 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T03:10:14.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.569+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3553571893 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd8c406a550 msgr2=0x7fd8c406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:14.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.569+0000 7fd8e7bf0640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd8c406a550 0x7fd8c406ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0099d0 tx=0x7fd8dc008040 comp rx=0 tx=0).stop 2023-12-01T03:10:14.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.569+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3553571893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e01f1530 msgr2=0x7fd8e01f19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:14.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.569+0000 7fd8e7bf0640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e01f1530 0x7fd8e01f19b0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00d3ca0 tx=0x7fd8d800b4a0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.569+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3553571893 shutdown_connections 2023-12-01T03:10:14.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.569+0000 7fd8e7bf0640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd8c406a550 0x7fd8c406ca10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.569+0000 7fd8e7bf0640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8e01f1530 0x7fd8e01f19b0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.569+0000 7fd8e7bf0640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8e015f2d0 0x7fd8e015f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.569+0000 7fd8e7bf0640 1 --2- 172.21.15.115:0/3553571893 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8cc0047c0 0x7fd8e015d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.569+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3553571893 >> 172.21.15.115:0/3553571893 conn(0x7fd8e0065f90 msgr2=0x7fd8e00b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:14.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.571+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3553571893 shutdown_connections 2023-12-01T03:10:14.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.571+0000 7fd8e7bf0640 1 -- 172.21.15.115:0/3553571893 wait complete. 2023-12-01T03:10:14.594 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T03:10:14.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.790+0000 7f64fb2ce640 1 Processor -- start 2023-12-01T03:10:14.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.791+0000 7f64fb2ce640 1 -- start start 2023-12-01T03:10:14.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.792+0000 7f64fb2ce640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f64f415cd90 0x7f64f40cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:14.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.792+0000 7f64fb2ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f40cfd60 0x7f64f4161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:14.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.792+0000 7f64fb2ce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f40ce5e0 con 0x7f64f40b5fa0 2023-12-01T03:10:14.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.792+0000 7f64fb2ce640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f40d0ec0 con 0x7f64f415cd90 2023-12-01T03:10:14.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.792+0000 7f64fb2ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4161950 con 0x7f64f40cfd60 2023-12-01T03:10:14.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.792+0000 7f64f8869640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f64f40b5fa0 0x7f64f40cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42614/0 (socket says 172.21.15.115:42614) 2023-12-01T03:10:14.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.792+0000 7f64f8869640 1 -- 172.21.15.115:0/261346163 learned_addr learned my addr 172.21.15.115:0/261346163 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9884455 0 0) 0x7f64e0001110 con 0x7f64f40b5fa0 2023-12-01T03:10:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e4003520 con 0x7f64f40b5fa0 2023-12-01T03:10:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f906a640 1 --2- 172.21.15.115:0/261346163 >> v2:172.21.15.142:3300/0 conn(0x7f64f415cd90 0x7f64f40cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f986b640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f40cfd60 0x7f64f4161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:14.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2768968846 0 0) 0x7f64e4003520 con 0x7f64f40b5fa0 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f40cfd60 msgr2=0x7f64f4161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f40cfd60 0x7f64f4161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 >> v2:172.21.15.142:3300/0 conn(0x7f64f415cd90 msgr2=0x7f64f40cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 --2- 172.21.15.115:0/261346163 >> v2:172.21.15.142:3300/0 conn(0x7f64f415cd90 0x7f64f40cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f40ce300 con 0x7f64f40b5fa0 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f64e0001800 con 0x7f64f40b5fa0 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 >> v1:172.21.15.115:6789/0 conn(0x7f64f40b5fa0 legacy=0x7f64f40cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64e4003970 0x7f64e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64e40047c0 0x7f64e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f40cfd60 0x7f64e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f40d0ec0 con 0x7f64e4003970 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4161950 con 0x7f64e40047c0 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.793+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e400ba30 con 0x7f64f40cfd60 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.794+0000 7f64f906a640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64e4003970 0x7f64e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:14.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.794+0000 7f64f986b640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f40cfd60 0x7f64e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:14.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.794+0000 7f64f906a640 1 -- 172.21.15.115:0/261346163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f40cfd60 msgr2=0x7f64e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:14.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.794+0000 7f64f906a640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64f40cfd60 0x7f64e4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.794+0000 7f64f906a640 1 -- 172.21.15.115:0/261346163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64e40047c0 msgr2=0x7f64e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:14.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.794+0000 7f64f906a640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64e40047c0 0x7f64e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.794+0000 7f64f906a640 1 -- 172.21.15.115:0/261346163 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4003520 con 0x7f64e4003970 2023-12-01T03:10:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.795+0000 7f64f906a640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64e4003970 0x7f64e4003d90 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f64e8007330 tx=0x7f64e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71350c36de9abec server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.795+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e802ccb0 con 0x7f64e4003970 2023-12-01T03:10:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.795+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e802ce10 con 0x7f64e4003970 2023-12-01T03:10:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.795+0000 7f64f3fff640 1 -- 172.21.15.115:0/261346163 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8034660 con 0x7f64e4003970 2023-12-01T03:10:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.796+0000 7f64fb2ce640 1 -- 172.21.15.115:0/261346163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64e4003970 msgr2=0x7f64e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.796+0000 7f64fb2ce640 1 --2- 172.21.15.115:0/261346163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64e4003970 0x7f64e4003d90 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f64e8007330 tx=0x7f64e40057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.796+0000 7f64f986b640 1 -- 172.21.15.115:0/261346163 reap_dead start 2023-12-01T03:10:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.796+0000 7f64fb2ce640 1 -- 172.21.15.115:0/261346163 shutdown_connections 2023-12-01T03:10:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.796+0000 7f64fb2ce640 1 -- 172.21.15.115:0/261346163 >> 172.21.15.115:0/261346163 conn(0x7f64f4065960 msgr2=0x7f64f4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:14.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.796+0000 7f64fb2ce640 1 -- 172.21.15.115:0/261346163 shutdown_connections 2023-12-01T03:10:14.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.796+0000 7f64fb2ce640 1 -- 172.21.15.115:0/261346163 wait complete. 2023-12-01T03:10:14.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.798+0000 7f64fb2ce640 1 Processor -- start 2023-12-01T03:10:14.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.798+0000 7f64fb2ce640 1 -- start start 2023-12-01T03:10:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.799+0000 7f64fb2ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64e4003970 0x7f64f4154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.799+0000 7f64fb2ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f40cade0 0x7f64f40cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.799+0000 7f64fb2ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f415b430 0x7f64f415b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.799+0000 7f64fb2ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f415bdf0 con 0x7f64f415b430 2023-12-01T03:10:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.799+0000 7f64fb2ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4141490 con 0x7f64f40cade0 2023-12-01T03:10:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.799+0000 7f64fb2ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f40cb830 con 0x7f64e4003970 2023-12-01T03:10:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.799+0000 7f64f986b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f415b430 0x7f64f415b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.799+0000 7f64f986b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f415b430 0x7f64f415b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60738/0 (socket says 172.21.15.115:60738) 2023-12-01T03:10:14.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.799+0000 7f64f986b640 1 -- 172.21.15.115:0/4026921574 learned_addr learned my addr 172.21.15.115:0/4026921574 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f906a640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64e4003970 0x7f64f4154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f8869640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f40cade0 0x7f64f40cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f986b640 1 -- 172.21.15.115:0/4026921574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64e4003970 msgr2=0x7f64f4154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f986b640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64e4003970 0x7f64f4154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f986b640 1 -- 172.21.15.115:0/4026921574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f40cade0 msgr2=0x7f64f40cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f986b640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f40cade0 0x7f64f40cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f986b640 1 -- 172.21.15.115:0/4026921574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f40cbb10 con 0x7f64f415b430 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f8869640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f40cade0 0x7f64f40cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f906a640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64e4003970 0x7f64f4154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f986b640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f415b430 0x7f64f415b8b0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006fd0 tx=0x7f64ec00e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.800+0000 7f64f1ffb640 1 -- 172.21.15.115:0/4026921574 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec00c070 con 0x7f64f415b430 2023-12-01T03:10:14.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.801+0000 7f64f1ffb640 1 -- 172.21.15.115:0/4026921574 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec013610 con 0x7f64f415b430 2023-12-01T03:10:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.801+0000 7f64f1ffb640 1 -- 172.21.15.115:0/4026921574 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0139c0 con 0x7f64f415b430 2023-12-01T03:10:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.801+0000 7f64fb2ce640 1 -- 172.21.15.115:0/4026921574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f4141120 con 0x7f64f415b430 2023-12-01T03:10:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.801+0000 7f64fb2ce640 1 -- 172.21.15.115:0/4026921574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f4154850 con 0x7f64f415b430 2023-12-01T03:10:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.802+0000 7f64fb2ce640 1 -- 172.21.15.115:0/4026921574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f4140f80 con 0x7f64f415b430 2023-12-01T03:10:14.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.803+0000 7f64f1ffb640 1 -- 172.21.15.115:0/4026921574 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f64ec013b20 con 0x7f64f415b430 2023-12-01T03:10:14.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.804+0000 7f64f1ffb640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f64d806a550 0x7f64d806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:14.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.804+0000 7f64f1ffb640 1 -- 172.21.15.115:0/4026921574 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5675+0+0 (secure 0 0 0) 0x7f64ec088300 con 0x7f64f415b430 2023-12-01T03:10:14.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.804+0000 7f64f906a640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f64d806a550 0x7f64d806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:14.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.805+0000 7f64f906a640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f64d806a550 0x7f64d806ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64e8036000 tx=0x7f64e8030000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:14.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:14.808+0000 7f64f1ffb640 1 -- 172.21.15.115:0/4026921574 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f64f4140f80 con 0x7f64f415b430 2023-12-01T03:10:15.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.002+0000 7f64fb2ce640 1 -- 172.21.15.115:0/4026921574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f64f4140f80 con 0x7f64f415b430 2023-12-01T03:10:15.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.008+0000 7f64f1ffb640 1 -- 172.21.15.115:0/4026921574 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f64f4140f80 con 0x7f64f415b430 2023-12-01T03:10:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 -- 172.21.15.115:0/4026921574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f64d806a550 msgr2=0x7f64d806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f64d806a550 0x7f64d806ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64e8036000 tx=0x7f64e8030000 comp rx=0 tx=0).stop 2023-12-01T03:10:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 -- 172.21.15.115:0/4026921574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f415b430 msgr2=0x7f64f415b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f415b430 0x7f64f415b8b0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006fd0 tx=0x7f64ec00e8b0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 -- 172.21.15.115:0/4026921574 shutdown_connections 2023-12-01T03:10:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f64d806a550 0x7f64d806ca10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64f415b430 0x7f64f415b8b0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64f40cade0 0x7f64f40cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 --2- 172.21.15.115:0/4026921574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f64e4003970 0x7f64f4154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 -- 172.21.15.115:0/4026921574 >> 172.21.15.115:0/4026921574 conn(0x7f64f4065960 msgr2=0x7f64f41598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:15.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 -- 172.21.15.115:0/4026921574 shutdown_connections 2023-12-01T03:10:15.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.011+0000 7f64fb2ce640 1 -- 172.21.15.115:0/4026921574 wait complete. 2023-12-01T03:10:15.031 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T03:10:15.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.245+0000 7fa9267bf640 1 Processor -- start 2023-12-01T03:10:15.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.246+0000 7fa9267bf640 1 -- start start 2023-12-01T03:10:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.247+0000 7fa9267bf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa92015ac80 0x7fa9200cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.247+0000 7fa9267bf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9200cc9a0 con 0x7fa9200b0e60 2023-12-01T03:10:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.247+0000 7fa9267bf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9200cbd90 con 0x7fa9200ce1b0 2023-12-01T03:10:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.247+0000 7fa9267bf640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92015b060 con 0x7fa92015ac80 2023-12-01T03:10:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.247+0000 7fa91ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa92015ac80 0x7fa9200cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.248+0000 7fa917fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa9200b0e60 0x7fa9200cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42622/0 (socket says 172.21.15.115:42622) 2023-12-01T03:10:15.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.248+0000 7fa917fff640 1 -- 172.21.15.115:0/282175324 learned_addr learned my addr 172.21.15.115:0/282175324 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.248+0000 7fa91f7fe640 1 -- 172.21.15.115:0/282175324 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421558258 0 0) 0x7fa9200cc9a0 con 0x7fa9200b0e60 2023-12-01T03:10:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.248+0000 7fa91ffff640 1 -- 172.21.15.115:0/282175324 >> v1:172.21.15.142:6789/0 conn(0x7fa9200ce1b0 legacy=0x7fa92006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.248+0000 7fa91ffff640 1 -- 172.21.15.115:0/282175324 >> v1:172.21.15.115:6789/0 conn(0x7fa9200b0e60 legacy=0x7fa9200cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.248+0000 7fa91ffff640 1 -- 172.21.15.115:0/282175324 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9200cc070 con 0x7fa92015ac80 2023-12-01T03:10:15.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.249+0000 7fa91ffff640 1 --2- 172.21.15.115:0/282175324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa92015ac80 0x7fa9200cce30 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097a0 tx=0x7fa91002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c4981fdcafd152 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:15.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.250+0000 7fa91f7fe640 1 -- 172.21.15.115:0/282175324 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910031830 con 0x7fa92015ac80 2023-12-01T03:10:15.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.250+0000 7fa91f7fe640 1 -- 172.21.15.115:0/282175324 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910031990 con 0x7fa92015ac80 2023-12-01T03:10:15.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.250+0000 7fa91f7fe640 1 -- 172.21.15.115:0/282175324 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910031d20 con 0x7fa92015ac80 2023-12-01T03:10:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.250+0000 7fa9267bf640 1 -- 172.21.15.115:0/282175324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa92015ac80 msgr2=0x7fa9200cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.250+0000 7fa9267bf640 1 --2- 172.21.15.115:0/282175324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa92015ac80 0x7fa9200cce30 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097a0 tx=0x7fa91002b8d0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.250+0000 7fa9267bf640 1 -- 172.21.15.115:0/282175324 shutdown_connections 2023-12-01T03:10:15.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.250+0000 7fa9267bf640 1 --2- 172.21.15.115:0/282175324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa92015ac80 0x7fa9200cce30 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.250+0000 7fa9267bf640 1 -- 172.21.15.115:0/282175324 >> 172.21.15.115:0/282175324 conn(0x7fa920065960 msgr2=0x7fa920065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:15.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.250+0000 7fa9267bf640 1 -- 172.21.15.115:0/282175324 shutdown_connections 2023-12-01T03:10:15.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.250+0000 7fa9267bf640 1 -- 172.21.15.115:0/282175324 wait complete. 2023-12-01T03:10:15.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.252+0000 7fa9267bf640 1 Processor -- start 2023-12-01T03:10:15.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.252+0000 7fa9267bf640 1 -- start start 2023-12-01T03:10:15.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.253+0000 7fa9267bf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9200b0e60 0x7fa9201f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.253+0000 7fa9267bf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9200ce1b0 0x7fa920070120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.253+0000 7fa9267bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa92015ac80 0x7fa920070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.253+0000 7fa9267bf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9201f3680 con 0x7fa9200b0e60 2023-12-01T03:10:15.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.253+0000 7fa9267bf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9201460b0 con 0x7fa92015ac80 2023-12-01T03:10:15.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.253+0000 7fa9267bf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa920146390 con 0x7fa9200ce1b0 2023-12-01T03:10:15.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.253+0000 7fa91ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9200b0e60 0x7fa9201f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:15.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.253+0000 7fa91ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9200b0e60 0x7fa9201f5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60758/0 (socket says 172.21.15.115:60758) 2023-12-01T03:10:15.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.253+0000 7fa91ffff640 1 -- 172.21.15.115:0/2255834240 learned_addr learned my addr 172.21.15.115:0/2255834240 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:15.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.254+0000 7fa91ffff640 1 -- 172.21.15.115:0/2255834240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9200ce1b0 msgr2=0x7fa920070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.254+0000 7fa91ffff640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9200ce1b0 0x7fa920070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.254+0000 7fa91ffff640 1 -- 172.21.15.115:0/2255834240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa92015ac80 msgr2=0x7fa920070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.254+0000 7fa91ffff640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa92015ac80 0x7fa920070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.254+0000 7fa91ffff640 1 -- 172.21.15.115:0/2255834240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910009400 con 0x7fa9200b0e60 2023-12-01T03:10:15.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.254+0000 7fa91ffff640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9200b0e60 0x7fa9201f5630 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097a0 tx=0x7fa9100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:15.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.254+0000 7fa91d7fa640 1 -- 172.21.15.115:0/2255834240 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100317d0 con 0x7fa9200b0e60 2023-12-01T03:10:15.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.255+0000 7fa91d7fa640 1 -- 172.21.15.115:0/2255834240 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910004030 con 0x7fa9200b0e60 2023-12-01T03:10:15.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.255+0000 7fa9267bf640 1 -- 172.21.15.115:0/2255834240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa920146670 con 0x7fa9200b0e60 2023-12-01T03:10:15.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.255+0000 7fa91d7fa640 1 -- 172.21.15.115:0/2255834240 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100043e0 con 0x7fa9200b0e60 2023-12-01T03:10:15.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.255+0000 7fa9267bf640 1 -- 172.21.15.115:0/2255834240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9201f3800 con 0x7fa9200b0e60 2023-12-01T03:10:15.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.256+0000 7fa9267bf640 1 -- 172.21.15.115:0/2255834240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa920146b90 con 0x7fa9200b0e60 2023-12-01T03:10:15.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.257+0000 7fa91d7fa640 1 -- 172.21.15.115:0/2255834240 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa910043070 con 0x7fa9200b0e60 2023-12-01T03:10:15.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.258+0000 7fa91d7fa640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa90006a580 0x7fa90006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.258+0000 7fa91d7fa640 1 -- 172.21.15.115:0/2255834240 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5675+0+0 (secure 0 0 0) 0x7fa9100aa8d0 con 0x7fa9200b0e60 2023-12-01T03:10:15.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.258+0000 7fa917fff640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa90006a580 0x7fa90006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:15.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.259+0000 7fa917fff640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa90006a580 0x7fa90006ca40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa908007fd0 tx=0x7fa908007920 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:15.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.263+0000 7fa91d7fa640 1 -- 172.21.15.115:0/2255834240 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa910075cc0 con 0x7fa9200b0e60 2023-12-01T03:10:15.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.451+0000 7fa9267bf640 1 -- 172.21.15.115:0/2255834240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fa9201464d0 con 0x7fa9200b0e60 2023-12-01T03:10:15.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.460+0000 7fa91d7fa640 1 -- 172.21.15.115:0/2255834240 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v68) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa9201464d0 con 0x7fa9200b0e60 2023-12-01T03:10:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.463+0000 7fa9267bf640 1 -- 172.21.15.115:0/2255834240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa90006a580 msgr2=0x7fa90006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.463+0000 7fa9267bf640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa90006a580 0x7fa90006ca40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa908007fd0 tx=0x7fa908007920 comp rx=0 tx=0).stop 2023-12-01T03:10:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.463+0000 7fa9267bf640 1 -- 172.21.15.115:0/2255834240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9200b0e60 msgr2=0x7fa9201f5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.463+0000 7fa9267bf640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9200b0e60 0x7fa9201f5630 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097a0 tx=0x7fa9100046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.463+0000 7fa9267bf640 1 -- 172.21.15.115:0/2255834240 shutdown_connections 2023-12-01T03:10:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.463+0000 7fa9267bf640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa90006a580 0x7fa90006ca40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.463+0000 7fa9267bf640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9200ce1b0 0x7fa920070120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.463+0000 7fa9267bf640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa92015ac80 0x7fa920070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.463+0000 7fa9267bf640 1 --2- 172.21.15.115:0/2255834240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa9200b0e60 0x7fa9201f5630 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.463+0000 7fa9267bf640 1 -- 172.21.15.115:0/2255834240 >> 172.21.15.115:0/2255834240 conn(0x7fa920065960 msgr2=0x7fa92006bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.464+0000 7fa9267bf640 1 -- 172.21.15.115:0/2255834240 shutdown_connections 2023-12-01T03:10:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.464+0000 7fa9267bf640 1 -- 172.21.15.115:0/2255834240 wait complete. 2023-12-01T03:10:15.484 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T03:10:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.694+0000 7f5423320640 1 Processor -- start 2023-12-01T03:10:15.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.694+0000 7f5423320640 1 -- start start 2023-12-01T03:10:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.695+0000 7f5423320640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f541c15b0e0 0x7f541c164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.695+0000 7f5423320640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0b2f30 con 0x7f541c0cd1e0 2023-12-01T03:10:15.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.696+0000 7f5423320640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0b2320 con 0x7f541c142870 2023-12-01T03:10:15.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.696+0000 7f5423320640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541c0b3ae0 con 0x7f541c15b0e0 2023-12-01T03:10:15.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.696+0000 7f54208bb640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f541c0cd1e0 0x7f541c0b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:42626/0 (socket says 172.21.15.115:42626) 2023-12-01T03:10:15.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.696+0000 7f54208bb640 1 -- 172.21.15.115:0/3673983878 learned_addr learned my addr 172.21.15.115:0/3673983878 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:15.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.696+0000 7f54218bd640 1 --2- 172.21.15.115:0/3673983878 >> v2:172.21.15.142:3300/0 conn(0x7f541c15b0e0 0x7f541c164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655525193 0 0) 0x7f541c0b2f30 con 0x7f541c0cd1e0 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5408003520 con 0x7f541c0cd1e0 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 755651087 0 0) 0x7f5408003520 con 0x7f541c0cd1e0 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 >> v2:172.21.15.142:3300/0 conn(0x7f541c15b0e0 msgr2=0x7f541c164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 --2- 172.21.15.115:0/3673983878 >> v2:172.21.15.142:3300/0 conn(0x7f541c15b0e0 0x7f541c164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 >> v1:172.21.15.142:6789/0 conn(0x7f541c142870 legacy=0x7f541c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c0b2600 con 0x7f541c0cd1e0 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f540c001670 con 0x7f541c0cd1e0 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 >> v1:172.21.15.115:6789/0 conn(0x7f541c0cd1e0 legacy=0x7f541c0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 --2- 172.21.15.115:0/3673983878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5408003970 0x7f5408003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 --2- 172.21.15.115:0/3673983878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54080047c0 0x7f5408004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 --2- 172.21.15.115:0/3673983878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5408005180 0x7f540800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0b3ae0 con 0x7f5408005180 2023-12-01T03:10:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540800c020 con 0x7f5408003970 2023-12-01T03:10:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.697+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540800c1e0 con 0x7f54080047c0 2023-12-01T03:10:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.698+0000 7f54218bd640 1 --2- 172.21.15.115:0/3673983878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5408005180 0x7f540800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.698+0000 7f54210bc640 1 --2- 172.21.15.115:0/3673983878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5408003970 0x7f5408003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.698+0000 7f54218bd640 1 -- 172.21.15.115:0/3673983878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54080047c0 msgr2=0x7f5408004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.698+0000 7f54218bd640 1 --2- 172.21.15.115:0/3673983878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f54080047c0 0x7f5408004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.698+0000 7f54218bd640 1 -- 172.21.15.115:0/3673983878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5408003970 msgr2=0x7f5408003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.698+0000 7f54218bd640 1 --2- 172.21.15.115:0/3673983878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5408003970 0x7f5408003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.698+0000 7f54218bd640 1 -- 172.21.15.115:0/3673983878 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5408003520 con 0x7f5408005180 2023-12-01T03:10:15.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.699+0000 7f54218bd640 1 -- 172.21.15.115:0/3673983878 reap_dead start 2023-12-01T03:10:15.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.699+0000 7f54218bd640 1 --2- 172.21.15.115:0/3673983878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5408005180 0x7f540800b980 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f5414009fc0 tx=0x7f54080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd6225ee97cebb1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:15.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.699+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541402cc60 con 0x7f5408005180 2023-12-01T03:10:15.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.699+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541400f040 con 0x7f5408005180 2023-12-01T03:10:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.699+0000 7f5413fff640 1 -- 172.21.15.115:0/3673983878 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414003990 con 0x7f5408005180 2023-12-01T03:10:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.700+0000 7f5423320640 1 -- 172.21.15.115:0/3673983878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5408005180 msgr2=0x7f540800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.700+0000 7f5423320640 1 --2- 172.21.15.115:0/3673983878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5408005180 0x7f540800b980 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f5414009fc0 tx=0x7f54080057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.700+0000 7f5423320640 1 -- 172.21.15.115:0/3673983878 shutdown_connections 2023-12-01T03:10:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.700+0000 7f5423320640 1 --2- 172.21.15.115:0/3673983878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5408005180 0x7f540800b980 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.700+0000 7f5423320640 1 -- 172.21.15.115:0/3673983878 >> 172.21.15.115:0/3673983878 conn(0x7f541c065960 msgr2=0x7f541c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.700+0000 7f5423320640 1 -- 172.21.15.115:0/3673983878 shutdown_connections 2023-12-01T03:10:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.700+0000 7f5423320640 1 -- 172.21.15.115:0/3673983878 wait complete. 2023-12-01T03:10:15.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.702+0000 7f5423320640 1 Processor -- start 2023-12-01T03:10:15.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.702+0000 7f5423320640 1 -- start start 2023-12-01T03:10:15.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.703+0000 7f5423320640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5408005180 0x7f541c0c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.703+0000 7f5423320640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f541c1f0190 0x7f541c12db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.703+0000 7f5423320640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541c12bae0 0x7f541c140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.703+0000 7f5423320640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0c3b60 con 0x7f541c12bae0 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.703+0000 7f5423320640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c12e080 con 0x7f5408005180 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.703+0000 7f5423320640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c12e350 con 0x7f541c1f0190 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.703+0000 7f54218bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541c12bae0 0x7f541c140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.703+0000 7f54218bd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541c12bae0 0x7f541c140740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60782/0 (socket says 172.21.15.115:60782) 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.703+0000 7f54218bd640 1 -- 172.21.15.115:0/1687626864 learned_addr learned my addr 172.21.15.115:0/1687626864 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f54218bd640 1 -- 172.21.15.115:0/1687626864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f541c1f0190 msgr2=0x7f541c12db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f54218bd640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f541c1f0190 0x7f541c12db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f54210bc640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5408005180 0x7f541c0c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f54218bd640 1 -- 172.21.15.115:0/1687626864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5408005180 msgr2=0x7f541c0c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f54218bd640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5408005180 0x7f541c0c5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f54218bd640 1 -- 172.21.15.115:0/1687626864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c12e630 con 0x7f541c12bae0 2023-12-01T03:10:15.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f54218bd640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541c12bae0 0x7f541c140740 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f541402c6d0 tx=0x7f5414009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:15.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f5411ffb640 1 -- 172.21.15.115:0/1687626864 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414003da0 con 0x7f541c12bae0 2023-12-01T03:10:15.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f5411ffb640 1 -- 172.21.15.115:0/1687626864 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5414035950 con 0x7f541c12bae0 2023-12-01T03:10:15.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f5411ffb640 1 -- 172.21.15.115:0/1687626864 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414035d40 con 0x7f541c12bae0 2023-12-01T03:10:15.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f5423320640 1 -- 172.21.15.115:0/1687626864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c12e910 con 0x7f541c12bae0 2023-12-01T03:10:15.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.704+0000 7f5423320640 1 -- 172.21.15.115:0/1687626864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c1f0610 con 0x7f541c12bae0 2023-12-01T03:10:15.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.706+0000 7f5423320640 1 -- 172.21.15.115:0/1687626864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c0c3fb0 con 0x7f541c12bae0 2023-12-01T03:10:15.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.706+0000 7f5411ffb640 1 -- 172.21.15.115:0/1687626864 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f5414004030 con 0x7f541c12bae0 2023-12-01T03:10:15.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.707+0000 7f5411ffb640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f540006a580 0x7f540006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:15.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.707+0000 7f5411ffb640 1 -- 172.21.15.115:0/1687626864 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5675+0+0 (secure 0 0 0) 0x7f54140ab6b0 con 0x7f541c12bae0 2023-12-01T03:10:15.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.708+0000 7f54210bc640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f540006a580 0x7f540006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:15.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.708+0000 7f54210bc640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f540006a580 0x7f540006ca40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54180014d0 tx=0x7f5418009040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:15.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.712+0000 7f5411ffb640 1 -- 172.21.15.115:0/1687626864 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f541c0c3fb0 con 0x7f541c12bae0 2023-12-01T03:10:15.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.901+0000 7f5423320640 1 -- 172.21.15.115:0/1687626864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f541c12eb00 con 0x7f541c12bae0 2023-12-01T03:10:15.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.909+0000 7f5411ffb640 1 -- 172.21.15.115:0/1687626864 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v69) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5414076bb0 con 0x7f541c12bae0 2023-12-01T03:10:15.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 -- 172.21.15.115:0/1687626864 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f540006a580 msgr2=0x7f540006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f540006a580 0x7f540006ca40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f54180014d0 tx=0x7f5418009040 comp rx=0 tx=0).stop 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 -- 172.21.15.115:0/1687626864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541c12bae0 msgr2=0x7f541c140740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541c12bae0 0x7f541c140740 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f541402c6d0 tx=0x7f5414009750 comp rx=0 tx=0).stop 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 -- 172.21.15.115:0/1687626864 shutdown_connections 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f540006a580 0x7f540006ca40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f541c1f0190 0x7f541c12db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541c12bae0 0x7f541c140740 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 --2- 172.21.15.115:0/1687626864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5408005180 0x7f541c0c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 -- 172.21.15.115:0/1687626864 >> 172.21.15.115:0/1687626864 conn(0x7f541c065960 msgr2=0x7f541c0b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.912+0000 7f5423320640 1 -- 172.21.15.115:0/1687626864 shutdown_connections 2023-12-01T03:10:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:15.913+0000 7f5423320640 1 -- 172.21.15.115:0/1687626864 wait complete. 2023-12-01T03:10:15.935 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-12-01T03:10:16.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.147+0000 7f59cd870640 1 Processor -- start 2023-12-01T03:10:16.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.147+0000 7f59cd870640 1 -- start start 2023-12-01T03:10:16.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.149+0000 7f59cd870640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c80cad70 0x7f59c806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.149+0000 7f59cd870640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f59c806f2f0 0x7f59c806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.149+0000 7f59cd870640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c80add10 con 0x7f59c815e2f0 2023-12-01T03:10:16.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.149+0000 7f59cd870640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c80c5760 con 0x7f59c806f2f0 2023-12-01T03:10:16.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.149+0000 7f59cd870640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c815e6d0 con 0x7f59c80cad70 2023-12-01T03:10:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.149+0000 7f59c77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f59c806f2f0 0x7f59c806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.149+0000 7f59c77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f59c806f2f0 0x7f59c806bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60784/0 (socket says 172.21.15.115:60784) 2023-12-01T03:10:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.149+0000 7f59c77fe640 1 -- 172.21.15.115:0/1092373120 learned_addr learned my addr 172.21.15.115:0/1092373120 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.150+0000 7f59c67fc640 1 --2- 172.21.15.115:0/1092373120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c80cad70 0x7f59c806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.150+0000 7f59c77fe640 1 -- 172.21.15.115:0/1092373120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c80cad70 msgr2=0x7f59c806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.150+0000 7f59c77fe640 1 --2- 172.21.15.115:0/1092373120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c80cad70 0x7f59c806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.150+0000 7f59c77fe640 1 -- 172.21.15.115:0/1092373120 >> v1:172.21.15.142:6789/0 conn(0x7f59c815e2f0 legacy=0x7f59c80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:16.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.150+0000 7f59c77fe640 1 -- 172.21.15.115:0/1092373120 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c80cf7b0 con 0x7f59c806f2f0 2023-12-01T03:10:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.151+0000 7f59c77fe640 1 --2- 172.21.15.115:0/1092373120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c806f2f0 0x7f59c806bb00 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009e50 tx=0x7f59c002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe274f1594d8f532 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.152+0000 7f59c5ffb640 1 -- 172.21.15.115:0/1092373120 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c002c9e0 con 0x7f59c806f2f0 2023-12-01T03:10:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.152+0000 7f59c5ffb640 1 -- 172.21.15.115:0/1092373120 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c002cb40 con 0x7f59c806f2f0 2023-12-01T03:10:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.152+0000 7f59c5ffb640 1 -- 172.21.15.115:0/1092373120 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00343f0 con 0x7f59c806f2f0 2023-12-01T03:10:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.152+0000 7f59cd870640 1 -- 172.21.15.115:0/1092373120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c806f2f0 msgr2=0x7f59c806bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.152+0000 7f59cd870640 1 --2- 172.21.15.115:0/1092373120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c806f2f0 0x7f59c806bb00 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009e50 tx=0x7f59c002be40 comp rx=0 tx=0).stop 2023-12-01T03:10:16.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.153+0000 7f59cd870640 1 -- 172.21.15.115:0/1092373120 shutdown_connections 2023-12-01T03:10:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.153+0000 7f59cd870640 1 --2- 172.21.15.115:0/1092373120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c806f2f0 0x7f59c806bb00 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.153+0000 7f59cd870640 1 --2- 172.21.15.115:0/1092373120 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c80cad70 0x7f59c806b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.153+0000 7f59cd870640 1 -- 172.21.15.115:0/1092373120 >> 172.21.15.115:0/1092373120 conn(0x7f59c80ce880 msgr2=0x7f59c80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.153+0000 7f59cd870640 1 -- 172.21.15.115:0/1092373120 shutdown_connections 2023-12-01T03:10:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.153+0000 7f59cd870640 1 -- 172.21.15.115:0/1092373120 wait complete. 2023-12-01T03:10:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.154+0000 7f59cd870640 1 Processor -- start 2023-12-01T03:10:16.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.154+0000 7f59cd870640 1 -- start start 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.155+0000 7f59cd870640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c806f2f0 0x7f59c814b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.155+0000 7f59cd870640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c80cad70 0x7f59c80c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.155+0000 7f59cd870640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c815e2f0 0x7f59c80c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.155+0000 7f59cd870640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c814b7c0 con 0x7f59c80cad70 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.155+0000 7f59cd870640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c814bf00 con 0x7f59c815e2f0 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.155+0000 7f59cd870640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c814c1e0 con 0x7f59c806f2f0 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.156+0000 7f59c67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c80cad70 0x7f59c80c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.156+0000 7f59c67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c80cad70 0x7f59c80c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60792/0 (socket says 172.21.15.115:60792) 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.156+0000 7f59c67fc640 1 -- 172.21.15.115:0/1648001083 learned_addr learned my addr 172.21.15.115:0/1648001083 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.156+0000 7f59c77fe640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c815e2f0 0x7f59c80c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.156+0000 7f59c67fc640 1 -- 172.21.15.115:0/1648001083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c806f2f0 msgr2=0x7f59c814b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.156+0000 7f59c67fc640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c806f2f0 0x7f59c814b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.156+0000 7f59c67fc640 1 -- 172.21.15.115:0/1648001083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c815e2f0 msgr2=0x7f59c80c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.156+0000 7f59c67fc640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c815e2f0 0x7f59c80c2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.156+0000 7f59c67fc640 1 -- 172.21.15.115:0/1648001083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c0009b30 con 0x7f59c80cad70 2023-12-01T03:10:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.156+0000 7f59c67fc640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c80cad70 0x7f59c80c7bd0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00d5f0 tx=0x7f59bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.157+0000 7f59affff640 1 -- 172.21.15.115:0/1648001083 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0125f0 con 0x7f59c80cad70 2023-12-01T03:10:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.157+0000 7f59affff640 1 -- 172.21.15.115:0/1648001083 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc012750 con 0x7f59c80cad70 2023-12-01T03:10:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.157+0000 7f59cd870640 1 -- 172.21.15.115:0/1648001083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c814c4c0 con 0x7f59c80cad70 2023-12-01T03:10:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.157+0000 7f59affff640 1 -- 172.21.15.115:0/1648001083 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc012b00 con 0x7f59c80cad70 2023-12-01T03:10:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.157+0000 7f59cd870640 1 -- 172.21.15.115:0/1648001083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c814ca80 con 0x7f59c80cad70 2023-12-01T03:10:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.158+0000 7f59cd870640 1 -- 172.21.15.115:0/1648001083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c814cc40 con 0x7f59c80cad70 2023-12-01T03:10:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.159+0000 7f59affff640 1 -- 172.21.15.115:0/1648001083 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f59bc012c60 con 0x7f59c80cad70 2023-12-01T03:10:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.160+0000 7f59affff640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f59b406a580 0x7f59b406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.160+0000 7f59affff640 1 -- 172.21.15.115:0/1648001083 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5675+0+0 (secure 0 0 0) 0x7f59bc088e40 con 0x7f59c80cad70 2023-12-01T03:10:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.161+0000 7f59c6ffd640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f59b406a580 0x7f59b406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.161+0000 7f59c6ffd640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f59b406a580 0x7f59b406ca40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59b8000f60 tx=0x7f59b80079b0 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.165+0000 7f59affff640 1 -- 172.21.15.115:0/1648001083 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f59bc0543b0 con 0x7f59c80cad70 2023-12-01T03:10:16.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.422+0000 7f59cd870640 1 -- 172.21.15.115:0/1648001083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f59c806fda0 con 0x7f59c80cad70 2023-12-01T03:10:16.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.572+0000 7f59affff640 1 -- 172.21.15.115:0/1648001083 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3056) v1 ==== 163+0+0 (secure 0 0 0) 0x7f59bc0579d0 con 0x7f59c80cad70 2023-12-01T03:10:16.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 -- 172.21.15.115:0/1648001083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f59b406a580 msgr2=0x7f59b406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:16.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f59b406a580 0x7f59b406ca40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59b8000f60 tx=0x7f59b80079b0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 -- 172.21.15.115:0/1648001083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c80cad70 msgr2=0x7f59c80c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:16.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c80cad70 0x7f59c80c7bd0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00d5f0 tx=0x7f59bc00dab0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 -- 172.21.15.115:0/1648001083 shutdown_connections 2023-12-01T03:10:16.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f59b406a580 0x7f59b406ca40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59c815e2f0 0x7f59c80c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59c80cad70 0x7f59c80c7bd0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 --2- 172.21.15.115:0/1648001083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59c806f2f0 0x7f59c814b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 -- 172.21.15.115:0/1648001083 >> 172.21.15.115:0/1648001083 conn(0x7f59c80ce880 msgr2=0x7f59c80cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:16.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 -- 172.21.15.115:0/1648001083 shutdown_connections 2023-12-01T03:10:16.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.575+0000 7f59cd870640 1 -- 172.21.15.115:0/1648001083 wait complete. 2023-12-01T03:10:16.598 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-12-01T03:10:16.599 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command succes with --fs argument for non-default fs 2023-12-01T03:10:16.599 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-12-01T03:10:16.599 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-12-01T03:10:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.809+0000 7f9585ece640 1 Processor -- start 2023-12-01T03:10:16.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.809+0000 7f9585ece640 1 -- start start 2023-12-01T03:10:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.810+0000 7f9585ece640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f95800cad70 0x7f958006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.810+0000 7f9585ece640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f958006f2f0 0x7f958006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.810+0000 7f9585ece640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95800add10 con 0x7f958015e2f0 2023-12-01T03:10:16.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.810+0000 7f9585ece640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95800c5760 con 0x7f958006f2f0 2023-12-01T03:10:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.810+0000 7f9585ece640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958015e6d0 con 0x7f95800cad70 2023-12-01T03:10:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.811+0000 7f957ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f958006f2f0 0x7f958006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.811+0000 7f957ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f958006f2f0 0x7f958006bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60806/0 (socket says 172.21.15.115:60806) 2023-12-01T03:10:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.811+0000 7f957ffff640 1 -- 172.21.15.115:0/906468186 learned_addr learned my addr 172.21.15.115:0/906468186 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:16.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.811+0000 7f9577fff640 1 --2- 172.21.15.115:0/906468186 >> v2:172.21.15.142:3300/0 conn(0x7f95800cad70 0x7f958006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.811+0000 7f957ffff640 1 -- 172.21.15.115:0/906468186 >> v2:172.21.15.142:3300/0 conn(0x7f95800cad70 msgr2=0x7f958006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.811+0000 7f957ffff640 1 --2- 172.21.15.115:0/906468186 >> v2:172.21.15.142:3300/0 conn(0x7f95800cad70 0x7f958006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.811+0000 7f957ffff640 1 -- 172.21.15.115:0/906468186 >> v1:172.21.15.115:6789/0 conn(0x7f958015e2f0 legacy=0x7f95800ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.811+0000 7f957ffff640 1 -- 172.21.15.115:0/906468186 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95800cf7b0 con 0x7f958006f2f0 2023-12-01T03:10:16.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.811+0000 7f957effd640 1 -- 172.21.15.115:0/906468186 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951854577 0 0) 0x7f95800add10 con 0x7f958015e2f0 2023-12-01T03:10:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.812+0000 7f957ffff640 1 --2- 172.21.15.115:0/906468186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958006f2f0 0x7f958006bb00 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f957802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a5839ede33d8d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.813+0000 7f957effd640 1 -- 172.21.15.115:0/906468186 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957802c9f0 con 0x7f958006f2f0 2023-12-01T03:10:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.813+0000 7f957effd640 1 -- 172.21.15.115:0/906468186 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957802cb50 con 0x7f958006f2f0 2023-12-01T03:10:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.813+0000 7f957effd640 1 -- 172.21.15.115:0/906468186 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780343f0 con 0x7f958006f2f0 2023-12-01T03:10:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.813+0000 7f9585ece640 1 -- 172.21.15.115:0/906468186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958006f2f0 msgr2=0x7f958006bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.813+0000 7f9585ece640 1 --2- 172.21.15.115:0/906468186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958006f2f0 0x7f958006bb00 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f957802be50 comp rx=0 tx=0).stop 2023-12-01T03:10:16.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.813+0000 7f9585ece640 1 -- 172.21.15.115:0/906468186 shutdown_connections 2023-12-01T03:10:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.813+0000 7f9585ece640 1 --2- 172.21.15.115:0/906468186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958006f2f0 0x7f958006bb00 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.813+0000 7f9585ece640 1 --2- 172.21.15.115:0/906468186 >> v2:172.21.15.142:3300/0 conn(0x7f95800cad70 0x7f958006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.813+0000 7f9585ece640 1 -- 172.21.15.115:0/906468186 >> 172.21.15.115:0/906468186 conn(0x7f95800ce880 msgr2=0x7f95800cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.813+0000 7f9585ece640 1 -- 172.21.15.115:0/906468186 shutdown_connections 2023-12-01T03:10:16.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.814+0000 7f9585ece640 1 -- 172.21.15.115:0/906468186 wait complete. 2023-12-01T03:10:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.815+0000 7f9585ece640 1 Processor -- start 2023-12-01T03:10:16.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.815+0000 7f9585ece640 1 -- start start 2023-12-01T03:10:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.816+0000 7f9585ece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f958006f2f0 0x7f9580157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.816+0000 7f9585ece640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95800cad70 0x7f95801e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.816+0000 7f9585ece640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958015e2f0 0x7f95801e35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.816+0000 7f9585ece640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95800c3a50 con 0x7f958015e2f0 2023-12-01T03:10:16.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.816+0000 7f9585ece640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801e4e00 con 0x7f95800cad70 2023-12-01T03:10:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.816+0000 7f9585ece640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95800c97e0 con 0x7f958006f2f0 2023-12-01T03:10:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.817+0000 7f957ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958015e2f0 0x7f95801e35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.817+0000 7f957ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958015e2f0 0x7f95801e35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60814/0 (socket says 172.21.15.115:60814) 2023-12-01T03:10:16.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.817+0000 7f957ffff640 1 -- 172.21.15.115:0/351334340 learned_addr learned my addr 172.21.15.115:0/351334340 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.817+0000 7f957ffff640 1 -- 172.21.15.115:0/351334340 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f958006f2f0 msgr2=0x7f9580157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.817+0000 7f957ffff640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f958006f2f0 0x7f9580157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.817+0000 7f957ffff640 1 -- 172.21.15.115:0/351334340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95800cad70 msgr2=0x7f95801e45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.817+0000 7f957ffff640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95800cad70 0x7f95801e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.817+0000 7f957ffff640 1 -- 172.21.15.115:0/351334340 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578009b30 con 0x7f958015e2f0 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.817+0000 7f957ffff640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958015e2f0 0x7f95801e35b0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f9578004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.818+0000 7f957cff9640 1 -- 172.21.15.115:0/351334340 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578009510 con 0x7f958015e2f0 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.818+0000 7f957cff9640 1 -- 172.21.15.115:0/351334340 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957803e450 con 0x7f958015e2f0 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.818+0000 7f957cff9640 1 -- 172.21.15.115:0/351334340 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957803e800 con 0x7f958015e2f0 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.818+0000 7f9585ece640 1 -- 172.21.15.115:0/351334340 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95800c30b0 con 0x7f958015e2f0 2023-12-01T03:10:16.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.818+0000 7f9585ece640 1 -- 172.21.15.115:0/351334340 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95800c3670 con 0x7f958015e2f0 2023-12-01T03:10:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.819+0000 7f9585ece640 1 -- 172.21.15.115:0/351334340 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958006fda0 con 0x7f958015e2f0 2023-12-01T03:10:16.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.820+0000 7f957cff9640 1 -- 172.21.15.115:0/351334340 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f957803e960 con 0x7f958015e2f0 2023-12-01T03:10:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.821+0000 7f957cff9640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f956006a550 0x7f956006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:16.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.821+0000 7f957cff9640 1 -- 172.21.15.115:0/351334340 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5675+0+0 (secure 0 0 0) 0x7f95780ab8b0 con 0x7f958015e2f0 2023-12-01T03:10:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.821+0000 7f957f7fe640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f956006a550 0x7f956006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:16.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.822+0000 7f957f7fe640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f956006a550 0x7f956006ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95800adea0 tx=0x7f9570007950 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:16.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:16.825+0000 7f957cff9640 1 -- 172.21.15.115:0/351334340 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9578076d10 con 0x7f958015e2f0 2023-12-01T03:10:17.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.089+0000 7f9585ece640 1 -- 172.21.15.115:0/351334340 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f95800c3830 con 0x7f958015e2f0 2023-12-01T03:10:17.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.575+0000 7f957cff9640 1 -- 172.21.15.115:0/351334340 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3057) v1 ==== 189+0+0 (secure 0 0 0) 0x7f957807a330 con 0x7f958015e2f0 2023-12-01T03:10:17.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 -- 172.21.15.115:0/351334340 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f956006a550 msgr2=0x7f956006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:17.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f956006a550 0x7f956006ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f95800adea0 tx=0x7f9570007950 comp rx=0 tx=0).stop 2023-12-01T03:10:17.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 -- 172.21.15.115:0/351334340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958015e2f0 msgr2=0x7f95801e35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:17.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958015e2f0 0x7f95801e35b0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f9578004a40 comp rx=0 tx=0).stop 2023-12-01T03:10:17.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 -- 172.21.15.115:0/351334340 shutdown_connections 2023-12-01T03:10:17.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f956006a550 0x7f956006ca10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f958015e2f0 0x7f95801e35b0 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95800cad70 0x7f95801e45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 --2- 172.21.15.115:0/351334340 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f958006f2f0 0x7f9580157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 -- 172.21.15.115:0/351334340 >> 172.21.15.115:0/351334340 conn(0x7f95800ce880 msgr2=0x7f95800adff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.579+0000 7f9585ece640 1 -- 172.21.15.115:0/351334340 shutdown_connections 2023-12-01T03:10:17.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.580+0000 7f9585ece640 1 -- 172.21.15.115:0/351334340 wait complete. 2023-12-01T03:10:17.603 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T03:10:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.822+0000 7f0acd7b1640 1 Processor -- start 2023-12-01T03:10:17.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.822+0000 7f0acd7b1640 1 -- start start 2023-12-01T03:10:17.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.823+0000 7f0acd7b1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0ac8142870 0x7f0ac80ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.823+0000 7f0acd7b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ac80cd010 0x7f0ac80b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.823+0000 7f0acd7b1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0ac80b1c50 0x7f0ac815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0acd7b1640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac80b3ae0 con 0x7f0ac80b1c50 2023-12-01T03:10:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0acd7b1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac8142c50 con 0x7f0ac8142870 2023-12-01T03:10:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0acd7b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80cd470 con 0x7f0ac80cd010 2023-12-01T03:10:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0ac77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0ac80b1c50 0x7f0ac815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0ac77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0ac80b1c50 0x7f0ac815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35118/0 (socket says 172.21.15.115:35118) 2023-12-01T03:10:17.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0ac77fe640 1 -- 172.21.15.115:0/1734801188 learned_addr learned my addr 172.21.15.115:0/1734801188 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0abffff640 1 --2- 172.21.15.115:0/1734801188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ac80cd010 0x7f0ac80b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0ac6ffd640 1 --2- 172.21.15.115:0/1734801188 >> v2:172.21.15.142:3300/0 conn(0x7f0ac8142870 0x7f0ac80ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0ac77fe640 1 -- 172.21.15.115:0/1734801188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ac80cd010 msgr2=0x7f0ac80b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0ac77fe640 1 --2- 172.21.15.115:0/1734801188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ac80cd010 0x7f0ac80b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0ac77fe640 1 -- 172.21.15.115:0/1734801188 >> v2:172.21.15.142:3300/0 conn(0x7f0ac8142870 msgr2=0x7f0ac80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0ac77fe640 1 --2- 172.21.15.115:0/1734801188 >> v2:172.21.15.142:3300/0 conn(0x7f0ac8142870 0x7f0ac80ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0ac77fe640 1 -- 172.21.15.115:0/1734801188 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac80b20e0 con 0x7f0ac80b1c50 2023-12-01T03:10:17.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.824+0000 7f0ac6ffd640 1 --2- 172.21.15.115:0/1734801188 >> v2:172.21.15.142:3300/0 conn(0x7f0ac8142870 0x7f0ac80ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.826+0000 7f0ac77fe640 1 --2- 172.21.15.115:0/1734801188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80b1c50 0x7f0ac815d690 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80b14d0 tx=0x7f0ac002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85784706211eee2f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.826+0000 7f0ac67fc640 1 -- 172.21.15.115:0/1734801188 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac002ca70 con 0x7f0ac80b1c50 2023-12-01T03:10:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.826+0000 7f0ac67fc640 1 -- 172.21.15.115:0/1734801188 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac002cbd0 con 0x7f0ac80b1c50 2023-12-01T03:10:17.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.826+0000 7f0ac67fc640 1 -- 172.21.15.115:0/1734801188 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00343f0 con 0x7f0ac80b1c50 2023-12-01T03:10:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.827+0000 7f0acd7b1640 1 -- 172.21.15.115:0/1734801188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80b1c50 msgr2=0x7f0ac815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.827+0000 7f0acd7b1640 1 --2- 172.21.15.115:0/1734801188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80b1c50 0x7f0ac815d690 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80b14d0 tx=0x7f0ac002bed0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.827+0000 7f0acd7b1640 1 -- 172.21.15.115:0/1734801188 shutdown_connections 2023-12-01T03:10:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.827+0000 7f0acd7b1640 1 --2- 172.21.15.115:0/1734801188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80b1c50 0x7f0ac815d690 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.827+0000 7f0acd7b1640 1 --2- 172.21.15.115:0/1734801188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ac80cd010 0x7f0ac80b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.827+0000 7f0acd7b1640 1 --2- 172.21.15.115:0/1734801188 >> v2:172.21.15.142:3300/0 conn(0x7f0ac8142870 0x7f0ac80ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.827+0000 7f0acd7b1640 1 -- 172.21.15.115:0/1734801188 >> 172.21.15.115:0/1734801188 conn(0x7f0ac8065960 msgr2=0x7f0ac8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.827+0000 7f0acd7b1640 1 -- 172.21.15.115:0/1734801188 shutdown_connections 2023-12-01T03:10:17.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.827+0000 7f0acd7b1640 1 -- 172.21.15.115:0/1734801188 wait complete. 2023-12-01T03:10:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.829+0000 7f0acd7b1640 1 Processor -- start 2023-12-01T03:10:17.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.829+0000 7f0acd7b1640 1 -- start start 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0acd7b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ac80b1c50 0x7f0ac81f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0acd7b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80cd010 0x7f0ac80c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0acd7b1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac8142870 0x7f0ac813fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0acd7b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81f6920 con 0x7f0ac80cd010 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0acd7b1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80cb920 con 0x7f0ac8142870 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0acd7b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81f6080 con 0x7f0ac80b1c50 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0abffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80cd010 0x7f0ac80c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0abffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80cd010 0x7f0ac80c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35130/0 (socket says 172.21.15.115:35130) 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0abffff640 1 -- 172.21.15.115:0/722092335 learned_addr learned my addr 172.21.15.115:0/722092335 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0ac6ffd640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ac80b1c50 0x7f0ac81f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:17.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0ac77fe640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac8142870 0x7f0ac813fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0abffff640 1 -- 172.21.15.115:0/722092335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ac80b1c50 msgr2=0x7f0ac81f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:17.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0abffff640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ac80b1c50 0x7f0ac81f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0abffff640 1 -- 172.21.15.115:0/722092335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac8142870 msgr2=0x7f0ac813fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.831+0000 7f0abffff640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac8142870 0x7f0ac813fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.832+0000 7f0abffff640 1 -- 172.21.15.115:0/722092335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac0009b30 con 0x7f0ac80cd010 2023-12-01T03:10:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.832+0000 7f0abffff640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80cd010 0x7f0ac80c91e0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c7f0 tx=0x7f0ab000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.832+0000 7f0abf7fe640 1 -- 172.21.15.115:0/722092335 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0004220 con 0x7f0ac80cd010 2023-12-01T03:10:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.832+0000 7f0abf7fe640 1 -- 172.21.15.115:0/722092335 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab0004380 con 0x7f0ac80cd010 2023-12-01T03:10:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.832+0000 7f0acd7b1640 1 -- 172.21.15.115:0/722092335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac81f6360 con 0x7f0ac80cd010 2023-12-01T03:10:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.832+0000 7f0abf7fe640 1 -- 172.21.15.115:0/722092335 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0007860 con 0x7f0ac80cd010 2023-12-01T03:10:17.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.832+0000 7f0acd7b1640 1 -- 172.21.15.115:0/722092335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac80c7e30 con 0x7f0ac80cd010 2023-12-01T03:10:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.833+0000 7f0acd7b1640 1 -- 172.21.15.115:0/722092335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac80c8050 con 0x7f0ac80cd010 2023-12-01T03:10:17.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.834+0000 7f0abf7fe640 1 -- 172.21.15.115:0/722092335 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0ab0007bd0 con 0x7f0ac80cd010 2023-12-01T03:10:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.835+0000 7f0abf7fe640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0aa806a550 0x7f0aa806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.835+0000 7f0abf7fe640 1 -- 172.21.15.115:0/722092335 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5675+0+0 (secure 0 0 0) 0x7f0ab0088be0 con 0x7f0ac80cd010 2023-12-01T03:10:17.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.835+0000 7f0ac6ffd640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0aa806a550 0x7f0aa806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:17.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.836+0000 7f0ac6ffd640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0aa806a550 0x7f0aa806ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800a950 tx=0x7f0ab8008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:17.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:17.840+0000 7f0abf7fe640 1 -- 172.21.15.115:0/722092335 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0ab0053f30 con 0x7f0ac80cd010 2023-12-01T03:10:18.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.030+0000 7f0acd7b1640 1 -- 172.21.15.115:0/722092335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0ac804e890 con 0x7f0ac80cd010 2023-12-01T03:10:18.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.037+0000 7f0abf7fe640 1 -- 172.21.15.115:0/722092335 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0ab0057550 con 0x7f0ac80cd010 2023-12-01T03:10:18.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.039+0000 7f0acd7b1640 1 -- 172.21.15.115:0/722092335 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0aa806a550 msgr2=0x7f0aa806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.039+0000 7f0acd7b1640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0aa806a550 0x7f0aa806ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800a950 tx=0x7f0ab8008040 comp rx=0 tx=0).stop 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.039+0000 7f0acd7b1640 1 -- 172.21.15.115:0/722092335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80cd010 msgr2=0x7f0ac80c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.039+0000 7f0acd7b1640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80cd010 0x7f0ac80c91e0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c7f0 tx=0x7f0ab000ccb0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.040+0000 7f0acd7b1640 1 -- 172.21.15.115:0/722092335 shutdown_connections 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.040+0000 7f0acd7b1640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0aa806a550 0x7f0aa806ca10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.040+0000 7f0acd7b1640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ac8142870 0x7f0ac813fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.040+0000 7f0acd7b1640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ac80cd010 0x7f0ac80c91e0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.040+0000 7f0acd7b1640 1 --2- 172.21.15.115:0/722092335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ac80b1c50 0x7f0ac81f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.040+0000 7f0acd7b1640 1 -- 172.21.15.115:0/722092335 >> 172.21.15.115:0/722092335 conn(0x7f0ac8065960 msgr2=0x7f0ac8141870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.040+0000 7f0acd7b1640 1 -- 172.21.15.115:0/722092335 shutdown_connections 2023-12-01T03:10:18.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.040+0000 7f0acd7b1640 1 -- 172.21.15.115:0/722092335 wait complete. 2023-12-01T03:10:18.059 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T03:10:18.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:10:18.177 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:10:18.304 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T03:10:18.304 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:10:18.304 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:10:18.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:10:18.365 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:10:18.365 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:10:18.425 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:10:18.426 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:10:18.426 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:10:18.488 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:10:18.488 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:10:18.488 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:10:18.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:10:18.550 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T03:10:18.550 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:10:18.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.772+0000 7fcda25b1640 1 Processor -- start 2023-12-01T03:10:18.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.772+0000 7fcda25b1640 1 -- start start 2023-12-01T03:10:18.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.774+0000 7fcda25b1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd9c15d220 0x7fcd9c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.774+0000 7fcda25b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0ad2b0 0x7fcd9c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.774+0000 7fcda25b1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c15d600 con 0x7fcd9c0ade80 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.774+0000 7fcda25b1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c06cc10 con 0x7fcd9c15d220 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.774+0000 7fcda25b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c06c720 con 0x7fcd9c0ad2b0 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.774+0000 7fcda0b4e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fcd9c0ade80 0x7fcd9c06b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56982/0 (socket says 172.21.15.115:56982) 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.774+0000 7fcda0b4e640 1 -- 172.21.15.115:0/3985087571 learned_addr learned my addr 172.21.15.115:0/3985087571 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.774+0000 7fcd9b7fe640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0ad2b0 0x7fcd9c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.774+0000 7fcd9bfff640 1 --2- 172.21.15.115:0/3985087571 >> v2:172.21.15.142:3300/0 conn(0x7fcd9c15d220 0x7fcd9c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407503462 0 0) 0x7fcd94001490 con 0x7fcd9c0ade80 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd84003520 con 0x7fcd9c0ade80 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3006600939 0 0) 0x7fcd84003520 con 0x7fcd9c0ade80 2023-12-01T03:10:18.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0ad2b0 msgr2=0x7fcd9c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:18.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0ad2b0 0x7fcd9c0ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 >> v2:172.21.15.142:3300/0 conn(0x7fcd9c15d220 msgr2=0x7fcd9c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:18.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 --2- 172.21.15.115:0/3985087571 >> v2:172.21.15.142:3300/0 conn(0x7fcd9c15d220 0x7fcd9c0cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c06c4b0 con 0x7fcd9c0ade80 2023-12-01T03:10:18.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fcd94001b80 con 0x7fcd9c0ade80 2023-12-01T03:10:18.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 >> v1:172.21.15.115:6789/0 conn(0x7fcd9c0ade80 legacy=0x7fcd9c06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:18.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0ad2b0 0x7fcd840039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:18.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd84003f60 0x7fcd840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:18.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd84004d00 0x7fcd84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:18.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c06cc10 con 0x7fcd84003f60 2023-12-01T03:10:18.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c06c720 con 0x7fcd84004d00 2023-12-01T03:10:18.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.775+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8400ba30 con 0x7fcd9c0ad2b0 2023-12-01T03:10:18.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.776+0000 7fcd9b7fe640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd84003f60 0x7fcd840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:18.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.776+0000 7fcda0b4e640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd84004d00 0x7fcd84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:18.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.776+0000 7fcd9bfff640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0ad2b0 0x7fcd840039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:18.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.776+0000 7fcd9b7fe640 1 -- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0ad2b0 msgr2=0x7fcd840039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:18.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.776+0000 7fcd9b7fe640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0ad2b0 0x7fcd840039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.776+0000 7fcd9b7fe640 1 -- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd84004d00 msgr2=0x7fcd84005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:18.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.776+0000 7fcd9b7fe640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd84004d00 0x7fcd84005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.776+0000 7fcd9b7fe640 1 -- 172.21.15.115:0/3985087571 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd84003520 con 0x7fcd84003f60 2023-12-01T03:10:18.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.777+0000 7fcd9b7fe640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd84003f60 0x7fcd840047c0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009ac0 tx=0x7fcd840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821a75eba5ffe992 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:18.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.777+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9002c6b0 con 0x7fcd84003f60 2023-12-01T03:10:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.777+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9002c810 con 0x7fcd84003f60 2023-12-01T03:10:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.777+0000 7fcd9affd640 1 -- 172.21.15.115:0/3985087571 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9002cba0 con 0x7fcd84003f60 2023-12-01T03:10:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.778+0000 7fcda25b1640 1 -- 172.21.15.115:0/3985087571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd84003f60 msgr2=0x7fcd840047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.778+0000 7fcda25b1640 1 --2- 172.21.15.115:0/3985087571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd84003f60 0x7fcd840047c0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009ac0 tx=0x7fcd840057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.778+0000 7fcda0b4e640 1 -- 172.21.15.115:0/3985087571 reap_dead start 2023-12-01T03:10:18.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.778+0000 7fcda25b1640 1 -- 172.21.15.115:0/3985087571 shutdown_connections 2023-12-01T03:10:18.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.778+0000 7fcda25b1640 1 -- 172.21.15.115:0/3985087571 >> 172.21.15.115:0/3985087571 conn(0x7fcd9c0ce880 msgr2=0x7fcd9c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:18.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.778+0000 7fcda25b1640 1 -- 172.21.15.115:0/3985087571 shutdown_connections 2023-12-01T03:10:18.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.778+0000 7fcda25b1640 1 -- 172.21.15.115:0/3985087571 wait complete. 2023-12-01T03:10:18.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.780+0000 7fcda25b1640 1 Processor -- start 2023-12-01T03:10:18.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.780+0000 7fcda25b1640 1 -- start start 2023-12-01T03:10:18.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.781+0000 7fcda25b1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd84003f60 0x7fcd9c1f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.781+0000 7fcda25b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd9c1e4e50 0x7fcd9c1e3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.781+0000 7fcda25b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0af820 0x7fcd9c0afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.781+0000 7fcda25b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0b01e0 con 0x7fcd9c1e4e50 2023-12-01T03:10:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.781+0000 7fcda25b1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0cbc60 con 0x7fcd84003f60 2023-12-01T03:10:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.781+0000 7fcda25b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1e3c50 con 0x7fcd9c0af820 2023-12-01T03:10:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.781+0000 7fcd9b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd9c1e4e50 0x7fcd9c1e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.781+0000 7fcd9b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd9c1e4e50 0x7fcd9c1e3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35146/0 (socket says 172.21.15.115:35146) 2023-12-01T03:10:18.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.781+0000 7fcd9b7fe640 1 -- 172.21.15.115:0/1132250928 learned_addr learned my addr 172.21.15.115:0/1132250928 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcda0b4e640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0af820 0x7fcd9c0afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcd9bfff640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd84003f60 0x7fcd9c1f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcd9b7fe640 1 -- 172.21.15.115:0/1132250928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0af820 msgr2=0x7fcd9c0afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcd9b7fe640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0af820 0x7fcd9c0afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcd9b7fe640 1 -- 172.21.15.115:0/1132250928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd84003f60 msgr2=0x7fcd9c1f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcd9b7fe640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd84003f60 0x7fcd9c1f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcd9b7fe640 1 -- 172.21.15.115:0/1132250928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c1e3f30 con 0x7fcd9c1e4e50 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcd9b7fe640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd9c1e4e50 0x7fcd9c1e3570 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcd9002c250 tx=0x7fcd90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcd98ff9640 1 -- 172.21.15.115:0/1132250928 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90003810 con 0x7fcd9c1e4e50 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcd98ff9640 1 -- 172.21.15.115:0/1132250928 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd90003970 con 0x7fcd9c1e4e50 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcda25b1640 1 -- 172.21.15.115:0/1132250928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c1e4210 con 0x7fcd9c1e4e50 2023-12-01T03:10:18.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcd98ff9640 1 -- 172.21.15.115:0/1132250928 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90003ce0 con 0x7fcd9c1e4e50 2023-12-01T03:10:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.782+0000 7fcda25b1640 1 -- 172.21.15.115:0/1132250928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c1e47d0 con 0x7fcd9c1e4e50 2023-12-01T03:10:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.784+0000 7fcda25b1640 1 -- 172.21.15.115:0/1132250928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c04e890 con 0x7fcd9c1e4e50 2023-12-01T03:10:18.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.784+0000 7fcd98ff9640 1 -- 172.21.15.115:0/1132250928 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fcd90047450 con 0x7fcd9c1e4e50 2023-12-01T03:10:18.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.785+0000 7fcd98ff9640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd8806a580 0x7fcd8806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:18.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.785+0000 7fcd98ff9640 1 -- 172.21.15.115:0/1132250928 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5675+0+0 (secure 0 0 0) 0x7fcd900aab90 con 0x7fcd9c1e4e50 2023-12-01T03:10:18.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.786+0000 7fcd9bfff640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd8806a580 0x7fcd8806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:18.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.786+0000 7fcd9bfff640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd8806a580 0x7fcd8806ca40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009990 tx=0x7fcd8c008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:18.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:18.790+0000 7fcd98ff9640 1 -- 172.21.15.115:0/1132250928 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcd900761d0 con 0x7fcd9c1e4e50 2023-12-01T03:10:19.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.017+0000 7fcda25b1640 1 -- 172.21.15.115:0/1132250928 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd9c0b01e0 con 0x7fcd9c1e4e50 2023-12-01T03:10:19.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.018+0000 7fcd98ff9640 1 -- 172.21.15.115:0/1132250928 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fcd9004cde0 con 0x7fcd9c1e4e50 2023-12-01T03:10:19.021 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:19.021 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":89,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/974148243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":974148243},{"type":"v1","addr":"172.21.15.115:6837","nonce":974148243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/476969842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":476969842},{"type":"v1","addr":"172.21.15.142:6834","nonce":476969842}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":6533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4028158492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":4028158492},{"type":"v1","addr":"172.21.15.142:6835","nonce":4028158492}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/1103499235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":1103499235},{"type":"v1","addr":"172.21.15.115:6835","nonce":1103499235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[]} 2023-12-01T03:10:19.021 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 89 2023-12-01T03:10:19.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.021+0000 7fcda25b1640 1 -- 172.21.15.115:0/1132250928 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd8806a580 msgr2=0x7fcd8806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:19.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.021+0000 7fcda25b1640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd8806a580 0x7fcd8806ca40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009990 tx=0x7fcd8c008040 comp rx=0 tx=0).stop 2023-12-01T03:10:19.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.021+0000 7fcda25b1640 1 -- 172.21.15.115:0/1132250928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd9c1e4e50 msgr2=0x7fcd9c1e3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:19.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.021+0000 7fcda25b1640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd9c1e4e50 0x7fcd9c1e3570 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcd9002c250 tx=0x7fcd90004730 comp rx=0 tx=0).stop 2023-12-01T03:10:19.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.021+0000 7fcda25b1640 1 -- 172.21.15.115:0/1132250928 shutdown_connections 2023-12-01T03:10:19.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.021+0000 7fcda25b1640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd8806a580 0x7fcd8806ca40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:19.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.021+0000 7fcda25b1640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd9c0af820 0x7fcd9c0afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:19.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.021+0000 7fcda25b1640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd9c1e4e50 0x7fcd9c1e3570 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:19.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.021+0000 7fcda25b1640 1 --2- 172.21.15.115:0/1132250928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd84003f60 0x7fcd9c1f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:19.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.021+0000 7fcda25b1640 1 -- 172.21.15.115:0/1132250928 >> 172.21.15.115:0/1132250928 conn(0x7fcd9c0ce880 msgr2=0x7fcd9c15b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:19.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.022+0000 7fcda25b1640 1 -- 172.21.15.115:0/1132250928 shutdown_connections 2023-12-01T03:10:19.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.022+0000 7fcda25b1640 1 -- 172.21.15.115:0/1132250928 wait complete. 2023-12-01T03:10:19.042 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T03:10:19.042 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T03:10:19.042 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T03:10:19.042 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:10:19.042 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T03:10:19.042 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T03:10:19.043 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T03:10:19.043 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:10:19.043 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T03:10:19.043 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T03:10:19.043 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T03:10:19.043 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:10:19.043 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T03:10:19.044 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T03:10:19.044 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T03:10:19.044 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:10:19.064 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T03:10:19.064 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T03:10:19.065 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T03:10:19.065 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T03:10:19.067 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T03:10:19.067 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T03:10:19.067 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T03:10:19.068 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T03:10:19.113 INFO:tasks.ceph.mds.a:Started 2023-12-01T03:10:19.114 DEBUG:teuthology.parallel:result is None 2023-12-01T03:10:19.116 INFO:tasks.ceph.mds.b:Started 2023-12-01T03:10:19.116 DEBUG:teuthology.parallel:result is None 2023-12-01T03:10:19.121 INFO:tasks.ceph.mds.c:Started 2023-12-01T03:10:19.121 DEBUG:teuthology.parallel:result is None 2023-12-01T03:10:19.123 INFO:tasks.ceph.mds.d:Started 2023-12-01T03:10:19.124 DEBUG:teuthology.parallel:result is None 2023-12-01T03:10:19.124 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T03:10:19.275 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T03:10:19.276 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T03:10:19.279 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T03:10:19.279 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T03:10:19.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.316+0000 7f2a5258a640 1 Processor -- start 2023-12-01T03:10:19.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.316+0000 7f2a5258a640 1 -- start start 2023-12-01T03:10:19.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.317+0000 7f2a5258a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a4c15c9c0 0x7f2a4c0b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:19.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.317+0000 7f2a5258a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2a4c0cfad0 0x7f2a4c0d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:19.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a5258a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c0d0ab0 con 0x7f2a4c0b59d0 2023-12-01T03:10:19.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a5258a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4c0d0d90 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a5258a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4c0d1070 con 0x7f2a4c15c9c0 2023-12-01T03:10:19.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a51d89640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2a4c0cfad0 0x7f2a4c0d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:19.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a51d89640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2a4c0cfad0 0x7f2a4c0d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35180/0 (socket says 172.21.15.115:35180) 2023-12-01T03:10:19.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a51d89640 1 -- 172.21.15.115:0/2745574027 learned_addr learned my addr 172.21.15.115:0/2745574027 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:19.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a51588640 1 --2- 172.21.15.115:0/2745574027 >> v2:172.21.15.142:3300/0 conn(0x7f2a4c15c9c0 0x7f2a4c0b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:19.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a51d89640 1 -- 172.21.15.115:0/2745574027 >> v2:172.21.15.142:3300/0 conn(0x7f2a4c15c9c0 msgr2=0x7f2a4c0b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:19.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a51d89640 1 --2- 172.21.15.115:0/2745574027 >> v2:172.21.15.142:3300/0 conn(0x7f2a4c15c9c0 0x7f2a4c0b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:19.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a51d89640 1 -- 172.21.15.115:0/2745574027 >> v1:172.21.15.142:6789/0 conn(0x7f2a4c0b59d0 legacy=0x7f2a4c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:19.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.318+0000 7f2a51d89640 1 -- 172.21.15.115:0/2745574027 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c0d1350 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.319+0000 7f2a51d89640 1 --2- 172.21.15.115:0/2745574027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 0x7f2a4c0d1850 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0668c0 tx=0x7f2a4402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41ac69fe364623f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:19.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.320+0000 7f2a43fff640 1 -- 172.21.15.115:0/2745574027 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4402cb20 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.320+0000 7f2a43fff640 1 -- 172.21.15.115:0/2745574027 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4402cc80 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.320+0000 7f2a43fff640 1 -- 172.21.15.115:0/2745574027 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a440343f0 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.320+0000 7f2a5258a640 1 -- 172.21.15.115:0/2745574027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 msgr2=0x7f2a4c0d1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:19.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.320+0000 7f2a5258a640 1 --2- 172.21.15.115:0/2745574027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 0x7f2a4c0d1850 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0668c0 tx=0x7f2a4402bf80 comp rx=0 tx=0).stop 2023-12-01T03:10:19.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.320+0000 7f2a5258a640 1 -- 172.21.15.115:0/2745574027 shutdown_connections 2023-12-01T03:10:19.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.320+0000 7f2a5258a640 1 --2- 172.21.15.115:0/2745574027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 0x7f2a4c0d1850 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:19.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.320+0000 7f2a5258a640 1 --2- 172.21.15.115:0/2745574027 >> v2:172.21.15.142:3300/0 conn(0x7f2a4c15c9c0 0x7f2a4c0b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:19.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.320+0000 7f2a5258a640 1 -- 172.21.15.115:0/2745574027 >> 172.21.15.115:0/2745574027 conn(0x7f2a4c065960 msgr2=0x7f2a4c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:19.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.321+0000 7f2a5258a640 1 -- 172.21.15.115:0/2745574027 shutdown_connections 2023-12-01T03:10:19.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.321+0000 7f2a5258a640 1 -- 172.21.15.115:0/2745574027 wait complete. 2023-12-01T03:10:19.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.322+0000 7f2a5258a640 1 Processor -- start 2023-12-01T03:10:19.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.322+0000 7f2a5258a640 1 -- start start 2023-12-01T03:10:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.323+0000 7f2a5258a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a4c0b59d0 0x7f2a4c14d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.323+0000 7f2a5258a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 0x7f2a4c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.323+0000 7f2a5258a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c15c9c0 0x7f2a4c159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.323+0000 7f2a5258a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0b2a30 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.323+0000 7f2a5258a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0c9030 con 0x7f2a4c15c9c0 2023-12-01T03:10:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.323+0000 7f2a5258a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0c7100 con 0x7f2a4c0b59d0 2023-12-01T03:10:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.324+0000 7f2a50d87640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 0x7f2a4c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.324+0000 7f2a50d87640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 0x7f2a4c158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35194/0 (socket says 172.21.15.115:35194) 2023-12-01T03:10:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.324+0000 7f2a50d87640 1 -- 172.21.15.115:0/2704172765 learned_addr learned my addr 172.21.15.115:0/2704172765 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.324+0000 7f2a51d89640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c15c9c0 0x7f2a4c159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:19.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.324+0000 7f2a51588640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a4c0b59d0 0x7f2a4c14d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.324+0000 7f2a50d87640 1 -- 172.21.15.115:0/2704172765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a4c0b59d0 msgr2=0x7f2a4c14d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.324+0000 7f2a50d87640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a4c0b59d0 0x7f2a4c14d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.324+0000 7f2a50d87640 1 -- 172.21.15.115:0/2704172765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c15c9c0 msgr2=0x7f2a4c159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.324+0000 7f2a50d87640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c15c9c0 0x7f2a4c159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.324+0000 7f2a50d87640 1 -- 172.21.15.115:0/2704172765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a44009b30 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.325+0000 7f2a50d87640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 0x7f2a4c158f10 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00c3c0 tx=0x7f2a3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.325+0000 7f2a41ffb640 1 -- 172.21.15.115:0/2704172765 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c0052d0 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.325+0000 7f2a41ffb640 1 -- 172.21.15.115:0/2704172765 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c005450 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.325+0000 7f2a41ffb640 1 -- 172.21.15.115:0/2704172765 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c003830 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.325+0000 7f2a5258a640 1 -- 172.21.15.115:0/2704172765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c0c73e0 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.325+0000 7f2a5258a640 1 -- 172.21.15.115:0/2704172765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c0b26c0 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.326+0000 7f2a5258a640 1 -- 172.21.15.115:0/2704172765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4c0b2ba0 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.327+0000 7f2a41ffb640 1 -- 172.21.15.115:0/2704172765 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f2a3c003990 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.328+0000 7f2a41ffb640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 0x7f2a3006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:19.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.328+0000 7f2a41ffb640 1 -- 172.21.15.115:0/2704172765 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5675+0+0 (secure 0 0 0) 0x7f2a3c087f40 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.328+0000 7f2a51588640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 0x7f2a3006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:19.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.329+0000 7f2a51588640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 0x7f2a3006ca40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009b40 tx=0x7f2a48009190 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:19.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.332+0000 7f2a41ffb640 1 -- 172.21.15.115:0/2704172765 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2a3c053450 con 0x7f2a4c0cfad0 2023-12-01T03:10:19.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:19.533+0000 7f2a5258a640 1 -- 172.21.15.115:0/2704172765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2a4c0673a0 con 0x7f2a4c0cfad0 2023-12-01T03:10:20.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.323+0000 7f2a41ffb640 1 -- 172.21.15.115:0/2704172765 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v210) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2a3c056a70 con 0x7f2a4c0cfad0 2023-12-01T03:10:20.325 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T03:10:20.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.326+0000 7f2a5258a640 1 -- 172.21.15.115:0/2704172765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 msgr2=0x7f2a3006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.326+0000 7f2a5258a640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 0x7f2a3006ca40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009b40 tx=0x7f2a48009190 comp rx=0 tx=0).stop 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.326+0000 7f2a5258a640 1 -- 172.21.15.115:0/2704172765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 msgr2=0x7f2a4c158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.326+0000 7f2a5258a640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 0x7f2a4c158f10 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00c3c0 tx=0x7f2a3c00c880 comp rx=0 tx=0).stop 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.326+0000 7f2a5258a640 1 -- 172.21.15.115:0/2704172765 shutdown_connections 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.326+0000 7f2a5258a640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 0x7f2a3006ca40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.327+0000 7f2a5258a640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a4c15c9c0 0x7f2a4c159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.327+0000 7f2a5258a640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a4c0cfad0 0x7f2a4c158f10 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.327+0000 7f2a5258a640 1 --2- 172.21.15.115:0/2704172765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a4c0b59d0 0x7f2a4c14d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.327+0000 7f2a5258a640 1 -- 172.21.15.115:0/2704172765 >> 172.21.15.115:0/2704172765 conn(0x7f2a4c065960 msgr2=0x7f2a4c154960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.327+0000 7f2a5258a640 1 -- 172.21.15.115:0/2704172765 shutdown_connections 2023-12-01T03:10:20.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.327+0000 7f2a5258a640 1 -- 172.21.15.115:0/2704172765 wait complete. 2023-12-01T03:10:20.349 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T03:10:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.562+0000 7fcd2a221640 1 Processor -- start 2023-12-01T03:10:20.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.562+0000 7fcd2a221640 1 -- start start 2023-12-01T03:10:20.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.564+0000 7fcd2a221640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd24159dd0 0x7fcd240c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.564+0000 7fcd2a221640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd24070d30 0x7fcd240c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.564+0000 7fcd2a221640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2406f8b0 con 0x7fcd2406b6f0 2023-12-01T03:10:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.564+0000 7fcd2a221640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd240adcb0 con 0x7fcd24070d30 2023-12-01T03:10:20.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.564+0000 7fcd2a221640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd240ad9d0 con 0x7fcd24159dd0 2023-12-01T03:10:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.564+0000 7fcd23fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd24070d30 0x7fcd240c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.564+0000 7fcd23fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcd24070d30 0x7fcd240c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35196/0 (socket says 172.21.15.115:35196) 2023-12-01T03:10:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.564+0000 7fcd23fff640 1 -- 172.21.15.115:0/2167885398 learned_addr learned my addr 172.21.15.115:0/2167885398 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.565+0000 7fcd23fff640 1 -- 172.21.15.115:0/2167885398 >> v2:172.21.15.142:3300/0 conn(0x7fcd24159dd0 msgr2=0x7fcd240c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.565+0000 7fcd23fff640 1 --2- 172.21.15.115:0/2167885398 >> v2:172.21.15.142:3300/0 conn(0x7fcd24159dd0 0x7fcd240c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.565+0000 7fcd23fff640 1 -- 172.21.15.115:0/2167885398 >> v1:172.21.15.142:6789/0 conn(0x7fcd2406b6f0 legacy=0x7fcd240cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:20.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.565+0000 7fcd23fff640 1 -- 172.21.15.115:0/2167885398 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd240c9000 con 0x7fcd24070d30 2023-12-01T03:10:20.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.566+0000 7fcd23fff640 1 --2- 172.21.15.115:0/2167885398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24070d30 0x7fcd240c9d20 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fcd240cf4e0 tx=0x7fcd1c02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a27997f9335b9d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:20.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.566+0000 7fcd22ffd640 1 -- 172.21.15.115:0/2167885398 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c02c960 con 0x7fcd24070d30 2023-12-01T03:10:20.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.566+0000 7fcd22ffd640 1 -- 172.21.15.115:0/2167885398 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c02cac0 con 0x7fcd24070d30 2023-12-01T03:10:20.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.566+0000 7fcd22ffd640 1 -- 172.21.15.115:0/2167885398 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c02ce50 con 0x7fcd24070d30 2023-12-01T03:10:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.567+0000 7fcd2a221640 1 -- 172.21.15.115:0/2167885398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24070d30 msgr2=0x7fcd240c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.567+0000 7fcd2a221640 1 --2- 172.21.15.115:0/2167885398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24070d30 0x7fcd240c9d20 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fcd240cf4e0 tx=0x7fcd1c02bdc0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.567+0000 7fcd2a221640 1 -- 172.21.15.115:0/2167885398 shutdown_connections 2023-12-01T03:10:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.567+0000 7fcd2a221640 1 --2- 172.21.15.115:0/2167885398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24070d30 0x7fcd240c9d20 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.567+0000 7fcd2a221640 1 --2- 172.21.15.115:0/2167885398 >> v2:172.21.15.142:3300/0 conn(0x7fcd24159dd0 0x7fcd240c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.567+0000 7fcd2a221640 1 -- 172.21.15.115:0/2167885398 >> 172.21.15.115:0/2167885398 conn(0x7fcd240ce880 msgr2=0x7fcd240cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.567+0000 7fcd2a221640 1 -- 172.21.15.115:0/2167885398 shutdown_connections 2023-12-01T03:10:20.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.567+0000 7fcd2a221640 1 -- 172.21.15.115:0/2167885398 wait complete. 2023-12-01T03:10:20.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.568+0000 7fcd2a221640 1 Processor -- start 2023-12-01T03:10:20.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.569+0000 7fcd2a221640 1 -- start start 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.569+0000 7fcd2a221640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd2406b6f0 0x7fcd2406c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.569+0000 7fcd2a221640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd24070d30 0x7fcd24154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.569+0000 7fcd2a221640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24159dd0 0x7fcd2415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.569+0000 7fcd2a221640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2415b850 con 0x7fcd24159dd0 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.569+0000 7fcd2a221640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd240c3e10 con 0x7fcd2406b6f0 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.569+0000 7fcd2a221640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2415d1f0 con 0x7fcd24070d30 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd23fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24159dd0 0x7fcd2415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd23fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24159dd0 0x7fcd2415c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35198/0 (socket says 172.21.15.115:35198) 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd23fff640 1 -- 172.21.15.115:0/175330496 learned_addr learned my addr 172.21.15.115:0/175330496 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd1bfff640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd24070d30 0x7fcd24154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:20.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd237fe640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd2406b6f0 0x7fcd2406c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:20.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd23fff640 1 -- 172.21.15.115:0/175330496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd24070d30 msgr2=0x7fcd24154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:20.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd23fff640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd24070d30 0x7fcd24154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd23fff640 1 -- 172.21.15.115:0/175330496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd2406b6f0 msgr2=0x7fcd2406c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:20.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd23fff640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd2406b6f0 0x7fcd2406c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd23fff640 1 -- 172.21.15.115:0/175330496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c009b30 con 0x7fcd24159dd0 2023-12-01T03:10:20.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.570+0000 7fcd23fff640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24159dd0 0x7fcd2415c170 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c036040 tx=0x7fcd1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:20.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.571+0000 7fcd20ff9640 1 -- 172.21.15.115:0/175330496 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c03e450 con 0x7fcd24159dd0 2023-12-01T03:10:20.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.571+0000 7fcd20ff9640 1 -- 172.21.15.115:0/175330496 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c03e5d0 con 0x7fcd24159dd0 2023-12-01T03:10:20.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.571+0000 7fcd20ff9640 1 -- 172.21.15.115:0/175330496 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c03e980 con 0x7fcd24159dd0 2023-12-01T03:10:20.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.571+0000 7fcd2a221640 1 -- 172.21.15.115:0/175330496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd240c4060 con 0x7fcd24159dd0 2023-12-01T03:10:20.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.571+0000 7fcd2a221640 1 -- 172.21.15.115:0/175330496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd240c4620 con 0x7fcd24159dd0 2023-12-01T03:10:20.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.572+0000 7fcd2a221640 1 -- 172.21.15.115:0/175330496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd240cc270 con 0x7fcd24159dd0 2023-12-01T03:10:20.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.573+0000 7fcd20ff9640 1 -- 172.21.15.115:0/175330496 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fcd1c03eae0 con 0x7fcd24159dd0 2023-12-01T03:10:20.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.574+0000 7fcd20ff9640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd0406a550 0x7fcd0406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:20.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.574+0000 7fcd20ff9640 1 -- 172.21.15.115:0/175330496 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fcd1c0ab6d0 con 0x7fcd24159dd0 2023-12-01T03:10:20.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.574+0000 7fcd237fe640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd0406a550 0x7fcd0406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:20.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.575+0000 7fcd237fe640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd0406a550 0x7fcd0406ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400a770 tx=0x7fcd14008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:20.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.578+0000 7fcd20ff9640 1 -- 172.21.15.115:0/175330496 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcd1c076f90 con 0x7fcd24159dd0 2023-12-01T03:10:20.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.839+0000 7fcd2a221640 1 -- 172.21.15.115:0/175330496 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcd2404e890 con 0x7fcd24159dd0 2023-12-01T03:10:20.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.840+0000 7fcd20ff9640 1 -- 172.21.15.115:0/175330496 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v47) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcd1c07a5b0 con 0x7fcd24159dd0 2023-12-01T03:10:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:20.842 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:10:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T03:10:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T03:10:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T03:10:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:10:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:10:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:10:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T03:10:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:10:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T03:10:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:10:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T03:10:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T03:10:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T03:10:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:10:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:10:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T03:10:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:10:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T03:10:20.855 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:10:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.842+0000 7fcd2a221640 1 -- 172.21.15.115:0/175330496 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd0406a550 msgr2=0x7fcd0406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.842+0000 7fcd2a221640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd0406a550 0x7fcd0406ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400a770 tx=0x7fcd14008040 comp rx=0 tx=0).stop 2023-12-01T03:10:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.842+0000 7fcd2a221640 1 -- 172.21.15.115:0/175330496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24159dd0 msgr2=0x7fcd2415c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.842+0000 7fcd2a221640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24159dd0 0x7fcd2415c170 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c036040 tx=0x7fcd1c004a40 comp rx=0 tx=0).stop 2023-12-01T03:10:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.843+0000 7fcd2a221640 1 -- 172.21.15.115:0/175330496 shutdown_connections 2023-12-01T03:10:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.843+0000 7fcd2a221640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd0406a550 0x7fcd0406ca10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.843+0000 7fcd2a221640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd24070d30 0x7fcd24154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.843+0000 7fcd2a221640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd24159dd0 0x7fcd2415c170 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.843+0000 7fcd2a221640 1 --2- 172.21.15.115:0/175330496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd2406b6f0 0x7fcd2406c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:20.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.843+0000 7fcd2a221640 1 -- 172.21.15.115:0/175330496 >> 172.21.15.115:0/175330496 conn(0x7fcd240ce880 msgr2=0x7fcd24158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:20.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.843+0000 7fcd2a221640 1 -- 172.21.15.115:0/175330496 shutdown_connections 2023-12-01T03:10:20.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:20.843+0000 7fcd2a221640 1 -- 172.21.15.115:0/175330496 wait complete. 2023-12-01T03:10:20.864 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:21.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.074+0000 7fbcc84f3640 1 Processor -- start 2023-12-01T03:10:21.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.074+0000 7fbcc84f3640 1 -- start start 2023-12-01T03:10:21.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.075+0000 7fbcc84f3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbcc00cad70 0x7fbcc006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.075+0000 7fbcc84f3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc00add10 con 0x7fbcc006f2f0 2023-12-01T03:10:21.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.075+0000 7fbcc84f3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc00c5760 con 0x7fbcc015e2f0 2023-12-01T03:10:21.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.075+0000 7fbcc84f3640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc015e6d0 con 0x7fbcc00cad70 2023-12-01T03:10:21.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.076+0000 7fbcc6a90640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fbcc006f2f0 0x7fbcc006bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57000/0 (socket says 172.21.15.115:57000) 2023-12-01T03:10:21.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.076+0000 7fbcc6a90640 1 -- 172.21.15.115:0/3031693005 learned_addr learned my addr 172.21.15.115:0/3031693005 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:21.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.076+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579087162 0 0) 0x7fbcc00add10 con 0x7fbcc006f2f0 2023-12-01T03:10:21.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.076+0000 7fbcc5a8e640 1 --2- 172.21.15.115:0/3031693005 >> v2:172.21.15.142:3300/0 conn(0x7fbcc00cad70 0x7fbcc006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.076+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcac003520 con 0x7fbcc006f2f0 2023-12-01T03:10:21.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2706584896 0 0) 0x7fbcac003520 con 0x7fbcc006f2f0 2023-12-01T03:10:21.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 >> v2:172.21.15.142:3300/0 conn(0x7fbcc00cad70 msgr2=0x7fbcc006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:21.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 --2- 172.21.15.115:0/3031693005 >> v2:172.21.15.142:3300/0 conn(0x7fbcc00cad70 0x7fbcc006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 >> v1:172.21.15.142:6789/0 conn(0x7fbcc015e2f0 legacy=0x7fbcc00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc00cf7b0 con 0x7fbcc006f2f0 2023-12-01T03:10:21.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fbcb8001a10 con 0x7fbcc006f2f0 2023-12-01T03:10:21.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 >> v1:172.21.15.115:6789/0 conn(0x7fbcc006f2f0 legacy=0x7fbcc006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 --2- 172.21.15.115:0/3031693005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcac003a30 0x7fbcac003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 --2- 172.21.15.115:0/3031693005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcac0047c0 0x7fbcac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 --2- 172.21.15.115:0/3031693005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcac00b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc015e6d0 con 0x7fbcac005380 2023-12-01T03:10:21.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac00c2c0 con 0x7fbcac003a30 2023-12-01T03:10:21.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac00c480 con 0x7fbcac0047c0 2023-12-01T03:10:21.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc5a8e640 1 --2- 172.21.15.115:0/3031693005 >> v2:172.21.15.142:3300/0 conn(0x7fbcc00cad70 0x7fbcc006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:10:21.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc6a90640 1 --2- 172.21.15.115:0/3031693005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcac00b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc6a90640 1 -- 172.21.15.115:0/3031693005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcac0047c0 msgr2=0x7fbcac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:21.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc6a90640 1 --2- 172.21.15.115:0/3031693005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcac0047c0 0x7fbcac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc6a90640 1 -- 172.21.15.115:0/3031693005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcac003a30 msgr2=0x7fbcac003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:21.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc6a90640 1 --2- 172.21.15.115:0/3031693005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcac003a30 0x7fbcac003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.077+0000 7fbcc6a90640 1 -- 172.21.15.115:0/3031693005 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac003520 con 0x7fbcac005380 2023-12-01T03:10:21.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.078+0000 7fbcc6a90640 1 -- 172.21.15.115:0/3031693005 reap_dead start 2023-12-01T03:10:21.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc6a90640 1 --2- 172.21.15.115:0/3031693005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcac00b9e0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fbcc006f700 tx=0x7fbcac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdbf774bd36d576 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:21.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8034630 con 0x7fbcac005380 2023-12-01T03:10:21.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb802f070 con 0x7fbcac005380 2023-12-01T03:10:21.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc528d640 1 -- 172.21.15.115:0/3031693005 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80349f0 con 0x7fbcac005380 2023-12-01T03:10:21.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc84f3640 1 -- 172.21.15.115:0/3031693005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 msgr2=0x7fbcac00b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc84f3640 1 --2- 172.21.15.115:0/3031693005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcac00b9e0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fbcc006f700 tx=0x7fbcac0057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc84f3640 1 -- 172.21.15.115:0/3031693005 shutdown_connections 2023-12-01T03:10:21.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc84f3640 1 --2- 172.21.15.115:0/3031693005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcac00b9e0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc84f3640 1 -- 172.21.15.115:0/3031693005 >> 172.21.15.115:0/3031693005 conn(0x7fbcc00ce880 msgr2=0x7fbcc00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:21.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc84f3640 1 -- 172.21.15.115:0/3031693005 shutdown_connections 2023-12-01T03:10:21.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.079+0000 7fbcc84f3640 1 -- 172.21.15.115:0/3031693005 wait complete. 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.081+0000 7fbcc84f3640 1 Processor -- start 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.081+0000 7fbcc84f3640 1 -- start start 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.082+0000 7fbcc84f3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcc01f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.082+0000 7fbcc84f3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcc00c6330 0x7fbcc00c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.082+0000 7fbcc84f3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcc014f440 0x7fbcc014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.082+0000 7fbcc84f3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc00c79c0 con 0x7fbcac005380 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.082+0000 7fbcc84f3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc00c6db0 con 0x7fbcc00c6330 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.082+0000 7fbcc84f3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc00c7090 con 0x7fbcc014f440 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.082+0000 7fbcc628f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcc01f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.082+0000 7fbcc628f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcc01f3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35218/0 (socket says 172.21.15.115:35218) 2023-12-01T03:10:21.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.082+0000 7fbcc628f640 1 -- 172.21.15.115:0/1229729703 learned_addr learned my addr 172.21.15.115:0/1229729703 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcc628f640 1 -- 172.21.15.115:0/1229729703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcc014f440 msgr2=0x7fbcc014f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcc628f640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcc014f440 0x7fbcc014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcc628f640 1 -- 172.21.15.115:0/1229729703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcc00c6330 msgr2=0x7fbcc00c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcc628f640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcc00c6330 0x7fbcc00c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcc628f640 1 -- 172.21.15.115:0/1229729703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc00c7370 con 0x7fbcac005380 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcc628f640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcc01f3010 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00c500 tx=0x7fbcbc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcb6ffd640 1 -- 172.21.15.115:0/1229729703 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0052d0 con 0x7fbcac005380 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcb6ffd640 1 -- 172.21.15.115:0/1229729703 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc005430 con 0x7fbcac005380 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcb6ffd640 1 -- 172.21.15.115:0/1229729703 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc012870 con 0x7fbcac005380 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcc84f3640 1 -- 172.21.15.115:0/1229729703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc00c7650 con 0x7fbcac005380 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.083+0000 7fbcc84f3640 1 -- 172.21.15.115:0/1229729703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc014fe00 con 0x7fbcac005380 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.084+0000 7fbcc84f3640 1 -- 172.21.15.115:0/1229729703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc006fda0 con 0x7fbcac005380 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.085+0000 7fbcb6ffd640 1 -- 172.21.15.115:0/1229729703 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fbcbc0129d0 con 0x7fbcac005380 2023-12-01T03:10:21.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.086+0000 7fbcb6ffd640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbca406a550 0x7fbca406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.086+0000 7fbcb6ffd640 1 -- 172.21.15.115:0/1229729703 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fbcbc088340 con 0x7fbcac005380 2023-12-01T03:10:21.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.087+0000 7fbcc5a8e640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbca406a550 0x7fbca406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.087+0000 7fbcc5a8e640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbca406a550 0x7fbca406ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00096b0 tx=0x7fbcb0008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:21.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.091+0000 7fbcb6ffd640 1 -- 172.21.15.115:0/1229729703 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbcbc057660 con 0x7fbcac005380 2023-12-01T03:10:21.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.280+0000 7fbcc84f3640 1 -- 172.21.15.115:0/1229729703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcc004e890 con 0x7fbcac005380 2023-12-01T03:10:21.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.282+0000 7fbcb6ffd640 1 -- 172.21.15.115:0/1229729703 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) v1 ==== 74+0+12517 (secure 0 0 0) 0x7fbcbc0571f0 con 0x7fbcac005380 2023-12-01T03:10:21.284 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:21.285 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":210,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:20.315817+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:21.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 -- 172.21.15.115:0/1229729703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbca406a550 msgr2=0x7fbca406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbca406a550 0x7fbca406ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00096b0 tx=0x7fbcb0008040 comp rx=0 tx=0).stop 2023-12-01T03:10:21.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 -- 172.21.15.115:0/1229729703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 msgr2=0x7fbcc01f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcc01f3010 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00c500 tx=0x7fbcbc00c9c0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 -- 172.21.15.115:0/1229729703 shutdown_connections 2023-12-01T03:10:21.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbca406a550 0x7fbca406ca10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcc014f440 0x7fbcc014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcc00c6330 0x7fbcc00c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 --2- 172.21.15.115:0/1229729703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcac005380 0x7fbcc01f3010 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 -- 172.21.15.115:0/1229729703 >> 172.21.15.115:0/1229729703 conn(0x7fbcc00ce880 msgr2=0x7fbcc015b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:21.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.285+0000 7fbcc84f3640 1 -- 172.21.15.115:0/1229729703 shutdown_connections 2023-12-01T03:10:21.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.286+0000 7fbcc84f3640 1 -- 172.21.15.115:0/1229729703 wait complete. 2023-12-01T03:10:21.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:10:21.307 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:10:21.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.520+0000 7f6b07df0640 1 Processor -- start 2023-12-01T03:10:21.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.520+0000 7f6b07df0640 1 -- start start 2023-12-01T03:10:21.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.522+0000 7f6b07df0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b000cc440 0x7f6b000b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.522+0000 7f6b07df0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b000cb290 con 0x7f6b00160200 2023-12-01T03:10:21.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.522+0000 7f6b07df0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b000b0ae0 con 0x7f6b000b0f80 2023-12-01T03:10:21.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.522+0000 7f6b07df0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b000b35f0 con 0x7f6b000cc440 2023-12-01T03:10:21.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.522+0000 7f6b05b8c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6b00160200 0x7f6b0006b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57010/0 (socket says 172.21.15.115:57010) 2023-12-01T03:10:21.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.522+0000 7f6b05b8c640 1 -- 172.21.15.115:0/4039388692 learned_addr learned my addr 172.21.15.115:0/4039388692 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:21.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.522+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710749415 0 0) 0x7f6b000cb290 con 0x7f6b00160200 2023-12-01T03:10:21.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aec003520 con 0x7f6b00160200 2023-12-01T03:10:21.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b0638d640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b000cc440 0x7f6b000b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3643986442 0 0) 0x7f6aec003520 con 0x7f6b00160200 2023-12-01T03:10:21.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b000cc440 msgr2=0x7f6b000b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:21.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b000cc440 0x7f6b000b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 >> v1:172.21.15.142:6789/0 conn(0x7f6b000b0f80 legacy=0x7f6b000cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b000b3310 con 0x7f6b00160200 2023-12-01T03:10:21.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6afc001670 con 0x7f6b00160200 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 >> v1:172.21.15.115:6789/0 conn(0x7f6b00160200 legacy=0x7f6b0006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b000cc440 0x7f6aec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 0x7f6aec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aec004d00 0x7f6aec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b000b35f0 con 0x7f6aec004020 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec00bbc0 con 0x7f6aec004d00 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec00bd80 con 0x7f6b000cc440 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.523+0000 7f6b0538b640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 0x7f6aec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.524+0000 7f6b05b8c640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b000cc440 0x7f6aec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.524+0000 7f6b0638d640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aec004d00 0x7f6aec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.524+0000 7f6b0538b640 1 -- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b000cc440 msgr2=0x7f6aec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.524+0000 7f6b0538b640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b000cc440 0x7f6aec0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.524+0000 7f6b0538b640 1 -- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aec004d00 msgr2=0x7f6aec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.524+0000 7f6b0538b640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6aec004d00 0x7f6aec005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.524+0000 7f6b0538b640 1 -- 172.21.15.115:0/4039388692 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec003520 con 0x7f6aec004020 2023-12-01T03:10:21.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.525+0000 7f6b0538b640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 0x7f6aec0047c0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f6af0001750 tx=0x7f6aec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7a0fceb162f17b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:21.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.525+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00052d0 con 0x7f6aec004020 2023-12-01T03:10:21.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.525+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af002e050 con 0x7f6aec004020 2023-12-01T03:10:21.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.525+0000 7f6b04b8a640 1 -- 172.21.15.115:0/4039388692 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0003830 con 0x7f6aec004020 2023-12-01T03:10:21.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.526+0000 7f6b07df0640 1 -- 172.21.15.115:0/4039388692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 msgr2=0x7f6aec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.526+0000 7f6b07df0640 1 --2- 172.21.15.115:0/4039388692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 0x7f6aec0047c0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f6af0001750 tx=0x7f6aec0057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.526+0000 7f6b0638d640 1 -- 172.21.15.115:0/4039388692 reap_dead start 2023-12-01T03:10:21.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.526+0000 7f6b07df0640 1 -- 172.21.15.115:0/4039388692 shutdown_connections 2023-12-01T03:10:21.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.526+0000 7f6b07df0640 1 -- 172.21.15.115:0/4039388692 >> 172.21.15.115:0/4039388692 conn(0x7f6b00065960 msgr2=0x7f6b00065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:21.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.526+0000 7f6b07df0640 1 -- 172.21.15.115:0/4039388692 shutdown_connections 2023-12-01T03:10:21.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.526+0000 7f6b07df0640 1 -- 172.21.15.115:0/4039388692 wait complete. 2023-12-01T03:10:21.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.527+0000 7f6b07df0640 1 Processor -- start 2023-12-01T03:10:21.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.528+0000 7f6b07df0640 1 -- start start 2023-12-01T03:10:21.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b07df0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 0x7f6b001485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b07df0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b00148e60 0x7f6b00155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b07df0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b00149390 0x7f6b00156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b07df0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00157750 con 0x7f6aec004020 2023-12-01T03:10:21.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b07df0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b001565d0 con 0x7f6b00149390 2023-12-01T03:10:21.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b07df0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b001544d0 con 0x7f6b00148e60 2023-12-01T03:10:21.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b05b8c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 0x7f6b001485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b05b8c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 0x7f6b001485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35240/0 (socket says 172.21.15.115:35240) 2023-12-01T03:10:21.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b05b8c640 1 -- 172.21.15.115:0/1670689228 learned_addr learned my addr 172.21.15.115:0/1670689228 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b05b8c640 1 -- 172.21.15.115:0/1670689228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b00148e60 msgr2=0x7f6b00155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b05b8c640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b00148e60 0x7f6b00155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b05b8c640 1 -- 172.21.15.115:0/1670689228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b00149390 msgr2=0x7f6b00156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b05b8c640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b00149390 0x7f6b00156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b05b8c640 1 -- 172.21.15.115:0/1670689228 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b001579a0 con 0x7f6aec004020 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.529+0000 7f6b05b8c640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 0x7f6b001485f0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00d780 tx=0x7f6afc00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.530+0000 7f6af67fc640 1 -- 172.21.15.115:0/1670689228 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0074e0 con 0x7f6aec004020 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.530+0000 7f6af67fc640 1 -- 172.21.15.115:0/1670689228 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc00de90 con 0x7f6aec004020 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.530+0000 7f6af67fc640 1 -- 172.21.15.115:0/1670689228 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0049e0 con 0x7f6aec004020 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.530+0000 7f6b07df0640 1 -- 172.21.15.115:0/1670689228 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b00157c80 con 0x7f6aec004020 2023-12-01T03:10:21.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.530+0000 7f6b07df0640 1 -- 172.21.15.115:0/1670689228 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b00154180 con 0x7f6aec004020 2023-12-01T03:10:21.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.531+0000 7f6b07df0640 1 -- 172.21.15.115:0/1670689228 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b001581a0 con 0x7f6aec004020 2023-12-01T03:10:21.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.532+0000 7f6af67fc640 1 -- 172.21.15.115:0/1670689228 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6afc004b40 con 0x7f6aec004020 2023-12-01T03:10:21.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.533+0000 7f6af67fc640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ae406a580 0x7f6ae406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.533+0000 7f6af67fc640 1 -- 172.21.15.115:0/1670689228 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f6afc088160 con 0x7f6aec004020 2023-12-01T03:10:21.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.533+0000 7f6b0538b640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ae406a580 0x7f6ae406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.534+0000 7f6b0538b640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ae406a580 0x7f6ae406ca40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6af0033000 tx=0x7f6af002f000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:21.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.537+0000 7f6af67fc640 1 -- 172.21.15.115:0/1670689228 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6b001581a0 con 0x7f6aec004020 2023-12-01T03:10:21.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.731+0000 7f6b07df0640 1 -- 172.21.15.115:0/1670689228 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b001581a0 con 0x7f6aec004020 2023-12-01T03:10:21.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.731+0000 7f6af67fc640 1 -- 172.21.15.115:0/1670689228 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v210) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6afc019020 con 0x7f6aec004020 2023-12-01T03:10:21.733 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:10:21.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 -- 172.21.15.115:0/1670689228 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ae406a580 msgr2=0x7f6ae406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ae406a580 0x7f6ae406ca40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6af0033000 tx=0x7f6af002f000 comp rx=0 tx=0).stop 2023-12-01T03:10:21.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 -- 172.21.15.115:0/1670689228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 msgr2=0x7f6b001485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 0x7f6b001485f0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00d780 tx=0x7f6afc00dc40 comp rx=0 tx=0).stop 2023-12-01T03:10:21.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 -- 172.21.15.115:0/1670689228 shutdown_connections 2023-12-01T03:10:21.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ae406a580 0x7f6ae406ca40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b00148e60 0x7f6b00155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b00149390 0x7f6b00156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 --2- 172.21.15.115:0/1670689228 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6aec004020 0x7f6b001485f0 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 -- 172.21.15.115:0/1670689228 >> 172.21.15.115:0/1670689228 conn(0x7f6b00065960 msgr2=0x7f6b000ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:21.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 -- 172.21.15.115:0/1670689228 shutdown_connections 2023-12-01T03:10:21.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.734+0000 7f6b07df0640 1 -- 172.21.15.115:0/1670689228 wait complete. 2023-12-01T03:10:21.755 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T03:10:21.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.968+0000 7ffa83fff640 1 Processor -- start 2023-12-01T03:10:21.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.968+0000 7ffa83fff640 1 -- start start 2023-12-01T03:10:21.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa83fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415cfc0 0x7ffa8406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa83fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa8406fb10 0x7ffa8406ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa83fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffa8406b6e0 0x7ffa8406bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa83fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa840704d0 con 0x7ffa8406b6e0 2023-12-01T03:10:21.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa83fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8415d3a0 con 0x7ffa8406fb10 2023-12-01T03:10:21.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa83fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8406f1b0 con 0x7ffa8415cfc0 2023-12-01T03:10:21.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa837fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffa8406b6e0 0x7ffa8406bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa837fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ffa8406b6e0 0x7ffa8406bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35242/0 (socket says 172.21.15.115:35242) 2023-12-01T03:10:21.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa837fe640 1 -- 172.21.15.115:0/397862309 learned_addr learned my addr 172.21.15.115:0/397862309 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:21.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa82ffd640 1 --2- 172.21.15.115:0/397862309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415cfc0 0x7ffa8406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.970+0000 7ffa7bfff640 1 --2- 172.21.15.115:0/397862309 >> v2:172.21.15.142:3300/0 conn(0x7ffa8406fb10 0x7ffa8406ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.971+0000 7ffa837fe640 1 -- 172.21.15.115:0/397862309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415cfc0 msgr2=0x7ffa8406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:21.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.971+0000 7ffa837fe640 1 --2- 172.21.15.115:0/397862309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415cfc0 0x7ffa8406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.971+0000 7ffa837fe640 1 -- 172.21.15.115:0/397862309 >> v2:172.21.15.142:3300/0 conn(0x7ffa8406fb10 msgr2=0x7ffa8406ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:21.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.971+0000 7ffa837fe640 1 --2- 172.21.15.115:0/397862309 >> v2:172.21.15.142:3300/0 conn(0x7ffa8406fb10 0x7ffa8406ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.971+0000 7ffa837fe640 1 -- 172.21.15.115:0/397862309 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa84161670 con 0x7ffa8406b6e0 2023-12-01T03:10:21.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.971+0000 7ffa7bfff640 1 --2- 172.21.15.115:0/397862309 >> v2:172.21.15.142:3300/0 conn(0x7ffa8406fb10 0x7ffa8406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:21.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.972+0000 7ffa837fe640 1 --2- 172.21.15.115:0/397862309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 0x7ffa8406bb70 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7ffa8406e4b0 tx=0x7ffa7c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85bc2f3ff4ca4f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:21.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.972+0000 7ffa827fc640 1 -- 172.21.15.115:0/397862309 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c02c9e0 con 0x7ffa8406b6e0 2023-12-01T03:10:21.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.972+0000 7ffa827fc640 1 -- 172.21.15.115:0/397862309 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c02cb40 con 0x7ffa8406b6e0 2023-12-01T03:10:21.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.972+0000 7ffa827fc640 1 -- 172.21.15.115:0/397862309 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0343f0 con 0x7ffa8406b6e0 2023-12-01T03:10:21.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.972+0000 7ffa83fff640 1 -- 172.21.15.115:0/397862309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 msgr2=0x7ffa8406bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.972+0000 7ffa83fff640 1 --2- 172.21.15.115:0/397862309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 0x7ffa8406bb70 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7ffa8406e4b0 tx=0x7ffa7c02be40 comp rx=0 tx=0).stop 2023-12-01T03:10:21.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.973+0000 7ffa83fff640 1 -- 172.21.15.115:0/397862309 shutdown_connections 2023-12-01T03:10:21.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.973+0000 7ffa83fff640 1 --2- 172.21.15.115:0/397862309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 0x7ffa8406bb70 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.973+0000 7ffa83fff640 1 --2- 172.21.15.115:0/397862309 >> v2:172.21.15.142:3300/0 conn(0x7ffa8406fb10 0x7ffa8406ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.973+0000 7ffa83fff640 1 --2- 172.21.15.115:0/397862309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415cfc0 0x7ffa8406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.973+0000 7ffa83fff640 1 -- 172.21.15.115:0/397862309 >> 172.21.15.115:0/397862309 conn(0x7ffa840ce620 msgr2=0x7ffa840cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:21.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.973+0000 7ffa83fff640 1 -- 172.21.15.115:0/397862309 shutdown_connections 2023-12-01T03:10:21.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.973+0000 7ffa83fff640 1 -- 172.21.15.115:0/397862309 wait complete. 2023-12-01T03:10:21.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.974+0000 7ffa83fff640 1 Processor -- start 2023-12-01T03:10:21.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.975+0000 7ffa83fff640 1 -- start start 2023-12-01T03:10:21.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.975+0000 7ffa83fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 0x7ffa8414edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa83fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8406fb10 0x7ffa8414da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa83fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415cfc0 0x7ffa84153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa83fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa840cb0a0 con 0x7ffa8406b6e0 2023-12-01T03:10:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa83fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa84150040 con 0x7ffa8406fb10 2023-12-01T03:10:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa83fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8414f3c0 con 0x7ffa8415cfc0 2023-12-01T03:10:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa82ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 0x7ffa8414edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa82ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 0x7ffa8414edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35248/0 (socket says 172.21.15.115:35248) 2023-12-01T03:10:21.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa82ffd640 1 -- 172.21.15.115:0/3586285290 learned_addr learned my addr 172.21.15.115:0/3586285290 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa7bfff640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8406fb10 0x7ffa8414da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa82ffd640 1 -- 172.21.15.115:0/3586285290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415cfc0 msgr2=0x7ffa84153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa837fe640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415cfc0 0x7ffa84153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa82ffd640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415cfc0 0x7ffa84153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa82ffd640 1 -- 172.21.15.115:0/3586285290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8406fb10 msgr2=0x7ffa8414da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa82ffd640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8406fb10 0x7ffa8414da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.976+0000 7ffa82ffd640 1 -- 172.21.15.115:0/3586285290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c009b30 con 0x7ffa8406b6e0 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.977+0000 7ffa82ffd640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 0x7ffa8414edd0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400c820 tx=0x7ffa7400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.977+0000 7ffa7b7fe640 1 -- 172.21.15.115:0/3586285290 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74004220 con 0x7ffa8406b6e0 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.977+0000 7ffa7b7fe640 1 -- 172.21.15.115:0/3586285290 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa740043a0 con 0x7ffa8406b6e0 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.977+0000 7ffa7b7fe640 1 -- 172.21.15.115:0/3586285290 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74007a70 con 0x7ffa8406b6e0 2023-12-01T03:10:21.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.977+0000 7ffa83fff640 1 -- 172.21.15.115:0/3586285290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8414f6a0 con 0x7ffa8406b6e0 2023-12-01T03:10:21.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.977+0000 7ffa83fff640 1 -- 172.21.15.115:0/3586285290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8414fc60 con 0x7ffa8406b6e0 2023-12-01T03:10:21.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.978+0000 7ffa83fff640 1 -- 172.21.15.115:0/3586285290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa84067850 con 0x7ffa8406b6e0 2023-12-01T03:10:21.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.979+0000 7ffa7b7fe640 1 -- 172.21.15.115:0/3586285290 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ffa74007bd0 con 0x7ffa8406b6e0 2023-12-01T03:10:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.980+0000 7ffa7b7fe640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffa6806a550 0x7ffa6806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.980+0000 7ffa7b7fe640 1 -- 172.21.15.115:0/3586285290 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4901+0+0 (secure 0 0 0) 0x7ffa740886e0 con 0x7ffa8406b6e0 2023-12-01T03:10:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.980+0000 7ffa7bfff640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffa6806a550 0x7ffa6806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:21.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.981+0000 7ffa7bfff640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffa6806a550 0x7ffa6806ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c006fd0 tx=0x7ffa6c008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:21.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:21.984+0000 7ffa7b7fe640 1 -- 172.21.15.115:0/3586285290 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ffa74053d40 con 0x7ffa8406b6e0 2023-12-01T03:10:22.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.176+0000 7ffa83fff640 1 -- 172.21.15.115:0/3586285290 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ffa840cb1e0 con 0x7ffa8406b6e0 2023-12-01T03:10:22.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.176+0000 7ffa7b7fe640 1 -- 172.21.15.115:0/3586285290 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v210) v1 ==== 84+0+65 (secure 0 0 0) 0x7ffa74057360 con 0x7ffa8406b6e0 2023-12-01T03:10:22.178 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:22.179 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T03:10:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T03:10:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T03:10:22.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:22.179 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T03:10:22.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 -- 172.21.15.115:0/3586285290 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffa6806a550 msgr2=0x7ffa6806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:22.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffa6806a550 0x7ffa6806ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c006fd0 tx=0x7ffa6c008040 comp rx=0 tx=0).stop 2023-12-01T03:10:22.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 -- 172.21.15.115:0/3586285290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 msgr2=0x7ffa8414edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:22.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 0x7ffa8414edd0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400c820 tx=0x7ffa7400cce0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 -- 172.21.15.115:0/3586285290 shutdown_connections 2023-12-01T03:10:22.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ffa6806a550 0x7ffa6806ca10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8415cfc0 0x7ffa84153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8406fb10 0x7ffa8414da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 --2- 172.21.15.115:0/3586285290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa8406b6e0 0x7ffa8414edd0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 -- 172.21.15.115:0/3586285290 >> 172.21.15.115:0/3586285290 conn(0x7ffa840ce620 msgr2=0x7ffa840ca380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:22.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 -- 172.21.15.115:0/3586285290 shutdown_connections 2023-12-01T03:10:22.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.179+0000 7ffa83fff640 1 -- 172.21.15.115:0/3586285290 wait complete. 2023-12-01T03:10:22.200 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T03:10:22.201 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T03:10:22.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.415+0000 7f9751fab640 1 Processor -- start 2023-12-01T03:10:22.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.415+0000 7f9751fab640 1 -- start start 2023-12-01T03:10:22.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.416+0000 7f9751fab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c159bf0 0x7f974c15e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:22.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.416+0000 7f9751fab640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974c15e570 con 0x7f974c0cb8c0 2023-12-01T03:10:22.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.416+0000 7f9751fab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974c0c9440 con 0x7f974c0ca7c0 2023-12-01T03:10:22.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.416+0000 7f9751fab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c159fd0 con 0x7f974c159bf0 2023-12-01T03:10:22.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.417+0000 7f974bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f974c0cb8c0 0x7f974c06f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57020/0 (socket says 172.21.15.115:57020) 2023-12-01T03:10:22.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.417+0000 7f974bfff640 1 -- 172.21.15.115:0/881424660 learned_addr learned my addr 172.21.15.115:0/881424660 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:22.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.417+0000 7f974b7fe640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c159bf0 0x7f974c15e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:22.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.417+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386957423 0 0) 0x7f974c15e570 con 0x7f974c0cb8c0 2023-12-01T03:10:22.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.417+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9738003520 con 0x7f974c0cb8c0 2023-12-01T03:10:22.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.417+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3903401942 0 0) 0x7f9738003520 con 0x7f974c0cb8c0 2023-12-01T03:10:22.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c159bf0 msgr2=0x7f974c15e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:22.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c159bf0 0x7f974c15e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 >> v1:172.21.15.142:6789/0 conn(0x7f974c0ca7c0 legacy=0x7f974c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:22.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c0c9720 con 0x7f974c0cb8c0 2023-12-01T03:10:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f97440019f0 con 0x7f974c0cb8c0 2023-12-01T03:10:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 >> v1:172.21.15.115:6789/0 conn(0x7f974c0cb8c0 legacy=0x7f974c06f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97380039a0 0x7f9738003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97380047c0 0x7f9738004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c159bf0 0x7f9738005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:22.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973800dc40 con 0x7f97380047c0 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973800dd80 con 0x7f97380039a0 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973800df40 con 0x7f974c159bf0 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f9743fff640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97380047c0 0x7f9738004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f9743fff640 1 -- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c159bf0 msgr2=0x7f9738005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.418+0000 7f974bfff640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c159bf0 0x7f9738005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.419+0000 7f974b7fe640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97380039a0 0x7f9738003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.419+0000 7f9743fff640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c159bf0 0x7f9738005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.419+0000 7f9743fff640 1 -- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97380039a0 msgr2=0x7f9738003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.419+0000 7f9743fff640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97380039a0 0x7f9738003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.419+0000 7f9743fff640 1 -- 172.21.15.115:0/881424660 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9738003520 con 0x7f97380047c0 2023-12-01T03:10:22.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.419+0000 7f974bfff640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c159bf0 0x7f9738005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.420+0000 7f9743fff640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97380047c0 0x7f9738004c40 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f9734001720 tx=0x7f97380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba93e2039b70555 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.420+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340052d0 con 0x7f97380047c0 2023-12-01T03:10:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.420+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973402e050 con 0x7f97380047c0 2023-12-01T03:10:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.420+0000 7f974affd640 1 -- 172.21.15.115:0/881424660 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340035f0 con 0x7f97380047c0 2023-12-01T03:10:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.421+0000 7f9751fab640 1 -- 172.21.15.115:0/881424660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97380047c0 msgr2=0x7f9738004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.421+0000 7f9751fab640 1 --2- 172.21.15.115:0/881424660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97380047c0 0x7f9738004c40 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f9734001720 tx=0x7f97380057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.421+0000 7f974bfff640 1 -- 172.21.15.115:0/881424660 reap_dead start 2023-12-01T03:10:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.421+0000 7f9751fab640 1 -- 172.21.15.115:0/881424660 shutdown_connections 2023-12-01T03:10:22.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.421+0000 7f9751fab640 1 -- 172.21.15.115:0/881424660 >> 172.21.15.115:0/881424660 conn(0x7f974c0662c0 msgr2=0x7f974c0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.421+0000 7f9751fab640 1 -- 172.21.15.115:0/881424660 shutdown_connections 2023-12-01T03:10:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.421+0000 7f9751fab640 1 -- 172.21.15.115:0/881424660 wait complete. 2023-12-01T03:10:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.423+0000 7f9751fab640 1 Processor -- start 2023-12-01T03:10:22.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.423+0000 7f9751fab640 1 -- start start 2023-12-01T03:10:22.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.424+0000 7f9751fab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97380047c0 0x7f974c0c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:22.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.424+0000 7f9751fab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c0c8080 0x7f974c1f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:22.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.424+0000 7f9751fab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c1f2850 0x7f974c13d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.424+0000 7f9751fab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c15d0c0 con 0x7f974c1f2850 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.424+0000 7f9751fab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c0c4fa0 con 0x7f97380047c0 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.424+0000 7f9751fab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c1f2cd0 con 0x7f974c0c8080 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.424+0000 7f974bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c1f2850 0x7f974c13d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.424+0000 7f974bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c1f2850 0x7f974c13d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35272/0 (socket says 172.21.15.115:35272) 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.424+0000 7f974bfff640 1 -- 172.21.15.115:0/927543310 learned_addr learned my addr 172.21.15.115:0/927543310 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.424+0000 7f9743fff640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c0c8080 0x7f974c1f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f974b7fe640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97380047c0 0x7f974c0c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f974bfff640 1 -- 172.21.15.115:0/927543310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c0c8080 msgr2=0x7f974c1f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f974bfff640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c0c8080 0x7f974c1f2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f974bfff640 1 -- 172.21.15.115:0/927543310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97380047c0 msgr2=0x7f974c0c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f974bfff640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97380047c0 0x7f974c0c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f974bfff640 1 -- 172.21.15.115:0/927543310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c15cde0 con 0x7f974c1f2850 2023-12-01T03:10:22.428 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f974bfff640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c1f2850 0x7f974c13d570 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f9744000f80 tx=0x7f974400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:22.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f9748ff9640 1 -- 172.21.15.115:0/927543310 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974400de90 con 0x7f974c1f2850 2023-12-01T03:10:22.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f9748ff9640 1 -- 172.21.15.115:0/927543310 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974400bb80 con 0x7f974c1f2850 2023-12-01T03:10:22.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f9748ff9640 1 -- 172.21.15.115:0/927543310 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744007070 con 0x7f974c1f2850 2023-12-01T03:10:22.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f9751fab640 1 -- 172.21.15.115:0/927543310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c0c75a0 con 0x7f974c1f2850 2023-12-01T03:10:22.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.425+0000 7f9751fab640 1 -- 172.21.15.115:0/927543310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c0c7b60 con 0x7f974c1f2850 2023-12-01T03:10:22.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.426+0000 7f9751fab640 1 -- 172.21.15.115:0/927543310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c0c7400 con 0x7f974c1f2850 2023-12-01T03:10:22.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.427+0000 7f9748ff9640 1 -- 172.21.15.115:0/927543310 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f97440071d0 con 0x7f974c1f2850 2023-12-01T03:10:22.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.428+0000 7f9748ff9640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f972c06a580 0x7f972c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:22.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.428+0000 7f9748ff9640 1 -- 172.21.15.115:0/927543310 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f9744088aa0 con 0x7f974c1f2850 2023-12-01T03:10:22.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.429+0000 7f974b7fe640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f972c06a580 0x7f972c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:22.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.429+0000 7f974b7fe640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f972c06a580 0x7f972c06ca40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f974c0ad5c0 tx=0x7f973c008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:22.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.433+0000 7f9748ff9640 1 -- 172.21.15.115:0/927543310 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f974c0c7400 con 0x7f974c1f2850 2023-12-01T03:10:22.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:22.626+0000 7f9751fab640 1 -- 172.21.15.115:0/927543310 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f974c0c7400 con 0x7f974c1f2850 2023-12-01T03:10:23.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.641+0000 7f9748ff9640 1 -- 172.21.15.115:0/927543310 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v211) v1 ==== 138+0+0 (secure 0 0 0) 0x7f974c0c7400 con 0x7f974c1f2850 2023-12-01T03:10:23.643 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T03:10:23.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.646+0000 7f9751fab640 1 -- 172.21.15.115:0/927543310 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f972c06a580 msgr2=0x7f972c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:23.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.646+0000 7f9751fab640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f972c06a580 0x7f972c06ca40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f974c0ad5c0 tx=0x7f973c008040 comp rx=0 tx=0).stop 2023-12-01T03:10:23.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.646+0000 7f9751fab640 1 -- 172.21.15.115:0/927543310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c1f2850 msgr2=0x7f974c13d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:23.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.646+0000 7f9751fab640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c1f2850 0x7f974c13d570 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f9744000f80 tx=0x7f974400b4a0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.647+0000 7f9751fab640 1 -- 172.21.15.115:0/927543310 shutdown_connections 2023-12-01T03:10:23.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.647+0000 7f9751fab640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f972c06a580 0x7f972c06ca40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.647+0000 7f9751fab640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f974c0c8080 0x7f974c1f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.647+0000 7f9751fab640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f974c1f2850 0x7f974c13d570 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.647+0000 7f9751fab640 1 --2- 172.21.15.115:0/927543310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97380047c0 0x7f974c0c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.647+0000 7f9751fab640 1 -- 172.21.15.115:0/927543310 >> 172.21.15.115:0/927543310 conn(0x7f974c0662c0 msgr2=0x7f974c0b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:23.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.647+0000 7f9751fab640 1 -- 172.21.15.115:0/927543310 shutdown_connections 2023-12-01T03:10:23.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.647+0000 7f9751fab640 1 -- 172.21.15.115:0/927543310 wait complete. 2023-12-01T03:10:23.673 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T03:10:23.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.890+0000 7f1f5928c640 1 Processor -- start 2023-12-01T03:10:23.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.890+0000 7f1f5928c640 1 -- start start 2023-12-01T03:10:23.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.892+0000 7f1f5928c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541599f0 0x7f1f540ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:23.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.892+0000 7f1f5928c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f540cc460 con 0x7f1f540cd6a0 2023-12-01T03:10:23.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.892+0000 7f1f5928c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f540cdba0 con 0x7f1f540b00e0 2023-12-01T03:10:23.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.892+0000 7f1f5928c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f540ce440 con 0x7f1f541599f0 2023-12-01T03:10:23.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.892+0000 7f1f4b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1f540cd6a0 0x7f1f540c8cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57024/0 (socket says 172.21.15.115:57024) 2023-12-01T03:10:23.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.892+0000 7f1f4b7fe640 1 -- 172.21.15.115:0/4214964088 learned_addr learned my addr 172.21.15.115:0/4214964088 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:23.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253696641 0 0) 0x7f1f540cc460 con 0x7f1f540cd6a0 2023-12-01T03:10:23.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f40003520 con 0x7f1f540cd6a0 2023-12-01T03:10:23.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f53fff640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541599f0 0x7f1f540ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:23.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 955033724 0 0) 0x7f1f40003520 con 0x7f1f540cd6a0 2023-12-01T03:10:23.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541599f0 msgr2=0x7f1f540ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:23.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541599f0 0x7f1f540ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 >> v1:172.21.15.142:6789/0 conn(0x7f1f540b00e0 legacy=0x7f1f540cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:23.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f540cde80 con 0x7f1f540cd6a0 2023-12-01T03:10:23.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f1f3c001670 con 0x7f1f540cd6a0 2023-12-01T03:10:23.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 >> v1:172.21.15.115:6789/0 conn(0x7f1f540cd6a0 legacy=0x7f1f540c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:23.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f400039a0 0x7f1f40003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:23.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f400047c0 0x7f1f40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:23.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541599f0 0x7f1f4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:23.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f540ce440 con 0x7f1f400039a0 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f400054e0 con 0x7f1f400047c0 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.893+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f40005650 con 0x7f1f541599f0 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.894+0000 7f1f53fff640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f400039a0 0x7f1f40003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.894+0000 7f1f53fff640 1 -- 172.21.15.115:0/4214964088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541599f0 msgr2=0x7f1f4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.894+0000 7f1f53fff640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541599f0 0x7f1f4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.894+0000 7f1f53fff640 1 -- 172.21.15.115:0/4214964088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f400047c0 msgr2=0x7f1f40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.894+0000 7f1f4b7fe640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f400047c0 0x7f1f40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.894+0000 7f1f53fff640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f400047c0 0x7f1f40004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.894+0000 7f1f53fff640 1 -- 172.21.15.115:0/4214964088 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f40003520 con 0x7f1f400039a0 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.895+0000 7f1f53fff640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f400039a0 0x7f1f40003da0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400ba60 tx=0x7f1f400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e76171d3c13e09 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:23.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.895+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44032680 con 0x7f1f400039a0 2023-12-01T03:10:23.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.895+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f440327e0 con 0x7f1f400039a0 2023-12-01T03:10:23.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.895+0000 7f1f537fe640 1 -- 172.21.15.115:0/4214964088 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44032b70 con 0x7f1f400039a0 2023-12-01T03:10:23.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.896+0000 7f1f5928c640 1 -- 172.21.15.115:0/4214964088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f400039a0 msgr2=0x7f1f40003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.896+0000 7f1f5928c640 1 --2- 172.21.15.115:0/4214964088 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f400039a0 0x7f1f40003da0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400ba60 tx=0x7f1f400057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.896+0000 7f1f58a8b640 1 -- 172.21.15.115:0/4214964088 reap_dead start 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.896+0000 7f1f5928c640 1 -- 172.21.15.115:0/4214964088 shutdown_connections 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.896+0000 7f1f5928c640 1 -- 172.21.15.115:0/4214964088 >> 172.21.15.115:0/4214964088 conn(0x7f1f54066400 msgr2=0x7f1f540cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.896+0000 7f1f5928c640 1 -- 172.21.15.115:0/4214964088 shutdown_connections 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.896+0000 7f1f5928c640 1 -- 172.21.15.115:0/4214964088 wait complete. 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.898+0000 7f1f5928c640 1 Processor -- start 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.898+0000 7f1f5928c640 1 -- start start 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.899+0000 7f1f5928c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f400039a0 0x7f1f54071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.899+0000 7f1f5928c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541f6100 0x7f1f540c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.899+0000 7f1f5928c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f540c5c70 0x7f1f54157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.899+0000 7f1f5928c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f540c61f0 con 0x7f1f540c5c70 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.899+0000 7f1f5928c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f541f5620 con 0x7f1f400039a0 2023-12-01T03:10:23.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.899+0000 7f1f5928c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f541f5900 con 0x7f1f541f6100 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.899+0000 7f1f58a8b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f540c5c70 0x7f1f54157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.899+0000 7f1f58a8b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f540c5c70 0x7f1f54157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35294/0 (socket says 172.21.15.115:35294) 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.899+0000 7f1f58a8b640 1 -- 172.21.15.115:0/3573518518 learned_addr learned my addr 172.21.15.115:0/3573518518 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f4b7fe640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541f6100 0x7f1f540c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f53fff640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f400039a0 0x7f1f54071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f58a8b640 1 -- 172.21.15.115:0/3573518518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541f6100 msgr2=0x7f1f540c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f58a8b640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541f6100 0x7f1f540c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f58a8b640 1 -- 172.21.15.115:0/3573518518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f400039a0 msgr2=0x7f1f54071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f58a8b640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f400039a0 0x7f1f54071310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f58a8b640 1 -- 172.21.15.115:0/3573518518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f541f5be0 con 0x7f1f540c5c70 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f58a8b640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f540c5c70 0x7f1f54157760 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f1f540cbd30 tx=0x7f1f4c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f517fa640 1 -- 172.21.15.115:0/3573518518 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c007070 con 0x7f1f540c5c70 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f517fa640 1 -- 172.21.15.115:0/3573518518 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c0071f0 con 0x7f1f540c5c70 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f517fa640 1 -- 172.21.15.115:0/3573518518 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0075a0 con 0x7f1f540c5c70 2023-12-01T03:10:23.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.900+0000 7f1f5928c640 1 -- 172.21.15.115:0/3573518518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f540c64d0 con 0x7f1f540c5c70 2023-12-01T03:10:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.901+0000 7f1f5928c640 1 -- 172.21.15.115:0/3573518518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f54071850 con 0x7f1f540c5c70 2023-12-01T03:10:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.902+0000 7f1f5928c640 1 -- 172.21.15.115:0/3573518518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f540618d0 con 0x7f1f540c5c70 2023-12-01T03:10:23.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.903+0000 7f1f517fa640 1 -- 172.21.15.115:0/3573518518 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1f4c007700 con 0x7f1f540c5c70 2023-12-01T03:10:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.904+0000 7f1f517fa640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f3406a550 0x7f1f3406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.904+0000 7f1f517fa640 1 -- 172.21.15.115:0/3573518518 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 5271+0+0 (secure 0 0 0) 0x7f1f4c088250 con 0x7f1f540c5c70 2023-12-01T03:10:23.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.905+0000 7f1f53fff640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f3406a550 0x7f1f3406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:23.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.905+0000 7f1f53fff640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f3406a550 0x7f1f3406ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400ba30 tx=0x7f1f44030000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:23.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:23.908+0000 7f1f517fa640 1 -- 172.21.15.115:0/3573518518 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1f4c0538c0 con 0x7f1f540c5c70 2023-12-01T03:10:24.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.099+0000 7f1f5928c640 1 -- 172.21.15.115:0/3573518518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1f540cd480 con 0x7f1f540c5c70 2023-12-01T03:10:24.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.666+0000 7f1f517fa640 1 -- 172.21.15.115:0/3573518518 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v212) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1f4c0170d0 con 0x7f1f540c5c70 2023-12-01T03:10:24.669 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T03:10:24.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.675+0000 7f1f5928c640 1 -- 172.21.15.115:0/3573518518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f3406a550 msgr2=0x7f1f3406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:24.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.675+0000 7f1f5928c640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f3406a550 0x7f1f3406ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400ba30 tx=0x7f1f44030000 comp rx=0 tx=0).stop 2023-12-01T03:10:24.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.675+0000 7f1f5928c640 1 -- 172.21.15.115:0/3573518518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f540c5c70 msgr2=0x7f1f54157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:24.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.675+0000 7f1f5928c640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f540c5c70 0x7f1f54157760 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f1f540cbd30 tx=0x7f1f4c00c980 comp rx=0 tx=0).stop 2023-12-01T03:10:24.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.676+0000 7f1f5928c640 1 -- 172.21.15.115:0/3573518518 shutdown_connections 2023-12-01T03:10:24.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.676+0000 7f1f5928c640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f3406a550 0x7f1f3406ca10 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400ba30 tx=0x7f1f44030000 comp rx=0 tx=0).stop 2023-12-01T03:10:24.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.676+0000 7f1f5928c640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f541f6100 0x7f1f540c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:24.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.676+0000 7f1f5928c640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f540c5c70 0x7f1f54157760 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:24.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.676+0000 7f1f5928c640 1 --2- 172.21.15.115:0/3573518518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f400039a0 0x7f1f54071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:24.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.676+0000 7f1f5928c640 1 -- 172.21.15.115:0/3573518518 >> 172.21.15.115:0/3573518518 conn(0x7f1f54066400 msgr2=0x7f1f540c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:24.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.679+0000 7f1f5928c640 1 -- 172.21.15.115:0/3573518518 shutdown_connections 2023-12-01T03:10:24.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.680+0000 7f1f5928c640 1 -- 172.21.15.115:0/3573518518 wait complete. 2023-12-01T03:10:24.709 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T03:10:24.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.894+0000 7f588d527640 1 Processor -- start 2023-12-01T03:10:24.896 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.894+0000 7f588d527640 1 -- start start 2023-12-01T03:10:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.895+0000 7f588d527640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f588815cd90 0x7f58880cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.895+0000 7f588d527640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58880ce5e0 con 0x7f58880b5fa0 2023-12-01T03:10:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.895+0000 7f588d527640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58880ce300 con 0x7f58880cfd60 2023-12-01T03:10:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.895+0000 7f588d527640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58880d0ec0 con 0x7f588815cd90 2023-12-01T03:10:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.896+0000 7f58867fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f58880b5fa0 0x7f58880cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57034/0 (socket says 172.21.15.115:57034) 2023-12-01T03:10:24.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.896+0000 7f58867fc640 1 -- 172.21.15.115:0/3919467499 learned_addr learned my addr 172.21.15.115:0/3919467499 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:24.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.896+0000 7f5886ffd640 1 --2- 172.21.15.115:0/3919467499 >> v2:172.21.15.142:3300/0 conn(0x7f588815cd90 0x7f58880cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752968114 0 0) 0x7f587c001110 con 0x7f58880b5fa0 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5870003520 con 0x7f58880b5fa0 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2906227314 0 0) 0x7f5870003520 con 0x7f58880b5fa0 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 >> v2:172.21.15.142:3300/0 conn(0x7f588815cd90 msgr2=0x7f58880cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 --2- 172.21.15.115:0/3919467499 >> v2:172.21.15.142:3300/0 conn(0x7f588815cd90 0x7f58880cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 >> v1:172.21.15.142:6789/0 conn(0x7f58880cfd60 legacy=0x7f58880d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58880d10a0 con 0x7f58880b5fa0 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f587c001800 con 0x7f58880b5fa0 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 >> v1:172.21.15.115:6789/0 conn(0x7f58880b5fa0 legacy=0x7f58880cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 --2- 172.21.15.115:0/3919467499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5870003970 0x7f5870003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:24.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 --2- 172.21.15.115:0/3919467499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58700047c0 0x7f5870004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 --2- 172.21.15.115:0/3919467499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5870005180 0x7f587000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880d0ec0 con 0x7f5870003970 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587000c020 con 0x7f5870005180 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.897+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587000c1e0 con 0x7f58700047c0 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.898+0000 7f5886ffd640 1 --2- 172.21.15.115:0/3919467499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5870003970 0x7f5870003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.898+0000 7f5886ffd640 1 -- 172.21.15.115:0/3919467499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58700047c0 msgr2=0x7f5870004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.898+0000 7f5886ffd640 1 --2- 172.21.15.115:0/3919467499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58700047c0 0x7f5870004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.898+0000 7f5886ffd640 1 -- 172.21.15.115:0/3919467499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5870005180 msgr2=0x7f587000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.898+0000 7f5886ffd640 1 --2- 172.21.15.115:0/3919467499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5870005180 0x7f587000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.898+0000 7f5886ffd640 1 -- 172.21.15.115:0/3919467499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5870003520 con 0x7f5870003970 2023-12-01T03:10:24.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.898+0000 7f58877fe640 1 -- 172.21.15.115:0/3919467499 reap_dead start 2023-12-01T03:10:24.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.899+0000 7f5886ffd640 1 --2- 172.21.15.115:0/3919467499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5870003970 0x7f5870003d90 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f5878009ac0 tx=0x7f58700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1789859e2d6132 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:24.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.899+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587802c710 con 0x7f5870003970 2023-12-01T03:10:24.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.899+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587802c870 con 0x7f5870003970 2023-12-01T03:10:24.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.899+0000 7f5885ffb640 1 -- 172.21.15.115:0/3919467499 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587802cc00 con 0x7f5870003970 2023-12-01T03:10:24.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.900+0000 7f588d527640 1 -- 172.21.15.115:0/3919467499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5870003970 msgr2=0x7f5870003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:24.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.900+0000 7f588d527640 1 --2- 172.21.15.115:0/3919467499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5870003970 0x7f5870003d90 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f5878009ac0 tx=0x7f58700057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:24.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.900+0000 7f588d527640 1 -- 172.21.15.115:0/3919467499 shutdown_connections 2023-12-01T03:10:24.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.900+0000 7f588d527640 1 --2- 172.21.15.115:0/3919467499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5870003970 0x7f5870003d90 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:24.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.900+0000 7f588d527640 1 -- 172.21.15.115:0/3919467499 >> 172.21.15.115:0/3919467499 conn(0x7f5888065960 msgr2=0x7f5888065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:24.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.900+0000 7f588d527640 1 -- 172.21.15.115:0/3919467499 shutdown_connections 2023-12-01T03:10:24.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.900+0000 7f588d527640 1 -- 172.21.15.115:0/3919467499 wait complete. 2023-12-01T03:10:24.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.902+0000 7f588d527640 1 Processor -- start 2023-12-01T03:10:24.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.902+0000 7f588d527640 1 -- start start 2023-12-01T03:10:24.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.903+0000 7f588d527640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5870003970 0x7f58881f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:24.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.903+0000 7f588d527640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58880caff0 0x7f58881f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:24.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.903+0000 7f588d527640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58880cc920 0x7f5888153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:24.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.903+0000 7f588d527640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880cb450 con 0x7f58880cc920 2023-12-01T03:10:24.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.903+0000 7f588d527640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588815a9c0 con 0x7f5870003970 2023-12-01T03:10:24.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.903+0000 7f588d527640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58880cd190 con 0x7f58880caff0 2023-12-01T03:10:24.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.903+0000 7f58877fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58880cc920 0x7f5888153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:24.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.903+0000 7f58877fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58880cc920 0x7f5888153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35320/0 (socket says 172.21.15.115:35320) 2023-12-01T03:10:24.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.903+0000 7f58877fe640 1 -- 172.21.15.115:0/4039898079 learned_addr learned my addr 172.21.15.115:0/4039898079 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:24.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f58877fe640 1 -- 172.21.15.115:0/4039898079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58880caff0 msgr2=0x7f58881f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f58867fc640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58880caff0 0x7f58881f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f58877fe640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58880caff0 0x7f58881f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f58877fe640 1 -- 172.21.15.115:0/4039898079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5870003970 msgr2=0x7f58881f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f58877fe640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5870003970 0x7f58881f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f58877fe640 1 -- 172.21.15.115:0/4039898079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588815a090 con 0x7f58880cc920 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f58877fe640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58880cc920 0x7f5888153fc0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f58880cf260 tx=0x7f588000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f586f7fe640 1 -- 172.21.15.115:0/4039898079 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58800070d0 con 0x7f58880cc920 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f586f7fe640 1 -- 172.21.15.115:0/4039898079 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5880007250 con 0x7f58880cc920 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f586f7fe640 1 -- 172.21.15.115:0/4039898079 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880007600 con 0x7f58880cc920 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.904+0000 7f588d527640 1 -- 172.21.15.115:0/4039898079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588815a370 con 0x7f58880cc920 2023-12-01T03:10:24.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.905+0000 7f588d527640 1 -- 172.21.15.115:0/4039898079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58880ccda0 con 0x7f58880cc920 2023-12-01T03:10:24.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.906+0000 7f588d527640 1 -- 172.21.15.115:0/4039898079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588804e890 con 0x7f58880cc920 2023-12-01T03:10:24.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.907+0000 7f586f7fe640 1 -- 172.21.15.115:0/4039898079 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f5880007760 con 0x7f58880cc920 2023-12-01T03:10:24.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.908+0000 7f586f7fe640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f587406a580 0x7f587406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:24.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.909+0000 7f586f7fe640 1 -- 172.21.15.115:0/4039898079 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5653+0+0 (secure 0 0 0) 0x7f5880088650 con 0x7f58880cc920 2023-12-01T03:10:24.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.909+0000 7f5886ffd640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f587406a580 0x7f587406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:24.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.909+0000 7f5886ffd640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f587406a580 0x7f587406ca40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5878036000 tx=0x7f587802e000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:24.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:24.913+0000 7f586f7fe640 1 -- 172.21.15.115:0/4039898079 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5880053bd0 con 0x7f58880cc920 2023-12-01T03:10:25.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.143+0000 7f588d527640 1 -- 172.21.15.115:0/4039898079 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f588815ab00 con 0x7f58880cc920 2023-12-01T03:10:25.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.155+0000 7f586f7fe640 1 -- 172.21.15.115:0/4039898079 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v91) v1 ==== 174+0+0 (secure 0 0 0) 0x7f588815ab00 con 0x7f58880cc920 2023-12-01T03:10:25.157 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 24 and data pool 25 2023-12-01T03:10:25.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.159+0000 7f588d527640 1 -- 172.21.15.115:0/4039898079 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f587406a580 msgr2=0x7f587406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:25.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.159+0000 7f588d527640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f587406a580 0x7f587406ca40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5878036000 tx=0x7f587802e000 comp rx=0 tx=0).stop 2023-12-01T03:10:25.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.159+0000 7f588d527640 1 -- 172.21.15.115:0/4039898079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58880cc920 msgr2=0x7f5888153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:25.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.159+0000 7f588d527640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58880cc920 0x7f5888153fc0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f58880cf260 tx=0x7f588000c960 comp rx=0 tx=0).stop 2023-12-01T03:10:25.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.159+0000 7f588d527640 1 -- 172.21.15.115:0/4039898079 shutdown_connections 2023-12-01T03:10:25.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.159+0000 7f588d527640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f587406a580 0x7f587406ca40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.159+0000 7f588d527640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58880caff0 0x7f58881f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.159+0000 7f588d527640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58880cc920 0x7f5888153fc0 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.160+0000 7f588d527640 1 --2- 172.21.15.115:0/4039898079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5870003970 0x7f58881f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.160+0000 7f588d527640 1 -- 172.21.15.115:0/4039898079 >> 172.21.15.115:0/4039898079 conn(0x7f5888065960 msgr2=0x7f58880b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:25.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.165+0000 7f588d527640 1 -- 172.21.15.115:0/4039898079 shutdown_connections 2023-12-01T03:10:25.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.166+0000 7f588d527640 1 -- 172.21.15.115:0/4039898079 wait complete. 2023-12-01T03:10:25.198 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:25.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.374+0000 7f319d3ca640 1 Processor -- start 2023-12-01T03:10:25.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.374+0000 7f319d3ca640 1 -- start start 2023-12-01T03:10:25.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.375+0000 7f319d3ca640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31981584b0 0x7f31980cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.375+0000 7f319d3ca640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f319815afe0 0x7f319806b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f319d3ca640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31980cc180 con 0x7f319806b7b0 2023-12-01T03:10:25.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f319d3ca640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319806bc40 con 0x7f319815afe0 2023-12-01T03:10:25.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f319d3ca640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319815b550 con 0x7f31981584b0 2023-12-01T03:10:25.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f31977fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f319815afe0 0x7f319806b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:25.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f31977fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f319815afe0 0x7f319806b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35330/0 (socket says 172.21.15.115:35330) 2023-12-01T03:10:25.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f31977fe640 1 -- 172.21.15.115:0/1528258402 learned_addr learned my addr 172.21.15.115:0/1528258402 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f31977fe640 1 -- 172.21.15.115:0/1528258402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31981584b0 msgr2=0x7f31980cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f3197fff640 1 --2- 172.21.15.115:0/1528258402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31981584b0 0x7f31980cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f31977fe640 1 --2- 172.21.15.115:0/1528258402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31981584b0 0x7f31980cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f31977fe640 1 -- 172.21.15.115:0/1528258402 >> v1:172.21.15.142:6789/0 conn(0x7f319806b7b0 legacy=0x7f319815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f31977fe640 1 -- 172.21.15.115:0/1528258402 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319815b830 con 0x7f319815afe0 2023-12-01T03:10:25.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.376+0000 7f3197fff640 1 --2- 172.21.15.115:0/1528258402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31981584b0 0x7f31980cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.377+0000 7f31977fe640 1 --2- 172.21.15.115:0/1528258402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f319815afe0 0x7f319806b270 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f318c009870 tx=0x7f318c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d366588c7ae8e772 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.378+0000 7f3196ffd640 1 -- 172.21.15.115:0/1528258402 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c031640 con 0x7f319815afe0 2023-12-01T03:10:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.378+0000 7f3196ffd640 1 -- 172.21.15.115:0/1528258402 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c0317a0 con 0x7f319815afe0 2023-12-01T03:10:25.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.378+0000 7f3196ffd640 1 -- 172.21.15.115:0/1528258402 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c031b30 con 0x7f319815afe0 2023-12-01T03:10:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.378+0000 7f319d3ca640 1 -- 172.21.15.115:0/1528258402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f319815afe0 msgr2=0x7f319806b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.378+0000 7f319d3ca640 1 --2- 172.21.15.115:0/1528258402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f319815afe0 0x7f319806b270 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f318c009870 tx=0x7f318c02b8f0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.379+0000 7f319d3ca640 1 -- 172.21.15.115:0/1528258402 shutdown_connections 2023-12-01T03:10:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.379+0000 7f319d3ca640 1 --2- 172.21.15.115:0/1528258402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f319815afe0 0x7f319806b270 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.379+0000 7f319d3ca640 1 --2- 172.21.15.115:0/1528258402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31981584b0 0x7f31980cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.379+0000 7f319d3ca640 1 -- 172.21.15.115:0/1528258402 >> 172.21.15.115:0/1528258402 conn(0x7f31980ce640 msgr2=0x7f31980cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.379+0000 7f319d3ca640 1 -- 172.21.15.115:0/1528258402 shutdown_connections 2023-12-01T03:10:25.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.379+0000 7f319d3ca640 1 -- 172.21.15.115:0/1528258402 wait complete. 2023-12-01T03:10:25.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.380+0000 7f319d3ca640 1 Processor -- start 2023-12-01T03:10:25.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.381+0000 7f319d3ca640 1 -- start start 2023-12-01T03:10:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.381+0000 7f319d3ca640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f319806b7b0 0x7f31980c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.381+0000 7f319d3ca640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31981584b0 0x7f31981f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f319d3ca640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f319815afe0 0x7f31981f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f319d3ca640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319815a1c0 con 0x7f31981584b0 2023-12-01T03:10:25.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f319d3ca640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3198159540 con 0x7f319806b7b0 2023-12-01T03:10:25.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f319d3ca640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3198159820 con 0x7f319815afe0 2023-12-01T03:10:25.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f31977fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31981584b0 0x7f31981f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:25.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f31977fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31981584b0 0x7f31981f1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35346/0 (socket says 172.21.15.115:35346) 2023-12-01T03:10:25.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f31977fe640 1 -- 172.21.15.115:0/3569122342 learned_addr learned my addr 172.21.15.115:0/3569122342 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:25.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f3197fff640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f319806b7b0 0x7f31980c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:25.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f319cbc9640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f319815afe0 0x7f31981f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:25.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f31977fe640 1 -- 172.21.15.115:0/3569122342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f319815afe0 msgr2=0x7f31981f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:25.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f31977fe640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f319815afe0 0x7f31981f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f31977fe640 1 -- 172.21.15.115:0/3569122342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f319806b7b0 msgr2=0x7f31980c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:25.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f31977fe640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f319806b7b0 0x7f31980c5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f31977fe640 1 -- 172.21.15.115:0/3569122342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c0094d0 con 0x7f31981584b0 2023-12-01T03:10:25.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.382+0000 7f319cbc9640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f319815afe0 0x7f31981f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:25.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.383+0000 7f31977fe640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31981584b0 0x7f31981f1c80 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:25.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.383+0000 7f3194ff9640 1 -- 172.21.15.115:0/3569122342 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c03ac60 con 0x7f31981584b0 2023-12-01T03:10:25.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.383+0000 7f3194ff9640 1 -- 172.21.15.115:0/3569122342 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c03ade0 con 0x7f31981584b0 2023-12-01T03:10:25.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.383+0000 7f3194ff9640 1 -- 172.21.15.115:0/3569122342 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c044630 con 0x7f31981584b0 2023-12-01T03:10:25.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.383+0000 7f319d3ca640 1 -- 172.21.15.115:0/3569122342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3198159b00 con 0x7f31981584b0 2023-12-01T03:10:25.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.383+0000 7f319d3ca640 1 -- 172.21.15.115:0/3569122342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31980c6530 con 0x7f31981584b0 2023-12-01T03:10:25.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.384+0000 7f319d3ca640 1 -- 172.21.15.115:0/3569122342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31980673a0 con 0x7f31981584b0 2023-12-01T03:10:25.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.385+0000 7f3194ff9640 1 -- 172.21.15.115:0/3569122342 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f318c02f050 con 0x7f31981584b0 2023-12-01T03:10:25.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.386+0000 7f3194ff9640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f318406a550 0x7f318406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.386+0000 7f3194ff9640 1 -- 172.21.15.115:0/3569122342 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f318c0aabf0 con 0x7f31981584b0 2023-12-01T03:10:25.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.387+0000 7f3197fff640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f318406a550 0x7f318406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:25.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.387+0000 7f3197fff640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f318406a550 0x7f318406ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f318800a990 tx=0x7f3188008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.391+0000 7f3194ff9640 1 -- 172.21.15.115:0/3569122342 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f318c0760e0 con 0x7f31981584b0 2023-12-01T03:10:25.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.575+0000 7f319d3ca640 1 -- 172.21.15.115:0/3569122342 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31980673a0 con 0x7f31981584b0 2023-12-01T03:10:25.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.578+0000 7f3194ff9640 1 -- 172.21.15.115:0/3569122342 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) v1 ==== 74+0+16186 (secure 0 0 0) 0x7f318c079700 con 0x7f31981584b0 2023-12-01T03:10:25.580 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:25.581 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":213,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:25.145152+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:25.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 -- 172.21.15.115:0/3569122342 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f318406a550 msgr2=0x7f318406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:25.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f318406a550 0x7f318406ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f318800a990 tx=0x7f3188008040 comp rx=0 tx=0).stop 2023-12-01T03:10:25.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 -- 172.21.15.115:0/3569122342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31981584b0 msgr2=0x7f31981f1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:25.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31981584b0 0x7f31981f1c80 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f318c000c00 tx=0x7f318c0046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 -- 172.21.15.115:0/3569122342 shutdown_connections 2023-12-01T03:10:25.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f318406a550 0x7f318406ca10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f319815afe0 0x7f31981f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31981584b0 0x7f31981f1c80 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 --2- 172.21.15.115:0/3569122342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f319806b7b0 0x7f31980c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 -- 172.21.15.115:0/3569122342 >> 172.21.15.115:0/3569122342 conn(0x7f31980ce640 msgr2=0x7f31980b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:25.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 -- 172.21.15.115:0/3569122342 shutdown_connections 2023-12-01T03:10:25.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.581+0000 7f319d3ca640 1 -- 172.21.15.115:0/3569122342 wait complete. 2023-12-01T03:10:25.601 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:25.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.797+0000 7ff4148af640 1 Processor -- start 2023-12-01T03:10:25.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.798+0000 7ff4148af640 1 -- start start 2023-12-01T03:10:25.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.799+0000 7ff4148af640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff4100b5d80 0x7ff4100d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.799+0000 7ff4148af640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff4100d1230 0x7ff4100d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.799+0000 7ff4148af640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4100d1b90 con 0x7ff41015cd20 2023-12-01T03:10:25.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.799+0000 7ff4148af640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4100b2160 con 0x7ff4100d1230 2023-12-01T03:10:25.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.799+0000 7ff4148af640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4100b8c80 con 0x7ff4100b5d80 2023-12-01T03:10:25.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.799+0000 7ff40ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff4100d1230 0x7ff4100d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:25.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.799+0000 7ff40ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff4100d1230 0x7ff4100d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35354/0 (socket says 172.21.15.115:35354) 2023-12-01T03:10:25.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.799+0000 7ff40ed9d640 1 -- 172.21.15.115:0/599446697 learned_addr learned my addr 172.21.15.115:0/599446697 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:25.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.800+0000 7ff40ed9d640 1 -- 172.21.15.115:0/599446697 >> v2:172.21.15.142:3300/0 conn(0x7ff4100b5d80 msgr2=0x7ff4100d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:25.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.800+0000 7ff40ed9d640 1 --2- 172.21.15.115:0/599446697 >> v2:172.21.15.142:3300/0 conn(0x7ff4100b5d80 0x7ff4100d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.800+0000 7ff40ed9d640 1 -- 172.21.15.115:0/599446697 >> v1:172.21.15.142:6789/0 conn(0x7ff41015cd20 legacy=0x7ff4100b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:25.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.800+0000 7ff40ed9d640 1 -- 172.21.15.115:0/599446697 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4100ce2e0 con 0x7ff4100d1230 2023-12-01T03:10:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.801+0000 7ff40ed9d640 1 --2- 172.21.15.115:0/599446697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 0x7ff4100d1650 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7ff4100cf630 tx=0x7ff40802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bbd6086ed59760 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.801+0000 7ff40d59a640 1 -- 172.21.15.115:0/599446697 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40802c9f0 con 0x7ff4100d1230 2023-12-01T03:10:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.801+0000 7ff40d59a640 1 -- 172.21.15.115:0/599446697 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40802cb50 con 0x7ff4100d1230 2023-12-01T03:10:25.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.801+0000 7ff40d59a640 1 -- 172.21.15.115:0/599446697 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080343f0 con 0x7ff4100d1230 2023-12-01T03:10:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.802+0000 7ff4148af640 1 -- 172.21.15.115:0/599446697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 msgr2=0x7ff4100d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.802+0000 7ff4148af640 1 --2- 172.21.15.115:0/599446697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 0x7ff4100d1650 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7ff4100cf630 tx=0x7ff40802be50 comp rx=0 tx=0).stop 2023-12-01T03:10:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.802+0000 7ff4148af640 1 -- 172.21.15.115:0/599446697 shutdown_connections 2023-12-01T03:10:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.802+0000 7ff4148af640 1 --2- 172.21.15.115:0/599446697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 0x7ff4100d1650 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.802+0000 7ff4148af640 1 --2- 172.21.15.115:0/599446697 >> v2:172.21.15.142:3300/0 conn(0x7ff4100b5d80 0x7ff4100d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.802+0000 7ff4148af640 1 -- 172.21.15.115:0/599446697 >> 172.21.15.115:0/599446697 conn(0x7ff41006a0e0 msgr2=0x7ff41006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.802+0000 7ff4148af640 1 -- 172.21.15.115:0/599446697 shutdown_connections 2023-12-01T03:10:25.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.802+0000 7ff4148af640 1 -- 172.21.15.115:0/599446697 wait complete. 2023-12-01T03:10:25.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.804+0000 7ff4148af640 1 Processor -- start 2023-12-01T03:10:25.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.804+0000 7ff4148af640 1 -- start start 2023-12-01T03:10:25.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff4148af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4100b5d80 0x7ff410158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff4148af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 0x7ff410155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff4148af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff41015cd20 0x7ff4100cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff4148af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100b4e20 con 0x7ff4100d1230 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff4148af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100b0e40 con 0x7ff41015cd20 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff4148af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100b1120 con 0x7ff4100b5d80 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff40dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 0x7ff410155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff40dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 0x7ff410155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35370/0 (socket says 172.21.15.115:35370) 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff40dd9b640 1 -- 172.21.15.115:0/148992931 learned_addr learned my addr 172.21.15.115:0/148992931 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff40e59c640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4100b5d80 0x7ff410158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff40dd9b640 1 -- 172.21.15.115:0/148992931 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4100b5d80 msgr2=0x7ff410158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff40dd9b640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4100b5d80 0x7ff410158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff40dd9b640 1 -- 172.21.15.115:0/148992931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff41015cd20 msgr2=0x7ff4100cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff40dd9b640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff41015cd20 0x7ff4100cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.805+0000 7ff40dd9b640 1 -- 172.21.15.115:0/148992931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408009b30 con 0x7ff4100d1230 2023-12-01T03:10:25.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.806+0000 7ff40dd9b640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 0x7ff410155cd0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800b4a0 tx=0x7ff3f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:25.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.806+0000 7ff406ffd640 1 -- 172.21.15.115:0/148992931 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8010610 con 0x7ff4100d1230 2023-12-01T03:10:25.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.806+0000 7ff406ffd640 1 -- 172.21.15.115:0/148992931 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f8010770 con 0x7ff4100d1230 2023-12-01T03:10:25.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.806+0000 7ff406ffd640 1 -- 172.21.15.115:0/148992931 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8010b20 con 0x7ff4100d1230 2023-12-01T03:10:25.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.806+0000 7ff4148af640 1 -- 172.21.15.115:0/148992931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4100b1400 con 0x7ff4100d1230 2023-12-01T03:10:25.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.806+0000 7ff4148af640 1 -- 172.21.15.115:0/148992931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff410156210 con 0x7ff4100d1230 2023-12-01T03:10:25.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.807+0000 7ff4148af640 1 -- 172.21.15.115:0/148992931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4100b4fd0 con 0x7ff4100d1230 2023-12-01T03:10:25.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.808+0000 7ff406ffd640 1 -- 172.21.15.115:0/148992931 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff3f8010c80 con 0x7ff4100d1230 2023-12-01T03:10:25.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.809+0000 7ff406ffd640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3f006a550 0x7ff3f006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:25.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.809+0000 7ff406ffd640 1 -- 172.21.15.115:0/148992931 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 5753+0+0 (secure 0 0 0) 0x7ff3f8088630 con 0x7ff4100d1230 2023-12-01T03:10:25.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.809+0000 7ff40e59c640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3f006a550 0x7ff3f006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:25.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.810+0000 7ff40e59c640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3f006a550 0x7ff3f006ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff40000a750 tx=0x7ff400008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:25.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:25.813+0000 7ff406ffd640 1 -- 172.21.15.115:0/148992931 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff4100b4fd0 con 0x7ff4100d1230 2023-12-01T03:10:26.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.001+0000 7ff4148af640 1 -- 172.21.15.115:0/148992931 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4100b4fd0 con 0x7ff4100d1230 2023-12-01T03:10:26.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.003+0000 7ff406ffd640 1 -- 172.21.15.115:0/148992931 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) v1 ==== 74+0+16186 (secure 0 0 0) 0x7ff4100b4fd0 con 0x7ff4100d1230 2023-12-01T03:10:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:26.005 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":213,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:25.145152+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:26.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.005+0000 7ff4148af640 1 -- 172.21.15.115:0/148992931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3f006a550 msgr2=0x7ff3f006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.005+0000 7ff4148af640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3f006a550 0x7ff3f006ca10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff40000a750 tx=0x7ff400008040 comp rx=0 tx=0).stop 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.005+0000 7ff4148af640 1 -- 172.21.15.115:0/148992931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 msgr2=0x7ff410155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.005+0000 7ff4148af640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 0x7ff410155cd0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800b4a0 tx=0x7ff3f800b960 comp rx=0 tx=0).stop 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.006+0000 7ff4148af640 1 -- 172.21.15.115:0/148992931 shutdown_connections 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.006+0000 7ff4148af640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3f006a550 0x7ff3f006ca10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.006+0000 7ff4148af640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff41015cd20 0x7ff4100cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.006+0000 7ff4148af640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4100d1230 0x7ff410155cd0 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.006+0000 7ff4148af640 1 --2- 172.21.15.115:0/148992931 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4100b5d80 0x7ff410158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.006+0000 7ff4148af640 1 -- 172.21.15.115:0/148992931 >> 172.21.15.115:0/148992931 conn(0x7ff41006a0e0 msgr2=0x7ff41006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.006+0000 7ff4148af640 1 -- 172.21.15.115:0/148992931 shutdown_connections 2023-12-01T03:10:26.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.006+0000 7ff4148af640 1 -- 172.21.15.115:0/148992931 wait complete. 2023-12-01T03:10:26.027 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T03:10:26.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.240+0000 7f750958b640 1 Processor -- start 2023-12-01T03:10:26.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.241+0000 7f750958b640 1 -- start start 2023-12-01T03:10:26.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f750958b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7504160200 0x7f75040b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:26.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f750958b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75040cbd30 0x7f75040cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:26.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f750958b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75040cb290 con 0x7f75040b0f80 2023-12-01T03:10:26.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f750958b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75040b35f0 con 0x7f7504160200 2023-12-01T03:10:26.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f750958b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75040cc6a0 con 0x7f75040cbd30 2023-12-01T03:10:26.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f7502ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7504160200 0x7f75040b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:26.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f74fbfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f75040b0f80 0x7f750406b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:57046/0 (socket says 172.21.15.115:57046) 2023-12-01T03:10:26.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f74fbfff640 1 -- 172.21.15.115:0/2271329262 learned_addr learned my addr 172.21.15.115:0/2271329262 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:26.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f7502ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7504160200 0x7f75040b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35376/0 (socket says 172.21.15.115:35376) 2023-12-01T03:10:26.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f7502ffd640 1 -- 172.21.15.115:0/2271329262 >> v2:172.21.15.142:3300/0 conn(0x7f75040cbd30 msgr2=0x7f75040cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:26.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f7502ffd640 1 --2- 172.21.15.115:0/2271329262 >> v2:172.21.15.142:3300/0 conn(0x7f75040cbd30 0x7f75040cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:26.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f7502ffd640 1 -- 172.21.15.115:0/2271329262 >> v1:172.21.15.115:6789/0 conn(0x7f75040b0f80 legacy=0x7f750406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:26.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f75027fc640 1 -- 172.21.15.115:0/2271329262 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962431156 0 0) 0x7f75040cb290 con 0x7f75040b0f80 2023-12-01T03:10:26.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.243+0000 7f7502ffd640 1 -- 172.21.15.115:0/2271329262 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040b0ae0 con 0x7f7504160200 2023-12-01T03:10:26.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.244+0000 7f7502ffd640 1 --2- 172.21.15.115:0/2271329262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7504160200 0x7f75040b2c80 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f75040b1b30 tx=0x7f74f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9d8c0ba43874ba server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:26.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.245+0000 7f75027fc640 1 -- 172.21.15.115:0/2271329262 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4031830 con 0x7f7504160200 2023-12-01T03:10:26.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.245+0000 7f75027fc640 1 -- 172.21.15.115:0/2271329262 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f4031990 con 0x7f7504160200 2023-12-01T03:10:26.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.245+0000 7f75027fc640 1 -- 172.21.15.115:0/2271329262 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f4031d20 con 0x7f7504160200 2023-12-01T03:10:26.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.245+0000 7f750958b640 1 -- 172.21.15.115:0/2271329262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7504160200 msgr2=0x7f75040b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.245+0000 7f750958b640 1 --2- 172.21.15.115:0/2271329262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7504160200 0x7f75040b2c80 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f75040b1b30 tx=0x7f74f402b8d0 comp rx=0 tx=0).stop 2023-12-01T03:10:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.245+0000 7f750958b640 1 -- 172.21.15.115:0/2271329262 shutdown_connections 2023-12-01T03:10:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.245+0000 7f750958b640 1 --2- 172.21.15.115:0/2271329262 >> v2:172.21.15.142:3300/0 conn(0x7f75040cbd30 0x7f75040cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.245+0000 7f750958b640 1 --2- 172.21.15.115:0/2271329262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7504160200 0x7f75040b2c80 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.245+0000 7f750958b640 1 -- 172.21.15.115:0/2271329262 >> 172.21.15.115:0/2271329262 conn(0x7f7504065960 msgr2=0x7f7504065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.245+0000 7f750958b640 1 -- 172.21.15.115:0/2271329262 shutdown_connections 2023-12-01T03:10:26.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.246+0000 7f750958b640 1 -- 172.21.15.115:0/2271329262 wait complete. 2023-12-01T03:10:26.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.247+0000 7f750958b640 1 Processor -- start 2023-12-01T03:10:26.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.247+0000 7f750958b640 1 -- start start 2023-12-01T03:10:26.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.248+0000 7f750958b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75040b0f80 0x7f75041481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.248+0000 7f750958b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75040cbd30 0x7f750406f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.248+0000 7f750958b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7504160200 0x7f750406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.248+0000 7f750958b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750414bcc0 con 0x7f75040cbd30 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.248+0000 7f750958b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75041f2980 con 0x7f7504160200 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.248+0000 7f750958b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750414b420 con 0x7f75040b0f80 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.249+0000 7f74fbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75040cbd30 0x7f750406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.249+0000 7f74fbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75040cbd30 0x7f750406f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35382/0 (socket says 172.21.15.115:35382) 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.249+0000 7f74fbfff640 1 -- 172.21.15.115:0/1197653941 learned_addr learned my addr 172.21.15.115:0/1197653941 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.249+0000 7f74fbfff640 1 -- 172.21.15.115:0/1197653941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75040b0f80 msgr2=0x7f75041481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.249+0000 7f74fbfff640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75040b0f80 0x7f75041481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.249+0000 7f74fbfff640 1 -- 172.21.15.115:0/1197653941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7504160200 msgr2=0x7f750406fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.249+0000 7f74fbfff640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7504160200 0x7f750406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:26.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.249+0000 7f74fbfff640 1 -- 172.21.15.115:0/1197653941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f4009400 con 0x7f75040cbd30 2023-12-01T03:10:26.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.249+0000 7f74fbfff640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75040cbd30 0x7f750406f700 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00c4d0 tx=0x7f74ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:26.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.250+0000 7f74fb7fe640 1 -- 172.21.15.115:0/1197653941 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec0074e0 con 0x7f75040cbd30 2023-12-01T03:10:26.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.250+0000 7f74fb7fe640 1 -- 172.21.15.115:0/1197653941 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ec007660 con 0x7f75040cbd30 2023-12-01T03:10:26.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.250+0000 7f74fb7fe640 1 -- 172.21.15.115:0/1197653941 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec004a20 con 0x7f75040cbd30 2023-12-01T03:10:26.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.250+0000 7f750958b640 1 -- 172.21.15.115:0/1197653941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750414b700 con 0x7f75040cbd30 2023-12-01T03:10:26.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.250+0000 7f750958b640 1 -- 172.21.15.115:0/1197653941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75041f2bb0 con 0x7f75040cbd30 2023-12-01T03:10:26.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.251+0000 7f750958b640 1 -- 172.21.15.115:0/1197653941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750404e890 con 0x7f75040cbd30 2023-12-01T03:10:26.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.252+0000 7f74fb7fe640 1 -- 172.21.15.115:0/1197653941 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f74ec004b80 con 0x7f75040cbd30 2023-12-01T03:10:26.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.253+0000 7f74fb7fe640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74e406a550 0x7f74e406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:26.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.253+0000 7f74fb7fe640 1 -- 172.21.15.115:0/1197653941 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f74ec087c10 con 0x7f75040cbd30 2023-12-01T03:10:26.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.253+0000 7f7502ffd640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74e406a550 0x7f74e406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:26.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.254+0000 7f7502ffd640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74e406a550 0x7f74e406ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f74f4036000 tx=0x7f74f4030000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:26.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.257+0000 7f74fb7fe640 1 -- 172.21.15.115:0/1197653941 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f74ec053150 con 0x7f75040cbd30 2023-12-01T03:10:26.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:26.467+0000 7f750958b640 1 -- 172.21.15.115:0/1197653941 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f75040cc1f0 con 0x7f75040cbd30 2023-12-01T03:10:27.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.168+0000 7f74fb7fe640 1 -- 172.21.15.115:0/1197653941 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v94) v1 ==== 118+0+0 (secure 0 0 0) 0x7f74ec018020 con 0x7f75040cbd30 2023-12-01T03:10:27.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.171+0000 7f750958b640 1 -- 172.21.15.115:0/1197653941 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74e406a550 msgr2=0x7f74e406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:27.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.171+0000 7f750958b640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74e406a550 0x7f74e406ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f74f4036000 tx=0x7f74f4030000 comp rx=0 tx=0).stop 2023-12-01T03:10:27.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.171+0000 7f750958b640 1 -- 172.21.15.115:0/1197653941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75040cbd30 msgr2=0x7f750406f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:27.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.171+0000 7f750958b640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75040cbd30 0x7f750406f700 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f74ec00c4d0 tx=0x7f74ec00c990 comp rx=0 tx=0).stop 2023-12-01T03:10:27.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.172+0000 7f750958b640 1 -- 172.21.15.115:0/1197653941 shutdown_connections 2023-12-01T03:10:27.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.172+0000 7f750958b640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74e406a550 0x7f74e406ca10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:27.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.172+0000 7f750958b640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7504160200 0x7f750406fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:27.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.172+0000 7f750958b640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75040cbd30 0x7f750406f700 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:27.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.172+0000 7f750958b640 1 --2- 172.21.15.115:0/1197653941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75040b0f80 0x7f75041481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:27.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.172+0000 7f750958b640 1 -- 172.21.15.115:0/1197653941 >> 172.21.15.115:0/1197653941 conn(0x7f7504065960 msgr2=0x7f7504156460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:27.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.172+0000 7f750958b640 1 -- 172.21.15.115:0/1197653941 shutdown_connections 2023-12-01T03:10:27.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.172+0000 7f750958b640 1 -- 172.21.15.115:0/1197653941 wait complete. 2023-12-01T03:10:27.193 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T03:10:27.193 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T03:10:27.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.406+0000 7faa82484640 1 Processor -- start 2023-12-01T03:10:27.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.406+0000 7faa82484640 1 -- start start 2023-12-01T03:10:27.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.408+0000 7faa82484640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faa7c15b0e0 0x7faa7c164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:27.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.408+0000 7faa82484640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c0b2f30 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.408+0000 7faa82484640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c0b2320 con 0x7faa7c142870 2023-12-01T03:10:27.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.408+0000 7faa82484640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c0b3ae0 con 0x7faa7c15b0e0 2023-12-01T03:10:27.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.408+0000 7faa80a21640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faa7c15b0e0 0x7faa7c164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:27.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.408+0000 7faa80a21640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faa7c15b0e0 0x7faa7c164a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35398/0 (socket says 172.21.15.115:35398) 2023-12-01T03:10:27.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.408+0000 7faa80a21640 1 -- 172.21.15.115:0/995241051 learned_addr learned my addr 172.21.15.115:0/995241051 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:27.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.409+0000 7faa80a21640 1 -- 172.21.15.115:0/995241051 >> v1:172.21.15.142:6789/0 conn(0x7faa7c142870 legacy=0x7faa7c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:27.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.409+0000 7faa80a21640 1 -- 172.21.15.115:0/995241051 >> v1:172.21.15.115:6789/0 conn(0x7faa7c0cd1e0 legacy=0x7faa7c0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:27.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.409+0000 7faa80a21640 1 -- 172.21.15.115:0/995241051 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c0b2600 con 0x7faa7c15b0e0 2023-12-01T03:10:27.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.409+0000 7faa7b7fe640 1 -- 172.21.15.115:0/995241051 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366382965 0 0) 0x7faa7c0b2f30 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.410+0000 7faa80a21640 1 --2- 172.21.15.115:0/995241051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c15b0e0 0x7faa7c164a20 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7faa7400d150 tx=0x7faa7402e200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d832e13187f4db10 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:27.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.410+0000 7faa7b7fe640 1 -- 172.21.15.115:0/995241051 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7400b0d0 con 0x7faa7c15b0e0 2023-12-01T03:10:27.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.411+0000 7faa7b7fe640 1 -- 172.21.15.115:0/995241051 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7400b230 con 0x7faa7c15b0e0 2023-12-01T03:10:27.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.411+0000 7faa7b7fe640 1 -- 172.21.15.115:0/995241051 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74009600 con 0x7faa7c15b0e0 2023-12-01T03:10:27.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.411+0000 7faa82484640 1 -- 172.21.15.115:0/995241051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c15b0e0 msgr2=0x7faa7c164a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:27.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.411+0000 7faa82484640 1 --2- 172.21.15.115:0/995241051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c15b0e0 0x7faa7c164a20 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7faa7400d150 tx=0x7faa7402e200 comp rx=0 tx=0).stop 2023-12-01T03:10:27.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.411+0000 7faa82484640 1 -- 172.21.15.115:0/995241051 shutdown_connections 2023-12-01T03:10:27.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.411+0000 7faa82484640 1 --2- 172.21.15.115:0/995241051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c15b0e0 0x7faa7c164a20 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:27.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.411+0000 7faa82484640 1 -- 172.21.15.115:0/995241051 >> 172.21.15.115:0/995241051 conn(0x7faa7c065960 msgr2=0x7faa7c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:27.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.411+0000 7faa82484640 1 -- 172.21.15.115:0/995241051 shutdown_connections 2023-12-01T03:10:27.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.412+0000 7faa82484640 1 -- 172.21.15.115:0/995241051 wait complete. 2023-12-01T03:10:27.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.413+0000 7faa82484640 1 Processor -- start 2023-12-01T03:10:27.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.413+0000 7faa82484640 1 -- start start 2023-12-01T03:10:27.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.414+0000 7faa82484640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c0cd1e0 0x7faa7c13cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:27.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.414+0000 7faa82484640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa7c142870 0x7faa7c13bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:27.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa82484640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa7c13c4e0 0x7faa7c0cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:27.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa82484640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c13c970 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa82484640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0cb8d0 con 0x7faa7c13c4e0 2023-12-01T03:10:27.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa82484640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0cbbb0 con 0x7faa7c142870 2023-12-01T03:10:27.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa7bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c0cd1e0 0x7faa7c13cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:27.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa7bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c0cd1e0 0x7faa7c13cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:35410/0 (socket says 172.21.15.115:35410) 2023-12-01T03:10:27.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa7bfff640 1 -- 172.21.15.115:0/1007055086 learned_addr learned my addr 172.21.15.115:0/1007055086 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:27.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa73fff640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa7c142870 0x7faa7c13bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:27.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa7bfff640 1 -- 172.21.15.115:0/1007055086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa7c142870 msgr2=0x7faa7c13bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:27.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa7bfff640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa7c142870 0x7faa7c13bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:27.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa7bfff640 1 -- 172.21.15.115:0/1007055086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa7c13c4e0 msgr2=0x7faa7c0cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:27.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa7bfff640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa7c13c4e0 0x7faa7c0cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:27.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.415+0000 7faa7bfff640 1 -- 172.21.15.115:0/1007055086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7400bd40 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.416+0000 7faa7bfff640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c0cd1e0 0x7faa7c13cde0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00c450 tx=0x7faa6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:27.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.417+0000 7faa797fa640 1 -- 172.21.15.115:0/1007055086 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0052d0 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.417+0000 7faa797fa640 1 -- 172.21.15.115:0/1007055086 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c005430 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.417+0000 7faa797fa640 1 -- 172.21.15.115:0/1007055086 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c012870 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.417+0000 7faa82484640 1 -- 172.21.15.115:0/1007055086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c0a8600 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.417+0000 7faa82484640 1 -- 172.21.15.115:0/1007055086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c13d320 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.418+0000 7faa82484640 1 -- 172.21.15.115:0/1007055086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c0a8b20 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.419+0000 7faa797fa640 1 -- 172.21.15.115:0/1007055086 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7faa6c0129d0 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.420+0000 7faa797fa640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa5c06a550 0x7faa5c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.420+0000 7faa797fa640 1 -- 172.21.15.115:0/1007055086 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7faa6c0886d0 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.420+0000 7faa73fff640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa5c06a550 0x7faa5c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:27.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.421+0000 7faa73fff640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa5c06a550 0x7faa5c06ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0b0500 tx=0x7faa64009040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:27.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.425+0000 7faa797fa640 1 -- 172.21.15.115:0/1007055086 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faa7c0a8b20 con 0x7faa7c0cd1e0 2023-12-01T03:10:27.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:27.653+0000 7faa82484640 1 -- 172.21.15.115:0/1007055086 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faa7c0a8b20 con 0x7faa7c0cd1e0 2023-12-01T03:10:28.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.423+0000 7faa797fa640 1 -- 172.21.15.115:0/1007055086 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v95) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa7c0a8b20 con 0x7faa7c0cd1e0 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.426+0000 7faa82484640 1 -- 172.21.15.115:0/1007055086 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa5c06a550 msgr2=0x7faa5c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.426+0000 7faa82484640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa5c06a550 0x7faa5c06ca10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0b0500 tx=0x7faa64009040 comp rx=0 tx=0).stop 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.426+0000 7faa82484640 1 -- 172.21.15.115:0/1007055086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c0cd1e0 msgr2=0x7faa7c13cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.426+0000 7faa82484640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c0cd1e0 0x7faa7c13cde0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00c450 tx=0x7faa6c00c910 comp rx=0 tx=0).stop 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.427+0000 7faa82484640 1 -- 172.21.15.115:0/1007055086 shutdown_connections 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.427+0000 7faa82484640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa5c06a550 0x7faa5c06ca10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.427+0000 7faa82484640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa7c142870 0x7faa7c13bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.427+0000 7faa82484640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa7c13c4e0 0x7faa7c0cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.427+0000 7faa82484640 1 --2- 172.21.15.115:0/1007055086 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa7c0cd1e0 0x7faa7c13cde0 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.427+0000 7faa82484640 1 -- 172.21.15.115:0/1007055086 >> 172.21.15.115:0/1007055086 conn(0x7faa7c065960 msgr2=0x7faa7c1419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.427+0000 7faa82484640 1 -- 172.21.15.115:0/1007055086 shutdown_connections 2023-12-01T03:10:28.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.427+0000 7faa82484640 1 -- 172.21.15.115:0/1007055086 wait complete. 2023-12-01T03:10:28.448 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T03:10:28.449 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:10:28.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.653+0000 7f17d91c9640 1 Processor -- start 2023-12-01T03:10:28.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.653+0000 7f17d91c9640 1 -- start start 2023-12-01T03:10:28.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d91c9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f17d415cd20 0x7f17d40d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:28.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d91c9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f17d40b5d80 0x7f17d40b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:28.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d91c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d40d1230 0x7f17d40d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:28.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d91c9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d40b2160 con 0x7f17d40b5d80 2023-12-01T03:10:28.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d91c9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d40d2a10 con 0x7f17d415cd20 2023-12-01T03:10:28.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d91c9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40d1b90 con 0x7f17d40d1230 2023-12-01T03:10:28.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f17d40b5d80 0x7f17d40b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:28.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f17d40b5d80 0x7f17d40b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33658/0 (socket says 172.21.15.115:33658) 2023-12-01T03:10:28.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d259c640 1 -- 172.21.15.115:0/3811006789 learned_addr learned my addr 172.21.15.115:0/3811006789 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:28.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d2d9d640 1 --2- 172.21.15.115:0/3811006789 >> v2:172.21.15.142:3300/0 conn(0x7f17d415cd20 0x7f17d40d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:28.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d359e640 1 --2- 172.21.15.115:0/3811006789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d40d1230 0x7f17d40d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:28.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d259c640 1 -- 172.21.15.115:0/3811006789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d40d1230 msgr2=0x7f17d40d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:28.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d259c640 1 --2- 172.21.15.115:0/3811006789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d40d1230 0x7f17d40d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d259c640 1 -- 172.21.15.115:0/3811006789 >> v2:172.21.15.142:3300/0 conn(0x7f17d415cd20 msgr2=0x7f17d40d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:28.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d259c640 1 --2- 172.21.15.115:0/3811006789 >> v2:172.21.15.142:3300/0 conn(0x7f17d415cd20 0x7f17d40d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.655+0000 7f17d259c640 1 -- 172.21.15.115:0/3811006789 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d40ce2e0 con 0x7f17d40b5d80 2023-12-01T03:10:28.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.657+0000 7f17d259c640 1 --2- 172.21.15.115:0/3811006789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40b5d80 0x7f17d40b8570 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f17d408bdb0 tx=0x7f17bc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7876174b40102222 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:28.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.657+0000 7f17d1d9b640 1 -- 172.21.15.115:0/3811006789 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0317c0 con 0x7f17d40b5d80 2023-12-01T03:10:28.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.657+0000 7f17d1d9b640 1 -- 172.21.15.115:0/3811006789 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc031920 con 0x7f17d40b5d80 2023-12-01T03:10:28.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.657+0000 7f17d1d9b640 1 -- 172.21.15.115:0/3811006789 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc031cb0 con 0x7f17d40b5d80 2023-12-01T03:10:28.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.658+0000 7f17d91c9640 1 -- 172.21.15.115:0/3811006789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40b5d80 msgr2=0x7f17d40b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:28.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.658+0000 7f17d91c9640 1 --2- 172.21.15.115:0/3811006789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40b5d80 0x7f17d40b8570 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f17d408bdb0 tx=0x7f17bc02b860 comp rx=0 tx=0).stop 2023-12-01T03:10:28.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.658+0000 7f17d91c9640 1 -- 172.21.15.115:0/3811006789 shutdown_connections 2023-12-01T03:10:28.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.658+0000 7f17d91c9640 1 --2- 172.21.15.115:0/3811006789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d40d1230 0x7f17d40d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.658+0000 7f17d91c9640 1 --2- 172.21.15.115:0/3811006789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40b5d80 0x7f17d40b8570 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.658+0000 7f17d91c9640 1 --2- 172.21.15.115:0/3811006789 >> v2:172.21.15.142:3300/0 conn(0x7f17d415cd20 0x7f17d40d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.658+0000 7f17d91c9640 1 -- 172.21.15.115:0/3811006789 >> 172.21.15.115:0/3811006789 conn(0x7f17d406a0e0 msgr2=0x7f17d406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:28.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.658+0000 7f17d91c9640 1 -- 172.21.15.115:0/3811006789 shutdown_connections 2023-12-01T03:10:28.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.658+0000 7f17d91c9640 1 -- 172.21.15.115:0/3811006789 wait complete. 2023-12-01T03:10:28.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.660+0000 7f17d91c9640 1 Processor -- start 2023-12-01T03:10:28.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.660+0000 7f17d91c9640 1 -- start start 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.661+0000 7f17d91c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d40b5d80 0x7f17d414d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.661+0000 7f17d91c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40d1230 0x7f17d40b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.661+0000 7f17d91c9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17d415cd20 0x7f17d40b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.661+0000 7f17d91c9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40c6870 con 0x7f17d40d1230 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.661+0000 7f17d91c9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40b5310 con 0x7f17d415cd20 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.661+0000 7f17d91c9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40b4a70 con 0x7f17d40b5d80 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.661+0000 7f17d259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40d1230 0x7f17d40b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.661+0000 7f17d259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40d1230 0x7f17d40b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33660/0 (socket says 172.21.15.115:33660) 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.662+0000 7f17d259c640 1 -- 172.21.15.115:0/3436229083 learned_addr learned my addr 172.21.15.115:0/3436229083 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.662+0000 7f17d2d9d640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d40b5d80 0x7f17d414d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.662+0000 7f17d259c640 1 -- 172.21.15.115:0/3436229083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d40b5d80 msgr2=0x7f17d414d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.662+0000 7f17d259c640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d40b5d80 0x7f17d414d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.662+0000 7f17d259c640 1 -- 172.21.15.115:0/3436229083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17d415cd20 msgr2=0x7f17d40b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:28.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.662+0000 7f17d359e640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17d415cd20 0x7f17d40b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:28.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.662+0000 7f17d259c640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17d415cd20 0x7f17d40b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.662+0000 7f17d259c640 1 -- 172.21.15.115:0/3436229083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc0094d0 con 0x7f17d40d1230 2023-12-01T03:10:28.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.662+0000 7f17d259c640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40d1230 0x7f17d40b3f00 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f17bc02b8a0 tx=0x7f17bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:28.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.662+0000 7f17cb7fe640 1 -- 172.21.15.115:0/3436229083 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc032860 con 0x7f17d40d1230 2023-12-01T03:10:28.665 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.663+0000 7f17cb7fe640 1 -- 172.21.15.115:0/3436229083 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc0329e0 con 0x7f17d40d1230 2023-12-01T03:10:28.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.663+0000 7f17cb7fe640 1 -- 172.21.15.115:0/3436229083 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc032d90 con 0x7f17d40d1230 2023-12-01T03:10:28.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.663+0000 7f17d91c9640 1 -- 172.21.15.115:0/3436229083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d40b4d50 con 0x7f17d40d1230 2023-12-01T03:10:28.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.663+0000 7f17d91c9640 1 -- 172.21.15.115:0/3436229083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d40cc630 con 0x7f17d40d1230 2023-12-01T03:10:28.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.664+0000 7f17d91c9640 1 -- 172.21.15.115:0/3436229083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d404e890 con 0x7f17d40d1230 2023-12-01T03:10:28.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.665+0000 7f17cb7fe640 1 -- 172.21.15.115:0/3436229083 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f17bc04e450 con 0x7f17d40d1230 2023-12-01T03:10:28.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.666+0000 7f17cb7fe640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f17b406a550 0x7f17b406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:28.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.666+0000 7f17cb7fe640 1 -- 172.21.15.115:0/3436229083 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f17bc0aacb0 con 0x7f17d40d1230 2023-12-01T03:10:28.667 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.666+0000 7f17d2d9d640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f17b406a550 0x7f17b406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:28.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.666+0000 7f17d2d9d640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f17b406a550 0x7f17b406ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009eb0 tx=0x7f17c400b040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:28.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.670+0000 7f17cb7fe640 1 -- 172.21.15.115:0/3436229083 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f17bc076270 con 0x7f17d40d1230 2023-12-01T03:10:28.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.897+0000 7f17d91c9640 1 -- 172.21.15.115:0/3436229083 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17d41e7010 con 0x7f17d40d1230 2023-12-01T03:10:28.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.898+0000 7f17cb7fe640 1 -- 172.21.15.115:0/3436229083 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4664 (secure 0 0 0) 0x7f17bc079890 con 0x7f17d40d1230 2023-12-01T03:10:28.900 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:28.900 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:25.146035+0000","modified":"2023-12-01T03:10:28.416554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10653":{"gid":10653,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/720518620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":720518620},{"type":"v1","addr":"172.21.15.142:6834","nonce":720518620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-12-01T03:10:28.900 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 95 2023-12-01T03:10:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.900+0000 7f17d91c9640 1 -- 172.21.15.115:0/3436229083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f17b406a550 msgr2=0x7f17b406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.900+0000 7f17d91c9640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f17b406a550 0x7f17b406ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009eb0 tx=0x7f17c400b040 comp rx=0 tx=0).stop 2023-12-01T03:10:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.900+0000 7f17d91c9640 1 -- 172.21.15.115:0/3436229083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40d1230 msgr2=0x7f17d40b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.900+0000 7f17d91c9640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40d1230 0x7f17d40b3f00 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f17bc02b8a0 tx=0x7f17bc0046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.900+0000 7f17d91c9640 1 -- 172.21.15.115:0/3436229083 shutdown_connections 2023-12-01T03:10:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.900+0000 7f17d91c9640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f17b406a550 0x7f17b406ca10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.900+0000 7f17d91c9640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17d415cd20 0x7f17d40b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.900+0000 7f17d91c9640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17d40d1230 0x7f17d40b3f00 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.900+0000 7f17d91c9640 1 --2- 172.21.15.115:0/3436229083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17d40b5d80 0x7f17d414d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:28.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.900+0000 7f17d91c9640 1 -- 172.21.15.115:0/3436229083 >> 172.21.15.115:0/3436229083 conn(0x7f17d406a0e0 msgr2=0x7f17d406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:28.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.901+0000 7f17d91c9640 1 -- 172.21.15.115:0/3436229083 shutdown_connections 2023-12-01T03:10:28.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:28.901+0000 7f17d91c9640 1 -- 172.21.15.115:0/3436229083 wait complete. 2023-12-01T03:10:28.922 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:29.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.124+0000 7fdf04bb5640 1 Processor -- start 2023-12-01T03:10:29.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.125+0000 7fdf04bb5640 1 -- start start 2023-12-01T03:10:29.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.126+0000 7fdf04bb5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdf0015cf90 0x7fdf000d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.126+0000 7fdf04bb5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdf000b8610 0x7fdf000cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.126+0000 7fdf04bb5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf000ce330 con 0x7fdf000cdf00 2023-12-01T03:10:29.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.126+0000 7fdf04bb5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0015d370 con 0x7fdf000b8610 2023-12-01T03:10:29.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.126+0000 7fdf04bb5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf000d19a0 con 0x7fdf0015cf90 2023-12-01T03:10:29.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.126+0000 7fdef7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdf000b8610 0x7fdf000cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.126+0000 7fdef7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fdf000b8610 0x7fdf000cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33674/0 (socket says 172.21.15.115:33674) 2023-12-01T03:10:29.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.127+0000 7fdef7fff640 1 -- 172.21.15.115:0/2941143184 learned_addr learned my addr 172.21.15.115:0/2941143184 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:29.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.127+0000 7fdef7fff640 1 -- 172.21.15.115:0/2941143184 >> v2:172.21.15.142:3300/0 conn(0x7fdf0015cf90 msgr2=0x7fdf000d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:29.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.127+0000 7fdefe59c640 1 --2- 172.21.15.115:0/2941143184 >> v2:172.21.15.142:3300/0 conn(0x7fdf0015cf90 0x7fdf000d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.127+0000 7fdef7fff640 1 --2- 172.21.15.115:0/2941143184 >> v2:172.21.15.142:3300/0 conn(0x7fdf0015cf90 0x7fdf000d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.127+0000 7fdef7fff640 1 -- 172.21.15.115:0/2941143184 >> v1:172.21.15.142:6789/0 conn(0x7fdf000cdf00 legacy=0x7fdf000b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:29.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.127+0000 7fdef7fff640 1 -- 172.21.15.115:0/2941143184 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf000d1c80 con 0x7fdf000b8610 2023-12-01T03:10:29.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.127+0000 7fdefe59c640 1 --2- 172.21.15.115:0/2941143184 >> v2:172.21.15.142:3300/0 conn(0x7fdf0015cf90 0x7fdf000d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:29.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.128+0000 7fdef7fff640 1 --2- 172.21.15.115:0/2941143184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000b8610 0x7fdf000cd9c0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fdf0008b560 tx=0x7fdee802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780b1b21e21a0135 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:29.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.128+0000 7fdefdd9b640 1 -- 172.21.15.115:0/2941143184 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8031640 con 0x7fdf000b8610 2023-12-01T03:10:29.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.129+0000 7fdefdd9b640 1 -- 172.21.15.115:0/2941143184 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee80317a0 con 0x7fdf000b8610 2023-12-01T03:10:29.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.129+0000 7fdefdd9b640 1 -- 172.21.15.115:0/2941143184 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8031b30 con 0x7fdf000b8610 2023-12-01T03:10:29.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.129+0000 7fdf04bb5640 1 -- 172.21.15.115:0/2941143184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000b8610 msgr2=0x7fdf000cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.129+0000 7fdf04bb5640 1 --2- 172.21.15.115:0/2941143184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000b8610 0x7fdf000cd9c0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fdf0008b560 tx=0x7fdee802b8f0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.129+0000 7fdf04bb5640 1 -- 172.21.15.115:0/2941143184 shutdown_connections 2023-12-01T03:10:29.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.129+0000 7fdf04bb5640 1 --2- 172.21.15.115:0/2941143184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000b8610 0x7fdf000cd9c0 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.129+0000 7fdf04bb5640 1 --2- 172.21.15.115:0/2941143184 >> v2:172.21.15.142:3300/0 conn(0x7fdf0015cf90 0x7fdf000d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.129+0000 7fdf04bb5640 1 -- 172.21.15.115:0/2941143184 >> 172.21.15.115:0/2941143184 conn(0x7fdf00065960 msgr2=0x7fdf00065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:29.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.129+0000 7fdf04bb5640 1 -- 172.21.15.115:0/2941143184 shutdown_connections 2023-12-01T03:10:29.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.129+0000 7fdf04bb5640 1 -- 172.21.15.115:0/2941143184 wait complete. 2023-12-01T03:10:29.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.131+0000 7fdf04bb5640 1 Processor -- start 2023-12-01T03:10:29.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.131+0000 7fdf04bb5640 1 -- start start 2023-12-01T03:10:29.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.132+0000 7fdf04bb5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf000b8610 0x7fdf0015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.132+0000 7fdf04bb5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000cdf00 0x7fdf000cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.132+0000 7fdf04bb5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0015cf90 0x7fdf00149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.132+0000 7fdf04bb5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0014a820 con 0x7fdf000cdf00 2023-12-01T03:10:29.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.132+0000 7fdf04bb5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf00159c10 con 0x7fdf000b8610 2023-12-01T03:10:29.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.132+0000 7fdf04bb5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf000cdb80 con 0x7fdf0015cf90 2023-12-01T03:10:29.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdef7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000cdf00 0x7fdf000cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdef7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000cdf00 0x7fdf000cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33676/0 (socket says 172.21.15.115:33676) 2023-12-01T03:10:29.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdef7fff640 1 -- 172.21.15.115:0/242390115 learned_addr learned my addr 172.21.15.115:0/242390115 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdef7fff640 1 -- 172.21.15.115:0/242390115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0015cf90 msgr2=0x7fdf00149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdef7fff640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0015cf90 0x7fdf00149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdefe59c640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf000b8610 0x7fdf0015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdef7fff640 1 -- 172.21.15.115:0/242390115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf000b8610 msgr2=0x7fdf0015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdef7fff640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf000b8610 0x7fdf0015dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdef7fff640 1 -- 172.21.15.115:0/242390115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80094d0 con 0x7fdf000cdf00 2023-12-01T03:10:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdefe59c640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf000b8610 0x7fdf0015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.133+0000 7fdef7fff640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000cdf00 0x7fdf000cd640 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fdee802b930 tx=0x7fdee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:29.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.134+0000 7fdef6ffd640 1 -- 172.21.15.115:0/242390115 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee803ac60 con 0x7fdf000cdf00 2023-12-01T03:10:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.134+0000 7fdef6ffd640 1 -- 172.21.15.115:0/242390115 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee803adc0 con 0x7fdf000cdf00 2023-12-01T03:10:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.134+0000 7fdef6ffd640 1 -- 172.21.15.115:0/242390115 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8044630 con 0x7fdf000cdf00 2023-12-01T03:10:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.134+0000 7fdf04bb5640 1 -- 172.21.15.115:0/242390115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0015e4b0 con 0x7fdf000cdf00 2023-12-01T03:10:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.134+0000 7fdf04bb5640 1 -- 172.21.15.115:0/242390115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0015ea70 con 0x7fdf000cdf00 2023-12-01T03:10:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.135+0000 7fdf04bb5640 1 -- 172.21.15.115:0/242390115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0004e890 con 0x7fdf000cdf00 2023-12-01T03:10:29.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.136+0000 7fdef6ffd640 1 -- 172.21.15.115:0/242390115 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fdee802f050 con 0x7fdf000cdf00 2023-12-01T03:10:29.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.137+0000 7fdef6ffd640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdee006a550 0x7fdee006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.137+0000 7fdef6ffd640 1 -- 172.21.15.115:0/242390115 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fdee80aad30 con 0x7fdf000cdf00 2023-12-01T03:10:29.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.137+0000 7fdefe59c640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdee006a550 0x7fdee006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.138+0000 7fdefe59c640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdee006a550 0x7fdee006ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdef0006fd0 tx=0x7fdef0008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:29.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.141+0000 7fdef6ffd640 1 -- 172.21.15.115:0/242390115 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdee8076160 con 0x7fdf000cdf00 2023-12-01T03:10:29.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.333+0000 7fdf04bb5640 1 -- 172.21.15.115:0/242390115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf000b1920 con 0x7fdf000cdf00 2023-12-01T03:10:29.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.334+0000 7fdef6ffd640 1 -- 172.21.15.115:0/242390115 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v214) v1 ==== 74+0+16173 (secure 0 0 0) 0x7fdee8035090 con 0x7fdf000cdf00 2023-12-01T03:10:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:29.337 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":214,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:26.151466+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:29.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 -- 172.21.15.115:0/242390115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdee006a550 msgr2=0x7fdee006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdee006a550 0x7fdee006ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdef0006fd0 tx=0x7fdef0008040 comp rx=0 tx=0).stop 2023-12-01T03:10:29.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 -- 172.21.15.115:0/242390115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000cdf00 msgr2=0x7fdf000cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000cdf00 0x7fdf000cd640 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fdee802b930 tx=0x7fdee80046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 -- 172.21.15.115:0/242390115 shutdown_connections 2023-12-01T03:10:29.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdee006a550 0x7fdee006ca10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0015cf90 0x7fdf00149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf000cdf00 0x7fdf000cd640 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 --2- 172.21.15.115:0/242390115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf000b8610 0x7fdf0015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 -- 172.21.15.115:0/242390115 >> 172.21.15.115:0/242390115 conn(0x7fdf00065960 msgr2=0x7fdf000c6840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:29.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 -- 172.21.15.115:0/242390115 shutdown_connections 2023-12-01T03:10:29.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.337+0000 7fdf04bb5640 1 -- 172.21.15.115:0/242390115 wait complete. 2023-12-01T03:10:29.358 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T03:10:29.359 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:10:29.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.572+0000 7ff1a3918640 1 Processor -- start 2023-12-01T03:10:29.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.572+0000 7ff1a3918640 1 -- start start 2023-12-01T03:10:29.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.573+0000 7ff1a3918640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff19c15e2f0 0x7ff19c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.573+0000 7ff1a3918640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff19c06f2f0 0x7ff19c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a3918640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c0add10 con 0x7ff19c0adf10 2023-12-01T03:10:29.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a3918640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c0c5760 con 0x7ff19c15e2f0 2023-12-01T03:10:29.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a3918640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c15e6d0 con 0x7ff19c06f2f0 2023-12-01T03:10:29.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a16b4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff19c15e2f0 0x7ff19c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a16b4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff19c15e2f0 0x7ff19c0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33678/0 (socket says 172.21.15.115:33678) 2023-12-01T03:10:29.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a16b4640 1 -- 172.21.15.115:0/2123810212 learned_addr learned my addr 172.21.15.115:0/2123810212 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:29.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a1eb5640 1 --1- 172.21.15.115:0/2123810212 >> v1:172.21.15.115:6789/0 conn(0x7ff19c0adf10 0x7ff19c0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37208/0 (socket says 172.21.15.115:37208) 2023-12-01T03:10:29.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a0eb3640 1 --2- 172.21.15.115:0/2123810212 >> v2:172.21.15.142:3300/0 conn(0x7ff19c06f2f0 0x7ff19c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a16b4640 1 -- 172.21.15.115:0/2123810212 >> v2:172.21.15.142:3300/0 conn(0x7ff19c06f2f0 msgr2=0x7ff19c0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:29.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a16b4640 1 --2- 172.21.15.115:0/2123810212 >> v2:172.21.15.142:3300/0 conn(0x7ff19c06f2f0 0x7ff19c0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a16b4640 1 -- 172.21.15.115:0/2123810212 >> v1:172.21.15.115:6789/0 conn(0x7ff19c0adf10 legacy=0x7ff19c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a16b4640 1 -- 172.21.15.115:0/2123810212 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0cf7b0 con 0x7ff19c15e2f0 2023-12-01T03:10:29.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff193fff640 1 -- 172.21.15.115:0/2123810212 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899444754 0 0) 0x7ff19c0add10 con 0x7ff19c0adf10 2023-12-01T03:10:29.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.574+0000 7ff1a0eb3640 1 --2- 172.21.15.115:0/2123810212 >> v2:172.21.15.142:3300/0 conn(0x7ff19c06f2f0 0x7ff19c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:29.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.575+0000 7ff1a16b4640 1 --2- 172.21.15.115:0/2123810212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c15e2f0 0x7ff19c0ca660 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7ff19c06de80 tx=0x7ff19802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978643888160c5ed server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:29.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.576+0000 7ff193fff640 1 -- 172.21.15.115:0/2123810212 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980317e0 con 0x7ff19c15e2f0 2023-12-01T03:10:29.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.576+0000 7ff193fff640 1 -- 172.21.15.115:0/2123810212 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff198031940 con 0x7ff19c15e2f0 2023-12-01T03:10:29.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.576+0000 7ff193fff640 1 -- 172.21.15.115:0/2123810212 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198031cd0 con 0x7ff19c15e2f0 2023-12-01T03:10:29.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.576+0000 7ff1a3918640 1 -- 172.21.15.115:0/2123810212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c15e2f0 msgr2=0x7ff19c0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.576+0000 7ff1a3918640 1 --2- 172.21.15.115:0/2123810212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c15e2f0 0x7ff19c0ca660 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7ff19c06de80 tx=0x7ff19802b880 comp rx=0 tx=0).stop 2023-12-01T03:10:29.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.576+0000 7ff1a3918640 1 -- 172.21.15.115:0/2123810212 shutdown_connections 2023-12-01T03:10:29.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.576+0000 7ff1a3918640 1 --2- 172.21.15.115:0/2123810212 >> v2:172.21.15.142:3300/0 conn(0x7ff19c06f2f0 0x7ff19c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.576+0000 7ff1a3918640 1 --2- 172.21.15.115:0/2123810212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c15e2f0 0x7ff19c0ca660 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.576+0000 7ff1a3918640 1 -- 172.21.15.115:0/2123810212 >> 172.21.15.115:0/2123810212 conn(0x7ff19c0ce880 msgr2=0x7ff19c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:29.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.576+0000 7ff1a3918640 1 -- 172.21.15.115:0/2123810212 shutdown_connections 2023-12-01T03:10:29.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.577+0000 7ff1a3918640 1 -- 172.21.15.115:0/2123810212 wait complete. 2023-12-01T03:10:29.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.578+0000 7ff1a3918640 1 Processor -- start 2023-12-01T03:10:29.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.578+0000 7ff1a3918640 1 -- start start 2023-12-01T03:10:29.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.579+0000 7ff1a3918640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff19c06f2f0 0x7ff19c0c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.579+0000 7ff1a3918640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c0adf10 0x7ff19c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.579+0000 7ff1a3918640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff19c15e2f0 0x7ff19c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.579+0000 7ff1a3918640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0c3680 con 0x7ff19c0adf10 2023-12-01T03:10:29.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.579+0000 7ff1a3918640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0c3960 con 0x7ff19c15e2f0 2023-12-01T03:10:29.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.579+0000 7ff1a3918640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0c3c40 con 0x7ff19c06f2f0 2023-12-01T03:10:29.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.580+0000 7ff1a0eb3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c0adf10 0x7ff19c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.580+0000 7ff1a0eb3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c0adf10 0x7ff19c15d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33680/0 (socket says 172.21.15.115:33680) 2023-12-01T03:10:29.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.580+0000 7ff1a0eb3640 1 -- 172.21.15.115:0/2164988174 learned_addr learned my addr 172.21.15.115:0/2164988174 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:29.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.580+0000 7ff1a1eb5640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff19c15e2f0 0x7ff19c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.580+0000 7ff1a16b4640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff19c06f2f0 0x7ff19c0c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.580+0000 7ff1a0eb3640 1 -- 172.21.15.115:0/2164988174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff19c06f2f0 msgr2=0x7ff19c0c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.580+0000 7ff1a0eb3640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff19c06f2f0 0x7ff19c0c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.580+0000 7ff1a0eb3640 1 -- 172.21.15.115:0/2164988174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff19c15e2f0 msgr2=0x7ff19c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.580+0000 7ff1a0eb3640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff19c15e2f0 0x7ff19c0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.580+0000 7ff1a0eb3640 1 -- 172.21.15.115:0/2164988174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980094d0 con 0x7ff19c0adf10 2023-12-01T03:10:29.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.581+0000 7ff1a0eb3640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c0adf10 0x7ff19c15d230 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c7f0 tx=0x7ff18c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:29.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.581+0000 7ff191ffb640 1 -- 172.21.15.115:0/2164988174 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c007820 con 0x7ff19c0adf10 2023-12-01T03:10:29.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.581+0000 7ff191ffb640 1 -- 172.21.15.115:0/2164988174 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c007980 con 0x7ff19c0adf10 2023-12-01T03:10:29.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.581+0000 7ff191ffb640 1 -- 172.21.15.115:0/2164988174 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c007cf0 con 0x7ff19c0adf10 2023-12-01T03:10:29.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.581+0000 7ff1a3918640 1 -- 172.21.15.115:0/2164988174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c0c3f20 con 0x7ff19c0adf10 2023-12-01T03:10:29.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.581+0000 7ff1a3918640 1 -- 172.21.15.115:0/2164988174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c15d770 con 0x7ff19c0adf10 2023-12-01T03:10:29.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.582+0000 7ff1a3918640 1 -- 172.21.15.115:0/2164988174 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c06fda0 con 0x7ff19c0adf10 2023-12-01T03:10:29.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.583+0000 7ff191ffb640 1 -- 172.21.15.115:0/2164988174 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff18c0113f0 con 0x7ff19c0adf10 2023-12-01T03:10:29.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.584+0000 7ff191ffb640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff18006a550 0x7ff18006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:29.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.584+0000 7ff191ffb640 1 -- 172.21.15.115:0/2164988174 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7ff18c088e70 con 0x7ff19c0adf10 2023-12-01T03:10:29.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.584+0000 7ff1a16b4640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff18006a550 0x7ff18006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:29.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.585+0000 7ff1a16b4640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff18006a550 0x7ff18006ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff198036000 tx=0x7ff198008030 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:29.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.589+0000 7ff191ffb640 1 -- 172.21.15.115:0/2164988174 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff18c0542a0 con 0x7ff19c0adf10 2023-12-01T03:10:29.610 INFO:tasks.ceph.mds.d.smithi142.stderr:2023-12-01T03:10:29.608+0000 7f0be1784640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:10:29.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.773+0000 7ff1a3918640 1 -- 172.21.15.115:0/2164988174 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff19c0c3ad0 con 0x7ff18006a550 2023-12-01T03:10:29.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.784+0000 7ff191ffb640 1 -- 172.21.15.115:0/2164988174 <== mgr.10407 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276036 (secure 0 0 0) 0x7ff19c0c3ad0 con 0x7ff18006a550 2023-12-01T03:10:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:29.791 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:10:29.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 -- 172.21.15.115:0/2164988174 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff18006a550 msgr2=0x7ff18006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff18006a550 0x7ff18006ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff198036000 tx=0x7ff198008030 comp rx=0 tx=0).stop 2023-12-01T03:10:29.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 -- 172.21.15.115:0/2164988174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c0adf10 msgr2=0x7ff19c15d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:29.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c0adf10 0x7ff19c15d230 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c7f0 tx=0x7ff18c00ccb0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 -- 172.21.15.115:0/2164988174 shutdown_connections 2023-12-01T03:10:29.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff18006a550 0x7ff18006ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff19c15e2f0 0x7ff19c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff19c0adf10 0x7ff19c15d230 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 --2- 172.21.15.115:0/2164988174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff19c06f2f0 0x7ff19c0c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:29.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 -- 172.21.15.115:0/2164988174 >> 172.21.15.115:0/2164988174 conn(0x7ff19c0ce880 msgr2=0x7ff19c06b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:29.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 -- 172.21.15.115:0/2164988174 shutdown_connections 2023-12-01T03:10:29.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:29.790+0000 7ff1a3918640 1 -- 172.21.15.115:0/2164988174 wait complete. 2023-12-01T03:10:29.811 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-01T03:10:27.795296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":412844,"kb_used_data":21740,"kb_used_omap":0,"kb_used_meta":391040,"kb_avail":754561876,"statfs":{"total":773094113280,"available":772671361024,"internally_reserved":0,"allocated":22261760,"data_stored":11436535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":400424960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.156404"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.658898+0000","last_change":"2023-12-01T03:10:23.658898+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.658898+0000","last_undegraded":"2023-12-01T03:10:23.658898+0000","last_fullsized":"2023-12-01T03:10:23.658898+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:07.101180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.651481+0000","last_change":"2023-12-01T03:10:23.651481+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.651481+0000","last_undegraded":"2023-12-01T03:10:23.651481+0000","last_fullsized":"2023-12-01T03:10:23.651481+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.657525+0000","last_change":"2023-12-01T03:10:23.657525+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.657525+0000","last_undegraded":"2023-12-01T03:10:23.657525+0000","last_fullsized":"2023-12-01T03:10:23.657525+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.651487+0000","last_change":"2023-12-01T03:10:23.651487+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.651487+0000","last_undegraded":"2023-12-01T03:10:23.651487+0000","last_fullsized":"2023-12-01T03:10:23.651487+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.659800+0000","last_change":"2023-12-01T03:10:23.659800+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.659800+0000","last_undegraded":"2023-12-01T03:10:23.659800+0000","last_fullsized":"2023-12-01T03:10:23.659800+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.653187+0000","last_change":"2023-12-01T03:10:23.653187+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.653187+0000","last_undegraded":"2023-12-01T03:10:23.653187+0000","last_fullsized":"2023-12-01T03:10:23.653187+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:19.256208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.649487+0000","last_change":"2023-12-01T03:10:23.649487+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.649487+0000","last_undegraded":"2023-12-01T03:10:23.649487+0000","last_fullsized":"2023-12-01T03:10:23.649487+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:18.663097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.653351+0000","last_change":"2023-12-01T03:10:23.653351+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.653351+0000","last_undegraded":"2023-12-01T03:10:23.653351+0000","last_fullsized":"2023-12-01T03:10:23.653351+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:39.126749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.651863+0000","last_change":"2023-12-01T03:10:23.651863+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.651863+0000","last_undegraded":"2023-12-01T03:10:23.651863+0000","last_fullsized":"2023-12-01T03:10:23.651863+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.656300+0000","last_change":"2023-12-01T03:10:23.656300+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.656300+0000","last_undegraded":"2023-12-01T03:10:23.656300+0000","last_fullsized":"2023-12-01T03:10:23.656300+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:07.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.654346+0000","last_change":"2023-12-01T03:10:23.654346+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.654346+0000","last_undegraded":"2023-12-01T03:10:23.654346+0000","last_fullsized":"2023-12-01T03:10:23.654346+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.650921+0000","last_change":"2023-12-01T03:10:23.650921+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.650921+0000","last_undegraded":"2023-12-01T03:10:23.650921+0000","last_fullsized":"2023-12-01T03:10:23.650921+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.654255+0000","last_change":"2023-12-01T03:10:23.654255+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.654255+0000","last_undegraded":"2023-12-01T03:10:23.654255+0000","last_fullsized":"2023-12-01T03:10:23.654255+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.652614+0000","last_change":"2023-12-01T03:10:23.652614+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.652614+0000","last_undegraded":"2023-12-01T03:10:23.652614+0000","last_fullsized":"2023-12-01T03:10:23.652614+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:38:02.332793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.648143+0000","last_change":"2023-12-01T03:10:23.648143+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.648143+0000","last_undegraded":"2023-12-01T03:10:23.648143+0000","last_fullsized":"2023-12-01T03:10:23.648143+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:29.129070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.655714+0000","last_change":"2023-12-01T03:10:23.655714+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.655714+0000","last_undegraded":"2023-12-01T03:10:23.655714+0000","last_fullsized":"2023-12-01T03:10:23.655714+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.646593+0000","last_change":"2023-12-01T03:10:23.646593+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.646593+0000","last_undegraded":"2023-12-01T03:10:23.646593+0000","last_fullsized":"2023-12-01T03:10:23.646593+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:44.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.650652+0000","last_change":"2023-12-01T03:10:23.650652+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.650652+0000","last_undegraded":"2023-12-01T03:10:23.650652+0000","last_fullsized":"2023-12-01T03:10:23.650652+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:24.670899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.650715+0000","last_change":"2023-12-01T03:10:23.650715+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.650715+0000","last_undegraded":"2023-12-01T03:10:23.650715+0000","last_fullsized":"2023-12-01T03:10:23.650715+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:12.511044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.656617+0000","last_change":"2023-12-01T03:10:23.656617+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.656617+0000","last_undegraded":"2023-12-01T03:10:23.656617+0000","last_fullsized":"2023-12-01T03:10:23.656617+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.646369+0000","last_change":"2023-12-01T03:10:23.646369+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.646369+0000","last_undegraded":"2023-12-01T03:10:23.646369+0000","last_fullsized":"2023-12-01T03:10:23.646369+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:21.942226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.645340+0000","last_change":"2023-12-01T03:10:23.645340+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.645340+0000","last_undegraded":"2023-12-01T03:10:23.645340+0000","last_fullsized":"2023-12-01T03:10:23.645340+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.651158+0000","last_change":"2023-12-01T03:10:23.651158+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.651158+0000","last_undegraded":"2023-12-01T03:10:23.651158+0000","last_fullsized":"2023-12-01T03:10:23.651158+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:45.173492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.652059+0000","last_change":"2023-12-01T03:10:23.652059+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.652059+0000","last_undegraded":"2023-12-01T03:10:23.652059+0000","last_fullsized":"2023-12-01T03:10:23.652059+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:05.006954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.657655+0000","last_change":"2023-12-01T03:10:23.657655+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.657655+0000","last_undegraded":"2023-12-01T03:10:23.657655+0000","last_fullsized":"2023-12-01T03:10:23.657655+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.657536+0000","last_change":"2023-12-01T03:10:23.657536+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.657536+0000","last_undegraded":"2023-12-01T03:10:23.657536+0000","last_fullsized":"2023-12-01T03:10:23.657536+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.653116+0000","last_change":"2023-12-01T03:10:23.653116+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.653116+0000","last_undegraded":"2023-12-01T03:10:23.653116+0000","last_fullsized":"2023-12-01T03:10:23.653116+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:45.743456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.645193+0000","last_change":"2023-12-01T03:10:23.645193+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.645193+0000","last_undegraded":"2023-12-01T03:10:23.645193+0000","last_fullsized":"2023-12-01T03:10:23.645193+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:36:57.969605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.645515+0000","last_change":"2023-12-01T03:10:23.645515+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.645515+0000","last_undegraded":"2023-12-01T03:10:23.645515+0000","last_fullsized":"2023-12-01T03:10:23.645515+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:40.211424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.645026+0000","last_change":"2023-12-01T03:10:23.645026+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.645026+0000","last_undegraded":"2023-12-01T03:10:23.645026+0000","last_fullsized":"2023-12-01T03:10:23.645026+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:19:02.474558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.645317+0000","last_change":"2023-12-01T03:10:23.645317+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.645317+0000","last_undegraded":"2023-12-01T03:10:23.645317+0000","last_fullsized":"2023-12-01T03:10:23.645317+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-01T03:10:24.639162+0000","last_change":"2023-12-01T03:10:24.639162+0000","last_active":"2023-12-01T03:10:24.639162+0000","last_peered":"2023-12-01T03:10:24.639162+0000","last_clean":"2023-12-01T03:10:24.639162+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:24.639162+0000","last_undegraded":"2023-12-01T03:10:24.639162+0000","last_fullsized":"2023-12-01T03:10:24.639162+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":211,"state":"creating+peering","last_fresh":"2023-12-01T03:10:23.651879+0000","last_change":"2023-12-01T03:10:23.651879+0000","last_active":"2023-12-01T03:10:23.622088+0000","last_peered":"2023-12-01T03:10:23.622088+0000","last_clean":"2023-12-01T03:10:23.622088+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:23.651879+0000","last_undegraded":"2023-12-01T03:10:23.651879+0000","last_fullsized":"2023-12-01T03:10:23.651879+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"201'326","reported_seq":1115,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-12-01T03:10:20.740281+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:10:20.740281+0000","last_peered":"2023-12-01T03:10:20.740281+0000","last_clean":"2023-12-01T03:10:20.740281+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:10:20.740281+0000","last_undegraded":"2023-12-01T03:10:20.740281+0000","last_fullsized":"2023-12-01T03:10:20.740281+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55760,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316080,"statfs":{"total":96636764160,"available":96579665920,"internally_reserved":0,"allocated":2629632,"data_stored":1279777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0780000000000001,"15min":1.0820000000000001},"min":{"1min":0.78200000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5069999999999999,"5min":1.6000000000000001,"15min":1.98},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":0.67800000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.556,"5min":1.7,"15min":1.7},"last":1.0469999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.2889999999999999},"min":{"1min":0.80000000000000004,"5min":0.745,"15min":0.73099999999999998},"max":{"1min":1.603,"5min":2,"15min":2.1419999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3149999999999999,"15min":1.3320000000000001},"min":{"1min":1.137,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.514,"5min":1.8169999999999999,"15min":1.8700000000000001},"last":1.2150000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.157,"15min":1.1719999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.476,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.26,"15min":1.254},"min":{"1min":1.0549999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6020000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.095}]},{"osd":3,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2529999999999999,"15min":1.3049999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6559999999999999,"5min":1.712,"15min":2.431},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.415,"15min":1.4199999999999999},"min":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.92900000000000005},"max":{"1min":1.5920000000000001,"5min":1.879,"15min":2.198},"last":1.1990000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3140000000000001,"15min":1.3400000000000001},"min":{"1min":1.006,"5min":0.82599999999999996,"15min":0.81200000000000006},"max":{"1min":1.901,"5min":1.901,"15min":2.5899999999999999},"last":1.147},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":1.1240000000000001,"5min":0.92800000000000005,"15min":0.79500000000000004},"max":{"1min":1.9690000000000001,"5min":2.3279999999999998,"15min":2.569},"last":1.3129999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3009999999999999,"15min":1.3149999999999999},"min":{"1min":0.93600000000000005,"5min":0.85999999999999999,"15min":0.84599999999999997},"max":{"1min":2.0099999999999998,"5min":2.016,"15min":2.7269999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5800000000000001,"15min":1.589},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":0.93200000000000005},"max":{"1min":2.0569999999999999,"5min":2.206,"15min":2.484},"last":1.4199999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4359999999999999,"15min":1.446},"min":{"1min":1.0229999999999999,"5min":0.98599999999999999,"15min":0.82599999999999996},"max":{"1min":1.921,"5min":2.2789999999999999,"15min":2.532},"last":1.383},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":1.8819999999999999,"15min":2.762},"last":1.355}]}]},{"osd":6,"up_from":13,"seq":55834575977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41808,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330032,"statfs":{"total":96636764160,"available":96593952768,"internally_reserved":0,"allocated":2629632,"data_stored":1279777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0589999999999999,"15min":1.113},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.4039999999999999,"5min":1.6639999999999999,"15min":1.9219999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1160000000000001,"15min":1.1659999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.623,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.976,"5min":1.976,"15min":2.0449999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1950000000000001,"15min":1.2210000000000001},"min":{"1min":0.627,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.694,"5min":1.694,"15min":1.835},"last":1.038}]},{"osd":2,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.163,"15min":1.2090000000000001},"min":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.976},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2629999999999999,"15min":1.2849999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.1359999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2849999999999999,"15min":1.319},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8100000000000001,"5min":1.927,"15min":2.0289999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.405,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.2929999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2130000000000001,"15min":1.262},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.25},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.367,"15min":1.3540000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.417}]},{"osd":5,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3360000000000001,"15min":1.4039999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.383},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4790000000000001,"15min":1.484},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.506}]},{"osd":7,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4890000000000001,"15min":1.5249999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.556,"15min":1.579},"min":{"1min":0.95499999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2999999999999998,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.5840000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575976,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48148,"kb_used_data":3148,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323692,"statfs":{"total":96636764160,"available":96587460608,"internally_reserved":0,"allocated":3223552,"data_stored":1870145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.125},"min":{"1min":0.82299999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.548,"5min":1.776,"15min":1.8899999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.238,"5min":1.1639999999999999,"15min":1.163},"min":{"1min":0.89800000000000002,"5min":0.69399999999999995,"15min":0.67300000000000004},"max":{"1min":1.619,"5min":1.752,"15min":1.8009999999999999},"last":1.0720000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.133,"15min":1.1539999999999999},"min":{"1min":0.94699999999999995,"5min":0.81200000000000006,"15min":0.79900000000000004},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.696},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.272,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.88,"5min":0.79500000000000004,"15min":0.60899999999999999},"max":{"1min":1.619,"5min":1.6659999999999999,"15min":1.7649999999999999},"last":1.1279999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2390000000000001,"15min":1.246},"min":{"1min":0.93700000000000006,"5min":0.84499999999999997,"15min":0.81399999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.774},"last":1.359},{"interface":"front","average":{"1min":1.284,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.93899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.502,"5min":1.899,"15min":1.9219999999999999},"last":1.335}]},{"osd":3,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.306,"15min":1.3200000000000001},"min":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.93100000000000005},"max":{"1min":1.7929999999999999,"5min":1.8979999999999999,"15min":1.994},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.343,"15min":1.363},"min":{"1min":1.02,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.724,"5min":1.97,"15min":2.1859999999999999},"last":1.268}]},{"osd":4,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2789999999999999,"15min":1.254},"min":{"1min":0.96199999999999997,"5min":0.78000000000000003,"15min":0.74099999999999999},"max":{"1min":1.726,"5min":1.726,"15min":2.6040000000000001},"last":1.236},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.399,"15min":1.385},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.84599999999999997},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.931},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3879999999999999,"15min":1.3759999999999999},"min":{"1min":0.97599999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.9279999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.173},{"interface":"front","average":{"1min":1.478,"5min":1.427,"15min":1.419},"min":{"1min":1.222,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":1.756,"5min":1.9219999999999999,"15min":2.577},"last":1.0149999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5349999999999999,"15min":1.5369999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.962,"5min":2.0790000000000002,"15min":3.1709999999999998},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.575,"15min":1.5740000000000001},"min":{"1min":1.3720000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.831,"5min":2.101,"15min":2.8740000000000001},"last":1.3779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47580,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324260,"statfs":{"total":96636764160,"available":96588042240,"internally_reserved":0,"allocated":2641920,"data_stored":1285638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.147,"15min":1.1339999999999999},"min":{"1min":0.73099999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.143},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.21,"15min":1.1990000000000001},"min":{"1min":0.82099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5840000000000001,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":0.86799999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.242,"15min":1.2589999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.613,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.82599999999999996},"max":{"1min":1.6080000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.3899999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.3240000000000001,"15min":1.2470000000000001},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.573,"5min":11.311,"15min":11.311},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.329,"15min":1.341},"min":{"1min":1.0680000000000001,"5min":1.024,"15min":0.94099999999999995},"max":{"1min":1.647,"5min":1.9379999999999999,"15min":1.9450000000000001},"last":1.117}]},{"osd":3,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3100000000000001,"15min":1.29},"min":{"1min":1.0529999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7749999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.526},{"interface":"front","average":{"1min":1.343,"5min":1.365,"15min":1.3380000000000001},"min":{"1min":0.99199999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.599,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.4430000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.357,"15min":1.329},"min":{"1min":0.94799999999999995,"5min":0.91200000000000003,"15min":0.755},"max":{"1min":1.615,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.216},{"interface":"front","average":{"1min":1.419,"5min":1.379,"15min":1.3120000000000001},"min":{"1min":0.94599999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7150000000000001,"5min":3.71,"15min":3.71},"last":1.6579999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3740000000000001,"15min":1.375},"min":{"1min":1.095,"5min":0.745,"15min":0.745},"max":{"1min":1.609,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.569},{"interface":"front","average":{"1min":1.514,"5min":1.4850000000000001,"15min":1.48},"min":{"1min":1.335,"5min":1.0660000000000001,"15min":1.0149999999999999},"max":{"1min":1.738,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.6060000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.52,"15min":1.486},"min":{"1min":1.157,"5min":0.95499999999999996,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.72},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6000000000000001,"15min":1.5800000000000001},"min":{"1min":1.3600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.806,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.6910000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51676,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320164,"statfs":{"total":96636764160,"available":96583847936,"internally_reserved":0,"allocated":2641920,"data_stored":1285638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91100000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.085},"min":{"1min":0.74199999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.571,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.226}]},{"osd":1,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.163,"15min":1.171},"min":{"1min":0.64700000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.121},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.008,"15min":0.999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41099999999999998},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":1.1639999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0900000000000001,"15min":1.103},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.27,"5min":1.2230000000000001,"15min":1.252},"min":{"1min":0.80100000000000005,"5min":0.747,"15min":0.58599999999999997},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.3169999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2989999999999999,"15min":1.2929999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.3},"min":{"1min":0.96799999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.999}]},{"osd":5,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3240000000000001,"15min":1.331},"min":{"1min":0.86599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.373,"15min":1.3759999999999999},"min":{"1min":0.88400000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.0349999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.78400000000000003},"max":{"1min":2.5649999999999999,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.54,"5min":1.5049999999999999,"15min":1.514},"min":{"1min":1.075,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.1879999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.516,"15min":1.5329999999999999},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.409},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.54,"15min":1.5309999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.878},"max":{"1min":2.4830000000000001,"5min":2.984,"15min":2.984},"last":1.3500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2641920,"data_stored":1285638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89300000000000002,"15min":0.89400000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1140000000000001,"5min":1.405,"15min":1.4730000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.054,"5min":1.056,"15min":1.0569999999999999},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5640000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.252}]},{"osd":1,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0449999999999999,"15min":1.0589999999999999},"min":{"1min":0.66100000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.321,"5min":1.698,"15min":1.698},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.038,"5min":1.125,"15min":1.151},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.522,"5min":1.635,"15min":1.71},"last":0.97799999999999998}]},{"osd":3,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0640000000000001,"15min":1.054},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.419,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1990000000000001,"15min":1.2170000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.516,"5min":1.6879999999999999,"15min":1.79},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":0.85199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.7569999999999999,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.714,"5min":1.714,"15min":1.835},"last":1.012}]},{"osd":5,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3919999999999999,"15min":1.391},"min":{"1min":0.88,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.8380000000000001,"15min":2.2040000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.419,"15min":1.4179999999999999},"min":{"1min":1.095,"5min":1.004,"15min":1.004},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.016},"last":1.321}]},{"osd":6,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.379,"15min":1.3819999999999999},"min":{"1min":0.89600000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.131},{"interface":"front","average":{"1min":1.46,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":1.087,"5min":0.879,"15min":0.879},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.179}]},{"osd":7,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5069999999999999,"15min":1.4910000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0979999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.117},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4330000000000001,"15min":1.458},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.012,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.27}]}]},{"osd":1,"up_from":13,"seq":55834575974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48148,"kb_used_data":3148,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323692,"statfs":{"total":96636764160,"available":96587460608,"internally_reserved":0,"allocated":3223552,"data_stored":1870145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.879,"15min":0.90900000000000003},"min":{"1min":0.58199999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.099,"5min":0.99299999999999999,"15min":1.046},"min":{"1min":0.81000000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.5069999999999999,"5min":1.5660000000000001,"15min":1.681},"last":1.0920000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.56299999999999994,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.115,"15min":1.1419999999999999},"min":{"1min":0.75600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.581,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":1.3220000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.258,"15min":1.3069999999999999},"min":{"1min":1.026,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5780000000000001,"5min":1.587,"15min":1.788},"last":1.212},{"interface":"front","average":{"1min":1.171,"5min":1.107,"15min":1.135},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.714,"5min":1.714,"15min":1.8049999999999999},"last":1.268}]},{"osd":4,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.21,"15min":1.2230000000000001},"min":{"1min":0.89900000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6859999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.389,"15min":1.409},"min":{"1min":1.0740000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.573}]},{"osd":5,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.85899999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.802,"5min":1.8520000000000001,"15min":2.101},"last":1.421},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.304,"15min":1.339},"min":{"1min":1.006,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.609,"5min":1.927,"15min":2.1659999999999999},"last":1.3740000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4019999999999999,"15min":1.4379999999999999},"min":{"1min":0.99399999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3799999999999999,"15min":1.4119999999999999},"min":{"1min":1.1779999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.768,"5min":2.0019999999999998,"15min":2.0209999999999999},"last":1.5}]},{"osd":7,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4870000000000001,"15min":1.5269999999999999},"min":{"1min":1.2549999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.911,"5min":2.0329999999999999,"15min":2.1190000000000002},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.524,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.708}]}]},{"osd":0,"up_from":13,"seq":55834575976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2629632,"data_stored":1279777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.86699999999999999,"15min":0.89400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1559999999999999,"5min":1.4379999999999999,"15min":1.994},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.04,"15min":1.056},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.427,"5min":1.583,"15min":1.6799999999999999},"last":0.80500000000000005}]},{"osd":2,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0329999999999999,"15min":1.069},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.446,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":1.262},{"interface":"front","average":{"1min":1.133,"5min":1.1020000000000001,"15min":1.1419999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.726,"5min":1.726,"15min":1.792},"last":1.1140000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0580000000000001,"15min":1.071},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.8839999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.655,"5min":1.655,"15min":2.6469999999999998},"last":1.5700000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3260000000000001,"15min":1.363},"min":{"1min":0.88400000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.284,"15min":1.3089999999999999},"min":{"1min":0.82099999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.677,"5min":1.74,"15min":2.3610000000000002},"last":1.214}]},{"osd":5,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.355,"15min":1.397},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5589999999999999,"5min":2.1030000000000002,"15min":2.7719999999999998},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.4099999999999999,"15min":1.4410000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.79600000000000004},"max":{"1min":1.9259999999999999,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":1.2030000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.367,"15min":1.3859999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7649999999999999,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.458,"15min":1.484},"min":{"1min":1.048,"5min":0.85299999999999998,"15min":0.75800000000000001},"max":{"1min":1.736,"5min":1.9870000000000001,"15min":2.8759999999999999},"last":1.833}]},{"osd":7,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.49,"15min":1.524},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.841,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4750000000000001,"15min":1.4910000000000001},"min":{"1min":0.97299999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.0619999999999998,"5min":2.1190000000000002,"15min":2.431},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:10:29.816 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:10:30.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.065+0000 7f4545527640 1 Processor -- start 2023-12-01T03:10:30.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.065+0000 7f4545527640 1 -- start start 2023-12-01T03:10:30.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.067+0000 7f4545527640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f454015d220 0x7f45400cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.067+0000 7f4545527640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45400ad2b0 0x7f45400ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.067+0000 7f4545527640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454015d600 con 0x7f45400ade80 2023-12-01T03:10:30.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.067+0000 7f4545527640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454006cc10 con 0x7f454015d220 2023-12-01T03:10:30.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.067+0000 7f4545527640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454006c720 con 0x7f45400ad2b0 2023-12-01T03:10:30.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.067+0000 7f453f7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f45400ade80 0x7f454006b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37220/0 (socket says 172.21.15.115:37220) 2023-12-01T03:10:30.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.067+0000 7f453f7fe640 1 -- 172.21.15.115:0/860972282 learned_addr learned my addr 172.21.15.115:0/860972282 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:30.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.067+0000 7f453effd640 1 --2- 172.21.15.115:0/860972282 >> v2:172.21.15.142:3300/0 conn(0x7f454015d220 0x7f45400cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.067+0000 7f453e7fc640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45400ad2b0 0x7f45400ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267752466 0 0) 0x7f4538001490 con 0x7f45400ade80 2023-12-01T03:10:30.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4528003520 con 0x7f45400ade80 2023-12-01T03:10:30.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3302144742 0 0) 0x7f4528003520 con 0x7f45400ade80 2023-12-01T03:10:30.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45400ad2b0 msgr2=0x7f45400ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:30.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45400ad2b0 0x7f45400ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 >> v2:172.21.15.142:3300/0 conn(0x7f454015d220 msgr2=0x7f45400cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:30.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 --2- 172.21.15.115:0/860972282 >> v2:172.21.15.142:3300/0 conn(0x7f454015d220 0x7f45400cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454006c4b0 con 0x7f45400ade80 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4538001b80 con 0x7f45400ade80 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 >> v1:172.21.15.115:6789/0 conn(0x7f45400ade80 legacy=0x7f454006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45400ad2b0 0x7f45280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4528003f60 0x7f45280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4528004d00 0x7f4528005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454006cc10 con 0x7f4528003f60 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454006c720 con 0x7f4528004d00 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452800ba30 con 0x7f45400ad2b0 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.068+0000 7f453e7fc640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4528003f60 0x7f45280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.069+0000 7f453f7fe640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4528004d00 0x7f4528005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.069+0000 7f453e7fc640 1 -- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45400ad2b0 msgr2=0x7f45280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.069+0000 7f453e7fc640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f45400ad2b0 0x7f45280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.069+0000 7f453e7fc640 1 -- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4528004d00 msgr2=0x7f4528005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.069+0000 7f453e7fc640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4528004d00 0x7f4528005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.069+0000 7f453e7fc640 1 -- 172.21.15.115:0/860972282 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528003520 con 0x7f4528003f60 2023-12-01T03:10:30.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.069+0000 7f453f7fe640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4528004d00 0x7f4528005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:30.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.070+0000 7f453e7fc640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4528003f60 0x7f45280047c0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f4534009ac0 tx=0x7f45280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba81de025e3f96c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.070+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453402c710 con 0x7f4528003f60 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.070+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453402c870 con 0x7f4528003f60 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.070+0000 7f453dffb640 1 -- 172.21.15.115:0/860972282 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453402cc00 con 0x7f4528003f60 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.071+0000 7f4545527640 1 -- 172.21.15.115:0/860972282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4528003f60 msgr2=0x7f45280047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.071+0000 7f4545527640 1 --2- 172.21.15.115:0/860972282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4528003f60 0x7f45280047c0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f4534009ac0 tx=0x7f45280057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.071+0000 7f453f7fe640 1 -- 172.21.15.115:0/860972282 reap_dead start 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.071+0000 7f4545527640 1 -- 172.21.15.115:0/860972282 shutdown_connections 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.071+0000 7f4545527640 1 -- 172.21.15.115:0/860972282 >> 172.21.15.115:0/860972282 conn(0x7f45400ce880 msgr2=0x7f45400cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.071+0000 7f4545527640 1 -- 172.21.15.115:0/860972282 shutdown_connections 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.071+0000 7f4545527640 1 -- 172.21.15.115:0/860972282 wait complete. 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.073+0000 7f4545527640 1 Processor -- start 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.073+0000 7f4545527640 1 -- start start 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f4545527640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4528003f60 0x7f45400b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f4545527640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454014ec90 0x7f45401e4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f4545527640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45401e4d90 0x7f45401f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f4545527640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45401e5210 con 0x7f454014ec90 2023-12-01T03:10:30.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f4545527640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45401e76c0 con 0x7f45401e4d90 2023-12-01T03:10:30.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f4545527640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400b1d80 con 0x7f4528003f60 2023-12-01T03:10:30.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f453e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454014ec90 0x7f45401e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f453e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454014ec90 0x7f45401e4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33700/0 (socket says 172.21.15.115:33700) 2023-12-01T03:10:30.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f453e7fc640 1 -- 172.21.15.115:0/3739102131 learned_addr learned my addr 172.21.15.115:0/3739102131 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:30.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f453e7fc640 1 -- 172.21.15.115:0/3739102131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4528003f60 msgr2=0x7f45400b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f453effd640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4528003f60 0x7f45400b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f453e7fc640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4528003f60 0x7f45400b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f453e7fc640 1 -- 172.21.15.115:0/3739102131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45401e4d90 msgr2=0x7f45401f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f453f7fe640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45401e4d90 0x7f45401f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f453e7fc640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45401e4d90 0x7f45401f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.074+0000 7f453e7fc640 1 -- 172.21.15.115:0/3739102131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45401e6ab0 con 0x7f454014ec90 2023-12-01T03:10:30.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.075+0000 7f453e7fc640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454014ec90 0x7f45401e4850 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f4534009ac0 tx=0x7f4534004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:30.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.075+0000 7f45277fe640 1 -- 172.21.15.115:0/3739102131 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534003810 con 0x7f454014ec90 2023-12-01T03:10:30.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.075+0000 7f45277fe640 1 -- 172.21.15.115:0/3739102131 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534003970 con 0x7f454014ec90 2023-12-01T03:10:30.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.075+0000 7f45277fe640 1 -- 172.21.15.115:0/3739102131 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534003d00 con 0x7f454014ec90 2023-12-01T03:10:30.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.075+0000 7f4545527640 1 -- 172.21.15.115:0/3739102131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45401e6d90 con 0x7f454014ec90 2023-12-01T03:10:30.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.075+0000 7f4545527640 1 -- 172.21.15.115:0/3739102131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45401e7350 con 0x7f454014ec90 2023-12-01T03:10:30.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.076+0000 7f4545527640 1 -- 172.21.15.115:0/3739102131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454014f6d0 con 0x7f454014ec90 2023-12-01T03:10:30.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.077+0000 7f45277fe640 1 -- 172.21.15.115:0/3739102131 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4534040450 con 0x7f454014ec90 2023-12-01T03:10:30.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.078+0000 7f45277fe640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f452c06a580 0x7f452c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.079+0000 7f45277fe640 1 -- 172.21.15.115:0/3739102131 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f45340aa990 con 0x7f454014ec90 2023-12-01T03:10:30.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.079+0000 7f453effd640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f452c06a580 0x7f452c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.079+0000 7f453effd640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f452c06a580 0x7f452c06ca40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f45401f5e80 tx=0x7f4530008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:30.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.083+0000 7f45277fe640 1 -- 172.21.15.115:0/3739102131 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f454014f6d0 con 0x7f454014ec90 2023-12-01T03:10:30.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.268+0000 7f4545527640 1 -- 172.21.15.115:0/3739102131 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f454006ce10 con 0x7f452c06a580 2023-12-01T03:10:30.286 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:10:30.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:10:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:30.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.296+0000 7f45277fe640 1 -- 172.21.15.115:0/3739102131 <== mgr.10407 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286956 (secure 0 0 0) 0x7f45300107f0 con 0x7f452c06a580 2023-12-01T03:10:30.304 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:10:30.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.303+0000 7f4545527640 1 -- 172.21.15.115:0/3739102131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f452c06a580 msgr2=0x7f452c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.303+0000 7f4545527640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f452c06a580 0x7f452c06ca40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f45401f5e80 tx=0x7f4530008040 comp rx=0 tx=0).stop 2023-12-01T03:10:30.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.303+0000 7f4545527640 1 -- 172.21.15.115:0/3739102131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454014ec90 msgr2=0x7f45401e4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.303+0000 7f4545527640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454014ec90 0x7f45401e4850 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f4534009ac0 tx=0x7f4534004730 comp rx=0 tx=0).stop 2023-12-01T03:10:30.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.303+0000 7f4545527640 1 -- 172.21.15.115:0/3739102131 shutdown_connections 2023-12-01T03:10:30.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.303+0000 7f4545527640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f452c06a580 0x7f452c06ca40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.303+0000 7f4545527640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45401e4d90 0x7f45401f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.303+0000 7f4545527640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f454014ec90 0x7f45401e4850 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.303+0000 7f4545527640 1 --2- 172.21.15.115:0/3739102131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4528003f60 0x7f45400b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.303+0000 7f4545527640 1 -- 172.21.15.115:0/3739102131 >> 172.21.15.115:0/3739102131 conn(0x7f45400ce880 msgr2=0x7f4540067c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:30.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.304+0000 7f4545527640 1 -- 172.21.15.115:0/3739102131 shutdown_connections 2023-12-01T03:10:30.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.304+0000 7f4545527640 1 -- 172.21.15.115:0/3739102131 wait complete. 2023-12-01T03:10:30.323 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-12-01T03:10:29.795982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417352,"kb_used_data":22152,"kb_used_omap":0,"kb_used_meta":395136,"kb_avail":754557368,"statfs":{"total":773094113280,"available":772666744832,"internally_reserved":0,"allocated":22683648,"data_stored":11626420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":404619264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001675"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133152+0000","last_change":"2023-12-01T03:10:25.171115+0000","last_active":"2023-12-01T03:10:27.133152+0000","last_peered":"2023-12-01T03:10:27.133152+0000","last_clean":"2023-12-01T03:10:27.133152+0000","last_became_active":"2023-12-01T03:10:25.170871+0000","last_became_peered":"2023-12-01T03:10:25.170871+0000","last_unstale":"2023-12-01T03:10:27.133152+0000","last_undegraded":"2023-12-01T03:10:27.133152+0000","last_fullsized":"2023-12-01T03:10:27.133152+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:24.370036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.927261+0000","last_change":"2023-12-01T03:10:25.181088+0000","last_active":"2023-12-01T03:10:26.927261+0000","last_peered":"2023-12-01T03:10:26.927261+0000","last_clean":"2023-12-01T03:10:26.927261+0000","last_became_active":"2023-12-01T03:10:25.180741+0000","last_became_peered":"2023-12-01T03:10:25.180741+0000","last_unstale":"2023-12-01T03:10:26.927261+0000","last_undegraded":"2023-12-01T03:10:26.927261+0000","last_fullsized":"2023-12-01T03:10:26.927261+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:26.321285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.166171+0000","last_change":"2023-12-01T03:10:25.181720+0000","last_active":"2023-12-01T03:10:26.166171+0000","last_peered":"2023-12-01T03:10:26.166171+0000","last_clean":"2023-12-01T03:10:26.166171+0000","last_became_active":"2023-12-01T03:10:25.180144+0000","last_became_peered":"2023-12-01T03:10:25.180144+0000","last_unstale":"2023-12-01T03:10:26.166171+0000","last_undegraded":"2023-12-01T03:10:26.166171+0000","last_fullsized":"2023-12-01T03:10:26.166171+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:53.999589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.926976+0000","last_change":"2023-12-01T03:10:25.171460+0000","last_active":"2023-12-01T03:10:26.926976+0000","last_peered":"2023-12-01T03:10:26.926976+0000","last_clean":"2023-12-01T03:10:26.926976+0000","last_became_active":"2023-12-01T03:10:25.171148+0000","last_became_peered":"2023-12-01T03:10:25.171148+0000","last_unstale":"2023-12-01T03:10:26.926976+0000","last_undegraded":"2023-12-01T03:10:26.926976+0000","last_fullsized":"2023-12-01T03:10:26.926976+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:01:37.278740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.169078+0000","last_change":"2023-12-01T03:10:25.191253+0000","last_active":"2023-12-01T03:10:26.169078+0000","last_peered":"2023-12-01T03:10:26.169078+0000","last_clean":"2023-12-01T03:10:26.169078+0000","last_became_active":"2023-12-01T03:10:25.188485+0000","last_became_peered":"2023-12-01T03:10:25.188485+0000","last_unstale":"2023-12-01T03:10:26.169078+0000","last_undegraded":"2023-12-01T03:10:26.169078+0000","last_fullsized":"2023-12-01T03:10:26.169078+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.170682+0000","last_change":"2023-12-01T03:10:25.182648+0000","last_active":"2023-12-01T03:10:26.170682+0000","last_peered":"2023-12-01T03:10:26.170682+0000","last_clean":"2023-12-01T03:10:26.170682+0000","last_became_active":"2023-12-01T03:10:25.182343+0000","last_became_peered":"2023-12-01T03:10:25.182343+0000","last_unstale":"2023-12-01T03:10:26.170682+0000","last_undegraded":"2023-12-01T03:10:26.170682+0000","last_fullsized":"2023-12-01T03:10:26.170682+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:55.408381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133271+0000","last_change":"2023-12-01T03:10:25.186300+0000","last_active":"2023-12-01T03:10:27.133271+0000","last_peered":"2023-12-01T03:10:27.133271+0000","last_clean":"2023-12-01T03:10:27.133271+0000","last_became_active":"2023-12-01T03:10:25.185030+0000","last_became_peered":"2023-12-01T03:10:25.185030+0000","last_unstale":"2023-12-01T03:10:27.133271+0000","last_undegraded":"2023-12-01T03:10:27.133271+0000","last_fullsized":"2023-12-01T03:10:27.133271+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.168135+0000","last_change":"2023-12-01T03:10:25.185623+0000","last_active":"2023-12-01T03:10:26.168135+0000","last_peered":"2023-12-01T03:10:26.168135+0000","last_clean":"2023-12-01T03:10:26.168135+0000","last_became_active":"2023-12-01T03:10:25.185311+0000","last_became_peered":"2023-12-01T03:10:25.185311+0000","last_unstale":"2023-12-01T03:10:26.168135+0000","last_undegraded":"2023-12-01T03:10:26.168135+0000","last_fullsized":"2023-12-01T03:10:26.168135+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.927337+0000","last_change":"2023-12-01T03:10:25.172560+0000","last_active":"2023-12-01T03:10:26.927337+0000","last_peered":"2023-12-01T03:10:26.927337+0000","last_clean":"2023-12-01T03:10:26.927337+0000","last_became_active":"2023-12-01T03:10:25.172005+0000","last_became_peered":"2023-12-01T03:10:25.172005+0000","last_unstale":"2023-12-01T03:10:26.927337+0000","last_undegraded":"2023-12-01T03:10:26.927337+0000","last_fullsized":"2023-12-01T03:10:26.927337+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:44.541671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133663+0000","last_change":"2023-12-01T03:10:25.177089+0000","last_active":"2023-12-01T03:10:27.133663+0000","last_peered":"2023-12-01T03:10:27.133663+0000","last_clean":"2023-12-01T03:10:27.133663+0000","last_became_active":"2023-12-01T03:10:25.176742+0000","last_became_peered":"2023-12-01T03:10:25.176742+0000","last_unstale":"2023-12-01T03:10:27.133663+0000","last_undegraded":"2023-12-01T03:10:27.133663+0000","last_fullsized":"2023-12-01T03:10:27.133663+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.132542+0000","last_change":"2023-12-01T03:10:25.181052+0000","last_active":"2023-12-01T03:10:27.132542+0000","last_peered":"2023-12-01T03:10:27.132542+0000","last_clean":"2023-12-01T03:10:27.132542+0000","last_became_active":"2023-12-01T03:10:25.180810+0000","last_became_peered":"2023-12-01T03:10:25.180810+0000","last_unstale":"2023-12-01T03:10:27.132542+0000","last_undegraded":"2023-12-01T03:10:27.132542+0000","last_fullsized":"2023-12-01T03:10:27.132542+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:35.663128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.532964+0000","last_change":"2023-12-01T03:10:25.172316+0000","last_active":"2023-12-01T03:10:26.532964+0000","last_peered":"2023-12-01T03:10:26.532964+0000","last_clean":"2023-12-01T03:10:26.532964+0000","last_became_active":"2023-12-01T03:10:25.169042+0000","last_became_peered":"2023-12-01T03:10:25.169042+0000","last_unstale":"2023-12-01T03:10:26.532964+0000","last_undegraded":"2023-12-01T03:10:26.532964+0000","last_fullsized":"2023-12-01T03:10:26.532964+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.132446+0000","last_change":"2023-12-01T03:10:25.178225+0000","last_active":"2023-12-01T03:10:27.132446+0000","last_peered":"2023-12-01T03:10:27.132446+0000","last_clean":"2023-12-01T03:10:27.132446+0000","last_became_active":"2023-12-01T03:10:25.177541+0000","last_became_peered":"2023-12-01T03:10:25.177541+0000","last_unstale":"2023-12-01T03:10:27.132446+0000","last_undegraded":"2023-12-01T03:10:27.132446+0000","last_fullsized":"2023-12-01T03:10:27.132446+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:18.470068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.926461+0000","last_change":"2023-12-01T03:10:25.173081+0000","last_active":"2023-12-01T03:10:26.926461+0000","last_peered":"2023-12-01T03:10:26.926461+0000","last_clean":"2023-12-01T03:10:26.926461+0000","last_became_active":"2023-12-01T03:10:25.172845+0000","last_became_peered":"2023-12-01T03:10:25.172845+0000","last_unstale":"2023-12-01T03:10:26.926461+0000","last_undegraded":"2023-12-01T03:10:26.926461+0000","last_fullsized":"2023-12-01T03:10:26.926461+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:18.173873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.132011+0000","last_change":"2023-12-01T03:10:25.170590+0000","last_active":"2023-12-01T03:10:27.132011+0000","last_peered":"2023-12-01T03:10:27.132011+0000","last_clean":"2023-12-01T03:10:27.132011+0000","last_became_active":"2023-12-01T03:10:25.170334+0000","last_became_peered":"2023-12-01T03:10:25.170334+0000","last_unstale":"2023-12-01T03:10:27.132011+0000","last_undegraded":"2023-12-01T03:10:27.132011+0000","last_fullsized":"2023-12-01T03:10:27.132011+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:35.058136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.166630+0000","last_change":"2023-12-01T03:10:25.189550+0000","last_active":"2023-12-01T03:10:26.166630+0000","last_peered":"2023-12-01T03:10:26.166630+0000","last_clean":"2023-12-01T03:10:26.166630+0000","last_became_active":"2023-12-01T03:10:25.189001+0000","last_became_peered":"2023-12-01T03:10:25.189001+0000","last_unstale":"2023-12-01T03:10:26.166630+0000","last_undegraded":"2023-12-01T03:10:26.166630+0000","last_fullsized":"2023-12-01T03:10:26.166630+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.171174+0000","last_change":"2023-12-01T03:10:25.173100+0000","last_active":"2023-12-01T03:10:26.171174+0000","last_peered":"2023-12-01T03:10:26.171174+0000","last_clean":"2023-12-01T03:10:26.171174+0000","last_became_active":"2023-12-01T03:10:25.172808+0000","last_became_peered":"2023-12-01T03:10:25.172808+0000","last_unstale":"2023-12-01T03:10:26.171174+0000","last_undegraded":"2023-12-01T03:10:26.171174+0000","last_fullsized":"2023-12-01T03:10:26.171174+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133900+0000","last_change":"2023-12-01T03:10:25.182676+0000","last_active":"2023-12-01T03:10:27.133900+0000","last_peered":"2023-12-01T03:10:27.133900+0000","last_clean":"2023-12-01T03:10:27.133900+0000","last_became_active":"2023-12-01T03:10:25.182429+0000","last_became_peered":"2023-12-01T03:10:25.182429+0000","last_unstale":"2023-12-01T03:10:27.133900+0000","last_undegraded":"2023-12-01T03:10:27.133900+0000","last_fullsized":"2023-12-01T03:10:27.133900+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.166246+0000","last_change":"2023-12-01T03:10:25.180992+0000","last_active":"2023-12-01T03:10:26.166246+0000","last_peered":"2023-12-01T03:10:26.166246+0000","last_clean":"2023-12-01T03:10:26.166246+0000","last_became_active":"2023-12-01T03:10:25.180712+0000","last_became_peered":"2023-12-01T03:10:25.180712+0000","last_unstale":"2023-12-01T03:10:26.166246+0000","last_undegraded":"2023-12-01T03:10:26.166246+0000","last_fullsized":"2023-12-01T03:10:26.166246+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:43.827830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.170255+0000","last_change":"2023-12-01T03:10:25.189264+0000","last_active":"2023-12-01T03:10:26.170255+0000","last_peered":"2023-12-01T03:10:26.170255+0000","last_clean":"2023-12-01T03:10:26.170255+0000","last_became_active":"2023-12-01T03:10:25.189007+0000","last_became_peered":"2023-12-01T03:10:25.189007+0000","last_unstale":"2023-12-01T03:10:26.170255+0000","last_undegraded":"2023-12-01T03:10:26.170255+0000","last_fullsized":"2023-12-01T03:10:26.170255+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:14.783134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.165855+0000","last_change":"2023-12-01T03:10:25.174568+0000","last_active":"2023-12-01T03:10:26.165855+0000","last_peered":"2023-12-01T03:10:26.165855+0000","last_clean":"2023-12-01T03:10:26.165855+0000","last_became_active":"2023-12-01T03:10:25.174229+0000","last_became_peered":"2023-12-01T03:10:25.174229+0000","last_unstale":"2023-12-01T03:10:26.165855+0000","last_undegraded":"2023-12-01T03:10:26.165855+0000","last_fullsized":"2023-12-01T03:10:26.165855+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:13.528118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.168917+0000","last_change":"2023-12-01T03:10:25.188998+0000","last_active":"2023-12-01T03:10:26.168917+0000","last_peered":"2023-12-01T03:10:26.168917+0000","last_clean":"2023-12-01T03:10:26.168917+0000","last_became_active":"2023-12-01T03:10:25.188657+0000","last_became_peered":"2023-12-01T03:10:25.188657+0000","last_unstale":"2023-12-01T03:10:26.168917+0000","last_undegraded":"2023-12-01T03:10:26.168917+0000","last_fullsized":"2023-12-01T03:10:26.168917+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:23:58.752943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.132880+0000","last_change":"2023-12-01T03:10:25.181678+0000","last_active":"2023-12-01T03:10:27.132880+0000","last_peered":"2023-12-01T03:10:27.132880+0000","last_clean":"2023-12-01T03:10:27.132880+0000","last_became_active":"2023-12-01T03:10:25.181430+0000","last_became_peered":"2023-12-01T03:10:25.181430+0000","last_unstale":"2023-12-01T03:10:27.132880+0000","last_undegraded":"2023-12-01T03:10:27.132880+0000","last_fullsized":"2023-12-01T03:10:27.132880+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.168152+0000","last_change":"2023-12-01T03:10:25.175356+0000","last_active":"2023-12-01T03:10:26.168152+0000","last_peered":"2023-12-01T03:10:26.168152+0000","last_clean":"2023-12-01T03:10:26.168152+0000","last_became_active":"2023-12-01T03:10:25.175086+0000","last_became_peered":"2023-12-01T03:10:25.175086+0000","last_unstale":"2023-12-01T03:10:26.168152+0000","last_undegraded":"2023-12-01T03:10:26.168152+0000","last_fullsized":"2023-12-01T03:10:26.168152+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.927377+0000","last_change":"2023-12-01T03:10:25.179679+0000","last_active":"2023-12-01T03:10:26.927377+0000","last_peered":"2023-12-01T03:10:26.927377+0000","last_clean":"2023-12-01T03:10:26.927377+0000","last_became_active":"2023-12-01T03:10:25.179442+0000","last_became_peered":"2023-12-01T03:10:25.179442+0000","last_unstale":"2023-12-01T03:10:26.927377+0000","last_undegraded":"2023-12-01T03:10:26.927377+0000","last_fullsized":"2023-12-01T03:10:26.927377+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:57:25.728865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.532797+0000","last_change":"2023-12-01T03:10:25.181469+0000","last_active":"2023-12-01T03:10:26.532797+0000","last_peered":"2023-12-01T03:10:26.532797+0000","last_clean":"2023-12-01T03:10:26.532797+0000","last_became_active":"2023-12-01T03:10:25.180646+0000","last_became_peered":"2023-12-01T03:10:25.180646+0000","last_unstale":"2023-12-01T03:10:26.532797+0000","last_undegraded":"2023-12-01T03:10:26.532797+0000","last_fullsized":"2023-12-01T03:10:26.532797+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:53.001661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.166316+0000","last_change":"2023-12-01T03:10:25.178072+0000","last_active":"2023-12-01T03:10:26.166316+0000","last_peered":"2023-12-01T03:10:26.166316+0000","last_clean":"2023-12-01T03:10:26.166316+0000","last_became_active":"2023-12-01T03:10:25.177778+0000","last_became_peered":"2023-12-01T03:10:25.177778+0000","last_unstale":"2023-12-01T03:10:26.166316+0000","last_undegraded":"2023-12-01T03:10:26.166316+0000","last_fullsized":"2023-12-01T03:10:26.166316+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:48.799119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133187+0000","last_change":"2023-12-01T03:10:25.176284+0000","last_active":"2023-12-01T03:10:27.133187+0000","last_peered":"2023-12-01T03:10:27.133187+0000","last_clean":"2023-12-01T03:10:27.133187+0000","last_became_active":"2023-12-01T03:10:25.175494+0000","last_became_peered":"2023-12-01T03:10:25.175494+0000","last_unstale":"2023-12-01T03:10:27.133187+0000","last_undegraded":"2023-12-01T03:10:27.133187+0000","last_fullsized":"2023-12-01T03:10:27.133187+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.532629+0000","last_change":"2023-12-01T03:10:25.178098+0000","last_active":"2023-12-01T03:10:26.532629+0000","last_peered":"2023-12-01T03:10:26.532629+0000","last_clean":"2023-12-01T03:10:26.532629+0000","last_became_active":"2023-12-01T03:10:25.177031+0000","last_became_peered":"2023-12-01T03:10:25.177031+0000","last_unstale":"2023-12-01T03:10:26.532629+0000","last_undegraded":"2023-12-01T03:10:26.532629+0000","last_fullsized":"2023-12-01T03:10:26.532629+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.532748+0000","last_change":"2023-12-01T03:10:25.175090+0000","last_active":"2023-12-01T03:10:26.532748+0000","last_peered":"2023-12-01T03:10:26.532748+0000","last_clean":"2023-12-01T03:10:26.532748+0000","last_became_active":"2023-12-01T03:10:25.174835+0000","last_became_peered":"2023-12-01T03:10:25.174835+0000","last_unstale":"2023-12-01T03:10:26.532748+0000","last_undegraded":"2023-12-01T03:10:26.532748+0000","last_fullsized":"2023-12-01T03:10:26.532748+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.168516+0000","last_change":"2023-12-01T03:10:25.186126+0000","last_active":"2023-12-01T03:10:26.168516+0000","last_peered":"2023-12-01T03:10:26.168516+0000","last_clean":"2023-12-01T03:10:26.168516+0000","last_became_active":"2023-12-01T03:10:25.185741+0000","last_became_peered":"2023-12-01T03:10:25.185741+0000","last_unstale":"2023-12-01T03:10:26.168516+0000","last_undegraded":"2023-12-01T03:10:26.168516+0000","last_fullsized":"2023-12-01T03:10:26.168516+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.168282+0000","last_change":"2023-12-01T03:10:25.185385+0000","last_active":"2023-12-01T03:10:26.168282+0000","last_peered":"2023-12-01T03:10:26.168282+0000","last_clean":"2023-12-01T03:10:26.168282+0000","last_became_active":"2023-12-01T03:10:25.185098+0000","last_became_peered":"2023-12-01T03:10:25.185098+0000","last_unstale":"2023-12-01T03:10:26.168282+0000","last_undegraded":"2023-12-01T03:10:26.168282+0000","last_fullsized":"2023-12-01T03:10:26.168282+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.170047+0000","last_change":"2023-12-01T03:10:25.191865+0000","last_active":"2023-12-01T03:10:26.170047+0000","last_peered":"2023-12-01T03:10:26.170047+0000","last_clean":"2023-12-01T03:10:26.170047+0000","last_became_active":"2023-12-01T03:10:25.189277+0000","last_became_peered":"2023-12-01T03:10:25.189277+0000","last_unstale":"2023-12-01T03:10:26.170047+0000","last_undegraded":"2023-12-01T03:10:26.170047+0000","last_fullsized":"2023-12-01T03:10:26.170047+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"213'4","reported_seq":15,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.184078+0000","last_change":"2023-12-01T03:10:24.697099+0000","last_active":"2023-12-01T03:10:25.184078+0000","last_peered":"2023-12-01T03:10:25.184078+0000","last_clean":"2023-12-01T03:10:25.184078+0000","last_became_active":"2023-12-01T03:10:24.696783+0000","last_became_peered":"2023-12-01T03:10:24.696783+0000","last_unstale":"2023-12-01T03:10:25.184078+0000","last_undegraded":"2023-12-01T03:10:25.184078+0000","last_fullsized":"2023-12-01T03:10:25.184078+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:07.101180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133877+0000","last_change":"2023-12-01T03:10:25.180335+0000","last_active":"2023-12-01T03:10:27.133877+0000","last_peered":"2023-12-01T03:10:27.133877+0000","last_clean":"2023-12-01T03:10:27.133877+0000","last_became_active":"2023-12-01T03:10:25.180020+0000","last_became_peered":"2023-12-01T03:10:25.180020+0000","last_unstale":"2023-12-01T03:10:27.133877+0000","last_undegraded":"2023-12-01T03:10:27.133877+0000","last_fullsized":"2023-12-01T03:10:27.133877+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"213'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.184319+0000","last_change":"2023-12-01T03:10:24.688812+0000","last_active":"2023-12-01T03:10:25.184319+0000","last_peered":"2023-12-01T03:10:25.184319+0000","last_clean":"2023-12-01T03:10:25.184319+0000","last_became_active":"2023-12-01T03:10:24.687577+0000","last_became_peered":"2023-12-01T03:10:24.687577+0000","last_unstale":"2023-12-01T03:10:25.184319+0000","last_undegraded":"2023-12-01T03:10:25.184319+0000","last_fullsized":"2023-12-01T03:10:25.184319+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.927273+0000","last_change":"2023-12-01T03:10:25.179131+0000","last_active":"2023-12-01T03:10:26.927273+0000","last_peered":"2023-12-01T03:10:26.927273+0000","last_clean":"2023-12-01T03:10:26.927273+0000","last_became_active":"2023-12-01T03:10:25.178708+0000","last_became_peered":"2023-12-01T03:10:25.178708+0000","last_unstale":"2023-12-01T03:10:26.927273+0000","last_undegraded":"2023-12-01T03:10:26.927273+0000","last_fullsized":"2023-12-01T03:10:26.927273+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:43.370389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"213'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.192862+0000","last_change":"2023-12-01T03:10:24.700378+0000","last_active":"2023-12-01T03:10:25.192862+0000","last_peered":"2023-12-01T03:10:25.192862+0000","last_clean":"2023-12-01T03:10:25.192862+0000","last_became_active":"2023-12-01T03:10:24.700126+0000","last_became_peered":"2023-12-01T03:10:24.700126+0000","last_unstale":"2023-12-01T03:10:25.192862+0000","last_undegraded":"2023-12-01T03:10:25.192862+0000","last_fullsized":"2023-12-01T03:10:25.192862+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.927157+0000","last_change":"2023-12-01T03:10:25.175999+0000","last_active":"2023-12-01T03:10:26.927157+0000","last_peered":"2023-12-01T03:10:26.927157+0000","last_clean":"2023-12-01T03:10:26.927157+0000","last_became_active":"2023-12-01T03:10:25.175764+0000","last_became_peered":"2023-12-01T03:10:25.175764+0000","last_unstale":"2023-12-01T03:10:26.927157+0000","last_undegraded":"2023-12-01T03:10:26.927157+0000","last_fullsized":"2023-12-01T03:10:26.927157+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:43.098776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"213'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.184131+0000","last_change":"2023-12-01T03:10:24.686931+0000","last_active":"2023-12-01T03:10:25.184131+0000","last_peered":"2023-12-01T03:10:25.184131+0000","last_clean":"2023-12-01T03:10:25.184131+0000","last_became_active":"2023-12-01T03:10:24.686642+0000","last_became_peered":"2023-12-01T03:10:24.686642+0000","last_unstale":"2023-12-01T03:10:25.184131+0000","last_undegraded":"2023-12-01T03:10:25.184131+0000","last_fullsized":"2023-12-01T03:10:25.184131+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.170594+0000","last_change":"2023-12-01T03:10:25.175874+0000","last_active":"2023-12-01T03:10:26.170594+0000","last_peered":"2023-12-01T03:10:26.170594+0000","last_clean":"2023-12-01T03:10:26.170594+0000","last_became_active":"2023-12-01T03:10:25.175584+0000","last_became_peered":"2023-12-01T03:10:25.175584+0000","last_unstale":"2023-12-01T03:10:26.170594+0000","last_undegraded":"2023-12-01T03:10:26.170594+0000","last_fullsized":"2023-12-01T03:10:26.170594+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:46.157610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"213'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.193084+0000","last_change":"2023-12-01T03:10:24.695067+0000","last_active":"2023-12-01T03:10:25.193084+0000","last_peered":"2023-12-01T03:10:25.193084+0000","last_clean":"2023-12-01T03:10:25.193084+0000","last_became_active":"2023-12-01T03:10:24.694560+0000","last_became_peered":"2023-12-01T03:10:24.694560+0000","last_unstale":"2023-12-01T03:10:25.193084+0000","last_undegraded":"2023-12-01T03:10:25.193084+0000","last_fullsized":"2023-12-01T03:10:25.193084+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.169212+0000","last_change":"2023-12-01T03:10:25.188350+0000","last_active":"2023-12-01T03:10:26.169212+0000","last_peered":"2023-12-01T03:10:26.169212+0000","last_clean":"2023-12-01T03:10:26.169212+0000","last_became_active":"2023-12-01T03:10:25.187212+0000","last_became_peered":"2023-12-01T03:10:25.187212+0000","last_unstale":"2023-12-01T03:10:26.169212+0000","last_undegraded":"2023-12-01T03:10:26.169212+0000","last_fullsized":"2023-12-01T03:10:26.169212+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"213'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.194545+0000","last_change":"2023-12-01T03:10:24.695315+0000","last_active":"2023-12-01T03:10:25.194545+0000","last_peered":"2023-12-01T03:10:25.194545+0000","last_clean":"2023-12-01T03:10:25.194545+0000","last_became_active":"2023-12-01T03:10:24.694274+0000","last_became_peered":"2023-12-01T03:10:24.694274+0000","last_unstale":"2023-12-01T03:10:25.194545+0000","last_undegraded":"2023-12-01T03:10:25.194545+0000","last_fullsized":"2023-12-01T03:10:25.194545+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:19.256208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.132262+0000","last_change":"2023-12-01T03:10:25.182793+0000","last_active":"2023-12-01T03:10:27.132262+0000","last_peered":"2023-12-01T03:10:27.132262+0000","last_clean":"2023-12-01T03:10:27.132262+0000","last_became_active":"2023-12-01T03:10:25.182524+0000","last_became_peered":"2023-12-01T03:10:25.182524+0000","last_unstale":"2023-12-01T03:10:27.132262+0000","last_undegraded":"2023-12-01T03:10:27.132262+0000","last_fullsized":"2023-12-01T03:10:27.132262+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:25.370249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.176883+0000","last_change":"2023-12-01T03:10:24.686553+0000","last_active":"2023-12-01T03:10:25.176883+0000","last_peered":"2023-12-01T03:10:25.176883+0000","last_clean":"2023-12-01T03:10:25.176883+0000","last_became_active":"2023-12-01T03:10:24.686136+0000","last_became_peered":"2023-12-01T03:10:24.686136+0000","last_unstale":"2023-12-01T03:10:25.176883+0000","last_undegraded":"2023-12-01T03:10:25.176883+0000","last_fullsized":"2023-12-01T03:10:25.176883+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:18.663097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.166591+0000","last_change":"2023-12-01T03:10:25.176941+0000","last_active":"2023-12-01T03:10:26.166591+0000","last_peered":"2023-12-01T03:10:26.166591+0000","last_clean":"2023-12-01T03:10:26.166591+0000","last_became_active":"2023-12-01T03:10:25.175692+0000","last_became_peered":"2023-12-01T03:10:25.175692+0000","last_unstale":"2023-12-01T03:10:26.166591+0000","last_undegraded":"2023-12-01T03:10:26.166591+0000","last_fullsized":"2023-12-01T03:10:26.166591+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.162769+0000","last_change":"2023-12-01T03:10:24.687709+0000","last_active":"2023-12-01T03:10:25.162769+0000","last_peered":"2023-12-01T03:10:25.162769+0000","last_clean":"2023-12-01T03:10:25.162769+0000","last_became_active":"2023-12-01T03:10:24.687452+0000","last_became_peered":"2023-12-01T03:10:24.687452+0000","last_unstale":"2023-12-01T03:10:25.162769+0000","last_undegraded":"2023-12-01T03:10:25.162769+0000","last_fullsized":"2023-12-01T03:10:25.162769+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:39.126749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133880+0000","last_change":"2023-12-01T03:10:25.178648+0000","last_active":"2023-12-01T03:10:27.133880+0000","last_peered":"2023-12-01T03:10:27.133880+0000","last_clean":"2023-12-01T03:10:27.133880+0000","last_became_active":"2023-12-01T03:10:25.178352+0000","last_became_peered":"2023-12-01T03:10:25.178352+0000","last_unstale":"2023-12-01T03:10:27.133880+0000","last_undegraded":"2023-12-01T03:10:27.133880+0000","last_fullsized":"2023-12-01T03:10:27.133880+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.164647+0000","last_change":"2023-12-01T03:10:24.691537+0000","last_active":"2023-12-01T03:10:25.164647+0000","last_peered":"2023-12-01T03:10:25.164647+0000","last_clean":"2023-12-01T03:10:25.164647+0000","last_became_active":"2023-12-01T03:10:24.691168+0000","last_became_peered":"2023-12-01T03:10:24.691168+0000","last_unstale":"2023-12-01T03:10:25.164647+0000","last_undegraded":"2023-12-01T03:10:25.164647+0000","last_fullsized":"2023-12-01T03:10:25.164647+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.533268+0000","last_change":"2023-12-01T03:10:25.187487+0000","last_active":"2023-12-01T03:10:26.533268+0000","last_peered":"2023-12-01T03:10:26.533268+0000","last_clean":"2023-12-01T03:10:26.533268+0000","last_became_active":"2023-12-01T03:10:25.187198+0000","last_became_peered":"2023-12-01T03:10:25.187198+0000","last_unstale":"2023-12-01T03:10:26.533268+0000","last_undegraded":"2023-12-01T03:10:26.533268+0000","last_fullsized":"2023-12-01T03:10:26.533268+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:00.948964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"213'3","reported_seq":14,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.196079+0000","last_change":"2023-12-01T03:10:24.699528+0000","last_active":"2023-12-01T03:10:25.196079+0000","last_peered":"2023-12-01T03:10:25.196079+0000","last_clean":"2023-12-01T03:10:25.196079+0000","last_became_active":"2023-12-01T03:10:24.699152+0000","last_became_peered":"2023-12-01T03:10:24.699152+0000","last_unstale":"2023-12-01T03:10:25.196079+0000","last_undegraded":"2023-12-01T03:10:25.196079+0000","last_fullsized":"2023-12-01T03:10:25.196079+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:07.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133721+0000","last_change":"2023-12-01T03:10:25.176864+0000","last_active":"2023-12-01T03:10:27.133721+0000","last_peered":"2023-12-01T03:10:27.133721+0000","last_clean":"2023-12-01T03:10:27.133721+0000","last_became_active":"2023-12-01T03:10:25.176588+0000","last_became_peered":"2023-12-01T03:10:25.176588+0000","last_unstale":"2023-12-01T03:10:27.133721+0000","last_undegraded":"2023-12-01T03:10:27.133721+0000","last_fullsized":"2023-12-01T03:10:27.133721+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"213'6","reported_seq":17,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.196958+0000","last_change":"2023-12-01T03:10:24.697328+0000","last_active":"2023-12-01T03:10:25.196958+0000","last_peered":"2023-12-01T03:10:25.196958+0000","last_clean":"2023-12-01T03:10:25.196958+0000","last_became_active":"2023-12-01T03:10:24.696986+0000","last_became_peered":"2023-12-01T03:10:24.696986+0000","last_unstale":"2023-12-01T03:10:25.196958+0000","last_undegraded":"2023-12-01T03:10:25.196958+0000","last_fullsized":"2023-12-01T03:10:25.196958+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133261+0000","last_change":"2023-12-01T03:10:25.186716+0000","last_active":"2023-12-01T03:10:27.133261+0000","last_peered":"2023-12-01T03:10:27.133261+0000","last_clean":"2023-12-01T03:10:27.133261+0000","last_became_active":"2023-12-01T03:10:25.186422+0000","last_became_peered":"2023-12-01T03:10:25.186422+0000","last_unstale":"2023-12-01T03:10:27.133261+0000","last_undegraded":"2023-12-01T03:10:27.133261+0000","last_fullsized":"2023-12-01T03:10:27.133261+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"213'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.183586+0000","last_change":"2023-12-01T03:10:24.692854+0000","last_active":"2023-12-01T03:10:25.183586+0000","last_peered":"2023-12-01T03:10:25.183586+0000","last_clean":"2023-12-01T03:10:25.183586+0000","last_became_active":"2023-12-01T03:10:24.692563+0000","last_became_peered":"2023-12-01T03:10:24.692563+0000","last_unstale":"2023-12-01T03:10:25.183586+0000","last_undegraded":"2023-12-01T03:10:25.183586+0000","last_fullsized":"2023-12-01T03:10:25.183586+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133143+0000","last_change":"2023-12-01T03:10:25.183804+0000","last_active":"2023-12-01T03:10:27.133143+0000","last_peered":"2023-12-01T03:10:27.133143+0000","last_clean":"2023-12-01T03:10:27.133143+0000","last_became_active":"2023-12-01T03:10:25.183079+0000","last_became_peered":"2023-12-01T03:10:25.183079+0000","last_unstale":"2023-12-01T03:10:27.133143+0000","last_undegraded":"2023-12-01T03:10:27.133143+0000","last_fullsized":"2023-12-01T03:10:27.133143+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.168010+0000","last_change":"2023-12-01T03:10:24.685823+0000","last_active":"2023-12-01T03:10:25.168010+0000","last_peered":"2023-12-01T03:10:25.168010+0000","last_clean":"2023-12-01T03:10:25.168010+0000","last_became_active":"2023-12-01T03:10:24.685560+0000","last_became_peered":"2023-12-01T03:10:24.685560+0000","last_unstale":"2023-12-01T03:10:25.168010+0000","last_undegraded":"2023-12-01T03:10:25.168010+0000","last_fullsized":"2023-12-01T03:10:25.168010+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.532374+0000","last_change":"2023-12-01T03:10:25.164054+0000","last_active":"2023-12-01T03:10:26.532374+0000","last_peered":"2023-12-01T03:10:26.532374+0000","last_clean":"2023-12-01T03:10:26.532374+0000","last_became_active":"2023-12-01T03:10:25.163787+0000","last_became_peered":"2023-12-01T03:10:25.163787+0000","last_unstale":"2023-12-01T03:10:26.532374+0000","last_undegraded":"2023-12-01T03:10:26.532374+0000","last_fullsized":"2023-12-01T03:10:26.532374+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:04.568421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.173391+0000","last_change":"2023-12-01T03:10:24.694942+0000","last_active":"2023-12-01T03:10:25.173391+0000","last_peered":"2023-12-01T03:10:25.173391+0000","last_clean":"2023-12-01T03:10:25.173391+0000","last_became_active":"2023-12-01T03:10:24.694664+0000","last_became_peered":"2023-12-01T03:10:24.694664+0000","last_unstale":"2023-12-01T03:10:25.173391+0000","last_undegraded":"2023-12-01T03:10:25.173391+0000","last_fullsized":"2023-12-01T03:10:25.173391+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:38:02.332793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.168680+0000","last_change":"2023-12-01T03:10:25.193021+0000","last_active":"2023-12-01T03:10:26.168680+0000","last_peered":"2023-12-01T03:10:26.168680+0000","last_clean":"2023-12-01T03:10:26.168680+0000","last_became_active":"2023-12-01T03:10:25.192051+0000","last_became_peered":"2023-12-01T03:10:25.192051+0000","last_unstale":"2023-12-01T03:10:26.168680+0000","last_undegraded":"2023-12-01T03:10:26.168680+0000","last_fullsized":"2023-12-01T03:10:26.168680+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"213'8","reported_seq":19,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.191664+0000","last_change":"2023-12-01T03:10:24.685759+0000","last_active":"2023-12-01T03:10:25.191664+0000","last_peered":"2023-12-01T03:10:25.191664+0000","last_clean":"2023-12-01T03:10:25.191664+0000","last_became_active":"2023-12-01T03:10:24.684800+0000","last_became_peered":"2023-12-01T03:10:24.684800+0000","last_unstale":"2023-12-01T03:10:25.191664+0000","last_undegraded":"2023-12-01T03:10:25.191664+0000","last_fullsized":"2023-12-01T03:10:25.191664+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:29.129070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133723+0000","last_change":"2023-12-01T03:10:25.181692+0000","last_active":"2023-12-01T03:10:27.133723+0000","last_peered":"2023-12-01T03:10:27.133723+0000","last_clean":"2023-12-01T03:10:27.133723+0000","last_became_active":"2023-12-01T03:10:25.181316+0000","last_became_peered":"2023-12-01T03:10:25.181316+0000","last_unstale":"2023-12-01T03:10:27.133723+0000","last_undegraded":"2023-12-01T03:10:27.133723+0000","last_fullsized":"2023-12-01T03:10:27.133723+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:57:07.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"213'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.192165+0000","last_change":"2023-12-01T03:10:24.699945+0000","last_active":"2023-12-01T03:10:25.192165+0000","last_peered":"2023-12-01T03:10:25.192165+0000","last_clean":"2023-12-01T03:10:25.192165+0000","last_became_active":"2023-12-01T03:10:24.699598+0000","last_became_peered":"2023-12-01T03:10:24.699598+0000","last_unstale":"2023-12-01T03:10:25.192165+0000","last_undegraded":"2023-12-01T03:10:25.192165+0000","last_fullsized":"2023-12-01T03:10:25.192165+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.533362+0000","last_change":"2023-12-01T03:10:25.180913+0000","last_active":"2023-12-01T03:10:26.533362+0000","last_peered":"2023-12-01T03:10:26.533362+0000","last_clean":"2023-12-01T03:10:26.533362+0000","last_became_active":"2023-12-01T03:10:25.180626+0000","last_became_peered":"2023-12-01T03:10:25.180626+0000","last_unstale":"2023-12-01T03:10:26.533362+0000","last_undegraded":"2023-12-01T03:10:26.533362+0000","last_fullsized":"2023-12-01T03:10:26.533362+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"213'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.177507+0000","last_change":"2023-12-01T03:10:24.688107+0000","last_active":"2023-12-01T03:10:25.177507+0000","last_peered":"2023-12-01T03:10:25.177507+0000","last_clean":"2023-12-01T03:10:25.177507+0000","last_became_active":"2023-12-01T03:10:24.686930+0000","last_became_peered":"2023-12-01T03:10:24.686930+0000","last_unstale":"2023-12-01T03:10:25.177507+0000","last_undegraded":"2023-12-01T03:10:25.177507+0000","last_fullsized":"2023-12-01T03:10:25.177507+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:44.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.927320+0000","last_change":"2023-12-01T03:10:25.178959+0000","last_active":"2023-12-01T03:10:26.927320+0000","last_peered":"2023-12-01T03:10:26.927320+0000","last_clean":"2023-12-01T03:10:26.927320+0000","last_became_active":"2023-12-01T03:10:25.178268+0000","last_became_peered":"2023-12-01T03:10:25.178268+0000","last_unstale":"2023-12-01T03:10:26.927320+0000","last_undegraded":"2023-12-01T03:10:26.927320+0000","last_fullsized":"2023-12-01T03:10:26.927320+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:55:31.814432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"213'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.186992+0000","last_change":"2023-12-01T03:10:24.691398+0000","last_active":"2023-12-01T03:10:25.186992+0000","last_peered":"2023-12-01T03:10:25.186992+0000","last_clean":"2023-12-01T03:10:25.186992+0000","last_became_active":"2023-12-01T03:10:24.691110+0000","last_became_peered":"2023-12-01T03:10:24.691110+0000","last_unstale":"2023-12-01T03:10:25.186992+0000","last_undegraded":"2023-12-01T03:10:25.186992+0000","last_fullsized":"2023-12-01T03:10:25.186992+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:24.670899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.533051+0000","last_change":"2023-12-01T03:10:25.172837+0000","last_active":"2023-12-01T03:10:26.533051+0000","last_peered":"2023-12-01T03:10:26.533051+0000","last_clean":"2023-12-01T03:10:26.533051+0000","last_became_active":"2023-12-01T03:10:25.172602+0000","last_became_peered":"2023-12-01T03:10:25.172602+0000","last_unstale":"2023-12-01T03:10:26.533051+0000","last_undegraded":"2023-12-01T03:10:26.533051+0000","last_fullsized":"2023-12-01T03:10:26.533051+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.160900+0000","last_change":"2023-12-01T03:10:24.683594+0000","last_active":"2023-12-01T03:10:25.160900+0000","last_peered":"2023-12-01T03:10:25.160900+0000","last_clean":"2023-12-01T03:10:25.160900+0000","last_became_active":"2023-12-01T03:10:24.679979+0000","last_became_peered":"2023-12-01T03:10:24.679979+0000","last_unstale":"2023-12-01T03:10:25.160900+0000","last_undegraded":"2023-12-01T03:10:25.160900+0000","last_fullsized":"2023-12-01T03:10:25.160900+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:12.511044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.170771+0000","last_change":"2023-12-01T03:10:25.169678+0000","last_active":"2023-12-01T03:10:26.170771+0000","last_peered":"2023-12-01T03:10:26.170771+0000","last_clean":"2023-12-01T03:10:26.170771+0000","last_became_active":"2023-12-01T03:10:25.168714+0000","last_became_peered":"2023-12-01T03:10:25.168714+0000","last_unstale":"2023-12-01T03:10:26.170771+0000","last_undegraded":"2023-12-01T03:10:26.170771+0000","last_fullsized":"2023-12-01T03:10:26.170771+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"213'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.193172+0000","last_change":"2023-12-01T03:10:24.695154+0000","last_active":"2023-12-01T03:10:25.193172+0000","last_peered":"2023-12-01T03:10:25.193172+0000","last_clean":"2023-12-01T03:10:25.193172+0000","last_became_active":"2023-12-01T03:10:24.694868+0000","last_became_peered":"2023-12-01T03:10:24.694868+0000","last_unstale":"2023-12-01T03:10:25.193172+0000","last_undegraded":"2023-12-01T03:10:25.193172+0000","last_fullsized":"2023-12-01T03:10:25.193172+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133431+0000","last_change":"2023-12-01T03:10:25.180768+0000","last_active":"2023-12-01T03:10:27.133431+0000","last_peered":"2023-12-01T03:10:27.133431+0000","last_clean":"2023-12-01T03:10:27.133431+0000","last_became_active":"2023-12-01T03:10:25.179796+0000","last_became_peered":"2023-12-01T03:10:25.179796+0000","last_unstale":"2023-12-01T03:10:27.133431+0000","last_undegraded":"2023-12-01T03:10:27.133431+0000","last_fullsized":"2023-12-01T03:10:27.133431+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.157201+0000","last_change":"2023-12-01T03:10:24.685775+0000","last_active":"2023-12-01T03:10:25.157201+0000","last_peered":"2023-12-01T03:10:25.157201+0000","last_clean":"2023-12-01T03:10:25.157201+0000","last_became_active":"2023-12-01T03:10:24.685178+0000","last_became_peered":"2023-12-01T03:10:24.685178+0000","last_unstale":"2023-12-01T03:10:25.157201+0000","last_undegraded":"2023-12-01T03:10:25.157201+0000","last_fullsized":"2023-12-01T03:10:25.157201+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:21.942226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.131901+0000","last_change":"2023-12-01T03:10:25.173728+0000","last_active":"2023-12-01T03:10:27.131901+0000","last_peered":"2023-12-01T03:10:27.131901+0000","last_clean":"2023-12-01T03:10:27.131901+0000","last_became_active":"2023-12-01T03:10:25.173449+0000","last_became_peered":"2023-12-01T03:10:25.173449+0000","last_unstale":"2023-12-01T03:10:27.131901+0000","last_undegraded":"2023-12-01T03:10:27.131901+0000","last_fullsized":"2023-12-01T03:10:27.131901+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:20.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.157045+0000","last_change":"2023-12-01T03:10:24.694594+0000","last_active":"2023-12-01T03:10:25.157045+0000","last_peered":"2023-12-01T03:10:25.157045+0000","last_clean":"2023-12-01T03:10:25.157045+0000","last_became_active":"2023-12-01T03:10:24.693808+0000","last_became_peered":"2023-12-01T03:10:24.693808+0000","last_unstale":"2023-12-01T03:10:25.157045+0000","last_undegraded":"2023-12-01T03:10:25.157045+0000","last_fullsized":"2023-12-01T03:10:25.157045+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.133063+0000","last_change":"2023-12-01T03:10:25.174479+0000","last_active":"2023-12-01T03:10:27.133063+0000","last_peered":"2023-12-01T03:10:27.133063+0000","last_clean":"2023-12-01T03:10:27.133063+0000","last_became_active":"2023-12-01T03:10:25.174194+0000","last_became_peered":"2023-12-01T03:10:25.174194+0000","last_unstale":"2023-12-01T03:10:27.133063+0000","last_undegraded":"2023-12-01T03:10:27.133063+0000","last_fullsized":"2023-12-01T03:10:27.133063+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.162169+0000","last_change":"2023-12-01T03:10:24.685003+0000","last_active":"2023-12-01T03:10:25.162169+0000","last_peered":"2023-12-01T03:10:25.162169+0000","last_clean":"2023-12-01T03:10:25.162169+0000","last_became_active":"2023-12-01T03:10:24.684425+0000","last_became_peered":"2023-12-01T03:10:24.684425+0000","last_unstale":"2023-12-01T03:10:25.162169+0000","last_undegraded":"2023-12-01T03:10:25.162169+0000","last_fullsized":"2023-12-01T03:10:25.162169+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:45.173492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.166592+0000","last_change":"2023-12-01T03:10:25.184992+0000","last_active":"2023-12-01T03:10:26.166592+0000","last_peered":"2023-12-01T03:10:26.166592+0000","last_clean":"2023-12-01T03:10:26.166592+0000","last_became_active":"2023-12-01T03:10:25.184670+0000","last_became_peered":"2023-12-01T03:10:25.184670+0000","last_unstale":"2023-12-01T03:10:26.166592+0000","last_undegraded":"2023-12-01T03:10:26.166592+0000","last_fullsized":"2023-12-01T03:10:26.166592+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:40.143822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"213'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.176809+0000","last_change":"2023-12-01T03:10:24.687160+0000","last_active":"2023-12-01T03:10:25.176809+0000","last_peered":"2023-12-01T03:10:25.176809+0000","last_clean":"2023-12-01T03:10:25.176809+0000","last_became_active":"2023-12-01T03:10:24.686184+0000","last_became_peered":"2023-12-01T03:10:24.686184+0000","last_unstale":"2023-12-01T03:10:25.176809+0000","last_undegraded":"2023-12-01T03:10:25.176809+0000","last_fullsized":"2023-12-01T03:10:25.176809+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:05.006954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.166301+0000","last_change":"2023-12-01T03:10:25.168359+0000","last_active":"2023-12-01T03:10:26.166301+0000","last_peered":"2023-12-01T03:10:26.166301+0000","last_clean":"2023-12-01T03:10:26.166301+0000","last_became_active":"2023-12-01T03:10:25.168082+0000","last_became_peered":"2023-12-01T03:10:25.168082+0000","last_unstale":"2023-12-01T03:10:26.166301+0000","last_undegraded":"2023-12-01T03:10:26.166301+0000","last_fullsized":"2023-12-01T03:10:26.166301+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:06.024028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.164216+0000","last_change":"2023-12-01T03:10:24.697890+0000","last_active":"2023-12-01T03:10:25.164216+0000","last_peered":"2023-12-01T03:10:25.164216+0000","last_clean":"2023-12-01T03:10:25.164216+0000","last_became_active":"2023-12-01T03:10:24.697679+0000","last_became_peered":"2023-12-01T03:10:24.697679+0000","last_unstale":"2023-12-01T03:10:25.164216+0000","last_undegraded":"2023-12-01T03:10:25.164216+0000","last_fullsized":"2023-12-01T03:10:25.164216+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.166028+0000","last_change":"2023-12-01T03:10:25.175476+0000","last_active":"2023-12-01T03:10:26.166028+0000","last_peered":"2023-12-01T03:10:26.166028+0000","last_clean":"2023-12-01T03:10:26.166028+0000","last_became_active":"2023-12-01T03:10:25.175211+0000","last_became_peered":"2023-12-01T03:10:25.175211+0000","last_unstale":"2023-12-01T03:10:26.166028+0000","last_undegraded":"2023-12-01T03:10:26.166028+0000","last_fullsized":"2023-12-01T03:10:26.166028+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:22.358205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.163443+0000","last_change":"2023-12-01T03:10:24.697992+0000","last_active":"2023-12-01T03:10:25.163443+0000","last_peered":"2023-12-01T03:10:25.163443+0000","last_clean":"2023-12-01T03:10:25.163443+0000","last_became_active":"2023-12-01T03:10:24.696883+0000","last_became_peered":"2023-12-01T03:10:24.696883+0000","last_unstale":"2023-12-01T03:10:25.163443+0000","last_undegraded":"2023-12-01T03:10:25.163443+0000","last_fullsized":"2023-12-01T03:10:25.163443+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.169225+0000","last_change":"2023-12-01T03:10:25.192028+0000","last_active":"2023-12-01T03:10:26.169225+0000","last_peered":"2023-12-01T03:10:26.169225+0000","last_clean":"2023-12-01T03:10:26.169225+0000","last_became_active":"2023-12-01T03:10:25.191747+0000","last_became_peered":"2023-12-01T03:10:25.191747+0000","last_unstale":"2023-12-01T03:10:26.169225+0000","last_undegraded":"2023-12-01T03:10:26.169225+0000","last_fullsized":"2023-12-01T03:10:26.169225+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.169342+0000","last_change":"2023-12-01T03:10:24.697595+0000","last_active":"2023-12-01T03:10:25.169342+0000","last_peered":"2023-12-01T03:10:25.169342+0000","last_clean":"2023-12-01T03:10:25.169342+0000","last_became_active":"2023-12-01T03:10:24.697311+0000","last_became_peered":"2023-12-01T03:10:24.697311+0000","last_unstale":"2023-12-01T03:10:25.169342+0000","last_undegraded":"2023-12-01T03:10:25.169342+0000","last_fullsized":"2023-12-01T03:10:25.169342+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:45.743456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.926457+0000","last_change":"2023-12-01T03:10:25.190210+0000","last_active":"2023-12-01T03:10:26.926457+0000","last_peered":"2023-12-01T03:10:26.926457+0000","last_clean":"2023-12-01T03:10:26.926457+0000","last_became_active":"2023-12-01T03:10:25.189957+0000","last_became_peered":"2023-12-01T03:10:25.189957+0000","last_unstale":"2023-12-01T03:10:26.926457+0000","last_undegraded":"2023-12-01T03:10:26.926457+0000","last_fullsized":"2023-12-01T03:10:26.926457+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:01.783625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.161249+0000","last_change":"2023-12-01T03:10:24.687326+0000","last_active":"2023-12-01T03:10:25.161249+0000","last_peered":"2023-12-01T03:10:25.161249+0000","last_clean":"2023-12-01T03:10:25.161249+0000","last_became_active":"2023-12-01T03:10:24.687036+0000","last_became_peered":"2023-12-01T03:10:24.687036+0000","last_unstale":"2023-12-01T03:10:25.161249+0000","last_undegraded":"2023-12-01T03:10:25.161249+0000","last_fullsized":"2023-12-01T03:10:25.161249+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:36:57.969605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:27.132564+0000","last_change":"2023-12-01T03:10:25.181396+0000","last_active":"2023-12-01T03:10:27.132564+0000","last_peered":"2023-12-01T03:10:27.132564+0000","last_clean":"2023-12-01T03:10:27.132564+0000","last_became_active":"2023-12-01T03:10:25.181134+0000","last_became_peered":"2023-12-01T03:10:25.181134+0000","last_unstale":"2023-12-01T03:10:27.132564+0000","last_undegraded":"2023-12-01T03:10:27.132564+0000","last_fullsized":"2023-12-01T03:10:27.132564+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:04.776013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.161397+0000","last_change":"2023-12-01T03:10:24.683240+0000","last_active":"2023-12-01T03:10:25.161397+0000","last_peered":"2023-12-01T03:10:25.161397+0000","last_clean":"2023-12-01T03:10:25.161397+0000","last_became_active":"2023-12-01T03:10:24.682933+0000","last_became_peered":"2023-12-01T03:10:24.682933+0000","last_unstale":"2023-12-01T03:10:25.161397+0000","last_undegraded":"2023-12-01T03:10:25.161397+0000","last_fullsized":"2023-12-01T03:10:25.161397+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:40.211424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.926462+0000","last_change":"2023-12-01T03:10:25.174085+0000","last_active":"2023-12-01T03:10:26.926462+0000","last_peered":"2023-12-01T03:10:26.926462+0000","last_clean":"2023-12-01T03:10:26.926462+0000","last_became_active":"2023-12-01T03:10:25.173802+0000","last_became_peered":"2023-12-01T03:10:25.173802+0000","last_unstale":"2023-12-01T03:10:26.926462+0000","last_undegraded":"2023-12-01T03:10:26.926462+0000","last_fullsized":"2023-12-01T03:10:26.926462+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:39.145914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.161307+0000","last_change":"2023-12-01T03:10:24.684248+0000","last_active":"2023-12-01T03:10:25.161307+0000","last_peered":"2023-12-01T03:10:25.161307+0000","last_clean":"2023-12-01T03:10:25.161307+0000","last_became_active":"2023-12-01T03:10:24.680714+0000","last_became_peered":"2023-12-01T03:10:24.680714+0000","last_unstale":"2023-12-01T03:10:25.161307+0000","last_undegraded":"2023-12-01T03:10:25.161307+0000","last_fullsized":"2023-12-01T03:10:25.161307+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:19:02.474558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.168657+0000","last_change":"2023-12-01T03:10:25.191828+0000","last_active":"2023-12-01T03:10:26.168657+0000","last_peered":"2023-12-01T03:10:26.168657+0000","last_clean":"2023-12-01T03:10:26.168657+0000","last_became_active":"2023-12-01T03:10:25.191436+0000","last_became_peered":"2023-12-01T03:10:25.191436+0000","last_unstale":"2023-12-01T03:10:26.168657+0000","last_undegraded":"2023-12-01T03:10:26.168657+0000","last_fullsized":"2023-12-01T03:10:26.168657+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.158178+0000","last_change":"2023-12-01T03:10:24.690155+0000","last_active":"2023-12-01T03:10:25.158178+0000","last_peered":"2023-12-01T03:10:25.158178+0000","last_clean":"2023-12-01T03:10:25.158178+0000","last_became_active":"2023-12-01T03:10:24.689122+0000","last_became_peered":"2023-12-01T03:10:24.689122+0000","last_unstale":"2023-12-01T03:10:25.158178+0000","last_undegraded":"2023-12-01T03:10:25.158178+0000","last_fullsized":"2023-12-01T03:10:25.158178+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-12-01T03:10:26.171004+0000","last_change":"2023-12-01T03:10:25.189213+0000","last_active":"2023-12-01T03:10:26.171004+0000","last_peered":"2023-12-01T03:10:26.171004+0000","last_clean":"2023-12-01T03:10:26.171004+0000","last_became_active":"2023-12-01T03:10:25.188927+0000","last_became_peered":"2023-12-01T03:10:25.188927+0000","last_unstale":"2023-12-01T03:10:26.171004+0000","last_undegraded":"2023-12-01T03:10:26.171004+0000","last_fullsized":"2023-12-01T03:10:26.171004+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:21:53.633800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.159009+0000","last_change":"2023-12-01T03:10:24.685430+0000","last_active":"2023-12-01T03:10:25.159009+0000","last_peered":"2023-12-01T03:10:25.159009+0000","last_clean":"2023-12-01T03:10:25.159009+0000","last_became_active":"2023-12-01T03:10:24.684919+0000","last_became_peered":"2023-12-01T03:10:24.684919+0000","last_unstale":"2023-12-01T03:10:25.159009+0000","last_undegraded":"2023-12-01T03:10:25.159009+0000","last_fullsized":"2023-12-01T03:10:25.159009+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"201'326","reported_seq":1118,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-01T03:10:25.160535+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:10:25.160535+0000","last_peered":"2023-12-01T03:10:25.160535+0000","last_clean":"2023-12-01T03:10:25.160535+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:10:25.160535+0000","last_undegraded":"2023-12-01T03:10:25.160535+0000","last_fullsized":"2023-12-01T03:10:25.160535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575977,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59924,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311916,"statfs":{"total":96636764160,"available":96575401984,"internally_reserved":0,"allocated":2699264,"data_stored":1306796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0780000000000001,"15min":1.0820000000000001},"min":{"1min":0.78200000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5069999999999999,"5min":1.6000000000000001,"15min":1.98},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":0.67800000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.556,"5min":1.7,"15min":1.7},"last":1.0269999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.2889999999999999},"min":{"1min":0.80000000000000004,"5min":0.745,"15min":0.73099999999999998},"max":{"1min":1.603,"5min":2,"15min":2.1419999999999999},"last":1.369},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3149999999999999,"15min":1.3320000000000001},"min":{"1min":1.137,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.514,"5min":1.8169999999999999,"15min":1.8700000000000001},"last":1.284}]},{"osd":2,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.157,"15min":1.1719999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.476,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.26,"15min":1.254},"min":{"1min":1.0549999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6020000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.2070000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2529999999999999,"15min":1.3049999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6559999999999999,"5min":1.712,"15min":2.431},"last":1.639},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.415,"15min":1.4199999999999999},"min":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.92900000000000005},"max":{"1min":1.5920000000000001,"5min":1.879,"15min":2.198},"last":1.5760000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3140000000000001,"15min":1.3400000000000001},"min":{"1min":1.006,"5min":0.82599999999999996,"15min":0.81200000000000006},"max":{"1min":1.901,"5min":1.901,"15min":2.5899999999999999},"last":1.407},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":1.1240000000000001,"5min":0.92800000000000005,"15min":0.79500000000000004},"max":{"1min":1.9690000000000001,"5min":2.3279999999999998,"15min":2.569},"last":1.839}]},{"osd":5,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3009999999999999,"15min":1.3149999999999999},"min":{"1min":0.93600000000000005,"5min":0.85999999999999999,"15min":0.84599999999999997},"max":{"1min":2.0099999999999998,"5min":2.016,"15min":2.7269999999999999},"last":1.698},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5800000000000001,"15min":1.589},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":0.93200000000000005},"max":{"1min":2.0569999999999999,"5min":2.206,"15min":2.484},"last":1.923}]},{"osd":6,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4359999999999999,"15min":1.446},"min":{"1min":1.0229999999999999,"5min":0.98599999999999999,"15min":0.82599999999999996},"max":{"1min":1.921,"5min":2.2789999999999999,"15min":2.532},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":1.8819999999999999,"15min":2.762},"last":1.752}]}]},{"osd":6,"up_from":13,"seq":55834575978,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41860,"kb_used_data":2620,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329980,"statfs":{"total":96636764160,"available":96593899520,"internally_reserved":0,"allocated":2682880,"data_stored":1305210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0589999999999999,"15min":1.113},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.4039999999999999,"5min":1.6639999999999999,"15min":1.9219999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1160000000000001,"15min":1.1659999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.623,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":0.85499999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.976,"5min":1.976,"15min":2.0449999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1950000000000001,"15min":1.2210000000000001},"min":{"1min":0.627,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.694,"5min":1.694,"15min":1.835},"last":1.214}]},{"osd":2,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.163,"15min":1.2090000000000001},"min":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.976},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2629999999999999,"15min":1.2849999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.0329999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2849999999999999,"15min":1.319},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8100000000000001,"5min":1.927,"15min":2.0289999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.405,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.3129999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2130000000000001,"15min":1.262},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.363},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.367,"15min":1.3540000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.397}]},{"osd":5,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3360000000000001,"15min":1.4039999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.282},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4790000000000001,"15min":1.484},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.4490000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4890000000000001,"15min":1.5249999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.556,"15min":1.579},"min":{"1min":0.95499999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2999999999999998,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.502}]}]},{"osd":5,"up_from":13,"seq":55834575977,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48204,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323636,"statfs":{"total":96636764160,"available":96587403264,"internally_reserved":0,"allocated":3280896,"data_stored":1896092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.125},"min":{"1min":0.82299999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.548,"5min":1.776,"15min":1.8899999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.238,"5min":1.1639999999999999,"15min":1.163},"min":{"1min":0.89800000000000002,"5min":0.69399999999999995,"15min":0.67300000000000004},"max":{"1min":1.619,"5min":1.752,"15min":1.8009999999999999},"last":1.0720000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.133,"15min":1.1539999999999999},"min":{"1min":0.94699999999999995,"5min":0.81200000000000006,"15min":0.79900000000000004},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.696},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.272,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.88,"5min":0.79500000000000004,"15min":0.60899999999999999},"max":{"1min":1.619,"5min":1.6659999999999999,"15min":1.7649999999999999},"last":1.1279999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2390000000000001,"15min":1.246},"min":{"1min":0.93700000000000006,"5min":0.84499999999999997,"15min":0.81399999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.774},"last":1.359},{"interface":"front","average":{"1min":1.284,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.93899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.502,"5min":1.899,"15min":1.9219999999999999},"last":1.335}]},{"osd":3,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.306,"15min":1.3200000000000001},"min":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.93100000000000005},"max":{"1min":1.7929999999999999,"5min":1.8979999999999999,"15min":1.994},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.343,"15min":1.363},"min":{"1min":1.02,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.724,"5min":1.97,"15min":2.1859999999999999},"last":1.268}]},{"osd":4,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2789999999999999,"15min":1.254},"min":{"1min":0.96199999999999997,"5min":0.78000000000000003,"15min":0.74099999999999999},"max":{"1min":1.726,"5min":1.726,"15min":2.6040000000000001},"last":1.236},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.399,"15min":1.385},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.84599999999999997},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.931},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3879999999999999,"15min":1.3759999999999999},"min":{"1min":0.97599999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.9279999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.173},{"interface":"front","average":{"1min":1.478,"5min":1.427,"15min":1.419},"min":{"1min":1.222,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":1.756,"5min":1.9219999999999999,"15min":2.577},"last":1.0149999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5349999999999999,"15min":1.5369999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.962,"5min":2.0790000000000002,"15min":3.1709999999999998},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.575,"15min":1.5740000000000001},"min":{"1min":1.3720000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.831,"5min":2.101,"15min":2.8740000000000001},"last":1.3779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575977,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47620,"kb_used_data":2620,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324220,"statfs":{"total":96636764160,"available":96588001280,"internally_reserved":0,"allocated":2682880,"data_stored":1305154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.147,"15min":1.1339999999999999},"min":{"1min":0.73099999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.21,"15min":1.1990000000000001},"min":{"1min":0.82099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5840000000000001,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.1339999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.242,"15min":1.2589999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.613,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.82599999999999996},"max":{"1min":1.6080000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.022}]},{"osd":2,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.3240000000000001,"15min":1.2470000000000001},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.573,"5min":11.311,"15min":11.311},"last":1.466},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.329,"15min":1.341},"min":{"1min":1.0680000000000001,"5min":1.024,"15min":0.94099999999999995},"max":{"1min":1.647,"5min":1.9379999999999999,"15min":1.9450000000000001},"last":1.3480000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3100000000000001,"15min":1.29},"min":{"1min":1.0529999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7749999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.23},{"interface":"front","average":{"1min":1.343,"5min":1.365,"15min":1.3380000000000001},"min":{"1min":0.99199999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.599,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.5349999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.357,"15min":1.329},"min":{"1min":0.94799999999999995,"5min":0.91200000000000003,"15min":0.755},"max":{"1min":1.615,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.419,"5min":1.379,"15min":1.3120000000000001},"min":{"1min":0.94599999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7150000000000001,"5min":3.71,"15min":3.71},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3740000000000001,"15min":1.375},"min":{"1min":1.095,"5min":0.745,"15min":0.745},"max":{"1min":1.609,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.514,"5min":1.4850000000000001,"15min":1.48},"min":{"1min":1.335,"5min":1.0660000000000001,"15min":1.0149999999999999},"max":{"1min":1.738,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.5820000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.52,"15min":1.486},"min":{"1min":1.157,"5min":0.95499999999999996,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.331},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6000000000000001,"15min":1.5800000000000001},"min":{"1min":1.3600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.806,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.6639999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575976,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51728,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320112,"statfs":{"total":96636764160,"available":96583794688,"internally_reserved":0,"allocated":2695168,"data_stored":1306754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91100000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.085},"min":{"1min":0.74199999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.571,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.4099999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.163,"15min":1.171},"min":{"1min":0.64700000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.008,"15min":0.999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41099999999999998},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":1.788}]},{"osd":2,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0900000000000001,"15min":1.103},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":1.224},{"interface":"front","average":{"1min":1.27,"5min":1.2230000000000001,"15min":1.252},"min":{"1min":0.80100000000000005,"5min":0.747,"15min":0.58599999999999997},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.264}]},{"osd":4,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2989999999999999,"15min":1.2929999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":1.367},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.3},"min":{"1min":0.96799999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.3169999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3240000000000001,"15min":1.331},"min":{"1min":0.86599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.373,"15min":1.3759999999999999},"min":{"1min":0.88400000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.633}]},{"osd":6,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.78400000000000003},"max":{"1min":2.5649999999999999,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.5049999999999999,"15min":1.514},"min":{"1min":1.075,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.73}]},{"osd":7,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.516,"15min":1.5329999999999999},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.601},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.54,"15min":1.5309999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.878},"max":{"1min":2.4830000000000001,"5min":2.984,"15min":2.984},"last":1.681}]}]},{"osd":2,"up_from":13,"seq":55834575976,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59908,"kb_used_data":2620,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311932,"statfs":{"total":96636764160,"available":96575418368,"internally_reserved":0,"allocated":2682880,"data_stored":1305190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89300000000000002,"15min":0.89400000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1140000000000001,"5min":1.405,"15min":1.4730000000000001},"last":1.411},{"interface":"front","average":{"1min":1.054,"5min":1.056,"15min":1.0569999999999999},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5640000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.80200000000000005}]},{"osd":1,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0449999999999999,"15min":1.0589999999999999},"min":{"1min":0.66100000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.321,"5min":1.698,"15min":1.698},"last":0.877},{"interface":"front","average":{"1min":1.038,"5min":1.125,"15min":1.151},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.522,"5min":1.635,"15min":1.71},"last":1.04}]},{"osd":3,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0640000000000001,"15min":1.054},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.419,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1990000000000001,"15min":1.2170000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.516,"5min":1.6879999999999999,"15min":1.79},"last":1.169}]},{"osd":4,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":0.85199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.7569999999999999,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.714,"5min":1.714,"15min":1.835},"last":1.075}]},{"osd":5,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3919999999999999,"15min":1.391},"min":{"1min":0.88,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.8380000000000001,"15min":2.2040000000000002},"last":1.575},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.419,"15min":1.4179999999999999},"min":{"1min":1.095,"5min":1.004,"15min":1.004},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.016},"last":1.456}]},{"osd":6,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.379,"15min":1.3819999999999999},"min":{"1min":0.89600000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.502},{"interface":"front","average":{"1min":1.46,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":1.087,"5min":0.879,"15min":0.879},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.6799999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5069999999999999,"15min":1.4910000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0979999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4330000000000001,"15min":1.458},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.012,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.6100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575975,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48196,"kb_used_data":3196,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323644,"statfs":{"total":96636764160,"available":96587411456,"internally_reserved":0,"allocated":3272704,"data_stored":1895488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.879,"15min":0.90900000000000003},"min":{"1min":0.58199999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.099,"5min":0.99299999999999999,"15min":1.046},"min":{"1min":0.81000000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.5069999999999999,"5min":1.5660000000000001,"15min":1.681},"last":1.248}]},{"osd":2,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.56299999999999994,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.747},{"interface":"front","average":{"1min":1.147,"5min":1.115,"15min":1.1419999999999999},"min":{"1min":0.75600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.581,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":1.1899999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.258,"15min":1.3069999999999999},"min":{"1min":1.026,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5780000000000001,"5min":1.587,"15min":1.788},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.107,"15min":1.135},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.714,"5min":1.714,"15min":1.8049999999999999},"last":0.95699999999999996}]},{"osd":4,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.21,"15min":1.2230000000000001},"min":{"1min":0.89900000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6859999999999999},"last":1.024},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.389,"15min":1.409},"min":{"1min":1.0740000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.4219999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.85899999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.802,"5min":1.8520000000000001,"15min":2.101},"last":1.649},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.304,"15min":1.339},"min":{"1min":1.006,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.609,"5min":1.927,"15min":2.1659999999999999},"last":1.3220000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4019999999999999,"15min":1.4379999999999999},"min":{"1min":0.99399999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3799999999999999,"15min":1.4119999999999999},"min":{"1min":1.1779999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.768,"5min":2.0019999999999998,"15min":2.0209999999999999},"last":1.6319999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4870000000000001,"15min":1.5269999999999999},"min":{"1min":1.2549999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.911,"5min":2.0329999999999999,"15min":2.1190000000000002},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.524,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.5549999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575977,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59912,"kb_used_data":2624,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311928,"statfs":{"total":96636764160,"available":96575414272,"internally_reserved":0,"allocated":2686976,"data_stored":1305736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.86699999999999999,"15min":0.89400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1559999999999999,"5min":1.4379999999999999,"15min":1.994},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.04,"15min":1.056},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.427,"5min":1.583,"15min":1.6799999999999999},"last":0.82899999999999996}]},{"osd":2,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0329999999999999,"15min":1.069},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.446,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.133,"5min":1.1020000000000001,"15min":1.1419999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.726,"5min":1.726,"15min":1.792},"last":1.1100000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0580000000000001,"15min":1.071},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.8839999999999999},"last":1.274},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.655,"5min":1.655,"15min":2.6469999999999998},"last":1.2250000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3260000000000001,"15min":1.363},"min":{"1min":0.88400000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.284,"15min":1.3089999999999999},"min":{"1min":0.82099999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.677,"5min":1.74,"15min":2.3610000000000002},"last":1.4359999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.355,"15min":1.397},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5589999999999999,"5min":2.1030000000000002,"15min":2.7719999999999998},"last":1.859},{"interface":"front","average":{"1min":1.431,"5min":1.4099999999999999,"15min":1.4410000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.79600000000000004},"max":{"1min":1.9259999999999999,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":1.681}]},{"osd":6,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.367,"15min":1.3859999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7649999999999999,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.458,"15min":1.484},"min":{"1min":1.048,"5min":0.85299999999999998,"15min":0.75800000000000001},"max":{"1min":1.736,"5min":1.9870000000000001,"15min":2.8759999999999999},"last":2.0569999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.49,"15min":1.524},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.841,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":1.7969999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4750000000000001,"15min":1.4910000000000001},"min":{"1min":0.97299999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.0619999999999998,"5min":2.1190000000000002,"15min":2.431},"last":1.8999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:10:30.328 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T03:10:30.328 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T03:10:30.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.557+0000 7ff1981d2640 1 Processor -- start 2023-12-01T03:10:30.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.558+0000 7ff1981d2640 1 -- start start 2023-12-01T03:10:30.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.559+0000 7ff1981d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900ca690 0x7ff1900ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.559+0000 7ff1981d2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff190066500 con 0x7ff1901604b0 2023-12-01T03:10:30.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.559+0000 7ff1981d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19006ea80 con 0x7ff1900712e0 2023-12-01T03:10:30.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.559+0000 7ff1981d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19006e7a0 con 0x7ff1900ca690 2023-12-01T03:10:30.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.559+0000 7ff195f6e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff1901604b0 0x7ff1900c9f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37230/0 (socket says 172.21.15.115:37230) 2023-12-01T03:10:30.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.559+0000 7ff195f6e640 1 -- 172.21.15.115:0/3835761350 learned_addr learned my addr 172.21.15.115:0/3835761350 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:30.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.560+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745277800 0 0) 0x7ff190066500 con 0x7ff1901604b0 2023-12-01T03:10:30.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.560+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff17c003520 con 0x7ff1901604b0 2023-12-01T03:10:30.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.560+0000 7ff19676f640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900ca690 0x7ff1900ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.560+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4143009842 0 0) 0x7ff17c003520 con 0x7ff1901604b0 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.560+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900ca690 msgr2=0x7ff1900ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.560+0000 7ff194f6c640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900ca690 0x7ff1900ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.560+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 >> v1:172.21.15.142:6789/0 conn(0x7ff1900712e0 legacy=0x7ff1900ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.560+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900cd5a0 con 0x7ff1901604b0 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.560+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7ff184001670 con 0x7ff1901604b0 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.560+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 >> v1:172.21.15.115:6789/0 conn(0x7ff1901604b0 legacy=0x7ff1900c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff194f6c640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900ca690 0x7ff17c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff194f6c640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c003f70 0x7ff17c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff194f6c640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 0x7ff17c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19006e7a0 con 0x7ff17c004d00 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c00bb70 con 0x7ff17c003f70 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c00bd30 con 0x7ff1900ca690 2023-12-01T03:10:30.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff19676f640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 0x7ff17c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff195f6e640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900ca690 0x7ff17c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff19576d640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c003f70 0x7ff17c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff19676f640 1 -- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900ca690 msgr2=0x7ff17c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:30.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff19676f640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900ca690 0x7ff17c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff19676f640 1 -- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c003f70 msgr2=0x7ff17c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:30.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff19676f640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c003f70 0x7ff17c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff19676f640 1 -- 172.21.15.115:0/3835761350 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c003520 con 0x7ff17c004d00 2023-12-01T03:10:30.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff195f6e640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900ca690 0x7ff17c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:10:30.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.561+0000 7ff19576d640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c003f70 0x7ff17c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.562+0000 7ff19676f640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 0x7ff17c005180 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7ff190071790 tx=0x7ff17c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7a05fb36583cbf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.563+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18802cb90 con 0x7ff17c004d00 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.563+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18800f040 con 0x7ff17c004d00 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.563+0000 7ff194f6c640 1 -- 172.21.15.115:0/3835761350 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188034440 con 0x7ff17c004d00 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.563+0000 7ff1981d2640 1 -- 172.21.15.115:0/3835761350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 msgr2=0x7ff17c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.563+0000 7ff1981d2640 1 --2- 172.21.15.115:0/3835761350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 0x7ff17c005180 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7ff190071790 tx=0x7ff17c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.563+0000 7ff19676f640 1 -- 172.21.15.115:0/3835761350 reap_dead start 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.563+0000 7ff1981d2640 1 -- 172.21.15.115:0/3835761350 shutdown_connections 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.563+0000 7ff1981d2640 1 -- 172.21.15.115:0/3835761350 >> 172.21.15.115:0/3835761350 conn(0x7ff1900b1590 msgr2=0x7ff1900b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.563+0000 7ff1981d2640 1 -- 172.21.15.115:0/3835761350 shutdown_connections 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.563+0000 7ff1981d2640 1 -- 172.21.15.115:0/3835761350 wait complete. 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.565+0000 7ff1981d2640 1 Processor -- start 2023-12-01T03:10:30.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.565+0000 7ff1981d2640 1 -- start start 2023-12-01T03:10:30.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.566+0000 7ff1981d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 0x7ff190158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.566+0000 7ff1981d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900c6a00 0x7ff1900c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.566+0000 7ff1981d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1900c90f0 0x7ff1900c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.566+0000 7ff1981d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900c73c0 con 0x7ff17c004d00 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.566+0000 7ff1981d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900c7c40 con 0x7ff1900c90f0 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.566+0000 7ff1981d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900c7680 con 0x7ff1900c6a00 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.566+0000 7ff195f6e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 0x7ff190158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.566+0000 7ff195f6e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 0x7ff190158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33722/0 (socket says 172.21.15.115:33722) 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.566+0000 7ff195f6e640 1 -- 172.21.15.115:0/3618860072 learned_addr learned my addr 172.21.15.115:0/3618860072 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.567+0000 7ff195f6e640 1 -- 172.21.15.115:0/3618860072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900c6a00 msgr2=0x7ff1900c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.567+0000 7ff195f6e640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900c6a00 0x7ff1900c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.567+0000 7ff19676f640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1900c90f0 0x7ff1900c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.567+0000 7ff195f6e640 1 -- 172.21.15.115:0/3618860072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1900c90f0 msgr2=0x7ff1900c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.567+0000 7ff195f6e640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1900c90f0 0x7ff1900c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.567+0000 7ff195f6e640 1 -- 172.21.15.115:0/3618860072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900c7960 con 0x7ff17c004d00 2023-12-01T03:10:30.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.567+0000 7ff195f6e640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 0x7ff190158f80 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7ff184000fb0 tx=0x7ff18400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:30.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.567+0000 7ff1827fc640 1 -- 172.21.15.115:0/3618860072 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18400c840 con 0x7ff17c004d00 2023-12-01T03:10:30.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.568+0000 7ff1827fc640 1 -- 172.21.15.115:0/3618860072 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18400c9a0 con 0x7ff17c004d00 2023-12-01T03:10:30.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.568+0000 7ff1827fc640 1 -- 172.21.15.115:0/3618860072 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18400cd30 con 0x7ff17c004d00 2023-12-01T03:10:30.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.568+0000 7ff1981d2640 1 -- 172.21.15.115:0/3618860072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1900b3570 con 0x7ff17c004d00 2023-12-01T03:10:30.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.568+0000 7ff1981d2640 1 -- 172.21.15.115:0/3618860072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1900c9ae0 con 0x7ff17c004d00 2023-12-01T03:10:30.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.569+0000 7ff1981d2640 1 -- 172.21.15.115:0/3618860072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1900b33b0 con 0x7ff17c004d00 2023-12-01T03:10:30.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.570+0000 7ff1827fc640 1 -- 172.21.15.115:0/3618860072 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff18401d460 con 0x7ff17c004d00 2023-12-01T03:10:30.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.570+0000 7ff1827fc640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff17406a580 0x7ff17406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:30.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.571+0000 7ff1827fc640 1 -- 172.21.15.115:0/3618860072 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7ff184088ae0 con 0x7ff17c004d00 2023-12-01T03:10:30.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.571+0000 7ff19576d640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff17406a580 0x7ff17406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:30.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.572+0000 7ff19576d640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff17406a580 0x7ff17406ca40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff18c000f60 tx=0x7ff18c009040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:30.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.575+0000 7ff1827fc640 1 -- 172.21.15.115:0/3618860072 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff184053f90 con 0x7ff17c004d00 2023-12-01T03:10:30.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.834+0000 7ff1981d2640 1 -- 172.21.15.115:0/3618860072 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff19004e890 con 0x7ff17c004d00 2023-12-01T03:10:30.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.841+0000 7ff1827fc640 1 -- 172.21.15.115:0/3618860072 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v48) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff1840575b0 con 0x7ff17c004d00 2023-12-01T03:10:30.844 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T03:10:30.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 -- 172.21.15.115:0/3618860072 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff17406a580 msgr2=0x7ff17406ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff17406a580 0x7ff17406ca40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff18c000f60 tx=0x7ff18c009040 comp rx=0 tx=0).stop 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 -- 172.21.15.115:0/3618860072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 msgr2=0x7ff190158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 0x7ff190158f80 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7ff184000fb0 tx=0x7ff18400ed50 comp rx=0 tx=0).stop 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 -- 172.21.15.115:0/3618860072 shutdown_connections 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff17406a580 0x7ff17406ca40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff1900c6a00 0x7ff1900c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1900c90f0 0x7ff1900c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 --2- 172.21.15.115:0/3618860072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff17c004d00 0x7ff190158f80 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 -- 172.21.15.115:0/3618860072 >> 172.21.15.115:0/3618860072 conn(0x7ff1900b1590 msgr2=0x7ff190160b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.844+0000 7ff1981d2640 1 -- 172.21.15.115:0/3618860072 shutdown_connections 2023-12-01T03:10:30.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:30.845+0000 7ff1981d2640 1 -- 172.21.15.115:0/3618860072 wait complete. 2023-12-01T03:10:30.865 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:10:31.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.076+0000 7f4775b8b640 1 Processor -- start 2023-12-01T03:10:31.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.076+0000 7f4775b8b640 1 -- start start 2023-12-01T03:10:31.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.078+0000 7f4775b8b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f477015d220 0x7f47700cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.078+0000 7f4775b8b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4770070ac0 0x7f47700ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.078+0000 7f4775b8b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477015d600 con 0x7f47700ad2b0 2023-12-01T03:10:31.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.078+0000 7f4775b8b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477006cc10 con 0x7f477015d220 2023-12-01T03:10:31.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.078+0000 7f4775b8b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477006c720 con 0x7f4770070ac0 2023-12-01T03:10:31.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.079+0000 7f476ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4770070ac0 0x7f47700ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.079+0000 7f476f7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f477015d220 0x7f47700cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.079+0000 7f476ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4770070ac0 0x7f47700ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:48530/0 (socket says 172.21.15.115:48530) 2023-12-01T03:10:31.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.079+0000 7f476ffff640 1 -- 172.21.15.115:0/580502814 learned_addr learned my addr 172.21.15.115:0/580502814 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.079+0000 7f476effd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f47700ad2b0 0x7f477006b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:60312/0 (socket says 172.21.15.115:60312) 2023-12-01T03:10:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.079+0000 7f476f7fe640 1 -- 172.21.15.115:0/580502814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4770070ac0 msgr2=0x7f47700ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.079+0000 7f476f7fe640 1 --2- 172.21.15.115:0/580502814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4770070ac0 0x7f47700ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.079+0000 7f476f7fe640 1 -- 172.21.15.115:0/580502814 >> v1:172.21.15.142:6789/0 conn(0x7f47700ad2b0 legacy=0x7f477006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.079+0000 7f476f7fe640 1 -- 172.21.15.115:0/580502814 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477006c4b0 con 0x7f477015d220 2023-12-01T03:10:31.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.079+0000 7f476ffff640 1 --2- 172.21.15.115:0/580502814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4770070ac0 0x7f47700ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:10:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.081+0000 7f476f7fe640 1 --2- 172.21.15.115:0/580502814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f477015d220 0x7f47700cc570 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f47600097a0 tx=0x7f476002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63edd9e79947360 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.081+0000 7f476e7fc640 1 -- 172.21.15.115:0/580502814 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476002e070 con 0x7f477015d220 2023-12-01T03:10:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.081+0000 7f476e7fc640 1 -- 172.21.15.115:0/580502814 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47600315a0 con 0x7f477015d220 2023-12-01T03:10:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.081+0000 7f476e7fc640 1 -- 172.21.15.115:0/580502814 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760031930 con 0x7f477015d220 2023-12-01T03:10:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.082+0000 7f4775b8b640 1 -- 172.21.15.115:0/580502814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f477015d220 msgr2=0x7f47700cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.082+0000 7f4775b8b640 1 --2- 172.21.15.115:0/580502814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f477015d220 0x7f47700cc570 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f47600097a0 tx=0x7f476002b840 comp rx=0 tx=0).stop 2023-12-01T03:10:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.082+0000 7f4775b8b640 1 -- 172.21.15.115:0/580502814 shutdown_connections 2023-12-01T03:10:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.082+0000 7f4775b8b640 1 --2- 172.21.15.115:0/580502814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4770070ac0 0x7f47700ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.082+0000 7f4775b8b640 1 --2- 172.21.15.115:0/580502814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f477015d220 0x7f47700cc570 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.082+0000 7f4775b8b640 1 -- 172.21.15.115:0/580502814 >> 172.21.15.115:0/580502814 conn(0x7f47700ce880 msgr2=0x7f47700cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:31.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.082+0000 7f4775b8b640 1 -- 172.21.15.115:0/580502814 shutdown_connections 2023-12-01T03:10:31.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.082+0000 7f4775b8b640 1 -- 172.21.15.115:0/580502814 wait complete. 2023-12-01T03:10:31.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.084+0000 7f4775b8b640 1 Processor -- start 2023-12-01T03:10:31.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.085+0000 7f4775b8b640 1 -- start start 2023-12-01T03:10:31.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.086+0000 7f4775b8b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4770070ac0 0x7f477014c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.086+0000 7f4775b8b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47700ad2b0 0x7f477015a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.086+0000 7f4775b8b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f477015d220 0x7f477015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.086+0000 7f4775b8b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477014ceb0 con 0x7f477015d220 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.086+0000 7f4775b8b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477014d190 con 0x7f4770070ac0 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.086+0000 7f4775b8b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477014d470 con 0x7f47700ad2b0 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.086+0000 7f476ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f477015d220 0x7f477015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.086+0000 7f476ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f477015d220 0x7f477015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33728/0 (socket says 172.21.15.115:33728) 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.086+0000 7f476ffff640 1 -- 172.21.15.115:0/3996325600 learned_addr learned my addr 172.21.15.115:0/3996325600 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f476f7fe640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4770070ac0 0x7f477014c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f476effd640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47700ad2b0 0x7f477015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f476ffff640 1 -- 172.21.15.115:0/3996325600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47700ad2b0 msgr2=0x7f477015a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f476ffff640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47700ad2b0 0x7f477015a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f476ffff640 1 -- 172.21.15.115:0/3996325600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4770070ac0 msgr2=0x7f477014c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f476ffff640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4770070ac0 0x7f477014c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f476ffff640 1 -- 172.21.15.115:0/3996325600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4768009b30 con 0x7f477015d220 2023-12-01T03:10:31.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f476effd640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47700ad2b0 0x7f477015a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:31.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f476ffff640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f477015d220 0x7f477015aaa0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f476800efc0 tx=0x7f476800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:31.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f4757fff640 1 -- 172.21.15.115:0/3996325600 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768018070 con 0x7f477015d220 2023-12-01T03:10:31.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f4757fff640 1 -- 172.21.15.115:0/3996325600 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476800f040 con 0x7f477015d220 2023-12-01T03:10:31.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.087+0000 7f4775b8b640 1 -- 172.21.15.115:0/3996325600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4760009400 con 0x7f477015d220 2023-12-01T03:10:31.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.088+0000 7f4757fff640 1 -- 172.21.15.115:0/3996325600 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476800ce00 con 0x7f477015d220 2023-12-01T03:10:31.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.088+0000 7f4775b8b640 1 -- 172.21.15.115:0/3996325600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477014a870 con 0x7f477015d220 2023-12-01T03:10:31.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.089+0000 7f4775b8b640 1 -- 172.21.15.115:0/3996325600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477004e890 con 0x7f477015d220 2023-12-01T03:10:31.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.090+0000 7f4757fff640 1 -- 172.21.15.115:0/3996325600 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4768014480 con 0x7f477015d220 2023-12-01T03:10:31.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.090+0000 7f4757fff640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f475c06a550 0x7f475c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.091+0000 7f4757fff640 1 -- 172.21.15.115:0/3996325600 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f4768088e40 con 0x7f477015d220 2023-12-01T03:10:31.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.091+0000 7f476f7fe640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f475c06a550 0x7f475c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.092+0000 7f476f7fe640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f475c06a550 0x7f475c06ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4760036000 tx=0x7f476003d040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:31.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.095+0000 7f4757fff640 1 -- 172.21.15.115:0/3996325600 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4768054250 con 0x7f477015d220 2023-12-01T03:10:31.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.324+0000 7f4775b8b640 1 -- 172.21.15.115:0/3996325600 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f477015c6b0 con 0x7f477015d220 2023-12-01T03:10:31.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.325+0000 7f4757fff640 1 -- 172.21.15.115:0/3996325600 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4664 (secure 0 0 0) 0x7f4768057870 con 0x7f477015d220 2023-12-01T03:10:31.327 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:31.327 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7333,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:25.146035+0000","modified":"2023-12-01T03:10:28.416554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10653":{"gid":10653,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/720518620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":720518620},{"type":"v1","addr":"172.21.15.142:6834","nonce":720518620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-12-01T03:10:31.327 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 95 2023-12-01T03:10:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 -- 172.21.15.115:0/3996325600 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f475c06a550 msgr2=0x7f475c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f475c06a550 0x7f475c06ca10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4760036000 tx=0x7f476003d040 comp rx=0 tx=0).stop 2023-12-01T03:10:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 -- 172.21.15.115:0/3996325600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f477015d220 msgr2=0x7f477015aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f477015d220 0x7f477015aaa0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f476800efc0 tx=0x7f476800c490 comp rx=0 tx=0).stop 2023-12-01T03:10:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 -- 172.21.15.115:0/3996325600 shutdown_connections 2023-12-01T03:10:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f475c06a550 0x7f475c06ca10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47700ad2b0 0x7f477015a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f477015d220 0x7f477015aaa0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 --2- 172.21.15.115:0/3996325600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4770070ac0 0x7f477014c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 -- 172.21.15.115:0/3996325600 >> 172.21.15.115:0/3996325600 conn(0x7f47700ce880 msgr2=0x7f47700afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:31.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 -- 172.21.15.115:0/3996325600 shutdown_connections 2023-12-01T03:10:31.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.330+0000 7f4775b8b640 1 -- 172.21.15.115:0/3996325600 wait complete. 2023-12-01T03:10:31.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T03:10:25.146035+0000', 'modified': '2023-12-01T03:10:28.416554+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10653}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10653': {'gid': 10653, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6834/720518620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 720518620}, {'type': 'v1', 'addr': '172.21.15.142:6834', 'nonce': 720518620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T03:10:31.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T03:10:31.755 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-01T03:10:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.980+0000 7fe7701fd640 1 Processor -- start 2023-12-01T03:10:31.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.981+0000 7fe7701fd640 1 -- start start 2023-12-01T03:10:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.982+0000 7fe7701fd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe76815cf90 0x7fe7680d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.982+0000 7fe7701fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680ce0d0 0x7fe7680b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.982+0000 7fe7701fd640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76815d370 con 0x7fe7680b8610 2023-12-01T03:10:31.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.982+0000 7fe7701fd640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7680d19a0 con 0x7fe76815cf90 2023-12-01T03:10:31.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.982+0000 7fe7701fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680d1c80 con 0x7fe7680ce0d0 2023-12-01T03:10:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.983+0000 7fe76e79a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680ce0d0 0x7fe7680b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.983+0000 7fe76e79a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680ce0d0 0x7fe7680b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:48544/0 (socket says 172.21.15.115:48544) 2023-12-01T03:10:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.983+0000 7fe76e79a640 1 -- 172.21.15.115:0/2187206068 learned_addr learned my addr 172.21.15.115:0/2187206068 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.983+0000 7fe76df99640 1 --2- 172.21.15.115:0/2187206068 >> v2:172.21.15.142:3300/0 conn(0x7fe76815cf90 0x7fe7680d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.983+0000 7fe76e79a640 1 -- 172.21.15.115:0/2187206068 >> v2:172.21.15.142:3300/0 conn(0x7fe76815cf90 msgr2=0x7fe7680d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:31.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.984+0000 7fe76e79a640 1 --2- 172.21.15.115:0/2187206068 >> v2:172.21.15.142:3300/0 conn(0x7fe76815cf90 0x7fe7680d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.984+0000 7fe76e79a640 1 -- 172.21.15.115:0/2187206068 >> v1:172.21.15.142:6789/0 conn(0x7fe7680b8610 legacy=0x7fe7680cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:31.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.984+0000 7fe76e79a640 1 -- 172.21.15.115:0/2187206068 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7680d1f60 con 0x7fe7680ce0d0 2023-12-01T03:10:31.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.985+0000 7fe76e79a640 1 --2- 172.21.15.115:0/2187206068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680ce0d0 0x7fe7680b5d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe760009870 tx=0x7fe76002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51cdd577f9bd7732 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:31.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.985+0000 7fe76cf97640 1 -- 172.21.15.115:0/2187206068 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760030840 con 0x7fe7680ce0d0 2023-12-01T03:10:31.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.985+0000 7fe76cf97640 1 -- 172.21.15.115:0/2187206068 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7600309a0 con 0x7fe7680ce0d0 2023-12-01T03:10:31.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.986+0000 7fe76cf97640 1 -- 172.21.15.115:0/2187206068 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760030d30 con 0x7fe7680ce0d0 2023-12-01T03:10:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.986+0000 7fe7701fd640 1 -- 172.21.15.115:0/2187206068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680ce0d0 msgr2=0x7fe7680b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.986+0000 7fe7701fd640 1 --2- 172.21.15.115:0/2187206068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680ce0d0 0x7fe7680b5d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe760009870 tx=0x7fe76002b980 comp rx=0 tx=0).stop 2023-12-01T03:10:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.986+0000 7fe7701fd640 1 -- 172.21.15.115:0/2187206068 shutdown_connections 2023-12-01T03:10:31.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.986+0000 7fe7701fd640 1 --2- 172.21.15.115:0/2187206068 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680ce0d0 0x7fe7680b5d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.986+0000 7fe7701fd640 1 --2- 172.21.15.115:0/2187206068 >> v2:172.21.15.142:3300/0 conn(0x7fe76815cf90 0x7fe7680d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.986+0000 7fe7701fd640 1 -- 172.21.15.115:0/2187206068 >> 172.21.15.115:0/2187206068 conn(0x7fe768065960 msgr2=0x7fe768065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.986+0000 7fe7701fd640 1 -- 172.21.15.115:0/2187206068 shutdown_connections 2023-12-01T03:10:31.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.987+0000 7fe7701fd640 1 -- 172.21.15.115:0/2187206068 wait complete. 2023-12-01T03:10:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.988+0000 7fe7701fd640 1 Processor -- start 2023-12-01T03:10:31.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.988+0000 7fe7701fd640 1 -- start start 2023-12-01T03:10:31.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.989+0000 7fe7701fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680b8610 0x7fe76815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.989+0000 7fe7701fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7680ce0d0 0x7fe7680d2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.989+0000 7fe7701fd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe76815cf90 0x7fe7680d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.989+0000 7fe7701fd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe768144940 con 0x7fe7680ce0d0 2023-12-01T03:10:31.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.989+0000 7fe7701fd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7681f69b0 con 0x7fe76815cf90 2023-12-01T03:10:31.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.989+0000 7fe7701fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7681604d0 con 0x7fe7680b8610 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76d798640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7680ce0d0 0x7fe7680d2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76d798640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7680ce0d0 0x7fe7680d2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33742/0 (socket says 172.21.15.115:33742) 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76d798640 1 -- 172.21.15.115:0/3020249384 learned_addr learned my addr 172.21.15.115:0/3020249384 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76df99640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680b8610 0x7fe76815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76e79a640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe76815cf90 0x7fe7680d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76d798640 1 -- 172.21.15.115:0/3020249384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680b8610 msgr2=0x7fe76815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76d798640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680b8610 0x7fe76815ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76d798640 1 -- 172.21.15.115:0/3020249384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe76815cf90 msgr2=0x7fe7680d31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76d798640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe76815cf90 0x7fe7680d31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76d798640 1 -- 172.21.15.115:0/3020249384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600094d0 con 0x7fe7680ce0d0 2023-12-01T03:10:31.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76df99640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680b8610 0x7fe76815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:10:31.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.990+0000 7fe76e79a640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe76815cf90 0x7fe7680d31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:31.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.991+0000 7fe76d798640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7680ce0d0 0x7fe7680d2c60 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c470 tx=0x7fe754001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:31.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.991+0000 7fe7667fc640 1 -- 172.21.15.115:0/3020249384 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7540052d0 con 0x7fe7680ce0d0 2023-12-01T03:10:31.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.991+0000 7fe7667fc640 1 -- 172.21.15.115:0/3020249384 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754005430 con 0x7fe7680ce0d0 2023-12-01T03:10:31.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.991+0000 7fe7667fc640 1 -- 172.21.15.115:0/3020249384 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754007260 con 0x7fe7680ce0d0 2023-12-01T03:10:31.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.991+0000 7fe7701fd640 1 -- 172.21.15.115:0/3020249384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7681f5bf0 con 0x7fe7680ce0d0 2023-12-01T03:10:31.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.991+0000 7fe7701fd640 1 -- 172.21.15.115:0/3020249384 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7681f61b0 con 0x7fe7680ce0d0 2023-12-01T03:10:31.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.993+0000 7fe7667fc640 1 -- 172.21.15.115:0/3020249384 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fe7540073c0 con 0x7fe7680ce0d0 2023-12-01T03:10:31.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.994+0000 7fe7667fc640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe74c06a550 0x7fe74c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.994+0000 7fe7667fc640 1 -- 172.21.15.115:0/3020249384 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fe754087fb0 con 0x7fe7680ce0d0 2023-12-01T03:10:31.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.994+0000 7fe76df99640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe74c06a550 0x7fe74c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.995+0000 7fe772d74740 1 Processor -- start 2023-12-01T03:10:31.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.995+0000 7fe772d74740 1 -- start start 2023-12-01T03:10:31.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.995+0000 7fe76df99640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe74c06a550 0x7fe74c06ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009fd0 tx=0x7fe75c00b040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.996+0000 7fe772d74740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 0x55e8d60c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.996+0000 7fe772d74740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60998a0 0x55e8d60c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.996+0000 7fe772d74740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.996+0000 7fe772d74740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d60752f0 con 0x55e8d609d4a0 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.996+0000 7fe772d74740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d6009f80 con 0x55e8d60c9920 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.996+0000 7fe772d74740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8d6007710 con 0x55e8d60998a0 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.996+0000 7fe76e79a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.996+0000 7fe76e79a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33750/0 (socket says 172.21.15.115:33750) 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.996+0000 7fe76e79a640 1 -- 172.21.15.115:0/268538289 learned_addr learned my addr 172.21.15.115:0/268538289 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.997+0000 7fe76df99640 1 --2- 172.21.15.115:0/268538289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 0x55e8d60c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:31.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.997+0000 7fe76d798640 1 --2- 172.21.15.115:0/268538289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60998a0 0x55e8d60c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.997+0000 7fe76e79a640 1 -- 172.21.15.115:0/268538289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60998a0 msgr2=0x55e8d60c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:32.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.997+0000 7fe76e79a640 1 --2- 172.21.15.115:0/268538289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60998a0 0x55e8d60c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.997+0000 7fe76e79a640 1 -- 172.21.15.115:0/268538289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 msgr2=0x55e8d60c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:32.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.997+0000 7fe76e79a640 1 --2- 172.21.15.115:0/268538289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 0x55e8d60c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.997+0000 7fe76e79a640 1 -- 172.21.15.115:0/268538289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d5fca690 con 0x55e8d609d4a0 2023-12-01T03:10:32.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.997+0000 7fe76e79a640 1 --2- 172.21.15.115:0/268538289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60c70d0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fe76002b9c0 tx=0x7fe760008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5e13768d29aeaa server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:32.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.997+0000 7fe74b7fe640 1 -- 172.21.15.115:0/268538289 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760030b10 con 0x55e8d609d4a0 2023-12-01T03:10:32.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.997+0000 7fe74b7fe640 1 -- 172.21.15.115:0/268538289 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760030c70 con 0x55e8d609d4a0 2023-12-01T03:10:32.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.998+0000 7fe74b7fe640 1 -- 172.21.15.115:0/268538289 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760044470 con 0x55e8d609d4a0 2023-12-01T03:10:32.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.998+0000 7fe772d74740 1 -- 172.21.15.115:0/268538289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 msgr2=0x55e8d60c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.998+0000 7fe772d74740 1 --2- 172.21.15.115:0/268538289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60c70d0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fe76002b9c0 tx=0x7fe760008030 comp rx=0 tx=0).stop 2023-12-01T03:10:32.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.998+0000 7fe772d74740 1 -- 172.21.15.115:0/268538289 shutdown_connections 2023-12-01T03:10:32.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.998+0000 7fe772d74740 1 --2- 172.21.15.115:0/268538289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60998a0 0x55e8d60c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.998+0000 7fe772d74740 1 --2- 172.21.15.115:0/268538289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60c70d0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.998+0000 7fe772d74740 1 --2- 172.21.15.115:0/268538289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 0x55e8d60c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.998+0000 7fe772d74740 1 -- 172.21.15.115:0/268538289 >> 172.21.15.115:0/268538289 conn(0x55e8d609bbb0 msgr2=0x55e8d60e4730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:32.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.998+0000 7fe772d74740 1 -- 172.21.15.115:0/268538289 shutdown_connections 2023-12-01T03:10:32.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.998+0000 7fe772d74740 1 -- 172.21.15.115:0/268538289 wait complete. 2023-12-01T03:10:32.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.999+0000 7fe772d74740 1 -- start start 2023-12-01T03:10:32.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:31.999+0000 7fe772d74740 1 Processor -- start 2023-12-01T03:10:32.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.000+0000 7fe772d74740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 0x55e8d60cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.000+0000 7fe772d74740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c9920 0x55e8d60e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.000+0000 7fe772d74740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c5f10 0x55e8d60e4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.000+0000 7fe772d74740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d6003000 con 0x55e8d60c9920 2023-12-01T03:10:32.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.000+0000 7fe772d74740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d5fa47d0 con 0x55e8d609d4a0 2023-12-01T03:10:32.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.000+0000 7fe772d74740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8d5fa4470 con 0x55e8d60c5f10 2023-12-01T03:10:32.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.000+0000 7fe772d74740 10 client.0 _ll_register_callbacks cb 0x55e8d5ff31b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:10:32.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76d798640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c9920 0x55e8d60e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76d798640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c9920 0x55e8d60e3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33752/0 (socket says 172.21.15.115:33752) 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76d798640 1 -- 172.21.15.115:0/3920220078 learned_addr learned my addr 172.21.15.115:0/3920220078 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76e79a640 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c5f10 0x55e8d60e4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76df99640 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 0x55e8d60cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76d798640 1 -- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c5f10 msgr2=0x55e8d60e4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76d798640 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c5f10 0x55e8d60e4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76d798640 1 -- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 msgr2=0x55e8d60cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76d798640 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 0x55e8d60cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76d798640 1 -- 172.21.15.115:0/3920220078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7680d1f60 con 0x55e8d60c9920 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76df99640 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 0x55e8d60cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe76d798640 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c9920 0x55e8d60e3d40 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fe754057010 tx=0x7fe754057100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556d3b7f9706cf94 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.001+0000 7fe748ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe748ff9640 1 -- 172.21.15.115:0/3920220078 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754086610 con 0x55e8d60c9920 2023-12-01T03:10:32.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe772d74740 1 -- 172.21.15.115:0/3920220078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8d602d550 con 0x55e8d60c9920 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe748ff9640 1 -- 172.21.15.115:0/3920220078 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fe75405ebd0 con 0x55e8d60c9920 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe748ff9640 1 -- 172.21.15.115:0/3920220078 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe75405ed90 con 0x55e8d60c9920 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe772d74740 10 client.10776 fetch_fsmap learned FSMap version 95 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe772d74740 1 -- 172.21.15.115:0/3920220078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x55e8d6001680 con 0x55e8d60c9920 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe748ff9640 1 -- 172.21.15.115:0/3920220078 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8d602d550 con 0x55e8d60c9920 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe748ff9640 1 -- 172.21.15.115:0/3920220078 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 95) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe754052e70 con 0x55e8d60c9920 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe772d74740 10 client.10776 fetch_fsmap finished waiting for FSMap version 95 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe772d74740 10 client.10776 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe772d74740 20 client.10776 populate_metadata read hostname 'smithi115' 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe772d74740 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:6832/720518620,v1:172.21.15.142:6834/720518620] conn(0x55e8d60b7510 0x55e8d60c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe772d74740 4 client.10776 mds_command: new command op to 10653 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.002+0000 7fe772d74740 1 -- 172.21.15.115:0/3920220078 --> [v2:172.21.15.142:6832/720518620,v1:172.21.15.142:6834/720518620] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8d5ff5b50 con 0x55e8d60b7510 2023-12-01T03:10:32.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.003+0000 7fe76df99640 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:6832/720518620,v1:172.21.15.142:6834/720518620] conn(0x55e8d60b7510 0x55e8d60c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.004+0000 7fe76df99640 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:6832/720518620,v1:172.21.15.142:6834/720518620] conn(0x55e8d60b7510 0x55e8d60c9d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caad4816c1fdba63 server_cookie=6e38058107045397 in_seq=0 out_seq=0 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.004+0000 7fe748ff9640 10 client.10776 ms_handle_connect on v2:172.21.15.142:6832/720518620 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe748ff9640 1 -- 172.21.15.115:0/3920220078 <== mds.0 v2:172.21.15.142:6832/720518620 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e8d5ff5b50 con 0x55e8d60b7510 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe748ff9640 10 client.10776 handle_command_reply: tid=0 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 client.10776 shutdown 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 -- 172.21.15.115:0/3920220078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c9920 msgr2=0x55e8d60e3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c9920 0x55e8d60e3d40 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fe754057010 tx=0x7fe754057100 comp rx=0 tx=0).stop 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 -- 172.21.15.115:0/3920220078 shutdown_connections 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:6832/720518620,v1:172.21.15.142:6834/720518620] conn(0x55e8d60b7510 0x55e8d60c9d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c5f10 0x55e8d60e4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c9920 0x55e8d60e3d40 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 --2- 172.21.15.115:0/3920220078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 0x55e8d60cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe748ff9640 0 client.10776 ms_handle_reset on v2:172.21.15.142:6832/720518620 2023-12-01T03:10:32.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 -- 172.21.15.115:0/3920220078 >> 172.21.15.115:0/3920220078 conn(0x55e8d60998a0 msgr2=0x55e8d609dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:32.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 -- 172.21.15.115:0/3920220078 shutdown_connections 2023-12-01T03:10:32.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.006+0000 7fe772d74740 1 -- 172.21.15.115:0/3920220078 wait complete. 2023-12-01T03:10:32.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.007+0000 7fe772d74740 20 client.10776 trim_cache size 0 max 16384 2023-12-01T03:10:32.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.023+0000 7fe772d74740 1 Processor -- start 2023-12-01T03:10:32.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.023+0000 7fe772d74740 1 -- start start 2023-12-01T03:10:32.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.023+0000 7fe772d74740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.023+0000 7fe772d74740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c5f10 0x55e8d60c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe772d74740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 0x55e8d60c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe772d74740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d5d6af20 con 0x55e8d609d4a0 2023-12-01T03:10:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe772d74740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d60e45c0 con 0x55e8d60c9920 2023-12-01T03:10:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe772d74740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8d60cb510 con 0x55e8d60c5f10 2023-12-01T03:10:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76df99640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76df99640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33766/0 (socket says 172.21.15.115:33766) 2023-12-01T03:10:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76df99640 1 -- 172.21.15.115:0/145568718 learned_addr learned my addr 172.21.15.115:0/145568718 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:32.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76e79a640 1 --2- 172.21.15.115:0/145568718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 0x55e8d60c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76df99640 1 -- 172.21.15.115:0/145568718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c5f10 msgr2=0x55e8d60c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:32.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76d798640 1 --2- 172.21.15.115:0/145568718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c5f10 0x55e8d60c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76df99640 1 --2- 172.21.15.115:0/145568718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c5f10 0x55e8d60c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76df99640 1 -- 172.21.15.115:0/145568718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 msgr2=0x55e8d60c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76df99640 1 --2- 172.21.15.115:0/145568718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 0x55e8d60c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76df99640 1 -- 172.21.15.115:0/145568718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d5fa39e0 con 0x55e8d609d4a0 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.024+0000 7fe76e79a640 1 --2- 172.21.15.115:0/145568718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 0x55e8d60c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.025+0000 7fe76df99640 1 --2- 172.21.15.115:0/145568718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60dbf80 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x55e8d60f7eb0 tx=0x7fe75c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cd82cb5756fde0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.025+0000 7fe74bfff640 1 -- 172.21.15.115:0/145568718 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c003db0 con 0x55e8d609d4a0 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.025+0000 7fe74bfff640 1 -- 172.21.15.115:0/145568718 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00f050 con 0x55e8d609d4a0 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.025+0000 7fe74bfff640 1 -- 172.21.15.115:0/145568718 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c026820 con 0x55e8d609d4a0 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.025+0000 7fe772d74740 1 -- 172.21.15.115:0/145568718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 msgr2=0x55e8d60dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.025+0000 7fe772d74740 1 --2- 172.21.15.115:0/145568718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60dbf80 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x55e8d60f7eb0 tx=0x7fe75c014000 comp rx=0 tx=0).stop 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.026+0000 7fe772d74740 1 -- 172.21.15.115:0/145568718 shutdown_connections 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.026+0000 7fe772d74740 1 --2- 172.21.15.115:0/145568718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c5f10 0x55e8d60c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.026+0000 7fe772d74740 1 --2- 172.21.15.115:0/145568718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d60c9920 0x55e8d60c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.026+0000 7fe772d74740 1 --2- 172.21.15.115:0/145568718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d609d4a0 0x55e8d60dbf80 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.026+0000 7fe772d74740 1 -- 172.21.15.115:0/145568718 >> 172.21.15.115:0/145568718 conn(0x55e8d60998a0 msgr2=0x55e8d6095040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.026+0000 7fe772d74740 1 -- 172.21.15.115:0/145568718 shutdown_connections 2023-12-01T03:10:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.026+0000 7fe772d74740 1 -- 172.21.15.115:0/145568718 wait complete. 2023-12-01T03:10:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.026+0000 7fe772d74740 1 -- start start 2023-12-01T03:10:32.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.027+0000 7fe772d74740 1 Processor -- start 2023-12-01T03:10:32.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe772d74740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 0x55e8d60e6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe772d74740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c5f10 0x55e8d60b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe772d74740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c9920 0x55e8d60b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe772d74740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d60b5ba0 con 0x55e8d60c5f10 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe772d74740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d6099300 con 0x55e8d609d4a0 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe772d74740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8d6106ec0 con 0x55e8d60c9920 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe772d74740 10 client.0 _ll_register_callbacks cb 0x7fe75c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76d798640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c5f10 0x55e8d60b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76d798640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c5f10 0x55e8d60b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33770/0 (socket says 172.21.15.115:33770) 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76d798640 1 -- 172.21.15.115:0/3767221882 learned_addr learned my addr 172.21.15.115:0/3767221882 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76d798640 1 -- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c9920 msgr2=0x55e8d60b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76d798640 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c9920 0x55e8d60b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76d798640 1 -- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 msgr2=0x55e8d60e6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76df99640 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 0x55e8d60e6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76d798640 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 0x55e8d60e6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76d798640 1 -- 172.21.15.115:0/3767221882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c020040 con 0x55e8d60c5f10 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76df99640 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 0x55e8d60e6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe76d798640 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c5f10 0x55e8d60b4c80 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fe75405eb30 tx=0x7fe75400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c03c4bb46fcc13 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.028+0000 7fe748ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe748ff9640 1 -- 172.21.15.115:0/3767221882 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754086610 con 0x55e8d60c5f10 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe748ff9640 1 -- 172.21.15.115:0/3767221882 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fe75405aeb0 con 0x55e8d60c5f10 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe772d74740 1 -- 172.21.15.115:0/3767221882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe760030c70 con 0x55e8d60c5f10 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe748ff9640 1 -- 172.21.15.115:0/3767221882 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7540577f0 con 0x55e8d60c5f10 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe772d74740 10 client.10782 fetch_fsmap learned FSMap version 95 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe772d74740 1 -- 172.21.15.115:0/3767221882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x55e8d5fc8ef0 con 0x55e8d60c5f10 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe748ff9640 1 -- 172.21.15.115:0/3767221882 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760030c70 con 0x55e8d60c5f10 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe748ff9640 1 -- 172.21.15.115:0/3767221882 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 95) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe754060080 con 0x55e8d60c5f10 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe772d74740 10 client.10782 fetch_fsmap finished waiting for FSMap version 95 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe772d74740 10 client.10782 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-01T03:10:32.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe772d74740 20 client.10782 populate_metadata read hostname 'smithi115' 2023-12-01T03:10:32.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe772d74740 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:6832/720518620,v1:172.21.15.142:6834/720518620] conn(0x55e8d60e2420 0x55e8d609e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe772d74740 4 client.10782 mds_command: new command op to 10653 tid=0[{"prefix": "status"}] 2023-12-01T03:10:32.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.029+0000 7fe772d74740 1 -- 172.21.15.115:0/3767221882 --> [v2:172.21.15.142:6832/720518620,v1:172.21.15.142:6834/720518620] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e8d5ffc080 con 0x55e8d60e2420 2023-12-01T03:10:32.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.030+0000 7fe76df99640 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:6832/720518620,v1:172.21.15.142:6834/720518620] conn(0x55e8d60e2420 0x55e8d609e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.031+0000 7fe76df99640 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:6832/720518620,v1:172.21.15.142:6834/720518620] conn(0x55e8d60e2420 0x55e8d609e6e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4a415036e5e220 server_cookie=6579b3859c0ce56c in_seq=0 out_seq=0 2023-12-01T03:10:32.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.031+0000 7fe748ff9640 10 client.10782 ms_handle_connect on v2:172.21.15.142:6832/720518620 2023-12-01T03:10:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe748ff9640 1 -- 172.21.15.115:0/3767221882 <== mds.0 v2:172.21.15.142:6832/720518620 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55e8d5ffc080 con 0x55e8d60e2420 2023-12-01T03:10:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe748ff9640 10 client.10782 handle_command_reply: tid=0 2023-12-01T03:10:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 client.10782 shutdown 2023-12-01T03:10:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 -- 172.21.15.115:0/3767221882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c5f10 msgr2=0x55e8d60b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c5f10 0x55e8d60b4c80 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fe75405eb30 tx=0x7fe75400a000 comp rx=0 tx=0).stop 2023-12-01T03:10:32.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 -- 172.21.15.115:0/3767221882 shutdown_connections 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:6832/720518620,v1:172.21.15.142:6834/720518620] conn(0x55e8d60e2420 0x55e8d609e6e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e8d60c9920 0x55e8d60b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe748ff9640 0 client.10782 ms_handle_reset on v2:172.21.15.142:6832/720518620 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55e8d60c5f10 0x55e8d60b4c80 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 --2- 172.21.15.115:0/3767221882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e8d609d4a0 0x55e8d60e6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 -- 172.21.15.115:0/3767221882 >> 172.21.15.115:0/3767221882 conn(0x55e8d60998a0 msgr2=0x55e8d60c6600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 -- 172.21.15.115:0/3767221882 shutdown_connections 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 1 -- 172.21.15.115:0/3767221882 wait complete. 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.032+0000 7fe772d74740 20 client.10782 trim_cache size 0 max 16384 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:10:32.037 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 10653, 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 6.8801387829999996, 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 95, 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 214, 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 214, 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 12.749877262 2023-12-01T03:10:32.038 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:10:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 -- 172.21.15.115:0/3020249384 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe74c06a550 msgr2=0x7fe74c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe74c06a550 0x7fe74c06ca10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009fd0 tx=0x7fe75c00b040 comp rx=0 tx=0).stop 2023-12-01T03:10:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 -- 172.21.15.115:0/3020249384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7680ce0d0 msgr2=0x7fe7680d2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7680ce0d0 0x7fe7680d2c60 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c470 tx=0x7fe754001e70 comp rx=0 tx=0).stop 2023-12-01T03:10:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 -- 172.21.15.115:0/3020249384 shutdown_connections 2023-12-01T03:10:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe74c06a550 0x7fe74c06ca10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe76815cf90 0x7fe7680d31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7680ce0d0 0x7fe7680d2c60 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 --2- 172.21.15.115:0/3020249384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7680b8610 0x7fe76815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 -- 172.21.15.115:0/3020249384 >> 172.21.15.115:0/3020249384 conn(0x7fe768065960 msgr2=0x7fe7680b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 -- 172.21.15.115:0/3020249384 shutdown_connections 2023-12-01T03:10:32.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.033+0000 7fe7701fd640 1 -- 172.21.15.115:0/3020249384 wait complete. 2023-12-01T03:10:32.052 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T03:10:32.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.252+0000 7f2ab1228640 1 Processor -- start 2023-12-01T03:10:32.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.252+0000 7f2ab1228640 1 -- start start 2023-12-01T03:10:32.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.253+0000 7f2ab1228640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2aac0c9ef0 0x7f2aac06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.253+0000 7f2ab1228640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2aac06b6f0 0x7f2aac070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2ab1228640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac06f8b0 con 0x7f2aac159dd0 2023-12-01T03:10:32.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2ab1228640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aac0adcb0 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2ab1228640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aac0ad9d0 con 0x7f2aac06b6f0 2023-12-01T03:10:32.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2aaa59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2aac0c9ef0 0x7f2aac06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2aaa59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2aac0c9ef0 0x7f2aac06fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33774/0 (socket says 172.21.15.115:33774) 2023-12-01T03:10:32.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2aaa59c640 1 -- 172.21.15.115:0/1266058859 learned_addr learned my addr 172.21.15.115:0/1266058859 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:32.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2aab59e640 1 --2- 172.21.15.115:0/1266058859 >> v2:172.21.15.142:3300/0 conn(0x7f2aac06b6f0 0x7f2aac070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2aaa59c640 1 -- 172.21.15.115:0/1266058859 >> v2:172.21.15.142:3300/0 conn(0x7f2aac06b6f0 msgr2=0x7f2aac070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:32.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2aaa59c640 1 --2- 172.21.15.115:0/1266058859 >> v2:172.21.15.142:3300/0 conn(0x7f2aac06b6f0 0x7f2aac070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2aaa59c640 1 -- 172.21.15.115:0/1266058859 >> v1:172.21.15.142:6789/0 conn(0x7f2aac159dd0 legacy=0x7f2aac0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:32.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2aaa59c640 1 -- 172.21.15.115:0/1266058859 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac0c9000 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.254+0000 7f2aab59e640 1 --2- 172.21.15.115:0/1266058859 >> v2:172.21.15.142:3300/0 conn(0x7f2aac06b6f0 0x7f2aac070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:32.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.256+0000 7f2aaa59c640 1 --2- 172.21.15.115:0/1266058859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 0x7f2aac06fd20 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f2aac06d470 tx=0x7f2aa002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c1005a98fb1ed2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:32.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.256+0000 7f2aa9d9b640 1 -- 172.21.15.115:0/1266058859 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0031750 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.256+0000 7f2aa9d9b640 1 -- 172.21.15.115:0/1266058859 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa00318b0 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.256+0000 7f2aa9d9b640 1 -- 172.21.15.115:0/1266058859 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0031c40 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.257+0000 7f2ab1228640 1 -- 172.21.15.115:0/1266058859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 msgr2=0x7f2aac06fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.257+0000 7f2ab1228640 1 --2- 172.21.15.115:0/1266058859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 0x7f2aac06fd20 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f2aac06d470 tx=0x7f2aa002b7f0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.257+0000 7f2ab1228640 1 -- 172.21.15.115:0/1266058859 shutdown_connections 2023-12-01T03:10:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.257+0000 7f2ab1228640 1 --2- 172.21.15.115:0/1266058859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 0x7f2aac06fd20 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.257+0000 7f2ab1228640 1 --2- 172.21.15.115:0/1266058859 >> v2:172.21.15.142:3300/0 conn(0x7f2aac06b6f0 0x7f2aac070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.257+0000 7f2ab1228640 1 -- 172.21.15.115:0/1266058859 >> 172.21.15.115:0/1266058859 conn(0x7f2aac0ce880 msgr2=0x7f2aac0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.257+0000 7f2ab1228640 1 -- 172.21.15.115:0/1266058859 shutdown_connections 2023-12-01T03:10:32.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.257+0000 7f2ab1228640 1 -- 172.21.15.115:0/1266058859 wait complete. 2023-12-01T03:10:32.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.259+0000 7f2ab1228640 1 Processor -- start 2023-12-01T03:10:32.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.259+0000 7f2ab1228640 1 -- start start 2023-12-01T03:10:32.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2ab1228640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac06b6f0 0x7f2aac0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2ab1228640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 0x7f2aac0c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2ab1228640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aac159dd0 0x7f2aac0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2ab1228640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0ae4b0 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2ab1228640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0af020 con 0x7f2aac159dd0 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2ab1228640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0c6720 con 0x7f2aac06b6f0 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aaa59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 0x7f2aac0c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aaa59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 0x7f2aac0c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33778/0 (socket says 172.21.15.115:33778) 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aaa59c640 1 -- 172.21.15.115:0/1109213799 learned_addr learned my addr 172.21.15.115:0/1109213799 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aaad9d640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac06b6f0 0x7f2aac0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aab59e640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aac159dd0 0x7f2aac0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aaa59c640 1 -- 172.21.15.115:0/1109213799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac06b6f0 msgr2=0x7f2aac0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aaa59c640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac06b6f0 0x7f2aac0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aaa59c640 1 -- 172.21.15.115:0/1109213799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aac159dd0 msgr2=0x7f2aac0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aaa59c640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aac159dd0 0x7f2aac0acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aaa59c640 1 -- 172.21.15.115:0/1109213799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa00094d0 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aaad9d640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac06b6f0 0x7f2aac0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.260+0000 7f2aab59e640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aac159dd0 0x7f2aac0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:32.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.261+0000 7f2aaa59c640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 0x7f2aac0c5950 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000c00 tx=0x7f2aa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:32.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.261+0000 7f2a937fe640 1 -- 172.21.15.115:0/1109213799 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0033c80 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.261+0000 7f2a937fe640 1 -- 172.21.15.115:0/1109213799 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa0033de0 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.261+0000 7f2a937fe640 1 -- 172.21.15.115:0/1109213799 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa003c620 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.261+0000 7f2ab1228640 1 -- 172.21.15.115:0/1109213799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac0c6a00 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.261+0000 7f2ab1228640 1 -- 172.21.15.115:0/1109213799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac0af5a0 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.262+0000 7f2ab1228640 1 -- 172.21.15.115:0/1109213799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac04e890 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.263+0000 7f2a937fe640 1 -- 172.21.15.115:0/1109213799 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f2aa003a040 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.265+0000 7f2a937fe640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a9806a580 0x7f2a9806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:32.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.265+0000 7f2a937fe640 1 -- 172.21.15.115:0/1109213799 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f2aa00aa9a0 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.265+0000 7f2aaad9d640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a9806a580 0x7f2a9806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:32.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.266+0000 7f2aaad9d640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a9806a580 0x7f2a9806ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00a830 tx=0x7f2a9c008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:32.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.270+0000 7f2a937fe640 1 -- 172.21.15.115:0/1109213799 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2aa0075db0 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.500+0000 7f2ab1228640 1 -- 172.21.15.115:0/1109213799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2aac071b00 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.500+0000 7f2a937fe640 1 -- 172.21.15.115:0/1109213799 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v95) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2aa0038020 con 0x7f2aac0c9ef0 2023-12-01T03:10:32.502 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T03:10:32.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.503+0000 7f2ab1228640 1 -- 172.21.15.115:0/1109213799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a9806a580 msgr2=0x7f2a9806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.503+0000 7f2ab1228640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a9806a580 0x7f2a9806ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00a830 tx=0x7f2a9c008040 comp rx=0 tx=0).stop 2023-12-01T03:10:32.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.503+0000 7f2ab1228640 1 -- 172.21.15.115:0/1109213799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 msgr2=0x7f2aac0c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:32.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.503+0000 7f2ab1228640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 0x7f2aac0c5950 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000c00 tx=0x7f2aa00046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.504+0000 7f2ab1228640 1 -- 172.21.15.115:0/1109213799 shutdown_connections 2023-12-01T03:10:32.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.504+0000 7f2ab1228640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a9806a580 0x7f2a9806ca40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.504+0000 7f2ab1228640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aac159dd0 0x7f2aac0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.504+0000 7f2ab1228640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2aac0c9ef0 0x7f2aac0c5950 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.504+0000 7f2ab1228640 1 --2- 172.21.15.115:0/1109213799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aac06b6f0 0x7f2aac0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:32.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.504+0000 7f2ab1228640 1 -- 172.21.15.115:0/1109213799 >> 172.21.15.115:0/1109213799 conn(0x7f2aac0ce880 msgr2=0x7f2aac158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:32.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.504+0000 7f2ab1228640 1 -- 172.21.15.115:0/1109213799 shutdown_connections 2023-12-01T03:10:32.506 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:32.504+0000 7f2ab1228640 1 -- 172.21.15.115:0/1109213799 wait complete. 2023-12-01T03:10:32.523 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T03:10:32.524 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T03:10:32.524 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T03:10:32.524 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T03:10:32.524 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T03:10:32.524 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T03:10:32.524 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T03:10:32.524 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T03:10:32.524 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:10:32.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:32.524 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T03:10:32.574 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:10:32.575 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 36724sec preferred_lft 36724sec 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T03:10:32.576 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T03:10:32.577 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:10:32.577 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T03:10:32.577 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T03:10:32.577 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T03:10:32.577 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:10:32.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:32.577 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T03:10:32.666 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T03:10:32.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:32.670 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T03:10:32.725 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T03:10:32.725 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T03:10:32.725 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T03:10:32.725 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T03:10:32.725 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T03:10:32.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:32.726 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T03:10:32.726 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T03:10:32.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T03:10:32.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T03:10:32.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T03:10:32.726 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T03:10:32.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:32.910 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T03:10:32.960 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T03:10:32.960 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T03:10:32.960 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:10:32.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:32.960 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T03:10:33.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T03:10:33.050 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T03:10:33.104 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:10:33.104 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T03:10:33.105 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T03:10:33.155 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T03:10:33.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:33.156 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:10:33.292 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:10:33.292 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:10:33.292 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:10:33.313 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:10:33.313 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:10:33.313 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:10:33.314 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T03:10:33.314 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T03:10:33.339 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:10:33.339 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:10:33.339 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:10:33.339 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T03:10:33.339 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T03:10:33.339 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T03:10:33.339 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T03:10:33.339 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T03:10:33.340 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T03:10:33.341 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T03:10:33.341 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T03:10:33.341 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T03:10:33.341 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T03:10:33.344 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T03:10:33.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:33.345 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T03:10:33.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:33.426 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T03:10:33.502 INFO:teuthology.orchestra.run.smithi115.stdout:client10791 2023-12-01T03:10:33.502 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:10:33.502 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:10:33.559 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-12-01T03:10:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.774+0000 7f7ac9e4d640 1 Processor -- start 2023-12-01T03:10:33.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.774+0000 7f7ac9e4d640 1 -- start start 2023-12-01T03:10:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.775+0000 7f7ac9e4d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7ac41599e0 0x7f7ac4159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.775+0000 7f7ac9e4d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7ac40af370 0x7f7ac40c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.775+0000 7f7ac9e4d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac415e440 con 0x7f7ac40c95a0 2023-12-01T03:10:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.775+0000 7f7ac9e4d640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac415a300 con 0x7f7ac41599e0 2023-12-01T03:10:33.777 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.775+0000 7f7ac9e4d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac415d550 con 0x7f7ac40af370 2023-12-01T03:10:33.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.776+0000 7f7ac8e4b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7ac41599e0 0x7f7ac4159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:33.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.776+0000 7f7ac8e4b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7ac41599e0 0x7f7ac4159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33782/0 (socket says 172.21.15.115:33782) 2023-12-01T03:10:33.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.776+0000 7f7ac8e4b640 1 -- 172.21.15.115:0/1595197264 learned_addr learned my addr 172.21.15.115:0/1595197264 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:33.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.776+0000 7f7ac8e4b640 1 -- 172.21.15.115:0/1595197264 >> v2:172.21.15.142:3300/0 conn(0x7f7ac40af370 msgr2=0x7f7ac40c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:33.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.776+0000 7f7ac8e4b640 1 --2- 172.21.15.115:0/1595197264 >> v2:172.21.15.142:3300/0 conn(0x7f7ac40af370 0x7f7ac40c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:33.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.776+0000 7f7ac8e4b640 1 -- 172.21.15.115:0/1595197264 >> v1:172.21.15.142:6789/0 conn(0x7f7ac40c95a0 legacy=0x7f7ac406f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:33.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.776+0000 7f7ac8e4b640 1 -- 172.21.15.115:0/1595197264 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac415d830 con 0x7f7ac41599e0 2023-12-01T03:10:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.777+0000 7f7ac8e4b640 1 --2- 172.21.15.115:0/1595197264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac41599e0 0x7f7ac4159dc0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40668e0 tx=0x7f7ab802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695d3d8e2b75295e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.777+0000 7f7ac37fe640 1 -- 172.21.15.115:0/1595197264 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8031870 con 0x7f7ac41599e0 2023-12-01T03:10:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.777+0000 7f7ac37fe640 1 -- 172.21.15.115:0/1595197264 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab80319d0 con 0x7f7ac41599e0 2023-12-01T03:10:33.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.777+0000 7f7ac37fe640 1 -- 172.21.15.115:0/1595197264 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8031d60 con 0x7f7ac41599e0 2023-12-01T03:10:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.778+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/1595197264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac41599e0 msgr2=0x7f7ac4159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.778+0000 7f7ac9e4d640 1 --2- 172.21.15.115:0/1595197264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac41599e0 0x7f7ac4159dc0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40668e0 tx=0x7f7ab802b910 comp rx=0 tx=0).stop 2023-12-01T03:10:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.778+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/1595197264 shutdown_connections 2023-12-01T03:10:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.778+0000 7f7ac9e4d640 1 --2- 172.21.15.115:0/1595197264 >> v2:172.21.15.142:3300/0 conn(0x7f7ac40af370 0x7f7ac40c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.778+0000 7f7ac9e4d640 1 --2- 172.21.15.115:0/1595197264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac41599e0 0x7f7ac4159dc0 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.778+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/1595197264 >> 172.21.15.115:0/1595197264 conn(0x7f7ac4065960 msgr2=0x7f7ac4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:33.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.778+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/1595197264 shutdown_connections 2023-12-01T03:10:33.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.778+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/1595197264 wait complete. 2023-12-01T03:10:33.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.780+0000 7f7ac9e4d640 1 Processor -- start 2023-12-01T03:10:33.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.780+0000 7f7ac9e4d640 1 -- start start 2023-12-01T03:10:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.781+0000 7f7ac9e4d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac40af370 0x7f7ac40c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.781+0000 7f7ac9e4d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac40c95a0 0x7f7ac40cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.781+0000 7f7ac9e4d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac41599e0 0x7f7ac40cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.781+0000 7f7ac9e4d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac4144a70 con 0x7f7ac40af370 2023-12-01T03:10:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.781+0000 7f7ac9e4d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac40c4380 con 0x7f7ac40c95a0 2023-12-01T03:10:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.781+0000 7f7ac9e4d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac4144cc0 con 0x7f7ac41599e0 2023-12-01T03:10:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.781+0000 7f7ac8e4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac40af370 0x7f7ac40c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.781+0000 7f7ac8e4b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac40af370 0x7f7ac40c3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33796/0 (socket says 172.21.15.115:33796) 2023-12-01T03:10:33.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.781+0000 7f7ac8e4b640 1 -- 172.21.15.115:0/2256749695 learned_addr learned my addr 172.21.15.115:0/2256749695 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.782+0000 7f7ac3fff640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac40c95a0 0x7f7ac40cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:33.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.782+0000 7f7ac964c640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac41599e0 0x7f7ac40cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.782+0000 7f7ac8e4b640 1 -- 172.21.15.115:0/2256749695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac41599e0 msgr2=0x7f7ac40cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.782+0000 7f7ac8e4b640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac41599e0 0x7f7ac40cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.782+0000 7f7ac8e4b640 1 -- 172.21.15.115:0/2256749695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac40c95a0 msgr2=0x7f7ac40cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.782+0000 7f7ac8e4b640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac40c95a0 0x7f7ac40cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.782+0000 7f7ac8e4b640 1 -- 172.21.15.115:0/2256749695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80094d0 con 0x7f7ac40af370 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.782+0000 7f7ac3fff640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac40c95a0 0x7f7ac40cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.782+0000 7f7ac8e4b640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac40af370 0x7f7ac40c3e40 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8000c00 tx=0x7f7ab80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.782+0000 7f7ac17fa640 1 -- 172.21.15.115:0/2256749695 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab803ad10 con 0x7f7ac40af370 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.783+0000 7f7ac17fa640 1 -- 172.21.15.115:0/2256749695 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab803ae70 con 0x7f7ac40af370 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.783+0000 7f7ac17fa640 1 -- 172.21.15.115:0/2256749695 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8044610 con 0x7f7ac40af370 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.783+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/2256749695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac4144fa0 con 0x7f7ac40af370 2023-12-01T03:10:33.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.783+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/2256749695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac4145560 con 0x7f7ac40af370 2023-12-01T03:10:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.784+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/2256749695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac4145750 con 0x7f7ac40af370 2023-12-01T03:10:33.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.785+0000 7f7ac17fa640 1 -- 172.21.15.115:0/2256749695 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f7ab804d450 con 0x7f7ac40af370 2023-12-01T03:10:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.786+0000 7f7ac17fa640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7aa806a550 0x7f7aa806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.786+0000 7f7ac17fa640 1 -- 172.21.15.115:0/2256749695 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f7ab802d040 con 0x7f7ac40af370 2023-12-01T03:10:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.786+0000 7f7ac3fff640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7aa806a550 0x7f7aa806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:33.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.787+0000 7f7ac3fff640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7aa806a550 0x7f7aa806ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40c5f60 tx=0x7f7ab0008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:33.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:33.792+0000 7f7ac17fa640 1 -- 172.21.15.115:0/2256749695 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7ab8076700 con 0x7f7ac40af370 2023-12-01T03:10:34.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.021+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/2256749695 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7ac40618d0 con 0x7f7ac40af370 2023-12-01T03:10:34.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.446+0000 7f7ac17fa640 1 -- 172.21.15.115:0/2256749695 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v96) v1 ==== 136+0+0 (secure 0 0 0) 0x7f7ab8079d20 con 0x7f7ac40af370 2023-12-01T03:10:34.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.450+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/2256749695 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7aa806a550 msgr2=0x7f7aa806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:34.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.450+0000 7f7ac9e4d640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7aa806a550 0x7f7aa806ca10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40c5f60 tx=0x7f7ab0008040 comp rx=0 tx=0).stop 2023-12-01T03:10:34.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.450+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/2256749695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac40af370 msgr2=0x7f7ac40c3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:34.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.450+0000 7f7ac9e4d640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac40af370 0x7f7ac40c3e40 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8000c00 tx=0x7f7ab80046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.450+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/2256749695 shutdown_connections 2023-12-01T03:10:34.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.450+0000 7f7ac9e4d640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7aa806a550 0x7f7aa806ca10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.450+0000 7f7ac9e4d640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ac41599e0 0x7f7ac40cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.450+0000 7f7ac9e4d640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ac40c95a0 0x7f7ac40cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.450+0000 7f7ac9e4d640 1 --2- 172.21.15.115:0/2256749695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ac40af370 0x7f7ac40c3e40 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.450+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/2256749695 >> 172.21.15.115:0/2256749695 conn(0x7f7ac4065960 msgr2=0x7f7ac406bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:34.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.451+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/2256749695 shutdown_connections 2023-12-01T03:10:34.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.451+0000 7f7ac9e4d640 1 -- 172.21.15.115:0/2256749695 wait complete. 2023-12-01T03:10:34.475 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:34.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.686+0000 7ff73c7b5640 1 Processor -- start 2023-12-01T03:10:34.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.687+0000 7ff73c7b5640 1 -- start start 2023-12-01T03:10:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.688+0000 7ff73c7b5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff7340adac0 0x7ff7340cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.688+0000 7ff73c7b5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff7340c8ed0 0x7ff7340cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.688+0000 7ff73c7b5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340cced0 0x7ff734161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.688+0000 7ff73c7b5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7340cf3a0 con 0x7ff7340c8ed0 2023-12-01T03:10:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.688+0000 7ff73c7b5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7340ce5e0 con 0x7ff7340adac0 2023-12-01T03:10:34.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.688+0000 7ff73c7b5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340ce150 con 0x7ff7340cced0 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff739d50640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff7340c8ed0 0x7ff7340cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff739d50640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff7340c8ed0 0x7ff7340cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33798/0 (socket says 172.21.15.115:33798) 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff739d50640 1 -- 172.21.15.115:0/471352638 learned_addr learned my addr 172.21.15.115:0/471352638 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff73a551640 1 --2- 172.21.15.115:0/471352638 >> v2:172.21.15.142:3300/0 conn(0x7ff7340adac0 0x7ff7340cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff73ad52640 1 --2- 172.21.15.115:0/471352638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340cced0 0x7ff734161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff739d50640 1 -- 172.21.15.115:0/471352638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340cced0 msgr2=0x7ff734161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff739d50640 1 --2- 172.21.15.115:0/471352638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340cced0 0x7ff734161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff739d50640 1 -- 172.21.15.115:0/471352638 >> v2:172.21.15.142:3300/0 conn(0x7ff7340adac0 msgr2=0x7ff7340cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff739d50640 1 --2- 172.21.15.115:0/471352638 >> v2:172.21.15.142:3300/0 conn(0x7ff7340adac0 0x7ff7340cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff739d50640 1 -- 172.21.15.115:0/471352638 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73406d620 con 0x7ff7340c8ed0 2023-12-01T03:10:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.689+0000 7ff73a551640 1 --2- 172.21.15.115:0/471352638 >> v2:172.21.15.142:3300/0 conn(0x7ff7340adac0 0x7ff7340cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:10:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.691+0000 7ff739d50640 1 --2- 172.21.15.115:0/471352638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340c8ed0 0x7ff7340cc990 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7ff7340cb260 tx=0x7ff72402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f064ff092c53cc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.691+0000 7ff73954f640 1 -- 172.21.15.115:0/471352638 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724031870 con 0x7ff7340c8ed0 2023-12-01T03:10:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.691+0000 7ff73954f640 1 -- 172.21.15.115:0/471352638 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7240319d0 con 0x7ff7340c8ed0 2023-12-01T03:10:34.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.691+0000 7ff73954f640 1 -- 172.21.15.115:0/471352638 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724031d60 con 0x7ff7340c8ed0 2023-12-01T03:10:34.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.692+0000 7ff73c7b5640 1 -- 172.21.15.115:0/471352638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340c8ed0 msgr2=0x7ff7340cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:34.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.692+0000 7ff73c7b5640 1 --2- 172.21.15.115:0/471352638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340c8ed0 0x7ff7340cc990 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7ff7340cb260 tx=0x7ff72402b910 comp rx=0 tx=0).stop 2023-12-01T03:10:34.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.692+0000 7ff73c7b5640 1 -- 172.21.15.115:0/471352638 shutdown_connections 2023-12-01T03:10:34.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.692+0000 7ff73c7b5640 1 --2- 172.21.15.115:0/471352638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340cced0 0x7ff734161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.692+0000 7ff73c7b5640 1 --2- 172.21.15.115:0/471352638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340c8ed0 0x7ff7340cc990 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.692+0000 7ff73c7b5640 1 --2- 172.21.15.115:0/471352638 >> v2:172.21.15.142:3300/0 conn(0x7ff7340adac0 0x7ff7340cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.692+0000 7ff73c7b5640 1 -- 172.21.15.115:0/471352638 >> 172.21.15.115:0/471352638 conn(0x7ff7340ce880 msgr2=0x7ff7340cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:34.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.692+0000 7ff73c7b5640 1 -- 172.21.15.115:0/471352638 shutdown_connections 2023-12-01T03:10:34.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.692+0000 7ff73c7b5640 1 -- 172.21.15.115:0/471352638 wait complete. 2023-12-01T03:10:34.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.694+0000 7ff73c7b5640 1 Processor -- start 2023-12-01T03:10:34.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.694+0000 7ff73c7b5640 1 -- start start 2023-12-01T03:10:34.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff73c7b5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7340adac0 0x7ff73414e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:34.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff73c7b5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340c8ed0 0x7ff7340c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:34.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff73c7b5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340cced0 0x7ff734151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff73c7b5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff734151760 con 0x7ff7340cced0 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff73c7b5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340c5fe0 con 0x7ff7340adac0 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff73c7b5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340c62c0 con 0x7ff7340c8ed0 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff73ad52640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340cced0 0x7ff734151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff73ad52640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340cced0 0x7ff734151220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33806/0 (socket says 172.21.15.115:33806) 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff73ad52640 1 -- 172.21.15.115:0/2845403718 learned_addr learned my addr 172.21.15.115:0/2845403718 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff73a551640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7340adac0 0x7ff73414e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.695+0000 7ff739d50640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340c8ed0 0x7ff7340c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff73ad52640 1 -- 172.21.15.115:0/2845403718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340c8ed0 msgr2=0x7ff7340c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff73ad52640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340c8ed0 0x7ff7340c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff73ad52640 1 -- 172.21.15.115:0/2845403718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7340adac0 msgr2=0x7ff73414e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff73ad52640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7340adac0 0x7ff73414e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff73ad52640 1 -- 172.21.15.115:0/2845403718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240094d0 con 0x7ff7340cced0 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff739d50640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340c8ed0 0x7ff7340c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:34.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff73ad52640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340cced0 0x7ff734151220 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7ff72c007e40 tx=0x7ff72c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff72affd640 1 -- 172.21.15.115:0/2845403718 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c014620 con 0x7ff7340cced0 2023-12-01T03:10:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff72affd640 1 -- 172.21.15.115:0/2845403718 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c00ee90 con 0x7ff7340cced0 2023-12-01T03:10:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff73c7b5640 1 -- 172.21.15.115:0/2845403718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7340c65a0 con 0x7ff7340cced0 2023-12-01T03:10:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff72affd640 1 -- 172.21.15.115:0/2845403718 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0149e0 con 0x7ff7340cced0 2023-12-01T03:10:34.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.696+0000 7ff73c7b5640 1 -- 172.21.15.115:0/2845403718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73414e760 con 0x7ff7340cced0 2023-12-01T03:10:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.697+0000 7ff73c7b5640 1 -- 172.21.15.115:0/2845403718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73404e890 con 0x7ff7340cced0 2023-12-01T03:10:34.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.698+0000 7ff72affd640 1 -- 172.21.15.115:0/2845403718 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff72c014b40 con 0x7ff7340cced0 2023-12-01T03:10:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.699+0000 7ff72affd640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff71806a550 0x7ff71806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.699+0000 7ff72affd640 1 -- 172.21.15.115:0/2845403718 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7ff72c089510 con 0x7ff7340cced0 2023-12-01T03:10:34.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.699+0000 7ff73a551640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff71806a550 0x7ff71806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:34.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.700+0000 7ff73a551640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff71806a550 0x7ff71806ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff730006fd0 tx=0x7ff730008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:34.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.704+0000 7ff72affd640 1 -- 172.21.15.115:0/2845403718 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff72c054920 con 0x7ff7340cced0 2023-12-01T03:10:34.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.895+0000 7ff73c7b5640 1 -- 172.21.15.115:0/2845403718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7340b4a10 con 0x7ff7340cced0 2023-12-01T03:10:34.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.898+0000 7ff72affd640 1 -- 172.21.15.115:0/2845403718 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v214) v1 ==== 74+0+16173 (secure 0 0 0) 0x7ff72c057f40 con 0x7ff7340cced0 2023-12-01T03:10:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:34.900 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":214,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:26.151466+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:34.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 -- 172.21.15.115:0/2845403718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff71806a550 msgr2=0x7ff71806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:34.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff71806a550 0x7ff71806ca10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff730006fd0 tx=0x7ff730008040 comp rx=0 tx=0).stop 2023-12-01T03:10:34.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 -- 172.21.15.115:0/2845403718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340cced0 msgr2=0x7ff734151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:34.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340cced0 0x7ff734151220 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7ff72c007e40 tx=0x7ff72c00e4b0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 -- 172.21.15.115:0/2845403718 shutdown_connections 2023-12-01T03:10:34.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff71806a550 0x7ff71806ca10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7340c8ed0 0x7ff7340c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7340cced0 0x7ff734151220 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 --2- 172.21.15.115:0/2845403718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7340adac0 0x7ff73414e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:34.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 -- 172.21.15.115:0/2845403718 >> 172.21.15.115:0/2845403718 conn(0x7ff7340ce880 msgr2=0x7ff7340aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:34.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.900+0000 7ff73c7b5640 1 -- 172.21.15.115:0/2845403718 shutdown_connections 2023-12-01T03:10:34.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:34.901+0000 7ff73c7b5640 1 -- 172.21.15.115:0/2845403718 wait complete. 2023-12-01T03:10:34.922 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T03:10:22.683496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '213', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T03:10:24.156854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '214', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:10:34.923 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:10:35.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.138+0000 7f1961970640 1 Processor -- start 2023-12-01T03:10:35.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.139+0000 7f1961970640 1 -- start start 2023-12-01T03:10:35.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.140+0000 7f1961970640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f195c153c30 0x7f195c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.140+0000 7f1961970640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f195c154010 0x7f195c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.140+0000 7f1961970640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c0cf9b0 0x7f195c160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.140+0000 7f1961970640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c0cdcd0 con 0x7f195c153c30 2023-12-01T03:10:35.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.140+0000 7f1961970640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c0cfe20 con 0x7f195c154010 2023-12-01T03:10:35.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.140+0000 7f1961970640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c161830 con 0x7f195c0cf9b0 2023-12-01T03:10:35.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f196096e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f195c153c30 0x7f195c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f196096e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f195c153c30 0x7f195c0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33818/0 (socket says 172.21.15.115:33818) 2023-12-01T03:10:35.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f196096e640 1 -- 172.21.15.115:0/1235890761 learned_addr learned my addr 172.21.15.115:0/1235890761 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:35.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f195bfff640 1 --2- 172.21.15.115:0/1235890761 >> v2:172.21.15.142:3300/0 conn(0x7f195c154010 0x7f195c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f196116f640 1 --2- 172.21.15.115:0/1235890761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c0cf9b0 0x7f195c160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f196096e640 1 -- 172.21.15.115:0/1235890761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c0cf9b0 msgr2=0x7f195c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:35.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f196096e640 1 --2- 172.21.15.115:0/1235890761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c0cf9b0 0x7f195c160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f196096e640 1 -- 172.21.15.115:0/1235890761 >> v2:172.21.15.142:3300/0 conn(0x7f195c154010 msgr2=0x7f195c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:35.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f196096e640 1 --2- 172.21.15.115:0/1235890761 >> v2:172.21.15.142:3300/0 conn(0x7f195c154010 0x7f195c0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f196096e640 1 -- 172.21.15.115:0/1235890761 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0cd3a0 con 0x7f195c153c30 2023-12-01T03:10:35.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.141+0000 7f196116f640 1 --2- 172.21.15.115:0/1235890761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c0cf9b0 0x7f195c160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:35.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.143+0000 7f196096e640 1 --2- 172.21.15.115:0/1235890761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c153c30 0x7f195c0ce160 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f195c0d3520 tx=0x7f194c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27cd0b356e2a09a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:35.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.143+0000 7f195b7fe640 1 -- 172.21.15.115:0/1235890761 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c02ca80 con 0x7f195c153c30 2023-12-01T03:10:35.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.143+0000 7f195b7fe640 1 -- 172.21.15.115:0/1235890761 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c02cbe0 con 0x7f195c153c30 2023-12-01T03:10:35.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.143+0000 7f195b7fe640 1 -- 172.21.15.115:0/1235890761 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c0343f0 con 0x7f195c153c30 2023-12-01T03:10:35.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.144+0000 7f1961970640 1 -- 172.21.15.115:0/1235890761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c153c30 msgr2=0x7f195c0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.144+0000 7f1961970640 1 --2- 172.21.15.115:0/1235890761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c153c30 0x7f195c0ce160 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f195c0d3520 tx=0x7f194c02bee0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.144+0000 7f1961970640 1 -- 172.21.15.115:0/1235890761 shutdown_connections 2023-12-01T03:10:35.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.144+0000 7f1961970640 1 --2- 172.21.15.115:0/1235890761 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c0cf9b0 0x7f195c160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.144+0000 7f1961970640 1 --2- 172.21.15.115:0/1235890761 >> v2:172.21.15.142:3300/0 conn(0x7f195c154010 0x7f195c0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.144+0000 7f1961970640 1 --2- 172.21.15.115:0/1235890761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c153c30 0x7f195c0ce160 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.144+0000 7f1961970640 1 -- 172.21.15.115:0/1235890761 >> 172.21.15.115:0/1235890761 conn(0x7f195c06a0b0 msgr2=0x7f195c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:35.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.144+0000 7f1961970640 1 -- 172.21.15.115:0/1235890761 shutdown_connections 2023-12-01T03:10:35.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.144+0000 7f1961970640 1 -- 172.21.15.115:0/1235890761 wait complete. 2023-12-01T03:10:35.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.146+0000 7f1961970640 1 Processor -- start 2023-12-01T03:10:35.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.146+0000 7f1961970640 1 -- start start 2023-12-01T03:10:35.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.147+0000 7f1961970640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c0cf9b0 0x7f195c0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.147+0000 7f1961970640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c153c30 0x7f195c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.147+0000 7f1961970640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c154010 0x7f195c155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.147+0000 7f1961970640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c154cd0 con 0x7f195c154010 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.147+0000 7f1961970640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c154fb0 con 0x7f195c0cf9b0 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.147+0000 7f1961970640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c155290 con 0x7f195c153c30 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.147+0000 7f196116f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c154010 0x7f195c155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.147+0000 7f196116f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c154010 0x7f195c155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33830/0 (socket says 172.21.15.115:33830) 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.147+0000 7f196116f640 1 -- 172.21.15.115:0/326544575 learned_addr learned my addr 172.21.15.115:0/326544575 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.147+0000 7f196096e640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c0cf9b0 0x7f195c0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f195bfff640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c153c30 0x7f195c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f196116f640 1 -- 172.21.15.115:0/326544575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c153c30 msgr2=0x7f195c1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f196116f640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c153c30 0x7f195c1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f196116f640 1 -- 172.21.15.115:0/326544575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c0cf9b0 msgr2=0x7f195c0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f196116f640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c0cf9b0 0x7f195c0cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f196116f640 1 -- 172.21.15.115:0/326544575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c009b30 con 0x7f195c154010 2023-12-01T03:10:35.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f196116f640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c154010 0x7f195c155d70 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f195400c7f0 tx=0x7f195400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:35.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f19597fa640 1 -- 172.21.15.115:0/326544575 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954007820 con 0x7f195c154010 2023-12-01T03:10:35.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f19597fa640 1 -- 172.21.15.115:0/326544575 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1954007980 con 0x7f195c154010 2023-12-01T03:10:35.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f1961970640 1 -- 172.21.15.115:0/326544575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c155570 con 0x7f195c154010 2023-12-01T03:10:35.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.148+0000 7f19597fa640 1 -- 172.21.15.115:0/326544575 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954007d30 con 0x7f195c154010 2023-12-01T03:10:35.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.149+0000 7f1961970640 1 -- 172.21.15.115:0/326544575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c1549e0 con 0x7f195c154010 2023-12-01T03:10:35.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.149+0000 7f1961970640 1 -- 172.21.15.115:0/326544575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195c155a90 con 0x7f195c154010 2023-12-01T03:10:35.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.151+0000 7f19597fa640 1 -- 172.21.15.115:0/326544575 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f19540113f0 con 0x7f195c154010 2023-12-01T03:10:35.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.151+0000 7f19597fa640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f193c06a580 0x7f193c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.152+0000 7f19597fa640 1 -- 172.21.15.115:0/326544575 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f1954088f40 con 0x7f195c154010 2023-12-01T03:10:35.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.152+0000 7f196096e640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f193c06a580 0x7f193c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.153+0000 7f196096e640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f193c06a580 0x7f193c06ca40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f194c035000 tx=0x7f194c030000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:35.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.156+0000 7f19597fa640 1 -- 172.21.15.115:0/326544575 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1954054300 con 0x7f195c154010 2023-12-01T03:10:35.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.344+0000 7f1961970640 1 -- 172.21.15.115:0/326544575 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f195c0cd650 con 0x7f195c154010 2023-12-01T03:10:35.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.345+0000 7f19597fa640 1 -- 172.21.15.115:0/326544575 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v214) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1954057920 con 0x7f195c154010 2023-12-01T03:10:35.347 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:10:35.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 -- 172.21.15.115:0/326544575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f193c06a580 msgr2=0x7f193c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f193c06a580 0x7f193c06ca40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f194c035000 tx=0x7f194c030000 comp rx=0 tx=0).stop 2023-12-01T03:10:35.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 -- 172.21.15.115:0/326544575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c154010 msgr2=0x7f195c155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c154010 0x7f195c155d70 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f195400c7f0 tx=0x7f195400ccb0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 -- 172.21.15.115:0/326544575 shutdown_connections 2023-12-01T03:10:35.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f193c06a580 0x7f193c06ca40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f195c153c30 0x7f195c1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f195c154010 0x7f195c155d70 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 --2- 172.21.15.115:0/326544575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c0cf9b0 0x7f195c0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 -- 172.21.15.115:0/326544575 >> 172.21.15.115:0/326544575 conn(0x7f195c06a0b0 msgr2=0x7f195c15a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:35.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 -- 172.21.15.115:0/326544575 shutdown_connections 2023-12-01T03:10:35.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.348+0000 7f1961970640 1 -- 172.21.15.115:0/326544575 wait complete. 2023-12-01T03:10:35.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:10:35.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.581+0000 7fcabaa69640 1 Processor -- start 2023-12-01T03:10:35.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.581+0000 7fcabaa69640 1 -- start start 2023-12-01T03:10:35.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.583+0000 7fcabaa69640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41590b0 0x7fcab4159490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.583+0000 7fcabaa69640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcab41599d0 0x7fcab40b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.583+0000 7fcabaa69640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab40cac30 con 0x7fcab40b1c50 2023-12-01T03:10:35.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.583+0000 7fcabaa69640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab40b2140 con 0x7fcab41599d0 2023-12-01T03:10:35.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.583+0000 7fcabaa69640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b2420 con 0x7fcab41590b0 2023-12-01T03:10:35.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.583+0000 7fcab9006640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fcab40b1c50 0x7fcab406baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37232/0 (socket says 172.21.15.115:37232) 2023-12-01T03:10:35.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.583+0000 7fcab9006640 1 -- 172.21.15.115:0/1396252209 learned_addr learned my addr 172.21.15.115:0/1396252209 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:35.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.583+0000 7fcab8805640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41590b0 0x7fcab4159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.583+0000 7fcab3fff640 1 --2- 172.21.15.115:0/1396252209 >> v2:172.21.15.142:3300/0 conn(0x7fcab41599d0 0x7fcab40b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.583+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130570292 0 0) 0x7fcab40cac30 con 0x7fcab40b1c50 2023-12-01T03:10:35.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa4003520 con 0x7fcab40b1c50 2023-12-01T03:10:35.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4099625457 0 0) 0x7fcaa4003520 con 0x7fcab40b1c50 2023-12-01T03:10:35.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41590b0 msgr2=0x7fcab4159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41590b0 0x7fcab4159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 >> v2:172.21.15.142:3300/0 conn(0x7fcab41599d0 msgr2=0x7fcab40b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 --2- 172.21.15.115:0/1396252209 >> v2:172.21.15.142:3300/0 conn(0x7fcab41599d0 0x7fcab40b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab40caed0 con 0x7fcab40b1c50 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fcaac0019f0 con 0x7fcab40b1c50 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 >> v1:172.21.15.115:6789/0 conn(0x7fcab40b1c50 legacy=0x7fcab406baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41590b0 0x7fcaa4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaa4003f40 0x7fcaa40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 0x7fcaa4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b2140 con 0x7fcaa4004d00 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b2420 con 0x7fcaa4003f40 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa400b980 con 0x7fcab41590b0 2023-12-01T03:10:35.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.584+0000 7fcab9006640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 0x7fcaa4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.585+0000 7fcab3fff640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaa4003f40 0x7fcaa40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.585+0000 7fcab8805640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41590b0 0x7fcaa4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.585+0000 7fcab9006640 1 -- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41590b0 msgr2=0x7fcaa4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:35.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.585+0000 7fcab9006640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41590b0 0x7fcaa4003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.585+0000 7fcab9006640 1 -- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaa4003f40 msgr2=0x7fcaa40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:35.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.585+0000 7fcab9006640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaa4003f40 0x7fcaa40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.585+0000 7fcab9006640 1 -- 172.21.15.115:0/1396252209 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa4003520 con 0x7fcaa4004d00 2023-12-01T03:10:35.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.585+0000 7fcab3fff640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcaa4003f40 0x7fcaa40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.586+0000 7fcab9006640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 0x7fcaa4005180 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00a840 tx=0x7fcaa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb1001e943a4405 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.586+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac034630 con 0x7fcaa4004d00 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.586+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac02f070 con 0x7fcaa4004d00 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.586+0000 7fcab37fe640 1 -- 172.21.15.115:0/1396252209 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac034a30 con 0x7fcaa4004d00 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.587+0000 7fcabaa69640 1 -- 172.21.15.115:0/1396252209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 msgr2=0x7fcaa4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.587+0000 7fcabaa69640 1 --2- 172.21.15.115:0/1396252209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 0x7fcaa4005180 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00a840 tx=0x7fcaa40057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.587+0000 7fcab9006640 1 -- 172.21.15.115:0/1396252209 reap_dead start 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.587+0000 7fcabaa69640 1 -- 172.21.15.115:0/1396252209 shutdown_connections 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.587+0000 7fcabaa69640 1 -- 172.21.15.115:0/1396252209 >> 172.21.15.115:0/1396252209 conn(0x7fcab4065960 msgr2=0x7fcab4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.587+0000 7fcabaa69640 1 -- 172.21.15.115:0/1396252209 shutdown_connections 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.587+0000 7fcabaa69640 1 -- 172.21.15.115:0/1396252209 wait complete. 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.589+0000 7fcabaa69640 1 Processor -- start 2023-12-01T03:10:35.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.589+0000 7fcabaa69640 1 -- start start 2023-12-01T03:10:35.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcabaa69640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 0x7fcab41f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcabaa69640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41510f0 0x7fcab41e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcabaa69640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab41535b0 0x7fcab4156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcabaa69640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4151570 con 0x7fcaa4004d00 2023-12-01T03:10:35.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcabaa69640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4153e30 con 0x7fcab41535b0 2023-12-01T03:10:35.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcabaa69640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4154110 con 0x7fcab41510f0 2023-12-01T03:10:35.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcab8805640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 0x7fcab41f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcab8805640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 0x7fcab41f2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33844/0 (socket says 172.21.15.115:33844) 2023-12-01T03:10:35.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcab8805640 1 -- 172.21.15.115:0/3583070810 learned_addr learned my addr 172.21.15.115:0/3583070810 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:35.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcab8805640 1 -- 172.21.15.115:0/3583070810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41510f0 msgr2=0x7fcab41e6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcab9006640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab41535b0 0x7fcab4156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcab8805640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41510f0 0x7fcab41e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcab8805640 1 -- 172.21.15.115:0/3583070810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab41535b0 msgr2=0x7fcab4156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcab8805640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab41535b0 0x7fcab4156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.590+0000 7fcab8805640 1 -- 172.21.15.115:0/3583070810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab41f26b0 con 0x7fcaa4004d00 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.591+0000 7fcab8805640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 0x7fcab41f2eb0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8010860 tx=0x7fcaa8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.591+0000 7fcab17fa640 1 -- 172.21.15.115:0/3583070810 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8009610 con 0x7fcaa4004d00 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.591+0000 7fcab17fa640 1 -- 172.21.15.115:0/3583070810 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa800eb00 con 0x7fcaa4004d00 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.591+0000 7fcab17fa640 1 -- 172.21.15.115:0/3583070810 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa800ee90 con 0x7fcaa4004d00 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.591+0000 7fcabaa69640 1 -- 172.21.15.115:0/3583070810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab41f2990 con 0x7fcaa4004d00 2023-12-01T03:10:35.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.591+0000 7fcabaa69640 1 -- 172.21.15.115:0/3583070810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab4153a60 con 0x7fcaa4004d00 2023-12-01T03:10:35.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.592+0000 7fcabaa69640 1 -- 172.21.15.115:0/3583070810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab4073950 con 0x7fcaa4004d00 2023-12-01T03:10:35.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.594+0000 7fcab17fa640 1 -- 172.21.15.115:0/3583070810 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fcaa801c400 con 0x7fcaa4004d00 2023-12-01T03:10:35.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.594+0000 7fcab17fa640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fca9806a550 0x7fca9806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:35.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.595+0000 7fcab17fa640 1 -- 172.21.15.115:0/3583070810 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fcaa8089040 con 0x7fcaa4004d00 2023-12-01T03:10:35.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.595+0000 7fcab3fff640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fca9806a550 0x7fca9806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:35.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.595+0000 7fcab3fff640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fca9806a550 0x7fca9806ca10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcab41566c0 tx=0x7fcaa0006f90 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:35.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.599+0000 7fcab17fa640 1 -- 172.21.15.115:0/3583070810 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcaa80544f0 con 0x7fcaa4004d00 2023-12-01T03:10:35.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.788+0000 7fcabaa69640 1 -- 172.21.15.115:0/3583070810 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcab40cd6c0 con 0x7fcaa4004d00 2023-12-01T03:10:35.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.789+0000 7fcab17fa640 1 -- 172.21.15.115:0/3583070810 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v214) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcaa8057b10 con 0x7fcaa4004d00 2023-12-01T03:10:35.791 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:10:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 -- 172.21.15.115:0/3583070810 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fca9806a550 msgr2=0x7fca9806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fca9806a550 0x7fca9806ca10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcab41566c0 tx=0x7fcaa0006f90 comp rx=0 tx=0).stop 2023-12-01T03:10:35.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 -- 172.21.15.115:0/3583070810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 msgr2=0x7fcab41f2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:35.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 0x7fcab41f2eb0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8010860 tx=0x7fcaa8010d20 comp rx=0 tx=0).stop 2023-12-01T03:10:35.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 -- 172.21.15.115:0/3583070810 shutdown_connections 2023-12-01T03:10:35.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fca9806a550 0x7fca9806ca10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcab41510f0 0x7fcab41e6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab41535b0 0x7fcab4156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 --2- 172.21.15.115:0/3583070810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcaa4004d00 0x7fcab41f2eb0 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:35.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 -- 172.21.15.115:0/3583070810 >> 172.21.15.115:0/3583070810 conn(0x7fcab4065960 msgr2=0x7fcab40b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:35.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 -- 172.21.15.115:0/3583070810 shutdown_connections 2023-12-01T03:10:35.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:35.792+0000 7fcabaa69640 1 -- 172.21.15.115:0/3583070810 wait complete. 2023-12-01T03:10:35.813 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:10:36.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.027+0000 7f75a505f640 1 Processor -- start 2023-12-01T03:10:36.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.028+0000 7f75a505f640 1 -- start start 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.029+0000 7f75a505f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a0142870 0x7f75a00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.029+0000 7f75a505f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f75a00b1e20 0x7f75a015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.029+0000 7f75a505f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a00cd470 con 0x7f75a00cd010 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.029+0000 7f75a505f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a00b3ae0 con 0x7f75a00b1e20 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.029+0000 7f75a505f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a0142c50 con 0x7f75a0142870 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.030+0000 7f759f59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f75a00b1e20 0x7f75a015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.030+0000 7f759f59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f75a00b1e20 0x7f75a015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33854/0 (socket says 172.21.15.115:33854) 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.030+0000 7f759f59e640 1 -- 172.21.15.115:0/246716118 learned_addr learned my addr 172.21.15.115:0/246716118 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.030+0000 7f7597fff640 1 --1- 172.21.15.115:0/246716118 >> v1:172.21.15.115:6789/0 conn(0x7f75a00cd010 0x7f75a00b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37240/0 (socket says 172.21.15.115:37240) 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.030+0000 7f759ed9d640 1 --2- 172.21.15.115:0/246716118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a0142870 0x7f75a00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.030+0000 7f759f59e640 1 -- 172.21.15.115:0/246716118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a0142870 msgr2=0x7f75a00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:36.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.030+0000 7f759f59e640 1 --2- 172.21.15.115:0/246716118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a0142870 0x7f75a00ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.030+0000 7f759f59e640 1 -- 172.21.15.115:0/246716118 >> v1:172.21.15.115:6789/0 conn(0x7f75a00cd010 legacy=0x7f75a00b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.030+0000 7f759f59e640 1 -- 172.21.15.115:0/246716118 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00b2f30 con 0x7f75a00b1e20 2023-12-01T03:10:36.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.030+0000 7f759ed9d640 1 --2- 172.21.15.115:0/246716118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a0142870 0x7f75a00ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:36.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.031+0000 7f759f59e640 1 --2- 172.21.15.115:0/246716118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a00b1e20 0x7f75a015a0d0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f7598009e80 tx=0x7f759802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13973fd20f613e73 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:36.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.031+0000 7f759e59c640 1 -- 172.21.15.115:0/246716118 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759802cae0 con 0x7f75a00b1e20 2023-12-01T03:10:36.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.031+0000 7f759e59c640 1 -- 172.21.15.115:0/246716118 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759802cc40 con 0x7f75a00b1e20 2023-12-01T03:10:36.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.031+0000 7f759e59c640 1 -- 172.21.15.115:0/246716118 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75980343f0 con 0x7f75a00b1e20 2023-12-01T03:10:36.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.032+0000 7f75a505f640 1 -- 172.21.15.115:0/246716118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a00b1e20 msgr2=0x7f75a015a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.032+0000 7f75a505f640 1 --2- 172.21.15.115:0/246716118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a00b1e20 0x7f75a015a0d0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f7598009e80 tx=0x7f759802bf40 comp rx=0 tx=0).stop 2023-12-01T03:10:36.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.032+0000 7f75a505f640 1 -- 172.21.15.115:0/246716118 shutdown_connections 2023-12-01T03:10:36.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.032+0000 7f75a505f640 1 --2- 172.21.15.115:0/246716118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a00b1e20 0x7f75a015a0d0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.032+0000 7f75a505f640 1 --2- 172.21.15.115:0/246716118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a0142870 0x7f75a00ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.032+0000 7f75a505f640 1 -- 172.21.15.115:0/246716118 >> 172.21.15.115:0/246716118 conn(0x7f75a0065960 msgr2=0x7f75a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:36.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.032+0000 7f75a505f640 1 -- 172.21.15.115:0/246716118 shutdown_connections 2023-12-01T03:10:36.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.032+0000 7f75a505f640 1 -- 172.21.15.115:0/246716118 wait complete. 2023-12-01T03:10:36.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.034+0000 7f75a505f640 1 Processor -- start 2023-12-01T03:10:36.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.034+0000 7f75a505f640 1 -- start start 2023-12-01T03:10:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.035+0000 7f75a505f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a00b1e20 0x7f75a01edc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.035+0000 7f75a505f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a00cd010 0x7f75a015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.035+0000 7f75a505f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a0142870 0x7f75a00ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.035+0000 7f75a505f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a015d890 con 0x7f75a0142870 2023-12-01T03:10:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.035+0000 7f75a505f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00a9700 con 0x7f75a00b1e20 2023-12-01T03:10:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.035+0000 7f75a505f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a01ee180 con 0x7f75a00cd010 2023-12-01T03:10:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.035+0000 7f759f59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a0142870 0x7f75a00ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.035+0000 7f759f59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a0142870 0x7f75a00ab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33864/0 (socket says 172.21.15.115:33864) 2023-12-01T03:10:36.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.035+0000 7f759f59e640 1 -- 172.21.15.115:0/640950650 learned_addr learned my addr 172.21.15.115:0/640950650 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:36.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.036+0000 7f759f59e640 1 -- 172.21.15.115:0/640950650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a00cd010 msgr2=0x7f75a015c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.036+0000 7f759f59e640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a00cd010 0x7f75a015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.036+0000 7f759f59e640 1 -- 172.21.15.115:0/640950650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a00b1e20 msgr2=0x7f75a01edc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.036+0000 7f759ed9d640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a00b1e20 0x7f75a01edc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.036+0000 7f759f59e640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a00b1e20 0x7f75a01edc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.036+0000 7f759f59e640 1 -- 172.21.15.115:0/640950650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7598009b30 con 0x7f75a0142870 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.036+0000 7f759ed9d640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a00b1e20 0x7f75a01edc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.036+0000 7f759f59e640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a0142870 0x7f75a00ab9c0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f7598009e80 tx=0x7f7598004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.036+0000 7f75977fe640 1 -- 172.21.15.115:0/640950650 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598009510 con 0x7f75a0142870 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.037+0000 7f75977fe640 1 -- 172.21.15.115:0/640950650 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759803e450 con 0x7f75a0142870 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.037+0000 7f75a505f640 1 -- 172.21.15.115:0/640950650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a015d2d0 con 0x7f75a0142870 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.037+0000 7f75977fe640 1 -- 172.21.15.115:0/640950650 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759803e800 con 0x7f75a0142870 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.037+0000 7f75a505f640 1 -- 172.21.15.115:0/640950650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a00a9390 con 0x7f75a0142870 2023-12-01T03:10:36.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.038+0000 7f75a505f640 1 -- 172.21.15.115:0/640950650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a00a91f0 con 0x7f75a0142870 2023-12-01T03:10:36.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.039+0000 7f75977fe640 1 -- 172.21.15.115:0/640950650 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f759803e960 con 0x7f75a0142870 2023-12-01T03:10:36.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.040+0000 7f75977fe640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f758006a580 0x7f758006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.040+0000 7f75977fe640 1 -- 172.21.15.115:0/640950650 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f75980ab970 con 0x7f75a0142870 2023-12-01T03:10:36.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.040+0000 7f759ed9d640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f758006a580 0x7f758006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.041+0000 7f759ed9d640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f758006a580 0x7f758006ca40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7590006fd0 tx=0x7f7590008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:36.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.045+0000 7f75977fe640 1 -- 172.21.15.115:0/640950650 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f75a00a91f0 con 0x7f75a0142870 2023-12-01T03:10:36.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.234+0000 7f75a505f640 1 -- 172.21.15.115:0/640950650 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f75a00a91f0 con 0x7f75a0142870 2023-12-01T03:10:36.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.235+0000 7f75977fe640 1 -- 172.21.15.115:0/640950650 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v214) v1 ==== 100+0+11 (secure 0 0 0) 0x7f75a00a91f0 con 0x7f75a0142870 2023-12-01T03:10:36.238 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:10:36.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.237+0000 7f75a505f640 1 -- 172.21.15.115:0/640950650 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f758006a580 msgr2=0x7f758006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f758006a580 0x7f758006ca40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7590006fd0 tx=0x7f7590008040 comp rx=0 tx=0).stop 2023-12-01T03:10:36.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 -- 172.21.15.115:0/640950650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a0142870 msgr2=0x7f75a00ab9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a0142870 0x7f75a00ab9c0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f7598009e80 tx=0x7f7598004a40 comp rx=0 tx=0).stop 2023-12-01T03:10:36.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 -- 172.21.15.115:0/640950650 shutdown_connections 2023-12-01T03:10:36.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f758006a580 0x7f758006ca40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75a00cd010 0x7f75a015c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75a0142870 0x7f75a00ab9c0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 --2- 172.21.15.115:0/640950650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75a00b1e20 0x7f75a01edc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 -- 172.21.15.115:0/640950650 >> 172.21.15.115:0/640950650 conn(0x7f75a0065960 msgr2=0x7f75a0070550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:36.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 -- 172.21.15.115:0/640950650 shutdown_connections 2023-12-01T03:10:36.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.238+0000 7f75a505f640 1 -- 172.21.15.115:0/640950650 wait complete. 2023-12-01T03:10:36.258 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T03:10:36.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.469+0000 7ff874c0e640 1 Processor -- start 2023-12-01T03:10:36.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.470+0000 7ff874c0e640 1 -- start start 2023-12-01T03:10:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.471+0000 7ff874c0e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff8700b0e70 0x7ff8700c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.471+0000 7ff874c0e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff8700b3ba0 0x7ff8700b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.471+0000 7ff874c0e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87006b1c0 con 0x7ff87006b6a0 2023-12-01T03:10:36.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.471+0000 7ff874c0e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8700c8af0 con 0x7ff8700b3ba0 2023-12-01T03:10:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.471+0000 7ff874c0e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87015f630 con 0x7ff8700b0e70 2023-12-01T03:10:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.471+0000 7ff86ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff87006b6a0 0x7ff87006baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37254/0 (socket says 172.21.15.115:37254) 2023-12-01T03:10:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.471+0000 7ff86ffff640 1 -- 172.21.15.115:0/1164175300 learned_addr learned my addr 172.21.15.115:0/1164175300 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.472+0000 7ff867fff640 1 --2- 172.21.15.115:0/1164175300 >> v2:172.21.15.115:3300/0 conn(0x7ff8700b3ba0 0x7ff8700b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.472+0000 7ff86f7fe640 1 --2- 172.21.15.115:0/1164175300 >> v2:172.21.15.142:3300/0 conn(0x7ff8700b0e70 0x7ff8700c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.472+0000 7ff86effd640 1 -- 172.21.15.115:0/1164175300 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103762827 0 0) 0x7ff87006b1c0 con 0x7ff87006b6a0 2023-12-01T03:10:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.472+0000 7ff86effd640 1 -- 172.21.15.115:0/1164175300 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff85c003520 con 0x7ff87006b6a0 2023-12-01T03:10:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.472+0000 7ff867fff640 1 -- 172.21.15.115:0/1164175300 >> v2:172.21.15.142:3300/0 conn(0x7ff8700b0e70 msgr2=0x7ff8700c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.472+0000 7ff867fff640 1 --2- 172.21.15.115:0/1164175300 >> v2:172.21.15.142:3300/0 conn(0x7ff8700b0e70 0x7ff8700c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.472+0000 7ff867fff640 1 -- 172.21.15.115:0/1164175300 >> v1:172.21.15.115:6789/0 conn(0x7ff87006b6a0 legacy=0x7ff87006baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.472+0000 7ff867fff640 1 -- 172.21.15.115:0/1164175300 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8700b2040 con 0x7ff8700b3ba0 2023-12-01T03:10:36.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.473+0000 7ff867fff640 1 --2- 172.21.15.115:0/1164175300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b3ba0 0x7ff8700b3fe0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7ff8700ce480 tx=0x7ff85802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed19dec5c67d3aee server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:36.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff86effd640 1 -- 172.21.15.115:0/1164175300 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85802e070 con 0x7ff8700b3ba0 2023-12-01T03:10:36.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff86effd640 1 -- 172.21.15.115:0/1164175300 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8580315a0 con 0x7ff8700b3ba0 2023-12-01T03:10:36.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff86effd640 1 -- 172.21.15.115:0/1164175300 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858031930 con 0x7ff8700b3ba0 2023-12-01T03:10:36.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff874c0e640 1 -- 172.21.15.115:0/1164175300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b3ba0 msgr2=0x7ff8700b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff874c0e640 1 --2- 172.21.15.115:0/1164175300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b3ba0 0x7ff8700b3fe0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7ff8700ce480 tx=0x7ff85802b840 comp rx=0 tx=0).stop 2023-12-01T03:10:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff874c0e640 1 -- 172.21.15.115:0/1164175300 shutdown_connections 2023-12-01T03:10:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff874c0e640 1 --2- 172.21.15.115:0/1164175300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b3ba0 0x7ff8700b3fe0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff874c0e640 1 --2- 172.21.15.115:0/1164175300 >> v2:172.21.15.142:3300/0 conn(0x7ff8700b0e70 0x7ff8700c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff874c0e640 1 -- 172.21.15.115:0/1164175300 >> 172.21.15.115:0/1164175300 conn(0x7ff8700ce620 msgr2=0x7ff8700cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff874c0e640 1 -- 172.21.15.115:0/1164175300 shutdown_connections 2023-12-01T03:10:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.474+0000 7ff874c0e640 1 -- 172.21.15.115:0/1164175300 wait complete. 2023-12-01T03:10:36.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.476+0000 7ff874c0e640 1 Processor -- start 2023-12-01T03:10:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.476+0000 7ff874c0e640 1 -- start start 2023-12-01T03:10:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.477+0000 7ff874c0e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff87006b6a0 0x7ff8701f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.477+0000 7ff874c0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b0e70 0x7ff8700cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.477+0000 7ff874c0e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700b3ba0 0x7ff8701566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.477+0000 7ff874c0e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8701f09f0 con 0x7ff8700b0e70 2023-12-01T03:10:36.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.477+0000 7ff874c0e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8700c3be0 con 0x7ff8700b3ba0 2023-12-01T03:10:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.477+0000 7ff874c0e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8700c6100 con 0x7ff87006b6a0 2023-12-01T03:10:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.478+0000 7ff867fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b0e70 0x7ff8700cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.478+0000 7ff867fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b0e70 0x7ff8700cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33874/0 (socket says 172.21.15.115:33874) 2023-12-01T03:10:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.478+0000 7ff867fff640 1 -- 172.21.15.115:0/2441235410 learned_addr learned my addr 172.21.15.115:0/2441235410 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.478+0000 7ff86f7fe640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff87006b6a0 0x7ff8701f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.478+0000 7ff867fff640 1 -- 172.21.15.115:0/2441235410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff87006b6a0 msgr2=0x7ff8701f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.478+0000 7ff867fff640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff87006b6a0 0x7ff8701f04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.478+0000 7ff867fff640 1 -- 172.21.15.115:0/2441235410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700b3ba0 msgr2=0x7ff8701566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.478+0000 7ff867fff640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700b3ba0 0x7ff8701566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.478+0000 7ff867fff640 1 -- 172.21.15.115:0/2441235410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff858009400 con 0x7ff8700b0e70 2023-12-01T03:10:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.478+0000 7ff867fff640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b0e70 0x7ff8700cb230 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7ff858009890 tx=0x7ff8580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.479+0000 7ff86cff9640 1 -- 172.21.15.115:0/2441235410 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858032840 con 0x7ff8700b0e70 2023-12-01T03:10:36.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.479+0000 7ff86cff9640 1 -- 172.21.15.115:0/2441235410 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff858031f00 con 0x7ff8700b0e70 2023-12-01T03:10:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.479+0000 7ff874c0e640 1 -- 172.21.15.115:0/2441235410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8700c6350 con 0x7ff8700b0e70 2023-12-01T03:10:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.479+0000 7ff86cff9640 1 -- 172.21.15.115:0/2441235410 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858032be0 con 0x7ff8700b0e70 2023-12-01T03:10:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.479+0000 7ff874c0e640 1 -- 172.21.15.115:0/2441235410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8700c6bf0 con 0x7ff8700b0e70 2023-12-01T03:10:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.480+0000 7ff874c0e640 1 -- 172.21.15.115:0/2441235410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff870067850 con 0x7ff8700b0e70 2023-12-01T03:10:36.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.481+0000 7ff86cff9640 1 -- 172.21.15.115:0/2441235410 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff858032d40 con 0x7ff8700b0e70 2023-12-01T03:10:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.482+0000 7ff86cff9640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff85006a550 0x7ff85006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.482+0000 7ff86cff9640 1 -- 172.21.15.115:0/2441235410 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7ff8580aacf0 con 0x7ff8700b0e70 2023-12-01T03:10:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.483+0000 7ff86f7fe640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff85006a550 0x7ff85006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.483+0000 7ff86f7fe640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff85006a550 0x7ff85006ca10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff860006fd0 tx=0x7ff860008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:36.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.487+0000 7ff86cff9640 1 -- 172.21.15.115:0/2441235410 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff858076110 con 0x7ff8700b0e70 2023-12-01T03:10:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.678+0000 7ff874c0e640 1 -- 172.21.15.115:0/2441235410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff870073830 con 0x7ff8700b0e70 2023-12-01T03:10:36.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.679+0000 7ff86cff9640 1 -- 172.21.15.115:0/2441235410 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v214) v1 ==== 84+0+209 (secure 0 0 0) 0x7ff858079730 con 0x7ff8700b0e70 2023-12-01T03:10:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 24, 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": "cephfs_metadata" 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 25, 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": "cephfs_data" 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:36.681 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T03:10:36.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.681+0000 7ff874c0e640 1 -- 172.21.15.115:0/2441235410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff85006a550 msgr2=0x7ff85006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.681+0000 7ff874c0e640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff85006a550 0x7ff85006ca10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff860006fd0 tx=0x7ff860008040 comp rx=0 tx=0).stop 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.681+0000 7ff874c0e640 1 -- 172.21.15.115:0/2441235410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b0e70 msgr2=0x7ff8700cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.681+0000 7ff874c0e640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b0e70 0x7ff8700cb230 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7ff858009890 tx=0x7ff8580046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.681+0000 7ff874c0e640 1 -- 172.21.15.115:0/2441235410 shutdown_connections 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.681+0000 7ff874c0e640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff85006a550 0x7ff85006ca10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.681+0000 7ff874c0e640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8700b3ba0 0x7ff8701566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.681+0000 7ff874c0e640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8700b0e70 0x7ff8700cb230 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.681+0000 7ff874c0e640 1 --2- 172.21.15.115:0/2441235410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff87006b6a0 0x7ff8701f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.682+0000 7ff874c0e640 1 -- 172.21.15.115:0/2441235410 >> 172.21.15.115:0/2441235410 conn(0x7ff8700ce620 msgr2=0x7ff8700cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.682+0000 7ff874c0e640 1 -- 172.21.15.115:0/2441235410 shutdown_connections 2023-12-01T03:10:36.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.682+0000 7ff874c0e640 1 -- 172.21.15.115:0/2441235410 wait complete. 2023-12-01T03:10:36.702 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-12-01T03:10:36.702 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-12-01T03:10:36.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.914+0000 7f616493d640 1 Processor -- start 2023-12-01T03:10:36.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.914+0000 7f616493d640 1 -- start start 2023-12-01T03:10:36.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.915+0000 7f616493d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f616015cb60 0x7f61600caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.916+0000 7f616493d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6160070000 con 0x7f61600cb430 2023-12-01T03:10:36.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.916+0000 7f616493d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61600702e0 con 0x7f61600ca2d0 2023-12-01T03:10:36.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.916+0000 7f616493d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61600ca740 con 0x7f616015cb60 2023-12-01T03:10:36.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.916+0000 7f615effd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f61600cb430 0x7f61600c9bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37264/0 (socket says 172.21.15.115:37264) 2023-12-01T03:10:36.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.916+0000 7f615effd640 1 -- 172.21.15.115:0/1459643748 learned_addr learned my addr 172.21.15.115:0/1459643748 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.916+0000 7f615f7fe640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f616015cb60 0x7f61600caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017123593 0 0) 0x7f6160070000 con 0x7f61600cb430 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f614c003520 con 0x7f61600cb430 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 668394422 0 0) 0x7f614c003520 con 0x7f61600cb430 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f616015cb60 msgr2=0x7f61600caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f616015cb60 0x7f61600caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 >> v1:172.21.15.142:6789/0 conn(0x7f61600ca2d0 legacy=0x7f61600ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61600705c0 con 0x7f61600cb430 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6148001670 con 0x7f61600cb430 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 >> v1:172.21.15.115:6789/0 conn(0x7f61600cb430 legacy=0x7f61600c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f614c0039a0 0x7f614c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f614c0047c0 0x7f614c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f616015cb60 0x7f614c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c00ba30 con 0x7f614c0047c0 2023-12-01T03:10:36.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c00bba0 con 0x7f614c0039a0 2023-12-01T03:10:36.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c00bd60 con 0x7f616015cb60 2023-12-01T03:10:36.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.917+0000 7f615effd640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f614c0047c0 0x7f614c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.918+0000 7f615f7fe640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f614c0039a0 0x7f614c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.918+0000 7f615effd640 1 -- 172.21.15.115:0/1459643748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f616015cb60 msgr2=0x7f614c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:36.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.918+0000 7f615effd640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f616015cb60 0x7f614c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.918+0000 7f615effd640 1 -- 172.21.15.115:0/1459643748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f614c0039a0 msgr2=0x7f614c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:36.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.918+0000 7f615effd640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f614c0039a0 0x7f614c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.918+0000 7f615effd640 1 -- 172.21.15.115:0/1459643748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c003520 con 0x7f614c0047c0 2023-12-01T03:10:36.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.919+0000 7f615effd640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f614c0047c0 0x7f614c004c40 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f6148001eb0 tx=0x7f614c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1974c9099ea296a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:36.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.919+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148005ad0 con 0x7f614c0047c0 2023-12-01T03:10:36.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.919+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6148005c50 con 0x7f614c0047c0 2023-12-01T03:10:36.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.919+0000 7f615e7fc640 1 -- 172.21.15.115:0/1459643748 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614802c7a0 con 0x7f614c0047c0 2023-12-01T03:10:36.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.920+0000 7f616493d640 1 -- 172.21.15.115:0/1459643748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f614c0047c0 msgr2=0x7f614c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.920+0000 7f616493d640 1 --2- 172.21.15.115:0/1459643748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f614c0047c0 0x7f614c004c40 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f6148001eb0 tx=0x7f614c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.920+0000 7f615ffff640 1 -- 172.21.15.115:0/1459643748 reap_dead start 2023-12-01T03:10:36.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.920+0000 7f616493d640 1 -- 172.21.15.115:0/1459643748 shutdown_connections 2023-12-01T03:10:36.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.920+0000 7f616493d640 1 -- 172.21.15.115:0/1459643748 >> 172.21.15.115:0/1459643748 conn(0x7f6160065960 msgr2=0x7f6160065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:36.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.920+0000 7f616493d640 1 -- 172.21.15.115:0/1459643748 shutdown_connections 2023-12-01T03:10:36.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.920+0000 7f616493d640 1 -- 172.21.15.115:0/1459643748 wait complete. 2023-12-01T03:10:36.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.922+0000 7f616493d640 1 Processor -- start 2023-12-01T03:10:36.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.922+0000 7f616493d640 1 -- start start 2023-12-01T03:10:36.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.923+0000 7f616493d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f614c0047c0 0x7f61600c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.923+0000 7f616493d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f616015e190 0x7f6160153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.923+0000 7f616493d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61600c68c0 0x7f61600cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.923+0000 7f616493d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61600c7e40 con 0x7f616015e190 2023-12-01T03:10:36.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.923+0000 7f616493d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6160154260 con 0x7f614c0047c0 2023-12-01T03:10:36.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.923+0000 7f616493d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61600c71c0 con 0x7f61600c68c0 2023-12-01T03:10:36.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.924+0000 7f615effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f616015e190 0x7f6160153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.924+0000 7f615effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f616015e190 0x7f6160153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:33900/0 (socket says 172.21.15.115:33900) 2023-12-01T03:10:36.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.924+0000 7f615effd640 1 -- 172.21.15.115:0/4265536563 learned_addr learned my addr 172.21.15.115:0/4265536563 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:36.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.924+0000 7f615ffff640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61600c68c0 0x7f61600cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.924+0000 7f615f7fe640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f614c0047c0 0x7f61600c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.924+0000 7f615effd640 1 -- 172.21.15.115:0/4265536563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61600c68c0 msgr2=0x7f61600cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.924+0000 7f615effd640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61600c68c0 0x7f61600cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.924+0000 7f615effd640 1 -- 172.21.15.115:0/4265536563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f614c0047c0 msgr2=0x7f61600c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:36.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.924+0000 7f615effd640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f614c0047c0 0x7f61600c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:36.927 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.924+0000 7f615effd640 1 -- 172.21.15.115:0/4265536563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61600c74a0 con 0x7f616015e190 2023-12-01T03:10:36.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.925+0000 7f615effd640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f616015e190 0x7f6160153d20 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f614802c340 tx=0x7f6148001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:36.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.925+0000 7f6157fff640 1 -- 172.21.15.115:0/4265536563 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148039070 con 0x7f616015e190 2023-12-01T03:10:36.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.925+0000 7f6157fff640 1 -- 172.21.15.115:0/4265536563 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614802cbb0 con 0x7f616015e190 2023-12-01T03:10:36.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.925+0000 7f6157fff640 1 -- 172.21.15.115:0/4265536563 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614803c040 con 0x7f616015e190 2023-12-01T03:10:36.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.925+0000 7f616493d640 1 -- 172.21.15.115:0/4265536563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61600c7780 con 0x7f616015e190 2023-12-01T03:10:36.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.925+0000 7f616493d640 1 -- 172.21.15.115:0/4265536563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61600c6d40 con 0x7f616015e190 2023-12-01T03:10:36.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.926+0000 7f616493d640 1 -- 172.21.15.115:0/4265536563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61600c7f80 con 0x7f616015e190 2023-12-01T03:10:36.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.927+0000 7f6157fff640 1 -- 172.21.15.115:0/4265536563 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6148032070 con 0x7f616015e190 2023-12-01T03:10:36.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.928+0000 7f6157fff640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f614006a550 0x7f614006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:36.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.928+0000 7f6157fff640 1 -- 172.21.15.115:0/4265536563 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f61480a9e00 con 0x7f616015e190 2023-12-01T03:10:36.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.928+0000 7f615f7fe640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f614006a550 0x7f614006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:36.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.929+0000 7f615f7fe640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f614006a550 0x7f614006ca10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6150009fd0 tx=0x7f615000b040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:36.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:36.932+0000 7f6157fff640 1 -- 172.21.15.115:0/4265536563 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f61600c7f80 con 0x7f616015e190 2023-12-01T03:10:37.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.119+0000 7f616493d640 1 -- 172.21.15.115:0/4265536563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f61600c7f80 con 0x7f616015e190 2023-12-01T03:10:37.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.472+0000 7f6157fff640 1 -- 172.21.15.115:0/4265536563 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v215) v1 ==== 144+0+0 (secure 0 0 0) 0x7f61600c7f80 con 0x7f616015e190 2023-12-01T03:10:37.474 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'backup_fs_metadata' created 2023-12-01T03:10:37.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.477+0000 7f616493d640 1 -- 172.21.15.115:0/4265536563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f614006a550 msgr2=0x7f614006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:37.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.477+0000 7f616493d640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f614006a550 0x7f614006ca10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6150009fd0 tx=0x7f615000b040 comp rx=0 tx=0).stop 2023-12-01T03:10:37.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.477+0000 7f616493d640 1 -- 172.21.15.115:0/4265536563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f616015e190 msgr2=0x7f6160153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:37.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.477+0000 7f616493d640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f616015e190 0x7f6160153d20 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f614802c340 tx=0x7f6148001930 comp rx=0 tx=0).stop 2023-12-01T03:10:37.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.478+0000 7f616493d640 1 -- 172.21.15.115:0/4265536563 shutdown_connections 2023-12-01T03:10:37.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.478+0000 7f616493d640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f614006a550 0x7f614006ca10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.478+0000 7f616493d640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f61600c68c0 0x7f61600cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.478+0000 7f616493d640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f616015e190 0x7f6160153d20 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.478+0000 7f616493d640 1 --2- 172.21.15.115:0/4265536563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f614c0047c0 0x7f61600c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.478+0000 7f616493d640 1 -- 172.21.15.115:0/4265536563 >> 172.21.15.115:0/4265536563 conn(0x7f6160065960 msgr2=0x7f61600ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:37.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.478+0000 7f616493d640 1 -- 172.21.15.115:0/4265536563 shutdown_connections 2023-12-01T03:10:37.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.479+0000 7f616493d640 1 -- 172.21.15.115:0/4265536563 wait complete. 2023-12-01T03:10:37.505 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T03:10:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.704+0000 7fc0cda5b640 1 Processor -- start 2023-12-01T03:10:37.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.705+0000 7fc0cda5b640 1 -- start start 2023-12-01T03:10:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.706+0000 7fc0cda5b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c815ac30 0x7fc0c80b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.706+0000 7fc0cda5b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc0c80b1d00 0x7fc0c80b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.706+0000 7fc0cda5b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c80b2560 con 0x7fc0c80b0620 2023-12-01T03:10:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.706+0000 7fc0cda5b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c80b0ab0 con 0x7fc0c80b1d00 2023-12-01T03:10:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.706+0000 7fc0cda5b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80c99d0 con 0x7fc0c815ac30 2023-12-01T03:10:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.706+0000 7fc0cd25a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc0c80b0620 0x7fc0c806b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37020/0 (socket says 172.21.15.115:37020) 2023-12-01T03:10:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.706+0000 7fc0cd25a640 1 -- 172.21.15.115:0/3522919748 learned_addr learned my addr 172.21.15.115:0/3522919748 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:37.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.706+0000 7fc0c7fff640 1 --2- 172.21.15.115:0/3522919748 >> v2:172.21.15.142:3300/0 conn(0x7fc0c80b1d00 0x7fc0c80b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.706+0000 7fc0cca59640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c815ac30 0x7fc0c80b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946872053 0 0) 0x7fc0c80b2560 con 0x7fc0c80b0620 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b8003520 con 0x7fc0c80b0620 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2107842871 0 0) 0x7fc0b8003520 con 0x7fc0c80b0620 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c815ac30 msgr2=0x7fc0c80b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c815ac30 0x7fc0c80b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 >> v2:172.21.15.142:3300/0 conn(0x7fc0c80b1d00 msgr2=0x7fc0c80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 --2- 172.21.15.115:0/3522919748 >> v2:172.21.15.142:3300/0 conn(0x7fc0c80b1d00 0x7fc0c80b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c80cae70 con 0x7fc0c80b0620 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc0c00019f0 con 0x7fc0c80b0620 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 >> v1:172.21.15.115:6789/0 conn(0x7fc0c80b0620 legacy=0x7fc0c806b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c815ac30 0x7fc0b8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b8003ef0 0x7fc0b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:37.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0b8004d00 0x7fc0b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80b0ab0 con 0x7fc0b8004d00 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80c99d0 con 0x7fc0b8003ef0 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b800ba80 con 0x7fc0c815ac30 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.707+0000 7fc0cd25a640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0b8004d00 0x7fc0b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.708+0000 7fc0c7fff640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b8003ef0 0x7fc0b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.708+0000 7fc0cca59640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c815ac30 0x7fc0b8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.708+0000 7fc0cd25a640 1 -- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c815ac30 msgr2=0x7fc0b8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.708+0000 7fc0cd25a640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c815ac30 0x7fc0b8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.708+0000 7fc0cd25a640 1 -- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b8003ef0 msgr2=0x7fc0b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.708+0000 7fc0cd25a640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b8003ef0 0x7fc0b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.708+0000 7fc0cd25a640 1 -- 172.21.15.115:0/3522919748 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b8003520 con 0x7fc0b8004d00 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.709+0000 7fc0cd25a640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0b8004d00 0x7fc0b8005180 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000a890 tx=0x7fc0b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf07be912ec45bb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.709+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0034630 con 0x7fc0b8004d00 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.709+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c002f070 con 0x7fc0b8004d00 2023-12-01T03:10:37.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.709+0000 7fc0c77fe640 1 -- 172.21.15.115:0/3522919748 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0034a30 con 0x7fc0b8004d00 2023-12-01T03:10:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.710+0000 7fc0cda5b640 1 -- 172.21.15.115:0/3522919748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0b8004d00 msgr2=0x7fc0b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.710+0000 7fc0cda5b640 1 --2- 172.21.15.115:0/3522919748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0b8004d00 0x7fc0b8005180 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000a890 tx=0x7fc0b80057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.710+0000 7fc0cd25a640 1 -- 172.21.15.115:0/3522919748 reap_dead start 2023-12-01T03:10:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.710+0000 7fc0cda5b640 1 -- 172.21.15.115:0/3522919748 shutdown_connections 2023-12-01T03:10:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.710+0000 7fc0cda5b640 1 -- 172.21.15.115:0/3522919748 >> 172.21.15.115:0/3522919748 conn(0x7fc0c8066400 msgr2=0x7fc0c80cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.710+0000 7fc0cda5b640 1 -- 172.21.15.115:0/3522919748 shutdown_connections 2023-12-01T03:10:37.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.710+0000 7fc0cda5b640 1 -- 172.21.15.115:0/3522919748 wait complete. 2023-12-01T03:10:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.711+0000 7fc0cda5b640 1 Processor -- start 2023-12-01T03:10:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.712+0000 7fc0cda5b640 1 -- start start 2023-12-01T03:10:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cda5b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b8004d00 0x7fc0c80c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cda5b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c814fb10 0x7fc0c80c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cda5b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c814ffc0 0x7fc0c80c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cda5b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80c85c0 con 0x7fc0c814ffc0 2023-12-01T03:10:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cda5b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80c5c00 con 0x7fc0b8004d00 2023-12-01T03:10:37.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cda5b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80c5e50 con 0x7fc0c814fb10 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cd25a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c814ffc0 0x7fc0c80c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cd25a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c814ffc0 0x7fc0c80c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55332/0 (socket says 172.21.15.115:55332) 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cd25a640 1 -- 172.21.15.115:0/450848490 learned_addr learned my addr 172.21.15.115:0/450848490 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0c7fff640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c814fb10 0x7fc0c80c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cca59640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b8004d00 0x7fc0c80c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cd25a640 1 -- 172.21.15.115:0/450848490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c814fb10 msgr2=0x7fc0c80c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cd25a640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c814fb10 0x7fc0c80c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cd25a640 1 -- 172.21.15.115:0/450848490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b8004d00 msgr2=0x7fc0c80c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cd25a640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b8004d00 0x7fc0c80c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cd25a640 1 -- 172.21.15.115:0/450848490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c80c6130 con 0x7fc0c814ffc0 2023-12-01T03:10:37.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.713+0000 7fc0cca59640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b8004d00 0x7fc0c80c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.714+0000 7fc0cd25a640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c814ffc0 0x7fc0c80c8080 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000a890 tx=0x7fc0c0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.714+0000 7fc0c57fa640 1 -- 172.21.15.115:0/450848490 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0043070 con 0x7fc0c814ffc0 2023-12-01T03:10:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.714+0000 7fc0c57fa640 1 -- 172.21.15.115:0/450848490 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c0001fd0 con 0x7fc0c814ffc0 2023-12-01T03:10:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.714+0000 7fc0c57fa640 1 -- 172.21.15.115:0/450848490 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0034840 con 0x7fc0c814ffc0 2023-12-01T03:10:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.714+0000 7fc0cda5b640 1 -- 172.21.15.115:0/450848490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c80c6410 con 0x7fc0c814ffc0 2023-12-01T03:10:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.714+0000 7fc0cda5b640 1 -- 172.21.15.115:0/450848490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c80c7780 con 0x7fc0c814ffc0 2023-12-01T03:10:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.715+0000 7fc0cda5b640 1 -- 172.21.15.115:0/450848490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c80c6930 con 0x7fc0c814ffc0 2023-12-01T03:10:37.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.716+0000 7fc0c57fa640 1 -- 172.21.15.115:0/450848490 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc0c0031030 con 0x7fc0c814ffc0 2023-12-01T03:10:37.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.717+0000 7fc0c57fa640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ac06a550 0x7fc0ac06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:37.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.717+0000 7fc0c57fa640 1 -- 172.21.15.115:0/450848490 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6126+0+0 (secure 0 0 0) 0x7fc0c00aa290 con 0x7fc0c814ffc0 2023-12-01T03:10:37.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.718+0000 7fc0cca59640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ac06a550 0x7fc0ac06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:37.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.718+0000 7fc0cca59640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ac06a550 0x7fc0ac06ca10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8153a10 tx=0x7fc0bc008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:37.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.723+0000 7fc0c57fa640 1 -- 172.21.15.115:0/450848490 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc0c00790c0 con 0x7fc0c814ffc0 2023-12-01T03:10:37.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:37.900+0000 7fc0cda5b640 1 -- 172.21.15.115:0/450848490 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc0c80c6930 con 0x7fc0c814ffc0 2023-12-01T03:10:38.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.492+0000 7fc0c57fa640 1 -- 172.21.15.115:0/450848490 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v216) v1 ==== 176+0+0 (secure 0 0 0) 0x7fc0c80c6930 con 0x7fc0c814ffc0 2023-12-01T03:10:38.495 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'backup_fs_data' created 2023-12-01T03:10:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.503+0000 7fc0cda5b640 1 -- 172.21.15.115:0/450848490 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ac06a550 msgr2=0x7fc0ac06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.503+0000 7fc0cda5b640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ac06a550 0x7fc0ac06ca10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8153a10 tx=0x7fc0bc008040 comp rx=0 tx=0).stop 2023-12-01T03:10:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.503+0000 7fc0cda5b640 1 -- 172.21.15.115:0/450848490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c814ffc0 msgr2=0x7fc0c80c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:38.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.503+0000 7fc0cda5b640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c814ffc0 0x7fc0c80c8080 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000a890 tx=0x7fc0c0001d80 comp rx=0 tx=0).stop 2023-12-01T03:10:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.505+0000 7fc0cda5b640 1 -- 172.21.15.115:0/450848490 shutdown_connections 2023-12-01T03:10:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.505+0000 7fc0cda5b640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ac06a550 0x7fc0ac06ca10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.505+0000 7fc0cda5b640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0c814fb10 0x7fc0c80c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.505+0000 7fc0cda5b640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c814ffc0 0x7fc0c80c8080 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.505+0000 7fc0cda5b640 1 --2- 172.21.15.115:0/450848490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b8004d00 0x7fc0c80c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.505+0000 7fc0cda5b640 1 -- 172.21.15.115:0/450848490 >> 172.21.15.115:0/450848490 conn(0x7fc0c8066400 msgr2=0x7fc0c80ca670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.506+0000 7fc0cda5b640 1 -- 172.21.15.115:0/450848490 shutdown_connections 2023-12-01T03:10:38.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.506+0000 7fc0cda5b640 1 -- 172.21.15.115:0/450848490 wait complete. 2023-12-01T03:10:38.537 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-12-01T03:10:38.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.716+0000 7f13a6d73640 1 Processor -- start 2023-12-01T03:10:38.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.716+0000 7f13a6d73640 1 -- start start 2023-12-01T03:10:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.717+0000 7f13a6d73640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f13a00b1710 0x7f13a006baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.717+0000 7f13a6d73640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13a00b1ba0 0x7f13a006c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.717+0000 7f13a6d73640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a00cac30 con 0x7f13a01590b0 2023-12-01T03:10:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.717+0000 7f13a6d73640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a00b2140 con 0x7f13a00b1710 2023-12-01T03:10:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.717+0000 7f13a6d73640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00b2420 con 0x7f13a00b1ba0 2023-12-01T03:10:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.718+0000 7f139ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f13a00b1710 0x7f13a006baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.718+0000 7f139ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f13a00b1710 0x7f13a006baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55334/0 (socket says 172.21.15.115:55334) 2023-12-01T03:10:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.718+0000 7f139ffff640 1 -- 172.21.15.115:0/1709977192 learned_addr learned my addr 172.21.15.115:0/1709977192 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.718+0000 7f139ffff640 1 -- 172.21.15.115:0/1709977192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13a00b1ba0 msgr2=0x7f13a006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:38.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.718+0000 7f13a5310640 1 --2- 172.21.15.115:0/1709977192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13a00b1ba0 0x7f13a006c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:38.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.718+0000 7f139ffff640 1 --2- 172.21.15.115:0/1709977192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13a00b1ba0 0x7f13a006c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.718+0000 7f139ffff640 1 -- 172.21.15.115:0/1709977192 >> v1:172.21.15.142:6789/0 conn(0x7f13a01590b0 legacy=0x7f13a0159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:38.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.718+0000 7f139ffff640 1 -- 172.21.15.115:0/1709977192 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a00caed0 con 0x7f13a00b1710 2023-12-01T03:10:38.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.719+0000 7f139ffff640 1 --2- 172.21.15.115:0/1709977192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1710 0x7f13a006baf0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f1388009e50 tx=0x7f138802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a13f8cf221e8e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:38.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f139f7fe640 1 -- 172.21.15.115:0/1709977192 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138802ca70 con 0x7f13a00b1710 2023-12-01T03:10:38.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f139f7fe640 1 -- 172.21.15.115:0/1709977192 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138802cbd0 con 0x7f13a00b1710 2023-12-01T03:10:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f139f7fe640 1 -- 172.21.15.115:0/1709977192 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13880343f0 con 0x7f13a00b1710 2023-12-01T03:10:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f13a6d73640 1 -- 172.21.15.115:0/1709977192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1710 msgr2=0x7f13a006baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f13a6d73640 1 --2- 172.21.15.115:0/1709977192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1710 0x7f13a006baf0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f1388009e50 tx=0x7f138802bed0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f13a6d73640 1 -- 172.21.15.115:0/1709977192 shutdown_connections 2023-12-01T03:10:38.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f13a6d73640 1 --2- 172.21.15.115:0/1709977192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13a00b1ba0 0x7f13a006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f13a6d73640 1 --2- 172.21.15.115:0/1709977192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1710 0x7f13a006baf0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f13a6d73640 1 -- 172.21.15.115:0/1709977192 >> 172.21.15.115:0/1709977192 conn(0x7f13a0065960 msgr2=0x7f13a0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:38.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f13a6d73640 1 -- 172.21.15.115:0/1709977192 shutdown_connections 2023-12-01T03:10:38.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.720+0000 7f13a6d73640 1 -- 172.21.15.115:0/1709977192 wait complete. 2023-12-01T03:10:38.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.722+0000 7f13a6d73640 1 Processor -- start 2023-12-01T03:10:38.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.722+0000 7f13a6d73640 1 -- start start 2023-12-01T03:10:38.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.723+0000 7f13a6d73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13a00b1710 0x7f13a0148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:38.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.723+0000 7f13a6d73640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1ba0 0x7f13a01f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:38.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.723+0000 7f13a6d73640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13a01590b0 0x7f13a014fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:38.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.723+0000 7f13a6d73640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a01f1630 con 0x7f13a00b1ba0 2023-12-01T03:10:38.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.723+0000 7f13a6d73640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a0148670 con 0x7f13a00b1710 2023-12-01T03:10:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.723+0000 7f13a6d73640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a01f1070 con 0x7f13a01590b0 2023-12-01T03:10:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.723+0000 7f139ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1ba0 0x7f13a01f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.723+0000 7f139ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1ba0 0x7f13a01f0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55342/0 (socket says 172.21.15.115:55342) 2023-12-01T03:10:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.723+0000 7f139ffff640 1 -- 172.21.15.115:0/3379465515 learned_addr learned my addr 172.21.15.115:0/3379465515 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f13a4b0f640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13a00b1710 0x7f13a0148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f13a5310640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13a01590b0 0x7f13a014fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f139ffff640 1 -- 172.21.15.115:0/3379465515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13a01590b0 msgr2=0x7f13a014fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f139ffff640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13a01590b0 0x7f13a014fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f139ffff640 1 -- 172.21.15.115:0/3379465515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13a00b1710 msgr2=0x7f13a0148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f139ffff640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13a00b1710 0x7f13a0148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f139ffff640 1 -- 172.21.15.115:0/3379465515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1388009b30 con 0x7f13a00b1ba0 2023-12-01T03:10:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f13a4b0f640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13a00b1710 0x7f13a0148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f139ffff640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1ba0 0x7f13a01f0640 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f1388009e50 tx=0x7f1388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f139d7fa640 1 -- 172.21.15.115:0/3379465515 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138803e450 con 0x7f13a00b1ba0 2023-12-01T03:10:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f139d7fa640 1 -- 172.21.15.115:0/3379465515 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138803e5f0 con 0x7f13a00b1ba0 2023-12-01T03:10:38.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f13a6d73640 1 -- 172.21.15.115:0/3379465515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a01f1350 con 0x7f13a00b1ba0 2023-12-01T03:10:38.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.724+0000 7f139d7fa640 1 -- 172.21.15.115:0/3379465515 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138803e9a0 con 0x7f13a00b1ba0 2023-12-01T03:10:38.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.725+0000 7f13a6d73640 1 -- 172.21.15.115:0/3379465515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a01f1c60 con 0x7f13a00b1ba0 2023-12-01T03:10:38.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.726+0000 7f13a6d73640 1 -- 172.21.15.115:0/3379465515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a01f17e0 con 0x7f13a00b1ba0 2023-12-01T03:10:38.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.727+0000 7f139d7fa640 1 -- 172.21.15.115:0/3379465515 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f138803eb00 con 0x7f13a00b1ba0 2023-12-01T03:10:38.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.728+0000 7f139d7fa640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f138006a550 0x7f138006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:38.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.728+0000 7f139d7fa640 1 -- 172.21.15.115:0/3379465515 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f13880abfc0 con 0x7f13a00b1ba0 2023-12-01T03:10:38.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.728+0000 7f13a4b0f640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f138006a550 0x7f138006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:38.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.729+0000 7f13a4b0f640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f138006a550 0x7f138006ca10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13a01e5a60 tx=0x7f1390008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:38.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.732+0000 7f139d7fa640 1 -- 172.21.15.115:0/3379465515 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1388077280 con 0x7f13a00b1ba0 2023-12-01T03:10:38.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.962+0000 7f13a6d73640 1 -- 172.21.15.115:0/3379465515 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f13a01f17e0 con 0x7f13a00b1ba0 2023-12-01T03:10:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 26 and data pool 27 2023-12-01T03:10:38.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.977+0000 7f139d7fa640 1 -- 172.21.15.115:0/3379465515 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 26 and data pool 27 v97) v1 ==== 183+0+0 (secure 0 0 0) 0x7f13a01f17e0 con 0x7f13a00b1ba0 2023-12-01T03:10:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.982+0000 7f13a6d73640 1 -- 172.21.15.115:0/3379465515 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f138006a550 msgr2=0x7f138006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.982+0000 7f13a6d73640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f138006a550 0x7f138006ca10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13a01e5a60 tx=0x7f1390008040 comp rx=0 tx=0).stop 2023-12-01T03:10:38.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.983+0000 7f13a6d73640 1 -- 172.21.15.115:0/3379465515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1ba0 msgr2=0x7f13a01f0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.983+0000 7f13a6d73640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1ba0 0x7f13a01f0640 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f1388009e50 tx=0x7f1388004a40 comp rx=0 tx=0).stop 2023-12-01T03:10:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.983+0000 7f13a6d73640 1 -- 172.21.15.115:0/3379465515 shutdown_connections 2023-12-01T03:10:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.983+0000 7f13a6d73640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f138006a550 0x7f138006ca10 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13a01e5a60 tx=0x7f1390008040 comp rx=0 tx=0).stop 2023-12-01T03:10:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.983+0000 7f13a6d73640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13a01590b0 0x7f13a014fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.983+0000 7f13a6d73640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13a00b1ba0 0x7f13a01f0640 secure :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f1388009e50 tx=0x7f1388004a40 comp rx=0 tx=0).stop 2023-12-01T03:10:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.983+0000 7f13a6d73640 1 --2- 172.21.15.115:0/3379465515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13a00b1710 0x7f13a0148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:38.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.983+0000 7f13a6d73640 1 -- 172.21.15.115:0/3379465515 >> 172.21.15.115:0/3379465515 conn(0x7f13a0065960 msgr2=0x7f13a0070920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:38.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.985+0000 7f13a6d73640 1 -- 172.21.15.115:0/3379465515 shutdown_connections 2023-12-01T03:10:38.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:38.986+0000 7f13a6d73640 1 -- 172.21.15.115:0/3379465515 wait complete. 2023-12-01T03:10:39.018 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:39.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.191+0000 7f9e6dbae640 1 Processor -- start 2023-12-01T03:10:39.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.191+0000 7f9e6dbae640 1 -- start start 2023-12-01T03:10:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.193+0000 7f9e6dbae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6815cd20 0x7f9e680d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.193+0000 7f9e6dbae640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9e680b5d80 0x7f9e680b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.193+0000 7f9e6dbae640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e680ce2e0 con 0x7f9e680d1230 2023-12-01T03:10:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.193+0000 7f9e6dbae640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e680b2160 con 0x7f9e680b5d80 2023-12-01T03:10:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.193+0000 7f9e6dbae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680d2a10 con 0x7f9e6815cd20 2023-12-01T03:10:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.193+0000 7f9e67fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9e680d1230 0x7f9e680d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37034/0 (socket says 172.21.15.115:37034) 2023-12-01T03:10:39.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.193+0000 7f9e67fff640 1 -- 172.21.15.115:0/1466054426 learned_addr learned my addr 172.21.15.115:0/1466054426 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e66ffd640 1 --2- 172.21.15.115:0/1466054426 >> v2:172.21.15.142:3300/0 conn(0x7f9e680b5d80 0x7f9e680b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e677fe640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6815cd20 0x7f9e680d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731884671 0 0) 0x7f9e680ce2e0 con 0x7f9e680d1230 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e54003520 con 0x7f9e680d1230 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3728738632 0 0) 0x7f9e54003520 con 0x7f9e680d1230 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6815cd20 msgr2=0x7f9e680d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6815cd20 0x7f9e680d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 >> v2:172.21.15.142:3300/0 conn(0x7f9e680b5d80 msgr2=0x7f9e680b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 --2- 172.21.15.115:0/1466054426 >> v2:172.21.15.142:3300/0 conn(0x7f9e680b5d80 0x7f9e680b8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680ce000 con 0x7f9e680d1230 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f9e60001670 con 0x7f9e680d1230 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 >> v1:172.21.15.115:6789/0 conn(0x7f9e680d1230 legacy=0x7f9e680d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6815cd20 0x7f9e54003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e54003f90 0x7f9e540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e54004d00 0x7f9e54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680d2a10 con 0x7f9e54004d00 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5400ba90 con 0x7f9e54003f90 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5400bc50 con 0x7f9e6815cd20 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.194+0000 7f9e67fff640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e54004d00 0x7f9e54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.195+0000 7f9e677fe640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6815cd20 0x7f9e54003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.195+0000 7f9e67fff640 1 -- 172.21.15.115:0/1466054426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6815cd20 msgr2=0x7f9e54003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.195+0000 7f9e67fff640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6815cd20 0x7f9e54003a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.195+0000 7f9e67fff640 1 -- 172.21.15.115:0/1466054426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e54003f90 msgr2=0x7f9e540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.195+0000 7f9e67fff640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e54003f90 0x7f9e540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.195+0000 7f9e67fff640 1 -- 172.21.15.115:0/1466054426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e54003520 con 0x7f9e54004d00 2023-12-01T03:10:39.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.196+0000 7f9e67fff640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e54004d00 0x7f9e54005180 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000c240 tx=0x7f9e540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d84be0df5b1ea2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:39.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.196+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60005ad0 con 0x7f9e54004d00 2023-12-01T03:10:39.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.196+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e60005c30 con 0x7f9e54004d00 2023-12-01T03:10:39.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.196+0000 7f9e667fc640 1 -- 172.21.15.115:0/1466054426 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6002caf0 con 0x7f9e54004d00 2023-12-01T03:10:39.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.197+0000 7f9e6dbae640 1 -- 172.21.15.115:0/1466054426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e54004d00 msgr2=0x7f9e54005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.197+0000 7f9e6dbae640 1 --2- 172.21.15.115:0/1466054426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e54004d00 0x7f9e54005180 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000c240 tx=0x7f9e540057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.197+0000 7f9e67fff640 1 -- 172.21.15.115:0/1466054426 reap_dead start 2023-12-01T03:10:39.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.197+0000 7f9e6dbae640 1 -- 172.21.15.115:0/1466054426 shutdown_connections 2023-12-01T03:10:39.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.197+0000 7f9e6dbae640 1 -- 172.21.15.115:0/1466054426 >> 172.21.15.115:0/1466054426 conn(0x7f9e6806a0e0 msgr2=0x7f9e6806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:39.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.197+0000 7f9e6dbae640 1 -- 172.21.15.115:0/1466054426 shutdown_connections 2023-12-01T03:10:39.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.197+0000 7f9e6dbae640 1 -- 172.21.15.115:0/1466054426 wait complete. 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.198+0000 7f9e6dbae640 1 Processor -- start 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.199+0000 7f9e6dbae640 1 -- start start 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.199+0000 7f9e6dbae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e54004d00 0x7f9e6814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.199+0000 7f9e6dbae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e680b52f0 0x7f9e68155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.199+0000 7f9e6dbae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e68158520 0x7f9e681589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.199+0000 7f9e6dbae640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68158ee0 con 0x7f9e68158520 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.199+0000 7f9e6dbae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680b2c20 con 0x7f9e680b52f0 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.199+0000 7f9e6dbae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e680b2610 con 0x7f9e54004d00 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e68158520 0x7f9e681589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e68158520 0x7f9e681589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55366/0 (socket says 172.21.15.115:55366) 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e67fff640 1 -- 172.21.15.115:0/2942845023 learned_addr learned my addr 172.21.15.115:0/2942845023 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e66ffd640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e680b52f0 0x7f9e68155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e677fe640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e54004d00 0x7f9e6814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e67fff640 1 -- 172.21.15.115:0/2942845023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e54004d00 msgr2=0x7f9e6814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e67fff640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e54004d00 0x7f9e6814f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e67fff640 1 -- 172.21.15.115:0/2942845023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e680b52f0 msgr2=0x7f9e68155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e67fff640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e680b52f0 0x7f9e68155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e67fff640 1 -- 172.21.15.115:0/2942845023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e681563a0 con 0x7f9e68158520 2023-12-01T03:10:39.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.200+0000 7f9e67fff640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e68158520 0x7f9e681589a0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f9e6002c520 tx=0x7f9e6002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:39.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.201+0000 7f9e5ffff640 1 -- 172.21.15.115:0/2942845023 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6003c040 con 0x7f9e68158520 2023-12-01T03:10:39.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.201+0000 7f9e5ffff640 1 -- 172.21.15.115:0/2942845023 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6002cac0 con 0x7f9e68158520 2023-12-01T03:10:39.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.201+0000 7f9e5ffff640 1 -- 172.21.15.115:0/2942845023 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6002ce30 con 0x7f9e68158520 2023-12-01T03:10:39.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.201+0000 7f9e6dbae640 1 -- 172.21.15.115:0/2942845023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e680b28b0 con 0x7f9e68158520 2023-12-01T03:10:39.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.201+0000 7f9e6dbae640 1 -- 172.21.15.115:0/2942845023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e68156090 con 0x7f9e68158520 2023-12-01T03:10:39.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.202+0000 7f9e6dbae640 1 -- 172.21.15.115:0/2942845023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6804e890 con 0x7f9e68158520 2023-12-01T03:10:39.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.204+0000 7f9e5ffff640 1 -- 172.21.15.115:0/2942845023 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9e60032070 con 0x7f9e68158520 2023-12-01T03:10:39.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.205+0000 7f9e5ffff640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4806a580 0x7f9e4806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.205+0000 7f9e5ffff640 1 -- 172.21.15.115:0/2942845023 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f9e600aa110 con 0x7f9e68158520 2023-12-01T03:10:39.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.205+0000 7f9e677fe640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4806a580 0x7f9e4806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.206+0000 7f9e677fe640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4806a580 0x7f9e4806ca40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e680cb4f0 tx=0x7f9e5800b040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:39.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.209+0000 7f9e5ffff640 1 -- 172.21.15.115:0/2942845023 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9e60075240 con 0x7f9e68158520 2023-12-01T03:10:39.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.387+0000 7f9e6dbae640 1 -- 172.21.15.115:0/2942845023 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e680b56d0 con 0x7f9e68158520 2023-12-01T03:10:39.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.389+0000 7f9e5ffff640 1 -- 172.21.15.115:0/2942845023 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v217) v1 ==== 74+0+19846 (secure 0 0 0) 0x7f9e680b56d0 con 0x7f9e68158520 2023-12-01T03:10:39.391 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:39.392 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":217,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:38.964577+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:10:37.180599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:10:37.961783+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 -- 172.21.15.115:0/2942845023 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4806a580 msgr2=0x7f9e4806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4806a580 0x7f9e4806ca40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e680cb4f0 tx=0x7f9e5800b040 comp rx=0 tx=0).stop 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 -- 172.21.15.115:0/2942845023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e68158520 msgr2=0x7f9e681589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e68158520 0x7f9e681589a0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f9e6002c520 tx=0x7f9e6002e070 comp rx=0 tx=0).stop 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 -- 172.21.15.115:0/2942845023 shutdown_connections 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4806a580 0x7f9e4806ca40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e68158520 0x7f9e681589a0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e680b52f0 0x7f9e68155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 --2- 172.21.15.115:0/2942845023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e54004d00 0x7f9e6814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 -- 172.21.15.115:0/2942845023 >> 172.21.15.115:0/2942845023 conn(0x7f9e6806a0e0 msgr2=0x7f9e680c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 -- 172.21.15.115:0/2942845023 shutdown_connections 2023-12-01T03:10:39.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.392+0000 7f9e6dbae640 1 -- 172.21.15.115:0/2942845023 wait complete. 2023-12-01T03:10:39.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.614+0000 7f0823212640 1 Processor -- start 2023-12-01T03:10:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.614+0000 7f0823212640 1 -- start start 2023-12-01T03:10:39.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.615+0000 7f0823212640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f081c0ccc80 0x7f081c06fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.615+0000 7f0823212640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ad2b0 0x7f081c06fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.616+0000 7f0823212640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081c15d600 con 0x7f081c15d220 2023-12-01T03:10:39.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.616+0000 7f0823212640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081c06cc10 con 0x7f081c0ccc80 2023-12-01T03:10:39.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.616+0000 7f0823212640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c06c720 con 0x7f081c0ad2b0 2023-12-01T03:10:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.616+0000 7f08217af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ad2b0 0x7f081c06fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.616+0000 7f08217af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ad2b0 0x7f081c06fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:49614/0 (socket says 172.21.15.115:49614) 2023-12-01T03:10:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.616+0000 7f08217af640 1 -- 172.21.15.115:0/4063197461 learned_addr learned my addr 172.21.15.115:0/4063197461 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:39.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.616+0000 7f0813fff640 1 --2- 172.21.15.115:0/4063197461 >> v2:172.21.15.142:3300/0 conn(0x7f081c0ccc80 0x7f081c06fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.617+0000 7f08217af640 1 -- 172.21.15.115:0/4063197461 >> v2:172.21.15.142:3300/0 conn(0x7f081c0ccc80 msgr2=0x7f081c06fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.617+0000 7f08217af640 1 --2- 172.21.15.115:0/4063197461 >> v2:172.21.15.142:3300/0 conn(0x7f081c0ccc80 0x7f081c06fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.617+0000 7f08217af640 1 -- 172.21.15.115:0/4063197461 >> v1:172.21.15.142:6789/0 conn(0x7f081c15d220 legacy=0x7f081c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.617+0000 7f08217af640 1 -- 172.21.15.115:0/4063197461 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c06c4b0 con 0x7f081c0ad2b0 2023-12-01T03:10:39.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.619+0000 7f08217af640 1 --2- 172.21.15.115:0/4063197461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ad2b0 0x7f081c06fff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0814009e50 tx=0x7f081402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbf10138274d11ce server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:39.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.619+0000 7f08137fe640 1 -- 172.21.15.115:0/4063197461 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081402ca70 con 0x7f081c0ad2b0 2023-12-01T03:10:39.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.619+0000 7f08137fe640 1 -- 172.21.15.115:0/4063197461 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081402cbd0 con 0x7f081c0ad2b0 2023-12-01T03:10:39.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.619+0000 7f08137fe640 1 -- 172.21.15.115:0/4063197461 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140343f0 con 0x7f081c0ad2b0 2023-12-01T03:10:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.620+0000 7f0823212640 1 -- 172.21.15.115:0/4063197461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ad2b0 msgr2=0x7f081c06fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.620+0000 7f0823212640 1 --2- 172.21.15.115:0/4063197461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ad2b0 0x7f081c06fff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0814009e50 tx=0x7f081402bed0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.620+0000 7f0823212640 1 -- 172.21.15.115:0/4063197461 shutdown_connections 2023-12-01T03:10:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.620+0000 7f0823212640 1 --2- 172.21.15.115:0/4063197461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ad2b0 0x7f081c06fff0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.620+0000 7f0823212640 1 --2- 172.21.15.115:0/4063197461 >> v2:172.21.15.142:3300/0 conn(0x7f081c0ccc80 0x7f081c06fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.620+0000 7f0823212640 1 -- 172.21.15.115:0/4063197461 >> 172.21.15.115:0/4063197461 conn(0x7f081c0ce880 msgr2=0x7f081c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.620+0000 7f0823212640 1 -- 172.21.15.115:0/4063197461 shutdown_connections 2023-12-01T03:10:39.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.620+0000 7f0823212640 1 -- 172.21.15.115:0/4063197461 wait complete. 2023-12-01T03:10:39.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.622+0000 7f0823212640 1 Processor -- start 2023-12-01T03:10:39.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.622+0000 7f0823212640 1 -- start start 2023-12-01T03:10:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.623+0000 7f0823212640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f081c0ad2b0 0x7f081c156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.623+0000 7f0823212640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ccc80 0x7f081c1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.623+0000 7f0823212640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f081c15d220 0x7f081c1597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.623+0000 7f0823212640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0b15c0 con 0x7f081c15d220 2023-12-01T03:10:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.623+0000 7f0823212640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c157250 con 0x7f081c0ad2b0 2023-12-01T03:10:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.623+0000 7f0823212640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0c8f50 con 0x7f081c0ccc80 2023-12-01T03:10:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.623+0000 7f08217af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f081c15d220 0x7f081c1597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.623+0000 7f08217af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f081c15d220 0x7f081c1597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55372/0 (socket says 172.21.15.115:55372) 2023-12-01T03:10:39.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.623+0000 7f08217af640 1 -- 172.21.15.115:0/1226773240 learned_addr learned my addr 172.21.15.115:0/1226773240 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:39.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f08217af640 1 -- 172.21.15.115:0/1226773240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ccc80 msgr2=0x7f081c1592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f0813fff640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ccc80 0x7f081c1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f08217af640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ccc80 0x7f081c1592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f08217af640 1 -- 172.21.15.115:0/1226773240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f081c0ad2b0 msgr2=0x7f081c156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f08217af640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f081c0ad2b0 0x7f081c156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f08217af640 1 -- 172.21.15.115:0/1226773240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814009b30 con 0x7f081c15d220 2023-12-01T03:10:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f08217af640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f081c15d220 0x7f081c1597e0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f0814009e50 tx=0x7f0814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f08117fa640 1 -- 172.21.15.115:0/1226773240 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814009510 con 0x7f081c15d220 2023-12-01T03:10:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f08117fa640 1 -- 172.21.15.115:0/1226773240 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081403e450 con 0x7f081c15d220 2023-12-01T03:10:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f08117fa640 1 -- 172.21.15.115:0/1226773240 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081403e800 con 0x7f081c15d220 2023-12-01T03:10:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f0823212640 1 -- 172.21.15.115:0/1226773240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c0c9230 con 0x7f081c15d220 2023-12-01T03:10:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.624+0000 7f0823212640 1 -- 172.21.15.115:0/1226773240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c0c97f0 con 0x7f081c15d220 2023-12-01T03:10:39.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.626+0000 7f0823212640 1 -- 172.21.15.115:0/1226773240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081c04e0e0 con 0x7f081c15d220 2023-12-01T03:10:39.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.626+0000 7f08117fa640 1 -- 172.21.15.115:0/1226773240 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f081403e960 con 0x7f081c15d220 2023-12-01T03:10:39.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.627+0000 7f08117fa640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f080806a550 0x7f080806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:39.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.627+0000 7f08117fa640 1 -- 172.21.15.115:0/1226773240 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f08140abb50 con 0x7f081c15d220 2023-12-01T03:10:39.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.628+0000 7f0820fae640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f080806a550 0x7f080806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:39.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.628+0000 7f0820fae640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f080806a550 0x7f080806ca10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0818000f60 tx=0x7f08180079b0 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:39.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.632+0000 7f08117fa640 1 -- 172.21.15.115:0/1226773240 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0814076c90 con 0x7f081c15d220 2023-12-01T03:10:39.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.822+0000 7f0823212640 1 -- 172.21.15.115:0/1226773240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f081c06c900 con 0x7f081c15d220 2023-12-01T03:10:39.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.824+0000 7f08117fa640 1 -- 172.21.15.115:0/1226773240 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v217) v1 ==== 74+0+19846 (secure 0 0 0) 0x7f081407a2b0 con 0x7f081c15d220 2023-12-01T03:10:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:39.826 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":217,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:38.964577+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:10:37.180599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:10:37.961783+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:39.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 -- 172.21.15.115:0/1226773240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f080806a550 msgr2=0x7f080806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f080806a550 0x7f080806ca10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0818000f60 tx=0x7f08180079b0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 -- 172.21.15.115:0/1226773240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f081c15d220 msgr2=0x7f081c1597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:39.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f081c15d220 0x7f081c1597e0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f0814009e50 tx=0x7f0814004a40 comp rx=0 tx=0).stop 2023-12-01T03:10:39.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 -- 172.21.15.115:0/1226773240 shutdown_connections 2023-12-01T03:10:39.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f080806a550 0x7f080806ca10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f081c0ccc80 0x7f081c1592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f081c15d220 0x7f081c1597e0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 --2- 172.21.15.115:0/1226773240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f081c0ad2b0 0x7f081c156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:39.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 -- 172.21.15.115:0/1226773240 >> 172.21.15.115:0/1226773240 conn(0x7f081c0ce880 msgr2=0x7f081c0af430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:39.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 -- 172.21.15.115:0/1226773240 shutdown_connections 2023-12-01T03:10:39.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:39.827+0000 7f0823212640 1 -- 172.21.15.115:0/1226773240 wait complete. 2023-12-01T03:10:39.848 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-12-01T03:10:40.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.047+0000 7efeda1c9640 1 Processor -- start 2023-12-01T03:10:40.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.047+0000 7efeda1c9640 1 -- start start 2023-12-01T03:10:40.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.049+0000 7efeda1c9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efed40ce1b0 0x7efed40cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:40.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.049+0000 7efeda1c9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed40cc9a0 con 0x7efed40b0e60 2023-12-01T03:10:40.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.049+0000 7efeda1c9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed40cbd90 con 0x7efed415ac80 2023-12-01T03:10:40.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.049+0000 7efeda1c9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed415b060 con 0x7efed40ce1b0 2023-12-01T03:10:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.049+0000 7efecbfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efed40ce1b0 0x7efed40cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.049+0000 7efed3fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efed40b0e60 0x7efed406b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37042/0 (socket says 172.21.15.115:37042) 2023-12-01T03:10:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.049+0000 7efecbfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efed40ce1b0 0x7efed40cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55384/0 (socket says 172.21.15.115:55384) 2023-12-01T03:10:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.049+0000 7efed3fff640 1 -- 172.21.15.115:0/2608686830 learned_addr learned my addr 172.21.15.115:0/2608686830 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.050+0000 7efecbfff640 1 -- 172.21.15.115:0/2608686830 >> v1:172.21.15.142:6789/0 conn(0x7efed415ac80 legacy=0x7efed40cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.050+0000 7efecbfff640 1 -- 172.21.15.115:0/2608686830 >> v1:172.21.15.115:6789/0 conn(0x7efed40b0e60 legacy=0x7efed406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.050+0000 7efecbfff640 1 -- 172.21.15.115:0/2608686830 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40cc070 con 0x7efed40ce1b0 2023-12-01T03:10:40.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.050+0000 7efed2ffd640 1 -- 172.21.15.115:0/2608686830 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104093257 0 0) 0x7efed40cc9a0 con 0x7efed40b0e60 2023-12-01T03:10:40.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.051+0000 7efecbfff640 1 --2- 172.21.15.115:0/2608686830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40ce1b0 0x7efed40cce30 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7efebc0097a0 tx=0x7efebc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d9829a7cb52b4d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:40.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.051+0000 7efed2ffd640 1 -- 172.21.15.115:0/2608686830 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc031830 con 0x7efed40ce1b0 2023-12-01T03:10:40.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.051+0000 7efed2ffd640 1 -- 172.21.15.115:0/2608686830 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc031990 con 0x7efed40ce1b0 2023-12-01T03:10:40.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.051+0000 7efed2ffd640 1 -- 172.21.15.115:0/2608686830 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc031d20 con 0x7efed40ce1b0 2023-12-01T03:10:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.052+0000 7efeda1c9640 1 -- 172.21.15.115:0/2608686830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40ce1b0 msgr2=0x7efed40cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.052+0000 7efeda1c9640 1 --2- 172.21.15.115:0/2608686830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40ce1b0 0x7efed40cce30 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7efebc0097a0 tx=0x7efebc02b8d0 comp rx=0 tx=0).stop 2023-12-01T03:10:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.052+0000 7efeda1c9640 1 -- 172.21.15.115:0/2608686830 shutdown_connections 2023-12-01T03:10:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.052+0000 7efeda1c9640 1 --2- 172.21.15.115:0/2608686830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40ce1b0 0x7efed40cce30 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.052+0000 7efeda1c9640 1 -- 172.21.15.115:0/2608686830 >> 172.21.15.115:0/2608686830 conn(0x7efed4065960 msgr2=0x7efed4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.052+0000 7efeda1c9640 1 -- 172.21.15.115:0/2608686830 shutdown_connections 2023-12-01T03:10:40.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.052+0000 7efeda1c9640 1 -- 172.21.15.115:0/2608686830 wait complete. 2023-12-01T03:10:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.053+0000 7efeda1c9640 1 Processor -- start 2023-12-01T03:10:40.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.054+0000 7efeda1c9640 1 -- start start 2023-12-01T03:10:40.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efeda1c9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40b0e60 0x7efed413efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:40.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efeda1c9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efed40ce1b0 0x7efed4156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:40.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efeda1c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efed415ac80 0x7efed41571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:40.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efeda1c9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed4142040 con 0x7efed40b0e60 2023-12-01T03:10:40.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efeda1c9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed413ed30 con 0x7efed40ce1b0 2023-12-01T03:10:40.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efeda1c9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed413f500 con 0x7efed415ac80 2023-12-01T03:10:40.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efed37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40b0e60 0x7efed413efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:40.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efed37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40b0e60 0x7efed413efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55398/0 (socket says 172.21.15.115:55398) 2023-12-01T03:10:40.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efed37fe640 1 -- 172.21.15.115:0/856529480 learned_addr learned my addr 172.21.15.115:0/856529480 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:40.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efed37fe640 1 -- 172.21.15.115:0/856529480 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efed415ac80 msgr2=0x7efed41571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:40.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efed37fe640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efed415ac80 0x7efed41571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:40.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efed37fe640 1 -- 172.21.15.115:0/856529480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efed40ce1b0 msgr2=0x7efed4156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:40.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efed37fe640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efed40ce1b0 0x7efed4156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:40.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efed37fe640 1 -- 172.21.15.115:0/856529480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc009400 con 0x7efed40b0e60 2023-12-01T03:10:40.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.055+0000 7efed37fe640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40b0e60 0x7efed413efc0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7efec400b420 tx=0x7efec400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:40.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.056+0000 7efed0ff9640 1 -- 172.21.15.115:0/856529480 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4011830 con 0x7efed40b0e60 2023-12-01T03:10:40.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.056+0000 7efed0ff9640 1 -- 172.21.15.115:0/856529480 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec4011990 con 0x7efed40b0e60 2023-12-01T03:10:40.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.056+0000 7efed0ff9640 1 -- 172.21.15.115:0/856529480 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4011d40 con 0x7efed40b0e60 2023-12-01T03:10:40.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.056+0000 7efeda1c9640 1 -- 172.21.15.115:0/856529480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed4158b30 con 0x7efed40b0e60 2023-12-01T03:10:40.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.056+0000 7efeda1c9640 1 -- 172.21.15.115:0/856529480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed4158dd0 con 0x7efed40b0e60 2023-12-01T03:10:40.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.057+0000 7efeda1c9640 1 -- 172.21.15.115:0/856529480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed404e890 con 0x7efed40b0e60 2023-12-01T03:10:40.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.058+0000 7efed0ff9640 1 -- 172.21.15.115:0/856529480 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7efec401b460 con 0x7efed40b0e60 2023-12-01T03:10:40.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.059+0000 7efed0ff9640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efeb406a550 0x7efeb406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:40.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.059+0000 7efed0ff9640 1 -- 172.21.15.115:0/856529480 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7efec4088ab0 con 0x7efed40b0e60 2023-12-01T03:10:40.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.059+0000 7efecbfff640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efeb406a550 0x7efeb406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:40.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.059+0000 7efecbfff640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efeb406a550 0x7efeb406ca10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efebc036000 tx=0x7efebc030000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:40.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.063+0000 7efed0ff9640 1 -- 172.21.15.115:0/856529480 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efec4053b60 con 0x7efed40b0e60 2023-12-01T03:10:40.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.289+0000 7efeda1c9640 1 -- 172.21.15.115:0/856529480 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efed40cc320 con 0x7efed40b0e60 2023-12-01T03:10:40.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.987+0000 7efed0ff9640 1 -- 172.21.15.115:0/856529480 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v100) v1 ==== 121+0+0 (secure 0 0 0) 0x7efec4057180 con 0x7efed40b0e60 2023-12-01T03:10:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 -- 172.21.15.115:0/856529480 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efeb406a550 msgr2=0x7efeb406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efeb406a550 0x7efeb406ca10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efebc036000 tx=0x7efebc030000 comp rx=0 tx=0).stop 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 -- 172.21.15.115:0/856529480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40b0e60 msgr2=0x7efed413efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40b0e60 0x7efed413efc0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7efec400b420 tx=0x7efec400b8e0 comp rx=0 tx=0).stop 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 -- 172.21.15.115:0/856529480 shutdown_connections 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efeb406a550 0x7efeb406ca10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efed415ac80 0x7efed41571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efed40ce1b0 0x7efed4156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 --2- 172.21.15.115:0/856529480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efed40b0e60 0x7efed413efc0 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 -- 172.21.15.115:0/856529480 >> 172.21.15.115:0/856529480 conn(0x7efed4065960 msgr2=0x7efed406f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 -- 172.21.15.115:0/856529480 shutdown_connections 2023-12-01T03:10:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:40.991+0000 7efeda1c9640 1 -- 172.21.15.115:0/856529480 wait complete. 2023-12-01T03:10:41.014 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T03:10:41.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-12-01T03:10:41.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.225+0000 7efe80e21640 1 Processor -- start 2023-12-01T03:10:41.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.225+0000 7efe80e21640 1 -- start start 2023-12-01T03:10:41.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.226+0000 7efe80e21640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efe7c15cf70 0x7efe7c0b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:41.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.226+0000 7efe80e21640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7c0b7760 con 0x7efe7c0d0260 2023-12-01T03:10:41.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.226+0000 7efe80e21640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7c0b8b70 con 0x7efe7c0b18f0 2023-12-01T03:10:41.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.226+0000 7efe80e21640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7c0b2170 con 0x7efe7c15cf70 2023-12-01T03:10:41.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.226+0000 7efe7ad9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efe7c0d0260 0x7efe7c0d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37050/0 (socket says 172.21.15.115:37050) 2023-12-01T03:10:41.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7ad9d640 1 -- 172.21.15.115:0/1560600619 learned_addr learned my addr 172.21.15.115:0/1560600619 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:41.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389403877 0 0) 0x7efe7c0b7760 con 0x7efe7c0d0260 2023-12-01T03:10:41.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe68003520 con 0x7efe7c0d0260 2023-12-01T03:10:41.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7a59c640 1 --2- 172.21.15.115:0/1560600619 >> v2:172.21.15.142:3300/0 conn(0x7efe7c15cf70 0x7efe7c0b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:41.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3862247626 0 0) 0x7efe68003520 con 0x7efe7c0d0260 2023-12-01T03:10:41.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 >> v2:172.21.15.142:3300/0 conn(0x7efe7c15cf70 msgr2=0x7efe7c0b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:41.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7959a640 1 --2- 172.21.15.115:0/1560600619 >> v2:172.21.15.142:3300/0 conn(0x7efe7c15cf70 0x7efe7c0b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 >> v1:172.21.15.142:6789/0 conn(0x7efe7c0b18f0 legacy=0x7efe7c0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:41.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c0b6e30 con 0x7efe7c0d0260 2023-12-01T03:10:41.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7efe74001670 con 0x7efe7c0d0260 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.227+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 >> v1:172.21.15.115:6789/0 conn(0x7efe7c0d0260 legacy=0x7efe7c0d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7959a640 1 --2- 172.21.15.115:0/1560600619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe68003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7959a640 1 --2- 172.21.15.115:0/1560600619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe680047c0 0x7efe68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7959a640 1 --2- 172.21.15.115:0/1560600619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe680051d0 0x7efe6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0b2170 con 0x7efe680039c0 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6800c0e0 con 0x7efe680051d0 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6800c2a0 con 0x7efe680047c0 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7a59c640 1 --2- 172.21.15.115:0/1560600619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe68003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7a59c640 1 -- 172.21.15.115:0/1560600619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe680047c0 msgr2=0x7efe68004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7a59c640 1 --2- 172.21.15.115:0/1560600619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe680047c0 0x7efe68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7a59c640 1 -- 172.21.15.115:0/1560600619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe680051d0 msgr2=0x7efe6800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7a59c640 1 --2- 172.21.15.115:0/1560600619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe680051d0 0x7efe6800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7a59c640 1 -- 172.21.15.115:0/1560600619 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe68003520 con 0x7efe680039c0 2023-12-01T03:10:41.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.228+0000 7efe7ad9d640 1 -- 172.21.15.115:0/1560600619 reap_dead start 2023-12-01T03:10:41.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.230+0000 7efe7a59c640 1 --2- 172.21.15.115:0/1560600619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe68003dc0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7efe6c007360 tx=0x7efe680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d50fd100ae90868 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:41.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.230+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c02ccf0 con 0x7efe680039c0 2023-12-01T03:10:41.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.230+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c02ce50 con 0x7efe680039c0 2023-12-01T03:10:41.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.230+0000 7efe7959a640 1 -- 172.21.15.115:0/1560600619 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c034660 con 0x7efe680039c0 2023-12-01T03:10:41.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.231+0000 7efe80e21640 1 -- 172.21.15.115:0/1560600619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 msgr2=0x7efe68003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:41.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.231+0000 7efe80e21640 1 --2- 172.21.15.115:0/1560600619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe68003dc0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7efe6c007360 tx=0x7efe680057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.232+0000 7efe80e21640 1 -- 172.21.15.115:0/1560600619 shutdown_connections 2023-12-01T03:10:41.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.232+0000 7efe80e21640 1 --2- 172.21.15.115:0/1560600619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe68003dc0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.232+0000 7efe80e21640 1 -- 172.21.15.115:0/1560600619 >> 172.21.15.115:0/1560600619 conn(0x7efe7c06a0e0 msgr2=0x7efe7c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:41.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.233+0000 7efe80e21640 1 -- 172.21.15.115:0/1560600619 shutdown_connections 2023-12-01T03:10:41.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.233+0000 7efe80e21640 1 -- 172.21.15.115:0/1560600619 wait complete. 2023-12-01T03:10:41.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.235+0000 7efe80e21640 1 Processor -- start 2023-12-01T03:10:41.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.235+0000 7efe80e21640 1 -- start start 2023-12-01T03:10:41.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.236+0000 7efe80e21640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe7c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.236+0000 7efe80e21640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7c153070 0x7efe7c158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.236+0000 7efe80e21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7c158570 0x7efe7c1f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.236+0000 7efe80e21640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c1589f0 con 0x7efe680039c0 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.236+0000 7efe80e21640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0ccbc0 con 0x7efe7c153070 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.236+0000 7efe80e21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0d39f0 con 0x7efe7c158570 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.236+0000 7efe7a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe7c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.236+0000 7efe7a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe7c0d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55410/0 (socket says 172.21.15.115:55410) 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.236+0000 7efe7a59c640 1 -- 172.21.15.115:0/2492538902 learned_addr learned my addr 172.21.15.115:0/2492538902 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.237+0000 7efe7a59c640 1 -- 172.21.15.115:0/2492538902 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7c158570 msgr2=0x7efe7c1f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.237+0000 7efe7a59c640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7c158570 0x7efe7c1f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.237+0000 7efe7a59c640 1 -- 172.21.15.115:0/2492538902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7c153070 msgr2=0x7efe7c158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.237+0000 7efe7a59c640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7c153070 0x7efe7c158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.237+0000 7efe7a59c640 1 -- 172.21.15.115:0/2492538902 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c0ca130 con 0x7efe680039c0 2023-12-01T03:10:41.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.237+0000 7efe7a59c640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe7c0d34b0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7efe6c000f80 tx=0x7efe6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:41.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.237+0000 7efe72ffd640 1 -- 172.21.15.115:0/2492538902 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c003b90 con 0x7efe680039c0 2023-12-01T03:10:41.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.238+0000 7efe72ffd640 1 -- 172.21.15.115:0/2492538902 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c003d30 con 0x7efe680039c0 2023-12-01T03:10:41.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.238+0000 7efe80e21640 1 -- 172.21.15.115:0/2492538902 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c0ca410 con 0x7efe680039c0 2023-12-01T03:10:41.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.238+0000 7efe72ffd640 1 -- 172.21.15.115:0/2492538902 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c003ea0 con 0x7efe680039c0 2023-12-01T03:10:41.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.238+0000 7efe80e21640 1 -- 172.21.15.115:0/2492538902 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c0ca9d0 con 0x7efe680039c0 2023-12-01T03:10:41.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.239+0000 7efe80e21640 1 -- 172.21.15.115:0/2492538902 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7c158990 con 0x7efe680039c0 2023-12-01T03:10:41.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.240+0000 7efe72ffd640 1 -- 172.21.15.115:0/2492538902 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7efe6c03b040 con 0x7efe680039c0 2023-12-01T03:10:41.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.241+0000 7efe72ffd640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efe5c06a580 0x7efe5c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:41.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.241+0000 7efe72ffd640 1 -- 172.21.15.115:0/2492538902 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7efe6c0aaab0 con 0x7efe680039c0 2023-12-01T03:10:41.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.241+0000 7efe79d9b640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efe5c06a580 0x7efe5c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:41.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.242+0000 7efe79d9b640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efe5c06a580 0x7efe5c06ca40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0c9520 tx=0x7efe640079b0 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:41.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.245+0000 7efe72ffd640 1 -- 172.21.15.115:0/2492538902 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efe7c158990 con 0x7efe680039c0 2023-12-01T03:10:41.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.474+0000 7efe80e21640 1 -- 172.21.15.115:0/2492538902 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efe7c158990 con 0x7efe680039c0 2023-12-01T03:10:41.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.991+0000 7efe72ffd640 1 -- 172.21.15.115:0/2492538902 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v101) v1 ==== 125+0+0 (secure 0 0 0) 0x7efe6c039020 con 0x7efe680039c0 2023-12-01T03:10:41.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.995+0000 7efe80e21640 1 -- 172.21.15.115:0/2492538902 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efe5c06a580 msgr2=0x7efe5c06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:41.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.995+0000 7efe80e21640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efe5c06a580 0x7efe5c06ca40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0c9520 tx=0x7efe640079b0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.995+0000 7efe80e21640 1 -- 172.21.15.115:0/2492538902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 msgr2=0x7efe7c0d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:41.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.995+0000 7efe80e21640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe7c0d34b0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7efe6c000f80 tx=0x7efe6c004ab0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.996+0000 7efe80e21640 1 -- 172.21.15.115:0/2492538902 shutdown_connections 2023-12-01T03:10:41.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.996+0000 7efe80e21640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efe5c06a580 0x7efe5c06ca40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.996+0000 7efe80e21640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe7c158570 0x7efe7c1f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.996+0000 7efe80e21640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe7c153070 0x7efe7c158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.996+0000 7efe80e21640 1 --2- 172.21.15.115:0/2492538902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe680039c0 0x7efe7c0d34b0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:41.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.996+0000 7efe80e21640 1 -- 172.21.15.115:0/2492538902 >> 172.21.15.115:0/2492538902 conn(0x7efe7c06a0e0 msgr2=0x7efe7c15aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:41.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.996+0000 7efe80e21640 1 -- 172.21.15.115:0/2492538902 shutdown_connections 2023-12-01T03:10:41.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:41.996+0000 7efe80e21640 1 -- 172.21.15.115:0/2492538902 wait complete. 2023-12-01T03:10:42.018 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-12-01T03:10:42.019 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:10:42.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.225+0000 7f2a558ee640 1 Processor -- start 2023-12-01T03:10:42.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.225+0000 7f2a558ee640 1 -- start start 2023-12-01T03:10:42.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.227+0000 7f2a558ee640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a50159dd0 0x7f2a500c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.227+0000 7f2a558ee640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5006f8b0 con 0x7f2a50070d30 2023-12-01T03:10:42.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.227+0000 7f2a558ee640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a500adcb0 con 0x7f2a5006b6f0 2023-12-01T03:10:42.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.227+0000 7f2a558ee640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a500cd270 con 0x7f2a50159dd0 2023-12-01T03:10:42.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4f7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2a50070d30 0x7f2a500ccb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37056/0 (socket says 172.21.15.115:37056) 2023-12-01T03:10:42.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4f7fe640 1 -- 172.21.15.115:0/4288511071 learned_addr learned my addr 172.21.15.115:0/4288511071 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:42.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933632990 0 0) 0x7f2a5006f8b0 con 0x7f2a50070d30 2023-12-01T03:10:42.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4effd640 1 --2- 172.21.15.115:0/4288511071 >> v2:172.21.15.142:3300/0 conn(0x7f2a50159dd0 0x7f2a500c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a3c003520 con 0x7f2a50070d30 2023-12-01T03:10:42.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2697599695 0 0) 0x7f2a3c003520 con 0x7f2a50070d30 2023-12-01T03:10:42.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 >> v2:172.21.15.142:3300/0 conn(0x7f2a50159dd0 msgr2=0x7f2a500c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4e7fc640 1 --2- 172.21.15.115:0/4288511071 >> v2:172.21.15.142:3300/0 conn(0x7f2a50159dd0 0x7f2a500c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 >> v1:172.21.15.142:6789/0 conn(0x7f2a5006b6f0 legacy=0x7f2a500cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a500ad9d0 con 0x7f2a50070d30 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.228+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2a48001670 con 0x7f2a50070d30 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 >> v1:172.21.15.115:6789/0 conn(0x7f2a50070d30 legacy=0x7f2a500ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4e7fc640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a3c003a30 0x7f2a3c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4e7fc640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c0047c0 0x7f2a3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4e7fc640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c0052d0 0x7f2a3c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500cd270 con 0x7f2a3c0052d0 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c00e420 con 0x7f2a3c0047c0 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c00e5e0 con 0x7f2a3c003a30 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4f7fe640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c0052d0 0x7f2a3c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a467fc640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c0047c0 0x7f2a3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4f7fe640 1 -- 172.21.15.115:0/4288511071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a3c003a30 msgr2=0x7f2a3c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:42.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4f7fe640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a3c003a30 0x7f2a3c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4f7fe640 1 -- 172.21.15.115:0/4288511071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c0047c0 msgr2=0x7f2a3c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:42.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4f7fe640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c0047c0 0x7f2a3c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a4f7fe640 1 -- 172.21.15.115:0/4288511071 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c003520 con 0x7f2a3c0052d0 2023-12-01T03:10:42.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.229+0000 7f2a467fc640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c0047c0 0x7f2a3c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:42.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.230+0000 7f2a4f7fe640 1 -- 172.21.15.115:0/4288511071 reap_dead start 2023-12-01T03:10:42.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.230+0000 7f2a4f7fe640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c0052d0 0x7f2a3c00db90 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c0e0 tx=0x7f2a3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42910665f43314b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:42.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.231+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48005ad0 con 0x7f2a3c0052d0 2023-12-01T03:10:42.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.231+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a48005c70 con 0x7f2a3c0052d0 2023-12-01T03:10:42.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.231+0000 7f2a4e7fc640 1 -- 172.21.15.115:0/4288511071 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4802c810 con 0x7f2a3c0052d0 2023-12-01T03:10:42.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.231+0000 7f2a558ee640 1 -- 172.21.15.115:0/4288511071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c0052d0 msgr2=0x7f2a3c00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.231+0000 7f2a558ee640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c0052d0 0x7f2a3c00db90 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c0e0 tx=0x7f2a3c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.231+0000 7f2a558ee640 1 -- 172.21.15.115:0/4288511071 shutdown_connections 2023-12-01T03:10:42.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.231+0000 7f2a558ee640 1 --2- 172.21.15.115:0/4288511071 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a3c0052d0 0x7f2a3c00db90 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.231+0000 7f2a558ee640 1 -- 172.21.15.115:0/4288511071 >> 172.21.15.115:0/4288511071 conn(0x7f2a500ce880 msgr2=0x7f2a500cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:42.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.231+0000 7f2a558ee640 1 -- 172.21.15.115:0/4288511071 shutdown_connections 2023-12-01T03:10:42.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.231+0000 7f2a558ee640 1 -- 172.21.15.115:0/4288511071 wait complete. 2023-12-01T03:10:42.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.233+0000 7f2a558ee640 1 Processor -- start 2023-12-01T03:10:42.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.233+0000 7f2a558ee640 1 -- start start 2023-12-01T03:10:42.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.234+0000 7f2a558ee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c0052d0 0x7f2a500cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.234+0000 7f2a558ee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a5015c950 0x7f2a500cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.234+0000 7f2a558ee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a500b14e0 0x7f2a50158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.234+0000 7f2a558ee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5015d3f0 con 0x7f2a500b14e0 2023-12-01T03:10:42.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.234+0000 7f2a558ee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5015ce30 con 0x7f2a3c0052d0 2023-12-01T03:10:42.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.234+0000 7f2a558ee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5015d110 con 0x7f2a5015c950 2023-12-01T03:10:42.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a4f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a500b14e0 0x7f2a50158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a4f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a500b14e0 0x7f2a50158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55430/0 (socket says 172.21.15.115:55430) 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a4f7fe640 1 -- 172.21.15.115:0/2637675723 learned_addr learned my addr 172.21.15.115:0/2637675723 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a467fc640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a5015c950 0x7f2a500cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a4f7fe640 1 -- 172.21.15.115:0/2637675723 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a5015c950 msgr2=0x7f2a500cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a4f7fe640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a5015c950 0x7f2a500cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a4effd640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c0052d0 0x7f2a500cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a4f7fe640 1 -- 172.21.15.115:0/2637675723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c0052d0 msgr2=0x7f2a500cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a4f7fe640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c0052d0 0x7f2a500cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a4f7fe640 1 -- 172.21.15.115:0/2637675723 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a501e57f0 con 0x7f2a500b14e0 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.235+0000 7f2a4f7fe640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a500b14e0 0x7f2a50158970 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c0e0 tx=0x7f2a48001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.236+0000 7f2a4cff9640 1 -- 172.21.15.115:0/2637675723 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48001670 con 0x7f2a500b14e0 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.236+0000 7f2a4cff9640 1 -- 172.21.15.115:0/2637675723 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a480047e0 con 0x7f2a500b14e0 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.236+0000 7f2a4cff9640 1 -- 172.21.15.115:0/2637675723 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48004b50 con 0x7f2a500b14e0 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.236+0000 7f2a558ee640 1 -- 172.21.15.115:0/2637675723 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a501e5ad0 con 0x7f2a500b14e0 2023-12-01T03:10:42.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.236+0000 7f2a558ee640 1 -- 172.21.15.115:0/2637675723 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a501e7800 con 0x7f2a500b14e0 2023-12-01T03:10:42.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.237+0000 7f2a558ee640 1 -- 172.21.15.115:0/2637675723 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a501e5cc0 con 0x7f2a500b14e0 2023-12-01T03:10:42.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.238+0000 7f2a4cff9640 1 -- 172.21.15.115:0/2637675723 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f2a48032070 con 0x7f2a500b14e0 2023-12-01T03:10:42.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.239+0000 7f2a4cff9640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 0x7f2a3006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.239+0000 7f2a4cff9640 1 -- 172.21.15.115:0/2637675723 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f2a4803c070 con 0x7f2a500b14e0 2023-12-01T03:10:42.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.239+0000 7f2a4effd640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 0x7f2a3006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.240+0000 7f2a4effd640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 0x7f2a3006ca40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a50159570 tx=0x7f2a40008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:42.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.243+0000 7f2a4cff9640 1 -- 172.21.15.115:0/2637675723 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2a48074d80 con 0x7f2a500b14e0 2023-12-01T03:10:42.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.469+0000 7f2a558ee640 1 -- 172.21.15.115:0/2637675723 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a5004e890 con 0x7f2a500b14e0 2023-12-01T03:10:42.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.470+0000 7f2a4cff9640 1 -- 172.21.15.115:0/2637675723 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5793 (secure 0 0 0) 0x7f2a48039070 con 0x7f2a500b14e0 2023-12-01T03:10:42.471 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:42.471 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":101,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:25.146035+0000","modified":"2023-12-01T03:10:28.416554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10653":{"gid":10653,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/720518620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":720518620},{"type":"v1","addr":"172.21.15.142:6834","nonce":720518620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:38.965513+0000","modified":"2023-12-01T03:10:41.983462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7333":{"gid":7333,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-01T03:10:42.471 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 101 2023-12-01T03:10:42.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 -- 172.21.15.115:0/2637675723 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 msgr2=0x7f2a3006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 0x7f2a3006ca40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a50159570 tx=0x7f2a40008040 comp rx=0 tx=0).stop 2023-12-01T03:10:42.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 -- 172.21.15.115:0/2637675723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a500b14e0 msgr2=0x7f2a50158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a500b14e0 0x7f2a50158970 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c0e0 tx=0x7f2a48001930 comp rx=0 tx=0).stop 2023-12-01T03:10:42.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 -- 172.21.15.115:0/2637675723 shutdown_connections 2023-12-01T03:10:42.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a3006a580 0x7f2a3006ca40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a5015c950 0x7f2a500cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a500b14e0 0x7f2a50158970 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 --2- 172.21.15.115:0/2637675723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a3c0052d0 0x7f2a500cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 -- 172.21.15.115:0/2637675723 >> 172.21.15.115:0/2637675723 conn(0x7f2a500ce880 msgr2=0x7f2a500afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:42.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 -- 172.21.15.115:0/2637675723 shutdown_connections 2023-12-01T03:10:42.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.473+0000 7f2a558ee640 1 -- 172.21.15.115:0/2637675723 wait complete. 2023-12-01T03:10:42.493 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:10:42.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.695+0000 7f922e900640 1 Processor -- start 2023-12-01T03:10:42.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.695+0000 7f922e900640 1 -- start start 2023-12-01T03:10:42.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.697+0000 7f922e900640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922815cc10 0x7f922806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.697+0000 7f922e900640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f922815cff0 0x7f92280c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.697+0000 7f922e900640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f92280cc350 0x7f92280cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.697+0000 7f922e900640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92280c9f00 con 0x7f92280cc350 2023-12-01T03:10:42.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.697+0000 7f922e900640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92280ccd20 con 0x7f922815cff0 2023-12-01T03:10:42.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.697+0000 7f922e900640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9228071110 con 0x7f922815cc10 2023-12-01T03:10:42.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.697+0000 7f922ce9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f92280cc350 0x7f92280cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.697+0000 7f922ce9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f92280cc350 0x7f92280cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55442/0 (socket says 172.21.15.115:55442) 2023-12-01T03:10:42.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.697+0000 7f922ce9d640 1 -- 172.21.15.115:0/1619528350 learned_addr learned my addr 172.21.15.115:0/1619528350 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:42.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.698+0000 7f9227fff640 1 --2- 172.21.15.115:0/1619528350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922815cc10 0x7f922806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.698+0000 7f922ce9d640 1 -- 172.21.15.115:0/1619528350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922815cc10 msgr2=0x7f922806d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:42.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.698+0000 7f921ffff640 1 --2- 172.21.15.115:0/1619528350 >> v2:172.21.15.142:3300/0 conn(0x7f922815cff0 0x7f92280c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.698+0000 7f922ce9d640 1 --2- 172.21.15.115:0/1619528350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922815cc10 0x7f922806d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.698+0000 7f922ce9d640 1 -- 172.21.15.115:0/1619528350 >> v2:172.21.15.142:3300/0 conn(0x7f922815cff0 msgr2=0x7f92280c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:42.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.698+0000 7f922ce9d640 1 --2- 172.21.15.115:0/1619528350 >> v2:172.21.15.142:3300/0 conn(0x7f922815cff0 0x7f92280c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.698+0000 7f922ce9d640 1 -- 172.21.15.115:0/1619528350 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280c6750 con 0x7f92280cc350 2023-12-01T03:10:42.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.699+0000 7f922ce9d640 1 --2- 172.21.15.115:0/1619528350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92280cc350 0x7f92280cc7e0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f92280ccfa0 tx=0x7f922002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45421f9f5b4cce99 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:42.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.699+0000 7f92277fe640 1 -- 172.21.15.115:0/1619528350 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922002c9e0 con 0x7f92280cc350 2023-12-01T03:10:42.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.699+0000 7f92277fe640 1 -- 172.21.15.115:0/1619528350 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922002cb40 con 0x7f92280cc350 2023-12-01T03:10:42.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.699+0000 7f92277fe640 1 -- 172.21.15.115:0/1619528350 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92200343f0 con 0x7f92280cc350 2023-12-01T03:10:42.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.700+0000 7f922e900640 1 -- 172.21.15.115:0/1619528350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92280cc350 msgr2=0x7f92280cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.700+0000 7f922e900640 1 --2- 172.21.15.115:0/1619528350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92280cc350 0x7f92280cc7e0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f92280ccfa0 tx=0x7f922002be40 comp rx=0 tx=0).stop 2023-12-01T03:10:42.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.700+0000 7f922e900640 1 -- 172.21.15.115:0/1619528350 shutdown_connections 2023-12-01T03:10:42.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.700+0000 7f922e900640 1 --2- 172.21.15.115:0/1619528350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92280cc350 0x7f92280cc7e0 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.700+0000 7f922e900640 1 --2- 172.21.15.115:0/1619528350 >> v2:172.21.15.142:3300/0 conn(0x7f922815cff0 0x7f92280c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.700+0000 7f922e900640 1 --2- 172.21.15.115:0/1619528350 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922815cc10 0x7f922806d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.700+0000 7f922e900640 1 -- 172.21.15.115:0/1619528350 >> 172.21.15.115:0/1619528350 conn(0x7f9228069cc0 msgr2=0x7f922806a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:42.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.700+0000 7f922e900640 1 -- 172.21.15.115:0/1619528350 shutdown_connections 2023-12-01T03:10:42.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.700+0000 7f922e900640 1 -- 172.21.15.115:0/1619528350 wait complete. 2023-12-01T03:10:42.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.702+0000 7f922e900640 1 Processor -- start 2023-12-01T03:10:42.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.702+0000 7f922e900640 1 -- start start 2023-12-01T03:10:42.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.702+0000 7f922e900640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92280cc350 0x7f9228145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f922e900640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f922815cc10 0x7f9228141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f922e900640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92280c8950 0x7f92280c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f922e900640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92280ce4d0 con 0x7f922815cc10 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f922e900640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92280c9260 con 0x7f92280cc350 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f922e900640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92280c9540 con 0x7f92280c8950 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f921ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f922815cc10 0x7f9228141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f921ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f922815cc10 0x7f9228141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55458/0 (socket says 172.21.15.115:55458) 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f921ffff640 1 -- 172.21.15.115:0/942583355 learned_addr learned my addr 172.21.15.115:0/942583355 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f922ce9d640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92280c8950 0x7f92280c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f921ffff640 1 -- 172.21.15.115:0/942583355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92280c8950 msgr2=0x7f92280c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f921ffff640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92280c8950 0x7f92280c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f921ffff640 1 -- 172.21.15.115:0/942583355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92280cc350 msgr2=0x7f9228145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f921ffff640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92280cc350 0x7f9228145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.703+0000 7f921ffff640 1 -- 172.21.15.115:0/942583355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220009b30 con 0x7f922815cc10 2023-12-01T03:10:42.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.704+0000 7f921ffff640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f922815cc10 0x7f9228141aa0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f921000c7f0 tx=0x7f921000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:42.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.704+0000 7f92257fa640 1 -- 172.21.15.115:0/942583355 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210004220 con 0x7f922815cc10 2023-12-01T03:10:42.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.704+0000 7f92257fa640 1 -- 172.21.15.115:0/942583355 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9210004380 con 0x7f922815cc10 2023-12-01T03:10:42.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.704+0000 7f92257fa640 1 -- 172.21.15.115:0/942583355 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210007840 con 0x7f922815cc10 2023-12-01T03:10:42.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.704+0000 7f922e900640 1 -- 172.21.15.115:0/942583355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92280c9820 con 0x7f922815cc10 2023-12-01T03:10:42.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.704+0000 7f922e900640 1 -- 172.21.15.115:0/942583355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9228141fe0 con 0x7f922815cc10 2023-12-01T03:10:42.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.705+0000 7f922e900640 1 -- 172.21.15.115:0/942583355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922804e890 con 0x7f922815cc10 2023-12-01T03:10:42.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.706+0000 7f92257fa640 1 -- 172.21.15.115:0/942583355 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9210007bb0 con 0x7f922815cc10 2023-12-01T03:10:42.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.707+0000 7f92257fa640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f920c06a550 0x7f920c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:42.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.707+0000 7f92257fa640 1 -- 172.21.15.115:0/942583355 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f9210088de0 con 0x7f922815cc10 2023-12-01T03:10:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.707+0000 7f9227fff640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f920c06a550 0x7f920c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:42.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.708+0000 7f9227fff640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f920c06a550 0x7f920c06ca10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9218006fd0 tx=0x7f9218008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:42.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.713+0000 7f92257fa640 1 -- 172.21.15.115:0/942583355 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9210053eb0 con 0x7f922815cc10 2023-12-01T03:10:42.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.906+0000 7f922e900640 1 -- 172.21.15.115:0/942583355 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92280ce610 con 0x7f922815cc10 2023-12-01T03:10:42.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.910+0000 7f92257fa640 1 -- 172.21.15.115:0/942583355 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) v1 ==== 74+0+19833 (secure 0 0 0) 0x7f9210007e60 con 0x7f922815cc10 2023-12-01T03:10:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:42.911 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":218,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:39.970384+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:10:37.180599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:10:37.961783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:10:42.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 -- 172.21.15.115:0/942583355 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f920c06a550 msgr2=0x7f920c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f920c06a550 0x7f920c06ca10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9218006fd0 tx=0x7f9218008040 comp rx=0 tx=0).stop 2023-12-01T03:10:42.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 -- 172.21.15.115:0/942583355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f922815cc10 msgr2=0x7f9228141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:42.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f922815cc10 0x7f9228141aa0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f921000c7f0 tx=0x7f921000ccb0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 -- 172.21.15.115:0/942583355 shutdown_connections 2023-12-01T03:10:42.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f920c06a550 0x7f920c06ca10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92280c8950 0x7f92280c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f922815cc10 0x7f9228141aa0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 --2- 172.21.15.115:0/942583355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92280cc350 0x7f9228145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:42.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 -- 172.21.15.115:0/942583355 >> 172.21.15.115:0/942583355 conn(0x7f9228069cc0 msgr2=0x7f92280af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:42.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 -- 172.21.15.115:0/942583355 shutdown_connections 2023-12-01T03:10:42.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:42.913+0000 7f922e900640 1 -- 172.21.15.115:0/942583355 wait complete. 2023-12-01T03:10:42.932 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T03:10:42.933 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:10:43.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.146+0000 7f0df6bb9640 1 Processor -- start 2023-12-01T03:10:43.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.146+0000 7f0df6bb9640 1 -- start start 2023-12-01T03:10:43.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.148+0000 7f0df6bb9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0df015d220 0x7f0df00cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.148+0000 7f0df6bb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df0070ac0 0x7f0df00ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.148+0000 7f0df6bb9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df015d600 con 0x7f0df00ad2b0 2023-12-01T03:10:43.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.148+0000 7f0df6bb9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df006cc10 con 0x7f0df015d220 2023-12-01T03:10:43.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.148+0000 7f0df6bb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df006c720 con 0x7f0df0070ac0 2023-12-01T03:10:43.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.148+0000 7f0df4955640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0df015d220 0x7f0df00cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.148+0000 7f0df4955640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0df015d220 0x7f0df00cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55462/0 (socket says 172.21.15.115:55462) 2023-12-01T03:10:43.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.148+0000 7f0df4955640 1 -- 172.21.15.115:0/3759658216 learned_addr learned my addr 172.21.15.115:0/3759658216 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:43.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.148+0000 7f0df5156640 1 --2- 172.21.15.115:0/3759658216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df0070ac0 0x7f0df00ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.149+0000 7f0df4955640 1 -- 172.21.15.115:0/3759658216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df0070ac0 msgr2=0x7f0df00ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:43.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.149+0000 7f0df4955640 1 --2- 172.21.15.115:0/3759658216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df0070ac0 0x7f0df00ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.149+0000 7f0df4955640 1 -- 172.21.15.115:0/3759658216 >> v1:172.21.15.115:6789/0 conn(0x7f0df00ad2b0 legacy=0x7f0df006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.149+0000 7f0df4955640 1 -- 172.21.15.115:0/3759658216 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df006c4b0 con 0x7f0df015d220 2023-12-01T03:10:43.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.150+0000 7f0df4955640 1 --2- 172.21.15.115:0/3759658216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df015d220 0x7f0df00cc570 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009870 tx=0x7f0de402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f046f65fc4a296c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:43.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.150+0000 7f0def7fe640 1 -- 172.21.15.115:0/3759658216 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4031640 con 0x7f0df015d220 2023-12-01T03:10:43.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.150+0000 7f0def7fe640 1 -- 172.21.15.115:0/3759658216 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de40317a0 con 0x7f0df015d220 2023-12-01T03:10:43.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.150+0000 7f0def7fe640 1 -- 172.21.15.115:0/3759658216 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4031b30 con 0x7f0df015d220 2023-12-01T03:10:43.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.151+0000 7f0df6bb9640 1 -- 172.21.15.115:0/3759658216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df015d220 msgr2=0x7f0df00cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.151+0000 7f0df6bb9640 1 --2- 172.21.15.115:0/3759658216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df015d220 0x7f0df00cc570 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009870 tx=0x7f0de402b8f0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.151+0000 7f0df6bb9640 1 -- 172.21.15.115:0/3759658216 shutdown_connections 2023-12-01T03:10:43.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.151+0000 7f0df6bb9640 1 --2- 172.21.15.115:0/3759658216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df0070ac0 0x7f0df00ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.151+0000 7f0df6bb9640 1 --2- 172.21.15.115:0/3759658216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df015d220 0x7f0df00cc570 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.151+0000 7f0df6bb9640 1 -- 172.21.15.115:0/3759658216 >> 172.21.15.115:0/3759658216 conn(0x7f0df00ce880 msgr2=0x7f0df00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:43.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.151+0000 7f0df6bb9640 1 -- 172.21.15.115:0/3759658216 shutdown_connections 2023-12-01T03:10:43.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.151+0000 7f0df6bb9640 1 -- 172.21.15.115:0/3759658216 wait complete. 2023-12-01T03:10:43.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.153+0000 7f0df6bb9640 1 Processor -- start 2023-12-01T03:10:43.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.153+0000 7f0df6bb9640 1 -- start start 2023-12-01T03:10:43.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df6bb9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df0070ac0 0x7f0df00c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df6bb9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00ad2b0 0x7f0df00c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df6bb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df015d220 0x7f0df00c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df6bb9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df015b0d0 con 0x7f0df0070ac0 2023-12-01T03:10:43.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df6bb9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df015b3b0 con 0x7f0df00ad2b0 2023-12-01T03:10:43.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df6bb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df015b690 con 0x7f0df015d220 2023-12-01T03:10:43.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df4955640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df0070ac0 0x7f0df00c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df4955640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df0070ac0 0x7f0df00c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55466/0 (socket says 172.21.15.115:55466) 2023-12-01T03:10:43.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df4955640 1 -- 172.21.15.115:0/809747069 learned_addr learned my addr 172.21.15.115:0/809747069 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df4955640 1 -- 172.21.15.115:0/809747069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df015d220 msgr2=0x7f0df00c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.154+0000 7f0df5156640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df015d220 0x7f0df00c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0df4955640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df015d220 0x7f0df00c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0df4955640 1 -- 172.21.15.115:0/809747069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00ad2b0 msgr2=0x7f0df00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0deffff640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00ad2b0 0x7f0df00c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0df4955640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00ad2b0 0x7f0df00c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0df4955640 1 -- 172.21.15.115:0/809747069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de40094d0 con 0x7f0df0070ac0 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0df4955640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df0070ac0 0x7f0df00c4a20 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f0de40090b0 tx=0x7f0de40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0ded7fa640 1 -- 172.21.15.115:0/809747069 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de403ac60 con 0x7f0df0070ac0 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0ded7fa640 1 -- 172.21.15.115:0/809747069 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de403ade0 con 0x7f0df0070ac0 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0ded7fa640 1 -- 172.21.15.115:0/809747069 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4044630 con 0x7f0df0070ac0 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0df6bb9640 1 -- 172.21.15.115:0/809747069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df015b970 con 0x7f0df0070ac0 2023-12-01T03:10:43.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.155+0000 7f0df6bb9640 1 -- 172.21.15.115:0/809747069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df015a310 con 0x7f0df0070ac0 2023-12-01T03:10:43.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.156+0000 7f0df6bb9640 1 -- 172.21.15.115:0/809747069 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df004e890 con 0x7f0df0070ac0 2023-12-01T03:10:43.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.157+0000 7f0ded7fa640 1 -- 172.21.15.115:0/809747069 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0de402f050 con 0x7f0df0070ac0 2023-12-01T03:10:43.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.158+0000 7f0ded7fa640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0dd406a550 0x7f0dd406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.158+0000 7f0ded7fa640 1 -- 172.21.15.115:0/809747069 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f0de40aad90 con 0x7f0df0070ac0 2023-12-01T03:10:43.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.158+0000 7f0deffff640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0dd406a550 0x7f0dd406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.159+0000 7f0deffff640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0dd406a550 0x7f0dd406ca10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0df00cf420 tx=0x7f0ddc009040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:43.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.162+0000 7f0ded7fa640 1 -- 172.21.15.115:0/809747069 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0de4075dd0 con 0x7f0df0070ac0 2023-12-01T03:10:43.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.347+0000 7f0df6bb9640 1 -- 172.21.15.115:0/809747069 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0df015b520 con 0x7f0dd406a550 2023-12-01T03:10:43.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.364+0000 7f0ded7fa640 1 -- 172.21.15.115:0/809747069 <== mgr.10407 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+523091 (secure 0 0 0) 0x7f0df015b520 con 0x7f0dd406a550 2023-12-01T03:10:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:43.376 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:10:43.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 -- 172.21.15.115:0/809747069 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0dd406a550 msgr2=0x7f0dd406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0dd406a550 0x7f0dd406ca10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0df00cf420 tx=0x7f0ddc009040 comp rx=0 tx=0).stop 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 -- 172.21.15.115:0/809747069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df0070ac0 msgr2=0x7f0df00c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df0070ac0 0x7f0df00c4a20 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f0de40090b0 tx=0x7f0de40046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 -- 172.21.15.115:0/809747069 shutdown_connections 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0dd406a550 0x7f0dd406ca10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0df015d220 0x7f0df00c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0df00ad2b0 0x7f0df00c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 --2- 172.21.15.115:0/809747069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0df0070ac0 0x7f0df00c4a20 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 -- 172.21.15.115:0/809747069 >> 172.21.15.115:0/809747069 conn(0x7f0df00ce880 msgr2=0x7f0df00cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 -- 172.21.15.115:0/809747069 shutdown_connections 2023-12-01T03:10:43.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.372+0000 7f0df6bb9640 1 -- 172.21.15.115:0/809747069 wait complete. 2023-12-01T03:10:43.391 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-12-01T03:10:41.799590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593971,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":586,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":367,"ondisk_log_size":367,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":202,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425676,"kb_used_data":22284,"kb_used_omap":0,"kb_used_meta":403328,"kb_avail":754549044,"statfs":{"total":773094113280,"available":772658221056,"internally_reserved":0,"allocated":22818816,"data_stored":11704194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":413007872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.330819"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507374+0000","last_change":"2023-12-01T03:10:38.507374+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507374+0000","last_undegraded":"2023-12-01T03:10:38.507374+0000","last_fullsized":"2023-12-01T03:10:38.507374+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:14.969594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"201'326","reported_seq":1121,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481853+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:10:37.481853+0000","last_peered":"2023-12-01T03:10:37.481853+0000","last_clean":"2023-12-01T03:10:37.481853+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:10:37.481853+0000","last_undegraded":"2023-12-01T03:10:37.481853+0000","last_fullsized":"2023-12-01T03:10:37.481853+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485007+0000","last_change":"2023-12-01T03:10:24.685430+0000","last_active":"2023-12-01T03:10:37.485007+0000","last_peered":"2023-12-01T03:10:37.485007+0000","last_clean":"2023-12-01T03:10:37.485007+0000","last_became_active":"2023-12-01T03:10:24.684919+0000","last_became_peered":"2023-12-01T03:10:24.684919+0000","last_unstale":"2023-12-01T03:10:37.485007+0000","last_undegraded":"2023-12-01T03:10:37.485007+0000","last_fullsized":"2023-12-01T03:10:37.485007+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481058+0000","last_change":"2023-12-01T03:10:25.189213+0000","last_active":"2023-12-01T03:10:37.481058+0000","last_peered":"2023-12-01T03:10:37.481058+0000","last_clean":"2023-12-01T03:10:37.481058+0000","last_became_active":"2023-12-01T03:10:25.188927+0000","last_became_peered":"2023-12-01T03:10:25.188927+0000","last_unstale":"2023-12-01T03:10:37.481058+0000","last_undegraded":"2023-12-01T03:10:37.481058+0000","last_fullsized":"2023-12-01T03:10:37.481058+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:21:53.633800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520651+0000","last_change":"2023-12-01T03:10:38.517472+0000","last_active":"2023-12-01T03:10:38.520651+0000","last_peered":"2023-12-01T03:10:38.520651+0000","last_clean":"2023-12-01T03:10:38.520651+0000","last_became_active":"2023-12-01T03:10:38.517218+0000","last_became_peered":"2023-12-01T03:10:38.517218+0000","last_unstale":"2023-12-01T03:10:38.520651+0000","last_undegraded":"2023-12-01T03:10:38.520651+0000","last_fullsized":"2023-12-01T03:10:38.520651+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:57.653086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499505+0000","last_change":"2023-12-01T03:10:38.499505+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499505+0000","last_undegraded":"2023-12-01T03:10:38.499505+0000","last_fullsized":"2023-12-01T03:10:38.499505+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481362+0000","last_change":"2023-12-01T03:10:24.690155+0000","last_active":"2023-12-01T03:10:37.481362+0000","last_peered":"2023-12-01T03:10:37.481362+0000","last_clean":"2023-12-01T03:10:37.481362+0000","last_became_active":"2023-12-01T03:10:24.689122+0000","last_became_peered":"2023-12-01T03:10:24.689122+0000","last_unstale":"2023-12-01T03:10:37.481362+0000","last_undegraded":"2023-12-01T03:10:37.481362+0000","last_fullsized":"2023-12-01T03:10:37.481362+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479122+0000","last_change":"2023-12-01T03:10:25.191828+0000","last_active":"2023-12-01T03:10:37.479122+0000","last_peered":"2023-12-01T03:10:37.479122+0000","last_clean":"2023-12-01T03:10:37.479122+0000","last_became_active":"2023-12-01T03:10:25.191436+0000","last_became_peered":"2023-12-01T03:10:25.191436+0000","last_unstale":"2023-12-01T03:10:37.479122+0000","last_undegraded":"2023-12-01T03:10:37.479122+0000","last_fullsized":"2023-12-01T03:10:37.479122+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525987+0000","last_change":"2023-12-01T03:10:38.524843+0000","last_active":"2023-12-01T03:10:38.525987+0000","last_peered":"2023-12-01T03:10:38.525987+0000","last_clean":"2023-12-01T03:10:38.525987+0000","last_became_active":"2023-12-01T03:10:38.524424+0000","last_became_peered":"2023-12-01T03:10:38.524424+0000","last_unstale":"2023-12-01T03:10:38.525987+0000","last_undegraded":"2023-12-01T03:10:38.525987+0000","last_fullsized":"2023-12-01T03:10:38.525987+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:32.967785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.509995+0000","last_change":"2023-12-01T03:10:38.509995+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.509995+0000","last_undegraded":"2023-12-01T03:10:38.509995+0000","last_fullsized":"2023-12-01T03:10:38.509995+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481866+0000","last_change":"2023-12-01T03:10:24.684248+0000","last_active":"2023-12-01T03:10:37.481866+0000","last_peered":"2023-12-01T03:10:37.481866+0000","last_clean":"2023-12-01T03:10:37.481866+0000","last_became_active":"2023-12-01T03:10:24.680714+0000","last_became_peered":"2023-12-01T03:10:24.680714+0000","last_unstale":"2023-12-01T03:10:37.481866+0000","last_undegraded":"2023-12-01T03:10:37.481866+0000","last_fullsized":"2023-12-01T03:10:37.481866+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:19:02.474558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481748+0000","last_change":"2023-12-01T03:10:25.174085+0000","last_active":"2023-12-01T03:10:37.481748+0000","last_peered":"2023-12-01T03:10:37.481748+0000","last_clean":"2023-12-01T03:10:37.481748+0000","last_became_active":"2023-12-01T03:10:25.173802+0000","last_became_peered":"2023-12-01T03:10:25.173802+0000","last_unstale":"2023-12-01T03:10:37.481748+0000","last_undegraded":"2023-12-01T03:10:37.481748+0000","last_fullsized":"2023-12-01T03:10:37.481748+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:39.145914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.516150+0000","last_change":"2023-12-01T03:10:38.512523+0000","last_active":"2023-12-01T03:10:38.516150+0000","last_peered":"2023-12-01T03:10:38.516150+0000","last_clean":"2023-12-01T03:10:38.516150+0000","last_became_active":"2023-12-01T03:10:38.512225+0000","last_became_peered":"2023-12-01T03:10:38.512225+0000","last_unstale":"2023-12-01T03:10:38.516150+0000","last_undegraded":"2023-12-01T03:10:38.516150+0000","last_fullsized":"2023-12-01T03:10:38.516150+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:05.819515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504879+0000","last_change":"2023-12-01T03:10:38.504879+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504879+0000","last_undegraded":"2023-12-01T03:10:38.504879+0000","last_fullsized":"2023-12-01T03:10:38.504879+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:39.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482088+0000","last_change":"2023-12-01T03:10:24.683240+0000","last_active":"2023-12-01T03:10:37.482088+0000","last_peered":"2023-12-01T03:10:37.482088+0000","last_clean":"2023-12-01T03:10:37.482088+0000","last_became_active":"2023-12-01T03:10:24.682933+0000","last_became_peered":"2023-12-01T03:10:24.682933+0000","last_unstale":"2023-12-01T03:10:37.482088+0000","last_undegraded":"2023-12-01T03:10:37.482088+0000","last_fullsized":"2023-12-01T03:10:37.482088+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:40.211424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472633+0000","last_change":"2023-12-01T03:10:25.181396+0000","last_active":"2023-12-01T03:10:37.472633+0000","last_peered":"2023-12-01T03:10:37.472633+0000","last_clean":"2023-12-01T03:10:37.472633+0000","last_became_active":"2023-12-01T03:10:25.181134+0000","last_became_peered":"2023-12-01T03:10:25.181134+0000","last_unstale":"2023-12-01T03:10:37.472633+0000","last_undegraded":"2023-12-01T03:10:37.472633+0000","last_fullsized":"2023-12-01T03:10:37.472633+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:04.776013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.523891+0000","last_change":"2023-12-01T03:10:38.522641+0000","last_active":"2023-12-01T03:10:38.523891+0000","last_peered":"2023-12-01T03:10:38.523891+0000","last_clean":"2023-12-01T03:10:38.523891+0000","last_became_active":"2023-12-01T03:10:38.522225+0000","last_became_peered":"2023-12-01T03:10:38.522225+0000","last_unstale":"2023-12-01T03:10:38.523891+0000","last_undegraded":"2023-12-01T03:10:38.523891+0000","last_fullsized":"2023-12-01T03:10:38.523891+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:13.766818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499285+0000","last_change":"2023-12-01T03:10:38.499285+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499285+0000","last_undegraded":"2023-12-01T03:10:38.499285+0000","last_fullsized":"2023-12-01T03:10:38.499285+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482608+0000","last_change":"2023-12-01T03:10:24.687326+0000","last_active":"2023-12-01T03:10:37.482608+0000","last_peered":"2023-12-01T03:10:37.482608+0000","last_clean":"2023-12-01T03:10:37.482608+0000","last_became_active":"2023-12-01T03:10:24.687036+0000","last_became_peered":"2023-12-01T03:10:24.687036+0000","last_unstale":"2023-12-01T03:10:37.482608+0000","last_undegraded":"2023-12-01T03:10:37.482608+0000","last_fullsized":"2023-12-01T03:10:37.482608+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:36:57.969605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482001+0000","last_change":"2023-12-01T03:10:25.190210+0000","last_active":"2023-12-01T03:10:37.482001+0000","last_peered":"2023-12-01T03:10:37.482001+0000","last_clean":"2023-12-01T03:10:37.482001+0000","last_became_active":"2023-12-01T03:10:25.189957+0000","last_became_peered":"2023-12-01T03:10:25.189957+0000","last_unstale":"2023-12-01T03:10:37.482001+0000","last_undegraded":"2023-12-01T03:10:37.482001+0000","last_fullsized":"2023-12-01T03:10:37.482001+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:01.783625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.518073+0000","last_change":"2023-12-01T03:10:38.514437+0000","last_active":"2023-12-01T03:10:38.518073+0000","last_peered":"2023-12-01T03:10:38.518073+0000","last_clean":"2023-12-01T03:10:38.518073+0000","last_became_active":"2023-12-01T03:10:38.514155+0000","last_became_peered":"2023-12-01T03:10:38.514155+0000","last_unstale":"2023-12-01T03:10:38.518073+0000","last_undegraded":"2023-12-01T03:10:38.518073+0000","last_fullsized":"2023-12-01T03:10:38.518073+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:06.817444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.503490+0000","last_change":"2023-12-01T03:10:38.503490+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.503490+0000","last_undegraded":"2023-12-01T03:10:38.503490+0000","last_fullsized":"2023-12-01T03:10:38.503490+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481503+0000","last_change":"2023-12-01T03:10:24.697595+0000","last_active":"2023-12-01T03:10:37.481503+0000","last_peered":"2023-12-01T03:10:37.481503+0000","last_clean":"2023-12-01T03:10:37.481503+0000","last_became_active":"2023-12-01T03:10:24.697311+0000","last_became_peered":"2023-12-01T03:10:24.697311+0000","last_unstale":"2023-12-01T03:10:37.481503+0000","last_undegraded":"2023-12-01T03:10:37.481503+0000","last_fullsized":"2023-12-01T03:10:37.481503+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:45.743456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483634+0000","last_change":"2023-12-01T03:10:25.192028+0000","last_active":"2023-12-01T03:10:37.483634+0000","last_peered":"2023-12-01T03:10:37.483634+0000","last_clean":"2023-12-01T03:10:37.483634+0000","last_became_active":"2023-12-01T03:10:25.191747+0000","last_became_peered":"2023-12-01T03:10:25.191747+0000","last_unstale":"2023-12-01T03:10:37.483634+0000","last_undegraded":"2023-12-01T03:10:37.483634+0000","last_fullsized":"2023-12-01T03:10:37.483634+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520576+0000","last_change":"2023-12-01T03:10:38.517012+0000","last_active":"2023-12-01T03:10:38.520576+0000","last_peered":"2023-12-01T03:10:38.520576+0000","last_clean":"2023-12-01T03:10:38.520576+0000","last_became_active":"2023-12-01T03:10:38.516749+0000","last_became_peered":"2023-12-01T03:10:38.516749+0000","last_unstale":"2023-12-01T03:10:38.520576+0000","last_undegraded":"2023-12-01T03:10:38.520576+0000","last_fullsized":"2023-12-01T03:10:38.520576+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:29.962162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.505124+0000","last_change":"2023-12-01T03:10:38.505124+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.505124+0000","last_undegraded":"2023-12-01T03:10:38.505124+0000","last_fullsized":"2023-12-01T03:10:38.505124+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488889+0000","last_change":"2023-12-01T03:10:24.697992+0000","last_active":"2023-12-01T03:10:37.488889+0000","last_peered":"2023-12-01T03:10:37.488889+0000","last_clean":"2023-12-01T03:10:37.488889+0000","last_became_active":"2023-12-01T03:10:24.696883+0000","last_became_peered":"2023-12-01T03:10:24.696883+0000","last_unstale":"2023-12-01T03:10:37.488889+0000","last_undegraded":"2023-12-01T03:10:37.488889+0000","last_fullsized":"2023-12-01T03:10:37.488889+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.487148+0000","last_change":"2023-12-01T03:10:25.175476+0000","last_active":"2023-12-01T03:10:37.487148+0000","last_peered":"2023-12-01T03:10:37.487148+0000","last_clean":"2023-12-01T03:10:37.487148+0000","last_became_active":"2023-12-01T03:10:25.175211+0000","last_became_peered":"2023-12-01T03:10:25.175211+0000","last_unstale":"2023-12-01T03:10:37.487148+0000","last_undegraded":"2023-12-01T03:10:37.487148+0000","last_fullsized":"2023-12-01T03:10:37.487148+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:22.358205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515962+0000","last_change":"2023-12-01T03:10:38.513347+0000","last_active":"2023-12-01T03:10:38.515962+0000","last_peered":"2023-12-01T03:10:38.515962+0000","last_clean":"2023-12-01T03:10:38.515962+0000","last_became_active":"2023-12-01T03:10:38.512782+0000","last_became_peered":"2023-12-01T03:10:38.512782+0000","last_unstale":"2023-12-01T03:10:38.515962+0000","last_undegraded":"2023-12-01T03:10:38.515962+0000","last_fullsized":"2023-12-01T03:10:38.515962+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:06.647413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.501239+0000","last_change":"2023-12-01T03:10:38.501239+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.501239+0000","last_undegraded":"2023-12-01T03:10:38.501239+0000","last_fullsized":"2023-12-01T03:10:38.501239+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.492807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"214'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489570+0000","last_change":"2023-12-01T03:10:24.697890+0000","last_active":"2023-12-01T03:10:37.489570+0000","last_peered":"2023-12-01T03:10:37.489570+0000","last_clean":"2023-12-01T03:10:37.489570+0000","last_became_active":"2023-12-01T03:10:24.697679+0000","last_became_peered":"2023-12-01T03:10:24.697679+0000","last_unstale":"2023-12-01T03:10:37.489570+0000","last_undegraded":"2023-12-01T03:10:37.489570+0000","last_fullsized":"2023-12-01T03:10:37.489570+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488204+0000","last_change":"2023-12-01T03:10:25.168359+0000","last_active":"2023-12-01T03:10:37.488204+0000","last_peered":"2023-12-01T03:10:37.488204+0000","last_clean":"2023-12-01T03:10:37.488204+0000","last_became_active":"2023-12-01T03:10:25.168082+0000","last_became_peered":"2023-12-01T03:10:25.168082+0000","last_unstale":"2023-12-01T03:10:37.488204+0000","last_undegraded":"2023-12-01T03:10:37.488204+0000","last_fullsized":"2023-12-01T03:10:37.488204+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:06.024028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525349+0000","last_change":"2023-12-01T03:10:38.524232+0000","last_active":"2023-12-01T03:10:38.525349+0000","last_peered":"2023-12-01T03:10:38.525349+0000","last_clean":"2023-12-01T03:10:38.525349+0000","last_became_active":"2023-12-01T03:10:38.523871+0000","last_became_peered":"2023-12-01T03:10:38.523871+0000","last_unstale":"2023-12-01T03:10:38.525349+0000","last_undegraded":"2023-12-01T03:10:38.525349+0000","last_fullsized":"2023-12-01T03:10:38.525349+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:06.647413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499505+0000","last_change":"2023-12-01T03:10:38.499505+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499505+0000","last_undegraded":"2023-12-01T03:10:38.499505+0000","last_fullsized":"2023-12-01T03:10:38.499505+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.7","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472938+0000","last_change":"2023-12-01T03:10:24.687160+0000","last_active":"2023-12-01T03:10:37.472938+0000","last_peered":"2023-12-01T03:10:37.472938+0000","last_clean":"2023-12-01T03:10:37.472938+0000","last_became_active":"2023-12-01T03:10:24.686184+0000","last_became_peered":"2023-12-01T03:10:24.686184+0000","last_unstale":"2023-12-01T03:10:37.472938+0000","last_undegraded":"2023-12-01T03:10:37.472938+0000","last_fullsized":"2023-12-01T03:10:37.472938+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:05.006954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489573+0000","last_change":"2023-12-01T03:10:25.184992+0000","last_active":"2023-12-01T03:10:37.489573+0000","last_peered":"2023-12-01T03:10:37.489573+0000","last_clean":"2023-12-01T03:10:37.489573+0000","last_became_active":"2023-12-01T03:10:25.184670+0000","last_became_peered":"2023-12-01T03:10:25.184670+0000","last_unstale":"2023-12-01T03:10:37.489573+0000","last_undegraded":"2023-12-01T03:10:37.489573+0000","last_fullsized":"2023-12-01T03:10:37.489573+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:40.143822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519720+0000","last_change":"2023-12-01T03:10:38.517303+0000","last_active":"2023-12-01T03:10:38.519720+0000","last_peered":"2023-12-01T03:10:38.519720+0000","last_clean":"2023-12-01T03:10:38.519720+0000","last_became_active":"2023-12-01T03:10:38.516948+0000","last_became_peered":"2023-12-01T03:10:38.516948+0000","last_unstale":"2023-12-01T03:10:38.519720+0000","last_undegraded":"2023-12-01T03:10:38.519720+0000","last_fullsized":"2023-12-01T03:10:38.519720+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:58:41.627485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504217+0000","last_change":"2023-12-01T03:10:38.504217+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504217+0000","last_undegraded":"2023-12-01T03:10:38.504217+0000","last_fullsized":"2023-12-01T03:10:38.504217+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:18.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472354+0000","last_change":"2023-12-01T03:10:24.685003+0000","last_active":"2023-12-01T03:10:37.472354+0000","last_peered":"2023-12-01T03:10:37.472354+0000","last_clean":"2023-12-01T03:10:37.472354+0000","last_became_active":"2023-12-01T03:10:24.684425+0000","last_became_peered":"2023-12-01T03:10:24.684425+0000","last_unstale":"2023-12-01T03:10:37.472354+0000","last_undegraded":"2023-12-01T03:10:37.472354+0000","last_fullsized":"2023-12-01T03:10:37.472354+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:45.173492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491598+0000","last_change":"2023-12-01T03:10:25.174479+0000","last_active":"2023-12-01T03:10:37.491598+0000","last_peered":"2023-12-01T03:10:37.491598+0000","last_clean":"2023-12-01T03:10:37.491598+0000","last_became_active":"2023-12-01T03:10:25.174194+0000","last_became_peered":"2023-12-01T03:10:25.174194+0000","last_unstale":"2023-12-01T03:10:37.491598+0000","last_undegraded":"2023-12-01T03:10:37.491598+0000","last_fullsized":"2023-12-01T03:10:37.491598+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519343+0000","last_change":"2023-12-01T03:10:38.516993+0000","last_active":"2023-12-01T03:10:38.519343+0000","last_peered":"2023-12-01T03:10:38.519343+0000","last_clean":"2023-12-01T03:10:38.519343+0000","last_became_active":"2023-12-01T03:10:38.516717+0000","last_became_peered":"2023-12-01T03:10:38.516717+0000","last_unstale":"2023-12-01T03:10:38.519343+0000","last_undegraded":"2023-12-01T03:10:38.519343+0000","last_fullsized":"2023-12-01T03:10:38.519343+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:51.689717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508392+0000","last_change":"2023-12-01T03:10:38.508392+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508392+0000","last_undegraded":"2023-12-01T03:10:38.508392+0000","last_fullsized":"2023-12-01T03:10:38.508392+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:53.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481282+0000","last_change":"2023-12-01T03:10:24.694594+0000","last_active":"2023-12-01T03:10:37.481282+0000","last_peered":"2023-12-01T03:10:37.481282+0000","last_clean":"2023-12-01T03:10:37.481282+0000","last_became_active":"2023-12-01T03:10:24.693808+0000","last_became_peered":"2023-12-01T03:10:24.693808+0000","last_unstale":"2023-12-01T03:10:37.481282+0000","last_undegraded":"2023-12-01T03:10:37.481282+0000","last_fullsized":"2023-12-01T03:10:37.481282+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472327+0000","last_change":"2023-12-01T03:10:25.173728+0000","last_active":"2023-12-01T03:10:37.472327+0000","last_peered":"2023-12-01T03:10:37.472327+0000","last_clean":"2023-12-01T03:10:37.472327+0000","last_became_active":"2023-12-01T03:10:25.173449+0000","last_became_peered":"2023-12-01T03:10:25.173449+0000","last_unstale":"2023-12-01T03:10:37.472327+0000","last_undegraded":"2023-12-01T03:10:37.472327+0000","last_fullsized":"2023-12-01T03:10:37.472327+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:20.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.513730+0000","last_change":"2023-12-01T03:10:38.508401+0000","last_active":"2023-12-01T03:10:38.513730+0000","last_peered":"2023-12-01T03:10:38.513730+0000","last_clean":"2023-12-01T03:10:38.513730+0000","last_became_active":"2023-12-01T03:10:38.508089+0000","last_became_peered":"2023-12-01T03:10:38.508089+0000","last_unstale":"2023-12-01T03:10:38.513730+0000","last_undegraded":"2023-12-01T03:10:38.513730+0000","last_fullsized":"2023-12-01T03:10:38.513730+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:56.645685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.509503+0000","last_change":"2023-12-01T03:10:38.509503+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.509503+0000","last_undegraded":"2023-12-01T03:10:38.509503+0000","last_fullsized":"2023-12-01T03:10:38.509503+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:27.357798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482736+0000","last_change":"2023-12-01T03:10:24.685775+0000","last_active":"2023-12-01T03:10:37.482736+0000","last_peered":"2023-12-01T03:10:37.482736+0000","last_clean":"2023-12-01T03:10:37.482736+0000","last_became_active":"2023-12-01T03:10:24.685178+0000","last_became_peered":"2023-12-01T03:10:24.685178+0000","last_unstale":"2023-12-01T03:10:37.482736+0000","last_undegraded":"2023-12-01T03:10:37.482736+0000","last_fullsized":"2023-12-01T03:10:37.482736+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:21.942226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483463+0000","last_change":"2023-12-01T03:10:25.180768+0000","last_active":"2023-12-01T03:10:37.483463+0000","last_peered":"2023-12-01T03:10:37.483463+0000","last_clean":"2023-12-01T03:10:37.483463+0000","last_became_active":"2023-12-01T03:10:25.179796+0000","last_became_peered":"2023-12-01T03:10:25.179796+0000","last_unstale":"2023-12-01T03:10:37.483463+0000","last_undegraded":"2023-12-01T03:10:37.483463+0000","last_fullsized":"2023-12-01T03:10:37.483463+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515225+0000","last_change":"2023-12-01T03:10:38.511676+0000","last_active":"2023-12-01T03:10:38.515225+0000","last_peered":"2023-12-01T03:10:38.515225+0000","last_clean":"2023-12-01T03:10:38.515225+0000","last_became_active":"2023-12-01T03:10:38.511406+0000","last_became_peered":"2023-12-01T03:10:38.511406+0000","last_unstale":"2023-12-01T03:10:38.515225+0000","last_undegraded":"2023-12-01T03:10:38.515225+0000","last_fullsized":"2023-12-01T03:10:38.515225+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:44.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.505476+0000","last_change":"2023-12-01T03:10:38.505476+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.505476+0000","last_undegraded":"2023-12-01T03:10:38.505476+0000","last_fullsized":"2023-12-01T03:10:38.505476+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:49:09.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.b","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481673+0000","last_change":"2023-12-01T03:10:24.695154+0000","last_active":"2023-12-01T03:10:37.481673+0000","last_peered":"2023-12-01T03:10:37.481673+0000","last_clean":"2023-12-01T03:10:37.481673+0000","last_became_active":"2023-12-01T03:10:24.694868+0000","last_became_peered":"2023-12-01T03:10:24.694868+0000","last_unstale":"2023-12-01T03:10:37.481673+0000","last_undegraded":"2023-12-01T03:10:37.481673+0000","last_fullsized":"2023-12-01T03:10:37.481673+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482517+0000","last_change":"2023-12-01T03:10:25.169678+0000","last_active":"2023-12-01T03:10:37.482517+0000","last_peered":"2023-12-01T03:10:37.482517+0000","last_clean":"2023-12-01T03:10:37.482517+0000","last_became_active":"2023-12-01T03:10:25.168714+0000","last_became_peered":"2023-12-01T03:10:25.168714+0000","last_unstale":"2023-12-01T03:10:37.482517+0000","last_undegraded":"2023-12-01T03:10:37.482517+0000","last_fullsized":"2023-12-01T03:10:37.482517+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.526019+0000","last_change":"2023-12-01T03:10:38.524888+0000","last_active":"2023-12-01T03:10:38.526019+0000","last_peered":"2023-12-01T03:10:38.526019+0000","last_clean":"2023-12-01T03:10:38.526019+0000","last_became_active":"2023-12-01T03:10:38.524601+0000","last_became_peered":"2023-12-01T03:10:38.524601+0000","last_unstale":"2023-12-01T03:10:38.526019+0000","last_undegraded":"2023-12-01T03:10:38.526019+0000","last_fullsized":"2023-12-01T03:10:38.526019+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:47.875991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.506299+0000","last_change":"2023-12-01T03:10:38.506299+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.506299+0000","last_undegraded":"2023-12-01T03:10:38.506299+0000","last_fullsized":"2023-12-01T03:10:38.506299+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:19.853707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473031+0000","last_change":"2023-12-01T03:10:24.683594+0000","last_active":"2023-12-01T03:10:37.473031+0000","last_peered":"2023-12-01T03:10:37.473031+0000","last_clean":"2023-12-01T03:10:37.473031+0000","last_became_active":"2023-12-01T03:10:24.679979+0000","last_became_peered":"2023-12-01T03:10:24.679979+0000","last_unstale":"2023-12-01T03:10:37.473031+0000","last_undegraded":"2023-12-01T03:10:37.473031+0000","last_fullsized":"2023-12-01T03:10:37.473031+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:12.511044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483931+0000","last_change":"2023-12-01T03:10:25.172837+0000","last_active":"2023-12-01T03:10:37.483931+0000","last_peered":"2023-12-01T03:10:37.483931+0000","last_clean":"2023-12-01T03:10:37.483931+0000","last_became_active":"2023-12-01T03:10:25.172602+0000","last_became_peered":"2023-12-01T03:10:25.172602+0000","last_unstale":"2023-12-01T03:10:37.483931+0000","last_undegraded":"2023-12-01T03:10:37.483931+0000","last_fullsized":"2023-12-01T03:10:37.483931+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521284+0000","last_change":"2023-12-01T03:10:38.517877+0000","last_active":"2023-12-01T03:10:38.521284+0000","last_peered":"2023-12-01T03:10:38.521284+0000","last_clean":"2023-12-01T03:10:38.521284+0000","last_became_active":"2023-12-01T03:10:38.517241+0000","last_became_peered":"2023-12-01T03:10:38.517241+0000","last_unstale":"2023-12-01T03:10:38.521284+0000","last_undegraded":"2023-12-01T03:10:38.521284+0000","last_fullsized":"2023-12-01T03:10:38.521284+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:32.967785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507192+0000","last_change":"2023-12-01T03:10:38.507192+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507192+0000","last_undegraded":"2023-12-01T03:10:38.507192+0000","last_fullsized":"2023-12-01T03:10:38.507192+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.d","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473150+0000","last_change":"2023-12-01T03:10:24.691398+0000","last_active":"2023-12-01T03:10:37.473150+0000","last_peered":"2023-12-01T03:10:37.473150+0000","last_clean":"2023-12-01T03:10:37.473150+0000","last_became_active":"2023-12-01T03:10:24.691110+0000","last_became_peered":"2023-12-01T03:10:24.691110+0000","last_unstale":"2023-12-01T03:10:37.473150+0000","last_undegraded":"2023-12-01T03:10:37.473150+0000","last_fullsized":"2023-12-01T03:10:37.473150+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:24.670899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483125+0000","last_change":"2023-12-01T03:10:25.178959+0000","last_active":"2023-12-01T03:10:37.483125+0000","last_peered":"2023-12-01T03:10:37.483125+0000","last_clean":"2023-12-01T03:10:37.483125+0000","last_became_active":"2023-12-01T03:10:25.178268+0000","last_became_peered":"2023-12-01T03:10:25.178268+0000","last_unstale":"2023-12-01T03:10:37.483125+0000","last_undegraded":"2023-12-01T03:10:37.483125+0000","last_fullsized":"2023-12-01T03:10:37.483125+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:55:31.814432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525071+0000","last_change":"2023-12-01T03:10:38.523799+0000","last_active":"2023-12-01T03:10:38.525071+0000","last_peered":"2023-12-01T03:10:38.525071+0000","last_clean":"2023-12-01T03:10:38.525071+0000","last_became_active":"2023-12-01T03:10:38.523512+0000","last_became_peered":"2023-12-01T03:10:38.523512+0000","last_unstale":"2023-12-01T03:10:38.525071+0000","last_undegraded":"2023-12-01T03:10:38.525071+0000","last_fullsized":"2023-12-01T03:10:38.525071+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:44.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504526+0000","last_change":"2023-12-01T03:10:38.504526+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504526+0000","last_undegraded":"2023-12-01T03:10:38.504526+0000","last_fullsized":"2023-12-01T03:10:38.504526+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:41.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482803+0000","last_change":"2023-12-01T03:10:24.688107+0000","last_active":"2023-12-01T03:10:37.482803+0000","last_peered":"2023-12-01T03:10:37.482803+0000","last_clean":"2023-12-01T03:10:37.482803+0000","last_became_active":"2023-12-01T03:10:24.686930+0000","last_became_peered":"2023-12-01T03:10:24.686930+0000","last_unstale":"2023-12-01T03:10:37.482803+0000","last_undegraded":"2023-12-01T03:10:37.482803+0000","last_fullsized":"2023-12-01T03:10:37.482803+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:44.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485627+0000","last_change":"2023-12-01T03:10:25.180913+0000","last_active":"2023-12-01T03:10:37.485627+0000","last_peered":"2023-12-01T03:10:37.485627+0000","last_clean":"2023-12-01T03:10:37.485627+0000","last_became_active":"2023-12-01T03:10:25.180626+0000","last_became_peered":"2023-12-01T03:10:25.180626+0000","last_unstale":"2023-12-01T03:10:37.485627+0000","last_undegraded":"2023-12-01T03:10:37.485627+0000","last_fullsized":"2023-12-01T03:10:37.485627+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520264+0000","last_change":"2023-12-01T03:10:38.519139+0000","last_active":"2023-12-01T03:10:38.520264+0000","last_peered":"2023-12-01T03:10:38.520264+0000","last_clean":"2023-12-01T03:10:38.520264+0000","last_became_active":"2023-12-01T03:10:38.518855+0000","last_became_peered":"2023-12-01T03:10:38.518855+0000","last_unstale":"2023-12-01T03:10:38.520264+0000","last_undegraded":"2023-12-01T03:10:38.520264+0000","last_fullsized":"2023-12-01T03:10:38.520264+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:52.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511251+0000","last_change":"2023-12-01T03:10:38.511251+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511251+0000","last_undegraded":"2023-12-01T03:10:38.511251+0000","last_fullsized":"2023-12-01T03:10:38.511251+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:47:30.021172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.f","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482669+0000","last_change":"2023-12-01T03:10:24.699945+0000","last_active":"2023-12-01T03:10:37.482669+0000","last_peered":"2023-12-01T03:10:37.482669+0000","last_clean":"2023-12-01T03:10:37.482669+0000","last_became_active":"2023-12-01T03:10:24.699598+0000","last_became_peered":"2023-12-01T03:10:24.699598+0000","last_unstale":"2023-12-01T03:10:37.482669+0000","last_undegraded":"2023-12-01T03:10:37.482669+0000","last_fullsized":"2023-12-01T03:10:37.482669+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492690+0000","last_change":"2023-12-01T03:10:25.181692+0000","last_active":"2023-12-01T03:10:37.492690+0000","last_peered":"2023-12-01T03:10:37.492690+0000","last_clean":"2023-12-01T03:10:37.492690+0000","last_became_active":"2023-12-01T03:10:25.181316+0000","last_became_peered":"2023-12-01T03:10:25.181316+0000","last_unstale":"2023-12-01T03:10:37.492690+0000","last_undegraded":"2023-12-01T03:10:37.492690+0000","last_fullsized":"2023-12-01T03:10:37.492690+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:57:07.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.522721+0000","last_change":"2023-12-01T03:10:38.518068+0000","last_active":"2023-12-01T03:10:38.522721+0000","last_peered":"2023-12-01T03:10:38.522721+0000","last_clean":"2023-12-01T03:10:38.522721+0000","last_became_active":"2023-12-01T03:10:38.517792+0000","last_became_peered":"2023-12-01T03:10:38.517792+0000","last_unstale":"2023-12-01T03:10:38.522721+0000","last_undegraded":"2023-12-01T03:10:38.522721+0000","last_fullsized":"2023-12-01T03:10:38.522721+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:01.616973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511262+0000","last_change":"2023-12-01T03:10:38.511262+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511262+0000","last_undegraded":"2023-12-01T03:10:38.511262+0000","last_fullsized":"2023-12-01T03:10:38.511262+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"213'8","reported_seq":21,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483272+0000","last_change":"2023-12-01T03:10:24.685759+0000","last_active":"2023-12-01T03:10:37.483272+0000","last_peered":"2023-12-01T03:10:37.483272+0000","last_clean":"2023-12-01T03:10:37.483272+0000","last_became_active":"2023-12-01T03:10:24.684800+0000","last_became_peered":"2023-12-01T03:10:24.684800+0000","last_unstale":"2023-12-01T03:10:37.483272+0000","last_undegraded":"2023-12-01T03:10:37.483272+0000","last_fullsized":"2023-12-01T03:10:37.483272+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:29.129070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478917+0000","last_change":"2023-12-01T03:10:25.193021+0000","last_active":"2023-12-01T03:10:37.478917+0000","last_peered":"2023-12-01T03:10:37.478917+0000","last_clean":"2023-12-01T03:10:37.478917+0000","last_became_active":"2023-12-01T03:10:25.192051+0000","last_became_peered":"2023-12-01T03:10:25.192051+0000","last_unstale":"2023-12-01T03:10:37.478917+0000","last_undegraded":"2023-12-01T03:10:37.478917+0000","last_fullsized":"2023-12-01T03:10:37.478917+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521581+0000","last_change":"2023-12-01T03:10:38.517969+0000","last_active":"2023-12-01T03:10:38.521581+0000","last_peered":"2023-12-01T03:10:38.521581+0000","last_clean":"2023-12-01T03:10:38.521581+0000","last_became_active":"2023-12-01T03:10:38.517693+0000","last_became_peered":"2023-12-01T03:10:38.517693+0000","last_unstale":"2023-12-01T03:10:38.521581+0000","last_undegraded":"2023-12-01T03:10:38.521581+0000","last_fullsized":"2023-12-01T03:10:38.521581+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:18.447021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.498276+0000","last_change":"2023-12-01T03:10:38.498276+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.498276+0000","last_undegraded":"2023-12-01T03:10:38.498276+0000","last_fullsized":"2023-12-01T03:10:38.498276+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479511+0000","last_change":"2023-12-01T03:10:24.694942+0000","last_active":"2023-12-01T03:10:37.479511+0000","last_peered":"2023-12-01T03:10:37.479511+0000","last_clean":"2023-12-01T03:10:37.479511+0000","last_became_active":"2023-12-01T03:10:24.694664+0000","last_became_peered":"2023-12-01T03:10:24.694664+0000","last_unstale":"2023-12-01T03:10:37.479511+0000","last_undegraded":"2023-12-01T03:10:37.479511+0000","last_fullsized":"2023-12-01T03:10:37.479511+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:38:02.332793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482916+0000","last_change":"2023-12-01T03:10:25.164054+0000","last_active":"2023-12-01T03:10:37.482916+0000","last_peered":"2023-12-01T03:10:37.482916+0000","last_clean":"2023-12-01T03:10:37.482916+0000","last_became_active":"2023-12-01T03:10:25.163787+0000","last_became_peered":"2023-12-01T03:10:25.163787+0000","last_unstale":"2023-12-01T03:10:37.482916+0000","last_undegraded":"2023-12-01T03:10:37.482916+0000","last_fullsized":"2023-12-01T03:10:37.482916+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:04.568421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521758+0000","last_change":"2023-12-01T03:10:38.520314+0000","last_active":"2023-12-01T03:10:38.521758+0000","last_peered":"2023-12-01T03:10:38.521758+0000","last_clean":"2023-12-01T03:10:38.521758+0000","last_became_active":"2023-12-01T03:10:38.519056+0000","last_became_peered":"2023-12-01T03:10:38.519056+0000","last_unstale":"2023-12-01T03:10:38.521758+0000","last_undegraded":"2023-12-01T03:10:38.521758+0000","last_fullsized":"2023-12-01T03:10:38.521758+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:40.686905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.502663+0000","last_change":"2023-12-01T03:10:38.502663+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.502663+0000","last_undegraded":"2023-12-01T03:10:38.502663+0000","last_fullsized":"2023-12-01T03:10:38.502663+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491751+0000","last_change":"2023-12-01T03:10:24.685823+0000","last_active":"2023-12-01T03:10:37.491751+0000","last_peered":"2023-12-01T03:10:37.491751+0000","last_clean":"2023-12-01T03:10:37.491751+0000","last_became_active":"2023-12-01T03:10:24.685560+0000","last_became_peered":"2023-12-01T03:10:24.685560+0000","last_unstale":"2023-12-01T03:10:37.491751+0000","last_undegraded":"2023-12-01T03:10:37.491751+0000","last_fullsized":"2023-12-01T03:10:37.491751+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483380+0000","last_change":"2023-12-01T03:10:25.183804+0000","last_active":"2023-12-01T03:10:37.483380+0000","last_peered":"2023-12-01T03:10:37.483380+0000","last_clean":"2023-12-01T03:10:37.483380+0000","last_became_active":"2023-12-01T03:10:25.183079+0000","last_became_peered":"2023-12-01T03:10:25.183079+0000","last_unstale":"2023-12-01T03:10:37.483380+0000","last_undegraded":"2023-12-01T03:10:37.483380+0000","last_fullsized":"2023-12-01T03:10:37.483380+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.510374+0000","last_change":"2023-12-01T03:10:38.508144+0000","last_active":"2023-12-01T03:10:38.510374+0000","last_peered":"2023-12-01T03:10:38.510374+0000","last_clean":"2023-12-01T03:10:38.510374+0000","last_became_active":"2023-12-01T03:10:38.507850+0000","last_became_peered":"2023-12-01T03:10:38.507850+0000","last_unstale":"2023-12-01T03:10:38.510374+0000","last_undegraded":"2023-12-01T03:10:38.510374+0000","last_fullsized":"2023-12-01T03:10:38.510374+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:52.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507590+0000","last_change":"2023-12-01T03:10:38.507590+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507590+0000","last_undegraded":"2023-12-01T03:10:38.507590+0000","last_fullsized":"2023-12-01T03:10:38.507590+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:45.156063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.13","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.484613+0000","last_change":"2023-12-01T03:10:24.692854+0000","last_active":"2023-12-01T03:10:37.484613+0000","last_peered":"2023-12-01T03:10:37.484613+0000","last_clean":"2023-12-01T03:10:37.484613+0000","last_became_active":"2023-12-01T03:10:24.692563+0000","last_became_peered":"2023-12-01T03:10:24.692563+0000","last_unstale":"2023-12-01T03:10:37.484613+0000","last_undegraded":"2023-12-01T03:10:37.484613+0000","last_fullsized":"2023-12-01T03:10:37.484613+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491687+0000","last_change":"2023-12-01T03:10:25.186716+0000","last_active":"2023-12-01T03:10:37.491687+0000","last_peered":"2023-12-01T03:10:37.491687+0000","last_clean":"2023-12-01T03:10:37.491687+0000","last_became_active":"2023-12-01T03:10:25.186422+0000","last_became_peered":"2023-12-01T03:10:25.186422+0000","last_unstale":"2023-12-01T03:10:37.491687+0000","last_undegraded":"2023-12-01T03:10:37.491687+0000","last_fullsized":"2023-12-01T03:10:37.491687+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.510490+0000","last_change":"2023-12-01T03:10:38.508098+0000","last_active":"2023-12-01T03:10:38.510490+0000","last_peered":"2023-12-01T03:10:38.510490+0000","last_clean":"2023-12-01T03:10:38.510490+0000","last_became_active":"2023-12-01T03:10:38.506510+0000","last_became_peered":"2023-12-01T03:10:38.506510+0000","last_unstale":"2023-12-01T03:10:38.510490+0000","last_undegraded":"2023-12-01T03:10:38.510490+0000","last_fullsized":"2023-12-01T03:10:38.510490+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:01.074406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504758+0000","last_change":"2023-12-01T03:10:38.504758+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504758+0000","last_undegraded":"2023-12-01T03:10:38.504758+0000","last_fullsized":"2023-12-01T03:10:38.504758+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:18.605693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"214'8","reported_seq":21,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481263+0000","last_change":"2023-12-01T03:10:24.697328+0000","last_active":"2023-12-01T03:10:37.481263+0000","last_peered":"2023-12-01T03:10:37.481263+0000","last_clean":"2023-12-01T03:10:37.481263+0000","last_became_active":"2023-12-01T03:10:24.696986+0000","last_became_peered":"2023-12-01T03:10:24.696986+0000","last_unstale":"2023-12-01T03:10:37.481263+0000","last_undegraded":"2023-12-01T03:10:37.481263+0000","last_fullsized":"2023-12-01T03:10:37.481263+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492498+0000","last_change":"2023-12-01T03:10:25.176864+0000","last_active":"2023-12-01T03:10:37.492498+0000","last_peered":"2023-12-01T03:10:37.492498+0000","last_clean":"2023-12-01T03:10:37.492498+0000","last_became_active":"2023-12-01T03:10:25.176588+0000","last_became_peered":"2023-12-01T03:10:25.176588+0000","last_unstale":"2023-12-01T03:10:37.492498+0000","last_undegraded":"2023-12-01T03:10:37.492498+0000","last_fullsized":"2023-12-01T03:10:37.492498+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520226+0000","last_change":"2023-12-01T03:10:38.519450+0000","last_active":"2023-12-01T03:10:38.520226+0000","last_peered":"2023-12-01T03:10:38.520226+0000","last_clean":"2023-12-01T03:10:38.520226+0000","last_became_active":"2023-12-01T03:10:38.519108+0000","last_became_peered":"2023-12-01T03:10:38.519108+0000","last_unstale":"2023-12-01T03:10:38.520226+0000","last_undegraded":"2023-12-01T03:10:38.520226+0000","last_fullsized":"2023-12-01T03:10:38.520226+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:58.926662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.506676+0000","last_change":"2023-12-01T03:10:38.506676+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.506676+0000","last_undegraded":"2023-12-01T03:10:38.506676+0000","last_fullsized":"2023-12-01T03:10:38.506676+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:07.829269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"213'3","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481279+0000","last_change":"2023-12-01T03:10:24.699528+0000","last_active":"2023-12-01T03:10:37.481279+0000","last_peered":"2023-12-01T03:10:37.481279+0000","last_clean":"2023-12-01T03:10:37.481279+0000","last_became_active":"2023-12-01T03:10:24.699152+0000","last_became_peered":"2023-12-01T03:10:24.699152+0000","last_unstale":"2023-12-01T03:10:37.481279+0000","last_undegraded":"2023-12-01T03:10:37.481279+0000","last_fullsized":"2023-12-01T03:10:37.481279+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:07.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.484358+0000","last_change":"2023-12-01T03:10:25.187487+0000","last_active":"2023-12-01T03:10:37.484358+0000","last_peered":"2023-12-01T03:10:37.484358+0000","last_clean":"2023-12-01T03:10:37.484358+0000","last_became_active":"2023-12-01T03:10:25.187198+0000","last_became_peered":"2023-12-01T03:10:25.187198+0000","last_unstale":"2023-12-01T03:10:37.484358+0000","last_undegraded":"2023-12-01T03:10:37.484358+0000","last_fullsized":"2023-12-01T03:10:37.484358+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:00.948964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.526023+0000","last_change":"2023-12-01T03:10:38.525203+0000","last_active":"2023-12-01T03:10:38.526023+0000","last_peered":"2023-12-01T03:10:38.526023+0000","last_clean":"2023-12-01T03:10:38.526023+0000","last_became_active":"2023-12-01T03:10:38.524926+0000","last_became_peered":"2023-12-01T03:10:38.524926+0000","last_unstale":"2023-12-01T03:10:38.526023+0000","last_undegraded":"2023-12-01T03:10:38.526023+0000","last_fullsized":"2023-12-01T03:10:38.526023+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:17.593868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.503576+0000","last_change":"2023-12-01T03:10:38.503576+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.503576+0000","last_undegraded":"2023-12-01T03:10:38.503576+0000","last_fullsized":"2023-12-01T03:10:38.503576+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492851+0000","last_change":"2023-12-01T03:10:24.691537+0000","last_active":"2023-12-01T03:10:37.492851+0000","last_peered":"2023-12-01T03:10:37.492851+0000","last_clean":"2023-12-01T03:10:37.492851+0000","last_became_active":"2023-12-01T03:10:24.691168+0000","last_became_peered":"2023-12-01T03:10:24.691168+0000","last_unstale":"2023-12-01T03:10:37.492851+0000","last_undegraded":"2023-12-01T03:10:37.492851+0000","last_fullsized":"2023-12-01T03:10:37.492851+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492811+0000","last_change":"2023-12-01T03:10:25.178648+0000","last_active":"2023-12-01T03:10:37.492811+0000","last_peered":"2023-12-01T03:10:37.492811+0000","last_clean":"2023-12-01T03:10:37.492811+0000","last_became_active":"2023-12-01T03:10:25.178352+0000","last_became_peered":"2023-12-01T03:10:25.178352+0000","last_unstale":"2023-12-01T03:10:37.492811+0000","last_undegraded":"2023-12-01T03:10:37.492811+0000","last_fullsized":"2023-12-01T03:10:37.492811+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519785+0000","last_change":"2023-12-01T03:10:38.515822+0000","last_active":"2023-12-01T03:10:38.519785+0000","last_peered":"2023-12-01T03:10:38.519785+0000","last_clean":"2023-12-01T03:10:38.519785+0000","last_became_active":"2023-12-01T03:10:38.515536+0000","last_became_peered":"2023-12-01T03:10:38.515536+0000","last_unstale":"2023-12-01T03:10:38.519785+0000","last_undegraded":"2023-12-01T03:10:38.519785+0000","last_fullsized":"2023-12-01T03:10:38.519785+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:10:36.574945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.510307+0000","last_change":"2023-12-01T03:10:38.510307+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.510307+0000","last_undegraded":"2023-12-01T03:10:38.510307+0000","last_fullsized":"2023-12-01T03:10:38.510307+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472906+0000","last_change":"2023-12-01T03:10:24.687709+0000","last_active":"2023-12-01T03:10:37.472906+0000","last_peered":"2023-12-01T03:10:37.472906+0000","last_clean":"2023-12-01T03:10:37.472906+0000","last_became_active":"2023-12-01T03:10:24.687452+0000","last_became_peered":"2023-12-01T03:10:24.687452+0000","last_unstale":"2023-12-01T03:10:37.472906+0000","last_undegraded":"2023-12-01T03:10:37.472906+0000","last_fullsized":"2023-12-01T03:10:37.472906+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:39.126749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489332+0000","last_change":"2023-12-01T03:10:25.176941+0000","last_active":"2023-12-01T03:10:37.489332+0000","last_peered":"2023-12-01T03:10:37.489332+0000","last_clean":"2023-12-01T03:10:37.489332+0000","last_became_active":"2023-12-01T03:10:25.175692+0000","last_became_peered":"2023-12-01T03:10:25.175692+0000","last_unstale":"2023-12-01T03:10:37.489332+0000","last_undegraded":"2023-12-01T03:10:37.489332+0000","last_fullsized":"2023-12-01T03:10:37.489332+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.513115+0000","last_change":"2023-12-01T03:10:38.509844+0000","last_active":"2023-12-01T03:10:38.513115+0000","last_peered":"2023-12-01T03:10:38.513115+0000","last_clean":"2023-12-01T03:10:38.513115+0000","last_became_active":"2023-12-01T03:10:38.509319+0000","last_became_peered":"2023-12-01T03:10:38.509319+0000","last_unstale":"2023-12-01T03:10:38.513115+0000","last_undegraded":"2023-12-01T03:10:38.513115+0000","last_fullsized":"2023-12-01T03:10:38.513115+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:12.432439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507312+0000","last_change":"2023-12-01T03:10:38.507312+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507312+0000","last_undegraded":"2023-12-01T03:10:38.507312+0000","last_fullsized":"2023-12-01T03:10:38.507312+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:53.973502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483145+0000","last_change":"2023-12-01T03:10:24.686553+0000","last_active":"2023-12-01T03:10:37.483145+0000","last_peered":"2023-12-01T03:10:37.483145+0000","last_clean":"2023-12-01T03:10:37.483145+0000","last_became_active":"2023-12-01T03:10:24.686136+0000","last_became_peered":"2023-12-01T03:10:24.686136+0000","last_unstale":"2023-12-01T03:10:37.483145+0000","last_undegraded":"2023-12-01T03:10:37.483145+0000","last_fullsized":"2023-12-01T03:10:37.483145+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:18.663097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472405+0000","last_change":"2023-12-01T03:10:25.182793+0000","last_active":"2023-12-01T03:10:37.472405+0000","last_peered":"2023-12-01T03:10:37.472405+0000","last_clean":"2023-12-01T03:10:37.472405+0000","last_became_active":"2023-12-01T03:10:25.182524+0000","last_became_peered":"2023-12-01T03:10:25.182524+0000","last_unstale":"2023-12-01T03:10:37.472405+0000","last_undegraded":"2023-12-01T03:10:37.472405+0000","last_fullsized":"2023-12-01T03:10:37.472405+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:25.370249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525096+0000","last_change":"2023-12-01T03:10:38.523078+0000","last_active":"2023-12-01T03:10:38.525096+0000","last_peered":"2023-12-01T03:10:38.525096+0000","last_clean":"2023-12-01T03:10:38.525096+0000","last_became_active":"2023-12-01T03:10:38.522603+0000","last_became_peered":"2023-12-01T03:10:38.522603+0000","last_unstale":"2023-12-01T03:10:38.525096+0000","last_undegraded":"2023-12-01T03:10:38.525096+0000","last_fullsized":"2023-12-01T03:10:38.525096+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:12.432439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.506778+0000","last_change":"2023-12-01T03:10:38.506778+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.506778+0000","last_undegraded":"2023-12-01T03:10:38.506778+0000","last_fullsized":"2023-12-01T03:10:38.506778+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1a","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480595+0000","last_change":"2023-12-01T03:10:24.695315+0000","last_active":"2023-12-01T03:10:37.480595+0000","last_peered":"2023-12-01T03:10:37.480595+0000","last_clean":"2023-12-01T03:10:37.480595+0000","last_became_active":"2023-12-01T03:10:24.694274+0000","last_became_peered":"2023-12-01T03:10:24.694274+0000","last_unstale":"2023-12-01T03:10:37.480595+0000","last_undegraded":"2023-12-01T03:10:37.480595+0000","last_fullsized":"2023-12-01T03:10:37.480595+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:19.256208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479974+0000","last_change":"2023-12-01T03:10:25.188350+0000","last_active":"2023-12-01T03:10:37.479974+0000","last_peered":"2023-12-01T03:10:37.479974+0000","last_clean":"2023-12-01T03:10:37.479974+0000","last_became_active":"2023-12-01T03:10:25.187212+0000","last_became_peered":"2023-12-01T03:10:25.187212+0000","last_unstale":"2023-12-01T03:10:37.479974+0000","last_undegraded":"2023-12-01T03:10:37.479974+0000","last_fullsized":"2023-12-01T03:10:37.479974+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.522745+0000","last_change":"2023-12-01T03:10:38.521794+0000","last_active":"2023-12-01T03:10:38.522745+0000","last_peered":"2023-12-01T03:10:38.522745+0000","last_clean":"2023-12-01T03:10:38.522745+0000","last_became_active":"2023-12-01T03:10:38.521546+0000","last_became_peered":"2023-12-01T03:10:38.521546+0000","last_unstale":"2023-12-01T03:10:38.522745+0000","last_undegraded":"2023-12-01T03:10:38.522745+0000","last_fullsized":"2023-12-01T03:10:38.522745+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:17.386276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.510675+0000","last_change":"2023-12-01T03:10:38.510675+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.510675+0000","last_undegraded":"2023-12-01T03:10:38.510675+0000","last_fullsized":"2023-12-01T03:10:38.510675+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:33:51.590745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480220+0000","last_change":"2023-12-01T03:10:24.695067+0000","last_active":"2023-12-01T03:10:37.480220+0000","last_peered":"2023-12-01T03:10:37.480220+0000","last_clean":"2023-12-01T03:10:37.480220+0000","last_became_active":"2023-12-01T03:10:24.694560+0000","last_became_peered":"2023-12-01T03:10:24.694560+0000","last_unstale":"2023-12-01T03:10:37.480220+0000","last_undegraded":"2023-12-01T03:10:37.480220+0000","last_fullsized":"2023-12-01T03:10:37.480220+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482278+0000","last_change":"2023-12-01T03:10:25.175874+0000","last_active":"2023-12-01T03:10:37.482278+0000","last_peered":"2023-12-01T03:10:37.482278+0000","last_clean":"2023-12-01T03:10:37.482278+0000","last_became_active":"2023-12-01T03:10:25.175584+0000","last_became_peered":"2023-12-01T03:10:25.175584+0000","last_unstale":"2023-12-01T03:10:37.482278+0000","last_undegraded":"2023-12-01T03:10:37.482278+0000","last_fullsized":"2023-12-01T03:10:37.482278+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:46.157610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519254+0000","last_change":"2023-12-01T03:10:38.515775+0000","last_active":"2023-12-01T03:10:38.519254+0000","last_peered":"2023-12-01T03:10:38.519254+0000","last_clean":"2023-12-01T03:10:38.519254+0000","last_became_active":"2023-12-01T03:10:38.515021+0000","last_became_peered":"2023-12-01T03:10:38.515021+0000","last_unstale":"2023-12-01T03:10:38.519254+0000","last_undegraded":"2023-12-01T03:10:38.519254+0000","last_fullsized":"2023-12-01T03:10:38.519254+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:35.878213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.509877+0000","last_change":"2023-12-01T03:10:38.509877+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.509877+0000","last_undegraded":"2023-12-01T03:10:38.509877+0000","last_fullsized":"2023-12-01T03:10:38.509877+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:59.987290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515498+0000","last_change":"2023-12-01T03:10:38.513581+0000","last_active":"2023-12-01T03:10:38.515498+0000","last_peered":"2023-12-01T03:10:38.515498+0000","last_clean":"2023-12-01T03:10:38.515498+0000","last_became_active":"2023-12-01T03:10:38.513306+0000","last_became_peered":"2023-12-01T03:10:38.513306+0000","last_unstale":"2023-12-01T03:10:38.515498+0000","last_undegraded":"2023-12-01T03:10:38.515498+0000","last_fullsized":"2023-12-01T03:10:38.515498+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:13.957769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.484449+0000","last_change":"2023-12-01T03:10:24.686931+0000","last_active":"2023-12-01T03:10:37.484449+0000","last_peered":"2023-12-01T03:10:37.484449+0000","last_clean":"2023-12-01T03:10:37.484449+0000","last_became_active":"2023-12-01T03:10:24.686642+0000","last_became_peered":"2023-12-01T03:10:24.686642+0000","last_unstale":"2023-12-01T03:10:37.484449+0000","last_undegraded":"2023-12-01T03:10:37.484449+0000","last_fullsized":"2023-12-01T03:10:37.484449+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483966+0000","last_change":"2023-12-01T03:10:25.175999+0000","last_active":"2023-12-01T03:10:37.483966+0000","last_peered":"2023-12-01T03:10:37.483966+0000","last_clean":"2023-12-01T03:10:37.483966+0000","last_became_active":"2023-12-01T03:10:25.175764+0000","last_became_peered":"2023-12-01T03:10:25.175764+0000","last_unstale":"2023-12-01T03:10:37.483966+0000","last_undegraded":"2023-12-01T03:10:37.483966+0000","last_fullsized":"2023-12-01T03:10:37.483966+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:43.098776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511444+0000","last_change":"2023-12-01T03:10:38.511444+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511444+0000","last_undegraded":"2023-12-01T03:10:38.511444+0000","last_fullsized":"2023-12-01T03:10:38.511444+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.522053+0000","last_change":"2023-12-01T03:10:38.520344+0000","last_active":"2023-12-01T03:10:38.522053+0000","last_peered":"2023-12-01T03:10:38.522053+0000","last_clean":"2023-12-01T03:10:38.522053+0000","last_became_active":"2023-12-01T03:10:38.520086+0000","last_became_peered":"2023-12-01T03:10:38.520086+0000","last_unstale":"2023-12-01T03:10:38.522053+0000","last_undegraded":"2023-12-01T03:10:38.522053+0000","last_fullsized":"2023-12-01T03:10:38.522053+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:01.616973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480900+0000","last_change":"2023-12-01T03:10:24.700378+0000","last_active":"2023-12-01T03:10:37.480900+0000","last_peered":"2023-12-01T03:10:37.480900+0000","last_clean":"2023-12-01T03:10:37.480900+0000","last_became_active":"2023-12-01T03:10:24.700126+0000","last_became_peered":"2023-12-01T03:10:24.700126+0000","last_unstale":"2023-12-01T03:10:37.480900+0000","last_undegraded":"2023-12-01T03:10:37.480900+0000","last_fullsized":"2023-12-01T03:10:37.480900+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483015+0000","last_change":"2023-12-01T03:10:25.179131+0000","last_active":"2023-12-01T03:10:37.483015+0000","last_peered":"2023-12-01T03:10:37.483015+0000","last_clean":"2023-12-01T03:10:37.483015+0000","last_became_active":"2023-12-01T03:10:25.178708+0000","last_became_peered":"2023-12-01T03:10:25.178708+0000","last_unstale":"2023-12-01T03:10:37.483015+0000","last_undegraded":"2023-12-01T03:10:37.483015+0000","last_fullsized":"2023-12-01T03:10:37.483015+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:43.370389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513293+0000","last_change":"2023-12-01T03:10:38.513293+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513293+0000","last_undegraded":"2023-12-01T03:10:38.513293+0000","last_fullsized":"2023-12-01T03:10:38.513293+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:46:18.989087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1e","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485050+0000","last_change":"2023-12-01T03:10:24.688812+0000","last_active":"2023-12-01T03:10:37.485050+0000","last_peered":"2023-12-01T03:10:37.485050+0000","last_clean":"2023-12-01T03:10:37.485050+0000","last_became_active":"2023-12-01T03:10:24.687577+0000","last_became_peered":"2023-12-01T03:10:24.687577+0000","last_unstale":"2023-12-01T03:10:37.485050+0000","last_undegraded":"2023-12-01T03:10:37.485050+0000","last_fullsized":"2023-12-01T03:10:37.485050+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492668+0000","last_change":"2023-12-01T03:10:25.180335+0000","last_active":"2023-12-01T03:10:37.492668+0000","last_peered":"2023-12-01T03:10:37.492668+0000","last_clean":"2023-12-01T03:10:37.492668+0000","last_became_active":"2023-12-01T03:10:25.180020+0000","last_became_peered":"2023-12-01T03:10:25.180020+0000","last_unstale":"2023-12-01T03:10:37.492668+0000","last_undegraded":"2023-12-01T03:10:37.492668+0000","last_fullsized":"2023-12-01T03:10:37.492668+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521149+0000","last_change":"2023-12-01T03:10:38.519061+0000","last_active":"2023-12-01T03:10:38.521149+0000","last_peered":"2023-12-01T03:10:38.521149+0000","last_clean":"2023-12-01T03:10:38.521149+0000","last_became_active":"2023-12-01T03:10:38.518091+0000","last_became_peered":"2023-12-01T03:10:38.518091+0000","last_unstale":"2023-12-01T03:10:38.521149+0000","last_undegraded":"2023-12-01T03:10:38.521149+0000","last_fullsized":"2023-12-01T03:10:38.521149+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:42.963999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512195+0000","last_change":"2023-12-01T03:10:38.512195+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512195+0000","last_undegraded":"2023-12-01T03:10:38.512195+0000","last_fullsized":"2023-12-01T03:10:38.512195+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"213'4","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489298+0000","last_change":"2023-12-01T03:10:24.697099+0000","last_active":"2023-12-01T03:10:37.489298+0000","last_peered":"2023-12-01T03:10:37.489298+0000","last_clean":"2023-12-01T03:10:37.489298+0000","last_became_active":"2023-12-01T03:10:24.696783+0000","last_became_peered":"2023-12-01T03:10:24.696783+0000","last_unstale":"2023-12-01T03:10:37.489298+0000","last_undegraded":"2023-12-01T03:10:37.489298+0000","last_fullsized":"2023-12-01T03:10:37.489298+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:07.101180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479580+0000","last_change":"2023-12-01T03:10:25.191865+0000","last_active":"2023-12-01T03:10:37.479580+0000","last_peered":"2023-12-01T03:10:37.479580+0000","last_clean":"2023-12-01T03:10:37.479580+0000","last_became_active":"2023-12-01T03:10:25.189277+0000","last_became_peered":"2023-12-01T03:10:25.189277+0000","last_unstale":"2023-12-01T03:10:37.479580+0000","last_undegraded":"2023-12-01T03:10:37.479580+0000","last_fullsized":"2023-12-01T03:10:37.479580+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521580+0000","last_change":"2023-12-01T03:10:38.517851+0000","last_active":"2023-12-01T03:10:38.521580+0000","last_peered":"2023-12-01T03:10:38.521580+0000","last_clean":"2023-12-01T03:10:38.521580+0000","last_became_active":"2023-12-01T03:10:38.517507+0000","last_became_peered":"2023-12-01T03:10:38.517507+0000","last_unstale":"2023-12-01T03:10:38.521580+0000","last_undegraded":"2023-12-01T03:10:38.521580+0000","last_fullsized":"2023-12-01T03:10:38.521580+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:22.592854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.509424+0000","last_change":"2023-12-01T03:10:38.509424+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.509424+0000","last_undegraded":"2023-12-01T03:10:38.509424+0000","last_fullsized":"2023-12-01T03:10:38.509424+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:36.187885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478313+0000","last_change":"2023-12-01T03:10:25.185385+0000","last_active":"2023-12-01T03:10:37.478313+0000","last_peered":"2023-12-01T03:10:37.478313+0000","last_clean":"2023-12-01T03:10:37.478313+0000","last_became_active":"2023-12-01T03:10:25.185098+0000","last_became_peered":"2023-12-01T03:10:25.185098+0000","last_unstale":"2023-12-01T03:10:37.478313+0000","last_undegraded":"2023-12-01T03:10:37.478313+0000","last_fullsized":"2023-12-01T03:10:37.478313+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500232+0000","last_change":"2023-12-01T03:10:38.500232+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500232+0000","last_undegraded":"2023-12-01T03:10:38.500232+0000","last_fullsized":"2023-12-01T03:10:38.500232+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478702+0000","last_change":"2023-12-01T03:10:25.186126+0000","last_active":"2023-12-01T03:10:37.478702+0000","last_peered":"2023-12-01T03:10:37.478702+0000","last_clean":"2023-12-01T03:10:37.478702+0000","last_became_active":"2023-12-01T03:10:25.185741+0000","last_became_peered":"2023-12-01T03:10:25.185741+0000","last_unstale":"2023-12-01T03:10:37.478702+0000","last_undegraded":"2023-12-01T03:10:37.478702+0000","last_fullsized":"2023-12-01T03:10:37.478702+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.501528+0000","last_change":"2023-12-01T03:10:38.501528+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.501528+0000","last_undegraded":"2023-12-01T03:10:38.501528+0000","last_fullsized":"2023-12-01T03:10:38.501528+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483578+0000","last_change":"2023-12-01T03:10:25.175090+0000","last_active":"2023-12-01T03:10:37.483578+0000","last_peered":"2023-12-01T03:10:37.483578+0000","last_clean":"2023-12-01T03:10:37.483578+0000","last_became_active":"2023-12-01T03:10:25.174835+0000","last_became_peered":"2023-12-01T03:10:25.174835+0000","last_unstale":"2023-12-01T03:10:37.483578+0000","last_undegraded":"2023-12-01T03:10:37.483578+0000","last_fullsized":"2023-12-01T03:10:37.483578+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.505382+0000","last_change":"2023-12-01T03:10:38.505382+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.505382+0000","last_undegraded":"2023-12-01T03:10:38.505382+0000","last_fullsized":"2023-12-01T03:10:38.505382+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:52.701542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.484004+0000","last_change":"2023-12-01T03:10:25.178098+0000","last_active":"2023-12-01T03:10:37.484004+0000","last_peered":"2023-12-01T03:10:37.484004+0000","last_clean":"2023-12-01T03:10:37.484004+0000","last_became_active":"2023-12-01T03:10:25.177031+0000","last_became_peered":"2023-12-01T03:10:25.177031+0000","last_unstale":"2023-12-01T03:10:37.484004+0000","last_undegraded":"2023-12-01T03:10:37.484004+0000","last_fullsized":"2023-12-01T03:10:37.484004+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.496510+0000","last_change":"2023-12-01T03:10:38.496510+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.496510+0000","last_undegraded":"2023-12-01T03:10:38.496510+0000","last_fullsized":"2023-12-01T03:10:38.496510+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491934+0000","last_change":"2023-12-01T03:10:25.176284+0000","last_active":"2023-12-01T03:10:37.491934+0000","last_peered":"2023-12-01T03:10:37.491934+0000","last_clean":"2023-12-01T03:10:37.491934+0000","last_became_active":"2023-12-01T03:10:25.175494+0000","last_became_peered":"2023-12-01T03:10:25.175494+0000","last_unstale":"2023-12-01T03:10:37.491934+0000","last_undegraded":"2023-12-01T03:10:37.491934+0000","last_fullsized":"2023-12-01T03:10:37.491934+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.510351+0000","last_change":"2023-12-01T03:10:38.510351+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.510351+0000","last_undegraded":"2023-12-01T03:10:38.510351+0000","last_fullsized":"2023-12-01T03:10:38.510351+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:02.628798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488821+0000","last_change":"2023-12-01T03:10:25.178072+0000","last_active":"2023-12-01T03:10:37.488821+0000","last_peered":"2023-12-01T03:10:37.488821+0000","last_clean":"2023-12-01T03:10:37.488821+0000","last_became_active":"2023-12-01T03:10:25.177778+0000","last_became_peered":"2023-12-01T03:10:25.177778+0000","last_unstale":"2023-12-01T03:10:37.488821+0000","last_undegraded":"2023-12-01T03:10:37.488821+0000","last_fullsized":"2023-12-01T03:10:37.488821+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:48.799119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.496580+0000","last_change":"2023-12-01T03:10:38.496580+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.496580+0000","last_undegraded":"2023-12-01T03:10:38.496580+0000","last_fullsized":"2023-12-01T03:10:38.496580+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.484882+0000","last_change":"2023-12-01T03:10:25.181469+0000","last_active":"2023-12-01T03:10:37.484882+0000","last_peered":"2023-12-01T03:10:37.484882+0000","last_clean":"2023-12-01T03:10:37.484882+0000","last_became_active":"2023-12-01T03:10:25.180646+0000","last_became_peered":"2023-12-01T03:10:25.180646+0000","last_unstale":"2023-12-01T03:10:37.484882+0000","last_undegraded":"2023-12-01T03:10:37.484882+0000","last_fullsized":"2023-12-01T03:10:37.484882+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:53.001661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504005+0000","last_change":"2023-12-01T03:10:38.504005+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504005+0000","last_undegraded":"2023-12-01T03:10:38.504005+0000","last_fullsized":"2023-12-01T03:10:38.504005+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.484309+0000","last_change":"2023-12-01T03:10:25.179679+0000","last_active":"2023-12-01T03:10:37.484309+0000","last_peered":"2023-12-01T03:10:37.484309+0000","last_clean":"2023-12-01T03:10:37.484309+0000","last_became_active":"2023-12-01T03:10:25.179442+0000","last_became_peered":"2023-12-01T03:10:25.179442+0000","last_unstale":"2023-12-01T03:10:37.484309+0000","last_undegraded":"2023-12-01T03:10:37.484309+0000","last_fullsized":"2023-12-01T03:10:37.484309+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:57:25.728865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500994+0000","last_change":"2023-12-01T03:10:38.500994+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500994+0000","last_undegraded":"2023-12-01T03:10:38.500994+0000","last_fullsized":"2023-12-01T03:10:38.500994+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478175+0000","last_change":"2023-12-01T03:10:25.175356+0000","last_active":"2023-12-01T03:10:37.478175+0000","last_peered":"2023-12-01T03:10:37.478175+0000","last_clean":"2023-12-01T03:10:37.478175+0000","last_became_active":"2023-12-01T03:10:25.175086+0000","last_became_peered":"2023-12-01T03:10:25.175086+0000","last_unstale":"2023-12-01T03:10:37.478175+0000","last_undegraded":"2023-12-01T03:10:37.478175+0000","last_fullsized":"2023-12-01T03:10:37.478175+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504261+0000","last_change":"2023-12-01T03:10:38.504261+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504261+0000","last_undegraded":"2023-12-01T03:10:38.504261+0000","last_fullsized":"2023-12-01T03:10:38.504261+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:38.199716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491490+0000","last_change":"2023-12-01T03:10:25.181678+0000","last_active":"2023-12-01T03:10:37.491490+0000","last_peered":"2023-12-01T03:10:37.491490+0000","last_clean":"2023-12-01T03:10:37.491490+0000","last_became_active":"2023-12-01T03:10:25.181430+0000","last_became_peered":"2023-12-01T03:10:25.181430+0000","last_unstale":"2023-12-01T03:10:37.491490+0000","last_undegraded":"2023-12-01T03:10:37.491490+0000","last_fullsized":"2023-12-01T03:10:37.491490+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.506833+0000","last_change":"2023-12-01T03:10:38.506833+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.506833+0000","last_undegraded":"2023-12-01T03:10:38.506833+0000","last_fullsized":"2023-12-01T03:10:38.506833+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:57:07.895399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480143+0000","last_change":"2023-12-01T03:10:25.188998+0000","last_active":"2023-12-01T03:10:37.480143+0000","last_peered":"2023-12-01T03:10:37.480143+0000","last_clean":"2023-12-01T03:10:37.480143+0000","last_became_active":"2023-12-01T03:10:25.188657+0000","last_became_peered":"2023-12-01T03:10:25.188657+0000","last_unstale":"2023-12-01T03:10:37.480143+0000","last_undegraded":"2023-12-01T03:10:37.480143+0000","last_fullsized":"2023-12-01T03:10:37.480143+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:23:58.752943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.509989+0000","last_change":"2023-12-01T03:10:38.509989+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.509989+0000","last_undegraded":"2023-12-01T03:10:38.509989+0000","last_fullsized":"2023-12-01T03:10:38.509989+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:25:51.393448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485796+0000","last_change":"2023-12-01T03:10:25.174568+0000","last_active":"2023-12-01T03:10:37.485796+0000","last_peered":"2023-12-01T03:10:37.485796+0000","last_clean":"2023-12-01T03:10:37.485796+0000","last_became_active":"2023-12-01T03:10:25.174229+0000","last_became_peered":"2023-12-01T03:10:25.174229+0000","last_unstale":"2023-12-01T03:10:37.485796+0000","last_undegraded":"2023-12-01T03:10:37.485796+0000","last_fullsized":"2023-12-01T03:10:37.485796+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:13.528118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512138+0000","last_change":"2023-12-01T03:10:38.512138+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512138+0000","last_undegraded":"2023-12-01T03:10:38.512138+0000","last_fullsized":"2023-12-01T03:10:38.512138+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481725+0000","last_change":"2023-12-01T03:10:25.189264+0000","last_active":"2023-12-01T03:10:37.481725+0000","last_peered":"2023-12-01T03:10:37.481725+0000","last_clean":"2023-12-01T03:10:37.481725+0000","last_became_active":"2023-12-01T03:10:25.189007+0000","last_became_peered":"2023-12-01T03:10:25.189007+0000","last_unstale":"2023-12-01T03:10:37.481725+0000","last_undegraded":"2023-12-01T03:10:37.481725+0000","last_fullsized":"2023-12-01T03:10:37.481725+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:14.783134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511167+0000","last_change":"2023-12-01T03:10:38.511167+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511167+0000","last_undegraded":"2023-12-01T03:10:38.511167+0000","last_fullsized":"2023-12-01T03:10:38.511167+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:38:07.828693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488698+0000","last_change":"2023-12-01T03:10:25.180992+0000","last_active":"2023-12-01T03:10:37.488698+0000","last_peered":"2023-12-01T03:10:37.488698+0000","last_clean":"2023-12-01T03:10:37.488698+0000","last_became_active":"2023-12-01T03:10:25.180712+0000","last_became_peered":"2023-12-01T03:10:25.180712+0000","last_unstale":"2023-12-01T03:10:37.488698+0000","last_undegraded":"2023-12-01T03:10:37.488698+0000","last_fullsized":"2023-12-01T03:10:37.488698+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:43.827830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507312+0000","last_change":"2023-12-01T03:10:38.507312+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507312+0000","last_undegraded":"2023-12-01T03:10:38.507312+0000","last_fullsized":"2023-12-01T03:10:38.507312+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:58:42.639310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485503+0000","last_change":"2023-12-01T03:10:25.182676+0000","last_active":"2023-12-01T03:10:37.485503+0000","last_peered":"2023-12-01T03:10:37.485503+0000","last_clean":"2023-12-01T03:10:37.485503+0000","last_became_active":"2023-12-01T03:10:25.182429+0000","last_became_peered":"2023-12-01T03:10:25.182429+0000","last_unstale":"2023-12-01T03:10:37.485503+0000","last_undegraded":"2023-12-01T03:10:37.485503+0000","last_fullsized":"2023-12-01T03:10:37.485503+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508726+0000","last_change":"2023-12-01T03:10:38.508726+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508726+0000","last_undegraded":"2023-12-01T03:10:38.508726+0000","last_fullsized":"2023-12-01T03:10:38.508726+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:54.829130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483923+0000","last_change":"2023-12-01T03:10:25.173100+0000","last_active":"2023-12-01T03:10:37.483923+0000","last_peered":"2023-12-01T03:10:37.483923+0000","last_clean":"2023-12-01T03:10:37.483923+0000","last_became_active":"2023-12-01T03:10:25.172808+0000","last_became_peered":"2023-12-01T03:10:25.172808+0000","last_unstale":"2023-12-01T03:10:37.483923+0000","last_undegraded":"2023-12-01T03:10:37.483923+0000","last_fullsized":"2023-12-01T03:10:37.483923+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507042+0000","last_change":"2023-12-01T03:10:38.507042+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507042+0000","last_undegraded":"2023-12-01T03:10:38.507042+0000","last_fullsized":"2023-12-01T03:10:38.507042+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:48.887816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483463+0000","last_change":"2023-12-01T03:10:25.189550+0000","last_active":"2023-12-01T03:10:37.483463+0000","last_peered":"2023-12-01T03:10:37.483463+0000","last_clean":"2023-12-01T03:10:37.483463+0000","last_became_active":"2023-12-01T03:10:25.189001+0000","last_became_peered":"2023-12-01T03:10:25.189001+0000","last_unstale":"2023-12-01T03:10:37.483463+0000","last_undegraded":"2023-12-01T03:10:37.483463+0000","last_fullsized":"2023-12-01T03:10:37.483463+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507724+0000","last_change":"2023-12-01T03:10:38.507724+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507724+0000","last_undegraded":"2023-12-01T03:10:38.507724+0000","last_fullsized":"2023-12-01T03:10:38.507724+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:27:00.020245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472732+0000","last_change":"2023-12-01T03:10:25.170590+0000","last_active":"2023-12-01T03:10:37.472732+0000","last_peered":"2023-12-01T03:10:37.472732+0000","last_clean":"2023-12-01T03:10:37.472732+0000","last_became_active":"2023-12-01T03:10:25.170334+0000","last_became_peered":"2023-12-01T03:10:25.170334+0000","last_unstale":"2023-12-01T03:10:37.472732+0000","last_undegraded":"2023-12-01T03:10:37.472732+0000","last_fullsized":"2023-12-01T03:10:37.472732+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:35.058136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513592+0000","last_change":"2023-12-01T03:10:38.513592+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513592+0000","last_undegraded":"2023-12-01T03:10:38.513592+0000","last_fullsized":"2023-12-01T03:10:38.513592+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481909+0000","last_change":"2023-12-01T03:10:25.173081+0000","last_active":"2023-12-01T03:10:37.481909+0000","last_peered":"2023-12-01T03:10:37.481909+0000","last_clean":"2023-12-01T03:10:37.481909+0000","last_became_active":"2023-12-01T03:10:25.172845+0000","last_became_peered":"2023-12-01T03:10:25.172845+0000","last_unstale":"2023-12-01T03:10:37.481909+0000","last_undegraded":"2023-12-01T03:10:37.481909+0000","last_fullsized":"2023-12-01T03:10:37.481909+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:18.173873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512530+0000","last_change":"2023-12-01T03:10:38.512530+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512530+0000","last_undegraded":"2023-12-01T03:10:38.512530+0000","last_fullsized":"2023-12-01T03:10:38.512530+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:24:41.123796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472369+0000","last_change":"2023-12-01T03:10:25.178225+0000","last_active":"2023-12-01T03:10:37.472369+0000","last_peered":"2023-12-01T03:10:37.472369+0000","last_clean":"2023-12-01T03:10:37.472369+0000","last_became_active":"2023-12-01T03:10:25.177541+0000","last_became_peered":"2023-12-01T03:10:25.177541+0000","last_unstale":"2023-12-01T03:10:37.472369+0000","last_undegraded":"2023-12-01T03:10:37.472369+0000","last_fullsized":"2023-12-01T03:10:37.472369+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:18.470068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.516591+0000","last_change":"2023-12-01T03:10:38.516591+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.516591+0000","last_undegraded":"2023-12-01T03:10:38.516591+0000","last_fullsized":"2023-12-01T03:10:38.516591+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.484276+0000","last_change":"2023-12-01T03:10:25.172316+0000","last_active":"2023-12-01T03:10:37.484276+0000","last_peered":"2023-12-01T03:10:37.484276+0000","last_clean":"2023-12-01T03:10:37.484276+0000","last_became_active":"2023-12-01T03:10:25.169042+0000","last_became_peered":"2023-12-01T03:10:25.169042+0000","last_unstale":"2023-12-01T03:10:37.484276+0000","last_undegraded":"2023-12-01T03:10:37.484276+0000","last_fullsized":"2023-12-01T03:10:37.484276+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507019+0000","last_change":"2023-12-01T03:10:38.507019+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507019+0000","last_undegraded":"2023-12-01T03:10:38.507019+0000","last_fullsized":"2023-12-01T03:10:38.507019+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473026+0000","last_change":"2023-12-01T03:10:25.181052+0000","last_active":"2023-12-01T03:10:37.473026+0000","last_peered":"2023-12-01T03:10:37.473026+0000","last_clean":"2023-12-01T03:10:37.473026+0000","last_became_active":"2023-12-01T03:10:25.180810+0000","last_became_peered":"2023-12-01T03:10:25.180810+0000","last_unstale":"2023-12-01T03:10:37.473026+0000","last_undegraded":"2023-12-01T03:10:37.473026+0000","last_fullsized":"2023-12-01T03:10:37.473026+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:35.663128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504365+0000","last_change":"2023-12-01T03:10:38.504365+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504365+0000","last_undegraded":"2023-12-01T03:10:38.504365+0000","last_fullsized":"2023-12-01T03:10:38.504365+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492510+0000","last_change":"2023-12-01T03:10:25.177089+0000","last_active":"2023-12-01T03:10:37.492510+0000","last_peered":"2023-12-01T03:10:37.492510+0000","last_clean":"2023-12-01T03:10:37.492510+0000","last_became_active":"2023-12-01T03:10:25.176742+0000","last_became_peered":"2023-12-01T03:10:25.176742+0000","last_unstale":"2023-12-01T03:10:37.492510+0000","last_undegraded":"2023-12-01T03:10:37.492510+0000","last_fullsized":"2023-12-01T03:10:37.492510+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513196+0000","last_change":"2023-12-01T03:10:38.513196+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513196+0000","last_undegraded":"2023-12-01T03:10:38.513196+0000","last_fullsized":"2023-12-01T03:10:38.513196+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:19.824447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.484180+0000","last_change":"2023-12-01T03:10:25.172560+0000","last_active":"2023-12-01T03:10:37.484180+0000","last_peered":"2023-12-01T03:10:37.484180+0000","last_clean":"2023-12-01T03:10:37.484180+0000","last_became_active":"2023-12-01T03:10:25.172005+0000","last_became_peered":"2023-12-01T03:10:25.172005+0000","last_unstale":"2023-12-01T03:10:37.484180+0000","last_undegraded":"2023-12-01T03:10:37.484180+0000","last_fullsized":"2023-12-01T03:10:37.484180+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:44.541671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.515497+0000","last_change":"2023-12-01T03:10:38.515497+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.515497+0000","last_undegraded":"2023-12-01T03:10:38.515497+0000","last_fullsized":"2023-12-01T03:10:38.515497+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:10:20.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.477932+0000","last_change":"2023-12-01T03:10:25.185623+0000","last_active":"2023-12-01T03:10:37.477932+0000","last_peered":"2023-12-01T03:10:37.477932+0000","last_clean":"2023-12-01T03:10:37.477932+0000","last_became_active":"2023-12-01T03:10:25.185311+0000","last_became_peered":"2023-12-01T03:10:25.185311+0000","last_unstale":"2023-12-01T03:10:37.477932+0000","last_undegraded":"2023-12-01T03:10:37.477932+0000","last_fullsized":"2023-12-01T03:10:37.477932+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508313+0000","last_change":"2023-12-01T03:10:38.508313+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508313+0000","last_undegraded":"2023-12-01T03:10:38.508313+0000","last_fullsized":"2023-12-01T03:10:38.508313+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:38.801763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483215+0000","last_change":"2023-12-01T03:10:25.186300+0000","last_active":"2023-12-01T03:10:37.483215+0000","last_peered":"2023-12-01T03:10:37.483215+0000","last_clean":"2023-12-01T03:10:37.483215+0000","last_became_active":"2023-12-01T03:10:25.185030+0000","last_became_peered":"2023-12-01T03:10:25.185030+0000","last_unstale":"2023-12-01T03:10:37.483215+0000","last_undegraded":"2023-12-01T03:10:37.483215+0000","last_fullsized":"2023-12-01T03:10:37.483215+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.501552+0000","last_change":"2023-12-01T03:10:38.501552+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.501552+0000","last_undegraded":"2023-12-01T03:10:38.501552+0000","last_fullsized":"2023-12-01T03:10:38.501552+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482005+0000","last_change":"2023-12-01T03:10:25.182648+0000","last_active":"2023-12-01T03:10:37.482005+0000","last_peered":"2023-12-01T03:10:37.482005+0000","last_clean":"2023-12-01T03:10:37.482005+0000","last_became_active":"2023-12-01T03:10:25.182343+0000","last_became_peered":"2023-12-01T03:10:25.182343+0000","last_unstale":"2023-12-01T03:10:37.482005+0000","last_undegraded":"2023-12-01T03:10:37.482005+0000","last_fullsized":"2023-12-01T03:10:37.482005+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:55.408381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.514336+0000","last_change":"2023-12-01T03:10:38.514336+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.514336+0000","last_undegraded":"2023-12-01T03:10:38.514336+0000","last_fullsized":"2023-12-01T03:10:38.514336+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479210+0000","last_change":"2023-12-01T03:10:25.191253+0000","last_active":"2023-12-01T03:10:37.479210+0000","last_peered":"2023-12-01T03:10:37.479210+0000","last_clean":"2023-12-01T03:10:37.479210+0000","last_became_active":"2023-12-01T03:10:25.188485+0000","last_became_peered":"2023-12-01T03:10:25.188485+0000","last_unstale":"2023-12-01T03:10:37.479210+0000","last_undegraded":"2023-12-01T03:10:37.479210+0000","last_fullsized":"2023-12-01T03:10:37.479210+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500633+0000","last_change":"2023-12-01T03:10:38.500633+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500633+0000","last_undegraded":"2023-12-01T03:10:38.500633+0000","last_fullsized":"2023-12-01T03:10:38.500633+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482898+0000","last_change":"2023-12-01T03:10:25.171460+0000","last_active":"2023-12-01T03:10:37.482898+0000","last_peered":"2023-12-01T03:10:37.482898+0000","last_clean":"2023-12-01T03:10:37.482898+0000","last_became_active":"2023-12-01T03:10:25.171148+0000","last_became_peered":"2023-12-01T03:10:25.171148+0000","last_unstale":"2023-12-01T03:10:37.482898+0000","last_undegraded":"2023-12-01T03:10:37.482898+0000","last_fullsized":"2023-12-01T03:10:37.482898+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:01:37.278740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.516041+0000","last_change":"2023-12-01T03:10:38.516041+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.516041+0000","last_undegraded":"2023-12-01T03:10:38.516041+0000","last_fullsized":"2023-12-01T03:10:38.516041+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:07:32.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488617+0000","last_change":"2023-12-01T03:10:25.181720+0000","last_active":"2023-12-01T03:10:37.488617+0000","last_peered":"2023-12-01T03:10:37.488617+0000","last_clean":"2023-12-01T03:10:37.488617+0000","last_became_active":"2023-12-01T03:10:25.180144+0000","last_became_peered":"2023-12-01T03:10:25.180144+0000","last_unstale":"2023-12-01T03:10:37.488617+0000","last_undegraded":"2023-12-01T03:10:37.488617+0000","last_fullsized":"2023-12-01T03:10:37.488617+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:53.999589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507044+0000","last_change":"2023-12-01T03:10:38.507044+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507044+0000","last_undegraded":"2023-12-01T03:10:38.507044+0000","last_fullsized":"2023-12-01T03:10:38.507044+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:44.253867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.484995+0000","last_change":"2023-12-01T03:10:25.181088+0000","last_active":"2023-12-01T03:10:37.484995+0000","last_peered":"2023-12-01T03:10:37.484995+0000","last_clean":"2023-12-01T03:10:37.484995+0000","last_became_active":"2023-12-01T03:10:25.180741+0000","last_became_peered":"2023-12-01T03:10:25.180741+0000","last_unstale":"2023-12-01T03:10:37.484995+0000","last_undegraded":"2023-12-01T03:10:37.484995+0000","last_fullsized":"2023-12-01T03:10:37.484995+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:26.321285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513800+0000","last_change":"2023-12-01T03:10:38.513800+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513800+0000","last_undegraded":"2023-12-01T03:10:38.513800+0000","last_fullsized":"2023-12-01T03:10:38.513800+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472688+0000","last_change":"2023-12-01T03:10:25.171115+0000","last_active":"2023-12-01T03:10:37.472688+0000","last_peered":"2023-12-01T03:10:37.472688+0000","last_clean":"2023-12-01T03:10:37.472688+0000","last_became_active":"2023-12-01T03:10:25.170871+0000","last_became_peered":"2023-12-01T03:10:25.170871+0000","last_unstale":"2023-12-01T03:10:37.472688+0000","last_undegraded":"2023-12-01T03:10:37.472688+0000","last_fullsized":"2023-12-01T03:10:37.472688+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:24.370036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59936,"kb_used_data":2648,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311904,"statfs":{"total":96636764160,"available":96575389696,"internally_reserved":0,"allocated":2711552,"data_stored":1314756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0780000000000001,"15min":1.0820000000000001},"min":{"1min":0.78200000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5069999999999999,"5min":1.6000000000000001,"15min":1.98},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.093,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":0.67800000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.556,"5min":1.7,"15min":1.7},"last":1.331}]},{"osd":1,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.2889999999999999},"min":{"1min":0.80000000000000004,"5min":0.745,"15min":0.73099999999999998},"max":{"1min":1.603,"5min":2,"15min":2.1419999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3149999999999999,"15min":1.3320000000000001},"min":{"1min":1.137,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.514,"5min":1.8169999999999999,"15min":1.8700000000000001},"last":1.0700000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.157,"15min":1.1719999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.476,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.26,"15min":1.254},"min":{"1min":1.0549999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6020000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.268}]},{"osd":3,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2529999999999999,"15min":1.3049999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6559999999999999,"5min":1.712,"15min":2.431},"last":1.014},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.415,"15min":1.4199999999999999},"min":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.92900000000000005},"max":{"1min":1.5920000000000001,"5min":1.879,"15min":2.198},"last":1.109}]},{"osd":4,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3140000000000001,"15min":1.3400000000000001},"min":{"1min":1.006,"5min":0.82599999999999996,"15min":0.81200000000000006},"max":{"1min":1.901,"5min":1.901,"15min":2.5899999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":1.1240000000000001,"5min":0.92800000000000005,"15min":0.79500000000000004},"max":{"1min":1.9690000000000001,"5min":2.3279999999999998,"15min":2.569},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3009999999999999,"15min":1.3149999999999999},"min":{"1min":0.93600000000000005,"5min":0.85999999999999999,"15min":0.84599999999999997},"max":{"1min":2.0099999999999998,"5min":2.016,"15min":2.7269999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5800000000000001,"15min":1.589},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":0.93200000000000005},"max":{"1min":2.0569999999999999,"5min":2.206,"15min":2.484},"last":1.4570000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4359999999999999,"15min":1.446},"min":{"1min":1.0229999999999999,"5min":0.98599999999999999,"15min":0.82599999999999996},"max":{"1min":1.921,"5min":2.2789999999999999,"15min":2.532},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":1.8819999999999999,"15min":2.762},"last":1.2370000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575980,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41872,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329968,"statfs":{"total":96636764160,"available":96593887232,"internally_reserved":0,"allocated":2695168,"data_stored":1311929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0589999999999999,"15min":1.113},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.4039999999999999,"5min":1.6639999999999999,"15min":1.9219999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1160000000000001,"15min":1.1659999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.623,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":0.97699999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.976,"5min":1.976,"15min":2.0449999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1950000000000001,"15min":1.2210000000000001},"min":{"1min":0.627,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.694,"5min":1.694,"15min":1.835},"last":1.2390000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.163,"15min":1.2090000000000001},"min":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.976},"last":1.127},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2629999999999999,"15min":1.2849999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.167}]},{"osd":3,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2849999999999999,"15min":1.319},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8100000000000001,"5min":1.927,"15min":2.0289999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.3720000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2130000000000001,"15min":1.262},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.367,"15min":1.3540000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.522}]},{"osd":5,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3360000000000001,"15min":1.4039999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.272},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4790000000000001,"15min":1.484},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.6319999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4890000000000001,"15min":1.5249999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.452},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.556,"15min":1.579},"min":{"1min":0.95499999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2999999999999998,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.794}]}]},{"osd":5,"up_from":13,"seq":55834575979,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48216,"kb_used_data":3216,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323624,"statfs":{"total":96636764160,"available":96587390976,"internally_reserved":0,"allocated":3293184,"data_stored":1902811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.125},"min":{"1min":0.82299999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.548,"5min":1.776,"15min":1.8899999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.238,"5min":1.1639999999999999,"15min":1.163},"min":{"1min":0.89800000000000002,"5min":0.69399999999999995,"15min":0.67300000000000004},"max":{"1min":1.619,"5min":1.752,"15min":1.8009999999999999},"last":0.84999999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.133,"15min":1.1539999999999999},"min":{"1min":0.94699999999999995,"5min":0.81200000000000006,"15min":0.79900000000000004},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.696},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.272,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.88,"5min":0.79500000000000004,"15min":0.60899999999999999},"max":{"1min":1.619,"5min":1.6659999999999999,"15min":1.7649999999999999},"last":1.0289999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2390000000000001,"15min":1.246},"min":{"1min":0.93700000000000006,"5min":0.84499999999999997,"15min":0.81399999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.774},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.284,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.93899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.502,"5min":1.899,"15min":1.9219999999999999},"last":0.90200000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.306,"15min":1.3200000000000001},"min":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.93100000000000005},"max":{"1min":1.7929999999999999,"5min":1.8979999999999999,"15min":1.994},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.343,"15min":1.363},"min":{"1min":1.02,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.724,"5min":1.97,"15min":2.1859999999999999},"last":1.1080000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2789999999999999,"15min":1.254},"min":{"1min":0.96199999999999997,"5min":0.78000000000000003,"15min":0.74099999999999999},"max":{"1min":1.726,"5min":1.726,"15min":2.6040000000000001},"last":1.389},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.399,"15min":1.385},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.84599999999999997},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.931},"last":1.1579999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3879999999999999,"15min":1.3759999999999999},"min":{"1min":0.97599999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.9279999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.427,"15min":1.419},"min":{"1min":1.222,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":1.756,"5min":1.9219999999999999,"15min":2.577},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5349999999999999,"15min":1.5369999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.962,"5min":2.0790000000000002,"15min":3.1709999999999998},"last":1.294},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.575,"15min":1.5740000000000001},"min":{"1min":1.3720000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.831,"5min":2.101,"15min":2.8740000000000001},"last":1.343}]}]},{"osd":4,"up_from":13,"seq":55834575979,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2707456,"data_stored":1319053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.147,"15min":1.1339999999999999},"min":{"1min":0.73099999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.21,"15min":1.1990000000000001},"min":{"1min":0.82099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5840000000000001,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.006}]},{"osd":1,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.242,"15min":1.2589999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.613,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.228},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.82599999999999996},"max":{"1min":1.6080000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.5549999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.3240000000000001,"15min":1.2470000000000001},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.573,"5min":11.311,"15min":11.311},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.329,"15min":1.341},"min":{"1min":1.0680000000000001,"5min":1.024,"15min":0.94099999999999995},"max":{"1min":1.647,"5min":1.9379999999999999,"15min":1.9450000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3100000000000001,"15min":1.29},"min":{"1min":1.0529999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7749999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.365,"15min":1.3380000000000001},"min":{"1min":0.99199999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.599,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.5920000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.357,"15min":1.329},"min":{"1min":0.94799999999999995,"5min":0.91200000000000003,"15min":0.755},"max":{"1min":1.615,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.379,"15min":1.3120000000000001},"min":{"1min":0.94599999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7150000000000001,"5min":3.71,"15min":3.71},"last":1.2809999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3740000000000001,"15min":1.375},"min":{"1min":1.095,"5min":0.745,"15min":0.745},"max":{"1min":1.609,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.4850000000000001,"15min":1.48},"min":{"1min":1.335,"5min":1.0660000000000001,"15min":1.0149999999999999},"max":{"1min":1.738,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.52,"15min":1.486},"min":{"1min":1.157,"5min":0.95499999999999996,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6000000000000001,"15min":1.5800000000000001},"min":{"1min":1.3600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.806,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.7070000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575978,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51752,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320088,"statfs":{"total":96636764160,"available":96583770112,"internally_reserved":0,"allocated":2719744,"data_stored":1321894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91100000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.085},"min":{"1min":0.74199999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.571,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.163,"15min":1.171},"min":{"1min":0.64700000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.008,"15min":0.999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41099999999999998},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0900000000000001,"15min":1.103},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2230000000000001,"15min":1.252},"min":{"1min":0.80100000000000005,"5min":0.747,"15min":0.58599999999999997},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.57899999999999996}]},{"osd":4,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2989999999999999,"15min":1.2929999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.3},"min":{"1min":0.96799999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0229999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3240000000000001,"15min":1.331},"min":{"1min":0.86599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.373,"15min":1.3759999999999999},"min":{"1min":0.88400000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.78400000000000003},"max":{"1min":2.5649999999999999,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.54,"5min":1.5049999999999999,"15min":1.514},"min":{"1min":1.075,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.075}]},{"osd":7,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.516,"15min":1.5329999999999999},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.042},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.54,"15min":1.5309999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.878},"max":{"1min":2.4830000000000001,"5min":2.984,"15min":2.984},"last":0.83899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575978,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64028,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307812,"statfs":{"total":96636764160,"available":96571199488,"internally_reserved":0,"allocated":2707456,"data_stored":1319089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89300000000000002,"15min":0.89400000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1140000000000001,"5min":1.405,"15min":1.4730000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.056,"15min":1.0569999999999999},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5640000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0449999999999999,"15min":1.0589999999999999},"min":{"1min":0.66100000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.321,"5min":1.698,"15min":1.698},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.038,"5min":1.125,"15min":1.151},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.522,"5min":1.635,"15min":1.71},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0640000000000001,"15min":1.054},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.419,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1990000000000001,"15min":1.2170000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.516,"5min":1.6879999999999999,"15min":1.79},"last":1.01}]},{"osd":4,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":0.85199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.7569999999999999,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.133},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.714,"5min":1.714,"15min":1.835},"last":1.2390000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3919999999999999,"15min":1.391},"min":{"1min":0.88,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.8380000000000001,"15min":2.2040000000000002},"last":1.042},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.419,"15min":1.4179999999999999},"min":{"1min":1.095,"5min":1.004,"15min":1.004},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.016},"last":0.91800000000000004}]},{"osd":6,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.379,"15min":1.3819999999999999},"min":{"1min":0.89600000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.337},{"interface":"front","average":{"1min":1.46,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":1.087,"5min":0.879,"15min":0.879},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.327}]},{"osd":7,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5069999999999999,"15min":1.4910000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0979999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4330000000000001,"15min":1.458},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.012,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.0860000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575977,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48208,"kb_used_data":3208,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323632,"statfs":{"total":96636764160,"available":96587399168,"internally_reserved":0,"allocated":3284992,"data_stored":1902207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.879,"15min":0.90900000000000003},"min":{"1min":0.58199999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.163},{"interface":"front","average":{"1min":1.099,"5min":0.99299999999999999,"15min":1.046},"min":{"1min":0.81000000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.5069999999999999,"5min":1.5660000000000001,"15min":1.681},"last":1.04}]},{"osd":2,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.56299999999999994,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.115,"15min":1.1419999999999999},"min":{"1min":0.75600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.581,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":1.244}]},{"osd":3,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.258,"15min":1.3069999999999999},"min":{"1min":1.026,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5780000000000001,"5min":1.587,"15min":1.788},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.107,"15min":1.135},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.714,"5min":1.714,"15min":1.8049999999999999},"last":1.302}]},{"osd":4,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.21,"15min":1.2230000000000001},"min":{"1min":0.89900000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6859999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.389,"15min":1.409},"min":{"1min":1.0740000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.47}]},{"osd":5,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.85899999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.802,"5min":1.8520000000000001,"15min":2.101},"last":1.9399999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.304,"15min":1.339},"min":{"1min":1.006,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.609,"5min":1.927,"15min":2.1659999999999999},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4019999999999999,"15min":1.4379999999999999},"min":{"1min":0.99399999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":2.016},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3799999999999999,"15min":1.4119999999999999},"min":{"1min":1.1779999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.768,"5min":2.0019999999999998,"15min":2.0209999999999999},"last":1.5369999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4870000000000001,"15min":1.5269999999999999},"min":{"1min":1.2549999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.911,"5min":2.0329999999999999,"15min":2.1190000000000002},"last":1.8939999999999999},{"interface":"front","average":{"1min":1.524,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":2.0699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575979,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59924,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311916,"statfs":{"total":96636764160,"available":96575401984,"internally_reserved":0,"allocated":2699264,"data_stored":1312455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.86699999999999999,"15min":0.89400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1559999999999999,"5min":1.4379999999999999,"15min":1.994},"last":0.625},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.04,"15min":1.056},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.427,"5min":1.583,"15min":1.6799999999999999},"last":0.76800000000000002}]},{"osd":2,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0329999999999999,"15min":1.069},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.446,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.1020000000000001,"15min":1.1419999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.726,"5min":1.726,"15min":1.792},"last":0.88}]},{"osd":3,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0580000000000001,"15min":1.071},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.8839999999999999},"last":1.22},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.655,"5min":1.655,"15min":2.6469999999999998},"last":1.351}]},{"osd":4,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3260000000000001,"15min":1.363},"min":{"1min":0.88400000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.284,"15min":1.3089999999999999},"min":{"1min":0.82099999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.677,"5min":1.74,"15min":2.3610000000000002},"last":1.5029999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.355,"15min":1.397},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5589999999999999,"5min":2.1030000000000002,"15min":2.7719999999999998},"last":1.554},{"interface":"front","average":{"1min":1.431,"5min":1.4099999999999999,"15min":1.4410000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.79600000000000004},"max":{"1min":1.9259999999999999,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":1.1579999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.367,"15min":1.3859999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7649999999999999,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.458,"15min":1.484},"min":{"1min":1.048,"5min":0.85299999999999998,"15min":0.75800000000000001},"max":{"1min":1.736,"5min":1.9870000000000001,"15min":2.8759999999999999},"last":1.4630000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.49,"15min":1.524},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.841,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4750000000000001,"15min":1.4910000000000001},"min":{"1min":0.97299999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.0619999999999998,"5min":2.1190000000000002,"15min":2.431},"last":1.401}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:10:43.400 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:10:43.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.643+0000 7f4bd1969640 1 Processor -- start 2023-12-01T03:10:43.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.643+0000 7f4bd1969640 1 -- start start 2023-12-01T03:10:43.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.645+0000 7f4bd1969640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4bcc0c9ef0 0x7f4bcc06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.645+0000 7f4bd1969640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc06f8b0 con 0x7f4bcc06b6f0 2023-12-01T03:10:43.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.645+0000 7f4bd1969640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc0adcb0 con 0x7f4bcc159dd0 2023-12-01T03:10:43.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.645+0000 7f4bd1969640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bcc0ad9d0 con 0x7f4bcc0c9ef0 2023-12-01T03:10:43.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.645+0000 7f4bcb7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4bcc06b6f0 0x7f4bcc070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37066/0 (socket says 172.21.15.115:37066) 2023-12-01T03:10:43.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.645+0000 7f4bcb7fe640 1 -- 172.21.15.115:0/3820073324 learned_addr learned my addr 172.21.15.115:0/3820073324 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:43.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bca7fc640 1 --2- 172.21.15.115:0/3820073324 >> v2:172.21.15.142:3300/0 conn(0x7f4bcc0c9ef0 0x7f4bcc06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274765040 0 0) 0x7f4bcc06f8b0 con 0x7f4bcc06b6f0 2023-12-01T03:10:43.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bb4003520 con 0x7f4bcc06b6f0 2023-12-01T03:10:43.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 229505700 0 0) 0x7f4bb4003520 con 0x7f4bcc06b6f0 2023-12-01T03:10:43.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 >> v2:172.21.15.142:3300/0 conn(0x7f4bcc0c9ef0 msgr2=0x7f4bcc06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:43.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 --2- 172.21.15.115:0/3820073324 >> v2:172.21.15.142:3300/0 conn(0x7f4bcc0c9ef0 0x7f4bcc06fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 >> v1:172.21.15.142:6789/0 conn(0x7f4bcc159dd0 legacy=0x7f4bcc0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc0c9000 con 0x7f4bcc06b6f0 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4bc40019f0 con 0x7f4bcc06b6f0 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 >> v1:172.21.15.115:6789/0 conn(0x7f4bcc06b6f0 legacy=0x7f4bcc070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 --2- 172.21.15.115:0/3820073324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bb40039e0 0x7f4bb4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 --2- 172.21.15.115:0/3820073324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb40047c0 0x7f4bb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 --2- 172.21.15.115:0/3820073324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb4005280 0x7f4bb400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc0ad9d0 con 0x7f4bb40039e0 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb400c1c0 con 0x7f4bb4005280 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.646+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb400e590 con 0x7f4bb40047c0 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.647+0000 7f4bcaffd640 1 --2- 172.21.15.115:0/3820073324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bb40039e0 0x7f4bb4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.647+0000 7f4bca7fc640 1 --2- 172.21.15.115:0/3820073324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb40047c0 0x7f4bb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.647+0000 7f4bcaffd640 1 -- 172.21.15.115:0/3820073324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb40047c0 msgr2=0x7f4bb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.647+0000 7f4bcaffd640 1 --2- 172.21.15.115:0/3820073324 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bb40047c0 0x7f4bb4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.647+0000 7f4bcaffd640 1 -- 172.21.15.115:0/3820073324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb4005280 msgr2=0x7f4bb400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.647+0000 7f4bcaffd640 1 --2- 172.21.15.115:0/3820073324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb4005280 0x7f4bb400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.647+0000 7f4bcaffd640 1 -- 172.21.15.115:0/3820073324 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb4003520 con 0x7f4bb40039e0 2023-12-01T03:10:43.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.647+0000 7f4bcb7fe640 1 -- 172.21.15.115:0/3820073324 reap_dead start 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.648+0000 7f4bcaffd640 1 --2- 172.21.15.115:0/3820073324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bb40039e0 0x7f4bb4003e00 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc001750 tx=0x7f4bb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99f487eea27616d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.648+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0052d0 con 0x7f4bb40039e0 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.648+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc02e050 con 0x7f4bb40039e0 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.648+0000 7f4bc9ffb640 1 -- 172.21.15.115:0/3820073324 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc003830 con 0x7f4bb40039e0 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.649+0000 7f4bd1969640 1 -- 172.21.15.115:0/3820073324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bb40039e0 msgr2=0x7f4bb4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.649+0000 7f4bd1969640 1 --2- 172.21.15.115:0/3820073324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bb40039e0 0x7f4bb4003e00 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc001750 tx=0x7f4bb40057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.649+0000 7f4bd1969640 1 -- 172.21.15.115:0/3820073324 shutdown_connections 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.649+0000 7f4bd1969640 1 --2- 172.21.15.115:0/3820073324 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bb40039e0 0x7f4bb4003e00 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.649+0000 7f4bd1969640 1 -- 172.21.15.115:0/3820073324 >> 172.21.15.115:0/3820073324 conn(0x7f4bcc0ce880 msgr2=0x7f4bcc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.649+0000 7f4bd1969640 1 -- 172.21.15.115:0/3820073324 shutdown_connections 2023-12-01T03:10:43.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.649+0000 7f4bd1969640 1 -- 172.21.15.115:0/3820073324 wait complete. 2023-12-01T03:10:43.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.651+0000 7f4bd1969640 1 Processor -- start 2023-12-01T03:10:43.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.651+0000 7f4bd1969640 1 -- start start 2023-12-01T03:10:43.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.652+0000 7f4bd1969640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb40039e0 0x7f4bcc1f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.652+0000 7f4bd1969640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bcc15ac50 0x7f4bcc0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.652+0000 7f4bd1969640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bcc0b3230 0x7f4bcc156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.652+0000 7f4bd1969640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc0b36b0 con 0x7f4bcc15ac50 2023-12-01T03:10:43.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.652+0000 7f4bd1969640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc1573a0 con 0x7f4bb40039e0 2023-12-01T03:10:43.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.652+0000 7f4bd1969640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc1570c0 con 0x7f4bcc0b3230 2023-12-01T03:10:43.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.652+0000 7f4bca7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bcc15ac50 0x7f4bcc0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.652+0000 7f4bca7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bcc15ac50 0x7f4bcc0b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55486/0 (socket says 172.21.15.115:55486) 2023-12-01T03:10:43.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.652+0000 7f4bca7fc640 1 -- 172.21.15.115:0/3125859434 learned_addr learned my addr 172.21.15.115:0/3125859434 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:43.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bca7fc640 1 -- 172.21.15.115:0/3125859434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bcc0b3230 msgr2=0x7f4bcc156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bcb7fe640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bcc0b3230 0x7f4bcc156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bcaffd640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb40039e0 0x7f4bcc1f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bca7fc640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bcc0b3230 0x7f4bcc156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bca7fc640 1 -- 172.21.15.115:0/3125859434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb40039e0 msgr2=0x7f4bcc1f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bca7fc640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb40039e0 0x7f4bcc1f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bca7fc640 1 -- 172.21.15.115:0/3125859434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc1e48f0 con 0x7f4bcc15ac50 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bca7fc640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bcc15ac50 0x7f4bcc0b2cf0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0010790 tx=0x7f4bc0010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bb3fff640 1 -- 172.21.15.115:0/3125859434 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0009630 con 0x7f4bcc15ac50 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bb3fff640 1 -- 172.21.15.115:0/3125859434 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc000eb60 con 0x7f4bcc15ac50 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bb3fff640 1 -- 172.21.15.115:0/3125859434 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0010ea0 con 0x7f4bcc15ac50 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bd1969640 1 -- 172.21.15.115:0/3125859434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc1e4bd0 con 0x7f4bcc15ac50 2023-12-01T03:10:43.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.653+0000 7f4bd1969640 1 -- 172.21.15.115:0/3125859434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc1f28f0 con 0x7f4bcc15ac50 2023-12-01T03:10:43.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.654+0000 7f4bd1969640 1 -- 172.21.15.115:0/3125859434 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc04e890 con 0x7f4bcc15ac50 2023-12-01T03:10:43.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.655+0000 7f4bb3fff640 1 -- 172.21.15.115:0/3125859434 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f4bc001c400 con 0x7f4bcc15ac50 2023-12-01T03:10:43.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.656+0000 7f4bb3fff640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bb806a580 0x7f4bb806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:43.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.656+0000 7f4bb3fff640 1 -- 172.21.15.115:0/3125859434 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f4bc0089270 con 0x7f4bcc15ac50 2023-12-01T03:10:43.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.656+0000 7f4bcaffd640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bb806a580 0x7f4bb806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:43.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.657+0000 7f4bcaffd640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bb806a580 0x7f4bb806ca40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc033000 tx=0x7f4bbc009030 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:43.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.660+0000 7f4bb3fff640 1 -- 172.21.15.115:0/3125859434 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4bc00543d0 con 0x7f4bcc15ac50 2023-12-01T03:10:43.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.849+0000 7f4bd1969640 1 -- 172.21.15.115:0/3125859434 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4bcc06d0d0 con 0x7f4bb806a580 2023-12-01T03:10:43.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.886+0000 7f4bb3fff640 1 -- 172.21.15.115:0/3125859434 <== mgr.10407 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+524098 (secure 0 0 0) 0x7f4bbc005380 con 0x7f4bb806a580 2023-12-01T03:10:43.894 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:43.898 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.893+0000 7f4bd1969640 1 -- 172.21.15.115:0/3125859434 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bb806a580 msgr2=0x7f4bb806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bb806a580 0x7f4bb806ca40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc033000 tx=0x7f4bbc009030 comp rx=0 tx=0).stop 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 -- 172.21.15.115:0/3125859434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bcc15ac50 msgr2=0x7f4bcc0b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bcc15ac50 0x7f4bcc0b2cf0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0010790 tx=0x7f4bc0010c50 comp rx=0 tx=0).stop 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 -- 172.21.15.115:0/3125859434 shutdown_connections 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bb806a580 0x7f4bb806ca40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4bcc0b3230 0x7f4bcc156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bcc15ac50 0x7f4bcc0b2cf0 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 --2- 172.21.15.115:0/3125859434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb40039e0 0x7f4bcc1f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 -- 172.21.15.115:0/3125859434 >> 172.21.15.115:0/3125859434 conn(0x7f4bcc0ce880 msgr2=0x7f4bcc158e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 -- 172.21.15.115:0/3125859434 shutdown_connections 2023-12-01T03:10:43.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:43.894+0000 7f4bd1969640 1 -- 172.21.15.115:0/3125859434 wait complete. 2023-12-01T03:10:43.914 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-12-01T03:10:43.800176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594553,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":593,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":373,"ondisk_log_size":373,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":250,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425752,"kb_used_data":22360,"kb_used_omap":0,"kb_used_meta":403328,"kb_avail":754548968,"statfs":{"total":773094113280,"available":772658143232,"internally_reserved":0,"allocated":22896640,"data_stored":11748936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":413007872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1569,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001845"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507374+0000","last_change":"2023-12-01T03:10:38.507374+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507374+0000","last_undegraded":"2023-12-01T03:10:38.507374+0000","last_fullsized":"2023-12-01T03:10:38.507374+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:14.969594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"201'326","reported_seq":1123,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983105+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:10:38.983105+0000","last_peered":"2023-12-01T03:10:38.983105+0000","last_clean":"2023-12-01T03:10:38.983105+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:10:38.983105+0000","last_undegraded":"2023-12-01T03:10:38.983105+0000","last_fullsized":"2023-12-01T03:10:38.983105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988127+0000","last_change":"2023-12-01T03:10:24.685430+0000","last_active":"2023-12-01T03:10:38.988127+0000","last_peered":"2023-12-01T03:10:38.988127+0000","last_clean":"2023-12-01T03:10:38.988127+0000","last_became_active":"2023-12-01T03:10:24.684919+0000","last_became_peered":"2023-12-01T03:10:24.684919+0000","last_unstale":"2023-12-01T03:10:38.988127+0000","last_undegraded":"2023-12-01T03:10:38.988127+0000","last_fullsized":"2023-12-01T03:10:38.988127+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481058+0000","last_change":"2023-12-01T03:10:25.189213+0000","last_active":"2023-12-01T03:10:37.481058+0000","last_peered":"2023-12-01T03:10:37.481058+0000","last_clean":"2023-12-01T03:10:37.481058+0000","last_became_active":"2023-12-01T03:10:25.188927+0000","last_became_peered":"2023-12-01T03:10:25.188927+0000","last_unstale":"2023-12-01T03:10:37.481058+0000","last_undegraded":"2023-12-01T03:10:37.481058+0000","last_fullsized":"2023-12-01T03:10:37.481058+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:21:53.633800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520651+0000","last_change":"2023-12-01T03:10:38.517472+0000","last_active":"2023-12-01T03:10:38.520651+0000","last_peered":"2023-12-01T03:10:38.520651+0000","last_clean":"2023-12-01T03:10:38.520651+0000","last_became_active":"2023-12-01T03:10:38.517218+0000","last_became_peered":"2023-12-01T03:10:38.517218+0000","last_unstale":"2023-12-01T03:10:38.520651+0000","last_undegraded":"2023-12-01T03:10:38.520651+0000","last_fullsized":"2023-12-01T03:10:38.520651+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:57.653086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499505+0000","last_change":"2023-12-01T03:10:38.499505+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499505+0000","last_undegraded":"2023-12-01T03:10:38.499505+0000","last_fullsized":"2023-12-01T03:10:38.499505+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481362+0000","last_change":"2023-12-01T03:10:24.690155+0000","last_active":"2023-12-01T03:10:37.481362+0000","last_peered":"2023-12-01T03:10:37.481362+0000","last_clean":"2023-12-01T03:10:37.481362+0000","last_became_active":"2023-12-01T03:10:24.689122+0000","last_became_peered":"2023-12-01T03:10:24.689122+0000","last_unstale":"2023-12-01T03:10:37.481362+0000","last_undegraded":"2023-12-01T03:10:37.481362+0000","last_fullsized":"2023-12-01T03:10:37.481362+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479122+0000","last_change":"2023-12-01T03:10:25.191828+0000","last_active":"2023-12-01T03:10:37.479122+0000","last_peered":"2023-12-01T03:10:37.479122+0000","last_clean":"2023-12-01T03:10:37.479122+0000","last_became_active":"2023-12-01T03:10:25.191436+0000","last_became_peered":"2023-12-01T03:10:25.191436+0000","last_unstale":"2023-12-01T03:10:37.479122+0000","last_undegraded":"2023-12-01T03:10:37.479122+0000","last_fullsized":"2023-12-01T03:10:37.479122+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525987+0000","last_change":"2023-12-01T03:10:38.524843+0000","last_active":"2023-12-01T03:10:38.525987+0000","last_peered":"2023-12-01T03:10:38.525987+0000","last_clean":"2023-12-01T03:10:38.525987+0000","last_became_active":"2023-12-01T03:10:38.524424+0000","last_became_peered":"2023-12-01T03:10:38.524424+0000","last_unstale":"2023-12-01T03:10:38.525987+0000","last_undegraded":"2023-12-01T03:10:38.525987+0000","last_fullsized":"2023-12-01T03:10:38.525987+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:32.967785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.509995+0000","last_change":"2023-12-01T03:10:38.509995+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.509995+0000","last_undegraded":"2023-12-01T03:10:38.509995+0000","last_fullsized":"2023-12-01T03:10:38.509995+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983588+0000","last_change":"2023-12-01T03:10:24.684248+0000","last_active":"2023-12-01T03:10:38.983588+0000","last_peered":"2023-12-01T03:10:38.983588+0000","last_clean":"2023-12-01T03:10:38.983588+0000","last_became_active":"2023-12-01T03:10:24.680714+0000","last_became_peered":"2023-12-01T03:10:24.680714+0000","last_unstale":"2023-12-01T03:10:38.983588+0000","last_undegraded":"2023-12-01T03:10:38.983588+0000","last_fullsized":"2023-12-01T03:10:38.983588+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:19:02.474558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.982169+0000","last_change":"2023-12-01T03:10:25.174085+0000","last_active":"2023-12-01T03:10:38.982169+0000","last_peered":"2023-12-01T03:10:38.982169+0000","last_clean":"2023-12-01T03:10:38.982169+0000","last_became_active":"2023-12-01T03:10:25.173802+0000","last_became_peered":"2023-12-01T03:10:25.173802+0000","last_unstale":"2023-12-01T03:10:38.982169+0000","last_undegraded":"2023-12-01T03:10:38.982169+0000","last_fullsized":"2023-12-01T03:10:38.982169+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:39.145914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.982926+0000","last_change":"2023-12-01T03:10:38.512523+0000","last_active":"2023-12-01T03:10:38.982926+0000","last_peered":"2023-12-01T03:10:38.982926+0000","last_clean":"2023-12-01T03:10:38.982926+0000","last_became_active":"2023-12-01T03:10:38.512225+0000","last_became_peered":"2023-12-01T03:10:38.512225+0000","last_unstale":"2023-12-01T03:10:38.982926+0000","last_undegraded":"2023-12-01T03:10:38.982926+0000","last_fullsized":"2023-12-01T03:10:38.982926+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:05.819515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003662+0000","last_change":"2023-12-01T03:10:39.001307+0000","last_active":"2023-12-01T03:10:39.003662+0000","last_peered":"2023-12-01T03:10:39.003662+0000","last_clean":"2023-12-01T03:10:39.003662+0000","last_became_active":"2023-12-01T03:10:39.000379+0000","last_became_peered":"2023-12-01T03:10:39.000379+0000","last_unstale":"2023-12-01T03:10:39.003662+0000","last_undegraded":"2023-12-01T03:10:39.003662+0000","last_fullsized":"2023-12-01T03:10:39.003662+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:39.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986591+0000","last_change":"2023-12-01T03:10:24.683240+0000","last_active":"2023-12-01T03:10:38.986591+0000","last_peered":"2023-12-01T03:10:38.986591+0000","last_clean":"2023-12-01T03:10:38.986591+0000","last_became_active":"2023-12-01T03:10:24.682933+0000","last_became_peered":"2023-12-01T03:10:24.682933+0000","last_unstale":"2023-12-01T03:10:38.986591+0000","last_undegraded":"2023-12-01T03:10:38.986591+0000","last_fullsized":"2023-12-01T03:10:38.986591+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:40.211424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472633+0000","last_change":"2023-12-01T03:10:25.181396+0000","last_active":"2023-12-01T03:10:37.472633+0000","last_peered":"2023-12-01T03:10:37.472633+0000","last_clean":"2023-12-01T03:10:37.472633+0000","last_became_active":"2023-12-01T03:10:25.181134+0000","last_became_peered":"2023-12-01T03:10:25.181134+0000","last_unstale":"2023-12-01T03:10:37.472633+0000","last_undegraded":"2023-12-01T03:10:37.472633+0000","last_fullsized":"2023-12-01T03:10:37.472633+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:04.776013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.981977+0000","last_change":"2023-12-01T03:10:38.522641+0000","last_active":"2023-12-01T03:10:38.981977+0000","last_peered":"2023-12-01T03:10:38.981977+0000","last_clean":"2023-12-01T03:10:38.981977+0000","last_became_active":"2023-12-01T03:10:38.522225+0000","last_became_peered":"2023-12-01T03:10:38.522225+0000","last_unstale":"2023-12-01T03:10:38.981977+0000","last_undegraded":"2023-12-01T03:10:38.981977+0000","last_fullsized":"2023-12-01T03:10:38.981977+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:13.766818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499285+0000","last_change":"2023-12-01T03:10:38.499285+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499285+0000","last_undegraded":"2023-12-01T03:10:38.499285+0000","last_fullsized":"2023-12-01T03:10:38.499285+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985936+0000","last_change":"2023-12-01T03:10:24.687326+0000","last_active":"2023-12-01T03:10:38.985936+0000","last_peered":"2023-12-01T03:10:38.985936+0000","last_clean":"2023-12-01T03:10:38.985936+0000","last_became_active":"2023-12-01T03:10:24.687036+0000","last_became_peered":"2023-12-01T03:10:24.687036+0000","last_unstale":"2023-12-01T03:10:38.985936+0000","last_undegraded":"2023-12-01T03:10:38.985936+0000","last_fullsized":"2023-12-01T03:10:38.985936+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:36:57.969605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986462+0000","last_change":"2023-12-01T03:10:25.190210+0000","last_active":"2023-12-01T03:10:38.986462+0000","last_peered":"2023-12-01T03:10:38.986462+0000","last_clean":"2023-12-01T03:10:38.986462+0000","last_became_active":"2023-12-01T03:10:25.189957+0000","last_became_peered":"2023-12-01T03:10:25.189957+0000","last_unstale":"2023-12-01T03:10:38.986462+0000","last_undegraded":"2023-12-01T03:10:38.986462+0000","last_fullsized":"2023-12-01T03:10:38.986462+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:01.783625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988039+0000","last_change":"2023-12-01T03:10:38.514437+0000","last_active":"2023-12-01T03:10:38.988039+0000","last_peered":"2023-12-01T03:10:38.988039+0000","last_clean":"2023-12-01T03:10:38.988039+0000","last_became_active":"2023-12-01T03:10:38.514155+0000","last_became_peered":"2023-12-01T03:10:38.514155+0000","last_unstale":"2023-12-01T03:10:38.988039+0000","last_undegraded":"2023-12-01T03:10:38.988039+0000","last_fullsized":"2023-12-01T03:10:38.988039+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:06.817444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.503490+0000","last_change":"2023-12-01T03:10:38.503490+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.503490+0000","last_undegraded":"2023-12-01T03:10:38.503490+0000","last_fullsized":"2023-12-01T03:10:38.503490+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481503+0000","last_change":"2023-12-01T03:10:24.697595+0000","last_active":"2023-12-01T03:10:37.481503+0000","last_peered":"2023-12-01T03:10:37.481503+0000","last_clean":"2023-12-01T03:10:37.481503+0000","last_became_active":"2023-12-01T03:10:24.697311+0000","last_became_peered":"2023-12-01T03:10:24.697311+0000","last_unstale":"2023-12-01T03:10:37.481503+0000","last_undegraded":"2023-12-01T03:10:37.481503+0000","last_fullsized":"2023-12-01T03:10:37.481503+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:45.743456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483634+0000","last_change":"2023-12-01T03:10:25.192028+0000","last_active":"2023-12-01T03:10:37.483634+0000","last_peered":"2023-12-01T03:10:37.483634+0000","last_clean":"2023-12-01T03:10:37.483634+0000","last_became_active":"2023-12-01T03:10:25.191747+0000","last_became_peered":"2023-12-01T03:10:25.191747+0000","last_unstale":"2023-12-01T03:10:37.483634+0000","last_undegraded":"2023-12-01T03:10:37.483634+0000","last_fullsized":"2023-12-01T03:10:37.483634+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520576+0000","last_change":"2023-12-01T03:10:38.517012+0000","last_active":"2023-12-01T03:10:38.520576+0000","last_peered":"2023-12-01T03:10:38.520576+0000","last_clean":"2023-12-01T03:10:38.520576+0000","last_became_active":"2023-12-01T03:10:38.516749+0000","last_became_peered":"2023-12-01T03:10:38.516749+0000","last_unstale":"2023-12-01T03:10:38.520576+0000","last_undegraded":"2023-12-01T03:10:38.520576+0000","last_fullsized":"2023-12-01T03:10:38.520576+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:29.962162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008414+0000","last_change":"2023-12-01T03:10:39.005634+0000","last_active":"2023-12-01T03:10:39.008414+0000","last_peered":"2023-12-01T03:10:39.008414+0000","last_clean":"2023-12-01T03:10:39.008414+0000","last_became_active":"2023-12-01T03:10:39.005386+0000","last_became_peered":"2023-12-01T03:10:39.005386+0000","last_unstale":"2023-12-01T03:10:39.008414+0000","last_undegraded":"2023-12-01T03:10:39.008414+0000","last_fullsized":"2023-12-01T03:10:39.008414+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488889+0000","last_change":"2023-12-01T03:10:24.697992+0000","last_active":"2023-12-01T03:10:37.488889+0000","last_peered":"2023-12-01T03:10:37.488889+0000","last_clean":"2023-12-01T03:10:37.488889+0000","last_became_active":"2023-12-01T03:10:24.696883+0000","last_became_peered":"2023-12-01T03:10:24.696883+0000","last_unstale":"2023-12-01T03:10:37.488889+0000","last_undegraded":"2023-12-01T03:10:37.488889+0000","last_fullsized":"2023-12-01T03:10:37.488889+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.487148+0000","last_change":"2023-12-01T03:10:25.175476+0000","last_active":"2023-12-01T03:10:37.487148+0000","last_peered":"2023-12-01T03:10:37.487148+0000","last_clean":"2023-12-01T03:10:37.487148+0000","last_became_active":"2023-12-01T03:10:25.175211+0000","last_became_peered":"2023-12-01T03:10:25.175211+0000","last_unstale":"2023-12-01T03:10:37.487148+0000","last_undegraded":"2023-12-01T03:10:37.487148+0000","last_fullsized":"2023-12-01T03:10:37.487148+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:22.358205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515962+0000","last_change":"2023-12-01T03:10:38.513347+0000","last_active":"2023-12-01T03:10:38.515962+0000","last_peered":"2023-12-01T03:10:38.515962+0000","last_clean":"2023-12-01T03:10:38.515962+0000","last_became_active":"2023-12-01T03:10:38.512782+0000","last_became_peered":"2023-12-01T03:10:38.512782+0000","last_unstale":"2023-12-01T03:10:38.515962+0000","last_undegraded":"2023-12-01T03:10:38.515962+0000","last_fullsized":"2023-12-01T03:10:38.515962+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:06.647413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.010194+0000","last_change":"2023-12-01T03:10:39.007372+0000","last_active":"2023-12-01T03:10:39.010194+0000","last_peered":"2023-12-01T03:10:39.010194+0000","last_clean":"2023-12-01T03:10:39.010194+0000","last_became_active":"2023-12-01T03:10:39.007059+0000","last_became_peered":"2023-12-01T03:10:39.007059+0000","last_unstale":"2023-12-01T03:10:39.010194+0000","last_undegraded":"2023-12-01T03:10:39.010194+0000","last_fullsized":"2023-12-01T03:10:39.010194+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.492807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"214'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489570+0000","last_change":"2023-12-01T03:10:24.697890+0000","last_active":"2023-12-01T03:10:37.489570+0000","last_peered":"2023-12-01T03:10:37.489570+0000","last_clean":"2023-12-01T03:10:37.489570+0000","last_became_active":"2023-12-01T03:10:24.697679+0000","last_became_peered":"2023-12-01T03:10:24.697679+0000","last_unstale":"2023-12-01T03:10:37.489570+0000","last_undegraded":"2023-12-01T03:10:37.489570+0000","last_fullsized":"2023-12-01T03:10:37.489570+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488204+0000","last_change":"2023-12-01T03:10:25.168359+0000","last_active":"2023-12-01T03:10:37.488204+0000","last_peered":"2023-12-01T03:10:37.488204+0000","last_clean":"2023-12-01T03:10:37.488204+0000","last_became_active":"2023-12-01T03:10:25.168082+0000","last_became_peered":"2023-12-01T03:10:25.168082+0000","last_unstale":"2023-12-01T03:10:37.488204+0000","last_undegraded":"2023-12-01T03:10:37.488204+0000","last_fullsized":"2023-12-01T03:10:37.488204+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:06.024028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525349+0000","last_change":"2023-12-01T03:10:38.524232+0000","last_active":"2023-12-01T03:10:38.525349+0000","last_peered":"2023-12-01T03:10:38.525349+0000","last_clean":"2023-12-01T03:10:38.525349+0000","last_became_active":"2023-12-01T03:10:38.523871+0000","last_became_peered":"2023-12-01T03:10:38.523871+0000","last_unstale":"2023-12-01T03:10:38.525349+0000","last_undegraded":"2023-12-01T03:10:38.525349+0000","last_fullsized":"2023-12-01T03:10:38.525349+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:06.647413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499505+0000","last_change":"2023-12-01T03:10:38.499505+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499505+0000","last_undegraded":"2023-12-01T03:10:38.499505+0000","last_fullsized":"2023-12-01T03:10:38.499505+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.7","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472938+0000","last_change":"2023-12-01T03:10:24.687160+0000","last_active":"2023-12-01T03:10:37.472938+0000","last_peered":"2023-12-01T03:10:37.472938+0000","last_clean":"2023-12-01T03:10:37.472938+0000","last_became_active":"2023-12-01T03:10:24.686184+0000","last_became_peered":"2023-12-01T03:10:24.686184+0000","last_unstale":"2023-12-01T03:10:37.472938+0000","last_undegraded":"2023-12-01T03:10:37.472938+0000","last_fullsized":"2023-12-01T03:10:37.472938+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:05.006954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489573+0000","last_change":"2023-12-01T03:10:25.184992+0000","last_active":"2023-12-01T03:10:37.489573+0000","last_peered":"2023-12-01T03:10:37.489573+0000","last_clean":"2023-12-01T03:10:37.489573+0000","last_became_active":"2023-12-01T03:10:25.184670+0000","last_became_peered":"2023-12-01T03:10:25.184670+0000","last_unstale":"2023-12-01T03:10:37.489573+0000","last_undegraded":"2023-12-01T03:10:37.489573+0000","last_fullsized":"2023-12-01T03:10:37.489573+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:40.143822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519720+0000","last_change":"2023-12-01T03:10:38.517303+0000","last_active":"2023-12-01T03:10:38.519720+0000","last_peered":"2023-12-01T03:10:38.519720+0000","last_clean":"2023-12-01T03:10:38.519720+0000","last_became_active":"2023-12-01T03:10:38.516948+0000","last_became_peered":"2023-12-01T03:10:38.516948+0000","last_unstale":"2023-12-01T03:10:38.519720+0000","last_undegraded":"2023-12-01T03:10:38.519720+0000","last_fullsized":"2023-12-01T03:10:38.519720+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:58:41.627485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504217+0000","last_change":"2023-12-01T03:10:38.504217+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504217+0000","last_undegraded":"2023-12-01T03:10:38.504217+0000","last_fullsized":"2023-12-01T03:10:38.504217+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:18.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472354+0000","last_change":"2023-12-01T03:10:24.685003+0000","last_active":"2023-12-01T03:10:37.472354+0000","last_peered":"2023-12-01T03:10:37.472354+0000","last_clean":"2023-12-01T03:10:37.472354+0000","last_became_active":"2023-12-01T03:10:24.684425+0000","last_became_peered":"2023-12-01T03:10:24.684425+0000","last_unstale":"2023-12-01T03:10:37.472354+0000","last_undegraded":"2023-12-01T03:10:37.472354+0000","last_fullsized":"2023-12-01T03:10:37.472354+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:45.173492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491598+0000","last_change":"2023-12-01T03:10:25.174479+0000","last_active":"2023-12-01T03:10:37.491598+0000","last_peered":"2023-12-01T03:10:37.491598+0000","last_clean":"2023-12-01T03:10:37.491598+0000","last_became_active":"2023-12-01T03:10:25.174194+0000","last_became_peered":"2023-12-01T03:10:25.174194+0000","last_unstale":"2023-12-01T03:10:37.491598+0000","last_undegraded":"2023-12-01T03:10:37.491598+0000","last_fullsized":"2023-12-01T03:10:37.491598+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519343+0000","last_change":"2023-12-01T03:10:38.516993+0000","last_active":"2023-12-01T03:10:38.519343+0000","last_peered":"2023-12-01T03:10:38.519343+0000","last_clean":"2023-12-01T03:10:38.519343+0000","last_became_active":"2023-12-01T03:10:38.516717+0000","last_became_peered":"2023-12-01T03:10:38.516717+0000","last_unstale":"2023-12-01T03:10:38.519343+0000","last_undegraded":"2023-12-01T03:10:38.519343+0000","last_fullsized":"2023-12-01T03:10:38.519343+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:51.689717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508392+0000","last_change":"2023-12-01T03:10:38.508392+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508392+0000","last_undegraded":"2023-12-01T03:10:38.508392+0000","last_fullsized":"2023-12-01T03:10:38.508392+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:53.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481282+0000","last_change":"2023-12-01T03:10:24.694594+0000","last_active":"2023-12-01T03:10:37.481282+0000","last_peered":"2023-12-01T03:10:37.481282+0000","last_clean":"2023-12-01T03:10:37.481282+0000","last_became_active":"2023-12-01T03:10:24.693808+0000","last_became_peered":"2023-12-01T03:10:24.693808+0000","last_unstale":"2023-12-01T03:10:37.481282+0000","last_undegraded":"2023-12-01T03:10:37.481282+0000","last_fullsized":"2023-12-01T03:10:37.481282+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472327+0000","last_change":"2023-12-01T03:10:25.173728+0000","last_active":"2023-12-01T03:10:37.472327+0000","last_peered":"2023-12-01T03:10:37.472327+0000","last_clean":"2023-12-01T03:10:37.472327+0000","last_became_active":"2023-12-01T03:10:25.173449+0000","last_became_peered":"2023-12-01T03:10:25.173449+0000","last_unstale":"2023-12-01T03:10:37.472327+0000","last_undegraded":"2023-12-01T03:10:37.472327+0000","last_fullsized":"2023-12-01T03:10:37.472327+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:20.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.513730+0000","last_change":"2023-12-01T03:10:38.508401+0000","last_active":"2023-12-01T03:10:38.513730+0000","last_peered":"2023-12-01T03:10:38.513730+0000","last_clean":"2023-12-01T03:10:38.513730+0000","last_became_active":"2023-12-01T03:10:38.508089+0000","last_became_peered":"2023-12-01T03:10:38.508089+0000","last_unstale":"2023-12-01T03:10:38.513730+0000","last_undegraded":"2023-12-01T03:10:38.513730+0000","last_fullsized":"2023-12-01T03:10:38.513730+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:56.645685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003642+0000","last_change":"2023-12-01T03:10:39.000679+0000","last_active":"2023-12-01T03:10:39.003642+0000","last_peered":"2023-12-01T03:10:39.003642+0000","last_clean":"2023-12-01T03:10:39.003642+0000","last_became_active":"2023-12-01T03:10:38.997903+0000","last_became_peered":"2023-12-01T03:10:38.997903+0000","last_unstale":"2023-12-01T03:10:39.003642+0000","last_undegraded":"2023-12-01T03:10:39.003642+0000","last_fullsized":"2023-12-01T03:10:39.003642+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:27.357798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482736+0000","last_change":"2023-12-01T03:10:24.685775+0000","last_active":"2023-12-01T03:10:37.482736+0000","last_peered":"2023-12-01T03:10:37.482736+0000","last_clean":"2023-12-01T03:10:37.482736+0000","last_became_active":"2023-12-01T03:10:24.685178+0000","last_became_peered":"2023-12-01T03:10:24.685178+0000","last_unstale":"2023-12-01T03:10:37.482736+0000","last_undegraded":"2023-12-01T03:10:37.482736+0000","last_fullsized":"2023-12-01T03:10:37.482736+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:21.942226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483463+0000","last_change":"2023-12-01T03:10:25.180768+0000","last_active":"2023-12-01T03:10:37.483463+0000","last_peered":"2023-12-01T03:10:37.483463+0000","last_clean":"2023-12-01T03:10:37.483463+0000","last_became_active":"2023-12-01T03:10:25.179796+0000","last_became_peered":"2023-12-01T03:10:25.179796+0000","last_unstale":"2023-12-01T03:10:37.483463+0000","last_undegraded":"2023-12-01T03:10:37.483463+0000","last_fullsized":"2023-12-01T03:10:37.483463+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515225+0000","last_change":"2023-12-01T03:10:38.511676+0000","last_active":"2023-12-01T03:10:38.515225+0000","last_peered":"2023-12-01T03:10:38.515225+0000","last_clean":"2023-12-01T03:10:38.515225+0000","last_became_active":"2023-12-01T03:10:38.511406+0000","last_became_peered":"2023-12-01T03:10:38.511406+0000","last_unstale":"2023-12-01T03:10:38.515225+0000","last_undegraded":"2023-12-01T03:10:38.515225+0000","last_fullsized":"2023-12-01T03:10:38.515225+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:44.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003720+0000","last_change":"2023-12-01T03:10:38.997814+0000","last_active":"2023-12-01T03:10:39.003720+0000","last_peered":"2023-12-01T03:10:39.003720+0000","last_clean":"2023-12-01T03:10:39.003720+0000","last_became_active":"2023-12-01T03:10:38.997558+0000","last_became_peered":"2023-12-01T03:10:38.997558+0000","last_unstale":"2023-12-01T03:10:39.003720+0000","last_undegraded":"2023-12-01T03:10:39.003720+0000","last_fullsized":"2023-12-01T03:10:39.003720+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:49:09.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.b","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481673+0000","last_change":"2023-12-01T03:10:24.695154+0000","last_active":"2023-12-01T03:10:37.481673+0000","last_peered":"2023-12-01T03:10:37.481673+0000","last_clean":"2023-12-01T03:10:37.481673+0000","last_became_active":"2023-12-01T03:10:24.694868+0000","last_became_peered":"2023-12-01T03:10:24.694868+0000","last_unstale":"2023-12-01T03:10:37.481673+0000","last_undegraded":"2023-12-01T03:10:37.481673+0000","last_fullsized":"2023-12-01T03:10:37.481673+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482517+0000","last_change":"2023-12-01T03:10:25.169678+0000","last_active":"2023-12-01T03:10:37.482517+0000","last_peered":"2023-12-01T03:10:37.482517+0000","last_clean":"2023-12-01T03:10:37.482517+0000","last_became_active":"2023-12-01T03:10:25.168714+0000","last_became_peered":"2023-12-01T03:10:25.168714+0000","last_unstale":"2023-12-01T03:10:37.482517+0000","last_undegraded":"2023-12-01T03:10:37.482517+0000","last_fullsized":"2023-12-01T03:10:37.482517+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.526019+0000","last_change":"2023-12-01T03:10:38.524888+0000","last_active":"2023-12-01T03:10:38.526019+0000","last_peered":"2023-12-01T03:10:38.526019+0000","last_clean":"2023-12-01T03:10:38.526019+0000","last_became_active":"2023-12-01T03:10:38.524601+0000","last_became_peered":"2023-12-01T03:10:38.524601+0000","last_unstale":"2023-12-01T03:10:38.526019+0000","last_undegraded":"2023-12-01T03:10:38.526019+0000","last_fullsized":"2023-12-01T03:10:38.526019+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:47.875991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.011924+0000","last_change":"2023-12-01T03:10:39.009719+0000","last_active":"2023-12-01T03:10:39.011924+0000","last_peered":"2023-12-01T03:10:39.011924+0000","last_clean":"2023-12-01T03:10:39.011924+0000","last_became_active":"2023-12-01T03:10:39.009561+0000","last_became_peered":"2023-12-01T03:10:39.009561+0000","last_unstale":"2023-12-01T03:10:39.011924+0000","last_undegraded":"2023-12-01T03:10:39.011924+0000","last_fullsized":"2023-12-01T03:10:39.011924+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:19.853707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473031+0000","last_change":"2023-12-01T03:10:24.683594+0000","last_active":"2023-12-01T03:10:37.473031+0000","last_peered":"2023-12-01T03:10:37.473031+0000","last_clean":"2023-12-01T03:10:37.473031+0000","last_became_active":"2023-12-01T03:10:24.679979+0000","last_became_peered":"2023-12-01T03:10:24.679979+0000","last_unstale":"2023-12-01T03:10:37.473031+0000","last_undegraded":"2023-12-01T03:10:37.473031+0000","last_fullsized":"2023-12-01T03:10:37.473031+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:12.511044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987554+0000","last_change":"2023-12-01T03:10:25.172837+0000","last_active":"2023-12-01T03:10:38.987554+0000","last_peered":"2023-12-01T03:10:38.987554+0000","last_clean":"2023-12-01T03:10:38.987554+0000","last_became_active":"2023-12-01T03:10:25.172602+0000","last_became_peered":"2023-12-01T03:10:25.172602+0000","last_unstale":"2023-12-01T03:10:38.987554+0000","last_undegraded":"2023-12-01T03:10:38.987554+0000","last_fullsized":"2023-12-01T03:10:38.987554+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521284+0000","last_change":"2023-12-01T03:10:38.517877+0000","last_active":"2023-12-01T03:10:38.521284+0000","last_peered":"2023-12-01T03:10:38.521284+0000","last_clean":"2023-12-01T03:10:38.521284+0000","last_became_active":"2023-12-01T03:10:38.517241+0000","last_became_peered":"2023-12-01T03:10:38.517241+0000","last_unstale":"2023-12-01T03:10:38.521284+0000","last_undegraded":"2023-12-01T03:10:38.521284+0000","last_fullsized":"2023-12-01T03:10:38.521284+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:32.967785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.004933+0000","last_change":"2023-12-01T03:10:39.003555+0000","last_active":"2023-12-01T03:10:39.004933+0000","last_peered":"2023-12-01T03:10:39.004933+0000","last_clean":"2023-12-01T03:10:39.004933+0000","last_became_active":"2023-12-01T03:10:39.003311+0000","last_became_peered":"2023-12-01T03:10:39.003311+0000","last_unstale":"2023-12-01T03:10:39.004933+0000","last_undegraded":"2023-12-01T03:10:39.004933+0000","last_fullsized":"2023-12-01T03:10:39.004933+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.d","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473150+0000","last_change":"2023-12-01T03:10:24.691398+0000","last_active":"2023-12-01T03:10:37.473150+0000","last_peered":"2023-12-01T03:10:37.473150+0000","last_clean":"2023-12-01T03:10:37.473150+0000","last_became_active":"2023-12-01T03:10:24.691110+0000","last_became_peered":"2023-12-01T03:10:24.691110+0000","last_unstale":"2023-12-01T03:10:37.473150+0000","last_undegraded":"2023-12-01T03:10:37.473150+0000","last_fullsized":"2023-12-01T03:10:37.473150+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:24.670899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999898+0000","last_change":"2023-12-01T03:10:25.178959+0000","last_active":"2023-12-01T03:10:38.999898+0000","last_peered":"2023-12-01T03:10:38.999898+0000","last_clean":"2023-12-01T03:10:38.999898+0000","last_became_active":"2023-12-01T03:10:25.178268+0000","last_became_peered":"2023-12-01T03:10:25.178268+0000","last_unstale":"2023-12-01T03:10:38.999898+0000","last_undegraded":"2023-12-01T03:10:38.999898+0000","last_fullsized":"2023-12-01T03:10:38.999898+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:55:31.814432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525071+0000","last_change":"2023-12-01T03:10:38.523799+0000","last_active":"2023-12-01T03:10:38.525071+0000","last_peered":"2023-12-01T03:10:38.525071+0000","last_clean":"2023-12-01T03:10:38.525071+0000","last_became_active":"2023-12-01T03:10:38.523512+0000","last_became_peered":"2023-12-01T03:10:38.523512+0000","last_unstale":"2023-12-01T03:10:38.525071+0000","last_undegraded":"2023-12-01T03:10:38.525071+0000","last_fullsized":"2023-12-01T03:10:38.525071+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:44.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504526+0000","last_change":"2023-12-01T03:10:38.504526+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504526+0000","last_undegraded":"2023-12-01T03:10:38.504526+0000","last_fullsized":"2023-12-01T03:10:38.504526+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:41.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482803+0000","last_change":"2023-12-01T03:10:24.688107+0000","last_active":"2023-12-01T03:10:37.482803+0000","last_peered":"2023-12-01T03:10:37.482803+0000","last_clean":"2023-12-01T03:10:37.482803+0000","last_became_active":"2023-12-01T03:10:24.686930+0000","last_became_peered":"2023-12-01T03:10:24.686930+0000","last_unstale":"2023-12-01T03:10:37.482803+0000","last_undegraded":"2023-12-01T03:10:37.482803+0000","last_fullsized":"2023-12-01T03:10:37.482803+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:44.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986385+0000","last_change":"2023-12-01T03:10:25.180913+0000","last_active":"2023-12-01T03:10:38.986385+0000","last_peered":"2023-12-01T03:10:38.986385+0000","last_clean":"2023-12-01T03:10:38.986385+0000","last_became_active":"2023-12-01T03:10:25.180626+0000","last_became_peered":"2023-12-01T03:10:25.180626+0000","last_unstale":"2023-12-01T03:10:38.986385+0000","last_undegraded":"2023-12-01T03:10:38.986385+0000","last_fullsized":"2023-12-01T03:10:38.986385+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520264+0000","last_change":"2023-12-01T03:10:38.519139+0000","last_active":"2023-12-01T03:10:38.520264+0000","last_peered":"2023-12-01T03:10:38.520264+0000","last_clean":"2023-12-01T03:10:38.520264+0000","last_became_active":"2023-12-01T03:10:38.518855+0000","last_became_peered":"2023-12-01T03:10:38.518855+0000","last_unstale":"2023-12-01T03:10:38.520264+0000","last_undegraded":"2023-12-01T03:10:38.520264+0000","last_fullsized":"2023-12-01T03:10:38.520264+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:52.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511251+0000","last_change":"2023-12-01T03:10:38.511251+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511251+0000","last_undegraded":"2023-12-01T03:10:38.511251+0000","last_fullsized":"2023-12-01T03:10:38.511251+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:47:30.021172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.f","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482669+0000","last_change":"2023-12-01T03:10:24.699945+0000","last_active":"2023-12-01T03:10:37.482669+0000","last_peered":"2023-12-01T03:10:37.482669+0000","last_clean":"2023-12-01T03:10:37.482669+0000","last_became_active":"2023-12-01T03:10:24.699598+0000","last_became_peered":"2023-12-01T03:10:24.699598+0000","last_unstale":"2023-12-01T03:10:37.482669+0000","last_undegraded":"2023-12-01T03:10:37.482669+0000","last_fullsized":"2023-12-01T03:10:37.482669+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492690+0000","last_change":"2023-12-01T03:10:25.181692+0000","last_active":"2023-12-01T03:10:37.492690+0000","last_peered":"2023-12-01T03:10:37.492690+0000","last_clean":"2023-12-01T03:10:37.492690+0000","last_became_active":"2023-12-01T03:10:25.181316+0000","last_became_peered":"2023-12-01T03:10:25.181316+0000","last_unstale":"2023-12-01T03:10:37.492690+0000","last_undegraded":"2023-12-01T03:10:37.492690+0000","last_fullsized":"2023-12-01T03:10:37.492690+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:57:07.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.522721+0000","last_change":"2023-12-01T03:10:38.518068+0000","last_active":"2023-12-01T03:10:38.522721+0000","last_peered":"2023-12-01T03:10:38.522721+0000","last_clean":"2023-12-01T03:10:38.522721+0000","last_became_active":"2023-12-01T03:10:38.517792+0000","last_became_peered":"2023-12-01T03:10:38.517792+0000","last_unstale":"2023-12-01T03:10:38.522721+0000","last_undegraded":"2023-12-01T03:10:38.522721+0000","last_fullsized":"2023-12-01T03:10:38.522721+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:01.616973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511262+0000","last_change":"2023-12-01T03:10:38.511262+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511262+0000","last_undegraded":"2023-12-01T03:10:38.511262+0000","last_fullsized":"2023-12-01T03:10:38.511262+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"213'8","reported_seq":21,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483272+0000","last_change":"2023-12-01T03:10:24.685759+0000","last_active":"2023-12-01T03:10:37.483272+0000","last_peered":"2023-12-01T03:10:37.483272+0000","last_clean":"2023-12-01T03:10:37.483272+0000","last_became_active":"2023-12-01T03:10:24.684800+0000","last_became_peered":"2023-12-01T03:10:24.684800+0000","last_unstale":"2023-12-01T03:10:37.483272+0000","last_undegraded":"2023-12-01T03:10:37.483272+0000","last_fullsized":"2023-12-01T03:10:37.483272+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:29.129070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478917+0000","last_change":"2023-12-01T03:10:25.193021+0000","last_active":"2023-12-01T03:10:37.478917+0000","last_peered":"2023-12-01T03:10:37.478917+0000","last_clean":"2023-12-01T03:10:37.478917+0000","last_became_active":"2023-12-01T03:10:25.192051+0000","last_became_peered":"2023-12-01T03:10:25.192051+0000","last_unstale":"2023-12-01T03:10:37.478917+0000","last_undegraded":"2023-12-01T03:10:37.478917+0000","last_fullsized":"2023-12-01T03:10:37.478917+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521581+0000","last_change":"2023-12-01T03:10:38.517969+0000","last_active":"2023-12-01T03:10:38.521581+0000","last_peered":"2023-12-01T03:10:38.521581+0000","last_clean":"2023-12-01T03:10:38.521581+0000","last_became_active":"2023-12-01T03:10:38.517693+0000","last_became_peered":"2023-12-01T03:10:38.517693+0000","last_unstale":"2023-12-01T03:10:38.521581+0000","last_undegraded":"2023-12-01T03:10:38.521581+0000","last_fullsized":"2023-12-01T03:10:38.521581+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:18.447021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.498276+0000","last_change":"2023-12-01T03:10:38.498276+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.498276+0000","last_undegraded":"2023-12-01T03:10:38.498276+0000","last_fullsized":"2023-12-01T03:10:38.498276+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479511+0000","last_change":"2023-12-01T03:10:24.694942+0000","last_active":"2023-12-01T03:10:37.479511+0000","last_peered":"2023-12-01T03:10:37.479511+0000","last_clean":"2023-12-01T03:10:37.479511+0000","last_became_active":"2023-12-01T03:10:24.694664+0000","last_became_peered":"2023-12-01T03:10:24.694664+0000","last_unstale":"2023-12-01T03:10:37.479511+0000","last_undegraded":"2023-12-01T03:10:37.479511+0000","last_fullsized":"2023-12-01T03:10:37.479511+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:38:02.332793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988703+0000","last_change":"2023-12-01T03:10:25.164054+0000","last_active":"2023-12-01T03:10:38.988703+0000","last_peered":"2023-12-01T03:10:38.988703+0000","last_clean":"2023-12-01T03:10:38.988703+0000","last_became_active":"2023-12-01T03:10:25.163787+0000","last_became_peered":"2023-12-01T03:10:25.163787+0000","last_unstale":"2023-12-01T03:10:38.988703+0000","last_undegraded":"2023-12-01T03:10:38.988703+0000","last_fullsized":"2023-12-01T03:10:38.988703+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:04.568421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521758+0000","last_change":"2023-12-01T03:10:38.520314+0000","last_active":"2023-12-01T03:10:38.521758+0000","last_peered":"2023-12-01T03:10:38.521758+0000","last_clean":"2023-12-01T03:10:38.521758+0000","last_became_active":"2023-12-01T03:10:38.519056+0000","last_became_peered":"2023-12-01T03:10:38.519056+0000","last_unstale":"2023-12-01T03:10:38.521758+0000","last_undegraded":"2023-12-01T03:10:38.521758+0000","last_fullsized":"2023-12-01T03:10:38.521758+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:40.686905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.502663+0000","last_change":"2023-12-01T03:10:38.502663+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.502663+0000","last_undegraded":"2023-12-01T03:10:38.502663+0000","last_fullsized":"2023-12-01T03:10:38.502663+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491751+0000","last_change":"2023-12-01T03:10:24.685823+0000","last_active":"2023-12-01T03:10:37.491751+0000","last_peered":"2023-12-01T03:10:37.491751+0000","last_clean":"2023-12-01T03:10:37.491751+0000","last_became_active":"2023-12-01T03:10:24.685560+0000","last_became_peered":"2023-12-01T03:10:24.685560+0000","last_unstale":"2023-12-01T03:10:37.491751+0000","last_undegraded":"2023-12-01T03:10:37.491751+0000","last_fullsized":"2023-12-01T03:10:37.491751+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483380+0000","last_change":"2023-12-01T03:10:25.183804+0000","last_active":"2023-12-01T03:10:37.483380+0000","last_peered":"2023-12-01T03:10:37.483380+0000","last_clean":"2023-12-01T03:10:37.483380+0000","last_became_active":"2023-12-01T03:10:25.183079+0000","last_became_peered":"2023-12-01T03:10:25.183079+0000","last_unstale":"2023-12-01T03:10:37.483380+0000","last_undegraded":"2023-12-01T03:10:37.483380+0000","last_fullsized":"2023-12-01T03:10:37.483380+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.510374+0000","last_change":"2023-12-01T03:10:38.508144+0000","last_active":"2023-12-01T03:10:38.510374+0000","last_peered":"2023-12-01T03:10:38.510374+0000","last_clean":"2023-12-01T03:10:38.510374+0000","last_became_active":"2023-12-01T03:10:38.507850+0000","last_became_peered":"2023-12-01T03:10:38.507850+0000","last_unstale":"2023-12-01T03:10:38.510374+0000","last_undegraded":"2023-12-01T03:10:38.510374+0000","last_fullsized":"2023-12-01T03:10:38.510374+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:52.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507590+0000","last_change":"2023-12-01T03:10:38.507590+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507590+0000","last_undegraded":"2023-12-01T03:10:38.507590+0000","last_fullsized":"2023-12-01T03:10:38.507590+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:45.156063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.13","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987606+0000","last_change":"2023-12-01T03:10:24.692854+0000","last_active":"2023-12-01T03:10:38.987606+0000","last_peered":"2023-12-01T03:10:38.987606+0000","last_clean":"2023-12-01T03:10:38.987606+0000","last_became_active":"2023-12-01T03:10:24.692563+0000","last_became_peered":"2023-12-01T03:10:24.692563+0000","last_unstale":"2023-12-01T03:10:38.987606+0000","last_undegraded":"2023-12-01T03:10:38.987606+0000","last_fullsized":"2023-12-01T03:10:38.987606+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491687+0000","last_change":"2023-12-01T03:10:25.186716+0000","last_active":"2023-12-01T03:10:37.491687+0000","last_peered":"2023-12-01T03:10:37.491687+0000","last_clean":"2023-12-01T03:10:37.491687+0000","last_became_active":"2023-12-01T03:10:25.186422+0000","last_became_peered":"2023-12-01T03:10:25.186422+0000","last_unstale":"2023-12-01T03:10:37.491687+0000","last_undegraded":"2023-12-01T03:10:37.491687+0000","last_fullsized":"2023-12-01T03:10:37.491687+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.510490+0000","last_change":"2023-12-01T03:10:38.508098+0000","last_active":"2023-12-01T03:10:38.510490+0000","last_peered":"2023-12-01T03:10:38.510490+0000","last_clean":"2023-12-01T03:10:38.510490+0000","last_became_active":"2023-12-01T03:10:38.506510+0000","last_became_peered":"2023-12-01T03:10:38.506510+0000","last_unstale":"2023-12-01T03:10:38.510490+0000","last_undegraded":"2023-12-01T03:10:38.510490+0000","last_fullsized":"2023-12-01T03:10:38.510490+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:01.074406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504758+0000","last_change":"2023-12-01T03:10:38.504758+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504758+0000","last_undegraded":"2023-12-01T03:10:38.504758+0000","last_fullsized":"2023-12-01T03:10:38.504758+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:18.605693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"214'8","reported_seq":21,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481263+0000","last_change":"2023-12-01T03:10:24.697328+0000","last_active":"2023-12-01T03:10:37.481263+0000","last_peered":"2023-12-01T03:10:37.481263+0000","last_clean":"2023-12-01T03:10:37.481263+0000","last_became_active":"2023-12-01T03:10:24.696986+0000","last_became_peered":"2023-12-01T03:10:24.696986+0000","last_unstale":"2023-12-01T03:10:37.481263+0000","last_undegraded":"2023-12-01T03:10:37.481263+0000","last_fullsized":"2023-12-01T03:10:37.481263+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492498+0000","last_change":"2023-12-01T03:10:25.176864+0000","last_active":"2023-12-01T03:10:37.492498+0000","last_peered":"2023-12-01T03:10:37.492498+0000","last_clean":"2023-12-01T03:10:37.492498+0000","last_became_active":"2023-12-01T03:10:25.176588+0000","last_became_peered":"2023-12-01T03:10:25.176588+0000","last_unstale":"2023-12-01T03:10:37.492498+0000","last_undegraded":"2023-12-01T03:10:37.492498+0000","last_fullsized":"2023-12-01T03:10:37.492498+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987075+0000","last_change":"2023-12-01T03:10:38.519450+0000","last_active":"2023-12-01T03:10:38.987075+0000","last_peered":"2023-12-01T03:10:38.987075+0000","last_clean":"2023-12-01T03:10:38.987075+0000","last_became_active":"2023-12-01T03:10:38.519108+0000","last_became_peered":"2023-12-01T03:10:38.519108+0000","last_unstale":"2023-12-01T03:10:38.987075+0000","last_undegraded":"2023-12-01T03:10:38.987075+0000","last_fullsized":"2023-12-01T03:10:38.987075+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:58.926662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007897+0000","last_change":"2023-12-01T03:10:39.005006+0000","last_active":"2023-12-01T03:10:39.007897+0000","last_peered":"2023-12-01T03:10:39.007897+0000","last_clean":"2023-12-01T03:10:39.007897+0000","last_became_active":"2023-12-01T03:10:39.004573+0000","last_became_peered":"2023-12-01T03:10:39.004573+0000","last_unstale":"2023-12-01T03:10:39.007897+0000","last_undegraded":"2023-12-01T03:10:39.007897+0000","last_fullsized":"2023-12-01T03:10:39.007897+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:07.829269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"213'3","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481279+0000","last_change":"2023-12-01T03:10:24.699528+0000","last_active":"2023-12-01T03:10:37.481279+0000","last_peered":"2023-12-01T03:10:37.481279+0000","last_clean":"2023-12-01T03:10:37.481279+0000","last_became_active":"2023-12-01T03:10:24.699152+0000","last_became_peered":"2023-12-01T03:10:24.699152+0000","last_unstale":"2023-12-01T03:10:37.481279+0000","last_undegraded":"2023-12-01T03:10:37.481279+0000","last_fullsized":"2023-12-01T03:10:37.481279+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:07.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987736+0000","last_change":"2023-12-01T03:10:25.187487+0000","last_active":"2023-12-01T03:10:38.987736+0000","last_peered":"2023-12-01T03:10:38.987736+0000","last_clean":"2023-12-01T03:10:38.987736+0000","last_became_active":"2023-12-01T03:10:25.187198+0000","last_became_peered":"2023-12-01T03:10:25.187198+0000","last_unstale":"2023-12-01T03:10:38.987736+0000","last_undegraded":"2023-12-01T03:10:38.987736+0000","last_fullsized":"2023-12-01T03:10:38.987736+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:00.948964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.526023+0000","last_change":"2023-12-01T03:10:38.525203+0000","last_active":"2023-12-01T03:10:38.526023+0000","last_peered":"2023-12-01T03:10:38.526023+0000","last_clean":"2023-12-01T03:10:38.526023+0000","last_became_active":"2023-12-01T03:10:38.524926+0000","last_became_peered":"2023-12-01T03:10:38.524926+0000","last_unstale":"2023-12-01T03:10:38.526023+0000","last_undegraded":"2023-12-01T03:10:38.526023+0000","last_fullsized":"2023-12-01T03:10:38.526023+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:17.593868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.503576+0000","last_change":"2023-12-01T03:10:38.503576+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.503576+0000","last_undegraded":"2023-12-01T03:10:38.503576+0000","last_fullsized":"2023-12-01T03:10:38.503576+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492851+0000","last_change":"2023-12-01T03:10:24.691537+0000","last_active":"2023-12-01T03:10:37.492851+0000","last_peered":"2023-12-01T03:10:37.492851+0000","last_clean":"2023-12-01T03:10:37.492851+0000","last_became_active":"2023-12-01T03:10:24.691168+0000","last_became_peered":"2023-12-01T03:10:24.691168+0000","last_unstale":"2023-12-01T03:10:37.492851+0000","last_undegraded":"2023-12-01T03:10:37.492851+0000","last_fullsized":"2023-12-01T03:10:37.492851+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492811+0000","last_change":"2023-12-01T03:10:25.178648+0000","last_active":"2023-12-01T03:10:37.492811+0000","last_peered":"2023-12-01T03:10:37.492811+0000","last_clean":"2023-12-01T03:10:37.492811+0000","last_became_active":"2023-12-01T03:10:25.178352+0000","last_became_peered":"2023-12-01T03:10:25.178352+0000","last_unstale":"2023-12-01T03:10:37.492811+0000","last_undegraded":"2023-12-01T03:10:37.492811+0000","last_fullsized":"2023-12-01T03:10:37.492811+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519785+0000","last_change":"2023-12-01T03:10:38.515822+0000","last_active":"2023-12-01T03:10:38.519785+0000","last_peered":"2023-12-01T03:10:38.519785+0000","last_clean":"2023-12-01T03:10:38.519785+0000","last_became_active":"2023-12-01T03:10:38.515536+0000","last_became_peered":"2023-12-01T03:10:38.515536+0000","last_unstale":"2023-12-01T03:10:38.519785+0000","last_undegraded":"2023-12-01T03:10:38.519785+0000","last_fullsized":"2023-12-01T03:10:38.519785+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:10:36.574945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.510307+0000","last_change":"2023-12-01T03:10:38.510307+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.510307+0000","last_undegraded":"2023-12-01T03:10:38.510307+0000","last_fullsized":"2023-12-01T03:10:38.510307+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472906+0000","last_change":"2023-12-01T03:10:24.687709+0000","last_active":"2023-12-01T03:10:37.472906+0000","last_peered":"2023-12-01T03:10:37.472906+0000","last_clean":"2023-12-01T03:10:37.472906+0000","last_became_active":"2023-12-01T03:10:24.687452+0000","last_became_peered":"2023-12-01T03:10:24.687452+0000","last_unstale":"2023-12-01T03:10:37.472906+0000","last_undegraded":"2023-12-01T03:10:37.472906+0000","last_fullsized":"2023-12-01T03:10:37.472906+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:39.126749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489332+0000","last_change":"2023-12-01T03:10:25.176941+0000","last_active":"2023-12-01T03:10:37.489332+0000","last_peered":"2023-12-01T03:10:37.489332+0000","last_clean":"2023-12-01T03:10:37.489332+0000","last_became_active":"2023-12-01T03:10:25.175692+0000","last_became_peered":"2023-12-01T03:10:25.175692+0000","last_unstale":"2023-12-01T03:10:37.489332+0000","last_undegraded":"2023-12-01T03:10:37.489332+0000","last_fullsized":"2023-12-01T03:10:37.489332+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.513115+0000","last_change":"2023-12-01T03:10:38.509844+0000","last_active":"2023-12-01T03:10:38.513115+0000","last_peered":"2023-12-01T03:10:38.513115+0000","last_clean":"2023-12-01T03:10:38.513115+0000","last_became_active":"2023-12-01T03:10:38.509319+0000","last_became_peered":"2023-12-01T03:10:38.509319+0000","last_unstale":"2023-12-01T03:10:38.513115+0000","last_undegraded":"2023-12-01T03:10:38.513115+0000","last_fullsized":"2023-12-01T03:10:38.513115+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:12.432439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012565+0000","last_change":"2023-12-01T03:10:39.010700+0000","last_active":"2023-12-01T03:10:39.012565+0000","last_peered":"2023-12-01T03:10:39.012565+0000","last_clean":"2023-12-01T03:10:39.012565+0000","last_became_active":"2023-12-01T03:10:39.010389+0000","last_became_peered":"2023-12-01T03:10:39.010389+0000","last_unstale":"2023-12-01T03:10:39.012565+0000","last_undegraded":"2023-12-01T03:10:39.012565+0000","last_fullsized":"2023-12-01T03:10:39.012565+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:53.973502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483145+0000","last_change":"2023-12-01T03:10:24.686553+0000","last_active":"2023-12-01T03:10:37.483145+0000","last_peered":"2023-12-01T03:10:37.483145+0000","last_clean":"2023-12-01T03:10:37.483145+0000","last_became_active":"2023-12-01T03:10:24.686136+0000","last_became_peered":"2023-12-01T03:10:24.686136+0000","last_unstale":"2023-12-01T03:10:37.483145+0000","last_undegraded":"2023-12-01T03:10:37.483145+0000","last_fullsized":"2023-12-01T03:10:37.483145+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:18.663097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472405+0000","last_change":"2023-12-01T03:10:25.182793+0000","last_active":"2023-12-01T03:10:37.472405+0000","last_peered":"2023-12-01T03:10:37.472405+0000","last_clean":"2023-12-01T03:10:37.472405+0000","last_became_active":"2023-12-01T03:10:25.182524+0000","last_became_peered":"2023-12-01T03:10:25.182524+0000","last_unstale":"2023-12-01T03:10:37.472405+0000","last_undegraded":"2023-12-01T03:10:37.472405+0000","last_fullsized":"2023-12-01T03:10:37.472405+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:25.370249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525096+0000","last_change":"2023-12-01T03:10:38.523078+0000","last_active":"2023-12-01T03:10:38.525096+0000","last_peered":"2023-12-01T03:10:38.525096+0000","last_clean":"2023-12-01T03:10:38.525096+0000","last_became_active":"2023-12-01T03:10:38.522603+0000","last_became_peered":"2023-12-01T03:10:38.522603+0000","last_unstale":"2023-12-01T03:10:38.525096+0000","last_undegraded":"2023-12-01T03:10:38.525096+0000","last_fullsized":"2023-12-01T03:10:38.525096+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:12.432439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.506778+0000","last_change":"2023-12-01T03:10:38.506778+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.506778+0000","last_undegraded":"2023-12-01T03:10:38.506778+0000","last_fullsized":"2023-12-01T03:10:38.506778+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1a","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480595+0000","last_change":"2023-12-01T03:10:24.695315+0000","last_active":"2023-12-01T03:10:37.480595+0000","last_peered":"2023-12-01T03:10:37.480595+0000","last_clean":"2023-12-01T03:10:37.480595+0000","last_became_active":"2023-12-01T03:10:24.694274+0000","last_became_peered":"2023-12-01T03:10:24.694274+0000","last_unstale":"2023-12-01T03:10:37.480595+0000","last_undegraded":"2023-12-01T03:10:37.480595+0000","last_fullsized":"2023-12-01T03:10:37.480595+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:19.256208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479974+0000","last_change":"2023-12-01T03:10:25.188350+0000","last_active":"2023-12-01T03:10:37.479974+0000","last_peered":"2023-12-01T03:10:37.479974+0000","last_clean":"2023-12-01T03:10:37.479974+0000","last_became_active":"2023-12-01T03:10:25.187212+0000","last_became_peered":"2023-12-01T03:10:25.187212+0000","last_unstale":"2023-12-01T03:10:37.479974+0000","last_undegraded":"2023-12-01T03:10:37.479974+0000","last_fullsized":"2023-12-01T03:10:37.479974+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.981897+0000","last_change":"2023-12-01T03:10:38.521794+0000","last_active":"2023-12-01T03:10:38.981897+0000","last_peered":"2023-12-01T03:10:38.981897+0000","last_clean":"2023-12-01T03:10:38.981897+0000","last_became_active":"2023-12-01T03:10:38.521546+0000","last_became_peered":"2023-12-01T03:10:38.521546+0000","last_unstale":"2023-12-01T03:10:38.981897+0000","last_undegraded":"2023-12-01T03:10:38.981897+0000","last_fullsized":"2023-12-01T03:10:38.981897+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:17.386276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.510675+0000","last_change":"2023-12-01T03:10:38.510675+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.510675+0000","last_undegraded":"2023-12-01T03:10:38.510675+0000","last_fullsized":"2023-12-01T03:10:38.510675+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:33:51.590745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480220+0000","last_change":"2023-12-01T03:10:24.695067+0000","last_active":"2023-12-01T03:10:37.480220+0000","last_peered":"2023-12-01T03:10:37.480220+0000","last_clean":"2023-12-01T03:10:37.480220+0000","last_became_active":"2023-12-01T03:10:24.694560+0000","last_became_peered":"2023-12-01T03:10:24.694560+0000","last_unstale":"2023-12-01T03:10:37.480220+0000","last_undegraded":"2023-12-01T03:10:37.480220+0000","last_fullsized":"2023-12-01T03:10:37.480220+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482278+0000","last_change":"2023-12-01T03:10:25.175874+0000","last_active":"2023-12-01T03:10:37.482278+0000","last_peered":"2023-12-01T03:10:37.482278+0000","last_clean":"2023-12-01T03:10:37.482278+0000","last_became_active":"2023-12-01T03:10:25.175584+0000","last_became_peered":"2023-12-01T03:10:25.175584+0000","last_unstale":"2023-12-01T03:10:37.482278+0000","last_undegraded":"2023-12-01T03:10:37.482278+0000","last_fullsized":"2023-12-01T03:10:37.482278+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:46.157610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519254+0000","last_change":"2023-12-01T03:10:38.515775+0000","last_active":"2023-12-01T03:10:38.519254+0000","last_peered":"2023-12-01T03:10:38.519254+0000","last_clean":"2023-12-01T03:10:38.519254+0000","last_became_active":"2023-12-01T03:10:38.515021+0000","last_became_peered":"2023-12-01T03:10:38.515021+0000","last_unstale":"2023-12-01T03:10:38.519254+0000","last_undegraded":"2023-12-01T03:10:38.519254+0000","last_fullsized":"2023-12-01T03:10:38.519254+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:35.878213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003601+0000","last_change":"2023-12-01T03:10:39.000307+0000","last_active":"2023-12-01T03:10:39.003601+0000","last_peered":"2023-12-01T03:10:39.003601+0000","last_clean":"2023-12-01T03:10:39.003601+0000","last_became_active":"2023-12-01T03:10:38.997156+0000","last_became_peered":"2023-12-01T03:10:38.997156+0000","last_unstale":"2023-12-01T03:10:39.003601+0000","last_undegraded":"2023-12-01T03:10:39.003601+0000","last_fullsized":"2023-12-01T03:10:39.003601+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:59.987290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515498+0000","last_change":"2023-12-01T03:10:38.513581+0000","last_active":"2023-12-01T03:10:38.515498+0000","last_peered":"2023-12-01T03:10:38.515498+0000","last_clean":"2023-12-01T03:10:38.515498+0000","last_became_active":"2023-12-01T03:10:38.513306+0000","last_became_peered":"2023-12-01T03:10:38.513306+0000","last_unstale":"2023-12-01T03:10:38.515498+0000","last_undegraded":"2023-12-01T03:10:38.515498+0000","last_fullsized":"2023-12-01T03:10:38.515498+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:13.957769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987848+0000","last_change":"2023-12-01T03:10:24.686931+0000","last_active":"2023-12-01T03:10:38.987848+0000","last_peered":"2023-12-01T03:10:38.987848+0000","last_clean":"2023-12-01T03:10:38.987848+0000","last_became_active":"2023-12-01T03:10:24.686642+0000","last_became_peered":"2023-12-01T03:10:24.686642+0000","last_unstale":"2023-12-01T03:10:38.987848+0000","last_undegraded":"2023-12-01T03:10:38.987848+0000","last_fullsized":"2023-12-01T03:10:38.987848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983478+0000","last_change":"2023-12-01T03:10:25.175999+0000","last_active":"2023-12-01T03:10:38.983478+0000","last_peered":"2023-12-01T03:10:38.983478+0000","last_clean":"2023-12-01T03:10:38.983478+0000","last_became_active":"2023-12-01T03:10:25.175764+0000","last_became_peered":"2023-12-01T03:10:25.175764+0000","last_unstale":"2023-12-01T03:10:38.983478+0000","last_undegraded":"2023-12-01T03:10:38.983478+0000","last_fullsized":"2023-12-01T03:10:38.983478+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:43.098776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012583+0000","last_change":"2023-12-01T03:10:39.009798+0000","last_active":"2023-12-01T03:10:39.012583+0000","last_peered":"2023-12-01T03:10:39.012583+0000","last_clean":"2023-12-01T03:10:39.012583+0000","last_became_active":"2023-12-01T03:10:39.009564+0000","last_became_peered":"2023-12-01T03:10:39.009564+0000","last_unstale":"2023-12-01T03:10:39.012583+0000","last_undegraded":"2023-12-01T03:10:39.012583+0000","last_fullsized":"2023-12-01T03:10:39.012583+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"217'4","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.011227+0000","last_change":"2023-12-01T03:10:38.520344+0000","last_active":"2023-12-01T03:10:39.011227+0000","last_peered":"2023-12-01T03:10:39.011227+0000","last_clean":"2023-12-01T03:10:39.011227+0000","last_became_active":"2023-12-01T03:10:38.520086+0000","last_became_peered":"2023-12-01T03:10:38.520086+0000","last_unstale":"2023-12-01T03:10:39.011227+0000","last_undegraded":"2023-12-01T03:10:39.011227+0000","last_fullsized":"2023-12-01T03:10:39.011227+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:01.616973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480900+0000","last_change":"2023-12-01T03:10:24.700378+0000","last_active":"2023-12-01T03:10:37.480900+0000","last_peered":"2023-12-01T03:10:37.480900+0000","last_clean":"2023-12-01T03:10:37.480900+0000","last_became_active":"2023-12-01T03:10:24.700126+0000","last_became_peered":"2023-12-01T03:10:24.700126+0000","last_unstale":"2023-12-01T03:10:37.480900+0000","last_undegraded":"2023-12-01T03:10:37.480900+0000","last_fullsized":"2023-12-01T03:10:37.480900+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999828+0000","last_change":"2023-12-01T03:10:25.179131+0000","last_active":"2023-12-01T03:10:38.999828+0000","last_peered":"2023-12-01T03:10:38.999828+0000","last_clean":"2023-12-01T03:10:38.999828+0000","last_became_active":"2023-12-01T03:10:25.178708+0000","last_became_peered":"2023-12-01T03:10:25.178708+0000","last_unstale":"2023-12-01T03:10:38.999828+0000","last_undegraded":"2023-12-01T03:10:38.999828+0000","last_fullsized":"2023-12-01T03:10:38.999828+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:43.370389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513293+0000","last_change":"2023-12-01T03:10:38.513293+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513293+0000","last_undegraded":"2023-12-01T03:10:38.513293+0000","last_fullsized":"2023-12-01T03:10:38.513293+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:46:18.989087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1e","version":"213'1","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989529+0000","last_change":"2023-12-01T03:10:24.688812+0000","last_active":"2023-12-01T03:10:38.989529+0000","last_peered":"2023-12-01T03:10:38.989529+0000","last_clean":"2023-12-01T03:10:38.989529+0000","last_became_active":"2023-12-01T03:10:24.687577+0000","last_became_peered":"2023-12-01T03:10:24.687577+0000","last_unstale":"2023-12-01T03:10:38.989529+0000","last_undegraded":"2023-12-01T03:10:38.989529+0000","last_fullsized":"2023-12-01T03:10:38.989529+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492668+0000","last_change":"2023-12-01T03:10:25.180335+0000","last_active":"2023-12-01T03:10:37.492668+0000","last_peered":"2023-12-01T03:10:37.492668+0000","last_clean":"2023-12-01T03:10:37.492668+0000","last_became_active":"2023-12-01T03:10:25.180020+0000","last_became_peered":"2023-12-01T03:10:25.180020+0000","last_unstale":"2023-12-01T03:10:37.492668+0000","last_undegraded":"2023-12-01T03:10:37.492668+0000","last_fullsized":"2023-12-01T03:10:37.492668+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"217'2","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.006339+0000","last_change":"2023-12-01T03:10:38.519061+0000","last_active":"2023-12-01T03:10:39.006339+0000","last_peered":"2023-12-01T03:10:39.006339+0000","last_clean":"2023-12-01T03:10:39.006339+0000","last_became_active":"2023-12-01T03:10:38.518091+0000","last_became_peered":"2023-12-01T03:10:38.518091+0000","last_unstale":"2023-12-01T03:10:39.006339+0000","last_undegraded":"2023-12-01T03:10:39.006339+0000","last_fullsized":"2023-12-01T03:10:39.006339+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:42.963999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512195+0000","last_change":"2023-12-01T03:10:38.512195+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512195+0000","last_undegraded":"2023-12-01T03:10:38.512195+0000","last_fullsized":"2023-12-01T03:10:38.512195+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"213'4","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489298+0000","last_change":"2023-12-01T03:10:24.697099+0000","last_active":"2023-12-01T03:10:37.489298+0000","last_peered":"2023-12-01T03:10:37.489298+0000","last_clean":"2023-12-01T03:10:37.489298+0000","last_became_active":"2023-12-01T03:10:24.696783+0000","last_became_peered":"2023-12-01T03:10:24.696783+0000","last_unstale":"2023-12-01T03:10:37.489298+0000","last_undegraded":"2023-12-01T03:10:37.489298+0000","last_fullsized":"2023-12-01T03:10:37.489298+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:07.101180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479580+0000","last_change":"2023-12-01T03:10:25.191865+0000","last_active":"2023-12-01T03:10:37.479580+0000","last_peered":"2023-12-01T03:10:37.479580+0000","last_clean":"2023-12-01T03:10:37.479580+0000","last_became_active":"2023-12-01T03:10:25.189277+0000","last_became_peered":"2023-12-01T03:10:25.189277+0000","last_unstale":"2023-12-01T03:10:37.479580+0000","last_undegraded":"2023-12-01T03:10:37.479580+0000","last_fullsized":"2023-12-01T03:10:37.479580+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521580+0000","last_change":"2023-12-01T03:10:38.517851+0000","last_active":"2023-12-01T03:10:38.521580+0000","last_peered":"2023-12-01T03:10:38.521580+0000","last_clean":"2023-12-01T03:10:38.521580+0000","last_became_active":"2023-12-01T03:10:38.517507+0000","last_became_peered":"2023-12-01T03:10:38.517507+0000","last_unstale":"2023-12-01T03:10:38.521580+0000","last_undegraded":"2023-12-01T03:10:38.521580+0000","last_fullsized":"2023-12-01T03:10:38.521580+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:22.592854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008442+0000","last_change":"2023-12-01T03:10:39.005982+0000","last_active":"2023-12-01T03:10:39.008442+0000","last_peered":"2023-12-01T03:10:39.008442+0000","last_clean":"2023-12-01T03:10:39.008442+0000","last_became_active":"2023-12-01T03:10:39.005729+0000","last_became_peered":"2023-12-01T03:10:39.005729+0000","last_unstale":"2023-12-01T03:10:39.008442+0000","last_undegraded":"2023-12-01T03:10:39.008442+0000","last_fullsized":"2023-12-01T03:10:39.008442+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:36.187885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478313+0000","last_change":"2023-12-01T03:10:25.185385+0000","last_active":"2023-12-01T03:10:37.478313+0000","last_peered":"2023-12-01T03:10:37.478313+0000","last_clean":"2023-12-01T03:10:37.478313+0000","last_became_active":"2023-12-01T03:10:25.185098+0000","last_became_peered":"2023-12-01T03:10:25.185098+0000","last_unstale":"2023-12-01T03:10:37.478313+0000","last_undegraded":"2023-12-01T03:10:37.478313+0000","last_fullsized":"2023-12-01T03:10:37.478313+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500232+0000","last_change":"2023-12-01T03:10:38.500232+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500232+0000","last_undegraded":"2023-12-01T03:10:38.500232+0000","last_fullsized":"2023-12-01T03:10:38.500232+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478702+0000","last_change":"2023-12-01T03:10:25.186126+0000","last_active":"2023-12-01T03:10:37.478702+0000","last_peered":"2023-12-01T03:10:37.478702+0000","last_clean":"2023-12-01T03:10:37.478702+0000","last_became_active":"2023-12-01T03:10:25.185741+0000","last_became_peered":"2023-12-01T03:10:25.185741+0000","last_unstale":"2023-12-01T03:10:37.478702+0000","last_undegraded":"2023-12-01T03:10:37.478702+0000","last_fullsized":"2023-12-01T03:10:37.478702+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.501528+0000","last_change":"2023-12-01T03:10:38.501528+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.501528+0000","last_undegraded":"2023-12-01T03:10:38.501528+0000","last_fullsized":"2023-12-01T03:10:38.501528+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985671+0000","last_change":"2023-12-01T03:10:25.175090+0000","last_active":"2023-12-01T03:10:38.985671+0000","last_peered":"2023-12-01T03:10:38.985671+0000","last_clean":"2023-12-01T03:10:38.985671+0000","last_became_active":"2023-12-01T03:10:25.174835+0000","last_became_peered":"2023-12-01T03:10:25.174835+0000","last_unstale":"2023-12-01T03:10:38.985671+0000","last_undegraded":"2023-12-01T03:10:38.985671+0000","last_fullsized":"2023-12-01T03:10:38.985671+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.505382+0000","last_change":"2023-12-01T03:10:38.505382+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.505382+0000","last_undegraded":"2023-12-01T03:10:38.505382+0000","last_fullsized":"2023-12-01T03:10:38.505382+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:52.701542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987378+0000","last_change":"2023-12-01T03:10:25.178098+0000","last_active":"2023-12-01T03:10:38.987378+0000","last_peered":"2023-12-01T03:10:38.987378+0000","last_clean":"2023-12-01T03:10:38.987378+0000","last_became_active":"2023-12-01T03:10:25.177031+0000","last_became_peered":"2023-12-01T03:10:25.177031+0000","last_unstale":"2023-12-01T03:10:38.987378+0000","last_undegraded":"2023-12-01T03:10:38.987378+0000","last_fullsized":"2023-12-01T03:10:38.987378+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.496510+0000","last_change":"2023-12-01T03:10:38.496510+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.496510+0000","last_undegraded":"2023-12-01T03:10:38.496510+0000","last_fullsized":"2023-12-01T03:10:38.496510+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491934+0000","last_change":"2023-12-01T03:10:25.176284+0000","last_active":"2023-12-01T03:10:37.491934+0000","last_peered":"2023-12-01T03:10:37.491934+0000","last_clean":"2023-12-01T03:10:37.491934+0000","last_became_active":"2023-12-01T03:10:25.175494+0000","last_became_peered":"2023-12-01T03:10:25.175494+0000","last_unstale":"2023-12-01T03:10:37.491934+0000","last_undegraded":"2023-12-01T03:10:37.491934+0000","last_fullsized":"2023-12-01T03:10:37.491934+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008362+0000","last_change":"2023-12-01T03:10:39.006064+0000","last_active":"2023-12-01T03:10:39.008362+0000","last_peered":"2023-12-01T03:10:39.008362+0000","last_clean":"2023-12-01T03:10:39.008362+0000","last_became_active":"2023-12-01T03:10:39.005720+0000","last_became_peered":"2023-12-01T03:10:39.005720+0000","last_unstale":"2023-12-01T03:10:39.008362+0000","last_undegraded":"2023-12-01T03:10:39.008362+0000","last_fullsized":"2023-12-01T03:10:39.008362+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:02.628798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488821+0000","last_change":"2023-12-01T03:10:25.178072+0000","last_active":"2023-12-01T03:10:37.488821+0000","last_peered":"2023-12-01T03:10:37.488821+0000","last_clean":"2023-12-01T03:10:37.488821+0000","last_became_active":"2023-12-01T03:10:25.177778+0000","last_became_peered":"2023-12-01T03:10:25.177778+0000","last_unstale":"2023-12-01T03:10:37.488821+0000","last_undegraded":"2023-12-01T03:10:37.488821+0000","last_fullsized":"2023-12-01T03:10:37.488821+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:48.799119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.496580+0000","last_change":"2023-12-01T03:10:38.496580+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.496580+0000","last_undegraded":"2023-12-01T03:10:38.496580+0000","last_fullsized":"2023-12-01T03:10:38.496580+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989375+0000","last_change":"2023-12-01T03:10:25.181469+0000","last_active":"2023-12-01T03:10:38.989375+0000","last_peered":"2023-12-01T03:10:38.989375+0000","last_clean":"2023-12-01T03:10:38.989375+0000","last_became_active":"2023-12-01T03:10:25.180646+0000","last_became_peered":"2023-12-01T03:10:25.180646+0000","last_unstale":"2023-12-01T03:10:38.989375+0000","last_undegraded":"2023-12-01T03:10:38.989375+0000","last_fullsized":"2023-12-01T03:10:38.989375+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:53.001661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008512+0000","last_change":"2023-12-01T03:10:39.006804+0000","last_active":"2023-12-01T03:10:39.008512+0000","last_peered":"2023-12-01T03:10:39.008512+0000","last_clean":"2023-12-01T03:10:39.008512+0000","last_became_active":"2023-12-01T03:10:39.006552+0000","last_became_peered":"2023-12-01T03:10:39.006552+0000","last_unstale":"2023-12-01T03:10:39.008512+0000","last_undegraded":"2023-12-01T03:10:39.008512+0000","last_fullsized":"2023-12-01T03:10:39.008512+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.992143+0000","last_change":"2023-12-01T03:10:25.179679+0000","last_active":"2023-12-01T03:10:38.992143+0000","last_peered":"2023-12-01T03:10:38.992143+0000","last_clean":"2023-12-01T03:10:38.992143+0000","last_became_active":"2023-12-01T03:10:25.179442+0000","last_became_peered":"2023-12-01T03:10:25.179442+0000","last_unstale":"2023-12-01T03:10:38.992143+0000","last_undegraded":"2023-12-01T03:10:38.992143+0000","last_fullsized":"2023-12-01T03:10:38.992143+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:57:25.728865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500994+0000","last_change":"2023-12-01T03:10:38.500994+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500994+0000","last_undegraded":"2023-12-01T03:10:38.500994+0000","last_fullsized":"2023-12-01T03:10:38.500994+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478175+0000","last_change":"2023-12-01T03:10:25.175356+0000","last_active":"2023-12-01T03:10:37.478175+0000","last_peered":"2023-12-01T03:10:37.478175+0000","last_clean":"2023-12-01T03:10:37.478175+0000","last_became_active":"2023-12-01T03:10:25.175086+0000","last_became_peered":"2023-12-01T03:10:25.175086+0000","last_unstale":"2023-12-01T03:10:37.478175+0000","last_undegraded":"2023-12-01T03:10:37.478175+0000","last_fullsized":"2023-12-01T03:10:37.478175+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.000224+0000","last_change":"2023-12-01T03:10:38.998643+0000","last_active":"2023-12-01T03:10:39.000224+0000","last_peered":"2023-12-01T03:10:39.000224+0000","last_clean":"2023-12-01T03:10:39.000224+0000","last_became_active":"2023-12-01T03:10:38.998393+0000","last_became_peered":"2023-12-01T03:10:38.998393+0000","last_unstale":"2023-12-01T03:10:39.000224+0000","last_undegraded":"2023-12-01T03:10:39.000224+0000","last_fullsized":"2023-12-01T03:10:39.000224+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:38.199716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491490+0000","last_change":"2023-12-01T03:10:25.181678+0000","last_active":"2023-12-01T03:10:37.491490+0000","last_peered":"2023-12-01T03:10:37.491490+0000","last_clean":"2023-12-01T03:10:37.491490+0000","last_became_active":"2023-12-01T03:10:25.181430+0000","last_became_peered":"2023-12-01T03:10:25.181430+0000","last_unstale":"2023-12-01T03:10:37.491490+0000","last_undegraded":"2023-12-01T03:10:37.491490+0000","last_fullsized":"2023-12-01T03:10:37.491490+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.506833+0000","last_change":"2023-12-01T03:10:38.506833+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.506833+0000","last_undegraded":"2023-12-01T03:10:38.506833+0000","last_fullsized":"2023-12-01T03:10:38.506833+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:57:07.895399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480143+0000","last_change":"2023-12-01T03:10:25.188998+0000","last_active":"2023-12-01T03:10:37.480143+0000","last_peered":"2023-12-01T03:10:37.480143+0000","last_clean":"2023-12-01T03:10:37.480143+0000","last_became_active":"2023-12-01T03:10:25.188657+0000","last_became_peered":"2023-12-01T03:10:25.188657+0000","last_unstale":"2023-12-01T03:10:37.480143+0000","last_undegraded":"2023-12-01T03:10:37.480143+0000","last_fullsized":"2023-12-01T03:10:37.480143+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:23:58.752943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003091+0000","last_change":"2023-12-01T03:10:38.997151+0000","last_active":"2023-12-01T03:10:39.003091+0000","last_peered":"2023-12-01T03:10:39.003091+0000","last_clean":"2023-12-01T03:10:39.003091+0000","last_became_active":"2023-12-01T03:10:38.989885+0000","last_became_peered":"2023-12-01T03:10:38.989885+0000","last_unstale":"2023-12-01T03:10:39.003091+0000","last_undegraded":"2023-12-01T03:10:39.003091+0000","last_fullsized":"2023-12-01T03:10:39.003091+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:25:51.393448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485796+0000","last_change":"2023-12-01T03:10:25.174568+0000","last_active":"2023-12-01T03:10:37.485796+0000","last_peered":"2023-12-01T03:10:37.485796+0000","last_clean":"2023-12-01T03:10:37.485796+0000","last_became_active":"2023-12-01T03:10:25.174229+0000","last_became_peered":"2023-12-01T03:10:25.174229+0000","last_unstale":"2023-12-01T03:10:37.485796+0000","last_undegraded":"2023-12-01T03:10:37.485796+0000","last_fullsized":"2023-12-01T03:10:37.485796+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:13.528118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512138+0000","last_change":"2023-12-01T03:10:38.512138+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512138+0000","last_undegraded":"2023-12-01T03:10:38.512138+0000","last_fullsized":"2023-12-01T03:10:38.512138+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481725+0000","last_change":"2023-12-01T03:10:25.189264+0000","last_active":"2023-12-01T03:10:37.481725+0000","last_peered":"2023-12-01T03:10:37.481725+0000","last_clean":"2023-12-01T03:10:37.481725+0000","last_became_active":"2023-12-01T03:10:25.189007+0000","last_became_peered":"2023-12-01T03:10:25.189007+0000","last_unstale":"2023-12-01T03:10:37.481725+0000","last_undegraded":"2023-12-01T03:10:37.481725+0000","last_fullsized":"2023-12-01T03:10:37.481725+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:14.783134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511167+0000","last_change":"2023-12-01T03:10:38.511167+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511167+0000","last_undegraded":"2023-12-01T03:10:38.511167+0000","last_fullsized":"2023-12-01T03:10:38.511167+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:38:07.828693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488698+0000","last_change":"2023-12-01T03:10:25.180992+0000","last_active":"2023-12-01T03:10:37.488698+0000","last_peered":"2023-12-01T03:10:37.488698+0000","last_clean":"2023-12-01T03:10:37.488698+0000","last_became_active":"2023-12-01T03:10:25.180712+0000","last_became_peered":"2023-12-01T03:10:25.180712+0000","last_unstale":"2023-12-01T03:10:37.488698+0000","last_undegraded":"2023-12-01T03:10:37.488698+0000","last_fullsized":"2023-12-01T03:10:37.488698+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:43.827830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507312+0000","last_change":"2023-12-01T03:10:38.507312+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507312+0000","last_undegraded":"2023-12-01T03:10:38.507312+0000","last_fullsized":"2023-12-01T03:10:38.507312+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:58:42.639310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485503+0000","last_change":"2023-12-01T03:10:25.182676+0000","last_active":"2023-12-01T03:10:37.485503+0000","last_peered":"2023-12-01T03:10:37.485503+0000","last_clean":"2023-12-01T03:10:37.485503+0000","last_became_active":"2023-12-01T03:10:25.182429+0000","last_became_peered":"2023-12-01T03:10:25.182429+0000","last_unstale":"2023-12-01T03:10:37.485503+0000","last_undegraded":"2023-12-01T03:10:37.485503+0000","last_fullsized":"2023-12-01T03:10:37.485503+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508726+0000","last_change":"2023-12-01T03:10:38.508726+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508726+0000","last_undegraded":"2023-12-01T03:10:38.508726+0000","last_fullsized":"2023-12-01T03:10:38.508726+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:54.829130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483923+0000","last_change":"2023-12-01T03:10:25.173100+0000","last_active":"2023-12-01T03:10:37.483923+0000","last_peered":"2023-12-01T03:10:37.483923+0000","last_clean":"2023-12-01T03:10:37.483923+0000","last_became_active":"2023-12-01T03:10:25.172808+0000","last_became_peered":"2023-12-01T03:10:25.172808+0000","last_unstale":"2023-12-01T03:10:37.483923+0000","last_undegraded":"2023-12-01T03:10:37.483923+0000","last_fullsized":"2023-12-01T03:10:37.483923+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507042+0000","last_change":"2023-12-01T03:10:38.507042+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507042+0000","last_undegraded":"2023-12-01T03:10:38.507042+0000","last_fullsized":"2023-12-01T03:10:38.507042+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:48.887816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483463+0000","last_change":"2023-12-01T03:10:25.189550+0000","last_active":"2023-12-01T03:10:37.483463+0000","last_peered":"2023-12-01T03:10:37.483463+0000","last_clean":"2023-12-01T03:10:37.483463+0000","last_became_active":"2023-12-01T03:10:25.189001+0000","last_became_peered":"2023-12-01T03:10:25.189001+0000","last_unstale":"2023-12-01T03:10:37.483463+0000","last_undegraded":"2023-12-01T03:10:37.483463+0000","last_fullsized":"2023-12-01T03:10:37.483463+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003543+0000","last_change":"2023-12-01T03:10:38.997977+0000","last_active":"2023-12-01T03:10:39.003543+0000","last_peered":"2023-12-01T03:10:39.003543+0000","last_clean":"2023-12-01T03:10:39.003543+0000","last_became_active":"2023-12-01T03:10:38.997683+0000","last_became_peered":"2023-12-01T03:10:38.997683+0000","last_unstale":"2023-12-01T03:10:39.003543+0000","last_undegraded":"2023-12-01T03:10:39.003543+0000","last_fullsized":"2023-12-01T03:10:39.003543+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:27:00.020245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472732+0000","last_change":"2023-12-01T03:10:25.170590+0000","last_active":"2023-12-01T03:10:37.472732+0000","last_peered":"2023-12-01T03:10:37.472732+0000","last_clean":"2023-12-01T03:10:37.472732+0000","last_became_active":"2023-12-01T03:10:25.170334+0000","last_became_peered":"2023-12-01T03:10:25.170334+0000","last_unstale":"2023-12-01T03:10:37.472732+0000","last_undegraded":"2023-12-01T03:10:37.472732+0000","last_fullsized":"2023-12-01T03:10:37.472732+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:35.058136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513592+0000","last_change":"2023-12-01T03:10:38.513592+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513592+0000","last_undegraded":"2023-12-01T03:10:38.513592+0000","last_fullsized":"2023-12-01T03:10:38.513592+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986339+0000","last_change":"2023-12-01T03:10:25.173081+0000","last_active":"2023-12-01T03:10:38.986339+0000","last_peered":"2023-12-01T03:10:38.986339+0000","last_clean":"2023-12-01T03:10:38.986339+0000","last_became_active":"2023-12-01T03:10:25.172845+0000","last_became_peered":"2023-12-01T03:10:25.172845+0000","last_unstale":"2023-12-01T03:10:38.986339+0000","last_undegraded":"2023-12-01T03:10:38.986339+0000","last_fullsized":"2023-12-01T03:10:38.986339+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:18.173873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512530+0000","last_change":"2023-12-01T03:10:38.512530+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512530+0000","last_undegraded":"2023-12-01T03:10:38.512530+0000","last_fullsized":"2023-12-01T03:10:38.512530+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:24:41.123796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472369+0000","last_change":"2023-12-01T03:10:25.178225+0000","last_active":"2023-12-01T03:10:37.472369+0000","last_peered":"2023-12-01T03:10:37.472369+0000","last_clean":"2023-12-01T03:10:37.472369+0000","last_became_active":"2023-12-01T03:10:25.177541+0000","last_became_peered":"2023-12-01T03:10:25.177541+0000","last_unstale":"2023-12-01T03:10:37.472369+0000","last_undegraded":"2023-12-01T03:10:37.472369+0000","last_fullsized":"2023-12-01T03:10:37.472369+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:18.470068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.516591+0000","last_change":"2023-12-01T03:10:38.516591+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.516591+0000","last_undegraded":"2023-12-01T03:10:38.516591+0000","last_fullsized":"2023-12-01T03:10:38.516591+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987497+0000","last_change":"2023-12-01T03:10:25.172316+0000","last_active":"2023-12-01T03:10:38.987497+0000","last_peered":"2023-12-01T03:10:38.987497+0000","last_clean":"2023-12-01T03:10:38.987497+0000","last_became_active":"2023-12-01T03:10:25.169042+0000","last_became_peered":"2023-12-01T03:10:25.169042+0000","last_unstale":"2023-12-01T03:10:38.987497+0000","last_undegraded":"2023-12-01T03:10:38.987497+0000","last_fullsized":"2023-12-01T03:10:38.987497+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507019+0000","last_change":"2023-12-01T03:10:38.507019+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507019+0000","last_undegraded":"2023-12-01T03:10:38.507019+0000","last_fullsized":"2023-12-01T03:10:38.507019+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473026+0000","last_change":"2023-12-01T03:10:25.181052+0000","last_active":"2023-12-01T03:10:37.473026+0000","last_peered":"2023-12-01T03:10:37.473026+0000","last_clean":"2023-12-01T03:10:37.473026+0000","last_became_active":"2023-12-01T03:10:25.180810+0000","last_became_peered":"2023-12-01T03:10:25.180810+0000","last_unstale":"2023-12-01T03:10:37.473026+0000","last_undegraded":"2023-12-01T03:10:37.473026+0000","last_fullsized":"2023-12-01T03:10:37.473026+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:35.663128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.013053+0000","last_change":"2023-12-01T03:10:39.011983+0000","last_active":"2023-12-01T03:10:39.013053+0000","last_peered":"2023-12-01T03:10:39.013053+0000","last_clean":"2023-12-01T03:10:39.013053+0000","last_became_active":"2023-12-01T03:10:39.011721+0000","last_became_peered":"2023-12-01T03:10:39.011721+0000","last_unstale":"2023-12-01T03:10:39.013053+0000","last_undegraded":"2023-12-01T03:10:39.013053+0000","last_fullsized":"2023-12-01T03:10:39.013053+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492510+0000","last_change":"2023-12-01T03:10:25.177089+0000","last_active":"2023-12-01T03:10:37.492510+0000","last_peered":"2023-12-01T03:10:37.492510+0000","last_clean":"2023-12-01T03:10:37.492510+0000","last_became_active":"2023-12-01T03:10:25.176742+0000","last_became_peered":"2023-12-01T03:10:25.176742+0000","last_unstale":"2023-12-01T03:10:37.492510+0000","last_undegraded":"2023-12-01T03:10:37.492510+0000","last_fullsized":"2023-12-01T03:10:37.492510+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513196+0000","last_change":"2023-12-01T03:10:38.513196+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513196+0000","last_undegraded":"2023-12-01T03:10:38.513196+0000","last_fullsized":"2023-12-01T03:10:38.513196+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:19.824447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.992054+0000","last_change":"2023-12-01T03:10:25.172560+0000","last_active":"2023-12-01T03:10:38.992054+0000","last_peered":"2023-12-01T03:10:38.992054+0000","last_clean":"2023-12-01T03:10:38.992054+0000","last_became_active":"2023-12-01T03:10:25.172005+0000","last_became_peered":"2023-12-01T03:10:25.172005+0000","last_unstale":"2023-12-01T03:10:38.992054+0000","last_undegraded":"2023-12-01T03:10:38.992054+0000","last_fullsized":"2023-12-01T03:10:38.992054+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:44.541671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.515497+0000","last_change":"2023-12-01T03:10:38.515497+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.515497+0000","last_undegraded":"2023-12-01T03:10:38.515497+0000","last_fullsized":"2023-12-01T03:10:38.515497+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:10:20.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.477932+0000","last_change":"2023-12-01T03:10:25.185623+0000","last_active":"2023-12-01T03:10:37.477932+0000","last_peered":"2023-12-01T03:10:37.477932+0000","last_clean":"2023-12-01T03:10:37.477932+0000","last_became_active":"2023-12-01T03:10:25.185311+0000","last_became_peered":"2023-12-01T03:10:25.185311+0000","last_unstale":"2023-12-01T03:10:37.477932+0000","last_undegraded":"2023-12-01T03:10:37.477932+0000","last_fullsized":"2023-12-01T03:10:37.477932+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508313+0000","last_change":"2023-12-01T03:10:38.508313+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508313+0000","last_undegraded":"2023-12-01T03:10:38.508313+0000","last_fullsized":"2023-12-01T03:10:38.508313+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:38.801763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483215+0000","last_change":"2023-12-01T03:10:25.186300+0000","last_active":"2023-12-01T03:10:37.483215+0000","last_peered":"2023-12-01T03:10:37.483215+0000","last_clean":"2023-12-01T03:10:37.483215+0000","last_became_active":"2023-12-01T03:10:25.185030+0000","last_became_peered":"2023-12-01T03:10:25.185030+0000","last_unstale":"2023-12-01T03:10:37.483215+0000","last_undegraded":"2023-12-01T03:10:37.483215+0000","last_fullsized":"2023-12-01T03:10:37.483215+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.501552+0000","last_change":"2023-12-01T03:10:38.501552+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.501552+0000","last_undegraded":"2023-12-01T03:10:38.501552+0000","last_fullsized":"2023-12-01T03:10:38.501552+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482005+0000","last_change":"2023-12-01T03:10:25.182648+0000","last_active":"2023-12-01T03:10:37.482005+0000","last_peered":"2023-12-01T03:10:37.482005+0000","last_clean":"2023-12-01T03:10:37.482005+0000","last_became_active":"2023-12-01T03:10:25.182343+0000","last_became_peered":"2023-12-01T03:10:25.182343+0000","last_unstale":"2023-12-01T03:10:37.482005+0000","last_undegraded":"2023-12-01T03:10:37.482005+0000","last_fullsized":"2023-12-01T03:10:37.482005+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:55.408381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.514336+0000","last_change":"2023-12-01T03:10:38.514336+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.514336+0000","last_undegraded":"2023-12-01T03:10:38.514336+0000","last_fullsized":"2023-12-01T03:10:38.514336+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479210+0000","last_change":"2023-12-01T03:10:25.191253+0000","last_active":"2023-12-01T03:10:37.479210+0000","last_peered":"2023-12-01T03:10:37.479210+0000","last_clean":"2023-12-01T03:10:37.479210+0000","last_became_active":"2023-12-01T03:10:25.188485+0000","last_became_peered":"2023-12-01T03:10:25.188485+0000","last_unstale":"2023-12-01T03:10:37.479210+0000","last_undegraded":"2023-12-01T03:10:37.479210+0000","last_fullsized":"2023-12-01T03:10:37.479210+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500633+0000","last_change":"2023-12-01T03:10:38.500633+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500633+0000","last_undegraded":"2023-12-01T03:10:38.500633+0000","last_fullsized":"2023-12-01T03:10:38.500633+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999753+0000","last_change":"2023-12-01T03:10:25.171460+0000","last_active":"2023-12-01T03:10:38.999753+0000","last_peered":"2023-12-01T03:10:38.999753+0000","last_clean":"2023-12-01T03:10:38.999753+0000","last_became_active":"2023-12-01T03:10:25.171148+0000","last_became_peered":"2023-12-01T03:10:25.171148+0000","last_unstale":"2023-12-01T03:10:38.999753+0000","last_undegraded":"2023-12-01T03:10:38.999753+0000","last_fullsized":"2023-12-01T03:10:38.999753+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:01:37.278740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.516041+0000","last_change":"2023-12-01T03:10:38.516041+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.516041+0000","last_undegraded":"2023-12-01T03:10:38.516041+0000","last_fullsized":"2023-12-01T03:10:38.516041+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:07:32.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488617+0000","last_change":"2023-12-01T03:10:25.181720+0000","last_active":"2023-12-01T03:10:37.488617+0000","last_peered":"2023-12-01T03:10:37.488617+0000","last_clean":"2023-12-01T03:10:37.488617+0000","last_became_active":"2023-12-01T03:10:25.180144+0000","last_became_peered":"2023-12-01T03:10:25.180144+0000","last_unstale":"2023-12-01T03:10:37.488617+0000","last_undegraded":"2023-12-01T03:10:37.488617+0000","last_fullsized":"2023-12-01T03:10:37.488617+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:53.999589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007029+0000","last_change":"2023-12-01T03:10:39.004848+0000","last_active":"2023-12-01T03:10:39.007029+0000","last_peered":"2023-12-01T03:10:39.007029+0000","last_clean":"2023-12-01T03:10:39.007029+0000","last_became_active":"2023-12-01T03:10:39.004604+0000","last_became_peered":"2023-12-01T03:10:39.004604+0000","last_unstale":"2023-12-01T03:10:39.007029+0000","last_undegraded":"2023-12-01T03:10:39.007029+0000","last_fullsized":"2023-12-01T03:10:39.007029+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:44.253867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983363+0000","last_change":"2023-12-01T03:10:25.181088+0000","last_active":"2023-12-01T03:10:38.983363+0000","last_peered":"2023-12-01T03:10:38.983363+0000","last_clean":"2023-12-01T03:10:38.983363+0000","last_became_active":"2023-12-01T03:10:25.180741+0000","last_became_peered":"2023-12-01T03:10:25.180741+0000","last_unstale":"2023-12-01T03:10:38.983363+0000","last_undegraded":"2023-12-01T03:10:38.983363+0000","last_fullsized":"2023-12-01T03:10:38.983363+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:26.321285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513800+0000","last_change":"2023-12-01T03:10:38.513800+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513800+0000","last_undegraded":"2023-12-01T03:10:38.513800+0000","last_fullsized":"2023-12-01T03:10:38.513800+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472688+0000","last_change":"2023-12-01T03:10:25.171115+0000","last_active":"2023-12-01T03:10:37.472688+0000","last_peered":"2023-12-01T03:10:37.472688+0000","last_clean":"2023-12-01T03:10:37.472688+0000","last_became_active":"2023-12-01T03:10:25.170871+0000","last_became_peered":"2023-12-01T03:10:25.170871+0000","last_unstale":"2023-12-01T03:10:37.472688+0000","last_undegraded":"2023-12-01T03:10:37.472688+0000","last_fullsized":"2023-12-01T03:10:37.472688+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:24.370036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59936,"kb_used_data":2648,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311904,"statfs":{"total":96636764160,"available":96575389696,"internally_reserved":0,"allocated":2711552,"data_stored":1314756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0780000000000001,"15min":1.0820000000000001},"min":{"1min":0.78200000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5069999999999999,"5min":1.6000000000000001,"15min":1.98},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.093,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":0.67800000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.556,"5min":1.7,"15min":1.7},"last":1.331}]},{"osd":1,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.2889999999999999},"min":{"1min":0.80000000000000004,"5min":0.745,"15min":0.73099999999999998},"max":{"1min":1.603,"5min":2,"15min":2.1419999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3149999999999999,"15min":1.3320000000000001},"min":{"1min":1.137,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.514,"5min":1.8169999999999999,"15min":1.8700000000000001},"last":1.0700000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.157,"15min":1.1719999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.476,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.26,"15min":1.254},"min":{"1min":1.0549999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6020000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.268}]},{"osd":3,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2529999999999999,"15min":1.3049999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6559999999999999,"5min":1.712,"15min":2.431},"last":1.014},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.415,"15min":1.4199999999999999},"min":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.92900000000000005},"max":{"1min":1.5920000000000001,"5min":1.879,"15min":2.198},"last":1.109}]},{"osd":4,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3140000000000001,"15min":1.3400000000000001},"min":{"1min":1.006,"5min":0.82599999999999996,"15min":0.81200000000000006},"max":{"1min":1.901,"5min":1.901,"15min":2.5899999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":1.1240000000000001,"5min":0.92800000000000005,"15min":0.79500000000000004},"max":{"1min":1.9690000000000001,"5min":2.3279999999999998,"15min":2.569},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3009999999999999,"15min":1.3149999999999999},"min":{"1min":0.93600000000000005,"5min":0.85999999999999999,"15min":0.84599999999999997},"max":{"1min":2.0099999999999998,"5min":2.016,"15min":2.7269999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5800000000000001,"15min":1.589},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":0.93200000000000005},"max":{"1min":2.0569999999999999,"5min":2.206,"15min":2.484},"last":1.4570000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4359999999999999,"15min":1.446},"min":{"1min":1.0229999999999999,"5min":0.98599999999999999,"15min":0.82599999999999996},"max":{"1min":1.921,"5min":2.2789999999999999,"15min":2.532},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":1.8819999999999999,"15min":2.762},"last":1.2370000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575980,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41872,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329968,"statfs":{"total":96636764160,"available":96593887232,"internally_reserved":0,"allocated":2695168,"data_stored":1311929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0589999999999999,"15min":1.113},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.4039999999999999,"5min":1.6639999999999999,"15min":1.9219999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1160000000000001,"15min":1.1659999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.623,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":0.97699999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.976,"5min":1.976,"15min":2.0449999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1950000000000001,"15min":1.2210000000000001},"min":{"1min":0.627,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.694,"5min":1.694,"15min":1.835},"last":1.2390000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.163,"15min":1.2090000000000001},"min":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.976},"last":1.127},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2629999999999999,"15min":1.2849999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.167}]},{"osd":3,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2849999999999999,"15min":1.319},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8100000000000001,"5min":1.927,"15min":2.0289999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.3720000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2130000000000001,"15min":1.262},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.367,"15min":1.3540000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.522}]},{"osd":5,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3360000000000001,"15min":1.4039999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.272},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4790000000000001,"15min":1.484},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.6319999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4890000000000001,"15min":1.5249999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.452},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.556,"15min":1.579},"min":{"1min":0.95499999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2999999999999998,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.794}]}]},{"osd":5,"up_from":13,"seq":55834575979,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48216,"kb_used_data":3216,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323624,"statfs":{"total":96636764160,"available":96587390976,"internally_reserved":0,"allocated":3293184,"data_stored":1902811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.125},"min":{"1min":0.82299999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.548,"5min":1.776,"15min":1.8899999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.238,"5min":1.1639999999999999,"15min":1.163},"min":{"1min":0.89800000000000002,"5min":0.69399999999999995,"15min":0.67300000000000004},"max":{"1min":1.619,"5min":1.752,"15min":1.8009999999999999},"last":0.84999999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.133,"15min":1.1539999999999999},"min":{"1min":0.94699999999999995,"5min":0.81200000000000006,"15min":0.79900000000000004},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.696},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.272,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.88,"5min":0.79500000000000004,"15min":0.60899999999999999},"max":{"1min":1.619,"5min":1.6659999999999999,"15min":1.7649999999999999},"last":1.0289999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2390000000000001,"15min":1.246},"min":{"1min":0.93700000000000006,"5min":0.84499999999999997,"15min":0.81399999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.774},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.284,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.93899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.502,"5min":1.899,"15min":1.9219999999999999},"last":0.90200000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.306,"15min":1.3200000000000001},"min":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.93100000000000005},"max":{"1min":1.7929999999999999,"5min":1.8979999999999999,"15min":1.994},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.343,"15min":1.363},"min":{"1min":1.02,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.724,"5min":1.97,"15min":2.1859999999999999},"last":1.1080000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2789999999999999,"15min":1.254},"min":{"1min":0.96199999999999997,"5min":0.78000000000000003,"15min":0.74099999999999999},"max":{"1min":1.726,"5min":1.726,"15min":2.6040000000000001},"last":1.389},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.399,"15min":1.385},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.84599999999999997},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.931},"last":1.1579999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3879999999999999,"15min":1.3759999999999999},"min":{"1min":0.97599999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.9279999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.427,"15min":1.419},"min":{"1min":1.222,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":1.756,"5min":1.9219999999999999,"15min":2.577},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5349999999999999,"15min":1.5369999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.962,"5min":2.0790000000000002,"15min":3.1709999999999998},"last":1.294},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.575,"15min":1.5740000000000001},"min":{"1min":1.3720000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.831,"5min":2.101,"15min":2.8740000000000001},"last":1.343}]}]},{"osd":4,"up_from":13,"seq":55834575979,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2707456,"data_stored":1319053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.147,"15min":1.1339999999999999},"min":{"1min":0.73099999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.21,"15min":1.1990000000000001},"min":{"1min":0.82099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5840000000000001,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.006}]},{"osd":1,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.242,"15min":1.2589999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.613,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.228},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.82599999999999996},"max":{"1min":1.6080000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.5549999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.3240000000000001,"15min":1.2470000000000001},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.573,"5min":11.311,"15min":11.311},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.329,"15min":1.341},"min":{"1min":1.0680000000000001,"5min":1.024,"15min":0.94099999999999995},"max":{"1min":1.647,"5min":1.9379999999999999,"15min":1.9450000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3100000000000001,"15min":1.29},"min":{"1min":1.0529999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7749999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.365,"15min":1.3380000000000001},"min":{"1min":0.99199999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.599,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.5920000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.357,"15min":1.329},"min":{"1min":0.94799999999999995,"5min":0.91200000000000003,"15min":0.755},"max":{"1min":1.615,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.379,"15min":1.3120000000000001},"min":{"1min":0.94599999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7150000000000001,"5min":3.71,"15min":3.71},"last":1.2809999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3740000000000001,"15min":1.375},"min":{"1min":1.095,"5min":0.745,"15min":0.745},"max":{"1min":1.609,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.4850000000000001,"15min":1.48},"min":{"1min":1.335,"5min":1.0660000000000001,"15min":1.0149999999999999},"max":{"1min":1.738,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.52,"15min":1.486},"min":{"1min":1.157,"5min":0.95499999999999996,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6000000000000001,"15min":1.5800000000000001},"min":{"1min":1.3600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.806,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.7070000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575978,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51752,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320088,"statfs":{"total":96636764160,"available":96583770112,"internally_reserved":0,"allocated":2719744,"data_stored":1321894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91100000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.085},"min":{"1min":0.74199999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.571,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.163,"15min":1.171},"min":{"1min":0.64700000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.008,"15min":0.999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41099999999999998},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0900000000000001,"15min":1.103},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2230000000000001,"15min":1.252},"min":{"1min":0.80100000000000005,"5min":0.747,"15min":0.58599999999999997},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.57899999999999996}]},{"osd":4,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2989999999999999,"15min":1.2929999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.3},"min":{"1min":0.96799999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0229999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3240000000000001,"15min":1.331},"min":{"1min":0.86599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.373,"15min":1.3759999999999999},"min":{"1min":0.88400000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.78400000000000003},"max":{"1min":2.5649999999999999,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.54,"5min":1.5049999999999999,"15min":1.514},"min":{"1min":1.075,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.075}]},{"osd":7,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.516,"15min":1.5329999999999999},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.042},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.54,"15min":1.5309999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.878},"max":{"1min":2.4830000000000001,"5min":2.984,"15min":2.984},"last":0.83899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575978,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64028,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307812,"statfs":{"total":96636764160,"available":96571199488,"internally_reserved":0,"allocated":2707456,"data_stored":1319089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89300000000000002,"15min":0.89400000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1140000000000001,"5min":1.405,"15min":1.4730000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.056,"15min":1.0569999999999999},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5640000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0449999999999999,"15min":1.0589999999999999},"min":{"1min":0.66100000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.321,"5min":1.698,"15min":1.698},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.038,"5min":1.125,"15min":1.151},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.522,"5min":1.635,"15min":1.71},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0640000000000001,"15min":1.054},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.419,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1990000000000001,"15min":1.2170000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.516,"5min":1.6879999999999999,"15min":1.79},"last":1.01}]},{"osd":4,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":0.85199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.7569999999999999,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.133},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.714,"5min":1.714,"15min":1.835},"last":1.2390000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3919999999999999,"15min":1.391},"min":{"1min":0.88,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.8380000000000001,"15min":2.2040000000000002},"last":1.042},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.419,"15min":1.4179999999999999},"min":{"1min":1.095,"5min":1.004,"15min":1.004},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.016},"last":0.91800000000000004}]},{"osd":6,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.379,"15min":1.3819999999999999},"min":{"1min":0.89600000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.337},{"interface":"front","average":{"1min":1.46,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":1.087,"5min":0.879,"15min":0.879},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.327}]},{"osd":7,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5069999999999999,"15min":1.4910000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0979999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4330000000000001,"15min":1.458},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.012,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.0860000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575978,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48244,"kb_used_data":3244,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323596,"statfs":{"total":96636764160,"available":96587362304,"internally_reserved":0,"allocated":3321856,"data_stored":1924287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.879,"15min":0.90900000000000003},"min":{"1min":0.58199999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.099,"5min":0.99299999999999999,"15min":1.046},"min":{"1min":0.81000000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.5069999999999999,"5min":1.5660000000000001,"15min":1.681},"last":0.498}]},{"osd":2,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.56299999999999994,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.115,"15min":1.1419999999999999},"min":{"1min":0.75600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.581,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.258,"15min":1.3069999999999999},"min":{"1min":1.026,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5780000000000001,"5min":1.587,"15min":1.788},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.107,"15min":1.135},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.714,"5min":1.714,"15min":1.8049999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.21,"15min":1.2230000000000001},"min":{"1min":0.89900000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6859999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.389,"15min":1.409},"min":{"1min":1.0740000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.0269999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.85899999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.802,"5min":1.8520000000000001,"15min":2.101},"last":1.056},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.304,"15min":1.339},"min":{"1min":1.006,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.609,"5min":1.927,"15min":2.1659999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4019999999999999,"15min":1.4379999999999999},"min":{"1min":0.99399999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3799999999999999,"15min":1.4119999999999999},"min":{"1min":1.1779999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.768,"5min":2.0019999999999998,"15min":2.0209999999999999},"last":0.98499999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4870000000000001,"15min":1.5269999999999999},"min":{"1min":1.2549999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.911,"5min":2.0329999999999999,"15min":2.1190000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.524,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.091}]}]},{"osd":0,"up_from":13,"seq":55834575980,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":2676,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2740224,"data_stored":1335117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.86699999999999999,"15min":0.89400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1559999999999999,"5min":1.4379999999999999,"15min":1.994},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.04,"15min":1.056},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.427,"5min":1.583,"15min":1.6799999999999999},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0329999999999999,"15min":1.069},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.446,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.1020000000000001,"15min":1.1419999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.726,"5min":1.726,"15min":1.792},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0580000000000001,"15min":1.071},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.8839999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.655,"5min":1.655,"15min":2.6469999999999998},"last":0.78200000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3260000000000001,"15min":1.363},"min":{"1min":0.88400000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.274,"5min":1.284,"15min":1.3089999999999999},"min":{"1min":0.82099999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.677,"5min":1.74,"15min":2.3610000000000002},"last":1.0740000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.355,"15min":1.397},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5589999999999999,"5min":2.1030000000000002,"15min":2.7719999999999998},"last":1.054},{"interface":"front","average":{"1min":1.431,"5min":1.4099999999999999,"15min":1.4410000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.79600000000000004},"max":{"1min":1.9259999999999999,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":0.97499999999999998}]},{"osd":6,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.367,"15min":1.3859999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7649999999999999,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.458,"15min":1.484},"min":{"1min":1.048,"5min":0.85299999999999998,"15min":0.75800000000000001},"max":{"1min":1.736,"5min":1.9870000000000001,"15min":2.8759999999999999},"last":1.2569999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.49,"15min":1.524},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.841,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4750000000000001,"15min":1.4910000000000001},"min":{"1min":0.97299999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.0619999999999998,"5min":2.1190000000000002,"15min":2.431},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:10:43.923 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:10:44.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.164+0000 7fd7ab0e5640 1 Processor -- start 2023-12-01T03:10:44.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.164+0000 7fd7ab0e5640 1 -- start start 2023-12-01T03:10:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.165+0000 7fd7ab0e5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd7a415e2f0 0x7fd7a40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.165+0000 7fd7ab0e5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd7a40adf10 0x7fd7a40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.165+0000 7fd7ab0e5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a40add10 con 0x7fd7a406f2f0 2023-12-01T03:10:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.165+0000 7fd7ab0e5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a40c5760 con 0x7fd7a415e2f0 2023-12-01T03:10:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.165+0000 7fd7ab0e5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a415e6d0 con 0x7fd7a40adf10 2023-12-01T03:10:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.166+0000 7fd7a8e81640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd7a415e2f0 0x7fd7a40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.166+0000 7fd7a3fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd7a406f2f0 0x7fd7a40c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37078/0 (socket says 172.21.15.115:37078) 2023-12-01T03:10:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.166+0000 7fd7a3fff640 1 -- 172.21.15.115:0/420369255 learned_addr learned my addr 172.21.15.115:0/420369255 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:44.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.166+0000 7fd7a8e81640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd7a415e2f0 0x7fd7a40ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55500/0 (socket says 172.21.15.115:55500) 2023-12-01T03:10:44.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.166+0000 7fd7a9682640 1 --2- 172.21.15.115:0/420369255 >> v2:172.21.15.142:3300/0 conn(0x7fd7a40adf10 0x7fd7a40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.166+0000 7fd7a8e81640 1 -- 172.21.15.115:0/420369255 >> v2:172.21.15.142:3300/0 conn(0x7fd7a40adf10 msgr2=0x7fd7a40caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:44.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.166+0000 7fd7a8e81640 1 --2- 172.21.15.115:0/420369255 >> v2:172.21.15.142:3300/0 conn(0x7fd7a40adf10 0x7fd7a40caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.166+0000 7fd7a8e81640 1 -- 172.21.15.115:0/420369255 >> v1:172.21.15.115:6789/0 conn(0x7fd7a406f2f0 legacy=0x7fd7a40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.166+0000 7fd7a8e81640 1 -- 172.21.15.115:0/420369255 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40cf7b0 con 0x7fd7a415e2f0 2023-12-01T03:10:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.167+0000 7fd7a8e81640 1 --2- 172.21.15.115:0/420369255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 0x7fd7a40ca660 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40c8e80 tx=0x7fd79402b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c756806ee62b57 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.168+0000 7fd7a37fe640 1 -- 172.21.15.115:0/420369255 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794031520 con 0x7fd7a415e2f0 2023-12-01T03:10:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.168+0000 7fd7a37fe640 1 -- 172.21.15.115:0/420369255 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd794031680 con 0x7fd7a415e2f0 2023-12-01T03:10:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.168+0000 7fd7a37fe640 1 -- 172.21.15.115:0/420369255 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794031a10 con 0x7fd7a415e2f0 2023-12-01T03:10:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.168+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/420369255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 msgr2=0x7fd7a40ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.168+0000 7fd7ab0e5640 1 --2- 172.21.15.115:0/420369255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 0x7fd7a40ca660 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40c8e80 tx=0x7fd79402b7c0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.168+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/420369255 shutdown_connections 2023-12-01T03:10:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.168+0000 7fd7ab0e5640 1 --2- 172.21.15.115:0/420369255 >> v2:172.21.15.142:3300/0 conn(0x7fd7a40adf10 0x7fd7a40caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.168+0000 7fd7ab0e5640 1 --2- 172.21.15.115:0/420369255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 0x7fd7a40ca660 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.168+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/420369255 >> 172.21.15.115:0/420369255 conn(0x7fd7a40ce880 msgr2=0x7fd7a40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.168+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/420369255 shutdown_connections 2023-12-01T03:10:44.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.169+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/420369255 wait complete. 2023-12-01T03:10:44.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.170+0000 7fd7ab0e5640 1 Processor -- start 2023-12-01T03:10:44.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.170+0000 7fd7ab0e5640 1 -- start start 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.171+0000 7fd7ab0e5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a406f2f0 0x7fd7a40c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.171+0000 7fd7ab0e5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7a40adf10 0x7fd7a40c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.171+0000 7fd7ab0e5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 0x7fd7a40c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.171+0000 7fd7ab0e5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40c8430 con 0x7fd7a415e2f0 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.171+0000 7fd7ab0e5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40c8710 con 0x7fd7a406f2f0 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.171+0000 7fd7ab0e5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40c46b0 con 0x7fd7a40adf10 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a9682640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 0x7fd7a40c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a9682640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 0x7fd7a40c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55512/0 (socket says 172.21.15.115:55512) 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a9682640 1 -- 172.21.15.115:0/2902730733 learned_addr learned my addr 172.21.15.115:0/2902730733 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a8e81640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a406f2f0 0x7fd7a40c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a3fff640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7a40adf10 0x7fd7a40c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a9682640 1 -- 172.21.15.115:0/2902730733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7a40adf10 msgr2=0x7fd7a40c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a9682640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7a40adf10 0x7fd7a40c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a9682640 1 -- 172.21.15.115:0/2902730733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a406f2f0 msgr2=0x7fd7a40c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a9682640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a406f2f0 0x7fd7a40c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a9682640 1 -- 172.21.15.115:0/2902730733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd794009400 con 0x7fd7a415e2f0 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.172+0000 7fd7a3fff640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7a40adf10 0x7fd7a40c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.173+0000 7fd7a9682640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 0x7fd7a40c63d0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00d5d0 tx=0x7fd79c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.173+0000 7fd7a17fa640 1 -- 172.21.15.115:0/2902730733 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0125f0 con 0x7fd7a415e2f0 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.173+0000 7fd7a17fa640 1 -- 172.21.15.115:0/2902730733 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c012770 con 0x7fd7a415e2f0 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.173+0000 7fd7a17fa640 1 -- 172.21.15.115:0/2902730733 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c012b20 con 0x7fd7a415e2f0 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.173+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/2902730733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a40c4990 con 0x7fd7a415e2f0 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.173+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/2902730733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a40c5190 con 0x7fd7a415e2f0 2023-12-01T03:10:44.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.174+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/2902730733 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a406fda0 con 0x7fd7a415e2f0 2023-12-01T03:10:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.175+0000 7fd7a17fa640 1 -- 172.21.15.115:0/2902730733 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fd79c012c80 con 0x7fd7a415e2f0 2023-12-01T03:10:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.176+0000 7fd7a17fa640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd78806a550 0x7fd78806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.177+0000 7fd7a17fa640 1 -- 172.21.15.115:0/2902730733 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fd79c089360 con 0x7fd7a415e2f0 2023-12-01T03:10:44.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.177+0000 7fd7a8e81640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd78806a550 0x7fd78806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.177+0000 7fd7a8e81640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd78806a550 0x7fd78806ca10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd794035000 tx=0x7fd794030000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:44.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.181+0000 7fd7a17fa640 1 -- 172.21.15.115:0/2902730733 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd79c054420 con 0x7fd7a415e2f0 2023-12-01T03:10:44.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.365+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/2902730733 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7a40b43d0 con 0x7fd78806a550 2023-12-01T03:10:44.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.394+0000 7fd7a17fa640 1 -- 172.21.15.115:0/2902730733 <== mgr.10407 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+524098 (secure 0 0 0) 0x7fd794006e10 con 0x7fd78806a550 2023-12-01T03:10:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:44.408 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:10:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.399+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/2902730733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd78806a550 msgr2=0x7fd78806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.399+0000 7fd7ab0e5640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd78806a550 0x7fd78806ca10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd794035000 tx=0x7fd794030000 comp rx=0 tx=0).stop 2023-12-01T03:10:44.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.399+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/2902730733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 msgr2=0x7fd7a40c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.399+0000 7fd7ab0e5640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 0x7fd7a40c63d0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00d5d0 tx=0x7fd79c00da90 comp rx=0 tx=0).stop 2023-12-01T03:10:44.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.400+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/2902730733 shutdown_connections 2023-12-01T03:10:44.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.400+0000 7fd7ab0e5640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd78806a550 0x7fd78806ca10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.400+0000 7fd7ab0e5640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd7a40adf10 0x7fd7a40c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.400+0000 7fd7ab0e5640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a415e2f0 0x7fd7a40c63d0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.400+0000 7fd7ab0e5640 1 --2- 172.21.15.115:0/2902730733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7a406f2f0 0x7fd7a40c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.400+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/2902730733 >> 172.21.15.115:0/2902730733 conn(0x7fd7a40ce880 msgr2=0x7fd7a40aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:44.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.400+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/2902730733 shutdown_connections 2023-12-01T03:10:44.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.400+0000 7fd7ab0e5640 1 -- 172.21.15.115:0/2902730733 wait complete. 2023-12-01T03:10:44.418 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-12-01T03:10:43.800176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594553,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":593,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":373,"ondisk_log_size":373,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":250,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425752,"kb_used_data":22360,"kb_used_omap":0,"kb_used_meta":403328,"kb_avail":754548968,"statfs":{"total":773094113280,"available":772658143232,"internally_reserved":0,"allocated":22896640,"data_stored":11748936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":413007872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1569,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001845"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507374+0000","last_change":"2023-12-01T03:10:38.507374+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507374+0000","last_undegraded":"2023-12-01T03:10:38.507374+0000","last_fullsized":"2023-12-01T03:10:38.507374+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:14.969594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"201'326","reported_seq":1123,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983105+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:10:38.983105+0000","last_peered":"2023-12-01T03:10:38.983105+0000","last_clean":"2023-12-01T03:10:38.983105+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:10:38.983105+0000","last_undegraded":"2023-12-01T03:10:38.983105+0000","last_fullsized":"2023-12-01T03:10:38.983105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988127+0000","last_change":"2023-12-01T03:10:24.685430+0000","last_active":"2023-12-01T03:10:38.988127+0000","last_peered":"2023-12-01T03:10:38.988127+0000","last_clean":"2023-12-01T03:10:38.988127+0000","last_became_active":"2023-12-01T03:10:24.684919+0000","last_became_peered":"2023-12-01T03:10:24.684919+0000","last_unstale":"2023-12-01T03:10:38.988127+0000","last_undegraded":"2023-12-01T03:10:38.988127+0000","last_fullsized":"2023-12-01T03:10:38.988127+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481058+0000","last_change":"2023-12-01T03:10:25.189213+0000","last_active":"2023-12-01T03:10:37.481058+0000","last_peered":"2023-12-01T03:10:37.481058+0000","last_clean":"2023-12-01T03:10:37.481058+0000","last_became_active":"2023-12-01T03:10:25.188927+0000","last_became_peered":"2023-12-01T03:10:25.188927+0000","last_unstale":"2023-12-01T03:10:37.481058+0000","last_undegraded":"2023-12-01T03:10:37.481058+0000","last_fullsized":"2023-12-01T03:10:37.481058+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:21:53.633800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520651+0000","last_change":"2023-12-01T03:10:38.517472+0000","last_active":"2023-12-01T03:10:38.520651+0000","last_peered":"2023-12-01T03:10:38.520651+0000","last_clean":"2023-12-01T03:10:38.520651+0000","last_became_active":"2023-12-01T03:10:38.517218+0000","last_became_peered":"2023-12-01T03:10:38.517218+0000","last_unstale":"2023-12-01T03:10:38.520651+0000","last_undegraded":"2023-12-01T03:10:38.520651+0000","last_fullsized":"2023-12-01T03:10:38.520651+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:57.653086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499505+0000","last_change":"2023-12-01T03:10:38.499505+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499505+0000","last_undegraded":"2023-12-01T03:10:38.499505+0000","last_fullsized":"2023-12-01T03:10:38.499505+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481362+0000","last_change":"2023-12-01T03:10:24.690155+0000","last_active":"2023-12-01T03:10:37.481362+0000","last_peered":"2023-12-01T03:10:37.481362+0000","last_clean":"2023-12-01T03:10:37.481362+0000","last_became_active":"2023-12-01T03:10:24.689122+0000","last_became_peered":"2023-12-01T03:10:24.689122+0000","last_unstale":"2023-12-01T03:10:37.481362+0000","last_undegraded":"2023-12-01T03:10:37.481362+0000","last_fullsized":"2023-12-01T03:10:37.481362+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479122+0000","last_change":"2023-12-01T03:10:25.191828+0000","last_active":"2023-12-01T03:10:37.479122+0000","last_peered":"2023-12-01T03:10:37.479122+0000","last_clean":"2023-12-01T03:10:37.479122+0000","last_became_active":"2023-12-01T03:10:25.191436+0000","last_became_peered":"2023-12-01T03:10:25.191436+0000","last_unstale":"2023-12-01T03:10:37.479122+0000","last_undegraded":"2023-12-01T03:10:37.479122+0000","last_fullsized":"2023-12-01T03:10:37.479122+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525987+0000","last_change":"2023-12-01T03:10:38.524843+0000","last_active":"2023-12-01T03:10:38.525987+0000","last_peered":"2023-12-01T03:10:38.525987+0000","last_clean":"2023-12-01T03:10:38.525987+0000","last_became_active":"2023-12-01T03:10:38.524424+0000","last_became_peered":"2023-12-01T03:10:38.524424+0000","last_unstale":"2023-12-01T03:10:38.525987+0000","last_undegraded":"2023-12-01T03:10:38.525987+0000","last_fullsized":"2023-12-01T03:10:38.525987+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:32.967785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.509995+0000","last_change":"2023-12-01T03:10:38.509995+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.509995+0000","last_undegraded":"2023-12-01T03:10:38.509995+0000","last_fullsized":"2023-12-01T03:10:38.509995+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983588+0000","last_change":"2023-12-01T03:10:24.684248+0000","last_active":"2023-12-01T03:10:38.983588+0000","last_peered":"2023-12-01T03:10:38.983588+0000","last_clean":"2023-12-01T03:10:38.983588+0000","last_became_active":"2023-12-01T03:10:24.680714+0000","last_became_peered":"2023-12-01T03:10:24.680714+0000","last_unstale":"2023-12-01T03:10:38.983588+0000","last_undegraded":"2023-12-01T03:10:38.983588+0000","last_fullsized":"2023-12-01T03:10:38.983588+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:19:02.474558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.982169+0000","last_change":"2023-12-01T03:10:25.174085+0000","last_active":"2023-12-01T03:10:38.982169+0000","last_peered":"2023-12-01T03:10:38.982169+0000","last_clean":"2023-12-01T03:10:38.982169+0000","last_became_active":"2023-12-01T03:10:25.173802+0000","last_became_peered":"2023-12-01T03:10:25.173802+0000","last_unstale":"2023-12-01T03:10:38.982169+0000","last_undegraded":"2023-12-01T03:10:38.982169+0000","last_fullsized":"2023-12-01T03:10:38.982169+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:39.145914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.982926+0000","last_change":"2023-12-01T03:10:38.512523+0000","last_active":"2023-12-01T03:10:38.982926+0000","last_peered":"2023-12-01T03:10:38.982926+0000","last_clean":"2023-12-01T03:10:38.982926+0000","last_became_active":"2023-12-01T03:10:38.512225+0000","last_became_peered":"2023-12-01T03:10:38.512225+0000","last_unstale":"2023-12-01T03:10:38.982926+0000","last_undegraded":"2023-12-01T03:10:38.982926+0000","last_fullsized":"2023-12-01T03:10:38.982926+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:05.819515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003662+0000","last_change":"2023-12-01T03:10:39.001307+0000","last_active":"2023-12-01T03:10:39.003662+0000","last_peered":"2023-12-01T03:10:39.003662+0000","last_clean":"2023-12-01T03:10:39.003662+0000","last_became_active":"2023-12-01T03:10:39.000379+0000","last_became_peered":"2023-12-01T03:10:39.000379+0000","last_unstale":"2023-12-01T03:10:39.003662+0000","last_undegraded":"2023-12-01T03:10:39.003662+0000","last_fullsized":"2023-12-01T03:10:39.003662+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:39.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986591+0000","last_change":"2023-12-01T03:10:24.683240+0000","last_active":"2023-12-01T03:10:38.986591+0000","last_peered":"2023-12-01T03:10:38.986591+0000","last_clean":"2023-12-01T03:10:38.986591+0000","last_became_active":"2023-12-01T03:10:24.682933+0000","last_became_peered":"2023-12-01T03:10:24.682933+0000","last_unstale":"2023-12-01T03:10:38.986591+0000","last_undegraded":"2023-12-01T03:10:38.986591+0000","last_fullsized":"2023-12-01T03:10:38.986591+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:40.211424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472633+0000","last_change":"2023-12-01T03:10:25.181396+0000","last_active":"2023-12-01T03:10:37.472633+0000","last_peered":"2023-12-01T03:10:37.472633+0000","last_clean":"2023-12-01T03:10:37.472633+0000","last_became_active":"2023-12-01T03:10:25.181134+0000","last_became_peered":"2023-12-01T03:10:25.181134+0000","last_unstale":"2023-12-01T03:10:37.472633+0000","last_undegraded":"2023-12-01T03:10:37.472633+0000","last_fullsized":"2023-12-01T03:10:37.472633+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:04.776013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.981977+0000","last_change":"2023-12-01T03:10:38.522641+0000","last_active":"2023-12-01T03:10:38.981977+0000","last_peered":"2023-12-01T03:10:38.981977+0000","last_clean":"2023-12-01T03:10:38.981977+0000","last_became_active":"2023-12-01T03:10:38.522225+0000","last_became_peered":"2023-12-01T03:10:38.522225+0000","last_unstale":"2023-12-01T03:10:38.981977+0000","last_undegraded":"2023-12-01T03:10:38.981977+0000","last_fullsized":"2023-12-01T03:10:38.981977+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:13.766818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499285+0000","last_change":"2023-12-01T03:10:38.499285+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499285+0000","last_undegraded":"2023-12-01T03:10:38.499285+0000","last_fullsized":"2023-12-01T03:10:38.499285+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985936+0000","last_change":"2023-12-01T03:10:24.687326+0000","last_active":"2023-12-01T03:10:38.985936+0000","last_peered":"2023-12-01T03:10:38.985936+0000","last_clean":"2023-12-01T03:10:38.985936+0000","last_became_active":"2023-12-01T03:10:24.687036+0000","last_became_peered":"2023-12-01T03:10:24.687036+0000","last_unstale":"2023-12-01T03:10:38.985936+0000","last_undegraded":"2023-12-01T03:10:38.985936+0000","last_fullsized":"2023-12-01T03:10:38.985936+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:36:57.969605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986462+0000","last_change":"2023-12-01T03:10:25.190210+0000","last_active":"2023-12-01T03:10:38.986462+0000","last_peered":"2023-12-01T03:10:38.986462+0000","last_clean":"2023-12-01T03:10:38.986462+0000","last_became_active":"2023-12-01T03:10:25.189957+0000","last_became_peered":"2023-12-01T03:10:25.189957+0000","last_unstale":"2023-12-01T03:10:38.986462+0000","last_undegraded":"2023-12-01T03:10:38.986462+0000","last_fullsized":"2023-12-01T03:10:38.986462+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:01.783625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988039+0000","last_change":"2023-12-01T03:10:38.514437+0000","last_active":"2023-12-01T03:10:38.988039+0000","last_peered":"2023-12-01T03:10:38.988039+0000","last_clean":"2023-12-01T03:10:38.988039+0000","last_became_active":"2023-12-01T03:10:38.514155+0000","last_became_peered":"2023-12-01T03:10:38.514155+0000","last_unstale":"2023-12-01T03:10:38.988039+0000","last_undegraded":"2023-12-01T03:10:38.988039+0000","last_fullsized":"2023-12-01T03:10:38.988039+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:06.817444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.503490+0000","last_change":"2023-12-01T03:10:38.503490+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.503490+0000","last_undegraded":"2023-12-01T03:10:38.503490+0000","last_fullsized":"2023-12-01T03:10:38.503490+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481503+0000","last_change":"2023-12-01T03:10:24.697595+0000","last_active":"2023-12-01T03:10:37.481503+0000","last_peered":"2023-12-01T03:10:37.481503+0000","last_clean":"2023-12-01T03:10:37.481503+0000","last_became_active":"2023-12-01T03:10:24.697311+0000","last_became_peered":"2023-12-01T03:10:24.697311+0000","last_unstale":"2023-12-01T03:10:37.481503+0000","last_undegraded":"2023-12-01T03:10:37.481503+0000","last_fullsized":"2023-12-01T03:10:37.481503+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:45.743456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483634+0000","last_change":"2023-12-01T03:10:25.192028+0000","last_active":"2023-12-01T03:10:37.483634+0000","last_peered":"2023-12-01T03:10:37.483634+0000","last_clean":"2023-12-01T03:10:37.483634+0000","last_became_active":"2023-12-01T03:10:25.191747+0000","last_became_peered":"2023-12-01T03:10:25.191747+0000","last_unstale":"2023-12-01T03:10:37.483634+0000","last_undegraded":"2023-12-01T03:10:37.483634+0000","last_fullsized":"2023-12-01T03:10:37.483634+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520576+0000","last_change":"2023-12-01T03:10:38.517012+0000","last_active":"2023-12-01T03:10:38.520576+0000","last_peered":"2023-12-01T03:10:38.520576+0000","last_clean":"2023-12-01T03:10:38.520576+0000","last_became_active":"2023-12-01T03:10:38.516749+0000","last_became_peered":"2023-12-01T03:10:38.516749+0000","last_unstale":"2023-12-01T03:10:38.520576+0000","last_undegraded":"2023-12-01T03:10:38.520576+0000","last_fullsized":"2023-12-01T03:10:38.520576+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:29.962162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008414+0000","last_change":"2023-12-01T03:10:39.005634+0000","last_active":"2023-12-01T03:10:39.008414+0000","last_peered":"2023-12-01T03:10:39.008414+0000","last_clean":"2023-12-01T03:10:39.008414+0000","last_became_active":"2023-12-01T03:10:39.005386+0000","last_became_peered":"2023-12-01T03:10:39.005386+0000","last_unstale":"2023-12-01T03:10:39.008414+0000","last_undegraded":"2023-12-01T03:10:39.008414+0000","last_fullsized":"2023-12-01T03:10:39.008414+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488889+0000","last_change":"2023-12-01T03:10:24.697992+0000","last_active":"2023-12-01T03:10:37.488889+0000","last_peered":"2023-12-01T03:10:37.488889+0000","last_clean":"2023-12-01T03:10:37.488889+0000","last_became_active":"2023-12-01T03:10:24.696883+0000","last_became_peered":"2023-12-01T03:10:24.696883+0000","last_unstale":"2023-12-01T03:10:37.488889+0000","last_undegraded":"2023-12-01T03:10:37.488889+0000","last_fullsized":"2023-12-01T03:10:37.488889+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.487148+0000","last_change":"2023-12-01T03:10:25.175476+0000","last_active":"2023-12-01T03:10:37.487148+0000","last_peered":"2023-12-01T03:10:37.487148+0000","last_clean":"2023-12-01T03:10:37.487148+0000","last_became_active":"2023-12-01T03:10:25.175211+0000","last_became_peered":"2023-12-01T03:10:25.175211+0000","last_unstale":"2023-12-01T03:10:37.487148+0000","last_undegraded":"2023-12-01T03:10:37.487148+0000","last_fullsized":"2023-12-01T03:10:37.487148+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:22.358205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515962+0000","last_change":"2023-12-01T03:10:38.513347+0000","last_active":"2023-12-01T03:10:38.515962+0000","last_peered":"2023-12-01T03:10:38.515962+0000","last_clean":"2023-12-01T03:10:38.515962+0000","last_became_active":"2023-12-01T03:10:38.512782+0000","last_became_peered":"2023-12-01T03:10:38.512782+0000","last_unstale":"2023-12-01T03:10:38.515962+0000","last_undegraded":"2023-12-01T03:10:38.515962+0000","last_fullsized":"2023-12-01T03:10:38.515962+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:06.647413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.010194+0000","last_change":"2023-12-01T03:10:39.007372+0000","last_active":"2023-12-01T03:10:39.010194+0000","last_peered":"2023-12-01T03:10:39.010194+0000","last_clean":"2023-12-01T03:10:39.010194+0000","last_became_active":"2023-12-01T03:10:39.007059+0000","last_became_peered":"2023-12-01T03:10:39.007059+0000","last_unstale":"2023-12-01T03:10:39.010194+0000","last_undegraded":"2023-12-01T03:10:39.010194+0000","last_fullsized":"2023-12-01T03:10:39.010194+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.492807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"214'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489570+0000","last_change":"2023-12-01T03:10:24.697890+0000","last_active":"2023-12-01T03:10:37.489570+0000","last_peered":"2023-12-01T03:10:37.489570+0000","last_clean":"2023-12-01T03:10:37.489570+0000","last_became_active":"2023-12-01T03:10:24.697679+0000","last_became_peered":"2023-12-01T03:10:24.697679+0000","last_unstale":"2023-12-01T03:10:37.489570+0000","last_undegraded":"2023-12-01T03:10:37.489570+0000","last_fullsized":"2023-12-01T03:10:37.489570+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488204+0000","last_change":"2023-12-01T03:10:25.168359+0000","last_active":"2023-12-01T03:10:37.488204+0000","last_peered":"2023-12-01T03:10:37.488204+0000","last_clean":"2023-12-01T03:10:37.488204+0000","last_became_active":"2023-12-01T03:10:25.168082+0000","last_became_peered":"2023-12-01T03:10:25.168082+0000","last_unstale":"2023-12-01T03:10:37.488204+0000","last_undegraded":"2023-12-01T03:10:37.488204+0000","last_fullsized":"2023-12-01T03:10:37.488204+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:06.024028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525349+0000","last_change":"2023-12-01T03:10:38.524232+0000","last_active":"2023-12-01T03:10:38.525349+0000","last_peered":"2023-12-01T03:10:38.525349+0000","last_clean":"2023-12-01T03:10:38.525349+0000","last_became_active":"2023-12-01T03:10:38.523871+0000","last_became_peered":"2023-12-01T03:10:38.523871+0000","last_unstale":"2023-12-01T03:10:38.525349+0000","last_undegraded":"2023-12-01T03:10:38.525349+0000","last_fullsized":"2023-12-01T03:10:38.525349+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:06.647413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499505+0000","last_change":"2023-12-01T03:10:38.499505+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499505+0000","last_undegraded":"2023-12-01T03:10:38.499505+0000","last_fullsized":"2023-12-01T03:10:38.499505+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.7","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472938+0000","last_change":"2023-12-01T03:10:24.687160+0000","last_active":"2023-12-01T03:10:37.472938+0000","last_peered":"2023-12-01T03:10:37.472938+0000","last_clean":"2023-12-01T03:10:37.472938+0000","last_became_active":"2023-12-01T03:10:24.686184+0000","last_became_peered":"2023-12-01T03:10:24.686184+0000","last_unstale":"2023-12-01T03:10:37.472938+0000","last_undegraded":"2023-12-01T03:10:37.472938+0000","last_fullsized":"2023-12-01T03:10:37.472938+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:05.006954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489573+0000","last_change":"2023-12-01T03:10:25.184992+0000","last_active":"2023-12-01T03:10:37.489573+0000","last_peered":"2023-12-01T03:10:37.489573+0000","last_clean":"2023-12-01T03:10:37.489573+0000","last_became_active":"2023-12-01T03:10:25.184670+0000","last_became_peered":"2023-12-01T03:10:25.184670+0000","last_unstale":"2023-12-01T03:10:37.489573+0000","last_undegraded":"2023-12-01T03:10:37.489573+0000","last_fullsized":"2023-12-01T03:10:37.489573+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:40.143822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519720+0000","last_change":"2023-12-01T03:10:38.517303+0000","last_active":"2023-12-01T03:10:38.519720+0000","last_peered":"2023-12-01T03:10:38.519720+0000","last_clean":"2023-12-01T03:10:38.519720+0000","last_became_active":"2023-12-01T03:10:38.516948+0000","last_became_peered":"2023-12-01T03:10:38.516948+0000","last_unstale":"2023-12-01T03:10:38.519720+0000","last_undegraded":"2023-12-01T03:10:38.519720+0000","last_fullsized":"2023-12-01T03:10:38.519720+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:58:41.627485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504217+0000","last_change":"2023-12-01T03:10:38.504217+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504217+0000","last_undegraded":"2023-12-01T03:10:38.504217+0000","last_fullsized":"2023-12-01T03:10:38.504217+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:18.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472354+0000","last_change":"2023-12-01T03:10:24.685003+0000","last_active":"2023-12-01T03:10:37.472354+0000","last_peered":"2023-12-01T03:10:37.472354+0000","last_clean":"2023-12-01T03:10:37.472354+0000","last_became_active":"2023-12-01T03:10:24.684425+0000","last_became_peered":"2023-12-01T03:10:24.684425+0000","last_unstale":"2023-12-01T03:10:37.472354+0000","last_undegraded":"2023-12-01T03:10:37.472354+0000","last_fullsized":"2023-12-01T03:10:37.472354+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:45.173492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491598+0000","last_change":"2023-12-01T03:10:25.174479+0000","last_active":"2023-12-01T03:10:37.491598+0000","last_peered":"2023-12-01T03:10:37.491598+0000","last_clean":"2023-12-01T03:10:37.491598+0000","last_became_active":"2023-12-01T03:10:25.174194+0000","last_became_peered":"2023-12-01T03:10:25.174194+0000","last_unstale":"2023-12-01T03:10:37.491598+0000","last_undegraded":"2023-12-01T03:10:37.491598+0000","last_fullsized":"2023-12-01T03:10:37.491598+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519343+0000","last_change":"2023-12-01T03:10:38.516993+0000","last_active":"2023-12-01T03:10:38.519343+0000","last_peered":"2023-12-01T03:10:38.519343+0000","last_clean":"2023-12-01T03:10:38.519343+0000","last_became_active":"2023-12-01T03:10:38.516717+0000","last_became_peered":"2023-12-01T03:10:38.516717+0000","last_unstale":"2023-12-01T03:10:38.519343+0000","last_undegraded":"2023-12-01T03:10:38.519343+0000","last_fullsized":"2023-12-01T03:10:38.519343+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:51.689717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508392+0000","last_change":"2023-12-01T03:10:38.508392+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508392+0000","last_undegraded":"2023-12-01T03:10:38.508392+0000","last_fullsized":"2023-12-01T03:10:38.508392+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:53.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481282+0000","last_change":"2023-12-01T03:10:24.694594+0000","last_active":"2023-12-01T03:10:37.481282+0000","last_peered":"2023-12-01T03:10:37.481282+0000","last_clean":"2023-12-01T03:10:37.481282+0000","last_became_active":"2023-12-01T03:10:24.693808+0000","last_became_peered":"2023-12-01T03:10:24.693808+0000","last_unstale":"2023-12-01T03:10:37.481282+0000","last_undegraded":"2023-12-01T03:10:37.481282+0000","last_fullsized":"2023-12-01T03:10:37.481282+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472327+0000","last_change":"2023-12-01T03:10:25.173728+0000","last_active":"2023-12-01T03:10:37.472327+0000","last_peered":"2023-12-01T03:10:37.472327+0000","last_clean":"2023-12-01T03:10:37.472327+0000","last_became_active":"2023-12-01T03:10:25.173449+0000","last_became_peered":"2023-12-01T03:10:25.173449+0000","last_unstale":"2023-12-01T03:10:37.472327+0000","last_undegraded":"2023-12-01T03:10:37.472327+0000","last_fullsized":"2023-12-01T03:10:37.472327+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:20.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.513730+0000","last_change":"2023-12-01T03:10:38.508401+0000","last_active":"2023-12-01T03:10:38.513730+0000","last_peered":"2023-12-01T03:10:38.513730+0000","last_clean":"2023-12-01T03:10:38.513730+0000","last_became_active":"2023-12-01T03:10:38.508089+0000","last_became_peered":"2023-12-01T03:10:38.508089+0000","last_unstale":"2023-12-01T03:10:38.513730+0000","last_undegraded":"2023-12-01T03:10:38.513730+0000","last_fullsized":"2023-12-01T03:10:38.513730+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:56.645685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003642+0000","last_change":"2023-12-01T03:10:39.000679+0000","last_active":"2023-12-01T03:10:39.003642+0000","last_peered":"2023-12-01T03:10:39.003642+0000","last_clean":"2023-12-01T03:10:39.003642+0000","last_became_active":"2023-12-01T03:10:38.997903+0000","last_became_peered":"2023-12-01T03:10:38.997903+0000","last_unstale":"2023-12-01T03:10:39.003642+0000","last_undegraded":"2023-12-01T03:10:39.003642+0000","last_fullsized":"2023-12-01T03:10:39.003642+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:27.357798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482736+0000","last_change":"2023-12-01T03:10:24.685775+0000","last_active":"2023-12-01T03:10:37.482736+0000","last_peered":"2023-12-01T03:10:37.482736+0000","last_clean":"2023-12-01T03:10:37.482736+0000","last_became_active":"2023-12-01T03:10:24.685178+0000","last_became_peered":"2023-12-01T03:10:24.685178+0000","last_unstale":"2023-12-01T03:10:37.482736+0000","last_undegraded":"2023-12-01T03:10:37.482736+0000","last_fullsized":"2023-12-01T03:10:37.482736+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:21.942226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483463+0000","last_change":"2023-12-01T03:10:25.180768+0000","last_active":"2023-12-01T03:10:37.483463+0000","last_peered":"2023-12-01T03:10:37.483463+0000","last_clean":"2023-12-01T03:10:37.483463+0000","last_became_active":"2023-12-01T03:10:25.179796+0000","last_became_peered":"2023-12-01T03:10:25.179796+0000","last_unstale":"2023-12-01T03:10:37.483463+0000","last_undegraded":"2023-12-01T03:10:37.483463+0000","last_fullsized":"2023-12-01T03:10:37.483463+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515225+0000","last_change":"2023-12-01T03:10:38.511676+0000","last_active":"2023-12-01T03:10:38.515225+0000","last_peered":"2023-12-01T03:10:38.515225+0000","last_clean":"2023-12-01T03:10:38.515225+0000","last_became_active":"2023-12-01T03:10:38.511406+0000","last_became_peered":"2023-12-01T03:10:38.511406+0000","last_unstale":"2023-12-01T03:10:38.515225+0000","last_undegraded":"2023-12-01T03:10:38.515225+0000","last_fullsized":"2023-12-01T03:10:38.515225+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:44.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003720+0000","last_change":"2023-12-01T03:10:38.997814+0000","last_active":"2023-12-01T03:10:39.003720+0000","last_peered":"2023-12-01T03:10:39.003720+0000","last_clean":"2023-12-01T03:10:39.003720+0000","last_became_active":"2023-12-01T03:10:38.997558+0000","last_became_peered":"2023-12-01T03:10:38.997558+0000","last_unstale":"2023-12-01T03:10:39.003720+0000","last_undegraded":"2023-12-01T03:10:39.003720+0000","last_fullsized":"2023-12-01T03:10:39.003720+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:49:09.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.b","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481673+0000","last_change":"2023-12-01T03:10:24.695154+0000","last_active":"2023-12-01T03:10:37.481673+0000","last_peered":"2023-12-01T03:10:37.481673+0000","last_clean":"2023-12-01T03:10:37.481673+0000","last_became_active":"2023-12-01T03:10:24.694868+0000","last_became_peered":"2023-12-01T03:10:24.694868+0000","last_unstale":"2023-12-01T03:10:37.481673+0000","last_undegraded":"2023-12-01T03:10:37.481673+0000","last_fullsized":"2023-12-01T03:10:37.481673+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482517+0000","last_change":"2023-12-01T03:10:25.169678+0000","last_active":"2023-12-01T03:10:37.482517+0000","last_peered":"2023-12-01T03:10:37.482517+0000","last_clean":"2023-12-01T03:10:37.482517+0000","last_became_active":"2023-12-01T03:10:25.168714+0000","last_became_peered":"2023-12-01T03:10:25.168714+0000","last_unstale":"2023-12-01T03:10:37.482517+0000","last_undegraded":"2023-12-01T03:10:37.482517+0000","last_fullsized":"2023-12-01T03:10:37.482517+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.526019+0000","last_change":"2023-12-01T03:10:38.524888+0000","last_active":"2023-12-01T03:10:38.526019+0000","last_peered":"2023-12-01T03:10:38.526019+0000","last_clean":"2023-12-01T03:10:38.526019+0000","last_became_active":"2023-12-01T03:10:38.524601+0000","last_became_peered":"2023-12-01T03:10:38.524601+0000","last_unstale":"2023-12-01T03:10:38.526019+0000","last_undegraded":"2023-12-01T03:10:38.526019+0000","last_fullsized":"2023-12-01T03:10:38.526019+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:47.875991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.011924+0000","last_change":"2023-12-01T03:10:39.009719+0000","last_active":"2023-12-01T03:10:39.011924+0000","last_peered":"2023-12-01T03:10:39.011924+0000","last_clean":"2023-12-01T03:10:39.011924+0000","last_became_active":"2023-12-01T03:10:39.009561+0000","last_became_peered":"2023-12-01T03:10:39.009561+0000","last_unstale":"2023-12-01T03:10:39.011924+0000","last_undegraded":"2023-12-01T03:10:39.011924+0000","last_fullsized":"2023-12-01T03:10:39.011924+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:19.853707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473031+0000","last_change":"2023-12-01T03:10:24.683594+0000","last_active":"2023-12-01T03:10:37.473031+0000","last_peered":"2023-12-01T03:10:37.473031+0000","last_clean":"2023-12-01T03:10:37.473031+0000","last_became_active":"2023-12-01T03:10:24.679979+0000","last_became_peered":"2023-12-01T03:10:24.679979+0000","last_unstale":"2023-12-01T03:10:37.473031+0000","last_undegraded":"2023-12-01T03:10:37.473031+0000","last_fullsized":"2023-12-01T03:10:37.473031+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:12.511044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987554+0000","last_change":"2023-12-01T03:10:25.172837+0000","last_active":"2023-12-01T03:10:38.987554+0000","last_peered":"2023-12-01T03:10:38.987554+0000","last_clean":"2023-12-01T03:10:38.987554+0000","last_became_active":"2023-12-01T03:10:25.172602+0000","last_became_peered":"2023-12-01T03:10:25.172602+0000","last_unstale":"2023-12-01T03:10:38.987554+0000","last_undegraded":"2023-12-01T03:10:38.987554+0000","last_fullsized":"2023-12-01T03:10:38.987554+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521284+0000","last_change":"2023-12-01T03:10:38.517877+0000","last_active":"2023-12-01T03:10:38.521284+0000","last_peered":"2023-12-01T03:10:38.521284+0000","last_clean":"2023-12-01T03:10:38.521284+0000","last_became_active":"2023-12-01T03:10:38.517241+0000","last_became_peered":"2023-12-01T03:10:38.517241+0000","last_unstale":"2023-12-01T03:10:38.521284+0000","last_undegraded":"2023-12-01T03:10:38.521284+0000","last_fullsized":"2023-12-01T03:10:38.521284+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:32.967785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.004933+0000","last_change":"2023-12-01T03:10:39.003555+0000","last_active":"2023-12-01T03:10:39.004933+0000","last_peered":"2023-12-01T03:10:39.004933+0000","last_clean":"2023-12-01T03:10:39.004933+0000","last_became_active":"2023-12-01T03:10:39.003311+0000","last_became_peered":"2023-12-01T03:10:39.003311+0000","last_unstale":"2023-12-01T03:10:39.004933+0000","last_undegraded":"2023-12-01T03:10:39.004933+0000","last_fullsized":"2023-12-01T03:10:39.004933+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.d","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473150+0000","last_change":"2023-12-01T03:10:24.691398+0000","last_active":"2023-12-01T03:10:37.473150+0000","last_peered":"2023-12-01T03:10:37.473150+0000","last_clean":"2023-12-01T03:10:37.473150+0000","last_became_active":"2023-12-01T03:10:24.691110+0000","last_became_peered":"2023-12-01T03:10:24.691110+0000","last_unstale":"2023-12-01T03:10:37.473150+0000","last_undegraded":"2023-12-01T03:10:37.473150+0000","last_fullsized":"2023-12-01T03:10:37.473150+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:24.670899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999898+0000","last_change":"2023-12-01T03:10:25.178959+0000","last_active":"2023-12-01T03:10:38.999898+0000","last_peered":"2023-12-01T03:10:38.999898+0000","last_clean":"2023-12-01T03:10:38.999898+0000","last_became_active":"2023-12-01T03:10:25.178268+0000","last_became_peered":"2023-12-01T03:10:25.178268+0000","last_unstale":"2023-12-01T03:10:38.999898+0000","last_undegraded":"2023-12-01T03:10:38.999898+0000","last_fullsized":"2023-12-01T03:10:38.999898+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:55:31.814432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525071+0000","last_change":"2023-12-01T03:10:38.523799+0000","last_active":"2023-12-01T03:10:38.525071+0000","last_peered":"2023-12-01T03:10:38.525071+0000","last_clean":"2023-12-01T03:10:38.525071+0000","last_became_active":"2023-12-01T03:10:38.523512+0000","last_became_peered":"2023-12-01T03:10:38.523512+0000","last_unstale":"2023-12-01T03:10:38.525071+0000","last_undegraded":"2023-12-01T03:10:38.525071+0000","last_fullsized":"2023-12-01T03:10:38.525071+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:44.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504526+0000","last_change":"2023-12-01T03:10:38.504526+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504526+0000","last_undegraded":"2023-12-01T03:10:38.504526+0000","last_fullsized":"2023-12-01T03:10:38.504526+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:41.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482803+0000","last_change":"2023-12-01T03:10:24.688107+0000","last_active":"2023-12-01T03:10:37.482803+0000","last_peered":"2023-12-01T03:10:37.482803+0000","last_clean":"2023-12-01T03:10:37.482803+0000","last_became_active":"2023-12-01T03:10:24.686930+0000","last_became_peered":"2023-12-01T03:10:24.686930+0000","last_unstale":"2023-12-01T03:10:37.482803+0000","last_undegraded":"2023-12-01T03:10:37.482803+0000","last_fullsized":"2023-12-01T03:10:37.482803+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:44.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986385+0000","last_change":"2023-12-01T03:10:25.180913+0000","last_active":"2023-12-01T03:10:38.986385+0000","last_peered":"2023-12-01T03:10:38.986385+0000","last_clean":"2023-12-01T03:10:38.986385+0000","last_became_active":"2023-12-01T03:10:25.180626+0000","last_became_peered":"2023-12-01T03:10:25.180626+0000","last_unstale":"2023-12-01T03:10:38.986385+0000","last_undegraded":"2023-12-01T03:10:38.986385+0000","last_fullsized":"2023-12-01T03:10:38.986385+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520264+0000","last_change":"2023-12-01T03:10:38.519139+0000","last_active":"2023-12-01T03:10:38.520264+0000","last_peered":"2023-12-01T03:10:38.520264+0000","last_clean":"2023-12-01T03:10:38.520264+0000","last_became_active":"2023-12-01T03:10:38.518855+0000","last_became_peered":"2023-12-01T03:10:38.518855+0000","last_unstale":"2023-12-01T03:10:38.520264+0000","last_undegraded":"2023-12-01T03:10:38.520264+0000","last_fullsized":"2023-12-01T03:10:38.520264+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:52.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511251+0000","last_change":"2023-12-01T03:10:38.511251+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511251+0000","last_undegraded":"2023-12-01T03:10:38.511251+0000","last_fullsized":"2023-12-01T03:10:38.511251+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:47:30.021172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.f","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482669+0000","last_change":"2023-12-01T03:10:24.699945+0000","last_active":"2023-12-01T03:10:37.482669+0000","last_peered":"2023-12-01T03:10:37.482669+0000","last_clean":"2023-12-01T03:10:37.482669+0000","last_became_active":"2023-12-01T03:10:24.699598+0000","last_became_peered":"2023-12-01T03:10:24.699598+0000","last_unstale":"2023-12-01T03:10:37.482669+0000","last_undegraded":"2023-12-01T03:10:37.482669+0000","last_fullsized":"2023-12-01T03:10:37.482669+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492690+0000","last_change":"2023-12-01T03:10:25.181692+0000","last_active":"2023-12-01T03:10:37.492690+0000","last_peered":"2023-12-01T03:10:37.492690+0000","last_clean":"2023-12-01T03:10:37.492690+0000","last_became_active":"2023-12-01T03:10:25.181316+0000","last_became_peered":"2023-12-01T03:10:25.181316+0000","last_unstale":"2023-12-01T03:10:37.492690+0000","last_undegraded":"2023-12-01T03:10:37.492690+0000","last_fullsized":"2023-12-01T03:10:37.492690+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:57:07.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.522721+0000","last_change":"2023-12-01T03:10:38.518068+0000","last_active":"2023-12-01T03:10:38.522721+0000","last_peered":"2023-12-01T03:10:38.522721+0000","last_clean":"2023-12-01T03:10:38.522721+0000","last_became_active":"2023-12-01T03:10:38.517792+0000","last_became_peered":"2023-12-01T03:10:38.517792+0000","last_unstale":"2023-12-01T03:10:38.522721+0000","last_undegraded":"2023-12-01T03:10:38.522721+0000","last_fullsized":"2023-12-01T03:10:38.522721+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:01.616973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511262+0000","last_change":"2023-12-01T03:10:38.511262+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511262+0000","last_undegraded":"2023-12-01T03:10:38.511262+0000","last_fullsized":"2023-12-01T03:10:38.511262+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"213'8","reported_seq":21,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483272+0000","last_change":"2023-12-01T03:10:24.685759+0000","last_active":"2023-12-01T03:10:37.483272+0000","last_peered":"2023-12-01T03:10:37.483272+0000","last_clean":"2023-12-01T03:10:37.483272+0000","last_became_active":"2023-12-01T03:10:24.684800+0000","last_became_peered":"2023-12-01T03:10:24.684800+0000","last_unstale":"2023-12-01T03:10:37.483272+0000","last_undegraded":"2023-12-01T03:10:37.483272+0000","last_fullsized":"2023-12-01T03:10:37.483272+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:29.129070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478917+0000","last_change":"2023-12-01T03:10:25.193021+0000","last_active":"2023-12-01T03:10:37.478917+0000","last_peered":"2023-12-01T03:10:37.478917+0000","last_clean":"2023-12-01T03:10:37.478917+0000","last_became_active":"2023-12-01T03:10:25.192051+0000","last_became_peered":"2023-12-01T03:10:25.192051+0000","last_unstale":"2023-12-01T03:10:37.478917+0000","last_undegraded":"2023-12-01T03:10:37.478917+0000","last_fullsized":"2023-12-01T03:10:37.478917+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521581+0000","last_change":"2023-12-01T03:10:38.517969+0000","last_active":"2023-12-01T03:10:38.521581+0000","last_peered":"2023-12-01T03:10:38.521581+0000","last_clean":"2023-12-01T03:10:38.521581+0000","last_became_active":"2023-12-01T03:10:38.517693+0000","last_became_peered":"2023-12-01T03:10:38.517693+0000","last_unstale":"2023-12-01T03:10:38.521581+0000","last_undegraded":"2023-12-01T03:10:38.521581+0000","last_fullsized":"2023-12-01T03:10:38.521581+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:18.447021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.498276+0000","last_change":"2023-12-01T03:10:38.498276+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.498276+0000","last_undegraded":"2023-12-01T03:10:38.498276+0000","last_fullsized":"2023-12-01T03:10:38.498276+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479511+0000","last_change":"2023-12-01T03:10:24.694942+0000","last_active":"2023-12-01T03:10:37.479511+0000","last_peered":"2023-12-01T03:10:37.479511+0000","last_clean":"2023-12-01T03:10:37.479511+0000","last_became_active":"2023-12-01T03:10:24.694664+0000","last_became_peered":"2023-12-01T03:10:24.694664+0000","last_unstale":"2023-12-01T03:10:37.479511+0000","last_undegraded":"2023-12-01T03:10:37.479511+0000","last_fullsized":"2023-12-01T03:10:37.479511+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:38:02.332793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988703+0000","last_change":"2023-12-01T03:10:25.164054+0000","last_active":"2023-12-01T03:10:38.988703+0000","last_peered":"2023-12-01T03:10:38.988703+0000","last_clean":"2023-12-01T03:10:38.988703+0000","last_became_active":"2023-12-01T03:10:25.163787+0000","last_became_peered":"2023-12-01T03:10:25.163787+0000","last_unstale":"2023-12-01T03:10:38.988703+0000","last_undegraded":"2023-12-01T03:10:38.988703+0000","last_fullsized":"2023-12-01T03:10:38.988703+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:04.568421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521758+0000","last_change":"2023-12-01T03:10:38.520314+0000","last_active":"2023-12-01T03:10:38.521758+0000","last_peered":"2023-12-01T03:10:38.521758+0000","last_clean":"2023-12-01T03:10:38.521758+0000","last_became_active":"2023-12-01T03:10:38.519056+0000","last_became_peered":"2023-12-01T03:10:38.519056+0000","last_unstale":"2023-12-01T03:10:38.521758+0000","last_undegraded":"2023-12-01T03:10:38.521758+0000","last_fullsized":"2023-12-01T03:10:38.521758+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:40.686905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.502663+0000","last_change":"2023-12-01T03:10:38.502663+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.502663+0000","last_undegraded":"2023-12-01T03:10:38.502663+0000","last_fullsized":"2023-12-01T03:10:38.502663+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491751+0000","last_change":"2023-12-01T03:10:24.685823+0000","last_active":"2023-12-01T03:10:37.491751+0000","last_peered":"2023-12-01T03:10:37.491751+0000","last_clean":"2023-12-01T03:10:37.491751+0000","last_became_active":"2023-12-01T03:10:24.685560+0000","last_became_peered":"2023-12-01T03:10:24.685560+0000","last_unstale":"2023-12-01T03:10:37.491751+0000","last_undegraded":"2023-12-01T03:10:37.491751+0000","last_fullsized":"2023-12-01T03:10:37.491751+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483380+0000","last_change":"2023-12-01T03:10:25.183804+0000","last_active":"2023-12-01T03:10:37.483380+0000","last_peered":"2023-12-01T03:10:37.483380+0000","last_clean":"2023-12-01T03:10:37.483380+0000","last_became_active":"2023-12-01T03:10:25.183079+0000","last_became_peered":"2023-12-01T03:10:25.183079+0000","last_unstale":"2023-12-01T03:10:37.483380+0000","last_undegraded":"2023-12-01T03:10:37.483380+0000","last_fullsized":"2023-12-01T03:10:37.483380+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.510374+0000","last_change":"2023-12-01T03:10:38.508144+0000","last_active":"2023-12-01T03:10:38.510374+0000","last_peered":"2023-12-01T03:10:38.510374+0000","last_clean":"2023-12-01T03:10:38.510374+0000","last_became_active":"2023-12-01T03:10:38.507850+0000","last_became_peered":"2023-12-01T03:10:38.507850+0000","last_unstale":"2023-12-01T03:10:38.510374+0000","last_undegraded":"2023-12-01T03:10:38.510374+0000","last_fullsized":"2023-12-01T03:10:38.510374+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:52.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507590+0000","last_change":"2023-12-01T03:10:38.507590+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507590+0000","last_undegraded":"2023-12-01T03:10:38.507590+0000","last_fullsized":"2023-12-01T03:10:38.507590+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:45.156063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.13","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987606+0000","last_change":"2023-12-01T03:10:24.692854+0000","last_active":"2023-12-01T03:10:38.987606+0000","last_peered":"2023-12-01T03:10:38.987606+0000","last_clean":"2023-12-01T03:10:38.987606+0000","last_became_active":"2023-12-01T03:10:24.692563+0000","last_became_peered":"2023-12-01T03:10:24.692563+0000","last_unstale":"2023-12-01T03:10:38.987606+0000","last_undegraded":"2023-12-01T03:10:38.987606+0000","last_fullsized":"2023-12-01T03:10:38.987606+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491687+0000","last_change":"2023-12-01T03:10:25.186716+0000","last_active":"2023-12-01T03:10:37.491687+0000","last_peered":"2023-12-01T03:10:37.491687+0000","last_clean":"2023-12-01T03:10:37.491687+0000","last_became_active":"2023-12-01T03:10:25.186422+0000","last_became_peered":"2023-12-01T03:10:25.186422+0000","last_unstale":"2023-12-01T03:10:37.491687+0000","last_undegraded":"2023-12-01T03:10:37.491687+0000","last_fullsized":"2023-12-01T03:10:37.491687+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.510490+0000","last_change":"2023-12-01T03:10:38.508098+0000","last_active":"2023-12-01T03:10:38.510490+0000","last_peered":"2023-12-01T03:10:38.510490+0000","last_clean":"2023-12-01T03:10:38.510490+0000","last_became_active":"2023-12-01T03:10:38.506510+0000","last_became_peered":"2023-12-01T03:10:38.506510+0000","last_unstale":"2023-12-01T03:10:38.510490+0000","last_undegraded":"2023-12-01T03:10:38.510490+0000","last_fullsized":"2023-12-01T03:10:38.510490+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:01.074406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504758+0000","last_change":"2023-12-01T03:10:38.504758+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504758+0000","last_undegraded":"2023-12-01T03:10:38.504758+0000","last_fullsized":"2023-12-01T03:10:38.504758+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:18.605693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"214'8","reported_seq":21,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481263+0000","last_change":"2023-12-01T03:10:24.697328+0000","last_active":"2023-12-01T03:10:37.481263+0000","last_peered":"2023-12-01T03:10:37.481263+0000","last_clean":"2023-12-01T03:10:37.481263+0000","last_became_active":"2023-12-01T03:10:24.696986+0000","last_became_peered":"2023-12-01T03:10:24.696986+0000","last_unstale":"2023-12-01T03:10:37.481263+0000","last_undegraded":"2023-12-01T03:10:37.481263+0000","last_fullsized":"2023-12-01T03:10:37.481263+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492498+0000","last_change":"2023-12-01T03:10:25.176864+0000","last_active":"2023-12-01T03:10:37.492498+0000","last_peered":"2023-12-01T03:10:37.492498+0000","last_clean":"2023-12-01T03:10:37.492498+0000","last_became_active":"2023-12-01T03:10:25.176588+0000","last_became_peered":"2023-12-01T03:10:25.176588+0000","last_unstale":"2023-12-01T03:10:37.492498+0000","last_undegraded":"2023-12-01T03:10:37.492498+0000","last_fullsized":"2023-12-01T03:10:37.492498+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987075+0000","last_change":"2023-12-01T03:10:38.519450+0000","last_active":"2023-12-01T03:10:38.987075+0000","last_peered":"2023-12-01T03:10:38.987075+0000","last_clean":"2023-12-01T03:10:38.987075+0000","last_became_active":"2023-12-01T03:10:38.519108+0000","last_became_peered":"2023-12-01T03:10:38.519108+0000","last_unstale":"2023-12-01T03:10:38.987075+0000","last_undegraded":"2023-12-01T03:10:38.987075+0000","last_fullsized":"2023-12-01T03:10:38.987075+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:58.926662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007897+0000","last_change":"2023-12-01T03:10:39.005006+0000","last_active":"2023-12-01T03:10:39.007897+0000","last_peered":"2023-12-01T03:10:39.007897+0000","last_clean":"2023-12-01T03:10:39.007897+0000","last_became_active":"2023-12-01T03:10:39.004573+0000","last_became_peered":"2023-12-01T03:10:39.004573+0000","last_unstale":"2023-12-01T03:10:39.007897+0000","last_undegraded":"2023-12-01T03:10:39.007897+0000","last_fullsized":"2023-12-01T03:10:39.007897+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:07.829269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"213'3","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481279+0000","last_change":"2023-12-01T03:10:24.699528+0000","last_active":"2023-12-01T03:10:37.481279+0000","last_peered":"2023-12-01T03:10:37.481279+0000","last_clean":"2023-12-01T03:10:37.481279+0000","last_became_active":"2023-12-01T03:10:24.699152+0000","last_became_peered":"2023-12-01T03:10:24.699152+0000","last_unstale":"2023-12-01T03:10:37.481279+0000","last_undegraded":"2023-12-01T03:10:37.481279+0000","last_fullsized":"2023-12-01T03:10:37.481279+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:07.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987736+0000","last_change":"2023-12-01T03:10:25.187487+0000","last_active":"2023-12-01T03:10:38.987736+0000","last_peered":"2023-12-01T03:10:38.987736+0000","last_clean":"2023-12-01T03:10:38.987736+0000","last_became_active":"2023-12-01T03:10:25.187198+0000","last_became_peered":"2023-12-01T03:10:25.187198+0000","last_unstale":"2023-12-01T03:10:38.987736+0000","last_undegraded":"2023-12-01T03:10:38.987736+0000","last_fullsized":"2023-12-01T03:10:38.987736+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:00.948964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.526023+0000","last_change":"2023-12-01T03:10:38.525203+0000","last_active":"2023-12-01T03:10:38.526023+0000","last_peered":"2023-12-01T03:10:38.526023+0000","last_clean":"2023-12-01T03:10:38.526023+0000","last_became_active":"2023-12-01T03:10:38.524926+0000","last_became_peered":"2023-12-01T03:10:38.524926+0000","last_unstale":"2023-12-01T03:10:38.526023+0000","last_undegraded":"2023-12-01T03:10:38.526023+0000","last_fullsized":"2023-12-01T03:10:38.526023+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:17.593868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.503576+0000","last_change":"2023-12-01T03:10:38.503576+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.503576+0000","last_undegraded":"2023-12-01T03:10:38.503576+0000","last_fullsized":"2023-12-01T03:10:38.503576+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492851+0000","last_change":"2023-12-01T03:10:24.691537+0000","last_active":"2023-12-01T03:10:37.492851+0000","last_peered":"2023-12-01T03:10:37.492851+0000","last_clean":"2023-12-01T03:10:37.492851+0000","last_became_active":"2023-12-01T03:10:24.691168+0000","last_became_peered":"2023-12-01T03:10:24.691168+0000","last_unstale":"2023-12-01T03:10:37.492851+0000","last_undegraded":"2023-12-01T03:10:37.492851+0000","last_fullsized":"2023-12-01T03:10:37.492851+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492811+0000","last_change":"2023-12-01T03:10:25.178648+0000","last_active":"2023-12-01T03:10:37.492811+0000","last_peered":"2023-12-01T03:10:37.492811+0000","last_clean":"2023-12-01T03:10:37.492811+0000","last_became_active":"2023-12-01T03:10:25.178352+0000","last_became_peered":"2023-12-01T03:10:25.178352+0000","last_unstale":"2023-12-01T03:10:37.492811+0000","last_undegraded":"2023-12-01T03:10:37.492811+0000","last_fullsized":"2023-12-01T03:10:37.492811+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519785+0000","last_change":"2023-12-01T03:10:38.515822+0000","last_active":"2023-12-01T03:10:38.519785+0000","last_peered":"2023-12-01T03:10:38.519785+0000","last_clean":"2023-12-01T03:10:38.519785+0000","last_became_active":"2023-12-01T03:10:38.515536+0000","last_became_peered":"2023-12-01T03:10:38.515536+0000","last_unstale":"2023-12-01T03:10:38.519785+0000","last_undegraded":"2023-12-01T03:10:38.519785+0000","last_fullsized":"2023-12-01T03:10:38.519785+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:10:36.574945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.510307+0000","last_change":"2023-12-01T03:10:38.510307+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.510307+0000","last_undegraded":"2023-12-01T03:10:38.510307+0000","last_fullsized":"2023-12-01T03:10:38.510307+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472906+0000","last_change":"2023-12-01T03:10:24.687709+0000","last_active":"2023-12-01T03:10:37.472906+0000","last_peered":"2023-12-01T03:10:37.472906+0000","last_clean":"2023-12-01T03:10:37.472906+0000","last_became_active":"2023-12-01T03:10:24.687452+0000","last_became_peered":"2023-12-01T03:10:24.687452+0000","last_unstale":"2023-12-01T03:10:37.472906+0000","last_undegraded":"2023-12-01T03:10:37.472906+0000","last_fullsized":"2023-12-01T03:10:37.472906+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:39.126749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489332+0000","last_change":"2023-12-01T03:10:25.176941+0000","last_active":"2023-12-01T03:10:37.489332+0000","last_peered":"2023-12-01T03:10:37.489332+0000","last_clean":"2023-12-01T03:10:37.489332+0000","last_became_active":"2023-12-01T03:10:25.175692+0000","last_became_peered":"2023-12-01T03:10:25.175692+0000","last_unstale":"2023-12-01T03:10:37.489332+0000","last_undegraded":"2023-12-01T03:10:37.489332+0000","last_fullsized":"2023-12-01T03:10:37.489332+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.513115+0000","last_change":"2023-12-01T03:10:38.509844+0000","last_active":"2023-12-01T03:10:38.513115+0000","last_peered":"2023-12-01T03:10:38.513115+0000","last_clean":"2023-12-01T03:10:38.513115+0000","last_became_active":"2023-12-01T03:10:38.509319+0000","last_became_peered":"2023-12-01T03:10:38.509319+0000","last_unstale":"2023-12-01T03:10:38.513115+0000","last_undegraded":"2023-12-01T03:10:38.513115+0000","last_fullsized":"2023-12-01T03:10:38.513115+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:12.432439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012565+0000","last_change":"2023-12-01T03:10:39.010700+0000","last_active":"2023-12-01T03:10:39.012565+0000","last_peered":"2023-12-01T03:10:39.012565+0000","last_clean":"2023-12-01T03:10:39.012565+0000","last_became_active":"2023-12-01T03:10:39.010389+0000","last_became_peered":"2023-12-01T03:10:39.010389+0000","last_unstale":"2023-12-01T03:10:39.012565+0000","last_undegraded":"2023-12-01T03:10:39.012565+0000","last_fullsized":"2023-12-01T03:10:39.012565+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:53.973502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483145+0000","last_change":"2023-12-01T03:10:24.686553+0000","last_active":"2023-12-01T03:10:37.483145+0000","last_peered":"2023-12-01T03:10:37.483145+0000","last_clean":"2023-12-01T03:10:37.483145+0000","last_became_active":"2023-12-01T03:10:24.686136+0000","last_became_peered":"2023-12-01T03:10:24.686136+0000","last_unstale":"2023-12-01T03:10:37.483145+0000","last_undegraded":"2023-12-01T03:10:37.483145+0000","last_fullsized":"2023-12-01T03:10:37.483145+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:18.663097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472405+0000","last_change":"2023-12-01T03:10:25.182793+0000","last_active":"2023-12-01T03:10:37.472405+0000","last_peered":"2023-12-01T03:10:37.472405+0000","last_clean":"2023-12-01T03:10:37.472405+0000","last_became_active":"2023-12-01T03:10:25.182524+0000","last_became_peered":"2023-12-01T03:10:25.182524+0000","last_unstale":"2023-12-01T03:10:37.472405+0000","last_undegraded":"2023-12-01T03:10:37.472405+0000","last_fullsized":"2023-12-01T03:10:37.472405+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:25.370249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525096+0000","last_change":"2023-12-01T03:10:38.523078+0000","last_active":"2023-12-01T03:10:38.525096+0000","last_peered":"2023-12-01T03:10:38.525096+0000","last_clean":"2023-12-01T03:10:38.525096+0000","last_became_active":"2023-12-01T03:10:38.522603+0000","last_became_peered":"2023-12-01T03:10:38.522603+0000","last_unstale":"2023-12-01T03:10:38.525096+0000","last_undegraded":"2023-12-01T03:10:38.525096+0000","last_fullsized":"2023-12-01T03:10:38.525096+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:12.432439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.506778+0000","last_change":"2023-12-01T03:10:38.506778+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.506778+0000","last_undegraded":"2023-12-01T03:10:38.506778+0000","last_fullsized":"2023-12-01T03:10:38.506778+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1a","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480595+0000","last_change":"2023-12-01T03:10:24.695315+0000","last_active":"2023-12-01T03:10:37.480595+0000","last_peered":"2023-12-01T03:10:37.480595+0000","last_clean":"2023-12-01T03:10:37.480595+0000","last_became_active":"2023-12-01T03:10:24.694274+0000","last_became_peered":"2023-12-01T03:10:24.694274+0000","last_unstale":"2023-12-01T03:10:37.480595+0000","last_undegraded":"2023-12-01T03:10:37.480595+0000","last_fullsized":"2023-12-01T03:10:37.480595+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:19.256208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479974+0000","last_change":"2023-12-01T03:10:25.188350+0000","last_active":"2023-12-01T03:10:37.479974+0000","last_peered":"2023-12-01T03:10:37.479974+0000","last_clean":"2023-12-01T03:10:37.479974+0000","last_became_active":"2023-12-01T03:10:25.187212+0000","last_became_peered":"2023-12-01T03:10:25.187212+0000","last_unstale":"2023-12-01T03:10:37.479974+0000","last_undegraded":"2023-12-01T03:10:37.479974+0000","last_fullsized":"2023-12-01T03:10:37.479974+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.981897+0000","last_change":"2023-12-01T03:10:38.521794+0000","last_active":"2023-12-01T03:10:38.981897+0000","last_peered":"2023-12-01T03:10:38.981897+0000","last_clean":"2023-12-01T03:10:38.981897+0000","last_became_active":"2023-12-01T03:10:38.521546+0000","last_became_peered":"2023-12-01T03:10:38.521546+0000","last_unstale":"2023-12-01T03:10:38.981897+0000","last_undegraded":"2023-12-01T03:10:38.981897+0000","last_fullsized":"2023-12-01T03:10:38.981897+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:17.386276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.510675+0000","last_change":"2023-12-01T03:10:38.510675+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.510675+0000","last_undegraded":"2023-12-01T03:10:38.510675+0000","last_fullsized":"2023-12-01T03:10:38.510675+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:33:51.590745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480220+0000","last_change":"2023-12-01T03:10:24.695067+0000","last_active":"2023-12-01T03:10:37.480220+0000","last_peered":"2023-12-01T03:10:37.480220+0000","last_clean":"2023-12-01T03:10:37.480220+0000","last_became_active":"2023-12-01T03:10:24.694560+0000","last_became_peered":"2023-12-01T03:10:24.694560+0000","last_unstale":"2023-12-01T03:10:37.480220+0000","last_undegraded":"2023-12-01T03:10:37.480220+0000","last_fullsized":"2023-12-01T03:10:37.480220+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482278+0000","last_change":"2023-12-01T03:10:25.175874+0000","last_active":"2023-12-01T03:10:37.482278+0000","last_peered":"2023-12-01T03:10:37.482278+0000","last_clean":"2023-12-01T03:10:37.482278+0000","last_became_active":"2023-12-01T03:10:25.175584+0000","last_became_peered":"2023-12-01T03:10:25.175584+0000","last_unstale":"2023-12-01T03:10:37.482278+0000","last_undegraded":"2023-12-01T03:10:37.482278+0000","last_fullsized":"2023-12-01T03:10:37.482278+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:46.157610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519254+0000","last_change":"2023-12-01T03:10:38.515775+0000","last_active":"2023-12-01T03:10:38.519254+0000","last_peered":"2023-12-01T03:10:38.519254+0000","last_clean":"2023-12-01T03:10:38.519254+0000","last_became_active":"2023-12-01T03:10:38.515021+0000","last_became_peered":"2023-12-01T03:10:38.515021+0000","last_unstale":"2023-12-01T03:10:38.519254+0000","last_undegraded":"2023-12-01T03:10:38.519254+0000","last_fullsized":"2023-12-01T03:10:38.519254+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:35.878213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003601+0000","last_change":"2023-12-01T03:10:39.000307+0000","last_active":"2023-12-01T03:10:39.003601+0000","last_peered":"2023-12-01T03:10:39.003601+0000","last_clean":"2023-12-01T03:10:39.003601+0000","last_became_active":"2023-12-01T03:10:38.997156+0000","last_became_peered":"2023-12-01T03:10:38.997156+0000","last_unstale":"2023-12-01T03:10:39.003601+0000","last_undegraded":"2023-12-01T03:10:39.003601+0000","last_fullsized":"2023-12-01T03:10:39.003601+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:59.987290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515498+0000","last_change":"2023-12-01T03:10:38.513581+0000","last_active":"2023-12-01T03:10:38.515498+0000","last_peered":"2023-12-01T03:10:38.515498+0000","last_clean":"2023-12-01T03:10:38.515498+0000","last_became_active":"2023-12-01T03:10:38.513306+0000","last_became_peered":"2023-12-01T03:10:38.513306+0000","last_unstale":"2023-12-01T03:10:38.515498+0000","last_undegraded":"2023-12-01T03:10:38.515498+0000","last_fullsized":"2023-12-01T03:10:38.515498+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:13.957769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987848+0000","last_change":"2023-12-01T03:10:24.686931+0000","last_active":"2023-12-01T03:10:38.987848+0000","last_peered":"2023-12-01T03:10:38.987848+0000","last_clean":"2023-12-01T03:10:38.987848+0000","last_became_active":"2023-12-01T03:10:24.686642+0000","last_became_peered":"2023-12-01T03:10:24.686642+0000","last_unstale":"2023-12-01T03:10:38.987848+0000","last_undegraded":"2023-12-01T03:10:38.987848+0000","last_fullsized":"2023-12-01T03:10:38.987848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983478+0000","last_change":"2023-12-01T03:10:25.175999+0000","last_active":"2023-12-01T03:10:38.983478+0000","last_peered":"2023-12-01T03:10:38.983478+0000","last_clean":"2023-12-01T03:10:38.983478+0000","last_became_active":"2023-12-01T03:10:25.175764+0000","last_became_peered":"2023-12-01T03:10:25.175764+0000","last_unstale":"2023-12-01T03:10:38.983478+0000","last_undegraded":"2023-12-01T03:10:38.983478+0000","last_fullsized":"2023-12-01T03:10:38.983478+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:43.098776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012583+0000","last_change":"2023-12-01T03:10:39.009798+0000","last_active":"2023-12-01T03:10:39.012583+0000","last_peered":"2023-12-01T03:10:39.012583+0000","last_clean":"2023-12-01T03:10:39.012583+0000","last_became_active":"2023-12-01T03:10:39.009564+0000","last_became_peered":"2023-12-01T03:10:39.009564+0000","last_unstale":"2023-12-01T03:10:39.012583+0000","last_undegraded":"2023-12-01T03:10:39.012583+0000","last_fullsized":"2023-12-01T03:10:39.012583+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"217'4","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.011227+0000","last_change":"2023-12-01T03:10:38.520344+0000","last_active":"2023-12-01T03:10:39.011227+0000","last_peered":"2023-12-01T03:10:39.011227+0000","last_clean":"2023-12-01T03:10:39.011227+0000","last_became_active":"2023-12-01T03:10:38.520086+0000","last_became_peered":"2023-12-01T03:10:38.520086+0000","last_unstale":"2023-12-01T03:10:39.011227+0000","last_undegraded":"2023-12-01T03:10:39.011227+0000","last_fullsized":"2023-12-01T03:10:39.011227+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:01.616973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480900+0000","last_change":"2023-12-01T03:10:24.700378+0000","last_active":"2023-12-01T03:10:37.480900+0000","last_peered":"2023-12-01T03:10:37.480900+0000","last_clean":"2023-12-01T03:10:37.480900+0000","last_became_active":"2023-12-01T03:10:24.700126+0000","last_became_peered":"2023-12-01T03:10:24.700126+0000","last_unstale":"2023-12-01T03:10:37.480900+0000","last_undegraded":"2023-12-01T03:10:37.480900+0000","last_fullsized":"2023-12-01T03:10:37.480900+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999828+0000","last_change":"2023-12-01T03:10:25.179131+0000","last_active":"2023-12-01T03:10:38.999828+0000","last_peered":"2023-12-01T03:10:38.999828+0000","last_clean":"2023-12-01T03:10:38.999828+0000","last_became_active":"2023-12-01T03:10:25.178708+0000","last_became_peered":"2023-12-01T03:10:25.178708+0000","last_unstale":"2023-12-01T03:10:38.999828+0000","last_undegraded":"2023-12-01T03:10:38.999828+0000","last_fullsized":"2023-12-01T03:10:38.999828+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:43.370389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513293+0000","last_change":"2023-12-01T03:10:38.513293+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513293+0000","last_undegraded":"2023-12-01T03:10:38.513293+0000","last_fullsized":"2023-12-01T03:10:38.513293+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:46:18.989087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1e","version":"213'1","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989529+0000","last_change":"2023-12-01T03:10:24.688812+0000","last_active":"2023-12-01T03:10:38.989529+0000","last_peered":"2023-12-01T03:10:38.989529+0000","last_clean":"2023-12-01T03:10:38.989529+0000","last_became_active":"2023-12-01T03:10:24.687577+0000","last_became_peered":"2023-12-01T03:10:24.687577+0000","last_unstale":"2023-12-01T03:10:38.989529+0000","last_undegraded":"2023-12-01T03:10:38.989529+0000","last_fullsized":"2023-12-01T03:10:38.989529+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492668+0000","last_change":"2023-12-01T03:10:25.180335+0000","last_active":"2023-12-01T03:10:37.492668+0000","last_peered":"2023-12-01T03:10:37.492668+0000","last_clean":"2023-12-01T03:10:37.492668+0000","last_became_active":"2023-12-01T03:10:25.180020+0000","last_became_peered":"2023-12-01T03:10:25.180020+0000","last_unstale":"2023-12-01T03:10:37.492668+0000","last_undegraded":"2023-12-01T03:10:37.492668+0000","last_fullsized":"2023-12-01T03:10:37.492668+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"217'2","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.006339+0000","last_change":"2023-12-01T03:10:38.519061+0000","last_active":"2023-12-01T03:10:39.006339+0000","last_peered":"2023-12-01T03:10:39.006339+0000","last_clean":"2023-12-01T03:10:39.006339+0000","last_became_active":"2023-12-01T03:10:38.518091+0000","last_became_peered":"2023-12-01T03:10:38.518091+0000","last_unstale":"2023-12-01T03:10:39.006339+0000","last_undegraded":"2023-12-01T03:10:39.006339+0000","last_fullsized":"2023-12-01T03:10:39.006339+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:42.963999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512195+0000","last_change":"2023-12-01T03:10:38.512195+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512195+0000","last_undegraded":"2023-12-01T03:10:38.512195+0000","last_fullsized":"2023-12-01T03:10:38.512195+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"213'4","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489298+0000","last_change":"2023-12-01T03:10:24.697099+0000","last_active":"2023-12-01T03:10:37.489298+0000","last_peered":"2023-12-01T03:10:37.489298+0000","last_clean":"2023-12-01T03:10:37.489298+0000","last_became_active":"2023-12-01T03:10:24.696783+0000","last_became_peered":"2023-12-01T03:10:24.696783+0000","last_unstale":"2023-12-01T03:10:37.489298+0000","last_undegraded":"2023-12-01T03:10:37.489298+0000","last_fullsized":"2023-12-01T03:10:37.489298+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:07.101180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479580+0000","last_change":"2023-12-01T03:10:25.191865+0000","last_active":"2023-12-01T03:10:37.479580+0000","last_peered":"2023-12-01T03:10:37.479580+0000","last_clean":"2023-12-01T03:10:37.479580+0000","last_became_active":"2023-12-01T03:10:25.189277+0000","last_became_peered":"2023-12-01T03:10:25.189277+0000","last_unstale":"2023-12-01T03:10:37.479580+0000","last_undegraded":"2023-12-01T03:10:37.479580+0000","last_fullsized":"2023-12-01T03:10:37.479580+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521580+0000","last_change":"2023-12-01T03:10:38.517851+0000","last_active":"2023-12-01T03:10:38.521580+0000","last_peered":"2023-12-01T03:10:38.521580+0000","last_clean":"2023-12-01T03:10:38.521580+0000","last_became_active":"2023-12-01T03:10:38.517507+0000","last_became_peered":"2023-12-01T03:10:38.517507+0000","last_unstale":"2023-12-01T03:10:38.521580+0000","last_undegraded":"2023-12-01T03:10:38.521580+0000","last_fullsized":"2023-12-01T03:10:38.521580+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:22.592854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008442+0000","last_change":"2023-12-01T03:10:39.005982+0000","last_active":"2023-12-01T03:10:39.008442+0000","last_peered":"2023-12-01T03:10:39.008442+0000","last_clean":"2023-12-01T03:10:39.008442+0000","last_became_active":"2023-12-01T03:10:39.005729+0000","last_became_peered":"2023-12-01T03:10:39.005729+0000","last_unstale":"2023-12-01T03:10:39.008442+0000","last_undegraded":"2023-12-01T03:10:39.008442+0000","last_fullsized":"2023-12-01T03:10:39.008442+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:36.187885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478313+0000","last_change":"2023-12-01T03:10:25.185385+0000","last_active":"2023-12-01T03:10:37.478313+0000","last_peered":"2023-12-01T03:10:37.478313+0000","last_clean":"2023-12-01T03:10:37.478313+0000","last_became_active":"2023-12-01T03:10:25.185098+0000","last_became_peered":"2023-12-01T03:10:25.185098+0000","last_unstale":"2023-12-01T03:10:37.478313+0000","last_undegraded":"2023-12-01T03:10:37.478313+0000","last_fullsized":"2023-12-01T03:10:37.478313+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500232+0000","last_change":"2023-12-01T03:10:38.500232+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500232+0000","last_undegraded":"2023-12-01T03:10:38.500232+0000","last_fullsized":"2023-12-01T03:10:38.500232+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478702+0000","last_change":"2023-12-01T03:10:25.186126+0000","last_active":"2023-12-01T03:10:37.478702+0000","last_peered":"2023-12-01T03:10:37.478702+0000","last_clean":"2023-12-01T03:10:37.478702+0000","last_became_active":"2023-12-01T03:10:25.185741+0000","last_became_peered":"2023-12-01T03:10:25.185741+0000","last_unstale":"2023-12-01T03:10:37.478702+0000","last_undegraded":"2023-12-01T03:10:37.478702+0000","last_fullsized":"2023-12-01T03:10:37.478702+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.501528+0000","last_change":"2023-12-01T03:10:38.501528+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.501528+0000","last_undegraded":"2023-12-01T03:10:38.501528+0000","last_fullsized":"2023-12-01T03:10:38.501528+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985671+0000","last_change":"2023-12-01T03:10:25.175090+0000","last_active":"2023-12-01T03:10:38.985671+0000","last_peered":"2023-12-01T03:10:38.985671+0000","last_clean":"2023-12-01T03:10:38.985671+0000","last_became_active":"2023-12-01T03:10:25.174835+0000","last_became_peered":"2023-12-01T03:10:25.174835+0000","last_unstale":"2023-12-01T03:10:38.985671+0000","last_undegraded":"2023-12-01T03:10:38.985671+0000","last_fullsized":"2023-12-01T03:10:38.985671+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.505382+0000","last_change":"2023-12-01T03:10:38.505382+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.505382+0000","last_undegraded":"2023-12-01T03:10:38.505382+0000","last_fullsized":"2023-12-01T03:10:38.505382+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:52.701542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987378+0000","last_change":"2023-12-01T03:10:25.178098+0000","last_active":"2023-12-01T03:10:38.987378+0000","last_peered":"2023-12-01T03:10:38.987378+0000","last_clean":"2023-12-01T03:10:38.987378+0000","last_became_active":"2023-12-01T03:10:25.177031+0000","last_became_peered":"2023-12-01T03:10:25.177031+0000","last_unstale":"2023-12-01T03:10:38.987378+0000","last_undegraded":"2023-12-01T03:10:38.987378+0000","last_fullsized":"2023-12-01T03:10:38.987378+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.496510+0000","last_change":"2023-12-01T03:10:38.496510+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.496510+0000","last_undegraded":"2023-12-01T03:10:38.496510+0000","last_fullsized":"2023-12-01T03:10:38.496510+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491934+0000","last_change":"2023-12-01T03:10:25.176284+0000","last_active":"2023-12-01T03:10:37.491934+0000","last_peered":"2023-12-01T03:10:37.491934+0000","last_clean":"2023-12-01T03:10:37.491934+0000","last_became_active":"2023-12-01T03:10:25.175494+0000","last_became_peered":"2023-12-01T03:10:25.175494+0000","last_unstale":"2023-12-01T03:10:37.491934+0000","last_undegraded":"2023-12-01T03:10:37.491934+0000","last_fullsized":"2023-12-01T03:10:37.491934+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008362+0000","last_change":"2023-12-01T03:10:39.006064+0000","last_active":"2023-12-01T03:10:39.008362+0000","last_peered":"2023-12-01T03:10:39.008362+0000","last_clean":"2023-12-01T03:10:39.008362+0000","last_became_active":"2023-12-01T03:10:39.005720+0000","last_became_peered":"2023-12-01T03:10:39.005720+0000","last_unstale":"2023-12-01T03:10:39.008362+0000","last_undegraded":"2023-12-01T03:10:39.008362+0000","last_fullsized":"2023-12-01T03:10:39.008362+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:02.628798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488821+0000","last_change":"2023-12-01T03:10:25.178072+0000","last_active":"2023-12-01T03:10:37.488821+0000","last_peered":"2023-12-01T03:10:37.488821+0000","last_clean":"2023-12-01T03:10:37.488821+0000","last_became_active":"2023-12-01T03:10:25.177778+0000","last_became_peered":"2023-12-01T03:10:25.177778+0000","last_unstale":"2023-12-01T03:10:37.488821+0000","last_undegraded":"2023-12-01T03:10:37.488821+0000","last_fullsized":"2023-12-01T03:10:37.488821+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:48.799119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.496580+0000","last_change":"2023-12-01T03:10:38.496580+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.496580+0000","last_undegraded":"2023-12-01T03:10:38.496580+0000","last_fullsized":"2023-12-01T03:10:38.496580+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989375+0000","last_change":"2023-12-01T03:10:25.181469+0000","last_active":"2023-12-01T03:10:38.989375+0000","last_peered":"2023-12-01T03:10:38.989375+0000","last_clean":"2023-12-01T03:10:38.989375+0000","last_became_active":"2023-12-01T03:10:25.180646+0000","last_became_peered":"2023-12-01T03:10:25.180646+0000","last_unstale":"2023-12-01T03:10:38.989375+0000","last_undegraded":"2023-12-01T03:10:38.989375+0000","last_fullsized":"2023-12-01T03:10:38.989375+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:53.001661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008512+0000","last_change":"2023-12-01T03:10:39.006804+0000","last_active":"2023-12-01T03:10:39.008512+0000","last_peered":"2023-12-01T03:10:39.008512+0000","last_clean":"2023-12-01T03:10:39.008512+0000","last_became_active":"2023-12-01T03:10:39.006552+0000","last_became_peered":"2023-12-01T03:10:39.006552+0000","last_unstale":"2023-12-01T03:10:39.008512+0000","last_undegraded":"2023-12-01T03:10:39.008512+0000","last_fullsized":"2023-12-01T03:10:39.008512+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.992143+0000","last_change":"2023-12-01T03:10:25.179679+0000","last_active":"2023-12-01T03:10:38.992143+0000","last_peered":"2023-12-01T03:10:38.992143+0000","last_clean":"2023-12-01T03:10:38.992143+0000","last_became_active":"2023-12-01T03:10:25.179442+0000","last_became_peered":"2023-12-01T03:10:25.179442+0000","last_unstale":"2023-12-01T03:10:38.992143+0000","last_undegraded":"2023-12-01T03:10:38.992143+0000","last_fullsized":"2023-12-01T03:10:38.992143+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:57:25.728865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500994+0000","last_change":"2023-12-01T03:10:38.500994+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500994+0000","last_undegraded":"2023-12-01T03:10:38.500994+0000","last_fullsized":"2023-12-01T03:10:38.500994+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478175+0000","last_change":"2023-12-01T03:10:25.175356+0000","last_active":"2023-12-01T03:10:37.478175+0000","last_peered":"2023-12-01T03:10:37.478175+0000","last_clean":"2023-12-01T03:10:37.478175+0000","last_became_active":"2023-12-01T03:10:25.175086+0000","last_became_peered":"2023-12-01T03:10:25.175086+0000","last_unstale":"2023-12-01T03:10:37.478175+0000","last_undegraded":"2023-12-01T03:10:37.478175+0000","last_fullsized":"2023-12-01T03:10:37.478175+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.000224+0000","last_change":"2023-12-01T03:10:38.998643+0000","last_active":"2023-12-01T03:10:39.000224+0000","last_peered":"2023-12-01T03:10:39.000224+0000","last_clean":"2023-12-01T03:10:39.000224+0000","last_became_active":"2023-12-01T03:10:38.998393+0000","last_became_peered":"2023-12-01T03:10:38.998393+0000","last_unstale":"2023-12-01T03:10:39.000224+0000","last_undegraded":"2023-12-01T03:10:39.000224+0000","last_fullsized":"2023-12-01T03:10:39.000224+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:38.199716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491490+0000","last_change":"2023-12-01T03:10:25.181678+0000","last_active":"2023-12-01T03:10:37.491490+0000","last_peered":"2023-12-01T03:10:37.491490+0000","last_clean":"2023-12-01T03:10:37.491490+0000","last_became_active":"2023-12-01T03:10:25.181430+0000","last_became_peered":"2023-12-01T03:10:25.181430+0000","last_unstale":"2023-12-01T03:10:37.491490+0000","last_undegraded":"2023-12-01T03:10:37.491490+0000","last_fullsized":"2023-12-01T03:10:37.491490+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.506833+0000","last_change":"2023-12-01T03:10:38.506833+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.506833+0000","last_undegraded":"2023-12-01T03:10:38.506833+0000","last_fullsized":"2023-12-01T03:10:38.506833+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:57:07.895399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480143+0000","last_change":"2023-12-01T03:10:25.188998+0000","last_active":"2023-12-01T03:10:37.480143+0000","last_peered":"2023-12-01T03:10:37.480143+0000","last_clean":"2023-12-01T03:10:37.480143+0000","last_became_active":"2023-12-01T03:10:25.188657+0000","last_became_peered":"2023-12-01T03:10:25.188657+0000","last_unstale":"2023-12-01T03:10:37.480143+0000","last_undegraded":"2023-12-01T03:10:37.480143+0000","last_fullsized":"2023-12-01T03:10:37.480143+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:23:58.752943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003091+0000","last_change":"2023-12-01T03:10:38.997151+0000","last_active":"2023-12-01T03:10:39.003091+0000","last_peered":"2023-12-01T03:10:39.003091+0000","last_clean":"2023-12-01T03:10:39.003091+0000","last_became_active":"2023-12-01T03:10:38.989885+0000","last_became_peered":"2023-12-01T03:10:38.989885+0000","last_unstale":"2023-12-01T03:10:39.003091+0000","last_undegraded":"2023-12-01T03:10:39.003091+0000","last_fullsized":"2023-12-01T03:10:39.003091+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:25:51.393448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485796+0000","last_change":"2023-12-01T03:10:25.174568+0000","last_active":"2023-12-01T03:10:37.485796+0000","last_peered":"2023-12-01T03:10:37.485796+0000","last_clean":"2023-12-01T03:10:37.485796+0000","last_became_active":"2023-12-01T03:10:25.174229+0000","last_became_peered":"2023-12-01T03:10:25.174229+0000","last_unstale":"2023-12-01T03:10:37.485796+0000","last_undegraded":"2023-12-01T03:10:37.485796+0000","last_fullsized":"2023-12-01T03:10:37.485796+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:13.528118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512138+0000","last_change":"2023-12-01T03:10:38.512138+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512138+0000","last_undegraded":"2023-12-01T03:10:38.512138+0000","last_fullsized":"2023-12-01T03:10:38.512138+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481725+0000","last_change":"2023-12-01T03:10:25.189264+0000","last_active":"2023-12-01T03:10:37.481725+0000","last_peered":"2023-12-01T03:10:37.481725+0000","last_clean":"2023-12-01T03:10:37.481725+0000","last_became_active":"2023-12-01T03:10:25.189007+0000","last_became_peered":"2023-12-01T03:10:25.189007+0000","last_unstale":"2023-12-01T03:10:37.481725+0000","last_undegraded":"2023-12-01T03:10:37.481725+0000","last_fullsized":"2023-12-01T03:10:37.481725+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:14.783134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511167+0000","last_change":"2023-12-01T03:10:38.511167+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511167+0000","last_undegraded":"2023-12-01T03:10:38.511167+0000","last_fullsized":"2023-12-01T03:10:38.511167+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:38:07.828693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488698+0000","last_change":"2023-12-01T03:10:25.180992+0000","last_active":"2023-12-01T03:10:37.488698+0000","last_peered":"2023-12-01T03:10:37.488698+0000","last_clean":"2023-12-01T03:10:37.488698+0000","last_became_active":"2023-12-01T03:10:25.180712+0000","last_became_peered":"2023-12-01T03:10:25.180712+0000","last_unstale":"2023-12-01T03:10:37.488698+0000","last_undegraded":"2023-12-01T03:10:37.488698+0000","last_fullsized":"2023-12-01T03:10:37.488698+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:43.827830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507312+0000","last_change":"2023-12-01T03:10:38.507312+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507312+0000","last_undegraded":"2023-12-01T03:10:38.507312+0000","last_fullsized":"2023-12-01T03:10:38.507312+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:58:42.639310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485503+0000","last_change":"2023-12-01T03:10:25.182676+0000","last_active":"2023-12-01T03:10:37.485503+0000","last_peered":"2023-12-01T03:10:37.485503+0000","last_clean":"2023-12-01T03:10:37.485503+0000","last_became_active":"2023-12-01T03:10:25.182429+0000","last_became_peered":"2023-12-01T03:10:25.182429+0000","last_unstale":"2023-12-01T03:10:37.485503+0000","last_undegraded":"2023-12-01T03:10:37.485503+0000","last_fullsized":"2023-12-01T03:10:37.485503+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508726+0000","last_change":"2023-12-01T03:10:38.508726+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508726+0000","last_undegraded":"2023-12-01T03:10:38.508726+0000","last_fullsized":"2023-12-01T03:10:38.508726+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:54.829130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483923+0000","last_change":"2023-12-01T03:10:25.173100+0000","last_active":"2023-12-01T03:10:37.483923+0000","last_peered":"2023-12-01T03:10:37.483923+0000","last_clean":"2023-12-01T03:10:37.483923+0000","last_became_active":"2023-12-01T03:10:25.172808+0000","last_became_peered":"2023-12-01T03:10:25.172808+0000","last_unstale":"2023-12-01T03:10:37.483923+0000","last_undegraded":"2023-12-01T03:10:37.483923+0000","last_fullsized":"2023-12-01T03:10:37.483923+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507042+0000","last_change":"2023-12-01T03:10:38.507042+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507042+0000","last_undegraded":"2023-12-01T03:10:38.507042+0000","last_fullsized":"2023-12-01T03:10:38.507042+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:48.887816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483463+0000","last_change":"2023-12-01T03:10:25.189550+0000","last_active":"2023-12-01T03:10:37.483463+0000","last_peered":"2023-12-01T03:10:37.483463+0000","last_clean":"2023-12-01T03:10:37.483463+0000","last_became_active":"2023-12-01T03:10:25.189001+0000","last_became_peered":"2023-12-01T03:10:25.189001+0000","last_unstale":"2023-12-01T03:10:37.483463+0000","last_undegraded":"2023-12-01T03:10:37.483463+0000","last_fullsized":"2023-12-01T03:10:37.483463+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003543+0000","last_change":"2023-12-01T03:10:38.997977+0000","last_active":"2023-12-01T03:10:39.003543+0000","last_peered":"2023-12-01T03:10:39.003543+0000","last_clean":"2023-12-01T03:10:39.003543+0000","last_became_active":"2023-12-01T03:10:38.997683+0000","last_became_peered":"2023-12-01T03:10:38.997683+0000","last_unstale":"2023-12-01T03:10:39.003543+0000","last_undegraded":"2023-12-01T03:10:39.003543+0000","last_fullsized":"2023-12-01T03:10:39.003543+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:27:00.020245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472732+0000","last_change":"2023-12-01T03:10:25.170590+0000","last_active":"2023-12-01T03:10:37.472732+0000","last_peered":"2023-12-01T03:10:37.472732+0000","last_clean":"2023-12-01T03:10:37.472732+0000","last_became_active":"2023-12-01T03:10:25.170334+0000","last_became_peered":"2023-12-01T03:10:25.170334+0000","last_unstale":"2023-12-01T03:10:37.472732+0000","last_undegraded":"2023-12-01T03:10:37.472732+0000","last_fullsized":"2023-12-01T03:10:37.472732+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:35.058136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513592+0000","last_change":"2023-12-01T03:10:38.513592+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513592+0000","last_undegraded":"2023-12-01T03:10:38.513592+0000","last_fullsized":"2023-12-01T03:10:38.513592+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986339+0000","last_change":"2023-12-01T03:10:25.173081+0000","last_active":"2023-12-01T03:10:38.986339+0000","last_peered":"2023-12-01T03:10:38.986339+0000","last_clean":"2023-12-01T03:10:38.986339+0000","last_became_active":"2023-12-01T03:10:25.172845+0000","last_became_peered":"2023-12-01T03:10:25.172845+0000","last_unstale":"2023-12-01T03:10:38.986339+0000","last_undegraded":"2023-12-01T03:10:38.986339+0000","last_fullsized":"2023-12-01T03:10:38.986339+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:18.173873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512530+0000","last_change":"2023-12-01T03:10:38.512530+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512530+0000","last_undegraded":"2023-12-01T03:10:38.512530+0000","last_fullsized":"2023-12-01T03:10:38.512530+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:24:41.123796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472369+0000","last_change":"2023-12-01T03:10:25.178225+0000","last_active":"2023-12-01T03:10:37.472369+0000","last_peered":"2023-12-01T03:10:37.472369+0000","last_clean":"2023-12-01T03:10:37.472369+0000","last_became_active":"2023-12-01T03:10:25.177541+0000","last_became_peered":"2023-12-01T03:10:25.177541+0000","last_unstale":"2023-12-01T03:10:37.472369+0000","last_undegraded":"2023-12-01T03:10:37.472369+0000","last_fullsized":"2023-12-01T03:10:37.472369+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:18.470068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.516591+0000","last_change":"2023-12-01T03:10:38.516591+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.516591+0000","last_undegraded":"2023-12-01T03:10:38.516591+0000","last_fullsized":"2023-12-01T03:10:38.516591+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987497+0000","last_change":"2023-12-01T03:10:25.172316+0000","last_active":"2023-12-01T03:10:38.987497+0000","last_peered":"2023-12-01T03:10:38.987497+0000","last_clean":"2023-12-01T03:10:38.987497+0000","last_became_active":"2023-12-01T03:10:25.169042+0000","last_became_peered":"2023-12-01T03:10:25.169042+0000","last_unstale":"2023-12-01T03:10:38.987497+0000","last_undegraded":"2023-12-01T03:10:38.987497+0000","last_fullsized":"2023-12-01T03:10:38.987497+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507019+0000","last_change":"2023-12-01T03:10:38.507019+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507019+0000","last_undegraded":"2023-12-01T03:10:38.507019+0000","last_fullsized":"2023-12-01T03:10:38.507019+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473026+0000","last_change":"2023-12-01T03:10:25.181052+0000","last_active":"2023-12-01T03:10:37.473026+0000","last_peered":"2023-12-01T03:10:37.473026+0000","last_clean":"2023-12-01T03:10:37.473026+0000","last_became_active":"2023-12-01T03:10:25.180810+0000","last_became_peered":"2023-12-01T03:10:25.180810+0000","last_unstale":"2023-12-01T03:10:37.473026+0000","last_undegraded":"2023-12-01T03:10:37.473026+0000","last_fullsized":"2023-12-01T03:10:37.473026+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:35.663128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.013053+0000","last_change":"2023-12-01T03:10:39.011983+0000","last_active":"2023-12-01T03:10:39.013053+0000","last_peered":"2023-12-01T03:10:39.013053+0000","last_clean":"2023-12-01T03:10:39.013053+0000","last_became_active":"2023-12-01T03:10:39.011721+0000","last_became_peered":"2023-12-01T03:10:39.011721+0000","last_unstale":"2023-12-01T03:10:39.013053+0000","last_undegraded":"2023-12-01T03:10:39.013053+0000","last_fullsized":"2023-12-01T03:10:39.013053+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492510+0000","last_change":"2023-12-01T03:10:25.177089+0000","last_active":"2023-12-01T03:10:37.492510+0000","last_peered":"2023-12-01T03:10:37.492510+0000","last_clean":"2023-12-01T03:10:37.492510+0000","last_became_active":"2023-12-01T03:10:25.176742+0000","last_became_peered":"2023-12-01T03:10:25.176742+0000","last_unstale":"2023-12-01T03:10:37.492510+0000","last_undegraded":"2023-12-01T03:10:37.492510+0000","last_fullsized":"2023-12-01T03:10:37.492510+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513196+0000","last_change":"2023-12-01T03:10:38.513196+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513196+0000","last_undegraded":"2023-12-01T03:10:38.513196+0000","last_fullsized":"2023-12-01T03:10:38.513196+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:19.824447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.992054+0000","last_change":"2023-12-01T03:10:25.172560+0000","last_active":"2023-12-01T03:10:38.992054+0000","last_peered":"2023-12-01T03:10:38.992054+0000","last_clean":"2023-12-01T03:10:38.992054+0000","last_became_active":"2023-12-01T03:10:25.172005+0000","last_became_peered":"2023-12-01T03:10:25.172005+0000","last_unstale":"2023-12-01T03:10:38.992054+0000","last_undegraded":"2023-12-01T03:10:38.992054+0000","last_fullsized":"2023-12-01T03:10:38.992054+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:44.541671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.515497+0000","last_change":"2023-12-01T03:10:38.515497+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.515497+0000","last_undegraded":"2023-12-01T03:10:38.515497+0000","last_fullsized":"2023-12-01T03:10:38.515497+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:10:20.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.477932+0000","last_change":"2023-12-01T03:10:25.185623+0000","last_active":"2023-12-01T03:10:37.477932+0000","last_peered":"2023-12-01T03:10:37.477932+0000","last_clean":"2023-12-01T03:10:37.477932+0000","last_became_active":"2023-12-01T03:10:25.185311+0000","last_became_peered":"2023-12-01T03:10:25.185311+0000","last_unstale":"2023-12-01T03:10:37.477932+0000","last_undegraded":"2023-12-01T03:10:37.477932+0000","last_fullsized":"2023-12-01T03:10:37.477932+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508313+0000","last_change":"2023-12-01T03:10:38.508313+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508313+0000","last_undegraded":"2023-12-01T03:10:38.508313+0000","last_fullsized":"2023-12-01T03:10:38.508313+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:38.801763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483215+0000","last_change":"2023-12-01T03:10:25.186300+0000","last_active":"2023-12-01T03:10:37.483215+0000","last_peered":"2023-12-01T03:10:37.483215+0000","last_clean":"2023-12-01T03:10:37.483215+0000","last_became_active":"2023-12-01T03:10:25.185030+0000","last_became_peered":"2023-12-01T03:10:25.185030+0000","last_unstale":"2023-12-01T03:10:37.483215+0000","last_undegraded":"2023-12-01T03:10:37.483215+0000","last_fullsized":"2023-12-01T03:10:37.483215+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.501552+0000","last_change":"2023-12-01T03:10:38.501552+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.501552+0000","last_undegraded":"2023-12-01T03:10:38.501552+0000","last_fullsized":"2023-12-01T03:10:38.501552+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482005+0000","last_change":"2023-12-01T03:10:25.182648+0000","last_active":"2023-12-01T03:10:37.482005+0000","last_peered":"2023-12-01T03:10:37.482005+0000","last_clean":"2023-12-01T03:10:37.482005+0000","last_became_active":"2023-12-01T03:10:25.182343+0000","last_became_peered":"2023-12-01T03:10:25.182343+0000","last_unstale":"2023-12-01T03:10:37.482005+0000","last_undegraded":"2023-12-01T03:10:37.482005+0000","last_fullsized":"2023-12-01T03:10:37.482005+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:55.408381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.514336+0000","last_change":"2023-12-01T03:10:38.514336+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.514336+0000","last_undegraded":"2023-12-01T03:10:38.514336+0000","last_fullsized":"2023-12-01T03:10:38.514336+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479210+0000","last_change":"2023-12-01T03:10:25.191253+0000","last_active":"2023-12-01T03:10:37.479210+0000","last_peered":"2023-12-01T03:10:37.479210+0000","last_clean":"2023-12-01T03:10:37.479210+0000","last_became_active":"2023-12-01T03:10:25.188485+0000","last_became_peered":"2023-12-01T03:10:25.188485+0000","last_unstale":"2023-12-01T03:10:37.479210+0000","last_undegraded":"2023-12-01T03:10:37.479210+0000","last_fullsized":"2023-12-01T03:10:37.479210+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500633+0000","last_change":"2023-12-01T03:10:38.500633+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500633+0000","last_undegraded":"2023-12-01T03:10:38.500633+0000","last_fullsized":"2023-12-01T03:10:38.500633+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999753+0000","last_change":"2023-12-01T03:10:25.171460+0000","last_active":"2023-12-01T03:10:38.999753+0000","last_peered":"2023-12-01T03:10:38.999753+0000","last_clean":"2023-12-01T03:10:38.999753+0000","last_became_active":"2023-12-01T03:10:25.171148+0000","last_became_peered":"2023-12-01T03:10:25.171148+0000","last_unstale":"2023-12-01T03:10:38.999753+0000","last_undegraded":"2023-12-01T03:10:38.999753+0000","last_fullsized":"2023-12-01T03:10:38.999753+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:01:37.278740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.516041+0000","last_change":"2023-12-01T03:10:38.516041+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.516041+0000","last_undegraded":"2023-12-01T03:10:38.516041+0000","last_fullsized":"2023-12-01T03:10:38.516041+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:07:32.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488617+0000","last_change":"2023-12-01T03:10:25.181720+0000","last_active":"2023-12-01T03:10:37.488617+0000","last_peered":"2023-12-01T03:10:37.488617+0000","last_clean":"2023-12-01T03:10:37.488617+0000","last_became_active":"2023-12-01T03:10:25.180144+0000","last_became_peered":"2023-12-01T03:10:25.180144+0000","last_unstale":"2023-12-01T03:10:37.488617+0000","last_undegraded":"2023-12-01T03:10:37.488617+0000","last_fullsized":"2023-12-01T03:10:37.488617+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:53.999589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007029+0000","last_change":"2023-12-01T03:10:39.004848+0000","last_active":"2023-12-01T03:10:39.007029+0000","last_peered":"2023-12-01T03:10:39.007029+0000","last_clean":"2023-12-01T03:10:39.007029+0000","last_became_active":"2023-12-01T03:10:39.004604+0000","last_became_peered":"2023-12-01T03:10:39.004604+0000","last_unstale":"2023-12-01T03:10:39.007029+0000","last_undegraded":"2023-12-01T03:10:39.007029+0000","last_fullsized":"2023-12-01T03:10:39.007029+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:44.253867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983363+0000","last_change":"2023-12-01T03:10:25.181088+0000","last_active":"2023-12-01T03:10:38.983363+0000","last_peered":"2023-12-01T03:10:38.983363+0000","last_clean":"2023-12-01T03:10:38.983363+0000","last_became_active":"2023-12-01T03:10:25.180741+0000","last_became_peered":"2023-12-01T03:10:25.180741+0000","last_unstale":"2023-12-01T03:10:38.983363+0000","last_undegraded":"2023-12-01T03:10:38.983363+0000","last_fullsized":"2023-12-01T03:10:38.983363+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:26.321285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513800+0000","last_change":"2023-12-01T03:10:38.513800+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513800+0000","last_undegraded":"2023-12-01T03:10:38.513800+0000","last_fullsized":"2023-12-01T03:10:38.513800+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472688+0000","last_change":"2023-12-01T03:10:25.171115+0000","last_active":"2023-12-01T03:10:37.472688+0000","last_peered":"2023-12-01T03:10:37.472688+0000","last_clean":"2023-12-01T03:10:37.472688+0000","last_became_active":"2023-12-01T03:10:25.170871+0000","last_became_peered":"2023-12-01T03:10:25.170871+0000","last_unstale":"2023-12-01T03:10:37.472688+0000","last_undegraded":"2023-12-01T03:10:37.472688+0000","last_fullsized":"2023-12-01T03:10:37.472688+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:24.370036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59936,"kb_used_data":2648,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311904,"statfs":{"total":96636764160,"available":96575389696,"internally_reserved":0,"allocated":2711552,"data_stored":1314756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0780000000000001,"15min":1.0820000000000001},"min":{"1min":0.78200000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5069999999999999,"5min":1.6000000000000001,"15min":1.98},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.093,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":0.67800000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.556,"5min":1.7,"15min":1.7},"last":1.331}]},{"osd":1,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.2889999999999999},"min":{"1min":0.80000000000000004,"5min":0.745,"15min":0.73099999999999998},"max":{"1min":1.603,"5min":2,"15min":2.1419999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3149999999999999,"15min":1.3320000000000001},"min":{"1min":1.137,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.514,"5min":1.8169999999999999,"15min":1.8700000000000001},"last":1.0700000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.157,"15min":1.1719999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.476,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.26,"15min":1.254},"min":{"1min":1.0549999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6020000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.268}]},{"osd":3,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2529999999999999,"15min":1.3049999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6559999999999999,"5min":1.712,"15min":2.431},"last":1.014},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.415,"15min":1.4199999999999999},"min":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.92900000000000005},"max":{"1min":1.5920000000000001,"5min":1.879,"15min":2.198},"last":1.109}]},{"osd":4,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3140000000000001,"15min":1.3400000000000001},"min":{"1min":1.006,"5min":0.82599999999999996,"15min":0.81200000000000006},"max":{"1min":1.901,"5min":1.901,"15min":2.5899999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":1.1240000000000001,"5min":0.92800000000000005,"15min":0.79500000000000004},"max":{"1min":1.9690000000000001,"5min":2.3279999999999998,"15min":2.569},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3009999999999999,"15min":1.3149999999999999},"min":{"1min":0.93600000000000005,"5min":0.85999999999999999,"15min":0.84599999999999997},"max":{"1min":2.0099999999999998,"5min":2.016,"15min":2.7269999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5800000000000001,"15min":1.589},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":0.93200000000000005},"max":{"1min":2.0569999999999999,"5min":2.206,"15min":2.484},"last":1.4570000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4359999999999999,"15min":1.446},"min":{"1min":1.0229999999999999,"5min":0.98599999999999999,"15min":0.82599999999999996},"max":{"1min":1.921,"5min":2.2789999999999999,"15min":2.532},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":1.8819999999999999,"15min":2.762},"last":1.2370000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575980,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41872,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329968,"statfs":{"total":96636764160,"available":96593887232,"internally_reserved":0,"allocated":2695168,"data_stored":1311929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0589999999999999,"15min":1.113},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.4039999999999999,"5min":1.6639999999999999,"15min":1.9219999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1160000000000001,"15min":1.1659999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.623,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":0.97699999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.976,"5min":1.976,"15min":2.0449999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1950000000000001,"15min":1.2210000000000001},"min":{"1min":0.627,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.694,"5min":1.694,"15min":1.835},"last":1.2390000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.163,"15min":1.2090000000000001},"min":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.976},"last":1.127},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2629999999999999,"15min":1.2849999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.167}]},{"osd":3,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2849999999999999,"15min":1.319},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8100000000000001,"5min":1.927,"15min":2.0289999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.3720000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2130000000000001,"15min":1.262},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.367,"15min":1.3540000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.522}]},{"osd":5,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3360000000000001,"15min":1.4039999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.272},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4790000000000001,"15min":1.484},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.6319999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4890000000000001,"15min":1.5249999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.452},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.556,"15min":1.579},"min":{"1min":0.95499999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2999999999999998,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.794}]}]},{"osd":5,"up_from":13,"seq":55834575979,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48216,"kb_used_data":3216,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323624,"statfs":{"total":96636764160,"available":96587390976,"internally_reserved":0,"allocated":3293184,"data_stored":1902811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.125},"min":{"1min":0.82299999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.548,"5min":1.776,"15min":1.8899999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.238,"5min":1.1639999999999999,"15min":1.163},"min":{"1min":0.89800000000000002,"5min":0.69399999999999995,"15min":0.67300000000000004},"max":{"1min":1.619,"5min":1.752,"15min":1.8009999999999999},"last":0.84999999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.133,"15min":1.1539999999999999},"min":{"1min":0.94699999999999995,"5min":0.81200000000000006,"15min":0.79900000000000004},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.696},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.272,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.88,"5min":0.79500000000000004,"15min":0.60899999999999999},"max":{"1min":1.619,"5min":1.6659999999999999,"15min":1.7649999999999999},"last":1.0289999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2390000000000001,"15min":1.246},"min":{"1min":0.93700000000000006,"5min":0.84499999999999997,"15min":0.81399999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.774},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.284,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.93899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.502,"5min":1.899,"15min":1.9219999999999999},"last":0.90200000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.306,"15min":1.3200000000000001},"min":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.93100000000000005},"max":{"1min":1.7929999999999999,"5min":1.8979999999999999,"15min":1.994},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.343,"15min":1.363},"min":{"1min":1.02,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.724,"5min":1.97,"15min":2.1859999999999999},"last":1.1080000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2789999999999999,"15min":1.254},"min":{"1min":0.96199999999999997,"5min":0.78000000000000003,"15min":0.74099999999999999},"max":{"1min":1.726,"5min":1.726,"15min":2.6040000000000001},"last":1.389},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.399,"15min":1.385},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.84599999999999997},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.931},"last":1.1579999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3879999999999999,"15min":1.3759999999999999},"min":{"1min":0.97599999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.9279999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.427,"15min":1.419},"min":{"1min":1.222,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":1.756,"5min":1.9219999999999999,"15min":2.577},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5349999999999999,"15min":1.5369999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.962,"5min":2.0790000000000002,"15min":3.1709999999999998},"last":1.294},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.575,"15min":1.5740000000000001},"min":{"1min":1.3720000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.831,"5min":2.101,"15min":2.8740000000000001},"last":1.343}]}]},{"osd":4,"up_from":13,"seq":55834575979,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2707456,"data_stored":1319053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.147,"15min":1.1339999999999999},"min":{"1min":0.73099999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.21,"15min":1.1990000000000001},"min":{"1min":0.82099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5840000000000001,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.006}]},{"osd":1,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.242,"15min":1.2589999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.613,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.228},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.82599999999999996},"max":{"1min":1.6080000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.5549999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.3240000000000001,"15min":1.2470000000000001},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.573,"5min":11.311,"15min":11.311},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.329,"15min":1.341},"min":{"1min":1.0680000000000001,"5min":1.024,"15min":0.94099999999999995},"max":{"1min":1.647,"5min":1.9379999999999999,"15min":1.9450000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3100000000000001,"15min":1.29},"min":{"1min":1.0529999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7749999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.365,"15min":1.3380000000000001},"min":{"1min":0.99199999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.599,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.5920000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.357,"15min":1.329},"min":{"1min":0.94799999999999995,"5min":0.91200000000000003,"15min":0.755},"max":{"1min":1.615,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.379,"15min":1.3120000000000001},"min":{"1min":0.94599999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7150000000000001,"5min":3.71,"15min":3.71},"last":1.2809999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3740000000000001,"15min":1.375},"min":{"1min":1.095,"5min":0.745,"15min":0.745},"max":{"1min":1.609,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.4850000000000001,"15min":1.48},"min":{"1min":1.335,"5min":1.0660000000000001,"15min":1.0149999999999999},"max":{"1min":1.738,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.52,"15min":1.486},"min":{"1min":1.157,"5min":0.95499999999999996,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6000000000000001,"15min":1.5800000000000001},"min":{"1min":1.3600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.806,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.7070000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575978,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51752,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320088,"statfs":{"total":96636764160,"available":96583770112,"internally_reserved":0,"allocated":2719744,"data_stored":1321894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91100000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.085},"min":{"1min":0.74199999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.571,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.163,"15min":1.171},"min":{"1min":0.64700000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.008,"15min":0.999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41099999999999998},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0900000000000001,"15min":1.103},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2230000000000001,"15min":1.252},"min":{"1min":0.80100000000000005,"5min":0.747,"15min":0.58599999999999997},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.57899999999999996}]},{"osd":4,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2989999999999999,"15min":1.2929999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.3},"min":{"1min":0.96799999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0229999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3240000000000001,"15min":1.331},"min":{"1min":0.86599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.373,"15min":1.3759999999999999},"min":{"1min":0.88400000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.78400000000000003},"max":{"1min":2.5649999999999999,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.54,"5min":1.5049999999999999,"15min":1.514},"min":{"1min":1.075,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.075}]},{"osd":7,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.516,"15min":1.5329999999999999},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.042},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.54,"15min":1.5309999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.878},"max":{"1min":2.4830000000000001,"5min":2.984,"15min":2.984},"last":0.83899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575978,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64028,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307812,"statfs":{"total":96636764160,"available":96571199488,"internally_reserved":0,"allocated":2707456,"data_stored":1319089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89300000000000002,"15min":0.89400000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1140000000000001,"5min":1.405,"15min":1.4730000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.056,"15min":1.0569999999999999},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5640000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0449999999999999,"15min":1.0589999999999999},"min":{"1min":0.66100000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.321,"5min":1.698,"15min":1.698},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.038,"5min":1.125,"15min":1.151},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.522,"5min":1.635,"15min":1.71},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0640000000000001,"15min":1.054},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.419,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1990000000000001,"15min":1.2170000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.516,"5min":1.6879999999999999,"15min":1.79},"last":1.01}]},{"osd":4,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":0.85199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.7569999999999999,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.133},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.714,"5min":1.714,"15min":1.835},"last":1.2390000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3919999999999999,"15min":1.391},"min":{"1min":0.88,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.8380000000000001,"15min":2.2040000000000002},"last":1.042},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.419,"15min":1.4179999999999999},"min":{"1min":1.095,"5min":1.004,"15min":1.004},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.016},"last":0.91800000000000004}]},{"osd":6,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.379,"15min":1.3819999999999999},"min":{"1min":0.89600000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.337},{"interface":"front","average":{"1min":1.46,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":1.087,"5min":0.879,"15min":0.879},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.327}]},{"osd":7,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5069999999999999,"15min":1.4910000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0979999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4330000000000001,"15min":1.458},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.012,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.0860000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575978,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48244,"kb_used_data":3244,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323596,"statfs":{"total":96636764160,"available":96587362304,"internally_reserved":0,"allocated":3321856,"data_stored":1924287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.879,"15min":0.90900000000000003},"min":{"1min":0.58199999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.099,"5min":0.99299999999999999,"15min":1.046},"min":{"1min":0.81000000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.5069999999999999,"5min":1.5660000000000001,"15min":1.681},"last":0.498}]},{"osd":2,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.56299999999999994,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.115,"15min":1.1419999999999999},"min":{"1min":0.75600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.581,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.258,"15min":1.3069999999999999},"min":{"1min":1.026,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5780000000000001,"5min":1.587,"15min":1.788},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.107,"15min":1.135},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.714,"5min":1.714,"15min":1.8049999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.21,"15min":1.2230000000000001},"min":{"1min":0.89900000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6859999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.389,"15min":1.409},"min":{"1min":1.0740000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.0269999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.85899999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.802,"5min":1.8520000000000001,"15min":2.101},"last":1.056},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.304,"15min":1.339},"min":{"1min":1.006,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.609,"5min":1.927,"15min":2.1659999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4019999999999999,"15min":1.4379999999999999},"min":{"1min":0.99399999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3799999999999999,"15min":1.4119999999999999},"min":{"1min":1.1779999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.768,"5min":2.0019999999999998,"15min":2.0209999999999999},"last":0.98499999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4870000000000001,"15min":1.5269999999999999},"min":{"1min":1.2549999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.911,"5min":2.0329999999999999,"15min":2.1190000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.524,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.091}]}]},{"osd":0,"up_from":13,"seq":55834575980,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":2676,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2740224,"data_stored":1335117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.86699999999999999,"15min":0.89400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1559999999999999,"5min":1.4379999999999999,"15min":1.994},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.04,"15min":1.056},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.427,"5min":1.583,"15min":1.6799999999999999},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0329999999999999,"15min":1.069},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.446,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.1020000000000001,"15min":1.1419999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.726,"5min":1.726,"15min":1.792},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0580000000000001,"15min":1.071},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.8839999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.655,"5min":1.655,"15min":2.6469999999999998},"last":0.78200000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3260000000000001,"15min":1.363},"min":{"1min":0.88400000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.274,"5min":1.284,"15min":1.3089999999999999},"min":{"1min":0.82099999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.677,"5min":1.74,"15min":2.3610000000000002},"last":1.0740000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.355,"15min":1.397},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5589999999999999,"5min":2.1030000000000002,"15min":2.7719999999999998},"last":1.054},{"interface":"front","average":{"1min":1.431,"5min":1.4099999999999999,"15min":1.4410000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.79600000000000004},"max":{"1min":1.9259999999999999,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":0.97499999999999998}]},{"osd":6,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.367,"15min":1.3859999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7649999999999999,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.458,"15min":1.484},"min":{"1min":1.048,"5min":0.85299999999999998,"15min":0.75800000000000001},"max":{"1min":1.736,"5min":1.9870000000000001,"15min":2.8759999999999999},"last":1.2569999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.49,"15min":1.524},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.841,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4750000000000001,"15min":1.4910000000000001},"min":{"1min":0.97299999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.0619999999999998,"5min":2.1190000000000002,"15min":2.431},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:10:44.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.1a is not active+clean 2023-12-01T03:10:44.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.507374+0000', 'last_change': '2023-12-01T03:10:38.507374+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.507374+0000', 'last_undegraded': '2023-12-01T03:10:38.507374+0000', 'last_fullsized': '2023-12-01T03:10:38.507374+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:26:14.969594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3 is not active+clean 2023-12-01T03:10:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.499505+0000', 'last_change': '2023-12-01T03:10:38.499505+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.499505+0000', 'last_undegraded': '2023-12-01T03:10:38.499505+0000', 'last_fullsized': '2023-12-01T03:10:38.499505+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:10:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2 is not active+clean 2023-12-01T03:10:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.509995+0000', 'last_change': '2023-12-01T03:10:38.509995+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.509995+0000', 'last_undegraded': '2023-12-01T03:10:38.509995+0000', 'last_fullsized': '2023-12-01T03:10:38.509995+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.0 is not active+clean 2023-12-01T03:10:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.499285+0000', 'last_change': '2023-12-01T03:10:38.499285+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.499285+0000', 'last_undegraded': '2023-12-01T03:10:38.499285+0000', 'last_fullsized': '2023-12-01T03:10:38.499285+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:10:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.7 is not active+clean 2023-12-01T03:10:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.503490+0000', 'last_change': '2023-12-01T03:10:38.503490+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.503490+0000', 'last_undegraded': '2023-12-01T03:10:38.503490+0000', 'last_fullsized': '2023-12-01T03:10:38.503490+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.4 is not active+clean 2023-12-01T03:10:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.499505+0000', 'last_change': '2023-12-01T03:10:38.499505+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.499505+0000', 'last_undegraded': '2023-12-01T03:10:38.499505+0000', 'last_fullsized': '2023-12-01T03:10:38.499505+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.b is not active+clean 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.504217+0000', 'last_change': '2023-12-01T03:10:38.504217+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.504217+0000', 'last_undegraded': '2023-12-01T03:10:38.504217+0000', 'last_fullsized': '2023-12-01T03:10:38.504217+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:41:18.081282+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.a is not active+clean 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.508392+0000', 'last_change': '2023-12-01T03:10:38.508392+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.508392+0000', 'last_undegraded': '2023-12-01T03:10:38.508392+0000', 'last_fullsized': '2023-12-01T03:10:38.508392+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:43:53.750334+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.d is not active+clean 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.504526+0000', 'last_change': '2023-12-01T03:10:38.504526+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.504526+0000', 'last_undegraded': '2023-12-01T03:10:38.504526+0000', 'last_fullsized': '2023-12-01T03:10:38.504526+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:00:41.392942+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.c is not active+clean 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.511251+0000', 'last_change': '2023-12-01T03:10:38.511251+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.511251+0000', 'last_undegraded': '2023-12-01T03:10:38.511251+0000', 'last_fullsized': '2023-12-01T03:10:38.511251+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:47:30.021172+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.13 is not active+clean 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.511262+0000', 'last_change': '2023-12-01T03:10:38.511262+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.511262+0000', 'last_undegraded': '2023-12-01T03:10:38.511262+0000', 'last_fullsized': '2023-12-01T03:10:38.511262+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.12 is not active+clean 2023-12-01T03:10:44.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.498276+0000', 'last_change': '2023-12-01T03:10:38.498276+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.498276+0000', 'last_undegraded': '2023-12-01T03:10:38.498276+0000', 'last_fullsized': '2023-12-01T03:10:38.498276+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.11 is not active+clean 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.502663+0000', 'last_change': '2023-12-01T03:10:38.502663+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.502663+0000', 'last_undegraded': '2023-12-01T03:10:38.502663+0000', 'last_fullsized': '2023-12-01T03:10:38.502663+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.10 is not active+clean 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.507590+0000', 'last_change': '2023-12-01T03:10:38.507590+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.507590+0000', 'last_undegraded': '2023-12-01T03:10:38.507590+0000', 'last_fullsized': '2023-12-01T03:10:38.507590+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:00:45.156063+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.17 is not active+clean 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.504758+0000', 'last_change': '2023-12-01T03:10:38.504758+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.504758+0000', 'last_undegraded': '2023-12-01T03:10:38.504758+0000', 'last_fullsized': '2023-12-01T03:10:38.504758+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:13:18.605693+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.15 is not active+clean 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.503576+0000', 'last_change': '2023-12-01T03:10:38.503576+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.503576+0000', 'last_undegraded': '2023-12-01T03:10:38.503576+0000', 'last_fullsized': '2023-12-01T03:10:38.503576+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.14 is not active+clean 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.510307+0000', 'last_change': '2023-12-01T03:10:38.510307+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.510307+0000', 'last_undegraded': '2023-12-01T03:10:38.510307+0000', 'last_fullsized': '2023-12-01T03:10:38.510307+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.19 is not active+clean 2023-12-01T03:10:44.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.506778+0000', 'last_change': '2023-12-01T03:10:38.506778+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.506778+0000', 'last_undegraded': '2023-12-01T03:10:38.506778+0000', 'last_fullsized': '2023-12-01T03:10:38.506778+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.18 is not active+clean 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.510675+0000', 'last_change': '2023-12-01T03:10:38.510675+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.510675+0000', 'last_undegraded': '2023-12-01T03:10:38.510675+0000', 'last_fullsized': '2023-12-01T03:10:38.510675+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:33:51.590745+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.1d is not active+clean 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.513293+0000', 'last_change': '2023-12-01T03:10:38.513293+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.513293+0000', 'last_undegraded': '2023-12-01T03:10:38.513293+0000', 'last_fullsized': '2023-12-01T03:10:38.513293+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:46:18.989087+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.1c is not active+clean 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.512195+0000', 'last_change': '2023-12-01T03:10:38.512195+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.512195+0000', 'last_undegraded': '2023-12-01T03:10:38.512195+0000', 'last_fullsized': '2023-12-01T03:10:38.512195+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.23 is not active+clean 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.500232+0000', 'last_change': '2023-12-01T03:10:38.500232+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.500232+0000', 'last_undegraded': '2023-12-01T03:10:38.500232+0000', 'last_fullsized': '2023-12-01T03:10:38.500232+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.20 is not active+clean 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.501528+0000', 'last_change': '2023-12-01T03:10:38.501528+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.501528+0000', 'last_undegraded': '2023-12-01T03:10:38.501528+0000', 'last_fullsized': '2023-12-01T03:10:38.501528+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.21 is not active+clean 2023-12-01T03:10:44.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.505382+0000', 'last_change': '2023-12-01T03:10:38.505382+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.505382+0000', 'last_undegraded': '2023-12-01T03:10:38.505382+0000', 'last_fullsized': '2023-12-01T03:10:38.505382+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:37:52.701542+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.26 is not active+clean 2023-12-01T03:10:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.496510+0000', 'last_change': '2023-12-01T03:10:38.496510+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.496510+0000', 'last_undegraded': '2023-12-01T03:10:38.496510+0000', 'last_fullsized': '2023-12-01T03:10:38.496510+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:10:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.24 is not active+clean 2023-12-01T03:10:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.496580+0000', 'last_change': '2023-12-01T03:10:38.496580+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.496580+0000', 'last_undegraded': '2023-12-01T03:10:38.496580+0000', 'last_fullsized': '2023-12-01T03:10:38.496580+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:10:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2a is not active+clean 2023-12-01T03:10:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.500994+0000', 'last_change': '2023-12-01T03:10:38.500994+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.500994+0000', 'last_undegraded': '2023-12-01T03:10:38.500994+0000', 'last_fullsized': '2023-12-01T03:10:38.500994+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:10:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.28 is not active+clean 2023-12-01T03:10:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.506833+0000', 'last_change': '2023-12-01T03:10:38.506833+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.506833+0000', 'last_undegraded': '2023-12-01T03:10:38.506833+0000', 'last_fullsized': '2023-12-01T03:10:38.506833+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:57:07.895399+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:10:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2e is not active+clean 2023-12-01T03:10:44.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.512138+0000', 'last_change': '2023-12-01T03:10:38.512138+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.512138+0000', 'last_undegraded': '2023-12-01T03:10:38.512138+0000', 'last_fullsized': '2023-12-01T03:10:38.512138+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2f is not active+clean 2023-12-01T03:10:44.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.511167+0000', 'last_change': '2023-12-01T03:10:38.511167+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.511167+0000', 'last_undegraded': '2023-12-01T03:10:38.511167+0000', 'last_fullsized': '2023-12-01T03:10:38.511167+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:38:07.828693+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2c is not active+clean 2023-12-01T03:10:44.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.507312+0000', 'last_change': '2023-12-01T03:10:38.507312+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.507312+0000', 'last_undegraded': '2023-12-01T03:10:38.507312+0000', 'last_fullsized': '2023-12-01T03:10:38.507312+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:58:42.639310+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2d is not active+clean 2023-12-01T03:10:44.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.508726+0000', 'last_change': '2023-12-01T03:10:38.508726+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.508726+0000', 'last_undegraded': '2023-12-01T03:10:38.508726+0000', 'last_fullsized': '2023-12-01T03:10:38.508726+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:34:54.829130+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.32 is not active+clean 2023-12-01T03:10:44.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.507042+0000', 'last_change': '2023-12-01T03:10:38.507042+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.507042+0000', 'last_undegraded': '2023-12-01T03:10:38.507042+0000', 'last_fullsized': '2023-12-01T03:10:38.507042+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:31:48.887816+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:10:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.30 is not active+clean 2023-12-01T03:10:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.513592+0000', 'last_change': '2023-12-01T03:10:38.513592+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.513592+0000', 'last_undegraded': '2023-12-01T03:10:38.513592+0000', 'last_fullsized': '2023-12-01T03:10:38.513592+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.31 is not active+clean 2023-12-01T03:10:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.512530+0000', 'last_change': '2023-12-01T03:10:38.512530+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.512530+0000', 'last_undegraded': '2023-12-01T03:10:38.512530+0000', 'last_fullsized': '2023-12-01T03:10:38.512530+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:24:41.123796+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.36 is not active+clean 2023-12-01T03:10:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.516591+0000', 'last_change': '2023-12-01T03:10:38.516591+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.516591+0000', 'last_undegraded': '2023-12-01T03:10:38.516591+0000', 'last_fullsized': '2023-12-01T03:10:38.516591+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.37 is not active+clean 2023-12-01T03:10:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.507019+0000', 'last_change': '2023-12-01T03:10:38.507019+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.507019+0000', 'last_undegraded': '2023-12-01T03:10:38.507019+0000', 'last_fullsized': '2023-12-01T03:10:38.507019+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:10:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.35 is not active+clean 2023-12-01T03:10:44.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.513196+0000', 'last_change': '2023-12-01T03:10:38.513196+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.513196+0000', 'last_undegraded': '2023-12-01T03:10:38.513196+0000', 'last_fullsized': '2023-12-01T03:10:38.513196+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:48:19.824447+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3a is not active+clean 2023-12-01T03:10:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.515497+0000', 'last_change': '2023-12-01T03:10:38.515497+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.515497+0000', 'last_undegraded': '2023-12-01T03:10:38.515497+0000', 'last_fullsized': '2023-12-01T03:10:38.515497+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:10:20.781162+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3b is not active+clean 2023-12-01T03:10:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.508313+0000', 'last_change': '2023-12-01T03:10:38.508313+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.508313+0000', 'last_undegraded': '2023-12-01T03:10:38.508313+0000', 'last_fullsized': '2023-12-01T03:10:38.508313+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:49:38.801763+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.38 is not active+clean 2023-12-01T03:10:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.501552+0000', 'last_change': '2023-12-01T03:10:38.501552+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.501552+0000', 'last_undegraded': '2023-12-01T03:10:38.501552+0000', 'last_fullsized': '2023-12-01T03:10:38.501552+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:10:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.39 is not active+clean 2023-12-01T03:10:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.514336+0000', 'last_change': '2023-12-01T03:10:38.514336+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.514336+0000', 'last_undegraded': '2023-12-01T03:10:38.514336+0000', 'last_fullsized': '2023-12-01T03:10:38.514336+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3e is not active+clean 2023-12-01T03:10:44.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.500633+0000', 'last_change': '2023-12-01T03:10:38.500633+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.500633+0000', 'last_undegraded': '2023-12-01T03:10:38.500633+0000', 'last_fullsized': '2023-12-01T03:10:38.500633+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:10:44.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3f is not active+clean 2023-12-01T03:10:44.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.516041+0000', 'last_change': '2023-12-01T03:10:38.516041+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.516041+0000', 'last_undegraded': '2023-12-01T03:10:38.516041+0000', 'last_fullsized': '2023-12-01T03:10:38.516041+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:07:32.945465+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:10:44.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3d is not active+clean 2023-12-01T03:10:44.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 216, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:10:38.513800+0000', 'last_change': '2023-12-01T03:10:38.513800+0000', 'last_active': '2023-12-01T03:10:38.468841+0000', 'last_peered': '2023-12-01T03:10:38.468841+0000', 'last_clean': '2023-12-01T03:10:38.468841+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:10:38.513800+0000', 'last_undegraded': '2023-12-01T03:10:38.513800+0000', 'last_fullsized': '2023-12-01T03:10:38.513800+0000', 'mapping_epoch': 216, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 216, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'last_clean_scrub_stamp': '2023-12-01T03:10:38.468841+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:10:44.435 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-01T03:10:44.612 INFO:tasks.ceph.mds.c.smithi115.stderr:2023-12-01T03:10:44.610+0000 7f735d95a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:10:44.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.675+0000 7fb1c65b5640 1 Processor -- start 2023-12-01T03:10:44.676 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.676+0000 7fb1c65b5640 1 -- start start 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.677+0000 7fb1c65b5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1c00cad70 0x7fb1c006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.677+0000 7fb1c65b5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb1c006f2f0 0x7fb1c006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.677+0000 7fb1c65b5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c00add10 con 0x7fb1c015e2f0 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.677+0000 7fb1c65b5640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c00c5760 con 0x7fb1c006f2f0 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.677+0000 7fb1c65b5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c015e6d0 con 0x7fb1c00cad70 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.677+0000 7fb1bffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb1c015e2f0 0x7fb1c00ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37082/0 (socket says 172.21.15.115:37082) 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.677+0000 7fb1c4b52640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb1c006f2f0 0x7fb1c006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.677+0000 7fb1bffff640 1 -- 172.21.15.115:0/234477058 learned_addr learned my addr 172.21.15.115:0/234477058 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.678+0000 7fb1bf7fe640 1 --2- 172.21.15.115:0/234477058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1c00cad70 0x7fb1c006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.678+0000 7fb1c4b52640 1 -- 172.21.15.115:0/234477058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1c00cad70 msgr2=0x7fb1c006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.678+0000 7fb1c4b52640 1 --2- 172.21.15.115:0/234477058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1c00cad70 0x7fb1c006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.678+0000 7fb1c4b52640 1 -- 172.21.15.115:0/234477058 >> v1:172.21.15.115:6789/0 conn(0x7fb1c015e2f0 legacy=0x7fb1c00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.678+0000 7fb1c4b52640 1 -- 172.21.15.115:0/234477058 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00cf7b0 con 0x7fb1c006f2f0 2023-12-01T03:10:44.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.678+0000 7fb1beffd640 1 -- 172.21.15.115:0/234477058 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504424808 0 0) 0x7fb1c00add10 con 0x7fb1c015e2f0 2023-12-01T03:10:44.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.679+0000 7fb1c4b52640 1 --2- 172.21.15.115:0/234477058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c006f2f0 0x7fb1c006bb00 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009d90 tx=0x7fb1b802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67747ef204a43a51 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:44.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.679+0000 7fb1beffd640 1 -- 172.21.15.115:0/234477058 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b802cb30 con 0x7fb1c006f2f0 2023-12-01T03:10:44.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.679+0000 7fb1beffd640 1 -- 172.21.15.115:0/234477058 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b802cc90 con 0x7fb1c006f2f0 2023-12-01T03:10:44.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.679+0000 7fb1beffd640 1 -- 172.21.15.115:0/234477058 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8034440 con 0x7fb1c006f2f0 2023-12-01T03:10:44.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.680+0000 7fb1c65b5640 1 -- 172.21.15.115:0/234477058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c006f2f0 msgr2=0x7fb1c006bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.680+0000 7fb1c65b5640 1 --2- 172.21.15.115:0/234477058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c006f2f0 0x7fb1c006bb00 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009d90 tx=0x7fb1b802bd80 comp rx=0 tx=0).stop 2023-12-01T03:10:44.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.680+0000 7fb1c65b5640 1 -- 172.21.15.115:0/234477058 shutdown_connections 2023-12-01T03:10:44.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.680+0000 7fb1c65b5640 1 --2- 172.21.15.115:0/234477058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c006f2f0 0x7fb1c006bb00 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.680+0000 7fb1c65b5640 1 --2- 172.21.15.115:0/234477058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1c00cad70 0x7fb1c006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.680+0000 7fb1c65b5640 1 -- 172.21.15.115:0/234477058 >> 172.21.15.115:0/234477058 conn(0x7fb1c00ce880 msgr2=0x7fb1c00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:44.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.680+0000 7fb1c65b5640 1 -- 172.21.15.115:0/234477058 shutdown_connections 2023-12-01T03:10:44.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.680+0000 7fb1c65b5640 1 -- 172.21.15.115:0/234477058 wait complete. 2023-12-01T03:10:44.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.682+0000 7fb1c65b5640 1 Processor -- start 2023-12-01T03:10:44.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.682+0000 7fb1c65b5640 1 -- start start 2023-12-01T03:10:44.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.683+0000 7fb1c65b5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1c006f2f0 0x7fb1c0159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.683+0000 7fb1c65b5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1c00cad70 0x7fb1c006c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.683+0000 7fb1c65b5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c015e2f0 0x7fb1c01f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.683+0000 7fb1c65b5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c015a4a0 con 0x7fb1c015e2f0 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.683+0000 7fb1c65b5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00c68b0 con 0x7fb1c00cad70 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.683+0000 7fb1c65b5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00c6b90 con 0x7fb1c006f2f0 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.683+0000 7fb1c4b52640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c015e2f0 0x7fb1c01f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.683+0000 7fb1c4b52640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c015e2f0 0x7fb1c01f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55524/0 (socket says 172.21.15.115:55524) 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.683+0000 7fb1c4b52640 1 -- 172.21.15.115:0/385273356 learned_addr learned my addr 172.21.15.115:0/385273356 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1bffff640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1c006f2f0 0x7fb1c0159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1bf7fe640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1c00cad70 0x7fb1c006c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1c4b52640 1 -- 172.21.15.115:0/385273356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1c006f2f0 msgr2=0x7fb1c0159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1c4b52640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1c006f2f0 0x7fb1c0159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1c4b52640 1 -- 172.21.15.115:0/385273356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1c00cad70 msgr2=0x7fb1c006c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1c4b52640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1c00cad70 0x7fb1c006c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1c4b52640 1 -- 172.21.15.115:0/385273356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b8009a70 con 0x7fb1c015e2f0 2023-12-01T03:10:44.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1c4b52640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c015e2f0 0x7fb1c01f3790 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009d90 tx=0x7fb1b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:44.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1bcff9640 1 -- 172.21.15.115:0/385273356 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8003990 con 0x7fb1c015e2f0 2023-12-01T03:10:44.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1bcff9640 1 -- 172.21.15.115:0/385273356 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b8003af0 con 0x7fb1c015e2f0 2023-12-01T03:10:44.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1c65b5640 1 -- 172.21.15.115:0/385273356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c00c3400 con 0x7fb1c015e2f0 2023-12-01T03:10:44.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1bcff9640 1 -- 172.21.15.115:0/385273356 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8003ea0 con 0x7fb1c015e2f0 2023-12-01T03:10:44.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.684+0000 7fb1c65b5640 1 -- 172.21.15.115:0/385273356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c00c39c0 con 0x7fb1c015e2f0 2023-12-01T03:10:44.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.685+0000 7fb1c65b5640 1 -- 172.21.15.115:0/385273356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c006fda0 con 0x7fb1c015e2f0 2023-12-01T03:10:44.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.686+0000 7fb1bcff9640 1 -- 172.21.15.115:0/385273356 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fb1b803b050 con 0x7fb1c015e2f0 2023-12-01T03:10:44.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.688+0000 7fb1bcff9640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb1ac06a550 0x7fb1ac06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:44.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.688+0000 7fb1bcff9640 1 -- 172.21.15.115:0/385273356 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fb1b80ab210 con 0x7fb1c015e2f0 2023-12-01T03:10:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.688+0000 7fb1bffff640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb1ac06a550 0x7fb1ac06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.689+0000 7fb1bffff640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb1ac06a550 0x7fb1ac06ca10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0000f60 tx=0x7fb1b0009040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:44.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.692+0000 7fb1bcff9640 1 -- 172.21.15.115:0/385273356 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb1b80763f0 con 0x7fb1c015e2f0 2023-12-01T03:10:44.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.951+0000 7fb1c65b5640 1 -- 172.21.15.115:0/385273356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb1c004e890 con 0x7fb1c015e2f0 2023-12-01T03:10:44.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.952+0000 7fb1bcff9640 1 -- 172.21.15.115:0/385273356 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3404 (secure 0 0 0) 0x7fb1b8039d60 con 0x7fb1c015e2f0 2023-12-01T03:10:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:44.954 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5699,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":218,"num_osds":8,"num_up_osds":8,"osd_up_since":1701394558,"num_in_osds":8,"osd_in_since":1701394549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":148},{"state_name":"creating+peering","count":45}],"num_pgs":193,"num_pools":5,"num_objects":27,"data_bytes":594553,"bytes_used":435970048,"bytes_avail":772658143232,"bytes_total":773094113280,"inactive_pgs_ratio":0.23316061496734619,"write_bytes_sec":341,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":101,"id":12,"up":1,"in":1,"max":1,"id":13,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":13,"rank":0,"name":"c","status":"up:active","gid":7333},{"filesystem_id":12,"rank":0,"name":"d","status":"up:active","gid":10653}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2023-12-01T03:10:11.791326+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-01T03:10:44.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 -- 172.21.15.115:0/385273356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb1ac06a550 msgr2=0x7fb1ac06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb1ac06a550 0x7fb1ac06ca10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0000f60 tx=0x7fb1b0009040 comp rx=0 tx=0).stop 2023-12-01T03:10:44.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 -- 172.21.15.115:0/385273356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c015e2f0 msgr2=0x7fb1c01f3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:44.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c015e2f0 0x7fb1c01f3790 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009d90 tx=0x7fb1b8004a40 comp rx=0 tx=0).stop 2023-12-01T03:10:44.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 -- 172.21.15.115:0/385273356 shutdown_connections 2023-12-01T03:10:44.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb1ac06a550 0x7fb1ac06ca10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1c015e2f0 0x7fb1c01f3790 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1c00cad70 0x7fb1c006c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 --2- 172.21.15.115:0/385273356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1c006f2f0 0x7fb1c0159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:44.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 -- 172.21.15.115:0/385273356 >> 172.21.15.115:0/385273356 conn(0x7fb1c00ce880 msgr2=0x7fb1c015c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:44.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 -- 172.21.15.115:0/385273356 shutdown_connections 2023-12-01T03:10:44.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:44.955+0000 7fb1c65b5640 1 -- 172.21.15.115:0/385273356 wait complete. 2023-12-01T03:10:44.974 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-01T03:10:44.974 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:10:45.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.181+0000 7feba5ffd640 1 Processor -- start 2023-12-01T03:10:45.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.182+0000 7feba5ffd640 1 -- start start 2023-12-01T03:10:45.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.183+0000 7feba5ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba015a940 0x7feba00cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:45.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.183+0000 7feba5ffd640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba006c970 con 0x7feba00cb830 2023-12-01T03:10:45.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.183+0000 7feba5ffd640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba006cc50 con 0x7feba015ad20 2023-12-01T03:10:45.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.183+0000 7feba5ffd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba006cf30 con 0x7feba015a940 2023-12-01T03:10:45.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.183+0000 7feb9ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7feba00cb830 0x7feba00ad4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37092/0 (socket says 172.21.15.115:37092) 2023-12-01T03:10:45.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.183+0000 7feb9ffff640 1 -- 172.21.15.115:0/3063349714 learned_addr learned my addr 172.21.15.115:0/3063349714 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:45.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9f7fe640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba015a940 0x7feba00cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:45.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562217929 0 0) 0x7feba006c970 con 0x7feba00cb830 2023-12-01T03:10:45.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb8c003520 con 0x7feba00cb830 2023-12-01T03:10:45.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1351544113 0 0) 0x7feb8c003520 con 0x7feba00cb830 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba015a940 msgr2=0x7feba00cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba015a940 0x7feba00cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 >> v1:172.21.15.142:6789/0 conn(0x7feba015ad20 legacy=0x7feba00cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba006d210 con 0x7feba00cb830 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7feb980019f0 con 0x7feba00cb830 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 >> v1:172.21.15.115:6789/0 conn(0x7feba00cb830 legacy=0x7feba00ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8c0039a0 0x7feb8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.184+0000 7feb9effd640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba015a940 0x7feb8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9effd640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c004d00 0x7feb8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba006cf30 con 0x7feb8c004d00 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c00ba30 con 0x7feb8c0039a0 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c00bbf0 con 0x7feba015a940 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9ffff640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c004d00 0x7feb8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb97fff640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba015a940 0x7feb8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:45.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9f7fe640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8c0039a0 0x7feb8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:45.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9ffff640 1 -- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba015a940 msgr2=0x7feb8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:45.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9ffff640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba015a940 0x7feb8c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:45.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9ffff640 1 -- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8c0039a0 msgr2=0x7feb8c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:45.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9ffff640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8c0039a0 0x7feb8c003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:45.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.185+0000 7feb9ffff640 1 -- 172.21.15.115:0/3063349714 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c003520 con 0x7feb8c004d00 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.186+0000 7feb9ffff640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c004d00 0x7feb8c005180 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7feba00cbeb0 tx=0x7feb8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5344d9864d381064 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.187+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980072d0 con 0x7feb8c004d00 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.187+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9802f070 con 0x7feb8c004d00 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.187+0000 7feb9effd640 1 -- 172.21.15.115:0/3063349714 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980076b0 con 0x7feb8c004d00 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.187+0000 7feba5ffd640 1 -- 172.21.15.115:0/3063349714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c004d00 msgr2=0x7feb8c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.187+0000 7feba5ffd640 1 --2- 172.21.15.115:0/3063349714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb8c004d00 0x7feb8c005180 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7feba00cbeb0 tx=0x7feb8c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.187+0000 7feb9ffff640 1 -- 172.21.15.115:0/3063349714 reap_dead start 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.187+0000 7feba5ffd640 1 -- 172.21.15.115:0/3063349714 shutdown_connections 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.187+0000 7feba5ffd640 1 -- 172.21.15.115:0/3063349714 >> 172.21.15.115:0/3063349714 conn(0x7feba0066210 msgr2=0x7feba00b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.187+0000 7feba5ffd640 1 -- 172.21.15.115:0/3063349714 shutdown_connections 2023-12-01T03:10:45.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.188+0000 7feba5ffd640 1 -- 172.21.15.115:0/3063349714 wait complete. 2023-12-01T03:10:45.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.189+0000 7feba5ffd640 1 Processor -- start 2023-12-01T03:10:45.190 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.189+0000 7feba5ffd640 1 -- start start 2023-12-01T03:10:45.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.190+0000 7feba5ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8c004d00 0x7feba0158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:45.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.190+0000 7feba5ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba00ac8a0 0x7feba01f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:45.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.190+0000 7feba5ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba00cc610 0x7feba00cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:45.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.190+0000 7feba5ffd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba00ccfd0 con 0x7feba00cc610 2023-12-01T03:10:45.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.190+0000 7feba5ffd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba01f68d0 con 0x7feb8c004d00 2023-12-01T03:10:45.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.190+0000 7feba5ffd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba00af450 con 0x7feba00ac8a0 2023-12-01T03:10:45.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.191+0000 7feb9ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba00cc610 0x7feba00cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:45.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.191+0000 7feb9ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba00cc610 0x7feba00cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:55540/0 (socket says 172.21.15.115:55540) 2023-12-01T03:10:45.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.191+0000 7feb9ffff640 1 -- 172.21.15.115:0/2955403343 learned_addr learned my addr 172.21.15.115:0/2955403343 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:45.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.191+0000 7feb9ffff640 1 -- 172.21.15.115:0/2955403343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba00ac8a0 msgr2=0x7feba01f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:45.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.191+0000 7feb9ffff640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba00ac8a0 0x7feba01f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:45.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.191+0000 7feb9ffff640 1 -- 172.21.15.115:0/2955403343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8c004d00 msgr2=0x7feba0158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:45.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.191+0000 7feb9ffff640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8c004d00 0x7feba0158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:45.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.191+0000 7feb9ffff640 1 -- 172.21.15.115:0/2955403343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba00ae840 con 0x7feba00cc610 2023-12-01T03:10:45.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.191+0000 7feb9ffff640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba00cc610 0x7feba00cca90 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7feb9802cb20 tx=0x7feb98001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:45.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.192+0000 7feb9cff9640 1 -- 172.21.15.115:0/2955403343 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980019b0 con 0x7feba00cc610 2023-12-01T03:10:45.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.192+0000 7feb9cff9640 1 -- 172.21.15.115:0/2955403343 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb98004e90 con 0x7feba00cc610 2023-12-01T03:10:45.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.192+0000 7feba5ffd640 1 -- 172.21.15.115:0/2955403343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba00aeb20 con 0x7feba00cc610 2023-12-01T03:10:45.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.192+0000 7feb9cff9640 1 -- 172.21.15.115:0/2955403343 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980072d0 con 0x7feba00cc610 2023-12-01T03:10:45.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.192+0000 7feba5ffd640 1 -- 172.21.15.115:0/2955403343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba00af0e0 con 0x7feba00cc610 2023-12-01T03:10:45.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.193+0000 7feba5ffd640 1 -- 172.21.15.115:0/2955403343 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba0074210 con 0x7feba00cc610 2023-12-01T03:10:45.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.194+0000 7feb9cff9640 1 -- 172.21.15.115:0/2955403343 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7feb98031030 con 0x7feba00cc610 2023-12-01T03:10:45.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.195+0000 7feb9cff9640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb8006a550 0x7feb8006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:45.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.195+0000 7feb9cff9640 1 -- 172.21.15.115:0/2955403343 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7feb980aa9e0 con 0x7feba00cc610 2023-12-01T03:10:45.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.195+0000 7feb9f7fe640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb8006a550 0x7feb8006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:45.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.196+0000 7feb9f7fe640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb8006a550 0x7feb8006ca10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb90009950 tx=0x7feb90008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:45.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.199+0000 7feb9cff9640 1 -- 172.21.15.115:0/2955403343 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7feb98075a10 con 0x7feba00cc610 2023-12-01T03:10:45.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.386+0000 7feba5ffd640 1 -- 172.21.15.115:0/2955403343 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feba00af5c0 con 0x7feb8006a550 2023-12-01T03:10:45.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.423+0000 7feb9cff9640 1 -- 172.21.15.115:0/2955403343 <== mgr.10407 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+524098 (secure 0 0 0) 0x7feba00af5c0 con 0x7feb8006a550 2023-12-01T03:10:45.427 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:45.435 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.435+0000 7feba5ffd640 1 -- 172.21.15.115:0/2955403343 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb8006a550 msgr2=0x7feb8006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.435+0000 7feba5ffd640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb8006a550 0x7feb8006ca10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb90009950 tx=0x7feb90008040 comp rx=0 tx=0).stop 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.435+0000 7feba5ffd640 1 -- 172.21.15.115:0/2955403343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba00cc610 msgr2=0x7feba00cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.435+0000 7feba5ffd640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba00cc610 0x7feba00cca90 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7feb9802cb20 tx=0x7feb98001d80 comp rx=0 tx=0).stop 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.435+0000 7feba5ffd640 1 -- 172.21.15.115:0/2955403343 shutdown_connections 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.435+0000 7feba5ffd640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb8006a550 0x7feb8006ca10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.435+0000 7feba5ffd640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feba00ac8a0 0x7feba01f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.435+0000 7feba5ffd640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba00cc610 0x7feba00cca90 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.435+0000 7feba5ffd640 1 --2- 172.21.15.115:0/2955403343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8c004d00 0x7feba0158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.435+0000 7feba5ffd640 1 -- 172.21.15.115:0/2955403343 >> 172.21.15.115:0/2955403343 conn(0x7feba0066210 msgr2=0x7feba00b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.436+0000 7feba5ffd640 1 -- 172.21.15.115:0/2955403343 shutdown_connections 2023-12-01T03:10:45.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:45.436+0000 7feba5ffd640 1 -- 172.21.15.115:0/2955403343 wait complete. 2023-12-01T03:10:45.457 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-12-01T03:10:43.800176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594553,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":593,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":373,"ondisk_log_size":373,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":250,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425752,"kb_used_data":22360,"kb_used_omap":0,"kb_used_meta":403328,"kb_avail":754548968,"statfs":{"total":773094113280,"available":772658143232,"internally_reserved":0,"allocated":22896640,"data_stored":11748936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":413007872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1569,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001845"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507374+0000","last_change":"2023-12-01T03:10:38.507374+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507374+0000","last_undegraded":"2023-12-01T03:10:38.507374+0000","last_fullsized":"2023-12-01T03:10:38.507374+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:14.969594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"201'326","reported_seq":1123,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983105+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:10:38.983105+0000","last_peered":"2023-12-01T03:10:38.983105+0000","last_clean":"2023-12-01T03:10:38.983105+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:10:38.983105+0000","last_undegraded":"2023-12-01T03:10:38.983105+0000","last_fullsized":"2023-12-01T03:10:38.983105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988127+0000","last_change":"2023-12-01T03:10:24.685430+0000","last_active":"2023-12-01T03:10:38.988127+0000","last_peered":"2023-12-01T03:10:38.988127+0000","last_clean":"2023-12-01T03:10:38.988127+0000","last_became_active":"2023-12-01T03:10:24.684919+0000","last_became_peered":"2023-12-01T03:10:24.684919+0000","last_unstale":"2023-12-01T03:10:38.988127+0000","last_undegraded":"2023-12-01T03:10:38.988127+0000","last_fullsized":"2023-12-01T03:10:38.988127+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481058+0000","last_change":"2023-12-01T03:10:25.189213+0000","last_active":"2023-12-01T03:10:37.481058+0000","last_peered":"2023-12-01T03:10:37.481058+0000","last_clean":"2023-12-01T03:10:37.481058+0000","last_became_active":"2023-12-01T03:10:25.188927+0000","last_became_peered":"2023-12-01T03:10:25.188927+0000","last_unstale":"2023-12-01T03:10:37.481058+0000","last_undegraded":"2023-12-01T03:10:37.481058+0000","last_fullsized":"2023-12-01T03:10:37.481058+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:21:53.633800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520651+0000","last_change":"2023-12-01T03:10:38.517472+0000","last_active":"2023-12-01T03:10:38.520651+0000","last_peered":"2023-12-01T03:10:38.520651+0000","last_clean":"2023-12-01T03:10:38.520651+0000","last_became_active":"2023-12-01T03:10:38.517218+0000","last_became_peered":"2023-12-01T03:10:38.517218+0000","last_unstale":"2023-12-01T03:10:38.520651+0000","last_undegraded":"2023-12-01T03:10:38.520651+0000","last_fullsized":"2023-12-01T03:10:38.520651+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:57.653086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499505+0000","last_change":"2023-12-01T03:10:38.499505+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499505+0000","last_undegraded":"2023-12-01T03:10:38.499505+0000","last_fullsized":"2023-12-01T03:10:38.499505+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481362+0000","last_change":"2023-12-01T03:10:24.690155+0000","last_active":"2023-12-01T03:10:37.481362+0000","last_peered":"2023-12-01T03:10:37.481362+0000","last_clean":"2023-12-01T03:10:37.481362+0000","last_became_active":"2023-12-01T03:10:24.689122+0000","last_became_peered":"2023-12-01T03:10:24.689122+0000","last_unstale":"2023-12-01T03:10:37.481362+0000","last_undegraded":"2023-12-01T03:10:37.481362+0000","last_fullsized":"2023-12-01T03:10:37.481362+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479122+0000","last_change":"2023-12-01T03:10:25.191828+0000","last_active":"2023-12-01T03:10:37.479122+0000","last_peered":"2023-12-01T03:10:37.479122+0000","last_clean":"2023-12-01T03:10:37.479122+0000","last_became_active":"2023-12-01T03:10:25.191436+0000","last_became_peered":"2023-12-01T03:10:25.191436+0000","last_unstale":"2023-12-01T03:10:37.479122+0000","last_undegraded":"2023-12-01T03:10:37.479122+0000","last_fullsized":"2023-12-01T03:10:37.479122+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525987+0000","last_change":"2023-12-01T03:10:38.524843+0000","last_active":"2023-12-01T03:10:38.525987+0000","last_peered":"2023-12-01T03:10:38.525987+0000","last_clean":"2023-12-01T03:10:38.525987+0000","last_became_active":"2023-12-01T03:10:38.524424+0000","last_became_peered":"2023-12-01T03:10:38.524424+0000","last_unstale":"2023-12-01T03:10:38.525987+0000","last_undegraded":"2023-12-01T03:10:38.525987+0000","last_fullsized":"2023-12-01T03:10:38.525987+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:32.967785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.509995+0000","last_change":"2023-12-01T03:10:38.509995+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.509995+0000","last_undegraded":"2023-12-01T03:10:38.509995+0000","last_fullsized":"2023-12-01T03:10:38.509995+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983588+0000","last_change":"2023-12-01T03:10:24.684248+0000","last_active":"2023-12-01T03:10:38.983588+0000","last_peered":"2023-12-01T03:10:38.983588+0000","last_clean":"2023-12-01T03:10:38.983588+0000","last_became_active":"2023-12-01T03:10:24.680714+0000","last_became_peered":"2023-12-01T03:10:24.680714+0000","last_unstale":"2023-12-01T03:10:38.983588+0000","last_undegraded":"2023-12-01T03:10:38.983588+0000","last_fullsized":"2023-12-01T03:10:38.983588+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:19:02.474558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.982169+0000","last_change":"2023-12-01T03:10:25.174085+0000","last_active":"2023-12-01T03:10:38.982169+0000","last_peered":"2023-12-01T03:10:38.982169+0000","last_clean":"2023-12-01T03:10:38.982169+0000","last_became_active":"2023-12-01T03:10:25.173802+0000","last_became_peered":"2023-12-01T03:10:25.173802+0000","last_unstale":"2023-12-01T03:10:38.982169+0000","last_undegraded":"2023-12-01T03:10:38.982169+0000","last_fullsized":"2023-12-01T03:10:38.982169+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:39.145914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.982926+0000","last_change":"2023-12-01T03:10:38.512523+0000","last_active":"2023-12-01T03:10:38.982926+0000","last_peered":"2023-12-01T03:10:38.982926+0000","last_clean":"2023-12-01T03:10:38.982926+0000","last_became_active":"2023-12-01T03:10:38.512225+0000","last_became_peered":"2023-12-01T03:10:38.512225+0000","last_unstale":"2023-12-01T03:10:38.982926+0000","last_undegraded":"2023-12-01T03:10:38.982926+0000","last_fullsized":"2023-12-01T03:10:38.982926+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:05.819515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003662+0000","last_change":"2023-12-01T03:10:39.001307+0000","last_active":"2023-12-01T03:10:39.003662+0000","last_peered":"2023-12-01T03:10:39.003662+0000","last_clean":"2023-12-01T03:10:39.003662+0000","last_became_active":"2023-12-01T03:10:39.000379+0000","last_became_peered":"2023-12-01T03:10:39.000379+0000","last_unstale":"2023-12-01T03:10:39.003662+0000","last_undegraded":"2023-12-01T03:10:39.003662+0000","last_fullsized":"2023-12-01T03:10:39.003662+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:39.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986591+0000","last_change":"2023-12-01T03:10:24.683240+0000","last_active":"2023-12-01T03:10:38.986591+0000","last_peered":"2023-12-01T03:10:38.986591+0000","last_clean":"2023-12-01T03:10:38.986591+0000","last_became_active":"2023-12-01T03:10:24.682933+0000","last_became_peered":"2023-12-01T03:10:24.682933+0000","last_unstale":"2023-12-01T03:10:38.986591+0000","last_undegraded":"2023-12-01T03:10:38.986591+0000","last_fullsized":"2023-12-01T03:10:38.986591+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:40.211424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472633+0000","last_change":"2023-12-01T03:10:25.181396+0000","last_active":"2023-12-01T03:10:37.472633+0000","last_peered":"2023-12-01T03:10:37.472633+0000","last_clean":"2023-12-01T03:10:37.472633+0000","last_became_active":"2023-12-01T03:10:25.181134+0000","last_became_peered":"2023-12-01T03:10:25.181134+0000","last_unstale":"2023-12-01T03:10:37.472633+0000","last_undegraded":"2023-12-01T03:10:37.472633+0000","last_fullsized":"2023-12-01T03:10:37.472633+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:04.776013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.981977+0000","last_change":"2023-12-01T03:10:38.522641+0000","last_active":"2023-12-01T03:10:38.981977+0000","last_peered":"2023-12-01T03:10:38.981977+0000","last_clean":"2023-12-01T03:10:38.981977+0000","last_became_active":"2023-12-01T03:10:38.522225+0000","last_became_peered":"2023-12-01T03:10:38.522225+0000","last_unstale":"2023-12-01T03:10:38.981977+0000","last_undegraded":"2023-12-01T03:10:38.981977+0000","last_fullsized":"2023-12-01T03:10:38.981977+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:13.766818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499285+0000","last_change":"2023-12-01T03:10:38.499285+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499285+0000","last_undegraded":"2023-12-01T03:10:38.499285+0000","last_fullsized":"2023-12-01T03:10:38.499285+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985936+0000","last_change":"2023-12-01T03:10:24.687326+0000","last_active":"2023-12-01T03:10:38.985936+0000","last_peered":"2023-12-01T03:10:38.985936+0000","last_clean":"2023-12-01T03:10:38.985936+0000","last_became_active":"2023-12-01T03:10:24.687036+0000","last_became_peered":"2023-12-01T03:10:24.687036+0000","last_unstale":"2023-12-01T03:10:38.985936+0000","last_undegraded":"2023-12-01T03:10:38.985936+0000","last_fullsized":"2023-12-01T03:10:38.985936+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:36:57.969605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986462+0000","last_change":"2023-12-01T03:10:25.190210+0000","last_active":"2023-12-01T03:10:38.986462+0000","last_peered":"2023-12-01T03:10:38.986462+0000","last_clean":"2023-12-01T03:10:38.986462+0000","last_became_active":"2023-12-01T03:10:25.189957+0000","last_became_peered":"2023-12-01T03:10:25.189957+0000","last_unstale":"2023-12-01T03:10:38.986462+0000","last_undegraded":"2023-12-01T03:10:38.986462+0000","last_fullsized":"2023-12-01T03:10:38.986462+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:01.783625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988039+0000","last_change":"2023-12-01T03:10:38.514437+0000","last_active":"2023-12-01T03:10:38.988039+0000","last_peered":"2023-12-01T03:10:38.988039+0000","last_clean":"2023-12-01T03:10:38.988039+0000","last_became_active":"2023-12-01T03:10:38.514155+0000","last_became_peered":"2023-12-01T03:10:38.514155+0000","last_unstale":"2023-12-01T03:10:38.988039+0000","last_undegraded":"2023-12-01T03:10:38.988039+0000","last_fullsized":"2023-12-01T03:10:38.988039+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:06.817444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.503490+0000","last_change":"2023-12-01T03:10:38.503490+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.503490+0000","last_undegraded":"2023-12-01T03:10:38.503490+0000","last_fullsized":"2023-12-01T03:10:38.503490+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481503+0000","last_change":"2023-12-01T03:10:24.697595+0000","last_active":"2023-12-01T03:10:37.481503+0000","last_peered":"2023-12-01T03:10:37.481503+0000","last_clean":"2023-12-01T03:10:37.481503+0000","last_became_active":"2023-12-01T03:10:24.697311+0000","last_became_peered":"2023-12-01T03:10:24.697311+0000","last_unstale":"2023-12-01T03:10:37.481503+0000","last_undegraded":"2023-12-01T03:10:37.481503+0000","last_fullsized":"2023-12-01T03:10:37.481503+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:45.743456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483634+0000","last_change":"2023-12-01T03:10:25.192028+0000","last_active":"2023-12-01T03:10:37.483634+0000","last_peered":"2023-12-01T03:10:37.483634+0000","last_clean":"2023-12-01T03:10:37.483634+0000","last_became_active":"2023-12-01T03:10:25.191747+0000","last_became_peered":"2023-12-01T03:10:25.191747+0000","last_unstale":"2023-12-01T03:10:37.483634+0000","last_undegraded":"2023-12-01T03:10:37.483634+0000","last_fullsized":"2023-12-01T03:10:37.483634+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520576+0000","last_change":"2023-12-01T03:10:38.517012+0000","last_active":"2023-12-01T03:10:38.520576+0000","last_peered":"2023-12-01T03:10:38.520576+0000","last_clean":"2023-12-01T03:10:38.520576+0000","last_became_active":"2023-12-01T03:10:38.516749+0000","last_became_peered":"2023-12-01T03:10:38.516749+0000","last_unstale":"2023-12-01T03:10:38.520576+0000","last_undegraded":"2023-12-01T03:10:38.520576+0000","last_fullsized":"2023-12-01T03:10:38.520576+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:29.962162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008414+0000","last_change":"2023-12-01T03:10:39.005634+0000","last_active":"2023-12-01T03:10:39.008414+0000","last_peered":"2023-12-01T03:10:39.008414+0000","last_clean":"2023-12-01T03:10:39.008414+0000","last_became_active":"2023-12-01T03:10:39.005386+0000","last_became_peered":"2023-12-01T03:10:39.005386+0000","last_unstale":"2023-12-01T03:10:39.008414+0000","last_undegraded":"2023-12-01T03:10:39.008414+0000","last_fullsized":"2023-12-01T03:10:39.008414+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488889+0000","last_change":"2023-12-01T03:10:24.697992+0000","last_active":"2023-12-01T03:10:37.488889+0000","last_peered":"2023-12-01T03:10:37.488889+0000","last_clean":"2023-12-01T03:10:37.488889+0000","last_became_active":"2023-12-01T03:10:24.696883+0000","last_became_peered":"2023-12-01T03:10:24.696883+0000","last_unstale":"2023-12-01T03:10:37.488889+0000","last_undegraded":"2023-12-01T03:10:37.488889+0000","last_fullsized":"2023-12-01T03:10:37.488889+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.487148+0000","last_change":"2023-12-01T03:10:25.175476+0000","last_active":"2023-12-01T03:10:37.487148+0000","last_peered":"2023-12-01T03:10:37.487148+0000","last_clean":"2023-12-01T03:10:37.487148+0000","last_became_active":"2023-12-01T03:10:25.175211+0000","last_became_peered":"2023-12-01T03:10:25.175211+0000","last_unstale":"2023-12-01T03:10:37.487148+0000","last_undegraded":"2023-12-01T03:10:37.487148+0000","last_fullsized":"2023-12-01T03:10:37.487148+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:22.358205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515962+0000","last_change":"2023-12-01T03:10:38.513347+0000","last_active":"2023-12-01T03:10:38.515962+0000","last_peered":"2023-12-01T03:10:38.515962+0000","last_clean":"2023-12-01T03:10:38.515962+0000","last_became_active":"2023-12-01T03:10:38.512782+0000","last_became_peered":"2023-12-01T03:10:38.512782+0000","last_unstale":"2023-12-01T03:10:38.515962+0000","last_undegraded":"2023-12-01T03:10:38.515962+0000","last_fullsized":"2023-12-01T03:10:38.515962+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:06.647413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.010194+0000","last_change":"2023-12-01T03:10:39.007372+0000","last_active":"2023-12-01T03:10:39.010194+0000","last_peered":"2023-12-01T03:10:39.010194+0000","last_clean":"2023-12-01T03:10:39.010194+0000","last_became_active":"2023-12-01T03:10:39.007059+0000","last_became_peered":"2023-12-01T03:10:39.007059+0000","last_unstale":"2023-12-01T03:10:39.010194+0000","last_undegraded":"2023-12-01T03:10:39.010194+0000","last_fullsized":"2023-12-01T03:10:39.010194+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.492807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"214'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489570+0000","last_change":"2023-12-01T03:10:24.697890+0000","last_active":"2023-12-01T03:10:37.489570+0000","last_peered":"2023-12-01T03:10:37.489570+0000","last_clean":"2023-12-01T03:10:37.489570+0000","last_became_active":"2023-12-01T03:10:24.697679+0000","last_became_peered":"2023-12-01T03:10:24.697679+0000","last_unstale":"2023-12-01T03:10:37.489570+0000","last_undegraded":"2023-12-01T03:10:37.489570+0000","last_fullsized":"2023-12-01T03:10:37.489570+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488204+0000","last_change":"2023-12-01T03:10:25.168359+0000","last_active":"2023-12-01T03:10:37.488204+0000","last_peered":"2023-12-01T03:10:37.488204+0000","last_clean":"2023-12-01T03:10:37.488204+0000","last_became_active":"2023-12-01T03:10:25.168082+0000","last_became_peered":"2023-12-01T03:10:25.168082+0000","last_unstale":"2023-12-01T03:10:37.488204+0000","last_undegraded":"2023-12-01T03:10:37.488204+0000","last_fullsized":"2023-12-01T03:10:37.488204+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:06.024028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525349+0000","last_change":"2023-12-01T03:10:38.524232+0000","last_active":"2023-12-01T03:10:38.525349+0000","last_peered":"2023-12-01T03:10:38.525349+0000","last_clean":"2023-12-01T03:10:38.525349+0000","last_became_active":"2023-12-01T03:10:38.523871+0000","last_became_peered":"2023-12-01T03:10:38.523871+0000","last_unstale":"2023-12-01T03:10:38.525349+0000","last_undegraded":"2023-12-01T03:10:38.525349+0000","last_fullsized":"2023-12-01T03:10:38.525349+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:06.647413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.499505+0000","last_change":"2023-12-01T03:10:38.499505+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.499505+0000","last_undegraded":"2023-12-01T03:10:38.499505+0000","last_fullsized":"2023-12-01T03:10:38.499505+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.7","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472938+0000","last_change":"2023-12-01T03:10:24.687160+0000","last_active":"2023-12-01T03:10:37.472938+0000","last_peered":"2023-12-01T03:10:37.472938+0000","last_clean":"2023-12-01T03:10:37.472938+0000","last_became_active":"2023-12-01T03:10:24.686184+0000","last_became_peered":"2023-12-01T03:10:24.686184+0000","last_unstale":"2023-12-01T03:10:37.472938+0000","last_undegraded":"2023-12-01T03:10:37.472938+0000","last_fullsized":"2023-12-01T03:10:37.472938+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:05.006954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489573+0000","last_change":"2023-12-01T03:10:25.184992+0000","last_active":"2023-12-01T03:10:37.489573+0000","last_peered":"2023-12-01T03:10:37.489573+0000","last_clean":"2023-12-01T03:10:37.489573+0000","last_became_active":"2023-12-01T03:10:25.184670+0000","last_became_peered":"2023-12-01T03:10:25.184670+0000","last_unstale":"2023-12-01T03:10:37.489573+0000","last_undegraded":"2023-12-01T03:10:37.489573+0000","last_fullsized":"2023-12-01T03:10:37.489573+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:40.143822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519720+0000","last_change":"2023-12-01T03:10:38.517303+0000","last_active":"2023-12-01T03:10:38.519720+0000","last_peered":"2023-12-01T03:10:38.519720+0000","last_clean":"2023-12-01T03:10:38.519720+0000","last_became_active":"2023-12-01T03:10:38.516948+0000","last_became_peered":"2023-12-01T03:10:38.516948+0000","last_unstale":"2023-12-01T03:10:38.519720+0000","last_undegraded":"2023-12-01T03:10:38.519720+0000","last_fullsized":"2023-12-01T03:10:38.519720+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:58:41.627485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504217+0000","last_change":"2023-12-01T03:10:38.504217+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504217+0000","last_undegraded":"2023-12-01T03:10:38.504217+0000","last_fullsized":"2023-12-01T03:10:38.504217+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:18.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472354+0000","last_change":"2023-12-01T03:10:24.685003+0000","last_active":"2023-12-01T03:10:37.472354+0000","last_peered":"2023-12-01T03:10:37.472354+0000","last_clean":"2023-12-01T03:10:37.472354+0000","last_became_active":"2023-12-01T03:10:24.684425+0000","last_became_peered":"2023-12-01T03:10:24.684425+0000","last_unstale":"2023-12-01T03:10:37.472354+0000","last_undegraded":"2023-12-01T03:10:37.472354+0000","last_fullsized":"2023-12-01T03:10:37.472354+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:45.173492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491598+0000","last_change":"2023-12-01T03:10:25.174479+0000","last_active":"2023-12-01T03:10:37.491598+0000","last_peered":"2023-12-01T03:10:37.491598+0000","last_clean":"2023-12-01T03:10:37.491598+0000","last_became_active":"2023-12-01T03:10:25.174194+0000","last_became_peered":"2023-12-01T03:10:25.174194+0000","last_unstale":"2023-12-01T03:10:37.491598+0000","last_undegraded":"2023-12-01T03:10:37.491598+0000","last_fullsized":"2023-12-01T03:10:37.491598+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519343+0000","last_change":"2023-12-01T03:10:38.516993+0000","last_active":"2023-12-01T03:10:38.519343+0000","last_peered":"2023-12-01T03:10:38.519343+0000","last_clean":"2023-12-01T03:10:38.519343+0000","last_became_active":"2023-12-01T03:10:38.516717+0000","last_became_peered":"2023-12-01T03:10:38.516717+0000","last_unstale":"2023-12-01T03:10:38.519343+0000","last_undegraded":"2023-12-01T03:10:38.519343+0000","last_fullsized":"2023-12-01T03:10:38.519343+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:51.689717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508392+0000","last_change":"2023-12-01T03:10:38.508392+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508392+0000","last_undegraded":"2023-12-01T03:10:38.508392+0000","last_fullsized":"2023-12-01T03:10:38.508392+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:53.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481282+0000","last_change":"2023-12-01T03:10:24.694594+0000","last_active":"2023-12-01T03:10:37.481282+0000","last_peered":"2023-12-01T03:10:37.481282+0000","last_clean":"2023-12-01T03:10:37.481282+0000","last_became_active":"2023-12-01T03:10:24.693808+0000","last_became_peered":"2023-12-01T03:10:24.693808+0000","last_unstale":"2023-12-01T03:10:37.481282+0000","last_undegraded":"2023-12-01T03:10:37.481282+0000","last_fullsized":"2023-12-01T03:10:37.481282+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472327+0000","last_change":"2023-12-01T03:10:25.173728+0000","last_active":"2023-12-01T03:10:37.472327+0000","last_peered":"2023-12-01T03:10:37.472327+0000","last_clean":"2023-12-01T03:10:37.472327+0000","last_became_active":"2023-12-01T03:10:25.173449+0000","last_became_peered":"2023-12-01T03:10:25.173449+0000","last_unstale":"2023-12-01T03:10:37.472327+0000","last_undegraded":"2023-12-01T03:10:37.472327+0000","last_fullsized":"2023-12-01T03:10:37.472327+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:20.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.513730+0000","last_change":"2023-12-01T03:10:38.508401+0000","last_active":"2023-12-01T03:10:38.513730+0000","last_peered":"2023-12-01T03:10:38.513730+0000","last_clean":"2023-12-01T03:10:38.513730+0000","last_became_active":"2023-12-01T03:10:38.508089+0000","last_became_peered":"2023-12-01T03:10:38.508089+0000","last_unstale":"2023-12-01T03:10:38.513730+0000","last_undegraded":"2023-12-01T03:10:38.513730+0000","last_fullsized":"2023-12-01T03:10:38.513730+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:56.645685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003642+0000","last_change":"2023-12-01T03:10:39.000679+0000","last_active":"2023-12-01T03:10:39.003642+0000","last_peered":"2023-12-01T03:10:39.003642+0000","last_clean":"2023-12-01T03:10:39.003642+0000","last_became_active":"2023-12-01T03:10:38.997903+0000","last_became_peered":"2023-12-01T03:10:38.997903+0000","last_unstale":"2023-12-01T03:10:39.003642+0000","last_undegraded":"2023-12-01T03:10:39.003642+0000","last_fullsized":"2023-12-01T03:10:39.003642+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:27.357798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482736+0000","last_change":"2023-12-01T03:10:24.685775+0000","last_active":"2023-12-01T03:10:37.482736+0000","last_peered":"2023-12-01T03:10:37.482736+0000","last_clean":"2023-12-01T03:10:37.482736+0000","last_became_active":"2023-12-01T03:10:24.685178+0000","last_became_peered":"2023-12-01T03:10:24.685178+0000","last_unstale":"2023-12-01T03:10:37.482736+0000","last_undegraded":"2023-12-01T03:10:37.482736+0000","last_fullsized":"2023-12-01T03:10:37.482736+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:21.942226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483463+0000","last_change":"2023-12-01T03:10:25.180768+0000","last_active":"2023-12-01T03:10:37.483463+0000","last_peered":"2023-12-01T03:10:37.483463+0000","last_clean":"2023-12-01T03:10:37.483463+0000","last_became_active":"2023-12-01T03:10:25.179796+0000","last_became_peered":"2023-12-01T03:10:25.179796+0000","last_unstale":"2023-12-01T03:10:37.483463+0000","last_undegraded":"2023-12-01T03:10:37.483463+0000","last_fullsized":"2023-12-01T03:10:37.483463+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515225+0000","last_change":"2023-12-01T03:10:38.511676+0000","last_active":"2023-12-01T03:10:38.515225+0000","last_peered":"2023-12-01T03:10:38.515225+0000","last_clean":"2023-12-01T03:10:38.515225+0000","last_became_active":"2023-12-01T03:10:38.511406+0000","last_became_peered":"2023-12-01T03:10:38.511406+0000","last_unstale":"2023-12-01T03:10:38.515225+0000","last_undegraded":"2023-12-01T03:10:38.515225+0000","last_fullsized":"2023-12-01T03:10:38.515225+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:44.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003720+0000","last_change":"2023-12-01T03:10:38.997814+0000","last_active":"2023-12-01T03:10:39.003720+0000","last_peered":"2023-12-01T03:10:39.003720+0000","last_clean":"2023-12-01T03:10:39.003720+0000","last_became_active":"2023-12-01T03:10:38.997558+0000","last_became_peered":"2023-12-01T03:10:38.997558+0000","last_unstale":"2023-12-01T03:10:39.003720+0000","last_undegraded":"2023-12-01T03:10:39.003720+0000","last_fullsized":"2023-12-01T03:10:39.003720+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:49:09.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.b","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481673+0000","last_change":"2023-12-01T03:10:24.695154+0000","last_active":"2023-12-01T03:10:37.481673+0000","last_peered":"2023-12-01T03:10:37.481673+0000","last_clean":"2023-12-01T03:10:37.481673+0000","last_became_active":"2023-12-01T03:10:24.694868+0000","last_became_peered":"2023-12-01T03:10:24.694868+0000","last_unstale":"2023-12-01T03:10:37.481673+0000","last_undegraded":"2023-12-01T03:10:37.481673+0000","last_fullsized":"2023-12-01T03:10:37.481673+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482517+0000","last_change":"2023-12-01T03:10:25.169678+0000","last_active":"2023-12-01T03:10:37.482517+0000","last_peered":"2023-12-01T03:10:37.482517+0000","last_clean":"2023-12-01T03:10:37.482517+0000","last_became_active":"2023-12-01T03:10:25.168714+0000","last_became_peered":"2023-12-01T03:10:25.168714+0000","last_unstale":"2023-12-01T03:10:37.482517+0000","last_undegraded":"2023-12-01T03:10:37.482517+0000","last_fullsized":"2023-12-01T03:10:37.482517+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.526019+0000","last_change":"2023-12-01T03:10:38.524888+0000","last_active":"2023-12-01T03:10:38.526019+0000","last_peered":"2023-12-01T03:10:38.526019+0000","last_clean":"2023-12-01T03:10:38.526019+0000","last_became_active":"2023-12-01T03:10:38.524601+0000","last_became_peered":"2023-12-01T03:10:38.524601+0000","last_unstale":"2023-12-01T03:10:38.526019+0000","last_undegraded":"2023-12-01T03:10:38.526019+0000","last_fullsized":"2023-12-01T03:10:38.526019+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:47.875991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.011924+0000","last_change":"2023-12-01T03:10:39.009719+0000","last_active":"2023-12-01T03:10:39.011924+0000","last_peered":"2023-12-01T03:10:39.011924+0000","last_clean":"2023-12-01T03:10:39.011924+0000","last_became_active":"2023-12-01T03:10:39.009561+0000","last_became_peered":"2023-12-01T03:10:39.009561+0000","last_unstale":"2023-12-01T03:10:39.011924+0000","last_undegraded":"2023-12-01T03:10:39.011924+0000","last_fullsized":"2023-12-01T03:10:39.011924+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:19.853707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473031+0000","last_change":"2023-12-01T03:10:24.683594+0000","last_active":"2023-12-01T03:10:37.473031+0000","last_peered":"2023-12-01T03:10:37.473031+0000","last_clean":"2023-12-01T03:10:37.473031+0000","last_became_active":"2023-12-01T03:10:24.679979+0000","last_became_peered":"2023-12-01T03:10:24.679979+0000","last_unstale":"2023-12-01T03:10:37.473031+0000","last_undegraded":"2023-12-01T03:10:37.473031+0000","last_fullsized":"2023-12-01T03:10:37.473031+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:12.511044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987554+0000","last_change":"2023-12-01T03:10:25.172837+0000","last_active":"2023-12-01T03:10:38.987554+0000","last_peered":"2023-12-01T03:10:38.987554+0000","last_clean":"2023-12-01T03:10:38.987554+0000","last_became_active":"2023-12-01T03:10:25.172602+0000","last_became_peered":"2023-12-01T03:10:25.172602+0000","last_unstale":"2023-12-01T03:10:38.987554+0000","last_undegraded":"2023-12-01T03:10:38.987554+0000","last_fullsized":"2023-12-01T03:10:38.987554+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521284+0000","last_change":"2023-12-01T03:10:38.517877+0000","last_active":"2023-12-01T03:10:38.521284+0000","last_peered":"2023-12-01T03:10:38.521284+0000","last_clean":"2023-12-01T03:10:38.521284+0000","last_became_active":"2023-12-01T03:10:38.517241+0000","last_became_peered":"2023-12-01T03:10:38.517241+0000","last_unstale":"2023-12-01T03:10:38.521284+0000","last_undegraded":"2023-12-01T03:10:38.521284+0000","last_fullsized":"2023-12-01T03:10:38.521284+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:32.967785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.004933+0000","last_change":"2023-12-01T03:10:39.003555+0000","last_active":"2023-12-01T03:10:39.004933+0000","last_peered":"2023-12-01T03:10:39.004933+0000","last_clean":"2023-12-01T03:10:39.004933+0000","last_became_active":"2023-12-01T03:10:39.003311+0000","last_became_peered":"2023-12-01T03:10:39.003311+0000","last_unstale":"2023-12-01T03:10:39.004933+0000","last_undegraded":"2023-12-01T03:10:39.004933+0000","last_fullsized":"2023-12-01T03:10:39.004933+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.d","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473150+0000","last_change":"2023-12-01T03:10:24.691398+0000","last_active":"2023-12-01T03:10:37.473150+0000","last_peered":"2023-12-01T03:10:37.473150+0000","last_clean":"2023-12-01T03:10:37.473150+0000","last_became_active":"2023-12-01T03:10:24.691110+0000","last_became_peered":"2023-12-01T03:10:24.691110+0000","last_unstale":"2023-12-01T03:10:37.473150+0000","last_undegraded":"2023-12-01T03:10:37.473150+0000","last_fullsized":"2023-12-01T03:10:37.473150+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:24.670899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999898+0000","last_change":"2023-12-01T03:10:25.178959+0000","last_active":"2023-12-01T03:10:38.999898+0000","last_peered":"2023-12-01T03:10:38.999898+0000","last_clean":"2023-12-01T03:10:38.999898+0000","last_became_active":"2023-12-01T03:10:25.178268+0000","last_became_peered":"2023-12-01T03:10:25.178268+0000","last_unstale":"2023-12-01T03:10:38.999898+0000","last_undegraded":"2023-12-01T03:10:38.999898+0000","last_fullsized":"2023-12-01T03:10:38.999898+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:55:31.814432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525071+0000","last_change":"2023-12-01T03:10:38.523799+0000","last_active":"2023-12-01T03:10:38.525071+0000","last_peered":"2023-12-01T03:10:38.525071+0000","last_clean":"2023-12-01T03:10:38.525071+0000","last_became_active":"2023-12-01T03:10:38.523512+0000","last_became_peered":"2023-12-01T03:10:38.523512+0000","last_unstale":"2023-12-01T03:10:38.525071+0000","last_undegraded":"2023-12-01T03:10:38.525071+0000","last_fullsized":"2023-12-01T03:10:38.525071+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:44.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504526+0000","last_change":"2023-12-01T03:10:38.504526+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504526+0000","last_undegraded":"2023-12-01T03:10:38.504526+0000","last_fullsized":"2023-12-01T03:10:38.504526+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:41.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482803+0000","last_change":"2023-12-01T03:10:24.688107+0000","last_active":"2023-12-01T03:10:37.482803+0000","last_peered":"2023-12-01T03:10:37.482803+0000","last_clean":"2023-12-01T03:10:37.482803+0000","last_became_active":"2023-12-01T03:10:24.686930+0000","last_became_peered":"2023-12-01T03:10:24.686930+0000","last_unstale":"2023-12-01T03:10:37.482803+0000","last_undegraded":"2023-12-01T03:10:37.482803+0000","last_fullsized":"2023-12-01T03:10:37.482803+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:44.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986385+0000","last_change":"2023-12-01T03:10:25.180913+0000","last_active":"2023-12-01T03:10:38.986385+0000","last_peered":"2023-12-01T03:10:38.986385+0000","last_clean":"2023-12-01T03:10:38.986385+0000","last_became_active":"2023-12-01T03:10:25.180626+0000","last_became_peered":"2023-12-01T03:10:25.180626+0000","last_unstale":"2023-12-01T03:10:38.986385+0000","last_undegraded":"2023-12-01T03:10:38.986385+0000","last_fullsized":"2023-12-01T03:10:38.986385+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.520264+0000","last_change":"2023-12-01T03:10:38.519139+0000","last_active":"2023-12-01T03:10:38.520264+0000","last_peered":"2023-12-01T03:10:38.520264+0000","last_clean":"2023-12-01T03:10:38.520264+0000","last_became_active":"2023-12-01T03:10:38.518855+0000","last_became_peered":"2023-12-01T03:10:38.518855+0000","last_unstale":"2023-12-01T03:10:38.520264+0000","last_undegraded":"2023-12-01T03:10:38.520264+0000","last_fullsized":"2023-12-01T03:10:38.520264+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:52.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511251+0000","last_change":"2023-12-01T03:10:38.511251+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511251+0000","last_undegraded":"2023-12-01T03:10:38.511251+0000","last_fullsized":"2023-12-01T03:10:38.511251+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:47:30.021172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.f","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482669+0000","last_change":"2023-12-01T03:10:24.699945+0000","last_active":"2023-12-01T03:10:37.482669+0000","last_peered":"2023-12-01T03:10:37.482669+0000","last_clean":"2023-12-01T03:10:37.482669+0000","last_became_active":"2023-12-01T03:10:24.699598+0000","last_became_peered":"2023-12-01T03:10:24.699598+0000","last_unstale":"2023-12-01T03:10:37.482669+0000","last_undegraded":"2023-12-01T03:10:37.482669+0000","last_fullsized":"2023-12-01T03:10:37.482669+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492690+0000","last_change":"2023-12-01T03:10:25.181692+0000","last_active":"2023-12-01T03:10:37.492690+0000","last_peered":"2023-12-01T03:10:37.492690+0000","last_clean":"2023-12-01T03:10:37.492690+0000","last_became_active":"2023-12-01T03:10:25.181316+0000","last_became_peered":"2023-12-01T03:10:25.181316+0000","last_unstale":"2023-12-01T03:10:37.492690+0000","last_undegraded":"2023-12-01T03:10:37.492690+0000","last_fullsized":"2023-12-01T03:10:37.492690+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:57:07.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.522721+0000","last_change":"2023-12-01T03:10:38.518068+0000","last_active":"2023-12-01T03:10:38.522721+0000","last_peered":"2023-12-01T03:10:38.522721+0000","last_clean":"2023-12-01T03:10:38.522721+0000","last_became_active":"2023-12-01T03:10:38.517792+0000","last_became_peered":"2023-12-01T03:10:38.517792+0000","last_unstale":"2023-12-01T03:10:38.522721+0000","last_undegraded":"2023-12-01T03:10:38.522721+0000","last_fullsized":"2023-12-01T03:10:38.522721+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:01.616973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511262+0000","last_change":"2023-12-01T03:10:38.511262+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511262+0000","last_undegraded":"2023-12-01T03:10:38.511262+0000","last_fullsized":"2023-12-01T03:10:38.511262+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"213'8","reported_seq":21,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483272+0000","last_change":"2023-12-01T03:10:24.685759+0000","last_active":"2023-12-01T03:10:37.483272+0000","last_peered":"2023-12-01T03:10:37.483272+0000","last_clean":"2023-12-01T03:10:37.483272+0000","last_became_active":"2023-12-01T03:10:24.684800+0000","last_became_peered":"2023-12-01T03:10:24.684800+0000","last_unstale":"2023-12-01T03:10:37.483272+0000","last_undegraded":"2023-12-01T03:10:37.483272+0000","last_fullsized":"2023-12-01T03:10:37.483272+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:29.129070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478917+0000","last_change":"2023-12-01T03:10:25.193021+0000","last_active":"2023-12-01T03:10:37.478917+0000","last_peered":"2023-12-01T03:10:37.478917+0000","last_clean":"2023-12-01T03:10:37.478917+0000","last_became_active":"2023-12-01T03:10:25.192051+0000","last_became_peered":"2023-12-01T03:10:25.192051+0000","last_unstale":"2023-12-01T03:10:37.478917+0000","last_undegraded":"2023-12-01T03:10:37.478917+0000","last_fullsized":"2023-12-01T03:10:37.478917+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521581+0000","last_change":"2023-12-01T03:10:38.517969+0000","last_active":"2023-12-01T03:10:38.521581+0000","last_peered":"2023-12-01T03:10:38.521581+0000","last_clean":"2023-12-01T03:10:38.521581+0000","last_became_active":"2023-12-01T03:10:38.517693+0000","last_became_peered":"2023-12-01T03:10:38.517693+0000","last_unstale":"2023-12-01T03:10:38.521581+0000","last_undegraded":"2023-12-01T03:10:38.521581+0000","last_fullsized":"2023-12-01T03:10:38.521581+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:18.447021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.498276+0000","last_change":"2023-12-01T03:10:38.498276+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.498276+0000","last_undegraded":"2023-12-01T03:10:38.498276+0000","last_fullsized":"2023-12-01T03:10:38.498276+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479511+0000","last_change":"2023-12-01T03:10:24.694942+0000","last_active":"2023-12-01T03:10:37.479511+0000","last_peered":"2023-12-01T03:10:37.479511+0000","last_clean":"2023-12-01T03:10:37.479511+0000","last_became_active":"2023-12-01T03:10:24.694664+0000","last_became_peered":"2023-12-01T03:10:24.694664+0000","last_unstale":"2023-12-01T03:10:37.479511+0000","last_undegraded":"2023-12-01T03:10:37.479511+0000","last_fullsized":"2023-12-01T03:10:37.479511+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:38:02.332793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988703+0000","last_change":"2023-12-01T03:10:25.164054+0000","last_active":"2023-12-01T03:10:38.988703+0000","last_peered":"2023-12-01T03:10:38.988703+0000","last_clean":"2023-12-01T03:10:38.988703+0000","last_became_active":"2023-12-01T03:10:25.163787+0000","last_became_peered":"2023-12-01T03:10:25.163787+0000","last_unstale":"2023-12-01T03:10:38.988703+0000","last_undegraded":"2023-12-01T03:10:38.988703+0000","last_fullsized":"2023-12-01T03:10:38.988703+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:04.568421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521758+0000","last_change":"2023-12-01T03:10:38.520314+0000","last_active":"2023-12-01T03:10:38.521758+0000","last_peered":"2023-12-01T03:10:38.521758+0000","last_clean":"2023-12-01T03:10:38.521758+0000","last_became_active":"2023-12-01T03:10:38.519056+0000","last_became_peered":"2023-12-01T03:10:38.519056+0000","last_unstale":"2023-12-01T03:10:38.521758+0000","last_undegraded":"2023-12-01T03:10:38.521758+0000","last_fullsized":"2023-12-01T03:10:38.521758+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:40.686905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.502663+0000","last_change":"2023-12-01T03:10:38.502663+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.502663+0000","last_undegraded":"2023-12-01T03:10:38.502663+0000","last_fullsized":"2023-12-01T03:10:38.502663+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491751+0000","last_change":"2023-12-01T03:10:24.685823+0000","last_active":"2023-12-01T03:10:37.491751+0000","last_peered":"2023-12-01T03:10:37.491751+0000","last_clean":"2023-12-01T03:10:37.491751+0000","last_became_active":"2023-12-01T03:10:24.685560+0000","last_became_peered":"2023-12-01T03:10:24.685560+0000","last_unstale":"2023-12-01T03:10:37.491751+0000","last_undegraded":"2023-12-01T03:10:37.491751+0000","last_fullsized":"2023-12-01T03:10:37.491751+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483380+0000","last_change":"2023-12-01T03:10:25.183804+0000","last_active":"2023-12-01T03:10:37.483380+0000","last_peered":"2023-12-01T03:10:37.483380+0000","last_clean":"2023-12-01T03:10:37.483380+0000","last_became_active":"2023-12-01T03:10:25.183079+0000","last_became_peered":"2023-12-01T03:10:25.183079+0000","last_unstale":"2023-12-01T03:10:37.483380+0000","last_undegraded":"2023-12-01T03:10:37.483380+0000","last_fullsized":"2023-12-01T03:10:37.483380+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.510374+0000","last_change":"2023-12-01T03:10:38.508144+0000","last_active":"2023-12-01T03:10:38.510374+0000","last_peered":"2023-12-01T03:10:38.510374+0000","last_clean":"2023-12-01T03:10:38.510374+0000","last_became_active":"2023-12-01T03:10:38.507850+0000","last_became_peered":"2023-12-01T03:10:38.507850+0000","last_unstale":"2023-12-01T03:10:38.510374+0000","last_undegraded":"2023-12-01T03:10:38.510374+0000","last_fullsized":"2023-12-01T03:10:38.510374+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:52.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507590+0000","last_change":"2023-12-01T03:10:38.507590+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507590+0000","last_undegraded":"2023-12-01T03:10:38.507590+0000","last_fullsized":"2023-12-01T03:10:38.507590+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:45.156063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.13","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987606+0000","last_change":"2023-12-01T03:10:24.692854+0000","last_active":"2023-12-01T03:10:38.987606+0000","last_peered":"2023-12-01T03:10:38.987606+0000","last_clean":"2023-12-01T03:10:38.987606+0000","last_became_active":"2023-12-01T03:10:24.692563+0000","last_became_peered":"2023-12-01T03:10:24.692563+0000","last_unstale":"2023-12-01T03:10:38.987606+0000","last_undegraded":"2023-12-01T03:10:38.987606+0000","last_fullsized":"2023-12-01T03:10:38.987606+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491687+0000","last_change":"2023-12-01T03:10:25.186716+0000","last_active":"2023-12-01T03:10:37.491687+0000","last_peered":"2023-12-01T03:10:37.491687+0000","last_clean":"2023-12-01T03:10:37.491687+0000","last_became_active":"2023-12-01T03:10:25.186422+0000","last_became_peered":"2023-12-01T03:10:25.186422+0000","last_unstale":"2023-12-01T03:10:37.491687+0000","last_undegraded":"2023-12-01T03:10:37.491687+0000","last_fullsized":"2023-12-01T03:10:37.491687+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.510490+0000","last_change":"2023-12-01T03:10:38.508098+0000","last_active":"2023-12-01T03:10:38.510490+0000","last_peered":"2023-12-01T03:10:38.510490+0000","last_clean":"2023-12-01T03:10:38.510490+0000","last_became_active":"2023-12-01T03:10:38.506510+0000","last_became_peered":"2023-12-01T03:10:38.506510+0000","last_unstale":"2023-12-01T03:10:38.510490+0000","last_undegraded":"2023-12-01T03:10:38.510490+0000","last_fullsized":"2023-12-01T03:10:38.510490+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:01.074406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.504758+0000","last_change":"2023-12-01T03:10:38.504758+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.504758+0000","last_undegraded":"2023-12-01T03:10:38.504758+0000","last_fullsized":"2023-12-01T03:10:38.504758+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:18.605693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"214'8","reported_seq":21,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481263+0000","last_change":"2023-12-01T03:10:24.697328+0000","last_active":"2023-12-01T03:10:37.481263+0000","last_peered":"2023-12-01T03:10:37.481263+0000","last_clean":"2023-12-01T03:10:37.481263+0000","last_became_active":"2023-12-01T03:10:24.696986+0000","last_became_peered":"2023-12-01T03:10:24.696986+0000","last_unstale":"2023-12-01T03:10:37.481263+0000","last_undegraded":"2023-12-01T03:10:37.481263+0000","last_fullsized":"2023-12-01T03:10:37.481263+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492498+0000","last_change":"2023-12-01T03:10:25.176864+0000","last_active":"2023-12-01T03:10:37.492498+0000","last_peered":"2023-12-01T03:10:37.492498+0000","last_clean":"2023-12-01T03:10:37.492498+0000","last_became_active":"2023-12-01T03:10:25.176588+0000","last_became_peered":"2023-12-01T03:10:25.176588+0000","last_unstale":"2023-12-01T03:10:37.492498+0000","last_undegraded":"2023-12-01T03:10:37.492498+0000","last_fullsized":"2023-12-01T03:10:37.492498+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987075+0000","last_change":"2023-12-01T03:10:38.519450+0000","last_active":"2023-12-01T03:10:38.987075+0000","last_peered":"2023-12-01T03:10:38.987075+0000","last_clean":"2023-12-01T03:10:38.987075+0000","last_became_active":"2023-12-01T03:10:38.519108+0000","last_became_peered":"2023-12-01T03:10:38.519108+0000","last_unstale":"2023-12-01T03:10:38.987075+0000","last_undegraded":"2023-12-01T03:10:38.987075+0000","last_fullsized":"2023-12-01T03:10:38.987075+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:58.926662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007897+0000","last_change":"2023-12-01T03:10:39.005006+0000","last_active":"2023-12-01T03:10:39.007897+0000","last_peered":"2023-12-01T03:10:39.007897+0000","last_clean":"2023-12-01T03:10:39.007897+0000","last_became_active":"2023-12-01T03:10:39.004573+0000","last_became_peered":"2023-12-01T03:10:39.004573+0000","last_unstale":"2023-12-01T03:10:39.007897+0000","last_undegraded":"2023-12-01T03:10:39.007897+0000","last_fullsized":"2023-12-01T03:10:39.007897+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:07.829269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"213'3","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481279+0000","last_change":"2023-12-01T03:10:24.699528+0000","last_active":"2023-12-01T03:10:37.481279+0000","last_peered":"2023-12-01T03:10:37.481279+0000","last_clean":"2023-12-01T03:10:37.481279+0000","last_became_active":"2023-12-01T03:10:24.699152+0000","last_became_peered":"2023-12-01T03:10:24.699152+0000","last_unstale":"2023-12-01T03:10:37.481279+0000","last_undegraded":"2023-12-01T03:10:37.481279+0000","last_fullsized":"2023-12-01T03:10:37.481279+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:07.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987736+0000","last_change":"2023-12-01T03:10:25.187487+0000","last_active":"2023-12-01T03:10:38.987736+0000","last_peered":"2023-12-01T03:10:38.987736+0000","last_clean":"2023-12-01T03:10:38.987736+0000","last_became_active":"2023-12-01T03:10:25.187198+0000","last_became_peered":"2023-12-01T03:10:25.187198+0000","last_unstale":"2023-12-01T03:10:38.987736+0000","last_undegraded":"2023-12-01T03:10:38.987736+0000","last_fullsized":"2023-12-01T03:10:38.987736+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:00.948964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.526023+0000","last_change":"2023-12-01T03:10:38.525203+0000","last_active":"2023-12-01T03:10:38.526023+0000","last_peered":"2023-12-01T03:10:38.526023+0000","last_clean":"2023-12-01T03:10:38.526023+0000","last_became_active":"2023-12-01T03:10:38.524926+0000","last_became_peered":"2023-12-01T03:10:38.524926+0000","last_unstale":"2023-12-01T03:10:38.526023+0000","last_undegraded":"2023-12-01T03:10:38.526023+0000","last_fullsized":"2023-12-01T03:10:38.526023+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:17.593868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.503576+0000","last_change":"2023-12-01T03:10:38.503576+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.503576+0000","last_undegraded":"2023-12-01T03:10:38.503576+0000","last_fullsized":"2023-12-01T03:10:38.503576+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492851+0000","last_change":"2023-12-01T03:10:24.691537+0000","last_active":"2023-12-01T03:10:37.492851+0000","last_peered":"2023-12-01T03:10:37.492851+0000","last_clean":"2023-12-01T03:10:37.492851+0000","last_became_active":"2023-12-01T03:10:24.691168+0000","last_became_peered":"2023-12-01T03:10:24.691168+0000","last_unstale":"2023-12-01T03:10:37.492851+0000","last_undegraded":"2023-12-01T03:10:37.492851+0000","last_fullsized":"2023-12-01T03:10:37.492851+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492811+0000","last_change":"2023-12-01T03:10:25.178648+0000","last_active":"2023-12-01T03:10:37.492811+0000","last_peered":"2023-12-01T03:10:37.492811+0000","last_clean":"2023-12-01T03:10:37.492811+0000","last_became_active":"2023-12-01T03:10:25.178352+0000","last_became_peered":"2023-12-01T03:10:25.178352+0000","last_unstale":"2023-12-01T03:10:37.492811+0000","last_undegraded":"2023-12-01T03:10:37.492811+0000","last_fullsized":"2023-12-01T03:10:37.492811+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519785+0000","last_change":"2023-12-01T03:10:38.515822+0000","last_active":"2023-12-01T03:10:38.519785+0000","last_peered":"2023-12-01T03:10:38.519785+0000","last_clean":"2023-12-01T03:10:38.519785+0000","last_became_active":"2023-12-01T03:10:38.515536+0000","last_became_peered":"2023-12-01T03:10:38.515536+0000","last_unstale":"2023-12-01T03:10:38.519785+0000","last_undegraded":"2023-12-01T03:10:38.519785+0000","last_fullsized":"2023-12-01T03:10:38.519785+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:10:36.574945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.510307+0000","last_change":"2023-12-01T03:10:38.510307+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.510307+0000","last_undegraded":"2023-12-01T03:10:38.510307+0000","last_fullsized":"2023-12-01T03:10:38.510307+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472906+0000","last_change":"2023-12-01T03:10:24.687709+0000","last_active":"2023-12-01T03:10:37.472906+0000","last_peered":"2023-12-01T03:10:37.472906+0000","last_clean":"2023-12-01T03:10:37.472906+0000","last_became_active":"2023-12-01T03:10:24.687452+0000","last_became_peered":"2023-12-01T03:10:24.687452+0000","last_unstale":"2023-12-01T03:10:37.472906+0000","last_undegraded":"2023-12-01T03:10:37.472906+0000","last_fullsized":"2023-12-01T03:10:37.472906+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:39.126749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489332+0000","last_change":"2023-12-01T03:10:25.176941+0000","last_active":"2023-12-01T03:10:37.489332+0000","last_peered":"2023-12-01T03:10:37.489332+0000","last_clean":"2023-12-01T03:10:37.489332+0000","last_became_active":"2023-12-01T03:10:25.175692+0000","last_became_peered":"2023-12-01T03:10:25.175692+0000","last_unstale":"2023-12-01T03:10:37.489332+0000","last_undegraded":"2023-12-01T03:10:37.489332+0000","last_fullsized":"2023-12-01T03:10:37.489332+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.513115+0000","last_change":"2023-12-01T03:10:38.509844+0000","last_active":"2023-12-01T03:10:38.513115+0000","last_peered":"2023-12-01T03:10:38.513115+0000","last_clean":"2023-12-01T03:10:38.513115+0000","last_became_active":"2023-12-01T03:10:38.509319+0000","last_became_peered":"2023-12-01T03:10:38.509319+0000","last_unstale":"2023-12-01T03:10:38.513115+0000","last_undegraded":"2023-12-01T03:10:38.513115+0000","last_fullsized":"2023-12-01T03:10:38.513115+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:12.432439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012565+0000","last_change":"2023-12-01T03:10:39.010700+0000","last_active":"2023-12-01T03:10:39.012565+0000","last_peered":"2023-12-01T03:10:39.012565+0000","last_clean":"2023-12-01T03:10:39.012565+0000","last_became_active":"2023-12-01T03:10:39.010389+0000","last_became_peered":"2023-12-01T03:10:39.010389+0000","last_unstale":"2023-12-01T03:10:39.012565+0000","last_undegraded":"2023-12-01T03:10:39.012565+0000","last_fullsized":"2023-12-01T03:10:39.012565+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:53.973502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":13,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483145+0000","last_change":"2023-12-01T03:10:24.686553+0000","last_active":"2023-12-01T03:10:37.483145+0000","last_peered":"2023-12-01T03:10:37.483145+0000","last_clean":"2023-12-01T03:10:37.483145+0000","last_became_active":"2023-12-01T03:10:24.686136+0000","last_became_peered":"2023-12-01T03:10:24.686136+0000","last_unstale":"2023-12-01T03:10:37.483145+0000","last_undegraded":"2023-12-01T03:10:37.483145+0000","last_fullsized":"2023-12-01T03:10:37.483145+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:18.663097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472405+0000","last_change":"2023-12-01T03:10:25.182793+0000","last_active":"2023-12-01T03:10:37.472405+0000","last_peered":"2023-12-01T03:10:37.472405+0000","last_clean":"2023-12-01T03:10:37.472405+0000","last_became_active":"2023-12-01T03:10:25.182524+0000","last_became_peered":"2023-12-01T03:10:25.182524+0000","last_unstale":"2023-12-01T03:10:37.472405+0000","last_undegraded":"2023-12-01T03:10:37.472405+0000","last_fullsized":"2023-12-01T03:10:37.472405+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:25.370249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.525096+0000","last_change":"2023-12-01T03:10:38.523078+0000","last_active":"2023-12-01T03:10:38.525096+0000","last_peered":"2023-12-01T03:10:38.525096+0000","last_clean":"2023-12-01T03:10:38.525096+0000","last_became_active":"2023-12-01T03:10:38.522603+0000","last_became_peered":"2023-12-01T03:10:38.522603+0000","last_unstale":"2023-12-01T03:10:38.525096+0000","last_undegraded":"2023-12-01T03:10:38.525096+0000","last_fullsized":"2023-12-01T03:10:38.525096+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:12.432439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.506778+0000","last_change":"2023-12-01T03:10:38.506778+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.506778+0000","last_undegraded":"2023-12-01T03:10:38.506778+0000","last_fullsized":"2023-12-01T03:10:38.506778+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1a","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480595+0000","last_change":"2023-12-01T03:10:24.695315+0000","last_active":"2023-12-01T03:10:37.480595+0000","last_peered":"2023-12-01T03:10:37.480595+0000","last_clean":"2023-12-01T03:10:37.480595+0000","last_became_active":"2023-12-01T03:10:24.694274+0000","last_became_peered":"2023-12-01T03:10:24.694274+0000","last_unstale":"2023-12-01T03:10:37.480595+0000","last_undegraded":"2023-12-01T03:10:37.480595+0000","last_fullsized":"2023-12-01T03:10:37.480595+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:19.256208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479974+0000","last_change":"2023-12-01T03:10:25.188350+0000","last_active":"2023-12-01T03:10:37.479974+0000","last_peered":"2023-12-01T03:10:37.479974+0000","last_clean":"2023-12-01T03:10:37.479974+0000","last_became_active":"2023-12-01T03:10:25.187212+0000","last_became_peered":"2023-12-01T03:10:25.187212+0000","last_unstale":"2023-12-01T03:10:37.479974+0000","last_undegraded":"2023-12-01T03:10:37.479974+0000","last_fullsized":"2023-12-01T03:10:37.479974+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.981897+0000","last_change":"2023-12-01T03:10:38.521794+0000","last_active":"2023-12-01T03:10:38.981897+0000","last_peered":"2023-12-01T03:10:38.981897+0000","last_clean":"2023-12-01T03:10:38.981897+0000","last_became_active":"2023-12-01T03:10:38.521546+0000","last_became_peered":"2023-12-01T03:10:38.521546+0000","last_unstale":"2023-12-01T03:10:38.981897+0000","last_undegraded":"2023-12-01T03:10:38.981897+0000","last_fullsized":"2023-12-01T03:10:38.981897+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:17.386276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.510675+0000","last_change":"2023-12-01T03:10:38.510675+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.510675+0000","last_undegraded":"2023-12-01T03:10:38.510675+0000","last_fullsized":"2023-12-01T03:10:38.510675+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:33:51.590745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"213'1","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480220+0000","last_change":"2023-12-01T03:10:24.695067+0000","last_active":"2023-12-01T03:10:37.480220+0000","last_peered":"2023-12-01T03:10:37.480220+0000","last_clean":"2023-12-01T03:10:37.480220+0000","last_became_active":"2023-12-01T03:10:24.694560+0000","last_became_peered":"2023-12-01T03:10:24.694560+0000","last_unstale":"2023-12-01T03:10:37.480220+0000","last_undegraded":"2023-12-01T03:10:37.480220+0000","last_fullsized":"2023-12-01T03:10:37.480220+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482278+0000","last_change":"2023-12-01T03:10:25.175874+0000","last_active":"2023-12-01T03:10:37.482278+0000","last_peered":"2023-12-01T03:10:37.482278+0000","last_clean":"2023-12-01T03:10:37.482278+0000","last_became_active":"2023-12-01T03:10:25.175584+0000","last_became_peered":"2023-12-01T03:10:25.175584+0000","last_unstale":"2023-12-01T03:10:37.482278+0000","last_undegraded":"2023-12-01T03:10:37.482278+0000","last_fullsized":"2023-12-01T03:10:37.482278+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:46.157610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.519254+0000","last_change":"2023-12-01T03:10:38.515775+0000","last_active":"2023-12-01T03:10:38.519254+0000","last_peered":"2023-12-01T03:10:38.519254+0000","last_clean":"2023-12-01T03:10:38.519254+0000","last_became_active":"2023-12-01T03:10:38.515021+0000","last_became_peered":"2023-12-01T03:10:38.515021+0000","last_unstale":"2023-12-01T03:10:38.519254+0000","last_undegraded":"2023-12-01T03:10:38.519254+0000","last_fullsized":"2023-12-01T03:10:38.519254+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:35.878213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003601+0000","last_change":"2023-12-01T03:10:39.000307+0000","last_active":"2023-12-01T03:10:39.003601+0000","last_peered":"2023-12-01T03:10:39.003601+0000","last_clean":"2023-12-01T03:10:39.003601+0000","last_became_active":"2023-12-01T03:10:38.997156+0000","last_became_peered":"2023-12-01T03:10:38.997156+0000","last_unstale":"2023-12-01T03:10:39.003601+0000","last_undegraded":"2023-12-01T03:10:39.003601+0000","last_fullsized":"2023-12-01T03:10:39.003601+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:59.987290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.515498+0000","last_change":"2023-12-01T03:10:38.513581+0000","last_active":"2023-12-01T03:10:38.515498+0000","last_peered":"2023-12-01T03:10:38.515498+0000","last_clean":"2023-12-01T03:10:38.515498+0000","last_became_active":"2023-12-01T03:10:38.513306+0000","last_became_peered":"2023-12-01T03:10:38.513306+0000","last_unstale":"2023-12-01T03:10:38.515498+0000","last_undegraded":"2023-12-01T03:10:38.515498+0000","last_fullsized":"2023-12-01T03:10:38.515498+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:13.957769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987848+0000","last_change":"2023-12-01T03:10:24.686931+0000","last_active":"2023-12-01T03:10:38.987848+0000","last_peered":"2023-12-01T03:10:38.987848+0000","last_clean":"2023-12-01T03:10:38.987848+0000","last_became_active":"2023-12-01T03:10:24.686642+0000","last_became_peered":"2023-12-01T03:10:24.686642+0000","last_unstale":"2023-12-01T03:10:38.987848+0000","last_undegraded":"2023-12-01T03:10:38.987848+0000","last_fullsized":"2023-12-01T03:10:38.987848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983478+0000","last_change":"2023-12-01T03:10:25.175999+0000","last_active":"2023-12-01T03:10:38.983478+0000","last_peered":"2023-12-01T03:10:38.983478+0000","last_clean":"2023-12-01T03:10:38.983478+0000","last_became_active":"2023-12-01T03:10:25.175764+0000","last_became_peered":"2023-12-01T03:10:25.175764+0000","last_unstale":"2023-12-01T03:10:38.983478+0000","last_undegraded":"2023-12-01T03:10:38.983478+0000","last_fullsized":"2023-12-01T03:10:38.983478+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:43.098776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012583+0000","last_change":"2023-12-01T03:10:39.009798+0000","last_active":"2023-12-01T03:10:39.012583+0000","last_peered":"2023-12-01T03:10:39.012583+0000","last_clean":"2023-12-01T03:10:39.012583+0000","last_became_active":"2023-12-01T03:10:39.009564+0000","last_became_peered":"2023-12-01T03:10:39.009564+0000","last_unstale":"2023-12-01T03:10:39.012583+0000","last_undegraded":"2023-12-01T03:10:39.012583+0000","last_fullsized":"2023-12-01T03:10:39.012583+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"217'4","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.011227+0000","last_change":"2023-12-01T03:10:38.520344+0000","last_active":"2023-12-01T03:10:39.011227+0000","last_peered":"2023-12-01T03:10:39.011227+0000","last_clean":"2023-12-01T03:10:39.011227+0000","last_became_active":"2023-12-01T03:10:38.520086+0000","last_became_peered":"2023-12-01T03:10:38.520086+0000","last_unstale":"2023-12-01T03:10:39.011227+0000","last_undegraded":"2023-12-01T03:10:39.011227+0000","last_fullsized":"2023-12-01T03:10:39.011227+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:01.616973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"213'2","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480900+0000","last_change":"2023-12-01T03:10:24.700378+0000","last_active":"2023-12-01T03:10:37.480900+0000","last_peered":"2023-12-01T03:10:37.480900+0000","last_clean":"2023-12-01T03:10:37.480900+0000","last_became_active":"2023-12-01T03:10:24.700126+0000","last_became_peered":"2023-12-01T03:10:24.700126+0000","last_unstale":"2023-12-01T03:10:37.480900+0000","last_undegraded":"2023-12-01T03:10:37.480900+0000","last_fullsized":"2023-12-01T03:10:37.480900+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999828+0000","last_change":"2023-12-01T03:10:25.179131+0000","last_active":"2023-12-01T03:10:38.999828+0000","last_peered":"2023-12-01T03:10:38.999828+0000","last_clean":"2023-12-01T03:10:38.999828+0000","last_became_active":"2023-12-01T03:10:25.178708+0000","last_became_peered":"2023-12-01T03:10:25.178708+0000","last_unstale":"2023-12-01T03:10:38.999828+0000","last_undegraded":"2023-12-01T03:10:38.999828+0000","last_fullsized":"2023-12-01T03:10:38.999828+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:43.370389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513293+0000","last_change":"2023-12-01T03:10:38.513293+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513293+0000","last_undegraded":"2023-12-01T03:10:38.513293+0000","last_fullsized":"2023-12-01T03:10:38.513293+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:46:18.989087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1e","version":"213'1","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989529+0000","last_change":"2023-12-01T03:10:24.688812+0000","last_active":"2023-12-01T03:10:38.989529+0000","last_peered":"2023-12-01T03:10:38.989529+0000","last_clean":"2023-12-01T03:10:38.989529+0000","last_became_active":"2023-12-01T03:10:24.687577+0000","last_became_peered":"2023-12-01T03:10:24.687577+0000","last_unstale":"2023-12-01T03:10:38.989529+0000","last_undegraded":"2023-12-01T03:10:38.989529+0000","last_fullsized":"2023-12-01T03:10:38.989529+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492668+0000","last_change":"2023-12-01T03:10:25.180335+0000","last_active":"2023-12-01T03:10:37.492668+0000","last_peered":"2023-12-01T03:10:37.492668+0000","last_clean":"2023-12-01T03:10:37.492668+0000","last_became_active":"2023-12-01T03:10:25.180020+0000","last_became_peered":"2023-12-01T03:10:25.180020+0000","last_unstale":"2023-12-01T03:10:37.492668+0000","last_undegraded":"2023-12-01T03:10:37.492668+0000","last_fullsized":"2023-12-01T03:10:37.492668+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"217'2","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.006339+0000","last_change":"2023-12-01T03:10:38.519061+0000","last_active":"2023-12-01T03:10:39.006339+0000","last_peered":"2023-12-01T03:10:39.006339+0000","last_clean":"2023-12-01T03:10:39.006339+0000","last_became_active":"2023-12-01T03:10:38.518091+0000","last_became_peered":"2023-12-01T03:10:38.518091+0000","last_unstale":"2023-12-01T03:10:39.006339+0000","last_undegraded":"2023-12-01T03:10:39.006339+0000","last_fullsized":"2023-12-01T03:10:39.006339+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:42.963999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512195+0000","last_change":"2023-12-01T03:10:38.512195+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512195+0000","last_undegraded":"2023-12-01T03:10:38.512195+0000","last_fullsized":"2023-12-01T03:10:38.512195+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"213'4","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.489298+0000","last_change":"2023-12-01T03:10:24.697099+0000","last_active":"2023-12-01T03:10:37.489298+0000","last_peered":"2023-12-01T03:10:37.489298+0000","last_clean":"2023-12-01T03:10:37.489298+0000","last_became_active":"2023-12-01T03:10:24.696783+0000","last_became_peered":"2023-12-01T03:10:24.696783+0000","last_unstale":"2023-12-01T03:10:37.489298+0000","last_undegraded":"2023-12-01T03:10:37.489298+0000","last_fullsized":"2023-12-01T03:10:37.489298+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:07.101180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479580+0000","last_change":"2023-12-01T03:10:25.191865+0000","last_active":"2023-12-01T03:10:37.479580+0000","last_peered":"2023-12-01T03:10:37.479580+0000","last_clean":"2023-12-01T03:10:37.479580+0000","last_became_active":"2023-12-01T03:10:25.189277+0000","last_became_peered":"2023-12-01T03:10:25.189277+0000","last_unstale":"2023-12-01T03:10:37.479580+0000","last_undegraded":"2023-12-01T03:10:37.479580+0000","last_fullsized":"2023-12-01T03:10:37.479580+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.521580+0000","last_change":"2023-12-01T03:10:38.517851+0000","last_active":"2023-12-01T03:10:38.521580+0000","last_peered":"2023-12-01T03:10:38.521580+0000","last_clean":"2023-12-01T03:10:38.521580+0000","last_became_active":"2023-12-01T03:10:38.517507+0000","last_became_peered":"2023-12-01T03:10:38.517507+0000","last_unstale":"2023-12-01T03:10:38.521580+0000","last_undegraded":"2023-12-01T03:10:38.521580+0000","last_fullsized":"2023-12-01T03:10:38.521580+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:22.592854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008442+0000","last_change":"2023-12-01T03:10:39.005982+0000","last_active":"2023-12-01T03:10:39.008442+0000","last_peered":"2023-12-01T03:10:39.008442+0000","last_clean":"2023-12-01T03:10:39.008442+0000","last_became_active":"2023-12-01T03:10:39.005729+0000","last_became_peered":"2023-12-01T03:10:39.005729+0000","last_unstale":"2023-12-01T03:10:39.008442+0000","last_undegraded":"2023-12-01T03:10:39.008442+0000","last_fullsized":"2023-12-01T03:10:39.008442+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:36.187885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478313+0000","last_change":"2023-12-01T03:10:25.185385+0000","last_active":"2023-12-01T03:10:37.478313+0000","last_peered":"2023-12-01T03:10:37.478313+0000","last_clean":"2023-12-01T03:10:37.478313+0000","last_became_active":"2023-12-01T03:10:25.185098+0000","last_became_peered":"2023-12-01T03:10:25.185098+0000","last_unstale":"2023-12-01T03:10:37.478313+0000","last_undegraded":"2023-12-01T03:10:37.478313+0000","last_fullsized":"2023-12-01T03:10:37.478313+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500232+0000","last_change":"2023-12-01T03:10:38.500232+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500232+0000","last_undegraded":"2023-12-01T03:10:38.500232+0000","last_fullsized":"2023-12-01T03:10:38.500232+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478702+0000","last_change":"2023-12-01T03:10:25.186126+0000","last_active":"2023-12-01T03:10:37.478702+0000","last_peered":"2023-12-01T03:10:37.478702+0000","last_clean":"2023-12-01T03:10:37.478702+0000","last_became_active":"2023-12-01T03:10:25.185741+0000","last_became_peered":"2023-12-01T03:10:25.185741+0000","last_unstale":"2023-12-01T03:10:37.478702+0000","last_undegraded":"2023-12-01T03:10:37.478702+0000","last_fullsized":"2023-12-01T03:10:37.478702+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.501528+0000","last_change":"2023-12-01T03:10:38.501528+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.501528+0000","last_undegraded":"2023-12-01T03:10:38.501528+0000","last_fullsized":"2023-12-01T03:10:38.501528+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985671+0000","last_change":"2023-12-01T03:10:25.175090+0000","last_active":"2023-12-01T03:10:38.985671+0000","last_peered":"2023-12-01T03:10:38.985671+0000","last_clean":"2023-12-01T03:10:38.985671+0000","last_became_active":"2023-12-01T03:10:25.174835+0000","last_became_peered":"2023-12-01T03:10:25.174835+0000","last_unstale":"2023-12-01T03:10:38.985671+0000","last_undegraded":"2023-12-01T03:10:38.985671+0000","last_fullsized":"2023-12-01T03:10:38.985671+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.505382+0000","last_change":"2023-12-01T03:10:38.505382+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.505382+0000","last_undegraded":"2023-12-01T03:10:38.505382+0000","last_fullsized":"2023-12-01T03:10:38.505382+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:52.701542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987378+0000","last_change":"2023-12-01T03:10:25.178098+0000","last_active":"2023-12-01T03:10:38.987378+0000","last_peered":"2023-12-01T03:10:38.987378+0000","last_clean":"2023-12-01T03:10:38.987378+0000","last_became_active":"2023-12-01T03:10:25.177031+0000","last_became_peered":"2023-12-01T03:10:25.177031+0000","last_unstale":"2023-12-01T03:10:38.987378+0000","last_undegraded":"2023-12-01T03:10:38.987378+0000","last_fullsized":"2023-12-01T03:10:38.987378+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.496510+0000","last_change":"2023-12-01T03:10:38.496510+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.496510+0000","last_undegraded":"2023-12-01T03:10:38.496510+0000","last_fullsized":"2023-12-01T03:10:38.496510+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491934+0000","last_change":"2023-12-01T03:10:25.176284+0000","last_active":"2023-12-01T03:10:37.491934+0000","last_peered":"2023-12-01T03:10:37.491934+0000","last_clean":"2023-12-01T03:10:37.491934+0000","last_became_active":"2023-12-01T03:10:25.175494+0000","last_became_peered":"2023-12-01T03:10:25.175494+0000","last_unstale":"2023-12-01T03:10:37.491934+0000","last_undegraded":"2023-12-01T03:10:37.491934+0000","last_fullsized":"2023-12-01T03:10:37.491934+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008362+0000","last_change":"2023-12-01T03:10:39.006064+0000","last_active":"2023-12-01T03:10:39.008362+0000","last_peered":"2023-12-01T03:10:39.008362+0000","last_clean":"2023-12-01T03:10:39.008362+0000","last_became_active":"2023-12-01T03:10:39.005720+0000","last_became_peered":"2023-12-01T03:10:39.005720+0000","last_unstale":"2023-12-01T03:10:39.008362+0000","last_undegraded":"2023-12-01T03:10:39.008362+0000","last_fullsized":"2023-12-01T03:10:39.008362+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:02.628798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488821+0000","last_change":"2023-12-01T03:10:25.178072+0000","last_active":"2023-12-01T03:10:37.488821+0000","last_peered":"2023-12-01T03:10:37.488821+0000","last_clean":"2023-12-01T03:10:37.488821+0000","last_became_active":"2023-12-01T03:10:25.177778+0000","last_became_peered":"2023-12-01T03:10:25.177778+0000","last_unstale":"2023-12-01T03:10:37.488821+0000","last_undegraded":"2023-12-01T03:10:37.488821+0000","last_fullsized":"2023-12-01T03:10:37.488821+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:48.799119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.496580+0000","last_change":"2023-12-01T03:10:38.496580+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.496580+0000","last_undegraded":"2023-12-01T03:10:38.496580+0000","last_fullsized":"2023-12-01T03:10:38.496580+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989375+0000","last_change":"2023-12-01T03:10:25.181469+0000","last_active":"2023-12-01T03:10:38.989375+0000","last_peered":"2023-12-01T03:10:38.989375+0000","last_clean":"2023-12-01T03:10:38.989375+0000","last_became_active":"2023-12-01T03:10:25.180646+0000","last_became_peered":"2023-12-01T03:10:25.180646+0000","last_unstale":"2023-12-01T03:10:38.989375+0000","last_undegraded":"2023-12-01T03:10:38.989375+0000","last_fullsized":"2023-12-01T03:10:38.989375+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:53.001661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008512+0000","last_change":"2023-12-01T03:10:39.006804+0000","last_active":"2023-12-01T03:10:39.008512+0000","last_peered":"2023-12-01T03:10:39.008512+0000","last_clean":"2023-12-01T03:10:39.008512+0000","last_became_active":"2023-12-01T03:10:39.006552+0000","last_became_peered":"2023-12-01T03:10:39.006552+0000","last_unstale":"2023-12-01T03:10:39.008512+0000","last_undegraded":"2023-12-01T03:10:39.008512+0000","last_fullsized":"2023-12-01T03:10:39.008512+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.992143+0000","last_change":"2023-12-01T03:10:25.179679+0000","last_active":"2023-12-01T03:10:38.992143+0000","last_peered":"2023-12-01T03:10:38.992143+0000","last_clean":"2023-12-01T03:10:38.992143+0000","last_became_active":"2023-12-01T03:10:25.179442+0000","last_became_peered":"2023-12-01T03:10:25.179442+0000","last_unstale":"2023-12-01T03:10:38.992143+0000","last_undegraded":"2023-12-01T03:10:38.992143+0000","last_fullsized":"2023-12-01T03:10:38.992143+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:57:25.728865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500994+0000","last_change":"2023-12-01T03:10:38.500994+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500994+0000","last_undegraded":"2023-12-01T03:10:38.500994+0000","last_fullsized":"2023-12-01T03:10:38.500994+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.478175+0000","last_change":"2023-12-01T03:10:25.175356+0000","last_active":"2023-12-01T03:10:37.478175+0000","last_peered":"2023-12-01T03:10:37.478175+0000","last_clean":"2023-12-01T03:10:37.478175+0000","last_became_active":"2023-12-01T03:10:25.175086+0000","last_became_peered":"2023-12-01T03:10:25.175086+0000","last_unstale":"2023-12-01T03:10:37.478175+0000","last_undegraded":"2023-12-01T03:10:37.478175+0000","last_fullsized":"2023-12-01T03:10:37.478175+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.000224+0000","last_change":"2023-12-01T03:10:38.998643+0000","last_active":"2023-12-01T03:10:39.000224+0000","last_peered":"2023-12-01T03:10:39.000224+0000","last_clean":"2023-12-01T03:10:39.000224+0000","last_became_active":"2023-12-01T03:10:38.998393+0000","last_became_peered":"2023-12-01T03:10:38.998393+0000","last_unstale":"2023-12-01T03:10:39.000224+0000","last_undegraded":"2023-12-01T03:10:39.000224+0000","last_fullsized":"2023-12-01T03:10:39.000224+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:38.199716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.491490+0000","last_change":"2023-12-01T03:10:25.181678+0000","last_active":"2023-12-01T03:10:37.491490+0000","last_peered":"2023-12-01T03:10:37.491490+0000","last_clean":"2023-12-01T03:10:37.491490+0000","last_became_active":"2023-12-01T03:10:25.181430+0000","last_became_peered":"2023-12-01T03:10:25.181430+0000","last_unstale":"2023-12-01T03:10:37.491490+0000","last_undegraded":"2023-12-01T03:10:37.491490+0000","last_fullsized":"2023-12-01T03:10:37.491490+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.506833+0000","last_change":"2023-12-01T03:10:38.506833+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.506833+0000","last_undegraded":"2023-12-01T03:10:38.506833+0000","last_fullsized":"2023-12-01T03:10:38.506833+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:57:07.895399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.480143+0000","last_change":"2023-12-01T03:10:25.188998+0000","last_active":"2023-12-01T03:10:37.480143+0000","last_peered":"2023-12-01T03:10:37.480143+0000","last_clean":"2023-12-01T03:10:37.480143+0000","last_became_active":"2023-12-01T03:10:25.188657+0000","last_became_peered":"2023-12-01T03:10:25.188657+0000","last_unstale":"2023-12-01T03:10:37.480143+0000","last_undegraded":"2023-12-01T03:10:37.480143+0000","last_fullsized":"2023-12-01T03:10:37.480143+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:23:58.752943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003091+0000","last_change":"2023-12-01T03:10:38.997151+0000","last_active":"2023-12-01T03:10:39.003091+0000","last_peered":"2023-12-01T03:10:39.003091+0000","last_clean":"2023-12-01T03:10:39.003091+0000","last_became_active":"2023-12-01T03:10:38.989885+0000","last_became_peered":"2023-12-01T03:10:38.989885+0000","last_unstale":"2023-12-01T03:10:39.003091+0000","last_undegraded":"2023-12-01T03:10:39.003091+0000","last_fullsized":"2023-12-01T03:10:39.003091+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:25:51.393448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485796+0000","last_change":"2023-12-01T03:10:25.174568+0000","last_active":"2023-12-01T03:10:37.485796+0000","last_peered":"2023-12-01T03:10:37.485796+0000","last_clean":"2023-12-01T03:10:37.485796+0000","last_became_active":"2023-12-01T03:10:25.174229+0000","last_became_peered":"2023-12-01T03:10:25.174229+0000","last_unstale":"2023-12-01T03:10:37.485796+0000","last_undegraded":"2023-12-01T03:10:37.485796+0000","last_fullsized":"2023-12-01T03:10:37.485796+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:13.528118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512138+0000","last_change":"2023-12-01T03:10:38.512138+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512138+0000","last_undegraded":"2023-12-01T03:10:38.512138+0000","last_fullsized":"2023-12-01T03:10:38.512138+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.481725+0000","last_change":"2023-12-01T03:10:25.189264+0000","last_active":"2023-12-01T03:10:37.481725+0000","last_peered":"2023-12-01T03:10:37.481725+0000","last_clean":"2023-12-01T03:10:37.481725+0000","last_became_active":"2023-12-01T03:10:25.189007+0000","last_became_peered":"2023-12-01T03:10:25.189007+0000","last_unstale":"2023-12-01T03:10:37.481725+0000","last_undegraded":"2023-12-01T03:10:37.481725+0000","last_fullsized":"2023-12-01T03:10:37.481725+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:14.783134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.511167+0000","last_change":"2023-12-01T03:10:38.511167+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.511167+0000","last_undegraded":"2023-12-01T03:10:38.511167+0000","last_fullsized":"2023-12-01T03:10:38.511167+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:38:07.828693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488698+0000","last_change":"2023-12-01T03:10:25.180992+0000","last_active":"2023-12-01T03:10:37.488698+0000","last_peered":"2023-12-01T03:10:37.488698+0000","last_clean":"2023-12-01T03:10:37.488698+0000","last_became_active":"2023-12-01T03:10:25.180712+0000","last_became_peered":"2023-12-01T03:10:25.180712+0000","last_unstale":"2023-12-01T03:10:37.488698+0000","last_undegraded":"2023-12-01T03:10:37.488698+0000","last_fullsized":"2023-12-01T03:10:37.488698+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:43.827830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507312+0000","last_change":"2023-12-01T03:10:38.507312+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507312+0000","last_undegraded":"2023-12-01T03:10:38.507312+0000","last_fullsized":"2023-12-01T03:10:38.507312+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:58:42.639310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.485503+0000","last_change":"2023-12-01T03:10:25.182676+0000","last_active":"2023-12-01T03:10:37.485503+0000","last_peered":"2023-12-01T03:10:37.485503+0000","last_clean":"2023-12-01T03:10:37.485503+0000","last_became_active":"2023-12-01T03:10:25.182429+0000","last_became_peered":"2023-12-01T03:10:25.182429+0000","last_unstale":"2023-12-01T03:10:37.485503+0000","last_undegraded":"2023-12-01T03:10:37.485503+0000","last_fullsized":"2023-12-01T03:10:37.485503+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508726+0000","last_change":"2023-12-01T03:10:38.508726+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508726+0000","last_undegraded":"2023-12-01T03:10:38.508726+0000","last_fullsized":"2023-12-01T03:10:38.508726+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:54.829130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483923+0000","last_change":"2023-12-01T03:10:25.173100+0000","last_active":"2023-12-01T03:10:37.483923+0000","last_peered":"2023-12-01T03:10:37.483923+0000","last_clean":"2023-12-01T03:10:37.483923+0000","last_became_active":"2023-12-01T03:10:25.172808+0000","last_became_peered":"2023-12-01T03:10:25.172808+0000","last_unstale":"2023-12-01T03:10:37.483923+0000","last_undegraded":"2023-12-01T03:10:37.483923+0000","last_fullsized":"2023-12-01T03:10:37.483923+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507042+0000","last_change":"2023-12-01T03:10:38.507042+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507042+0000","last_undegraded":"2023-12-01T03:10:38.507042+0000","last_fullsized":"2023-12-01T03:10:38.507042+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:48.887816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483463+0000","last_change":"2023-12-01T03:10:25.189550+0000","last_active":"2023-12-01T03:10:37.483463+0000","last_peered":"2023-12-01T03:10:37.483463+0000","last_clean":"2023-12-01T03:10:37.483463+0000","last_became_active":"2023-12-01T03:10:25.189001+0000","last_became_peered":"2023-12-01T03:10:25.189001+0000","last_unstale":"2023-12-01T03:10:37.483463+0000","last_undegraded":"2023-12-01T03:10:37.483463+0000","last_fullsized":"2023-12-01T03:10:37.483463+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003543+0000","last_change":"2023-12-01T03:10:38.997977+0000","last_active":"2023-12-01T03:10:39.003543+0000","last_peered":"2023-12-01T03:10:39.003543+0000","last_clean":"2023-12-01T03:10:39.003543+0000","last_became_active":"2023-12-01T03:10:38.997683+0000","last_became_peered":"2023-12-01T03:10:38.997683+0000","last_unstale":"2023-12-01T03:10:39.003543+0000","last_undegraded":"2023-12-01T03:10:39.003543+0000","last_fullsized":"2023-12-01T03:10:39.003543+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:27:00.020245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472732+0000","last_change":"2023-12-01T03:10:25.170590+0000","last_active":"2023-12-01T03:10:37.472732+0000","last_peered":"2023-12-01T03:10:37.472732+0000","last_clean":"2023-12-01T03:10:37.472732+0000","last_became_active":"2023-12-01T03:10:25.170334+0000","last_became_peered":"2023-12-01T03:10:25.170334+0000","last_unstale":"2023-12-01T03:10:37.472732+0000","last_undegraded":"2023-12-01T03:10:37.472732+0000","last_fullsized":"2023-12-01T03:10:37.472732+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:35.058136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513592+0000","last_change":"2023-12-01T03:10:38.513592+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513592+0000","last_undegraded":"2023-12-01T03:10:38.513592+0000","last_fullsized":"2023-12-01T03:10:38.513592+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986339+0000","last_change":"2023-12-01T03:10:25.173081+0000","last_active":"2023-12-01T03:10:38.986339+0000","last_peered":"2023-12-01T03:10:38.986339+0000","last_clean":"2023-12-01T03:10:38.986339+0000","last_became_active":"2023-12-01T03:10:25.172845+0000","last_became_peered":"2023-12-01T03:10:25.172845+0000","last_unstale":"2023-12-01T03:10:38.986339+0000","last_undegraded":"2023-12-01T03:10:38.986339+0000","last_fullsized":"2023-12-01T03:10:38.986339+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:18.173873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.512530+0000","last_change":"2023-12-01T03:10:38.512530+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.512530+0000","last_undegraded":"2023-12-01T03:10:38.512530+0000","last_fullsized":"2023-12-01T03:10:38.512530+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:24:41.123796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472369+0000","last_change":"2023-12-01T03:10:25.178225+0000","last_active":"2023-12-01T03:10:37.472369+0000","last_peered":"2023-12-01T03:10:37.472369+0000","last_clean":"2023-12-01T03:10:37.472369+0000","last_became_active":"2023-12-01T03:10:25.177541+0000","last_became_peered":"2023-12-01T03:10:25.177541+0000","last_unstale":"2023-12-01T03:10:37.472369+0000","last_undegraded":"2023-12-01T03:10:37.472369+0000","last_fullsized":"2023-12-01T03:10:37.472369+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:18.470068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.516591+0000","last_change":"2023-12-01T03:10:38.516591+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.516591+0000","last_undegraded":"2023-12-01T03:10:38.516591+0000","last_fullsized":"2023-12-01T03:10:38.516591+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987497+0000","last_change":"2023-12-01T03:10:25.172316+0000","last_active":"2023-12-01T03:10:38.987497+0000","last_peered":"2023-12-01T03:10:38.987497+0000","last_clean":"2023-12-01T03:10:38.987497+0000","last_became_active":"2023-12-01T03:10:25.169042+0000","last_became_peered":"2023-12-01T03:10:25.169042+0000","last_unstale":"2023-12-01T03:10:38.987497+0000","last_undegraded":"2023-12-01T03:10:38.987497+0000","last_fullsized":"2023-12-01T03:10:38.987497+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.507019+0000","last_change":"2023-12-01T03:10:38.507019+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.507019+0000","last_undegraded":"2023-12-01T03:10:38.507019+0000","last_fullsized":"2023-12-01T03:10:38.507019+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.473026+0000","last_change":"2023-12-01T03:10:25.181052+0000","last_active":"2023-12-01T03:10:37.473026+0000","last_peered":"2023-12-01T03:10:37.473026+0000","last_clean":"2023-12-01T03:10:37.473026+0000","last_became_active":"2023-12-01T03:10:25.180810+0000","last_became_peered":"2023-12-01T03:10:25.180810+0000","last_unstale":"2023-12-01T03:10:37.473026+0000","last_undegraded":"2023-12-01T03:10:37.473026+0000","last_fullsized":"2023-12-01T03:10:37.473026+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:35.663128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.013053+0000","last_change":"2023-12-01T03:10:39.011983+0000","last_active":"2023-12-01T03:10:39.013053+0000","last_peered":"2023-12-01T03:10:39.013053+0000","last_clean":"2023-12-01T03:10:39.013053+0000","last_became_active":"2023-12-01T03:10:39.011721+0000","last_became_peered":"2023-12-01T03:10:39.011721+0000","last_unstale":"2023-12-01T03:10:39.013053+0000","last_undegraded":"2023-12-01T03:10:39.013053+0000","last_fullsized":"2023-12-01T03:10:39.013053+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.492510+0000","last_change":"2023-12-01T03:10:25.177089+0000","last_active":"2023-12-01T03:10:37.492510+0000","last_peered":"2023-12-01T03:10:37.492510+0000","last_clean":"2023-12-01T03:10:37.492510+0000","last_became_active":"2023-12-01T03:10:25.176742+0000","last_became_peered":"2023-12-01T03:10:25.176742+0000","last_unstale":"2023-12-01T03:10:37.492510+0000","last_undegraded":"2023-12-01T03:10:37.492510+0000","last_fullsized":"2023-12-01T03:10:37.492510+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513196+0000","last_change":"2023-12-01T03:10:38.513196+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513196+0000","last_undegraded":"2023-12-01T03:10:38.513196+0000","last_fullsized":"2023-12-01T03:10:38.513196+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:19.824447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.992054+0000","last_change":"2023-12-01T03:10:25.172560+0000","last_active":"2023-12-01T03:10:38.992054+0000","last_peered":"2023-12-01T03:10:38.992054+0000","last_clean":"2023-12-01T03:10:38.992054+0000","last_became_active":"2023-12-01T03:10:25.172005+0000","last_became_peered":"2023-12-01T03:10:25.172005+0000","last_unstale":"2023-12-01T03:10:38.992054+0000","last_undegraded":"2023-12-01T03:10:38.992054+0000","last_fullsized":"2023-12-01T03:10:38.992054+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:44.541671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.515497+0000","last_change":"2023-12-01T03:10:38.515497+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.515497+0000","last_undegraded":"2023-12-01T03:10:38.515497+0000","last_fullsized":"2023-12-01T03:10:38.515497+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:10:20.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.477932+0000","last_change":"2023-12-01T03:10:25.185623+0000","last_active":"2023-12-01T03:10:37.477932+0000","last_peered":"2023-12-01T03:10:37.477932+0000","last_clean":"2023-12-01T03:10:37.477932+0000","last_became_active":"2023-12-01T03:10:25.185311+0000","last_became_peered":"2023-12-01T03:10:25.185311+0000","last_unstale":"2023-12-01T03:10:37.477932+0000","last_undegraded":"2023-12-01T03:10:37.477932+0000","last_fullsized":"2023-12-01T03:10:37.477932+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.508313+0000","last_change":"2023-12-01T03:10:38.508313+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.508313+0000","last_undegraded":"2023-12-01T03:10:38.508313+0000","last_fullsized":"2023-12-01T03:10:38.508313+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:38.801763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.483215+0000","last_change":"2023-12-01T03:10:25.186300+0000","last_active":"2023-12-01T03:10:37.483215+0000","last_peered":"2023-12-01T03:10:37.483215+0000","last_clean":"2023-12-01T03:10:37.483215+0000","last_became_active":"2023-12-01T03:10:25.185030+0000","last_became_peered":"2023-12-01T03:10:25.185030+0000","last_unstale":"2023-12-01T03:10:37.483215+0000","last_undegraded":"2023-12-01T03:10:37.483215+0000","last_fullsized":"2023-12-01T03:10:37.483215+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.501552+0000","last_change":"2023-12-01T03:10:38.501552+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.501552+0000","last_undegraded":"2023-12-01T03:10:38.501552+0000","last_fullsized":"2023-12-01T03:10:38.501552+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.482005+0000","last_change":"2023-12-01T03:10:25.182648+0000","last_active":"2023-12-01T03:10:37.482005+0000","last_peered":"2023-12-01T03:10:37.482005+0000","last_clean":"2023-12-01T03:10:37.482005+0000","last_became_active":"2023-12-01T03:10:25.182343+0000","last_became_peered":"2023-12-01T03:10:25.182343+0000","last_unstale":"2023-12-01T03:10:37.482005+0000","last_undegraded":"2023-12-01T03:10:37.482005+0000","last_fullsized":"2023-12-01T03:10:37.482005+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:55.408381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.514336+0000","last_change":"2023-12-01T03:10:38.514336+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.514336+0000","last_undegraded":"2023-12-01T03:10:38.514336+0000","last_fullsized":"2023-12-01T03:10:38.514336+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.479210+0000","last_change":"2023-12-01T03:10:25.191253+0000","last_active":"2023-12-01T03:10:37.479210+0000","last_peered":"2023-12-01T03:10:37.479210+0000","last_clean":"2023-12-01T03:10:37.479210+0000","last_became_active":"2023-12-01T03:10:25.188485+0000","last_became_peered":"2023-12-01T03:10:25.188485+0000","last_unstale":"2023-12-01T03:10:37.479210+0000","last_undegraded":"2023-12-01T03:10:37.479210+0000","last_fullsized":"2023-12-01T03:10:37.479210+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.500633+0000","last_change":"2023-12-01T03:10:38.500633+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.500633+0000","last_undegraded":"2023-12-01T03:10:38.500633+0000","last_fullsized":"2023-12-01T03:10:38.500633+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999753+0000","last_change":"2023-12-01T03:10:25.171460+0000","last_active":"2023-12-01T03:10:38.999753+0000","last_peered":"2023-12-01T03:10:38.999753+0000","last_clean":"2023-12-01T03:10:38.999753+0000","last_became_active":"2023-12-01T03:10:25.171148+0000","last_became_peered":"2023-12-01T03:10:25.171148+0000","last_unstale":"2023-12-01T03:10:38.999753+0000","last_undegraded":"2023-12-01T03:10:38.999753+0000","last_fullsized":"2023-12-01T03:10:38.999753+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:01:37.278740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.516041+0000","last_change":"2023-12-01T03:10:38.516041+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.516041+0000","last_undegraded":"2023-12-01T03:10:38.516041+0000","last_fullsized":"2023-12-01T03:10:38.516041+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:07:32.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.488617+0000","last_change":"2023-12-01T03:10:25.181720+0000","last_active":"2023-12-01T03:10:37.488617+0000","last_peered":"2023-12-01T03:10:37.488617+0000","last_clean":"2023-12-01T03:10:37.488617+0000","last_became_active":"2023-12-01T03:10:25.180144+0000","last_became_peered":"2023-12-01T03:10:25.180144+0000","last_unstale":"2023-12-01T03:10:37.488617+0000","last_undegraded":"2023-12-01T03:10:37.488617+0000","last_fullsized":"2023-12-01T03:10:37.488617+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:53.999589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007029+0000","last_change":"2023-12-01T03:10:39.004848+0000","last_active":"2023-12-01T03:10:39.007029+0000","last_peered":"2023-12-01T03:10:39.007029+0000","last_clean":"2023-12-01T03:10:39.007029+0000","last_became_active":"2023-12-01T03:10:39.004604+0000","last_became_peered":"2023-12-01T03:10:39.004604+0000","last_unstale":"2023-12-01T03:10:39.007029+0000","last_undegraded":"2023-12-01T03:10:39.007029+0000","last_fullsized":"2023-12-01T03:10:39.007029+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:44.253867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983363+0000","last_change":"2023-12-01T03:10:25.181088+0000","last_active":"2023-12-01T03:10:38.983363+0000","last_peered":"2023-12-01T03:10:38.983363+0000","last_clean":"2023-12-01T03:10:38.983363+0000","last_became_active":"2023-12-01T03:10:25.180741+0000","last_became_peered":"2023-12-01T03:10:25.180741+0000","last_unstale":"2023-12-01T03:10:38.983363+0000","last_undegraded":"2023-12-01T03:10:38.983363+0000","last_fullsized":"2023-12-01T03:10:38.983363+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:26.321285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":1,"reported_epoch":216,"state":"creating+peering","last_fresh":"2023-12-01T03:10:38.513800+0000","last_change":"2023-12-01T03:10:38.513800+0000","last_active":"2023-12-01T03:10:38.468841+0000","last_peered":"2023-12-01T03:10:38.468841+0000","last_clean":"2023-12-01T03:10:38.468841+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:10:38.513800+0000","last_undegraded":"2023-12-01T03:10:38.513800+0000","last_fullsized":"2023-12-01T03:10:38.513800+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-01T03:10:37.472688+0000","last_change":"2023-12-01T03:10:25.171115+0000","last_active":"2023-12-01T03:10:37.472688+0000","last_peered":"2023-12-01T03:10:37.472688+0000","last_clean":"2023-12-01T03:10:37.472688+0000","last_became_active":"2023-12-01T03:10:25.170871+0000","last_became_peered":"2023-12-01T03:10:25.170871+0000","last_unstale":"2023-12-01T03:10:37.472688+0000","last_undegraded":"2023-12-01T03:10:37.472688+0000","last_fullsized":"2023-12-01T03:10:37.472688+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:24.370036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59936,"kb_used_data":2648,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311904,"statfs":{"total":96636764160,"available":96575389696,"internally_reserved":0,"allocated":2711552,"data_stored":1314756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0780000000000001,"15min":1.0820000000000001},"min":{"1min":0.78200000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5069999999999999,"5min":1.6000000000000001,"15min":1.98},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.093,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":0.67800000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.556,"5min":1.7,"15min":1.7},"last":1.331}]},{"osd":1,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.2889999999999999},"min":{"1min":0.80000000000000004,"5min":0.745,"15min":0.73099999999999998},"max":{"1min":1.603,"5min":2,"15min":2.1419999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3149999999999999,"15min":1.3320000000000001},"min":{"1min":1.137,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.514,"5min":1.8169999999999999,"15min":1.8700000000000001},"last":1.0700000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.157,"15min":1.1719999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.476,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.26,"15min":1.254},"min":{"1min":1.0549999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6020000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.268}]},{"osd":3,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2529999999999999,"15min":1.3049999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6559999999999999,"5min":1.712,"15min":2.431},"last":1.014},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.415,"15min":1.4199999999999999},"min":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.92900000000000005},"max":{"1min":1.5920000000000001,"5min":1.879,"15min":2.198},"last":1.109}]},{"osd":4,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3140000000000001,"15min":1.3400000000000001},"min":{"1min":1.006,"5min":0.82599999999999996,"15min":0.81200000000000006},"max":{"1min":1.901,"5min":1.901,"15min":2.5899999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":1.1240000000000001,"5min":0.92800000000000005,"15min":0.79500000000000004},"max":{"1min":1.9690000000000001,"5min":2.3279999999999998,"15min":2.569},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3009999999999999,"15min":1.3149999999999999},"min":{"1min":0.93600000000000005,"5min":0.85999999999999999,"15min":0.84599999999999997},"max":{"1min":2.0099999999999998,"5min":2.016,"15min":2.7269999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5800000000000001,"15min":1.589},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":0.93200000000000005},"max":{"1min":2.0569999999999999,"5min":2.206,"15min":2.484},"last":1.4570000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4359999999999999,"15min":1.446},"min":{"1min":1.0229999999999999,"5min":0.98599999999999999,"15min":0.82599999999999996},"max":{"1min":1.921,"5min":2.2789999999999999,"15min":2.532},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":1.8819999999999999,"15min":2.762},"last":1.2370000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575980,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41872,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329968,"statfs":{"total":96636764160,"available":96593887232,"internally_reserved":0,"allocated":2695168,"data_stored":1311929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0589999999999999,"15min":1.113},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.4039999999999999,"5min":1.6639999999999999,"15min":1.9219999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1160000000000001,"15min":1.1659999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.623,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":0.97699999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.976,"5min":1.976,"15min":2.0449999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1950000000000001,"15min":1.2210000000000001},"min":{"1min":0.627,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.694,"5min":1.694,"15min":1.835},"last":1.2390000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.163,"15min":1.2090000000000001},"min":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.976},"last":1.127},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2629999999999999,"15min":1.2849999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.167}]},{"osd":3,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2849999999999999,"15min":1.319},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8100000000000001,"5min":1.927,"15min":2.0289999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.3720000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2130000000000001,"15min":1.262},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.367,"15min":1.3540000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.522}]},{"osd":5,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3360000000000001,"15min":1.4039999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.272},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4790000000000001,"15min":1.484},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.6319999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4890000000000001,"15min":1.5249999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.452},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.556,"15min":1.579},"min":{"1min":0.95499999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2999999999999998,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.794}]}]},{"osd":5,"up_from":13,"seq":55834575979,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48216,"kb_used_data":3216,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323624,"statfs":{"total":96636764160,"available":96587390976,"internally_reserved":0,"allocated":3293184,"data_stored":1902811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.125},"min":{"1min":0.82299999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.548,"5min":1.776,"15min":1.8899999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.238,"5min":1.1639999999999999,"15min":1.163},"min":{"1min":0.89800000000000002,"5min":0.69399999999999995,"15min":0.67300000000000004},"max":{"1min":1.619,"5min":1.752,"15min":1.8009999999999999},"last":0.84999999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.133,"15min":1.1539999999999999},"min":{"1min":0.94699999999999995,"5min":0.81200000000000006,"15min":0.79900000000000004},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.696},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.272,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.88,"5min":0.79500000000000004,"15min":0.60899999999999999},"max":{"1min":1.619,"5min":1.6659999999999999,"15min":1.7649999999999999},"last":1.0289999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2390000000000001,"15min":1.246},"min":{"1min":0.93700000000000006,"5min":0.84499999999999997,"15min":0.81399999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.774},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.284,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.93899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.502,"5min":1.899,"15min":1.9219999999999999},"last":0.90200000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.306,"15min":1.3200000000000001},"min":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.93100000000000005},"max":{"1min":1.7929999999999999,"5min":1.8979999999999999,"15min":1.994},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.343,"15min":1.363},"min":{"1min":1.02,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.724,"5min":1.97,"15min":2.1859999999999999},"last":1.1080000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2789999999999999,"15min":1.254},"min":{"1min":0.96199999999999997,"5min":0.78000000000000003,"15min":0.74099999999999999},"max":{"1min":1.726,"5min":1.726,"15min":2.6040000000000001},"last":1.389},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.399,"15min":1.385},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.84599999999999997},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.931},"last":1.1579999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3879999999999999,"15min":1.3759999999999999},"min":{"1min":0.97599999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.9279999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.427,"15min":1.419},"min":{"1min":1.222,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":1.756,"5min":1.9219999999999999,"15min":2.577},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5349999999999999,"15min":1.5369999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.962,"5min":2.0790000000000002,"15min":3.1709999999999998},"last":1.294},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.575,"15min":1.5740000000000001},"min":{"1min":1.3720000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.831,"5min":2.101,"15min":2.8740000000000001},"last":1.343}]}]},{"osd":4,"up_from":13,"seq":55834575979,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2707456,"data_stored":1319053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.147,"15min":1.1339999999999999},"min":{"1min":0.73099999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.21,"15min":1.1990000000000001},"min":{"1min":0.82099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5840000000000001,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.006}]},{"osd":1,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.242,"15min":1.2589999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.613,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.228},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.82599999999999996},"max":{"1min":1.6080000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.5549999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.3240000000000001,"15min":1.2470000000000001},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.573,"5min":11.311,"15min":11.311},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.329,"15min":1.341},"min":{"1min":1.0680000000000001,"5min":1.024,"15min":0.94099999999999995},"max":{"1min":1.647,"5min":1.9379999999999999,"15min":1.9450000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3100000000000001,"15min":1.29},"min":{"1min":1.0529999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7749999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.365,"15min":1.3380000000000001},"min":{"1min":0.99199999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.599,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.5920000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.357,"15min":1.329},"min":{"1min":0.94799999999999995,"5min":0.91200000000000003,"15min":0.755},"max":{"1min":1.615,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.379,"15min":1.3120000000000001},"min":{"1min":0.94599999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7150000000000001,"5min":3.71,"15min":3.71},"last":1.2809999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3740000000000001,"15min":1.375},"min":{"1min":1.095,"5min":0.745,"15min":0.745},"max":{"1min":1.609,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.4850000000000001,"15min":1.48},"min":{"1min":1.335,"5min":1.0660000000000001,"15min":1.0149999999999999},"max":{"1min":1.738,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.52,"15min":1.486},"min":{"1min":1.157,"5min":0.95499999999999996,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6000000000000001,"15min":1.5800000000000001},"min":{"1min":1.3600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.806,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.7070000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575978,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51752,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320088,"statfs":{"total":96636764160,"available":96583770112,"internally_reserved":0,"allocated":2719744,"data_stored":1321894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91100000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.085},"min":{"1min":0.74199999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.571,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.163,"15min":1.171},"min":{"1min":0.64700000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.008,"15min":0.999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41099999999999998},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0900000000000001,"15min":1.103},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2230000000000001,"15min":1.252},"min":{"1min":0.80100000000000005,"5min":0.747,"15min":0.58599999999999997},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.57899999999999996}]},{"osd":4,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2989999999999999,"15min":1.2929999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.3},"min":{"1min":0.96799999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0229999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3240000000000001,"15min":1.331},"min":{"1min":0.86599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.373,"15min":1.3759999999999999},"min":{"1min":0.88400000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.78400000000000003},"max":{"1min":2.5649999999999999,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.54,"5min":1.5049999999999999,"15min":1.514},"min":{"1min":1.075,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.075}]},{"osd":7,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.516,"15min":1.5329999999999999},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.042},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.54,"15min":1.5309999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.878},"max":{"1min":2.4830000000000001,"5min":2.984,"15min":2.984},"last":0.83899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575978,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64028,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307812,"statfs":{"total":96636764160,"available":96571199488,"internally_reserved":0,"allocated":2707456,"data_stored":1319089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89300000000000002,"15min":0.89400000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1140000000000001,"5min":1.405,"15min":1.4730000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.056,"15min":1.0569999999999999},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5640000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0449999999999999,"15min":1.0589999999999999},"min":{"1min":0.66100000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.321,"5min":1.698,"15min":1.698},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.038,"5min":1.125,"15min":1.151},"min":{"1min":0.67200000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.522,"5min":1.635,"15min":1.71},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0640000000000001,"15min":1.054},"min":{"1min":0.64100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.419,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1990000000000001,"15min":1.2170000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.516,"5min":1.6879999999999999,"15min":1.79},"last":1.01}]},{"osd":4,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3480000000000001,"15min":1.3380000000000001},"min":{"1min":0.85199999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.7569999999999999,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.133},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2909999999999999,"15min":1.3009999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.714,"5min":1.714,"15min":1.835},"last":1.2390000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3919999999999999,"15min":1.391},"min":{"1min":0.88,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.8380000000000001,"15min":2.2040000000000002},"last":1.042},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.419,"15min":1.4179999999999999},"min":{"1min":1.095,"5min":1.004,"15min":1.004},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.016},"last":0.91800000000000004}]},{"osd":6,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.379,"15min":1.3819999999999999},"min":{"1min":0.89600000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.337},{"interface":"front","average":{"1min":1.46,"5min":1.5069999999999999,"15min":1.5049999999999999},"min":{"1min":1.087,"5min":0.879,"15min":0.879},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.327}]},{"osd":7,"last update":"Fri Dec 1 03:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5069999999999999,"15min":1.4910000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":2.0979999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4330000000000001,"15min":1.458},"min":{"1min":0.94399999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.012,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.0860000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575978,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48244,"kb_used_data":3244,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323596,"statfs":{"total":96636764160,"available":96587362304,"internally_reserved":0,"allocated":3321856,"data_stored":1924287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.879,"15min":0.90900000000000003},"min":{"1min":0.58199999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.099,"5min":0.99299999999999999,"15min":1.046},"min":{"1min":0.81000000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.5069999999999999,"5min":1.5660000000000001,"15min":1.681},"last":0.498}]},{"osd":2,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.56299999999999994,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.115,"15min":1.1419999999999999},"min":{"1min":0.75600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.581,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.258,"15min":1.3069999999999999},"min":{"1min":1.026,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5780000000000001,"5min":1.587,"15min":1.788},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.107,"15min":1.135},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.714,"5min":1.714,"15min":1.8049999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.21,"15min":1.2230000000000001},"min":{"1min":0.89900000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6859999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.389,"15min":1.409},"min":{"1min":1.0740000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.0269999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.85899999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.802,"5min":1.8520000000000001,"15min":2.101},"last":1.056},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.304,"15min":1.339},"min":{"1min":1.006,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.609,"5min":1.927,"15min":2.1659999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4019999999999999,"15min":1.4379999999999999},"min":{"1min":0.99399999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3799999999999999,"15min":1.4119999999999999},"min":{"1min":1.1779999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.768,"5min":2.0019999999999998,"15min":2.0209999999999999},"last":0.98499999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4870000000000001,"15min":1.5269999999999999},"min":{"1min":1.2549999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.911,"5min":2.0329999999999999,"15min":2.1190000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.524,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.091}]}]},{"osd":0,"up_from":13,"seq":55834575980,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":2676,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2740224,"data_stored":1335117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.86699999999999999,"15min":0.89400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1559999999999999,"5min":1.4379999999999999,"15min":1.994},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.04,"15min":1.056},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.427,"5min":1.583,"15min":1.6799999999999999},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0329999999999999,"15min":1.069},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.446,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.1020000000000001,"15min":1.1419999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.726,"5min":1.726,"15min":1.792},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0580000000000001,"15min":1.071},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.8839999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.655,"5min":1.655,"15min":2.6469999999999998},"last":0.78200000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3260000000000001,"15min":1.363},"min":{"1min":0.88400000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.274,"5min":1.284,"15min":1.3089999999999999},"min":{"1min":0.82099999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.677,"5min":1.74,"15min":2.3610000000000002},"last":1.0740000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.355,"15min":1.397},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5589999999999999,"5min":2.1030000000000002,"15min":2.7719999999999998},"last":1.054},{"interface":"front","average":{"1min":1.431,"5min":1.4099999999999999,"15min":1.4410000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.79600000000000004},"max":{"1min":1.9259999999999999,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":0.97499999999999998}]},{"osd":6,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.367,"15min":1.3859999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7649999999999999,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.458,"15min":1.484},"min":{"1min":1.048,"5min":0.85299999999999998,"15min":0.75800000000000001},"max":{"1min":1.736,"5min":1.9870000000000001,"15min":2.8759999999999999},"last":1.2569999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.49,"15min":1.524},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.841,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4750000000000001,"15min":1.4910000000000001},"min":{"1min":0.97299999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.0619999999999998,"5min":2.1190000000000002,"15min":2.431},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:10:48.466 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:10:48.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.693+0000 7f0b21837640 1 Processor -- start 2023-12-01T03:10:48.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.693+0000 7f0b21837640 1 -- start start 2023-12-01T03:10:48.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.694+0000 7f0b21837640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1c160130 0x7f0b1c0d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:48.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.694+0000 7f0b21837640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0b1c0d26e0 0x7f0b1c0b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:48.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.695+0000 7f0b21837640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1c0b8b60 con 0x7f0b1c0cfb20 2023-12-01T03:10:48.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.695+0000 7f0b21837640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1c160510 con 0x7f0b1c0d26e0 2023-12-01T03:10:48.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.695+0000 7f0b21837640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c0b6d20 con 0x7f0b1c160130 2023-12-01T03:10:48.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.695+0000 7f0b21036640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0b1c0cfb20 0x7f0b1c0d2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43638/0 (socket says 172.21.15.115:43638) 2023-12-01T03:10:48.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.695+0000 7f0b21036640 1 -- 172.21.15.115:0/1876144919 learned_addr learned my addr 172.21.15.115:0/1876144919 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.695+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000951460 0 0) 0x7f0b1c0b8b60 con 0x7f0b1c0cfb20 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.695+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b0c003520 con 0x7f0b1c0cfb20 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1bfff640 1 --2- 172.21.15.115:0/1876144919 >> v2:172.21.15.142:3300/0 conn(0x7f0b1c0d26e0 0x7f0b1c0b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b20835640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1c160130 0x7f0b1c0d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3630329689 0 0) 0x7f0b0c003520 con 0x7f0b1c0cfb20 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1c160130 msgr2=0x7f0b1c0d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1c160130 0x7f0b1c0d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 >> v2:172.21.15.142:3300/0 conn(0x7f0b1c0d26e0 msgr2=0x7f0b1c0b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 --2- 172.21.15.115:0/1876144919 >> v2:172.21.15.142:3300/0 conn(0x7f0b1c0d26e0 0x7f0b1c0b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c0d1640 con 0x7f0b1c0cfb20 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0b140019f0 con 0x7f0b1c0cfb20 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 >> v1:172.21.15.115:6789/0 conn(0x7f0b1c0cfb20 legacy=0x7f0b1c0d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:48.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1c160130 0x7f0b0c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0c004060 0x7f0b0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b0c004d00 0x7f0b0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c160510 con 0x7f0b0c004d00 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c0b6d20 con 0x7f0b0c004060 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c00bb60 con 0x7f0b1c160130 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.696+0000 7f0b21036640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b0c004d00 0x7f0b0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.697+0000 7f0b20835640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1c160130 0x7f0b0c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.697+0000 7f0b21036640 1 -- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1c160130 msgr2=0x7f0b0c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.697+0000 7f0b21036640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1c160130 0x7f0b0c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.697+0000 7f0b21036640 1 -- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0c004060 msgr2=0x7f0b0c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.697+0000 7f0b21036640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0c004060 0x7f0b0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.697+0000 7f0b21036640 1 -- 172.21.15.115:0/1876144919 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c003520 con 0x7f0b0c004d00 2023-12-01T03:10:48.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.697+0000 7f0b20835640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1c160130 0x7f0b0c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:48.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.698+0000 7f0b21036640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b0c004d00 0x7f0b0c005180 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400a790 tx=0x7f0b0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd075498246bf0cb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:48.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.699+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14034630 con 0x7f0b0c004d00 2023-12-01T03:10:48.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.699+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1402f070 con 0x7f0b0c004d00 2023-12-01T03:10:48.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.699+0000 7f0b1b7fe640 1 -- 172.21.15.115:0/1876144919 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14034a30 con 0x7f0b0c004d00 2023-12-01T03:10:48.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.699+0000 7f0b21837640 1 -- 172.21.15.115:0/1876144919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b0c004d00 msgr2=0x7f0b0c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:48.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.699+0000 7f0b21837640 1 --2- 172.21.15.115:0/1876144919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b0c004d00 0x7f0b0c005180 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400a790 tx=0x7f0b0c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.699+0000 7f0b21036640 1 -- 172.21.15.115:0/1876144919 reap_dead start 2023-12-01T03:10:48.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.700+0000 7f0b21837640 1 -- 172.21.15.115:0/1876144919 shutdown_connections 2023-12-01T03:10:48.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.700+0000 7f0b21837640 1 -- 172.21.15.115:0/1876144919 >> 172.21.15.115:0/1876144919 conn(0x7f0b1c06a0b0 msgr2=0x7f0b1c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:48.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.700+0000 7f0b21837640 1 -- 172.21.15.115:0/1876144919 shutdown_connections 2023-12-01T03:10:48.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.700+0000 7f0b21837640 1 -- 172.21.15.115:0/1876144919 wait complete. 2023-12-01T03:10:48.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.702+0000 7f0b21837640 1 Processor -- start 2023-12-01T03:10:48.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.702+0000 7f0b21837640 1 -- start start 2023-12-01T03:10:48.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.703+0000 7f0b21837640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b0c004d00 0x7f0b1c144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:48.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.703+0000 7f0b21837640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c0ca6c0 0x7f0b1c1f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:48.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.703+0000 7f0b21837640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b1c142e10 0x7f0b1c1432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:48.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b21837640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c145e60 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b21837640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c1455c0 con 0x7f0b1c142e10 2023-12-01T03:10:48.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b21837640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c1458a0 con 0x7f0b0c004d00 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c0ca6c0 0x7f0b1c1f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c0ca6c0 0x7f0b1c1f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50154/0 (socket says 172.21.15.115:50154) 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b1bfff640 1 -- 172.21.15.115:0/3197601934 learned_addr learned my addr 172.21.15.115:0/3197601934 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b21036640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b1c142e10 0x7f0b1c1432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b20835640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b0c004d00 0x7f0b1c144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b1bfff640 1 -- 172.21.15.115:0/3197601934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b0c004d00 msgr2=0x7f0b1c144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b1bfff640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b0c004d00 0x7f0b1c144f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b1bfff640 1 -- 172.21.15.115:0/3197601934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b1c142e10 msgr2=0x7f0b1c1432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b1bfff640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b1c142e10 0x7f0b1c1432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.704+0000 7f0b1bfff640 1 -- 172.21.15.115:0/3197601934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c145b80 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.705+0000 7f0b1bfff640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c0ca6c0 0x7f0b1c1f53e0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800d5b0 tx=0x7f0b0800da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:48.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.705+0000 7f0b197fa640 1 -- 172.21.15.115:0/3197601934 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b080125f0 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.705+0000 7f0b197fa640 1 -- 172.21.15.115:0/3197601934 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b08012770 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.705+0000 7f0b197fa640 1 -- 172.21.15.115:0/3197601934 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b08012b20 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.705+0000 7f0b21837640 1 -- 172.21.15.115:0/3197601934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1c1f5e10 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.705+0000 7f0b21837640 1 -- 172.21.15.115:0/3197601934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c1f5920 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.707+0000 7f0b21837640 1 -- 172.21.15.115:0/3197601934 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1c0673a0 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.707+0000 7f0b197fa640 1 -- 172.21.15.115:0/3197601934 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0b08012c80 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.708+0000 7f0b197fa640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0b0006a580 0x7f0b0006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:48.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.708+0000 7f0b197fa640 1 -- 172.21.15.115:0/3197601934 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f0b08089210 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.709+0000 7f0b20835640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0b0006a580 0x7f0b0006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:48.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.709+0000 7f0b20835640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0b0006a580 0x7f0b0006ca40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c870 tx=0x7f0b10008040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:48.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.713+0000 7f0b197fa640 1 -- 172.21.15.115:0/3197601934 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0b080543f0 con 0x7f0b1c0ca6c0 2023-12-01T03:10:48.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.900+0000 7f0b21837640 1 -- 172.21.15.115:0/3197601934 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b1c0d1c00 con 0x7f0b0006a580 2023-12-01T03:10:48.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.940+0000 7f0b197fa640 1 -- 172.21.15.115:0/3197601934 <== mgr.10407 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526097 (secure 0 0 0) 0x7f0b1000ce50 con 0x7f0b0006a580 2023-12-01T03:10:48.950 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:48.953 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:10:48.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 -- 172.21.15.115:0/3197601934 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0b0006a580 msgr2=0x7f0b0006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:48.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0b0006a580 0x7f0b0006ca40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c870 tx=0x7f0b10008040 comp rx=0 tx=0).stop 2023-12-01T03:10:48.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 -- 172.21.15.115:0/3197601934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c0ca6c0 msgr2=0x7f0b1c1f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:48.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c0ca6c0 0x7f0b1c1f53e0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800d5b0 tx=0x7f0b0800da70 comp rx=0 tx=0).stop 2023-12-01T03:10:48.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 -- 172.21.15.115:0/3197601934 shutdown_connections 2023-12-01T03:10:48.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0b0006a580 0x7f0b0006ca40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b1c142e10 0x7f0b1c1432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b1c0ca6c0 0x7f0b1c1f53e0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 --2- 172.21.15.115:0/3197601934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b0c004d00 0x7f0b1c144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:48.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 -- 172.21.15.115:0/3197601934 >> 172.21.15.115:0/3197601934 conn(0x7f0b1c06a0b0 msgr2=0x7f0b1c1e8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:48.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 -- 172.21.15.115:0/3197601934 shutdown_connections 2023-12-01T03:10:48.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:48.948+0000 7f0b21837640 1 -- 172.21.15.115:0/3197601934 wait complete. 2023-12-01T03:10:48.968 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-12-01T03:10:47.802368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":630,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":405,"ondisk_log_size":405,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425992,"kb_used_data":22600,"kb_used_omap":0,"kb_used_meta":403328,"kb_avail":754548728,"statfs":{"total":773094113280,"available":772657897472,"internally_reserved":0,"allocated":23142400,"data_stored":11864018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":413007872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.830871"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.021319+0000","last_change":"2023-12-01T03:10:39.020284+0000","last_active":"2023-12-01T03:10:39.021319+0000","last_peered":"2023-12-01T03:10:39.021319+0000","last_clean":"2023-12-01T03:10:39.021319+0000","last_became_active":"2023-12-01T03:10:39.019930+0000","last_became_peered":"2023-12-01T03:10:39.019930+0000","last_unstale":"2023-12-01T03:10:39.021319+0000","last_undegraded":"2023-12-01T03:10:39.021319+0000","last_fullsized":"2023-12-01T03:10:39.021319+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:14.969594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"201'326","reported_seq":1123,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983105+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:10:38.983105+0000","last_peered":"2023-12-01T03:10:38.983105+0000","last_clean":"2023-12-01T03:10:38.983105+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:10:38.983105+0000","last_undegraded":"2023-12-01T03:10:38.983105+0000","last_fullsized":"2023-12-01T03:10:38.983105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988127+0000","last_change":"2023-12-01T03:10:24.685430+0000","last_active":"2023-12-01T03:10:38.988127+0000","last_peered":"2023-12-01T03:10:38.988127+0000","last_clean":"2023-12-01T03:10:38.988127+0000","last_became_active":"2023-12-01T03:10:24.684919+0000","last_became_peered":"2023-12-01T03:10:24.684919+0000","last_unstale":"2023-12-01T03:10:38.988127+0000","last_undegraded":"2023-12-01T03:10:38.988127+0000","last_fullsized":"2023-12-01T03:10:38.988127+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985828+0000","last_change":"2023-12-01T03:10:25.189213+0000","last_active":"2023-12-01T03:10:38.985828+0000","last_peered":"2023-12-01T03:10:38.985828+0000","last_clean":"2023-12-01T03:10:38.985828+0000","last_became_active":"2023-12-01T03:10:25.188927+0000","last_became_peered":"2023-12-01T03:10:25.188927+0000","last_unstale":"2023-12-01T03:10:38.985828+0000","last_undegraded":"2023-12-01T03:10:38.985828+0000","last_fullsized":"2023-12-01T03:10:38.985828+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:21:53.633800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1b","version":"217'1","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996159+0000","last_change":"2023-12-01T03:10:38.517472+0000","last_active":"2023-12-01T03:10:38.996159+0000","last_peered":"2023-12-01T03:10:38.996159+0000","last_clean":"2023-12-01T03:10:38.996159+0000","last_became_active":"2023-12-01T03:10:38.517218+0000","last_became_peered":"2023-12-01T03:10:38.517218+0000","last_unstale":"2023-12-01T03:10:38.996159+0000","last_undegraded":"2023-12-01T03:10:38.996159+0000","last_fullsized":"2023-12-01T03:10:38.996159+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:57.653086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.013169+0000","last_change":"2023-12-01T03:10:39.012642+0000","last_active":"2023-12-01T03:10:39.013169+0000","last_peered":"2023-12-01T03:10:39.013169+0000","last_clean":"2023-12-01T03:10:39.013169+0000","last_became_active":"2023-12-01T03:10:39.011556+0000","last_became_peered":"2023-12-01T03:10:39.011556+0000","last_unstale":"2023-12-01T03:10:39.013169+0000","last_undegraded":"2023-12-01T03:10:39.013169+0000","last_fullsized":"2023-12-01T03:10:39.013169+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.984956+0000","last_change":"2023-12-01T03:10:24.690155+0000","last_active":"2023-12-01T03:10:38.984956+0000","last_peered":"2023-12-01T03:10:38.984956+0000","last_clean":"2023-12-01T03:10:38.984956+0000","last_became_active":"2023-12-01T03:10:24.689122+0000","last_became_peered":"2023-12-01T03:10:24.689122+0000","last_unstale":"2023-12-01T03:10:38.984956+0000","last_undegraded":"2023-12-01T03:10:38.984956+0000","last_fullsized":"2023-12-01T03:10:38.984956+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.000534+0000","last_change":"2023-12-01T03:10:25.191828+0000","last_active":"2023-12-01T03:10:39.000534+0000","last_peered":"2023-12-01T03:10:39.000534+0000","last_clean":"2023-12-01T03:10:39.000534+0000","last_became_active":"2023-12-01T03:10:25.191436+0000","last_became_peered":"2023-12-01T03:10:25.191436+0000","last_unstale":"2023-12-01T03:10:39.000534+0000","last_undegraded":"2023-12-01T03:10:39.000534+0000","last_fullsized":"2023-12-01T03:10:39.000534+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.004256+0000","last_change":"2023-12-01T03:10:38.524843+0000","last_active":"2023-12-01T03:10:39.004256+0000","last_peered":"2023-12-01T03:10:39.004256+0000","last_clean":"2023-12-01T03:10:39.004256+0000","last_became_active":"2023-12-01T03:10:38.524424+0000","last_became_peered":"2023-12-01T03:10:38.524424+0000","last_unstale":"2023-12-01T03:10:39.004256+0000","last_undegraded":"2023-12-01T03:10:39.004256+0000","last_fullsized":"2023-12-01T03:10:39.004256+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:32.967785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012786+0000","last_change":"2023-12-01T03:10:39.008570+0000","last_active":"2023-12-01T03:10:39.012786+0000","last_peered":"2023-12-01T03:10:39.012786+0000","last_clean":"2023-12-01T03:10:39.012786+0000","last_became_active":"2023-12-01T03:10:39.004783+0000","last_became_peered":"2023-12-01T03:10:39.004783+0000","last_unstale":"2023-12-01T03:10:39.012786+0000","last_undegraded":"2023-12-01T03:10:39.012786+0000","last_fullsized":"2023-12-01T03:10:39.012786+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983588+0000","last_change":"2023-12-01T03:10:24.684248+0000","last_active":"2023-12-01T03:10:38.983588+0000","last_peered":"2023-12-01T03:10:38.983588+0000","last_clean":"2023-12-01T03:10:38.983588+0000","last_became_active":"2023-12-01T03:10:24.680714+0000","last_became_peered":"2023-12-01T03:10:24.680714+0000","last_unstale":"2023-12-01T03:10:38.983588+0000","last_undegraded":"2023-12-01T03:10:38.983588+0000","last_fullsized":"2023-12-01T03:10:38.983588+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:19:02.474558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.982169+0000","last_change":"2023-12-01T03:10:25.174085+0000","last_active":"2023-12-01T03:10:38.982169+0000","last_peered":"2023-12-01T03:10:38.982169+0000","last_clean":"2023-12-01T03:10:38.982169+0000","last_became_active":"2023-12-01T03:10:25.173802+0000","last_became_peered":"2023-12-01T03:10:25.173802+0000","last_unstale":"2023-12-01T03:10:38.982169+0000","last_undegraded":"2023-12-01T03:10:38.982169+0000","last_fullsized":"2023-12-01T03:10:38.982169+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:39.145914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.982926+0000","last_change":"2023-12-01T03:10:38.512523+0000","last_active":"2023-12-01T03:10:38.982926+0000","last_peered":"2023-12-01T03:10:38.982926+0000","last_clean":"2023-12-01T03:10:38.982926+0000","last_became_active":"2023-12-01T03:10:38.512225+0000","last_became_peered":"2023-12-01T03:10:38.512225+0000","last_unstale":"2023-12-01T03:10:38.982926+0000","last_undegraded":"2023-12-01T03:10:38.982926+0000","last_fullsized":"2023-12-01T03:10:38.982926+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:55:05.819515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003662+0000","last_change":"2023-12-01T03:10:39.001307+0000","last_active":"2023-12-01T03:10:39.003662+0000","last_peered":"2023-12-01T03:10:39.003662+0000","last_clean":"2023-12-01T03:10:39.003662+0000","last_became_active":"2023-12-01T03:10:39.000379+0000","last_became_peered":"2023-12-01T03:10:39.000379+0000","last_unstale":"2023-12-01T03:10:39.003662+0000","last_undegraded":"2023-12-01T03:10:39.003662+0000","last_fullsized":"2023-12-01T03:10:39.003662+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:39.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986591+0000","last_change":"2023-12-01T03:10:24.683240+0000","last_active":"2023-12-01T03:10:38.986591+0000","last_peered":"2023-12-01T03:10:38.986591+0000","last_clean":"2023-12-01T03:10:38.986591+0000","last_became_active":"2023-12-01T03:10:24.682933+0000","last_became_peered":"2023-12-01T03:10:24.682933+0000","last_unstale":"2023-12-01T03:10:38.986591+0000","last_undegraded":"2023-12-01T03:10:38.986591+0000","last_fullsized":"2023-12-01T03:10:38.986591+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:30:40.211424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.975090+0000","last_change":"2023-12-01T03:10:25.181396+0000","last_active":"2023-12-01T03:10:38.975090+0000","last_peered":"2023-12-01T03:10:38.975090+0000","last_clean":"2023-12-01T03:10:38.975090+0000","last_became_active":"2023-12-01T03:10:25.181134+0000","last_became_peered":"2023-12-01T03:10:25.181134+0000","last_unstale":"2023-12-01T03:10:38.975090+0000","last_undegraded":"2023-12-01T03:10:38.975090+0000","last_fullsized":"2023-12-01T03:10:38.975090+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:04.776013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.981977+0000","last_change":"2023-12-01T03:10:38.522641+0000","last_active":"2023-12-01T03:10:38.981977+0000","last_peered":"2023-12-01T03:10:38.981977+0000","last_clean":"2023-12-01T03:10:38.981977+0000","last_became_active":"2023-12-01T03:10:38.522225+0000","last_became_peered":"2023-12-01T03:10:38.522225+0000","last_unstale":"2023-12-01T03:10:38.981977+0000","last_undegraded":"2023-12-01T03:10:38.981977+0000","last_fullsized":"2023-12-01T03:10:38.981977+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:13.766818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.010103+0000","last_change":"2023-12-01T03:10:38.993126+0000","last_active":"2023-12-01T03:10:39.010103+0000","last_peered":"2023-12-01T03:10:39.010103+0000","last_clean":"2023-12-01T03:10:39.010103+0000","last_became_active":"2023-12-01T03:10:38.992847+0000","last_became_peered":"2023-12-01T03:10:38.992847+0000","last_unstale":"2023-12-01T03:10:39.010103+0000","last_undegraded":"2023-12-01T03:10:39.010103+0000","last_fullsized":"2023-12-01T03:10:39.010103+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985936+0000","last_change":"2023-12-01T03:10:24.687326+0000","last_active":"2023-12-01T03:10:38.985936+0000","last_peered":"2023-12-01T03:10:38.985936+0000","last_clean":"2023-12-01T03:10:38.985936+0000","last_became_active":"2023-12-01T03:10:24.687036+0000","last_became_peered":"2023-12-01T03:10:24.687036+0000","last_unstale":"2023-12-01T03:10:38.985936+0000","last_undegraded":"2023-12-01T03:10:38.985936+0000","last_fullsized":"2023-12-01T03:10:38.985936+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:36:57.969605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986462+0000","last_change":"2023-12-01T03:10:25.190210+0000","last_active":"2023-12-01T03:10:38.986462+0000","last_peered":"2023-12-01T03:10:38.986462+0000","last_clean":"2023-12-01T03:10:38.986462+0000","last_became_active":"2023-12-01T03:10:25.189957+0000","last_became_peered":"2023-12-01T03:10:25.189957+0000","last_unstale":"2023-12-01T03:10:38.986462+0000","last_undegraded":"2023-12-01T03:10:38.986462+0000","last_fullsized":"2023-12-01T03:10:38.986462+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:52:01.783625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988039+0000","last_change":"2023-12-01T03:10:38.514437+0000","last_active":"2023-12-01T03:10:38.988039+0000","last_peered":"2023-12-01T03:10:38.988039+0000","last_clean":"2023-12-01T03:10:38.988039+0000","last_became_active":"2023-12-01T03:10:38.514155+0000","last_became_peered":"2023-12-01T03:10:38.514155+0000","last_unstale":"2023-12-01T03:10:38.988039+0000","last_undegraded":"2023-12-01T03:10:38.988039+0000","last_fullsized":"2023-12-01T03:10:38.988039+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:06.817444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.015220+0000","last_change":"2023-12-01T03:10:39.013154+0000","last_active":"2023-12-01T03:10:39.015220+0000","last_peered":"2023-12-01T03:10:39.015220+0000","last_clean":"2023-12-01T03:10:39.015220+0000","last_became_active":"2023-12-01T03:10:39.012155+0000","last_became_peered":"2023-12-01T03:10:39.012155+0000","last_unstale":"2023-12-01T03:10:39.015220+0000","last_undegraded":"2023-12-01T03:10:39.015220+0000","last_fullsized":"2023-12-01T03:10:39.015220+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:10:53.596012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.009471+0000","last_change":"2023-12-01T03:10:24.697595+0000","last_active":"2023-12-01T03:10:39.009471+0000","last_peered":"2023-12-01T03:10:39.009471+0000","last_clean":"2023-12-01T03:10:39.009471+0000","last_became_active":"2023-12-01T03:10:24.697311+0000","last_became_peered":"2023-12-01T03:10:24.697311+0000","last_unstale":"2023-12-01T03:10:39.009471+0000","last_undegraded":"2023-12-01T03:10:39.009471+0000","last_fullsized":"2023-12-01T03:10:39.009471+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:45.743456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.004827+0000","last_change":"2023-12-01T03:10:25.192028+0000","last_active":"2023-12-01T03:10:39.004827+0000","last_peered":"2023-12-01T03:10:39.004827+0000","last_clean":"2023-12-01T03:10:39.004827+0000","last_became_active":"2023-12-01T03:10:25.191747+0000","last_became_peered":"2023-12-01T03:10:25.191747+0000","last_unstale":"2023-12-01T03:10:39.004827+0000","last_undegraded":"2023-12-01T03:10:39.004827+0000","last_fullsized":"2023-12-01T03:10:39.004827+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.976006+0000","last_change":"2023-12-01T03:10:38.517012+0000","last_active":"2023-12-01T03:10:38.976006+0000","last_peered":"2023-12-01T03:10:38.976006+0000","last_clean":"2023-12-01T03:10:38.976006+0000","last_became_active":"2023-12-01T03:10:38.516749+0000","last_became_peered":"2023-12-01T03:10:38.516749+0000","last_unstale":"2023-12-01T03:10:38.976006+0000","last_undegraded":"2023-12-01T03:10:38.976006+0000","last_fullsized":"2023-12-01T03:10:38.976006+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:29.962162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008414+0000","last_change":"2023-12-01T03:10:39.005634+0000","last_active":"2023-12-01T03:10:39.008414+0000","last_peered":"2023-12-01T03:10:39.008414+0000","last_clean":"2023-12-01T03:10:39.008414+0000","last_became_active":"2023-12-01T03:10:39.005386+0000","last_became_peered":"2023-12-01T03:10:39.005386+0000","last_unstale":"2023-12-01T03:10:39.008414+0000","last_undegraded":"2023-12-01T03:10:39.008414+0000","last_fullsized":"2023-12-01T03:10:39.008414+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996938+0000","last_change":"2023-12-01T03:10:24.697992+0000","last_active":"2023-12-01T03:10:38.996938+0000","last_peered":"2023-12-01T03:10:38.996938+0000","last_clean":"2023-12-01T03:10:38.996938+0000","last_became_active":"2023-12-01T03:10:24.696883+0000","last_became_peered":"2023-12-01T03:10:24.696883+0000","last_unstale":"2023-12-01T03:10:38.996938+0000","last_undegraded":"2023-12-01T03:10:38.996938+0000","last_fullsized":"2023-12-01T03:10:38.996938+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:40.815147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989989+0000","last_change":"2023-12-01T03:10:25.175476+0000","last_active":"2023-12-01T03:10:38.989989+0000","last_peered":"2023-12-01T03:10:38.989989+0000","last_clean":"2023-12-01T03:10:38.989989+0000","last_became_active":"2023-12-01T03:10:25.175211+0000","last_became_peered":"2023-12-01T03:10:25.175211+0000","last_unstale":"2023-12-01T03:10:38.989989+0000","last_undegraded":"2023-12-01T03:10:38.989989+0000","last_fullsized":"2023-12-01T03:10:38.989989+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:22.358205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.7","version":"217'2","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.010053+0000","last_change":"2023-12-01T03:10:38.513347+0000","last_active":"2023-12-01T03:10:39.010053+0000","last_peered":"2023-12-01T03:10:39.010053+0000","last_clean":"2023-12-01T03:10:39.010053+0000","last_became_active":"2023-12-01T03:10:38.512782+0000","last_became_peered":"2023-12-01T03:10:38.512782+0000","last_unstale":"2023-12-01T03:10:39.010053+0000","last_undegraded":"2023-12-01T03:10:39.010053+0000","last_fullsized":"2023-12-01T03:10:39.010053+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:06.647413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.010194+0000","last_change":"2023-12-01T03:10:39.007372+0000","last_active":"2023-12-01T03:10:39.010194+0000","last_peered":"2023-12-01T03:10:39.010194+0000","last_clean":"2023-12-01T03:10:39.010194+0000","last_became_active":"2023-12-01T03:10:39.007059+0000","last_became_peered":"2023-12-01T03:10:39.007059+0000","last_unstale":"2023-12-01T03:10:39.010194+0000","last_undegraded":"2023-12-01T03:10:39.010194+0000","last_fullsized":"2023-12-01T03:10:39.010194+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.492807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"214'1","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.992263+0000","last_change":"2023-12-01T03:10:24.697890+0000","last_active":"2023-12-01T03:10:38.992263+0000","last_peered":"2023-12-01T03:10:38.992263+0000","last_clean":"2023-12-01T03:10:38.992263+0000","last_became_active":"2023-12-01T03:10:24.697679+0000","last_became_peered":"2023-12-01T03:10:24.697679+0000","last_unstale":"2023-12-01T03:10:38.992263+0000","last_undegraded":"2023-12-01T03:10:38.992263+0000","last_fullsized":"2023-12-01T03:10:38.992263+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996107+0000","last_change":"2023-12-01T03:10:25.168359+0000","last_active":"2023-12-01T03:10:38.996107+0000","last_peered":"2023-12-01T03:10:38.996107+0000","last_clean":"2023-12-01T03:10:38.996107+0000","last_became_active":"2023-12-01T03:10:25.168082+0000","last_became_peered":"2023-12-01T03:10:25.168082+0000","last_unstale":"2023-12-01T03:10:38.996107+0000","last_undegraded":"2023-12-01T03:10:38.996107+0000","last_fullsized":"2023-12-01T03:10:38.996107+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:06.024028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989583+0000","last_change":"2023-12-01T03:10:38.524232+0000","last_active":"2023-12-01T03:10:38.989583+0000","last_peered":"2023-12-01T03:10:38.989583+0000","last_clean":"2023-12-01T03:10:38.989583+0000","last_became_active":"2023-12-01T03:10:38.523871+0000","last_became_peered":"2023-12-01T03:10:38.523871+0000","last_unstale":"2023-12-01T03:10:38.989583+0000","last_undegraded":"2023-12-01T03:10:38.989583+0000","last_fullsized":"2023-12-01T03:10:38.989583+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:47:06.647413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.992943+0000","last_change":"2023-12-01T03:10:38.991914+0000","last_active":"2023-12-01T03:10:38.992943+0000","last_peered":"2023-12-01T03:10:38.992943+0000","last_clean":"2023-12-01T03:10:38.992943+0000","last_became_active":"2023-12-01T03:10:38.991635+0000","last_became_peered":"2023-12-01T03:10:38.991635+0000","last_unstale":"2023-12-01T03:10:38.992943+0000","last_undegraded":"2023-12-01T03:10:38.992943+0000","last_fullsized":"2023-12-01T03:10:38.992943+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.7","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.976303+0000","last_change":"2023-12-01T03:10:24.687160+0000","last_active":"2023-12-01T03:10:38.976303+0000","last_peered":"2023-12-01T03:10:38.976303+0000","last_clean":"2023-12-01T03:10:38.976303+0000","last_became_active":"2023-12-01T03:10:24.686184+0000","last_became_peered":"2023-12-01T03:10:24.686184+0000","last_unstale":"2023-12-01T03:10:38.976303+0000","last_undegraded":"2023-12-01T03:10:38.976303+0000","last_fullsized":"2023-12-01T03:10:38.976303+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:05.006954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.991332+0000","last_change":"2023-12-01T03:10:25.184992+0000","last_active":"2023-12-01T03:10:38.991332+0000","last_peered":"2023-12-01T03:10:38.991332+0000","last_clean":"2023-12-01T03:10:38.991332+0000","last_became_active":"2023-12-01T03:10:25.184670+0000","last_became_peered":"2023-12-01T03:10:25.184670+0000","last_unstale":"2023-12-01T03:10:38.991332+0000","last_undegraded":"2023-12-01T03:10:38.991332+0000","last_fullsized":"2023-12-01T03:10:38.991332+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:40.143822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996407+0000","last_change":"2023-12-01T03:10:38.517303+0000","last_active":"2023-12-01T03:10:38.996407+0000","last_peered":"2023-12-01T03:10:38.996407+0000","last_clean":"2023-12-01T03:10:38.996407+0000","last_became_active":"2023-12-01T03:10:38.516948+0000","last_became_peered":"2023-12-01T03:10:38.516948+0000","last_unstale":"2023-12-01T03:10:38.996407+0000","last_undegraded":"2023-12-01T03:10:38.996407+0000","last_fullsized":"2023-12-01T03:10:38.996407+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:58:41.627485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.017202+0000","last_change":"2023-12-01T03:10:39.015761+0000","last_active":"2023-12-01T03:10:39.017202+0000","last_peered":"2023-12-01T03:10:39.017202+0000","last_clean":"2023-12-01T03:10:39.017202+0000","last_became_active":"2023-12-01T03:10:39.015474+0000","last_became_peered":"2023-12-01T03:10:39.015474+0000","last_unstale":"2023-12-01T03:10:39.017202+0000","last_undegraded":"2023-12-01T03:10:39.017202+0000","last_fullsized":"2023-12-01T03:10:39.017202+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:18.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.973980+0000","last_change":"2023-12-01T03:10:24.685003+0000","last_active":"2023-12-01T03:10:38.973980+0000","last_peered":"2023-12-01T03:10:38.973980+0000","last_clean":"2023-12-01T03:10:38.973980+0000","last_became_active":"2023-12-01T03:10:24.684425+0000","last_became_peered":"2023-12-01T03:10:24.684425+0000","last_unstale":"2023-12-01T03:10:38.973980+0000","last_undegraded":"2023-12-01T03:10:38.973980+0000","last_fullsized":"2023-12-01T03:10:38.973980+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:45.173492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.993444+0000","last_change":"2023-12-01T03:10:25.174479+0000","last_active":"2023-12-01T03:10:38.993444+0000","last_peered":"2023-12-01T03:10:38.993444+0000","last_clean":"2023-12-01T03:10:38.993444+0000","last_became_active":"2023-12-01T03:10:25.174194+0000","last_became_peered":"2023-12-01T03:10:25.174194+0000","last_unstale":"2023-12-01T03:10:38.993444+0000","last_undegraded":"2023-12-01T03:10:38.993444+0000","last_fullsized":"2023-12-01T03:10:38.993444+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985021+0000","last_change":"2023-12-01T03:10:38.516993+0000","last_active":"2023-12-01T03:10:38.985021+0000","last_peered":"2023-12-01T03:10:38.985021+0000","last_clean":"2023-12-01T03:10:38.985021+0000","last_became_active":"2023-12-01T03:10:38.516717+0000","last_became_peered":"2023-12-01T03:10:38.516717+0000","last_unstale":"2023-12-01T03:10:38.985021+0000","last_undegraded":"2023-12-01T03:10:38.985021+0000","last_fullsized":"2023-12-01T03:10:38.985021+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:51.689717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.018399+0000","last_change":"2023-12-01T03:10:39.013312+0000","last_active":"2023-12-01T03:10:39.018399+0000","last_peered":"2023-12-01T03:10:39.018399+0000","last_clean":"2023-12-01T03:10:39.018399+0000","last_became_active":"2023-12-01T03:10:39.012886+0000","last_became_peered":"2023-12-01T03:10:39.012886+0000","last_unstale":"2023-12-01T03:10:39.018399+0000","last_undegraded":"2023-12-01T03:10:39.018399+0000","last_fullsized":"2023-12-01T03:10:39.018399+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:53.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985939+0000","last_change":"2023-12-01T03:10:24.694594+0000","last_active":"2023-12-01T03:10:38.985939+0000","last_peered":"2023-12-01T03:10:38.985939+0000","last_clean":"2023-12-01T03:10:38.985939+0000","last_became_active":"2023-12-01T03:10:24.693808+0000","last_became_peered":"2023-12-01T03:10:24.693808+0000","last_unstale":"2023-12-01T03:10:38.985939+0000","last_undegraded":"2023-12-01T03:10:38.985939+0000","last_fullsized":"2023-12-01T03:10:38.985939+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:12.933987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.973878+0000","last_change":"2023-12-01T03:10:25.173728+0000","last_active":"2023-12-01T03:10:38.973878+0000","last_peered":"2023-12-01T03:10:38.973878+0000","last_clean":"2023-12-01T03:10:38.973878+0000","last_became_active":"2023-12-01T03:10:25.173449+0000","last_became_peered":"2023-12-01T03:10:25.173449+0000","last_unstale":"2023-12-01T03:10:38.973878+0000","last_undegraded":"2023-12-01T03:10:38.973878+0000","last_fullsized":"2023-12-01T03:10:38.973878+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:20.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"217'2","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012868+0000","last_change":"2023-12-01T03:10:38.508401+0000","last_active":"2023-12-01T03:10:39.012868+0000","last_peered":"2023-12-01T03:10:39.012868+0000","last_clean":"2023-12-01T03:10:39.012868+0000","last_became_active":"2023-12-01T03:10:38.508089+0000","last_became_peered":"2023-12-01T03:10:38.508089+0000","last_unstale":"2023-12-01T03:10:39.012868+0000","last_undegraded":"2023-12-01T03:10:39.012868+0000","last_fullsized":"2023-12-01T03:10:39.012868+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:56.645685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003642+0000","last_change":"2023-12-01T03:10:39.000679+0000","last_active":"2023-12-01T03:10:39.003642+0000","last_peered":"2023-12-01T03:10:39.003642+0000","last_clean":"2023-12-01T03:10:39.003642+0000","last_became_active":"2023-12-01T03:10:38.997903+0000","last_became_peered":"2023-12-01T03:10:38.997903+0000","last_unstale":"2023-12-01T03:10:39.003642+0000","last_undegraded":"2023-12-01T03:10:39.003642+0000","last_fullsized":"2023-12-01T03:10:39.003642+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:27.357798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985640+0000","last_change":"2023-12-01T03:10:24.685775+0000","last_active":"2023-12-01T03:10:38.985640+0000","last_peered":"2023-12-01T03:10:38.985640+0000","last_clean":"2023-12-01T03:10:38.985640+0000","last_became_active":"2023-12-01T03:10:24.685178+0000","last_became_peered":"2023-12-01T03:10:24.685178+0000","last_unstale":"2023-12-01T03:10:38.985640+0000","last_undegraded":"2023-12-01T03:10:38.985640+0000","last_fullsized":"2023-12-01T03:10:38.985640+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:21.942226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.981531+0000","last_change":"2023-12-01T03:10:25.180768+0000","last_active":"2023-12-01T03:10:38.981531+0000","last_peered":"2023-12-01T03:10:38.981531+0000","last_clean":"2023-12-01T03:10:38.981531+0000","last_became_active":"2023-12-01T03:10:25.179796+0000","last_became_peered":"2023-12-01T03:10:25.179796+0000","last_unstale":"2023-12-01T03:10:38.981531+0000","last_undegraded":"2023-12-01T03:10:38.981531+0000","last_fullsized":"2023-12-01T03:10:38.981531+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983624+0000","last_change":"2023-12-01T03:10:38.511676+0000","last_active":"2023-12-01T03:10:38.983624+0000","last_peered":"2023-12-01T03:10:38.983624+0000","last_clean":"2023-12-01T03:10:38.983624+0000","last_became_active":"2023-12-01T03:10:38.511406+0000","last_became_peered":"2023-12-01T03:10:38.511406+0000","last_unstale":"2023-12-01T03:10:38.983624+0000","last_undegraded":"2023-12-01T03:10:38.983624+0000","last_fullsized":"2023-12-01T03:10:38.983624+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:44.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003720+0000","last_change":"2023-12-01T03:10:38.997814+0000","last_active":"2023-12-01T03:10:39.003720+0000","last_peered":"2023-12-01T03:10:39.003720+0000","last_clean":"2023-12-01T03:10:39.003720+0000","last_became_active":"2023-12-01T03:10:38.997558+0000","last_became_peered":"2023-12-01T03:10:38.997558+0000","last_unstale":"2023-12-01T03:10:39.003720+0000","last_undegraded":"2023-12-01T03:10:39.003720+0000","last_fullsized":"2023-12-01T03:10:39.003720+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:49:09.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.b","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007581+0000","last_change":"2023-12-01T03:10:24.695154+0000","last_active":"2023-12-01T03:10:39.007581+0000","last_peered":"2023-12-01T03:10:39.007581+0000","last_clean":"2023-12-01T03:10:39.007581+0000","last_became_active":"2023-12-01T03:10:24.694868+0000","last_became_peered":"2023-12-01T03:10:24.694868+0000","last_unstale":"2023-12-01T03:10:39.007581+0000","last_undegraded":"2023-12-01T03:10:39.007581+0000","last_fullsized":"2023-12-01T03:10:39.007581+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985442+0000","last_change":"2023-12-01T03:10:25.169678+0000","last_active":"2023-12-01T03:10:38.985442+0000","last_peered":"2023-12-01T03:10:38.985442+0000","last_clean":"2023-12-01T03:10:38.985442+0000","last_became_active":"2023-12-01T03:10:25.168714+0000","last_became_peered":"2023-12-01T03:10:25.168714+0000","last_unstale":"2023-12-01T03:10:38.985442+0000","last_undegraded":"2023-12-01T03:10:38.985442+0000","last_fullsized":"2023-12-01T03:10:38.985442+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999750+0000","last_change":"2023-12-01T03:10:38.524888+0000","last_active":"2023-12-01T03:10:38.999750+0000","last_peered":"2023-12-01T03:10:38.999750+0000","last_clean":"2023-12-01T03:10:38.999750+0000","last_became_active":"2023-12-01T03:10:38.524601+0000","last_became_peered":"2023-12-01T03:10:38.524601+0000","last_unstale":"2023-12-01T03:10:38.999750+0000","last_undegraded":"2023-12-01T03:10:38.999750+0000","last_fullsized":"2023-12-01T03:10:38.999750+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:47.875991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.011924+0000","last_change":"2023-12-01T03:10:39.009719+0000","last_active":"2023-12-01T03:10:39.011924+0000","last_peered":"2023-12-01T03:10:39.011924+0000","last_clean":"2023-12-01T03:10:39.011924+0000","last_became_active":"2023-12-01T03:10:39.009561+0000","last_became_peered":"2023-12-01T03:10:39.009561+0000","last_unstale":"2023-12-01T03:10:39.011924+0000","last_undegraded":"2023-12-01T03:10:39.011924+0000","last_fullsized":"2023-12-01T03:10:39.011924+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:35:19.853707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.975519+0000","last_change":"2023-12-01T03:10:24.683594+0000","last_active":"2023-12-01T03:10:38.975519+0000","last_peered":"2023-12-01T03:10:38.975519+0000","last_clean":"2023-12-01T03:10:38.975519+0000","last_became_active":"2023-12-01T03:10:24.679979+0000","last_became_peered":"2023-12-01T03:10:24.679979+0000","last_unstale":"2023-12-01T03:10:38.975519+0000","last_undegraded":"2023-12-01T03:10:38.975519+0000","last_fullsized":"2023-12-01T03:10:38.975519+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:12.511044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987554+0000","last_change":"2023-12-01T03:10:25.172837+0000","last_active":"2023-12-01T03:10:38.987554+0000","last_peered":"2023-12-01T03:10:38.987554+0000","last_clean":"2023-12-01T03:10:38.987554+0000","last_became_active":"2023-12-01T03:10:25.172602+0000","last_became_peered":"2023-12-01T03:10:25.172602+0000","last_unstale":"2023-12-01T03:10:38.987554+0000","last_undegraded":"2023-12-01T03:10:38.987554+0000","last_fullsized":"2023-12-01T03:10:38.987554+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"217'2","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.011646+0000","last_change":"2023-12-01T03:10:38.517877+0000","last_active":"2023-12-01T03:10:39.011646+0000","last_peered":"2023-12-01T03:10:39.011646+0000","last_clean":"2023-12-01T03:10:39.011646+0000","last_became_active":"2023-12-01T03:10:38.517241+0000","last_became_peered":"2023-12-01T03:10:38.517241+0000","last_unstale":"2023-12-01T03:10:39.011646+0000","last_undegraded":"2023-12-01T03:10:39.011646+0000","last_fullsized":"2023-12-01T03:10:39.011646+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:17:32.967785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.004933+0000","last_change":"2023-12-01T03:10:39.003555+0000","last_active":"2023-12-01T03:10:39.004933+0000","last_peered":"2023-12-01T03:10:39.004933+0000","last_clean":"2023-12-01T03:10:39.004933+0000","last_became_active":"2023-12-01T03:10:39.003311+0000","last_became_peered":"2023-12-01T03:10:39.003311+0000","last_unstale":"2023-12-01T03:10:39.004933+0000","last_undegraded":"2023-12-01T03:10:39.004933+0000","last_fullsized":"2023-12-01T03:10:39.004933+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.d","version":"213'1","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.975698+0000","last_change":"2023-12-01T03:10:24.691398+0000","last_active":"2023-12-01T03:10:38.975698+0000","last_peered":"2023-12-01T03:10:38.975698+0000","last_clean":"2023-12-01T03:10:38.975698+0000","last_became_active":"2023-12-01T03:10:24.691110+0000","last_became_peered":"2023-12-01T03:10:24.691110+0000","last_unstale":"2023-12-01T03:10:38.975698+0000","last_undegraded":"2023-12-01T03:10:38.975698+0000","last_fullsized":"2023-12-01T03:10:38.975698+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:24.670899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999898+0000","last_change":"2023-12-01T03:10:25.178959+0000","last_active":"2023-12-01T03:10:38.999898+0000","last_peered":"2023-12-01T03:10:38.999898+0000","last_clean":"2023-12-01T03:10:38.999898+0000","last_became_active":"2023-12-01T03:10:25.178268+0000","last_became_peered":"2023-12-01T03:10:25.178268+0000","last_unstale":"2023-12-01T03:10:38.999898+0000","last_undegraded":"2023-12-01T03:10:38.999898+0000","last_fullsized":"2023-12-01T03:10:38.999898+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:55:31.814432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.f","version":"217'1","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.017436+0000","last_change":"2023-12-01T03:10:38.523799+0000","last_active":"2023-12-01T03:10:39.017436+0000","last_peered":"2023-12-01T03:10:39.017436+0000","last_clean":"2023-12-01T03:10:39.017436+0000","last_became_active":"2023-12-01T03:10:38.523512+0000","last_became_peered":"2023-12-01T03:10:38.523512+0000","last_unstale":"2023-12-01T03:10:39.017436+0000","last_undegraded":"2023-12-01T03:10:39.017436+0000","last_fullsized":"2023-12-01T03:10:39.017436+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:44.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.017857+0000","last_change":"2023-12-01T03:10:39.015122+0000","last_active":"2023-12-01T03:10:39.017857+0000","last_peered":"2023-12-01T03:10:39.017857+0000","last_clean":"2023-12-01T03:10:39.017857+0000","last_became_active":"2023-12-01T03:10:39.011700+0000","last_became_peered":"2023-12-01T03:10:39.011700+0000","last_unstale":"2023-12-01T03:10:39.017857+0000","last_undegraded":"2023-12-01T03:10:39.017857+0000","last_fullsized":"2023-12-01T03:10:39.017857+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:00:41.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.984279+0000","last_change":"2023-12-01T03:10:24.688107+0000","last_active":"2023-12-01T03:10:38.984279+0000","last_peered":"2023-12-01T03:10:38.984279+0000","last_clean":"2023-12-01T03:10:38.984279+0000","last_became_active":"2023-12-01T03:10:24.686930+0000","last_became_peered":"2023-12-01T03:10:24.686930+0000","last_unstale":"2023-12-01T03:10:38.984279+0000","last_undegraded":"2023-12-01T03:10:38.984279+0000","last_fullsized":"2023-12-01T03:10:38.984279+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:44.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986385+0000","last_change":"2023-12-01T03:10:25.180913+0000","last_active":"2023-12-01T03:10:38.986385+0000","last_peered":"2023-12-01T03:10:38.986385+0000","last_clean":"2023-12-01T03:10:38.986385+0000","last_became_active":"2023-12-01T03:10:25.180626+0000","last_became_peered":"2023-12-01T03:10:25.180626+0000","last_unstale":"2023-12-01T03:10:38.986385+0000","last_undegraded":"2023-12-01T03:10:38.986385+0000","last_fullsized":"2023-12-01T03:10:38.986385+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987806+0000","last_change":"2023-12-01T03:10:38.519139+0000","last_active":"2023-12-01T03:10:38.987806+0000","last_peered":"2023-12-01T03:10:38.987806+0000","last_clean":"2023-12-01T03:10:38.987806+0000","last_became_active":"2023-12-01T03:10:38.518855+0000","last_became_peered":"2023-12-01T03:10:38.518855+0000","last_unstale":"2023-12-01T03:10:38.987806+0000","last_undegraded":"2023-12-01T03:10:38.987806+0000","last_fullsized":"2023-12-01T03:10:38.987806+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:52.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.019862+0000","last_change":"2023-12-01T03:10:39.017561+0000","last_active":"2023-12-01T03:10:39.019862+0000","last_peered":"2023-12-01T03:10:39.019862+0000","last_clean":"2023-12-01T03:10:39.019862+0000","last_became_active":"2023-12-01T03:10:39.016757+0000","last_became_peered":"2023-12-01T03:10:39.016757+0000","last_unstale":"2023-12-01T03:10:39.019862+0000","last_undegraded":"2023-12-01T03:10:39.019862+0000","last_fullsized":"2023-12-01T03:10:39.019862+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:47:30.021172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.f","version":"213'1","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.006354+0000","last_change":"2023-12-01T03:10:24.699945+0000","last_active":"2023-12-01T03:10:39.006354+0000","last_peered":"2023-12-01T03:10:39.006354+0000","last_clean":"2023-12-01T03:10:39.006354+0000","last_became_active":"2023-12-01T03:10:24.699598+0000","last_became_peered":"2023-12-01T03:10:24.699598+0000","last_unstale":"2023-12-01T03:10:39.006354+0000","last_undegraded":"2023-12-01T03:10:39.006354+0000","last_fullsized":"2023-12-01T03:10:39.006354+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.993689+0000","last_change":"2023-12-01T03:10:25.181692+0000","last_active":"2023-12-01T03:10:38.993689+0000","last_peered":"2023-12-01T03:10:38.993689+0000","last_clean":"2023-12-01T03:10:38.993689+0000","last_became_active":"2023-12-01T03:10:25.181316+0000","last_became_peered":"2023-12-01T03:10:25.181316+0000","last_unstale":"2023-12-01T03:10:38.993689+0000","last_undegraded":"2023-12-01T03:10:38.993689+0000","last_fullsized":"2023-12-01T03:10:38.993689+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:57:07.414252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.d","version":"217'1","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.005022+0000","last_change":"2023-12-01T03:10:38.518068+0000","last_active":"2023-12-01T03:10:39.005022+0000","last_peered":"2023-12-01T03:10:39.005022+0000","last_clean":"2023-12-01T03:10:39.005022+0000","last_became_active":"2023-12-01T03:10:38.517792+0000","last_became_peered":"2023-12-01T03:10:38.517792+0000","last_unstale":"2023-12-01T03:10:39.005022+0000","last_undegraded":"2023-12-01T03:10:39.005022+0000","last_fullsized":"2023-12-01T03:10:39.005022+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:01.616973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.017706+0000","last_change":"2023-12-01T03:10:39.014057+0000","last_active":"2023-12-01T03:10:39.017706+0000","last_peered":"2023-12-01T03:10:39.017706+0000","last_clean":"2023-12-01T03:10:39.017706+0000","last_became_active":"2023-12-01T03:10:39.012967+0000","last_became_peered":"2023-12-01T03:10:39.012967+0000","last_unstale":"2023-12-01T03:10:39.017706+0000","last_undegraded":"2023-12-01T03:10:39.017706+0000","last_fullsized":"2023-12-01T03:10:39.017706+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"213'8","reported_seq":23,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987714+0000","last_change":"2023-12-01T03:10:24.685759+0000","last_active":"2023-12-01T03:10:38.987714+0000","last_peered":"2023-12-01T03:10:38.987714+0000","last_clean":"2023-12-01T03:10:38.987714+0000","last_became_active":"2023-12-01T03:10:24.684800+0000","last_became_peered":"2023-12-01T03:10:24.684800+0000","last_unstale":"2023-12-01T03:10:38.987714+0000","last_undegraded":"2023-12-01T03:10:38.987714+0000","last_fullsized":"2023-12-01T03:10:38.987714+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:29.129070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.000458+0000","last_change":"2023-12-01T03:10:25.193021+0000","last_active":"2023-12-01T03:10:39.000458+0000","last_peered":"2023-12-01T03:10:39.000458+0000","last_clean":"2023-12-01T03:10:39.000458+0000","last_became_active":"2023-12-01T03:10:25.192051+0000","last_became_peered":"2023-12-01T03:10:25.192051+0000","last_unstale":"2023-12-01T03:10:39.000458+0000","last_undegraded":"2023-12-01T03:10:39.000458+0000","last_fullsized":"2023-12-01T03:10:39.000458+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.974245+0000","last_change":"2023-12-01T03:10:38.517969+0000","last_active":"2023-12-01T03:10:38.974245+0000","last_peered":"2023-12-01T03:10:38.974245+0000","last_clean":"2023-12-01T03:10:38.974245+0000","last_became_active":"2023-12-01T03:10:38.517693+0000","last_became_peered":"2023-12-01T03:10:38.517693+0000","last_unstale":"2023-12-01T03:10:38.974245+0000","last_undegraded":"2023-12-01T03:10:38.974245+0000","last_fullsized":"2023-12-01T03:10:38.974245+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:18.447021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.009010+0000","last_change":"2023-12-01T03:10:39.008089+0000","last_active":"2023-12-01T03:10:39.009010+0000","last_peered":"2023-12-01T03:10:39.009010+0000","last_clean":"2023-12-01T03:10:39.009010+0000","last_became_active":"2023-12-01T03:10:39.007836+0000","last_became_peered":"2023-12-01T03:10:39.007836+0000","last_unstale":"2023-12-01T03:10:39.009010+0000","last_undegraded":"2023-12-01T03:10:39.009010+0000","last_fullsized":"2023-12-01T03:10:39.009010+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.000678+0000","last_change":"2023-12-01T03:10:24.694942+0000","last_active":"2023-12-01T03:10:39.000678+0000","last_peered":"2023-12-01T03:10:39.000678+0000","last_clean":"2023-12-01T03:10:39.000678+0000","last_became_active":"2023-12-01T03:10:24.694664+0000","last_became_peered":"2023-12-01T03:10:24.694664+0000","last_unstale":"2023-12-01T03:10:39.000678+0000","last_undegraded":"2023-12-01T03:10:39.000678+0000","last_fullsized":"2023-12-01T03:10:39.000678+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:38:02.332793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.988703+0000","last_change":"2023-12-01T03:10:25.164054+0000","last_active":"2023-12-01T03:10:38.988703+0000","last_peered":"2023-12-01T03:10:38.988703+0000","last_clean":"2023-12-01T03:10:38.988703+0000","last_became_active":"2023-12-01T03:10:25.163787+0000","last_became_peered":"2023-12-01T03:10:25.163787+0000","last_unstale":"2023-12-01T03:10:38.988703+0000","last_undegraded":"2023-12-01T03:10:38.988703+0000","last_fullsized":"2023-12-01T03:10:38.988703+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:04.568421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"217'2","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.006485+0000","last_change":"2023-12-01T03:10:38.520314+0000","last_active":"2023-12-01T03:10:39.006485+0000","last_peered":"2023-12-01T03:10:39.006485+0000","last_clean":"2023-12-01T03:10:39.006485+0000","last_became_active":"2023-12-01T03:10:38.519056+0000","last_became_peered":"2023-12-01T03:10:38.519056+0000","last_unstale":"2023-12-01T03:10:39.006485+0000","last_undegraded":"2023-12-01T03:10:39.006485+0000","last_fullsized":"2023-12-01T03:10:39.006485+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:40.686905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008940+0000","last_change":"2023-12-01T03:10:39.002983+0000","last_active":"2023-12-01T03:10:39.008940+0000","last_peered":"2023-12-01T03:10:39.008940+0000","last_clean":"2023-12-01T03:10:39.008940+0000","last_became_active":"2023-12-01T03:10:39.001294+0000","last_became_peered":"2023-12-01T03:10:39.001294+0000","last_unstale":"2023-12-01T03:10:39.008940+0000","last_undegraded":"2023-12-01T03:10:39.008940+0000","last_fullsized":"2023-12-01T03:10:39.008940+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989754+0000","last_change":"2023-12-01T03:10:24.685823+0000","last_active":"2023-12-01T03:10:38.989754+0000","last_peered":"2023-12-01T03:10:38.989754+0000","last_clean":"2023-12-01T03:10:38.989754+0000","last_became_active":"2023-12-01T03:10:24.685560+0000","last_became_peered":"2023-12-01T03:10:24.685560+0000","last_unstale":"2023-12-01T03:10:38.989754+0000","last_undegraded":"2023-12-01T03:10:38.989754+0000","last_fullsized":"2023-12-01T03:10:38.989754+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.981520+0000","last_change":"2023-12-01T03:10:25.183804+0000","last_active":"2023-12-01T03:10:38.981520+0000","last_peered":"2023-12-01T03:10:38.981520+0000","last_clean":"2023-12-01T03:10:38.981520+0000","last_became_active":"2023-12-01T03:10:25.183079+0000","last_became_peered":"2023-12-01T03:10:25.183079+0000","last_unstale":"2023-12-01T03:10:38.981520+0000","last_undegraded":"2023-12-01T03:10:38.981520+0000","last_fullsized":"2023-12-01T03:10:38.981520+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.10","version":"217'8","reported_seq":19,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.016315+0000","last_change":"2023-12-01T03:10:38.508144+0000","last_active":"2023-12-01T03:10:39.016315+0000","last_peered":"2023-12-01T03:10:39.016315+0000","last_clean":"2023-12-01T03:10:39.016315+0000","last_became_active":"2023-12-01T03:10:38.507850+0000","last_became_peered":"2023-12-01T03:10:38.507850+0000","last_unstale":"2023-12-01T03:10:39.016315+0000","last_undegraded":"2023-12-01T03:10:39.016315+0000","last_fullsized":"2023-12-01T03:10:39.016315+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:43:52.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.018299+0000","last_change":"2023-12-01T03:10:39.009153+0000","last_active":"2023-12-01T03:10:39.018299+0000","last_peered":"2023-12-01T03:10:39.018299+0000","last_clean":"2023-12-01T03:10:39.018299+0000","last_became_active":"2023-12-01T03:10:39.007426+0000","last_became_peered":"2023-12-01T03:10:39.007426+0000","last_unstale":"2023-12-01T03:10:39.018299+0000","last_undegraded":"2023-12-01T03:10:39.018299+0000","last_fullsized":"2023-12-01T03:10:39.018299+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:00:45.156063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.13","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987606+0000","last_change":"2023-12-01T03:10:24.692854+0000","last_active":"2023-12-01T03:10:38.987606+0000","last_peered":"2023-12-01T03:10:38.987606+0000","last_clean":"2023-12-01T03:10:38.987606+0000","last_became_active":"2023-12-01T03:10:24.692563+0000","last_became_peered":"2023-12-01T03:10:24.692563+0000","last_unstale":"2023-12-01T03:10:38.987606+0000","last_undegraded":"2023-12-01T03:10:38.987606+0000","last_fullsized":"2023-12-01T03:10:38.987606+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989554+0000","last_change":"2023-12-01T03:10:25.186716+0000","last_active":"2023-12-01T03:10:38.989554+0000","last_peered":"2023-12-01T03:10:38.989554+0000","last_clean":"2023-12-01T03:10:38.989554+0000","last_became_active":"2023-12-01T03:10:25.186422+0000","last_became_peered":"2023-12-01T03:10:25.186422+0000","last_unstale":"2023-12-01T03:10:38.989554+0000","last_undegraded":"2023-12-01T03:10:38.989554+0000","last_fullsized":"2023-12-01T03:10:38.989554+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.984292+0000","last_change":"2023-12-01T03:10:38.508098+0000","last_active":"2023-12-01T03:10:38.984292+0000","last_peered":"2023-12-01T03:10:38.984292+0000","last_clean":"2023-12-01T03:10:38.984292+0000","last_became_active":"2023-12-01T03:10:38.506510+0000","last_became_peered":"2023-12-01T03:10:38.506510+0000","last_unstale":"2023-12-01T03:10:38.984292+0000","last_undegraded":"2023-12-01T03:10:38.984292+0000","last_fullsized":"2023-12-01T03:10:38.984292+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:01.074406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.000941+0000","last_change":"2023-12-01T03:10:38.998284+0000","last_active":"2023-12-01T03:10:39.000941+0000","last_peered":"2023-12-01T03:10:39.000941+0000","last_clean":"2023-12-01T03:10:39.000941+0000","last_became_active":"2023-12-01T03:10:38.997970+0000","last_became_peered":"2023-12-01T03:10:38.997970+0000","last_unstale":"2023-12-01T03:10:39.000941+0000","last_undegraded":"2023-12-01T03:10:39.000941+0000","last_fullsized":"2023-12-01T03:10:39.000941+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:18.605693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"214'8","reported_seq":23,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.009397+0000","last_change":"2023-12-01T03:10:24.697328+0000","last_active":"2023-12-01T03:10:39.009397+0000","last_peered":"2023-12-01T03:10:39.009397+0000","last_clean":"2023-12-01T03:10:39.009397+0000","last_became_active":"2023-12-01T03:10:24.696986+0000","last_became_peered":"2023-12-01T03:10:24.696986+0000","last_unstale":"2023-12-01T03:10:39.009397+0000","last_undegraded":"2023-12-01T03:10:39.009397+0000","last_fullsized":"2023-12-01T03:10:39.009397+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:39:23.655063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.994525+0000","last_change":"2023-12-01T03:10:25.176864+0000","last_active":"2023-12-01T03:10:38.994525+0000","last_peered":"2023-12-01T03:10:38.994525+0000","last_clean":"2023-12-01T03:10:38.994525+0000","last_became_active":"2023-12-01T03:10:25.176588+0000","last_became_peered":"2023-12-01T03:10:25.176588+0000","last_unstale":"2023-12-01T03:10:38.994525+0000","last_undegraded":"2023-12-01T03:10:38.994525+0000","last_fullsized":"2023-12-01T03:10:38.994525+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:08:41.832221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987075+0000","last_change":"2023-12-01T03:10:38.519450+0000","last_active":"2023-12-01T03:10:38.987075+0000","last_peered":"2023-12-01T03:10:38.987075+0000","last_clean":"2023-12-01T03:10:38.987075+0000","last_became_active":"2023-12-01T03:10:38.519108+0000","last_became_peered":"2023-12-01T03:10:38.519108+0000","last_unstale":"2023-12-01T03:10:38.987075+0000","last_undegraded":"2023-12-01T03:10:38.987075+0000","last_fullsized":"2023-12-01T03:10:38.987075+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:50:58.926662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007897+0000","last_change":"2023-12-01T03:10:39.005006+0000","last_active":"2023-12-01T03:10:39.007897+0000","last_peered":"2023-12-01T03:10:39.007897+0000","last_clean":"2023-12-01T03:10:39.007897+0000","last_became_active":"2023-12-01T03:10:39.004573+0000","last_became_peered":"2023-12-01T03:10:39.004573+0000","last_unstale":"2023-12-01T03:10:39.007897+0000","last_undegraded":"2023-12-01T03:10:39.007897+0000","last_fullsized":"2023-12-01T03:10:39.007897+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:07.829269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"213'3","reported_seq":18,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.005034+0000","last_change":"2023-12-01T03:10:24.699528+0000","last_active":"2023-12-01T03:10:39.005034+0000","last_peered":"2023-12-01T03:10:39.005034+0000","last_clean":"2023-12-01T03:10:39.005034+0000","last_became_active":"2023-12-01T03:10:24.699152+0000","last_became_peered":"2023-12-01T03:10:24.699152+0000","last_unstale":"2023-12-01T03:10:39.005034+0000","last_undegraded":"2023-12-01T03:10:39.005034+0000","last_fullsized":"2023-12-01T03:10:39.005034+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:07.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987736+0000","last_change":"2023-12-01T03:10:25.187487+0000","last_active":"2023-12-01T03:10:38.987736+0000","last_peered":"2023-12-01T03:10:38.987736+0000","last_clean":"2023-12-01T03:10:38.987736+0000","last_became_active":"2023-12-01T03:10:25.187198+0000","last_became_peered":"2023-12-01T03:10:25.187198+0000","last_unstale":"2023-12-01T03:10:38.987736+0000","last_undegraded":"2023-12-01T03:10:38.987736+0000","last_fullsized":"2023-12-01T03:10:38.987736+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:32:00.948964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008959+0000","last_change":"2023-12-01T03:10:38.525203+0000","last_active":"2023-12-01T03:10:39.008959+0000","last_peered":"2023-12-01T03:10:39.008959+0000","last_clean":"2023-12-01T03:10:39.008959+0000","last_became_active":"2023-12-01T03:10:38.524926+0000","last_became_peered":"2023-12-01T03:10:38.524926+0000","last_unstale":"2023-12-01T03:10:39.008959+0000","last_undegraded":"2023-12-01T03:10:39.008959+0000","last_fullsized":"2023-12-01T03:10:39.008959+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:17.593868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012144+0000","last_change":"2023-12-01T03:10:39.002492+0000","last_active":"2023-12-01T03:10:39.012144+0000","last_peered":"2023-12-01T03:10:39.012144+0000","last_clean":"2023-12-01T03:10:39.012144+0000","last_became_active":"2023-12-01T03:10:39.001865+0000","last_became_peered":"2023-12-01T03:10:39.001865+0000","last_unstale":"2023-12-01T03:10:39.012144+0000","last_undegraded":"2023-12-01T03:10:39.012144+0000","last_fullsized":"2023-12-01T03:10:39.012144+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:39:30.973988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.993765+0000","last_change":"2023-12-01T03:10:24.691537+0000","last_active":"2023-12-01T03:10:38.993765+0000","last_peered":"2023-12-01T03:10:38.993765+0000","last_clean":"2023-12-01T03:10:38.993765+0000","last_became_active":"2023-12-01T03:10:24.691168+0000","last_became_peered":"2023-12-01T03:10:24.691168+0000","last_unstale":"2023-12-01T03:10:38.993765+0000","last_undegraded":"2023-12-01T03:10:38.993765+0000","last_fullsized":"2023-12-01T03:10:38.993765+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.994339+0000","last_change":"2023-12-01T03:10:25.178648+0000","last_active":"2023-12-01T03:10:38.994339+0000","last_peered":"2023-12-01T03:10:38.994339+0000","last_clean":"2023-12-01T03:10:38.994339+0000","last_became_active":"2023-12-01T03:10:25.178352+0000","last_became_peered":"2023-12-01T03:10:25.178352+0000","last_unstale":"2023-12-01T03:10:38.994339+0000","last_undegraded":"2023-12-01T03:10:38.994339+0000","last_fullsized":"2023-12-01T03:10:38.994339+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.14","version":"217'6","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007381+0000","last_change":"2023-12-01T03:10:38.515822+0000","last_active":"2023-12-01T03:10:39.007381+0000","last_peered":"2023-12-01T03:10:39.007381+0000","last_clean":"2023-12-01T03:10:39.007381+0000","last_became_active":"2023-12-01T03:10:38.515536+0000","last_became_peered":"2023-12-01T03:10:38.515536+0000","last_unstale":"2023-12-01T03:10:39.007381+0000","last_undegraded":"2023-12-01T03:10:39.007381+0000","last_fullsized":"2023-12-01T03:10:39.007381+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:10:36.574945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.994146+0000","last_change":"2023-12-01T03:10:38.988792+0000","last_active":"2023-12-01T03:10:38.994146+0000","last_peered":"2023-12-01T03:10:38.994146+0000","last_clean":"2023-12-01T03:10:38.994146+0000","last_became_active":"2023-12-01T03:10:38.986942+0000","last_became_peered":"2023-12-01T03:10:38.986942+0000","last_unstale":"2023-12-01T03:10:38.994146+0000","last_undegraded":"2023-12-01T03:10:38.994146+0000","last_fullsized":"2023-12-01T03:10:38.994146+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:27.972469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.976223+0000","last_change":"2023-12-01T03:10:24.687709+0000","last_active":"2023-12-01T03:10:38.976223+0000","last_peered":"2023-12-01T03:10:38.976223+0000","last_clean":"2023-12-01T03:10:38.976223+0000","last_became_active":"2023-12-01T03:10:24.687452+0000","last_became_peered":"2023-12-01T03:10:24.687452+0000","last_unstale":"2023-12-01T03:10:38.976223+0000","last_undegraded":"2023-12-01T03:10:38.976223+0000","last_fullsized":"2023-12-01T03:10:38.976223+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:39.126749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.991237+0000","last_change":"2023-12-01T03:10:25.176941+0000","last_active":"2023-12-01T03:10:38.991237+0000","last_peered":"2023-12-01T03:10:38.991237+0000","last_clean":"2023-12-01T03:10:38.991237+0000","last_became_active":"2023-12-01T03:10:25.175692+0000","last_became_peered":"2023-12-01T03:10:25.175692+0000","last_unstale":"2023-12-01T03:10:38.991237+0000","last_undegraded":"2023-12-01T03:10:38.991237+0000","last_fullsized":"2023-12-01T03:10:38.991237+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.15","version":"217'3","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012097+0000","last_change":"2023-12-01T03:10:38.509844+0000","last_active":"2023-12-01T03:10:39.012097+0000","last_peered":"2023-12-01T03:10:39.012097+0000","last_clean":"2023-12-01T03:10:39.012097+0000","last_became_active":"2023-12-01T03:10:38.509319+0000","last_became_peered":"2023-12-01T03:10:38.509319+0000","last_unstale":"2023-12-01T03:10:39.012097+0000","last_undegraded":"2023-12-01T03:10:39.012097+0000","last_fullsized":"2023-12-01T03:10:39.012097+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:12.432439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012565+0000","last_change":"2023-12-01T03:10:39.010700+0000","last_active":"2023-12-01T03:10:39.012565+0000","last_peered":"2023-12-01T03:10:39.012565+0000","last_clean":"2023-12-01T03:10:39.012565+0000","last_became_active":"2023-12-01T03:10:39.010389+0000","last_became_peered":"2023-12-01T03:10:39.010389+0000","last_unstale":"2023-12-01T03:10:39.012565+0000","last_undegraded":"2023-12-01T03:10:39.012565+0000","last_fullsized":"2023-12-01T03:10:39.012565+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:58:53.973502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987602+0000","last_change":"2023-12-01T03:10:24.686553+0000","last_active":"2023-12-01T03:10:38.987602+0000","last_peered":"2023-12-01T03:10:38.987602+0000","last_clean":"2023-12-01T03:10:38.987602+0000","last_became_active":"2023-12-01T03:10:24.686136+0000","last_became_peered":"2023-12-01T03:10:24.686136+0000","last_unstale":"2023-12-01T03:10:38.987602+0000","last_undegraded":"2023-12-01T03:10:38.987602+0000","last_fullsized":"2023-12-01T03:10:38.987602+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:18.663097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.973989+0000","last_change":"2023-12-01T03:10:25.182793+0000","last_active":"2023-12-01T03:10:38.973989+0000","last_peered":"2023-12-01T03:10:38.973989+0000","last_clean":"2023-12-01T03:10:38.973989+0000","last_became_active":"2023-12-01T03:10:25.182524+0000","last_became_peered":"2023-12-01T03:10:25.182524+0000","last_unstale":"2023-12-01T03:10:38.973989+0000","last_undegraded":"2023-12-01T03:10:38.973989+0000","last_fullsized":"2023-12-01T03:10:38.973989+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:25.370249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1a","version":"217'1","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.014656+0000","last_change":"2023-12-01T03:10:38.523078+0000","last_active":"2023-12-01T03:10:39.014656+0000","last_peered":"2023-12-01T03:10:39.014656+0000","last_clean":"2023-12-01T03:10:39.014656+0000","last_became_active":"2023-12-01T03:10:38.522603+0000","last_became_peered":"2023-12-01T03:10:38.522603+0000","last_unstale":"2023-12-01T03:10:39.014656+0000","last_undegraded":"2023-12-01T03:10:39.014656+0000","last_fullsized":"2023-12-01T03:10:39.014656+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:12.432439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996745+0000","last_change":"2023-12-01T03:10:38.995075+0000","last_active":"2023-12-01T03:10:38.996745+0000","last_peered":"2023-12-01T03:10:38.996745+0000","last_clean":"2023-12-01T03:10:38.996745+0000","last_became_active":"2023-12-01T03:10:38.994813+0000","last_became_peered":"2023-12-01T03:10:38.994813+0000","last_unstale":"2023-12-01T03:10:38.996745+0000","last_undegraded":"2023-12-01T03:10:38.996745+0000","last_fullsized":"2023-12-01T03:10:38.996745+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:22.679001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1a","version":"213'1","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.005161+0000","last_change":"2023-12-01T03:10:24.695315+0000","last_active":"2023-12-01T03:10:39.005161+0000","last_peered":"2023-12-01T03:10:39.005161+0000","last_clean":"2023-12-01T03:10:39.005161+0000","last_became_active":"2023-12-01T03:10:24.694274+0000","last_became_peered":"2023-12-01T03:10:24.694274+0000","last_unstale":"2023-12-01T03:10:39.005161+0000","last_undegraded":"2023-12-01T03:10:39.005161+0000","last_fullsized":"2023-12-01T03:10:39.005161+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:19.256208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003736+0000","last_change":"2023-12-01T03:10:25.188350+0000","last_active":"2023-12-01T03:10:39.003736+0000","last_peered":"2023-12-01T03:10:39.003736+0000","last_clean":"2023-12-01T03:10:39.003736+0000","last_became_active":"2023-12-01T03:10:25.187212+0000","last_became_peered":"2023-12-01T03:10:25.187212+0000","last_unstale":"2023-12-01T03:10:39.003736+0000","last_undegraded":"2023-12-01T03:10:39.003736+0000","last_fullsized":"2023-12-01T03:10:39.003736+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.981897+0000","last_change":"2023-12-01T03:10:38.521794+0000","last_active":"2023-12-01T03:10:38.981897+0000","last_peered":"2023-12-01T03:10:38.981897+0000","last_clean":"2023-12-01T03:10:38.981897+0000","last_became_active":"2023-12-01T03:10:38.521546+0000","last_became_peered":"2023-12-01T03:10:38.521546+0000","last_unstale":"2023-12-01T03:10:38.981897+0000","last_undegraded":"2023-12-01T03:10:38.981897+0000","last_fullsized":"2023-12-01T03:10:38.981897+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:17.386276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.010471+0000","last_change":"2023-12-01T03:10:39.005449+0000","last_active":"2023-12-01T03:10:39.010471+0000","last_peered":"2023-12-01T03:10:39.010471+0000","last_clean":"2023-12-01T03:10:39.010471+0000","last_became_active":"2023-12-01T03:10:39.004324+0000","last_became_peered":"2023-12-01T03:10:39.004324+0000","last_unstale":"2023-12-01T03:10:39.010471+0000","last_undegraded":"2023-12-01T03:10:39.010471+0000","last_fullsized":"2023-12-01T03:10:39.010471+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:33:51.590745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"213'1","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.005301+0000","last_change":"2023-12-01T03:10:24.695067+0000","last_active":"2023-12-01T03:10:39.005301+0000","last_peered":"2023-12-01T03:10:39.005301+0000","last_clean":"2023-12-01T03:10:39.005301+0000","last_became_active":"2023-12-01T03:10:24.694560+0000","last_became_peered":"2023-12-01T03:10:24.694560+0000","last_unstale":"2023-12-01T03:10:39.005301+0000","last_undegraded":"2023-12-01T03:10:39.005301+0000","last_fullsized":"2023-12-01T03:10:39.005301+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:11.746292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985232+0000","last_change":"2023-12-01T03:10:25.175874+0000","last_active":"2023-12-01T03:10:38.985232+0000","last_peered":"2023-12-01T03:10:38.985232+0000","last_clean":"2023-12-01T03:10:38.985232+0000","last_became_active":"2023-12-01T03:10:25.175584+0000","last_became_peered":"2023-12-01T03:10:25.175584+0000","last_unstale":"2023-12-01T03:10:38.985232+0000","last_undegraded":"2023-12-01T03:10:38.985232+0000","last_fullsized":"2023-12-01T03:10:38.985232+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:46.157610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.984005+0000","last_change":"2023-12-01T03:10:38.515775+0000","last_active":"2023-12-01T03:10:38.984005+0000","last_peered":"2023-12-01T03:10:38.984005+0000","last_clean":"2023-12-01T03:10:38.984005+0000","last_became_active":"2023-12-01T03:10:38.515021+0000","last_became_peered":"2023-12-01T03:10:38.515021+0000","last_unstale":"2023-12-01T03:10:38.984005+0000","last_undegraded":"2023-12-01T03:10:38.984005+0000","last_fullsized":"2023-12-01T03:10:38.984005+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:35.878213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003601+0000","last_change":"2023-12-01T03:10:39.000307+0000","last_active":"2023-12-01T03:10:39.003601+0000","last_peered":"2023-12-01T03:10:39.003601+0000","last_clean":"2023-12-01T03:10:39.003601+0000","last_became_active":"2023-12-01T03:10:38.997156+0000","last_became_peered":"2023-12-01T03:10:38.997156+0000","last_unstale":"2023-12-01T03:10:39.003601+0000","last_undegraded":"2023-12-01T03:10:39.003601+0000","last_fullsized":"2023-12-01T03:10:39.003601+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:59.987290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"217'1","reported_seq":12,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.995537+0000","last_change":"2023-12-01T03:10:38.513581+0000","last_active":"2023-12-01T03:10:38.995537+0000","last_peered":"2023-12-01T03:10:38.995537+0000","last_clean":"2023-12-01T03:10:38.995537+0000","last_became_active":"2023-12-01T03:10:38.513306+0000","last_became_peered":"2023-12-01T03:10:38.513306+0000","last_unstale":"2023-12-01T03:10:38.995537+0000","last_undegraded":"2023-12-01T03:10:38.995537+0000","last_fullsized":"2023-12-01T03:10:38.995537+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:13.957769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987848+0000","last_change":"2023-12-01T03:10:24.686931+0000","last_active":"2023-12-01T03:10:38.987848+0000","last_peered":"2023-12-01T03:10:38.987848+0000","last_clean":"2023-12-01T03:10:38.987848+0000","last_became_active":"2023-12-01T03:10:24.686642+0000","last_became_peered":"2023-12-01T03:10:24.686642+0000","last_unstale":"2023-12-01T03:10:38.987848+0000","last_undegraded":"2023-12-01T03:10:38.987848+0000","last_fullsized":"2023-12-01T03:10:38.987848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:44:11.026775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983478+0000","last_change":"2023-12-01T03:10:25.175999+0000","last_active":"2023-12-01T03:10:38.983478+0000","last_peered":"2023-12-01T03:10:38.983478+0000","last_clean":"2023-12-01T03:10:38.983478+0000","last_became_active":"2023-12-01T03:10:25.175764+0000","last_became_peered":"2023-12-01T03:10:25.175764+0000","last_unstale":"2023-12-01T03:10:38.983478+0000","last_undegraded":"2023-12-01T03:10:38.983478+0000","last_fullsized":"2023-12-01T03:10:38.983478+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:43.098776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012583+0000","last_change":"2023-12-01T03:10:39.009798+0000","last_active":"2023-12-01T03:10:39.012583+0000","last_peered":"2023-12-01T03:10:39.012583+0000","last_clean":"2023-12-01T03:10:39.012583+0000","last_became_active":"2023-12-01T03:10:39.009564+0000","last_became_peered":"2023-12-01T03:10:39.009564+0000","last_unstale":"2023-12-01T03:10:39.012583+0000","last_undegraded":"2023-12-01T03:10:39.012583+0000","last_fullsized":"2023-12-01T03:10:39.012583+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"217'4","reported_seq":15,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.011227+0000","last_change":"2023-12-01T03:10:38.520344+0000","last_active":"2023-12-01T03:10:39.011227+0000","last_peered":"2023-12-01T03:10:39.011227+0000","last_clean":"2023-12-01T03:10:39.011227+0000","last_became_active":"2023-12-01T03:10:38.520086+0000","last_became_peered":"2023-12-01T03:10:38.520086+0000","last_unstale":"2023-12-01T03:10:39.011227+0000","last_undegraded":"2023-12-01T03:10:39.011227+0000","last_fullsized":"2023-12-01T03:10:39.011227+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:01.616973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"213'2","reported_seq":17,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.004934+0000","last_change":"2023-12-01T03:10:24.700378+0000","last_active":"2023-12-01T03:10:39.004934+0000","last_peered":"2023-12-01T03:10:39.004934+0000","last_clean":"2023-12-01T03:10:39.004934+0000","last_became_active":"2023-12-01T03:10:24.700126+0000","last_became_peered":"2023-12-01T03:10:24.700126+0000","last_unstale":"2023-12-01T03:10:39.004934+0000","last_undegraded":"2023-12-01T03:10:39.004934+0000","last_fullsized":"2023-12-01T03:10:39.004934+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:37:25.396669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999828+0000","last_change":"2023-12-01T03:10:25.179131+0000","last_active":"2023-12-01T03:10:38.999828+0000","last_peered":"2023-12-01T03:10:38.999828+0000","last_clean":"2023-12-01T03:10:38.999828+0000","last_became_active":"2023-12-01T03:10:25.178708+0000","last_became_peered":"2023-12-01T03:10:25.178708+0000","last_unstale":"2023-12-01T03:10:38.999828+0000","last_undegraded":"2023-12-01T03:10:38.999828+0000","last_fullsized":"2023-12-01T03:10:38.999828+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:43.370389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.020373+0000","last_change":"2023-12-01T03:10:39.018720+0000","last_active":"2023-12-01T03:10:39.020373+0000","last_peered":"2023-12-01T03:10:39.020373+0000","last_clean":"2023-12-01T03:10:39.020373+0000","last_became_active":"2023-12-01T03:10:39.018223+0000","last_became_peered":"2023-12-01T03:10:39.018223+0000","last_unstale":"2023-12-01T03:10:39.020373+0000","last_undegraded":"2023-12-01T03:10:39.020373+0000","last_fullsized":"2023-12-01T03:10:39.020373+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:46:18.989087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1e","version":"213'1","reported_seq":16,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989529+0000","last_change":"2023-12-01T03:10:24.688812+0000","last_active":"2023-12-01T03:10:38.989529+0000","last_peered":"2023-12-01T03:10:38.989529+0000","last_clean":"2023-12-01T03:10:38.989529+0000","last_became_active":"2023-12-01T03:10:24.687577+0000","last_became_peered":"2023-12-01T03:10:24.687577+0000","last_unstale":"2023-12-01T03:10:38.989529+0000","last_undegraded":"2023-12-01T03:10:38.989529+0000","last_fullsized":"2023-12-01T03:10:38.989529+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:47:46.402722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.994955+0000","last_change":"2023-12-01T03:10:25.180335+0000","last_active":"2023-12-01T03:10:38.994955+0000","last_peered":"2023-12-01T03:10:38.994955+0000","last_clean":"2023-12-01T03:10:38.994955+0000","last_became_active":"2023-12-01T03:10:25.180020+0000","last_became_peered":"2023-12-01T03:10:25.180020+0000","last_unstale":"2023-12-01T03:10:38.994955+0000","last_undegraded":"2023-12-01T03:10:38.994955+0000","last_fullsized":"2023-12-01T03:10:38.994955+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"217'2","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.006339+0000","last_change":"2023-12-01T03:10:38.519061+0000","last_active":"2023-12-01T03:10:39.006339+0000","last_peered":"2023-12-01T03:10:39.006339+0000","last_clean":"2023-12-01T03:10:39.006339+0000","last_became_active":"2023-12-01T03:10:38.518091+0000","last_became_peered":"2023-12-01T03:10:38.518091+0000","last_unstale":"2023-12-01T03:10:39.006339+0000","last_undegraded":"2023-12-01T03:10:39.006339+0000","last_fullsized":"2023-12-01T03:10:39.006339+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:42.963999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.019819+0000","last_change":"2023-12-01T03:10:39.017358+0000","last_active":"2023-12-01T03:10:39.019819+0000","last_peered":"2023-12-01T03:10:39.019819+0000","last_clean":"2023-12-01T03:10:39.019819+0000","last_became_active":"2023-12-01T03:10:39.016704+0000","last_became_peered":"2023-12-01T03:10:39.016704+0000","last_unstale":"2023-12-01T03:10:39.019819+0000","last_undegraded":"2023-12-01T03:10:39.019819+0000","last_fullsized":"2023-12-01T03:10:39.019819+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"213'4","reported_seq":19,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996372+0000","last_change":"2023-12-01T03:10:24.697099+0000","last_active":"2023-12-01T03:10:38.996372+0000","last_peered":"2023-12-01T03:10:38.996372+0000","last_clean":"2023-12-01T03:10:38.996372+0000","last_became_active":"2023-12-01T03:10:24.696783+0000","last_became_peered":"2023-12-01T03:10:24.696783+0000","last_unstale":"2023-12-01T03:10:38.996372+0000","last_undegraded":"2023-12-01T03:10:38.996372+0000","last_fullsized":"2023-12-01T03:10:38.996372+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:23.622088+0000","last_clean_scrub_stamp":"2023-12-01T03:10:23.622088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:07.101180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007708+0000","last_change":"2023-12-01T03:10:25.191865+0000","last_active":"2023-12-01T03:10:39.007708+0000","last_peered":"2023-12-01T03:10:39.007708+0000","last_clean":"2023-12-01T03:10:39.007708+0000","last_became_active":"2023-12-01T03:10:25.189277+0000","last_became_peered":"2023-12-01T03:10:25.189277+0000","last_unstale":"2023-12-01T03:10:39.007708+0000","last_undegraded":"2023-12-01T03:10:39.007708+0000","last_fullsized":"2023-12-01T03:10:39.007708+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"217'2","reported_seq":13,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012020+0000","last_change":"2023-12-01T03:10:38.517851+0000","last_active":"2023-12-01T03:10:39.012020+0000","last_peered":"2023-12-01T03:10:39.012020+0000","last_clean":"2023-12-01T03:10:39.012020+0000","last_became_active":"2023-12-01T03:10:38.517507+0000","last_became_peered":"2023-12-01T03:10:38.517507+0000","last_unstale":"2023-12-01T03:10:39.012020+0000","last_undegraded":"2023-12-01T03:10:39.012020+0000","last_fullsized":"2023-12-01T03:10:39.012020+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:37.457016+0000","last_clean_scrub_stamp":"2023-12-01T03:10:37.457016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:22.592854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008442+0000","last_change":"2023-12-01T03:10:39.005982+0000","last_active":"2023-12-01T03:10:39.008442+0000","last_peered":"2023-12-01T03:10:39.008442+0000","last_clean":"2023-12-01T03:10:39.008442+0000","last_became_active":"2023-12-01T03:10:39.005729+0000","last_became_peered":"2023-12-01T03:10:39.005729+0000","last_unstale":"2023-12-01T03:10:39.008442+0000","last_undegraded":"2023-12-01T03:10:39.008442+0000","last_fullsized":"2023-12-01T03:10:39.008442+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:14:36.187885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996020+0000","last_change":"2023-12-01T03:10:25.185385+0000","last_active":"2023-12-01T03:10:38.996020+0000","last_peered":"2023-12-01T03:10:38.996020+0000","last_clean":"2023-12-01T03:10:38.996020+0000","last_became_active":"2023-12-01T03:10:25.185098+0000","last_became_peered":"2023-12-01T03:10:25.185098+0000","last_unstale":"2023-12-01T03:10:38.996020+0000","last_undegraded":"2023-12-01T03:10:38.996020+0000","last_fullsized":"2023-12-01T03:10:38.996020+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:01:45.233397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996396+0000","last_change":"2023-12-01T03:10:38.994136+0000","last_active":"2023-12-01T03:10:38.996396+0000","last_peered":"2023-12-01T03:10:38.996396+0000","last_clean":"2023-12-01T03:10:38.996396+0000","last_became_active":"2023-12-01T03:10:38.993889+0000","last_became_peered":"2023-12-01T03:10:38.993889+0000","last_unstale":"2023-12-01T03:10:38.996396+0000","last_undegraded":"2023-12-01T03:10:38.996396+0000","last_fullsized":"2023-12-01T03:10:38.996396+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:58:52.287938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.000359+0000","last_change":"2023-12-01T03:10:25.186126+0000","last_active":"2023-12-01T03:10:39.000359+0000","last_peered":"2023-12-01T03:10:39.000359+0000","last_clean":"2023-12-01T03:10:39.000359+0000","last_became_active":"2023-12-01T03:10:25.185741+0000","last_became_peered":"2023-12-01T03:10:25.185741+0000","last_unstale":"2023-12-01T03:10:39.000359+0000","last_undegraded":"2023-12-01T03:10:39.000359+0000","last_fullsized":"2023-12-01T03:10:39.000359+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:08.118254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008978+0000","last_change":"2023-12-01T03:10:39.006774+0000","last_active":"2023-12-01T03:10:39.008978+0000","last_peered":"2023-12-01T03:10:39.008978+0000","last_clean":"2023-12-01T03:10:39.008978+0000","last_became_active":"2023-12-01T03:10:39.006446+0000","last_became_peered":"2023-12-01T03:10:39.006446+0000","last_unstale":"2023-12-01T03:10:39.008978+0000","last_undegraded":"2023-12-01T03:10:39.008978+0000","last_fullsized":"2023-12-01T03:10:39.008978+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.985671+0000","last_change":"2023-12-01T03:10:25.175090+0000","last_active":"2023-12-01T03:10:38.985671+0000","last_peered":"2023-12-01T03:10:38.985671+0000","last_clean":"2023-12-01T03:10:38.985671+0000","last_became_active":"2023-12-01T03:10:25.174835+0000","last_became_peered":"2023-12-01T03:10:25.174835+0000","last_unstale":"2023-12-01T03:10:38.985671+0000","last_undegraded":"2023-12-01T03:10:38.985671+0000","last_fullsized":"2023-12-01T03:10:38.985671+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:05.658105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.017535+0000","last_change":"2023-12-01T03:10:39.014298+0000","last_active":"2023-12-01T03:10:39.017535+0000","last_peered":"2023-12-01T03:10:39.017535+0000","last_clean":"2023-12-01T03:10:39.017535+0000","last_became_active":"2023-12-01T03:10:39.013826+0000","last_became_peered":"2023-12-01T03:10:39.013826+0000","last_unstale":"2023-12-01T03:10:39.017535+0000","last_undegraded":"2023-12-01T03:10:39.017535+0000","last_fullsized":"2023-12-01T03:10:39.017535+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:37:52.701542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987378+0000","last_change":"2023-12-01T03:10:25.178098+0000","last_active":"2023-12-01T03:10:38.987378+0000","last_peered":"2023-12-01T03:10:38.987378+0000","last_clean":"2023-12-01T03:10:38.987378+0000","last_became_active":"2023-12-01T03:10:25.177031+0000","last_became_peered":"2023-12-01T03:10:25.177031+0000","last_unstale":"2023-12-01T03:10:38.987378+0000","last_undegraded":"2023-12-01T03:10:38.987378+0000","last_fullsized":"2023-12-01T03:10:38.987378+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:44:19.680171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.010642+0000","last_change":"2023-12-01T03:10:38.995152+0000","last_active":"2023-12-01T03:10:39.010642+0000","last_peered":"2023-12-01T03:10:39.010642+0000","last_clean":"2023-12-01T03:10:39.010642+0000","last_became_active":"2023-12-01T03:10:38.994160+0000","last_became_peered":"2023-12-01T03:10:38.994160+0000","last_unstale":"2023-12-01T03:10:39.010642+0000","last_undegraded":"2023-12-01T03:10:39.010642+0000","last_fullsized":"2023-12-01T03:10:39.010642+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.993367+0000","last_change":"2023-12-01T03:10:25.176284+0000","last_active":"2023-12-01T03:10:38.993367+0000","last_peered":"2023-12-01T03:10:38.993367+0000","last_clean":"2023-12-01T03:10:38.993367+0000","last_became_active":"2023-12-01T03:10:25.175494+0000","last_became_peered":"2023-12-01T03:10:25.175494+0000","last_unstale":"2023-12-01T03:10:38.993367+0000","last_undegraded":"2023-12-01T03:10:38.993367+0000","last_fullsized":"2023-12-01T03:10:38.993367+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:22:31.828247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008362+0000","last_change":"2023-12-01T03:10:39.006064+0000","last_active":"2023-12-01T03:10:39.008362+0000","last_peered":"2023-12-01T03:10:39.008362+0000","last_clean":"2023-12-01T03:10:39.008362+0000","last_became_active":"2023-12-01T03:10:39.005720+0000","last_became_peered":"2023-12-01T03:10:39.005720+0000","last_unstale":"2023-12-01T03:10:39.008362+0000","last_undegraded":"2023-12-01T03:10:39.008362+0000","last_fullsized":"2023-12-01T03:10:39.008362+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:22:02.628798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996850+0000","last_change":"2023-12-01T03:10:25.178072+0000","last_active":"2023-12-01T03:10:38.996850+0000","last_peered":"2023-12-01T03:10:38.996850+0000","last_clean":"2023-12-01T03:10:38.996850+0000","last_became_active":"2023-12-01T03:10:25.177778+0000","last_became_peered":"2023-12-01T03:10:25.177778+0000","last_unstale":"2023-12-01T03:10:38.996850+0000","last_undegraded":"2023-12-01T03:10:38.996850+0000","last_fullsized":"2023-12-01T03:10:38.996850+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:48.799119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.013237+0000","last_change":"2023-12-01T03:10:39.006374+0000","last_active":"2023-12-01T03:10:39.013237+0000","last_peered":"2023-12-01T03:10:39.013237+0000","last_clean":"2023-12-01T03:10:39.013237+0000","last_became_active":"2023-12-01T03:10:39.006094+0000","last_became_peered":"2023-12-01T03:10:39.006094+0000","last_unstale":"2023-12-01T03:10:39.013237+0000","last_undegraded":"2023-12-01T03:10:39.013237+0000","last_fullsized":"2023-12-01T03:10:39.013237+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989375+0000","last_change":"2023-12-01T03:10:25.181469+0000","last_active":"2023-12-01T03:10:38.989375+0000","last_peered":"2023-12-01T03:10:38.989375+0000","last_clean":"2023-12-01T03:10:38.989375+0000","last_became_active":"2023-12-01T03:10:25.180646+0000","last_became_peered":"2023-12-01T03:10:25.180646+0000","last_unstale":"2023-12-01T03:10:38.989375+0000","last_undegraded":"2023-12-01T03:10:38.989375+0000","last_fullsized":"2023-12-01T03:10:38.989375+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:53.001661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.008512+0000","last_change":"2023-12-01T03:10:39.006804+0000","last_active":"2023-12-01T03:10:39.008512+0000","last_peered":"2023-12-01T03:10:39.008512+0000","last_clean":"2023-12-01T03:10:39.008512+0000","last_became_active":"2023-12-01T03:10:39.006552+0000","last_became_peered":"2023-12-01T03:10:39.006552+0000","last_unstale":"2023-12-01T03:10:39.008512+0000","last_undegraded":"2023-12-01T03:10:39.008512+0000","last_fullsized":"2023-12-01T03:10:39.008512+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:32.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.992143+0000","last_change":"2023-12-01T03:10:25.179679+0000","last_active":"2023-12-01T03:10:38.992143+0000","last_peered":"2023-12-01T03:10:38.992143+0000","last_clean":"2023-12-01T03:10:38.992143+0000","last_became_active":"2023-12-01T03:10:25.179442+0000","last_became_peered":"2023-12-01T03:10:25.179442+0000","last_unstale":"2023-12-01T03:10:38.992143+0000","last_undegraded":"2023-12-01T03:10:38.992143+0000","last_fullsized":"2023-12-01T03:10:38.992143+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:57:25.728865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.013027+0000","last_change":"2023-12-01T03:10:39.010942+0000","last_active":"2023-12-01T03:10:39.013027+0000","last_peered":"2023-12-01T03:10:39.013027+0000","last_clean":"2023-12-01T03:10:39.013027+0000","last_became_active":"2023-12-01T03:10:39.010216+0000","last_became_peered":"2023-12-01T03:10:39.010216+0000","last_unstale":"2023-12-01T03:10:39.013027+0000","last_undegraded":"2023-12-01T03:10:39.013027+0000","last_fullsized":"2023-12-01T03:10:39.013027+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:09.328075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.995915+0000","last_change":"2023-12-01T03:10:25.175356+0000","last_active":"2023-12-01T03:10:38.995915+0000","last_peered":"2023-12-01T03:10:38.995915+0000","last_clean":"2023-12-01T03:10:38.995915+0000","last_became_active":"2023-12-01T03:10:25.175086+0000","last_became_peered":"2023-12-01T03:10:25.175086+0000","last_unstale":"2023-12-01T03:10:38.995915+0000","last_undegraded":"2023-12-01T03:10:38.995915+0000","last_fullsized":"2023-12-01T03:10:38.995915+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.000224+0000","last_change":"2023-12-01T03:10:38.998643+0000","last_active":"2023-12-01T03:10:39.000224+0000","last_peered":"2023-12-01T03:10:39.000224+0000","last_clean":"2023-12-01T03:10:39.000224+0000","last_became_active":"2023-12-01T03:10:38.998393+0000","last_became_peered":"2023-12-01T03:10:38.998393+0000","last_unstale":"2023-12-01T03:10:39.000224+0000","last_undegraded":"2023-12-01T03:10:39.000224+0000","last_fullsized":"2023-12-01T03:10:39.000224+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:38.199716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.993349+0000","last_change":"2023-12-01T03:10:25.181678+0000","last_active":"2023-12-01T03:10:38.993349+0000","last_peered":"2023-12-01T03:10:38.993349+0000","last_clean":"2023-12-01T03:10:38.993349+0000","last_became_active":"2023-12-01T03:10:25.181430+0000","last_became_peered":"2023-12-01T03:10:25.181430+0000","last_unstale":"2023-12-01T03:10:38.993349+0000","last_undegraded":"2023-12-01T03:10:38.993349+0000","last_fullsized":"2023-12-01T03:10:38.993349+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:22.959300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.012953+0000","last_change":"2023-12-01T03:10:39.011762+0000","last_active":"2023-12-01T03:10:39.012953+0000","last_peered":"2023-12-01T03:10:39.012953+0000","last_clean":"2023-12-01T03:10:39.012953+0000","last_became_active":"2023-12-01T03:10:39.011419+0000","last_became_peered":"2023-12-01T03:10:39.011419+0000","last_unstale":"2023-12-01T03:10:39.012953+0000","last_undegraded":"2023-12-01T03:10:39.012953+0000","last_fullsized":"2023-12-01T03:10:39.012953+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:57:07.895399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.005003+0000","last_change":"2023-12-01T03:10:25.188998+0000","last_active":"2023-12-01T03:10:39.005003+0000","last_peered":"2023-12-01T03:10:39.005003+0000","last_clean":"2023-12-01T03:10:39.005003+0000","last_became_active":"2023-12-01T03:10:25.188657+0000","last_became_peered":"2023-12-01T03:10:25.188657+0000","last_unstale":"2023-12-01T03:10:39.005003+0000","last_undegraded":"2023-12-01T03:10:39.005003+0000","last_fullsized":"2023-12-01T03:10:39.005003+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:23:58.752943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003091+0000","last_change":"2023-12-01T03:10:38.997151+0000","last_active":"2023-12-01T03:10:39.003091+0000","last_peered":"2023-12-01T03:10:39.003091+0000","last_clean":"2023-12-01T03:10:39.003091+0000","last_became_active":"2023-12-01T03:10:38.989885+0000","last_became_peered":"2023-12-01T03:10:38.989885+0000","last_unstale":"2023-12-01T03:10:39.003091+0000","last_undegraded":"2023-12-01T03:10:39.003091+0000","last_fullsized":"2023-12-01T03:10:39.003091+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:25:51.393448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.994238+0000","last_change":"2023-12-01T03:10:25.174568+0000","last_active":"2023-12-01T03:10:38.994238+0000","last_peered":"2023-12-01T03:10:38.994238+0000","last_clean":"2023-12-01T03:10:38.994238+0000","last_became_active":"2023-12-01T03:10:25.174229+0000","last_became_peered":"2023-12-01T03:10:25.174229+0000","last_unstale":"2023-12-01T03:10:38.994238+0000","last_undegraded":"2023-12-01T03:10:38.994238+0000","last_fullsized":"2023-12-01T03:10:38.994238+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:13.528118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.019795+0000","last_change":"2023-12-01T03:10:39.017406+0000","last_active":"2023-12-01T03:10:39.019795+0000","last_peered":"2023-12-01T03:10:39.019795+0000","last_clean":"2023-12-01T03:10:39.019795+0000","last_became_active":"2023-12-01T03:10:39.016547+0000","last_became_peered":"2023-12-01T03:10:39.016547+0000","last_unstale":"2023-12-01T03:10:39.019795+0000","last_undegraded":"2023-12-01T03:10:39.019795+0000","last_fullsized":"2023-12-01T03:10:39.019795+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:19:19.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.009300+0000","last_change":"2023-12-01T03:10:25.189264+0000","last_active":"2023-12-01T03:10:39.009300+0000","last_peered":"2023-12-01T03:10:39.009300+0000","last_clean":"2023-12-01T03:10:39.009300+0000","last_became_active":"2023-12-01T03:10:25.189007+0000","last_became_peered":"2023-12-01T03:10:25.189007+0000","last_unstale":"2023-12-01T03:10:39.009300+0000","last_undegraded":"2023-12-01T03:10:39.009300+0000","last_fullsized":"2023-12-01T03:10:39.009300+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:14.783134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.015245+0000","last_change":"2023-12-01T03:10:39.013208+0000","last_active":"2023-12-01T03:10:39.015245+0000","last_peered":"2023-12-01T03:10:39.015245+0000","last_clean":"2023-12-01T03:10:39.015245+0000","last_became_active":"2023-12-01T03:10:39.012491+0000","last_became_peered":"2023-12-01T03:10:39.012491+0000","last_unstale":"2023-12-01T03:10:39.015245+0000","last_undegraded":"2023-12-01T03:10:39.015245+0000","last_fullsized":"2023-12-01T03:10:39.015245+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:38:07.828693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996751+0000","last_change":"2023-12-01T03:10:25.180992+0000","last_active":"2023-12-01T03:10:38.996751+0000","last_peered":"2023-12-01T03:10:38.996751+0000","last_clean":"2023-12-01T03:10:38.996751+0000","last_became_active":"2023-12-01T03:10:25.180712+0000","last_became_peered":"2023-12-01T03:10:25.180712+0000","last_unstale":"2023-12-01T03:10:38.996751+0000","last_undegraded":"2023-12-01T03:10:38.996751+0000","last_fullsized":"2023-12-01T03:10:38.996751+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:43.827830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.019841+0000","last_change":"2023-12-01T03:10:39.017405+0000","last_active":"2023-12-01T03:10:39.019841+0000","last_peered":"2023-12-01T03:10:39.019841+0000","last_clean":"2023-12-01T03:10:39.019841+0000","last_became_active":"2023-12-01T03:10:39.017058+0000","last_became_peered":"2023-12-01T03:10:39.017058+0000","last_unstale":"2023-12-01T03:10:39.019841+0000","last_undegraded":"2023-12-01T03:10:39.019841+0000","last_fullsized":"2023-12-01T03:10:39.019841+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:58:42.639310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.982053+0000","last_change":"2023-12-01T03:10:25.182676+0000","last_active":"2023-12-01T03:10:38.982053+0000","last_peered":"2023-12-01T03:10:38.982053+0000","last_clean":"2023-12-01T03:10:38.982053+0000","last_became_active":"2023-12-01T03:10:25.182429+0000","last_became_peered":"2023-12-01T03:10:25.182429+0000","last_unstale":"2023-12-01T03:10:38.982053+0000","last_undegraded":"2023-12-01T03:10:38.982053+0000","last_fullsized":"2023-12-01T03:10:38.982053+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:51:09.855540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.020393+0000","last_change":"2023-12-01T03:10:39.018941+0000","last_active":"2023-12-01T03:10:39.020393+0000","last_peered":"2023-12-01T03:10:39.020393+0000","last_clean":"2023-12-01T03:10:39.020393+0000","last_became_active":"2023-12-01T03:10:39.018597+0000","last_became_peered":"2023-12-01T03:10:39.018597+0000","last_unstale":"2023-12-01T03:10:39.020393+0000","last_undegraded":"2023-12-01T03:10:39.020393+0000","last_fullsized":"2023-12-01T03:10:39.020393+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:54.829130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986606+0000","last_change":"2023-12-01T03:10:25.173100+0000","last_active":"2023-12-01T03:10:38.986606+0000","last_peered":"2023-12-01T03:10:38.986606+0000","last_clean":"2023-12-01T03:10:38.986606+0000","last_became_active":"2023-12-01T03:10:25.172808+0000","last_became_peered":"2023-12-01T03:10:25.172808+0000","last_unstale":"2023-12-01T03:10:38.986606+0000","last_undegraded":"2023-12-01T03:10:38.986606+0000","last_fullsized":"2023-12-01T03:10:38.986606+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:16:25.687973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.000690+0000","last_change":"2023-12-01T03:10:38.998980+0000","last_active":"2023-12-01T03:10:39.000690+0000","last_peered":"2023-12-01T03:10:39.000690+0000","last_clean":"2023-12-01T03:10:39.000690+0000","last_became_active":"2023-12-01T03:10:38.998712+0000","last_became_peered":"2023-12-01T03:10:38.998712+0000","last_unstale":"2023-12-01T03:10:39.000690+0000","last_undegraded":"2023-12-01T03:10:39.000690+0000","last_fullsized":"2023-12-01T03:10:39.000690+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:48.887816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.998595+0000","last_change":"2023-12-01T03:10:25.189550+0000","last_active":"2023-12-01T03:10:38.998595+0000","last_peered":"2023-12-01T03:10:38.998595+0000","last_clean":"2023-12-01T03:10:38.998595+0000","last_became_active":"2023-12-01T03:10:25.189001+0000","last_became_peered":"2023-12-01T03:10:25.189001+0000","last_unstale":"2023-12-01T03:10:38.998595+0000","last_undegraded":"2023-12-01T03:10:38.998595+0000","last_fullsized":"2023-12-01T03:10:38.998595+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:26:46.190565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.003543+0000","last_change":"2023-12-01T03:10:38.997977+0000","last_active":"2023-12-01T03:10:39.003543+0000","last_peered":"2023-12-01T03:10:39.003543+0000","last_clean":"2023-12-01T03:10:39.003543+0000","last_became_active":"2023-12-01T03:10:38.997683+0000","last_became_peered":"2023-12-01T03:10:38.997683+0000","last_unstale":"2023-12-01T03:10:39.003543+0000","last_undegraded":"2023-12-01T03:10:39.003543+0000","last_fullsized":"2023-12-01T03:10:39.003543+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:27:00.020245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.973897+0000","last_change":"2023-12-01T03:10:25.170590+0000","last_active":"2023-12-01T03:10:38.973897+0000","last_peered":"2023-12-01T03:10:38.973897+0000","last_clean":"2023-12-01T03:10:38.973897+0000","last_became_active":"2023-12-01T03:10:25.170334+0000","last_became_peered":"2023-12-01T03:10:25.170334+0000","last_unstale":"2023-12-01T03:10:38.973897+0000","last_undegraded":"2023-12-01T03:10:38.973897+0000","last_fullsized":"2023-12-01T03:10:38.973897+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:31:35.058136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.009182+0000","last_change":"2023-12-01T03:10:39.005096+0000","last_active":"2023-12-01T03:10:39.009182+0000","last_peered":"2023-12-01T03:10:39.009182+0000","last_clean":"2023-12-01T03:10:39.009182+0000","last_became_active":"2023-12-01T03:10:39.004792+0000","last_became_peered":"2023-12-01T03:10:39.004792+0000","last_unstale":"2023-12-01T03:10:39.009182+0000","last_undegraded":"2023-12-01T03:10:39.009182+0000","last_fullsized":"2023-12-01T03:10:39.009182+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:23.743010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.986339+0000","last_change":"2023-12-01T03:10:25.173081+0000","last_active":"2023-12-01T03:10:38.986339+0000","last_peered":"2023-12-01T03:10:38.986339+0000","last_clean":"2023-12-01T03:10:38.986339+0000","last_became_active":"2023-12-01T03:10:25.172845+0000","last_became_peered":"2023-12-01T03:10:25.172845+0000","last_unstale":"2023-12-01T03:10:38.986339+0000","last_undegraded":"2023-12-01T03:10:38.986339+0000","last_fullsized":"2023-12-01T03:10:38.986339+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:18.173873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.014218+0000","last_change":"2023-12-01T03:10:39.011331+0000","last_active":"2023-12-01T03:10:39.014218+0000","last_peered":"2023-12-01T03:10:39.014218+0000","last_clean":"2023-12-01T03:10:39.014218+0000","last_became_active":"2023-12-01T03:10:39.011009+0000","last_became_peered":"2023-12-01T03:10:39.011009+0000","last_unstale":"2023-12-01T03:10:39.014218+0000","last_undegraded":"2023-12-01T03:10:39.014218+0000","last_fullsized":"2023-12-01T03:10:39.014218+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:24:41.123796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.975137+0000","last_change":"2023-12-01T03:10:25.178225+0000","last_active":"2023-12-01T03:10:38.975137+0000","last_peered":"2023-12-01T03:10:38.975137+0000","last_clean":"2023-12-01T03:10:38.975137+0000","last_became_active":"2023-12-01T03:10:25.177541+0000","last_became_peered":"2023-12-01T03:10:25.177541+0000","last_unstale":"2023-12-01T03:10:38.975137+0000","last_undegraded":"2023-12-01T03:10:38.975137+0000","last_fullsized":"2023-12-01T03:10:38.975137+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:22:18.470068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.019820+0000","last_change":"2023-12-01T03:10:39.018239+0000","last_active":"2023-12-01T03:10:39.019820+0000","last_peered":"2023-12-01T03:10:39.019820+0000","last_clean":"2023-12-01T03:10:39.019820+0000","last_became_active":"2023-12-01T03:10:39.017956+0000","last_became_peered":"2023-12-01T03:10:39.017956+0000","last_unstale":"2023-12-01T03:10:39.019820+0000","last_undegraded":"2023-12-01T03:10:39.019820+0000","last_fullsized":"2023-12-01T03:10:39.019820+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:30:27.744078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.987497+0000","last_change":"2023-12-01T03:10:25.172316+0000","last_active":"2023-12-01T03:10:38.987497+0000","last_peered":"2023-12-01T03:10:38.987497+0000","last_clean":"2023-12-01T03:10:38.987497+0000","last_became_active":"2023-12-01T03:10:25.169042+0000","last_became_peered":"2023-12-01T03:10:25.169042+0000","last_unstale":"2023-12-01T03:10:38.987497+0000","last_undegraded":"2023-12-01T03:10:38.987497+0000","last_fullsized":"2023-12-01T03:10:38.987497+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:30.146144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.989770+0000","last_change":"2023-12-01T03:10:38.987305+0000","last_active":"2023-12-01T03:10:38.989770+0000","last_peered":"2023-12-01T03:10:38.989770+0000","last_clean":"2023-12-01T03:10:38.989770+0000","last_became_active":"2023-12-01T03:10:38.987037+0000","last_became_peered":"2023-12-01T03:10:38.987037+0000","last_unstale":"2023-12-01T03:10:38.989770+0000","last_undegraded":"2023-12-01T03:10:38.989770+0000","last_fullsized":"2023-12-01T03:10:38.989770+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:49.632963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.975696+0000","last_change":"2023-12-01T03:10:25.181052+0000","last_active":"2023-12-01T03:10:38.975696+0000","last_peered":"2023-12-01T03:10:38.975696+0000","last_clean":"2023-12-01T03:10:38.975696+0000","last_became_active":"2023-12-01T03:10:25.180810+0000","last_became_peered":"2023-12-01T03:10:25.180810+0000","last_unstale":"2023-12-01T03:10:38.975696+0000","last_undegraded":"2023-12-01T03:10:38.975696+0000","last_fullsized":"2023-12-01T03:10:38.975696+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:35.663128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.013053+0000","last_change":"2023-12-01T03:10:39.011983+0000","last_active":"2023-12-01T03:10:39.013053+0000","last_peered":"2023-12-01T03:10:39.013053+0000","last_clean":"2023-12-01T03:10:39.013053+0000","last_became_active":"2023-12-01T03:10:39.011721+0000","last_became_peered":"2023-12-01T03:10:39.011721+0000","last_unstale":"2023-12-01T03:10:39.013053+0000","last_undegraded":"2023-12-01T03:10:39.013053+0000","last_fullsized":"2023-12-01T03:10:39.013053+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:04:39.199929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.993611+0000","last_change":"2023-12-01T03:10:25.177089+0000","last_active":"2023-12-01T03:10:38.993611+0000","last_peered":"2023-12-01T03:10:38.993611+0000","last_clean":"2023-12-01T03:10:38.993611+0000","last_became_active":"2023-12-01T03:10:25.176742+0000","last_became_peered":"2023-12-01T03:10:25.176742+0000","last_unstale":"2023-12-01T03:10:38.993611+0000","last_undegraded":"2023-12-01T03:10:38.993611+0000","last_fullsized":"2023-12-01T03:10:38.993611+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.016878+0000","last_change":"2023-12-01T03:10:39.016041+0000","last_active":"2023-12-01T03:10:39.016878+0000","last_peered":"2023-12-01T03:10:39.016878+0000","last_clean":"2023-12-01T03:10:39.016878+0000","last_became_active":"2023-12-01T03:10:39.012071+0000","last_became_peered":"2023-12-01T03:10:39.012071+0000","last_unstale":"2023-12-01T03:10:39.016878+0000","last_undegraded":"2023-12-01T03:10:39.016878+0000","last_fullsized":"2023-12-01T03:10:39.016878+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:19.824447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.992054+0000","last_change":"2023-12-01T03:10:25.172560+0000","last_active":"2023-12-01T03:10:38.992054+0000","last_peered":"2023-12-01T03:10:38.992054+0000","last_clean":"2023-12-01T03:10:38.992054+0000","last_became_active":"2023-12-01T03:10:25.172005+0000","last_became_peered":"2023-12-01T03:10:25.172005+0000","last_unstale":"2023-12-01T03:10:38.992054+0000","last_undegraded":"2023-12-01T03:10:38.992054+0000","last_fullsized":"2023-12-01T03:10:38.992054+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:48:44.541671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.010327+0000","last_change":"2023-12-01T03:10:39.004021+0000","last_active":"2023-12-01T03:10:39.010327+0000","last_peered":"2023-12-01T03:10:39.010327+0000","last_clean":"2023-12-01T03:10:39.010327+0000","last_became_active":"2023-12-01T03:10:39.000850+0000","last_became_peered":"2023-12-01T03:10:39.000850+0000","last_unstale":"2023-12-01T03:10:39.010327+0000","last_undegraded":"2023-12-01T03:10:39.010327+0000","last_fullsized":"2023-12-01T03:10:39.010327+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:10:20.781162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.995816+0000","last_change":"2023-12-01T03:10:25.185623+0000","last_active":"2023-12-01T03:10:38.995816+0000","last_peered":"2023-12-01T03:10:38.995816+0000","last_clean":"2023-12-01T03:10:38.995816+0000","last_became_active":"2023-12-01T03:10:25.185311+0000","last_became_peered":"2023-12-01T03:10:25.185311+0000","last_unstale":"2023-12-01T03:10:38.995816+0000","last_undegraded":"2023-12-01T03:10:38.995816+0000","last_fullsized":"2023-12-01T03:10:38.995816+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:13:13.951061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.019737+0000","last_change":"2023-12-01T03:10:39.016613+0000","last_active":"2023-12-01T03:10:39.019737+0000","last_peered":"2023-12-01T03:10:39.019737+0000","last_clean":"2023-12-01T03:10:39.019737+0000","last_became_active":"2023-12-01T03:10:39.016189+0000","last_became_peered":"2023-12-01T03:10:39.016189+0000","last_unstale":"2023-12-01T03:10:39.019737+0000","last_undegraded":"2023-12-01T03:10:39.019737+0000","last_fullsized":"2023-12-01T03:10:39.019737+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:38.801763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.980354+0000","last_change":"2023-12-01T03:10:25.186300+0000","last_active":"2023-12-01T03:10:38.980354+0000","last_peered":"2023-12-01T03:10:38.980354+0000","last_clean":"2023-12-01T03:10:38.980354+0000","last_became_active":"2023-12-01T03:10:25.185030+0000","last_became_peered":"2023-12-01T03:10:25.185030+0000","last_unstale":"2023-12-01T03:10:38.980354+0000","last_undegraded":"2023-12-01T03:10:38.980354+0000","last_fullsized":"2023-12-01T03:10:38.980354+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:20:36.072429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.014958+0000","last_change":"2023-12-01T03:10:39.010156+0000","last_active":"2023-12-01T03:10:39.014958+0000","last_peered":"2023-12-01T03:10:39.014958+0000","last_clean":"2023-12-01T03:10:39.014958+0000","last_became_active":"2023-12-01T03:10:39.008161+0000","last_became_peered":"2023-12-01T03:10:39.008161+0000","last_unstale":"2023-12-01T03:10:39.014958+0000","last_undegraded":"2023-12-01T03:10:39.014958+0000","last_fullsized":"2023-12-01T03:10:39.014958+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:58.664911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.984764+0000","last_change":"2023-12-01T03:10:25.182648+0000","last_active":"2023-12-01T03:10:38.984764+0000","last_peered":"2023-12-01T03:10:38.984764+0000","last_clean":"2023-12-01T03:10:38.984764+0000","last_became_active":"2023-12-01T03:10:25.182343+0000","last_became_peered":"2023-12-01T03:10:25.182343+0000","last_unstale":"2023-12-01T03:10:38.984764+0000","last_undegraded":"2023-12-01T03:10:38.984764+0000","last_fullsized":"2023-12-01T03:10:38.984764+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:55.408381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.011206+0000","last_change":"2023-12-01T03:10:39.005646+0000","last_active":"2023-12-01T03:10:39.011206+0000","last_peered":"2023-12-01T03:10:39.011206+0000","last_clean":"2023-12-01T03:10:39.011206+0000","last_became_active":"2023-12-01T03:10:39.005198+0000","last_became_peered":"2023-12-01T03:10:39.005198+0000","last_unstale":"2023-12-01T03:10:39.011206+0000","last_undegraded":"2023-12-01T03:10:39.011206+0000","last_fullsized":"2023-12-01T03:10:39.011206+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.001701+0000","last_change":"2023-12-01T03:10:25.191253+0000","last_active":"2023-12-01T03:10:39.001701+0000","last_peered":"2023-12-01T03:10:39.001701+0000","last_clean":"2023-12-01T03:10:39.001701+0000","last_became_active":"2023-12-01T03:10:25.188485+0000","last_became_peered":"2023-12-01T03:10:25.188485+0000","last_unstale":"2023-12-01T03:10:39.001701+0000","last_undegraded":"2023-12-01T03:10:39.001701+0000","last_fullsized":"2023-12-01T03:10:39.001701+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:03.118823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.993068+0000","last_change":"2023-12-01T03:10:38.992005+0000","last_active":"2023-12-01T03:10:38.993068+0000","last_peered":"2023-12-01T03:10:38.993068+0000","last_clean":"2023-12-01T03:10:38.993068+0000","last_became_active":"2023-12-01T03:10:38.991740+0000","last_became_peered":"2023-12-01T03:10:38.991740+0000","last_unstale":"2023-12-01T03:10:38.993068+0000","last_undegraded":"2023-12-01T03:10:38.993068+0000","last_fullsized":"2023-12-01T03:10:38.993068+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:53.169568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.999753+0000","last_change":"2023-12-01T03:10:25.171460+0000","last_active":"2023-12-01T03:10:38.999753+0000","last_peered":"2023-12-01T03:10:38.999753+0000","last_clean":"2023-12-01T03:10:38.999753+0000","last_became_active":"2023-12-01T03:10:25.171148+0000","last_became_peered":"2023-12-01T03:10:25.171148+0000","last_unstale":"2023-12-01T03:10:38.999753+0000","last_undegraded":"2023-12-01T03:10:38.999753+0000","last_fullsized":"2023-12-01T03:10:38.999753+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:01:37.278740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.009106+0000","last_change":"2023-12-01T03:10:39.003963+0000","last_active":"2023-12-01T03:10:39.009106+0000","last_peered":"2023-12-01T03:10:39.009106+0000","last_clean":"2023-12-01T03:10:39.009106+0000","last_became_active":"2023-12-01T03:10:39.003567+0000","last_became_peered":"2023-12-01T03:10:39.003567+0000","last_unstale":"2023-12-01T03:10:39.009106+0000","last_undegraded":"2023-12-01T03:10:39.009106+0000","last_fullsized":"2023-12-01T03:10:39.009106+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:07:32.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.996650+0000","last_change":"2023-12-01T03:10:25.181720+0000","last_active":"2023-12-01T03:10:38.996650+0000","last_peered":"2023-12-01T03:10:38.996650+0000","last_clean":"2023-12-01T03:10:38.996650+0000","last_became_active":"2023-12-01T03:10:25.180144+0000","last_became_peered":"2023-12-01T03:10:25.180144+0000","last_unstale":"2023-12-01T03:10:38.996650+0000","last_undegraded":"2023-12-01T03:10:38.996650+0000","last_fullsized":"2023-12-01T03:10:38.996650+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:08:53.999589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.007029+0000","last_change":"2023-12-01T03:10:39.004848+0000","last_active":"2023-12-01T03:10:39.007029+0000","last_peered":"2023-12-01T03:10:39.007029+0000","last_clean":"2023-12-01T03:10:39.007029+0000","last_became_active":"2023-12-01T03:10:39.004604+0000","last_became_peered":"2023-12-01T03:10:39.004604+0000","last_unstale":"2023-12-01T03:10:39.007029+0000","last_undegraded":"2023-12-01T03:10:39.007029+0000","last_fullsized":"2023-12-01T03:10:39.007029+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:33:44.253867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.983363+0000","last_change":"2023-12-01T03:10:25.181088+0000","last_active":"2023-12-01T03:10:38.983363+0000","last_peered":"2023-12-01T03:10:38.983363+0000","last_clean":"2023-12-01T03:10:38.983363+0000","last_became_active":"2023-12-01T03:10:25.180741+0000","last_became_peered":"2023-12-01T03:10:25.180741+0000","last_unstale":"2023-12-01T03:10:38.983363+0000","last_undegraded":"2023-12-01T03:10:38.983363+0000","last_fullsized":"2023-12-01T03:10:38.983363+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:26.321285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:39.020800+0000","last_change":"2023-12-01T03:10:39.019550+0000","last_active":"2023-12-01T03:10:39.020800+0000","last_peered":"2023-12-01T03:10:39.020800+0000","last_clean":"2023-12-01T03:10:39.020800+0000","last_became_active":"2023-12-01T03:10:39.019194+0000","last_became_peered":"2023-12-01T03:10:39.019194+0000","last_unstale":"2023-12-01T03:10:39.020800+0000","last_undegraded":"2023-12-01T03:10:39.020800+0000","last_fullsized":"2023-12-01T03:10:39.020800+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:38.468841+0000","last_clean_scrub_stamp":"2023-12-01T03:10:38.468841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:46:59.546982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":14,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-12-01T03:10:38.976177+0000","last_change":"2023-12-01T03:10:25.171115+0000","last_active":"2023-12-01T03:10:38.976177+0000","last_peered":"2023-12-01T03:10:38.976177+0000","last_clean":"2023-12-01T03:10:38.976177+0000","last_became_active":"2023-12-01T03:10:25.170871+0000","last_became_peered":"2023-12-01T03:10:25.170871+0000","last_unstale":"2023-12-01T03:10:38.976177+0000","last_undegraded":"2023-12-01T03:10:38.976177+0000","last_fullsized":"2023-12-01T03:10:38.976177+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:10:24.639162+0000","last_clean_scrub_stamp":"2023-12-01T03:10:24.639162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:34:24.370036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575980,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59976,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311864,"statfs":{"total":96636764160,"available":96575348736,"internally_reserved":0,"allocated":2752512,"data_stored":1336858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0780000000000001,"15min":1.0820000000000001},"min":{"1min":0.78200000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5069999999999999,"5min":1.6000000000000001,"15min":1.98},"last":1.052},{"interface":"front","average":{"1min":1.093,"5min":1.1120000000000001,"15min":1.1080000000000001},"min":{"1min":0.67800000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.556,"5min":1.7,"15min":1.7},"last":0.83199999999999996}]},{"osd":1,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2949999999999999,"15min":1.2889999999999999},"min":{"1min":0.80000000000000004,"5min":0.745,"15min":0.73099999999999998},"max":{"1min":1.603,"5min":2,"15min":2.1419999999999999},"last":1.222},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3149999999999999,"15min":1.3320000000000001},"min":{"1min":1.137,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.514,"5min":1.8169999999999999,"15min":1.8700000000000001},"last":1.325}]},{"osd":2,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.157,"15min":1.1719999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.476,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.26,"15min":1.254},"min":{"1min":1.0549999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6020000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.643}]},{"osd":3,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2529999999999999,"15min":1.3049999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.6559999999999999,"5min":1.712,"15min":2.431},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.415,"15min":1.4199999999999999},"min":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.92900000000000005},"max":{"1min":1.5920000000000001,"5min":1.879,"15min":2.198},"last":1.7529999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3140000000000001,"15min":1.3400000000000001},"min":{"1min":1.006,"5min":0.82599999999999996,"15min":0.81200000000000006},"max":{"1min":1.901,"5min":1.901,"15min":2.5899999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":1.1240000000000001,"5min":0.92800000000000005,"15min":0.79500000000000004},"max":{"1min":1.9690000000000001,"5min":2.3279999999999998,"15min":2.569},"last":1.8799999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3009999999999999,"15min":1.3149999999999999},"min":{"1min":0.93600000000000005,"5min":0.85999999999999999,"15min":0.84599999999999997},"max":{"1min":2.0099999999999998,"5min":2.016,"15min":2.7269999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5800000000000001,"15min":1.589},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":0.93200000000000005},"max":{"1min":2.0569999999999999,"5min":2.206,"15min":2.484},"last":2.0470000000000002}]},{"osd":6,"last update":"Fri Dec 1 03:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.4359999999999999,"15min":1.446},"min":{"1min":1.0229999999999999,"5min":0.98599999999999999,"15min":0.82599999999999996},"max":{"1min":1.921,"5min":2.2789999999999999,"15min":2.532},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4370000000000001,"15min":1.462},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":1.8819999999999999,"15min":2.762},"last":1.8200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575981,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41912,"kb_used_data":2672,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329928,"statfs":{"total":96636764160,"available":96593846272,"internally_reserved":0,"allocated":2736128,"data_stored":1334079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0589999999999999,"15min":1.113},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.4039999999999999,"5min":1.6639999999999999,"15min":1.9219999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1160000000000001,"15min":1.1659999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.623,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":0.94299999999999995}]},{"osd":1,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.218},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.976,"5min":1.976,"15min":2.0449999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1950000000000001,"15min":1.2210000000000001},"min":{"1min":0.627,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.694,"5min":1.694,"15min":1.835},"last":1.1579999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.163,"15min":1.2090000000000001},"min":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.742,"5min":1.742,"15min":1.976},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2629999999999999,"15min":1.2849999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.2270000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2849999999999999,"15min":1.319},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8100000000000001,"5min":1.927,"15min":2.0289999999999999},"last":1.278},{"interface":"front","average":{"1min":1.405,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.464}]},{"osd":4,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2130000000000001,"15min":1.262},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.367,"15min":1.3540000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.5720000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3360000000000001,"15min":1.4039999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4790000000000001,"15min":1.484},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.6779999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4890000000000001,"15min":1.5249999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.778},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.556,"15min":1.579},"min":{"1min":0.95499999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2999999999999998,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.744}]}]},{"osd":5,"up_from":13,"seq":55834575980,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48268,"kb_used_data":3268,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323572,"statfs":{"total":96636764160,"available":96587337728,"internally_reserved":0,"allocated":3346432,"data_stored":1925619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.125},"min":{"1min":0.82299999999999995,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.548,"5min":1.776,"15min":1.8899999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.1639999999999999,"15min":1.163},"min":{"1min":0.89800000000000002,"5min":0.69399999999999995,"15min":0.67300000000000004},"max":{"1min":1.619,"5min":1.752,"15min":1.8009999999999999},"last":1.0289999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.133,"15min":1.1539999999999999},"min":{"1min":0.94699999999999995,"5min":0.81200000000000006,"15min":0.79900000000000004},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.696},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.204,"15min":1.2350000000000001},"min":{"1min":0.88,"5min":0.79500000000000004,"15min":0.60899999999999999},"max":{"1min":1.619,"5min":1.6659999999999999,"15min":1.7649999999999999},"last":1.4099999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2390000000000001,"15min":1.246},"min":{"1min":0.93700000000000006,"5min":0.84499999999999997,"15min":0.81399999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.774},"last":1.569},{"interface":"front","average":{"1min":1.284,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.93899999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.502,"5min":1.899,"15min":1.9219999999999999},"last":1.0820000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.306,"15min":1.3200000000000001},"min":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.93100000000000005},"max":{"1min":1.7929999999999999,"5min":1.8979999999999999,"15min":1.994},"last":1.466},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.343,"15min":1.363},"min":{"1min":1.02,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.724,"5min":1.97,"15min":2.1859999999999999},"last":1.202}]},{"osd":4,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2789999999999999,"15min":1.254},"min":{"1min":0.96199999999999997,"5min":0.78000000000000003,"15min":0.74099999999999999},"max":{"1min":1.726,"5min":1.726,"15min":2.6040000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.399,"15min":1.385},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.84599999999999997},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.931},"last":1.2669999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3879999999999999,"15min":1.3759999999999999},"min":{"1min":0.97599999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.9279999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.427,"15min":1.419},"min":{"1min":1.222,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":1.756,"5min":1.9219999999999999,"15min":2.577},"last":1.1599999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.5349999999999999,"15min":1.5369999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.962,"5min":2.0790000000000002,"15min":3.1709999999999998},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.575,"15min":1.5740000000000001},"min":{"1min":1.3720000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.831,"5min":2.101,"15min":2.8740000000000001},"last":1.51}]}]},{"osd":4,"up_from":13,"seq":55834575980,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51768,"kb_used_data":2672,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320072,"statfs":{"total":96636764160,"available":96583753728,"internally_reserved":0,"allocated":2736128,"data_stored":1334043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.147,"15min":1.1339999999999999},"min":{"1min":0.73099999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.4850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.21,"15min":1.1990000000000001},"min":{"1min":0.82099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5840000000000001,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":0.93799999999999994}]},{"osd":1,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.242,"15min":1.2589999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.613,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1930000000000001,"15min":1.1890000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.82599999999999996},"max":{"1min":1.6080000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.0900000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.3240000000000001,"15min":1.2470000000000001},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.573,"5min":11.311,"15min":11.311},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.329,"15min":1.341},"min":{"1min":1.0680000000000001,"5min":1.024,"15min":0.94099999999999995},"max":{"1min":1.647,"5min":1.9379999999999999,"15min":1.9450000000000001},"last":1.171}]},{"osd":3,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3100000000000001,"15min":1.29},"min":{"1min":1.0529999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7749999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.2},{"interface":"front","average":{"1min":1.343,"5min":1.365,"15min":1.3380000000000001},"min":{"1min":0.99199999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.599,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.29}]},{"osd":5,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.357,"15min":1.329},"min":{"1min":0.94799999999999995,"5min":0.91200000000000003,"15min":0.755},"max":{"1min":1.615,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.379,"15min":1.3120000000000001},"min":{"1min":0.94599999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7150000000000001,"5min":3.71,"15min":3.71},"last":0.995}]},{"osd":6,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3740000000000001,"15min":1.375},"min":{"1min":1.095,"5min":0.745,"15min":0.745},"max":{"1min":1.609,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.514,"5min":1.4850000000000001,"15min":1.48},"min":{"1min":1.335,"5min":1.0660000000000001,"15min":1.0149999999999999},"max":{"1min":1.738,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.2210000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:09:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.52,"15min":1.486},"min":{"1min":1.157,"5min":0.95499999999999996,"15min":0.93799999999999994},"max":{"1min":1.9019999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.26},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6000000000000001,"15min":1.5800000000000001},"min":{"1min":1.3600000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.806,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.381}]}]},{"osd":3,"up_from":13,"seq":55834575979,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51788,"kb_used_data":2692,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320052,"statfs":{"total":96636764160,"available":96583733248,"internally_reserved":0,"allocated":2756608,"data_stored":1338358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.92400000000000004,"15min":0.91100000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.085},"min":{"1min":0.74199999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.571,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.3580000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.163,"15min":1.171},"min":{"1min":0.64700000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.135},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.008,"15min":0.999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.41099999999999998},"max":{"1min":3.863,"5min":3.863,"15min":3.863},"last":0.95999999999999996}]},{"osd":2,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0900000000000001,"15min":1.103},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":1.292},{"interface":"front","average":{"1min":1.27,"5min":1.2230000000000001,"15min":1.252},"min":{"1min":0.80100000000000005,"5min":0.747,"15min":0.58599999999999997},"max":{"1min":3.0790000000000002,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.2509999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2989999999999999,"15min":1.2929999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.282,"15min":1.3},"min":{"1min":0.96799999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.3999999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3240000000000001,"15min":1.331},"min":{"1min":0.86599999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.508},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.373,"15min":1.3759999999999999},"min":{"1min":0.88400000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.845}]},{"osd":6,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.78400000000000003},"max":{"1min":2.5649999999999999,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.54,"5min":1.5049999999999999,"15min":1.514},"min":{"1min":1.075,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":1.7450000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.516,"15min":1.5329999999999999},"min":{"1min":1.085,"5min":1.085,"15min":1.085},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.823},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.54,"15min":1.5309999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.878},"max":{"1min":2.4830000000000001,"5min":2.984,"15min":2.984},"last":1.7170000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575979,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64072,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307768,"statfs":{"total":96636764160,"available":96571154432,"internally_reserved":0,"allocated":2752512,"data_stored":1335657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.411,"5min":1.411,"15min":1.4730000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.052,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3080000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.29}]},{"osd":1,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.0229999999999999,"15min":1.0489999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.343,"5min":1.698,"15min":1.698},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.097,"15min":1.135},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.4730000000000001,"5min":1.635,"15min":1.6599999999999999},"last":1.175}]},{"osd":3,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.026,"15min":1.042},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.393,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.232},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.159,"15min":1.208},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4950000000000001,"5min":1.6879999999999999,"15min":1.79},"last":0.89000000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.319,"15min":1.327},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5700000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.2689999999999999,"15min":1.294},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5189999999999999,"5min":1.714,"15min":1.835},"last":1.3560000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3560000000000001,"15min":1.3839999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.782,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.407,"15min":1.411},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.873,"5min":1.8740000000000001,"15min":2.016},"last":1.5349999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3660000000000001,"15min":1.3799999999999999},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.714,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4730000000000001,"15min":1.4970000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.2210000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.478,"15min":1.4850000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.9350000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.4259999999999999,"15min":1.454},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8149999999999999,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.8029999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575978,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48244,"kb_used_data":3244,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323596,"statfs":{"total":96636764160,"available":96587362304,"internally_reserved":0,"allocated":3321856,"data_stored":1924287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.879,"15min":0.90900000000000003},"min":{"1min":0.58199999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.099,"5min":0.99299999999999999,"15min":1.046},"min":{"1min":0.81000000000000005,"5min":0.37,"15min":0.37},"max":{"1min":1.5069999999999999,"5min":1.5660000000000001,"15min":1.681},"last":0.498}]},{"osd":2,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.56299999999999994,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.147,"5min":1.115,"15min":1.1419999999999999},"min":{"1min":0.75600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.581,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.258,"15min":1.3069999999999999},"min":{"1min":1.026,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5780000000000001,"5min":1.587,"15min":1.788},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.107,"15min":1.135},"min":{"1min":0.70999999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.714,"5min":1.714,"15min":1.8049999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.21,"15min":1.2230000000000001},"min":{"1min":0.89900000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6859999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.389,"15min":1.409},"min":{"1min":1.0740000000000001,"5min":0.872,"15min":0.872},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.0269999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.85899999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.802,"5min":1.8520000000000001,"15min":2.101},"last":1.056},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.304,"15min":1.339},"min":{"1min":1.006,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.609,"5min":1.927,"15min":2.1659999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4019999999999999,"15min":1.4379999999999999},"min":{"1min":0.99399999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3799999999999999,"15min":1.4119999999999999},"min":{"1min":1.1779999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.768,"5min":2.0019999999999998,"15min":2.0209999999999999},"last":0.98499999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4870000000000001,"15min":1.5269999999999999},"min":{"1min":1.2549999999999999,"5min":0.871,"15min":0.871},"max":{"1min":1.911,"5min":2.0329999999999999,"15min":2.1190000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.524,"5min":1.4870000000000001,"15min":1.5149999999999999},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.091}]}]},{"osd":0,"up_from":13,"seq":55834575980,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":2676,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2740224,"data_stored":1335117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.86699999999999999,"15min":0.89400000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1559999999999999,"5min":1.4379999999999999,"15min":1.994},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.04,"15min":1.056},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.427,"5min":1.583,"15min":1.6799999999999999},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0329999999999999,"15min":1.069},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.446,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.1020000000000001,"15min":1.1419999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.726,"5min":1.726,"15min":1.792},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0580000000000001,"15min":1.071},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.8839999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.655,"5min":1.655,"15min":2.6469999999999998},"last":0.78200000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3260000000000001,"15min":1.363},"min":{"1min":0.88400000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.274,"5min":1.284,"15min":1.3089999999999999},"min":{"1min":0.82099999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.677,"5min":1.74,"15min":2.3610000000000002},"last":1.0740000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.355,"15min":1.397},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.5589999999999999,"5min":2.1030000000000002,"15min":2.7719999999999998},"last":1.054},{"interface":"front","average":{"1min":1.431,"5min":1.4099999999999999,"15min":1.4410000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.79600000000000004},"max":{"1min":1.9259999999999999,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":0.97499999999999998}]},{"osd":6,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.367,"15min":1.3859999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7649999999999999,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.458,"15min":1.484},"min":{"1min":1.048,"5min":0.85299999999999998,"15min":0.75800000000000001},"max":{"1min":1.736,"5min":1.9870000000000001,"15min":2.8759999999999999},"last":1.2569999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.49,"15min":1.524},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.841,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4750000000000001,"15min":1.4910000000000001},"min":{"1min":0.97299999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.0619999999999998,"5min":2.1190000000000002,"15min":2.431},"last":1.1379999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:10:48.977 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T03:10:48.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:10:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.231+0000 7fc1d545f640 1 Processor -- start 2023-12-01T03:10:49.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.232+0000 7fc1d545f640 1 -- start start 2023-12-01T03:10:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.233+0000 7fc1d545f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc1d00b5d80 0x7fc1d00d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.233+0000 7fc1d545f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d00ce2e0 con 0x7fc1d00d1230 2023-12-01T03:10:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.233+0000 7fc1d545f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d00ce000 con 0x7fc1d015cd20 2023-12-01T03:10:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.233+0000 7fc1d545f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d00b2160 con 0x7fc1d00b5d80 2023-12-01T03:10:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.233+0000 7fc1cf7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc1d00d1230 0x7fc1d00d2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43648/0 (socket says 172.21.15.115:43648) 2023-12-01T03:10:49.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.233+0000 7fc1cf7fe640 1 -- 172.21.15.115:0/1143169403 learned_addr learned my addr 172.21.15.115:0/1143169403 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:49.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066097844 0 0) 0x7fc1d00ce2e0 con 0x7fc1d00d1230 2023-12-01T03:10:49.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1bc003520 con 0x7fc1d00d1230 2023-12-01T03:10:49.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1ce7fc640 1 --2- 172.21.15.115:0/1143169403 >> v2:172.21.15.142:3300/0 conn(0x7fc1d00b5d80 0x7fc1d00d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:49.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2110935604 0 0) 0x7fc1bc003520 con 0x7fc1d00d1230 2023-12-01T03:10:49.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 >> v2:172.21.15.142:3300/0 conn(0x7fc1d00b5d80 msgr2=0x7fc1d00d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 --2- 172.21.15.115:0/1143169403 >> v2:172.21.15.142:3300/0 conn(0x7fc1d00b5d80 0x7fc1d00d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 >> v1:172.21.15.142:6789/0 conn(0x7fc1d015cd20 legacy=0x7fc1d00b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d00cd3d0 con 0x7fc1d00d1230 2023-12-01T03:10:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc1c8001670 con 0x7fc1d00d1230 2023-12-01T03:10:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 >> v1:172.21.15.115:6789/0 conn(0x7fc1d00d1230 legacy=0x7fc1d00d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1bc0039e0 0x7fc1bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:49.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc0047c0 0x7fc1bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.234+0000 7fc1cdffb640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0052a0 0x7fc1bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc00c1c0 con 0x7fc1bc0052a0 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc00c330 con 0x7fc1bc0047c0 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc00c4f0 con 0x7fc1bc0039e0 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1cf7fe640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0052a0 0x7fc1bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1cf7fe640 1 -- 172.21.15.115:0/1143169403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1bc0039e0 msgr2=0x7fc1bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1ceffd640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1bc0039e0 0x7fc1bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1ce7fc640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc0047c0 0x7fc1bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1cf7fe640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1bc0039e0 0x7fc1bc003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1cf7fe640 1 -- 172.21.15.115:0/1143169403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc0047c0 msgr2=0x7fc1bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:49.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1cf7fe640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc0047c0 0x7fc1bc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1cf7fe640 1 -- 172.21.15.115:0/1143169403 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc003520 con 0x7fc1bc0052a0 2023-12-01T03:10:49.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.235+0000 7fc1ce7fc640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc0047c0 0x7fc1bc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:49.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.236+0000 7fc1cf7fe640 1 -- 172.21.15.115:0/1143169403 reap_dead start 2023-12-01T03:10:49.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.236+0000 7fc1cf7fe640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0052a0 0x7fc1bc00b980 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800c210 tx=0x7fc1bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45412b3a5e30f24 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:49.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.237+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8005ad0 con 0x7fc1bc0052a0 2023-12-01T03:10:49.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.237+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c8005c30 con 0x7fc1bc0052a0 2023-12-01T03:10:49.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.237+0000 7fc1cdffb640 1 -- 172.21.15.115:0/1143169403 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c802c890 con 0x7fc1bc0052a0 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.237+0000 7fc1d545f640 1 -- 172.21.15.115:0/1143169403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0052a0 msgr2=0x7fc1bc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.237+0000 7fc1d545f640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0052a0 0x7fc1bc00b980 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800c210 tx=0x7fc1bc0057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.237+0000 7fc1d545f640 1 -- 172.21.15.115:0/1143169403 shutdown_connections 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.237+0000 7fc1d545f640 1 --2- 172.21.15.115:0/1143169403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0052a0 0x7fc1bc00b980 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.237+0000 7fc1d545f640 1 -- 172.21.15.115:0/1143169403 >> 172.21.15.115:0/1143169403 conn(0x7fc1d006a0e0 msgr2=0x7fc1d006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.237+0000 7fc1d545f640 1 -- 172.21.15.115:0/1143169403 shutdown_connections 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.237+0000 7fc1d545f640 1 -- 172.21.15.115:0/1143169403 wait complete. 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.239+0000 7fc1d545f640 1 Processor -- start 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.239+0000 7fc1d545f640 1 -- start start 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.240+0000 7fc1d545f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1bc0052a0 0x7fc1d014ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.240+0000 7fc1d545f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1d014f860 0x7fc1d015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:49.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.240+0000 7fc1d545f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d015fb20 0x7fc1d01e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.240+0000 7fc1d545f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d015d370 con 0x7fc1d015fb20 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.240+0000 7fc1d545f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d015d650 con 0x7fc1d014f860 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.240+0000 7fc1d545f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d015d930 con 0x7fc1bc0052a0 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.240+0000 7fc1cf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d015fb20 0x7fc1d01e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.240+0000 7fc1cf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d015fb20 0x7fc1d01e66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50166/0 (socket says 172.21.15.115:50166) 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.240+0000 7fc1cf7fe640 1 -- 172.21.15.115:0/1740541561 learned_addr learned my addr 172.21.15.115:0/1740541561 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.241+0000 7fc1ce7fc640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1d014f860 0x7fc1d015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.241+0000 7fc1ceffd640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1bc0052a0 0x7fc1d014ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.241+0000 7fc1cf7fe640 1 -- 172.21.15.115:0/1740541561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1bc0052a0 msgr2=0x7fc1d014ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.241+0000 7fc1cf7fe640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1bc0052a0 0x7fc1d014ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.241+0000 7fc1cf7fe640 1 -- 172.21.15.115:0/1740541561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1d014f860 msgr2=0x7fc1d015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.241+0000 7fc1cf7fe640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1d014f860 0x7fc1d015f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.241+0000 7fc1cf7fe640 1 -- 172.21.15.115:0/1740541561 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d015dc10 con 0x7fc1d015fb20 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.241+0000 7fc1ceffd640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1bc0052a0 0x7fc1d014ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:49.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.241+0000 7fc1cf7fe640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d015fb20 0x7fc1d01e66d0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8000c00 tx=0x7fc1c8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:49.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.241+0000 7fc1c77fe640 1 -- 172.21.15.115:0/1740541561 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80049e0 con 0x7fc1d015fb20 2023-12-01T03:10:49.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.242+0000 7fc1c77fe640 1 -- 172.21.15.115:0/1740541561 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c8004b40 con 0x7fc1d015fb20 2023-12-01T03:10:49.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.242+0000 7fc1d545f640 1 -- 172.21.15.115:0/1740541561 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d015def0 con 0x7fc1d015fb20 2023-12-01T03:10:49.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.242+0000 7fc1c77fe640 1 -- 172.21.15.115:0/1740541561 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8000ed0 con 0x7fc1d015fb20 2023-12-01T03:10:49.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.242+0000 7fc1d545f640 1 -- 172.21.15.115:0/1740541561 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d01e6c10 con 0x7fc1d015fb20 2023-12-01T03:10:49.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.243+0000 7fc1d545f640 1 -- 172.21.15.115:0/1740541561 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d01f5850 con 0x7fc1d015fb20 2023-12-01T03:10:49.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.244+0000 7fc1c77fe640 1 -- 172.21.15.115:0/1740541561 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc1c8032070 con 0x7fc1d015fb20 2023-12-01T03:10:49.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.245+0000 7fc1c77fe640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1b006a550 0x7fc1b006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:49.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.245+0000 7fc1c77fe640 1 -- 172.21.15.115:0/1740541561 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fc1c803c070 con 0x7fc1d015fb20 2023-12-01T03:10:49.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.245+0000 7fc1ceffd640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1b006a550 0x7fc1b006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:49.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.246+0000 7fc1ceffd640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1b006a550 0x7fc1b006ca10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0001ce0 tx=0x7fc1c0009040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:49.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.249+0000 7fc1c77fe640 1 -- 172.21.15.115:0/1740541561 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc1d01f5850 con 0x7fc1d015fb20 2023-12-01T03:10:49.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.480+0000 7fc1d545f640 1 -- 172.21.15.115:0/1740541561 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1d004e890 con 0x7fc1d015fb20 2023-12-01T03:10:49.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.481+0000 7fc1c77fe640 1 -- 172.21.15.115:0/1740541561 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5793 (secure 0 0 0) 0x7fc1c8045090 con 0x7fc1d015fb20 2023-12-01T03:10:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:49.483 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":101,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:25.146035+0000","modified":"2023-12-01T03:10:28.416554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10653":{"gid":10653,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/720518620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":720518620},{"type":"v1","addr":"172.21.15.142:6834","nonce":720518620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:38.965513+0000","modified":"2023-12-01T03:10:41.983462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7333":{"gid":7333,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-01T03:10:49.483 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 101 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.483+0000 7fc1d545f640 1 -- 172.21.15.115:0/1740541561 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1b006a550 msgr2=0x7fc1b006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.483+0000 7fc1d545f640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1b006a550 0x7fc1b006ca10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0001ce0 tx=0x7fc1c0009040 comp rx=0 tx=0).stop 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.483+0000 7fc1d545f640 1 -- 172.21.15.115:0/1740541561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d015fb20 msgr2=0x7fc1d01e66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.483+0000 7fc1d545f640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d015fb20 0x7fc1d01e66d0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8000c00 tx=0x7fc1c8001a00 comp rx=0 tx=0).stop 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.484+0000 7fc1d545f640 1 -- 172.21.15.115:0/1740541561 shutdown_connections 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.484+0000 7fc1d545f640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1b006a550 0x7fc1b006ca10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.484+0000 7fc1d545f640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1d015fb20 0x7fc1d01e66d0 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.484+0000 7fc1d545f640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1d014f860 0x7fc1d015f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.484+0000 7fc1d545f640 1 --2- 172.21.15.115:0/1740541561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1bc0052a0 0x7fc1d014ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.484+0000 7fc1d545f640 1 -- 172.21.15.115:0/1740541561 >> 172.21.15.115:0/1740541561 conn(0x7fc1d006a0e0 msgr2=0x7fc1d00d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.484+0000 7fc1d545f640 1 -- 172.21.15.115:0/1740541561 shutdown_connections 2023-12-01T03:10:49.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:49.484+0000 7fc1d545f640 1 -- 172.21.15.115:0/1740541561 wait complete. 2023-12-01T03:10:49.907 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T03:10:38.965513+0000', 'modified': '2023-12-01T03:10:41.983462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7333}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7333': {'gid': 7333, 'name': 'c', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.115:6837/1399179434', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6835', 'nonce': 1399179434}, {'type': 'v1', 'addr': '172.21.15.115:6837', 'nonce': 1399179434}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T03:10:49.907 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T03:10:49.907 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-01T03:10:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.134+0000 7f8ae7ffb640 1 Processor -- start 2023-12-01T03:10:50.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.134+0000 7f8ae7ffb640 1 -- start start 2023-12-01T03:10:50.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.136+0000 7f8ae7ffb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8ae00c9ef0 0x7f8ae006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.136+0000 7f8ae7ffb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae006f8b0 con 0x7f8ae0159dd0 2023-12-01T03:10:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.136+0000 7f8ae7ffb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae00adcb0 con 0x7f8ae006b6f0 2023-12-01T03:10:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.136+0000 7f8ae7ffb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae00ad9d0 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.136+0000 7f8ae5596640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8ae00c9ef0 0x7f8ae006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.136+0000 7f8ae5d97640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8ae0159dd0 0x7f8ae00c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43656/0 (socket says 172.21.15.115:43656) 2023-12-01T03:10:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.136+0000 7f8ae5d97640 1 -- 172.21.15.115:0/3900381893 learned_addr learned my addr 172.21.15.115:0/3900381893 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.137+0000 7f8ae5596640 1 -- 172.21.15.115:0/3900381893 >> v1:172.21.15.142:6789/0 conn(0x7f8ae006b6f0 legacy=0x7f8ae0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.137+0000 7f8ae5596640 1 -- 172.21.15.115:0/3900381893 >> v1:172.21.15.115:6789/0 conn(0x7f8ae0159dd0 legacy=0x7f8ae00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.137+0000 7f8ae5596640 1 -- 172.21.15.115:0/3900381893 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae00c9000 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.138+0000 7f8ae5596640 1 --2- 172.21.15.115:0/3900381893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 0x7f8ae006fd20 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000ba40 tx=0x7f8ad002df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9050a378fb32c1c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.138+0000 7f8ae4d95640 1 -- 172.21.15.115:0/3900381893 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad002ed20 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.138+0000 7f8ae4d95640 1 -- 172.21.15.115:0/3900381893 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad002ee80 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.138+0000 7f8ae4d95640 1 -- 172.21.15.115:0/3900381893 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0036660 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.139+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3900381893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 msgr2=0x7f8ae006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.139+0000 7f8ae7ffb640 1 --2- 172.21.15.115:0/3900381893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 0x7f8ae006fd20 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000ba40 tx=0x7f8ad002df70 comp rx=0 tx=0).stop 2023-12-01T03:10:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.139+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3900381893 shutdown_connections 2023-12-01T03:10:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.139+0000 7f8ae7ffb640 1 --2- 172.21.15.115:0/3900381893 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 0x7f8ae006fd20 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.139+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3900381893 >> 172.21.15.115:0/3900381893 conn(0x7f8ae00ce880 msgr2=0x7f8ae00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.139+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3900381893 shutdown_connections 2023-12-01T03:10:50.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.139+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3900381893 wait complete. 2023-12-01T03:10:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.141+0000 7f8ae7ffb640 1 Processor -- start 2023-12-01T03:10:50.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.141+0000 7f8ae7ffb640 1 -- start start 2023-12-01T03:10:50.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.142+0000 7f8ae7ffb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae006b6f0 0x7f8ae00c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.142+0000 7f8ae7ffb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 0x7f8ae01ee220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.142+0000 7f8ae7ffb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ae0159dd0 0x7f8ae0156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.142+0000 7f8ae7ffb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae015b4d0 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.142+0000 7f8ae7ffb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae01ee760 con 0x7f8ae006b6f0 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.142+0000 7f8ae7ffb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae01eea00 con 0x7f8ae0159dd0 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.142+0000 7f8ae5596640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 0x7f8ae01ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.142+0000 7f8ae5596640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 0x7f8ae01ee220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50194/0 (socket says 172.21.15.115:50194) 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.142+0000 7f8ae5596640 1 -- 172.21.15.115:0/3457145409 learned_addr learned my addr 172.21.15.115:0/3457145409 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae006b6f0 0x7f8ae00c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ae5596640 1 -- 172.21.15.115:0/3457145409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ae0159dd0 msgr2=0x7f8ae0156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ae5596640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ae0159dd0 0x7f8ae0156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ae5596640 1 -- 172.21.15.115:0/3457145409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae006b6f0 msgr2=0x7f8ae00c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ae5596640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae006b6f0 0x7f8ae00c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ae5596640 1 -- 172.21.15.115:0/3457145409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad000b6a0 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae006b6f0 0x7f8ae00c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:50.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ae5596640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 0x7f8ae01ee220 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000bb60 tx=0x7f8ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ad67fc640 1 -- 172.21.15.115:0/3457145409 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0040820 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ad67fc640 1 -- 172.21.15.115:0/3457145409 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad0040980 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ad67fc640 1 -- 172.21.15.115:0/3457145409 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0040d30 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3457145409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae01eece0 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.143+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3457145409 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae015b6b0 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.145+0000 7f8ad67fc640 1 -- 172.21.15.115:0/3457145409 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8ad003d050 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.146+0000 7f8ad67fc640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ac406a550 0x7f8ac406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.146+0000 7f8ad67fc640 1 -- 172.21.15.115:0/3457145409 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f8ad00ad610 con 0x7f8ae00c9ef0 2023-12-01T03:10:50.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.147+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ac406a550 0x7f8ac406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.147+0000 7f8aeab6f740 1 Processor -- start 2023-12-01T03:10:50.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.147+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ac406a550 0x7f8ac406ca10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8adc005880 tx=0x7f8adc009040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.147+0000 7f8aeab6f740 1 -- start start 2023-12-01T03:10:50.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.148+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b47110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.148+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b16cb0 0x555c51b43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.148+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b1a8b0 0x555c51b444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.148+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c51aee2f0 con 0x555c51b46d30 2023-12-01T03:10:50.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.148+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c51a82f80 con 0x555c51b1a8b0 2023-12-01T03:10:50.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.148+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c51a80710 con 0x555c51b16cb0 2023-12-01T03:10:50.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.148+0000 7f8ae5d97640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b47110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.148+0000 7f8ae5d97640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b47110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50200/0 (socket says 172.21.15.115:50200) 2023-12-01T03:10:50.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.148+0000 7f8ae5d97640 1 -- 172.21.15.115:0/866607620 learned_addr learned my addr 172.21.15.115:0/866607620 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:50.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.149+0000 7f8ae5d97640 1 -- 172.21.15.115:0/866607620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b16cb0 msgr2=0x555c51b43fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.149+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/866607620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b16cb0 0x555c51b43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.149+0000 7f8ae5d97640 1 -- 172.21.15.115:0/866607620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b1a8b0 msgr2=0x555c51b444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.149+0000 7f8ae6598640 1 --2- 172.21.15.115:0/866607620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b1a8b0 0x555c51b444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.149+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/866607620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b1a8b0 0x555c51b444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.149+0000 7f8ae5d97640 1 -- 172.21.15.115:0/866607620 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c51a43690 con 0x555c51b46d30 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.149+0000 7f8ae6598640 1 --2- 172.21.15.115:0/866607620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b1a8b0 0x555c51b444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.149+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/866607620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b47110 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f8adc001e60 tx=0x7f8adc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fb13281564d003 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.149+0000 7f8ac37fe640 1 -- 172.21.15.115:0/866607620 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00d9a0 con 0x555c51b46d30 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8ac37fe640 1 -- 172.21.15.115:0/866607620 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc00db00 con 0x555c51b46d30 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8ac37fe640 1 -- 172.21.15.115:0/866607620 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00de70 con 0x555c51b46d30 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8aeab6f740 1 -- 172.21.15.115:0/866607620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 msgr2=0x555c51b47110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/866607620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b47110 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f8adc001e60 tx=0x7f8adc010000 comp rx=0 tx=0).stop 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8aeab6f740 1 -- 172.21.15.115:0/866607620 shutdown_connections 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/866607620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b16cb0 0x555c51b43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/866607620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b1a8b0 0x555c51b444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/866607620 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b47110 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8aeab6f740 1 -- 172.21.15.115:0/866607620 >> 172.21.15.115:0/866607620 conn(0x555c51b18fc0 msgr2=0x555c51b61b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:50.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8aeab6f740 1 -- 172.21.15.115:0/866607620 shutdown_connections 2023-12-01T03:10:50.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.150+0000 7f8aeab6f740 1 -- 172.21.15.115:0/866607620 wait complete. 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.151+0000 7f8aeab6f740 1 -- start start 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.151+0000 7f8aeab6f740 1 Processor -- start 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.152+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b1a8b0 0x555c51b48eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.152+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 0x555c51b5f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.152+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b3b940 0x555c51b61040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.152+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c51a7c000 con 0x555c51b1a8b0 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.152+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c51a1d7d0 con 0x555c51b46d30 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.152+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c51a1d470 con 0x555c51b3b940 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.152+0000 7f8aeab6f740 10 client.0 _ll_register_callbacks cb 0x555c51a6c1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5d97640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b1a8b0 0x555c51b48eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5d97640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b1a8b0 0x555c51b48eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50216/0 (socket says 172.21.15.115:50216) 2023-12-01T03:10:50.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5d97640 1 -- 172.21.15.115:0/2991743201 learned_addr learned my addr 172.21.15.115:0/2991743201 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae6598640 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b3b940 0x555c51b61040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5596640 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 0x555c51b5f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5d97640 1 -- 172.21.15.115:0/2991743201 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b3b940 msgr2=0x555c51b61040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b3b940 0x555c51b61040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5d97640 1 -- 172.21.15.115:0/2991743201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 msgr2=0x555c51b5f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 0x555c51b5f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5d97640 1 -- 172.21.15.115:0/2991743201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae006f8b0 con 0x555c51b1a8b0 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5596640 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 0x555c51b5f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b1a8b0 0x555c51b48eb0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f8adc000f60 tx=0x7f8adc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a79f7e9511febdf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.153+0000 7f8ac0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/2991743201 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc027070 con 0x555c51b1a8b0 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/2991743201 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f8adc020720 con 0x555c51b1a8b0 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8aeab6f740 1 -- 172.21.15.115:0/2991743201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c51aa6550 con 0x555c51b1a8b0 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/2991743201 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8adc0208e0 con 0x555c51b1a8b0 2023-12-01T03:10:50.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8aeab6f740 10 client.10953 fetch_fsmap learned FSMap version 101 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8aeab6f740 1 -- 172.21.15.115:0/2991743201 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) v3 -- 0x555c51a7a680 con 0x555c51b1a8b0 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/2991743201 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c51aa6550 con 0x555c51b1a8b0 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/2991743201 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 101) v1 ==== 3665+0+0 (secure 0 0 0) 0x7f8adc020a40 con 0x555c51b1a8b0 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8aeab6f740 10 client.10953 fetch_fsmap finished waiting for FSMap version 101 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8aeab6f740 10 client.10953 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8aeab6f740 20 client.10953 populate_metadata read hostname 'smithi115' 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.115:6835/1399179434,v1:172.21.15.115:6837/1399179434] conn(0x555c51b64970 0x555c51b3ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8aeab6f740 4 client.10953 mds_command: new command op to 7333 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.154+0000 7f8aeab6f740 1 -- 172.21.15.115:0/2991743201 --> [v2:172.21.15.115:6835/1399179434,v1:172.21.15.115:6837/1399179434] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c51a6eb50 con 0x555c51b64970 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.155+0000 7f8ae5596640 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.115:6835/1399179434,v1:172.21.15.115:6837/1399179434] conn(0x555c51b64970 0x555c51b3ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.155+0000 7f8ac0ff9640 10 client.10953 ms_handle_connect on v2:172.21.15.115:6835/1399179434 2023-12-01T03:10:50.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.155+0000 7f8ae5596640 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.115:6835/1399179434,v1:172.21.15.115:6837/1399179434] conn(0x555c51b64970 0x555c51b3ff10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ffa46dab09ba34b server_cookie=de14458f91783fea in_seq=0 out_seq=0 2023-12-01T03:10:50.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/2991743201 <== mds.0 v2:172.21.15.115:6835/1399179434 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555c51a6eb50 con 0x555c51b64970 2023-12-01T03:10:50.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8ac0ff9640 10 client.10953 handle_command_reply: tid=0 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8aeab6f740 1 client.10953 shutdown 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8aeab6f740 1 -- 172.21.15.115:0/2991743201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b1a8b0 msgr2=0x555c51b48eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b1a8b0 0x555c51b48eb0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f8adc000f60 tx=0x7f8adc012000 comp rx=0 tx=0).stop 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8aeab6f740 1 -- 172.21.15.115:0/2991743201 shutdown_connections 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.115:6835/1399179434,v1:172.21.15.115:6837/1399179434] conn(0x555c51b64970 0x555c51b3ff10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b3b940 0x555c51b61040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 0x555c51b5f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/2991743201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b1a8b0 0x555c51b48eb0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8ac0ff9640 0 client.10953 ms_handle_reset on v2:172.21.15.115:6835/1399179434 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.156+0000 7f8aeab6f740 1 -- 172.21.15.115:0/2991743201 >> 172.21.15.115:0/2991743201 conn(0x555c51b16cb0 msgr2=0x555c51b1b160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.157+0000 7f8aeab6f740 1 -- 172.21.15.115:0/2991743201 shutdown_connections 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.157+0000 7f8aeab6f740 1 -- 172.21.15.115:0/2991743201 wait complete. 2023-12-01T03:10:50.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.157+0000 7f8aeab6f740 20 client.10953 trim_cache size 0 max 16384 2023-12-01T03:10:50.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.173+0000 7f8aeab6f740 1 Processor -- start 2023-12-01T03:10:50.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.173+0000 7f8aeab6f740 1 -- start start 2023-12-01T03:10:50.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.174+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b1a8b0 0x555c51b65c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.174+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b3b940 0x555c51b1ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.174+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b41460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.174+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c517e3f20 con 0x555c51b46d30 2023-12-01T03:10:50.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.174+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c51b39c10 con 0x555c51b1a8b0 2023-12-01T03:10:50.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.174+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c51b119d0 con 0x555c51b3b940 2023-12-01T03:10:50.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.174+0000 7f8ae6598640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b41460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.174+0000 7f8ae6598640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b41460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50222/0 (socket says 172.21.15.115:50222) 2023-12-01T03:10:50.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.174+0000 7f8ae6598640 1 -- 172.21.15.115:0/58342407 learned_addr learned my addr 172.21.15.115:0/58342407 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:50.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.175+0000 7f8ae6598640 1 -- 172.21.15.115:0/58342407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b3b940 msgr2=0x555c51b1ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.175+0000 7f8ae6598640 1 --2- 172.21.15.115:0/58342407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b3b940 0x555c51b1ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.175+0000 7f8ae6598640 1 -- 172.21.15.115:0/58342407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b1a8b0 msgr2=0x555c51b65c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.175+0000 7f8ae6598640 1 --2- 172.21.15.115:0/58342407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b1a8b0 0x555c51b65c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.175+0000 7f8ae6598640 1 -- 172.21.15.115:0/58342407 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c51a1c9e0 con 0x555c51b46d30 2023-12-01T03:10:50.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.175+0000 7f8ae6598640 1 --2- 172.21.15.115:0/58342407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b41460 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8005330 tx=0x7f8ad80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759af0c048006a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.175+0000 7f8ac3fff640 1 -- 172.21.15.115:0/58342407 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8006b50 con 0x555c51b46d30 2023-12-01T03:10:50.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.175+0000 7f8ac3fff640 1 -- 172.21.15.115:0/58342407 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad800d040 con 0x555c51b46d30 2023-12-01T03:10:50.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.175+0000 7f8ac3fff640 1 -- 172.21.15.115:0/58342407 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800ea70 con 0x555c51b46d30 2023-12-01T03:10:50.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.176+0000 7f8aeab6f740 1 -- 172.21.15.115:0/58342407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 msgr2=0x555c51b41460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.176+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/58342407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b41460 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8005330 tx=0x7f8ad80057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.176+0000 7f8aeab6f740 1 -- 172.21.15.115:0/58342407 shutdown_connections 2023-12-01T03:10:50.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.176+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/58342407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b3b940 0x555c51b1ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.176+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/58342407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b46d30 0x555c51b41460 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.176+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/58342407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b1a8b0 0x555c51b65c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.176+0000 7f8aeab6f740 1 -- 172.21.15.115:0/58342407 >> 172.21.15.115:0/58342407 conn(0x555c51b16cb0 msgr2=0x555c51b11bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:50.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.176+0000 7f8aeab6f740 1 -- 172.21.15.115:0/58342407 shutdown_connections 2023-12-01T03:10:50.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.176+0000 7f8aeab6f740 1 -- 172.21.15.115:0/58342407 wait complete. 2023-12-01T03:10:50.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.177+0000 7f8aeab6f740 1 -- start start 2023-12-01T03:10:50.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.177+0000 7f8aeab6f740 1 Processor -- start 2023-12-01T03:10:50.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.178+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b1a8b0 0x555c51bc78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.178+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b3b940 0x555c51b7a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.178+0000 7f8aeab6f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 0x555c51b7af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.178+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c51bc7e80 con 0x555c51b3b940 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.178+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c51b3ed10 con 0x555c51b46d30 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.178+0000 7f8aeab6f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c51b3ac10 con 0x555c51b1a8b0 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.178+0000 7f8aeab6f740 10 client.0 _ll_register_callbacks cb 0x7f8adc0013d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.178+0000 7f8ae5596640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b3b940 0x555c51b7a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.178+0000 7f8ae5596640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b3b940 0x555c51b7a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50228/0 (socket says 172.21.15.115:50228) 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.178+0000 7f8ae5596640 1 -- 172.21.15.115:0/3137134634 learned_addr learned my addr 172.21.15.115:0/3137134634 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ae5596640 1 -- 172.21.15.115:0/3137134634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b1a8b0 msgr2=0x555c51bc78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ae5596640 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b1a8b0 0x555c51bc78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ae5596640 1 -- 172.21.15.115:0/3137134634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 msgr2=0x555c51b7af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ae6598640 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 0x555c51b7af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ae5596640 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 0x555c51b7af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ae5596640 1 -- 172.21.15.115:0/3137134634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad8004f00 con 0x555c51b3b940 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ae6598640 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 0x555c51b7af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ae5596640 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b3b940 0x555c51b7a9d0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00789b0 tx=0x7f8ad00788f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748d292f6ce2f607 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ac0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/3137134634 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0078bc0 con 0x555c51b3b940 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/3137134634 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f8ad0078d20 con 0x555c51b3b940 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8aeab6f740 1 -- 172.21.15.115:0/3137134634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8adc00db00 con 0x555c51b3b940 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/3137134634 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad00ab770 con 0x555c51b3b940 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8aeab6f740 10 client.10959 fetch_fsmap learned FSMap version 101 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8aeab6f740 1 -- 172.21.15.115:0/3137134634 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) v3 -- 0x555c51a41ef0 con 0x555c51b3b940 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.179+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/3137134634 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc00db00 con 0x555c51b3b940 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.180+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/3137134634 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 101) v1 ==== 3665+0+0 (secure 0 0 0) 0x7f8ad0052020 con 0x555c51b3b940 2023-12-01T03:10:50.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.180+0000 7f8aeab6f740 10 client.10959 fetch_fsmap finished waiting for FSMap version 101 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.180+0000 7f8aeab6f740 10 client.10959 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.180+0000 7f8aeab6f740 20 client.10959 populate_metadata read hostname 'smithi115' 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.180+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.115:6835/1399179434,v1:172.21.15.115:6837/1399179434] conn(0x555c51b42260 0x555c51b42640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.180+0000 7f8aeab6f740 4 client.10959 mds_command: new command op to 7333 tid=0[{"prefix": "status"}] 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.180+0000 7f8aeab6f740 1 -- 172.21.15.115:0/3137134634 --> [v2:172.21.15.115:6835/1399179434,v1:172.21.15.115:6837/1399179434] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555c51a75080 con 0x555c51b42260 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.180+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.115:6835/1399179434,v1:172.21.15.115:6837/1399179434] conn(0x555c51b42260 0x555c51b42640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.180+0000 7f8ac0ff9640 10 client.10959 ms_handle_connect on v2:172.21.15.115:6835/1399179434 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.180+0000 7f8ae5d97640 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.115:6835/1399179434,v1:172.21.15.115:6837/1399179434] conn(0x555c51b42260 0x555c51b42640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795f7d4ef2bec01 server_cookie=233ff85c6c95860c in_seq=0 out_seq=0 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8ac0ff9640 1 -- 172.21.15.115:0/3137134634 <== mds.0 v2:172.21.15.115:6835/1399179434 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555c51a75080 con 0x555c51b42260 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8ac0ff9640 10 client.10959 handle_command_reply: tid=0 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 client.10959 shutdown 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 -- 172.21.15.115:0/3137134634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b3b940 msgr2=0x555c51b7a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b3b940 0x555c51b7a9d0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00789b0 tx=0x7f8ad00788f0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 -- 172.21.15.115:0/3137134634 shutdown_connections 2023-12-01T03:10:50.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.115:6835/1399179434,v1:172.21.15.115:6837/1399179434] conn(0x555c51b42260 0x555c51b42640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555c51b46d30 0x555c51b7af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x555c51b3b940 0x555c51b7a9d0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 --2- 172.21.15.115:0/3137134634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x555c51b1a8b0 0x555c51bc78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8ac0ff9640 0 client.10959 ms_handle_reset on v2:172.21.15.115:6835/1399179434 2023-12-01T03:10:50.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 -- 172.21.15.115:0/3137134634 >> 172.21.15.115:0/3137134634 conn(0x555c51b16cb0 msgr2=0x555c51b1ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:50.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 -- 172.21.15.115:0/3137134634 shutdown_connections 2023-12-01T03:10:50.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 1 -- 172.21.15.115:0/3137134634 wait complete. 2023-12-01T03:10:50.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.181+0000 7f8aeab6f740 20 client.10959 trim_cache size 0 max 16384 2023-12-01T03:10:50.186 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:10:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 7333, 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "backup_fs", 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 11.208296808, 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 101, 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 218, 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 218, 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 30.901673126999999 2023-12-01T03:10:50.187 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:10:50.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3457145409 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ac406a550 msgr2=0x7f8ac406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ac406a550 0x7f8ac406ca10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8adc005880 tx=0x7f8adc009040 comp rx=0 tx=0).stop 2023-12-01T03:10:50.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3457145409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 msgr2=0x7f8ae01ee220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 0x7f8ae01ee220 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000bb60 tx=0x7f8ad00046c0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3457145409 shutdown_connections 2023-12-01T03:10:50.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ac406a550 0x7f8ac406ca10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ae0159dd0 0x7f8ae0156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00c9ef0 0x7f8ae01ee220 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 --2- 172.21.15.115:0/3457145409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ae006b6f0 0x7f8ae00c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3457145409 >> 172.21.15.115:0/3457145409 conn(0x7f8ae00ce880 msgr2=0x7f8ae0157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:50.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3457145409 shutdown_connections 2023-12-01T03:10:50.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.182+0000 7f8ae7ffb640 1 -- 172.21.15.115:0/3457145409 wait complete. 2023-12-01T03:10:50.199 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T03:10:50.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.389+0000 7f6efc173640 1 Processor -- start 2023-12-01T03:10:50.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.390+0000 7f6efc173640 1 -- start start 2023-12-01T03:10:50.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.391+0000 7f6efc173640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef406f2f0 0x7f6ef406bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.391+0000 7f6efc173640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef40add10 con 0x7f6ef40cad70 2023-12-01T03:10:50.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.391+0000 7f6efc173640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef40c5760 con 0x7f6ef415e2f0 2023-12-01T03:10:50.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.391+0000 7f6efc173640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef415e6d0 con 0x7f6ef406f2f0 2023-12-01T03:10:50.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.391+0000 7f6ef970e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6ef40cad70 0x7f6ef406b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43662/0 (socket says 172.21.15.115:43662) 2023-12-01T03:10:50.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.391+0000 7f6ef970e640 1 -- 172.21.15.115:0/3374556034 learned_addr learned my addr 172.21.15.115:0/3374556034 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6efa710640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef406f2f0 0x7f6ef406bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419169074 0 0) 0x7f6ef40add10 con 0x7f6ef40cad70 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee0003520 con 0x7f6ef40cad70 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 613527034 0 0) 0x7f6ee0003520 con 0x7f6ef40cad70 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef406f2f0 msgr2=0x7f6ef406bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef406f2f0 0x7f6ef406bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 >> v1:172.21.15.142:6789/0 conn(0x7f6ef415e2f0 legacy=0x7f6ef40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef40cf7b0 con 0x7f6ef40cad70 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6ee4001670 con 0x7f6ef40cad70 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 >> v1:172.21.15.115:6789/0 conn(0x7f6ef40cad70 legacy=0x7f6ef406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 0x7f6ee0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef406f2f0 0x7f6ee00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ee0004e00 0x7f6ee0005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef415e6d0 con 0x7f6ee0003a10 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee000bcc0 con 0x7f6ee0004e00 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee000be80 con 0x7f6ef406f2f0 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.392+0000 7f6ef9f0f640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 0x7f6ee0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.393+0000 7f6ef9f0f640 1 -- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef406f2f0 msgr2=0x7f6ee00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.393+0000 7f6efa710640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ee0004e00 0x7f6ee0005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.393+0000 7f6ef970e640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef406f2f0 0x7f6ee00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.393+0000 7f6ef9f0f640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef406f2f0 0x7f6ee00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.393+0000 7f6ef9f0f640 1 -- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ee0004e00 msgr2=0x7f6ee0005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.393+0000 7f6ef9f0f640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ee0004e00 0x7f6ee0005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.393+0000 7f6ef9f0f640 1 -- 172.21.15.115:0/3374556034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee0003520 con 0x7f6ee0003a10 2023-12-01T03:10:50.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.393+0000 7f6efa710640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ee0004e00 0x7f6ee0005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:50.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.393+0000 7f6ef970e640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef406f2f0 0x7f6ee00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.394+0000 7f6ef9f0f640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 0x7f6ee0003e30 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0001750 tx=0x7f6ee00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90497e3350d534ef server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.394+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00052d0 con 0x7f6ee0003a10 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.394+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef002e050 con 0x7f6ee0003a10 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.394+0000 7f6ef8f0d640 1 -- 172.21.15.115:0/3374556034 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00035f0 con 0x7f6ee0003a10 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.395+0000 7f6efc173640 1 -- 172.21.15.115:0/3374556034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 msgr2=0x7f6ee0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.395+0000 7f6efc173640 1 --2- 172.21.15.115:0/3374556034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 0x7f6ee0003e30 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0001750 tx=0x7f6ee00057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.395+0000 7f6efa710640 1 -- 172.21.15.115:0/3374556034 reap_dead start 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.395+0000 7f6efc173640 1 -- 172.21.15.115:0/3374556034 shutdown_connections 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.395+0000 7f6efc173640 1 -- 172.21.15.115:0/3374556034 >> 172.21.15.115:0/3374556034 conn(0x7f6ef40ce880 msgr2=0x7f6ef40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.395+0000 7f6efc173640 1 -- 172.21.15.115:0/3374556034 shutdown_connections 2023-12-01T03:10:50.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.395+0000 7f6efc173640 1 -- 172.21.15.115:0/3374556034 wait complete. 2023-12-01T03:10:50.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.397+0000 7f6efc173640 1 Processor -- start 2023-12-01T03:10:50.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.397+0000 7f6efc173640 1 -- start start 2023-12-01T03:10:50.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.398+0000 7f6efc173640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 0x7f6ef40c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.398+0000 7f6efc173640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40c23b0 0x7f6ef41f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.398+0000 7f6efc173640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4147250 0x7f6ef41437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.398+0000 7f6efc173640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4144780 con 0x7f6ee0003a10 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.398+0000 7f6efc173640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40c8880 con 0x7f6ef4147250 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.398+0000 7f6efc173640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41444a0 con 0x7f6ef40c23b0 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.398+0000 7f6ef9f0f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 0x7f6ef40c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.398+0000 7f6ef9f0f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 0x7f6ef40c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50254/0 (socket says 172.21.15.115:50254) 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.398+0000 7f6ef9f0f640 1 -- 172.21.15.115:0/2295547354 learned_addr learned my addr 172.21.15.115:0/2295547354 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.398+0000 7f6ef970e640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40c23b0 0x7f6ef41f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6efa710640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4147250 0x7f6ef41437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6ef9f0f640 1 -- 172.21.15.115:0/2295547354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40c23b0 msgr2=0x7f6ef41f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6ef9f0f640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40c23b0 0x7f6ef41f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6ef9f0f640 1 -- 172.21.15.115:0/2295547354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4147250 msgr2=0x7f6ef41437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6ef9f0f640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4147250 0x7f6ef41437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6ef9f0f640 1 -- 172.21.15.115:0/2295547354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef40c82c0 con 0x7f6ee0003a10 2023-12-01T03:10:50.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6efa710640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4147250 0x7f6ef41437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:50.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6ef9f0f640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 0x7f6ef40c4d50 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0008fd0 tx=0x7f6ef0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6eea7fc640 1 -- 172.21.15.115:0/2295547354 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef00076c0 con 0x7f6ee0003a10 2023-12-01T03:10:50.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6eea7fc640 1 -- 172.21.15.115:0/2295547354 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef0007a30 con 0x7f6ee0003a10 2023-12-01T03:10:50.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6eea7fc640 1 -- 172.21.15.115:0/2295547354 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0006be0 con 0x7f6ee0003a10 2023-12-01T03:10:50.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6efc173640 1 -- 172.21.15.115:0/2295547354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef40c85a0 con 0x7f6ee0003a10 2023-12-01T03:10:50.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.399+0000 7f6efc173640 1 -- 172.21.15.115:0/2295547354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef41f3a60 con 0x7f6ee0003a10 2023-12-01T03:10:50.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.400+0000 7f6efc173640 1 -- 172.21.15.115:0/2295547354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef406fda0 con 0x7f6ee0003a10 2023-12-01T03:10:50.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.401+0000 7f6eea7fc640 1 -- 172.21.15.115:0/2295547354 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f6ef0006d40 con 0x7f6ee0003a10 2023-12-01T03:10:50.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.402+0000 7f6eea7fc640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ed806a550 0x7f6ed806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.402+0000 7f6eea7fc640 1 -- 172.21.15.115:0/2295547354 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f6ef00ab070 con 0x7f6ee0003a10 2023-12-01T03:10:50.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.403+0000 7f6ef970e640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ed806a550 0x7f6ed806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.403+0000 7f6ef970e640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ed806a550 0x7f6ed806ca10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40c4a40 tx=0x7f6ee4009040 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.406+0000 7f6eea7fc640 1 -- 172.21.15.115:0/2295547354 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6ef00ddc80 con 0x7f6ee0003a10 2023-12-01T03:10:50.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.617+0000 7f6efc173640 1 -- 172.21.15.115:0/2295547354 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ef40b43d0 con 0x7f6ed806a550 2023-12-01T03:10:50.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.619+0000 7f6eea7fc640 1 -- 172.21.15.115:0/2295547354 <== mgr.10407 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f6ee400d050 con 0x7f6ed806a550 2023-12-01T03:10:50.620 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T03:10:50.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:50.620 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T03:10:50.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:10:50.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:10:50.620 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "backup_fs" 2023-12-01T03:10:50.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:10:50.620 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T03:10:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 -- 172.21.15.115:0/2295547354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ed806a550 msgr2=0x7f6ed806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ed806a550 0x7f6ed806ca10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40c4a40 tx=0x7f6ee4009040 comp rx=0 tx=0).stop 2023-12-01T03:10:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 -- 172.21.15.115:0/2295547354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 msgr2=0x7f6ef40c4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 0x7f6ef40c4d50 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0008fd0 tx=0x7f6ef0001120 comp rx=0 tx=0).stop 2023-12-01T03:10:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 -- 172.21.15.115:0/2295547354 shutdown_connections 2023-12-01T03:10:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6ed806a550 0x7f6ed806ca10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40c23b0 0x7f6ef41f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4147250 0x7f6ef41437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 --2- 172.21.15.115:0/2295547354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ee0003a10 0x7f6ef40c4d50 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 -- 172.21.15.115:0/2295547354 >> 172.21.15.115:0/2295547354 conn(0x7f6ef40ce880 msgr2=0x7f6ef406c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:50.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 -- 172.21.15.115:0/2295547354 shutdown_connections 2023-12-01T03:10:50.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.621+0000 7f6efc173640 1 -- 172.21.15.115:0/2295547354 wait complete. 2023-12-01T03:10:50.640 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T03:10:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.847+0000 7f1089d7e640 1 Processor -- start 2023-12-01T03:10:50.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.848+0000 7f1089d7e640 1 -- start start 2023-12-01T03:10:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.849+0000 7f1089d7e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f10840cde50 0x7f10840d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.849+0000 7f1089d7e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f10840b8400 0x7f108406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.849+0000 7f1089d7e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10840d1e00 con 0x7f108415cb40 2023-12-01T03:10:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.849+0000 7f1089d7e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10840cd250 con 0x7f10840cde50 2023-12-01T03:10:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.849+0000 7f1089d7e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1084069cc0 con 0x7f10840b8400 2023-12-01T03:10:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.849+0000 7f1083fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f10840cde50 0x7f10840d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.849+0000 7f1083fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f10840cde50 0x7f10840d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50270/0 (socket says 172.21.15.115:50270) 2023-12-01T03:10:50.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.849+0000 7f1083fff640 1 -- 172.21.15.115:0/349197686 learned_addr learned my addr 172.21.15.115:0/349197686 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.850+0000 7f108957d640 1 --2- 172.21.15.115:0/349197686 >> v2:172.21.15.142:3300/0 conn(0x7f10840b8400 0x7f108406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.850+0000 7f1083fff640 1 -- 172.21.15.115:0/349197686 >> v2:172.21.15.142:3300/0 conn(0x7f10840b8400 msgr2=0x7f108406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.850+0000 7f1083fff640 1 --2- 172.21.15.115:0/349197686 >> v2:172.21.15.142:3300/0 conn(0x7f10840b8400 0x7f108406a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.850+0000 7f1083fff640 1 -- 172.21.15.115:0/349197686 >> v1:172.21.15.142:6789/0 conn(0x7f108415cb40 legacy=0x7f10840cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:50.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.850+0000 7f1083fff640 1 -- 172.21.15.115:0/349197686 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1084069f60 con 0x7f10840cde50 2023-12-01T03:10:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.851+0000 7f1083fff640 1 --2- 172.21.15.115:0/349197686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840cde50 0x7f10840d18c0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f10840c9af0 tx=0x7f106c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be87c7ca8e861d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.851+0000 7f10837fe640 1 -- 172.21.15.115:0/349197686 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c02ca80 con 0x7f10840cde50 2023-12-01T03:10:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.851+0000 7f10837fe640 1 -- 172.21.15.115:0/349197686 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c02cbe0 con 0x7f10840cde50 2023-12-01T03:10:50.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.851+0000 7f10837fe640 1 -- 172.21.15.115:0/349197686 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c0343f0 con 0x7f10840cde50 2023-12-01T03:10:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.852+0000 7f1089d7e640 1 -- 172.21.15.115:0/349197686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840cde50 msgr2=0x7f10840d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.852+0000 7f1089d7e640 1 --2- 172.21.15.115:0/349197686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840cde50 0x7f10840d18c0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f10840c9af0 tx=0x7f106c02bee0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.852+0000 7f1089d7e640 1 -- 172.21.15.115:0/349197686 shutdown_connections 2023-12-01T03:10:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.852+0000 7f1089d7e640 1 --2- 172.21.15.115:0/349197686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840cde50 0x7f10840d18c0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.852+0000 7f1089d7e640 1 --2- 172.21.15.115:0/349197686 >> v2:172.21.15.142:3300/0 conn(0x7f10840b8400 0x7f108406a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.852+0000 7f1089d7e640 1 -- 172.21.15.115:0/349197686 >> 172.21.15.115:0/349197686 conn(0x7f10840d2a70 msgr2=0x7f10840d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:50.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.852+0000 7f1089d7e640 1 -- 172.21.15.115:0/349197686 shutdown_connections 2023-12-01T03:10:50.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.852+0000 7f1089d7e640 1 -- 172.21.15.115:0/349197686 wait complete. 2023-12-01T03:10:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.854+0000 7f1089d7e640 1 Processor -- start 2023-12-01T03:10:50.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.854+0000 7f1089d7e640 1 -- start start 2023-12-01T03:10:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1089d7e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840b8400 0x7f1084155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1089d7e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10840cde50 0x7f10840b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1089d7e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f108415cb40 0x7f10841528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1089d7e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1084154860 con 0x7f10840b8400 2023-12-01T03:10:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1089d7e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1084156470 con 0x7f108415cb40 2023-12-01T03:10:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1089d7e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1084153be0 con 0x7f10840cde50 2023-12-01T03:10:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1088d7c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840b8400 0x7f1084155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1088d7c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840b8400 0x7f1084155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50280/0 (socket says 172.21.15.115:50280) 2023-12-01T03:10:50.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1088d7c640 1 -- 172.21.15.115:0/3186300563 learned_addr learned my addr 172.21.15.115:0/3186300563 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1083fff640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10840cde50 0x7f10840b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f108957d640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f108415cb40 0x7f10841528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1088d7c640 1 -- 172.21.15.115:0/3186300563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10840cde50 msgr2=0x7f10840b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1088d7c640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10840cde50 0x7f10840b6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1088d7c640 1 -- 172.21.15.115:0/3186300563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f108415cb40 msgr2=0x7f10841528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1088d7c640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f108415cb40 0x7f10841528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:50.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.856+0000 7f1088d7c640 1 -- 172.21.15.115:0/3186300563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c009b30 con 0x7f10840b8400 2023-12-01T03:10:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.857+0000 7f1088d7c640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840b8400 0x7f1084155f30 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f107400b3f0 tx=0x7f107400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.857+0000 7f10817fa640 1 -- 172.21.15.115:0/3186300563 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074011830 con 0x7f10840b8400 2023-12-01T03:10:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.857+0000 7f10817fa640 1 -- 172.21.15.115:0/3186300563 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1074011990 con 0x7f10840b8400 2023-12-01T03:10:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.857+0000 7f10817fa640 1 -- 172.21.15.115:0/3186300563 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074011d40 con 0x7f10840b8400 2023-12-01T03:10:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.857+0000 7f1089d7e640 1 -- 172.21.15.115:0/3186300563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1084153ec0 con 0x7f10840b8400 2023-12-01T03:10:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.857+0000 7f1089d7e640 1 -- 172.21.15.115:0/3186300563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1084154480 con 0x7f10840b8400 2023-12-01T03:10:50.859 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.858+0000 7f1089d7e640 1 -- 172.21.15.115:0/3186300563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10841549a0 con 0x7f10840b8400 2023-12-01T03:10:50.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.859+0000 7f10817fa640 1 -- 172.21.15.115:0/3186300563 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 124) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f107401a400 con 0x7f10840b8400 2023-12-01T03:10:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.860+0000 7f10817fa640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f106806a550 0x7f106806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.860+0000 7f10817fa640 1 -- 172.21.15.115:0/3186300563 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f1074088d30 con 0x7f10840b8400 2023-12-01T03:10:50.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.860+0000 7f1083fff640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f106806a550 0x7f106806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:50.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.861+0000 7f1083fff640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f106806a550 0x7f106806ca10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f106c035000 tx=0x7f106c030000 comp rx=0 tx=0).ready entity=mgr.10407 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:50.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:50.865+0000 7f10817fa640 1 -- 172.21.15.115:0/3186300563 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f10841549a0 con 0x7f10840b8400 2023-12-01T03:10:51.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.102+0000 7f1089d7e640 1 -- 172.21.15.115:0/3186300563 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f10840618d0 con 0x7f10840b8400 2023-12-01T03:10:51.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.469+0000 7f10817fa640 1 -- 172.21.15.115:0/3186300563 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 125) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f1074086ec0 con 0x7f10840b8400 2023-12-01T03:10:51.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.472+0000 7f10817fa640 1 -- 172.21.15.115:0/3186300563 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v125) v1 ==== 92+0+0 (secure 0 0 0) 0x7f10740711b0 con 0x7f10840b8400 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 -- 172.21.15.115:0/3186300563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f106806a550 msgr2=0x7f106806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f106806a550 0x7f106806ca10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f106c035000 tx=0x7f106c030000 comp rx=0 tx=0).stop 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 -- 172.21.15.115:0/3186300563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840b8400 msgr2=0x7f1084155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840b8400 0x7f1084155f30 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f107400b3f0 tx=0x7f107400b8b0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 -- 172.21.15.115:0/3186300563 shutdown_connections 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f106806a550 0x7f106806ca10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10840cde50 0x7f10840b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f108415cb40 0x7f10841528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 --2- 172.21.15.115:0/3186300563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10840b8400 0x7f1084155f30 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 -- 172.21.15.115:0/3186300563 >> 172.21.15.115:0/3186300563 conn(0x7f10840d2a70 msgr2=0x7f10840cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:51.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 -- 172.21.15.115:0/3186300563 shutdown_connections 2023-12-01T03:10:51.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.475+0000 7f1089d7e640 1 -- 172.21.15.115:0/3186300563 wait complete. 2023-12-01T03:10:51.497 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T03:10:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.691+0000 7ff42359e640 1 Processor -- start 2023-12-01T03:10:51.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.691+0000 7ff42359e640 1 -- start start 2023-12-01T03:10:51.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.692+0000 7ff42359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff424153c30 0x7ff4240ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:51.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.692+0000 7ff42359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff424154010 0x7ff4240cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:51.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.692+0000 7ff42359e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4240cdcd0 con 0x7ff4240cf9b0 2023-12-01T03:10:51.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.692+0000 7ff42359e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4240cfe20 con 0x7ff424153c30 2023-12-01T03:10:51.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.692+0000 7ff42359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff424161830 con 0x7ff424154010 2023-12-01T03:10:51.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.692+0000 7ff42259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff424153c30 0x7ff4240ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:51.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.693+0000 7ff42259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff424153c30 0x7ff4240ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50308/0 (socket says 172.21.15.115:50308) 2023-12-01T03:10:51.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.693+0000 7ff42259c640 1 -- 172.21.15.115:0/1839681212 learned_addr learned my addr 172.21.15.115:0/1839681212 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:51.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.693+0000 7ff42259c640 1 -- 172.21.15.115:0/1839681212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff424154010 msgr2=0x7ff4240cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:51.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.693+0000 7ff42259c640 1 --2- 172.21.15.115:0/1839681212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff424154010 0x7ff4240cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.693+0000 7ff42259c640 1 -- 172.21.15.115:0/1839681212 >> v1:172.21.15.142:6789/0 conn(0x7ff4240cf9b0 legacy=0x7ff424160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:51.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.693+0000 7ff42259c640 1 -- 172.21.15.115:0/1839681212 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240cd3a0 con 0x7ff424153c30 2023-12-01T03:10:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.694+0000 7ff42259c640 1 --2- 172.21.15.115:0/1839681212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424153c30 0x7ff4240ce160 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7ff414009870 tx=0x7ff41402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befeeb6f282cedd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.694+0000 7ff42159a640 1 -- 172.21.15.115:0/1839681212 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414031640 con 0x7ff424153c30 2023-12-01T03:10:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.694+0000 7ff42159a640 1 -- 172.21.15.115:0/1839681212 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4140317a0 con 0x7ff424153c30 2023-12-01T03:10:51.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.694+0000 7ff42159a640 1 -- 172.21.15.115:0/1839681212 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414031b30 con 0x7ff424153c30 2023-12-01T03:10:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.695+0000 7ff42359e640 1 -- 172.21.15.115:0/1839681212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424153c30 msgr2=0x7ff4240ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.695+0000 7ff42359e640 1 --2- 172.21.15.115:0/1839681212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424153c30 0x7ff4240ce160 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7ff414009870 tx=0x7ff41402b8f0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.695+0000 7ff42359e640 1 -- 172.21.15.115:0/1839681212 shutdown_connections 2023-12-01T03:10:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.695+0000 7ff42359e640 1 --2- 172.21.15.115:0/1839681212 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff424154010 0x7ff4240cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.695+0000 7ff42359e640 1 --2- 172.21.15.115:0/1839681212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424153c30 0x7ff4240ce160 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.695+0000 7ff42359e640 1 -- 172.21.15.115:0/1839681212 >> 172.21.15.115:0/1839681212 conn(0x7ff42406a0b0 msgr2=0x7ff42406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.695+0000 7ff42359e640 1 -- 172.21.15.115:0/1839681212 shutdown_connections 2023-12-01T03:10:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.695+0000 7ff42359e640 1 -- 172.21.15.115:0/1839681212 wait complete. 2023-12-01T03:10:51.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.697+0000 7ff42359e640 1 Processor -- start 2023-12-01T03:10:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.697+0000 7ff42359e640 1 -- start start 2023-12-01T03:10:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.698+0000 7ff42359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4240cf9b0 0x7ff424154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.698+0000 7ff42359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff424153c30 0x7ff4241549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.698+0000 7ff42359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424154010 0x7ff4241559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.698+0000 7ff42359e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4241f29b0 con 0x7ff424154010 2023-12-01T03:10:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.698+0000 7ff42359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4241f6ed0 con 0x7ff4240cf9b0 2023-12-01T03:10:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.698+0000 7ff42359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4241f34c0 con 0x7ff424153c30 2023-12-01T03:10:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.698+0000 7ff422d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424154010 0x7ff4241559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.698+0000 7ff422d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424154010 0x7ff4241559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50324/0 (socket says 172.21.15.115:50324) 2023-12-01T03:10:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.698+0000 7ff422d9d640 1 -- 172.21.15.115:0/3045558269 learned_addr learned my addr 172.21.15.115:0/3045558269 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.699+0000 7ff42259c640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4240cf9b0 0x7ff424154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.699+0000 7ff421d9b640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff424153c30 0x7ff4241549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.699+0000 7ff422d9d640 1 -- 172.21.15.115:0/3045558269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff424153c30 msgr2=0x7ff4241549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.699+0000 7ff422d9d640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff424153c30 0x7ff4241549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.699+0000 7ff422d9d640 1 -- 172.21.15.115:0/3045558269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4240cf9b0 msgr2=0x7ff424154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.699+0000 7ff422d9d640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4240cf9b0 0x7ff424154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.699+0000 7ff422d9d640 1 -- 172.21.15.115:0/3045558269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4140094d0 con 0x7ff424154010 2023-12-01T03:10:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.699+0000 7ff422d9d640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424154010 0x7ff4241559d0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00c780 tx=0x7ff41c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.699+0000 7ff412ffd640 1 -- 172.21.15.115:0/3045558269 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c003d80 con 0x7ff424154010 2023-12-01T03:10:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.700+0000 7ff412ffd640 1 -- 172.21.15.115:0/3045558269 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41c00ce90 con 0x7ff424154010 2023-12-01T03:10:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.700+0000 7ff412ffd640 1 -- 172.21.15.115:0/3045558269 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0052b0 con 0x7ff424154010 2023-12-01T03:10:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.700+0000 7ff42359e640 1 -- 172.21.15.115:0/3045558269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4241f3950 con 0x7ff424154010 2023-12-01T03:10:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.700+0000 7ff42359e640 1 -- 172.21.15.115:0/3045558269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241f2e70 con 0x7ff424154010 2023-12-01T03:10:51.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.701+0000 7ff42359e640 1 -- 172.21.15.115:0/3045558269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4241f3630 con 0x7ff424154010 2023-12-01T03:10:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.702+0000 7ff412ffd640 1 -- 172.21.15.115:0/3045558269 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 125) v1 ==== 88726+0+0 (secure 0 0 0) 0x7ff41c005410 con 0x7ff424154010 2023-12-01T03:10:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.702+0000 7ff412ffd640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff40406a5a0 0x7ff40406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.703+0000 7ff412ffd640 1 -- 172.21.15.115:0/3045558269 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7ff41c088400 con 0x7ff424154010 2023-12-01T03:10:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.703+0000 7ff42259c640 1 -- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff40406a5a0 msgr2=0x7ff40406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.703+0000 7ff42259c640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff40406a5a0 0x7ff40406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:51.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.707+0000 7ff412ffd640 1 -- 172.21.15.115:0/3045558269 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff41c0534b0 con 0x7ff424154010 2023-12-01T03:10:51.899 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.899+0000 7ff42359e640 1 -- 172.21.15.115:0/3045558269 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff4240618d0 con 0x7ff424154010 2023-12-01T03:10:51.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.904+0000 7ff42259c640 1 -- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff40406a5a0 msgr2=0x7ff40406ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:51.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.904+0000 7ff42259c640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff40406a5a0 0x7ff40406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:10:51.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.904+0000 7ff412ffd640 1 -- 172.21.15.115:0/3045558269 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff41c056ad0 con 0x7ff424154010 2023-12-01T03:10:51.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 -- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff40406a5a0 msgr2=0x7ff40406ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:51.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff40406a5a0 0x7ff40406ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 -- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424154010 msgr2=0x7ff4241559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:51.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424154010 0x7ff4241559d0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00c780 tx=0x7ff41c00cc40 comp rx=0 tx=0).stop 2023-12-01T03:10:51.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 -- 172.21.15.115:0/3045558269 shutdown_connections 2023-12-01T03:10:51.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff40406a5a0 0x7ff40406ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff424153c30 0x7ff4241549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff424154010 0x7ff4241559d0 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 --2- 172.21.15.115:0/3045558269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4240cf9b0 0x7ff424154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:51.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 -- 172.21.15.115:0/3045558269 >> 172.21.15.115:0/3045558269 conn(0x7ff42406a0b0 msgr2=0x7ff42415c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:51.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.907+0000 7ff42359e640 1 -- 172.21.15.115:0/3045558269 shutdown_connections 2023-12-01T03:10:51.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:51.908+0000 7ff42359e640 1 -- 172.21.15.115:0/3045558269 wait complete. 2023-12-01T03:10:51.925 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T03:10:51.995 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:51.993+0000 7f3717019040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:10:52.003 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:52.002+0000 7f727982b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:10:52.048 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:52.046+0000 7f3717019040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:10:52.059 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:52.058+0000 7f727982b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:10:52.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.126+0000 7f16f6566640 1 Processor -- start 2023-12-01T03:10:52.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.126+0000 7f16f6566640 1 -- start start 2023-12-01T03:10:52.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16f6566640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f16f015cd90 0x7f16f00cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16f6566640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f16f00cfd60 0x7f16f0161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16f6566640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f00ce5e0 con 0x7f16f00b5fa0 2023-12-01T03:10:52.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16f6566640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f00d0ec0 con 0x7f16f00cfd60 2023-12-01T03:10:52.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16f6566640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f0161950 con 0x7f16f015cd90 2023-12-01T03:10:52.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16f4b03640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f16f00cfd60 0x7f16f0161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:52.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16e7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f16f00b5fa0 0x7f16f00cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:43674/0 (socket says 172.21.15.115:43674) 2023-12-01T03:10:52.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16e7fff640 1 -- 172.21.15.115:0/2606883930 learned_addr learned my addr 172.21.15.115:0/2606883930 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:52.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16effff640 1 --2- 172.21.15.115:0/2606883930 >> v2:172.21.15.142:3300/0 conn(0x7f16f015cd90 0x7f16f00cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:52.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16f4b03640 1 -- 172.21.15.115:0/2606883930 >> v2:172.21.15.142:3300/0 conn(0x7f16f015cd90 msgr2=0x7f16f00cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:52.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16f4b03640 1 --2- 172.21.15.115:0/2606883930 >> v2:172.21.15.142:3300/0 conn(0x7f16f015cd90 0x7f16f00cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16f4b03640 1 -- 172.21.15.115:0/2606883930 >> v1:172.21.15.115:6789/0 conn(0x7f16f00b5fa0 legacy=0x7f16f00cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:52.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.128+0000 7f16ef7fe640 1 -- 172.21.15.115:0/2606883930 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000656973 0 0) 0x7f16d8001110 con 0x7f16f00b5fa0 2023-12-01T03:10:52.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.129+0000 7f16f4b03640 1 -- 172.21.15.115:0/2606883930 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f00ce300 con 0x7f16f00cfd60 2023-12-01T03:10:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.130+0000 7f16f4b03640 1 --2- 172.21.15.115:0/2606883930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f00cfd60 0x7f16f0161410 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f16f006bd10 tx=0x7f16e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3d28587f55c262 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.130+0000 7f16ef7fe640 1 -- 172.21.15.115:0/2606883930 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e802cc80 con 0x7f16f00cfd60 2023-12-01T03:10:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.130+0000 7f16ef7fe640 1 -- 172.21.15.115:0/2606883930 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e802cde0 con 0x7f16f00cfd60 2023-12-01T03:10:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.130+0000 7f16ef7fe640 1 -- 172.21.15.115:0/2606883930 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8034660 con 0x7f16f00cfd60 2023-12-01T03:10:52.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.130+0000 7f16f6566640 1 -- 172.21.15.115:0/2606883930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f00cfd60 msgr2=0x7f16f0161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:52.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.130+0000 7f16f6566640 1 --2- 172.21.15.115:0/2606883930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f00cfd60 0x7f16f0161410 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f16f006bd10 tx=0x7f16e802bed0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.131+0000 7f16f6566640 1 -- 172.21.15.115:0/2606883930 shutdown_connections 2023-12-01T03:10:52.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.131+0000 7f16f6566640 1 --2- 172.21.15.115:0/2606883930 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f00cfd60 0x7f16f0161410 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.131+0000 7f16f6566640 1 --2- 172.21.15.115:0/2606883930 >> v2:172.21.15.142:3300/0 conn(0x7f16f015cd90 0x7f16f00cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.131+0000 7f16f6566640 1 -- 172.21.15.115:0/2606883930 >> 172.21.15.115:0/2606883930 conn(0x7f16f0065960 msgr2=0x7f16f0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:52.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.131+0000 7f16f6566640 1 -- 172.21.15.115:0/2606883930 shutdown_connections 2023-12-01T03:10:52.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.131+0000 7f16f6566640 1 -- 172.21.15.115:0/2606883930 wait complete. 2023-12-01T03:10:52.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.132+0000 7f16f6566640 1 Processor -- start 2023-12-01T03:10:52.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.132+0000 7f16f6566640 1 -- start start 2023-12-01T03:10:52.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.133+0000 7f16f6566640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16f00b5fa0 0x7f16f0153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.133+0000 7f16f6566640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16f00cfd60 0x7f16f0154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.133+0000 7f16f6566640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f015cd90 0x7f16f0155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.133+0000 7f16f6566640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f0153570 con 0x7f16f015cd90 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.133+0000 7f16f6566640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f00ca7a0 con 0x7f16f00b5fa0 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.133+0000 7f16f6566640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f015def0 con 0x7f16f00cfd60 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.134+0000 7f16f4b03640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f015cd90 0x7f16f0155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.134+0000 7f16f4b03640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f015cd90 0x7f16f0155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50330/0 (socket says 172.21.15.115:50330) 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.134+0000 7f16f4b03640 1 -- 172.21.15.115:0/3357520232 learned_addr learned my addr 172.21.15.115:0/3357520232 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.134+0000 7f16f4b03640 1 -- 172.21.15.115:0/3357520232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16f00cfd60 msgr2=0x7f16f0154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.134+0000 7f16f4b03640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16f00cfd60 0x7f16f0154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.134+0000 7f16f4b03640 1 -- 172.21.15.115:0/3357520232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16f00b5fa0 msgr2=0x7f16f0153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.134+0000 7f16f4b03640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16f00b5fa0 0x7f16f0153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.134+0000 7f16f4b03640 1 -- 172.21.15.115:0/3357520232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8009a70 con 0x7f16f015cd90 2023-12-01T03:10:52.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.134+0000 7f16f4b03640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f015cd90 0x7f16f0155130 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f16e8037040 tx=0x7f16e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:52.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.134+0000 7f16ed7fa640 1 -- 172.21.15.115:0/3357520232 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8040650 con 0x7f16f015cd90 2023-12-01T03:10:52.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.135+0000 7f16ed7fa640 1 -- 172.21.15.115:0/3357520232 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e80407b0 con 0x7f16f015cd90 2023-12-01T03:10:52.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.135+0000 7f16ed7fa640 1 -- 172.21.15.115:0/3357520232 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8040b60 con 0x7f16f015cd90 2023-12-01T03:10:52.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.135+0000 7f16f6566640 1 -- 172.21.15.115:0/3357520232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f015e1d0 con 0x7f16f015cd90 2023-12-01T03:10:52.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.135+0000 7f16f6566640 1 -- 172.21.15.115:0/3357520232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f00cad40 con 0x7f16f015cd90 2023-12-01T03:10:52.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.136+0000 7f16f6566640 1 -- 172.21.15.115:0/3357520232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f015e390 con 0x7f16f015cd90 2023-12-01T03:10:52.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.137+0000 7f16ed7fa640 1 -- 172.21.15.115:0/3357520232 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 125) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f16e8040cc0 con 0x7f16f015cd90 2023-12-01T03:10:52.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.138+0000 7f16ed7fa640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f16d006a5d0 0x7f16d006ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.138+0000 7f16effff640 1 -- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f16d006a5d0 msgr2=0x7f16d006ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:52.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.138+0000 7f16ed7fa640 1 -- 172.21.15.115:0/3357520232 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f16e80aba10 con 0x7f16f015cd90 2023-12-01T03:10:52.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.138+0000 7f16effff640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f16d006a5d0 0x7f16d006ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:52.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.142+0000 7f16ed7fa640 1 -- 172.21.15.115:0/3357520232 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f16e8076b60 con 0x7f16f015cd90 2023-12-01T03:10:52.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.339+0000 7f16effff640 1 -- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f16d006a5d0 msgr2=0x7f16d006ca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:52.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.339+0000 7f16effff640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f16d006a5d0 0x7f16d006ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:10:52.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.347+0000 7f16f6566640 1 -- 172.21.15.115:0/3357520232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f16f004e890 con 0x7f16f015cd90 2023-12-01T03:10:52.353 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.352+0000 7f16ed7fa640 1 -- 172.21.15.115:0/3357520232 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f16e8009450 con 0x7f16f015cd90 2023-12-01T03:10:52.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.354+0000 7f16f6566640 1 -- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f16d006a5d0 msgr2=0x7f16d006ca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:52.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.354+0000 7f16f6566640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f16d006a5d0 0x7f16d006ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.354+0000 7f16f6566640 1 -- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f015cd90 msgr2=0x7f16f0155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:52.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.354+0000 7f16f6566640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f015cd90 0x7f16f0155130 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f16e8037040 tx=0x7f16e8004a40 comp rx=0 tx=0).stop 2023-12-01T03:10:52.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.354+0000 7f16f6566640 1 -- 172.21.15.115:0/3357520232 shutdown_connections 2023-12-01T03:10:52.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.354+0000 7f16f6566640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f16d006a5d0 0x7f16d006ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.354+0000 7f16f6566640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16f00cfd60 0x7f16f0154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.354+0000 7f16f6566640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16f015cd90 0x7f16f0155130 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.354+0000 7f16f6566640 1 --2- 172.21.15.115:0/3357520232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16f00b5fa0 0x7f16f0153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.354+0000 7f16f6566640 1 -- 172.21.15.115:0/3357520232 >> 172.21.15.115:0/3357520232 conn(0x7f16f0065960 msgr2=0x7f16f00b7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:52.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.355+0000 7f16f6566640 1 -- 172.21.15.115:0/3357520232 shutdown_connections 2023-12-01T03:10:52.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.355+0000 7f16f6566640 1 -- 172.21.15.115:0/3357520232 wait complete. 2023-12-01T03:10:52.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:10:52.373 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-12-01T03:10:52.427 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-01T03:10:52.503 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:52.501+0000 7f3717019040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:10:52.541 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:52.540+0000 7f727982b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:10:52.555 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:52.553+0000 7f3717019040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:10:52.596 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:52.594+0000 7f727982b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:10:52.606 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:52.604+0000 7f3717019040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:10:52.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.627+0000 7f067bfff640 1 Processor -- start 2023-12-01T03:10:52.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.627+0000 7f067bfff640 1 -- start start 2023-12-01T03:10:52.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.629+0000 7f067bfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f067c157470 0x7f067c070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.629+0000 7f067bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c15af50 0x7f067c070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.629+0000 7f067bfff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067c06c370 con 0x7f067c15a000 2023-12-01T03:10:52.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.630+0000 7f067bfff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067c15ada0 con 0x7f067c157470 2023-12-01T03:10:52.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.630+0000 7f067bfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067c15abc0 con 0x7f067c15af50 2023-12-01T03:10:52.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.630+0000 7f067a7fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c15af50 0x7f067c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:52.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.630+0000 7f067a7fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c15af50 0x7f067c070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:35434/0 (socket says 172.21.15.115:35434) 2023-12-01T03:10:52.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.630+0000 7f067a7fc640 1 -- 172.21.15.115:0/3222599149 learned_addr learned my addr 172.21.15.115:0/3222599149 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:52.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.630+0000 7f067affd640 1 --2- 172.21.15.115:0/3222599149 >> v2:172.21.15.142:3300/0 conn(0x7f067c157470 0x7f067c070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:52.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.631+0000 7f067affd640 1 -- 172.21.15.115:0/3222599149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c15af50 msgr2=0x7f067c070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:52.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.631+0000 7f067affd640 1 --2- 172.21.15.115:0/3222599149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c15af50 0x7f067c070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.631+0000 7f067affd640 1 -- 172.21.15.115:0/3222599149 >> v1:172.21.15.142:6789/0 conn(0x7f067c15a000 legacy=0x7f067c15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:52.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.631+0000 7f067affd640 1 -- 172.21.15.115:0/3222599149 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c06c650 con 0x7f067c157470 2023-12-01T03:10:52.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.631+0000 7f067a7fc640 1 --2- 172.21.15.115:0/3222599149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c15af50 0x7f067c070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:10:52.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.632+0000 7f067affd640 1 --2- 172.21.15.115:0/3222599149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f067c157470 0x7f067c070140 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f066c0097a0 tx=0x7f066c02dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81dcfd021f5b7954 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:52.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f0679ffb640 1 -- 172.21.15.115:0/3222599149 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c02f8a0 con 0x7f067c157470 2023-12-01T03:10:52.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f0679ffb640 1 -- 172.21.15.115:0/3222599149 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c02fa00 con 0x7f067c157470 2023-12-01T03:10:52.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f0679ffb640 1 -- 172.21.15.115:0/3222599149 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c02fd90 con 0x7f067c157470 2023-12-01T03:10:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f067bfff640 1 -- 172.21.15.115:0/3222599149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f067c157470 msgr2=0x7f067c070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f067bfff640 1 --2- 172.21.15.115:0/3222599149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f067c157470 0x7f067c070140 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f066c0097a0 tx=0x7f066c02dae0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f067bfff640 1 -- 172.21.15.115:0/3222599149 shutdown_connections 2023-12-01T03:10:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f067bfff640 1 --2- 172.21.15.115:0/3222599149 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c15af50 0x7f067c070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f067bfff640 1 --2- 172.21.15.115:0/3222599149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f067c157470 0x7f067c070140 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f067bfff640 1 -- 172.21.15.115:0/3222599149 >> 172.21.15.115:0/3222599149 conn(0x7f067c073330 msgr2=0x7f067c0cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:52.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f067bfff640 1 -- 172.21.15.115:0/3222599149 shutdown_connections 2023-12-01T03:10:52.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.633+0000 7f067bfff640 1 -- 172.21.15.115:0/3222599149 wait complete. 2023-12-01T03:10:52.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.635+0000 7f067bfff640 1 Processor -- start 2023-12-01T03:10:52.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.635+0000 7f067bfff640 1 -- start start 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.636+0000 7f067bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c157470 0x7f067c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.636+0000 7f067bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f067c15a000 0x7f067c0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.636+0000 7f067bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f067c15af50 0x7f067c1f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.636+0000 7f067bfff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c14dad0 con 0x7f067c15a000 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.636+0000 7f067bfff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c157ab0 con 0x7f067c15af50 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.636+0000 7f067bfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067c0ca7a0 con 0x7f067c157470 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067a7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f067c15a000 0x7f067c0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067a7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f067c15a000 0x7f067c0b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50338/0 (socket says 172.21.15.115:50338) 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067a7fc640 1 -- 172.21.15.115:0/565139532 learned_addr learned my addr 172.21.15.115:0/565139532 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067affd640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c157470 0x7f067c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067b7fe640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f067c15af50 0x7f067c1f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:52.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067a7fc640 1 -- 172.21.15.115:0/565139532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c157470 msgr2=0x7f067c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:52.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067a7fc640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c157470 0x7f067c0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067a7fc640 1 -- 172.21.15.115:0/565139532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f067c15af50 msgr2=0x7f067c1f1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:52.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067a7fc640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f067c15af50 0x7f067c1f1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:52.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067a7fc640 1 -- 172.21.15.115:0/565139532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06700094d0 con 0x7f067c15a000 2023-12-01T03:10:52.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.637+0000 7f067a7fc640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f067c15a000 0x7f067c0b2e80 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f067000e980 tx=0x7f067000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:52.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.638+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067000ca00 con 0x7f067c15a000 2023-12-01T03:10:52.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.638+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067000cb60 con 0x7f067c15a000 2023-12-01T03:10:52.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.638+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670013470 con 0x7f067c15a000 2023-12-01T03:10:52.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.638+0000 7f067bfff640 1 -- 172.21.15.115:0/565139532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c009400 con 0x7f067c15a000 2023-12-01T03:10:52.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.638+0000 7f067bfff640 1 -- 172.21.15.115:0/565139532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c1582c0 con 0x7f067c15a000 2023-12-01T03:10:52.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.639+0000 7f067bfff640 1 -- 172.21.15.115:0/565139532 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067c04e890 con 0x7f067c15a000 2023-12-01T03:10:52.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.640+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 125) v1 ==== 88726+0+0 (secure 0 0 0) 0x7f06700135d0 con 0x7f067c15a000 2023-12-01T03:10:52.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.641+0000 7f06637fe640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:52.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.641+0000 7f067affd640 1 -- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 msgr2=0x7f066806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:52.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.641+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f06700891a0 con 0x7f067c15a000 2023-12-01T03:10:52.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.641+0000 7f067affd640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:10:52.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.645+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f06700542e0 con 0x7f067c15a000 2023-12-01T03:10:52.649 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:52.648+0000 7f727982b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:10:52.668 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:52.666+0000 7f3717019040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:10:52.712 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:52.710+0000 7f727982b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:10:52.717 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:52.715+0000 7f3717019040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:10:52.762 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:52.760+0000 7f727982b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:10:52.826 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:52.824+0000 7f3717019040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:10:52.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.842+0000 7f067affd640 1 -- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 msgr2=0x7f066806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:52.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.842+0000 7f067affd640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:10:52.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:52.860+0000 7f067bfff640 1 -- 172.21.15.115:0/565139532 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f067c06cc10 con 0x7f066806a5a0 2023-12-01T03:10:52.873 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:52.871+0000 7f727982b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:10:52.888 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:52.886+0000 7f3717019040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:10:52.935 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:52.933+0000 7f727982b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:10:52.938 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:52.936+0000 7f3717019040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:10:52.984 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:52.983+0000 7f727982b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:10:52.995 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:52.993+0000 7f3717019040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:10:53.041 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:53.039+0000 7f727982b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:10:53.045 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:53.044+0000 7f3717019040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:10:53.091 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:53.089+0000 7f727982b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:10:53.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:53.243+0000 7f067affd640 1 -- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 msgr2=0x7f066806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:53.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:53.243+0000 7f067affd640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T03:10:53.428 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:53.426+0000 7f3717019040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:10:53.468 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:53.466+0000 7f727982b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:10:53.587 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:53.585+0000 7f3717019040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:10:53.625 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:53.623+0000 7f727982b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:10:53.695 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:53.692+0000 7f3717019040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:10:53.732 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:53.730+0000 7f727982b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:10:53.744 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:53.742+0000 7f3717019040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:10:53.781 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:53.779+0000 7f727982b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:10:53.853 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:53.851+0000 7f3717019040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:10:53.888 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:53.886+0000 7f727982b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:10:53.959 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:53.957+0000 7f3717019040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:10:53.994 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:53.992+0000 7f727982b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:10:54.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:54.044+0000 7f067affd640 1 -- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 msgr2=0x7f066806ca60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:10:54.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:54.044+0000 7f067affd640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T03:10:54.117 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:54.114+0000 7f3717019040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:10:54.151 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:54.149+0000 7f727982b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:10:54.164 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:54.163+0000 7f3717019040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:10:54.199 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:54.197+0000 7f727982b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:10:54.284 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:54.281+0000 7f3717019040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:10:54.317 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:54.314+0000 7f727982b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:10:55.109 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:55.107+0000 7f3717019040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:10:55.161 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.159+0000 7f727982b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:10:55.405 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:55.403+0000 7f3717019040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:10:55.456 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.454+0000 7f727982b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:10:55.462 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:55.461+0000 7f3717019040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:10:55.512 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.510+0000 7f727982b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:10:55.576 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:10:55.575+0000 7f3717019040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:10:55.625 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.623+0000 7f727982b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:10:55.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:55.644+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 126) v1 ==== 88335+0+0 (secure 0 0 0) 0x7f06700546b0 con 0x7f067c15a000 2023-12-01T03:10:55.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:55.645+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 msgr2=0x7f066806ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:10:55.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:55.645+0000 7f06637fe640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:55.946 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.943+0000 7f71f8412640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.946 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.943+0000 7f71f8412640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.946 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.943+0000 7f71f8412640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.946 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.943+0000 7f71f8412640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.946 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.943+0000 7f71f8412640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.946 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.944+0000 7f71f540c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.946 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.944+0000 7f71f540c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.946 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.944+0000 7f71f540c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.946 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.944+0000 7f71f540c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.946 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.944+0000 7f71f540c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.965 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.963+0000 7f71f540c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.965 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.963+0000 7f71f540c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.965 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.963+0000 7f71f540c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.965 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.963+0000 7f71f540c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.965 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.963+0000 7f71f540c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.965 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.963+0000 7f71f8412640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.965 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.963+0000 7f71f8412640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.965 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.963+0000 7f71f8412640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.965 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.963+0000 7f71f8412640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:55.965 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:10:55.963+0000 7f71f8412640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:10:56.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.681+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 127) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f0670016170 con 0x7f067c15a000 2023-12-01T03:10:56.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.682+0000 7f06637fe640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:56.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.682+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0668027f80 con 0x7f066806a5a0 2023-12-01T03:10:56.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.693+0000 7f067affd640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:56.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.701+0000 7f067affd640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f066c038000 tx=0x7f066c008030 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:56.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.854+0000 7f06637fe640 1 -- 172.21.15.115:0/565139532 <== mgr.10977 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+53 (secure 0 0 0) 0x7f0668027f80 con 0x7f066806a5a0 2023-12-01T03:10:56.856 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path mandatory_fs_argument_test_dir 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 -- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 msgr2=0x7f066806ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f066c038000 tx=0x7f066c008030 comp rx=0 tx=0).stop 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 -- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f067c15a000 msgr2=0x7f067c0b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f067c15a000 0x7f067c0b2e80 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f067000e980 tx=0x7f067000ee40 comp rx=0 tx=0).stop 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 -- 172.21.15.115:0/565139532 shutdown_connections 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f066806a5a0 0x7f066806ca60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f067c15af50 0x7f067c1f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f067c15a000 0x7f067c0b2e80 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 --2- 172.21.15.115:0/565139532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f067c157470 0x7f067c0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 -- 172.21.15.115:0/565139532 >> 172.21.15.115:0/565139532 conn(0x7f067c073330 msgr2=0x7f067c06d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:56.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 -- 172.21.15.115:0/565139532 shutdown_connections 2023-12-01T03:10:56.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:56.858+0000 7f067bfff640 1 -- 172.21.15.115:0/565139532 wait complete. 2023-12-01T03:10:56.880 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path mandatory_fs_argument_test_dir 2023-12-01T03:10:56.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-01T03:10:57.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.090+0000 7f66de75c640 1 Processor -- start 2023-12-01T03:10:57.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.091+0000 7f66de75c640 1 -- start start 2023-12-01T03:10:57.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.092+0000 7f66de75c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f66d815ce40 0x7f66d80b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.092+0000 7f66de75c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80d1690 con 0x7f66d80d2730 2023-12-01T03:10:57.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.092+0000 7f66de75c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80d1970 con 0x7f66d80b4d50 2023-12-01T03:10:57.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.092+0000 7f66de75c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d815d220 con 0x7f66d815ce40 2023-12-01T03:10:57.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.092+0000 7f66d7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f66d815ce40 0x7f66d80b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.092+0000 7f66d7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f66d815ce40 0x7f66d80b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50524/0 (socket says 172.21.15.115:50524) 2023-12-01T03:10:57.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.093+0000 7f66d7fff640 1 -- 172.21.15.115:0/4177088167 learned_addr learned my addr 172.21.15.115:0/4177088167 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:57.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.093+0000 7f66d7fff640 1 -- 172.21.15.115:0/4177088167 >> v1:172.21.15.142:6789/0 conn(0x7f66d80b4d50 legacy=0x7f66d80b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:57.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.093+0000 7f66d7fff640 1 -- 172.21.15.115:0/4177088167 >> v1:172.21.15.115:6789/0 conn(0x7f66d80d2730 legacy=0x7f66d80b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:57.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.093+0000 7f66d7fff640 1 -- 172.21.15.115:0/4177088167 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80d1c50 con 0x7f66d815ce40 2023-12-01T03:10:57.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.094+0000 7f66d7fff640 1 --2- 172.21.15.115:0/4177088167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 0x7f66d80b8710 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009e50 tx=0x7f66c802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9fdf2fb36cd870 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:57.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.094+0000 7f66d6ffd640 1 -- 172.21.15.115:0/4177088167 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c802ca80 con 0x7f66d815ce40 2023-12-01T03:10:57.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.094+0000 7f66d6ffd640 1 -- 172.21.15.115:0/4177088167 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c802cbe0 con 0x7f66d815ce40 2023-12-01T03:10:57.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.094+0000 7f66d6ffd640 1 -- 172.21.15.115:0/4177088167 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c80343f0 con 0x7f66d815ce40 2023-12-01T03:10:57.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.095+0000 7f66de75c640 1 -- 172.21.15.115:0/4177088167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 msgr2=0x7f66d80b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:57.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.095+0000 7f66de75c640 1 --2- 172.21.15.115:0/4177088167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 0x7f66d80b8710 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009e50 tx=0x7f66c802bee0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.095+0000 7f66de75c640 1 -- 172.21.15.115:0/4177088167 shutdown_connections 2023-12-01T03:10:57.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.095+0000 7f66de75c640 1 --2- 172.21.15.115:0/4177088167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 0x7f66d80b8710 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.095+0000 7f66de75c640 1 -- 172.21.15.115:0/4177088167 >> 172.21.15.115:0/4177088167 conn(0x7f66d8065f90 msgr2=0x7f66d80cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:57.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.095+0000 7f66de75c640 1 -- 172.21.15.115:0/4177088167 shutdown_connections 2023-12-01T03:10:57.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.095+0000 7f66de75c640 1 -- 172.21.15.115:0/4177088167 wait complete. 2023-12-01T03:10:57.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.097+0000 7f66de75c640 1 Processor -- start 2023-12-01T03:10:57.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.097+0000 7f66de75c640 1 -- start start 2023-12-01T03:10:57.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.098+0000 7f66de75c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66d80b4d50 0x7f66d81f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.098+0000 7f66de75c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66d80d2730 0x7f66d80cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.098+0000 7f66de75c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 0x7f66d80cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.098+0000 7f66de75c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d815d530 con 0x7f66d815ce40 2023-12-01T03:10:57.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.098+0000 7f66de75c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8148a90 con 0x7f66d80d2730 2023-12-01T03:10:57.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.098+0000 7f66de75c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8148f40 con 0x7f66d80b4d50 2023-12-01T03:10:57.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66dccf9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 0x7f66d80cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66dccf9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 0x7f66d80cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50534/0 (socket says 172.21.15.115:50534) 2023-12-01T03:10:57.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66dccf9640 1 -- 172.21.15.115:0/2508568404 learned_addr learned my addr 172.21.15.115:0/2508568404 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:57.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66d7fff640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66d80b4d50 0x7f66d81f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66d77fe640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66d80d2730 0x7f66d80cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66dccf9640 1 -- 172.21.15.115:0/2508568404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66d80b4d50 msgr2=0x7f66d81f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:57.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66dccf9640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66d80b4d50 0x7f66d81f1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66dccf9640 1 -- 172.21.15.115:0/2508568404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66d80d2730 msgr2=0x7f66d80cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:57.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66dccf9640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66d80d2730 0x7f66d80cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66dccf9640 1 -- 172.21.15.115:0/2508568404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8009b30 con 0x7f66d815ce40 2023-12-01T03:10:57.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.099+0000 7f66dccf9640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 0x7f66d80cbf40 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c3e0 tx=0x7f66d000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:57.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.100+0000 7f66d4ff9640 1 -- 172.21.15.115:0/2508568404 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0011790 con 0x7f66d815ce40 2023-12-01T03:10:57.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.100+0000 7f66d4ff9640 1 -- 172.21.15.115:0/2508568404 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d00118f0 con 0x7f66d815ce40 2023-12-01T03:10:57.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.100+0000 7f66d4ff9640 1 -- 172.21.15.115:0/2508568404 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0011ce0 con 0x7f66d815ce40 2023-12-01T03:10:57.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.100+0000 7f66de75c640 1 -- 172.21.15.115:0/2508568404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d80ca8e0 con 0x7f66d815ce40 2023-12-01T03:10:57.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.100+0000 7f66de75c640 1 -- 172.21.15.115:0/2508568404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d80caea0 con 0x7f66d815ce40 2023-12-01T03:10:57.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.101+0000 7f66de75c640 1 -- 172.21.15.115:0/2508568404 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d80d0ea0 con 0x7f66d815ce40 2023-12-01T03:10:57.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.102+0000 7f66d4ff9640 1 -- 172.21.15.115:0/2508568404 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 127) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f66d001c460 con 0x7f66d815ce40 2023-12-01T03:10:57.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.103+0000 7f66d4ff9640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f66b806a2d0 0x7f66b806c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.103+0000 7f66d4ff9640 1 -- 172.21.15.115:0/2508568404 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f66d0088930 con 0x7f66d815ce40 2023-12-01T03:10:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.103+0000 7f66d7fff640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f66b806a2d0 0x7f66b806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.104+0000 7f66d7fff640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f66b806a2d0 0x7f66b806c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f66c8035000 tx=0x7f66c8030000 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:57.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.108+0000 7f66d4ff9640 1 -- 172.21.15.115:0/2508568404 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f66d0053d70 con 0x7f66d815ce40 2023-12-01T03:10:57.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.324+0000 7f66de75c640 1 -- 172.21.15.115:0/2508568404 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f66d80d2210 con 0x7f66b806a2d0 2023-12-01T03:10:57.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.421+0000 7f66d4ff9640 1 -- 172.21.15.115:0/2508568404 <== mgr.10977 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f66c8040450 con 0x7f66b806a2d0 2023-12-01T03:10:57.423 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule activated for path mandatory_fs_argument_test_dir 2023-12-01T03:10:57.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 -- 172.21.15.115:0/2508568404 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f66b806a2d0 msgr2=0x7f66b806c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:57.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f66b806a2d0 0x7f66b806c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f66c8035000 tx=0x7f66c8030000 comp rx=0 tx=0).stop 2023-12-01T03:10:57.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 -- 172.21.15.115:0/2508568404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 msgr2=0x7f66d80cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:57.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 0x7f66d80cbf40 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c3e0 tx=0x7f66d000c8a0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.426 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 -- 172.21.15.115:0/2508568404 shutdown_connections 2023-12-01T03:10:57.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f66b806a2d0 0x7f66b806c790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66d815ce40 0x7f66d80cbf40 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66d80d2730 0x7f66d80cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 --2- 172.21.15.115:0/2508568404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66d80b4d50 0x7f66d81f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 -- 172.21.15.115:0/2508568404 >> 172.21.15.115:0/2508568404 conn(0x7f66d8065f90 msgr2=0x7f66d8154120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:57.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 -- 172.21.15.115:0/2508568404 shutdown_connections 2023-12-01T03:10:57.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.424+0000 7f66de75c640 1 -- 172.21.15.115:0/2508568404 wait complete. 2023-12-01T03:10:57.446 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path mandatory_fs_argument_test_dir 2023-12-01T03:10:57.449 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-01T03:10:57.657 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.656+0000 7f6a32fc9640 1 Processor -- start 2023-12-01T03:10:57.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.656+0000 7f6a32fc9640 1 -- start start 2023-12-01T03:10:57.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.657+0000 7f6a32fc9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6a2c15cd90 0x7f6a2c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.657+0000 7f6a32fc9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0cfd60 0x7f6a2c161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.657+0000 7f6a32fc9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a2c0ce5e0 con 0x7f6a2c0b5fa0 2023-12-01T03:10:57.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.657+0000 7f6a32fc9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2c0d0ec0 con 0x7f6a2c15cd90 2023-12-01T03:10:57.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.657+0000 7f6a32fc9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c161950 con 0x7f6a2c0cfd60 2023-12-01T03:10:57.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.658+0000 7f6a2bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6a2c0b5fa0 0x7f6a2c0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58404/0 (socket says 172.21.15.115:58404) 2023-12-01T03:10:57.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.658+0000 7f6a2bfff640 1 -- 172.21.15.115:0/2148539536 learned_addr learned my addr 172.21.15.115:0/2148539536 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:57.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.658+0000 7f6a31566640 1 --2- 172.21.15.115:0/2148539536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0cfd60 0x7f6a2c161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.658+0000 7f6a30d65640 1 --2- 172.21.15.115:0/2148539536 >> v2:172.21.15.142:3300/0 conn(0x7f6a2c15cd90 0x7f6a2c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.659+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/2148539536 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244425140 0 0) 0x7f6a20001110 con 0x7f6a2c0b5fa0 2023-12-01T03:10:57.660 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.659+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/2148539536 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a14003520 con 0x7f6a2c0b5fa0 2023-12-01T03:10:57.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.659+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/2148539536 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4113947692 0 0) 0x7f6a14003520 con 0x7f6a2c0b5fa0 2023-12-01T03:10:57.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.659+0000 7f6a30d65640 1 -- 172.21.15.115:0/2148539536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0cfd60 msgr2=0x7f6a2c161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:57.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.659+0000 7f6a30d65640 1 --2- 172.21.15.115:0/2148539536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0cfd60 0x7f6a2c161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.659+0000 7f6a30d65640 1 -- 172.21.15.115:0/2148539536 >> v1:172.21.15.115:6789/0 conn(0x7f6a2c0b5fa0 legacy=0x7f6a2c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:57.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.659+0000 7f6a30d65640 1 -- 172.21.15.115:0/2148539536 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c0ce300 con 0x7f6a2c15cd90 2023-12-01T03:10:57.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.659+0000 7f6a31566640 1 --2- 172.21.15.115:0/2148539536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0cfd60 0x7f6a2c161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:10:57.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.660+0000 7f6a30d65640 1 --2- 172.21.15.115:0/2148539536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c15cd90 0x7f6a2c0cdd20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009d90 tx=0x7f6a1c02bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940b69792247cb60 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:57.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.661+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/2148539536 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20001800 con 0x7f6a2c15cd90 2023-12-01T03:10:57.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.661+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/2148539536 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c02c8a0 con 0x7f6a2c15cd90 2023-12-01T03:10:57.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.661+0000 7f6a2b7fe640 1 -- 172.21.15.115:0/2148539536 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c02cc30 con 0x7f6a2c15cd90 2023-12-01T03:10:57.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.662+0000 7f6a32fc9640 1 -- 172.21.15.115:0/2148539536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c15cd90 msgr2=0x7f6a2c0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:57.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.662+0000 7f6a32fc9640 1 --2- 172.21.15.115:0/2148539536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c15cd90 0x7f6a2c0cdd20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009d90 tx=0x7f6a1c02bd70 comp rx=0 tx=0).stop 2023-12-01T03:10:57.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.662+0000 7f6a32fc9640 1 -- 172.21.15.115:0/2148539536 shutdown_connections 2023-12-01T03:10:57.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.662+0000 7f6a32fc9640 1 --2- 172.21.15.115:0/2148539536 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0cfd60 0x7f6a2c161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.662+0000 7f6a32fc9640 1 --2- 172.21.15.115:0/2148539536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c15cd90 0x7f6a2c0cdd20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.662+0000 7f6a32fc9640 1 -- 172.21.15.115:0/2148539536 >> 172.21.15.115:0/2148539536 conn(0x7f6a2c065960 msgr2=0x7f6a2c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:57.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.662+0000 7f6a32fc9640 1 -- 172.21.15.115:0/2148539536 shutdown_connections 2023-12-01T03:10:57.664 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.662+0000 7f6a32fc9640 1 -- 172.21.15.115:0/2148539536 wait complete. 2023-12-01T03:10:57.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.664+0000 7f6a32fc9640 1 Processor -- start 2023-12-01T03:10:57.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.665+0000 7f6a32fc9640 1 -- start start 2023-12-01T03:10:57.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.666+0000 7f6a32fc9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0b5fa0 0x7f6a2c15f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.666+0000 7f6a32fc9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c0cfd60 0x7f6a2c156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.666+0000 7f6a32fc9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c15cd90 0x7f6a2c157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.666+0000 7f6a32fc9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c0a9f70 con 0x7f6a2c15cd90 2023-12-01T03:10:57.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.666+0000 7f6a32fc9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c1f21d0 con 0x7f6a2c0cfd60 2023-12-01T03:10:57.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.666+0000 7f6a32fc9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c15fa70 con 0x7f6a2c0b5fa0 2023-12-01T03:10:57.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.666+0000 7f6a31566640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c15cd90 0x7f6a2c157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.666+0000 7f6a31566640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c15cd90 0x7f6a2c157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46806/0 (socket says 172.21.15.115:46806) 2023-12-01T03:10:57.668 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.666+0000 7f6a31566640 1 -- 172.21.15.115:0/126293385 learned_addr learned my addr 172.21.15.115:0/126293385 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a30d65640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0b5fa0 0x7f6a2c15f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a2bfff640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c0cfd60 0x7f6a2c156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a31566640 1 -- 172.21.15.115:0/126293385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0b5fa0 msgr2=0x7f6a2c15f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a31566640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0b5fa0 0x7f6a2c15f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a31566640 1 -- 172.21.15.115:0/126293385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c0cfd60 msgr2=0x7f6a2c156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a31566640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c0cfd60 0x7f6a2c156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a31566640 1 -- 172.21.15.115:0/126293385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a24009400 con 0x7f6a2c15cd90 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a30d65640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0b5fa0 0x7f6a2c15f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a31566640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c15cd90 0x7f6a2c157470 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e900 tx=0x7f6a2400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a297fa640 1 -- 172.21.15.115:0/126293385 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a240096e0 con 0x7f6a2c15cd90 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a297fa640 1 -- 172.21.15.115:0/126293385 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2400ca00 con 0x7f6a2c15cd90 2023-12-01T03:10:57.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a297fa640 1 -- 172.21.15.115:0/126293385 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400cd70 con 0x7f6a2c15cd90 2023-12-01T03:10:57.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.667+0000 7f6a32fc9640 1 -- 172.21.15.115:0/126293385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c009a70 con 0x7f6a2c15cd90 2023-12-01T03:10:57.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.668+0000 7f6a32fc9640 1 -- 172.21.15.115:0/126293385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2c0a98b0 con 0x7f6a2c15cd90 2023-12-01T03:10:57.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.669+0000 7f6a32fc9640 1 -- 172.21.15.115:0/126293385 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2c04e890 con 0x7f6a2c15cd90 2023-12-01T03:10:57.671 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.670+0000 7f6a297fa640 1 -- 172.21.15.115:0/126293385 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 127) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f6a24013470 con 0x7f6a2c15cd90 2023-12-01T03:10:57.672 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.671+0000 7f6a297fa640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2d0 0x7f6a1806c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:57.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.671+0000 7f6a297fa640 1 -- 172.21.15.115:0/126293385 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f6a24088e40 con 0x7f6a2c15cd90 2023-12-01T03:10:57.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.671+0000 7f6a30d65640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2d0 0x7f6a1806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:57.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.672+0000 7f6a30d65640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2d0 0x7f6a1806c790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c036000 tx=0x7f6a1c030000 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:57.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.676+0000 7f6a297fa640 1 -- 172.21.15.115:0/126293385 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6a240541b0 con 0x7f6a2c15cd90 2023-12-01T03:10:57.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:57.903+0000 7f6a32fc9640 1 -- 172.21.15.115:0/126293385 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a2c0d1380 con 0x7f6a1806a2d0 2023-12-01T03:10:58.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.026+0000 7f6a297fa640 1 -- 172.21.15.115:0/126293385 <== mgr.10977 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f6a1c0450f0 con 0x7f6a1806a2d0 2023-12-01T03:10:58.028 INFO:teuthology.orchestra.run.smithi115.stdout:Retention added to path mandatory_fs_argument_test_dir 2023-12-01T03:10:58.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 -- 172.21.15.115:0/126293385 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2d0 msgr2=0x7f6a1806c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2d0 0x7f6a1806c790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c036000 tx=0x7f6a1c030000 comp rx=0 tx=0).stop 2023-12-01T03:10:58.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 -- 172.21.15.115:0/126293385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c15cd90 msgr2=0x7f6a2c157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c15cd90 0x7f6a2c157470 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e900 tx=0x7f6a2400edc0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 -- 172.21.15.115:0/126293385 shutdown_connections 2023-12-01T03:10:58.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2d0 0x7f6a1806c790 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a2c15cd90 0x7f6a2c157470 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a2c0cfd60 0x7f6a2c156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 --2- 172.21.15.115:0/126293385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a2c0b5fa0 0x7f6a2c15f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 -- 172.21.15.115:0/126293385 >> 172.21.15.115:0/126293385 conn(0x7f6a2c065960 msgr2=0x7f6a2c158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:58.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 -- 172.21.15.115:0/126293385 shutdown_connections 2023-12-01T03:10:58.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.029+0000 7f6a32fc9640 1 -- 172.21.15.115:0/126293385 wait complete. 2023-12-01T03:10:58.053 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path mandatory_fs_argument_test_dir 2023-12-01T03:10:58.054 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-12-01T03:10:58.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.267+0000 7f6c01096640 1 Processor -- start 2023-12-01T03:10:58.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.267+0000 7f6c01096640 1 -- start start 2023-12-01T03:10:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.268+0000 7f6c01096640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bfc15ac70 0x7f6bfc0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.268+0000 7f6c01096640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6bfc0b4070 0x7f6bfc0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.268+0000 7f6c01096640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc06c860 con 0x7f6bfc0adca0 2023-12-01T03:10:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.268+0000 7f6c01096640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc15b050 con 0x7f6bfc0b4070 2023-12-01T03:10:58.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.268+0000 7f6c01096640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0ce0b0 con 0x7f6bfc15ac70 2023-12-01T03:10:58.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.269+0000 7f6bfb59e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6bfc0adca0 0x7f6bfc06b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58418/0 (socket says 172.21.15.115:58418) 2023-12-01T03:10:58.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.269+0000 7f6bfb59e640 1 -- 172.21.15.115:0/4140813995 learned_addr learned my addr 172.21.15.115:0/4140813995 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.269+0000 7f6bfa59c640 1 --2- 172.21.15.115:0/4140813995 >> v2:172.21.15.142:3300/0 conn(0x7f6bfc0b4070 0x7f6bfc0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.269+0000 7f6bfad9d640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bfc15ac70 0x7f6bfc0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.269+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112441621 0 0) 0x7f6bfc06c860 con 0x7f6bfc0adca0 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.269+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6be8003520 con 0x7f6bfc0adca0 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3299450424 0 0) 0x7f6be8003520 con 0x7f6bfc0adca0 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bfc15ac70 msgr2=0x7f6bfc0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bfc15ac70 0x7f6bfc0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 >> v2:172.21.15.142:3300/0 conn(0x7f6bfc0b4070 msgr2=0x7f6bfc0cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 --2- 172.21.15.115:0/4140813995 >> v2:172.21.15.142:3300/0 conn(0x7f6bfc0b4070 0x7f6bfc0cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0ce390 con 0x7f6bfc0adca0 2023-12-01T03:10:58.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6bf4001670 con 0x7f6bfc0adca0 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 >> v1:172.21.15.115:6789/0 conn(0x7f6bfc0adca0 legacy=0x7f6bfc06b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bfc15ac70 0x7f6be8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6be8003ef0 0x7f6be80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6be8004d00 0x7f6be8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc15b050 con 0x7f6be8004d00 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0ce0b0 con 0x7f6be8003ef0 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be800b9d0 con 0x7f6bfc15ac70 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bfb59e640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6be8004d00 0x7f6be8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bfad9d640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bfc15ac70 0x7f6be8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.270+0000 7f6bfa59c640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6be8003ef0 0x7f6be80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.271+0000 7f6bfb59e640 1 -- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bfc15ac70 msgr2=0x7f6be8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.271+0000 7f6bfb59e640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6bfc15ac70 0x7f6be8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.271+0000 7f6bfb59e640 1 -- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6be8003ef0 msgr2=0x7f6be80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.271+0000 7f6bfb59e640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6be8003ef0 0x7f6be80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.271+0000 7f6bfb59e640 1 -- 172.21.15.115:0/4140813995 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be8003520 con 0x7f6be8004d00 2023-12-01T03:10:58.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.271+0000 7f6bfa59c640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6be8003ef0 0x7f6be80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.272+0000 7f6bfb59e640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6be8004d00 0x7f6be8005180 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400c2c0 tx=0x7f6be80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de17fe0979f1bf8c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.272+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4005ad0 con 0x7f6be8004d00 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.272+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf4005c30 con 0x7f6be8004d00 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.272+0000 7f6bf9d9b640 1 -- 172.21.15.115:0/4140813995 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf402c8b0 con 0x7f6be8004d00 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.273+0000 7f6c01096640 1 -- 172.21.15.115:0/4140813995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6be8004d00 msgr2=0x7f6be8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.273+0000 7f6c01096640 1 --2- 172.21.15.115:0/4140813995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6be8004d00 0x7f6be8005180 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400c2c0 tx=0x7f6be80057f0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.273+0000 7f6bfb59e640 1 -- 172.21.15.115:0/4140813995 reap_dead start 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.273+0000 7f6c01096640 1 -- 172.21.15.115:0/4140813995 shutdown_connections 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.273+0000 7f6c01096640 1 -- 172.21.15.115:0/4140813995 >> 172.21.15.115:0/4140813995 conn(0x7f6bfc0662c0 msgr2=0x7f6bfc0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.273+0000 7f6c01096640 1 -- 172.21.15.115:0/4140813995 shutdown_connections 2023-12-01T03:10:58.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.273+0000 7f6c01096640 1 -- 172.21.15.115:0/4140813995 wait complete. 2023-12-01T03:10:58.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.275+0000 7f6c01096640 1 Processor -- start 2023-12-01T03:10:58.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.275+0000 7f6c01096640 1 -- start start 2023-12-01T03:10:58.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.276+0000 7f6c01096640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6be8004d00 0x7f6bfc1f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.276+0000 7f6c01096640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bfc0b3450 0x7f6bfc0c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.276+0000 7f6c01096640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc0c3670 0x7f6bfc0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.276+0000 7f6c01096640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c3af0 con 0x7f6bfc0b3450 2023-12-01T03:10:58.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.276+0000 7f6c01096640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c3dd0 con 0x7f6bfc0c3670 2023-12-01T03:10:58.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.276+0000 7f6c01096640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0c40b0 con 0x7f6be8004d00 2023-12-01T03:10:58.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.276+0000 7f6bfa59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bfc0b3450 0x7f6bfc0c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.276+0000 7f6bfa59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bfc0b3450 0x7f6bfc0c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46836/0 (socket says 172.21.15.115:46836) 2023-12-01T03:10:58.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.276+0000 7f6bfa59c640 1 -- 172.21.15.115:0/1759339765 learned_addr learned my addr 172.21.15.115:0/1759339765 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bfa59c640 1 -- 172.21.15.115:0/1759339765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6be8004d00 msgr2=0x7f6bfc1f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bfad9d640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6be8004d00 0x7f6bfc1f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bfb59e640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc0c3670 0x7f6bfc0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bfa59c640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6be8004d00 0x7f6bfc1f1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bfa59c640 1 -- 172.21.15.115:0/1759339765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc0c3670 msgr2=0x7f6bfc0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bfa59c640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc0c3670 0x7f6bfc0ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bfa59c640 1 -- 172.21.15.115:0/1759339765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0c4390 con 0x7f6bfc0b3450 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bfb59e640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc0c3670 0x7f6bfc0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bfa59c640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bfc0b3450 0x7f6bfc0c3130 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f6be40116d0 tx=0x7f6be4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bf37fe640 1 -- 172.21.15.115:0/1759339765 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be400f9c0 con 0x7f6bfc0b3450 2023-12-01T03:10:58.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bf37fe640 1 -- 172.21.15.115:0/1759339765 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be400fb20 con 0x7f6bfc0b3450 2023-12-01T03:10:58.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6bf37fe640 1 -- 172.21.15.115:0/1759339765 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be400fed0 con 0x7f6bfc0b3450 2023-12-01T03:10:58.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6c01096640 1 -- 172.21.15.115:0/1759339765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc0acde0 con 0x7f6bfc0b3450 2023-12-01T03:10:58.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.277+0000 7f6c01096640 1 -- 172.21.15.115:0/1759339765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc0c8df0 con 0x7f6bfc0b3450 2023-12-01T03:10:58.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.278+0000 7f6c01096640 1 -- 172.21.15.115:0/1759339765 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc0ad2a0 con 0x7f6bfc0b3450 2023-12-01T03:10:58.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.279+0000 7f6bf37fe640 1 -- 172.21.15.115:0/1759339765 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 127) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f6be4015460 con 0x7f6bfc0b3450 2023-12-01T03:10:58.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.280+0000 7f6bf37fe640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6bdc06a2d0 0x7f6bdc06c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.280+0000 7f6bf37fe640 1 -- 172.21.15.115:0/1759339765 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f6be40894e0 con 0x7f6bfc0b3450 2023-12-01T03:10:58.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.280+0000 7f6bfad9d640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6bdc06a2d0 0x7f6bdc06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.281+0000 7f6bfad9d640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6bdc06a2d0 0x7f6bdc06c790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6bec007440 tx=0x7f6bec00b040 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:58.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.285+0000 7f6bf37fe640 1 -- 172.21.15.115:0/1759339765 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6be4054850 con 0x7f6bfc0b3450 2023-12-01T03:10:58.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.448+0000 7f6bf37fe640 1 -- 172.21.15.115:0/1759339765 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 128) v1 ==== 88829+0+0 (secure 0 0 0) 0x7f6be4054c20 con 0x7f6bfc0b3450 2023-12-01T03:10:58.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.502+0000 7f6c01096640 1 -- 172.21.15.115:0/1759339765 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6bfc0c3f40 con 0x7f6bdc06a2d0 2023-12-01T03:10:58.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.514+0000 7f6bf37fe640 1 -- 172.21.15.115:0/1759339765 <== mgr.10977 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+88 (secure 0 0 0) 0x7f6bfc0c3f40 con 0x7f6bdc06a2d0 2023-12-01T03:10:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:58.516 INFO:teuthology.orchestra.run.smithi115.stdout:{"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-01T03:10:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 -- 172.21.15.115:0/1759339765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6bdc06a2d0 msgr2=0x7f6bdc06c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6bdc06a2d0 0x7f6bdc06c790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6bec007440 tx=0x7f6bec00b040 comp rx=0 tx=0).stop 2023-12-01T03:10:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 -- 172.21.15.115:0/1759339765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bfc0b3450 msgr2=0x7f6bfc0c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bfc0b3450 0x7f6bfc0c3130 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f6be40116d0 tx=0x7f6be4011b90 comp rx=0 tx=0).stop 2023-12-01T03:10:58.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 -- 172.21.15.115:0/1759339765 shutdown_connections 2023-12-01T03:10:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6bdc06a2d0 0x7f6bdc06c790 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc0c3670 0x7f6bfc0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bfc0b3450 0x7f6bfc0c3130 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 --2- 172.21.15.115:0/1759339765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6be8004d00 0x7f6bfc1f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 -- 172.21.15.115:0/1759339765 >> 172.21.15.115:0/1759339765 conn(0x7f6bfc0662c0 msgr2=0x7f6bfc070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 -- 172.21.15.115:0/1759339765 shutdown_connections 2023-12-01T03:10:58.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.517+0000 7f6c01096640 1 -- 172.21.15.115:0/1759339765 wait complete. 2023-12-01T03:10:58.539 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-01T03:10:58.539 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-12-01T03:10:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.747+0000 7f3aa44c7640 1 Processor -- start 2023-12-01T03:10:58.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.748+0000 7f3aa44c7640 1 -- start start 2023-12-01T03:10:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.749+0000 7f3aa44c7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a9c159bf0 0x7f3a9c15e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.749+0000 7f3aa44c7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0ca7c0 0x7f3a9c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.749+0000 7f3aa44c7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3a9c0cb6f0 0x7f3a9c06f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.749+0000 7f3aa44c7640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c159fd0 con 0x7f3a9c0cb6f0 2023-12-01T03:10:58.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.749+0000 7f3aa44c7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c0cbb80 con 0x7f3a9c159bf0 2023-12-01T03:10:58.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.749+0000 7f3aa44c7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c15e570 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.749+0000 7f3aa2a64640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3a9c0cb6f0 0x7f3a9c06f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.749+0000 7f3aa2a64640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3a9c0cb6f0 0x7f3a9c06f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46846/0 (socket says 172.21.15.115:46846) 2023-12-01T03:10:58.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.749+0000 7f3aa2a64640 1 -- 172.21.15.115:0/2998960424 learned_addr learned my addr 172.21.15.115:0/2998960424 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:58.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.750+0000 7f3aa2263640 1 --2- 172.21.15.115:0/2998960424 >> v2:172.21.15.142:3300/0 conn(0x7f3a9c159bf0 0x7f3a9c15e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.750+0000 7f3aa2a64640 1 -- 172.21.15.115:0/2998960424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0ca7c0 msgr2=0x7f3a9c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:58.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.750+0000 7f3aa1a62640 1 --2- 172.21.15.115:0/2998960424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0ca7c0 0x7f3a9c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.750+0000 7f3aa2a64640 1 --2- 172.21.15.115:0/2998960424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0ca7c0 0x7f3a9c0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.750+0000 7f3aa2a64640 1 -- 172.21.15.115:0/2998960424 >> v2:172.21.15.142:3300/0 conn(0x7f3a9c159bf0 msgr2=0x7f3a9c15e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:58.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.750+0000 7f3aa2a64640 1 --2- 172.21.15.115:0/2998960424 >> v2:172.21.15.142:3300/0 conn(0x7f3a9c159bf0 0x7f3a9c15e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.750+0000 7f3aa2a64640 1 -- 172.21.15.115:0/2998960424 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0c9440 con 0x7f3a9c0cb6f0 2023-12-01T03:10:58.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.750+0000 7f3aa2263640 1 --2- 172.21.15.115:0/2998960424 >> v2:172.21.15.142:3300/0 conn(0x7f3a9c159bf0 0x7f3a9c15e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:58.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.750+0000 7f3aa1a62640 1 --2- 172.21.15.115:0/2998960424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0ca7c0 0x7f3a9c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:10:58.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.751+0000 7f3aa2a64640 1 --2- 172.21.15.115:0/2998960424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0cb6f0 0x7f3a9c06f650 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0b4930 tx=0x7f3a9402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed8b2584de73860 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.751+0000 7f3aa1261640 1 -- 172.21.15.115:0/2998960424 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9402ec80 con 0x7f3a9c0cb6f0 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.751+0000 7f3aa1261640 1 -- 172.21.15.115:0/2998960424 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9402ede0 con 0x7f3a9c0cb6f0 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.751+0000 7f3aa1261640 1 -- 172.21.15.115:0/2998960424 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94036610 con 0x7f3a9c0cb6f0 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.752+0000 7f3aa44c7640 1 -- 172.21.15.115:0/2998960424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0cb6f0 msgr2=0x7f3a9c06f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.752+0000 7f3aa44c7640 1 --2- 172.21.15.115:0/2998960424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0cb6f0 0x7f3a9c06f650 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0b4930 tx=0x7f3a9402e0e0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.752+0000 7f3aa44c7640 1 -- 172.21.15.115:0/2998960424 shutdown_connections 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.752+0000 7f3aa44c7640 1 --2- 172.21.15.115:0/2998960424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0cb6f0 0x7f3a9c06f650 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.752+0000 7f3aa44c7640 1 --2- 172.21.15.115:0/2998960424 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0ca7c0 0x7f3a9c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.752+0000 7f3aa44c7640 1 --2- 172.21.15.115:0/2998960424 >> v2:172.21.15.142:3300/0 conn(0x7f3a9c159bf0 0x7f3a9c15e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.752+0000 7f3aa44c7640 1 -- 172.21.15.115:0/2998960424 >> 172.21.15.115:0/2998960424 conn(0x7f3a9c0662c0 msgr2=0x7f3a9c0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.752+0000 7f3aa44c7640 1 -- 172.21.15.115:0/2998960424 shutdown_connections 2023-12-01T03:10:58.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.752+0000 7f3aa44c7640 1 -- 172.21.15.115:0/2998960424 wait complete. 2023-12-01T03:10:58.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.754+0000 7f3aa44c7640 1 Processor -- start 2023-12-01T03:10:58.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.754+0000 7f3aa44c7640 1 -- start start 2023-12-01T03:10:58.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.755+0000 7f3aa44c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0ca7c0 0x7f3a9c1f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.755+0000 7f3aa44c7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0cb6f0 0x7f3a9c15a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.755+0000 7f3aa44c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a9c159bf0 0x7f3a9c0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.755+0000 7f3aa44c7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c15c810 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.755+0000 7f3aa44c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c15af10 con 0x7f3a9c159bf0 2023-12-01T03:10:58.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.755+0000 7f3aa44c7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c15b3a0 con 0x7f3a9c0cb6f0 2023-12-01T03:10:58.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa2263640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0ca7c0 0x7f3a9c1f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa2263640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0ca7c0 0x7f3a9c1f1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46856/0 (socket says 172.21.15.115:46856) 2023-12-01T03:10:58.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa2263640 1 -- 172.21.15.115:0/3186529323 learned_addr learned my addr 172.21.15.115:0/3186529323 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:58.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa1a62640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0cb6f0 0x7f3a9c15a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa2263640 1 -- 172.21.15.115:0/3186529323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0cb6f0 msgr2=0x7f3a9c15a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa2263640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0cb6f0 0x7f3a9c15a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa2263640 1 -- 172.21.15.115:0/3186529323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a9c159bf0 msgr2=0x7f3a9c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa2a64640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a9c159bf0 0x7f3a9c0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa2263640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a9c159bf0 0x7f3a9c0c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa2263640 1 -- 172.21.15.115:0/3186529323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a94009b30 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.756+0000 7f3aa2263640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0ca7c0 0x7f3a9c1f1ce0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c7f0 tx=0x7f3a9800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:58.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.757+0000 7f3a92ffd640 1 -- 172.21.15.115:0/3186529323 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98007820 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.757+0000 7f3a92ffd640 1 -- 172.21.15.115:0/3186529323 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a98007980 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.757+0000 7f3a92ffd640 1 -- 172.21.15.115:0/3186529323 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98007cf0 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.757+0000 7f3aa44c7640 1 -- 172.21.15.115:0/3186529323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c15d000 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.757+0000 7f3aa44c7640 1 -- 172.21.15.115:0/3186529323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c15cd20 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.758+0000 7f3aa44c7640 1 -- 172.21.15.115:0/3186529323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c074210 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.759+0000 7f3a92ffd640 1 -- 172.21.15.115:0/3186529323 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 128) v1 ==== 88829+0+0 (secure 0 0 0) 0x7f3a980113f0 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.760+0000 7f3a92ffd640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3a8006a6c0 0x7f3a8006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:58.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.760+0000 7f3a92ffd640 1 -- 172.21.15.115:0/3186529323 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f3a980896b0 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.760+0000 7f3aa1a62640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3a8006a6c0 0x7f3a8006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:58.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.761+0000 7f3aa1a62640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3a8006a6c0 0x7f3a8006cb80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0cdcb0 tx=0x7f3a8c008040 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:58.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.764+0000 7f3a92ffd640 1 -- 172.21.15.115:0/3186529323 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3a980542a0 con 0x7f3a9c0ca7c0 2023-12-01T03:10:58.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.975+0000 7f3aa44c7640 1 -- 172.21.15.115:0/3186529323 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a9c0c9a00 con 0x7f3a8006a6c0 2023-12-01T03:10:58.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.987+0000 7f3a92ffd640 1 -- 172.21.15.115:0/3186529323 <== mgr.10977 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+338 (secure 0 0 0) 0x7f3a8c00c040 con 0x7f3a8006a6c0 2023-12-01T03:10:58.989 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:10:58.989 INFO:teuthology.orchestra.run.smithi115.stdout:[{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T03:10:56", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-01T03:10:58.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.989+0000 7f3aa44c7640 1 -- 172.21.15.115:0/3186529323 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3a8006a6c0 msgr2=0x7f3a8006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.989+0000 7f3aa44c7640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3a8006a6c0 0x7f3a8006cb80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0cdcb0 tx=0x7f3a8c008040 comp rx=0 tx=0).stop 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.989+0000 7f3aa44c7640 1 -- 172.21.15.115:0/3186529323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0ca7c0 msgr2=0x7f3a9c1f1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.989+0000 7f3aa44c7640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0ca7c0 0x7f3a9c1f1ce0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c7f0 tx=0x7f3a9800ccb0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.990+0000 7f3aa44c7640 1 -- 172.21.15.115:0/3186529323 shutdown_connections 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.990+0000 7f3aa44c7640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3a8006a6c0 0x7f3a8006cb80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.990+0000 7f3aa44c7640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a9c0cb6f0 0x7f3a9c15a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.990+0000 7f3aa44c7640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a9c159bf0 0x7f3a9c0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.990+0000 7f3aa44c7640 1 --2- 172.21.15.115:0/3186529323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a9c0ca7c0 0x7f3a9c1f1ce0 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.990+0000 7f3aa44c7640 1 -- 172.21.15.115:0/3186529323 >> 172.21.15.115:0/3186529323 conn(0x7f3a9c0662c0 msgr2=0x7f3a9c06f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.990+0000 7f3aa44c7640 1 -- 172.21.15.115:0/3186529323 shutdown_connections 2023-12-01T03:10:58.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:58.990+0000 7f3aa44c7640 1 -- 172.21.15.115:0/3186529323 wait complete. 2023-12-01T03:10:59.011 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-12-01T00:00:00", "created": "2023-12-01T03:10:56", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-01T03:10:59.011 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-01T03:10:59.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.231+0000 7fe534363640 1 Processor -- start 2023-12-01T03:10:59.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.231+0000 7fe534363640 1 -- start start 2023-12-01T03:10:59.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.232+0000 7fe534363640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe52c0ce1b0 0x7fe52c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe534363640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe52c0b0e60 0x7fe52c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe534363640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52c0cc9a0 con 0x7fe52c15ac80 2023-12-01T03:10:59.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe534363640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52c15b060 con 0x7fe52c0b0e60 2023-12-01T03:10:59.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe534363640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52c0b0ce0 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe5320ff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe52c15ac80 0x7fe52c0cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58434/0 (socket says 172.21.15.115:58434) 2023-12-01T03:10:59.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe5320ff640 1 -- 172.21.15.115:0/2395751016 learned_addr learned my addr 172.21.15.115:0/2395751016 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:10:59.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe532900640 1 --2- 172.21.15.115:0/2395751016 >> v2:172.21.15.115:3300/0 conn(0x7fe52c0b0e60 0x7fe52c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe532900640 1 -- 172.21.15.115:0/2395751016 >> v2:172.21.15.142:3300/0 conn(0x7fe52c0ce1b0 msgr2=0x7fe52c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:59.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe5318fe640 1 --2- 172.21.15.115:0/2395751016 >> v2:172.21.15.142:3300/0 conn(0x7fe52c0ce1b0 0x7fe52c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe532900640 1 --2- 172.21.15.115:0/2395751016 >> v2:172.21.15.142:3300/0 conn(0x7fe52c0ce1b0 0x7fe52c0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe532900640 1 -- 172.21.15.115:0/2395751016 >> v1:172.21.15.115:6789/0 conn(0x7fe52c15ac80 legacy=0x7fe52c0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:59.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe532900640 1 -- 172.21.15.115:0/2395751016 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52c0cbd90 con 0x7fe52c0b0e60 2023-12-01T03:10:59.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.233+0000 7fe5310fd640 1 -- 172.21.15.115:0/2395751016 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278225993 0 0) 0x7fe52c0cc9a0 con 0x7fe52c15ac80 2023-12-01T03:10:59.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.234+0000 7fe532900640 1 --2- 172.21.15.115:0/2395751016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0b0e60 0x7fe52c0b07a0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fe52c15e680 tx=0x7fe52402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c18b49ec392f807 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:59.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.235+0000 7fe5310fd640 1 -- 172.21.15.115:0/2395751016 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52402cc80 con 0x7fe52c0b0e60 2023-12-01T03:10:59.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.235+0000 7fe5310fd640 1 -- 172.21.15.115:0/2395751016 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52402cde0 con 0x7fe52c0b0e60 2023-12-01T03:10:59.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.235+0000 7fe5310fd640 1 -- 172.21.15.115:0/2395751016 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524034660 con 0x7fe52c0b0e60 2023-12-01T03:10:59.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.235+0000 7fe534363640 1 -- 172.21.15.115:0/2395751016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0b0e60 msgr2=0x7fe52c0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:59.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.235+0000 7fe534363640 1 --2- 172.21.15.115:0/2395751016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0b0e60 0x7fe52c0b07a0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fe52c15e680 tx=0x7fe52402bed0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.236+0000 7fe534363640 1 -- 172.21.15.115:0/2395751016 shutdown_connections 2023-12-01T03:10:59.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.236+0000 7fe534363640 1 --2- 172.21.15.115:0/2395751016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0b0e60 0x7fe52c0b07a0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.236+0000 7fe534363640 1 --2- 172.21.15.115:0/2395751016 >> v2:172.21.15.142:3300/0 conn(0x7fe52c0ce1b0 0x7fe52c0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.236+0000 7fe534363640 1 -- 172.21.15.115:0/2395751016 >> 172.21.15.115:0/2395751016 conn(0x7fe52c065960 msgr2=0x7fe52c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:59.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.236+0000 7fe534363640 1 -- 172.21.15.115:0/2395751016 shutdown_connections 2023-12-01T03:10:59.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.236+0000 7fe534363640 1 -- 172.21.15.115:0/2395751016 wait complete. 2023-12-01T03:10:59.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.238+0000 7fe534363640 1 Processor -- start 2023-12-01T03:10:59.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.238+0000 7fe534363640 1 -- start start 2023-12-01T03:10:59.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe534363640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe52c0b0e60 0x7fe52c150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe534363640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0ce1b0 0x7fe52c1ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe534363640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c15ac80 0x7fe52c153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe534363640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c141840 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe534363640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c140c30 con 0x7fe52c15ac80 2023-12-01T03:10:59.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe534363640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52c140f10 con 0x7fe52c0b0e60 2023-12-01T03:10:59.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe5318fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0ce1b0 0x7fe52c1ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe5318fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0ce1b0 0x7fe52c1ef9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46874/0 (socket says 172.21.15.115:46874) 2023-12-01T03:10:59.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe5318fe640 1 -- 172.21.15.115:0/709118558 learned_addr learned my addr 172.21.15.115:0/709118558 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:59.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe5320ff640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe52c0b0e60 0x7fe52c150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe5318fe640 1 -- 172.21.15.115:0/709118558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe52c0b0e60 msgr2=0x7fe52c150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:59.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe5318fe640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe52c0b0e60 0x7fe52c150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe532900640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c15ac80 0x7fe52c153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe5318fe640 1 -- 172.21.15.115:0/709118558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c15ac80 msgr2=0x7fe52c153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:59.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe5318fe640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c15ac80 0x7fe52c153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.239+0000 7fe5318fe640 1 -- 172.21.15.115:0/709118558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524009a70 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.240+0000 7fe5318fe640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0ce1b0 0x7fe52c1ef9a0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00c7f0 tx=0x7fe51c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:59.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.240+0000 7fe522ffd640 1 -- 172.21.15.115:0/709118558 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c004220 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.240+0000 7fe522ffd640 1 -- 172.21.15.115:0/709118558 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c0043a0 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.240+0000 7fe522ffd640 1 -- 172.21.15.115:0/709118558 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c007860 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.240+0000 7fe534363640 1 -- 172.21.15.115:0/709118558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52c1411f0 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.240+0000 7fe534363640 1 -- 172.21.15.115:0/709118558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52c1efee0 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.241+0000 7fe534363640 1 -- 172.21.15.115:0/709118558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52c04e890 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.242+0000 7fe522ffd640 1 -- 172.21.15.115:0/709118558 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 128) v1 ==== 88829+0+0 (secure 0 0 0) 0x7fe51c007bd0 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.243+0000 7fe522ffd640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe51006a6c0 0x7fe51006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.243+0000 7fe522ffd640 1 -- 172.21.15.115:0/709118558 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fe51c0894d0 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.243+0000 7fe5320ff640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe51006a6c0 0x7fe51006cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.244+0000 7fe5320ff640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe51006a6c0 0x7fe51006cb80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe528000f60 tx=0x7fe528009040 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:59.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.247+0000 7fe522ffd640 1 -- 172.21.15.115:0/709118558 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe51c0540c0 con 0x7fe52c0ce1b0 2023-12-01T03:10:59.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.466+0000 7fe534363640 1 -- 172.21.15.115:0/709118558 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe52c0cc350 con 0x7fe51006a6c0 2023-12-01T03:10:59.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.592+0000 7fe522ffd640 1 -- 172.21.15.115:0/709118558 <== mgr.10977 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fe528001270 con 0x7fe51006a6c0 2023-12-01T03:10:59.595 INFO:teuthology.orchestra.run.smithi115.stdout:Retention removed from path mandatory_fs_argument_test_dir 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 -- 172.21.15.115:0/709118558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe51006a6c0 msgr2=0x7fe51006cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe51006a6c0 0x7fe51006cb80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe528000f60 tx=0x7fe528009040 comp rx=0 tx=0).stop 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 -- 172.21.15.115:0/709118558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0ce1b0 msgr2=0x7fe52c1ef9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0ce1b0 0x7fe52c1ef9a0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00c7f0 tx=0x7fe51c00ccb0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 -- 172.21.15.115:0/709118558 shutdown_connections 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe51006a6c0 0x7fe51006cb80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe52c15ac80 0x7fe52c153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe52c0ce1b0 0x7fe52c1ef9a0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 --2- 172.21.15.115:0/709118558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe52c0b0e60 0x7fe52c150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 -- 172.21.15.115:0/709118558 >> 172.21.15.115:0/709118558 conn(0x7fe52c065960 msgr2=0x7fe52c06f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:59.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 -- 172.21.15.115:0/709118558 shutdown_connections 2023-12-01T03:10:59.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.596+0000 7fe534363640 1 -- 172.21.15.115:0/709118558 wait complete. 2023-12-01T03:10:59.618 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention removed from path mandatory_fs_argument_test_dir 2023-12-01T03:10:59.618 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-01T03:10:59.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.835+0000 7fe1cab9c640 1 Processor -- start 2023-12-01T03:10:59.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.835+0000 7fe1cab9c640 1 -- start start 2023-12-01T03:10:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.837+0000 7fe1cab9c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe1c415ce50 0x7fe1c40ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.837+0000 7fe1cab9c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1c40cb8c0 0x7fe1c406b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.837+0000 7fe1cab9c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c40ca120 con 0x7fe1c406caa0 2023-12-01T03:10:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.837+0000 7fe1cab9c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c406d2f0 con 0x7fe1c415ce50 2023-12-01T03:10:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.837+0000 7fe1cab9c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c40c9380 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.837+0000 7fe1c8938640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe1c415ce50 0x7fe1c40ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.837+0000 7fe1c8938640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe1c415ce50 0x7fe1c40ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46878/0 (socket says 172.21.15.115:46878) 2023-12-01T03:10:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.837+0000 7fe1c8938640 1 -- 172.21.15.115:0/493613944 learned_addr learned my addr 172.21.15.115:0/493613944 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.838+0000 7fe1c8938640 1 -- 172.21.15.115:0/493613944 >> v2:172.21.15.142:3300/0 conn(0x7fe1c40cb8c0 msgr2=0x7fe1c406b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.838+0000 7fe1c8938640 1 --2- 172.21.15.115:0/493613944 >> v2:172.21.15.142:3300/0 conn(0x7fe1c40cb8c0 0x7fe1c406b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.838+0000 7fe1c8938640 1 -- 172.21.15.115:0/493613944 >> v1:172.21.15.142:6789/0 conn(0x7fe1c406caa0 legacy=0x7fe1c40cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:10:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.838+0000 7fe1c8938640 1 -- 172.21.15.115:0/493613944 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c40c9660 con 0x7fe1c415ce50 2023-12-01T03:10:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.839+0000 7fe1c8938640 1 --2- 172.21.15.115:0/493613944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c415ce50 0x7fe1c40ccdf0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40adf70 tx=0x7fe1b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8449f74396eb5740 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.839+0000 7fe1c37fe640 1 -- 172.21.15.115:0/493613944 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8031870 con 0x7fe1c415ce50 2023-12-01T03:10:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.839+0000 7fe1c37fe640 1 -- 172.21.15.115:0/493613944 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b80319d0 con 0x7fe1c415ce50 2023-12-01T03:10:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.839+0000 7fe1c37fe640 1 -- 172.21.15.115:0/493613944 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8031d60 con 0x7fe1c415ce50 2023-12-01T03:10:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.840+0000 7fe1cab9c640 1 -- 172.21.15.115:0/493613944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c415ce50 msgr2=0x7fe1c40ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.840+0000 7fe1cab9c640 1 --2- 172.21.15.115:0/493613944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c415ce50 0x7fe1c40ccdf0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40adf70 tx=0x7fe1b802b910 comp rx=0 tx=0).stop 2023-12-01T03:10:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.840+0000 7fe1cab9c640 1 -- 172.21.15.115:0/493613944 shutdown_connections 2023-12-01T03:10:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.840+0000 7fe1cab9c640 1 --2- 172.21.15.115:0/493613944 >> v2:172.21.15.142:3300/0 conn(0x7fe1c40cb8c0 0x7fe1c406b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.840+0000 7fe1cab9c640 1 --2- 172.21.15.115:0/493613944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c415ce50 0x7fe1c40ccdf0 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.840+0000 7fe1cab9c640 1 -- 172.21.15.115:0/493613944 >> 172.21.15.115:0/493613944 conn(0x7fe1c40662c0 msgr2=0x7fe1c40b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:10:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.840+0000 7fe1cab9c640 1 -- 172.21.15.115:0/493613944 shutdown_connections 2023-12-01T03:10:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.840+0000 7fe1cab9c640 1 -- 172.21.15.115:0/493613944 wait complete. 2023-12-01T03:10:59.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.842+0000 7fe1cab9c640 1 Processor -- start 2023-12-01T03:10:59.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.842+0000 7fe1cab9c640 1 -- start start 2023-12-01T03:10:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.843+0000 7fe1cab9c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c406caa0 0x7fe1c41f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.843+0000 7fe1cab9c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c40cb8c0 0x7fe1c406f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.843+0000 7fe1cab9c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c415ce50 0x7fe1c41f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.843+0000 7fe1cab9c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c41f66d0 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.843+0000 7fe1cab9c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40b3540 con 0x7fe1c406caa0 2023-12-01T03:10:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.843+0000 7fe1cab9c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40c5e10 con 0x7fe1c415ce50 2023-12-01T03:10:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.843+0000 7fe1c3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c40cb8c0 0x7fe1c406f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.843+0000 7fe1c3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c40cb8c0 0x7fe1c406f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46888/0 (socket says 172.21.15.115:46888) 2023-12-01T03:10:59.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.843+0000 7fe1c3fff640 1 -- 172.21.15.115:0/1422598741 learned_addr learned my addr 172.21.15.115:0/1422598741 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:10:59.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c9139640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c415ce50 0x7fe1c41f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c3fff640 1 -- 172.21.15.115:0/1422598741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c415ce50 msgr2=0x7fe1c41f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c3fff640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c415ce50 0x7fe1c41f2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c8938640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c406caa0 0x7fe1c41f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c3fff640 1 -- 172.21.15.115:0/1422598741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c406caa0 msgr2=0x7fe1c41f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c3fff640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c406caa0 0x7fe1c41f6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c3fff640 1 -- 172.21.15.115:0/1422598741 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b80094d0 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c3fff640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c40cb8c0 0x7fe1c406f340 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c460 tx=0x7fe1b000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c17fa640 1 -- 172.21.15.115:0/1422598741 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0012830 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c17fa640 1 -- 172.21.15.115:0/1422598741 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b00129d0 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1c17fa640 1 -- 172.21.15.115:0/1422598741 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0012d80 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.844+0000 7fe1cab9c640 1 -- 172.21.15.115:0/1422598741 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c40c60f0 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.845+0000 7fe1cab9c640 1 -- 172.21.15.115:0/1422598741 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c40c66b0 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.845+0000 7fe1cab9c640 1 -- 172.21.15.115:0/1422598741 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c4074210 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.847+0000 7fe1c17fa640 1 -- 172.21.15.115:0/1422598741 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 128) v1 ==== 88829+0+0 (secure 0 0 0) 0x7fe1b001a3b0 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.848+0000 7fe1c17fa640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1a806a6c0 0x7fe1a806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:10:59.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.848+0000 7fe1c17fa640 1 -- 172.21.15.115:0/1422598741 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fe1b0088ee0 con 0x7fe1c40cb8c0 2023-12-01T03:10:59.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.848+0000 7fe1c8938640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1a806a6c0 0x7fe1a806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:10:59.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.849+0000 7fe1c8938640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1a806a6c0 0x7fe1a806cb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8036000 tx=0x7fe1b803c040 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:10:59.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:10:59.852+0000 7fe1c17fa640 1 -- 172.21.15.115:0/1422598741 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe1b0053cb0 con 0x7fe1c40cb8c0 2023-12-01T03:11:00.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.068+0000 7fe1cab9c640 1 -- 172.21.15.115:0/1422598741 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1c40c9c20 con 0x7fe1a806a6c0 2023-12-01T03:11:00.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.195+0000 7fe1c17fa640 1 -- 172.21.15.115:0/1422598741 <== mgr.10977 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+61 (secure 0 0 0) 0x7fe1b8031cc0 con 0x7fe1a806a6c0 2023-12-01T03:11:00.197 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule deactivated for path mandatory_fs_argument_test_dir 2023-12-01T03:11:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.198+0000 7fe1cab9c640 1 -- 172.21.15.115:0/1422598741 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1a806a6c0 msgr2=0x7fe1a806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.198+0000 7fe1cab9c640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1a806a6c0 0x7fe1a806cb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8036000 tx=0x7fe1b803c040 comp rx=0 tx=0).stop 2023-12-01T03:11:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.198+0000 7fe1cab9c640 1 -- 172.21.15.115:0/1422598741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c40cb8c0 msgr2=0x7fe1c406f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.198+0000 7fe1cab9c640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c40cb8c0 0x7fe1c406f340 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c460 tx=0x7fe1b000c920 comp rx=0 tx=0).stop 2023-12-01T03:11:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.198+0000 7fe1cab9c640 1 -- 172.21.15.115:0/1422598741 shutdown_connections 2023-12-01T03:11:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.198+0000 7fe1cab9c640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1a806a6c0 0x7fe1a806cb80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.198+0000 7fe1cab9c640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c415ce50 0x7fe1c41f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.198+0000 7fe1cab9c640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1c40cb8c0 0x7fe1c406f340 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.198+0000 7fe1cab9c640 1 --2- 172.21.15.115:0/1422598741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c406caa0 0x7fe1c41f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.198+0000 7fe1cab9c640 1 -- 172.21.15.115:0/1422598741 >> 172.21.15.115:0/1422598741 conn(0x7fe1c40662c0 msgr2=0x7fe1c40cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.199+0000 7fe1cab9c640 1 -- 172.21.15.115:0/1422598741 shutdown_connections 2023-12-01T03:11:00.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.199+0000 7fe1cab9c640 1 -- 172.21.15.115:0/1422598741 wait complete. 2023-12-01T03:11:00.221 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path mandatory_fs_argument_test_dir 2023-12-01T03:11:00.221 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-01T03:11:00.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.433+0000 7f626b451640 1 Processor -- start 2023-12-01T03:11:00.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.433+0000 7f626b451640 1 -- start start 2023-12-01T03:11:00.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.434+0000 7f626b451640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f62640cb8c0 0x7f62640cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:00.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.434+0000 7f626b451640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62640b4070 0x7f62640cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:00.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.434+0000 7f626b451640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626406c860 con 0x7f626415ac70 2023-12-01T03:11:00.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.434+0000 7f626b451640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626415b050 con 0x7f62640cb8c0 2023-12-01T03:11:00.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.434+0000 7f626b451640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62640ce0b0 con 0x7f62640b4070 2023-12-01T03:11:00.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.435+0000 7f62699ee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62640b4070 0x7f62640cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:00.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.435+0000 7f62689ec640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f62640cb8c0 0x7f62640cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:00.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.435+0000 7f62691ed640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f626415ac70 0x7f62640cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:44518/0 (socket says 172.21.15.115:44518) 2023-12-01T03:11:00.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.435+0000 7f62691ed640 1 -- 172.21.15.115:0/2827339428 learned_addr learned my addr 172.21.15.115:0/2827339428 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:00.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.436+0000 7f625bfff640 1 -- 172.21.15.115:0/2827339428 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853510186 0 0) 0x7f626406c860 con 0x7f626415ac70 2023-12-01T03:11:00.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.436+0000 7f625bfff640 1 -- 172.21.15.115:0/2827339428 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6250003520 con 0x7f626415ac70 2023-12-01T03:11:00.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.436+0000 7f62699ee640 1 -- 172.21.15.115:0/2827339428 >> v2:172.21.15.142:3300/0 conn(0x7f62640cb8c0 msgr2=0x7f62640cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:00.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.436+0000 7f62699ee640 1 --2- 172.21.15.115:0/2827339428 >> v2:172.21.15.142:3300/0 conn(0x7f62640cb8c0 0x7f62640cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.436+0000 7f62699ee640 1 -- 172.21.15.115:0/2827339428 >> v1:172.21.15.142:6789/0 conn(0x7f626415ac70 legacy=0x7f62640cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:00.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.436+0000 7f62699ee640 1 -- 172.21.15.115:0/2827339428 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640ce390 con 0x7f62640b4070 2023-12-01T03:11:00.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.436+0000 7f62689ec640 1 --2- 172.21.15.115:0/2827339428 >> v2:172.21.15.142:3300/0 conn(0x7f62640cb8c0 0x7f62640cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:11:00.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.438+0000 7f62699ee640 1 --2- 172.21.15.115:0/2827339428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62640b4070 0x7f62640cc8b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f625c009d90 tx=0x7f625c02beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6725dc7e30f8e36 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:00.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.438+0000 7f625bfff640 1 -- 172.21.15.115:0/2827339428 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c02cc60 con 0x7f62640b4070 2023-12-01T03:11:00.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.438+0000 7f625bfff640 1 -- 172.21.15.115:0/2827339428 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c02cdc0 con 0x7f62640b4070 2023-12-01T03:11:00.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.438+0000 7f625bfff640 1 -- 172.21.15.115:0/2827339428 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c034660 con 0x7f62640b4070 2023-12-01T03:11:00.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.439+0000 7f626b451640 1 -- 172.21.15.115:0/2827339428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62640b4070 msgr2=0x7f62640cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:00.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.439+0000 7f626b451640 1 --2- 172.21.15.115:0/2827339428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62640b4070 0x7f62640cc8b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f625c009d90 tx=0x7f625c02beb0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.439+0000 7f626b451640 1 -- 172.21.15.115:0/2827339428 shutdown_connections 2023-12-01T03:11:00.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.439+0000 7f626b451640 1 --2- 172.21.15.115:0/2827339428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62640b4070 0x7f62640cc8b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.439+0000 7f626b451640 1 --2- 172.21.15.115:0/2827339428 >> v2:172.21.15.142:3300/0 conn(0x7f62640cb8c0 0x7f62640cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.439+0000 7f626b451640 1 -- 172.21.15.115:0/2827339428 >> 172.21.15.115:0/2827339428 conn(0x7f62640662c0 msgr2=0x7f62640b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:00.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.439+0000 7f626b451640 1 -- 172.21.15.115:0/2827339428 shutdown_connections 2023-12-01T03:11:00.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.439+0000 7f626b451640 1 -- 172.21.15.115:0/2827339428 wait complete. 2023-12-01T03:11:00.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.441+0000 7f626b451640 1 Processor -- start 2023-12-01T03:11:00.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.442+0000 7f626b451640 1 -- start start 2023-12-01T03:11:00.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f626b451640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62640b4070 0x7f62641ed930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:00.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f626b451640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62640cb8c0 0x7f62640c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:00.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f626b451640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626415ac70 0x7f62640c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:00.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f626b451640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640c1e10 con 0x7f62640b4070 2023-12-01T03:11:00.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f626b451640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62641431f0 con 0x7f626415ac70 2023-12-01T03:11:00.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f626b451640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62641425e0 con 0x7f62640cb8c0 2023-12-01T03:11:00.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f62691ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62640b4070 0x7f62641ed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:00.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f62691ed640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62640b4070 0x7f62641ed930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46892/0 (socket says 172.21.15.115:46892) 2023-12-01T03:11:00.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f62691ed640 1 -- 172.21.15.115:0/2080981272 learned_addr learned my addr 172.21.15.115:0/2080981272 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:00.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f62691ed640 1 -- 172.21.15.115:0/2080981272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62640cb8c0 msgr2=0x7f62640c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:00.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f62691ed640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62640cb8c0 0x7f62640c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f62691ed640 1 -- 172.21.15.115:0/2080981272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626415ac70 msgr2=0x7f62640c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:00.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f62691ed640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626415ac70 0x7f62640c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.443+0000 7f62691ed640 1 -- 172.21.15.115:0/2080981272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254009400 con 0x7f62640b4070 2023-12-01T03:11:00.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.444+0000 7f62691ed640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62640b4070 0x7f62641ed930 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f626000c430 tx=0x7f626000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:00.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.445+0000 7f6259ffb640 1 -- 172.21.15.115:0/2080981272 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260005ce0 con 0x7f62640b4070 2023-12-01T03:11:00.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.445+0000 7f6259ffb640 1 -- 172.21.15.115:0/2080981272 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6260005e40 con 0x7f62640b4070 2023-12-01T03:11:00.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.445+0000 7f6259ffb640 1 -- 172.21.15.115:0/2080981272 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260007310 con 0x7f62640b4070 2023-12-01T03:11:00.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.445+0000 7f626b451640 1 -- 172.21.15.115:0/2080981272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625c009a70 con 0x7f62640b4070 2023-12-01T03:11:00.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.445+0000 7f626b451640 1 -- 172.21.15.115:0/2080981272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6264142ba0 con 0x7f62640b4070 2023-12-01T03:11:00.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.446+0000 7f626b451640 1 -- 172.21.15.115:0/2080981272 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6264074210 con 0x7f62640b4070 2023-12-01T03:11:00.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.447+0000 7f6259ffb640 1 -- 172.21.15.115:0/2080981272 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 128) v1 ==== 88829+0+0 (secure 0 0 0) 0x7f6260007470 con 0x7f62640b4070 2023-12-01T03:11:00.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.448+0000 7f6259ffb640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f624806a6c0 0x7f624806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:00.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.448+0000 7f6259ffb640 1 -- 172.21.15.115:0/2080981272 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f62600888f0 con 0x7f62640b4070 2023-12-01T03:11:00.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.448+0000 7f62689ec640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f624806a6c0 0x7f624806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:00.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.449+0000 7f62689ec640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f624806a6c0 0x7f624806cb80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6254006e80 tx=0x7f6254006dd0 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:00.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.452+0000 7f6259ffb640 1 -- 172.21.15.115:0/2080981272 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6260053670 con 0x7f62640b4070 2023-12-01T03:11:00.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.462+0000 7f6259ffb640 1 -- 172.21.15.115:0/2080981272 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 129) v1 ==== 88885+0+0 (secure 0 0 0) 0x7f6260053a40 con 0x7f62640b4070 2023-12-01T03:11:00.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:11:00.550 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:11:00.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.671+0000 7f626b451640 1 -- 172.21.15.115:0/2080981272 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f626406cde0 con 0x7f624806a6c0 2023-12-01T03:11:00.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.805+0000 7f6259ffb640 1 -- 172.21.15.115:0/2080981272 <== mgr.10977 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f6254009830 con 0x7f624806a6c0 2023-12-01T03:11:00.807 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path mandatory_fs_argument_test_dir 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.808+0000 7f626b451640 1 -- 172.21.15.115:0/2080981272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f624806a6c0 msgr2=0x7f624806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.808+0000 7f626b451640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f624806a6c0 0x7f624806cb80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6254006e80 tx=0x7f6254006dd0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.808+0000 7f626b451640 1 -- 172.21.15.115:0/2080981272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62640b4070 msgr2=0x7f62641ed930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.808+0000 7f626b451640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62640b4070 0x7f62641ed930 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f626000c430 tx=0x7f626000c8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.809+0000 7f626b451640 1 -- 172.21.15.115:0/2080981272 shutdown_connections 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.809+0000 7f626b451640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f624806a6c0 0x7f624806cb80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.809+0000 7f626b451640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62640cb8c0 0x7f62640c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.809+0000 7f626b451640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626415ac70 0x7f62640c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.809+0000 7f626b451640 1 --2- 172.21.15.115:0/2080981272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62640b4070 0x7f62641ed930 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.809+0000 7f626b451640 1 -- 172.21.15.115:0/2080981272 >> 172.21.15.115:0/2080981272 conn(0x7f62640662c0 msgr2=0x7f62640cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.809+0000 7f626b451640 1 -- 172.21.15.115:0/2080981272 shutdown_connections 2023-12-01T03:11:00.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:00.809+0000 7f626b451640 1 -- 172.21.15.115:0/2080981272 wait complete. 2023-12-01T03:11:00.831 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path mandatory_fs_argument_test_dir 2023-12-01T03:11:00.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:00.831 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-12-01T03:11:00.937 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T03:11:01.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.151+0000 7f3be4012640 1 Processor -- start 2023-12-01T03:11:01.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.151+0000 7f3be4012640 1 -- start start 2023-12-01T03:11:01.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.153+0000 7f3be4012640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3bdc15cd90 0x7f3bdc0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:01.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.153+0000 7f3be4012640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3bdc0b5fa0 0x7f3bdc161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:01.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.153+0000 7f3be4012640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bdc0ce5e0 con 0x7f3bdc0cf650 2023-12-01T03:11:01.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.153+0000 7f3be4012640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc0d0ec0 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.153+0000 7f3be4012640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc161950 con 0x7f3bdc15cd90 2023-12-01T03:11:01.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.153+0000 7f3be15ad640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3bdc0b5fa0 0x7f3bdc161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:01.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.153+0000 7f3be15ad640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3bdc0b5fa0 0x7f3bdc161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46898/0 (socket says 172.21.15.115:46898) 2023-12-01T03:11:01.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.153+0000 7f3be15ad640 1 -- 172.21.15.115:0/795457546 learned_addr learned my addr 172.21.15.115:0/795457546 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:01.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.154+0000 7f3be15ad640 1 -- 172.21.15.115:0/795457546 >> v2:172.21.15.142:3300/0 conn(0x7f3bdc15cd90 msgr2=0x7f3bdc0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:01.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.154+0000 7f3be15ad640 1 --2- 172.21.15.115:0/795457546 >> v2:172.21.15.142:3300/0 conn(0x7f3bdc15cd90 0x7f3bdc0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:01.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.154+0000 7f3be15ad640 1 -- 172.21.15.115:0/795457546 >> v1:172.21.15.142:6789/0 conn(0x7f3bdc0cf650 legacy=0x7f3bdc0cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:01.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.154+0000 7f3be15ad640 1 -- 172.21.15.115:0/795457546 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc0ce300 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.155+0000 7f3be15ad640 1 --2- 172.21.15.115:0/795457546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 0x7f3bdc161410 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc06bd10 tx=0x7f3bd002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e86545cd001c3e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:01.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.155+0000 7f3be0dac640 1 -- 172.21.15.115:0/795457546 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0031870 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.155+0000 7f3be0dac640 1 -- 172.21.15.115:0/795457546 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd00319d0 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.155+0000 7f3be0dac640 1 -- 172.21.15.115:0/795457546 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0031d60 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.156+0000 7f3be4012640 1 -- 172.21.15.115:0/795457546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 msgr2=0x7f3bdc161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:01.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.156+0000 7f3be4012640 1 --2- 172.21.15.115:0/795457546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 0x7f3bdc161410 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc06bd10 tx=0x7f3bd002b910 comp rx=0 tx=0).stop 2023-12-01T03:11:01.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.156+0000 7f3be4012640 1 -- 172.21.15.115:0/795457546 shutdown_connections 2023-12-01T03:11:01.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.156+0000 7f3be4012640 1 --2- 172.21.15.115:0/795457546 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 0x7f3bdc161410 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:01.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.156+0000 7f3be4012640 1 --2- 172.21.15.115:0/795457546 >> v2:172.21.15.142:3300/0 conn(0x7f3bdc15cd90 0x7f3bdc0cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:01.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.156+0000 7f3be4012640 1 -- 172.21.15.115:0/795457546 >> 172.21.15.115:0/795457546 conn(0x7f3bdc065960 msgr2=0x7f3bdc065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:01.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.156+0000 7f3be4012640 1 -- 172.21.15.115:0/795457546 shutdown_connections 2023-12-01T03:11:01.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.156+0000 7f3be4012640 1 -- 172.21.15.115:0/795457546 wait complete. 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.157+0000 7f3be4012640 1 Processor -- start 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.158+0000 7f3be4012640 1 -- start start 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.158+0000 7f3be4012640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 0x7f3bdc0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be4012640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bdc0cf650 0x7f3bdc153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be4012640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bdc15cd90 0x7f3bdc154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be4012640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0cd0b0 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be4012640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc153950 con 0x7f3bdc0cf650 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be4012640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0ccdd0 con 0x7f3bdc15cd90 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be1dae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 0x7f3bdc0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be1dae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 0x7f3bdc0ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46900/0 (socket says 172.21.15.115:46900) 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be1dae640 1 -- 172.21.15.115:0/3922571776 learned_addr learned my addr 172.21.15.115:0/3922571776 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be15ad640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bdc0cf650 0x7f3bdc153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be1dae640 1 -- 172.21.15.115:0/3922571776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bdc15cd90 msgr2=0x7f3bdc154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be1dae640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bdc15cd90 0x7f3bdc154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be1dae640 1 -- 172.21.15.115:0/3922571776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bdc0cf650 msgr2=0x7f3bdc153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be1dae640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bdc0cf650 0x7f3bdc153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.159+0000 7f3be1dae640 1 -- 172.21.15.115:0/3922571776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00094d0 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.160+0000 7f3be1dae640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 0x7f3bdc0ca9a0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00ba50 tx=0x7f3bcc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.160+0000 7f3bda7fc640 1 -- 172.21.15.115:0/3922571776 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00cbe0 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.160+0000 7f3bda7fc640 1 -- 172.21.15.115:0/3922571776 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc00cd60 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.160+0000 7f3bda7fc640 1 -- 172.21.15.115:0/3922571776 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc014680 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.160+0000 7f3be4012640 1 -- 172.21.15.115:0/3922571776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bdc0c9ec0 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.160+0000 7f3be4012640 1 -- 172.21.15.115:0/3922571776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc0ca480 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.161+0000 7f3be4012640 1 -- 172.21.15.115:0/3922571776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bdc04e890 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.162+0000 7f3bda7fc640 1 -- 172.21.15.115:0/3922571776 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 129) v1 ==== 88885+0+0 (secure 0 0 0) 0x7f3bcc01d3b0 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.163+0000 7f3bda7fc640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bc806a750 0x7f3bc806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:01.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.163+0000 7f3bda7fc640 1 -- 172.21.15.115:0/3922571776 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f3bcc089310 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.163+0000 7f3be15ad640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bc806a750 0x7f3bc806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:01.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.164+0000 7f3be15ad640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bc806a750 0x7f3bc806cc10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0036000 tx=0x7f3bd003c040 comp rx=0 tx=0).ready entity=mgr.10977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.168+0000 7f3bda7fc640 1 -- 172.21.15.115:0/3922571776 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3bcc054070 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.408+0000 7f3be4012640 1 -- 172.21.15.115:0/3922571776 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f3bdc0ca670 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.473+0000 7f3bda7fc640 1 -- 172.21.15.115:0/3922571776 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 130) v1 ==== 88868+0+0 (secure 0 0 0) 0x7f3bcc054440 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.474+0000 7f3bda7fc640 1 -- 172.21.15.115:0/3922571776 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v130) v1 ==== 93+0+0 (secure 0 0 0) 0x7f3bcc050880 con 0x7f3bdc0b5fa0 2023-12-01T03:11:01.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.476+0000 7f3be4012640 1 -- 172.21.15.115:0/3922571776 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bc806a750 msgr2=0x7f3bc806cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:01.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.476+0000 7f3be4012640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bc806a750 0x7f3bc806cc10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0036000 tx=0x7f3bd003c040 comp rx=0 tx=0).stop 2023-12-01T03:11:01.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.476+0000 7f3be4012640 1 -- 172.21.15.115:0/3922571776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 msgr2=0x7f3bdc0ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:01.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.476+0000 7f3be4012640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 0x7f3bdc0ca9a0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00ba50 tx=0x7f3bcc00bf10 comp rx=0 tx=0).stop 2023-12-01T03:11:01.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.476+0000 7f3be4012640 1 -- 172.21.15.115:0/3922571776 shutdown_connections 2023-12-01T03:11:01.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.476+0000 7f3be4012640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bc806a750 0x7f3bc806cc10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:01.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.476+0000 7f3be4012640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bdc15cd90 0x7f3bdc154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:01.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.476+0000 7f3be4012640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bdc0cf650 0x7f3bdc153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:01.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.476+0000 7f3be4012640 1 --2- 172.21.15.115:0/3922571776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bdc0b5fa0 0x7f3bdc0ca9a0 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:01.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.476+0000 7f3be4012640 1 -- 172.21.15.115:0/3922571776 >> 172.21.15.115:0/3922571776 conn(0x7f3bdc065960 msgr2=0x7f3bdc0b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:01.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.477+0000 7f3be4012640 1 -- 172.21.15.115:0/3922571776 shutdown_connections 2023-12-01T03:11:01.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:01.477+0000 7f3be4012640 1 -- 172.21.15.115:0/3922571776 wait complete. 2023-12-01T03:11:01.498 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T03:11:01.498 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:11:01.620 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:11:01.722 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T03:11:01.722 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:11:01.722 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:11:01.774 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:11:01.775 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:11:01.822 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:01.986 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:01.985+0000 7f5690638040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:11:02.013 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:02.012+0000 7f16dfa15040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:11:02.038 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:02.037+0000 7f5690638040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:11:02.069 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:02.067+0000 7f16dfa15040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:11:02.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.069+0000 7ff140961640 1 Processor -- start 2023-12-01T03:11:02.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.070+0000 7ff140961640 1 -- start start 2023-12-01T03:11:02.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.071+0000 7ff140961640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff13c15d7b0 0x7ff13c15db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.071+0000 7ff140961640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c160670 0x7ff13c0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.071+0000 7ff140961640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff13c0cddc0 0x7ff13c0d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.071+0000 7ff140961640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13c0cd3f0 con 0x7ff13c0cddc0 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.071+0000 7ff140961640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13c0cd0f0 con 0x7ff13c15d7b0 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.071+0000 7ff140961640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c160ec0 con 0x7ff13c160670 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.071+0000 7ff13bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff13c0cddc0 0x7ff13c0d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.071+0000 7ff13bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff13c0cddc0 0x7ff13c0d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46904/0 (socket says 172.21.15.115:46904) 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.071+0000 7ff13bfff640 1 -- 172.21.15.115:0/3161917413 learned_addr learned my addr 172.21.15.115:0/3161917413 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.072+0000 7ff13b7fe640 1 --2- 172.21.15.115:0/3161917413 >> v2:172.21.15.142:3300/0 conn(0x7ff13c15d7b0 0x7ff13c15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.072+0000 7ff13affd640 1 --2- 172.21.15.115:0/3161917413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c160670 0x7ff13c0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.072+0000 7ff13bfff640 1 -- 172.21.15.115:0/3161917413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c160670 msgr2=0x7ff13c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.072+0000 7ff13bfff640 1 --2- 172.21.15.115:0/3161917413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c160670 0x7ff13c0cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.072+0000 7ff13bfff640 1 -- 172.21.15.115:0/3161917413 >> v2:172.21.15.142:3300/0 conn(0x7ff13c15d7b0 msgr2=0x7ff13c15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.072+0000 7ff13bfff640 1 --2- 172.21.15.115:0/3161917413 >> v2:172.21.15.142:3300/0 conn(0x7ff13c15d7b0 0x7ff13c15db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.072+0000 7ff13bfff640 1 -- 172.21.15.115:0/3161917413 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c1611a0 con 0x7ff13c0cddc0 2023-12-01T03:11:02.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.073+0000 7ff13bfff640 1 --2- 172.21.15.115:0/3161917413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c0cddc0 0x7ff13c0d05d0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0720e0 tx=0x7ff13402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749687fd3b1bc79d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.073+0000 7ff13a7fc640 1 -- 172.21.15.115:0/3161917413 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13402cb00 con 0x7ff13c0cddc0 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.073+0000 7ff13a7fc640 1 -- 172.21.15.115:0/3161917413 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13402cc60 con 0x7ff13c0cddc0 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.073+0000 7ff13a7fc640 1 -- 172.21.15.115:0/3161917413 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1340343f0 con 0x7ff13c0cddc0 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.074+0000 7ff140961640 1 -- 172.21.15.115:0/3161917413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c0cddc0 msgr2=0x7ff13c0d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.074+0000 7ff140961640 1 --2- 172.21.15.115:0/3161917413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c0cddc0 0x7ff13c0d05d0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0720e0 tx=0x7ff13402bf60 comp rx=0 tx=0).stop 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.074+0000 7ff140961640 1 -- 172.21.15.115:0/3161917413 shutdown_connections 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.074+0000 7ff140961640 1 --2- 172.21.15.115:0/3161917413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c0cddc0 0x7ff13c0d05d0 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.074+0000 7ff140961640 1 --2- 172.21.15.115:0/3161917413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c160670 0x7ff13c0cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.074+0000 7ff140961640 1 --2- 172.21.15.115:0/3161917413 >> v2:172.21.15.142:3300/0 conn(0x7ff13c15d7b0 0x7ff13c15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.074+0000 7ff140961640 1 -- 172.21.15.115:0/3161917413 >> 172.21.15.115:0/3161917413 conn(0x7ff13c06a0e0 msgr2=0x7ff13c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.074+0000 7ff140961640 1 -- 172.21.15.115:0/3161917413 shutdown_connections 2023-12-01T03:11:02.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.074+0000 7ff140961640 1 -- 172.21.15.115:0/3161917413 wait complete. 2023-12-01T03:11:02.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.076+0000 7ff140961640 1 Processor -- start 2023-12-01T03:11:02.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.076+0000 7ff140961640 1 -- start start 2023-12-01T03:11:02.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.076+0000 7ff140961640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0cddc0 0x7ff13c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff140961640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c15d7b0 0x7ff13c15f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff140961640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c160670 0x7ff13c15faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff140961640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c1f6580 con 0x7ff13c15d7b0 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff140961640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c155e00 con 0x7ff13c0cddc0 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff140961640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c1f59b0 con 0x7ff13c160670 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c15d7b0 0x7ff13c15f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c15d7b0 0x7ff13c15f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46908/0 (socket says 172.21.15.115:46908) 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13affd640 1 -- 172.21.15.115:0/54710232 learned_addr learned my addr 172.21.15.115:0/54710232 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13b7fe640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0cddc0 0x7ff13c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13affd640 1 -- 172.21.15.115:0/54710232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c160670 msgr2=0x7ff13c15faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13affd640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c160670 0x7ff13c15faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13affd640 1 -- 172.21.15.115:0/54710232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0cddc0 msgr2=0x7ff13c1558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13affd640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0cddc0 0x7ff13c1558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13affd640 1 -- 172.21.15.115:0/54710232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff134009b30 con 0x7ff13c15d7b0 2023-12-01T03:11:02.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13b7fe640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0cddc0 0x7ff13c1558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:02.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.077+0000 7ff13affd640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c15d7b0 0x7ff13c15f5b0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7ff13000d5f0 tx=0x7ff13000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:02.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.078+0000 7ff123fff640 1 -- 172.21.15.115:0/54710232 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1300125f0 con 0x7ff13c15d7b0 2023-12-01T03:11:02.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.078+0000 7ff123fff640 1 -- 172.21.15.115:0/54710232 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff130012750 con 0x7ff13c15d7b0 2023-12-01T03:11:02.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.078+0000 7ff140961640 1 -- 172.21.15.115:0/54710232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c1f5c50 con 0x7ff13c15d7b0 2023-12-01T03:11:02.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.078+0000 7ff123fff640 1 -- 172.21.15.115:0/54710232 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130012b00 con 0x7ff13c15d7b0 2023-12-01T03:11:02.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.078+0000 7ff140961640 1 -- 172.21.15.115:0/54710232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c1f6210 con 0x7ff13c15d7b0 2023-12-01T03:11:02.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.079+0000 7ff140961640 1 -- 172.21.15.115:0/54710232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c1f63d0 con 0x7ff13c15d7b0 2023-12-01T03:11:02.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.080+0000 7ff123fff640 1 -- 172.21.15.115:0/54710232 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 130) v1 ==== 88868+0+0 (secure 0 0 0) 0x7ff130012c60 con 0x7ff13c15d7b0 2023-12-01T03:11:02.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.081+0000 7ff123fff640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff12806a700 0x7ff12806cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.081+0000 7ff123fff640 1 -- 172.21.15.115:0/54710232 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7ff1300896b0 con 0x7ff13c15d7b0 2023-12-01T03:11:02.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.081+0000 7ff13b7fe640 1 -- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff12806a700 msgr2=0x7ff12806cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:02.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.081+0000 7ff13b7fe640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff12806a700 0x7ff12806cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:02.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.085+0000 7ff123fff640 1 -- 172.21.15.115:0/54710232 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff130054420 con 0x7ff13c15d7b0 2023-12-01T03:11:02.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.278+0000 7ff140961640 1 -- 172.21.15.115:0/54710232 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff13c0673a0 con 0x7ff13c15d7b0 2023-12-01T03:11:02.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.280+0000 7ff123fff640 1 -- 172.21.15.115:0/54710232 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v219) v1 ==== 74+0+20333 (secure 0 0 0) 0x7ff130057a40 con 0x7ff13c15d7b0 2023-12-01T03:11:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:02.283 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":219,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:10:55.627426+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:10:37.180599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:10:37.961783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:6822/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/2913786291":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/846563507":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3736942314":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/1606716517":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/2954409059":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3112535338":"2023-12-02T03:10:55.627314+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:02.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.282+0000 7ff13b7fe640 1 -- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff12806a700 msgr2=0x7ff12806cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:02.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.282+0000 7ff13b7fe640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff12806a700 0x7ff12806cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:11:02.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 -- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff12806a700 msgr2=0x7ff12806cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:11:02.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff12806a700 0x7ff12806cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 -- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c15d7b0 msgr2=0x7ff13c15f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c15d7b0 0x7ff13c15f5b0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7ff13000d5f0 tx=0x7ff13000dab0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 -- 172.21.15.115:0/54710232 shutdown_connections 2023-12-01T03:11:02.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff12806a700 0x7ff12806cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff13c160670 0x7ff13c15faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff13c15d7b0 0x7ff13c15f5b0 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 --2- 172.21.15.115:0/54710232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff13c0cddc0 0x7ff13c1558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 -- 172.21.15.115:0/54710232 >> 172.21.15.115:0/54710232 conn(0x7ff13c06a0e0 msgr2=0x7ff13c0b83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:02.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 -- 172.21.15.115:0/54710232 shutdown_connections 2023-12-01T03:11:02.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.283+0000 7ff140961640 1 -- 172.21.15.115:0/54710232 wait complete. 2023-12-01T03:11:02.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T03:10:22.683496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '213', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T03:10:24.156854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '214', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-01T03:10:37.180599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '217', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-01T03:10:37.961783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '218', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:11:02.306 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:11:02.488 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:02.487+0000 7f5690638040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:11:02.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.513+0000 7f4f2244a640 1 Processor -- start 2023-12-01T03:11:02.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.513+0000 7f4f2244a640 1 -- start start 2023-12-01T03:11:02.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.514+0000 7f4f2244a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f1c0b5fa0 0x7f4f1c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.514+0000 7f4f2244a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c0ce5e0 con 0x7f4f1c15cd90 2023-12-01T03:11:02.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.515+0000 7f4f2244a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c0ce300 con 0x7f4f1c0cfd60 2023-12-01T03:11:02.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.515+0000 7f4f2244a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0d0ec0 con 0x7f4f1c0b5fa0 2023-12-01T03:11:02.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.515+0000 7f4f1bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4f1c15cd90 0x7f4f1c0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58450/0 (socket says 172.21.15.115:58450) 2023-12-01T03:11:02.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.515+0000 7f4f1bfff640 1 -- 172.21.15.115:0/2086262206 learned_addr learned my addr 172.21.15.115:0/2086262206 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:02.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.515+0000 7f4f209e7640 1 --2- 172.21.15.115:0/2086262206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f1c0b5fa0 0x7f4f1c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.515+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027024871 0 0) 0x7f4f0c001110 con 0x7f4f1c15cd90 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.515+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f08003520 con 0x7f4f1c15cd90 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757562942 0 0) 0x7f4f1c0ce300 con 0x7f4f1c0cfd60 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f0c001110 con 0x7f4f1c0cfd60 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2014694083 0 0) 0x7f4f08003520 con 0x7f4f1c15cd90 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f1c0b5fa0 msgr2=0x7f4f1c0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 --2- 172.21.15.115:0/2086262206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f1c0b5fa0 0x7f4f1c0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 >> v1:172.21.15.142:6789/0 conn(0x7f4f1c0cfd60 legacy=0x7f4f1c0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c0d10a0 con 0x7f4f1c15cd90 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4f0c001800 con 0x7f4f1c15cd90 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 >> v1:172.21.15.115:6789/0 conn(0x7f4f1c15cd90 legacy=0x7f4f1c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 --2- 172.21.15.115:0/2086262206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f08003b30 0x7f4f08003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 --2- 172.21.15.115:0/2086262206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f080047c0 0x7f4f08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 --2- 172.21.15.115:0/2086262206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f1c0b5fa0 0x7f4f08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0d0ec0 con 0x7f4f08003b30 2023-12-01T03:11:02.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0800ba30 con 0x7f4f080047c0 2023-12-01T03:11:02.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0800bbf0 con 0x7f4f1c0b5fa0 2023-12-01T03:11:02.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.516+0000 7f4f1bfff640 1 --2- 172.21.15.115:0/2086262206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f08003b30 0x7f4f08003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.517+0000 7f4f1bfff640 1 -- 172.21.15.115:0/2086262206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f1c0b5fa0 msgr2=0x7f4f08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:02.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.517+0000 7f4f1bfff640 1 --2- 172.21.15.115:0/2086262206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f1c0b5fa0 0x7f4f08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.517+0000 7f4f1bfff640 1 -- 172.21.15.115:0/2086262206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f080047c0 msgr2=0x7f4f08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:02.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.517+0000 7f4f1bfff640 1 --2- 172.21.15.115:0/2086262206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f080047c0 0x7f4f08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.517+0000 7f4f1bfff640 1 -- 172.21.15.115:0/2086262206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f08003520 con 0x7f4f08003b30 2023-12-01T03:11:02.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.518+0000 7f4f1bfff640 1 --2- 172.21.15.115:0/2086262206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f08003b30 0x7f4f08003f30 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c007fa0 tx=0x7f4f080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c7f9fe9ebc2c32 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:02.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.518+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c005ad0 con 0x7f4f08003b30 2023-12-01T03:11:02.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.518+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c005c30 con 0x7f4f08003b30 2023-12-01T03:11:02.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.518+0000 7f4f1b7fe640 1 -- 172.21.15.115:0/2086262206 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c02c790 con 0x7f4f08003b30 2023-12-01T03:11:02.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.519+0000 7f4f2244a640 1 -- 172.21.15.115:0/2086262206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f08003b30 msgr2=0x7f4f08003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.519+0000 7f4f2244a640 1 --2- 172.21.15.115:0/2086262206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f08003b30 0x7f4f08003f30 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c007fa0 tx=0x7f4f080057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.519+0000 7f4f209e7640 1 -- 172.21.15.115:0/2086262206 reap_dead start 2023-12-01T03:11:02.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.519+0000 7f4f2244a640 1 -- 172.21.15.115:0/2086262206 shutdown_connections 2023-12-01T03:11:02.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.519+0000 7f4f2244a640 1 -- 172.21.15.115:0/2086262206 >> 172.21.15.115:0/2086262206 conn(0x7f4f1c065960 msgr2=0x7f4f1c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:02.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.519+0000 7f4f2244a640 1 -- 172.21.15.115:0/2086262206 shutdown_connections 2023-12-01T03:11:02.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.519+0000 7f4f2244a640 1 -- 172.21.15.115:0/2086262206 wait complete. 2023-12-01T03:11:02.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.521+0000 7f4f2244a640 1 Processor -- start 2023-12-01T03:11:02.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.521+0000 7f4f2244a640 1 -- start start 2023-12-01T03:11:02.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.522+0000 7f4f2244a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f08003b30 0x7f4f1c1f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.522+0000 7f4f2244a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f1c1f53f0 0x7f4f1c152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.522+0000 7f4f2244a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f1c1530d0 0x7f4f1c0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.522+0000 7f4f2244a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0c93b0 con 0x7f4f1c1530d0 2023-12-01T03:11:02.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.522+0000 7f4f2244a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c153550 con 0x7f4f1c1f53f0 2023-12-01T03:11:02.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.522+0000 7f4f2244a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1e6b80 con 0x7f4f08003b30 2023-12-01T03:11:02.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.522+0000 7f4f209e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f1c1530d0 0x7f4f1c0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.523+0000 7f4f209e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f1c1530d0 0x7f4f1c0cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46940/0 (socket says 172.21.15.115:46940) 2023-12-01T03:11:02.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.523+0000 7f4f209e7640 1 -- 172.21.15.115:0/3608557468 learned_addr learned my addr 172.21.15.115:0/3608557468 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:02.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.523+0000 7f4f13fff640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f1c1f53f0 0x7f4f1c152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.523+0000 7f4f1bfff640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f08003b30 0x7f4f1c1f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.523+0000 7f4f209e7640 1 -- 172.21.15.115:0/3608557468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f08003b30 msgr2=0x7f4f1c1f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.523+0000 7f4f209e7640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f08003b30 0x7f4f1c1f2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.523+0000 7f4f209e7640 1 -- 172.21.15.115:0/3608557468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f1c1f53f0 msgr2=0x7f4f1c152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.523+0000 7f4f209e7640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f1c1f53f0 0x7f4f1c152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.523+0000 7f4f209e7640 1 -- 172.21.15.115:0/3608557468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c1f5850 con 0x7f4f1c1530d0 2023-12-01T03:11:02.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.523+0000 7f4f209e7640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f1c1530d0 0x7f4f1c0cb7a0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c071ac0 tx=0x7f4f1400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:02.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.524+0000 7f4f197fa640 1 -- 172.21.15.115:0/3608557468 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14010040 con 0x7f4f1c1530d0 2023-12-01T03:11:02.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.524+0000 7f4f197fa640 1 -- 172.21.15.115:0/3608557468 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1400bce0 con 0x7f4f1c1530d0 2023-12-01T03:11:02.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.524+0000 7f4f197fa640 1 -- 172.21.15.115:0/3608557468 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14014640 con 0x7f4f1c1530d0 2023-12-01T03:11:02.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.524+0000 7f4f2244a640 1 -- 172.21.15.115:0/3608557468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c0cc560 con 0x7f4f1c1530d0 2023-12-01T03:11:02.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.524+0000 7f4f2244a640 1 -- 172.21.15.115:0/3608557468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c1e70b0 con 0x7f4f1c1530d0 2023-12-01T03:11:02.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.525+0000 7f4f2244a640 1 -- 172.21.15.115:0/3608557468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c04e890 con 0x7f4f1c1530d0 2023-12-01T03:11:02.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.526+0000 7f4f197fa640 1 -- 172.21.15.115:0/3608557468 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 130) v1 ==== 88868+0+0 (secure 0 0 0) 0x7f4f140147a0 con 0x7f4f1c1530d0 2023-12-01T03:11:02.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.527+0000 7f4f197fa640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4efc06a700 0x7f4efc06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.527+0000 7f4f197fa640 1 -- 172.21.15.115:0/3608557468 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f4f14089a70 con 0x7f4f1c1530d0 2023-12-01T03:11:02.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.527+0000 7f4f1bfff640 1 -- 172.21.15.115:0/3608557468 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4efc06a700 msgr2=0x7f4efc06cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:02.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.527+0000 7f4f1bfff640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4efc06a700 0x7f4efc06cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:02.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.531+0000 7f4f197fa640 1 -- 172.21.15.115:0/3608557468 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4f140547a0 con 0x7f4f1c1530d0 2023-12-01T03:11:02.543 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:02.542+0000 7f16dfa15040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:11:02.545 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:02.544+0000 7f5690638040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:11:02.595 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:02.595+0000 7f5690638040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:11:02.598 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:02.596+0000 7f16dfa15040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:11:02.650 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:02.648+0000 7f16dfa15040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:11:02.656 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:02.656+0000 7f5690638040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:11:02.706 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:02.705+0000 7f5690638040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:11:02.712 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:02.711+0000 7f16dfa15040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:11:02.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.723+0000 7f4f2244a640 1 -- 172.21.15.115:0/3608557468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f1c0d0a70 con 0x7f4f1c1530d0 2023-12-01T03:11:02.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.724+0000 7f4f197fa640 1 -- 172.21.15.115:0/3608557468 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v219) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f14057dc0 con 0x7f4f1c1530d0 2023-12-01T03:11:02.726 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:11:02.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.726+0000 7f4f2244a640 1 -- 172.21.15.115:0/3608557468 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4efc06a700 msgr2=0x7f4efc06cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:11:02.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.726+0000 7f4f2244a640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4efc06a700 0x7f4efc06cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.726+0000 7f4f2244a640 1 -- 172.21.15.115:0/3608557468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f1c1530d0 msgr2=0x7f4f1c0cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.726+0000 7f4f2244a640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f1c1530d0 0x7f4f1c0cb7a0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c071ac0 tx=0x7f4f1400b480 comp rx=0 tx=0).stop 2023-12-01T03:11:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.726+0000 7f4f2244a640 1 -- 172.21.15.115:0/3608557468 shutdown_connections 2023-12-01T03:11:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.726+0000 7f4f2244a640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4efc06a700 0x7f4efc06cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.726+0000 7f4f2244a640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f1c1530d0 0x7f4f1c0cb7a0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.726+0000 7f4f2244a640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f1c1f53f0 0x7f4f1c152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.727+0000 7f4f2244a640 1 --2- 172.21.15.115:0/3608557468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f08003b30 0x7f4f1c1f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.727+0000 7f4f2244a640 1 -- 172.21.15.115:0/3608557468 >> 172.21.15.115:0/3608557468 conn(0x7f4f1c065960 msgr2=0x7f4f1c0d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.727+0000 7f4f2244a640 1 -- 172.21.15.115:0/3608557468 shutdown_connections 2023-12-01T03:11:02.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.727+0000 7f4f2244a640 1 -- 172.21.15.115:0/3608557468 wait complete. 2023-12-01T03:11:02.747 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:11:02.766 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:02.764+0000 7f16dfa15040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:11:02.817 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:02.816+0000 7f5690638040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:11:02.880 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:02.879+0000 7f5690638040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:11:02.883 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:02.881+0000 7f16dfa15040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:11:02.930 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:02.929+0000 7f5690638040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:11:02.948 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:02.946+0000 7f16dfa15040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:11:02.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.949+0000 7fcd559fa640 1 Processor -- start 2023-12-01T03:11:02.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.950+0000 7fcd559fa640 1 -- start start 2023-12-01T03:11:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.951+0000 7fcd559fa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd500b2b30 0x7fcd5006b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.951+0000 7fcd559fa640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd50069d60 con 0x7fcd50071460 2023-12-01T03:11:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.951+0000 7fcd559fa640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5006a000 con 0x7fcd5015abb0 2023-12-01T03:11:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.951+0000 7fcd559fa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd5015af90 con 0x7fcd500b2b30 2023-12-01T03:11:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.951+0000 7fcd551f9640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fcd50071460 0x7fcd5015d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58458/0 (socket says 172.21.15.115:58458) 2023-12-01T03:11:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.951+0000 7fcd549f8640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fcd5015abb0 0x7fcd500b2420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:44560/0 (socket says 172.21.15.115:44560) 2023-12-01T03:11:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.951+0000 7fcd551f9640 1 -- 172.21.15.115:0/468589518 learned_addr learned my addr 172.21.15.115:0/468589518 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:02.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.951+0000 7fcd4ffff640 1 --2- 172.21.15.115:0/468589518 >> v2:172.21.15.142:3300/0 conn(0x7fcd500b2b30 0x7fcd5006b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460816063 0 0) 0x7fcd50069d60 con 0x7fcd50071460 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd40003520 con 0x7fcd50071460 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815866442 0 0) 0x7fcd5006a000 con 0x7fcd5015abb0 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd50069d60 con 0x7fcd5015abb0 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2456876819 0 0) 0x7fcd40003520 con 0x7fcd50071460 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 >> v2:172.21.15.142:3300/0 conn(0x7fcd500b2b30 msgr2=0x7fcd5006b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 --2- 172.21.15.115:0/468589518 >> v2:172.21.15.142:3300/0 conn(0x7fcd500b2b30 0x7fcd5006b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 >> v1:172.21.15.142:6789/0 conn(0x7fcd5015abb0 legacy=0x7fcd500b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5006a2e0 con 0x7fcd50071460 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fcd480019f0 con 0x7fcd50071460 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 >> v1:172.21.15.115:6789/0 conn(0x7fcd50071460 legacy=0x7fcd5015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd40004bb0 0x7fcd40004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd40003760 0x7fcd40003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd40004120 0x7fcd4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5015af90 con 0x7fcd40004120 2023-12-01T03:11:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000c080 con 0x7fcd40004bb0 2023-12-01T03:11:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.952+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000c240 con 0x7fcd40003760 2023-12-01T03:11:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.953+0000 7fcd551f9640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd40004120 0x7fcd4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.953+0000 7fcd549f8640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd40004bb0 0x7fcd40004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.953+0000 7fcd4ffff640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd40003760 0x7fcd40003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.953+0000 7fcd551f9640 1 -- 172.21.15.115:0/468589518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd40003760 msgr2=0x7fcd40003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.953+0000 7fcd551f9640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd40003760 0x7fcd40003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.953+0000 7fcd551f9640 1 -- 172.21.15.115:0/468589518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd40004bb0 msgr2=0x7fcd40004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.953+0000 7fcd551f9640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd40004bb0 0x7fcd40004fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.953+0000 7fcd551f9640 1 -- 172.21.15.115:0/468589518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40003520 con 0x7fcd40004120 2023-12-01T03:11:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.954+0000 7fcd551f9640 1 -- 172.21.15.115:0/468589518 reap_dead start 2023-12-01T03:11:02.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.954+0000 7fcd551f9640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd40004120 0x7fcd4000b980 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800a710 tx=0x7fcd400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219dd94c7dd39b65 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:02.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.955+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48034630 con 0x7fcd40004120 2023-12-01T03:11:02.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.955+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4802f070 con 0x7fcd40004120 2023-12-01T03:11:02.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.955+0000 7fcd4f7fe640 1 -- 172.21.15.115:0/468589518 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48034a10 con 0x7fcd40004120 2023-12-01T03:11:02.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.955+0000 7fcd559fa640 1 -- 172.21.15.115:0/468589518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd40004120 msgr2=0x7fcd4000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.955+0000 7fcd559fa640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd40004120 0x7fcd4000b980 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800a710 tx=0x7fcd400057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.955+0000 7fcd559fa640 1 -- 172.21.15.115:0/468589518 shutdown_connections 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.955+0000 7fcd559fa640 1 --2- 172.21.15.115:0/468589518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd40004120 0x7fcd4000b980 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.955+0000 7fcd559fa640 1 -- 172.21.15.115:0/468589518 >> 172.21.15.115:0/468589518 conn(0x7fcd500cd420 msgr2=0x7fcd500cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.955+0000 7fcd559fa640 1 -- 172.21.15.115:0/468589518 shutdown_connections 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.955+0000 7fcd559fa640 1 -- 172.21.15.115:0/468589518 wait complete. 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.957+0000 7fcd559fa640 1 Processor -- start 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.957+0000 7fcd559fa640 1 -- start start 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.958+0000 7fcd559fa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd40004120 0x7fcd500b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.958+0000 7fcd559fa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd500c6e00 0x7fcd50156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.958+0000 7fcd559fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd500aea70 0x7fcd500aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.958+0000 7fcd559fa640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500af430 con 0x7fcd500c6e00 2023-12-01T03:11:02.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.958+0000 7fcd559fa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500c76c0 con 0x7fcd40004120 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.958+0000 7fcd559fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500b36f0 con 0x7fcd500aea70 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.958+0000 7fcd4ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd500c6e00 0x7fcd50156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.958+0000 7fcd4ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd500c6e00 0x7fcd50156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46966/0 (socket says 172.21.15.115:46966) 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.958+0000 7fcd4ffff640 1 -- 172.21.15.115:0/412259722 learned_addr learned my addr 172.21.15.115:0/412259722 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.958+0000 7fcd549f8640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd40004120 0x7fcd500b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.959+0000 7fcd551f9640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd500aea70 0x7fcd500aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.959+0000 7fcd4ffff640 1 -- 172.21.15.115:0/412259722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd500aea70 msgr2=0x7fcd500aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.959+0000 7fcd4ffff640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd500aea70 0x7fcd500aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.959+0000 7fcd4ffff640 1 -- 172.21.15.115:0/412259722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd40004120 msgr2=0x7fcd500b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.959+0000 7fcd4ffff640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd40004120 0x7fcd500b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.959+0000 7fcd4ffff640 1 -- 172.21.15.115:0/412259722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd500b3990 con 0x7fcd500c6e00 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.959+0000 7fcd549f8640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd40004120 0x7fcd500b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.959+0000 7fcd4ffff640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd500c6e00 0x7fcd50156f30 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c011590 tx=0x7fcd3c011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.960+0000 7fcd4d7fa640 1 -- 172.21.15.115:0/412259722 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c004220 con 0x7fcd500c6e00 2023-12-01T03:11:02.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.960+0000 7fcd4d7fa640 1 -- 172.21.15.115:0/412259722 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c004380 con 0x7fcd500c6e00 2023-12-01T03:11:02.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.960+0000 7fcd559fa640 1 -- 172.21.15.115:0/412259722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd500b3c70 con 0x7fcd500c6e00 2023-12-01T03:11:02.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.960+0000 7fcd4d7fa640 1 -- 172.21.15.115:0/412259722 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c015930 con 0x7fcd500c6e00 2023-12-01T03:11:02.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.960+0000 7fcd559fa640 1 -- 172.21.15.115:0/412259722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd500b4470 con 0x7fcd500c6e00 2023-12-01T03:11:02.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.961+0000 7fcd559fa640 1 -- 172.21.15.115:0/412259722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5006a5a0 con 0x7fcd500c6e00 2023-12-01T03:11:02.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.962+0000 7fcd4d7fa640 1 -- 172.21.15.115:0/412259722 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 130) v1 ==== 88868+0+0 (secure 0 0 0) 0x7fcd3c015a90 con 0x7fcd500c6e00 2023-12-01T03:11:02.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.963+0000 7fcd4d7fa640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd3406a700 0x7fcd3406cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:02.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.963+0000 7fcd549f8640 1 -- 172.21.15.115:0/412259722 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd3406a700 msgr2=0x7fcd3406cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:02.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.963+0000 7fcd4d7fa640 1 -- 172.21.15.115:0/412259722 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fcd3c0892c0 con 0x7fcd500c6e00 2023-12-01T03:11:02.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.963+0000 7fcd549f8640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd3406a700 0x7fcd3406cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:02.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:02.967+0000 7fcd4d7fa640 1 -- 172.21.15.115:0/412259722 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcd5006a5a0 con 0x7fcd500c6e00 2023-12-01T03:11:02.989 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:02.988+0000 7f5690638040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:11:03.003 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:03.001+0000 7f16dfa15040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:11:03.045 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:03.044+0000 7f5690638040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:11:03.064 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:03.063+0000 7f16dfa15040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:11:03.116 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:03.114+0000 7f16dfa15040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:11:03.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.159+0000 7fcd559fa640 1 -- 172.21.15.115:0/412259722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd5006a5a0 con 0x7fcd500c6e00 2023-12-01T03:11:03.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.160+0000 7fcd4d7fa640 1 -- 172.21.15.115:0/412259722 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v219) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd5006a5a0 con 0x7fcd500c6e00 2023-12-01T03:11:03.162 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 -- 172.21.15.115:0/412259722 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd3406a700 msgr2=0x7fcd3406cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd3406a700 0x7fcd3406cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 -- 172.21.15.115:0/412259722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd500c6e00 msgr2=0x7fcd50156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd500c6e00 0x7fcd50156f30 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c011590 tx=0x7fcd3c011990 comp rx=0 tx=0).stop 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 -- 172.21.15.115:0/412259722 shutdown_connections 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcd3406a700 0x7fcd3406cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd500aea70 0x7fcd500aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd500c6e00 0x7fcd50156f30 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 --2- 172.21.15.115:0/412259722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd40004120 0x7fcd500b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 -- 172.21.15.115:0/412259722 >> 172.21.15.115:0/412259722 conn(0x7fcd500cd420 msgr2=0x7fcd500b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 -- 172.21.15.115:0/412259722 shutdown_connections 2023-12-01T03:11:03.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.163+0000 7fcd559fa640 1 -- 172.21.15.115:0/412259722 wait complete. 2023-12-01T03:11:03.182 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:11:03.428 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:03.427+0000 7f5690638040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:11:03.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.439+0000 7f38698df640 1 Processor -- start 2023-12-01T03:11:03.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.440+0000 7f38698df640 1 -- start start 2023-12-01T03:11:03.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.441+0000 7f38698df640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3864153c30 0x7f38640ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.441+0000 7f38698df640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3864154010 0x7f38640cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.441+0000 7f38698df640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38640cdcd0 con 0x7f38640cf9b0 2023-12-01T03:11:03.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.441+0000 7f38698df640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38640cfe20 con 0x7f3864154010 2023-12-01T03:11:03.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f38698df640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3864161830 con 0x7f3864153c30 2023-12-01T03:11:03.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f3863fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3864154010 0x7f38640cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:03.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f3863fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3864154010 0x7f38640cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46974/0 (socket says 172.21.15.115:46974) 2023-12-01T03:11:03.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f3863fff640 1 -- 172.21.15.115:0/1246825436 learned_addr learned my addr 172.21.15.115:0/1246825436 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:03.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f38690de640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f38640cf9b0 0x7f3864160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58472/0 (socket says 172.21.15.115:58472) 2023-12-01T03:11:03.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f38688dd640 1 --2- 172.21.15.115:0/1246825436 >> v2:172.21.15.142:3300/0 conn(0x7f3864153c30 0x7f38640ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:03.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f3863fff640 1 -- 172.21.15.115:0/1246825436 >> v2:172.21.15.142:3300/0 conn(0x7f3864153c30 msgr2=0x7f38640ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:03.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f3863fff640 1 --2- 172.21.15.115:0/1246825436 >> v2:172.21.15.142:3300/0 conn(0x7f3864153c30 0x7f38640ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f3863fff640 1 -- 172.21.15.115:0/1246825436 >> v1:172.21.15.115:6789/0 conn(0x7f38640cf9b0 legacy=0x7f3864160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:03.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f3863fff640 1 -- 172.21.15.115:0/1246825436 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38640cd3a0 con 0x7f3864154010 2023-12-01T03:11:03.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.442+0000 7f38637fe640 1 -- 172.21.15.115:0/1246825436 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456898782 0 0) 0x7f38640cdcd0 con 0x7f38640cf9b0 2023-12-01T03:11:03.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.444+0000 7f3863fff640 1 --2- 172.21.15.115:0/1246825436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3864154010 0x7f38640cf470 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f38640d3800 tx=0x7f385002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb00724a489ff60b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:03.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.444+0000 7f38637fe640 1 -- 172.21.15.115:0/1246825436 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850030860 con 0x7f3864154010 2023-12-01T03:11:03.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.444+0000 7f38637fe640 1 -- 172.21.15.115:0/1246825436 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38500309c0 con 0x7f3864154010 2023-12-01T03:11:03.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.444+0000 7f38637fe640 1 -- 172.21.15.115:0/1246825436 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850030d50 con 0x7f3864154010 2023-12-01T03:11:03.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.445+0000 7f38698df640 1 -- 172.21.15.115:0/1246825436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3864154010 msgr2=0x7f38640cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:03.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.445+0000 7f38698df640 1 --2- 172.21.15.115:0/1246825436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3864154010 0x7f38640cf470 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f38640d3800 tx=0x7f385002b9a0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.445+0000 7f38698df640 1 -- 172.21.15.115:0/1246825436 shutdown_connections 2023-12-01T03:11:03.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.445+0000 7f38698df640 1 --2- 172.21.15.115:0/1246825436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3864154010 0x7f38640cf470 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.445+0000 7f38698df640 1 --2- 172.21.15.115:0/1246825436 >> v2:172.21.15.142:3300/0 conn(0x7f3864153c30 0x7f38640ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.445+0000 7f38698df640 1 -- 172.21.15.115:0/1246825436 >> 172.21.15.115:0/1246825436 conn(0x7f386406a0b0 msgr2=0x7f386406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:03.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.445+0000 7f38698df640 1 -- 172.21.15.115:0/1246825436 shutdown_connections 2023-12-01T03:11:03.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.445+0000 7f38698df640 1 -- 172.21.15.115:0/1246825436 wait complete. 2023-12-01T03:11:03.448 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.447+0000 7f38698df640 1 Processor -- start 2023-12-01T03:11:03.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.447+0000 7f38698df640 1 -- start start 2023-12-01T03:11:03.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f38698df640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38640cf9b0 0x7f3864155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f38698df640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3864153c30 0x7f38640c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f38698df640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3864154010 0x7f3864152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f38698df640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38641584c0 con 0x7f38640cf9b0 2023-12-01T03:11:03.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f38698df640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38640c7a10 con 0x7f3864154010 2023-12-01T03:11:03.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f38698df640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3864157b20 con 0x7f3864153c30 2023-12-01T03:11:03.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f38688dd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38640cf9b0 0x7f3864155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:03.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f38688dd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38640cf9b0 0x7f3864155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46990/0 (socket says 172.21.15.115:46990) 2023-12-01T03:11:03.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f38688dd640 1 -- 172.21.15.115:0/4199754206 learned_addr learned my addr 172.21.15.115:0/4199754206 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:03.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f38690de640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3864154010 0x7f3864152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:03.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.448+0000 7f3863fff640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3864153c30 0x7f38640c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:03.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38688dd640 1 -- 172.21.15.115:0/4199754206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3864153c30 msgr2=0x7f38640c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:03.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38688dd640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3864153c30 0x7f38640c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38688dd640 1 -- 172.21.15.115:0/4199754206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3864154010 msgr2=0x7f3864152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:03.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38688dd640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3864154010 0x7f3864152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38688dd640 1 -- 172.21.15.115:0/4199754206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500094d0 con 0x7f38640cf9b0 2023-12-01T03:11:03.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38690de640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3864154010 0x7f3864152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:11:03.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38688dd640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38640cf9b0 0x7f3864155980 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f385800d800 tx=0x7f385800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:03.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38617fa640 1 -- 172.21.15.115:0/4199754206 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800b6c0 con 0x7f38640cf9b0 2023-12-01T03:11:03.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38617fa640 1 -- 172.21.15.115:0/4199754206 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385800b820 con 0x7f38640cf9b0 2023-12-01T03:11:03.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38617fa640 1 -- 172.21.15.115:0/4199754206 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800bbb0 con 0x7f38640cf9b0 2023-12-01T03:11:03.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.449+0000 7f38698df640 1 -- 172.21.15.115:0/4199754206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3864157e00 con 0x7f38640cf9b0 2023-12-01T03:11:03.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.450+0000 7f38698df640 1 -- 172.21.15.115:0/4199754206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3864155ec0 con 0x7f38640cf9b0 2023-12-01T03:11:03.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.451+0000 7f38698df640 1 -- 172.21.15.115:0/4199754206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38640618d0 con 0x7f38640cf9b0 2023-12-01T03:11:03.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.452+0000 7f38617fa640 1 -- 172.21.15.115:0/4199754206 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 130) v1 ==== 88868+0+0 (secure 0 0 0) 0x7f385800bd10 con 0x7f38640cf9b0 2023-12-01T03:11:03.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.453+0000 7f38617fa640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f384806a700 0x7f384806cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.453+0000 7f38617fa640 1 -- 172.21.15.115:0/4199754206 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f3858089720 con 0x7f38640cf9b0 2023-12-01T03:11:03.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.453+0000 7f3863fff640 1 -- 172.21.15.115:0/4199754206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f384806a700 msgr2=0x7f384806cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:03.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.453+0000 7f3863fff640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f384806a700 0x7f384806cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:03.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.458+0000 7f38617fa640 1 -- 172.21.15.115:0/4199754206 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3858054560 con 0x7f38640cf9b0 2023-12-01T03:11:03.517 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:03.515+0000 7f16dfa15040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:11:03.589 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:03.588+0000 7f5690638040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:11:03.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.648+0000 7f38698df640 1 -- 172.21.15.115:0/4199754206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38640d0a20 con 0x7f38640cf9b0 2023-12-01T03:11:03.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.649+0000 7f38617fa640 1 -- 172.21.15.115:0/4199754206 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v219) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3858057b80 con 0x7f38640cf9b0 2023-12-01T03:11:03.651 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:11:03.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.651+0000 7f38698df640 1 -- 172.21.15.115:0/4199754206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f384806a700 msgr2=0x7f384806cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:11:03.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.651+0000 7f38698df640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f384806a700 0x7f384806cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.651+0000 7f38698df640 1 -- 172.21.15.115:0/4199754206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38640cf9b0 msgr2=0x7f3864155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:03.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.651+0000 7f38698df640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38640cf9b0 0x7f3864155980 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f385800d800 tx=0x7f385800dcc0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.652+0000 7f38698df640 1 -- 172.21.15.115:0/4199754206 shutdown_connections 2023-12-01T03:11:03.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.652+0000 7f38698df640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f384806a700 0x7f384806cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.652+0000 7f38698df640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3864153c30 0x7f38640c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.652+0000 7f38698df640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3864154010 0x7f3864152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.652+0000 7f38698df640 1 --2- 172.21.15.115:0/4199754206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38640cf9b0 0x7f3864155980 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.652+0000 7f38698df640 1 -- 172.21.15.115:0/4199754206 >> 172.21.15.115:0/4199754206 conn(0x7f386406a0b0 msgr2=0x7f386415bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:03.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.652+0000 7f38698df640 1 -- 172.21.15.115:0/4199754206 shutdown_connections 2023-12-01T03:11:03.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.652+0000 7f38698df640 1 -- 172.21.15.115:0/4199754206 wait complete. 2023-12-01T03:11:03.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-01T03:11:03.683 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:03.679+0000 7f16dfa15040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:11:03.698 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:03.697+0000 7f5690638040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:11:03.749 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:03.748+0000 7f5690638040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:11:03.795 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:03.793+0000 7f16dfa15040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:11:03.847 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:03.845+0000 7f16dfa15040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:11:03.856 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:03.855+0000 7f5690638040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:11:03.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.874+0000 7f7d76347640 1 Processor -- start 2023-12-01T03:11:03.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.874+0000 7f7d76347640 1 -- start start 2023-12-01T03:11:03.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.875+0000 7f7d76347640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 0x7f7d700b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.875+0000 7f7d76347640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7d700b18f0 0x7f7d700b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.876+0000 7f7d76347640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d700b7760 con 0x7f7d700cfb50 2023-12-01T03:11:03.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.876+0000 7f7d76347640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d700b2170 con 0x7f7d700b18f0 2023-12-01T03:11:03.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.876+0000 7f7d76347640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700b8b70 con 0x7f7d7015cf70 2023-12-01T03:11:03.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.876+0000 7f7d6f7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7d700b18f0 0x7f7d700b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:03.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.876+0000 7f7d6ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 0x7f7d700b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:03.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.876+0000 7f7d6f7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7d700b18f0 0x7f7d700b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:44368/0 (socket says 172.21.15.115:44368) 2023-12-01T03:11:03.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.876+0000 7f7d748e4640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f7d700cfb50 0x7f7d700d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:44568/0 (socket says 172.21.15.115:44568) 2023-12-01T03:11:03.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.876+0000 7f7d6f7fe640 1 -- 172.21.15.115:0/2468543602 learned_addr learned my addr 172.21.15.115:0/2468543602 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:03.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.877+0000 7f7d6f7fe640 1 -- 172.21.15.115:0/2468543602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 msgr2=0x7f7d700b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:03.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.877+0000 7f7d6f7fe640 1 --2- 172.21.15.115:0/2468543602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 0x7f7d700b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.877+0000 7f7d6f7fe640 1 -- 172.21.15.115:0/2468543602 >> v1:172.21.15.142:6789/0 conn(0x7f7d700cfb50 legacy=0x7f7d700d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:03.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.877+0000 7f7d6f7fe640 1 -- 172.21.15.115:0/2468543602 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d700b6e30 con 0x7f7d700b18f0 2023-12-01T03:11:03.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.877+0000 7f7d6ffff640 1 --2- 172.21.15.115:0/2468543602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 0x7f7d700b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:11:03.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.878+0000 7f7d6f7fe640 1 --2- 172.21.15.115:0/2468543602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d700b18f0 0x7f7d700b7bf0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009870 tx=0x7f7d6402ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2320a10f59d9e2aa server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:03.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d6effd640 1 -- 172.21.15.115:0/2468543602 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6402f830 con 0x7f7d700b18f0 2023-12-01T03:11:03.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d6effd640 1 -- 172.21.15.115:0/2468543602 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6402f990 con 0x7f7d700b18f0 2023-12-01T03:11:03.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d6effd640 1 -- 172.21.15.115:0/2468543602 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6402fd20 con 0x7f7d700b18f0 2023-12-01T03:11:03.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d76347640 1 -- 172.21.15.115:0/2468543602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d700b18f0 msgr2=0x7f7d700b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:03.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d76347640 1 --2- 172.21.15.115:0/2468543602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d700b18f0 0x7f7d700b7bf0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009870 tx=0x7f7d6402ba10 comp rx=0 tx=0).stop 2023-12-01T03:11:03.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d76347640 1 -- 172.21.15.115:0/2468543602 shutdown_connections 2023-12-01T03:11:03.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d76347640 1 --2- 172.21.15.115:0/2468543602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d700b18f0 0x7f7d700b7bf0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d76347640 1 --2- 172.21.15.115:0/2468543602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 0x7f7d700b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d76347640 1 -- 172.21.15.115:0/2468543602 >> 172.21.15.115:0/2468543602 conn(0x7f7d7006a0e0 msgr2=0x7f7d7006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:03.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d76347640 1 -- 172.21.15.115:0/2468543602 shutdown_connections 2023-12-01T03:11:03.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.879+0000 7f7d76347640 1 -- 172.21.15.115:0/2468543602 wait complete. 2023-12-01T03:11:03.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.881+0000 7f7d76347640 1 Processor -- start 2023-12-01T03:11:03.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.881+0000 7f7d76347640 1 -- start start 2023-12-01T03:11:03.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.882+0000 7f7d76347640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d700b18f0 0x7f7d7015d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.882+0000 7f7d76347640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d700cfb50 0x7f7d701f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.882+0000 7f7d76347640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 0x7f7d700d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.882+0000 7f7d76347640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701f2630 con 0x7f7d700cfb50 2023-12-01T03:11:03.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.882+0000 7f7d76347640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7015d930 con 0x7f7d700b18f0 2023-12-01T03:11:03.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.882+0000 7f7d76347640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7015dc10 con 0x7f7d7015cf70 2023-12-01T03:11:03.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d6f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d700cfb50 0x7f7d701f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d6f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d700cfb50 0x7f7d701f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46992/0 (socket says 172.21.15.115:46992) 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d6f7fe640 1 -- 172.21.15.115:0/350519503 learned_addr learned my addr 172.21.15.115:0/350519503 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d6ffff640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d700b18f0 0x7f7d7015d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d748e4640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 0x7f7d700d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d6f7fe640 1 -- 172.21.15.115:0/350519503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 msgr2=0x7f7d700d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d6f7fe640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 0x7f7d700d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d6f7fe640 1 -- 172.21.15.115:0/350519503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d700b18f0 msgr2=0x7f7d7015d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d6f7fe640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d700b18f0 0x7f7d7015d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d6f7fe640 1 -- 172.21.15.115:0/350519503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d60009400 con 0x7f7d700cfb50 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.883+0000 7f7d6f7fe640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d700cfb50 0x7f7d701f1f70 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f7d64000c00 tx=0x7f7d64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.884+0000 7f7d6cff9640 1 -- 172.21.15.115:0/350519503 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64037c40 con 0x7f7d700cfb50 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.884+0000 7f7d6cff9640 1 -- 172.21.15.115:0/350519503 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d64037da0 con 0x7f7d700cfb50 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.884+0000 7f7d6cff9640 1 -- 172.21.15.115:0/350519503 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64041640 con 0x7f7d700cfb50 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.884+0000 7f7d76347640 1 -- 172.21.15.115:0/350519503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d640094d0 con 0x7f7d700cfb50 2023-12-01T03:11:03.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.884+0000 7f7d76347640 1 -- 172.21.15.115:0/350519503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7015e1d0 con 0x7f7d700cfb50 2023-12-01T03:11:03.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.885+0000 7f7d76347640 1 -- 172.21.15.115:0/350519503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d701f2770 con 0x7f7d700cfb50 2023-12-01T03:11:03.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.886+0000 7f7d6cff9640 1 -- 172.21.15.115:0/350519503 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 130) v1 ==== 88868+0+0 (secure 0 0 0) 0x7f7d640417a0 con 0x7f7d700cfb50 2023-12-01T03:11:03.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.886+0000 7f7d6cff9640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7d5c06a700 0x7f7d5c06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:03.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.887+0000 7f7d6cff9640 1 -- 172.21.15.115:0/350519503 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f7d6403c070 con 0x7f7d700cfb50 2023-12-01T03:11:03.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.887+0000 7f7d6ffff640 1 -- 172.21.15.115:0/350519503 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7d5c06a700 msgr2=0x7f7d5c06cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:03.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.887+0000 7f7d6ffff640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7d5c06a700 0x7f7d5c06cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:03.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:03.891+0000 7f7d6cff9640 1 -- 172.21.15.115:0/350519503 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7d701f2770 con 0x7f7d700cfb50 2023-12-01T03:11:03.961 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:03.959+0000 7f16dfa15040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:11:03.966 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:03.965+0000 7f5690638040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:11:04.072 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:04.070+0000 7f16dfa15040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:11:04.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.082+0000 7f7d76347640 1 -- 172.21.15.115:0/350519503 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d701f2770 con 0x7f7d700cfb50 2023-12-01T03:11:04.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.083+0000 7f7d6cff9640 1 -- 172.21.15.115:0/350519503 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v219) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7d701f2770 con 0x7f7d700cfb50 2023-12-01T03:11:04.085 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:11:04.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.085+0000 7f7d76347640 1 -- 172.21.15.115:0/350519503 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7d5c06a700 msgr2=0x7f7d5c06cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:11:04.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.085+0000 7f7d76347640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7d5c06a700 0x7f7d5c06cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.086+0000 7f7d76347640 1 -- 172.21.15.115:0/350519503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d700cfb50 msgr2=0x7f7d701f1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:04.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.086+0000 7f7d76347640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d700cfb50 0x7f7d701f1f70 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f7d64000c00 tx=0x7f7d64004730 comp rx=0 tx=0).stop 2023-12-01T03:11:04.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.086+0000 7f7d76347640 1 -- 172.21.15.115:0/350519503 shutdown_connections 2023-12-01T03:11:04.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.086+0000 7f7d76347640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7d5c06a700 0x7f7d5c06cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.086+0000 7f7d76347640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d7015cf70 0x7f7d700d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.086+0000 7f7d76347640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d700cfb50 0x7f7d701f1f70 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.086+0000 7f7d76347640 1 --2- 172.21.15.115:0/350519503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d700b18f0 0x7f7d7015d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.086+0000 7f7d76347640 1 -- 172.21.15.115:0/350519503 >> 172.21.15.115:0/350519503 conn(0x7f7d7006a0e0 msgr2=0x7f7d700cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:04.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.086+0000 7f7d76347640 1 -- 172.21.15.115:0/350519503 shutdown_connections 2023-12-01T03:11:04.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.086+0000 7f7d76347640 1 -- 172.21.15.115:0/350519503 wait complete. 2023-12-01T03:11:04.107 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-01T03:11:04.125 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:04.124+0000 7f5690638040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:11:04.174 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:04.173+0000 7f5690638040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:11:04.237 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:04.235+0000 7f16dfa15040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:11:04.289 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:04.287+0000 7f16dfa15040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:11:04.294 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:04.293+0000 7f5690638040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:11:04.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.362+0000 7f0a17fff640 1 Processor -- start 2023-12-01T03:11:04.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.362+0000 7f0a17fff640 1 -- start start 2023-12-01T03:11:04.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.364+0000 7f0a17fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0a180acd00 0x7f0a180ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.364+0000 7f0a17fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a180cd120 0x7f0a180c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.364+0000 7f0a17fff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a180c98a0 con 0x7f0a18146440 2023-12-01T03:11:04.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.364+0000 7f0a17fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a180adbb0 con 0x7f0a180acd00 2023-12-01T03:11:04.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.364+0000 7f0a17fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180ade90 con 0x7f0a180cd120 2023-12-01T03:11:04.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.365+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a180cd120 0x7f0a180c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:04.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.365+0000 7f0a0ffff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f0a18146440 0x7f0a180cca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:44570/0 (socket says 172.21.15.115:44570) 2023-12-01T03:11:04.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.365+0000 7f0a0ffff640 1 -- 172.21.15.115:0/2409017122 learned_addr learned my addr 172.21.15.115:0/2409017122 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:04.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.365+0000 7f0a16ffd640 1 --2- 172.21.15.115:0/2409017122 >> v2:172.21.15.142:3300/0 conn(0x7f0a180acd00 0x7f0a180ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:04.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.366+0000 7f0a167fc640 1 -- 172.21.15.115:0/2409017122 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090866714 0 0) 0x7f0a180c98a0 con 0x7f0a18146440 2023-12-01T03:11:04.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.366+0000 7f0a167fc640 1 -- 172.21.15.115:0/2409017122 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a04003520 con 0x7f0a18146440 2023-12-01T03:11:04.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.366+0000 7f0a16ffd640 1 -- 172.21.15.115:0/2409017122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a180cd120 msgr2=0x7f0a180c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:04.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.366+0000 7f0a16ffd640 1 --2- 172.21.15.115:0/2409017122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a180cd120 0x7f0a180c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.366+0000 7f0a16ffd640 1 -- 172.21.15.115:0/2409017122 >> v1:172.21.15.142:6789/0 conn(0x7f0a18146440 legacy=0x7f0a180cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:04.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.366+0000 7f0a16ffd640 1 -- 172.21.15.115:0/2409017122 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a180ae170 con 0x7f0a180acd00 2023-12-01T03:11:04.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.367+0000 7f0a16ffd640 1 --2- 172.21.15.115:0/2409017122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a180acd00 0x7f0a180ad0e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0a080097a0 tx=0x7f0a0802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5f8666b1639135d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:04.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.367+0000 7f0a167fc640 1 -- 172.21.15.115:0/2409017122 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08031830 con 0x7f0a180acd00 2023-12-01T03:11:04.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.367+0000 7f0a167fc640 1 -- 172.21.15.115:0/2409017122 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a08031990 con 0x7f0a180acd00 2023-12-01T03:11:04.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.367+0000 7f0a167fc640 1 -- 172.21.15.115:0/2409017122 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08031d00 con 0x7f0a180acd00 2023-12-01T03:11:04.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.368+0000 7f0a17fff640 1 -- 172.21.15.115:0/2409017122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a180acd00 msgr2=0x7f0a180ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:04.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.368+0000 7f0a17fff640 1 --2- 172.21.15.115:0/2409017122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a180acd00 0x7f0a180ad0e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0a080097a0 tx=0x7f0a0802b8d0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.368+0000 7f0a17fff640 1 -- 172.21.15.115:0/2409017122 shutdown_connections 2023-12-01T03:11:04.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.368+0000 7f0a17fff640 1 --2- 172.21.15.115:0/2409017122 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a180cd120 0x7f0a180c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.368+0000 7f0a17fff640 1 --2- 172.21.15.115:0/2409017122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a180acd00 0x7f0a180ad0e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.368+0000 7f0a17fff640 1 -- 172.21.15.115:0/2409017122 >> 172.21.15.115:0/2409017122 conn(0x7f0a180cee40 msgr2=0x7f0a180cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:04.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.368+0000 7f0a17fff640 1 -- 172.21.15.115:0/2409017122 shutdown_connections 2023-12-01T03:11:04.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.368+0000 7f0a17fff640 1 -- 172.21.15.115:0/2409017122 wait complete. 2023-12-01T03:11:04.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.370+0000 7f0a17fff640 1 Processor -- start 2023-12-01T03:11:04.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.370+0000 7f0a17fff640 1 -- start start 2023-12-01T03:11:04.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a17fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a180acd00 0x7f0a180b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a17fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a180cd120 0x7f0a180c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a17fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a18146440 0x7f0a180c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a17fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180b2a30 con 0x7f0a18146440 2023-12-01T03:11:04.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a17fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180b2d10 con 0x7f0a180acd00 2023-12-01T03:11:04.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a17fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180b2ff0 con 0x7f0a180cd120 2023-12-01T03:11:04.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a18146440 0x7f0a180c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:04.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a18146440 0x7f0a180c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46998/0 (socket says 172.21.15.115:46998) 2023-12-01T03:11:04.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a177fe640 1 -- 172.21.15.115:0/2809084356 learned_addr learned my addr 172.21.15.115:0/2809084356 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:04.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a16ffd640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a180acd00 0x7f0a180b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:04.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.371+0000 7f0a0ffff640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a180cd120 0x7f0a180c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:04.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a177fe640 1 -- 172.21.15.115:0/2809084356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a180cd120 msgr2=0x7f0a180c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:04.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a177fe640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a180cd120 0x7f0a180c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a177fe640 1 -- 172.21.15.115:0/2809084356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a180acd00 msgr2=0x7f0a180b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:04.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a177fe640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a180acd00 0x7f0a180b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a177fe640 1 -- 172.21.15.115:0/2809084356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a08009400 con 0x7f0a18146440 2023-12-01T03:11:04.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a177fe640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a18146440 0x7f0a180c6450 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000efc0 tx=0x7f0a1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:04.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a0f7fe640 1 -- 172.21.15.115:0/2809084356 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10014480 con 0x7f0a18146440 2023-12-01T03:11:04.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a0f7fe640 1 -- 172.21.15.115:0/2809084356 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1000f040 con 0x7f0a18146440 2023-12-01T03:11:04.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a0f7fe640 1 -- 172.21.15.115:0/2809084356 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10014820 con 0x7f0a18146440 2023-12-01T03:11:04.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a17fff640 1 -- 172.21.15.115:0/2809084356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a180b32d0 con 0x7f0a18146440 2023-12-01T03:11:04.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.372+0000 7f0a17fff640 1 -- 172.21.15.115:0/2809084356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a181514a0 con 0x7f0a18146440 2023-12-01T03:11:04.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.373+0000 7f0a17fff640 1 -- 172.21.15.115:0/2809084356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a180b37f0 con 0x7f0a18146440 2023-12-01T03:11:04.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.375+0000 7f0a0f7fe640 1 -- 172.21.15.115:0/2809084356 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 130) v1 ==== 88868+0+0 (secure 0 0 0) 0x7f0a10014980 con 0x7f0a18146440 2023-12-01T03:11:04.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.375+0000 7f0a0f7fe640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f09f806a700 0x7f09f806cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.376+0000 7f0a0f7fe640 1 -- 172.21.15.115:0/2809084356 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f0a10089ec0 con 0x7f0a18146440 2023-12-01T03:11:04.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.376+0000 7f0a16ffd640 1 -- 172.21.15.115:0/2809084356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f09f806a700 msgr2=0x7f09f806cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:04.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.376+0000 7f0a16ffd640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f09f806a700 0x7f09f806cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:04.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.380+0000 7f0a0f7fe640 1 -- 172.21.15.115:0/2809084356 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0a180b37f0 con 0x7f0a18146440 2023-12-01T03:11:04.415 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:04.414+0000 7f16dfa15040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:11:04.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.570+0000 7f0a17fff640 1 -- 172.21.15.115:0/2809084356 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a180b37f0 con 0x7f0a18146440 2023-12-01T03:11:04.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.571+0000 7f0a0f7fe640 1 -- 172.21.15.115:0/2809084356 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v219) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0a180b37f0 con 0x7f0a18146440 2023-12-01T03:11:04.573 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:11:04.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.573+0000 7f0a17fff640 1 -- 172.21.15.115:0/2809084356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f09f806a700 msgr2=0x7f09f806cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:11:04.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.573+0000 7f0a17fff640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f09f806a700 0x7f09f806cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.573+0000 7f0a17fff640 1 -- 172.21.15.115:0/2809084356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a18146440 msgr2=0x7f0a180c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:04.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.573+0000 7f0a17fff640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a18146440 0x7f0a180c6450 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000efc0 tx=0x7f0a1000c490 comp rx=0 tx=0).stop 2023-12-01T03:11:04.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.573+0000 7f0a17fff640 1 -- 172.21.15.115:0/2809084356 shutdown_connections 2023-12-01T03:11:04.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.574+0000 7f0a17fff640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f09f806a700 0x7f09f806cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.574+0000 7f0a17fff640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a180cd120 0x7f0a180c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.574+0000 7f0a17fff640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a18146440 0x7f0a180c6450 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.574+0000 7f0a17fff640 1 --2- 172.21.15.115:0/2809084356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a180acd00 0x7f0a180b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.574+0000 7f0a17fff640 1 -- 172.21.15.115:0/2809084356 >> 172.21.15.115:0/2809084356 conn(0x7f0a180cee40 msgr2=0x7f0a18071170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:04.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.574+0000 7f0a17fff640 1 -- 172.21.15.115:0/2809084356 shutdown_connections 2023-12-01T03:11:04.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.574+0000 7f0a17fff640 1 -- 172.21.15.115:0/2809084356 wait complete. 2023-12-01T03:11:04.594 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T03:11:04.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.796+0000 7f101272e640 1 Processor -- start 2023-12-01T03:11:04.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.796+0000 7f101272e640 1 -- start start 2023-12-01T03:11:04.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.797+0000 7f101272e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f100c15cd20 0x7f100c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.797+0000 7f101272e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f100c0d1230 0x7f100c0d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.797+0000 7f101272e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100c0ce2e0 con 0x7f100c0b5d80 2023-12-01T03:11:04.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.797+0000 7f101272e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100c0b2160 con 0x7f100c0d1230 2023-12-01T03:11:04.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.797+0000 7f101272e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0d1be0 con 0x7f100c15cd20 2023-12-01T03:11:04.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.798+0000 7f1010ccb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f100c0d1230 0x7f100c0d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:04.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.798+0000 7f1003fff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f100c0b5d80 0x7f100c0b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:44582/0 (socket says 172.21.15.115:44582) 2023-12-01T03:11:04.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.798+0000 7f1010ccb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f100c0d1230 0x7f100c0d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47010/0 (socket says 172.21.15.115:47010) 2023-12-01T03:11:04.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.798+0000 7f1003fff640 1 -- 172.21.15.115:0/3496325884 learned_addr learned my addr 172.21.15.115:0/3496325884 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:04.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.798+0000 7f100bfff640 1 --2- 172.21.15.115:0/3496325884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f100c15cd20 0x7f100c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:04.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.798+0000 7f1010ccb640 1 -- 172.21.15.115:0/3496325884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f100c15cd20 msgr2=0x7f100c0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:04.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.798+0000 7f1010ccb640 1 --2- 172.21.15.115:0/3496325884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f100c15cd20 0x7f100c0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.798+0000 7f1010ccb640 1 -- 172.21.15.115:0/3496325884 >> v1:172.21.15.142:6789/0 conn(0x7f100c0b5d80 legacy=0x7f100c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:04.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.798+0000 7f1010ccb640 1 -- 172.21.15.115:0/3496325884 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c0ce000 con 0x7f100c0d1230 2023-12-01T03:11:04.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.800+0000 7f1010ccb640 1 --2- 172.21.15.115:0/3496325884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0d1230 0x7f100c0d16a0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f1004009d90 tx=0x7f100402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c59cc86459f681e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:04.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.800+0000 7f100b7fe640 1 -- 172.21.15.115:0/3496325884 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100402cbc0 con 0x7f100c0d1230 2023-12-01T03:11:04.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.800+0000 7f100b7fe640 1 -- 172.21.15.115:0/3496325884 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100402cd20 con 0x7f100c0d1230 2023-12-01T03:11:04.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.800+0000 7f100b7fe640 1 -- 172.21.15.115:0/3496325884 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004034660 con 0x7f100c0d1230 2023-12-01T03:11:04.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.801+0000 7f101272e640 1 -- 172.21.15.115:0/3496325884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0d1230 msgr2=0x7f100c0d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:04.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.801+0000 7f101272e640 1 --2- 172.21.15.115:0/3496325884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0d1230 0x7f100c0d16a0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f1004009d90 tx=0x7f100402be10 comp rx=0 tx=0).stop 2023-12-01T03:11:04.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.801+0000 7f101272e640 1 -- 172.21.15.115:0/3496325884 shutdown_connections 2023-12-01T03:11:04.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.801+0000 7f101272e640 1 --2- 172.21.15.115:0/3496325884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0d1230 0x7f100c0d16a0 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.801+0000 7f101272e640 1 --2- 172.21.15.115:0/3496325884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f100c15cd20 0x7f100c0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.801+0000 7f101272e640 1 -- 172.21.15.115:0/3496325884 >> 172.21.15.115:0/3496325884 conn(0x7f100c06a0e0 msgr2=0x7f100c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:04.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.801+0000 7f101272e640 1 -- 172.21.15.115:0/3496325884 shutdown_connections 2023-12-01T03:11:04.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.801+0000 7f101272e640 1 -- 172.21.15.115:0/3496325884 wait complete. 2023-12-01T03:11:04.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.803+0000 7f101272e640 1 Processor -- start 2023-12-01T03:11:04.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.803+0000 7f101272e640 1 -- start start 2023-12-01T03:11:04.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.804+0000 7f101272e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c0b5d80 0x7f100c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.804+0000 7f101272e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f100c0d1230 0x7f100c0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.804+0000 7f101272e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c15cd20 0x7f100c0cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.804+0000 7f101272e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0b3bb0 con 0x7f100c15cd20 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.804+0000 7f101272e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c15d100 con 0x7f100c0b5d80 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.804+0000 7f101272e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0c84c0 con 0x7f100c0d1230 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.804+0000 7f1010ccb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c15cd20 0x7f100c0cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.804+0000 7f1010ccb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c15cd20 0x7f100c0cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47022/0 (socket says 172.21.15.115:47022) 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.804+0000 7f1010ccb640 1 -- 172.21.15.115:0/1743409694 learned_addr learned my addr 172.21.15.115:0/1743409694 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.805+0000 7f1003fff640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f100c0d1230 0x7f100c0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.805+0000 7f100bfff640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c0b5d80 0x7f100c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.805+0000 7f1010ccb640 1 -- 172.21.15.115:0/1743409694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f100c0d1230 msgr2=0x7f100c0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.805+0000 7f1010ccb640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f100c0d1230 0x7f100c0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.805+0000 7f1010ccb640 1 -- 172.21.15.115:0/1743409694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c0b5d80 msgr2=0x7f100c156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.805+0000 7f1010ccb640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c0b5d80 0x7f100c156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.805+0000 7f1010ccb640 1 -- 172.21.15.115:0/1743409694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1004009a70 con 0x7f100c15cd20 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.805+0000 7f100bfff640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c0b5d80 0x7f100c156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:04.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.805+0000 7f1010ccb640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c15cd20 0x7f100c0cbef0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f1004009d90 tx=0x7f1004004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.805+0000 7f10097fa640 1 -- 172.21.15.115:0/1743409694 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004040650 con 0x7f100c15cd20 2023-12-01T03:11:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.806+0000 7f10097fa640 1 -- 172.21.15.115:0/1743409694 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100402ce90 con 0x7f100c15cd20 2023-12-01T03:11:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.806+0000 7f101272e640 1 -- 172.21.15.115:0/1743409694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c0b49a0 con 0x7f100c15cd20 2023-12-01T03:11:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.806+0000 7f10097fa640 1 -- 172.21.15.115:0/1743409694 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10040409f0 con 0x7f100c15cd20 2023-12-01T03:11:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.806+0000 7f101272e640 1 -- 172.21.15.115:0/1743409694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c156dd0 con 0x7f100c15cd20 2023-12-01T03:11:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.807+0000 7f101272e640 1 -- 172.21.15.115:0/1743409694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c0b4b30 con 0x7f100c15cd20 2023-12-01T03:11:04.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.808+0000 7f10097fa640 1 -- 172.21.15.115:0/1743409694 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 130) v1 ==== 88868+0+0 (secure 0 0 0) 0x7f1004040b50 con 0x7f100c15cd20 2023-12-01T03:11:04.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.809+0000 7f10097fa640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0fec06a700 0x7f0fec06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:04.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.809+0000 7f100bfff640 1 -- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0fec06a700 msgr2=0x7f0fec06cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:04.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.809+0000 7f10097fa640 1 -- 172.21.15.115:0/1743409694 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f10040abee0 con 0x7f100c15cd20 2023-12-01T03:11:04.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.809+0000 7f100bfff640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0fec06a700 0x7f0fec06cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:04.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:04.814+0000 7f10097fa640 1 -- 172.21.15.115:0/1743409694 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1004076c50 con 0x7f100c15cd20 2023-12-01T03:11:05.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.010+0000 7f100bfff640 1 -- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0fec06a700 msgr2=0x7f0fec06cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:05.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.010+0000 7f100bfff640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0fec06a700 0x7f0fec06cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:11:05.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.026+0000 7f101272e640 1 -- 172.21.15.115:0/1743409694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f100c04e890 con 0x7f100c15cd20 2023-12-01T03:11:05.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.026+0000 7f10097fa640 1 -- 172.21.15.115:0/1743409694 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v219) v1 ==== 80+0+469 (secure 0 0 0) 0x7f1004009450 con 0x7f100c15cd20 2023-12-01T03:11:05.028 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/138932 2023-12-02T03:10:55.627314+0000 2023-12-01T03:11:05.028 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/138932 2023-12-02T03:10:55.627314+0000 2023-12-01T03:11:05.028 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2913786291 2023-12-02T03:10:55.627314+0000 2023-12-01T03:11:05.029 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/846563507 2023-12-02T03:10:55.627314+0000 2023-12-01T03:11:05.029 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3736942314 2023-12-02T03:10:55.627314+0000 2023-12-01T03:11:05.029 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1606716517 2023-12-02T03:10:55.627314+0000 2023-12-01T03:11:05.029 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2954409059 2023-12-02T03:10:55.627314+0000 2023-12-01T03:11:05.029 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3112535338 2023-12-02T03:10:55.627314+0000 2023-12-01T03:11:05.029 INFO:teuthology.orchestra.run.smithi115.stderr:listed 8 entries 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 -- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0fec06a700 msgr2=0x7f0fec06cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0fec06a700 0x7f0fec06cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 -- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c15cd20 msgr2=0x7f100c0cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c15cd20 0x7f100c0cbef0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f1004009d90 tx=0x7f1004004a40 comp rx=0 tx=0).stop 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 -- 172.21.15.115:0/1743409694 shutdown_connections 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0fec06a700 0x7f0fec06cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f100c0d1230 0x7f100c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c15cd20 0x7f100c0cbef0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 --2- 172.21.15.115:0/1743409694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f100c0b5d80 0x7f100c156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 -- 172.21.15.115:0/1743409694 >> 172.21.15.115:0/1743409694 conn(0x7f100c06a0e0 msgr2=0x7f100c15bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 -- 172.21.15.115:0/1743409694 shutdown_connections 2023-12-01T03:11:05.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.029+0000 7f101272e640 1 -- 172.21.15.115:0/1743409694 wait complete. 2023-12-01T03:11:05.050 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T03:11:05.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:05.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T03:11:05.133 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:05.132+0000 7f5690638040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:11:05.141 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T03:11:05.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:05.141 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T03:11:05.188 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:11:05.188 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:11:05.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:11:05.242 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:11:05.242 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:11:05.283 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.281+0000 7f16dfa15040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:11:05.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:11:05.291 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:11:05.291 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:11:05.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:11:05.343 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:05.446 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:05.445+0000 7f5690638040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:11:05.506 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:05.505+0000 7f5690638040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:11:05.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.542+0000 7f7b60a57640 1 Processor -- start 2023-12-01T03:11:05.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.542+0000 7f7b60a57640 1 -- start start 2023-12-01T03:11:05.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.543+0000 7f7b60a57640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c15d7b0 0x7f7b5c15db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:05.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.543+0000 7f7b60a57640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7b5c160670 0x7f7b5c0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:05.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.544+0000 7f7b60a57640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5c0cd3f0 con 0x7f7b5c0cddc0 2023-12-01T03:11:05.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.544+0000 7f7b60a57640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5c0cd0f0 con 0x7f7b5c160670 2023-12-01T03:11:05.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.544+0000 7f7b60a57640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c160ec0 con 0x7f7b5c15d7b0 2023-12-01T03:11:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.544+0000 7f7b5b7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c15d7b0 0x7f7b5c15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.544+0000 7f7b5affd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7b5c160670 0x7f7b5c0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.544+0000 7f7b5b7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c15d7b0 0x7f7b5c15db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:40492/0 (socket says 172.21.15.115:40492) 2023-12-01T03:11:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.544+0000 7f7b5b7fe640 1 -- 172.21.15.115:0/4281688155 learned_addr learned my addr 172.21.15.115:0/4281688155 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.545+0000 7f7b5b7fe640 1 -- 172.21.15.115:0/4281688155 >> v2:172.21.15.142:3300/0 conn(0x7f7b5c160670 msgr2=0x7f7b5c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.545+0000 7f7b5b7fe640 1 --2- 172.21.15.115:0/4281688155 >> v2:172.21.15.142:3300/0 conn(0x7f7b5c160670 0x7f7b5c0cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.545+0000 7f7b5b7fe640 1 -- 172.21.15.115:0/4281688155 >> v1:172.21.15.142:6789/0 conn(0x7f7b5c0cddc0 legacy=0x7f7b5c0d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.545+0000 7f7b5b7fe640 1 -- 172.21.15.115:0/4281688155 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c1611a0 con 0x7f7b5c15d7b0 2023-12-01T03:11:05.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.547+0000 7f7b5b7fe640 1 --2- 172.21.15.115:0/4281688155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c15d7b0 0x7f7b5c15db90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009870 tx=0x7f7b4c02ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=585f0a652dcc5316 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.547+0000 7f7b5a7fc640 1 -- 172.21.15.115:0/4281688155 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c02f830 con 0x7f7b5c15d7b0 2023-12-01T03:11:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.547+0000 7f7b5a7fc640 1 -- 172.21.15.115:0/4281688155 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c02f990 con 0x7f7b5c15d7b0 2023-12-01T03:11:05.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.547+0000 7f7b5a7fc640 1 -- 172.21.15.115:0/4281688155 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c02fd20 con 0x7f7b5c15d7b0 2023-12-01T03:11:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.548+0000 7f7b60a57640 1 -- 172.21.15.115:0/4281688155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c15d7b0 msgr2=0x7f7b5c15db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.548+0000 7f7b60a57640 1 --2- 172.21.15.115:0/4281688155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c15d7b0 0x7f7b5c15db90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009870 tx=0x7f7b4c02ba10 comp rx=0 tx=0).stop 2023-12-01T03:11:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.548+0000 7f7b60a57640 1 -- 172.21.15.115:0/4281688155 shutdown_connections 2023-12-01T03:11:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.548+0000 7f7b60a57640 1 --2- 172.21.15.115:0/4281688155 >> v2:172.21.15.142:3300/0 conn(0x7f7b5c160670 0x7f7b5c0cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.548+0000 7f7b60a57640 1 --2- 172.21.15.115:0/4281688155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c15d7b0 0x7f7b5c15db90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.548+0000 7f7b60a57640 1 -- 172.21.15.115:0/4281688155 >> 172.21.15.115:0/4281688155 conn(0x7f7b5c06a0e0 msgr2=0x7f7b5c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.548+0000 7f7b60a57640 1 -- 172.21.15.115:0/4281688155 shutdown_connections 2023-12-01T03:11:05.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.548+0000 7f7b60a57640 1 -- 172.21.15.115:0/4281688155 wait complete. 2023-12-01T03:11:05.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.550+0000 7f7b60a57640 1 Processor -- start 2023-12-01T03:11:05.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.550+0000 7f7b60a57640 1 -- start start 2023-12-01T03:11:05.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.551+0000 7f7b60a57640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c0cddc0 0x7f7b5c0b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.551+0000 7f7b60a57640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b5c15d7b0 0x7f7b5c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.551+0000 7f7b60a57640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b5c160670 0x7f7b5c0c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.551+0000 7f7b60a57640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c0c8bc0 con 0x7f7b5c160670 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.551+0000 7f7b60a57640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c15b090 con 0x7f7b5c15d7b0 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.551+0000 7f7b60a57640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c15bed0 con 0x7f7b5c0cddc0 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.551+0000 7f7b5bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b5c160670 0x7f7b5c0c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b5c160670 0x7f7b5c0c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47032/0 (socket says 172.21.15.115:47032) 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5bfff640 1 -- 172.21.15.115:0/4026555644 learned_addr learned my addr 172.21.15.115:0/4026555644 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5b7fe640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c0cddc0 0x7f7b5c0b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5affd640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b5c15d7b0 0x7f7b5c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:05.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5bfff640 1 -- 172.21.15.115:0/4026555644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c0cddc0 msgr2=0x7f7b5c0b85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5bfff640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c0cddc0 0x7f7b5c0b85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5bfff640 1 -- 172.21.15.115:0/4026555644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b5c15d7b0 msgr2=0x7f7b5c15c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5bfff640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b5c15d7b0 0x7f7b5c15c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5bfff640 1 -- 172.21.15.115:0/4026555644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c0094d0 con 0x7f7b5c160670 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5b7fe640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c0cddc0 0x7f7b5c0b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b5bfff640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b5c160670 0x7f7b5c0c8680 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400c7a0 tx=0x7f7b5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.552+0000 7f7b43fff640 1 -- 172.21.15.115:0/4026555644 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54003b70 con 0x7f7b5c160670 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.553+0000 7f7b43fff640 1 -- 172.21.15.115:0/4026555644 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b54003cf0 con 0x7f7b5c160670 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.553+0000 7f7b60a57640 1 -- 172.21.15.115:0/4026555644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5c15b2c0 con 0x7f7b5c160670 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.553+0000 7f7b43fff640 1 -- 172.21.15.115:0/4026555644 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54003ea0 con 0x7f7b5c160670 2023-12-01T03:11:05.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.553+0000 7f7b60a57640 1 -- 172.21.15.115:0/4026555644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c15b880 con 0x7f7b5c160670 2023-12-01T03:11:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.554+0000 7f7b60a57640 1 -- 172.21.15.115:0/4026555644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5c15c8a0 con 0x7f7b5c160670 2023-12-01T03:11:05.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.555+0000 7f7b43fff640 1 -- 172.21.15.115:0/4026555644 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 130) v1 ==== 88868+0+0 (secure 0 0 0) 0x7f7b5400aa90 con 0x7f7b5c160670 2023-12-01T03:11:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.556+0000 7f7b43fff640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b4806a700 0x7f7b4806cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.556+0000 7f7b43fff640 1 -- 172.21.15.115:0/4026555644 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f7b540890b0 con 0x7f7b5c160670 2023-12-01T03:11:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.556+0000 7f7b5b7fe640 1 -- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b4806a700 msgr2=0x7f7b4806cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:05.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.556+0000 7f7b5b7fe640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b4806a700 0x7f7b4806cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:05.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.560+0000 7f7b43fff640 1 -- 172.21.15.115:0/4026555644 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7b54053e60 con 0x7f7b5c160670 2023-12-01T03:11:05.604 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.602+0000 7f16dfa15040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:11:05.621 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:05.620+0000 7f5690638040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:11:05.664 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.662+0000 7f16dfa15040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:11:05.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.757+0000 7f7b5b7fe640 1 -- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b4806a700 msgr2=0x7f7b4806cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:05.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.757+0000 7f7b5b7fe640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b4806a700 0x7f7b4806cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:11:05.781 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.779+0000 7f16dfa15040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:11:05.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.793+0000 7f7b60a57640 1 -- 172.21.15.115:0/4026555644 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b5c160160 con 0x7f7b5c160670 2023-12-01T03:11:05.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.800+0000 7f7b43fff640 1 -- 172.21.15.115:0/4026555644 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 131) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f7b5400ad40 con 0x7f7b5c160670 2023-12-01T03:11:05.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.801+0000 7f7b43fff640 1 -- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b4806a700 msgr2=0x7f7b4806cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:11:05.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.801+0000 7f7b43fff640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b4806a700 0x7f7b4806cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.803+0000 7f7b43fff640 1 -- 172.21.15.115:0/4026555644 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5793 (secure 0 0 0) 0x7f7b54070e00 con 0x7f7b5c160670 2023-12-01T03:11:05.806 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:05.806 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":101,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:25.146035+0000","modified":"2023-12-01T03:10:28.416554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10653":{"gid":10653,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/720518620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":720518620},{"type":"v1","addr":"172.21.15.142:6834","nonce":720518620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:38.965513+0000","modified":"2023-12-01T03:10:41.983462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7333":{"gid":7333,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-01T03:11:05.806 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 101 2023-12-01T03:11:05.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.807+0000 7f7b60a57640 1 -- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b5c160670 msgr2=0x7f7b5c0c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:05.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.807+0000 7f7b60a57640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b5c160670 0x7f7b5c0c8680 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400c7a0 tx=0x7f7b5400cc60 comp rx=0 tx=0).stop 2023-12-01T03:11:05.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.807+0000 7f7b60a57640 1 -- 172.21.15.115:0/4026555644 shutdown_connections 2023-12-01T03:11:05.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.807+0000 7f7b60a57640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b4806a700 0x7f7b4806cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.807+0000 7f7b60a57640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b5c160670 0x7f7b5c0c8680 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.807+0000 7f7b60a57640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b5c15d7b0 0x7f7b5c15c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.807+0000 7f7b60a57640 1 --2- 172.21.15.115:0/4026555644 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5c0cddc0 0x7f7b5c0b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:05.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.807+0000 7f7b60a57640 1 -- 172.21.15.115:0/4026555644 >> 172.21.15.115:0/4026555644 conn(0x7f7b5c06a0e0 msgr2=0x7f7b5c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:05.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.807+0000 7f7b60a57640 1 -- 172.21.15.115:0/4026555644 shutdown_connections 2023-12-01T03:11:05.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:05.807+0000 7f7b60a57640 1 -- 172.21.15.115:0/4026555644 wait complete. 2023-12-01T03:11:05.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:05.868 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.866+0000 7f1661ab4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.869 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.866+0000 7f1661ab4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.869 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.866+0000 7f1661ab4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.869 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.866+0000 7f1661ab4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.869 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.866+0000 7f1661ab4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.869 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.867+0000 7f16642b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.869 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.867+0000 7f16642b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.869 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.867+0000 7f16642b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.869 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.867+0000 7f16642b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.869 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.867+0000 7f16642b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.886 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.883+0000 7f1661ab4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.886 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.883+0000 7f1661ab4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.886 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.883+0000 7f1661ab4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.886 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.883+0000 7f1661ab4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.886 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.883+0000 7f1661ab4640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.886 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.884+0000 7f16642b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.886 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.884+0000 7f16642b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.886 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.884+0000 7f16642b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.886 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.884+0000 7f16642b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:05.886 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:05.884+0000 7f16642b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:11:06.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.011+0000 7f919306c640 1 Processor -- start 2023-12-01T03:11:06.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.011+0000 7f919306c640 1 -- start start 2023-12-01T03:11:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.012+0000 7f919306c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f918c15afe0 0x7f918c06b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.012+0000 7f919306c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918c0ce4a0 con 0x7f918c1584b0 2023-12-01T03:11:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.012+0000 7f919306c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918c0cc180 con 0x7f918c06b7b0 2023-12-01T03:11:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.012+0000 7f919306c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918c06bc40 con 0x7f918c15afe0 2023-12-01T03:11:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.012+0000 7f919206a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f918c1584b0 0x7f918c0cdd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58486/0 (socket says 172.21.15.115:58486) 2023-12-01T03:11:06.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.012+0000 7f919206a640 1 -- 172.21.15.115:0/4170075863 learned_addr learned my addr 172.21.15.115:0/4170075863 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:06.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915671388 0 0) 0x7f9188001110 con 0x7f918c1584b0 2023-12-01T03:11:06.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191869640 1 --2- 172.21.15.115:0/4170075863 >> v2:172.21.15.142:3300/0 conn(0x7f918c15afe0 0x7f918c06b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9178003520 con 0x7f918c1584b0 2023-12-01T03:11:06.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1351153551 0 0) 0x7f9178003520 con 0x7f918c1584b0 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 >> v2:172.21.15.142:3300/0 conn(0x7f918c15afe0 msgr2=0x7f918c06b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 --2- 172.21.15.115:0/4170075863 >> v2:172.21.15.142:3300/0 conn(0x7f918c15afe0 0x7f918c06b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 >> v1:172.21.15.142:6789/0 conn(0x7f918c06b7b0 legacy=0x7f918c0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c15b550 con 0x7f918c1584b0 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f9188001800 con 0x7f918c1584b0 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 >> v1:172.21.15.115:6789/0 conn(0x7f918c1584b0 legacy=0x7f918c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91780039e0 0x7f9178003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91780047c0 0x7f9178004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91780051d0 0x7f917800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c06bc40 con 0x7f91780047c0 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917800c120 con 0x7f91780039e0 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.013+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917800c2e0 con 0x7f91780051d0 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.014+0000 7f9191869640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91780047c0 0x7f9178004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.014+0000 7f919206a640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91780039e0 0x7f9178003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.014+0000 7f9191869640 1 -- 172.21.15.115:0/4170075863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91780051d0 msgr2=0x7f917800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:06.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.014+0000 7f919286b640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91780051d0 0x7f917800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.014+0000 7f9191869640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91780051d0 0x7f917800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.014+0000 7f9191869640 1 -- 172.21.15.115:0/4170075863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91780039e0 msgr2=0x7f9178003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:06.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.014+0000 7f9191869640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91780039e0 0x7f9178003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.014+0000 7f9191869640 1 -- 172.21.15.115:0/4170075863 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178003520 con 0x7f91780047c0 2023-12-01T03:11:06.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.014+0000 7f919286b640 1 -- 172.21.15.115:0/4170075863 reap_dead start 2023-12-01T03:11:06.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.015+0000 7f9191869640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91780047c0 0x7f9178004c40 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f917c00ba30 tx=0x7f91780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb4b7616417bf1c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:06.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.015+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c032660 con 0x7f91780047c0 2023-12-01T03:11:06.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.015+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c0327c0 con 0x7f91780047c0 2023-12-01T03:11:06.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.015+0000 7f9191068640 1 -- 172.21.15.115:0/4170075863 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c032b50 con 0x7f91780047c0 2023-12-01T03:11:06.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.016+0000 7f919306c640 1 -- 172.21.15.115:0/4170075863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91780047c0 msgr2=0x7f9178004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.016+0000 7f919306c640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91780047c0 0x7f9178004c40 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f917c00ba30 tx=0x7f91780057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.016+0000 7f919306c640 1 -- 172.21.15.115:0/4170075863 shutdown_connections 2023-12-01T03:11:06.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.016+0000 7f919306c640 1 --2- 172.21.15.115:0/4170075863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91780047c0 0x7f9178004c40 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.016+0000 7f919306c640 1 -- 172.21.15.115:0/4170075863 >> 172.21.15.115:0/4170075863 conn(0x7f918c0ce640 msgr2=0x7f918c0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:06.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.016+0000 7f919306c640 1 -- 172.21.15.115:0/4170075863 shutdown_connections 2023-12-01T03:11:06.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.016+0000 7f919306c640 1 -- 172.21.15.115:0/4170075863 wait complete. 2023-12-01T03:11:06.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.018+0000 7f919306c640 1 Processor -- start 2023-12-01T03:11:06.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.018+0000 7f919306c640 1 -- start start 2023-12-01T03:11:06.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f919306c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91780047c0 0x7f918c14c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f919306c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f918c06feb0 0x7f918c0c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f919306c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918c0c6c10 0x7f918c0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f919306c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c14c8f0 con 0x7f918c06feb0 2023-12-01T03:11:06.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f919306c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c0c76e0 con 0x7f918c0c6c10 2023-12-01T03:11:06.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f919306c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918c0c7120 con 0x7f91780047c0 2023-12-01T03:11:06.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f9191869640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f918c06feb0 0x7f918c0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f9191869640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f918c06feb0 0x7f918c0c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47146/0 (socket says 172.21.15.115:47146) 2023-12-01T03:11:06.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f9191869640 1 -- 172.21.15.115:0/1563771176 learned_addr learned my addr 172.21.15.115:0/1563771176 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:06.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f919206a640 1 --2- 172.21.15.115:0/1563771176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91780047c0 0x7f918c14c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f919286b640 1 --2- 172.21.15.115:0/1563771176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918c0c6c10 0x7f918c0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f9191869640 1 -- 172.21.15.115:0/1563771176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91780047c0 msgr2=0x7f918c14c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.019+0000 7f9191869640 1 --2- 172.21.15.115:0/1563771176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91780047c0 0x7f918c14c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.020+0000 7f9191869640 1 -- 172.21.15.115:0/1563771176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918c0c6c10 msgr2=0x7f918c0b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.020+0000 7f9191869640 1 --2- 172.21.15.115:0/1563771176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918c0c6c10 0x7f918c0b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.020+0000 7f9191869640 1 -- 172.21.15.115:0/1563771176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c0c7400 con 0x7f918c06feb0 2023-12-01T03:11:06.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.020+0000 7f9191869640 1 --2- 172.21.15.115:0/1563771176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f918c06feb0 0x7f918c0c66d0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f917c02bee0 tx=0x7f917c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:06.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.020+0000 7f9182ffd640 1 -- 172.21.15.115:0/1563771176 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c004220 con 0x7f918c06feb0 2023-12-01T03:11:06.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.020+0000 7f9182ffd640 1 -- 172.21.15.115:0/1563771176 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c0043a0 con 0x7f918c06feb0 2023-12-01T03:11:06.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.020+0000 7f9182ffd640 1 -- 172.21.15.115:0/1563771176 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c030aa0 con 0x7f918c06feb0 2023-12-01T03:11:06.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.020+0000 7f919306c640 1 -- 172.21.15.115:0/1563771176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918c14cee0 con 0x7f918c06feb0 2023-12-01T03:11:06.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.020+0000 7f919306c640 1 -- 172.21.15.115:0/1563771176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918c0c8160 con 0x7f918c06feb0 2023-12-01T03:11:06.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.021+0000 7f919306c640 1 -- 172.21.15.115:0/1563771176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918c0c82f0 con 0x7f918c06feb0 2023-12-01T03:11:06.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.022+0000 7f9182ffd640 1 -- 172.21.15.115:0/1563771176 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 131) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f917c030c00 con 0x7f918c06feb0 2023-12-01T03:11:06.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.023+0000 7f9182ffd640 1 -- 172.21.15.115:0/1563771176 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7f917c0ab6a0 con 0x7f918c06feb0 2023-12-01T03:11:06.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.027+0000 7f9182ffd640 1 -- 172.21.15.115:0/1563771176 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f918c0c82f0 con 0x7f918c06feb0 2023-12-01T03:11:06.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.217+0000 7f919306c640 1 -- 172.21.15.115:0/1563771176 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f918c0c82f0 con 0x7f918c06feb0 2023-12-01T03:11:06.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.220+0000 7f9182ffd640 1 -- 172.21.15.115:0/1563771176 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+20900 (secure 0 0 0) 0x7f918c0c82f0 con 0x7f918c06feb0 2023-12-01T03:11:06.222 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:06.223 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":220,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:05.783522+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:10:37.180599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:10:37.961783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/1558237180":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3235712957":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1161447385":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3956813425":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/4128909410":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3433151698":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1087193141":"2023-12-02T03:11:05.783435+0000","172.21.15.115:6824/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:6822/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3133490080":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2913786291":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/846563507":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3736942314":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/1606716517":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3870112353":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2954409059":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3112535338":"2023-12-02T03:10:55.627314+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:06.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.222+0000 7f919306c640 1 -- 172.21.15.115:0/1563771176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f918c06feb0 msgr2=0x7f918c0c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.222+0000 7f919306c640 1 --2- 172.21.15.115:0/1563771176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f918c06feb0 0x7f918c0c66d0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f917c02bee0 tx=0x7f917c004730 comp rx=0 tx=0).stop 2023-12-01T03:11:06.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.223+0000 7f919306c640 1 -- 172.21.15.115:0/1563771176 shutdown_connections 2023-12-01T03:11:06.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.223+0000 7f919306c640 1 --2- 172.21.15.115:0/1563771176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f918c0c6c10 0x7f918c0b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.223+0000 7f919306c640 1 --2- 172.21.15.115:0/1563771176 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f918c06feb0 0x7f918c0c66d0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.223+0000 7f919306c640 1 --2- 172.21.15.115:0/1563771176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91780047c0 0x7f918c14c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.223+0000 7f919306c640 1 -- 172.21.15.115:0/1563771176 >> 172.21.15.115:0/1563771176 conn(0x7f918c0ce640 msgr2=0x7f918c158890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:06.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.223+0000 7f919306c640 1 -- 172.21.15.115:0/1563771176 shutdown_connections 2023-12-01T03:11:06.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.223+0000 7f919306c640 1 -- 172.21.15.115:0/1563771176 wait complete. 2023-12-01T03:11:06.243 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T03:10:22.683496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '213', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T03:10:24.156854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '214', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-01T03:10:37.180599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '217', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-01T03:10:37.961783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '218', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:11:06.244 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:11:06.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.505+0000 7f9b5b241640 1 Processor -- start 2023-12-01T03:11:06.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.506+0000 7f9b5b241640 1 -- start start 2023-12-01T03:11:06.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b5b241640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b5415cd90 0x7f9b540cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b5b241640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9b540b5fa0 0x7f9b54161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b5b241640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9b540cf650 0x7f9b540cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b5b241640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b540d0ec0 con 0x7f9b540b5fa0 2023-12-01T03:11:06.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b5b241640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b540ce5e0 con 0x7f9b540cf650 2023-12-01T03:11:06.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b5b241640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b54161950 con 0x7f9b5415cd90 2023-12-01T03:11:06.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b53fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9b540b5fa0 0x7f9b54161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b53fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9b540b5fa0 0x7f9b54161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47162/0 (socket says 172.21.15.115:47162) 2023-12-01T03:11:06.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b53fff640 1 -- 172.21.15.115:0/868890226 learned_addr learned my addr 172.21.15.115:0/868890226 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:06.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b597de640 1 --2- 172.21.15.115:0/868890226 >> v2:172.21.15.142:3300/0 conn(0x7f9b540cf650 0x7f9b540cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.507+0000 7f9b58fdd640 1 --2- 172.21.15.115:0/868890226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b5415cd90 0x7f9b540cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.508+0000 7f9b53fff640 1 -- 172.21.15.115:0/868890226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b5415cd90 msgr2=0x7f9b540cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:06.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.508+0000 7f9b53fff640 1 --2- 172.21.15.115:0/868890226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b5415cd90 0x7f9b540cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.508+0000 7f9b53fff640 1 -- 172.21.15.115:0/868890226 >> v2:172.21.15.142:3300/0 conn(0x7f9b540cf650 msgr2=0x7f9b540cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:06.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.508+0000 7f9b53fff640 1 --2- 172.21.15.115:0/868890226 >> v2:172.21.15.142:3300/0 conn(0x7f9b540cf650 0x7f9b540cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.508+0000 7f9b53fff640 1 -- 172.21.15.115:0/868890226 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b540d0000 con 0x7f9b540b5fa0 2023-12-01T03:11:06.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.509+0000 7f9b53fff640 1 --2- 172.21.15.115:0/868890226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 0x7f9b54161410 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f9b540cd3a0 tx=0x7f9b4002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357181df08456194 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.509+0000 7f9b537fe640 1 -- 172.21.15.115:0/868890226 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40031870 con 0x7f9b540b5fa0 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.509+0000 7f9b537fe640 1 -- 172.21.15.115:0/868890226 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b400319d0 con 0x7f9b540b5fa0 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.509+0000 7f9b537fe640 1 -- 172.21.15.115:0/868890226 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40031d60 con 0x7f9b540b5fa0 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.510+0000 7f9b5b241640 1 -- 172.21.15.115:0/868890226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 msgr2=0x7f9b54161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.510+0000 7f9b5b241640 1 --2- 172.21.15.115:0/868890226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 0x7f9b54161410 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f9b540cd3a0 tx=0x7f9b4002b910 comp rx=0 tx=0).stop 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.510+0000 7f9b5b241640 1 -- 172.21.15.115:0/868890226 shutdown_connections 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.510+0000 7f9b5b241640 1 --2- 172.21.15.115:0/868890226 >> v2:172.21.15.142:3300/0 conn(0x7f9b540cf650 0x7f9b540cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.510+0000 7f9b5b241640 1 --2- 172.21.15.115:0/868890226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 0x7f9b54161410 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.510+0000 7f9b5b241640 1 --2- 172.21.15.115:0/868890226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b5415cd90 0x7f9b540cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.510+0000 7f9b5b241640 1 -- 172.21.15.115:0/868890226 >> 172.21.15.115:0/868890226 conn(0x7f9b54065960 msgr2=0x7f9b54065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.510+0000 7f9b5b241640 1 -- 172.21.15.115:0/868890226 shutdown_connections 2023-12-01T03:11:06.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.510+0000 7f9b5b241640 1 -- 172.21.15.115:0/868890226 wait complete. 2023-12-01T03:11:06.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.511+0000 7f9b5b241640 1 Processor -- start 2023-12-01T03:11:06.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.512+0000 7f9b5b241640 1 -- start start 2023-12-01T03:11:06.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.512+0000 7f9b5b241640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 0x7f9b541f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.512+0000 7f9b5b241640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540cf650 0x7f9b540b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.512+0000 7f9b5b241640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b5415cd90 0x7f9b541f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b5b241640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b540cb800 con 0x7f9b540b5fa0 2023-12-01T03:11:06.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b5b241640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5415e960 con 0x7f9b5415cd90 2023-12-01T03:11:06.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b5b241640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b540caf10 con 0x7f9b540cf650 2023-12-01T03:11:06.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b58fdd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 0x7f9b541f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b58fdd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 0x7f9b541f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47170/0 (socket says 172.21.15.115:47170) 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b58fdd640 1 -- 172.21.15.115:0/3559743188 learned_addr learned my addr 172.21.15.115:0/3559743188 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b53fff640 1 --2- 172.21.15.115:0/3559743188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540cf650 0x7f9b540b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b597de640 1 --2- 172.21.15.115:0/3559743188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b5415cd90 0x7f9b541f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b58fdd640 1 -- 172.21.15.115:0/3559743188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540cf650 msgr2=0x7f9b540b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b58fdd640 1 --2- 172.21.15.115:0/3559743188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540cf650 0x7f9b540b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b58fdd640 1 -- 172.21.15.115:0/3559743188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b5415cd90 msgr2=0x7f9b541f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b58fdd640 1 --2- 172.21.15.115:0/3559743188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b5415cd90 0x7f9b541f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b58fdd640 1 -- 172.21.15.115:0/3559743188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b400094d0 con 0x7f9b540b5fa0 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.513+0000 7f9b597de640 1 --2- 172.21.15.115:0/3559743188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b5415cd90 0x7f9b541f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:06.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.514+0000 7f9b58fdd640 1 --2- 172.21.15.115:0/3559743188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 0x7f9b541f23c0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800d5f0 tx=0x7f9b4800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:06.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.514+0000 7f9b517fa640 1 -- 172.21.15.115:0/3559743188 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b480125f0 con 0x7f9b540b5fa0 2023-12-01T03:11:06.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.514+0000 7f9b517fa640 1 -- 172.21.15.115:0/3559743188 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b48012750 con 0x7f9b540b5fa0 2023-12-01T03:11:06.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.514+0000 7f9b517fa640 1 -- 172.21.15.115:0/3559743188 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48012b00 con 0x7f9b540b5fa0 2023-12-01T03:11:06.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.514+0000 7f9b5b241640 1 -- 172.21.15.115:0/3559743188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b540b53a0 con 0x7f9b540b5fa0 2023-12-01T03:11:06.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.514+0000 7f9b5b241640 1 -- 172.21.15.115:0/3559743188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b540cb490 con 0x7f9b540b5fa0 2023-12-01T03:11:06.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.515+0000 7f9b5b241640 1 -- 172.21.15.115:0/3559743188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5404e890 con 0x7f9b540b5fa0 2023-12-01T03:11:06.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.516+0000 7f9b517fa640 1 -- 172.21.15.115:0/3559743188 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 131) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f9b48012c60 con 0x7f9b540b5fa0 2023-12-01T03:11:06.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.518+0000 7f9b517fa640 1 -- 172.21.15.115:0/3559743188 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7f9b48088f50 con 0x7f9b540b5fa0 2023-12-01T03:11:06.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.522+0000 7f9b517fa640 1 -- 172.21.15.115:0/3559743188 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9b48054230 con 0x7f9b540b5fa0 2023-12-01T03:11:06.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.711+0000 7f9b5b241640 1 -- 172.21.15.115:0/3559743188 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b540d0a70 con 0x7f9b540b5fa0 2023-12-01T03:11:06.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.712+0000 7f9b517fa640 1 -- 172.21.15.115:0/3559743188 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v220) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b48057850 con 0x7f9b540b5fa0 2023-12-01T03:11:06.714 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:11:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.715+0000 7f9b5b241640 1 -- 172.21.15.115:0/3559743188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 msgr2=0x7f9b541f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.715+0000 7f9b5b241640 1 --2- 172.21.15.115:0/3559743188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 0x7f9b541f23c0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800d5f0 tx=0x7f9b4800dab0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.715+0000 7f9b5b241640 1 -- 172.21.15.115:0/3559743188 shutdown_connections 2023-12-01T03:11:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.715+0000 7f9b5b241640 1 --2- 172.21.15.115:0/3559743188 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9b540cf650 0x7f9b540b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.715+0000 7f9b5b241640 1 --2- 172.21.15.115:0/3559743188 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b5415cd90 0x7f9b541f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.715+0000 7f9b5b241640 1 --2- 172.21.15.115:0/3559743188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b540b5fa0 0x7f9b541f23c0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.715+0000 7f9b5b241640 1 -- 172.21.15.115:0/3559743188 >> 172.21.15.115:0/3559743188 conn(0x7f9b54065960 msgr2=0x7f9b5415a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:06.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.715+0000 7f9b5b241640 1 -- 172.21.15.115:0/3559743188 shutdown_connections 2023-12-01T03:11:06.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.715+0000 7f9b5b241640 1 -- 172.21.15.115:0/3559743188 wait complete. 2023-12-01T03:11:06.737 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:11:06.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.945+0000 7fe995638640 1 Processor -- start 2023-12-01T03:11:06.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.945+0000 7fe995638640 1 -- start start 2023-12-01T03:11:06.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.947+0000 7fe995638640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe990160200 0x7fe9900b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.947+0000 7fe995638640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9900b0f80 0x7fe99006b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.947+0000 7fe995638640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9900cb290 con 0x7fe99006b7a0 2023-12-01T03:11:06.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.947+0000 7fe995638640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9900b35f0 con 0x7fe990160200 2023-12-01T03:11:06.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.947+0000 7fe995638640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9900b0ae0 con 0x7fe9900b0f80 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.948+0000 7fe98f7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe99006b7a0 0x7fe9900cbd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58502/0 (socket says 172.21.15.115:58502) 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.948+0000 7fe98f7fe640 1 -- 172.21.15.115:0/960456013 learned_addr learned my addr 172.21.15.115:0/960456013 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.948+0000 7fe9867fc640 1 --2- 172.21.15.115:0/960456013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9900b0f80 0x7fe99006b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.948+0000 7fe98effd640 1 --2- 172.21.15.115:0/960456013 >> v2:172.21.15.142:3300/0 conn(0x7fe990160200 0x7fe9900b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.949+0000 7fe98e7fc640 1 -- 172.21.15.115:0/960456013 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828072655 0 0) 0x7fe9900cb290 con 0x7fe99006b7a0 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.949+0000 7fe98e7fc640 1 -- 172.21.15.115:0/960456013 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe97c003520 con 0x7fe99006b7a0 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.949+0000 7fe9867fc640 1 -- 172.21.15.115:0/960456013 >> v2:172.21.15.142:3300/0 conn(0x7fe990160200 msgr2=0x7fe9900b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.949+0000 7fe9867fc640 1 --2- 172.21.15.115:0/960456013 >> v2:172.21.15.142:3300/0 conn(0x7fe990160200 0x7fe9900b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.949+0000 7fe98e7fc640 1 -- 172.21.15.115:0/960456013 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2329893455 0 0) 0x7fe97c003520 con 0x7fe99006b7a0 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.949+0000 7fe9867fc640 1 -- 172.21.15.115:0/960456013 >> v1:172.21.15.115:6789/0 conn(0x7fe99006b7a0 legacy=0x7fe9900cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.949+0000 7fe9867fc640 1 -- 172.21.15.115:0/960456013 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9900b3310 con 0x7fe9900b0f80 2023-12-01T03:11:06.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.950+0000 7fe9867fc640 1 --2- 172.21.15.115:0/960456013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9900b0f80 0x7fe99006b260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe9780097a0 tx=0x7fe97802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50bf1ca922ea1d46 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:06.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.951+0000 7fe98e7fc640 1 -- 172.21.15.115:0/960456013 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97802d040 con 0x7fe9900b0f80 2023-12-01T03:11:06.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.951+0000 7fe98e7fc640 1 -- 172.21.15.115:0/960456013 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978031810 con 0x7fe9900b0f80 2023-12-01T03:11:06.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.951+0000 7fe98e7fc640 1 -- 172.21.15.115:0/960456013 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978031ba0 con 0x7fe9900b0f80 2023-12-01T03:11:06.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.952+0000 7fe995638640 1 -- 172.21.15.115:0/960456013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9900b0f80 msgr2=0x7fe99006b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.952+0000 7fe995638640 1 --2- 172.21.15.115:0/960456013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9900b0f80 0x7fe99006b260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe9780097a0 tx=0x7fe97802b8b0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.952+0000 7fe995638640 1 -- 172.21.15.115:0/960456013 shutdown_connections 2023-12-01T03:11:06.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.952+0000 7fe995638640 1 --2- 172.21.15.115:0/960456013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9900b0f80 0x7fe99006b260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.952+0000 7fe995638640 1 --2- 172.21.15.115:0/960456013 >> v2:172.21.15.142:3300/0 conn(0x7fe990160200 0x7fe9900b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.952+0000 7fe995638640 1 -- 172.21.15.115:0/960456013 >> 172.21.15.115:0/960456013 conn(0x7fe990065960 msgr2=0x7fe990065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:06.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.952+0000 7fe995638640 1 -- 172.21.15.115:0/960456013 shutdown_connections 2023-12-01T03:11:06.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.952+0000 7fe995638640 1 -- 172.21.15.115:0/960456013 wait complete. 2023-12-01T03:11:06.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.954+0000 7fe995638640 1 Processor -- start 2023-12-01T03:11:06.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.954+0000 7fe995638640 1 -- start start 2023-12-01T03:11:06.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.955+0000 7fe995638640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe99006b7a0 0x7fe99015ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.955+0000 7fe995638640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9900b0f80 0x7fe9901510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.955+0000 7fe995638640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe990160200 0x7fe9901515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.955+0000 7fe995638640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe990150c10 con 0x7fe990160200 2023-12-01T03:11:06.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.955+0000 7fe995638640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe990147a50 con 0x7fe9900b0f80 2023-12-01T03:11:06.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.955+0000 7fe995638640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99015f690 con 0x7fe99006b7a0 2023-12-01T03:11:06.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe98f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe990160200 0x7fe9901515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe98f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe990160200 0x7fe9901515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47172/0 (socket says 172.21.15.115:47172) 2023-12-01T03:11:06.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe98f7fe640 1 -- 172.21.15.115:0/1338720027 learned_addr learned my addr 172.21.15.115:0/1338720027 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:06.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe98effd640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe99006b7a0 0x7fe99015ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe9867fc640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9900b0f80 0x7fe9901510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe98f7fe640 1 -- 172.21.15.115:0/1338720027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe99006b7a0 msgr2=0x7fe99015ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe98f7fe640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe99006b7a0 0x7fe99015ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe98f7fe640 1 -- 172.21.15.115:0/1338720027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9900b0f80 msgr2=0x7fe9901510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:06.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe98f7fe640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9900b0f80 0x7fe9901510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:06.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe98f7fe640 1 -- 172.21.15.115:0/1338720027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe978009400 con 0x7fe990160200 2023-12-01T03:11:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.956+0000 7fe98effd640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe99006b7a0 0x7fe99015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:11:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.957+0000 7fe98f7fe640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe990160200 0x7fe9901515e0 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fe98800cb20 tx=0x7fe98800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.957+0000 7fe987fff640 1 -- 172.21.15.115:0/1338720027 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98800aca0 con 0x7fe990160200 2023-12-01T03:11:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.957+0000 7fe987fff640 1 -- 172.21.15.115:0/1338720027 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98800ae00 con 0x7fe990160200 2023-12-01T03:11:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.957+0000 7fe987fff640 1 -- 172.21.15.115:0/1338720027 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988012640 con 0x7fe990160200 2023-12-01T03:11:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.957+0000 7fe995638640 1 -- 172.21.15.115:0/1338720027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99015f970 con 0x7fe990160200 2023-12-01T03:11:06.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.957+0000 7fe995638640 1 -- 172.21.15.115:0/1338720027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9901508a0 con 0x7fe990160200 2023-12-01T03:11:06.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.958+0000 7fe995638640 1 -- 172.21.15.115:0/1338720027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe990150dc0 con 0x7fe990160200 2023-12-01T03:11:06.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.959+0000 7fe987fff640 1 -- 172.21.15.115:0/1338720027 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 132) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fe9880127a0 con 0x7fe990160200 2023-12-01T03:11:06.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.961+0000 7fe987fff640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe97006a2b0 0x7fe97006c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:06.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.961+0000 7fe987fff640 1 -- 172.21.15.115:0/1338720027 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7fe988089120 con 0x7fe990160200 2023-12-01T03:11:06.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.962+0000 7fe98effd640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe97006a2b0 0x7fe97006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:06.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.963+0000 7fe98effd640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe97006a2b0 0x7fe97006c770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe99015e170 tx=0x7fe980008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:06.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:06.966+0000 7fe987fff640 1 -- 172.21.15.115:0/1338720027 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe9880543e0 con 0x7fe990160200 2023-12-01T03:11:07.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.156+0000 7fe995638640 1 -- 172.21.15.115:0/1338720027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe990147b90 con 0x7fe990160200 2023-12-01T03:11:07.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.157+0000 7fe987fff640 1 -- 172.21.15.115:0/1338720027 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v220) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe990147b90 con 0x7fe990160200 2023-12-01T03:11:07.159 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:11:07.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.159+0000 7fe995638640 1 -- 172.21.15.115:0/1338720027 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe97006a2b0 msgr2=0x7fe97006c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:07.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.159+0000 7fe995638640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe97006a2b0 0x7fe97006c770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe99015e170 tx=0x7fe980008040 comp rx=0 tx=0).stop 2023-12-01T03:11:07.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.159+0000 7fe995638640 1 -- 172.21.15.115:0/1338720027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe990160200 msgr2=0x7fe9901515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:07.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.159+0000 7fe995638640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe990160200 0x7fe9901515e0 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fe98800cb20 tx=0x7fe98800a060 comp rx=0 tx=0).stop 2023-12-01T03:11:07.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.160+0000 7fe995638640 1 -- 172.21.15.115:0/1338720027 shutdown_connections 2023-12-01T03:11:07.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.160+0000 7fe995638640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe97006a2b0 0x7fe97006c770 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.160+0000 7fe995638640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe990160200 0x7fe9901515e0 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.160+0000 7fe995638640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9900b0f80 0x7fe9901510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.160+0000 7fe995638640 1 --2- 172.21.15.115:0/1338720027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe99006b7a0 0x7fe99015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.160+0000 7fe995638640 1 -- 172.21.15.115:0/1338720027 >> 172.21.15.115:0/1338720027 conn(0x7fe990065960 msgr2=0x7fe9900cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:07.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.160+0000 7fe995638640 1 -- 172.21.15.115:0/1338720027 shutdown_connections 2023-12-01T03:11:07.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.160+0000 7fe995638640 1 -- 172.21.15.115:0/1338720027 wait complete. 2023-12-01T03:11:07.181 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:11:07.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.396+0000 7f6ec90c4640 1 Processor -- start 2023-12-01T03:11:07.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.396+0000 7f6ec90c4640 1 -- start start 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.397+0000 7f6ec90c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec415ac80 0x7f6ec40cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.397+0000 7f6ec90c4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6ec40b0e60 0x7f6ec40b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.398+0000 7f6ec90c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6ec40cdaa0 0x7f6ec40cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.398+0000 7f6ec90c4640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec415b060 con 0x7f6ec40cdaa0 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.398+0000 7f6ec90c4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec40b0ce0 con 0x7f6ec40b0e60 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.398+0000 7f6ec90c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40ce470 con 0x7f6ec415ac80 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.398+0000 7f6ec359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6ec40cdaa0 0x7f6ec40cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.398+0000 7f6ec359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6ec40cdaa0 0x7f6ec40cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47176/0 (socket says 172.21.15.115:47176) 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.398+0000 7f6ec359e640 1 -- 172.21.15.115:0/3064461532 learned_addr learned my addr 172.21.15.115:0/3064461532 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.398+0000 7f6ec259c640 1 --2- 172.21.15.115:0/3064461532 >> v2:172.21.15.142:3300/0 conn(0x7f6ec40b0e60 0x7f6ec40b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.398+0000 7f6ec2d9d640 1 --2- 172.21.15.115:0/3064461532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec415ac80 0x7f6ec40cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.399+0000 7f6ec359e640 1 -- 172.21.15.115:0/3064461532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec415ac80 msgr2=0x7f6ec40cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:07.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.399+0000 7f6ec359e640 1 --2- 172.21.15.115:0/3064461532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec415ac80 0x7f6ec40cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.399+0000 7f6ec359e640 1 -- 172.21.15.115:0/3064461532 >> v2:172.21.15.142:3300/0 conn(0x7f6ec40b0e60 msgr2=0x7f6ec40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:07.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.399+0000 7f6ec359e640 1 --2- 172.21.15.115:0/3064461532 >> v2:172.21.15.142:3300/0 conn(0x7f6ec40b0e60 0x7f6ec40b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.399+0000 7f6ec359e640 1 -- 172.21.15.115:0/3064461532 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec40cc9a0 con 0x7f6ec40cdaa0 2023-12-01T03:11:07.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.400+0000 7f6ec359e640 1 --2- 172.21.15.115:0/3064461532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec40cdaa0 0x7f6ec40cdf30 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4066990 tx=0x7f6ebc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cee53a191d53ec server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.400+0000 7f6ec1d9b640 1 -- 172.21.15.115:0/3064461532 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc02ca70 con 0x7f6ec40cdaa0 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.400+0000 7f6ec1d9b640 1 -- 172.21.15.115:0/3064461532 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc02cbd0 con 0x7f6ec40cdaa0 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.400+0000 7f6ec1d9b640 1 -- 172.21.15.115:0/3064461532 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0343f0 con 0x7f6ec40cdaa0 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.401+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3064461532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec40cdaa0 msgr2=0x7f6ec40cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.401+0000 7f6ec90c4640 1 --2- 172.21.15.115:0/3064461532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec40cdaa0 0x7f6ec40cdf30 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4066990 tx=0x7f6ebc02bed0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.401+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3064461532 shutdown_connections 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.401+0000 7f6ec90c4640 1 --2- 172.21.15.115:0/3064461532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec40cdaa0 0x7f6ec40cdf30 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.401+0000 7f6ec90c4640 1 --2- 172.21.15.115:0/3064461532 >> v2:172.21.15.142:3300/0 conn(0x7f6ec40b0e60 0x7f6ec40b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.401+0000 7f6ec90c4640 1 --2- 172.21.15.115:0/3064461532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec415ac80 0x7f6ec40cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.401+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3064461532 >> 172.21.15.115:0/3064461532 conn(0x7f6ec4065960 msgr2=0x7f6ec4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:07.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.401+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3064461532 shutdown_connections 2023-12-01T03:11:07.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.401+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3064461532 wait complete. 2023-12-01T03:11:07.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.403+0000 7f6ec90c4640 1 Processor -- start 2023-12-01T03:11:07.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.403+0000 7f6ec90c4640 1 -- start start 2023-12-01T03:11:07.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec90c4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec40b0e60 0x7f6ec4143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec90c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec40cdaa0 0x7f6ec41411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec90c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec415ac80 0x7f6ec4141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec90c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec4141c50 con 0x7f6ec415ac80 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec90c4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec4141f00 con 0x7f6ec40b0e60 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec90c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec41421e0 con 0x7f6ec40cdaa0 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec415ac80 0x7f6ec4141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec415ac80 0x7f6ec4141710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47184/0 (socket says 172.21.15.115:47184) 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec359e640 1 -- 172.21.15.115:0/3715006026 learned_addr learned my addr 172.21.15.115:0/3715006026 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec2d9d640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec40b0e60 0x7f6ec4143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec259c640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec40cdaa0 0x7f6ec41411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec359e640 1 -- 172.21.15.115:0/3715006026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec40cdaa0 msgr2=0x7f6ec41411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec359e640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec40cdaa0 0x7f6ec41411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec359e640 1 -- 172.21.15.115:0/3715006026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec40b0e60 msgr2=0x7f6ec4143310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.404+0000 7f6ec359e640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec40b0e60 0x7f6ec4143310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.405+0000 7f6ec359e640 1 -- 172.21.15.115:0/3715006026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc009b30 con 0x7f6ec415ac80 2023-12-01T03:11:07.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.405+0000 7f6ec259c640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec40cdaa0 0x7f6ec41411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:07.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.405+0000 7f6ec359e640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec415ac80 0x7f6ec4141710 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc036040 tx=0x7f6ebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:07.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.405+0000 7f6eab7fe640 1 -- 172.21.15.115:0/3715006026 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc009510 con 0x7f6ec415ac80 2023-12-01T03:11:07.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.405+0000 7f6eab7fe640 1 -- 172.21.15.115:0/3715006026 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc03e450 con 0x7f6ec415ac80 2023-12-01T03:11:07.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.405+0000 7f6eab7fe640 1 -- 172.21.15.115:0/3715006026 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc03e800 con 0x7f6ec415ac80 2023-12-01T03:11:07.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.405+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3715006026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec41424c0 con 0x7f6ec415ac80 2023-12-01T03:11:07.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.405+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3715006026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec4142a80 con 0x7f6ec415ac80 2023-12-01T03:11:07.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.406+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3715006026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec4142ca0 con 0x7f6ec415ac80 2023-12-01T03:11:07.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.407+0000 7f6eab7fe640 1 -- 172.21.15.115:0/3715006026 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 132) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f6ebc03e960 con 0x7f6ec415ac80 2023-12-01T03:11:07.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.408+0000 7f6eab7fe640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6eb006a2b0 0x7f6eb006c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.408+0000 7f6eab7fe640 1 -- 172.21.15.115:0/3715006026 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7f6ebc0ab9e0 con 0x7f6ec415ac80 2023-12-01T03:11:07.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.408+0000 7f6ec2d9d640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6eb006a2b0 0x7f6eb006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.409+0000 7f6ec2d9d640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6eb006a2b0 0x7f6eb006c770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4006fd0 tx=0x7f6eb4008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:07.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.413+0000 7f6eab7fe640 1 -- 172.21.15.115:0/3715006026 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6ebc076bd0 con 0x7f6ec415ac80 2023-12-01T03:11:07.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.605+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3715006026 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ec404e890 con 0x7f6ec415ac80 2023-12-01T03:11:07.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.606+0000 7f6eab7fe640 1 -- 172.21.15.115:0/3715006026 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v220) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ebc07a1f0 con 0x7f6ec415ac80 2023-12-01T03:11:07.608 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.608+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3715006026 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6eb006a2b0 msgr2=0x7f6eb006c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.608+0000 7f6ec90c4640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6eb006a2b0 0x7f6eb006c770 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4006fd0 tx=0x7f6eb4008040 comp rx=0 tx=0).stop 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.608+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3715006026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec415ac80 msgr2=0x7f6ec4141710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.608+0000 7f6ec90c4640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec415ac80 0x7f6ec4141710 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc036040 tx=0x7f6ebc004a40 comp rx=0 tx=0).stop 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.609+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3715006026 shutdown_connections 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.609+0000 7f6ec90c4640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6eb006a2b0 0x7f6eb006c770 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.609+0000 7f6ec90c4640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ec40cdaa0 0x7f6ec41411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.609+0000 7f6ec90c4640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ec415ac80 0x7f6ec4141710 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.609+0000 7f6ec90c4640 1 --2- 172.21.15.115:0/3715006026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ec40b0e60 0x7f6ec4143310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.609+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3715006026 >> 172.21.15.115:0/3715006026 conn(0x7f6ec4065960 msgr2=0x7f6ec4070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:07.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.609+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3715006026 shutdown_connections 2023-12-01T03:11:07.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.609+0000 7f6ec90c4640 1 -- 172.21.15.115:0/3715006026 wait complete. 2023-12-01T03:11:07.627 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-01T03:11:07.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.828+0000 7f126f59e640 1 Processor -- start 2023-12-01T03:11:07.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.829+0000 7f126f59e640 1 -- start start 2023-12-01T03:11:07.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.830+0000 7f126f59e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f127015cb60 0x7f12700caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.830+0000 7f126f59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f12700ca2d0 0x7f12700ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.830+0000 7f126f59e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12700ca700 con 0x7f12700cb430 2023-12-01T03:11:07.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.830+0000 7f126f59e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12700cb8c0 con 0x7f12700ca2d0 2023-12-01T03:11:07.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.830+0000 7f126f59e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1270070000 con 0x7f127015cb60 2023-12-01T03:11:07.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.830+0000 7f126ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f12700ca2d0 0x7f12700ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.831+0000 7f126ed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f12700ca2d0 0x7f12700ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54292/0 (socket says 172.21.15.115:54292) 2023-12-01T03:11:07.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.831+0000 7f126ed9d640 1 -- 172.21.15.115:0/275784200 learned_addr learned my addr 172.21.15.115:0/275784200 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:07.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.831+0000 7f126ed9d640 1 -- 172.21.15.115:0/275784200 >> v2:172.21.15.142:3300/0 conn(0x7f127015cb60 msgr2=0x7f12700caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:07.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.831+0000 7f126ed9d640 1 --2- 172.21.15.115:0/275784200 >> v2:172.21.15.142:3300/0 conn(0x7f127015cb60 0x7f12700caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.831+0000 7f126ed9d640 1 -- 172.21.15.115:0/275784200 >> v1:172.21.15.142:6789/0 conn(0x7f12700cb430 legacy=0x7f12700c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:07.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.831+0000 7f126ed9d640 1 -- 172.21.15.115:0/275784200 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12700702e0 con 0x7f12700ca2d0 2023-12-01T03:11:07.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.832+0000 7f126ed9d640 1 --2- 172.21.15.115:0/275784200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 0x7f12700ac8a0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f12700667c0 tx=0x7f126802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf8db83ee445e80 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:07.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.832+0000 7f126dd9b640 1 -- 172.21.15.115:0/275784200 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126802ca80 con 0x7f12700ca2d0 2023-12-01T03:11:07.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.832+0000 7f126dd9b640 1 -- 172.21.15.115:0/275784200 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126802cbe0 con 0x7f12700ca2d0 2023-12-01T03:11:07.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.832+0000 7f126dd9b640 1 -- 172.21.15.115:0/275784200 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12680343f0 con 0x7f12700ca2d0 2023-12-01T03:11:07.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.833+0000 7f126f59e640 1 -- 172.21.15.115:0/275784200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 msgr2=0x7f12700ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:07.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.833+0000 7f126f59e640 1 --2- 172.21.15.115:0/275784200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 0x7f12700ac8a0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f12700667c0 tx=0x7f126802bee0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.833+0000 7f126f59e640 1 -- 172.21.15.115:0/275784200 shutdown_connections 2023-12-01T03:11:07.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.833+0000 7f126f59e640 1 --2- 172.21.15.115:0/275784200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 0x7f12700ac8a0 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.833+0000 7f126f59e640 1 --2- 172.21.15.115:0/275784200 >> v2:172.21.15.142:3300/0 conn(0x7f127015cb60 0x7f12700caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.833+0000 7f126f59e640 1 -- 172.21.15.115:0/275784200 >> 172.21.15.115:0/275784200 conn(0x7f1270065960 msgr2=0x7f1270065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:07.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.833+0000 7f126f59e640 1 -- 172.21.15.115:0/275784200 shutdown_connections 2023-12-01T03:11:07.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.833+0000 7f126f59e640 1 -- 172.21.15.115:0/275784200 wait complete. 2023-12-01T03:11:07.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.835+0000 7f126f59e640 1 Processor -- start 2023-12-01T03:11:07.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.835+0000 7f126f59e640 1 -- start start 2023-12-01T03:11:07.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.836+0000 7f126f59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 0x7f12701f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.836+0000 7f126f59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12700cb430 0x7f12701546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.836+0000 7f126f59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f127015cb60 0x7f12700c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.836+0000 7f126f59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12701f37c0 con 0x7f12700ca2d0 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.836+0000 7f126f59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270156120 con 0x7f12700cb430 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.836+0000 7f126f59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1270155080 con 0x7f127015cb60 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.837+0000 7f126e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 0x7f12701f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.837+0000 7f126e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 0x7f12701f3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54306/0 (socket says 172.21.15.115:54306) 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.837+0000 7f126e59c640 1 -- 172.21.15.115:0/3835596588 learned_addr learned my addr 172.21.15.115:0/3835596588 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.837+0000 7f1267fff640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12700cb430 0x7f12701546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.837+0000 7f126e59c640 1 -- 172.21.15.115:0/3835596588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f127015cb60 msgr2=0x7f12700c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.837+0000 7f126e59c640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f127015cb60 0x7f12700c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.837+0000 7f126e59c640 1 -- 172.21.15.115:0/3835596588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12700cb430 msgr2=0x7f12701546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.837+0000 7f126e59c640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12700cb430 0x7f12701546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.837+0000 7f126e59c640 1 -- 172.21.15.115:0/3835596588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1268009b30 con 0x7f12700ca2d0 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.837+0000 7f126e59c640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 0x7f12701f3280 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f126000b410 tx=0x7f126000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:07.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.838+0000 7f1266ffd640 1 -- 172.21.15.115:0/3835596588 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260011830 con 0x7f12700ca2d0 2023-12-01T03:11:07.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.838+0000 7f1266ffd640 1 -- 172.21.15.115:0/3835596588 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12600119b0 con 0x7f12700ca2d0 2023-12-01T03:11:07.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.838+0000 7f1266ffd640 1 -- 172.21.15.115:0/3835596588 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260011d60 con 0x7f12700ca2d0 2023-12-01T03:11:07.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.838+0000 7f126f59e640 1 -- 172.21.15.115:0/3835596588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1270155360 con 0x7f12700ca2d0 2023-12-01T03:11:07.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.838+0000 7f126f59e640 1 -- 172.21.15.115:0/3835596588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1270155920 con 0x7f12700ca2d0 2023-12-01T03:11:07.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.839+0000 7f126f59e640 1 -- 172.21.15.115:0/3835596588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12700618d0 con 0x7f12700ca2d0 2023-12-01T03:11:07.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.840+0000 7f1266ffd640 1 -- 172.21.15.115:0/3835596588 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 132) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f126001a400 con 0x7f12700ca2d0 2023-12-01T03:11:07.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.841+0000 7f1266ffd640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f125006a280 0x7f125006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:07.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.841+0000 7f1266ffd640 1 -- 172.21.15.115:0/3835596588 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7f1260088d70 con 0x7f12700ca2d0 2023-12-01T03:11:07.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.841+0000 7f1267fff640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f125006a280 0x7f125006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:07.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.842+0000 7f1267fff640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f125006a280 0x7f125006c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f12580015d0 tx=0x7f1258009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:07.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:07.846+0000 7f1266ffd640 1 -- 172.21.15.115:0/3835596588 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1260054030 con 0x7f12700ca2d0 2023-12-01T03:11:08.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.041+0000 7f126f59e640 1 -- 172.21.15.115:0/3835596588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12700c8b40 con 0x7f12700ca2d0 2023-12-01T03:11:08.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.042+0000 7f1266ffd640 1 -- 172.21.15.115:0/3835596588 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v220) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1260015020 con 0x7f12700ca2d0 2023-12-01T03:11:08.044 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:11:08.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.044+0000 7f126f59e640 1 -- 172.21.15.115:0/3835596588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f125006a280 msgr2=0x7f125006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.044+0000 7f126f59e640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f125006a280 0x7f125006c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f12580015d0 tx=0x7f1258009040 comp rx=0 tx=0).stop 2023-12-01T03:11:08.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.044+0000 7f126f59e640 1 -- 172.21.15.115:0/3835596588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 msgr2=0x7f12701f3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.044+0000 7f126f59e640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 0x7f12701f3280 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f126000b410 tx=0x7f126000b8d0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.044+0000 7f126f59e640 1 -- 172.21.15.115:0/3835596588 shutdown_connections 2023-12-01T03:11:08.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.044+0000 7f126f59e640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f125006a280 0x7f125006c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.045+0000 7f126f59e640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f127015cb60 0x7f12700c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.045+0000 7f126f59e640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12700cb430 0x7f12701546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.045+0000 7f126f59e640 1 --2- 172.21.15.115:0/3835596588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12700ca2d0 0x7f12701f3280 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.045+0000 7f126f59e640 1 -- 172.21.15.115:0/3835596588 >> 172.21.15.115:0/3835596588 conn(0x7f1270065960 msgr2=0x7f12700ad790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:08.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.045+0000 7f126f59e640 1 -- 172.21.15.115:0/3835596588 shutdown_connections 2023-12-01T03:11:08.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.045+0000 7f126f59e640 1 -- 172.21.15.115:0/3835596588 wait complete. 2023-12-01T03:11:08.064 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-01T03:11:08.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.276+0000 7fcc23ece640 1 Processor -- start 2023-12-01T03:11:08.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.276+0000 7fcc23ece640 1 -- start start 2023-12-01T03:11:08.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.277+0000 7fcc23ece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c142870 0x7fcc1c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc23ece640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcc1c0cd010 0x7fcc1c0b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc23ece640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1c0cd470 con 0x7fcc1c0b1c50 2023-12-01T03:11:08.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc23ece640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1c0b3ae0 con 0x7fcc1c0cd010 2023-12-01T03:11:08.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc23ece640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c142c50 con 0x7fcc1c142870 2023-12-01T03:11:08.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc21469640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcc1c0cd010 0x7fcc1c0b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc21469640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcc1c0cd010 0x7fcc1c0b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54340/0 (socket says 172.21.15.115:54340) 2023-12-01T03:11:08.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc21469640 1 -- 172.21.15.115:0/2959552251 learned_addr learned my addr 172.21.15.115:0/2959552251 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:08.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc21c6a640 1 --2- 172.21.15.115:0/2959552251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c142870 0x7fcc1c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc21469640 1 -- 172.21.15.115:0/2959552251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c142870 msgr2=0x7fcc1c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:08.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc21469640 1 --2- 172.21.15.115:0/2959552251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c142870 0x7fcc1c0ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc21469640 1 -- 172.21.15.115:0/2959552251 >> v1:172.21.15.142:6789/0 conn(0x7fcc1c0b1c50 legacy=0x7fcc1c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:08.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.278+0000 7fcc21469640 1 -- 172.21.15.115:0/2959552251 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c0b20e0 con 0x7fcc1c0cd010 2023-12-01T03:11:08.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.279+0000 7fcc21469640 1 --2- 172.21.15.115:0/2959552251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 0x7fcc1c0b1710 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0098a0 tx=0x7fcc0c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed23e21a6916b08b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:08.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.280+0000 7fcc20c68640 1 -- 172.21.15.115:0/2959552251 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c031640 con 0x7fcc1c0cd010 2023-12-01T03:11:08.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.280+0000 7fcc20c68640 1 -- 172.21.15.115:0/2959552251 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c0317a0 con 0x7fcc1c0cd010 2023-12-01T03:11:08.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.280+0000 7fcc20c68640 1 -- 172.21.15.115:0/2959552251 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c031b30 con 0x7fcc1c0cd010 2023-12-01T03:11:08.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.280+0000 7fcc23ece640 1 -- 172.21.15.115:0/2959552251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 msgr2=0x7fcc1c0b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.280+0000 7fcc23ece640 1 --2- 172.21.15.115:0/2959552251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 0x7fcc1c0b1710 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0098a0 tx=0x7fcc0c02b8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.280+0000 7fcc23ece640 1 -- 172.21.15.115:0/2959552251 shutdown_connections 2023-12-01T03:11:08.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.280+0000 7fcc23ece640 1 --2- 172.21.15.115:0/2959552251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 0x7fcc1c0b1710 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.280+0000 7fcc23ece640 1 --2- 172.21.15.115:0/2959552251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c142870 0x7fcc1c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.280+0000 7fcc23ece640 1 -- 172.21.15.115:0/2959552251 >> 172.21.15.115:0/2959552251 conn(0x7fcc1c065960 msgr2=0x7fcc1c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:08.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.281+0000 7fcc23ece640 1 -- 172.21.15.115:0/2959552251 shutdown_connections 2023-12-01T03:11:08.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.281+0000 7fcc23ece640 1 -- 172.21.15.115:0/2959552251 wait complete. 2023-12-01T03:11:08.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.282+0000 7fcc23ece640 1 Processor -- start 2023-12-01T03:11:08.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.282+0000 7fcc23ece640 1 -- start start 2023-12-01T03:11:08.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.283+0000 7fcc23ece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c0b1c50 0x7fcc1c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.283+0000 7fcc23ece640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 0x7fcc1c1f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.283+0000 7fcc23ece640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc1c142870 0x7fcc1c1ef900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.283+0000 7fcc23ece640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c140750 con 0x7fcc1c0cd010 2023-12-01T03:11:08.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.283+0000 7fcc23ece640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c13fc80 con 0x7fcc1c142870 2023-12-01T03:11:08.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.283+0000 7fcc23ece640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c1f3970 con 0x7fcc1c0b1c50 2023-12-01T03:11:08.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21469640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 0x7fcc1c1f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21469640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 0x7fcc1c1f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54354/0 (socket says 172.21.15.115:54354) 2023-12-01T03:11:08.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21469640 1 -- 172.21.15.115:0/3089491500 learned_addr learned my addr 172.21.15.115:0/3089491500 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:08.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21c6a640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c0b1c50 0x7fcc1c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21469640 1 -- 172.21.15.115:0/3089491500 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c0b1c50 msgr2=0x7fcc1c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21469640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c0b1c50 0x7fcc1c13f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21469640 1 -- 172.21.15.115:0/3089491500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc1c142870 msgr2=0x7fcc1c1ef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21469640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc1c142870 0x7fcc1c1ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21469640 1 -- 172.21.15.115:0/3089491500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0094d0 con 0x7fcc1c0cd010 2023-12-01T03:11:08.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21c6a640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c0b1c50 0x7fcc1c13f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:08.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.284+0000 7fcc21469640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 0x7fcc1c1f30e0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c000c00 tx=0x7fcc0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:08.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.285+0000 7fcc127fc640 1 -- 172.21.15.115:0/3089491500 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c03ac60 con 0x7fcc1c0cd010 2023-12-01T03:11:08.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.285+0000 7fcc127fc640 1 -- 172.21.15.115:0/3089491500 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c03ade0 con 0x7fcc1c0cd010 2023-12-01T03:11:08.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.285+0000 7fcc23ece640 1 -- 172.21.15.115:0/3089491500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c1f3620 con 0x7fcc1c0cd010 2023-12-01T03:11:08.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.285+0000 7fcc127fc640 1 -- 172.21.15.115:0/3089491500 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c044630 con 0x7fcc1c0cd010 2023-12-01T03:11:08.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.285+0000 7fcc23ece640 1 -- 172.21.15.115:0/3089491500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1c140190 con 0x7fcc1c0cd010 2023-12-01T03:11:08.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.286+0000 7fcc23ece640 1 -- 172.21.15.115:0/3089491500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1c04e890 con 0x7fcc1c0cd010 2023-12-01T03:11:08.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.287+0000 7fcc127fc640 1 -- 172.21.15.115:0/3089491500 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 132) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fcc0c02f050 con 0x7fcc1c0cd010 2023-12-01T03:11:08.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.288+0000 7fcc127fc640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc0006a280 0x7fcc0006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.288+0000 7fcc127fc640 1 -- 172.21.15.115:0/3089491500 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7fcc0c0aac70 con 0x7fcc1c0cd010 2023-12-01T03:11:08.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.288+0000 7fcc21c6a640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc0006a280 0x7fcc0006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.289+0000 7fcc21c6a640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc0006a280 0x7fcc0006c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800b820 tx=0x7fcc18008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:08.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.292+0000 7fcc127fc640 1 -- 172.21.15.115:0/3089491500 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcc0c075e60 con 0x7fcc1c0cd010 2023-12-01T03:11:08.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.468+0000 7fcc23ece640 1 -- 172.21.15.115:0/3089491500 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc1c0b20e0 con 0x7fcc1c0cd010 2023-12-01T03:11:08.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.470+0000 7fcc127fc640 1 -- 172.21.15.115:0/3089491500 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v220) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcc1c0b20e0 con 0x7fcc1c0cd010 2023-12-01T03:11:08.471 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:11:08.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.472+0000 7fcc23ece640 1 -- 172.21.15.115:0/3089491500 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc0006a280 msgr2=0x7fcc0006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.472+0000 7fcc23ece640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc0006a280 0x7fcc0006c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800b820 tx=0x7fcc18008040 comp rx=0 tx=0).stop 2023-12-01T03:11:08.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.472+0000 7fcc23ece640 1 -- 172.21.15.115:0/3089491500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 msgr2=0x7fcc1c1f30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.472+0000 7fcc23ece640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 0x7fcc1c1f30e0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c000c00 tx=0x7fcc0c0046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.472+0000 7fcc23ece640 1 -- 172.21.15.115:0/3089491500 shutdown_connections 2023-12-01T03:11:08.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.472+0000 7fcc23ece640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc0006a280 0x7fcc0006c740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.472+0000 7fcc23ece640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc1c142870 0x7fcc1c1ef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.472+0000 7fcc23ece640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc1c0cd010 0x7fcc1c1f30e0 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.472+0000 7fcc23ece640 1 --2- 172.21.15.115:0/3089491500 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc1c0b1c50 0x7fcc1c13f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.473+0000 7fcc23ece640 1 -- 172.21.15.115:0/3089491500 >> 172.21.15.115:0/3089491500 conn(0x7fcc1c065960 msgr2=0x7fcc1c15a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:08.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.473+0000 7fcc23ece640 1 -- 172.21.15.115:0/3089491500 shutdown_connections 2023-12-01T03:11:08.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.473+0000 7fcc23ece640 1 -- 172.21.15.115:0/3089491500 wait complete. 2023-12-01T03:11:08.493 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.697+0000 7f1cab870640 1 Processor -- start 2023-12-01T03:11:08.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.697+0000 7f1cab870640 1 -- start start 2023-12-01T03:11:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.698+0000 7f1cab870640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 0x7f1ca40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.698+0000 7f1cab870640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1ca40adf10 0x7f1ca40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.699+0000 7f1cab870640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca40add10 con 0x7f1ca406f2f0 2023-12-01T03:11:08.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.699+0000 7f1cab870640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca40c5760 con 0x7f1ca40adf10 2023-12-01T03:11:08.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.699+0000 7f1cab870640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca415e6d0 con 0x7f1ca415e2f0 2023-12-01T03:11:08.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.699+0000 7f1ca960c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 0x7f1ca40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.699+0000 7f1ca8e0b640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f1ca406f2f0 0x7f1ca40c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:60292/0 (socket says 172.21.15.115:60292) 2023-12-01T03:11:08.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.699+0000 7f1ca8e0b640 1 -- 172.21.15.115:0/1134222694 learned_addr learned my addr 172.21.15.115:0/1134222694 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:08.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.699+0000 7f1ca9e0d640 1 --2- 172.21.15.115:0/1134222694 >> v2:172.21.15.142:3300/0 conn(0x7f1ca40adf10 0x7f1ca40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.700+0000 7f1c97fff640 1 -- 172.21.15.115:0/1134222694 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491665381 0 0) 0x7f1ca40add10 con 0x7f1ca406f2f0 2023-12-01T03:11:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.700+0000 7f1c97fff640 1 -- 172.21.15.115:0/1134222694 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c90003520 con 0x7f1ca406f2f0 2023-12-01T03:11:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.700+0000 7f1ca960c640 1 -- 172.21.15.115:0/1134222694 >> v2:172.21.15.142:3300/0 conn(0x7f1ca40adf10 msgr2=0x7f1ca40caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.700+0000 7f1ca960c640 1 --2- 172.21.15.115:0/1134222694 >> v2:172.21.15.142:3300/0 conn(0x7f1ca40adf10 0x7f1ca40caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.700+0000 7f1ca960c640 1 -- 172.21.15.115:0/1134222694 >> v1:172.21.15.142:6789/0 conn(0x7f1ca406f2f0 legacy=0x7f1ca40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.700+0000 7f1ca960c640 1 -- 172.21.15.115:0/1134222694 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40cf7b0 con 0x7f1ca415e2f0 2023-12-01T03:11:08.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.700+0000 7f1ca9e0d640 1 --2- 172.21.15.115:0/1134222694 >> v2:172.21.15.142:3300/0 conn(0x7f1ca40adf10 0x7f1ca40caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:11:08.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.702+0000 7f1ca960c640 1 --2- 172.21.15.115:0/1134222694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 0x7f1ca40ca660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1c980097a0 tx=0x7f1c9802b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28af036a5156e7ab server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:08.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.702+0000 7f1c97fff640 1 -- 172.21.15.115:0/1134222694 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9802f050 con 0x7f1ca415e2f0 2023-12-01T03:11:08.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.702+0000 7f1c97fff640 1 -- 172.21.15.115:0/1134222694 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c98031700 con 0x7f1ca415e2f0 2023-12-01T03:11:08.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.702+0000 7f1c97fff640 1 -- 172.21.15.115:0/1134222694 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98031a70 con 0x7f1ca415e2f0 2023-12-01T03:11:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.703+0000 7f1cab870640 1 -- 172.21.15.115:0/1134222694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 msgr2=0x7f1ca40ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.703+0000 7f1cab870640 1 --2- 172.21.15.115:0/1134222694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 0x7f1ca40ca660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1c980097a0 tx=0x7f1c9802b7a0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.703+0000 7f1cab870640 1 -- 172.21.15.115:0/1134222694 shutdown_connections 2023-12-01T03:11:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.703+0000 7f1cab870640 1 --2- 172.21.15.115:0/1134222694 >> v2:172.21.15.142:3300/0 conn(0x7f1ca40adf10 0x7f1ca40caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.703+0000 7f1cab870640 1 --2- 172.21.15.115:0/1134222694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 0x7f1ca40ca660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.703+0000 7f1cab870640 1 -- 172.21.15.115:0/1134222694 >> 172.21.15.115:0/1134222694 conn(0x7f1ca40ce880 msgr2=0x7f1ca40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.703+0000 7f1cab870640 1 -- 172.21.15.115:0/1134222694 shutdown_connections 2023-12-01T03:11:08.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.703+0000 7f1cab870640 1 -- 172.21.15.115:0/1134222694 wait complete. 2023-12-01T03:11:08.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.705+0000 7f1cab870640 1 Processor -- start 2023-12-01T03:11:08.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.705+0000 7f1cab870640 1 -- start start 2023-12-01T03:11:08.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.706+0000 7f1cab870640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ca406f2f0 0x7f1ca4159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.706+0000 7f1cab870640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ca40adf10 0x7f1ca40c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.706+0000 7f1cab870640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 0x7f1ca40c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.706+0000 7f1cab870640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40c6ad0 con 0x7f1ca40adf10 2023-12-01T03:11:08.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.706+0000 7f1cab870640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40c5ec0 con 0x7f1ca406f2f0 2023-12-01T03:11:08.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.706+0000 7f1cab870640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40c61a0 con 0x7f1ca415e2f0 2023-12-01T03:11:08.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca8e0b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ca40adf10 0x7f1ca40c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca8e0b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ca40adf10 0x7f1ca40c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54362/0 (socket says 172.21.15.115:54362) 2023-12-01T03:11:08.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca8e0b640 1 -- 172.21.15.115:0/1780543196 learned_addr learned my addr 172.21.15.115:0/1780543196 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:08.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca960c640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ca406f2f0 0x7f1ca4159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca9e0d640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 0x7f1ca40c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca8e0b640 1 -- 172.21.15.115:0/1780543196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 msgr2=0x7f1ca40c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca8e0b640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 0x7f1ca40c4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca8e0b640 1 -- 172.21.15.115:0/1780543196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ca406f2f0 msgr2=0x7f1ca4159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca8e0b640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ca406f2f0 0x7f1ca4159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca8e0b640 1 -- 172.21.15.115:0/1780543196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c009400 con 0x7f1ca40adf10 2023-12-01T03:11:08.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca960c640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ca406f2f0 0x7f1ca4159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:08.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.707+0000 7f1ca8e0b640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ca40adf10 0x7f1ca40c6f60 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000cbc0 tx=0x7f1ca000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:08.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.708+0000 7f1c95ffb640 1 -- 172.21.15.115:0/1780543196 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000ad40 con 0x7f1ca40adf10 2023-12-01T03:11:08.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.708+0000 7f1c95ffb640 1 -- 172.21.15.115:0/1780543196 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca0012400 con 0x7f1ca40adf10 2023-12-01T03:11:08.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.708+0000 7f1c95ffb640 1 -- 172.21.15.115:0/1780543196 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000aea0 con 0x7f1ca40adf10 2023-12-01T03:11:08.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.708+0000 7f1cab870640 1 -- 172.21.15.115:0/1780543196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c98009400 con 0x7f1ca40adf10 2023-12-01T03:11:08.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.708+0000 7f1cab870640 1 -- 172.21.15.115:0/1780543196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca40c6760 con 0x7f1ca40adf10 2023-12-01T03:11:08.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.709+0000 7f1cab870640 1 -- 172.21.15.115:0/1780543196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca406fda0 con 0x7f1ca40adf10 2023-12-01T03:11:08.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.710+0000 7f1c95ffb640 1 -- 172.21.15.115:0/1780543196 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 133) v1 ==== 88755+0+0 (secure 0 0 0) 0x7f1ca0007070 con 0x7f1ca40adf10 2023-12-01T03:11:08.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.711+0000 7f1c95ffb640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1c8806a5e0 0x7f1c8806caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:08.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.712+0000 7f1c95ffb640 1 -- 172.21.15.115:0/1780543196 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7f1ca0089340 con 0x7f1ca40adf10 2023-12-01T03:11:08.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.712+0000 7f1ca960c640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1c8806a5e0 0x7f1c8806caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:08.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.713+0000 7f1ca960c640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1c8806a5e0 0x7f1c8806caa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c98035000 tx=0x7f1c98008030 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:08.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.716+0000 7f1c95ffb640 1 -- 172.21.15.115:0/1780543196 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1ca0053fa0 con 0x7f1ca40adf10 2023-12-01T03:11:08.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.946+0000 7f1cab870640 1 -- 172.21.15.115:0/1780543196 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ca404e890 con 0x7f1ca40adf10 2023-12-01T03:11:08.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.947+0000 7f1c95ffb640 1 -- 172.21.15.115:0/1780543196 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5793 (secure 0 0 0) 0x7f1ca00575c0 con 0x7f1ca40adf10 2023-12-01T03:11:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:08.949 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":101,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:25.146035+0000","modified":"2023-12-01T03:10:28.416554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10653":{"gid":10653,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/720518620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":720518620},{"type":"v1","addr":"172.21.15.142:6834","nonce":720518620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:38.965513+0000","modified":"2023-12-01T03:10:41.983462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7333":{"gid":7333,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-01T03:11:08.949 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 101 2023-12-01T03:11:08.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.949+0000 7f1cab870640 1 -- 172.21.15.115:0/1780543196 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1c8806a5e0 msgr2=0x7f1c8806caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.949+0000 7f1cab870640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1c8806a5e0 0x7f1c8806caa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c98035000 tx=0x7f1c98008030 comp rx=0 tx=0).stop 2023-12-01T03:11:08.951 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.949+0000 7f1cab870640 1 -- 172.21.15.115:0/1780543196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ca40adf10 msgr2=0x7f1ca40c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:08.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.949+0000 7f1cab870640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ca40adf10 0x7f1ca40c6f60 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000cbc0 tx=0x7f1ca000a430 comp rx=0 tx=0).stop 2023-12-01T03:11:08.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.949+0000 7f1cab870640 1 -- 172.21.15.115:0/1780543196 shutdown_connections 2023-12-01T03:11:08.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.949+0000 7f1cab870640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1c8806a5e0 0x7f1c8806caa0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.949+0000 7f1cab870640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ca415e2f0 0x7f1ca40c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.949+0000 7f1cab870640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ca40adf10 0x7f1ca40c6f60 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.949+0000 7f1cab870640 1 --2- 172.21.15.115:0/1780543196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ca406f2f0 0x7f1ca4159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:08.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.949+0000 7f1cab870640 1 -- 172.21.15.115:0/1780543196 >> 172.21.15.115:0/1780543196 conn(0x7f1ca40ce880 msgr2=0x7f1ca406b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:08.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.950+0000 7f1cab870640 1 -- 172.21.15.115:0/1780543196 shutdown_connections 2023-12-01T03:11:08.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:08.950+0000 7f1cab870640 1 -- 172.21.15.115:0/1780543196 wait complete. 2023-12-01T03:11:08.970 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:09.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.174+0000 7fcb3fa4a640 1 Processor -- start 2023-12-01T03:11:09.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.174+0000 7fcb3fa4a640 1 -- start start 2023-12-01T03:11:09.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.176+0000 7fcb3fa4a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcb3815d220 0x7fcb380cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.176+0000 7fcb3fa4a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcb380ad2b0 0x7fcb380ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.176+0000 7fcb3fa4a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3815d600 con 0x7fcb380ade80 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.176+0000 7fcb3fa4a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3806cc10 con 0x7fcb3815d220 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.176+0000 7fcb3fa4a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3806c720 con 0x7fcb380ad2b0 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.176+0000 7fcb3d7e6640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcb3815d220 0x7fcb380cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.176+0000 7fcb3dfe7640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fcb380ade80 0x7fcb3806b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34678/0 (socket says 172.21.15.115:34678) 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.176+0000 7fcb3d7e6640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcb3815d220 0x7fcb380cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54364/0 (socket says 172.21.15.115:54364) 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.176+0000 7fcb3dfe7640 1 -- 172.21.15.115:0/2713247571 learned_addr learned my addr 172.21.15.115:0/2713247571 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.177+0000 7fcb3cfe5640 1 --2- 172.21.15.115:0/2713247571 >> v2:172.21.15.142:3300/0 conn(0x7fcb380ad2b0 0x7fcb380ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.177+0000 7fcb3d7e6640 1 -- 172.21.15.115:0/2713247571 >> v2:172.21.15.142:3300/0 conn(0x7fcb380ad2b0 msgr2=0x7fcb380ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.177+0000 7fcb3d7e6640 1 --2- 172.21.15.115:0/2713247571 >> v2:172.21.15.142:3300/0 conn(0x7fcb380ad2b0 0x7fcb380ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.177+0000 7fcb3d7e6640 1 -- 172.21.15.115:0/2713247571 >> v1:172.21.15.115:6789/0 conn(0x7fcb380ade80 legacy=0x7fcb3806b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.177+0000 7fcb3d7e6640 1 -- 172.21.15.115:0/2713247571 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3806c4b0 con 0x7fcb3815d220 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.177+0000 7fcb2ffff640 1 -- 172.21.15.115:0/2713247571 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030671975 0 0) 0x7fcb30001490 con 0x7fcb380ade80 2023-12-01T03:11:09.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.177+0000 7fcb3cfe5640 1 --2- 172.21.15.115:0/2713247571 >> v2:172.21.15.142:3300/0 conn(0x7fcb380ad2b0 0x7fcb380ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:09.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.178+0000 7fcb3d7e6640 1 --2- 172.21.15.115:0/2713247571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb3815d220 0x7fcb380cc570 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fcb380cc4e0 tx=0x7fcb3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41592fc10738da6b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:09.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.178+0000 7fcb2ffff640 1 -- 172.21.15.115:0/2713247571 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3402e070 con 0x7fcb3815d220 2023-12-01T03:11:09.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.178+0000 7fcb2ffff640 1 -- 172.21.15.115:0/2713247571 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb340315a0 con 0x7fcb3815d220 2023-12-01T03:11:09.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.178+0000 7fcb2ffff640 1 -- 172.21.15.115:0/2713247571 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34031930 con 0x7fcb3815d220 2023-12-01T03:11:09.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.179+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/2713247571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb3815d220 msgr2=0x7fcb380cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.179+0000 7fcb3fa4a640 1 --2- 172.21.15.115:0/2713247571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb3815d220 0x7fcb380cc570 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fcb380cc4e0 tx=0x7fcb3402b840 comp rx=0 tx=0).stop 2023-12-01T03:11:09.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.179+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/2713247571 shutdown_connections 2023-12-01T03:11:09.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.179+0000 7fcb3fa4a640 1 --2- 172.21.15.115:0/2713247571 >> v2:172.21.15.142:3300/0 conn(0x7fcb380ad2b0 0x7fcb380ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.179+0000 7fcb3fa4a640 1 --2- 172.21.15.115:0/2713247571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb3815d220 0x7fcb380cc570 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.179+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/2713247571 >> 172.21.15.115:0/2713247571 conn(0x7fcb380ce880 msgr2=0x7fcb380cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:09.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.179+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/2713247571 shutdown_connections 2023-12-01T03:11:09.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.179+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/2713247571 wait complete. 2023-12-01T03:11:09.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.181+0000 7fcb3fa4a640 1 Processor -- start 2023-12-01T03:11:09.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.181+0000 7fcb3fa4a640 1 -- start start 2023-12-01T03:11:09.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.182+0000 7fcb3fa4a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb380ad2b0 0x7fcb38157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.182+0000 7fcb3fa4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb380ade80 0x7fcb380cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.182+0000 7fcb3fa4a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb3815d220 0x7fcb380ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.182+0000 7fcb3fa4a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380ca080 con 0x7fcb380ad2b0 2023-12-01T03:11:09.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.182+0000 7fcb3fa4a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380b3990 con 0x7fcb3815d220 2023-12-01T03:11:09.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.182+0000 7fcb3fa4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380c9400 con 0x7fcb380ade80 2023-12-01T03:11:09.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.182+0000 7fcb3d7e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb380ad2b0 0x7fcb38157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.182+0000 7fcb3d7e6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb380ad2b0 0x7fcb38157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54374/0 (socket says 172.21.15.115:54374) 2023-12-01T03:11:09.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.182+0000 7fcb3d7e6640 1 -- 172.21.15.115:0/3894526729 learned_addr learned my addr 172.21.15.115:0/3894526729 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:09.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb3d7e6640 1 -- 172.21.15.115:0/3894526729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb380ade80 msgr2=0x7fcb380cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb3d7e6640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb380ade80 0x7fcb380cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb3dfe7640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb3815d220 0x7fcb380ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb3d7e6640 1 -- 172.21.15.115:0/3894526729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb3815d220 msgr2=0x7fcb380ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb3d7e6640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb3815d220 0x7fcb380ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb3d7e6640 1 -- 172.21.15.115:0/3894526729 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34009400 con 0x7fcb380ad2b0 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb3d7e6640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb380ad2b0 0x7fcb38157fd0 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009890 tx=0x7fcb340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb2dffb640 1 -- 172.21.15.115:0/3894526729 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34032900 con 0x7fcb380ad2b0 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb2dffb640 1 -- 172.21.15.115:0/3894526729 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb34032a60 con 0x7fcb380ad2b0 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb2dffb640 1 -- 172.21.15.115:0/3894526729 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34032e10 con 0x7fcb380ad2b0 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.183+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/3894526729 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb380c96e0 con 0x7fcb380ad2b0 2023-12-01T03:11:09.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.184+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/3894526729 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb380c9ca0 con 0x7fcb380ad2b0 2023-12-01T03:11:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.185+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/3894526729 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3804e890 con 0x7fcb380ad2b0 2023-12-01T03:11:09.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.185+0000 7fcb2dffb640 1 -- 172.21.15.115:0/3894526729 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 133) v1 ==== 88755+0+0 (secure 0 0 0) 0x7fcb3403b3f0 con 0x7fcb380ad2b0 2023-12-01T03:11:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.186+0000 7fcb2dffb640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcb1c06a5e0 0x7fcb1c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.186+0000 7fcb2dffb640 1 -- 172.21.15.115:0/3894526729 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7fcb340ab4e0 con 0x7fcb380ad2b0 2023-12-01T03:11:09.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.187+0000 7fcb3cfe5640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcb1c06a5e0 0x7fcb1c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.187+0000 7fcb3cfe5640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcb1c06a5e0 0x7fcb1c06caa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcb28006fd0 tx=0x7fcb28008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:09.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.190+0000 7fcb2dffb640 1 -- 172.21.15.115:0/3894526729 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcb340760c0 con 0x7fcb380ad2b0 2023-12-01T03:11:09.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.380+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/3894526729 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb380c9140 con 0x7fcb380ad2b0 2023-12-01T03:11:09.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.384+0000 7fcb2dffb640 1 -- 172.21.15.115:0/3894526729 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+20900 (secure 0 0 0) 0x7fcb340796e0 con 0x7fcb380ad2b0 2023-12-01T03:11:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:09.386 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":220,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:05.783522+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:10:37.180599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:10:37.961783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/1558237180":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3235712957":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1161447385":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3956813425":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/4128909410":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3433151698":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1087193141":"2023-12-02T03:11:05.783435+0000","172.21.15.115:6824/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:6822/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3133490080":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2913786291":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/846563507":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3736942314":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/1606716517":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3870112353":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2954409059":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3112535338":"2023-12-02T03:10:55.627314+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:09.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.386+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/3894526729 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcb1c06a5e0 msgr2=0x7fcb1c06caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.386+0000 7fcb3fa4a640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcb1c06a5e0 0x7fcb1c06caa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcb28006fd0 tx=0x7fcb28008040 comp rx=0 tx=0).stop 2023-12-01T03:11:09.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.386+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/3894526729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb380ad2b0 msgr2=0x7fcb38157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.386+0000 7fcb3fa4a640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb380ad2b0 0x7fcb38157fd0 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009890 tx=0x7fcb340046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.386+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/3894526729 shutdown_connections 2023-12-01T03:11:09.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.386+0000 7fcb3fa4a640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcb1c06a5e0 0x7fcb1c06caa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.386+0000 7fcb3fa4a640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb380ade80 0x7fcb380cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.386+0000 7fcb3fa4a640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb3815d220 0x7fcb380ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.386+0000 7fcb3fa4a640 1 --2- 172.21.15.115:0/3894526729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb380ad2b0 0x7fcb38157fd0 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.386+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/3894526729 >> 172.21.15.115:0/3894526729 conn(0x7fcb380ce880 msgr2=0x7fcb3815b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:09.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.387+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/3894526729 shutdown_connections 2023-12-01T03:11:09.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.387+0000 7fcb3fa4a640 1 -- 172.21.15.115:0/3894526729 wait complete. 2023-12-01T03:11:09.408 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T03:11:09.408 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:09.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.614+0000 7f09328f8640 1 Processor -- start 2023-12-01T03:11:09.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.614+0000 7f09328f8640 1 -- start start 2023-12-01T03:11:09.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.615+0000 7f09328f8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c0cad70 0x7f092c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.615+0000 7f09328f8640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c0add10 con 0x7f092c06f2f0 2023-12-01T03:11:09.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.615+0000 7f09328f8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c0c5760 con 0x7f092c15e2f0 2023-12-01T03:11:09.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.615+0000 7f09328f8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c15e6d0 con 0x7f092c0cad70 2023-12-01T03:11:09.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.616+0000 7f0930e95640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f092c06f2f0 0x7f092c06bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34692/0 (socket says 172.21.15.115:34692) 2023-12-01T03:11:09.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.616+0000 7f0930e95640 1 -- 172.21.15.115:0/146919047 learned_addr learned my addr 172.21.15.115:0/146919047 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:09.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.616+0000 7f092b7fe640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c0cad70 0x7f092c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.616+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299082331 0 0) 0x7f092c0add10 con 0x7f092c06f2f0 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.616+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0914003520 con 0x7f092c06f2f0 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.616+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1160543811 0 0) 0x7f0914003520 con 0x7f092c06f2f0 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.616+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c0cad70 msgr2=0x7f092c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.616+0000 7f092affd640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c0cad70 0x7f092c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.616+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 >> v1:172.21.15.142:6789/0 conn(0x7f092c15e2f0 legacy=0x7f092c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.616+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c0cf7b0 con 0x7f092c06f2f0 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f09240019f0 con 0x7f092c06f2f0 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 >> v1:172.21.15.115:6789/0 conn(0x7f092c06f2f0 legacy=0x7f092c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f092affd640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0914003a10 0x7f0914003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f092affd640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c0cad70 0x7f09140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f092affd640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 0x7f09140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c15e6d0 con 0x7f0914004d50 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091400bc00 con 0x7f0914003a10 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091400bdc0 con 0x7f092c0cad70 2023-12-01T03:11:09.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f0930e95640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 0x7f09140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f092b7fe640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c0cad70 0x7f09140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f092bfff640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0914003a10 0x7f0914003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f0930e95640 1 -- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c0cad70 msgr2=0x7f09140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:09.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f0930e95640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c0cad70 0x7f09140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f0930e95640 1 -- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0914003a10 msgr2=0x7f0914003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:09.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f0930e95640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0914003a10 0x7f0914003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.617+0000 7f0930e95640 1 -- 172.21.15.115:0/146919047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914003520 con 0x7f0914004d50 2023-12-01T03:11:09.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.618+0000 7f0930e95640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 0x7f09140051d0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f092400a760 tx=0x7f09140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7c85f1008bc6bb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:09.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.619+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924034630 con 0x7f0914004d50 2023-12-01T03:11:09.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.619+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092402f070 con 0x7f0914004d50 2023-12-01T03:11:09.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.619+0000 7f092affd640 1 -- 172.21.15.115:0/146919047 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924034a10 con 0x7f0914004d50 2023-12-01T03:11:09.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.619+0000 7f09328f8640 1 -- 172.21.15.115:0/146919047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 msgr2=0x7f09140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.619+0000 7f09328f8640 1 --2- 172.21.15.115:0/146919047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 0x7f09140051d0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f092400a760 tx=0x7f09140057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.619+0000 7f0930e95640 1 -- 172.21.15.115:0/146919047 reap_dead start 2023-12-01T03:11:09.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.619+0000 7f09328f8640 1 -- 172.21.15.115:0/146919047 shutdown_connections 2023-12-01T03:11:09.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.619+0000 7f09328f8640 1 -- 172.21.15.115:0/146919047 >> 172.21.15.115:0/146919047 conn(0x7f092c0ce880 msgr2=0x7f092c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:09.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.619+0000 7f09328f8640 1 -- 172.21.15.115:0/146919047 shutdown_connections 2023-12-01T03:11:09.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.619+0000 7f09328f8640 1 -- 172.21.15.115:0/146919047 wait complete. 2023-12-01T03:11:09.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.621+0000 7f09328f8640 1 Processor -- start 2023-12-01T03:11:09.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.621+0000 7f09328f8640 1 -- start start 2023-12-01T03:11:09.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.622+0000 7f09328f8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 0x7f092c1e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.622+0000 7f09328f8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c1e6190 0x7f092c1e79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.622+0000 7f09328f8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f092c1e7ef0 0x7f092c0c3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.622+0000 7f09328f8640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c1e8380 con 0x7f0914004d50 2023-12-01T03:11:09.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.622+0000 7f09328f8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0c2770 con 0x7f092c1e7ef0 2023-12-01T03:11:09.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.622+0000 7f09328f8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0c2a50 con 0x7f092c1e6190 2023-12-01T03:11:09.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.622+0000 7f092bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 0x7f092c1e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.622+0000 7f092bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 0x7f092c1e5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54380/0 (socket says 172.21.15.115:54380) 2023-12-01T03:11:09.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.622+0000 7f092bfff640 1 -- 172.21.15.115:0/2291355645 learned_addr learned my addr 172.21.15.115:0/2291355645 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f0930e95640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f092c1e7ef0 0x7f092c0c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f092b7fe640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c1e6190 0x7f092c1e79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f092bfff640 1 -- 172.21.15.115:0/2291355645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c1e6190 msgr2=0x7f092c1e79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f092bfff640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c1e6190 0x7f092c1e79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f092bfff640 1 -- 172.21.15.115:0/2291355645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f092c1e7ef0 msgr2=0x7f092c0c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f092bfff640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f092c1e7ef0 0x7f092c0c3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f092bfff640 1 -- 172.21.15.115:0/2291355645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c0c2d30 con 0x7f0914004d50 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f092bfff640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 0x7f092c1e5c50 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f091c001eb0 tx=0x7f091c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f0928ff9640 1 -- 172.21.15.115:0/2291355645 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c005480 con 0x7f0914004d50 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f0928ff9640 1 -- 172.21.15.115:0/2291355645 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c003610 con 0x7f0914004d50 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f0928ff9640 1 -- 172.21.15.115:0/2291355645 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c003980 con 0x7f0914004d50 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.623+0000 7f09328f8640 1 -- 172.21.15.115:0/2291355645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c0c3010 con 0x7f0914004d50 2023-12-01T03:11:09.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.624+0000 7f09328f8640 1 -- 172.21.15.115:0/2291355645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c0c23b0 con 0x7f0914004d50 2023-12-01T03:11:09.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.625+0000 7f09328f8640 1 -- 172.21.15.115:0/2291355645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c06fda0 con 0x7f0914004d50 2023-12-01T03:11:09.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.626+0000 7f0928ff9640 1 -- 172.21.15.115:0/2291355645 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 133) v1 ==== 88755+0+0 (secure 0 0 0) 0x7f091c003ae0 con 0x7f0914004d50 2023-12-01T03:11:09.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.627+0000 7f0928ff9640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f091806a5e0 0x7f091806caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:09.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.627+0000 7f0928ff9640 1 -- 172.21.15.115:0/2291355645 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7f091c089230 con 0x7f0914004d50 2023-12-01T03:11:09.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.627+0000 7f092b7fe640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f091806a5e0 0x7f091806caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:09.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.628+0000 7f092b7fe640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f091806a5e0 0x7f091806caa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f092c0cca80 tx=0x7f0920008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:09.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.631+0000 7f0928ff9640 1 -- 172.21.15.115:0/2291355645 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f091c053eb0 con 0x7f0914004d50 2023-12-01T03:11:09.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.848+0000 7f09328f8640 1 -- 172.21.15.115:0/2291355645 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f092c04e890 con 0x7f0914004d50 2023-12-01T03:11:09.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.849+0000 7f0928ff9640 1 -- 172.21.15.115:0/2291355645 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5793 (secure 0 0 0) 0x7f091c0574d0 con 0x7f0914004d50 2023-12-01T03:11:09.851 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:09.851 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":101,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:25.146035+0000","modified":"2023-12-01T03:10:28.416554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10653":{"gid":10653,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/720518620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":720518620},{"type":"v1","addr":"172.21.15.142:6834","nonce":720518620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:38.965513+0000","modified":"2023-12-01T03:10:41.983462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7333":{"gid":7333,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-01T03:11:09.851 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 101 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 -- 172.21.15.115:0/2291355645 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f091806a5e0 msgr2=0x7f091806caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f091806a5e0 0x7f091806caa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f092c0cca80 tx=0x7f0920008040 comp rx=0 tx=0).stop 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 -- 172.21.15.115:0/2291355645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 msgr2=0x7f092c1e5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 0x7f092c1e5c50 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f091c001eb0 tx=0x7f091c00d9d0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 -- 172.21.15.115:0/2291355645 shutdown_connections 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f091806a5e0 0x7f091806caa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f092c1e6190 0x7f092c1e79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f092c1e7ef0 0x7f092c0c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 --2- 172.21.15.115:0/2291355645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0914004d50 0x7f092c1e5c50 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 -- 172.21.15.115:0/2291355645 >> 172.21.15.115:0/2291355645 conn(0x7f092c0ce880 msgr2=0x7f092c0ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:09.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 -- 172.21.15.115:0/2291355645 shutdown_connections 2023-12-01T03:11:09.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:09.852+0000 7f09328f8640 1 -- 172.21.15.115:0/2291355645 wait complete. 2023-12-01T03:11:09.872 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:10.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.077+0000 7ff05359e640 1 Processor -- start 2023-12-01T03:11:10.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.077+0000 7ff05359e640 1 -- start start 2023-12-01T03:11:10.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.078+0000 7ff05359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff054145620 0x7ff0540ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05359e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff05406b980 0x7ff0540c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05359e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0540ad7e0 con 0x7ff0540cdd80 2023-12-01T03:11:10.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05359e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0540cc8e0 con 0x7ff054145620 2023-12-01T03:11:10.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05359e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0540ce260 con 0x7ff05406b980 2023-12-01T03:11:10.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff054145620 0x7ff0540ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:10.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff051d9b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff0540cdd80 0x7ff05406b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34696/0 (socket says 172.21.15.115:34696) 2023-12-01T03:11:10.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff054145620 0x7ff0540ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54390/0 (socket says 172.21.15.115:54390) 2023-12-01T03:11:10.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff051d9b640 1 -- 172.21.15.115:0/2483243146 learned_addr learned my addr 172.21.15.115:0/2483243146 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:10.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff052d9d640 1 --2- 172.21.15.115:0/2483243146 >> v2:172.21.15.142:3300/0 conn(0x7ff05406b980 0x7ff0540c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:10.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05259c640 1 -- 172.21.15.115:0/2483243146 >> v2:172.21.15.142:3300/0 conn(0x7ff05406b980 msgr2=0x7ff0540c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:10.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05259c640 1 --2- 172.21.15.115:0/2483243146 >> v2:172.21.15.142:3300/0 conn(0x7ff05406b980 0x7ff0540c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05259c640 1 -- 172.21.15.115:0/2483243146 >> v1:172.21.15.115:6789/0 conn(0x7ff0540cdd80 legacy=0x7ff05406b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05259c640 1 -- 172.21.15.115:0/2483243146 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540cc320 con 0x7ff054145620 2023-12-01T03:11:10.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.079+0000 7ff05159a640 1 -- 172.21.15.115:0/2483243146 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111404067 0 0) 0x7ff0540ad7e0 con 0x7ff0540cdd80 2023-12-01T03:11:10.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.080+0000 7ff05259c640 1 --2- 172.21.15.115:0/2483243146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 0x7ff0540ad2a0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7ff0440097a0 tx=0x7ff04402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41902d843af8fe01 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:10.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.081+0000 7ff05159a640 1 -- 172.21.15.115:0/2483243146 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044031640 con 0x7ff054145620 2023-12-01T03:11:10.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.081+0000 7ff05159a640 1 -- 172.21.15.115:0/2483243146 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0440317a0 con 0x7ff054145620 2023-12-01T03:11:10.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.081+0000 7ff05159a640 1 -- 172.21.15.115:0/2483243146 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044031b30 con 0x7ff054145620 2023-12-01T03:11:10.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.081+0000 7ff05359e640 1 -- 172.21.15.115:0/2483243146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 msgr2=0x7ff0540ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.081+0000 7ff05359e640 1 --2- 172.21.15.115:0/2483243146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 0x7ff0540ad2a0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7ff0440097a0 tx=0x7ff04402b8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.081+0000 7ff05359e640 1 -- 172.21.15.115:0/2483243146 shutdown_connections 2023-12-01T03:11:10.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.081+0000 7ff05359e640 1 --2- 172.21.15.115:0/2483243146 >> v2:172.21.15.142:3300/0 conn(0x7ff05406b980 0x7ff0540c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.081+0000 7ff05359e640 1 --2- 172.21.15.115:0/2483243146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 0x7ff0540ad2a0 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.081+0000 7ff05359e640 1 -- 172.21.15.115:0/2483243146 >> 172.21.15.115:0/2483243146 conn(0x7ff0540ce640 msgr2=0x7ff0540cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:10.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.082+0000 7ff05359e640 1 -- 172.21.15.115:0/2483243146 shutdown_connections 2023-12-01T03:11:10.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.082+0000 7ff05359e640 1 -- 172.21.15.115:0/2483243146 wait complete. 2023-12-01T03:11:10.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.083+0000 7ff05359e640 1 Processor -- start 2023-12-01T03:11:10.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.084+0000 7ff05359e640 1 -- start start 2023-12-01T03:11:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.084+0000 7ff05359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05406b980 0x7ff05415fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff05359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0540cdd80 0x7ff054071480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff05359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 0x7ff0540719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff05359e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff054144270 con 0x7ff054145620 2023-12-01T03:11:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff05359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05406f5f0 con 0x7ff05406b980 2023-12-01T03:11:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff05359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0541414d0 con 0x7ff0540cdd80 2023-12-01T03:11:10.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff052d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 0x7ff0540719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff052d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 0x7ff0540719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54398/0 (socket says 172.21.15.115:54398) 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff052d9d640 1 -- 172.21.15.115:0/2480605544 learned_addr learned my addr 172.21.15.115:0/2480605544 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff05259c640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05406b980 0x7ff05415fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff051d9b640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0540cdd80 0x7ff054071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff052d9d640 1 -- 172.21.15.115:0/2480605544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0540cdd80 msgr2=0x7ff054071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff052d9d640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0540cdd80 0x7ff054071480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff052d9d640 1 -- 172.21.15.115:0/2480605544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05406b980 msgr2=0x7ff05415fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff052d9d640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05406b980 0x7ff05415fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff052d9d640 1 -- 172.21.15.115:0/2480605544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044009400 con 0x7ff054145620 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.085+0000 7ff05259c640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05406b980 0x7ff05415fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:10.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.086+0000 7ff052d9d640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 0x7ff0540719c0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00c5c0 tx=0x7ff04c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:10.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.086+0000 7ff04affd640 1 -- 172.21.15.115:0/2480605544 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c004220 con 0x7ff054145620 2023-12-01T03:11:10.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.086+0000 7ff04affd640 1 -- 172.21.15.115:0/2480605544 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c004380 con 0x7ff054145620 2023-12-01T03:11:10.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.086+0000 7ff04affd640 1 -- 172.21.15.115:0/2480605544 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c003830 con 0x7ff054145620 2023-12-01T03:11:10.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.086+0000 7ff05359e640 1 -- 172.21.15.115:0/2480605544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff054160070 con 0x7ff054145620 2023-12-01T03:11:10.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.086+0000 7ff05359e640 1 -- 172.21.15.115:0/2480605544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff054143cb0 con 0x7ff054145620 2023-12-01T03:11:10.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.087+0000 7ff05359e640 1 -- 172.21.15.115:0/2480605544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05406f3c0 con 0x7ff054145620 2023-12-01T03:11:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.088+0000 7ff04affd640 1 -- 172.21.15.115:0/2480605544 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 133) v1 ==== 88755+0+0 (secure 0 0 0) 0x7ff04c003990 con 0x7ff054145620 2023-12-01T03:11:10.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.089+0000 7ff04affd640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff03406a610 0x7ff03406cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.089+0000 7ff04affd640 1 -- 172.21.15.115:0/2480605544 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7ff04c089140 con 0x7ff054145620 2023-12-01T03:11:10.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.089+0000 7ff05259c640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff03406a610 0x7ff03406cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:10.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.090+0000 7ff05259c640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff03406a610 0x7ff03406cad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff044036000 tx=0x7ff04403c040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:10.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.094+0000 7ff04affd640 1 -- 172.21.15.115:0/2480605544 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff05406f3c0 con 0x7ff054145620 2023-12-01T03:11:10.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.318+0000 7ff05359e640 1 -- 172.21.15.115:0/2480605544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff05406f3c0 con 0x7ff054145620 2023-12-01T03:11:10.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.320+0000 7ff04affd640 1 -- 172.21.15.115:0/2480605544 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5793 (secure 0 0 0) 0x7ff05406f3c0 con 0x7ff054145620 2023-12-01T03:11:10.322 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:10.322 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":101,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:25.146035+0000","modified":"2023-12-01T03:10:28.416554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10653},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10653":{"gid":10653,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/720518620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":720518620},{"type":"v1","addr":"172.21.15.142:6834","nonce":720518620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:38.965513+0000","modified":"2023-12-01T03:10:41.983462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7333":{"gid":7333,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-01T03:11:10.322 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 101 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.322+0000 7ff05359e640 1 -- 172.21.15.115:0/2480605544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff03406a610 msgr2=0x7ff03406cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.322+0000 7ff05359e640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff03406a610 0x7ff03406cad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff044036000 tx=0x7ff04403c040 comp rx=0 tx=0).stop 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.322+0000 7ff05359e640 1 -- 172.21.15.115:0/2480605544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 msgr2=0x7ff0540719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.322+0000 7ff05359e640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 0x7ff0540719c0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00c5c0 tx=0x7ff04c00ca80 comp rx=0 tx=0).stop 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.323+0000 7ff05359e640 1 -- 172.21.15.115:0/2480605544 shutdown_connections 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.323+0000 7ff05359e640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff03406a610 0x7ff03406cad0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.323+0000 7ff05359e640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff0540cdd80 0x7ff054071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.323+0000 7ff05359e640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff054145620 0x7ff0540719c0 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.323+0000 7ff05359e640 1 --2- 172.21.15.115:0/2480605544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05406b980 0x7ff05415fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.323+0000 7ff05359e640 1 -- 172.21.15.115:0/2480605544 >> 172.21.15.115:0/2480605544 conn(0x7ff0540ce640 msgr2=0x7ff0540c9560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.323+0000 7ff05359e640 1 -- 172.21.15.115:0/2480605544 shutdown_connections 2023-12-01T03:11:10.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.323+0000 7ff05359e640 1 -- 172.21.15.115:0/2480605544 wait complete. 2023-12-01T03:11:10.344 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:10.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.566+0000 7f8f96599640 1 Processor -- start 2023-12-01T03:11:10.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.566+0000 7f8f96599640 1 -- start start 2023-12-01T03:11:10.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.567+0000 7f8f96599640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8f900cad70 0x7f8f9006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.567+0000 7f8f96599640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f900add10 con 0x7f8f9006f2f0 2023-12-01T03:11:10.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.567+0000 7f8f96599640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f900c5760 con 0x7f8f9015e2f0 2023-12-01T03:11:10.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.567+0000 7f8f96599640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9015e6d0 con 0x7f8f900cad70 2023-12-01T03:11:10.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.568+0000 7f8f94b36640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8f9006f2f0 0x7f8f9006bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34706/0 (socket says 172.21.15.115:34706) 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.568+0000 7f8f94b36640 1 -- 172.21.15.115:0/223250687 learned_addr learned my addr 172.21.15.115:0/223250687 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.568+0000 7f8f87fff640 1 --2- 172.21.15.115:0/223250687 >> v2:172.21.15.115:3300/0 conn(0x7f8f900cad70 0x7f8f9006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.568+0000 7f8f8f7fe640 1 -- 172.21.15.115:0/223250687 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512916436 0 0) 0x7f8f900add10 con 0x7f8f9006f2f0 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.568+0000 7f8f8f7fe640 1 -- 172.21.15.115:0/223250687 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f7c003520 con 0x7f8f9006f2f0 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.568+0000 7f8f87fff640 1 -- 172.21.15.115:0/223250687 >> v1:172.21.15.142:6789/0 conn(0x7f8f9015e2f0 legacy=0x7f8f900ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.568+0000 7f8f87fff640 1 -- 172.21.15.115:0/223250687 >> v1:172.21.15.115:6789/0 conn(0x7f8f9006f2f0 legacy=0x7f8f9006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.568+0000 7f8f87fff640 1 -- 172.21.15.115:0/223250687 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900cf7b0 con 0x7f8f900cad70 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.569+0000 7f8f87fff640 1 --2- 172.21.15.115:0/223250687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f900cad70 0x7f8f9006b5c0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f8f780097d0 tx=0x7f8f7802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55746cfb47a0164 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.570+0000 7f8f8f7fe640 1 -- 172.21.15.115:0/223250687 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f780316f0 con 0x7f8f900cad70 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.570+0000 7f8f8f7fe640 1 -- 172.21.15.115:0/223250687 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f78031850 con 0x7f8f900cad70 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.570+0000 7f8f8f7fe640 1 -- 172.21.15.115:0/223250687 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78031be0 con 0x7f8f900cad70 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.570+0000 7f8f96599640 1 -- 172.21.15.115:0/223250687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f900cad70 msgr2=0x7f8f9006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.570+0000 7f8f96599640 1 --2- 172.21.15.115:0/223250687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f900cad70 0x7f8f9006b5c0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f8f780097d0 tx=0x7f8f7802b790 comp rx=0 tx=0).stop 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.570+0000 7f8f96599640 1 -- 172.21.15.115:0/223250687 shutdown_connections 2023-12-01T03:11:10.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.570+0000 7f8f96599640 1 --2- 172.21.15.115:0/223250687 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f900cad70 0x7f8f9006b5c0 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.570+0000 7f8f96599640 1 -- 172.21.15.115:0/223250687 >> 172.21.15.115:0/223250687 conn(0x7f8f900ce880 msgr2=0x7f8f900cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.570+0000 7f8f96599640 1 -- 172.21.15.115:0/223250687 shutdown_connections 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.570+0000 7f8f96599640 1 -- 172.21.15.115:0/223250687 wait complete. 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.572+0000 7f8f96599640 1 Processor -- start 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.572+0000 7f8f96599640 1 -- start start 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.573+0000 7f8f96599640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f9006f2f0 0x7f8f90153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.573+0000 7f8f96599640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f900cad70 0x7f8f90153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.573+0000 7f8f96599640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f9015e2f0 0x7f8f900c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.573+0000 7f8f96599640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900ae440 con 0x7f8f9006f2f0 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.573+0000 7f8f96599640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900ae720 con 0x7f8f9015e2f0 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.573+0000 7f8f96599640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900aea00 con 0x7f8f900cad70 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.573+0000 7f8f8ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f9006f2f0 0x7f8f90153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.573+0000 7f8f8ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f9006f2f0 0x7f8f90153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54414/0 (socket says 172.21.15.115:54414) 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.573+0000 7f8f8ffff640 1 -- 172.21.15.115:0/3752224278 learned_addr learned my addr 172.21.15.115:0/3752224278 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:10.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.574+0000 7f8f8ffff640 1 -- 172.21.15.115:0/3752224278 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f900cad70 msgr2=0x7f8f90153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.574+0000 7f8f8ffff640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f900cad70 0x7f8f90153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.574+0000 7f8f8ffff640 1 -- 172.21.15.115:0/3752224278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f9015e2f0 msgr2=0x7f8f900c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.574+0000 7f8f8ffff640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f9015e2f0 0x7f8f900c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.574+0000 7f8f8ffff640 1 -- 172.21.15.115:0/3752224278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78009400 con 0x7f8f9006f2f0 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.574+0000 7f8f8ffff640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f9006f2f0 0x7f8f90153e20 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000c430 tx=0x7f8f8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.574+0000 7f8f8d7fa640 1 -- 172.21.15.115:0/3752224278 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80011790 con 0x7f8f9006f2f0 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.574+0000 7f8f8d7fa640 1 -- 172.21.15.115:0/3752224278 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f800118f0 con 0x7f8f9006f2f0 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.574+0000 7f8f8d7fa640 1 -- 172.21.15.115:0/3752224278 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80011ce0 con 0x7f8f9006f2f0 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.574+0000 7f8f96599640 1 -- 172.21.15.115:0/3752224278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f900aece0 con 0x7f8f9006f2f0 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.575+0000 7f8f96599640 1 -- 172.21.15.115:0/3752224278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f900c28f0 con 0x7f8f9006f2f0 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.576+0000 7f8f96599640 1 -- 172.21.15.115:0/3752224278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9006fda0 con 0x7f8f9006f2f0 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.577+0000 7f8f8d7fa640 1 -- 172.21.15.115:0/3752224278 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f8f8001c460 con 0x7f8f9006f2f0 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.577+0000 7f8f8d7fa640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8f7006a670 0x7f8f7006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:10.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.578+0000 7f8f8d7fa640 1 -- 172.21.15.115:0/3752224278 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7f8f80089350 con 0x7f8f9006f2f0 2023-12-01T03:11:10.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.578+0000 7f8f87fff640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8f7006a670 0x7f8f7006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:10.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.578+0000 7f8f87fff640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8f7006a670 0x7f8f7006cb30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8f78036000 tx=0x7f8f78030000 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:10.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.582+0000 7f8f8d7fa640 1 -- 172.21.15.115:0/3752224278 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8f80053fa0 con 0x7f8f9006f2f0 2023-12-01T03:11:10.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.772+0000 7f8f96599640 1 -- 172.21.15.115:0/3752224278 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f900af200 con 0x7f8f9006f2f0 2023-12-01T03:11:10.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.776+0000 7f8f8d7fa640 1 -- 172.21.15.115:0/3752224278 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+20900 (secure 0 0 0) 0x7f8f800575c0 con 0x7f8f9006f2f0 2023-12-01T03:11:10.787 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:10.787 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":220,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:05.783522+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:10:22.683496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-01T03:10:24.156854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:10:37.180599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:10:37.961783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/1558237180":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3235712957":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1161447385":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3956813425":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/4128909410":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3433151698":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1087193141":"2023-12-02T03:11:05.783435+0000","172.21.15.115:6824/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:6822/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3133490080":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2913786291":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/846563507":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3736942314":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/1606716517":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3870112353":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2954409059":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3112535338":"2023-12-02T03:10:55.627314+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:10.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.778+0000 7f8f96599640 1 -- 172.21.15.115:0/3752224278 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8f7006a670 msgr2=0x7f8f7006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.778+0000 7f8f96599640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8f7006a670 0x7f8f7006cb30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8f78036000 tx=0x7f8f78030000 comp rx=0 tx=0).stop 2023-12-01T03:11:10.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.778+0000 7f8f96599640 1 -- 172.21.15.115:0/3752224278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f9006f2f0 msgr2=0x7f8f90153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:10.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.778+0000 7f8f96599640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f9006f2f0 0x7f8f90153e20 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000c430 tx=0x7f8f8000c8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.779+0000 7f8f96599640 1 -- 172.21.15.115:0/3752224278 shutdown_connections 2023-12-01T03:11:10.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.779+0000 7f8f96599640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8f7006a670 0x7f8f7006cb30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.779+0000 7f8f96599640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f900cad70 0x7f8f90153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.779+0000 7f8f96599640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f9015e2f0 0x7f8f900c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.779+0000 7f8f96599640 1 --2- 172.21.15.115:0/3752224278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f9006f2f0 0x7f8f90153e20 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:10.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.779+0000 7f8f96599640 1 -- 172.21.15.115:0/3752224278 >> 172.21.15.115:0/3752224278 conn(0x7f8f900ce880 msgr2=0x7f8f900c1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:10.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.779+0000 7f8f96599640 1 -- 172.21.15.115:0/3752224278 shutdown_connections 2023-12-01T03:11:10.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:10.779+0000 7f8f96599640 1 -- 172.21.15.115:0/3752224278 wait complete. 2023-12-01T03:11:10.801 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T03:11:11.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.013+0000 7f388f327640 1 Processor -- start 2023-12-01T03:11:11.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.013+0000 7f388f327640 1 -- start start 2023-12-01T03:11:11.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.014+0000 7f388f327640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f38880cfad0 0x7f38880b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.014+0000 7f388f327640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38880d0ab0 con 0x7f38880b59d0 2023-12-01T03:11:11.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.014+0000 7f388f327640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38880d0d90 con 0x7f388815c9c0 2023-12-01T03:11:11.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.014+0000 7f388f327640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38880d1070 con 0x7f38880cfad0 2023-12-01T03:11:11.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388eb26640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f38880b59d0 0x7f38880d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34712/0 (socket says 172.21.15.115:34712) 2023-12-01T03:11:11.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388eb26640 1 -- 172.21.15.115:0/564632280 learned_addr learned my addr 172.21.15.115:0/564632280 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:11.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388db24640 1 --2- 172.21.15.115:0/564632280 >> v2:172.21.15.142:3300/0 conn(0x7f38880cfad0 0x7f38880b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052090034 0 0) 0x7f38880d0ab0 con 0x7f38880b59d0 2023-12-01T03:11:11.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3874003520 con 0x7f38880b59d0 2023-12-01T03:11:11.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2257594514 0 0) 0x7f3874003520 con 0x7f38880b59d0 2023-12-01T03:11:11.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 >> v2:172.21.15.142:3300/0 conn(0x7f38880cfad0 msgr2=0x7f38880b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388d323640 1 --2- 172.21.15.115:0/564632280 >> v2:172.21.15.142:3300/0 conn(0x7f38880cfad0 0x7f38880b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 >> v1:172.21.15.142:6789/0 conn(0x7f388815c9c0 legacy=0x7f38880cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38880d1350 con 0x7f38880b59d0 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f38800019f0 con 0x7f38880b59d0 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 >> v1:172.21.15.115:6789/0 conn(0x7f38880b59d0 legacy=0x7f38880d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.015+0000 7f388d323640 1 --2- 172.21.15.115:0/564632280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38740039e0 0x7f3874003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388d323640 1 --2- 172.21.15.115:0/564632280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38740047c0 0x7f3874004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388d323640 1 --2- 172.21.15.115:0/564632280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f387400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38880d1070 con 0x7f3874005180 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387400c0b0 con 0x7f38740047c0 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387400c270 con 0x7f38740039e0 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388eb26640 1 --2- 172.21.15.115:0/564632280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f387400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388eb26640 1 -- 172.21.15.115:0/564632280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38740039e0 msgr2=0x7f3874003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:11.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388eb26640 1 --2- 172.21.15.115:0/564632280 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38740039e0 0x7f3874003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388eb26640 1 -- 172.21.15.115:0/564632280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38740047c0 msgr2=0x7f3874004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:11.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388eb26640 1 --2- 172.21.15.115:0/564632280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38740047c0 0x7f3874004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.016+0000 7f388eb26640 1 -- 172.21.15.115:0/564632280 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3874003520 con 0x7f3874005180 2023-12-01T03:11:11.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.017+0000 7f388eb26640 1 -- 172.21.15.115:0/564632280 reap_dead start 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.017+0000 7f388eb26640 1 --2- 172.21.15.115:0/564632280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f387400b980 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f388000a800 tx=0x7f38740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a016df01face9e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.018+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800072d0 con 0x7f3874005180 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.018+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388002f070 con 0x7f3874005180 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.018+0000 7f388d323640 1 -- 172.21.15.115:0/564632280 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880007670 con 0x7f3874005180 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.018+0000 7f388f327640 1 -- 172.21.15.115:0/564632280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 msgr2=0x7f387400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.018+0000 7f388f327640 1 --2- 172.21.15.115:0/564632280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f387400b980 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f388000a800 tx=0x7f38740057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.018+0000 7f388f327640 1 -- 172.21.15.115:0/564632280 shutdown_connections 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.018+0000 7f388f327640 1 --2- 172.21.15.115:0/564632280 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f387400b980 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.018+0000 7f388f327640 1 -- 172.21.15.115:0/564632280 >> 172.21.15.115:0/564632280 conn(0x7f3888065960 msgr2=0x7f3888065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.019+0000 7f388f327640 1 -- 172.21.15.115:0/564632280 shutdown_connections 2023-12-01T03:11:11.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.019+0000 7f388f327640 1 -- 172.21.15.115:0/564632280 wait complete. 2023-12-01T03:11:11.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.020+0000 7f388f327640 1 Processor -- start 2023-12-01T03:11:11.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.021+0000 7f388f327640 1 -- start start 2023-12-01T03:11:11.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388f327640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f38880c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388f327640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38880cc1a0 0x7f388814c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388f327640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38881f5950 0x7f38881f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388f327640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38881f6310 con 0x7f3874005180 2023-12-01T03:11:11.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388f327640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3888152bc0 con 0x7f38880cc1a0 2023-12-01T03:11:11.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388f327640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38880cb6c0 con 0x7f38881f5950 2023-12-01T03:11:11.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388e325640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f38880c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388e325640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f38880c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54436/0 (socket says 172.21.15.115:54436) 2023-12-01T03:11:11.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388e325640 1 -- 172.21.15.115:0/1254775236 learned_addr learned my addr 172.21.15.115:0/1254775236 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:11.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388e325640 1 -- 172.21.15.115:0/1254775236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38881f5950 msgr2=0x7f38881f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388e325640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38881f5950 0x7f38881f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388e325640 1 -- 172.21.15.115:0/1254775236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38880cc1a0 msgr2=0x7f388814c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388db24640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38880cc1a0 0x7f388814c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388e325640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38880cc1a0 0x7f388814c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.022+0000 7f388e325640 1 -- 172.21.15.115:0/1254775236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38880cb9a0 con 0x7f3874005180 2023-12-01T03:11:11.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.023+0000 7f388e325640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f38880c6520 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f388400c500 tx=0x7f388400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:11.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.023+0000 7f387effd640 1 -- 172.21.15.115:0/1254775236 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38840052d0 con 0x7f3874005180 2023-12-01T03:11:11.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.023+0000 7f387effd640 1 -- 172.21.15.115:0/1254775236 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3884005430 con 0x7f3874005180 2023-12-01T03:11:11.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.023+0000 7f388f327640 1 -- 172.21.15.115:0/1254775236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38880cbc80 con 0x7f3874005180 2023-12-01T03:11:11.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.023+0000 7f387effd640 1 -- 172.21.15.115:0/1254775236 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884012870 con 0x7f3874005180 2023-12-01T03:11:11.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.023+0000 7f388f327640 1 -- 172.21.15.115:0/1254775236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388814ccf0 con 0x7f3874005180 2023-12-01T03:11:11.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.024+0000 7f388f327640 1 -- 172.21.15.115:0/1254775236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38880ce050 con 0x7f3874005180 2023-12-01T03:11:11.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.025+0000 7f387effd640 1 -- 172.21.15.115:0/1254775236 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f38840129d0 con 0x7f3874005180 2023-12-01T03:11:11.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.026+0000 7f387effd640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f386c06a670 0x7f386c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.026+0000 7f387effd640 1 -- 172.21.15.115:0/1254775236 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7348+0+0 (secure 0 0 0) 0x7f3884088f70 con 0x7f3874005180 2023-12-01T03:11:11.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.026+0000 7f388db24640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f386c06a670 0x7f386c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.027+0000 7f388db24640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f386c06a670 0x7f386c06cb30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f38880b54c0 tx=0x7f3878008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:11.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.030+0000 7f387effd640 1 -- 172.21.15.115:0/1254775236 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f38840bbc80 con 0x7f3874005180 2023-12-01T03:11:11.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.253+0000 7f388f327640 1 -- 172.21.15.115:0/1254775236 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f38880cb520 con 0x7f3874005180 2023-12-01T03:11:11.257 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T03:11:11.255+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T03:11:11.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.274+0000 7f387effd640 1 -- 172.21.15.115:0/1254775236 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) v1 ==== 161+0+0 (secure 0 0 0) 0x7f38880cb520 con 0x7f3874005180 2023-12-01T03:11:11.276 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 -- 172.21.15.115:0/1254775236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f386c06a670 msgr2=0x7f386c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f386c06a670 0x7f386c06cb30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f38880b54c0 tx=0x7f3878008040 comp rx=0 tx=0).stop 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 -- 172.21.15.115:0/1254775236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 msgr2=0x7f38880c6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f38880c6520 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f388400c500 tx=0x7f388400c9c0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 -- 172.21.15.115:0/1254775236 shutdown_connections 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f386c06a670 0x7f386c06cb30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38881f5950 0x7f38881f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38880cc1a0 0x7f388814c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 --2- 172.21.15.115:0/1254775236 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3874005180 0x7f38880c6520 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 -- 172.21.15.115:0/1254775236 >> 172.21.15.115:0/1254775236 conn(0x7f3888065960 msgr2=0x7f38880d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 -- 172.21.15.115:0/1254775236 shutdown_connections 2023-12-01T03:11:11.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.277+0000 7f388f327640 1 -- 172.21.15.115:0/1254775236 wait complete. 2023-12-01T03:11:11.297 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T03:11:11.301 INFO:tasks.ceph.mds.d.smithi142.stderr: -4690> 2023-12-01T03:10:29.608+0000 7f0be1784640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:11:11.393 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T03:11:11.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.501+0000 7f58a8316640 1 Processor -- start 2023-12-01T03:11:11.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.501+0000 7f58a8316640 1 -- start start 2023-12-01T03:11:11.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.503+0000 7f58a8316640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f58a015cd90 0x7f58a00cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.503+0000 7f58a8316640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a00b5fa0 0x7f58a0161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.503+0000 7f58a8316640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a00ce5e0 con 0x7f58a00cf650 2023-12-01T03:11:11.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.503+0000 7f58a8316640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a00d0ec0 con 0x7f58a015cd90 2023-12-01T03:11:11.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.503+0000 7f58a8316640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0161950 con 0x7f58a00b5fa0 2023-12-01T03:11:11.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.503+0000 7f58a68b3640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f58a00cf650 0x7f58a00b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34720/0 (socket says 172.21.15.115:34720) 2023-12-01T03:11:11.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.503+0000 7f58a68b3640 1 -- 172.21.15.115:0/1402869156 learned_addr learned my addr 172.21.15.115:0/1402869156 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:11.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a58b1640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a00b5fa0 0x7f58a0161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a60b2640 1 --2- 172.21.15.115:0/1402869156 >> v2:172.21.15.142:3300/0 conn(0x7f58a015cd90 0x7f58a00cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6954326 0 0) 0x7f5898001490 con 0x7f58a00cf650 2023-12-01T03:11:11.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f588c003520 con 0x7f58a00cf650 2023-12-01T03:11:11.507 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046734706 0 0) 0x7f588c003520 con 0x7f58a00cf650 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a00b5fa0 msgr2=0x7f58a0161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a00b5fa0 0x7f58a0161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 >> v2:172.21.15.142:3300/0 conn(0x7f58a015cd90 msgr2=0x7f58a00cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 --2- 172.21.15.115:0/1402869156 >> v2:172.21.15.142:3300/0 conn(0x7f58a015cd90 0x7f58a00cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00ce300 con 0x7f58a00cf650 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5898001b80 con 0x7f58a00cf650 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 >> v1:172.21.15.115:6789/0 conn(0x7f58a00cf650 legacy=0x7f58a00b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c003970 0x7f588c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a00b5fa0 0x7f588c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c004d00 0x7f588c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00d0ec0 con 0x7f588c004d00 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0161950 con 0x7f588c003970 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c00b9b0 con 0x7f58a00b5fa0 2023-12-01T03:11:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.504+0000 7f58a60b2640 1 --2- 172.21.15.115:0/1402869156 >> v2:172.21.15.142:3300/0 conn(0x7f58a015cd90 0x7f58a00cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:11:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.505+0000 7f58a68b3640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c004d00 0x7f588c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.505+0000 7f58a58b1640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a00b5fa0 0x7f588c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.505+0000 7f58a68b3640 1 -- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a00b5fa0 msgr2=0x7f588c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.505+0000 7f58a68b3640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a00b5fa0 0x7f588c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.505+0000 7f58a60b2640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c003970 0x7f588c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.505+0000 7f58a68b3640 1 -- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c003970 msgr2=0x7f588c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.505+0000 7f58a68b3640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c003970 0x7f588c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.505+0000 7f58a68b3640 1 -- 172.21.15.115:0/1402869156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c003520 con 0x7f588c004d00 2023-12-01T03:11:11.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.505+0000 7f58a60b2640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c003970 0x7f588c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:11.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.506+0000 7f58a68b3640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c004d00 0x7f588c005180 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f589800a830 tx=0x7f588c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad1fe3355ca91cc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.506+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898034630 con 0x7f588c004d00 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.506+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589802f070 con 0x7f588c004d00 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.506+0000 7f58a50b0640 1 -- 172.21.15.115:0/1402869156 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58980349d0 con 0x7f588c004d00 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.507+0000 7f58a8316640 1 -- 172.21.15.115:0/1402869156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c004d00 msgr2=0x7f588c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.507+0000 7f58a8316640 1 --2- 172.21.15.115:0/1402869156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f588c004d00 0x7f588c005180 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f589800a830 tx=0x7f588c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.507+0000 7f58a68b3640 1 -- 172.21.15.115:0/1402869156 reap_dead start 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.507+0000 7f58a8316640 1 -- 172.21.15.115:0/1402869156 shutdown_connections 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.507+0000 7f58a8316640 1 -- 172.21.15.115:0/1402869156 >> 172.21.15.115:0/1402869156 conn(0x7f58a0065960 msgr2=0x7f58a0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.507+0000 7f58a8316640 1 -- 172.21.15.115:0/1402869156 shutdown_connections 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.507+0000 7f58a8316640 1 -- 172.21.15.115:0/1402869156 wait complete. 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.509+0000 7f58a8316640 1 Processor -- start 2023-12-01T03:11:11.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.509+0000 7f58a8316640 1 -- start start 2023-12-01T03:11:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.510+0000 7f58a8316640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c004d00 0x7f58a015aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.510+0000 7f58a8316640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58a015d9f0 0x7f58a015de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.510+0000 7f58a8316640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a0152dc0 0x7f58a0153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.510+0000 7f58a8316640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a015e3b0 con 0x7f58a015d9f0 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.510+0000 7f58a8316640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00c9d90 con 0x7f588c004d00 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.510+0000 7f58a8316640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00c9fe0 con 0x7f58a0152dc0 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.510+0000 7f58a58b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58a015d9f0 0x7f58a015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.510+0000 7f58a58b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58a015d9f0 0x7f58a015de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54446/0 (socket says 172.21.15.115:54446) 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.510+0000 7f58a58b1640 1 -- 172.21.15.115:0/2326202396 learned_addr learned my addr 172.21.15.115:0/2326202396 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f58a58b1640 1 -- 172.21.15.115:0/2326202396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a0152dc0 msgr2=0x7f58a0153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f58a60b2640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c004d00 0x7f58a015aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f58a58b1640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a0152dc0 0x7f58a0153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f58a58b1640 1 -- 172.21.15.115:0/2326202396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c004d00 msgr2=0x7f58a015aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f58a58b1640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c004d00 0x7f58a015aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f58a58b1640 1 -- 172.21.15.115:0/2326202396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00ca2c0 con 0x7f58a015d9f0 2023-12-01T03:11:11.513 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f58a58b1640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58a015d9f0 0x7f58a015de70 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f58900107e0 tx=0x7f5890010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f5896ffd640 1 -- 172.21.15.115:0/2326202396 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890009610 con 0x7f58a015d9f0 2023-12-01T03:11:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f5896ffd640 1 -- 172.21.15.115:0/2326202396 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589000eb60 con 0x7f58a015d9f0 2023-12-01T03:11:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f58a8316640 1 -- 172.21.15.115:0/2326202396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a00ca5a0 con 0x7f58a015d9f0 2023-12-01T03:11:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f5896ffd640 1 -- 172.21.15.115:0/2326202396 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589001c400 con 0x7f58a015d9f0 2023-12-01T03:11:11.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.511+0000 7f58a8316640 1 -- 172.21.15.115:0/2326202396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a00cab60 con 0x7f58a015d9f0 2023-12-01T03:11:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.512+0000 7f58a8316640 1 -- 172.21.15.115:0/2326202396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a004e890 con 0x7f58a015d9f0 2023-12-01T03:11:11.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.514+0000 7f5896ffd640 1 -- 172.21.15.115:0/2326202396 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f589001c560 con 0x7f58a015d9f0 2023-12-01T03:11:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.515+0000 7f5896ffd640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f588406a6a0 0x7f588406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.515+0000 7f5896ffd640 1 -- 172.21.15.115:0/2326202396 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7434+0+0 (secure 0 0 0) 0x7f58900899a0 con 0x7f58a015d9f0 2023-12-01T03:11:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.515+0000 7f58a60b2640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f588406a6a0 0x7f588406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:11.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.516+0000 7f58a60b2640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f588406a6a0 0x7f588406cb60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f589c006fd0 tx=0x7f589c008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:11.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.519+0000 7f5896ffd640 1 -- 172.21.15.115:0/2326202396 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5890054580 con 0x7f58a015d9f0 2023-12-01T03:11:11.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:11.751+0000 7f58a8316640 1 -- 172.21.15.115:0/2326202396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f58a0153780 con 0x7f58a015d9f0 2023-12-01T03:11:12.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.278+0000 7f5896ffd640 1 -- 172.21.15.115:0/2326202396 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v103) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5890057ba0 con 0x7f58a015d9f0 2023-12-01T03:11:12.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.281+0000 7f58a8316640 1 -- 172.21.15.115:0/2326202396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f588406a6a0 msgr2=0x7f588406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.281+0000 7f58a8316640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f588406a6a0 0x7f588406cb60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f589c006fd0 tx=0x7f589c008040 comp rx=0 tx=0).stop 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.281+0000 7f58a8316640 1 -- 172.21.15.115:0/2326202396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58a015d9f0 msgr2=0x7f58a015de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.281+0000 7f58a8316640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58a015d9f0 0x7f58a015de70 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f58900107e0 tx=0x7f5890010ca0 comp rx=0 tx=0).stop 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.282+0000 7f58a8316640 1 -- 172.21.15.115:0/2326202396 shutdown_connections 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.282+0000 7f58a8316640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f588406a6a0 0x7f588406cb60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.282+0000 7f58a8316640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58a0152dc0 0x7f58a0153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.282+0000 7f58a8316640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58a015d9f0 0x7f58a015de70 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.282+0000 7f58a8316640 1 --2- 172.21.15.115:0/2326202396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c004d00 0x7f58a015aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.282+0000 7f58a8316640 1 -- 172.21.15.115:0/2326202396 >> 172.21.15.115:0/2326202396 conn(0x7f58a0065960 msgr2=0x7f58a00d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.282+0000 7f58a8316640 1 -- 172.21.15.115:0/2326202396 shutdown_connections 2023-12-01T03:11:12.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.282+0000 7f58a8316640 1 -- 172.21.15.115:0/2326202396 wait complete. 2023-12-01T03:11:12.305 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T03:11:12.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.522+0000 7f4c20ce7640 1 Processor -- start 2023-12-01T03:11:12.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.522+0000 7f4c20ce7640 1 -- start start 2023-12-01T03:11:12.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.523+0000 7f4c20ce7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4c1c15ac70 0x7f4c1c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:12.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.523+0000 7f4c20ce7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c06c0b0 con 0x7f4c1c0adca0 2023-12-01T03:11:12.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.523+0000 7f4c20ce7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c06c860 con 0x7f4c1c0b4070 2023-12-01T03:11:12.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.523+0000 7f4c20ce7640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c15b050 con 0x7f4c1c15ac70 2023-12-01T03:11:12.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.524+0000 7f4c1ad9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4c1c0adca0 0x7f4c1c06b9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34724/0 (socket says 172.21.15.115:34724) 2023-12-01T03:11:12.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.524+0000 7f4c1ad9d640 1 -- 172.21.15.115:0/968120594 learned_addr learned my addr 172.21.15.115:0/968120594 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:12.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.524+0000 7f4c1a59c640 1 --2- 172.21.15.115:0/968120594 >> v2:172.21.15.115:3300/0 conn(0x7f4c1c15ac70 0x7f4c1c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:12.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.524+0000 7f4c19d9b640 1 -- 172.21.15.115:0/968120594 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517886242 0 0) 0x7f4c1c06c0b0 con 0x7f4c1c0adca0 2023-12-01T03:11:12.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.524+0000 7f4c19d9b640 1 -- 172.21.15.115:0/968120594 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c08003520 con 0x7f4c1c0adca0 2023-12-01T03:11:12.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.524+0000 7f4c1a59c640 1 -- 172.21.15.115:0/968120594 >> v1:172.21.15.142:6789/0 conn(0x7f4c1c0b4070 legacy=0x7f4c1c06b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:12.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.524+0000 7f4c1a59c640 1 -- 172.21.15.115:0/968120594 >> v1:172.21.15.115:6789/0 conn(0x7f4c1c0adca0 legacy=0x7f4c1c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:12.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.524+0000 7f4c1a59c640 1 -- 172.21.15.115:0/968120594 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c0ce0b0 con 0x7f4c1c15ac70 2023-12-01T03:11:12.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.525+0000 7f4c1a59c640 1 --2- 172.21.15.115:0/968120594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15ac70 0x7f4c1c0cb1b0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0097a0 tx=0x7f4c0c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228c65de54b6b2e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:12.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.526+0000 7f4c19d9b640 1 -- 172.21.15.115:0/968120594 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c02e070 con 0x7f4c1c15ac70 2023-12-01T03:11:12.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.526+0000 7f4c19d9b640 1 -- 172.21.15.115:0/968120594 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c0315a0 con 0x7f4c1c15ac70 2023-12-01T03:11:12.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.526+0000 7f4c19d9b640 1 -- 172.21.15.115:0/968120594 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c031910 con 0x7f4c1c15ac70 2023-12-01T03:11:12.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.526+0000 7f4c20ce7640 1 -- 172.21.15.115:0/968120594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15ac70 msgr2=0x7f4c1c0cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:12.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.526+0000 7f4c20ce7640 1 --2- 172.21.15.115:0/968120594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15ac70 0x7f4c1c0cb1b0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0097a0 tx=0x7f4c0c02b840 comp rx=0 tx=0).stop 2023-12-01T03:11:12.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.526+0000 7f4c20ce7640 1 -- 172.21.15.115:0/968120594 shutdown_connections 2023-12-01T03:11:12.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.526+0000 7f4c20ce7640 1 --2- 172.21.15.115:0/968120594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c15ac70 0x7f4c1c0cb1b0 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:12.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.526+0000 7f4c20ce7640 1 -- 172.21.15.115:0/968120594 >> 172.21.15.115:0/968120594 conn(0x7f4c1c0662c0 msgr2=0x7f4c1c0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:12.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.527+0000 7f4c20ce7640 1 -- 172.21.15.115:0/968120594 shutdown_connections 2023-12-01T03:11:12.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.527+0000 7f4c20ce7640 1 -- 172.21.15.115:0/968120594 wait complete. 2023-12-01T03:11:12.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.528+0000 7f4c20ce7640 1 Processor -- start 2023-12-01T03:11:12.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.528+0000 7f4c20ce7640 1 -- start start 2023-12-01T03:11:12.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.529+0000 7f4c20ce7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0adca0 0x7f4c1c0c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:12.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.529+0000 7f4c20ce7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c1c0b4070 0x7f4c1c151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:12.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.529+0000 7f4c20ce7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c1c15ac70 0x7f4c1c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:12.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.529+0000 7f4c20ce7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1f2200 con 0x7f4c1c0adca0 2023-12-01T03:11:12.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.529+0000 7f4c20ce7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c152450 con 0x7f4c1c0b4070 2023-12-01T03:11:12.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.529+0000 7f4c20ce7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c152730 con 0x7f4c1c15ac70 2023-12-01T03:11:12.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.530+0000 7f4c1a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0adca0 0x7f4c1c0c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:12.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.530+0000 7f4c1a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0adca0 0x7f4c1c0c62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54468/0 (socket says 172.21.15.115:54468) 2023-12-01T03:11:12.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.530+0000 7f4c1a59c640 1 -- 172.21.15.115:0/2284449505 learned_addr learned my addr 172.21.15.115:0/2284449505 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:12.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.530+0000 7f4c1a59c640 1 -- 172.21.15.115:0/2284449505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c1c15ac70 msgr2=0x7f4c1c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:12.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.530+0000 7f4c1a59c640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c1c15ac70 0x7f4c1c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:12.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.530+0000 7f4c1a59c640 1 -- 172.21.15.115:0/2284449505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c1c0b4070 msgr2=0x7f4c1c151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:12.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.530+0000 7f4c1a59c640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c1c0b4070 0x7f4c1c151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:12.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.530+0000 7f4c1a59c640 1 -- 172.21.15.115:0/2284449505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c009400 con 0x7f4c1c0adca0 2023-12-01T03:11:12.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.530+0000 7f4c1a59c640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0adca0 0x7f4c1c0c62a0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c000c00 tx=0x7f4c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:12.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.531+0000 7f4c12ffd640 1 -- 172.21.15.115:0/2284449505 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c032630 con 0x7f4c1c0adca0 2023-12-01T03:11:12.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.531+0000 7f4c12ffd640 1 -- 172.21.15.115:0/2284449505 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c032790 con 0x7f4c1c0adca0 2023-12-01T03:11:12.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.531+0000 7f4c12ffd640 1 -- 172.21.15.115:0/2284449505 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c032b40 con 0x7f4c1c0adca0 2023-12-01T03:11:12.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.531+0000 7f4c20ce7640 1 -- 172.21.15.115:0/2284449505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c152a10 con 0x7f4c1c0adca0 2023-12-01T03:11:12.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.531+0000 7f4c20ce7640 1 -- 172.21.15.115:0/2284449505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c152fd0 con 0x7f4c1c0adca0 2023-12-01T03:11:12.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.532+0000 7f4c20ce7640 1 -- 172.21.15.115:0/2284449505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c1531c0 con 0x7f4c1c0adca0 2023-12-01T03:11:12.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.533+0000 7f4c12ffd640 1 -- 172.21.15.115:0/2284449505 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f4c0c032ca0 con 0x7f4c1c0adca0 2023-12-01T03:11:12.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.534+0000 7f4c12ffd640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bfc06a670 0x7f4bfc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:12.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.534+0000 7f4c12ffd640 1 -- 172.21.15.115:0/2284449505 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7434+0+0 (secure 0 0 0) 0x7f4c0c0aafa0 con 0x7f4c1c0adca0 2023-12-01T03:11:12.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.534+0000 7f4c13fff640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bfc06a670 0x7f4bfc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:12.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.535+0000 7f4c13fff640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bfc06a670 0x7f4bfc06cb30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4c04006fd0 tx=0x7f4c04006bc0 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:12.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.539+0000 7f4c12ffd640 1 -- 172.21.15.115:0/2284449505 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4c0c0ddc80 con 0x7f4c1c0adca0 2023-12-01T03:11:12.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:12.731+0000 7f4c20ce7640 1 -- 172.21.15.115:0/2284449505 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4c1c074210 con 0x7f4c1c0adca0 2023-12-01T03:11:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.568+0000 7f4c12ffd640 1 -- 172.21.15.115:0/2284449505 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v222) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4c0c0795a0 con 0x7f4c1c0adca0 2023-12-01T03:11:13.573 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T03:11:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.576+0000 7f4c20ce7640 1 -- 172.21.15.115:0/2284449505 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bfc06a670 msgr2=0x7f4bfc06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.576+0000 7f4c20ce7640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bfc06a670 0x7f4bfc06cb30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4c04006fd0 tx=0x7f4c04006bc0 comp rx=0 tx=0).stop 2023-12-01T03:11:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.576+0000 7f4c20ce7640 1 -- 172.21.15.115:0/2284449505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0adca0 msgr2=0x7f4c1c0c62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.576+0000 7f4c20ce7640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0adca0 0x7f4c1c0c62a0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c000c00 tx=0x7f4c0c0046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:13.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.577+0000 7f4c20ce7640 1 -- 172.21.15.115:0/2284449505 shutdown_connections 2023-12-01T03:11:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.577+0000 7f4c20ce7640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4bfc06a670 0x7f4bfc06cb30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.577+0000 7f4c20ce7640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4c1c15ac70 0x7f4c1c0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.577+0000 7f4c20ce7640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c1c0b4070 0x7f4c1c151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.577+0000 7f4c20ce7640 1 --2- 172.21.15.115:0/2284449505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c1c0adca0 0x7f4c1c0c62a0 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.577+0000 7f4c20ce7640 1 -- 172.21.15.115:0/2284449505 >> 172.21.15.115:0/2284449505 conn(0x7f4c1c0662c0 msgr2=0x7f4c1c0aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.578+0000 7f4c20ce7640 1 -- 172.21.15.115:0/2284449505 shutdown_connections 2023-12-01T03:11:13.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.578+0000 7f4c20ce7640 1 -- 172.21.15.115:0/2284449505 wait complete. 2023-12-01T03:11:13.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T03:11:13.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.804+0000 7f4650acf640 1 Processor -- start 2023-12-01T03:11:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.805+0000 7f4650acf640 1 -- start start 2023-12-01T03:11:13.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.806+0000 7f4650acf640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f464c0b4d50 0x7f464c0b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:13.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.806+0000 7f4650acf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464c0d1690 con 0x7f464c0d2730 2023-12-01T03:11:13.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.806+0000 7f4650acf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464c0d1970 con 0x7f464c15ce40 2023-12-01T03:11:13.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.806+0000 7f4650acf640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464c15d220 con 0x7f464c0b4d50 2023-12-01T03:11:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.806+0000 7f464ad9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f464c0d2730 0x7f464c0b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34734/0 (socket says 172.21.15.115:34734) 2023-12-01T03:11:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.806+0000 7f464ad9d640 1 -- 172.21.15.115:0/744454381 learned_addr learned my addr 172.21.15.115:0/744454381 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.806+0000 7f4649d9b640 1 --2- 172.21.15.115:0/744454381 >> v2:172.21.15.115:3300/0 conn(0x7f464c0b4d50 0x7f464c0b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.807+0000 7f464959a640 1 -- 172.21.15.115:0/744454381 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201098774 0 0) 0x7f464c0d1690 con 0x7f464c0d2730 2023-12-01T03:11:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.807+0000 7f464959a640 1 -- 172.21.15.115:0/744454381 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4638003520 con 0x7f464c0d2730 2023-12-01T03:11:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.807+0000 7f4649d9b640 1 -- 172.21.15.115:0/744454381 >> v1:172.21.15.142:6789/0 conn(0x7f464c15ce40 legacy=0x7f464c0b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.807+0000 7f4649d9b640 1 -- 172.21.15.115:0/744454381 >> v1:172.21.15.115:6789/0 conn(0x7f464c0d2730 legacy=0x7f464c0b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.807+0000 7f4649d9b640 1 -- 172.21.15.115:0/744454381 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c0d1c50 con 0x7f464c0b4d50 2023-12-01T03:11:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.808+0000 7f4649d9b640 1 --2- 172.21.15.115:0/744454381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c0b4d50 0x7f464c0b8710 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f46340097a0 tx=0x7f463402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c34f2111f578bc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.808+0000 7f464959a640 1 -- 172.21.15.115:0/744454381 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634031830 con 0x7f464c0b4d50 2023-12-01T03:11:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.809+0000 7f464959a640 1 -- 172.21.15.115:0/744454381 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4634031990 con 0x7f464c0b4d50 2023-12-01T03:11:13.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.809+0000 7f464959a640 1 -- 172.21.15.115:0/744454381 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634031d00 con 0x7f464c0b4d50 2023-12-01T03:11:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.809+0000 7f4650acf640 1 -- 172.21.15.115:0/744454381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c0b4d50 msgr2=0x7f464c0b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.809+0000 7f4650acf640 1 --2- 172.21.15.115:0/744454381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c0b4d50 0x7f464c0b8710 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f46340097a0 tx=0x7f463402b8d0 comp rx=0 tx=0).stop 2023-12-01T03:11:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.809+0000 7f4650acf640 1 -- 172.21.15.115:0/744454381 shutdown_connections 2023-12-01T03:11:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.809+0000 7f4650acf640 1 --2- 172.21.15.115:0/744454381 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c0b4d50 0x7f464c0b8710 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.809+0000 7f4650acf640 1 -- 172.21.15.115:0/744454381 >> 172.21.15.115:0/744454381 conn(0x7f464c065f90 msgr2=0x7f464c0cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.809+0000 7f4650acf640 1 -- 172.21.15.115:0/744454381 shutdown_connections 2023-12-01T03:11:13.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.809+0000 7f4650acf640 1 -- 172.21.15.115:0/744454381 wait complete. 2023-12-01T03:11:13.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.811+0000 7f4650acf640 1 Processor -- start 2023-12-01T03:11:13.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.811+0000 7f4650acf640 1 -- start start 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.812+0000 7f4650acf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f464c0b4d50 0x7f464c14a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.812+0000 7f4650acf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c0d2730 0x7f464c158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.812+0000 7f4650acf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c15ce40 0x7f464c1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.812+0000 7f4650acf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c14a6a0 con 0x7f464c15ce40 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.812+0000 7f4650acf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c1f5e50 con 0x7f464c0d2730 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.812+0000 7f4650acf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464c1f60f0 con 0x7f464c0b4d50 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.812+0000 7f464ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c15ce40 0x7f464c1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.813+0000 7f464ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c15ce40 0x7f464c1586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54496/0 (socket says 172.21.15.115:54496) 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.813+0000 7f464ad9d640 1 -- 172.21.15.115:0/2187818530 learned_addr learned my addr 172.21.15.115:0/2187818530 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.813+0000 7f464ad9d640 1 -- 172.21.15.115:0/2187818530 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f464c0b4d50 msgr2=0x7f464c14a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.813+0000 7f4649d9b640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c0d2730 0x7f464c158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.813+0000 7f464ad9d640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f464c0b4d50 0x7f464c14a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.813+0000 7f464ad9d640 1 -- 172.21.15.115:0/2187818530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c0d2730 msgr2=0x7f464c158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:13.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.813+0000 7f464ad9d640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c0d2730 0x7f464c158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:13.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.813+0000 7f464ad9d640 1 -- 172.21.15.115:0/2187818530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634009400 con 0x7f464c15ce40 2023-12-01T03:11:13.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.813+0000 7f464ad9d640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c15ce40 0x7f464c1586a0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f464400c430 tx=0x7f464400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:13.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.813+0000 7f4642ffd640 1 -- 172.21.15.115:0/2187818530 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644005ce0 con 0x7f464c15ce40 2023-12-01T03:11:13.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.814+0000 7f4642ffd640 1 -- 172.21.15.115:0/2187818530 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644005e40 con 0x7f464c15ce40 2023-12-01T03:11:13.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.814+0000 7f4642ffd640 1 -- 172.21.15.115:0/2187818530 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644007310 con 0x7f464c15ce40 2023-12-01T03:11:13.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.814+0000 7f4650acf640 1 -- 172.21.15.115:0/2187818530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c1f63d0 con 0x7f464c15ce40 2023-12-01T03:11:13.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.814+0000 7f4650acf640 1 -- 172.21.15.115:0/2187818530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c1f6990 con 0x7f464c15ce40 2023-12-01T03:11:13.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.815+0000 7f4650acf640 1 -- 172.21.15.115:0/2187818530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c0d0ea0 con 0x7f464c15ce40 2023-12-01T03:11:13.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.816+0000 7f4642ffd640 1 -- 172.21.15.115:0/2187818530 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f4644007470 con 0x7f464c15ce40 2023-12-01T03:11:13.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.817+0000 7f4642ffd640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f462c06a670 0x7f462c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:13.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.817+0000 7f4642ffd640 1 -- 172.21.15.115:0/2187818530 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6996+0+0 (secure 0 0 0) 0x7f4644088970 con 0x7f464c15ce40 2023-12-01T03:11:13.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.817+0000 7f464a59c640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f462c06a670 0x7f462c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:13.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.818+0000 7f464a59c640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f462c06a670 0x7f462c06cb30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f463c0012e0 tx=0x7f463c007d30 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:13.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:13.821+0000 7f4642ffd640 1 -- 172.21.15.115:0/2187818530 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4644053650 con 0x7f464c15ce40 2023-12-01T03:11:14.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.019+0000 7f4650acf640 1 -- 172.21.15.115:0/2187818530 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f464c074210 con 0x7f464c15ce40 2023-12-01T03:11:14.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.585+0000 7f4642ffd640 1 -- 172.21.15.115:0/2187818530 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v223) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4644056c70 con 0x7f464c15ce40 2023-12-01T03:11:14.587 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T03:11:14.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 -- 172.21.15.115:0/2187818530 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f462c06a670 msgr2=0x7f462c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:14.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f462c06a670 0x7f462c06cb30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f463c0012e0 tx=0x7f463c007d30 comp rx=0 tx=0).stop 2023-12-01T03:11:14.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 -- 172.21.15.115:0/2187818530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c15ce40 msgr2=0x7f464c1586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:14.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c15ce40 0x7f464c1586a0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f464400c430 tx=0x7f464400c8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 -- 172.21.15.115:0/2187818530 shutdown_connections 2023-12-01T03:11:14.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f462c06a670 0x7f462c06cb30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c15ce40 0x7f464c1586a0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f464c0d2730 0x7f464c158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 --2- 172.21.15.115:0/2187818530 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f464c0b4d50 0x7f464c14a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 -- 172.21.15.115:0/2187818530 >> 172.21.15.115:0/2187818530 conn(0x7f464c065f90 msgr2=0x7f464c15b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:14.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 -- 172.21.15.115:0/2187818530 shutdown_connections 2023-12-01T03:11:14.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.589+0000 7f4650acf640 1 -- 172.21.15.115:0/2187818530 wait complete. 2023-12-01T03:11:14.614 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:14.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.817+0000 7fb7a2b9f640 1 Processor -- start 2023-12-01T03:11:14.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.817+0000 7fb7a2b9f640 1 -- start start 2023-12-01T03:11:14.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.819+0000 7fb7a2b9f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb79c15d220 0x7fb79c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:14.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.819+0000 7fb7a2b9f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79c15d600 con 0x7fb79c070ac0 2023-12-01T03:11:14.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.819+0000 7fb7a2b9f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79c06c170 con 0x7fb79c0ad2b0 2023-12-01T03:11:14.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.819+0000 7fb7a2b9f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79c06cc10 con 0x7fb79c15d220 2023-12-01T03:11:14.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.819+0000 7fb7a113c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb79c070ac0 0x7fb79c06ba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34746/0 (socket says 172.21.15.115:34746) 2023-12-01T03:11:14.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.819+0000 7fb7a113c640 1 -- 172.21.15.115:0/3932746323 learned_addr learned my addr 172.21.15.115:0/3932746323 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:14.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616520734 0 0) 0x7fb794001490 con 0x7fb79c070ac0 2023-12-01T03:11:14.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb784003520 con 0x7fb79c070ac0 2023-12-01T03:11:14.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7a093b640 1 --2- 172.21.15.115:0/3932746323 >> v2:172.21.15.142:3300/0 conn(0x7fb79c15d220 0x7fb79c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1953548383 0 0) 0x7fb784003520 con 0x7fb79c070ac0 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 >> v2:172.21.15.142:3300/0 conn(0x7fb79c15d220 msgr2=0x7fb79c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 --2- 172.21.15.115:0/3932746323 >> v2:172.21.15.142:3300/0 conn(0x7fb79c15d220 0x7fb79c0cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 >> v1:172.21.15.142:6789/0 conn(0x7fb79c0ad2b0 legacy=0x7fb79c06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c06c4b0 con 0x7fb79c070ac0 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb794001b80 con 0x7fb79c070ac0 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 >> v1:172.21.15.115:6789/0 conn(0x7fb79c070ac0 legacy=0x7fb79c06ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb784003a30 0x7fb784003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7840047c0 0x7fb784004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb784005380 0x7fb78400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c06cc10 con 0x7fb7840047c0 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78400c220 con 0x7fb784005380 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.820+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78400c3e0 con 0x7fb784003a30 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.821+0000 7fb793fff640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7840047c0 0x7fb784004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:14.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.821+0000 7fb7a093b640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb784003a30 0x7fb784003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:14.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.821+0000 7fb793fff640 1 -- 172.21.15.115:0/3932746323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb784003a30 msgr2=0x7fb784003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:14.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.821+0000 7fb793fff640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb784003a30 0x7fb784003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.821+0000 7fb7a113c640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb784005380 0x7fb78400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:14.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.821+0000 7fb793fff640 1 -- 172.21.15.115:0/3932746323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb784005380 msgr2=0x7fb78400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:14.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.821+0000 7fb793fff640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb784005380 0x7fb78400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.821+0000 7fb7a113c640 1 -- 172.21.15.115:0/3932746323 reap_dead start 2023-12-01T03:11:14.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.821+0000 7fb793fff640 1 -- 172.21.15.115:0/3932746323 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb784003520 con 0x7fb7840047c0 2023-12-01T03:11:14.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.822+0000 7fb793fff640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7840047c0 0x7fb784004c40 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7fb78c001750 tx=0x7fb7840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2923723907b5f0d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:14.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.822+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0052d0 con 0x7fb7840047c0 2023-12-01T03:11:14.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.822+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c02e050 con 0x7fb7840047c0 2023-12-01T03:11:14.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.822+0000 7fb7937fe640 1 -- 172.21.15.115:0/3932746323 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c003830 con 0x7fb7840047c0 2023-12-01T03:11:14.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.823+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/3932746323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7840047c0 msgr2=0x7fb784004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:14.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.823+0000 7fb7a2b9f640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7840047c0 0x7fb784004c40 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7fb78c001750 tx=0x7fb7840057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.823+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/3932746323 shutdown_connections 2023-12-01T03:11:14.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.823+0000 7fb7a2b9f640 1 --2- 172.21.15.115:0/3932746323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7840047c0 0x7fb784004c40 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.823+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/3932746323 >> 172.21.15.115:0/3932746323 conn(0x7fb79c0ce880 msgr2=0x7fb79c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:14.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.823+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/3932746323 shutdown_connections 2023-12-01T03:11:14.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.823+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/3932746323 wait complete. 2023-12-01T03:11:14.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.825+0000 7fb7a2b9f640 1 Processor -- start 2023-12-01T03:11:14.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.825+0000 7fb7a2b9f640 1 -- start start 2023-12-01T03:11:14.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.826+0000 7fb7a2b9f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840047c0 0x7fb79c14cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:14.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.826+0000 7fb7a2b9f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb79c0c23b0 0x7fb79c0c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:14.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.826+0000 7fb7a2b9f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79c0c9f70 0x7fb79c0ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:14.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.826+0000 7fb7a2b9f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0c2d70 con 0x7fb79c0c9f70 2023-12-01T03:11:14.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.826+0000 7fb7a2b9f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0cb8b0 con 0x7fb7840047c0 2023-12-01T03:11:14.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.826+0000 7fb7a2b9f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0caca0 con 0x7fb79c0c23b0 2023-12-01T03:11:14.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.826+0000 7fb7a113c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79c0c9f70 0x7fb79c0ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:14.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.826+0000 7fb7a113c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79c0c9f70 0x7fb79c0ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54512/0 (socket says 172.21.15.115:54512) 2023-12-01T03:11:14.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.826+0000 7fb7a113c640 1 -- 172.21.15.115:0/933045891 learned_addr learned my addr 172.21.15.115:0/933045891 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:14.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7a093b640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840047c0 0x7fb79c14cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:14.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7a113c640 1 -- 172.21.15.115:0/933045891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb79c0c23b0 msgr2=0x7fb79c0c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7a113c640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb79c0c23b0 0x7fb79c0c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7a113c640 1 -- 172.21.15.115:0/933045891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840047c0 msgr2=0x7fb79c14cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7a113c640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840047c0 0x7fb79c14cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7a113c640 1 -- 172.21.15.115:0/933045891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c0caf80 con 0x7fb79c0c9f70 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7a093b640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840047c0 0x7fb79c14cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7a113c640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79c0c9f70 0x7fb79c0ca420 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fb79c06ed00 tx=0x7fb79400b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7917fa640 1 -- 172.21.15.115:0/933045891 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79400b870 con 0x7fb79c0c9f70 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7917fa640 1 -- 172.21.15.115:0/933045891 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79400b9d0 con 0x7fb79c0c9f70 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7917fa640 1 -- 172.21.15.115:0/933045891 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79400bd80 con 0x7fb79c0c9f70 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/933045891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79c0cb260 con 0x7fb79c0c9f70 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.827+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/933045891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79c14d290 con 0x7fb79c0c9f70 2023-12-01T03:11:14.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.829+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/933045891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79c14d5d0 con 0x7fb79c0c9f70 2023-12-01T03:11:14.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.829+0000 7fb7917fa640 1 -- 172.21.15.115:0/933045891 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fb794007070 con 0x7fb79c0c9f70 2023-12-01T03:11:14.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.830+0000 7fb7917fa640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb78806a6a0 0x7fb78806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:14.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.830+0000 7fb7917fa640 1 -- 172.21.15.115:0/933045891 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb7940893d0 con 0x7fb79c0c9f70 2023-12-01T03:11:14.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.831+0000 7fb7a093b640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb78806a6a0 0x7fb78806cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:14.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.831+0000 7fb7a093b640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb78806a6a0 0x7fb78806cb60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb79c1f5870 tx=0x7fb798008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:14.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:14.835+0000 7fb7917fa640 1 -- 172.21.15.115:0/933045891 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb79c14d5d0 con 0x7fb79c0c9f70 2023-12-01T03:11:15.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.042+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/933045891 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb79c15d600 con 0x7fb79c0c9f70 2023-12-01T03:11:15.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.043+0000 7fb7917fa640 1 -- 172.21.15.115:0/933045891 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v223) v1 ==== 74+0+17374 (secure 0 0 0) 0x7fb79c15d600 con 0x7fb79c0c9f70 2023-12-01T03:11:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:15.046 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":223,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:14.561274+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:10:37.180599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:10:37.961783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6834/720518620":"2023-12-02T03:11:11.255312+0000","172.21.15.115:0/1558237180":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3235712957":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1161447385":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3956813425":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/4128909410":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3433151698":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1087193141":"2023-12-02T03:11:05.783435+0000","172.21.15.115:6824/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:6822/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.142:6832/720518620":"2023-12-02T03:11:11.255312+0000","172.21.15.115:0/3133490080":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2913786291":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/846563507":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3736942314":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/1606716517":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3870112353":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2954409059":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3112535338":"2023-12-02T03:10:55.627314+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:15.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/933045891 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb78806a6a0 msgr2=0x7fb78806cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb78806a6a0 0x7fb78806cb60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb79c1f5870 tx=0x7fb798008040 comp rx=0 tx=0).stop 2023-12-01T03:11:15.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/933045891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79c0c9f70 msgr2=0x7fb79c0ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79c0c9f70 0x7fb79c0ca420 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fb79c06ed00 tx=0x7fb79400b0d0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/933045891 shutdown_connections 2023-12-01T03:11:15.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb78806a6a0 0x7fb78806cb60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb79c0c23b0 0x7fb79c0c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb79c0c9f70 0x7fb79c0ca420 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 --2- 172.21.15.115:0/933045891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7840047c0 0x7fb79c14cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/933045891 >> 172.21.15.115:0/933045891 conn(0x7fb79c0ce880 msgr2=0x7fb79c0ade70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:15.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/933045891 shutdown_connections 2023-12-01T03:11:15.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.046+0000 7fb7a2b9f640 1 -- 172.21.15.115:0/933045891 wait complete. 2023-12-01T03:11:15.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-01T03:10:37.180599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '217', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-01T03:10:37.961783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '218', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:11:15.067 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:11:15.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.260+0000 7f7fb359e640 1 Processor -- start 2023-12-01T03:11:15.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.261+0000 7f7fb359e640 1 -- start start 2023-12-01T03:11:15.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.262+0000 7f7fb359e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7fb40b04c0 0x7f7fb40cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.262+0000 7f7fb359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb40cb630 0x7f7fb406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.262+0000 7f7fb359e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb40cbac0 con 0x7f7fb415ac00 2023-12-01T03:11:15.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.262+0000 7f7fb359e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb40b12b0 con 0x7f7fb40b04c0 2023-12-01T03:11:15.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.262+0000 7f7fb359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb415afe0 con 0x7f7fb40cb630 2023-12-01T03:11:15.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.262+0000 7f7fb259c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7fb415ac00 0x7f7fb40ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34752/0 (socket says 172.21.15.115:34752) 2023-12-01T03:11:15.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.262+0000 7f7fb259c640 1 -- 172.21.15.115:0/3184104864 learned_addr learned my addr 172.21.15.115:0/3184104864 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:15.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fabfff640 1 --2- 172.21.15.115:0/3184104864 >> v2:172.21.15.142:3300/0 conn(0x7f7fb40b04c0 0x7f7fb40cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677515530 0 0) 0x7f7fb40cbac0 con 0x7f7fb415ac00 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb2d9d640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb40cb630 0x7f7fb406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fa0003520 con 0x7f7fb415ac00 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2316808959 0 0) 0x7f7fa0003520 con 0x7f7fb415ac00 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb40cb630 msgr2=0x7f7fb406fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb40cb630 0x7f7fb406fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 >> v2:172.21.15.142:3300/0 conn(0x7f7fb40b04c0 msgr2=0x7f7fb40cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 --2- 172.21.15.115:0/3184104864 >> v2:172.21.15.142:3300/0 conn(0x7f7fb40b04c0 0x7f7fb40cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40b0a10 con 0x7f7fb415ac00 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f7fa4001670 con 0x7f7fb415ac00 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb2d9d640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb40cb630 0x7f7fb406fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 >> v1:172.21.15.115:6789/0 conn(0x7f7fb415ac00 legacy=0x7f7fb40ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.263+0000 7f7fb1d9b640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa0003970 0x7f7fa0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb1d9b640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb40cb630 0x7f7fa00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb1d9b640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa0004d00 0x7f7fa0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40b12b0 con 0x7f7fa0003970 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb415afe0 con 0x7f7fa0004d00 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa000b9d0 con 0x7f7fb40cb630 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb259c640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa0003970 0x7f7fa0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb259c640 1 -- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb40cb630 msgr2=0x7f7fa00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb2d9d640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa0004d00 0x7f7fa0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb259c640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb40cb630 0x7f7fa00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb259c640 1 -- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa0004d00 msgr2=0x7f7fa0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb259c640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa0004d00 0x7f7fa0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb259c640 1 -- 172.21.15.115:0/3184104864 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa0003520 con 0x7f7fa0003970 2023-12-01T03:11:15.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.264+0000 7f7fb2d9d640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa0004d00 0x7f7fa0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:15.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.265+0000 7f7fb259c640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa0003970 0x7f7fa0003d90 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400c240 tx=0x7f7fa00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540e85193a4fe2b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:15.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.266+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa402cb00 con 0x7f7fa0003970 2023-12-01T03:11:15.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.266+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa402cc60 con 0x7f7fa0003970 2023-12-01T03:11:15.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.266+0000 7f7fb1d9b640 1 -- 172.21.15.115:0/3184104864 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4005af0 con 0x7f7fa0003970 2023-12-01T03:11:15.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.266+0000 7f7fb359e640 1 -- 172.21.15.115:0/3184104864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa0003970 msgr2=0x7f7fa0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.266+0000 7f7fb359e640 1 --2- 172.21.15.115:0/3184104864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa0003970 0x7f7fa0003d90 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400c240 tx=0x7f7fa00057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.266+0000 7f7fb2d9d640 1 -- 172.21.15.115:0/3184104864 reap_dead start 2023-12-01T03:11:15.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.267+0000 7f7fb359e640 1 -- 172.21.15.115:0/3184104864 shutdown_connections 2023-12-01T03:11:15.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.267+0000 7f7fb359e640 1 -- 172.21.15.115:0/3184104864 >> 172.21.15.115:0/3184104864 conn(0x7f7fb4065960 msgr2=0x7f7fb4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:15.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.267+0000 7f7fb359e640 1 -- 172.21.15.115:0/3184104864 shutdown_connections 2023-12-01T03:11:15.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.267+0000 7f7fb359e640 1 -- 172.21.15.115:0/3184104864 wait complete. 2023-12-01T03:11:15.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.268+0000 7f7fb359e640 1 Processor -- start 2023-12-01T03:11:15.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.269+0000 7f7fb359e640 1 -- start start 2023-12-01T03:11:15.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa0003970 0x7f7fb4153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb415cba0 0x7f7fb41520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb415d000 0x7f7fb41525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb359e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40c6e60 con 0x7f7fb415d000 2023-12-01T03:11:15.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb4152b30 con 0x7f7fa0003970 2023-12-01T03:11:15.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40c61e0 con 0x7f7fb415cba0 2023-12-01T03:11:15.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb2d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb415d000 0x7f7fb41525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb2d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb415d000 0x7f7fb41525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54526/0 (socket says 172.21.15.115:54526) 2023-12-01T03:11:15.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb2d9d640 1 -- 172.21.15.115:0/331317668 learned_addr learned my addr 172.21.15.115:0/331317668 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:15.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb259c640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa0003970 0x7f7fb4153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fabfff640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb415cba0 0x7f7fb41520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb2d9d640 1 -- 172.21.15.115:0/331317668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb415cba0 msgr2=0x7f7fb41520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb2d9d640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb415cba0 0x7f7fb41520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb2d9d640 1 -- 172.21.15.115:0/331317668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa0003970 msgr2=0x7f7fb4153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb2d9d640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa0003970 0x7f7fb4153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.270+0000 7f7fb2d9d640 1 -- 172.21.15.115:0/331317668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40c64c0 con 0x7f7fb415d000 2023-12-01T03:11:15.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.271+0000 7f7fb2d9d640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb415d000 0x7f7fb41525f0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40c8d30 tx=0x7f7fac00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:15.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.271+0000 7f7faaffd640 1 -- 172.21.15.115:0/331317668 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac00fe00 con 0x7f7fb415d000 2023-12-01T03:11:15.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.271+0000 7f7faaffd640 1 -- 172.21.15.115:0/331317668 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac0143f0 con 0x7f7fb415d000 2023-12-01T03:11:15.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.271+0000 7f7faaffd640 1 -- 172.21.15.115:0/331317668 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0147a0 con 0x7f7fb415d000 2023-12-01T03:11:15.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.271+0000 7f7fb359e640 1 -- 172.21.15.115:0/331317668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb40c67a0 con 0x7f7fb415d000 2023-12-01T03:11:15.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.271+0000 7f7fb359e640 1 -- 172.21.15.115:0/331317668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb4153850 con 0x7f7fb415d000 2023-12-01T03:11:15.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.272+0000 7f7fb359e640 1 -- 172.21.15.115:0/331317668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb40b0fb0 con 0x7f7fb415d000 2023-12-01T03:11:15.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.273+0000 7f7faaffd640 1 -- 172.21.15.115:0/331317668 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f7fac014900 con 0x7f7fb415d000 2023-12-01T03:11:15.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.274+0000 7f7faaffd640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f9406a670 0x7f7f9406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.274+0000 7f7faaffd640 1 -- 172.21.15.115:0/331317668 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f7fac089af0 con 0x7f7fb415d000 2023-12-01T03:11:15.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.274+0000 7f7fb259c640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f9406a670 0x7f7f9406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.275+0000 7f7fb259c640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f9406a670 0x7f7f9406cb30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4034000 tx=0x7f7fa4007000 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:15.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.278+0000 7f7faaffd640 1 -- 172.21.15.115:0/331317668 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7fb40b0fb0 con 0x7f7fb415d000 2023-12-01T03:11:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.467+0000 7f7fb359e640 1 -- 172.21.15.115:0/331317668 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fb40c6fa0 con 0x7f7fb415d000 2023-12-01T03:11:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.468+0000 7f7faaffd640 1 -- 172.21.15.115:0/331317668 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v223) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fb40c6fa0 con 0x7f7fb415d000 2023-12-01T03:11:15.470 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:11:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 -- 172.21.15.115:0/331317668 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f9406a670 msgr2=0x7f7f9406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f9406a670 0x7f7f9406cb30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4034000 tx=0x7f7fa4007000 comp rx=0 tx=0).stop 2023-12-01T03:11:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 -- 172.21.15.115:0/331317668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb415d000 msgr2=0x7f7fb41525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb415d000 0x7f7fb41525f0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40c8d30 tx=0x7f7fac00f520 comp rx=0 tx=0).stop 2023-12-01T03:11:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 -- 172.21.15.115:0/331317668 shutdown_connections 2023-12-01T03:11:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f9406a670 0x7f7f9406cb30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fb415cba0 0x7f7fb41520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fb415d000 0x7f7fb41525f0 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 --2- 172.21.15.115:0/331317668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa0003970 0x7f7fb4153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 -- 172.21.15.115:0/331317668 >> 172.21.15.115:0/331317668 conn(0x7f7fb4065960 msgr2=0x7f7fb40ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:15.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 -- 172.21.15.115:0/331317668 shutdown_connections 2023-12-01T03:11:15.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.471+0000 7f7fb359e640 1 -- 172.21.15.115:0/331317668 wait complete. 2023-12-01T03:11:15.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-01T03:11:15.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.706+0000 7f6810d00640 1 Processor -- start 2023-12-01T03:11:15.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.706+0000 7f6810d00640 1 -- start start 2023-12-01T03:11:15.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.707+0000 7f6810d00640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f680c0ce1b0 0x7f680c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.707+0000 7f6810d00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f680c0b0e60 0x7f680c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.707+0000 7f6810d00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c0cc9a0 con 0x7f680c15ac80 2023-12-01T03:11:15.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.707+0000 7f6810d00640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c15b060 con 0x7f680c0ce1b0 2023-12-01T03:11:15.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.707+0000 7f6810d00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c0b0ce0 con 0x7f680c0b0e60 2023-12-01T03:11:15.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.708+0000 7f6809d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f680c0ce1b0 0x7f680c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.708+0000 7f6809d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f680c0ce1b0 0x7f680c0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54542/0 (socket says 172.21.15.115:54542) 2023-12-01T03:11:15.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.708+0000 7f6809d9b640 1 -- 172.21.15.115:0/347896625 learned_addr learned my addr 172.21.15.115:0/347896625 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:15.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.708+0000 7f6809d9b640 1 -- 172.21.15.115:0/347896625 >> v2:172.21.15.142:3300/0 conn(0x7f680c0b0e60 msgr2=0x7f680c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:15.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.708+0000 7f680ad9d640 1 --2- 172.21.15.115:0/347896625 >> v2:172.21.15.142:3300/0 conn(0x7f680c0b0e60 0x7f680c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.708+0000 7f6809d9b640 1 --2- 172.21.15.115:0/347896625 >> v2:172.21.15.142:3300/0 conn(0x7f680c0b0e60 0x7f680c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.708+0000 7f6809d9b640 1 -- 172.21.15.115:0/347896625 >> v1:172.21.15.142:6789/0 conn(0x7f680c15ac80 legacy=0x7f680c0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:15.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.708+0000 7f6809d9b640 1 -- 172.21.15.115:0/347896625 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c0cbd90 con 0x7f680c0ce1b0 2023-12-01T03:11:15.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.708+0000 7f680ad9d640 1 --2- 172.21.15.115:0/347896625 >> v2:172.21.15.142:3300/0 conn(0x7f680c0b0e60 0x7f680c0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:15.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.709+0000 7f6809d9b640 1 --2- 172.21.15.115:0/347896625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0ce1b0 0x7f680c0cce30 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f680c0cd6e0 tx=0x7f67f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baad52a5a767a2a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:15.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.710+0000 7f680959a640 1 -- 172.21.15.115:0/347896625 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4031870 con 0x7f680c0ce1b0 2023-12-01T03:11:15.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.710+0000 7f680959a640 1 -- 172.21.15.115:0/347896625 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f40319d0 con 0x7f680c0ce1b0 2023-12-01T03:11:15.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.710+0000 7f680959a640 1 -- 172.21.15.115:0/347896625 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4031d60 con 0x7f680c0ce1b0 2023-12-01T03:11:15.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.710+0000 7f6810d00640 1 -- 172.21.15.115:0/347896625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0ce1b0 msgr2=0x7f680c0cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.710+0000 7f6810d00640 1 --2- 172.21.15.115:0/347896625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0ce1b0 0x7f680c0cce30 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f680c0cd6e0 tx=0x7f67f402b910 comp rx=0 tx=0).stop 2023-12-01T03:11:15.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.711+0000 7f6810d00640 1 -- 172.21.15.115:0/347896625 shutdown_connections 2023-12-01T03:11:15.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.711+0000 7f6810d00640 1 --2- 172.21.15.115:0/347896625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0ce1b0 0x7f680c0cce30 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.711+0000 7f6810d00640 1 --2- 172.21.15.115:0/347896625 >> v2:172.21.15.142:3300/0 conn(0x7f680c0b0e60 0x7f680c0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.711+0000 7f6810d00640 1 -- 172.21.15.115:0/347896625 >> 172.21.15.115:0/347896625 conn(0x7f680c065960 msgr2=0x7f680c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:15.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.711+0000 7f6810d00640 1 -- 172.21.15.115:0/347896625 shutdown_connections 2023-12-01T03:11:15.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.711+0000 7f6810d00640 1 -- 172.21.15.115:0/347896625 wait complete. 2023-12-01T03:11:15.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.712+0000 7f6810d00640 1 Processor -- start 2023-12-01T03:11:15.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.713+0000 7f6810d00640 1 -- start start 2023-12-01T03:11:15.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f6810d00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0b0e60 0x7f680c142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f6810d00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f680c0ce1b0 0x7f680c155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f6810d00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f680c15ac80 0x7f680c155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f6810d00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c1513f0 con 0x7f680c0b0e60 2023-12-01T03:11:15.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f6810d00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c150500 con 0x7f680c0ce1b0 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f6810d00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c1507e0 con 0x7f680c15ac80 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f680a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0b0e60 0x7f680c142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f680a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0b0e60 0x7f680c142190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54554/0 (socket says 172.21.15.115:54554) 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f680a59c640 1 -- 172.21.15.115:0/3582183830 learned_addr learned my addr 172.21.15.115:0/3582183830 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f6809d9b640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f680c0ce1b0 0x7f680c155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f680a59c640 1 -- 172.21.15.115:0/3582183830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f680c15ac80 msgr2=0x7f680c155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f680a59c640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f680c15ac80 0x7f680c155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f680a59c640 1 -- 172.21.15.115:0/3582183830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f680c0ce1b0 msgr2=0x7f680c155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f680a59c640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f680c0ce1b0 0x7f680c155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.714+0000 7f680a59c640 1 -- 172.21.15.115:0/3582183830 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f40094d0 con 0x7f680c0b0e60 2023-12-01T03:11:15.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.715+0000 7f680a59c640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0b0e60 0x7f680c142190 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00ba50 tx=0x7f67fc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:15.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.715+0000 7f6802ffd640 1 -- 172.21.15.115:0/3582183830 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc00cbe0 con 0x7f680c0b0e60 2023-12-01T03:11:15.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.715+0000 7f6802ffd640 1 -- 172.21.15.115:0/3582183830 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc00cd80 con 0x7f680c0b0e60 2023-12-01T03:11:15.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.715+0000 7f6810d00640 1 -- 172.21.15.115:0/3582183830 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c150ac0 con 0x7f680c0b0e60 2023-12-01T03:11:15.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.715+0000 7f6802ffd640 1 -- 172.21.15.115:0/3582183830 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc014680 con 0x7f680c0b0e60 2023-12-01T03:11:15.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.715+0000 7f6810d00640 1 -- 172.21.15.115:0/3582183830 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c151080 con 0x7f680c0b0e60 2023-12-01T03:11:15.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.716+0000 7f6810d00640 1 -- 172.21.15.115:0/3582183830 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680c04e890 con 0x7f680c0b0e60 2023-12-01T03:11:15.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.717+0000 7f6802ffd640 1 -- 172.21.15.115:0/3582183830 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f67fc0147e0 con 0x7f680c0b0e60 2023-12-01T03:11:15.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.718+0000 7f6802ffd640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f67ec06a670 0x7f67ec06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:15.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.718+0000 7f6802ffd640 1 -- 172.21.15.115:0/3582183830 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f67fc088fb0 con 0x7f680c0b0e60 2023-12-01T03:11:15.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.718+0000 7f6809d9b640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f67ec06a670 0x7f67ec06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:15.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.719+0000 7f6809d9b640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f67ec06a670 0x7f67ec06cb30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67f4036000 tx=0x7f67f403c040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:15.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.723+0000 7f6802ffd640 1 -- 172.21.15.115:0/3582183830 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f67fc0bbc80 con 0x7f680c0b0e60 2023-12-01T03:11:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.913+0000 7f6810d00640 1 -- 172.21.15.115:0/3582183830 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f680c157db0 con 0x7f680c0b0e60 2023-12-01T03:11:15.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.914+0000 7f6802ffd640 1 -- 172.21.15.115:0/3582183830 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v223) v1 ==== 107+0+11 (secure 0 0 0) 0x7f67fc0190b0 con 0x7f680c0b0e60 2023-12-01T03:11:15.916 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:11:15.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 -- 172.21.15.115:0/3582183830 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f67ec06a670 msgr2=0x7f67ec06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f67ec06a670 0x7f67ec06cb30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67f4036000 tx=0x7f67f403c040 comp rx=0 tx=0).stop 2023-12-01T03:11:15.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 -- 172.21.15.115:0/3582183830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0b0e60 msgr2=0x7f680c142190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:15.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0b0e60 0x7f680c142190 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00ba50 tx=0x7f67fc00bf10 comp rx=0 tx=0).stop 2023-12-01T03:11:15.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 -- 172.21.15.115:0/3582183830 shutdown_connections 2023-12-01T03:11:15.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f67ec06a670 0x7f67ec06cb30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f680c15ac80 0x7f680c155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f680c0ce1b0 0x7f680c155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 --2- 172.21.15.115:0/3582183830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f680c0b0e60 0x7f680c142190 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:15.919 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 -- 172.21.15.115:0/3582183830 >> 172.21.15.115:0/3582183830 conn(0x7f680c065960 msgr2=0x7f680c06fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:15.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 -- 172.21.15.115:0/3582183830 shutdown_connections 2023-12-01T03:11:15.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:15.917+0000 7f6810d00640 1 -- 172.21.15.115:0/3582183830 wait complete. 2023-12-01T03:11:15.938 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-01T03:11:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.151+0000 7fedf507e640 1 Processor -- start 2023-12-01T03:11:16.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.151+0000 7fedf507e640 1 -- start start 2023-12-01T03:11:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.153+0000 7fedf507e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf00b04c0 0x7fedf00cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.153+0000 7fedf507e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf00cbac0 con 0x7fedf015ac00 2023-12-01T03:11:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.153+0000 7fedf507e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf015afe0 con 0x7fedf00cb630 2023-12-01T03:11:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.153+0000 7fedf507e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00b12b0 con 0x7fedf00b04c0 2023-12-01T03:11:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.153+0000 7fedeffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fedf015ac00 0x7fedf00ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34768/0 (socket says 172.21.15.115:34768) 2023-12-01T03:11:16.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.153+0000 7fedeffff640 1 -- 172.21.15.115:0/282381975 learned_addr learned my addr 172.21.15.115:0/282381975 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.153+0000 7fede77fe640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf00b04c0 0x7fedf00cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684542281 0 0) 0x7fedf00cbac0 con 0x7fedf015ac00 2023-12-01T03:11:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feddc003520 con 0x7fedf015ac00 2023-12-01T03:11:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021116682 0 0) 0x7fedf015afe0 con 0x7fedf00cb630 2023-12-01T03:11:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedf00cbac0 con 0x7fedf00cb630 2023-12-01T03:11:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1224886429 0 0) 0x7feddc003520 con 0x7fedf015ac00 2023-12-01T03:11:16.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf00b04c0 msgr2=0x7fedf00cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf00b04c0 0x7fedf00cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 >> v1:172.21.15.142:6789/0 conn(0x7fedf00cb630 legacy=0x7fedf006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf00b0a10 con 0x7fedf015ac00 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fede0001670 con 0x7fedf015ac00 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 >> v1:172.21.15.115:6789/0 conn(0x7fedf015ac00 legacy=0x7fedf00ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 0x7feddc003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf00b04c0 0x7feddc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feddc004d00 0x7feddc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00b12b0 con 0x7feddc003b30 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc00ff20 con 0x7feddc004d00 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.154+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0100b0 con 0x7fedf00b04c0 2023-12-01T03:11:16.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.155+0000 7fedeffff640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 0x7feddc003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.155+0000 7fede77fe640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf00b04c0 0x7feddc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.155+0000 7fedf487d640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feddc004d00 0x7feddc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.155+0000 7fedeffff640 1 -- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf00b04c0 msgr2=0x7feddc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.155+0000 7fedeffff640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf00b04c0 0x7feddc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.155+0000 7fedeffff640 1 -- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feddc004d00 msgr2=0x7feddc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.155+0000 7fedeffff640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feddc004d00 0x7feddc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.155+0000 7fedeffff640 1 -- 172.21.15.115:0/282381975 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc003520 con 0x7feddc003b30 2023-12-01T03:11:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.157+0000 7fedeffff640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 0x7feddc003f30 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fede0007fc0 tx=0x7feddc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba2e9ad9aae63f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.157+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0005ad0 con 0x7feddc003b30 2023-12-01T03:11:16.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.157+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede0005c30 con 0x7feddc003b30 2023-12-01T03:11:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.157+0000 7fedef7fe640 1 -- 172.21.15.115:0/282381975 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede002c790 con 0x7feddc003b30 2023-12-01T03:11:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.157+0000 7fedf507e640 1 -- 172.21.15.115:0/282381975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 msgr2=0x7feddc003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.157+0000 7fedf507e640 1 --2- 172.21.15.115:0/282381975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 0x7feddc003f30 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fede0007fc0 tx=0x7feddc0057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.157+0000 7fedf487d640 1 -- 172.21.15.115:0/282381975 reap_dead start 2023-12-01T03:11:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.157+0000 7fedf507e640 1 -- 172.21.15.115:0/282381975 shutdown_connections 2023-12-01T03:11:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.157+0000 7fedf507e640 1 -- 172.21.15.115:0/282381975 >> 172.21.15.115:0/282381975 conn(0x7fedf0065960 msgr2=0x7fedf0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.158+0000 7fedf507e640 1 -- 172.21.15.115:0/282381975 shutdown_connections 2023-12-01T03:11:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.158+0000 7fedf507e640 1 -- 172.21.15.115:0/282381975 wait complete. 2023-12-01T03:11:16.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.159+0000 7fedf507e640 1 Processor -- start 2023-12-01T03:11:16.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.160+0000 7fedf507e640 1 -- start start 2023-12-01T03:11:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.160+0000 7fedf507e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 0x7fedf0155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.160+0000 7fedf507e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf015b7b0 0x7fedf00c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.160+0000 7fedf507e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf006b350 0x7fedf014d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.160+0000 7fedf507e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf015b590 con 0x7feddc003b30 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.160+0000 7fedf507e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0155800 con 0x7fedf015b7b0 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.160+0000 7fedf507e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0155ae0 con 0x7fedf006b350 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fedeffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 0x7fedf0155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fedeffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 0x7fedf0155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54564/0 (socket says 172.21.15.115:54564) 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fedeffff640 1 -- 172.21.15.115:0/3105990426 learned_addr learned my addr 172.21.15.115:0/3105990426 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fede77fe640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf015b7b0 0x7fedf00c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fedf487d640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf006b350 0x7fedf014d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fedeffff640 1 -- 172.21.15.115:0/3105990426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf006b350 msgr2=0x7fedf014d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fedeffff640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf006b350 0x7fedf014d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fedeffff640 1 -- 172.21.15.115:0/3105990426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf015b7b0 msgr2=0x7fedf00c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fedeffff640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf015b7b0 0x7fedf00c2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fedeffff640 1 -- 172.21.15.115:0/3105990426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0155dc0 con 0x7feddc003b30 2023-12-01T03:11:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.161+0000 7fede77fe640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf015b7b0 0x7fedf00c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.162+0000 7fedeffff640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 0x7fedf0155100 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fede000c1d0 tx=0x7fede0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.162+0000 7feded7fa640 1 -- 172.21.15.115:0/3105990426 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0039070 con 0x7feddc003b30 2023-12-01T03:11:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.162+0000 7feded7fa640 1 -- 172.21.15.115:0/3105990426 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede002cba0 con 0x7feddc003b30 2023-12-01T03:11:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.162+0000 7feded7fa640 1 -- 172.21.15.115:0/3105990426 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede003c040 con 0x7feddc003b30 2023-12-01T03:11:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.162+0000 7fedf507e640 1 -- 172.21.15.115:0/3105990426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf01560a0 con 0x7feddc003b30 2023-12-01T03:11:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.162+0000 7fedf507e640 1 -- 172.21.15.115:0/3105990426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf00c27a0 con 0x7feddc003b30 2023-12-01T03:11:16.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.163+0000 7fedf507e640 1 -- 172.21.15.115:0/3105990426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf00c2930 con 0x7feddc003b30 2023-12-01T03:11:16.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.164+0000 7feded7fa640 1 -- 172.21.15.115:0/3105990426 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fede0032070 con 0x7feddc003b30 2023-12-01T03:11:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.165+0000 7feded7fa640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fedd006a670 0x7fedd006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.165+0000 7feded7fa640 1 -- 172.21.15.115:0/3105990426 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fede00aa3c0 con 0x7feddc003b30 2023-12-01T03:11:16.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.165+0000 7fede77fe640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fedd006a670 0x7fedd006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.166+0000 7fede77fe640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fedd006a670 0x7fedd006cb30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedf015dfe0 tx=0x7fedd8008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:16.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.170+0000 7feded7fa640 1 -- 172.21.15.115:0/3105990426 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fede00753f0 con 0x7feddc003b30 2023-12-01T03:11:16.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.358+0000 7fedf507e640 1 -- 172.21.15.115:0/3105990426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fedf00618d0 con 0x7feddc003b30 2023-12-01T03:11:16.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.359+0000 7feded7fa640 1 -- 172.21.15.115:0/3105990426 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v223) v1 ==== 103+0+11 (secure 0 0 0) 0x7fede0078a10 con 0x7feddc003b30 2023-12-01T03:11:16.361 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:11:16.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 -- 172.21.15.115:0/3105990426 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fedd006a670 msgr2=0x7fedd006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fedd006a670 0x7fedd006cb30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedf015dfe0 tx=0x7fedd8008040 comp rx=0 tx=0).stop 2023-12-01T03:11:16.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 -- 172.21.15.115:0/3105990426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 msgr2=0x7fedf0155100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 0x7fedf0155100 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fede000c1d0 tx=0x7fede0001930 comp rx=0 tx=0).stop 2023-12-01T03:11:16.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 -- 172.21.15.115:0/3105990426 shutdown_connections 2023-12-01T03:11:16.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fedd006a670 0x7fedd006cb30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedf006b350 0x7fedf014d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedf015b7b0 0x7fedf00c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 --2- 172.21.15.115:0/3105990426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feddc003b30 0x7fedf0155100 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 -- 172.21.15.115:0/3105990426 >> 172.21.15.115:0/3105990426 conn(0x7fedf0065960 msgr2=0x7fedf00aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:16.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.362+0000 7fedf507e640 1 -- 172.21.15.115:0/3105990426 shutdown_connections 2023-12-01T03:11:16.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.363+0000 7fedf507e640 1 -- 172.21.15.115:0/3105990426 wait complete. 2023-12-01T03:11:16.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.588+0000 7fc1655e1640 1 Processor -- start 2023-12-01T03:11:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.588+0000 7fc1655e1640 1 -- start start 2023-12-01T03:11:16.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.589+0000 7fc1655e1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1601584b0 0x7fc1600cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.589+0000 7fc1655e1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc16015afe0 0x7fc16006b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.590+0000 7fc1655e1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1600cc180 con 0x7fc16006b7b0 2023-12-01T03:11:16.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.590+0000 7fc1655e1640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16006bc40 con 0x7fc1601584b0 2023-12-01T03:11:16.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.590+0000 7fc1655e1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16015b550 con 0x7fc16015afe0 2023-12-01T03:11:16.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.590+0000 7fc15ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1601584b0 0x7fc1600cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.590+0000 7fc15ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1601584b0 0x7fc1600cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54576/0 (socket says 172.21.15.115:54576) 2023-12-01T03:11:16.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.590+0000 7fc15ffff640 1 -- 172.21.15.115:0/1767069020 learned_addr learned my addr 172.21.15.115:0/1767069020 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:16.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.590+0000 7fc15ffff640 1 -- 172.21.15.115:0/1767069020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc16015afe0 msgr2=0x7fc16006b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.590+0000 7fc15ffff640 1 --2- 172.21.15.115:0/1767069020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc16015afe0 0x7fc16006b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.590+0000 7fc15ffff640 1 -- 172.21.15.115:0/1767069020 >> v1:172.21.15.142:6789/0 conn(0x7fc16006b7b0 legacy=0x7fc16015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.590+0000 7fc15ffff640 1 -- 172.21.15.115:0/1767069020 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16015b830 con 0x7fc1601584b0 2023-12-01T03:11:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.591+0000 7fc15ffff640 1 --2- 172.21.15.115:0/1767069020 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 0x7fc1600cdd90 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fc150009870 tx=0x7fc15002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1527d0817d34e149 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.592+0000 7fc15effd640 1 -- 172.21.15.115:0/1767069020 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150031640 con 0x7fc1601584b0 2023-12-01T03:11:16.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.592+0000 7fc15effd640 1 -- 172.21.15.115:0/1767069020 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1500317a0 con 0x7fc1601584b0 2023-12-01T03:11:16.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.592+0000 7fc15effd640 1 -- 172.21.15.115:0/1767069020 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150031b30 con 0x7fc1601584b0 2023-12-01T03:11:16.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.592+0000 7fc1655e1640 1 -- 172.21.15.115:0/1767069020 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 msgr2=0x7fc1600cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.592+0000 7fc1655e1640 1 --2- 172.21.15.115:0/1767069020 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 0x7fc1600cdd90 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fc150009870 tx=0x7fc15002b8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.593+0000 7fc1655e1640 1 -- 172.21.15.115:0/1767069020 shutdown_connections 2023-12-01T03:11:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.593+0000 7fc1655e1640 1 --2- 172.21.15.115:0/1767069020 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc16015afe0 0x7fc16006b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.593+0000 7fc1655e1640 1 --2- 172.21.15.115:0/1767069020 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 0x7fc1600cdd90 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.593+0000 7fc1655e1640 1 -- 172.21.15.115:0/1767069020 >> 172.21.15.115:0/1767069020 conn(0x7fc1600ce640 msgr2=0x7fc1600cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.593+0000 7fc1655e1640 1 -- 172.21.15.115:0/1767069020 shutdown_connections 2023-12-01T03:11:16.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.593+0000 7fc1655e1640 1 -- 172.21.15.115:0/1767069020 wait complete. 2023-12-01T03:11:16.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.594+0000 7fc1655e1640 1 Processor -- start 2023-12-01T03:11:16.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.595+0000 7fc1655e1640 1 -- start start 2023-12-01T03:11:16.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc1655e1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc16006b7b0 0x7fc1601f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc1655e1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 0x7fc1600ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc1655e1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16015afe0 0x7fc1600acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc1655e1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1600c8c00 con 0x7fc1601584b0 2023-12-01T03:11:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc1655e1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1600c8e50 con 0x7fc16015afe0 2023-12-01T03:11:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc1655e1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1600c9130 con 0x7fc16006b7b0 2023-12-01T03:11:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc15f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 0x7fc1600ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc15f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 0x7fc1600ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54584/0 (socket says 172.21.15.115:54584) 2023-12-01T03:11:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc15f7fe640 1 -- 172.21.15.115:0/4098954354 learned_addr learned my addr 172.21.15.115:0/4098954354 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc15ffff640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc16006b7b0 0x7fc1601f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc15f7fe640 1 -- 172.21.15.115:0/4098954354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc16006b7b0 msgr2=0x7fc1601f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc15f7fe640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc16006b7b0 0x7fc1601f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc164de0640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16015afe0 0x7fc1600acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc15f7fe640 1 -- 172.21.15.115:0/4098954354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16015afe0 msgr2=0x7fc1600acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc15f7fe640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16015afe0 0x7fc1600acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc15f7fe640 1 -- 172.21.15.115:0/4098954354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1500094d0 con 0x7fc1601584b0 2023-12-01T03:11:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.596+0000 7fc15ffff640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc16006b7b0 0x7fc1601f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:16.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.597+0000 7fc15f7fe640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 0x7fc1600ac890 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fc15400b410 tx=0x7fc15400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:16.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.597+0000 7fc15cff9640 1 -- 172.21.15.115:0/4098954354 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154011830 con 0x7fc1601584b0 2023-12-01T03:11:16.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.597+0000 7fc15cff9640 1 -- 172.21.15.115:0/4098954354 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1540119b0 con 0x7fc1601584b0 2023-12-01T03:11:16.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.597+0000 7fc1655e1640 1 -- 172.21.15.115:0/4098954354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1600c9410 con 0x7fc1601584b0 2023-12-01T03:11:16.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.597+0000 7fc15cff9640 1 -- 172.21.15.115:0/4098954354 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154011d60 con 0x7fc1601584b0 2023-12-01T03:11:16.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.597+0000 7fc1655e1640 1 -- 172.21.15.115:0/4098954354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1600c99d0 con 0x7fc1601584b0 2023-12-01T03:11:16.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.598+0000 7fc1655e1640 1 -- 172.21.15.115:0/4098954354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1600673a0 con 0x7fc1601584b0 2023-12-01T03:11:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.599+0000 7fc15cff9640 1 -- 172.21.15.115:0/4098954354 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fc15401a400 con 0x7fc1601584b0 2023-12-01T03:11:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.600+0000 7fc15cff9640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc14c06a670 0x7fc14c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.600+0000 7fc15cff9640 1 -- 172.21.15.115:0/4098954354 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc1540892c0 con 0x7fc1601584b0 2023-12-01T03:11:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.600+0000 7fc15ffff640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc14c06a670 0x7fc14c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:16.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.601+0000 7fc15ffff640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc14c06a670 0x7fc14c06cb30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc150036000 tx=0x7fc15003c040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:16.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.604+0000 7fc15cff9640 1 -- 172.21.15.115:0/4098954354 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc154054140 con 0x7fc1601584b0 2023-12-01T03:11:16.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.825+0000 7fc1655e1640 1 -- 172.21.15.115:0/4098954354 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc16006fa20 con 0x7fc1601584b0 2023-12-01T03:11:16.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.826+0000 7fc15cff9640 1 -- 172.21.15.115:0/4098954354 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4670 (secure 0 0 0) 0x7fc154057760 con 0x7fc1601584b0 2023-12-01T03:11:16.827 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:16.828 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":103,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":6797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1762895106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1762895106},{"type":"v1","addr":"172.21.15.142:6834","nonce":1762895106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":103},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:38.965513+0000","modified":"2023-12-01T03:10:41.983462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7333":{"gid":7333,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-01T03:11:16.828 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 103 2023-12-01T03:11:16.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.828+0000 7fc1655e1640 1 -- 172.21.15.115:0/4098954354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc14c06a670 msgr2=0x7fc14c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.828+0000 7fc1655e1640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc14c06a670 0x7fc14c06cb30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc150036000 tx=0x7fc15003c040 comp rx=0 tx=0).stop 2023-12-01T03:11:16.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.828+0000 7fc1655e1640 1 -- 172.21.15.115:0/4098954354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 msgr2=0x7fc1600ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.828+0000 7fc1655e1640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 0x7fc1600ac890 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fc15400b410 tx=0x7fc15400b8d0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.828+0000 7fc1655e1640 1 -- 172.21.15.115:0/4098954354 shutdown_connections 2023-12-01T03:11:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.828+0000 7fc1655e1640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc14c06a670 0x7fc14c06cb30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.828+0000 7fc1655e1640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16015afe0 0x7fc1600acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.828+0000 7fc1655e1640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1601584b0 0x7fc1600ac890 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.828+0000 7fc1655e1640 1 --2- 172.21.15.115:0/4098954354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc16006b7b0 0x7fc1601f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.828+0000 7fc1655e1640 1 -- 172.21.15.115:0/4098954354 >> 172.21.15.115:0/4098954354 conn(0x7fc1600ce640 msgr2=0x7fc1600c35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.829+0000 7fc1655e1640 1 -- 172.21.15.115:0/4098954354 shutdown_connections 2023-12-01T03:11:16.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:16.829+0000 7fc1655e1640 1 -- 172.21.15.115:0/4098954354 wait complete. 2023-12-01T03:11:16.846 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:17.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.049+0000 7fa73af9f640 1 Processor -- start 2023-12-01T03:11:17.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.050+0000 7fa73af9f640 1 -- start start 2023-12-01T03:11:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.051+0000 7fa73af9f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa73415e2f0 0x7fa7340ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.051+0000 7fa73af9f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7340add10 con 0x7fa7340adf10 2023-12-01T03:11:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.051+0000 7fa73af9f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7340c5760 con 0x7fa73406f2f0 2023-12-01T03:11:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.051+0000 7fa73af9f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7340c9cf0 con 0x7fa73415e2f0 2023-12-01T03:11:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.051+0000 7fa738d3b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa73415e2f0 0x7fa7340ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.051+0000 7fa738d3b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa73415e2f0 0x7fa7340ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54600/0 (socket says 172.21.15.115:54600) 2023-12-01T03:11:17.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.051+0000 7fa738d3b640 1 -- 172.21.15.115:0/3225261209 learned_addr learned my addr 172.21.15.115:0/3225261209 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:17.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.052+0000 7fa738d3b640 1 -- 172.21.15.115:0/3225261209 >> v1:172.21.15.142:6789/0 conn(0x7fa73406f2f0 legacy=0x7fa7340c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:17.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.052+0000 7fa738d3b640 1 -- 172.21.15.115:0/3225261209 >> v1:172.21.15.115:6789/0 conn(0x7fa7340adf10 legacy=0x7fa7340c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.052+0000 7fa738d3b640 1 -- 172.21.15.115:0/3225261209 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73415e6d0 con 0x7fa73415e2f0 2023-12-01T03:11:17.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.053+0000 7fa738d3b640 1 --2- 172.21.15.115:0/3225261209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa73415e2f0 0x7fa7340ca660 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7fa724009e50 tx=0x7fa72402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2044856464115f4d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:17.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.053+0000 7fa7337fe640 1 -- 172.21.15.115:0/3225261209 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72402c9f0 con 0x7fa73415e2f0 2023-12-01T03:11:17.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.053+0000 7fa7337fe640 1 -- 172.21.15.115:0/3225261209 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72402cb50 con 0x7fa73415e2f0 2023-12-01T03:11:17.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.053+0000 7fa7337fe640 1 -- 172.21.15.115:0/3225261209 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7240343f0 con 0x7fa73415e2f0 2023-12-01T03:11:17.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.054+0000 7fa73af9f640 1 -- 172.21.15.115:0/3225261209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa73415e2f0 msgr2=0x7fa7340ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.054+0000 7fa73af9f640 1 --2- 172.21.15.115:0/3225261209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa73415e2f0 0x7fa7340ca660 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7fa724009e50 tx=0x7fa72402be50 comp rx=0 tx=0).stop 2023-12-01T03:11:17.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.054+0000 7fa73af9f640 1 -- 172.21.15.115:0/3225261209 shutdown_connections 2023-12-01T03:11:17.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.054+0000 7fa73af9f640 1 --2- 172.21.15.115:0/3225261209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa73415e2f0 0x7fa7340ca660 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.054+0000 7fa73af9f640 1 -- 172.21.15.115:0/3225261209 >> 172.21.15.115:0/3225261209 conn(0x7fa7340ce880 msgr2=0x7fa7340cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:17.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.054+0000 7fa73af9f640 1 -- 172.21.15.115:0/3225261209 shutdown_connections 2023-12-01T03:11:17.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.054+0000 7fa73af9f640 1 -- 172.21.15.115:0/3225261209 wait complete. 2023-12-01T03:11:17.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.056+0000 7fa73af9f640 1 Processor -- start 2023-12-01T03:11:17.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.056+0000 7fa73af9f640 1 -- start start 2023-12-01T03:11:17.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa73af9f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa73406f2f0 0x7fa734153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa73af9f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340adf10 0x7fa7340ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa73af9f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa73415e2f0 0x7fa7340acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa73af9f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7341f29e0 con 0x7fa7340adf10 2023-12-01T03:11:17.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa73af9f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340c8100 con 0x7fa73406f2f0 2023-12-01T03:11:17.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa73af9f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340c7580 con 0x7fa73415e2f0 2023-12-01T03:11:17.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa733fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340adf10 0x7fa7340ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa733fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340adf10 0x7fa7340ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54614/0 (socket says 172.21.15.115:54614) 2023-12-01T03:11:17.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa733fff640 1 -- 172.21.15.115:0/2076973410 learned_addr learned my addr 172.21.15.115:0/2076973410 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:17.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa738d3b640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa73406f2f0 0x7fa734153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa733fff640 1 -- 172.21.15.115:0/2076973410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa73415e2f0 msgr2=0x7fa7340acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa73953c640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa73415e2f0 0x7fa7340acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa733fff640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa73415e2f0 0x7fa7340acde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa733fff640 1 -- 172.21.15.115:0/2076973410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa73406f2f0 msgr2=0x7fa734153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa733fff640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa73406f2f0 0x7fa734153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa733fff640 1 -- 172.21.15.115:0/2076973410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724009b30 con 0x7fa7340adf10 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa738d3b640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa73406f2f0 0x7fa734153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.057+0000 7fa73953c640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa73415e2f0 0x7fa7340acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.058+0000 7fa733fff640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340adf10 0x7fa7340ac8a0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00b3f0 tx=0x7fa71c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.058+0000 7fa7317fa640 1 -- 172.21.15.115:0/2076973410 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c011830 con 0x7fa7340adf10 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.058+0000 7fa7317fa640 1 -- 172.21.15.115:0/2076973410 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c011990 con 0x7fa7340adf10 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.058+0000 7fa7317fa640 1 -- 172.21.15.115:0/2076973410 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c011d40 con 0x7fa7340adf10 2023-12-01T03:11:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.058+0000 7fa73af9f640 1 -- 172.21.15.115:0/2076973410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7340c7860 con 0x7fa7340adf10 2023-12-01T03:11:17.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.058+0000 7fa73af9f640 1 -- 172.21.15.115:0/2076973410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7340c7e20 con 0x7fa7340adf10 2023-12-01T03:11:17.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.059+0000 7fa73af9f640 1 -- 172.21.15.115:0/2076973410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73406fda0 con 0x7fa7340adf10 2023-12-01T03:11:17.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.060+0000 7fa7317fa640 1 -- 172.21.15.115:0/2076973410 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fa71c01a400 con 0x7fa7340adf10 2023-12-01T03:11:17.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.061+0000 7fa7317fa640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa71406a670 0x7fa71406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.062+0000 7fa7317fa640 1 -- 172.21.15.115:0/2076973410 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa71c088fa0 con 0x7fa7340adf10 2023-12-01T03:11:17.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.062+0000 7fa738d3b640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa71406a670 0x7fa71406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.062+0000 7fa738d3b640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa71406a670 0x7fa71406cb30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa724035000 tx=0x7fa724030000 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:17.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.066+0000 7fa7317fa640 1 -- 172.21.15.115:0/2076973410 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa71c053e20 con 0x7fa7340adf10 2023-12-01T03:11:17.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.243+0000 7fa73af9f640 1 -- 172.21.15.115:0/2076973410 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa73404e890 con 0x7fa7340adf10 2023-12-01T03:11:17.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.246+0000 7fa7317fa640 1 -- 172.21.15.115:0/2076973410 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v223) v1 ==== 74+0+17374 (secure 0 0 0) 0x7fa71c015090 con 0x7fa7340adf10 2023-12-01T03:11:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:17.249 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":223,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:14.561274+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:10:37.180599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:10:37.961783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6834/720518620":"2023-12-02T03:11:11.255312+0000","172.21.15.115:0/1558237180":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3235712957":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1161447385":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3956813425":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/4128909410":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3433151698":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1087193141":"2023-12-02T03:11:05.783435+0000","172.21.15.115:6824/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:6822/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.142:6832/720518620":"2023-12-02T03:11:11.255312+0000","172.21.15.115:0/3133490080":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2913786291":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/846563507":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3736942314":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/1606716517":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3870112353":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2954409059":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3112535338":"2023-12-02T03:10:55.627314+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:17.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 -- 172.21.15.115:0/2076973410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa71406a670 msgr2=0x7fa71406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa71406a670 0x7fa71406cb30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa724035000 tx=0x7fa724030000 comp rx=0 tx=0).stop 2023-12-01T03:11:17.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 -- 172.21.15.115:0/2076973410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340adf10 msgr2=0x7fa7340ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340adf10 0x7fa7340ac8a0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00b3f0 tx=0x7fa71c00b8b0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 -- 172.21.15.115:0/2076973410 shutdown_connections 2023-12-01T03:11:17.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa71406a670 0x7fa71406cb30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa73415e2f0 0x7fa7340acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7340adf10 0x7fa7340ac8a0 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 --2- 172.21.15.115:0/2076973410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa73406f2f0 0x7fa734153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 -- 172.21.15.115:0/2076973410 >> 172.21.15.115:0/2076973410 conn(0x7fa7340ce880 msgr2=0x7fa7340b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:17.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 -- 172.21.15.115:0/2076973410 shutdown_connections 2023-12-01T03:11:17.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.249+0000 7fa73af9f640 1 -- 172.21.15.115:0/2076973410 wait complete. 2023-12-01T03:11:17.268 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-12-01T03:11:17.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.470+0000 7f2c2b59e640 1 Processor -- start 2023-12-01T03:11:17.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.470+0000 7f2c2b59e640 1 -- start start 2023-12-01T03:11:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.471+0000 7f2c2b59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2c2c0cdcc0 0x7f2c2c06b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.471+0000 7f2c2b59e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2c2c0adf00 0x7f2c2c0ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.471+0000 7f2c2b59e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2c0ce120 con 0x7f2c2c1597d0 2023-12-01T03:11:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.471+0000 7f2c2b59e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c06b7b0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.471+0000 7f2c2b59e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c06b9c0 con 0x7f2c2c0adf00 2023-12-01T03:11:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.472+0000 7f2c23fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2c2c0cdcc0 0x7f2c2c06b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.472+0000 7f2c23fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2c2c0cdcc0 0x7f2c2c06b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54620/0 (socket says 172.21.15.115:54620) 2023-12-01T03:11:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.472+0000 7f2c23fff640 1 -- 172.21.15.115:0/1123038419 learned_addr learned my addr 172.21.15.115:0/1123038419 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.472+0000 7f2c2a59c640 1 --1- 172.21.15.115:0/1123038419 >> v1:172.21.15.142:6789/0 conn(0x7f2c2c1597d0 0x7f2c2c15db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:60374/0 (socket says 172.21.15.115:60374) 2023-12-01T03:11:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.472+0000 7f2c2ad9d640 1 --2- 172.21.15.115:0/1123038419 >> v2:172.21.15.142:3300/0 conn(0x7f2c2c0adf00 0x7f2c2c0ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.472+0000 7f2c23fff640 1 -- 172.21.15.115:0/1123038419 >> v2:172.21.15.142:3300/0 conn(0x7f2c2c0adf00 msgr2=0x7f2c2c0ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:17.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.472+0000 7f2c23fff640 1 --2- 172.21.15.115:0/1123038419 >> v2:172.21.15.142:3300/0 conn(0x7f2c2c0adf00 0x7f2c2c0ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.472+0000 7f2c23fff640 1 -- 172.21.15.115:0/1123038419 >> v1:172.21.15.142:6789/0 conn(0x7f2c2c1597d0 legacy=0x7f2c2c15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.472+0000 7f2c23fff640 1 -- 172.21.15.115:0/1123038419 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c06bca0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.472+0000 7f2c2ad9d640 1 --2- 172.21.15.115:0/1123038419 >> v2:172.21.15.142:3300/0 conn(0x7f2c2c0adf00 0x7f2c2c0ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.474+0000 7f2c23fff640 1 --2- 172.21.15.115:0/1123038419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 0x7f2c2c06b270 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0ad8f0 tx=0x7f2c1402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcb63e250eb8d07 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.474+0000 7f2c29d9b640 1 -- 172.21.15.115:0/1123038419 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14031890 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.474+0000 7f2c29d9b640 1 -- 172.21.15.115:0/1123038419 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c140319f0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.474+0000 7f2c29d9b640 1 -- 172.21.15.115:0/1123038419 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14031d80 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.475+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1123038419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 msgr2=0x7f2c2c06b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.475+0000 7f2c2b59e640 1 --2- 172.21.15.115:0/1123038419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 0x7f2c2c06b270 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0ad8f0 tx=0x7f2c1402b930 comp rx=0 tx=0).stop 2023-12-01T03:11:17.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.475+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1123038419 shutdown_connections 2023-12-01T03:11:17.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.475+0000 7f2c2b59e640 1 --2- 172.21.15.115:0/1123038419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 0x7f2c2c06b270 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.475+0000 7f2c2b59e640 1 --2- 172.21.15.115:0/1123038419 >> v2:172.21.15.142:3300/0 conn(0x7f2c2c0adf00 0x7f2c2c0ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.475+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1123038419 >> 172.21.15.115:0/1123038419 conn(0x7f2c2c0ce640 msgr2=0x7f2c2c0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:17.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.475+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1123038419 shutdown_connections 2023-12-01T03:11:17.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.475+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1123038419 wait complete. 2023-12-01T03:11:17.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.477+0000 7f2c2b59e640 1 Processor -- start 2023-12-01T03:11:17.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.477+0000 7f2c2b59e640 1 -- start start 2023-12-01T03:11:17.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c2b59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0adf00 0x7f2c2c0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c2b59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 0x7f2c2c0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c2b59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c2c1597d0 0x7f2c2c1f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c2b59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c15b5d0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c2b59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c1f66b0 con 0x7f2c2c0adf00 2023-12-01T03:11:17.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c2b59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c15aca0 con 0x7f2c2c1597d0 2023-12-01T03:11:17.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c23fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 0x7f2c2c0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c23fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 0x7f2c2c0acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54626/0 (socket says 172.21.15.115:54626) 2023-12-01T03:11:17.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c23fff640 1 -- 172.21.15.115:0/1294989635 learned_addr learned my addr 172.21.15.115:0/1294989635 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:17.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c23fff640 1 -- 172.21.15.115:0/1294989635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c2c1597d0 msgr2=0x7f2c2c1f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c23fff640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c2c1597d0 0x7f2c2c1f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c23fff640 1 -- 172.21.15.115:0/1294989635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0adf00 msgr2=0x7f2c2c0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.478+0000 7f2c23fff640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0adf00 0x7f2c2c0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.479+0000 7f2c23fff640 1 -- 172.21.15.115:0/1294989635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c140094d0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.479+0000 7f2c23fff640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 0x7f2c2c0acdd0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f2c1402b970 tx=0x7f2c140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.479+0000 7f2c22ffd640 1 -- 172.21.15.115:0/1294989635 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14031860 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.479+0000 7f2c22ffd640 1 -- 172.21.15.115:0/1294989635 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c140443f0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.479+0000 7f2c22ffd640 1 -- 172.21.15.115:0/1294989635 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c140447a0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.479+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1294989635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c15af80 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.479+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1294989635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c15ba60 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.480+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1294989635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c15d5f0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.481+0000 7f2c22ffd640 1 -- 172.21.15.115:0/1294989635 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f2c1404d450 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.482+0000 7f2c22ffd640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2c0c06a670 0x7f2c0c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.482+0000 7f2c22ffd640 1 -- 172.21.15.115:0/1294989635 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f2c140ab9f0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.482+0000 7f2c2a59c640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2c0c06a670 0x7f2c0c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.483+0000 7f2c2a59c640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2c0c06a670 0x7f2c0c06cb30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0c44e0 tx=0x7f2c1c0079c0 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.487+0000 7f2c22ffd640 1 -- 172.21.15.115:0/1294989635 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2c2c15d5f0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.699+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1294989635 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c2c15d5f0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.700+0000 7f2c22ffd640 1 -- 172.21.15.115:0/1294989635 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4670 (secure 0 0 0) 0x7f2c2c15d5f0 con 0x7f2c2c0cdcc0 2023-12-01T03:11:17.702 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:17.703 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":103,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":6797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1762895106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1762895106},{"type":"v1","addr":"172.21.15.142:6834","nonce":1762895106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":103},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:38.965513+0000","modified":"2023-12-01T03:10:41.983462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7333":{"gid":7333,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-01T03:11:17.703 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 103 2023-12-01T03:11:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.703+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1294989635 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2c0c06a670 msgr2=0x7f2c0c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.703+0000 7f2c2b59e640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2c0c06a670 0x7f2c0c06cb30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0c44e0 tx=0x7f2c1c0079c0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.703+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1294989635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 msgr2=0x7f2c2c0acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.703+0000 7f2c2b59e640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 0x7f2c2c0acdd0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f2c1402b970 tx=0x7f2c140046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.703+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1294989635 shutdown_connections 2023-12-01T03:11:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.703+0000 7f2c2b59e640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2c0c06a670 0x7f2c0c06cb30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.703+0000 7f2c2b59e640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c2c1597d0 0x7f2c2c1f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.703+0000 7f2c2b59e640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c2c0cdcc0 0x7f2c2c0acdd0 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.703+0000 7f2c2b59e640 1 --2- 172.21.15.115:0/1294989635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c2c0adf00 0x7f2c2c0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.703+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1294989635 >> 172.21.15.115:0/1294989635 conn(0x7f2c2c0ce640 msgr2=0x7f2c2c0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.704+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1294989635 shutdown_connections 2023-12-01T03:11:17.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.704+0000 7f2c2b59e640 1 -- 172.21.15.115:0/1294989635 wait complete. 2023-12-01T03:11:17.722 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:17.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.918+0000 7faab5f99640 1 Processor -- start 2023-12-01T03:11:17.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.918+0000 7faab5f99640 1 -- start start 2023-12-01T03:11:17.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.920+0000 7faab5f99640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faab00cad70 0x7faab006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.920+0000 7faab5f99640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab00add10 con 0x7faab006f2f0 2023-12-01T03:11:17.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.920+0000 7faab5f99640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab00c5760 con 0x7faab015e2f0 2023-12-01T03:11:17.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.920+0000 7faab5f99640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab015e6d0 con 0x7faab00cad70 2023-12-01T03:11:17.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.920+0000 7faaaffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7faab006f2f0 0x7faab006bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46374/0 (socket says 172.21.15.115:46374) 2023-12-01T03:11:17.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.920+0000 7faaaffff640 1 -- 172.21.15.115:0/3929096420 learned_addr learned my addr 172.21.15.115:0/3929096420 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:17.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.920+0000 7faaa7fff640 1 --2- 172.21.15.115:0/3929096420 >> v2:172.21.15.115:3300/0 conn(0x7faab00cad70 0x7faab006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.921+0000 7faaaeffd640 1 -- 172.21.15.115:0/3929096420 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237844005 0 0) 0x7faab00add10 con 0x7faab006f2f0 2023-12-01T03:11:17.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.921+0000 7faaaeffd640 1 -- 172.21.15.115:0/3929096420 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa9c003520 con 0x7faab006f2f0 2023-12-01T03:11:17.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.921+0000 7faaa7fff640 1 -- 172.21.15.115:0/3929096420 >> v1:172.21.15.142:6789/0 conn(0x7faab015e2f0 legacy=0x7faab00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.921+0000 7faaa7fff640 1 -- 172.21.15.115:0/3929096420 >> v1:172.21.15.115:6789/0 conn(0x7faab006f2f0 legacy=0x7faab006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.921+0000 7faaa7fff640 1 -- 172.21.15.115:0/3929096420 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab00cf7b0 con 0x7faab00cad70 2023-12-01T03:11:17.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.922+0000 7faaa7fff640 1 --2- 172.21.15.115:0/3929096420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab00cad70 0x7faab006b5c0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7faa980097d0 tx=0x7faa9802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825185ea6319c4db server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:17.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.922+0000 7faaaeffd640 1 -- 172.21.15.115:0/3929096420 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9802e070 con 0x7faab00cad70 2023-12-01T03:11:17.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.922+0000 7faaaeffd640 1 -- 172.21.15.115:0/3929096420 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa980315a0 con 0x7faab00cad70 2023-12-01T03:11:17.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.922+0000 7faaaeffd640 1 -- 172.21.15.115:0/3929096420 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98031930 con 0x7faab00cad70 2023-12-01T03:11:17.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.923+0000 7faab5f99640 1 -- 172.21.15.115:0/3929096420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab00cad70 msgr2=0x7faab006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.923+0000 7faab5f99640 1 --2- 172.21.15.115:0/3929096420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab00cad70 0x7faab006b5c0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7faa980097d0 tx=0x7faa9802b840 comp rx=0 tx=0).stop 2023-12-01T03:11:17.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.923+0000 7faab5f99640 1 -- 172.21.15.115:0/3929096420 shutdown_connections 2023-12-01T03:11:17.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.923+0000 7faab5f99640 1 --2- 172.21.15.115:0/3929096420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab00cad70 0x7faab006b5c0 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.923+0000 7faab5f99640 1 -- 172.21.15.115:0/3929096420 >> 172.21.15.115:0/3929096420 conn(0x7faab00ce880 msgr2=0x7faab00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:17.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.923+0000 7faab5f99640 1 -- 172.21.15.115:0/3929096420 shutdown_connections 2023-12-01T03:11:17.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.923+0000 7faab5f99640 1 -- 172.21.15.115:0/3929096420 wait complete. 2023-12-01T03:11:17.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.925+0000 7faab5f99640 1 Processor -- start 2023-12-01T03:11:17.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.925+0000 7faab5f99640 1 -- start start 2023-12-01T03:11:17.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faab5f99640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab006f2f0 0x7faab00c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faab5f99640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab00cad70 0x7faab00c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faab5f99640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab015e2f0 0x7faab00c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.928 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faab5f99640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab00c3a70 con 0x7faab015e2f0 2023-12-01T03:11:17.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faab5f99640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab00c3d50 con 0x7faab00cad70 2023-12-01T03:11:17.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faab5f99640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab00c4030 con 0x7faab006f2f0 2023-12-01T03:11:17.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faaaffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab015e2f0 0x7faab00c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faaaffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab015e2f0 0x7faab00c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46876/0 (socket says 172.21.15.115:46876) 2023-12-01T03:11:17.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faaaffff640 1 -- 172.21.15.115:0/2844576206 learned_addr learned my addr 172.21.15.115:0/2844576206 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:17.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faaa7fff640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab00cad70 0x7faab00c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.926+0000 7faaaf7fe640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab006f2f0 0x7faab00c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faaaffff640 1 -- 172.21.15.115:0/2844576206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab006f2f0 msgr2=0x7faab00c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faaaffff640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab006f2f0 0x7faab00c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faaaffff640 1 -- 172.21.15.115:0/2844576206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab00cad70 msgr2=0x7faab00c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:17.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faaaffff640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab00cad70 0x7faab00c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:17.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faaaffff640 1 -- 172.21.15.115:0/2844576206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa98009400 con 0x7faab015e2f0 2023-12-01T03:11:17.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faaaffff640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab015e2f0 0x7faab00c6b90 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7faaa800caa0 tx=0x7faaa800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:17.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faaacff9640 1 -- 172.21.15.115:0/2844576206 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800aba0 con 0x7faab015e2f0 2023-12-01T03:11:17.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faaacff9640 1 -- 172.21.15.115:0/2844576206 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa800ad00 con 0x7faab015e2f0 2023-12-01T03:11:17.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faaacff9640 1 -- 172.21.15.115:0/2844576206 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8012600 con 0x7faab015e2f0 2023-12-01T03:11:17.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faab5f99640 1 -- 172.21.15.115:0/2844576206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab00c4310 con 0x7faab015e2f0 2023-12-01T03:11:17.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.927+0000 7faab5f99640 1 -- 172.21.15.115:0/2844576206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab00c9670 con 0x7faab015e2f0 2023-12-01T03:11:17.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.928+0000 7faab5f99640 1 -- 172.21.15.115:0/2844576206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab006fda0 con 0x7faab015e2f0 2023-12-01T03:11:17.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.929+0000 7faaacff9640 1 -- 172.21.15.115:0/2844576206 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7faaa8012760 con 0x7faab015e2f0 2023-12-01T03:11:17.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.930+0000 7faaacff9640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa9006a6a0 0x7faa9006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:17.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.930+0000 7faaacff9640 1 -- 172.21.15.115:0/2844576206 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6582+0+0 (secure 0 0 0) 0x7faaa8088e90 con 0x7faab015e2f0 2023-12-01T03:11:17.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.930+0000 7faaaf7fe640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa9006a6a0 0x7faa9006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.931+0000 7faaaf7fe640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa9006a6a0 0x7faa9006cb60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faab0071040 tx=0x7faaa0009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:17.935+0000 7faaacff9640 1 -- 172.21.15.115:0/2844576206 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faaa8053e50 con 0x7faab015e2f0 2023-12-01T03:11:18.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.161+0000 7faab5f99640 1 -- 172.21.15.115:0/2844576206 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faab00c4830 con 0x7faab015e2f0 2023-12-01T03:11:18.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.162+0000 7faaacff9640 1 -- 172.21.15.115:0/2844576206 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4670 (secure 0 0 0) 0x7faab00c4830 con 0x7faab015e2f0 2023-12-01T03:11:18.164 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:18.164 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":103,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":6797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1762895106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1762895106},{"type":"v1","addr":"172.21.15.142:6834","nonce":1762895106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":103},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:10:38.965513+0000","modified":"2023-12-01T03:10:41.983462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7333":{"gid":7333,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":6,"addr":"172.21.15.115:6837/1399179434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":1399179434},{"type":"v1","addr":"172.21.15.115:6837","nonce":1399179434}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-01T03:11:18.165 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 103 2023-12-01T03:11:18.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 -- 172.21.15.115:0/2844576206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa9006a6a0 msgr2=0x7faa9006cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa9006a6a0 0x7faa9006cb60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faab0071040 tx=0x7faaa0009040 comp rx=0 tx=0).stop 2023-12-01T03:11:18.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 -- 172.21.15.115:0/2844576206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab015e2f0 msgr2=0x7faab00c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab015e2f0 0x7faab00c6b90 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7faaa800caa0 tx=0x7faaa800cf60 comp rx=0 tx=0).stop 2023-12-01T03:11:18.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 -- 172.21.15.115:0/2844576206 shutdown_connections 2023-12-01T03:11:18.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faa9006a6a0 0x7faa9006cb60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faab015e2f0 0x7faab00c6b90 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab00cad70 0x7faab00c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 --2- 172.21.15.115:0/2844576206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab006f2f0 0x7faab00c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 -- 172.21.15.115:0/2844576206 >> 172.21.15.115:0/2844576206 conn(0x7faab00ce880 msgr2=0x7faab00b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:18.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.165+0000 7faab5f99640 1 -- 172.21.15.115:0/2844576206 shutdown_connections 2023-12-01T03:11:18.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.166+0000 7faab5f99640 1 -- 172.21.15.115:0/2844576206 wait complete. 2023-12-01T03:11:18.184 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:18.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.384+0000 7fd0835ec640 1 Processor -- start 2023-12-01T03:11:18.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.384+0000 7fd0835ec640 1 -- start start 2023-12-01T03:11:18.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.386+0000 7fd0835ec640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd07c0b8610 0x7fd07c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.386+0000 7fd0835ec640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd07c0cdf00 0x7fd07c0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.386+0000 7fd0835ec640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07c0d1740 con 0x7fd07c15cf90 2023-12-01T03:11:18.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.386+0000 7fd0835ec640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07c0ce330 con 0x7fd07c0b8610 2023-12-01T03:11:18.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.386+0000 7fd0835ec640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07c15d370 con 0x7fd07c0cdf00 2023-12-01T03:11:18.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.386+0000 7fd080b87640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd07c0b8610 0x7fd07c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.386+0000 7fd080b87640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd07c0b8610 0x7fd07c0cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46880/0 (socket says 172.21.15.115:46880) 2023-12-01T03:11:18.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.386+0000 7fd080b87640 1 -- 172.21.15.115:0/2726615246 learned_addr learned my addr 172.21.15.115:0/2726615246 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:18.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.387+0000 7fd081b89640 1 --2- 172.21.15.115:0/2726615246 >> v2:172.21.15.142:3300/0 conn(0x7fd07c0cdf00 0x7fd07c0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.387+0000 7fd080b87640 1 -- 172.21.15.115:0/2726615246 >> v2:172.21.15.142:3300/0 conn(0x7fd07c0cdf00 msgr2=0x7fd07c0b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.387+0000 7fd080b87640 1 --2- 172.21.15.115:0/2726615246 >> v2:172.21.15.142:3300/0 conn(0x7fd07c0cdf00 0x7fd07c0b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.387+0000 7fd080b87640 1 -- 172.21.15.115:0/2726615246 >> v1:172.21.15.142:6789/0 conn(0x7fd07c15cf90 legacy=0x7fd07c0d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.387+0000 7fd080b87640 1 -- 172.21.15.115:0/2726615246 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c0d19a0 con 0x7fd07c0b8610 2023-12-01T03:11:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.388+0000 7fd080b87640 1 --2- 172.21.15.115:0/2726615246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0b8610 0x7fd07c0cd9c0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fd07c08b560 tx=0x7fd06c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac7917fbd27add8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:18.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.388+0000 7fd073fff640 1 -- 172.21.15.115:0/2726615246 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c031640 con 0x7fd07c0b8610 2023-12-01T03:11:18.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.388+0000 7fd073fff640 1 -- 172.21.15.115:0/2726615246 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c0317a0 con 0x7fd07c0b8610 2023-12-01T03:11:18.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.388+0000 7fd073fff640 1 -- 172.21.15.115:0/2726615246 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c031b30 con 0x7fd07c0b8610 2023-12-01T03:11:18.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.389+0000 7fd0835ec640 1 -- 172.21.15.115:0/2726615246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0b8610 msgr2=0x7fd07c0cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.389+0000 7fd0835ec640 1 --2- 172.21.15.115:0/2726615246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0b8610 0x7fd07c0cd9c0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fd07c08b560 tx=0x7fd06c02b8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.389+0000 7fd0835ec640 1 -- 172.21.15.115:0/2726615246 shutdown_connections 2023-12-01T03:11:18.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.389+0000 7fd0835ec640 1 --2- 172.21.15.115:0/2726615246 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0b8610 0x7fd07c0cd9c0 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.389+0000 7fd0835ec640 1 --2- 172.21.15.115:0/2726615246 >> v2:172.21.15.142:3300/0 conn(0x7fd07c0cdf00 0x7fd07c0b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.389+0000 7fd0835ec640 1 -- 172.21.15.115:0/2726615246 >> 172.21.15.115:0/2726615246 conn(0x7fd07c065960 msgr2=0x7fd07c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:18.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.389+0000 7fd0835ec640 1 -- 172.21.15.115:0/2726615246 shutdown_connections 2023-12-01T03:11:18.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.389+0000 7fd0835ec640 1 -- 172.21.15.115:0/2726615246 wait complete. 2023-12-01T03:11:18.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.391+0000 7fd0835ec640 1 Processor -- start 2023-12-01T03:11:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.391+0000 7fd0835ec640 1 -- start start 2023-12-01T03:11:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.392+0000 7fd0835ec640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c0b8610 0x7fd07c151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.392+0000 7fd0835ec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0cdf00 0x7fd07c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.392+0000 7fd0835ec640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c15cf90 0x7fd07c151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.392+0000 7fd0835ec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c15e630 con 0x7fd07c0cdf00 2023-12-01T03:11:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.392+0000 7fd0835ec640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c0b5470 con 0x7fd07c15cf90 2023-12-01T03:11:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.392+0000 7fd0835ec640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c15e8a0 con 0x7fd07c0b8610 2023-12-01T03:11:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.392+0000 7fd080b87640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0cdf00 0x7fd07c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.392+0000 7fd080b87640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0cdf00 0x7fd07c0b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46882/0 (socket says 172.21.15.115:46882) 2023-12-01T03:11:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.392+0000 7fd080b87640 1 -- 172.21.15.115:0/4079502877 learned_addr learned my addr 172.21.15.115:0/4079502877 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd081388640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c0b8610 0x7fd07c151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd081b89640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c15cf90 0x7fd07c151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd080b87640 1 -- 172.21.15.115:0/4079502877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c0b8610 msgr2=0x7fd07c151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd080b87640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c0b8610 0x7fd07c151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd080b87640 1 -- 172.21.15.115:0/4079502877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c15cf90 msgr2=0x7fd07c151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd080b87640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c15cf90 0x7fd07c151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd080b87640 1 -- 172.21.15.115:0/4079502877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0094d0 con 0x7fd07c0cdf00 2023-12-01T03:11:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd081b89640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c15cf90 0x7fd07c151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:18.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd080b87640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0cdf00 0x7fd07c0b4f30 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7fd06c000c00 tx=0x7fd06c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd071ffb640 1 -- 172.21.15.115:0/4079502877 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c03ac60 con 0x7fd07c0cdf00 2023-12-01T03:11:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd071ffb640 1 -- 172.21.15.115:0/4079502877 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c03adc0 con 0x7fd07c0cdf00 2023-12-01T03:11:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd071ffb640 1 -- 172.21.15.115:0/4079502877 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c044630 con 0x7fd07c0cdf00 2023-12-01T03:11:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.393+0000 7fd0835ec640 1 -- 172.21.15.115:0/4079502877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c15eb80 con 0x7fd07c0cdf00 2023-12-01T03:11:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.394+0000 7fd0835ec640 1 -- 172.21.15.115:0/4079502877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07c15f140 con 0x7fd07c0cdf00 2023-12-01T03:11:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.395+0000 7fd0835ec640 1 -- 172.21.15.115:0/4079502877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07c04e890 con 0x7fd07c0cdf00 2023-12-01T03:11:18.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.396+0000 7fd071ffb640 1 -- 172.21.15.115:0/4079502877 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fd06c02f050 con 0x7fd07c0cdf00 2023-12-01T03:11:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.396+0000 7fd071ffb640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd06006a670 0x7fd06006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.397+0000 7fd071ffb640 1 -- 172.21.15.115:0/4079502877 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd06c0ab2d0 con 0x7fd07c0cdf00 2023-12-01T03:11:18.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.397+0000 7fd081388640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd06006a670 0x7fd06006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.397+0000 7fd081388640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd06006a670 0x7fd06006cb30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd078006fd0 tx=0x7fd078008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:18.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.401+0000 7fd071ffb640 1 -- 172.21.15.115:0/4079502877 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd06c0761a0 con 0x7fd07c0cdf00 2023-12-01T03:11:18.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.599+0000 7fd0835ec640 1 -- 172.21.15.115:0/4079502877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd07c0c9320 con 0x7fd07c0cdf00 2023-12-01T03:11:18.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.602+0000 7fd071ffb640 1 -- 172.21.15.115:0/4079502877 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v223) v1 ==== 74+0+17374 (secure 0 0 0) 0x7fd06c0797c0 con 0x7fd07c0cdf00 2023-12-01T03:11:18.605 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:18.605 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":223,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:14.561274+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:10:37.180599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:10:37.961783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6834/720518620":"2023-12-02T03:11:11.255312+0000","172.21.15.115:0/1558237180":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3235712957":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1161447385":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3956813425":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/4128909410":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/3433151698":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/1087193141":"2023-12-02T03:11:05.783435+0000","172.21.15.115:6824/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.115:6822/138932":"2023-12-02T03:10:55.627314+0000","172.21.15.142:6832/720518620":"2023-12-02T03:11:11.255312+0000","172.21.15.115:0/3133490080":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2913786291":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/846563507":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3736942314":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/1606716517":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3870112353":"2023-12-02T03:11:05.783435+0000","172.21.15.115:0/2954409059":"2023-12-02T03:10:55.627314+0000","172.21.15.115:0/3112535338":"2023-12-02T03:10:55.627314+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:18.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 -- 172.21.15.115:0/4079502877 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd06006a670 msgr2=0x7fd06006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd06006a670 0x7fd06006cb30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd078006fd0 tx=0x7fd078008040 comp rx=0 tx=0).stop 2023-12-01T03:11:18.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 -- 172.21.15.115:0/4079502877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0cdf00 msgr2=0x7fd07c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0cdf00 0x7fd07c0b4f30 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7fd06c000c00 tx=0x7fd06c0046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 -- 172.21.15.115:0/4079502877 shutdown_connections 2023-12-01T03:11:18.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd06006a670 0x7fd06006cb30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd07c15cf90 0x7fd07c151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0cdf00 0x7fd07c0b4f30 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 --2- 172.21.15.115:0/4079502877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd07c0b8610 0x7fd07c151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 -- 172.21.15.115:0/4079502877 >> 172.21.15.115:0/4079502877 conn(0x7fd07c065960 msgr2=0x7fd07c0b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:18.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 -- 172.21.15.115:0/4079502877 shutdown_connections 2023-12-01T03:11:18.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.605+0000 7fd0835ec640 1 -- 172.21.15.115:0/4079502877 wait complete. 2023-12-01T03:11:18.625 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-12-01T03:11:18.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.839+0000 7f55c6b73640 1 Processor -- start 2023-12-01T03:11:18.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.839+0000 7f55c6b73640 1 -- start start 2023-12-01T03:11:18.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.841+0000 7f55c6b73640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55c015e2f0 0x7f55c00ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.841+0000 7f55c6b73640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f55c006f2f0 0x7f55c00caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.841+0000 7f55c6b73640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c00add10 con 0x7f55c00adf10 2023-12-01T03:11:18.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.841+0000 7f55c6b73640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c00c5760 con 0x7f55c006f2f0 2023-12-01T03:11:18.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.841+0000 7f55c6b73640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c015e6d0 con 0x7f55c015e2f0 2023-12-01T03:11:18.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.841+0000 7f55c5110640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f55c00adf10 0x7f55c00c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46390/0 (socket says 172.21.15.115:46390) 2023-12-01T03:11:18.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.841+0000 7f55c5110640 1 -- 172.21.15.115:0/382012312 learned_addr learned my addr 172.21.15.115:0/382012312 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:18.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bffff640 1 --2- 172.21.15.115:0/382012312 >> v2:172.21.15.142:3300/0 conn(0x7f55c006f2f0 0x7f55c00caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55c490f640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55c015e2f0 0x7f55c00ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245821738 0 0) 0x7f55c00add10 con 0x7f55c00adf10 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b0003520 con 0x7f55c00adf10 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3495356757 0 0) 0x7f55b0003520 con 0x7f55c00adf10 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55c015e2f0 msgr2=0x7f55c00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55c015e2f0 0x7f55c00ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 >> v2:172.21.15.142:3300/0 conn(0x7f55c006f2f0 msgr2=0x7f55c00caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 --2- 172.21.15.115:0/382012312 >> v2:172.21.15.142:3300/0 conn(0x7f55c006f2f0 0x7f55c00caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c00cf7b0 con 0x7f55c00adf10 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f55b80019f0 con 0x7f55c00adf10 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 >> v1:172.21.15.115:6789/0 conn(0x7f55c00adf10 legacy=0x7f55c00c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b00039e0 0x7f55b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55c015e2f0 0x7f55b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.842+0000 7f55bf7fe640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b0004d00 0x7f55b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00c5760 con 0x7f55b00039e0 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c015e6d0 con 0x7f55b0004d00 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b000ba80 con 0x7f55c015e2f0 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55c490f640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b00039e0 0x7f55b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55c5110640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b0004d00 0x7f55b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55bffff640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55c015e2f0 0x7f55b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55c490f640 1 -- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55c015e2f0 msgr2=0x7f55b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55c490f640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55c015e2f0 0x7f55b00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55c490f640 1 -- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b0004d00 msgr2=0x7f55b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55c490f640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b0004d00 0x7f55b0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55c490f640 1 -- 172.21.15.115:0/382012312 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b0003520 con 0x7f55b00039e0 2023-12-01T03:11:18.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.843+0000 7f55bffff640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55c015e2f0 0x7f55b00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:18.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.845+0000 7f55c490f640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b00039e0 0x7f55b0003e00 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009af0 tx=0x7f55b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db13dbe07d59a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:18.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.845+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b402c6b0 con 0x7f55b00039e0 2023-12-01T03:11:18.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.845+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b402c810 con 0x7f55b00039e0 2023-12-01T03:11:18.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.845+0000 7f55bf7fe640 1 -- 172.21.15.115:0/382012312 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b402cba0 con 0x7f55b00039e0 2023-12-01T03:11:18.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.845+0000 7f55c6b73640 1 -- 172.21.15.115:0/382012312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b00039e0 msgr2=0x7f55b0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.845+0000 7f55c6b73640 1 --2- 172.21.15.115:0/382012312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55b00039e0 0x7f55b0003e00 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009af0 tx=0x7f55b00057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.846+0000 7f55c5110640 1 -- 172.21.15.115:0/382012312 reap_dead start 2023-12-01T03:11:18.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.846+0000 7f55c6b73640 1 -- 172.21.15.115:0/382012312 shutdown_connections 2023-12-01T03:11:18.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.846+0000 7f55c6b73640 1 -- 172.21.15.115:0/382012312 >> 172.21.15.115:0/382012312 conn(0x7f55c00ce880 msgr2=0x7f55c00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:18.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.846+0000 7f55c6b73640 1 -- 172.21.15.115:0/382012312 shutdown_connections 2023-12-01T03:11:18.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.846+0000 7f55c6b73640 1 -- 172.21.15.115:0/382012312 wait complete. 2023-12-01T03:11:18.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.848+0000 7f55c6b73640 1 Processor -- start 2023-12-01T03:11:18.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.848+0000 7f55c6b73640 1 -- start start 2023-12-01T03:11:18.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c6b73640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00039e0 0x7f55c015a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c6b73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c015d3e0 0x7f55c01506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c6b73640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55c0150d60 0x7f55c00c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c6b73640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c015b4d0 con 0x7f55c0150d60 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c6b73640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0150c00 con 0x7f55c015d3e0 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c6b73640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00ca990 con 0x7f55b00039e0 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c5110640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55c0150d60 0x7f55c00c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c5110640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55c0150d60 0x7f55c00c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46906/0 (socket says 172.21.15.115:46906) 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c5110640 1 -- 172.21.15.115:0/2820081865 learned_addr learned my addr 172.21.15.115:0/2820081865 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55bffff640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c015d3e0 0x7f55c01506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c490f640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00039e0 0x7f55c015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c5110640 1 -- 172.21.15.115:0/2820081865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00039e0 msgr2=0x7f55c015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c5110640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00039e0 0x7f55c015a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c5110640 1 -- 172.21.15.115:0/2820081865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c015d3e0 msgr2=0x7f55c01506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:18.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c5110640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c015d3e0 0x7f55c01506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:18.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.849+0000 7f55c5110640 1 -- 172.21.15.115:0/2820081865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c015af10 con 0x7f55c0150d60 2023-12-01T03:11:18.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.850+0000 7f55c5110640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55c0150d60 0x7f55c00c8220 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f55c00b1490 tx=0x7f55b8007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:18.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.850+0000 7f55bd7fa640 1 -- 172.21.15.115:0/2820081865 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800eb40 con 0x7f55c0150d60 2023-12-01T03:11:18.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.850+0000 7f55bd7fa640 1 -- 172.21.15.115:0/2820081865 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b800eca0 con 0x7f55c0150d60 2023-12-01T03:11:18.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.850+0000 7f55bd7fa640 1 -- 172.21.15.115:0/2820081865 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8014480 con 0x7f55c0150d60 2023-12-01T03:11:18.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.850+0000 7f55c6b73640 1 -- 172.21.15.115:0/2820081865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c015b1f0 con 0x7f55c0150d60 2023-12-01T03:11:18.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.850+0000 7f55c6b73640 1 -- 172.21.15.115:0/2820081865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c015cec0 con 0x7f55c0150d60 2023-12-01T03:11:18.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.851+0000 7f55c6b73640 1 -- 172.21.15.115:0/2820081865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c006fda0 con 0x7f55c0150d60 2023-12-01T03:11:18.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.852+0000 7f55bd7fa640 1 -- 172.21.15.115:0/2820081865 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f55b80145e0 con 0x7f55c0150d60 2023-12-01T03:11:18.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.854+0000 7f55bd7fa640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f55a406a6a0 0x7f55a406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:18.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.854+0000 7f55bd7fa640 1 -- 172.21.15.115:0/2820081865 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f55b8088c50 con 0x7f55c0150d60 2023-12-01T03:11:18.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.854+0000 7f55c490f640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f55a406a6a0 0x7f55a406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:18.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.855+0000 7f55c490f640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f55a406a6a0 0x7f55a406cb60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55b4036000 tx=0x7f55b4030000 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:18.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:18.858+0000 7f55bd7fa640 1 -- 172.21.15.115:0/2820081865 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f55b8053c30 con 0x7f55c0150d60 2023-12-01T03:11:19.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.089+0000 7f55c6b73640 1 -- 172.21.15.115:0/2820081865 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7f55c015ca40 con 0x7f55c0150d60 2023-12-01T03:11:19.093 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T03:11:19.091+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T03:11:19.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.111+0000 7f55bd7fa640 1 -- 172.21.15.115:0/2820081865 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) v1 ==== 167+0+0 (secure 0 0 0) 0x7f55c015ca40 con 0x7f55c0150d60 2023-12-01T03:11:19.113 INFO:teuthology.orchestra.run.smithi115.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T03:11:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 -- 172.21.15.115:0/2820081865 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f55a406a6a0 msgr2=0x7f55a406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f55a406a6a0 0x7f55a406cb60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55b4036000 tx=0x7f55b4030000 comp rx=0 tx=0).stop 2023-12-01T03:11:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 -- 172.21.15.115:0/2820081865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55c0150d60 msgr2=0x7f55c00c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55c0150d60 0x7f55c00c8220 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f55c00b1490 tx=0x7f55b8007bc0 comp rx=0 tx=0).stop 2023-12-01T03:11:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 -- 172.21.15.115:0/2820081865 shutdown_connections 2023-12-01T03:11:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f55a406a6a0 0x7f55a406cb60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55c0150d60 0x7f55c00c8220 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55c015d3e0 0x7f55c01506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:19.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 --2- 172.21.15.115:0/2820081865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00039e0 0x7f55c015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 -- 172.21.15.115:0/2820081865 >> 172.21.15.115:0/2820081865 conn(0x7f55c00ce880 msgr2=0x7f55c00b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 -- 172.21.15.115:0/2820081865 shutdown_connections 2023-12-01T03:11:19.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.114+0000 7f55c6b73640 1 -- 172.21.15.115:0/2820081865 wait complete. 2023-12-01T03:11:19.129 INFO:tasks.ceph.mds.c.smithi115.stderr: -1993> 2023-12-01T03:10:44.610+0000 7f735d95a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:11:19.135 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-12-01T03:11:19.191 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T03:11:19.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.320+0000 7f8881815640 1 Processor -- start 2023-12-01T03:11:19.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.320+0000 7f8881815640 1 -- start start 2023-12-01T03:11:19.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f8881815640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f887c15ac00 0x7f887c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:19.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f8881815640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f887c0cb800 0x7f887c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f8881815640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c15afe0 con 0x7f887c0aede0 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f8881815640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c0b12b0 con 0x7f887c15ac00 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f8881815640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0b0a10 con 0x7f887c0cb800 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f8880813640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f887c15ac00 0x7f887c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f887bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f887c0aede0 0x7f887c0cb0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46398/0 (socket says 172.21.15.115:46398) 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f887bfff640 1 -- 172.21.15.115:0/2314692342 learned_addr learned my addr 172.21.15.115:0/2314692342 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f8880813640 1 -- 172.21.15.115:0/2314692342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f887c0cb800 msgr2=0x7f887c06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f8881014640 1 --2- 172.21.15.115:0/2314692342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f887c0cb800 0x7f887c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.322+0000 7f887b7fe640 1 -- 172.21.15.115:0/2314692342 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497496852 0 0) 0x7f887c15afe0 con 0x7f887c0aede0 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.323+0000 7f8880813640 1 --2- 172.21.15.115:0/2314692342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f887c0cb800 0x7f887c06fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.323+0000 7f8880813640 1 -- 172.21.15.115:0/2314692342 >> v1:172.21.15.115:6789/0 conn(0x7f887c0aede0 legacy=0x7f887c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.323+0000 7f8880813640 1 -- 172.21.15.115:0/2314692342 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0b0cf0 con 0x7f887c15ac00 2023-12-01T03:11:19.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.323+0000 7f8881014640 1 --2- 172.21.15.115:0/2314692342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f887c0cb800 0x7f887c06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f8880813640 1 --2- 172.21.15.115:0/2314692342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 0x7f887c0ae8a0 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f88700097a0 tx=0x7f887002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b121ac3d71cafa server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f887b7fe640 1 -- 172.21.15.115:0/2314692342 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8870031780 con 0x7f887c15ac00 2023-12-01T03:11:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f887b7fe640 1 -- 172.21.15.115:0/2314692342 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88700318e0 con 0x7f887c15ac00 2023-12-01T03:11:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f887b7fe640 1 -- 172.21.15.115:0/2314692342 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8870031c70 con 0x7f887c15ac00 2023-12-01T03:11:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f8881815640 1 -- 172.21.15.115:0/2314692342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 msgr2=0x7f887c0ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f8881815640 1 --2- 172.21.15.115:0/2314692342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 0x7f887c0ae8a0 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f88700097a0 tx=0x7f887002b820 comp rx=0 tx=0).stop 2023-12-01T03:11:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f8881815640 1 -- 172.21.15.115:0/2314692342 shutdown_connections 2023-12-01T03:11:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f8881815640 1 --2- 172.21.15.115:0/2314692342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f887c0cb800 0x7f887c06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:19.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f8881815640 1 --2- 172.21.15.115:0/2314692342 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 0x7f887c0ae8a0 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f8881815640 1 -- 172.21.15.115:0/2314692342 >> 172.21.15.115:0/2314692342 conn(0x7f887c065960 msgr2=0x7f887c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.324+0000 7f8881815640 1 -- 172.21.15.115:0/2314692342 shutdown_connections 2023-12-01T03:11:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.325+0000 7f8881815640 1 -- 172.21.15.115:0/2314692342 wait complete. 2023-12-01T03:11:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.326+0000 7f8881815640 1 Processor -- start 2023-12-01T03:11:19.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.326+0000 7f8881815640 1 -- start start 2023-12-01T03:11:19.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.327+0000 7f8881815640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f887c0aede0 0x7f887c144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.327+0000 7f8881815640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f887c0cb800 0x7f887c144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.327+0000 7f8881815640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 0x7f887c145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.327+0000 7f8881815640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c144690 con 0x7f887c15ac00 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.327+0000 7f8881815640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0b25c0 con 0x7f887c0cb800 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.327+0000 7f8881815640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0b19b0 con 0x7f887c0aede0 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.327+0000 7f8881014640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 0x7f887c145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.327+0000 7f8881014640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 0x7f887c145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46946/0 (socket says 172.21.15.115:46946) 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.327+0000 7f8881014640 1 -- 172.21.15.115:0/603038171 learned_addr learned my addr 172.21.15.115:0/603038171 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f8881014640 1 -- 172.21.15.115:0/603038171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f887c0aede0 msgr2=0x7f887c144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f8881014640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f887c0aede0 0x7f887c144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:19.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f8881014640 1 -- 172.21.15.115:0/603038171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f887c0cb800 msgr2=0x7f887c144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f8881014640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f887c0cb800 0x7f887c144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f8881014640 1 -- 172.21.15.115:0/603038171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8870009400 con 0x7f887c15ac00 2023-12-01T03:11:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f8881014640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 0x7f887c145380 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f887400cef0 tx=0x7f887400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f88797fa640 1 -- 172.21.15.115:0/603038171 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400d840 con 0x7f887c15ac00 2023-12-01T03:11:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f88797fa640 1 -- 172.21.15.115:0/603038171 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887400d9c0 con 0x7f887c15ac00 2023-12-01T03:11:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f88797fa640 1 -- 172.21.15.115:0/603038171 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400dd70 con 0x7f887c15ac00 2023-12-01T03:11:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f8881815640 1 -- 172.21.15.115:0/603038171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c0b1c90 con 0x7f887c15ac00 2023-12-01T03:11:19.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.328+0000 7f8881815640 1 -- 172.21.15.115:0/603038171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c0b2250 con 0x7f887c15ac00 2023-12-01T03:11:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.329+0000 7f8881815640 1 -- 172.21.15.115:0/603038171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c0618d0 con 0x7f887c15ac00 2023-12-01T03:11:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.330+0000 7f88797fa640 1 -- 172.21.15.115:0/603038171 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f887401e3b0 con 0x7f887c15ac00 2023-12-01T03:11:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.331+0000 7f88797fa640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f886006a670 0x7f886006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.331+0000 7f88797fa640 1 -- 172.21.15.115:0/603038171 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f8874089be0 con 0x7f887c15ac00 2023-12-01T03:11:19.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.331+0000 7f8880813640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f886006a670 0x7f886006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:19.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.332+0000 7f8880813640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f886006a670 0x7f886006cb30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8870036000 tx=0x7f8870030000 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:19.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.337+0000 7f88797fa640 1 -- 172.21.15.115:0/603038171 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8874054b60 con 0x7f887c15ac00 2023-12-01T03:11:19.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:19.563+0000 7f8881815640 1 -- 172.21.15.115:0/603038171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f887c0b2730 con 0x7f887c15ac00 2023-12-01T03:11:20.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.115+0000 7f88797fa640 1 -- 172.21.15.115:0/603038171 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v105) v1 ==== 107+0+0 (secure 0 0 0) 0x7f887401a020 con 0x7f887c15ac00 2023-12-01T03:11:20.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.118+0000 7f8881815640 1 -- 172.21.15.115:0/603038171 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f886006a670 msgr2=0x7f886006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:20.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.118+0000 7f8881815640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f886006a670 0x7f886006cb30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8870036000 tx=0x7f8870030000 comp rx=0 tx=0).stop 2023-12-01T03:11:20.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.118+0000 7f8881815640 1 -- 172.21.15.115:0/603038171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 msgr2=0x7f887c145380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:20.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.118+0000 7f8881815640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 0x7f887c145380 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f887400cef0 tx=0x7f887400a710 comp rx=0 tx=0).stop 2023-12-01T03:11:20.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.118+0000 7f8881815640 1 -- 172.21.15.115:0/603038171 shutdown_connections 2023-12-01T03:11:20.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.119+0000 7f8881815640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f886006a670 0x7f886006cb30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:20.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.119+0000 7f8881815640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f887c15ac00 0x7f887c145380 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:20.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.119+0000 7f8881815640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f887c0cb800 0x7f887c144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:20.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.119+0000 7f8881815640 1 --2- 172.21.15.115:0/603038171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f887c0aede0 0x7f887c144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:20.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.119+0000 7f8881815640 1 -- 172.21.15.115:0/603038171 >> 172.21.15.115:0/603038171 conn(0x7f887c065960 msgr2=0x7f887c0cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:20.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.119+0000 7f8881815640 1 -- 172.21.15.115:0/603038171 shutdown_connections 2023-12-01T03:11:20.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.119+0000 7f8881815640 1 -- 172.21.15.115:0/603038171 wait complete. 2023-12-01T03:11:20.140 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-12-01T03:11:20.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.352+0000 7fe3074ce640 1 Processor -- start 2023-12-01T03:11:20.354 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.352+0000 7fe3074ce640 1 -- start start 2023-12-01T03:11:20.355 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.354+0000 7fe3074ce640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe30015ac70 0x7fe3000cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:20.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.354+0000 7fe3074ce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30006c0b0 con 0x7fe3000b4070 2023-12-01T03:11:20.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.354+0000 7fe3074ce640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30006c860 con 0x7fe3000adca0 2023-12-01T03:11:20.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.354+0000 7fe3074ce640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30015b050 con 0x7fe30015ac70 2023-12-01T03:11:20.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.354+0000 7fe30526a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe30015ac70 0x7fe3000cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:20.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.354+0000 7fe30526a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe30015ac70 0x7fe3000cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46958/0 (socket says 172.21.15.115:46958) 2023-12-01T03:11:20.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.354+0000 7fe30526a640 1 -- 172.21.15.115:0/1930612748 learned_addr learned my addr 172.21.15.115:0/1930612748 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:20.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.355+0000 7fe30526a640 1 -- 172.21.15.115:0/1930612748 >> v1:172.21.15.142:6789/0 conn(0x7fe3000adca0 legacy=0x7fe30006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:20.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.355+0000 7fe30526a640 1 -- 172.21.15.115:0/1930612748 >> v1:172.21.15.115:6789/0 conn(0x7fe3000b4070 legacy=0x7fe30006b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:20.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.355+0000 7fe30526a640 1 -- 172.21.15.115:0/1930612748 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3000ce0b0 con 0x7fe30015ac70 2023-12-01T03:11:20.357 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.356+0000 7fe30526a640 1 --2- 172.21.15.115:0/1930612748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 0x7fe3000cb1b0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009870 tx=0x7fe2fc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860618c55c724cf6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:20.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.356+0000 7fe2f7fff640 1 -- 172.21.15.115:0/1930612748 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc031860 con 0x7fe30015ac70 2023-12-01T03:11:20.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.356+0000 7fe2f7fff640 1 -- 172.21.15.115:0/1930612748 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc0319c0 con 0x7fe30015ac70 2023-12-01T03:11:20.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.356+0000 7fe2f7fff640 1 -- 172.21.15.115:0/1930612748 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc031d50 con 0x7fe30015ac70 2023-12-01T03:11:20.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.356+0000 7fe3074ce640 1 -- 172.21.15.115:0/1930612748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 msgr2=0x7fe3000cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:20.358 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.356+0000 7fe3074ce640 1 --2- 172.21.15.115:0/1930612748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 0x7fe3000cb1b0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009870 tx=0x7fe2fc02b900 comp rx=0 tx=0).stop 2023-12-01T03:11:20.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.357+0000 7fe3074ce640 1 -- 172.21.15.115:0/1930612748 shutdown_connections 2023-12-01T03:11:20.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.357+0000 7fe3074ce640 1 --2- 172.21.15.115:0/1930612748 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 0x7fe3000cb1b0 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:20.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.357+0000 7fe3074ce640 1 -- 172.21.15.115:0/1930612748 >> 172.21.15.115:0/1930612748 conn(0x7fe3000662c0 msgr2=0x7fe3000b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:20.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.357+0000 7fe3074ce640 1 -- 172.21.15.115:0/1930612748 shutdown_connections 2023-12-01T03:11:20.359 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.357+0000 7fe3074ce640 1 -- 172.21.15.115:0/1930612748 wait complete. 2023-12-01T03:11:20.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.358+0000 7fe3074ce640 1 Processor -- start 2023-12-01T03:11:20.360 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.358+0000 7fe3074ce640 1 -- start start 2023-12-01T03:11:20.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.359+0000 7fe3074ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3000adca0 0x7fe300150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:20.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.359+0000 7fe3074ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3000b4070 0x7fe3000c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.359+0000 7fe3074ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 0x7fe3000ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.359+0000 7fe3074ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe300150d50 con 0x7fe30015ac70 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.359+0000 7fe3074ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3000c31a0 con 0x7fe3000adca0 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.359+0000 7fe3074ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3000c8df0 con 0x7fe3000b4070 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe305a6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 0x7fe3000ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe305a6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 0x7fe3000ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46968/0 (socket says 172.21.15.115:46968) 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe305a6b640 1 -- 172.21.15.115:0/4241640654 learned_addr learned my addr 172.21.15.115:0/4241640654 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe30526a640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3000adca0 0x7fe300150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe305a6b640 1 -- 172.21.15.115:0/4241640654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3000b4070 msgr2=0x7fe3000c2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe304a69640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3000b4070 0x7fe3000c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe305a6b640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3000b4070 0x7fe3000c2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe305a6b640 1 -- 172.21.15.115:0/4241640654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3000adca0 msgr2=0x7fe300150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:20.363 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe305a6b640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3000adca0 0x7fe300150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe305a6b640 1 -- 172.21.15.115:0/4241640654 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc0094d0 con 0x7fe30015ac70 2023-12-01T03:11:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe30526a640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3000adca0 0x7fe300150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe304a69640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3000b4070 0x7fe3000c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.360+0000 7fe305a6b640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 0x7fe3000ac8a0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800b970 tx=0x7fe2f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.361+0000 7fe2f5ffb640 1 -- 172.21.15.115:0/4241640654 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f800cb40 con 0x7fe30015ac70 2023-12-01T03:11:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.361+0000 7fe2f5ffb640 1 -- 172.21.15.115:0/4241640654 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f800cca0 con 0x7fe30015ac70 2023-12-01T03:11:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.361+0000 7fe2f5ffb640 1 -- 172.21.15.115:0/4241640654 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8014680 con 0x7fe30015ac70 2023-12-01T03:11:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.361+0000 7fe3074ce640 1 -- 172.21.15.115:0/4241640654 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3000c90d0 con 0x7fe30015ac70 2023-12-01T03:11:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.361+0000 7fe3074ce640 1 -- 172.21.15.115:0/4241640654 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3000c9690 con 0x7fe30015ac70 2023-12-01T03:11:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.362+0000 7fe3074ce640 1 -- 172.21.15.115:0/4241640654 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3000c98b0 con 0x7fe30015ac70 2023-12-01T03:11:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.363+0000 7fe2f5ffb640 1 -- 172.21.15.115:0/4241640654 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fe2f801d3b0 con 0x7fe30015ac70 2023-12-01T03:11:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.364+0000 7fe2f5ffb640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe2e406a6a0 0x7fe2e406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.364+0000 7fe2f5ffb640 1 -- 172.21.15.115:0/4241640654 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fe2f8088f80 con 0x7fe30015ac70 2023-12-01T03:11:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.364+0000 7fe30526a640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe2e406a6a0 0x7fe2e406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:20.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.365+0000 7fe30526a640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe2e406a6a0 0x7fe2e406cb60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc036000 tx=0x7fe2fc03c040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:20.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.368+0000 7fe2f5ffb640 1 -- 172.21.15.115:0/4241640654 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe2f8053db0 con 0x7fe30015ac70 2023-12-01T03:11:20.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:20.544+0000 7fe3074ce640 1 -- 172.21.15.115:0/4241640654 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe300074210 con 0x7fe30015ac70 2023-12-01T03:11:21.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.120+0000 7fe2f5ffb640 1 -- 172.21.15.115:0/4241640654 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v225) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe2f80573d0 con 0x7fe30015ac70 2023-12-01T03:11:21.122 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'backup_fs_metadata' removed 2023-12-01T03:11:21.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 -- 172.21.15.115:0/4241640654 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe2e406a6a0 msgr2=0x7fe2e406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:21.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe2e406a6a0 0x7fe2e406cb60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc036000 tx=0x7fe2fc03c040 comp rx=0 tx=0).stop 2023-12-01T03:11:21.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 -- 172.21.15.115:0/4241640654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 msgr2=0x7fe3000ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:21.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 0x7fe3000ac8a0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800b970 tx=0x7fe2f800be30 comp rx=0 tx=0).stop 2023-12-01T03:11:21.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 -- 172.21.15.115:0/4241640654 shutdown_connections 2023-12-01T03:11:21.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe2e406a6a0 0x7fe2e406cb60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3000b4070 0x7fe3000c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe30015ac70 0x7fe3000ac8a0 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 --2- 172.21.15.115:0/4241640654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3000adca0 0x7fe300150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 -- 172.21.15.115:0/4241640654 >> 172.21.15.115:0/4241640654 conn(0x7fe3000662c0 msgr2=0x7fe3000b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:21.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 -- 172.21.15.115:0/4241640654 shutdown_connections 2023-12-01T03:11:21.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.124+0000 7fe3074ce640 1 -- 172.21.15.115:0/4241640654 wait complete. 2023-12-01T03:11:21.146 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-12-01T03:11:21.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.367+0000 7fa01e2cc640 1 Processor -- start 2023-12-01T03:11:21.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.367+0000 7fa01e2cc640 1 -- start start 2023-12-01T03:11:21.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.368+0000 7fa01e2cc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0181599e0 0x7fa018159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:21.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.368+0000 7fa01e2cc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa0180af370 0x7fa0180c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:21.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.369+0000 7fa01e2cc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01815e440 con 0x7fa0180c95a0 2023-12-01T03:11:21.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.369+0000 7fa01e2cc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01815a300 con 0x7fa0180af370 2023-12-01T03:11:21.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.369+0000 7fa01e2cc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01815d550 con 0x7fa0181599e0 2023-12-01T03:11:21.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.369+0000 7fa01dacb640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa0180c95a0 0x7fa01806f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46420/0 (socket says 172.21.15.115:46420) 2023-12-01T03:11:21.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.369+0000 7fa01dacb640 1 -- 172.21.15.115:0/1309557949 learned_addr learned my addr 172.21.15.115:0/1309557949 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:21.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.369+0000 7fa01d2ca640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0181599e0 0x7fa018159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:21.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.369+0000 7fa01cac9640 1 --2- 172.21.15.115:0/1309557949 >> v2:172.21.15.142:3300/0 conn(0x7fa0180af370 0x7fa0180c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:21.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.369+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918099524 0 0) 0x7fa01815e440 con 0x7fa0180c95a0 2023-12-01T03:11:21.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa004003520 con 0x7fa0180c95a0 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 612277495 0 0) 0x7fa004003520 con 0x7fa0180c95a0 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0181599e0 msgr2=0x7fa018159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0181599e0 0x7fa018159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 >> v2:172.21.15.142:3300/0 conn(0x7fa0180af370 msgr2=0x7fa0180c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 --2- 172.21.15.115:0/1309557949 >> v2:172.21.15.142:3300/0 conn(0x7fa0180af370 0x7fa0180c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01815d830 con 0x7fa0180c95a0 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa0100019f0 con 0x7fa0180c95a0 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 >> v1:172.21.15.115:6789/0 conn(0x7fa0180c95a0 legacy=0x7fa01806f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa004003970 0x7fa004003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0181599e0 0x7fa0040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa004004d00 0x7fa004005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01815a300 con 0x7fa004004d00 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01815d550 con 0x7fa004003970 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00400b980 con 0x7fa0181599e0 2023-12-01T03:11:21.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.370+0000 7fa01dacb640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa004004d00 0x7fa004005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:21.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.371+0000 7fa01d2ca640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa004003970 0x7fa004003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:21.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.371+0000 7fa01cac9640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0181599e0 0x7fa0040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:21.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.371+0000 7fa01dacb640 1 -- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0181599e0 msgr2=0x7fa0040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:21.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.371+0000 7fa01dacb640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0181599e0 0x7fa0040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.371+0000 7fa01dacb640 1 -- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa004003970 msgr2=0x7fa004003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:21.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.371+0000 7fa01dacb640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa004003970 0x7fa004003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.371+0000 7fa01dacb640 1 -- 172.21.15.115:0/1309557949 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa004003520 con 0x7fa004004d00 2023-12-01T03:11:21.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.372+0000 7fa01dacb640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa004004d00 0x7fa004005180 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fa01000a790 tx=0x7fa0040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f447c295555e4b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.372+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010034630 con 0x7fa004004d00 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.372+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01002f070 con 0x7fa004004d00 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.372+0000 7fa00ffff640 1 -- 172.21.15.115:0/1309557949 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100349f0 con 0x7fa004004d00 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.373+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1309557949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa004004d00 msgr2=0x7fa004005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.373+0000 7fa01e2cc640 1 --2- 172.21.15.115:0/1309557949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa004004d00 0x7fa004005180 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fa01000a790 tx=0x7fa0040057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.373+0000 7fa01dacb640 1 -- 172.21.15.115:0/1309557949 reap_dead start 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.373+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1309557949 shutdown_connections 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.373+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1309557949 >> 172.21.15.115:0/1309557949 conn(0x7fa018065960 msgr2=0x7fa018065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.373+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1309557949 shutdown_connections 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.373+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1309557949 wait complete. 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.375+0000 7fa01e2cc640 1 Processor -- start 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.375+0000 7fa01e2cc640 1 -- start start 2023-12-01T03:11:21.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.376+0000 7fa01e2cc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa004004d00 0x7fa018151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.376+0000 7fa01e2cc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa01815c170 0x7fa018153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.376+0000 7fa01e2cc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0181f1dd0 0x7fa0181f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.376+0000 7fa01e2cc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0181522d0 con 0x7fa01815c170 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.376+0000 7fa01e2cc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0181535a0 con 0x7fa0181f1dd0 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.376+0000 7fa01e2cc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01815ced0 con 0x7fa004004d00 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.376+0000 7fa01cac9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa01815c170 0x7fa018153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.376+0000 7fa01cac9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa01815c170 0x7fa018153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46984/0 (socket says 172.21.15.115:46984) 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.376+0000 7fa01cac9640 1 -- 172.21.15.115:0/1123720148 learned_addr learned my addr 172.21.15.115:0/1123720148 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa01d2ca640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa004004d00 0x7fa018151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa01dacb640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0181f1dd0 0x7fa0181f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa01cac9640 1 -- 172.21.15.115:0/1123720148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa004004d00 msgr2=0x7fa018151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa01cac9640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa004004d00 0x7fa018151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa01cac9640 1 -- 172.21.15.115:0/1123720148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0181f1dd0 msgr2=0x7fa0181f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:21.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa01cac9640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0181f1dd0 0x7fa0181f2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa01cac9640 1 -- 172.21.15.115:0/1123720148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180cd6f0 con 0x7fa01815c170 2023-12-01T03:11:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa01cac9640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa01815c170 0x7fa018153060 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fa0080116d0 tx=0x7fa008011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa00dffb640 1 -- 172.21.15.115:0/1123720148 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00800f960 con 0x7fa01815c170 2023-12-01T03:11:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa00dffb640 1 -- 172.21.15.115:0/1123720148 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00800fac0 con 0x7fa01815c170 2023-12-01T03:11:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa00dffb640 1 -- 172.21.15.115:0/1123720148 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00800fe30 con 0x7fa01815c170 2023-12-01T03:11:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1123720148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01815c630 con 0x7fa01815c170 2023-12-01T03:11:21.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.377+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1123720148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01815cbf0 con 0x7fa01815c170 2023-12-01T03:11:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.378+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1123720148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0180618d0 con 0x7fa01815c170 2023-12-01T03:11:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.379+0000 7fa00dffb640 1 -- 172.21.15.115:0/1123720148 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fa008015460 con 0x7fa01815c170 2023-12-01T03:11:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.381+0000 7fa00dffb640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9ffc06a670 0x7f9ffc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.381+0000 7fa00dffb640 1 -- 172.21.15.115:0/1123720148 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6224+0+0 (secure 0 0 0) 0x7fa008089530 con 0x7fa01815c170 2023-12-01T03:11:21.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.381+0000 7fa01d2ca640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9ffc06a670 0x7f9ffc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:21.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.382+0000 7fa01d2ca640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9ffc06a670 0x7f9ffc06cb30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa014009950 tx=0x7fa014008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:21.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.385+0000 7fa00dffb640 1 -- 172.21.15.115:0/1123720148 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa0080545f0 con 0x7fa01815c170 2023-12-01T03:11:21.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:21.581+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1123720148 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa0180c9e60 con 0x7fa01815c170 2023-12-01T03:11:22.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.124+0000 7fa00dffb640 1 -- 172.21.15.115:0/1123720148 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v226) v1 ==== 178+0+0 (secure 0 0 0) 0x7fa008057c10 con 0x7fa01815c170 2023-12-01T03:11:22.126 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'backup_fs_data' removed 2023-12-01T03:11:22.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.127+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1123720148 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9ffc06a670 msgr2=0x7f9ffc06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.127+0000 7fa01e2cc640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9ffc06a670 0x7f9ffc06cb30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa014009950 tx=0x7fa014008040 comp rx=0 tx=0).stop 2023-12-01T03:11:22.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.127+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1123720148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa01815c170 msgr2=0x7fa018153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.127+0000 7fa01e2cc640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa01815c170 0x7fa018153060 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fa0080116d0 tx=0x7fa008011b90 comp rx=0 tx=0).stop 2023-12-01T03:11:22.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.128+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1123720148 shutdown_connections 2023-12-01T03:11:22.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.128+0000 7fa01e2cc640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9ffc06a670 0x7f9ffc06cb30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.128+0000 7fa01e2cc640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0181f1dd0 0x7fa0181f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.128+0000 7fa01e2cc640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa01815c170 0x7fa018153060 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.128+0000 7fa01e2cc640 1 --2- 172.21.15.115:0/1123720148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa004004d00 0x7fa018151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.128+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1123720148 >> 172.21.15.115:0/1123720148 conn(0x7fa018065960 msgr2=0x7fa0180b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:22.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.128+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1123720148 shutdown_connections 2023-12-01T03:11:22.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.128+0000 7fa01e2cc640 1 -- 172.21.15.115:0/1123720148 wait complete. 2023-12-01T03:11:22.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T03:11:22.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.362+0000 7fde7cffb640 1 Processor -- start 2023-12-01T03:11:22.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.363+0000 7fde7cffb640 1 -- start start 2023-12-01T03:11:22.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.364+0000 7fde7cffb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fde780ce1b0 0x7fde780cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.364+0000 7fde7cffb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde780cc9a0 con 0x7fde7815ac80 2023-12-01T03:11:22.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.364+0000 7fde7cffb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde780cbd90 con 0x7fde780b0e60 2023-12-01T03:11:22.366 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.364+0000 7fde7cffb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde7815b060 con 0x7fde780ce1b0 2023-12-01T03:11:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.365+0000 7fde7659c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fde7815ac80 0x7fde780cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46430/0 (socket says 172.21.15.115:46430) 2023-12-01T03:11:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.365+0000 7fde7659c640 1 -- 172.21.15.115:0/2517046468 learned_addr learned my addr 172.21.15.115:0/2517046468 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.365+0000 7fde75d9b640 1 --2- 172.21.15.115:0/2517046468 >> v2:172.21.15.142:3300/0 conn(0x7fde780ce1b0 0x7fde780cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.365+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816631829 0 0) 0x7fde780cc9a0 con 0x7fde7815ac80 2023-12-01T03:11:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.365+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde60003520 con 0x7fde7815ac80 2023-12-01T03:11:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.365+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892254533 0 0) 0x7fde60003520 con 0x7fde7815ac80 2023-12-01T03:11:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.365+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 >> v2:172.21.15.142:3300/0 conn(0x7fde780ce1b0 msgr2=0x7fde780cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:22.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.365+0000 7fde7559a640 1 --2- 172.21.15.115:0/2517046468 >> v2:172.21.15.142:3300/0 conn(0x7fde780ce1b0 0x7fde780cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.365+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 >> v1:172.21.15.142:6789/0 conn(0x7fde780b0e60 legacy=0x7fde7806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde780cc070 con 0x7fde7815ac80 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fde68001670 con 0x7fde7815ac80 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 >> v1:172.21.15.115:6789/0 conn(0x7fde7815ac80 legacy=0x7fde780cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde7559a640 1 --2- 172.21.15.115:0/2517046468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde60003970 0x7fde60003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde7559a640 1 --2- 172.21.15.115:0/2517046468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde600047c0 0x7fde60004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde7559a640 1 --2- 172.21.15.115:0/2517046468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde60005180 0x7fde6000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7815b060 con 0x7fde600047c0 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6000c020 con 0x7fde60005180 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6000c1e0 con 0x7fde60003970 2023-12-01T03:11:22.369 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde75d9b640 1 --2- 172.21.15.115:0/2517046468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde600047c0 0x7fde60004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde75d9b640 1 -- 172.21.15.115:0/2517046468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde60003970 msgr2=0x7fde60003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde75d9b640 1 --2- 172.21.15.115:0/2517046468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde60003970 0x7fde60003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde75d9b640 1 -- 172.21.15.115:0/2517046468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde60005180 msgr2=0x7fde6000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde75d9b640 1 --2- 172.21.15.115:0/2517046468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde60005180 0x7fde6000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde75d9b640 1 -- 172.21.15.115:0/2517046468 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60003520 con 0x7fde600047c0 2023-12-01T03:11:22.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.366+0000 7fde76d9d640 1 -- 172.21.15.115:0/2517046468 reap_dead start 2023-12-01T03:11:22.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.367+0000 7fde75d9b640 1 --2- 172.21.15.115:0/2517046468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde600047c0 0x7fde60004c40 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009fc0 tx=0x7fde600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304289cdffc0f459 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:22.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.368+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c02cc30 con 0x7fde600047c0 2023-12-01T03:11:22.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.368+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c00f040 con 0x7fde600047c0 2023-12-01T03:11:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.368+0000 7fde7559a640 1 -- 172.21.15.115:0/2517046468 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c034440 con 0x7fde600047c0 2023-12-01T03:11:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.368+0000 7fde7cffb640 1 -- 172.21.15.115:0/2517046468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde600047c0 msgr2=0x7fde60004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.368+0000 7fde7cffb640 1 --2- 172.21.15.115:0/2517046468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde600047c0 0x7fde60004c40 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009fc0 tx=0x7fde600057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.368+0000 7fde7cffb640 1 -- 172.21.15.115:0/2517046468 shutdown_connections 2023-12-01T03:11:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.368+0000 7fde7cffb640 1 --2- 172.21.15.115:0/2517046468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde600047c0 0x7fde60004c40 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.368+0000 7fde7cffb640 1 -- 172.21.15.115:0/2517046468 >> 172.21.15.115:0/2517046468 conn(0x7fde78065960 msgr2=0x7fde78065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:22.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.368+0000 7fde7cffb640 1 -- 172.21.15.115:0/2517046468 shutdown_connections 2023-12-01T03:11:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.368+0000 7fde7cffb640 1 -- 172.21.15.115:0/2517046468 wait complete. 2023-12-01T03:11:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.370+0000 7fde7cffb640 1 Processor -- start 2023-12-01T03:11:22.374 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.370+0000 7fde7cffb640 1 -- start start 2023-12-01T03:11:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.371+0000 7fde7cffb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde600047c0 0x7fde7814f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.371+0000 7fde7cffb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde78070320 0x7fde780707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde7cffb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde78156d80 0x7fde78157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde7cffb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78070ce0 con 0x7fde78070320 2023-12-01T03:11:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde7cffb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78158ae0 con 0x7fde78156d80 2023-12-01T03:11:22.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde7cffb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde78157a40 con 0x7fde600047c0 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde75d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde78070320 0x7fde780707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde75d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde78070320 0x7fde780707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47012/0 (socket says 172.21.15.115:47012) 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde75d9b640 1 -- 172.21.15.115:0/212451766 learned_addr learned my addr 172.21.15.115:0/212451766 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde7659c640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde600047c0 0x7fde7814f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde76d9d640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde78156d80 0x7fde78157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde75d9b640 1 -- 172.21.15.115:0/212451766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde600047c0 msgr2=0x7fde7814f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde75d9b640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde600047c0 0x7fde7814f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde75d9b640 1 -- 172.21.15.115:0/212451766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde78156d80 msgr2=0x7fde78157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde75d9b640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde78156d80 0x7fde78157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde75d9b640 1 -- 172.21.15.115:0/212451766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde78157d20 con 0x7fde78070320 2023-12-01T03:11:22.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.372+0000 7fde76d9d640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde78156d80 0x7fde78157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.373+0000 7fde75d9b640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde78070320 0x7fde780707a0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7fde6c037040 tx=0x7fde6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.373+0000 7fde66ffd640 1 -- 172.21.15.115:0/212451766 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c034bd0 con 0x7fde78070320 2023-12-01T03:11:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.373+0000 7fde66ffd640 1 -- 172.21.15.115:0/212451766 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c034d30 con 0x7fde78070320 2023-12-01T03:11:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.373+0000 7fde66ffd640 1 -- 172.21.15.115:0/212451766 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c034ea0 con 0x7fde78070320 2023-12-01T03:11:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.373+0000 7fde7cffb640 1 -- 172.21.15.115:0/212451766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde78158000 con 0x7fde78070320 2023-12-01T03:11:22.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.373+0000 7fde7cffb640 1 -- 172.21.15.115:0/212451766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde781585c0 con 0x7fde78070320 2023-12-01T03:11:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.375+0000 7fde7cffb640 1 -- 172.21.15.115:0/212451766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7806c980 con 0x7fde78070320 2023-12-01T03:11:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.376+0000 7fde66ffd640 1 -- 172.21.15.115:0/212451766 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fde6c040450 con 0x7fde78070320 2023-12-01T03:11:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.377+0000 7fde66ffd640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde5806a670 0x7fde5806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.377+0000 7fde66ffd640 1 -- 172.21.15.115:0/212451766 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fde6c0ab630 con 0x7fde78070320 2023-12-01T03:11:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.377+0000 7fde7659c640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde5806a670 0x7fde5806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.378+0000 7fde7659c640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde5806a670 0x7fde5806cb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fde680011e0 tx=0x7fde68001680 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:22.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.381+0000 7fde66ffd640 1 -- 172.21.15.115:0/212451766 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fde6c076970 con 0x7fde78070320 2023-12-01T03:11:22.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.570+0000 7fde7cffb640 1 -- 172.21.15.115:0/212451766 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fde7804e890 con 0x7fde78070320 2023-12-01T03:11:22.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.576+0000 7fde66ffd640 1 -- 172.21.15.115:0/212451766 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7fde6c079f90 con 0x7fde78070320 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.579+0000 7fde7cffb640 1 -- 172.21.15.115:0/212451766 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde5806a670 msgr2=0x7fde5806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.579+0000 7fde7cffb640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde5806a670 0x7fde5806cb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fde680011e0 tx=0x7fde68001680 comp rx=0 tx=0).stop 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.580+0000 7fde7cffb640 1 -- 172.21.15.115:0/212451766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde78070320 msgr2=0x7fde780707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.580+0000 7fde7cffb640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde78070320 0x7fde780707a0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7fde6c037040 tx=0x7fde6c009750 comp rx=0 tx=0).stop 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.580+0000 7fde7cffb640 1 -- 172.21.15.115:0/212451766 shutdown_connections 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.580+0000 7fde7cffb640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fde5806a670 0x7fde5806cb30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.580+0000 7fde7cffb640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde78156d80 0x7fde78157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.580+0000 7fde7cffb640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fde78070320 0x7fde780707a0 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.580+0000 7fde7cffb640 1 --2- 172.21.15.115:0/212451766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde600047c0 0x7fde7814f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.580+0000 7fde7cffb640 1 -- 172.21.15.115:0/212451766 >> 172.21.15.115:0/212451766 conn(0x7fde78065960 msgr2=0x7fde780ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.580+0000 7fde7cffb640 1 -- 172.21.15.115:0/212451766 shutdown_connections 2023-12-01T03:11:22.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.580+0000 7fde7cffb640 1 -- 172.21.15.115:0/212451766 wait complete. 2023-12-01T03:11:22.601 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T03:11:22.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.806+0000 7f462ae4a640 1 Processor -- start 2023-12-01T03:11:22.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.806+0000 7f462ae4a640 1 -- start start 2023-12-01T03:11:22.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.808+0000 7f462ae4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462415ac80 0x7f46240cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.808+0000 7f462ae4a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f46240b0e60 0x7f46240b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.808+0000 7f462ae4a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46240cc9a0 con 0x7f46240cdaa0 2023-12-01T03:11:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.808+0000 7f462ae4a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462415b060 con 0x7f46240b0e60 2023-12-01T03:11:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.808+0000 7f462ae4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46240b0ce0 con 0x7f462415ac80 2023-12-01T03:11:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.808+0000 7f4623fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f46240b0e60 0x7f46240b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.808+0000 7f4628be6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462415ac80 0x7f46240cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.808+0000 7f4623fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f46240b0e60 0x7f46240b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:42476/0 (socket says 172.21.15.115:42476) 2023-12-01T03:11:22.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.808+0000 7f4623fff640 1 -- 172.21.15.115:0/2069706392 learned_addr learned my addr 172.21.15.115:0/2069706392 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:22.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.809+0000 7f4623fff640 1 -- 172.21.15.115:0/2069706392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462415ac80 msgr2=0x7f46240cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:22.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.809+0000 7f4623fff640 1 --2- 172.21.15.115:0/2069706392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462415ac80 0x7f46240cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.809+0000 7f4623fff640 1 -- 172.21.15.115:0/2069706392 >> v1:172.21.15.142:6789/0 conn(0x7f46240cdaa0 legacy=0x7f462406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.809+0000 7f4623fff640 1 -- 172.21.15.115:0/2069706392 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240cbd90 con 0x7f46240b0e60 2023-12-01T03:11:22.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.809+0000 7f4628be6640 1 --2- 172.21.15.115:0/2069706392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462415ac80 0x7f46240cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:11:22.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.811+0000 7f4623fff640 1 --2- 172.21.15.115:0/2069706392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46240b0e60 0x7f46240b07a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4610009870 tx=0x7f461002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e4940553d4b806 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:22.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.811+0000 7f46237fe640 1 -- 172.21.15.115:0/2069706392 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610030840 con 0x7f46240b0e60 2023-12-01T03:11:22.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.811+0000 7f46237fe640 1 -- 172.21.15.115:0/2069706392 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46100309a0 con 0x7f46240b0e60 2023-12-01T03:11:22.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.811+0000 7f46237fe640 1 -- 172.21.15.115:0/2069706392 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610030d30 con 0x7f46240b0e60 2023-12-01T03:11:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.812+0000 7f462ae4a640 1 -- 172.21.15.115:0/2069706392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46240b0e60 msgr2=0x7f46240b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.812+0000 7f462ae4a640 1 --2- 172.21.15.115:0/2069706392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46240b0e60 0x7f46240b07a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4610009870 tx=0x7f461002b980 comp rx=0 tx=0).stop 2023-12-01T03:11:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.812+0000 7f462ae4a640 1 -- 172.21.15.115:0/2069706392 shutdown_connections 2023-12-01T03:11:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.812+0000 7f462ae4a640 1 --2- 172.21.15.115:0/2069706392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46240b0e60 0x7f46240b07a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.812+0000 7f462ae4a640 1 --2- 172.21.15.115:0/2069706392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462415ac80 0x7f46240cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.812+0000 7f462ae4a640 1 -- 172.21.15.115:0/2069706392 >> 172.21.15.115:0/2069706392 conn(0x7f4624065960 msgr2=0x7f4624065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:22.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.812+0000 7f462ae4a640 1 -- 172.21.15.115:0/2069706392 shutdown_connections 2023-12-01T03:11:22.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.812+0000 7f462ae4a640 1 -- 172.21.15.115:0/2069706392 wait complete. 2023-12-01T03:11:22.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.814+0000 7f462ae4a640 1 Processor -- start 2023-12-01T03:11:22.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.814+0000 7f462ae4a640 1 -- start start 2023-12-01T03:11:22.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.815+0000 7f462ae4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46240b0e60 0x7f46241f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.815+0000 7f462ae4a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46240cdaa0 0x7f462414e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.815+0000 7f462ae4a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462415ac80 0x7f4624151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.815+0000 7f462ae4a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4624152ef0 con 0x7f462415ac80 2023-12-01T03:11:22.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.815+0000 7f462ae4a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462414e6d0 con 0x7f46240cdaa0 2023-12-01T03:11:22.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.815+0000 7f462ae4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4624153190 con 0x7f46240b0e60 2023-12-01T03:11:22.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f46293e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462415ac80 0x7f4624151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f46293e7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462415ac80 0x7f4624151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47024/0 (socket says 172.21.15.115:47024) 2023-12-01T03:11:22.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f46293e7640 1 -- 172.21.15.115:0/2990319017 learned_addr learned my addr 172.21.15.115:0/2990319017 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:22.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f4623fff640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46240cdaa0 0x7f462414e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f4628be6640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46240b0e60 0x7f46241f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f46293e7640 1 -- 172.21.15.115:0/2990319017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46240b0e60 msgr2=0x7f46241f2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f46293e7640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46240b0e60 0x7f46241f2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f46293e7640 1 -- 172.21.15.115:0/2990319017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46240cdaa0 msgr2=0x7f462414e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:22.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f46293e7640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46240cdaa0 0x7f462414e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:22.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f46293e7640 1 -- 172.21.15.115:0/2990319017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618009400 con 0x7f462415ac80 2023-12-01T03:11:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.816+0000 7f4623fff640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46240cdaa0 0x7f462414e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:11:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.817+0000 7f46293e7640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462415ac80 0x7f4624151000 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f461c00c7a0 tx=0x7f461c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.817+0000 7f46217fa640 1 -- 172.21.15.115:0/2990319017 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c003b70 con 0x7f462415ac80 2023-12-01T03:11:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.817+0000 7f46217fa640 1 -- 172.21.15.115:0/2990319017 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c003cf0 con 0x7f462415ac80 2023-12-01T03:11:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.817+0000 7f46217fa640 1 -- 172.21.15.115:0/2990319017 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c003ea0 con 0x7f462415ac80 2023-12-01T03:11:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.817+0000 7f462ae4a640 1 -- 172.21.15.115:0/2990319017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46100094d0 con 0x7f462415ac80 2023-12-01T03:11:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.817+0000 7f462ae4a640 1 -- 172.21.15.115:0/2990319017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4624153750 con 0x7f462415ac80 2023-12-01T03:11:22.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.818+0000 7f462ae4a640 1 -- 172.21.15.115:0/2990319017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4624153940 con 0x7f462415ac80 2023-12-01T03:11:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.819+0000 7f46217fa640 1 -- 172.21.15.115:0/2990319017 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f461c00aa90 con 0x7f462415ac80 2023-12-01T03:11:22.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.820+0000 7f46217fa640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 0x7f460806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.820+0000 7f46217fa640 1 -- 172.21.15.115:0/2990319017 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f461c088c90 con 0x7f462415ac80 2023-12-01T03:11:22.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.820+0000 7f4628be6640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 0x7f460806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:22.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.821+0000 7f4628be6640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 0x7f460806cb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4618009bc0 tx=0x7f4618008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:22.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:22.825+0000 7f46217fa640 1 -- 172.21.15.115:0/2990319017 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f461c053e20 con 0x7f462415ac80 2023-12-01T03:11:23.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.026+0000 7f462ae4a640 1 -- 172.21.15.115:0/2990319017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f462404e890 con 0x7f462415ac80 2023-12-01T03:11:23.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.031+0000 7f46217fa640 1 -- 172.21.15.115:0/2990319017 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v74) v1 ==== 120+0+0 (secure 0 0 0) 0x7f461c057440 con 0x7f462415ac80 2023-12-01T03:11:23.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.034+0000 7f462ae4a640 1 -- 172.21.15.115:0/2990319017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 msgr2=0x7f460806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.034+0000 7f462ae4a640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 0x7f460806cb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4618009bc0 tx=0x7f4618008040 comp rx=0 tx=0).stop 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.034+0000 7f462ae4a640 1 -- 172.21.15.115:0/2990319017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462415ac80 msgr2=0x7f4624151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.034+0000 7f462ae4a640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462415ac80 0x7f4624151000 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f461c00c7a0 tx=0x7f461c00cc60 comp rx=0 tx=0).stop 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.035+0000 7f462ae4a640 1 -- 172.21.15.115:0/2990319017 shutdown_connections 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.035+0000 7f462ae4a640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 0x7f460806cb30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.035+0000 7f462ae4a640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462415ac80 0x7f4624151000 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.035+0000 7f462ae4a640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46240cdaa0 0x7f462414e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.035+0000 7f462ae4a640 1 --2- 172.21.15.115:0/2990319017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46240b0e60 0x7f46241f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.035+0000 7f462ae4a640 1 -- 172.21.15.115:0/2990319017 >> 172.21.15.115:0/2990319017 conn(0x7f4624065960 msgr2=0x7f462414eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.035+0000 7f462ae4a640 1 -- 172.21.15.115:0/2990319017 shutdown_connections 2023-12-01T03:11:23.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.035+0000 7f462ae4a640 1 -- 172.21.15.115:0/2990319017 wait complete. 2023-12-01T03:11:23.054 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T03:11:23.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.251+0000 7f5412f01640 1 Processor -- start 2023-12-01T03:11:23.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.251+0000 7f5412f01640 1 -- start start 2023-12-01T03:11:23.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.252+0000 7f5412f01640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f540c15bdf0 0x7f540c15f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.252+0000 7f5412f01640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c15b040 con 0x7f540c0cb7e0 2023-12-01T03:11:23.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.252+0000 7f5412f01640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c0ae1a0 con 0x7f540c0addc0 2023-12-01T03:11:23.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.253+0000 7f5412f01640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c15f5d0 con 0x7f540c15bdf0 2023-12-01T03:11:23.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.253+0000 7f5412700640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f540c0cb7e0 0x7f540c0b2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46438/0 (socket says 172.21.15.115:46438) 2023-12-01T03:11:23.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.253+0000 7f5412700640 1 -- 172.21.15.115:0/3672919107 learned_addr learned my addr 172.21.15.115:0/3672919107 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:23.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.253+0000 7f54116fe640 1 --2- 172.21.15.115:0/3672919107 >> v2:172.21.15.142:3300/0 conn(0x7f540c15bdf0 0x7f540c15f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.253+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972283392 0 0) 0x7f540c15b040 con 0x7f540c0cb7e0 2023-12-01T03:11:23.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.253+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53f8003520 con 0x7f540c0cb7e0 2023-12-01T03:11:23.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3258547097 0 0) 0x7f53f8003520 con 0x7f540c0cb7e0 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 >> v2:172.21.15.142:3300/0 conn(0x7f540c15bdf0 msgr2=0x7f540c15f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 --2- 172.21.15.115:0/3672919107 >> v2:172.21.15.142:3300/0 conn(0x7f540c15bdf0 0x7f540c15f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 >> v1:172.21.15.142:6789/0 conn(0x7f540c0addc0 legacy=0x7f540c0cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c0c9d40 con 0x7f540c0cb7e0 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f54040019f0 con 0x7f540c0cb7e0 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 >> v1:172.21.15.115:6789/0 conn(0x7f540c0cb7e0 legacy=0x7f540c0b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 --2- 172.21.15.115:0/3672919107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53f80039c0 0x7f53f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 --2- 172.21.15.115:0/3672919107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53f80047c0 0x7f53f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 --2- 172.21.15.115:0/3672919107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53f8005180 0x7f53f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c15f5d0 con 0x7f53f8005180 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f800c050 con 0x7f53f80039c0 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f800c210 con 0x7f53f80047c0 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.254+0000 7f5412700640 1 --2- 172.21.15.115:0/3672919107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53f8005180 0x7f53f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.255+0000 7f5412700640 1 -- 172.21.15.115:0/3672919107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53f80047c0 msgr2=0x7f53f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.255+0000 7f5412700640 1 --2- 172.21.15.115:0/3672919107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53f80047c0 0x7f53f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.255+0000 7f5412700640 1 -- 172.21.15.115:0/3672919107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53f80039c0 msgr2=0x7f53f8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.255+0000 7f5412700640 1 --2- 172.21.15.115:0/3672919107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53f80039c0 0x7f53f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.255+0000 7f5412700640 1 -- 172.21.15.115:0/3672919107 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f8003520 con 0x7f53f8005180 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.255+0000 7f5412700640 1 -- 172.21.15.115:0/3672919107 reap_dead start 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.256+0000 7f5412700640 1 --2- 172.21.15.115:0/3672919107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53f8005180 0x7f53f800b980 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f540400a7e0 tx=0x7f53f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfcde23bcc7706d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.256+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404034630 con 0x7f53f8005180 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.256+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540402f070 con 0x7f53f8005180 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.256+0000 7f5410efd640 1 -- 172.21.15.115:0/3672919107 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54040349d0 con 0x7f53f8005180 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.256+0000 7f5412f01640 1 -- 172.21.15.115:0/3672919107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53f8005180 msgr2=0x7f53f800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.256+0000 7f5412f01640 1 --2- 172.21.15.115:0/3672919107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53f8005180 0x7f53f800b980 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f540400a7e0 tx=0x7f53f80057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.257+0000 7f5412f01640 1 -- 172.21.15.115:0/3672919107 shutdown_connections 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.257+0000 7f5412f01640 1 --2- 172.21.15.115:0/3672919107 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53f8005180 0x7f53f800b980 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.257+0000 7f5412f01640 1 -- 172.21.15.115:0/3672919107 >> 172.21.15.115:0/3672919107 conn(0x7f540c065960 msgr2=0x7f540c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.257+0000 7f5412f01640 1 -- 172.21.15.115:0/3672919107 shutdown_connections 2023-12-01T03:11:23.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.257+0000 7f5412f01640 1 -- 172.21.15.115:0/3672919107 wait complete. 2023-12-01T03:11:23.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.258+0000 7f5412f01640 1 Processor -- start 2023-12-01T03:11:23.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.259+0000 7f5412f01640 1 -- start start 2023-12-01T03:11:23.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.259+0000 7f5412f01640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53f8005180 0x7f540c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.259+0000 7f5412f01640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f540c0c3670 0x7f540c14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412f01640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f540c0c44d0 0x7f540c0c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412f01640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c1f1f30 con 0x7f540c0c44d0 2023-12-01T03:11:23.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412f01640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c158fd0 con 0x7f540c0c3670 2023-12-01T03:11:23.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412f01640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c1592b0 con 0x7f53f8005180 2023-12-01T03:11:23.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412700640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f540c0c44d0 0x7f540c0c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412700640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f540c0c44d0 0x7f540c0c62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47036/0 (socket says 172.21.15.115:47036) 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412700640 1 -- 172.21.15.115:0/2520640837 learned_addr learned my addr 172.21.15.115:0/2520640837 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f54116fe640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f540c0c3670 0x7f540c14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5411eff640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53f8005180 0x7f540c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412700640 1 -- 172.21.15.115:0/2520640837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53f8005180 msgr2=0x7f540c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412700640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53f8005180 0x7f540c0cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412700640 1 -- 172.21.15.115:0/2520640837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f540c0c3670 msgr2=0x7f540c14ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412700640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f540c0c3670 0x7f540c14ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.260+0000 7f5412700640 1 -- 172.21.15.115:0/2520640837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c159590 con 0x7f540c0c44d0 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.261+0000 7f5412700640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f540c0c44d0 0x7f540c0c62e0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f540400a7e0 tx=0x7f5404001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.261+0000 7f53fe7fc640 1 -- 172.21.15.115:0/2520640837 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404004ea0 con 0x7f540c0c44d0 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.261+0000 7f53fe7fc640 1 -- 172.21.15.115:0/2520640837 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540403f470 con 0x7f540c0c44d0 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.261+0000 7f53fe7fc640 1 -- 172.21.15.115:0/2520640837 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540403f800 con 0x7f540c0c44d0 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.261+0000 7f5412f01640 1 -- 172.21.15.115:0/2520640837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540c159870 con 0x7f540c0c44d0 2023-12-01T03:11:23.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.261+0000 7f5412f01640 1 -- 172.21.15.115:0/2520640837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c14f130 con 0x7f540c0c44d0 2023-12-01T03:11:23.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.262+0000 7f5412f01640 1 -- 172.21.15.115:0/2520640837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540c159ab0 con 0x7f540c0c44d0 2023-12-01T03:11:23.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.263+0000 7f53fe7fc640 1 -- 172.21.15.115:0/2520640837 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f5404031030 con 0x7f540c0c44d0 2023-12-01T03:11:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.265+0000 7f53fe7fc640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f53f006a670 0x7f53f006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.265+0000 7f53fe7fc640 1 -- 172.21.15.115:0/2520640837 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f54040aa300 con 0x7f540c0c44d0 2023-12-01T03:11:23.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.265+0000 7f5411eff640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f53f006a670 0x7f53f006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.265+0000 7f5411eff640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f53f006a670 0x7f53f006cb30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5408003d20 tx=0x7f5408009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:23.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.269+0000 7f53fe7fc640 1 -- 172.21.15.115:0/2520640837 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f540c159ab0 con 0x7f540c0c44d0 2023-12-01T03:11:23.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.456+0000 7f5412f01640 1 -- 172.21.15.115:0/2520640837 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f540c159ab0 con 0x7f540c0c44d0 2023-12-01T03:11:23.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.465+0000 7f53fe7fc640 1 -- 172.21.15.115:0/2520640837 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v75) v1 ==== 115+0+0 (secure 0 0 0) 0x7f540c159ab0 con 0x7f540c0c44d0 2023-12-01T03:11:23.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 -- 172.21.15.115:0/2520640837 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f53f006a670 msgr2=0x7f53f006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f53f006a670 0x7f53f006cb30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5408003d20 tx=0x7f5408009040 comp rx=0 tx=0).stop 2023-12-01T03:11:23.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 -- 172.21.15.115:0/2520640837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f540c0c44d0 msgr2=0x7f540c0c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f540c0c44d0 0x7f540c0c62e0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f540400a7e0 tx=0x7f5404001cb0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 -- 172.21.15.115:0/2520640837 shutdown_connections 2023-12-01T03:11:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f53f006a670 0x7f53f006cb30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f540c0c44d0 0x7f540c0c62e0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f540c0c3670 0x7f540c14ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 --2- 172.21.15.115:0/2520640837 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53f8005180 0x7f540c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 -- 172.21.15.115:0/2520640837 >> 172.21.15.115:0/2520640837 conn(0x7f540c065960 msgr2=0x7f540c15c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.469+0000 7f5412f01640 1 -- 172.21.15.115:0/2520640837 shutdown_connections 2023-12-01T03:11:23.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.470+0000 7f5412f01640 1 -- 172.21.15.115:0/2520640837 wait complete. 2023-12-01T03:11:23.490 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-12-01T03:11:23.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.701+0000 7f2706e17640 1 Processor -- start 2023-12-01T03:11:23.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.702+0000 7f2706e17640 1 -- start start 2023-12-01T03:11:23.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.703+0000 7f2706e17640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f27000b5d80 0x7f27000d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.703+0000 7f2706e17640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27000d1230 0x7f27000d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.703+0000 7f2706e17640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27000d1b90 con 0x7f270015cd20 2023-12-01T03:11:23.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.703+0000 7f2706e17640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27000b2160 con 0x7f27000b5d80 2023-12-01T03:11:23.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.703+0000 7f2706e17640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27000b8c80 con 0x7f27000d1230 2023-12-01T03:11:23.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.704+0000 7f26f7fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f27000b5d80 0x7f27000d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.704+0000 7f27053b4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27000d1230 0x7f27000d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.704+0000 7f26f7fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f27000b5d80 0x7f27000d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:42520/0 (socket says 172.21.15.115:42520) 2023-12-01T03:11:23.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.704+0000 7f26f7fff640 1 -- 172.21.15.115:0/484598606 learned_addr learned my addr 172.21.15.115:0/484598606 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:23.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.705+0000 7f26f7fff640 1 -- 172.21.15.115:0/484598606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27000d1230 msgr2=0x7f27000d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:23.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.705+0000 7f26f7fff640 1 --2- 172.21.15.115:0/484598606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27000d1230 0x7f27000d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.705+0000 7f26f7fff640 1 -- 172.21.15.115:0/484598606 >> v1:172.21.15.142:6789/0 conn(0x7f270015cd20 legacy=0x7f27000b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.705+0000 7f26f7fff640 1 -- 172.21.15.115:0/484598606 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27000ce2e0 con 0x7f27000b5d80 2023-12-01T03:11:23.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.706+0000 7f26f7fff640 1 --2- 172.21.15.115:0/484598606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27000b5d80 0x7f27000d24d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009870 tx=0x7f26f002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54103a049a626fa0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:23.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.706+0000 7f26f77fe640 1 -- 172.21.15.115:0/484598606 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0031640 con 0x7f27000b5d80 2023-12-01T03:11:23.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.706+0000 7f26f77fe640 1 -- 172.21.15.115:0/484598606 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f00317a0 con 0x7f27000b5d80 2023-12-01T03:11:23.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.706+0000 7f26f77fe640 1 -- 172.21.15.115:0/484598606 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0031b30 con 0x7f27000b5d80 2023-12-01T03:11:23.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.707+0000 7f2706e17640 1 -- 172.21.15.115:0/484598606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27000b5d80 msgr2=0x7f27000d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.707+0000 7f2706e17640 1 --2- 172.21.15.115:0/484598606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27000b5d80 0x7f27000d24d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009870 tx=0x7f26f002b8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.707+0000 7f2706e17640 1 -- 172.21.15.115:0/484598606 shutdown_connections 2023-12-01T03:11:23.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.707+0000 7f2706e17640 1 --2- 172.21.15.115:0/484598606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27000d1230 0x7f27000d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.707+0000 7f2706e17640 1 --2- 172.21.15.115:0/484598606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27000b5d80 0x7f27000d24d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.707+0000 7f2706e17640 1 -- 172.21.15.115:0/484598606 >> 172.21.15.115:0/484598606 conn(0x7f270006a0e0 msgr2=0x7f270006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:23.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.707+0000 7f2706e17640 1 -- 172.21.15.115:0/484598606 shutdown_connections 2023-12-01T03:11:23.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.707+0000 7f2706e17640 1 -- 172.21.15.115:0/484598606 wait complete. 2023-12-01T03:11:23.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.709+0000 7f2706e17640 1 Processor -- start 2023-12-01T03:11:23.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.709+0000 7f2706e17640 1 -- start start 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.710+0000 7f2706e17640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27000b5d80 0x7f27000ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.710+0000 7f2706e17640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27000d1230 0x7f27000cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.710+0000 7f2706e17640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270015cd20 0x7f27001f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.710+0000 7f2706e17640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27000b55d0 con 0x7f27000d1230 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.710+0000 7f2706e17640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270015d100 con 0x7f270015cd20 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.710+0000 7f2706e17640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27000cb2a0 con 0x7f27000b5d80 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.710+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27000d1230 0x7f27000cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.710+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27000d1230 0x7f27000cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47040/0 (socket says 172.21.15.115:47040) 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.710+0000 7f26f7fff640 1 -- 172.21.15.115:0/2954518374 learned_addr learned my addr 172.21.15.115:0/2954518374 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.711+0000 7f2704bb3640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27000b5d80 0x7f27000ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.711+0000 7f27053b4640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270015cd20 0x7f27001f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.711+0000 7f26f7fff640 1 -- 172.21.15.115:0/2954518374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27000b5d80 msgr2=0x7f27000ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.711+0000 7f26f7fff640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27000b5d80 0x7f27000ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.711+0000 7f26f7fff640 1 -- 172.21.15.115:0/2954518374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270015cd20 msgr2=0x7f27001f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:23.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.711+0000 7f26f7fff640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270015cd20 0x7f27001f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:23.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.711+0000 7f26f7fff640 1 -- 172.21.15.115:0/2954518374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f00094d0 con 0x7f27000d1230 2023-12-01T03:11:23.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.711+0000 7f27053b4640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270015cd20 0x7f27001f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:23.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.711+0000 7f26f7fff640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27000d1230 0x7f27000cac70 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f26f0000c00 tx=0x7f26f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:23.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.712+0000 7f26f57fa640 1 -- 172.21.15.115:0/2954518374 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f003a930 con 0x7f27000d1230 2023-12-01T03:11:23.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.712+0000 7f26f57fa640 1 -- 172.21.15.115:0/2954518374 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f003aa90 con 0x7f27000d1230 2023-12-01T03:11:23.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.712+0000 7f2706e17640 1 -- 172.21.15.115:0/2954518374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27000cb580 con 0x7f27000d1230 2023-12-01T03:11:23.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.712+0000 7f26f57fa640 1 -- 172.21.15.115:0/2954518374 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f003ae40 con 0x7f27000d1230 2023-12-01T03:11:23.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.712+0000 7f2706e17640 1 -- 172.21.15.115:0/2954518374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270015d630 con 0x7f27000d1230 2023-12-01T03:11:23.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.713+0000 7f2706e17640 1 -- 172.21.15.115:0/2954518374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270015d820 con 0x7f27000d1230 2023-12-01T03:11:23.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.714+0000 7f26f57fa640 1 -- 172.21.15.115:0/2954518374 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f26f002f050 con 0x7f27000d1230 2023-12-01T03:11:23.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.715+0000 7f26f57fa640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f26ec06a6a0 0x7f26ec06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:23.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.715+0000 7f26f57fa640 1 -- 172.21.15.115:0/2954518374 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f26f00aacd0 con 0x7f27000d1230 2023-12-01T03:11:23.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.715+0000 7f2704bb3640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f26ec06a6a0 0x7f26ec06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:23.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.715+0000 7f2704bb3640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f26ec06a6a0 0x7f26ec06cb60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f26fc007fd0 tx=0x7f26fc009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:23.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.719+0000 7f26f57fa640 1 -- 172.21.15.115:0/2954518374 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f26f0075df0 con 0x7f27000d1230 2023-12-01T03:11:23.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:23.986+0000 7f2706e17640 1 -- 172.21.15.115:0/2954518374 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f270004e890 con 0x7f27000d1230 2023-12-01T03:11:24.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.145+0000 7f26f57fa640 1 -- 172.21.15.115:0/2954518374 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3115) v1 ==== 186+0+0 (secure 0 0 0) 0x7f26f0079410 con 0x7f27000d1230 2023-12-01T03:11:24.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.149+0000 7f2706e17640 1 -- 172.21.15.115:0/2954518374 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f26ec06a6a0 msgr2=0x7f26ec06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:24.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.149+0000 7f2706e17640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f26ec06a6a0 0x7f26ec06cb60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f26fc007fd0 tx=0x7f26fc009040 comp rx=0 tx=0).stop 2023-12-01T03:11:24.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.149+0000 7f2706e17640 1 -- 172.21.15.115:0/2954518374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27000d1230 msgr2=0x7f27000cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:24.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.149+0000 7f2706e17640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27000d1230 0x7f27000cac70 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f26f0000c00 tx=0x7f26f00046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:24.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.149+0000 7f2706e17640 1 -- 172.21.15.115:0/2954518374 shutdown_connections 2023-12-01T03:11:24.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.149+0000 7f2706e17640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f26ec06a6a0 0x7f26ec06cb60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:24.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.149+0000 7f2706e17640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f270015cd20 0x7f27001f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:24.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.149+0000 7f2706e17640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27000d1230 0x7f27000cac70 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:24.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.149+0000 7f2706e17640 1 --2- 172.21.15.115:0/2954518374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27000b5d80 0x7f27000ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:24.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.149+0000 7f2706e17640 1 -- 172.21.15.115:0/2954518374 >> 172.21.15.115:0/2954518374 conn(0x7f270006a0e0 msgr2=0x7f2700157400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:24.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.150+0000 7f2706e17640 1 -- 172.21.15.115:0/2954518374 shutdown_connections 2023-12-01T03:11:24.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.150+0000 7f2706e17640 1 -- 172.21.15.115:0/2954518374 wait complete. 2023-12-01T03:11:24.170 INFO:tasks.cephfs_test_runner:Test command succes with --fs argument for non-default fs ... ok 2023-12-01T03:11:24.170 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command fails without --fs argument in presence of multiple fs 2023-12-01T03:11:24.170 INFO:tasks.cephfs_test_runner:test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-12-01T03:11:24.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-12-01T03:11:24.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.379+0000 7f863adf0640 1 Processor -- start 2023-12-01T03:11:24.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.380+0000 7f863adf0640 1 -- start start 2023-12-01T03:11:24.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.381+0000 7f863adf0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86340b5d80 0x7f86340d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:24.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.381+0000 7f863adf0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f86340d1230 0x7f86340d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:24.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.381+0000 7f863adf0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86340d1b90 con 0x7f863415cd20 2023-12-01T03:11:24.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.381+0000 7f863adf0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86340b2160 con 0x7f86340b5d80 2023-12-01T03:11:24.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.381+0000 7f863adf0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86340b8c80 con 0x7f86340d1230 2023-12-01T03:11:24.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.381+0000 7f8633fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86340b5d80 0x7f86340d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:24.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.381+0000 7f8633fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86340b5d80 0x7f86340d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47042/0 (socket says 172.21.15.115:47042) 2023-12-01T03:11:24.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.381+0000 7f8633fff640 1 -- 172.21.15.115:0/593022630 learned_addr learned my addr 172.21.15.115:0/593022630 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:24.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.382+0000 7f863938d640 1 --2- 172.21.15.115:0/593022630 >> v2:172.21.15.142:3300/0 conn(0x7f86340d1230 0x7f86340d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:24.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.382+0000 7f8633fff640 1 -- 172.21.15.115:0/593022630 >> v2:172.21.15.142:3300/0 conn(0x7f86340d1230 msgr2=0x7f86340d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:24.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.382+0000 7f8633fff640 1 --2- 172.21.15.115:0/593022630 >> v2:172.21.15.142:3300/0 conn(0x7f86340d1230 0x7f86340d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:24.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.382+0000 7f8633fff640 1 -- 172.21.15.115:0/593022630 >> v1:172.21.15.142:6789/0 conn(0x7f863415cd20 legacy=0x7f86340b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:24.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.382+0000 7f8633fff640 1 -- 172.21.15.115:0/593022630 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86340ce2e0 con 0x7f86340b5d80 2023-12-01T03:11:24.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.382+0000 7f863938d640 1 --2- 172.21.15.115:0/593022630 >> v2:172.21.15.142:3300/0 conn(0x7f86340d1230 0x7f86340d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.383+0000 7f8633fff640 1 --2- 172.21.15.115:0/593022630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340b5d80 0x7f86340d24d0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f86340c6720 tx=0x7f862402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b75dad59f1ec31 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.383+0000 7f86337fe640 1 -- 172.21.15.115:0/593022630 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862402c9f0 con 0x7f86340b5d80 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.383+0000 7f86337fe640 1 -- 172.21.15.115:0/593022630 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862402cb50 con 0x7f86340b5d80 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.383+0000 7f86337fe640 1 -- 172.21.15.115:0/593022630 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86240343f0 con 0x7f86340b5d80 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.384+0000 7f863adf0640 1 -- 172.21.15.115:0/593022630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340b5d80 msgr2=0x7f86340d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.384+0000 7f863adf0640 1 --2- 172.21.15.115:0/593022630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340b5d80 0x7f86340d24d0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f86340c6720 tx=0x7f862402be50 comp rx=0 tx=0).stop 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.384+0000 7f863adf0640 1 -- 172.21.15.115:0/593022630 shutdown_connections 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.384+0000 7f863adf0640 1 --2- 172.21.15.115:0/593022630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340b5d80 0x7f86340d24d0 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.384+0000 7f863adf0640 1 --2- 172.21.15.115:0/593022630 >> v2:172.21.15.142:3300/0 conn(0x7f86340d1230 0x7f86340d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.384+0000 7f863adf0640 1 -- 172.21.15.115:0/593022630 >> 172.21.15.115:0/593022630 conn(0x7f863406a0e0 msgr2=0x7f863406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.384+0000 7f863adf0640 1 -- 172.21.15.115:0/593022630 shutdown_connections 2023-12-01T03:11:24.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.384+0000 7f863adf0640 1 -- 172.21.15.115:0/593022630 wait complete. 2023-12-01T03:11:24.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.386+0000 7f863adf0640 1 Processor -- start 2023-12-01T03:11:24.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.386+0000 7f863adf0640 1 -- start start 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.387+0000 7f863adf0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86340b5d80 0x7f86341f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.387+0000 7f863adf0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340d1230 0x7f8634156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.387+0000 7f863adf0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f863415cd20 0x7f86340cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.387+0000 7f863adf0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86341f3ac0 con 0x7f86340d1230 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.387+0000 7f863adf0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86340b25b0 con 0x7f86340b5d80 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.387+0000 7f863adf0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8634156760 con 0x7f863415cd20 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.387+0000 7f8633fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340d1230 0x7f8634156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.387+0000 7f8633fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340d1230 0x7f8634156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47046/0 (socket says 172.21.15.115:47046) 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.387+0000 7f8633fff640 1 -- 172.21.15.115:0/239220150 learned_addr learned my addr 172.21.15.115:0/239220150 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.388+0000 7f8638b8c640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86340b5d80 0x7f86341f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.388+0000 7f863938d640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f863415cd20 0x7f86340cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:24.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.388+0000 7f8633fff640 1 -- 172.21.15.115:0/239220150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f863415cd20 msgr2=0x7f86340cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:24.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.388+0000 7f8633fff640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f863415cd20 0x7f86340cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:24.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.388+0000 7f8633fff640 1 -- 172.21.15.115:0/239220150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86340b5d80 msgr2=0x7f86341f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:24.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.388+0000 7f8633fff640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86340b5d80 0x7f86341f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:24.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.388+0000 7f8633fff640 1 -- 172.21.15.115:0/239220150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8624009b30 con 0x7f86340d1230 2023-12-01T03:11:24.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.388+0000 7f8638b8c640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86340b5d80 0x7f86341f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:24.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.388+0000 7f8633fff640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340d1230 0x7f8634156220 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f8624000f80 tx=0x7f8624004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:24.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.389+0000 7f86317fa640 1 -- 172.21.15.115:0/239220150 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862403e450 con 0x7f86340d1230 2023-12-01T03:11:24.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.389+0000 7f86317fa640 1 -- 172.21.15.115:0/239220150 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862403e5f0 con 0x7f86340d1230 2023-12-01T03:11:24.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.389+0000 7f86317fa640 1 -- 172.21.15.115:0/239220150 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862403e9a0 con 0x7f86340d1230 2023-12-01T03:11:24.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.389+0000 7f863adf0640 1 -- 172.21.15.115:0/239220150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86340b2800 con 0x7f86340d1230 2023-12-01T03:11:24.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.389+0000 7f863adf0640 1 -- 172.21.15.115:0/239220150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86340b2dc0 con 0x7f86340d1230 2023-12-01T03:11:24.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.390+0000 7f863adf0640 1 -- 172.21.15.115:0/239220150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86340d2740 con 0x7f86340d1230 2023-12-01T03:11:24.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.391+0000 7f86317fa640 1 -- 172.21.15.115:0/239220150 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f862403eb00 con 0x7f86340d1230 2023-12-01T03:11:24.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.392+0000 7f86317fa640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f861406a670 0x7f861406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:24.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.392+0000 7f86317fa640 1 -- 172.21.15.115:0/239220150 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f86240abb10 con 0x7f86340d1230 2023-12-01T03:11:24.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.392+0000 7f8638b8c640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f861406a670 0x7f861406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:24.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.393+0000 7f8638b8c640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f861406a670 0x7f861406cb30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f861c00a830 tx=0x7f861c008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:24.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.396+0000 7f86317fa640 1 -- 172.21.15.115:0/239220150 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8624076d30 con 0x7f86340d1230 2023-12-01T03:11:24.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:24.661+0000 7f863adf0640 1 -- 172.21.15.115:0/239220150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7f86340d2990 con 0x7f86340d1230 2023-12-01T03:11:25.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.148+0000 7f86317fa640 1 -- 172.21.15.115:0/239220150 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3116) v1 ==== 190+0+0 (secure 0 0 0) 0x7f862403edb0 con 0x7f86340d1230 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 -- 172.21.15.115:0/239220150 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f861406a670 msgr2=0x7f861406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f861406a670 0x7f861406cb30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f861c00a830 tx=0x7f861c008040 comp rx=0 tx=0).stop 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 -- 172.21.15.115:0/239220150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340d1230 msgr2=0x7f8634156220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340d1230 0x7f8634156220 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f8624000f80 tx=0x7f8624004a40 comp rx=0 tx=0).stop 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 -- 172.21.15.115:0/239220150 shutdown_connections 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f861406a670 0x7f861406cb30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f863415cd20 0x7f86340cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86340d1230 0x7f8634156220 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 --2- 172.21.15.115:0/239220150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86340b5d80 0x7f86341f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 -- 172.21.15.115:0/239220150 >> 172.21.15.115:0/239220150 conn(0x7f863406a0e0 msgr2=0x7f86340c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 -- 172.21.15.115:0/239220150 shutdown_connections 2023-12-01T03:11:25.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.152+0000 7f863adf0640 1 -- 172.21.15.115:0/239220150 wait complete. 2023-12-01T03:11:25.173 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T03:11:25.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.388+0000 7f3729720640 1 Processor -- start 2023-12-01T03:11:25.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.388+0000 7f3729720640 1 -- start start 2023-12-01T03:11:25.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.390+0000 7f3729720640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3724142870 0x7f37240ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:25.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.390+0000 7f3729720640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f37240cd010 0x7f37240b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:25.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.390+0000 7f3729720640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37240b2f30 con 0x7f37240b1c50 2023-12-01T03:11:25.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.390+0000 7f3729720640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37240b3ae0 con 0x7f37240cd010 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.390+0000 7f3729720640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37240cd4a0 con 0x7f3724142870 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.390+0000 7f37227fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f37240cd010 0x7f37240b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.390+0000 7f37227fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f37240cd010 0x7f37240b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47056/0 (socket says 172.21.15.115:47056) 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.390+0000 7f37227fc640 1 -- 172.21.15.115:0/2222226251 learned_addr learned my addr 172.21.15.115:0/2222226251 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.390+0000 7f37237fe640 1 --1- 172.21.15.115:0/2222226251 >> v1:172.21.15.115:6789/0 conn(0x7f37240b1c50 0x7f372415d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46440/0 (socket says 172.21.15.115:46440) 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.390+0000 7f3722ffd640 1 --2- 172.21.15.115:0/2222226251 >> v2:172.21.15.142:3300/0 conn(0x7f3724142870 0x7f37240ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.391+0000 7f37227fc640 1 -- 172.21.15.115:0/2222226251 >> v2:172.21.15.142:3300/0 conn(0x7f3724142870 msgr2=0x7f37240ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.391+0000 7f37227fc640 1 --2- 172.21.15.115:0/2222226251 >> v2:172.21.15.142:3300/0 conn(0x7f3724142870 0x7f37240ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.391+0000 7f37227fc640 1 -- 172.21.15.115:0/2222226251 >> v1:172.21.15.115:6789/0 conn(0x7f37240b1c50 legacy=0x7f372415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.391+0000 7f37227fc640 1 -- 172.21.15.115:0/2222226251 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240b20c0 con 0x7f37240cd010 2023-12-01T03:11:25.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.391+0000 7f3721ffb640 1 -- 172.21.15.115:0/2222226251 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912335287 0 0) 0x7f37240b2f30 con 0x7f37240b1c50 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.392+0000 7f37227fc640 1 --2- 172.21.15.115:0/2222226251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 0x7f37240b1710 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f37240b14d0 tx=0x7f371802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b8037872f8b3b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.392+0000 7f3721ffb640 1 -- 172.21.15.115:0/2222226251 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718030860 con 0x7f37240cd010 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.392+0000 7f3721ffb640 1 -- 172.21.15.115:0/2222226251 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37180309c0 con 0x7f37240cd010 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.392+0000 7f3721ffb640 1 -- 172.21.15.115:0/2222226251 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718030d50 con 0x7f37240cd010 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.393+0000 7f3729720640 1 -- 172.21.15.115:0/2222226251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 msgr2=0x7f37240b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.393+0000 7f3729720640 1 --2- 172.21.15.115:0/2222226251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 0x7f37240b1710 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f37240b14d0 tx=0x7f371802b9a0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.393+0000 7f3729720640 1 -- 172.21.15.115:0/2222226251 shutdown_connections 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.393+0000 7f3729720640 1 --2- 172.21.15.115:0/2222226251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 0x7f37240b1710 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.393+0000 7f3729720640 1 --2- 172.21.15.115:0/2222226251 >> v2:172.21.15.142:3300/0 conn(0x7f3724142870 0x7f37240ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.393+0000 7f3729720640 1 -- 172.21.15.115:0/2222226251 >> 172.21.15.115:0/2222226251 conn(0x7f3724065960 msgr2=0x7f3724065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.393+0000 7f3729720640 1 -- 172.21.15.115:0/2222226251 shutdown_connections 2023-12-01T03:11:25.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.393+0000 7f3729720640 1 -- 172.21.15.115:0/2222226251 wait complete. 2023-12-01T03:11:25.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.395+0000 7f3729720640 1 Processor -- start 2023-12-01T03:11:25.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.395+0000 7f3729720640 1 -- start start 2023-12-01T03:11:25.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f3729720640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37240b1c50 0x7f372413bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:25.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f3729720640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 0x7f372413e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f3729720640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3724142870 0x7f372406f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f3729720640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372413ed70 con 0x7f37240cd010 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f3729720640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37241f2300 con 0x7f37240b1c50 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f3729720640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37241f25e0 con 0x7f3724142870 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f37227fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 0x7f372413e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f37227fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 0x7f372413e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47062/0 (socket says 172.21.15.115:47062) 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f37227fc640 1 -- 172.21.15.115:0/1715069078 learned_addr learned my addr 172.21.15.115:0/1715069078 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f3722ffd640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37240b1c50 0x7f372413bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.396+0000 7f37237fe640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3724142870 0x7f372406f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f37227fc640 1 -- 172.21.15.115:0/1715069078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3724142870 msgr2=0x7f372406f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f37227fc640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3724142870 0x7f372406f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f37227fc640 1 -- 172.21.15.115:0/1715069078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37240b1c50 msgr2=0x7f372413bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f37227fc640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37240b1c50 0x7f372413bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f37227fc640 1 -- 172.21.15.115:0/1715069078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37180094d0 con 0x7f37240cd010 2023-12-01T03:11:25.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f37237fe640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3724142870 0x7f372406f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:25.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f37227fc640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 0x7f372413e270 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f371802b9e0 tx=0x7f3718004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:25.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f370b7fe640 1 -- 172.21.15.115:0/1715069078 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718039d60 con 0x7f37240cd010 2023-12-01T03:11:25.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f370b7fe640 1 -- 172.21.15.115:0/1715069078 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3718043400 con 0x7f37240cd010 2023-12-01T03:11:25.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f370b7fe640 1 -- 172.21.15.115:0/1715069078 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37180437b0 con 0x7f37240cd010 2023-12-01T03:11:25.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f3729720640 1 -- 172.21.15.115:0/1715069078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37241f28c0 con 0x7f37240cd010 2023-12-01T03:11:25.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.397+0000 7f3729720640 1 -- 172.21.15.115:0/1715069078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372413f050 con 0x7f37240cd010 2023-12-01T03:11:25.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.399+0000 7f3729720640 1 -- 172.21.15.115:0/1715069078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372404e890 con 0x7f37240cd010 2023-12-01T03:11:25.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.399+0000 7f370b7fe640 1 -- 172.21.15.115:0/1715069078 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f3718043910 con 0x7f37240cd010 2023-12-01T03:11:25.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.400+0000 7f370b7fe640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f371006a670 0x7f371006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:25.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.400+0000 7f370b7fe640 1 -- 172.21.15.115:0/1715069078 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f371803e070 con 0x7f37240cd010 2023-12-01T03:11:25.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.400+0000 7f3722ffd640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f371006a670 0x7f371006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:25.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.401+0000 7f3722ffd640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f371006a670 0x7f371006cb30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3714006fd0 tx=0x7f3714008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:25.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.405+0000 7f370b7fe640 1 -- 172.21.15.115:0/1715069078 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3718077180 con 0x7f37240cd010 2023-12-01T03:11:25.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.585+0000 7f3729720640 1 -- 172.21.15.115:0/1715069078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f372406b6d0 con 0x7f37240cd010 2023-12-01T03:11:25.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.595+0000 7f370b7fe640 1 -- 172.21.15.115:0/1715069078 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v76)=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f37180340c0 con 0x7f37240cd010 2023-12-01T03:11:25.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.597+0000 7f3729720640 1 -- 172.21.15.115:0/1715069078 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f371006a670 msgr2=0x7f371006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:25.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.597+0000 7f3729720640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f371006a670 0x7f371006cb30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3714006fd0 tx=0x7f3714008040 comp rx=0 tx=0).stop 2023-12-01T03:11:25.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.598+0000 7f3729720640 1 -- 172.21.15.115:0/1715069078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 msgr2=0x7f372413e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:25.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.598+0000 7f3729720640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 0x7f372413e270 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f371802b9e0 tx=0x7f3718004730 comp rx=0 tx=0).stop 2023-12-01T03:11:25.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.598+0000 7f3729720640 1 -- 172.21.15.115:0/1715069078 shutdown_connections 2023-12-01T03:11:25.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.598+0000 7f3729720640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f371006a670 0x7f371006cb30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.598+0000 7f3729720640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3724142870 0x7f372406f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.598+0000 7f3729720640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37240cd010 0x7f372413e270 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.598+0000 7f3729720640 1 --2- 172.21.15.115:0/1715069078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37240b1c50 0x7f372413bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.598+0000 7f3729720640 1 -- 172.21.15.115:0/1715069078 >> 172.21.15.115:0/1715069078 conn(0x7f3724065960 msgr2=0x7f372413cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.598+0000 7f3729720640 1 -- 172.21.15.115:0/1715069078 shutdown_connections 2023-12-01T03:11:25.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:25.598+0000 7f3729720640 1 -- 172.21.15.115:0/1715069078 wait complete. 2023-12-01T03:11:25.617 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T03:11:25.617 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:11:25.732 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:11:25.852 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T03:11:25.852 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:11:25.853 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:11:25.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:11:25.912 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:11:25.912 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:11:25.971 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:11:25.971 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:11:25.972 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:11:26.032 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:11:26.032 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:11:26.032 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:11:26.092 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:11:26.092 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T03:11:26.092 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:26.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.312+0000 7f4232b20640 1 Processor -- start 2023-12-01T03:11:26.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.312+0000 7f4232b20640 1 -- start start 2023-12-01T03:11:26.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.313+0000 7f4232b20640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f422c160470 0x7f422c06a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.313+0000 7f4232b20640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f422c0b2e60 0x7f422c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.313+0000 7f4232b20640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c069ed0 con 0x7f422c0cb860 2023-12-01T03:11:26.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.313+0000 7f4232b20640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c06a400 con 0x7f422c160470 2023-12-01T03:11:26.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.313+0000 7f4232b20640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c06a120 con 0x7f422c0b2e60 2023-12-01T03:11:26.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.313+0000 7f42308bc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f422c160470 0x7f422c06a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:26.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.313+0000 7f42308bc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f422c160470 0x7f422c06a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47072/0 (socket says 172.21.15.115:47072) 2023-12-01T03:11:26.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.313+0000 7f42308bc640 1 -- 172.21.15.115:0/2273212214 learned_addr learned my addr 172.21.15.115:0/2273212214 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:26.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.314+0000 7f42308bc640 1 -- 172.21.15.115:0/2273212214 >> v2:172.21.15.142:3300/0 conn(0x7f422c0b2e60 msgr2=0x7f422c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:26.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.314+0000 7f42308bc640 1 --2- 172.21.15.115:0/2273212214 >> v2:172.21.15.142:3300/0 conn(0x7f422c0b2e60 0x7f422c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.314+0000 7f42308bc640 1 -- 172.21.15.115:0/2273212214 >> v1:172.21.15.142:6789/0 conn(0x7f422c0cb860 legacy=0x7f422c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:26.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.314+0000 7f42308bc640 1 -- 172.21.15.115:0/2273212214 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0cb060 con 0x7f422c160470 2023-12-01T03:11:26.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.315+0000 7f42308bc640 1 --2- 172.21.15.115:0/2273212214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c160470 0x7f422c06a6d0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f422c06fc90 tx=0x7f421402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588664f55c8430de server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:26.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.315+0000 7f422b7fe640 1 -- 172.21.15.115:0/2273212214 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214031870 con 0x7f422c160470 2023-12-01T03:11:26.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.315+0000 7f422b7fe640 1 -- 172.21.15.115:0/2273212214 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42140319d0 con 0x7f422c160470 2023-12-01T03:11:26.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.315+0000 7f422b7fe640 1 -- 172.21.15.115:0/2273212214 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214031d60 con 0x7f422c160470 2023-12-01T03:11:26.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.316+0000 7f4232b20640 1 -- 172.21.15.115:0/2273212214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c160470 msgr2=0x7f422c06a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:26.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.316+0000 7f4232b20640 1 --2- 172.21.15.115:0/2273212214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c160470 0x7f422c06a6d0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f422c06fc90 tx=0x7f421402b910 comp rx=0 tx=0).stop 2023-12-01T03:11:26.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.316+0000 7f4232b20640 1 -- 172.21.15.115:0/2273212214 shutdown_connections 2023-12-01T03:11:26.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.316+0000 7f4232b20640 1 --2- 172.21.15.115:0/2273212214 >> v2:172.21.15.142:3300/0 conn(0x7f422c0b2e60 0x7f422c06ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.316+0000 7f4232b20640 1 --2- 172.21.15.115:0/2273212214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c160470 0x7f422c06a6d0 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.316+0000 7f4232b20640 1 -- 172.21.15.115:0/2273212214 >> 172.21.15.115:0/2273212214 conn(0x7f422c0d0a00 msgr2=0x7f422c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:26.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.316+0000 7f4232b20640 1 -- 172.21.15.115:0/2273212214 shutdown_connections 2023-12-01T03:11:26.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.316+0000 7f4232b20640 1 -- 172.21.15.115:0/2273212214 wait complete. 2023-12-01T03:11:26.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.318+0000 7f4232b20640 1 Processor -- start 2023-12-01T03:11:26.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.318+0000 7f4232b20640 1 -- start start 2023-12-01T03:11:26.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f4232b20640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f422c0b2e60 0x7f422c0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f4232b20640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c0cb860 0x7f422c14ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f4232b20640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f422c160470 0x7f422c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f4232b20640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c06e090 con 0x7f422c0cb860 2023-12-01T03:11:26.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f4232b20640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c14b3a0 con 0x7f422c160470 2023-12-01T03:11:26.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f4232b20640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0caaa0 con 0x7f422c0b2e60 2023-12-01T03:11:26.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f422bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c0cb860 0x7f422c14ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:26.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f422bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c0cb860 0x7f422c14ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47084/0 (socket says 172.21.15.115:47084) 2023-12-01T03:11:26.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f422bfff640 1 -- 172.21.15.115:0/188801542 learned_addr learned my addr 172.21.15.115:0/188801542 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:26.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f422bfff640 1 -- 172.21.15.115:0/188801542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f422c0b2e60 msgr2=0x7f422c0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:26.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f422bfff640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f422c0b2e60 0x7f422c0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f422bfff640 1 -- 172.21.15.115:0/188801542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f422c160470 msgr2=0x7f422c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:26.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f422bfff640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f422c160470 0x7f422c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.319+0000 7f422bfff640 1 -- 172.21.15.115:0/188801542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4214009470 con 0x7f422c0cb860 2023-12-01T03:11:26.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.320+0000 7f422bfff640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c0cb860 0x7f422c14ae60 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f421c00b9c0 tx=0x7f421c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:26.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.320+0000 7f42297fa640 1 -- 172.21.15.115:0/188801542 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c00cba0 con 0x7f422c0cb860 2023-12-01T03:11:26.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.320+0000 7f42297fa640 1 -- 172.21.15.115:0/188801542 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c00cd20 con 0x7f422c0cb860 2023-12-01T03:11:26.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.320+0000 7f42297fa640 1 -- 172.21.15.115:0/188801542 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c014680 con 0x7f422c0cb860 2023-12-01T03:11:26.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.320+0000 7f4232b20640 1 -- 172.21.15.115:0/188801542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c0c9f20 con 0x7f422c0cb860 2023-12-01T03:11:26.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.320+0000 7f4232b20640 1 -- 172.21.15.115:0/188801542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c0ca7c0 con 0x7f422c0cb860 2023-12-01T03:11:26.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.321+0000 7f4232b20640 1 -- 172.21.15.115:0/188801542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c06be50 con 0x7f422c0cb860 2023-12-01T03:11:26.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.322+0000 7f42297fa640 1 -- 172.21.15.115:0/188801542 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f421c01d3b0 con 0x7f422c0cb860 2023-12-01T03:11:26.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.324+0000 7f42297fa640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f420c06a670 0x7f420c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.324+0000 7f42297fa640 1 -- 172.21.15.115:0/188801542 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f421c088e00 con 0x7f422c0cb860 2023-12-01T03:11:26.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.324+0000 7f42308bc640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f420c06a670 0x7f420c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:26.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.324+0000 7f42308bc640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f420c06a670 0x7f420c06cb30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4214036000 tx=0x7f4214030000 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:26.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.328+0000 7f42297fa640 1 -- 172.21.15.115:0/188801542 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f421c053f90 con 0x7f422c0cb860 2023-12-01T03:11:26.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.558+0000 7f4232b20640 1 -- 172.21.15.115:0/188801542 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f422c04e890 con 0x7f422c0cb860 2023-12-01T03:11:26.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.559+0000 7f42297fa640 1 -- 172.21.15.115:0/188801542 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+3545 (secure 0 0 0) 0x7f421c019020 con 0x7f422c0cb860 2023-12-01T03:11:26.561 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:26.561 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1584748728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":1584748728},{"type":"v1","addr":"172.21.15.142:6835","nonce":1584748728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":6797,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1762895106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1762895106},{"type":"v1","addr":"172.21.15.142:6834","nonce":1762895106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":103},{"gid":7330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6836/2203823530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":2203823530},{"type":"v1","addr":"172.21.15.115:6836","nonce":2203823530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":11394,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/4271245035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6835","nonce":4271245035},{"type":"v1","addr":"172.21.15.115:6837","nonce":4271245035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[]} 2023-12-01T03:11:26.561 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 105 2023-12-01T03:11:26.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 -- 172.21.15.115:0/188801542 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f420c06a670 msgr2=0x7f420c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:26.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f420c06a670 0x7f420c06cb30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4214036000 tx=0x7f4214030000 comp rx=0 tx=0).stop 2023-12-01T03:11:26.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 -- 172.21.15.115:0/188801542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c0cb860 msgr2=0x7f422c14ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:26.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c0cb860 0x7f422c14ae60 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f421c00b9c0 tx=0x7f421c00be80 comp rx=0 tx=0).stop 2023-12-01T03:11:26.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 -- 172.21.15.115:0/188801542 shutdown_connections 2023-12-01T03:11:26.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f420c06a670 0x7f420c06cb30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f422c160470 0x7f422c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f422c0cb860 0x7f422c14ae60 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 --2- 172.21.15.115:0/188801542 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f422c0b2e60 0x7f422c0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 -- 172.21.15.115:0/188801542 >> 172.21.15.115:0/188801542 conn(0x7f422c0d0a00 msgr2=0x7f422c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:26.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.561+0000 7f4232b20640 1 -- 172.21.15.115:0/188801542 shutdown_connections 2023-12-01T03:11:26.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.562+0000 7f4232b20640 1 -- 172.21.15.115:0/188801542 wait complete. 2023-12-01T03:11:26.583 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T03:11:26.583 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T03:11:26.583 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T03:11:26.583 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:11:26.583 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T03:11:26.583 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T03:11:26.584 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T03:11:26.584 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:11:26.584 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T03:11:26.584 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T03:11:26.584 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T03:11:26.584 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:11:26.584 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T03:11:26.584 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T03:11:26.585 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T03:11:26.585 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:11:26.603 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T03:11:26.603 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T03:11:26.613 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T03:11:26.613 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T03:11:26.615 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T03:11:26.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T03:11:26.616 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T03:11:26.616 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T03:11:26.662 INFO:tasks.ceph.mds.c:Started 2023-12-01T03:11:26.662 DEBUG:teuthology.parallel:result is None 2023-12-01T03:11:26.664 INFO:tasks.ceph.mds.b:Started 2023-12-01T03:11:26.665 DEBUG:teuthology.parallel:result is None 2023-12-01T03:11:26.669 INFO:tasks.ceph.mds.a:Started 2023-12-01T03:11:26.670 DEBUG:teuthology.parallel:result is None 2023-12-01T03:11:26.671 INFO:tasks.ceph.mds.d:Started 2023-12-01T03:11:26.672 DEBUG:teuthology.parallel:result is None 2023-12-01T03:11:26.672 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T03:11:26.824 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T03:11:26.827 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T03:11:26.832 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T03:11:26.844 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T03:11:26.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.861+0000 7efcd01fd640 1 Processor -- start 2023-12-01T03:11:26.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.862+0000 7efcd01fd640 1 -- start start 2023-12-01T03:11:26.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.863+0000 7efcd01fd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efcc8160470 0x7efcc806a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.863+0000 7efcd01fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 0x7efcc806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.863+0000 7efcd01fd640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc8069ed0 con 0x7efcc80b2e60 2023-12-01T03:11:26.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.863+0000 7efcd01fd640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc806a400 con 0x7efcc8160470 2023-12-01T03:11:26.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.863+0000 7efcd01fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc806a120 con 0x7efcc80cfcb0 2023-12-01T03:11:26.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.863+0000 7efccd798640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efcc80b2e60 0x7efcc80cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:46462/0 (socket says 172.21.15.115:46462) 2023-12-01T03:11:26.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.863+0000 7efccd798640 1 -- 172.21.15.115:0/855379574 learned_addr learned my addr 172.21.15.115:0/855379574 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:26.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.863+0000 7efccdf99640 1 --2- 172.21.15.115:0/855379574 >> v2:172.21.15.142:3300/0 conn(0x7efcc8160470 0x7efcc806a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:26.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.863+0000 7efcce79a640 1 --2- 172.21.15.115:0/855379574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 0x7efcc806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:26.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.864+0000 7efcce79a640 1 -- 172.21.15.115:0/855379574 >> v2:172.21.15.142:3300/0 conn(0x7efcc8160470 msgr2=0x7efcc806a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:26.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.864+0000 7efcce79a640 1 --2- 172.21.15.115:0/855379574 >> v2:172.21.15.142:3300/0 conn(0x7efcc8160470 0x7efcc806a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.864+0000 7efcce79a640 1 -- 172.21.15.115:0/855379574 >> v1:172.21.15.115:6789/0 conn(0x7efcc80b2e60 legacy=0x7efcc80cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:26.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.864+0000 7efcce79a640 1 -- 172.21.15.115:0/855379574 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc80cb060 con 0x7efcc80cfcb0 2023-12-01T03:11:26.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.865+0000 7efcce79a640 1 --2- 172.21.15.115:0/855379574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 0x7efcc806ac10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009d90 tx=0x7efcc002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fef5db3d825ed18 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:26.867 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.865+0000 7efcccf97640 1 -- 172.21.15.115:0/855379574 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc002cc60 con 0x7efcc80cfcb0 2023-12-01T03:11:26.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.872+0000 7efcccf97640 1 -- 172.21.15.115:0/855379574 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc002cdc0 con 0x7efcc80cfcb0 2023-12-01T03:11:26.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.872+0000 7efcccf97640 1 -- 172.21.15.115:0/855379574 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0034660 con 0x7efcc80cfcb0 2023-12-01T03:11:26.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.873+0000 7efcd01fd640 1 -- 172.21.15.115:0/855379574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 msgr2=0x7efcc806ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:26.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.873+0000 7efcd01fd640 1 --2- 172.21.15.115:0/855379574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 0x7efcc806ac10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009d90 tx=0x7efcc002beb0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.873+0000 7efcd01fd640 1 -- 172.21.15.115:0/855379574 shutdown_connections 2023-12-01T03:11:26.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.873+0000 7efcd01fd640 1 --2- 172.21.15.115:0/855379574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 0x7efcc806ac10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.873+0000 7efcd01fd640 1 --2- 172.21.15.115:0/855379574 >> v2:172.21.15.142:3300/0 conn(0x7efcc8160470 0x7efcc806a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.873+0000 7efcd01fd640 1 -- 172.21.15.115:0/855379574 >> 172.21.15.115:0/855379574 conn(0x7efcc80d0a00 msgr2=0x7efcc80d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:26.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.873+0000 7efcd01fd640 1 -- 172.21.15.115:0/855379574 shutdown_connections 2023-12-01T03:11:26.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.873+0000 7efcd01fd640 1 -- 172.21.15.115:0/855379574 wait complete. 2023-12-01T03:11:26.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.875+0000 7efcd01fd640 1 Processor -- start 2023-12-01T03:11:26.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.875+0000 7efcd01fd640 1 -- start start 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.876+0000 7efcd01fd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc80b2e60 0x7efcc8159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.876+0000 7efcd01fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 0x7efcc806e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.876+0000 7efcd01fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8160470 0x7efcc80c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.876+0000 7efcd01fd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc815a9a0 con 0x7efcc8160470 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.876+0000 7efcd01fd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80b61a0 con 0x7efcc80b2e60 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.876+0000 7efcd01fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80b6480 con 0x7efcc80cfcb0 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.876+0000 7efcce79a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8160470 0x7efcc80c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.876+0000 7efcce79a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8160470 0x7efcc80c4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47124/0 (socket says 172.21.15.115:47124) 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.877+0000 7efcce79a640 1 -- 172.21.15.115:0/3840669455 learned_addr learned my addr 172.21.15.115:0/3840669455 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.877+0000 7efccdf99640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc80b2e60 0x7efcc8159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.877+0000 7efccd798640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 0x7efcc806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.877+0000 7efcce79a640 1 -- 172.21.15.115:0/3840669455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 msgr2=0x7efcc806e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:26.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.877+0000 7efcce79a640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 0x7efcc806e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.877+0000 7efcce79a640 1 -- 172.21.15.115:0/3840669455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc80b2e60 msgr2=0x7efcc8159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:26.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.877+0000 7efcce79a640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc80b2e60 0x7efcc8159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:26.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.877+0000 7efcce79a640 1 -- 172.21.15.115:0/3840669455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0009a70 con 0x7efcc8160470 2023-12-01T03:11:26.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.877+0000 7efcce79a640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8160470 0x7efcc80c4480 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009eb0 tx=0x7efcc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:26.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.878+0000 7efcbe7fc640 1 -- 172.21.15.115:0/3840669455 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc003e610 con 0x7efcc8160470 2023-12-01T03:11:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.878+0000 7efcbe7fc640 1 -- 172.21.15.115:0/3840669455 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc003e770 con 0x7efcc8160470 2023-12-01T03:11:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.878+0000 7efcbe7fc640 1 -- 172.21.15.115:0/3840669455 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc003eb60 con 0x7efcc8160470 2023-12-01T03:11:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.878+0000 7efcd01fd640 1 -- 172.21.15.115:0/3840669455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc80ca920 con 0x7efcc8160470 2023-12-01T03:11:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.878+0000 7efcd01fd640 1 -- 172.21.15.115:0/3840669455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc8159d60 con 0x7efcc8160470 2023-12-01T03:11:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.879+0000 7efcd01fd640 1 -- 172.21.15.115:0/3840669455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc80cae40 con 0x7efcc8160470 2023-12-01T03:11:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.880+0000 7efcbe7fc640 1 -- 172.21.15.115:0/3840669455 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7efcc003b050 con 0x7efcc8160470 2023-12-01T03:11:26.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.881+0000 7efcbe7fc640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efcac06a670 0x7efcac06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:26.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.881+0000 7efcbe7fc640 1 -- 172.21.15.115:0/3840669455 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5804+0+0 (secure 0 0 0) 0x7efcc00aae60 con 0x7efcc8160470 2023-12-01T03:11:26.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.881+0000 7efccdf99640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efcac06a670 0x7efcac06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:26.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.882+0000 7efccdf99640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efcac06a670 0x7efcac06cb30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efcc80d03b0 tx=0x7efcc4009110 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:26.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:26.885+0000 7efcbe7fc640 1 -- 172.21.15.115:0/3840669455 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efcc80cae40 con 0x7efcc8160470 2023-12-01T03:11:27.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.090+0000 7efcd01fd640 1 -- 172.21.15.115:0/3840669455 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7efcc80cae40 con 0x7efcc8160470 2023-12-01T03:11:27.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.884+0000 7efcbe7fc640 1 -- 172.21.15.115:0/3840669455 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v231) v1 ==== 97+0+0 (secure 0 0 0) 0x7efcc80cae40 con 0x7efcc8160470 2023-12-01T03:11:27.886 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T03:11:27.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.887+0000 7efcd01fd640 1 -- 172.21.15.115:0/3840669455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efcac06a670 msgr2=0x7efcac06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:27.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.887+0000 7efcd01fd640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efcac06a670 0x7efcac06cb30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efcc80d03b0 tx=0x7efcc4009110 comp rx=0 tx=0).stop 2023-12-01T03:11:27.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.887+0000 7efcd01fd640 1 -- 172.21.15.115:0/3840669455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8160470 msgr2=0x7efcc80c4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:27.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.887+0000 7efcd01fd640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8160470 0x7efcc80c4480 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009eb0 tx=0x7efcc0004a40 comp rx=0 tx=0).stop 2023-12-01T03:11:27.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.888+0000 7efcd01fd640 1 -- 172.21.15.115:0/3840669455 shutdown_connections 2023-12-01T03:11:27.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.888+0000 7efcd01fd640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efcac06a670 0x7efcac06cb30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:27.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.888+0000 7efcd01fd640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efcc80cfcb0 0x7efcc806e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:27.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.888+0000 7efcd01fd640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efcc8160470 0x7efcc80c4480 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:27.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.888+0000 7efcd01fd640 1 --2- 172.21.15.115:0/3840669455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcc80b2e60 0x7efcc8159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:27.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.888+0000 7efcd01fd640 1 -- 172.21.15.115:0/3840669455 >> 172.21.15.115:0/3840669455 conn(0x7efcc80d0a00 msgr2=0x7efcc815ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:27.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.888+0000 7efcd01fd640 1 -- 172.21.15.115:0/3840669455 shutdown_connections 2023-12-01T03:11:27.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:27.888+0000 7efcd01fd640 1 -- 172.21.15.115:0/3840669455 wait complete. 2023-12-01T03:11:27.909 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T03:11:28.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.125+0000 7f13cf7f0640 1 Processor -- start 2023-12-01T03:11:28.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.125+0000 7f13cf7f0640 1 -- start start 2023-12-01T03:11:28.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.126+0000 7f13cf7f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c815cd20 0x7f13c80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.126+0000 7f13cf7f0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f13c80d1230 0x7f13c80d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.126+0000 7f13cf7f0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80ce2e0 con 0x7f13c80b5d80 2023-12-01T03:11:28.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.126+0000 7f13cf7f0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c80b2160 con 0x7f13c80d1230 2023-12-01T03:11:28.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.126+0000 7f13cf7f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c80d1be0 con 0x7f13c815cd20 2023-12-01T03:11:28.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.127+0000 7f13cdd8d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f13c80d1230 0x7f13c80d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.127+0000 7f13cdd8d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f13c80d1230 0x7f13c80d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38628/0 (socket says 172.21.15.115:38628) 2023-12-01T03:11:28.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.127+0000 7f13cdd8d640 1 -- 172.21.15.115:0/2288724860 learned_addr learned my addr 172.21.15.115:0/2288724860 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:28.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.127+0000 7f13cdd8d640 1 -- 172.21.15.115:0/2288724860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c815cd20 msgr2=0x7f13c80d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:28.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.127+0000 7f13cd58c640 1 --2- 172.21.15.115:0/2288724860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c815cd20 0x7f13c80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.127+0000 7f13cdd8d640 1 --2- 172.21.15.115:0/2288724860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c815cd20 0x7f13c80d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.127+0000 7f13cdd8d640 1 -- 172.21.15.115:0/2288724860 >> v1:172.21.15.142:6789/0 conn(0x7f13c80b5d80 legacy=0x7f13c80b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:28.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.127+0000 7f13cdd8d640 1 -- 172.21.15.115:0/2288724860 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80ce000 con 0x7f13c80d1230 2023-12-01T03:11:28.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.127+0000 7f13cd58c640 1 --2- 172.21.15.115:0/2288724860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c815cd20 0x7f13c80d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:28.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.128+0000 7f13cdd8d640 1 --2- 172.21.15.115:0/2288724860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c80d1230 0x7f13c80d16a0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009e50 tx=0x7f13c002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46406182c08345e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:28.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.129+0000 7f13bffff640 1 -- 172.21.15.115:0/2288724860 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c002c9f0 con 0x7f13c80d1230 2023-12-01T03:11:28.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.129+0000 7f13bffff640 1 -- 172.21.15.115:0/2288724860 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c002cb50 con 0x7f13c80d1230 2023-12-01T03:11:28.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.129+0000 7f13bffff640 1 -- 172.21.15.115:0/2288724860 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00343f0 con 0x7f13c80d1230 2023-12-01T03:11:28.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.129+0000 7f13cf7f0640 1 -- 172.21.15.115:0/2288724860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c80d1230 msgr2=0x7f13c80d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.129+0000 7f13cf7f0640 1 --2- 172.21.15.115:0/2288724860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c80d1230 0x7f13c80d16a0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009e50 tx=0x7f13c002be50 comp rx=0 tx=0).stop 2023-12-01T03:11:28.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.130+0000 7f13cf7f0640 1 -- 172.21.15.115:0/2288724860 shutdown_connections 2023-12-01T03:11:28.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.130+0000 7f13cf7f0640 1 --2- 172.21.15.115:0/2288724860 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c80d1230 0x7f13c80d16a0 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.130+0000 7f13cf7f0640 1 --2- 172.21.15.115:0/2288724860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c815cd20 0x7f13c80d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.130+0000 7f13cf7f0640 1 -- 172.21.15.115:0/2288724860 >> 172.21.15.115:0/2288724860 conn(0x7f13c806a0e0 msgr2=0x7f13c806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:28.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.130+0000 7f13cf7f0640 1 -- 172.21.15.115:0/2288724860 shutdown_connections 2023-12-01T03:11:28.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.130+0000 7f13cf7f0640 1 -- 172.21.15.115:0/2288724860 wait complete. 2023-12-01T03:11:28.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.131+0000 7f13cf7f0640 1 Processor -- start 2023-12-01T03:11:28.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.132+0000 7f13cf7f0640 1 -- start start 2023-12-01T03:11:28.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.132+0000 7f13cf7f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13c80b5d80 0x7f13c81f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.132+0000 7f13cf7f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c80d1230 0x7f13c81449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cf7f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c815cd20 0x7f13c8144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cf7f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c80cc650 con 0x7f13c815cd20 2023-12-01T03:11:28.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cf7f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c80cb780 con 0x7f13c80b5d80 2023-12-01T03:11:28.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cf7f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c80cb9d0 con 0x7f13c80d1230 2023-12-01T03:11:28.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cdd8d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c815cd20 0x7f13c8144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cdd8d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c815cd20 0x7f13c8144f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38632/0 (socket says 172.21.15.115:38632) 2023-12-01T03:11:28.135 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cdd8d640 1 -- 172.21.15.115:0/4271916961 learned_addr learned my addr 172.21.15.115:0/4271916961 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:28.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cd58c640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13c80b5d80 0x7f13c81f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13ccd8b640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c80d1230 0x7f13c81449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cdd8d640 1 -- 172.21.15.115:0/4271916961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c80d1230 msgr2=0x7f13c81449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cdd8d640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c80d1230 0x7f13c81449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cdd8d640 1 -- 172.21.15.115:0/4271916961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13c80b5d80 msgr2=0x7f13c81f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cdd8d640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13c80b5d80 0x7f13c81f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.133+0000 7f13cdd8d640 1 -- 172.21.15.115:0/4271916961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c0009b30 con 0x7f13c815cd20 2023-12-01T03:11:28.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.134+0000 7f13cdd8d640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c815cd20 0x7f13c8144f00 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f13c0036040 tx=0x7f13c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:28.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.134+0000 7f13bdffb640 1 -- 172.21.15.115:0/4271916961 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c003e450 con 0x7f13c815cd20 2023-12-01T03:11:28.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.134+0000 7f13bdffb640 1 -- 172.21.15.115:0/4271916961 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c003e5d0 con 0x7f13c815cd20 2023-12-01T03:11:28.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.134+0000 7f13bdffb640 1 -- 172.21.15.115:0/4271916961 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c003e980 con 0x7f13c815cd20 2023-12-01T03:11:28.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.134+0000 7f13cf7f0640 1 -- 172.21.15.115:0/4271916961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c80cbcb0 con 0x7f13c815cd20 2023-12-01T03:11:28.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.134+0000 7f13cf7f0640 1 -- 172.21.15.115:0/4271916961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c80cc270 con 0x7f13c815cd20 2023-12-01T03:11:28.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.135+0000 7f13cf7f0640 1 -- 172.21.15.115:0/4271916961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c804e890 con 0x7f13c815cd20 2023-12-01T03:11:28.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.136+0000 7f13bdffb640 1 -- 172.21.15.115:0/4271916961 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f13c003eae0 con 0x7f13c815cd20 2023-12-01T03:11:28.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.137+0000 7f13bdffb640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f13ac06a670 0x7f13ac06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.137+0000 7f13bdffb640 1 -- 172.21.15.115:0/4271916961 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f13c00ab990 con 0x7f13c815cd20 2023-12-01T03:11:28.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.137+0000 7f13cd58c640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f13ac06a670 0x7f13ac06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.138+0000 7f13cd58c640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f13ac06a670 0x7f13ac06cb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f13c400a950 tx=0x7f13c4008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:28.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.142+0000 7f13bdffb640 1 -- 172.21.15.115:0/4271916961 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f13c0076e10 con 0x7f13c815cd20 2023-12-01T03:11:28.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.400+0000 7f13cf7f0640 1 -- 172.21.15.115:0/4271916961 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f13c80cc430 con 0x7f13c815cd20 2023-12-01T03:11:28.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.401+0000 7f13bdffb640 1 -- 172.21.15.115:0/4271916961 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v48) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f13c0043090 con 0x7f13c815cd20 2023-12-01T03:11:28.403 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:28.403 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:11:28.403 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T03:11:28.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.404 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T03:11:28.404 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T03:11:28.404 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.404 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:11:28.404 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:11:28.404 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:11:28.404 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.406 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T03:11:28.406 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T03:11:28.406 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.406 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:11:28.406 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:11:28.406 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:11:28.406 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:11:28.408 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.409 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:11:28.410 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.411 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T03:11:28.411 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T03:11:28.411 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.411 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:11:28.411 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:11:28.411 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.412 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T03:11:28.412 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T03:11:28.412 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.412 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:11:28.412 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:11:28.412 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.412 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:11:28.413 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:11:28.415 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:11:28.416 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:11:28.417 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.417 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T03:11:28.418 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T03:11:28.418 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.418 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:11:28.418 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T03:11:28.418 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:28.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:28.419 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T03:11:28.419 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T03:11:28.419 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:11:28.420 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:11:28.420 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T03:11:28.420 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:11:28.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:28.420 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T03:11:28.420 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:11:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 -- 172.21.15.115:0/4271916961 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f13ac06a670 msgr2=0x7f13ac06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f13ac06a670 0x7f13ac06cb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f13c400a950 tx=0x7f13c4008040 comp rx=0 tx=0).stop 2023-12-01T03:11:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 -- 172.21.15.115:0/4271916961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c815cd20 msgr2=0x7f13c8144f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c815cd20 0x7f13c8144f00 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f13c0036040 tx=0x7f13c0004a40 comp rx=0 tx=0).stop 2023-12-01T03:11:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 -- 172.21.15.115:0/4271916961 shutdown_connections 2023-12-01T03:11:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f13ac06a670 0x7f13ac06cb30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13c80d1230 0x7f13c81449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13c815cd20 0x7f13c8144f00 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 --2- 172.21.15.115:0/4271916961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13c80b5d80 0x7f13c81f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 -- 172.21.15.115:0/4271916961 >> 172.21.15.115:0/4271916961 conn(0x7f13c806a0e0 msgr2=0x7f13c81575e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 -- 172.21.15.115:0/4271916961 shutdown_connections 2023-12-01T03:11:28.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.404+0000 7f13cf7f0640 1 -- 172.21.15.115:0/4271916961 wait complete. 2023-12-01T03:11:28.425 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:28.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.636+0000 7f4edb39c640 1 Processor -- start 2023-12-01T03:11:28.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.636+0000 7f4edb39c640 1 -- start start 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.638+0000 7f4edb39c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40adac0 0x7f4ed40cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.638+0000 7f4edb39c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed40ce150 con 0x7f4ed40c8ed0 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.638+0000 7f4edb39c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed406d620 con 0x7f4ed406b350 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.638+0000 7f4edb39c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40cf3a0 con 0x7f4ed40adac0 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.638+0000 7f4ed8937640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4ed40c8ed0 0x7f4ed40cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56908/0 (socket says 172.21.15.115:56908) 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.638+0000 7f4ed8937640 1 -- 172.21.15.115:0/4280962341 learned_addr learned my addr 172.21.15.115:0/4280962341 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836586355 0 0) 0x7f4ed40ce150 con 0x7f4ed40c8ed0 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec0003520 con 0x7f4ed40c8ed0 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ed9138640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40adac0 0x7f4ed40cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2673307739 0 0) 0x7f4ec0003520 con 0x7f4ed40c8ed0 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40adac0 msgr2=0x7f4ed40cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:28.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40adac0 0x7f4ed40cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 >> v1:172.21.15.142:6789/0 conn(0x7f4ed406b350 legacy=0x7f4ed406b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40c1d90 con 0x7f4ed40c8ed0 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4ec4001670 con 0x7f4ed40c8ed0 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 >> v1:172.21.15.115:6789/0 conn(0x7f4ed40c8ed0 legacy=0x7f4ed40cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec0003a10 0x7f4ec0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40adac0 0x7f4ec00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ec0004d50 0x7f4ec00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40cf3a0 con 0x7f4ec0004d50 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec000bb60 con 0x7f4ec0003a10 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.639+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec000bd20 con 0x7f4ed40adac0 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.640+0000 7f4ed9939640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ec0004d50 0x7f4ec00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.640+0000 7f4ed9138640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec0003a10 0x7f4ec0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.640+0000 7f4ed9939640 1 -- 172.21.15.115:0/4280962341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40adac0 msgr2=0x7f4ec00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.640+0000 7f4ed9939640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ed40adac0 0x7f4ec00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.640+0000 7f4ed9939640 1 -- 172.21.15.115:0/4280962341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec0003a10 msgr2=0x7f4ec0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:28.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.640+0000 7f4ed9939640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec0003a10 0x7f4ec0003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.640+0000 7f4ed9939640 1 -- 172.21.15.115:0/4280962341 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0003520 con 0x7f4ec0004d50 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.641+0000 7f4ed9939640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ec0004d50 0x7f4ec00051d0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc004480 tx=0x7f4ec00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca9d3d0f07dfc20 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.641+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc031670 con 0x7f4ec0004d50 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.641+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc0317d0 con 0x7f4ec0004d50 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.641+0000 7f4ecbfff640 1 -- 172.21.15.115:0/4280962341 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc031b80 con 0x7f4ec0004d50 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.642+0000 7f4edb39c640 1 -- 172.21.15.115:0/4280962341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ec0004d50 msgr2=0x7f4ec00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.642+0000 7f4edb39c640 1 --2- 172.21.15.115:0/4280962341 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ec0004d50 0x7f4ec00051d0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc004480 tx=0x7f4ec00057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.642+0000 7f4ed9939640 1 -- 172.21.15.115:0/4280962341 reap_dead start 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.642+0000 7f4edb39c640 1 -- 172.21.15.115:0/4280962341 shutdown_connections 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.642+0000 7f4edb39c640 1 -- 172.21.15.115:0/4280962341 >> 172.21.15.115:0/4280962341 conn(0x7f4ed40ce880 msgr2=0x7f4ed40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.642+0000 7f4edb39c640 1 -- 172.21.15.115:0/4280962341 shutdown_connections 2023-12-01T03:11:28.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.642+0000 7f4edb39c640 1 -- 172.21.15.115:0/4280962341 wait complete. 2023-12-01T03:11:28.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.644+0000 7f4edb39c640 1 Processor -- start 2023-12-01T03:11:28.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.644+0000 7f4edb39c640 1 -- start start 2023-12-01T03:11:28.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.645+0000 7f4edb39c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec0004d50 0x7f4ed40c2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.645+0000 7f4edb39c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ae2a0 0x7f4ed40ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.645+0000 7f4edb39c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40c45f0 0x7f4ed40c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.645+0000 7f4edb39c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40c3640 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.645+0000 7f4edb39c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40aec40 con 0x7f4ed40c45f0 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.645+0000 7f4edb39c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40c29c0 con 0x7f4ec0004d50 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.645+0000 7f4ed8937640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ae2a0 0x7f4ed40ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.645+0000 7f4ed8937640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ae2a0 0x7f4ed40ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38638/0 (socket says 172.21.15.115:38638) 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.645+0000 7f4ed8937640 1 -- 172.21.15.115:0/4055725102 learned_addr learned my addr 172.21.15.115:0/4055725102 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ed9138640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec0004d50 0x7f4ed40c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ed9939640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40c45f0 0x7f4ed40c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ed8937640 1 -- 172.21.15.115:0/4055725102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec0004d50 msgr2=0x7f4ed40c2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ed8937640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec0004d50 0x7f4ed40c2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ed8937640 1 -- 172.21.15.115:0/4055725102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40c45f0 msgr2=0x7f4ed40c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ed8937640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40c45f0 0x7f4ed40c4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ed8937640 1 -- 172.21.15.115:0/4055725102 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40c2ca0 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ed8937640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ae2a0 0x7f4ed40ae700 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c680 tx=0x7f4ec400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ec9ffb640 1 -- 172.21.15.115:0/4055725102 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40072d0 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ec9ffb640 1 -- 172.21.15.115:0/4055725102 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec4007430 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4ec9ffb640 1 -- 172.21.15.115:0/4055725102 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40047e0 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4edb39c640 1 -- 172.21.15.115:0/4055725102 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed40c2f80 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.646+0000 7f4edb39c640 1 -- 172.21.15.115:0/4055725102 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed40c4fb0 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.647+0000 7f4edb39c640 1 -- 172.21.15.115:0/4055725102 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed404e890 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.648+0000 7f4ec9ffb640 1 -- 172.21.15.115:0/4055725102 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f4ec4004940 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.649+0000 7f4ec9ffb640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a670 0x7f4eb806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:28.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.649+0000 7f4ec9ffb640 1 -- 172.21.15.115:0/4055725102 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f4ec4087ed0 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.650+0000 7f4ed9138640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a670 0x7f4eb806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:28.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.650+0000 7f4ed9138640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a670 0x7f4eb806cb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006fd0 tx=0x7f4ed0008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:28.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.653+0000 7f4ec9ffb640 1 -- 172.21.15.115:0/4055725102 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4ec4056f00 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.844+0000 7f4edb39c640 1 -- 172.21.15.115:0/4055725102 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ed4161440 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.845+0000 7f4ec9ffb640 1 -- 172.21.15.115:0/4055725102 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v231) v1 ==== 74+0+12517 (secure 0 0 0) 0x7f4ec4018020 con 0x7f4ed40ae2a0 2023-12-01T03:11:28.847 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:28.847 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":231,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:27.874489+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:28.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 -- 172.21.15.115:0/4055725102 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a670 msgr2=0x7f4eb806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a670 0x7f4eb806cb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006fd0 tx=0x7f4ed0008040 comp rx=0 tx=0).stop 2023-12-01T03:11:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 -- 172.21.15.115:0/4055725102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ae2a0 msgr2=0x7f4ed40ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ae2a0 0x7f4ed40ae700 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c680 tx=0x7f4ec400cb40 comp rx=0 tx=0).stop 2023-12-01T03:11:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 -- 172.21.15.115:0/4055725102 shutdown_connections 2023-12-01T03:11:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4eb806a670 0x7f4eb806cb30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ed40c45f0 0x7f4ed40c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ed40ae2a0 0x7f4ed40ae700 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 --2- 172.21.15.115:0/4055725102 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec0004d50 0x7f4ed40c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 -- 172.21.15.115:0/4055725102 >> 172.21.15.115:0/4055725102 conn(0x7f4ed40ce880 msgr2=0x7f4ed415bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 -- 172.21.15.115:0/4055725102 shutdown_connections 2023-12-01T03:11:28.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:28.848+0000 7f4edb39c640 1 -- 172.21.15.115:0/4055725102 wait complete. 2023-12-01T03:11:28.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:11:28.869 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:11:29.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.078+0000 7fc224821640 1 Processor -- start 2023-12-01T03:11:29.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.078+0000 7fc224821640 1 -- start start 2023-12-01T03:11:29.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc224821640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc220142870 0x7fc2200ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc224821640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200cd010 0x7fc2200b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc224821640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc2200b1c50 0x7fc22015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc224821640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2200b3ae0 con 0x7fc220142870 2023-12-01T03:11:29.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc224821640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc220142c50 con 0x7fc2200b1c50 2023-12-01T03:11:29.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc224821640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200cd470 con 0x7fc2200cd010 2023-12-01T03:11:29.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc21e59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc220142870 0x7fc2200ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc21e59c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc220142870 0x7fc2200ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38650/0 (socket says 172.21.15.115:38650) 2023-12-01T03:11:29.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc21e59c640 1 -- 172.21.15.115:0/2100641279 learned_addr learned my addr 172.21.15.115:0/2100641279 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:29.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc21ed9d640 1 --2- 172.21.15.115:0/2100641279 >> v2:172.21.15.142:3300/0 conn(0x7fc2200b1c50 0x7fc22015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc21e59c640 1 -- 172.21.15.115:0/2100641279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200cd010 msgr2=0x7fc2200b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.080+0000 7fc21dd9b640 1 --2- 172.21.15.115:0/2100641279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200cd010 0x7fc2200b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.081+0000 7fc21e59c640 1 --2- 172.21.15.115:0/2100641279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200cd010 0x7fc2200b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.081+0000 7fc21e59c640 1 -- 172.21.15.115:0/2100641279 >> v2:172.21.15.142:3300/0 conn(0x7fc2200b1c50 msgr2=0x7fc22015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.081+0000 7fc21e59c640 1 --2- 172.21.15.115:0/2100641279 >> v2:172.21.15.142:3300/0 conn(0x7fc2200b1c50 0x7fc22015d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.081+0000 7fc21e59c640 1 -- 172.21.15.115:0/2100641279 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2200b20e0 con 0x7fc220142870 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.081+0000 7fc21ed9d640 1 --2- 172.21.15.115:0/2100641279 >> v2:172.21.15.142:3300/0 conn(0x7fc2200b1c50 0x7fc22015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.081+0000 7fc21dd9b640 1 --2- 172.21.15.115:0/2100641279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200cd010 0x7fc2200b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.082+0000 7fc21e59c640 1 --2- 172.21.15.115:0/2100641279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 0x7fc2200ccad0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fc2200b0f50 tx=0x7fc21002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933079de5a627857 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.082+0000 7fc21d59a640 1 -- 172.21.15.115:0/2100641279 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21002ca60 con 0x7fc220142870 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.082+0000 7fc21d59a640 1 -- 172.21.15.115:0/2100641279 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21002cbc0 con 0x7fc220142870 2023-12-01T03:11:29.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.082+0000 7fc21d59a640 1 -- 172.21.15.115:0/2100641279 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100343f0 con 0x7fc220142870 2023-12-01T03:11:29.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.083+0000 7fc224821640 1 -- 172.21.15.115:0/2100641279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 msgr2=0x7fc2200ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.083+0000 7fc224821640 1 --2- 172.21.15.115:0/2100641279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 0x7fc2200ccad0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fc2200b0f50 tx=0x7fc21002bec0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.083+0000 7fc224821640 1 -- 172.21.15.115:0/2100641279 shutdown_connections 2023-12-01T03:11:29.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.083+0000 7fc224821640 1 --2- 172.21.15.115:0/2100641279 >> v2:172.21.15.142:3300/0 conn(0x7fc2200b1c50 0x7fc22015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.083+0000 7fc224821640 1 --2- 172.21.15.115:0/2100641279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200cd010 0x7fc2200b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.083+0000 7fc224821640 1 --2- 172.21.15.115:0/2100641279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 0x7fc2200ccad0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.083+0000 7fc224821640 1 -- 172.21.15.115:0/2100641279 >> 172.21.15.115:0/2100641279 conn(0x7fc220065960 msgr2=0x7fc220065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:29.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.083+0000 7fc224821640 1 -- 172.21.15.115:0/2100641279 shutdown_connections 2023-12-01T03:11:29.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.083+0000 7fc224821640 1 -- 172.21.15.115:0/2100641279 wait complete. 2023-12-01T03:11:29.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.084+0000 7fc224821640 1 Processor -- start 2023-12-01T03:11:29.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.085+0000 7fc224821640 1 -- start start 2023-12-01T03:11:29.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.085+0000 7fc224821640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200b1c50 0x7fc2200cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.085+0000 7fc224821640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2200cd010 0x7fc2200c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.085+0000 7fc224821640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 0x7fc2200c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc224821640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200cbd10 con 0x7fc220142870 2023-12-01T03:11:29.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc224821640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200c90c0 con 0x7fc2200cd010 2023-12-01T03:11:29.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc224821640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200c6120 con 0x7fc2200b1c50 2023-12-01T03:11:29.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc21ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 0x7fc2200c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc21ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 0x7fc2200c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38656/0 (socket says 172.21.15.115:38656) 2023-12-01T03:11:29.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc21ed9d640 1 -- 172.21.15.115:0/3986818813 learned_addr learned my addr 172.21.15.115:0/3986818813 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:29.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc21dd9b640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2200cd010 0x7fc2200c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc21e59c640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200b1c50 0x7fc2200cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc21ed9d640 1 -- 172.21.15.115:0/3986818813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200b1c50 msgr2=0x7fc2200cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc21ed9d640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200b1c50 0x7fc2200cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc21ed9d640 1 -- 172.21.15.115:0/3986818813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2200cd010 msgr2=0x7fc2200c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc21ed9d640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2200cd010 0x7fc2200c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.086+0000 7fc21ed9d640 1 -- 172.21.15.115:0/3986818813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc210009b30 con 0x7fc220142870 2023-12-01T03:11:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.087+0000 7fc21ed9d640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 0x7fc2200c7320 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fc21800afd0 tx=0x7fc21800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.087+0000 7fc216ffd640 1 -- 172.21.15.115:0/3986818813 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218004440 con 0x7fc220142870 2023-12-01T03:11:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.087+0000 7fc216ffd640 1 -- 172.21.15.115:0/3986818813 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc218003610 con 0x7fc220142870 2023-12-01T03:11:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.087+0000 7fc224821640 1 -- 172.21.15.115:0/3986818813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2200c6400 con 0x7fc220142870 2023-12-01T03:11:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.087+0000 7fc216ffd640 1 -- 172.21.15.115:0/3986818813 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180039c0 con 0x7fc220142870 2023-12-01T03:11:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.087+0000 7fc224821640 1 -- 172.21.15.115:0/3986818813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2200c69c0 con 0x7fc220142870 2023-12-01T03:11:29.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.088+0000 7fc224821640 1 -- 172.21.15.115:0/3986818813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22004e890 con 0x7fc220142870 2023-12-01T03:11:29.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.089+0000 7fc216ffd640 1 -- 172.21.15.115:0/3986818813 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fc218003b20 con 0x7fc220142870 2023-12-01T03:11:29.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.090+0000 7fc216ffd640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc20006a670 0x7fc20006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.090+0000 7fc216ffd640 1 -- 172.21.15.115:0/3986818813 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fc218088d30 con 0x7fc220142870 2023-12-01T03:11:29.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.090+0000 7fc21e59c640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc20006a670 0x7fc20006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.091+0000 7fc21e59c640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc20006a670 0x7fc20006cb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc210035000 tx=0x7fc210030000 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:29.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.094+0000 7fc216ffd640 1 -- 172.21.15.115:0/3986818813 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc2180541d0 con 0x7fc220142870 2023-12-01T03:11:29.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.284+0000 7fc224821640 1 -- 172.21.15.115:0/3986818813 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc22006b6d0 con 0x7fc220142870 2023-12-01T03:11:29.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.285+0000 7fc216ffd640 1 -- 172.21.15.115:0/3986818813 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v231) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2180150a0 con 0x7fc220142870 2023-12-01T03:11:29.287 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:11:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 -- 172.21.15.115:0/3986818813 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc20006a670 msgr2=0x7fc20006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc20006a670 0x7fc20006cb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc210035000 tx=0x7fc210030000 comp rx=0 tx=0).stop 2023-12-01T03:11:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 -- 172.21.15.115:0/3986818813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 msgr2=0x7fc2200c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 0x7fc2200c7320 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fc21800afd0 tx=0x7fc21800ca20 comp rx=0 tx=0).stop 2023-12-01T03:11:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 -- 172.21.15.115:0/3986818813 shutdown_connections 2023-12-01T03:11:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc20006a670 0x7fc20006cb30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc220142870 0x7fc2200c7320 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2200cd010 0x7fc2200c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 --2- 172.21.15.115:0/3986818813 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc2200b1c50 0x7fc2200cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 -- 172.21.15.115:0/3986818813 >> 172.21.15.115:0/3986818813 conn(0x7fc220065960 msgr2=0x7fc220065fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 -- 172.21.15.115:0/3986818813 shutdown_connections 2023-12-01T03:11:29.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.288+0000 7fc224821640 1 -- 172.21.15.115:0/3986818813 wait complete. 2023-12-01T03:11:29.309 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T03:11:29.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.519+0000 7fa2e699f640 1 Processor -- start 2023-12-01T03:11:29.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.519+0000 7fa2e699f640 1 -- start start 2023-12-01T03:11:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.521+0000 7fa2e699f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa2e015cd20 0x7fa2e00d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.521+0000 7fa2e699f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e00d1230 0x7fa2e00d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.521+0000 7fa2e699f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e00d1b90 con 0x7fa2e00b5d80 2023-12-01T03:11:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.521+0000 7fa2e699f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e00b2160 con 0x7fa2e015cd20 2023-12-01T03:11:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.521+0000 7fa2e699f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00b8c80 con 0x7fa2e00d1230 2023-12-01T03:11:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.521+0000 7fa2dffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa2e015cd20 0x7fa2e00d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.521+0000 7fa2dffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa2e015cd20 0x7fa2e00d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38658/0 (socket says 172.21.15.115:38658) 2023-12-01T03:11:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.521+0000 7fa2dffff640 1 -- 172.21.15.115:0/3387197260 learned_addr learned my addr 172.21.15.115:0/3387197260 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.521+0000 7fa2df7fe640 1 --1- 172.21.15.115:0/3387197260 >> v1:172.21.15.115:6789/0 conn(0x7fa2e00b5d80 0x7fa2e00b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56912/0 (socket says 172.21.15.115:56912) 2023-12-01T03:11:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.522+0000 7fa2dffff640 1 -- 172.21.15.115:0/3387197260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e00d1230 msgr2=0x7fa2e00d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.522+0000 7fa2dffff640 1 --2- 172.21.15.115:0/3387197260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e00d1230 0x7fa2e00d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.522+0000 7fa2dffff640 1 -- 172.21.15.115:0/3387197260 >> v1:172.21.15.115:6789/0 conn(0x7fa2e00b5d80 legacy=0x7fa2e00b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.522+0000 7fa2dffff640 1 -- 172.21.15.115:0/3387197260 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00ce2e0 con 0x7fa2e015cd20 2023-12-01T03:11:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.522+0000 7fa2deffd640 1 -- 172.21.15.115:0/3387197260 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389229360 0 0) 0x7fa2c8001110 con 0x7fa2e00b5d80 2023-12-01T03:11:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.523+0000 7fa2dffff640 1 --2- 172.21.15.115:0/3387197260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 0x7fa2e00d24d0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009870 tx=0x7fa2d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2b7adb6e72a52e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:29.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.523+0000 7fa2deffd640 1 -- 172.21.15.115:0/3387197260 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0031640 con 0x7fa2e015cd20 2023-12-01T03:11:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.523+0000 7fa2deffd640 1 -- 172.21.15.115:0/3387197260 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d00317a0 con 0x7fa2e015cd20 2023-12-01T03:11:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.523+0000 7fa2deffd640 1 -- 172.21.15.115:0/3387197260 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0031b30 con 0x7fa2e015cd20 2023-12-01T03:11:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.524+0000 7fa2e699f640 1 -- 172.21.15.115:0/3387197260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 msgr2=0x7fa2e00d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.524+0000 7fa2e699f640 1 --2- 172.21.15.115:0/3387197260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 0x7fa2e00d24d0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009870 tx=0x7fa2d002b8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.524+0000 7fa2e699f640 1 -- 172.21.15.115:0/3387197260 shutdown_connections 2023-12-01T03:11:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.524+0000 7fa2e699f640 1 --2- 172.21.15.115:0/3387197260 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e00d1230 0x7fa2e00d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.524+0000 7fa2e699f640 1 --2- 172.21.15.115:0/3387197260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 0x7fa2e00d24d0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.524+0000 7fa2e699f640 1 -- 172.21.15.115:0/3387197260 >> 172.21.15.115:0/3387197260 conn(0x7fa2e006a0e0 msgr2=0x7fa2e006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.524+0000 7fa2e699f640 1 -- 172.21.15.115:0/3387197260 shutdown_connections 2023-12-01T03:11:29.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.524+0000 7fa2e699f640 1 -- 172.21.15.115:0/3387197260 wait complete. 2023-12-01T03:11:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.526+0000 7fa2e699f640 1 Processor -- start 2023-12-01T03:11:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.526+0000 7fa2e699f640 1 -- start start 2023-12-01T03:11:29.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e699f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e00b5d80 0x7fa2e0159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e699f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e00d1230 0x7fa2e00b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e699f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 0x7fa2e015f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e699f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00c6be0 con 0x7fa2e015cd20 2023-12-01T03:11:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e699f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00b2980 con 0x7fa2e00d1230 2023-12-01T03:11:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e699f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00b2c50 con 0x7fa2e00b5d80 2023-12-01T03:11:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e4f3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 0x7fa2e015f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e4f3c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 0x7fa2e015f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38666/0 (socket says 172.21.15.115:38666) 2023-12-01T03:11:29.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e4f3c640 1 -- 172.21.15.115:0/1975318737 learned_addr learned my addr 172.21.15.115:0/1975318737 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2df7fe640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e00d1230 0x7fa2e00b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2dffff640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e00b5d80 0x7fa2e0159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e4f3c640 1 -- 172.21.15.115:0/1975318737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e00b5d80 msgr2=0x7fa2e0159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e4f3c640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e00b5d80 0x7fa2e0159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e4f3c640 1 -- 172.21.15.115:0/1975318737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e00d1230 msgr2=0x7fa2e00b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e4f3c640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e00d1230 0x7fa2e00b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2e4f3c640 1 -- 172.21.15.115:0/1975318737 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00094d0 con 0x7fa2e015cd20 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.527+0000 7fa2df7fe640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e00d1230 0x7fa2e00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.528+0000 7fa2e4f3c640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 0x7fa2e015f2d0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800b970 tx=0x7fa2d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.528+0000 7fa2dcff9640 1 -- 172.21.15.115:0/1975318737 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800cb40 con 0x7fa2e015cd20 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.528+0000 7fa2dcff9640 1 -- 172.21.15.115:0/1975318737 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d800ccc0 con 0x7fa2e015cd20 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.528+0000 7fa2dcff9640 1 -- 172.21.15.115:0/1975318737 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8014680 con 0x7fa2e015cd20 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.528+0000 7fa2e699f640 1 -- 172.21.15.115:0/1975318737 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e00b2f30 con 0x7fa2e015cd20 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.528+0000 7fa2e699f640 1 -- 172.21.15.115:0/1975318737 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e00b3210 con 0x7fa2e015cd20 2023-12-01T03:11:29.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.529+0000 7fa2e699f640 1 -- 172.21.15.115:0/1975318737 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e00c6d90 con 0x7fa2e015cd20 2023-12-01T03:11:29.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.530+0000 7fa2dcff9640 1 -- 172.21.15.115:0/1975318737 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fa2d801d3b0 con 0x7fa2e015cd20 2023-12-01T03:11:29.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.531+0000 7fa2dcff9640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2c006a6a0 0x7fa2c006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.531+0000 7fa2dcff9640 1 -- 172.21.15.115:0/1975318737 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fa2d8088960 con 0x7fa2e015cd20 2023-12-01T03:11:29.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.531+0000 7fa2dffff640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2c006a6a0 0x7fa2c006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.532+0000 7fa2dffff640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2c006a6a0 0x7fa2c006cb60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0036000 tx=0x7fa2d003c040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:29.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.535+0000 7fa2dcff9640 1 -- 172.21.15.115:0/1975318737 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa2e00c6d90 con 0x7fa2e015cd20 2023-12-01T03:11:29.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.732+0000 7fa2e699f640 1 -- 172.21.15.115:0/1975318737 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa2e004e890 con 0x7fa2e015cd20 2023-12-01T03:11:29.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.733+0000 7fa2dcff9640 1 -- 172.21.15.115:0/1975318737 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v231) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa2d8019020 con 0x7fa2e015cd20 2023-12-01T03:11:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:29.735 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T03:11:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T03:11:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T03:11:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:29.735 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T03:11:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.735+0000 7fa2e699f640 1 -- 172.21.15.115:0/1975318737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2c006a6a0 msgr2=0x7fa2c006cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.735+0000 7fa2e699f640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2c006a6a0 0x7fa2c006cb60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0036000 tx=0x7fa2d003c040 comp rx=0 tx=0).stop 2023-12-01T03:11:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.735+0000 7fa2e699f640 1 -- 172.21.15.115:0/1975318737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 msgr2=0x7fa2e015f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.735+0000 7fa2e699f640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 0x7fa2e015f2d0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800b970 tx=0x7fa2d800be30 comp rx=0 tx=0).stop 2023-12-01T03:11:29.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.735+0000 7fa2e699f640 1 -- 172.21.15.115:0/1975318737 shutdown_connections 2023-12-01T03:11:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.735+0000 7fa2e699f640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2c006a6a0 0x7fa2c006cb60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.735+0000 7fa2e699f640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2e015cd20 0x7fa2e015f2d0 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.735+0000 7fa2e699f640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2e00d1230 0x7fa2e00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.735+0000 7fa2e699f640 1 --2- 172.21.15.115:0/1975318737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2e00b5d80 0x7fa2e0159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.735+0000 7fa2e699f640 1 -- 172.21.15.115:0/1975318737 >> 172.21.15.115:0/1975318737 conn(0x7fa2e006a0e0 msgr2=0x7fa2e006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.736+0000 7fa2e699f640 1 -- 172.21.15.115:0/1975318737 shutdown_connections 2023-12-01T03:11:29.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.736+0000 7fa2e699f640 1 -- 172.21.15.115:0/1975318737 wait complete. 2023-12-01T03:11:29.756 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T03:11:29.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T03:11:29.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.967+0000 7fa3996b9640 1 Processor -- start 2023-12-01T03:11:29.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.968+0000 7fa3996b9640 1 -- start start 2023-12-01T03:11:29.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.969+0000 7fa3996b9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3940d2730 0x7fa3940b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.969+0000 7fa3996b9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3940d1690 con 0x7fa3940b4d50 2023-12-01T03:11:29.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.969+0000 7fa3996b9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3940d1970 con 0x7fa39415ce40 2023-12-01T03:11:29.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.969+0000 7fa3996b9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39415d220 con 0x7fa3940d2730 2023-12-01T03:11:29.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.969+0000 7fa38a7fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa3940b4d50 0x7fa3940b5cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56924/0 (socket says 172.21.15.115:56924) 2023-12-01T03:11:29.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.969+0000 7fa38a7fc640 1 -- 172.21.15.115:0/2929557396 learned_addr learned my addr 172.21.15.115:0/2929557396 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:29.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818342936 0 0) 0x7fa3940d1690 con 0x7fa3940b4d50 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa380003520 con 0x7fa3940b4d50 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3937fe640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3940d2730 0x7fa3940b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3235053220 0 0) 0x7fa380003520 con 0x7fa3940b4d50 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3940d2730 msgr2=0x7fa3940b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3927fc640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3940d2730 0x7fa3940b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 >> v1:172.21.15.142:6789/0 conn(0x7fa39415ce40 legacy=0x7fa3940b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3940d1c50 con 0x7fa3940b4d50 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa37c001690 con 0x7fa3940b4d50 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3937fe640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3940d2730 0x7fa3940b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 >> v1:172.21.15.115:6789/0 conn(0x7fa3940b4d50 legacy=0x7fa3940b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.970+0000 7fa3927fc640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3800039a0 0x7fa380003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.971+0000 7fa3927fc640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800047c0 0x7fa380004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.971+0000 7fa3927fc640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3940d2730 0x7fa380005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.971+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39415d220 con 0x7fa3800039a0 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.971+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38000bb60 con 0x7fa3800047c0 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.971+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38000bd20 con 0x7fa3940d2730 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.971+0000 7fa392ffd640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3800039a0 0x7fa380003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.971+0000 7fa38a7fc640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800047c0 0x7fa380004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.971+0000 7fa3937fe640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3940d2730 0x7fa380005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.971+0000 7fa392ffd640 1 -- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3940d2730 msgr2=0x7fa380005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.971+0000 7fa392ffd640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3940d2730 0x7fa380005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.972+0000 7fa392ffd640 1 -- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800047c0 msgr2=0x7fa380004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.972+0000 7fa392ffd640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800047c0 0x7fa380004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.972+0000 7fa392ffd640 1 -- 172.21.15.115:0/2929557396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa380003520 con 0x7fa3800039a0 2023-12-01T03:11:29.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.972+0000 7fa38a7fc640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3800047c0 0x7fa380004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:11:29.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.973+0000 7fa392ffd640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3800039a0 0x7fa380003dc0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fa384001750 tx=0x7fa3800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed505b5f37ffcc05 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:29.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.973+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3840052d0 con 0x7fa3800039a0 2023-12-01T03:11:29.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.973+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38402e050 con 0x7fa3800039a0 2023-12-01T03:11:29.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.973+0000 7fa3927fc640 1 -- 172.21.15.115:0/2929557396 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3840035f0 con 0x7fa3800039a0 2023-12-01T03:11:29.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.974+0000 7fa3996b9640 1 -- 172.21.15.115:0/2929557396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3800039a0 msgr2=0x7fa380003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.974+0000 7fa3996b9640 1 --2- 172.21.15.115:0/2929557396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3800039a0 0x7fa380003dc0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fa384001750 tx=0x7fa3800057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.974+0000 7fa3937fe640 1 -- 172.21.15.115:0/2929557396 reap_dead start 2023-12-01T03:11:29.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.974+0000 7fa3996b9640 1 -- 172.21.15.115:0/2929557396 shutdown_connections 2023-12-01T03:11:29.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.974+0000 7fa3996b9640 1 -- 172.21.15.115:0/2929557396 >> 172.21.15.115:0/2929557396 conn(0x7fa394065f90 msgr2=0x7fa3940cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:29.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.974+0000 7fa3996b9640 1 -- 172.21.15.115:0/2929557396 shutdown_connections 2023-12-01T03:11:29.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.974+0000 7fa3996b9640 1 -- 172.21.15.115:0/2929557396 wait complete. 2023-12-01T03:11:29.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.976+0000 7fa3996b9640 1 Processor -- start 2023-12-01T03:11:29.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.976+0000 7fa3996b9640 1 -- start start 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa3996b9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3800039a0 0x7fa39414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa3996b9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941506e0 0x7fa394150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa3996b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3940b72b0 0x7fa3940b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa3996b9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940b7c70 con 0x7fa3941506e0 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa3996b9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39415d390 con 0x7fa3940b72b0 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa3996b9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39414e100 con 0x7fa3800039a0 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa38a7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941506e0 0x7fa394150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa38a7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941506e0 0x7fa394150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38674/0 (socket says 172.21.15.115:38674) 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa38a7fc640 1 -- 172.21.15.115:0/1202141195 learned_addr learned my addr 172.21.15.115:0/1202141195 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa3937fe640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3940b72b0 0x7fa3940b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa38a7fc640 1 -- 172.21.15.115:0/1202141195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3800039a0 msgr2=0x7fa39414dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa392ffd640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3800039a0 0x7fa39414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.977+0000 7fa38a7fc640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3800039a0 0x7fa39414dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.978+0000 7fa38a7fc640 1 -- 172.21.15.115:0/1202141195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3940b72b0 msgr2=0x7fa3940b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.978+0000 7fa38a7fc640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3940b72b0 0x7fa3940b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.978+0000 7fa38a7fc640 1 -- 172.21.15.115:0/1202141195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa394151160 con 0x7fa3941506e0 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.978+0000 7fa3937fe640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3940b72b0 0x7fa3940b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.978+0000 7fa38a7fc640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941506e0 0x7fa394150b40 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00e700 tx=0x7fa37c00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.978+0000 7fa38bfff640 1 -- 172.21.15.115:0/1202141195 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c00ee10 con 0x7fa3941506e0 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.978+0000 7fa38bfff640 1 -- 172.21.15.115:0/1202141195 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37c00c870 con 0x7fa3941506e0 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.978+0000 7fa3996b9640 1 -- 172.21.15.115:0/1202141195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39415d650 con 0x7fa3941506e0 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.978+0000 7fa38bfff640 1 -- 172.21.15.115:0/1202141195 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c00cc00 con 0x7fa3941506e0 2023-12-01T03:11:29.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.978+0000 7fa3996b9640 1 -- 172.21.15.115:0/1202141195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39415dc10 con 0x7fa3941506e0 2023-12-01T03:11:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.979+0000 7fa3996b9640 1 -- 172.21.15.115:0/1202141195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa394074210 con 0x7fa3941506e0 2023-12-01T03:11:29.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.980+0000 7fa38bfff640 1 -- 172.21.15.115:0/1202141195 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fa37c01d460 con 0x7fa3941506e0 2023-12-01T03:11:29.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.982+0000 7fa38bfff640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa37406a670 0x7fa37406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:29.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.982+0000 7fa38bfff640 1 -- 172.21.15.115:0/1202141195 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fa37c088980 con 0x7fa3941506e0 2023-12-01T03:11:29.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.982+0000 7fa392ffd640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa37406a670 0x7fa37406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:29.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.983+0000 7fa392ffd640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa37406a670 0x7fa37406cb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa384033000 tx=0x7fa384009030 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:29.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:29.986+0000 7fa38bfff640 1 -- 172.21.15.115:0/1202141195 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa37c053f60 con 0x7fa3941506e0 2023-12-01T03:11:30.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.175+0000 7fa3996b9640 1 -- 172.21.15.115:0/1202141195 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa394158470 con 0x7fa3941506e0 2023-12-01T03:11:30.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.637+0000 7fa38bfff640 1 -- 172.21.15.115:0/1202141195 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v232) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa37c057580 con 0x7fa3941506e0 2023-12-01T03:11:30.640 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T03:11:30.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.643+0000 7fa3996b9640 1 -- 172.21.15.115:0/1202141195 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa37406a670 msgr2=0x7fa37406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:30.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.643+0000 7fa3996b9640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa37406a670 0x7fa37406cb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa384033000 tx=0x7fa384009030 comp rx=0 tx=0).stop 2023-12-01T03:11:30.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.644+0000 7fa3996b9640 1 -- 172.21.15.115:0/1202141195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941506e0 msgr2=0x7fa394150b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:30.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.644+0000 7fa3996b9640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941506e0 0x7fa394150b40 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00e700 tx=0x7fa37c00ebc0 comp rx=0 tx=0).stop 2023-12-01T03:11:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.646+0000 7fa3996b9640 1 -- 172.21.15.115:0/1202141195 shutdown_connections 2023-12-01T03:11:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.646+0000 7fa3996b9640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa37406a670 0x7fa37406cb30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.646+0000 7fa3996b9640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3940b72b0 0x7fa3940b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:30.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.646+0000 7fa3996b9640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3941506e0 0x7fa394150b40 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:30.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.646+0000 7fa3996b9640 1 --2- 172.21.15.115:0/1202141195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3800039a0 0x7fa39414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:30.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.646+0000 7fa3996b9640 1 -- 172.21.15.115:0/1202141195 >> 172.21.15.115:0/1202141195 conn(0x7fa394065f90 msgr2=0x7fa39415bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:30.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.647+0000 7fa3996b9640 1 -- 172.21.15.115:0/1202141195 shutdown_connections 2023-12-01T03:11:30.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.648+0000 7fa3996b9640 1 -- 172.21.15.115:0/1202141195 wait complete. 2023-12-01T03:11:30.673 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T03:11:30.688 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:11:30.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:11:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.874+0000 7efd8a25f640 1 Processor -- start 2023-12-01T03:11:30.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.874+0000 7efd8a25f640 1 -- start start 2023-12-01T03:11:30.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.876+0000 7efd8a25f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efd8406c390 0x7efd8406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:30.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.876+0000 7efd8a25f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd840b2b00 con 0x7efd84143240 2023-12-01T03:11:30.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.876+0000 7efd8a25f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd840b2de0 con 0x7efd840ce0c0 2023-12-01T03:11:30.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.876+0000 7efd8a25f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8406cd40 con 0x7efd8406c390 2023-12-01T03:11:30.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.876+0000 7efd837fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efd84143240 0x7efd840b20a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56934/0 (socket says 172.21.15.115:56934) 2023-12-01T03:11:30.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.876+0000 7efd837fe640 1 -- 172.21.15.115:0/2552529072 learned_addr learned my addr 172.21.15.115:0/2552529072 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:30.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.876+0000 7efd82ffd640 1 --2- 172.21.15.115:0/2552529072 >> v2:172.21.15.115:3300/0 conn(0x7efd8406c390 0x7efd8406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:30.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.877+0000 7efd827fc640 1 -- 172.21.15.115:0/2552529072 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453333673 0 0) 0x7efd840b2b00 con 0x7efd84143240 2023-12-01T03:11:30.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.877+0000 7efd827fc640 1 -- 172.21.15.115:0/2552529072 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd70003520 con 0x7efd84143240 2023-12-01T03:11:30.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.877+0000 7efd82ffd640 1 -- 172.21.15.115:0/2552529072 >> v1:172.21.15.142:6789/0 conn(0x7efd840ce0c0 legacy=0x7efd840ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:30.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.877+0000 7efd82ffd640 1 -- 172.21.15.115:0/2552529072 >> v1:172.21.15.115:6789/0 conn(0x7efd84143240 legacy=0x7efd840b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:30.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.877+0000 7efd82ffd640 1 -- 172.21.15.115:0/2552529072 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd840b30c0 con 0x7efd8406c390 2023-12-01T03:11:30.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.878+0000 7efd82ffd640 1 --2- 172.21.15.115:0/2552529072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd8406c390 0x7efd8406c800 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7efd74009d90 tx=0x7efd7402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4206d41294ba456 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:30.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.878+0000 7efd827fc640 1 -- 172.21.15.115:0/2552529072 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7402cd00 con 0x7efd8406c390 2023-12-01T03:11:30.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.878+0000 7efd827fc640 1 -- 172.21.15.115:0/2552529072 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7402ce60 con 0x7efd8406c390 2023-12-01T03:11:30.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.878+0000 7efd827fc640 1 -- 172.21.15.115:0/2552529072 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd74034660 con 0x7efd8406c390 2023-12-01T03:11:30.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.879+0000 7efd8a25f640 1 -- 172.21.15.115:0/2552529072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd8406c390 msgr2=0x7efd8406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:30.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.879+0000 7efd8a25f640 1 --2- 172.21.15.115:0/2552529072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd8406c390 0x7efd8406c800 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7efd74009d90 tx=0x7efd7402bf50 comp rx=0 tx=0).stop 2023-12-01T03:11:30.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.879+0000 7efd8a25f640 1 -- 172.21.15.115:0/2552529072 shutdown_connections 2023-12-01T03:11:30.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.879+0000 7efd8a25f640 1 --2- 172.21.15.115:0/2552529072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd8406c390 0x7efd8406c800 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:30.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.879+0000 7efd8a25f640 1 -- 172.21.15.115:0/2552529072 >> 172.21.15.115:0/2552529072 conn(0x7efd84065960 msgr2=0x7efd84065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:30.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.879+0000 7efd8a25f640 1 -- 172.21.15.115:0/2552529072 shutdown_connections 2023-12-01T03:11:30.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.879+0000 7efd8a25f640 1 -- 172.21.15.115:0/2552529072 wait complete. 2023-12-01T03:11:30.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.881+0000 7efd8a25f640 1 Processor -- start 2023-12-01T03:11:30.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.881+0000 7efd8a25f640 1 -- start start 2023-12-01T03:11:30.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.882+0000 7efd8a25f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd8406c390 0x7efd84132860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:30.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.882+0000 7efd8a25f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd840ce0c0 0x7efd8413a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:30.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.882+0000 7efd8a25f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd84143240 0x7efd84133d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:30.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.882+0000 7efd8a25f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8413bb50 con 0x7efd84143240 2023-12-01T03:11:30.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.882+0000 7efd8a25f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8413af40 con 0x7efd8406c390 2023-12-01T03:11:30.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.882+0000 7efd8a25f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8413b220 con 0x7efd840ce0c0 2023-12-01T03:11:30.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.882+0000 7efd83fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd84143240 0x7efd84133d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:30.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.882+0000 7efd83fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd84143240 0x7efd84133d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38694/0 (socket says 172.21.15.115:38694) 2023-12-01T03:11:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.882+0000 7efd83fff640 1 -- 172.21.15.115:0/1085308752 learned_addr learned my addr 172.21.15.115:0/1085308752 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd837fe640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd8406c390 0x7efd84132860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd83fff640 1 -- 172.21.15.115:0/1085308752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd840ce0c0 msgr2=0x7efd8413a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd83fff640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd840ce0c0 0x7efd8413a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd83fff640 1 -- 172.21.15.115:0/1085308752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd8406c390 msgr2=0x7efd84132860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd83fff640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd8406c390 0x7efd84132860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:30.889 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd83fff640 1 -- 172.21.15.115:0/1085308752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd74009a70 con 0x7efd84143240 2023-12-01T03:11:30.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd83fff640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd84143240 0x7efd84133d80 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7efd7c001eb0 tx=0x7efd7c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:30.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd7bfff640 1 -- 172.21.15.115:0/1085308752 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0052d0 con 0x7efd84143240 2023-12-01T03:11:30.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd7bfff640 1 -- 172.21.15.115:0/1085308752 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c005430 con 0x7efd84143240 2023-12-01T03:11:30.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd8a25f640 1 -- 172.21.15.115:0/1085308752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8413b500 con 0x7efd84143240 2023-12-01T03:11:30.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.883+0000 7efd7bfff640 1 -- 172.21.15.115:0/1085308752 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c003810 con 0x7efd84143240 2023-12-01T03:11:30.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.884+0000 7efd8a25f640 1 -- 172.21.15.115:0/1085308752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8413aaa0 con 0x7efd84143240 2023-12-01T03:11:30.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.885+0000 7efd8a25f640 1 -- 172.21.15.115:0/1085308752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8406cbb0 con 0x7efd84143240 2023-12-01T03:11:30.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.886+0000 7efd7bfff640 1 -- 172.21.15.115:0/1085308752 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7efd7c003970 con 0x7efd84143240 2023-12-01T03:11:30.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.887+0000 7efd7bfff640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a6a0 0x7efd6406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:30.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.887+0000 7efd7bfff640 1 -- 172.21.15.115:0/1085308752 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 5271+0+0 (secure 0 0 0) 0x7efd7c0883f0 con 0x7efd84143240 2023-12-01T03:11:30.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.887+0000 7efd837fe640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a6a0 0x7efd6406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:30.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.888+0000 7efd837fe640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a6a0 0x7efd6406cb60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0011e0 tx=0x7efd6c009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:30.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:30.891+0000 7efd7bfff640 1 -- 172.21.15.115:0/1085308752 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efd7c0539e0 con 0x7efd84143240 2023-12-01T03:11:31.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.085+0000 7efd8a25f640 1 -- 172.21.15.115:0/1085308752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efd8413acc0 con 0x7efd84143240 2023-12-01T03:11:31.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.650+0000 7efd7bfff640 1 -- 172.21.15.115:0/1085308752 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v233) v1 ==== 170+0+0 (secure 0 0 0) 0x7efd7c057000 con 0x7efd84143240 2023-12-01T03:11:31.655 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T03:11:31.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.656+0000 7efd8a25f640 1 -- 172.21.15.115:0/1085308752 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a6a0 msgr2=0x7efd6406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:31.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.656+0000 7efd8a25f640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a6a0 0x7efd6406cb60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0011e0 tx=0x7efd6c009040 comp rx=0 tx=0).stop 2023-12-01T03:11:31.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.656+0000 7efd8a25f640 1 -- 172.21.15.115:0/1085308752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd84143240 msgr2=0x7efd84133d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:31.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.656+0000 7efd8a25f640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd84143240 0x7efd84133d80 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7efd7c001eb0 tx=0x7efd7c00c890 comp rx=0 tx=0).stop 2023-12-01T03:11:31.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.659+0000 7efd8a25f640 1 -- 172.21.15.115:0/1085308752 shutdown_connections 2023-12-01T03:11:31.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.659+0000 7efd8a25f640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a6a0 0x7efd6406cb60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:31.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.659+0000 7efd8a25f640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd840ce0c0 0x7efd8413a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:31.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.659+0000 7efd8a25f640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd84143240 0x7efd84133d80 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:31.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.659+0000 7efd8a25f640 1 --2- 172.21.15.115:0/1085308752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd8406c390 0x7efd84132860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:31.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.659+0000 7efd8a25f640 1 -- 172.21.15.115:0/1085308752 >> 172.21.15.115:0/1085308752 conn(0x7efd84065960 msgr2=0x7efd8413bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:31.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.661+0000 7efd8a25f640 1 -- 172.21.15.115:0/1085308752 shutdown_connections 2023-12-01T03:11:31.666 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.663+0000 7efd8a25f640 1 -- 172.21.15.115:0/1085308752 wait complete. 2023-12-01T03:11:31.694 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T03:11:31.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.929+0000 7f352bfff640 1 Processor -- start 2023-12-01T03:11:31.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.929+0000 7f352bfff640 1 -- start start 2023-12-01T03:11:31.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f352bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f352c0cb600 0x7f352c0c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:31.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f352bfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f352c0ca100 0x7f352c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:31.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f352bfff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c070000 con 0x7f352c15cb60 2023-12-01T03:11:31.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f352bfff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352c0702e0 con 0x7f352c0cb600 2023-12-01T03:11:31.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f352bfff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352c0705c0 con 0x7f352c0ca100 2023-12-01T03:11:31.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f3523fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f352c0cb600 0x7f352c0c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:31.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f3523fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f352c0cb600 0x7f352c0c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38704/0 (socket says 172.21.15.115:38704) 2023-12-01T03:11:31.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f3523fff640 1 -- 172.21.15.115:0/1703731680 learned_addr learned my addr 172.21.15.115:0/1703731680 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:31.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f352b7fe640 1 --2- 172.21.15.115:0/1703731680 >> v2:172.21.15.142:3300/0 conn(0x7f352c0ca100 0x7f352c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:31.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f3523fff640 1 -- 172.21.15.115:0/1703731680 >> v2:172.21.15.142:3300/0 conn(0x7f352c0ca100 msgr2=0x7f352c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:31.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f3523fff640 1 --2- 172.21.15.115:0/1703731680 >> v2:172.21.15.142:3300/0 conn(0x7f352c0ca100 0x7f352c0ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:31.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f3523fff640 1 -- 172.21.15.115:0/1703731680 >> v1:172.21.15.142:6789/0 conn(0x7f352c15cb60 legacy=0x7f352c0caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:31.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.931+0000 7f3523fff640 1 -- 172.21.15.115:0/1703731680 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0708a0 con 0x7f352c0cb600 2023-12-01T03:11:31.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.932+0000 7f3523fff640 1 --2- 172.21.15.115:0/1703731680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c0cb600 0x7f352c0c9bc0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f352c0c5790 tx=0x7f351402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9fbbc61945fa97 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:31.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.933+0000 7f352a7fc640 1 -- 172.21.15.115:0/1703731680 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514031870 con 0x7f352c0cb600 2023-12-01T03:11:31.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.933+0000 7f352a7fc640 1 -- 172.21.15.115:0/1703731680 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35140319d0 con 0x7f352c0cb600 2023-12-01T03:11:31.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.933+0000 7f352a7fc640 1 -- 172.21.15.115:0/1703731680 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514031d60 con 0x7f352c0cb600 2023-12-01T03:11:31.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.933+0000 7f352bfff640 1 -- 172.21.15.115:0/1703731680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c0cb600 msgr2=0x7f352c0c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:31.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.933+0000 7f352bfff640 1 --2- 172.21.15.115:0/1703731680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c0cb600 0x7f352c0c9bc0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f352c0c5790 tx=0x7f351402b910 comp rx=0 tx=0).stop 2023-12-01T03:11:31.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.934+0000 7f352bfff640 1 -- 172.21.15.115:0/1703731680 shutdown_connections 2023-12-01T03:11:31.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.934+0000 7f352bfff640 1 --2- 172.21.15.115:0/1703731680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c0cb600 0x7f352c0c9bc0 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:31.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.934+0000 7f352bfff640 1 --2- 172.21.15.115:0/1703731680 >> v2:172.21.15.142:3300/0 conn(0x7f352c0ca100 0x7f352c0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:31.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.934+0000 7f352bfff640 1 -- 172.21.15.115:0/1703731680 >> 172.21.15.115:0/1703731680 conn(0x7f352c065960 msgr2=0x7f352c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:31.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.934+0000 7f352bfff640 1 -- 172.21.15.115:0/1703731680 shutdown_connections 2023-12-01T03:11:31.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.934+0000 7f352bfff640 1 -- 172.21.15.115:0/1703731680 wait complete. 2023-12-01T03:11:31.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.936+0000 7f352bfff640 1 Processor -- start 2023-12-01T03:11:31.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.936+0000 7f352bfff640 1 -- start start 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.937+0000 7f352bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f352c0ca100 0x7f352c1f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.937+0000 7f352bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f352c0cb600 0x7f352c1ef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.937+0000 7f352bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c15cb60 0x7f352c153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.937+0000 7f352bfff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c1f2670 con 0x7f352c15cb60 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.937+0000 7f352bfff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c1f6a10 con 0x7f352c0ca100 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.937+0000 7f352bfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c15fd40 con 0x7f352c0cb600 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.937+0000 7f352b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c15cb60 0x7f352c153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.937+0000 7f352b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c15cb60 0x7f352c153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38714/0 (socket says 172.21.15.115:38714) 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.937+0000 7f352b7fe640 1 -- 172.21.15.115:0/3053315321 learned_addr learned my addr 172.21.15.115:0/3053315321 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.937+0000 7f352affd640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f352c0ca100 0x7f352c1f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:31.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f352b7fe640 1 -- 172.21.15.115:0/3053315321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f352c0cb600 msgr2=0x7f352c1ef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f352b7fe640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f352c0cb600 0x7f352c1ef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f352b7fe640 1 -- 172.21.15.115:0/3053315321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f352c0ca100 msgr2=0x7f352c1f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f352b7fe640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f352c0ca100 0x7f352c1f64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f352b7fe640 1 -- 172.21.15.115:0/3053315321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35140094d0 con 0x7f352c15cb60 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f352affd640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f352c0ca100 0x7f352c1f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f352b7fe640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c15cb60 0x7f352c153650 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f3524007e40 tx=0x7f352400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f35237fe640 1 -- 172.21.15.115:0/3053315321 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524014620 con 0x7f352c15cb60 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f35237fe640 1 -- 172.21.15.115:0/3053315321 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352400ee90 con 0x7f352c15cb60 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f35237fe640 1 -- 172.21.15.115:0/3053315321 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35240149e0 con 0x7f352c15cb60 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f352bfff640 1 -- 172.21.15.115:0/3053315321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c15e8b0 con 0x7f352c15cb60 2023-12-01T03:11:31.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.938+0000 7f352bfff640 1 -- 172.21.15.115:0/3053315321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c15ee70 con 0x7f352c15cb60 2023-12-01T03:11:31.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.939+0000 7f352bfff640 1 -- 172.21.15.115:0/3053315321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c1f27b0 con 0x7f352c15cb60 2023-12-01T03:11:31.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.941+0000 7f35237fe640 1 -- 172.21.15.115:0/3053315321 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f3524014b40 con 0x7f352c15cb60 2023-12-01T03:11:31.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.943+0000 7f35237fe640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f350c06a670 0x7f350c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:31.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.943+0000 7f35237fe640 1 -- 172.21.15.115:0/3053315321 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 5653+0+0 (secure 0 0 0) 0x7f352408a260 con 0x7f352c15cb60 2023-12-01T03:11:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.943+0000 7f352affd640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f350c06a670 0x7f350c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:31.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.944+0000 7f352affd640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f350c06a670 0x7f350c06cb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f352c0c8410 tx=0x7f351c008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:31.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:31.947+0000 7f35237fe640 1 -- 172.21.15.115:0/3053315321 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f35240556d0 con 0x7f352c15cb60 2023-12-01T03:11:32.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.174+0000 7f352bfff640 1 -- 172.21.15.115:0/3053315321 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f352c1f27b0 con 0x7f352c15cb60 2023-12-01T03:11:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.186+0000 7f35237fe640 1 -- 172.21.15.115:0/3053315321 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 28 and data pool 29 v107) v1 ==== 174+0+0 (secure 0 0 0) 0x7f352c1f27b0 con 0x7f352c15cb60 2023-12-01T03:11:32.188 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 28 and data pool 29 2023-12-01T03:11:32.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.190+0000 7f352bfff640 1 -- 172.21.15.115:0/3053315321 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f350c06a670 msgr2=0x7f350c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.190+0000 7f352bfff640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f350c06a670 0x7f350c06cb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f352c0c8410 tx=0x7f351c008040 comp rx=0 tx=0).stop 2023-12-01T03:11:32.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.190+0000 7f352bfff640 1 -- 172.21.15.115:0/3053315321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c15cb60 msgr2=0x7f352c153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.192 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.190+0000 7f352bfff640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c15cb60 0x7f352c153650 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f3524007e40 tx=0x7f352400e4b0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.191+0000 7f352bfff640 1 -- 172.21.15.115:0/3053315321 shutdown_connections 2023-12-01T03:11:32.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.191+0000 7f352bfff640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f350c06a670 0x7f350c06cb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.191+0000 7f352bfff640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f352c0cb600 0x7f352c1ef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.191+0000 7f352bfff640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f352c15cb60 0x7f352c153650 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.191+0000 7f352bfff640 1 --2- 172.21.15.115:0/3053315321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f352c0ca100 0x7f352c1f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.191+0000 7f352bfff640 1 -- 172.21.15.115:0/3053315321 >> 172.21.15.115:0/3053315321 conn(0x7f352c065960 msgr2=0x7f352c0ad420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:32.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.193+0000 7f352bfff640 1 -- 172.21.15.115:0/3053315321 shutdown_connections 2023-12-01T03:11:32.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.193+0000 7f352bfff640 1 -- 172.21.15.115:0/3053315321 wait complete. 2023-12-01T03:11:32.228 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.398+0000 7f85c1cec640 1 Processor -- start 2023-12-01T03:11:32.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.398+0000 7f85c1cec640 1 -- start start 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85c1cec640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85bc15db90 0x7f85bc0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85c1cec640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f85bc0cddc0 0x7f85bc0d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85c1cec640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc0cd3f0 con 0x7f85bc15d7b0 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85c1cec640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc15dff0 con 0x7f85bc0cddc0 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85c1cec640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0cd0f0 con 0x7f85bc15db90 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85c0cea640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f85bc15d7b0 0x7f85bc160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56948/0 (socket says 172.21.15.115:56948) 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85c0cea640 1 -- 172.21.15.115:0/1426710861 learned_addr learned my addr 172.21.15.115:0/1426710861 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85c14eb640 1 --2- 172.21.15.115:0/1426710861 >> v2:172.21.15.115:3300/0 conn(0x7f85bc0cddc0 0x7f85bc0d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85b37fe640 1 -- 172.21.15.115:0/1426710861 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480764374 0 0) 0x7f85bc0cd3f0 con 0x7f85bc15d7b0 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85b37fe640 1 -- 172.21.15.115:0/1426710861 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a4003520 con 0x7f85bc15d7b0 2023-12-01T03:11:32.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.400+0000 7f85b3fff640 1 --2- 172.21.15.115:0/1426710861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85bc15db90 0x7f85bc0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:32.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.401+0000 7f85b37fe640 1 -- 172.21.15.115:0/1426710861 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2850007607 0 0) 0x7f85a4003520 con 0x7f85bc15d7b0 2023-12-01T03:11:32.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.401+0000 7f85c14eb640 1 -- 172.21.15.115:0/1426710861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85bc15db90 msgr2=0x7f85bc0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:32.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.401+0000 7f85c14eb640 1 --2- 172.21.15.115:0/1426710861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85bc15db90 0x7f85bc0cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.401+0000 7f85c14eb640 1 -- 172.21.15.115:0/1426710861 >> v1:172.21.15.115:6789/0 conn(0x7f85bc15d7b0 legacy=0x7f85bc160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.401+0000 7f85c14eb640 1 -- 172.21.15.115:0/1426710861 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc160ec0 con 0x7f85bc0cddc0 2023-12-01T03:11:32.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.402+0000 7f85c14eb640 1 --2- 172.21.15.115:0/1426710861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc0cddc0 0x7f85bc0d05d0 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009d90 tx=0x7f85b402be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb1859916b04723 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.402+0000 7f85b37fe640 1 -- 172.21.15.115:0/1426710861 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8001670 con 0x7f85bc0cddc0 2023-12-01T03:11:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.402+0000 7f85b37fe640 1 -- 172.21.15.115:0/1426710861 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b402c930 con 0x7f85bc0cddc0 2023-12-01T03:11:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.402+0000 7f85b37fe640 1 -- 172.21.15.115:0/1426710861 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b402ccc0 con 0x7f85bc0cddc0 2023-12-01T03:11:32.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.403+0000 7f85c1cec640 1 -- 172.21.15.115:0/1426710861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc0cddc0 msgr2=0x7f85bc0d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.403+0000 7f85c1cec640 1 --2- 172.21.15.115:0/1426710861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc0cddc0 0x7f85bc0d05d0 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009d90 tx=0x7f85b402be00 comp rx=0 tx=0).stop 2023-12-01T03:11:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.403+0000 7f85c1cec640 1 -- 172.21.15.115:0/1426710861 shutdown_connections 2023-12-01T03:11:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.403+0000 7f85c1cec640 1 --2- 172.21.15.115:0/1426710861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc0cddc0 0x7f85bc0d05d0 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.403+0000 7f85c1cec640 1 --2- 172.21.15.115:0/1426710861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85bc15db90 0x7f85bc0cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.403+0000 7f85c1cec640 1 -- 172.21.15.115:0/1426710861 >> 172.21.15.115:0/1426710861 conn(0x7f85bc06a0e0 msgr2=0x7f85bc06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.403+0000 7f85c1cec640 1 -- 172.21.15.115:0/1426710861 shutdown_connections 2023-12-01T03:11:32.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.403+0000 7f85c1cec640 1 -- 172.21.15.115:0/1426710861 wait complete. 2023-12-01T03:11:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.404+0000 7f85c1cec640 1 Processor -- start 2023-12-01T03:11:32.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.405+0000 7f85c1cec640 1 -- start start 2023-12-01T03:11:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.405+0000 7f85c1cec640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85bc0cddc0 0x7f85bc0b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.405+0000 7f85c1cec640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc15d7b0 0x7f85bc0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c1cec640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc15db90 0x7f85bc0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c1cec640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0cb470 con 0x7f85bc15db90 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c1cec640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0cb750 con 0x7f85bc15d7b0 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c1cec640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0cba30 con 0x7f85bc0cddc0 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c14eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc15db90 0x7f85bc0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c14eb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc15db90 0x7f85bc0b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38742/0 (socket says 172.21.15.115:38742) 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c14eb640 1 -- 172.21.15.115:0/582883624 learned_addr learned my addr 172.21.15.115:0/582883624 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c14eb640 1 -- 172.21.15.115:0/582883624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85bc0cddc0 msgr2=0x7f85bc0b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c14eb640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85bc0cddc0 0x7f85bc0b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c14eb640 1 -- 172.21.15.115:0/582883624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc15d7b0 msgr2=0x7f85bc0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c14eb640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc15d7b0 0x7f85bc0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c14eb640 1 -- 172.21.15.115:0/582883624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4009a70 con 0x7f85bc15db90 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.406+0000 7f85c14eb640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc15db90 0x7f85bc0b23f0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009d90 tx=0x7f85b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.407+0000 7f85b17fa640 1 -- 172.21.15.115:0/582883624 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4009450 con 0x7f85bc15db90 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.407+0000 7f85b17fa640 1 -- 172.21.15.115:0/582883624 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b4040450 con 0x7f85bc15db90 2023-12-01T03:11:32.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.407+0000 7f85c1cec640 1 -- 172.21.15.115:0/582883624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc143f00 con 0x7f85bc15db90 2023-12-01T03:11:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.407+0000 7f85b17fa640 1 -- 172.21.15.115:0/582883624 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4040800 con 0x7f85bc15db90 2023-12-01T03:11:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.407+0000 7f85c1cec640 1 -- 172.21.15.115:0/582883624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc0cb240 con 0x7f85bc15db90 2023-12-01T03:11:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.408+0000 7f85c1cec640 1 -- 172.21.15.115:0/582883624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc143d60 con 0x7f85bc15db90 2023-12-01T03:11:32.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.409+0000 7f85b17fa640 1 -- 172.21.15.115:0/582883624 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f85b4040960 con 0x7f85bc15db90 2023-12-01T03:11:32.413 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.410+0000 7f85b17fa640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f85a806a670 0x7f85a806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.410+0000 7f85b17fa640 1 -- 172.21.15.115:0/582883624 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f85b40aba30 con 0x7f85bc15db90 2023-12-01T03:11:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.410+0000 7f85c0cea640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f85a806a670 0x7f85a806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:32.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.410+0000 7f85c0cea640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f85a806a670 0x7f85a806cb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001eb0 tx=0x7f85b8001a20 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:32.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.414+0000 7f85b17fa640 1 -- 172.21.15.115:0/582883624 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f85bc143d60 con 0x7f85bc15db90 2023-12-01T03:11:32.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.609+0000 7f85c1cec640 1 -- 172.21.15.115:0/582883624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85bc143d60 con 0x7f85bc15db90 2023-12-01T03:11:32.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.611+0000 7f85b17fa640 1 -- 172.21.15.115:0/582883624 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v234) v1 ==== 74+0+16246 (secure 0 0 0) 0x7f85bc143d60 con 0x7f85bc15db90 2023-12-01T03:11:32.613 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:32.613 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":234,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:32.175969+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:32.617 INFO:tasks.ceph.mds.d.smithi142.stderr:2023-12-01T03:11:32.613+0000 7fc6116fc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:11:32.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 -- 172.21.15.115:0/582883624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f85a806a670 msgr2=0x7f85a806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f85a806a670 0x7f85a806cb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85b8001eb0 tx=0x7f85b8001a20 comp rx=0 tx=0).stop 2023-12-01T03:11:32.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 -- 172.21.15.115:0/582883624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc15db90 msgr2=0x7f85bc0b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc15db90 0x7f85bc0b23f0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009d90 tx=0x7f85b4004a40 comp rx=0 tx=0).stop 2023-12-01T03:11:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 -- 172.21.15.115:0/582883624 shutdown_connections 2023-12-01T03:11:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f85a806a670 0x7f85a806cb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85bc15db90 0x7f85bc0b23f0 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85bc15d7b0 0x7f85bc0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 --2- 172.21.15.115:0/582883624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f85bc0cddc0 0x7f85bc0b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 -- 172.21.15.115:0/582883624 >> 172.21.15.115:0/582883624 conn(0x7f85bc06a0e0 msgr2=0x7f85bc15fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 -- 172.21.15.115:0/582883624 shutdown_connections 2023-12-01T03:11:32.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.614+0000 7f85c1cec640 1 -- 172.21.15.115:0/582883624 wait complete. 2023-12-01T03:11:32.635 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:32.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.827+0000 7f37013e1640 1 Processor -- start 2023-12-01T03:11:32.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.827+0000 7f37013e1640 1 -- start start 2023-12-01T03:11:32.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f37013e1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36fc1584b0 0x7f36fc0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f37013e1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f36fc06b980 0x7f36fc0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f37013e1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36fc0cc180 con 0x7f36fc15afe0 2023-12-01T03:11:32.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f37013e1640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36fc15b550 con 0x7f36fc06b980 2023-12-01T03:11:32.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f37013e1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc15b830 con 0x7f36fc1584b0 2023-12-01T03:11:32.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f3700be0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f36fc06b980 0x7f36fc0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:32.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f3700be0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f36fc06b980 0x7f36fc0cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38750/0 (socket says 172.21.15.115:38750) 2023-12-01T03:11:32.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f3700be0640 1 -- 172.21.15.115:0/2314810477 learned_addr learned my addr 172.21.15.115:0/2314810477 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:32.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f3700be0640 1 -- 172.21.15.115:0/2314810477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36fc1584b0 msgr2=0x7f36fc0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:32.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f3700be0640 1 --2- 172.21.15.115:0/2314810477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36fc1584b0 0x7f36fc0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f3700be0640 1 -- 172.21.15.115:0/2314810477 >> v1:172.21.15.142:6789/0 conn(0x7f36fc15afe0 legacy=0x7f36fc06b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:32.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.829+0000 7f3700be0640 1 -- 172.21.15.115:0/2314810477 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc15bb10 con 0x7f36fc06b980 2023-12-01T03:11:32.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.830+0000 7f3700be0640 1 --2- 172.21.15.115:0/2314810477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc06b980 0x7f36fc0cc680 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009e50 tx=0x7f36f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfd86fb735e74cf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:32.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.831+0000 7f36faffd640 1 -- 172.21.15.115:0/2314810477 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f402ca70 con 0x7f36fc06b980 2023-12-01T03:11:32.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.831+0000 7f36faffd640 1 -- 172.21.15.115:0/2314810477 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f402cbd0 con 0x7f36fc06b980 2023-12-01T03:11:32.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.831+0000 7f36faffd640 1 -- 172.21.15.115:0/2314810477 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f40343f0 con 0x7f36fc06b980 2023-12-01T03:11:32.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.831+0000 7f37013e1640 1 -- 172.21.15.115:0/2314810477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc06b980 msgr2=0x7f36fc0cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.831+0000 7f37013e1640 1 --2- 172.21.15.115:0/2314810477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc06b980 0x7f36fc0cc680 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009e50 tx=0x7f36f402bed0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.832+0000 7f37013e1640 1 -- 172.21.15.115:0/2314810477 shutdown_connections 2023-12-01T03:11:32.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.832+0000 7f37013e1640 1 --2- 172.21.15.115:0/2314810477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc06b980 0x7f36fc0cc680 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.832+0000 7f37013e1640 1 --2- 172.21.15.115:0/2314810477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36fc1584b0 0x7f36fc0cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.832+0000 7f37013e1640 1 -- 172.21.15.115:0/2314810477 >> 172.21.15.115:0/2314810477 conn(0x7f36fc0ce640 msgr2=0x7f36fc0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:32.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.832+0000 7f37013e1640 1 -- 172.21.15.115:0/2314810477 shutdown_connections 2023-12-01T03:11:32.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.832+0000 7f37013e1640 1 -- 172.21.15.115:0/2314810477 wait complete. 2023-12-01T03:11:32.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.834+0000 7f37013e1640 1 Processor -- start 2023-12-01T03:11:32.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.834+0000 7f37013e1640 1 -- start start 2023-12-01T03:11:32.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.835+0000 7f37013e1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36fc06b980 0x7f36fc1efff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.835+0000 7f37013e1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc1584b0 0x7f36fc0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.835+0000 7f37013e1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36fc15afe0 0x7f36fc1f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.835+0000 7f37013e1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc0b1a20 con 0x7f36fc1584b0 2023-12-01T03:11:32.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.835+0000 7f37013e1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc1f0840 con 0x7f36fc15afe0 2023-12-01T03:11:32.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.835+0000 7f37013e1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc1f0b20 con 0x7f36fc06b980 2023-12-01T03:11:32.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f36fb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc1584b0 0x7f36fc0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:32.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f36fb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc1584b0 0x7f36fc0b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38752/0 (socket says 172.21.15.115:38752) 2023-12-01T03:11:32.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f36fb7fe640 1 -- 172.21.15.115:0/2109612803 learned_addr learned my addr 172.21.15.115:0/2109612803 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:32.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f3700be0640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36fc15afe0 0x7f36fc1f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:32.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f36fbfff640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36fc06b980 0x7f36fc1efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:32.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f36fb7fe640 1 -- 172.21.15.115:0/2109612803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36fc06b980 msgr2=0x7f36fc1efff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f36fb7fe640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36fc06b980 0x7f36fc1efff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f36fb7fe640 1 -- 172.21.15.115:0/2109612803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36fc15afe0 msgr2=0x7f36fc1f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:32.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f36fb7fe640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36fc15afe0 0x7f36fc1f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:32.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f36fb7fe640 1 -- 172.21.15.115:0/2109612803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f4009b30 con 0x7f36fc1584b0 2023-12-01T03:11:32.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.836+0000 7f36fb7fe640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc1584b0 0x7f36fc0b3df0 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f36f000c410 tx=0x7f36f000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:32.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.837+0000 7f36f8ff9640 1 -- 172.21.15.115:0/2109612803 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f00035f0 con 0x7f36fc1584b0 2023-12-01T03:11:32.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.837+0000 7f36f8ff9640 1 -- 172.21.15.115:0/2109612803 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f0003750 con 0x7f36fc1584b0 2023-12-01T03:11:32.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.837+0000 7f36f8ff9640 1 -- 172.21.15.115:0/2109612803 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0003b40 con 0x7f36fc1584b0 2023-12-01T03:11:32.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.837+0000 7f37013e1640 1 -- 172.21.15.115:0/2109612803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc1f0e00 con 0x7f36fc1584b0 2023-12-01T03:11:32.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.837+0000 7f37013e1640 1 -- 172.21.15.115:0/2109612803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc1f0530 con 0x7f36fc1584b0 2023-12-01T03:11:32.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.838+0000 7f37013e1640 1 -- 172.21.15.115:0/2109612803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36fc0b1e70 con 0x7f36fc1584b0 2023-12-01T03:11:32.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.839+0000 7f36f8ff9640 1 -- 172.21.15.115:0/2109612803 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f36f0003ca0 con 0x7f36fc1584b0 2023-12-01T03:11:32.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.840+0000 7f36f8ff9640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f36e806a670 0x7f36e806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:32.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.840+0000 7f36f8ff9640 1 -- 172.21.15.115:0/2109612803 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f36f0087fc0 con 0x7f36fc1584b0 2023-12-01T03:11:32.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.840+0000 7f36fbfff640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f36e806a670 0x7f36e806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:32.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.841+0000 7f36fbfff640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f36e806a670 0x7f36e806cb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36fc1edbe0 tx=0x7f36ec008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:32.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:32.844+0000 7f36f8ff9640 1 -- 172.21.15.115:0/2109612803 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f36fc0b1e70 con 0x7f36fc1584b0 2023-12-01T03:11:33.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.019+0000 7f37013e1640 1 -- 172.21.15.115:0/2109612803 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36fc0b1e70 con 0x7f36fc1584b0 2023-12-01T03:11:33.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.022+0000 7f36f8ff9640 1 -- 172.21.15.115:0/2109612803 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v234) v1 ==== 74+0+16246 (secure 0 0 0) 0x7f36fc0b1e70 con 0x7f36fc1584b0 2023-12-01T03:11:33.024 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:33.025 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":234,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:32.175969+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:33.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 -- 172.21.15.115:0/2109612803 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f36e806a670 msgr2=0x7f36e806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:33.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f36e806a670 0x7f36e806cb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36fc1edbe0 tx=0x7f36ec008040 comp rx=0 tx=0).stop 2023-12-01T03:11:33.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 -- 172.21.15.115:0/2109612803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc1584b0 msgr2=0x7f36fc0b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:33.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc1584b0 0x7f36fc0b3df0 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f36f000c410 tx=0x7f36f000c8d0 comp rx=0 tx=0).stop 2023-12-01T03:11:33.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 -- 172.21.15.115:0/2109612803 shutdown_connections 2023-12-01T03:11:33.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f36e806a670 0x7f36e806cb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:33.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36fc15afe0 0x7f36fc1f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:33.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36fc1584b0 0x7f36fc0b3df0 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:33.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 --2- 172.21.15.115:0/2109612803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36fc06b980 0x7f36fc1efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:33.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 -- 172.21.15.115:0/2109612803 >> 172.21.15.115:0/2109612803 conn(0x7f36fc0ce640 msgr2=0x7f36fc0c3730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 -- 172.21.15.115:0/2109612803 shutdown_connections 2023-12-01T03:11:33.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.025+0000 7f37013e1640 1 -- 172.21.15.115:0/2109612803 wait complete. 2023-12-01T03:11:33.045 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T03:11:33.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.253+0000 7f01dae5b640 1 Processor -- start 2023-12-01T03:11:33.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.253+0000 7f01dae5b640 1 -- start start 2023-12-01T03:11:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.255+0000 7f01dae5b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f01d40b04c0 0x7f01d40cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.255+0000 7f01dae5b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d40cbaa0 con 0x7f01d40cb630 2023-12-01T03:11:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.255+0000 7f01dae5b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d415afe0 con 0x7f01d415ac00 2023-12-01T03:11:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.255+0000 7f01dae5b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d40b12b0 con 0x7f01d40b04c0 2023-12-01T03:11:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.255+0000 7f01d9658640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f01d40b04c0 0x7f01d40cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.255+0000 7f01da65a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f01d40cb630 0x7f01d406fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56960/0 (socket says 172.21.15.115:56960) 2023-12-01T03:11:33.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.255+0000 7f01da65a640 1 -- 172.21.15.115:0/3138061069 learned_addr learned my addr 172.21.15.115:0/3138061069 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.256+0000 7f01d9658640 1 -- 172.21.15.115:0/3138061069 >> v1:172.21.15.142:6789/0 conn(0x7f01d415ac00 legacy=0x7f01d40ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.256+0000 7f01d9658640 1 -- 172.21.15.115:0/3138061069 >> v1:172.21.15.115:6789/0 conn(0x7f01d40cb630 legacy=0x7f01d406fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:33.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.256+0000 7f01d9658640 1 -- 172.21.15.115:0/3138061069 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d40b0a10 con 0x7f01d40b04c0 2023-12-01T03:11:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.257+0000 7f01d9658640 1 --2- 172.21.15.115:0/3138061069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40b04c0 0x7f01d40cb0f0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f01c40097a0 tx=0x7f01c402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0bde132c129e0c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.257+0000 7f01d8e57640 1 -- 172.21.15.115:0/3138061069 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c402e070 con 0x7f01d40b04c0 2023-12-01T03:11:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.257+0000 7f01d8e57640 1 -- 172.21.15.115:0/3138061069 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c40315a0 con 0x7f01d40b04c0 2023-12-01T03:11:33.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.257+0000 7f01d8e57640 1 -- 172.21.15.115:0/3138061069 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4031930 con 0x7f01d40b04c0 2023-12-01T03:11:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.258+0000 7f01dae5b640 1 -- 172.21.15.115:0/3138061069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40b04c0 msgr2=0x7f01d40cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.258+0000 7f01dae5b640 1 --2- 172.21.15.115:0/3138061069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40b04c0 0x7f01d40cb0f0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f01c40097a0 tx=0x7f01c402b840 comp rx=0 tx=0).stop 2023-12-01T03:11:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.258+0000 7f01dae5b640 1 -- 172.21.15.115:0/3138061069 shutdown_connections 2023-12-01T03:11:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.258+0000 7f01dae5b640 1 --2- 172.21.15.115:0/3138061069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40b04c0 0x7f01d40cb0f0 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.258+0000 7f01dae5b640 1 -- 172.21.15.115:0/3138061069 >> 172.21.15.115:0/3138061069 conn(0x7f01d4065960 msgr2=0x7f01d4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.258+0000 7f01dae5b640 1 -- 172.21.15.115:0/3138061069 shutdown_connections 2023-12-01T03:11:33.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.258+0000 7f01dae5b640 1 -- 172.21.15.115:0/3138061069 wait complete. 2023-12-01T03:11:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.260+0000 7f01dae5b640 1 Processor -- start 2023-12-01T03:11:33.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.260+0000 7f01dae5b640 1 -- start start 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01dae5b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01d40b04c0 0x7f01d415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01dae5b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40cb630 0x7f01d41f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01dae5b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01d415ac00 0x7f01d40c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01dae5b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d41f3930 con 0x7f01d40cb630 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01dae5b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d41f3bd0 con 0x7f01d415ac00 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01dae5b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d41f3eb0 con 0x7f01d40b04c0 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01d9658640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40cb630 0x7f01d41f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01d9658640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40cb630 0x7f01d41f33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38766/0 (socket says 172.21.15.115:38766) 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01d9658640 1 -- 172.21.15.115:0/4172680607 learned_addr learned my addr 172.21.15.115:0/4172680607 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01da65a640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01d415ac00 0x7f01d40c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01d9658640 1 -- 172.21.15.115:0/4172680607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01d40b04c0 msgr2=0x7f01d415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01d9e59640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01d40b04c0 0x7f01d415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01d9658640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01d40b04c0 0x7f01d415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:33.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01d9658640 1 -- 172.21.15.115:0/4172680607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01d415ac00 msgr2=0x7f01d40c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:33.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01d9658640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01d415ac00 0x7f01d40c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:33.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01d9658640 1 -- 172.21.15.115:0/4172680607 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c4009400 con 0x7f01d40cb630 2023-12-01T03:11:33.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.261+0000 7f01d9e59640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01d40b04c0 0x7f01d415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:33.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.262+0000 7f01d9658640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40cb630 0x7f01d41f33f0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f01c4034040 tx=0x7f01c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:33.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.262+0000 7f01ca7fc640 1 -- 172.21.15.115:0/4172680607 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4032900 con 0x7f01d40cb630 2023-12-01T03:11:33.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.262+0000 7f01ca7fc640 1 -- 172.21.15.115:0/4172680607 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c4032a80 con 0x7f01d40cb630 2023-12-01T03:11:33.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.262+0000 7f01ca7fc640 1 -- 172.21.15.115:0/4172680607 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4032e30 con 0x7f01d40cb630 2023-12-01T03:11:33.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.262+0000 7f01dae5b640 1 -- 172.21.15.115:0/4172680607 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d41f4190 con 0x7f01d40cb630 2023-12-01T03:11:33.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.262+0000 7f01dae5b640 1 -- 172.21.15.115:0/4172680607 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d41f4750 con 0x7f01d40cb630 2023-12-01T03:11:33.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.263+0000 7f01dae5b640 1 -- 172.21.15.115:0/4172680607 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d41f4940 con 0x7f01d40cb630 2023-12-01T03:11:33.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.264+0000 7f01ca7fc640 1 -- 172.21.15.115:0/4172680607 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f01c403b3f0 con 0x7f01d40cb630 2023-12-01T03:11:33.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.266+0000 7f01ca7fc640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01b806a670 0x7f01b806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:33.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.266+0000 7f01ca7fc640 1 -- 172.21.15.115:0/4172680607 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f01c40aafc0 con 0x7f01d40cb630 2023-12-01T03:11:33.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.266+0000 7f01d9e59640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01b806a670 0x7f01b806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:33.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.267+0000 7f01d9e59640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01b806a670 0x7f01b806cb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01d000a830 tx=0x7f01d0008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:33.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.270+0000 7f01ca7fc640 1 -- 172.21.15.115:0/4172680607 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f01c4076070 con 0x7f01d40cb630 2023-12-01T03:11:33.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:33.495+0000 7f01dae5b640 1 -- 172.21.15.115:0/4172680607 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f01d40618d0 con 0x7f01d40cb630 2023-12-01T03:11:34.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.191+0000 7f01ca7fc640 1 -- 172.21.15.115:0/4172680607 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v109) v1 ==== 118+0+0 (secure 0 0 0) 0x7f01c4079690 con 0x7f01d40cb630 2023-12-01T03:11:34.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.194+0000 7f01dae5b640 1 -- 172.21.15.115:0/4172680607 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01b806a670 msgr2=0x7f01b806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:34.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.194+0000 7f01dae5b640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01b806a670 0x7f01b806cb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01d000a830 tx=0x7f01d0008040 comp rx=0 tx=0).stop 2023-12-01T03:11:34.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.194+0000 7f01dae5b640 1 -- 172.21.15.115:0/4172680607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40cb630 msgr2=0x7f01d41f33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:34.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.194+0000 7f01dae5b640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40cb630 0x7f01d41f33f0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f01c4034040 tx=0x7f01c40046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.194+0000 7f01dae5b640 1 -- 172.21.15.115:0/4172680607 shutdown_connections 2023-12-01T03:11:34.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.194+0000 7f01dae5b640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f01b806a670 0x7f01b806cb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.194+0000 7f01dae5b640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01d415ac00 0x7f01d40c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.194+0000 7f01dae5b640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01d40cb630 0x7f01d41f33f0 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.194+0000 7f01dae5b640 1 --2- 172.21.15.115:0/4172680607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f01d40b04c0 0x7f01d415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.194+0000 7f01dae5b640 1 -- 172.21.15.115:0/4172680607 >> 172.21.15.115:0/4172680607 conn(0x7f01d4065960 msgr2=0x7f01d4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:34.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.195+0000 7f01dae5b640 1 -- 172.21.15.115:0/4172680607 shutdown_connections 2023-12-01T03:11:34.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.195+0000 7f01dae5b640 1 -- 172.21.15.115:0/4172680607 wait complete. 2023-12-01T03:11:34.217 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T03:11:34.218 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T03:11:34.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.432+0000 7f4ab27cb640 1 Processor -- start 2023-12-01T03:11:34.434 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.432+0000 7f4ab27cb640 1 -- start start 2023-12-01T03:11:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.434+0000 7f4ab27cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0ce1b0 0x7f4aac0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.434+0000 7f4ab27cb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac0cc9a0 con 0x7f4aac15ac80 2023-12-01T03:11:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.434+0000 7f4ab27cb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac0cbd90 con 0x7f4aac0b0e60 2023-12-01T03:11:34.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.434+0000 7f4ab27cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac15b060 con 0x7f4aac0ce1b0 2023-12-01T03:11:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.434+0000 7f4aabfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4aac15ac80 0x7f4aac0cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56972/0 (socket says 172.21.15.115:56972) 2023-12-01T03:11:34.436 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.434+0000 7f4aabfff640 1 -- 172.21.15.115:0/4184672333 learned_addr learned my addr 172.21.15.115:0/4184672333 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.434+0000 7f4aa3fff640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0ce1b0 0x7f4aac0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.434+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322218781 0 0) 0x7f4aac0cc9a0 con 0x7f4aac15ac80 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.434+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a98003520 con 0x7f4aac15ac80 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1374306220 0 0) 0x7f4a98003520 con 0x7f4aac15ac80 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0ce1b0 msgr2=0x7f4aac0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0ce1b0 0x7f4aac0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 >> v1:172.21.15.142:6789/0 conn(0x7f4aac0b0e60 legacy=0x7f4aac06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac0cc070 con 0x7f4aac15ac80 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f4a9c001670 con 0x7f4aac15ac80 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 >> v1:172.21.15.115:6789/0 conn(0x7f4aac15ac80 legacy=0x7f4aac0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0ce1b0 0x7f4a980039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a98003fd0 0x7f4a980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:34.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a98004d00 0x7f4a98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac15b060 con 0x7f4a98003fd0 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9800bb40 con 0x7f4a98004d00 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9800bd00 con 0x7f4aac0ce1b0 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aa3fff640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a98003fd0 0x7f4a980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4aabfff640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0ce1b0 0x7f4a980039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.435+0000 7f4ab0d68640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a98004d00 0x7f4a98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.436+0000 7f4aa3fff640 1 -- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0ce1b0 msgr2=0x7f4a980039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.436+0000 7f4aa3fff640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0ce1b0 0x7f4a980039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.436+0000 7f4aa3fff640 1 -- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a98004d00 msgr2=0x7f4a98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.436+0000 7f4aa3fff640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a98004d00 0x7f4a98005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.436+0000 7f4aa3fff640 1 -- 172.21.15.115:0/4184672333 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a98003520 con 0x7f4a98003fd0 2023-12-01T03:11:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.437+0000 7f4aa3fff640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a98003fd0 0x7f4a980047c0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400ba30 tx=0x7f4a980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af19414489b5ac19 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:34.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.437+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94032640 con 0x7f4a98003fd0 2023-12-01T03:11:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.437+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a940327a0 con 0x7f4a98003fd0 2023-12-01T03:11:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.437+0000 7f4aab7fe640 1 -- 172.21.15.115:0/4184672333 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94032b30 con 0x7f4a98003fd0 2023-12-01T03:11:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.438+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4184672333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a98003fd0 msgr2=0x7f4a980047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.438+0000 7f4ab27cb640 1 --2- 172.21.15.115:0/4184672333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a98003fd0 0x7f4a980047c0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400ba30 tx=0x7f4a980057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.438+0000 7f4ab0d68640 1 -- 172.21.15.115:0/4184672333 reap_dead start 2023-12-01T03:11:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.438+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4184672333 shutdown_connections 2023-12-01T03:11:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.438+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4184672333 >> 172.21.15.115:0/4184672333 conn(0x7f4aac065960 msgr2=0x7f4aac065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.438+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4184672333 shutdown_connections 2023-12-01T03:11:34.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.438+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4184672333 wait complete. 2023-12-01T03:11:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.440+0000 7f4ab27cb640 1 Processor -- start 2023-12-01T03:11:34.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.440+0000 7f4ab27cb640 1 -- start start 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab27cb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a98003fd0 0x7f4aac14b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab27cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0c4c50 0x7f4aac0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab27cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aac15d300 0x7f4aac15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab27cb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac15dcf0 con 0x7f4aac15d300 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab27cb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0a8580 con 0x7f4a98003fd0 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab27cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0c5c90 con 0x7f4aac0c4c50 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab0d68640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aac15d300 0x7f4aac15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab0d68640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aac15d300 0x7f4aac15d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38790/0 (socket says 172.21.15.115:38790) 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab0d68640 1 -- 172.21.15.115:0/4082480574 learned_addr learned my addr 172.21.15.115:0/4082480574 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4aabfff640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a98003fd0 0x7f4aac14b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab0d68640 1 -- 172.21.15.115:0/4082480574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0c4c50 msgr2=0x7f4aac0c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab0d68640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0c4c50 0x7f4aac0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab0d68640 1 -- 172.21.15.115:0/4082480574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a98003fd0 msgr2=0x7f4aac14b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab0d68640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a98003fd0 0x7f4aac14b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:34.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.441+0000 7f4ab0d68640 1 -- 172.21.15.115:0/4082480574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac14bbf0 con 0x7f4aac15d300 2023-12-01T03:11:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.442+0000 7f4ab0d68640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aac15d300 0x7f4aac15d7b0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0ca270 tx=0x7f4aa400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.442+0000 7f4aa97fa640 1 -- 172.21.15.115:0/4082480574 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4003970 con 0x7f4aac15d300 2023-12-01T03:11:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.442+0000 7f4aa97fa640 1 -- 172.21.15.115:0/4082480574 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa4003ad0 con 0x7f4aac15d300 2023-12-01T03:11:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.442+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4082480574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac0c56d0 con 0x7f4aac15d300 2023-12-01T03:11:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.442+0000 7f4aa97fa640 1 -- 172.21.15.115:0/4082480574 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4003e80 con 0x7f4aac15d300 2023-12-01T03:11:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.442+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4082480574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac0c66b0 con 0x7f4aac15d300 2023-12-01T03:11:34.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.443+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4082480574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac04e890 con 0x7f4aac15d300 2023-12-01T03:11:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.444+0000 7f4aa97fa640 1 -- 172.21.15.115:0/4082480574 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f4aa40050d0 con 0x7f4aac15d300 2023-12-01T03:11:34.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.445+0000 7f4aa97fa640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4a8c06a6a0 0x7f4a8c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:34.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.445+0000 7f4aa97fa640 1 -- 172.21.15.115:0/4082480574 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f4aa4088b20 con 0x7f4aac15d300 2023-12-01T03:11:34.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.445+0000 7f4aabfff640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4a8c06a6a0 0x7f4a8c06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:34.447 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.446+0000 7f4aabfff640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4a8c06a6a0 0x7f4a8c06cb60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c001eb0 tx=0x7f4a9c000ff0 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:34.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.449+0000 7f4aa97fa640 1 -- 172.21.15.115:0/4082480574 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4aa4053d70 con 0x7f4aac15d300 2023-12-01T03:11:34.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:34.678+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4082480574 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4aac15bab0 con 0x7f4aac15d300 2023-12-01T03:11:35.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.679+0000 7f4aa97fa640 1 -- 172.21.15.115:0/4082480574 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v110) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4aa4057390 con 0x7f4aac15d300 2023-12-01T03:11:35.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4082480574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4a8c06a6a0 msgr2=0x7f4a8c06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:35.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4a8c06a6a0 0x7f4a8c06cb60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c001eb0 tx=0x7f4a9c000ff0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4082480574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aac15d300 msgr2=0x7f4aac15d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:35.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aac15d300 0x7f4aac15d7b0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0ca270 tx=0x7f4aa400ec10 comp rx=0 tx=0).stop 2023-12-01T03:11:35.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4082480574 shutdown_connections 2023-12-01T03:11:35.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4a8c06a6a0 0x7f4a8c06cb60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4aac0c4c50 0x7f4aac0c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4aac15d300 0x7f4aac15d7b0 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 --2- 172.21.15.115:0/4082480574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a98003fd0 0x7f4aac14b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4082480574 >> 172.21.15.115:0/4082480574 conn(0x7f4aac065960 msgr2=0x7f4aac0708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:35.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4082480574 shutdown_connections 2023-12-01T03:11:35.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.683+0000 7f4ab27cb640 1 -- 172.21.15.115:0/4082480574 wait complete. 2023-12-01T03:11:35.704 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T03:11:35.704 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:35.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.909+0000 7fc892d73640 1 Processor -- start 2023-12-01T03:11:35.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.909+0000 7fc892d73640 1 -- start start 2023-12-01T03:11:35.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.910+0000 7fc892d73640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc88c15d220 0x7fc88c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:35.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.910+0000 7fc892d73640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 0x7fc88c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:35.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.910+0000 7fc892d73640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c15d600 con 0x7fc88c0ade80 2023-12-01T03:11:35.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.911+0000 7fc892d73640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c06cc10 con 0x7fc88c15d220 2023-12-01T03:11:35.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.911+0000 7fc892d73640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c06c720 con 0x7fc88c0ad2b0 2023-12-01T03:11:35.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.911+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 0x7fc88c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:35.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.911+0000 7fc890b0f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc88c15d220 0x7fc88c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:35.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.911+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 0x7fc88c0ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:49084/0 (socket says 172.21.15.115:49084) 2023-12-01T03:11:35.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.911+0000 7fc88bfff640 1 -- 172.21.15.115:0/2836571846 learned_addr learned my addr 172.21.15.115:0/2836571846 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:35.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.912+0000 7fc88b7fe640 1 -- 172.21.15.115:0/2836571846 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421680628 0 0) 0x7fc884001490 con 0x7fc88c0ade80 2023-12-01T03:11:35.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.912+0000 7fc88bfff640 1 -- 172.21.15.115:0/2836571846 >> v2:172.21.15.142:3300/0 conn(0x7fc88c15d220 msgr2=0x7fc88c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:35.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.912+0000 7fc88bfff640 1 --2- 172.21.15.115:0/2836571846 >> v2:172.21.15.142:3300/0 conn(0x7fc88c15d220 0x7fc88c0cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.912+0000 7fc88bfff640 1 -- 172.21.15.115:0/2836571846 >> v1:172.21.15.142:6789/0 conn(0x7fc88c0ade80 legacy=0x7fc88c06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:35.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.912+0000 7fc88bfff640 1 -- 172.21.15.115:0/2836571846 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c06c4b0 con 0x7fc88c0ad2b0 2023-12-01T03:11:35.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.914+0000 7fc88bfff640 1 --2- 172.21.15.115:0/2836571846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 0x7fc88c0ccab0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc878009870 tx=0x7fc87802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9108405da554d3d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:35.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.914+0000 7fc88b7fe640 1 -- 172.21.15.115:0/2836571846 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878031640 con 0x7fc88c0ad2b0 2023-12-01T03:11:35.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.914+0000 7fc88b7fe640 1 -- 172.21.15.115:0/2836571846 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8780317a0 con 0x7fc88c0ad2b0 2023-12-01T03:11:35.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.914+0000 7fc88b7fe640 1 -- 172.21.15.115:0/2836571846 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878031b30 con 0x7fc88c0ad2b0 2023-12-01T03:11:35.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.915+0000 7fc892d73640 1 -- 172.21.15.115:0/2836571846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 msgr2=0x7fc88c0ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:35.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.915+0000 7fc892d73640 1 --2- 172.21.15.115:0/2836571846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 0x7fc88c0ccab0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc878009870 tx=0x7fc87802b8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.915+0000 7fc892d73640 1 -- 172.21.15.115:0/2836571846 shutdown_connections 2023-12-01T03:11:35.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.915+0000 7fc892d73640 1 --2- 172.21.15.115:0/2836571846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 0x7fc88c0ccab0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.915+0000 7fc892d73640 1 --2- 172.21.15.115:0/2836571846 >> v2:172.21.15.142:3300/0 conn(0x7fc88c15d220 0x7fc88c0cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.915+0000 7fc892d73640 1 -- 172.21.15.115:0/2836571846 >> 172.21.15.115:0/2836571846 conn(0x7fc88c0ce880 msgr2=0x7fc88c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:35.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.915+0000 7fc892d73640 1 -- 172.21.15.115:0/2836571846 shutdown_connections 2023-12-01T03:11:35.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.915+0000 7fc892d73640 1 -- 172.21.15.115:0/2836571846 wait complete. 2023-12-01T03:11:35.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.918+0000 7fc892d73640 1 Processor -- start 2023-12-01T03:11:35.918 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.918+0000 7fc892d73640 1 -- start start 2023-12-01T03:11:35.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.919+0000 7fc892d73640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 0x7fc88c158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:35.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.919+0000 7fc892d73640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0ade80 0x7fc88c0ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:35.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.919+0000 7fc892d73640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c15d220 0x7fc88c0c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:35.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.919+0000 7fc892d73640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0b1e60 con 0x7fc88c0ade80 2023-12-01T03:11:35.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.919+0000 7fc892d73640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0cb160 con 0x7fc88c15d220 2023-12-01T03:11:35.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.919+0000 7fc892d73640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0cb440 con 0x7fc88c0ad2b0 2023-12-01T03:11:35.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.919+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0ade80 0x7fc88c0ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:35.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.919+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0ade80 0x7fc88c0ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38792/0 (socket says 172.21.15.115:38792) 2023-12-01T03:11:35.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.919+0000 7fc88bfff640 1 -- 172.21.15.115:0/3347768758 learned_addr learned my addr 172.21.15.115:0/3347768758 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:35.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.919+0000 7fc890b0f640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 0x7fc88c158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:35.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.920+0000 7fc891310640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c15d220 0x7fc88c0c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:35.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.920+0000 7fc88bfff640 1 -- 172.21.15.115:0/3347768758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 msgr2=0x7fc88c158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:35.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.920+0000 7fc88bfff640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 0x7fc88c158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.920+0000 7fc88bfff640 1 -- 172.21.15.115:0/3347768758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c15d220 msgr2=0x7fc88c0c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:35.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.920+0000 7fc88bfff640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c15d220 0x7fc88c0c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:35.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.920+0000 7fc88bfff640 1 -- 172.21.15.115:0/3347768758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780094d0 con 0x7fc88c0ade80 2023-12-01T03:11:35.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.920+0000 7fc88bfff640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0ade80 0x7fc88c0ca910 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7fc878000c00 tx=0x7fc8780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:35.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.920+0000 7fc8897fa640 1 -- 172.21.15.115:0/3347768758 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87803a930 con 0x7fc88c0ade80 2023-12-01T03:11:35.922 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.921+0000 7fc8897fa640 1 -- 172.21.15.115:0/3347768758 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87803aa90 con 0x7fc88c0ade80 2023-12-01T03:11:35.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.921+0000 7fc892d73640 1 -- 172.21.15.115:0/3347768758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c0cb720 con 0x7fc88c0ade80 2023-12-01T03:11:35.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.921+0000 7fc8897fa640 1 -- 172.21.15.115:0/3347768758 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87803ae40 con 0x7fc88c0ade80 2023-12-01T03:11:35.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.921+0000 7fc892d73640 1 -- 172.21.15.115:0/3347768758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c1591d0 con 0x7fc88c0ade80 2023-12-01T03:11:35.923 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.922+0000 7fc892d73640 1 -- 172.21.15.115:0/3347768758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c0cb8b0 con 0x7fc88c0ade80 2023-12-01T03:11:35.924 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.923+0000 7fc8897fa640 1 -- 172.21.15.115:0/3347768758 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fc87802f050 con 0x7fc88c0ade80 2023-12-01T03:11:35.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.924+0000 7fc8897fa640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc87006a6a0 0x7fc87006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:35.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.924+0000 7fc8897fa640 1 -- 172.21.15.115:0/3347768758 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fc8780aabe0 con 0x7fc88c0ade80 2023-12-01T03:11:35.925 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.924+0000 7fc890b0f640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc87006a6a0 0x7fc87006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:35.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.925+0000 7fc890b0f640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc87006a6a0 0x7fc87006cb60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc880009a30 tx=0x7fc880008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:35.929 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:35.929+0000 7fc8897fa640 1 -- 172.21.15.115:0/3347768758 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc878075d30 con 0x7fc88c0ade80 2023-12-01T03:11:36.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.156+0000 7fc892d73640 1 -- 172.21.15.115:0/3347768758 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc88c04e890 con 0x7fc88c0ade80 2023-12-01T03:11:36.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.157+0000 7fc8897fa640 1 -- 172.21.15.115:0/3347768758 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4658 (secure 0 0 0) 0x7fc878035090 con 0x7fc88c0ade80 2023-12-01T03:11:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:36.158 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":110,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:32.176438+0000","modified":"2023-12-01T03:11:35.672576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7735":{"gid":7735,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1837882056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1837882056},{"type":"v1","addr":"172.21.15.142:6835","nonce":1837882056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-12-01T03:11:36.158 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 110 2023-12-01T03:11:36.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 -- 172.21.15.115:0/3347768758 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc87006a6a0 msgr2=0x7fc87006cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc87006a6a0 0x7fc87006cb60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc880009a30 tx=0x7fc880008040 comp rx=0 tx=0).stop 2023-12-01T03:11:36.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 -- 172.21.15.115:0/3347768758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0ade80 msgr2=0x7fc88c0ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0ade80 0x7fc88c0ca910 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7fc878000c00 tx=0x7fc8780046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 -- 172.21.15.115:0/3347768758 shutdown_connections 2023-12-01T03:11:36.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc87006a6a0 0x7fc87006cb60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc88c15d220 0x7fc88c0c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc88c0ade80 0x7fc88c0ca910 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 --2- 172.21.15.115:0/3347768758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc88c0ad2b0 0x7fc88c158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 -- 172.21.15.115:0/3347768758 >> 172.21.15.115:0/3347768758 conn(0x7fc88c0ce880 msgr2=0x7fc88c159d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:36.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 -- 172.21.15.115:0/3347768758 shutdown_connections 2023-12-01T03:11:36.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.160+0000 7fc892d73640 1 -- 172.21.15.115:0/3347768758 wait complete. 2023-12-01T03:11:36.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:36.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.392+0000 7f51ae100640 1 Processor -- start 2023-12-01T03:11:36.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.393+0000 7f51ae100640 1 -- start start 2023-12-01T03:11:36.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.394+0000 7f51ae100640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a806b6f0 0x7f51a8070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.394+0000 7f51ae100640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a806f8b0 con 0x7f51a80c9ef0 2023-12-01T03:11:36.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.394+0000 7f51ae100640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a80adcb0 con 0x7f51a8159dd0 2023-12-01T03:11:36.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.394+0000 7f51ae100640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80ad9d0 con 0x7f51a806b6f0 2023-12-01T03:11:36.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.394+0000 7f519effd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f51a80c9ef0 0x7f51a806fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56978/0 (socket says 172.21.15.115:56978) 2023-12-01T03:11:36.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.394+0000 7f519effd640 1 -- 172.21.15.115:0/2443395271 learned_addr learned my addr 172.21.15.115:0/2443395271 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.394+0000 7f51a7fff640 1 --2- 172.21.15.115:0/2443395271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a806b6f0 0x7f51a8070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456360654 0 0) 0x7f51a806f8b0 con 0x7f51a80c9ef0 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5194003520 con 0x7f51a80c9ef0 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1833974834 0 0) 0x7f5194003520 con 0x7f51a80c9ef0 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a806b6f0 msgr2=0x7f51a8070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 --2- 172.21.15.115:0/2443395271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a806b6f0 0x7f51a8070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 >> v1:172.21.15.142:6789/0 conn(0x7f51a8159dd0 legacy=0x7f51a80c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a80c9000 con 0x7f51a80c9ef0 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5190001670 con 0x7f51a80c9ef0 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 >> v1:172.21.15.115:6789/0 conn(0x7f51a80c9ef0 legacy=0x7f51a806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 --2- 172.21.15.115:0/2443395271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5194003a10 0x7f5194003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 --2- 172.21.15.115:0/2443395271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a806b6f0 0x7f51940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 --2- 172.21.15.115:0/2443395271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5194004d00 0x7f5194005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80ad9d0 con 0x7f5194003a10 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519400ddc0 con 0x7f5194004d00 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519400df80 con 0x7f51a806b6f0 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.395+0000 7f51a77fe640 1 --2- 172.21.15.115:0/2443395271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5194003a10 0x7f5194003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.396+0000 7f51a77fe640 1 -- 172.21.15.115:0/2443395271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a806b6f0 msgr2=0x7f51940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.396+0000 7f51a77fe640 1 --2- 172.21.15.115:0/2443395271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a806b6f0 0x7f51940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.396+0000 7f51a77fe640 1 -- 172.21.15.115:0/2443395271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5194004d00 msgr2=0x7f5194005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.396+0000 7f51a77fe640 1 --2- 172.21.15.115:0/2443395271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5194004d00 0x7f5194005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.396+0000 7f51a77fe640 1 -- 172.21.15.115:0/2443395271 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194003520 con 0x7f5194003a10 2023-12-01T03:11:36.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.397+0000 7f51a77fe640 1 --2- 172.21.15.115:0/2443395271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5194003a10 0x7f5194003e30 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f5198001750 tx=0x7f51940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee087eaf274bf1e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:36.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.397+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980052d0 con 0x7f5194003a10 2023-12-01T03:11:36.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.397+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519802e050 con 0x7f5194003a10 2023-12-01T03:11:36.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.397+0000 7f51a6ffd640 1 -- 172.21.15.115:0/2443395271 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980035f0 con 0x7f5194003a10 2023-12-01T03:11:36.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.398+0000 7f51ae100640 1 -- 172.21.15.115:0/2443395271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5194003a10 msgr2=0x7f5194003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.398+0000 7f51ae100640 1 --2- 172.21.15.115:0/2443395271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5194003a10 0x7f5194003e30 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f5198001750 tx=0x7f51940057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.398+0000 7f51a7fff640 1 -- 172.21.15.115:0/2443395271 reap_dead start 2023-12-01T03:11:36.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.398+0000 7f51ae100640 1 -- 172.21.15.115:0/2443395271 shutdown_connections 2023-12-01T03:11:36.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.398+0000 7f51ae100640 1 -- 172.21.15.115:0/2443395271 >> 172.21.15.115:0/2443395271 conn(0x7f51a80ce880 msgr2=0x7f51a80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:36.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.398+0000 7f51ae100640 1 -- 172.21.15.115:0/2443395271 shutdown_connections 2023-12-01T03:11:36.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.398+0000 7f51ae100640 1 -- 172.21.15.115:0/2443395271 wait complete. 2023-12-01T03:11:36.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.400+0000 7f51ae100640 1 Processor -- start 2023-12-01T03:11:36.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.400+0000 7f51ae100640 1 -- start start 2023-12-01T03:11:36.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.401+0000 7f51ae100640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5194003a10 0x7f51a80c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.401+0000 7f51ae100640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a80ad370 0x7f51a8158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.401+0000 7f51ae100640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a8159080 0x7f51a80c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.401+0000 7f51ae100640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80c73b0 con 0x7f51a80ad370 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.401+0000 7f51ae100640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80c6ac0 con 0x7f5194003a10 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.401+0000 7f51ae100640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8159510 con 0x7f51a8159080 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.401+0000 7f519effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a80ad370 0x7f51a8158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.401+0000 7f519effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a80ad370 0x7f51a8158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38812/0 (socket says 172.21.15.115:38812) 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.401+0000 7f519effd640 1 -- 172.21.15.115:0/1416685718 learned_addr learned my addr 172.21.15.115:0/1416685718 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f519effd640 1 -- 172.21.15.115:0/1416685718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a8159080 msgr2=0x7f51a80c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f519effd640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a8159080 0x7f51a80c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f519effd640 1 -- 172.21.15.115:0/1416685718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5194003a10 msgr2=0x7f51a80c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f51a77fe640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5194003a10 0x7f51a80c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f519effd640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5194003a10 0x7f51a80c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f519effd640 1 -- 172.21.15.115:0/1416685718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a81f3490 con 0x7f51a80ad370 2023-12-01T03:11:36.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f519effd640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a80ad370 0x7f51a8158b40 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f519000c6b0 tx=0x7f519000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:36.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f51a4ff9640 1 -- 172.21.15.115:0/1416685718 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51900072d0 con 0x7f51a80ad370 2023-12-01T03:11:36.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f51a4ff9640 1 -- 172.21.15.115:0/1416685718 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5190007450 con 0x7f51a80ad370 2023-12-01T03:11:36.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f51a4ff9640 1 -- 172.21.15.115:0/1416685718 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51900047e0 con 0x7f51a80ad370 2023-12-01T03:11:36.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.402+0000 7f51ae100640 1 -- 172.21.15.115:0/1416685718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a80c6500 con 0x7f51a80ad370 2023-12-01T03:11:36.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.403+0000 7f51ae100640 1 -- 172.21.15.115:0/1416685718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a80c6fd0 con 0x7f51a80ad370 2023-12-01T03:11:36.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.404+0000 7f51ae100640 1 -- 172.21.15.115:0/1416685718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a80c74f0 con 0x7f51a80ad370 2023-12-01T03:11:36.405 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.405+0000 7f51a4ff9640 1 -- 172.21.15.115:0/1416685718 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f5190004960 con 0x7f51a80ad370 2023-12-01T03:11:36.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.406+0000 7f51a4ff9640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f518806a670 0x7f518806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.406+0000 7f51a4ff9640 1 -- 172.21.15.115:0/1416685718 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f5190087fc0 con 0x7f51a80ad370 2023-12-01T03:11:36.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.406+0000 7f51a77fe640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f518806a670 0x7f518806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.407+0000 7f51a77fe640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f518806a670 0x7f518806cb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5198033000 tx=0x7f5198009030 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:36.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.410+0000 7f51a4ff9640 1 -- 172.21.15.115:0/1416685718 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f51a80c74f0 con 0x7f51a80ad370 2023-12-01T03:11:36.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.597+0000 7f51ae100640 1 -- 172.21.15.115:0/1416685718 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51a80c74f0 con 0x7f51a80ad370 2023-12-01T03:11:36.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.599+0000 7f51a4ff9640 1 -- 172.21.15.115:0/1416685718 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v235) v1 ==== 74+0+16233 (secure 0 0 0) 0x7f51a80c74f0 con 0x7f51a80ad370 2023-12-01T03:11:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:36.600 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":235,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:33.180191+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 -- 172.21.15.115:0/1416685718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f518806a670 msgr2=0x7f518806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f518806a670 0x7f518806cb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5198033000 tx=0x7f5198009030 comp rx=0 tx=0).stop 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 -- 172.21.15.115:0/1416685718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a80ad370 msgr2=0x7f51a8158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a80ad370 0x7f51a8158b40 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f519000c6b0 tx=0x7f519000cb70 comp rx=0 tx=0).stop 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 -- 172.21.15.115:0/1416685718 shutdown_connections 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f518806a670 0x7f518806cb30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51a8159080 0x7f51a80c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a80ad370 0x7f51a8158b40 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 --2- 172.21.15.115:0/1416685718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5194003a10 0x7f51a80c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 -- 172.21.15.115:0/1416685718 >> 172.21.15.115:0/1416685718 conn(0x7f51a80ce880 msgr2=0x7f51a806c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 -- 172.21.15.115:0/1416685718 shutdown_connections 2023-12-01T03:11:36.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.602+0000 7f51ae100640 1 -- 172.21.15.115:0/1416685718 wait complete. 2023-12-01T03:11:36.622 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T03:11:36.622 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:11:36.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.823+0000 7f3da9570640 1 Processor -- start 2023-12-01T03:11:36.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.823+0000 7f3da9570640 1 -- start start 2023-12-01T03:11:36.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.824+0000 7f3da9570640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40d05d0 0x7f3da40d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.824+0000 7f3da9570640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da415e020 con 0x7f3da415db90 2023-12-01T03:11:36.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.824+0000 7f3da9570640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da40cd3f0 con 0x7f3da415d7b0 2023-12-01T03:11:36.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.824+0000 7f3da9570640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40cd0f0 con 0x7f3da40d05d0 2023-12-01T03:11:36.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.824+0000 7f3da37fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3da415db90 0x7f3da40cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56992/0 (socket says 172.21.15.115:56992) 2023-12-01T03:11:36.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.824+0000 7f3da37fe640 1 -- 172.21.15.115:0/59700210 learned_addr learned my addr 172.21.15.115:0/59700210 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da8d6f640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40d05d0 0x7f3da40d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217803987 0 0) 0x7f3da415e020 con 0x7f3da415db90 2023-12-01T03:11:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d8c003520 con 0x7f3da415db90 2023-12-01T03:11:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3977418715 0 0) 0x7f3d8c003520 con 0x7f3da415db90 2023-12-01T03:11:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40d05d0 msgr2=0x7f3da40d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40d05d0 0x7f3da40d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 >> v1:172.21.15.142:6789/0 conn(0x7f3da415d7b0 legacy=0x7f3da4160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da4160ec0 con 0x7f3da415db90 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3d98001670 con 0x7f3da415db90 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 >> v1:172.21.15.115:6789/0 conn(0x7f3da415db90 legacy=0x7f3da40cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d8c003a10 0x7f3d8c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.825+0000 7f3da2ffd640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40d05d0 0x7f3d8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da2ffd640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d8c004d00 0x7f3d8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40cd0f0 con 0x7f3d8c004d00 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c00bba0 con 0x7f3d8c003a10 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c00bd30 con 0x7f3da40d05d0 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da8d6f640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d8c004d00 0x7f3d8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da3fff640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d8c003a10 0x7f3d8c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da8d6f640 1 -- 172.21.15.115:0/59700210 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40d05d0 msgr2=0x7f3d8c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da8d6f640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da40d05d0 0x7f3d8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da8d6f640 1 -- 172.21.15.115:0/59700210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d8c003a10 msgr2=0x7f3d8c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:36.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da8d6f640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d8c003a10 0x7f3d8c003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.826+0000 7f3da8d6f640 1 -- 172.21.15.115:0/59700210 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c003520 con 0x7f3d8c004d00 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.827+0000 7f3da8d6f640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d8c004d00 0x7f3d8c005180 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009fc0 tx=0x7f3d8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6351114db14b5705 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.827+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c02cbf0 con 0x7f3d8c004d00 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.827+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c00f040 con 0x7f3d8c004d00 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.827+0000 7f3da2ffd640 1 -- 172.21.15.115:0/59700210 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c034440 con 0x7f3d8c004d00 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.828+0000 7f3da9570640 1 -- 172.21.15.115:0/59700210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d8c004d00 msgr2=0x7f3d8c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.828+0000 7f3da9570640 1 --2- 172.21.15.115:0/59700210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d8c004d00 0x7f3d8c005180 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009fc0 tx=0x7f3d8c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.828+0000 7f3da8d6f640 1 -- 172.21.15.115:0/59700210 reap_dead start 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.828+0000 7f3da9570640 1 -- 172.21.15.115:0/59700210 shutdown_connections 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.828+0000 7f3da9570640 1 -- 172.21.15.115:0/59700210 >> 172.21.15.115:0/59700210 conn(0x7f3da406a0e0 msgr2=0x7f3da406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.828+0000 7f3da9570640 1 -- 172.21.15.115:0/59700210 shutdown_connections 2023-12-01T03:11:36.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.828+0000 7f3da9570640 1 -- 172.21.15.115:0/59700210 wait complete. 2023-12-01T03:11:36.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.830+0000 7f3da9570640 1 Processor -- start 2023-12-01T03:11:36.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.830+0000 7f3da9570640 1 -- start start 2023-12-01T03:11:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.831+0000 7f3da9570640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da415b820 0x7f3da40ab030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.831+0000 7f3da9570640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da40b4e50 0x7f3da40a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.831+0000 7f3da9570640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da40a9940 0x7f3da41f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.831+0000 7f3da9570640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40a9dc0 con 0x7f3da40b4e50 2023-12-01T03:11:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.831+0000 7f3da9570640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40abdd0 con 0x7f3da40a9940 2023-12-01T03:11:36.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.831+0000 7f3da9570640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40b23f0 con 0x7f3da415b820 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.831+0000 7f3da37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da40b4e50 0x7f3da40a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.831+0000 7f3da37fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da40b4e50 0x7f3da40a9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38832/0 (socket says 172.21.15.115:38832) 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.831+0000 7f3da37fe640 1 -- 172.21.15.115:0/1790328416 learned_addr learned my addr 172.21.15.115:0/1790328416 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da37fe640 1 -- 172.21.15.115:0/1790328416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da415b820 msgr2=0x7f3da40ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da3fff640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da415b820 0x7f3da40ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da8d6f640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da40a9940 0x7f3da41f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da37fe640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da415b820 0x7f3da40ab030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da37fe640 1 -- 172.21.15.115:0/1790328416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da40a9940 msgr2=0x7f3da41f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da37fe640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da40a9940 0x7f3da41f5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da37fe640 1 -- 172.21.15.115:0/1790328416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da40ab570 con 0x7f3da40b4e50 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da37fe640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da40b4e50 0x7f3da40a9400 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c760 tx=0x7f3d9800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da0ff9640 1 -- 172.21.15.115:0/1790328416 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98004cb0 con 0x7f3da40b4e50 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da0ff9640 1 -- 172.21.15.115:0/1790328416 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d98004e30 con 0x7f3da40b4e50 2023-12-01T03:11:36.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da0ff9640 1 -- 172.21.15.115:0/1790328416 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98007310 con 0x7f3da40b4e50 2023-12-01T03:11:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da9570640 1 -- 172.21.15.115:0/1790328416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da40ab810 con 0x7f3da40b4e50 2023-12-01T03:11:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.832+0000 7f3da9570640 1 -- 172.21.15.115:0/1790328416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da40b2640 con 0x7f3da40b4e50 2023-12-01T03:11:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.834+0000 7f3da9570640 1 -- 172.21.15.115:0/1790328416 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da40673a0 con 0x7f3da40b4e50 2023-12-01T03:11:36.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.834+0000 7f3da0ff9640 1 -- 172.21.15.115:0/1790328416 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f3d98007470 con 0x7f3da40b4e50 2023-12-01T03:11:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.835+0000 7f3da0ff9640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3d9006a6a0 0x7f3d9006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.835+0000 7f3da0ff9640 1 -- 172.21.15.115:0/1790328416 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f3d980881b0 con 0x7f3da40b4e50 2023-12-01T03:11:36.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.836+0000 7f3da3fff640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3d9006a6a0 0x7f3d9006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:36.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.836+0000 7f3da3fff640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3d9006a6a0 0x7f3d9006cb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3da41f6c10 tx=0x7f3d94009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:36.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:36.840+0000 7f3da0ff9640 1 -- 172.21.15.115:0/1790328416 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3d98053410 con 0x7f3da40b4e50 2023-12-01T03:11:37.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.011+0000 7f3da9570640 1 -- 172.21.15.115:0/1790328416 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3da4161480 con 0x7f3d9006a6a0 2023-12-01T03:11:37.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.026+0000 7f3da0ff9640 1 -- 172.21.15.115:0/1790328416 <== mgr.11121 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284410 (secure 0 0 0) 0x7f3d9400d040 con 0x7f3d9006a6a0 2023-12-01T03:11:37.031 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:37.033 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:11:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 -- 172.21.15.115:0/1790328416 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3d9006a6a0 msgr2=0x7f3d9006cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3d9006a6a0 0x7f3d9006cb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3da41f6c10 tx=0x7f3d94009040 comp rx=0 tx=0).stop 2023-12-01T03:11:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 -- 172.21.15.115:0/1790328416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da40b4e50 msgr2=0x7f3da40a9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da40b4e50 0x7f3da40a9400 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c760 tx=0x7f3d9800cc20 comp rx=0 tx=0).stop 2023-12-01T03:11:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 -- 172.21.15.115:0/1790328416 shutdown_connections 2023-12-01T03:11:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3d9006a6a0 0x7f3d9006cb60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da40a9940 0x7f3da41f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3da40b4e50 0x7f3da40a9400 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 --2- 172.21.15.115:0/1790328416 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3da415b820 0x7f3da40ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 -- 172.21.15.115:0/1790328416 >> 172.21.15.115:0/1790328416 conn(0x7f3da406a0e0 msgr2=0x7f3da40b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 -- 172.21.15.115:0/1790328416 shutdown_connections 2023-12-01T03:11:37.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.032+0000 7f3da9570640 1 -- 172.21.15.115:0/1790328416 wait complete. 2023-12-01T03:11:37.051 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-12-01T03:11:35.822351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":419168,"kb_used_data":23968,"kb_used_omap":0,"kb_used_meta":395136,"kb_avail":754555552,"statfs":{"total":773094113280,"available":772664885248,"internally_reserved":0,"allocated":24543232,"data_stored":12617262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":404619264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190259"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.672100+0000","last_change":"2023-12-01T03:11:31.672100+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.672100+0000","last_undegraded":"2023-12-01T03:11:31.672100+0000","last_fullsized":"2023-12-01T03:11:31.672100+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662282+0000","last_change":"2023-12-01T03:11:31.662282+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662282+0000","last_undegraded":"2023-12-01T03:11:31.662282+0000","last_fullsized":"2023-12-01T03:11:31.662282+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:10.849908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.667239+0000","last_change":"2023-12-01T03:11:31.667239+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.667239+0000","last_undegraded":"2023-12-01T03:11:31.667239+0000","last_fullsized":"2023-12-01T03:11:31.667239+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:38.174827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.646957+0000","last_change":"2023-12-01T03:11:31.646957+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.646957+0000","last_undegraded":"2023-12-01T03:11:31.646957+0000","last_fullsized":"2023-12-01T03:11:31.646957+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:43.961392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.649740+0000","last_change":"2023-12-01T03:11:31.649740+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.649740+0000","last_undegraded":"2023-12-01T03:11:31.649740+0000","last_fullsized":"2023-12-01T03:11:31.649740+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.655118+0000","last_change":"2023-12-01T03:11:31.655118+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.655118+0000","last_undegraded":"2023-12-01T03:11:31.655118+0000","last_fullsized":"2023-12-01T03:11:31.655118+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.666619+0000","last_change":"2023-12-01T03:11:31.666619+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.666619+0000","last_undegraded":"2023-12-01T03:11:31.666619+0000","last_fullsized":"2023-12-01T03:11:31.666619+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:19.277681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.655923+0000","last_change":"2023-12-01T03:11:31.655923+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.655923+0000","last_undegraded":"2023-12-01T03:11:31.655923+0000","last_fullsized":"2023-12-01T03:11:31.655923+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:51.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.653833+0000","last_change":"2023-12-01T03:11:31.653833+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.653833+0000","last_undegraded":"2023-12-01T03:11:31.653833+0000","last_fullsized":"2023-12-01T03:11:31.653833+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.650471+0000","last_change":"2023-12-01T03:11:31.650471+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.650471+0000","last_undegraded":"2023-12-01T03:11:31.650471+0000","last_fullsized":"2023-12-01T03:11:31.650471+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:44:42.356924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662851+0000","last_change":"2023-12-01T03:11:31.662851+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662851+0000","last_undegraded":"2023-12-01T03:11:31.662851+0000","last_fullsized":"2023-12-01T03:11:31.662851+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:35.627224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.659273+0000","last_change":"2023-12-01T03:11:31.659273+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.659273+0000","last_undegraded":"2023-12-01T03:11:31.659273+0000","last_fullsized":"2023-12-01T03:11:31.659273+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665375+0000","last_change":"2023-12-01T03:11:31.665375+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665375+0000","last_undegraded":"2023-12-01T03:11:31.665375+0000","last_fullsized":"2023-12-01T03:11:31.665375+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:04.079396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665123+0000","last_change":"2023-12-01T03:11:31.665123+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665123+0000","last_undegraded":"2023-12-01T03:11:31.665123+0000","last_fullsized":"2023-12-01T03:11:31.665123+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:21.336618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.646269+0000","last_change":"2023-12-01T03:11:31.646269+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.646269+0000","last_undegraded":"2023-12-01T03:11:31.646269+0000","last_fullsized":"2023-12-01T03:11:31.646269+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.653126+0000","last_change":"2023-12-01T03:11:31.653126+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.653126+0000","last_undegraded":"2023-12-01T03:11:31.653126+0000","last_fullsized":"2023-12-01T03:11:31.653126+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.670984+0000","last_change":"2023-12-01T03:11:31.670984+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.670984+0000","last_undegraded":"2023-12-01T03:11:31.670984+0000","last_fullsized":"2023-12-01T03:11:31.670984+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665505+0000","last_change":"2023-12-01T03:11:31.665505+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665505+0000","last_undegraded":"2023-12-01T03:11:31.665505+0000","last_fullsized":"2023-12-01T03:11:31.665505+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:42.720564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.652287+0000","last_change":"2023-12-01T03:11:31.652287+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.652287+0000","last_undegraded":"2023-12-01T03:11:31.652287+0000","last_fullsized":"2023-12-01T03:11:31.652287+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665034+0000","last_change":"2023-12-01T03:11:31.665034+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665034+0000","last_undegraded":"2023-12-01T03:11:31.665034+0000","last_fullsized":"2023-12-01T03:11:31.665034+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:10.441679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662847+0000","last_change":"2023-12-01T03:11:31.662847+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662847+0000","last_undegraded":"2023-12-01T03:11:31.662847+0000","last_fullsized":"2023-12-01T03:11:31.662847+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:53:43.178969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.651584+0000","last_change":"2023-12-01T03:11:31.651584+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.651584+0000","last_undegraded":"2023-12-01T03:11:31.651584+0000","last_fullsized":"2023-12-01T03:11:31.651584+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665570+0000","last_change":"2023-12-01T03:11:31.665570+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665570+0000","last_undegraded":"2023-12-01T03:11:31.665570+0000","last_fullsized":"2023-12-01T03:11:31.665570+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:00.674458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.649271+0000","last_change":"2023-12-01T03:11:31.649271+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.649271+0000","last_undegraded":"2023-12-01T03:11:31.649271+0000","last_fullsized":"2023-12-01T03:11:31.649271+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:29.661974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.661942+0000","last_change":"2023-12-01T03:11:31.661942+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.661942+0000","last_undegraded":"2023-12-01T03:11:31.661942+0000","last_fullsized":"2023-12-01T03:11:31.661942+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:09.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.648127+0000","last_change":"2023-12-01T03:11:31.648127+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.648127+0000","last_undegraded":"2023-12-01T03:11:31.648127+0000","last_fullsized":"2023-12-01T03:11:31.648127+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:35.975600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.663265+0000","last_change":"2023-12-01T03:11:31.663265+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.663265+0000","last_undegraded":"2023-12-01T03:11:31.663265+0000","last_fullsized":"2023-12-01T03:11:31.663265+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:28.852429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.654435+0000","last_change":"2023-12-01T03:11:31.654435+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.654435+0000","last_undegraded":"2023-12-01T03:11:31.654435+0000","last_fullsized":"2023-12-01T03:11:31.654435+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:14.716448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.655436+0000","last_change":"2023-12-01T03:11:31.655436+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.655436+0000","last_undegraded":"2023-12-01T03:11:31.655436+0000","last_fullsized":"2023-12-01T03:11:31.655436+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:28:34.324085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.658499+0000","last_change":"2023-12-01T03:11:31.658499+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.658499+0000","last_undegraded":"2023-12-01T03:11:31.658499+0000","last_fullsized":"2023-12-01T03:11:31.658499+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662619+0000","last_change":"2023-12-01T03:11:31.662619+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662619+0000","last_undegraded":"2023-12-01T03:11:31.662619+0000","last_fullsized":"2023-12-01T03:11:31.662619+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:36.571608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.654612+0000","last_change":"2023-12-01T03:11:31.654612+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.654612+0000","last_undegraded":"2023-12-01T03:11:31.654612+0000","last_fullsized":"2023-12-01T03:11:31.654612+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:14.760284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.661833+0000","last_change":"2023-12-01T03:11:31.661833+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.661833+0000","last_undegraded":"2023-12-01T03:11:31.661833+0000","last_fullsized":"2023-12-01T03:11:31.661833+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:24.244094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.681672+0000","last_change":"2023-12-01T03:11:31.680864+0000","last_active":"2023-12-01T03:11:31.681672+0000","last_peered":"2023-12-01T03:11:31.681672+0000","last_clean":"2023-12-01T03:11:31.681672+0000","last_became_active":"2023-12-01T03:11:31.680279+0000","last_became_peered":"2023-12-01T03:11:31.680279+0000","last_unstale":"2023-12-01T03:11:31.681672+0000","last_undegraded":"2023-12-01T03:11:31.681672+0000","last_fullsized":"2023-12-01T03:11:31.681672+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.377359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.650323+0000","last_change":"2023-12-01T03:11:31.650323+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.650323+0000","last_undegraded":"2023-12-01T03:11:31.650323+0000","last_fullsized":"2023-12-01T03:11:31.650323+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.681207+0000","last_change":"2023-12-01T03:11:31.679798+0000","last_active":"2023-12-01T03:11:31.681207+0000","last_peered":"2023-12-01T03:11:31.681207+0000","last_clean":"2023-12-01T03:11:31.681207+0000","last_became_active":"2023-12-01T03:11:31.679030+0000","last_became_peered":"2023-12-01T03:11:31.679030+0000","last_unstale":"2023-12-01T03:11:31.681207+0000","last_undegraded":"2023-12-01T03:11:31.681207+0000","last_fullsized":"2023-12-01T03:11:31.681207+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:49.261314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.655235+0000","last_change":"2023-12-01T03:11:31.655235+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.655235+0000","last_undegraded":"2023-12-01T03:11:31.655235+0000","last_fullsized":"2023-12-01T03:11:31.655235+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:17.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.666885+0000","last_change":"2023-12-01T03:11:31.663745+0000","last_active":"2023-12-01T03:11:31.666885+0000","last_peered":"2023-12-01T03:11:31.666885+0000","last_clean":"2023-12-01T03:11:31.666885+0000","last_became_active":"2023-12-01T03:11:31.662706+0000","last_became_peered":"2023-12-01T03:11:31.662706+0000","last_unstale":"2023-12-01T03:11:31.666885+0000","last_undegraded":"2023-12-01T03:11:31.666885+0000","last_fullsized":"2023-12-01T03:11:31.666885+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:42.945449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.664632+0000","last_change":"2023-12-01T03:11:31.664632+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.664632+0000","last_undegraded":"2023-12-01T03:11:31.664632+0000","last_fullsized":"2023-12-01T03:11:31.664632+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:22:51.687377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.661731+0000","last_change":"2023-12-01T03:11:31.660758+0000","last_active":"2023-12-01T03:11:31.661731+0000","last_peered":"2023-12-01T03:11:31.661731+0000","last_clean":"2023-12-01T03:11:31.661731+0000","last_became_active":"2023-12-01T03:11:31.660001+0000","last_became_peered":"2023-12-01T03:11:31.660001+0000","last_unstale":"2023-12-01T03:11:31.661731+0000","last_undegraded":"2023-12-01T03:11:31.661731+0000","last_fullsized":"2023-12-01T03:11:31.661731+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.646204+0000","last_change":"2023-12-01T03:11:31.646204+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.646204+0000","last_undegraded":"2023-12-01T03:11:31.646204+0000","last_fullsized":"2023-12-01T03:11:31.646204+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.682460+0000","last_change":"2023-12-01T03:11:31.681531+0000","last_active":"2023-12-01T03:11:31.682460+0000","last_peered":"2023-12-01T03:11:31.682460+0000","last_clean":"2023-12-01T03:11:31.682460+0000","last_became_active":"2023-12-01T03:11:31.680782+0000","last_became_peered":"2023-12-01T03:11:31.680782+0000","last_unstale":"2023-12-01T03:11:31.682460+0000","last_undegraded":"2023-12-01T03:11:31.682460+0000","last_fullsized":"2023-12-01T03:11:31.682460+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:18.018055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.649611+0000","last_change":"2023-12-01T03:11:31.649611+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.649611+0000","last_undegraded":"2023-12-01T03:11:31.649611+0000","last_fullsized":"2023-12-01T03:11:31.649611+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.666436+0000","last_change":"2023-12-01T03:11:31.663166+0000","last_active":"2023-12-01T03:11:31.666436+0000","last_peered":"2023-12-01T03:11:31.666436+0000","last_clean":"2023-12-01T03:11:31.666436+0000","last_became_active":"2023-12-01T03:11:31.662263+0000","last_became_peered":"2023-12-01T03:11:31.662263+0000","last_unstale":"2023-12-01T03:11:31.666436+0000","last_undegraded":"2023-12-01T03:11:31.666436+0000","last_fullsized":"2023-12-01T03:11:31.666436+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:57:00.965499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.664744+0000","last_change":"2023-12-01T03:11:31.664744+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.664744+0000","last_undegraded":"2023-12-01T03:11:31.664744+0000","last_fullsized":"2023-12-01T03:11:31.664744+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.674834+0000","last_change":"2023-12-01T03:11:31.670524+0000","last_active":"2023-12-01T03:11:31.674834+0000","last_peered":"2023-12-01T03:11:31.674834+0000","last_clean":"2023-12-01T03:11:31.674834+0000","last_became_active":"2023-12-01T03:11:31.670190+0000","last_became_peered":"2023-12-01T03:11:31.670190+0000","last_unstale":"2023-12-01T03:11:31.674834+0000","last_undegraded":"2023-12-01T03:11:31.674834+0000","last_fullsized":"2023-12-01T03:11:31.674834+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:45.874739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.658638+0000","last_change":"2023-12-01T03:11:31.658638+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.658638+0000","last_undegraded":"2023-12-01T03:11:31.658638+0000","last_fullsized":"2023-12-01T03:11:31.658638+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.677746+0000","last_change":"2023-12-01T03:11:31.676322+0000","last_active":"2023-12-01T03:11:31.677746+0000","last_peered":"2023-12-01T03:11:31.677746+0000","last_clean":"2023-12-01T03:11:31.677746+0000","last_became_active":"2023-12-01T03:11:31.674451+0000","last_became_peered":"2023-12-01T03:11:31.674451+0000","last_unstale":"2023-12-01T03:11:31.677746+0000","last_undegraded":"2023-12-01T03:11:31.677746+0000","last_fullsized":"2023-12-01T03:11:31.677746+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:44.820233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.661499+0000","last_change":"2023-12-01T03:11:31.661499+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.661499+0000","last_undegraded":"2023-12-01T03:11:31.661499+0000","last_fullsized":"2023-12-01T03:11:31.661499+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:40.557913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.664811+0000","last_change":"2023-12-01T03:11:31.662929+0000","last_active":"2023-12-01T03:11:31.664811+0000","last_peered":"2023-12-01T03:11:31.664811+0000","last_clean":"2023-12-01T03:11:31.664811+0000","last_became_active":"2023-12-01T03:11:31.662617+0000","last_became_peered":"2023-12-01T03:11:31.662617+0000","last_unstale":"2023-12-01T03:11:31.664811+0000","last_undegraded":"2023-12-01T03:11:31.664811+0000","last_fullsized":"2023-12-01T03:11:31.664811+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:34.611280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.657685+0000","last_change":"2023-12-01T03:11:31.657685+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.657685+0000","last_undegraded":"2023-12-01T03:11:31.657685+0000","last_fullsized":"2023-12-01T03:11:31.657685+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.670983+0000","last_change":"2023-12-01T03:11:31.664798+0000","last_active":"2023-12-01T03:11:31.670983+0000","last_peered":"2023-12-01T03:11:31.670983+0000","last_clean":"2023-12-01T03:11:31.670983+0000","last_became_active":"2023-12-01T03:11:31.664276+0000","last_became_peered":"2023-12-01T03:11:31.664276+0000","last_unstale":"2023-12-01T03:11:31.670983+0000","last_undegraded":"2023-12-01T03:11:31.670983+0000","last_fullsized":"2023-12-01T03:11:31.670983+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:18:18.909277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.659121+0000","last_change":"2023-12-01T03:11:31.659121+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.659121+0000","last_undegraded":"2023-12-01T03:11:31.659121+0000","last_fullsized":"2023-12-01T03:11:31.659121+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.670077+0000","last_change":"2023-12-01T03:11:31.664162+0000","last_active":"2023-12-01T03:11:31.670077+0000","last_peered":"2023-12-01T03:11:31.670077+0000","last_clean":"2023-12-01T03:11:31.670077+0000","last_became_active":"2023-12-01T03:11:31.663854+0000","last_became_peered":"2023-12-01T03:11:31.663854+0000","last_unstale":"2023-12-01T03:11:31.670077+0000","last_undegraded":"2023-12-01T03:11:31.670077+0000","last_fullsized":"2023-12-01T03:11:31.670077+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:15.367696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.650462+0000","last_change":"2023-12-01T03:11:31.650462+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.650462+0000","last_undegraded":"2023-12-01T03:11:31.650462+0000","last_fullsized":"2023-12-01T03:11:31.650462+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:46.890682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.671195+0000","last_change":"2023-12-01T03:11:31.665987+0000","last_active":"2023-12-01T03:11:31.671195+0000","last_peered":"2023-12-01T03:11:31.671195+0000","last_clean":"2023-12-01T03:11:31.671195+0000","last_became_active":"2023-12-01T03:11:31.665550+0000","last_became_peered":"2023-12-01T03:11:31.665550+0000","last_unstale":"2023-12-01T03:11:31.671195+0000","last_undegraded":"2023-12-01T03:11:31.671195+0000","last_fullsized":"2023-12-01T03:11:31.671195+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:16.324155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.657130+0000","last_change":"2023-12-01T03:11:31.657130+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.657130+0000","last_undegraded":"2023-12-01T03:11:31.657130+0000","last_fullsized":"2023-12-01T03:11:31.657130+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.676301+0000","last_change":"2023-12-01T03:11:31.675387+0000","last_active":"2023-12-01T03:11:31.676301+0000","last_peered":"2023-12-01T03:11:31.676301+0000","last_clean":"2023-12-01T03:11:31.676301+0000","last_became_active":"2023-12-01T03:11:31.675108+0000","last_became_peered":"2023-12-01T03:11:31.675108+0000","last_unstale":"2023-12-01T03:11:31.676301+0000","last_undegraded":"2023-12-01T03:11:31.676301+0000","last_fullsized":"2023-12-01T03:11:31.676301+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.836486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.652517+0000","last_change":"2023-12-01T03:11:31.652517+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.652517+0000","last_undegraded":"2023-12-01T03:11:31.652517+0000","last_fullsized":"2023-12-01T03:11:31.652517+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.672130+0000","last_change":"2023-12-01T03:11:31.671174+0000","last_active":"2023-12-01T03:11:31.672130+0000","last_peered":"2023-12-01T03:11:31.672130+0000","last_clean":"2023-12-01T03:11:31.672130+0000","last_became_active":"2023-12-01T03:11:31.670862+0000","last_became_peered":"2023-12-01T03:11:31.670862+0000","last_unstale":"2023-12-01T03:11:31.672130+0000","last_undegraded":"2023-12-01T03:11:31.672130+0000","last_fullsized":"2023-12-01T03:11:31.672130+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:37.158883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.666137+0000","last_change":"2023-12-01T03:11:31.666137+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.666137+0000","last_undegraded":"2023-12-01T03:11:31.666137+0000","last_fullsized":"2023-12-01T03:11:31.666137+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:22.924077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.676600+0000","last_change":"2023-12-01T03:11:31.673143+0000","last_active":"2023-12-01T03:11:31.676600+0000","last_peered":"2023-12-01T03:11:31.676600+0000","last_clean":"2023-12-01T03:11:31.676600+0000","last_became_active":"2023-12-01T03:11:31.671261+0000","last_became_peered":"2023-12-01T03:11:31.671261+0000","last_unstale":"2023-12-01T03:11:31.676600+0000","last_undegraded":"2023-12-01T03:11:31.676600+0000","last_fullsized":"2023-12-01T03:11:31.676600+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:13.744340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.648564+0000","last_change":"2023-12-01T03:11:31.648564+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.648564+0000","last_undegraded":"2023-12-01T03:11:31.648564+0000","last_fullsized":"2023-12-01T03:11:31.648564+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:46.532354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.678553+0000","last_change":"2023-12-01T03:11:31.677841+0000","last_active":"2023-12-01T03:11:31.678553+0000","last_peered":"2023-12-01T03:11:31.678553+0000","last_clean":"2023-12-01T03:11:31.678553+0000","last_became_active":"2023-12-01T03:11:31.677483+0000","last_became_peered":"2023-12-01T03:11:31.677483+0000","last_unstale":"2023-12-01T03:11:31.678553+0000","last_undegraded":"2023-12-01T03:11:31.678553+0000","last_fullsized":"2023-12-01T03:11:31.678553+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:31.837132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662025+0000","last_change":"2023-12-01T03:11:31.662025+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662025+0000","last_undegraded":"2023-12-01T03:11:31.662025+0000","last_fullsized":"2023-12-01T03:11:31.662025+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:29:48.204277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.675090+0000","last_change":"2023-12-01T03:11:31.670701+0000","last_active":"2023-12-01T03:11:31.675090+0000","last_peered":"2023-12-01T03:11:31.675090+0000","last_clean":"2023-12-01T03:11:31.675090+0000","last_became_active":"2023-12-01T03:11:31.670179+0000","last_became_peered":"2023-12-01T03:11:31.670179+0000","last_unstale":"2023-12-01T03:11:31.675090+0000","last_undegraded":"2023-12-01T03:11:31.675090+0000","last_fullsized":"2023-12-01T03:11:31.675090+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:01.094744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665097+0000","last_change":"2023-12-01T03:11:31.665097+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665097+0000","last_undegraded":"2023-12-01T03:11:31.665097+0000","last_fullsized":"2023-12-01T03:11:31.665097+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:31.875817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.680353+0000","last_change":"2023-12-01T03:11:31.679287+0000","last_active":"2023-12-01T03:11:31.680353+0000","last_peered":"2023-12-01T03:11:31.680353+0000","last_clean":"2023-12-01T03:11:31.680353+0000","last_became_active":"2023-12-01T03:11:31.678925+0000","last_became_peered":"2023-12-01T03:11:31.678925+0000","last_unstale":"2023-12-01T03:11:31.680353+0000","last_undegraded":"2023-12-01T03:11:31.680353+0000","last_fullsized":"2023-12-01T03:11:31.680353+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:28:46.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.653733+0000","last_change":"2023-12-01T03:11:31.653733+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.653733+0000","last_undegraded":"2023-12-01T03:11:31.653733+0000","last_fullsized":"2023-12-01T03:11:31.653733+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:18:18.055289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.676041+0000","last_change":"2023-12-01T03:11:31.674632+0000","last_active":"2023-12-01T03:11:31.676041+0000","last_peered":"2023-12-01T03:11:31.676041+0000","last_clean":"2023-12-01T03:11:31.676041+0000","last_became_active":"2023-12-01T03:11:31.674319+0000","last_became_peered":"2023-12-01T03:11:31.674319+0000","last_unstale":"2023-12-01T03:11:31.676041+0000","last_undegraded":"2023-12-01T03:11:31.676041+0000","last_fullsized":"2023-12-01T03:11:31.676041+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:21.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662041+0000","last_change":"2023-12-01T03:11:31.662041+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662041+0000","last_undegraded":"2023-12-01T03:11:31.662041+0000","last_fullsized":"2023-12-01T03:11:31.662041+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:27:53.473015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.678086+0000","last_change":"2023-12-01T03:11:31.676885+0000","last_active":"2023-12-01T03:11:31.678086+0000","last_peered":"2023-12-01T03:11:31.678086+0000","last_clean":"2023-12-01T03:11:31.678086+0000","last_became_active":"2023-12-01T03:11:31.675545+0000","last_became_peered":"2023-12-01T03:11:31.675545+0000","last_unstale":"2023-12-01T03:11:31.678086+0000","last_undegraded":"2023-12-01T03:11:31.678086+0000","last_fullsized":"2023-12-01T03:11:31.678086+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:03.962061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.656877+0000","last_change":"2023-12-01T03:11:31.656877+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.656877+0000","last_undegraded":"2023-12-01T03:11:31.656877+0000","last_fullsized":"2023-12-01T03:11:31.656877+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:15:30.930202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.676977+0000","last_change":"2023-12-01T03:11:31.675947+0000","last_active":"2023-12-01T03:11:31.676977+0000","last_peered":"2023-12-01T03:11:31.676977+0000","last_clean":"2023-12-01T03:11:31.676977+0000","last_became_active":"2023-12-01T03:11:31.675372+0000","last_became_peered":"2023-12-01T03:11:31.675372+0000","last_unstale":"2023-12-01T03:11:31.676977+0000","last_undegraded":"2023-12-01T03:11:31.676977+0000","last_fullsized":"2023-12-01T03:11:31.676977+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:45.516411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.646063+0000","last_change":"2023-12-01T03:11:31.646063+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.646063+0000","last_undegraded":"2023-12-01T03:11:31.646063+0000","last_fullsized":"2023-12-01T03:11:31.646063+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:20.713703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.678244+0000","last_change":"2023-12-01T03:11:31.673899+0000","last_active":"2023-12-01T03:11:31.678244+0000","last_peered":"2023-12-01T03:11:31.678244+0000","last_clean":"2023-12-01T03:11:31.678244+0000","last_became_active":"2023-12-01T03:11:31.673474+0000","last_became_peered":"2023-12-01T03:11:31.673474+0000","last_unstale":"2023-12-01T03:11:31.678244+0000","last_undegraded":"2023-12-01T03:11:31.678244+0000","last_fullsized":"2023-12-01T03:11:31.678244+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:50.332038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.654447+0000","last_change":"2023-12-01T03:11:31.654447+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.654447+0000","last_undegraded":"2023-12-01T03:11:31.654447+0000","last_fullsized":"2023-12-01T03:11:31.654447+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.683474+0000","last_change":"2023-12-01T03:11:31.682949+0000","last_active":"2023-12-01T03:11:31.683474+0000","last_peered":"2023-12-01T03:11:31.683474+0000","last_clean":"2023-12-01T03:11:31.683474+0000","last_became_active":"2023-12-01T03:11:31.682454+0000","last_became_peered":"2023-12-01T03:11:31.682454+0000","last_unstale":"2023-12-01T03:11:31.683474+0000","last_undegraded":"2023-12-01T03:11:31.683474+0000","last_fullsized":"2023-12-01T03:11:31.683474+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:49.777291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.663231+0000","last_change":"2023-12-01T03:11:31.663231+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.663231+0000","last_undegraded":"2023-12-01T03:11:31.663231+0000","last_fullsized":"2023-12-01T03:11:31.663231+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:33:45.498838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.673399+0000","last_change":"2023-12-01T03:11:31.668375+0000","last_active":"2023-12-01T03:11:31.673399+0000","last_peered":"2023-12-01T03:11:31.673399+0000","last_clean":"2023-12-01T03:11:31.673399+0000","last_became_active":"2023-12-01T03:11:31.668109+0000","last_became_peered":"2023-12-01T03:11:31.668109+0000","last_unstale":"2023-12-01T03:11:31.673399+0000","last_undegraded":"2023-12-01T03:11:31.673399+0000","last_fullsized":"2023-12-01T03:11:31.673399+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:34.959657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.663085+0000","last_change":"2023-12-01T03:11:31.663085+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.663085+0000","last_undegraded":"2023-12-01T03:11:31.663085+0000","last_fullsized":"2023-12-01T03:11:31.663085+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:53.273884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.680434+0000","last_change":"2023-12-01T03:11:31.679248+0000","last_active":"2023-12-01T03:11:31.680434+0000","last_peered":"2023-12-01T03:11:31.680434+0000","last_clean":"2023-12-01T03:11:31.680434+0000","last_became_active":"2023-12-01T03:11:31.678973+0000","last_became_peered":"2023-12-01T03:11:31.678973+0000","last_unstale":"2023-12-01T03:11:31.680434+0000","last_undegraded":"2023-12-01T03:11:31.680434+0000","last_fullsized":"2023-12-01T03:11:31.680434+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.658835+0000","last_change":"2023-12-01T03:11:31.658835+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.658835+0000","last_undegraded":"2023-12-01T03:11:31.658835+0000","last_fullsized":"2023-12-01T03:11:31.658835+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.669200+0000","last_change":"2023-12-01T03:11:31.663771+0000","last_active":"2023-12-01T03:11:31.669200+0000","last_peered":"2023-12-01T03:11:31.669200+0000","last_clean":"2023-12-01T03:11:31.669200+0000","last_became_active":"2023-12-01T03:11:31.662877+0000","last_became_peered":"2023-12-01T03:11:31.662877+0000","last_unstale":"2023-12-01T03:11:31.669200+0000","last_undegraded":"2023-12-01T03:11:31.669200+0000","last_fullsized":"2023-12-01T03:11:31.669200+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:59.396265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.648564+0000","last_change":"2023-12-01T03:11:31.648564+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.648564+0000","last_undegraded":"2023-12-01T03:11:31.648564+0000","last_fullsized":"2023-12-01T03:11:31.648564+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:17.846927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.677011+0000","last_change":"2023-12-01T03:11:31.673043+0000","last_active":"2023-12-01T03:11:31.677011+0000","last_peered":"2023-12-01T03:11:31.677011+0000","last_clean":"2023-12-01T03:11:31.677011+0000","last_became_active":"2023-12-01T03:11:31.672780+0000","last_became_peered":"2023-12-01T03:11:31.672780+0000","last_unstale":"2023-12-01T03:11:31.677011+0000","last_undegraded":"2023-12-01T03:11:31.677011+0000","last_fullsized":"2023-12-01T03:11:31.677011+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:40.478556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.664707+0000","last_change":"2023-12-01T03:11:31.664707+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.664707+0000","last_undegraded":"2023-12-01T03:11:31.664707+0000","last_fullsized":"2023-12-01T03:11:31.664707+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.678278+0000","last_change":"2023-12-01T03:11:31.677162+0000","last_active":"2023-12-01T03:11:31.678278+0000","last_peered":"2023-12-01T03:11:31.678278+0000","last_clean":"2023-12-01T03:11:31.678278+0000","last_became_active":"2023-12-01T03:11:31.674178+0000","last_became_peered":"2023-12-01T03:11:31.674178+0000","last_unstale":"2023-12-01T03:11:31.678278+0000","last_undegraded":"2023-12-01T03:11:31.678278+0000","last_fullsized":"2023-12-01T03:11:31.678278+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:13.700505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.660932+0000","last_change":"2023-12-01T03:11:31.660932+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.660932+0000","last_undegraded":"2023-12-01T03:11:31.660932+0000","last_fullsized":"2023-12-01T03:11:31.660932+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.673944+0000","last_change":"2023-12-01T03:11:31.672908+0000","last_active":"2023-12-01T03:11:31.673944+0000","last_peered":"2023-12-01T03:11:31.673944+0000","last_clean":"2023-12-01T03:11:31.673944+0000","last_became_active":"2023-12-01T03:11:31.671738+0000","last_became_peered":"2023-12-01T03:11:31.671738+0000","last_unstale":"2023-12-01T03:11:31.673944+0000","last_undegraded":"2023-12-01T03:11:31.673944+0000","last_fullsized":"2023-12-01T03:11:31.673944+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.660812+0000","last_change":"2023-12-01T03:11:31.660812+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.660812+0000","last_undegraded":"2023-12-01T03:11:31.660812+0000","last_fullsized":"2023-12-01T03:11:31.660812+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:47:03.446609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.677149+0000","last_change":"2023-12-01T03:11:31.676480+0000","last_active":"2023-12-01T03:11:31.677149+0000","last_peered":"2023-12-01T03:11:31.677149+0000","last_clean":"2023-12-01T03:11:31.677149+0000","last_became_active":"2023-12-01T03:11:31.676127+0000","last_became_peered":"2023-12-01T03:11:31.676127+0000","last_unstale":"2023-12-01T03:11:31.677149+0000","last_undegraded":"2023-12-01T03:11:31.677149+0000","last_fullsized":"2023-12-01T03:11:31.677149+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.654692+0000","last_change":"2023-12-01T03:11:31.654692+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.654692+0000","last_undegraded":"2023-12-01T03:11:31.654692+0000","last_fullsized":"2023-12-01T03:11:31.654692+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.681381+0000","last_change":"2023-12-01T03:11:31.680251+0000","last_active":"2023-12-01T03:11:31.681381+0000","last_peered":"2023-12-01T03:11:31.681381+0000","last_clean":"2023-12-01T03:11:31.681381+0000","last_became_active":"2023-12-01T03:11:31.679936+0000","last_became_peered":"2023-12-01T03:11:31.679936+0000","last_unstale":"2023-12-01T03:11:31.681381+0000","last_undegraded":"2023-12-01T03:11:31.681381+0000","last_fullsized":"2023-12-01T03:11:31.681381+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:39.296836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.653762+0000","last_change":"2023-12-01T03:11:31.653762+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.653762+0000","last_undegraded":"2023-12-01T03:11:31.653762+0000","last_fullsized":"2023-12-01T03:11:31.653762+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.682554+0000","last_change":"2023-12-01T03:11:31.681391+0000","last_active":"2023-12-01T03:11:31.682554+0000","last_peered":"2023-12-01T03:11:31.682554+0000","last_clean":"2023-12-01T03:11:31.682554+0000","last_became_active":"2023-12-01T03:11:31.680773+0000","last_became_peered":"2023-12-01T03:11:31.680773+0000","last_unstale":"2023-12-01T03:11:31.682554+0000","last_undegraded":"2023-12-01T03:11:31.682554+0000","last_fullsized":"2023-12-01T03:11:31.682554+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:40:13.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"220'348","reported_seq":1212,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-01T03:11:30.647156+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:11:30.647156+0000","last_peered":"2023-12-01T03:11:30.647156+0000","last_clean":"2023-12-01T03:11:30.647156+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:11:30.647156+0000","last_undegraded":"2023-12-01T03:11:30.647156+0000","last_fullsized":"2023-12-01T03:11:30.647156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575988,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60136,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311704,"statfs":{"total":96636764160,"available":96575184896,"internally_reserved":0,"allocated":2916352,"data_stored":1427986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.05,"15min":1.071},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5409999999999999,"5min":1.6000000000000001,"15min":1.7390000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6599999999999999,"5min":1.6699999999999999,"15min":1.7},"last":0.873}]},{"osd":1,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.236,"15min":1.2769999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4990000000000001,"5min":1.78,"15min":2.1419999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.294,"15min":1.3260000000000001},"min":{"1min":0.873,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5740000000000001,"5min":1.6240000000000001,"15min":1.8700000000000001},"last":0.99399999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1439999999999999,"15min":1.173},"min":{"1min":0.71699999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.573,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.2549999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.643,"5min":1.7050000000000001,"15min":2.1120000000000001},"last":1.4119999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2509999999999999,"15min":1.302},"min":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.712,"15min":2.431},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3720000000000001,"15min":1.4119999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7529999999999999,"5min":1.879,"15min":2.198},"last":1.4610000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3009999999999999,"15min":1.335},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7689999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.4510000000000001,"15min":1.49},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8799999999999999,"5min":1.97,"15min":2.569},"last":1.514}]},{"osd":5,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2869999999999999,"15min":1.3160000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7150000000000001,"5min":2.0099999999999998,"15min":2.7269999999999999},"last":1.159},{"interface":"front","average":{"1min":1.498,"5min":1.5469999999999999,"15min":1.585},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.0470000000000002,"5min":2.0569999999999999,"15min":2.484},"last":1.5589999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.403,"15min":1.4359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7749999999999999,"5min":1.921,"15min":2.532},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4299999999999999,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8260000000000001,"5min":1.8819999999999999,"15min":2.762},"last":1.296}]}]},{"osd":6,"up_from":13,"seq":55834575989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42088,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329752,"statfs":{"total":96636764160,"available":96593666048,"internally_reserved":0,"allocated":2916352,"data_stored":1427986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0509999999999999,"15min":1.1080000000000001},"min":{"1min":0.66100000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.1040000000000001,"15min":1.157},"min":{"1min":0.78400000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6910000000000001,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.333}]},{"osd":1,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.141,"15min":1.2150000000000001},"min":{"1min":0.68999999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8460000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1830000000000001,"15min":1.22},"min":{"1min":0.86099999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.6220000000000001,"5min":1.694,"15min":1.835},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.157,"15min":1.206},"min":{"1min":0.71799999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.464,"5min":1.742,"15min":1.976},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.222,"15min":1.278},"min":{"1min":0.71699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5009999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.2}]},{"osd":3,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2829999999999999,"15min":1.3140000000000001},"min":{"1min":0.89500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6839999999999999,"5min":1.927,"15min":2.0289999999999999},"last":1.105},{"interface":"front","average":{"1min":1.454,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":1.0069999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0190000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.3979999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":0.93600000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.589,"5min":1.407,"15min":1.367},"min":{"1min":1.2090000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.222,"5min":2.48,"15min":2.48},"last":1.157}]},{"osd":5,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.3700000000000001,"15min":1.413},"min":{"1min":1.127,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.962,"5min":2.262,"15min":2.262},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5169999999999999,"15min":1.4970000000000001},"min":{"1min":1.2629999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.194,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.2490000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5029999999999999,"15min":1.5309999999999999},"min":{"1min":1.155,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1320000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.746,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":1.371,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2770000000000001,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.48}]}]},{"osd":5,"up_from":13,"seq":55834575988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52536,"kb_used_data":3440,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319304,"statfs":{"total":96636764160,"available":96582967296,"internally_reserved":0,"allocated":3522560,"data_stored":2024673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1000000000000001,"15min":1.123},"min":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.7170000000000001,"5min":1.776,"15min":1.776},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1539999999999999,"15min":1.161},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.752},"last":0.83499999999999996}]},{"osd":1,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.1559999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6120000000000001,"5min":1.6519999999999999,"15min":1.696},"last":0.747},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.198,"15min":1.2330000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":0.98699999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2270000000000001,"15min":1.2470000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.81399999999999995},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.2450000000000001,"15min":1.254},"min":{"1min":0.75,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.867,"5min":1.899,"15min":1.9219999999999999},"last":1.1339999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3089999999999999,"15min":1.3149999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.889,"5min":1.889,"15min":1.8979999999999999},"last":1.29},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.329,"15min":1.359},"min":{"1min":1.0109999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.97,"5min":1.97,"15min":2.1859999999999999},"last":1.256}]},{"osd":4,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.869,"5min":1.869,"15min":2.6040000000000001},"last":1.196},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.407}]},{"osd":6,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3779999999999999,"15min":1.377},"min":{"1min":0.99099999999999999,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.7649999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.367},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.409,"15min":1.4219999999999999},"min":{"1min":1.0149999999999999,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":2.032,"5min":2.032,"15min":2.577},"last":1.5329999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.534,"15min":1.538},"min":{"1min":1.2010000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":2.077,"5min":2.077,"15min":3.1709999999999998},"last":1.591},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5760000000000001,"15min":1.5780000000000001},"min":{"1min":1.3200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.101,"15min":2.8740000000000001},"last":1.472}]}]},{"osd":4,"up_from":13,"seq":55834575988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51956,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319884,"statfs":{"total":96636764160,"available":96583561216,"internally_reserved":0,"allocated":2928640,"data_stored":1434305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1200000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8089999999999999,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.212},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.198,"15min":1.1899999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5309999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":0.89700000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2290000000000001,"15min":1.256},"min":{"1min":0.83599999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8580000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1719999999999999,"15min":1.175},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.778,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":0.81200000000000006}]},{"osd":2,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3300000000000001,"15min":1.244},"min":{"1min":0.74399999999999999,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.637,"5min":11.311,"15min":11.311},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3320000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9330000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.056}]},{"osd":3,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.298,"15min":1.29},"min":{"1min":0.80000000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.716,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.095},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3759999999999999,"15min":1.3340000000000001},"min":{"1min":0.96399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7430000000000001,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.4399999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.357,"15min":1.3200000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.738,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.375,"15min":1.3160000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6579999999999999,"5min":3.71,"15min":3.71},"last":1.302}]},{"osd":6,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3799999999999999,"15min":1.3740000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8009999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.361},{"interface":"front","average":{"1min":1.419,"5min":1.4790000000000001,"15min":1.476},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.76,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.494}]},{"osd":7,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.514,"15min":1.488},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9830000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.655},{"interface":"front","average":{"1min":1.532,"5min":1.597,"15min":1.575},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.907,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.706}]}]},{"osd":3,"up_from":13,"seq":55834575987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56040,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315800,"statfs":{"total":96636764160,"available":96579379200,"internally_reserved":0,"allocated":2916352,"data_stored":1427986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.91300000000000003,"15min":0.90700000000000003},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.503},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1399999999999999,"15min":1.167},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.788,"5min":3.863,"15min":3.863},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.073,"15min":1.0960000000000001},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.637,"5min":3.597,"15min":3.597},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.127,"5min":1.2090000000000001,"15min":1.2410000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.8319999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2969999999999999,"15min":1.294},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.272,"15min":1.296},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.736,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0049999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.331,"15min":1.329},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.1240000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.373,"15min":1.373},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.425,"15min":1.444},"min":{"1min":0.80500000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.252},{"interface":"front","average":{"1min":1.504,"5min":1.5089999999999999,"15min":1.5129999999999999},"min":{"1min":1.0089999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.92200000000000004}]},{"osd":7,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5129999999999999,"15min":1.5249999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.899,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.242},{"interface":"front","average":{"1min":1.45,"5min":1.5209999999999999,"15min":1.526},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":1.3129999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575987,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47848,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323992,"statfs":{"total":96636764160,"available":96587767808,"internally_reserved":0,"allocated":2916352,"data_stored":1427986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.411,"5min":1.411,"15min":1.4730000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.052,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3080000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.0229999999999999,"15min":1.0489999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.343,"5min":1.698,"15min":1.698},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.097,"15min":1.135},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.4730000000000001,"5min":1.635,"15min":1.6599999999999999},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.026,"15min":1.042},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.393,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.497},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.159,"15min":1.208},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4950000000000001,"5min":1.6879999999999999,"15min":1.79},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.319,"15min":1.327},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5700000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.752},{"interface":"front","average":{"1min":1.194,"5min":1.2689999999999999,"15min":1.294},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5189999999999999,"5min":1.714,"15min":1.835},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3560000000000001,"15min":1.3839999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.782,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.081},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.407,"15min":1.411},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.873,"5min":1.8740000000000001,"15min":2.016},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3660000000000001,"15min":1.3799999999999999},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.714,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4730000000000001,"15min":1.4970000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.1870000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.478,"15min":1.4850000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.9350000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.389,"5min":1.4259999999999999,"15min":1.454},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8149999999999999,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.145}]}]},{"osd":1,"up_from":13,"seq":55834575986,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48428,"kb_used_data":3428,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323412,"statfs":{"total":96636764160,"available":96587173888,"internally_reserved":0,"allocated":3510272,"data_stored":2018354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.89800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.238,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.98499999999999999,"15min":1.038},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.512,"5min":1.5660000000000001,"15min":1.681},"last":1.167}]},{"osd":2,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2190000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.048,"5min":1.1020000000000001,"15min":1.141},"min":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5489999999999999,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.208,"15min":1.2949999999999999},"min":{"1min":0.60799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5860000000000001,"5min":1.587,"15min":1.788},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.0780000000000001,"15min":1.127},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.383,"5min":1.714,"15min":1.8049999999999999},"last":1.139}]},{"osd":4,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2030000000000001,"15min":1.218},"min":{"1min":0.84399999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.623,"5min":1.623,"15min":1.6859999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3700000000000001,"15min":1.4039999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.657,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.268}]},{"osd":5,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3109999999999999,"15min":1.333},"min":{"1min":0.95899999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":2.101},"last":1.722},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.298,"15min":1.331},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.9019999999999999,"5min":1.927,"15min":1.927},"last":1.7470000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0529999999999999,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3939999999999999,"15min":1.413},"min":{"1min":0.91600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.6719999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4650000000000001,"15min":1.522},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.9430000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":1.577},{"interface":"front","average":{"1min":1.524,"5min":1.48,"15min":1.5109999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.0699999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.863}]}]},{"osd":0,"up_from":13,"seq":55834575988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60136,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311704,"statfs":{"total":96636764160,"available":96575184896,"internally_reserved":0,"allocated":2916352,"data_stored":1427986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.88900000000000001},"min":{"1min":0.314,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.994},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.98899999999999999,"15min":1.048},"min":{"1min":0.58999999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6799999999999999},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0169999999999999,"15min":1.0640000000000001},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.635,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0589999999999999,"15min":1.1259999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3100000000000001,"5min":1.726,"15min":1.792},"last":0.96099999999999997}]},{"osd":3,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.036,"15min":1.0669999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.8839999999999999},"last":0.999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.171,"15min":1.224},"min":{"1min":0.70499999999999996,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.6469999999999998},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.296,"15min":1.3520000000000001},"min":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.681,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":1.05},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.3029999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1279999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.90700000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.899,"15min":2.7719999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.383,"15min":1.427},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.681,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":1.3420000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3520000000000001,"15min":1.381},"min":{"1min":0.84599999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7270000000000001,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":1.22},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.45,"15min":1.4830000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.1200000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4670000000000001,"15min":1.516},"min":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7969999999999999,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":1.286},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4730000000000001,"15min":1.4870000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.927,"5min":2.1190000000000002,"15min":2.431},"last":1.3120000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:11:37.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:11:37.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.305+0000 7f2b3a539640 1 Processor -- start 2023-12-01T03:11:37.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.305+0000 7f2b3a539640 1 -- start start 2023-12-01T03:11:37.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.306+0000 7f2b3a539640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2b3406f2f0 0x7f2b3406bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.306+0000 7f2b3a539640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b340add10 con 0x7f2b3415e2f0 2023-12-01T03:11:37.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.306+0000 7f2b3a539640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b340c5760 con 0x7f2b340cad70 2023-12-01T03:11:37.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.306+0000 7f2b3a539640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3415e6d0 con 0x7f2b3406f2f0 2023-12-01T03:11:37.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.306+0000 7f2b33fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2b3415e2f0 0x7f2b340ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56994/0 (socket says 172.21.15.115:56994) 2023-12-01T03:11:37.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.306+0000 7f2b33fff640 1 -- 172.21.15.115:0/2319413411 learned_addr learned my addr 172.21.15.115:0/2319413411 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:37.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825414988 0 0) 0x7f2b340add10 con 0x7f2b3415e2f0 2023-12-01T03:11:37.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b1c003520 con 0x7f2b3415e2f0 2023-12-01T03:11:37.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b38ad6640 1 --2- 172.21.15.115:0/2319413411 >> v2:172.21.15.142:3300/0 conn(0x7f2b3406f2f0 0x7f2b3406bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 853184297 0 0) 0x7f2b1c003520 con 0x7f2b3415e2f0 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 >> v2:172.21.15.142:3300/0 conn(0x7f2b3406f2f0 msgr2=0x7f2b3406bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 --2- 172.21.15.115:0/2319413411 >> v2:172.21.15.142:3300/0 conn(0x7f2b3406f2f0 0x7f2b3406bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 >> v1:172.21.15.142:6789/0 conn(0x7f2b340cad70 legacy=0x7f2b3406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340cf7b0 con 0x7f2b3415e2f0 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2b24001690 con 0x7f2b3415e2f0 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 >> v1:172.21.15.115:6789/0 conn(0x7f2b3415e2f0 legacy=0x7f2b340ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 --2- 172.21.15.115:0/2319413411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b1c003a60 0x7f2b1c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 --2- 172.21.15.115:0/2319413411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b1c0047c0 0x7f2b1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 --2- 172.21.15.115:0/2319413411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b1c0053d0 0x7f2b1c00ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3415e6d0 con 0x7f2b1c0047c0 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c00c2e0 con 0x7f2b1c003a60 2023-12-01T03:11:37.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.307+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c00c4a0 con 0x7f2b1c0053d0 2023-12-01T03:11:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.308+0000 7f2b337fe640 1 --2- 172.21.15.115:0/2319413411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b1c0047c0 0x7f2b1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.308+0000 7f2b33fff640 1 --2- 172.21.15.115:0/2319413411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b1c003a60 0x7f2b1c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.308+0000 7f2b337fe640 1 -- 172.21.15.115:0/2319413411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b1c0053d0 msgr2=0x7f2b1c00ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.308+0000 7f2b337fe640 1 --2- 172.21.15.115:0/2319413411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b1c0053d0 0x7f2b1c00ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.308+0000 7f2b337fe640 1 -- 172.21.15.115:0/2319413411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b1c003a60 msgr2=0x7f2b1c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.308+0000 7f2b337fe640 1 --2- 172.21.15.115:0/2319413411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b1c003a60 0x7f2b1c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.308+0000 7f2b337fe640 1 -- 172.21.15.115:0/2319413411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c003520 con 0x7f2b1c0047c0 2023-12-01T03:11:37.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.308+0000 7f2b38ad6640 1 -- 172.21.15.115:0/2319413411 reap_dead start 2023-12-01T03:11:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.309+0000 7f2b337fe640 1 --2- 172.21.15.115:0/2319413411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b1c0047c0 0x7f2b1c004c40 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f2b28003db0 tx=0x7f2b1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec29995b8d8eb09 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.309+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28032600 con 0x7f2b1c0047c0 2023-12-01T03:11:37.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.309+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2802e050 con 0x7f2b1c0047c0 2023-12-01T03:11:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.309+0000 7f2b32ffd640 1 -- 172.21.15.115:0/2319413411 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b280329e0 con 0x7f2b1c0047c0 2023-12-01T03:11:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.310+0000 7f2b3a539640 1 -- 172.21.15.115:0/2319413411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b1c0047c0 msgr2=0x7f2b1c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.310+0000 7f2b3a539640 1 --2- 172.21.15.115:0/2319413411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b1c0047c0 0x7f2b1c004c40 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f2b28003db0 tx=0x7f2b1c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.310+0000 7f2b3a539640 1 -- 172.21.15.115:0/2319413411 shutdown_connections 2023-12-01T03:11:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.310+0000 7f2b3a539640 1 --2- 172.21.15.115:0/2319413411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b1c0047c0 0x7f2b1c004c40 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.310+0000 7f2b3a539640 1 -- 172.21.15.115:0/2319413411 >> 172.21.15.115:0/2319413411 conn(0x7f2b340ce880 msgr2=0x7f2b340cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.310+0000 7f2b3a539640 1 -- 172.21.15.115:0/2319413411 shutdown_connections 2023-12-01T03:11:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.310+0000 7f2b3a539640 1 -- 172.21.15.115:0/2319413411 wait complete. 2023-12-01T03:11:37.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.312+0000 7f2b3a539640 1 Processor -- start 2023-12-01T03:11:37.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.312+0000 7f2b3a539640 1 -- start start 2023-12-01T03:11:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b3a539640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b1c0047c0 0x7f2b3415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b3a539640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b340c4fd0 0x7f2b341f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b3a539640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b340c5950 0x7f2b341f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b3a539640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340af920 con 0x7f2b340c4fd0 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b3a539640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3415a6a0 con 0x7f2b340c5950 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b3a539640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340c5e30 con 0x7f2b1c0047c0 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b340c4fd0 0x7f2b341f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b337fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b340c4fd0 0x7f2b341f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38840/0 (socket says 172.21.15.115:38840) 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b337fe640 1 -- 172.21.15.115:0/308510753 learned_addr learned my addr 172.21.15.115:0/308510753 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b337fe640 1 -- 172.21.15.115:0/308510753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b1c0047c0 msgr2=0x7f2b3415a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b337fe640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b1c0047c0 0x7f2b3415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b337fe640 1 -- 172.21.15.115:0/308510753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b340c5950 msgr2=0x7f2b341f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b337fe640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b340c5950 0x7f2b341f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.313+0000 7f2b337fe640 1 -- 172.21.15.115:0/308510753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340afbc0 con 0x7f2b340c4fd0 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.314+0000 7f2b337fe640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b340c4fd0 0x7f2b341f5a50 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f2b28035040 tx=0x7f2b280036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.314+0000 7f2b30ff9640 1 -- 172.21.15.115:0/308510753 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2803c610 con 0x7f2b340c4fd0 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.314+0000 7f2b30ff9640 1 -- 172.21.15.115:0/308510753 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2803c770 con 0x7f2b340c4fd0 2023-12-01T03:11:37.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.314+0000 7f2b3a539640 1 -- 172.21.15.115:0/308510753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b340af920 con 0x7f2b340c4fd0 2023-12-01T03:11:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.314+0000 7f2b30ff9640 1 -- 172.21.15.115:0/308510753 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2803cb00 con 0x7f2b340c4fd0 2023-12-01T03:11:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.314+0000 7f2b3a539640 1 -- 172.21.15.115:0/308510753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b340b0180 con 0x7f2b340c4fd0 2023-12-01T03:11:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.315+0000 7f2b3a539640 1 -- 172.21.15.115:0/308510753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3406fda0 con 0x7f2b340c4fd0 2023-12-01T03:11:37.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.316+0000 7f2b30ff9640 1 -- 172.21.15.115:0/308510753 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f2b28039050 con 0x7f2b340c4fd0 2023-12-01T03:11:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.317+0000 7f2b30ff9640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2b2006a670 0x7f2b2006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.317+0000 7f2b30ff9640 1 -- 172.21.15.115:0/308510753 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f2b280aa5c0 con 0x7f2b340c4fd0 2023-12-01T03:11:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.317+0000 7f2b33fff640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2b2006a670 0x7f2b2006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.318+0000 7f2b33fff640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2b2006a670 0x7f2b2006cb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2b340adea0 tx=0x7f2b24001cb0 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:37.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.321+0000 7f2b30ff9640 1 -- 172.21.15.115:0/308510753 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2b28075740 con 0x7f2b340c4fd0 2023-12-01T03:11:37.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.508+0000 7f2b3a539640 1 -- 172.21.15.115:0/308510753 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b340b43d0 con 0x7f2b2006a670 2023-12-01T03:11:37.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.530+0000 7f2b30ff9640 1 -- 172.21.15.115:0/308510753 <== mgr.11121 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284410 (secure 0 0 0) 0x7f2b24005da0 con 0x7f2b2006a670 2023-12-01T03:11:37.537 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:37.539 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:11:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.535+0000 7f2b3a539640 1 -- 172.21.15.115:0/308510753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2b2006a670 msgr2=0x7f2b2006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.535+0000 7f2b3a539640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2b2006a670 0x7f2b2006cb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2b340adea0 tx=0x7f2b24001cb0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.535+0000 7f2b3a539640 1 -- 172.21.15.115:0/308510753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b340c4fd0 msgr2=0x7f2b341f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.535+0000 7f2b3a539640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b340c4fd0 0x7f2b341f5a50 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f2b28035040 tx=0x7f2b280036f0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.535+0000 7f2b3a539640 1 -- 172.21.15.115:0/308510753 shutdown_connections 2023-12-01T03:11:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.535+0000 7f2b3a539640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2b2006a670 0x7f2b2006cb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.535+0000 7f2b3a539640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b340c5950 0x7f2b341f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.535+0000 7f2b3a539640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b340c4fd0 0x7f2b341f5a50 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.535+0000 7f2b3a539640 1 --2- 172.21.15.115:0/308510753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b1c0047c0 0x7f2b3415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.535+0000 7f2b3a539640 1 -- 172.21.15.115:0/308510753 >> 172.21.15.115:0/308510753 conn(0x7f2b340ce880 msgr2=0x7f2b3415cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.536+0000 7f2b3a539640 1 -- 172.21.15.115:0/308510753 shutdown_connections 2023-12-01T03:11:37.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.536+0000 7f2b3a539640 1 -- 172.21.15.115:0/308510753 wait complete. 2023-12-01T03:11:37.555 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-12-01T03:11:35.822351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":419168,"kb_used_data":23968,"kb_used_omap":0,"kb_used_meta":395136,"kb_avail":754555552,"statfs":{"total":773094113280,"available":772664885248,"internally_reserved":0,"allocated":24543232,"data_stored":12617262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":404619264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190259"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.672100+0000","last_change":"2023-12-01T03:11:31.672100+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.672100+0000","last_undegraded":"2023-12-01T03:11:31.672100+0000","last_fullsized":"2023-12-01T03:11:31.672100+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662282+0000","last_change":"2023-12-01T03:11:31.662282+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662282+0000","last_undegraded":"2023-12-01T03:11:31.662282+0000","last_fullsized":"2023-12-01T03:11:31.662282+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:10.849908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.667239+0000","last_change":"2023-12-01T03:11:31.667239+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.667239+0000","last_undegraded":"2023-12-01T03:11:31.667239+0000","last_fullsized":"2023-12-01T03:11:31.667239+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:38.174827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.646957+0000","last_change":"2023-12-01T03:11:31.646957+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.646957+0000","last_undegraded":"2023-12-01T03:11:31.646957+0000","last_fullsized":"2023-12-01T03:11:31.646957+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:43.961392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.649740+0000","last_change":"2023-12-01T03:11:31.649740+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.649740+0000","last_undegraded":"2023-12-01T03:11:31.649740+0000","last_fullsized":"2023-12-01T03:11:31.649740+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.655118+0000","last_change":"2023-12-01T03:11:31.655118+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.655118+0000","last_undegraded":"2023-12-01T03:11:31.655118+0000","last_fullsized":"2023-12-01T03:11:31.655118+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.666619+0000","last_change":"2023-12-01T03:11:31.666619+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.666619+0000","last_undegraded":"2023-12-01T03:11:31.666619+0000","last_fullsized":"2023-12-01T03:11:31.666619+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:19.277681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.655923+0000","last_change":"2023-12-01T03:11:31.655923+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.655923+0000","last_undegraded":"2023-12-01T03:11:31.655923+0000","last_fullsized":"2023-12-01T03:11:31.655923+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:51.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.653833+0000","last_change":"2023-12-01T03:11:31.653833+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.653833+0000","last_undegraded":"2023-12-01T03:11:31.653833+0000","last_fullsized":"2023-12-01T03:11:31.653833+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.650471+0000","last_change":"2023-12-01T03:11:31.650471+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.650471+0000","last_undegraded":"2023-12-01T03:11:31.650471+0000","last_fullsized":"2023-12-01T03:11:31.650471+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:44:42.356924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662851+0000","last_change":"2023-12-01T03:11:31.662851+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662851+0000","last_undegraded":"2023-12-01T03:11:31.662851+0000","last_fullsized":"2023-12-01T03:11:31.662851+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:35.627224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.659273+0000","last_change":"2023-12-01T03:11:31.659273+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.659273+0000","last_undegraded":"2023-12-01T03:11:31.659273+0000","last_fullsized":"2023-12-01T03:11:31.659273+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665375+0000","last_change":"2023-12-01T03:11:31.665375+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665375+0000","last_undegraded":"2023-12-01T03:11:31.665375+0000","last_fullsized":"2023-12-01T03:11:31.665375+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:04.079396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665123+0000","last_change":"2023-12-01T03:11:31.665123+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665123+0000","last_undegraded":"2023-12-01T03:11:31.665123+0000","last_fullsized":"2023-12-01T03:11:31.665123+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:21.336618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.646269+0000","last_change":"2023-12-01T03:11:31.646269+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.646269+0000","last_undegraded":"2023-12-01T03:11:31.646269+0000","last_fullsized":"2023-12-01T03:11:31.646269+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.653126+0000","last_change":"2023-12-01T03:11:31.653126+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.653126+0000","last_undegraded":"2023-12-01T03:11:31.653126+0000","last_fullsized":"2023-12-01T03:11:31.653126+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.670984+0000","last_change":"2023-12-01T03:11:31.670984+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.670984+0000","last_undegraded":"2023-12-01T03:11:31.670984+0000","last_fullsized":"2023-12-01T03:11:31.670984+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665505+0000","last_change":"2023-12-01T03:11:31.665505+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665505+0000","last_undegraded":"2023-12-01T03:11:31.665505+0000","last_fullsized":"2023-12-01T03:11:31.665505+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:42.720564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.652287+0000","last_change":"2023-12-01T03:11:31.652287+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.652287+0000","last_undegraded":"2023-12-01T03:11:31.652287+0000","last_fullsized":"2023-12-01T03:11:31.652287+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665034+0000","last_change":"2023-12-01T03:11:31.665034+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665034+0000","last_undegraded":"2023-12-01T03:11:31.665034+0000","last_fullsized":"2023-12-01T03:11:31.665034+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:10.441679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662847+0000","last_change":"2023-12-01T03:11:31.662847+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662847+0000","last_undegraded":"2023-12-01T03:11:31.662847+0000","last_fullsized":"2023-12-01T03:11:31.662847+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:53:43.178969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.651584+0000","last_change":"2023-12-01T03:11:31.651584+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.651584+0000","last_undegraded":"2023-12-01T03:11:31.651584+0000","last_fullsized":"2023-12-01T03:11:31.651584+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665570+0000","last_change":"2023-12-01T03:11:31.665570+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665570+0000","last_undegraded":"2023-12-01T03:11:31.665570+0000","last_fullsized":"2023-12-01T03:11:31.665570+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:00.674458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.649271+0000","last_change":"2023-12-01T03:11:31.649271+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.649271+0000","last_undegraded":"2023-12-01T03:11:31.649271+0000","last_fullsized":"2023-12-01T03:11:31.649271+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:29.661974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.661942+0000","last_change":"2023-12-01T03:11:31.661942+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.661942+0000","last_undegraded":"2023-12-01T03:11:31.661942+0000","last_fullsized":"2023-12-01T03:11:31.661942+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:09.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.648127+0000","last_change":"2023-12-01T03:11:31.648127+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.648127+0000","last_undegraded":"2023-12-01T03:11:31.648127+0000","last_fullsized":"2023-12-01T03:11:31.648127+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:35.975600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.663265+0000","last_change":"2023-12-01T03:11:31.663265+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.663265+0000","last_undegraded":"2023-12-01T03:11:31.663265+0000","last_fullsized":"2023-12-01T03:11:31.663265+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:28.852429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.654435+0000","last_change":"2023-12-01T03:11:31.654435+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.654435+0000","last_undegraded":"2023-12-01T03:11:31.654435+0000","last_fullsized":"2023-12-01T03:11:31.654435+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:14.716448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.655436+0000","last_change":"2023-12-01T03:11:31.655436+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.655436+0000","last_undegraded":"2023-12-01T03:11:31.655436+0000","last_fullsized":"2023-12-01T03:11:31.655436+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:28:34.324085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.658499+0000","last_change":"2023-12-01T03:11:31.658499+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.658499+0000","last_undegraded":"2023-12-01T03:11:31.658499+0000","last_fullsized":"2023-12-01T03:11:31.658499+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662619+0000","last_change":"2023-12-01T03:11:31.662619+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662619+0000","last_undegraded":"2023-12-01T03:11:31.662619+0000","last_fullsized":"2023-12-01T03:11:31.662619+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:36.571608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.654612+0000","last_change":"2023-12-01T03:11:31.654612+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.654612+0000","last_undegraded":"2023-12-01T03:11:31.654612+0000","last_fullsized":"2023-12-01T03:11:31.654612+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:14.760284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.661833+0000","last_change":"2023-12-01T03:11:31.661833+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.661833+0000","last_undegraded":"2023-12-01T03:11:31.661833+0000","last_fullsized":"2023-12-01T03:11:31.661833+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:24.244094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.681672+0000","last_change":"2023-12-01T03:11:31.680864+0000","last_active":"2023-12-01T03:11:31.681672+0000","last_peered":"2023-12-01T03:11:31.681672+0000","last_clean":"2023-12-01T03:11:31.681672+0000","last_became_active":"2023-12-01T03:11:31.680279+0000","last_became_peered":"2023-12-01T03:11:31.680279+0000","last_unstale":"2023-12-01T03:11:31.681672+0000","last_undegraded":"2023-12-01T03:11:31.681672+0000","last_fullsized":"2023-12-01T03:11:31.681672+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.377359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.650323+0000","last_change":"2023-12-01T03:11:31.650323+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.650323+0000","last_undegraded":"2023-12-01T03:11:31.650323+0000","last_fullsized":"2023-12-01T03:11:31.650323+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.681207+0000","last_change":"2023-12-01T03:11:31.679798+0000","last_active":"2023-12-01T03:11:31.681207+0000","last_peered":"2023-12-01T03:11:31.681207+0000","last_clean":"2023-12-01T03:11:31.681207+0000","last_became_active":"2023-12-01T03:11:31.679030+0000","last_became_peered":"2023-12-01T03:11:31.679030+0000","last_unstale":"2023-12-01T03:11:31.681207+0000","last_undegraded":"2023-12-01T03:11:31.681207+0000","last_fullsized":"2023-12-01T03:11:31.681207+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:49.261314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.655235+0000","last_change":"2023-12-01T03:11:31.655235+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.655235+0000","last_undegraded":"2023-12-01T03:11:31.655235+0000","last_fullsized":"2023-12-01T03:11:31.655235+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:17.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.666885+0000","last_change":"2023-12-01T03:11:31.663745+0000","last_active":"2023-12-01T03:11:31.666885+0000","last_peered":"2023-12-01T03:11:31.666885+0000","last_clean":"2023-12-01T03:11:31.666885+0000","last_became_active":"2023-12-01T03:11:31.662706+0000","last_became_peered":"2023-12-01T03:11:31.662706+0000","last_unstale":"2023-12-01T03:11:31.666885+0000","last_undegraded":"2023-12-01T03:11:31.666885+0000","last_fullsized":"2023-12-01T03:11:31.666885+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:42.945449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.664632+0000","last_change":"2023-12-01T03:11:31.664632+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.664632+0000","last_undegraded":"2023-12-01T03:11:31.664632+0000","last_fullsized":"2023-12-01T03:11:31.664632+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:22:51.687377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.661731+0000","last_change":"2023-12-01T03:11:31.660758+0000","last_active":"2023-12-01T03:11:31.661731+0000","last_peered":"2023-12-01T03:11:31.661731+0000","last_clean":"2023-12-01T03:11:31.661731+0000","last_became_active":"2023-12-01T03:11:31.660001+0000","last_became_peered":"2023-12-01T03:11:31.660001+0000","last_unstale":"2023-12-01T03:11:31.661731+0000","last_undegraded":"2023-12-01T03:11:31.661731+0000","last_fullsized":"2023-12-01T03:11:31.661731+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.646204+0000","last_change":"2023-12-01T03:11:31.646204+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.646204+0000","last_undegraded":"2023-12-01T03:11:31.646204+0000","last_fullsized":"2023-12-01T03:11:31.646204+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.682460+0000","last_change":"2023-12-01T03:11:31.681531+0000","last_active":"2023-12-01T03:11:31.682460+0000","last_peered":"2023-12-01T03:11:31.682460+0000","last_clean":"2023-12-01T03:11:31.682460+0000","last_became_active":"2023-12-01T03:11:31.680782+0000","last_became_peered":"2023-12-01T03:11:31.680782+0000","last_unstale":"2023-12-01T03:11:31.682460+0000","last_undegraded":"2023-12-01T03:11:31.682460+0000","last_fullsized":"2023-12-01T03:11:31.682460+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:18.018055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.649611+0000","last_change":"2023-12-01T03:11:31.649611+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.649611+0000","last_undegraded":"2023-12-01T03:11:31.649611+0000","last_fullsized":"2023-12-01T03:11:31.649611+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.666436+0000","last_change":"2023-12-01T03:11:31.663166+0000","last_active":"2023-12-01T03:11:31.666436+0000","last_peered":"2023-12-01T03:11:31.666436+0000","last_clean":"2023-12-01T03:11:31.666436+0000","last_became_active":"2023-12-01T03:11:31.662263+0000","last_became_peered":"2023-12-01T03:11:31.662263+0000","last_unstale":"2023-12-01T03:11:31.666436+0000","last_undegraded":"2023-12-01T03:11:31.666436+0000","last_fullsized":"2023-12-01T03:11:31.666436+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:57:00.965499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.664744+0000","last_change":"2023-12-01T03:11:31.664744+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.664744+0000","last_undegraded":"2023-12-01T03:11:31.664744+0000","last_fullsized":"2023-12-01T03:11:31.664744+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.674834+0000","last_change":"2023-12-01T03:11:31.670524+0000","last_active":"2023-12-01T03:11:31.674834+0000","last_peered":"2023-12-01T03:11:31.674834+0000","last_clean":"2023-12-01T03:11:31.674834+0000","last_became_active":"2023-12-01T03:11:31.670190+0000","last_became_peered":"2023-12-01T03:11:31.670190+0000","last_unstale":"2023-12-01T03:11:31.674834+0000","last_undegraded":"2023-12-01T03:11:31.674834+0000","last_fullsized":"2023-12-01T03:11:31.674834+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:45.874739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.658638+0000","last_change":"2023-12-01T03:11:31.658638+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.658638+0000","last_undegraded":"2023-12-01T03:11:31.658638+0000","last_fullsized":"2023-12-01T03:11:31.658638+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.677746+0000","last_change":"2023-12-01T03:11:31.676322+0000","last_active":"2023-12-01T03:11:31.677746+0000","last_peered":"2023-12-01T03:11:31.677746+0000","last_clean":"2023-12-01T03:11:31.677746+0000","last_became_active":"2023-12-01T03:11:31.674451+0000","last_became_peered":"2023-12-01T03:11:31.674451+0000","last_unstale":"2023-12-01T03:11:31.677746+0000","last_undegraded":"2023-12-01T03:11:31.677746+0000","last_fullsized":"2023-12-01T03:11:31.677746+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:44.820233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.661499+0000","last_change":"2023-12-01T03:11:31.661499+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.661499+0000","last_undegraded":"2023-12-01T03:11:31.661499+0000","last_fullsized":"2023-12-01T03:11:31.661499+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:40.557913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.664811+0000","last_change":"2023-12-01T03:11:31.662929+0000","last_active":"2023-12-01T03:11:31.664811+0000","last_peered":"2023-12-01T03:11:31.664811+0000","last_clean":"2023-12-01T03:11:31.664811+0000","last_became_active":"2023-12-01T03:11:31.662617+0000","last_became_peered":"2023-12-01T03:11:31.662617+0000","last_unstale":"2023-12-01T03:11:31.664811+0000","last_undegraded":"2023-12-01T03:11:31.664811+0000","last_fullsized":"2023-12-01T03:11:31.664811+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:34.611280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.657685+0000","last_change":"2023-12-01T03:11:31.657685+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.657685+0000","last_undegraded":"2023-12-01T03:11:31.657685+0000","last_fullsized":"2023-12-01T03:11:31.657685+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.670983+0000","last_change":"2023-12-01T03:11:31.664798+0000","last_active":"2023-12-01T03:11:31.670983+0000","last_peered":"2023-12-01T03:11:31.670983+0000","last_clean":"2023-12-01T03:11:31.670983+0000","last_became_active":"2023-12-01T03:11:31.664276+0000","last_became_peered":"2023-12-01T03:11:31.664276+0000","last_unstale":"2023-12-01T03:11:31.670983+0000","last_undegraded":"2023-12-01T03:11:31.670983+0000","last_fullsized":"2023-12-01T03:11:31.670983+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:18:18.909277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.659121+0000","last_change":"2023-12-01T03:11:31.659121+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.659121+0000","last_undegraded":"2023-12-01T03:11:31.659121+0000","last_fullsized":"2023-12-01T03:11:31.659121+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.670077+0000","last_change":"2023-12-01T03:11:31.664162+0000","last_active":"2023-12-01T03:11:31.670077+0000","last_peered":"2023-12-01T03:11:31.670077+0000","last_clean":"2023-12-01T03:11:31.670077+0000","last_became_active":"2023-12-01T03:11:31.663854+0000","last_became_peered":"2023-12-01T03:11:31.663854+0000","last_unstale":"2023-12-01T03:11:31.670077+0000","last_undegraded":"2023-12-01T03:11:31.670077+0000","last_fullsized":"2023-12-01T03:11:31.670077+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:15.367696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.650462+0000","last_change":"2023-12-01T03:11:31.650462+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.650462+0000","last_undegraded":"2023-12-01T03:11:31.650462+0000","last_fullsized":"2023-12-01T03:11:31.650462+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:46.890682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.671195+0000","last_change":"2023-12-01T03:11:31.665987+0000","last_active":"2023-12-01T03:11:31.671195+0000","last_peered":"2023-12-01T03:11:31.671195+0000","last_clean":"2023-12-01T03:11:31.671195+0000","last_became_active":"2023-12-01T03:11:31.665550+0000","last_became_peered":"2023-12-01T03:11:31.665550+0000","last_unstale":"2023-12-01T03:11:31.671195+0000","last_undegraded":"2023-12-01T03:11:31.671195+0000","last_fullsized":"2023-12-01T03:11:31.671195+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:16.324155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.657130+0000","last_change":"2023-12-01T03:11:31.657130+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.657130+0000","last_undegraded":"2023-12-01T03:11:31.657130+0000","last_fullsized":"2023-12-01T03:11:31.657130+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.676301+0000","last_change":"2023-12-01T03:11:31.675387+0000","last_active":"2023-12-01T03:11:31.676301+0000","last_peered":"2023-12-01T03:11:31.676301+0000","last_clean":"2023-12-01T03:11:31.676301+0000","last_became_active":"2023-12-01T03:11:31.675108+0000","last_became_peered":"2023-12-01T03:11:31.675108+0000","last_unstale":"2023-12-01T03:11:31.676301+0000","last_undegraded":"2023-12-01T03:11:31.676301+0000","last_fullsized":"2023-12-01T03:11:31.676301+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.836486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.652517+0000","last_change":"2023-12-01T03:11:31.652517+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.652517+0000","last_undegraded":"2023-12-01T03:11:31.652517+0000","last_fullsized":"2023-12-01T03:11:31.652517+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.672130+0000","last_change":"2023-12-01T03:11:31.671174+0000","last_active":"2023-12-01T03:11:31.672130+0000","last_peered":"2023-12-01T03:11:31.672130+0000","last_clean":"2023-12-01T03:11:31.672130+0000","last_became_active":"2023-12-01T03:11:31.670862+0000","last_became_peered":"2023-12-01T03:11:31.670862+0000","last_unstale":"2023-12-01T03:11:31.672130+0000","last_undegraded":"2023-12-01T03:11:31.672130+0000","last_fullsized":"2023-12-01T03:11:31.672130+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:37.158883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.666137+0000","last_change":"2023-12-01T03:11:31.666137+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.666137+0000","last_undegraded":"2023-12-01T03:11:31.666137+0000","last_fullsized":"2023-12-01T03:11:31.666137+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:22.924077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.676600+0000","last_change":"2023-12-01T03:11:31.673143+0000","last_active":"2023-12-01T03:11:31.676600+0000","last_peered":"2023-12-01T03:11:31.676600+0000","last_clean":"2023-12-01T03:11:31.676600+0000","last_became_active":"2023-12-01T03:11:31.671261+0000","last_became_peered":"2023-12-01T03:11:31.671261+0000","last_unstale":"2023-12-01T03:11:31.676600+0000","last_undegraded":"2023-12-01T03:11:31.676600+0000","last_fullsized":"2023-12-01T03:11:31.676600+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:13.744340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.648564+0000","last_change":"2023-12-01T03:11:31.648564+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.648564+0000","last_undegraded":"2023-12-01T03:11:31.648564+0000","last_fullsized":"2023-12-01T03:11:31.648564+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:46.532354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.678553+0000","last_change":"2023-12-01T03:11:31.677841+0000","last_active":"2023-12-01T03:11:31.678553+0000","last_peered":"2023-12-01T03:11:31.678553+0000","last_clean":"2023-12-01T03:11:31.678553+0000","last_became_active":"2023-12-01T03:11:31.677483+0000","last_became_peered":"2023-12-01T03:11:31.677483+0000","last_unstale":"2023-12-01T03:11:31.678553+0000","last_undegraded":"2023-12-01T03:11:31.678553+0000","last_fullsized":"2023-12-01T03:11:31.678553+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:31.837132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662025+0000","last_change":"2023-12-01T03:11:31.662025+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662025+0000","last_undegraded":"2023-12-01T03:11:31.662025+0000","last_fullsized":"2023-12-01T03:11:31.662025+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:29:48.204277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.675090+0000","last_change":"2023-12-01T03:11:31.670701+0000","last_active":"2023-12-01T03:11:31.675090+0000","last_peered":"2023-12-01T03:11:31.675090+0000","last_clean":"2023-12-01T03:11:31.675090+0000","last_became_active":"2023-12-01T03:11:31.670179+0000","last_became_peered":"2023-12-01T03:11:31.670179+0000","last_unstale":"2023-12-01T03:11:31.675090+0000","last_undegraded":"2023-12-01T03:11:31.675090+0000","last_fullsized":"2023-12-01T03:11:31.675090+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:01.094744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.665097+0000","last_change":"2023-12-01T03:11:31.665097+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.665097+0000","last_undegraded":"2023-12-01T03:11:31.665097+0000","last_fullsized":"2023-12-01T03:11:31.665097+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:31.875817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.680353+0000","last_change":"2023-12-01T03:11:31.679287+0000","last_active":"2023-12-01T03:11:31.680353+0000","last_peered":"2023-12-01T03:11:31.680353+0000","last_clean":"2023-12-01T03:11:31.680353+0000","last_became_active":"2023-12-01T03:11:31.678925+0000","last_became_peered":"2023-12-01T03:11:31.678925+0000","last_unstale":"2023-12-01T03:11:31.680353+0000","last_undegraded":"2023-12-01T03:11:31.680353+0000","last_fullsized":"2023-12-01T03:11:31.680353+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:28:46.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.653733+0000","last_change":"2023-12-01T03:11:31.653733+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.653733+0000","last_undegraded":"2023-12-01T03:11:31.653733+0000","last_fullsized":"2023-12-01T03:11:31.653733+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:18:18.055289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.676041+0000","last_change":"2023-12-01T03:11:31.674632+0000","last_active":"2023-12-01T03:11:31.676041+0000","last_peered":"2023-12-01T03:11:31.676041+0000","last_clean":"2023-12-01T03:11:31.676041+0000","last_became_active":"2023-12-01T03:11:31.674319+0000","last_became_peered":"2023-12-01T03:11:31.674319+0000","last_unstale":"2023-12-01T03:11:31.676041+0000","last_undegraded":"2023-12-01T03:11:31.676041+0000","last_fullsized":"2023-12-01T03:11:31.676041+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:21.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.662041+0000","last_change":"2023-12-01T03:11:31.662041+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.662041+0000","last_undegraded":"2023-12-01T03:11:31.662041+0000","last_fullsized":"2023-12-01T03:11:31.662041+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:27:53.473015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.678086+0000","last_change":"2023-12-01T03:11:31.676885+0000","last_active":"2023-12-01T03:11:31.678086+0000","last_peered":"2023-12-01T03:11:31.678086+0000","last_clean":"2023-12-01T03:11:31.678086+0000","last_became_active":"2023-12-01T03:11:31.675545+0000","last_became_peered":"2023-12-01T03:11:31.675545+0000","last_unstale":"2023-12-01T03:11:31.678086+0000","last_undegraded":"2023-12-01T03:11:31.678086+0000","last_fullsized":"2023-12-01T03:11:31.678086+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:03.962061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.656877+0000","last_change":"2023-12-01T03:11:31.656877+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.656877+0000","last_undegraded":"2023-12-01T03:11:31.656877+0000","last_fullsized":"2023-12-01T03:11:31.656877+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:15:30.930202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.676977+0000","last_change":"2023-12-01T03:11:31.675947+0000","last_active":"2023-12-01T03:11:31.676977+0000","last_peered":"2023-12-01T03:11:31.676977+0000","last_clean":"2023-12-01T03:11:31.676977+0000","last_became_active":"2023-12-01T03:11:31.675372+0000","last_became_peered":"2023-12-01T03:11:31.675372+0000","last_unstale":"2023-12-01T03:11:31.676977+0000","last_undegraded":"2023-12-01T03:11:31.676977+0000","last_fullsized":"2023-12-01T03:11:31.676977+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:45.516411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.646063+0000","last_change":"2023-12-01T03:11:31.646063+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.646063+0000","last_undegraded":"2023-12-01T03:11:31.646063+0000","last_fullsized":"2023-12-01T03:11:31.646063+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:20.713703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.678244+0000","last_change":"2023-12-01T03:11:31.673899+0000","last_active":"2023-12-01T03:11:31.678244+0000","last_peered":"2023-12-01T03:11:31.678244+0000","last_clean":"2023-12-01T03:11:31.678244+0000","last_became_active":"2023-12-01T03:11:31.673474+0000","last_became_peered":"2023-12-01T03:11:31.673474+0000","last_unstale":"2023-12-01T03:11:31.678244+0000","last_undegraded":"2023-12-01T03:11:31.678244+0000","last_fullsized":"2023-12-01T03:11:31.678244+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:50.332038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.654447+0000","last_change":"2023-12-01T03:11:31.654447+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.654447+0000","last_undegraded":"2023-12-01T03:11:31.654447+0000","last_fullsized":"2023-12-01T03:11:31.654447+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.683474+0000","last_change":"2023-12-01T03:11:31.682949+0000","last_active":"2023-12-01T03:11:31.683474+0000","last_peered":"2023-12-01T03:11:31.683474+0000","last_clean":"2023-12-01T03:11:31.683474+0000","last_became_active":"2023-12-01T03:11:31.682454+0000","last_became_peered":"2023-12-01T03:11:31.682454+0000","last_unstale":"2023-12-01T03:11:31.683474+0000","last_undegraded":"2023-12-01T03:11:31.683474+0000","last_fullsized":"2023-12-01T03:11:31.683474+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:49.777291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.663231+0000","last_change":"2023-12-01T03:11:31.663231+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.663231+0000","last_undegraded":"2023-12-01T03:11:31.663231+0000","last_fullsized":"2023-12-01T03:11:31.663231+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:33:45.498838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.673399+0000","last_change":"2023-12-01T03:11:31.668375+0000","last_active":"2023-12-01T03:11:31.673399+0000","last_peered":"2023-12-01T03:11:31.673399+0000","last_clean":"2023-12-01T03:11:31.673399+0000","last_became_active":"2023-12-01T03:11:31.668109+0000","last_became_peered":"2023-12-01T03:11:31.668109+0000","last_unstale":"2023-12-01T03:11:31.673399+0000","last_undegraded":"2023-12-01T03:11:31.673399+0000","last_fullsized":"2023-12-01T03:11:31.673399+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:34.959657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.663085+0000","last_change":"2023-12-01T03:11:31.663085+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.663085+0000","last_undegraded":"2023-12-01T03:11:31.663085+0000","last_fullsized":"2023-12-01T03:11:31.663085+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:53.273884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.680434+0000","last_change":"2023-12-01T03:11:31.679248+0000","last_active":"2023-12-01T03:11:31.680434+0000","last_peered":"2023-12-01T03:11:31.680434+0000","last_clean":"2023-12-01T03:11:31.680434+0000","last_became_active":"2023-12-01T03:11:31.678973+0000","last_became_peered":"2023-12-01T03:11:31.678973+0000","last_unstale":"2023-12-01T03:11:31.680434+0000","last_undegraded":"2023-12-01T03:11:31.680434+0000","last_fullsized":"2023-12-01T03:11:31.680434+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.658835+0000","last_change":"2023-12-01T03:11:31.658835+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.658835+0000","last_undegraded":"2023-12-01T03:11:31.658835+0000","last_fullsized":"2023-12-01T03:11:31.658835+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.669200+0000","last_change":"2023-12-01T03:11:31.663771+0000","last_active":"2023-12-01T03:11:31.669200+0000","last_peered":"2023-12-01T03:11:31.669200+0000","last_clean":"2023-12-01T03:11:31.669200+0000","last_became_active":"2023-12-01T03:11:31.662877+0000","last_became_peered":"2023-12-01T03:11:31.662877+0000","last_unstale":"2023-12-01T03:11:31.669200+0000","last_undegraded":"2023-12-01T03:11:31.669200+0000","last_fullsized":"2023-12-01T03:11:31.669200+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:59.396265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.648564+0000","last_change":"2023-12-01T03:11:31.648564+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.648564+0000","last_undegraded":"2023-12-01T03:11:31.648564+0000","last_fullsized":"2023-12-01T03:11:31.648564+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:17.846927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.677011+0000","last_change":"2023-12-01T03:11:31.673043+0000","last_active":"2023-12-01T03:11:31.677011+0000","last_peered":"2023-12-01T03:11:31.677011+0000","last_clean":"2023-12-01T03:11:31.677011+0000","last_became_active":"2023-12-01T03:11:31.672780+0000","last_became_peered":"2023-12-01T03:11:31.672780+0000","last_unstale":"2023-12-01T03:11:31.677011+0000","last_undegraded":"2023-12-01T03:11:31.677011+0000","last_fullsized":"2023-12-01T03:11:31.677011+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:40.478556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.664707+0000","last_change":"2023-12-01T03:11:31.664707+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.664707+0000","last_undegraded":"2023-12-01T03:11:31.664707+0000","last_fullsized":"2023-12-01T03:11:31.664707+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.678278+0000","last_change":"2023-12-01T03:11:31.677162+0000","last_active":"2023-12-01T03:11:31.678278+0000","last_peered":"2023-12-01T03:11:31.678278+0000","last_clean":"2023-12-01T03:11:31.678278+0000","last_became_active":"2023-12-01T03:11:31.674178+0000","last_became_peered":"2023-12-01T03:11:31.674178+0000","last_unstale":"2023-12-01T03:11:31.678278+0000","last_undegraded":"2023-12-01T03:11:31.678278+0000","last_fullsized":"2023-12-01T03:11:31.678278+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:13.700505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.660932+0000","last_change":"2023-12-01T03:11:31.660932+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.660932+0000","last_undegraded":"2023-12-01T03:11:31.660932+0000","last_fullsized":"2023-12-01T03:11:31.660932+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.673944+0000","last_change":"2023-12-01T03:11:31.672908+0000","last_active":"2023-12-01T03:11:31.673944+0000","last_peered":"2023-12-01T03:11:31.673944+0000","last_clean":"2023-12-01T03:11:31.673944+0000","last_became_active":"2023-12-01T03:11:31.671738+0000","last_became_peered":"2023-12-01T03:11:31.671738+0000","last_unstale":"2023-12-01T03:11:31.673944+0000","last_undegraded":"2023-12-01T03:11:31.673944+0000","last_fullsized":"2023-12-01T03:11:31.673944+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.660812+0000","last_change":"2023-12-01T03:11:31.660812+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.660812+0000","last_undegraded":"2023-12-01T03:11:31.660812+0000","last_fullsized":"2023-12-01T03:11:31.660812+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:47:03.446609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.677149+0000","last_change":"2023-12-01T03:11:31.676480+0000","last_active":"2023-12-01T03:11:31.677149+0000","last_peered":"2023-12-01T03:11:31.677149+0000","last_clean":"2023-12-01T03:11:31.677149+0000","last_became_active":"2023-12-01T03:11:31.676127+0000","last_became_peered":"2023-12-01T03:11:31.676127+0000","last_unstale":"2023-12-01T03:11:31.677149+0000","last_undegraded":"2023-12-01T03:11:31.677149+0000","last_fullsized":"2023-12-01T03:11:31.677149+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.654692+0000","last_change":"2023-12-01T03:11:31.654692+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.654692+0000","last_undegraded":"2023-12-01T03:11:31.654692+0000","last_fullsized":"2023-12-01T03:11:31.654692+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.681381+0000","last_change":"2023-12-01T03:11:31.680251+0000","last_active":"2023-12-01T03:11:31.681381+0000","last_peered":"2023-12-01T03:11:31.681381+0000","last_clean":"2023-12-01T03:11:31.681381+0000","last_became_active":"2023-12-01T03:11:31.679936+0000","last_became_peered":"2023-12-01T03:11:31.679936+0000","last_unstale":"2023-12-01T03:11:31.681381+0000","last_undegraded":"2023-12-01T03:11:31.681381+0000","last_fullsized":"2023-12-01T03:11:31.681381+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:39.296836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":1,"reported_epoch":233,"state":"creating+peering","last_fresh":"2023-12-01T03:11:31.653762+0000","last_change":"2023-12-01T03:11:31.653762+0000","last_active":"2023-12-01T03:11:31.630683+0000","last_peered":"2023-12-01T03:11:31.630683+0000","last_clean":"2023-12-01T03:11:31.630683+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:11:31.653762+0000","last_undegraded":"2023-12-01T03:11:31.653762+0000","last_fullsized":"2023-12-01T03:11:31.653762+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-01T03:11:31.682554+0000","last_change":"2023-12-01T03:11:31.681391+0000","last_active":"2023-12-01T03:11:31.682554+0000","last_peered":"2023-12-01T03:11:31.682554+0000","last_clean":"2023-12-01T03:11:31.682554+0000","last_became_active":"2023-12-01T03:11:31.680773+0000","last_became_peered":"2023-12-01T03:11:31.680773+0000","last_unstale":"2023-12-01T03:11:31.682554+0000","last_undegraded":"2023-12-01T03:11:31.682554+0000","last_fullsized":"2023-12-01T03:11:31.682554+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:40:13.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"220'348","reported_seq":1212,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-01T03:11:30.647156+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:11:30.647156+0000","last_peered":"2023-12-01T03:11:30.647156+0000","last_clean":"2023-12-01T03:11:30.647156+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:11:30.647156+0000","last_undegraded":"2023-12-01T03:11:30.647156+0000","last_fullsized":"2023-12-01T03:11:30.647156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575988,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60136,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311704,"statfs":{"total":96636764160,"available":96575184896,"internally_reserved":0,"allocated":2916352,"data_stored":1427986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.05,"15min":1.071},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5409999999999999,"5min":1.6000000000000001,"15min":1.7390000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6599999999999999,"5min":1.6699999999999999,"15min":1.7},"last":0.873}]},{"osd":1,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.236,"15min":1.2769999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4990000000000001,"5min":1.78,"15min":2.1419999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.294,"15min":1.3260000000000001},"min":{"1min":0.873,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5740000000000001,"5min":1.6240000000000001,"15min":1.8700000000000001},"last":0.99399999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1439999999999999,"15min":1.173},"min":{"1min":0.71699999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.573,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.2549999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.643,"5min":1.7050000000000001,"15min":2.1120000000000001},"last":1.4119999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2509999999999999,"15min":1.302},"min":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.712,"15min":2.431},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3720000000000001,"15min":1.4119999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7529999999999999,"5min":1.879,"15min":2.198},"last":1.4610000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3009999999999999,"15min":1.335},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7689999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.4510000000000001,"15min":1.49},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8799999999999999,"5min":1.97,"15min":2.569},"last":1.514}]},{"osd":5,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2869999999999999,"15min":1.3160000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7150000000000001,"5min":2.0099999999999998,"15min":2.7269999999999999},"last":1.159},{"interface":"front","average":{"1min":1.498,"5min":1.5469999999999999,"15min":1.585},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.0470000000000002,"5min":2.0569999999999999,"15min":2.484},"last":1.5589999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.403,"15min":1.4359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7749999999999999,"5min":1.921,"15min":2.532},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4299999999999999,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8260000000000001,"5min":1.8819999999999999,"15min":2.762},"last":1.296}]}]},{"osd":6,"up_from":13,"seq":55834575989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42088,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329752,"statfs":{"total":96636764160,"available":96593666048,"internally_reserved":0,"allocated":2916352,"data_stored":1427986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0509999999999999,"15min":1.1080000000000001},"min":{"1min":0.66100000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.1040000000000001,"15min":1.157},"min":{"1min":0.78400000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6910000000000001,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.333}]},{"osd":1,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.141,"15min":1.2150000000000001},"min":{"1min":0.68999999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8460000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1830000000000001,"15min":1.22},"min":{"1min":0.86099999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.6220000000000001,"5min":1.694,"15min":1.835},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.157,"15min":1.206},"min":{"1min":0.71799999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.464,"5min":1.742,"15min":1.976},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.222,"15min":1.278},"min":{"1min":0.71699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5009999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.2}]},{"osd":3,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2829999999999999,"15min":1.3140000000000001},"min":{"1min":0.89500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6839999999999999,"5min":1.927,"15min":2.0289999999999999},"last":1.105},{"interface":"front","average":{"1min":1.454,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":1.0069999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0190000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.3979999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":0.93600000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.589,"5min":1.407,"15min":1.367},"min":{"1min":1.2090000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.222,"5min":2.48,"15min":2.48},"last":1.157}]},{"osd":5,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.3700000000000001,"15min":1.413},"min":{"1min":1.127,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.962,"5min":2.262,"15min":2.262},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5169999999999999,"15min":1.4970000000000001},"min":{"1min":1.2629999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.194,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.2490000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5029999999999999,"15min":1.5309999999999999},"min":{"1min":1.155,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1320000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.746,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":1.371,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2770000000000001,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.48}]}]},{"osd":5,"up_from":13,"seq":55834575988,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52536,"kb_used_data":3440,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319304,"statfs":{"total":96636764160,"available":96582967296,"internally_reserved":0,"allocated":3522560,"data_stored":2024673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1000000000000001,"15min":1.123},"min":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.7170000000000001,"5min":1.776,"15min":1.776},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1539999999999999,"15min":1.161},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.752},"last":0.83499999999999996}]},{"osd":1,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.1559999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6120000000000001,"5min":1.6519999999999999,"15min":1.696},"last":0.747},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.198,"15min":1.2330000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":0.98699999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2270000000000001,"15min":1.2470000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.81399999999999995},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.2450000000000001,"15min":1.254},"min":{"1min":0.75,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.867,"5min":1.899,"15min":1.9219999999999999},"last":1.1339999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3089999999999999,"15min":1.3149999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.889,"5min":1.889,"15min":1.8979999999999999},"last":1.29},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.329,"15min":1.359},"min":{"1min":1.0109999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.97,"5min":1.97,"15min":2.1859999999999999},"last":1.256}]},{"osd":4,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.869,"5min":1.869,"15min":2.6040000000000001},"last":1.196},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.407}]},{"osd":6,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3779999999999999,"15min":1.377},"min":{"1min":0.99099999999999999,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.7649999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.367},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.409,"15min":1.4219999999999999},"min":{"1min":1.0149999999999999,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":2.032,"5min":2.032,"15min":2.577},"last":1.5329999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.534,"15min":1.538},"min":{"1min":1.2010000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":2.077,"5min":2.077,"15min":3.1709999999999998},"last":1.591},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5760000000000001,"15min":1.5780000000000001},"min":{"1min":1.3200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.101,"15min":2.8740000000000001},"last":1.472}]}]},{"osd":4,"up_from":13,"seq":55834575988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51956,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319884,"statfs":{"total":96636764160,"available":96583561216,"internally_reserved":0,"allocated":2928640,"data_stored":1434305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1200000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8089999999999999,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.212},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.198,"15min":1.1899999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5309999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":0.89700000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2290000000000001,"15min":1.256},"min":{"1min":0.83599999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8580000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1719999999999999,"15min":1.175},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.778,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":0.81200000000000006}]},{"osd":2,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3300000000000001,"15min":1.244},"min":{"1min":0.74399999999999999,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.637,"5min":11.311,"15min":11.311},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3320000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9330000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.056}]},{"osd":3,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.298,"15min":1.29},"min":{"1min":0.80000000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.716,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.095},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3759999999999999,"15min":1.3340000000000001},"min":{"1min":0.96399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7430000000000001,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.4399999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.357,"15min":1.3200000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.738,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.375,"15min":1.3160000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6579999999999999,"5min":3.71,"15min":3.71},"last":1.302}]},{"osd":6,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3799999999999999,"15min":1.3740000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8009999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.361},{"interface":"front","average":{"1min":1.419,"5min":1.4790000000000001,"15min":1.476},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.76,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.494}]},{"osd":7,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.514,"15min":1.488},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9830000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.655},{"interface":"front","average":{"1min":1.532,"5min":1.597,"15min":1.575},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.907,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.706}]}]},{"osd":3,"up_from":13,"seq":55834575987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56040,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315800,"statfs":{"total":96636764160,"available":96579379200,"internally_reserved":0,"allocated":2916352,"data_stored":1427986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.91300000000000003,"15min":0.90700000000000003},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.503},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1399999999999999,"15min":1.167},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.788,"5min":3.863,"15min":3.863},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.073,"15min":1.0960000000000001},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.637,"5min":3.597,"15min":3.597},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.127,"5min":1.2090000000000001,"15min":1.2410000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.8319999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2969999999999999,"15min":1.294},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.272,"15min":1.296},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.736,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0049999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.331,"15min":1.329},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.1240000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.373,"15min":1.373},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.425,"15min":1.444},"min":{"1min":0.80500000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.252},{"interface":"front","average":{"1min":1.504,"5min":1.5089999999999999,"15min":1.5129999999999999},"min":{"1min":1.0089999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.92200000000000004}]},{"osd":7,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5129999999999999,"15min":1.5249999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.899,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.242},{"interface":"front","average":{"1min":1.45,"5min":1.5209999999999999,"15min":1.526},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":1.3129999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575987,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47848,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323992,"statfs":{"total":96636764160,"available":96587767808,"internally_reserved":0,"allocated":2916352,"data_stored":1427986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.411,"5min":1.411,"15min":1.4730000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.052,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3080000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.0229999999999999,"15min":1.0489999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.343,"5min":1.698,"15min":1.698},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.097,"15min":1.135},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.4730000000000001,"5min":1.635,"15min":1.6599999999999999},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.026,"15min":1.042},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.393,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.497},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.159,"15min":1.208},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4950000000000001,"5min":1.6879999999999999,"15min":1.79},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.319,"15min":1.327},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5700000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.752},{"interface":"front","average":{"1min":1.194,"5min":1.2689999999999999,"15min":1.294},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5189999999999999,"5min":1.714,"15min":1.835},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3560000000000001,"15min":1.3839999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.782,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.081},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.407,"15min":1.411},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.873,"5min":1.8740000000000001,"15min":2.016},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3660000000000001,"15min":1.3799999999999999},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.714,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4730000000000001,"15min":1.4970000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.1870000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.478,"15min":1.4850000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.9350000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.389,"5min":1.4259999999999999,"15min":1.454},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8149999999999999,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.145}]}]},{"osd":1,"up_from":13,"seq":55834575986,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48428,"kb_used_data":3428,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323412,"statfs":{"total":96636764160,"available":96587173888,"internally_reserved":0,"allocated":3510272,"data_stored":2018354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.89800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.238,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.98499999999999999,"15min":1.038},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.512,"5min":1.5660000000000001,"15min":1.681},"last":1.167}]},{"osd":2,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2190000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.048,"5min":1.1020000000000001,"15min":1.141},"min":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5489999999999999,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.208,"15min":1.2949999999999999},"min":{"1min":0.60799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5860000000000001,"5min":1.587,"15min":1.788},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.0780000000000001,"15min":1.127},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.383,"5min":1.714,"15min":1.8049999999999999},"last":1.139}]},{"osd":4,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2030000000000001,"15min":1.218},"min":{"1min":0.84399999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.623,"5min":1.623,"15min":1.6859999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3700000000000001,"15min":1.4039999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.657,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.268}]},{"osd":5,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3109999999999999,"15min":1.333},"min":{"1min":0.95899999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":2.101},"last":1.722},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.298,"15min":1.331},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.9019999999999999,"5min":1.927,"15min":1.927},"last":1.7470000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0529999999999999,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3939999999999999,"15min":1.413},"min":{"1min":0.91600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.6719999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4650000000000001,"15min":1.522},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.9430000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":1.577},{"interface":"front","average":{"1min":1.524,"5min":1.48,"15min":1.5109999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.0699999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.863}]}]},{"osd":0,"up_from":13,"seq":55834575988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60136,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311704,"statfs":{"total":96636764160,"available":96575184896,"internally_reserved":0,"allocated":2916352,"data_stored":1427986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.88900000000000001},"min":{"1min":0.314,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.994},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.98899999999999999,"15min":1.048},"min":{"1min":0.58999999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6799999999999999},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0169999999999999,"15min":1.0640000000000001},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.635,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0589999999999999,"15min":1.1259999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3100000000000001,"5min":1.726,"15min":1.792},"last":0.96099999999999997}]},{"osd":3,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.036,"15min":1.0669999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.8839999999999999},"last":0.999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.171,"15min":1.224},"min":{"1min":0.70499999999999996,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.6469999999999998},"last":0.54400000000000004}]},{"osd":4,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.296,"15min":1.3520000000000001},"min":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.681,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":1.05},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.3029999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1279999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.90700000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.899,"15min":2.7719999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.383,"15min":1.427},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.681,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":1.3420000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3520000000000001,"15min":1.381},"min":{"1min":0.84599999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7270000000000001,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":1.22},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.45,"15min":1.4830000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.1200000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4670000000000001,"15min":1.516},"min":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7969999999999999,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":1.286},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4730000000000001,"15min":1.4870000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.927,"5min":2.1190000000000002,"15min":2.431},"last":1.3120000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:11:37.560 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:11:37.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.809+0000 7fd297fff640 1 Processor -- start 2023-12-01T03:11:37.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.809+0000 7fd297fff640 1 -- start start 2023-12-01T03:11:37.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.811+0000 7fd297fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 0x7fd2980cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.811+0000 7fd297fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd29806b270 0x7fd29815db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.811+0000 7fd297fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd29815e070 0x7fd2980adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.811+0000 7fd297fff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29806b9c0 con 0x7fd29806b270 2023-12-01T03:11:37.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.811+0000 7fd297fff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29806bca0 con 0x7fd29815e070 2023-12-01T03:11:37.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.811+0000 7fd297fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29806bf80 con 0x7fd2981597d0 2023-12-01T03:11:37.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.811+0000 7fd28ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd29806b270 0x7fd29815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.811+0000 7fd28ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd29806b270 0x7fd29815db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37522/0 (socket says 172.21.15.115:37522) 2023-12-01T03:11:37.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.811+0000 7fd28ffff640 1 -- 172.21.15.115:0/1574586871 learned_addr learned my addr 172.21.15.115:0/1574586871 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.812+0000 7fd296ffd640 1 --2- 172.21.15.115:0/1574586871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 0x7fd2980cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.812+0000 7fd2977fe640 1 --2- 172.21.15.115:0/1574586871 >> v2:172.21.15.142:3300/0 conn(0x7fd29815e070 0x7fd2980adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.812+0000 7fd28ffff640 1 -- 172.21.15.115:0/1574586871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 msgr2=0x7fd2980cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.812+0000 7fd28ffff640 1 --2- 172.21.15.115:0/1574586871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 0x7fd2980cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.812+0000 7fd28ffff640 1 -- 172.21.15.115:0/1574586871 >> v2:172.21.15.142:3300/0 conn(0x7fd29815e070 msgr2=0x7fd2980adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.812+0000 7fd28ffff640 1 --2- 172.21.15.115:0/1574586871 >> v2:172.21.15.142:3300/0 conn(0x7fd29815e070 0x7fd2980adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.812+0000 7fd28ffff640 1 -- 172.21.15.115:0/1574586871 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29815d610 con 0x7fd29806b270 2023-12-01T03:11:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.812+0000 7fd296ffd640 1 --2- 172.21.15.115:0/1574586871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 0x7fd2980cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.812+0000 7fd2977fe640 1 --2- 172.21.15.115:0/1574586871 >> v2:172.21.15.142:3300/0 conn(0x7fd29815e070 0x7fd2980adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:37.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.813+0000 7fd28ffff640 1 --2- 172.21.15.115:0/1574586871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29806b270 0x7fd29815db30 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7fd2980adad0 tx=0x7fd28002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3ad57cb0441d79 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:37.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.813+0000 7fd2967fc640 1 -- 172.21.15.115:0/1574586871 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800317d0 con 0x7fd29806b270 2023-12-01T03:11:37.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.813+0000 7fd2967fc640 1 -- 172.21.15.115:0/1574586871 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280031930 con 0x7fd29806b270 2023-12-01T03:11:37.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.813+0000 7fd2967fc640 1 -- 172.21.15.115:0/1574586871 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280031cc0 con 0x7fd29806b270 2023-12-01T03:11:37.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.814+0000 7fd297fff640 1 -- 172.21.15.115:0/1574586871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29806b270 msgr2=0x7fd29815db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.814+0000 7fd297fff640 1 --2- 172.21.15.115:0/1574586871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29806b270 0x7fd29815db30 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7fd2980adad0 tx=0x7fd28002b870 comp rx=0 tx=0).stop 2023-12-01T03:11:37.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.814+0000 7fd297fff640 1 -- 172.21.15.115:0/1574586871 shutdown_connections 2023-12-01T03:11:37.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.814+0000 7fd297fff640 1 --2- 172.21.15.115:0/1574586871 >> v2:172.21.15.142:3300/0 conn(0x7fd29815e070 0x7fd2980adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.814+0000 7fd297fff640 1 --2- 172.21.15.115:0/1574586871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29806b270 0x7fd29815db30 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.814+0000 7fd297fff640 1 --2- 172.21.15.115:0/1574586871 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 0x7fd2980cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.814+0000 7fd297fff640 1 -- 172.21.15.115:0/1574586871 >> 172.21.15.115:0/1574586871 conn(0x7fd2980ce640 msgr2=0x7fd2980cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:37.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.814+0000 7fd297fff640 1 -- 172.21.15.115:0/1574586871 shutdown_connections 2023-12-01T03:11:37.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.814+0000 7fd297fff640 1 -- 172.21.15.115:0/1574586871 wait complete. 2023-12-01T03:11:37.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.816+0000 7fd297fff640 1 Processor -- start 2023-12-01T03:11:37.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.816+0000 7fd297fff640 1 -- start start 2023-12-01T03:11:37.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd297fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29806b270 0x7fd2980711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd297fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 0x7fd29815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd297fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29815e070 0x7fd29814bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd297fff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29806f8e0 con 0x7fd29815e070 2023-12-01T03:11:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd297fff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29806fac0 con 0x7fd29806b270 2023-12-01T03:11:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd297fff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29806fd60 con 0x7fd2981597d0 2023-12-01T03:11:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd2977fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29815e070 0x7fd29814bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd2977fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29815e070 0x7fd29814bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37528/0 (socket says 172.21.15.115:37528) 2023-12-01T03:11:37.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd2977fe640 1 -- 172.21.15.115:0/2642304466 learned_addr learned my addr 172.21.15.115:0/2642304466 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd28ffff640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 0x7fd29815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd296ffd640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29806b270 0x7fd2980711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd2977fe640 1 -- 172.21.15.115:0/2642304466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 msgr2=0x7fd29815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd2977fe640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 0x7fd29815c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd2977fe640 1 -- 172.21.15.115:0/2642304466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29806b270 msgr2=0x7fd2980711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:37.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd2977fe640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29806b270 0x7fd2980711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.817+0000 7fd2977fe640 1 -- 172.21.15.115:0/2642304466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2800094d0 con 0x7fd29815e070 2023-12-01T03:11:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.818+0000 7fd2977fe640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29815e070 0x7fd29814bc80 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7fd29000cef0 tx=0x7fd29000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.818+0000 7fd294ff9640 1 -- 172.21.15.115:0/2642304466 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29000d840 con 0x7fd29815e070 2023-12-01T03:11:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.818+0000 7fd294ff9640 1 -- 172.21.15.115:0/2642304466 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29000d9c0 con 0x7fd29815e070 2023-12-01T03:11:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.818+0000 7fd297fff640 1 -- 172.21.15.115:0/2642304466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd298070040 con 0x7fd29815e070 2023-12-01T03:11:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.818+0000 7fd294ff9640 1 -- 172.21.15.115:0/2642304466 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29000dd70 con 0x7fd29815e070 2023-12-01T03:11:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.818+0000 7fd297fff640 1 -- 172.21.15.115:0/2642304466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd298070600 con 0x7fd29815e070 2023-12-01T03:11:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.819+0000 7fd297fff640 1 -- 172.21.15.115:0/2642304466 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2980707f0 con 0x7fd29815e070 2023-12-01T03:11:37.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.820+0000 7fd294ff9640 1 -- 172.21.15.115:0/2642304466 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fd29001e3b0 con 0x7fd29815e070 2023-12-01T03:11:37.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.821+0000 7fd294ff9640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27806a670 0x7fd27806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:37.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.822+0000 7fd294ff9640 1 -- 172.21.15.115:0/2642304466 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fd2900898e0 con 0x7fd29815e070 2023-12-01T03:11:37.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.822+0000 7fd296ffd640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27806a670 0x7fd27806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:37.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.822+0000 7fd296ffd640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27806a670 0x7fd27806cb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd28800b7c0 tx=0x7fd288008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:37.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:37.826+0000 7fd294ff9640 1 -- 172.21.15.115:0/2642304466 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd290054b60 con 0x7fd29815e070 2023-12-01T03:11:38.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.011+0000 7fd297fff640 1 -- 172.21.15.115:0/2642304466 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd29806b6d0 con 0x7fd27806a670 2023-12-01T03:11:38.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.020+0000 7fd294ff9640 1 -- 172.21.15.115:0/2642304466 <== mgr.11121 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287573 (secure 0 0 0) 0x7fd28800c040 con 0x7fd27806a670 2023-12-01T03:11:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:38.027 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:11:38.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.025+0000 7fd297fff640 1 -- 172.21.15.115:0/2642304466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27806a670 msgr2=0x7fd27806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.025+0000 7fd297fff640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27806a670 0x7fd27806cb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd28800b7c0 tx=0x7fd288008040 comp rx=0 tx=0).stop 2023-12-01T03:11:38.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.025+0000 7fd297fff640 1 -- 172.21.15.115:0/2642304466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29815e070 msgr2=0x7fd29814bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.025+0000 7fd297fff640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29815e070 0x7fd29814bc80 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7fd29000cef0 tx=0x7fd29000a710 comp rx=0 tx=0).stop 2023-12-01T03:11:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.026+0000 7fd297fff640 1 -- 172.21.15.115:0/2642304466 shutdown_connections 2023-12-01T03:11:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.026+0000 7fd297fff640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd27806a670 0x7fd27806cb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.026+0000 7fd297fff640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2981597d0 0x7fd29815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.026+0000 7fd297fff640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd29815e070 0x7fd29814bc80 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.026+0000 7fd297fff640 1 --2- 172.21.15.115:0/2642304466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29806b270 0x7fd2980711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.026+0000 7fd297fff640 1 -- 172.21.15.115:0/2642304466 >> 172.21.15.115:0/2642304466 conn(0x7fd2980ce640 msgr2=0x7fd2980ad940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.026+0000 7fd297fff640 1 -- 172.21.15.115:0/2642304466 shutdown_connections 2023-12-01T03:11:38.029 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.026+0000 7fd297fff640 1 -- 172.21.15.115:0/2642304466 wait complete. 2023-12-01T03:11:38.044 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-12-01T03:11:37.822769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":623,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":419496,"kb_used_data":24296,"kb_used_omap":0,"kb_used_meta":395136,"kb_avail":754555224,"statfs":{"total":773094113280,"available":772664549376,"internally_reserved":0,"allocated":24879104,"data_stored":12765204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":404619264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.183091"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222918+0000","last_change":"2023-12-01T03:11:32.212603+0000","last_active":"2023-12-01T03:11:32.222918+0000","last_peered":"2023-12-01T03:11:32.222918+0000","last_clean":"2023-12-01T03:11:32.222918+0000","last_became_active":"2023-12-01T03:11:32.212287+0000","last_became_peered":"2023-12-01T03:11:32.212287+0000","last_unstale":"2023-12-01T03:11:32.222918+0000","last_undegraded":"2023-12-01T03:11:32.222918+0000","last_fullsized":"2023-12-01T03:11:32.222918+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220449+0000","last_change":"2023-12-01T03:11:32.217098+0000","last_active":"2023-12-01T03:11:32.220449+0000","last_peered":"2023-12-01T03:11:32.220449+0000","last_clean":"2023-12-01T03:11:32.220449+0000","last_became_active":"2023-12-01T03:11:32.216867+0000","last_became_peered":"2023-12-01T03:11:32.216867+0000","last_unstale":"2023-12-01T03:11:32.220449+0000","last_undegraded":"2023-12-01T03:11:32.220449+0000","last_fullsized":"2023-12-01T03:11:32.220449+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:10.849908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210187+0000","last_change":"2023-12-01T03:11:32.208423+0000","last_active":"2023-12-01T03:11:32.210187+0000","last_peered":"2023-12-01T03:11:32.210187+0000","last_clean":"2023-12-01T03:11:32.210187+0000","last_became_active":"2023-12-01T03:11:32.208113+0000","last_became_peered":"2023-12-01T03:11:32.208113+0000","last_unstale":"2023-12-01T03:11:32.210187+0000","last_undegraded":"2023-12-01T03:11:32.210187+0000","last_fullsized":"2023-12-01T03:11:32.210187+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:38.174827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213451+0000","last_change":"2023-12-01T03:11:32.207840+0000","last_active":"2023-12-01T03:11:32.213451+0000","last_peered":"2023-12-01T03:11:32.213451+0000","last_clean":"2023-12-01T03:11:32.213451+0000","last_became_active":"2023-12-01T03:11:32.207048+0000","last_became_peered":"2023-12-01T03:11:32.207048+0000","last_unstale":"2023-12-01T03:11:32.213451+0000","last_undegraded":"2023-12-01T03:11:32.213451+0000","last_fullsized":"2023-12-01T03:11:32.213451+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:43.961392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212780+0000","last_change":"2023-12-01T03:11:32.206960+0000","last_active":"2023-12-01T03:11:32.212780+0000","last_peered":"2023-12-01T03:11:32.212780+0000","last_clean":"2023-12-01T03:11:32.212780+0000","last_became_active":"2023-12-01T03:11:32.203345+0000","last_became_peered":"2023-12-01T03:11:32.203345+0000","last_unstale":"2023-12-01T03:11:32.212780+0000","last_undegraded":"2023-12-01T03:11:32.212780+0000","last_fullsized":"2023-12-01T03:11:32.212780+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211014+0000","last_change":"2023-12-01T03:11:32.205496+0000","last_active":"2023-12-01T03:11:32.211014+0000","last_peered":"2023-12-01T03:11:32.211014+0000","last_clean":"2023-12-01T03:11:32.211014+0000","last_became_active":"2023-12-01T03:11:32.205074+0000","last_became_peered":"2023-12-01T03:11:32.205074+0000","last_unstale":"2023-12-01T03:11:32.211014+0000","last_undegraded":"2023-12-01T03:11:32.211014+0000","last_fullsized":"2023-12-01T03:11:32.211014+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209538+0000","last_change":"2023-12-01T03:11:32.202465+0000","last_active":"2023-12-01T03:11:32.209538+0000","last_peered":"2023-12-01T03:11:32.209538+0000","last_clean":"2023-12-01T03:11:32.209538+0000","last_became_active":"2023-12-01T03:11:32.200165+0000","last_became_peered":"2023-12-01T03:11:32.200165+0000","last_unstale":"2023-12-01T03:11:32.209538+0000","last_undegraded":"2023-12-01T03:11:32.209538+0000","last_fullsized":"2023-12-01T03:11:32.209538+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:19.277681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220081+0000","last_change":"2023-12-01T03:11:32.217253+0000","last_active":"2023-12-01T03:11:32.220081+0000","last_peered":"2023-12-01T03:11:32.220081+0000","last_clean":"2023-12-01T03:11:32.220081+0000","last_became_active":"2023-12-01T03:11:32.216425+0000","last_became_peered":"2023-12-01T03:11:32.216425+0000","last_unstale":"2023-12-01T03:11:32.220081+0000","last_undegraded":"2023-12-01T03:11:32.220081+0000","last_fullsized":"2023-12-01T03:11:32.220081+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:51.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.214233+0000","last_change":"2023-12-01T03:11:32.211881+0000","last_active":"2023-12-01T03:11:32.214233+0000","last_peered":"2023-12-01T03:11:32.214233+0000","last_clean":"2023-12-01T03:11:32.214233+0000","last_became_active":"2023-12-01T03:11:32.211602+0000","last_became_peered":"2023-12-01T03:11:32.211602+0000","last_unstale":"2023-12-01T03:11:32.214233+0000","last_undegraded":"2023-12-01T03:11:32.214233+0000","last_fullsized":"2023-12-01T03:11:32.214233+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.201292+0000","last_change":"2023-12-01T03:11:32.200099+0000","last_active":"2023-12-01T03:11:32.201292+0000","last_peered":"2023-12-01T03:11:32.201292+0000","last_clean":"2023-12-01T03:11:32.201292+0000","last_became_active":"2023-12-01T03:11:32.199703+0000","last_became_peered":"2023-12-01T03:11:32.199703+0000","last_unstale":"2023-12-01T03:11:32.201292+0000","last_undegraded":"2023-12-01T03:11:32.201292+0000","last_fullsized":"2023-12-01T03:11:32.201292+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:44:42.356924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213688+0000","last_change":"2023-12-01T03:11:32.211517+0000","last_active":"2023-12-01T03:11:32.213688+0000","last_peered":"2023-12-01T03:11:32.213688+0000","last_clean":"2023-12-01T03:11:32.213688+0000","last_became_active":"2023-12-01T03:11:32.211273+0000","last_became_peered":"2023-12-01T03:11:32.211273+0000","last_unstale":"2023-12-01T03:11:32.213688+0000","last_undegraded":"2023-12-01T03:11:32.213688+0000","last_fullsized":"2023-12-01T03:11:32.213688+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:35.627224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.206460+0000","last_change":"2023-12-01T03:11:32.202825+0000","last_active":"2023-12-01T03:11:32.206460+0000","last_peered":"2023-12-01T03:11:32.206460+0000","last_clean":"2023-12-01T03:11:32.206460+0000","last_became_active":"2023-12-01T03:11:32.202452+0000","last_became_peered":"2023-12-01T03:11:32.202452+0000","last_unstale":"2023-12-01T03:11:32.206460+0000","last_undegraded":"2023-12-01T03:11:32.206460+0000","last_fullsized":"2023-12-01T03:11:32.206460+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225985+0000","last_change":"2023-12-01T03:11:32.224888+0000","last_active":"2023-12-01T03:11:32.225985+0000","last_peered":"2023-12-01T03:11:32.225985+0000","last_clean":"2023-12-01T03:11:32.225985+0000","last_became_active":"2023-12-01T03:11:32.224585+0000","last_became_peered":"2023-12-01T03:11:32.224585+0000","last_unstale":"2023-12-01T03:11:32.225985+0000","last_undegraded":"2023-12-01T03:11:32.225985+0000","last_fullsized":"2023-12-01T03:11:32.225985+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:04.079396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210154+0000","last_change":"2023-12-01T03:11:32.208938+0000","last_active":"2023-12-01T03:11:32.210154+0000","last_peered":"2023-12-01T03:11:32.210154+0000","last_clean":"2023-12-01T03:11:32.210154+0000","last_became_active":"2023-12-01T03:11:32.208695+0000","last_became_peered":"2023-12-01T03:11:32.208695+0000","last_unstale":"2023-12-01T03:11:32.210154+0000","last_undegraded":"2023-12-01T03:11:32.210154+0000","last_fullsized":"2023-12-01T03:11:32.210154+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:21.336618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218399+0000","last_change":"2023-12-01T03:11:32.214557+0000","last_active":"2023-12-01T03:11:32.218399+0000","last_peered":"2023-12-01T03:11:32.218399+0000","last_clean":"2023-12-01T03:11:32.218399+0000","last_became_active":"2023-12-01T03:11:32.214252+0000","last_became_peered":"2023-12-01T03:11:32.214252+0000","last_unstale":"2023-12-01T03:11:32.218399+0000","last_undegraded":"2023-12-01T03:11:32.218399+0000","last_fullsized":"2023-12-01T03:11:32.218399+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218473+0000","last_change":"2023-12-01T03:11:32.212018+0000","last_active":"2023-12-01T03:11:32.218473+0000","last_peered":"2023-12-01T03:11:32.218473+0000","last_clean":"2023-12-01T03:11:32.218473+0000","last_became_active":"2023-12-01T03:11:32.211742+0000","last_became_peered":"2023-12-01T03:11:32.211742+0000","last_unstale":"2023-12-01T03:11:32.218473+0000","last_undegraded":"2023-12-01T03:11:32.218473+0000","last_fullsized":"2023-12-01T03:11:32.218473+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222963+0000","last_change":"2023-12-01T03:11:32.219580+0000","last_active":"2023-12-01T03:11:32.222963+0000","last_peered":"2023-12-01T03:11:32.222963+0000","last_clean":"2023-12-01T03:11:32.222963+0000","last_became_active":"2023-12-01T03:11:32.218935+0000","last_became_peered":"2023-12-01T03:11:32.218935+0000","last_unstale":"2023-12-01T03:11:32.222963+0000","last_undegraded":"2023-12-01T03:11:32.222963+0000","last_fullsized":"2023-12-01T03:11:32.222963+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223790+0000","last_change":"2023-12-01T03:11:32.222587+0000","last_active":"2023-12-01T03:11:32.223790+0000","last_peered":"2023-12-01T03:11:32.223790+0000","last_clean":"2023-12-01T03:11:32.223790+0000","last_became_active":"2023-12-01T03:11:32.222267+0000","last_became_peered":"2023-12-01T03:11:32.222267+0000","last_unstale":"2023-12-01T03:11:32.223790+0000","last_undegraded":"2023-12-01T03:11:32.223790+0000","last_fullsized":"2023-12-01T03:11:32.223790+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:42.720564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210046+0000","last_change":"2023-12-01T03:11:32.204026+0000","last_active":"2023-12-01T03:11:32.210046+0000","last_peered":"2023-12-01T03:11:32.210046+0000","last_clean":"2023-12-01T03:11:32.210046+0000","last_became_active":"2023-12-01T03:11:32.200832+0000","last_became_peered":"2023-12-01T03:11:32.200832+0000","last_unstale":"2023-12-01T03:11:32.210046+0000","last_undegraded":"2023-12-01T03:11:32.210046+0000","last_fullsized":"2023-12-01T03:11:32.210046+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.215183+0000","last_change":"2023-12-01T03:11:32.207642+0000","last_active":"2023-12-01T03:11:32.215183+0000","last_peered":"2023-12-01T03:11:32.215183+0000","last_clean":"2023-12-01T03:11:32.215183+0000","last_became_active":"2023-12-01T03:11:32.207344+0000","last_became_peered":"2023-12-01T03:11:32.207344+0000","last_unstale":"2023-12-01T03:11:32.215183+0000","last_undegraded":"2023-12-01T03:11:32.215183+0000","last_fullsized":"2023-12-01T03:11:32.215183+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:10.441679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223757+0000","last_change":"2023-12-01T03:11:32.220696+0000","last_active":"2023-12-01T03:11:32.223757+0000","last_peered":"2023-12-01T03:11:32.223757+0000","last_clean":"2023-12-01T03:11:32.223757+0000","last_became_active":"2023-12-01T03:11:32.220353+0000","last_became_peered":"2023-12-01T03:11:32.220353+0000","last_unstale":"2023-12-01T03:11:32.223757+0000","last_undegraded":"2023-12-01T03:11:32.223757+0000","last_fullsized":"2023-12-01T03:11:32.223757+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:53:43.178969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211636+0000","last_change":"2023-12-01T03:11:32.207709+0000","last_active":"2023-12-01T03:11:32.211636+0000","last_peered":"2023-12-01T03:11:32.211636+0000","last_clean":"2023-12-01T03:11:32.211636+0000","last_became_active":"2023-12-01T03:11:32.205510+0000","last_became_peered":"2023-12-01T03:11:32.205510+0000","last_unstale":"2023-12-01T03:11:32.211636+0000","last_undegraded":"2023-12-01T03:11:32.211636+0000","last_fullsized":"2023-12-01T03:11:32.211636+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216610+0000","last_change":"2023-12-01T03:11:32.210643+0000","last_active":"2023-12-01T03:11:32.216610+0000","last_peered":"2023-12-01T03:11:32.216610+0000","last_clean":"2023-12-01T03:11:32.216610+0000","last_became_active":"2023-12-01T03:11:32.210367+0000","last_became_peered":"2023-12-01T03:11:32.210367+0000","last_unstale":"2023-12-01T03:11:32.216610+0000","last_undegraded":"2023-12-01T03:11:32.216610+0000","last_fullsized":"2023-12-01T03:11:32.216610+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:00.674458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.214317+0000","last_change":"2023-12-01T03:11:32.209242+0000","last_active":"2023-12-01T03:11:32.214317+0000","last_peered":"2023-12-01T03:11:32.214317+0000","last_clean":"2023-12-01T03:11:32.214317+0000","last_became_active":"2023-12-01T03:11:32.208985+0000","last_became_peered":"2023-12-01T03:11:32.208985+0000","last_unstale":"2023-12-01T03:11:32.214317+0000","last_undegraded":"2023-12-01T03:11:32.214317+0000","last_fullsized":"2023-12-01T03:11:32.214317+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:29.661974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223614+0000","last_change":"2023-12-01T03:11:32.220828+0000","last_active":"2023-12-01T03:11:32.223614+0000","last_peered":"2023-12-01T03:11:32.223614+0000","last_clean":"2023-12-01T03:11:32.223614+0000","last_became_active":"2023-12-01T03:11:32.219607+0000","last_became_peered":"2023-12-01T03:11:32.219607+0000","last_unstale":"2023-12-01T03:11:32.223614+0000","last_undegraded":"2023-12-01T03:11:32.223614+0000","last_fullsized":"2023-12-01T03:11:32.223614+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:09.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.205877+0000","last_change":"2023-12-01T03:11:32.198609+0000","last_active":"2023-12-01T03:11:32.205877+0000","last_peered":"2023-12-01T03:11:32.205877+0000","last_clean":"2023-12-01T03:11:32.205877+0000","last_became_active":"2023-12-01T03:11:32.197869+0000","last_became_peered":"2023-12-01T03:11:32.197869+0000","last_unstale":"2023-12-01T03:11:32.205877+0000","last_undegraded":"2023-12-01T03:11:32.205877+0000","last_fullsized":"2023-12-01T03:11:32.205877+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:35.975600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217603+0000","last_change":"2023-12-01T03:11:32.212954+0000","last_active":"2023-12-01T03:11:32.217603+0000","last_peered":"2023-12-01T03:11:32.217603+0000","last_clean":"2023-12-01T03:11:32.217603+0000","last_became_active":"2023-12-01T03:11:32.212678+0000","last_became_peered":"2023-12-01T03:11:32.212678+0000","last_unstale":"2023-12-01T03:11:32.217603+0000","last_undegraded":"2023-12-01T03:11:32.217603+0000","last_fullsized":"2023-12-01T03:11:32.217603+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:28.852429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220068+0000","last_change":"2023-12-01T03:11:32.215698+0000","last_active":"2023-12-01T03:11:32.220068+0000","last_peered":"2023-12-01T03:11:32.220068+0000","last_clean":"2023-12-01T03:11:32.220068+0000","last_became_active":"2023-12-01T03:11:32.214531+0000","last_became_peered":"2023-12-01T03:11:32.214531+0000","last_unstale":"2023-12-01T03:11:32.220068+0000","last_undegraded":"2023-12-01T03:11:32.220068+0000","last_fullsized":"2023-12-01T03:11:32.220068+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:14.716448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212370+0000","last_change":"2023-12-01T03:11:32.204327+0000","last_active":"2023-12-01T03:11:32.212370+0000","last_peered":"2023-12-01T03:11:32.212370+0000","last_clean":"2023-12-01T03:11:32.212370+0000","last_became_active":"2023-12-01T03:11:32.203765+0000","last_became_peered":"2023-12-01T03:11:32.203765+0000","last_unstale":"2023-12-01T03:11:32.212370+0000","last_undegraded":"2023-12-01T03:11:32.212370+0000","last_fullsized":"2023-12-01T03:11:32.212370+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:28:34.324085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213777+0000","last_change":"2023-12-01T03:11:32.212023+0000","last_active":"2023-12-01T03:11:32.213777+0000","last_peered":"2023-12-01T03:11:32.213777+0000","last_clean":"2023-12-01T03:11:32.213777+0000","last_became_active":"2023-12-01T03:11:32.210975+0000","last_became_peered":"2023-12-01T03:11:32.210975+0000","last_unstale":"2023-12-01T03:11:32.213777+0000","last_undegraded":"2023-12-01T03:11:32.213777+0000","last_fullsized":"2023-12-01T03:11:32.213777+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217256+0000","last_change":"2023-12-01T03:11:32.213074+0000","last_active":"2023-12-01T03:11:32.217256+0000","last_peered":"2023-12-01T03:11:32.217256+0000","last_clean":"2023-12-01T03:11:32.217256+0000","last_became_active":"2023-12-01T03:11:32.212775+0000","last_became_peered":"2023-12-01T03:11:32.212775+0000","last_unstale":"2023-12-01T03:11:32.217256+0000","last_undegraded":"2023-12-01T03:11:32.217256+0000","last_fullsized":"2023-12-01T03:11:32.217256+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:36.571608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225974+0000","last_change":"2023-12-01T03:11:32.225479+0000","last_active":"2023-12-01T03:11:32.225974+0000","last_peered":"2023-12-01T03:11:32.225974+0000","last_clean":"2023-12-01T03:11:32.225974+0000","last_became_active":"2023-12-01T03:11:32.225154+0000","last_became_peered":"2023-12-01T03:11:32.225154+0000","last_unstale":"2023-12-01T03:11:32.225974+0000","last_undegraded":"2023-12-01T03:11:32.225974+0000","last_fullsized":"2023-12-01T03:11:32.225974+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:14.760284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220523+0000","last_change":"2023-12-01T03:11:32.214972+0000","last_active":"2023-12-01T03:11:32.220523+0000","last_peered":"2023-12-01T03:11:32.220523+0000","last_clean":"2023-12-01T03:11:32.220523+0000","last_became_active":"2023-12-01T03:11:32.214729+0000","last_became_peered":"2023-12-01T03:11:32.214729+0000","last_unstale":"2023-12-01T03:11:32.220523+0000","last_undegraded":"2023-12-01T03:11:32.220523+0000","last_fullsized":"2023-12-01T03:11:32.220523+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:24.244094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1f","version":"234'4","reported_seq":15,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225296+0000","last_change":"2023-12-01T03:11:31.680864+0000","last_active":"2023-12-01T03:11:32.225296+0000","last_peered":"2023-12-01T03:11:32.225296+0000","last_clean":"2023-12-01T03:11:32.225296+0000","last_became_active":"2023-12-01T03:11:31.680279+0000","last_became_peered":"2023-12-01T03:11:31.680279+0000","last_unstale":"2023-12-01T03:11:32.225296+0000","last_undegraded":"2023-12-01T03:11:32.225296+0000","last_fullsized":"2023-12-01T03:11:32.225296+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.377359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217958+0000","last_change":"2023-12-01T03:11:32.211743+0000","last_active":"2023-12-01T03:11:32.217958+0000","last_peered":"2023-12-01T03:11:32.217958+0000","last_clean":"2023-12-01T03:11:32.217958+0000","last_became_active":"2023-12-01T03:11:32.211443+0000","last_became_peered":"2023-12-01T03:11:32.211443+0000","last_unstale":"2023-12-01T03:11:32.217958+0000","last_undegraded":"2023-12-01T03:11:32.217958+0000","last_fullsized":"2023-12-01T03:11:32.217958+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224872+0000","last_change":"2023-12-01T03:11:31.679798+0000","last_active":"2023-12-01T03:11:32.224872+0000","last_peered":"2023-12-01T03:11:32.224872+0000","last_clean":"2023-12-01T03:11:32.224872+0000","last_became_active":"2023-12-01T03:11:31.679030+0000","last_became_peered":"2023-12-01T03:11:31.679030+0000","last_unstale":"2023-12-01T03:11:32.224872+0000","last_undegraded":"2023-12-01T03:11:32.224872+0000","last_fullsized":"2023-12-01T03:11:32.224872+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:49.261314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223595+0000","last_change":"2023-12-01T03:11:32.222760+0000","last_active":"2023-12-01T03:11:32.223595+0000","last_peered":"2023-12-01T03:11:32.223595+0000","last_clean":"2023-12-01T03:11:32.223595+0000","last_became_active":"2023-12-01T03:11:32.222500+0000","last_became_peered":"2023-12-01T03:11:32.222500+0000","last_unstale":"2023-12-01T03:11:32.223595+0000","last_undegraded":"2023-12-01T03:11:32.223595+0000","last_fullsized":"2023-12-01T03:11:32.223595+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:17.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218820+0000","last_change":"2023-12-01T03:11:31.663745+0000","last_active":"2023-12-01T03:11:32.218820+0000","last_peered":"2023-12-01T03:11:32.218820+0000","last_clean":"2023-12-01T03:11:32.218820+0000","last_became_active":"2023-12-01T03:11:31.662706+0000","last_became_peered":"2023-12-01T03:11:31.662706+0000","last_unstale":"2023-12-01T03:11:32.218820+0000","last_undegraded":"2023-12-01T03:11:32.218820+0000","last_fullsized":"2023-12-01T03:11:32.218820+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:42.945449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217635+0000","last_change":"2023-12-01T03:11:32.213316+0000","last_active":"2023-12-01T03:11:32.217635+0000","last_peered":"2023-12-01T03:11:32.217635+0000","last_clean":"2023-12-01T03:11:32.217635+0000","last_became_active":"2023-12-01T03:11:32.213070+0000","last_became_peered":"2023-12-01T03:11:32.213070+0000","last_unstale":"2023-12-01T03:11:32.217635+0000","last_undegraded":"2023-12-01T03:11:32.217635+0000","last_fullsized":"2023-12-01T03:11:32.217635+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:22:51.687377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216557+0000","last_change":"2023-12-01T03:11:31.660758+0000","last_active":"2023-12-01T03:11:32.216557+0000","last_peered":"2023-12-01T03:11:32.216557+0000","last_clean":"2023-12-01T03:11:32.216557+0000","last_became_active":"2023-12-01T03:11:31.660001+0000","last_became_peered":"2023-12-01T03:11:31.660001+0000","last_unstale":"2023-12-01T03:11:32.216557+0000","last_undegraded":"2023-12-01T03:11:32.216557+0000","last_fullsized":"2023-12-01T03:11:32.216557+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218251+0000","last_change":"2023-12-01T03:11:32.213961+0000","last_active":"2023-12-01T03:11:32.218251+0000","last_peered":"2023-12-01T03:11:32.218251+0000","last_clean":"2023-12-01T03:11:32.218251+0000","last_became_active":"2023-12-01T03:11:32.213692+0000","last_became_peered":"2023-12-01T03:11:32.213692+0000","last_unstale":"2023-12-01T03:11:32.218251+0000","last_undegraded":"2023-12-01T03:11:32.218251+0000","last_fullsized":"2023-12-01T03:11:32.218251+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224769+0000","last_change":"2023-12-01T03:11:31.681531+0000","last_active":"2023-12-01T03:11:32.224769+0000","last_peered":"2023-12-01T03:11:32.224769+0000","last_clean":"2023-12-01T03:11:32.224769+0000","last_became_active":"2023-12-01T03:11:31.680782+0000","last_became_peered":"2023-12-01T03:11:31.680782+0000","last_unstale":"2023-12-01T03:11:32.224769+0000","last_undegraded":"2023-12-01T03:11:32.224769+0000","last_fullsized":"2023-12-01T03:11:32.224769+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:18.018055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211538+0000","last_change":"2023-12-01T03:11:32.206659+0000","last_active":"2023-12-01T03:11:32.211538+0000","last_peered":"2023-12-01T03:11:32.211538+0000","last_clean":"2023-12-01T03:11:32.211538+0000","last_became_active":"2023-12-01T03:11:32.206346+0000","last_became_peered":"2023-12-01T03:11:32.206346+0000","last_unstale":"2023-12-01T03:11:32.211538+0000","last_undegraded":"2023-12-01T03:11:32.211538+0000","last_fullsized":"2023-12-01T03:11:32.211538+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.196693+0000","last_change":"2023-12-01T03:11:31.663166+0000","last_active":"2023-12-01T03:11:32.196693+0000","last_peered":"2023-12-01T03:11:32.196693+0000","last_clean":"2023-12-01T03:11:32.196693+0000","last_became_active":"2023-12-01T03:11:31.662263+0000","last_became_peered":"2023-12-01T03:11:31.662263+0000","last_unstale":"2023-12-01T03:11:32.196693+0000","last_undegraded":"2023-12-01T03:11:32.196693+0000","last_fullsized":"2023-12-01T03:11:32.196693+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:57:00.965499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216627+0000","last_change":"2023-12-01T03:11:32.210988+0000","last_active":"2023-12-01T03:11:32.216627+0000","last_peered":"2023-12-01T03:11:32.216627+0000","last_clean":"2023-12-01T03:11:32.216627+0000","last_became_active":"2023-12-01T03:11:32.207844+0000","last_became_peered":"2023-12-01T03:11:32.207844+0000","last_unstale":"2023-12-01T03:11:32.216627+0000","last_undegraded":"2023-12-01T03:11:32.216627+0000","last_fullsized":"2023-12-01T03:11:32.216627+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.206110+0000","last_change":"2023-12-01T03:11:31.670524+0000","last_active":"2023-12-01T03:11:32.206110+0000","last_peered":"2023-12-01T03:11:32.206110+0000","last_clean":"2023-12-01T03:11:32.206110+0000","last_became_active":"2023-12-01T03:11:31.670190+0000","last_became_peered":"2023-12-01T03:11:31.670190+0000","last_unstale":"2023-12-01T03:11:32.206110+0000","last_undegraded":"2023-12-01T03:11:32.206110+0000","last_fullsized":"2023-12-01T03:11:32.206110+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:45.874739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.214311+0000","last_change":"2023-12-01T03:11:32.212987+0000","last_active":"2023-12-01T03:11:32.214311+0000","last_peered":"2023-12-01T03:11:32.214311+0000","last_clean":"2023-12-01T03:11:32.214311+0000","last_became_active":"2023-12-01T03:11:32.212426+0000","last_became_peered":"2023-12-01T03:11:32.212426+0000","last_unstale":"2023-12-01T03:11:32.214311+0000","last_undegraded":"2023-12-01T03:11:32.214311+0000","last_fullsized":"2023-12-01T03:11:32.214311+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.219351+0000","last_change":"2023-12-01T03:11:31.676322+0000","last_active":"2023-12-01T03:11:32.219351+0000","last_peered":"2023-12-01T03:11:32.219351+0000","last_clean":"2023-12-01T03:11:32.219351+0000","last_became_active":"2023-12-01T03:11:31.674451+0000","last_became_peered":"2023-12-01T03:11:31.674451+0000","last_unstale":"2023-12-01T03:11:32.219351+0000","last_undegraded":"2023-12-01T03:11:32.219351+0000","last_fullsized":"2023-12-01T03:11:32.219351+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:44.820233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222710+0000","last_change":"2023-12-01T03:11:32.219383+0000","last_active":"2023-12-01T03:11:32.222710+0000","last_peered":"2023-12-01T03:11:32.222710+0000","last_clean":"2023-12-01T03:11:32.222710+0000","last_became_active":"2023-12-01T03:11:32.219139+0000","last_became_peered":"2023-12-01T03:11:32.219139+0000","last_unstale":"2023-12-01T03:11:32.222710+0000","last_undegraded":"2023-12-01T03:11:32.222710+0000","last_fullsized":"2023-12-01T03:11:32.222710+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:40.557913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.192088+0000","last_change":"2023-12-01T03:11:31.662929+0000","last_active":"2023-12-01T03:11:32.192088+0000","last_peered":"2023-12-01T03:11:32.192088+0000","last_clean":"2023-12-01T03:11:32.192088+0000","last_became_active":"2023-12-01T03:11:31.662617+0000","last_became_peered":"2023-12-01T03:11:31.662617+0000","last_unstale":"2023-12-01T03:11:32.192088+0000","last_undegraded":"2023-12-01T03:11:32.192088+0000","last_fullsized":"2023-12-01T03:11:32.192088+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:34.611280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210926+0000","last_change":"2023-12-01T03:11:32.209244+0000","last_active":"2023-12-01T03:11:32.210926+0000","last_peered":"2023-12-01T03:11:32.210926+0000","last_clean":"2023-12-01T03:11:32.210926+0000","last_became_active":"2023-12-01T03:11:32.206294+0000","last_became_peered":"2023-12-01T03:11:32.206294+0000","last_unstale":"2023-12-01T03:11:32.210926+0000","last_undegraded":"2023-12-01T03:11:32.210926+0000","last_fullsized":"2023-12-01T03:11:32.210926+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"234'3","reported_seq":14,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218935+0000","last_change":"2023-12-01T03:11:31.664798+0000","last_active":"2023-12-01T03:11:32.218935+0000","last_peered":"2023-12-01T03:11:32.218935+0000","last_clean":"2023-12-01T03:11:32.218935+0000","last_became_active":"2023-12-01T03:11:31.664276+0000","last_became_peered":"2023-12-01T03:11:31.664276+0000","last_unstale":"2023-12-01T03:11:32.218935+0000","last_undegraded":"2023-12-01T03:11:32.218935+0000","last_fullsized":"2023-12-01T03:11:32.218935+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:18:18.909277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.215970+0000","last_change":"2023-12-01T03:11:32.213607+0000","last_active":"2023-12-01T03:11:32.215970+0000","last_peered":"2023-12-01T03:11:32.215970+0000","last_clean":"2023-12-01T03:11:32.215970+0000","last_became_active":"2023-12-01T03:11:32.211875+0000","last_became_peered":"2023-12-01T03:11:32.211875+0000","last_unstale":"2023-12-01T03:11:32.215970+0000","last_undegraded":"2023-12-01T03:11:32.215970+0000","last_fullsized":"2023-12-01T03:11:32.215970+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"234'6","reported_seq":17,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223391+0000","last_change":"2023-12-01T03:11:31.664162+0000","last_active":"2023-12-01T03:11:32.223391+0000","last_peered":"2023-12-01T03:11:32.223391+0000","last_clean":"2023-12-01T03:11:32.223391+0000","last_became_active":"2023-12-01T03:11:31.663854+0000","last_became_peered":"2023-12-01T03:11:31.663854+0000","last_unstale":"2023-12-01T03:11:32.223391+0000","last_undegraded":"2023-12-01T03:11:32.223391+0000","last_fullsized":"2023-12-01T03:11:32.223391+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:15.367696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209163+0000","last_change":"2023-12-01T03:11:32.203625+0000","last_active":"2023-12-01T03:11:32.209163+0000","last_peered":"2023-12-01T03:11:32.209163+0000","last_clean":"2023-12-01T03:11:32.209163+0000","last_became_active":"2023-12-01T03:11:32.203300+0000","last_became_peered":"2023-12-01T03:11:32.203300+0000","last_unstale":"2023-12-01T03:11:32.209163+0000","last_undegraded":"2023-12-01T03:11:32.209163+0000","last_fullsized":"2023-12-01T03:11:32.209163+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:46.890682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222667+0000","last_change":"2023-12-01T03:11:31.665987+0000","last_active":"2023-12-01T03:11:32.222667+0000","last_peered":"2023-12-01T03:11:32.222667+0000","last_clean":"2023-12-01T03:11:32.222667+0000","last_became_active":"2023-12-01T03:11:31.665550+0000","last_became_peered":"2023-12-01T03:11:31.665550+0000","last_unstale":"2023-12-01T03:11:32.222667+0000","last_undegraded":"2023-12-01T03:11:32.222667+0000","last_fullsized":"2023-12-01T03:11:32.222667+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:16.324155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209603+0000","last_change":"2023-12-01T03:11:32.208104+0000","last_active":"2023-12-01T03:11:32.209603+0000","last_peered":"2023-12-01T03:11:32.209603+0000","last_clean":"2023-12-01T03:11:32.209603+0000","last_became_active":"2023-12-01T03:11:32.207859+0000","last_became_peered":"2023-12-01T03:11:32.207859+0000","last_unstale":"2023-12-01T03:11:32.209603+0000","last_undegraded":"2023-12-01T03:11:32.209603+0000","last_fullsized":"2023-12-01T03:11:32.209603+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.188460+0000","last_change":"2023-12-01T03:11:31.675387+0000","last_active":"2023-12-01T03:11:32.188460+0000","last_peered":"2023-12-01T03:11:32.188460+0000","last_clean":"2023-12-01T03:11:32.188460+0000","last_became_active":"2023-12-01T03:11:31.675108+0000","last_became_peered":"2023-12-01T03:11:31.675108+0000","last_unstale":"2023-12-01T03:11:32.188460+0000","last_undegraded":"2023-12-01T03:11:32.188460+0000","last_fullsized":"2023-12-01T03:11:32.188460+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.836486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211907+0000","last_change":"2023-12-01T03:11:32.204685+0000","last_active":"2023-12-01T03:11:32.211907+0000","last_peered":"2023-12-01T03:11:32.211907+0000","last_clean":"2023-12-01T03:11:32.211907+0000","last_became_active":"2023-12-01T03:11:32.204422+0000","last_became_peered":"2023-12-01T03:11:32.204422+0000","last_unstale":"2023-12-01T03:11:32.211907+0000","last_undegraded":"2023-12-01T03:11:32.211907+0000","last_fullsized":"2023-12-01T03:11:32.211907+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.188647+0000","last_change":"2023-12-01T03:11:31.671174+0000","last_active":"2023-12-01T03:11:32.188647+0000","last_peered":"2023-12-01T03:11:32.188647+0000","last_clean":"2023-12-01T03:11:32.188647+0000","last_became_active":"2023-12-01T03:11:31.670862+0000","last_became_peered":"2023-12-01T03:11:31.670862+0000","last_unstale":"2023-12-01T03:11:32.188647+0000","last_undegraded":"2023-12-01T03:11:32.188647+0000","last_fullsized":"2023-12-01T03:11:32.188647+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:37.158883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209558+0000","last_change":"2023-12-01T03:11:32.203594+0000","last_active":"2023-12-01T03:11:32.209558+0000","last_peered":"2023-12-01T03:11:32.209558+0000","last_clean":"2023-12-01T03:11:32.209558+0000","last_became_active":"2023-12-01T03:11:32.203346+0000","last_became_peered":"2023-12-01T03:11:32.203346+0000","last_unstale":"2023-12-01T03:11:32.209558+0000","last_undegraded":"2023-12-01T03:11:32.209558+0000","last_fullsized":"2023-12-01T03:11:32.209558+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:22.924077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.10","version":"234'8","reported_seq":19,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218897+0000","last_change":"2023-12-01T03:11:31.673143+0000","last_active":"2023-12-01T03:11:32.218897+0000","last_peered":"2023-12-01T03:11:32.218897+0000","last_clean":"2023-12-01T03:11:32.218897+0000","last_became_active":"2023-12-01T03:11:31.671261+0000","last_became_peered":"2023-12-01T03:11:31.671261+0000","last_unstale":"2023-12-01T03:11:32.218897+0000","last_undegraded":"2023-12-01T03:11:32.218897+0000","last_fullsized":"2023-12-01T03:11:32.218897+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:13.744340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213478+0000","last_change":"2023-12-01T03:11:32.209735+0000","last_active":"2023-12-01T03:11:32.213478+0000","last_peered":"2023-12-01T03:11:32.213478+0000","last_clean":"2023-12-01T03:11:32.213478+0000","last_became_active":"2023-12-01T03:11:32.209148+0000","last_became_peered":"2023-12-01T03:11:32.209148+0000","last_unstale":"2023-12-01T03:11:32.213478+0000","last_undegraded":"2023-12-01T03:11:32.213478+0000","last_fullsized":"2023-12-01T03:11:32.213478+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:46.532354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225412+0000","last_change":"2023-12-01T03:11:31.677841+0000","last_active":"2023-12-01T03:11:32.225412+0000","last_peered":"2023-12-01T03:11:32.225412+0000","last_clean":"2023-12-01T03:11:32.225412+0000","last_became_active":"2023-12-01T03:11:31.677483+0000","last_became_peered":"2023-12-01T03:11:31.677483+0000","last_unstale":"2023-12-01T03:11:32.225412+0000","last_undegraded":"2023-12-01T03:11:32.225412+0000","last_fullsized":"2023-12-01T03:11:32.225412+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:31.837132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217241+0000","last_change":"2023-12-01T03:11:32.213539+0000","last_active":"2023-12-01T03:11:32.217241+0000","last_peered":"2023-12-01T03:11:32.217241+0000","last_clean":"2023-12-01T03:11:32.217241+0000","last_became_active":"2023-12-01T03:11:32.212492+0000","last_became_peered":"2023-12-01T03:11:32.212492+0000","last_unstale":"2023-12-01T03:11:32.217241+0000","last_undegraded":"2023-12-01T03:11:32.217241+0000","last_fullsized":"2023-12-01T03:11:32.217241+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:29:48.204277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.e","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222148+0000","last_change":"2023-12-01T03:11:31.670701+0000","last_active":"2023-12-01T03:11:32.222148+0000","last_peered":"2023-12-01T03:11:32.222148+0000","last_clean":"2023-12-01T03:11:32.222148+0000","last_became_active":"2023-12-01T03:11:31.670179+0000","last_became_peered":"2023-12-01T03:11:31.670179+0000","last_unstale":"2023-12-01T03:11:32.222148+0000","last_undegraded":"2023-12-01T03:11:32.222148+0000","last_fullsized":"2023-12-01T03:11:32.222148+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:01.094744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216871+0000","last_change":"2023-12-01T03:11:32.212327+0000","last_active":"2023-12-01T03:11:32.216871+0000","last_peered":"2023-12-01T03:11:32.216871+0000","last_clean":"2023-12-01T03:11:32.216871+0000","last_became_active":"2023-12-01T03:11:32.212068+0000","last_became_peered":"2023-12-01T03:11:32.212068+0000","last_unstale":"2023-12-01T03:11:32.216871+0000","last_undegraded":"2023-12-01T03:11:32.216871+0000","last_fullsized":"2023-12-01T03:11:32.216871+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:31.875817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224912+0000","last_change":"2023-12-01T03:11:31.679287+0000","last_active":"2023-12-01T03:11:32.224912+0000","last_peered":"2023-12-01T03:11:32.224912+0000","last_clean":"2023-12-01T03:11:32.224912+0000","last_became_active":"2023-12-01T03:11:31.678925+0000","last_became_peered":"2023-12-01T03:11:31.678925+0000","last_unstale":"2023-12-01T03:11:32.224912+0000","last_undegraded":"2023-12-01T03:11:32.224912+0000","last_fullsized":"2023-12-01T03:11:32.224912+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:28:46.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.221614+0000","last_change":"2023-12-01T03:11:32.214436+0000","last_active":"2023-12-01T03:11:32.221614+0000","last_peered":"2023-12-01T03:11:32.221614+0000","last_clean":"2023-12-01T03:11:32.221614+0000","last_became_active":"2023-12-01T03:11:32.214162+0000","last_became_peered":"2023-12-01T03:11:32.214162+0000","last_unstale":"2023-12-01T03:11:32.221614+0000","last_undegraded":"2023-12-01T03:11:32.221614+0000","last_fullsized":"2023-12-01T03:11:32.221614+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:18:18.055289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.192683+0000","last_change":"2023-12-01T03:11:31.674632+0000","last_active":"2023-12-01T03:11:32.192683+0000","last_peered":"2023-12-01T03:11:32.192683+0000","last_clean":"2023-12-01T03:11:32.192683+0000","last_became_active":"2023-12-01T03:11:31.674319+0000","last_became_peered":"2023-12-01T03:11:31.674319+0000","last_unstale":"2023-12-01T03:11:32.192683+0000","last_undegraded":"2023-12-01T03:11:32.192683+0000","last_fullsized":"2023-12-01T03:11:32.192683+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:21.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216607+0000","last_change":"2023-12-01T03:11:32.209024+0000","last_active":"2023-12-01T03:11:32.216607+0000","last_peered":"2023-12-01T03:11:32.216607+0000","last_clean":"2023-12-01T03:11:32.216607+0000","last_became_active":"2023-12-01T03:11:32.208729+0000","last_became_peered":"2023-12-01T03:11:32.208729+0000","last_unstale":"2023-12-01T03:11:32.216607+0000","last_undegraded":"2023-12-01T03:11:32.216607+0000","last_fullsized":"2023-12-01T03:11:32.216607+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:27:53.473015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.b","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223478+0000","last_change":"2023-12-01T03:11:31.676885+0000","last_active":"2023-12-01T03:11:32.223478+0000","last_peered":"2023-12-01T03:11:32.223478+0000","last_clean":"2023-12-01T03:11:32.223478+0000","last_became_active":"2023-12-01T03:11:31.675545+0000","last_became_peered":"2023-12-01T03:11:31.675545+0000","last_unstale":"2023-12-01T03:11:32.223478+0000","last_undegraded":"2023-12-01T03:11:32.223478+0000","last_fullsized":"2023-12-01T03:11:32.223478+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:03.962061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212390+0000","last_change":"2023-12-01T03:11:32.204502+0000","last_active":"2023-12-01T03:11:32.212390+0000","last_peered":"2023-12-01T03:11:32.212390+0000","last_clean":"2023-12-01T03:11:32.212390+0000","last_became_active":"2023-12-01T03:11:32.204048+0000","last_became_peered":"2023-12-01T03:11:32.204048+0000","last_unstale":"2023-12-01T03:11:32.212390+0000","last_undegraded":"2023-12-01T03:11:32.212390+0000","last_fullsized":"2023-12-01T03:11:32.212390+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:15:30.930202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.183774+0000","last_change":"2023-12-01T03:11:31.675947+0000","last_active":"2023-12-01T03:11:32.183774+0000","last_peered":"2023-12-01T03:11:32.183774+0000","last_clean":"2023-12-01T03:11:32.183774+0000","last_became_active":"2023-12-01T03:11:31.675372+0000","last_became_peered":"2023-12-01T03:11:31.675372+0000","last_unstale":"2023-12-01T03:11:32.183774+0000","last_undegraded":"2023-12-01T03:11:32.183774+0000","last_fullsized":"2023-12-01T03:11:32.183774+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:45.516411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216820+0000","last_change":"2023-12-01T03:11:32.214460+0000","last_active":"2023-12-01T03:11:32.216820+0000","last_peered":"2023-12-01T03:11:32.216820+0000","last_clean":"2023-12-01T03:11:32.216820+0000","last_became_active":"2023-12-01T03:11:32.213987+0000","last_became_peered":"2023-12-01T03:11:32.213987+0000","last_unstale":"2023-12-01T03:11:32.216820+0000","last_undegraded":"2023-12-01T03:11:32.216820+0000","last_fullsized":"2023-12-01T03:11:32.216820+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:20.713703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.194822+0000","last_change":"2023-12-01T03:11:31.673899+0000","last_active":"2023-12-01T03:11:32.194822+0000","last_peered":"2023-12-01T03:11:32.194822+0000","last_clean":"2023-12-01T03:11:32.194822+0000","last_became_active":"2023-12-01T03:11:31.673474+0000","last_became_peered":"2023-12-01T03:11:31.673474+0000","last_unstale":"2023-12-01T03:11:32.194822+0000","last_undegraded":"2023-12-01T03:11:32.194822+0000","last_fullsized":"2023-12-01T03:11:32.194822+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:50.332038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218479+0000","last_change":"2023-12-01T03:11:32.214835+0000","last_active":"2023-12-01T03:11:32.218479+0000","last_peered":"2023-12-01T03:11:32.218479+0000","last_clean":"2023-12-01T03:11:32.218479+0000","last_became_active":"2023-12-01T03:11:32.214553+0000","last_became_peered":"2023-12-01T03:11:32.214553+0000","last_unstale":"2023-12-01T03:11:32.218479+0000","last_undegraded":"2023-12-01T03:11:32.218479+0000","last_fullsized":"2023-12-01T03:11:32.218479+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.190392+0000","last_change":"2023-12-01T03:11:31.682949+0000","last_active":"2023-12-01T03:11:32.190392+0000","last_peered":"2023-12-01T03:11:32.190392+0000","last_clean":"2023-12-01T03:11:32.190392+0000","last_became_active":"2023-12-01T03:11:31.682454+0000","last_became_peered":"2023-12-01T03:11:31.682454+0000","last_unstale":"2023-12-01T03:11:32.190392+0000","last_undegraded":"2023-12-01T03:11:32.190392+0000","last_fullsized":"2023-12-01T03:11:32.190392+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:49.777291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.208510+0000","last_change":"2023-12-01T03:11:32.204430+0000","last_active":"2023-12-01T03:11:32.208510+0000","last_peered":"2023-12-01T03:11:32.208510+0000","last_clean":"2023-12-01T03:11:32.208510+0000","last_became_active":"2023-12-01T03:11:32.204043+0000","last_became_peered":"2023-12-01T03:11:32.204043+0000","last_unstale":"2023-12-01T03:11:32.208510+0000","last_undegraded":"2023-12-01T03:11:32.208510+0000","last_fullsized":"2023-12-01T03:11:32.208510+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:33:45.498838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222532+0000","last_change":"2023-12-01T03:11:31.668375+0000","last_active":"2023-12-01T03:11:32.222532+0000","last_peered":"2023-12-01T03:11:32.222532+0000","last_clean":"2023-12-01T03:11:32.222532+0000","last_became_active":"2023-12-01T03:11:31.668109+0000","last_became_peered":"2023-12-01T03:11:31.668109+0000","last_unstale":"2023-12-01T03:11:32.222532+0000","last_undegraded":"2023-12-01T03:11:32.222532+0000","last_fullsized":"2023-12-01T03:11:32.222532+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:34.959657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225708+0000","last_change":"2023-12-01T03:11:32.223409+0000","last_active":"2023-12-01T03:11:32.225708+0000","last_peered":"2023-12-01T03:11:32.225708+0000","last_clean":"2023-12-01T03:11:32.225708+0000","last_became_active":"2023-12-01T03:11:32.221541+0000","last_became_peered":"2023-12-01T03:11:32.221541+0000","last_unstale":"2023-12-01T03:11:32.225708+0000","last_undegraded":"2023-12-01T03:11:32.225708+0000","last_fullsized":"2023-12-01T03:11:32.225708+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:53.273884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.219881+0000","last_change":"2023-12-01T03:11:31.679248+0000","last_active":"2023-12-01T03:11:32.219881+0000","last_peered":"2023-12-01T03:11:32.219881+0000","last_clean":"2023-12-01T03:11:32.219881+0000","last_became_active":"2023-12-01T03:11:31.678973+0000","last_became_peered":"2023-12-01T03:11:31.678973+0000","last_unstale":"2023-12-01T03:11:32.219881+0000","last_undegraded":"2023-12-01T03:11:32.219881+0000","last_fullsized":"2023-12-01T03:11:32.219881+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212063+0000","last_change":"2023-12-01T03:11:32.207147+0000","last_active":"2023-12-01T03:11:32.212063+0000","last_peered":"2023-12-01T03:11:32.212063+0000","last_clean":"2023-12-01T03:11:32.212063+0000","last_became_active":"2023-12-01T03:11:32.206872+0000","last_became_peered":"2023-12-01T03:11:32.206872+0000","last_unstale":"2023-12-01T03:11:32.212063+0000","last_undegraded":"2023-12-01T03:11:32.212063+0000","last_fullsized":"2023-12-01T03:11:32.212063+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.202418+0000","last_change":"2023-12-01T03:11:31.663771+0000","last_active":"2023-12-01T03:11:32.202418+0000","last_peered":"2023-12-01T03:11:32.202418+0000","last_clean":"2023-12-01T03:11:32.202418+0000","last_became_active":"2023-12-01T03:11:31.662877+0000","last_became_peered":"2023-12-01T03:11:31.662877+0000","last_unstale":"2023-12-01T03:11:32.202418+0000","last_undegraded":"2023-12-01T03:11:32.202418+0000","last_fullsized":"2023-12-01T03:11:32.202418+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:59.396265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213526+0000","last_change":"2023-12-01T03:11:32.208042+0000","last_active":"2023-12-01T03:11:32.213526+0000","last_peered":"2023-12-01T03:11:32.213526+0000","last_clean":"2023-12-01T03:11:32.213526+0000","last_became_active":"2023-12-01T03:11:32.207767+0000","last_became_peered":"2023-12-01T03:11:32.207767+0000","last_unstale":"2023-12-01T03:11:32.213526+0000","last_undegraded":"2023-12-01T03:11:32.213526+0000","last_fullsized":"2023-12-01T03:11:32.213526+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:17.846927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.195227+0000","last_change":"2023-12-01T03:11:31.673043+0000","last_active":"2023-12-01T03:11:32.195227+0000","last_peered":"2023-12-01T03:11:32.195227+0000","last_clean":"2023-12-01T03:11:32.195227+0000","last_became_active":"2023-12-01T03:11:31.672780+0000","last_became_peered":"2023-12-01T03:11:31.672780+0000","last_unstale":"2023-12-01T03:11:32.195227+0000","last_undegraded":"2023-12-01T03:11:32.195227+0000","last_fullsized":"2023-12-01T03:11:32.195227+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:40.478556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225682+0000","last_change":"2023-12-01T03:11:32.224518+0000","last_active":"2023-12-01T03:11:32.225682+0000","last_peered":"2023-12-01T03:11:32.225682+0000","last_clean":"2023-12-01T03:11:32.225682+0000","last_became_active":"2023-12-01T03:11:32.220831+0000","last_became_peered":"2023-12-01T03:11:32.220831+0000","last_unstale":"2023-12-01T03:11:32.225682+0000","last_undegraded":"2023-12-01T03:11:32.225682+0000","last_fullsized":"2023-12-01T03:11:32.225682+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.189860+0000","last_change":"2023-12-01T03:11:31.677162+0000","last_active":"2023-12-01T03:11:32.189860+0000","last_peered":"2023-12-01T03:11:32.189860+0000","last_clean":"2023-12-01T03:11:32.189860+0000","last_became_active":"2023-12-01T03:11:31.674178+0000","last_became_peered":"2023-12-01T03:11:31.674178+0000","last_unstale":"2023-12-01T03:11:32.189860+0000","last_undegraded":"2023-12-01T03:11:32.189860+0000","last_fullsized":"2023-12-01T03:11:32.189860+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:13.700505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225199+0000","last_change":"2023-12-01T03:11:32.222439+0000","last_active":"2023-12-01T03:11:32.225199+0000","last_peered":"2023-12-01T03:11:32.225199+0000","last_clean":"2023-12-01T03:11:32.225199+0000","last_became_active":"2023-12-01T03:11:32.221662+0000","last_became_peered":"2023-12-01T03:11:32.221662+0000","last_unstale":"2023-12-01T03:11:32.225199+0000","last_undegraded":"2023-12-01T03:11:32.225199+0000","last_fullsized":"2023-12-01T03:11:32.225199+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.188580+0000","last_change":"2023-12-01T03:11:31.672908+0000","last_active":"2023-12-01T03:11:32.188580+0000","last_peered":"2023-12-01T03:11:32.188580+0000","last_clean":"2023-12-01T03:11:32.188580+0000","last_became_active":"2023-12-01T03:11:31.671738+0000","last_became_peered":"2023-12-01T03:11:31.671738+0000","last_unstale":"2023-12-01T03:11:32.188580+0000","last_undegraded":"2023-12-01T03:11:32.188580+0000","last_fullsized":"2023-12-01T03:11:32.188580+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220159+0000","last_change":"2023-12-01T03:11:32.214055+0000","last_active":"2023-12-01T03:11:32.220159+0000","last_peered":"2023-12-01T03:11:32.220159+0000","last_clean":"2023-12-01T03:11:32.220159+0000","last_became_active":"2023-12-01T03:11:32.213767+0000","last_became_peered":"2023-12-01T03:11:32.213767+0000","last_unstale":"2023-12-01T03:11:32.220159+0000","last_undegraded":"2023-12-01T03:11:32.220159+0000","last_fullsized":"2023-12-01T03:11:32.220159+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:47:03.446609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.201156+0000","last_change":"2023-12-01T03:11:31.676480+0000","last_active":"2023-12-01T03:11:32.201156+0000","last_peered":"2023-12-01T03:11:32.201156+0000","last_clean":"2023-12-01T03:11:32.201156+0000","last_became_active":"2023-12-01T03:11:31.676127+0000","last_became_peered":"2023-12-01T03:11:31.676127+0000","last_unstale":"2023-12-01T03:11:32.201156+0000","last_undegraded":"2023-12-01T03:11:32.201156+0000","last_fullsized":"2023-12-01T03:11:32.201156+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.208183+0000","last_change":"2023-12-01T03:11:32.203024+0000","last_active":"2023-12-01T03:11:32.208183+0000","last_peered":"2023-12-01T03:11:32.208183+0000","last_clean":"2023-12-01T03:11:32.208183+0000","last_became_active":"2023-12-01T03:11:32.202779+0000","last_became_peered":"2023-12-01T03:11:32.202779+0000","last_unstale":"2023-12-01T03:11:32.208183+0000","last_undegraded":"2023-12-01T03:11:32.208183+0000","last_fullsized":"2023-12-01T03:11:32.208183+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.205725+0000","last_change":"2023-12-01T03:11:31.680251+0000","last_active":"2023-12-01T03:11:32.205725+0000","last_peered":"2023-12-01T03:11:32.205725+0000","last_clean":"2023-12-01T03:11:32.205725+0000","last_became_active":"2023-12-01T03:11:31.679936+0000","last_became_peered":"2023-12-01T03:11:31.679936+0000","last_unstale":"2023-12-01T03:11:32.205725+0000","last_undegraded":"2023-12-01T03:11:32.205725+0000","last_fullsized":"2023-12-01T03:11:32.205725+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:39.296836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211446+0000","last_change":"2023-12-01T03:11:32.210270+0000","last_active":"2023-12-01T03:11:32.211446+0000","last_peered":"2023-12-01T03:11:32.211446+0000","last_clean":"2023-12-01T03:11:32.211446+0000","last_became_active":"2023-12-01T03:11:32.209831+0000","last_became_peered":"2023-12-01T03:11:32.209831+0000","last_unstale":"2023-12-01T03:11:32.211446+0000","last_undegraded":"2023-12-01T03:11:32.211446+0000","last_fullsized":"2023-12-01T03:11:32.211446+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224896+0000","last_change":"2023-12-01T03:11:31.681391+0000","last_active":"2023-12-01T03:11:32.224896+0000","last_peered":"2023-12-01T03:11:32.224896+0000","last_clean":"2023-12-01T03:11:32.224896+0000","last_became_active":"2023-12-01T03:11:31.680773+0000","last_became_peered":"2023-12-01T03:11:31.680773+0000","last_unstale":"2023-12-01T03:11:32.224896+0000","last_undegraded":"2023-12-01T03:11:32.224896+0000","last_fullsized":"2023-12-01T03:11:32.224896+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:40:13.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"220'348","reported_seq":1214,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.205820+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:11:32.205820+0000","last_peered":"2023-12-01T03:11:32.205820+0000","last_clean":"2023-12-01T03:11:32.205820+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:11:32.205820+0000","last_undegraded":"2023-12-01T03:11:32.205820+0000","last_fullsized":"2023-12-01T03:11:32.205820+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575989,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60180,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311660,"statfs":{"total":96636764160,"available":96575139840,"internally_reserved":0,"allocated":2961408,"data_stored":1448140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.05,"15min":1.071},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5409999999999999,"5min":1.6000000000000001,"15min":1.7390000000000001},"last":1.613},{"interface":"front","average":{"1min":1.111,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6599999999999999,"5min":1.6699999999999999,"15min":1.7},"last":0.90500000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.236,"15min":1.2769999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4990000000000001,"5min":1.78,"15min":2.1419999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.294,"15min":1.3260000000000001},"min":{"1min":0.873,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5740000000000001,"5min":1.6240000000000001,"15min":1.8700000000000001},"last":1.5649999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1439999999999999,"15min":1.173},"min":{"1min":0.71699999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.573,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":1.647},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.2549999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.643,"5min":1.7050000000000001,"15min":2.1120000000000001},"last":1.147}]},{"osd":3,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2509999999999999,"15min":1.302},"min":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.712,"15min":2.431},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3720000000000001,"15min":1.4119999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7529999999999999,"5min":1.879,"15min":2.198},"last":1.476}]},{"osd":4,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3009999999999999,"15min":1.335},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7689999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.4510000000000001,"15min":1.49},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8799999999999999,"5min":1.97,"15min":2.569},"last":1.694}]},{"osd":5,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2869999999999999,"15min":1.3160000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7150000000000001,"5min":2.0099999999999998,"15min":2.7269999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.5469999999999999,"15min":1.585},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.0470000000000002,"5min":2.0569999999999999,"15min":2.484},"last":1.7589999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.403,"15min":1.4359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7749999999999999,"5min":1.921,"15min":2.532},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4299999999999999,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8260000000000001,"5min":1.8819999999999999,"15min":2.762},"last":1.726}]}]},{"osd":6,"up_from":13,"seq":55834575990,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42136,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329704,"statfs":{"total":96636764160,"available":96593616896,"internally_reserved":0,"allocated":2965504,"data_stored":1447614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0509999999999999,"15min":1.1080000000000001},"min":{"1min":0.66100000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.073,"5min":1.1040000000000001,"15min":1.157},"min":{"1min":0.78400000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6910000000000001,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.6439999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.141,"15min":1.2150000000000001},"min":{"1min":0.68999999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8460000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1830000000000001,"15min":1.22},"min":{"1min":0.86099999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.6220000000000001,"5min":1.694,"15min":1.835},"last":1.3939999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.157,"15min":1.206},"min":{"1min":0.71799999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.464,"5min":1.742,"15min":1.976},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.222,"15min":1.278},"min":{"1min":0.71699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5009999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.2909999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2829999999999999,"15min":1.3140000000000001},"min":{"1min":0.89500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6839999999999999,"5min":1.927,"15min":2.0289999999999999},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":1.0069999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0190000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.599}]},{"osd":4,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":0.93600000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.532},{"interface":"front","average":{"1min":1.589,"5min":1.407,"15min":1.367},"min":{"1min":1.2090000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.222,"5min":2.48,"15min":2.48},"last":2.1019999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.3700000000000001,"15min":1.413},"min":{"1min":1.127,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.962,"5min":2.262,"15min":2.262},"last":1.167},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5169999999999999,"15min":1.4970000000000001},"min":{"1min":1.2629999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.194,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.74}]},{"osd":7,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5029999999999999,"15min":1.5309999999999999},"min":{"1min":1.155,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1320000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.865},{"interface":"front","average":{"1min":1.746,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":1.371,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2770000000000001,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.841}]}]},{"osd":5,"up_from":13,"seq":55834575989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52576,"kb_used_data":3480,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319264,"statfs":{"total":96636764160,"available":96582926336,"internally_reserved":0,"allocated":3563520,"data_stored":2039470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1000000000000001,"15min":1.123},"min":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.7170000000000001,"5min":1.776,"15min":1.776},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1539999999999999,"15min":1.161},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.752},"last":0.97599999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.1559999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6120000000000001,"5min":1.6519999999999999,"15min":1.696},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.198,"15min":1.2330000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":1.506}]},{"osd":2,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2270000000000001,"15min":1.2470000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.81399999999999995},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.2450000000000001,"15min":1.254},"min":{"1min":0.75,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.867,"5min":1.899,"15min":1.9219999999999999},"last":1.5609999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3089999999999999,"15min":1.3149999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.889,"5min":1.889,"15min":1.8979999999999999},"last":1.175},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.329,"15min":1.359},"min":{"1min":1.0109999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.97,"5min":1.97,"15min":2.1859999999999999},"last":1.444}]},{"osd":4,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.869,"5min":1.869,"15min":2.6040000000000001},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.2470000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3779999999999999,"15min":1.377},"min":{"1min":0.99099999999999999,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.7649999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.409,"15min":1.4219999999999999},"min":{"1min":1.0149999999999999,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":2.032,"5min":2.032,"15min":2.577},"last":1.6830000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.534,"15min":1.538},"min":{"1min":1.2010000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":2.077,"5min":2.077,"15min":3.1709999999999998},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5760000000000001,"15min":1.5780000000000001},"min":{"1min":1.3200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.101,"15min":2.8740000000000001},"last":1.704}]}]},{"osd":4,"up_from":13,"seq":55834575989,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51992,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319848,"statfs":{"total":96636764160,"available":96583524352,"internally_reserved":0,"allocated":2965504,"data_stored":1449032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1200000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8089999999999999,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.198,"15min":1.1899999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5309999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.2669999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2290000000000001,"15min":1.256},"min":{"1min":0.83599999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8580000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1719999999999999,"15min":1.175},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.778,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.093}]},{"osd":2,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3300000000000001,"15min":1.244},"min":{"1min":0.74399999999999999,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.637,"5min":11.311,"15min":11.311},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3320000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9330000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.89700000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.298,"15min":1.29},"min":{"1min":0.80000000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.716,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3759999999999999,"15min":1.3340000000000001},"min":{"1min":0.96399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7430000000000001,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.357,"15min":1.3200000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.738,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.375,"15min":1.3160000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6579999999999999,"5min":3.71,"15min":3.71},"last":1.411}]},{"osd":6,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3799999999999999,"15min":1.3740000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8009999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4790000000000001,"15min":1.476},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.76,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.3939999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.514,"15min":1.488},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9830000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.597,"15min":1.575},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.907,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.5860000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575988,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56076,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315764,"statfs":{"total":96636764160,"available":96579342336,"internally_reserved":0,"allocated":2953216,"data_stored":1447468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.91300000000000003,"15min":0.90700000000000003},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.44},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1399999999999999,"15min":1.167},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.788,"5min":3.863,"15min":3.863},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.073,"15min":1.0960000000000001},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.637,"5min":3.597,"15min":3.597},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.127,"5min":1.2090000000000001,"15min":1.2410000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.8319999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.745}]},{"osd":4,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2969999999999999,"15min":1.294},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.272,"15min":1.296},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.736,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.331,"15min":1.329},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.1240000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.373,"15min":1.373},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.425,"15min":1.444},"min":{"1min":0.80500000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.5089999999999999,"15min":1.5129999999999999},"min":{"1min":1.0089999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":1.087}]},{"osd":7,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5129999999999999,"15min":1.5249999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.899,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.5209999999999999,"15min":1.526},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":1.1859999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575988,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47888,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323952,"statfs":{"total":96636764160,"available":96587726848,"internally_reserved":0,"allocated":2957312,"data_stored":1447502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.411,"5min":1.411,"15min":1.4730000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.052,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3080000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.216}]},{"osd":1,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.0229999999999999,"15min":1.0489999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.343,"5min":1.698,"15min":1.698},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.097,"15min":1.135},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.4730000000000001,"5min":1.635,"15min":1.6599999999999999},"last":1.5229999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.026,"15min":1.042},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.393,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.159,"15min":1.208},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4950000000000001,"5min":1.6879999999999999,"15min":1.79},"last":1.4970000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.319,"15min":1.327},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5700000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.621},{"interface":"front","average":{"1min":1.194,"5min":1.2689999999999999,"15min":1.294},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5189999999999999,"5min":1.714,"15min":1.835},"last":1.411}]},{"osd":5,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3560000000000001,"15min":1.3839999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.782,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.266},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.407,"15min":1.411},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.873,"5min":1.8740000000000001,"15min":2.016},"last":1.4530000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3660000000000001,"15min":1.3799999999999999},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.714,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4730000000000001,"15min":1.4970000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.353}]},{"osd":7,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.478,"15min":1.4850000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.9350000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.4259999999999999,"15min":1.454},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8149999999999999,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.5509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575987,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48476,"kb_used_data":3476,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323364,"statfs":{"total":96636764160,"available":96587124736,"internally_reserved":0,"allocated":3559424,"data_stored":2038510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.89800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.238,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.98499999999999999,"15min":1.038},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.512,"5min":1.5660000000000001,"15min":1.681},"last":1.089}]},{"osd":2,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2190000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.1020000000000001,"15min":1.141},"min":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5489999999999999,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":1.0429999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.208,"15min":1.2949999999999999},"min":{"1min":0.60799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5860000000000001,"5min":1.587,"15min":1.788},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.02,"5min":1.0780000000000001,"15min":1.127},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.383,"5min":1.714,"15min":1.8049999999999999},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2030000000000001,"15min":1.218},"min":{"1min":0.84399999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.623,"5min":1.623,"15min":1.6859999999999999},"last":1.536},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3700000000000001,"15min":1.4039999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.657,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.7450000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3109999999999999,"15min":1.333},"min":{"1min":0.95899999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":2.101},"last":1.506},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.298,"15min":1.331},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.9019999999999999,"5min":1.927,"15min":1.927},"last":1.7030000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0529999999999999,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3939999999999999,"15min":1.413},"min":{"1min":0.91600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.901}]},{"osd":7,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4650000000000001,"15min":1.522},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.9430000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.524,"5min":1.48,"15min":1.5109999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.0699999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.6060000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60172,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311668,"statfs":{"total":96636764160,"available":96575148032,"internally_reserved":0,"allocated":2953216,"data_stored":1447468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.88900000000000001},"min":{"1min":0.314,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.994},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.98899999999999999,"15min":1.048},"min":{"1min":0.58999999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6799999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0169999999999999,"15min":1.0640000000000001},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.635,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0589999999999999,"15min":1.1259999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3100000000000001,"5min":1.726,"15min":1.792},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.036,"15min":1.0669999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.8839999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.171,"15min":1.224},"min":{"1min":0.70499999999999996,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.6469999999999998},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.296,"15min":1.3520000000000001},"min":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.681,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.3029999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1240000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.90700000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.899,"15min":2.7719999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.383,"15min":1.427},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.681,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":1.1479999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3520000000000001,"15min":1.381},"min":{"1min":0.84599999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7270000000000001,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.45,"15min":1.4830000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.0660000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4670000000000001,"15min":1.516},"min":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7969999999999999,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4730000000000001,"15min":1.4870000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.927,"5min":2.1190000000000002,"15min":2.431},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:11:38.049 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-01T03:11:38.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.297+0000 7f6fbb127640 1 Processor -- start 2023-12-01T03:11:38.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.298+0000 7f6fbb127640 1 -- start start 2023-12-01T03:11:38.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.299+0000 7f6fbb127640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6fb415e2f0 0x7f6fb40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.299+0000 7f6fbb127640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb40add10 con 0x7f6fb40adf10 2023-12-01T03:11:38.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.299+0000 7f6fbb127640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb40c5760 con 0x7f6fb406f2f0 2023-12-01T03:11:38.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.299+0000 7f6fbb127640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb40c9cf0 con 0x7f6fb415e2f0 2023-12-01T03:11:38.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.299+0000 7f6fb96c4640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6fb40adf10 0x7f6fb40c95e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34714/0 (socket says 172.21.15.115:34714) 2023-12-01T03:11:38.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.299+0000 7f6fb96c4640 1 -- 172.21.15.115:0/2573416581 learned_addr learned my addr 172.21.15.115:0/2573416581 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:38.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb8ec3640 1 --2- 172.21.15.115:0/2573416581 >> v2:172.21.15.142:3300/0 conn(0x7f6fb415e2f0 0x7f6fb40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:38.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606775913 0 0) 0x7f6fb40add10 con 0x7f6fb40adf10 2023-12-01T03:11:38.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa4003520 con 0x7f6fb40adf10 2023-12-01T03:11:38.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3413706871 0 0) 0x7f6fa4003520 con 0x7f6fb40adf10 2023-12-01T03:11:38.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 >> v2:172.21.15.142:3300/0 conn(0x7f6fb415e2f0 msgr2=0x7f6fb40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:38.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 --2- 172.21.15.115:0/2573416581 >> v2:172.21.15.142:3300/0 conn(0x7f6fb415e2f0 0x7f6fb40ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 >> v1:172.21.15.142:6789/0 conn(0x7f6fb406f2f0 legacy=0x7f6fb40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb415e6d0 con 0x7f6fb40adf10 2023-12-01T03:11:38.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6fac0019f0 con 0x7f6fb40adf10 2023-12-01T03:11:38.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 >> v1:172.21.15.115:6789/0 conn(0x7f6fb40adf10 legacy=0x7f6fb40c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fa40039e0 0x7f6fa4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fa40047c0 0x7f6fa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.300+0000 7f6fb37fe640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa4005280 0x7f6fa400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40c9cf0 con 0x7f6fa40047c0 2023-12-01T03:11:38.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa400c1c0 con 0x7f6fa4005280 2023-12-01T03:11:38.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa400c380 con 0x7f6fa40039e0 2023-12-01T03:11:38.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb3fff640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fa40047c0 0x7f6fa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:38.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb8ec3640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fa40039e0 0x7f6fa4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:38.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb96c4640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa4005280 0x7f6fa400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:38.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb3fff640 1 -- 172.21.15.115:0/2573416581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fa40039e0 msgr2=0x7f6fa4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:38.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb3fff640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fa40039e0 0x7f6fa4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb3fff640 1 -- 172.21.15.115:0/2573416581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa4005280 msgr2=0x7f6fa400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:38.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb3fff640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa4005280 0x7f6fa400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb3fff640 1 -- 172.21.15.115:0/2573416581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4003520 con 0x7f6fa40047c0 2023-12-01T03:11:38.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb96c4640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa4005280 0x7f6fa400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:38.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.301+0000 7f6fb96c4640 1 -- 172.21.15.115:0/2573416581 reap_dead start 2023-12-01T03:11:38.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.302+0000 7f6fb3fff640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fa40047c0 0x7f6fa4004c40 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0001750 tx=0x7f6fa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa30bacec1b1bf9b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:38.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.303+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00052d0 con 0x7f6fa40047c0 2023-12-01T03:11:38.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.303+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa002e050 con 0x7f6fa40047c0 2023-12-01T03:11:38.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.303+0000 7f6fb37fe640 1 -- 172.21.15.115:0/2573416581 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0003830 con 0x7f6fa40047c0 2023-12-01T03:11:38.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.303+0000 7f6fbb127640 1 -- 172.21.15.115:0/2573416581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fa40047c0 msgr2=0x7f6fa4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.303+0000 7f6fbb127640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fa40047c0 0x7f6fa4004c40 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0001750 tx=0x7f6fa40057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.303+0000 7f6fbb127640 1 -- 172.21.15.115:0/2573416581 shutdown_connections 2023-12-01T03:11:38.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.303+0000 7f6fbb127640 1 --2- 172.21.15.115:0/2573416581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fa40047c0 0x7f6fa4004c40 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.303+0000 7f6fbb127640 1 -- 172.21.15.115:0/2573416581 >> 172.21.15.115:0/2573416581 conn(0x7f6fb40ce880 msgr2=0x7f6fb40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:38.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.303+0000 7f6fbb127640 1 -- 172.21.15.115:0/2573416581 shutdown_connections 2023-12-01T03:11:38.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.303+0000 7f6fbb127640 1 -- 172.21.15.115:0/2573416581 wait complete. 2023-12-01T03:11:38.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.305+0000 7f6fbb127640 1 Processor -- start 2023-12-01T03:11:38.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.305+0000 7f6fbb127640 1 -- start start 2023-12-01T03:11:38.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.306+0000 7f6fbb127640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa40047c0 0x7f6fb41f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.306+0000 7f6fbb127640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb40cdec0 0x7f6fb40c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.306+0000 7f6fbb127640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb40c28f0 0x7f6fb40c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.306+0000 7f6fbb127640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40b1c40 con 0x7f6fb40cdec0 2023-12-01T03:11:38.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.306+0000 7f6fbb127640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40c2d90 con 0x7f6fa40047c0 2023-12-01T03:11:38.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.306+0000 7f6fbb127640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40c3070 con 0x7f6fb40c28f0 2023-12-01T03:11:38.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.306+0000 7f6fb3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb40cdec0 0x7f6fb40c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:38.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb3fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb40cdec0 0x7f6fb40c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37544/0 (socket says 172.21.15.115:37544) 2023-12-01T03:11:38.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb3fff640 1 -- 172.21.15.115:0/2684507702 learned_addr learned my addr 172.21.15.115:0/2684507702 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:38.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb3fff640 1 -- 172.21.15.115:0/2684507702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb40c28f0 msgr2=0x7f6fb40c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb3fff640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb40c28f0 0x7f6fb40c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb3fff640 1 -- 172.21.15.115:0/2684507702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa40047c0 msgr2=0x7f6fb41f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb3fff640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa40047c0 0x7f6fb41f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb3fff640 1 -- 172.21.15.115:0/2684507702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40b1860 con 0x7f6fb40cdec0 2023-12-01T03:11:38.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb3fff640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb40cdec0 0x7f6fb40c23b0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f6fa002bc00 tx=0x7f6fa0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:38.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb17fa640 1 -- 172.21.15.115:0/2684507702 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0007640 con 0x7f6fb40cdec0 2023-12-01T03:11:38.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb17fa640 1 -- 172.21.15.115:0/2684507702 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa00069e0 con 0x7f6fb40cdec0 2023-12-01T03:11:38.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fb17fa640 1 -- 172.21.15.115:0/2684507702 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0006d70 con 0x7f6fb40cdec0 2023-12-01T03:11:38.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.307+0000 7f6fbb127640 1 -- 172.21.15.115:0/2684507702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb415acd0 con 0x7f6fb40cdec0 2023-12-01T03:11:38.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.308+0000 7f6fbb127640 1 -- 172.21.15.115:0/2684507702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb40c3f60 con 0x7f6fb40cdec0 2023-12-01T03:11:38.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.309+0000 7f6fbb127640 1 -- 172.21.15.115:0/2684507702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb406fda0 con 0x7f6fb40cdec0 2023-12-01T03:11:38.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.310+0000 7f6fb17fa640 1 -- 172.21.15.115:0/2684507702 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f6fa00035f0 con 0x7f6fb40cdec0 2023-12-01T03:11:38.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.311+0000 7f6fb17fa640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6f9806a670 0x7f6f9806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.311+0000 7f6fb17fa640 1 -- 172.21.15.115:0/2684507702 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f6fa00aab00 con 0x7f6fb40cdec0 2023-12-01T03:11:38.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.311+0000 7f6fb8ec3640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6f9806a670 0x7f6f9806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:38.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.311+0000 7f6fb8ec3640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6f9806a670 0x7f6f9806cb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009990 tx=0x7f6fa8008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:38.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.315+0000 7f6fb17fa640 1 -- 172.21.15.115:0/2684507702 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6fa0075d20 con 0x7f6fb40cdec0 2023-12-01T03:11:38.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.576+0000 7f6fbb127640 1 -- 172.21.15.115:0/2684507702 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6fb406bc80 con 0x7f6fb40cdec0 2023-12-01T03:11:38.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.577+0000 7f6fb17fa640 1 -- 172.21.15.115:0/2684507702 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3213 (secure 0 0 0) 0x7f6fa004b080 con 0x7f6fb40cdec0 2023-12-01T03:11:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:38.579 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5753,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":235,"num_osds":8,"num_up_osds":8,"osd_up_since":1701394558,"num_in_osds":8,"osd_in_since":1701394549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":429563904,"bytes_avail":772664549376,"bytes_total":773094113280,"write_bytes_sec":2318,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":110,"id":14,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":14,"rank":0,"name":"d","status":"up:active","gid":7735}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2023-12-01T03:11:19.817508+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-01T03:11:38.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 -- 172.21.15.115:0/2684507702 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6f9806a670 msgr2=0x7f6f9806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6f9806a670 0x7f6f9806cb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009990 tx=0x7f6fa8008040 comp rx=0 tx=0).stop 2023-12-01T03:11:38.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 -- 172.21.15.115:0/2684507702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb40cdec0 msgr2=0x7f6fb40c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb40cdec0 0x7f6fb40c23b0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f6fa002bc00 tx=0x7f6fa0001120 comp rx=0 tx=0).stop 2023-12-01T03:11:38.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 -- 172.21.15.115:0/2684507702 shutdown_connections 2023-12-01T03:11:38.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6f9806a670 0x7f6f9806cb30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb40c28f0 0x7f6fb40c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6fb40cdec0 0x7f6fb40c23b0 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 --2- 172.21.15.115:0/2684507702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fa40047c0 0x7f6fb41f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 -- 172.21.15.115:0/2684507702 >> 172.21.15.115:0/2684507702 conn(0x7f6fb40ce880 msgr2=0x7f6fb406bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:38.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.580+0000 7f6fbb127640 1 -- 172.21.15.115:0/2684507702 shutdown_connections 2023-12-01T03:11:38.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.581+0000 7f6fbb127640 1 -- 172.21.15.115:0/2684507702 wait complete. 2023-12-01T03:11:38.599 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-01T03:11:38.599 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:11:38.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.805+0000 7faf3fa0b640 1 Processor -- start 2023-12-01T03:11:38.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.805+0000 7faf3fa0b640 1 -- start start 2023-12-01T03:11:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.806+0000 7faf3fa0b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3815e2f0 0x7faf380ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.806+0000 7faf3fa0b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faf380adf10 0x7faf380caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.806+0000 7faf3fa0b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf380add10 con 0x7faf3806f2f0 2023-12-01T03:11:38.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.806+0000 7faf3fa0b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf380c5760 con 0x7faf380adf10 2023-12-01T03:11:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.806+0000 7faf3fa0b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815e6d0 con 0x7faf3815e2f0 2023-12-01T03:11:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.807+0000 7faf3cfa6640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7faf3806f2f0 0x7faf380c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34728/0 (socket says 172.21.15.115:34728) 2023-12-01T03:11:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.807+0000 7faf3cfa6640 1 -- 172.21.15.115:0/90745300 learned_addr learned my addr 172.21.15.115:0/90745300 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.807+0000 7faf3dfa8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faf380adf10 0x7faf380caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.807+0000 7faf3d7a7640 1 --2- 172.21.15.115:0/90745300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3815e2f0 0x7faf380ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:38.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.807+0000 7faf3dfa8640 1 -- 172.21.15.115:0/90745300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3815e2f0 msgr2=0x7faf380ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.807+0000 7faf3dfa8640 1 --2- 172.21.15.115:0/90745300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3815e2f0 0x7faf380ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.807+0000 7faf3dfa8640 1 -- 172.21.15.115:0/90745300 >> v1:172.21.15.115:6789/0 conn(0x7faf3806f2f0 legacy=0x7faf380c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.807+0000 7faf2ffff640 1 -- 172.21.15.115:0/90745300 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690429858 0 0) 0x7faf380add10 con 0x7faf3806f2f0 2023-12-01T03:11:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.807+0000 7faf3dfa8640 1 -- 172.21.15.115:0/90745300 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380cf7b0 con 0x7faf380adf10 2023-12-01T03:11:38.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.809+0000 7faf3dfa8640 1 --2- 172.21.15.115:0/90745300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380adf10 0x7faf380caba0 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7faf30009d90 tx=0x7faf3002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8753a1f66a104b1c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.809+0000 7faf2ffff640 1 -- 172.21.15.115:0/90745300 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3002cb40 con 0x7faf380adf10 2023-12-01T03:11:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.809+0000 7faf2ffff640 1 -- 172.21.15.115:0/90745300 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3002cca0 con 0x7faf380adf10 2023-12-01T03:11:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.809+0000 7faf2ffff640 1 -- 172.21.15.115:0/90745300 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30034660 con 0x7faf380adf10 2023-12-01T03:11:38.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.809+0000 7faf3fa0b640 1 -- 172.21.15.115:0/90745300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380adf10 msgr2=0x7faf380caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.809+0000 7faf3fa0b640 1 --2- 172.21.15.115:0/90745300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380adf10 0x7faf380caba0 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7faf30009d90 tx=0x7faf3002bd90 comp rx=0 tx=0).stop 2023-12-01T03:11:38.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.810+0000 7faf3fa0b640 1 -- 172.21.15.115:0/90745300 shutdown_connections 2023-12-01T03:11:38.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.810+0000 7faf3fa0b640 1 --2- 172.21.15.115:0/90745300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380adf10 0x7faf380caba0 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.810+0000 7faf3fa0b640 1 --2- 172.21.15.115:0/90745300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3815e2f0 0x7faf380ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.810+0000 7faf3fa0b640 1 -- 172.21.15.115:0/90745300 >> 172.21.15.115:0/90745300 conn(0x7faf380ce880 msgr2=0x7faf380cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:38.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.810+0000 7faf3fa0b640 1 -- 172.21.15.115:0/90745300 shutdown_connections 2023-12-01T03:11:38.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.810+0000 7faf3fa0b640 1 -- 172.21.15.115:0/90745300 wait complete. 2023-12-01T03:11:38.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.811+0000 7faf3fa0b640 1 Processor -- start 2023-12-01T03:11:38.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.812+0000 7faf3fa0b640 1 -- start start 2023-12-01T03:11:38.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.812+0000 7faf3fa0b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3806f2f0 0x7faf380b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.812+0000 7faf3fa0b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf380adf10 0x7faf380c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.812+0000 7faf3fa0b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf3815e2f0 0x7faf380c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.812+0000 7faf3fa0b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815d5f0 con 0x7faf3815e2f0 2023-12-01T03:11:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.812+0000 7faf3fa0b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815d8d0 con 0x7faf380adf10 2023-12-01T03:11:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.812+0000 7faf3fa0b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380c6a00 con 0x7faf3806f2f0 2023-12-01T03:11:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.813+0000 7faf3dfa8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf3815e2f0 0x7faf380c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.813+0000 7faf3dfa8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf3815e2f0 0x7faf380c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37562/0 (socket says 172.21.15.115:37562) 2023-12-01T03:11:38.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.813+0000 7faf3dfa8640 1 -- 172.21.15.115:0/189261447 learned_addr learned my addr 172.21.15.115:0/189261447 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.813+0000 7faf3dfa8640 1 -- 172.21.15.115:0/189261447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3806f2f0 msgr2=0x7faf380b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.813+0000 7faf3dfa8640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3806f2f0 0x7faf380b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.813+0000 7faf3dfa8640 1 -- 172.21.15.115:0/189261447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf380adf10 msgr2=0x7faf380c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.813+0000 7faf3dfa8640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf380adf10 0x7faf380c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.813+0000 7faf3dfa8640 1 -- 172.21.15.115:0/189261447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf30009a70 con 0x7faf3815e2f0 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.813+0000 7faf3dfa8640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf3815e2f0 0x7faf380c23b0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7faf30009d90 tx=0x7faf30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.814+0000 7faf2dffb640 1 -- 172.21.15.115:0/189261447 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30040650 con 0x7faf3815e2f0 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.814+0000 7faf2dffb640 1 -- 172.21.15.115:0/189261447 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf300407d0 con 0x7faf3815e2f0 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.814+0000 7faf2dffb640 1 -- 172.21.15.115:0/189261447 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30040b80 con 0x7faf3815e2f0 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.814+0000 7faf3fa0b640 1 -- 172.21.15.115:0/189261447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf380c6ce0 con 0x7faf3815e2f0 2023-12-01T03:11:38.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.814+0000 7faf3fa0b640 1 -- 172.21.15.115:0/189261447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf380c66b0 con 0x7faf3815e2f0 2023-12-01T03:11:38.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.815+0000 7faf3fa0b640 1 -- 172.21.15.115:0/189261447 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3806fda0 con 0x7faf3815e2f0 2023-12-01T03:11:38.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.816+0000 7faf2dffb640 1 -- 172.21.15.115:0/189261447 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7faf30040ce0 con 0x7faf3815e2f0 2023-12-01T03:11:38.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.817+0000 7faf2dffb640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1c06a670 0x7faf1c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:38.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.817+0000 7faf2dffb640 1 -- 172.21.15.115:0/189261447 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7faf300abc00 con 0x7faf3815e2f0 2023-12-01T03:11:38.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.817+0000 7faf3d7a7640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1c06a670 0x7faf1c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:38.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.818+0000 7faf3d7a7640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1c06a670 0x7faf1c06cb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf34006fd0 tx=0x7faf34008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:38.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:38.821+0000 7faf2dffb640 1 -- 172.21.15.115:0/189261447 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faf30076e50 con 0x7faf3815e2f0 2023-12-01T03:11:39.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.008+0000 7faf3fa0b640 1 -- 172.21.15.115:0/189261447 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf380b43d0 con 0x7faf1c06a670 2023-12-01T03:11:39.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.031+0000 7faf2dffb640 1 -- 172.21.15.115:0/189261447 <== mgr.11121 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287573 (secure 0 0 0) 0x7faf3400be20 con 0x7faf1c06a670 2023-12-01T03:11:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:39.037 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.036+0000 7faf3fa0b640 1 -- 172.21.15.115:0/189261447 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1c06a670 msgr2=0x7faf1c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.036+0000 7faf3fa0b640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1c06a670 0x7faf1c06cb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf34006fd0 tx=0x7faf34008040 comp rx=0 tx=0).stop 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.036+0000 7faf3fa0b640 1 -- 172.21.15.115:0/189261447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf3815e2f0 msgr2=0x7faf380c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.036+0000 7faf3fa0b640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf3815e2f0 0x7faf380c23b0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7faf30009d90 tx=0x7faf30004a40 comp rx=0 tx=0).stop 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.036+0000 7faf3fa0b640 1 -- 172.21.15.115:0/189261447 shutdown_connections 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.037+0000 7faf3fa0b640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1c06a670 0x7faf1c06cb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.037+0000 7faf3fa0b640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf3815e2f0 0x7faf380c23b0 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.037+0000 7faf3fa0b640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf380adf10 0x7faf380c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.037+0000 7faf3fa0b640 1 --2- 172.21.15.115:0/189261447 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf3806f2f0 0x7faf380b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.037+0000 7faf3fa0b640 1 -- 172.21.15.115:0/189261447 >> 172.21.15.115:0/189261447 conn(0x7faf380ce880 msgr2=0x7faf380ca330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.037+0000 7faf3fa0b640 1 -- 172.21.15.115:0/189261447 shutdown_connections 2023-12-01T03:11:39.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:39.037+0000 7faf3fa0b640 1 -- 172.21.15.115:0/189261447 wait complete. 2023-12-01T03:11:39.057 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-12-01T03:11:37.822769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":623,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":419496,"kb_used_data":24296,"kb_used_omap":0,"kb_used_meta":395136,"kb_avail":754555224,"statfs":{"total":773094113280,"available":772664549376,"internally_reserved":0,"allocated":24879104,"data_stored":12765204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":404619264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.183091"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222918+0000","last_change":"2023-12-01T03:11:32.212603+0000","last_active":"2023-12-01T03:11:32.222918+0000","last_peered":"2023-12-01T03:11:32.222918+0000","last_clean":"2023-12-01T03:11:32.222918+0000","last_became_active":"2023-12-01T03:11:32.212287+0000","last_became_peered":"2023-12-01T03:11:32.212287+0000","last_unstale":"2023-12-01T03:11:32.222918+0000","last_undegraded":"2023-12-01T03:11:32.222918+0000","last_fullsized":"2023-12-01T03:11:32.222918+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220449+0000","last_change":"2023-12-01T03:11:32.217098+0000","last_active":"2023-12-01T03:11:32.220449+0000","last_peered":"2023-12-01T03:11:32.220449+0000","last_clean":"2023-12-01T03:11:32.220449+0000","last_became_active":"2023-12-01T03:11:32.216867+0000","last_became_peered":"2023-12-01T03:11:32.216867+0000","last_unstale":"2023-12-01T03:11:32.220449+0000","last_undegraded":"2023-12-01T03:11:32.220449+0000","last_fullsized":"2023-12-01T03:11:32.220449+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:10.849908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210187+0000","last_change":"2023-12-01T03:11:32.208423+0000","last_active":"2023-12-01T03:11:32.210187+0000","last_peered":"2023-12-01T03:11:32.210187+0000","last_clean":"2023-12-01T03:11:32.210187+0000","last_became_active":"2023-12-01T03:11:32.208113+0000","last_became_peered":"2023-12-01T03:11:32.208113+0000","last_unstale":"2023-12-01T03:11:32.210187+0000","last_undegraded":"2023-12-01T03:11:32.210187+0000","last_fullsized":"2023-12-01T03:11:32.210187+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:38.174827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213451+0000","last_change":"2023-12-01T03:11:32.207840+0000","last_active":"2023-12-01T03:11:32.213451+0000","last_peered":"2023-12-01T03:11:32.213451+0000","last_clean":"2023-12-01T03:11:32.213451+0000","last_became_active":"2023-12-01T03:11:32.207048+0000","last_became_peered":"2023-12-01T03:11:32.207048+0000","last_unstale":"2023-12-01T03:11:32.213451+0000","last_undegraded":"2023-12-01T03:11:32.213451+0000","last_fullsized":"2023-12-01T03:11:32.213451+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:43.961392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212780+0000","last_change":"2023-12-01T03:11:32.206960+0000","last_active":"2023-12-01T03:11:32.212780+0000","last_peered":"2023-12-01T03:11:32.212780+0000","last_clean":"2023-12-01T03:11:32.212780+0000","last_became_active":"2023-12-01T03:11:32.203345+0000","last_became_peered":"2023-12-01T03:11:32.203345+0000","last_unstale":"2023-12-01T03:11:32.212780+0000","last_undegraded":"2023-12-01T03:11:32.212780+0000","last_fullsized":"2023-12-01T03:11:32.212780+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211014+0000","last_change":"2023-12-01T03:11:32.205496+0000","last_active":"2023-12-01T03:11:32.211014+0000","last_peered":"2023-12-01T03:11:32.211014+0000","last_clean":"2023-12-01T03:11:32.211014+0000","last_became_active":"2023-12-01T03:11:32.205074+0000","last_became_peered":"2023-12-01T03:11:32.205074+0000","last_unstale":"2023-12-01T03:11:32.211014+0000","last_undegraded":"2023-12-01T03:11:32.211014+0000","last_fullsized":"2023-12-01T03:11:32.211014+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209538+0000","last_change":"2023-12-01T03:11:32.202465+0000","last_active":"2023-12-01T03:11:32.209538+0000","last_peered":"2023-12-01T03:11:32.209538+0000","last_clean":"2023-12-01T03:11:32.209538+0000","last_became_active":"2023-12-01T03:11:32.200165+0000","last_became_peered":"2023-12-01T03:11:32.200165+0000","last_unstale":"2023-12-01T03:11:32.209538+0000","last_undegraded":"2023-12-01T03:11:32.209538+0000","last_fullsized":"2023-12-01T03:11:32.209538+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:19.277681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220081+0000","last_change":"2023-12-01T03:11:32.217253+0000","last_active":"2023-12-01T03:11:32.220081+0000","last_peered":"2023-12-01T03:11:32.220081+0000","last_clean":"2023-12-01T03:11:32.220081+0000","last_became_active":"2023-12-01T03:11:32.216425+0000","last_became_peered":"2023-12-01T03:11:32.216425+0000","last_unstale":"2023-12-01T03:11:32.220081+0000","last_undegraded":"2023-12-01T03:11:32.220081+0000","last_fullsized":"2023-12-01T03:11:32.220081+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:51.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.214233+0000","last_change":"2023-12-01T03:11:32.211881+0000","last_active":"2023-12-01T03:11:32.214233+0000","last_peered":"2023-12-01T03:11:32.214233+0000","last_clean":"2023-12-01T03:11:32.214233+0000","last_became_active":"2023-12-01T03:11:32.211602+0000","last_became_peered":"2023-12-01T03:11:32.211602+0000","last_unstale":"2023-12-01T03:11:32.214233+0000","last_undegraded":"2023-12-01T03:11:32.214233+0000","last_fullsized":"2023-12-01T03:11:32.214233+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.201292+0000","last_change":"2023-12-01T03:11:32.200099+0000","last_active":"2023-12-01T03:11:32.201292+0000","last_peered":"2023-12-01T03:11:32.201292+0000","last_clean":"2023-12-01T03:11:32.201292+0000","last_became_active":"2023-12-01T03:11:32.199703+0000","last_became_peered":"2023-12-01T03:11:32.199703+0000","last_unstale":"2023-12-01T03:11:32.201292+0000","last_undegraded":"2023-12-01T03:11:32.201292+0000","last_fullsized":"2023-12-01T03:11:32.201292+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:44:42.356924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213688+0000","last_change":"2023-12-01T03:11:32.211517+0000","last_active":"2023-12-01T03:11:32.213688+0000","last_peered":"2023-12-01T03:11:32.213688+0000","last_clean":"2023-12-01T03:11:32.213688+0000","last_became_active":"2023-12-01T03:11:32.211273+0000","last_became_peered":"2023-12-01T03:11:32.211273+0000","last_unstale":"2023-12-01T03:11:32.213688+0000","last_undegraded":"2023-12-01T03:11:32.213688+0000","last_fullsized":"2023-12-01T03:11:32.213688+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:35.627224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.206460+0000","last_change":"2023-12-01T03:11:32.202825+0000","last_active":"2023-12-01T03:11:32.206460+0000","last_peered":"2023-12-01T03:11:32.206460+0000","last_clean":"2023-12-01T03:11:32.206460+0000","last_became_active":"2023-12-01T03:11:32.202452+0000","last_became_peered":"2023-12-01T03:11:32.202452+0000","last_unstale":"2023-12-01T03:11:32.206460+0000","last_undegraded":"2023-12-01T03:11:32.206460+0000","last_fullsized":"2023-12-01T03:11:32.206460+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225985+0000","last_change":"2023-12-01T03:11:32.224888+0000","last_active":"2023-12-01T03:11:32.225985+0000","last_peered":"2023-12-01T03:11:32.225985+0000","last_clean":"2023-12-01T03:11:32.225985+0000","last_became_active":"2023-12-01T03:11:32.224585+0000","last_became_peered":"2023-12-01T03:11:32.224585+0000","last_unstale":"2023-12-01T03:11:32.225985+0000","last_undegraded":"2023-12-01T03:11:32.225985+0000","last_fullsized":"2023-12-01T03:11:32.225985+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:04.079396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210154+0000","last_change":"2023-12-01T03:11:32.208938+0000","last_active":"2023-12-01T03:11:32.210154+0000","last_peered":"2023-12-01T03:11:32.210154+0000","last_clean":"2023-12-01T03:11:32.210154+0000","last_became_active":"2023-12-01T03:11:32.208695+0000","last_became_peered":"2023-12-01T03:11:32.208695+0000","last_unstale":"2023-12-01T03:11:32.210154+0000","last_undegraded":"2023-12-01T03:11:32.210154+0000","last_fullsized":"2023-12-01T03:11:32.210154+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:21.336618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218399+0000","last_change":"2023-12-01T03:11:32.214557+0000","last_active":"2023-12-01T03:11:32.218399+0000","last_peered":"2023-12-01T03:11:32.218399+0000","last_clean":"2023-12-01T03:11:32.218399+0000","last_became_active":"2023-12-01T03:11:32.214252+0000","last_became_peered":"2023-12-01T03:11:32.214252+0000","last_unstale":"2023-12-01T03:11:32.218399+0000","last_undegraded":"2023-12-01T03:11:32.218399+0000","last_fullsized":"2023-12-01T03:11:32.218399+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218473+0000","last_change":"2023-12-01T03:11:32.212018+0000","last_active":"2023-12-01T03:11:32.218473+0000","last_peered":"2023-12-01T03:11:32.218473+0000","last_clean":"2023-12-01T03:11:32.218473+0000","last_became_active":"2023-12-01T03:11:32.211742+0000","last_became_peered":"2023-12-01T03:11:32.211742+0000","last_unstale":"2023-12-01T03:11:32.218473+0000","last_undegraded":"2023-12-01T03:11:32.218473+0000","last_fullsized":"2023-12-01T03:11:32.218473+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222963+0000","last_change":"2023-12-01T03:11:32.219580+0000","last_active":"2023-12-01T03:11:32.222963+0000","last_peered":"2023-12-01T03:11:32.222963+0000","last_clean":"2023-12-01T03:11:32.222963+0000","last_became_active":"2023-12-01T03:11:32.218935+0000","last_became_peered":"2023-12-01T03:11:32.218935+0000","last_unstale":"2023-12-01T03:11:32.222963+0000","last_undegraded":"2023-12-01T03:11:32.222963+0000","last_fullsized":"2023-12-01T03:11:32.222963+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223790+0000","last_change":"2023-12-01T03:11:32.222587+0000","last_active":"2023-12-01T03:11:32.223790+0000","last_peered":"2023-12-01T03:11:32.223790+0000","last_clean":"2023-12-01T03:11:32.223790+0000","last_became_active":"2023-12-01T03:11:32.222267+0000","last_became_peered":"2023-12-01T03:11:32.222267+0000","last_unstale":"2023-12-01T03:11:32.223790+0000","last_undegraded":"2023-12-01T03:11:32.223790+0000","last_fullsized":"2023-12-01T03:11:32.223790+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:42.720564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210046+0000","last_change":"2023-12-01T03:11:32.204026+0000","last_active":"2023-12-01T03:11:32.210046+0000","last_peered":"2023-12-01T03:11:32.210046+0000","last_clean":"2023-12-01T03:11:32.210046+0000","last_became_active":"2023-12-01T03:11:32.200832+0000","last_became_peered":"2023-12-01T03:11:32.200832+0000","last_unstale":"2023-12-01T03:11:32.210046+0000","last_undegraded":"2023-12-01T03:11:32.210046+0000","last_fullsized":"2023-12-01T03:11:32.210046+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.215183+0000","last_change":"2023-12-01T03:11:32.207642+0000","last_active":"2023-12-01T03:11:32.215183+0000","last_peered":"2023-12-01T03:11:32.215183+0000","last_clean":"2023-12-01T03:11:32.215183+0000","last_became_active":"2023-12-01T03:11:32.207344+0000","last_became_peered":"2023-12-01T03:11:32.207344+0000","last_unstale":"2023-12-01T03:11:32.215183+0000","last_undegraded":"2023-12-01T03:11:32.215183+0000","last_fullsized":"2023-12-01T03:11:32.215183+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:10.441679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223757+0000","last_change":"2023-12-01T03:11:32.220696+0000","last_active":"2023-12-01T03:11:32.223757+0000","last_peered":"2023-12-01T03:11:32.223757+0000","last_clean":"2023-12-01T03:11:32.223757+0000","last_became_active":"2023-12-01T03:11:32.220353+0000","last_became_peered":"2023-12-01T03:11:32.220353+0000","last_unstale":"2023-12-01T03:11:32.223757+0000","last_undegraded":"2023-12-01T03:11:32.223757+0000","last_fullsized":"2023-12-01T03:11:32.223757+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:53:43.178969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211636+0000","last_change":"2023-12-01T03:11:32.207709+0000","last_active":"2023-12-01T03:11:32.211636+0000","last_peered":"2023-12-01T03:11:32.211636+0000","last_clean":"2023-12-01T03:11:32.211636+0000","last_became_active":"2023-12-01T03:11:32.205510+0000","last_became_peered":"2023-12-01T03:11:32.205510+0000","last_unstale":"2023-12-01T03:11:32.211636+0000","last_undegraded":"2023-12-01T03:11:32.211636+0000","last_fullsized":"2023-12-01T03:11:32.211636+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216610+0000","last_change":"2023-12-01T03:11:32.210643+0000","last_active":"2023-12-01T03:11:32.216610+0000","last_peered":"2023-12-01T03:11:32.216610+0000","last_clean":"2023-12-01T03:11:32.216610+0000","last_became_active":"2023-12-01T03:11:32.210367+0000","last_became_peered":"2023-12-01T03:11:32.210367+0000","last_unstale":"2023-12-01T03:11:32.216610+0000","last_undegraded":"2023-12-01T03:11:32.216610+0000","last_fullsized":"2023-12-01T03:11:32.216610+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:00.674458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.214317+0000","last_change":"2023-12-01T03:11:32.209242+0000","last_active":"2023-12-01T03:11:32.214317+0000","last_peered":"2023-12-01T03:11:32.214317+0000","last_clean":"2023-12-01T03:11:32.214317+0000","last_became_active":"2023-12-01T03:11:32.208985+0000","last_became_peered":"2023-12-01T03:11:32.208985+0000","last_unstale":"2023-12-01T03:11:32.214317+0000","last_undegraded":"2023-12-01T03:11:32.214317+0000","last_fullsized":"2023-12-01T03:11:32.214317+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:29.661974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223614+0000","last_change":"2023-12-01T03:11:32.220828+0000","last_active":"2023-12-01T03:11:32.223614+0000","last_peered":"2023-12-01T03:11:32.223614+0000","last_clean":"2023-12-01T03:11:32.223614+0000","last_became_active":"2023-12-01T03:11:32.219607+0000","last_became_peered":"2023-12-01T03:11:32.219607+0000","last_unstale":"2023-12-01T03:11:32.223614+0000","last_undegraded":"2023-12-01T03:11:32.223614+0000","last_fullsized":"2023-12-01T03:11:32.223614+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:09.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.205877+0000","last_change":"2023-12-01T03:11:32.198609+0000","last_active":"2023-12-01T03:11:32.205877+0000","last_peered":"2023-12-01T03:11:32.205877+0000","last_clean":"2023-12-01T03:11:32.205877+0000","last_became_active":"2023-12-01T03:11:32.197869+0000","last_became_peered":"2023-12-01T03:11:32.197869+0000","last_unstale":"2023-12-01T03:11:32.205877+0000","last_undegraded":"2023-12-01T03:11:32.205877+0000","last_fullsized":"2023-12-01T03:11:32.205877+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:35.975600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217603+0000","last_change":"2023-12-01T03:11:32.212954+0000","last_active":"2023-12-01T03:11:32.217603+0000","last_peered":"2023-12-01T03:11:32.217603+0000","last_clean":"2023-12-01T03:11:32.217603+0000","last_became_active":"2023-12-01T03:11:32.212678+0000","last_became_peered":"2023-12-01T03:11:32.212678+0000","last_unstale":"2023-12-01T03:11:32.217603+0000","last_undegraded":"2023-12-01T03:11:32.217603+0000","last_fullsized":"2023-12-01T03:11:32.217603+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:28.852429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220068+0000","last_change":"2023-12-01T03:11:32.215698+0000","last_active":"2023-12-01T03:11:32.220068+0000","last_peered":"2023-12-01T03:11:32.220068+0000","last_clean":"2023-12-01T03:11:32.220068+0000","last_became_active":"2023-12-01T03:11:32.214531+0000","last_became_peered":"2023-12-01T03:11:32.214531+0000","last_unstale":"2023-12-01T03:11:32.220068+0000","last_undegraded":"2023-12-01T03:11:32.220068+0000","last_fullsized":"2023-12-01T03:11:32.220068+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:14.716448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212370+0000","last_change":"2023-12-01T03:11:32.204327+0000","last_active":"2023-12-01T03:11:32.212370+0000","last_peered":"2023-12-01T03:11:32.212370+0000","last_clean":"2023-12-01T03:11:32.212370+0000","last_became_active":"2023-12-01T03:11:32.203765+0000","last_became_peered":"2023-12-01T03:11:32.203765+0000","last_unstale":"2023-12-01T03:11:32.212370+0000","last_undegraded":"2023-12-01T03:11:32.212370+0000","last_fullsized":"2023-12-01T03:11:32.212370+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:28:34.324085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213777+0000","last_change":"2023-12-01T03:11:32.212023+0000","last_active":"2023-12-01T03:11:32.213777+0000","last_peered":"2023-12-01T03:11:32.213777+0000","last_clean":"2023-12-01T03:11:32.213777+0000","last_became_active":"2023-12-01T03:11:32.210975+0000","last_became_peered":"2023-12-01T03:11:32.210975+0000","last_unstale":"2023-12-01T03:11:32.213777+0000","last_undegraded":"2023-12-01T03:11:32.213777+0000","last_fullsized":"2023-12-01T03:11:32.213777+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217256+0000","last_change":"2023-12-01T03:11:32.213074+0000","last_active":"2023-12-01T03:11:32.217256+0000","last_peered":"2023-12-01T03:11:32.217256+0000","last_clean":"2023-12-01T03:11:32.217256+0000","last_became_active":"2023-12-01T03:11:32.212775+0000","last_became_peered":"2023-12-01T03:11:32.212775+0000","last_unstale":"2023-12-01T03:11:32.217256+0000","last_undegraded":"2023-12-01T03:11:32.217256+0000","last_fullsized":"2023-12-01T03:11:32.217256+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:36.571608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225974+0000","last_change":"2023-12-01T03:11:32.225479+0000","last_active":"2023-12-01T03:11:32.225974+0000","last_peered":"2023-12-01T03:11:32.225974+0000","last_clean":"2023-12-01T03:11:32.225974+0000","last_became_active":"2023-12-01T03:11:32.225154+0000","last_became_peered":"2023-12-01T03:11:32.225154+0000","last_unstale":"2023-12-01T03:11:32.225974+0000","last_undegraded":"2023-12-01T03:11:32.225974+0000","last_fullsized":"2023-12-01T03:11:32.225974+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:14.760284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220523+0000","last_change":"2023-12-01T03:11:32.214972+0000","last_active":"2023-12-01T03:11:32.220523+0000","last_peered":"2023-12-01T03:11:32.220523+0000","last_clean":"2023-12-01T03:11:32.220523+0000","last_became_active":"2023-12-01T03:11:32.214729+0000","last_became_peered":"2023-12-01T03:11:32.214729+0000","last_unstale":"2023-12-01T03:11:32.220523+0000","last_undegraded":"2023-12-01T03:11:32.220523+0000","last_fullsized":"2023-12-01T03:11:32.220523+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:24.244094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1f","version":"234'4","reported_seq":15,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225296+0000","last_change":"2023-12-01T03:11:31.680864+0000","last_active":"2023-12-01T03:11:32.225296+0000","last_peered":"2023-12-01T03:11:32.225296+0000","last_clean":"2023-12-01T03:11:32.225296+0000","last_became_active":"2023-12-01T03:11:31.680279+0000","last_became_peered":"2023-12-01T03:11:31.680279+0000","last_unstale":"2023-12-01T03:11:32.225296+0000","last_undegraded":"2023-12-01T03:11:32.225296+0000","last_fullsized":"2023-12-01T03:11:32.225296+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.377359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217958+0000","last_change":"2023-12-01T03:11:32.211743+0000","last_active":"2023-12-01T03:11:32.217958+0000","last_peered":"2023-12-01T03:11:32.217958+0000","last_clean":"2023-12-01T03:11:32.217958+0000","last_became_active":"2023-12-01T03:11:32.211443+0000","last_became_peered":"2023-12-01T03:11:32.211443+0000","last_unstale":"2023-12-01T03:11:32.217958+0000","last_undegraded":"2023-12-01T03:11:32.217958+0000","last_fullsized":"2023-12-01T03:11:32.217958+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224872+0000","last_change":"2023-12-01T03:11:31.679798+0000","last_active":"2023-12-01T03:11:32.224872+0000","last_peered":"2023-12-01T03:11:32.224872+0000","last_clean":"2023-12-01T03:11:32.224872+0000","last_became_active":"2023-12-01T03:11:31.679030+0000","last_became_peered":"2023-12-01T03:11:31.679030+0000","last_unstale":"2023-12-01T03:11:32.224872+0000","last_undegraded":"2023-12-01T03:11:32.224872+0000","last_fullsized":"2023-12-01T03:11:32.224872+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:49.261314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223595+0000","last_change":"2023-12-01T03:11:32.222760+0000","last_active":"2023-12-01T03:11:32.223595+0000","last_peered":"2023-12-01T03:11:32.223595+0000","last_clean":"2023-12-01T03:11:32.223595+0000","last_became_active":"2023-12-01T03:11:32.222500+0000","last_became_peered":"2023-12-01T03:11:32.222500+0000","last_unstale":"2023-12-01T03:11:32.223595+0000","last_undegraded":"2023-12-01T03:11:32.223595+0000","last_fullsized":"2023-12-01T03:11:32.223595+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:17.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218820+0000","last_change":"2023-12-01T03:11:31.663745+0000","last_active":"2023-12-01T03:11:32.218820+0000","last_peered":"2023-12-01T03:11:32.218820+0000","last_clean":"2023-12-01T03:11:32.218820+0000","last_became_active":"2023-12-01T03:11:31.662706+0000","last_became_peered":"2023-12-01T03:11:31.662706+0000","last_unstale":"2023-12-01T03:11:32.218820+0000","last_undegraded":"2023-12-01T03:11:32.218820+0000","last_fullsized":"2023-12-01T03:11:32.218820+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:42.945449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217635+0000","last_change":"2023-12-01T03:11:32.213316+0000","last_active":"2023-12-01T03:11:32.217635+0000","last_peered":"2023-12-01T03:11:32.217635+0000","last_clean":"2023-12-01T03:11:32.217635+0000","last_became_active":"2023-12-01T03:11:32.213070+0000","last_became_peered":"2023-12-01T03:11:32.213070+0000","last_unstale":"2023-12-01T03:11:32.217635+0000","last_undegraded":"2023-12-01T03:11:32.217635+0000","last_fullsized":"2023-12-01T03:11:32.217635+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:22:51.687377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216557+0000","last_change":"2023-12-01T03:11:31.660758+0000","last_active":"2023-12-01T03:11:32.216557+0000","last_peered":"2023-12-01T03:11:32.216557+0000","last_clean":"2023-12-01T03:11:32.216557+0000","last_became_active":"2023-12-01T03:11:31.660001+0000","last_became_peered":"2023-12-01T03:11:31.660001+0000","last_unstale":"2023-12-01T03:11:32.216557+0000","last_undegraded":"2023-12-01T03:11:32.216557+0000","last_fullsized":"2023-12-01T03:11:32.216557+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218251+0000","last_change":"2023-12-01T03:11:32.213961+0000","last_active":"2023-12-01T03:11:32.218251+0000","last_peered":"2023-12-01T03:11:32.218251+0000","last_clean":"2023-12-01T03:11:32.218251+0000","last_became_active":"2023-12-01T03:11:32.213692+0000","last_became_peered":"2023-12-01T03:11:32.213692+0000","last_unstale":"2023-12-01T03:11:32.218251+0000","last_undegraded":"2023-12-01T03:11:32.218251+0000","last_fullsized":"2023-12-01T03:11:32.218251+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224769+0000","last_change":"2023-12-01T03:11:31.681531+0000","last_active":"2023-12-01T03:11:32.224769+0000","last_peered":"2023-12-01T03:11:32.224769+0000","last_clean":"2023-12-01T03:11:32.224769+0000","last_became_active":"2023-12-01T03:11:31.680782+0000","last_became_peered":"2023-12-01T03:11:31.680782+0000","last_unstale":"2023-12-01T03:11:32.224769+0000","last_undegraded":"2023-12-01T03:11:32.224769+0000","last_fullsized":"2023-12-01T03:11:32.224769+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:18.018055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211538+0000","last_change":"2023-12-01T03:11:32.206659+0000","last_active":"2023-12-01T03:11:32.211538+0000","last_peered":"2023-12-01T03:11:32.211538+0000","last_clean":"2023-12-01T03:11:32.211538+0000","last_became_active":"2023-12-01T03:11:32.206346+0000","last_became_peered":"2023-12-01T03:11:32.206346+0000","last_unstale":"2023-12-01T03:11:32.211538+0000","last_undegraded":"2023-12-01T03:11:32.211538+0000","last_fullsized":"2023-12-01T03:11:32.211538+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.196693+0000","last_change":"2023-12-01T03:11:31.663166+0000","last_active":"2023-12-01T03:11:32.196693+0000","last_peered":"2023-12-01T03:11:32.196693+0000","last_clean":"2023-12-01T03:11:32.196693+0000","last_became_active":"2023-12-01T03:11:31.662263+0000","last_became_peered":"2023-12-01T03:11:31.662263+0000","last_unstale":"2023-12-01T03:11:32.196693+0000","last_undegraded":"2023-12-01T03:11:32.196693+0000","last_fullsized":"2023-12-01T03:11:32.196693+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:57:00.965499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216627+0000","last_change":"2023-12-01T03:11:32.210988+0000","last_active":"2023-12-01T03:11:32.216627+0000","last_peered":"2023-12-01T03:11:32.216627+0000","last_clean":"2023-12-01T03:11:32.216627+0000","last_became_active":"2023-12-01T03:11:32.207844+0000","last_became_peered":"2023-12-01T03:11:32.207844+0000","last_unstale":"2023-12-01T03:11:32.216627+0000","last_undegraded":"2023-12-01T03:11:32.216627+0000","last_fullsized":"2023-12-01T03:11:32.216627+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.206110+0000","last_change":"2023-12-01T03:11:31.670524+0000","last_active":"2023-12-01T03:11:32.206110+0000","last_peered":"2023-12-01T03:11:32.206110+0000","last_clean":"2023-12-01T03:11:32.206110+0000","last_became_active":"2023-12-01T03:11:31.670190+0000","last_became_peered":"2023-12-01T03:11:31.670190+0000","last_unstale":"2023-12-01T03:11:32.206110+0000","last_undegraded":"2023-12-01T03:11:32.206110+0000","last_fullsized":"2023-12-01T03:11:32.206110+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:45.874739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.214311+0000","last_change":"2023-12-01T03:11:32.212987+0000","last_active":"2023-12-01T03:11:32.214311+0000","last_peered":"2023-12-01T03:11:32.214311+0000","last_clean":"2023-12-01T03:11:32.214311+0000","last_became_active":"2023-12-01T03:11:32.212426+0000","last_became_peered":"2023-12-01T03:11:32.212426+0000","last_unstale":"2023-12-01T03:11:32.214311+0000","last_undegraded":"2023-12-01T03:11:32.214311+0000","last_fullsized":"2023-12-01T03:11:32.214311+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.219351+0000","last_change":"2023-12-01T03:11:31.676322+0000","last_active":"2023-12-01T03:11:32.219351+0000","last_peered":"2023-12-01T03:11:32.219351+0000","last_clean":"2023-12-01T03:11:32.219351+0000","last_became_active":"2023-12-01T03:11:31.674451+0000","last_became_peered":"2023-12-01T03:11:31.674451+0000","last_unstale":"2023-12-01T03:11:32.219351+0000","last_undegraded":"2023-12-01T03:11:32.219351+0000","last_fullsized":"2023-12-01T03:11:32.219351+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:44.820233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222710+0000","last_change":"2023-12-01T03:11:32.219383+0000","last_active":"2023-12-01T03:11:32.222710+0000","last_peered":"2023-12-01T03:11:32.222710+0000","last_clean":"2023-12-01T03:11:32.222710+0000","last_became_active":"2023-12-01T03:11:32.219139+0000","last_became_peered":"2023-12-01T03:11:32.219139+0000","last_unstale":"2023-12-01T03:11:32.222710+0000","last_undegraded":"2023-12-01T03:11:32.222710+0000","last_fullsized":"2023-12-01T03:11:32.222710+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:40.557913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.192088+0000","last_change":"2023-12-01T03:11:31.662929+0000","last_active":"2023-12-01T03:11:32.192088+0000","last_peered":"2023-12-01T03:11:32.192088+0000","last_clean":"2023-12-01T03:11:32.192088+0000","last_became_active":"2023-12-01T03:11:31.662617+0000","last_became_peered":"2023-12-01T03:11:31.662617+0000","last_unstale":"2023-12-01T03:11:32.192088+0000","last_undegraded":"2023-12-01T03:11:32.192088+0000","last_fullsized":"2023-12-01T03:11:32.192088+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:34.611280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210926+0000","last_change":"2023-12-01T03:11:32.209244+0000","last_active":"2023-12-01T03:11:32.210926+0000","last_peered":"2023-12-01T03:11:32.210926+0000","last_clean":"2023-12-01T03:11:32.210926+0000","last_became_active":"2023-12-01T03:11:32.206294+0000","last_became_peered":"2023-12-01T03:11:32.206294+0000","last_unstale":"2023-12-01T03:11:32.210926+0000","last_undegraded":"2023-12-01T03:11:32.210926+0000","last_fullsized":"2023-12-01T03:11:32.210926+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"234'3","reported_seq":14,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218935+0000","last_change":"2023-12-01T03:11:31.664798+0000","last_active":"2023-12-01T03:11:32.218935+0000","last_peered":"2023-12-01T03:11:32.218935+0000","last_clean":"2023-12-01T03:11:32.218935+0000","last_became_active":"2023-12-01T03:11:31.664276+0000","last_became_peered":"2023-12-01T03:11:31.664276+0000","last_unstale":"2023-12-01T03:11:32.218935+0000","last_undegraded":"2023-12-01T03:11:32.218935+0000","last_fullsized":"2023-12-01T03:11:32.218935+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:18:18.909277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.215970+0000","last_change":"2023-12-01T03:11:32.213607+0000","last_active":"2023-12-01T03:11:32.215970+0000","last_peered":"2023-12-01T03:11:32.215970+0000","last_clean":"2023-12-01T03:11:32.215970+0000","last_became_active":"2023-12-01T03:11:32.211875+0000","last_became_peered":"2023-12-01T03:11:32.211875+0000","last_unstale":"2023-12-01T03:11:32.215970+0000","last_undegraded":"2023-12-01T03:11:32.215970+0000","last_fullsized":"2023-12-01T03:11:32.215970+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"234'6","reported_seq":17,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223391+0000","last_change":"2023-12-01T03:11:31.664162+0000","last_active":"2023-12-01T03:11:32.223391+0000","last_peered":"2023-12-01T03:11:32.223391+0000","last_clean":"2023-12-01T03:11:32.223391+0000","last_became_active":"2023-12-01T03:11:31.663854+0000","last_became_peered":"2023-12-01T03:11:31.663854+0000","last_unstale":"2023-12-01T03:11:32.223391+0000","last_undegraded":"2023-12-01T03:11:32.223391+0000","last_fullsized":"2023-12-01T03:11:32.223391+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:15.367696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209163+0000","last_change":"2023-12-01T03:11:32.203625+0000","last_active":"2023-12-01T03:11:32.209163+0000","last_peered":"2023-12-01T03:11:32.209163+0000","last_clean":"2023-12-01T03:11:32.209163+0000","last_became_active":"2023-12-01T03:11:32.203300+0000","last_became_peered":"2023-12-01T03:11:32.203300+0000","last_unstale":"2023-12-01T03:11:32.209163+0000","last_undegraded":"2023-12-01T03:11:32.209163+0000","last_fullsized":"2023-12-01T03:11:32.209163+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:46.890682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222667+0000","last_change":"2023-12-01T03:11:31.665987+0000","last_active":"2023-12-01T03:11:32.222667+0000","last_peered":"2023-12-01T03:11:32.222667+0000","last_clean":"2023-12-01T03:11:32.222667+0000","last_became_active":"2023-12-01T03:11:31.665550+0000","last_became_peered":"2023-12-01T03:11:31.665550+0000","last_unstale":"2023-12-01T03:11:32.222667+0000","last_undegraded":"2023-12-01T03:11:32.222667+0000","last_fullsized":"2023-12-01T03:11:32.222667+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:16.324155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209603+0000","last_change":"2023-12-01T03:11:32.208104+0000","last_active":"2023-12-01T03:11:32.209603+0000","last_peered":"2023-12-01T03:11:32.209603+0000","last_clean":"2023-12-01T03:11:32.209603+0000","last_became_active":"2023-12-01T03:11:32.207859+0000","last_became_peered":"2023-12-01T03:11:32.207859+0000","last_unstale":"2023-12-01T03:11:32.209603+0000","last_undegraded":"2023-12-01T03:11:32.209603+0000","last_fullsized":"2023-12-01T03:11:32.209603+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.188460+0000","last_change":"2023-12-01T03:11:31.675387+0000","last_active":"2023-12-01T03:11:32.188460+0000","last_peered":"2023-12-01T03:11:32.188460+0000","last_clean":"2023-12-01T03:11:32.188460+0000","last_became_active":"2023-12-01T03:11:31.675108+0000","last_became_peered":"2023-12-01T03:11:31.675108+0000","last_unstale":"2023-12-01T03:11:32.188460+0000","last_undegraded":"2023-12-01T03:11:32.188460+0000","last_fullsized":"2023-12-01T03:11:32.188460+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.836486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211907+0000","last_change":"2023-12-01T03:11:32.204685+0000","last_active":"2023-12-01T03:11:32.211907+0000","last_peered":"2023-12-01T03:11:32.211907+0000","last_clean":"2023-12-01T03:11:32.211907+0000","last_became_active":"2023-12-01T03:11:32.204422+0000","last_became_peered":"2023-12-01T03:11:32.204422+0000","last_unstale":"2023-12-01T03:11:32.211907+0000","last_undegraded":"2023-12-01T03:11:32.211907+0000","last_fullsized":"2023-12-01T03:11:32.211907+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.188647+0000","last_change":"2023-12-01T03:11:31.671174+0000","last_active":"2023-12-01T03:11:32.188647+0000","last_peered":"2023-12-01T03:11:32.188647+0000","last_clean":"2023-12-01T03:11:32.188647+0000","last_became_active":"2023-12-01T03:11:31.670862+0000","last_became_peered":"2023-12-01T03:11:31.670862+0000","last_unstale":"2023-12-01T03:11:32.188647+0000","last_undegraded":"2023-12-01T03:11:32.188647+0000","last_fullsized":"2023-12-01T03:11:32.188647+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:37.158883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209558+0000","last_change":"2023-12-01T03:11:32.203594+0000","last_active":"2023-12-01T03:11:32.209558+0000","last_peered":"2023-12-01T03:11:32.209558+0000","last_clean":"2023-12-01T03:11:32.209558+0000","last_became_active":"2023-12-01T03:11:32.203346+0000","last_became_peered":"2023-12-01T03:11:32.203346+0000","last_unstale":"2023-12-01T03:11:32.209558+0000","last_undegraded":"2023-12-01T03:11:32.209558+0000","last_fullsized":"2023-12-01T03:11:32.209558+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:22.924077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.10","version":"234'8","reported_seq":19,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218897+0000","last_change":"2023-12-01T03:11:31.673143+0000","last_active":"2023-12-01T03:11:32.218897+0000","last_peered":"2023-12-01T03:11:32.218897+0000","last_clean":"2023-12-01T03:11:32.218897+0000","last_became_active":"2023-12-01T03:11:31.671261+0000","last_became_peered":"2023-12-01T03:11:31.671261+0000","last_unstale":"2023-12-01T03:11:32.218897+0000","last_undegraded":"2023-12-01T03:11:32.218897+0000","last_fullsized":"2023-12-01T03:11:32.218897+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:13.744340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213478+0000","last_change":"2023-12-01T03:11:32.209735+0000","last_active":"2023-12-01T03:11:32.213478+0000","last_peered":"2023-12-01T03:11:32.213478+0000","last_clean":"2023-12-01T03:11:32.213478+0000","last_became_active":"2023-12-01T03:11:32.209148+0000","last_became_peered":"2023-12-01T03:11:32.209148+0000","last_unstale":"2023-12-01T03:11:32.213478+0000","last_undegraded":"2023-12-01T03:11:32.213478+0000","last_fullsized":"2023-12-01T03:11:32.213478+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:46.532354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225412+0000","last_change":"2023-12-01T03:11:31.677841+0000","last_active":"2023-12-01T03:11:32.225412+0000","last_peered":"2023-12-01T03:11:32.225412+0000","last_clean":"2023-12-01T03:11:32.225412+0000","last_became_active":"2023-12-01T03:11:31.677483+0000","last_became_peered":"2023-12-01T03:11:31.677483+0000","last_unstale":"2023-12-01T03:11:32.225412+0000","last_undegraded":"2023-12-01T03:11:32.225412+0000","last_fullsized":"2023-12-01T03:11:32.225412+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:31.837132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217241+0000","last_change":"2023-12-01T03:11:32.213539+0000","last_active":"2023-12-01T03:11:32.217241+0000","last_peered":"2023-12-01T03:11:32.217241+0000","last_clean":"2023-12-01T03:11:32.217241+0000","last_became_active":"2023-12-01T03:11:32.212492+0000","last_became_peered":"2023-12-01T03:11:32.212492+0000","last_unstale":"2023-12-01T03:11:32.217241+0000","last_undegraded":"2023-12-01T03:11:32.217241+0000","last_fullsized":"2023-12-01T03:11:32.217241+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:29:48.204277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.e","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222148+0000","last_change":"2023-12-01T03:11:31.670701+0000","last_active":"2023-12-01T03:11:32.222148+0000","last_peered":"2023-12-01T03:11:32.222148+0000","last_clean":"2023-12-01T03:11:32.222148+0000","last_became_active":"2023-12-01T03:11:31.670179+0000","last_became_peered":"2023-12-01T03:11:31.670179+0000","last_unstale":"2023-12-01T03:11:32.222148+0000","last_undegraded":"2023-12-01T03:11:32.222148+0000","last_fullsized":"2023-12-01T03:11:32.222148+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:01.094744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216871+0000","last_change":"2023-12-01T03:11:32.212327+0000","last_active":"2023-12-01T03:11:32.216871+0000","last_peered":"2023-12-01T03:11:32.216871+0000","last_clean":"2023-12-01T03:11:32.216871+0000","last_became_active":"2023-12-01T03:11:32.212068+0000","last_became_peered":"2023-12-01T03:11:32.212068+0000","last_unstale":"2023-12-01T03:11:32.216871+0000","last_undegraded":"2023-12-01T03:11:32.216871+0000","last_fullsized":"2023-12-01T03:11:32.216871+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:31.875817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224912+0000","last_change":"2023-12-01T03:11:31.679287+0000","last_active":"2023-12-01T03:11:32.224912+0000","last_peered":"2023-12-01T03:11:32.224912+0000","last_clean":"2023-12-01T03:11:32.224912+0000","last_became_active":"2023-12-01T03:11:31.678925+0000","last_became_peered":"2023-12-01T03:11:31.678925+0000","last_unstale":"2023-12-01T03:11:32.224912+0000","last_undegraded":"2023-12-01T03:11:32.224912+0000","last_fullsized":"2023-12-01T03:11:32.224912+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:28:46.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.221614+0000","last_change":"2023-12-01T03:11:32.214436+0000","last_active":"2023-12-01T03:11:32.221614+0000","last_peered":"2023-12-01T03:11:32.221614+0000","last_clean":"2023-12-01T03:11:32.221614+0000","last_became_active":"2023-12-01T03:11:32.214162+0000","last_became_peered":"2023-12-01T03:11:32.214162+0000","last_unstale":"2023-12-01T03:11:32.221614+0000","last_undegraded":"2023-12-01T03:11:32.221614+0000","last_fullsized":"2023-12-01T03:11:32.221614+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:18:18.055289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.192683+0000","last_change":"2023-12-01T03:11:31.674632+0000","last_active":"2023-12-01T03:11:32.192683+0000","last_peered":"2023-12-01T03:11:32.192683+0000","last_clean":"2023-12-01T03:11:32.192683+0000","last_became_active":"2023-12-01T03:11:31.674319+0000","last_became_peered":"2023-12-01T03:11:31.674319+0000","last_unstale":"2023-12-01T03:11:32.192683+0000","last_undegraded":"2023-12-01T03:11:32.192683+0000","last_fullsized":"2023-12-01T03:11:32.192683+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:21.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216607+0000","last_change":"2023-12-01T03:11:32.209024+0000","last_active":"2023-12-01T03:11:32.216607+0000","last_peered":"2023-12-01T03:11:32.216607+0000","last_clean":"2023-12-01T03:11:32.216607+0000","last_became_active":"2023-12-01T03:11:32.208729+0000","last_became_peered":"2023-12-01T03:11:32.208729+0000","last_unstale":"2023-12-01T03:11:32.216607+0000","last_undegraded":"2023-12-01T03:11:32.216607+0000","last_fullsized":"2023-12-01T03:11:32.216607+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:27:53.473015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.b","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223478+0000","last_change":"2023-12-01T03:11:31.676885+0000","last_active":"2023-12-01T03:11:32.223478+0000","last_peered":"2023-12-01T03:11:32.223478+0000","last_clean":"2023-12-01T03:11:32.223478+0000","last_became_active":"2023-12-01T03:11:31.675545+0000","last_became_peered":"2023-12-01T03:11:31.675545+0000","last_unstale":"2023-12-01T03:11:32.223478+0000","last_undegraded":"2023-12-01T03:11:32.223478+0000","last_fullsized":"2023-12-01T03:11:32.223478+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:03.962061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212390+0000","last_change":"2023-12-01T03:11:32.204502+0000","last_active":"2023-12-01T03:11:32.212390+0000","last_peered":"2023-12-01T03:11:32.212390+0000","last_clean":"2023-12-01T03:11:32.212390+0000","last_became_active":"2023-12-01T03:11:32.204048+0000","last_became_peered":"2023-12-01T03:11:32.204048+0000","last_unstale":"2023-12-01T03:11:32.212390+0000","last_undegraded":"2023-12-01T03:11:32.212390+0000","last_fullsized":"2023-12-01T03:11:32.212390+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:15:30.930202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.183774+0000","last_change":"2023-12-01T03:11:31.675947+0000","last_active":"2023-12-01T03:11:32.183774+0000","last_peered":"2023-12-01T03:11:32.183774+0000","last_clean":"2023-12-01T03:11:32.183774+0000","last_became_active":"2023-12-01T03:11:31.675372+0000","last_became_peered":"2023-12-01T03:11:31.675372+0000","last_unstale":"2023-12-01T03:11:32.183774+0000","last_undegraded":"2023-12-01T03:11:32.183774+0000","last_fullsized":"2023-12-01T03:11:32.183774+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:45.516411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216820+0000","last_change":"2023-12-01T03:11:32.214460+0000","last_active":"2023-12-01T03:11:32.216820+0000","last_peered":"2023-12-01T03:11:32.216820+0000","last_clean":"2023-12-01T03:11:32.216820+0000","last_became_active":"2023-12-01T03:11:32.213987+0000","last_became_peered":"2023-12-01T03:11:32.213987+0000","last_unstale":"2023-12-01T03:11:32.216820+0000","last_undegraded":"2023-12-01T03:11:32.216820+0000","last_fullsized":"2023-12-01T03:11:32.216820+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:20.713703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.194822+0000","last_change":"2023-12-01T03:11:31.673899+0000","last_active":"2023-12-01T03:11:32.194822+0000","last_peered":"2023-12-01T03:11:32.194822+0000","last_clean":"2023-12-01T03:11:32.194822+0000","last_became_active":"2023-12-01T03:11:31.673474+0000","last_became_peered":"2023-12-01T03:11:31.673474+0000","last_unstale":"2023-12-01T03:11:32.194822+0000","last_undegraded":"2023-12-01T03:11:32.194822+0000","last_fullsized":"2023-12-01T03:11:32.194822+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:50.332038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218479+0000","last_change":"2023-12-01T03:11:32.214835+0000","last_active":"2023-12-01T03:11:32.218479+0000","last_peered":"2023-12-01T03:11:32.218479+0000","last_clean":"2023-12-01T03:11:32.218479+0000","last_became_active":"2023-12-01T03:11:32.214553+0000","last_became_peered":"2023-12-01T03:11:32.214553+0000","last_unstale":"2023-12-01T03:11:32.218479+0000","last_undegraded":"2023-12-01T03:11:32.218479+0000","last_fullsized":"2023-12-01T03:11:32.218479+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.190392+0000","last_change":"2023-12-01T03:11:31.682949+0000","last_active":"2023-12-01T03:11:32.190392+0000","last_peered":"2023-12-01T03:11:32.190392+0000","last_clean":"2023-12-01T03:11:32.190392+0000","last_became_active":"2023-12-01T03:11:31.682454+0000","last_became_peered":"2023-12-01T03:11:31.682454+0000","last_unstale":"2023-12-01T03:11:32.190392+0000","last_undegraded":"2023-12-01T03:11:32.190392+0000","last_fullsized":"2023-12-01T03:11:32.190392+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:49.777291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.208510+0000","last_change":"2023-12-01T03:11:32.204430+0000","last_active":"2023-12-01T03:11:32.208510+0000","last_peered":"2023-12-01T03:11:32.208510+0000","last_clean":"2023-12-01T03:11:32.208510+0000","last_became_active":"2023-12-01T03:11:32.204043+0000","last_became_peered":"2023-12-01T03:11:32.204043+0000","last_unstale":"2023-12-01T03:11:32.208510+0000","last_undegraded":"2023-12-01T03:11:32.208510+0000","last_fullsized":"2023-12-01T03:11:32.208510+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:33:45.498838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222532+0000","last_change":"2023-12-01T03:11:31.668375+0000","last_active":"2023-12-01T03:11:32.222532+0000","last_peered":"2023-12-01T03:11:32.222532+0000","last_clean":"2023-12-01T03:11:32.222532+0000","last_became_active":"2023-12-01T03:11:31.668109+0000","last_became_peered":"2023-12-01T03:11:31.668109+0000","last_unstale":"2023-12-01T03:11:32.222532+0000","last_undegraded":"2023-12-01T03:11:32.222532+0000","last_fullsized":"2023-12-01T03:11:32.222532+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:34.959657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225708+0000","last_change":"2023-12-01T03:11:32.223409+0000","last_active":"2023-12-01T03:11:32.225708+0000","last_peered":"2023-12-01T03:11:32.225708+0000","last_clean":"2023-12-01T03:11:32.225708+0000","last_became_active":"2023-12-01T03:11:32.221541+0000","last_became_peered":"2023-12-01T03:11:32.221541+0000","last_unstale":"2023-12-01T03:11:32.225708+0000","last_undegraded":"2023-12-01T03:11:32.225708+0000","last_fullsized":"2023-12-01T03:11:32.225708+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:53.273884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.219881+0000","last_change":"2023-12-01T03:11:31.679248+0000","last_active":"2023-12-01T03:11:32.219881+0000","last_peered":"2023-12-01T03:11:32.219881+0000","last_clean":"2023-12-01T03:11:32.219881+0000","last_became_active":"2023-12-01T03:11:31.678973+0000","last_became_peered":"2023-12-01T03:11:31.678973+0000","last_unstale":"2023-12-01T03:11:32.219881+0000","last_undegraded":"2023-12-01T03:11:32.219881+0000","last_fullsized":"2023-12-01T03:11:32.219881+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212063+0000","last_change":"2023-12-01T03:11:32.207147+0000","last_active":"2023-12-01T03:11:32.212063+0000","last_peered":"2023-12-01T03:11:32.212063+0000","last_clean":"2023-12-01T03:11:32.212063+0000","last_became_active":"2023-12-01T03:11:32.206872+0000","last_became_peered":"2023-12-01T03:11:32.206872+0000","last_unstale":"2023-12-01T03:11:32.212063+0000","last_undegraded":"2023-12-01T03:11:32.212063+0000","last_fullsized":"2023-12-01T03:11:32.212063+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.202418+0000","last_change":"2023-12-01T03:11:31.663771+0000","last_active":"2023-12-01T03:11:32.202418+0000","last_peered":"2023-12-01T03:11:32.202418+0000","last_clean":"2023-12-01T03:11:32.202418+0000","last_became_active":"2023-12-01T03:11:31.662877+0000","last_became_peered":"2023-12-01T03:11:31.662877+0000","last_unstale":"2023-12-01T03:11:32.202418+0000","last_undegraded":"2023-12-01T03:11:32.202418+0000","last_fullsized":"2023-12-01T03:11:32.202418+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:59.396265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213526+0000","last_change":"2023-12-01T03:11:32.208042+0000","last_active":"2023-12-01T03:11:32.213526+0000","last_peered":"2023-12-01T03:11:32.213526+0000","last_clean":"2023-12-01T03:11:32.213526+0000","last_became_active":"2023-12-01T03:11:32.207767+0000","last_became_peered":"2023-12-01T03:11:32.207767+0000","last_unstale":"2023-12-01T03:11:32.213526+0000","last_undegraded":"2023-12-01T03:11:32.213526+0000","last_fullsized":"2023-12-01T03:11:32.213526+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:17.846927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.195227+0000","last_change":"2023-12-01T03:11:31.673043+0000","last_active":"2023-12-01T03:11:32.195227+0000","last_peered":"2023-12-01T03:11:32.195227+0000","last_clean":"2023-12-01T03:11:32.195227+0000","last_became_active":"2023-12-01T03:11:31.672780+0000","last_became_peered":"2023-12-01T03:11:31.672780+0000","last_unstale":"2023-12-01T03:11:32.195227+0000","last_undegraded":"2023-12-01T03:11:32.195227+0000","last_fullsized":"2023-12-01T03:11:32.195227+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:40.478556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225682+0000","last_change":"2023-12-01T03:11:32.224518+0000","last_active":"2023-12-01T03:11:32.225682+0000","last_peered":"2023-12-01T03:11:32.225682+0000","last_clean":"2023-12-01T03:11:32.225682+0000","last_became_active":"2023-12-01T03:11:32.220831+0000","last_became_peered":"2023-12-01T03:11:32.220831+0000","last_unstale":"2023-12-01T03:11:32.225682+0000","last_undegraded":"2023-12-01T03:11:32.225682+0000","last_fullsized":"2023-12-01T03:11:32.225682+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.189860+0000","last_change":"2023-12-01T03:11:31.677162+0000","last_active":"2023-12-01T03:11:32.189860+0000","last_peered":"2023-12-01T03:11:32.189860+0000","last_clean":"2023-12-01T03:11:32.189860+0000","last_became_active":"2023-12-01T03:11:31.674178+0000","last_became_peered":"2023-12-01T03:11:31.674178+0000","last_unstale":"2023-12-01T03:11:32.189860+0000","last_undegraded":"2023-12-01T03:11:32.189860+0000","last_fullsized":"2023-12-01T03:11:32.189860+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:13.700505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225199+0000","last_change":"2023-12-01T03:11:32.222439+0000","last_active":"2023-12-01T03:11:32.225199+0000","last_peered":"2023-12-01T03:11:32.225199+0000","last_clean":"2023-12-01T03:11:32.225199+0000","last_became_active":"2023-12-01T03:11:32.221662+0000","last_became_peered":"2023-12-01T03:11:32.221662+0000","last_unstale":"2023-12-01T03:11:32.225199+0000","last_undegraded":"2023-12-01T03:11:32.225199+0000","last_fullsized":"2023-12-01T03:11:32.225199+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.188580+0000","last_change":"2023-12-01T03:11:31.672908+0000","last_active":"2023-12-01T03:11:32.188580+0000","last_peered":"2023-12-01T03:11:32.188580+0000","last_clean":"2023-12-01T03:11:32.188580+0000","last_became_active":"2023-12-01T03:11:31.671738+0000","last_became_peered":"2023-12-01T03:11:31.671738+0000","last_unstale":"2023-12-01T03:11:32.188580+0000","last_undegraded":"2023-12-01T03:11:32.188580+0000","last_fullsized":"2023-12-01T03:11:32.188580+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220159+0000","last_change":"2023-12-01T03:11:32.214055+0000","last_active":"2023-12-01T03:11:32.220159+0000","last_peered":"2023-12-01T03:11:32.220159+0000","last_clean":"2023-12-01T03:11:32.220159+0000","last_became_active":"2023-12-01T03:11:32.213767+0000","last_became_peered":"2023-12-01T03:11:32.213767+0000","last_unstale":"2023-12-01T03:11:32.220159+0000","last_undegraded":"2023-12-01T03:11:32.220159+0000","last_fullsized":"2023-12-01T03:11:32.220159+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:47:03.446609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.201156+0000","last_change":"2023-12-01T03:11:31.676480+0000","last_active":"2023-12-01T03:11:32.201156+0000","last_peered":"2023-12-01T03:11:32.201156+0000","last_clean":"2023-12-01T03:11:32.201156+0000","last_became_active":"2023-12-01T03:11:31.676127+0000","last_became_peered":"2023-12-01T03:11:31.676127+0000","last_unstale":"2023-12-01T03:11:32.201156+0000","last_undegraded":"2023-12-01T03:11:32.201156+0000","last_fullsized":"2023-12-01T03:11:32.201156+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.208183+0000","last_change":"2023-12-01T03:11:32.203024+0000","last_active":"2023-12-01T03:11:32.208183+0000","last_peered":"2023-12-01T03:11:32.208183+0000","last_clean":"2023-12-01T03:11:32.208183+0000","last_became_active":"2023-12-01T03:11:32.202779+0000","last_became_peered":"2023-12-01T03:11:32.202779+0000","last_unstale":"2023-12-01T03:11:32.208183+0000","last_undegraded":"2023-12-01T03:11:32.208183+0000","last_fullsized":"2023-12-01T03:11:32.208183+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.205725+0000","last_change":"2023-12-01T03:11:31.680251+0000","last_active":"2023-12-01T03:11:32.205725+0000","last_peered":"2023-12-01T03:11:32.205725+0000","last_clean":"2023-12-01T03:11:32.205725+0000","last_became_active":"2023-12-01T03:11:31.679936+0000","last_became_peered":"2023-12-01T03:11:31.679936+0000","last_unstale":"2023-12-01T03:11:32.205725+0000","last_undegraded":"2023-12-01T03:11:32.205725+0000","last_fullsized":"2023-12-01T03:11:32.205725+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:39.296836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211446+0000","last_change":"2023-12-01T03:11:32.210270+0000","last_active":"2023-12-01T03:11:32.211446+0000","last_peered":"2023-12-01T03:11:32.211446+0000","last_clean":"2023-12-01T03:11:32.211446+0000","last_became_active":"2023-12-01T03:11:32.209831+0000","last_became_peered":"2023-12-01T03:11:32.209831+0000","last_unstale":"2023-12-01T03:11:32.211446+0000","last_undegraded":"2023-12-01T03:11:32.211446+0000","last_fullsized":"2023-12-01T03:11:32.211446+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224896+0000","last_change":"2023-12-01T03:11:31.681391+0000","last_active":"2023-12-01T03:11:32.224896+0000","last_peered":"2023-12-01T03:11:32.224896+0000","last_clean":"2023-12-01T03:11:32.224896+0000","last_became_active":"2023-12-01T03:11:31.680773+0000","last_became_peered":"2023-12-01T03:11:31.680773+0000","last_unstale":"2023-12-01T03:11:32.224896+0000","last_undegraded":"2023-12-01T03:11:32.224896+0000","last_fullsized":"2023-12-01T03:11:32.224896+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:40:13.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"220'348","reported_seq":1214,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.205820+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:11:32.205820+0000","last_peered":"2023-12-01T03:11:32.205820+0000","last_clean":"2023-12-01T03:11:32.205820+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:11:32.205820+0000","last_undegraded":"2023-12-01T03:11:32.205820+0000","last_fullsized":"2023-12-01T03:11:32.205820+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575989,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60180,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311660,"statfs":{"total":96636764160,"available":96575139840,"internally_reserved":0,"allocated":2961408,"data_stored":1448140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.05,"15min":1.071},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5409999999999999,"5min":1.6000000000000001,"15min":1.7390000000000001},"last":1.613},{"interface":"front","average":{"1min":1.111,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6599999999999999,"5min":1.6699999999999999,"15min":1.7},"last":0.90500000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.236,"15min":1.2769999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4990000000000001,"5min":1.78,"15min":2.1419999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.294,"15min":1.3260000000000001},"min":{"1min":0.873,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5740000000000001,"5min":1.6240000000000001,"15min":1.8700000000000001},"last":1.5649999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1439999999999999,"15min":1.173},"min":{"1min":0.71699999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.573,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":1.647},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.2549999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.643,"5min":1.7050000000000001,"15min":2.1120000000000001},"last":1.147}]},{"osd":3,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2509999999999999,"15min":1.302},"min":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.712,"15min":2.431},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3720000000000001,"15min":1.4119999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7529999999999999,"5min":1.879,"15min":2.198},"last":1.476}]},{"osd":4,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3009999999999999,"15min":1.335},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7689999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.4510000000000001,"15min":1.49},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8799999999999999,"5min":1.97,"15min":2.569},"last":1.694}]},{"osd":5,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2869999999999999,"15min":1.3160000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7150000000000001,"5min":2.0099999999999998,"15min":2.7269999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.5469999999999999,"15min":1.585},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.0470000000000002,"5min":2.0569999999999999,"15min":2.484},"last":1.7589999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.403,"15min":1.4359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7749999999999999,"5min":1.921,"15min":2.532},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4299999999999999,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8260000000000001,"5min":1.8819999999999999,"15min":2.762},"last":1.726}]}]},{"osd":6,"up_from":13,"seq":55834575990,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42136,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329704,"statfs":{"total":96636764160,"available":96593616896,"internally_reserved":0,"allocated":2965504,"data_stored":1447614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0509999999999999,"15min":1.1080000000000001},"min":{"1min":0.66100000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.073,"5min":1.1040000000000001,"15min":1.157},"min":{"1min":0.78400000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6910000000000001,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.6439999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.141,"15min":1.2150000000000001},"min":{"1min":0.68999999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8460000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1830000000000001,"15min":1.22},"min":{"1min":0.86099999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.6220000000000001,"5min":1.694,"15min":1.835},"last":1.3939999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.157,"15min":1.206},"min":{"1min":0.71799999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.464,"5min":1.742,"15min":1.976},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.222,"15min":1.278},"min":{"1min":0.71699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5009999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.2909999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2829999999999999,"15min":1.3140000000000001},"min":{"1min":0.89500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6839999999999999,"5min":1.927,"15min":2.0289999999999999},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":1.0069999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0190000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.599}]},{"osd":4,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":0.93600000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.532},{"interface":"front","average":{"1min":1.589,"5min":1.407,"15min":1.367},"min":{"1min":1.2090000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.222,"5min":2.48,"15min":2.48},"last":2.1019999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.3700000000000001,"15min":1.413},"min":{"1min":1.127,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.962,"5min":2.262,"15min":2.262},"last":1.167},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5169999999999999,"15min":1.4970000000000001},"min":{"1min":1.2629999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.194,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.74}]},{"osd":7,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5029999999999999,"15min":1.5309999999999999},"min":{"1min":1.155,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1320000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.865},{"interface":"front","average":{"1min":1.746,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":1.371,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2770000000000001,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.841}]}]},{"osd":5,"up_from":13,"seq":55834575989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52576,"kb_used_data":3480,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319264,"statfs":{"total":96636764160,"available":96582926336,"internally_reserved":0,"allocated":3563520,"data_stored":2039470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1000000000000001,"15min":1.123},"min":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.7170000000000001,"5min":1.776,"15min":1.776},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1539999999999999,"15min":1.161},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.752},"last":0.97599999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.1559999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6120000000000001,"5min":1.6519999999999999,"15min":1.696},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.198,"15min":1.2330000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":1.506}]},{"osd":2,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2270000000000001,"15min":1.2470000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.81399999999999995},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.2450000000000001,"15min":1.254},"min":{"1min":0.75,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.867,"5min":1.899,"15min":1.9219999999999999},"last":1.5609999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3089999999999999,"15min":1.3149999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.889,"5min":1.889,"15min":1.8979999999999999},"last":1.175},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.329,"15min":1.359},"min":{"1min":1.0109999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.97,"5min":1.97,"15min":2.1859999999999999},"last":1.444}]},{"osd":4,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.869,"5min":1.869,"15min":2.6040000000000001},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.2470000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3779999999999999,"15min":1.377},"min":{"1min":0.99099999999999999,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.7649999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.409,"15min":1.4219999999999999},"min":{"1min":1.0149999999999999,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":2.032,"5min":2.032,"15min":2.577},"last":1.6830000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.534,"15min":1.538},"min":{"1min":1.2010000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":2.077,"5min":2.077,"15min":3.1709999999999998},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5760000000000001,"15min":1.5780000000000001},"min":{"1min":1.3200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.101,"15min":2.8740000000000001},"last":1.704}]}]},{"osd":4,"up_from":13,"seq":55834575989,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51992,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319848,"statfs":{"total":96636764160,"available":96583524352,"internally_reserved":0,"allocated":2965504,"data_stored":1449032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1200000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8089999999999999,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.198,"15min":1.1899999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5309999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.2669999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2290000000000001,"15min":1.256},"min":{"1min":0.83599999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8580000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1719999999999999,"15min":1.175},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.778,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.093}]},{"osd":2,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3300000000000001,"15min":1.244},"min":{"1min":0.74399999999999999,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.637,"5min":11.311,"15min":11.311},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3320000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9330000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.89700000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.298,"15min":1.29},"min":{"1min":0.80000000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.716,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3759999999999999,"15min":1.3340000000000001},"min":{"1min":0.96399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7430000000000001,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.357,"15min":1.3200000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.738,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.375,"15min":1.3160000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6579999999999999,"5min":3.71,"15min":3.71},"last":1.411}]},{"osd":6,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3799999999999999,"15min":1.3740000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8009999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4790000000000001,"15min":1.476},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.76,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.3939999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.514,"15min":1.488},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9830000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.597,"15min":1.575},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.907,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.5860000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575988,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56076,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315764,"statfs":{"total":96636764160,"available":96579342336,"internally_reserved":0,"allocated":2953216,"data_stored":1447468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.91300000000000003,"15min":0.90700000000000003},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.44},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1399999999999999,"15min":1.167},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.788,"5min":3.863,"15min":3.863},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.073,"15min":1.0960000000000001},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.637,"5min":3.597,"15min":3.597},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.127,"5min":1.2090000000000001,"15min":1.2410000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.8319999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.745}]},{"osd":4,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2969999999999999,"15min":1.294},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.272,"15min":1.296},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.736,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.331,"15min":1.329},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.1240000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.373,"15min":1.373},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.425,"15min":1.444},"min":{"1min":0.80500000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.5089999999999999,"15min":1.5129999999999999},"min":{"1min":1.0089999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":1.087}]},{"osd":7,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5129999999999999,"15min":1.5249999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.899,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.5209999999999999,"15min":1.526},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":1.1859999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575988,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47888,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323952,"statfs":{"total":96636764160,"available":96587726848,"internally_reserved":0,"allocated":2957312,"data_stored":1447502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.411,"5min":1.411,"15min":1.4730000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.052,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3080000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.216}]},{"osd":1,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.0229999999999999,"15min":1.0489999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.343,"5min":1.698,"15min":1.698},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.097,"15min":1.135},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.4730000000000001,"5min":1.635,"15min":1.6599999999999999},"last":1.5229999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.026,"15min":1.042},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.393,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.159,"15min":1.208},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4950000000000001,"5min":1.6879999999999999,"15min":1.79},"last":1.4970000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.319,"15min":1.327},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5700000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.621},{"interface":"front","average":{"1min":1.194,"5min":1.2689999999999999,"15min":1.294},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5189999999999999,"5min":1.714,"15min":1.835},"last":1.411}]},{"osd":5,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3560000000000001,"15min":1.3839999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.782,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.266},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.407,"15min":1.411},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.873,"5min":1.8740000000000001,"15min":2.016},"last":1.4530000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3660000000000001,"15min":1.3799999999999999},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.714,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4730000000000001,"15min":1.4970000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.353}]},{"osd":7,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.478,"15min":1.4850000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.9350000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.4259999999999999,"15min":1.454},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8149999999999999,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.5509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575987,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48476,"kb_used_data":3476,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323364,"statfs":{"total":96636764160,"available":96587124736,"internally_reserved":0,"allocated":3559424,"data_stored":2038510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.89800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.238,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.98499999999999999,"15min":1.038},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.512,"5min":1.5660000000000001,"15min":1.681},"last":1.089}]},{"osd":2,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2190000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.1020000000000001,"15min":1.141},"min":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5489999999999999,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":1.0429999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.208,"15min":1.2949999999999999},"min":{"1min":0.60799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5860000000000001,"5min":1.587,"15min":1.788},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.02,"5min":1.0780000000000001,"15min":1.127},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.383,"5min":1.714,"15min":1.8049999999999999},"last":1.1899999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2030000000000001,"15min":1.218},"min":{"1min":0.84399999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.623,"5min":1.623,"15min":1.6859999999999999},"last":1.536},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3700000000000001,"15min":1.4039999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.657,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.7450000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3109999999999999,"15min":1.333},"min":{"1min":0.95899999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":2.101},"last":1.506},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.298,"15min":1.331},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.9019999999999999,"5min":1.927,"15min":1.927},"last":1.7030000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0529999999999999,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3939999999999999,"15min":1.413},"min":{"1min":0.91600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.901}]},{"osd":7,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4650000000000001,"15min":1.522},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.9430000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.524,"5min":1.48,"15min":1.5109999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.0699999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.6060000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60172,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311668,"statfs":{"total":96636764160,"available":96575148032,"internally_reserved":0,"allocated":2953216,"data_stored":1447468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.88900000000000001},"min":{"1min":0.314,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.994},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.98899999999999999,"15min":1.048},"min":{"1min":0.58999999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6799999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0169999999999999,"15min":1.0640000000000001},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.635,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0589999999999999,"15min":1.1259999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3100000000000001,"5min":1.726,"15min":1.792},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.036,"15min":1.0669999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.8839999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.171,"15min":1.224},"min":{"1min":0.70499999999999996,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.6469999999999998},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.296,"15min":1.3520000000000001},"min":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.681,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.3029999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1240000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.90700000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.899,"15min":2.7719999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.383,"15min":1.427},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.681,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":1.1479999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3520000000000001,"15min":1.381},"min":{"1min":0.84599999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7270000000000001,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.45,"15min":1.4830000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.0660000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4670000000000001,"15min":1.516},"min":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7969999999999999,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4730000000000001,"15min":1.4870000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.927,"5min":2.1190000000000002,"15min":2.431},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:11:42.062 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:11:42.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.283+0000 7fdb33716640 1 Processor -- start 2023-12-01T03:11:42.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.283+0000 7fdb33716640 1 -- start start 2023-12-01T03:11:42.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.285+0000 7fdb33716640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb2c15e2f0 0x7fdb2c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.285+0000 7fdb33716640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdb2c06f2f0 0x7fdb2c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.285+0000 7fdb33716640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2c0add10 con 0x7fdb2c0adf10 2023-12-01T03:11:42.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.285+0000 7fdb33716640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c5760 con 0x7fdb2c06f2f0 2023-12-01T03:11:42.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.285+0000 7fdb33716640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c15e6d0 con 0x7fdb2c15e2f0 2023-12-01T03:11:42.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.285+0000 7fdb31cb3640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fdb2c0adf10 0x7fdb2c0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34736/0 (socket says 172.21.15.115:34736) 2023-12-01T03:11:42.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.285+0000 7fdb31cb3640 1 -- 172.21.15.115:0/861662662 learned_addr learned my addr 172.21.15.115:0/861662662 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642138502 0 0) 0x7fdb2c0add10 con 0x7fdb2c0adf10 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb30cb1640 1 --2- 172.21.15.115:0/861662662 >> v2:172.21.15.142:3300/0 conn(0x7fdb2c06f2f0 0x7fdb2c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb18003520 con 0x7fdb2c0adf10 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb314b2640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb2c15e2f0 0x7fdb2c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 221666701 0 0) 0x7fdb18003520 con 0x7fdb2c0adf10 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb2c15e2f0 msgr2=0x7fdb2c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb2c15e2f0 0x7fdb2c0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 >> v2:172.21.15.142:3300/0 conn(0x7fdb2c06f2f0 msgr2=0x7fdb2c0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 --2- 172.21.15.115:0/861662662 >> v2:172.21.15.142:3300/0 conn(0x7fdb2c06f2f0 0x7fdb2c0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c0cf7b0 con 0x7fdb2c0adf10 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fdb240019f0 con 0x7fdb2c0adf10 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 >> v1:172.21.15.115:6789/0 conn(0x7fdb2c0adf10 legacy=0x7fdb2c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:42.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb2c15e2f0 0x7fdb180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb18004010 0x7fdb180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb18004d00 0x7fdb18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c5760 con 0x7fdb18004010 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c15e6d0 con 0x7fdb18004d00 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1800bc40 con 0x7fdb2c15e2f0 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.286+0000 7fdb30cb1640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb18004010 0x7fdb180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.287+0000 7fdb30cb1640 1 -- 172.21.15.115:0/861662662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb2c15e2f0 msgr2=0x7fdb180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.287+0000 7fdb30cb1640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb2c15e2f0 0x7fdb180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.287+0000 7fdb31cb3640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb18004d00 0x7fdb18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.287+0000 7fdb30cb1640 1 -- 172.21.15.115:0/861662662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb18004d00 msgr2=0x7fdb18005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.287+0000 7fdb30cb1640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb18004d00 0x7fdb18005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.287+0000 7fdb30cb1640 1 -- 172.21.15.115:0/861662662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18003520 con 0x7fdb18004010 2023-12-01T03:11:42.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.288+0000 7fdb30cb1640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb18004010 0x7fdb180047c0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00ba30 tx=0x7fdb180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e568b9a8e7719c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:42.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.288+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c032680 con 0x7fdb18004010 2023-12-01T03:11:42.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.288+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c0327e0 con 0x7fdb18004010 2023-12-01T03:11:42.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.288+0000 7fdb23fff640 1 -- 172.21.15.115:0/861662662 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c032b70 con 0x7fdb18004010 2023-12-01T03:11:42.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.289+0000 7fdb33716640 1 -- 172.21.15.115:0/861662662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb18004010 msgr2=0x7fdb180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:42.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.289+0000 7fdb33716640 1 --2- 172.21.15.115:0/861662662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb18004010 0x7fdb180047c0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00ba30 tx=0x7fdb180057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.289+0000 7fdb31cb3640 1 -- 172.21.15.115:0/861662662 reap_dead start 2023-12-01T03:11:42.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.289+0000 7fdb33716640 1 -- 172.21.15.115:0/861662662 shutdown_connections 2023-12-01T03:11:42.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.289+0000 7fdb33716640 1 -- 172.21.15.115:0/861662662 >> 172.21.15.115:0/861662662 conn(0x7fdb2c0ce880 msgr2=0x7fdb2c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:42.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.289+0000 7fdb33716640 1 -- 172.21.15.115:0/861662662 shutdown_connections 2023-12-01T03:11:42.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.289+0000 7fdb33716640 1 -- 172.21.15.115:0/861662662 wait complete. 2023-12-01T03:11:42.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.291+0000 7fdb33716640 1 Processor -- start 2023-12-01T03:11:42.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.291+0000 7fdb33716640 1 -- start start 2023-12-01T03:11:42.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb33716640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb18004010 0x7fdb2c0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb33716640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb2c150c10 0x7fdb2c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb33716640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb2c15c9b0 0x7fdb2c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb33716640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c0cc550 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb33716640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c3a90 con 0x7fdb2c150c10 2023-12-01T03:11:42.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb33716640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c3d70 con 0x7fdb18004010 2023-12-01T03:11:42.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb31cb3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb2c15c9b0 0x7fdb2c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb31cb3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb2c15c9b0 0x7fdb2c0ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37582/0 (socket says 172.21.15.115:37582) 2023-12-01T03:11:42.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb31cb3640 1 -- 172.21.15.115:0/1118991082 learned_addr learned my addr 172.21.15.115:0/1118991082 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:42.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb30cb1640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb2c150c10 0x7fdb2c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb31cb3640 1 -- 172.21.15.115:0/1118991082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb18004010 msgr2=0x7fdb2c0c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:42.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb31cb3640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb18004010 0x7fdb2c0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb31cb3640 1 -- 172.21.15.115:0/1118991082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb2c150c10 msgr2=0x7fdb2c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:42.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb31cb3640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb2c150c10 0x7fdb2c15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.292+0000 7fdb31cb3640 1 -- 172.21.15.115:0/1118991082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c0c4050 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.293+0000 7fdb31cb3640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb2c15c9b0 0x7fdb2c0ac8a0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c06ed00 tx=0x7fdb2400df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:42.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.293+0000 7fdb21ffb640 1 -- 172.21.15.115:0/1118991082 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400b860 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.293+0000 7fdb21ffb640 1 -- 172.21.15.115:0/1118991082 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2400b9c0 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.293+0000 7fdb33716640 1 -- 172.21.15.115:0/1118991082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2c0c4330 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.293+0000 7fdb21ffb640 1 -- 172.21.15.115:0/1118991082 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400bd70 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.293+0000 7fdb33716640 1 -- 172.21.15.115:0/1118991082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2c151090 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.294+0000 7fdb33716640 1 -- 172.21.15.115:0/1118991082 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2c06fda0 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.295+0000 7fdb21ffb640 1 -- 172.21.15.115:0/1118991082 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fdb24007110 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.296+0000 7fdb21ffb640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdb1006a6a0 0x7fdb1006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.296+0000 7fdb21ffb640 1 -- 172.21.15.115:0/1118991082 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fdb24089150 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.297+0000 7fdb314b2640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdb1006a6a0 0x7fdb1006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.297+0000 7fdb314b2640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdb1006a6a0 0x7fdb1006cb60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b730 tx=0x7fdb28008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:42.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.301+0000 7fdb21ffb640 1 -- 172.21.15.115:0/1118991082 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fdb24054330 con 0x7fdb2c15c9b0 2023-12-01T03:11:42.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.487+0000 7fdb33716640 1 -- 172.21.15.115:0/1118991082 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb2c0b43d0 con 0x7fdb1006a6a0 2023-12-01T03:11:42.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.499+0000 7fdb21ffb640 1 -- 172.21.15.115:0/1118991082 <== mgr.11121 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287581 (secure 0 0 0) 0x7fdb2800d530 con 0x7fdb1006a6a0 2023-12-01T03:11:42.504 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:42.506 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:11:42.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 -- 172.21.15.115:0/1118991082 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdb1006a6a0 msgr2=0x7fdb1006cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:42.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdb1006a6a0 0x7fdb1006cb60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b730 tx=0x7fdb28008040 comp rx=0 tx=0).stop 2023-12-01T03:11:42.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 -- 172.21.15.115:0/1118991082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb2c15c9b0 msgr2=0x7fdb2c0ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:42.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb2c15c9b0 0x7fdb2c0ac8a0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c06ed00 tx=0x7fdb2400df70 comp rx=0 tx=0).stop 2023-12-01T03:11:42.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 -- 172.21.15.115:0/1118991082 shutdown_connections 2023-12-01T03:11:42.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fdb1006a6a0 0x7fdb1006cb60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdb2c15c9b0 0x7fdb2c0ac8a0 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb2c150c10 0x7fdb2c15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 --2- 172.21.15.115:0/1118991082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb18004010 0x7fdb2c0c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 -- 172.21.15.115:0/1118991082 >> 172.21.15.115:0/1118991082 conn(0x7fdb2c0ce880 msgr2=0x7fdb2c0ca330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:42.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 -- 172.21.15.115:0/1118991082 shutdown_connections 2023-12-01T03:11:42.509 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.506+0000 7fdb33716640 1 -- 172.21.15.115:0/1118991082 wait complete. 2023-12-01T03:11:42.527 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-12-01T03:11:41.823933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":624,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":419496,"kb_used_data":24296,"kb_used_omap":0,"kb_used_meta":395136,"kb_avail":754555224,"statfs":{"total":773094113280,"available":772664549376,"internally_reserved":0,"allocated":24879104,"data_stored":12765204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":404619264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.641995"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222918+0000","last_change":"2023-12-01T03:11:32.212603+0000","last_active":"2023-12-01T03:11:32.222918+0000","last_peered":"2023-12-01T03:11:32.222918+0000","last_clean":"2023-12-01T03:11:32.222918+0000","last_became_active":"2023-12-01T03:11:32.212287+0000","last_became_peered":"2023-12-01T03:11:32.212287+0000","last_unstale":"2023-12-01T03:11:32.222918+0000","last_undegraded":"2023-12-01T03:11:32.222918+0000","last_fullsized":"2023-12-01T03:11:32.222918+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220449+0000","last_change":"2023-12-01T03:11:32.217098+0000","last_active":"2023-12-01T03:11:32.220449+0000","last_peered":"2023-12-01T03:11:32.220449+0000","last_clean":"2023-12-01T03:11:32.220449+0000","last_became_active":"2023-12-01T03:11:32.216867+0000","last_became_peered":"2023-12-01T03:11:32.216867+0000","last_unstale":"2023-12-01T03:11:32.220449+0000","last_undegraded":"2023-12-01T03:11:32.220449+0000","last_fullsized":"2023-12-01T03:11:32.220449+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:10.849908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210187+0000","last_change":"2023-12-01T03:11:32.208423+0000","last_active":"2023-12-01T03:11:32.210187+0000","last_peered":"2023-12-01T03:11:32.210187+0000","last_clean":"2023-12-01T03:11:32.210187+0000","last_became_active":"2023-12-01T03:11:32.208113+0000","last_became_peered":"2023-12-01T03:11:32.208113+0000","last_unstale":"2023-12-01T03:11:32.210187+0000","last_undegraded":"2023-12-01T03:11:32.210187+0000","last_fullsized":"2023-12-01T03:11:32.210187+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:38.174827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213451+0000","last_change":"2023-12-01T03:11:32.207840+0000","last_active":"2023-12-01T03:11:32.213451+0000","last_peered":"2023-12-01T03:11:32.213451+0000","last_clean":"2023-12-01T03:11:32.213451+0000","last_became_active":"2023-12-01T03:11:32.207048+0000","last_became_peered":"2023-12-01T03:11:32.207048+0000","last_unstale":"2023-12-01T03:11:32.213451+0000","last_undegraded":"2023-12-01T03:11:32.213451+0000","last_fullsized":"2023-12-01T03:11:32.213451+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:43.961392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212780+0000","last_change":"2023-12-01T03:11:32.206960+0000","last_active":"2023-12-01T03:11:32.212780+0000","last_peered":"2023-12-01T03:11:32.212780+0000","last_clean":"2023-12-01T03:11:32.212780+0000","last_became_active":"2023-12-01T03:11:32.203345+0000","last_became_peered":"2023-12-01T03:11:32.203345+0000","last_unstale":"2023-12-01T03:11:32.212780+0000","last_undegraded":"2023-12-01T03:11:32.212780+0000","last_fullsized":"2023-12-01T03:11:32.212780+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211014+0000","last_change":"2023-12-01T03:11:32.205496+0000","last_active":"2023-12-01T03:11:32.211014+0000","last_peered":"2023-12-01T03:11:32.211014+0000","last_clean":"2023-12-01T03:11:32.211014+0000","last_became_active":"2023-12-01T03:11:32.205074+0000","last_became_peered":"2023-12-01T03:11:32.205074+0000","last_unstale":"2023-12-01T03:11:32.211014+0000","last_undegraded":"2023-12-01T03:11:32.211014+0000","last_fullsized":"2023-12-01T03:11:32.211014+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209538+0000","last_change":"2023-12-01T03:11:32.202465+0000","last_active":"2023-12-01T03:11:32.209538+0000","last_peered":"2023-12-01T03:11:32.209538+0000","last_clean":"2023-12-01T03:11:32.209538+0000","last_became_active":"2023-12-01T03:11:32.200165+0000","last_became_peered":"2023-12-01T03:11:32.200165+0000","last_unstale":"2023-12-01T03:11:32.209538+0000","last_undegraded":"2023-12-01T03:11:32.209538+0000","last_fullsized":"2023-12-01T03:11:32.209538+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:19.277681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220081+0000","last_change":"2023-12-01T03:11:32.217253+0000","last_active":"2023-12-01T03:11:32.220081+0000","last_peered":"2023-12-01T03:11:32.220081+0000","last_clean":"2023-12-01T03:11:32.220081+0000","last_became_active":"2023-12-01T03:11:32.216425+0000","last_became_peered":"2023-12-01T03:11:32.216425+0000","last_unstale":"2023-12-01T03:11:32.220081+0000","last_undegraded":"2023-12-01T03:11:32.220081+0000","last_fullsized":"2023-12-01T03:11:32.220081+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:51.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.214233+0000","last_change":"2023-12-01T03:11:32.211881+0000","last_active":"2023-12-01T03:11:32.214233+0000","last_peered":"2023-12-01T03:11:32.214233+0000","last_clean":"2023-12-01T03:11:32.214233+0000","last_became_active":"2023-12-01T03:11:32.211602+0000","last_became_peered":"2023-12-01T03:11:32.211602+0000","last_unstale":"2023-12-01T03:11:32.214233+0000","last_undegraded":"2023-12-01T03:11:32.214233+0000","last_fullsized":"2023-12-01T03:11:32.214233+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.201292+0000","last_change":"2023-12-01T03:11:32.200099+0000","last_active":"2023-12-01T03:11:32.201292+0000","last_peered":"2023-12-01T03:11:32.201292+0000","last_clean":"2023-12-01T03:11:32.201292+0000","last_became_active":"2023-12-01T03:11:32.199703+0000","last_became_peered":"2023-12-01T03:11:32.199703+0000","last_unstale":"2023-12-01T03:11:32.201292+0000","last_undegraded":"2023-12-01T03:11:32.201292+0000","last_fullsized":"2023-12-01T03:11:32.201292+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:44:42.356924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213688+0000","last_change":"2023-12-01T03:11:32.211517+0000","last_active":"2023-12-01T03:11:32.213688+0000","last_peered":"2023-12-01T03:11:32.213688+0000","last_clean":"2023-12-01T03:11:32.213688+0000","last_became_active":"2023-12-01T03:11:32.211273+0000","last_became_peered":"2023-12-01T03:11:32.211273+0000","last_unstale":"2023-12-01T03:11:32.213688+0000","last_undegraded":"2023-12-01T03:11:32.213688+0000","last_fullsized":"2023-12-01T03:11:32.213688+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:35.627224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.206460+0000","last_change":"2023-12-01T03:11:32.202825+0000","last_active":"2023-12-01T03:11:32.206460+0000","last_peered":"2023-12-01T03:11:32.206460+0000","last_clean":"2023-12-01T03:11:32.206460+0000","last_became_active":"2023-12-01T03:11:32.202452+0000","last_became_peered":"2023-12-01T03:11:32.202452+0000","last_unstale":"2023-12-01T03:11:32.206460+0000","last_undegraded":"2023-12-01T03:11:32.206460+0000","last_fullsized":"2023-12-01T03:11:32.206460+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225985+0000","last_change":"2023-12-01T03:11:32.224888+0000","last_active":"2023-12-01T03:11:32.225985+0000","last_peered":"2023-12-01T03:11:32.225985+0000","last_clean":"2023-12-01T03:11:32.225985+0000","last_became_active":"2023-12-01T03:11:32.224585+0000","last_became_peered":"2023-12-01T03:11:32.224585+0000","last_unstale":"2023-12-01T03:11:32.225985+0000","last_undegraded":"2023-12-01T03:11:32.225985+0000","last_fullsized":"2023-12-01T03:11:32.225985+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:04.079396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210154+0000","last_change":"2023-12-01T03:11:32.208938+0000","last_active":"2023-12-01T03:11:32.210154+0000","last_peered":"2023-12-01T03:11:32.210154+0000","last_clean":"2023-12-01T03:11:32.210154+0000","last_became_active":"2023-12-01T03:11:32.208695+0000","last_became_peered":"2023-12-01T03:11:32.208695+0000","last_unstale":"2023-12-01T03:11:32.210154+0000","last_undegraded":"2023-12-01T03:11:32.210154+0000","last_fullsized":"2023-12-01T03:11:32.210154+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:21.336618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218399+0000","last_change":"2023-12-01T03:11:32.214557+0000","last_active":"2023-12-01T03:11:32.218399+0000","last_peered":"2023-12-01T03:11:32.218399+0000","last_clean":"2023-12-01T03:11:32.218399+0000","last_became_active":"2023-12-01T03:11:32.214252+0000","last_became_peered":"2023-12-01T03:11:32.214252+0000","last_unstale":"2023-12-01T03:11:32.218399+0000","last_undegraded":"2023-12-01T03:11:32.218399+0000","last_fullsized":"2023-12-01T03:11:32.218399+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218473+0000","last_change":"2023-12-01T03:11:32.212018+0000","last_active":"2023-12-01T03:11:32.218473+0000","last_peered":"2023-12-01T03:11:32.218473+0000","last_clean":"2023-12-01T03:11:32.218473+0000","last_became_active":"2023-12-01T03:11:32.211742+0000","last_became_peered":"2023-12-01T03:11:32.211742+0000","last_unstale":"2023-12-01T03:11:32.218473+0000","last_undegraded":"2023-12-01T03:11:32.218473+0000","last_fullsized":"2023-12-01T03:11:32.218473+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222963+0000","last_change":"2023-12-01T03:11:32.219580+0000","last_active":"2023-12-01T03:11:32.222963+0000","last_peered":"2023-12-01T03:11:32.222963+0000","last_clean":"2023-12-01T03:11:32.222963+0000","last_became_active":"2023-12-01T03:11:32.218935+0000","last_became_peered":"2023-12-01T03:11:32.218935+0000","last_unstale":"2023-12-01T03:11:32.222963+0000","last_undegraded":"2023-12-01T03:11:32.222963+0000","last_fullsized":"2023-12-01T03:11:32.222963+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223790+0000","last_change":"2023-12-01T03:11:32.222587+0000","last_active":"2023-12-01T03:11:32.223790+0000","last_peered":"2023-12-01T03:11:32.223790+0000","last_clean":"2023-12-01T03:11:32.223790+0000","last_became_active":"2023-12-01T03:11:32.222267+0000","last_became_peered":"2023-12-01T03:11:32.222267+0000","last_unstale":"2023-12-01T03:11:32.223790+0000","last_undegraded":"2023-12-01T03:11:32.223790+0000","last_fullsized":"2023-12-01T03:11:32.223790+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:42.720564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210046+0000","last_change":"2023-12-01T03:11:32.204026+0000","last_active":"2023-12-01T03:11:32.210046+0000","last_peered":"2023-12-01T03:11:32.210046+0000","last_clean":"2023-12-01T03:11:32.210046+0000","last_became_active":"2023-12-01T03:11:32.200832+0000","last_became_peered":"2023-12-01T03:11:32.200832+0000","last_unstale":"2023-12-01T03:11:32.210046+0000","last_undegraded":"2023-12-01T03:11:32.210046+0000","last_fullsized":"2023-12-01T03:11:32.210046+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.215183+0000","last_change":"2023-12-01T03:11:32.207642+0000","last_active":"2023-12-01T03:11:32.215183+0000","last_peered":"2023-12-01T03:11:32.215183+0000","last_clean":"2023-12-01T03:11:32.215183+0000","last_became_active":"2023-12-01T03:11:32.207344+0000","last_became_peered":"2023-12-01T03:11:32.207344+0000","last_unstale":"2023-12-01T03:11:32.215183+0000","last_undegraded":"2023-12-01T03:11:32.215183+0000","last_fullsized":"2023-12-01T03:11:32.215183+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:10.441679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223757+0000","last_change":"2023-12-01T03:11:32.220696+0000","last_active":"2023-12-01T03:11:32.223757+0000","last_peered":"2023-12-01T03:11:32.223757+0000","last_clean":"2023-12-01T03:11:32.223757+0000","last_became_active":"2023-12-01T03:11:32.220353+0000","last_became_peered":"2023-12-01T03:11:32.220353+0000","last_unstale":"2023-12-01T03:11:32.223757+0000","last_undegraded":"2023-12-01T03:11:32.223757+0000","last_fullsized":"2023-12-01T03:11:32.223757+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:53:43.178969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211636+0000","last_change":"2023-12-01T03:11:32.207709+0000","last_active":"2023-12-01T03:11:32.211636+0000","last_peered":"2023-12-01T03:11:32.211636+0000","last_clean":"2023-12-01T03:11:32.211636+0000","last_became_active":"2023-12-01T03:11:32.205510+0000","last_became_peered":"2023-12-01T03:11:32.205510+0000","last_unstale":"2023-12-01T03:11:32.211636+0000","last_undegraded":"2023-12-01T03:11:32.211636+0000","last_fullsized":"2023-12-01T03:11:32.211636+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216610+0000","last_change":"2023-12-01T03:11:32.210643+0000","last_active":"2023-12-01T03:11:32.216610+0000","last_peered":"2023-12-01T03:11:32.216610+0000","last_clean":"2023-12-01T03:11:32.216610+0000","last_became_active":"2023-12-01T03:11:32.210367+0000","last_became_peered":"2023-12-01T03:11:32.210367+0000","last_unstale":"2023-12-01T03:11:32.216610+0000","last_undegraded":"2023-12-01T03:11:32.216610+0000","last_fullsized":"2023-12-01T03:11:32.216610+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:00.674458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.214317+0000","last_change":"2023-12-01T03:11:32.209242+0000","last_active":"2023-12-01T03:11:32.214317+0000","last_peered":"2023-12-01T03:11:32.214317+0000","last_clean":"2023-12-01T03:11:32.214317+0000","last_became_active":"2023-12-01T03:11:32.208985+0000","last_became_peered":"2023-12-01T03:11:32.208985+0000","last_unstale":"2023-12-01T03:11:32.214317+0000","last_undegraded":"2023-12-01T03:11:32.214317+0000","last_fullsized":"2023-12-01T03:11:32.214317+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:29.661974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223614+0000","last_change":"2023-12-01T03:11:32.220828+0000","last_active":"2023-12-01T03:11:32.223614+0000","last_peered":"2023-12-01T03:11:32.223614+0000","last_clean":"2023-12-01T03:11:32.223614+0000","last_became_active":"2023-12-01T03:11:32.219607+0000","last_became_peered":"2023-12-01T03:11:32.219607+0000","last_unstale":"2023-12-01T03:11:32.223614+0000","last_undegraded":"2023-12-01T03:11:32.223614+0000","last_fullsized":"2023-12-01T03:11:32.223614+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:09.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.205877+0000","last_change":"2023-12-01T03:11:32.198609+0000","last_active":"2023-12-01T03:11:32.205877+0000","last_peered":"2023-12-01T03:11:32.205877+0000","last_clean":"2023-12-01T03:11:32.205877+0000","last_became_active":"2023-12-01T03:11:32.197869+0000","last_became_peered":"2023-12-01T03:11:32.197869+0000","last_unstale":"2023-12-01T03:11:32.205877+0000","last_undegraded":"2023-12-01T03:11:32.205877+0000","last_fullsized":"2023-12-01T03:11:32.205877+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:35.975600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217603+0000","last_change":"2023-12-01T03:11:32.212954+0000","last_active":"2023-12-01T03:11:32.217603+0000","last_peered":"2023-12-01T03:11:32.217603+0000","last_clean":"2023-12-01T03:11:32.217603+0000","last_became_active":"2023-12-01T03:11:32.212678+0000","last_became_peered":"2023-12-01T03:11:32.212678+0000","last_unstale":"2023-12-01T03:11:32.217603+0000","last_undegraded":"2023-12-01T03:11:32.217603+0000","last_fullsized":"2023-12-01T03:11:32.217603+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:28.852429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220068+0000","last_change":"2023-12-01T03:11:32.215698+0000","last_active":"2023-12-01T03:11:32.220068+0000","last_peered":"2023-12-01T03:11:32.220068+0000","last_clean":"2023-12-01T03:11:32.220068+0000","last_became_active":"2023-12-01T03:11:32.214531+0000","last_became_peered":"2023-12-01T03:11:32.214531+0000","last_unstale":"2023-12-01T03:11:32.220068+0000","last_undegraded":"2023-12-01T03:11:32.220068+0000","last_fullsized":"2023-12-01T03:11:32.220068+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:14.716448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212370+0000","last_change":"2023-12-01T03:11:32.204327+0000","last_active":"2023-12-01T03:11:32.212370+0000","last_peered":"2023-12-01T03:11:32.212370+0000","last_clean":"2023-12-01T03:11:32.212370+0000","last_became_active":"2023-12-01T03:11:32.203765+0000","last_became_peered":"2023-12-01T03:11:32.203765+0000","last_unstale":"2023-12-01T03:11:32.212370+0000","last_undegraded":"2023-12-01T03:11:32.212370+0000","last_fullsized":"2023-12-01T03:11:32.212370+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:28:34.324085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213777+0000","last_change":"2023-12-01T03:11:32.212023+0000","last_active":"2023-12-01T03:11:32.213777+0000","last_peered":"2023-12-01T03:11:32.213777+0000","last_clean":"2023-12-01T03:11:32.213777+0000","last_became_active":"2023-12-01T03:11:32.210975+0000","last_became_peered":"2023-12-01T03:11:32.210975+0000","last_unstale":"2023-12-01T03:11:32.213777+0000","last_undegraded":"2023-12-01T03:11:32.213777+0000","last_fullsized":"2023-12-01T03:11:32.213777+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217256+0000","last_change":"2023-12-01T03:11:32.213074+0000","last_active":"2023-12-01T03:11:32.217256+0000","last_peered":"2023-12-01T03:11:32.217256+0000","last_clean":"2023-12-01T03:11:32.217256+0000","last_became_active":"2023-12-01T03:11:32.212775+0000","last_became_peered":"2023-12-01T03:11:32.212775+0000","last_unstale":"2023-12-01T03:11:32.217256+0000","last_undegraded":"2023-12-01T03:11:32.217256+0000","last_fullsized":"2023-12-01T03:11:32.217256+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:36.571608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225974+0000","last_change":"2023-12-01T03:11:32.225479+0000","last_active":"2023-12-01T03:11:32.225974+0000","last_peered":"2023-12-01T03:11:32.225974+0000","last_clean":"2023-12-01T03:11:32.225974+0000","last_became_active":"2023-12-01T03:11:32.225154+0000","last_became_peered":"2023-12-01T03:11:32.225154+0000","last_unstale":"2023-12-01T03:11:32.225974+0000","last_undegraded":"2023-12-01T03:11:32.225974+0000","last_fullsized":"2023-12-01T03:11:32.225974+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:14.760284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220523+0000","last_change":"2023-12-01T03:11:32.214972+0000","last_active":"2023-12-01T03:11:32.220523+0000","last_peered":"2023-12-01T03:11:32.220523+0000","last_clean":"2023-12-01T03:11:32.220523+0000","last_became_active":"2023-12-01T03:11:32.214729+0000","last_became_peered":"2023-12-01T03:11:32.214729+0000","last_unstale":"2023-12-01T03:11:32.220523+0000","last_undegraded":"2023-12-01T03:11:32.220523+0000","last_fullsized":"2023-12-01T03:11:32.220523+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:24.244094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1f","version":"234'4","reported_seq":15,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225296+0000","last_change":"2023-12-01T03:11:31.680864+0000","last_active":"2023-12-01T03:11:32.225296+0000","last_peered":"2023-12-01T03:11:32.225296+0000","last_clean":"2023-12-01T03:11:32.225296+0000","last_became_active":"2023-12-01T03:11:31.680279+0000","last_became_peered":"2023-12-01T03:11:31.680279+0000","last_unstale":"2023-12-01T03:11:32.225296+0000","last_undegraded":"2023-12-01T03:11:32.225296+0000","last_fullsized":"2023-12-01T03:11:32.225296+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.377359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217958+0000","last_change":"2023-12-01T03:11:32.211743+0000","last_active":"2023-12-01T03:11:32.217958+0000","last_peered":"2023-12-01T03:11:32.217958+0000","last_clean":"2023-12-01T03:11:32.217958+0000","last_became_active":"2023-12-01T03:11:32.211443+0000","last_became_peered":"2023-12-01T03:11:32.211443+0000","last_unstale":"2023-12-01T03:11:32.217958+0000","last_undegraded":"2023-12-01T03:11:32.217958+0000","last_fullsized":"2023-12-01T03:11:32.217958+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224872+0000","last_change":"2023-12-01T03:11:31.679798+0000","last_active":"2023-12-01T03:11:32.224872+0000","last_peered":"2023-12-01T03:11:32.224872+0000","last_clean":"2023-12-01T03:11:32.224872+0000","last_became_active":"2023-12-01T03:11:31.679030+0000","last_became_peered":"2023-12-01T03:11:31.679030+0000","last_unstale":"2023-12-01T03:11:32.224872+0000","last_undegraded":"2023-12-01T03:11:32.224872+0000","last_fullsized":"2023-12-01T03:11:32.224872+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:49.261314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223595+0000","last_change":"2023-12-01T03:11:32.222760+0000","last_active":"2023-12-01T03:11:32.223595+0000","last_peered":"2023-12-01T03:11:32.223595+0000","last_clean":"2023-12-01T03:11:32.223595+0000","last_became_active":"2023-12-01T03:11:32.222500+0000","last_became_peered":"2023-12-01T03:11:32.222500+0000","last_unstale":"2023-12-01T03:11:32.223595+0000","last_undegraded":"2023-12-01T03:11:32.223595+0000","last_fullsized":"2023-12-01T03:11:32.223595+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:17.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218820+0000","last_change":"2023-12-01T03:11:31.663745+0000","last_active":"2023-12-01T03:11:32.218820+0000","last_peered":"2023-12-01T03:11:32.218820+0000","last_clean":"2023-12-01T03:11:32.218820+0000","last_became_active":"2023-12-01T03:11:31.662706+0000","last_became_peered":"2023-12-01T03:11:31.662706+0000","last_unstale":"2023-12-01T03:11:32.218820+0000","last_undegraded":"2023-12-01T03:11:32.218820+0000","last_fullsized":"2023-12-01T03:11:32.218820+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:42.945449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217635+0000","last_change":"2023-12-01T03:11:32.213316+0000","last_active":"2023-12-01T03:11:32.217635+0000","last_peered":"2023-12-01T03:11:32.217635+0000","last_clean":"2023-12-01T03:11:32.217635+0000","last_became_active":"2023-12-01T03:11:32.213070+0000","last_became_peered":"2023-12-01T03:11:32.213070+0000","last_unstale":"2023-12-01T03:11:32.217635+0000","last_undegraded":"2023-12-01T03:11:32.217635+0000","last_fullsized":"2023-12-01T03:11:32.217635+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:22:51.687377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216557+0000","last_change":"2023-12-01T03:11:31.660758+0000","last_active":"2023-12-01T03:11:32.216557+0000","last_peered":"2023-12-01T03:11:32.216557+0000","last_clean":"2023-12-01T03:11:32.216557+0000","last_became_active":"2023-12-01T03:11:31.660001+0000","last_became_peered":"2023-12-01T03:11:31.660001+0000","last_unstale":"2023-12-01T03:11:32.216557+0000","last_undegraded":"2023-12-01T03:11:32.216557+0000","last_fullsized":"2023-12-01T03:11:32.216557+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218251+0000","last_change":"2023-12-01T03:11:32.213961+0000","last_active":"2023-12-01T03:11:32.218251+0000","last_peered":"2023-12-01T03:11:32.218251+0000","last_clean":"2023-12-01T03:11:32.218251+0000","last_became_active":"2023-12-01T03:11:32.213692+0000","last_became_peered":"2023-12-01T03:11:32.213692+0000","last_unstale":"2023-12-01T03:11:32.218251+0000","last_undegraded":"2023-12-01T03:11:32.218251+0000","last_fullsized":"2023-12-01T03:11:32.218251+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224769+0000","last_change":"2023-12-01T03:11:31.681531+0000","last_active":"2023-12-01T03:11:32.224769+0000","last_peered":"2023-12-01T03:11:32.224769+0000","last_clean":"2023-12-01T03:11:32.224769+0000","last_became_active":"2023-12-01T03:11:31.680782+0000","last_became_peered":"2023-12-01T03:11:31.680782+0000","last_unstale":"2023-12-01T03:11:32.224769+0000","last_undegraded":"2023-12-01T03:11:32.224769+0000","last_fullsized":"2023-12-01T03:11:32.224769+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:18.018055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211538+0000","last_change":"2023-12-01T03:11:32.206659+0000","last_active":"2023-12-01T03:11:32.211538+0000","last_peered":"2023-12-01T03:11:32.211538+0000","last_clean":"2023-12-01T03:11:32.211538+0000","last_became_active":"2023-12-01T03:11:32.206346+0000","last_became_peered":"2023-12-01T03:11:32.206346+0000","last_unstale":"2023-12-01T03:11:32.211538+0000","last_undegraded":"2023-12-01T03:11:32.211538+0000","last_fullsized":"2023-12-01T03:11:32.211538+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.196693+0000","last_change":"2023-12-01T03:11:31.663166+0000","last_active":"2023-12-01T03:11:32.196693+0000","last_peered":"2023-12-01T03:11:32.196693+0000","last_clean":"2023-12-01T03:11:32.196693+0000","last_became_active":"2023-12-01T03:11:31.662263+0000","last_became_peered":"2023-12-01T03:11:31.662263+0000","last_unstale":"2023-12-01T03:11:32.196693+0000","last_undegraded":"2023-12-01T03:11:32.196693+0000","last_fullsized":"2023-12-01T03:11:32.196693+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:57:00.965499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216627+0000","last_change":"2023-12-01T03:11:32.210988+0000","last_active":"2023-12-01T03:11:32.216627+0000","last_peered":"2023-12-01T03:11:32.216627+0000","last_clean":"2023-12-01T03:11:32.216627+0000","last_became_active":"2023-12-01T03:11:32.207844+0000","last_became_peered":"2023-12-01T03:11:32.207844+0000","last_unstale":"2023-12-01T03:11:32.216627+0000","last_undegraded":"2023-12-01T03:11:32.216627+0000","last_fullsized":"2023-12-01T03:11:32.216627+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.206110+0000","last_change":"2023-12-01T03:11:31.670524+0000","last_active":"2023-12-01T03:11:32.206110+0000","last_peered":"2023-12-01T03:11:32.206110+0000","last_clean":"2023-12-01T03:11:32.206110+0000","last_became_active":"2023-12-01T03:11:31.670190+0000","last_became_peered":"2023-12-01T03:11:31.670190+0000","last_unstale":"2023-12-01T03:11:32.206110+0000","last_undegraded":"2023-12-01T03:11:32.206110+0000","last_fullsized":"2023-12-01T03:11:32.206110+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:45.874739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.214311+0000","last_change":"2023-12-01T03:11:32.212987+0000","last_active":"2023-12-01T03:11:32.214311+0000","last_peered":"2023-12-01T03:11:32.214311+0000","last_clean":"2023-12-01T03:11:32.214311+0000","last_became_active":"2023-12-01T03:11:32.212426+0000","last_became_peered":"2023-12-01T03:11:32.212426+0000","last_unstale":"2023-12-01T03:11:32.214311+0000","last_undegraded":"2023-12-01T03:11:32.214311+0000","last_fullsized":"2023-12-01T03:11:32.214311+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.219351+0000","last_change":"2023-12-01T03:11:31.676322+0000","last_active":"2023-12-01T03:11:32.219351+0000","last_peered":"2023-12-01T03:11:32.219351+0000","last_clean":"2023-12-01T03:11:32.219351+0000","last_became_active":"2023-12-01T03:11:31.674451+0000","last_became_peered":"2023-12-01T03:11:31.674451+0000","last_unstale":"2023-12-01T03:11:32.219351+0000","last_undegraded":"2023-12-01T03:11:32.219351+0000","last_fullsized":"2023-12-01T03:11:32.219351+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:44.820233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222710+0000","last_change":"2023-12-01T03:11:32.219383+0000","last_active":"2023-12-01T03:11:32.222710+0000","last_peered":"2023-12-01T03:11:32.222710+0000","last_clean":"2023-12-01T03:11:32.222710+0000","last_became_active":"2023-12-01T03:11:32.219139+0000","last_became_peered":"2023-12-01T03:11:32.219139+0000","last_unstale":"2023-12-01T03:11:32.222710+0000","last_undegraded":"2023-12-01T03:11:32.222710+0000","last_fullsized":"2023-12-01T03:11:32.222710+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:40.557913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.192088+0000","last_change":"2023-12-01T03:11:31.662929+0000","last_active":"2023-12-01T03:11:32.192088+0000","last_peered":"2023-12-01T03:11:32.192088+0000","last_clean":"2023-12-01T03:11:32.192088+0000","last_became_active":"2023-12-01T03:11:31.662617+0000","last_became_peered":"2023-12-01T03:11:31.662617+0000","last_unstale":"2023-12-01T03:11:32.192088+0000","last_undegraded":"2023-12-01T03:11:32.192088+0000","last_fullsized":"2023-12-01T03:11:32.192088+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:34.611280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.210926+0000","last_change":"2023-12-01T03:11:32.209244+0000","last_active":"2023-12-01T03:11:32.210926+0000","last_peered":"2023-12-01T03:11:32.210926+0000","last_clean":"2023-12-01T03:11:32.210926+0000","last_became_active":"2023-12-01T03:11:32.206294+0000","last_became_peered":"2023-12-01T03:11:32.206294+0000","last_unstale":"2023-12-01T03:11:32.210926+0000","last_undegraded":"2023-12-01T03:11:32.210926+0000","last_fullsized":"2023-12-01T03:11:32.210926+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"234'3","reported_seq":14,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218935+0000","last_change":"2023-12-01T03:11:31.664798+0000","last_active":"2023-12-01T03:11:32.218935+0000","last_peered":"2023-12-01T03:11:32.218935+0000","last_clean":"2023-12-01T03:11:32.218935+0000","last_became_active":"2023-12-01T03:11:31.664276+0000","last_became_peered":"2023-12-01T03:11:31.664276+0000","last_unstale":"2023-12-01T03:11:32.218935+0000","last_undegraded":"2023-12-01T03:11:32.218935+0000","last_fullsized":"2023-12-01T03:11:32.218935+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:18:18.909277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.215970+0000","last_change":"2023-12-01T03:11:32.213607+0000","last_active":"2023-12-01T03:11:32.215970+0000","last_peered":"2023-12-01T03:11:32.215970+0000","last_clean":"2023-12-01T03:11:32.215970+0000","last_became_active":"2023-12-01T03:11:32.211875+0000","last_became_peered":"2023-12-01T03:11:32.211875+0000","last_unstale":"2023-12-01T03:11:32.215970+0000","last_undegraded":"2023-12-01T03:11:32.215970+0000","last_fullsized":"2023-12-01T03:11:32.215970+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"234'6","reported_seq":17,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223391+0000","last_change":"2023-12-01T03:11:31.664162+0000","last_active":"2023-12-01T03:11:32.223391+0000","last_peered":"2023-12-01T03:11:32.223391+0000","last_clean":"2023-12-01T03:11:32.223391+0000","last_became_active":"2023-12-01T03:11:31.663854+0000","last_became_peered":"2023-12-01T03:11:31.663854+0000","last_unstale":"2023-12-01T03:11:32.223391+0000","last_undegraded":"2023-12-01T03:11:32.223391+0000","last_fullsized":"2023-12-01T03:11:32.223391+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:15.367696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209163+0000","last_change":"2023-12-01T03:11:32.203625+0000","last_active":"2023-12-01T03:11:32.209163+0000","last_peered":"2023-12-01T03:11:32.209163+0000","last_clean":"2023-12-01T03:11:32.209163+0000","last_became_active":"2023-12-01T03:11:32.203300+0000","last_became_peered":"2023-12-01T03:11:32.203300+0000","last_unstale":"2023-12-01T03:11:32.209163+0000","last_undegraded":"2023-12-01T03:11:32.209163+0000","last_fullsized":"2023-12-01T03:11:32.209163+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:46.890682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222667+0000","last_change":"2023-12-01T03:11:31.665987+0000","last_active":"2023-12-01T03:11:32.222667+0000","last_peered":"2023-12-01T03:11:32.222667+0000","last_clean":"2023-12-01T03:11:32.222667+0000","last_became_active":"2023-12-01T03:11:31.665550+0000","last_became_peered":"2023-12-01T03:11:31.665550+0000","last_unstale":"2023-12-01T03:11:32.222667+0000","last_undegraded":"2023-12-01T03:11:32.222667+0000","last_fullsized":"2023-12-01T03:11:32.222667+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:16.324155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209603+0000","last_change":"2023-12-01T03:11:32.208104+0000","last_active":"2023-12-01T03:11:32.209603+0000","last_peered":"2023-12-01T03:11:32.209603+0000","last_clean":"2023-12-01T03:11:32.209603+0000","last_became_active":"2023-12-01T03:11:32.207859+0000","last_became_peered":"2023-12-01T03:11:32.207859+0000","last_unstale":"2023-12-01T03:11:32.209603+0000","last_undegraded":"2023-12-01T03:11:32.209603+0000","last_fullsized":"2023-12-01T03:11:32.209603+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.188460+0000","last_change":"2023-12-01T03:11:31.675387+0000","last_active":"2023-12-01T03:11:32.188460+0000","last_peered":"2023-12-01T03:11:32.188460+0000","last_clean":"2023-12-01T03:11:32.188460+0000","last_became_active":"2023-12-01T03:11:31.675108+0000","last_became_peered":"2023-12-01T03:11:31.675108+0000","last_unstale":"2023-12-01T03:11:32.188460+0000","last_undegraded":"2023-12-01T03:11:32.188460+0000","last_fullsized":"2023-12-01T03:11:32.188460+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.836486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211907+0000","last_change":"2023-12-01T03:11:32.204685+0000","last_active":"2023-12-01T03:11:32.211907+0000","last_peered":"2023-12-01T03:11:32.211907+0000","last_clean":"2023-12-01T03:11:32.211907+0000","last_became_active":"2023-12-01T03:11:32.204422+0000","last_became_peered":"2023-12-01T03:11:32.204422+0000","last_unstale":"2023-12-01T03:11:32.211907+0000","last_undegraded":"2023-12-01T03:11:32.211907+0000","last_fullsized":"2023-12-01T03:11:32.211907+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.188647+0000","last_change":"2023-12-01T03:11:31.671174+0000","last_active":"2023-12-01T03:11:32.188647+0000","last_peered":"2023-12-01T03:11:32.188647+0000","last_clean":"2023-12-01T03:11:32.188647+0000","last_became_active":"2023-12-01T03:11:31.670862+0000","last_became_peered":"2023-12-01T03:11:31.670862+0000","last_unstale":"2023-12-01T03:11:32.188647+0000","last_undegraded":"2023-12-01T03:11:32.188647+0000","last_fullsized":"2023-12-01T03:11:32.188647+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:37.158883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.209558+0000","last_change":"2023-12-01T03:11:32.203594+0000","last_active":"2023-12-01T03:11:32.209558+0000","last_peered":"2023-12-01T03:11:32.209558+0000","last_clean":"2023-12-01T03:11:32.209558+0000","last_became_active":"2023-12-01T03:11:32.203346+0000","last_became_peered":"2023-12-01T03:11:32.203346+0000","last_unstale":"2023-12-01T03:11:32.209558+0000","last_undegraded":"2023-12-01T03:11:32.209558+0000","last_fullsized":"2023-12-01T03:11:32.209558+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:22.924077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.10","version":"234'8","reported_seq":19,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218897+0000","last_change":"2023-12-01T03:11:31.673143+0000","last_active":"2023-12-01T03:11:32.218897+0000","last_peered":"2023-12-01T03:11:32.218897+0000","last_clean":"2023-12-01T03:11:32.218897+0000","last_became_active":"2023-12-01T03:11:31.671261+0000","last_became_peered":"2023-12-01T03:11:31.671261+0000","last_unstale":"2023-12-01T03:11:32.218897+0000","last_undegraded":"2023-12-01T03:11:32.218897+0000","last_fullsized":"2023-12-01T03:11:32.218897+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:13.744340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213478+0000","last_change":"2023-12-01T03:11:32.209735+0000","last_active":"2023-12-01T03:11:32.213478+0000","last_peered":"2023-12-01T03:11:32.213478+0000","last_clean":"2023-12-01T03:11:32.213478+0000","last_became_active":"2023-12-01T03:11:32.209148+0000","last_became_peered":"2023-12-01T03:11:32.209148+0000","last_unstale":"2023-12-01T03:11:32.213478+0000","last_undegraded":"2023-12-01T03:11:32.213478+0000","last_fullsized":"2023-12-01T03:11:32.213478+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:46.532354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225412+0000","last_change":"2023-12-01T03:11:31.677841+0000","last_active":"2023-12-01T03:11:32.225412+0000","last_peered":"2023-12-01T03:11:32.225412+0000","last_clean":"2023-12-01T03:11:32.225412+0000","last_became_active":"2023-12-01T03:11:31.677483+0000","last_became_peered":"2023-12-01T03:11:31.677483+0000","last_unstale":"2023-12-01T03:11:32.225412+0000","last_undegraded":"2023-12-01T03:11:32.225412+0000","last_fullsized":"2023-12-01T03:11:32.225412+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:31.837132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.217241+0000","last_change":"2023-12-01T03:11:32.213539+0000","last_active":"2023-12-01T03:11:32.217241+0000","last_peered":"2023-12-01T03:11:32.217241+0000","last_clean":"2023-12-01T03:11:32.217241+0000","last_became_active":"2023-12-01T03:11:32.212492+0000","last_became_peered":"2023-12-01T03:11:32.212492+0000","last_unstale":"2023-12-01T03:11:32.217241+0000","last_undegraded":"2023-12-01T03:11:32.217241+0000","last_fullsized":"2023-12-01T03:11:32.217241+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:29:48.204277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.e","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222148+0000","last_change":"2023-12-01T03:11:31.670701+0000","last_active":"2023-12-01T03:11:32.222148+0000","last_peered":"2023-12-01T03:11:32.222148+0000","last_clean":"2023-12-01T03:11:32.222148+0000","last_became_active":"2023-12-01T03:11:31.670179+0000","last_became_peered":"2023-12-01T03:11:31.670179+0000","last_unstale":"2023-12-01T03:11:32.222148+0000","last_undegraded":"2023-12-01T03:11:32.222148+0000","last_fullsized":"2023-12-01T03:11:32.222148+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:01.094744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216871+0000","last_change":"2023-12-01T03:11:32.212327+0000","last_active":"2023-12-01T03:11:32.216871+0000","last_peered":"2023-12-01T03:11:32.216871+0000","last_clean":"2023-12-01T03:11:32.216871+0000","last_became_active":"2023-12-01T03:11:32.212068+0000","last_became_peered":"2023-12-01T03:11:32.212068+0000","last_unstale":"2023-12-01T03:11:32.216871+0000","last_undegraded":"2023-12-01T03:11:32.216871+0000","last_fullsized":"2023-12-01T03:11:32.216871+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:31.875817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"234'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224912+0000","last_change":"2023-12-01T03:11:31.679287+0000","last_active":"2023-12-01T03:11:32.224912+0000","last_peered":"2023-12-01T03:11:32.224912+0000","last_clean":"2023-12-01T03:11:32.224912+0000","last_became_active":"2023-12-01T03:11:31.678925+0000","last_became_peered":"2023-12-01T03:11:31.678925+0000","last_unstale":"2023-12-01T03:11:32.224912+0000","last_undegraded":"2023-12-01T03:11:32.224912+0000","last_fullsized":"2023-12-01T03:11:32.224912+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:28:46.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.221614+0000","last_change":"2023-12-01T03:11:32.214436+0000","last_active":"2023-12-01T03:11:32.221614+0000","last_peered":"2023-12-01T03:11:32.221614+0000","last_clean":"2023-12-01T03:11:32.221614+0000","last_became_active":"2023-12-01T03:11:32.214162+0000","last_became_peered":"2023-12-01T03:11:32.214162+0000","last_unstale":"2023-12-01T03:11:32.221614+0000","last_undegraded":"2023-12-01T03:11:32.221614+0000","last_fullsized":"2023-12-01T03:11:32.221614+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:18:18.055289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.192683+0000","last_change":"2023-12-01T03:11:31.674632+0000","last_active":"2023-12-01T03:11:32.192683+0000","last_peered":"2023-12-01T03:11:32.192683+0000","last_clean":"2023-12-01T03:11:32.192683+0000","last_became_active":"2023-12-01T03:11:31.674319+0000","last_became_peered":"2023-12-01T03:11:31.674319+0000","last_unstale":"2023-12-01T03:11:32.192683+0000","last_undegraded":"2023-12-01T03:11:32.192683+0000","last_fullsized":"2023-12-01T03:11:32.192683+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:21.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216607+0000","last_change":"2023-12-01T03:11:32.209024+0000","last_active":"2023-12-01T03:11:32.216607+0000","last_peered":"2023-12-01T03:11:32.216607+0000","last_clean":"2023-12-01T03:11:32.216607+0000","last_became_active":"2023-12-01T03:11:32.208729+0000","last_became_peered":"2023-12-01T03:11:32.208729+0000","last_unstale":"2023-12-01T03:11:32.216607+0000","last_undegraded":"2023-12-01T03:11:32.216607+0000","last_fullsized":"2023-12-01T03:11:32.216607+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:27:53.473015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.b","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.223478+0000","last_change":"2023-12-01T03:11:31.676885+0000","last_active":"2023-12-01T03:11:32.223478+0000","last_peered":"2023-12-01T03:11:32.223478+0000","last_clean":"2023-12-01T03:11:32.223478+0000","last_became_active":"2023-12-01T03:11:31.675545+0000","last_became_peered":"2023-12-01T03:11:31.675545+0000","last_unstale":"2023-12-01T03:11:32.223478+0000","last_undegraded":"2023-12-01T03:11:32.223478+0000","last_fullsized":"2023-12-01T03:11:32.223478+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:03.962061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212390+0000","last_change":"2023-12-01T03:11:32.204502+0000","last_active":"2023-12-01T03:11:32.212390+0000","last_peered":"2023-12-01T03:11:32.212390+0000","last_clean":"2023-12-01T03:11:32.212390+0000","last_became_active":"2023-12-01T03:11:32.204048+0000","last_became_peered":"2023-12-01T03:11:32.204048+0000","last_unstale":"2023-12-01T03:11:32.212390+0000","last_undegraded":"2023-12-01T03:11:32.212390+0000","last_fullsized":"2023-12-01T03:11:32.212390+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:15:30.930202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.183774+0000","last_change":"2023-12-01T03:11:31.675947+0000","last_active":"2023-12-01T03:11:32.183774+0000","last_peered":"2023-12-01T03:11:32.183774+0000","last_clean":"2023-12-01T03:11:32.183774+0000","last_became_active":"2023-12-01T03:11:31.675372+0000","last_became_peered":"2023-12-01T03:11:31.675372+0000","last_unstale":"2023-12-01T03:11:32.183774+0000","last_undegraded":"2023-12-01T03:11:32.183774+0000","last_fullsized":"2023-12-01T03:11:32.183774+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:45.516411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.216820+0000","last_change":"2023-12-01T03:11:32.214460+0000","last_active":"2023-12-01T03:11:32.216820+0000","last_peered":"2023-12-01T03:11:32.216820+0000","last_clean":"2023-12-01T03:11:32.216820+0000","last_became_active":"2023-12-01T03:11:32.213987+0000","last_became_peered":"2023-12-01T03:11:32.213987+0000","last_unstale":"2023-12-01T03:11:32.216820+0000","last_undegraded":"2023-12-01T03:11:32.216820+0000","last_fullsized":"2023-12-01T03:11:32.216820+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:20.713703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.194822+0000","last_change":"2023-12-01T03:11:31.673899+0000","last_active":"2023-12-01T03:11:32.194822+0000","last_peered":"2023-12-01T03:11:32.194822+0000","last_clean":"2023-12-01T03:11:32.194822+0000","last_became_active":"2023-12-01T03:11:31.673474+0000","last_became_peered":"2023-12-01T03:11:31.673474+0000","last_unstale":"2023-12-01T03:11:32.194822+0000","last_undegraded":"2023-12-01T03:11:32.194822+0000","last_fullsized":"2023-12-01T03:11:32.194822+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:50.332038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.218479+0000","last_change":"2023-12-01T03:11:32.214835+0000","last_active":"2023-12-01T03:11:32.218479+0000","last_peered":"2023-12-01T03:11:32.218479+0000","last_clean":"2023-12-01T03:11:32.218479+0000","last_became_active":"2023-12-01T03:11:32.214553+0000","last_became_peered":"2023-12-01T03:11:32.214553+0000","last_unstale":"2023-12-01T03:11:32.218479+0000","last_undegraded":"2023-12-01T03:11:32.218479+0000","last_fullsized":"2023-12-01T03:11:32.218479+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.190392+0000","last_change":"2023-12-01T03:11:31.682949+0000","last_active":"2023-12-01T03:11:32.190392+0000","last_peered":"2023-12-01T03:11:32.190392+0000","last_clean":"2023-12-01T03:11:32.190392+0000","last_became_active":"2023-12-01T03:11:31.682454+0000","last_became_peered":"2023-12-01T03:11:31.682454+0000","last_unstale":"2023-12-01T03:11:32.190392+0000","last_undegraded":"2023-12-01T03:11:32.190392+0000","last_fullsized":"2023-12-01T03:11:32.190392+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:49.777291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.208510+0000","last_change":"2023-12-01T03:11:32.204430+0000","last_active":"2023-12-01T03:11:32.208510+0000","last_peered":"2023-12-01T03:11:32.208510+0000","last_clean":"2023-12-01T03:11:32.208510+0000","last_became_active":"2023-12-01T03:11:32.204043+0000","last_became_peered":"2023-12-01T03:11:32.204043+0000","last_unstale":"2023-12-01T03:11:32.208510+0000","last_undegraded":"2023-12-01T03:11:32.208510+0000","last_fullsized":"2023-12-01T03:11:32.208510+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:33:45.498838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.222532+0000","last_change":"2023-12-01T03:11:31.668375+0000","last_active":"2023-12-01T03:11:32.222532+0000","last_peered":"2023-12-01T03:11:32.222532+0000","last_clean":"2023-12-01T03:11:32.222532+0000","last_became_active":"2023-12-01T03:11:31.668109+0000","last_became_peered":"2023-12-01T03:11:31.668109+0000","last_unstale":"2023-12-01T03:11:32.222532+0000","last_undegraded":"2023-12-01T03:11:32.222532+0000","last_fullsized":"2023-12-01T03:11:32.222532+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:34.959657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225708+0000","last_change":"2023-12-01T03:11:32.223409+0000","last_active":"2023-12-01T03:11:32.225708+0000","last_peered":"2023-12-01T03:11:32.225708+0000","last_clean":"2023-12-01T03:11:32.225708+0000","last_became_active":"2023-12-01T03:11:32.221541+0000","last_became_peered":"2023-12-01T03:11:32.221541+0000","last_unstale":"2023-12-01T03:11:32.225708+0000","last_undegraded":"2023-12-01T03:11:32.225708+0000","last_fullsized":"2023-12-01T03:11:32.225708+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:53.273884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.6","version":"235'1","reported_seq":12,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-01T03:11:33.191347+0000","last_change":"2023-12-01T03:11:31.679248+0000","last_active":"2023-12-01T03:11:33.191347+0000","last_peered":"2023-12-01T03:11:33.191347+0000","last_clean":"2023-12-01T03:11:33.191347+0000","last_became_active":"2023-12-01T03:11:31.678973+0000","last_became_peered":"2023-12-01T03:11:31.678973+0000","last_unstale":"2023-12-01T03:11:33.191347+0000","last_undegraded":"2023-12-01T03:11:33.191347+0000","last_fullsized":"2023-12-01T03:11:33.191347+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.212063+0000","last_change":"2023-12-01T03:11:32.207147+0000","last_active":"2023-12-01T03:11:32.212063+0000","last_peered":"2023-12-01T03:11:32.212063+0000","last_clean":"2023-12-01T03:11:32.212063+0000","last_became_active":"2023-12-01T03:11:32.206872+0000","last_became_peered":"2023-12-01T03:11:32.206872+0000","last_unstale":"2023-12-01T03:11:32.212063+0000","last_undegraded":"2023-12-01T03:11:32.212063+0000","last_fullsized":"2023-12-01T03:11:32.212063+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.202418+0000","last_change":"2023-12-01T03:11:31.663771+0000","last_active":"2023-12-01T03:11:32.202418+0000","last_peered":"2023-12-01T03:11:32.202418+0000","last_clean":"2023-12-01T03:11:32.202418+0000","last_became_active":"2023-12-01T03:11:31.662877+0000","last_became_peered":"2023-12-01T03:11:31.662877+0000","last_unstale":"2023-12-01T03:11:32.202418+0000","last_undegraded":"2023-12-01T03:11:32.202418+0000","last_fullsized":"2023-12-01T03:11:32.202418+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:59.396265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.213526+0000","last_change":"2023-12-01T03:11:32.208042+0000","last_active":"2023-12-01T03:11:32.213526+0000","last_peered":"2023-12-01T03:11:32.213526+0000","last_clean":"2023-12-01T03:11:32.213526+0000","last_became_active":"2023-12-01T03:11:32.207767+0000","last_became_peered":"2023-12-01T03:11:32.207767+0000","last_unstale":"2023-12-01T03:11:32.213526+0000","last_undegraded":"2023-12-01T03:11:32.213526+0000","last_fullsized":"2023-12-01T03:11:32.213526+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:17.846927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.195227+0000","last_change":"2023-12-01T03:11:31.673043+0000","last_active":"2023-12-01T03:11:32.195227+0000","last_peered":"2023-12-01T03:11:32.195227+0000","last_clean":"2023-12-01T03:11:32.195227+0000","last_became_active":"2023-12-01T03:11:31.672780+0000","last_became_peered":"2023-12-01T03:11:31.672780+0000","last_unstale":"2023-12-01T03:11:32.195227+0000","last_undegraded":"2023-12-01T03:11:32.195227+0000","last_fullsized":"2023-12-01T03:11:32.195227+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:40.478556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225682+0000","last_change":"2023-12-01T03:11:32.224518+0000","last_active":"2023-12-01T03:11:32.225682+0000","last_peered":"2023-12-01T03:11:32.225682+0000","last_clean":"2023-12-01T03:11:32.225682+0000","last_became_active":"2023-12-01T03:11:32.220831+0000","last_became_peered":"2023-12-01T03:11:32.220831+0000","last_unstale":"2023-12-01T03:11:32.225682+0000","last_undegraded":"2023-12-01T03:11:32.225682+0000","last_fullsized":"2023-12-01T03:11:32.225682+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.189860+0000","last_change":"2023-12-01T03:11:31.677162+0000","last_active":"2023-12-01T03:11:32.189860+0000","last_peered":"2023-12-01T03:11:32.189860+0000","last_clean":"2023-12-01T03:11:32.189860+0000","last_became_active":"2023-12-01T03:11:31.674178+0000","last_became_peered":"2023-12-01T03:11:31.674178+0000","last_unstale":"2023-12-01T03:11:32.189860+0000","last_undegraded":"2023-12-01T03:11:32.189860+0000","last_fullsized":"2023-12-01T03:11:32.189860+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:13.700505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.225199+0000","last_change":"2023-12-01T03:11:32.222439+0000","last_active":"2023-12-01T03:11:32.225199+0000","last_peered":"2023-12-01T03:11:32.225199+0000","last_clean":"2023-12-01T03:11:32.225199+0000","last_became_active":"2023-12-01T03:11:32.221662+0000","last_became_peered":"2023-12-01T03:11:32.221662+0000","last_unstale":"2023-12-01T03:11:32.225199+0000","last_undegraded":"2023-12-01T03:11:32.225199+0000","last_fullsized":"2023-12-01T03:11:32.225199+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.188580+0000","last_change":"2023-12-01T03:11:31.672908+0000","last_active":"2023-12-01T03:11:32.188580+0000","last_peered":"2023-12-01T03:11:32.188580+0000","last_clean":"2023-12-01T03:11:32.188580+0000","last_became_active":"2023-12-01T03:11:31.671738+0000","last_became_peered":"2023-12-01T03:11:31.671738+0000","last_unstale":"2023-12-01T03:11:32.188580+0000","last_undegraded":"2023-12-01T03:11:32.188580+0000","last_fullsized":"2023-12-01T03:11:32.188580+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.220159+0000","last_change":"2023-12-01T03:11:32.214055+0000","last_active":"2023-12-01T03:11:32.220159+0000","last_peered":"2023-12-01T03:11:32.220159+0000","last_clean":"2023-12-01T03:11:32.220159+0000","last_became_active":"2023-12-01T03:11:32.213767+0000","last_became_peered":"2023-12-01T03:11:32.213767+0000","last_unstale":"2023-12-01T03:11:32.220159+0000","last_undegraded":"2023-12-01T03:11:32.220159+0000","last_fullsized":"2023-12-01T03:11:32.220159+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:47:03.446609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.201156+0000","last_change":"2023-12-01T03:11:31.676480+0000","last_active":"2023-12-01T03:11:32.201156+0000","last_peered":"2023-12-01T03:11:32.201156+0000","last_clean":"2023-12-01T03:11:32.201156+0000","last_became_active":"2023-12-01T03:11:31.676127+0000","last_became_peered":"2023-12-01T03:11:31.676127+0000","last_unstale":"2023-12-01T03:11:32.201156+0000","last_undegraded":"2023-12-01T03:11:32.201156+0000","last_fullsized":"2023-12-01T03:11:32.201156+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.208183+0000","last_change":"2023-12-01T03:11:32.203024+0000","last_active":"2023-12-01T03:11:32.208183+0000","last_peered":"2023-12-01T03:11:32.208183+0000","last_clean":"2023-12-01T03:11:32.208183+0000","last_became_active":"2023-12-01T03:11:32.202779+0000","last_became_peered":"2023-12-01T03:11:32.202779+0000","last_unstale":"2023-12-01T03:11:32.208183+0000","last_undegraded":"2023-12-01T03:11:32.208183+0000","last_fullsized":"2023-12-01T03:11:32.208183+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.205725+0000","last_change":"2023-12-01T03:11:31.680251+0000","last_active":"2023-12-01T03:11:32.205725+0000","last_peered":"2023-12-01T03:11:32.205725+0000","last_clean":"2023-12-01T03:11:32.205725+0000","last_became_active":"2023-12-01T03:11:31.679936+0000","last_became_peered":"2023-12-01T03:11:31.679936+0000","last_unstale":"2023-12-01T03:11:32.205725+0000","last_undegraded":"2023-12-01T03:11:32.205725+0000","last_fullsized":"2023-12-01T03:11:32.205725+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:39.296836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.211446+0000","last_change":"2023-12-01T03:11:32.210270+0000","last_active":"2023-12-01T03:11:32.211446+0000","last_peered":"2023-12-01T03:11:32.211446+0000","last_clean":"2023-12-01T03:11:32.211446+0000","last_became_active":"2023-12-01T03:11:32.209831+0000","last_became_peered":"2023-12-01T03:11:32.209831+0000","last_unstale":"2023-12-01T03:11:32.211446+0000","last_undegraded":"2023-12-01T03:11:32.211446+0000","last_fullsized":"2023-12-01T03:11:32.211446+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"234'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.224896+0000","last_change":"2023-12-01T03:11:31.681391+0000","last_active":"2023-12-01T03:11:32.224896+0000","last_peered":"2023-12-01T03:11:32.224896+0000","last_clean":"2023-12-01T03:11:32.224896+0000","last_became_active":"2023-12-01T03:11:31.680773+0000","last_became_peered":"2023-12-01T03:11:31.680773+0000","last_unstale":"2023-12-01T03:11:32.224896+0000","last_undegraded":"2023-12-01T03:11:32.224896+0000","last_fullsized":"2023-12-01T03:11:32.224896+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:40:13.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"220'348","reported_seq":1214,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-01T03:11:32.205820+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:11:32.205820+0000","last_peered":"2023-12-01T03:11:32.205820+0000","last_clean":"2023-12-01T03:11:32.205820+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:11:32.205820+0000","last_undegraded":"2023-12-01T03:11:32.205820+0000","last_fullsized":"2023-12-01T03:11:32.205820+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575989,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60180,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311660,"statfs":{"total":96636764160,"available":96575139840,"internally_reserved":0,"allocated":2961408,"data_stored":1448140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.05,"15min":1.071},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5409999999999999,"5min":1.6000000000000001,"15min":1.7390000000000001},"last":1.613},{"interface":"front","average":{"1min":1.111,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6599999999999999,"5min":1.6699999999999999,"15min":1.7},"last":0.90500000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.236,"15min":1.2769999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4990000000000001,"5min":1.78,"15min":2.1419999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.294,"15min":1.3260000000000001},"min":{"1min":0.873,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5740000000000001,"5min":1.6240000000000001,"15min":1.8700000000000001},"last":1.5649999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1439999999999999,"15min":1.173},"min":{"1min":0.71699999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.573,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":1.647},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.2549999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.643,"5min":1.7050000000000001,"15min":2.1120000000000001},"last":1.147}]},{"osd":3,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2509999999999999,"15min":1.302},"min":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.712,"15min":2.431},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3720000000000001,"15min":1.4119999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7529999999999999,"5min":1.879,"15min":2.198},"last":1.476}]},{"osd":4,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3009999999999999,"15min":1.335},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7689999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.4510000000000001,"15min":1.49},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8799999999999999,"5min":1.97,"15min":2.569},"last":1.694}]},{"osd":5,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2869999999999999,"15min":1.3160000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7150000000000001,"5min":2.0099999999999998,"15min":2.7269999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.5469999999999999,"15min":1.585},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.0470000000000002,"5min":2.0569999999999999,"15min":2.484},"last":1.7589999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.403,"15min":1.4359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7749999999999999,"5min":1.921,"15min":2.532},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4299999999999999,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8260000000000001,"5min":1.8819999999999999,"15min":2.762},"last":1.726}]}]},{"osd":6,"up_from":13,"seq":55834575991,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42136,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329704,"statfs":{"total":96636764160,"available":96593616896,"internally_reserved":0,"allocated":2965504,"data_stored":1447614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0509999999999999,"15min":1.1080000000000001},"min":{"1min":0.66100000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.1040000000000001,"15min":1.157},"min":{"1min":0.78400000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6910000000000001,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.141,"15min":1.2150000000000001},"min":{"1min":0.68999999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8460000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1830000000000001,"15min":1.22},"min":{"1min":0.86099999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.6220000000000001,"5min":1.694,"15min":1.835},"last":1.0629999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.157,"15min":1.206},"min":{"1min":0.71799999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.464,"5min":1.742,"15min":1.976},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.222,"15min":1.278},"min":{"1min":0.71699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5009999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.3919999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2829999999999999,"15min":1.3140000000000001},"min":{"1min":0.89500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6839999999999999,"5min":1.927,"15min":2.0289999999999999},"last":1.169},{"interface":"front","average":{"1min":1.454,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":1.0069999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0190000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.532}]},{"osd":4,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":0.93600000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.661},{"interface":"front","average":{"1min":1.589,"5min":1.407,"15min":1.367},"min":{"1min":1.2090000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.222,"5min":2.48,"15min":2.48},"last":1.4470000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.3700000000000001,"15min":1.413},"min":{"1min":1.127,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.962,"5min":2.262,"15min":2.262},"last":1.581},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5169999999999999,"15min":1.4970000000000001},"min":{"1min":1.2629999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.194,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.6240000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5029999999999999,"15min":1.5309999999999999},"min":{"1min":1.155,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1320000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.746,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":1.371,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2770000000000001,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.726}]}]},{"osd":5,"up_from":13,"seq":55834575989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52576,"kb_used_data":3480,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319264,"statfs":{"total":96636764160,"available":96582926336,"internally_reserved":0,"allocated":3563520,"data_stored":2039470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1000000000000001,"15min":1.123},"min":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.7170000000000001,"5min":1.776,"15min":1.776},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1539999999999999,"15min":1.161},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.752},"last":0.97599999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.1559999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6120000000000001,"5min":1.6519999999999999,"15min":1.696},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.198,"15min":1.2330000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":1.506}]},{"osd":2,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2270000000000001,"15min":1.2470000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.81399999999999995},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.2450000000000001,"15min":1.254},"min":{"1min":0.75,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.867,"5min":1.899,"15min":1.9219999999999999},"last":1.5609999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3089999999999999,"15min":1.3149999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.889,"5min":1.889,"15min":1.8979999999999999},"last":1.175},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.329,"15min":1.359},"min":{"1min":1.0109999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.97,"5min":1.97,"15min":2.1859999999999999},"last":1.444}]},{"osd":4,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.869,"5min":1.869,"15min":2.6040000000000001},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.2470000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3779999999999999,"15min":1.377},"min":{"1min":0.99099999999999999,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.7649999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.409,"15min":1.4219999999999999},"min":{"1min":1.0149999999999999,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":2.032,"5min":2.032,"15min":2.577},"last":1.6830000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.534,"15min":1.538},"min":{"1min":1.2010000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":2.077,"5min":2.077,"15min":3.1709999999999998},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5760000000000001,"15min":1.5780000000000001},"min":{"1min":1.3200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.101,"15min":2.8740000000000001},"last":1.704}]}]},{"osd":4,"up_from":13,"seq":55834575989,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51992,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319848,"statfs":{"total":96636764160,"available":96583524352,"internally_reserved":0,"allocated":2965504,"data_stored":1449032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1200000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8089999999999999,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.198,"15min":1.1899999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5309999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.2669999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2290000000000001,"15min":1.256},"min":{"1min":0.83599999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8580000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1719999999999999,"15min":1.175},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.778,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.093}]},{"osd":2,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3300000000000001,"15min":1.244},"min":{"1min":0.74399999999999999,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.637,"5min":11.311,"15min":11.311},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3320000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9330000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.89700000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.298,"15min":1.29},"min":{"1min":0.80000000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.716,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3759999999999999,"15min":1.3340000000000001},"min":{"1min":0.96399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7430000000000001,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.357,"15min":1.3200000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.738,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.375,"15min":1.3160000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6579999999999999,"5min":3.71,"15min":3.71},"last":1.411}]},{"osd":6,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3799999999999999,"15min":1.3740000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8009999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4790000000000001,"15min":1.476},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.76,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.3939999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.514,"15min":1.488},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9830000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.597,"15min":1.575},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.907,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.5860000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575988,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56076,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315764,"statfs":{"total":96636764160,"available":96579342336,"internally_reserved":0,"allocated":2953216,"data_stored":1447468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.91300000000000003,"15min":0.90700000000000003},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.4260000000000002,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.44},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1399999999999999,"15min":1.167},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.98599999999999999,"15min":0.99299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.788,"5min":3.863,"15min":3.863},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.073,"15min":1.0960000000000001},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.637,"5min":3.597,"15min":3.597},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.127,"5min":1.2090000000000001,"15min":1.2410000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.8319999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.745}]},{"osd":4,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2969999999999999,"15min":1.294},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.272,"15min":1.296},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.736,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.331,"15min":1.329},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.1240000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.373,"15min":1.373},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.425,"15min":1.444},"min":{"1min":0.80500000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.5089999999999999,"15min":1.5129999999999999},"min":{"1min":1.0089999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":1.087}]},{"osd":7,"last update":"Fri Dec 1 03:10:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5129999999999999,"15min":1.5249999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":2.899,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.5209999999999999,"15min":1.526},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":1.1859999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575988,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47888,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323952,"statfs":{"total":96636764160,"available":96587726848,"internally_reserved":0,"allocated":2957312,"data_stored":1447502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.87,"15min":0.88900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.411,"5min":1.411,"15min":1.4730000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.95099999999999996,"5min":1.052,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3080000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.216}]},{"osd":1,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.0229999999999999,"15min":1.0489999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.343,"5min":1.698,"15min":1.698},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.097,"15min":1.135},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.4730000000000001,"5min":1.635,"15min":1.6599999999999999},"last":1.5229999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.026,"15min":1.042},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.393,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.159,"15min":1.208},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4950000000000001,"5min":1.6879999999999999,"15min":1.79},"last":1.4970000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.319,"15min":1.327},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.5700000000000001,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.621},{"interface":"front","average":{"1min":1.194,"5min":1.2689999999999999,"15min":1.294},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.5189999999999999,"5min":1.714,"15min":1.835},"last":1.411}]},{"osd":5,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3560000000000001,"15min":1.3839999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.782,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.266},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.407,"15min":1.411},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.873,"5min":1.8740000000000001,"15min":2.016},"last":1.4530000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3660000000000001,"15min":1.3799999999999999},"min":{"1min":0.82199999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.714,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4730000000000001,"15min":1.4970000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7989999999999999,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.353}]},{"osd":7,"last update":"Fri Dec 1 03:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.478,"15min":1.4850000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.9350000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.4259999999999999,"15min":1.454},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.8149999999999999,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.5509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575988,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48476,"kb_used_data":3476,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323364,"statfs":{"total":96636764160,"available":96587124736,"internally_reserved":0,"allocated":3559424,"data_stored":2038510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.89800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.238,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.98499999999999999,"15min":1.038},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.512,"5min":1.5660000000000001,"15min":1.681},"last":0.81299999999999994}]},{"osd":2,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2190000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.048,"5min":1.1020000000000001,"15min":1.141},"min":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5489999999999999,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":0.92900000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.208,"15min":1.2949999999999999},"min":{"1min":0.60799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5860000000000001,"5min":1.587,"15min":1.788},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.02,"5min":1.0780000000000001,"15min":1.127},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.383,"5min":1.714,"15min":1.8049999999999999},"last":0.89900000000000002}]},{"osd":4,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2030000000000001,"15min":1.218},"min":{"1min":0.84399999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.623,"5min":1.623,"15min":1.6859999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3700000000000001,"15min":1.4039999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.657,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.4950000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3109999999999999,"15min":1.333},"min":{"1min":0.95899999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":2.101},"last":1.581},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.298,"15min":1.331},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.9019999999999999,"5min":1.927,"15min":1.927},"last":1.429}]},{"osd":6,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0529999999999999,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3939999999999999,"15min":1.413},"min":{"1min":0.91600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.3520000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4650000000000001,"15min":1.522},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.9430000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":1.734},{"interface":"front","average":{"1min":1.524,"5min":1.48,"15min":1.5109999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.0699999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":1.6970000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60172,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311668,"statfs":{"total":96636764160,"available":96575148032,"internally_reserved":0,"allocated":2953216,"data_stored":1447468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.88900000000000001},"min":{"1min":0.314,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.994},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.98899999999999999,"15min":1.048},"min":{"1min":0.58999999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6799999999999999},"last":0.52700000000000002}]},{"osd":2,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0169999999999999,"15min":1.0640000000000001},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.635,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0589999999999999,"15min":1.1259999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3100000000000001,"5min":1.726,"15min":1.792},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.036,"15min":1.0669999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.8839999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.171,"15min":1.224},"min":{"1min":0.70499999999999996,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.6469999999999998},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.296,"15min":1.3520000000000001},"min":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.681,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.3029999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1240000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.90700000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.899,"15min":2.7719999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.383,"15min":1.427},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.681,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":1.1479999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3520000000000001,"15min":1.381},"min":{"1min":0.84599999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7270000000000001,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.45,"15min":1.4830000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.0660000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4670000000000001,"15min":1.516},"min":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7969999999999999,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4730000000000001,"15min":1.4870000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.927,"5min":2.1190000000000002,"15min":2.431},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:11:42.532 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T03:11:42.532 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T03:11:42.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.797+0000 7f35e27e0640 1 Processor -- start 2023-12-01T03:11:42.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.798+0000 7f35e27e0640 1 -- start start 2023-12-01T03:11:42.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.799+0000 7f35e27e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35dc157250 0x7f35dc15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.799+0000 7f35e27e0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f35dc0b29d0 0x7f35dc0afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.799+0000 7f35e27e0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35dc15e2b0 con 0x7f35dc0af820 2023-12-01T03:11:42.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.799+0000 7f35e27e0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35dc0b01e0 con 0x7f35dc0b29d0 2023-12-01T03:11:42.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.799+0000 7f35e27e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc06b280 con 0x7f35dc157250 2023-12-01T03:11:42.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.800+0000 7f35e0d7d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f35dc0b29d0 0x7f35dc0afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.800+0000 7f35dbfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35dc157250 0x7f35dc15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.800+0000 7f35e0d7d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f35dc0b29d0 0x7f35dc0afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:53274/0 (socket says 172.21.15.115:53274) 2023-12-01T03:11:42.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.800+0000 7f35e0d7d640 1 -- 172.21.15.115:0/3960079225 learned_addr learned my addr 172.21.15.115:0/3960079225 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:42.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.801+0000 7f35e0d7d640 1 -- 172.21.15.115:0/3960079225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35dc157250 msgr2=0x7f35dc15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:42.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.801+0000 7f35e0d7d640 1 --2- 172.21.15.115:0/3960079225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35dc157250 0x7f35dc15dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.801+0000 7f35e0d7d640 1 -- 172.21.15.115:0/3960079225 >> v1:172.21.15.142:6789/0 conn(0x7f35dc0af820 legacy=0x7f35dc1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:42.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.801+0000 7f35e0d7d640 1 -- 172.21.15.115:0/3960079225 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc06b520 con 0x7f35dc0b29d0 2023-12-01T03:11:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.802+0000 7f35e0d7d640 1 --2- 172.21.15.115:0/3960079225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35dc0b29d0 0x7f35dc0afca0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0b0320 tx=0x7f35d402bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dec0cf450de32b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.803+0000 7f35db7fe640 1 -- 172.21.15.115:0/3960079225 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d402c8a0 con 0x7f35dc0b29d0 2023-12-01T03:11:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.803+0000 7f35db7fe640 1 -- 172.21.15.115:0/3960079225 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d402ca00 con 0x7f35dc0b29d0 2023-12-01T03:11:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.803+0000 7f35db7fe640 1 -- 172.21.15.115:0/3960079225 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d402cd90 con 0x7f35dc0b29d0 2023-12-01T03:11:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.803+0000 7f35e27e0640 1 -- 172.21.15.115:0/3960079225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35dc0b29d0 msgr2=0x7f35dc0afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.803+0000 7f35e27e0640 1 --2- 172.21.15.115:0/3960079225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35dc0b29d0 0x7f35dc0afca0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0b0320 tx=0x7f35d402bd80 comp rx=0 tx=0).stop 2023-12-01T03:11:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.804+0000 7f35e27e0640 1 -- 172.21.15.115:0/3960079225 shutdown_connections 2023-12-01T03:11:42.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.804+0000 7f35e27e0640 1 --2- 172.21.15.115:0/3960079225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35dc0b29d0 0x7f35dc0afca0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.804+0000 7f35e27e0640 1 --2- 172.21.15.115:0/3960079225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35dc157250 0x7f35dc15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.804+0000 7f35e27e0640 1 -- 172.21.15.115:0/3960079225 >> 172.21.15.115:0/3960079225 conn(0x7f35dc065960 msgr2=0x7f35dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:42.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.804+0000 7f35e27e0640 1 -- 172.21.15.115:0/3960079225 shutdown_connections 2023-12-01T03:11:42.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.804+0000 7f35e27e0640 1 -- 172.21.15.115:0/3960079225 wait complete. 2023-12-01T03:11:42.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.806+0000 7f35e27e0640 1 Processor -- start 2023-12-01T03:11:42.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.806+0000 7f35e27e0640 1 -- start start 2023-12-01T03:11:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35e27e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35dc0af820 0x7f35dc153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35e27e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35dc0b29d0 0x7f35dc160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35e27e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35dc157250 0x7f35dc0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35e27e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc1f66e0 con 0x7f35dc157250 2023-12-01T03:11:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35e27e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc1f5aa0 con 0x7f35dc0af820 2023-12-01T03:11:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35e27e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc1f5d40 con 0x7f35dc0b29d0 2023-12-01T03:11:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35e0d7d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35dc157250 0x7f35dc0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35e0d7d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35dc157250 0x7f35dc0706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37586/0 (socket says 172.21.15.115:37586) 2023-12-01T03:11:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35e0d7d640 1 -- 172.21.15.115:0/2810845047 learned_addr learned my addr 172.21.15.115:0/2810845047 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35d3fff640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35dc0b29d0 0x7f35dc160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.807+0000 7f35dbfff640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35dc0af820 0x7f35dc153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.808+0000 7f35e0d7d640 1 -- 172.21.15.115:0/2810845047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35dc0b29d0 msgr2=0x7f35dc160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:42.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.808+0000 7f35e0d7d640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35dc0b29d0 0x7f35dc160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.808+0000 7f35e0d7d640 1 -- 172.21.15.115:0/2810845047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35dc0af820 msgr2=0x7f35dc153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:42.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.808+0000 7f35e0d7d640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35dc0af820 0x7f35dc153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:42.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.808+0000 7f35e0d7d640 1 -- 172.21.15.115:0/2810845047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4009b30 con 0x7f35dc157250 2023-12-01T03:11:42.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.808+0000 7f35e0d7d640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35dc157250 0x7f35dc0706a0 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f35d402bdc0 tx=0x7f35d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:42.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.808+0000 7f35d97fa640 1 -- 172.21.15.115:0/2810845047 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4009510 con 0x7f35dc157250 2023-12-01T03:11:42.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.808+0000 7f35d97fa640 1 -- 172.21.15.115:0/2810845047 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d40079b0 con 0x7f35dc157250 2023-12-01T03:11:42.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.808+0000 7f35d97fa640 1 -- 172.21.15.115:0/2810845047 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4007d60 con 0x7f35dc157250 2023-12-01T03:11:42.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.808+0000 7f35e27e0640 1 -- 172.21.15.115:0/2810845047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc1f6020 con 0x7f35dc157250 2023-12-01T03:11:42.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.809+0000 7f35e27e0640 1 -- 172.21.15.115:0/2810845047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc15f650 con 0x7f35dc157250 2023-12-01T03:11:42.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.810+0000 7f35e27e0640 1 -- 172.21.15.115:0/2810845047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc1f6820 con 0x7f35dc157250 2023-12-01T03:11:42.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.811+0000 7f35d97fa640 1 -- 172.21.15.115:0/2810845047 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f35d403e450 con 0x7f35dc157250 2023-12-01T03:11:42.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.811+0000 7f35d97fa640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f35bc06a670 0x7f35bc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:42.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.812+0000 7f35d97fa640 1 -- 172.21.15.115:0/2810845047 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f35d40ab6b0 con 0x7f35dc157250 2023-12-01T03:11:42.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.812+0000 7f35dbfff640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f35bc06a670 0x7f35bc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:42.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.813+0000 7f35dbfff640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f35bc06a670 0x7f35bc06cb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009a30 tx=0x7f35cc008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:42.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:42.816+0000 7f35d97fa640 1 -- 172.21.15.115:0/2810845047 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f35dc1f6820 con 0x7f35dc157250 2023-12-01T03:11:43.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.079+0000 7f35e27e0640 1 -- 172.21.15.115:0/2810845047 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f35dc1f6820 con 0x7f35dc157250 2023-12-01T03:11:43.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.088+0000 7f35d97fa640 1 -- 172.21.15.115:0/2810845047 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v49) v1 ==== 190+0+0 (secure 0 0 0) 0x7f35dc1f6820 con 0x7f35dc157250 2023-12-01T03:11:43.089 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T03:11:43.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.090+0000 7f35e27e0640 1 -- 172.21.15.115:0/2810845047 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f35bc06a670 msgr2=0x7f35bc06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:43.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.090+0000 7f35e27e0640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f35bc06a670 0x7f35bc06cb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009a30 tx=0x7f35cc008040 comp rx=0 tx=0).stop 2023-12-01T03:11:43.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.090+0000 7f35e27e0640 1 -- 172.21.15.115:0/2810845047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35dc157250 msgr2=0x7f35dc0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:43.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.090+0000 7f35e27e0640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35dc157250 0x7f35dc0706a0 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f35d402bdc0 tx=0x7f35d4004a40 comp rx=0 tx=0).stop 2023-12-01T03:11:43.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.091+0000 7f35e27e0640 1 -- 172.21.15.115:0/2810845047 shutdown_connections 2023-12-01T03:11:43.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.091+0000 7f35e27e0640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f35bc06a670 0x7f35bc06cb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.091+0000 7f35e27e0640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35dc0b29d0 0x7f35dc160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.091+0000 7f35e27e0640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35dc157250 0x7f35dc0706a0 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.091+0000 7f35e27e0640 1 --2- 172.21.15.115:0/2810845047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35dc0af820 0x7f35dc153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.091+0000 7f35e27e0640 1 -- 172.21.15.115:0/2810845047 >> 172.21.15.115:0/2810845047 conn(0x7f35dc065960 msgr2=0x7f35dc0b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:43.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.091+0000 7f35e27e0640 1 -- 172.21.15.115:0/2810845047 shutdown_connections 2023-12-01T03:11:43.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.091+0000 7f35e27e0640 1 -- 172.21.15.115:0/2810845047 wait complete. 2023-12-01T03:11:43.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:43.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.308+0000 7fbdca170640 1 Processor -- start 2023-12-01T03:11:43.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.309+0000 7fbdca170640 1 -- start start 2023-12-01T03:11:43.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.310+0000 7fbdca170640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b4e0 0x7fbdc4161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:43.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.310+0000 7fbdca170640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbdc415b0d0 0x7fbdc415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:43.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.310+0000 7fbdca170640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc4162530 con 0x7fbdc406b8c0 2023-12-01T03:11:43.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.310+0000 7fbdca170640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc40caf40 con 0x7fbdc415b0d0 2023-12-01T03:11:43.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.310+0000 7fbdca170640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40cd7f0 con 0x7fbdc406b4e0 2023-12-01T03:11:43.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.311+0000 7fbdc916e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b4e0 0x7fbdc4161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:43.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.311+0000 7fbdc996f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbdc415b0d0 0x7fbdc415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:43.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.311+0000 7fbdc916e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b4e0 0x7fbdc4161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:51498/0 (socket says 172.21.15.115:51498) 2023-12-01T03:11:43.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.311+0000 7fbdc916e640 1 -- 172.21.15.115:0/3456204316 learned_addr learned my addr 172.21.15.115:0/3456204316 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:43.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.312+0000 7fbdc916e640 1 -- 172.21.15.115:0/3456204316 >> v2:172.21.15.142:3300/0 conn(0x7fbdc415b0d0 msgr2=0x7fbdc415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:43.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.312+0000 7fbdc916e640 1 --2- 172.21.15.115:0/3456204316 >> v2:172.21.15.142:3300/0 conn(0x7fbdc415b0d0 0x7fbdc415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.312+0000 7fbdc916e640 1 -- 172.21.15.115:0/3456204316 >> v1:172.21.15.142:6789/0 conn(0x7fbdc406b8c0 legacy=0x7fbdc40cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:43.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.312+0000 7fbdc916e640 1 -- 172.21.15.115:0/3456204316 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc415ba10 con 0x7fbdc406b4e0 2023-12-01T03:11:43.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.313+0000 7fbdc916e640 1 --2- 172.21.15.115:0/3456204316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b4e0 0x7fbdc4161ff0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbdc415bc00 tx=0x7fbdc002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a98cdb5516f6a0ad server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:43.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.314+0000 7fbdbbfff640 1 -- 172.21.15.115:0/3456204316 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0030840 con 0x7fbdc406b4e0 2023-12-01T03:11:43.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.314+0000 7fbdbbfff640 1 -- 172.21.15.115:0/3456204316 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc00309a0 con 0x7fbdc406b4e0 2023-12-01T03:11:43.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.314+0000 7fbdbbfff640 1 -- 172.21.15.115:0/3456204316 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0030d30 con 0x7fbdc406b4e0 2023-12-01T03:11:43.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.314+0000 7fbdca170640 1 -- 172.21.15.115:0/3456204316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b4e0 msgr2=0x7fbdc4161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:43.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.314+0000 7fbdca170640 1 --2- 172.21.15.115:0/3456204316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b4e0 0x7fbdc4161ff0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbdc415bc00 tx=0x7fbdc002b980 comp rx=0 tx=0).stop 2023-12-01T03:11:43.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.314+0000 7fbdca170640 1 -- 172.21.15.115:0/3456204316 shutdown_connections 2023-12-01T03:11:43.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.314+0000 7fbdca170640 1 --2- 172.21.15.115:0/3456204316 >> v2:172.21.15.142:3300/0 conn(0x7fbdc415b0d0 0x7fbdc415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.314+0000 7fbdca170640 1 --2- 172.21.15.115:0/3456204316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b4e0 0x7fbdc4161ff0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.314+0000 7fbdca170640 1 -- 172.21.15.115:0/3456204316 >> 172.21.15.115:0/3456204316 conn(0x7fbdc40ce640 msgr2=0x7fbdc40cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:43.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.314+0000 7fbdca170640 1 -- 172.21.15.115:0/3456204316 shutdown_connections 2023-12-01T03:11:43.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.315+0000 7fbdca170640 1 -- 172.21.15.115:0/3456204316 wait complete. 2023-12-01T03:11:43.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.317+0000 7fbdca170640 1 Processor -- start 2023-12-01T03:11:43.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.317+0000 7fbdca170640 1 -- start start 2023-12-01T03:11:43.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.318+0000 7fbdca170640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdc406b4e0 0x7fbdc40c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.318+0000 7fbdca170640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b8c0 0x7fbdc4157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.318+0000 7fbdca170640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdc415b0d0 0x7fbdc40714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.318+0000 7fbdca170640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc41583b0 con 0x7fbdc406b4e0 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.318+0000 7fbdca170640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc406efb0 con 0x7fbdc415b0d0 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.318+0000 7fbdca170640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc4157e50 con 0x7fbdc406b8c0 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.318+0000 7fbdc916e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdc406b4e0 0x7fbdc40c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.318+0000 7fbdc916e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdc406b4e0 0x7fbdc40c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37596/0 (socket says 172.21.15.115:37596) 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.318+0000 7fbdc916e640 1 -- 172.21.15.115:0/1446028040 learned_addr learned my addr 172.21.15.115:0/1446028040 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdc996f640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdc415b0d0 0x7fbdc40714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdc896d640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b8c0 0x7fbdc4157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdc916e640 1 -- 172.21.15.115:0/1446028040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b8c0 msgr2=0x7fbdc4157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdc916e640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b8c0 0x7fbdc4157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdc916e640 1 -- 172.21.15.115:0/1446028040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdc415b0d0 msgr2=0x7fbdc40714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:43.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdc916e640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdc415b0d0 0x7fbdc40714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdc916e640 1 -- 172.21.15.115:0/1446028040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc00094d0 con 0x7fbdc406b4e0 2023-12-01T03:11:43.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdc996f640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdc415b0d0 0x7fbdc40714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:43.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdc896d640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b8c0 0x7fbdc4157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:43.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdc916e640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdc406b4e0 0x7fbdc40c7720 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:43.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdb9ffb640 1 -- 172.21.15.115:0/1446028040 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0039c90 con 0x7fbdc406b4e0 2023-12-01T03:11:43.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdb9ffb640 1 -- 172.21.15.115:0/1446028040 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc0039e10 con 0x7fbdc406b4e0 2023-12-01T03:11:43.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdb9ffb640 1 -- 172.21.15.115:0/1446028040 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00446c0 con 0x7fbdc406b4e0 2023-12-01T03:11:43.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.319+0000 7fbdca170640 1 -- 172.21.15.115:0/1446028040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc40c7e00 con 0x7fbdc406b4e0 2023-12-01T03:11:43.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.320+0000 7fbdca170640 1 -- 172.21.15.115:0/1446028040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc406f500 con 0x7fbdc406b4e0 2023-12-01T03:11:43.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.321+0000 7fbdca170640 1 -- 172.21.15.115:0/1446028040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc40722f0 con 0x7fbdc406b4e0 2023-12-01T03:11:43.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.322+0000 7fbdb9ffb640 1 -- 172.21.15.115:0/1446028040 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fbdc004d3b0 con 0x7fbdc406b4e0 2023-12-01T03:11:43.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.323+0000 7fbdb9ffb640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbda806a670 0x7fbda806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:43.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.323+0000 7fbdb9ffb640 1 -- 172.21.15.115:0/1446028040 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fbdc003e070 con 0x7fbdc406b4e0 2023-12-01T03:11:43.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.323+0000 7fbdc896d640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbda806a670 0x7fbda806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:43.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.324+0000 7fbdc896d640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbda806a670 0x7fbda806cb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40ad6d0 tx=0x7fbdb4009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:43.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.327+0000 7fbdb9ffb640 1 -- 172.21.15.115:0/1446028040 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbdc0076c20 con 0x7fbdc406b4e0 2023-12-01T03:11:43.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.554+0000 7fbdca170640 1 -- 172.21.15.115:0/1446028040 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdc40673a0 con 0x7fbdc406b4e0 2023-12-01T03:11:43.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.555+0000 7fbdb9ffb640 1 -- 172.21.15.115:0/1446028040 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4658 (secure 0 0 0) 0x7fbdc007a240 con 0x7fbdc406b4e0 2023-12-01T03:11:43.556 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:43.556 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":110,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:32.176438+0000","modified":"2023-12-01T03:11:35.672576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7735":{"gid":7735,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1837882056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1837882056},{"type":"v1","addr":"172.21.15.142:6835","nonce":1837882056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-12-01T03:11:43.557 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 110 2023-12-01T03:11:43.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 -- 172.21.15.115:0/1446028040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbda806a670 msgr2=0x7fbda806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:43.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbda806a670 0x7fbda806cb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40ad6d0 tx=0x7fbdb4009040 comp rx=0 tx=0).stop 2023-12-01T03:11:43.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 -- 172.21.15.115:0/1446028040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdc406b4e0 msgr2=0x7fbdc40c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:43.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdc406b4e0 0x7fbdc40c7720 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc0004730 comp rx=0 tx=0).stop 2023-12-01T03:11:43.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 -- 172.21.15.115:0/1446028040 shutdown_connections 2023-12-01T03:11:43.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbda806a670 0x7fbda806cb30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdc406b8c0 0x7fbdc4157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdc415b0d0 0x7fbdc40714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 --2- 172.21.15.115:0/1446028040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdc406b4e0 0x7fbdc40c7720 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:43.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 -- 172.21.15.115:0/1446028040 >> 172.21.15.115:0/1446028040 conn(0x7fbdc40ce640 msgr2=0x7fbdc415a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:43.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 -- 172.21.15.115:0/1446028040 shutdown_connections 2023-12-01T03:11:43.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:43.558+0000 7fbdca170640 1 -- 172.21.15.115:0/1446028040 wait complete. 2023-12-01T03:11:43.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T03:11:32.176438+0000', 'modified': '2023-12-01T03:11:35.672576+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7735}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7735': {'gid': 7735, 'name': 'd', 'rank': 0, 'incarnation': 107, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6835/1837882056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 1837882056}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 1837882056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T03:11:43.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T03:11:43.980 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-01T03:11:44.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.197+0000 7f6e9abcb640 1 Processor -- start 2023-12-01T03:11:44.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.198+0000 7f6e9abcb640 1 -- start start 2023-12-01T03:11:44.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.199+0000 7f6e9abcb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e9415e2f0 0x7f6e940ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.199+0000 7f6e9abcb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e9406f2f0 0x7f6e940caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.199+0000 7f6e9abcb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6e940adf10 0x7f6e940c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.199+0000 7f6e9abcb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e940add10 con 0x7f6e9406f2f0 2023-12-01T03:11:44.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.199+0000 7f6e9abcb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e940c5760 con 0x7f6e940adf10 2023-12-01T03:11:44.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.199+0000 7f6e9abcb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9415e6d0 con 0x7f6e9415e2f0 2023-12-01T03:11:44.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.199+0000 7f6e93fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e9406f2f0 0x7f6e940caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.199+0000 7f6e93fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6e9406f2f0 0x7f6e940caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37602/0 (socket says 172.21.15.115:37602) 2023-12-01T03:11:44.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.199+0000 7f6e93fff640 1 -- 172.21.15.115:0/1364735682 learned_addr learned my addr 172.21.15.115:0/1364735682 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:44.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.200+0000 7f6e93fff640 1 -- 172.21.15.115:0/1364735682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e9415e2f0 msgr2=0x7f6e940ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.200+0000 7f6e93fff640 1 --2- 172.21.15.115:0/1364735682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e9415e2f0 0x7f6e940ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.200+0000 7f6e93fff640 1 -- 172.21.15.115:0/1364735682 >> v2:172.21.15.142:3300/0 conn(0x7f6e940adf10 msgr2=0x7f6e940c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.200+0000 7f6e93fff640 1 --2- 172.21.15.115:0/1364735682 >> v2:172.21.15.142:3300/0 conn(0x7f6e940adf10 0x7f6e940c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.200+0000 7f6e93fff640 1 -- 172.21.15.115:0/1364735682 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e940cf7b0 con 0x7f6e9406f2f0 2023-12-01T03:11:44.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.201+0000 7f6e93fff640 1 --2- 172.21.15.115:0/1364735682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9406f2f0 0x7f6e940caba0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f6e940cb2b0 tx=0x7f6e8002b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5447981effc734c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.201+0000 7f6e937fe640 1 -- 172.21.15.115:0/1364735682 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80031760 con 0x7f6e9406f2f0 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.201+0000 7f6e937fe640 1 -- 172.21.15.115:0/1364735682 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e800318c0 con 0x7f6e9406f2f0 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.201+0000 7f6e937fe640 1 -- 172.21.15.115:0/1364735682 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80031c50 con 0x7f6e9406f2f0 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.202+0000 7f6e9abcb640 1 -- 172.21.15.115:0/1364735682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9406f2f0 msgr2=0x7f6e940caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.202+0000 7f6e9abcb640 1 --2- 172.21.15.115:0/1364735682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9406f2f0 0x7f6e940caba0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f6e940cb2b0 tx=0x7f6e8002b800 comp rx=0 tx=0).stop 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.202+0000 7f6e9abcb640 1 -- 172.21.15.115:0/1364735682 shutdown_connections 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.202+0000 7f6e9abcb640 1 --2- 172.21.15.115:0/1364735682 >> v2:172.21.15.142:3300/0 conn(0x7f6e940adf10 0x7f6e940c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.202+0000 7f6e9abcb640 1 --2- 172.21.15.115:0/1364735682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9406f2f0 0x7f6e940caba0 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.202+0000 7f6e9abcb640 1 --2- 172.21.15.115:0/1364735682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e9415e2f0 0x7f6e940ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.202+0000 7f6e9abcb640 1 -- 172.21.15.115:0/1364735682 >> 172.21.15.115:0/1364735682 conn(0x7f6e940ce880 msgr2=0x7f6e940cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:44.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.202+0000 7f6e9abcb640 1 -- 172.21.15.115:0/1364735682 shutdown_connections 2023-12-01T03:11:44.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.202+0000 7f6e9abcb640 1 -- 172.21.15.115:0/1364735682 wait complete. 2023-12-01T03:11:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.204+0000 7f6e9abcb640 1 Processor -- start 2023-12-01T03:11:44.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.204+0000 7f6e9abcb640 1 -- start start 2023-12-01T03:11:44.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.205+0000 7f6e9abcb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e9406f2f0 0x7f6e941f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.205+0000 7f6e9abcb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e940adf10 0x7f6e941580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.205+0000 7f6e9abcb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9415e2f0 0x7f6e940c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.205+0000 7f6e9abcb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e941f1e10 con 0x7f6e9415e2f0 2023-12-01T03:11:44.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.205+0000 7f6e9abcb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e941f20f0 con 0x7f6e940adf10 2023-12-01T03:11:44.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.205+0000 7f6e9abcb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e941f23d0 con 0x7f6e9406f2f0 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.205+0000 7f6e99168640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9415e2f0 0x7f6e940c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.205+0000 7f6e99168640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9415e2f0 0x7f6e940c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37604/0 (socket says 172.21.15.115:37604) 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.205+0000 7f6e99168640 1 -- 172.21.15.115:0/2571520980 learned_addr learned my addr 172.21.15.115:0/2571520980 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e93fff640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e940adf10 0x7f6e941580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e98967640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e9406f2f0 0x7f6e941f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e99168640 1 -- 172.21.15.115:0/2571520980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e9406f2f0 msgr2=0x7f6e941f15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e99168640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e9406f2f0 0x7f6e941f15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e99168640 1 -- 172.21.15.115:0/2571520980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e940adf10 msgr2=0x7f6e941580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e99168640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e940adf10 0x7f6e941580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e99168640 1 -- 172.21.15.115:0/2571520980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800094d0 con 0x7f6e9415e2f0 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e93fff640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e940adf10 0x7f6e941580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e99168640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9415e2f0 0x7f6e940c5950 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00b9e0 tx=0x7f6e8c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e917fa640 1 -- 172.21.15.115:0/2571520980 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00cb20 con 0x7f6e9415e2f0 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e917fa640 1 -- 172.21.15.115:0/2571520980 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c00cca0 con 0x7f6e9415e2f0 2023-12-01T03:11:44.208 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e917fa640 1 -- 172.21.15.115:0/2571520980 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c014680 con 0x7f6e9415e2f0 2023-12-01T03:11:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.206+0000 7f6e9abcb640 1 -- 172.21.15.115:0/2571520980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e941f26b0 con 0x7f6e9415e2f0 2023-12-01T03:11:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.207+0000 7f6e9abcb640 1 -- 172.21.15.115:0/2571520980 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e941f1b20 con 0x7f6e9415e2f0 2023-12-01T03:11:44.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.209+0000 7f6e917fa640 1 -- 172.21.15.115:0/2571520980 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f6e8c01d3b0 con 0x7f6e9415e2f0 2023-12-01T03:11:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.210+0000 7f6e917fa640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6e7806a670 0x7f6e7806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.210+0000 7f6e917fa640 1 -- 172.21.15.115:0/2571520980 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f6e8c088d70 con 0x7f6e9415e2f0 2023-12-01T03:11:44.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.210+0000 7f6e98967640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6e7806a670 0x7f6e7806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.210+0000 7f6e9d81d740 1 Processor -- start 2023-12-01T03:11:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.211+0000 7f6e9d81d740 1 -- start start 2023-12-01T03:11:44.211 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.211+0000 7f6e98967640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6e7806a670 0x7f6e7806cb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800a770 tx=0x7f6e88008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.211+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1bee730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.211+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1beec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.211+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1bebd60 0x5583b1bc24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.211+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b1b9a2f0 con 0x5583b1c0c650 2023-12-01T03:11:44.212 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.211+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b1b2ef80 con 0x5583b1c0a170 2023-12-01T03:11:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.211+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583b1b2c710 con 0x5583b1bebd60 2023-12-01T03:11:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.212+0000 7f6e98967640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1bee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.212+0000 7f6e98967640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1bee730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37612/0 (socket says 172.21.15.115:37612) 2023-12-01T03:11:44.213 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.212+0000 7f6e98967640 1 -- 172.21.15.115:0/2299235192 learned_addr learned my addr 172.21.15.115:0/2299235192 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:44.214 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.212+0000 7f6e98967640 1 -- 172.21.15.115:0/2299235192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1bebd60 msgr2=0x5583b1bc24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.212+0000 7f6e98967640 1 --2- 172.21.15.115:0/2299235192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1bebd60 0x5583b1bc24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.212+0000 7f6e93fff640 1 --2- 172.21.15.115:0/2299235192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1beec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.212+0000 7f6e98967640 1 -- 172.21.15.115:0/2299235192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 msgr2=0x5583b1beec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.212+0000 7f6e98967640 1 --2- 172.21.15.115:0/2299235192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1beec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.212+0000 7f6e98967640 1 -- 172.21.15.115:0/2299235192 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583b1aef690 con 0x5583b1c0c650 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.212+0000 7f6e98967640 1 --2- 172.21.15.115:0/2299235192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1bee730 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c9a0 tx=0x7f6e8800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e7c7a7b22c1a9d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.213+0000 7f6e7e7fc640 1 -- 172.21.15.115:0/2299235192 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8800c520 con 0x5583b1c0c650 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.213+0000 7f6e7e7fc640 1 -- 172.21.15.115:0/2299235192 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e88013b80 con 0x5583b1c0c650 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.213+0000 7f6e7e7fc640 1 -- 172.21.15.115:0/2299235192 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8801d3f0 con 0x5583b1c0c650 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.213+0000 7f6e9d81d740 1 -- 172.21.15.115:0/2299235192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 msgr2=0x5583b1bee730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.213+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/2299235192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1bee730 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c9a0 tx=0x7f6e8800f000 comp rx=0 tx=0).stop 2023-12-01T03:11:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.213+0000 7f6e9d81d740 1 -- 172.21.15.115:0/2299235192 shutdown_connections 2023-12-01T03:11:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.214+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/2299235192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1bebd60 0x5583b1bc24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.214+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/2299235192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1beec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.214+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/2299235192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1bee730 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.214+0000 7f6e9d81d740 1 -- 172.21.15.115:0/2299235192 >> 172.21.15.115:0/2299235192 conn(0x5583b1bc0bb0 msgr2=0x5583b1c09730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.214+0000 7f6e9d81d740 1 -- 172.21.15.115:0/2299235192 shutdown_connections 2023-12-01T03:11:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.214+0000 7f6e9d81d740 1 -- 172.21.15.115:0/2299235192 wait complete. 2023-12-01T03:11:44.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.214+0000 7f6e9d81d740 1 -- start start 2023-12-01T03:11:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.214+0000 7f6e9d81d740 1 Processor -- start 2023-12-01T03:11:44.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.215+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1c0a170 0x5583b1bf23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.215+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.215+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1be5e60 0x5583b1bc24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.215+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b1b28000 con 0x5583b1c0c650 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.215+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b1ac97d0 con 0x5583b1be5e60 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.215+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583b1ac9470 con 0x5583b1c0a170 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.215+0000 7f6e9d81d740 10 client.0 _ll_register_callbacks cb 0x5583b1b181b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e93fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e93fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37618/0 (socket says 172.21.15.115:37618) 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e93fff640 1 -- 172.21.15.115:0/1689002584 learned_addr learned my addr 172.21.15.115:0/1689002584 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e98967640 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1c0a170 0x5583b1bf23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e99168640 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1be5e60 0x5583b1bc24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e93fff640 1 -- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1c0a170 msgr2=0x5583b1bf23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e93fff640 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1c0a170 0x5583b1bf23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e93fff640 1 -- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1be5e60 msgr2=0x5583b1bc24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e93fff640 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1be5e60 0x5583b1bc24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e93fff640 1 -- 172.21.15.115:0/1689002584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8800c680 con 0x5583b1c0c650 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e93fff640 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be57f0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f6e8002b840 tx=0x7f6e8002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd71ded3485756 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.216+0000 7f6e737fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e737fe640 1 -- 172.21.15.115:0/1689002584 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8003b8c0 con 0x5583b1c0c650 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e9d81d740 1 -- 172.21.15.115:0/1689002584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583b1b52550 con 0x5583b1c0c650 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e737fe640 1 -- 172.21.15.115:0/1689002584 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f6e80040070 con 0x5583b1c0c650 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e737fe640 1 -- 172.21.15.115:0/1689002584 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e8003ba20 con 0x5583b1c0c650 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e9d81d740 10 client.11631 fetch_fsmap learned FSMap version 110 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e9d81d740 1 -- 172.21.15.115:0/1689002584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x5583b1b26680 con 0x5583b1c0c650 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e737fe640 1 -- 172.21.15.115:0/1689002584 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5583b1b52550 con 0x5583b1c0c650 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e737fe640 1 -- 172.21.15.115:0/1689002584 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 110) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6e80031070 con 0x5583b1c0c650 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e9d81d740 10 client.11631 fetch_fsmap finished waiting for FSMap version 110 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e9d81d740 10 client.11631 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-01T03:11:44.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e9d81d740 20 client.11631 populate_metadata read hostname 'smithi115' 2023-12-01T03:11:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:6834/1837882056,v1:172.21.15.142:6835/1837882056] conn(0x5583b1bef770 0x5583b1befb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e9d81d740 4 client.11631 mds_command: new command op to 7735 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T03:11:44.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.217+0000 7f6e9d81d740 1 -- 172.21.15.115:0/1689002584 --> [v2:172.21.15.142:6834/1837882056,v1:172.21.15.142:6835/1837882056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583b1b1ab50 con 0x5583b1bef770 2023-12-01T03:11:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.218+0000 7f6e98967640 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:6834/1837882056,v1:172.21.15.142:6835/1837882056] conn(0x5583b1bef770 0x5583b1befb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.219+0000 7f6e98967640 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:6834/1837882056,v1:172.21.15.142:6835/1837882056] conn(0x5583b1bef770 0x5583b1befb50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b49ec38573fe189 server_cookie=97e2468e3185b808 in_seq=0 out_seq=0 2023-12-01T03:11:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.219+0000 7f6e737fe640 10 client.11631 ms_handle_connect on v2:172.21.15.142:6834/1837882056 2023-12-01T03:11:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e737fe640 1 -- 172.21.15.115:0/1689002584 <== mds.0 v2:172.21.15.142:6834/1837882056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5583b1b1ab50 con 0x5583b1bef770 2023-12-01T03:11:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e737fe640 10 client.11631 handle_command_reply: tid=0 2023-12-01T03:11:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 client.11631 shutdown 2023-12-01T03:11:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 -- 172.21.15.115:0/1689002584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 msgr2=0x5583b1be57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be57f0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f6e8002b840 tx=0x7f6e8002e000 comp rx=0 tx=0).stop 2023-12-01T03:11:44.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 -- 172.21.15.115:0/1689002584 shutdown_connections 2023-12-01T03:11:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:6834/1837882056,v1:172.21.15.142:6835/1837882056] conn(0x5583b1bef770 0x5583b1befb50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1be5e60 0x5583b1bc24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be57f0 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e737fe640 0 client.11631 ms_handle_reset on v2:172.21.15.142:6834/1837882056 2023-12-01T03:11:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/1689002584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1c0a170 0x5583b1bf23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 -- 172.21.15.115:0/1689002584 >> 172.21.15.115:0/1689002584 conn(0x5583b1bebd60 msgr2=0x5583b1be76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 -- 172.21.15.115:0/1689002584 shutdown_connections 2023-12-01T03:11:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.221+0000 7f6e9d81d740 1 -- 172.21.15.115:0/1689002584 wait complete. 2023-12-01T03:11:44.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.222+0000 7f6e9d81d740 20 client.11631 trim_cache size 0 max 16384 2023-12-01T03:11:44.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.238+0000 7f6e9d81d740 1 Processor -- start 2023-12-01T03:11:44.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.238+0000 7f6e9d81d740 1 -- start start 2023-12-01T03:11:44.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.238+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1be5e60 0x5583b1c15690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1be76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b188ff20 con 0x5583b1c0c650 2023-12-01T03:11:44.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b1bc54f0 con 0x5583b1c0a170 2023-12-01T03:11:44.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583b1c27b10 con 0x5583b1be5e60 2023-12-01T03:11:44.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e99168640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e99168640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37624/0 (socket says 172.21.15.115:37624) 2023-12-01T03:11:44.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e99168640 1 -- 172.21.15.115:0/3423434284 learned_addr learned my addr 172.21.15.115:0/3423434284 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:44.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e98967640 1 --2- 172.21.15.115:0/3423434284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1be5e60 0x5583b1c15690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e93fff640 1 --2- 172.21.15.115:0/3423434284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1be76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e99168640 1 -- 172.21.15.115:0/3423434284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1be5e60 msgr2=0x5583b1c15690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e99168640 1 --2- 172.21.15.115:0/3423434284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1be5e60 0x5583b1c15690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e99168640 1 -- 172.21.15.115:0/3423434284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 msgr2=0x5583b1be76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e99168640 1 --2- 172.21.15.115:0/3423434284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1be76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e99168640 1 -- 172.21.15.115:0/3423434284 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583b1ac89e0 con 0x5583b1c0c650 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.239+0000 7f6e99168640 1 --2- 172.21.15.115:0/3423434284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be7bf0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00b9b0 tx=0x7f6e8c057bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa3236be04945cb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e7effd640 1 -- 172.21.15.115:0/3423434284 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00ce10 con 0x5583b1c0c650 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e7effd640 1 -- 172.21.15.115:0/3423434284 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c055f90 con 0x5583b1c0c650 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e7effd640 1 -- 172.21.15.115:0/3423434284 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c0570e0 con 0x5583b1c0c650 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3423434284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 msgr2=0x5583b1be7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/3423434284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be7bf0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00b9b0 tx=0x7f6e8c057bf0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3423434284 shutdown_connections 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/3423434284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1c0c650 0x5583b1be7bf0 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/3423434284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1be76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/3423434284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1be5e60 0x5583b1c15690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3423434284 >> 172.21.15.115:0/3423434284 conn(0x5583b1bebd60 msgr2=0x5583b1c135a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:44.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3423434284 shutdown_connections 2023-12-01T03:11:44.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.240+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3423434284 wait complete. 2023-12-01T03:11:44.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.241+0000 7f6e9d81d740 1 -- start start 2023-12-01T03:11:44.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.241+0000 7f6e9d81d740 1 Processor -- start 2023-12-01T03:11:44.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.242+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1be5e60 0x5583b1c271f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.242+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1bee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.242+0000 7f6e9d81d740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1c0c650 0x5583b1be7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.242+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b1bf0420 con 0x5583b1be5e60 2023-12-01T03:11:44.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.242+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583b1befb50 con 0x5583b1c0a170 2023-12-01T03:11:44.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.242+0000 7f6e9d81d740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583b1c1c310 con 0x5583b1c0c650 2023-12-01T03:11:44.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.242+0000 7f6e9d81d740 10 client.0 _ll_register_callbacks cb 0x7f6e8800a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:11:44.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.243+0000 7f6e98967640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1be5e60 0x5583b1c271f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.243+0000 7f6e98967640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1be5e60 0x5583b1c271f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37632/0 (socket says 172.21.15.115:37632) 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.243+0000 7f6e98967640 1 -- 172.21.15.115:0/3477398078 learned_addr learned my addr 172.21.15.115:0/3477398078 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.243+0000 7f6e98967640 1 -- 172.21.15.115:0/3477398078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1c0c650 msgr2=0x5583b1be7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.243+0000 7f6e98967640 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1c0c650 0x5583b1be7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.243+0000 7f6e98967640 1 -- 172.21.15.115:0/3477398078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 msgr2=0x5583b1bee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.243+0000 7f6e98967640 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1bee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.243+0000 7f6e98967640 1 -- 172.21.15.115:0/3477398078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e941f1b20 con 0x5583b1be5e60 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.243+0000 7f6e98967640 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1be5e60 0x5583b1c271f0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f6e88008010 tx=0x7f6e880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bd0106782c176b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.243+0000 7f6e737fe640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e737fe640 1 -- 172.21.15.115:0/3477398078 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8802c040 con 0x5583b1be5e60 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e737fe640 1 -- 172.21.15.115:0/3477398078 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f6e8802b040 con 0x5583b1be5e60 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3477398078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e88013b80 con 0x5583b1be5e60 2023-12-01T03:11:44.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e737fe640 1 -- 172.21.15.115:0/3477398078 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e88031980 con 0x5583b1be5e60 2023-12-01T03:11:44.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e9d81d740 10 client.11637 fetch_fsmap learned FSMap version 110 2023-12-01T03:11:44.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3477398078 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x5583b1aedef0 con 0x5583b1be5e60 2023-12-01T03:11:44.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e737fe640 1 -- 172.21.15.115:0/3477398078 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e88013b80 con 0x5583b1be5e60 2023-12-01T03:11:44.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e737fe640 1 -- 172.21.15.115:0/3477398078 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 110) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6e88031ae0 con 0x5583b1be5e60 2023-12-01T03:11:44.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e9d81d740 10 client.11637 fetch_fsmap finished waiting for FSMap version 110 2023-12-01T03:11:44.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e9d81d740 10 client.11637 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-01T03:11:44.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e9d81d740 20 client.11637 populate_metadata read hostname 'smithi115' 2023-12-01T03:11:44.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.142:6834/1837882056,v1:172.21.15.142:6835/1837882056] conn(0x5583b1c777f0 0x5583b1c77bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e9d81d740 4 client.11637 mds_command: new command op to 7735 tid=0[{"prefix": "status"}] 2023-12-01T03:11:44.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.244+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3477398078 --> [v2:172.21.15.142:6834/1837882056,v1:172.21.15.142:6835/1837882056] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5583b1b21080 con 0x5583b1c777f0 2023-12-01T03:11:44.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.245+0000 7f6e93fff640 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.142:6834/1837882056,v1:172.21.15.142:6835/1837882056] conn(0x5583b1c777f0 0x5583b1c77bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.246+0000 7f6e93fff640 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.142:6834/1837882056,v1:172.21.15.142:6835/1837882056] conn(0x5583b1c777f0 0x5583b1c77bf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a762b287e408cc7 server_cookie=eb5991435d3d4b8f in_seq=0 out_seq=0 2023-12-01T03:11:44.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.246+0000 7f6e737fe640 10 client.11637 ms_handle_connect on v2:172.21.15.142:6834/1837882056 2023-12-01T03:11:44.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.246+0000 7f6e737fe640 1 -- 172.21.15.115:0/3477398078 <== mds.0 v2:172.21.15.142:6834/1837882056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x5583b1b21080 con 0x5583b1c777f0 2023-12-01T03:11:44.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e737fe640 10 client.11637 handle_command_reply: tid=0 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 client.11637 shutdown 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3477398078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1be5e60 msgr2=0x5583b1c271f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1be5e60 0x5583b1c271f0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f6e88008010 tx=0x7f6e880046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3477398078 shutdown_connections 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.142:6834/1837882056,v1:172.21.15.142:6835/1837882056] conn(0x5583b1c777f0 0x5583b1c77bf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5583b1c0c650 0x5583b1be7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5583b1c0a170 0x5583b1bee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 --2- 172.21.15.115:0/3477398078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5583b1be5e60 0x5583b1c271f0 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e737fe640 0 client.11637 ms_handle_reset on v2:172.21.15.142:6834/1837882056 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3477398078 >> 172.21.15.115:0/3477398078 conn(0x5583b1bebd60 msgr2=0x5583b1c143f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3477398078 shutdown_connections 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 1 -- 172.21.15.115:0/3477398078 wait complete. 2023-12-01T03:11:44.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.247+0000 7f6e9d81d740 20 client.11637 trim_cache size 0 max 16384 2023-12-01T03:11:44.251 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:11:44.251 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T03:11:44.251 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T03:11:44.251 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 7735, 2023-12-01T03:11:44.251 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T03:11:44.251 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T03:11:44.251 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T03:11:44.251 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 12.062340508, 2023-12-01T03:11:44.251 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 110, 2023-12-01T03:11:44.251 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 235, 2023-12-01T03:11:44.252 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 235, 2023-12-01T03:11:44.252 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 17.39834127 2023-12-01T03:11:44.252 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:11:44.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 -- 172.21.15.115:0/2571520980 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6e7806a670 msgr2=0x7f6e7806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6e7806a670 0x7f6e7806cb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800a770 tx=0x7f6e88008040 comp rx=0 tx=0).stop 2023-12-01T03:11:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 -- 172.21.15.115:0/2571520980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9415e2f0 msgr2=0x7f6e940c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9415e2f0 0x7f6e940c5950 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00b9e0 tx=0x7f6e8c00bea0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 -- 172.21.15.115:0/2571520980 shutdown_connections 2023-12-01T03:11:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6e7806a670 0x7f6e7806cb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e9415e2f0 0x7f6e940c5950 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e940adf10 0x7f6e941580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 --2- 172.21.15.115:0/2571520980 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e9406f2f0 0x7f6e941f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 -- 172.21.15.115:0/2571520980 >> 172.21.15.115:0/2571520980 conn(0x7f6e940ce880 msgr2=0x7f6e9406c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 -- 172.21.15.115:0/2571520980 shutdown_connections 2023-12-01T03:11:44.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.248+0000 7f6e9abcb640 1 -- 172.21.15.115:0/2571520980 wait complete. 2023-12-01T03:11:44.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T03:11:44.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.473+0000 7f7b847a3640 1 Processor -- start 2023-12-01T03:11:44.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.474+0000 7f7b847a3640 1 -- start start 2023-12-01T03:11:44.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.475+0000 7f7b847a3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7b7c0b5d80 0x7f7b7c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.475+0000 7f7b847a3640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7c0ce2e0 con 0x7f7b7c15cd20 2023-12-01T03:11:44.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.475+0000 7f7b847a3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7c0ce000 con 0x7f7b7c0d1230 2023-12-01T03:11:44.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.475+0000 7f7b847a3640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b7c0b2160 con 0x7f7b7c0b5d80 2023-12-01T03:11:44.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.475+0000 7f7b8253f640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7b7c15cd20 0x7f7b7c0b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34740/0 (socket says 172.21.15.115:34740) 2023-12-01T03:11:44.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.475+0000 7f7b8253f640 1 -- 172.21.15.115:0/1198071662 learned_addr learned my addr 172.21.15.115:0/1198071662 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.475+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913218512 0 0) 0x7f7b7c0ce2e0 con 0x7f7b7c15cd20 2023-12-01T03:11:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b68003520 con 0x7f7b7c15cd20 2023-12-01T03:11:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b81d3e640 1 --2- 172.21.15.115:0/1198071662 >> v2:172.21.15.142:3300/0 conn(0x7f7b7c0b5d80 0x7f7b7c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 749086505 0 0) 0x7f7b68003520 con 0x7f7b7c15cd20 2023-12-01T03:11:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 >> v2:172.21.15.142:3300/0 conn(0x7f7b7c0b5d80 msgr2=0x7f7b7c0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 --2- 172.21.15.115:0/1198071662 >> v2:172.21.15.142:3300/0 conn(0x7f7b7c0b5d80 0x7f7b7c0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 >> v1:172.21.15.142:6789/0 conn(0x7f7b7c0d1230 legacy=0x7f7b7c0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c0cd3d0 con 0x7f7b7c15cd20 2023-12-01T03:11:44.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f7b78001670 con 0x7f7b7c15cd20 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 >> v1:172.21.15.115:6789/0 conn(0x7f7b7c15cd20 legacy=0x7f7b7c0b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b680039e0 0x7f7b68003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b680047c0 0x7f7b68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6800c190 con 0x7f7b680052a0 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6800c300 con 0x7f7b680047c0 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6800c4c0 con 0x7f7b680039e0 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.476+0000 7f7b82d40640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b6800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.477+0000 7f7b8253f640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b680039e0 0x7f7b68003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.477+0000 7f7b81d3e640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b680047c0 0x7f7b68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.477+0000 7f7b82d40640 1 -- 172.21.15.115:0/1198071662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b680039e0 msgr2=0x7f7b68003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.477+0000 7f7b82d40640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b680039e0 0x7f7b68003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.477+0000 7f7b82d40640 1 -- 172.21.15.115:0/1198071662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b680047c0 msgr2=0x7f7b68004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:44.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.477+0000 7f7b82d40640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b680047c0 0x7f7b68004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.477+0000 7f7b82d40640 1 -- 172.21.15.115:0/1198071662 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b68003520 con 0x7f7b680052a0 2023-12-01T03:11:44.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.477+0000 7f7b81d3e640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b680047c0 0x7f7b68004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.478+0000 7f7b82d40640 1 -- 172.21.15.115:0/1198071662 reap_dead start 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.478+0000 7f7b82d40640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b6800b980 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f7b740044b0 tx=0x7f7b680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9fe55eee99a11a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.478+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74006d50 con 0x7f7b680052a0 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.478+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b74007020 con 0x7f7b680052a0 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.478+0000 7f7b8153d640 1 -- 172.21.15.115:0/1198071662 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740073d0 con 0x7f7b680052a0 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.479+0000 7f7b847a3640 1 -- 172.21.15.115:0/1198071662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 msgr2=0x7f7b6800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.479+0000 7f7b847a3640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b6800b980 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f7b740044b0 tx=0x7f7b680057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.479+0000 7f7b847a3640 1 -- 172.21.15.115:0/1198071662 shutdown_connections 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.479+0000 7f7b847a3640 1 --2- 172.21.15.115:0/1198071662 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b6800b980 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.479+0000 7f7b847a3640 1 -- 172.21.15.115:0/1198071662 >> 172.21.15.115:0/1198071662 conn(0x7f7b7c06a0e0 msgr2=0x7f7b7c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:44.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.479+0000 7f7b847a3640 1 -- 172.21.15.115:0/1198071662 shutdown_connections 2023-12-01T03:11:44.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.479+0000 7f7b847a3640 1 -- 172.21.15.115:0/1198071662 wait complete. 2023-12-01T03:11:44.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.481+0000 7f7b847a3640 1 Processor -- start 2023-12-01T03:11:44.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.481+0000 7f7b847a3640 1 -- start start 2023-12-01T03:11:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.482+0000 7f7b847a3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b7c1e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.482+0000 7f7b847a3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b7c1e6ab0 0x7f7b7c15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.482+0000 7f7b847a3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b7c1e6f10 0x7f7b7c1f1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.482+0000 7f7b847a3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c1e5950 con 0x7f7b680052a0 2023-12-01T03:11:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.482+0000 7f7b847a3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c154580 con 0x7f7b7c1e6f10 2023-12-01T03:11:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.482+0000 7f7b847a3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c154860 con 0x7f7b7c1e6ab0 2023-12-01T03:11:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.482+0000 7f7b8253f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b7c1e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.482+0000 7f7b8253f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b7c1e5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37638/0 (socket says 172.21.15.115:37638) 2023-12-01T03:11:44.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.482+0000 7f7b8253f640 1 -- 172.21.15.115:0/1812839377 learned_addr learned my addr 172.21.15.115:0/1812839377 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b8253f640 1 -- 172.21.15.115:0/1812839377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b7c1e6ab0 msgr2=0x7f7b7c15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b8253f640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b7c1e6ab0 0x7f7b7c15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b8253f640 1 -- 172.21.15.115:0/1812839377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b7c1e6f10 msgr2=0x7f7b7c1f1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b8253f640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b7c1e6f10 0x7f7b7c1f1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b8253f640 1 -- 172.21.15.115:0/1812839377 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c1e5290 con 0x7f7b680052a0 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b8253f640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b7c1e5d70 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f7b780018e0 tx=0x7f7b7800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b6effd640 1 -- 172.21.15.115:0/1812839377 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7800ee00 con 0x7f7b680052a0 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b6effd640 1 -- 172.21.15.115:0/1812839377 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7800c870 con 0x7f7b680052a0 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b6effd640 1 -- 172.21.15.115:0/1812839377 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7800cc00 con 0x7f7b680052a0 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b847a3640 1 -- 172.21.15.115:0/1812839377 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7c1e5570 con 0x7f7b680052a0 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.483+0000 7f7b847a3640 1 -- 172.21.15.115:0/1812839377 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7c15abb0 con 0x7f7b680052a0 2023-12-01T03:11:44.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.485+0000 7f7b847a3640 1 -- 172.21.15.115:0/1812839377 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7c15add0 con 0x7f7b680052a0 2023-12-01T03:11:44.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.488+0000 7f7b6effd640 1 -- 172.21.15.115:0/1812839377 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f7b7801d460 con 0x7f7b680052a0 2023-12-01T03:11:44.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.489+0000 7f7b6effd640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b6006a670 0x7f7b6006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:44.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.489+0000 7f7b6effd640 1 -- 172.21.15.115:0/1812839377 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f7b78089510 con 0x7f7b680052a0 2023-12-01T03:11:44.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.489+0000 7f7b81d3e640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b6006a670 0x7f7b6006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:44.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.490+0000 7f7b81d3e640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b6006a670 0x7f7b6006cb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c15afc0 tx=0x7f7b70009350 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:44.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.491+0000 7f7b6effd640 1 -- 172.21.15.115:0/1812839377 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7b780549b0 con 0x7f7b680052a0 2023-12-01T03:11:44.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.720+0000 7f7b847a3640 1 -- 172.21.15.115:0/1812839377 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7b7c04e890 con 0x7f7b680052a0 2023-12-01T03:11:44.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.721+0000 7f7b6effd640 1 -- 172.21.15.115:0/1812839377 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v110) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7b78057fd0 con 0x7f7b680052a0 2023-12-01T03:11:44.722 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T03:11:44.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 -- 172.21.15.115:0/1812839377 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b6006a670 msgr2=0x7f7b6006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b6006a670 0x7f7b6006cb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c15afc0 tx=0x7f7b70009350 comp rx=0 tx=0).stop 2023-12-01T03:11:44.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 -- 172.21.15.115:0/1812839377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 msgr2=0x7f7b7c1e5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:44.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b7c1e5d70 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f7b780018e0 tx=0x7f7b7800ec10 comp rx=0 tx=0).stop 2023-12-01T03:11:44.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 -- 172.21.15.115:0/1812839377 shutdown_connections 2023-12-01T03:11:44.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7b6006a670 0x7f7b6006cb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b7c1e6ab0 0x7f7b7c15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b7c1e6f10 0x7f7b7c1f1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 --2- 172.21.15.115:0/1812839377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b680052a0 0x7f7b7c1e5d70 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:44.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 -- 172.21.15.115:0/1812839377 >> 172.21.15.115:0/1812839377 conn(0x7f7b7c06a0e0 msgr2=0x7f7b7c1561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:44.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.723+0000 7f7b847a3640 1 -- 172.21.15.115:0/1812839377 shutdown_connections 2023-12-01T03:11:44.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:44.724+0000 7f7b847a3640 1 -- 172.21.15.115:0/1812839377 wait complete. 2023-12-01T03:11:44.741 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T03:11:44.741 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T03:11:44.741 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T03:11:44.741 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T03:11:44.741 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T03:11:44.741 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T03:11:44.741 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T03:11:44.741 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T03:11:44.741 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:11:44.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:44.742 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:11:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 36652sec preferred_lft 36652sec 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:11:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T03:11:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:11:44.792 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T03:11:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T03:11:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T03:11:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:11:44.792 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T03:11:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T03:11:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T03:11:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:11:44.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:44.793 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T03:11:44.876 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T03:11:44.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:44.880 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T03:11:44.934 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T03:11:44.935 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T03:11:44.935 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T03:11:44.935 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T03:11:44.935 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T03:11:44.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:44.935 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T03:11:44.935 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T03:11:44.935 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T03:11:44.936 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T03:11:44.936 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T03:11:44.936 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T03:11:45.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:45.123 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T03:11:45.175 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T03:11:45.175 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T03:11:45.176 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:11:45.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:45.176 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T03:11:45.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T03:11:45.271 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T03:11:45.328 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:11:45.328 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T03:11:45.328 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T03:11:45.379 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T03:11:45.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:45.380 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:11:45.517 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:11:45.517 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:11:45.517 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:11:45.537 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:11:45.537 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:11:45.538 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:11:45.538 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T03:11:45.538 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T03:11:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:11:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:11:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:11:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T03:11:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T03:11:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T03:11:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T03:11:45.566 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T03:11:45.568 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T03:11:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T03:11:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T03:11:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T03:11:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T03:11:45.573 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T03:11:45.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:45.573 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T03:11:45.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:45.655 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T03:11:45.723 INFO:teuthology.orchestra.run.smithi115.stdout:client6911 2023-12-01T03:11:45.724 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:11:45.724 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:11:45.781 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-12-01T03:11:46.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.002+0000 7f462ca8c640 1 Processor -- start 2023-12-01T03:11:46.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.002+0000 7f462ca8c640 1 -- start start 2023-12-01T03:11:46.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.003+0000 7f462ca8c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f462815ac00 0x7f46280ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f462ca8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46280cb800 0x7f462806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f462ca8c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462815afe0 con 0x7f46280aede0 2023-12-01T03:11:46.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f462ca8c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46280b12b0 con 0x7f462815ac00 2023-12-01T03:11:46.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f462ca8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46280b0a10 con 0x7f46280cb800 2023-12-01T03:11:46.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f46277fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f462815ac00 0x7f46280ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f46277fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f462815ac00 0x7f46280ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37654/0 (socket says 172.21.15.115:37654) 2023-12-01T03:11:46.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f46277fe640 1 -- 172.21.15.115:0/1040792845 learned_addr learned my addr 172.21.15.115:0/1040792845 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:46.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f46277fe640 1 -- 172.21.15.115:0/1040792845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46280cb800 msgr2=0x7f462806fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:46.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f4627fff640 1 --2- 172.21.15.115:0/1040792845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46280cb800 0x7f462806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f46277fe640 1 --2- 172.21.15.115:0/1040792845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46280cb800 0x7f462806fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f46277fe640 1 -- 172.21.15.115:0/1040792845 >> v1:172.21.15.142:6789/0 conn(0x7f46280aede0 legacy=0x7f46280cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:46.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.004+0000 7f46277fe640 1 -- 172.21.15.115:0/1040792845 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46280b0cf0 con 0x7f462815ac00 2023-12-01T03:11:46.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.005+0000 7f46277fe640 1 --2- 172.21.15.115:0/1040792845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462815ac00 0x7f46280ae8a0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f4618009870 tx=0x7f461802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aff8b069203e751 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.006+0000 7f4626ffd640 1 -- 172.21.15.115:0/1040792845 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180317c0 con 0x7f462815ac00 2023-12-01T03:11:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.006+0000 7f4626ffd640 1 -- 172.21.15.115:0/1040792845 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4618031920 con 0x7f462815ac00 2023-12-01T03:11:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.006+0000 7f4626ffd640 1 -- 172.21.15.115:0/1040792845 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618031cb0 con 0x7f462815ac00 2023-12-01T03:11:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.007+0000 7f462ca8c640 1 -- 172.21.15.115:0/1040792845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462815ac00 msgr2=0x7f46280ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.007+0000 7f462ca8c640 1 --2- 172.21.15.115:0/1040792845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462815ac00 0x7f46280ae8a0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f4618009870 tx=0x7f461802b860 comp rx=0 tx=0).stop 2023-12-01T03:11:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.007+0000 7f462ca8c640 1 -- 172.21.15.115:0/1040792845 shutdown_connections 2023-12-01T03:11:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.007+0000 7f462ca8c640 1 --2- 172.21.15.115:0/1040792845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46280cb800 0x7f462806fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.007+0000 7f462ca8c640 1 --2- 172.21.15.115:0/1040792845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f462815ac00 0x7f46280ae8a0 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.007+0000 7f462ca8c640 1 -- 172.21.15.115:0/1040792845 >> 172.21.15.115:0/1040792845 conn(0x7f4628065960 msgr2=0x7f4628065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.007+0000 7f462ca8c640 1 -- 172.21.15.115:0/1040792845 shutdown_connections 2023-12-01T03:11:46.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.007+0000 7f462ca8c640 1 -- 172.21.15.115:0/1040792845 wait complete. 2023-12-01T03:11:46.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.009+0000 7f462ca8c640 1 Processor -- start 2023-12-01T03:11:46.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.009+0000 7f462ca8c640 1 -- start start 2023-12-01T03:11:46.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f462ca8c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280aede0 0x7f462815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f462ca8c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46280cb800 0x7f46280c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f462ca8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462815ac00 0x7f46280c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f462ca8c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46280c2260 con 0x7f46280aede0 2023-12-01T03:11:46.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f462ca8c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46280c50c0 con 0x7f46280cb800 2023-12-01T03:11:46.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f462ca8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462815c240 con 0x7f462815ac00 2023-12-01T03:11:46.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f46277fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280aede0 0x7f462815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f46277fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280aede0 0x7f462815bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37668/0 (socket says 172.21.15.115:37668) 2023-12-01T03:11:46.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f46277fe640 1 -- 172.21.15.115:0/395270956 learned_addr learned my addr 172.21.15.115:0/395270956 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:46.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f461ffff640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46280cb800 0x7f46280c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f4627fff640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462815ac00 0x7f46280c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f46277fe640 1 -- 172.21.15.115:0/395270956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462815ac00 msgr2=0x7f46280c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f46277fe640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462815ac00 0x7f46280c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f46277fe640 1 -- 172.21.15.115:0/395270956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46280cb800 msgr2=0x7f46280c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f46277fe640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46280cb800 0x7f46280c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.011+0000 7f46277fe640 1 -- 172.21.15.115:0/395270956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180094d0 con 0x7f46280aede0 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.012+0000 7f461ffff640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46280cb800 0x7f46280c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.012+0000 7f46277fe640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280aede0 0x7f462815bbb0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f4618000c00 tx=0x7f46180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.012+0000 7f4624ff9640 1 -- 172.21.15.115:0/395270956 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180329a0 con 0x7f46280aede0 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.012+0000 7f4624ff9640 1 -- 172.21.15.115:0/395270956 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4618032b20 con 0x7f46280aede0 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.012+0000 7f4624ff9640 1 -- 172.21.15.115:0/395270956 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180453f0 con 0x7f46280aede0 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.012+0000 7f462ca8c640 1 -- 172.21.15.115:0/395270956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462815c520 con 0x7f46280aede0 2023-12-01T03:11:46.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.012+0000 7f462ca8c640 1 -- 172.21.15.115:0/395270956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46280c2790 con 0x7f46280aede0 2023-12-01T03:11:46.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.014+0000 7f462ca8c640 1 -- 172.21.15.115:0/395270956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46280c2980 con 0x7f46280aede0 2023-12-01T03:11:46.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.014+0000 7f4624ff9640 1 -- 172.21.15.115:0/395270956 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f461804d3f0 con 0x7f46280aede0 2023-12-01T03:11:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.015+0000 7f4624ff9640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 0x7f460806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.015+0000 7f4624ff9640 1 -- 172.21.15.115:0/395270956 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f46180aad00 con 0x7f46280aede0 2023-12-01T03:11:46.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.016+0000 7f461ffff640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 0x7f460806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.016+0000 7f461ffff640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 0x7f460806cb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f462815d280 tx=0x7f4610009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:46.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.020+0000 7f4624ff9640 1 -- 172.21.15.115:0/395270956 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4618075ff0 con 0x7f46280aede0 2023-12-01T03:11:46.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.252+0000 7f462ca8c640 1 -- 172.21.15.115:0/395270956 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f46280618d0 con 0x7f46280aede0 2023-12-01T03:11:46.511 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.509+0000 7f4624ff9640 1 -- 172.21.15.115:0/395270956 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v111) v1 ==== 136+0+0 (secure 0 0 0) 0x7f4618079610 con 0x7f46280aede0 2023-12-01T03:11:46.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.512+0000 7f462ca8c640 1 -- 172.21.15.115:0/395270956 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 msgr2=0x7f460806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.512+0000 7f462ca8c640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 0x7f460806cb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f462815d280 tx=0x7f4610009040 comp rx=0 tx=0).stop 2023-12-01T03:11:46.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.512+0000 7f462ca8c640 1 -- 172.21.15.115:0/395270956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280aede0 msgr2=0x7f462815bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.512+0000 7f462ca8c640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280aede0 0x7f462815bbb0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f4618000c00 tx=0x7f46180046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.513+0000 7f462ca8c640 1 -- 172.21.15.115:0/395270956 shutdown_connections 2023-12-01T03:11:46.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.513+0000 7f462ca8c640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f460806a670 0x7f460806cb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.513+0000 7f462ca8c640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f462815ac00 0x7f46280c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.513+0000 7f462ca8c640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46280cb800 0x7f46280c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.513+0000 7f462ca8c640 1 --2- 172.21.15.115:0/395270956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280aede0 0x7f462815bbb0 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.513+0000 7f462ca8c640 1 -- 172.21.15.115:0/395270956 >> 172.21.15.115:0/395270956 conn(0x7f4628065960 msgr2=0x7f4628159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:46.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.513+0000 7f462ca8c640 1 -- 172.21.15.115:0/395270956 shutdown_connections 2023-12-01T03:11:46.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.513+0000 7f462ca8c640 1 -- 172.21.15.115:0/395270956 wait complete. 2023-12-01T03:11:46.536 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:46.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.746+0000 7fd9661bf640 1 Processor -- start 2023-12-01T03:11:46.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.746+0000 7fd9661bf640 1 -- start start 2023-12-01T03:11:46.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.748+0000 7fd9661bf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd9600cad70 0x7fd96006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.748+0000 7fd9661bf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9600add10 con 0x7fd96006f2f0 2023-12-01T03:11:46.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.748+0000 7fd9661bf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9600c5760 con 0x7fd96015e2f0 2023-12-01T03:11:46.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.748+0000 7fd9661bf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96015e6d0 con 0x7fd9600cad70 2023-12-01T03:11:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.748+0000 7fd95ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fd96006f2f0 0x7fd96006bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34748/0 (socket says 172.21.15.115:34748) 2023-12-01T03:11:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.748+0000 7fd95ffff640 1 -- 172.21.15.115:0/183017207 learned_addr learned my addr 172.21.15.115:0/183017207 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd957fff640 1 --2- 172.21.15.115:0/183017207 >> v2:172.21.15.142:3300/0 conn(0x7fd9600cad70 0x7fd96006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468587325 0 0) 0x7fd9600add10 con 0x7fd96006f2f0 2023-12-01T03:11:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd94c003520 con 0x7fd96006f2f0 2023-12-01T03:11:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 429568503 0 0) 0x7fd94c003520 con 0x7fd96006f2f0 2023-12-01T03:11:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 >> v2:172.21.15.142:3300/0 conn(0x7fd9600cad70 msgr2=0x7fd96006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:46.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 --2- 172.21.15.115:0/183017207 >> v2:172.21.15.142:3300/0 conn(0x7fd9600cad70 0x7fd96006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 >> v1:172.21.15.142:6789/0 conn(0x7fd96015e2f0 legacy=0x7fd9600ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9600cf7b0 con 0x7fd96006f2f0 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fd958001a10 con 0x7fd96006f2f0 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 >> v1:172.21.15.115:6789/0 conn(0x7fd96006f2f0 legacy=0x7fd96006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd94c0039e0 0x7fd94c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd94c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd94c005380 0x7fd94c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96015e6d0 con 0x7fd94c0047c0 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c00c260 con 0x7fd94c0039e0 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.749+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c00c420 con 0x7fd94c005380 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.750+0000 7fd957fff640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd94c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.750+0000 7fd95f7fe640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd94c0039e0 0x7fd94c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.750+0000 7fd957fff640 1 -- 172.21.15.115:0/183017207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd94c005380 msgr2=0x7fd94c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.750+0000 7fd957fff640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd94c005380 0x7fd94c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.750+0000 7fd957fff640 1 -- 172.21.15.115:0/183017207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd94c0039e0 msgr2=0x7fd94c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:46.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.750+0000 7fd957fff640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd94c0039e0 0x7fd94c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.750+0000 7fd957fff640 1 -- 172.21.15.115:0/183017207 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c003520 con 0x7fd94c0047c0 2023-12-01T03:11:46.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.750+0000 7fd95ffff640 1 -- 172.21.15.115:0/183017207 reap_dead start 2023-12-01T03:11:46.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.750+0000 7fd95f7fe640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd94c0039e0 0x7fd94c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.751+0000 7fd957fff640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd94c004c40 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fd94800ba30 tx=0x7fd94c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1b0818a0f5e121 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.752+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948032640 con 0x7fd94c0047c0 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.752+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9480327a0 con 0x7fd94c0047c0 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.752+0000 7fd95effd640 1 -- 172.21.15.115:0/183017207 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948032b30 con 0x7fd94c0047c0 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.752+0000 7fd9661bf640 1 -- 172.21.15.115:0/183017207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 msgr2=0x7fd94c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.752+0000 7fd9661bf640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd94c004c40 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fd94800ba30 tx=0x7fd94c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.752+0000 7fd9661bf640 1 -- 172.21.15.115:0/183017207 shutdown_connections 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.752+0000 7fd9661bf640 1 --2- 172.21.15.115:0/183017207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd94c004c40 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.752+0000 7fd9661bf640 1 -- 172.21.15.115:0/183017207 >> 172.21.15.115:0/183017207 conn(0x7fd9600ce880 msgr2=0x7fd9600cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.752+0000 7fd9661bf640 1 -- 172.21.15.115:0/183017207 shutdown_connections 2023-12-01T03:11:46.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.752+0000 7fd9661bf640 1 -- 172.21.15.115:0/183017207 wait complete. 2023-12-01T03:11:46.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.754+0000 7fd9661bf640 1 Processor -- start 2023-12-01T03:11:46.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.754+0000 7fd9661bf640 1 -- start start 2023-12-01T03:11:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.755+0000 7fd9661bf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd9601f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.755+0000 7fd9661bf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9601f1f00 0x7fd9600c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.755+0000 7fd9661bf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9600c5e90 0x7fd9600c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.755+0000 7fd9661bf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96014cad0 con 0x7fd94c0047c0 2023-12-01T03:11:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.755+0000 7fd9661bf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601f5c50 con 0x7fd9601f1f00 2023-12-01T03:11:46.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.755+0000 7fd9661bf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601f5f30 con 0x7fd9600c5e90 2023-12-01T03:11:46.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.755+0000 7fd95f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd9601f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd95f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd9601f19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37688/0 (socket says 172.21.15.115:37688) 2023-12-01T03:11:46.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd95f7fe640 1 -- 172.21.15.115:0/228528491 learned_addr learned my addr 172.21.15.115:0/228528491 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd95ffff640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9600c5e90 0x7fd9600c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd95f7fe640 1 -- 172.21.15.115:0/228528491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9600c5e90 msgr2=0x7fd9600c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd957fff640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9601f1f00 0x7fd9600c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd95f7fe640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9600c5e90 0x7fd9600c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd95f7fe640 1 -- 172.21.15.115:0/228528491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9601f1f00 msgr2=0x7fd9600c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd95f7fe640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9601f1f00 0x7fd9600c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd95f7fe640 1 -- 172.21.15.115:0/228528491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9601f6210 con 0x7fd94c0047c0 2023-12-01T03:11:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd95f7fe640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd9601f19c0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fd95000d430 tx=0x7fd95000d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:46.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.756+0000 7fd95cff9640 1 -- 172.21.15.115:0/228528491 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950007020 con 0x7fd94c0047c0 2023-12-01T03:11:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.757+0000 7fd95cff9640 1 -- 172.21.15.115:0/228528491 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950007180 con 0x7fd94c0047c0 2023-12-01T03:11:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.757+0000 7fd95cff9640 1 -- 172.21.15.115:0/228528491 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950007530 con 0x7fd94c0047c0 2023-12-01T03:11:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.757+0000 7fd9661bf640 1 -- 172.21.15.115:0/228528491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9601f64f0 con 0x7fd94c0047c0 2023-12-01T03:11:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.757+0000 7fd9661bf640 1 -- 172.21.15.115:0/228528491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9601f5a00 con 0x7fd94c0047c0 2023-12-01T03:11:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.758+0000 7fd9661bf640 1 -- 172.21.15.115:0/228528491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96006fda0 con 0x7fd94c0047c0 2023-12-01T03:11:46.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.759+0000 7fd95cff9640 1 -- 172.21.15.115:0/228528491 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fd9500067d0 con 0x7fd94c0047c0 2023-12-01T03:11:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.760+0000 7fd95cff9640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd94006a670 0x7fd94006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.760+0000 7fd95cff9640 1 -- 172.21.15.115:0/228528491 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fd950088cf0 con 0x7fd94c0047c0 2023-12-01T03:11:46.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.760+0000 7fd957fff640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd94006a670 0x7fd94006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:46.763 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.761+0000 7fd957fff640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd94006a670 0x7fd94006cb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd94800ba00 tx=0x7fd94803d040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:46.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.764+0000 7fd95cff9640 1 -- 172.21.15.115:0/228528491 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd95008d070 con 0x7fd94c0047c0 2023-12-01T03:11:46.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.956+0000 7fd9661bf640 1 -- 172.21.15.115:0/228528491 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd96004e890 con 0x7fd94c0047c0 2023-12-01T03:11:46.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.959+0000 7fd95cff9640 1 -- 172.21.15.115:0/228528491 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v235) v1 ==== 74+0+16233 (secure 0 0 0) 0x7fd950053e40 con 0x7fd94c0047c0 2023-12-01T03:11:46.961 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:46.961 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":235,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:33.180191+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.961+0000 7fd9661bf640 1 -- 172.21.15.115:0/228528491 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd94006a670 msgr2=0x7fd94006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.961+0000 7fd9661bf640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd94006a670 0x7fd94006cb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd94800ba00 tx=0x7fd94803d040 comp rx=0 tx=0).stop 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.961+0000 7fd9661bf640 1 -- 172.21.15.115:0/228528491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 msgr2=0x7fd9601f19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.961+0000 7fd9661bf640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd9601f19c0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fd95000d430 tx=0x7fd95000d8f0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.962+0000 7fd9661bf640 1 -- 172.21.15.115:0/228528491 shutdown_connections 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.962+0000 7fd9661bf640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd94006a670 0x7fd94006cb30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.962+0000 7fd9661bf640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9600c5e90 0x7fd9600c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.962+0000 7fd9661bf640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9601f1f00 0x7fd9600c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.962+0000 7fd9661bf640 1 --2- 172.21.15.115:0/228528491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd94c0047c0 0x7fd9601f19c0 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.962+0000 7fd9661bf640 1 -- 172.21.15.115:0/228528491 >> 172.21.15.115:0/228528491 conn(0x7fd9600ce880 msgr2=0x7fd96006bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.962+0000 7fd9661bf640 1 -- 172.21.15.115:0/228528491 shutdown_connections 2023-12-01T03:11:46.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:46.962+0000 7fd9661bf640 1 -- 172.21.15.115:0/228528491 wait complete. 2023-12-01T03:11:46.983 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T03:11:30.230656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '234', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T03:11:31.144773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '235', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:11:46.984 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:11:47.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.191+0000 7f82349e0640 1 Processor -- start 2023-12-01T03:11:47.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.191+0000 7f82349e0640 1 -- start start 2023-12-01T03:11:47.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.193+0000 7f82349e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f823015ac80 0x7f82300cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.193+0000 7f82349e0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82300cc9a0 con 0x7f82300ce1b0 2023-12-01T03:11:47.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.193+0000 7f82349e0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82300cbd90 con 0x7f82300b0e60 2023-12-01T03:11:47.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.193+0000 7f82349e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823015b060 con 0x7f823015ac80 2023-12-01T03:11:47.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.193+0000 7f822ed9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f82300ce1b0 0x7f823006b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:34750/0 (socket says 172.21.15.115:34750) 2023-12-01T03:11:47.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.193+0000 7f822ed9d640 1 -- 172.21.15.115:0/3229480116 learned_addr learned my addr 172.21.15.115:0/3229480116 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822e59c640 1 --2- 172.21.15.115:0/3229480116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f823015ac80 0x7f82300cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876085350 0 0) 0x7f82300cc9a0 con 0x7f82300ce1b0 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f821c003520 con 0x7f82300ce1b0 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 35352094 0 0) 0x7f821c003520 con 0x7f82300ce1b0 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f823015ac80 msgr2=0x7f82300cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 --2- 172.21.15.115:0/3229480116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f823015ac80 0x7f82300cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 >> v1:172.21.15.142:6789/0 conn(0x7f82300b0e60 legacy=0x7f82300cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82300cc070 con 0x7f82300ce1b0 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f8228001670 con 0x7f82300ce1b0 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 >> v1:172.21.15.115:6789/0 conn(0x7f82300ce1b0 legacy=0x7f823006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 --2- 172.21.15.115:0/3229480116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 0x7f821c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 --2- 172.21.15.115:0/3229480116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f823015ac80 0x7f821c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 --2- 172.21.15.115:0/3229480116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f821c004d00 0x7f821c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823015b060 con 0x7f821c0039a0 2023-12-01T03:11:47.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c00ba60 con 0x7f821c004d00 2023-12-01T03:11:47.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.194+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c00bc20 con 0x7f823015ac80 2023-12-01T03:11:47.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.195+0000 7f822e59c640 1 --2- 172.21.15.115:0/3229480116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 0x7f821c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.195+0000 7f822e59c640 1 -- 172.21.15.115:0/3229480116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f823015ac80 msgr2=0x7f821c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:47.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.195+0000 7f822e59c640 1 --2- 172.21.15.115:0/3229480116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f823015ac80 0x7f821c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.195+0000 7f822e59c640 1 -- 172.21.15.115:0/3229480116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f821c004d00 msgr2=0x7f821c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:47.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.195+0000 7f822e59c640 1 --2- 172.21.15.115:0/3229480116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f821c004d00 0x7f821c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.195+0000 7f822e59c640 1 -- 172.21.15.115:0/3229480116 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c003520 con 0x7f821c0039a0 2023-12-01T03:11:47.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.196+0000 7f822e59c640 1 --2- 172.21.15.115:0/3229480116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 0x7f821c003dc0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f8220009f90 tx=0x7f821c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3da9b2a3a9b13ff server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:47.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.196+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822002cc20 con 0x7f821c0039a0 2023-12-01T03:11:47.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.196+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822000f040 con 0x7f821c0039a0 2023-12-01T03:11:47.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.196+0000 7f822dd9b640 1 -- 172.21.15.115:0/3229480116 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220034440 con 0x7f821c0039a0 2023-12-01T03:11:47.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.197+0000 7f82349e0640 1 -- 172.21.15.115:0/3229480116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 msgr2=0x7f821c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.197+0000 7f82349e0640 1 --2- 172.21.15.115:0/3229480116 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 0x7f821c003dc0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f8220009f90 tx=0x7f821c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.197+0000 7f822ed9d640 1 -- 172.21.15.115:0/3229480116 reap_dead start 2023-12-01T03:11:47.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.197+0000 7f82349e0640 1 -- 172.21.15.115:0/3229480116 shutdown_connections 2023-12-01T03:11:47.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.197+0000 7f82349e0640 1 -- 172.21.15.115:0/3229480116 >> 172.21.15.115:0/3229480116 conn(0x7f8230065960 msgr2=0x7f8230065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:47.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.197+0000 7f82349e0640 1 -- 172.21.15.115:0/3229480116 shutdown_connections 2023-12-01T03:11:47.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.197+0000 7f82349e0640 1 -- 172.21.15.115:0/3229480116 wait complete. 2023-12-01T03:11:47.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.199+0000 7f82349e0640 1 Processor -- start 2023-12-01T03:11:47.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.199+0000 7f82349e0640 1 -- start start 2023-12-01T03:11:47.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.200+0000 7f82349e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 0x7f823014bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.200+0000 7f82349e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8230150470 0x7f823015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.200+0000 7f82349e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82301508f0 0x7f823015c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.200+0000 7f82349e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823015b2d0 con 0x7f821c0039a0 2023-12-01T03:11:47.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.200+0000 7f82349e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823015b5b0 con 0x7f8230150470 2023-12-01T03:11:47.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.200+0000 7f82349e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823015b890 con 0x7f82301508f0 2023-12-01T03:11:47.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.200+0000 7f822e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 0x7f823014bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.200+0000 7f822e59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 0x7f823014bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:37704/0 (socket says 172.21.15.115:37704) 2023-12-01T03:11:47.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.200+0000 7f822e59c640 1 -- 172.21.15.115:0/2134939712 learned_addr learned my addr 172.21.15.115:0/2134939712 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:47.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f822ed9d640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82301508f0 0x7f823015c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.200+0000 7f8225d9b640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8230150470 0x7f823015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f822e59c640 1 -- 172.21.15.115:0/2134939712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82301508f0 msgr2=0x7f823015c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f822e59c640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82301508f0 0x7f823015c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f822e59c640 1 -- 172.21.15.115:0/2134939712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8230150470 msgr2=0x7f823015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f822e59c640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8230150470 0x7f823015c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f822e59c640 1 -- 172.21.15.115:0/2134939712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823015bb70 con 0x7f821c0039a0 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f8225d9b640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8230150470 0x7f823015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f822e59c640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 0x7f823014bc00 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f822002c5c0 tx=0x7f8220009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f82277fe640 1 -- 172.21.15.115:0/2134939712 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220040450 con 0x7f821c0039a0 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f82277fe640 1 -- 172.21.15.115:0/2134939712 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82200405d0 con 0x7f821c0039a0 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f82349e0640 1 -- 172.21.15.115:0/2134939712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823015be50 con 0x7f821c0039a0 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.201+0000 7f82277fe640 1 -- 172.21.15.115:0/2134939712 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220040980 con 0x7f821c0039a0 2023-12-01T03:11:47.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.202+0000 7f82349e0640 1 -- 172.21.15.115:0/2134939712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823015d460 con 0x7f821c0039a0 2023-12-01T03:11:47.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.203+0000 7f82349e0640 1 -- 172.21.15.115:0/2134939712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823004e890 con 0x7f821c0039a0 2023-12-01T03:11:47.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.203+0000 7f82277fe640 1 -- 172.21.15.115:0/2134939712 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f8220040ae0 con 0x7f821c0039a0 2023-12-01T03:11:47.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.204+0000 7f82277fe640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f821006a670 0x7f821006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.204+0000 7f82277fe640 1 -- 172.21.15.115:0/2134939712 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f82200ab5b0 con 0x7f821c0039a0 2023-12-01T03:11:47.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.205+0000 7f8225d9b640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f821006a670 0x7f821006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.205+0000 7f8225d9b640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f821006a670 0x7f821006cb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f823015d040 tx=0x7f8218009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:47.210 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.209+0000 7f82277fe640 1 -- 172.21.15.115:0/2134939712 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8220076800 con 0x7f821c0039a0 2023-12-01T03:11:47.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.396+0000 7f82349e0640 1 -- 172.21.15.115:0/2134939712 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f823006f5a0 con 0x7f821c0039a0 2023-12-01T03:11:47.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.397+0000 7f82277fe640 1 -- 172.21.15.115:0/2134939712 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v235) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8220079e20 con 0x7f821c0039a0 2023-12-01T03:11:47.399 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:11:47.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 -- 172.21.15.115:0/2134939712 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f821006a670 msgr2=0x7f821006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f821006a670 0x7f821006cb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f823015d040 tx=0x7f8218009040 comp rx=0 tx=0).stop 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 -- 172.21.15.115:0/2134939712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 msgr2=0x7f823014bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 0x7f823014bc00 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f822002c5c0 tx=0x7f8220009750 comp rx=0 tx=0).stop 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 -- 172.21.15.115:0/2134939712 shutdown_connections 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f821006a670 0x7f821006cb30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82301508f0 0x7f823015c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8230150470 0x7f823015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 --2- 172.21.15.115:0/2134939712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f821c0039a0 0x7f823014bc00 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 -- 172.21.15.115:0/2134939712 >> 172.21.15.115:0/2134939712 conn(0x7f8230065960 msgr2=0x7f82300c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 -- 172.21.15.115:0/2134939712 shutdown_connections 2023-12-01T03:11:47.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.400+0000 7f82349e0640 1 -- 172.21.15.115:0/2134939712 wait complete. 2023-12-01T03:11:47.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:11:47.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.630+0000 7fc7ce373640 1 Processor -- start 2023-12-01T03:11:47.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.630+0000 7fc7ce373640 1 -- start start 2023-12-01T03:11:47.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.632+0000 7fc7ce373640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc7c80b0e60 0x7fc7c80cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.632+0000 7fc7ce373640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80cc9a0 con 0x7fc7c80ce1b0 2023-12-01T03:11:47.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.632+0000 7fc7ce373640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80cbd90 con 0x7fc7c815ac80 2023-12-01T03:11:47.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.632+0000 7fc7ce373640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c815b060 con 0x7fc7c80b0e60 2023-12-01T03:11:47.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.632+0000 7fc7bffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc7c80ce1b0 0x7fc7c806b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58784/0 (socket says 172.21.15.115:58784) 2023-12-01T03:11:47.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.632+0000 7fc7bffff640 1 -- 172.21.15.115:0/2869154854 learned_addr learned my addr 172.21.15.115:0/2869154854 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:47.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7cc910640 1 --2- 172.21.15.115:0/2869154854 >> v2:172.21.15.142:3300/0 conn(0x7fc7c80b0e60 0x7fc7c80cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911208299 0 0) 0x7fc7c80cc9a0 con 0x7fc7c80ce1b0 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b4003520 con 0x7fc7c80ce1b0 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3717440843 0 0) 0x7fc7b4003520 con 0x7fc7c80ce1b0 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 >> v2:172.21.15.142:3300/0 conn(0x7fc7c80b0e60 msgr2=0x7fc7c80cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 --2- 172.21.15.115:0/2869154854 >> v2:172.21.15.142:3300/0 conn(0x7fc7c80b0e60 0x7fc7c80cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 >> v1:172.21.15.142:6789/0 conn(0x7fc7c815ac80 legacy=0x7fc7c80cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80cc070 con 0x7fc7c80ce1b0 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc7b0001670 con 0x7fc7c80ce1b0 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 >> v1:172.21.15.115:6789/0 conn(0x7fc7c80ce1b0 legacy=0x7fc7c806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b40039c0 0x7fc7b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b40047c0 0x7fc7b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c815b060 con 0x7fc7b4005180 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b400c080 con 0x7fc7b40039c0 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.633+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b400c240 con 0x7fc7b40047c0 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.634+0000 7fc7cc910640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.634+0000 7fc7bffff640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b40047c0 0x7fc7b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.634+0000 7fc7c7fff640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b40039c0 0x7fc7b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.634+0000 7fc7cc910640 1 -- 172.21.15.115:0/2869154854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b40047c0 msgr2=0x7fc7b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.634+0000 7fc7cc910640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7b40047c0 0x7fc7b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.634+0000 7fc7cc910640 1 -- 172.21.15.115:0/2869154854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b40039c0 msgr2=0x7fc7b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.634+0000 7fc7cc910640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b40039c0 0x7fc7b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.634+0000 7fc7cc910640 1 -- 172.21.15.115:0/2869154854 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b4003520 con 0x7fc7b4005180 2023-12-01T03:11:47.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.634+0000 7fc7c7fff640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7b40039c0 0x7fc7b4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.635+0000 7fc7cc910640 1 -- 172.21.15.115:0/2869154854 reap_dead start 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.636+0000 7fc7cc910640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7b400b980 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009fc0 tx=0x7fc7b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903ba56536e2ddaf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.636+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c002cc60 con 0x7fc7b4005180 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.636+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c000f040 con 0x7fc7b4005180 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.636+0000 7fc7c77fe640 1 -- 172.21.15.115:0/2869154854 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0003990 con 0x7fc7b4005180 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.636+0000 7fc7ce373640 1 -- 172.21.15.115:0/2869154854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 msgr2=0x7fc7b400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.636+0000 7fc7ce373640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7b400b980 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009fc0 tx=0x7fc7b40057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.637+0000 7fc7ce373640 1 -- 172.21.15.115:0/2869154854 shutdown_connections 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.637+0000 7fc7ce373640 1 --2- 172.21.15.115:0/2869154854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7b400b980 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.637+0000 7fc7ce373640 1 -- 172.21.15.115:0/2869154854 >> 172.21.15.115:0/2869154854 conn(0x7fc7c8065960 msgr2=0x7fc7c8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:47.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.637+0000 7fc7ce373640 1 -- 172.21.15.115:0/2869154854 shutdown_connections 2023-12-01T03:11:47.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.637+0000 7fc7ce373640 1 -- 172.21.15.115:0/2869154854 wait complete. 2023-12-01T03:11:47.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.638+0000 7fc7ce373640 1 Processor -- start 2023-12-01T03:11:47.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.638+0000 7fc7ce373640 1 -- start start 2023-12-01T03:11:47.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.639+0000 7fc7ce373640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7c814bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.639+0000 7fc7ce373640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7c8152f80 0x7fc7c8150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.639+0000 7fc7ce373640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8150790 0x7fc7c81f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.639+0000 7fc7ce373640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8154e10 con 0x7fc7b4005180 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.639+0000 7fc7ce373640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8150c10 con 0x7fc7c8150790 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.639+0000 7fc7ce373640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8150ef0 con 0x7fc7c8152f80 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.640+0000 7fc7c7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7c814bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.640+0000 7fc7c7fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7c814bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52626/0 (socket says 172.21.15.115:52626) 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.640+0000 7fc7c7fff640 1 -- 172.21.15.115:0/3328672003 learned_addr learned my addr 172.21.15.115:0/3328672003 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.640+0000 7fc7c7fff640 1 -- 172.21.15.115:0/3328672003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7c8152f80 msgr2=0x7fc7c8150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.640+0000 7fc7c7fff640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7c8152f80 0x7fc7c8150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.640+0000 7fc7c7fff640 1 -- 172.21.15.115:0/3328672003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8150790 msgr2=0x7fc7c81f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.640+0000 7fc7c7fff640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8150790 0x7fc7c81f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.640+0000 7fc7c7fff640 1 -- 172.21.15.115:0/3328672003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c81547c0 con 0x7fc7b4005180 2023-12-01T03:11:47.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.640+0000 7fc7c7fff640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7c814bf70 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8000f60 tx=0x7fc7b800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:47.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.641+0000 7fc7c57fa640 1 -- 172.21.15.115:0/3328672003 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8005480 con 0x7fc7b4005180 2023-12-01T03:11:47.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.641+0000 7fc7c57fa640 1 -- 172.21.15.115:0/3328672003 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b8003610 con 0x7fc7b4005180 2023-12-01T03:11:47.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.641+0000 7fc7c57fa640 1 -- 172.21.15.115:0/3328672003 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80039c0 con 0x7fc7b4005180 2023-12-01T03:11:47.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.641+0000 7fc7ce373640 1 -- 172.21.15.115:0/3328672003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c8154aa0 con 0x7fc7b4005180 2023-12-01T03:11:47.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.641+0000 7fc7ce373640 1 -- 172.21.15.115:0/3328672003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c815b1d0 con 0x7fc7b4005180 2023-12-01T03:11:47.644 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.642+0000 7fc7ce373640 1 -- 172.21.15.115:0/3328672003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c806f930 con 0x7fc7b4005180 2023-12-01T03:11:47.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.643+0000 7fc7c57fa640 1 -- 172.21.15.115:0/3328672003 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fc7b8003b20 con 0x7fc7b4005180 2023-12-01T03:11:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.644+0000 7fc7c57fa640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc7a806a6a0 0x7fc7a806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.644+0000 7fc7c57fa640 1 -- 172.21.15.115:0/3328672003 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fc7b8088db0 con 0x7fc7b4005180 2023-12-01T03:11:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.644+0000 7fc7bffff640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc7a806a6a0 0x7fc7a806cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.645+0000 7fc7bffff640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc7a806a6a0 0x7fc7a806cb60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc7c815a3c0 tx=0x7fc7b0000ef0 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:47.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.648+0000 7fc7c57fa640 1 -- 172.21.15.115:0/3328672003 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc7b8053fb0 con 0x7fc7b4005180 2023-12-01T03:11:47.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.840+0000 7fc7ce373640 1 -- 172.21.15.115:0/3328672003 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7c80c9de0 con 0x7fc7b4005180 2023-12-01T03:11:47.845 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.841+0000 7fc7c57fa640 1 -- 172.21.15.115:0/3328672003 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v235) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7b8001230 con 0x7fc7b4005180 2023-12-01T03:11:47.846 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:11:47.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 -- 172.21.15.115:0/3328672003 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc7a806a6a0 msgr2=0x7fc7a806cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc7a806a6a0 0x7fc7a806cb60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc7c815a3c0 tx=0x7fc7b0000ef0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 -- 172.21.15.115:0/3328672003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 msgr2=0x7fc7c814bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7c814bf70 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8000f60 tx=0x7fc7b800d9f0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 -- 172.21.15.115:0/3328672003 shutdown_connections 2023-12-01T03:11:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc7a806a6a0 0x7fc7a806cb60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7c8152f80 0x7fc7c8150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8150790 0x7fc7c81f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 --2- 172.21.15.115:0/3328672003 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b4005180 0x7fc7c814bf70 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 -- 172.21.15.115:0/3328672003 >> 172.21.15.115:0/3328672003 conn(0x7fc7c8065960 msgr2=0x7fc7c806f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 -- 172.21.15.115:0/3328672003 shutdown_connections 2023-12-01T03:11:47.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:47.844+0000 7fc7ce373640 1 -- 172.21.15.115:0/3328672003 wait complete. 2023-12-01T03:11:47.866 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:11:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.078+0000 7fb2ce539640 1 Processor -- start 2023-12-01T03:11:48.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.078+0000 7fb2ce539640 1 -- start start 2023-12-01T03:11:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.080+0000 7fb2ce539640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80cd1e0 0x7fb2c80b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.080+0000 7fb2ce539640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb2c80b1c50 0x7fb2c815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.080+0000 7fb2ce539640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c80b20e0 con 0x7fb2c8142870 2023-12-01T03:11:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.080+0000 7fb2ce539640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c80b3ae0 con 0x7fb2c80b1c50 2023-12-01T03:11:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.080+0000 7fb2ce539640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8142c50 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.081+0000 7fb2bf7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80cd1e0 0x7fb2c80b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.081+0000 7fb2ccad6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb2c80b1c50 0x7fb2c815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.081+0000 7fb2bf7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80cd1e0 0x7fb2c80b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:55760/0 (socket says 172.21.15.115:55760) 2023-12-01T03:11:48.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.081+0000 7fb2bf7fe640 1 -- 172.21.15.115:0/3915847783 learned_addr learned my addr 172.21.15.115:0/3915847783 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:48.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.081+0000 7fb2ccad6640 1 -- 172.21.15.115:0/3915847783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80cd1e0 msgr2=0x7fb2c80b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:48.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.081+0000 7fb2ccad6640 1 --2- 172.21.15.115:0/3915847783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80cd1e0 0x7fb2c80b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.081+0000 7fb2ccad6640 1 -- 172.21.15.115:0/3915847783 >> v1:172.21.15.142:6789/0 conn(0x7fb2c8142870 legacy=0x7fb2c80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.081+0000 7fb2ccad6640 1 -- 172.21.15.115:0/3915847783 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80b2f30 con 0x7fb2c80b1c50 2023-12-01T03:11:48.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.083+0000 7fb2ccad6640 1 --2- 172.21.15.115:0/3915847783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c80b1c50 0x7fb2c815a0d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000ba70 tx=0x7fb2c002e080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=571176749aab3b15 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:48.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.084+0000 7fb2c77fe640 1 -- 172.21.15.115:0/3915847783 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c002ee30 con 0x7fb2c80b1c50 2023-12-01T03:11:48.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.084+0000 7fb2c77fe640 1 -- 172.21.15.115:0/3915847783 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c0036440 con 0x7fb2c80b1c50 2023-12-01T03:11:48.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.084+0000 7fb2c77fe640 1 -- 172.21.15.115:0/3915847783 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00367d0 con 0x7fb2c80b1c50 2023-12-01T03:11:48.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.084+0000 7fb2ce539640 1 -- 172.21.15.115:0/3915847783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c80b1c50 msgr2=0x7fb2c815a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.084+0000 7fb2ce539640 1 --2- 172.21.15.115:0/3915847783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c80b1c50 0x7fb2c815a0d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000ba70 tx=0x7fb2c002e080 comp rx=0 tx=0).stop 2023-12-01T03:11:48.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.085+0000 7fb2ce539640 1 -- 172.21.15.115:0/3915847783 shutdown_connections 2023-12-01T03:11:48.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.085+0000 7fb2ce539640 1 --2- 172.21.15.115:0/3915847783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80cd1e0 0x7fb2c80b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.085+0000 7fb2ce539640 1 --2- 172.21.15.115:0/3915847783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c80b1c50 0x7fb2c815a0d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.085+0000 7fb2ce539640 1 -- 172.21.15.115:0/3915847783 >> 172.21.15.115:0/3915847783 conn(0x7fb2c8065960 msgr2=0x7fb2c8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:48.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.085+0000 7fb2ce539640 1 -- 172.21.15.115:0/3915847783 shutdown_connections 2023-12-01T03:11:48.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.085+0000 7fb2ce539640 1 -- 172.21.15.115:0/3915847783 wait complete. 2023-12-01T03:11:48.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.086+0000 7fb2ce539640 1 Processor -- start 2023-12-01T03:11:48.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.087+0000 7fb2ce539640 1 -- start start 2023-12-01T03:11:48.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2ce539640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80b1c50 0x7fb2c80c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2ce539640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2c80cd1e0 0x7fb2c813eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2ce539640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c8142870 0x7fb2c813f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2ce539640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81f34e0 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2ce539640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81f2f20 con 0x7fb2c8142870 2023-12-01T03:11:48.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2ce539640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81f3200 con 0x7fb2c80b1c50 2023-12-01T03:11:48.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2bf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2c80cd1e0 0x7fb2c813eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2bf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2c80cd1e0 0x7fb2c813eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52636/0 (socket says 172.21.15.115:52636) 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2bf7fe640 1 -- 172.21.15.115:0/3116394558 learned_addr learned my addr 172.21.15.115:0/3116394558 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2c7fff640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80b1c50 0x7fb2c80c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2ccad6640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c8142870 0x7fb2c813f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2bf7fe640 1 -- 172.21.15.115:0/3116394558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80b1c50 msgr2=0x7fb2c80c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2bf7fe640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80b1c50 0x7fb2c80c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2bf7fe640 1 -- 172.21.15.115:0/3116394558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c8142870 msgr2=0x7fb2c813f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2bf7fe640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c8142870 0x7fb2c813f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2bf7fe640 1 -- 172.21.15.115:0/3116394558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c000b6a0 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.088+0000 7fb2ccad6640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c8142870 0x7fb2c813f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.089+0000 7fb2bf7fe640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2c80cd1e0 0x7fb2c813eb00 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009fd0 tx=0x7fb2b000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.089+0000 7fb2c57fa640 1 -- 172.21.15.115:0/3116394558 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000c940 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.089+0000 7fb2c57fa640 1 -- 172.21.15.115:0/3116394558 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b000caa0 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.089+0000 7fb2ce539640 1 -- 172.21.15.115:0/3116394558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c80c81a0 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.089+0000 7fb2c57fa640 1 -- 172.21.15.115:0/3116394558 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000ce30 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.089+0000 7fb2ce539640 1 -- 172.21.15.115:0/3116394558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c815dec0 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.090+0000 7fb2ce539640 1 -- 172.21.15.115:0/3116394558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c80c8000 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.091+0000 7fb2c57fa640 1 -- 172.21.15.115:0/3116394558 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fb2b0013470 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.092+0000 7fb2c57fa640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb2a806a670 0x7fb2a806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.092+0000 7fb2c57fa640 1 -- 172.21.15.115:0/3116394558 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fb2b0088fc0 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.092+0000 7fb2c7fff640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb2a806a670 0x7fb2a806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.093+0000 7fb2c7fff640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb2a806a670 0x7fb2a806cb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80012e0 tx=0x7fb2b8009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:48.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.097+0000 7fb2c57fa640 1 -- 172.21.15.115:0/3116394558 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb2c80c8000 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.292+0000 7fb2ce539640 1 -- 172.21.15.115:0/3116394558 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2c80c8000 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.293+0000 7fb2c57fa640 1 -- 172.21.15.115:0/3116394558 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v235) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2c80c8000 con 0x7fb2c80cd1e0 2023-12-01T03:11:48.294 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:11:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.295+0000 7fb2ce539640 1 -- 172.21.15.115:0/3116394558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb2a806a670 msgr2=0x7fb2a806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.295+0000 7fb2ce539640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb2a806a670 0x7fb2a806cb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80012e0 tx=0x7fb2b8009040 comp rx=0 tx=0).stop 2023-12-01T03:11:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.295+0000 7fb2ce539640 1 -- 172.21.15.115:0/3116394558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2c80cd1e0 msgr2=0x7fb2c813eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.295+0000 7fb2ce539640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2c80cd1e0 0x7fb2c813eb00 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009fd0 tx=0x7fb2b000ed40 comp rx=0 tx=0).stop 2023-12-01T03:11:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.295+0000 7fb2ce539640 1 -- 172.21.15.115:0/3116394558 shutdown_connections 2023-12-01T03:11:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.295+0000 7fb2ce539640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb2a806a670 0x7fb2a806cb30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.295+0000 7fb2ce539640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb2c8142870 0x7fb2c813f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.295+0000 7fb2ce539640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2c80cd1e0 0x7fb2c813eb00 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.295+0000 7fb2ce539640 1 --2- 172.21.15.115:0/3116394558 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb2c80b1c50 0x7fb2c80c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.295+0000 7fb2ce539640 1 -- 172.21.15.115:0/3116394558 >> 172.21.15.115:0/3116394558 conn(0x7fb2c8065960 msgr2=0x7fb2c813fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:48.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.296+0000 7fb2ce539640 1 -- 172.21.15.115:0/3116394558 shutdown_connections 2023-12-01T03:11:48.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.296+0000 7fb2ce539640 1 -- 172.21.15.115:0/3116394558 wait complete. 2023-12-01T03:11:48.316 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T03:11:48.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.524+0000 7fe64893d640 1 Processor -- start 2023-12-01T03:11:48.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.525+0000 7fe64893d640 1 -- start start 2023-12-01T03:11:48.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.526+0000 7fe64893d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe6440b1710 0x7fe64406d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.526+0000 7fe64893d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64415e560 con 0x7fe64406c2b0 2023-12-01T03:11:48.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.526+0000 7fe64893d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64406d8a0 con 0x7fe64415ac70 2023-12-01T03:11:48.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.526+0000 7fe64893d640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64406c870 con 0x7fe6440b1710 2023-12-01T03:11:48.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.526+0000 7fe64359e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe64406c2b0 0x7fe64415ecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58790/0 (socket says 172.21.15.115:58790) 2023-12-01T03:11:48.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.526+0000 7fe642d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe6440b1710 0x7fe64406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.526+0000 7fe64359e640 1 -- 172.21.15.115:0/2016923536 learned_addr learned my addr 172.21.15.115:0/2016923536 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:48.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.527+0000 7fe642d9d640 1 -- 172.21.15.115:0/2016923536 >> v1:172.21.15.142:6789/0 conn(0x7fe64415ac70 legacy=0x7fe6440b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:48.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.527+0000 7fe642d9d640 1 -- 172.21.15.115:0/2016923536 >> v1:172.21.15.115:6789/0 conn(0x7fe64406c2b0 legacy=0x7fe64415ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.527+0000 7fe642d9d640 1 -- 172.21.15.115:0/2016923536 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64406cb50 con 0x7fe6440b1710 2023-12-01T03:11:48.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.527+0000 7fe64259c640 1 -- 172.21.15.115:0/2016923536 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362508118 0 0) 0x7fe64415e560 con 0x7fe64406c2b0 2023-12-01T03:11:48.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.528+0000 7fe642d9d640 1 --2- 172.21.15.115:0/2016923536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 0x7fe64406d360 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fe634009de0 tx=0x7fe63402be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5f8b7363fdc9f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:48.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.528+0000 7fe64259c640 1 -- 172.21.15.115:0/2016923536 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63402cc20 con 0x7fe6440b1710 2023-12-01T03:11:48.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.528+0000 7fe64259c640 1 -- 172.21.15.115:0/2016923536 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63402cd80 con 0x7fe6440b1710 2023-12-01T03:11:48.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.528+0000 7fe64259c640 1 -- 172.21.15.115:0/2016923536 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634034660 con 0x7fe6440b1710 2023-12-01T03:11:48.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.529+0000 7fe64893d640 1 -- 172.21.15.115:0/2016923536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 msgr2=0x7fe64406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.529+0000 7fe64893d640 1 --2- 172.21.15.115:0/2016923536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 0x7fe64406d360 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fe634009de0 tx=0x7fe63402be70 comp rx=0 tx=0).stop 2023-12-01T03:11:48.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.529+0000 7fe64893d640 1 -- 172.21.15.115:0/2016923536 shutdown_connections 2023-12-01T03:11:48.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.529+0000 7fe64893d640 1 --2- 172.21.15.115:0/2016923536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 0x7fe64406d360 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.529+0000 7fe64893d640 1 -- 172.21.15.115:0/2016923536 >> 172.21.15.115:0/2016923536 conn(0x7fe644072d00 msgr2=0x7fe6440730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:48.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.529+0000 7fe64893d640 1 -- 172.21.15.115:0/2016923536 shutdown_connections 2023-12-01T03:11:48.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.529+0000 7fe64893d640 1 -- 172.21.15.115:0/2016923536 wait complete. 2023-12-01T03:11:48.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.531+0000 7fe64893d640 1 Processor -- start 2023-12-01T03:11:48.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.531+0000 7fe64893d640 1 -- start start 2023-12-01T03:11:48.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.532+0000 7fe64893d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe64406c2b0 0x7fe644141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.532+0000 7fe64893d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 0x7fe644143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.532+0000 7fe64893d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe644144340 0x7fe6440b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.532+0000 7fe64893d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe644141d70 con 0x7fe6440b1710 2023-12-01T03:11:48.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.532+0000 7fe64893d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6441448a0 con 0x7fe64406c2b0 2023-12-01T03:11:48.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.532+0000 7fe64893d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe644144b80 con 0x7fe644144340 2023-12-01T03:11:48.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.532+0000 7fe63a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 0x7fe644143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.532+0000 7fe63a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 0x7fe644143e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52662/0 (socket says 172.21.15.115:52662) 2023-12-01T03:11:48.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.532+0000 7fe63a59c640 1 -- 172.21.15.115:0/1992748584 learned_addr learned my addr 172.21.15.115:0/1992748584 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.532+0000 7fe642d9d640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe64406c2b0 0x7fe644141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe63a59c640 1 -- 172.21.15.115:0/1992748584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe644144340 msgr2=0x7fe6440b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe63a59c640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe644144340 0x7fe6440b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe63a59c640 1 -- 172.21.15.115:0/1992748584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe64406c2b0 msgr2=0x7fe644141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe63a59c640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe64406c2b0 0x7fe644141620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe63a59c640 1 -- 172.21.15.115:0/1992748584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe634009a70 con 0x7fe6440b1710 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe63a59c640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 0x7fe644143e00 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b3c0 tx=0x7fe62c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe63bfff640 1 -- 172.21.15.115:0/1992748584 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c011830 con 0x7fe6440b1710 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe63bfff640 1 -- 172.21.15.115:0/1992748584 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c011990 con 0x7fe6440b1710 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe63bfff640 1 -- 172.21.15.115:0/1992748584 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c011d80 con 0x7fe6440b1710 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe64893d640 1 -- 172.21.15.115:0/1992748584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe644141f80 con 0x7fe6440b1710 2023-12-01T03:11:48.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.533+0000 7fe64893d640 1 -- 172.21.15.115:0/1992748584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe644156cf0 con 0x7fe6440b1710 2023-12-01T03:11:48.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.535+0000 7fe64893d640 1 -- 172.21.15.115:0/1992748584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6441424a0 con 0x7fe6440b1710 2023-12-01T03:11:48.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.535+0000 7fe63bfff640 1 -- 172.21.15.115:0/1992748584 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fe62c01b460 con 0x7fe6440b1710 2023-12-01T03:11:48.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.537+0000 7fe63bfff640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe62406a670 0x7fe62406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.537+0000 7fe63bfff640 1 -- 172.21.15.115:0/1992748584 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fe62c0888f0 con 0x7fe6440b1710 2023-12-01T03:11:48.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.537+0000 7fe642d9d640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe62406a670 0x7fe62406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.538+0000 7fe642d9d640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe62406a670 0x7fe62406cb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe634036000 tx=0x7fe634030000 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:48.543 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.541+0000 7fe63bfff640 1 -- 172.21.15.115:0/1992748584 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe6441424a0 con 0x7fe6440b1710 2023-12-01T03:11:48.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.733+0000 7fe64893d640 1 -- 172.21.15.115:0/1992748584 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe6441424a0 con 0x7fe6440b1710 2023-12-01T03:11:48.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.734+0000 7fe63bfff640 1 -- 172.21.15.115:0/1992748584 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v235) v1 ==== 84+0+209 (secure 0 0 0) 0x7fe6441424a0 con 0x7fe6440b1710 2023-12-01T03:11:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:48.736 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T03:11:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T03:11:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T03:11:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 28, 2023-12-01T03:11:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": "cephfs_metadata" 2023-12-01T03:11:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 29, 2023-12-01T03:11:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": "cephfs_data" 2023-12-01T03:11:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:48.737 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.736+0000 7fe64893d640 1 -- 172.21.15.115:0/1992748584 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe62406a670 msgr2=0x7fe62406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.736+0000 7fe64893d640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe62406a670 0x7fe62406cb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe634036000 tx=0x7fe634030000 comp rx=0 tx=0).stop 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.736+0000 7fe64893d640 1 -- 172.21.15.115:0/1992748584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 msgr2=0x7fe644143e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.736+0000 7fe64893d640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 0x7fe644143e00 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b3c0 tx=0x7fe62c00b880 comp rx=0 tx=0).stop 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.737+0000 7fe64893d640 1 -- 172.21.15.115:0/1992748584 shutdown_connections 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.737+0000 7fe64893d640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe62406a670 0x7fe62406cb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.737+0000 7fe64893d640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe644144340 0x7fe6440b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.737+0000 7fe64893d640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6440b1710 0x7fe644143e00 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.737+0000 7fe64893d640 1 --2- 172.21.15.115:0/1992748584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe64406c2b0 0x7fe644141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.737+0000 7fe64893d640 1 -- 172.21.15.115:0/1992748584 >> 172.21.15.115:0/1992748584 conn(0x7fe644072d00 msgr2=0x7fe644065960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.737+0000 7fe64893d640 1 -- 172.21.15.115:0/1992748584 shutdown_connections 2023-12-01T03:11:48.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.737+0000 7fe64893d640 1 -- 172.21.15.115:0/1992748584 wait complete. 2023-12-01T03:11:48.757 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-12-01T03:11:48.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-12-01T03:11:48.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.966+0000 7fcb10f77640 1 Processor -- start 2023-12-01T03:11:48.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.966+0000 7fcb10f77640 1 -- start start 2023-12-01T03:11:48.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.967+0000 7fcb10f77640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcb0c15ce50 0x7fcb0c0ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.967+0000 7fcb10f77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 0x7fcb0c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.967+0000 7fcb10f77640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0c0ca120 con 0x7fcb0c0cb6f0 2023-12-01T03:11:48.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.967+0000 7fcb10f77640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0c0cbb80 con 0x7fcb0c15ce50 2023-12-01T03:11:48.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.967+0000 7fcb10f77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c06d2f0 con 0x7fcb0c06caa0 2023-12-01T03:11:48.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.968+0000 7fcb0a59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcb0c15ce50 0x7fcb0c0ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.968+0000 7fcb09d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 0x7fcb0c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.968+0000 7fcb0ad9d640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fcb0c0cb6f0 0x7fcb0c0ac8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:48792/0 (socket says 172.21.15.115:48792) 2023-12-01T03:11:48.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.968+0000 7fcb0a59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcb0c15ce50 0x7fcb0c0ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:59026/0 (socket says 172.21.15.115:59026) 2023-12-01T03:11:48.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.968+0000 7fcb0ad9d640 1 -- 172.21.15.115:0/2225742174 learned_addr learned my addr 172.21.15.115:0/2225742174 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:48.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.969+0000 7fcb0a59c640 1 -- 172.21.15.115:0/2225742174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 msgr2=0x7fcb0c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:48.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.969+0000 7fcb0a59c640 1 --2- 172.21.15.115:0/2225742174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 0x7fcb0c0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.969+0000 7fcb0a59c640 1 -- 172.21.15.115:0/2225742174 >> v1:172.21.15.142:6789/0 conn(0x7fcb0c0cb6f0 legacy=0x7fcb0c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.969+0000 7fcb0a59c640 1 -- 172.21.15.115:0/2225742174 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c0c9380 con 0x7fcb0c15ce50 2023-12-01T03:11:48.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.970+0000 7fcb0a59c640 1 --2- 172.21.15.115:0/2225742174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c15ce50 0x7fcb0c0ccdf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009d90 tx=0x7fcafc02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3014632354a08761 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:48.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.970+0000 7fcb0959a640 1 -- 172.21.15.115:0/2225742174 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc02cc50 con 0x7fcb0c15ce50 2023-12-01T03:11:48.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.971+0000 7fcb0959a640 1 -- 172.21.15.115:0/2225742174 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc02cdb0 con 0x7fcb0c15ce50 2023-12-01T03:11:48.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.971+0000 7fcb0959a640 1 -- 172.21.15.115:0/2225742174 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc034660 con 0x7fcb0c15ce50 2023-12-01T03:11:48.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.971+0000 7fcb10f77640 1 -- 172.21.15.115:0/2225742174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c15ce50 msgr2=0x7fcb0c0ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.971+0000 7fcb10f77640 1 --2- 172.21.15.115:0/2225742174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c15ce50 0x7fcb0c0ccdf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009d90 tx=0x7fcafc02bea0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.971+0000 7fcb10f77640 1 -- 172.21.15.115:0/2225742174 shutdown_connections 2023-12-01T03:11:48.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.971+0000 7fcb10f77640 1 --2- 172.21.15.115:0/2225742174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 0x7fcb0c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.971+0000 7fcb10f77640 1 --2- 172.21.15.115:0/2225742174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c15ce50 0x7fcb0c0ccdf0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.971+0000 7fcb10f77640 1 -- 172.21.15.115:0/2225742174 >> 172.21.15.115:0/2225742174 conn(0x7fcb0c0662c0 msgr2=0x7fcb0c0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:48.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.971+0000 7fcb10f77640 1 -- 172.21.15.115:0/2225742174 shutdown_connections 2023-12-01T03:11:48.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.971+0000 7fcb10f77640 1 -- 172.21.15.115:0/2225742174 wait complete. 2023-12-01T03:11:48.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.973+0000 7fcb10f77640 1 Processor -- start 2023-12-01T03:11:48.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.973+0000 7fcb10f77640 1 -- start start 2023-12-01T03:11:48.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.974+0000 7fcb10f77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 0x7fcb0c15f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.974+0000 7fcb10f77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c0cb6f0 0x7fcb0c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.974+0000 7fcb10f77640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c15ce50 0x7fcb0c15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.974+0000 7fcb10f77640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c1606b0 con 0x7fcb0c15ce50 2023-12-01T03:11:48.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.974+0000 7fcb10f77640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c15fb30 con 0x7fcb0c0cb6f0 2023-12-01T03:11:48.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.974+0000 7fcb10f77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c15fe10 con 0x7fcb0c06caa0 2023-12-01T03:11:48.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c15ce50 0x7fcb0c15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0ad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c15ce50 0x7fcb0c15d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52668/0 (socket says 172.21.15.115:52668) 2023-12-01T03:11:48.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0ad9d640 1 -- 172.21.15.115:0/4144362544 learned_addr learned my addr 172.21.15.115:0/4144362544 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:48.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0a59c640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 0x7fcb0c15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb09d9b640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c0cb6f0 0x7fcb0c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0ad9d640 1 -- 172.21.15.115:0/4144362544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 msgr2=0x7fcb0c15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0ad9d640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 0x7fcb0c15f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0ad9d640 1 -- 172.21.15.115:0/4144362544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c0cb6f0 msgr2=0x7fcb0c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:48.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0ad9d640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c0cb6f0 0x7fcb0c15d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:48.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0ad9d640 1 -- 172.21.15.115:0/4144362544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc009a70 con 0x7fcb0c15ce50 2023-12-01T03:11:48.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0a59c640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 0x7fcb0c15f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:48.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.975+0000 7fcb0ad9d640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c15ce50 0x7fcb0c15d770 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c420 tx=0x7fcb0400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:48.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.976+0000 7fcb02ffd640 1 -- 172.21.15.115:0/4144362544 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04005ce0 con 0x7fcb0c15ce50 2023-12-01T03:11:48.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.976+0000 7fcb02ffd640 1 -- 172.21.15.115:0/4144362544 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb04005e40 con 0x7fcb0c15ce50 2023-12-01T03:11:48.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.976+0000 7fcb02ffd640 1 -- 172.21.15.115:0/4144362544 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb040072d0 con 0x7fcb0c15ce50 2023-12-01T03:11:48.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.976+0000 7fcb10f77640 1 -- 172.21.15.115:0/4144362544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c1600f0 con 0x7fcb0c15ce50 2023-12-01T03:11:48.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.976+0000 7fcb10f77640 1 -- 172.21.15.115:0/4144362544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c15f760 con 0x7fcb0c15ce50 2023-12-01T03:11:48.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.977+0000 7fcb10f77640 1 -- 172.21.15.115:0/4144362544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c074210 con 0x7fcb0c15ce50 2023-12-01T03:11:48.980 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.978+0000 7fcb02ffd640 1 -- 172.21.15.115:0/4144362544 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fcb040074f0 con 0x7fcb0c15ce50 2023-12-01T03:11:48.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.979+0000 7fcb02ffd640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcaec06a670 0x7fcaec06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:48.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.979+0000 7fcb02ffd640 1 -- 172.21.15.115:0/4144362544 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fcb04088290 con 0x7fcb0c15ce50 2023-12-01T03:11:48.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.979+0000 7fcb0a59c640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcaec06a670 0x7fcaec06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:48.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.980+0000 7fcb0a59c640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcaec06a670 0x7fcaec06cb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcafc036000 tx=0x7fcafc030000 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:48.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:48.983+0000 7fcb02ffd640 1 -- 172.21.15.115:0/4144362544 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcb04053500 con 0x7fcb0c15ce50 2023-12-01T03:11:49.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.172+0000 7fcb10f77640 1 -- 172.21.15.115:0/4144362544 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcb0c06c670 con 0x7fcb0c15ce50 2023-12-01T03:11:49.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.544+0000 7fcb02ffd640 1 -- 172.21.15.115:0/4144362544 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v236) v1 ==== 144+0+0 (secure 0 0 0) 0x7fcb04056b20 con 0x7fcb0c15ce50 2023-12-01T03:11:49.547 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'backup_fs_metadata' created 2023-12-01T03:11:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.548+0000 7fcb10f77640 1 -- 172.21.15.115:0/4144362544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcaec06a670 msgr2=0x7fcaec06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.549+0000 7fcb10f77640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcaec06a670 0x7fcaec06cb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcafc036000 tx=0x7fcafc030000 comp rx=0 tx=0).stop 2023-12-01T03:11:49.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.549+0000 7fcb10f77640 1 -- 172.21.15.115:0/4144362544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c15ce50 msgr2=0x7fcb0c15d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:49.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.549+0000 7fcb10f77640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c15ce50 0x7fcb0c15d770 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c420 tx=0x7fcb0400c8e0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.549+0000 7fcb10f77640 1 -- 172.21.15.115:0/4144362544 shutdown_connections 2023-12-01T03:11:49.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.549+0000 7fcb10f77640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcaec06a670 0x7fcaec06cb30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.549+0000 7fcb10f77640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb0c15ce50 0x7fcb0c15d770 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.549+0000 7fcb10f77640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb0c0cb6f0 0x7fcb0c15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.549+0000 7fcb10f77640 1 --2- 172.21.15.115:0/4144362544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb0c06caa0 0x7fcb0c15f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.549+0000 7fcb10f77640 1 -- 172.21.15.115:0/4144362544 >> 172.21.15.115:0/4144362544 conn(0x7fcb0c0662c0 msgr2=0x7fcb0c0c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.550+0000 7fcb10f77640 1 -- 172.21.15.115:0/4144362544 shutdown_connections 2023-12-01T03:11:49.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.550+0000 7fcb10f77640 1 -- 172.21.15.115:0/4144362544 wait complete. 2023-12-01T03:11:49.578 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T03:11:49.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.784+0000 7f9597316640 1 Processor -- start 2023-12-01T03:11:49.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.784+0000 7f9597316640 1 -- start start 2023-12-01T03:11:49.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.786+0000 7f9597316640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9590143240 0x7f95900b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:49.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.786+0000 7f9597316640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f95900b25e0 0x7f959006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.786+0000 7f9597316640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f959006c8d0 0x7f959015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.786+0000 7f9597316640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959006cd40 con 0x7f959006c8d0 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.786+0000 7f9597316640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95900b2b00 con 0x7f95900b25e0 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.786+0000 7f9597316640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95900b2de0 con 0x7f9590143240 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.786+0000 7f95958b3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f959006c8d0 0x7f959015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.786+0000 7f95958b3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f959006c8d0 0x7f959015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52672/0 (socket says 172.21.15.115:52672) 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.786+0000 7f95958b3640 1 -- 172.21.15.115:0/1366318308 learned_addr learned my addr 172.21.15.115:0/1366318308 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.787+0000 7f95950b2640 1 --2- 172.21.15.115:0/1366318308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9590143240 0x7f95900b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.787+0000 7f95948b1640 1 --2- 172.21.15.115:0/1366318308 >> v2:172.21.15.142:3300/0 conn(0x7f95900b25e0 0x7f959006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.787+0000 7f95958b3640 1 -- 172.21.15.115:0/1366318308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9590143240 msgr2=0x7f95900b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.787+0000 7f95958b3640 1 --2- 172.21.15.115:0/1366318308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9590143240 0x7f95900b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.787+0000 7f95958b3640 1 -- 172.21.15.115:0/1366318308 >> v2:172.21.15.142:3300/0 conn(0x7f95900b25e0 msgr2=0x7f959006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.787+0000 7f95958b3640 1 --2- 172.21.15.115:0/1366318308 >> v2:172.21.15.142:3300/0 conn(0x7f95900b25e0 0x7f959006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.787+0000 7f95958b3640 1 -- 172.21.15.115:0/1366318308 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900b30c0 con 0x7f959006c8d0 2023-12-01T03:11:49.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.788+0000 7f95958b3640 1 --2- 172.21.15.115:0/1366318308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f959006c8d0 0x7f959015d5d0 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f95900cf870 tx=0x7f958802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e5e4df1a95ae47 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:49.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.788+0000 7f9583fff640 1 -- 172.21.15.115:0/1366318308 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958802ca80 con 0x7f959006c8d0 2023-12-01T03:11:49.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.788+0000 7f9583fff640 1 -- 172.21.15.115:0/1366318308 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958802cbe0 con 0x7f959006c8d0 2023-12-01T03:11:49.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.788+0000 7f9583fff640 1 -- 172.21.15.115:0/1366318308 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95880343f0 con 0x7f959006c8d0 2023-12-01T03:11:49.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.789+0000 7f9597316640 1 -- 172.21.15.115:0/1366318308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f959006c8d0 msgr2=0x7f959015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:49.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.789+0000 7f9597316640 1 --2- 172.21.15.115:0/1366318308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f959006c8d0 0x7f959015d5d0 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f95900cf870 tx=0x7f958802bee0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.789+0000 7f9597316640 1 -- 172.21.15.115:0/1366318308 shutdown_connections 2023-12-01T03:11:49.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.789+0000 7f9597316640 1 --2- 172.21.15.115:0/1366318308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f959006c8d0 0x7f959015d5d0 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.789+0000 7f9597316640 1 --2- 172.21.15.115:0/1366318308 >> v2:172.21.15.142:3300/0 conn(0x7f95900b25e0 0x7f959006c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.789+0000 7f9597316640 1 --2- 172.21.15.115:0/1366318308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9590143240 0x7f95900b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.789+0000 7f9597316640 1 -- 172.21.15.115:0/1366318308 >> 172.21.15.115:0/1366318308 conn(0x7f9590065960 msgr2=0x7f9590065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:49.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.789+0000 7f9597316640 1 -- 172.21.15.115:0/1366318308 shutdown_connections 2023-12-01T03:11:49.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.789+0000 7f9597316640 1 -- 172.21.15.115:0/1366318308 wait complete. 2023-12-01T03:11:49.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.792+0000 7f9597316640 1 Processor -- start 2023-12-01T03:11:49.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.792+0000 7f9597316640 1 -- start start 2023-12-01T03:11:49.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.793+0000 7f9597316640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f959006c8d0 0x7f9590138d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:49.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.793+0000 7f9597316640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95900b25e0 0x7f959013f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:49.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.793+0000 7f9597316640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9590143240 0x7f959006ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:49.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.793+0000 7f9597316640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959015dd50 con 0x7f9590143240 2023-12-01T03:11:49.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.793+0000 7f9597316640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959013f940 con 0x7f95900b25e0 2023-12-01T03:11:49.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.793+0000 7f9597316640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959015d420 con 0x7f959006c8d0 2023-12-01T03:11:49.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.793+0000 7f95958b3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9590143240 0x7f959006ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:49.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.793+0000 7f95958b3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9590143240 0x7f959006ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52680/0 (socket says 172.21.15.115:52680) 2023-12-01T03:11:49.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.793+0000 7f95958b3640 1 -- 172.21.15.115:0/2820279267 learned_addr learned my addr 172.21.15.115:0/2820279267 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:49.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.794+0000 7f95950b2640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f959006c8d0 0x7f9590138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.794+0000 7f95948b1640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95900b25e0 0x7f959013f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.794+0000 7f95958b3640 1 -- 172.21.15.115:0/2820279267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f959006c8d0 msgr2=0x7f9590138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.794+0000 7f95958b3640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f959006c8d0 0x7f9590138d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.794+0000 7f95958b3640 1 -- 172.21.15.115:0/2820279267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95900b25e0 msgr2=0x7f959013f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.794+0000 7f95958b3640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95900b25e0 0x7f959013f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.794+0000 7f95958b3640 1 -- 172.21.15.115:0/2820279267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588009b30 con 0x7f9590143240 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.794+0000 7f95958b3640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9590143240 0x7f959006ff10 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f9588036040 tx=0x7f9588004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.795+0000 7f9581ffb640 1 -- 172.21.15.115:0/2820279267 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9588009510 con 0x7f9590143240 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.795+0000 7f9581ffb640 1 -- 172.21.15.115:0/2820279267 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958803e450 con 0x7f9590143240 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.795+0000 7f9581ffb640 1 -- 172.21.15.115:0/2820279267 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958803e800 con 0x7f9590143240 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.795+0000 7f9597316640 1 -- 172.21.15.115:0/2820279267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959015d700 con 0x7f9590143240 2023-12-01T03:11:49.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.795+0000 7f9597316640 1 -- 172.21.15.115:0/2820279267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9590139330 con 0x7f9590143240 2023-12-01T03:11:49.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.796+0000 7f9597316640 1 -- 172.21.15.115:0/2820279267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959015df00 con 0x7f9590143240 2023-12-01T03:11:49.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.797+0000 7f9581ffb640 1 -- 172.21.15.115:0/2820279267 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f958802cd50 con 0x7f9590143240 2023-12-01T03:11:49.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.798+0000 7f9581ffb640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f957406a670 0x7f957406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:49.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.798+0000 7f9581ffb640 1 -- 172.21.15.115:0/2820279267 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6126+0+0 (secure 0 0 0) 0x7f95880abbb0 con 0x7f9590143240 2023-12-01T03:11:49.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.798+0000 7f95950b2640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f957406a670 0x7f957406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:49.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.799+0000 7f95950b2640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f957406a670 0x7f957406cb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f958c006fd0 tx=0x7f958c008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:49.804 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.802+0000 7f9581ffb640 1 -- 172.21.15.115:0/2820279267 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9588076d80 con 0x7f9590143240 2023-12-01T03:11:49.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:49.993+0000 7f9597316640 1 -- 172.21.15.115:0/2820279267 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f959004e890 con 0x7f9590143240 2023-12-01T03:11:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.564+0000 7f9581ffb640 1 -- 172.21.15.115:0/2820279267 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v237) v1 ==== 176+0+0 (secure 0 0 0) 0x7f958807a3a0 con 0x7f9590143240 2023-12-01T03:11:50.566 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'backup_fs_data' created 2023-12-01T03:11:50.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.569+0000 7f9597316640 1 -- 172.21.15.115:0/2820279267 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f957406a670 msgr2=0x7f957406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:50.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.569+0000 7f9597316640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f957406a670 0x7f957406cb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f958c006fd0 tx=0x7f958c008040 comp rx=0 tx=0).stop 2023-12-01T03:11:50.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.569+0000 7f9597316640 1 -- 172.21.15.115:0/2820279267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9590143240 msgr2=0x7f959006ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:50.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.569+0000 7f9597316640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9590143240 0x7f959006ff10 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f9588036040 tx=0x7f9588004a40 comp rx=0 tx=0).stop 2023-12-01T03:11:50.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.569+0000 7f9597316640 1 -- 172.21.15.115:0/2820279267 shutdown_connections 2023-12-01T03:11:50.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.569+0000 7f9597316640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f957406a670 0x7f957406cb30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:50.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.569+0000 7f9597316640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9590143240 0x7f959006ff10 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:50.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.570+0000 7f9597316640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95900b25e0 0x7f959013f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:50.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.570+0000 7f9597316640 1 --2- 172.21.15.115:0/2820279267 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f959006c8d0 0x7f9590138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:50.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.570+0000 7f9597316640 1 -- 172.21.15.115:0/2820279267 >> 172.21.15.115:0/2820279267 conn(0x7f9590065960 msgr2=0x7f959006b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:50.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.570+0000 7f9597316640 1 -- 172.21.15.115:0/2820279267 shutdown_connections 2023-12-01T03:11:50.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.570+0000 7f9597316640 1 -- 172.21.15.115:0/2820279267 wait complete. 2023-12-01T03:11:50.604 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-12-01T03:11:50.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.789+0000 7f34e4700640 1 Processor -- start 2023-12-01T03:11:50.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.789+0000 7f34e4700640 1 -- start start 2023-12-01T03:11:50.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.791+0000 7f34e4700640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0cb570 0x7f34dc0cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:50.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.791+0000 7f34e4700640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc0ce1b0 con 0x7f34dc164c60 2023-12-01T03:11:50.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.791+0000 7f34e4700640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc0ce490 con 0x7f34dc143eb0 2023-12-01T03:11:50.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.791+0000 7f34e4700640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc0cbef0 con 0x7f34dc0cb570 2023-12-01T03:11:50.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.791+0000 7f34e2c9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f34dc164c60 0x7f34dc0cd440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58806/0 (socket says 172.21.15.115:58806) 2023-12-01T03:11:50.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.792+0000 7f34e2c9d640 1 -- 172.21.15.115:0/917696885 learned_addr learned my addr 172.21.15.115:0/917696885 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:50.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.792+0000 7f34e1c9b640 1 --2- 172.21.15.115:0/917696885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0cb570 0x7f34dc0cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:50.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.792+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737804374 0 0) 0x7f34dc0ce1b0 con 0x7f34dc164c60 2023-12-01T03:11:50.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.792+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34c8003520 con 0x7f34dc164c60 2023-12-01T03:11:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.792+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1294369694 0 0) 0x7f34c8003520 con 0x7f34dc164c60 2023-12-01T03:11:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.792+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0cb570 msgr2=0x7f34dc0cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:50.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.792+0000 7f34e149a640 1 --2- 172.21.15.115:0/917696885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0cb570 0x7f34dc0cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.792+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 >> v1:172.21.15.142:6789/0 conn(0x7f34dc143eb0 legacy=0x7f34dc06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.792+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc0caa90 con 0x7f34dc164c60 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.792+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f34d40019f0 con 0x7f34dc164c60 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 >> v1:172.21.15.115:6789/0 conn(0x7f34dc164c60 legacy=0x7f34dc0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e149a640 1 --2- 172.21.15.115:0/917696885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 0x7f34c8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e149a640 1 --2- 172.21.15.115:0/917696885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c80047c0 0x7f34c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e149a640 1 --2- 172.21.15.115:0/917696885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0cb570 0x7f34c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc0cbef0 con 0x7f34c80039a0 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c800ba60 con 0x7f34c80047c0 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c800bc20 con 0x7f34dc0cb570 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e249c640 1 --2- 172.21.15.115:0/917696885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 0x7f34c8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e249c640 1 -- 172.21.15.115:0/917696885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0cb570 msgr2=0x7f34c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e249c640 1 --2- 172.21.15.115:0/917696885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0cb570 0x7f34c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:50.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e249c640 1 -- 172.21.15.115:0/917696885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c80047c0 msgr2=0x7f34c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e249c640 1 --2- 172.21.15.115:0/917696885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34c80047c0 0x7f34c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:50.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.793+0000 7f34e249c640 1 -- 172.21.15.115:0/917696885 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c8003520 con 0x7f34c80039a0 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.794+0000 7f34e249c640 1 --2- 172.21.15.115:0/917696885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 0x7f34c8003dc0 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f34d8001720 tx=0x7f34c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf41956f7870a5d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.795+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d80052d0 con 0x7f34c80039a0 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.795+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d802e050 con 0x7f34c80039a0 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.795+0000 7f34e149a640 1 -- 172.21.15.115:0/917696885 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d80035f0 con 0x7f34c80039a0 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.795+0000 7f34e4700640 1 -- 172.21.15.115:0/917696885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 msgr2=0x7f34c8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.795+0000 7f34e4700640 1 --2- 172.21.15.115:0/917696885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 0x7f34c8003dc0 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f34d8001720 tx=0x7f34c80057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.795+0000 7f34e2c9d640 1 -- 172.21.15.115:0/917696885 reap_dead start 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.795+0000 7f34e4700640 1 -- 172.21.15.115:0/917696885 shutdown_connections 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.795+0000 7f34e4700640 1 -- 172.21.15.115:0/917696885 >> 172.21.15.115:0/917696885 conn(0x7f34dc065960 msgr2=0x7f34dc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.795+0000 7f34e4700640 1 -- 172.21.15.115:0/917696885 shutdown_connections 2023-12-01T03:11:50.798 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.795+0000 7f34e4700640 1 -- 172.21.15.115:0/917696885 wait complete. 2023-12-01T03:11:50.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.797+0000 7f34e4700640 1 Processor -- start 2023-12-01T03:11:50.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.797+0000 7f34e4700640 1 -- start start 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.798+0000 7f34e4700640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 0x7f34dc13cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.798+0000 7f34e4700640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0c3fe0 0x7f34dc0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.798+0000 7f34e4700640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc141000 0x7f34dc141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.798+0000 7f34e4700640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc0c4980 con 0x7f34c80039a0 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.798+0000 7f34e4700640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc141da0 con 0x7f34dc141000 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.798+0000 7f34e4700640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc142080 con 0x7f34dc0c3fe0 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.798+0000 7f34e249c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 0x7f34dc13cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e249c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 0x7f34dc13cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52690/0 (socket says 172.21.15.115:52690) 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e249c640 1 -- 172.21.15.115:0/1384985577 learned_addr learned my addr 172.21.15.115:0/1384985577 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e2c9d640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc141000 0x7f34dc141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e1c9b640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0c3fe0 0x7f34dc0c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e249c640 1 -- 172.21.15.115:0/1384985577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0c3fe0 msgr2=0x7f34dc0c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:50.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e249c640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0c3fe0 0x7f34dc0c4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:50.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e249c640 1 -- 172.21.15.115:0/1384985577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc141000 msgr2=0x7f34dc141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:50.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e249c640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc141000 0x7f34dc141460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:50.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e249c640 1 -- 172.21.15.115:0/1384985577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc142360 con 0x7f34c80039a0 2023-12-01T03:11:50.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e1c9b640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0c3fe0 0x7f34dc0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:50.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e2c9d640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc141000 0x7f34dc141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:50.802 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.799+0000 7f34e249c640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 0x7f34dc13cbb0 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f34d8008fd0 tx=0x7f34d8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:50.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.800+0000 7f34ceffd640 1 -- 172.21.15.115:0/1384985577 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d80076c0 con 0x7f34c80039a0 2023-12-01T03:11:50.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.800+0000 7f34ceffd640 1 -- 172.21.15.115:0/1384985577 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d8007a30 con 0x7f34c80039a0 2023-12-01T03:11:50.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.800+0000 7f34e4700640 1 -- 172.21.15.115:0/1384985577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc142640 con 0x7f34c80039a0 2023-12-01T03:11:50.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.800+0000 7f34ceffd640 1 -- 172.21.15.115:0/1384985577 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8006be0 con 0x7f34c80039a0 2023-12-01T03:11:50.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.800+0000 7f34e4700640 1 -- 172.21.15.115:0/1384985577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc13d0f0 con 0x7f34c80039a0 2023-12-01T03:11:50.803 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.801+0000 7f34e4700640 1 -- 172.21.15.115:0/1384985577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc04e890 con 0x7f34c80039a0 2023-12-01T03:11:50.805 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.803+0000 7f34ceffd640 1 -- 172.21.15.115:0/1384985577 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f34d8006d40 con 0x7f34c80039a0 2023-12-01T03:11:50.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.804+0000 7f34ceffd640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f34c006a6a0 0x7f34c006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:50.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.804+0000 7f34ceffd640 1 -- 172.21.15.115:0/1384985577 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f34d80ab3b0 con 0x7f34c80039a0 2023-12-01T03:11:50.806 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.805+0000 7f34e1c9b640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f34c006a6a0 0x7f34c006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:50.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.805+0000 7f34e1c9b640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f34c006a6a0 0x7f34c006cb60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0c3da0 tx=0x7f34d0008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:50.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:50.808+0000 7f34ceffd640 1 -- 172.21.15.115:0/1384985577 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f34d80763c0 con 0x7f34c80039a0 2023-12-01T03:11:51.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.034+0000 7f34e4700640 1 -- 172.21.15.115:0/1384985577 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f34dc0c4980 con 0x7f34c80039a0 2023-12-01T03:11:51.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.044+0000 7f34ceffd640 1 -- 172.21.15.115:0/1384985577 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 30 and data pool 31 v112) v1 ==== 183+0+0 (secure 0 0 0) 0x7f34d80799e0 con 0x7f34c80039a0 2023-12-01T03:11:51.046 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 30 and data pool 31 2023-12-01T03:11:51.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.050+0000 7f34e4700640 1 -- 172.21.15.115:0/1384985577 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f34c006a6a0 msgr2=0x7f34c006cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.050+0000 7f34e4700640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f34c006a6a0 0x7f34c006cb60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0c3da0 tx=0x7f34d0008040 comp rx=0 tx=0).stop 2023-12-01T03:11:51.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.051+0000 7f34e4700640 1 -- 172.21.15.115:0/1384985577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 msgr2=0x7f34dc13cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.051+0000 7f34e4700640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 0x7f34dc13cbb0 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f34d8008fd0 tx=0x7f34d8001120 comp rx=0 tx=0).stop 2023-12-01T03:11:51.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.051+0000 7f34e4700640 1 -- 172.21.15.115:0/1384985577 shutdown_connections 2023-12-01T03:11:51.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.051+0000 7f34e4700640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f34c006a6a0 0x7f34c006cb60 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0c3da0 tx=0x7f34d0008040 comp rx=0 tx=0).stop 2023-12-01T03:11:51.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.051+0000 7f34e4700640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34dc0c3fe0 0x7f34dc0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.051+0000 7f34e4700640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34dc141000 0x7f34dc141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.051+0000 7f34e4700640 1 --2- 172.21.15.115:0/1384985577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34c80039a0 0x7f34dc13cbb0 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.051+0000 7f34e4700640 1 -- 172.21.15.115:0/1384985577 >> 172.21.15.115:0/1384985577 conn(0x7f34dc065960 msgr2=0x7f34dc0cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:51.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.055+0000 7f34e4700640 1 -- 172.21.15.115:0/1384985577 shutdown_connections 2023-12-01T03:11:51.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.057+0000 7f34e4700640 1 -- 172.21.15.115:0/1384985577 wait complete. 2023-12-01T03:11:51.087 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:51.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.263+0000 7fc1c359e640 1 Processor -- start 2023-12-01T03:11:51.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.263+0000 7fc1c359e640 1 -- start start 2023-12-01T03:11:51.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.265+0000 7fc1c359e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc1c40d26e0 0x7fc1c40d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.265+0000 7fc1c359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1c40cfb20 0x7fc1c40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.265+0000 7fc1c359e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c4160510 con 0x7fc1c4160130 2023-12-01T03:11:51.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.265+0000 7fc1c359e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c40d0510 con 0x7fc1c40cfb20 2023-12-01T03:11:51.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.265+0000 7fc1c359e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c40b6d20 con 0x7fc1c40d26e0 2023-12-01T03:11:51.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.265+0000 7fc1c2d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1c40cfb20 0x7fc1c40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.265+0000 7fc1c2d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc1c40cfb20 0x7fc1c40cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52698/0 (socket says 172.21.15.115:52698) 2023-12-01T03:11:51.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.265+0000 7fc1c2d9d640 1 -- 172.21.15.115:0/4071203136 learned_addr learned my addr 172.21.15.115:0/4071203136 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:51.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.265+0000 7fc1c1d9b640 1 --2- 172.21.15.115:0/4071203136 >> v2:172.21.15.142:3300/0 conn(0x7fc1c40d26e0 0x7fc1c40d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.266+0000 7fc1c2d9d640 1 -- 172.21.15.115:0/4071203136 >> v2:172.21.15.142:3300/0 conn(0x7fc1c40d26e0 msgr2=0x7fc1c40d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:51.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.266+0000 7fc1c2d9d640 1 --2- 172.21.15.115:0/4071203136 >> v2:172.21.15.142:3300/0 conn(0x7fc1c40d26e0 0x7fc1c40d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.266+0000 7fc1c2d9d640 1 -- 172.21.15.115:0/4071203136 >> v1:172.21.15.142:6789/0 conn(0x7fc1c4160130 legacy=0x7fc1c40b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:51.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.266+0000 7fc1c2d9d640 1 -- 172.21.15.115:0/4071203136 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c40d1640 con 0x7fc1c40cfb20 2023-12-01T03:11:51.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.267+0000 7fc1c2d9d640 1 --2- 172.21.15.115:0/4071203136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40cfb20 0x7fc1c40cffd0 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40d0d10 tx=0x7fc1b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05c4331b11082e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:51.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.267+0000 7fc1c159a640 1 -- 172.21.15.115:0/4071203136 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4031870 con 0x7fc1c40cfb20 2023-12-01T03:11:51.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.267+0000 7fc1c159a640 1 -- 172.21.15.115:0/4071203136 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b40319d0 con 0x7fc1c40cfb20 2023-12-01T03:11:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.267+0000 7fc1c159a640 1 -- 172.21.15.115:0/4071203136 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4031d60 con 0x7fc1c40cfb20 2023-12-01T03:11:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.268+0000 7fc1c359e640 1 -- 172.21.15.115:0/4071203136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40cfb20 msgr2=0x7fc1c40cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.268+0000 7fc1c359e640 1 --2- 172.21.15.115:0/4071203136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40cfb20 0x7fc1c40cffd0 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40d0d10 tx=0x7fc1b402b910 comp rx=0 tx=0).stop 2023-12-01T03:11:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.268+0000 7fc1c359e640 1 -- 172.21.15.115:0/4071203136 shutdown_connections 2023-12-01T03:11:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.268+0000 7fc1c359e640 1 --2- 172.21.15.115:0/4071203136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40cfb20 0x7fc1c40cffd0 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.268+0000 7fc1c359e640 1 --2- 172.21.15.115:0/4071203136 >> v2:172.21.15.142:3300/0 conn(0x7fc1c40d26e0 0x7fc1c40d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.268+0000 7fc1c359e640 1 -- 172.21.15.115:0/4071203136 >> 172.21.15.115:0/4071203136 conn(0x7fc1c406a0b0 msgr2=0x7fc1c406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.268+0000 7fc1c359e640 1 -- 172.21.15.115:0/4071203136 shutdown_connections 2023-12-01T03:11:51.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.268+0000 7fc1c359e640 1 -- 172.21.15.115:0/4071203136 wait complete. 2023-12-01T03:11:51.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.270+0000 7fc1c359e640 1 Processor -- start 2023-12-01T03:11:51.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.270+0000 7fc1c359e640 1 -- start start 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.271+0000 7fc1c359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1c40cfb20 0x7fc1c40c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.271+0000 7fc1c359e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40d26e0 0x7fc1c414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.271+0000 7fc1c359e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c4160130 0x7fc1c415c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.271+0000 7fc1c359e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c415aa80 con 0x7fc1c40d26e0 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.271+0000 7fc1c359e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c9cb0 con 0x7fc1c4160130 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.271+0000 7fc1c359e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c6ba0 con 0x7fc1c40cfb20 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.271+0000 7fc1c1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40d26e0 0x7fc1c414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.271+0000 7fc1c1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40d26e0 0x7fc1c414a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52700/0 (socket says 172.21.15.115:52700) 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.271+0000 7fc1c1d9b640 1 -- 172.21.15.115:0/1691141307 learned_addr learned my addr 172.21.15.115:0/1691141307 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.271+0000 7fc1c259c640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1c40cfb20 0x7fc1c40c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1c2d9d640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c4160130 0x7fc1c415c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1c1d9b640 1 -- 172.21.15.115:0/1691141307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1c40cfb20 msgr2=0x7fc1c40c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1c1d9b640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1c40cfb20 0x7fc1c40c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1c1d9b640 1 -- 172.21.15.115:0/1691141307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c4160130 msgr2=0x7fc1c415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1c1d9b640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c4160130 0x7fc1c415c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1c1d9b640 1 -- 172.21.15.115:0/1691141307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40094d0 con 0x7fc1c40d26e0 2023-12-01T03:11:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1c1d9b640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40d26e0 0x7fc1c414a080 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c7f0 tx=0x7fc1ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1baffd640 1 -- 172.21.15.115:0/1691141307 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac004220 con 0x7fc1c40d26e0 2023-12-01T03:11:51.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1baffd640 1 -- 172.21.15.115:0/1691141307 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac004380 con 0x7fc1c40d26e0 2023-12-01T03:11:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1c359e640 1 -- 172.21.15.115:0/1691141307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c40c6e80 con 0x7fc1c40d26e0 2023-12-01T03:11:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1baffd640 1 -- 172.21.15.115:0/1691141307 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac007840 con 0x7fc1c40d26e0 2023-12-01T03:11:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.272+0000 7fc1c359e640 1 -- 172.21.15.115:0/1691141307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c40c7440 con 0x7fc1c40d26e0 2023-12-01T03:11:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.274+0000 7fc1c359e640 1 -- 172.21.15.115:0/1691141307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c40673a0 con 0x7fc1c40d26e0 2023-12-01T03:11:51.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.274+0000 7fc1baffd640 1 -- 172.21.15.115:0/1691141307 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fc1ac007bb0 con 0x7fc1c40d26e0 2023-12-01T03:11:51.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.275+0000 7fc1baffd640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1a406a670 0x7fc1a406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.276+0000 7fc1baffd640 1 -- 172.21.15.115:0/1691141307 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fc1ac089150 con 0x7fc1c40d26e0 2023-12-01T03:11:51.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.276+0000 7fc1c259c640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1a406a670 0x7fc1a406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.276+0000 7fc1c259c640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1a406a670 0x7fc1a406cb30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009f60 tx=0x7fc1bc0094f0 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:51.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.280+0000 7fc1baffd640 1 -- 172.21.15.115:0/1691141307 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc1ac053f40 con 0x7fc1c40d26e0 2023-12-01T03:11:51.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.465+0000 7fc1c359e640 1 -- 172.21.15.115:0/1691141307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1c40c7dc0 con 0x7fc1c40d26e0 2023-12-01T03:11:51.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.469+0000 7fc1baffd640 1 -- 172.21.15.115:0/1691141307 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v238) v1 ==== 74+0+19906 (secure 0 0 0) 0x7fc1ac007e60 con 0x7fc1c40d26e0 2023-12-01T03:11:51.474 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:51.474 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":238,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:51.036774+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:11:49.229904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:11:50.056441+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 -- 172.21.15.115:0/1691141307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1a406a670 msgr2=0x7fc1a406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1a406a670 0x7fc1a406cb30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009f60 tx=0x7fc1bc0094f0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 -- 172.21.15.115:0/1691141307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40d26e0 msgr2=0x7fc1c414a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40d26e0 0x7fc1c414a080 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c7f0 tx=0x7fc1ac00ccb0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 -- 172.21.15.115:0/1691141307 shutdown_connections 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc1a406a670 0x7fc1a406cb30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c4160130 0x7fc1c415c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1c40d26e0 0x7fc1c414a080 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 --2- 172.21.15.115:0/1691141307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1c40cfb20 0x7fc1c40c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 -- 172.21.15.115:0/1691141307 >> 172.21.15.115:0/1691141307 conn(0x7fc1c406a0b0 msgr2=0x7fc1c4157490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.471+0000 7fc1c359e640 1 -- 172.21.15.115:0/1691141307 shutdown_connections 2023-12-01T03:11:51.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.472+0000 7fc1c359e640 1 -- 172.21.15.115:0/1691141307 wait complete. 2023-12-01T03:11:51.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.694+0000 7f8760347640 1 Processor -- start 2023-12-01T03:11:51.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.694+0000 7f8760347640 1 -- start start 2023-12-01T03:11:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f8760347640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f875815cd20 0x7f87580d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f8760347640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f87580d1230 0x7f87580d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f8760347640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87580d1b90 con 0x7f87580b5d80 2023-12-01T03:11:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f8760347640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87580b2160 con 0x7f875815cd20 2023-12-01T03:11:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f8760347640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87580b8c80 con 0x7f87580d1230 2023-12-01T03:11:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f875e0e3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f875815cd20 0x7f87580d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f875e0e3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f875815cd20 0x7f87580d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52704/0 (socket says 172.21.15.115:52704) 2023-12-01T03:11:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f875e0e3640 1 -- 172.21.15.115:0/1903844368 learned_addr learned my addr 172.21.15.115:0/1903844368 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f875e0e3640 1 -- 172.21.15.115:0/1903844368 >> v2:172.21.15.142:3300/0 conn(0x7f87580d1230 msgr2=0x7f87580d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:51.698 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f875e8e4640 1 --2- 172.21.15.115:0/1903844368 >> v2:172.21.15.142:3300/0 conn(0x7f87580d1230 0x7f87580d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f875e0e3640 1 --2- 172.21.15.115:0/1903844368 >> v2:172.21.15.142:3300/0 conn(0x7f87580d1230 0x7f87580d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f875e0e3640 1 -- 172.21.15.115:0/1903844368 >> v1:172.21.15.142:6789/0 conn(0x7f87580b5d80 legacy=0x7f87580b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.696+0000 7f875e0e3640 1 -- 172.21.15.115:0/1903844368 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87580ce2e0 con 0x7f875815cd20 2023-12-01T03:11:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.697+0000 7f875e0e3640 1 --2- 172.21.15.115:0/1903844368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f875815cd20 0x7f87580d24d0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f87580c6720 tx=0x7f875402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34478d26b9ea7b33 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f875d0e1640 1 -- 172.21.15.115:0/1903844368 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540317e0 con 0x7f875815cd20 2023-12-01T03:11:51.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f875d0e1640 1 -- 172.21.15.115:0/1903844368 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8754031940 con 0x7f875815cd20 2023-12-01T03:11:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f875d0e1640 1 -- 172.21.15.115:0/1903844368 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754031cd0 con 0x7f875815cd20 2023-12-01T03:11:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f8760347640 1 -- 172.21.15.115:0/1903844368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f875815cd20 msgr2=0x7f87580d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f8760347640 1 --2- 172.21.15.115:0/1903844368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f875815cd20 0x7f87580d24d0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f87580c6720 tx=0x7f875402b880 comp rx=0 tx=0).stop 2023-12-01T03:11:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f8760347640 1 -- 172.21.15.115:0/1903844368 shutdown_connections 2023-12-01T03:11:51.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f8760347640 1 --2- 172.21.15.115:0/1903844368 >> v2:172.21.15.142:3300/0 conn(0x7f87580d1230 0x7f87580d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f8760347640 1 --2- 172.21.15.115:0/1903844368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f875815cd20 0x7f87580d24d0 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f8760347640 1 -- 172.21.15.115:0/1903844368 >> 172.21.15.115:0/1903844368 conn(0x7f875806a0e0 msgr2=0x7f875806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f8760347640 1 -- 172.21.15.115:0/1903844368 shutdown_connections 2023-12-01T03:11:51.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.698+0000 7f8760347640 1 -- 172.21.15.115:0/1903844368 wait complete. 2023-12-01T03:11:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.700+0000 7f8760347640 1 Processor -- start 2023-12-01T03:11:51.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.700+0000 7f8760347640 1 -- start start 2023-12-01T03:11:51.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.701+0000 7f8760347640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87580b5d80 0x7f8758150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.701+0000 7f8760347640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87580d1230 0x7f87580cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.701+0000 7f8760347640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f875815cd20 0x7f87581f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.701+0000 7f8760347640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87581f6730 con 0x7f87580d1230 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.701+0000 7f8760347640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87581f6a10 con 0x7f875815cd20 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.701+0000 7f8760347640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87581517b0 con 0x7f87580b5d80 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.701+0000 7f875d8e2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87580d1230 0x7f87580cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875d8e2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87580d1230 0x7f87580cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52714/0 (socket says 172.21.15.115:52714) 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875d8e2640 1 -- 172.21.15.115:0/3822909303 learned_addr learned my addr 172.21.15.115:0/3822909303 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875e8e4640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f875815cd20 0x7f87581f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875e0e3640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87580b5d80 0x7f8758150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875d8e2640 1 -- 172.21.15.115:0/3822909303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87580b5d80 msgr2=0x7f8758150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875d8e2640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87580b5d80 0x7f8758150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875d8e2640 1 -- 172.21.15.115:0/3822909303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f875815cd20 msgr2=0x7f87581f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875d8e2640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f875815cd20 0x7f87581f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875d8e2640 1 -- 172.21.15.115:0/3822909303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540094d0 con 0x7f87580d1230 2023-12-01T03:11:51.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875e0e3640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87580b5d80 0x7f8758150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:51.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f875d8e2640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87580d1230 0x7f87580cae50 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f874800c410 tx=0x7f874800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:51.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f874effd640 1 -- 172.21.15.115:0/3822909303 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87480035f0 con 0x7f87580d1230 2023-12-01T03:11:51.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.702+0000 7f874effd640 1 -- 172.21.15.115:0/3822909303 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8748003750 con 0x7f87580d1230 2023-12-01T03:11:51.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.703+0000 7f8760347640 1 -- 172.21.15.115:0/3822909303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8758151a90 con 0x7f87580d1230 2023-12-01T03:11:51.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.703+0000 7f874effd640 1 -- 172.21.15.115:0/3822909303 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748003b40 con 0x7f87580d1230 2023-12-01T03:11:51.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.703+0000 7f8760347640 1 -- 172.21.15.115:0/3822909303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87580cb390 con 0x7f87580d1230 2023-12-01T03:11:51.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.704+0000 7f8760347640 1 -- 172.21.15.115:0/3822909303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8758151cd0 con 0x7f87580d1230 2023-12-01T03:11:51.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.705+0000 7f874effd640 1 -- 172.21.15.115:0/3822909303 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f8748003ca0 con 0x7f87580d1230 2023-12-01T03:11:51.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.705+0000 7f874effd640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f873c06a670 0x7f873c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:51.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.706+0000 7f874effd640 1 -- 172.21.15.115:0/3822909303 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f87480887d0 con 0x7f87580d1230 2023-12-01T03:11:51.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.706+0000 7f875e0e3640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f873c06a670 0x7f873c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:51.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.707+0000 7f875e0e3640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f873c06a670 0x7f873c06cb30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8754036000 tx=0x7f875403d040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:51.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.710+0000 7f874effd640 1 -- 172.21.15.115:0/3822909303 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f87480536c0 con 0x7f87580d1230 2023-12-01T03:11:51.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.901+0000 7f8760347640 1 -- 172.21.15.115:0/3822909303 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87581f6bd0 con 0x7f87580d1230 2023-12-01T03:11:51.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.905+0000 7f874effd640 1 -- 172.21.15.115:0/3822909303 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v238) v1 ==== 74+0+19906 (secure 0 0 0) 0x7f8748056ce0 con 0x7f87580d1230 2023-12-01T03:11:51.908 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:51.908 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":238,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:51.036774+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:11:49.229904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:11:50.056441+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:51.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 -- 172.21.15.115:0/3822909303 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f873c06a670 msgr2=0x7f873c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f873c06a670 0x7f873c06cb30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8754036000 tx=0x7f875403d040 comp rx=0 tx=0).stop 2023-12-01T03:11:51.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 -- 172.21.15.115:0/3822909303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87580d1230 msgr2=0x7f87580cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:51.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87580d1230 0x7f87580cae50 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f874800c410 tx=0x7f874800c8d0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 -- 172.21.15.115:0/3822909303 shutdown_connections 2023-12-01T03:11:51.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f873c06a670 0x7f873c06cb30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f875815cd20 0x7f87581f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87580d1230 0x7f87580cae50 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 --2- 172.21.15.115:0/3822909303 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87580b5d80 0x7f8758150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:51.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 -- 172.21.15.115:0/3822909303 >> 172.21.15.115:0/3822909303 conn(0x7f875806a0e0 msgr2=0x7f87581587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:51.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.908+0000 7f8760347640 1 -- 172.21.15.115:0/3822909303 shutdown_connections 2023-12-01T03:11:51.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:51.909+0000 7f8760347640 1 -- 172.21.15.115:0/3822909303 wait complete. 2023-12-01T03:11:51.930 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-12-01T03:11:52.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.149+0000 7fd476fb9640 1 Processor -- start 2023-12-01T03:11:52.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.150+0000 7fd476fb9640 1 -- start start 2023-12-01T03:11:52.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.151+0000 7fd476fb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd470071740 0x7fd470070440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:52.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.151+0000 7fd476fb9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd47015b200 0x7fd4700ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:52.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.151+0000 7fd476fb9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd47015b680 0x7fd4700cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:52.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.151+0000 7fd476fb9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4700cf950 con 0x7fd47015b680 2023-12-01T03:11:52.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.151+0000 7fd476fb9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47015c1f0 con 0x7fd47015b200 2023-12-01T03:11:52.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.151+0000 7fd476fb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700cba40 con 0x7fd470071740 2023-12-01T03:11:52.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.151+0000 7fd475556640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd47015b680 0x7fd4700cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:52.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.152+0000 7fd475556640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd47015b680 0x7fd4700cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52722/0 (socket says 172.21.15.115:52722) 2023-12-01T03:11:52.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.152+0000 7fd475556640 1 -- 172.21.15.115:0/2958823410 learned_addr learned my addr 172.21.15.115:0/2958823410 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:52.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.152+0000 7fd474d55640 1 --2- 172.21.15.115:0/2958823410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd470071740 0x7fd470070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:52.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.152+0000 7fd46ffff640 1 --2- 172.21.15.115:0/2958823410 >> v2:172.21.15.142:3300/0 conn(0x7fd47015b200 0x7fd4700ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:52.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.152+0000 7fd475556640 1 -- 172.21.15.115:0/2958823410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd470071740 msgr2=0x7fd470070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:52.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.152+0000 7fd475556640 1 --2- 172.21.15.115:0/2958823410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd470071740 0x7fd470070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:52.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.152+0000 7fd475556640 1 -- 172.21.15.115:0/2958823410 >> v2:172.21.15.142:3300/0 conn(0x7fd47015b200 msgr2=0x7fd4700ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:52.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.152+0000 7fd475556640 1 --2- 172.21.15.115:0/2958823410 >> v2:172.21.15.142:3300/0 conn(0x7fd47015b200 0x7fd4700ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:52.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.152+0000 7fd475556640 1 -- 172.21.15.115:0/2958823410 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4701556f0 con 0x7fd47015b680 2023-12-01T03:11:52.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.153+0000 7fd475556640 1 --2- 172.21.15.115:0/2958823410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47015b680 0x7fd4700cf410 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7fd470072430 tx=0x7fd46802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd400c88faf2558 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:52.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.153+0000 7fd46f7fe640 1 -- 172.21.15.115:0/2958823410 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680317e0 con 0x7fd47015b680 2023-12-01T03:11:52.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.154+0000 7fd46f7fe640 1 -- 172.21.15.115:0/2958823410 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468031940 con 0x7fd47015b680 2023-12-01T03:11:52.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.154+0000 7fd46f7fe640 1 -- 172.21.15.115:0/2958823410 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468031cd0 con 0x7fd47015b680 2023-12-01T03:11:52.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.154+0000 7fd476fb9640 1 -- 172.21.15.115:0/2958823410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47015b680 msgr2=0x7fd4700cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:52.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.154+0000 7fd476fb9640 1 --2- 172.21.15.115:0/2958823410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47015b680 0x7fd4700cf410 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7fd470072430 tx=0x7fd46802b880 comp rx=0 tx=0).stop 2023-12-01T03:11:52.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.154+0000 7fd476fb9640 1 -- 172.21.15.115:0/2958823410 shutdown_connections 2023-12-01T03:11:52.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.154+0000 7fd476fb9640 1 --2- 172.21.15.115:0/2958823410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd47015b680 0x7fd4700cf410 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:52.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.154+0000 7fd476fb9640 1 --2- 172.21.15.115:0/2958823410 >> v2:172.21.15.142:3300/0 conn(0x7fd47015b200 0x7fd4700ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:52.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.154+0000 7fd476fb9640 1 --2- 172.21.15.115:0/2958823410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd470071740 0x7fd470070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:52.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.154+0000 7fd476fb9640 1 -- 172.21.15.115:0/2958823410 >> 172.21.15.115:0/2958823410 conn(0x7fd47006bd00 msgr2=0x7fd47006c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:52.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.155+0000 7fd476fb9640 1 -- 172.21.15.115:0/2958823410 shutdown_connections 2023-12-01T03:11:52.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.155+0000 7fd476fb9640 1 -- 172.21.15.115:0/2958823410 wait complete. 2023-12-01T03:11:52.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.156+0000 7fd476fb9640 1 Processor -- start 2023-12-01T03:11:52.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.156+0000 7fd476fb9640 1 -- start start 2023-12-01T03:11:52.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.157+0000 7fd476fb9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd470071740 0x7fd470146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:52.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.157+0000 7fd476fb9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd47015b200 0x7fd470147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:52.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.157+0000 7fd476fb9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd47015b680 0x7fd47015d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:52.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.157+0000 7fd476fb9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4701f2990 con 0x7fd470071740 2023-12-01T03:11:52.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.157+0000 7fd476fb9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4701479f0 con 0x7fd47015b200 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.157+0000 7fd476fb9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd470147cd0 con 0x7fd47015b680 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.158+0000 7fd474d55640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd470071740 0x7fd470146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.158+0000 7fd474d55640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd470071740 0x7fd470146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52736/0 (socket says 172.21.15.115:52736) 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.158+0000 7fd474d55640 1 -- 172.21.15.115:0/1545325703 learned_addr learned my addr 172.21.15.115:0/1545325703 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.158+0000 7fd474d55640 1 -- 172.21.15.115:0/1545325703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd47015b680 msgr2=0x7fd47015d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.158+0000 7fd474d55640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd47015b680 0x7fd47015d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.158+0000 7fd474d55640 1 -- 172.21.15.115:0/1545325703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd47015b200 msgr2=0x7fd470147160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.158+0000 7fd474d55640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd47015b200 0x7fd470147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.158+0000 7fd474d55640 1 -- 172.21.15.115:0/1545325703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680094d0 con 0x7fd470071740 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.158+0000 7fd474d55640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd470071740 0x7fd470146100 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7fd460007e40 tx=0x7fd46000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.159+0000 7fd46d7fa640 1 -- 172.21.15.115:0/1545325703 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460014620 con 0x7fd470071740 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.159+0000 7fd46d7fa640 1 -- 172.21.15.115:0/1545325703 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46000ee90 con 0x7fd470071740 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.159+0000 7fd46d7fa640 1 -- 172.21.15.115:0/1545325703 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600149c0 con 0x7fd470071740 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.159+0000 7fd476fb9640 1 -- 172.21.15.115:0/1545325703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd470147fb0 con 0x7fd470071740 2023-12-01T03:11:52.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.159+0000 7fd476fb9640 1 -- 172.21.15.115:0/1545325703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4701476a0 con 0x7fd470071740 2023-12-01T03:11:52.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.160+0000 7fd476fb9640 1 -- 172.21.15.115:0/1545325703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd470073590 con 0x7fd470071740 2023-12-01T03:11:52.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.161+0000 7fd46d7fa640 1 -- 172.21.15.115:0/1545325703 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fd460014b20 con 0x7fd470071740 2023-12-01T03:11:52.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.162+0000 7fd46d7fa640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd45006a670 0x7fd45006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:52.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.162+0000 7fd46d7fa640 1 -- 172.21.15.115:0/1545325703 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fd460089980 con 0x7fd470071740 2023-12-01T03:11:52.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.162+0000 7fd46ffff640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd45006a670 0x7fd45006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:52.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.163+0000 7fd46ffff640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd45006a670 0x7fd45006cb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd47015e330 tx=0x7fd458008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:52.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.166+0000 7fd46d7fa640 1 -- 172.21.15.115:0/1545325703 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd460054950 con 0x7fd470071740 2023-12-01T03:11:52.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:52.397+0000 7fd476fb9640 1 -- 172.21.15.115:0/1545325703 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd4700b0830 con 0x7fd470071740 2023-12-01T03:11:52.616 INFO:tasks.ceph.mds.b.smithi142.stderr:2023-12-01T03:11:52.615+0000 7f6916e75640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:11:53.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.058+0000 7fd46d7fa640 1 -- 172.21.15.115:0/1545325703 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v115) v1 ==== 121+0+0 (secure 0 0 0) 0x7fd460057f70 con 0x7fd470071740 2023-12-01T03:11:53.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.061+0000 7fd476fb9640 1 -- 172.21.15.115:0/1545325703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd45006a670 msgr2=0x7fd45006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:53.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.061+0000 7fd476fb9640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd45006a670 0x7fd45006cb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd47015e330 tx=0x7fd458008040 comp rx=0 tx=0).stop 2023-12-01T03:11:53.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.061+0000 7fd476fb9640 1 -- 172.21.15.115:0/1545325703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd470071740 msgr2=0x7fd470146100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:53.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.061+0000 7fd476fb9640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd470071740 0x7fd470146100 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7fd460007e40 tx=0x7fd46000e4b0 comp rx=0 tx=0).stop 2023-12-01T03:11:53.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.061+0000 7fd476fb9640 1 -- 172.21.15.115:0/1545325703 shutdown_connections 2023-12-01T03:11:53.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.062+0000 7fd476fb9640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd45006a670 0x7fd45006cb30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:53.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.062+0000 7fd476fb9640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd47015b680 0x7fd47015d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:53.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.062+0000 7fd476fb9640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd47015b200 0x7fd470147160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:53.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.062+0000 7fd476fb9640 1 --2- 172.21.15.115:0/1545325703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd470071740 0x7fd470146100 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:53.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.062+0000 7fd476fb9640 1 -- 172.21.15.115:0/1545325703 >> 172.21.15.115:0/1545325703 conn(0x7fd47006bd00 msgr2=0x7fd4700ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:53.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.062+0000 7fd476fb9640 1 -- 172.21.15.115:0/1545325703 shutdown_connections 2023-12-01T03:11:53.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.062+0000 7fd476fb9640 1 -- 172.21.15.115:0/1545325703 wait complete. 2023-12-01T03:11:53.083 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T03:11:53.083 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-12-01T03:11:53.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.294+0000 7f97df199640 1 Processor -- start 2023-12-01T03:11:53.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.294+0000 7f97df199640 1 -- start start 2023-12-01T03:11:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.296+0000 7f97df199640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d815ac80 0x7f97d80cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.296+0000 7f97df199640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f97d80ce1b0 0x7f97d80b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.296+0000 7f97df199640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d80cc9a0 con 0x7f97d80b0e60 2023-12-01T03:11:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.296+0000 7f97df199640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d815b060 con 0x7f97d80ce1b0 2023-12-01T03:11:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.296+0000 7f97df199640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d80b0ce0 con 0x7f97d815ac80 2023-12-01T03:11:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.296+0000 7f97dd736640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f97d80ce1b0 0x7f97d80b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.296+0000 7f97dd736640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f97d80ce1b0 0x7f97d80b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52746/0 (socket says 172.21.15.115:52746) 2023-12-01T03:11:53.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.296+0000 7f97dd736640 1 -- 172.21.15.115:0/689106475 learned_addr learned my addr 172.21.15.115:0/689106475 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:53.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.296+0000 7f97dcf35640 1 --2- 172.21.15.115:0/689106475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d815ac80 0x7f97d80cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:53.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.297+0000 7f97dd736640 1 -- 172.21.15.115:0/689106475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d815ac80 msgr2=0x7f97d80cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:53.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.297+0000 7f97dd736640 1 --2- 172.21.15.115:0/689106475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d815ac80 0x7f97d80cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:53.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.297+0000 7f97dd736640 1 -- 172.21.15.115:0/689106475 >> v1:172.21.15.142:6789/0 conn(0x7f97d80b0e60 legacy=0x7f97d80cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:53.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.297+0000 7f97dd736640 1 -- 172.21.15.115:0/689106475 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d80cbd90 con 0x7f97d80ce1b0 2023-12-01T03:11:53.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.298+0000 7f97dd736640 1 --2- 172.21.15.115:0/689106475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d80ce1b0 0x7f97d80b07a0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009e50 tx=0x7f97d002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a1355520e5a765 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:53.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.298+0000 7f97d77fe640 1 -- 172.21.15.115:0/689106475 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d002cb00 con 0x7f97d80ce1b0 2023-12-01T03:11:53.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.298+0000 7f97d77fe640 1 -- 172.21.15.115:0/689106475 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d002cc60 con 0x7f97d80ce1b0 2023-12-01T03:11:53.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.298+0000 7f97d77fe640 1 -- 172.21.15.115:0/689106475 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d00343f0 con 0x7f97d80ce1b0 2023-12-01T03:11:53.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.299+0000 7f97df199640 1 -- 172.21.15.115:0/689106475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d80ce1b0 msgr2=0x7f97d80b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:53.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.299+0000 7f97df199640 1 --2- 172.21.15.115:0/689106475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d80ce1b0 0x7f97d80b07a0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009e50 tx=0x7f97d002bf60 comp rx=0 tx=0).stop 2023-12-01T03:11:53.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.299+0000 7f97df199640 1 -- 172.21.15.115:0/689106475 shutdown_connections 2023-12-01T03:11:53.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.299+0000 7f97df199640 1 --2- 172.21.15.115:0/689106475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d80ce1b0 0x7f97d80b07a0 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:53.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.299+0000 7f97df199640 1 --2- 172.21.15.115:0/689106475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d815ac80 0x7f97d80cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:53.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.299+0000 7f97df199640 1 -- 172.21.15.115:0/689106475 >> 172.21.15.115:0/689106475 conn(0x7f97d8065960 msgr2=0x7f97d8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:53.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.299+0000 7f97df199640 1 -- 172.21.15.115:0/689106475 shutdown_connections 2023-12-01T03:11:53.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.299+0000 7f97df199640 1 -- 172.21.15.115:0/689106475 wait complete. 2023-12-01T03:11:53.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.301+0000 7f97df199640 1 Processor -- start 2023-12-01T03:11:53.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.301+0000 7f97df199640 1 -- start start 2023-12-01T03:11:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97df199640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d80b0e60 0x7f97d8152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97df199640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d80ce1b0 0x7f97d814b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97df199640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d815ac80 0x7f97d814c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97df199640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d814c600 con 0x7f97d815ac80 2023-12-01T03:11:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97df199640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d815d130 con 0x7f97d80b0e60 2023-12-01T03:11:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97df199640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d814c930 con 0x7f97d80ce1b0 2023-12-01T03:11:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97dd736640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d815ac80 0x7f97d814c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97dd736640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d815ac80 0x7f97d814c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52758/0 (socket says 172.21.15.115:52758) 2023-12-01T03:11:53.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97dd736640 1 -- 172.21.15.115:0/1149820045 learned_addr learned my addr 172.21.15.115:0/1149820045 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:53.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97d7fff640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d80ce1b0 0x7f97d814b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.302+0000 7f97dcf35640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d80b0e60 0x7f97d8152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.303+0000 7f97dd736640 1 -- 172.21.15.115:0/1149820045 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d80ce1b0 msgr2=0x7f97d814b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.303+0000 7f97dd736640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d80ce1b0 0x7f97d814b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.303+0000 7f97dd736640 1 -- 172.21.15.115:0/1149820045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d80b0e60 msgr2=0x7f97d8152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.303+0000 7f97dd736640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d80b0e60 0x7f97d8152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.303+0000 7f97dd736640 1 -- 172.21.15.115:0/1149820045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d0009b30 con 0x7f97d815ac80 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.303+0000 7f97dd736640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d815ac80 0x7f97d814c0c0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009e50 tx=0x7f97d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.303+0000 7f97d57fa640 1 -- 172.21.15.115:0/1149820045 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0009510 con 0x7f97d815ac80 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.303+0000 7f97d57fa640 1 -- 172.21.15.115:0/1149820045 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d003e450 con 0x7f97d815ac80 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.303+0000 7f97d57fa640 1 -- 172.21.15.115:0/1149820045 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d003e800 con 0x7f97d815ac80 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.303+0000 7f97df199640 1 -- 172.21.15.115:0/1149820045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d814bde0 con 0x7f97d815ac80 2023-12-01T03:11:53.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.304+0000 7f97df199640 1 -- 172.21.15.115:0/1149820045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d815cdc0 con 0x7f97d815ac80 2023-12-01T03:11:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.305+0000 7f97df199640 1 -- 172.21.15.115:0/1149820045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d815d2c0 con 0x7f97d815ac80 2023-12-01T03:11:53.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.306+0000 7f97d57fa640 1 -- 172.21.15.115:0/1149820045 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f97d003e960 con 0x7f97d815ac80 2023-12-01T03:11:53.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.306+0000 7f97d57fa640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f97bc06a670 0x7f97bc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:53.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.307+0000 7f97d57fa640 1 -- 172.21.15.115:0/1149820045 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f97d00abe00 con 0x7f97d815ac80 2023-12-01T03:11:53.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.307+0000 7f97dcf35640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f97bc06a670 0x7f97bc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:53.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.307+0000 7f97dcf35640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f97bc06a670 0x7f97bc06cb30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f97d81f5b80 tx=0x7f97cc008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:53.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.311+0000 7f97d57fa640 1 -- 172.21.15.115:0/1149820045 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f97d0076cf0 con 0x7f97d815ac80 2023-12-01T03:11:53.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:53.539+0000 7f97df199640 1 -- 172.21.15.115:0/1149820045 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f97d804e890 con 0x7f97d815ac80 2023-12-01T03:11:54.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.062+0000 7f97d57fa640 1 -- 172.21.15.115:0/1149820045 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v116) v1 ==== 125+0+0 (secure 0 0 0) 0x7f97d007a310 con 0x7f97d815ac80 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.065+0000 7f97df199640 1 -- 172.21.15.115:0/1149820045 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f97bc06a670 msgr2=0x7f97bc06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.065+0000 7f97df199640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f97bc06a670 0x7f97bc06cb30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f97d81f5b80 tx=0x7f97cc008040 comp rx=0 tx=0).stop 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.065+0000 7f97df199640 1 -- 172.21.15.115:0/1149820045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d815ac80 msgr2=0x7f97d814c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.065+0000 7f97df199640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d815ac80 0x7f97d814c0c0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009e50 tx=0x7f97d0004a40 comp rx=0 tx=0).stop 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.066+0000 7f97df199640 1 -- 172.21.15.115:0/1149820045 shutdown_connections 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.066+0000 7f97df199640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f97bc06a670 0x7f97bc06cb30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.066+0000 7f97df199640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97d80ce1b0 0x7f97d814b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.066+0000 7f97df199640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97d815ac80 0x7f97d814c0c0 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.066+0000 7f97df199640 1 --2- 172.21.15.115:0/1149820045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97d80b0e60 0x7f97d8152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.066+0000 7f97df199640 1 -- 172.21.15.115:0/1149820045 >> 172.21.15.115:0/1149820045 conn(0x7f97d8065960 msgr2=0x7f97d80ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:54.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.066+0000 7f97df199640 1 -- 172.21.15.115:0/1149820045 shutdown_connections 2023-12-01T03:11:54.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.066+0000 7f97df199640 1 -- 172.21.15.115:0/1149820045 wait complete. 2023-12-01T03:11:54.087 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-12-01T03:11:54.087 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:54.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.305+0000 7ff8bf59e640 1 Processor -- start 2023-12-01T03:11:54.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.305+0000 7ff8bf59e640 1 -- start start 2023-12-01T03:11:54.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.306+0000 7ff8bf59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c00d05d0 0x7ff8c00d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.306+0000 7ff8bf59e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c015e020 con 0x7ff8c015db90 2023-12-01T03:11:54.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.306+0000 7ff8bf59e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c00cd3f0 con 0x7ff8c015d7b0 2023-12-01T03:11:54.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.306+0000 7ff8bf59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00cd0f0 con 0x7ff8c00d05d0 2023-12-01T03:11:54.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.307+0000 7ff8b7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff8c015db90 0x7ff8c00cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58810/0 (socket says 172.21.15.115:58810) 2023-12-01T03:11:54.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.307+0000 7ff8b7fff640 1 -- 172.21.15.115:0/929646663 learned_addr learned my addr 172.21.15.115:0/929646663 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:54.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.307+0000 7ff8bed9d640 1 --2- 172.21.15.115:0/929646663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c00d05d0 0x7ff8c00d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.307+0000 7ff8bdd9b640 1 -- 172.21.15.115:0/929646663 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728565305 0 0) 0x7ff8c015e020 con 0x7ff8c015db90 2023-12-01T03:11:54.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.307+0000 7ff8bdd9b640 1 -- 172.21.15.115:0/929646663 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8ac003520 con 0x7ff8c015db90 2023-12-01T03:11:54.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.307+0000 7ff8bdd9b640 1 -- 172.21.15.115:0/929646663 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528669620 0 0) 0x7ff8c00cd3f0 con 0x7ff8c015d7b0 2023-12-01T03:11:54.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.308+0000 7ff8bdd9b640 1 -- 172.21.15.115:0/929646663 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c015e020 con 0x7ff8c015d7b0 2023-12-01T03:11:54.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.308+0000 7ff8bdd9b640 1 -- 172.21.15.115:0/929646663 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333989428 0 0) 0x7ff8ac003520 con 0x7ff8c015db90 2023-12-01T03:11:54.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.308+0000 7ff8bed9d640 1 -- 172.21.15.115:0/929646663 >> v1:172.21.15.142:6789/0 conn(0x7ff8c015d7b0 legacy=0x7ff8c0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.308+0000 7ff8bed9d640 1 -- 172.21.15.115:0/929646663 >> v1:172.21.15.115:6789/0 conn(0x7ff8c015db90 legacy=0x7ff8c00cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.308+0000 7ff8bed9d640 1 -- 172.21.15.115:0/929646663 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c0160ec0 con 0x7ff8c00d05d0 2023-12-01T03:11:54.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.309+0000 7ff8bed9d640 1 --2- 172.21.15.115:0/929646663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c00d05d0 0x7ff8c00d0a60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009dc0 tx=0x7ff8b802be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e36aebd573b3493 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:54.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.310+0000 7ff8bdd9b640 1 -- 172.21.15.115:0/929646663 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8001670 con 0x7ff8c00d05d0 2023-12-01T03:11:54.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.310+0000 7ff8bdd9b640 1 -- 172.21.15.115:0/929646663 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b802c9a0 con 0x7ff8c00d05d0 2023-12-01T03:11:54.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.310+0000 7ff8bdd9b640 1 -- 172.21.15.115:0/929646663 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b802cd10 con 0x7ff8c00d05d0 2023-12-01T03:11:54.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.311+0000 7ff8bf59e640 1 -- 172.21.15.115:0/929646663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c00d05d0 msgr2=0x7ff8c00d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.311+0000 7ff8bf59e640 1 --2- 172.21.15.115:0/929646663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c00d05d0 0x7ff8c00d0a60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009dc0 tx=0x7ff8b802be70 comp rx=0 tx=0).stop 2023-12-01T03:11:54.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.311+0000 7ff8bf59e640 1 -- 172.21.15.115:0/929646663 shutdown_connections 2023-12-01T03:11:54.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.311+0000 7ff8bf59e640 1 --2- 172.21.15.115:0/929646663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c00d05d0 0x7ff8c00d0a60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.311+0000 7ff8bf59e640 1 -- 172.21.15.115:0/929646663 >> 172.21.15.115:0/929646663 conn(0x7ff8c006a0e0 msgr2=0x7ff8c006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:54.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.311+0000 7ff8bf59e640 1 -- 172.21.15.115:0/929646663 shutdown_connections 2023-12-01T03:11:54.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.311+0000 7ff8bf59e640 1 -- 172.21.15.115:0/929646663 wait complete. 2023-12-01T03:11:54.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.313+0000 7ff8bf59e640 1 Processor -- start 2023-12-01T03:11:54.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.313+0000 7ff8bf59e640 1 -- start start 2023-12-01T03:11:54.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8bf59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8c00d05d0 0x7ff8c01541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8bf59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c015d7b0 0x7ff8c0158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8bf59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c015db90 0x7ff8c00cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8bf59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0154710 con 0x7ff8c00d05d0 2023-12-01T03:11:54.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8bf59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00cac50 con 0x7ff8c015db90 2023-12-01T03:11:54.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8bf59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00caf30 con 0x7ff8c015d7b0 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8be59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8c00d05d0 0x7ff8c01541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8be59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8c00d05d0 0x7ff8c01541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52762/0 (socket says 172.21.15.115:52762) 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8be59c640 1 -- 172.21.15.115:0/3892189199 learned_addr learned my addr 172.21.15.115:0/3892189199 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8b7fff640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c015d7b0 0x7ff8c0158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8bed9d640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c015db90 0x7ff8c00cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8be59c640 1 -- 172.21.15.115:0/3892189199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c015d7b0 msgr2=0x7ff8c0158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8be59c640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c015d7b0 0x7ff8c0158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8be59c640 1 -- 172.21.15.115:0/3892189199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c015db90 msgr2=0x7ff8c00cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8be59c640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c015db90 0x7ff8c00cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.314+0000 7ff8be59c640 1 -- 172.21.15.115:0/3892189199 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8009a70 con 0x7ff8c00d05d0 2023-12-01T03:11:54.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.315+0000 7ff8be59c640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8c00d05d0 0x7ff8c01541d0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c500 tx=0x7ff8b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:54.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.315+0000 7ff8b6ffd640 1 -- 172.21.15.115:0/3892189199 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00052d0 con 0x7ff8c00d05d0 2023-12-01T03:11:54.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.315+0000 7ff8b6ffd640 1 -- 172.21.15.115:0/3892189199 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b0005470 con 0x7ff8c00d05d0 2023-12-01T03:11:54.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.315+0000 7ff8b6ffd640 1 -- 172.21.15.115:0/3892189199 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0003830 con 0x7ff8c00d05d0 2023-12-01T03:11:54.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.315+0000 7ff8bf59e640 1 -- 172.21.15.115:0/3892189199 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c00cb210 con 0x7ff8c00d05d0 2023-12-01T03:11:54.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.315+0000 7ff8bf59e640 1 -- 172.21.15.115:0/3892189199 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c00cb7d0 con 0x7ff8c00d05d0 2023-12-01T03:11:54.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.316+0000 7ff8bf59e640 1 -- 172.21.15.115:0/3892189199 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c00673a0 con 0x7ff8c00d05d0 2023-12-01T03:11:54.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.317+0000 7ff8b6ffd640 1 -- 172.21.15.115:0/3892189199 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7ff8b0003990 con 0x7ff8c00d05d0 2023-12-01T03:11:54.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.318+0000 7ff8b6ffd640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff8a006a670 0x7ff8a006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.318+0000 7ff8b6ffd640 1 -- 172.21.15.115:0/3892189199 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7ff8b0088640 con 0x7ff8c00d05d0 2023-12-01T03:11:54.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.318+0000 7ff8b7fff640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff8a006a670 0x7ff8a006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.319+0000 7ff8b7fff640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff8a006a670 0x7ff8a006cb30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8001380 tx=0x7ff8a8009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:54.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.322+0000 7ff8b6ffd640 1 -- 172.21.15.115:0/3892189199 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff8b0053560 con 0x7ff8c00d05d0 2023-12-01T03:11:54.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.548+0000 7ff8bf59e640 1 -- 172.21.15.115:0/3892189199 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8c00d3100 con 0x7ff8c00d05d0 2023-12-01T03:11:54.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.549+0000 7ff8b6ffd640 1 -- 172.21.15.115:0/3892189199 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5786 (secure 0 0 0) 0x7ff8b0056b80 con 0x7ff8c00d05d0 2023-12-01T03:11:54.551 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:54.551 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:32.176438+0000","modified":"2023-12-01T03:11:35.672576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7735":{"gid":7735,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1837882056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1837882056},{"type":"v1","addr":"172.21.15.142:6835","nonce":1837882056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:51.037525+0000","modified":"2023-12-01T03:11:54.055433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7732":{"gid":7732,"name":"b","rank":0,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-01T03:11:54.551 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 116 2023-12-01T03:11:54.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.551+0000 7ff8bf59e640 1 -- 172.21.15.115:0/3892189199 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff8a006a670 msgr2=0x7ff8a006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.551+0000 7ff8bf59e640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff8a006a670 0x7ff8a006cb30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8001380 tx=0x7ff8a8009040 comp rx=0 tx=0).stop 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.551+0000 7ff8bf59e640 1 -- 172.21.15.115:0/3892189199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8c00d05d0 msgr2=0x7ff8c01541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.551+0000 7ff8bf59e640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8c00d05d0 0x7ff8c01541d0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c500 tx=0x7ff8b000c9c0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.552+0000 7ff8bf59e640 1 -- 172.21.15.115:0/3892189199 shutdown_connections 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.552+0000 7ff8bf59e640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff8a006a670 0x7ff8a006cb30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.552+0000 7ff8bf59e640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8c015d7b0 0x7ff8c0158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.552+0000 7ff8bf59e640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8c015db90 0x7ff8c00cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.552+0000 7ff8bf59e640 1 --2- 172.21.15.115:0/3892189199 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8c00d05d0 0x7ff8c01541d0 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.552+0000 7ff8bf59e640 1 -- 172.21.15.115:0/3892189199 >> 172.21.15.115:0/3892189199 conn(0x7ff8c006a0e0 msgr2=0x7ff8c0067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.552+0000 7ff8bf59e640 1 -- 172.21.15.115:0/3892189199 shutdown_connections 2023-12-01T03:11:54.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.552+0000 7ff8bf59e640 1 -- 172.21.15.115:0/3892189199 wait complete. 2023-12-01T03:11:54.572 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:11:54.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.782+0000 7fc182817640 1 Processor -- start 2023-12-01T03:11:54.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.782+0000 7fc182817640 1 -- start start 2023-12-01T03:11:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc182817640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c159dd0 0x7fc17c0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc182817640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc17c070d30 0x7fc17c0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc182817640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17c06f8b0 con 0x7fc17c06b6f0 2023-12-01T03:11:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc182817640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17c0adcb0 con 0x7fc17c070d30 2023-12-01T03:11:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc182817640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0ad9d0 con 0x7fc17c159dd0 2023-12-01T03:11:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc180db4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc17c070d30 0x7fc17c0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc180db4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc17c070d30 0x7fc17c0c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52772/0 (socket says 172.21.15.115:52772) 2023-12-01T03:11:54.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc180db4640 1 -- 172.21.15.115:0/2416110012 learned_addr learned my addr 172.21.15.115:0/2416110012 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:54.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc17bfff640 1 --2- 172.21.15.115:0/2416110012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c159dd0 0x7fc17c0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc180db4640 1 -- 172.21.15.115:0/2416110012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c159dd0 msgr2=0x7fc17c0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:54.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc180db4640 1 --2- 172.21.15.115:0/2416110012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c159dd0 0x7fc17c0c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc180db4640 1 -- 172.21.15.115:0/2416110012 >> v1:172.21.15.142:6789/0 conn(0x7fc17c06b6f0 legacy=0x7fc17c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:54.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.784+0000 7fc180db4640 1 -- 172.21.15.115:0/2416110012 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c0c9000 con 0x7fc17c070d30 2023-12-01T03:11:54.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.785+0000 7fc17bfff640 1 --2- 172.21.15.115:0/2416110012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c159dd0 0x7fc17c0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.786+0000 7fc180db4640 1 --2- 172.21.15.115:0/2416110012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c070d30 0x7fc17c0c9d20 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7fc17400bb10 tx=0x7fc17402dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8fb265f3d6203c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.786+0000 7fc17affd640 1 -- 172.21.15.115:0/2416110012 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17402ed50 con 0x7fc17c070d30 2023-12-01T03:11:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.786+0000 7fc17affd640 1 -- 172.21.15.115:0/2416110012 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc174036440 con 0x7fc17c070d30 2023-12-01T03:11:54.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.786+0000 7fc17affd640 1 -- 172.21.15.115:0/2416110012 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1740367d0 con 0x7fc17c070d30 2023-12-01T03:11:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.786+0000 7fc182817640 1 -- 172.21.15.115:0/2416110012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c070d30 msgr2=0x7fc17c0c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.786+0000 7fc182817640 1 --2- 172.21.15.115:0/2416110012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c070d30 0x7fc17c0c9d20 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7fc17400bb10 tx=0x7fc17402dfa0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.787+0000 7fc182817640 1 -- 172.21.15.115:0/2416110012 shutdown_connections 2023-12-01T03:11:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.787+0000 7fc182817640 1 --2- 172.21.15.115:0/2416110012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c070d30 0x7fc17c0c9d20 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.787+0000 7fc182817640 1 --2- 172.21.15.115:0/2416110012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c159dd0 0x7fc17c0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.787+0000 7fc182817640 1 -- 172.21.15.115:0/2416110012 >> 172.21.15.115:0/2416110012 conn(0x7fc17c0ce880 msgr2=0x7fc17c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.787+0000 7fc182817640 1 -- 172.21.15.115:0/2416110012 shutdown_connections 2023-12-01T03:11:54.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.787+0000 7fc182817640 1 -- 172.21.15.115:0/2416110012 wait complete. 2023-12-01T03:11:54.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.789+0000 7fc182817640 1 Processor -- start 2023-12-01T03:11:54.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.789+0000 7fc182817640 1 -- start start 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc182817640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c06b6f0 0x7fc17c14da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc182817640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c070d30 0x7fc17c15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc182817640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc17c159dd0 0x7fc17c15bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc182817640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c15c170 con 0x7fc17c06b6f0 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc182817640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c1f16a0 con 0x7fc17c159dd0 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc182817640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c1f1980 con 0x7fc17c070d30 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc17bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c06b6f0 0x7fc17c14da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc17bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c06b6f0 0x7fc17c14da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52778/0 (socket says 172.21.15.115:52778) 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc17bfff640 1 -- 172.21.15.115:0/3788691706 learned_addr learned my addr 172.21.15.115:0/3788691706 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc17b7fe640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c070d30 0x7fc17c15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc180db4640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc17c159dd0 0x7fc17c15bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.793 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc17bfff640 1 -- 172.21.15.115:0/3788691706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c070d30 msgr2=0x7fc17c15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc17bfff640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c070d30 0x7fc17c15b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc17bfff640 1 -- 172.21.15.115:0/3788691706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc17c159dd0 msgr2=0x7fc17c15bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc17bfff640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc17c159dd0 0x7fc17c15bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc17bfff640 1 -- 172.21.15.115:0/3788691706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17400b770 con 0x7fc17c06b6f0 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.791+0000 7fc180db4640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc17c159dd0 0x7fc17c15bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.790+0000 7fc17b7fe640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c070d30 0x7fc17c15b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.791+0000 7fc17bfff640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c06b6f0 0x7fc17c14da70 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00ce00 tx=0x7fc16c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.791+0000 7fc178ff9640 1 -- 172.21.15.115:0/3788691706 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c00f890 con 0x7fc17c06b6f0 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.791+0000 7fc178ff9640 1 -- 172.21.15.115:0/3788691706 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c00f9f0 con 0x7fc17c06b6f0 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.791+0000 7fc178ff9640 1 -- 172.21.15.115:0/3788691706 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c00fdc0 con 0x7fc17c06b6f0 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.791+0000 7fc182817640 1 -- 172.21.15.115:0/3788691706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c1f1c60 con 0x7fc17c06b6f0 2023-12-01T03:11:54.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.791+0000 7fc182817640 1 -- 172.21.15.115:0/3788691706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c0c3dd0 con 0x7fc17c06b6f0 2023-12-01T03:11:54.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.792+0000 7fc182817640 1 -- 172.21.15.115:0/3788691706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c0cc270 con 0x7fc17c06b6f0 2023-12-01T03:11:54.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.793+0000 7fc178ff9640 1 -- 172.21.15.115:0/3788691706 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fc16c00b040 con 0x7fc17c06b6f0 2023-12-01T03:11:54.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.794+0000 7fc178ff9640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc15c06a670 0x7fc15c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:54.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.794+0000 7fc178ff9640 1 -- 172.21.15.115:0/3788691706 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fc16c089d00 con 0x7fc17c06b6f0 2023-12-01T03:11:54.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.795+0000 7fc17b7fe640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc15c06a670 0x7fc15c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:54.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.795+0000 7fc17b7fe640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc15c06a670 0x7fc15c06cb30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc164007fd0 tx=0x7fc164009040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:54.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.799+0000 7fc178ff9640 1 -- 172.21.15.115:0/3788691706 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc16c0549c0 con 0x7fc17c06b6f0 2023-12-01T03:11:54.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.996+0000 7fc182817640 1 -- 172.21.15.115:0/3788691706 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc17c04e890 con 0x7fc17c06b6f0 2023-12-01T03:11:55.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:54.998+0000 7fc178ff9640 1 -- 172.21.15.115:0/3788691706 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+19893 (secure 0 0 0) 0x7fc16c01d030 con 0x7fc17c06b6f0 2023-12-01T03:11:55.000 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:55.001 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":239,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:11:52.042614+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:11:49.229904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:11:50.056441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:11:55.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 -- 172.21.15.115:0/3788691706 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc15c06a670 msgr2=0x7fc15c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc15c06a670 0x7fc15c06cb30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc164007fd0 tx=0x7fc164009040 comp rx=0 tx=0).stop 2023-12-01T03:11:55.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 -- 172.21.15.115:0/3788691706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c06b6f0 msgr2=0x7fc17c14da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c06b6f0 0x7fc17c14da70 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00ce00 tx=0x7fc16c00a650 comp rx=0 tx=0).stop 2023-12-01T03:11:55.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 -- 172.21.15.115:0/3788691706 shutdown_connections 2023-12-01T03:11:55.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc15c06a670 0x7fc15c06cb30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc17c070d30 0x7fc17c15b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc17c159dd0 0x7fc17c15bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 --2- 172.21.15.115:0/3788691706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc17c06b6f0 0x7fc17c14da70 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 -- 172.21.15.115:0/3788691706 >> 172.21.15.115:0/3788691706 conn(0x7fc17c0ce880 msgr2=0x7fc17c0ade40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:55.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 -- 172.21.15.115:0/3788691706 shutdown_connections 2023-12-01T03:11:55.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.001+0000 7fc182817640 1 -- 172.21.15.115:0/3788691706 wait complete. 2023-12-01T03:11:55.023 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T03:11:55.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:11:55.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.236+0000 7f0a2da96640 1 Processor -- start 2023-12-01T03:11:55.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.236+0000 7f0a2da96640 1 -- start start 2023-12-01T03:11:55.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.238+0000 7f0a2da96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 0x7f0a280ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.238+0000 7f0a2da96640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0a2806b350 0x7f0a2806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.238+0000 7f0a2da96640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a280ce150 con 0x7f0a280adac0 2023-12-01T03:11:55.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.238+0000 7f0a2da96640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a280cf3a0 con 0x7f0a2806b350 2023-12-01T03:11:55.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.238+0000 7f0a2da96640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280ce5e0 con 0x7f0a280c8ed0 2023-12-01T03:11:55.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.238+0000 7f0a26ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 0x7f0a280ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:55.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.238+0000 7f0a27fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0a2806b350 0x7f0a2806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:55.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.238+0000 7f0a27fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0a2806b350 0x7f0a2806b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:59150/0 (socket says 172.21.15.115:59150) 2023-12-01T03:11:55.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.238+0000 7f0a27fff640 1 -- 172.21.15.115:0/2354089982 learned_addr learned my addr 172.21.15.115:0/2354089982 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:55.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.238+0000 7f0a26ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 0x7f0a280ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:55950/0 (socket says 172.21.15.115:55950) 2023-12-01T03:11:55.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.239+0000 7f0a26ffd640 1 -- 172.21.15.115:0/2354089982 >> v2:172.21.15.142:3300/0 conn(0x7f0a2806b350 msgr2=0x7f0a2806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:55.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.239+0000 7f0a26ffd640 1 --2- 172.21.15.115:0/2354089982 >> v2:172.21.15.142:3300/0 conn(0x7f0a2806b350 0x7f0a2806b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.239+0000 7f0a26ffd640 1 -- 172.21.15.115:0/2354089982 >> v1:172.21.15.142:6789/0 conn(0x7f0a280adac0 legacy=0x7f0a280cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.239+0000 7f0a26ffd640 1 -- 172.21.15.115:0/2354089982 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2806d620 con 0x7f0a280c8ed0 2023-12-01T03:11:55.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.241+0000 7f0a26ffd640 1 --2- 172.21.15.115:0/2354089982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 0x7f0a280ccb60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0097a0 tx=0x7f0a1c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d767c5ae5e345a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:55.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.241+0000 7f0a267fc640 1 -- 172.21.15.115:0/2354089982 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c02d040 con 0x7f0a280c8ed0 2023-12-01T03:11:55.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.241+0000 7f0a267fc640 1 -- 172.21.15.115:0/2354089982 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1c031810 con 0x7f0a280c8ed0 2023-12-01T03:11:55.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.241+0000 7f0a267fc640 1 -- 172.21.15.115:0/2354089982 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c031ba0 con 0x7f0a280c8ed0 2023-12-01T03:11:55.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.242+0000 7f0a2da96640 1 -- 172.21.15.115:0/2354089982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 msgr2=0x7f0a280ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.242+0000 7f0a2da96640 1 --2- 172.21.15.115:0/2354089982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 0x7f0a280ccb60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0097a0 tx=0x7f0a1c02b8b0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.242+0000 7f0a2da96640 1 -- 172.21.15.115:0/2354089982 shutdown_connections 2023-12-01T03:11:55.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.242+0000 7f0a2da96640 1 --2- 172.21.15.115:0/2354089982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 0x7f0a280ccb60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.242+0000 7f0a2da96640 1 --2- 172.21.15.115:0/2354089982 >> v2:172.21.15.142:3300/0 conn(0x7f0a2806b350 0x7f0a2806b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.242+0000 7f0a2da96640 1 -- 172.21.15.115:0/2354089982 >> 172.21.15.115:0/2354089982 conn(0x7f0a280ce880 msgr2=0x7f0a280cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:55.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.242+0000 7f0a2da96640 1 -- 172.21.15.115:0/2354089982 shutdown_connections 2023-12-01T03:11:55.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.242+0000 7f0a2da96640 1 -- 172.21.15.115:0/2354089982 wait complete. 2023-12-01T03:11:55.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.244+0000 7f0a2da96640 1 Processor -- start 2023-12-01T03:11:55.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.244+0000 7f0a2da96640 1 -- start start 2023-12-01T03:11:55.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a2da96640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a2806b350 0x7f0a2815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a2da96640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a280adac0 0x7f0a280c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a2da96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 0x7f0a28158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a2da96640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280c7220 con 0x7f0a2806b350 2023-12-01T03:11:55.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a2da96640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280c74c0 con 0x7f0a280adac0 2023-12-01T03:11:55.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a2da96640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280c77a0 con 0x7f0a280c8ed0 2023-12-01T03:11:55.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a277fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a2806b350 0x7f0a2815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:55.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a277fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a2806b350 0x7f0a2815a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52794/0 (socket says 172.21.15.115:52794) 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a277fe640 1 -- 172.21.15.115:0/1289418996 learned_addr learned my addr 172.21.15.115:0/1289418996 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a26ffd640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a280adac0 0x7f0a280c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a27fff640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 0x7f0a28158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a277fe640 1 -- 172.21.15.115:0/1289418996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 msgr2=0x7f0a28158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a277fe640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 0x7f0a28158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a277fe640 1 -- 172.21.15.115:0/1289418996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a280adac0 msgr2=0x7f0a280c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a277fe640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a280adac0 0x7f0a280c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.245+0000 7f0a277fe640 1 -- 172.21.15.115:0/1289418996 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c009400 con 0x7f0a2806b350 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.246+0000 7f0a26ffd640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a280adac0 0x7f0a280c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.246+0000 7f0a277fe640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a2806b350 0x7f0a2815a9b0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c450 tx=0x7f0a1800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.246+0000 7f0a0ffff640 1 -- 172.21.15.115:0/1289418996 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a180052d0 con 0x7f0a2806b350 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.246+0000 7f0a0ffff640 1 -- 172.21.15.115:0/1289418996 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a18005430 con 0x7f0a2806b350 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.246+0000 7f0a0ffff640 1 -- 172.21.15.115:0/1289418996 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18012870 con 0x7f0a2806b350 2023-12-01T03:11:55.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.246+0000 7f0a2da96640 1 -- 172.21.15.115:0/1289418996 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a280c7a80 con 0x7f0a2806b350 2023-12-01T03:11:55.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.246+0000 7f0a2da96640 1 -- 172.21.15.115:0/1289418996 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a280c8040 con 0x7f0a2806b350 2023-12-01T03:11:55.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.247+0000 7f0a2da96640 1 -- 172.21.15.115:0/1289418996 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2804e890 con 0x7f0a2806b350 2023-12-01T03:11:55.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.248+0000 7f0a0ffff640 1 -- 172.21.15.115:0/1289418996 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f0a180129d0 con 0x7f0a2806b350 2023-12-01T03:11:55.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.250+0000 7f0a0ffff640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0a1406a670 0x7f0a1406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.250+0000 7f0a0ffff640 1 -- 172.21.15.115:0/1289418996 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f0a18088c40 con 0x7f0a2806b350 2023-12-01T03:11:55.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.250+0000 7f0a26ffd640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0a1406a670 0x7f0a1406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:55.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.251+0000 7f0a26ffd640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0a1406a670 0x7f0a1406cb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c036000 tx=0x7f0a1c03c040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:55.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.254+0000 7f0a0ffff640 1 -- 172.21.15.115:0/1289418996 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0a18053b40 con 0x7f0a2806b350 2023-12-01T03:11:55.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.438+0000 7f0a2da96640 1 -- 172.21.15.115:0/1289418996 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a280cf5b0 con 0x7f0a1406a670 2023-12-01T03:11:55.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.482+0000 7f0a0ffff640 1 -- 172.21.15.115:0/1289418996 <== mgr.11121 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526590 (secure 0 0 0) 0x7f0a1c03a420 con 0x7f0a1406a670 2023-12-01T03:11:55.490 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:55.496 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:11:55.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 -- 172.21.15.115:0/1289418996 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0a1406a670 msgr2=0x7f0a1406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0a1406a670 0x7f0a1406cb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c036000 tx=0x7f0a1c03c040 comp rx=0 tx=0).stop 2023-12-01T03:11:55.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 -- 172.21.15.115:0/1289418996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a2806b350 msgr2=0x7f0a2815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a2806b350 0x7f0a2815a9b0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c450 tx=0x7f0a1800c910 comp rx=0 tx=0).stop 2023-12-01T03:11:55.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 -- 172.21.15.115:0/1289418996 shutdown_connections 2023-12-01T03:11:55.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0a1406a670 0x7f0a1406cb30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a280c8ed0 0x7f0a28158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a280adac0 0x7f0a280c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 --2- 172.21.15.115:0/1289418996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0a2806b350 0x7f0a2815a9b0 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 -- 172.21.15.115:0/1289418996 >> 172.21.15.115:0/1289418996 conn(0x7f0a280ce880 msgr2=0x7f0a280ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:55.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 -- 172.21.15.115:0/1289418996 shutdown_connections 2023-12-01T03:11:55.499 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.494+0000 7f0a2da96640 1 -- 172.21.15.115:0/1289418996 wait complete. 2023-12-01T03:11:55.523 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2023-12-01T03:11:53.828239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":670,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":427,"ondisk_log_size":427,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":267,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":419820,"kb_used_data":24620,"kb_used_omap":0,"kb_used_meta":395136,"kb_avail":754554900,"statfs":{"total":773094113280,"available":772664217600,"internally_reserved":0,"allocated":25210880,"data_stored":12935771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":404619264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3349,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.285741"},"pg_stats":[{"pgid":"31.3d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089847+0000","last_change":"2023-12-01T03:11:51.088748+0000","last_active":"2023-12-01T03:11:51.089847+0000","last_peered":"2023-12-01T03:11:51.089847+0000","last_clean":"2023-12-01T03:11:51.089847+0000","last_became_active":"2023-12-01T03:11:51.087164+0000","last_became_peered":"2023-12-01T03:11:51.087164+0000","last_unstale":"2023-12-01T03:11:51.089847+0000","last_undegraded":"2023-12-01T03:11:51.089847+0000","last_fullsized":"2023-12-01T03:11:51.089847+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:38:38.426875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582373+0000","last_change":"2023-12-01T03:11:32.212603+0000","last_active":"2023-12-01T03:11:50.582373+0000","last_peered":"2023-12-01T03:11:50.582373+0000","last_clean":"2023-12-01T03:11:50.582373+0000","last_became_active":"2023-12-01T03:11:32.212287+0000","last_became_peered":"2023-12-01T03:11:32.212287+0000","last_unstale":"2023-12-01T03:11:50.582373+0000","last_undegraded":"2023-12-01T03:11:50.582373+0000","last_fullsized":"2023-12-01T03:11:50.582373+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.087638+0000","last_change":"2023-12-01T03:11:51.073555+0000","last_active":"2023-12-01T03:11:51.087638+0000","last_peered":"2023-12-01T03:11:51.087638+0000","last_clean":"2023-12-01T03:11:51.087638+0000","last_became_active":"2023-12-01T03:11:51.073247+0000","last_became_peered":"2023-12-01T03:11:51.073247+0000","last_unstale":"2023-12-01T03:11:51.087638+0000","last_undegraded":"2023-12-01T03:11:51.087638+0000","last_fullsized":"2023-12-01T03:11:51.087638+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:10.005791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569672+0000","last_change":"2023-12-01T03:11:32.217098+0000","last_active":"2023-12-01T03:11:50.569672+0000","last_peered":"2023-12-01T03:11:50.569672+0000","last_clean":"2023-12-01T03:11:50.569672+0000","last_became_active":"2023-12-01T03:11:32.216867+0000","last_became_peered":"2023-12-01T03:11:32.216867+0000","last_unstale":"2023-12-01T03:11:50.569672+0000","last_undegraded":"2023-12-01T03:11:50.569672+0000","last_fullsized":"2023-12-01T03:11:50.569672+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:10.849908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.083721+0000","last_change":"2023-12-01T03:11:51.081937+0000","last_active":"2023-12-01T03:11:51.083721+0000","last_peered":"2023-12-01T03:11:51.083721+0000","last_clean":"2023-12-01T03:11:51.083721+0000","last_became_active":"2023-12-01T03:11:51.081497+0000","last_became_peered":"2023-12-01T03:11:51.081497+0000","last_unstale":"2023-12-01T03:11:51.083721+0000","last_undegraded":"2023-12-01T03:11:51.083721+0000","last_fullsized":"2023-12-01T03:11:51.083721+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:40:11.382428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569344+0000","last_change":"2023-12-01T03:11:32.208423+0000","last_active":"2023-12-01T03:11:50.569344+0000","last_peered":"2023-12-01T03:11:50.569344+0000","last_clean":"2023-12-01T03:11:50.569344+0000","last_became_active":"2023-12-01T03:11:32.208113+0000","last_became_peered":"2023-12-01T03:11:32.208113+0000","last_unstale":"2023-12-01T03:11:50.569344+0000","last_undegraded":"2023-12-01T03:11:50.569344+0000","last_fullsized":"2023-12-01T03:11:50.569344+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:38.174827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.072305+0000","last_change":"2023-12-01T03:11:51.066754+0000","last_active":"2023-12-01T03:11:51.072305+0000","last_peered":"2023-12-01T03:11:51.072305+0000","last_clean":"2023-12-01T03:11:51.072305+0000","last_became_active":"2023-12-01T03:11:51.065359+0000","last_became_peered":"2023-12-01T03:11:51.065359+0000","last_unstale":"2023-12-01T03:11:51.072305+0000","last_undegraded":"2023-12-01T03:11:51.072305+0000","last_fullsized":"2023-12-01T03:11:51.072305+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:38.991634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.584143+0000","last_change":"2023-12-01T03:11:32.207840+0000","last_active":"2023-12-01T03:11:50.584143+0000","last_peered":"2023-12-01T03:11:50.584143+0000","last_clean":"2023-12-01T03:11:50.584143+0000","last_became_active":"2023-12-01T03:11:32.207048+0000","last_became_peered":"2023-12-01T03:11:32.207048+0000","last_unstale":"2023-12-01T03:11:50.584143+0000","last_undegraded":"2023-12-01T03:11:50.584143+0000","last_fullsized":"2023-12-01T03:11:50.584143+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:43.961392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.081151+0000","last_change":"2023-12-01T03:11:51.080141+0000","last_active":"2023-12-01T03:11:51.081151+0000","last_peered":"2023-12-01T03:11:51.081151+0000","last_clean":"2023-12-01T03:11:51.081151+0000","last_became_active":"2023-12-01T03:11:51.079871+0000","last_became_peered":"2023-12-01T03:11:51.079871+0000","last_unstale":"2023-12-01T03:11:51.081151+0000","last_undegraded":"2023-12-01T03:11:51.081151+0000","last_fullsized":"2023-12-01T03:11:51.081151+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:18.202586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583207+0000","last_change":"2023-12-01T03:11:32.206960+0000","last_active":"2023-12-01T03:11:50.583207+0000","last_peered":"2023-12-01T03:11:50.583207+0000","last_clean":"2023-12-01T03:11:50.583207+0000","last_became_active":"2023-12-01T03:11:32.203345+0000","last_became_peered":"2023-12-01T03:11:32.203345+0000","last_unstale":"2023-12-01T03:11:50.583207+0000","last_undegraded":"2023-12-01T03:11:50.583207+0000","last_fullsized":"2023-12-01T03:11:50.583207+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070238+0000","last_change":"2023-12-01T03:11:51.068646+0000","last_active":"2023-12-01T03:11:51.070238+0000","last_peered":"2023-12-01T03:11:51.070238+0000","last_clean":"2023-12-01T03:11:51.070238+0000","last_became_active":"2023-12-01T03:11:51.068386+0000","last_became_peered":"2023-12-01T03:11:51.068386+0000","last_unstale":"2023-12-01T03:11:51.070238+0000","last_undegraded":"2023-12-01T03:11:51.070238+0000","last_fullsized":"2023-12-01T03:11:51.070238+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:29.351674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.586169+0000","last_change":"2023-12-01T03:11:32.205496+0000","last_active":"2023-12-01T03:11:50.586169+0000","last_peered":"2023-12-01T03:11:50.586169+0000","last_clean":"2023-12-01T03:11:50.586169+0000","last_became_active":"2023-12-01T03:11:32.205074+0000","last_became_peered":"2023-12-01T03:11:32.205074+0000","last_unstale":"2023-12-01T03:11:50.586169+0000","last_undegraded":"2023-12-01T03:11:50.586169+0000","last_fullsized":"2023-12-01T03:11:50.586169+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070482+0000","last_change":"2023-12-01T03:11:51.067697+0000","last_active":"2023-12-01T03:11:51.070482+0000","last_peered":"2023-12-01T03:11:51.070482+0000","last_clean":"2023-12-01T03:11:51.070482+0000","last_became_active":"2023-12-01T03:11:51.065158+0000","last_became_peered":"2023-12-01T03:11:51.065158+0000","last_unstale":"2023-12-01T03:11:51.070482+0000","last_undegraded":"2023-12-01T03:11:51.070482+0000","last_fullsized":"2023-12-01T03:11:51.070482+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:13.658416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568771+0000","last_change":"2023-12-01T03:11:32.202465+0000","last_active":"2023-12-01T03:11:50.568771+0000","last_peered":"2023-12-01T03:11:50.568771+0000","last_clean":"2023-12-01T03:11:50.568771+0000","last_became_active":"2023-12-01T03:11:32.200165+0000","last_became_peered":"2023-12-01T03:11:32.200165+0000","last_unstale":"2023-12-01T03:11:50.568771+0000","last_undegraded":"2023-12-01T03:11:50.568771+0000","last_fullsized":"2023-12-01T03:11:50.568771+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:19.277681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088652+0000","last_change":"2023-12-01T03:11:51.087856+0000","last_active":"2023-12-01T03:11:51.088652+0000","last_peered":"2023-12-01T03:11:51.088652+0000","last_clean":"2023-12-01T03:11:51.088652+0000","last_became_active":"2023-12-01T03:11:51.087320+0000","last_became_peered":"2023-12-01T03:11:51.087320+0000","last_unstale":"2023-12-01T03:11:51.088652+0000","last_undegraded":"2023-12-01T03:11:51.088652+0000","last_fullsized":"2023-12-01T03:11:51.088652+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:29.292487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.563843+0000","last_change":"2023-12-01T03:11:32.217253+0000","last_active":"2023-12-01T03:11:50.563843+0000","last_peered":"2023-12-01T03:11:50.563843+0000","last_clean":"2023-12-01T03:11:50.563843+0000","last_became_active":"2023-12-01T03:11:32.216425+0000","last_became_peered":"2023-12-01T03:11:32.216425+0000","last_unstale":"2023-12-01T03:11:50.563843+0000","last_undegraded":"2023-12-01T03:11:50.563843+0000","last_fullsized":"2023-12-01T03:11:50.563843+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:51.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089032+0000","last_change":"2023-12-01T03:11:51.083849+0000","last_active":"2023-12-01T03:11:51.089032+0000","last_peered":"2023-12-01T03:11:51.089032+0000","last_clean":"2023-12-01T03:11:51.089032+0000","last_became_active":"2023-12-01T03:11:51.079649+0000","last_became_peered":"2023-12-01T03:11:51.079649+0000","last_unstale":"2023-12-01T03:11:51.089032+0000","last_undegraded":"2023-12-01T03:11:51.089032+0000","last_fullsized":"2023-12-01T03:11:51.089032+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:24.492502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.593355+0000","last_change":"2023-12-01T03:11:32.211881+0000","last_active":"2023-12-01T03:11:50.593355+0000","last_peered":"2023-12-01T03:11:50.593355+0000","last_clean":"2023-12-01T03:11:50.593355+0000","last_became_active":"2023-12-01T03:11:32.211602+0000","last_became_peered":"2023-12-01T03:11:32.211602+0000","last_unstale":"2023-12-01T03:11:50.593355+0000","last_undegraded":"2023-12-01T03:11:50.593355+0000","last_fullsized":"2023-12-01T03:11:50.593355+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.081093+0000","last_change":"2023-12-01T03:11:51.080082+0000","last_active":"2023-12-01T03:11:51.081093+0000","last_peered":"2023-12-01T03:11:51.081093+0000","last_clean":"2023-12-01T03:11:51.081093+0000","last_became_active":"2023-12-01T03:11:51.079868+0000","last_became_peered":"2023-12-01T03:11:51.079868+0000","last_unstale":"2023-12-01T03:11:51.081093+0000","last_undegraded":"2023-12-01T03:11:51.081093+0000","last_fullsized":"2023-12-01T03:11:51.081093+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:45:06.821895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.585831+0000","last_change":"2023-12-01T03:11:32.200099+0000","last_active":"2023-12-01T03:11:50.585831+0000","last_peered":"2023-12-01T03:11:50.585831+0000","last_clean":"2023-12-01T03:11:50.585831+0000","last_became_active":"2023-12-01T03:11:32.199703+0000","last_became_peered":"2023-12-01T03:11:32.199703+0000","last_unstale":"2023-12-01T03:11:50.585831+0000","last_undegraded":"2023-12-01T03:11:50.585831+0000","last_fullsized":"2023-12-01T03:11:50.585831+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:44:42.356924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.071966+0000","last_change":"2023-12-01T03:11:51.070269+0000","last_active":"2023-12-01T03:11:51.071966+0000","last_peered":"2023-12-01T03:11:51.071966+0000","last_clean":"2023-12-01T03:11:51.071966+0000","last_became_active":"2023-12-01T03:11:51.069876+0000","last_became_peered":"2023-12-01T03:11:51.069876+0000","last_unstale":"2023-12-01T03:11:51.071966+0000","last_undegraded":"2023-12-01T03:11:51.071966+0000","last_fullsized":"2023-12-01T03:11:51.071966+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:04.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569551+0000","last_change":"2023-12-01T03:11:32.211517+0000","last_active":"2023-12-01T03:11:50.569551+0000","last_peered":"2023-12-01T03:11:50.569551+0000","last_clean":"2023-12-01T03:11:50.569551+0000","last_became_active":"2023-12-01T03:11:32.211273+0000","last_became_peered":"2023-12-01T03:11:32.211273+0000","last_unstale":"2023-12-01T03:11:50.569551+0000","last_undegraded":"2023-12-01T03:11:50.569551+0000","last_fullsized":"2023-12-01T03:11:50.569551+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:35.627224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.081422+0000","last_change":"2023-12-01T03:11:51.080839+0000","last_active":"2023-12-01T03:11:51.081422+0000","last_peered":"2023-12-01T03:11:51.081422+0000","last_clean":"2023-12-01T03:11:51.081422+0000","last_became_active":"2023-12-01T03:11:51.080590+0000","last_became_peered":"2023-12-01T03:11:51.080590+0000","last_unstale":"2023-12-01T03:11:51.081422+0000","last_undegraded":"2023-12-01T03:11:51.081422+0000","last_fullsized":"2023-12-01T03:11:51.081422+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:29.292487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.579811+0000","last_change":"2023-12-01T03:11:32.202825+0000","last_active":"2023-12-01T03:11:50.579811+0000","last_peered":"2023-12-01T03:11:50.579811+0000","last_clean":"2023-12-01T03:11:50.579811+0000","last_became_active":"2023-12-01T03:11:32.202452+0000","last_became_peered":"2023-12-01T03:11:32.202452+0000","last_unstale":"2023-12-01T03:11:50.579811+0000","last_undegraded":"2023-12-01T03:11:50.579811+0000","last_fullsized":"2023-12-01T03:11:50.579811+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.067508+0000","last_change":"2023-12-01T03:11:51.064149+0000","last_active":"2023-12-01T03:11:51.067508+0000","last_peered":"2023-12-01T03:11:51.067508+0000","last_clean":"2023-12-01T03:11:51.067508+0000","last_became_active":"2023-12-01T03:11:51.063847+0000","last_became_peered":"2023-12-01T03:11:51.063847+0000","last_unstale":"2023-12-01T03:11:51.067508+0000","last_undegraded":"2023-12-01T03:11:51.067508+0000","last_fullsized":"2023-12-01T03:11:51.067508+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:33:54.697716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581309+0000","last_change":"2023-12-01T03:11:32.224888+0000","last_active":"2023-12-01T03:11:50.581309+0000","last_peered":"2023-12-01T03:11:50.581309+0000","last_clean":"2023-12-01T03:11:50.581309+0000","last_became_active":"2023-12-01T03:11:32.224585+0000","last_became_peered":"2023-12-01T03:11:32.224585+0000","last_unstale":"2023-12-01T03:11:50.581309+0000","last_undegraded":"2023-12-01T03:11:50.581309+0000","last_fullsized":"2023-12-01T03:11:50.581309+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:04.079396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068457+0000","last_change":"2023-12-01T03:11:51.066331+0000","last_active":"2023-12-01T03:11:51.068457+0000","last_peered":"2023-12-01T03:11:51.068457+0000","last_clean":"2023-12-01T03:11:51.068457+0000","last_became_active":"2023-12-01T03:11:51.065335+0000","last_became_peered":"2023-12-01T03:11:51.065335+0000","last_unstale":"2023-12-01T03:11:51.068457+0000","last_undegraded":"2023-12-01T03:11:51.068457+0000","last_fullsized":"2023-12-01T03:11:51.068457+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:14.769992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.570401+0000","last_change":"2023-12-01T03:11:32.208938+0000","last_active":"2023-12-01T03:11:50.570401+0000","last_peered":"2023-12-01T03:11:50.570401+0000","last_clean":"2023-12-01T03:11:50.570401+0000","last_became_active":"2023-12-01T03:11:32.208695+0000","last_became_peered":"2023-12-01T03:11:32.208695+0000","last_unstale":"2023-12-01T03:11:50.570401+0000","last_undegraded":"2023-12-01T03:11:50.570401+0000","last_fullsized":"2023-12-01T03:11:50.570401+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:21.336618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.071601+0000","last_change":"2023-12-01T03:11:51.069976+0000","last_active":"2023-12-01T03:11:51.071601+0000","last_peered":"2023-12-01T03:11:51.071601+0000","last_clean":"2023-12-01T03:11:51.071601+0000","last_became_active":"2023-12-01T03:11:51.069153+0000","last_became_peered":"2023-12-01T03:11:51.069153+0000","last_unstale":"2023-12-01T03:11:51.071601+0000","last_undegraded":"2023-12-01T03:11:51.071601+0000","last_fullsized":"2023-12-01T03:11:51.071601+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:54:02.088965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583726+0000","last_change":"2023-12-01T03:11:32.214557+0000","last_active":"2023-12-01T03:11:50.583726+0000","last_peered":"2023-12-01T03:11:50.583726+0000","last_clean":"2023-12-01T03:11:50.583726+0000","last_became_active":"2023-12-01T03:11:32.214252+0000","last_became_peered":"2023-12-01T03:11:32.214252+0000","last_unstale":"2023-12-01T03:11:50.583726+0000","last_undegraded":"2023-12-01T03:11:50.583726+0000","last_fullsized":"2023-12-01T03:11:50.583726+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088066+0000","last_change":"2023-12-01T03:11:51.076162+0000","last_active":"2023-12-01T03:11:51.088066+0000","last_peered":"2023-12-01T03:11:51.088066+0000","last_clean":"2023-12-01T03:11:51.088066+0000","last_became_active":"2023-12-01T03:11:51.075899+0000","last_became_peered":"2023-12-01T03:11:51.075899+0000","last_unstale":"2023-12-01T03:11:51.088066+0000","last_undegraded":"2023-12-01T03:11:51.088066+0000","last_fullsized":"2023-12-01T03:11:51.088066+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:26:32.163371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.590521+0000","last_change":"2023-12-01T03:11:32.212018+0000","last_active":"2023-12-01T03:11:50.590521+0000","last_peered":"2023-12-01T03:11:50.590521+0000","last_clean":"2023-12-01T03:11:50.590521+0000","last_became_active":"2023-12-01T03:11:32.211742+0000","last_became_peered":"2023-12-01T03:11:32.211742+0000","last_unstale":"2023-12-01T03:11:50.590521+0000","last_undegraded":"2023-12-01T03:11:50.590521+0000","last_fullsized":"2023-12-01T03:11:50.590521+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070608+0000","last_change":"2023-12-01T03:11:51.069517+0000","last_active":"2023-12-01T03:11:51.070608+0000","last_peered":"2023-12-01T03:11:51.070608+0000","last_clean":"2023-12-01T03:11:51.070608+0000","last_became_active":"2023-12-01T03:11:51.068565+0000","last_became_peered":"2023-12-01T03:11:51.068565+0000","last_unstale":"2023-12-01T03:11:51.070608+0000","last_undegraded":"2023-12-01T03:11:51.070608+0000","last_fullsized":"2023-12-01T03:11:51.070608+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:33.446505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582658+0000","last_change":"2023-12-01T03:11:32.219580+0000","last_active":"2023-12-01T03:11:50.582658+0000","last_peered":"2023-12-01T03:11:50.582658+0000","last_clean":"2023-12-01T03:11:50.582658+0000","last_became_active":"2023-12-01T03:11:32.218935+0000","last_became_peered":"2023-12-01T03:11:32.218935+0000","last_unstale":"2023-12-01T03:11:50.582658+0000","last_undegraded":"2023-12-01T03:11:50.582658+0000","last_fullsized":"2023-12-01T03:11:50.582658+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.066372+0000","last_change":"2023-12-01T03:11:51.065776+0000","last_active":"2023-12-01T03:11:51.066372+0000","last_peered":"2023-12-01T03:11:51.066372+0000","last_clean":"2023-12-01T03:11:51.066372+0000","last_became_active":"2023-12-01T03:11:51.065512+0000","last_became_peered":"2023-12-01T03:11:51.065512+0000","last_unstale":"2023-12-01T03:11:51.066372+0000","last_undegraded":"2023-12-01T03:11:51.066372+0000","last_fullsized":"2023-12-01T03:11:51.066372+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:02:55.175453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581169+0000","last_change":"2023-12-01T03:11:32.222587+0000","last_active":"2023-12-01T03:11:50.581169+0000","last_peered":"2023-12-01T03:11:50.581169+0000","last_clean":"2023-12-01T03:11:50.581169+0000","last_became_active":"2023-12-01T03:11:32.222267+0000","last_became_peered":"2023-12-01T03:11:32.222267+0000","last_unstale":"2023-12-01T03:11:50.581169+0000","last_undegraded":"2023-12-01T03:11:50.581169+0000","last_fullsized":"2023-12-01T03:11:50.581169+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:42.720564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.067007+0000","last_change":"2023-12-01T03:11:51.066093+0000","last_active":"2023-12-01T03:11:51.067007+0000","last_peered":"2023-12-01T03:11:51.067007+0000","last_clean":"2023-12-01T03:11:51.067007+0000","last_became_active":"2023-12-01T03:11:51.065673+0000","last_became_peered":"2023-12-01T03:11:51.065673+0000","last_unstale":"2023-12-01T03:11:51.067007+0000","last_undegraded":"2023-12-01T03:11:51.067007+0000","last_fullsized":"2023-12-01T03:11:51.067007+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:31.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.587613+0000","last_change":"2023-12-01T03:11:32.204026+0000","last_active":"2023-12-01T03:11:50.587613+0000","last_peered":"2023-12-01T03:11:50.587613+0000","last_clean":"2023-12-01T03:11:50.587613+0000","last_became_active":"2023-12-01T03:11:32.200832+0000","last_became_peered":"2023-12-01T03:11:32.200832+0000","last_unstale":"2023-12-01T03:11:50.587613+0000","last_undegraded":"2023-12-01T03:11:50.587613+0000","last_fullsized":"2023-12-01T03:11:50.587613+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078852+0000","last_change":"2023-12-01T03:11:51.068191+0000","last_active":"2023-12-01T03:11:51.078852+0000","last_peered":"2023-12-01T03:11:51.078852+0000","last_clean":"2023-12-01T03:11:51.078852+0000","last_became_active":"2023-12-01T03:11:51.067752+0000","last_became_peered":"2023-12-01T03:11:51.067752+0000","last_unstale":"2023-12-01T03:11:51.078852+0000","last_undegraded":"2023-12-01T03:11:51.078852+0000","last_fullsized":"2023-12-01T03:11:51.078852+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:54:02.088965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.588717+0000","last_change":"2023-12-01T03:11:32.207642+0000","last_active":"2023-12-01T03:11:50.588717+0000","last_peered":"2023-12-01T03:11:50.588717+0000","last_clean":"2023-12-01T03:11:50.588717+0000","last_became_active":"2023-12-01T03:11:32.207344+0000","last_became_peered":"2023-12-01T03:11:32.207344+0000","last_unstale":"2023-12-01T03:11:50.588717+0000","last_undegraded":"2023-12-01T03:11:50.588717+0000","last_fullsized":"2023-12-01T03:11:50.588717+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:10.441679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.066428+0000","last_change":"2023-12-01T03:11:51.062269+0000","last_active":"2023-12-01T03:11:51.066428+0000","last_peered":"2023-12-01T03:11:51.066428+0000","last_clean":"2023-12-01T03:11:51.066428+0000","last_became_active":"2023-12-01T03:11:51.061924+0000","last_became_peered":"2023-12-01T03:11:51.061924+0000","last_unstale":"2023-12-01T03:11:51.066428+0000","last_undegraded":"2023-12-01T03:11:51.066428+0000","last_fullsized":"2023-12-01T03:11:51.066428+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:01.630559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.580945+0000","last_change":"2023-12-01T03:11:32.220696+0000","last_active":"2023-12-01T03:11:50.580945+0000","last_peered":"2023-12-01T03:11:50.580945+0000","last_clean":"2023-12-01T03:11:50.580945+0000","last_became_active":"2023-12-01T03:11:32.220353+0000","last_became_peered":"2023-12-01T03:11:32.220353+0000","last_unstale":"2023-12-01T03:11:50.580945+0000","last_undegraded":"2023-12-01T03:11:50.580945+0000","last_fullsized":"2023-12-01T03:11:50.580945+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:53:43.178969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088991+0000","last_change":"2023-12-01T03:11:51.080854+0000","last_active":"2023-12-01T03:11:51.088991+0000","last_peered":"2023-12-01T03:11:51.088991+0000","last_clean":"2023-12-01T03:11:51.088991+0000","last_became_active":"2023-12-01T03:11:51.080581+0000","last_became_peered":"2023-12-01T03:11:51.080581+0000","last_unstale":"2023-12-01T03:11:51.088991+0000","last_undegraded":"2023-12-01T03:11:51.088991+0000","last_fullsized":"2023-12-01T03:11:51.088991+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:57:38.081054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.586520+0000","last_change":"2023-12-01T03:11:32.207709+0000","last_active":"2023-12-01T03:11:50.586520+0000","last_peered":"2023-12-01T03:11:50.586520+0000","last_clean":"2023-12-01T03:11:50.586520+0000","last_became_active":"2023-12-01T03:11:32.205510+0000","last_became_peered":"2023-12-01T03:11:32.205510+0000","last_unstale":"2023-12-01T03:11:50.586520+0000","last_undegraded":"2023-12-01T03:11:50.586520+0000","last_fullsized":"2023-12-01T03:11:50.586520+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070463+0000","last_change":"2023-12-01T03:11:51.067527+0000","last_active":"2023-12-01T03:11:51.070463+0000","last_peered":"2023-12-01T03:11:51.070463+0000","last_clean":"2023-12-01T03:11:51.070463+0000","last_became_active":"2023-12-01T03:11:51.064895+0000","last_became_peered":"2023-12-01T03:11:51.064895+0000","last_unstale":"2023-12-01T03:11:51.070463+0000","last_undegraded":"2023-12-01T03:11:51.070463+0000","last_fullsized":"2023-12-01T03:11:51.070463+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:54:02.088965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.578605+0000","last_change":"2023-12-01T03:11:32.210643+0000","last_active":"2023-12-01T03:11:50.578605+0000","last_peered":"2023-12-01T03:11:50.578605+0000","last_clean":"2023-12-01T03:11:50.578605+0000","last_became_active":"2023-12-01T03:11:32.210367+0000","last_became_peered":"2023-12-01T03:11:32.210367+0000","last_unstale":"2023-12-01T03:11:50.578605+0000","last_undegraded":"2023-12-01T03:11:50.578605+0000","last_fullsized":"2023-12-01T03:11:50.578605+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:00.674458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.085719+0000","last_change":"2023-12-01T03:11:51.085229+0000","last_active":"2023-12-01T03:11:51.085719+0000","last_peered":"2023-12-01T03:11:51.085719+0000","last_clean":"2023-12-01T03:11:51.085719+0000","last_became_active":"2023-12-01T03:11:51.084930+0000","last_became_peered":"2023-12-01T03:11:51.084930+0000","last_unstale":"2023-12-01T03:11:51.085719+0000","last_undegraded":"2023-12-01T03:11:51.085719+0000","last_fullsized":"2023-12-01T03:11:51.085719+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:05:22.145638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582376+0000","last_change":"2023-12-01T03:11:32.209242+0000","last_active":"2023-12-01T03:11:50.582376+0000","last_peered":"2023-12-01T03:11:50.582376+0000","last_clean":"2023-12-01T03:11:50.582376+0000","last_became_active":"2023-12-01T03:11:32.208985+0000","last_became_peered":"2023-12-01T03:11:32.208985+0000","last_unstale":"2023-12-01T03:11:50.582376+0000","last_undegraded":"2023-12-01T03:11:50.582376+0000","last_fullsized":"2023-12-01T03:11:50.582376+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:29.661974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078326+0000","last_change":"2023-12-01T03:11:51.071579+0000","last_active":"2023-12-01T03:11:51.078326+0000","last_peered":"2023-12-01T03:11:51.078326+0000","last_clean":"2023-12-01T03:11:51.078326+0000","last_became_active":"2023-12-01T03:11:51.071042+0000","last_became_peered":"2023-12-01T03:11:51.071042+0000","last_unstale":"2023-12-01T03:11:51.078326+0000","last_undegraded":"2023-12-01T03:11:51.078326+0000","last_fullsized":"2023-12-01T03:11:51.078326+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:54:09.453554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569921+0000","last_change":"2023-12-01T03:11:32.220828+0000","last_active":"2023-12-01T03:11:50.569921+0000","last_peered":"2023-12-01T03:11:50.569921+0000","last_clean":"2023-12-01T03:11:50.569921+0000","last_became_active":"2023-12-01T03:11:32.219607+0000","last_became_peered":"2023-12-01T03:11:32.219607+0000","last_unstale":"2023-12-01T03:11:50.569921+0000","last_undegraded":"2023-12-01T03:11:50.569921+0000","last_fullsized":"2023-12-01T03:11:50.569921+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:09.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.087817+0000","last_change":"2023-12-01T03:11:51.074146+0000","last_active":"2023-12-01T03:11:51.087817+0000","last_peered":"2023-12-01T03:11:51.087817+0000","last_clean":"2023-12-01T03:11:51.087817+0000","last_became_active":"2023-12-01T03:11:51.073887+0000","last_became_peered":"2023-12-01T03:11:51.073887+0000","last_unstale":"2023-12-01T03:11:51.087817+0000","last_undegraded":"2023-12-01T03:11:51.087817+0000","last_fullsized":"2023-12-01T03:11:51.087817+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:49.251059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.591470+0000","last_change":"2023-12-01T03:11:32.198609+0000","last_active":"2023-12-01T03:11:50.591470+0000","last_peered":"2023-12-01T03:11:50.591470+0000","last_clean":"2023-12-01T03:11:50.591470+0000","last_became_active":"2023-12-01T03:11:32.197869+0000","last_became_peered":"2023-12-01T03:11:32.197869+0000","last_unstale":"2023-12-01T03:11:50.591470+0000","last_undegraded":"2023-12-01T03:11:50.591470+0000","last_fullsized":"2023-12-01T03:11:50.591470+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:35.975600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088036+0000","last_change":"2023-12-01T03:11:51.075313+0000","last_active":"2023-12-01T03:11:51.088036+0000","last_peered":"2023-12-01T03:11:51.088036+0000","last_clean":"2023-12-01T03:11:51.088036+0000","last_became_active":"2023-12-01T03:11:51.072486+0000","last_became_peered":"2023-12-01T03:11:51.072486+0000","last_unstale":"2023-12-01T03:11:51.088036+0000","last_undegraded":"2023-12-01T03:11:51.088036+0000","last_fullsized":"2023-12-01T03:11:51.088036+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:02:55.175453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569456+0000","last_change":"2023-12-01T03:11:32.212954+0000","last_active":"2023-12-01T03:11:50.569456+0000","last_peered":"2023-12-01T03:11:50.569456+0000","last_clean":"2023-12-01T03:11:50.569456+0000","last_became_active":"2023-12-01T03:11:32.212678+0000","last_became_peered":"2023-12-01T03:11:32.212678+0000","last_unstale":"2023-12-01T03:11:50.569456+0000","last_undegraded":"2023-12-01T03:11:50.569456+0000","last_fullsized":"2023-12-01T03:11:50.569456+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:28.852429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068257+0000","last_change":"2023-12-01T03:11:51.067112+0000","last_active":"2023-12-01T03:11:51.068257+0000","last_peered":"2023-12-01T03:11:51.068257+0000","last_clean":"2023-12-01T03:11:51.068257+0000","last_became_active":"2023-12-01T03:11:51.066722+0000","last_became_peered":"2023-12-01T03:11:51.066722+0000","last_unstale":"2023-12-01T03:11:51.068257+0000","last_undegraded":"2023-12-01T03:11:51.068257+0000","last_fullsized":"2023-12-01T03:11:51.068257+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:07.114272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.567075+0000","last_change":"2023-12-01T03:11:32.215698+0000","last_active":"2023-12-01T03:11:50.567075+0000","last_peered":"2023-12-01T03:11:50.567075+0000","last_clean":"2023-12-01T03:11:50.567075+0000","last_became_active":"2023-12-01T03:11:32.214531+0000","last_became_peered":"2023-12-01T03:11:32.214531+0000","last_unstale":"2023-12-01T03:11:50.567075+0000","last_undegraded":"2023-12-01T03:11:50.567075+0000","last_fullsized":"2023-12-01T03:11:50.567075+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:14.716448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088453+0000","last_change":"2023-12-01T03:11:51.078917+0000","last_active":"2023-12-01T03:11:51.088453+0000","last_peered":"2023-12-01T03:11:51.088453+0000","last_clean":"2023-12-01T03:11:51.088453+0000","last_became_active":"2023-12-01T03:11:51.078182+0000","last_became_peered":"2023-12-01T03:11:51.078182+0000","last_unstale":"2023-12-01T03:11:51.088453+0000","last_undegraded":"2023-12-01T03:11:51.088453+0000","last_fullsized":"2023-12-01T03:11:51.088453+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:38.991634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.593671+0000","last_change":"2023-12-01T03:11:32.204327+0000","last_active":"2023-12-01T03:11:50.593671+0000","last_peered":"2023-12-01T03:11:50.593671+0000","last_clean":"2023-12-01T03:11:50.593671+0000","last_became_active":"2023-12-01T03:11:32.203765+0000","last_became_peered":"2023-12-01T03:11:32.203765+0000","last_unstale":"2023-12-01T03:11:50.593671+0000","last_undegraded":"2023-12-01T03:11:50.593671+0000","last_fullsized":"2023-12-01T03:11:50.593671+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:28:34.324085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.086910+0000","last_change":"2023-12-01T03:11:51.085732+0000","last_active":"2023-12-01T03:11:51.086910+0000","last_peered":"2023-12-01T03:11:51.086910+0000","last_clean":"2023-12-01T03:11:51.086910+0000","last_became_active":"2023-12-01T03:11:51.083582+0000","last_became_peered":"2023-12-01T03:11:51.083582+0000","last_unstale":"2023-12-01T03:11:51.086910+0000","last_undegraded":"2023-12-01T03:11:51.086910+0000","last_fullsized":"2023-12-01T03:11:51.086910+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:26:32.163371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.577680+0000","last_change":"2023-12-01T03:11:32.212023+0000","last_active":"2023-12-01T03:11:50.577680+0000","last_peered":"2023-12-01T03:11:50.577680+0000","last_clean":"2023-12-01T03:11:50.577680+0000","last_became_active":"2023-12-01T03:11:32.210975+0000","last_became_peered":"2023-12-01T03:11:32.210975+0000","last_unstale":"2023-12-01T03:11:50.577680+0000","last_undegraded":"2023-12-01T03:11:50.577680+0000","last_fullsized":"2023-12-01T03:11:50.577680+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089326+0000","last_change":"2023-12-01T03:11:51.084960+0000","last_active":"2023-12-01T03:11:51.089326+0000","last_peered":"2023-12-01T03:11:51.089326+0000","last_clean":"2023-12-01T03:11:51.089326+0000","last_became_active":"2023-12-01T03:11:51.084228+0000","last_became_peered":"2023-12-01T03:11:51.084228+0000","last_unstale":"2023-12-01T03:11:51.089326+0000","last_undegraded":"2023-12-01T03:11:51.089326+0000","last_fullsized":"2023-12-01T03:11:51.089326+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:09:43.235099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.592399+0000","last_change":"2023-12-01T03:11:32.213074+0000","last_active":"2023-12-01T03:11:50.592399+0000","last_peered":"2023-12-01T03:11:50.592399+0000","last_clean":"2023-12-01T03:11:50.592399+0000","last_became_active":"2023-12-01T03:11:32.212775+0000","last_became_peered":"2023-12-01T03:11:32.212775+0000","last_unstale":"2023-12-01T03:11:50.592399+0000","last_undegraded":"2023-12-01T03:11:50.592399+0000","last_fullsized":"2023-12-01T03:11:50.592399+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:36.571608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070822+0000","last_change":"2023-12-01T03:11:51.069719+0000","last_active":"2023-12-01T03:11:51.070822+0000","last_peered":"2023-12-01T03:11:51.070822+0000","last_clean":"2023-12-01T03:11:51.070822+0000","last_became_active":"2023-12-01T03:11:51.069436+0000","last_became_peered":"2023-12-01T03:11:51.069436+0000","last_unstale":"2023-12-01T03:11:51.070822+0000","last_undegraded":"2023-12-01T03:11:51.070822+0000","last_fullsized":"2023-12-01T03:11:51.070822+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:07.114272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.564062+0000","last_change":"2023-12-01T03:11:32.225479+0000","last_active":"2023-12-01T03:11:50.564062+0000","last_peered":"2023-12-01T03:11:50.564062+0000","last_clean":"2023-12-01T03:11:50.564062+0000","last_became_active":"2023-12-01T03:11:32.225154+0000","last_became_peered":"2023-12-01T03:11:32.225154+0000","last_unstale":"2023-12-01T03:11:50.564062+0000","last_undegraded":"2023-12-01T03:11:50.564062+0000","last_fullsized":"2023-12-01T03:11:50.564062+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:14.760284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088235+0000","last_change":"2023-12-01T03:11:51.076925+0000","last_active":"2023-12-01T03:11:51.088235+0000","last_peered":"2023-12-01T03:11:51.088235+0000","last_clean":"2023-12-01T03:11:51.088235+0000","last_became_active":"2023-12-01T03:11:51.076518+0000","last_became_peered":"2023-12-01T03:11:51.076518+0000","last_unstale":"2023-12-01T03:11:51.088235+0000","last_undegraded":"2023-12-01T03:11:51.088235+0000","last_fullsized":"2023-12-01T03:11:51.088235+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:14.769992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1d","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089447+0000","last_change":"2023-12-01T03:11:50.599295+0000","last_active":"2023-12-01T03:11:51.089447+0000","last_peered":"2023-12-01T03:11:51.089447+0000","last_clean":"2023-12-01T03:11:51.089447+0000","last_became_active":"2023-12-01T03:11:50.598962+0000","last_became_peered":"2023-12-01T03:11:50.598962+0000","last_unstale":"2023-12-01T03:11:51.089447+0000","last_undegraded":"2023-12-01T03:11:51.089447+0000","last_fullsized":"2023-12-01T03:11:51.089447+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:11:14.566384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569753+0000","last_change":"2023-12-01T03:11:32.214972+0000","last_active":"2023-12-01T03:11:50.569753+0000","last_peered":"2023-12-01T03:11:50.569753+0000","last_clean":"2023-12-01T03:11:50.569753+0000","last_became_active":"2023-12-01T03:11:32.214729+0000","last_became_peered":"2023-12-01T03:11:32.214729+0000","last_unstale":"2023-12-01T03:11:50.569753+0000","last_undegraded":"2023-12-01T03:11:50.569753+0000","last_fullsized":"2023-12-01T03:11:50.569753+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:24.244094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1f","version":"234'4","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.590253+0000","last_change":"2023-12-01T03:11:31.680864+0000","last_active":"2023-12-01T03:11:50.590253+0000","last_peered":"2023-12-01T03:11:50.590253+0000","last_clean":"2023-12-01T03:11:50.590253+0000","last_became_active":"2023-12-01T03:11:31.680279+0000","last_became_peered":"2023-12-01T03:11:31.680279+0000","last_unstale":"2023-12-01T03:11:50.590253+0000","last_undegraded":"2023-12-01T03:11:50.590253+0000","last_fullsized":"2023-12-01T03:11:50.590253+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.377359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070624+0000","last_change":"2023-12-01T03:11:51.069693+0000","last_active":"2023-12-01T03:11:51.070624+0000","last_peered":"2023-12-01T03:11:51.070624+0000","last_clean":"2023-12-01T03:11:51.070624+0000","last_became_active":"2023-12-01T03:11:51.069280+0000","last_became_peered":"2023-12-01T03:11:51.069280+0000","last_unstale":"2023-12-01T03:11:51.070624+0000","last_undegraded":"2023-12-01T03:11:51.070624+0000","last_fullsized":"2023-12-01T03:11:51.070624+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:29.351674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1c","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078893+0000","last_change":"2023-12-01T03:11:50.585796+0000","last_active":"2023-12-01T03:11:51.078893+0000","last_peered":"2023-12-01T03:11:51.078893+0000","last_clean":"2023-12-01T03:11:51.078893+0000","last_became_active":"2023-12-01T03:11:50.585488+0000","last_became_peered":"2023-12-01T03:11:50.585488+0000","last_unstale":"2023-12-01T03:11:51.078893+0000","last_undegraded":"2023-12-01T03:11:51.078893+0000","last_fullsized":"2023-12-01T03:11:51.078893+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:45:00.247816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.590136+0000","last_change":"2023-12-01T03:11:32.211743+0000","last_active":"2023-12-01T03:11:50.590136+0000","last_peered":"2023-12-01T03:11:50.590136+0000","last_clean":"2023-12-01T03:11:50.590136+0000","last_became_active":"2023-12-01T03:11:32.211443+0000","last_became_peered":"2023-12-01T03:11:32.211443+0000","last_unstale":"2023-12-01T03:11:50.590136+0000","last_undegraded":"2023-12-01T03:11:50.590136+0000","last_fullsized":"2023-12-01T03:11:50.590136+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"234'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583824+0000","last_change":"2023-12-01T03:11:31.679798+0000","last_active":"2023-12-01T03:11:50.583824+0000","last_peered":"2023-12-01T03:11:50.583824+0000","last_clean":"2023-12-01T03:11:50.583824+0000","last_became_active":"2023-12-01T03:11:31.679030+0000","last_became_peered":"2023-12-01T03:11:31.679030+0000","last_unstale":"2023-12-01T03:11:50.583824+0000","last_undegraded":"2023-12-01T03:11:50.583824+0000","last_fullsized":"2023-12-01T03:11:50.583824+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:49.261314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.080966+0000","last_change":"2023-12-01T03:11:51.076089+0000","last_active":"2023-12-01T03:11:51.080966+0000","last_peered":"2023-12-01T03:11:51.080966+0000","last_clean":"2023-12-01T03:11:51.080966+0000","last_became_active":"2023-12-01T03:11:51.071097+0000","last_became_peered":"2023-12-01T03:11:51.071097+0000","last_unstale":"2023-12-01T03:11:51.080966+0000","last_undegraded":"2023-12-01T03:11:51.080966+0000","last_fullsized":"2023-12-01T03:11:51.080966+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:07.114272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.074468+0000","last_change":"2023-12-01T03:11:50.585199+0000","last_active":"2023-12-01T03:11:51.074468+0000","last_peered":"2023-12-01T03:11:51.074468+0000","last_clean":"2023-12-01T03:11:51.074468+0000","last_became_active":"2023-12-01T03:11:50.584909+0000","last_became_peered":"2023-12-01T03:11:50.584909+0000","last_unstale":"2023-12-01T03:11:51.074468+0000","last_undegraded":"2023-12-01T03:11:51.074468+0000","last_fullsized":"2023-12-01T03:11:51.074468+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:05:21.126535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.592447+0000","last_change":"2023-12-01T03:11:32.222760+0000","last_active":"2023-12-01T03:11:50.592447+0000","last_peered":"2023-12-01T03:11:50.592447+0000","last_clean":"2023-12-01T03:11:50.592447+0000","last_became_active":"2023-12-01T03:11:32.222500+0000","last_became_peered":"2023-12-01T03:11:32.222500+0000","last_unstale":"2023-12-01T03:11:50.592447+0000","last_undegraded":"2023-12-01T03:11:50.592447+0000","last_fullsized":"2023-12-01T03:11:50.592447+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:17.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.580065+0000","last_change":"2023-12-01T03:11:31.663745+0000","last_active":"2023-12-01T03:11:50.580065+0000","last_peered":"2023-12-01T03:11:50.580065+0000","last_clean":"2023-12-01T03:11:50.580065+0000","last_became_active":"2023-12-01T03:11:31.662706+0000","last_became_peered":"2023-12-01T03:11:31.662706+0000","last_unstale":"2023-12-01T03:11:50.580065+0000","last_undegraded":"2023-12-01T03:11:50.580065+0000","last_fullsized":"2023-12-01T03:11:50.580065+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:42.945449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.087580+0000","last_change":"2023-12-01T03:11:51.072177+0000","last_active":"2023-12-01T03:11:51.087580+0000","last_peered":"2023-12-01T03:11:51.087580+0000","last_clean":"2023-12-01T03:11:51.087580+0000","last_became_active":"2023-12-01T03:11:51.071919+0000","last_became_peered":"2023-12-01T03:11:51.071919+0000","last_unstale":"2023-12-01T03:11:51.087580+0000","last_undegraded":"2023-12-01T03:11:51.087580+0000","last_fullsized":"2023-12-01T03:11:51.087580+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:54:09.453554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.055288+0000","last_change":"2023-12-01T03:11:50.594435+0000","last_active":"2023-12-01T03:11:51.055288+0000","last_peered":"2023-12-01T03:11:51.055288+0000","last_clean":"2023-12-01T03:11:51.055288+0000","last_became_active":"2023-12-01T03:11:50.594170+0000","last_became_peered":"2023-12-01T03:11:50.594170+0000","last_unstale":"2023-12-01T03:11:51.055288+0000","last_undegraded":"2023-12-01T03:11:51.055288+0000","last_fullsized":"2023-12-01T03:11:51.055288+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:34.274531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.570081+0000","last_change":"2023-12-01T03:11:32.213316+0000","last_active":"2023-12-01T03:11:50.570081+0000","last_peered":"2023-12-01T03:11:50.570081+0000","last_clean":"2023-12-01T03:11:50.570081+0000","last_became_active":"2023-12-01T03:11:32.213070+0000","last_became_peered":"2023-12-01T03:11:32.213070+0000","last_unstale":"2023-12-01T03:11:50.570081+0000","last_undegraded":"2023-12-01T03:11:50.570081+0000","last_fullsized":"2023-12-01T03:11:50.570081+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:22:51.687377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"234'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583999+0000","last_change":"2023-12-01T03:11:31.660758+0000","last_active":"2023-12-01T03:11:50.583999+0000","last_peered":"2023-12-01T03:11:50.583999+0000","last_clean":"2023-12-01T03:11:50.583999+0000","last_became_active":"2023-12-01T03:11:31.660001+0000","last_became_peered":"2023-12-01T03:11:31.660001+0000","last_unstale":"2023-12-01T03:11:50.583999+0000","last_undegraded":"2023-12-01T03:11:50.583999+0000","last_fullsized":"2023-12-01T03:11:50.583999+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088994+0000","last_change":"2023-12-01T03:11:51.080585+0000","last_active":"2023-12-01T03:11:51.088994+0000","last_peered":"2023-12-01T03:11:51.088994+0000","last_clean":"2023-12-01T03:11:51.088994+0000","last_became_active":"2023-12-01T03:11:51.079004+0000","last_became_peered":"2023-12-01T03:11:51.079004+0000","last_unstale":"2023-12-01T03:11:51.088994+0000","last_undegraded":"2023-12-01T03:11:51.088994+0000","last_fullsized":"2023-12-01T03:11:51.088994+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:49.158244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.055065+0000","last_change":"2023-12-01T03:11:50.588931+0000","last_active":"2023-12-01T03:11:51.055065+0000","last_peered":"2023-12-01T03:11:51.055065+0000","last_clean":"2023-12-01T03:11:51.055065+0000","last_became_active":"2023-12-01T03:11:50.588657+0000","last_became_peered":"2023-12-01T03:11:50.588657+0000","last_unstale":"2023-12-01T03:11:51.055065+0000","last_undegraded":"2023-12-01T03:11:51.055065+0000","last_fullsized":"2023-12-01T03:11:51.055065+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:35.737819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.588967+0000","last_change":"2023-12-01T03:11:32.213961+0000","last_active":"2023-12-01T03:11:50.588967+0000","last_peered":"2023-12-01T03:11:50.588967+0000","last_clean":"2023-12-01T03:11:50.588967+0000","last_became_active":"2023-12-01T03:11:32.213692+0000","last_became_peered":"2023-12-01T03:11:32.213692+0000","last_unstale":"2023-12-01T03:11:50.588967+0000","last_undegraded":"2023-12-01T03:11:50.588967+0000","last_fullsized":"2023-12-01T03:11:50.588967+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"234'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.580111+0000","last_change":"2023-12-01T03:11:31.681531+0000","last_active":"2023-12-01T03:11:50.580111+0000","last_peered":"2023-12-01T03:11:50.580111+0000","last_clean":"2023-12-01T03:11:50.580111+0000","last_became_active":"2023-12-01T03:11:31.680782+0000","last_became_peered":"2023-12-01T03:11:31.680782+0000","last_unstale":"2023-12-01T03:11:50.580111+0000","last_undegraded":"2023-12-01T03:11:50.580111+0000","last_fullsized":"2023-12-01T03:11:50.580111+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:18.018055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.065085+0000","last_change":"2023-12-01T03:11:51.058395+0000","last_active":"2023-12-01T03:11:51.065085+0000","last_peered":"2023-12-01T03:11:51.065085+0000","last_clean":"2023-12-01T03:11:51.065085+0000","last_became_active":"2023-12-01T03:11:51.058097+0000","last_became_peered":"2023-12-01T03:11:51.058097+0000","last_unstale":"2023-12-01T03:11:51.065085+0000","last_undegraded":"2023-12-01T03:11:51.065085+0000","last_fullsized":"2023-12-01T03:11:51.065085+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:29.351674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1b","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078692+0000","last_change":"2023-12-01T03:11:50.585680+0000","last_active":"2023-12-01T03:11:51.078692+0000","last_peered":"2023-12-01T03:11:51.078692+0000","last_clean":"2023-12-01T03:11:51.078692+0000","last_became_active":"2023-12-01T03:11:50.585097+0000","last_became_peered":"2023-12-01T03:11:50.585097+0000","last_unstale":"2023-12-01T03:11:51.078692+0000","last_undegraded":"2023-12-01T03:11:51.078692+0000","last_fullsized":"2023-12-01T03:11:51.078692+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:20.001579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582405+0000","last_change":"2023-12-01T03:11:32.206659+0000","last_active":"2023-12-01T03:11:50.582405+0000","last_peered":"2023-12-01T03:11:50.582405+0000","last_clean":"2023-12-01T03:11:50.582405+0000","last_became_active":"2023-12-01T03:11:32.206346+0000","last_became_peered":"2023-12-01T03:11:32.206346+0000","last_unstale":"2023-12-01T03:11:50.582405+0000","last_undegraded":"2023-12-01T03:11:50.582405+0000","last_fullsized":"2023-12-01T03:11:50.582405+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.591287+0000","last_change":"2023-12-01T03:11:31.663166+0000","last_active":"2023-12-01T03:11:50.591287+0000","last_peered":"2023-12-01T03:11:50.591287+0000","last_clean":"2023-12-01T03:11:50.591287+0000","last_became_active":"2023-12-01T03:11:31.662263+0000","last_became_peered":"2023-12-01T03:11:31.662263+0000","last_unstale":"2023-12-01T03:11:50.591287+0000","last_undegraded":"2023-12-01T03:11:50.591287+0000","last_fullsized":"2023-12-01T03:11:50.591287+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:57:00.965499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.080276+0000","last_change":"2023-12-01T03:11:51.079275+0000","last_active":"2023-12-01T03:11:51.080276+0000","last_peered":"2023-12-01T03:11:51.080276+0000","last_clean":"2023-12-01T03:11:51.080276+0000","last_became_active":"2023-12-01T03:11:51.078910+0000","last_became_peered":"2023-12-01T03:11:51.078910+0000","last_unstale":"2023-12-01T03:11:51.080276+0000","last_undegraded":"2023-12-01T03:11:51.080276+0000","last_fullsized":"2023-12-01T03:11:51.080276+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:01.612426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1a","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.080017+0000","last_change":"2023-12-01T03:11:50.585007+0000","last_active":"2023-12-01T03:11:51.080017+0000","last_peered":"2023-12-01T03:11:51.080017+0000","last_clean":"2023-12-01T03:11:51.080017+0000","last_became_active":"2023-12-01T03:11:50.576065+0000","last_became_peered":"2023-12-01T03:11:50.576065+0000","last_unstale":"2023-12-01T03:11:51.080017+0000","last_undegraded":"2023-12-01T03:11:51.080017+0000","last_fullsized":"2023-12-01T03:11:51.080017+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:04.781574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.578742+0000","last_change":"2023-12-01T03:11:32.210988+0000","last_active":"2023-12-01T03:11:50.578742+0000","last_peered":"2023-12-01T03:11:50.578742+0000","last_clean":"2023-12-01T03:11:50.578742+0000","last_became_active":"2023-12-01T03:11:32.207844+0000","last_became_peered":"2023-12-01T03:11:32.207844+0000","last_unstale":"2023-12-01T03:11:50.578742+0000","last_undegraded":"2023-12-01T03:11:50.578742+0000","last_fullsized":"2023-12-01T03:11:50.578742+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.587802+0000","last_change":"2023-12-01T03:11:31.670524+0000","last_active":"2023-12-01T03:11:50.587802+0000","last_peered":"2023-12-01T03:11:50.587802+0000","last_clean":"2023-12-01T03:11:50.587802+0000","last_became_active":"2023-12-01T03:11:31.670190+0000","last_became_peered":"2023-12-01T03:11:31.670190+0000","last_unstale":"2023-12-01T03:11:50.587802+0000","last_undegraded":"2023-12-01T03:11:50.587802+0000","last_fullsized":"2023-12-01T03:11:50.587802+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:45.874739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.075912+0000","last_change":"2023-12-01T03:11:51.070027+0000","last_active":"2023-12-01T03:11:51.075912+0000","last_peered":"2023-12-01T03:11:51.075912+0000","last_clean":"2023-12-01T03:11:51.075912+0000","last_became_active":"2023-12-01T03:11:51.069121+0000","last_became_peered":"2023-12-01T03:11:51.069121+0000","last_unstale":"2023-12-01T03:11:51.075912+0000","last_undegraded":"2023-12-01T03:11:51.075912+0000","last_fullsized":"2023-12-01T03:11:51.075912+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:59.791669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.15","version":"238'3","reported_seq":13,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.086847+0000","last_change":"2023-12-01T03:11:50.597304+0000","last_active":"2023-12-01T03:11:51.086847+0000","last_peered":"2023-12-01T03:11:51.086847+0000","last_clean":"2023-12-01T03:11:51.086847+0000","last_became_active":"2023-12-01T03:11:50.596949+0000","last_became_peered":"2023-12-01T03:11:50.596949+0000","last_unstale":"2023-12-01T03:11:51.086847+0000","last_undegraded":"2023-12-01T03:11:51.086847+0000","last_fullsized":"2023-12-01T03:11:51.086847+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:54.254506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583534+0000","last_change":"2023-12-01T03:11:32.212987+0000","last_active":"2023-12-01T03:11:50.583534+0000","last_peered":"2023-12-01T03:11:50.583534+0000","last_clean":"2023-12-01T03:11:50.583534+0000","last_became_active":"2023-12-01T03:11:32.212426+0000","last_became_peered":"2023-12-01T03:11:32.212426+0000","last_unstale":"2023-12-01T03:11:50.583534+0000","last_undegraded":"2023-12-01T03:11:50.583534+0000","last_fullsized":"2023-12-01T03:11:50.583534+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582762+0000","last_change":"2023-12-01T03:11:31.676322+0000","last_active":"2023-12-01T03:11:50.582762+0000","last_peered":"2023-12-01T03:11:50.582762+0000","last_clean":"2023-12-01T03:11:50.582762+0000","last_became_active":"2023-12-01T03:11:31.674451+0000","last_became_peered":"2023-12-01T03:11:31.674451+0000","last_unstale":"2023-12-01T03:11:50.582762+0000","last_undegraded":"2023-12-01T03:11:50.582762+0000","last_fullsized":"2023-12-01T03:11:50.582762+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:44.820233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070390+0000","last_change":"2023-12-01T03:11:51.068922+0000","last_active":"2023-12-01T03:11:51.070390+0000","last_peered":"2023-12-01T03:11:51.070390+0000","last_clean":"2023-12-01T03:11:51.070390+0000","last_became_active":"2023-12-01T03:11:51.068655+0000","last_became_peered":"2023-12-01T03:11:51.068655+0000","last_unstale":"2023-12-01T03:11:51.070390+0000","last_undegraded":"2023-12-01T03:11:51.070390+0000","last_fullsized":"2023-12-01T03:11:51.070390+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:01.630559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.14","version":"238'6","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.083939+0000","last_change":"2023-12-01T03:11:50.592947+0000","last_active":"2023-12-01T03:11:51.083939+0000","last_peered":"2023-12-01T03:11:51.083939+0000","last_clean":"2023-12-01T03:11:51.083939+0000","last_became_active":"2023-12-01T03:11:50.592670+0000","last_became_peered":"2023-12-01T03:11:50.592670+0000","last_unstale":"2023-12-01T03:11:51.083939+0000","last_undegraded":"2023-12-01T03:11:51.083939+0000","last_fullsized":"2023-12-01T03:11:51.083939+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:33:53.678613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.570001+0000","last_change":"2023-12-01T03:11:32.219383+0000","last_active":"2023-12-01T03:11:50.570001+0000","last_peered":"2023-12-01T03:11:50.570001+0000","last_clean":"2023-12-01T03:11:50.570001+0000","last_became_active":"2023-12-01T03:11:32.219139+0000","last_became_peered":"2023-12-01T03:11:32.219139+0000","last_unstale":"2023-12-01T03:11:50.570001+0000","last_undegraded":"2023-12-01T03:11:50.570001+0000","last_fullsized":"2023-12-01T03:11:50.570001+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:40.557913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.584590+0000","last_change":"2023-12-01T03:11:31.662929+0000","last_active":"2023-12-01T03:11:50.584590+0000","last_peered":"2023-12-01T03:11:50.584590+0000","last_clean":"2023-12-01T03:11:50.584590+0000","last_became_active":"2023-12-01T03:11:31.662617+0000","last_became_peered":"2023-12-01T03:11:31.662617+0000","last_unstale":"2023-12-01T03:11:50.584590+0000","last_undegraded":"2023-12-01T03:11:50.584590+0000","last_fullsized":"2023-12-01T03:11:50.584590+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:34.611280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089058+0000","last_change":"2023-12-01T03:11:51.080389+0000","last_active":"2023-12-01T03:11:51.089058+0000","last_peered":"2023-12-01T03:11:51.089058+0000","last_clean":"2023-12-01T03:11:51.089058+0000","last_became_active":"2023-12-01T03:11:51.080109+0000","last_became_peered":"2023-12-01T03:11:51.080109+0000","last_unstale":"2023-12-01T03:11:51.089058+0000","last_undegraded":"2023-12-01T03:11:51.089058+0000","last_fullsized":"2023-12-01T03:11:51.089058+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:59.791669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.055263+0000","last_change":"2023-12-01T03:11:50.585244+0000","last_active":"2023-12-01T03:11:51.055263+0000","last_peered":"2023-12-01T03:11:51.055263+0000","last_clean":"2023-12-01T03:11:51.055263+0000","last_became_active":"2023-12-01T03:11:50.584935+0000","last_became_peered":"2023-12-01T03:11:50.584935+0000","last_unstale":"2023-12-01T03:11:51.055263+0000","last_undegraded":"2023-12-01T03:11:51.055263+0000","last_fullsized":"2023-12-01T03:11:51.055263+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:32.427402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.579929+0000","last_change":"2023-12-01T03:11:32.209244+0000","last_active":"2023-12-01T03:11:50.579929+0000","last_peered":"2023-12-01T03:11:50.579929+0000","last_clean":"2023-12-01T03:11:50.579929+0000","last_became_active":"2023-12-01T03:11:32.206294+0000","last_became_peered":"2023-12-01T03:11:32.206294+0000","last_unstale":"2023-12-01T03:11:50.579929+0000","last_undegraded":"2023-12-01T03:11:50.579929+0000","last_fullsized":"2023-12-01T03:11:50.579929+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"234'3","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.564927+0000","last_change":"2023-12-01T03:11:31.664798+0000","last_active":"2023-12-01T03:11:50.564927+0000","last_peered":"2023-12-01T03:11:50.564927+0000","last_clean":"2023-12-01T03:11:50.564927+0000","last_became_active":"2023-12-01T03:11:31.664276+0000","last_became_peered":"2023-12-01T03:11:31.664276+0000","last_unstale":"2023-12-01T03:11:50.564927+0000","last_undegraded":"2023-12-01T03:11:50.564927+0000","last_fullsized":"2023-12-01T03:11:50.564927+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:18:18.909277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.076730+0000","last_change":"2023-12-01T03:11:51.062845+0000","last_active":"2023-12-01T03:11:51.076730+0000","last_peered":"2023-12-01T03:11:51.076730+0000","last_clean":"2023-12-01T03:11:51.076730+0000","last_became_active":"2023-12-01T03:11:51.062504+0000","last_became_peered":"2023-12-01T03:11:51.062504+0000","last_unstale":"2023-12-01T03:11:51.076730+0000","last_undegraded":"2023-12-01T03:11:51.076730+0000","last_fullsized":"2023-12-01T03:11:51.076730+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:04.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070187+0000","last_change":"2023-12-01T03:11:50.595799+0000","last_active":"2023-12-01T03:11:51.070187+0000","last_peered":"2023-12-01T03:11:51.070187+0000","last_clean":"2023-12-01T03:11:51.070187+0000","last_became_active":"2023-12-01T03:11:50.594796+0000","last_became_peered":"2023-12-01T03:11:50.594796+0000","last_unstale":"2023-12-01T03:11:51.070187+0000","last_undegraded":"2023-12-01T03:11:51.070187+0000","last_fullsized":"2023-12-01T03:11:51.070187+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:04.343416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582898+0000","last_change":"2023-12-01T03:11:32.213607+0000","last_active":"2023-12-01T03:11:50.582898+0000","last_peered":"2023-12-01T03:11:50.582898+0000","last_clean":"2023-12-01T03:11:50.582898+0000","last_became_active":"2023-12-01T03:11:32.211875+0000","last_became_peered":"2023-12-01T03:11:32.211875+0000","last_unstale":"2023-12-01T03:11:50.582898+0000","last_undegraded":"2023-12-01T03:11:50.582898+0000","last_fullsized":"2023-12-01T03:11:50.582898+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"235'8","reported_seq":22,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.588826+0000","last_change":"2023-12-01T03:11:31.664162+0000","last_active":"2023-12-01T03:11:50.588826+0000","last_peered":"2023-12-01T03:11:50.588826+0000","last_clean":"2023-12-01T03:11:50.588826+0000","last_became_active":"2023-12-01T03:11:31.663854+0000","last_became_peered":"2023-12-01T03:11:31.663854+0000","last_unstale":"2023-12-01T03:11:50.588826+0000","last_undegraded":"2023-12-01T03:11:50.588826+0000","last_fullsized":"2023-12-01T03:11:50.588826+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:15.367696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.083231+0000","last_change":"2023-12-01T03:11:51.081832+0000","last_active":"2023-12-01T03:11:51.083231+0000","last_peered":"2023-12-01T03:11:51.083231+0000","last_clean":"2023-12-01T03:11:51.083231+0000","last_became_active":"2023-12-01T03:11:51.081477+0000","last_became_peered":"2023-12-01T03:11:51.081477+0000","last_unstale":"2023-12-01T03:11:51.083231+0000","last_undegraded":"2023-12-01T03:11:51.083231+0000","last_fullsized":"2023-12-01T03:11:51.083231+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:28:12.383010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.054962+0000","last_change":"2023-12-01T03:11:50.598270+0000","last_active":"2023-12-01T03:11:51.054962+0000","last_peered":"2023-12-01T03:11:51.054962+0000","last_clean":"2023-12-01T03:11:51.054962+0000","last_became_active":"2023-12-01T03:11:50.597997+0000","last_became_peered":"2023-12-01T03:11:50.597997+0000","last_unstale":"2023-12-01T03:11:51.054962+0000","last_undegraded":"2023-12-01T03:11:51.054962+0000","last_fullsized":"2023-12-01T03:11:51.054962+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:05:05.484343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.586610+0000","last_change":"2023-12-01T03:11:32.203625+0000","last_active":"2023-12-01T03:11:50.586610+0000","last_peered":"2023-12-01T03:11:50.586610+0000","last_clean":"2023-12-01T03:11:50.586610+0000","last_became_active":"2023-12-01T03:11:32.203300+0000","last_became_peered":"2023-12-01T03:11:32.203300+0000","last_unstale":"2023-12-01T03:11:50.586610+0000","last_undegraded":"2023-12-01T03:11:50.586610+0000","last_fullsized":"2023-12-01T03:11:50.586610+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:46.890682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.566369+0000","last_change":"2023-12-01T03:11:31.665987+0000","last_active":"2023-12-01T03:11:50.566369+0000","last_peered":"2023-12-01T03:11:50.566369+0000","last_clean":"2023-12-01T03:11:50.566369+0000","last_became_active":"2023-12-01T03:11:31.665550+0000","last_became_peered":"2023-12-01T03:11:31.665550+0000","last_unstale":"2023-12-01T03:11:50.566369+0000","last_undegraded":"2023-12-01T03:11:50.566369+0000","last_fullsized":"2023-12-01T03:11:50.566369+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:16.324155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.062718+0000","last_change":"2023-12-01T03:11:51.058517+0000","last_active":"2023-12-01T03:11:51.062718+0000","last_peered":"2023-12-01T03:11:51.062718+0000","last_clean":"2023-12-01T03:11:51.062718+0000","last_became_active":"2023-12-01T03:11:51.058244+0000","last_became_peered":"2023-12-01T03:11:51.058244+0000","last_unstale":"2023-12-01T03:11:51.062718+0000","last_undegraded":"2023-12-01T03:11:51.062718+0000","last_fullsized":"2023-12-01T03:11:51.062718+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:59:45.386653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"238'8","reported_seq":18,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088097+0000","last_change":"2023-12-01T03:11:50.598938+0000","last_active":"2023-12-01T03:11:51.088097+0000","last_peered":"2023-12-01T03:11:51.088097+0000","last_clean":"2023-12-01T03:11:51.088097+0000","last_became_active":"2023-12-01T03:11:50.598444+0000","last_became_peered":"2023-12-01T03:11:50.598444+0000","last_unstale":"2023-12-01T03:11:51.088097+0000","last_undegraded":"2023-12-01T03:11:51.088097+0000","last_fullsized":"2023-12-01T03:11:51.088097+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:47.245587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.578127+0000","last_change":"2023-12-01T03:11:32.208104+0000","last_active":"2023-12-01T03:11:50.578127+0000","last_peered":"2023-12-01T03:11:50.578127+0000","last_clean":"2023-12-01T03:11:50.578127+0000","last_became_active":"2023-12-01T03:11:32.207859+0000","last_became_peered":"2023-12-01T03:11:32.207859+0000","last_unstale":"2023-12-01T03:11:50.578127+0000","last_undegraded":"2023-12-01T03:11:50.578127+0000","last_fullsized":"2023-12-01T03:11:50.578127+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.586731+0000","last_change":"2023-12-01T03:11:31.675387+0000","last_active":"2023-12-01T03:11:50.586731+0000","last_peered":"2023-12-01T03:11:50.586731+0000","last_clean":"2023-12-01T03:11:50.586731+0000","last_became_active":"2023-12-01T03:11:31.675108+0000","last_became_peered":"2023-12-01T03:11:31.675108+0000","last_unstale":"2023-12-01T03:11:50.586731+0000","last_undegraded":"2023-12-01T03:11:50.586731+0000","last_fullsized":"2023-12-01T03:11:50.586731+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.836486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.085561+0000","last_change":"2023-12-01T03:11:51.084910+0000","last_active":"2023-12-01T03:11:51.085561+0000","last_peered":"2023-12-01T03:11:51.085561+0000","last_clean":"2023-12-01T03:11:51.085561+0000","last_became_active":"2023-12-01T03:11:51.084552+0000","last_became_peered":"2023-12-01T03:11:51.084552+0000","last_unstale":"2023-12-01T03:11:51.085561+0000","last_undegraded":"2023-12-01T03:11:51.085561+0000","last_fullsized":"2023-12-01T03:11:51.085561+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:36.041546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.13","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.083335+0000","last_change":"2023-12-01T03:11:50.598810+0000","last_active":"2023-12-01T03:11:51.083335+0000","last_peered":"2023-12-01T03:11:51.083335+0000","last_clean":"2023-12-01T03:11:51.083335+0000","last_became_active":"2023-12-01T03:11:50.597756+0000","last_became_peered":"2023-12-01T03:11:50.597756+0000","last_unstale":"2023-12-01T03:11:51.083335+0000","last_undegraded":"2023-12-01T03:11:51.083335+0000","last_fullsized":"2023-12-01T03:11:51.083335+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:29:58.374404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.590599+0000","last_change":"2023-12-01T03:11:32.204685+0000","last_active":"2023-12-01T03:11:50.590599+0000","last_peered":"2023-12-01T03:11:50.590599+0000","last_clean":"2023-12-01T03:11:50.590599+0000","last_became_active":"2023-12-01T03:11:32.204422+0000","last_became_peered":"2023-12-01T03:11:32.204422+0000","last_unstale":"2023-12-01T03:11:50.590599+0000","last_undegraded":"2023-12-01T03:11:50.590599+0000","last_fullsized":"2023-12-01T03:11:50.590599+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583882+0000","last_change":"2023-12-01T03:11:31.671174+0000","last_active":"2023-12-01T03:11:50.583882+0000","last_peered":"2023-12-01T03:11:50.583882+0000","last_clean":"2023-12-01T03:11:50.583882+0000","last_became_active":"2023-12-01T03:11:31.670862+0000","last_became_peered":"2023-12-01T03:11:31.670862+0000","last_unstale":"2023-12-01T03:11:50.583882+0000","last_undegraded":"2023-12-01T03:11:50.583882+0000","last_fullsized":"2023-12-01T03:11:50.583882+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:37.158883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.066075+0000","last_change":"2023-12-01T03:11:51.062334+0000","last_active":"2023-12-01T03:11:51.066075+0000","last_peered":"2023-12-01T03:11:51.066075+0000","last_clean":"2023-12-01T03:11:51.066075+0000","last_became_active":"2023-12-01T03:11:51.062033+0000","last_became_peered":"2023-12-01T03:11:51.062033+0000","last_unstale":"2023-12-01T03:11:51.066075+0000","last_undegraded":"2023-12-01T03:11:51.066075+0000","last_fullsized":"2023-12-01T03:11:51.066075+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:13.658416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068064+0000","last_change":"2023-12-01T03:11:50.596720+0000","last_active":"2023-12-01T03:11:51.068064+0000","last_peered":"2023-12-01T03:11:51.068064+0000","last_clean":"2023-12-01T03:11:51.068064+0000","last_became_active":"2023-12-01T03:11:50.596434+0000","last_became_peered":"2023-12-01T03:11:50.596434+0000","last_unstale":"2023-12-01T03:11:51.068064+0000","last_undegraded":"2023-12-01T03:11:51.068064+0000","last_fullsized":"2023-12-01T03:11:51.068064+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:35.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568403+0000","last_change":"2023-12-01T03:11:32.203594+0000","last_active":"2023-12-01T03:11:50.568403+0000","last_peered":"2023-12-01T03:11:50.568403+0000","last_clean":"2023-12-01T03:11:50.568403+0000","last_became_active":"2023-12-01T03:11:32.203346+0000","last_became_peered":"2023-12-01T03:11:32.203346+0000","last_unstale":"2023-12-01T03:11:50.568403+0000","last_undegraded":"2023-12-01T03:11:50.568403+0000","last_fullsized":"2023-12-01T03:11:50.568403+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:22.924077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.10","version":"234'8","reported_seq":22,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568113+0000","last_change":"2023-12-01T03:11:31.673143+0000","last_active":"2023-12-01T03:11:50.568113+0000","last_peered":"2023-12-01T03:11:50.568113+0000","last_clean":"2023-12-01T03:11:50.568113+0000","last_became_active":"2023-12-01T03:11:31.671261+0000","last_became_peered":"2023-12-01T03:11:31.671261+0000","last_unstale":"2023-12-01T03:11:50.568113+0000","last_undegraded":"2023-12-01T03:11:50.568113+0000","last_fullsized":"2023-12-01T03:11:50.568113+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:13.744340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.065327+0000","last_change":"2023-12-01T03:11:51.059709+0000","last_active":"2023-12-01T03:11:51.065327+0000","last_peered":"2023-12-01T03:11:51.065327+0000","last_clean":"2023-12-01T03:11:51.065327+0000","last_became_active":"2023-12-01T03:11:51.059353+0000","last_became_peered":"2023-12-01T03:11:51.059353+0000","last_unstale":"2023-12-01T03:11:51.065327+0000","last_undegraded":"2023-12-01T03:11:51.065327+0000","last_fullsized":"2023-12-01T03:11:51.065327+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:21:35.791011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.d","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.074545+0000","last_change":"2023-12-01T03:11:50.591624+0000","last_active":"2023-12-01T03:11:51.074545+0000","last_peered":"2023-12-01T03:11:51.074545+0000","last_clean":"2023-12-01T03:11:51.074545+0000","last_became_active":"2023-12-01T03:11:50.590955+0000","last_became_peered":"2023-12-01T03:11:50.590955+0000","last_unstale":"2023-12-01T03:11:51.074545+0000","last_undegraded":"2023-12-01T03:11:51.074545+0000","last_fullsized":"2023-12-01T03:11:51.074545+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:05.252744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.584501+0000","last_change":"2023-12-01T03:11:32.209735+0000","last_active":"2023-12-01T03:11:50.584501+0000","last_peered":"2023-12-01T03:11:50.584501+0000","last_clean":"2023-12-01T03:11:50.584501+0000","last_became_active":"2023-12-01T03:11:32.209148+0000","last_became_peered":"2023-12-01T03:11:32.209148+0000","last_unstale":"2023-12-01T03:11:50.584501+0000","last_undegraded":"2023-12-01T03:11:50.584501+0000","last_fullsized":"2023-12-01T03:11:50.584501+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:46.532354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"234'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582869+0000","last_change":"2023-12-01T03:11:31.677841+0000","last_active":"2023-12-01T03:11:50.582869+0000","last_peered":"2023-12-01T03:11:50.582869+0000","last_clean":"2023-12-01T03:11:50.582869+0000","last_became_active":"2023-12-01T03:11:31.677483+0000","last_became_peered":"2023-12-01T03:11:31.677483+0000","last_unstale":"2023-12-01T03:11:50.582869+0000","last_undegraded":"2023-12-01T03:11:50.582869+0000","last_fullsized":"2023-12-01T03:11:50.582869+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:31.837132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078397+0000","last_change":"2023-12-01T03:11:51.077820+0000","last_active":"2023-12-01T03:11:51.078397+0000","last_peered":"2023-12-01T03:11:51.078397+0000","last_clean":"2023-12-01T03:11:51.078397+0000","last_became_active":"2023-12-01T03:11:51.077536+0000","last_became_peered":"2023-12-01T03:11:51.077536+0000","last_unstale":"2023-12-01T03:11:51.078397+0000","last_undegraded":"2023-12-01T03:11:51.078397+0000","last_fullsized":"2023-12-01T03:11:51.078397+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:48.264690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068796+0000","last_change":"2023-12-01T03:11:50.594903+0000","last_active":"2023-12-01T03:11:51.068796+0000","last_peered":"2023-12-01T03:11:51.068796+0000","last_clean":"2023-12-01T03:11:51.068796+0000","last_became_active":"2023-12-01T03:11:50.594146+0000","last_became_peered":"2023-12-01T03:11:50.594146+0000","last_unstale":"2023-12-01T03:11:51.068796+0000","last_undegraded":"2023-12-01T03:11:51.068796+0000","last_fullsized":"2023-12-01T03:11:51.068796+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:21:34.771908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.589488+0000","last_change":"2023-12-01T03:11:32.213539+0000","last_active":"2023-12-01T03:11:50.589488+0000","last_peered":"2023-12-01T03:11:50.589488+0000","last_clean":"2023-12-01T03:11:50.589488+0000","last_became_active":"2023-12-01T03:11:32.212492+0000","last_became_peered":"2023-12-01T03:11:32.212492+0000","last_unstale":"2023-12-01T03:11:50.589488+0000","last_undegraded":"2023-12-01T03:11:50.589488+0000","last_fullsized":"2023-12-01T03:11:50.589488+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:29:48.204277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.e","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581381+0000","last_change":"2023-12-01T03:11:31.670701+0000","last_active":"2023-12-01T03:11:50.581381+0000","last_peered":"2023-12-01T03:11:50.581381+0000","last_clean":"2023-12-01T03:11:50.581381+0000","last_became_active":"2023-12-01T03:11:31.670179+0000","last_became_peered":"2023-12-01T03:11:31.670179+0000","last_unstale":"2023-12-01T03:11:50.581381+0000","last_undegraded":"2023-12-01T03:11:50.581381+0000","last_fullsized":"2023-12-01T03:11:50.581381+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:01.094744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078798+0000","last_change":"2023-12-01T03:11:51.068023+0000","last_active":"2023-12-01T03:11:51.078798+0000","last_peered":"2023-12-01T03:11:51.078798+0000","last_clean":"2023-12-01T03:11:51.078798+0000","last_became_active":"2023-12-01T03:11:51.067743+0000","last_became_peered":"2023-12-01T03:11:51.067743+0000","last_unstale":"2023-12-01T03:11:51.078798+0000","last_undegraded":"2023-12-01T03:11:51.078798+0000","last_fullsized":"2023-12-01T03:11:51.078798+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:23:20.739511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089230+0000","last_change":"2023-12-01T03:11:50.588920+0000","last_active":"2023-12-01T03:11:51.089230+0000","last_peered":"2023-12-01T03:11:51.089230+0000","last_clean":"2023-12-01T03:11:51.089230+0000","last_became_active":"2023-12-01T03:11:50.588620+0000","last_became_peered":"2023-12-01T03:11:50.588620+0000","last_unstale":"2023-12-01T03:11:51.089230+0000","last_undegraded":"2023-12-01T03:11:51.089230+0000","last_fullsized":"2023-12-01T03:11:51.089230+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:19:47.329072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568782+0000","last_change":"2023-12-01T03:11:32.212327+0000","last_active":"2023-12-01T03:11:50.568782+0000","last_peered":"2023-12-01T03:11:50.568782+0000","last_clean":"2023-12-01T03:11:50.568782+0000","last_became_active":"2023-12-01T03:11:32.212068+0000","last_became_peered":"2023-12-01T03:11:32.212068+0000","last_unstale":"2023-12-01T03:11:50.568782+0000","last_undegraded":"2023-12-01T03:11:50.568782+0000","last_fullsized":"2023-12-01T03:11:50.568782+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:31.875817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"234'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582571+0000","last_change":"2023-12-01T03:11:31.679287+0000","last_active":"2023-12-01T03:11:50.582571+0000","last_peered":"2023-12-01T03:11:50.582571+0000","last_clean":"2023-12-01T03:11:50.582571+0000","last_became_active":"2023-12-01T03:11:31.678925+0000","last_became_peered":"2023-12-01T03:11:31.678925+0000","last_unstale":"2023-12-01T03:11:50.582571+0000","last_undegraded":"2023-12-01T03:11:50.582571+0000","last_fullsized":"2023-12-01T03:11:50.582571+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:28:46.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.062638+0000","last_change":"2023-12-01T03:11:51.061097+0000","last_active":"2023-12-01T03:11:51.062638+0000","last_peered":"2023-12-01T03:11:51.062638+0000","last_clean":"2023-12-01T03:11:51.062638+0000","last_became_active":"2023-12-01T03:11:51.060803+0000","last_became_peered":"2023-12-01T03:11:51.060803+0000","last_unstale":"2023-12-01T03:11:51.062638+0000","last_undegraded":"2023-12-01T03:11:51.062638+0000","last_fullsized":"2023-12-01T03:11:51.062638+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:10:05.844579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.e","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089007+0000","last_change":"2023-12-01T03:11:50.599084+0000","last_active":"2023-12-01T03:11:51.089007+0000","last_peered":"2023-12-01T03:11:51.089007+0000","last_clean":"2023-12-01T03:11:51.089007+0000","last_became_active":"2023-12-01T03:11:50.594125+0000","last_became_peered":"2023-12-01T03:11:50.594125+0000","last_unstale":"2023-12-01T03:11:51.089007+0000","last_undegraded":"2023-12-01T03:11:51.089007+0000","last_fullsized":"2023-12-01T03:11:51.089007+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:28:15.840367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.592519+0000","last_change":"2023-12-01T03:11:32.214436+0000","last_active":"2023-12-01T03:11:50.592519+0000","last_peered":"2023-12-01T03:11:50.592519+0000","last_clean":"2023-12-01T03:11:50.592519+0000","last_became_active":"2023-12-01T03:11:32.214162+0000","last_became_peered":"2023-12-01T03:11:32.214162+0000","last_unstale":"2023-12-01T03:11:50.592519+0000","last_undegraded":"2023-12-01T03:11:50.592519+0000","last_fullsized":"2023-12-01T03:11:50.592519+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:18:18.055289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583700+0000","last_change":"2023-12-01T03:11:31.674632+0000","last_active":"2023-12-01T03:11:50.583700+0000","last_peered":"2023-12-01T03:11:50.583700+0000","last_clean":"2023-12-01T03:11:50.583700+0000","last_became_active":"2023-12-01T03:11:31.674319+0000","last_became_peered":"2023-12-01T03:11:31.674319+0000","last_unstale":"2023-12-01T03:11:50.583700+0000","last_undegraded":"2023-12-01T03:11:50.583700+0000","last_fullsized":"2023-12-01T03:11:50.583700+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:21.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.072464+0000","last_change":"2023-12-01T03:11:51.064064+0000","last_active":"2023-12-01T03:11:51.072464+0000","last_peered":"2023-12-01T03:11:51.072464+0000","last_clean":"2023-12-01T03:11:51.072464+0000","last_became_active":"2023-12-01T03:11:51.061405+0000","last_became_peered":"2023-12-01T03:11:51.061405+0000","last_unstale":"2023-12-01T03:11:51.072464+0000","last_undegraded":"2023-12-01T03:11:51.072464+0000","last_fullsized":"2023-12-01T03:11:51.072464+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:55.481603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.062954+0000","last_change":"2023-12-01T03:11:50.589295+0000","last_active":"2023-12-01T03:11:51.062954+0000","last_peered":"2023-12-01T03:11:51.062954+0000","last_clean":"2023-12-01T03:11:51.062954+0000","last_became_active":"2023-12-01T03:11:50.588119+0000","last_became_peered":"2023-12-01T03:11:50.588119+0000","last_unstale":"2023-12-01T03:11:51.062954+0000","last_undegraded":"2023-12-01T03:11:51.062954+0000","last_fullsized":"2023-12-01T03:11:51.062954+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:00.593323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.579907+0000","last_change":"2023-12-01T03:11:32.209024+0000","last_active":"2023-12-01T03:11:50.579907+0000","last_peered":"2023-12-01T03:11:50.579907+0000","last_clean":"2023-12-01T03:11:50.579907+0000","last_became_active":"2023-12-01T03:11:32.208729+0000","last_became_peered":"2023-12-01T03:11:32.208729+0000","last_unstale":"2023-12-01T03:11:50.579907+0000","last_undegraded":"2023-12-01T03:11:50.579907+0000","last_fullsized":"2023-12-01T03:11:50.579907+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:27:53.473015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.b","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581606+0000","last_change":"2023-12-01T03:11:31.676885+0000","last_active":"2023-12-01T03:11:50.581606+0000","last_peered":"2023-12-01T03:11:50.581606+0000","last_clean":"2023-12-01T03:11:50.581606+0000","last_became_active":"2023-12-01T03:11:31.675545+0000","last_became_peered":"2023-12-01T03:11:31.675545+0000","last_unstale":"2023-12-01T03:11:50.581606+0000","last_undegraded":"2023-12-01T03:11:50.581606+0000","last_fullsized":"2023-12-01T03:11:50.581606+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:03.962061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070436+0000","last_change":"2023-12-01T03:11:51.068349+0000","last_active":"2023-12-01T03:11:51.070436+0000","last_peered":"2023-12-01T03:11:51.070436+0000","last_clean":"2023-12-01T03:11:51.070436+0000","last_became_active":"2023-12-01T03:11:51.068096+0000","last_became_peered":"2023-12-01T03:11:51.068096+0000","last_unstale":"2023-12-01T03:11:51.070436+0000","last_undegraded":"2023-12-01T03:11:51.070436+0000","last_fullsized":"2023-12-01T03:11:51.070436+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:28:12.383010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.049893+0000","last_change":"2023-12-01T03:11:50.585375+0000","last_active":"2023-12-01T03:11:51.049893+0000","last_peered":"2023-12-01T03:11:51.049893+0000","last_clean":"2023-12-01T03:11:51.049893+0000","last_became_active":"2023-12-01T03:11:50.584765+0000","last_became_peered":"2023-12-01T03:11:50.584765+0000","last_unstale":"2023-12-01T03:11:51.049893+0000","last_undegraded":"2023-12-01T03:11:51.049893+0000","last_fullsized":"2023-12-01T03:11:51.049893+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:35.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.593405+0000","last_change":"2023-12-01T03:11:32.204502+0000","last_active":"2023-12-01T03:11:50.593405+0000","last_peered":"2023-12-01T03:11:50.593405+0000","last_clean":"2023-12-01T03:11:50.593405+0000","last_became_active":"2023-12-01T03:11:32.204048+0000","last_became_peered":"2023-12-01T03:11:32.204048+0000","last_unstale":"2023-12-01T03:11:50.593405+0000","last_undegraded":"2023-12-01T03:11:50.593405+0000","last_fullsized":"2023-12-01T03:11:50.593405+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:15:30.930202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.577647+0000","last_change":"2023-12-01T03:11:31.675947+0000","last_active":"2023-12-01T03:11:50.577647+0000","last_peered":"2023-12-01T03:11:50.577647+0000","last_clean":"2023-12-01T03:11:50.577647+0000","last_became_active":"2023-12-01T03:11:31.675372+0000","last_became_peered":"2023-12-01T03:11:31.675372+0000","last_unstale":"2023-12-01T03:11:50.577647+0000","last_undegraded":"2023-12-01T03:11:50.577647+0000","last_fullsized":"2023-12-01T03:11:50.577647+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:45.516411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.073967+0000","last_change":"2023-12-01T03:11:51.073355+0000","last_active":"2023-12-01T03:11:51.073967+0000","last_peered":"2023-12-01T03:11:51.073967+0000","last_clean":"2023-12-01T03:11:51.073967+0000","last_became_active":"2023-12-01T03:11:51.072973+0000","last_became_peered":"2023-12-01T03:11:51.072973+0000","last_unstale":"2023-12-01T03:11:51.073967+0000","last_undegraded":"2023-12-01T03:11:51.073967+0000","last_fullsized":"2023-12-01T03:11:51.073967+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:06.271847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.b","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088280+0000","last_change":"2023-12-01T03:11:50.591967+0000","last_active":"2023-12-01T03:11:51.088280+0000","last_peered":"2023-12-01T03:11:51.088280+0000","last_clean":"2023-12-01T03:11:51.088280+0000","last_became_active":"2023-12-01T03:11:50.591673+0000","last_became_peered":"2023-12-01T03:11:50.591673+0000","last_unstale":"2023-12-01T03:11:51.088280+0000","last_undegraded":"2023-12-01T03:11:51.088280+0000","last_fullsized":"2023-12-01T03:11:51.088280+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:30:19.059675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582499+0000","last_change":"2023-12-01T03:11:32.214460+0000","last_active":"2023-12-01T03:11:50.582499+0000","last_peered":"2023-12-01T03:11:50.582499+0000","last_clean":"2023-12-01T03:11:50.582499+0000","last_became_active":"2023-12-01T03:11:32.213987+0000","last_became_peered":"2023-12-01T03:11:32.213987+0000","last_unstale":"2023-12-01T03:11:50.582499+0000","last_undegraded":"2023-12-01T03:11:50.582499+0000","last_fullsized":"2023-12-01T03:11:50.582499+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:20.713703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568566+0000","last_change":"2023-12-01T03:11:31.673899+0000","last_active":"2023-12-01T03:11:50.568566+0000","last_peered":"2023-12-01T03:11:50.568566+0000","last_clean":"2023-12-01T03:11:50.568566+0000","last_became_active":"2023-12-01T03:11:31.673474+0000","last_became_peered":"2023-12-01T03:11:31.673474+0000","last_unstale":"2023-12-01T03:11:50.568566+0000","last_undegraded":"2023-12-01T03:11:50.568566+0000","last_fullsized":"2023-12-01T03:11:50.568566+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:50.332038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088255+0000","last_change":"2023-12-01T03:11:51.074892+0000","last_active":"2023-12-01T03:11:51.088255+0000","last_peered":"2023-12-01T03:11:51.088255+0000","last_clean":"2023-12-01T03:11:51.088255+0000","last_became_active":"2023-12-01T03:11:51.074651+0000","last_became_peered":"2023-12-01T03:11:51.074651+0000","last_unstale":"2023-12-01T03:11:51.088255+0000","last_undegraded":"2023-12-01T03:11:51.088255+0000","last_fullsized":"2023-12-01T03:11:51.088255+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:10:05.844579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068152+0000","last_change":"2023-12-01T03:11:50.593853+0000","last_active":"2023-12-01T03:11:51.068152+0000","last_peered":"2023-12-01T03:11:51.068152+0000","last_clean":"2023-12-01T03:11:51.068152+0000","last_became_active":"2023-12-01T03:11:50.593584+0000","last_became_peered":"2023-12-01T03:11:50.593584+0000","last_unstale":"2023-12-01T03:11:51.068152+0000","last_undegraded":"2023-12-01T03:11:51.068152+0000","last_fullsized":"2023-12-01T03:11:51.068152+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:00.593323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.564517+0000","last_change":"2023-12-01T03:11:32.214835+0000","last_active":"2023-12-01T03:11:50.564517+0000","last_peered":"2023-12-01T03:11:50.564517+0000","last_clean":"2023-12-01T03:11:50.564517+0000","last_became_active":"2023-12-01T03:11:32.214553+0000","last_became_peered":"2023-12-01T03:11:32.214553+0000","last_unstale":"2023-12-01T03:11:50.564517+0000","last_undegraded":"2023-12-01T03:11:50.564517+0000","last_fullsized":"2023-12-01T03:11:50.564517+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568234+0000","last_change":"2023-12-01T03:11:31.682949+0000","last_active":"2023-12-01T03:11:50.568234+0000","last_peered":"2023-12-01T03:11:50.568234+0000","last_clean":"2023-12-01T03:11:50.568234+0000","last_became_active":"2023-12-01T03:11:31.682454+0000","last_became_peered":"2023-12-01T03:11:31.682454+0000","last_unstale":"2023-12-01T03:11:50.568234+0000","last_undegraded":"2023-12-01T03:11:50.568234+0000","last_fullsized":"2023-12-01T03:11:50.568234+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:49.777291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.081777+0000","last_change":"2023-12-01T03:11:51.073657+0000","last_active":"2023-12-01T03:11:51.081777+0000","last_peered":"2023-12-01T03:11:51.081777+0000","last_clean":"2023-12-01T03:11:51.081777+0000","last_became_active":"2023-12-01T03:11:51.072728+0000","last_became_peered":"2023-12-01T03:11:51.072728+0000","last_unstale":"2023-12-01T03:11:51.081777+0000","last_undegraded":"2023-12-01T03:11:51.081777+0000","last_fullsized":"2023-12-01T03:11:51.081777+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:09.196828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.055603+0000","last_change":"2023-12-01T03:11:50.595863+0000","last_active":"2023-12-01T03:11:51.055603+0000","last_peered":"2023-12-01T03:11:51.055603+0000","last_clean":"2023-12-01T03:11:51.055603+0000","last_became_active":"2023-12-01T03:11:50.595589+0000","last_became_peered":"2023-12-01T03:11:50.595589+0000","last_unstale":"2023-12-01T03:11:51.055603+0000","last_undegraded":"2023-12-01T03:11:51.055603+0000","last_fullsized":"2023-12-01T03:11:51.055603+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:19.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581273+0000","last_change":"2023-12-01T03:11:32.204430+0000","last_active":"2023-12-01T03:11:50.581273+0000","last_peered":"2023-12-01T03:11:50.581273+0000","last_clean":"2023-12-01T03:11:50.581273+0000","last_became_active":"2023-12-01T03:11:32.204043+0000","last_became_peered":"2023-12-01T03:11:32.204043+0000","last_unstale":"2023-12-01T03:11:50.581273+0000","last_undegraded":"2023-12-01T03:11:50.581273+0000","last_fullsized":"2023-12-01T03:11:50.581273+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:33:45.498838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.589595+0000","last_change":"2023-12-01T03:11:31.668375+0000","last_active":"2023-12-01T03:11:50.589595+0000","last_peered":"2023-12-01T03:11:50.589595+0000","last_clean":"2023-12-01T03:11:50.589595+0000","last_became_active":"2023-12-01T03:11:31.668109+0000","last_became_peered":"2023-12-01T03:11:31.668109+0000","last_unstale":"2023-12-01T03:11:50.589595+0000","last_undegraded":"2023-12-01T03:11:50.589595+0000","last_fullsized":"2023-12-01T03:11:50.589595+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:34.959657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089053+0000","last_change":"2023-12-01T03:11:51.084028+0000","last_active":"2023-12-01T03:11:51.089053+0000","last_peered":"2023-12-01T03:11:51.089053+0000","last_clean":"2023-12-01T03:11:51.089053+0000","last_became_active":"2023-12-01T03:11:51.079967+0000","last_became_peered":"2023-12-01T03:11:51.079967+0000","last_unstale":"2023-12-01T03:11:51.089053+0000","last_undegraded":"2023-12-01T03:11:51.089053+0000","last_fullsized":"2023-12-01T03:11:51.089053+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:00.020345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068930+0000","last_change":"2023-12-01T03:11:50.599761+0000","last_active":"2023-12-01T03:11:51.068930+0000","last_peered":"2023-12-01T03:11:51.068930+0000","last_clean":"2023-12-01T03:11:51.068930+0000","last_became_active":"2023-12-01T03:11:50.599505+0000","last_became_peered":"2023-12-01T03:11:50.599505+0000","last_unstale":"2023-12-01T03:11:51.068930+0000","last_undegraded":"2023-12-01T03:11:51.068930+0000","last_fullsized":"2023-12-01T03:11:51.068930+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:33:53.678613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582481+0000","last_change":"2023-12-01T03:11:32.223409+0000","last_active":"2023-12-01T03:11:50.582481+0000","last_peered":"2023-12-01T03:11:50.582481+0000","last_clean":"2023-12-01T03:11:50.582481+0000","last_became_active":"2023-12-01T03:11:32.221541+0000","last_became_peered":"2023-12-01T03:11:32.221541+0000","last_unstale":"2023-12-01T03:11:50.582481+0000","last_undegraded":"2023-12-01T03:11:50.582481+0000","last_fullsized":"2023-12-01T03:11:50.582481+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:53.273884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.6","version":"235'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583935+0000","last_change":"2023-12-01T03:11:31.679248+0000","last_active":"2023-12-01T03:11:50.583935+0000","last_peered":"2023-12-01T03:11:50.583935+0000","last_clean":"2023-12-01T03:11:50.583935+0000","last_became_active":"2023-12-01T03:11:31.678973+0000","last_became_peered":"2023-12-01T03:11:31.678973+0000","last_unstale":"2023-12-01T03:11:50.583935+0000","last_undegraded":"2023-12-01T03:11:50.583935+0000","last_fullsized":"2023-12-01T03:11:50.583935+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.073271+0000","last_change":"2023-12-01T03:11:51.071661+0000","last_active":"2023-12-01T03:11:51.073271+0000","last_peered":"2023-12-01T03:11:51.073271+0000","last_clean":"2023-12-01T03:11:51.073271+0000","last_became_active":"2023-12-01T03:11:51.071413+0000","last_became_peered":"2023-12-01T03:11:51.071413+0000","last_unstale":"2023-12-01T03:11:51.073271+0000","last_undegraded":"2023-12-01T03:11:51.073271+0000","last_fullsized":"2023-12-01T03:11:51.073271+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:24:05.519021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.7","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088668+0000","last_change":"2023-12-01T03:11:50.594548+0000","last_active":"2023-12-01T03:11:51.088668+0000","last_peered":"2023-12-01T03:11:51.088668+0000","last_clean":"2023-12-01T03:11:51.088668+0000","last_became_active":"2023-12-01T03:11:50.593162+0000","last_became_peered":"2023-12-01T03:11:50.593162+0000","last_unstale":"2023-12-01T03:11:51.088668+0000","last_undegraded":"2023-12-01T03:11:51.088668+0000","last_fullsized":"2023-12-01T03:11:51.088668+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:59:44.367550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581818+0000","last_change":"2023-12-01T03:11:32.207147+0000","last_active":"2023-12-01T03:11:50.581818+0000","last_peered":"2023-12-01T03:11:50.581818+0000","last_clean":"2023-12-01T03:11:50.581818+0000","last_became_active":"2023-12-01T03:11:32.206872+0000","last_became_peered":"2023-12-01T03:11:32.206872+0000","last_unstale":"2023-12-01T03:11:50.581818+0000","last_undegraded":"2023-12-01T03:11:50.581818+0000","last_fullsized":"2023-12-01T03:11:50.581818+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.592609+0000","last_change":"2023-12-01T03:11:31.663771+0000","last_active":"2023-12-01T03:11:50.592609+0000","last_peered":"2023-12-01T03:11:50.592609+0000","last_clean":"2023-12-01T03:11:50.592609+0000","last_became_active":"2023-12-01T03:11:31.662877+0000","last_became_peered":"2023-12-01T03:11:31.662877+0000","last_unstale":"2023-12-01T03:11:50.592609+0000","last_undegraded":"2023-12-01T03:11:50.592609+0000","last_fullsized":"2023-12-01T03:11:50.592609+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:59.396265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.090184+0000","last_change":"2023-12-01T03:11:51.089350+0000","last_active":"2023-12-01T03:11:51.090184+0000","last_peered":"2023-12-01T03:11:51.090184+0000","last_clean":"2023-12-01T03:11:51.090184+0000","last_became_active":"2023-12-01T03:11:51.089010+0000","last_became_peered":"2023-12-01T03:11:51.089010+0000","last_unstale":"2023-12-01T03:11:51.090184+0000","last_undegraded":"2023-12-01T03:11:51.090184+0000","last_fullsized":"2023-12-01T03:11:51.090184+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:45:06.821895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.060978+0000","last_change":"2023-12-01T03:11:50.594295+0000","last_active":"2023-12-01T03:11:51.060978+0000","last_peered":"2023-12-01T03:11:51.060978+0000","last_clean":"2023-12-01T03:11:51.060978+0000","last_became_active":"2023-12-01T03:11:50.594018+0000","last_became_peered":"2023-12-01T03:11:50.594018+0000","last_unstale":"2023-12-01T03:11:51.060978+0000","last_undegraded":"2023-12-01T03:11:51.060978+0000","last_fullsized":"2023-12-01T03:11:51.060978+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:53.866492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583031+0000","last_change":"2023-12-01T03:11:32.208042+0000","last_active":"2023-12-01T03:11:50.583031+0000","last_peered":"2023-12-01T03:11:50.583031+0000","last_clean":"2023-12-01T03:11:50.583031+0000","last_became_active":"2023-12-01T03:11:32.207767+0000","last_became_peered":"2023-12-01T03:11:32.207767+0000","last_unstale":"2023-12-01T03:11:50.583031+0000","last_undegraded":"2023-12-01T03:11:50.583031+0000","last_fullsized":"2023-12-01T03:11:50.583031+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:17.846927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568923+0000","last_change":"2023-12-01T03:11:31.673043+0000","last_active":"2023-12-01T03:11:50.568923+0000","last_peered":"2023-12-01T03:11:50.568923+0000","last_clean":"2023-12-01T03:11:50.568923+0000","last_became_active":"2023-12-01T03:11:31.672780+0000","last_became_peered":"2023-12-01T03:11:31.672780+0000","last_unstale":"2023-12-01T03:11:50.568923+0000","last_undegraded":"2023-12-01T03:11:50.568923+0000","last_fullsized":"2023-12-01T03:11:50.568923+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:40.478556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.061603+0000","last_change":"2023-12-01T03:11:51.060543+0000","last_active":"2023-12-01T03:11:51.061603+0000","last_peered":"2023-12-01T03:11:51.061603+0000","last_clean":"2023-12-01T03:11:51.061603+0000","last_became_active":"2023-12-01T03:11:51.060251+0000","last_became_peered":"2023-12-01T03:11:51.060251+0000","last_unstale":"2023-12-01T03:11:51.061603+0000","last_undegraded":"2023-12-01T03:11:51.061603+0000","last_fullsized":"2023-12-01T03:11:51.061603+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:51:04.334452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.056881+0000","last_change":"2023-12-01T03:11:50.593239+0000","last_active":"2023-12-01T03:11:51.056881+0000","last_peered":"2023-12-01T03:11:51.056881+0000","last_clean":"2023-12-01T03:11:51.056881+0000","last_became_active":"2023-12-01T03:11:50.592971+0000","last_became_peered":"2023-12-01T03:11:50.592971+0000","last_unstale":"2023-12-01T03:11:51.056881+0000","last_undegraded":"2023-12-01T03:11:51.056881+0000","last_fullsized":"2023-12-01T03:11:51.056881+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:04.343416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.579962+0000","last_change":"2023-12-01T03:11:32.224518+0000","last_active":"2023-12-01T03:11:50.579962+0000","last_peered":"2023-12-01T03:11:50.579962+0000","last_clean":"2023-12-01T03:11:50.579962+0000","last_became_active":"2023-12-01T03:11:32.220831+0000","last_became_peered":"2023-12-01T03:11:32.220831+0000","last_unstale":"2023-12-01T03:11:50.579962+0000","last_undegraded":"2023-12-01T03:11:50.579962+0000","last_fullsized":"2023-12-01T03:11:50.579962+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568848+0000","last_change":"2023-12-01T03:11:31.677162+0000","last_active":"2023-12-01T03:11:50.568848+0000","last_peered":"2023-12-01T03:11:50.568848+0000","last_clean":"2023-12-01T03:11:50.568848+0000","last_became_active":"2023-12-01T03:11:31.674178+0000","last_became_peered":"2023-12-01T03:11:31.674178+0000","last_unstale":"2023-12-01T03:11:50.568848+0000","last_undegraded":"2023-12-01T03:11:50.568848+0000","last_fullsized":"2023-12-01T03:11:50.568848+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:13.700505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.072346+0000","last_change":"2023-12-01T03:11:51.069551+0000","last_active":"2023-12-01T03:11:51.072346+0000","last_peered":"2023-12-01T03:11:51.072346+0000","last_clean":"2023-12-01T03:11:51.072346+0000","last_became_active":"2023-12-01T03:11:51.069264+0000","last_became_peered":"2023-12-01T03:11:51.069264+0000","last_unstale":"2023-12-01T03:11:51.072346+0000","last_undegraded":"2023-12-01T03:11:51.072346+0000","last_fullsized":"2023-12-01T03:11:51.072346+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:48.264690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.052593+0000","last_change":"2023-12-01T03:11:50.599045+0000","last_active":"2023-12-01T03:11:51.052593+0000","last_peered":"2023-12-01T03:11:51.052593+0000","last_clean":"2023-12-01T03:11:51.052593+0000","last_became_active":"2023-12-01T03:11:50.598742+0000","last_became_peered":"2023-12-01T03:11:50.598742+0000","last_unstale":"2023-12-01T03:11:51.052593+0000","last_undegraded":"2023-12-01T03:11:51.052593+0000","last_fullsized":"2023-12-01T03:11:51.052593+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:45:05.802792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581427+0000","last_change":"2023-12-01T03:11:32.222439+0000","last_active":"2023-12-01T03:11:50.581427+0000","last_peered":"2023-12-01T03:11:50.581427+0000","last_clean":"2023-12-01T03:11:50.581427+0000","last_became_active":"2023-12-01T03:11:32.221662+0000","last_became_peered":"2023-12-01T03:11:32.221662+0000","last_unstale":"2023-12-01T03:11:50.581427+0000","last_undegraded":"2023-12-01T03:11:50.581427+0000","last_fullsized":"2023-12-01T03:11:50.581427+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.590846+0000","last_change":"2023-12-01T03:11:31.672908+0000","last_active":"2023-12-01T03:11:50.590846+0000","last_peered":"2023-12-01T03:11:50.590846+0000","last_clean":"2023-12-01T03:11:50.590846+0000","last_became_active":"2023-12-01T03:11:31.671738+0000","last_became_peered":"2023-12-01T03:11:31.671738+0000","last_unstale":"2023-12-01T03:11:50.590846+0000","last_undegraded":"2023-12-01T03:11:50.590846+0000","last_fullsized":"2023-12-01T03:11:50.590846+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068190+0000","last_change":"2023-12-01T03:11:51.065065+0000","last_active":"2023-12-01T03:11:51.068190+0000","last_peered":"2023-12-01T03:11:51.068190+0000","last_clean":"2023-12-01T03:11:51.068190+0000","last_became_active":"2023-12-01T03:11:51.064818+0000","last_became_peered":"2023-12-01T03:11:51.064818+0000","last_unstale":"2023-12-01T03:11:51.068190+0000","last_undegraded":"2023-12-01T03:11:51.068190+0000","last_fullsized":"2023-12-01T03:11:51.068190+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:28:12.383010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.055803+0000","last_change":"2023-12-01T03:11:50.591414+0000","last_active":"2023-12-01T03:11:51.055803+0000","last_peered":"2023-12-01T03:11:51.055803+0000","last_clean":"2023-12-01T03:11:51.055803+0000","last_became_active":"2023-12-01T03:11:50.591116+0000","last_became_peered":"2023-12-01T03:11:50.591116+0000","last_unstale":"2023-12-01T03:11:51.055803+0000","last_undegraded":"2023-12-01T03:11:51.055803+0000","last_fullsized":"2023-12-01T03:11:51.055803+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:51:00.345760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568008+0000","last_change":"2023-12-01T03:11:32.214055+0000","last_active":"2023-12-01T03:11:50.568008+0000","last_peered":"2023-12-01T03:11:50.568008+0000","last_clean":"2023-12-01T03:11:50.568008+0000","last_became_active":"2023-12-01T03:11:32.213767+0000","last_became_peered":"2023-12-01T03:11:32.213767+0000","last_unstale":"2023-12-01T03:11:50.568008+0000","last_undegraded":"2023-12-01T03:11:50.568008+0000","last_fullsized":"2023-12-01T03:11:50.568008+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:47:03.446609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.564835+0000","last_change":"2023-12-01T03:11:31.676480+0000","last_active":"2023-12-01T03:11:50.564835+0000","last_peered":"2023-12-01T03:11:50.564835+0000","last_clean":"2023-12-01T03:11:50.564835+0000","last_became_active":"2023-12-01T03:11:31.676127+0000","last_became_peered":"2023-12-01T03:11:31.676127+0000","last_unstale":"2023-12-01T03:11:50.564835+0000","last_undegraded":"2023-12-01T03:11:50.564835+0000","last_fullsized":"2023-12-01T03:11:50.564835+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070497+0000","last_change":"2023-12-01T03:11:51.069233+0000","last_active":"2023-12-01T03:11:51.070497+0000","last_peered":"2023-12-01T03:11:51.070497+0000","last_clean":"2023-12-01T03:11:51.070497+0000","last_became_active":"2023-12-01T03:11:51.068196+0000","last_became_peered":"2023-12-01T03:11:51.068196+0000","last_unstale":"2023-12-01T03:11:51.070497+0000","last_undegraded":"2023-12-01T03:11:51.070497+0000","last_fullsized":"2023-12-01T03:11:51.070497+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:04.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.054236+0000","last_change":"2023-12-01T03:11:50.594879+0000","last_active":"2023-12-01T03:11:51.054236+0000","last_peered":"2023-12-01T03:11:51.054236+0000","last_clean":"2023-12-01T03:11:51.054236+0000","last_became_active":"2023-12-01T03:11:50.594328+0000","last_became_peered":"2023-12-01T03:11:50.594328+0000","last_unstale":"2023-12-01T03:11:51.054236+0000","last_undegraded":"2023-12-01T03:11:51.054236+0000","last_fullsized":"2023-12-01T03:11:51.054236+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:47.552866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.591188+0000","last_change":"2023-12-01T03:11:32.203024+0000","last_active":"2023-12-01T03:11:50.591188+0000","last_peered":"2023-12-01T03:11:50.591188+0000","last_clean":"2023-12-01T03:11:50.591188+0000","last_became_active":"2023-12-01T03:11:32.202779+0000","last_became_peered":"2023-12-01T03:11:32.202779+0000","last_unstale":"2023-12-01T03:11:50.591188+0000","last_undegraded":"2023-12-01T03:11:50.591188+0000","last_fullsized":"2023-12-01T03:11:50.591188+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.580650+0000","last_change":"2023-12-01T03:11:31.680251+0000","last_active":"2023-12-01T03:11:50.580650+0000","last_peered":"2023-12-01T03:11:50.580650+0000","last_clean":"2023-12-01T03:11:50.580650+0000","last_became_active":"2023-12-01T03:11:31.679936+0000","last_became_peered":"2023-12-01T03:11:31.679936+0000","last_unstale":"2023-12-01T03:11:50.580650+0000","last_undegraded":"2023-12-01T03:11:50.580650+0000","last_fullsized":"2023-12-01T03:11:50.580650+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:39.296836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.081403+0000","last_change":"2023-12-01T03:11:51.080842+0000","last_active":"2023-12-01T03:11:51.081403+0000","last_peered":"2023-12-01T03:11:51.081403+0000","last_clean":"2023-12-01T03:11:51.081403+0000","last_became_active":"2023-12-01T03:11:51.080626+0000","last_became_peered":"2023-12-01T03:11:51.080626+0000","last_unstale":"2023-12-01T03:11:51.081403+0000","last_undegraded":"2023-12-01T03:11:51.081403+0000","last_fullsized":"2023-12-01T03:11:51.081403+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:55.273609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1f","version":"238'4","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.080614+0000","last_change":"2023-12-01T03:11:50.597722+0000","last_active":"2023-12-01T03:11:51.080614+0000","last_peered":"2023-12-01T03:11:51.080614+0000","last_clean":"2023-12-01T03:11:51.080614+0000","last_became_active":"2023-12-01T03:11:50.596938+0000","last_became_peered":"2023-12-01T03:11:50.596938+0000","last_unstale":"2023-12-01T03:11:51.080614+0000","last_undegraded":"2023-12-01T03:11:51.080614+0000","last_fullsized":"2023-12-01T03:11:51.080614+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:59:44.367550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.593756+0000","last_change":"2023-12-01T03:11:32.210270+0000","last_active":"2023-12-01T03:11:50.593756+0000","last_peered":"2023-12-01T03:11:50.593756+0000","last_clean":"2023-12-01T03:11:50.593756+0000","last_became_active":"2023-12-01T03:11:32.209831+0000","last_became_peered":"2023-12-01T03:11:32.209831+0000","last_unstale":"2023-12-01T03:11:50.593756+0000","last_undegraded":"2023-12-01T03:11:50.593756+0000","last_fullsized":"2023-12-01T03:11:50.593756+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582476+0000","last_change":"2023-12-01T03:11:31.681391+0000","last_active":"2023-12-01T03:11:50.582476+0000","last_peered":"2023-12-01T03:11:50.582476+0000","last_clean":"2023-12-01T03:11:50.582476+0000","last_became_active":"2023-12-01T03:11:31.680773+0000","last_became_peered":"2023-12-01T03:11:31.680773+0000","last_unstale":"2023-12-01T03:11:50.582476+0000","last_undegraded":"2023-12-01T03:11:50.582476+0000","last_fullsized":"2023-12-01T03:11:50.582476+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:40:13.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"220'348","reported_seq":1217,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.580775+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:11:50.580775+0000","last_peered":"2023-12-01T03:11:50.580775+0000","last_clean":"2023-12-01T03:11:50.580775+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:11:50.580775+0000","last_undegraded":"2023-12-01T03:11:50.580775+0000","last_fullsized":"2023-12-01T03:11:50.580775+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575992,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60236,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311604,"statfs":{"total":96636764160,"available":96575082496,"internally_reserved":0,"allocated":3018752,"data_stored":1471908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.05,"15min":1.071},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5409999999999999,"5min":1.6000000000000001,"15min":1.7390000000000001},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6599999999999999,"5min":1.6699999999999999,"15min":1.7},"last":5.5800000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.236,"15min":1.2769999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4990000000000001,"5min":1.78,"15min":2.1419999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.294,"15min":1.3260000000000001},"min":{"1min":0.873,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5740000000000001,"5min":1.6240000000000001,"15min":1.8700000000000001},"last":1.137}]},{"osd":2,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1439999999999999,"15min":1.173},"min":{"1min":0.71699999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.573,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.2549999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.643,"5min":1.7050000000000001,"15min":2.1120000000000001},"last":1.8640000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2509999999999999,"15min":1.302},"min":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.712,"15min":2.431},"last":4.2720000000000002},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3720000000000001,"15min":1.4119999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7529999999999999,"5min":1.879,"15min":2.198},"last":2.4860000000000002}]},{"osd":4,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3009999999999999,"15min":1.335},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7689999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.4510000000000001,"15min":1.49},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8799999999999999,"5min":1.97,"15min":2.569},"last":1.2629999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2869999999999999,"15min":1.3160000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7150000000000001,"5min":2.0099999999999998,"15min":2.7269999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.5469999999999999,"15min":1.585},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.0470000000000002,"5min":2.0569999999999999,"15min":2.484},"last":1.6679999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.403,"15min":1.4359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7749999999999999,"5min":1.921,"15min":2.532},"last":4.8579999999999997},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4299999999999999,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8260000000000001,"5min":1.8819999999999999,"15min":2.762},"last":0.81799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575993,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42160,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329680,"statfs":{"total":96636764160,"available":96593592320,"internally_reserved":0,"allocated":2990080,"data_stored":1461513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0509999999999999,"15min":1.1080000000000001},"min":{"1min":0.66100000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.1040000000000001,"15min":1.157},"min":{"1min":0.78400000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6910000000000001,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":0.89400000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.141,"15min":1.2150000000000001},"min":{"1min":0.68999999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8460000000000001,"5min":1.976,"15min":2.0449999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1830000000000001,"15min":1.22},"min":{"1min":0.86099999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.6220000000000001,"5min":1.694,"15min":1.835},"last":1.044}]},{"osd":2,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.157,"15min":1.206},"min":{"1min":0.71799999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.464,"5min":1.742,"15min":1.976},"last":1.095},{"interface":"front","average":{"1min":1.131,"5min":1.222,"15min":1.278},"min":{"1min":0.71699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5009999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.288}]},{"osd":3,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2829999999999999,"15min":1.3140000000000001},"min":{"1min":0.89500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6839999999999999,"5min":1.927,"15min":2.0289999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":1.0069999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0190000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.93500000000000005}]},{"osd":4,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":0.93600000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.498},{"interface":"front","average":{"1min":1.589,"5min":1.407,"15min":1.367},"min":{"1min":1.2090000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.222,"5min":2.48,"15min":2.48},"last":1.407}]},{"osd":5,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.3700000000000001,"15min":1.413},"min":{"1min":1.127,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.962,"5min":2.262,"15min":2.262},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5169999999999999,"15min":1.4970000000000001},"min":{"1min":1.2629999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.194,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.252}]},{"osd":7,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5029999999999999,"15min":1.5309999999999999},"min":{"1min":1.155,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1320000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.417},{"interface":"front","average":{"1min":1.746,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":1.371,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2770000000000001,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.3580000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575992,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52620,"kb_used_data":3524,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319220,"statfs":{"total":96636764160,"available":96582881280,"internally_reserved":0,"allocated":3608576,"data_stored":2062423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1000000000000001,"15min":1.123},"min":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.7170000000000001,"5min":1.776,"15min":1.776},"last":1.194},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1539999999999999,"15min":1.161},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.752},"last":1.3879999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.1559999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6120000000000001,"5min":1.6519999999999999,"15min":1.696},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.198,"15min":1.2330000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":1.087}]},{"osd":2,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2270000000000001,"15min":1.2470000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.81399999999999995},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.167,"5min":1.2450000000000001,"15min":1.254},"min":{"1min":0.75,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.867,"5min":1.899,"15min":1.9219999999999999},"last":1.0169999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3089999999999999,"15min":1.3149999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.889,"5min":1.889,"15min":1.8979999999999999},"last":1.498},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.329,"15min":1.359},"min":{"1min":1.0109999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.97,"5min":1.97,"15min":2.1859999999999999},"last":1.2909999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.869,"5min":1.869,"15min":2.6040000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.242}]},{"osd":6,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3779999999999999,"15min":1.377},"min":{"1min":0.99099999999999999,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.7649999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.409,"15min":1.4219999999999999},"min":{"1min":1.0149999999999999,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":2.032,"5min":2.032,"15min":2.577},"last":1.3149999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.534,"15min":1.538},"min":{"1min":1.2010000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":2.077,"5min":2.077,"15min":3.1709999999999998},"last":1.619},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5760000000000001,"15min":1.5780000000000001},"min":{"1min":1.3200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.101,"15min":2.8740000000000001},"last":1.7050000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575992,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52032,"kb_used_data":2936,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319808,"statfs":{"total":96636764160,"available":96583483392,"internally_reserved":0,"allocated":3006464,"data_stored":1471883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1200000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8089999999999999,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.163},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.198,"15min":1.1899999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5309999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.345}]},{"osd":1,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2290000000000001,"15min":1.256},"min":{"1min":0.83599999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8580000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.238},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1719999999999999,"15min":1.175},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.778,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.2909999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3300000000000001,"15min":1.244},"min":{"1min":0.74399999999999999,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.637,"5min":11.311,"15min":11.311},"last":1.387},{"interface":"front","average":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3320000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9330000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.4350000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.298,"15min":1.29},"min":{"1min":0.80000000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.716,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3759999999999999,"15min":1.3340000000000001},"min":{"1min":0.96399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7430000000000001,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.357,"15min":1.3200000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.738,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.375,"15min":1.3160000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6579999999999999,"5min":3.71,"15min":3.71},"last":1.4750000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3799999999999999,"15min":1.3740000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8009999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4790000000000001,"15min":1.476},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.76,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.5249999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.514,"15min":1.488},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9830000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.573},{"interface":"front","average":{"1min":1.532,"5min":1.597,"15min":1.575},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.907,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.548}]}]},{"osd":3,"up_from":13,"seq":55834575991,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56112,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315728,"statfs":{"total":96636764160,"available":96579305472,"internally_reserved":0,"allocated":2990080,"data_stored":1469056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.306,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0449999999999999,"15min":1.0609999999999999},"min":{"1min":0.39500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.573,"5min":1.95,"15min":1.95},"last":1.161}]},{"osd":1,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.083,"15min":1.1479999999999999},"min":{"1min":0.56200000000000006,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.417,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.94399999999999995,"15min":0.98499999999999999},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6970000000000001,"5min":3.863,"15min":3.863},"last":1.198}]},{"osd":2,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.038,"15min":1.085},"min":{"1min":0.49099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.224,"5min":3.597,"15min":3.597},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.175,"15min":1.228},"min":{"1min":0.54700000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.647,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.0640000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.264,"15min":1.286},"min":{"1min":0.86099999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5269999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.825},{"interface":"front","average":{"1min":1.248,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.79500000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9970000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.9890000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3109999999999999,"15min":1.325},"min":{"1min":0.92200000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":1.8979999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.3420000000000001,"15min":1.3600000000000001},"min":{"1min":0.82499999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6699999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.7609999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.375,"15min":1.4370000000000001},"min":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":2.3079999999999998},{"interface":"front","average":{"1min":1.29,"5min":1.456,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.6879999999999999,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":2.0129999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4750000000000001,"15min":1.514},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.788,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4770000000000001,"15min":1.522},"min":{"1min":1.0549999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.913,"5min":3.004,"15min":3.004},"last":2.2440000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575991,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47928,"kb_used_data":2928,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323912,"statfs":{"total":96636764160,"available":96587685888,"internally_reserved":0,"allocated":2998272,"data_stored":1469160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.81100000000000005,"15min":0.87},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92100000000000004,"5min":1.411,"15min":1.4730000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98899999999999999,"15min":1.024},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.216,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.97299999999999998,"15min":1.0309999999999999},"min":{"1min":0.57899999999999996,"5min":0.378,"15min":0.378},"max":{"1min":1.512,"5min":1.512,"15min":1.698},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.04,"15min":1.113},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5229999999999999,"5min":1.635,"15min":1.6599999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.97499999999999998,"15min":1.022},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2889999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.097,"15min":1.181},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.4970000000000001,"5min":1.615,"15min":1.79},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2749999999999999,"15min":1.3120000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.621,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.276},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4730000000000001,"5min":1.714,"15min":1.835},"last":0.89500000000000002}]},{"osd":5,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.3109999999999999,"15min":1.3680000000000001},"min":{"1min":0.86599999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.639,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.347,"15min":1.393},"min":{"1min":0.79000000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.573,"5min":1.8740000000000001,"15min":2.016},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.351,"15min":1.367},"min":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.865,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.268,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.86599999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7090000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.0509999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4370000000000001,"15min":1.4630000000000001},"min":{"1min":0.94299999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5740000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.4350000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.8100000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.0189999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575990,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48516,"kb_used_data":3516,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323324,"statfs":{"total":96636764160,"available":96587083776,"internally_reserved":0,"allocated":3600384,"data_stored":2060680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.89800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.238,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.98499999999999999,"15min":1.038},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.512,"5min":1.5660000000000001,"15min":1.681},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2190000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.048,"5min":1.1020000000000001,"15min":1.141},"min":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5489999999999999,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":0.36399999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.208,"15min":1.2949999999999999},"min":{"1min":0.60799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5860000000000001,"5min":1.587,"15min":1.788},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.02,"5min":1.0780000000000001,"15min":1.127},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.383,"5min":1.714,"15min":1.8049999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2030000000000001,"15min":1.218},"min":{"1min":0.84399999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.623,"5min":1.623,"15min":1.6859999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3700000000000001,"15min":1.4039999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.657,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":0.77800000000000002}]},{"osd":5,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3109999999999999,"15min":1.333},"min":{"1min":0.95899999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":2.101},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.298,"15min":1.331},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.9019999999999999,"5min":1.927,"15min":1.927},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0529999999999999,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3939999999999999,"15min":1.413},"min":{"1min":0.91600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.98699999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4650000000000001,"15min":1.522},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.9430000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.524,"5min":1.48,"15min":1.5109999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.0699999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":0.94099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575992,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60216,"kb_used_data":2928,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311624,"statfs":{"total":96636764160,"available":96575102976,"internally_reserved":0,"allocated":2998272,"data_stored":1469148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.88900000000000001},"min":{"1min":0.314,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.994},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.98899999999999999,"15min":1.048},"min":{"1min":0.58999999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6799999999999999},"last":1.113}]},{"osd":2,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0169999999999999,"15min":1.0640000000000001},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.635,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0589999999999999,"15min":1.1259999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3100000000000001,"5min":1.726,"15min":1.792},"last":0.98599999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.036,"15min":1.0669999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.8839999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.171,"15min":1.224},"min":{"1min":0.70499999999999996,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.6469999999999998},"last":1.3580000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.296,"15min":1.3520000000000001},"min":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.681,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":1.216},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.3029999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.286}]},{"osd":5,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.90700000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.899,"15min":2.7719999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.383,"15min":1.427},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.681,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":1.3819999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3520000000000001,"15min":1.381},"min":{"1min":0.84599999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7270000000000001,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":1.452},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.45,"15min":1.4830000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.419}]},{"osd":7,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4670000000000001,"15min":1.516},"min":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7969999999999999,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4730000000000001,"15min":1.4870000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.927,"5min":2.1190000000000002,"15min":2.431},"last":1.405}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:11:55.531 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:11:55.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.782+0000 7f2cc6dd2640 1 Processor -- start 2023-12-01T03:11:55.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.782+0000 7f2cc6dd2640 1 -- start start 2023-12-01T03:11:55.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.783+0000 7f2cc6dd2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2cc00cad70 0x7f2cc006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.783+0000 7f2cc6dd2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2cc006f2f0 0x7f2cc006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.783+0000 7f2cc6dd2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc00add10 con 0x7f2cc015e2f0 2023-12-01T03:11:55.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.783+0000 7f2cc6dd2640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc00c5760 con 0x7f2cc006f2f0 2023-12-01T03:11:55.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.784+0000 7f2cc6dd2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc015e6d0 con 0x7f2cc00cad70 2023-12-01T03:11:55.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.784+0000 7f2cc536f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2cc006f2f0 0x7f2cc006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:55.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.784+0000 7f2cc4b6e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f2cc015e2f0 0x7f2cc00ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58818/0 (socket says 172.21.15.115:58818) 2023-12-01T03:11:55.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.784+0000 7f2cc4b6e640 1 -- 172.21.15.115:0/1649049333 learned_addr learned my addr 172.21.15.115:0/1649049333 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:55.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.784+0000 7f2cbffff640 1 --2- 172.21.15.115:0/1649049333 >> v2:172.21.15.142:3300/0 conn(0x7f2cc00cad70 0x7f2cc006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:55.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.784+0000 7f2cc536f640 1 -- 172.21.15.115:0/1649049333 >> v2:172.21.15.142:3300/0 conn(0x7f2cc00cad70 msgr2=0x7f2cc006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:55.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.784+0000 7f2cc536f640 1 --2- 172.21.15.115:0/1649049333 >> v2:172.21.15.142:3300/0 conn(0x7f2cc00cad70 0x7f2cc006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.784+0000 7f2cc536f640 1 -- 172.21.15.115:0/1649049333 >> v1:172.21.15.115:6789/0 conn(0x7f2cc015e2f0 legacy=0x7f2cc00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.784+0000 7f2cc536f640 1 -- 172.21.15.115:0/1649049333 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc00cf7b0 con 0x7f2cc006f2f0 2023-12-01T03:11:55.788 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.785+0000 7f2cc536f640 1 --2- 172.21.15.115:0/1649049333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 0x7f2cc006bb00 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009d90 tx=0x7f2cb802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f615d97e50b3e5f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:55.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.786+0000 7f2cbf7fe640 1 -- 172.21.15.115:0/1649049333 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb802cb40 con 0x7f2cc006f2f0 2023-12-01T03:11:55.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.786+0000 7f2cbf7fe640 1 -- 172.21.15.115:0/1649049333 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb802cca0 con 0x7f2cc006f2f0 2023-12-01T03:11:55.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.786+0000 7f2cbf7fe640 1 -- 172.21.15.115:0/1649049333 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8034660 con 0x7f2cc006f2f0 2023-12-01T03:11:55.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.786+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/1649049333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 msgr2=0x7f2cc006bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.786+0000 7f2cc6dd2640 1 --2- 172.21.15.115:0/1649049333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 0x7f2cc006bb00 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009d90 tx=0x7f2cb802bd90 comp rx=0 tx=0).stop 2023-12-01T03:11:55.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.786+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/1649049333 shutdown_connections 2023-12-01T03:11:55.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.786+0000 7f2cc6dd2640 1 --2- 172.21.15.115:0/1649049333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 0x7f2cc006bb00 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.786+0000 7f2cc6dd2640 1 --2- 172.21.15.115:0/1649049333 >> v2:172.21.15.142:3300/0 conn(0x7f2cc00cad70 0x7f2cc006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.786+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/1649049333 >> 172.21.15.115:0/1649049333 conn(0x7f2cc00ce880 msgr2=0x7f2cc00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:55.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.786+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/1649049333 shutdown_connections 2023-12-01T03:11:55.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.787+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/1649049333 wait complete. 2023-12-01T03:11:55.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.788+0000 7f2cc6dd2640 1 Processor -- start 2023-12-01T03:11:55.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.788+0000 7f2cc6dd2640 1 -- start start 2023-12-01T03:11:55.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.789+0000 7f2cc6dd2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 0x7f2cc0147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.789+0000 7f2cc6dd2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc00cad70 0x7f2cc00c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.794 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.789+0000 7f2cc6dd2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc015e2f0 0x7f2cc00c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.789+0000 7f2cc6dd2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc00c3af0 con 0x7f2cc006f2f0 2023-12-01T03:11:55.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.789+0000 7f2cc6dd2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc00c2e70 con 0x7f2cc00cad70 2023-12-01T03:11:55.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.789+0000 7f2cc6dd2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc00c3150 con 0x7f2cc015e2f0 2023-12-01T03:11:55.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.790+0000 7f2cc4b6e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 0x7f2cc0147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:55.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.790+0000 7f2cc4b6e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 0x7f2cc0147ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52818/0 (socket says 172.21.15.115:52818) 2023-12-01T03:11:55.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.790+0000 7f2cc4b6e640 1 -- 172.21.15.115:0/3815051398 learned_addr learned my addr 172.21.15.115:0/3815051398 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:55.795 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.790+0000 7f2cc4b6e640 1 -- 172.21.15.115:0/3815051398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc015e2f0 msgr2=0x7f2cc00c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.790+0000 7f2cc4b6e640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc015e2f0 0x7f2cc00c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.790+0000 7f2cc4b6e640 1 -- 172.21.15.115:0/3815051398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc00cad70 msgr2=0x7f2cc00c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:55.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.790+0000 7f2cc4b6e640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc00cad70 0x7f2cc00c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:55.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.790+0000 7f2cc4b6e640 1 -- 172.21.15.115:0/3815051398 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb8009a70 con 0x7f2cc006f2f0 2023-12-01T03:11:55.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.790+0000 7f2cc4b6e640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 0x7f2cc0147ee0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400c400 tx=0x7f2cb400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:55.796 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.791+0000 7f2cbd7fa640 1 -- 172.21.15.115:0/3815051398 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4005ce0 con 0x7f2cc006f2f0 2023-12-01T03:11:55.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.791+0000 7f2cbd7fa640 1 -- 172.21.15.115:0/3815051398 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb4005e40 con 0x7f2cc006f2f0 2023-12-01T03:11:55.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.791+0000 7f2cbd7fa640 1 -- 172.21.15.115:0/3815051398 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4007310 con 0x7f2cc006f2f0 2023-12-01T03:11:55.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.791+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/3815051398 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc00c3430 con 0x7f2cc006f2f0 2023-12-01T03:11:55.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.791+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/3815051398 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc0148420 con 0x7f2cc006f2f0 2023-12-01T03:11:55.797 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.792+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/3815051398 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc006fda0 con 0x7f2cc006f2f0 2023-12-01T03:11:55.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.793+0000 7f2cbd7fa640 1 -- 172.21.15.115:0/3815051398 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f2cb4007470 con 0x7f2cc006f2f0 2023-12-01T03:11:55.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.794+0000 7f2cbd7fa640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2ca406a670 0x7f2ca406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:55.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.794+0000 7f2cbd7fa640 1 -- 172.21.15.115:0/3815051398 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f2cb4088680 con 0x7f2cc006f2f0 2023-12-01T03:11:55.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.794+0000 7f2cbffff640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2ca406a670 0x7f2ca406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:55.799 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.795+0000 7f2cbffff640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2ca406a670 0x7f2ca406cb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0148bd0 tx=0x7f2cac008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:55.800 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.798+0000 7f2cbd7fa640 1 -- 172.21.15.115:0/3815051398 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2cb4053570 con 0x7f2cc006f2f0 2023-12-01T03:11:55.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:55.985+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/3815051398 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2cc00b43d0 con 0x7f2ca406a670 2023-12-01T03:11:56.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.000+0000 7f2cbd7fa640 1 -- 172.21.15.115:0/3815051398 <== mgr.11121 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526590 (secure 0 0 0) 0x7f2cac00adc0 con 0x7f2ca406a670 2023-12-01T03:11:56.006 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:56.012 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:11:56.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/3815051398 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2ca406a670 msgr2=0x7f2ca406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:56.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2ca406a670 0x7f2ca406cb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0148bd0 tx=0x7f2cac008040 comp rx=0 tx=0).stop 2023-12-01T03:11:56.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/3815051398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 msgr2=0x7f2cc0147ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:56.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 0x7f2cc0147ee0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400c400 tx=0x7f2cb400c8c0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/3815051398 shutdown_connections 2023-12-01T03:11:56.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2ca406a670 0x7f2ca406cb30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc015e2f0 0x7f2cc00c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc00cad70 0x7f2cc00c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 --2- 172.21.15.115:0/3815051398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cc006f2f0 0x7f2cc0147ee0 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/3815051398 >> 172.21.15.115:0/3815051398 conn(0x7f2cc00ce880 msgr2=0x7f2cc00b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:56.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/3815051398 shutdown_connections 2023-12-01T03:11:56.015 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.010+0000 7f2cc6dd2640 1 -- 172.21.15.115:0/3815051398 wait complete. 2023-12-01T03:11:56.033 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2023-12-01T03:11:55.828829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":670,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":427,"ondisk_log_size":427,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":267,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":419820,"kb_used_data":24620,"kb_used_omap":0,"kb_used_meta":395136,"kb_avail":754554900,"statfs":{"total":773094113280,"available":772664217600,"internally_reserved":0,"allocated":25210880,"data_stored":12935771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":404619264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3349,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002247"},"pg_stats":[{"pgid":"31.3d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089847+0000","last_change":"2023-12-01T03:11:51.088748+0000","last_active":"2023-12-01T03:11:51.089847+0000","last_peered":"2023-12-01T03:11:51.089847+0000","last_clean":"2023-12-01T03:11:51.089847+0000","last_became_active":"2023-12-01T03:11:51.087164+0000","last_became_peered":"2023-12-01T03:11:51.087164+0000","last_unstale":"2023-12-01T03:11:51.089847+0000","last_undegraded":"2023-12-01T03:11:51.089847+0000","last_fullsized":"2023-12-01T03:11:51.089847+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:38:38.426875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582373+0000","last_change":"2023-12-01T03:11:32.212603+0000","last_active":"2023-12-01T03:11:50.582373+0000","last_peered":"2023-12-01T03:11:50.582373+0000","last_clean":"2023-12-01T03:11:50.582373+0000","last_became_active":"2023-12-01T03:11:32.212287+0000","last_became_peered":"2023-12-01T03:11:32.212287+0000","last_unstale":"2023-12-01T03:11:50.582373+0000","last_undegraded":"2023-12-01T03:11:50.582373+0000","last_fullsized":"2023-12-01T03:11:50.582373+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.087638+0000","last_change":"2023-12-01T03:11:51.073555+0000","last_active":"2023-12-01T03:11:51.087638+0000","last_peered":"2023-12-01T03:11:51.087638+0000","last_clean":"2023-12-01T03:11:51.087638+0000","last_became_active":"2023-12-01T03:11:51.073247+0000","last_became_peered":"2023-12-01T03:11:51.073247+0000","last_unstale":"2023-12-01T03:11:51.087638+0000","last_undegraded":"2023-12-01T03:11:51.087638+0000","last_fullsized":"2023-12-01T03:11:51.087638+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:10.005791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569672+0000","last_change":"2023-12-01T03:11:32.217098+0000","last_active":"2023-12-01T03:11:50.569672+0000","last_peered":"2023-12-01T03:11:50.569672+0000","last_clean":"2023-12-01T03:11:50.569672+0000","last_became_active":"2023-12-01T03:11:32.216867+0000","last_became_peered":"2023-12-01T03:11:32.216867+0000","last_unstale":"2023-12-01T03:11:50.569672+0000","last_undegraded":"2023-12-01T03:11:50.569672+0000","last_fullsized":"2023-12-01T03:11:50.569672+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:57:10.849908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.083721+0000","last_change":"2023-12-01T03:11:51.081937+0000","last_active":"2023-12-01T03:11:51.083721+0000","last_peered":"2023-12-01T03:11:51.083721+0000","last_clean":"2023-12-01T03:11:51.083721+0000","last_became_active":"2023-12-01T03:11:51.081497+0000","last_became_peered":"2023-12-01T03:11:51.081497+0000","last_unstale":"2023-12-01T03:11:51.083721+0000","last_undegraded":"2023-12-01T03:11:51.083721+0000","last_fullsized":"2023-12-01T03:11:51.083721+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:40:11.382428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569344+0000","last_change":"2023-12-01T03:11:32.208423+0000","last_active":"2023-12-01T03:11:50.569344+0000","last_peered":"2023-12-01T03:11:50.569344+0000","last_clean":"2023-12-01T03:11:50.569344+0000","last_became_active":"2023-12-01T03:11:32.208113+0000","last_became_peered":"2023-12-01T03:11:32.208113+0000","last_unstale":"2023-12-01T03:11:50.569344+0000","last_undegraded":"2023-12-01T03:11:50.569344+0000","last_fullsized":"2023-12-01T03:11:50.569344+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:38.174827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.072305+0000","last_change":"2023-12-01T03:11:51.066754+0000","last_active":"2023-12-01T03:11:51.072305+0000","last_peered":"2023-12-01T03:11:51.072305+0000","last_clean":"2023-12-01T03:11:51.072305+0000","last_became_active":"2023-12-01T03:11:51.065359+0000","last_became_peered":"2023-12-01T03:11:51.065359+0000","last_unstale":"2023-12-01T03:11:51.072305+0000","last_undegraded":"2023-12-01T03:11:51.072305+0000","last_fullsized":"2023-12-01T03:11:51.072305+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:38.991634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.584143+0000","last_change":"2023-12-01T03:11:32.207840+0000","last_active":"2023-12-01T03:11:50.584143+0000","last_peered":"2023-12-01T03:11:50.584143+0000","last_clean":"2023-12-01T03:11:50.584143+0000","last_became_active":"2023-12-01T03:11:32.207048+0000","last_became_peered":"2023-12-01T03:11:32.207048+0000","last_unstale":"2023-12-01T03:11:50.584143+0000","last_undegraded":"2023-12-01T03:11:50.584143+0000","last_fullsized":"2023-12-01T03:11:50.584143+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:43.961392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.081151+0000","last_change":"2023-12-01T03:11:51.080141+0000","last_active":"2023-12-01T03:11:51.081151+0000","last_peered":"2023-12-01T03:11:51.081151+0000","last_clean":"2023-12-01T03:11:51.081151+0000","last_became_active":"2023-12-01T03:11:51.079871+0000","last_became_peered":"2023-12-01T03:11:51.079871+0000","last_unstale":"2023-12-01T03:11:51.081151+0000","last_undegraded":"2023-12-01T03:11:51.081151+0000","last_fullsized":"2023-12-01T03:11:51.081151+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:13:18.202586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583207+0000","last_change":"2023-12-01T03:11:32.206960+0000","last_active":"2023-12-01T03:11:50.583207+0000","last_peered":"2023-12-01T03:11:50.583207+0000","last_clean":"2023-12-01T03:11:50.583207+0000","last_became_active":"2023-12-01T03:11:32.203345+0000","last_became_peered":"2023-12-01T03:11:32.203345+0000","last_unstale":"2023-12-01T03:11:50.583207+0000","last_undegraded":"2023-12-01T03:11:50.583207+0000","last_fullsized":"2023-12-01T03:11:50.583207+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070238+0000","last_change":"2023-12-01T03:11:51.068646+0000","last_active":"2023-12-01T03:11:51.070238+0000","last_peered":"2023-12-01T03:11:51.070238+0000","last_clean":"2023-12-01T03:11:51.070238+0000","last_became_active":"2023-12-01T03:11:51.068386+0000","last_became_peered":"2023-12-01T03:11:51.068386+0000","last_unstale":"2023-12-01T03:11:51.070238+0000","last_undegraded":"2023-12-01T03:11:51.070238+0000","last_fullsized":"2023-12-01T03:11:51.070238+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:29.351674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.586169+0000","last_change":"2023-12-01T03:11:32.205496+0000","last_active":"2023-12-01T03:11:50.586169+0000","last_peered":"2023-12-01T03:11:50.586169+0000","last_clean":"2023-12-01T03:11:50.586169+0000","last_became_active":"2023-12-01T03:11:32.205074+0000","last_became_peered":"2023-12-01T03:11:32.205074+0000","last_unstale":"2023-12-01T03:11:50.586169+0000","last_undegraded":"2023-12-01T03:11:50.586169+0000","last_fullsized":"2023-12-01T03:11:50.586169+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:02.110687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070482+0000","last_change":"2023-12-01T03:11:51.067697+0000","last_active":"2023-12-01T03:11:51.070482+0000","last_peered":"2023-12-01T03:11:51.070482+0000","last_clean":"2023-12-01T03:11:51.070482+0000","last_became_active":"2023-12-01T03:11:51.065158+0000","last_became_peered":"2023-12-01T03:11:51.065158+0000","last_unstale":"2023-12-01T03:11:51.070482+0000","last_undegraded":"2023-12-01T03:11:51.070482+0000","last_fullsized":"2023-12-01T03:11:51.070482+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:13.658416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568771+0000","last_change":"2023-12-01T03:11:32.202465+0000","last_active":"2023-12-01T03:11:50.568771+0000","last_peered":"2023-12-01T03:11:50.568771+0000","last_clean":"2023-12-01T03:11:50.568771+0000","last_became_active":"2023-12-01T03:11:32.200165+0000","last_became_peered":"2023-12-01T03:11:32.200165+0000","last_unstale":"2023-12-01T03:11:50.568771+0000","last_undegraded":"2023-12-01T03:11:50.568771+0000","last_fullsized":"2023-12-01T03:11:50.568771+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:35:19.277681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088652+0000","last_change":"2023-12-01T03:11:51.087856+0000","last_active":"2023-12-01T03:11:51.088652+0000","last_peered":"2023-12-01T03:11:51.088652+0000","last_clean":"2023-12-01T03:11:51.088652+0000","last_became_active":"2023-12-01T03:11:51.087320+0000","last_became_peered":"2023-12-01T03:11:51.087320+0000","last_unstale":"2023-12-01T03:11:51.088652+0000","last_undegraded":"2023-12-01T03:11:51.088652+0000","last_fullsized":"2023-12-01T03:11:51.088652+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:29.292487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.563843+0000","last_change":"2023-12-01T03:11:32.217253+0000","last_active":"2023-12-01T03:11:50.563843+0000","last_peered":"2023-12-01T03:11:50.563843+0000","last_clean":"2023-12-01T03:11:50.563843+0000","last_became_active":"2023-12-01T03:11:32.216425+0000","last_became_peered":"2023-12-01T03:11:32.216425+0000","last_unstale":"2023-12-01T03:11:50.563843+0000","last_undegraded":"2023-12-01T03:11:50.563843+0000","last_fullsized":"2023-12-01T03:11:50.563843+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:51.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089032+0000","last_change":"2023-12-01T03:11:51.083849+0000","last_active":"2023-12-01T03:11:51.089032+0000","last_peered":"2023-12-01T03:11:51.089032+0000","last_clean":"2023-12-01T03:11:51.089032+0000","last_became_active":"2023-12-01T03:11:51.079649+0000","last_became_peered":"2023-12-01T03:11:51.079649+0000","last_unstale":"2023-12-01T03:11:51.089032+0000","last_undegraded":"2023-12-01T03:11:51.089032+0000","last_fullsized":"2023-12-01T03:11:51.089032+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:24:24.492502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.593355+0000","last_change":"2023-12-01T03:11:32.211881+0000","last_active":"2023-12-01T03:11:50.593355+0000","last_peered":"2023-12-01T03:11:50.593355+0000","last_clean":"2023-12-01T03:11:50.593355+0000","last_became_active":"2023-12-01T03:11:32.211602+0000","last_became_peered":"2023-12-01T03:11:32.211602+0000","last_unstale":"2023-12-01T03:11:50.593355+0000","last_undegraded":"2023-12-01T03:11:50.593355+0000","last_fullsized":"2023-12-01T03:11:50.593355+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.081093+0000","last_change":"2023-12-01T03:11:51.080082+0000","last_active":"2023-12-01T03:11:51.081093+0000","last_peered":"2023-12-01T03:11:51.081093+0000","last_clean":"2023-12-01T03:11:51.081093+0000","last_became_active":"2023-12-01T03:11:51.079868+0000","last_became_peered":"2023-12-01T03:11:51.079868+0000","last_unstale":"2023-12-01T03:11:51.081093+0000","last_undegraded":"2023-12-01T03:11:51.081093+0000","last_fullsized":"2023-12-01T03:11:51.081093+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:45:06.821895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.585831+0000","last_change":"2023-12-01T03:11:32.200099+0000","last_active":"2023-12-01T03:11:50.585831+0000","last_peered":"2023-12-01T03:11:50.585831+0000","last_clean":"2023-12-01T03:11:50.585831+0000","last_became_active":"2023-12-01T03:11:32.199703+0000","last_became_peered":"2023-12-01T03:11:32.199703+0000","last_unstale":"2023-12-01T03:11:50.585831+0000","last_undegraded":"2023-12-01T03:11:50.585831+0000","last_fullsized":"2023-12-01T03:11:50.585831+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:44:42.356924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.071966+0000","last_change":"2023-12-01T03:11:51.070269+0000","last_active":"2023-12-01T03:11:51.071966+0000","last_peered":"2023-12-01T03:11:51.071966+0000","last_clean":"2023-12-01T03:11:51.071966+0000","last_became_active":"2023-12-01T03:11:51.069876+0000","last_became_peered":"2023-12-01T03:11:51.069876+0000","last_unstale":"2023-12-01T03:11:51.071966+0000","last_undegraded":"2023-12-01T03:11:51.071966+0000","last_fullsized":"2023-12-01T03:11:51.071966+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:04.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569551+0000","last_change":"2023-12-01T03:11:32.211517+0000","last_active":"2023-12-01T03:11:50.569551+0000","last_peered":"2023-12-01T03:11:50.569551+0000","last_clean":"2023-12-01T03:11:50.569551+0000","last_became_active":"2023-12-01T03:11:32.211273+0000","last_became_peered":"2023-12-01T03:11:32.211273+0000","last_unstale":"2023-12-01T03:11:50.569551+0000","last_undegraded":"2023-12-01T03:11:50.569551+0000","last_fullsized":"2023-12-01T03:11:50.569551+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:35.627224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.081422+0000","last_change":"2023-12-01T03:11:51.080839+0000","last_active":"2023-12-01T03:11:51.081422+0000","last_peered":"2023-12-01T03:11:51.081422+0000","last_clean":"2023-12-01T03:11:51.081422+0000","last_became_active":"2023-12-01T03:11:51.080590+0000","last_became_peered":"2023-12-01T03:11:51.080590+0000","last_unstale":"2023-12-01T03:11:51.081422+0000","last_undegraded":"2023-12-01T03:11:51.081422+0000","last_fullsized":"2023-12-01T03:11:51.081422+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:29.292487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.579811+0000","last_change":"2023-12-01T03:11:32.202825+0000","last_active":"2023-12-01T03:11:50.579811+0000","last_peered":"2023-12-01T03:11:50.579811+0000","last_clean":"2023-12-01T03:11:50.579811+0000","last_became_active":"2023-12-01T03:11:32.202452+0000","last_became_peered":"2023-12-01T03:11:32.202452+0000","last_unstale":"2023-12-01T03:11:50.579811+0000","last_undegraded":"2023-12-01T03:11:50.579811+0000","last_fullsized":"2023-12-01T03:11:50.579811+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.067508+0000","last_change":"2023-12-01T03:11:51.064149+0000","last_active":"2023-12-01T03:11:51.067508+0000","last_peered":"2023-12-01T03:11:51.067508+0000","last_clean":"2023-12-01T03:11:51.067508+0000","last_became_active":"2023-12-01T03:11:51.063847+0000","last_became_peered":"2023-12-01T03:11:51.063847+0000","last_unstale":"2023-12-01T03:11:51.067508+0000","last_undegraded":"2023-12-01T03:11:51.067508+0000","last_fullsized":"2023-12-01T03:11:51.067508+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:33:54.697716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581309+0000","last_change":"2023-12-01T03:11:32.224888+0000","last_active":"2023-12-01T03:11:50.581309+0000","last_peered":"2023-12-01T03:11:50.581309+0000","last_clean":"2023-12-01T03:11:50.581309+0000","last_became_active":"2023-12-01T03:11:32.224585+0000","last_became_peered":"2023-12-01T03:11:32.224585+0000","last_unstale":"2023-12-01T03:11:50.581309+0000","last_undegraded":"2023-12-01T03:11:50.581309+0000","last_fullsized":"2023-12-01T03:11:50.581309+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:10:04.079396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068457+0000","last_change":"2023-12-01T03:11:51.066331+0000","last_active":"2023-12-01T03:11:51.068457+0000","last_peered":"2023-12-01T03:11:51.068457+0000","last_clean":"2023-12-01T03:11:51.068457+0000","last_became_active":"2023-12-01T03:11:51.065335+0000","last_became_peered":"2023-12-01T03:11:51.065335+0000","last_unstale":"2023-12-01T03:11:51.068457+0000","last_undegraded":"2023-12-01T03:11:51.068457+0000","last_fullsized":"2023-12-01T03:11:51.068457+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:14.769992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.570401+0000","last_change":"2023-12-01T03:11:32.208938+0000","last_active":"2023-12-01T03:11:50.570401+0000","last_peered":"2023-12-01T03:11:50.570401+0000","last_clean":"2023-12-01T03:11:50.570401+0000","last_became_active":"2023-12-01T03:11:32.208695+0000","last_became_peered":"2023-12-01T03:11:32.208695+0000","last_unstale":"2023-12-01T03:11:50.570401+0000","last_undegraded":"2023-12-01T03:11:50.570401+0000","last_fullsized":"2023-12-01T03:11:50.570401+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:21.336618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.071601+0000","last_change":"2023-12-01T03:11:51.069976+0000","last_active":"2023-12-01T03:11:51.071601+0000","last_peered":"2023-12-01T03:11:51.071601+0000","last_clean":"2023-12-01T03:11:51.071601+0000","last_became_active":"2023-12-01T03:11:51.069153+0000","last_became_peered":"2023-12-01T03:11:51.069153+0000","last_unstale":"2023-12-01T03:11:51.071601+0000","last_undegraded":"2023-12-01T03:11:51.071601+0000","last_fullsized":"2023-12-01T03:11:51.071601+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:54:02.088965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583726+0000","last_change":"2023-12-01T03:11:32.214557+0000","last_active":"2023-12-01T03:11:50.583726+0000","last_peered":"2023-12-01T03:11:50.583726+0000","last_clean":"2023-12-01T03:11:50.583726+0000","last_became_active":"2023-12-01T03:11:32.214252+0000","last_became_peered":"2023-12-01T03:11:32.214252+0000","last_unstale":"2023-12-01T03:11:50.583726+0000","last_undegraded":"2023-12-01T03:11:50.583726+0000","last_fullsized":"2023-12-01T03:11:50.583726+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088066+0000","last_change":"2023-12-01T03:11:51.076162+0000","last_active":"2023-12-01T03:11:51.088066+0000","last_peered":"2023-12-01T03:11:51.088066+0000","last_clean":"2023-12-01T03:11:51.088066+0000","last_became_active":"2023-12-01T03:11:51.075899+0000","last_became_peered":"2023-12-01T03:11:51.075899+0000","last_unstale":"2023-12-01T03:11:51.088066+0000","last_undegraded":"2023-12-01T03:11:51.088066+0000","last_fullsized":"2023-12-01T03:11:51.088066+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:26:32.163371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.590521+0000","last_change":"2023-12-01T03:11:32.212018+0000","last_active":"2023-12-01T03:11:50.590521+0000","last_peered":"2023-12-01T03:11:50.590521+0000","last_clean":"2023-12-01T03:11:50.590521+0000","last_became_active":"2023-12-01T03:11:32.211742+0000","last_became_peered":"2023-12-01T03:11:32.211742+0000","last_unstale":"2023-12-01T03:11:50.590521+0000","last_undegraded":"2023-12-01T03:11:50.590521+0000","last_fullsized":"2023-12-01T03:11:50.590521+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070608+0000","last_change":"2023-12-01T03:11:51.069517+0000","last_active":"2023-12-01T03:11:51.070608+0000","last_peered":"2023-12-01T03:11:51.070608+0000","last_clean":"2023-12-01T03:11:51.070608+0000","last_became_active":"2023-12-01T03:11:51.068565+0000","last_became_peered":"2023-12-01T03:11:51.068565+0000","last_unstale":"2023-12-01T03:11:51.070608+0000","last_undegraded":"2023-12-01T03:11:51.070608+0000","last_fullsized":"2023-12-01T03:11:51.070608+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:33.446505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582658+0000","last_change":"2023-12-01T03:11:32.219580+0000","last_active":"2023-12-01T03:11:50.582658+0000","last_peered":"2023-12-01T03:11:50.582658+0000","last_clean":"2023-12-01T03:11:50.582658+0000","last_became_active":"2023-12-01T03:11:32.218935+0000","last_became_peered":"2023-12-01T03:11:32.218935+0000","last_unstale":"2023-12-01T03:11:50.582658+0000","last_undegraded":"2023-12-01T03:11:50.582658+0000","last_fullsized":"2023-12-01T03:11:50.582658+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:45.903635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.066372+0000","last_change":"2023-12-01T03:11:51.065776+0000","last_active":"2023-12-01T03:11:51.066372+0000","last_peered":"2023-12-01T03:11:51.066372+0000","last_clean":"2023-12-01T03:11:51.066372+0000","last_became_active":"2023-12-01T03:11:51.065512+0000","last_became_peered":"2023-12-01T03:11:51.065512+0000","last_unstale":"2023-12-01T03:11:51.066372+0000","last_undegraded":"2023-12-01T03:11:51.066372+0000","last_fullsized":"2023-12-01T03:11:51.066372+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:02:55.175453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581169+0000","last_change":"2023-12-01T03:11:32.222587+0000","last_active":"2023-12-01T03:11:50.581169+0000","last_peered":"2023-12-01T03:11:50.581169+0000","last_clean":"2023-12-01T03:11:50.581169+0000","last_became_active":"2023-12-01T03:11:32.222267+0000","last_became_peered":"2023-12-01T03:11:32.222267+0000","last_unstale":"2023-12-01T03:11:50.581169+0000","last_undegraded":"2023-12-01T03:11:50.581169+0000","last_fullsized":"2023-12-01T03:11:50.581169+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:41:42.720564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.067007+0000","last_change":"2023-12-01T03:11:51.066093+0000","last_active":"2023-12-01T03:11:51.067007+0000","last_peered":"2023-12-01T03:11:51.067007+0000","last_clean":"2023-12-01T03:11:51.067007+0000","last_became_active":"2023-12-01T03:11:51.065673+0000","last_became_peered":"2023-12-01T03:11:51.065673+0000","last_unstale":"2023-12-01T03:11:51.067007+0000","last_undegraded":"2023-12-01T03:11:51.067007+0000","last_fullsized":"2023-12-01T03:11:51.067007+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:00:31.691096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.587613+0000","last_change":"2023-12-01T03:11:32.204026+0000","last_active":"2023-12-01T03:11:50.587613+0000","last_peered":"2023-12-01T03:11:50.587613+0000","last_clean":"2023-12-01T03:11:50.587613+0000","last_became_active":"2023-12-01T03:11:32.200832+0000","last_became_peered":"2023-12-01T03:11:32.200832+0000","last_unstale":"2023-12-01T03:11:50.587613+0000","last_undegraded":"2023-12-01T03:11:50.587613+0000","last_fullsized":"2023-12-01T03:11:50.587613+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078852+0000","last_change":"2023-12-01T03:11:51.068191+0000","last_active":"2023-12-01T03:11:51.078852+0000","last_peered":"2023-12-01T03:11:51.078852+0000","last_clean":"2023-12-01T03:11:51.078852+0000","last_became_active":"2023-12-01T03:11:51.067752+0000","last_became_peered":"2023-12-01T03:11:51.067752+0000","last_unstale":"2023-12-01T03:11:51.078852+0000","last_undegraded":"2023-12-01T03:11:51.078852+0000","last_fullsized":"2023-12-01T03:11:51.078852+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:54:02.088965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.588717+0000","last_change":"2023-12-01T03:11:32.207642+0000","last_active":"2023-12-01T03:11:50.588717+0000","last_peered":"2023-12-01T03:11:50.588717+0000","last_clean":"2023-12-01T03:11:50.588717+0000","last_became_active":"2023-12-01T03:11:32.207344+0000","last_became_peered":"2023-12-01T03:11:32.207344+0000","last_unstale":"2023-12-01T03:11:50.588717+0000","last_undegraded":"2023-12-01T03:11:50.588717+0000","last_fullsized":"2023-12-01T03:11:50.588717+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:10.441679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.066428+0000","last_change":"2023-12-01T03:11:51.062269+0000","last_active":"2023-12-01T03:11:51.066428+0000","last_peered":"2023-12-01T03:11:51.066428+0000","last_clean":"2023-12-01T03:11:51.066428+0000","last_became_active":"2023-12-01T03:11:51.061924+0000","last_became_peered":"2023-12-01T03:11:51.061924+0000","last_unstale":"2023-12-01T03:11:51.066428+0000","last_undegraded":"2023-12-01T03:11:51.066428+0000","last_fullsized":"2023-12-01T03:11:51.066428+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:01.630559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.580945+0000","last_change":"2023-12-01T03:11:32.220696+0000","last_active":"2023-12-01T03:11:50.580945+0000","last_peered":"2023-12-01T03:11:50.580945+0000","last_clean":"2023-12-01T03:11:50.580945+0000","last_became_active":"2023-12-01T03:11:32.220353+0000","last_became_peered":"2023-12-01T03:11:32.220353+0000","last_unstale":"2023-12-01T03:11:50.580945+0000","last_undegraded":"2023-12-01T03:11:50.580945+0000","last_fullsized":"2023-12-01T03:11:50.580945+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:53:43.178969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088991+0000","last_change":"2023-12-01T03:11:51.080854+0000","last_active":"2023-12-01T03:11:51.088991+0000","last_peered":"2023-12-01T03:11:51.088991+0000","last_clean":"2023-12-01T03:11:51.088991+0000","last_became_active":"2023-12-01T03:11:51.080581+0000","last_became_peered":"2023-12-01T03:11:51.080581+0000","last_unstale":"2023-12-01T03:11:51.088991+0000","last_undegraded":"2023-12-01T03:11:51.088991+0000","last_fullsized":"2023-12-01T03:11:51.088991+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:57:38.081054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.586520+0000","last_change":"2023-12-01T03:11:32.207709+0000","last_active":"2023-12-01T03:11:50.586520+0000","last_peered":"2023-12-01T03:11:50.586520+0000","last_clean":"2023-12-01T03:11:50.586520+0000","last_became_active":"2023-12-01T03:11:32.205510+0000","last_became_peered":"2023-12-01T03:11:32.205510+0000","last_unstale":"2023-12-01T03:11:50.586520+0000","last_undegraded":"2023-12-01T03:11:50.586520+0000","last_fullsized":"2023-12-01T03:11:50.586520+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070463+0000","last_change":"2023-12-01T03:11:51.067527+0000","last_active":"2023-12-01T03:11:51.070463+0000","last_peered":"2023-12-01T03:11:51.070463+0000","last_clean":"2023-12-01T03:11:51.070463+0000","last_became_active":"2023-12-01T03:11:51.064895+0000","last_became_peered":"2023-12-01T03:11:51.064895+0000","last_unstale":"2023-12-01T03:11:51.070463+0000","last_undegraded":"2023-12-01T03:11:51.070463+0000","last_fullsized":"2023-12-01T03:11:51.070463+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:54:02.088965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.578605+0000","last_change":"2023-12-01T03:11:32.210643+0000","last_active":"2023-12-01T03:11:50.578605+0000","last_peered":"2023-12-01T03:11:50.578605+0000","last_clean":"2023-12-01T03:11:50.578605+0000","last_became_active":"2023-12-01T03:11:32.210367+0000","last_became_peered":"2023-12-01T03:11:32.210367+0000","last_unstale":"2023-12-01T03:11:50.578605+0000","last_undegraded":"2023-12-01T03:11:50.578605+0000","last_fullsized":"2023-12-01T03:11:50.578605+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:14:00.674458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.085719+0000","last_change":"2023-12-01T03:11:51.085229+0000","last_active":"2023-12-01T03:11:51.085719+0000","last_peered":"2023-12-01T03:11:51.085719+0000","last_clean":"2023-12-01T03:11:51.085719+0000","last_became_active":"2023-12-01T03:11:51.084930+0000","last_became_peered":"2023-12-01T03:11:51.084930+0000","last_unstale":"2023-12-01T03:11:51.085719+0000","last_undegraded":"2023-12-01T03:11:51.085719+0000","last_fullsized":"2023-12-01T03:11:51.085719+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:05:22.145638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582376+0000","last_change":"2023-12-01T03:11:32.209242+0000","last_active":"2023-12-01T03:11:50.582376+0000","last_peered":"2023-12-01T03:11:50.582376+0000","last_clean":"2023-12-01T03:11:50.582376+0000","last_became_active":"2023-12-01T03:11:32.208985+0000","last_became_peered":"2023-12-01T03:11:32.208985+0000","last_unstale":"2023-12-01T03:11:50.582376+0000","last_undegraded":"2023-12-01T03:11:50.582376+0000","last_fullsized":"2023-12-01T03:11:50.582376+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:29.661974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078326+0000","last_change":"2023-12-01T03:11:51.071579+0000","last_active":"2023-12-01T03:11:51.078326+0000","last_peered":"2023-12-01T03:11:51.078326+0000","last_clean":"2023-12-01T03:11:51.078326+0000","last_became_active":"2023-12-01T03:11:51.071042+0000","last_became_peered":"2023-12-01T03:11:51.071042+0000","last_unstale":"2023-12-01T03:11:51.078326+0000","last_undegraded":"2023-12-01T03:11:51.078326+0000","last_fullsized":"2023-12-01T03:11:51.078326+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:54:09.453554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569921+0000","last_change":"2023-12-01T03:11:32.220828+0000","last_active":"2023-12-01T03:11:50.569921+0000","last_peered":"2023-12-01T03:11:50.569921+0000","last_clean":"2023-12-01T03:11:50.569921+0000","last_became_active":"2023-12-01T03:11:32.219607+0000","last_became_peered":"2023-12-01T03:11:32.219607+0000","last_unstale":"2023-12-01T03:11:50.569921+0000","last_undegraded":"2023-12-01T03:11:50.569921+0000","last_fullsized":"2023-12-01T03:11:50.569921+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:09.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.087817+0000","last_change":"2023-12-01T03:11:51.074146+0000","last_active":"2023-12-01T03:11:51.087817+0000","last_peered":"2023-12-01T03:11:51.087817+0000","last_clean":"2023-12-01T03:11:51.087817+0000","last_became_active":"2023-12-01T03:11:51.073887+0000","last_became_peered":"2023-12-01T03:11:51.073887+0000","last_unstale":"2023-12-01T03:11:51.087817+0000","last_undegraded":"2023-12-01T03:11:51.087817+0000","last_fullsized":"2023-12-01T03:11:51.087817+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:49.251059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.591470+0000","last_change":"2023-12-01T03:11:32.198609+0000","last_active":"2023-12-01T03:11:50.591470+0000","last_peered":"2023-12-01T03:11:50.591470+0000","last_clean":"2023-12-01T03:11:50.591470+0000","last_became_active":"2023-12-01T03:11:32.197869+0000","last_became_peered":"2023-12-01T03:11:32.197869+0000","last_unstale":"2023-12-01T03:11:50.591470+0000","last_undegraded":"2023-12-01T03:11:50.591470+0000","last_fullsized":"2023-12-01T03:11:50.591470+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:35.975600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088036+0000","last_change":"2023-12-01T03:11:51.075313+0000","last_active":"2023-12-01T03:11:51.088036+0000","last_peered":"2023-12-01T03:11:51.088036+0000","last_clean":"2023-12-01T03:11:51.088036+0000","last_became_active":"2023-12-01T03:11:51.072486+0000","last_became_peered":"2023-12-01T03:11:51.072486+0000","last_unstale":"2023-12-01T03:11:51.088036+0000","last_undegraded":"2023-12-01T03:11:51.088036+0000","last_fullsized":"2023-12-01T03:11:51.088036+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:02:55.175453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569456+0000","last_change":"2023-12-01T03:11:32.212954+0000","last_active":"2023-12-01T03:11:50.569456+0000","last_peered":"2023-12-01T03:11:50.569456+0000","last_clean":"2023-12-01T03:11:50.569456+0000","last_became_active":"2023-12-01T03:11:32.212678+0000","last_became_peered":"2023-12-01T03:11:32.212678+0000","last_unstale":"2023-12-01T03:11:50.569456+0000","last_undegraded":"2023-12-01T03:11:50.569456+0000","last_fullsized":"2023-12-01T03:11:50.569456+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:28.852429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068257+0000","last_change":"2023-12-01T03:11:51.067112+0000","last_active":"2023-12-01T03:11:51.068257+0000","last_peered":"2023-12-01T03:11:51.068257+0000","last_clean":"2023-12-01T03:11:51.068257+0000","last_became_active":"2023-12-01T03:11:51.066722+0000","last_became_peered":"2023-12-01T03:11:51.066722+0000","last_unstale":"2023-12-01T03:11:51.068257+0000","last_undegraded":"2023-12-01T03:11:51.068257+0000","last_fullsized":"2023-12-01T03:11:51.068257+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:07.114272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.567075+0000","last_change":"2023-12-01T03:11:32.215698+0000","last_active":"2023-12-01T03:11:50.567075+0000","last_peered":"2023-12-01T03:11:50.567075+0000","last_clean":"2023-12-01T03:11:50.567075+0000","last_became_active":"2023-12-01T03:11:32.214531+0000","last_became_peered":"2023-12-01T03:11:32.214531+0000","last_unstale":"2023-12-01T03:11:50.567075+0000","last_undegraded":"2023-12-01T03:11:50.567075+0000","last_fullsized":"2023-12-01T03:11:50.567075+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:14.716448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088453+0000","last_change":"2023-12-01T03:11:51.078917+0000","last_active":"2023-12-01T03:11:51.088453+0000","last_peered":"2023-12-01T03:11:51.088453+0000","last_clean":"2023-12-01T03:11:51.088453+0000","last_became_active":"2023-12-01T03:11:51.078182+0000","last_became_peered":"2023-12-01T03:11:51.078182+0000","last_unstale":"2023-12-01T03:11:51.088453+0000","last_undegraded":"2023-12-01T03:11:51.088453+0000","last_fullsized":"2023-12-01T03:11:51.088453+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:38.991634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.593671+0000","last_change":"2023-12-01T03:11:32.204327+0000","last_active":"2023-12-01T03:11:50.593671+0000","last_peered":"2023-12-01T03:11:50.593671+0000","last_clean":"2023-12-01T03:11:50.593671+0000","last_became_active":"2023-12-01T03:11:32.203765+0000","last_became_peered":"2023-12-01T03:11:32.203765+0000","last_unstale":"2023-12-01T03:11:50.593671+0000","last_undegraded":"2023-12-01T03:11:50.593671+0000","last_fullsized":"2023-12-01T03:11:50.593671+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:28:34.324085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.086910+0000","last_change":"2023-12-01T03:11:51.085732+0000","last_active":"2023-12-01T03:11:51.086910+0000","last_peered":"2023-12-01T03:11:51.086910+0000","last_clean":"2023-12-01T03:11:51.086910+0000","last_became_active":"2023-12-01T03:11:51.083582+0000","last_became_peered":"2023-12-01T03:11:51.083582+0000","last_unstale":"2023-12-01T03:11:51.086910+0000","last_undegraded":"2023-12-01T03:11:51.086910+0000","last_fullsized":"2023-12-01T03:11:51.086910+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:26:32.163371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.577680+0000","last_change":"2023-12-01T03:11:32.212023+0000","last_active":"2023-12-01T03:11:50.577680+0000","last_peered":"2023-12-01T03:11:50.577680+0000","last_clean":"2023-12-01T03:11:50.577680+0000","last_became_active":"2023-12-01T03:11:32.210975+0000","last_became_peered":"2023-12-01T03:11:32.210975+0000","last_unstale":"2023-12-01T03:11:50.577680+0000","last_undegraded":"2023-12-01T03:11:50.577680+0000","last_fullsized":"2023-12-01T03:11:50.577680+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:13:01.202192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089326+0000","last_change":"2023-12-01T03:11:51.084960+0000","last_active":"2023-12-01T03:11:51.089326+0000","last_peered":"2023-12-01T03:11:51.089326+0000","last_clean":"2023-12-01T03:11:51.089326+0000","last_became_active":"2023-12-01T03:11:51.084228+0000","last_became_peered":"2023-12-01T03:11:51.084228+0000","last_unstale":"2023-12-01T03:11:51.089326+0000","last_undegraded":"2023-12-01T03:11:51.089326+0000","last_fullsized":"2023-12-01T03:11:51.089326+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:09:43.235099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.592399+0000","last_change":"2023-12-01T03:11:32.213074+0000","last_active":"2023-12-01T03:11:50.592399+0000","last_peered":"2023-12-01T03:11:50.592399+0000","last_clean":"2023-12-01T03:11:50.592399+0000","last_became_active":"2023-12-01T03:11:32.212775+0000","last_became_peered":"2023-12-01T03:11:32.212775+0000","last_unstale":"2023-12-01T03:11:50.592399+0000","last_undegraded":"2023-12-01T03:11:50.592399+0000","last_fullsized":"2023-12-01T03:11:50.592399+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:36.571608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070822+0000","last_change":"2023-12-01T03:11:51.069719+0000","last_active":"2023-12-01T03:11:51.070822+0000","last_peered":"2023-12-01T03:11:51.070822+0000","last_clean":"2023-12-01T03:11:51.070822+0000","last_became_active":"2023-12-01T03:11:51.069436+0000","last_became_peered":"2023-12-01T03:11:51.069436+0000","last_unstale":"2023-12-01T03:11:51.070822+0000","last_undegraded":"2023-12-01T03:11:51.070822+0000","last_fullsized":"2023-12-01T03:11:51.070822+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:07.114272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.564062+0000","last_change":"2023-12-01T03:11:32.225479+0000","last_active":"2023-12-01T03:11:50.564062+0000","last_peered":"2023-12-01T03:11:50.564062+0000","last_clean":"2023-12-01T03:11:50.564062+0000","last_became_active":"2023-12-01T03:11:32.225154+0000","last_became_peered":"2023-12-01T03:11:32.225154+0000","last_unstale":"2023-12-01T03:11:50.564062+0000","last_undegraded":"2023-12-01T03:11:50.564062+0000","last_fullsized":"2023-12-01T03:11:50.564062+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:14.760284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088235+0000","last_change":"2023-12-01T03:11:51.076925+0000","last_active":"2023-12-01T03:11:51.088235+0000","last_peered":"2023-12-01T03:11:51.088235+0000","last_clean":"2023-12-01T03:11:51.088235+0000","last_became_active":"2023-12-01T03:11:51.076518+0000","last_became_peered":"2023-12-01T03:11:51.076518+0000","last_unstale":"2023-12-01T03:11:51.088235+0000","last_undegraded":"2023-12-01T03:11:51.088235+0000","last_fullsized":"2023-12-01T03:11:51.088235+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:14.769992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1d","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089447+0000","last_change":"2023-12-01T03:11:50.599295+0000","last_active":"2023-12-01T03:11:51.089447+0000","last_peered":"2023-12-01T03:11:51.089447+0000","last_clean":"2023-12-01T03:11:51.089447+0000","last_became_active":"2023-12-01T03:11:50.598962+0000","last_became_peered":"2023-12-01T03:11:50.598962+0000","last_unstale":"2023-12-01T03:11:51.089447+0000","last_undegraded":"2023-12-01T03:11:51.089447+0000","last_fullsized":"2023-12-01T03:11:51.089447+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:11:14.566384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.569753+0000","last_change":"2023-12-01T03:11:32.214972+0000","last_active":"2023-12-01T03:11:50.569753+0000","last_peered":"2023-12-01T03:11:50.569753+0000","last_clean":"2023-12-01T03:11:50.569753+0000","last_became_active":"2023-12-01T03:11:32.214729+0000","last_became_peered":"2023-12-01T03:11:32.214729+0000","last_unstale":"2023-12-01T03:11:50.569753+0000","last_undegraded":"2023-12-01T03:11:50.569753+0000","last_fullsized":"2023-12-01T03:11:50.569753+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:24.244094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1f","version":"234'4","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.590253+0000","last_change":"2023-12-01T03:11:31.680864+0000","last_active":"2023-12-01T03:11:50.590253+0000","last_peered":"2023-12-01T03:11:50.590253+0000","last_clean":"2023-12-01T03:11:50.590253+0000","last_became_active":"2023-12-01T03:11:31.680279+0000","last_became_peered":"2023-12-01T03:11:31.680279+0000","last_unstale":"2023-12-01T03:11:50.590253+0000","last_undegraded":"2023-12-01T03:11:50.590253+0000","last_fullsized":"2023-12-01T03:11:50.590253+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.377359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070624+0000","last_change":"2023-12-01T03:11:51.069693+0000","last_active":"2023-12-01T03:11:51.070624+0000","last_peered":"2023-12-01T03:11:51.070624+0000","last_clean":"2023-12-01T03:11:51.070624+0000","last_became_active":"2023-12-01T03:11:51.069280+0000","last_became_peered":"2023-12-01T03:11:51.069280+0000","last_unstale":"2023-12-01T03:11:51.070624+0000","last_undegraded":"2023-12-01T03:11:51.070624+0000","last_fullsized":"2023-12-01T03:11:51.070624+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:29.351674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1c","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078893+0000","last_change":"2023-12-01T03:11:50.585796+0000","last_active":"2023-12-01T03:11:51.078893+0000","last_peered":"2023-12-01T03:11:51.078893+0000","last_clean":"2023-12-01T03:11:51.078893+0000","last_became_active":"2023-12-01T03:11:50.585488+0000","last_became_peered":"2023-12-01T03:11:50.585488+0000","last_unstale":"2023-12-01T03:11:51.078893+0000","last_undegraded":"2023-12-01T03:11:51.078893+0000","last_fullsized":"2023-12-01T03:11:51.078893+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:45:00.247816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.590136+0000","last_change":"2023-12-01T03:11:32.211743+0000","last_active":"2023-12-01T03:11:50.590136+0000","last_peered":"2023-12-01T03:11:50.590136+0000","last_clean":"2023-12-01T03:11:50.590136+0000","last_became_active":"2023-12-01T03:11:32.211443+0000","last_became_peered":"2023-12-01T03:11:32.211443+0000","last_unstale":"2023-12-01T03:11:50.590136+0000","last_undegraded":"2023-12-01T03:11:50.590136+0000","last_fullsized":"2023-12-01T03:11:50.590136+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"234'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583824+0000","last_change":"2023-12-01T03:11:31.679798+0000","last_active":"2023-12-01T03:11:50.583824+0000","last_peered":"2023-12-01T03:11:50.583824+0000","last_clean":"2023-12-01T03:11:50.583824+0000","last_became_active":"2023-12-01T03:11:31.679030+0000","last_became_peered":"2023-12-01T03:11:31.679030+0000","last_unstale":"2023-12-01T03:11:50.583824+0000","last_undegraded":"2023-12-01T03:11:50.583824+0000","last_fullsized":"2023-12-01T03:11:50.583824+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:55:49.261314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.080966+0000","last_change":"2023-12-01T03:11:51.076089+0000","last_active":"2023-12-01T03:11:51.080966+0000","last_peered":"2023-12-01T03:11:51.080966+0000","last_clean":"2023-12-01T03:11:51.080966+0000","last_became_active":"2023-12-01T03:11:51.071097+0000","last_became_peered":"2023-12-01T03:11:51.071097+0000","last_unstale":"2023-12-01T03:11:51.080966+0000","last_undegraded":"2023-12-01T03:11:51.080966+0000","last_fullsized":"2023-12-01T03:11:51.080966+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:07.114272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.074468+0000","last_change":"2023-12-01T03:11:50.585199+0000","last_active":"2023-12-01T03:11:51.074468+0000","last_peered":"2023-12-01T03:11:51.074468+0000","last_clean":"2023-12-01T03:11:51.074468+0000","last_became_active":"2023-12-01T03:11:50.584909+0000","last_became_peered":"2023-12-01T03:11:50.584909+0000","last_unstale":"2023-12-01T03:11:51.074468+0000","last_undegraded":"2023-12-01T03:11:51.074468+0000","last_fullsized":"2023-12-01T03:11:51.074468+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:05:21.126535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.592447+0000","last_change":"2023-12-01T03:11:32.222760+0000","last_active":"2023-12-01T03:11:50.592447+0000","last_peered":"2023-12-01T03:11:50.592447+0000","last_clean":"2023-12-01T03:11:50.592447+0000","last_became_active":"2023-12-01T03:11:32.222500+0000","last_became_peered":"2023-12-01T03:11:32.222500+0000","last_unstale":"2023-12-01T03:11:50.592447+0000","last_undegraded":"2023-12-01T03:11:50.592447+0000","last_fullsized":"2023-12-01T03:11:50.592447+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:17.670345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.580065+0000","last_change":"2023-12-01T03:11:31.663745+0000","last_active":"2023-12-01T03:11:50.580065+0000","last_peered":"2023-12-01T03:11:50.580065+0000","last_clean":"2023-12-01T03:11:50.580065+0000","last_became_active":"2023-12-01T03:11:31.662706+0000","last_became_peered":"2023-12-01T03:11:31.662706+0000","last_unstale":"2023-12-01T03:11:50.580065+0000","last_undegraded":"2023-12-01T03:11:50.580065+0000","last_fullsized":"2023-12-01T03:11:50.580065+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:41:42.945449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.087580+0000","last_change":"2023-12-01T03:11:51.072177+0000","last_active":"2023-12-01T03:11:51.087580+0000","last_peered":"2023-12-01T03:11:51.087580+0000","last_clean":"2023-12-01T03:11:51.087580+0000","last_became_active":"2023-12-01T03:11:51.071919+0000","last_became_peered":"2023-12-01T03:11:51.071919+0000","last_unstale":"2023-12-01T03:11:51.087580+0000","last_undegraded":"2023-12-01T03:11:51.087580+0000","last_fullsized":"2023-12-01T03:11:51.087580+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:54:09.453554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.055288+0000","last_change":"2023-12-01T03:11:50.594435+0000","last_active":"2023-12-01T03:11:51.055288+0000","last_peered":"2023-12-01T03:11:51.055288+0000","last_clean":"2023-12-01T03:11:51.055288+0000","last_became_active":"2023-12-01T03:11:50.594170+0000","last_became_peered":"2023-12-01T03:11:50.594170+0000","last_unstale":"2023-12-01T03:11:51.055288+0000","last_undegraded":"2023-12-01T03:11:51.055288+0000","last_fullsized":"2023-12-01T03:11:51.055288+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:34.274531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.570081+0000","last_change":"2023-12-01T03:11:32.213316+0000","last_active":"2023-12-01T03:11:50.570081+0000","last_peered":"2023-12-01T03:11:50.570081+0000","last_clean":"2023-12-01T03:11:50.570081+0000","last_became_active":"2023-12-01T03:11:32.213070+0000","last_became_peered":"2023-12-01T03:11:32.213070+0000","last_unstale":"2023-12-01T03:11:50.570081+0000","last_undegraded":"2023-12-01T03:11:50.570081+0000","last_fullsized":"2023-12-01T03:11:50.570081+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:22:51.687377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"234'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583999+0000","last_change":"2023-12-01T03:11:31.660758+0000","last_active":"2023-12-01T03:11:50.583999+0000","last_peered":"2023-12-01T03:11:50.583999+0000","last_clean":"2023-12-01T03:11:50.583999+0000","last_became_active":"2023-12-01T03:11:31.660001+0000","last_became_peered":"2023-12-01T03:11:31.660001+0000","last_unstale":"2023-12-01T03:11:50.583999+0000","last_undegraded":"2023-12-01T03:11:50.583999+0000","last_fullsized":"2023-12-01T03:11:50.583999+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088994+0000","last_change":"2023-12-01T03:11:51.080585+0000","last_active":"2023-12-01T03:11:51.088994+0000","last_peered":"2023-12-01T03:11:51.088994+0000","last_clean":"2023-12-01T03:11:51.088994+0000","last_became_active":"2023-12-01T03:11:51.079004+0000","last_became_peered":"2023-12-01T03:11:51.079004+0000","last_unstale":"2023-12-01T03:11:51.088994+0000","last_undegraded":"2023-12-01T03:11:51.088994+0000","last_fullsized":"2023-12-01T03:11:51.088994+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:57:49.158244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.055065+0000","last_change":"2023-12-01T03:11:50.588931+0000","last_active":"2023-12-01T03:11:51.055065+0000","last_peered":"2023-12-01T03:11:51.055065+0000","last_clean":"2023-12-01T03:11:51.055065+0000","last_became_active":"2023-12-01T03:11:50.588657+0000","last_became_peered":"2023-12-01T03:11:50.588657+0000","last_unstale":"2023-12-01T03:11:51.055065+0000","last_undegraded":"2023-12-01T03:11:51.055065+0000","last_fullsized":"2023-12-01T03:11:51.055065+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:35.737819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.588967+0000","last_change":"2023-12-01T03:11:32.213961+0000","last_active":"2023-12-01T03:11:50.588967+0000","last_peered":"2023-12-01T03:11:50.588967+0000","last_clean":"2023-12-01T03:11:50.588967+0000","last_became_active":"2023-12-01T03:11:32.213692+0000","last_became_peered":"2023-12-01T03:11:32.213692+0000","last_unstale":"2023-12-01T03:11:50.588967+0000","last_undegraded":"2023-12-01T03:11:50.588967+0000","last_fullsized":"2023-12-01T03:11:50.588967+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"234'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.580111+0000","last_change":"2023-12-01T03:11:31.681531+0000","last_active":"2023-12-01T03:11:50.580111+0000","last_peered":"2023-12-01T03:11:50.580111+0000","last_clean":"2023-12-01T03:11:50.580111+0000","last_became_active":"2023-12-01T03:11:31.680782+0000","last_became_peered":"2023-12-01T03:11:31.680782+0000","last_unstale":"2023-12-01T03:11:50.580111+0000","last_undegraded":"2023-12-01T03:11:50.580111+0000","last_fullsized":"2023-12-01T03:11:50.580111+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:18.018055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.065085+0000","last_change":"2023-12-01T03:11:51.058395+0000","last_active":"2023-12-01T03:11:51.065085+0000","last_peered":"2023-12-01T03:11:51.065085+0000","last_clean":"2023-12-01T03:11:51.065085+0000","last_became_active":"2023-12-01T03:11:51.058097+0000","last_became_peered":"2023-12-01T03:11:51.058097+0000","last_unstale":"2023-12-01T03:11:51.065085+0000","last_undegraded":"2023-12-01T03:11:51.065085+0000","last_fullsized":"2023-12-01T03:11:51.065085+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:59:29.351674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1b","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078692+0000","last_change":"2023-12-01T03:11:50.585680+0000","last_active":"2023-12-01T03:11:51.078692+0000","last_peered":"2023-12-01T03:11:51.078692+0000","last_clean":"2023-12-01T03:11:51.078692+0000","last_became_active":"2023-12-01T03:11:50.585097+0000","last_became_peered":"2023-12-01T03:11:50.585097+0000","last_unstale":"2023-12-01T03:11:51.078692+0000","last_undegraded":"2023-12-01T03:11:51.078692+0000","last_fullsized":"2023-12-01T03:11:51.078692+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:20.001579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582405+0000","last_change":"2023-12-01T03:11:32.206659+0000","last_active":"2023-12-01T03:11:50.582405+0000","last_peered":"2023-12-01T03:11:50.582405+0000","last_clean":"2023-12-01T03:11:50.582405+0000","last_became_active":"2023-12-01T03:11:32.206346+0000","last_became_peered":"2023-12-01T03:11:32.206346+0000","last_unstale":"2023-12-01T03:11:50.582405+0000","last_undegraded":"2023-12-01T03:11:50.582405+0000","last_fullsized":"2023-12-01T03:11:50.582405+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:01.902693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.591287+0000","last_change":"2023-12-01T03:11:31.663166+0000","last_active":"2023-12-01T03:11:50.591287+0000","last_peered":"2023-12-01T03:11:50.591287+0000","last_clean":"2023-12-01T03:11:50.591287+0000","last_became_active":"2023-12-01T03:11:31.662263+0000","last_became_peered":"2023-12-01T03:11:31.662263+0000","last_unstale":"2023-12-01T03:11:50.591287+0000","last_undegraded":"2023-12-01T03:11:50.591287+0000","last_fullsized":"2023-12-01T03:11:50.591287+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:57:00.965499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.080276+0000","last_change":"2023-12-01T03:11:51.079275+0000","last_active":"2023-12-01T03:11:51.080276+0000","last_peered":"2023-12-01T03:11:51.080276+0000","last_clean":"2023-12-01T03:11:51.080276+0000","last_became_active":"2023-12-01T03:11:51.078910+0000","last_became_peered":"2023-12-01T03:11:51.078910+0000","last_unstale":"2023-12-01T03:11:51.080276+0000","last_undegraded":"2023-12-01T03:11:51.080276+0000","last_fullsized":"2023-12-01T03:11:51.080276+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:01.612426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1a","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.080017+0000","last_change":"2023-12-01T03:11:50.585007+0000","last_active":"2023-12-01T03:11:51.080017+0000","last_peered":"2023-12-01T03:11:51.080017+0000","last_clean":"2023-12-01T03:11:51.080017+0000","last_became_active":"2023-12-01T03:11:50.576065+0000","last_became_peered":"2023-12-01T03:11:50.576065+0000","last_unstale":"2023-12-01T03:11:51.080017+0000","last_undegraded":"2023-12-01T03:11:51.080017+0000","last_fullsized":"2023-12-01T03:11:51.080017+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:04.781574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.578742+0000","last_change":"2023-12-01T03:11:32.210988+0000","last_active":"2023-12-01T03:11:50.578742+0000","last_peered":"2023-12-01T03:11:50.578742+0000","last_clean":"2023-12-01T03:11:50.578742+0000","last_became_active":"2023-12-01T03:11:32.207844+0000","last_became_peered":"2023-12-01T03:11:32.207844+0000","last_unstale":"2023-12-01T03:11:50.578742+0000","last_undegraded":"2023-12-01T03:11:50.578742+0000","last_fullsized":"2023-12-01T03:11:50.578742+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.587802+0000","last_change":"2023-12-01T03:11:31.670524+0000","last_active":"2023-12-01T03:11:50.587802+0000","last_peered":"2023-12-01T03:11:50.587802+0000","last_clean":"2023-12-01T03:11:50.587802+0000","last_became_active":"2023-12-01T03:11:31.670190+0000","last_became_peered":"2023-12-01T03:11:31.670190+0000","last_unstale":"2023-12-01T03:11:50.587802+0000","last_undegraded":"2023-12-01T03:11:50.587802+0000","last_fullsized":"2023-12-01T03:11:50.587802+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:45.874739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.075912+0000","last_change":"2023-12-01T03:11:51.070027+0000","last_active":"2023-12-01T03:11:51.075912+0000","last_peered":"2023-12-01T03:11:51.075912+0000","last_clean":"2023-12-01T03:11:51.075912+0000","last_became_active":"2023-12-01T03:11:51.069121+0000","last_became_peered":"2023-12-01T03:11:51.069121+0000","last_unstale":"2023-12-01T03:11:51.075912+0000","last_undegraded":"2023-12-01T03:11:51.075912+0000","last_fullsized":"2023-12-01T03:11:51.075912+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:59.791669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.15","version":"238'3","reported_seq":13,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.086847+0000","last_change":"2023-12-01T03:11:50.597304+0000","last_active":"2023-12-01T03:11:51.086847+0000","last_peered":"2023-12-01T03:11:51.086847+0000","last_clean":"2023-12-01T03:11:51.086847+0000","last_became_active":"2023-12-01T03:11:50.596949+0000","last_became_peered":"2023-12-01T03:11:50.596949+0000","last_unstale":"2023-12-01T03:11:51.086847+0000","last_undegraded":"2023-12-01T03:11:51.086847+0000","last_fullsized":"2023-12-01T03:11:51.086847+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:54.254506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583534+0000","last_change":"2023-12-01T03:11:32.212987+0000","last_active":"2023-12-01T03:11:50.583534+0000","last_peered":"2023-12-01T03:11:50.583534+0000","last_clean":"2023-12-01T03:11:50.583534+0000","last_became_active":"2023-12-01T03:11:32.212426+0000","last_became_peered":"2023-12-01T03:11:32.212426+0000","last_unstale":"2023-12-01T03:11:50.583534+0000","last_undegraded":"2023-12-01T03:11:50.583534+0000","last_fullsized":"2023-12-01T03:11:50.583534+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:21:50.286833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582762+0000","last_change":"2023-12-01T03:11:31.676322+0000","last_active":"2023-12-01T03:11:50.582762+0000","last_peered":"2023-12-01T03:11:50.582762+0000","last_clean":"2023-12-01T03:11:50.582762+0000","last_became_active":"2023-12-01T03:11:31.674451+0000","last_became_peered":"2023-12-01T03:11:31.674451+0000","last_unstale":"2023-12-01T03:11:50.582762+0000","last_undegraded":"2023-12-01T03:11:50.582762+0000","last_fullsized":"2023-12-01T03:11:50.582762+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:50:44.820233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070390+0000","last_change":"2023-12-01T03:11:51.068922+0000","last_active":"2023-12-01T03:11:51.070390+0000","last_peered":"2023-12-01T03:11:51.070390+0000","last_clean":"2023-12-01T03:11:51.070390+0000","last_became_active":"2023-12-01T03:11:51.068655+0000","last_became_peered":"2023-12-01T03:11:51.068655+0000","last_unstale":"2023-12-01T03:11:51.070390+0000","last_undegraded":"2023-12-01T03:11:51.070390+0000","last_fullsized":"2023-12-01T03:11:51.070390+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:01.630559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.14","version":"238'6","reported_seq":16,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.083939+0000","last_change":"2023-12-01T03:11:50.592947+0000","last_active":"2023-12-01T03:11:51.083939+0000","last_peered":"2023-12-01T03:11:51.083939+0000","last_clean":"2023-12-01T03:11:51.083939+0000","last_became_active":"2023-12-01T03:11:50.592670+0000","last_became_peered":"2023-12-01T03:11:50.592670+0000","last_unstale":"2023-12-01T03:11:51.083939+0000","last_undegraded":"2023-12-01T03:11:51.083939+0000","last_fullsized":"2023-12-01T03:11:51.083939+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:33:53.678613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.570001+0000","last_change":"2023-12-01T03:11:32.219383+0000","last_active":"2023-12-01T03:11:50.570001+0000","last_peered":"2023-12-01T03:11:50.570001+0000","last_clean":"2023-12-01T03:11:50.570001+0000","last_became_active":"2023-12-01T03:11:32.219139+0000","last_became_peered":"2023-12-01T03:11:32.219139+0000","last_unstale":"2023-12-01T03:11:50.570001+0000","last_undegraded":"2023-12-01T03:11:50.570001+0000","last_fullsized":"2023-12-01T03:11:50.570001+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:40.557913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.584590+0000","last_change":"2023-12-01T03:11:31.662929+0000","last_active":"2023-12-01T03:11:50.584590+0000","last_peered":"2023-12-01T03:11:50.584590+0000","last_clean":"2023-12-01T03:11:50.584590+0000","last_became_active":"2023-12-01T03:11:31.662617+0000","last_became_peered":"2023-12-01T03:11:31.662617+0000","last_unstale":"2023-12-01T03:11:50.584590+0000","last_undegraded":"2023-12-01T03:11:50.584590+0000","last_fullsized":"2023-12-01T03:11:50.584590+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:52:34.611280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089058+0000","last_change":"2023-12-01T03:11:51.080389+0000","last_active":"2023-12-01T03:11:51.089058+0000","last_peered":"2023-12-01T03:11:51.089058+0000","last_clean":"2023-12-01T03:11:51.089058+0000","last_became_active":"2023-12-01T03:11:51.080109+0000","last_became_peered":"2023-12-01T03:11:51.080109+0000","last_unstale":"2023-12-01T03:11:51.089058+0000","last_undegraded":"2023-12-01T03:11:51.089058+0000","last_fullsized":"2023-12-01T03:11:51.089058+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:10:59.791669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.055263+0000","last_change":"2023-12-01T03:11:50.585244+0000","last_active":"2023-12-01T03:11:51.055263+0000","last_peered":"2023-12-01T03:11:51.055263+0000","last_clean":"2023-12-01T03:11:51.055263+0000","last_became_active":"2023-12-01T03:11:50.584935+0000","last_became_peered":"2023-12-01T03:11:50.584935+0000","last_unstale":"2023-12-01T03:11:51.055263+0000","last_undegraded":"2023-12-01T03:11:51.055263+0000","last_fullsized":"2023-12-01T03:11:51.055263+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:32.427402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.579929+0000","last_change":"2023-12-01T03:11:32.209244+0000","last_active":"2023-12-01T03:11:50.579929+0000","last_peered":"2023-12-01T03:11:50.579929+0000","last_clean":"2023-12-01T03:11:50.579929+0000","last_became_active":"2023-12-01T03:11:32.206294+0000","last_became_peered":"2023-12-01T03:11:32.206294+0000","last_unstale":"2023-12-01T03:11:50.579929+0000","last_undegraded":"2023-12-01T03:11:50.579929+0000","last_fullsized":"2023-12-01T03:11:50.579929+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:04:47.593451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"234'3","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.564927+0000","last_change":"2023-12-01T03:11:31.664798+0000","last_active":"2023-12-01T03:11:50.564927+0000","last_peered":"2023-12-01T03:11:50.564927+0000","last_clean":"2023-12-01T03:11:50.564927+0000","last_became_active":"2023-12-01T03:11:31.664276+0000","last_became_peered":"2023-12-01T03:11:31.664276+0000","last_unstale":"2023-12-01T03:11:50.564927+0000","last_undegraded":"2023-12-01T03:11:50.564927+0000","last_fullsized":"2023-12-01T03:11:50.564927+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:18:18.909277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.076730+0000","last_change":"2023-12-01T03:11:51.062845+0000","last_active":"2023-12-01T03:11:51.076730+0000","last_peered":"2023-12-01T03:11:51.076730+0000","last_clean":"2023-12-01T03:11:51.076730+0000","last_became_active":"2023-12-01T03:11:51.062504+0000","last_became_peered":"2023-12-01T03:11:51.062504+0000","last_unstale":"2023-12-01T03:11:51.076730+0000","last_undegraded":"2023-12-01T03:11:51.076730+0000","last_fullsized":"2023-12-01T03:11:51.076730+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:04.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070187+0000","last_change":"2023-12-01T03:11:50.595799+0000","last_active":"2023-12-01T03:11:51.070187+0000","last_peered":"2023-12-01T03:11:51.070187+0000","last_clean":"2023-12-01T03:11:51.070187+0000","last_became_active":"2023-12-01T03:11:50.594796+0000","last_became_peered":"2023-12-01T03:11:50.594796+0000","last_unstale":"2023-12-01T03:11:51.070187+0000","last_undegraded":"2023-12-01T03:11:51.070187+0000","last_fullsized":"2023-12-01T03:11:51.070187+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:04.343416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582898+0000","last_change":"2023-12-01T03:11:32.213607+0000","last_active":"2023-12-01T03:11:50.582898+0000","last_peered":"2023-12-01T03:11:50.582898+0000","last_clean":"2023-12-01T03:11:50.582898+0000","last_became_active":"2023-12-01T03:11:32.211875+0000","last_became_peered":"2023-12-01T03:11:32.211875+0000","last_unstale":"2023-12-01T03:11:50.582898+0000","last_undegraded":"2023-12-01T03:11:50.582898+0000","last_fullsized":"2023-12-01T03:11:50.582898+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:16:15.404930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"235'8","reported_seq":22,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.588826+0000","last_change":"2023-12-01T03:11:31.664162+0000","last_active":"2023-12-01T03:11:50.588826+0000","last_peered":"2023-12-01T03:11:50.588826+0000","last_clean":"2023-12-01T03:11:50.588826+0000","last_became_active":"2023-12-01T03:11:31.663854+0000","last_became_peered":"2023-12-01T03:11:31.663854+0000","last_unstale":"2023-12-01T03:11:50.588826+0000","last_undegraded":"2023-12-01T03:11:50.588826+0000","last_fullsized":"2023-12-01T03:11:50.588826+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:15.367696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.083231+0000","last_change":"2023-12-01T03:11:51.081832+0000","last_active":"2023-12-01T03:11:51.083231+0000","last_peered":"2023-12-01T03:11:51.083231+0000","last_clean":"2023-12-01T03:11:51.083231+0000","last_became_active":"2023-12-01T03:11:51.081477+0000","last_became_peered":"2023-12-01T03:11:51.081477+0000","last_unstale":"2023-12-01T03:11:51.083231+0000","last_undegraded":"2023-12-01T03:11:51.083231+0000","last_fullsized":"2023-12-01T03:11:51.083231+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:28:12.383010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.054962+0000","last_change":"2023-12-01T03:11:50.598270+0000","last_active":"2023-12-01T03:11:51.054962+0000","last_peered":"2023-12-01T03:11:51.054962+0000","last_clean":"2023-12-01T03:11:51.054962+0000","last_became_active":"2023-12-01T03:11:50.597997+0000","last_became_peered":"2023-12-01T03:11:50.597997+0000","last_unstale":"2023-12-01T03:11:51.054962+0000","last_undegraded":"2023-12-01T03:11:51.054962+0000","last_fullsized":"2023-12-01T03:11:51.054962+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:05:05.484343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.586610+0000","last_change":"2023-12-01T03:11:32.203625+0000","last_active":"2023-12-01T03:11:50.586610+0000","last_peered":"2023-12-01T03:11:50.586610+0000","last_clean":"2023-12-01T03:11:50.586610+0000","last_became_active":"2023-12-01T03:11:32.203300+0000","last_became_peered":"2023-12-01T03:11:32.203300+0000","last_unstale":"2023-12-01T03:11:50.586610+0000","last_undegraded":"2023-12-01T03:11:50.586610+0000","last_fullsized":"2023-12-01T03:11:50.586610+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:38:46.890682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.566369+0000","last_change":"2023-12-01T03:11:31.665987+0000","last_active":"2023-12-01T03:11:50.566369+0000","last_peered":"2023-12-01T03:11:50.566369+0000","last_clean":"2023-12-01T03:11:50.566369+0000","last_became_active":"2023-12-01T03:11:31.665550+0000","last_became_peered":"2023-12-01T03:11:31.665550+0000","last_unstale":"2023-12-01T03:11:50.566369+0000","last_undegraded":"2023-12-01T03:11:50.566369+0000","last_fullsized":"2023-12-01T03:11:50.566369+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:16.324155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.062718+0000","last_change":"2023-12-01T03:11:51.058517+0000","last_active":"2023-12-01T03:11:51.062718+0000","last_peered":"2023-12-01T03:11:51.062718+0000","last_clean":"2023-12-01T03:11:51.062718+0000","last_became_active":"2023-12-01T03:11:51.058244+0000","last_became_peered":"2023-12-01T03:11:51.058244+0000","last_unstale":"2023-12-01T03:11:51.062718+0000","last_undegraded":"2023-12-01T03:11:51.062718+0000","last_fullsized":"2023-12-01T03:11:51.062718+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:59:45.386653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"238'8","reported_seq":18,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088097+0000","last_change":"2023-12-01T03:11:50.598938+0000","last_active":"2023-12-01T03:11:51.088097+0000","last_peered":"2023-12-01T03:11:51.088097+0000","last_clean":"2023-12-01T03:11:51.088097+0000","last_became_active":"2023-12-01T03:11:50.598444+0000","last_became_peered":"2023-12-01T03:11:50.598444+0000","last_unstale":"2023-12-01T03:11:51.088097+0000","last_undegraded":"2023-12-01T03:11:51.088097+0000","last_fullsized":"2023-12-01T03:11:51.088097+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:47.245587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.578127+0000","last_change":"2023-12-01T03:11:32.208104+0000","last_active":"2023-12-01T03:11:50.578127+0000","last_peered":"2023-12-01T03:11:50.578127+0000","last_clean":"2023-12-01T03:11:50.578127+0000","last_became_active":"2023-12-01T03:11:32.207859+0000","last_became_peered":"2023-12-01T03:11:32.207859+0000","last_unstale":"2023-12-01T03:11:50.578127+0000","last_undegraded":"2023-12-01T03:11:50.578127+0000","last_fullsized":"2023-12-01T03:11:50.578127+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:38:16.383639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.586731+0000","last_change":"2023-12-01T03:11:31.675387+0000","last_active":"2023-12-01T03:11:50.586731+0000","last_peered":"2023-12-01T03:11:50.586731+0000","last_clean":"2023-12-01T03:11:50.586731+0000","last_became_active":"2023-12-01T03:11:31.675108+0000","last_became_peered":"2023-12-01T03:11:31.675108+0000","last_unstale":"2023-12-01T03:11:50.586731+0000","last_undegraded":"2023-12-01T03:11:50.586731+0000","last_fullsized":"2023-12-01T03:11:50.586731+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:19:27.836486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.085561+0000","last_change":"2023-12-01T03:11:51.084910+0000","last_active":"2023-12-01T03:11:51.085561+0000","last_peered":"2023-12-01T03:11:51.085561+0000","last_clean":"2023-12-01T03:11:51.085561+0000","last_became_active":"2023-12-01T03:11:51.084552+0000","last_became_peered":"2023-12-01T03:11:51.084552+0000","last_unstale":"2023-12-01T03:11:51.085561+0000","last_undegraded":"2023-12-01T03:11:51.085561+0000","last_fullsized":"2023-12-01T03:11:51.085561+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:36.041546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.13","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.083335+0000","last_change":"2023-12-01T03:11:50.598810+0000","last_active":"2023-12-01T03:11:51.083335+0000","last_peered":"2023-12-01T03:11:51.083335+0000","last_clean":"2023-12-01T03:11:51.083335+0000","last_became_active":"2023-12-01T03:11:50.597756+0000","last_became_peered":"2023-12-01T03:11:50.597756+0000","last_unstale":"2023-12-01T03:11:51.083335+0000","last_undegraded":"2023-12-01T03:11:51.083335+0000","last_fullsized":"2023-12-01T03:11:51.083335+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:29:58.374404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.590599+0000","last_change":"2023-12-01T03:11:32.204685+0000","last_active":"2023-12-01T03:11:50.590599+0000","last_peered":"2023-12-01T03:11:50.590599+0000","last_clean":"2023-12-01T03:11:50.590599+0000","last_became_active":"2023-12-01T03:11:32.204422+0000","last_became_peered":"2023-12-01T03:11:32.204422+0000","last_unstale":"2023-12-01T03:11:50.590599+0000","last_undegraded":"2023-12-01T03:11:50.590599+0000","last_fullsized":"2023-12-01T03:11:50.590599+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583882+0000","last_change":"2023-12-01T03:11:31.671174+0000","last_active":"2023-12-01T03:11:50.583882+0000","last_peered":"2023-12-01T03:11:50.583882+0000","last_clean":"2023-12-01T03:11:50.583882+0000","last_became_active":"2023-12-01T03:11:31.670862+0000","last_became_peered":"2023-12-01T03:11:31.670862+0000","last_unstale":"2023-12-01T03:11:50.583882+0000","last_undegraded":"2023-12-01T03:11:50.583882+0000","last_fullsized":"2023-12-01T03:11:50.583882+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:14:37.158883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.066075+0000","last_change":"2023-12-01T03:11:51.062334+0000","last_active":"2023-12-01T03:11:51.066075+0000","last_peered":"2023-12-01T03:11:51.066075+0000","last_clean":"2023-12-01T03:11:51.066075+0000","last_became_active":"2023-12-01T03:11:51.062033+0000","last_became_peered":"2023-12-01T03:11:51.062033+0000","last_unstale":"2023-12-01T03:11:51.066075+0000","last_undegraded":"2023-12-01T03:11:51.066075+0000","last_fullsized":"2023-12-01T03:11:51.066075+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:13.658416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068064+0000","last_change":"2023-12-01T03:11:50.596720+0000","last_active":"2023-12-01T03:11:51.068064+0000","last_peered":"2023-12-01T03:11:51.068064+0000","last_clean":"2023-12-01T03:11:51.068064+0000","last_became_active":"2023-12-01T03:11:50.596434+0000","last_became_peered":"2023-12-01T03:11:50.596434+0000","last_unstale":"2023-12-01T03:11:51.068064+0000","last_undegraded":"2023-12-01T03:11:51.068064+0000","last_fullsized":"2023-12-01T03:11:51.068064+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:35.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568403+0000","last_change":"2023-12-01T03:11:32.203594+0000","last_active":"2023-12-01T03:11:50.568403+0000","last_peered":"2023-12-01T03:11:50.568403+0000","last_clean":"2023-12-01T03:11:50.568403+0000","last_became_active":"2023-12-01T03:11:32.203346+0000","last_became_peered":"2023-12-01T03:11:32.203346+0000","last_unstale":"2023-12-01T03:11:50.568403+0000","last_undegraded":"2023-12-01T03:11:50.568403+0000","last_fullsized":"2023-12-01T03:11:50.568403+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:22.924077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.10","version":"234'8","reported_seq":22,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568113+0000","last_change":"2023-12-01T03:11:31.673143+0000","last_active":"2023-12-01T03:11:50.568113+0000","last_peered":"2023-12-01T03:11:50.568113+0000","last_clean":"2023-12-01T03:11:50.568113+0000","last_became_active":"2023-12-01T03:11:31.671261+0000","last_became_peered":"2023-12-01T03:11:31.671261+0000","last_unstale":"2023-12-01T03:11:50.568113+0000","last_undegraded":"2023-12-01T03:11:50.568113+0000","last_fullsized":"2023-12-01T03:11:50.568113+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:14:13.744340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.065327+0000","last_change":"2023-12-01T03:11:51.059709+0000","last_active":"2023-12-01T03:11:51.065327+0000","last_peered":"2023-12-01T03:11:51.065327+0000","last_clean":"2023-12-01T03:11:51.065327+0000","last_became_active":"2023-12-01T03:11:51.059353+0000","last_became_peered":"2023-12-01T03:11:51.059353+0000","last_unstale":"2023-12-01T03:11:51.065327+0000","last_undegraded":"2023-12-01T03:11:51.065327+0000","last_fullsized":"2023-12-01T03:11:51.065327+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:21:35.791011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.d","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.074545+0000","last_change":"2023-12-01T03:11:50.591624+0000","last_active":"2023-12-01T03:11:51.074545+0000","last_peered":"2023-12-01T03:11:51.074545+0000","last_clean":"2023-12-01T03:11:51.074545+0000","last_became_active":"2023-12-01T03:11:50.590955+0000","last_became_peered":"2023-12-01T03:11:50.590955+0000","last_unstale":"2023-12-01T03:11:51.074545+0000","last_undegraded":"2023-12-01T03:11:51.074545+0000","last_fullsized":"2023-12-01T03:11:51.074545+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:05.252744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.584501+0000","last_change":"2023-12-01T03:11:32.209735+0000","last_active":"2023-12-01T03:11:50.584501+0000","last_peered":"2023-12-01T03:11:50.584501+0000","last_clean":"2023-12-01T03:11:50.584501+0000","last_became_active":"2023-12-01T03:11:32.209148+0000","last_became_peered":"2023-12-01T03:11:32.209148+0000","last_unstale":"2023-12-01T03:11:50.584501+0000","last_undegraded":"2023-12-01T03:11:50.584501+0000","last_fullsized":"2023-12-01T03:11:50.584501+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:46.532354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"234'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582869+0000","last_change":"2023-12-01T03:11:31.677841+0000","last_active":"2023-12-01T03:11:50.582869+0000","last_peered":"2023-12-01T03:11:50.582869+0000","last_clean":"2023-12-01T03:11:50.582869+0000","last_became_active":"2023-12-01T03:11:31.677483+0000","last_became_peered":"2023-12-01T03:11:31.677483+0000","last_unstale":"2023-12-01T03:11:50.582869+0000","last_undegraded":"2023-12-01T03:11:50.582869+0000","last_fullsized":"2023-12-01T03:11:50.582869+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:28:31.837132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078397+0000","last_change":"2023-12-01T03:11:51.077820+0000","last_active":"2023-12-01T03:11:51.078397+0000","last_peered":"2023-12-01T03:11:51.078397+0000","last_clean":"2023-12-01T03:11:51.078397+0000","last_became_active":"2023-12-01T03:11:51.077536+0000","last_became_peered":"2023-12-01T03:11:51.077536+0000","last_unstale":"2023-12-01T03:11:51.078397+0000","last_undegraded":"2023-12-01T03:11:51.078397+0000","last_fullsized":"2023-12-01T03:11:51.078397+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:48.264690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068796+0000","last_change":"2023-12-01T03:11:50.594903+0000","last_active":"2023-12-01T03:11:51.068796+0000","last_peered":"2023-12-01T03:11:51.068796+0000","last_clean":"2023-12-01T03:11:51.068796+0000","last_became_active":"2023-12-01T03:11:50.594146+0000","last_became_peered":"2023-12-01T03:11:50.594146+0000","last_unstale":"2023-12-01T03:11:51.068796+0000","last_undegraded":"2023-12-01T03:11:51.068796+0000","last_fullsized":"2023-12-01T03:11:51.068796+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:21:34.771908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.589488+0000","last_change":"2023-12-01T03:11:32.213539+0000","last_active":"2023-12-01T03:11:50.589488+0000","last_peered":"2023-12-01T03:11:50.589488+0000","last_clean":"2023-12-01T03:11:50.589488+0000","last_became_active":"2023-12-01T03:11:32.212492+0000","last_became_peered":"2023-12-01T03:11:32.212492+0000","last_unstale":"2023-12-01T03:11:50.589488+0000","last_undegraded":"2023-12-01T03:11:50.589488+0000","last_fullsized":"2023-12-01T03:11:50.589488+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:29:48.204277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.e","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581381+0000","last_change":"2023-12-01T03:11:31.670701+0000","last_active":"2023-12-01T03:11:50.581381+0000","last_peered":"2023-12-01T03:11:50.581381+0000","last_clean":"2023-12-01T03:11:50.581381+0000","last_became_active":"2023-12-01T03:11:31.670179+0000","last_became_peered":"2023-12-01T03:11:31.670179+0000","last_unstale":"2023-12-01T03:11:50.581381+0000","last_undegraded":"2023-12-01T03:11:50.581381+0000","last_fullsized":"2023-12-01T03:11:50.581381+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:15:01.094744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.078798+0000","last_change":"2023-12-01T03:11:51.068023+0000","last_active":"2023-12-01T03:11:51.078798+0000","last_peered":"2023-12-01T03:11:51.078798+0000","last_clean":"2023-12-01T03:11:51.078798+0000","last_became_active":"2023-12-01T03:11:51.067743+0000","last_became_peered":"2023-12-01T03:11:51.067743+0000","last_unstale":"2023-12-01T03:11:51.078798+0000","last_undegraded":"2023-12-01T03:11:51.078798+0000","last_fullsized":"2023-12-01T03:11:51.078798+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:23:20.739511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"238'1","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089230+0000","last_change":"2023-12-01T03:11:50.588920+0000","last_active":"2023-12-01T03:11:51.089230+0000","last_peered":"2023-12-01T03:11:51.089230+0000","last_clean":"2023-12-01T03:11:51.089230+0000","last_became_active":"2023-12-01T03:11:50.588620+0000","last_became_peered":"2023-12-01T03:11:50.588620+0000","last_unstale":"2023-12-01T03:11:51.089230+0000","last_undegraded":"2023-12-01T03:11:51.089230+0000","last_fullsized":"2023-12-01T03:11:51.089230+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:19:47.329072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568782+0000","last_change":"2023-12-01T03:11:32.212327+0000","last_active":"2023-12-01T03:11:50.568782+0000","last_peered":"2023-12-01T03:11:50.568782+0000","last_clean":"2023-12-01T03:11:50.568782+0000","last_became_active":"2023-12-01T03:11:32.212068+0000","last_became_peered":"2023-12-01T03:11:32.212068+0000","last_unstale":"2023-12-01T03:11:50.568782+0000","last_undegraded":"2023-12-01T03:11:50.568782+0000","last_fullsized":"2023-12-01T03:11:50.568782+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:36:31.875817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"234'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582571+0000","last_change":"2023-12-01T03:11:31.679287+0000","last_active":"2023-12-01T03:11:50.582571+0000","last_peered":"2023-12-01T03:11:50.582571+0000","last_clean":"2023-12-01T03:11:50.582571+0000","last_became_active":"2023-12-01T03:11:31.678925+0000","last_became_peered":"2023-12-01T03:11:31.678925+0000","last_unstale":"2023-12-01T03:11:50.582571+0000","last_undegraded":"2023-12-01T03:11:50.582571+0000","last_fullsized":"2023-12-01T03:11:50.582571+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:28:46.964303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.062638+0000","last_change":"2023-12-01T03:11:51.061097+0000","last_active":"2023-12-01T03:11:51.062638+0000","last_peered":"2023-12-01T03:11:51.062638+0000","last_clean":"2023-12-01T03:11:51.062638+0000","last_became_active":"2023-12-01T03:11:51.060803+0000","last_became_peered":"2023-12-01T03:11:51.060803+0000","last_unstale":"2023-12-01T03:11:51.062638+0000","last_undegraded":"2023-12-01T03:11:51.062638+0000","last_fullsized":"2023-12-01T03:11:51.062638+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:10:05.844579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.e","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089007+0000","last_change":"2023-12-01T03:11:50.599084+0000","last_active":"2023-12-01T03:11:51.089007+0000","last_peered":"2023-12-01T03:11:51.089007+0000","last_clean":"2023-12-01T03:11:51.089007+0000","last_became_active":"2023-12-01T03:11:50.594125+0000","last_became_peered":"2023-12-01T03:11:50.594125+0000","last_unstale":"2023-12-01T03:11:51.089007+0000","last_undegraded":"2023-12-01T03:11:51.089007+0000","last_fullsized":"2023-12-01T03:11:51.089007+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:28:15.840367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.592519+0000","last_change":"2023-12-01T03:11:32.214436+0000","last_active":"2023-12-01T03:11:50.592519+0000","last_peered":"2023-12-01T03:11:50.592519+0000","last_clean":"2023-12-01T03:11:50.592519+0000","last_became_active":"2023-12-01T03:11:32.214162+0000","last_became_peered":"2023-12-01T03:11:32.214162+0000","last_unstale":"2023-12-01T03:11:50.592519+0000","last_undegraded":"2023-12-01T03:11:50.592519+0000","last_fullsized":"2023-12-01T03:11:50.592519+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:18:18.055289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583700+0000","last_change":"2023-12-01T03:11:31.674632+0000","last_active":"2023-12-01T03:11:50.583700+0000","last_peered":"2023-12-01T03:11:50.583700+0000","last_clean":"2023-12-01T03:11:50.583700+0000","last_became_active":"2023-12-01T03:11:31.674319+0000","last_became_peered":"2023-12-01T03:11:31.674319+0000","last_unstale":"2023-12-01T03:11:50.583700+0000","last_undegraded":"2023-12-01T03:11:50.583700+0000","last_fullsized":"2023-12-01T03:11:50.583700+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:09:21.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.072464+0000","last_change":"2023-12-01T03:11:51.064064+0000","last_active":"2023-12-01T03:11:51.072464+0000","last_peered":"2023-12-01T03:11:51.072464+0000","last_clean":"2023-12-01T03:11:51.072464+0000","last_became_active":"2023-12-01T03:11:51.061405+0000","last_became_peered":"2023-12-01T03:11:51.061405+0000","last_unstale":"2023-12-01T03:11:51.072464+0000","last_undegraded":"2023-12-01T03:11:51.072464+0000","last_fullsized":"2023-12-01T03:11:51.072464+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:32:55.481603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.062954+0000","last_change":"2023-12-01T03:11:50.589295+0000","last_active":"2023-12-01T03:11:51.062954+0000","last_peered":"2023-12-01T03:11:51.062954+0000","last_clean":"2023-12-01T03:11:51.062954+0000","last_became_active":"2023-12-01T03:11:50.588119+0000","last_became_peered":"2023-12-01T03:11:50.588119+0000","last_unstale":"2023-12-01T03:11:51.062954+0000","last_undegraded":"2023-12-01T03:11:51.062954+0000","last_fullsized":"2023-12-01T03:11:51.062954+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:00.593323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.579907+0000","last_change":"2023-12-01T03:11:32.209024+0000","last_active":"2023-12-01T03:11:50.579907+0000","last_peered":"2023-12-01T03:11:50.579907+0000","last_clean":"2023-12-01T03:11:50.579907+0000","last_became_active":"2023-12-01T03:11:32.208729+0000","last_became_peered":"2023-12-01T03:11:32.208729+0000","last_unstale":"2023-12-01T03:11:50.579907+0000","last_undegraded":"2023-12-01T03:11:50.579907+0000","last_fullsized":"2023-12-01T03:11:50.579907+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:27:53.473015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.b","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581606+0000","last_change":"2023-12-01T03:11:31.676885+0000","last_active":"2023-12-01T03:11:50.581606+0000","last_peered":"2023-12-01T03:11:50.581606+0000","last_clean":"2023-12-01T03:11:50.581606+0000","last_became_active":"2023-12-01T03:11:31.675545+0000","last_became_peered":"2023-12-01T03:11:31.675545+0000","last_unstale":"2023-12-01T03:11:50.581606+0000","last_undegraded":"2023-12-01T03:11:50.581606+0000","last_fullsized":"2023-12-01T03:11:50.581606+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:21:03.962061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070436+0000","last_change":"2023-12-01T03:11:51.068349+0000","last_active":"2023-12-01T03:11:51.070436+0000","last_peered":"2023-12-01T03:11:51.070436+0000","last_clean":"2023-12-01T03:11:51.070436+0000","last_became_active":"2023-12-01T03:11:51.068096+0000","last_became_peered":"2023-12-01T03:11:51.068096+0000","last_unstale":"2023-12-01T03:11:51.070436+0000","last_undegraded":"2023-12-01T03:11:51.070436+0000","last_fullsized":"2023-12-01T03:11:51.070436+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:28:12.383010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.049893+0000","last_change":"2023-12-01T03:11:50.585375+0000","last_active":"2023-12-01T03:11:51.049893+0000","last_peered":"2023-12-01T03:11:51.049893+0000","last_clean":"2023-12-01T03:11:51.049893+0000","last_became_active":"2023-12-01T03:11:50.584765+0000","last_became_peered":"2023-12-01T03:11:50.584765+0000","last_unstale":"2023-12-01T03:11:51.049893+0000","last_undegraded":"2023-12-01T03:11:51.049893+0000","last_fullsized":"2023-12-01T03:11:51.049893+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:23:35.022443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.593405+0000","last_change":"2023-12-01T03:11:32.204502+0000","last_active":"2023-12-01T03:11:50.593405+0000","last_peered":"2023-12-01T03:11:50.593405+0000","last_clean":"2023-12-01T03:11:50.593405+0000","last_became_active":"2023-12-01T03:11:32.204048+0000","last_became_peered":"2023-12-01T03:11:32.204048+0000","last_unstale":"2023-12-01T03:11:50.593405+0000","last_undegraded":"2023-12-01T03:11:50.593405+0000","last_fullsized":"2023-12-01T03:11:50.593405+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:15:30.930202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.577647+0000","last_change":"2023-12-01T03:11:31.675947+0000","last_active":"2023-12-01T03:11:50.577647+0000","last_peered":"2023-12-01T03:11:50.577647+0000","last_clean":"2023-12-01T03:11:50.577647+0000","last_became_active":"2023-12-01T03:11:31.675372+0000","last_became_peered":"2023-12-01T03:11:31.675372+0000","last_unstale":"2023-12-01T03:11:50.577647+0000","last_undegraded":"2023-12-01T03:11:50.577647+0000","last_fullsized":"2023-12-01T03:11:50.577647+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:13:45.516411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.073967+0000","last_change":"2023-12-01T03:11:51.073355+0000","last_active":"2023-12-01T03:11:51.073967+0000","last_peered":"2023-12-01T03:11:51.073967+0000","last_clean":"2023-12-01T03:11:51.073967+0000","last_became_active":"2023-12-01T03:11:51.072973+0000","last_became_peered":"2023-12-01T03:11:51.072973+0000","last_unstale":"2023-12-01T03:11:51.073967+0000","last_undegraded":"2023-12-01T03:11:51.073967+0000","last_fullsized":"2023-12-01T03:11:51.073967+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:08:06.271847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.b","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088280+0000","last_change":"2023-12-01T03:11:50.591967+0000","last_active":"2023-12-01T03:11:51.088280+0000","last_peered":"2023-12-01T03:11:51.088280+0000","last_clean":"2023-12-01T03:11:51.088280+0000","last_became_active":"2023-12-01T03:11:50.591673+0000","last_became_peered":"2023-12-01T03:11:50.591673+0000","last_unstale":"2023-12-01T03:11:51.088280+0000","last_undegraded":"2023-12-01T03:11:51.088280+0000","last_fullsized":"2023-12-01T03:11:51.088280+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:30:19.059675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582499+0000","last_change":"2023-12-01T03:11:32.214460+0000","last_active":"2023-12-01T03:11:50.582499+0000","last_peered":"2023-12-01T03:11:50.582499+0000","last_clean":"2023-12-01T03:11:50.582499+0000","last_became_active":"2023-12-01T03:11:32.213987+0000","last_became_peered":"2023-12-01T03:11:32.213987+0000","last_unstale":"2023-12-01T03:11:50.582499+0000","last_undegraded":"2023-12-01T03:11:50.582499+0000","last_fullsized":"2023-12-01T03:11:50.582499+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:20.713703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568566+0000","last_change":"2023-12-01T03:11:31.673899+0000","last_active":"2023-12-01T03:11:50.568566+0000","last_peered":"2023-12-01T03:11:50.568566+0000","last_clean":"2023-12-01T03:11:50.568566+0000","last_became_active":"2023-12-01T03:11:31.673474+0000","last_became_peered":"2023-12-01T03:11:31.673474+0000","last_unstale":"2023-12-01T03:11:50.568566+0000","last_undegraded":"2023-12-01T03:11:50.568566+0000","last_fullsized":"2023-12-01T03:11:50.568566+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:50.332038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088255+0000","last_change":"2023-12-01T03:11:51.074892+0000","last_active":"2023-12-01T03:11:51.088255+0000","last_peered":"2023-12-01T03:11:51.088255+0000","last_clean":"2023-12-01T03:11:51.088255+0000","last_became_active":"2023-12-01T03:11:51.074651+0000","last_became_peered":"2023-12-01T03:11:51.074651+0000","last_unstale":"2023-12-01T03:11:51.088255+0000","last_undegraded":"2023-12-01T03:11:51.088255+0000","last_fullsized":"2023-12-01T03:11:51.088255+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:10:05.844579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068152+0000","last_change":"2023-12-01T03:11:50.593853+0000","last_active":"2023-12-01T03:11:51.068152+0000","last_peered":"2023-12-01T03:11:51.068152+0000","last_clean":"2023-12-01T03:11:51.068152+0000","last_became_active":"2023-12-01T03:11:50.593584+0000","last_became_peered":"2023-12-01T03:11:50.593584+0000","last_unstale":"2023-12-01T03:11:51.068152+0000","last_undegraded":"2023-12-01T03:11:51.068152+0000","last_fullsized":"2023-12-01T03:11:51.068152+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:00.593323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.564517+0000","last_change":"2023-12-01T03:11:32.214835+0000","last_active":"2023-12-01T03:11:50.564517+0000","last_peered":"2023-12-01T03:11:50.564517+0000","last_clean":"2023-12-01T03:11:50.564517+0000","last_became_active":"2023-12-01T03:11:32.214553+0000","last_became_peered":"2023-12-01T03:11:32.214553+0000","last_unstale":"2023-12-01T03:11:50.564517+0000","last_undegraded":"2023-12-01T03:11:50.564517+0000","last_fullsized":"2023-12-01T03:11:50.564517+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:41.494499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568234+0000","last_change":"2023-12-01T03:11:31.682949+0000","last_active":"2023-12-01T03:11:50.568234+0000","last_peered":"2023-12-01T03:11:50.568234+0000","last_clean":"2023-12-01T03:11:50.568234+0000","last_became_active":"2023-12-01T03:11:31.682454+0000","last_became_peered":"2023-12-01T03:11:31.682454+0000","last_unstale":"2023-12-01T03:11:50.568234+0000","last_undegraded":"2023-12-01T03:11:50.568234+0000","last_fullsized":"2023-12-01T03:11:50.568234+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:45:49.777291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.081777+0000","last_change":"2023-12-01T03:11:51.073657+0000","last_active":"2023-12-01T03:11:51.081777+0000","last_peered":"2023-12-01T03:11:51.081777+0000","last_clean":"2023-12-01T03:11:51.081777+0000","last_became_active":"2023-12-01T03:11:51.072728+0000","last_became_peered":"2023-12-01T03:11:51.072728+0000","last_unstale":"2023-12-01T03:11:51.081777+0000","last_undegraded":"2023-12-01T03:11:51.081777+0000","last_fullsized":"2023-12-01T03:11:51.081777+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:09.196828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.055603+0000","last_change":"2023-12-01T03:11:50.595863+0000","last_active":"2023-12-01T03:11:51.055603+0000","last_peered":"2023-12-01T03:11:51.055603+0000","last_clean":"2023-12-01T03:11:51.055603+0000","last_became_active":"2023-12-01T03:11:50.595589+0000","last_became_peered":"2023-12-01T03:11:50.595589+0000","last_unstale":"2023-12-01T03:11:51.055603+0000","last_undegraded":"2023-12-01T03:11:51.055603+0000","last_fullsized":"2023-12-01T03:11:51.055603+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:19.793585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581273+0000","last_change":"2023-12-01T03:11:32.204430+0000","last_active":"2023-12-01T03:11:50.581273+0000","last_peered":"2023-12-01T03:11:50.581273+0000","last_clean":"2023-12-01T03:11:50.581273+0000","last_became_active":"2023-12-01T03:11:32.204043+0000","last_became_peered":"2023-12-01T03:11:32.204043+0000","last_unstale":"2023-12-01T03:11:50.581273+0000","last_undegraded":"2023-12-01T03:11:50.581273+0000","last_fullsized":"2023-12-01T03:11:50.581273+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:33:45.498838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.589595+0000","last_change":"2023-12-01T03:11:31.668375+0000","last_active":"2023-12-01T03:11:50.589595+0000","last_peered":"2023-12-01T03:11:50.589595+0000","last_clean":"2023-12-01T03:11:50.589595+0000","last_became_active":"2023-12-01T03:11:31.668109+0000","last_became_peered":"2023-12-01T03:11:31.668109+0000","last_unstale":"2023-12-01T03:11:50.589595+0000","last_undegraded":"2023-12-01T03:11:50.589595+0000","last_fullsized":"2023-12-01T03:11:50.589595+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:34.959657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.089053+0000","last_change":"2023-12-01T03:11:51.084028+0000","last_active":"2023-12-01T03:11:51.089053+0000","last_peered":"2023-12-01T03:11:51.089053+0000","last_clean":"2023-12-01T03:11:51.089053+0000","last_became_active":"2023-12-01T03:11:51.079967+0000","last_became_peered":"2023-12-01T03:11:51.079967+0000","last_unstale":"2023-12-01T03:11:51.089053+0000","last_undegraded":"2023-12-01T03:11:51.089053+0000","last_fullsized":"2023-12-01T03:11:51.089053+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:00.020345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068930+0000","last_change":"2023-12-01T03:11:50.599761+0000","last_active":"2023-12-01T03:11:51.068930+0000","last_peered":"2023-12-01T03:11:51.068930+0000","last_clean":"2023-12-01T03:11:51.068930+0000","last_became_active":"2023-12-01T03:11:50.599505+0000","last_became_peered":"2023-12-01T03:11:50.599505+0000","last_unstale":"2023-12-01T03:11:51.068930+0000","last_undegraded":"2023-12-01T03:11:51.068930+0000","last_fullsized":"2023-12-01T03:11:51.068930+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:33:53.678613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582481+0000","last_change":"2023-12-01T03:11:32.223409+0000","last_active":"2023-12-01T03:11:50.582481+0000","last_peered":"2023-12-01T03:11:50.582481+0000","last_clean":"2023-12-01T03:11:50.582481+0000","last_became_active":"2023-12-01T03:11:32.221541+0000","last_became_peered":"2023-12-01T03:11:32.221541+0000","last_unstale":"2023-12-01T03:11:50.582481+0000","last_undegraded":"2023-12-01T03:11:50.582481+0000","last_fullsized":"2023-12-01T03:11:50.582481+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:53.273884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.6","version":"235'1","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583935+0000","last_change":"2023-12-01T03:11:31.679248+0000","last_active":"2023-12-01T03:11:50.583935+0000","last_peered":"2023-12-01T03:11:50.583935+0000","last_clean":"2023-12-01T03:11:50.583935+0000","last_became_active":"2023-12-01T03:11:31.678973+0000","last_became_peered":"2023-12-01T03:11:31.678973+0000","last_unstale":"2023-12-01T03:11:50.583935+0000","last_undegraded":"2023-12-01T03:11:50.583935+0000","last_fullsized":"2023-12-01T03:11:50.583935+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.073271+0000","last_change":"2023-12-01T03:11:51.071661+0000","last_active":"2023-12-01T03:11:51.073271+0000","last_peered":"2023-12-01T03:11:51.073271+0000","last_clean":"2023-12-01T03:11:51.073271+0000","last_became_active":"2023-12-01T03:11:51.071413+0000","last_became_peered":"2023-12-01T03:11:51.071413+0000","last_unstale":"2023-12-01T03:11:51.073271+0000","last_undegraded":"2023-12-01T03:11:51.073271+0000","last_fullsized":"2023-12-01T03:11:51.073271+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:24:05.519021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.7","version":"238'2","reported_seq":12,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.088668+0000","last_change":"2023-12-01T03:11:50.594548+0000","last_active":"2023-12-01T03:11:51.088668+0000","last_peered":"2023-12-01T03:11:51.088668+0000","last_clean":"2023-12-01T03:11:51.088668+0000","last_became_active":"2023-12-01T03:11:50.593162+0000","last_became_peered":"2023-12-01T03:11:50.593162+0000","last_unstale":"2023-12-01T03:11:51.088668+0000","last_undegraded":"2023-12-01T03:11:51.088668+0000","last_fullsized":"2023-12-01T03:11:51.088668+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:59:44.367550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581818+0000","last_change":"2023-12-01T03:11:32.207147+0000","last_active":"2023-12-01T03:11:50.581818+0000","last_peered":"2023-12-01T03:11:50.581818+0000","last_clean":"2023-12-01T03:11:50.581818+0000","last_became_active":"2023-12-01T03:11:32.206872+0000","last_became_peered":"2023-12-01T03:11:32.206872+0000","last_unstale":"2023-12-01T03:11:50.581818+0000","last_undegraded":"2023-12-01T03:11:50.581818+0000","last_fullsized":"2023-12-01T03:11:50.581818+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:09:57.557776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.592609+0000","last_change":"2023-12-01T03:11:31.663771+0000","last_active":"2023-12-01T03:11:50.592609+0000","last_peered":"2023-12-01T03:11:50.592609+0000","last_clean":"2023-12-01T03:11:50.592609+0000","last_became_active":"2023-12-01T03:11:31.662877+0000","last_became_peered":"2023-12-01T03:11:31.662877+0000","last_unstale":"2023-12-01T03:11:50.592609+0000","last_undegraded":"2023-12-01T03:11:50.592609+0000","last_fullsized":"2023-12-01T03:11:50.592609+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:43:59.396265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.090184+0000","last_change":"2023-12-01T03:11:51.089350+0000","last_active":"2023-12-01T03:11:51.090184+0000","last_peered":"2023-12-01T03:11:51.090184+0000","last_clean":"2023-12-01T03:11:51.090184+0000","last_became_active":"2023-12-01T03:11:51.089010+0000","last_became_peered":"2023-12-01T03:11:51.089010+0000","last_unstale":"2023-12-01T03:11:51.090184+0000","last_undegraded":"2023-12-01T03:11:51.090184+0000","last_fullsized":"2023-12-01T03:11:51.090184+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:45:06.821895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.060978+0000","last_change":"2023-12-01T03:11:50.594295+0000","last_active":"2023-12-01T03:11:51.060978+0000","last_peered":"2023-12-01T03:11:51.060978+0000","last_clean":"2023-12-01T03:11:51.060978+0000","last_became_active":"2023-12-01T03:11:50.594018+0000","last_became_peered":"2023-12-01T03:11:50.594018+0000","last_unstale":"2023-12-01T03:11:51.060978+0000","last_undegraded":"2023-12-01T03:11:51.060978+0000","last_fullsized":"2023-12-01T03:11:51.060978+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:10:53.866492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.583031+0000","last_change":"2023-12-01T03:11:32.208042+0000","last_active":"2023-12-01T03:11:50.583031+0000","last_peered":"2023-12-01T03:11:50.583031+0000","last_clean":"2023-12-01T03:11:50.583031+0000","last_became_active":"2023-12-01T03:11:32.207767+0000","last_became_peered":"2023-12-01T03:11:32.207767+0000","last_unstale":"2023-12-01T03:11:50.583031+0000","last_undegraded":"2023-12-01T03:11:50.583031+0000","last_fullsized":"2023-12-01T03:11:50.583031+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:36:17.846927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568923+0000","last_change":"2023-12-01T03:11:31.673043+0000","last_active":"2023-12-01T03:11:50.568923+0000","last_peered":"2023-12-01T03:11:50.568923+0000","last_clean":"2023-12-01T03:11:50.568923+0000","last_became_active":"2023-12-01T03:11:31.672780+0000","last_became_peered":"2023-12-01T03:11:31.672780+0000","last_unstale":"2023-12-01T03:11:50.568923+0000","last_undegraded":"2023-12-01T03:11:50.568923+0000","last_fullsized":"2023-12-01T03:11:50.568923+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:26:40.478556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.061603+0000","last_change":"2023-12-01T03:11:51.060543+0000","last_active":"2023-12-01T03:11:51.061603+0000","last_peered":"2023-12-01T03:11:51.061603+0000","last_clean":"2023-12-01T03:11:51.061603+0000","last_became_active":"2023-12-01T03:11:51.060251+0000","last_became_peered":"2023-12-01T03:11:51.060251+0000","last_unstale":"2023-12-01T03:11:51.061603+0000","last_undegraded":"2023-12-01T03:11:51.061603+0000","last_fullsized":"2023-12-01T03:11:51.061603+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:51:04.334452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.056881+0000","last_change":"2023-12-01T03:11:50.593239+0000","last_active":"2023-12-01T03:11:51.056881+0000","last_peered":"2023-12-01T03:11:51.056881+0000","last_clean":"2023-12-01T03:11:51.056881+0000","last_became_active":"2023-12-01T03:11:50.592971+0000","last_became_peered":"2023-12-01T03:11:50.592971+0000","last_unstale":"2023-12-01T03:11:51.056881+0000","last_undegraded":"2023-12-01T03:11:51.056881+0000","last_fullsized":"2023-12-01T03:11:51.056881+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:23:04.343416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.579962+0000","last_change":"2023-12-01T03:11:32.224518+0000","last_active":"2023-12-01T03:11:50.579962+0000","last_peered":"2023-12-01T03:11:50.579962+0000","last_clean":"2023-12-01T03:11:50.579962+0000","last_became_active":"2023-12-01T03:11:32.220831+0000","last_became_peered":"2023-12-01T03:11:32.220831+0000","last_unstale":"2023-12-01T03:11:50.579962+0000","last_undegraded":"2023-12-01T03:11:50.579962+0000","last_fullsized":"2023-12-01T03:11:50.579962+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568848+0000","last_change":"2023-12-01T03:11:31.677162+0000","last_active":"2023-12-01T03:11:50.568848+0000","last_peered":"2023-12-01T03:11:50.568848+0000","last_clean":"2023-12-01T03:11:50.568848+0000","last_became_active":"2023-12-01T03:11:31.674178+0000","last_became_peered":"2023-12-01T03:11:31.674178+0000","last_unstale":"2023-12-01T03:11:50.568848+0000","last_undegraded":"2023-12-01T03:11:50.568848+0000","last_fullsized":"2023-12-01T03:11:50.568848+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:38:13.700505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.072346+0000","last_change":"2023-12-01T03:11:51.069551+0000","last_active":"2023-12-01T03:11:51.072346+0000","last_peered":"2023-12-01T03:11:51.072346+0000","last_clean":"2023-12-01T03:11:51.072346+0000","last_became_active":"2023-12-01T03:11:51.069264+0000","last_became_peered":"2023-12-01T03:11:51.069264+0000","last_unstale":"2023-12-01T03:11:51.072346+0000","last_undegraded":"2023-12-01T03:11:51.072346+0000","last_fullsized":"2023-12-01T03:11:51.072346+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:18:48.264690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.052593+0000","last_change":"2023-12-01T03:11:50.599045+0000","last_active":"2023-12-01T03:11:51.052593+0000","last_peered":"2023-12-01T03:11:51.052593+0000","last_clean":"2023-12-01T03:11:51.052593+0000","last_became_active":"2023-12-01T03:11:50.598742+0000","last_became_peered":"2023-12-01T03:11:50.598742+0000","last_unstale":"2023-12-01T03:11:51.052593+0000","last_undegraded":"2023-12-01T03:11:51.052593+0000","last_fullsized":"2023-12-01T03:11:51.052593+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:45:05.802792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.581427+0000","last_change":"2023-12-01T03:11:32.222439+0000","last_active":"2023-12-01T03:11:50.581427+0000","last_peered":"2023-12-01T03:11:50.581427+0000","last_clean":"2023-12-01T03:11:50.581427+0000","last_became_active":"2023-12-01T03:11:32.221662+0000","last_became_peered":"2023-12-01T03:11:32.221662+0000","last_unstale":"2023-12-01T03:11:50.581427+0000","last_undegraded":"2023-12-01T03:11:50.581427+0000","last_fullsized":"2023-12-01T03:11:50.581427+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:49:30.651975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.590846+0000","last_change":"2023-12-01T03:11:31.672908+0000","last_active":"2023-12-01T03:11:50.590846+0000","last_peered":"2023-12-01T03:11:50.590846+0000","last_clean":"2023-12-01T03:11:50.590846+0000","last_became_active":"2023-12-01T03:11:31.671738+0000","last_became_peered":"2023-12-01T03:11:31.671738+0000","last_unstale":"2023-12-01T03:11:50.590846+0000","last_undegraded":"2023-12-01T03:11:50.590846+0000","last_fullsized":"2023-12-01T03:11:50.590846+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:12:39.541970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.068190+0000","last_change":"2023-12-01T03:11:51.065065+0000","last_active":"2023-12-01T03:11:51.068190+0000","last_peered":"2023-12-01T03:11:51.068190+0000","last_clean":"2023-12-01T03:11:51.068190+0000","last_became_active":"2023-12-01T03:11:51.064818+0000","last_became_peered":"2023-12-01T03:11:51.064818+0000","last_unstale":"2023-12-01T03:11:51.068190+0000","last_undegraded":"2023-12-01T03:11:51.068190+0000","last_fullsized":"2023-12-01T03:11:51.068190+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:28:12.383010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.055803+0000","last_change":"2023-12-01T03:11:50.591414+0000","last_active":"2023-12-01T03:11:51.055803+0000","last_peered":"2023-12-01T03:11:51.055803+0000","last_clean":"2023-12-01T03:11:51.055803+0000","last_became_active":"2023-12-01T03:11:50.591116+0000","last_became_peered":"2023-12-01T03:11:50.591116+0000","last_unstale":"2023-12-01T03:11:51.055803+0000","last_undegraded":"2023-12-01T03:11:51.055803+0000","last_fullsized":"2023-12-01T03:11:51.055803+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:51:00.345760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.568008+0000","last_change":"2023-12-01T03:11:32.214055+0000","last_active":"2023-12-01T03:11:50.568008+0000","last_peered":"2023-12-01T03:11:50.568008+0000","last_clean":"2023-12-01T03:11:50.568008+0000","last_became_active":"2023-12-01T03:11:32.213767+0000","last_became_peered":"2023-12-01T03:11:32.213767+0000","last_unstale":"2023-12-01T03:11:50.568008+0000","last_undegraded":"2023-12-01T03:11:50.568008+0000","last_fullsized":"2023-12-01T03:11:50.568008+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:47:03.446609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.564835+0000","last_change":"2023-12-01T03:11:31.676480+0000","last_active":"2023-12-01T03:11:50.564835+0000","last_peered":"2023-12-01T03:11:50.564835+0000","last_clean":"2023-12-01T03:11:50.564835+0000","last_became_active":"2023-12-01T03:11:31.676127+0000","last_became_peered":"2023-12-01T03:11:31.676127+0000","last_unstale":"2023-12-01T03:11:50.564835+0000","last_undegraded":"2023-12-01T03:11:50.564835+0000","last_fullsized":"2023-12-01T03:11:50.564835+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:44:17.083965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.070497+0000","last_change":"2023-12-01T03:11:51.069233+0000","last_active":"2023-12-01T03:11:51.070497+0000","last_peered":"2023-12-01T03:11:51.070497+0000","last_clean":"2023-12-01T03:11:51.070497+0000","last_became_active":"2023-12-01T03:11:51.068196+0000","last_became_peered":"2023-12-01T03:11:51.068196+0000","last_unstale":"2023-12-01T03:11:51.070497+0000","last_undegraded":"2023-12-01T03:11:51.070497+0000","last_fullsized":"2023-12-01T03:11:51.070497+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:34:04.408833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":10,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.054236+0000","last_change":"2023-12-01T03:11:50.594879+0000","last_active":"2023-12-01T03:11:51.054236+0000","last_peered":"2023-12-01T03:11:51.054236+0000","last_clean":"2023-12-01T03:11:51.054236+0000","last_became_active":"2023-12-01T03:11:50.594328+0000","last_became_peered":"2023-12-01T03:11:50.594328+0000","last_unstale":"2023-12-01T03:11:51.054236+0000","last_undegraded":"2023-12-01T03:11:51.054236+0000","last_fullsized":"2023-12-01T03:11:51.054236+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:47.552866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.591188+0000","last_change":"2023-12-01T03:11:32.203024+0000","last_active":"2023-12-01T03:11:50.591188+0000","last_peered":"2023-12-01T03:11:50.591188+0000","last_clean":"2023-12-01T03:11:50.591188+0000","last_became_active":"2023-12-01T03:11:32.202779+0000","last_became_peered":"2023-12-01T03:11:32.202779+0000","last_unstale":"2023-12-01T03:11:50.591188+0000","last_undegraded":"2023-12-01T03:11:50.591188+0000","last_fullsized":"2023-12-01T03:11:50.591188+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:25.249379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.580650+0000","last_change":"2023-12-01T03:11:31.680251+0000","last_active":"2023-12-01T03:11:50.580650+0000","last_peered":"2023-12-01T03:11:50.580650+0000","last_clean":"2023-12-01T03:11:50.580650+0000","last_became_active":"2023-12-01T03:11:31.679936+0000","last_became_peered":"2023-12-01T03:11:31.679936+0000","last_unstale":"2023-12-01T03:11:50.580650+0000","last_undegraded":"2023-12-01T03:11:50.580650+0000","last_fullsized":"2023-12-01T03:11:50.580650+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:47:39.296836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":9,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.081403+0000","last_change":"2023-12-01T03:11:51.080842+0000","last_active":"2023-12-01T03:11:51.081403+0000","last_peered":"2023-12-01T03:11:51.081403+0000","last_clean":"2023-12-01T03:11:51.081403+0000","last_became_active":"2023-12-01T03:11:51.080626+0000","last_became_peered":"2023-12-01T03:11:51.080626+0000","last_unstale":"2023-12-01T03:11:51.081403+0000","last_undegraded":"2023-12-01T03:11:51.081403+0000","last_fullsized":"2023-12-01T03:11:51.081403+0000","mapping_epoch":237,"log_start":"0'0","ondisk_log_start":"0'0","created":237,"last_epoch_clean":238,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:50.540678+0000","last_clean_scrub_stamp":"2023-12-01T03:11:50.540678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:02:55.273609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1f","version":"238'4","reported_seq":14,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-12-01T03:11:51.080614+0000","last_change":"2023-12-01T03:11:50.597722+0000","last_active":"2023-12-01T03:11:51.080614+0000","last_peered":"2023-12-01T03:11:51.080614+0000","last_clean":"2023-12-01T03:11:51.080614+0000","last_became_active":"2023-12-01T03:11:50.596938+0000","last_became_peered":"2023-12-01T03:11:50.596938+0000","last_unstale":"2023-12-01T03:11:51.080614+0000","last_undegraded":"2023-12-01T03:11:51.080614+0000","last_fullsized":"2023-12-01T03:11:51.080614+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:49.521575+0000","last_clean_scrub_stamp":"2023-12-01T03:11:49.521575+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:59:44.367550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.593756+0000","last_change":"2023-12-01T03:11:32.210270+0000","last_active":"2023-12-01T03:11:50.593756+0000","last_peered":"2023-12-01T03:11:50.593756+0000","last_clean":"2023-12-01T03:11:50.593756+0000","last_became_active":"2023-12-01T03:11:32.209831+0000","last_became_peered":"2023-12-01T03:11:32.209831+0000","last_unstale":"2023-12-01T03:11:50.593756+0000","last_undegraded":"2023-12-01T03:11:50.593756+0000","last_fullsized":"2023-12-01T03:11:50.593756+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":233,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:31.630683+0000","last_clean_scrub_stamp":"2023-12-01T03:11:31.630683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:37:32.333432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"234'2","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.582476+0000","last_change":"2023-12-01T03:11:31.681391+0000","last_active":"2023-12-01T03:11:50.582476+0000","last_peered":"2023-12-01T03:11:50.582476+0000","last_clean":"2023-12-01T03:11:50.582476+0000","last_became_active":"2023-12-01T03:11:31.680773+0000","last_became_peered":"2023-12-01T03:11:31.680773+0000","last_unstale":"2023-12-01T03:11:50.582476+0000","last_undegraded":"2023-12-01T03:11:50.582476+0000","last_fullsized":"2023-12-01T03:11:50.582476+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:11:30.614740+0000","last_clean_scrub_stamp":"2023-12-01T03:11:30.614740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:40:13.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"220'348","reported_seq":1217,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-01T03:11:50.580775+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:11:50.580775+0000","last_peered":"2023-12-01T03:11:50.580775+0000","last_clean":"2023-12-01T03:11:50.580775+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:11:50.580775+0000","last_undegraded":"2023-12-01T03:11:50.580775+0000","last_fullsized":"2023-12-01T03:11:50.580775+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575992,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60236,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311604,"statfs":{"total":96636764160,"available":96575082496,"internally_reserved":0,"allocated":3018752,"data_stored":1471908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.05,"15min":1.071},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5409999999999999,"5min":1.6000000000000001,"15min":1.7390000000000001},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6599999999999999,"5min":1.6699999999999999,"15min":1.7},"last":5.5800000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.236,"15min":1.2769999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4990000000000001,"5min":1.78,"15min":2.1419999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.294,"15min":1.3260000000000001},"min":{"1min":0.873,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5740000000000001,"5min":1.6240000000000001,"15min":1.8700000000000001},"last":1.137}]},{"osd":2,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1439999999999999,"15min":1.173},"min":{"1min":0.71699999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.573,"5min":1.7070000000000001,"15min":2.1179999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.23,"15min":1.2549999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.643,"5min":1.7050000000000001,"15min":2.1120000000000001},"last":1.8640000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2509999999999999,"15min":1.302},"min":{"1min":0.70099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.696,"5min":1.712,"15min":2.431},"last":4.2720000000000002},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3720000000000001,"15min":1.4119999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7529999999999999,"5min":1.879,"15min":2.198},"last":2.4860000000000002}]},{"osd":4,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3009999999999999,"15min":1.335},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7689999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.4510000000000001,"15min":1.49},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8799999999999999,"5min":1.97,"15min":2.569},"last":1.2629999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2869999999999999,"15min":1.3160000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7150000000000001,"5min":2.0099999999999998,"15min":2.7269999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.5469999999999999,"15min":1.585},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.0470000000000002,"5min":2.0569999999999999,"15min":2.484},"last":1.6679999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.403,"15min":1.4359999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.7749999999999999,"5min":1.921,"15min":2.532},"last":4.8579999999999997},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4299999999999999,"15min":1.4550000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8260000000000001,"5min":1.8819999999999999,"15min":2.762},"last":0.81799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575993,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42160,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94329680,"statfs":{"total":96636764160,"available":96593592320,"internally_reserved":0,"allocated":2990080,"data_stored":1461513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.0509999999999999,"15min":1.1080000000000001},"min":{"1min":0.66100000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.1040000000000001,"15min":1.157},"min":{"1min":0.78400000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6910000000000001,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":0.89400000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.141,"15min":1.2150000000000001},"min":{"1min":0.68999999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8460000000000001,"5min":1.976,"15min":2.0449999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1830000000000001,"15min":1.22},"min":{"1min":0.86099999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.6220000000000001,"5min":1.694,"15min":1.835},"last":1.044}]},{"osd":2,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.157,"15min":1.206},"min":{"1min":0.71799999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.464,"5min":1.742,"15min":1.976},"last":1.095},{"interface":"front","average":{"1min":1.131,"5min":1.222,"15min":1.278},"min":{"1min":0.71699999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.5009999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.288}]},{"osd":3,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2829999999999999,"15min":1.3140000000000001},"min":{"1min":0.89500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6839999999999999,"5min":1.927,"15min":2.0289999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":1.0069999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0190000000000001,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.93500000000000005}]},{"osd":4,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.2649999999999999,"15min":1.27},"min":{"1min":0.93600000000000005,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8440000000000001,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.498},{"interface":"front","average":{"1min":1.589,"5min":1.407,"15min":1.367},"min":{"1min":1.2090000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.222,"5min":2.48,"15min":2.48},"last":1.407}]},{"osd":5,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.3700000000000001,"15min":1.413},"min":{"1min":1.127,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.962,"5min":2.262,"15min":2.262},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.6830000000000001,"5min":1.5169999999999999,"15min":1.4970000000000001},"min":{"1min":1.2629999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.194,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.252}]},{"osd":7,"last update":"Fri Dec 1 03:11:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5029999999999999,"15min":1.5309999999999999},"min":{"1min":1.155,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1320000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.417},{"interface":"front","average":{"1min":1.746,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":1.371,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.2770000000000001,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.3580000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575992,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52620,"kb_used_data":3524,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319220,"statfs":{"total":96636764160,"available":96582881280,"internally_reserved":0,"allocated":3608576,"data_stored":2062423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1000000000000001,"15min":1.123},"min":{"1min":0.68300000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.7170000000000001,"5min":1.776,"15min":1.776},"last":1.194},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1539999999999999,"15min":1.161},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.752},"last":1.3879999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1399999999999999,"15min":1.1559999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6120000000000001,"5min":1.6519999999999999,"15min":1.696},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.198,"15min":1.2330000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":1.087}]},{"osd":2,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2270000000000001,"15min":1.2470000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.81399999999999995},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.167,"5min":1.2450000000000001,"15min":1.254},"min":{"1min":0.75,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.867,"5min":1.899,"15min":1.9219999999999999},"last":1.0169999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.3089999999999999,"15min":1.3149999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.889,"5min":1.889,"15min":1.8979999999999999},"last":1.498},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.329,"15min":1.359},"min":{"1min":1.0109999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.97,"5min":1.97,"15min":2.1859999999999999},"last":1.2909999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.869,"5min":1.869,"15min":2.6040000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.242}]},{"osd":6,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3779999999999999,"15min":1.377},"min":{"1min":0.99099999999999999,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.7649999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.409,"15min":1.4219999999999999},"min":{"1min":1.0149999999999999,"5min":0.95199999999999996,"15min":0.86099999999999999},"max":{"1min":2.032,"5min":2.032,"15min":2.577},"last":1.3149999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:02 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.534,"15min":1.538},"min":{"1min":1.2010000000000001,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":2.077,"5min":2.077,"15min":3.1709999999999998},"last":1.619},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.5760000000000001,"15min":1.5780000000000001},"min":{"1min":1.3200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.0710000000000002,"5min":2.101,"15min":2.8740000000000001},"last":1.7050000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575992,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52032,"kb_used_data":2936,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319808,"statfs":{"total":96636764160,"available":96583483392,"internally_reserved":0,"allocated":3006464,"data_stored":1471883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1200000000000001,"15min":1.1240000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8089999999999999,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.163},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.198,"15min":1.1899999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.5309999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.345}]},{"osd":1,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2290000000000001,"15min":1.256},"min":{"1min":0.83599999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8580000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.238},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1719999999999999,"15min":1.175},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.778,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.2909999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3300000000000001,"15min":1.244},"min":{"1min":0.74399999999999999,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.637,"5min":11.311,"15min":11.311},"last":1.387},{"interface":"front","average":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3320000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9330000000000001,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.4350000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.298,"15min":1.29},"min":{"1min":0.80000000000000004,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.716,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3759999999999999,"15min":1.3340000000000001},"min":{"1min":0.96399999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7430000000000001,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.357,"15min":1.3200000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.738,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.375,"15min":1.3160000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6579999999999999,"5min":3.71,"15min":3.71},"last":1.4750000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3799999999999999,"15min":1.3740000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8009999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4790000000000001,"15min":1.476},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.76,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.5249999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:57 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.514,"15min":1.488},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9830000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.573},{"interface":"front","average":{"1min":1.532,"5min":1.597,"15min":1.575},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.907,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.548}]}]},{"osd":3,"up_from":13,"seq":55834575991,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56112,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315728,"statfs":{"total":96636764160,"available":96579305472,"internally_reserved":0,"allocated":2990080,"data_stored":1469056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.306,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0449999999999999,"15min":1.0609999999999999},"min":{"1min":0.39500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.573,"5min":1.95,"15min":1.95},"last":1.161}]},{"osd":1,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.083,"15min":1.1479999999999999},"min":{"1min":0.56200000000000006,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.417,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.94399999999999995,"15min":0.98499999999999999},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6970000000000001,"5min":3.863,"15min":3.863},"last":1.198}]},{"osd":2,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.038,"15min":1.085},"min":{"1min":0.49099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.224,"5min":3.597,"15min":3.597},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.175,"15min":1.228},"min":{"1min":0.54700000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.647,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.0640000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.264,"15min":1.286},"min":{"1min":0.86099999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5269999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.825},{"interface":"front","average":{"1min":1.248,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.79500000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9970000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.9890000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3109999999999999,"15min":1.325},"min":{"1min":0.92200000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":1.8979999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.3420000000000001,"15min":1.3600000000000001},"min":{"1min":0.82499999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6699999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.7609999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.375,"15min":1.4370000000000001},"min":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":2.3079999999999998},{"interface":"front","average":{"1min":1.29,"5min":1.456,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.6879999999999999,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":2.0129999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4750000000000001,"15min":1.514},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.788,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4770000000000001,"15min":1.522},"min":{"1min":1.0549999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.913,"5min":3.004,"15min":3.004},"last":2.2440000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575991,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47928,"kb_used_data":2928,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323912,"statfs":{"total":96636764160,"available":96587685888,"internally_reserved":0,"allocated":2998272,"data_stored":1469160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.81100000000000005,"15min":0.87},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92100000000000004,"5min":1.411,"15min":1.4730000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98899999999999999,"15min":1.024},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.216,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.97299999999999998,"15min":1.0309999999999999},"min":{"1min":0.57899999999999996,"5min":0.378,"15min":0.378},"max":{"1min":1.512,"5min":1.512,"15min":1.698},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.04,"15min":1.113},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5229999999999999,"5min":1.635,"15min":1.6599999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.97499999999999998,"15min":1.022},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2889999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.097,"15min":1.181},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.4970000000000001,"5min":1.615,"15min":1.79},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2749999999999999,"15min":1.3120000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.621,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.276},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4730000000000001,"5min":1.714,"15min":1.835},"last":0.89500000000000002}]},{"osd":5,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.3109999999999999,"15min":1.3680000000000001},"min":{"1min":0.86599999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.639,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.347,"15min":1.393},"min":{"1min":0.79000000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.573,"5min":1.8740000000000001,"15min":2.016},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.351,"15min":1.367},"min":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.865,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.268,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.86599999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7090000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.0509999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4370000000000001,"15min":1.4630000000000001},"min":{"1min":0.94299999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5740000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.4350000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.8100000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.0189999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575990,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48516,"kb_used_data":3516,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323324,"statfs":{"total":96636764160,"available":96587083776,"internally_reserved":0,"allocated":3600384,"data_stored":2060680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.89800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.238,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.98499999999999999,"15min":1.038},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.512,"5min":1.5660000000000001,"15min":1.681},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.90300000000000002,"15min":0.93999999999999995},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2190000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.048,"5min":1.1020000000000001,"15min":1.141},"min":{"1min":0.52500000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5489999999999999,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":0.36399999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.208,"15min":1.2949999999999999},"min":{"1min":0.60799999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5860000000000001,"5min":1.587,"15min":1.788},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.02,"5min":1.0780000000000001,"15min":1.127},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.383,"5min":1.714,"15min":1.8049999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2030000000000001,"15min":1.218},"min":{"1min":0.84399999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.623,"5min":1.623,"15min":1.6859999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3700000000000001,"15min":1.4039999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.657,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":0.77800000000000002}]},{"osd":5,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3109999999999999,"15min":1.333},"min":{"1min":0.95899999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":2.101},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.298,"15min":1.331},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.9019999999999999,"5min":1.927,"15min":1.927},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.3979999999999999,"15min":1.4379999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0529999999999999,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3939999999999999,"15min":1.413},"min":{"1min":0.91600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.98699999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:10:59 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4650000000000001,"15min":1.522},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.9430000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.524,"5min":1.48,"15min":1.5109999999999999},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.0699999999999998,"5min":2.1389999999999998,"15min":2.2120000000000002},"last":0.94099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575992,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60216,"kb_used_data":2928,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311624,"statfs":{"total":96636764160,"available":96575102976,"internally_reserved":0,"allocated":2998272,"data_stored":1469148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.86399999999999999,"15min":0.88900000000000001},"min":{"1min":0.314,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.994},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.98899999999999999,"15min":1.048},"min":{"1min":0.58999999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6799999999999999},"last":1.113}]},{"osd":2,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0169999999999999,"15min":1.0640000000000001},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.48699999999999999},"max":{"1min":1.635,"5min":1.7669999999999999,"15min":2.1429999999999998},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0589999999999999,"15min":1.1259999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3100000000000001,"5min":1.726,"15min":1.792},"last":0.98599999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.036,"15min":1.0669999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.8839999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.171,"15min":1.224},"min":{"1min":0.70499999999999996,"5min":0.621,"15min":0.54200000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2.6469999999999998},"last":1.3580000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.296,"15min":1.3520000000000001},"min":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.681,"5min":1.8029999999999999,"15min":2.1400000000000001},"last":1.216},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.3029999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.286}]},{"osd":5,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.90700000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.899,"5min":1.899,"15min":2.7719999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.383,"15min":1.427},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.681,"5min":1.9730000000000001,"15min":2.5880000000000001},"last":1.3819999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3520000000000001,"15min":1.381},"min":{"1min":0.84599999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7270000000000001,"5min":1.8169999999999999,"15min":2.2029999999999998},"last":1.452},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.45,"15min":1.4830000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.419}]},{"osd":7,"last update":"Fri Dec 1 03:10:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4670000000000001,"15min":1.516},"min":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7969999999999999,"5min":2.0569999999999999,"15min":2.7040000000000002},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4730000000000001,"15min":1.4870000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.927,"5min":2.1190000000000002,"15min":2.431},"last":1.405}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:11:56.041 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T03:11:56.041 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:11:56.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.294+0000 7f3ec11ea640 1 Processor -- start 2023-12-01T03:11:56.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.295+0000 7f3ec11ea640 1 -- start start 2023-12-01T03:11:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.296+0000 7f3ec11ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ebc0c9ef0 0x7f3ebc06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.296+0000 7f3ec11ea640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3ebc06b6f0 0x7f3ebc070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.296+0000 7f3ec11ea640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc06f8b0 con 0x7f3ebc159dd0 2023-12-01T03:11:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.296+0000 7f3ec11ea640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ebc0adcb0 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.296+0000 7f3ec11ea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0ad9d0 con 0x7f3ebc0c9ef0 2023-12-01T03:11:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.296+0000 7f3ebb59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3ebc06b6f0 0x7f3ebc070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.296+0000 7f3ebb59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3ebc06b6f0 0x7f3ebc070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52826/0 (socket says 172.21.15.115:52826) 2023-12-01T03:11:56.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.297+0000 7f3ebb59e640 1 -- 172.21.15.115:0/4258290636 learned_addr learned my addr 172.21.15.115:0/4258290636 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:56.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.297+0000 7f3eb3fff640 1 --2- 172.21.15.115:0/4258290636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ebc0c9ef0 0x7f3ebc06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:56.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.297+0000 7f3ebb59e640 1 -- 172.21.15.115:0/4258290636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ebc0c9ef0 msgr2=0x7f3ebc06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:56.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.297+0000 7f3ebb59e640 1 --2- 172.21.15.115:0/4258290636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ebc0c9ef0 0x7f3ebc06fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.297+0000 7f3ebb59e640 1 -- 172.21.15.115:0/4258290636 >> v1:172.21.15.142:6789/0 conn(0x7f3ebc159dd0 legacy=0x7f3ebc0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:56.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.297+0000 7f3ebb59e640 1 -- 172.21.15.115:0/4258290636 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc0c9000 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.298+0000 7f3ebb59e640 1 --2- 172.21.15.115:0/4258290636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 0x7f3ebc070260 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009e50 tx=0x7f3eb402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123bfb2b76de316e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:56.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.298+0000 7f3eba59c640 1 -- 172.21.15.115:0/4258290636 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb402c9e0 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.298+0000 7f3eba59c640 1 -- 172.21.15.115:0/4258290636 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb402cb40 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.298+0000 7f3eba59c640 1 -- 172.21.15.115:0/4258290636 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb40343f0 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.299+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4258290636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 msgr2=0x7f3ebc070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:56.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.299+0000 7f3ec11ea640 1 --2- 172.21.15.115:0/4258290636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 0x7f3ebc070260 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009e50 tx=0x7f3eb402be40 comp rx=0 tx=0).stop 2023-12-01T03:11:56.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.299+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4258290636 shutdown_connections 2023-12-01T03:11:56.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.299+0000 7f3ec11ea640 1 --2- 172.21.15.115:0/4258290636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 0x7f3ebc070260 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.299+0000 7f3ec11ea640 1 --2- 172.21.15.115:0/4258290636 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ebc0c9ef0 0x7f3ebc06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.299+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4258290636 >> 172.21.15.115:0/4258290636 conn(0x7f3ebc0ce880 msgr2=0x7f3ebc0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:56.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.299+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4258290636 shutdown_connections 2023-12-01T03:11:56.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.299+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4258290636 wait complete. 2023-12-01T03:11:56.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.301+0000 7f3ec11ea640 1 Processor -- start 2023-12-01T03:11:56.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.301+0000 7f3ec11ea640 1 -- start start 2023-12-01T03:11:56.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.302+0000 7f3ec11ea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 0x7f3ebc147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:56.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.302+0000 7f3ec11ea640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ebc0c9ef0 0x7f3ebc15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:56.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.302+0000 7f3ec11ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ebc159dd0 0x7f3ebc15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:56.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.302+0000 7f3ec11ea640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc15af70 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.302+0000 7f3ec11ea640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc15b250 con 0x7f3ebc0c9ef0 2023-12-01T03:11:56.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.302+0000 7f3ec11ea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc15b530 con 0x7f3ebc159dd0 2023-12-01T03:11:56.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3ebad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 0x7f3ebc147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:56.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3ebad9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 0x7f3ebc147f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52836/0 (socket says 172.21.15.115:52836) 2023-12-01T03:11:56.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3ebad9d640 1 -- 172.21.15.115:0/4230608028 learned_addr learned my addr 172.21.15.115:0/4230608028 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:56.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3eb3fff640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ebc0c9ef0 0x7f3ebc15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:56.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3ebb59e640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ebc159dd0 0x7f3ebc15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:56.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3ebad9d640 1 -- 172.21.15.115:0/4230608028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ebc159dd0 msgr2=0x7f3ebc15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:56.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3ebad9d640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ebc159dd0 0x7f3ebc15ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3ebad9d640 1 -- 172.21.15.115:0/4230608028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ebc0c9ef0 msgr2=0x7f3ebc15a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:56.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3ebad9d640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ebc0c9ef0 0x7f3ebc15a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3ebad9d640 1 -- 172.21.15.115:0/4230608028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb4009b30 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.303+0000 7f3ebad9d640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 0x7f3ebc147f60 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c400 tx=0x7f3eac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:56.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.304+0000 7f3eb37fe640 1 -- 172.21.15.115:0/4230608028 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac0052d0 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.304+0000 7f3eb37fe640 1 -- 172.21.15.115:0/4230608028 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac005430 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.304+0000 7f3eb37fe640 1 -- 172.21.15.115:0/4230608028 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac003830 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.304+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4230608028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc15b810 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.304+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4230608028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc0c3940 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.305+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4230608028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc0cf560 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.306+0000 7f3eb37fe640 1 -- 172.21.15.115:0/4230608028 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f3eac003990 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.307+0000 7f3eb37fe640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3e9c06a670 0x7f3e9c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:56.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.307+0000 7f3eb37fe640 1 -- 172.21.15.115:0/4230608028 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f3eac0885b0 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.307+0000 7f3eb3fff640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3e9c06a670 0x7f3e9c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:56.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.308+0000 7f3eb3fff640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3e9c06a670 0x7f3e9c06cb30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0c8c50 tx=0x7f3ea4008040 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:56.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.311+0000 7f3eb37fe640 1 -- 172.21.15.115:0/4230608028 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3eac053580 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.538+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4230608028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ebc04e890 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.539+0000 7f3eb37fe640 1 -- 172.21.15.115:0/4230608028 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5786 (secure 0 0 0) 0x7f3eac056ba0 con 0x7f3ebc06b6f0 2023-12-01T03:11:56.541 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:11:56.542 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:32.176438+0000","modified":"2023-12-01T03:11:35.672576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7735":{"gid":7735,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1837882056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1837882056},{"type":"v1","addr":"172.21.15.142:6835","nonce":1837882056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:51.037525+0000","modified":"2023-12-01T03:11:54.055433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7732":{"gid":7732,"name":"b","rank":0,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-01T03:11:56.542 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 116 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4230608028 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3e9c06a670 msgr2=0x7f3e9c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3e9c06a670 0x7f3e9c06cb30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0c8c50 tx=0x7f3ea4008040 comp rx=0 tx=0).stop 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4230608028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 msgr2=0x7f3ebc147f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 0x7f3ebc147f60 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c400 tx=0x7f3eac00c8c0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4230608028 shutdown_connections 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3e9c06a670 0x7f3e9c06cb30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ebc159dd0 0x7f3ebc15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ebc0c9ef0 0x7f3ebc15a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 --2- 172.21.15.115:0/4230608028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ebc06b6f0 0x7f3ebc147f60 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4230608028 >> 172.21.15.115:0/4230608028 conn(0x7f3ebc0ce880 msgr2=0x7f3ebc0ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4230608028 shutdown_connections 2023-12-01T03:11:56.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:56.542+0000 7f3ec11ea640 1 -- 172.21.15.115:0/4230608028 wait complete. 2023-12-01T03:11:56.965 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 116, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T03:11:51.037525+0000', 'modified': '2023-12-01T03:11:54.055433+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7732}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7732': {'gid': 7732, 'name': 'b', 'rank': 0, 'incarnation': 112, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.142:6833/3596468657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 3596468657}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 3596468657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [31], 'metadata_pool': 30, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T03:11:56.965 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T03:11:56.965 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-01T03:11:57.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.189+0000 7f9b9f59e640 1 Processor -- start 2023-12-01T03:11:57.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.189+0000 7f9b9f59e640 1 -- start start 2023-12-01T03:11:57.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.190+0000 7f9b9f59e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9ba006b190 0x7f9ba00709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.190+0000 7f9b9f59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00702a0 0x7f9ba0070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.190+0000 7f9b9f59e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba00707a0 con 0x7f9ba0159d90 2023-12-01T03:11:57.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.190+0000 7f9b9f59e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba00c9380 con 0x7f9ba006b190 2023-12-01T03:11:57.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.190+0000 7f9b9f59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00b2f70 con 0x7f9ba00702a0 2023-12-01T03:11:57.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.191+0000 7f9b9dd9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9ba006b190 0x7f9ba00709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.191+0000 7f9b9dd9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9ba006b190 0x7f9ba00709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52846/0 (socket says 172.21.15.115:52846) 2023-12-01T03:11:57.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.191+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/1424808001 learned_addr learned my addr 172.21.15.115:0/1424808001 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:57.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.191+0000 7f9b9ed9d640 1 --2- 172.21.15.115:0/1424808001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00702a0 0x7f9ba0070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.191+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/1424808001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00702a0 msgr2=0x7f9ba0070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.191+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/1424808001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00702a0 0x7f9ba0070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.191+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/1424808001 >> v1:172.21.15.142:6789/0 conn(0x7f9ba0159d90 legacy=0x7f9ba00c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.191+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/1424808001 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba015e2e0 con 0x7f9ba006b190 2023-12-01T03:11:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.193+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/1424808001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba006b190 0x7f9ba00709a0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009870 tx=0x7f9b8802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a510e6175df61 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.193+0000 7f9b9d59a640 1 -- 172.21.15.115:0/1424808001 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b880317c0 con 0x7f9ba006b190 2023-12-01T03:11:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.193+0000 7f9b9d59a640 1 -- 172.21.15.115:0/1424808001 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b88031920 con 0x7f9ba006b190 2023-12-01T03:11:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.193+0000 7f9b9d59a640 1 -- 172.21.15.115:0/1424808001 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88031cb0 con 0x7f9ba006b190 2023-12-01T03:11:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.194+0000 7f9b9f59e640 1 -- 172.21.15.115:0/1424808001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba006b190 msgr2=0x7f9ba00709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.194+0000 7f9b9f59e640 1 --2- 172.21.15.115:0/1424808001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba006b190 0x7f9ba00709a0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009870 tx=0x7f9b8802b860 comp rx=0 tx=0).stop 2023-12-01T03:11:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.194+0000 7f9b9f59e640 1 -- 172.21.15.115:0/1424808001 shutdown_connections 2023-12-01T03:11:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.194+0000 7f9b9f59e640 1 --2- 172.21.15.115:0/1424808001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00702a0 0x7f9ba0070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.194+0000 7f9b9f59e640 1 --2- 172.21.15.115:0/1424808001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba006b190 0x7f9ba00709a0 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.194+0000 7f9b9f59e640 1 -- 172.21.15.115:0/1424808001 >> 172.21.15.115:0/1424808001 conn(0x7f9ba00ce620 msgr2=0x7f9ba00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:57.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.194+0000 7f9b9f59e640 1 -- 172.21.15.115:0/1424808001 shutdown_connections 2023-12-01T03:11:57.197 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.194+0000 7f9b9f59e640 1 -- 172.21.15.115:0/1424808001 wait complete. 2023-12-01T03:11:57.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.196+0000 7f9b9f59e640 1 Processor -- start 2023-12-01T03:11:57.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.196+0000 7f9b9f59e640 1 -- start start 2023-12-01T03:11:57.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.197+0000 7f9b9f59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba006b190 0x7f9ba00c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.197+0000 7f9b9f59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba00702a0 0x7f9ba00c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.197+0000 7f9b9f59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0159d90 0x7f9ba0154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.197+0000 7f9b9f59e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c61c0 con 0x7f9ba00702a0 2023-12-01T03:11:57.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.197+0000 7f9b9f59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c5e90 con 0x7f9ba0159d90 2023-12-01T03:11:57.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.197+0000 7f9b9f59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c2f40 con 0x7f9ba006b190 2023-12-01T03:11:57.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba00702a0 0x7f9ba00c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba00702a0 0x7f9ba00c2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52848/0 (socket says 172.21.15.115:52848) 2023-12-01T03:11:57.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/274951548 learned_addr learned my addr 172.21.15.115:0/274951548 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:57.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba006b190 0x7f9ba00c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9ed9d640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0159d90 0x7f9ba0154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/274951548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba006b190 msgr2=0x7f9ba00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba006b190 0x7f9ba00c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/274951548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0159d90 msgr2=0x7f9ba0154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0159d90 0x7f9ba0154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/274951548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b880094d0 con 0x7f9ba00702a0 2023-12-01T03:11:57.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.198+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba00702a0 0x7f9ba00c2a00 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f9b88000c00 tx=0x7f9b880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.199+0000 7f9b96ffd640 1 -- 172.21.15.115:0/274951548 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88032860 con 0x7f9ba00702a0 2023-12-01T03:11:57.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.199+0000 7f9b96ffd640 1 -- 172.21.15.115:0/274951548 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b880329c0 con 0x7f9ba00702a0 2023-12-01T03:11:57.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.199+0000 7f9b96ffd640 1 -- 172.21.15.115:0/274951548 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88032d70 con 0x7f9ba00702a0 2023-12-01T03:11:57.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.199+0000 7f9b9f59e640 1 -- 172.21.15.115:0/274951548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba00c3a10 con 0x7f9ba00702a0 2023-12-01T03:11:57.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.199+0000 7f9b9f59e640 1 -- 172.21.15.115:0/274951548 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba00c3450 con 0x7f9ba00702a0 2023-12-01T03:11:57.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.201+0000 7f9b96ffd640 1 -- 172.21.15.115:0/274951548 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f9b8804e450 con 0x7f9ba00702a0 2023-12-01T03:11:57.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.202+0000 7f9b96ffd640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9b8006a670 0x7f9b8006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.202+0000 7f9b96ffd640 1 -- 172.21.15.115:0/274951548 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f9b880ab3c0 con 0x7f9ba00702a0 2023-12-01T03:11:57.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.202+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9b8006a670 0x7f9b8006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.203+0000 7f9ba7c01740 1 Processor -- start 2023-12-01T03:11:57.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.203+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9b8006a670 0x7f9b8006cb30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9b90007bf0 tx=0x7f9b90007240 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.203+0000 7f9ba7c01740 1 -- start start 2023-12-01T03:11:57.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.204+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaaa1b00 0x5574eaaa1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.204+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa71a80 0x5574eaa9ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.204+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa75680 0x5574eaa9f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.204+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574eaa4d2f0 con 0x5574eaa75680 2023-12-01T03:11:57.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.204+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ea9e1f80 con 0x5574eaaa1b00 2023-12-01T03:11:57.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.204+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574ea9df710 con 0x5574eaa71a80 2023-12-01T03:11:57.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.204+0000 7f9b9ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa75680 0x5574eaa9f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.206 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.204+0000 7f9b9ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa75680 0x5574eaa9f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52850/0 (socket says 172.21.15.115:52850) 2023-12-01T03:11:57.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.204+0000 7f9b9ed9d640 1 -- 172.21.15.115:0/2321504725 learned_addr learned my addr 172.21.15.115:0/2321504725 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:57.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.205+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaaa1b00 0x5574eaaa1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.205+0000 7f9b9ed9d640 1 -- 172.21.15.115:0/2321504725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa71a80 msgr2=0x5574eaa9ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.205+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa71a80 0x5574eaa9ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.205+0000 7f9b9ed9d640 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa71a80 0x5574eaa9ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.205+0000 7f9b9ed9d640 1 -- 172.21.15.115:0/2321504725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaaa1b00 msgr2=0x5574eaaa1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.205+0000 7f9b9ed9d640 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaaa1b00 0x5574eaaa1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.205+0000 7f9b9ed9d640 1 -- 172.21.15.115:0/2321504725 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ea9a2690 con 0x5574eaa75680 2023-12-01T03:11:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.205+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaaa1b00 0x5574eaaa1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.205+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa71a80 0x5574eaa9ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:57.216 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.205+0000 7f9b9ed9d640 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa75680 0x5574eaa9f2b0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f9b980098d0 tx=0x7f9b98009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3faaf87ce4d21b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9b7ffff640 1 -- 172.21.15.115:0/2321504725 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9800c070 con 0x5574eaa75680 2023-12-01T03:11:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9b7ffff640 1 -- 172.21.15.115:0/2321504725 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9800e950 con 0x5574eaa75680 2023-12-01T03:11:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9b7ffff640 1 -- 172.21.15.115:0/2321504725 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9800ece0 con 0x5574eaa75680 2023-12-01T03:11:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9ba7c01740 1 -- 172.21.15.115:0/2321504725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa75680 msgr2=0x5574eaa9f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa75680 0x5574eaa9f2b0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f9b980098d0 tx=0x7f9b98009d90 comp rx=0 tx=0).stop 2023-12-01T03:11:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9ba7c01740 1 -- 172.21.15.115:0/2321504725 shutdown_connections 2023-12-01T03:11:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa71a80 0x5574eaa9ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa75680 0x5574eaa9f2b0 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/2321504725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaaa1b00 0x5574eaaa1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9ba7c01740 1 -- 172.21.15.115:0/2321504725 >> 172.21.15.115:0/2321504725 conn(0x5574eaa73d90 msgr2=0x5574eaabc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:57.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9ba7c01740 1 -- 172.21.15.115:0/2321504725 shutdown_connections 2023-12-01T03:11:57.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.206+0000 7f9ba7c01740 1 -- 172.21.15.115:0/2321504725 wait complete. 2023-12-01T03:11:57.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.207+0000 7f9ba7c01740 1 -- start start 2023-12-01T03:11:57.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.207+0000 7f9ba7c01740 1 Processor -- start 2023-12-01T03:11:57.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.208+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaa75680 0x5574eaa8c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.208+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa8ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.208+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa8b210 0x5574eaa8ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.208+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ea9db000 con 0x5574eaaa1b00 2023-12-01T03:11:57.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.208+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ea97c7d0 con 0x5574eaa75680 2023-12-01T03:11:57.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.208+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574ea97c470 con 0x5574eaa8b210 2023-12-01T03:11:57.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.208+0000 7f9ba7c01740 10 client.0 _ll_register_callbacks cb 0x5574ea9cb1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:11:57.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b9dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa8ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b9dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa8ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52864/0 (socket says 172.21.15.115:52864) 2023-12-01T03:11:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/1395596465 learned_addr learned my addr 172.21.15.115:0/1395596465 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/1395596465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa8b210 msgr2=0x5574eaa8ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa8b210 0x5574eaa8ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/1395596465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaa75680 msgr2=0x5574eaa8c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaa75680 0x5574eaa8c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/1395596465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b98009500 con 0x5574eaaa1b00 2023-12-01T03:11:57.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa8ccf0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f9b8802b8a0 tx=0x7f9b8807a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39366b58e2aa50b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/1395596465 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8807a460 con 0x5574eaaa1b00 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.209+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/1395596465 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f9b88081fe0 con 0x5574eaaa1b00 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9ba7c01740 1 -- 172.21.15.115:0/1395596465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574eaa05550 con 0x5574eaaa1b00 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/1395596465 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b880794f0 con 0x5574eaaa1b00 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9ba7c01740 10 client.11760 fetch_fsmap learned FSMap version 116 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9ba7c01740 1 -- 172.21.15.115:0/1395596465 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) v3 -- 0x5574ea9d9680 con 0x5574eaaa1b00 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/1395596465 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5574eaa05550 con 0x5574eaaa1b00 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/1395596465 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 116) v1 ==== 3665+0+0 (secure 0 0 0) 0x7f9b88081b50 con 0x5574eaaa1b00 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9ba7c01740 10 client.11760 fetch_fsmap finished waiting for FSMap version 116 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9ba7c01740 10 client.11760 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9ba7c01740 20 client.11760 populate_metadata read hostname 'smithi115' 2023-12-01T03:11:57.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.142:6832/3596468657,v1:172.21.15.142:6833/3596468657] conn(0x5574eaabbe40 0x5574eaad9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9ba7c01740 4 client.11760 mds_command: new command op to 7732 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T03:11:57.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.210+0000 7f9ba7c01740 1 -- 172.21.15.115:0/1395596465 --> [v2:172.21.15.142:6832/3596468657,v1:172.21.15.142:6833/3596468657] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574ea9cdb50 con 0x5574eaabbe40 2023-12-01T03:11:57.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.211+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.142:6832/3596468657,v1:172.21.15.142:6833/3596468657] conn(0x5574eaabbe40 0x5574eaad9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.212+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.142:6832/3596468657,v1:172.21.15.142:6833/3596468657] conn(0x5574eaabbe40 0x5574eaad9b50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aad8508a13c7d08 server_cookie=4b68ad255cdd914a in_seq=0 out_seq=0 2023-12-01T03:11:57.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.212+0000 7f9b7d7fa640 10 client.11760 ms_handle_connect on v2:172.21.15.142:6832/3596468657 2023-12-01T03:11:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.214+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/1395596465 <== mds.0 v2:172.21.15.142:6832/3596468657 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5574ea9cdb50 con 0x5574eaabbe40 2023-12-01T03:11:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.214+0000 7f9b7d7fa640 10 client.11760 handle_command_reply: tid=0 2023-12-01T03:11:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.214+0000 7f9ba7c01740 1 client.11760 shutdown 2023-12-01T03:11:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 1 -- 172.21.15.115:0/1395596465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 msgr2=0x5574eaa8ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa8ccf0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f9b8802b8a0 tx=0x7f9b8807a7c0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 1 -- 172.21.15.115:0/1395596465 shutdown_connections 2023-12-01T03:11:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.142:6832/3596468657,v1:172.21.15.142:6833/3596468657] conn(0x5574eaabbe40 0x5574eaad9b50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa8b210 0x5574eaa8ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa8ccf0 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/1395596465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaa75680 0x5574eaa8c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 1 -- 172.21.15.115:0/1395596465 >> 172.21.15.115:0/1395596465 conn(0x5574eaa71a80 msgr2=0x5574eaabfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9b7d7fa640 0 client.11760 ms_handle_reset on v2:172.21.15.142:6832/3596468657 2023-12-01T03:11:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 1 -- 172.21.15.115:0/1395596465 shutdown_connections 2023-12-01T03:11:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 1 -- 172.21.15.115:0/1395596465 wait complete. 2023-12-01T03:11:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.215+0000 7f9ba7c01740 20 client.11760 trim_cache size 0 max 16384 2023-12-01T03:11:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.231+0000 7f9ba7c01740 1 Processor -- start 2023-12-01T03:11:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.231+0000 7f9ba7c01740 1 -- start start 2023-12-01T03:11:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.232+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaa75680 0x5574eaa8c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.232+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa8b210 0x5574eaa8ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.232+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa9bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.232+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ea742f20 con 0x5574eaaa1b00 2023-12-01T03:11:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.232+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574eaa766c0 con 0x5574eaa75680 2023-12-01T03:11:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.232+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574eaabf1f0 con 0x5574eaa8b210 2023-12-01T03:11:57.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.232+0000 7f9b9ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa9bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa9bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52876/0 (socket says 172.21.15.115:52876) 2023-12-01T03:11:57.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9ed9d640 1 -- 172.21.15.115:0/122283049 learned_addr learned my addr 172.21.15.115:0/122283049 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:57.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/122283049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa8b210 0x5574eaa8ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9ed9d640 1 -- 172.21.15.115:0/122283049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa8b210 msgr2=0x5574eaa8ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9ed9d640 1 --2- 172.21.15.115:0/122283049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa8b210 0x5574eaa8ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/122283049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaa75680 0x5574eaa8c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9ed9d640 1 -- 172.21.15.115:0/122283049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaa75680 msgr2=0x5574eaa8c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9ed9d640 1 --2- 172.21.15.115:0/122283049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaa75680 0x5574eaa8c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9ed9d640 1 -- 172.21.15.115:0/122283049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ea97b9e0 con 0x5574eaaa1b00 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/122283049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaa75680 0x5574eaa8c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.233+0000 7f9b9ed9d640 1 --2- 172.21.15.115:0/122283049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa9bdc0 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f9b98006fd0 tx=0x7f9b98011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22148cb6ddf2b682 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9b94ff9640 1 -- 172.21.15.115:0/122283049 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9800eac0 con 0x5574eaaa1b00 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9b94ff9640 1 -- 172.21.15.115:0/122283049 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9800ec20 con 0x5574eaaa1b00 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9b94ff9640 1 -- 172.21.15.115:0/122283049 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b98004030 con 0x5574eaaa1b00 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9ba7c01740 1 -- 172.21.15.115:0/122283049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 msgr2=0x5574eaa9bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/122283049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa9bdc0 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f9b98006fd0 tx=0x7f9b98011000 comp rx=0 tx=0).stop 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9ba7c01740 1 -- 172.21.15.115:0/122283049 shutdown_connections 2023-12-01T03:11:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/122283049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa8b210 0x5574eaa8ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/122283049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaaa1b00 0x5574eaa9bdc0 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/122283049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaa75680 0x5574eaa8c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9ba7c01740 1 -- 172.21.15.115:0/122283049 >> 172.21.15.115:0/122283049 conn(0x5574eaa71a80 msgr2=0x5574eaa75f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:57.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9ba7c01740 1 -- 172.21.15.115:0/122283049 shutdown_connections 2023-12-01T03:11:57.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.234+0000 7f9ba7c01740 1 -- 172.21.15.115:0/122283049 wait complete. 2023-12-01T03:11:57.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.235+0000 7f9ba7c01740 1 -- start start 2023-12-01T03:11:57.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.235+0000 7f9ba7c01740 1 Processor -- start 2023-12-01T03:11:57.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.236+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa75680 0x5574eaa89930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.236+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa8b210 0x5574eaa89e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.236+0000 7f9ba7c01740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaaa1b00 0x5574eaa9cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.236+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574eaa8d450 con 0x5574eaa8b210 2023-12-01T03:11:57.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.236+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574eaa9df10 con 0x5574eaaa1b00 2023-12-01T03:11:57.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.236+0000 7f9ba7c01740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574eaa895f0 con 0x5574eaa75680 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.236+0000 7f9ba7c01740 10 client.0 _ll_register_callbacks cb 0x7f9b98009160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b9dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa8b210 0x5574eaa89e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b9dd9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa8b210 0x5574eaa89e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52878/0 (socket says 172.21.15.115:52878) 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/225494599 learned_addr learned my addr 172.21.15.115:0/225494599 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/225494599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa75680 msgr2=0x5574eaa89930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa75680 0x5574eaa89930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/225494599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaaa1b00 msgr2=0x5574eaa9cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaaa1b00 0x5574eaa9cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b9dd9b640 1 -- 172.21.15.115:0/225494599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574ea9a2690 con 0x5574eaa8b210 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b9dd9b640 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa8b210 0x5574eaa89e70 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f9b88079f90 tx=0x7f9b880ab040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32917a218f8d295a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.237+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/225494599 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b880794f0 con 0x5574eaa8b210 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/225494599 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f9b88081fe0 con 0x5574eaa8b210 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9ba7c01740 1 -- 172.21.15.115:0/225494599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b9800e950 con 0x5574eaa8b210 2023-12-01T03:11:57.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/225494599 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b88081b50 con 0x5574eaa8b210 2023-12-01T03:11:57.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9ba7c01740 10 client.11766 fetch_fsmap learned FSMap version 116 2023-12-01T03:11:57.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9ba7c01740 1 -- 172.21.15.115:0/225494599 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) v3 -- 0x5574ea9a0ef0 con 0x5574eaa8b210 2023-12-01T03:11:57.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/225494599 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9800e950 con 0x5574eaa8b210 2023-12-01T03:11:57.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/225494599 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 116) v1 ==== 3665+0+0 (secure 0 0 0) 0x7f9b880a9390 con 0x5574eaa8b210 2023-12-01T03:11:57.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9ba7c01740 10 client.11766 fetch_fsmap finished waiting for FSMap version 116 2023-12-01T03:11:57.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9ba7c01740 10 client.11766 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-01T03:11:57.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9ba7c01740 20 client.11766 populate_metadata read hostname 'smithi115' 2023-12-01T03:11:57.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.142:6832/3596468657,v1:172.21.15.142:6833/3596468657] conn(0x5574eab21900 0x5574eaa9d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9ba7c01740 4 client.11766 mds_command: new command op to 7732 tid=0[{"prefix": "status"}] 2023-12-01T03:11:57.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.238+0000 7f9ba7c01740 1 -- 172.21.15.115:0/225494599 --> [v2:172.21.15.142:6832/3596468657,v1:172.21.15.142:6833/3596468657] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5574ea9d4080 con 0x5574eab21900 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.239+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.142:6832/3596468657,v1:172.21.15.142:6833/3596468657] conn(0x5574eab21900 0x5574eaa9d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.240+0000 7f9b9e59c640 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.142:6832/3596468657,v1:172.21.15.142:6833/3596468657] conn(0x5574eab21900 0x5574eaa9d4c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c82088827f56463 server_cookie=29ab90b9d6a77029 in_seq=0 out_seq=0 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.240+0000 7f9b7d7fa640 10 client.11766 ms_handle_connect on v2:172.21.15.142:6832/3596468657 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.240+0000 7f9b7d7fa640 1 -- 172.21.15.115:0/225494599 <== mds.0 v2:172.21.15.142:6832/3596468657 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+337 (crc 0 0 0) 0x5574ea9d4080 con 0x5574eab21900 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.240+0000 7f9b7d7fa640 10 client.11766 handle_command_reply: tid=0 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.240+0000 7f9ba7c01740 1 client.11766 shutdown 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 1 -- 172.21.15.115:0/225494599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa8b210 msgr2=0x5574eaa89e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa8b210 0x5574eaa89e70 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f9b88079f90 tx=0x7f9b880ab040 comp rx=0 tx=0).stop 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 1 -- 172.21.15.115:0/225494599 shutdown_connections 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5574eaaa1b00 0x5574eaa9cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x5574eaa8b210 0x5574eaa89e70 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.142:6832/3596468657,v1:172.21.15.142:6833/3596468657] conn(0x5574eab21900 0x5574eaa9d4c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 1 --2- 172.21.15.115:0/225494599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5574eaa75680 0x5574eaa89930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 1 -- 172.21.15.115:0/225494599 >> 172.21.15.115:0/225494599 conn(0x5574eaa71a80 msgr2=0x5574eaaddb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9b7d7fa640 0 client.11766 ms_handle_reset on v2:172.21.15.142:6832/3596468657 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 1 -- 172.21.15.115:0/225494599 shutdown_connections 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 1 -- 172.21.15.115:0/225494599 wait complete. 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.241+0000 7f9ba7c01740 20 client.11766 trim_cache size 0 max 16384 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 7732, 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T03:11:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "backup_fs", 2023-12-01T03:11:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 6.1961476600000003, 2023-12-01T03:11:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 116, 2023-12-01T03:11:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 239, 2023-12-01T03:11:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 239, 2023-12-01T03:11:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 30.406022034999999 2023-12-01T03:11:57.246 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 -- 172.21.15.115:0/274951548 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9b8006a670 msgr2=0x7f9b8006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9b8006a670 0x7f9b8006cb30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9b90007bf0 tx=0x7f9b90007240 comp rx=0 tx=0).stop 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 -- 172.21.15.115:0/274951548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba00702a0 msgr2=0x7f9ba00c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba00702a0 0x7f9ba00c2a00 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f9b88000c00 tx=0x7f9b880046c0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 -- 172.21.15.115:0/274951548 shutdown_connections 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9b8006a670 0x7f9b8006cb30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba0159d90 0x7f9ba0154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ba00702a0 0x7f9ba00c2a00 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 --2- 172.21.15.115:0/274951548 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba006b190 0x7f9ba00c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 -- 172.21.15.115:0/274951548 >> 172.21.15.115:0/274951548 conn(0x7f9ba00ce620 msgr2=0x7f9ba015cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 -- 172.21.15.115:0/274951548 shutdown_connections 2023-12-01T03:11:57.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.242+0000 7f9b9f59e640 1 -- 172.21.15.115:0/274951548 wait complete. 2023-12-01T03:11:57.262 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T03:11:57.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.475+0000 7f0c5e3ea640 1 Processor -- start 2023-12-01T03:11:57.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.475+0000 7f0c5e3ea640 1 -- start start 2023-12-01T03:11:57.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.476+0000 7f0c5e3ea640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0c580c9ef0 0x7f0c5806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.477+0000 7f0c5e3ea640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5806f8b0 con 0x7f0c58159dd0 2023-12-01T03:11:57.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.477+0000 7f0c5e3ea640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c580adcb0 con 0x7f0c5806b6f0 2023-12-01T03:11:57.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.477+0000 7f0c5e3ea640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c580ad9d0 con 0x7f0c580c9ef0 2023-12-01T03:11:57.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.477+0000 7f0c57fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0c58159dd0 0x7f0c580c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58826/0 (socket says 172.21.15.115:58826) 2023-12-01T03:11:57.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.477+0000 7f0c57fff640 1 -- 172.21.15.115:0/2227376887 learned_addr learned my addr 172.21.15.115:0/2227376887 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:57.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.477+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503983333 0 0) 0x7f0c5806f8b0 con 0x7f0c58159dd0 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.477+0000 7f0c577fe640 1 --2- 172.21.15.115:0/2227376887 >> v2:172.21.15.142:3300/0 conn(0x7f0c580c9ef0 0x7f0c5806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.477+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c40003520 con 0x7f0c58159dd0 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 210992720 0 0) 0x7f0c40003520 con 0x7f0c58159dd0 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 >> v2:172.21.15.142:3300/0 conn(0x7f0c580c9ef0 msgr2=0x7f0c5806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 --2- 172.21.15.115:0/2227376887 >> v2:172.21.15.142:3300/0 conn(0x7f0c580c9ef0 0x7f0c5806fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 >> v1:172.21.15.142:6789/0 conn(0x7f0c5806b6f0 legacy=0x7f0c58070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c580c9000 con 0x7f0c58159dd0 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0c48001670 con 0x7f0c58159dd0 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 >> v1:172.21.15.115:6789/0 conn(0x7f0c58159dd0 legacy=0x7f0c580c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 --2- 172.21.15.115:0/2227376887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40003a30 0x7f0c40003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 --2- 172.21.15.115:0/2227376887 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c400047c0 0x7f0c40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 --2- 172.21.15.115:0/2227376887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c40005360 0x7f0c4000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580ad9d0 con 0x7f0c40003a30 2023-12-01T03:11:57.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4000e4b0 con 0x7f0c40005360 2023-12-01T03:11:57.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4000e670 con 0x7f0c400047c0 2023-12-01T03:11:57.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c57fff640 1 --2- 172.21.15.115:0/2227376887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40003a30 0x7f0c40003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c57fff640 1 -- 172.21.15.115:0/2227376887 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c400047c0 msgr2=0x7f0c40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c57fff640 1 --2- 172.21.15.115:0/2227376887 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c400047c0 0x7f0c40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c57fff640 1 -- 172.21.15.115:0/2227376887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c40005360 msgr2=0x7f0c4000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c57fff640 1 --2- 172.21.15.115:0/2227376887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c40005360 0x7f0c4000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c57fff640 1 -- 172.21.15.115:0/2227376887 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c40003520 con 0x7f0c40003a30 2023-12-01T03:11:57.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.478+0000 7f0c5c987640 1 -- 172.21.15.115:0/2227376887 reap_dead start 2023-12-01T03:11:57.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.480+0000 7f0c57fff640 1 --2- 172.21.15.115:0/2227376887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40003a30 0x7f0c40003e30 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800c190 tx=0x7f0c400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf9571db7a84a84 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.480+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48005ad0 con 0x7f0c40003a30 2023-12-01T03:11:57.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.480+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c48005c30 con 0x7f0c40003a30 2023-12-01T03:11:57.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.480+0000 7f0c56ffd640 1 -- 172.21.15.115:0/2227376887 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4802c810 con 0x7f0c40003a30 2023-12-01T03:11:57.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.480+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/2227376887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40003a30 msgr2=0x7f0c40003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.481+0000 7f0c5e3ea640 1 --2- 172.21.15.115:0/2227376887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40003a30 0x7f0c40003e30 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800c190 tx=0x7f0c400057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.481+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/2227376887 shutdown_connections 2023-12-01T03:11:57.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.481+0000 7f0c5e3ea640 1 --2- 172.21.15.115:0/2227376887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c40003a30 0x7f0c40003e30 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.481+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/2227376887 >> 172.21.15.115:0/2227376887 conn(0x7f0c580ce880 msgr2=0x7f0c580cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:57.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.481+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/2227376887 shutdown_connections 2023-12-01T03:11:57.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.481+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/2227376887 wait complete. 2023-12-01T03:11:57.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.483+0000 7f0c5e3ea640 1 Processor -- start 2023-12-01T03:11:57.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.483+0000 7f0c5e3ea640 1 -- start start 2023-12-01T03:11:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.484+0000 7f0c5e3ea640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c40003a30 0x7f0c58148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.484+0000 7f0c5e3ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c580c5950 0x7f0c581577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.484+0000 7f0c5e3ea640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c580c5dd0 0x7f0c58157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.484+0000 7f0c5e3ea640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580c6fb0 con 0x7f0c580c5dd0 2023-12-01T03:11:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.484+0000 7f0c5e3ea640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580c63a0 con 0x7f0c40003a30 2023-12-01T03:11:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.484+0000 7f0c5e3ea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580c6680 con 0x7f0c580c5950 2023-12-01T03:11:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.484+0000 7f0c5c987640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c580c5dd0 0x7f0c58157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.484+0000 7f0c5c987640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c580c5dd0 0x7f0c58157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52894/0 (socket says 172.21.15.115:52894) 2023-12-01T03:11:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.484+0000 7f0c5c987640 1 -- 172.21.15.115:0/181653817 learned_addr learned my addr 172.21.15.115:0/181653817 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:57.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.484+0000 7f0c57fff640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c40003a30 0x7f0c58148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c577fe640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c580c5950 0x7f0c581577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c5c987640 1 -- 172.21.15.115:0/181653817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c580c5950 msgr2=0x7f0c581577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c5c987640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c580c5950 0x7f0c581577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c5c987640 1 -- 172.21.15.115:0/181653817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c40003a30 msgr2=0x7f0c58148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c5c987640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c40003a30 0x7f0c58148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c5c987640 1 -- 172.21.15.115:0/181653817 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c580c6960 con 0x7f0c580c5dd0 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c57fff640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c40003a30 0x7f0c58148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c577fe640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c580c5950 0x7f0c581577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c5c987640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c580c5dd0 0x7f0c58157ce0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f0c58066e30 tx=0x7f0c5000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c54ff9640 1 -- 172.21.15.115:0/181653817 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c500070d0 con 0x7f0c580c5dd0 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c54ff9640 1 -- 172.21.15.115:0/181653817 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c50007230 con 0x7f0c580c5dd0 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c54ff9640 1 -- 172.21.15.115:0/181653817 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50007620 con 0x7f0c580c5dd0 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/181653817 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c580c6c40 con 0x7f0c580c5dd0 2023-12-01T03:11:57.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.485+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/181653817 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c58156150 con 0x7f0c580c5dd0 2023-12-01T03:11:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.486+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/181653817 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c580c7160 con 0x7f0c580c5dd0 2023-12-01T03:11:57.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.487+0000 7f0c54ff9640 1 -- 172.21.15.115:0/181653817 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f0c50007780 con 0x7f0c580c5dd0 2023-12-01T03:11:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.489+0000 7f0c54ff9640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c4406a670 0x7f0c4406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.489+0000 7f0c54ff9640 1 -- 172.21.15.115:0/181653817 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f0c500886b0 con 0x7f0c580c5dd0 2023-12-01T03:11:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.489+0000 7f0c57fff640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c4406a670 0x7f0c4406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.490+0000 7f0c57fff640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c4406a670 0x7f0c4406cb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0c48034000 tx=0x7f0c48009030 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.494+0000 7f0c54ff9640 1 -- 172.21.15.115:0/181653817 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0c580c7160 con 0x7f0c580c5dd0 2023-12-01T03:11:57.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.709+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/181653817 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c5806d0d0 con 0x7f0c4406a670 2023-12-01T03:11:57.714 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.712+0000 7f0c54ff9640 1 -- 172.21.15.115:0/181653817 <== mgr.11121 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f0c480059c0 con 0x7f0c4406a670 2023-12-01T03:11:57.714 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T03:11:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T03:11:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:11:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:11:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "backup_fs" 2023-12-01T03:11:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:11:57.715 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T03:11:57.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/181653817 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c4406a670 msgr2=0x7f0c4406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c4406a670 0x7f0c4406cb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0c48034000 tx=0x7f0c48009030 comp rx=0 tx=0).stop 2023-12-01T03:11:57.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/181653817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c580c5dd0 msgr2=0x7f0c58157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c580c5dd0 0x7f0c58157ce0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f0c58066e30 tx=0x7f0c5000c960 comp rx=0 tx=0).stop 2023-12-01T03:11:57.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/181653817 shutdown_connections 2023-12-01T03:11:57.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c4406a670 0x7f0c4406cb30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c580c5950 0x7f0c581577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c580c5dd0 0x7f0c58157ce0 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 --2- 172.21.15.115:0/181653817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c40003a30 0x7f0c58148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/181653817 >> 172.21.15.115:0/181653817 conn(0x7f0c580ce880 msgr2=0x7f0c58158de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:57.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/181653817 shutdown_connections 2023-12-01T03:11:57.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.715+0000 7f0c5e3ea640 1 -- 172.21.15.115:0/181653817 wait complete. 2023-12-01T03:11:57.737 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T03:11:57.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.946+0000 7fe799de7640 1 Processor -- start 2023-12-01T03:11:57.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.946+0000 7fe799de7640 1 -- start start 2023-12-01T03:11:57.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.948+0000 7fe799de7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe79415b0e0 0x7fe794164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.948+0000 7fe799de7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7940b2f30 con 0x7fe794142870 2023-12-01T03:11:57.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.948+0000 7fe799de7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7940b2320 con 0x7fe7940cd1e0 2023-12-01T03:11:57.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.948+0000 7fe799de7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940b3ae0 con 0x7fe79415b0e0 2023-12-01T03:11:57.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.948+0000 7fe7937fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe794142870 0x7fe7940ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56288/0 (socket says 172.21.15.115:56288) 2023-12-01T03:11:57.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.948+0000 7fe7937fe640 1 -- 172.21.15.115:0/2865104027 learned_addr learned my addr 172.21.15.115:0/2865104027 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.948+0000 7fe793fff640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe79415b0e0 0x7fe794164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.948+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362340892 0 0) 0x7fe7940b2f30 con 0x7fe794142870 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.948+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe780003520 con 0x7fe794142870 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2364126323 0 0) 0x7fe780003520 con 0x7fe794142870 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe79415b0e0 msgr2=0x7fe794164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe79415b0e0 0x7fe794164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 >> v1:172.21.15.142:6789/0 conn(0x7fe7940cd1e0 legacy=0x7fe7940b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940b2600 con 0x7fe794142870 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fe77c001670 con 0x7fe794142870 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 >> v1:172.21.15.115:6789/0 conn(0x7fe794142870 legacy=0x7fe7940ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7800039a0 0x7fe780003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7800047c0 0x7fe780004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe780005140 0x7fe78000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940b3ae0 con 0x7fe7800039a0 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78000e0d0 con 0x7fe780005140 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78000e210 con 0x7fe7800047c0 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.949+0000 7fe7937fe640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7800039a0 0x7fe780003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.950+0000 7fe792ffd640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7800047c0 0x7fe780004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.950+0000 7fe7937fe640 1 -- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7800047c0 msgr2=0x7fe780004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.950+0000 7fe7937fe640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7800047c0 0x7fe780004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.950+0000 7fe7937fe640 1 -- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe780005140 msgr2=0x7fe78000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.950+0000 7fe7937fe640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe780005140 0x7fe78000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.950+0000 7fe7937fe640 1 -- 172.21.15.115:0/2865104027 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe780003520 con 0x7fe7800039a0 2023-12-01T03:11:57.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.950+0000 7fe792ffd640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7800047c0 0x7fe780004c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:57.954 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.951+0000 7fe7937fe640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7800039a0 0x7fe780003dc0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00c240 tx=0x7fe7800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8eb75e4b167b231 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.951+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c005ad0 con 0x7fe7800039a0 2023-12-01T03:11:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.951+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c005c30 con 0x7fe7800039a0 2023-12-01T03:11:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.951+0000 7fe7927fc640 1 -- 172.21.15.115:0/2865104027 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c02c8a0 con 0x7fe7800039a0 2023-12-01T03:11:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.951+0000 7fe799de7640 1 -- 172.21.15.115:0/2865104027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7800039a0 msgr2=0x7fe780003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.951+0000 7fe799de7640 1 --2- 172.21.15.115:0/2865104027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7800039a0 0x7fe780003dc0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00c240 tx=0x7fe7800057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.951+0000 7fe793fff640 1 -- 172.21.15.115:0/2865104027 reap_dead start 2023-12-01T03:11:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.952+0000 7fe799de7640 1 -- 172.21.15.115:0/2865104027 shutdown_connections 2023-12-01T03:11:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.952+0000 7fe799de7640 1 -- 172.21.15.115:0/2865104027 >> 172.21.15.115:0/2865104027 conn(0x7fe794065960 msgr2=0x7fe794065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.952+0000 7fe799de7640 1 -- 172.21.15.115:0/2865104027 shutdown_connections 2023-12-01T03:11:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.952+0000 7fe799de7640 1 -- 172.21.15.115:0/2865104027 wait complete. 2023-12-01T03:11:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.953+0000 7fe799de7640 1 Processor -- start 2023-12-01T03:11:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.954+0000 7fe799de7640 1 -- start start 2023-12-01T03:11:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.954+0000 7fe799de7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7800039a0 0x7fe794150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.954+0000 7fe799de7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794150fc0 0x7fe79414f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.954+0000 7fe799de7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79414f5a0 0x7fe79414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.954+0000 7fe799de7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7941ee7f0 con 0x7fe79414f5a0 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.954+0000 7fe799de7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7941514d0 con 0x7fe794150fc0 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.954+0000 7fe799de7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7941edec0 con 0x7fe7800039a0 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe793fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79414f5a0 0x7fe79414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe793fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79414f5a0 0x7fe79414fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46810/0 (socket says 172.21.15.115:46810) 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe793fff640 1 -- 172.21.15.115:0/2430873401 learned_addr learned my addr 172.21.15.115:0/2430873401 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe7937fe640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7800039a0 0x7fe794150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe793fff640 1 -- 172.21.15.115:0/2430873401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7800039a0 msgr2=0x7fe794150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe793fff640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7800039a0 0x7fe794150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe792ffd640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794150fc0 0x7fe79414f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe793fff640 1 -- 172.21.15.115:0/2430873401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794150fc0 msgr2=0x7fe79414f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe793fff640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794150fc0 0x7fe79414f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe793fff640 1 -- 172.21.15.115:0/2430873401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7941ee1a0 con 0x7fe79414f5a0 2023-12-01T03:11:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.955+0000 7fe793fff640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79414f5a0 0x7fe79414fa00 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7fe79406efe0 tx=0x7fe78c00e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.956+0000 7fe78bfff640 1 -- 172.21.15.115:0/2430873401 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c013680 con 0x7fe79414f5a0 2023-12-01T03:11:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.956+0000 7fe78bfff640 1 -- 172.21.15.115:0/2430873401 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c0137e0 con 0x7fe79414f5a0 2023-12-01T03:11:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.956+0000 7fe78bfff640 1 -- 172.21.15.115:0/2430873401 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c013b90 con 0x7fe79414f5a0 2023-12-01T03:11:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.956+0000 7fe799de7640 1 -- 172.21.15.115:0/2430873401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7941ee480 con 0x7fe79414f5a0 2023-12-01T03:11:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.956+0000 7fe799de7640 1 -- 172.21.15.115:0/2430873401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe794151850 con 0x7fe79414f5a0 2023-12-01T03:11:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.957+0000 7fe799de7640 1 -- 172.21.15.115:0/2430873401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7941ee9a0 con 0x7fe79414f5a0 2023-12-01T03:11:57.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.958+0000 7fe78bfff640 1 -- 172.21.15.115:0/2430873401 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 134) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fe78c013cf0 con 0x7fe79414f5a0 2023-12-01T03:11:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.959+0000 7fe78bfff640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe77406a6a0 0x7fe77406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.959+0000 7fe78bfff640 1 -- 172.21.15.115:0/2430873401 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fe78c088d10 con 0x7fe79414f5a0 2023-12-01T03:11:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.959+0000 7fe7937fe640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe77406a6a0 0x7fe77406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:57.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.960+0000 7fe7937fe640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe77406a6a0 0x7fe77406cb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe77c034000 tx=0x7fe77c009030 comp rx=0 tx=0).ready entity=mgr.11121 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:57.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:57.963+0000 7fe78bfff640 1 -- 172.21.15.115:0/2430873401 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe7941ee9a0 con 0x7fe79414f5a0 2023-12-01T03:11:58.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.201+0000 7fe799de7640 1 -- 172.21.15.115:0/2430873401 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe79404e890 con 0x7fe79414f5a0 2023-12-01T03:11:58.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.488+0000 7fe78bfff640 1 -- 172.21.15.115:0/2430873401 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 135) v1 ==== 88828+0+0 (secure 0 0 0) 0x7fe78c053fd0 con 0x7fe79414f5a0 2023-12-01T03:11:58.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.490+0000 7fe78bfff640 1 -- 172.21.15.115:0/2430873401 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v135) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe78c053c00 con 0x7fe79414f5a0 2023-12-01T03:11:58.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 -- 172.21.15.115:0/2430873401 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe77406a6a0 msgr2=0x7fe77406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:58.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe77406a6a0 0x7fe77406cb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe77c034000 tx=0x7fe77c009030 comp rx=0 tx=0).stop 2023-12-01T03:11:58.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 -- 172.21.15.115:0/2430873401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79414f5a0 msgr2=0x7fe79414fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79414f5a0 0x7fe79414fa00 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7fe79406efe0 tx=0x7fe78c00e9f0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 -- 172.21.15.115:0/2430873401 shutdown_connections 2023-12-01T03:11:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe77406a6a0 0x7fe77406cb60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79414f5a0 0x7fe79414fa00 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe794150fc0 0x7fe79414f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 --2- 172.21.15.115:0/2430873401 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7800039a0 0x7fe794150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 -- 172.21.15.115:0/2430873401 >> 172.21.15.115:0/2430873401 conn(0x7fe794065960 msgr2=0x7fe79413f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.493+0000 7fe799de7640 1 -- 172.21.15.115:0/2430873401 shutdown_connections 2023-12-01T03:11:58.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.494+0000 7fe799de7640 1 -- 172.21.15.115:0/2430873401 wait complete. 2023-12-01T03:11:58.517 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T03:11:58.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.713+0000 7f867d22c640 1 Processor -- start 2023-12-01T03:11:58.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.714+0000 7f867d22c640 1 -- start start 2023-12-01T03:11:58.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.715+0000 7f867d22c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f867815cd90 0x7f86780cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:58.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.715+0000 7f867d22c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86780b5fa0 0x7f8678161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:58.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.715+0000 7f867d22c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f86780cf650 0x7f86780cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:58.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.715+0000 7f867d22c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86780d0ec0 con 0x7f86780b5fa0 2023-12-01T03:11:58.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.715+0000 7f867d22c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86780ce5e0 con 0x7f86780cf650 2023-12-01T03:11:58.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.715+0000 7f867d22c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8678161950 con 0x7f867815cd90 2023-12-01T03:11:58.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.715+0000 7f867659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86780b5fa0 0x7f8678161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:58.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.715+0000 7f867659c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f86780b5fa0 0x7f8678161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46832/0 (socket says 172.21.15.115:46832) 2023-12-01T03:11:58.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.715+0000 7f867659c640 1 -- 172.21.15.115:0/1181824170 learned_addr learned my addr 172.21.15.115:0/1181824170 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:58.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.716+0000 7f8676d9d640 1 --2- 172.21.15.115:0/1181824170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f867815cd90 0x7f86780cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:58.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.716+0000 7f867659c640 1 -- 172.21.15.115:0/1181824170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f867815cd90 msgr2=0x7f86780cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:58.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.716+0000 7f867759e640 1 --2- 172.21.15.115:0/1181824170 >> v2:172.21.15.142:3300/0 conn(0x7f86780cf650 0x7f86780cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:58.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.716+0000 7f867659c640 1 --2- 172.21.15.115:0/1181824170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f867815cd90 0x7f86780cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.716+0000 7f867659c640 1 -- 172.21.15.115:0/1181824170 >> v2:172.21.15.142:3300/0 conn(0x7f86780cf650 msgr2=0x7f86780cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:58.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.716+0000 7f867659c640 1 --2- 172.21.15.115:0/1181824170 >> v2:172.21.15.142:3300/0 conn(0x7f86780cf650 0x7f86780cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.716+0000 7f867659c640 1 -- 172.21.15.115:0/1181824170 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780d0000 con 0x7f86780b5fa0 2023-12-01T03:11:58.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.717+0000 7f867659c640 1 --2- 172.21.15.115:0/1181824170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 0x7f8678161410 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f86780cd3a0 tx=0x7f866c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b0f0b8725453a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:58.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.717+0000 7f8675d9b640 1 -- 172.21.15.115:0/1181824170 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c031860 con 0x7f86780b5fa0 2023-12-01T03:11:58.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.717+0000 7f8675d9b640 1 -- 172.21.15.115:0/1181824170 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c0319c0 con 0x7f86780b5fa0 2023-12-01T03:11:58.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.717+0000 7f8675d9b640 1 -- 172.21.15.115:0/1181824170 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c031d50 con 0x7f86780b5fa0 2023-12-01T03:11:58.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.718+0000 7f867d22c640 1 -- 172.21.15.115:0/1181824170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 msgr2=0x7f8678161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:58.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.718+0000 7f867d22c640 1 --2- 172.21.15.115:0/1181824170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 0x7f8678161410 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f86780cd3a0 tx=0x7f866c02b900 comp rx=0 tx=0).stop 2023-12-01T03:11:58.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.718+0000 7f867d22c640 1 -- 172.21.15.115:0/1181824170 shutdown_connections 2023-12-01T03:11:58.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.718+0000 7f867d22c640 1 --2- 172.21.15.115:0/1181824170 >> v2:172.21.15.142:3300/0 conn(0x7f86780cf650 0x7f86780cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.718+0000 7f867d22c640 1 --2- 172.21.15.115:0/1181824170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 0x7f8678161410 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.718+0000 7f867d22c640 1 --2- 172.21.15.115:0/1181824170 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f867815cd90 0x7f86780cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.718+0000 7f867d22c640 1 -- 172.21.15.115:0/1181824170 >> 172.21.15.115:0/1181824170 conn(0x7f8678065960 msgr2=0x7f8678065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:58.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.718+0000 7f867d22c640 1 -- 172.21.15.115:0/1181824170 shutdown_connections 2023-12-01T03:11:58.721 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.718+0000 7f867d22c640 1 -- 172.21.15.115:0/1181824170 wait complete. 2023-12-01T03:11:58.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.721+0000 7f867d22c640 1 Processor -- start 2023-12-01T03:11:58.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.721+0000 7f867d22c640 1 -- start start 2023-12-01T03:11:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f867d22c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 0x7f8678153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f867d22c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86780cf650 0x7f86780cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f867d22c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867815cd90 0x7f86780b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f867d22c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86781f2480 con 0x7f86780b5fa0 2023-12-01T03:11:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f867d22c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86781f26e0 con 0x7f867815cd90 2023-12-01T03:11:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f867d22c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86781f29c0 con 0x7f86780cf650 2023-12-01T03:11:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f8676d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 0x7f8678153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f8676d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 0x7f8678153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46844/0 (socket says 172.21.15.115:46844) 2023-12-01T03:11:58.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f8676d9d640 1 -- 172.21.15.115:0/1992561038 learned_addr learned my addr 172.21.15.115:0/1992561038 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f867659c640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86780cf650 0x7f86780cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.723+0000 7f867759e640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867815cd90 0x7f86780b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f8676d9d640 1 -- 172.21.15.115:0/1992561038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86780cf650 msgr2=0x7f86780cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f8676d9d640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86780cf650 0x7f86780cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f8676d9d640 1 -- 172.21.15.115:0/1992561038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867815cd90 msgr2=0x7f86780b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f8676d9d640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867815cd90 0x7f86780b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f8676d9d640 1 -- 172.21.15.115:0/1992561038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0094d0 con 0x7f86780b5fa0 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f867759e640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867815cd90 0x7f86780b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f8676d9d640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 0x7f8678153720 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f866800d5f0 tx=0x7f866800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f865f7fe640 1 -- 172.21.15.115:0/1992561038 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680125f0 con 0x7f86780b5fa0 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f865f7fe640 1 -- 172.21.15.115:0/1992561038 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668012750 con 0x7f86780b5fa0 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f867d22c640 1 -- 172.21.15.115:0/1992561038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86781f2ca0 con 0x7f86780b5fa0 2023-12-01T03:11:58.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f865f7fe640 1 -- 172.21.15.115:0/1992561038 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668012b00 con 0x7f86780b5fa0 2023-12-01T03:11:58.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.724+0000 7f867d22c640 1 -- 172.21.15.115:0/1992561038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86781f3260 con 0x7f86780b5fa0 2023-12-01T03:11:58.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.726+0000 7f867d22c640 1 -- 172.21.15.115:0/1992561038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86780d3bc0 con 0x7f86780b5fa0 2023-12-01T03:11:58.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.726+0000 7f865f7fe640 1 -- 172.21.15.115:0/1992561038 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 135) v1 ==== 88828+0+0 (secure 0 0 0) 0x7f8668012c60 con 0x7f86780b5fa0 2023-12-01T03:11:58.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.728+0000 7f865f7fe640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f866406a6c0 0x7f866406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:58.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.728+0000 7f867659c640 1 -- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f866406a6c0 msgr2=0x7f866406cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:58.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.728+0000 7f865f7fe640 1 -- 172.21.15.115:0/1992561038 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f8668089610 con 0x7f86780b5fa0 2023-12-01T03:11:58.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.728+0000 7f867659c640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f866406a6c0 0x7f866406cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:58.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.732+0000 7f865f7fe640 1 -- 172.21.15.115:0/1992561038 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8668054500 con 0x7f86780b5fa0 2023-12-01T03:11:58.926 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.924+0000 7f867d22c640 1 -- 172.21.15.115:0/1992561038 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f86781f33f0 con 0x7f86780b5fa0 2023-12-01T03:11:58.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.929+0000 7f867659c640 1 -- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f866406a6c0 msgr2=0x7f866406cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:58.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.929+0000 7f867659c640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f866406a6c0 0x7f866406cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:11:58.931 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.929+0000 7f865f7fe640 1 -- 172.21.15.115:0/1992561038 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v77)=0 v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8668057b20 con 0x7f86780b5fa0 2023-12-01T03:11:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 -- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f866406a6c0 msgr2=0x7f866406cb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:11:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f866406a6c0 0x7f866406cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 -- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 msgr2=0x7f8678153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 0x7f8678153720 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f866800d5f0 tx=0x7f866800dab0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 -- 172.21.15.115:0/1992561038 shutdown_connections 2023-12-01T03:11:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f866406a6c0 0x7f866406cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86780cf650 0x7f86780cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f867815cd90 0x7f86780b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 --2- 172.21.15.115:0/1992561038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86780b5fa0 0x7f8678153720 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:58.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 -- 172.21.15.115:0/1992561038 >> 172.21.15.115:0/1992561038 conn(0x7f8678065960 msgr2=0x7f8678157110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:58.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 -- 172.21.15.115:0/1992561038 shutdown_connections 2023-12-01T03:11:58.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:58.932+0000 7f867d22c640 1 -- 172.21.15.115:0/1992561038 wait complete. 2023-12-01T03:11:58.958 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T03:11:59.035 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:59.033+0000 7fb2e0217040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:11:59.036 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:59.034+0000 7fe3c30b6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:11:59.094 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:59.093+0000 7fb2e0217040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:11:59.095 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:59.094+0000 7fe3c30b6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:11:59.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.166+0000 7f872c73c640 1 Processor -- start 2023-12-01T03:11:59.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.166+0000 7f872c73c640 1 -- start start 2023-12-01T03:11:59.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.167+0000 7f872c73c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f87240b0f80 0x7f87240b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.167+0000 7f872c73c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240cb290 con 0x7f87240cbd30 2023-12-01T03:11:59.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.168+0000 7f872c73c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240b0ae0 con 0x7f8724160200 2023-12-01T03:11:59.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.168+0000 7f872c73c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87240b35f0 con 0x7f87240b0f80 2023-12-01T03:11:59.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.168+0000 7f872acd9640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f87240cbd30 0x7f87240cc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56300/0 (socket says 172.21.15.115:56300) 2023-12-01T03:11:59.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.168+0000 7f872acd9640 1 -- 172.21.15.115:0/1176021388 learned_addr learned my addr 172.21.15.115:0/1176021388 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:59.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.168+0000 7f8729cd7640 1 --2- 172.21.15.115:0/1176021388 >> v2:172.21.15.142:3300/0 conn(0x7f87240b0f80 0x7f87240b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.168+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284970182 0 0) 0x7f87240cb290 con 0x7f87240cbd30 2023-12-01T03:11:59.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.168+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8710003520 con 0x7f87240cbd30 2023-12-01T03:11:59.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2190962690 0 0) 0x7f8710003520 con 0x7f87240cbd30 2023-12-01T03:11:59.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 >> v2:172.21.15.142:3300/0 conn(0x7f87240b0f80 msgr2=0x7f87240b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:59.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 --2- 172.21.15.115:0/1176021388 >> v2:172.21.15.142:3300/0 conn(0x7f87240b0f80 0x7f87240b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 >> v1:172.21.15.142:6789/0 conn(0x7f8724160200 legacy=0x7f872406b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240b3310 con 0x7f87240cbd30 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f871c0019f0 con 0x7f87240cbd30 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 >> v1:172.21.15.115:6789/0 conn(0x7f87240cbd30 legacy=0x7f87240cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87100039c0 0x7f8710003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87100047c0 0x7f8710004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005180 0x7f871000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240b35f0 con 0x7f8710005180 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871000c080 con 0x7f87100039c0 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871000c240 con 0x7f87100047c0 2023-12-01T03:11:59.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f872acd9640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005180 0x7f871000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f8729cd7640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87100047c0 0x7f8710004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.169+0000 7f872a4d8640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87100039c0 0x7f8710003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.170+0000 7f872acd9640 1 -- 172.21.15.115:0/1176021388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87100047c0 msgr2=0x7f8710004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:59.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.170+0000 7f872acd9640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87100047c0 0x7f8710004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.170+0000 7f872acd9640 1 -- 172.21.15.115:0/1176021388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87100039c0 msgr2=0x7f8710003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:59.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.170+0000 7f872acd9640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87100039c0 0x7f8710003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.170+0000 7f872acd9640 1 -- 172.21.15.115:0/1176021388 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8710003520 con 0x7f8710005180 2023-12-01T03:11:59.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.171+0000 7f872acd9640 1 -- 172.21.15.115:0/1176021388 reap_dead start 2023-12-01T03:11:59.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.171+0000 7f872acd9640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005180 0x7f871000b980 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f871c00a760 tx=0x7f87100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497041a02c092a2d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:59.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.172+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c034630 con 0x7f8710005180 2023-12-01T03:11:59.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.172+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c02f070 con 0x7f8710005180 2023-12-01T03:11:59.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.172+0000 7f87294d6640 1 -- 172.21.15.115:0/1176021388 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c034a10 con 0x7f8710005180 2023-12-01T03:11:59.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.172+0000 7f872c73c640 1 -- 172.21.15.115:0/1176021388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005180 msgr2=0x7f871000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:59.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.172+0000 7f872c73c640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005180 0x7f871000b980 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f871c00a760 tx=0x7f87100057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.172+0000 7f872c73c640 1 -- 172.21.15.115:0/1176021388 shutdown_connections 2023-12-01T03:11:59.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.172+0000 7f872c73c640 1 --2- 172.21.15.115:0/1176021388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8710005180 0x7f871000b980 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.172+0000 7f872c73c640 1 -- 172.21.15.115:0/1176021388 >> 172.21.15.115:0/1176021388 conn(0x7f8724065960 msgr2=0x7f8724065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:59.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.173+0000 7f872c73c640 1 -- 172.21.15.115:0/1176021388 shutdown_connections 2023-12-01T03:11:59.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.173+0000 7f872c73c640 1 -- 172.21.15.115:0/1176021388 wait complete. 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.174+0000 7f872c73c640 1 Processor -- start 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.174+0000 7f872c73c640 1 -- start start 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.175+0000 7f872c73c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8710005180 0x7f8724142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.175+0000 7f872c73c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87240c8ae0 0x7f87240ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.175+0000 7f872c73c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87240cecc0 0x7f87240cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.175+0000 7f872c73c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240cf680 con 0x7f87240cecc0 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.175+0000 7f872c73c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87241430d0 con 0x7f87240c8ae0 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.175+0000 7f872c73c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240c9010 con 0x7f8710005180 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f872acd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87240cecc0 0x7f87240cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f872acd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87240cecc0 0x7f87240cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46858/0 (socket says 172.21.15.115:46858) 2023-12-01T03:11:59.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f872acd9640 1 -- 172.21.15.115:0/2354524035 learned_addr learned my addr 172.21.15.115:0/2354524035 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f872a4d8640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8710005180 0x7f8724142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f8729cd7640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87240c8ae0 0x7f87240ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f872acd9640 1 -- 172.21.15.115:0/2354524035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8710005180 msgr2=0x7f8724142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f872acd9640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8710005180 0x7f8724142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f872acd9640 1 -- 172.21.15.115:0/2354524035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87240c8ae0 msgr2=0x7f87240ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f872acd9640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87240c8ae0 0x7f87240ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f872acd9640 1 -- 172.21.15.115:0/2354524035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240c92f0 con 0x7f87240cecc0 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.176+0000 7f872acd9640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87240cecc0 0x7f87240cf140 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f871c002230 tx=0x7f871c001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.177+0000 7f871affd640 1 -- 172.21.15.115:0/2354524035 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c0349b0 con 0x7f87240cecc0 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.177+0000 7f871affd640 1 -- 172.21.15.115:0/2354524035 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c034b10 con 0x7f87240cecc0 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.177+0000 7f872c73c640 1 -- 172.21.15.115:0/2354524035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87240c95d0 con 0x7f87240cecc0 2023-12-01T03:11:59.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.177+0000 7f871affd640 1 -- 172.21.15.115:0/2354524035 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c034ea0 con 0x7f87240cecc0 2023-12-01T03:11:59.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.177+0000 7f872c73c640 1 -- 172.21.15.115:0/2354524035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872414dc20 con 0x7f87240cecc0 2023-12-01T03:11:59.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.178+0000 7f872c73c640 1 -- 172.21.15.115:0/2354524035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872404e890 con 0x7f87240cecc0 2023-12-01T03:11:59.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.180+0000 7f871affd640 1 -- 172.21.15.115:0/2354524035 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 135) v1 ==== 88828+0+0 (secure 0 0 0) 0x7f871c031030 con 0x7f87240cecc0 2023-12-01T03:11:59.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.181+0000 7f871affd640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f870806a6c0 0x7f870806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.181+0000 7f871affd640 1 -- 172.21.15.115:0/2354524035 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f871c0aab80 con 0x7f87240cecc0 2023-12-01T03:11:59.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.181+0000 7f872a4d8640 1 -- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f870806a6c0 msgr2=0x7f870806cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:59.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.181+0000 7f872a4d8640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f870806a6c0 0x7f870806cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:59.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.185+0000 7f871affd640 1 -- 172.21.15.115:0/2354524035 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f871c075a50 con 0x7f87240cecc0 2023-12-01T03:11:59.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.376+0000 7f872c73c640 1 -- 172.21.15.115:0/2354524035 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f87240b1e20 con 0x7f87240cecc0 2023-12-01T03:11:59.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.380+0000 7f871affd640 1 -- 172.21.15.115:0/2354524035 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v78)=0 v78) v1 ==== 133+0+0 (secure 0 0 0) 0x7f871c0aae60 con 0x7f87240cecc0 2023-12-01T03:11:59.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.382+0000 7f872a4d8640 1 -- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f870806a6c0 msgr2=0x7f870806cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:59.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.382+0000 7f872a4d8640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f870806a6c0 0x7f870806cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:11:59.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 -- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f870806a6c0 msgr2=0x7f870806cb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:11:59.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f870806a6c0 0x7f870806cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 -- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87240cecc0 msgr2=0x7f87240cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:59.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87240cecc0 0x7f87240cf140 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f871c002230 tx=0x7f871c001cb0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 -- 172.21.15.115:0/2354524035 shutdown_connections 2023-12-01T03:11:59.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f870806a6c0 0x7f870806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87240cecc0 0x7f87240cf140 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87240c8ae0 0x7f87240ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 --2- 172.21.15.115:0/2354524035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8710005180 0x7f8724142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.385 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 -- 172.21.15.115:0/2354524035 >> 172.21.15.115:0/2354524035 conn(0x7f8724065960 msgr2=0x7f8724156460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:59.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 -- 172.21.15.115:0/2354524035 shutdown_connections 2023-12-01T03:11:59.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.383+0000 7f872c73c640 1 -- 172.21.15.115:0/2354524035 wait complete. 2023-12-01T03:11:59.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:11:59.405 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-12-01T03:11:59.462 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir 1M 2023-12-01T03:11:59.546 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:59.545+0000 7fe3c30b6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:11:59.575 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:59.573+0000 7fb2e0217040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:11:59.598 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:59.597+0000 7fe3c30b6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:11:59.630 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:59.627+0000 7fb2e0217040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:11:59.648 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:59.647+0000 7fe3c30b6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:11:59.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.671+0000 7f8c10f89640 1 Processor -- start 2023-12-01T03:11:59.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.672+0000 7f8c10f89640 1 -- start start 2023-12-01T03:11:59.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.673+0000 7f8c10f89640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c15e720 0x7f8c0c0b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.673+0000 7f8c10f89640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8c0c0b2040 0x7f8c0c0c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.673+0000 7f8c10f89640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0c0cc2b0 con 0x7f8c0c0cd470 2023-12-01T03:11:59.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.673+0000 7f8c10f89640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c0c0ce0b0 con 0x7f8c0c0b2040 2023-12-01T03:11:59.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.673+0000 7f8c10f89640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c0cddd0 con 0x7f8c0c15e720 2023-12-01T03:11:59.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.673+0000 7f8c0a59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8c0c0cd470 0x7f8c0c15e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56304/0 (socket says 172.21.15.115:56304) 2023-12-01T03:11:59.675 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.673+0000 7f8c0a59c640 1 -- 172.21.15.115:0/223042154 learned_addr learned my addr 172.21.15.115:0/223042154 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c03fff640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c15e720 0x7f8c0c0b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c0ad9d640 1 --2- 172.21.15.115:0/223042154 >> v2:172.21.15.142:3300/0 conn(0x7f8c0c0b2040 0x7f8c0c0c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699548847 0 0) 0x7f8c0c0cc2b0 con 0x7f8c0c0cd470 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bf8003520 con 0x7f8c0c0cd470 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1960147924 0 0) 0x7f8bf8003520 con 0x7f8c0c0cd470 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c15e720 msgr2=0x7f8c0c0b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c15e720 0x7f8c0c0b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 >> v2:172.21.15.142:3300/0 conn(0x7f8c0c0b2040 msgr2=0x7f8c0c0c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 --2- 172.21.15.115:0/223042154 >> v2:172.21.15.142:3300/0 conn(0x7f8c0c0b2040 0x7f8c0c0c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c0d1780 con 0x7f8c0c0cd470 2023-12-01T03:11:59.677 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f8bfc001670 con 0x7f8c0c0cd470 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 >> v1:172.21.15.115:6789/0 conn(0x7f8c0c0cd470 legacy=0x7f8c0c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf8003970 0x7f8bf8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80047c0 0x7f8bf8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c15e720 0x7f8bf8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c0ce0b0 con 0x7f8bf80047c0 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c0cddd0 con 0x7f8bf8003970 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf800b9b0 con 0x7f8c0c15e720 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.674+0000 7f8c03fff640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80047c0 0x7f8bf8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.675+0000 7f8c0a59c640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf8003970 0x7f8bf8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.675+0000 7f8c0ad9d640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c15e720 0x7f8bf8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.675+0000 7f8c03fff640 1 -- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c15e720 msgr2=0x7f8bf8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.675+0000 7f8c03fff640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c15e720 0x7f8bf8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.675+0000 7f8c03fff640 1 -- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf8003970 msgr2=0x7f8bf8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.675+0000 7f8c03fff640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf8003970 0x7f8bf8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.678 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.675+0000 7f8c03fff640 1 -- 172.21.15.115:0/223042154 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf8003520 con 0x7f8bf80047c0 2023-12-01T03:11:59.679 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.676+0000 7f8c03fff640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80047c0 0x7f8bf8004c40 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400ba30 tx=0x7f8bf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388c3f8f1d1149a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:59.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.676+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4032630 con 0x7f8bf80047c0 2023-12-01T03:11:59.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.676+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4032790 con 0x7f8bf80047c0 2023-12-01T03:11:59.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.676+0000 7f8c09d9b640 1 -- 172.21.15.115:0/223042154 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4032b20 con 0x7f8bf80047c0 2023-12-01T03:11:59.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.677+0000 7f8c10f89640 1 -- 172.21.15.115:0/223042154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80047c0 msgr2=0x7f8bf8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:59.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.677+0000 7f8c10f89640 1 --2- 172.21.15.115:0/223042154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80047c0 0x7f8bf8004c40 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400ba30 tx=0x7f8bf80057f0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.677+0000 7f8c0ad9d640 1 -- 172.21.15.115:0/223042154 reap_dead start 2023-12-01T03:11:59.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.677+0000 7f8c10f89640 1 -- 172.21.15.115:0/223042154 shutdown_connections 2023-12-01T03:11:59.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.677+0000 7f8c10f89640 1 -- 172.21.15.115:0/223042154 >> 172.21.15.115:0/223042154 conn(0x7f8c0c0d0ba0 msgr2=0x7f8c0c0d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:11:59.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.677+0000 7f8c10f89640 1 -- 172.21.15.115:0/223042154 shutdown_connections 2023-12-01T03:11:59.680 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.677+0000 7f8c10f89640 1 -- 172.21.15.115:0/223042154 wait complete. 2023-12-01T03:11:59.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.679+0000 7f8c10f89640 1 Processor -- start 2023-12-01T03:11:59.681 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.679+0000 7f8c10f89640 1 -- start start 2023-12-01T03:11:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.680+0000 7f8c10f89640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80047c0 0x7f8c0c1e5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.680+0000 7f8c10f89640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c06dfe0 0x7f8c0c155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.680+0000 7f8c10f89640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c0c06a010 0x7f8c0c06a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.680+0000 7f8c10f89640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c06a9d0 con 0x7f8bf80047c0 2023-12-01T03:11:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.680+0000 7f8c10f89640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c0c93e0 con 0x7f8c0c06a010 2023-12-01T03:11:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.680+0000 7f8c10f89640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c0c9630 con 0x7f8c0c06dfe0 2023-12-01T03:11:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.680+0000 7f8c0a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80047c0 0x7f8c0c1e5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.680+0000 7f8c0a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80047c0 0x7f8c0c1e5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:46890/0 (socket says 172.21.15.115:46890) 2023-12-01T03:11:59.682 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.680+0000 7f8c0a59c640 1 -- 172.21.15.115:0/1746709578 learned_addr learned my addr 172.21.15.115:0/1746709578 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:11:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c03fff640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c06dfe0 0x7f8c0c155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:11:59.683 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c0a59c640 1 -- 172.21.15.115:0/1746709578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c06dfe0 msgr2=0x7f8c0c155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c0a59c640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c0c06dfe0 0x7f8c0c155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c0a59c640 1 -- 172.21.15.115:0/1746709578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c0c06a010 msgr2=0x7f8c0c06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:11:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c0a59c640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c0c06a010 0x7f8c0c06a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:11:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c0a59c640 1 -- 172.21.15.115:0/1746709578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c0c9910 con 0x7f8bf80047c0 2023-12-01T03:11:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c0a59c640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80047c0 0x7f8c0c1e5270 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc007340 tx=0x7f8bfc007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:11:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc00f7e0 con 0x7f8bf80047c0 2023-12-01T03:11:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc00f940 con 0x7f8bf80047c0 2023-12-01T03:11:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c10f89640 1 -- 172.21.15.115:0/1746709578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c0c9bf0 con 0x7f8bf80047c0 2023-12-01T03:11:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc00fcf0 con 0x7f8bf80047c0 2023-12-01T03:11:59.684 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.681+0000 7f8c10f89640 1 -- 172.21.15.115:0/1746709578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0c0ca1b0 con 0x7f8bf80047c0 2023-12-01T03:11:59.684 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:59.682+0000 7fb2e0217040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:11:59.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.682+0000 7f8c10f89640 1 -- 172.21.15.115:0/1746709578 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0c067850 con 0x7f8bf80047c0 2023-12-01T03:11:59.685 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.683+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 135) v1 ==== 88828+0+0 (secure 0 0 0) 0x7f8bfc01d460 con 0x7f8bf80047c0 2023-12-01T03:11:59.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.684+0000 7f8c02ffd640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 0x7f8bec06cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:11:59.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.684+0000 7f8c03fff640 1 -- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 msgr2=0x7f8bec06cbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:59.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.684+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f8bfc089280 con 0x7f8bf80047c0 2023-12-01T03:11:59.686 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.684+0000 7f8c03fff640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 0x7f8bec06cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:11:59.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.688+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8bfc0540e0 con 0x7f8bf80047c0 2023-12-01T03:11:59.710 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:59.709+0000 7fe3c30b6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:11:59.747 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:59.745+0000 7fb2e0217040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:11:59.759 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:59.758+0000 7fe3c30b6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:11:59.798 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:59.797+0000 7fb2e0217040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:11:59.871 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:59.870+0000 7fe3c30b6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:11:59.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.885+0000 7f8c03fff640 1 -- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 msgr2=0x7f8bec06cbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:11:59.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.885+0000 7f8c03fff640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 0x7f8bec06cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:11:59.897 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:11:59.895+0000 7f8c10f89640 1 -- 172.21.15.115:0/1746709578 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c0c0d1a60 con 0x7f8bec06a6f0 2023-12-01T03:11:59.914 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:59.912+0000 7fb2e0217040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:11:59.933 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:59.931+0000 7fe3c30b6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:11:59.977 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:11:59.975+0000 7fb2e0217040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:11:59.982 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:11:59.981+0000 7fe3c30b6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:12:00.026 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:00.024+0000 7fb2e0217040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:12:00.038 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:00.037+0000 7fe3c30b6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:12:00.083 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:00.081+0000 7fb2e0217040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:12:00.089 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:00.088+0000 7fe3c30b6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:12:00.133 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:00.132+0000 7fb2e0217040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:12:00.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:00.286+0000 7f8c03fff640 1 -- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 msgr2=0x7f8bec06cbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:00.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:00.286+0000 7f8c03fff640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 0x7f8bec06cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T03:12:00.468 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:00.467+0000 7fe3c30b6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:12:00.515 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:00.513+0000 7fb2e0217040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:12:00.628 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:00.627+0000 7fe3c30b6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:12:00.673 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:00.671+0000 7fb2e0217040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:12:00.739 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:00.738+0000 7fe3c30b6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:12:00.782 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:00.780+0000 7fb2e0217040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:12:00.790 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:00.789+0000 7fe3c30b6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:12:00.831 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:00.830+0000 7fb2e0217040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:12:00.898 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:00.897+0000 7fe3c30b6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:00.919 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:12:00.926 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:12:00.941 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:00.940+0000 7fb2e0217040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:01.009 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:01.008+0000 7fe3c30b6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:12:01.054 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:01.053+0000 7fb2e0217040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:12:01.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:01.087+0000 7f8c03fff640 1 -- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 msgr2=0x7f8bec06cbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:01.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:01.087+0000 7f8c03fff640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 0x7f8bec06cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T03:12:01.170 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:01.168+0000 7fe3c30b6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:01.211 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:01.209+0000 7fb2e0217040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:01.218 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:01.217+0000 7fe3c30b6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:12:01.259 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:01.258+0000 7fb2e0217040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:12:01.337 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:01.336+0000 7fe3c30b6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:12:01.378 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:01.376+0000 7fb2e0217040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:12:02.170 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:02.169+0000 7fe3c30b6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:12:02.212 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.210+0000 7fb2e0217040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:12:02.494 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:02.493+0000 7fe3c30b6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:12:02.505 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.503+0000 7fb2e0217040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:12:02.551 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:02.550+0000 7fe3c30b6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:12:02.563 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.561+0000 7fb2e0217040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:12:02.664 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:02.663+0000 7fe3c30b6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:12:02.676 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.674+0000 7fb2e0217040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:12:02.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:02.689+0000 7f8c03fff640 1 -- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 msgr2=0x7f8bec06cbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:02.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:02.690+0000 7f8c03fff640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 0x7f8bec06cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-01T03:12:02.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:02.694+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 136) v1 ==== 88335+0+0 (secure 0 0 0) 0x7f8bfc0544b0 con 0x7f8bf80047c0 2023-12-01T03:12:02.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:02.695+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 msgr2=0x7f8bec06cbb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:02.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:02.695+0000 7f8c02ffd640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 0x7f8bec06cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:02.953 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.951+0000 7fb25f2ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.953 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.951+0000 7fb25f2ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.953 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.951+0000 7fb25f2ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.953 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.951+0000 7fb25f2ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.953 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.951+0000 7fb25f2ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.953 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.951+0000 7fb25c2a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.954 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.951+0000 7fb25c2a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.954 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.951+0000 7fb25c2a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.954 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.951+0000 7fb25c2a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.954 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.951+0000 7fb25c2a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.971 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.969+0000 7fb25f2ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.971 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.969+0000 7fb25f2ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.971 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.969+0000 7fb25f2ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.972 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.969+0000 7fb25f2ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.972 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.969+0000 7fb25f2ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.972 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.969+0000 7fb25c2a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.972 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.969+0000 7fb25c2a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.972 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.969+0000 7fb25c2a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.972 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.969+0000 7fb25c2a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:02.972 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:02.969+0000 7fb25c2a8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:03.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:03.702+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 137) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f8bfc06f5a0 con 0x7f8bf80047c0 2023-12-01T03:12:03.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:03.703+0000 7f8c02ffd640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 0x7f8bec035500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:03.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:03.703+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bec0280d0 con 0x7f8bec06a6f0 2023-12-01T03:12:03.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:03.710+0000 7f8c03fff640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 0x7f8bec035500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:03.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:03.715+0000 7f8c03fff640 1 --2- 172.21.15.115:0/1746709578 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8bec06a6f0 0x7f8bec035500 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400ba00 tx=0x7f8bf403d040 comp rx=0 tx=0).ready entity=mgr.11784 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:03.721 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:03.719+0000 7fb2662bc640 -1 mgr.server reply reply (22) Invalid argument filesystem argument is required when there is more than one file system 2023-12-01T03:12:03.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:03.720+0000 7f8c02ffd640 1 -- 172.21.15.115:0/1746709578 <== mgr.11784 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: -22 filesystem argument is required when there is more than one file system) v1 ==== 79+0+0 (secure 0 0 0) 0x7f8bec0280d0 con 0x7f8bec06a6f0 2023-12-01T03:12:03.722 INFO:teuthology.orchestra.run.smithi115.stderr:Error EINVAL: filesystem argument is required when there is more than one file system 2023-12-01T03:12:03.739 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-01T03:12:03.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:03.740 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-12-01T03:12:03.843 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T03:12:04.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.052+0000 7f86c9843640 1 Processor -- start 2023-12-01T03:12:04.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.052+0000 7f86c9843640 1 -- start start 2023-12-01T03:12:04.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.053+0000 7f86c9843640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f86c41590b0 0x7f86c4159490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:04.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.053+0000 7f86c9843640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c41599d0 0x7f86c40b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:04.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.053+0000 7f86c9843640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c40cac30 con 0x7f86c40b1c50 2023-12-01T03:12:04.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.053+0000 7f86c9843640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c40b2140 con 0x7f86c41590b0 2023-12-01T03:12:04.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.053+0000 7f86c9843640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40b2420 con 0x7f86c41599d0 2023-12-01T03:12:04.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c37fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f86c40b1c50 0x7f86c406baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56312/0 (socket says 172.21.15.115:56312) 2023-12-01T03:12:04.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c37fe640 1 -- 172.21.15.115:0/492517311 learned_addr learned my addr 172.21.15.115:0/492517311 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:04.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791800732 0 0) 0x7f86c40cac30 con 0x7f86c40b1c50 2023-12-01T03:12:04.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b0003520 con 0x7f86c40b1c50 2023-12-01T03:12:04.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86bbfff640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c41599d0 0x7f86c40b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:04.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c2ffd640 1 --2- 172.21.15.115:0/492517311 >> v2:172.21.15.142:3300/0 conn(0x7f86c41590b0 0x7f86c4159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3019647380 0 0) 0x7f86b0003520 con 0x7f86c40b1c50 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c41599d0 msgr2=0x7f86c40b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c41599d0 0x7f86c40b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 >> v2:172.21.15.142:3300/0 conn(0x7f86c41590b0 msgr2=0x7f86c4159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 --2- 172.21.15.115:0/492517311 >> v2:172.21.15.142:3300/0 conn(0x7f86c41590b0 0x7f86c4159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c40caed0 con 0x7f86c40b1c50 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f86bc0019f0 con 0x7f86c40b1c50 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 >> v1:172.21.15.115:6789/0 conn(0x7f86c40b1c50 legacy=0x7f86c406baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c41599d0 0x7f86b0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b0003ff0 0x7f86b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86b0004d00 0x7f86b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:04.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40b2140 con 0x7f86b0004d00 2023-12-01T03:12:04.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40b2420 con 0x7f86b0003ff0 2023-12-01T03:12:04.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.054+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b000bae0 con 0x7f86c41599d0 2023-12-01T03:12:04.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.055+0000 7f86c37fe640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86b0004d00 0x7f86b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:04.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.055+0000 7f86bbfff640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b0003ff0 0x7f86b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:04.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.055+0000 7f86c2ffd640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c41599d0 0x7f86b0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:04.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.055+0000 7f86c37fe640 1 -- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c41599d0 msgr2=0x7f86b0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:04.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.055+0000 7f86c37fe640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c41599d0 0x7f86b0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.055+0000 7f86c37fe640 1 -- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b0003ff0 msgr2=0x7f86b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:04.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.055+0000 7f86c37fe640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b0003ff0 0x7f86b00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.055+0000 7f86c37fe640 1 -- 172.21.15.115:0/492517311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b0003520 con 0x7f86b0004d00 2023-12-01T03:12:04.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.055+0000 7f86bbfff640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b0003ff0 0x7f86b00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:04.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.056+0000 7f86c37fe640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86b0004d00 0x7f86b0005180 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00a810 tx=0x7f86b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f5067ff92abf9f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:04.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.056+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc034630 con 0x7f86b0004d00 2023-12-01T03:12:04.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.056+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc02f070 con 0x7f86b0004d00 2023-12-01T03:12:04.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.056+0000 7f86c27fc640 1 -- 172.21.15.115:0/492517311 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc034a50 con 0x7f86b0004d00 2023-12-01T03:12:04.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.057+0000 7f86c9843640 1 -- 172.21.15.115:0/492517311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86b0004d00 msgr2=0x7f86b0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:04.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.057+0000 7f86c9843640 1 --2- 172.21.15.115:0/492517311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86b0004d00 0x7f86b0005180 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00a810 tx=0x7f86b00057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.057+0000 7f86c37fe640 1 -- 172.21.15.115:0/492517311 reap_dead start 2023-12-01T03:12:04.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.057+0000 7f86c9843640 1 -- 172.21.15.115:0/492517311 shutdown_connections 2023-12-01T03:12:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.057+0000 7f86c9843640 1 -- 172.21.15.115:0/492517311 >> 172.21.15.115:0/492517311 conn(0x7f86c4065960 msgr2=0x7f86c4065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.057+0000 7f86c9843640 1 -- 172.21.15.115:0/492517311 shutdown_connections 2023-12-01T03:12:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.057+0000 7f86c9843640 1 -- 172.21.15.115:0/492517311 wait complete. 2023-12-01T03:12:04.065 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.059+0000 7f86c9843640 1 Processor -- start 2023-12-01T03:12:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.059+0000 7f86c9843640 1 -- start start 2023-12-01T03:12:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.060+0000 7f86c9843640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b0004d00 0x7f86c413e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.060+0000 7f86c9843640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c4153860 0x7f86c4153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.060+0000 7f86c9843640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c414eac0 0x7f86c414ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:04.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.060+0000 7f86c9843640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c4154200 con 0x7f86c4153860 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.060+0000 7f86c9843640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c414f780 con 0x7f86b0004d00 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.060+0000 7f86c9843640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c414fa60 con 0x7f86c414eac0 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.060+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c4153860 0x7f86c4153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.060+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c4153860 0x7f86c4153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47066/0 (socket says 172.21.15.115:47066) 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.060+0000 7f86bbfff640 1 -- 172.21.15.115:0/4098973265 learned_addr learned my addr 172.21.15.115:0/4098973265 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.061+0000 7f86bbfff640 1 -- 172.21.15.115:0/4098973265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c414eac0 msgr2=0x7f86c414ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.061+0000 7f86bbfff640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c414eac0 0x7f86c414ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.061+0000 7f86bbfff640 1 -- 172.21.15.115:0/4098973265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b0004d00 msgr2=0x7f86c413e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.061+0000 7f86c2ffd640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b0004d00 0x7f86c413e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.061+0000 7f86bbfff640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b0004d00 0x7f86c413e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.061+0000 7f86bbfff640 1 -- 172.21.15.115:0/4098973265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c414fd40 con 0x7f86c4153860 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.061+0000 7f86bbfff640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c4153860 0x7f86c4153cc0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e860 tx=0x7f86ac00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:04.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.061+0000 7f86c0ff9640 1 -- 172.21.15.115:0/4098973265 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00ab20 con 0x7f86c4153860 2023-12-01T03:12:04.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.061+0000 7f86c0ff9640 1 -- 172.21.15.115:0/4098973265 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac00ac80 con 0x7f86c4153860 2023-12-01T03:12:04.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.062+0000 7f86c0ff9640 1 -- 172.21.15.115:0/4098973265 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0113f0 con 0x7f86c4153860 2023-12-01T03:12:04.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.062+0000 7f86c9843640 1 -- 172.21.15.115:0/4098973265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c4150020 con 0x7f86c4153860 2023-12-01T03:12:04.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.062+0000 7f86c9843640 1 -- 172.21.15.115:0/4098973265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c414f480 con 0x7f86c4153860 2023-12-01T03:12:04.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.063+0000 7f86c9843640 1 -- 172.21.15.115:0/4098973265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c41ee1b0 con 0x7f86c4153860 2023-12-01T03:12:04.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.064+0000 7f86c0ff9640 1 -- 172.21.15.115:0/4098973265 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 137) v1 ==== 88462+0+0 (secure 0 0 0) 0x7f86ac011550 con 0x7f86c4153860 2023-12-01T03:12:04.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.065+0000 7f86c0ff9640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f86a406a2d0 0x7f86a406c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:04.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.065+0000 7f86c0ff9640 1 -- 172.21.15.115:0/4098973265 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f86ac088fd0 con 0x7f86c4153860 2023-12-01T03:12:04.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.065+0000 7f86c2ffd640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f86a406a2d0 0x7f86a406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:04.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.066+0000 7f86c2ffd640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f86a406a2d0 0x7f86a406c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f86b400be70 tx=0x7f86b400d040 comp rx=0 tx=0).ready entity=mgr.11784 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:04.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.070+0000 7f86c0ff9640 1 -- 172.21.15.115:0/4098973265 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f86ac0542a0 con 0x7f86c4153860 2023-12-01T03:12:04.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.317+0000 7f86c9843640 1 -- 172.21.15.115:0/4098973265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f86c404e890 con 0x7f86c4153860 2023-12-01T03:12:04.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.720+0000 7f86c0ff9640 1 -- 172.21.15.115:0/4098973265 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 138) v1 ==== 88812+0+0 (secure 0 0 0) 0x7f86ac054670 con 0x7f86c4153860 2023-12-01T03:12:04.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.721+0000 7f86c0ff9640 1 -- 172.21.15.115:0/4098973265 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v138) v1 ==== 93+0+0 (secure 0 0 0) 0x7f86ac0850b0 con 0x7f86c4153860 2023-12-01T03:12:04.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.725+0000 7f86c9843640 1 -- 172.21.15.115:0/4098973265 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f86a406a2d0 msgr2=0x7f86a406c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.725+0000 7f86c9843640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f86a406a2d0 0x7f86a406c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f86b400be70 tx=0x7f86b400d040 comp rx=0 tx=0).stop 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.725+0000 7f86c9843640 1 -- 172.21.15.115:0/4098973265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c4153860 msgr2=0x7f86c4153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.725+0000 7f86c9843640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c4153860 0x7f86c4153cc0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e860 tx=0x7f86ac00ed20 comp rx=0 tx=0).stop 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.726+0000 7f86c9843640 1 -- 172.21.15.115:0/4098973265 shutdown_connections 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.726+0000 7f86c9843640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f86a406a2d0 0x7f86a406c790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.726+0000 7f86c9843640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86c414eac0 0x7f86c414ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.726+0000 7f86c9843640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86c4153860 0x7f86c4153cc0 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.726+0000 7f86c9843640 1 --2- 172.21.15.115:0/4098973265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86b0004d00 0x7f86c413e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.726+0000 7f86c9843640 1 -- 172.21.15.115:0/4098973265 >> 172.21.15.115:0/4098973265 conn(0x7f86c4065960 msgr2=0x7f86c406fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.726+0000 7f86c9843640 1 -- 172.21.15.115:0/4098973265 shutdown_connections 2023-12-01T03:12:04.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:04.726+0000 7f86c9843640 1 -- 172.21.15.115:0/4098973265 wait complete. 2023-12-01T03:12:04.747 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T03:12:04.747 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:12:04.849 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:12:04.945 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T03:12:04.945 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:12:04.945 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:04.995 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:12:04.996 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:05.091 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:05.257 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:05.256+0000 7ff192c2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:12:05.270 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:05.268+0000 7f4a4e416040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:12:05.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.289+0000 7ff30e100640 1 Processor -- start 2023-12-01T03:12:05.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.289+0000 7ff30e100640 1 -- start start 2023-12-01T03:12:05.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.290+0000 7ff30e100640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff30815cd20 0x7ff3080d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.290+0000 7ff30e100640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3080b5d80 0x7ff3080b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.290+0000 7ff30e100640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3080d1230 0x7ff3080d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.290+0000 7ff30e100640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3080b2160 con 0x7ff3080d1230 2023-12-01T03:12:05.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.290+0000 7ff30e100640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3080d1be0 con 0x7ff30815cd20 2023-12-01T03:12:05.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.290+0000 7ff30e100640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080d2a10 con 0x7ff3080b5d80 2023-12-01T03:12:05.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.291+0000 7ff307fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3080d1230 0x7ff3080d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.291+0000 7ff307fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3080d1230 0x7ff3080d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47094/0 (socket says 172.21.15.115:47094) 2023-12-01T03:12:05.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.291+0000 7ff307fff640 1 -- 172.21.15.115:0/806281239 learned_addr learned my addr 172.21.15.115:0/806281239 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:05.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.291+0000 7ff3077fe640 1 --2- 172.21.15.115:0/806281239 >> v2:172.21.15.142:3300/0 conn(0x7ff30815cd20 0x7ff3080d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.291+0000 7ff306ffd640 1 --2- 172.21.15.115:0/806281239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3080b5d80 0x7ff3080b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.291+0000 7ff307fff640 1 -- 172.21.15.115:0/806281239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3080b5d80 msgr2=0x7ff3080b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:05.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.291+0000 7ff307fff640 1 --2- 172.21.15.115:0/806281239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3080b5d80 0x7ff3080b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.291+0000 7ff307fff640 1 -- 172.21.15.115:0/806281239 >> v2:172.21.15.142:3300/0 conn(0x7ff30815cd20 msgr2=0x7ff3080d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:05.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.291+0000 7ff307fff640 1 --2- 172.21.15.115:0/806281239 >> v2:172.21.15.142:3300/0 conn(0x7ff30815cd20 0x7ff3080d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.291+0000 7ff307fff640 1 -- 172.21.15.115:0/806281239 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3080ce2e0 con 0x7ff3080d1230 2023-12-01T03:12:05.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.292+0000 7ff307fff640 1 --2- 172.21.15.115:0/806281239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080d1230 0x7ff3080d16a0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7ff30808bdb0 tx=0x7ff30002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03c422347460aba server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:05.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.293+0000 7ff3067fc640 1 -- 172.21.15.115:0/806281239 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30002c9f0 con 0x7ff3080d1230 2023-12-01T03:12:05.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.293+0000 7ff3067fc640 1 -- 172.21.15.115:0/806281239 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30002cb50 con 0x7ff3080d1230 2023-12-01T03:12:05.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.293+0000 7ff3067fc640 1 -- 172.21.15.115:0/806281239 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3000343f0 con 0x7ff3080d1230 2023-12-01T03:12:05.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.293+0000 7ff30e100640 1 -- 172.21.15.115:0/806281239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080d1230 msgr2=0x7ff3080d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:05.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.293+0000 7ff30e100640 1 --2- 172.21.15.115:0/806281239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080d1230 0x7ff3080d16a0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7ff30808bdb0 tx=0x7ff30002be50 comp rx=0 tx=0).stop 2023-12-01T03:12:05.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.293+0000 7ff30e100640 1 -- 172.21.15.115:0/806281239 shutdown_connections 2023-12-01T03:12:05.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.293+0000 7ff30e100640 1 --2- 172.21.15.115:0/806281239 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080d1230 0x7ff3080d16a0 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.293+0000 7ff30e100640 1 --2- 172.21.15.115:0/806281239 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3080b5d80 0x7ff3080b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.293+0000 7ff30e100640 1 --2- 172.21.15.115:0/806281239 >> v2:172.21.15.142:3300/0 conn(0x7ff30815cd20 0x7ff3080d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.293+0000 7ff30e100640 1 -- 172.21.15.115:0/806281239 >> 172.21.15.115:0/806281239 conn(0x7ff30806a0e0 msgr2=0x7ff30806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:05.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.294+0000 7ff30e100640 1 -- 172.21.15.115:0/806281239 shutdown_connections 2023-12-01T03:12:05.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.294+0000 7ff30e100640 1 -- 172.21.15.115:0/806281239 wait complete. 2023-12-01T03:12:05.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.295+0000 7ff30e100640 1 Processor -- start 2023-12-01T03:12:05.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.295+0000 7ff30e100640 1 -- start start 2023-12-01T03:12:05.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.296+0000 7ff30e100640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080b5d80 0x7ff308155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.296+0000 7ff30e100640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3080d1230 0x7ff30815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.296+0000 7ff30e100640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30815cd20 0x7ff30815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.296+0000 7ff30e100640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080c8780 con 0x7ff3080b5d80 2023-12-01T03:12:05.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.296+0000 7ff30e100640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080c8a60 con 0x7ff3080d1230 2023-12-01T03:12:05.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.296+0000 7ff30e100640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080c8d40 con 0x7ff30815cd20 2023-12-01T03:12:05.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.296+0000 7ff3077fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080b5d80 0x7ff308155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.297+0000 7ff3077fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080b5d80 0x7ff308155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47096/0 (socket says 172.21.15.115:47096) 2023-12-01T03:12:05.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.297+0000 7ff3077fe640 1 -- 172.21.15.115:0/2282231909 learned_addr learned my addr 172.21.15.115:0/2282231909 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:05.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.297+0000 7ff306ffd640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3080d1230 0x7ff30815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.297+0000 7ff307fff640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30815cd20 0x7ff30815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.297+0000 7ff3077fe640 1 -- 172.21.15.115:0/2282231909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30815cd20 msgr2=0x7ff30815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:05.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.297+0000 7ff3077fe640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30815cd20 0x7ff30815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.297+0000 7ff3077fe640 1 -- 172.21.15.115:0/2282231909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3080d1230 msgr2=0x7ff30815b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:05.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.297+0000 7ff3077fe640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3080d1230 0x7ff30815b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.297+0000 7ff3077fe640 1 -- 172.21.15.115:0/2282231909 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300009b30 con 0x7ff3080b5d80 2023-12-01T03:12:05.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.297+0000 7ff3077fe640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080b5d80 0x7ff308155cd0 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800d800 tx=0x7ff2f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:05.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.298+0000 7ff2effff640 1 -- 172.21.15.115:0/2282231909 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800b6c0 con 0x7ff3080b5d80 2023-12-01T03:12:05.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.298+0000 7ff2effff640 1 -- 172.21.15.115:0/2282231909 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f800b820 con 0x7ff3080b5d80 2023-12-01T03:12:05.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.298+0000 7ff2effff640 1 -- 172.21.15.115:0/2282231909 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800bbb0 con 0x7ff3080b5d80 2023-12-01T03:12:05.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.298+0000 7ff30e100640 1 -- 172.21.15.115:0/2282231909 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3080c9020 con 0x7ff3080b5d80 2023-12-01T03:12:05.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.298+0000 7ff30e100640 1 -- 172.21.15.115:0/2282231909 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3080c8350 con 0x7ff3080b5d80 2023-12-01T03:12:05.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.299+0000 7ff30e100640 1 -- 172.21.15.115:0/2282231909 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3080d1a50 con 0x7ff3080b5d80 2023-12-01T03:12:05.301 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.300+0000 7ff2effff640 1 -- 172.21.15.115:0/2282231909 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 138) v1 ==== 88812+0+0 (secure 0 0 0) 0x7ff2f800bd10 con 0x7ff3080b5d80 2023-12-01T03:12:05.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.301+0000 7ff2effff640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff2f406a670 0x7ff2f406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.301+0000 7ff2effff640 1 -- 172.21.15.115:0/2282231909 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7047+0+0 (secure 0 0 0) 0x7ff2f80896b0 con 0x7ff3080b5d80 2023-12-01T03:12:05.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.301+0000 7ff306ffd640 1 -- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff2f406a670 msgr2=0x7ff2f406cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:05.303 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.301+0000 7ff306ffd640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff2f406a670 0x7ff2f406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:05.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.305+0000 7ff2effff640 1 -- 172.21.15.115:0/2282231909 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff2f80543f0 con 0x7ff3080b5d80 2023-12-01T03:12:05.311 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:05.310+0000 7ff192c2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:12:05.327 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:05.325+0000 7f4a4e416040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:12:05.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.496+0000 7ff30e100640 1 -- 172.21.15.115:0/2282231909 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3080c8510 con 0x7ff3080b5d80 2023-12-01T03:12:05.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.499+0000 7ff2effff640 1 -- 172.21.15.115:0/2282231909 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v240) v1 ==== 74+0+20517 (secure 0 0 0) 0x7ff2f8057a10 con 0x7ff3080b5d80 2023-12-01T03:12:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:05.502 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":240,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:02.677856+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:11:49.229904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:11:50.056441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/638241265":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/3713299174":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/4124642007":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6824/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/3863453309":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1160795001":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/524060219":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1801977880":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/203535105":"2023-12-02T03:12:02.677778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:05.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff306ffd640 1 -- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff2f406a670 msgr2=0x7ff2f406cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:05.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 -- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff2f406a670 msgr2=0x7ff2f406cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T03:12:05.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff306ffd640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff2f406a670 0x7ff2f406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:12:05.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff2f406a670 0x7ff2f406cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 -- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080b5d80 msgr2=0x7ff308155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:05.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080b5d80 0x7ff308155cd0 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800d800 tx=0x7ff2f800dcc0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 -- 172.21.15.115:0/2282231909 shutdown_connections 2023-12-01T03:12:05.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff2f406a670 0x7ff2f406cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff30815cd20 0x7ff30815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3080d1230 0x7ff30815b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 --2- 172.21.15.115:0/2282231909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3080b5d80 0x7ff308155cd0 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 -- 172.21.15.115:0/2282231909 >> 172.21.15.115:0/2282231909 conn(0x7ff30806a0e0 msgr2=0x7ff308157550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:05.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 -- 172.21.15.115:0/2282231909 shutdown_connections 2023-12-01T03:12:05.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.502+0000 7ff30e100640 1 -- 172.21.15.115:0/2282231909 wait complete. 2023-12-01T03:12:05.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T03:11:30.230656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '234', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T03:11:31.144773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '235', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-01T03:11:49.229904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '238', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-01T03:11:50.056441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '239', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:12:05.523 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:12:05.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.731+0000 7f43ae441640 1 Processor -- start 2023-12-01T03:12:05.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.731+0000 7f43ae441640 1 -- start start 2023-12-01T03:12:05.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.733+0000 7f43ae441640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43a815cd90 0x7f43a80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.733+0000 7f43ae441640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f43a80cfd60 0x7f43a8161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.733+0000 7f43ae441640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a80ce5e0 con 0x7f43a80b5fa0 2023-12-01T03:12:05.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.733+0000 7f43ae441640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a80d0ec0 con 0x7f43a80cfd60 2023-12-01T03:12:05.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.733+0000 7f43ae441640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8161950 con 0x7f43a815cd90 2023-12-01T03:12:05.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.733+0000 7f43ac9de640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f43a80cfd60 0x7f43a8161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.733+0000 7f43ac9de640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f43a80cfd60 0x7f43a8161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47106/0 (socket says 172.21.15.115:47106) 2023-12-01T03:12:05.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.733+0000 7f43ac9de640 1 -- 172.21.15.115:0/733307454 learned_addr learned my addr 172.21.15.115:0/733307454 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:05.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.733+0000 7f43a7fff640 1 --2- 172.21.15.115:0/733307454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43a815cd90 0x7f43a80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.734+0000 7f43ac9de640 1 -- 172.21.15.115:0/733307454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43a815cd90 msgr2=0x7f43a80cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:05.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.734+0000 7f43ac9de640 1 --2- 172.21.15.115:0/733307454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43a815cd90 0x7f43a80cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.734+0000 7f43ac9de640 1 -- 172.21.15.115:0/733307454 >> v1:172.21.15.142:6789/0 conn(0x7f43a80b5fa0 legacy=0x7f43a80cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:05.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.734+0000 7f43ac9de640 1 -- 172.21.15.115:0/733307454 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a80ce300 con 0x7f43a80cfd60 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.735+0000 7f43ac9de640 1 --2- 172.21.15.115:0/733307454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 0x7f43a8161410 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009e50 tx=0x7f43a002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51edf3ee729ce914 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.735+0000 7f43a77fe640 1 -- 172.21.15.115:0/733307454 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a002cb00 con 0x7f43a80cfd60 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.735+0000 7f43a77fe640 1 -- 172.21.15.115:0/733307454 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a002cc60 con 0x7f43a80cfd60 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.735+0000 7f43a77fe640 1 -- 172.21.15.115:0/733307454 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a00343f0 con 0x7f43a80cfd60 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.736+0000 7f43ae441640 1 -- 172.21.15.115:0/733307454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 msgr2=0x7f43a8161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.736+0000 7f43ae441640 1 --2- 172.21.15.115:0/733307454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 0x7f43a8161410 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009e50 tx=0x7f43a002bf60 comp rx=0 tx=0).stop 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.736+0000 7f43ae441640 1 -- 172.21.15.115:0/733307454 shutdown_connections 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.736+0000 7f43ae441640 1 --2- 172.21.15.115:0/733307454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 0x7f43a8161410 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.736+0000 7f43ae441640 1 --2- 172.21.15.115:0/733307454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43a815cd90 0x7f43a80cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.736+0000 7f43ae441640 1 -- 172.21.15.115:0/733307454 >> 172.21.15.115:0/733307454 conn(0x7f43a8065960 msgr2=0x7f43a8065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.736+0000 7f43ae441640 1 -- 172.21.15.115:0/733307454 shutdown_connections 2023-12-01T03:12:05.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.736+0000 7f43ae441640 1 -- 172.21.15.115:0/733307454 wait complete. 2023-12-01T03:12:05.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.738+0000 7f43ae441640 1 Processor -- start 2023-12-01T03:12:05.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.738+0000 7f43ae441640 1 -- start start 2023-12-01T03:12:05.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f43ae441640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43a80b5fa0 0x7f43a81f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f43ae441640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 0x7f43a81f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f43ae441640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43a815cd90 0x7f43a815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f43ae441640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80cade0 con 0x7f43a80cfd60 2023-12-01T03:12:05.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f43ae441640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80cb030 con 0x7f43a80b5fa0 2023-12-01T03:12:05.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f43ae441640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80cb310 con 0x7f43a815cd90 2023-12-01T03:12:05.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f439ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 0x7f43a81f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f439ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 0x7f43a81f3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47110/0 (socket says 172.21.15.115:47110) 2023-12-01T03:12:05.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f439ffff640 1 -- 172.21.15.115:0/1103930240 learned_addr learned my addr 172.21.15.115:0/1103930240 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:05.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f43a7fff640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43a80b5fa0 0x7f43a81f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.739+0000 7f43ac9de640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43a815cd90 0x7f43a815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.740+0000 7f439ffff640 1 -- 172.21.15.115:0/1103930240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43a815cd90 msgr2=0x7f43a815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.740+0000 7f439ffff640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43a815cd90 0x7f43a815e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.740+0000 7f439ffff640 1 -- 172.21.15.115:0/1103930240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43a80b5fa0 msgr2=0x7f43a81f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.740+0000 7f439ffff640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43a80b5fa0 0x7f43a81f5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.740+0000 7f439ffff640 1 -- 172.21.15.115:0/1103930240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a0009b30 con 0x7f43a80cfd60 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.740+0000 7f439ffff640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 0x7f43a81f3570 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f439000c400 tx=0x7f439000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.740+0000 7f43a57fa640 1 -- 172.21.15.115:0/1103930240 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43900052d0 con 0x7f43a80cfd60 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.740+0000 7f43a57fa640 1 -- 172.21.15.115:0/1103930240 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4390005430 con 0x7f43a80cfd60 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.740+0000 7f43a57fa640 1 -- 172.21.15.115:0/1103930240 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390003830 con 0x7f43a80cfd60 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.740+0000 7f43ae441640 1 -- 172.21.15.115:0/1103930240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a80cb5f0 con 0x7f43a80cfd60 2023-12-01T03:12:05.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.741+0000 7f43ae441640 1 -- 172.21.15.115:0/1103930240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a815d980 con 0x7f43a80cfd60 2023-12-01T03:12:05.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.742+0000 7f43ae441640 1 -- 172.21.15.115:0/1103930240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a80cbb10 con 0x7f43a80cfd60 2023-12-01T03:12:05.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.742+0000 7f43a57fa640 1 -- 172.21.15.115:0/1103930240 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 138) v1 ==== 88812+0+0 (secure 0 0 0) 0x7f4390003990 con 0x7f43a80cfd60 2023-12-01T03:12:05.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.743+0000 7f43a57fa640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f438806a6a0 0x7f438806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:05.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.743+0000 7f43a57fa640 1 -- 172.21.15.115:0/1103930240 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f4390088420 con 0x7f43a80cfd60 2023-12-01T03:12:05.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.743+0000 7f43a7fff640 1 -- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f438806a6a0 msgr2=0x7f438806cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:05.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.743+0000 7f43a7fff640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f438806a6a0 0x7f438806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:05.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.748+0000 7f43a57fa640 1 -- 172.21.15.115:0/1103930240 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f43a80cbb10 con 0x7f43a80cfd60 2023-12-01T03:12:05.759 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:05.758+0000 7ff192c2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:12:05.807 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:05.806+0000 7f4a4e416040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:12:05.812 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:05.812+0000 7ff192c2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:12:05.861 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:05.859+0000 7f4a4e416040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:12:05.862 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:05.861+0000 7ff192c2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:12:05.911 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:05.909+0000 7f4a4e416040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:12:05.923 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:05.922+0000 7ff192c2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:12:05.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.944+0000 7f43a7fff640 1 -- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f438806a6a0 msgr2=0x7f438806cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:05.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.944+0000 7f43a7fff640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f438806a6a0 0x7f438806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:12:05.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.947+0000 7f43ae441640 1 -- 172.21.15.115:0/1103930240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43a815db10 con 0x7f43a80cfd60 2023-12-01T03:12:05.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.948+0000 7f43a57fa640 1 -- 172.21.15.115:0/1103930240 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v240) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4390053380 con 0x7f43a80cfd60 2023-12-01T03:12:05.949 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:12:05.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 -- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f438806a6a0 msgr2=0x7f438806cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:05.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f438806a6a0 0x7f438806cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 -- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 msgr2=0x7f43a81f3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:05.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 0x7f43a81f3570 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f439000c400 tx=0x7f439000c8c0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 -- 172.21.15.115:0/1103930240 shutdown_connections 2023-12-01T03:12:05.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f438806a6a0 0x7f438806cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43a815cd90 0x7f43a815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43a80cfd60 0x7f43a81f3570 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 --2- 172.21.15.115:0/1103930240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43a80b5fa0 0x7f43a81f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:05.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 -- 172.21.15.115:0/1103930240 >> 172.21.15.115:0/1103930240 conn(0x7f43a8065960 msgr2=0x7f43a80b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:05.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 -- 172.21.15.115:0/1103930240 shutdown_connections 2023-12-01T03:12:05.953 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:05.950+0000 7f43ae441640 1 -- 172.21.15.115:0/1103930240 wait complete. 2023-12-01T03:12:05.969 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:12:05.971 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:05.970+0000 7ff192c2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:12:05.977 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:05.972+0000 7f4a4e416040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:12:06.026 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:06.024+0000 7f4a4e416040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:12:06.081 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:06.080+0000 7ff192c2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:12:06.142 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:06.140+0000 7f4a4e416040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:12:06.143 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:06.142+0000 7ff192c2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:12:06.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.170+0000 7f5e8759e640 1 Processor -- start 2023-12-01T03:12:06.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.170+0000 7f5e8759e640 1 -- start start 2023-12-01T03:12:06.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.172+0000 7f5e8759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e880cfb80 0x7f5e880ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.172+0000 7f5e8759e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e880cdcd0 con 0x7f5e88154010 2023-12-01T03:12:06.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.172+0000 7f5e8759e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e88161830 con 0x7f5e88153c30 2023-12-01T03:12:06.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.172+0000 7f5e8759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880cd3a0 con 0x7f5e880cfb80 2023-12-01T03:12:06.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.172+0000 7f5e86d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5e88154010 0x7f5e88160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56334/0 (socket says 172.21.15.115:56334) 2023-12-01T03:12:06.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.172+0000 7f5e86d9d640 1 -- 172.21.15.115:0/1028849778 learned_addr learned my addr 172.21.15.115:0/1028849778 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:06.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.172+0000 7f5e7ffff640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e880cfb80 0x7f5e880ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064833962 0 0) 0x7f5e880cdcd0 con 0x7f5e88154010 2023-12-01T03:12:06.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e74003520 con 0x7f5e88154010 2023-12-01T03:12:06.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1858628856 0 0) 0x7f5e74003520 con 0x7f5e88154010 2023-12-01T03:12:06.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e880cfb80 msgr2=0x7f5e880ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:06.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e880cfb80 0x7f5e880ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 >> v1:172.21.15.142:6789/0 conn(0x7f5e88153c30 legacy=0x7f5e880cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e880cd680 con 0x7f5e88154010 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f5e800019f0 con 0x7f5e88154010 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 >> v1:172.21.15.115:6789/0 conn(0x7f5e88154010 legacy=0x7f5e88160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e740039a0 0x7f5e74003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e74004350 0x7f5e74004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e880cfb80 0x7f5e74005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880cd3a0 con 0x7f5e740039a0 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7400b980 con 0x7f5e74004350 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.173+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7400bb40 con 0x7f5e880cfb80 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.174+0000 7f5e8659c640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e740039a0 0x7f5e74003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.174+0000 7f5e7ffff640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e74004350 0x7f5e74004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.174+0000 7f5e86d9d640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e880cfb80 0x7f5e74005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.174+0000 7f5e8659c640 1 -- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e880cfb80 msgr2=0x7f5e74005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:06.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.174+0000 7f5e8659c640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e880cfb80 0x7f5e74005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.174+0000 7f5e8659c640 1 -- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e74004350 msgr2=0x7f5e74004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:06.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.174+0000 7f5e8659c640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e74004350 0x7f5e74004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.174+0000 7f5e8659c640 1 -- 172.21.15.115:0/1028849778 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74003520 con 0x7f5e740039a0 2023-12-01T03:12:06.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.174+0000 7f5e86d9d640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e880cfb80 0x7f5e74005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:06.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.175+0000 7f5e8659c640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e740039a0 0x7f5e74003d80 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f5e78001860 tx=0x7f5e740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c374dccf504aa4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:06.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.175+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78033640 con 0x7f5e740039a0 2023-12-01T03:12:06.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.175+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e780337c0 con 0x7f5e740039a0 2023-12-01T03:12:06.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.175+0000 7f5e85d9b640 1 -- 172.21.15.115:0/1028849778 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78033b70 con 0x7f5e740039a0 2023-12-01T03:12:06.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.176+0000 7f5e8759e640 1 -- 172.21.15.115:0/1028849778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e740039a0 msgr2=0x7f5e74003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.176+0000 7f5e8759e640 1 --2- 172.21.15.115:0/1028849778 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e740039a0 0x7f5e74003d80 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f5e78001860 tx=0x7f5e740057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.176+0000 7f5e86d9d640 1 -- 172.21.15.115:0/1028849778 reap_dead start 2023-12-01T03:12:06.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.176+0000 7f5e8759e640 1 -- 172.21.15.115:0/1028849778 shutdown_connections 2023-12-01T03:12:06.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.176+0000 7f5e8759e640 1 -- 172.21.15.115:0/1028849778 >> 172.21.15.115:0/1028849778 conn(0x7f5e8806a0b0 msgr2=0x7f5e8806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:06.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.176+0000 7f5e8759e640 1 -- 172.21.15.115:0/1028849778 shutdown_connections 2023-12-01T03:12:06.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.176+0000 7f5e8759e640 1 -- 172.21.15.115:0/1028849778 wait complete. 2023-12-01T03:12:06.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.178+0000 7f5e8759e640 1 Processor -- start 2023-12-01T03:12:06.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.178+0000 7f5e8759e640 1 -- start start 2023-12-01T03:12:06.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e8759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e740039a0 0x7f5e880c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e8759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880c58a0 0x7f5e88156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e8759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e8813f890 0x7f5e8813fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e8759e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88157310 con 0x7f5e880c58a0 2023-12-01T03:12:06.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e8759e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88140250 con 0x7f5e8813f890 2023-12-01T03:12:06.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e8759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880c5d20 con 0x7f5e740039a0 2023-12-01T03:12:06.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e7ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880c58a0 0x7f5e88156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e7ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880c58a0 0x7f5e88156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47136/0 (socket says 172.21.15.115:47136) 2023-12-01T03:12:06.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e7ffff640 1 -- 172.21.15.115:0/992881502 learned_addr learned my addr 172.21.15.115:0/992881502 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:06.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e86d9d640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e8813f890 0x7f5e8813fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.179+0000 7f5e8659c640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e740039a0 0x7f5e880c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e7ffff640 1 -- 172.21.15.115:0/992881502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e740039a0 msgr2=0x7f5e880c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e7ffff640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e740039a0 0x7f5e880c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e7ffff640 1 -- 172.21.15.115:0/992881502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e8813f890 msgr2=0x7f5e8813fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e7ffff640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e8813f890 0x7f5e8813fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e7ffff640 1 -- 172.21.15.115:0/992881502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e880cb810 con 0x7f5e880c58a0 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e7ffff640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880c58a0 0x7f5e88156dd0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f5e70011600 tx=0x7f5e70011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e7effd640 1 -- 172.21.15.115:0/992881502 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000f920 con 0x7f5e880c58a0 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e7effd640 1 -- 172.21.15.115:0/992881502 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7000fa80 con 0x7f5e880c58a0 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e7effd640 1 -- 172.21.15.115:0/992881502 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000fdf0 con 0x7f5e880c58a0 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e8759e640 1 -- 172.21.15.115:0/992881502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e880cbaf0 con 0x7f5e880c58a0 2023-12-01T03:12:06.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.180+0000 7f5e8759e640 1 -- 172.21.15.115:0/992881502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e880cc0b0 con 0x7f5e880c58a0 2023-12-01T03:12:06.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.181+0000 7f5e8759e640 1 -- 172.21.15.115:0/992881502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e880618d0 con 0x7f5e880c58a0 2023-12-01T03:12:06.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.182+0000 7f5e7effd640 1 -- 172.21.15.115:0/992881502 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 138) v1 ==== 88812+0+0 (secure 0 0 0) 0x7f5e70015460 con 0x7f5e880c58a0 2023-12-01T03:12:06.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.183+0000 7f5e7effd640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5e6806a670 0x7f5e6806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.183+0000 7f5e7effd640 1 -- 172.21.15.115:0/992881502 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f5e700896d0 con 0x7f5e880c58a0 2023-12-01T03:12:06.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.183+0000 7f5e8659c640 1 -- 172.21.15.115:0/992881502 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5e6806a670 msgr2=0x7f5e6806cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:06.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.183+0000 7f5e8659c640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5e6806a670 0x7f5e6806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:06.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.187+0000 7f5e7effd640 1 -- 172.21.15.115:0/992881502 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5e70054420 con 0x7f5e880c58a0 2023-12-01T03:12:06.193 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:06.193+0000 7ff192c2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:12:06.210 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:06.209+0000 7f4a4e416040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:12:06.251 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:06.250+0000 7ff192c2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:12:06.262 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:06.260+0000 7f4a4e416040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:12:06.302 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:06.301+0000 7ff192c2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:12:06.321 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:06.319+0000 7f4a4e416040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:12:06.373 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:06.371+0000 7f4a4e416040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:12:06.378 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.377+0000 7f5e8759e640 1 -- 172.21.15.115:0/992881502 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e880b5540 con 0x7f5e880c58a0 2023-12-01T03:12:06.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.378+0000 7f5e7effd640 1 -- 172.21.15.115:0/992881502 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v240) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e70057a40 con 0x7f5e880c58a0 2023-12-01T03:12:06.379 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 -- 172.21.15.115:0/992881502 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5e6806a670 msgr2=0x7f5e6806cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5e6806a670 0x7f5e6806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 -- 172.21.15.115:0/992881502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880c58a0 msgr2=0x7f5e88156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880c58a0 0x7f5e88156dd0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f5e70011600 tx=0x7f5e70011ac0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 -- 172.21.15.115:0/992881502 shutdown_connections 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5e6806a670 0x7f5e6806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e8813f890 0x7f5e8813fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880c58a0 0x7f5e88156dd0 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 --2- 172.21.15.115:0/992881502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5e740039a0 0x7f5e880c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 -- 172.21.15.115:0/992881502 >> 172.21.15.115:0/992881502 conn(0x7f5e8806a0b0 msgr2=0x7f5e881518e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 -- 172.21.15.115:0/992881502 shutdown_connections 2023-12-01T03:12:06.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.380+0000 7f5e8759e640 1 -- 172.21.15.115:0/992881502 wait complete. 2023-12-01T03:12:06.399 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:12:06.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.599+0000 7f42619bc640 1 Processor -- start 2023-12-01T03:12:06.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.600+0000 7f42619bc640 1 -- start start 2023-12-01T03:12:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.601+0000 7f42619bc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f425c153c30 0x7f425c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.601+0000 7f42619bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f425c154010 0x7f425c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.601+0000 7f42619bc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c0cdcd0 con 0x7f425c0cf9b0 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.601+0000 7f42619bc640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425c0cfe20 con 0x7f425c153c30 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.601+0000 7f42619bc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c161830 con 0x7f425c154010 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.601+0000 7f42609ba640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f425c153c30 0x7f425c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.601+0000 7f42611bb640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f425c0cf9b0 0x7f425c160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:56340/0 (socket says 172.21.15.115:56340) 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.601+0000 7f42611bb640 1 -- 172.21.15.115:0/403196577 learned_addr learned my addr 172.21.15.115:0/403196577 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.601+0000 7f42609ba640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f425c153c30 0x7f425c0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47150/0 (socket says 172.21.15.115:47150) 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.601+0000 7f425bfff640 1 --2- 172.21.15.115:0/403196577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f425c154010 0x7f425c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.602+0000 7f42609ba640 1 -- 172.21.15.115:0/403196577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f425c154010 msgr2=0x7f425c0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.602+0000 7f42609ba640 1 --2- 172.21.15.115:0/403196577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f425c154010 0x7f425c0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.602+0000 7f42609ba640 1 -- 172.21.15.115:0/403196577 >> v1:172.21.15.115:6789/0 conn(0x7f425c0cf9b0 legacy=0x7f425c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.602+0000 7f42609ba640 1 -- 172.21.15.115:0/403196577 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0cd3a0 con 0x7f425c153c30 2023-12-01T03:12:06.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.602+0000 7f425b7fe640 1 -- 172.21.15.115:0/403196577 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366628437 0 0) 0x7f425c0cdcd0 con 0x7f425c0cf9b0 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.603+0000 7f42609ba640 1 --2- 172.21.15.115:0/403196577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 0x7f425c0ce160 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f4254009d90 tx=0x7f425402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c636c2b0d177f453 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.603+0000 7f425b7fe640 1 -- 172.21.15.115:0/403196577 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425402cc50 con 0x7f425c153c30 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.603+0000 7f425b7fe640 1 -- 172.21.15.115:0/403196577 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425402cdb0 con 0x7f425c153c30 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.603+0000 7f425b7fe640 1 -- 172.21.15.115:0/403196577 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254034660 con 0x7f425c153c30 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.603+0000 7f42619bc640 1 -- 172.21.15.115:0/403196577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 msgr2=0x7f425c0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.603+0000 7f42619bc640 1 --2- 172.21.15.115:0/403196577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 0x7f425c0ce160 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f4254009d90 tx=0x7f425402bea0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.604+0000 7f42619bc640 1 -- 172.21.15.115:0/403196577 shutdown_connections 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.604+0000 7f42619bc640 1 --2- 172.21.15.115:0/403196577 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f425c154010 0x7f425c0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.604+0000 7f42619bc640 1 --2- 172.21.15.115:0/403196577 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 0x7f425c0ce160 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.604+0000 7f42619bc640 1 -- 172.21.15.115:0/403196577 >> 172.21.15.115:0/403196577 conn(0x7f425c06a0b0 msgr2=0x7f425c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:06.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.604+0000 7f42619bc640 1 -- 172.21.15.115:0/403196577 shutdown_connections 2023-12-01T03:12:06.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.604+0000 7f42619bc640 1 -- 172.21.15.115:0/403196577 wait complete. 2023-12-01T03:12:06.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.605+0000 7f42619bc640 1 Processor -- start 2023-12-01T03:12:06.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.606+0000 7f42619bc640 1 -- start start 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.606+0000 7f42619bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f425c0cf9b0 0x7f425c1f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.606+0000 7f42619bc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 0x7f425c0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.606+0000 7f42619bc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f425c154010 0x7f425c0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.606+0000 7f42619bc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c1f2ea0 con 0x7f425c153c30 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.606+0000 7f42619bc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c160420 con 0x7f425c154010 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.606+0000 7f42619bc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c160140 con 0x7f425c0cf9b0 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.607+0000 7f425bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 0x7f425c0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.607+0000 7f425bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 0x7f425c0c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47160/0 (socket says 172.21.15.115:47160) 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.607+0000 7f425bfff640 1 -- 172.21.15.115:0/3589672734 learned_addr learned my addr 172.21.15.115:0/3589672734 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.607+0000 7f42611bb640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f425c154010 0x7f425c0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.607+0000 7f42609ba640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f425c0cf9b0 0x7f425c1f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:06.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.607+0000 7f425bfff640 1 -- 172.21.15.115:0/3589672734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f425c0cf9b0 msgr2=0x7f425c1f2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.607+0000 7f425bfff640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f425c0cf9b0 0x7f425c1f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.607+0000 7f425bfff640 1 -- 172.21.15.115:0/3589672734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f425c154010 msgr2=0x7f425c0c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.607+0000 7f425bfff640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f425c154010 0x7f425c0c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.607+0000 7f425bfff640 1 -- 172.21.15.115:0/3589672734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4254009a70 con 0x7f425c153c30 2023-12-01T03:12:06.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.608+0000 7f425bfff640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 0x7f425c0c57c0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f424400d5f0 tx=0x7f424400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:06.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.608+0000 7f42597fa640 1 -- 172.21.15.115:0/3589672734 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42440125f0 con 0x7f425c153c30 2023-12-01T03:12:06.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.608+0000 7f42597fa640 1 -- 172.21.15.115:0/3589672734 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4244012750 con 0x7f425c153c30 2023-12-01T03:12:06.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.608+0000 7f42597fa640 1 -- 172.21.15.115:0/3589672734 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244012b00 con 0x7f425c153c30 2023-12-01T03:12:06.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.608+0000 7f42619bc640 1 -- 172.21.15.115:0/3589672734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c15b0e0 con 0x7f425c153c30 2023-12-01T03:12:06.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.608+0000 7f42619bc640 1 -- 172.21.15.115:0/3589672734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c15b6a0 con 0x7f425c153c30 2023-12-01T03:12:06.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.609+0000 7f42619bc640 1 -- 172.21.15.115:0/3589672734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c15b860 con 0x7f425c153c30 2023-12-01T03:12:06.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.611+0000 7f42597fa640 1 -- 172.21.15.115:0/3589672734 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 138) v1 ==== 88812+0+0 (secure 0 0 0) 0x7f4244012c60 con 0x7f425c153c30 2023-12-01T03:12:06.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.612+0000 7f42597fa640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f423c06a6a0 0x7f423c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:06.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.612+0000 7f42597fa640 1 -- 172.21.15.115:0/3589672734 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f4244089620 con 0x7f425c153c30 2023-12-01T03:12:06.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.612+0000 7f42609ba640 1 -- 172.21.15.115:0/3589672734 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f423c06a6a0 msgr2=0x7f423c06cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:06.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.612+0000 7f42609ba640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f423c06a6a0 0x7f423c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:06.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.616+0000 7f42597fa640 1 -- 172.21.15.115:0/3589672734 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4244054400 con 0x7f425c153c30 2023-12-01T03:12:06.688 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:06.686+0000 7ff192c2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:12:06.774 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:06.772+0000 7f4a4e416040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:12:06.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.807+0000 7f42619bc640 1 -- 172.21.15.115:0/3589672734 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f425c0618d0 con 0x7f425c153c30 2023-12-01T03:12:06.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.808+0000 7f42597fa640 1 -- 172.21.15.115:0/3589672734 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v240) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4244057a20 con 0x7f425c153c30 2023-12-01T03:12:06.810 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:12:06.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.810+0000 7f42619bc640 1 -- 172.21.15.115:0/3589672734 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f423c06a6a0 msgr2=0x7f423c06cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:06.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.810+0000 7f42619bc640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f423c06a6a0 0x7f423c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.810+0000 7f42619bc640 1 -- 172.21.15.115:0/3589672734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 msgr2=0x7f425c0c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:06.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.810+0000 7f42619bc640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 0x7f425c0c57c0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f424400d5f0 tx=0x7f424400dab0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.810+0000 7f42619bc640 1 -- 172.21.15.115:0/3589672734 shutdown_connections 2023-12-01T03:12:06.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.810+0000 7f42619bc640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f423c06a6a0 0x7f423c06cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.810+0000 7f42619bc640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f425c154010 0x7f425c0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.810+0000 7f42619bc640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f425c153c30 0x7f425c0c57c0 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.810+0000 7f42619bc640 1 --2- 172.21.15.115:0/3589672734 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f425c0cf9b0 0x7f425c1f2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:06.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.810+0000 7f42619bc640 1 -- 172.21.15.115:0/3589672734 >> 172.21.15.115:0/3589672734 conn(0x7f425c06a0b0 msgr2=0x7f425c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:06.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.811+0000 7f42619bc640 1 -- 172.21.15.115:0/3589672734 shutdown_connections 2023-12-01T03:12:06.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:06.811+0000 7f42619bc640 1 -- 172.21.15.115:0/3589672734 wait complete. 2023-12-01T03:12:06.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-01T03:12:06.848 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:06.847+0000 7ff192c2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:12:06.940 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:06.938+0000 7f4a4e416040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:12:06.957 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:06.956+0000 7ff192c2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:12:07.007 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:07.006+0000 7ff192c2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:12:07.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.033+0000 7f4cd5a38640 1 Processor -- start 2023-12-01T03:12:07.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.033+0000 7f4cd5a38640 1 -- start start 2023-12-01T03:12:07.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cd5a38640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd015cf70 0x7f4cd00b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cd5a38640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4cd00b18f0 0x7f4cd00b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cd5a38640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd00b7760 con 0x7f4cd00cfb50 2023-12-01T03:12:07.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cd5a38640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd00b2170 con 0x7f4cd00b18f0 2023-12-01T03:12:07.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cd5a38640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd00b8b70 con 0x7f4cd015cf70 2023-12-01T03:12:07.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cce7fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4cd00b18f0 0x7f4cd00b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cce7fc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4cd00b18f0 0x7f4cd00b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47164/0 (socket says 172.21.15.115:47164) 2023-12-01T03:12:07.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cce7fc640 1 -- 172.21.15.115:0/3357470050 learned_addr learned my addr 172.21.15.115:0/3357470050 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:07.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cceffd640 1 --2- 172.21.15.115:0/3357470050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd015cf70 0x7f4cd00b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cce7fc640 1 -- 172.21.15.115:0/3357470050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd015cf70 msgr2=0x7f4cd00b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:07.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cce7fc640 1 --2- 172.21.15.115:0/3357470050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd015cf70 0x7f4cd00b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cce7fc640 1 -- 172.21.15.115:0/3357470050 >> v1:172.21.15.115:6789/0 conn(0x7f4cd00cfb50 legacy=0x7f4cd00d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4cce7fc640 1 -- 172.21.15.115:0/3357470050 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd00b6e30 con 0x7f4cd00b18f0 2023-12-01T03:12:07.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.035+0000 7f4ccdffb640 1 -- 172.21.15.115:0/3357470050 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082009545 0 0) 0x7f4cd00b7760 con 0x7f4cd00cfb50 2023-12-01T03:12:07.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.037+0000 7f4cce7fc640 1 --2- 172.21.15.115:0/3357470050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd00b18f0 0x7f4cd00b7bf0 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009870 tx=0x7f4cc402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1287e869e737c14d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:07.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.037+0000 7f4ccdffb640 1 -- 172.21.15.115:0/3357470050 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc402f840 con 0x7f4cd00b18f0 2023-12-01T03:12:07.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.037+0000 7f4ccdffb640 1 -- 172.21.15.115:0/3357470050 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc402f9a0 con 0x7f4cd00b18f0 2023-12-01T03:12:07.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.037+0000 7f4ccdffb640 1 -- 172.21.15.115:0/3357470050 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc402fd30 con 0x7f4cd00b18f0 2023-12-01T03:12:07.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.037+0000 7f4cd5a38640 1 -- 172.21.15.115:0/3357470050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd00b18f0 msgr2=0x7f4cd00b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.037+0000 7f4cd5a38640 1 --2- 172.21.15.115:0/3357470050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd00b18f0 0x7f4cd00b7bf0 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009870 tx=0x7f4cc402ba20 comp rx=0 tx=0).stop 2023-12-01T03:12:07.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.038+0000 7f4cd5a38640 1 -- 172.21.15.115:0/3357470050 shutdown_connections 2023-12-01T03:12:07.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.038+0000 7f4cd5a38640 1 --2- 172.21.15.115:0/3357470050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd00b18f0 0x7f4cd00b7bf0 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.038+0000 7f4cd5a38640 1 --2- 172.21.15.115:0/3357470050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd015cf70 0x7f4cd00b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.038+0000 7f4cd5a38640 1 -- 172.21.15.115:0/3357470050 >> 172.21.15.115:0/3357470050 conn(0x7f4cd006a0e0 msgr2=0x7f4cd006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:07.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.038+0000 7f4cd5a38640 1 -- 172.21.15.115:0/3357470050 shutdown_connections 2023-12-01T03:12:07.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.038+0000 7f4cd5a38640 1 -- 172.21.15.115:0/3357470050 wait complete. 2023-12-01T03:12:07.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.039+0000 7f4cd5a38640 1 Processor -- start 2023-12-01T03:12:07.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.039+0000 7f4cd5a38640 1 -- start start 2023-12-01T03:12:07.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.040+0000 7f4cd5a38640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd00b18f0 0x7f4cd01e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.040+0000 7f4cd5a38640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd00cfb50 0x7f4cd0151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.040+0000 7f4cd5a38640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd015cf70 0x7f4cd015ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.040+0000 7f4cd5a38640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd01e5600 con 0x7f4cd015cf70 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.040+0000 7f4cd5a38640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd01e5850 con 0x7f4cd00b18f0 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.040+0000 7f4cd5a38640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd01e5b30 con 0x7f4cd00cfb50 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4ccf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd015cf70 0x7f4cd015ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4ccf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd015cf70 0x7f4cd015ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47180/0 (socket says 172.21.15.115:47180) 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4ccf7fe640 1 -- 172.21.15.115:0/1846389932 learned_addr learned my addr 172.21.15.115:0/1846389932 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4cceffd640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd00b18f0 0x7f4cd01e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4cce7fc640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd00cfb50 0x7f4cd0151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4ccf7fe640 1 -- 172.21.15.115:0/1846389932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd00cfb50 msgr2=0x7f4cd0151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4ccf7fe640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd00cfb50 0x7f4cd0151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4ccf7fe640 1 -- 172.21.15.115:0/1846389932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd00b18f0 msgr2=0x7f4cd01e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4ccf7fe640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd00b18f0 0x7f4cd01e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4ccf7fe640 1 -- 172.21.15.115:0/1846389932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc40094d0 con 0x7f4cd015cf70 2023-12-01T03:12:07.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4cce7fc640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd00cfb50 0x7f4cd0151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:07.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.041+0000 7f4ccf7fe640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd015cf70 0x7f4cd015ba10 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8001560 tx=0x7f4cc800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:07.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.042+0000 7f4cb7fff640 1 -- 172.21.15.115:0/1846389932 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8003da0 con 0x7f4cd015cf70 2023-12-01T03:12:07.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.042+0000 7f4cb7fff640 1 -- 172.21.15.115:0/1846389932 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc8005070 con 0x7f4cd015cf70 2023-12-01T03:12:07.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.042+0000 7f4cb7fff640 1 -- 172.21.15.115:0/1846389932 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8005420 con 0x7f4cd015cf70 2023-12-01T03:12:07.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.042+0000 7f4cd5a38640 1 -- 172.21.15.115:0/1846389932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd01e5e10 con 0x7f4cd015cf70 2023-12-01T03:12:07.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.042+0000 7f4cd5a38640 1 -- 172.21.15.115:0/1846389932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd01e63d0 con 0x7f4cd015cf70 2023-12-01T03:12:07.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.043+0000 7f4cd5a38640 1 -- 172.21.15.115:0/1846389932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd01e65c0 con 0x7f4cd015cf70 2023-12-01T03:12:07.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.044+0000 7f4cb7fff640 1 -- 172.21.15.115:0/1846389932 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 138) v1 ==== 88812+0+0 (secure 0 0 0) 0x7f4cc8005580 con 0x7f4cd015cf70 2023-12-01T03:12:07.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.045+0000 7f4cb7fff640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4cbc06a670 0x7f4cbc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.045+0000 7f4cb7fff640 1 -- 172.21.15.115:0/1846389932 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f4cc8088aa0 con 0x7f4cd015cf70 2023-12-01T03:12:07.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.045+0000 7f4cceffd640 1 -- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4cbc06a670 msgr2=0x7f4cbc06cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:07.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.045+0000 7f4cceffd640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4cbc06a670 0x7f4cbc06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:07.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.049+0000 7f4cb7fff640 1 -- 172.21.15.115:0/1846389932 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4cc8053880 con 0x7f4cd015cf70 2023-12-01T03:12:07.053 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:07.051+0000 7f4a4e416040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:12:07.109 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:07.107+0000 7f4a4e416040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:12:07.121 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:07.121+0000 7ff192c2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:07.221 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:07.219+0000 7f4a4e416040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:07.229 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:07.228+0000 7ff192c2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:12:07.246 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.245+0000 7f4cd5a38640 1 -- 172.21.15.115:0/1846389932 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4cd004e890 con 0x7f4cd015cf70 2023-12-01T03:12:07.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.245+0000 7f4cb7fff640 1 -- 172.21.15.115:0/1846389932 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v240) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4cc8017020 con 0x7f4cd015cf70 2023-12-01T03:12:07.247 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.246+0000 7f4cceffd640 1 -- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4cbc06a670 msgr2=0x7f4cbc06cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:07.248 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:12:07.248 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.246+0000 7f4cceffd640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4cbc06a670 0x7f4cbc06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:12:07.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 -- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4cbc06a670 msgr2=0x7f4cbc06cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:07.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4cbc06a670 0x7f4cbc06cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 -- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd015cf70 msgr2=0x7f4cd015ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd015cf70 0x7f4cd015ba10 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8001560 tx=0x7f4cc800cc70 comp rx=0 tx=0).stop 2023-12-01T03:12:07.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 -- 172.21.15.115:0/1846389932 shutdown_connections 2023-12-01T03:12:07.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4cbc06a670 0x7f4cbc06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd00cfb50 0x7f4cd0151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4cd015cf70 0x7f4cd015ba10 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 --2- 172.21.15.115:0/1846389932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd00b18f0 0x7f4cd01e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 -- 172.21.15.115:0/1846389932 >> 172.21.15.115:0/1846389932 conn(0x7f4cd006a0e0 msgr2=0x7f4cd015a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:07.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 -- 172.21.15.115:0/1846389932 shutdown_connections 2023-12-01T03:12:07.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.248+0000 7f4cd5a38640 1 -- 172.21.15.115:0/1846389932 wait complete. 2023-12-01T03:12:07.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-01T03:12:07.334 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:07.332+0000 7f4a4e416040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:12:07.386 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:07.384+0000 7ff192c2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:07.435 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:07.433+0000 7ff192c2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:12:07.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.477+0000 7f378f1fd640 1 Processor -- start 2023-12-01T03:12:07.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.478+0000 7f378f1fd640 1 -- start start 2023-12-01T03:12:07.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.479+0000 7f378f1fd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f378815ddb0 0x7f378806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.479+0000 7f378f1fd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f378806b970 0x7f378806da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.479+0000 7f378f1fd640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37880cd800 con 0x7f378806df50 2023-12-01T03:12:07.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.479+0000 7f378f1fd640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378806e3c0 con 0x7f378815ddb0 2023-12-01T03:12:07.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.479+0000 7f378f1fd640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37880cdae0 con 0x7f378806b970 2023-12-01T03:12:07.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.479+0000 7f378e1fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f378815ddb0 0x7f378806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.479+0000 7f378e1fb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f378815ddb0 0x7f378806d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47186/0 (socket says 172.21.15.115:47186) 2023-12-01T03:12:07.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.479+0000 7f378e1fb640 1 -- 172.21.15.115:0/1161602329 learned_addr learned my addr 172.21.15.115:0/1161602329 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:07.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.480+0000 7f378d9fa640 1 --2- 172.21.15.115:0/1161602329 >> v2:172.21.15.142:3300/0 conn(0x7f378806b970 0x7f378806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.480+0000 7f378e1fb640 1 -- 172.21.15.115:0/1161602329 >> v2:172.21.15.142:3300/0 conn(0x7f378806b970 msgr2=0x7f378806da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:07.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.480+0000 7f378e1fb640 1 --2- 172.21.15.115:0/1161602329 >> v2:172.21.15.142:3300/0 conn(0x7f378806b970 0x7f378806da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.480+0000 7f378e1fb640 1 -- 172.21.15.115:0/1161602329 >> v1:172.21.15.115:6789/0 conn(0x7f378806df50 legacy=0x7f378815ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.480+0000 7f378e1fb640 1 -- 172.21.15.115:0/1161602329 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880cddc0 con 0x7f378815ddb0 2023-12-01T03:12:07.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.481+0000 7f378e1fb640 1 --2- 172.21.15.115:0/1161602329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378815ddb0 0x7f378806d4d0 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f3788072170 tx=0x7f378402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62790746e5b6db21 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:07.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.481+0000 7f378d1f9640 1 -- 172.21.15.115:0/1161602329 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784031890 con 0x7f378815ddb0 2023-12-01T03:12:07.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.481+0000 7f378d1f9640 1 -- 172.21.15.115:0/1161602329 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37840319f0 con 0x7f378815ddb0 2023-12-01T03:12:07.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.481+0000 7f378d1f9640 1 -- 172.21.15.115:0/1161602329 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784031d80 con 0x7f378815ddb0 2023-12-01T03:12:07.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.482+0000 7f378f1fd640 1 -- 172.21.15.115:0/1161602329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378815ddb0 msgr2=0x7f378806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.482+0000 7f378f1fd640 1 --2- 172.21.15.115:0/1161602329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378815ddb0 0x7f378806d4d0 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f3788072170 tx=0x7f378402b930 comp rx=0 tx=0).stop 2023-12-01T03:12:07.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.482+0000 7f378f1fd640 1 -- 172.21.15.115:0/1161602329 shutdown_connections 2023-12-01T03:12:07.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.482+0000 7f378f1fd640 1 --2- 172.21.15.115:0/1161602329 >> v2:172.21.15.142:3300/0 conn(0x7f378806b970 0x7f378806da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.482+0000 7f378f1fd640 1 --2- 172.21.15.115:0/1161602329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378815ddb0 0x7f378806d4d0 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.482+0000 7f378f1fd640 1 -- 172.21.15.115:0/1161602329 >> 172.21.15.115:0/1161602329 conn(0x7f37880cee40 msgr2=0x7f37880cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:07.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.482+0000 7f378f1fd640 1 -- 172.21.15.115:0/1161602329 shutdown_connections 2023-12-01T03:12:07.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.482+0000 7f378f1fd640 1 -- 172.21.15.115:0/1161602329 wait complete. 2023-12-01T03:12:07.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.484+0000 7f378f1fd640 1 Processor -- start 2023-12-01T03:12:07.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.484+0000 7f378f1fd640 1 -- start start 2023-12-01T03:12:07.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.485+0000 7f378f1fd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378806b970 0x7f37880adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.485+0000 7f378f1fd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378806df50 0x7f37881f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.485+0000 7f378f1fd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378815ddb0 0x7f3788159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.485+0000 7f378f1fd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37881f0e00 con 0x7f378806df50 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.485+0000 7f378f1fd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788159850 con 0x7f378806b970 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.485+0000 7f378f1fd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37880ae5e0 con 0x7f378815ddb0 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.485+0000 7f378d9fa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378806df50 0x7f37881f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.485+0000 7f378d9fa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378806df50 0x7f37881f08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47192/0 (socket says 172.21.15.115:47192) 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.485+0000 7f378d9fa640 1 -- 172.21.15.115:0/1717407156 learned_addr learned my addr 172.21.15.115:0/1717407156 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f378e1fb640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378806b970 0x7f37880adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f378d9fa640 1 -- 172.21.15.115:0/1717407156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378815ddb0 msgr2=0x7f3788159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f378d9fa640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378815ddb0 0x7f3788159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f378d9fa640 1 -- 172.21.15.115:0/1717407156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378806b970 msgr2=0x7f37880adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f378d9fa640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378806b970 0x7f37880adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f378d9fa640 1 -- 172.21.15.115:0/1717407156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37840094d0 con 0x7f378806df50 2023-12-01T03:12:07.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f378d9fa640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378806df50 0x7f37881f08c0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f377800d800 tx=0x7f377800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:07.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f377effd640 1 -- 172.21.15.115:0/1717407156 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377800b6c0 con 0x7f378806df50 2023-12-01T03:12:07.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f377effd640 1 -- 172.21.15.115:0/1717407156 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377800b820 con 0x7f378806df50 2023-12-01T03:12:07.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f377effd640 1 -- 172.21.15.115:0/1717407156 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377800bbb0 con 0x7f378806df50 2023-12-01T03:12:07.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.486+0000 7f378f1fd640 1 -- 172.21.15.115:0/1717407156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37880ae8c0 con 0x7f378806df50 2023-12-01T03:12:07.489 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.487+0000 7f378f1fd640 1 -- 172.21.15.115:0/1717407156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37880aee80 con 0x7f378806df50 2023-12-01T03:12:07.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.488+0000 7f378f1fd640 1 -- 172.21.15.115:0/1717407156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3788066850 con 0x7f378806df50 2023-12-01T03:12:07.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.489+0000 7f377effd640 1 -- 172.21.15.115:0/1717407156 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 138) v1 ==== 88812+0+0 (secure 0 0 0) 0x7f377800bd10 con 0x7f378806df50 2023-12-01T03:12:07.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.490+0000 7f377effd640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f376c06a6a0 0x7f376c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.490+0000 7f378e1fb640 1 -- 172.21.15.115:0/1717407156 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f376c06a6a0 msgr2=0x7f376c06cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:07.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.490+0000 7f377effd640 1 -- 172.21.15.115:0/1717407156 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f3778089700 con 0x7f378806df50 2023-12-01T03:12:07.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.490+0000 7f378e1fb640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f376c06a6a0 0x7f376c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:07.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.495+0000 7f377effd640 1 -- 172.21.15.115:0/1717407156 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f37780544e0 con 0x7f378806df50 2023-12-01T03:12:07.500 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:07.498+0000 7f4a4e416040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:07.552 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:07.550+0000 7f4a4e416040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:12:07.557 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:07.556+0000 7ff192c2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:12:07.677 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:07.675+0000 7f4a4e416040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:12:07.687 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.685+0000 7f378f1fd640 1 -- 172.21.15.115:0/1717407156 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f378806ddc0 con 0x7f378806df50 2023-12-01T03:12:07.688 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.686+0000 7f377effd640 1 -- 172.21.15.115:0/1717407156 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v240) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3778057b00 con 0x7f378806df50 2023-12-01T03:12:07.688 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:12:07.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.688+0000 7f378f1fd640 1 -- 172.21.15.115:0/1717407156 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f376c06a6a0 msgr2=0x7f376c06cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:07.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.688+0000 7f378f1fd640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f376c06a6a0 0x7f376c06cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.688+0000 7f378f1fd640 1 -- 172.21.15.115:0/1717407156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378806df50 msgr2=0x7f37881f08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.688+0000 7f378f1fd640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378806df50 0x7f37881f08c0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f377800d800 tx=0x7f377800dcc0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.688+0000 7f378f1fd640 1 -- 172.21.15.115:0/1717407156 shutdown_connections 2023-12-01T03:12:07.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.688+0000 7f378f1fd640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f376c06a6a0 0x7f376c06cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.688+0000 7f378f1fd640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f378815ddb0 0x7f3788159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.688+0000 7f378f1fd640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f378806df50 0x7f37881f08c0 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.688+0000 7f378f1fd640 1 --2- 172.21.15.115:0/1717407156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f378806b970 0x7f37880adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.688+0000 7f378f1fd640 1 -- 172.21.15.115:0/1717407156 >> 172.21.15.115:0/1717407156 conn(0x7f37880cee40 msgr2=0x7f37880b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:07.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.689+0000 7f378f1fd640 1 -- 172.21.15.115:0/1717407156 shutdown_connections 2023-12-01T03:12:07.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.689+0000 7f378f1fd640 1 -- 172.21.15.115:0/1717407156 wait complete. 2023-12-01T03:12:07.708 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T03:12:07.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.896+0000 7f9e5ffff640 1 Processor -- start 2023-12-01T03:12:07.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.896+0000 7f9e5ffff640 1 -- start start 2023-12-01T03:12:07.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.897+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015c9c0 0x7f9e600b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.897+0000 7f9e5ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9e600b59d0 0x7f9e600cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.897+0000 7f9e5ffff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e600b1f60 con 0x7f9e6015cda0 2023-12-01T03:12:07.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.897+0000 7f9e5ffff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e600d2860 con 0x7f9e600b59d0 2023-12-01T03:12:07.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.897+0000 7f9e5ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e600d0ab0 con 0x7f9e6015c9c0 2023-12-01T03:12:07.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.898+0000 7f9e57fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9e600b59d0 0x7f9e600cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.898+0000 7f9e57fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9e600b59d0 0x7f9e600cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52480/0 (socket says 172.21.15.115:52480) 2023-12-01T03:12:07.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.898+0000 7f9e57fff640 1 -- 172.21.15.115:0/1812995172 learned_addr learned my addr 172.21.15.115:0/1812995172 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:07.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.898+0000 7f9e5effd640 1 --2- 172.21.15.115:0/1812995172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015c9c0 0x7f9e600b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.898+0000 7f9e57fff640 1 -- 172.21.15.115:0/1812995172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015c9c0 msgr2=0x7f9e600b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:07.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.898+0000 7f9e57fff640 1 --2- 172.21.15.115:0/1812995172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015c9c0 0x7f9e600b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.898+0000 7f9e57fff640 1 -- 172.21.15.115:0/1812995172 >> v1:172.21.15.115:6789/0 conn(0x7f9e6015cda0 legacy=0x7f9e600b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.898+0000 7f9e57fff640 1 -- 172.21.15.115:0/1812995172 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e600d0d90 con 0x7f9e600b59d0 2023-12-01T03:12:07.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.898+0000 7f9e5e7fc640 1 -- 172.21.15.115:0/1812995172 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355088469 0 0) 0x7f9e600b1f60 con 0x7f9e6015cda0 2023-12-01T03:12:07.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.899+0000 7f9e57fff640 1 --2- 172.21.15.115:0/1812995172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 0x7f9e600cf3c0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009870 tx=0x7f9e4802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272058d85cd0e1de server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:07.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.900+0000 7f9e5e7fc640 1 -- 172.21.15.115:0/1812995172 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4802f840 con 0x7f9e600b59d0 2023-12-01T03:12:07.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.900+0000 7f9e5e7fc640 1 -- 172.21.15.115:0/1812995172 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4802f9a0 con 0x7f9e600b59d0 2023-12-01T03:12:07.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.900+0000 7f9e5e7fc640 1 -- 172.21.15.115:0/1812995172 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4802fd30 con 0x7f9e600b59d0 2023-12-01T03:12:07.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.900+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1812995172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 msgr2=0x7f9e600cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.900+0000 7f9e5ffff640 1 --2- 172.21.15.115:0/1812995172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 0x7f9e600cf3c0 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009870 tx=0x7f9e4802ba20 comp rx=0 tx=0).stop 2023-12-01T03:12:07.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.900+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1812995172 shutdown_connections 2023-12-01T03:12:07.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.900+0000 7f9e5ffff640 1 --2- 172.21.15.115:0/1812995172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 0x7f9e600cf3c0 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.900+0000 7f9e5ffff640 1 --2- 172.21.15.115:0/1812995172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015c9c0 0x7f9e600b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.900+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1812995172 >> 172.21.15.115:0/1812995172 conn(0x7f9e60065960 msgr2=0x7f9e60065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:07.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.901+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1812995172 shutdown_connections 2023-12-01T03:12:07.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.901+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1812995172 wait complete. 2023-12-01T03:12:07.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.902+0000 7f9e5ffff640 1 Processor -- start 2023-12-01T03:12:07.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.902+0000 7f9e5ffff640 1 -- start start 2023-12-01T03:12:07.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.903+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 0x7f9e6014a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.903+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6015c9c0 0x7f9e601f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.903+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015cda0 0x7f9e600b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.903+0000 7f9e5ffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6015e320 con 0x7f9e600b59d0 2023-12-01T03:12:07.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.903+0000 7f9e5ffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6014afa0 con 0x7f9e6015c9c0 2023-12-01T03:12:07.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.903+0000 7f9e5ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e601f6140 con 0x7f9e6015cda0 2023-12-01T03:12:07.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 0x7f9e6014a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5effd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 0x7f9e6014a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52482/0 (socket says 172.21.15.115:52482) 2023-12-01T03:12:07.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5effd640 1 -- 172.21.15.115:0/1740071034 learned_addr learned my addr 172.21.15.115:0/1740071034 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:07.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e57fff640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6015c9c0 0x7f9e601f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5f7fe640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015cda0 0x7f9e600b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:07.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5effd640 1 -- 172.21.15.115:0/1740071034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015cda0 msgr2=0x7f9e600b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5effd640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015cda0 0x7f9e600b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5effd640 1 -- 172.21.15.115:0/1740071034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6015c9c0 msgr2=0x7f9e601f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:07.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5effd640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6015c9c0 0x7f9e601f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:07.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5effd640 1 -- 172.21.15.115:0/1740071034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e480094d0 con 0x7f9e600b59d0 2023-12-01T03:12:07.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5f7fe640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015cda0 0x7f9e600b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:07.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e5effd640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 0x7f9e6014a790 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000d800 tx=0x7f9e5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:07.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.904+0000 7f9e577fe640 1 -- 172.21.15.115:0/1740071034 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000b6c0 con 0x7f9e600b59d0 2023-12-01T03:12:07.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.905+0000 7f9e577fe640 1 -- 172.21.15.115:0/1740071034 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5000b820 con 0x7f9e600b59d0 2023-12-01T03:12:07.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.905+0000 7f9e577fe640 1 -- 172.21.15.115:0/1740071034 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000bbb0 con 0x7f9e600b59d0 2023-12-01T03:12:07.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.905+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1740071034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e601f6420 con 0x7f9e600b59d0 2023-12-01T03:12:07.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.905+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1740071034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6015e810 con 0x7f9e600b59d0 2023-12-01T03:12:07.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.906+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1740071034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e601f6610 con 0x7f9e600b59d0 2023-12-01T03:12:07.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.908+0000 7f9e577fe640 1 -- 172.21.15.115:0/1740071034 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 138) v1 ==== 88812+0+0 (secure 0 0 0) 0x7f9e5000bd10 con 0x7f9e600b59d0 2023-12-01T03:12:07.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.908+0000 7f9e577fe640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4006a6a0 0x7f9e4006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:07.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.909+0000 7f9e577fe640 1 -- 172.21.15.115:0/1740071034 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f9e50089830 con 0x7f9e600b59d0 2023-12-01T03:12:07.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.909+0000 7f9e57fff640 1 -- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4006a6a0 msgr2=0x7f9e4006cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:07.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.909+0000 7f9e57fff640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4006a6a0 0x7f9e4006cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:07.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:07.913+0000 7f9e577fe640 1 -- 172.21.15.115:0/1740071034 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9e601f6610 con 0x7f9e600b59d0 2023-12-01T03:12:08.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.110+0000 7f9e57fff640 1 -- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4006a6a0 msgr2=0x7f9e4006cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:08.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.110+0000 7f9e57fff640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4006a6a0 0x7f9e4006cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:12:08.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.117+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1740071034 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9e601f6610 con 0x7f9e600b59d0 2023-12-01T03:12:08.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.118+0000 7f9e577fe640 1 -- 172.21.15.115:0/1740071034 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v240) v1 ==== 81+0+585 (secure 0 0 0) 0x7f9e601f6610 con 0x7f9e600b59d0 2023-12-01T03:12:08.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/638241265 2023-12-02T03:12:02.677778+0000 2023-12-01T03:12:08.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3713299174 2023-12-02T03:12:02.677778+0000 2023-12-01T03:12:08.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/4124642007 2023-12-02T03:12:02.677778+0000 2023-12-01T03:12:08.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/138932 2023-12-02T03:12:02.677778+0000 2023-12-01T03:12:08.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/138932 2023-12-02T03:12:02.677778+0000 2023-12-01T03:12:08.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3863453309 2023-12-02T03:12:02.677778+0000 2023-12-01T03:12:08.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1160795001 2023-12-02T03:12:02.677778+0000 2023-12-01T03:12:08.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/524060219 2023-12-02T03:12:02.677778+0000 2023-12-01T03:12:08.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/1801977880 2023-12-02T03:12:02.677778+0000 2023-12-01T03:12:08.120 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/203535105 2023-12-02T03:12:02.677778+0000 2023-12-01T03:12:08.121 INFO:teuthology.orchestra.run.smithi115.stderr:listed 10 entries 2023-12-01T03:12:08.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.120+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4006a6a0 msgr2=0x7f9e4006cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:08.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.120+0000 7f9e5ffff640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4006a6a0 0x7f9e4006cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.120+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 msgr2=0x7f9e6014a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:08.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.120+0000 7f9e5ffff640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 0x7f9e6014a790 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000d800 tx=0x7f9e5000dcc0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.120+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1740071034 shutdown_connections 2023-12-01T03:12:08.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.120+0000 7f9e5ffff640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9e4006a6a0 0x7f9e4006cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.120+0000 7f9e5ffff640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e6015cda0 0x7f9e600b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.120+0000 7f9e5ffff640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e6015c9c0 0x7f9e601f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.121+0000 7f9e5ffff640 1 --2- 172.21.15.115:0/1740071034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e600b59d0 0x7f9e6014a790 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.121+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1740071034 >> 172.21.15.115:0/1740071034 conn(0x7f9e60065960 msgr2=0x7f9e600cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:08.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.121+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1740071034 shutdown_connections 2023-12-01T03:12:08.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.121+0000 7f9e5ffff640 1 -- 172.21.15.115:0/1740071034 wait complete. 2023-12-01T03:12:08.140 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T03:12:08.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:08.140 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T03:12:08.221 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T03:12:08.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:08.221 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T03:12:08.266 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:12:08.266 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:08.318 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:12:08.318 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:12:08.319 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:08.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:12:08.373 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:12:08.373 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:08.409 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:08.408+0000 7ff192c2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:12:08.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:12:08.427 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:12:08.511 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:08.510+0000 7f4a4e416040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:12:08.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.629+0000 7f8f1b53d640 1 Processor -- start 2023-12-01T03:12:08.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.630+0000 7f8f1b53d640 1 -- start start 2023-12-01T03:12:08.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.631+0000 7f8f1b53d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8f1415d7b0 0x7f8f1415db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:08.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.631+0000 7f8f1b53d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8f14160670 0x7f8f140cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:08.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.631+0000 7f8f1b53d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f140cddc0 0x7f8f140d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:08.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.631+0000 7f8f1b53d640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f140cd3f0 con 0x7f8f1415d7b0 2023-12-01T03:12:08.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.631+0000 7f8f1b53d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f140cd0f0 con 0x7f8f14160670 2023-12-01T03:12:08.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.631+0000 7f8f1b53d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f14160ec0 con 0x7f8f140cddc0 2023-12-01T03:12:08.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.631+0000 7f8f1a53b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8f1415d7b0 0x7f8f1415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:08.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.631+0000 7f8f1a53b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8f1415d7b0 0x7f8f1415db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52484/0 (socket says 172.21.15.115:52484) 2023-12-01T03:12:08.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.631+0000 7f8f1a53b640 1 -- 172.21.15.115:0/611954880 learned_addr learned my addr 172.21.15.115:0/611954880 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:08.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.632+0000 7f8f19d3a640 1 --2- 172.21.15.115:0/611954880 >> v2:172.21.15.142:3300/0 conn(0x7f8f14160670 0x7f8f140cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:08.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.632+0000 7f8f1ad3c640 1 --2- 172.21.15.115:0/611954880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f140cddc0 0x7f8f140d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:08.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.632+0000 7f8f1a53b640 1 -- 172.21.15.115:0/611954880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f140cddc0 msgr2=0x7f8f140d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:08.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.632+0000 7f8f1a53b640 1 --2- 172.21.15.115:0/611954880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f140cddc0 0x7f8f140d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.632+0000 7f8f1a53b640 1 -- 172.21.15.115:0/611954880 >> v2:172.21.15.142:3300/0 conn(0x7f8f14160670 msgr2=0x7f8f140cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:08.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.632+0000 7f8f1a53b640 1 --2- 172.21.15.115:0/611954880 >> v2:172.21.15.142:3300/0 conn(0x7f8f14160670 0x7f8f140cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.632+0000 7f8f1a53b640 1 -- 172.21.15.115:0/611954880 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f141611a0 con 0x7f8f1415d7b0 2023-12-01T03:12:08.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.632+0000 7f8f19d3a640 1 --2- 172.21.15.115:0/611954880 >> v2:172.21.15.142:3300/0 conn(0x7f8f14160670 0x7f8f140cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:08.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.633+0000 7f8f1a53b640 1 --2- 172.21.15.115:0/611954880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1415d7b0 0x7f8f1415db90 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f8f140d1ba0 tx=0x7f8f1002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ea8ceb7be9e8c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:08.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.633+0000 7f8f19539640 1 -- 172.21.15.115:0/611954880 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10030850 con 0x7f8f1415d7b0 2023-12-01T03:12:08.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.633+0000 7f8f19539640 1 -- 172.21.15.115:0/611954880 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f100309b0 con 0x7f8f1415d7b0 2023-12-01T03:12:08.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.633+0000 7f8f19539640 1 -- 172.21.15.115:0/611954880 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10030d40 con 0x7f8f1415d7b0 2023-12-01T03:12:08.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.634+0000 7f8f1b53d640 1 -- 172.21.15.115:0/611954880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1415d7b0 msgr2=0x7f8f1415db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:08.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.634+0000 7f8f1b53d640 1 --2- 172.21.15.115:0/611954880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1415d7b0 0x7f8f1415db90 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f8f140d1ba0 tx=0x7f8f1002b990 comp rx=0 tx=0).stop 2023-12-01T03:12:08.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.634+0000 7f8f1b53d640 1 -- 172.21.15.115:0/611954880 shutdown_connections 2023-12-01T03:12:08.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.634+0000 7f8f1b53d640 1 --2- 172.21.15.115:0/611954880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f140cddc0 0x7f8f140d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.634+0000 7f8f1b53d640 1 --2- 172.21.15.115:0/611954880 >> v2:172.21.15.142:3300/0 conn(0x7f8f14160670 0x7f8f140cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.634+0000 7f8f1b53d640 1 --2- 172.21.15.115:0/611954880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f1415d7b0 0x7f8f1415db90 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.634+0000 7f8f1b53d640 1 -- 172.21.15.115:0/611954880 >> 172.21.15.115:0/611954880 conn(0x7f8f1406a0e0 msgr2=0x7f8f1406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:08.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.634+0000 7f8f1b53d640 1 -- 172.21.15.115:0/611954880 shutdown_connections 2023-12-01T03:12:08.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.634+0000 7f8f1b53d640 1 -- 172.21.15.115:0/611954880 wait complete. 2023-12-01T03:12:08.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.635+0000 7f8f1b53d640 1 Processor -- start 2023-12-01T03:12:08.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.636+0000 7f8f1b53d640 1 -- start start 2023-12-01T03:12:08.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.636+0000 7f8f1b53d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f140cddc0 0x7f8f14151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:08.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1b53d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f1415d7b0 0x7f8f140cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:08.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1b53d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f14160670 0x7f8f141f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:08.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1b53d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f14152230 con 0x7f8f140cddc0 2023-12-01T03:12:08.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1b53d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f140ca580 con 0x7f8f1415d7b0 2023-12-01T03:12:08.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1b53d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f140ca820 con 0x7f8f14160670 2023-12-01T03:12:08.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1a53b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f140cddc0 0x7f8f14151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:08.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1a53b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f140cddc0 0x7f8f14151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52500/0 (socket says 172.21.15.115:52500) 2023-12-01T03:12:08.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1a53b640 1 -- 172.21.15.115:0/2962999588 learned_addr learned my addr 172.21.15.115:0/2962999588 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:08.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f19d3a640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f1415d7b0 0x7f8f140cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:08.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1ad3c640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f14160670 0x7f8f141f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:08.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1a53b640 1 -- 172.21.15.115:0/2962999588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f14160670 msgr2=0x7f8f141f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:08.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1a53b640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f14160670 0x7f8f141f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1a53b640 1 -- 172.21.15.115:0/2962999588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f1415d7b0 msgr2=0x7f8f140cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:08.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1a53b640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f1415d7b0 0x7f8f140cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.637+0000 7f8f1a53b640 1 -- 172.21.15.115:0/2962999588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100094d0 con 0x7f8f140cddc0 2023-12-01T03:12:08.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.638+0000 7f8f1a53b640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f140cddc0 0x7f8f14151cf0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f8f1002b9d0 tx=0x7f8f10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:08.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.638+0000 7f8f0affd640 1 -- 172.21.15.115:0/2962999588 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10030840 con 0x7f8f140cddc0 2023-12-01T03:12:08.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.638+0000 7f8f0affd640 1 -- 172.21.15.115:0/2962999588 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f10043400 con 0x7f8f140cddc0 2023-12-01T03:12:08.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.638+0000 7f8f0affd640 1 -- 172.21.15.115:0/2962999588 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f100437b0 con 0x7f8f140cddc0 2023-12-01T03:12:08.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.638+0000 7f8f1b53d640 1 -- 172.21.15.115:0/2962999588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f140cab00 con 0x7f8f140cddc0 2023-12-01T03:12:08.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.638+0000 7f8f1b53d640 1 -- 172.21.15.115:0/2962999588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f140cb0c0 con 0x7f8f140cddc0 2023-12-01T03:12:08.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.639+0000 7f8f1b53d640 1 -- 172.21.15.115:0/2962999588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f140cb2b0 con 0x7f8f140cddc0 2023-12-01T03:12:08.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.640+0000 7f8f0affd640 1 -- 172.21.15.115:0/2962999588 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 138) v1 ==== 88812+0+0 (secure 0 0 0) 0x7f8f1002e070 con 0x7f8f140cddc0 2023-12-01T03:12:08.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.641+0000 7f8f0affd640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ef806a670 0x7f8ef806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:08.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.641+0000 7f8f19d3a640 1 -- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ef806a670 msgr2=0x7f8ef806cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:08.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.641+0000 7f8f0affd640 1 -- 172.21.15.115:0/2962999588 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7047+0+0 (secure 0 0 0) 0x7f8f1003e070 con 0x7f8f140cddc0 2023-12-01T03:12:08.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.641+0000 7f8f19d3a640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ef806a670 0x7f8ef806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:08.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.646+0000 7f8f0affd640 1 -- 172.21.15.115:0/2962999588 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8f10076fd0 con 0x7f8f140cddc0 2023-12-01T03:12:08.709 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:08.707+0000 7ff192c2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:12:08.767 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:08.766+0000 7ff192c2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:12:08.822 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:08.821+0000 7f4a4e416040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:12:08.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.842+0000 7f8f19d3a640 1 -- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ef806a670 msgr2=0x7f8ef806cb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:08.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.842+0000 7f8f19d3a640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ef806a670 0x7f8ef806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:12:08.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.864+0000 7f8f1b53d640 1 -- 172.21.15.115:0/2962999588 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f140673a0 con 0x7f8f140cddc0 2023-12-01T03:12:08.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.865+0000 7f8f0affd640 1 -- 172.21.15.115:0/2962999588 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5786 (secure 0 0 0) 0x7f8f1007a5f0 con 0x7f8f140cddc0 2023-12-01T03:12:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:08.867 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:32.176438+0000","modified":"2023-12-01T03:11:35.672576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7735":{"gid":7735,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1837882056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1837882056},{"type":"v1","addr":"172.21.15.142:6835","nonce":1837882056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:51.037525+0000","modified":"2023-12-01T03:11:54.055433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7732":{"gid":7732,"name":"b","rank":0,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-01T03:12:08.867 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 116 2023-12-01T03:12:08.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 -- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ef806a670 msgr2=0x7f8ef806cb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:08.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ef806a670 0x7f8ef806cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 -- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f140cddc0 msgr2=0x7f8f14151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:08.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f140cddc0 0x7f8f14151cf0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f8f1002b9d0 tx=0x7f8f10004730 comp rx=0 tx=0).stop 2023-12-01T03:12:08.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 -- 172.21.15.115:0/2962999588 shutdown_connections 2023-12-01T03:12:08.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8ef806a670 0x7f8ef806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f14160670 0x7f8f141f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f1415d7b0 0x7f8f140cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 --2- 172.21.15.115:0/2962999588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f140cddc0 0x7f8f14151cf0 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:08.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 -- 172.21.15.115:0/2962999588 >> 172.21.15.115:0/2962999588 conn(0x7f8f1406a0e0 msgr2=0x7f8f140cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:08.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 -- 172.21.15.115:0/2962999588 shutdown_connections 2023-12-01T03:12:08.872 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:08.869+0000 7f8f1b53d640 1 -- 172.21.15.115:0/2962999588 wait complete. 2023-12-01T03:12:08.883 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:08.879+0000 7ff192c2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:12:08.892 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:08.890+0000 7f4a4e416040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:12:08.897 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:09.012 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.010+0000 7f4a4e416040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:12:09.103 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.101+0000 7f49d2c18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.103 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.101+0000 7f49d2c18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.103 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.101+0000 7f49d2c18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.103 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.101+0000 7f49d2c18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.103 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.101+0000 7f49d2c18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.104 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.102+0000 7f49d0413640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.104 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.102+0000 7f49d0413640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.104 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.102+0000 7f49d0413640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.104 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.102+0000 7f49d0413640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.104 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.102+0000 7f49d0413640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.119 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.118+0000 7f49d0413640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.120 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.118+0000 7f49d0413640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.120 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.118+0000 7f49d0413640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.120 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.118+0000 7f49d0413640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.120 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.118+0000 7f49d0413640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.120 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.118+0000 7f49d2c18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.120 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.118+0000 7f49d2c18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.120 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.119+0000 7f49d2c18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.121 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.119+0000 7f49d2c18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.121 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:09.119+0000 7f49d2c18640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:09.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.120+0000 7f32d73d2640 1 Processor -- start 2023-12-01T03:12:09.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.121+0000 7f32d73d2640 1 -- start start 2023-12-01T03:12:09.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d73d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d015cd20 0x7f32d00d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d73d2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f32d00b5d80 0x7f32d00b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d73d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00ce2e0 con 0x7f32d00d1230 2023-12-01T03:12:09.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d73d2640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d00b2160 con 0x7f32d00b5d80 2023-12-01T03:12:09.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d73d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00d2a10 con 0x7f32d015cd20 2023-12-01T03:12:09.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d496d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f32d00b5d80 0x7f32d00b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d496d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f32d00b5d80 0x7f32d00b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52644/0 (socket says 172.21.15.115:52644) 2023-12-01T03:12:09.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d496d640 1 -- 172.21.15.115:0/3772120532 learned_addr learned my addr 172.21.15.115:0/3772120532 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:09.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d496d640 1 -- 172.21.15.115:0/3772120532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d015cd20 msgr2=0x7f32d00d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:09.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d496d640 1 --2- 172.21.15.115:0/3772120532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d015cd20 0x7f32d00d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d496d640 1 -- 172.21.15.115:0/3772120532 >> v1:172.21.15.142:6789/0 conn(0x7f32d00d1230 legacy=0x7f32d00d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.122+0000 7f32d496d640 1 -- 172.21.15.115:0/3772120532 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00ce000 con 0x7f32d00b5d80 2023-12-01T03:12:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.123+0000 7f32d496d640 1 --2- 172.21.15.115:0/3772120532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d00b5d80 0x7f32d00b8570 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f32c00098a0 tx=0x7f32c002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aead614d36eb479f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.124+0000 7f32c7fff640 1 -- 172.21.15.115:0/3772120532 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00317c0 con 0x7f32d00b5d80 2023-12-01T03:12:09.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.124+0000 7f32c7fff640 1 -- 172.21.15.115:0/3772120532 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0031920 con 0x7f32d00b5d80 2023-12-01T03:12:09.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.124+0000 7f32c7fff640 1 -- 172.21.15.115:0/3772120532 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0031cb0 con 0x7f32d00b5d80 2023-12-01T03:12:09.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.124+0000 7f32d73d2640 1 -- 172.21.15.115:0/3772120532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d00b5d80 msgr2=0x7f32d00b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.124+0000 7f32d73d2640 1 --2- 172.21.15.115:0/3772120532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d00b5d80 0x7f32d00b8570 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f32c00098a0 tx=0x7f32c002b860 comp rx=0 tx=0).stop 2023-12-01T03:12:09.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.124+0000 7f32d73d2640 1 -- 172.21.15.115:0/3772120532 shutdown_connections 2023-12-01T03:12:09.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.124+0000 7f32d73d2640 1 --2- 172.21.15.115:0/3772120532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d00b5d80 0x7f32d00b8570 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.124+0000 7f32d73d2640 1 --2- 172.21.15.115:0/3772120532 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d015cd20 0x7f32d00d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.124+0000 7f32d73d2640 1 -- 172.21.15.115:0/3772120532 >> 172.21.15.115:0/3772120532 conn(0x7f32d006a0e0 msgr2=0x7f32d006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:09.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.125+0000 7f32d73d2640 1 -- 172.21.15.115:0/3772120532 shutdown_connections 2023-12-01T03:12:09.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.125+0000 7f32d73d2640 1 -- 172.21.15.115:0/3772120532 wait complete. 2023-12-01T03:12:09.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.126+0000 7f32d73d2640 1 Processor -- start 2023-12-01T03:12:09.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.126+0000 7f32d73d2640 1 -- start start 2023-12-01T03:12:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.127+0000 7f32d73d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d00b5d80 0x7f32d015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.127+0000 7f32d73d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d00d1230 0x7f32d015fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.127+0000 7f32d73d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d015cd20 0x7f32d01f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.127+0000 7f32d73d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015eaf0 con 0x7f32d015cd20 2023-12-01T03:12:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.127+0000 7f32d73d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015d640 con 0x7f32d00d1230 2023-12-01T03:12:09.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.127+0000 7f32d73d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015ed70 con 0x7f32d00b5d80 2023-12-01T03:12:09.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d596f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d015cd20 0x7f32d01f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d596f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d015cd20 0x7f32d01f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52656/0 (socket says 172.21.15.115:52656) 2023-12-01T03:12:09.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d596f640 1 -- 172.21.15.115:0/1301977060 learned_addr learned my addr 172.21.15.115:0/1301977060 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:09.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d496d640 1 --2- 172.21.15.115:0/1301977060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d00d1230 0x7f32d015fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d516e640 1 --2- 172.21.15.115:0/1301977060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d00b5d80 0x7f32d015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d596f640 1 -- 172.21.15.115:0/1301977060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d00b5d80 msgr2=0x7f32d015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d596f640 1 --2- 172.21.15.115:0/1301977060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d00b5d80 0x7f32d015d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d596f640 1 -- 172.21.15.115:0/1301977060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d00d1230 msgr2=0x7f32d015fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d596f640 1 --2- 172.21.15.115:0/1301977060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d00d1230 0x7f32d015fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d596f640 1 -- 172.21.15.115:0/1301977060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c00094d0 con 0x7f32d015cd20 2023-12-01T03:12:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d496d640 1 --2- 172.21.15.115:0/1301977060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d00d1230 0x7f32d015fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.128+0000 7f32d596f640 1 --2- 172.21.15.115:0/1301977060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d015cd20 0x7f32d01f26d0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f32c800c780 tx=0x7f32c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.129+0000 7f32c5ffb640 1 -- 172.21.15.115:0/1301977060 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8003d80 con 0x7f32d015cd20 2023-12-01T03:12:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.129+0000 7f32c5ffb640 1 -- 172.21.15.115:0/1301977060 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c800ce90 con 0x7f32d015cd20 2023-12-01T03:12:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.129+0000 7f32c5ffb640 1 -- 172.21.15.115:0/1301977060 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c80052b0 con 0x7f32d015cd20 2023-12-01T03:12:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.129+0000 7f32d73d2640 1 -- 172.21.15.115:0/1301977060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d015f050 con 0x7f32d015cd20 2023-12-01T03:12:09.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.129+0000 7f32d73d2640 1 -- 172.21.15.115:0/1301977060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d015f610 con 0x7f32d015cd20 2023-12-01T03:12:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.130+0000 7f32d73d2640 1 -- 172.21.15.115:0/1301977060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d015f800 con 0x7f32d015cd20 2023-12-01T03:12:09.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.131+0000 7f32c5ffb640 1 -- 172.21.15.115:0/1301977060 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 139) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f32c8005410 con 0x7f32d015cd20 2023-12-01T03:12:09.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.132+0000 7f32c5ffb640 1 -- 172.21.15.115:0/1301977060 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7f32c8087fe0 con 0x7f32d015cd20 2023-12-01T03:12:09.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.136+0000 7f32c5ffb640 1 -- 172.21.15.115:0/1301977060 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f32c8053340 con 0x7f32d015cd20 2023-12-01T03:12:09.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.313+0000 7f32d73d2640 1 -- 172.21.15.115:0/1301977060 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32d004e890 con 0x7f32d015cd20 2023-12-01T03:12:09.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.317+0000 7f32c5ffb640 1 -- 172.21.15.115:0/1301977060 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v241) v1 ==== 74+0+21019 (secure 0 0 0) 0x7f32c8017090 con 0x7f32d015cd20 2023-12-01T03:12:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:09.319 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":241,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:09.014479+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:11:49.229904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:11:50.056441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2980649202":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/1025490173":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2408987063":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3619877240":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2405923893":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/957514002":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/638241265":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1722505219":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3713299174":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/4124642007":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6824/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/3863453309":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1160795001":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/988288848":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/524060219":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1801977880":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/203535105":"2023-12-02T03:12:02.677778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:09.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.320+0000 7f32d73d2640 1 -- 172.21.15.115:0/1301977060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d015cd20 msgr2=0x7f32d01f26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.320+0000 7f32d73d2640 1 --2- 172.21.15.115:0/1301977060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d015cd20 0x7f32d01f26d0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f32c800c780 tx=0x7f32c800cc40 comp rx=0 tx=0).stop 2023-12-01T03:12:09.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.320+0000 7f32d73d2640 1 -- 172.21.15.115:0/1301977060 shutdown_connections 2023-12-01T03:12:09.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.320+0000 7f32d73d2640 1 --2- 172.21.15.115:0/1301977060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32d015cd20 0x7f32d01f26d0 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.320+0000 7f32d73d2640 1 --2- 172.21.15.115:0/1301977060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32d00d1230 0x7f32d015fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.321+0000 7f32d73d2640 1 --2- 172.21.15.115:0/1301977060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32d00b5d80 0x7f32d015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.321+0000 7f32d73d2640 1 -- 172.21.15.115:0/1301977060 >> 172.21.15.115:0/1301977060 conn(0x7f32d006a0e0 msgr2=0x7f32d0157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:09.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.321+0000 7f32d73d2640 1 -- 172.21.15.115:0/1301977060 shutdown_connections 2023-12-01T03:12:09.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.321+0000 7f32d73d2640 1 -- 172.21.15.115:0/1301977060 wait complete. 2023-12-01T03:12:09.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T03:11:30.230656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '234', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T03:11:31.144773+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '235', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-01T03:11:49.229904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '238', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-01T03:11:50.056441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '239', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:12:09.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:12:09.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.546+0000 7f8f4eca1640 1 Processor -- start 2023-12-01T03:12:09.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.546+0000 7f8f4eca1640 1 -- start start 2023-12-01T03:12:09.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.547+0000 7f8f4eca1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8f4815ac80 0x7f8f480cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.547+0000 7f8f4eca1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8f480ce1b0 0x7f8f480b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.547+0000 7f8f4eca1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480cc9a0 con 0x7f8f480b0e60 2023-12-01T03:12:09.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f4eca1640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4815b060 con 0x7f8f480ce1b0 2023-12-01T03:12:09.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f4eca1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480b0ce0 con 0x7f8f4815ac80 2023-12-01T03:12:09.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f47fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8f480b0e60 0x7f8f480cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59952/0 (socket says 172.21.15.115:59952) 2023-12-01T03:12:09.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f4d23e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8f480ce1b0 0x7f8f480b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f47fff640 1 -- 172.21.15.115:0/1667418391 learned_addr learned my addr 172.21.15.115:0/1667418391 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:09.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f4ca3d640 1 --2- 172.21.15.115:0/1667418391 >> v2:172.21.15.142:3300/0 conn(0x7f8f4815ac80 0x7f8f480cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f477fe640 1 -- 172.21.15.115:0/1667418391 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228133186 0 0) 0x7f8f480cc9a0 con 0x7f8f480b0e60 2023-12-01T03:12:09.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f4d23e640 1 -- 172.21.15.115:0/1667418391 >> v2:172.21.15.142:3300/0 conn(0x7f8f4815ac80 msgr2=0x7f8f480cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:09.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f4d23e640 1 --2- 172.21.15.115:0/1667418391 >> v2:172.21.15.142:3300/0 conn(0x7f8f4815ac80 0x7f8f480cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f4d23e640 1 -- 172.21.15.115:0/1667418391 >> v1:172.21.15.115:6789/0 conn(0x7f8f480b0e60 legacy=0x7f8f480cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f4d23e640 1 -- 172.21.15.115:0/1667418391 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f480cbd90 con 0x7f8f480ce1b0 2023-12-01T03:12:09.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.548+0000 7f8f4ca3d640 1 --2- 172.21.15.115:0/1667418391 >> v2:172.21.15.142:3300/0 conn(0x7f8f4815ac80 0x7f8f480cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:09.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.549+0000 7f8f4d23e640 1 --2- 172.21.15.115:0/1667418391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480ce1b0 0x7f8f480b07a0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f8f480b3cc0 tx=0x7f8f4002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c21e6447ab9938 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:09.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.550+0000 7f8f477fe640 1 -- 172.21.15.115:0/1667418391 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4002cc80 con 0x7f8f480ce1b0 2023-12-01T03:12:09.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.550+0000 7f8f477fe640 1 -- 172.21.15.115:0/1667418391 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4002cde0 con 0x7f8f480ce1b0 2023-12-01T03:12:09.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.550+0000 7f8f477fe640 1 -- 172.21.15.115:0/1667418391 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40034660 con 0x7f8f480ce1b0 2023-12-01T03:12:09.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.550+0000 7f8f4eca1640 1 -- 172.21.15.115:0/1667418391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480ce1b0 msgr2=0x7f8f480b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.550+0000 7f8f4eca1640 1 --2- 172.21.15.115:0/1667418391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480ce1b0 0x7f8f480b07a0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f8f480b3cc0 tx=0x7f8f4002bed0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.550+0000 7f8f4eca1640 1 -- 172.21.15.115:0/1667418391 shutdown_connections 2023-12-01T03:12:09.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.550+0000 7f8f4eca1640 1 --2- 172.21.15.115:0/1667418391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480ce1b0 0x7f8f480b07a0 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.550+0000 7f8f4eca1640 1 --2- 172.21.15.115:0/1667418391 >> v2:172.21.15.142:3300/0 conn(0x7f8f4815ac80 0x7f8f480cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.550+0000 7f8f4eca1640 1 -- 172.21.15.115:0/1667418391 >> 172.21.15.115:0/1667418391 conn(0x7f8f48065960 msgr2=0x7f8f48065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:09.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.550+0000 7f8f4eca1640 1 -- 172.21.15.115:0/1667418391 shutdown_connections 2023-12-01T03:12:09.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.551+0000 7f8f4eca1640 1 -- 172.21.15.115:0/1667418391 wait complete. 2023-12-01T03:12:09.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.552+0000 7f8f4eca1640 1 Processor -- start 2023-12-01T03:12:09.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.552+0000 7f8f4eca1640 1 -- start start 2023-12-01T03:12:09.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.553+0000 7f8f4eca1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480b0e60 0x7f8f48153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.553+0000 7f8f4eca1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f480ce1b0 0x7f8f48157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.553+0000 7f8f4eca1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4815ac80 0x7f8f480c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.553+0000 7f8f4eca1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48153850 con 0x7f8f480b0e60 2023-12-01T03:12:09.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.553+0000 7f8f4eca1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48158220 con 0x7f8f480ce1b0 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.553+0000 7f8f4eca1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48158500 con 0x7f8f4815ac80 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.553+0000 7f8f4ca3d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480b0e60 0x7f8f48153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.553+0000 7f8f4ca3d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480b0e60 0x7f8f48153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52674/0 (socket says 172.21.15.115:52674) 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.553+0000 7f8f4ca3d640 1 -- 172.21.15.115:0/4267059419 learned_addr learned my addr 172.21.15.115:0/4267059419 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f4ca3d640 1 -- 172.21.15.115:0/4267059419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4815ac80 msgr2=0x7f8f480c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f4ca3d640 1 --2- 172.21.15.115:0/4267059419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4815ac80 0x7f8f480c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f47fff640 1 --2- 172.21.15.115:0/4267059419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f480ce1b0 0x7f8f48157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f4ca3d640 1 -- 172.21.15.115:0/4267059419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f480ce1b0 msgr2=0x7f8f48157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f4ca3d640 1 --2- 172.21.15.115:0/4267059419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f480ce1b0 0x7f8f48157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f4ca3d640 1 -- 172.21.15.115:0/4267059419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f40009a70 con 0x7f8f480b0e60 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f4ca3d640 1 --2- 172.21.15.115:0/4267059419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480b0e60 0x7f8f48153310 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c7f0 tx=0x7f8f3c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f457fa640 1 -- 172.21.15.115:0/4267059419 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c007820 con 0x7f8f480b0e60 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f457fa640 1 -- 172.21.15.115:0/4267059419 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c007980 con 0x7f8f480b0e60 2023-12-01T03:12:09.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f457fa640 1 -- 172.21.15.115:0/4267059419 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c007cf0 con 0x7f8f480b0e60 2023-12-01T03:12:09.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.554+0000 7f8f4eca1640 1 -- 172.21.15.115:0/4267059419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f481587e0 con 0x7f8f480b0e60 2023-12-01T03:12:09.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.555+0000 7f8f4eca1640 1 -- 172.21.15.115:0/4267059419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f48157e60 con 0x7f8f480b0e60 2023-12-01T03:12:09.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.556+0000 7f8f4eca1640 1 -- 172.21.15.115:0/4267059419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4804e890 con 0x7f8f480b0e60 2023-12-01T03:12:09.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.556+0000 7f8f457fa640 1 -- 172.21.15.115:0/4267059419 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 139) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f8f3c0113f0 con 0x7f8f480b0e60 2023-12-01T03:12:09.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.557+0000 7f8f457fa640 1 -- 172.21.15.115:0/4267059419 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7f8f3c088d60 con 0x7f8f480b0e60 2023-12-01T03:12:09.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.561+0000 7f8f457fa640 1 -- 172.21.15.115:0/4267059419 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8f3c054110 con 0x7f8f480b0e60 2023-12-01T03:12:09.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.737+0000 7f8f4eca1640 1 -- 172.21.15.115:0/4267059419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f480b1810 con 0x7f8f480b0e60 2023-12-01T03:12:09.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.738+0000 7f8f457fa640 1 -- 172.21.15.115:0/4267059419 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v241) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f3c01f0a0 con 0x7f8f480b0e60 2023-12-01T03:12:09.740 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:12:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.741+0000 7f8f4eca1640 1 -- 172.21.15.115:0/4267059419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480b0e60 msgr2=0x7f8f48153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.741+0000 7f8f4eca1640 1 --2- 172.21.15.115:0/4267059419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480b0e60 0x7f8f48153310 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c7f0 tx=0x7f8f3c00ccb0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.741+0000 7f8f4eca1640 1 -- 172.21.15.115:0/4267059419 shutdown_connections 2023-12-01T03:12:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.741+0000 7f8f4eca1640 1 --2- 172.21.15.115:0/4267059419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f4815ac80 0x7f8f480c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.741+0000 7f8f4eca1640 1 --2- 172.21.15.115:0/4267059419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f480ce1b0 0x7f8f48157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.741+0000 7f8f4eca1640 1 --2- 172.21.15.115:0/4267059419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f480b0e60 0x7f8f48153310 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.741+0000 7f8f4eca1640 1 -- 172.21.15.115:0/4267059419 >> 172.21.15.115:0/4267059419 conn(0x7f8f48065960 msgr2=0x7f8f4815d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.741+0000 7f8f4eca1640 1 -- 172.21.15.115:0/4267059419 shutdown_connections 2023-12-01T03:12:09.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.741+0000 7f8f4eca1640 1 -- 172.21.15.115:0/4267059419 wait complete. 2023-12-01T03:12:09.760 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:12:09.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.965+0000 7f5d1b6db640 1 Processor -- start 2023-12-01T03:12:09.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.965+0000 7f5d1b6db640 1 -- start start 2023-12-01T03:12:09.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.966+0000 7f5d1b6db640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5d1415b0e0 0x7f5d14164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d1b6db640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d140b2f30 con 0x7f5d140cd1e0 2023-12-01T03:12:09.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d1b6db640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d140b2320 con 0x7f5d14142870 2023-12-01T03:12:09.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d1b6db640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d140b3ae0 con 0x7f5d1415b0e0 2023-12-01T03:12:09.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d18c76640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5d140cd1e0 0x7f5d140b1710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59956/0 (socket says 172.21.15.115:59956) 2023-12-01T03:12:09.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d18c76640 1 -- 172.21.15.115:0/3406520175 learned_addr learned my addr 172.21.15.115:0/3406520175 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:09.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d19c78640 1 --2- 172.21.15.115:0/3406520175 >> v2:172.21.15.115:3300/0 conn(0x7f5d1415b0e0 0x7f5d14164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d0bfff640 1 -- 172.21.15.115:0/3406520175 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648864666 0 0) 0x7f5d140b2f30 con 0x7f5d140cd1e0 2023-12-01T03:12:09.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d0bfff640 1 -- 172.21.15.115:0/3406520175 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d00003520 con 0x7f5d140cd1e0 2023-12-01T03:12:09.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d19c78640 1 -- 172.21.15.115:0/3406520175 >> v1:172.21.15.142:6789/0 conn(0x7f5d14142870 legacy=0x7f5d140ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:09.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d19c78640 1 -- 172.21.15.115:0/3406520175 >> v1:172.21.15.115:6789/0 conn(0x7f5d140cd1e0 legacy=0x7f5d140b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.967+0000 7f5d19c78640 1 -- 172.21.15.115:0/3406520175 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d140b2600 con 0x7f5d1415b0e0 2023-12-01T03:12:09.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.969+0000 7f5d19c78640 1 --2- 172.21.15.115:0/3406520175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1415b0e0 0x7f5d14164a20 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009d90 tx=0x7f5d0c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1af921014811baa server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:09.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.969+0000 7f5d0bfff640 1 -- 172.21.15.115:0/3406520175 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c02cc60 con 0x7f5d1415b0e0 2023-12-01T03:12:09.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.969+0000 7f5d0bfff640 1 -- 172.21.15.115:0/3406520175 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c02cdc0 con 0x7f5d1415b0e0 2023-12-01T03:12:09.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.969+0000 7f5d0bfff640 1 -- 172.21.15.115:0/3406520175 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c034660 con 0x7f5d1415b0e0 2023-12-01T03:12:09.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.969+0000 7f5d1b6db640 1 -- 172.21.15.115:0/3406520175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1415b0e0 msgr2=0x7f5d14164a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.969+0000 7f5d1b6db640 1 --2- 172.21.15.115:0/3406520175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1415b0e0 0x7f5d14164a20 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009d90 tx=0x7f5d0c02beb0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.970+0000 7f5d1b6db640 1 -- 172.21.15.115:0/3406520175 shutdown_connections 2023-12-01T03:12:09.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.970+0000 7f5d1b6db640 1 --2- 172.21.15.115:0/3406520175 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1415b0e0 0x7f5d14164a20 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.970+0000 7f5d1b6db640 1 -- 172.21.15.115:0/3406520175 >> 172.21.15.115:0/3406520175 conn(0x7f5d14065960 msgr2=0x7f5d14065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:09.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.970+0000 7f5d1b6db640 1 -- 172.21.15.115:0/3406520175 shutdown_connections 2023-12-01T03:12:09.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.970+0000 7f5d1b6db640 1 -- 172.21.15.115:0/3406520175 wait complete. 2023-12-01T03:12:09.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.972+0000 7f5d1b6db640 1 Processor -- start 2023-12-01T03:12:09.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.972+0000 7f5d1b6db640 1 -- start start 2023-12-01T03:12:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.973+0000 7f5d1b6db640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d140cd1e0 0x7f5d140c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.973+0000 7f5d1b6db640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d14142870 0x7f5d141f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.973+0000 7f5d1b6db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1413ed30 0x7f5d141f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.973+0000 7f5d1b6db640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1413f2d0 con 0x7f5d1413ed30 2023-12-01T03:12:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.973+0000 7f5d1b6db640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1413f5b0 con 0x7f5d140cd1e0 2023-12-01T03:12:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.973+0000 7f5d1b6db640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1413f890 con 0x7f5d14142870 2023-12-01T03:12:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19c78640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1413ed30 0x7f5d141f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19c78640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1413ed30 0x7f5d141f3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52688/0 (socket says 172.21.15.115:52688) 2023-12-01T03:12:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19c78640 1 -- 172.21.15.115:0/178450383 learned_addr learned my addr 172.21.15.115:0/178450383 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:09.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d18c76640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d14142870 0x7f5d141f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19477640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d140cd1e0 0x7f5d140c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:09.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19c78640 1 -- 172.21.15.115:0/178450383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d14142870 msgr2=0x7f5d141f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19c78640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d14142870 0x7f5d141f2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19c78640 1 -- 172.21.15.115:0/178450383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d140cd1e0 msgr2=0x7f5d140c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:09.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19c78640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d140cd1e0 0x7f5d140c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:09.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19c78640 1 -- 172.21.15.115:0/178450383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c009a70 con 0x7f5d1413ed30 2023-12-01T03:12:09.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19477640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d140cd1e0 0x7f5d140c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:09.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.974+0000 7f5d19c78640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1413ed30 0x7f5d141f3270 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009d90 tx=0x7f5d0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:09.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.975+0000 7f5d09ffb640 1 -- 172.21.15.115:0/178450383 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c040650 con 0x7f5d1413ed30 2023-12-01T03:12:09.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.975+0000 7f5d09ffb640 1 -- 172.21.15.115:0/178450383 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c0407b0 con 0x7f5d1413ed30 2023-12-01T03:12:09.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.975+0000 7f5d09ffb640 1 -- 172.21.15.115:0/178450383 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c040b60 con 0x7f5d1413ed30 2023-12-01T03:12:09.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.975+0000 7f5d1b6db640 1 -- 172.21.15.115:0/178450383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1413fb70 con 0x7f5d1413ed30 2023-12-01T03:12:09.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.975+0000 7f5d1b6db640 1 -- 172.21.15.115:0/178450383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d140c5110 con 0x7f5d1413ed30 2023-12-01T03:12:09.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.976+0000 7f5d1b6db640 1 -- 172.21.15.115:0/178450383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d14140090 con 0x7f5d1413ed30 2023-12-01T03:12:09.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.977+0000 7f5d09ffb640 1 -- 172.21.15.115:0/178450383 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 139) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f5d0c040cc0 con 0x7f5d1413ed30 2023-12-01T03:12:09.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.978+0000 7f5d09ffb640 1 -- 172.21.15.115:0/178450383 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7f5d0c0ab810 con 0x7f5d1413ed30 2023-12-01T03:12:09.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:09.982+0000 7f5d09ffb640 1 -- 172.21.15.115:0/178450383 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5d14140090 con 0x7f5d1413ed30 2023-12-01T03:12:10.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.035+0000 7f5d09ffb640 1 -- 172.21.15.115:0/178450383 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 140) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f5d0c076730 con 0x7f5d1413ed30 2023-12-01T03:12:10.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.036+0000 7f5d09ffb640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5cf8072360 0x7f5cf8074750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.037+0000 7f5d19477640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5cf8072360 0x7f5cf8074750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.041+0000 7f5d19477640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5cf8072360 0x7f5cf8074750 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5d10003eb0 tx=0x7f5d10009040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:10.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.171+0000 7f5d1b6db640 1 -- 172.21.15.115:0/178450383 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d14140090 con 0x7f5d1413ed30 2023-12-01T03:12:10.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.172+0000 7f5d09ffb640 1 -- 172.21.15.115:0/178450383 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v241) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d14140090 con 0x7f5d1413ed30 2023-12-01T03:12:10.173 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:12:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 -- 172.21.15.115:0/178450383 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5cf8072360 msgr2=0x7f5cf8074750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5cf8072360 0x7f5cf8074750 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5d10003eb0 tx=0x7f5d10009040 comp rx=0 tx=0).stop 2023-12-01T03:12:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 -- 172.21.15.115:0/178450383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1413ed30 msgr2=0x7f5d141f3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1413ed30 0x7f5d141f3270 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009d90 tx=0x7f5d0c004a40 comp rx=0 tx=0).stop 2023-12-01T03:12:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 -- 172.21.15.115:0/178450383 shutdown_connections 2023-12-01T03:12:10.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5cf8072360 0x7f5cf8074750 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d14142870 0x7f5d141f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d1413ed30 0x7f5d141f3270 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 --2- 172.21.15.115:0/178450383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d140cd1e0 0x7f5d140c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 -- 172.21.15.115:0/178450383 >> 172.21.15.115:0/178450383 conn(0x7f5d14065960 msgr2=0x7f5d1406baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:10.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 -- 172.21.15.115:0/178450383 shutdown_connections 2023-12-01T03:12:10.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.174+0000 7f5d1b6db640 1 -- 172.21.15.115:0/178450383 wait complete. 2023-12-01T03:12:10.194 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:12:10.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.402+0000 7f834b163640 1 Processor -- start 2023-12-01T03:12:10.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.402+0000 7f834b163640 1 -- start start 2023-12-01T03:12:10.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.404+0000 7f834b163640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f83440cd1e0 0x7f83440b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.404+0000 7f834b163640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83440b2f30 con 0x7f8344142870 2023-12-01T03:12:10.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.404+0000 7f834b163640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83440b2080 con 0x7f83440b1c50 2023-12-01T03:12:10.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.404+0000 7f834b163640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83440b3ae0 con 0x7f83440cd1e0 2023-12-01T03:12:10.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.404+0000 7f8343fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f83440cd1e0 0x7f83440b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.406 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.404+0000 7f8343fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f83440cd1e0 0x7f83440b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52702/0 (socket says 172.21.15.115:52702) 2023-12-01T03:12:10.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.404+0000 7f8343fff640 1 -- 172.21.15.115:0/693156796 learned_addr learned my addr 172.21.15.115:0/693156796 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:10.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.404+0000 7f8348eff640 1 --1- 172.21.15.115:0/693156796 >> v1:172.21.15.115:6789/0 conn(0x7f8344142870 0x7f83440ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59968/0 (socket says 172.21.15.115:59968) 2023-12-01T03:12:10.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.405+0000 7f8343fff640 1 -- 172.21.15.115:0/693156796 >> v1:172.21.15.142:6789/0 conn(0x7f83440b1c50 legacy=0x7f834415a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:10.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.405+0000 7f8343fff640 1 -- 172.21.15.115:0/693156796 >> v1:172.21.15.115:6789/0 conn(0x7f8344142870 legacy=0x7f83440ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.405+0000 7f8343fff640 1 -- 172.21.15.115:0/693156796 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83440b2320 con 0x7f83440cd1e0 2023-12-01T03:12:10.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.406+0000 7f8343fff640 1 --2- 172.21.15.115:0/693156796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440cd1e0 0x7f83440b1710 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f833000bb10 tx=0x7f833002e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf729cd63d90df3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:10.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.407+0000 7f83437fe640 1 -- 172.21.15.115:0/693156796 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330036440 con 0x7f83440cd1e0 2023-12-01T03:12:10.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.407+0000 7f83437fe640 1 -- 172.21.15.115:0/693156796 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83300365a0 con 0x7f83440cd1e0 2023-12-01T03:12:10.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.407+0000 7f83437fe640 1 -- 172.21.15.115:0/693156796 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330036930 con 0x7f83440cd1e0 2023-12-01T03:12:10.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.408+0000 7f834b163640 1 -- 172.21.15.115:0/693156796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440cd1e0 msgr2=0x7f83440b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.408+0000 7f834b163640 1 --2- 172.21.15.115:0/693156796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440cd1e0 0x7f83440b1710 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f833000bb10 tx=0x7f833002e160 comp rx=0 tx=0).stop 2023-12-01T03:12:10.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.408+0000 7f834b163640 1 -- 172.21.15.115:0/693156796 shutdown_connections 2023-12-01T03:12:10.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.408+0000 7f834b163640 1 --2- 172.21.15.115:0/693156796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440cd1e0 0x7f83440b1710 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.408+0000 7f834b163640 1 -- 172.21.15.115:0/693156796 >> 172.21.15.115:0/693156796 conn(0x7f8344065960 msgr2=0x7f8344065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:10.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.408+0000 7f834b163640 1 -- 172.21.15.115:0/693156796 shutdown_connections 2023-12-01T03:12:10.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.408+0000 7f834b163640 1 -- 172.21.15.115:0/693156796 wait complete. 2023-12-01T03:12:10.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.410+0000 7f834b163640 1 Processor -- start 2023-12-01T03:12:10.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.410+0000 7f834b163640 1 -- start start 2023-12-01T03:12:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f834b163640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440b1c50 0x7f8344158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f834b163640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83440cd1e0 0x7f83440a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f834b163640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8344142870 0x7f83441e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f834b163640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83440aa230 con 0x7f83440b1c50 2023-12-01T03:12:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f834b163640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83440aa510 con 0x7f83440cd1e0 2023-12-01T03:12:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f834b163640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83440aa7f0 con 0x7f8344142870 2023-12-01T03:12:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f8348eff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440b1c50 0x7f8344158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f8348eff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440b1c50 0x7f8344158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52706/0 (socket says 172.21.15.115:52706) 2023-12-01T03:12:10.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f8348eff640 1 -- 172.21.15.115:0/1373584208 learned_addr learned my addr 172.21.15.115:0/1373584208 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f8343fff640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83440cd1e0 0x7f83440a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f8348eff640 1 -- 172.21.15.115:0/1373584208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8344142870 msgr2=0x7f83441e41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f8348eff640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8344142870 0x7f83441e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f8348eff640 1 -- 172.21.15.115:0/1373584208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83440cd1e0 msgr2=0x7f83440a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f8348eff640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83440cd1e0 0x7f83440a97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.412+0000 7f8348eff640 1 -- 172.21.15.115:0/1373584208 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833000b770 con 0x7f83440b1c50 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.413+0000 7f8348eff640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440b1c50 0x7f8344158bc0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f833800c3c0 tx=0x7f833800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.413+0000 7f83417fa640 1 -- 172.21.15.115:0/1373584208 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83380074e0 con 0x7f83440b1c50 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.413+0000 7f83417fa640 1 -- 172.21.15.115:0/1373584208 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8338007640 con 0x7f83440b1c50 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.413+0000 7f83417fa640 1 -- 172.21.15.115:0/1373584208 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338004a00 con 0x7f83440b1c50 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.413+0000 7f834b163640 1 -- 172.21.15.115:0/1373584208 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83440aaad0 con 0x7f83440b1c50 2023-12-01T03:12:10.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.413+0000 7f834b163640 1 -- 172.21.15.115:0/1373584208 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83440abc50 con 0x7f83440b1c50 2023-12-01T03:12:10.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.414+0000 7f834b163640 1 -- 172.21.15.115:0/1373584208 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83440aaff0 con 0x7f83440b1c50 2023-12-01T03:12:10.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.415+0000 7f83417fa640 1 -- 172.21.15.115:0/1373584208 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 140) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f8338004b60 con 0x7f83440b1c50 2023-12-01T03:12:10.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.416+0000 7f83417fa640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f832806a280 0x7f832806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.416+0000 7f83417fa640 1 -- 172.21.15.115:0/1373584208 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7f83380885a0 con 0x7f83440b1c50 2023-12-01T03:12:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.416+0000 7f8343fff640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f832806a280 0x7f832806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.417+0000 7f8343fff640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f832806a280 0x7f832806c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8330038000 tx=0x7f8330032000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:10.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.420+0000 7f83417fa640 1 -- 172.21.15.115:0/1373584208 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f83440aaff0 con 0x7f83440b1c50 2023-12-01T03:12:10.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.598+0000 7f834b163640 1 -- 172.21.15.115:0/1373584208 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83440aaff0 con 0x7f83440b1c50 2023-12-01T03:12:10.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.599+0000 7f83417fa640 1 -- 172.21.15.115:0/1373584208 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v241) v1 ==== 100+0+11 (secure 0 0 0) 0x7f83440aaff0 con 0x7f83440b1c50 2023-12-01T03:12:10.600 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:12:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 -- 172.21.15.115:0/1373584208 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f832806a280 msgr2=0x7f832806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f832806a280 0x7f832806c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8330038000 tx=0x7f8330032000 comp rx=0 tx=0).stop 2023-12-01T03:12:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 -- 172.21.15.115:0/1373584208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440b1c50 msgr2=0x7f8344158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440b1c50 0x7f8344158bc0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f833800c3c0 tx=0x7f833800c880 comp rx=0 tx=0).stop 2023-12-01T03:12:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 -- 172.21.15.115:0/1373584208 shutdown_connections 2023-12-01T03:12:10.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f832806a280 0x7f832806c740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8344142870 0x7f83441e41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83440cd1e0 0x7f83440a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 --2- 172.21.15.115:0/1373584208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83440b1c50 0x7f8344158bc0 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 -- 172.21.15.115:0/1373584208 >> 172.21.15.115:0/1373584208 conn(0x7f8344065960 msgr2=0x7f8344065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 -- 172.21.15.115:0/1373584208 shutdown_connections 2023-12-01T03:12:10.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.601+0000 7f834b163640 1 -- 172.21.15.115:0/1373584208 wait complete. 2023-12-01T03:12:10.621 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-01T03:12:10.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.828+0000 7f8cdd3a9640 1 Processor -- start 2023-12-01T03:12:10.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.828+0000 7f8cdd3a9640 1 -- start start 2023-12-01T03:12:10.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.830+0000 7f8cdd3a9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8cd8160200 0x7f8cd80b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.830+0000 7f8cdd3a9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd80b0f80 0x7f8cd806b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.830+0000 7f8cdd3a9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd80cb290 con 0x7f8cd806b7a0 2023-12-01T03:12:10.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.830+0000 7f8cdd3a9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd80b35f0 con 0x7f8cd8160200 2023-12-01T03:12:10.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.830+0000 7f8cdd3a9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80b0ae0 con 0x7f8cd80b0f80 2023-12-01T03:12:10.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.830+0000 7f8cd6ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8cd8160200 0x7f8cd80b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.830+0000 7f8cd77fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8cd806b7a0 0x7f8cd80cbd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59982/0 (socket says 172.21.15.115:59982) 2023-12-01T03:12:10.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.830+0000 7f8cd77fe640 1 -- 172.21.15.115:0/488943901 learned_addr learned my addr 172.21.15.115:0/488943901 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:10.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.830+0000 7f8cd67fc640 1 --2- 172.21.15.115:0/488943901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd80b0f80 0x7f8cd806b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.831+0000 7f8cd5ffb640 1 -- 172.21.15.115:0/488943901 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962882509 0 0) 0x7f8cd80cb290 con 0x7f8cd806b7a0 2023-12-01T03:12:10.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.831+0000 7f8cd5ffb640 1 -- 172.21.15.115:0/488943901 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cc0003520 con 0x7f8cd806b7a0 2023-12-01T03:12:10.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.831+0000 7f8cd6ffd640 1 -- 172.21.15.115:0/488943901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd80b0f80 msgr2=0x7f8cd806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:10.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.831+0000 7f8cd6ffd640 1 --2- 172.21.15.115:0/488943901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd80b0f80 0x7f8cd806b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.831+0000 7f8cd6ffd640 1 -- 172.21.15.115:0/488943901 >> v1:172.21.15.115:6789/0 conn(0x7f8cd806b7a0 legacy=0x7f8cd80cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.831+0000 7f8cd6ffd640 1 -- 172.21.15.115:0/488943901 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80b3310 con 0x7f8cd8160200 2023-12-01T03:12:10.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.832+0000 7f8cd6ffd640 1 --2- 172.21.15.115:0/488943901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd8160200 0x7f8cd80b2c80 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80097a0 tx=0x7f8cc802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3722d45ec32d85 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:10.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.832+0000 7f8cd5ffb640 1 -- 172.21.15.115:0/488943901 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8031830 con 0x7f8cd8160200 2023-12-01T03:12:10.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.832+0000 7f8cd5ffb640 1 -- 172.21.15.115:0/488943901 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc8031990 con 0x7f8cd8160200 2023-12-01T03:12:10.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.832+0000 7f8cd5ffb640 1 -- 172.21.15.115:0/488943901 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8031d20 con 0x7f8cd8160200 2023-12-01T03:12:10.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.833+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/488943901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd8160200 msgr2=0x7f8cd80b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.833+0000 7f8cdd3a9640 1 --2- 172.21.15.115:0/488943901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd8160200 0x7f8cd80b2c80 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80097a0 tx=0x7f8cc802b8d0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.833+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/488943901 shutdown_connections 2023-12-01T03:12:10.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.833+0000 7f8cdd3a9640 1 --2- 172.21.15.115:0/488943901 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd80b0f80 0x7f8cd806b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.833+0000 7f8cdd3a9640 1 --2- 172.21.15.115:0/488943901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd8160200 0x7f8cd80b2c80 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.833+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/488943901 >> 172.21.15.115:0/488943901 conn(0x7f8cd8065960 msgr2=0x7f8cd8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:10.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.833+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/488943901 shutdown_connections 2023-12-01T03:12:10.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.833+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/488943901 wait complete. 2023-12-01T03:12:10.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.835+0000 7f8cdd3a9640 1 Processor -- start 2023-12-01T03:12:10.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.835+0000 7f8cdd3a9640 1 -- start start 2023-12-01T03:12:10.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.836+0000 7f8cdd3a9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd806b7a0 0x7f8cd806bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.836+0000 7f8cdd3a9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd80b0f80 0x7f8cd806c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.836+0000 7f8cdd3a9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd8160200 0x7f8cd81f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.836+0000 7f8cdd3a9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd8150c90 con 0x7f8cd80b0f80 2023-12-01T03:12:10.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.836+0000 7f8cdd3a9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd8151810 con 0x7f8cd8160200 2023-12-01T03:12:10.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.836+0000 7f8cdd3a9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd8150ee0 con 0x7f8cd806b7a0 2023-12-01T03:12:10.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.836+0000 7f8cd67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd80b0f80 0x7f8cd806c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.836+0000 7f8cd67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd80b0f80 0x7f8cd806c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52734/0 (socket says 172.21.15.115:52734) 2023-12-01T03:12:10.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.836+0000 7f8cd67fc640 1 -- 172.21.15.115:0/2302907396 learned_addr learned my addr 172.21.15.115:0/2302907396 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:10.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cd6ffd640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd806b7a0 0x7f8cd806bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cd67fc640 1 -- 172.21.15.115:0/2302907396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd806b7a0 msgr2=0x7f8cd806bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cd77fe640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd8160200 0x7f8cd81f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cd67fc640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd806b7a0 0x7f8cd806bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cd67fc640 1 -- 172.21.15.115:0/2302907396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd8160200 msgr2=0x7f8cd81f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cd67fc640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd8160200 0x7f8cd81f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cd67fc640 1 -- 172.21.15.115:0/2302907396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc8009400 con 0x7f8cd80b0f80 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cd67fc640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd80b0f80 0x7f8cd806c180 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00d800 tx=0x7f8ccc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cbf7fe640 1 -- 172.21.15.115:0/2302907396 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00b6c0 con 0x7f8cd80b0f80 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cbf7fe640 1 -- 172.21.15.115:0/2302907396 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc00b820 con 0x7f8cd80b0f80 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cbf7fe640 1 -- 172.21.15.115:0/2302907396 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00bbb0 con 0x7f8cd80b0f80 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/2302907396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd81511c0 con 0x7f8cd80b0f80 2023-12-01T03:12:10.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.837+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/2302907396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd81f3620 con 0x7f8cd80b0f80 2023-12-01T03:12:10.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.838+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/2302907396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd804e890 con 0x7f8cd80b0f80 2023-12-01T03:12:10.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.839+0000 7f8cbf7fe640 1 -- 172.21.15.115:0/2302907396 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 140) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f8ccc00bd10 con 0x7f8cd80b0f80 2023-12-01T03:12:10.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.840+0000 7f8cbf7fe640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8cc406a280 0x7f8cc406c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:10.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.840+0000 7f8cbf7fe640 1 -- 172.21.15.115:0/2302907396 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7f8ccc0891e0 con 0x7f8cd80b0f80 2023-12-01T03:12:10.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.840+0000 7f8cd6ffd640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8cc406a280 0x7f8cc406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:10.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.841+0000 7f8cd6ffd640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8cc406a280 0x7f8cc406c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8036000 tx=0x7f8cc803c040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:10.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:10.844+0000 7f8cbf7fe640 1 -- 172.21.15.115:0/2302907396 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8ccc0543b0 con 0x7f8cd80b0f80 2023-12-01T03:12:11.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.038+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/2302907396 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8cd80b1e20 con 0x7f8cd80b0f80 2023-12-01T03:12:11.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.038+0000 7f8cbf7fe640 1 -- 172.21.15.115:0/2302907396 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v241) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8ccc0579d0 con 0x7f8cd80b0f80 2023-12-01T03:12:11.040 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:12:11.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.041+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/2302907396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8cc406a280 msgr2=0x7f8cc406c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.041+0000 7f8cdd3a9640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8cc406a280 0x7f8cc406c740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8036000 tx=0x7f8cc803c040 comp rx=0 tx=0).stop 2023-12-01T03:12:11.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.041+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/2302907396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd80b0f80 msgr2=0x7f8cd806c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.041+0000 7f8cdd3a9640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd80b0f80 0x7f8cd806c180 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00d800 tx=0x7f8ccc00dcc0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.042+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/2302907396 shutdown_connections 2023-12-01T03:12:11.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.042+0000 7f8cdd3a9640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8cc406a280 0x7f8cc406c740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.042+0000 7f8cdd3a9640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd8160200 0x7f8cd81f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.042+0000 7f8cdd3a9640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8cd80b0f80 0x7f8cd806c180 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.042+0000 7f8cdd3a9640 1 --2- 172.21.15.115:0/2302907396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd806b7a0 0x7f8cd806bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.042+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/2302907396 >> 172.21.15.115:0/2302907396 conn(0x7f8cd8065960 msgr2=0x7f8cd8156560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:11.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.042+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/2302907396 shutdown_connections 2023-12-01T03:12:11.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.042+0000 7f8cdd3a9640 1 -- 172.21.15.115:0/2302907396 wait complete. 2023-12-01T03:12:11.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-01T03:12:11.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.252+0000 7fbf4c518640 1 Processor -- start 2023-12-01T03:12:11.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.252+0000 7fbf4c518640 1 -- start start 2023-12-01T03:12:11.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.254+0000 7fbf4c518640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf440ce1b0 0x7fbf440cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.254+0000 7fbf4c518640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbf440b0e60 0x7fbf440b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.254+0000 7fbf4c518640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf440cc9a0 con 0x7fbf4415ac80 2023-12-01T03:12:11.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.254+0000 7fbf4c518640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4415b060 con 0x7fbf440b0e60 2023-12-01T03:12:11.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.254+0000 7fbf4c518640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf440b0ce0 con 0x7fbf440ce1b0 2023-12-01T03:12:11.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.254+0000 7fbf4aab5640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbf440b0e60 0x7fbf440b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:11.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.254+0000 7fbf4a2b4640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fbf4415ac80 0x7fbf440cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59992/0 (socket says 172.21.15.115:59992) 2023-12-01T03:12:11.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.254+0000 7fbf4a2b4640 1 -- 172.21.15.115:0/461768803 learned_addr learned my addr 172.21.15.115:0/461768803 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:11.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.254+0000 7fbf49ab3640 1 --2- 172.21.15.115:0/461768803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf440ce1b0 0x7fbf440cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:11.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.255+0000 7fbf4aab5640 1 -- 172.21.15.115:0/461768803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf440ce1b0 msgr2=0x7fbf440cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:11.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.255+0000 7fbf4aab5640 1 --2- 172.21.15.115:0/461768803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf440ce1b0 0x7fbf440cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.255+0000 7fbf4aab5640 1 -- 172.21.15.115:0/461768803 >> v1:172.21.15.115:6789/0 conn(0x7fbf4415ac80 legacy=0x7fbf440cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.255+0000 7fbf4aab5640 1 -- 172.21.15.115:0/461768803 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf440cbd90 con 0x7fbf440b0e60 2023-12-01T03:12:11.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.255+0000 7fbf492b2640 1 -- 172.21.15.115:0/461768803 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443323071 0 0) 0x7fbf440cc9a0 con 0x7fbf4415ac80 2023-12-01T03:12:11.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.256+0000 7fbf4aab5640 1 --2- 172.21.15.115:0/461768803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 0x7fbf440b07a0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009d90 tx=0x7fbf3c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f24f0ade7dc609 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:11.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.256+0000 7fbf492b2640 1 -- 172.21.15.115:0/461768803 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c02cc50 con 0x7fbf440b0e60 2023-12-01T03:12:11.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.256+0000 7fbf492b2640 1 -- 172.21.15.115:0/461768803 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c02cdb0 con 0x7fbf440b0e60 2023-12-01T03:12:11.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.256+0000 7fbf492b2640 1 -- 172.21.15.115:0/461768803 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c034660 con 0x7fbf440b0e60 2023-12-01T03:12:11.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.257+0000 7fbf4c518640 1 -- 172.21.15.115:0/461768803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 msgr2=0x7fbf440b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.257+0000 7fbf4c518640 1 --2- 172.21.15.115:0/461768803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 0x7fbf440b07a0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009d90 tx=0x7fbf3c02bea0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.257+0000 7fbf4c518640 1 -- 172.21.15.115:0/461768803 shutdown_connections 2023-12-01T03:12:11.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.257+0000 7fbf4c518640 1 --2- 172.21.15.115:0/461768803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 0x7fbf440b07a0 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.257+0000 7fbf4c518640 1 --2- 172.21.15.115:0/461768803 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf440ce1b0 0x7fbf440cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.257+0000 7fbf4c518640 1 -- 172.21.15.115:0/461768803 >> 172.21.15.115:0/461768803 conn(0x7fbf44065960 msgr2=0x7fbf44065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:11.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.257+0000 7fbf4c518640 1 -- 172.21.15.115:0/461768803 shutdown_connections 2023-12-01T03:12:11.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.257+0000 7fbf4c518640 1 -- 172.21.15.115:0/461768803 wait complete. 2023-12-01T03:12:11.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.259+0000 7fbf4c518640 1 Processor -- start 2023-12-01T03:12:11.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.259+0000 7fbf4c518640 1 -- start start 2023-12-01T03:12:11.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.260+0000 7fbf4c518640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 0x7fbf4415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.260+0000 7fbf4c518640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf440ce1b0 0x7fbf44147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.260+0000 7fbf4c518640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf4415ac80 0x7fbf4406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.260+0000 7fbf4c518640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4415d840 con 0x7fbf440b0e60 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.260+0000 7fbf4c518640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf44147660 con 0x7fbf440ce1b0 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.260+0000 7fbf4c518640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4415cea0 con 0x7fbf4415ac80 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.260+0000 7fbf4a2b4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 0x7fbf4415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.260+0000 7fbf4a2b4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 0x7fbf4415dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52770/0 (socket says 172.21.15.115:52770) 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.260+0000 7fbf4a2b4640 1 -- 172.21.15.115:0/1792483025 learned_addr learned my addr 172.21.15.115:0/1792483025 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf4a2b4640 1 -- 172.21.15.115:0/1792483025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf4415ac80 msgr2=0x7fbf4406faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf4a2b4640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf4415ac80 0x7fbf4406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf4a2b4640 1 -- 172.21.15.115:0/1792483025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf440ce1b0 msgr2=0x7fbf44147120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf4a2b4640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf440ce1b0 0x7fbf44147120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf4a2b4640 1 -- 172.21.15.115:0/1792483025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c009a70 con 0x7fbf440b0e60 2023-12-01T03:12:11.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf4a2b4640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 0x7fbf4415dc60 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000c430 tx=0x7fbf4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:11.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf3affd640 1 -- 172.21.15.115:0/1792483025 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf400074e0 con 0x7fbf440b0e60 2023-12-01T03:12:11.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf3affd640 1 -- 172.21.15.115:0/1792483025 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf40007660 con 0x7fbf440b0e60 2023-12-01T03:12:11.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf3affd640 1 -- 172.21.15.115:0/1792483025 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40004a20 con 0x7fbf440b0e60 2023-12-01T03:12:11.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf4c518640 1 -- 172.21.15.115:0/1792483025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4415d180 con 0x7fbf440b0e60 2023-12-01T03:12:11.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.261+0000 7fbf4c518640 1 -- 172.21.15.115:0/1792483025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf44150d00 con 0x7fbf440b0e60 2023-12-01T03:12:11.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.262+0000 7fbf4c518640 1 -- 172.21.15.115:0/1792483025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4404e890 con 0x7fbf440b0e60 2023-12-01T03:12:11.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.263+0000 7fbf3affd640 1 -- 172.21.15.115:0/1792483025 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 140) v1 ==== 88445+0+0 (secure 0 0 0) 0x7fbf40004b80 con 0x7fbf440b0e60 2023-12-01T03:12:11.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.264+0000 7fbf3affd640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2806a2b0 0x7fbf2806c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.264+0000 7fbf3affd640 1 -- 172.21.15.115:0/1792483025 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7fbf40088ae0 con 0x7fbf440b0e60 2023-12-01T03:12:11.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.265+0000 7fbf49ab3640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2806a2b0 0x7fbf2806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:11.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.265+0000 7fbf49ab3640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2806a2b0 0x7fbf2806c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbf4415a550 tx=0x7fbf34008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:11.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.268+0000 7fbf3affd640 1 -- 172.21.15.115:0/1792483025 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbf40053cb0 con 0x7fbf440b0e60 2023-12-01T03:12:11.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.456+0000 7fbf4c518640 1 -- 172.21.15.115:0/1792483025 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf440c3cf0 con 0x7fbf440b0e60 2023-12-01T03:12:11.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.457+0000 7fbf3affd640 1 -- 172.21.15.115:0/1792483025 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v241) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbf400572d0 con 0x7fbf440b0e60 2023-12-01T03:12:11.459 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:12:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 -- 172.21.15.115:0/1792483025 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2806a2b0 msgr2=0x7fbf2806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2806a2b0 0x7fbf2806c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbf4415a550 tx=0x7fbf34008040 comp rx=0 tx=0).stop 2023-12-01T03:12:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 -- 172.21.15.115:0/1792483025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 msgr2=0x7fbf4415dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 0x7fbf4415dc60 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000c430 tx=0x7fbf4000c8f0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 -- 172.21.15.115:0/1792483025 shutdown_connections 2023-12-01T03:12:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2806a2b0 0x7fbf2806c770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf4415ac80 0x7fbf4406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf440ce1b0 0x7fbf44147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 --2- 172.21.15.115:0/1792483025 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440b0e60 0x7fbf4415dc60 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 -- 172.21.15.115:0/1792483025 >> 172.21.15.115:0/1792483025 conn(0x7fbf44065960 msgr2=0x7fbf440c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.459+0000 7fbf4c518640 1 -- 172.21.15.115:0/1792483025 shutdown_connections 2023-12-01T03:12:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.460+0000 7fbf4c518640 1 -- 172.21.15.115:0/1792483025 wait complete. 2023-12-01T03:12:11.481 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:12:11.690 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.688+0000 7f4f6ffff640 1 Processor -- start 2023-12-01T03:12:11.691 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.688+0000 7f4f6ffff640 1 -- start start 2023-12-01T03:12:11.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.690+0000 7f4f6ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7015c5d0 0x7f4f7006f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.690+0000 7f4f6ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4f700ce830 0x7f4f7006ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.690+0000 7f4f6ffff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7006cbe0 con 0x7f4f7015cb90 2023-12-01T03:12:11.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.690+0000 7f4f6ffff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f7015c9b0 con 0x7f4f700ce830 2023-12-01T03:12:11.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.690+0000 7f4f6ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7006c340 con 0x7f4f7015c5d0 2023-12-01T03:12:11.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.690+0000 7f4f67fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4f700ce830 0x7f4f7006ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:11.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.690+0000 7f4f67fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4f700ce830 0x7f4f7006ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52786/0 (socket says 172.21.15.115:52786) 2023-12-01T03:12:11.693 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.690+0000 7f4f67fff640 1 -- 172.21.15.115:0/1837029753 learned_addr learned my addr 172.21.15.115:0/1837029753 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:11.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.691+0000 7f4f67fff640 1 -- 172.21.15.115:0/1837029753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7015c5d0 msgr2=0x7f4f7006f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:11.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.691+0000 7f4f6effd640 1 --2- 172.21.15.115:0/1837029753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7015c5d0 0x7f4f7006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:11.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.691+0000 7f4f67fff640 1 --2- 172.21.15.115:0/1837029753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7015c5d0 0x7f4f7006f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.691+0000 7f4f67fff640 1 -- 172.21.15.115:0/1837029753 >> v1:172.21.15.142:6789/0 conn(0x7f4f7015cb90 legacy=0x7f4f700cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:11.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.691+0000 7f4f67fff640 1 -- 172.21.15.115:0/1837029753 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7006c620 con 0x7f4f700ce830 2023-12-01T03:12:11.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.692+0000 7f4f67fff640 1 --2- 172.21.15.115:0/1837029753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f700ce830 0x7f4f7006ba60 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009870 tx=0x7f4f5802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3270504a8d8e0c12 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:11.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.692+0000 7f4f6e7fc640 1 -- 172.21.15.115:0/1837029753 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58031640 con 0x7f4f700ce830 2023-12-01T03:12:11.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.692+0000 7f4f6e7fc640 1 -- 172.21.15.115:0/1837029753 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f580317a0 con 0x7f4f700ce830 2023-12-01T03:12:11.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.692+0000 7f4f6e7fc640 1 -- 172.21.15.115:0/1837029753 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58031b30 con 0x7f4f700ce830 2023-12-01T03:12:11.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.693+0000 7f4f6ffff640 1 -- 172.21.15.115:0/1837029753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f700ce830 msgr2=0x7f4f7006ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.693+0000 7f4f6ffff640 1 --2- 172.21.15.115:0/1837029753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f700ce830 0x7f4f7006ba60 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009870 tx=0x7f4f5802b8f0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.695 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.693+0000 7f4f6ffff640 1 -- 172.21.15.115:0/1837029753 shutdown_connections 2023-12-01T03:12:11.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.693+0000 7f4f6ffff640 1 --2- 172.21.15.115:0/1837029753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f700ce830 0x7f4f7006ba60 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.693+0000 7f4f6ffff640 1 --2- 172.21.15.115:0/1837029753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f7015c5d0 0x7f4f7006f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.693+0000 7f4f6ffff640 1 -- 172.21.15.115:0/1837029753 >> 172.21.15.115:0/1837029753 conn(0x7f4f700ceed0 msgr2=0x7f4f700cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:11.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.693+0000 7f4f6ffff640 1 -- 172.21.15.115:0/1837029753 shutdown_connections 2023-12-01T03:12:11.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.693+0000 7f4f6ffff640 1 -- 172.21.15.115:0/1837029753 wait complete. 2023-12-01T03:12:11.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.695+0000 7f4f6ffff640 1 Processor -- start 2023-12-01T03:12:11.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.695+0000 7f4f6ffff640 1 -- start start 2023-12-01T03:12:11.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.696+0000 7f4f6ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f700ce830 0x7f4f70150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.696+0000 7f4f6ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f7015c5d0 0x7f4f700c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.696+0000 7f4f6ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7015cb90 0x7f4f701e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.696+0000 7f4f6ffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f700aff00 con 0x7f4f7015cb90 2023-12-01T03:12:11.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.696+0000 7f4f6ffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7015aab0 con 0x7f4f7015c5d0 2023-12-01T03:12:11.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.696+0000 7f4f6ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f70159bc0 con 0x7f4f700ce830 2023-12-01T03:12:11.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.696+0000 7f4f6f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7015cb90 0x7f4f701e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:11.699 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.696+0000 7f4f6f7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7015cb90 0x7f4f701e31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52790/0 (socket says 172.21.15.115:52790) 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.696+0000 7f4f6f7fe640 1 -- 172.21.15.115:0/3954320744 learned_addr learned my addr 172.21.15.115:0/3954320744 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.696+0000 7f4f6effd640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f700ce830 0x7f4f70150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f67fff640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f7015c5d0 0x7f4f700c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f6f7fe640 1 -- 172.21.15.115:0/3954320744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f700ce830 msgr2=0x7f4f70150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f6f7fe640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f700ce830 0x7f4f70150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f6f7fe640 1 -- 172.21.15.115:0/3954320744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f7015c5d0 msgr2=0x7f4f700c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f6f7fe640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f7015c5d0 0x7f4f700c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f6f7fe640 1 -- 172.21.15.115:0/3954320744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f580094d0 con 0x7f4f7015cb90 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f6effd640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f700ce830 0x7f4f70150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f6f7fe640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7015cb90 0x7f4f701e31e0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800c7b0 tx=0x7f4f6800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f677fe640 1 -- 172.21.15.115:0/3954320744 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68004b60 con 0x7f4f7015cb90 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f677fe640 1 -- 172.21.15.115:0/3954320744 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f68004cc0 con 0x7f4f7015cb90 2023-12-01T03:12:11.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f677fe640 1 -- 172.21.15.115:0/3954320744 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800aa90 con 0x7f4f7015cb90 2023-12-01T03:12:11.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f6ffff640 1 -- 172.21.15.115:0/3954320744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f70159ea0 con 0x7f4f7015cb90 2023-12-01T03:12:11.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.697+0000 7f4f6ffff640 1 -- 172.21.15.115:0/3954320744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7015a460 con 0x7f4f7015cb90 2023-12-01T03:12:11.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.698+0000 7f4f6ffff640 1 -- 172.21.15.115:0/3954320744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7015ac40 con 0x7f4f7015cb90 2023-12-01T03:12:11.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.699+0000 7f4f677fe640 1 -- 172.21.15.115:0/3954320744 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 140) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f4f6800abf0 con 0x7f4f7015cb90 2023-12-01T03:12:11.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.700+0000 7f4f677fe640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f5006a280 0x7f4f5006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:11.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.700+0000 7f4f677fe640 1 -- 172.21.15.115:0/3954320744 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7f4f68088e10 con 0x7f4f7015cb90 2023-12-01T03:12:11.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.700+0000 7f4f6effd640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f5006a280 0x7f4f5006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:11.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.701+0000 7f4f6effd640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f5006a280 0x7f4f5006c740 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f60008890 tx=0x7f4f60008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:11.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.704+0000 7f4f677fe640 1 -- 172.21.15.115:0/3954320744 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4f6808c030 con 0x7f4f7015cb90 2023-12-01T03:12:11.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.750+0000 7f4f677fe640 1 -- 172.21.15.115:0/3954320744 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 141) v1 ==== 88755+0+0 (secure 0 0 0) 0x7f4f680543b0 con 0x7f4f7015cb90 2023-12-01T03:12:11.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.937+0000 7f4f6ffff640 1 -- 172.21.15.115:0/3954320744 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f700673a0 con 0x7f4f7015cb90 2023-12-01T03:12:11.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.938+0000 7f4f677fe640 1 -- 172.21.15.115:0/3954320744 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5786 (secure 0 0 0) 0x7f4f680717d0 con 0x7f4f7015cb90 2023-12-01T03:12:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:11.940 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:32.176438+0000","modified":"2023-12-01T03:11:35.672576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7735":{"gid":7735,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1837882056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1837882056},{"type":"v1","addr":"172.21.15.142:6835","nonce":1837882056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:51.037525+0000","modified":"2023-12-01T03:11:54.055433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7732":{"gid":7732,"name":"b","rank":0,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-01T03:12:11.941 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 116 2023-12-01T03:12:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 -- 172.21.15.115:0/3954320744 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f5006a280 msgr2=0x7f4f5006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f5006a280 0x7f4f5006c740 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f60008890 tx=0x7f4f60008040 comp rx=0 tx=0).stop 2023-12-01T03:12:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 -- 172.21.15.115:0/3954320744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7015cb90 msgr2=0x7f4f701e31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7015cb90 0x7f4f701e31e0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800c7b0 tx=0x7f4f6800cc70 comp rx=0 tx=0).stop 2023-12-01T03:12:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 -- 172.21.15.115:0/3954320744 shutdown_connections 2023-12-01T03:12:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f5006a280 0x7f4f5006c740 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7015cb90 0x7f4f701e31e0 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f7015c5d0 0x7f4f700c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 --2- 172.21.15.115:0/3954320744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f700ce830 0x7f4f70150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 -- 172.21.15.115:0/3954320744 >> 172.21.15.115:0/3954320744 conn(0x7f4f700ceed0 msgr2=0x7f4f70153230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 -- 172.21.15.115:0/3954320744 shutdown_connections 2023-12-01T03:12:11.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:11.941+0000 7f4f6ffff640 1 -- 172.21.15.115:0/3954320744 wait complete. 2023-12-01T03:12:11.962 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:12.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.173+0000 7ff8278e0640 1 Processor -- start 2023-12-01T03:12:12.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.173+0000 7ff8278e0640 1 -- start start 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.174+0000 7ff8278e0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff82015e2f0 0x7ff8200ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.174+0000 7ff8278e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff82006f2f0 0x7ff8200caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.174+0000 7ff8278e0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8200add10 con 0x7ff8200adf10 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.174+0000 7ff8278e0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8200c5760 con 0x7ff82015e2f0 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.174+0000 7ff8278e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82015e6d0 con 0x7ff82006f2f0 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.175+0000 7ff82567c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff82015e2f0 0x7ff8200ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.175+0000 7ff82567c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff82015e2f0 0x7ff8200ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52806/0 (socket says 172.21.15.115:52806) 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.175+0000 7ff82567c640 1 -- 172.21.15.115:0/2381053609 learned_addr learned my addr 172.21.15.115:0/2381053609 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.175+0000 7ff82567c640 1 -- 172.21.15.115:0/2381053609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff82006f2f0 msgr2=0x7ff8200caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.175+0000 7ff82567c640 1 --2- 172.21.15.115:0/2381053609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff82006f2f0 0x7ff8200caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.175+0000 7ff82567c640 1 -- 172.21.15.115:0/2381053609 >> v1:172.21.15.142:6789/0 conn(0x7ff8200adf10 legacy=0x7ff8200c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:12.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.175+0000 7ff82567c640 1 -- 172.21.15.115:0/2381053609 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8200cf7b0 con 0x7ff82015e2f0 2023-12-01T03:12:12.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.176+0000 7ff82567c640 1 --2- 172.21.15.115:0/2381053609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 0x7ff8200ca660 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009870 tx=0x7ff81c02b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31a9b78e343a77a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:12.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.176+0000 7ff817fff640 1 -- 172.21.15.115:0/2381053609 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c033620 con 0x7ff82015e2f0 2023-12-01T03:12:12.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.176+0000 7ff817fff640 1 -- 172.21.15.115:0/2381053609 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c033780 con 0x7ff82015e2f0 2023-12-01T03:12:12.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.176+0000 7ff817fff640 1 -- 172.21.15.115:0/2381053609 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c033b10 con 0x7ff82015e2f0 2023-12-01T03:12:12.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.177+0000 7ff8278e0640 1 -- 172.21.15.115:0/2381053609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 msgr2=0x7ff8200ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.177+0000 7ff8278e0640 1 --2- 172.21.15.115:0/2381053609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 0x7ff8200ca660 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009870 tx=0x7ff81c02b7d0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.177+0000 7ff8278e0640 1 -- 172.21.15.115:0/2381053609 shutdown_connections 2023-12-01T03:12:12.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.177+0000 7ff8278e0640 1 --2- 172.21.15.115:0/2381053609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff82006f2f0 0x7ff8200caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.177+0000 7ff8278e0640 1 --2- 172.21.15.115:0/2381053609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 0x7ff8200ca660 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.177+0000 7ff8278e0640 1 -- 172.21.15.115:0/2381053609 >> 172.21.15.115:0/2381053609 conn(0x7ff8200ce880 msgr2=0x7ff8200cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:12.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.177+0000 7ff8278e0640 1 -- 172.21.15.115:0/2381053609 shutdown_connections 2023-12-01T03:12:12.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.177+0000 7ff8278e0640 1 -- 172.21.15.115:0/2381053609 wait complete. 2023-12-01T03:12:12.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.179+0000 7ff8278e0640 1 Processor -- start 2023-12-01T03:12:12.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.179+0000 7ff8278e0640 1 -- start start 2023-12-01T03:12:12.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff8278e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82006f2f0 0x7ff820157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff8278e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8200adf10 0x7ff82015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.182 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff8278e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 0x7ff8200c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff8278e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201591b0 con 0x7ff82015e2f0 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff8278e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff820159490 con 0x7ff82006f2f0 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff8278e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff820159770 con 0x7ff8200adf10 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff825e7d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 0x7ff8200c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff825e7d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 0x7ff8200c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52820/0 (socket says 172.21.15.115:52820) 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff825e7d640 1 -- 172.21.15.115:0/124579467 learned_addr learned my addr 172.21.15.115:0/124579467 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff82567c640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82006f2f0 0x7ff820157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.180+0000 7ff824e7b640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8200adf10 0x7ff82015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff825e7d640 1 -- 172.21.15.115:0/124579467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8200adf10 msgr2=0x7ff82015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff825e7d640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8200adf10 0x7ff82015a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff825e7d640 1 -- 172.21.15.115:0/124579467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82006f2f0 msgr2=0x7ff820157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff825e7d640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82006f2f0 0x7ff820157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff825e7d640 1 -- 172.21.15.115:0/124579467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0094d0 con 0x7ff82015e2f0 2023-12-01T03:12:12.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff825e7d640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 0x7ff8200c5950 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7ff81800b9e0 tx=0x7ff81800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:12.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff815ffb640 1 -- 172.21.15.115:0/124579467 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800cb20 con 0x7ff82015e2f0 2023-12-01T03:12:12.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff815ffb640 1 -- 172.21.15.115:0/124579467 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81800cc80 con 0x7ff82015e2f0 2023-12-01T03:12:12.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff8278e0640 1 -- 172.21.15.115:0/124579467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff820159a50 con 0x7ff82015e2f0 2023-12-01T03:12:12.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff815ffb640 1 -- 172.21.15.115:0/124579467 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818014680 con 0x7ff82015e2f0 2023-12-01T03:12:12.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.181+0000 7ff8278e0640 1 -- 172.21.15.115:0/124579467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8201583f0 con 0x7ff82015e2f0 2023-12-01T03:12:12.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.182+0000 7ff8278e0640 1 -- 172.21.15.115:0/124579467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82006fda0 con 0x7ff82015e2f0 2023-12-01T03:12:12.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.183+0000 7ff815ffb640 1 -- 172.21.15.115:0/124579467 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 141) v1 ==== 88755+0+0 (secure 0 0 0) 0x7ff81801d3b0 con 0x7ff82015e2f0 2023-12-01T03:12:12.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.185+0000 7ff815ffb640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff80406a5e0 0x7ff80406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.185+0000 7ff815ffb640 1 -- 172.21.15.115:0/124579467 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7ff818089270 con 0x7ff82015e2f0 2023-12-01T03:12:12.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.185+0000 7ff82567c640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff80406a5e0 0x7ff80406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.186+0000 7ff82567c640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff80406a5e0 0x7ff80406caa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff81c035000 tx=0x7ff81c008030 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:12.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.190+0000 7ff815ffb640 1 -- 172.21.15.115:0/124579467 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff818053eb0 con 0x7ff82015e2f0 2023-12-01T03:12:12.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.384+0000 7ff8278e0640 1 -- 172.21.15.115:0/124579467 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff820159f70 con 0x7ff82015e2f0 2023-12-01T03:12:12.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.388+0000 7ff815ffb640 1 -- 172.21.15.115:0/124579467 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v241) v1 ==== 74+0+21019 (secure 0 0 0) 0x7ff820159f70 con 0x7ff82015e2f0 2023-12-01T03:12:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:12.391 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":241,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:09.014479+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:11:49.229904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:11:50.056441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2980649202":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/1025490173":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2408987063":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3619877240":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2405923893":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/957514002":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/638241265":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1722505219":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3713299174":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/4124642007":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6824/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/3863453309":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1160795001":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/988288848":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/524060219":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1801977880":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/203535105":"2023-12-02T03:12:02.677778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:12.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 -- 172.21.15.115:0/124579467 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff80406a5e0 msgr2=0x7ff80406caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff80406a5e0 0x7ff80406caa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff81c035000 tx=0x7ff81c008030 comp rx=0 tx=0).stop 2023-12-01T03:12:12.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 -- 172.21.15.115:0/124579467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 msgr2=0x7ff8200c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 0x7ff8200c5950 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7ff81800b9e0 tx=0x7ff81800bea0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 -- 172.21.15.115:0/124579467 shutdown_connections 2023-12-01T03:12:12.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff80406a5e0 0x7ff80406caa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8200adf10 0x7ff82015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff82015e2f0 0x7ff8200c5950 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 --2- 172.21.15.115:0/124579467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff82006f2f0 0x7ff820157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 -- 172.21.15.115:0/124579467 >> 172.21.15.115:0/124579467 conn(0x7ff8200ce880 msgr2=0x7ff8200cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:12.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 -- 172.21.15.115:0/124579467 shutdown_connections 2023-12-01T03:12:12.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.391+0000 7ff8278e0640 1 -- 172.21.15.115:0/124579467 wait complete. 2023-12-01T03:12:12.412 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T03:12:12.413 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:12:12.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.613+0000 7fc3150c7640 1 Processor -- start 2023-12-01T03:12:12.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.613+0000 7fc3150c7640 1 -- start start 2023-12-01T03:12:12.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.615+0000 7fc3150c7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015e2f0 0x7fc3100ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.615+0000 7fc3150c7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc3100adf10 0x7fc3100caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.615+0000 7fc3150c7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3100add10 con 0x7fc31006f2f0 2023-12-01T03:12:12.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.615+0000 7fc3150c7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3100c5760 con 0x7fc3100adf10 2023-12-01T03:12:12.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.615+0000 7fc3150c7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31015e6d0 con 0x7fc31015e2f0 2023-12-01T03:12:12.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.615+0000 7fc307fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc31006f2f0 0x7fc3100c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60002/0 (socket says 172.21.15.115:60002) 2023-12-01T03:12:12.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.615+0000 7fc307fff640 1 -- 172.21.15.115:0/3808807773 learned_addr learned my addr 172.21.15.115:0/3808807773 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30ed9d640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015e2f0 0x7fc3100ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30f59e640 1 --2- 172.21.15.115:0/3808807773 >> v2:172.21.15.142:3300/0 conn(0x7fc3100adf10 0x7fc3100caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824458928 0 0) 0x7fc3100add10 con 0x7fc31006f2f0 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2fc003520 con 0x7fc31006f2f0 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 233512794 0 0) 0x7fc2fc003520 con 0x7fc31006f2f0 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015e2f0 msgr2=0x7fc3100ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015e2f0 0x7fc3100ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 >> v2:172.21.15.142:3300/0 conn(0x7fc3100adf10 msgr2=0x7fc3100caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 --2- 172.21.15.115:0/3808807773 >> v2:172.21.15.142:3300/0 conn(0x7fc3100adf10 0x7fc3100caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3100cf7b0 con 0x7fc31006f2f0 2023-12-01T03:12:12.619 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc2f8001670 con 0x7fc31006f2f0 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 >> v1:172.21.15.115:6789/0 conn(0x7fc31006f2f0 legacy=0x7fc3100c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015e2f0 0x7fc2fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 0x7fc2fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc004d00 0x7fc2fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3100c5760 con 0x7fc2fc003f60 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31015e6d0 con 0x7fc2fc004d00 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.616+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc00bae0 con 0x7fc31015e2f0 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.617+0000 7fc307fff640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 0x7fc2fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.617+0000 7fc30f59e640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc004d00 0x7fc2fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.617+0000 7fc307fff640 1 -- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015e2f0 msgr2=0x7fc2fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.617+0000 7fc307fff640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc31015e2f0 0x7fc2fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.617+0000 7fc307fff640 1 -- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc004d00 msgr2=0x7fc2fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.617+0000 7fc307fff640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc004d00 0x7fc2fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.617+0000 7fc307fff640 1 -- 172.21.15.115:0/3808807773 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc003520 con 0x7fc2fc003f60 2023-12-01T03:12:12.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.617+0000 7fc30f59e640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2fc004d00 0x7fc2fc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:12.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.618+0000 7fc307fff640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 0x7fc2fc0047c0 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800c240 tx=0x7fc2fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702c1beb7f0b4fe1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:12.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.618+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f802cb00 con 0x7fc2fc003f60 2023-12-01T03:12:12.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.618+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f802cc60 con 0x7fc2fc003f60 2023-12-01T03:12:12.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.618+0000 7fc30e59c640 1 -- 172.21.15.115:0/3808807773 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8005af0 con 0x7fc2fc003f60 2023-12-01T03:12:12.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.619+0000 7fc3150c7640 1 -- 172.21.15.115:0/3808807773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 msgr2=0x7fc2fc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.619+0000 7fc3150c7640 1 --2- 172.21.15.115:0/3808807773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 0x7fc2fc0047c0 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800c240 tx=0x7fc2fc0057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.619+0000 7fc30f59e640 1 -- 172.21.15.115:0/3808807773 reap_dead start 2023-12-01T03:12:12.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.619+0000 7fc3150c7640 1 -- 172.21.15.115:0/3808807773 shutdown_connections 2023-12-01T03:12:12.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.619+0000 7fc3150c7640 1 -- 172.21.15.115:0/3808807773 >> 172.21.15.115:0/3808807773 conn(0x7fc3100ce880 msgr2=0x7fc3100cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:12.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.619+0000 7fc3150c7640 1 -- 172.21.15.115:0/3808807773 shutdown_connections 2023-12-01T03:12:12.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.619+0000 7fc3150c7640 1 -- 172.21.15.115:0/3808807773 wait complete. 2023-12-01T03:12:12.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.621+0000 7fc3150c7640 1 Processor -- start 2023-12-01T03:12:12.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.621+0000 7fc3150c7640 1 -- start start 2023-12-01T03:12:12.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.622+0000 7fc3150c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 0x7fc3101f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.622+0000 7fc3150c7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3101e6d10 0x7fc3100af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.622+0000 7fc3150c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3100afbb0 0x7fc3100b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.622+0000 7fc3150c7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3100b0030 con 0x7fc2fc003f60 2023-12-01T03:12:12.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.622+0000 7fc3150c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3101f3780 con 0x7fc3100afbb0 2023-12-01T03:12:12.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.622+0000 7fc3150c7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3100c61d0 con 0x7fc3101e6d10 2023-12-01T03:12:12.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.622+0000 7fc30ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 0x7fc3101f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.622+0000 7fc30ed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 0x7fc3101f3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52828/0 (socket says 172.21.15.115:52828) 2023-12-01T03:12:12.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.622+0000 7fc30ed9d640 1 -- 172.21.15.115:0/4132054008 learned_addr learned my addr 172.21.15.115:0/4132054008 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc30ed9d640 1 -- 172.21.15.115:0/4132054008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3101e6d10 msgr2=0x7fc3100af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc30f59e640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3100afbb0 0x7fc3100b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.625 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc30ed9d640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3101e6d10 0x7fc3100af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc30ed9d640 1 -- 172.21.15.115:0/4132054008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3100afbb0 msgr2=0x7fc3100b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc30ed9d640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3100afbb0 0x7fc3100b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc30ed9d640 1 -- 172.21.15.115:0/4132054008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3100c64b0 con 0x7fc2fc003f60 2023-12-01T03:12:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc30ed9d640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 0x7fc3101f3240 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7fc30000e7c0 tx=0x7fc30000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc3077fe640 1 -- 172.21.15.115:0/4132054008 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000eed0 con 0x7fc2fc003f60 2023-12-01T03:12:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc3077fe640 1 -- 172.21.15.115:0/4132054008 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30000c940 con 0x7fc2fc003f60 2023-12-01T03:12:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc3150c7640 1 -- 172.21.15.115:0/4132054008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3100c6790 con 0x7fc2fc003f60 2023-12-01T03:12:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc3077fe640 1 -- 172.21.15.115:0/4132054008 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000ccd0 con 0x7fc2fc003f60 2023-12-01T03:12:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.623+0000 7fc3150c7640 1 -- 172.21.15.115:0/4132054008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3100c8630 con 0x7fc2fc003f60 2023-12-01T03:12:12.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.624+0000 7fc3150c7640 1 -- 172.21.15.115:0/4132054008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31006fda0 con 0x7fc2fc003f60 2023-12-01T03:12:12.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.626+0000 7fc3077fe640 1 -- 172.21.15.115:0/4132054008 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 141) v1 ==== 88755+0+0 (secure 0 0 0) 0x7fc300013470 con 0x7fc2fc003f60 2023-12-01T03:12:12.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.626+0000 7fc3077fe640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc2f006a610 0x7fc2f006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:12.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.627+0000 7fc3077fe640 1 -- 172.21.15.115:0/4132054008 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7fc3000892f0 con 0x7fc2fc003f60 2023-12-01T03:12:12.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.627+0000 7fc307fff640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc2f006a610 0x7fc2f006cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:12.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.627+0000 7fc307fff640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc2f006a610 0x7fc2f006cad0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8034000 tx=0x7fc2f802e000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:12.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.631+0000 7fc3077fe640 1 -- 172.21.15.115:0/4132054008 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc300053fc0 con 0x7fc2fc003f60 2023-12-01T03:12:12.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.851+0000 7fc3150c7640 1 -- 172.21.15.115:0/4132054008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc31004e890 con 0x7fc2fc003f60 2023-12-01T03:12:12.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.853+0000 7fc3077fe640 1 -- 172.21.15.115:0/4132054008 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5786 (secure 0 0 0) 0x7fc3000575e0 con 0x7fc2fc003f60 2023-12-01T03:12:12.855 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:12.855 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:32.176438+0000","modified":"2023-12-01T03:11:35.672576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7735":{"gid":7735,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1837882056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1837882056},{"type":"v1","addr":"172.21.15.142:6835","nonce":1837882056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:51.037525+0000","modified":"2023-12-01T03:11:54.055433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7732":{"gid":7732,"name":"b","rank":0,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-01T03:12:12.855 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 116 2023-12-01T03:12:12.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 -- 172.21.15.115:0/4132054008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc2f006a610 msgr2=0x7fc2f006cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc2f006a610 0x7fc2f006cad0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8034000 tx=0x7fc2f802e000 comp rx=0 tx=0).stop 2023-12-01T03:12:12.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 -- 172.21.15.115:0/4132054008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 msgr2=0x7fc3101f3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:12.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 0x7fc3101f3240 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7fc30000e7c0 tx=0x7fc30000ec80 comp rx=0 tx=0).stop 2023-12-01T03:12:12.857 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 -- 172.21.15.115:0/4132054008 shutdown_connections 2023-12-01T03:12:12.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc2f006a610 0x7fc2f006cad0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3101e6d10 0x7fc3100af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3100afbb0 0x7fc3100b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 --2- 172.21.15.115:0/4132054008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2fc003f60 0x7fc3101f3240 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:12.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 -- 172.21.15.115:0/4132054008 >> 172.21.15.115:0/4132054008 conn(0x7fc3100ce880 msgr2=0x7fc3100b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:12.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 -- 172.21.15.115:0/4132054008 shutdown_connections 2023-12-01T03:12:12.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:12.855+0000 7fc3150c7640 1 -- 172.21.15.115:0/4132054008 wait complete. 2023-12-01T03:12:12.875 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:12:13.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.079+0000 7f5d04a84640 1 Processor -- start 2023-12-01T03:12:13.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.080+0000 7f5d04a84640 1 -- start start 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.081+0000 7f5d04a84640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5d000b5d80 0x7f5d000d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.081+0000 7f5d04a84640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5d000d1230 0x7f5d000d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.081+0000 7f5d04a84640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d000d1b90 con 0x7f5d0015cd20 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.081+0000 7f5d04a84640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d000b2160 con 0x7f5d000b5d80 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.081+0000 7f5d04a84640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d000b8c80 con 0x7f5d000d1230 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.081+0000 7f5cf7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5d000b5d80 0x7f5d000d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.081+0000 7f5cf7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5d000b5d80 0x7f5d000d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52842/0 (socket says 172.21.15.115:52842) 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.081+0000 7f5cf7fff640 1 -- 172.21.15.115:0/293903895 learned_addr learned my addr 172.21.15.115:0/293903895 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.082+0000 7f5cfed9d640 1 --2- 172.21.15.115:0/293903895 >> v2:172.21.15.142:3300/0 conn(0x7f5d000d1230 0x7f5d000d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.082+0000 7f5cf7fff640 1 -- 172.21.15.115:0/293903895 >> v2:172.21.15.142:3300/0 conn(0x7f5d000d1230 msgr2=0x7f5d000d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.082+0000 7f5cf7fff640 1 --2- 172.21.15.115:0/293903895 >> v2:172.21.15.142:3300/0 conn(0x7f5d000d1230 0x7f5d000d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.082+0000 7f5cf7fff640 1 -- 172.21.15.115:0/293903895 >> v1:172.21.15.142:6789/0 conn(0x7f5d0015cd20 legacy=0x7f5d000b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:13.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.082+0000 7f5cf7fff640 1 -- 172.21.15.115:0/293903895 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d000ce2e0 con 0x7f5d000b5d80 2023-12-01T03:12:13.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.083+0000 7f5cf7fff640 1 --2- 172.21.15.115:0/293903895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d000b5d80 0x7f5d000d24d0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f5d000c6720 tx=0x7f5ce802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667f85326206c7d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:13.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.083+0000 7f5cfdd9b640 1 -- 172.21.15.115:0/293903895 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80317e0 con 0x7f5d000b5d80 2023-12-01T03:12:13.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.083+0000 7f5cfdd9b640 1 -- 172.21.15.115:0/293903895 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce8031940 con 0x7f5d000b5d80 2023-12-01T03:12:13.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.083+0000 7f5cfdd9b640 1 -- 172.21.15.115:0/293903895 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8031cd0 con 0x7f5d000b5d80 2023-12-01T03:12:13.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.084+0000 7f5d04a84640 1 -- 172.21.15.115:0/293903895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d000b5d80 msgr2=0x7f5d000d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.084+0000 7f5d04a84640 1 --2- 172.21.15.115:0/293903895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d000b5d80 0x7f5d000d24d0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f5d000c6720 tx=0x7f5ce802b880 comp rx=0 tx=0).stop 2023-12-01T03:12:13.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.084+0000 7f5d04a84640 1 -- 172.21.15.115:0/293903895 shutdown_connections 2023-12-01T03:12:13.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.084+0000 7f5d04a84640 1 --2- 172.21.15.115:0/293903895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d000b5d80 0x7f5d000d24d0 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.084+0000 7f5d04a84640 1 --2- 172.21.15.115:0/293903895 >> v2:172.21.15.142:3300/0 conn(0x7f5d000d1230 0x7f5d000d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.084+0000 7f5d04a84640 1 -- 172.21.15.115:0/293903895 >> 172.21.15.115:0/293903895 conn(0x7f5d0006a0e0 msgr2=0x7f5d0006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:13.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.084+0000 7f5d04a84640 1 -- 172.21.15.115:0/293903895 shutdown_connections 2023-12-01T03:12:13.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.084+0000 7f5d04a84640 1 -- 172.21.15.115:0/293903895 wait complete. 2023-12-01T03:12:13.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.085+0000 7f5d04a84640 1 Processor -- start 2023-12-01T03:12:13.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.086+0000 7f5d04a84640 1 -- start start 2023-12-01T03:12:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.086+0000 7f5d04a84640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d000b5d80 0x7f5d001f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.086+0000 7f5d04a84640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d000d1230 0x7f5d0015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5d04a84640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0015cd20 0x7f5d001586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5d04a84640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d001559c0 con 0x7f5d0015cd20 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5d04a84640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000cad70 con 0x7f5d000d1230 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5d04a84640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000cafc0 con 0x7f5d000b5d80 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5cfed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0015cd20 0x7f5d001586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5cfed9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0015cd20 0x7f5d001586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52854/0 (socket says 172.21.15.115:52854) 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5cfed9d640 1 -- 172.21.15.115:0/1069068052 learned_addr learned my addr 172.21.15.115:0/1069068052 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5cf7fff640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d000d1230 0x7f5d0015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5cfed9d640 1 -- 172.21.15.115:0/1069068052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d000b5d80 msgr2=0x7f5d001f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5cfed9d640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d000b5d80 0x7f5d001f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5cfed9d640 1 -- 172.21.15.115:0/1069068052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d000d1230 msgr2=0x7f5d0015f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5cfed9d640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d000d1230 0x7f5d0015f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5cfed9d640 1 -- 172.21.15.115:0/1069068052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce80094d0 con 0x7f5d0015cd20 2023-12-01T03:12:13.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.087+0000 7f5cf7fff640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d000d1230 0x7f5d0015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.088+0000 7f5cfed9d640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0015cd20 0x7f5d001586c0 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800cd70 tx=0x7f5cf800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.088+0000 7f5cf6ffd640 1 -- 172.21.15.115:0/1069068052 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8003b90 con 0x7f5d0015cd20 2023-12-01T03:12:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.088+0000 7f5cf6ffd640 1 -- 172.21.15.115:0/1069068052 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf8003cf0 con 0x7f5d0015cd20 2023-12-01T03:12:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.088+0000 7f5cf6ffd640 1 -- 172.21.15.115:0/1069068052 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8004240 con 0x7f5d0015cd20 2023-12-01T03:12:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.088+0000 7f5d04a84640 1 -- 172.21.15.115:0/1069068052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d000cb2a0 con 0x7f5d0015cd20 2023-12-01T03:12:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.088+0000 7f5d04a84640 1 -- 172.21.15.115:0/1069068052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d000cb860 con 0x7f5d0015cd20 2023-12-01T03:12:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.089+0000 7f5d04a84640 1 -- 172.21.15.115:0/1069068052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0004e890 con 0x7f5d0015cd20 2023-12-01T03:12:13.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.090+0000 7f5cf6ffd640 1 -- 172.21.15.115:0/1069068052 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 141) v1 ==== 88755+0+0 (secure 0 0 0) 0x7f5cf80043a0 con 0x7f5d0015cd20 2023-12-01T03:12:13.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.091+0000 7f5cf6ffd640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ce006a5e0 0x7f5ce006caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.091+0000 7f5cf6ffd640 1 -- 172.21.15.115:0/1069068052 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7f5cf8012070 con 0x7f5d0015cd20 2023-12-01T03:12:13.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.091+0000 7f5cfe59c640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ce006a5e0 0x7f5ce006caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.092+0000 7f5cfe59c640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ce006a5e0 0x7f5ce006caa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009650 tx=0x7f5cf0006ba0 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:13.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.095+0000 7f5cf6ffd640 1 -- 172.21.15.115:0/1069068052 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5cf8054970 con 0x7f5d0015cd20 2023-12-01T03:12:13.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.305+0000 7f5d04a84640 1 -- 172.21.15.115:0/1069068052 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d000c9a60 con 0x7f5d0015cd20 2023-12-01T03:12:13.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.307+0000 7f5cf6ffd640 1 -- 172.21.15.115:0/1069068052 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5786 (secure 0 0 0) 0x7f5cf8057f90 con 0x7f5d0015cd20 2023-12-01T03:12:13.309 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:13.309 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:32.176438+0000","modified":"2023-12-01T03:11:35.672576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7735},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7735":{"gid":7735,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6835/1837882056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1837882056},{"type":"v1","addr":"172.21.15.142:6835","nonce":1837882056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:51.037525+0000","modified":"2023-12-01T03:11:54.055433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7732":{"gid":7732,"name":"b","rank":0,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-01T03:12:13.309 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 116 2023-12-01T03:12:13.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 -- 172.21.15.115:0/1069068052 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ce006a5e0 msgr2=0x7f5ce006caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ce006a5e0 0x7f5ce006caa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009650 tx=0x7f5cf0006ba0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 -- 172.21.15.115:0/1069068052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0015cd20 msgr2=0x7f5d001586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0015cd20 0x7f5d001586c0 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800cd70 tx=0x7f5cf800a660 comp rx=0 tx=0).stop 2023-12-01T03:12:13.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 -- 172.21.15.115:0/1069068052 shutdown_connections 2023-12-01T03:12:13.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ce006a5e0 0x7f5ce006caa0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d0015cd20 0x7f5d001586c0 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d000d1230 0x7f5d0015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 --2- 172.21.15.115:0/1069068052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d000b5d80 0x7f5d001f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 -- 172.21.15.115:0/1069068052 >> 172.21.15.115:0/1069068052 conn(0x7f5d0006a0e0 msgr2=0x7f5d001598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:13.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 -- 172.21.15.115:0/1069068052 shutdown_connections 2023-12-01T03:12:13.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.309+0000 7f5d04a84640 1 -- 172.21.15.115:0/1069068052 wait complete. 2023-12-01T03:12:13.330 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:13.535 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.534+0000 7f9ed938b640 1 Processor -- start 2023-12-01T03:12:13.536 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.534+0000 7f9ed938b640 1 -- start start 2023-12-01T03:12:13.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.535+0000 7f9ed938b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9ed415e2f0 0x7f9ed40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.535+0000 7f9ed938b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ed406f2f0 0x7f9ed40caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9ed938b640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed40add10 con 0x7f9ed40adf10 2023-12-01T03:12:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9ed938b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed40c5760 con 0x7f9ed415e2f0 2023-12-01T03:12:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9ed938b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed415e6d0 con 0x7f9ed406f2f0 2023-12-01T03:12:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9ed2ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9ed415e2f0 0x7f9ed40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9ed37fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9ed40adf10 0x7f9ed40c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60004/0 (socket says 172.21.15.115:60004) 2023-12-01T03:12:13.538 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9ed37fe640 1 -- 172.21.15.115:0/4183298793 learned_addr learned my addr 172.21.15.115:0/4183298793 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9eca7fc640 1 --2- 172.21.15.115:0/4183298793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ed406f2f0 0x7f9ed40caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9ed2ffd640 1 -- 172.21.15.115:0/4183298793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ed406f2f0 msgr2=0x7f9ed40caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9ed2ffd640 1 --2- 172.21.15.115:0/4183298793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ed406f2f0 0x7f9ed40caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9ed2ffd640 1 -- 172.21.15.115:0/4183298793 >> v1:172.21.15.115:6789/0 conn(0x7f9ed40adf10 legacy=0x7f9ed40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.537+0000 7f9ed2ffd640 1 -- 172.21.15.115:0/4183298793 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40cf7b0 con 0x7f9ed415e2f0 2023-12-01T03:12:13.539 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.536+0000 7f9ed27fc640 1 -- 172.21.15.115:0/4183298793 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425671547 0 0) 0x7f9ed40add10 con 0x7f9ed40adf10 2023-12-01T03:12:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.538+0000 7f9ed2ffd640 1 --2- 172.21.15.115:0/4183298793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed415e2f0 0x7f9ed40ca660 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40097a0 tx=0x7f9ec402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2402a533c386f34 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.538+0000 7f9ed27fc640 1 -- 172.21.15.115:0/4183298793 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec402f050 con 0x7f9ed415e2f0 2023-12-01T03:12:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.538+0000 7f9ed27fc640 1 -- 172.21.15.115:0/4183298793 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec4031700 con 0x7f9ed415e2f0 2023-12-01T03:12:13.540 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.538+0000 7f9ed27fc640 1 -- 172.21.15.115:0/4183298793 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4031a90 con 0x7f9ed415e2f0 2023-12-01T03:12:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.538+0000 7f9ed938b640 1 -- 172.21.15.115:0/4183298793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed415e2f0 msgr2=0x7f9ed40ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.538+0000 7f9ed938b640 1 --2- 172.21.15.115:0/4183298793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed415e2f0 0x7f9ed40ca660 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40097a0 tx=0x7f9ec402b7a0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.538+0000 7f9ed938b640 1 -- 172.21.15.115:0/4183298793 shutdown_connections 2023-12-01T03:12:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.539+0000 7f9ed938b640 1 --2- 172.21.15.115:0/4183298793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ed406f2f0 0x7f9ed40caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.539+0000 7f9ed938b640 1 --2- 172.21.15.115:0/4183298793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed415e2f0 0x7f9ed40ca660 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.539+0000 7f9ed938b640 1 -- 172.21.15.115:0/4183298793 >> 172.21.15.115:0/4183298793 conn(0x7f9ed40ce880 msgr2=0x7f9ed40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.539+0000 7f9ed938b640 1 -- 172.21.15.115:0/4183298793 shutdown_connections 2023-12-01T03:12:13.541 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.539+0000 7f9ed938b640 1 -- 172.21.15.115:0/4183298793 wait complete. 2023-12-01T03:12:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.540+0000 7f9ed938b640 1 Processor -- start 2023-12-01T03:12:13.542 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.540+0000 7f9ed938b640 1 -- start start 2023-12-01T03:12:13.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.541+0000 7f9ed938b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ed406f2f0 0x7f9ed41f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.544 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.541+0000 7f9ed938b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed40adf10 0x7f9ed40c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.541+0000 7f9ed938b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ed415e2f0 0x7f9ed406bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.541+0000 7f9ed938b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40c32b0 con 0x7f9ed40adf10 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.541+0000 7f9ed938b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40c3590 con 0x7f9ed406f2f0 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.541+0000 7f9ed938b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40c3870 con 0x7f9ed415e2f0 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9eca7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed40adf10 0x7f9ed40c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9eca7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed40adf10 0x7f9ed40c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52884/0 (socket says 172.21.15.115:52884) 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9eca7fc640 1 -- 172.21.15.115:0/645184245 learned_addr learned my addr 172.21.15.115:0/645184245 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9eca7fc640 1 -- 172.21.15.115:0/645184245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ed415e2f0 msgr2=0x7f9ed406bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9ed37fe640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ed415e2f0 0x7f9ed406bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9ed2ffd640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ed406f2f0 0x7f9ed41f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9eca7fc640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ed415e2f0 0x7f9ed406bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9eca7fc640 1 -- 172.21.15.115:0/645184245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ed406f2f0 msgr2=0x7f9ed41f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9eca7fc640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ed406f2f0 0x7f9ed41f1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9eca7fc640 1 -- 172.21.15.115:0/645184245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec4009400 con 0x7f9ed40adf10 2023-12-01T03:12:13.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.542+0000 7f9eca7fc640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed40adf10 0x7f9ed40c5950 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00d5f0 tx=0x7f9ebc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.543+0000 7f9ecbfff640 1 -- 172.21.15.115:0/645184245 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0125f0 con 0x7f9ed40adf10 2023-12-01T03:12:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.543+0000 7f9ecbfff640 1 -- 172.21.15.115:0/645184245 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc012750 con 0x7f9ed40adf10 2023-12-01T03:12:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.543+0000 7f9ecbfff640 1 -- 172.21.15.115:0/645184245 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc012b00 con 0x7f9ed40adf10 2023-12-01T03:12:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.543+0000 7f9ed938b640 1 -- 172.21.15.115:0/645184245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed40c3b50 con 0x7f9ed40adf10 2023-12-01T03:12:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.543+0000 7f9ed938b640 1 -- 172.21.15.115:0/645184245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed40c5e90 con 0x7f9ed40adf10 2023-12-01T03:12:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.544+0000 7f9ed938b640 1 -- 172.21.15.115:0/645184245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed406fda0 con 0x7f9ed40adf10 2023-12-01T03:12:13.546 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.545+0000 7f9ecbfff640 1 -- 172.21.15.115:0/645184245 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 141) v1 ==== 88755+0+0 (secure 0 0 0) 0x7f9ebc012c60 con 0x7f9ed40adf10 2023-12-01T03:12:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.545+0000 7f9ecbfff640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9eb406a5b0 0x7f9eb406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.546+0000 7f9ecbfff640 1 -- 172.21.15.115:0/645184245 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7f9ebc0897f0 con 0x7f9ed40adf10 2023-12-01T03:12:13.547 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.546+0000 7f9ed2ffd640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9eb406a5b0 0x7f9eb406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.548 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.546+0000 7f9ed2ffd640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9eb406a5b0 0x7f9eb406ca70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4035000 tx=0x7f9ec4008030 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:13.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.550+0000 7f9ecbfff640 1 -- 172.21.15.115:0/645184245 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9ebc054430 con 0x7f9ed40adf10 2023-12-01T03:12:13.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.729+0000 7f9ed938b640 1 -- 172.21.15.115:0/645184245 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ed404e890 con 0x7f9ed40adf10 2023-12-01T03:12:13.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.733+0000 7f9ecbfff640 1 -- 172.21.15.115:0/645184245 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v241) v1 ==== 74+0+21019 (secure 0 0 0) 0x7f9ebc057a50 con 0x7f9ed40adf10 2023-12-01T03:12:13.735 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:13.735 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":241,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:09.014479+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:11:30.230656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-01T03:11:31.144773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"235","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:11:49.229904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:11:50.056441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2980649202":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/1025490173":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2408987063":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3619877240":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2405923893":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/957514002":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/638241265":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1722505219":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3713299174":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/4124642007":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6824/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/3863453309":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1160795001":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/988288848":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/524060219":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1801977880":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/203535105":"2023-12-02T03:12:02.677778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:13.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.735+0000 7f9ed938b640 1 -- 172.21.15.115:0/645184245 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9eb406a5b0 msgr2=0x7f9eb406ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.735+0000 7f9ed938b640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9eb406a5b0 0x7f9eb406ca70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4035000 tx=0x7f9ec4008030 comp rx=0 tx=0).stop 2023-12-01T03:12:13.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.735+0000 7f9ed938b640 1 -- 172.21.15.115:0/645184245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed40adf10 msgr2=0x7f9ed40c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.735+0000 7f9ed938b640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed40adf10 0x7f9ed40c5950 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00d5f0 tx=0x7f9ebc00dab0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.736+0000 7f9ed938b640 1 -- 172.21.15.115:0/645184245 shutdown_connections 2023-12-01T03:12:13.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.736+0000 7f9ed938b640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9eb406a5b0 0x7f9eb406ca70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.736+0000 7f9ed938b640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ed415e2f0 0x7f9ed406bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.736+0000 7f9ed938b640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ed40adf10 0x7f9ed40c5950 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.736+0000 7f9ed938b640 1 --2- 172.21.15.115:0/645184245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ed406f2f0 0x7f9ed41f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.736+0000 7f9ed938b640 1 -- 172.21.15.115:0/645184245 >> 172.21.15.115:0/645184245 conn(0x7f9ed40ce880 msgr2=0x7f9ed40ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:13.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.736+0000 7f9ed938b640 1 -- 172.21.15.115:0/645184245 shutdown_connections 2023-12-01T03:12:13.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.736+0000 7f9ed938b640 1 -- 172.21.15.115:0/645184245 wait complete. 2023-12-01T03:12:13.758 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T03:12:13.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.955+0000 7faed5b16640 1 Processor -- start 2023-12-01T03:12:13.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.956+0000 7faed5b16640 1 -- start start 2023-12-01T03:12:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.957+0000 7faed5b16640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faed0159dd0 0x7faed00c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.957+0000 7faed5b16640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faed006b6f0 0x7faed00c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.957+0000 7faed5b16640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed006f8b0 con 0x7faed00ade80 2023-12-01T03:12:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.957+0000 7faed5b16640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed00adcb0 con 0x7faed006b6f0 2023-12-01T03:12:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.957+0000 7faed5b16640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed00ad9d0 con 0x7faed0159dd0 2023-12-01T03:12:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.957+0000 7faec7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faed006b6f0 0x7faed00c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.957+0000 7faecffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7faed00ade80 0x7faed00cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60006/0 (socket says 172.21.15.115:60006) 2023-12-01T03:12:13.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.957+0000 7faecffff640 1 -- 172.21.15.115:0/3220830609 learned_addr learned my addr 172.21.15.115:0/3220830609 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.958+0000 7faec7fff640 1 -- 172.21.15.115:0/3220830609 >> v2:172.21.15.142:3300/0 conn(0x7faed0159dd0 msgr2=0x7faed00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.958+0000 7faecf7fe640 1 --2- 172.21.15.115:0/3220830609 >> v2:172.21.15.142:3300/0 conn(0x7faed0159dd0 0x7faed00c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.958+0000 7faec7fff640 1 --2- 172.21.15.115:0/3220830609 >> v2:172.21.15.142:3300/0 conn(0x7faed0159dd0 0x7faed00c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.958+0000 7faec7fff640 1 -- 172.21.15.115:0/3220830609 >> v1:172.21.15.115:6789/0 conn(0x7faed00ade80 legacy=0x7faed00cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.958+0000 7faec7fff640 1 -- 172.21.15.115:0/3220830609 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00c9000 con 0x7faed006b6f0 2023-12-01T03:12:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.958+0000 7faeceffd640 1 -- 172.21.15.115:0/3220830609 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627379507 0 0) 0x7faed006f8b0 con 0x7faed00ade80 2023-12-01T03:12:13.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.958+0000 7faecf7fe640 1 --2- 172.21.15.115:0/3220830609 >> v2:172.21.15.142:3300/0 conn(0x7faed0159dd0 0x7faed00c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.959+0000 7faec7fff640 1 --2- 172.21.15.115:0/3220830609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed006b6f0 0x7faed00c9d20 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7faed00cf4e0 tx=0x7faeb802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50507df267b8d894 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.959+0000 7faeceffd640 1 -- 172.21.15.115:0/3220830609 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8031710 con 0x7faed006b6f0 2023-12-01T03:12:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.959+0000 7faeceffd640 1 -- 172.21.15.115:0/3220830609 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb802f050 con 0x7faed006b6f0 2023-12-01T03:12:13.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.959+0000 7faeceffd640 1 -- 172.21.15.115:0/3220830609 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8031a90 con 0x7faed006b6f0 2023-12-01T03:12:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.960+0000 7faed5b16640 1 -- 172.21.15.115:0/3220830609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed006b6f0 msgr2=0x7faed00c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.960+0000 7faed5b16640 1 --2- 172.21.15.115:0/3220830609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed006b6f0 0x7faed00c9d20 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7faed00cf4e0 tx=0x7faeb802b7b0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.960+0000 7faed5b16640 1 -- 172.21.15.115:0/3220830609 shutdown_connections 2023-12-01T03:12:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.960+0000 7faed5b16640 1 --2- 172.21.15.115:0/3220830609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed006b6f0 0x7faed00c9d20 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.960+0000 7faed5b16640 1 --2- 172.21.15.115:0/3220830609 >> v2:172.21.15.142:3300/0 conn(0x7faed0159dd0 0x7faed00c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.960+0000 7faed5b16640 1 -- 172.21.15.115:0/3220830609 >> 172.21.15.115:0/3220830609 conn(0x7faed00ce880 msgr2=0x7faed00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.960+0000 7faed5b16640 1 -- 172.21.15.115:0/3220830609 shutdown_connections 2023-12-01T03:12:13.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.960+0000 7faed5b16640 1 -- 172.21.15.115:0/3220830609 wait complete. 2023-12-01T03:12:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.962+0000 7faed5b16640 1 Processor -- start 2023-12-01T03:12:13.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.962+0000 7faed5b16640 1 -- start start 2023-12-01T03:12:13.965 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faed5b16640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faed006b6f0 0x7faed01e5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faed5b16640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faed00ade80 0x7faed00c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faed5b16640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed0159dd0 0x7faed013e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faed5b16640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00c40b0 con 0x7faed0159dd0 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faed5b16640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00c34a0 con 0x7faed00ade80 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faed5b16640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed00c3780 con 0x7faed006b6f0 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faecffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed0159dd0 0x7faed013e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faecffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed0159dd0 0x7faed013e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52910/0 (socket says 172.21.15.115:52910) 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faecffff640 1 -- 172.21.15.115:0/1010596316 learned_addr learned my addr 172.21.15.115:0/1010596316 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faec7fff640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faed00ade80 0x7faed00c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faecffff640 1 -- 172.21.15.115:0/1010596316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faed006b6f0 msgr2=0x7faed01e5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faecffff640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faed006b6f0 0x7faed01e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faecffff640 1 -- 172.21.15.115:0/1010596316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faed00ade80 msgr2=0x7faed00c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faecffff640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faed00ade80 0x7faed00c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faecffff640 1 -- 172.21.15.115:0/1010596316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8009400 con 0x7faed0159dd0 2023-12-01T03:12:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.963+0000 7faec7fff640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faed00ade80 0x7faed00c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.964+0000 7faecffff640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed0159dd0 0x7faed013e9a0 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7faec800c7e0 tx=0x7faec800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.964+0000 7faeccff9640 1 -- 172.21.15.115:0/1010596316 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800a640 con 0x7faed0159dd0 2023-12-01T03:12:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.964+0000 7faeccff9640 1 -- 172.21.15.115:0/1010596316 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec800a7c0 con 0x7faed0159dd0 2023-12-01T03:12:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.964+0000 7faeccff9640 1 -- 172.21.15.115:0/1010596316 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800ab70 con 0x7faed0159dd0 2023-12-01T03:12:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.964+0000 7faed5b16640 1 -- 172.21.15.115:0/1010596316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed00c3a60 con 0x7faed0159dd0 2023-12-01T03:12:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.964+0000 7faed5b16640 1 -- 172.21.15.115:0/1010596316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed0150260 con 0x7faed0159dd0 2023-12-01T03:12:13.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.965+0000 7faed5b16640 1 -- 172.21.15.115:0/1010596316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed0071b00 con 0x7faed0159dd0 2023-12-01T03:12:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.966+0000 7faeccff9640 1 -- 172.21.15.115:0/1010596316 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7faec800acd0 con 0x7faed0159dd0 2023-12-01T03:12:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.967+0000 7faeccff9640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faeb006a670 0x7faeb006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.967+0000 7faeccff9640 1 -- 172.21.15.115:0/1010596316 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7391+0+0 (secure 0 0 0) 0x7faec8088a30 con 0x7faed0159dd0 2023-12-01T03:12:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.967+0000 7faecf7fe640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faeb006a670 0x7faeb006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.968+0000 7faecf7fe640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faeb006a670 0x7faeb006cb30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faec0006fd0 tx=0x7faec0008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:13.971+0000 7faeccff9640 1 -- 172.21.15.115:0/1010596316 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faec8053590 con 0x7faed0159dd0 2023-12-01T03:12:14.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.200+0000 7faed5b16640 1 -- 172.21.15.115:0/1010596316 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faed004e890 con 0x7faed0159dd0 2023-12-01T03:12:14.203 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T03:12:14.201+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T03:12:14.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.219+0000 7faeccff9640 1 -- 172.21.15.115:0/1010596316 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) v1 ==== 161+0+0 (secure 0 0 0) 0x7faec8018090 con 0x7faed0159dd0 2023-12-01T03:12:14.221 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T03:12:14.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.221+0000 7faed5b16640 1 -- 172.21.15.115:0/1010596316 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faeb006a670 msgr2=0x7faeb006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.221+0000 7faed5b16640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faeb006a670 0x7faeb006cb30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faec0006fd0 tx=0x7faec0008040 comp rx=0 tx=0).stop 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.222+0000 7faed5b16640 1 -- 172.21.15.115:0/1010596316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed0159dd0 msgr2=0x7faed013e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.222+0000 7faed5b16640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed0159dd0 0x7faed013e9a0 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7faec800c7e0 tx=0x7faec800cca0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.222+0000 7faed5b16640 1 -- 172.21.15.115:0/1010596316 shutdown_connections 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.222+0000 7faed5b16640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faeb006a670 0x7faeb006cb30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.222+0000 7faed5b16640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faed0159dd0 0x7faed013e9a0 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.222+0000 7faed5b16640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faed00ade80 0x7faed00c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.222+0000 7faed5b16640 1 --2- 172.21.15.115:0/1010596316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faed006b6f0 0x7faed01e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.222+0000 7faed5b16640 1 -- 172.21.15.115:0/1010596316 >> 172.21.15.115:0/1010596316 conn(0x7faed00ce880 msgr2=0x7faed00af930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.222+0000 7faed5b16640 1 -- 172.21.15.115:0/1010596316 shutdown_connections 2023-12-01T03:12:14.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.222+0000 7faed5b16640 1 -- 172.21.15.115:0/1010596316 wait complete. 2023-12-01T03:12:14.235 INFO:tasks.ceph.mds.d.smithi142.stderr: -4473> 2023-12-01T03:11:32.613+0000 7fc6116fc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:12:14.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T03:12:14.307 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T03:12:14.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.447+0000 7fe1fafb1640 1 Processor -- start 2023-12-01T03:12:14.449 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.448+0000 7fe1fafb1640 1 -- start start 2023-12-01T03:12:14.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.449+0000 7fe1fafb1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40cfd60 0x7fe1f40cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:14.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.449+0000 7fe1fafb1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1f40b5fa0 0x7fe1f4161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:14.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.449+0000 7fe1fafb1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f40ce5e0 con 0x7fe1f415cd90 2023-12-01T03:12:14.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.449+0000 7fe1fafb1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f40d0ec0 con 0x7fe1f40b5fa0 2023-12-01T03:12:14.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.449+0000 7fe1fafb1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f4161950 con 0x7fe1f40cfd60 2023-12-01T03:12:14.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.450+0000 7fe1f8d4d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe1f415cd90 0x7fe1f40cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60016/0 (socket says 172.21.15.115:60016) 2023-12-01T03:12:14.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.450+0000 7fe1f8d4d640 1 -- 172.21.15.115:0/3057833335 learned_addr learned my addr 172.21.15.115:0/3057833335 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:14.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.450+0000 7fe1f3fff640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40cfd60 0x7fe1f40cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:14.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.450+0000 7fe1f954e640 1 --2- 172.21.15.115:0/3057833335 >> v2:172.21.15.142:3300/0 conn(0x7fe1f40b5fa0 0x7fe1f4161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:14.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.450+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967562870 0 0) 0x7fe1e4001490 con 0x7fe1f415cd90 2023-12-01T03:12:14.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.450+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e0003520 con 0x7fe1f415cd90 2023-12-01T03:12:14.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3243965375 0 0) 0x7fe1e0003520 con 0x7fe1f415cd90 2023-12-01T03:12:14.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40cfd60 msgr2=0x7fe1f40cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:14.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40cfd60 0x7fe1f40cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 >> v2:172.21.15.142:3300/0 conn(0x7fe1f40b5fa0 msgr2=0x7fe1f4161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:14.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 --2- 172.21.15.115:0/3057833335 >> v2:172.21.15.142:3300/0 conn(0x7fe1f40b5fa0 0x7fe1f4161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f40ce300 con 0x7fe1f415cd90 2023-12-01T03:12:14.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fe1e4001b80 con 0x7fe1f415cd90 2023-12-01T03:12:14.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 >> v1:172.21.15.115:6789/0 conn(0x7fe1f415cd90 legacy=0x7fe1f40cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:14.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0003970 0x7fe1e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:14.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40cfd60 0x7fe1e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:14.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 0x7fe1e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:14.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40d0ec0 con 0x7fe1e0004d00 2023-12-01T03:12:14.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f4161950 con 0x7fe1e0003970 2023-12-01T03:12:14.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e000b980 con 0x7fe1f40cfd60 2023-12-01T03:12:14.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f954e640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 0x7fe1e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:14.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f8d4d640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0003970 0x7fe1e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:14.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f3fff640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40cfd60 0x7fe1e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:14.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f954e640 1 -- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40cfd60 msgr2=0x7fe1e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:14.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f954e640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40cfd60 0x7fe1e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f954e640 1 -- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0003970 msgr2=0x7fe1e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:14.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.451+0000 7fe1f954e640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0003970 0x7fe1e0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.452+0000 7fe1f954e640 1 -- 172.21.15.115:0/3057833335 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e0003520 con 0x7fe1e0004d00 2023-12-01T03:12:14.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.452+0000 7fe1f3fff640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40cfd60 0x7fe1e00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:14.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.453+0000 7fe1f954e640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 0x7fe1e0005180 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009ac0 tx=0x7fe1e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f32338774e1ffca server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:14.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.453+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec02c710 con 0x7fe1e0004d00 2023-12-01T03:12:14.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.453+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec02c870 con 0x7fe1e0004d00 2023-12-01T03:12:14.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.453+0000 7fe1f37fe640 1 -- 172.21.15.115:0/3057833335 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec02cc00 con 0x7fe1e0004d00 2023-12-01T03:12:14.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.454+0000 7fe1fafb1640 1 -- 172.21.15.115:0/3057833335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 msgr2=0x7fe1e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:14.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.454+0000 7fe1fafb1640 1 --2- 172.21.15.115:0/3057833335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 0x7fe1e0005180 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009ac0 tx=0x7fe1e00057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.454+0000 7fe1f954e640 1 -- 172.21.15.115:0/3057833335 reap_dead start 2023-12-01T03:12:14.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.454+0000 7fe1fafb1640 1 -- 172.21.15.115:0/3057833335 shutdown_connections 2023-12-01T03:12:14.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.454+0000 7fe1fafb1640 1 -- 172.21.15.115:0/3057833335 >> 172.21.15.115:0/3057833335 conn(0x7fe1f4065960 msgr2=0x7fe1f4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:14.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.454+0000 7fe1fafb1640 1 -- 172.21.15.115:0/3057833335 shutdown_connections 2023-12-01T03:12:14.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.454+0000 7fe1fafb1640 1 -- 172.21.15.115:0/3057833335 wait complete. 2023-12-01T03:12:14.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.456+0000 7fe1fafb1640 1 Processor -- start 2023-12-01T03:12:14.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.456+0000 7fe1fafb1640 1 -- start start 2023-12-01T03:12:14.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1fafb1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 0x7fe1f40cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:14.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1fafb1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1f40ccc40 0x7fe1f40a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:14.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1fafb1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40ab040 0x7fe1f40ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1fafb1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40aaaf0 con 0x7fe1e0004d00 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1fafb1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40c90f0 con 0x7fe1f40ccc40 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1fafb1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40aba00 con 0x7fe1f40ab040 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1f8d4d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 0x7fe1f40cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1f8d4d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 0x7fe1f40cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52930/0 (socket says 172.21.15.115:52930) 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1f8d4d640 1 -- 172.21.15.115:0/198969257 learned_addr learned my addr 172.21.15.115:0/198969257 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1f954e640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40ab040 0x7fe1f40ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1f8d4d640 1 -- 172.21.15.115:0/198969257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40ab040 msgr2=0x7fe1f40ab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1f8d4d640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40ab040 0x7fe1f40ab4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1f3fff640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1f40ccc40 0x7fe1f40a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1f8d4d640 1 -- 172.21.15.115:0/198969257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1f40ccc40 msgr2=0x7fe1f40a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1f8d4d640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1f40ccc40 0x7fe1f40a9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.457+0000 7fe1f8d4d640 1 -- 172.21.15.115:0/198969257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f40a9e70 con 0x7fe1e0004d00 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.458+0000 7fe1f8d4d640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 0x7fe1f40cb790 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4007ac0 tx=0x7fe1e400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:14.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.458+0000 7fe1f17fa640 1 -- 172.21.15.115:0/198969257 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4013420 con 0x7fe1e0004d00 2023-12-01T03:12:14.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.458+0000 7fe1f17fa640 1 -- 172.21.15.115:0/198969257 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e4013580 con 0x7fe1e0004d00 2023-12-01T03:12:14.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.458+0000 7fe1f17fa640 1 -- 172.21.15.115:0/198969257 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4013930 con 0x7fe1e0004d00 2023-12-01T03:12:14.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.458+0000 7fe1fafb1640 1 -- 172.21.15.115:0/198969257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f40aa150 con 0x7fe1e0004d00 2023-12-01T03:12:14.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.458+0000 7fe1fafb1640 1 -- 172.21.15.115:0/198969257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f40aa710 con 0x7fe1e0004d00 2023-12-01T03:12:14.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.459+0000 7fe1fafb1640 1 -- 172.21.15.115:0/198969257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f404e890 con 0x7fe1e0004d00 2023-12-01T03:12:14.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.460+0000 7fe1f17fa640 1 -- 172.21.15.115:0/198969257 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fe1e4013a90 con 0x7fe1e0004d00 2023-12-01T03:12:14.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.461+0000 7fe1f17fa640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1d406a6a0 0x7fe1d406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:14.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.461+0000 7fe1f17fa640 1 -- 172.21.15.115:0/198969257 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7477+0+0 (secure 0 0 0) 0x7fe1e4089b20 con 0x7fe1e0004d00 2023-12-01T03:12:14.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.461+0000 7fe1f3fff640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1d406a6a0 0x7fe1d406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:14.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.462+0000 7fe1f3fff640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1d406a6a0 0x7fe1d406cb60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40b5670 tx=0x7fe1dc008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:14.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.465+0000 7fe1f17fa640 1 -- 172.21.15.115:0/198969257 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe1e40546f0 con 0x7fe1e0004d00 2023-12-01T03:12:14.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:14.694+0000 7fe1fafb1640 1 -- 172.21.15.115:0/198969257 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe1f40ce5e0 con 0x7fe1e0004d00 2023-12-01T03:12:15.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.225+0000 7fe1f17fa640 1 -- 172.21.15.115:0/198969257 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v118) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1f40ce5e0 con 0x7fe1e0004d00 2023-12-01T03:12:15.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 -- 172.21.15.115:0/198969257 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1d406a6a0 msgr2=0x7fe1d406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:15.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1d406a6a0 0x7fe1d406cb60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40b5670 tx=0x7fe1dc008040 comp rx=0 tx=0).stop 2023-12-01T03:12:15.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 -- 172.21.15.115:0/198969257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 msgr2=0x7fe1f40cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:15.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 0x7fe1f40cb790 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4007ac0 tx=0x7fe1e400e530 comp rx=0 tx=0).stop 2023-12-01T03:12:15.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 -- 172.21.15.115:0/198969257 shutdown_connections 2023-12-01T03:12:15.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe1d406a6a0 0x7fe1d406cb60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:15.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1f40ab040 0x7fe1f40ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:15.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1f40ccc40 0x7fe1f40a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:15.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 --2- 172.21.15.115:0/198969257 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1e0004d00 0x7fe1f40cb790 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:15.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 -- 172.21.15.115:0/198969257 >> 172.21.15.115:0/198969257 conn(0x7fe1f4065960 msgr2=0x7fe1f4158eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:15.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 -- 172.21.15.115:0/198969257 shutdown_connections 2023-12-01T03:12:15.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.229+0000 7fe1fafb1640 1 -- 172.21.15.115:0/198969257 wait complete. 2023-12-01T03:12:15.252 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T03:12:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.462+0000 7fe027441640 1 Processor -- start 2023-12-01T03:12:15.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.462+0000 7fe027441640 1 -- start start 2023-12-01T03:12:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.464+0000 7fe027441640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe020159bf0 0x7fe02015e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.464+0000 7fe027441640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 0x7fe0200cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.464+0000 7fe027441640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02015e570 con 0x7fe0200cb6f0 2023-12-01T03:12:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.464+0000 7fe027441640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe020159fd0 con 0x7fe020159bf0 2023-12-01T03:12:15.466 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.464+0000 7fe027441640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200cbb80 con 0x7fe0200ca7c0 2023-12-01T03:12:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.465+0000 7fe0251dd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe020159bf0 0x7fe02015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.465+0000 7fe0249dc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 0x7fe0200cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.465+0000 7fe0249dc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 0x7fe0200cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:59418/0 (socket says 172.21.15.115:59418) 2023-12-01T03:12:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.465+0000 7fe0249dc640 1 -- 172.21.15.115:0/566096028 learned_addr learned my addr 172.21.15.115:0/566096028 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.465+0000 7fe0251dd640 1 -- 172.21.15.115:0/566096028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 msgr2=0x7fe0200cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.465+0000 7fe0251dd640 1 --2- 172.21.15.115:0/566096028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 0x7fe0200cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:15.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.465+0000 7fe0251dd640 1 -- 172.21.15.115:0/566096028 >> v1:172.21.15.142:6789/0 conn(0x7fe0200cb6f0 legacy=0x7fe02006f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.465+0000 7fe0251dd640 1 -- 172.21.15.115:0/566096028 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200c9440 con 0x7fe020159bf0 2023-12-01T03:12:15.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.466+0000 7fe0251dd640 1 --2- 172.21.15.115:0/566096028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe020159bf0 0x7fe02015e030 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0097a0 tx=0x7fe01c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a890d0dcd4c4ceb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.467+0000 7fe017fff640 1 -- 172.21.15.115:0/566096028 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c02d040 con 0x7fe020159bf0 2023-12-01T03:12:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.467+0000 7fe017fff640 1 -- 172.21.15.115:0/566096028 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c031810 con 0x7fe020159bf0 2023-12-01T03:12:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.467+0000 7fe017fff640 1 -- 172.21.15.115:0/566096028 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c031ba0 con 0x7fe020159bf0 2023-12-01T03:12:15.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.468+0000 7fe027441640 1 -- 172.21.15.115:0/566096028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe020159bf0 msgr2=0x7fe02015e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.468+0000 7fe027441640 1 --2- 172.21.15.115:0/566096028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe020159bf0 0x7fe02015e030 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0097a0 tx=0x7fe01c02b8b0 comp rx=0 tx=0).stop 2023-12-01T03:12:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.468+0000 7fe027441640 1 -- 172.21.15.115:0/566096028 shutdown_connections 2023-12-01T03:12:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.468+0000 7fe027441640 1 --2- 172.21.15.115:0/566096028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 0x7fe0200cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.468+0000 7fe027441640 1 --2- 172.21.15.115:0/566096028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe020159bf0 0x7fe02015e030 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.468+0000 7fe027441640 1 -- 172.21.15.115:0/566096028 >> 172.21.15.115:0/566096028 conn(0x7fe0200662c0 msgr2=0x7fe0200b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.468+0000 7fe027441640 1 -- 172.21.15.115:0/566096028 shutdown_connections 2023-12-01T03:12:15.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.468+0000 7fe027441640 1 -- 172.21.15.115:0/566096028 wait complete. 2023-12-01T03:12:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.470+0000 7fe027441640 1 Processor -- start 2023-12-01T03:12:15.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.470+0000 7fe027441640 1 -- start start 2023-12-01T03:12:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe027441640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 0x7fe02014e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:15.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe027441640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0200cb6f0 0x7fe0201583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe027441640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe020159bf0 0x7fe0201517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe027441640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0201662c0 con 0x7fe0200cb6f0 2023-12-01T03:12:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe027441640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200c6990 con 0x7fe020159bf0 2023-12-01T03:12:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe027441640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200c6c70 con 0x7fe0200ca7c0 2023-12-01T03:12:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe0249dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0200cb6f0 0x7fe0201583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe0249dc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0200cb6f0 0x7fe0201583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52942/0 (socket says 172.21.15.115:52942) 2023-12-01T03:12:15.473 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe0249dc640 1 -- 172.21.15.115:0/1894139689 learned_addr learned my addr 172.21.15.115:0/1894139689 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:15.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe0259de640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe020159bf0 0x7fe0201517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:15.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe0251dd640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 0x7fe02014e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:15.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe0249dc640 1 -- 172.21.15.115:0/1894139689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 msgr2=0x7fe02014e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:15.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe0249dc640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 0x7fe02014e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:15.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe0249dc640 1 -- 172.21.15.115:0/1894139689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe020159bf0 msgr2=0x7fe0201517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:15.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe0249dc640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe020159bf0 0x7fe0201517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:15.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.471+0000 7fe0249dc640 1 -- 172.21.15.115:0/1894139689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c009400 con 0x7fe0200cb6f0 2023-12-01T03:12:15.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.472+0000 7fe0249dc640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0200cb6f0 0x7fe0201583a0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fe010009fd0 tx=0x7fe01000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:15.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.472+0000 7fe015ffb640 1 -- 172.21.15.115:0/1894139689 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000c940 con 0x7fe0200cb6f0 2023-12-01T03:12:15.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.472+0000 7fe015ffb640 1 -- 172.21.15.115:0/1894139689 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01000caa0 con 0x7fe0200cb6f0 2023-12-01T03:12:15.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.472+0000 7fe015ffb640 1 -- 172.21.15.115:0/1894139689 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000ce30 con 0x7fe0200cb6f0 2023-12-01T03:12:15.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.472+0000 7fe027441640 1 -- 172.21.15.115:0/1894139689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe020165f50 con 0x7fe0200cb6f0 2023-12-01T03:12:15.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.472+0000 7fe027441640 1 -- 172.21.15.115:0/1894139689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0201588e0 con 0x7fe0200cb6f0 2023-12-01T03:12:15.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.473+0000 7fe027441640 1 -- 172.21.15.115:0/1894139689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe020074210 con 0x7fe0200cb6f0 2023-12-01T03:12:15.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.474+0000 7fe015ffb640 1 -- 172.21.15.115:0/1894139689 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fe010013470 con 0x7fe0200cb6f0 2023-12-01T03:12:15.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.475+0000 7fe015ffb640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe00406a670 0x7fe00406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:15.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.475+0000 7fe015ffb640 1 -- 172.21.15.115:0/1894139689 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7477+0+0 (secure 0 0 0) 0x7fe0100899b0 con 0x7fe0200cb6f0 2023-12-01T03:12:15.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.475+0000 7fe0251dd640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe00406a670 0x7fe00406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:15.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.476+0000 7fe0251dd640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe00406a670 0x7fe00406cb30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe01c036000 tx=0x7fe01c03c040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:15.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.479+0000 7fe015ffb640 1 -- 172.21.15.115:0/1894139689 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe010054540 con 0x7fe0200cb6f0 2023-12-01T03:12:15.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:15.656+0000 7fe027441640 1 -- 172.21.15.115:0/1894139689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe0200c9220 con 0x7fe0200cb6f0 2023-12-01T03:12:16.232 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.230+0000 7fe015ffb640 1 -- 172.21.15.115:0/1894139689 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v243) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe010057b60 con 0x7fe0200cb6f0 2023-12-01T03:12:16.233 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.235+0000 7fe027441640 1 -- 172.21.15.115:0/1894139689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe00406a670 msgr2=0x7fe00406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.235+0000 7fe027441640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe00406a670 0x7fe00406cb30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe01c036000 tx=0x7fe01c03c040 comp rx=0 tx=0).stop 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.235+0000 7fe027441640 1 -- 172.21.15.115:0/1894139689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0200cb6f0 msgr2=0x7fe0201583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.235+0000 7fe027441640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0200cb6f0 0x7fe0201583a0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fe010009fd0 tx=0x7fe01000ed40 comp rx=0 tx=0).stop 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.235+0000 7fe027441640 1 -- 172.21.15.115:0/1894139689 shutdown_connections 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.235+0000 7fe027441640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe00406a670 0x7fe00406cb30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.235+0000 7fe027441640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe020159bf0 0x7fe0201517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.235+0000 7fe027441640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0200cb6f0 0x7fe0201583a0 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.235+0000 7fe027441640 1 --2- 172.21.15.115:0/1894139689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0200ca7c0 0x7fe02014e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.235+0000 7fe027441640 1 -- 172.21.15.115:0/1894139689 >> 172.21.15.115:0/1894139689 conn(0x7fe0200662c0 msgr2=0x7fe02014f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.236+0000 7fe027441640 1 -- 172.21.15.115:0/1894139689 shutdown_connections 2023-12-01T03:12:16.238 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.236+0000 7fe027441640 1 -- 172.21.15.115:0/1894139689 wait complete. 2023-12-01T03:12:16.261 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T03:12:16.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.477+0000 7f912df00640 1 Processor -- start 2023-12-01T03:12:16.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.477+0000 7f912df00640 1 -- start start 2023-12-01T03:12:16.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.479+0000 7f912df00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280ca7c0 0x7f91280cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:16.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.479+0000 7f912df00640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280c9440 con 0x7f91280cb6f0 2023-12-01T03:12:16.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.479+0000 7f912df00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280c9720 con 0x7f9128159bf0 2023-12-01T03:12:16.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.479+0000 7f912df00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9128159fd0 con 0x7f91280ca7c0 2023-12-01T03:12:16.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.479+0000 7f9127fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f91280cb6f0 0x7f912806f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:60024/0 (socket says 172.21.15.115:60024) 2023-12-01T03:12:16.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.479+0000 7f9127fff640 1 -- 172.21.15.115:0/1447282345 learned_addr learned my addr 172.21.15.115:0/1447282345 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:16.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.479+0000 7f9126ffd640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280ca7c0 0x7f91280cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:16.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251090644 0 0) 0x7f91280c9440 con 0x7f91280cb6f0 2023-12-01T03:12:16.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9114005730 con 0x7f91280cb6f0 2023-12-01T03:12:16.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4157061119 0 0) 0x7f9114005730 con 0x7f91280cb6f0 2023-12-01T03:12:16.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280ca7c0 msgr2=0x7f91280cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280ca7c0 0x7f91280cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 >> v1:172.21.15.142:6789/0 conn(0x7f9128159bf0 legacy=0x7f912815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280c9a00 con 0x7f91280cb6f0 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f9120001670 con 0x7f91280cb6f0 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 >> v1:172.21.15.115:6789/0 conn(0x7f91280cb6f0 legacy=0x7f912806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280ca7c0 0x7f9114005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9114006130 0x7f91140069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9114006f10 0x7f9114007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911400dc40 con 0x7f9114006130 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911400ddb0 con 0x7f9114006f10 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.480+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911400df70 con 0x7f91280ca7c0 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.481+0000 7f9126ffd640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9114006130 0x7f91140069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.481+0000 7f91277fe640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280ca7c0 0x7f9114005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:16.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.481+0000 7f9127fff640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9114006f10 0x7f9114007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:16.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.481+0000 7f9126ffd640 1 -- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280ca7c0 msgr2=0x7f9114005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:16.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.481+0000 7f9126ffd640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280ca7c0 0x7f9114005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:16.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.481+0000 7f9126ffd640 1 -- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9114006f10 msgr2=0x7f9114007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:16.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.481+0000 7f9126ffd640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9114006f10 0x7f9114007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:16.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.481+0000 7f9126ffd640 1 -- 172.21.15.115:0/1447282345 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9114005730 con 0x7f9114006130 2023-12-01T03:12:16.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.481+0000 7f91277fe640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91280ca7c0 0x7f9114005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:16.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.481+0000 7f9127fff640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9114006f10 0x7f9114007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:16.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.482+0000 7f9126ffd640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9114006130 0x7f91140069d0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f9110009ac0 tx=0x7f9114007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f155b0b79c36c53a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.482+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911002c6b0 con 0x7f9114006130 2023-12-01T03:12:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.482+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911002c810 con 0x7f9114006130 2023-12-01T03:12:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.482+0000 7f91267fc640 1 -- 172.21.15.115:0/1447282345 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911002cba0 con 0x7f9114006130 2023-12-01T03:12:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.483+0000 7f912df00640 1 -- 172.21.15.115:0/1447282345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9114006130 msgr2=0x7f91140069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.483+0000 7f912df00640 1 --2- 172.21.15.115:0/1447282345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9114006130 0x7f91140069d0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f9110009ac0 tx=0x7f9114007a00 comp rx=0 tx=0).stop 2023-12-01T03:12:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.483+0000 7f9127fff640 1 -- 172.21.15.115:0/1447282345 reap_dead start 2023-12-01T03:12:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.483+0000 7f912df00640 1 -- 172.21.15.115:0/1447282345 shutdown_connections 2023-12-01T03:12:16.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.483+0000 7f912df00640 1 -- 172.21.15.115:0/1447282345 >> 172.21.15.115:0/1447282345 conn(0x7f91280662c0 msgr2=0x7f91280b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.483+0000 7f912df00640 1 -- 172.21.15.115:0/1447282345 shutdown_connections 2023-12-01T03:12:16.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.483+0000 7f912df00640 1 -- 172.21.15.115:0/1447282345 wait complete. 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.485+0000 7f912df00640 1 Processor -- start 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.485+0000 7f912df00640 1 -- start start 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f912df00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9114006130 0x7f9128146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f912df00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c29b0 0x7f91280c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f912df00640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f912815c6e0 0x7f912815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f912df00640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912815d0a0 con 0x7f912815c6e0 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f912df00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91280c34b0 con 0x7f91280c29b0 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f912df00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280c3790 con 0x7f9114006130 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f9127fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f912815c6e0 0x7f912815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f9127fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f912815c6e0 0x7f912815cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52968/0 (socket says 172.21.15.115:52968) 2023-12-01T03:12:16.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f9127fff640 1 -- 172.21.15.115:0/2291958419 learned_addr learned my addr 172.21.15.115:0/2291958419 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f9126ffd640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c29b0 0x7f91280c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.486+0000 7f91277fe640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9114006130 0x7f9128146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f9127fff640 1 -- 172.21.15.115:0/2291958419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9114006130 msgr2=0x7f9128146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f9127fff640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9114006130 0x7f9128146f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f9127fff640 1 -- 172.21.15.115:0/2291958419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c29b0 msgr2=0x7f91280c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f9127fff640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c29b0 0x7f91280c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f9127fff640 1 -- 172.21.15.115:0/2291958419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280c3a70 con 0x7f912815c6e0 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f9126ffd640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c29b0 0x7f91280c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f9127fff640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f912815c6e0 0x7f912815cb60 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f91280ce520 tx=0x7f912000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f911ffff640 1 -- 172.21.15.115:0/2291958419 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120007230 con 0x7f912815c6e0 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f911ffff640 1 -- 172.21.15.115:0/2291958419 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91200073b0 con 0x7f912815c6e0 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f911ffff640 1 -- 172.21.15.115:0/2291958419 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91200047e0 con 0x7f912815c6e0 2023-12-01T03:12:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f912df00640 1 -- 172.21.15.115:0/2291958419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91280c3d50 con 0x7f912815c6e0 2023-12-01T03:12:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.487+0000 7f912df00640 1 -- 172.21.15.115:0/2291958419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9128147a80 con 0x7f912815c6e0 2023-12-01T03:12:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.488+0000 7f912df00640 1 -- 172.21.15.115:0/2291958419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91280c4270 con 0x7f912815c6e0 2023-12-01T03:12:16.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.489+0000 7f911ffff640 1 -- 172.21.15.115:0/2291958419 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f9120004940 con 0x7f912815c6e0 2023-12-01T03:12:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.490+0000 7f911ffff640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f910806a670 0x7f910806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.490+0000 7f911ffff640 1 -- 172.21.15.115:0/2291958419 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7039+0+0 (secure 0 0 0) 0x7f9120089140 con 0x7f912815c6e0 2023-12-01T03:12:16.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.491+0000 7f91277fe640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f910806a670 0x7f910806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:16.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.491+0000 7f91277fe640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f910806a670 0x7f910806cb30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9118001d20 tx=0x7f9118009040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:16.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.495+0000 7f911ffff640 1 -- 172.21.15.115:0/2291958419 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f91280c4270 con 0x7f912815c6e0 2023-12-01T03:12:16.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:16.703+0000 7f912df00640 1 -- 172.21.15.115:0/2291958419 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f91280c4270 con 0x7f912815c6e0 2023-12-01T03:12:17.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.234+0000 7f911ffff640 1 -- 172.21.15.115:0/2291958419 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v244) v1 ==== 169+0+0 (secure 0 0 0) 0x7f91280c4270 con 0x7f912815c6e0 2023-12-01T03:12:17.236 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T03:12:17.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.237+0000 7f912df00640 1 -- 172.21.15.115:0/2291958419 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f910806a670 msgr2=0x7f910806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.237+0000 7f912df00640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f910806a670 0x7f910806cb30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9118001d20 tx=0x7f9118009040 comp rx=0 tx=0).stop 2023-12-01T03:12:17.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.237+0000 7f912df00640 1 -- 172.21.15.115:0/2291958419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f912815c6e0 msgr2=0x7f912815cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.239 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.237+0000 7f912df00640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f912815c6e0 0x7f912815cb60 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f91280ce520 tx=0x7f912000dc00 comp rx=0 tx=0).stop 2023-12-01T03:12:17.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.238+0000 7f912df00640 1 -- 172.21.15.115:0/2291958419 shutdown_connections 2023-12-01T03:12:17.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.238+0000 7f912df00640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f910806a670 0x7f910806cb30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.238+0000 7f912df00640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f912815c6e0 0x7f912815cb60 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.238+0000 7f912df00640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91280c29b0 0x7f91280c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.238+0000 7f912df00640 1 --2- 172.21.15.115:0/2291958419 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9114006130 0x7f9128146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.238+0000 7f912df00640 1 -- 172.21.15.115:0/2291958419 >> 172.21.15.115:0/2291958419 conn(0x7f91280662c0 msgr2=0x7f91280ae800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:17.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.238+0000 7f912df00640 1 -- 172.21.15.115:0/2291958419 shutdown_connections 2023-12-01T03:12:17.241 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.238+0000 7f912df00640 1 -- 172.21.15.115:0/2291958419 wait complete. 2023-12-01T03:12:17.262 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:17.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.475+0000 7f803ffff640 1 Processor -- start 2023-12-01T03:12:17.477 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.475+0000 7f803ffff640 1 -- start start 2023-12-01T03:12:17.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.477+0000 7f803ffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f80401584b0 0x7f80400cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.477+0000 7f803ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f804006b980 0x7f80400cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.477+0000 7f803ffff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80400cc180 con 0x7f804015afe0 2023-12-01T03:12:17.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.477+0000 7f803ffff640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804015b550 con 0x7f804006b980 2023-12-01T03:12:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.477+0000 7f803ffff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804015b830 con 0x7f80401584b0 2023-12-01T03:12:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.477+0000 7f803f7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f804006b980 0x7f80400cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.477+0000 7f803f7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f804006b980 0x7f80400cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52976/0 (socket says 172.21.15.115:52976) 2023-12-01T03:12:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.477+0000 7f803f7fe640 1 -- 172.21.15.115:0/2948473526 learned_addr learned my addr 172.21.15.115:0/2948473526 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.477+0000 7f803f7fe640 1 -- 172.21.15.115:0/2948473526 >> v2:172.21.15.142:3300/0 conn(0x7f80401584b0 msgr2=0x7f80400cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.477+0000 7f803effd640 1 --2- 172.21.15.115:0/2948473526 >> v2:172.21.15.142:3300/0 conn(0x7f80401584b0 0x7f80400cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.478+0000 7f803f7fe640 1 --2- 172.21.15.115:0/2948473526 >> v2:172.21.15.142:3300/0 conn(0x7f80401584b0 0x7f80400cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.478+0000 7f803f7fe640 1 -- 172.21.15.115:0/2948473526 >> v1:172.21.15.142:6789/0 conn(0x7f804015afe0 legacy=0x7f804006b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.478+0000 7f803f7fe640 1 -- 172.21.15.115:0/2948473526 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804015bb10 con 0x7f804006b980 2023-12-01T03:12:17.480 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.478+0000 7f803effd640 1 --2- 172.21.15.115:0/2948473526 >> v2:172.21.15.142:3300/0 conn(0x7f80401584b0 0x7f80400cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:17.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.479+0000 7f803f7fe640 1 --2- 172.21.15.115:0/2948473526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f804006b980 0x7f80400cc680 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f804006bfb0 tx=0x7f803802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d449207ec8eaac3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:17.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.479+0000 7f803dffb640 1 -- 172.21.15.115:0/2948473526 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803802ca80 con 0x7f804006b980 2023-12-01T03:12:17.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.479+0000 7f803dffb640 1 -- 172.21.15.115:0/2948473526 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803802cbe0 con 0x7f804006b980 2023-12-01T03:12:17.481 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.479+0000 7f803dffb640 1 -- 172.21.15.115:0/2948473526 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380343f0 con 0x7f804006b980 2023-12-01T03:12:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.480+0000 7f803ffff640 1 -- 172.21.15.115:0/2948473526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f804006b980 msgr2=0x7f80400cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.480+0000 7f803ffff640 1 --2- 172.21.15.115:0/2948473526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f804006b980 0x7f80400cc680 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f804006bfb0 tx=0x7f803802bee0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.480+0000 7f803ffff640 1 -- 172.21.15.115:0/2948473526 shutdown_connections 2023-12-01T03:12:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.480+0000 7f803ffff640 1 --2- 172.21.15.115:0/2948473526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f804006b980 0x7f80400cc680 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.480+0000 7f803ffff640 1 --2- 172.21.15.115:0/2948473526 >> v2:172.21.15.142:3300/0 conn(0x7f80401584b0 0x7f80400cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.480+0000 7f803ffff640 1 -- 172.21.15.115:0/2948473526 >> 172.21.15.115:0/2948473526 conn(0x7f80400ce640 msgr2=0x7f80400cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.480+0000 7f803ffff640 1 -- 172.21.15.115:0/2948473526 shutdown_connections 2023-12-01T03:12:17.482 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.480+0000 7f803ffff640 1 -- 172.21.15.115:0/2948473526 wait complete. 2023-12-01T03:12:17.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.482+0000 7f803ffff640 1 Processor -- start 2023-12-01T03:12:17.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.482+0000 7f803ffff640 1 -- start start 2023-12-01T03:12:17.485 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803ffff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f804006b980 0x7f80400c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80401584b0 0x7f804014c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803ffff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f804015afe0 0x7f8040147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803ffff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401475a0 con 0x7f80401584b0 2023-12-01T03:12:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803ffff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80400c9960 con 0x7f804015afe0 2023-12-01T03:12:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803ffff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80400c9c40 con 0x7f804006b980 2023-12-01T03:12:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80401584b0 0x7f804014c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80401584b0 0x7f804014c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52988/0 (socket says 172.21.15.115:52988) 2023-12-01T03:12:17.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803e7fc640 1 -- 172.21.15.115:0/2807874435 learned_addr learned my addr 172.21.15.115:0/2807874435 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803e7fc640 1 -- 172.21.15.115:0/2807874435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f804006b980 msgr2=0x7f80400c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803e7fc640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f804006b980 0x7f80400c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803e7fc640 1 -- 172.21.15.115:0/2807874435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f804015afe0 msgr2=0x7f8040147060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803f7fe640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f804015afe0 0x7f8040147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803e7fc640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f804015afe0 0x7f8040147060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.484+0000 7f803e7fc640 1 -- 172.21.15.115:0/2807874435 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038009b30 con 0x7f80401584b0 2023-12-01T03:12:17.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.485+0000 7f803e7fc640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80401584b0 0x7f804014c470 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f803400b3f0 tx=0x7f803400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:17.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.485+0000 7f80277fe640 1 -- 172.21.15.115:0/2807874435 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034011830 con 0x7f80401584b0 2023-12-01T03:12:17.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.485+0000 7f80277fe640 1 -- 172.21.15.115:0/2807874435 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034011990 con 0x7f80401584b0 2023-12-01T03:12:17.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.485+0000 7f80277fe640 1 -- 172.21.15.115:0/2807874435 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034011d40 con 0x7f80401584b0 2023-12-01T03:12:17.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.485+0000 7f803ffff640 1 -- 172.21.15.115:0/2807874435 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80400c9f20 con 0x7f80401584b0 2023-12-01T03:12:17.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.485+0000 7f803ffff640 1 -- 172.21.15.115:0/2807874435 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80400ca4e0 con 0x7f80401584b0 2023-12-01T03:12:17.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.486+0000 7f803ffff640 1 -- 172.21.15.115:0/2807874435 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80400673a0 con 0x7f80401584b0 2023-12-01T03:12:17.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.487+0000 7f80277fe640 1 -- 172.21.15.115:0/2807874435 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f803401a400 con 0x7f80401584b0 2023-12-01T03:12:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.488+0000 7f80277fe640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f802c06a6a0 0x7f802c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.488+0000 7f80277fe640 1 -- 172.21.15.115:0/2807874435 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6625+0+0 (secure 0 0 0) 0x7f80340890b0 con 0x7f80401584b0 2023-12-01T03:12:17.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.488+0000 7f803effd640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f802c06a6a0 0x7f802c06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:17.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.489+0000 7f803effd640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f802c06a6a0 0x7f802c06cb60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8030006fd0 tx=0x7f8030008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:17.494 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.493+0000 7f80277fe640 1 -- 172.21.15.115:0/2807874435 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8034053e70 con 0x7f80401584b0 2023-12-01T03:12:17.692 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.690+0000 7f803ffff640 1 -- 172.21.15.115:0/2807874435 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80400ca6a0 con 0x7f80401584b0 2023-12-01T03:12:17.694 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.692+0000 7f80277fe640 1 -- 172.21.15.115:0/2807874435 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v244) v1 ==== 74+0+17435 (secure 0 0 0) 0x7f8034015090 con 0x7f80401584b0 2023-12-01T03:12:17.694 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:17.695 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":244,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:17.226038+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:11:49.229904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:11:50.056441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6835/1837882056":"2023-12-02T03:12:14.201815+0000","172.21.15.142:6834/1837882056":"2023-12-02T03:12:14.201815+0000","172.21.15.115:0/2980649202":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/1025490173":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2408987063":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3619877240":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2405923893":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/957514002":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/638241265":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1722505219":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3713299174":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/4124642007":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6824/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/3863453309":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1160795001":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/988288848":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/524060219":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1801977880":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/203535105":"2023-12-02T03:12:02.677778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.694+0000 7f803ffff640 1 -- 172.21.15.115:0/2807874435 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f802c06a6a0 msgr2=0x7f802c06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.694+0000 7f803ffff640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f802c06a6a0 0x7f802c06cb60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8030006fd0 tx=0x7f8030008040 comp rx=0 tx=0).stop 2023-12-01T03:12:17.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.694+0000 7f803ffff640 1 -- 172.21.15.115:0/2807874435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80401584b0 msgr2=0x7f804014c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.694+0000 7f803ffff640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80401584b0 0x7f804014c470 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f803400b3f0 tx=0x7f803400b8b0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.694+0000 7f803ffff640 1 -- 172.21.15.115:0/2807874435 shutdown_connections 2023-12-01T03:12:17.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.694+0000 7f803ffff640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f802c06a6a0 0x7f802c06cb60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.694+0000 7f803ffff640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f804015afe0 0x7f8040147060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.694+0000 7f803ffff640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80401584b0 0x7f804014c470 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.694+0000 7f803ffff640 1 --2- 172.21.15.115:0/2807874435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f804006b980 0x7f80400c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.694+0000 7f803ffff640 1 -- 172.21.15.115:0/2807874435 >> 172.21.15.115:0/2807874435 conn(0x7f80400ce640 msgr2=0x7f80400b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:17.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.695+0000 7f803ffff640 1 -- 172.21.15.115:0/2807874435 shutdown_connections 2023-12-01T03:12:17.697 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.695+0000 7f803ffff640 1 -- 172.21.15.115:0/2807874435 wait complete. 2023-12-01T03:12:17.715 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-01T03:11:49.229904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '238', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-01T03:11:50.056441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '239', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:12:17.715 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:12:17.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.928+0000 7f2e51b16640 1 Processor -- start 2023-12-01T03:12:17.930 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.928+0000 7f2e51b16640 1 -- start start 2023-12-01T03:12:17.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.930+0000 7f2e51b16640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2e4c15cd90 0x7f2e4c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.930+0000 7f2e51b16640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2e4c0b5fa0 0x7f2e4c161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.930+0000 7f2e51b16640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c0ce5e0 con 0x7f2e4c0cf650 2023-12-01T03:12:17.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.930+0000 7f2e51b16640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c0d0ec0 con 0x7f2e4c15cd90 2023-12-01T03:12:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.930+0000 7f2e51b16640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c161950 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.931+0000 7f2e4b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2e4c15cd90 0x7f2e4c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.931+0000 7f2e4b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2e4c15cd90 0x7f2e4c0cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54094/0 (socket says 172.21.15.115:54094) 2023-12-01T03:12:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.931+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/2768349924 learned_addr learned my addr 172.21.15.115:0/2768349924 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:17.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.931+0000 7f2e4affd640 1 --2- 172.21.15.115:0/2768349924 >> v2:172.21.15.142:3300/0 conn(0x7f2e4c0b5fa0 0x7f2e4c161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:17.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.932+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/2768349924 >> v2:172.21.15.142:3300/0 conn(0x7f2e4c0b5fa0 msgr2=0x7f2e4c161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:17.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.932+0000 7f2e4b7fe640 1 --2- 172.21.15.115:0/2768349924 >> v2:172.21.15.142:3300/0 conn(0x7f2e4c0b5fa0 0x7f2e4c161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.932+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/2768349924 >> v1:172.21.15.142:6789/0 conn(0x7f2e4c0cf650 legacy=0x7f2e4c0cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.932+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/2768349924 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c0ce300 con 0x7f2e4c15cd90 2023-12-01T03:12:17.934 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.932+0000 7f2e4a7fc640 1 -- 172.21.15.115:0/2768349924 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055693230 0 0) 0x7f2e44001490 con 0x7f2e4c0cf650 2023-12-01T03:12:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.934+0000 7f2e4b7fe640 1 --2- 172.21.15.115:0/2768349924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c15cd90 0x7f2e4c0cdd20 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0d0760 tx=0x7f2e3c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b25c3480bf449f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.934+0000 7f2e4a7fc640 1 -- 172.21.15.115:0/2768349924 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c030860 con 0x7f2e4c15cd90 2023-12-01T03:12:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.935+0000 7f2e4a7fc640 1 -- 172.21.15.115:0/2768349924 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c0309c0 con 0x7f2e4c15cd90 2023-12-01T03:12:17.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.935+0000 7f2e4a7fc640 1 -- 172.21.15.115:0/2768349924 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c030d50 con 0x7f2e4c15cd90 2023-12-01T03:12:17.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.935+0000 7f2e51b16640 1 -- 172.21.15.115:0/2768349924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c15cd90 msgr2=0x7f2e4c0cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.935+0000 7f2e51b16640 1 --2- 172.21.15.115:0/2768349924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c15cd90 0x7f2e4c0cdd20 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0d0760 tx=0x7f2e3c02b9a0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.935+0000 7f2e51b16640 1 -- 172.21.15.115:0/2768349924 shutdown_connections 2023-12-01T03:12:17.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.935+0000 7f2e51b16640 1 --2- 172.21.15.115:0/2768349924 >> v2:172.21.15.142:3300/0 conn(0x7f2e4c0b5fa0 0x7f2e4c161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.935+0000 7f2e51b16640 1 --2- 172.21.15.115:0/2768349924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c15cd90 0x7f2e4c0cdd20 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.936+0000 7f2e51b16640 1 -- 172.21.15.115:0/2768349924 >> 172.21.15.115:0/2768349924 conn(0x7f2e4c065960 msgr2=0x7f2e4c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:17.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.936+0000 7f2e51b16640 1 -- 172.21.15.115:0/2768349924 shutdown_connections 2023-12-01T03:12:17.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.936+0000 7f2e51b16640 1 -- 172.21.15.115:0/2768349924 wait complete. 2023-12-01T03:12:17.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.937+0000 7f2e51b16640 1 Processor -- start 2023-12-01T03:12:17.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.938+0000 7f2e51b16640 1 -- start start 2023-12-01T03:12:17.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e51b16640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c0b5fa0 0x7f2e4c1f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e51b16640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e4c0cf650 0x7f2e4c0abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e51b16640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c15cd90 0x7f2e4c0ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e51b16640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c158a60 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e51b16640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c15de30 con 0x7f2e4c0cf650 2023-12-01T03:12:17.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e51b16640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c15e110 con 0x7f2e4c15cd90 2023-12-01T03:12:17.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e4b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c0b5fa0 0x7f2e4c1f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:17.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e4b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c0b5fa0 0x7f2e4c1f33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54104/0 (socket says 172.21.15.115:54104) 2023-12-01T03:12:17.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/3157440825 learned_addr learned my addr 172.21.15.115:0/3157440825 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:17.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e4affd640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e4c0cf650 0x7f2e4c0abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:17.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/3157440825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c15cd90 msgr2=0x7f2e4c0ac1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e4b7fe640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c15cd90 0x7f2e4c0ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/3157440825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e4c0cf650 msgr2=0x7f2e4c0abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e4b7fe640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e4c0cf650 0x7f2e4c0abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.939+0000 7f2e4b7fe640 1 -- 172.21.15.115:0/3157440825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c0094d0 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.940+0000 7f2e4b7fe640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c0b5fa0 0x7f2e4c1f33b0 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c000c00 tx=0x7f2e3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.940+0000 7f2e33fff640 1 -- 172.21.15.115:0/3157440825 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c039a90 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.940+0000 7f2e33fff640 1 -- 172.21.15.115:0/3157440825 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c039bf0 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.940+0000 7f2e33fff640 1 -- 172.21.15.115:0/3157440825 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0433f0 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.940+0000 7f2e51b16640 1 -- 172.21.15.115:0/3157440825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c15e3f0 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.940+0000 7f2e51b16640 1 -- 172.21.15.115:0/3157440825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c15e9b0 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.942+0000 7f2e51b16640 1 -- 172.21.15.115:0/3157440825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c15eb40 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.942+0000 7f2e33fff640 1 -- 172.21.15.115:0/3157440825 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f2e3c02f3b0 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.943+0000 7f2e33fff640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2e3806a670 0x7f2e3806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:17.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.943+0000 7f2e33fff640 1 -- 172.21.15.115:0/3157440825 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6625+0+0 (secure 0 0 0) 0x7f2e3c0ab400 con 0x7f2e4c0b5fa0 2023-12-01T03:12:17.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.943+0000 7f2e4affd640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2e3806a670 0x7f2e3806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:17.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.944+0000 7f2e4affd640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2e3806a670 0x7f2e3806cb30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c158fd0 tx=0x7f2e40008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:17.949 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:17.948+0000 7f2e33fff640 1 -- 172.21.15.115:0/3157440825 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2e3c076300 con 0x7f2e4c0b5fa0 2023-12-01T03:12:18.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.136+0000 7f2e51b16640 1 -- 172.21.15.115:0/3157440825 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e4c04e890 con 0x7f2e4c0b5fa0 2023-12-01T03:12:18.139 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.137+0000 7f2e33fff640 1 -- 172.21.15.115:0/3157440825 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v244) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e3c079920 con 0x7f2e4c0b5fa0 2023-12-01T03:12:18.139 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:12:18.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 -- 172.21.15.115:0/3157440825 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2e3806a670 msgr2=0x7f2e3806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2e3806a670 0x7f2e3806cb30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c158fd0 tx=0x7f2e40008040 comp rx=0 tx=0).stop 2023-12-01T03:12:18.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 -- 172.21.15.115:0/3157440825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c0b5fa0 msgr2=0x7f2e4c1f33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c0b5fa0 0x7f2e4c1f33b0 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c000c00 tx=0x7f2e3c004730 comp rx=0 tx=0).stop 2023-12-01T03:12:18.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 -- 172.21.15.115:0/3157440825 shutdown_connections 2023-12-01T03:12:18.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2e3806a670 0x7f2e3806cb30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2e4c15cd90 0x7f2e4c0ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e4c0cf650 0x7f2e4c0abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 --2- 172.21.15.115:0/3157440825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e4c0b5fa0 0x7f2e4c1f33b0 unknown :-1 s=CLOSED pgs=2879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 -- 172.21.15.115:0/3157440825 >> 172.21.15.115:0/3157440825 conn(0x7f2e4c065960 msgr2=0x7f2e4c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:18.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 -- 172.21.15.115:0/3157440825 shutdown_connections 2023-12-01T03:12:18.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.140+0000 7f2e51b16640 1 -- 172.21.15.115:0/3157440825 wait complete. 2023-12-01T03:12:18.161 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-01T03:12:18.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.366+0000 7f2961666640 1 Processor -- start 2023-12-01T03:12:18.368 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.367+0000 7f2961666640 1 -- start start 2023-12-01T03:12:18.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.368+0000 7f2961666640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c0cfd60 0x7f295c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.368+0000 7f2961666640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295c0ce5e0 con 0x7f295c0b5fa0 2023-12-01T03:12:18.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.368+0000 7f2961666640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295c0ce300 con 0x7f295c15cd90 2023-12-01T03:12:18.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.368+0000 7f2961666640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c0d0ec0 con 0x7f295c0cfd60 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.368+0000 7f295b7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f295c0b5fa0 0x7f295c0d1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37740/0 (socket says 172.21.15.115:37740) 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.368+0000 7f295b7fe640 1 -- 172.21.15.115:0/3189702397 learned_addr learned my addr 172.21.15.115:0/3189702397 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f295a7fc640 1 --2- 172.21.15.115:0/3189702397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c0cfd60 0x7f295c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756793797 0 0) 0x7f2954001110 con 0x7f295c0b5fa0 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2948003520 con 0x7f295c0b5fa0 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 994260957 0 0) 0x7f2948003520 con 0x7f295c0b5fa0 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c0cfd60 msgr2=0x7f295c0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 --2- 172.21.15.115:0/3189702397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c0cfd60 0x7f295c0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 >> v1:172.21.15.142:6789/0 conn(0x7f295c15cd90 legacy=0x7f295c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c0d10a0 con 0x7f295c0b5fa0 2023-12-01T03:12:18.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f2954001800 con 0x7f295c0b5fa0 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 >> v1:172.21.15.115:6789/0 conn(0x7f295c0b5fa0 legacy=0x7f295c0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 --2- 172.21.15.115:0/3189702397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 0x7f2948003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 --2- 172.21.15.115:0/3189702397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c0cfd60 0x7f29480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 --2- 172.21.15.115:0/3189702397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2948004d00 0x7f2948005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c0d0ec0 con 0x7f29480039a0 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294800bab0 con 0x7f2948004d00 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.369+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294800bc70 con 0x7f295c0cfd60 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.370+0000 7f295affd640 1 --2- 172.21.15.115:0/3189702397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 0x7f2948003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.370+0000 7f295affd640 1 -- 172.21.15.115:0/3189702397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c0cfd60 msgr2=0x7f29480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.370+0000 7f295affd640 1 --2- 172.21.15.115:0/3189702397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c0cfd60 0x7f29480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.370+0000 7f295affd640 1 -- 172.21.15.115:0/3189702397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2948004d00 msgr2=0x7f2948005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.370+0000 7f295affd640 1 --2- 172.21.15.115:0/3189702397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2948004d00 0x7f2948005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.370+0000 7f295affd640 1 -- 172.21.15.115:0/3189702397 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2948003520 con 0x7f29480039a0 2023-12-01T03:12:18.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.371+0000 7f295affd640 1 --2- 172.21.15.115:0/3189702397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 0x7f2948003dc0 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f294c001e50 tx=0x7f29480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d24ce6b424613b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:18.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.371+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c003b70 con 0x7f29480039a0 2023-12-01T03:12:18.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.371+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c003cd0 con 0x7f29480039a0 2023-12-01T03:12:18.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.371+0000 7f2959ffb640 1 -- 172.21.15.115:0/3189702397 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c005310 con 0x7f29480039a0 2023-12-01T03:12:18.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.372+0000 7f2961666640 1 -- 172.21.15.115:0/3189702397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 msgr2=0x7f2948003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.372+0000 7f2961666640 1 --2- 172.21.15.115:0/3189702397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 0x7f2948003dc0 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f294c001e50 tx=0x7f29480057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.376 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.372+0000 7f295b7fe640 1 -- 172.21.15.115:0/3189702397 reap_dead start 2023-12-01T03:12:18.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.372+0000 7f2961666640 1 -- 172.21.15.115:0/3189702397 shutdown_connections 2023-12-01T03:12:18.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.372+0000 7f2961666640 1 -- 172.21.15.115:0/3189702397 >> 172.21.15.115:0/3189702397 conn(0x7f295c065960 msgr2=0x7f295c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:18.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.372+0000 7f2961666640 1 -- 172.21.15.115:0/3189702397 shutdown_connections 2023-12-01T03:12:18.377 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.372+0000 7f2961666640 1 -- 172.21.15.115:0/3189702397 wait complete. 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.374+0000 7f2961666640 1 Processor -- start 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.374+0000 7f2961666640 1 -- start start 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.375+0000 7f2961666640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 0x7f295c15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.375+0000 7f2961666640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295c0cc860 0x7f295c158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.375+0000 7f2961666640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c14e2a0 0x7f295c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.375+0000 7f2961666640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c158d40 con 0x7f29480039a0 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.375+0000 7f2961666640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c15a450 con 0x7f295c0cc860 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.375+0000 7f2961666640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c15a730 con 0x7f295c14e2a0 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.375+0000 7f295affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 0x7f295c15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.375+0000 7f295affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 0x7f295c15b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54128/0 (socket says 172.21.15.115:54128) 2023-12-01T03:12:18.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.375+0000 7f295affd640 1 -- 172.21.15.115:0/2318844787 learned_addr learned my addr 172.21.15.115:0/2318844787 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f295affd640 1 -- 172.21.15.115:0/2318844787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c14e2a0 msgr2=0x7f295c15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f295a7fc640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295c0cc860 0x7f295c158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f295affd640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c14e2a0 0x7f295c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f295affd640 1 -- 172.21.15.115:0/2318844787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295c0cc860 msgr2=0x7f295c158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f295affd640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295c0cc860 0x7f295c158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f295affd640 1 -- 172.21.15.115:0/2318844787 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c15aa10 con 0x7f29480039a0 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f295affd640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 0x7f295c15b4f0 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f294c02bee0 tx=0x7f294c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f29537fe640 1 -- 172.21.15.115:0/2318844787 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c005850 con 0x7f29480039a0 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f29537fe640 1 -- 172.21.15.115:0/2318844787 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c003e40 con 0x7f29480039a0 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f29537fe640 1 -- 172.21.15.115:0/2318844787 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c007230 con 0x7f29480039a0 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f2961666640 1 -- 172.21.15.115:0/2318844787 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c15acf0 con 0x7f29480039a0 2023-12-01T03:12:18.380 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.376+0000 7f2961666640 1 -- 172.21.15.115:0/2318844787 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c14e720 con 0x7f29480039a0 2023-12-01T03:12:18.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.378+0000 7f2961666640 1 -- 172.21.15.115:0/2318844787 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295c04e890 con 0x7f29480039a0 2023-12-01T03:12:18.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.378+0000 7f29537fe640 1 -- 172.21.15.115:0/2318844787 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f294c007390 con 0x7f29480039a0 2023-12-01T03:12:18.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.379+0000 7f29537fe640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f293c06a6a0 0x7f293c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.381 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.380+0000 7f29537fe640 1 -- 172.21.15.115:0/2318844787 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6625+0+0 (secure 0 0 0) 0x7f294c0ab760 con 0x7f29480039a0 2023-12-01T03:12:18.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.380+0000 7f295a7fc640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f293c06a6a0 0x7f293c06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.384 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.381+0000 7f295a7fc640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f293c06a6a0 0x7f293c06cb60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f295c1515f0 tx=0x7f2944008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:18.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.389+0000 7f29537fe640 1 -- 172.21.15.115:0/2318844787 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f294c076530 con 0x7f29480039a0 2023-12-01T03:12:18.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.579+0000 7f2961666640 1 -- 172.21.15.115:0/2318844787 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f295c158d40 con 0x7f29480039a0 2023-12-01T03:12:18.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.580+0000 7f29537fe640 1 -- 172.21.15.115:0/2318844787 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v244) v1 ==== 107+0+11 (secure 0 0 0) 0x7f294c079b50 con 0x7f29480039a0 2023-12-01T03:12:18.582 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:12:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.582+0000 7f2961666640 1 -- 172.21.15.115:0/2318844787 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f293c06a6a0 msgr2=0x7f293c06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.582+0000 7f2961666640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f293c06a6a0 0x7f293c06cb60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f295c1515f0 tx=0x7f2944008040 comp rx=0 tx=0).stop 2023-12-01T03:12:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.582+0000 7f2961666640 1 -- 172.21.15.115:0/2318844787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 msgr2=0x7f295c15b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.582+0000 7f2961666640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 0x7f295c15b4f0 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f294c02bee0 tx=0x7f294c0014a0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.583+0000 7f2961666640 1 -- 172.21.15.115:0/2318844787 shutdown_connections 2023-12-01T03:12:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.583+0000 7f2961666640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f293c06a6a0 0x7f293c06cb60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.583+0000 7f2961666640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f295c14e2a0 0x7f295c15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.583+0000 7f2961666640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295c0cc860 0x7f295c158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.583+0000 7f2961666640 1 --2- 172.21.15.115:0/2318844787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29480039a0 0x7f295c15b4f0 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.583+0000 7f2961666640 1 -- 172.21.15.115:0/2318844787 >> 172.21.15.115:0/2318844787 conn(0x7f295c065960 msgr2=0x7f295c155d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.583+0000 7f2961666640 1 -- 172.21.15.115:0/2318844787 shutdown_connections 2023-12-01T03:12:18.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.583+0000 7f2961666640 1 -- 172.21.15.115:0/2318844787 wait complete. 2023-12-01T03:12:18.604 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-01T03:12:18.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.805+0000 7fd5117d6640 1 Processor -- start 2023-12-01T03:12:18.807 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.805+0000 7fd5117d6640 1 -- start start 2023-12-01T03:12:18.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.807+0000 7fd5117d6640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd50c15ac00 0x7fd50c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.807+0000 7fd5117d6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd50c0cb800 0x7fd50c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.807+0000 7fd5117d6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c15afe0 con 0x7fd50c0aede0 2023-12-01T03:12:18.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.807+0000 7fd5117d6640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c0b12b0 con 0x7fd50c15ac00 2023-12-01T03:12:18.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.807+0000 7fd5117d6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c0b0a10 con 0x7fd50c0cb800 2023-12-01T03:12:18.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.807+0000 7fd50bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd50c15ac00 0x7fd50c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.807+0000 7fd50bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd50c15ac00 0x7fd50c0ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54132/0 (socket says 172.21.15.115:54132) 2023-12-01T03:12:18.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.807+0000 7fd50bfff640 1 -- 172.21.15.115:0/2150310084 learned_addr learned my addr 172.21.15.115:0/2150310084 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:18.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.807+0000 7fd510fd5640 1 --2- 172.21.15.115:0/2150310084 >> v2:172.21.15.142:3300/0 conn(0x7fd50c0cb800 0x7fd50c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.808+0000 7fd50bfff640 1 -- 172.21.15.115:0/2150310084 >> v2:172.21.15.142:3300/0 conn(0x7fd50c0cb800 msgr2=0x7fd50c06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:18.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.808+0000 7fd50bfff640 1 --2- 172.21.15.115:0/2150310084 >> v2:172.21.15.142:3300/0 conn(0x7fd50c0cb800 0x7fd50c06fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.808+0000 7fd50bfff640 1 -- 172.21.15.115:0/2150310084 >> v1:172.21.15.142:6789/0 conn(0x7fd50c0aede0 legacy=0x7fd50c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:18.810 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.808+0000 7fd50bfff640 1 -- 172.21.15.115:0/2150310084 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0b0cf0 con 0x7fd50c15ac00 2023-12-01T03:12:18.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.809+0000 7fd50bfff640 1 --2- 172.21.15.115:0/2150310084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c15ac00 0x7fd50c0ae8a0 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fd50c066750 tx=0x7fd4fc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687ba35178519d7a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:18.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.809+0000 7fd50b7fe640 1 -- 172.21.15.115:0/2150310084 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc0317e0 con 0x7fd50c15ac00 2023-12-01T03:12:18.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.809+0000 7fd50b7fe640 1 -- 172.21.15.115:0/2150310084 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc031940 con 0x7fd50c15ac00 2023-12-01T03:12:18.811 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.809+0000 7fd50b7fe640 1 -- 172.21.15.115:0/2150310084 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc031cd0 con 0x7fd50c15ac00 2023-12-01T03:12:18.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.810+0000 7fd5117d6640 1 -- 172.21.15.115:0/2150310084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c15ac00 msgr2=0x7fd50c0ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.810+0000 7fd5117d6640 1 --2- 172.21.15.115:0/2150310084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c15ac00 0x7fd50c0ae8a0 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fd50c066750 tx=0x7fd4fc02b880 comp rx=0 tx=0).stop 2023-12-01T03:12:18.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.810+0000 7fd5117d6640 1 -- 172.21.15.115:0/2150310084 shutdown_connections 2023-12-01T03:12:18.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.810+0000 7fd5117d6640 1 --2- 172.21.15.115:0/2150310084 >> v2:172.21.15.142:3300/0 conn(0x7fd50c0cb800 0x7fd50c06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.810+0000 7fd5117d6640 1 --2- 172.21.15.115:0/2150310084 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c15ac00 0x7fd50c0ae8a0 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.810+0000 7fd5117d6640 1 -- 172.21.15.115:0/2150310084 >> 172.21.15.115:0/2150310084 conn(0x7fd50c065960 msgr2=0x7fd50c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:18.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.810+0000 7fd5117d6640 1 -- 172.21.15.115:0/2150310084 shutdown_connections 2023-12-01T03:12:18.812 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.810+0000 7fd5117d6640 1 -- 172.21.15.115:0/2150310084 wait complete. 2023-12-01T03:12:18.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.812+0000 7fd5117d6640 1 Processor -- start 2023-12-01T03:12:18.813 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.812+0000 7fd5117d6640 1 -- start start 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd5117d6640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c0aede0 0x7fd50c156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd5117d6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd50c0cb800 0x7fd50c0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd5117d6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50c15ac00 0x7fd50c0c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd5117d6640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0c8d80 con 0x7fd50c0aede0 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd5117d6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0c95e0 con 0x7fd50c0cb800 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd5117d6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c1573b0 con 0x7fd50c15ac00 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd50bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c0aede0 0x7fd50c156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd50bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c0aede0 0x7fd50c156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54146/0 (socket says 172.21.15.115:54146) 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd50bfff640 1 -- 172.21.15.115:0/706174918 learned_addr learned my addr 172.21.15.115:0/706174918 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd510fd5640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50c15ac00 0x7fd50c0c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd5037fe640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd50c0cb800 0x7fd50c0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd50bfff640 1 -- 172.21.15.115:0/706174918 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50c15ac00 msgr2=0x7fd50c0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd50bfff640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50c15ac00 0x7fd50c0c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd50bfff640 1 -- 172.21.15.115:0/706174918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd50c0cb800 msgr2=0x7fd50c0c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd50bfff640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd50c0cb800 0x7fd50c0c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd50bfff640 1 -- 172.21.15.115:0/706174918 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc0094d0 con 0x7fd50c0aede0 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd510fd5640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50c15ac00 0x7fd50c0c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.813+0000 7fd5037fe640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd50c0cb800 0x7fd50c0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.814+0000 7fd50bfff640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c0aede0 0x7fd50c156e70 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc02b8c0 tx=0x7fd4fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.814+0000 7fd5097fa640 1 -- 172.21.15.115:0/706174918 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc032880 con 0x7fd50c0aede0 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.814+0000 7fd5097fa640 1 -- 172.21.15.115:0/706174918 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc032a00 con 0x7fd50c0aede0 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.814+0000 7fd5117d6640 1 -- 172.21.15.115:0/706174918 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c15df10 con 0x7fd50c0aede0 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.814+0000 7fd5097fa640 1 -- 172.21.15.115:0/706174918 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc032db0 con 0x7fd50c0aede0 2023-12-01T03:12:18.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.814+0000 7fd5117d6640 1 -- 172.21.15.115:0/706174918 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c0c9270 con 0x7fd50c0aede0 2023-12-01T03:12:18.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.815+0000 7fd5117d6640 1 -- 172.21.15.115:0/706174918 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c0618d0 con 0x7fd50c0aede0 2023-12-01T03:12:18.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.816+0000 7fd5097fa640 1 -- 172.21.15.115:0/706174918 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fd4fc04d3f0 con 0x7fd50c0aede0 2023-12-01T03:12:18.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.817+0000 7fd5097fa640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd4ec06a6a0 0x7fd4ec06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:18.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.817+0000 7fd5097fa640 1 -- 172.21.15.115:0/706174918 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6625+0+0 (secure 0 0 0) 0x7fd4fc0aaf60 con 0x7fd50c0aede0 2023-12-01T03:12:18.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.818+0000 7fd5037fe640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd4ec06a6a0 0x7fd4ec06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:18.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.818+0000 7fd5037fe640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd4ec06a6a0 0x7fd4ec06cb60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0c8bf0 tx=0x7fd4f4009040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:18.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:18.822+0000 7fd5097fa640 1 -- 172.21.15.115:0/706174918 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd4fc075df0 con 0x7fd50c0aede0 2023-12-01T03:12:19.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.002+0000 7fd5117d6640 1 -- 172.21.15.115:0/706174918 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fd50c070120 con 0x7fd50c0aede0 2023-12-01T03:12:19.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.003+0000 7fd5097fa640 1 -- 172.21.15.115:0/706174918 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v244) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd4fc079410 con 0x7fd50c0aede0 2023-12-01T03:12:19.005 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:12:19.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 -- 172.21.15.115:0/706174918 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd4ec06a6a0 msgr2=0x7fd4ec06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd4ec06a6a0 0x7fd4ec06cb60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0c8bf0 tx=0x7fd4f4009040 comp rx=0 tx=0).stop 2023-12-01T03:12:19.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 -- 172.21.15.115:0/706174918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c0aede0 msgr2=0x7fd50c156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c0aede0 0x7fd50c156e70 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc02b8c0 tx=0x7fd4fc0046c0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 -- 172.21.15.115:0/706174918 shutdown_connections 2023-12-01T03:12:19.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd4ec06a6a0 0x7fd4ec06cb60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50c15ac00 0x7fd50c0c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd50c0cb800 0x7fd50c0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 --2- 172.21.15.115:0/706174918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd50c0aede0 0x7fd50c156e70 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 -- 172.21.15.115:0/706174918 >> 172.21.15.115:0/706174918 conn(0x7fd50c065960 msgr2=0x7fd50c0cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:19.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 -- 172.21.15.115:0/706174918 shutdown_connections 2023-12-01T03:12:19.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.006+0000 7fd5117d6640 1 -- 172.21.15.115:0/706174918 wait complete. 2023-12-01T03:12:19.027 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:12:19.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.238+0000 7f1f8c887640 1 Processor -- start 2023-12-01T03:12:19.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.239+0000 7f1f8c887640 1 -- start start 2023-12-01T03:12:19.242 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.240+0000 7f1f8c887640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1f88145fa0 0x7f1f88065960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.240+0000 7f1f8c887640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f881609a0 con 0x7f1f88065ea0 2023-12-01T03:12:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.241+0000 7f1f8c887640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f88160c80 con 0x7f1f88071e30 2023-12-01T03:12:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.241+0000 7f1f8c887640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f88161620 con 0x7f1f88145fa0 2023-12-01T03:12:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.241+0000 7f1f87fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f1f88065ea0 0x7f1f880701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37748/0 (socket says 172.21.15.115:37748) 2023-12-01T03:12:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.241+0000 7f1f87fff640 1 -- 172.21.15.115:0/4246664290 learned_addr learned my addr 172.21.15.115:0/4246664290 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:19.243 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.241+0000 7f1f86ffd640 1 --2- 172.21.15.115:0/4246664290 >> v2:172.21.15.142:3300/0 conn(0x7f1f88145fa0 0x7f1f88065960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.242+0000 7f1f86ffd640 1 -- 172.21.15.115:0/4246664290 >> v1:172.21.15.142:6789/0 conn(0x7f1f88071e30 legacy=0x7f1f880b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.242+0000 7f1f86ffd640 1 -- 172.21.15.115:0/4246664290 >> v1:172.21.15.115:6789/0 conn(0x7f1f88065ea0 legacy=0x7f1f880701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.244 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.242+0000 7f1f86ffd640 1 -- 172.21.15.115:0/4246664290 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f88160f60 con 0x7f1f88145fa0 2023-12-01T03:12:19.245 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.244+0000 7f1f86ffd640 1 --2- 172.21.15.115:0/4246664290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f88145fa0 0x7f1f88065960 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1f700097a0 tx=0x7f1f7002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba4fadfca1130b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:19.249 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.247+0000 7f1f867fc640 1 -- 172.21.15.115:0/4246664290 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70031830 con 0x7f1f88145fa0 2023-12-01T03:12:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.247+0000 7f1f867fc640 1 -- 172.21.15.115:0/4246664290 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70031990 con 0x7f1f88145fa0 2023-12-01T03:12:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.247+0000 7f1f867fc640 1 -- 172.21.15.115:0/4246664290 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70031d20 con 0x7f1f88145fa0 2023-12-01T03:12:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.248+0000 7f1f8c887640 1 -- 172.21.15.115:0/4246664290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f88145fa0 msgr2=0x7f1f88065960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.248+0000 7f1f8c887640 1 --2- 172.21.15.115:0/4246664290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f88145fa0 0x7f1f88065960 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1f700097a0 tx=0x7f1f7002b8d0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.248+0000 7f1f8c887640 1 -- 172.21.15.115:0/4246664290 shutdown_connections 2023-12-01T03:12:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.248+0000 7f1f8c887640 1 --2- 172.21.15.115:0/4246664290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f88145fa0 0x7f1f88065960 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.248+0000 7f1f8c887640 1 -- 172.21.15.115:0/4246664290 >> 172.21.15.115:0/4246664290 conn(0x7f1f88073330 msgr2=0x7f1f88069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.248+0000 7f1f8c887640 1 -- 172.21.15.115:0/4246664290 shutdown_connections 2023-12-01T03:12:19.250 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.248+0000 7f1f8c887640 1 -- 172.21.15.115:0/4246664290 wait complete. 2023-12-01T03:12:19.251 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.250+0000 7f1f8c887640 1 Processor -- start 2023-12-01T03:12:19.252 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.250+0000 7f1f8c887640 1 -- start start 2023-12-01T03:12:19.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.251+0000 7f1f8c887640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f88065ea0 0x7f1f8813e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.254 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.251+0000 7f1f8c887640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f88071e30 0x7f1f881e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.251+0000 7f1f8c887640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f88145fa0 0x7f1f880ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.251+0000 7f1f8c887640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88066350 con 0x7f1f88071e30 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.251+0000 7f1f8c887640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8813f280 con 0x7f1f88145fa0 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.251+0000 7f1f8c887640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f881e7020 con 0x7f1f88065ea0 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.251+0000 7f1f86ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f88071e30 0x7f1f881e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f86ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f88071e30 0x7f1f881e66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54148/0 (socket says 172.21.15.115:54148) 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f86ffd640 1 -- 172.21.15.115:0/1022063162 learned_addr learned my addr 172.21.15.115:0/1022063162 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f877fe640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f88065ea0 0x7f1f8813e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f87fff640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f88145fa0 0x7f1f880ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f86ffd640 1 -- 172.21.15.115:0/1022063162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f88065ea0 msgr2=0x7f1f8813e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f86ffd640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f88065ea0 0x7f1f8813e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f86ffd640 1 -- 172.21.15.115:0/1022063162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f88145fa0 msgr2=0x7f1f880ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f86ffd640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f88145fa0 0x7f1f880ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f86ffd640 1 -- 172.21.15.115:0/1022063162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f70009400 con 0x7f1f88071e30 2023-12-01T03:12:19.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f877fe640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f88065ea0 0x7f1f8813e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:19.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f87fff640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f88145fa0 0x7f1f880ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:19.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.252+0000 7f1f86ffd640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f88071e30 0x7f1f881e66d0 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f1f70034040 tx=0x7f1f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:19.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.253+0000 7f1f7ffff640 1 -- 172.21.15.115:0/1022063162 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7003aa60 con 0x7f1f88071e30 2023-12-01T03:12:19.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.253+0000 7f1f7ffff640 1 -- 172.21.15.115:0/1022063162 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7003abe0 con 0x7f1f88071e30 2023-12-01T03:12:19.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.253+0000 7f1f7ffff640 1 -- 172.21.15.115:0/1022063162 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70044400 con 0x7f1f88071e30 2023-12-01T03:12:19.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.253+0000 7f1f8c887640 1 -- 172.21.15.115:0/1022063162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f881e7300 con 0x7f1f88071e30 2023-12-01T03:12:19.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.253+0000 7f1f8c887640 1 -- 172.21.15.115:0/1022063162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8813f800 con 0x7f1f88071e30 2023-12-01T03:12:19.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.254+0000 7f1f8c887640 1 -- 172.21.15.115:0/1022063162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f881e7540 con 0x7f1f88071e30 2023-12-01T03:12:19.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.255+0000 7f1f7ffff640 1 -- 172.21.15.115:0/1022063162 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f1f7004d440 con 0x7f1f88071e30 2023-12-01T03:12:19.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.256+0000 7f1f7ffff640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f6806a6a0 0x7f1f6806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.256+0000 7f1f7ffff640 1 -- 172.21.15.115:0/1022063162 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6625+0+0 (secure 0 0 0) 0x7f1f700ab390 con 0x7f1f88071e30 2023-12-01T03:12:19.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.256+0000 7f1f877fe640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f6806a6a0 0x7f1f6806cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:19.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.257+0000 7f1f877fe640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f6806a6a0 0x7f1f6806cb60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f780012e0 tx=0x7f1f78009040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:19.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.261+0000 7f1f7ffff640 1 -- 172.21.15.115:0/1022063162 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1f881e7540 con 0x7f1f88071e30 2023-12-01T03:12:19.474 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.472+0000 7f1f8c887640 1 -- 172.21.15.115:0/1022063162 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f881e7540 con 0x7f1f88071e30 2023-12-01T03:12:19.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.474+0000 7f1f7ffff640 1 -- 172.21.15.115:0/1022063162 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4658 (secure 0 0 0) 0x7f1f881e7540 con 0x7f1f88071e30 2023-12-01T03:12:19.476 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:19.476 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":118,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/993295369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":993295369},{"type":"v1","addr":"172.21.15.142:6835","nonce":993295369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:51.037525+0000","modified":"2023-12-01T03:11:54.055433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7732":{"gid":7732,"name":"b","rank":0,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-01T03:12:19.476 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 118 2023-12-01T03:12:19.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 -- 172.21.15.115:0/1022063162 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f6806a6a0 msgr2=0x7f1f6806cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f6806a6a0 0x7f1f6806cb60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f780012e0 tx=0x7f1f78009040 comp rx=0 tx=0).stop 2023-12-01T03:12:19.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 -- 172.21.15.115:0/1022063162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f88071e30 msgr2=0x7f1f881e66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f88071e30 0x7f1f881e66d0 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f1f70034040 tx=0x7f1f700046c0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 -- 172.21.15.115:0/1022063162 shutdown_connections 2023-12-01T03:12:19.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1f6806a6a0 0x7f1f6806cb60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.478 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f88145fa0 0x7f1f880ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f88071e30 0x7f1f881e66d0 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 --2- 172.21.15.115:0/1022063162 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f88065ea0 0x7f1f8813e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 -- 172.21.15.115:0/1022063162 >> 172.21.15.115:0/1022063162 conn(0x7f1f88073330 msgr2=0x7f1f88069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.476+0000 7f1f8c887640 1 -- 172.21.15.115:0/1022063162 shutdown_connections 2023-12-01T03:12:19.479 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.477+0000 7f1f8c887640 1 -- 172.21.15.115:0/1022063162 wait complete. 2023-12-01T03:12:19.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:19.700 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.698+0000 7f12deb89640 1 Processor -- start 2023-12-01T03:12:19.701 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.699+0000 7f12deb89640 1 -- start start 2023-12-01T03:12:19.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.700+0000 7f12deb89640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f12d815e2f0 0x7f12d80ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.700+0000 7f12deb89640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12d80adf10 0x7f12d80caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.700+0000 7f12deb89640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d80add10 con 0x7f12d806f2f0 2023-12-01T03:12:19.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.700+0000 7f12deb89640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d80c5760 con 0x7f12d815e2f0 2023-12-01T03:12:19.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.700+0000 7f12deb89640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d815e6d0 con 0x7f12d80adf10 2023-12-01T03:12:19.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.700+0000 7f12dc925640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f12d815e2f0 0x7f12d80ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:19.702 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.700+0000 7f12dc925640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f12d815e2f0 0x7f12d80ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54152/0 (socket says 172.21.15.115:54152) 2023-12-01T03:12:19.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.700+0000 7f12dc925640 1 -- 172.21.15.115:0/3565685032 learned_addr learned my addr 172.21.15.115:0/3565685032 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:19.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.700+0000 7f12d7fff640 1 --1- 172.21.15.115:0/3565685032 >> v1:172.21.15.115:6789/0 conn(0x7f12d806f2f0 0x7f12d80c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37758/0 (socket says 172.21.15.115:37758) 2023-12-01T03:12:19.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.701+0000 7f12dd126640 1 --2- 172.21.15.115:0/3565685032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12d80adf10 0x7f12d80caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:19.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.701+0000 7f12dc925640 1 -- 172.21.15.115:0/3565685032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12d80adf10 msgr2=0x7f12d80caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:19.703 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.701+0000 7f12dc925640 1 --2- 172.21.15.115:0/3565685032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12d80adf10 0x7f12d80caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.701+0000 7f12dc925640 1 -- 172.21.15.115:0/3565685032 >> v1:172.21.15.115:6789/0 conn(0x7f12d806f2f0 legacy=0x7f12d80c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.701+0000 7f12dc925640 1 -- 172.21.15.115:0/3565685032 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d80cf7b0 con 0x7f12d815e2f0 2023-12-01T03:12:19.704 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.702+0000 7f12dc925640 1 --2- 172.21.15.115:0/3565685032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 0x7f12d80ca660 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009870 tx=0x7f12cc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576b321d1fa2b3a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.702+0000 7f12d77fe640 1 -- 172.21.15.115:0/3565685032 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc0317d0 con 0x7f12d815e2f0 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.702+0000 7f12d77fe640 1 -- 172.21.15.115:0/3565685032 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc031930 con 0x7f12d815e2f0 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.702+0000 7f12d77fe640 1 -- 172.21.15.115:0/3565685032 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc031cc0 con 0x7f12d815e2f0 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.703+0000 7f12deb89640 1 -- 172.21.15.115:0/3565685032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 msgr2=0x7f12d80ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.703+0000 7f12deb89640 1 --2- 172.21.15.115:0/3565685032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 0x7f12d80ca660 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009870 tx=0x7f12cc02b870 comp rx=0 tx=0).stop 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.703+0000 7f12deb89640 1 -- 172.21.15.115:0/3565685032 shutdown_connections 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.703+0000 7f12deb89640 1 --2- 172.21.15.115:0/3565685032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12d80adf10 0x7f12d80caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.703+0000 7f12deb89640 1 --2- 172.21.15.115:0/3565685032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 0x7f12d80ca660 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.703+0000 7f12deb89640 1 -- 172.21.15.115:0/3565685032 >> 172.21.15.115:0/3565685032 conn(0x7f12d80ce880 msgr2=0x7f12d80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.703+0000 7f12deb89640 1 -- 172.21.15.115:0/3565685032 shutdown_connections 2023-12-01T03:12:19.705 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.703+0000 7f12deb89640 1 -- 172.21.15.115:0/3565685032 wait complete. 2023-12-01T03:12:19.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.705+0000 7f12deb89640 1 Processor -- start 2023-12-01T03:12:19.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.705+0000 7f12deb89640 1 -- start start 2023-12-01T03:12:19.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.706+0000 7f12deb89640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12d806f2f0 0x7f12d80acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.706+0000 7f12deb89640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12d80adf10 0x7f12d80c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.706+0000 7f12deb89640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 0x7f12d80c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.706+0000 7f12deb89640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80c4ee0 con 0x7f12d815e2f0 2023-12-01T03:12:19.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.706+0000 7f12deb89640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80c4360 con 0x7f12d806f2f0 2023-12-01T03:12:19.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.706+0000 7f12deb89640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80c4640 con 0x7f12d80adf10 2023-12-01T03:12:19.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.706+0000 7f12dd126640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 0x7f12d80c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:19.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.706+0000 7f12dd126640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 0x7f12d80c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54160/0 (socket says 172.21.15.115:54160) 2023-12-01T03:12:19.708 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.706+0000 7f12dd126640 1 -- 172.21.15.115:0/2397209223 learned_addr learned my addr 172.21.15.115:0/2397209223 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:19.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12dd126640 1 -- 172.21.15.115:0/2397209223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12d80adf10 msgr2=0x7f12d80c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12dd126640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12d80adf10 0x7f12d80c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12dd126640 1 -- 172.21.15.115:0/2397209223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12d806f2f0 msgr2=0x7f12d80acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12dd126640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12d806f2f0 0x7f12d80acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12dd126640 1 -- 172.21.15.115:0/2397209223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0094d0 con 0x7f12d815e2f0 2023-12-01T03:12:19.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12dd126640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 0x7f12d80c8200 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f12d000cc30 tx=0x7f12d000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:19.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12d57fa640 1 -- 172.21.15.115:0/2397209223 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d000ce40 con 0x7f12d815e2f0 2023-12-01T03:12:19.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12d57fa640 1 -- 172.21.15.115:0/2397209223 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d0003990 con 0x7f12d815e2f0 2023-12-01T03:12:19.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12d57fa640 1 -- 172.21.15.115:0/2397209223 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0003d20 con 0x7f12d815e2f0 2023-12-01T03:12:19.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12deb89640 1 -- 172.21.15.115:0/2397209223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d80c4920 con 0x7f12d815e2f0 2023-12-01T03:12:19.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.707+0000 7f12deb89640 1 -- 172.21.15.115:0/2397209223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d80ad0b0 con 0x7f12d815e2f0 2023-12-01T03:12:19.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.709+0000 7f12deb89640 1 -- 172.21.15.115:0/2397209223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d806fda0 con 0x7f12d815e2f0 2023-12-01T03:12:19.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.709+0000 7f12d57fa640 1 -- 172.21.15.115:0/2397209223 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f12d0003e80 con 0x7f12d815e2f0 2023-12-01T03:12:19.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.710+0000 7f12d57fa640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f12bc06a670 0x7f12bc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:19.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.710+0000 7f12d57fa640 1 -- 172.21.15.115:0/2397209223 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6625+0+0 (secure 0 0 0) 0x7f12d00895d0 con 0x7f12d815e2f0 2023-12-01T03:12:19.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.711+0000 7f12dc925640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f12bc06a670 0x7f12bc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:19.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.711+0000 7f12dc925640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f12bc06a670 0x7f12bc06cb30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f12cc036000 tx=0x7f12cc030000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:19.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.714+0000 7f12d57fa640 1 -- 172.21.15.115:0/2397209223 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f12d0054440 con 0x7f12d815e2f0 2023-12-01T03:12:19.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.905+0000 7f12deb89640 1 -- 172.21.15.115:0/2397209223 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12d814ac20 con 0x7f12d815e2f0 2023-12-01T03:12:19.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.907+0000 7f12d57fa640 1 -- 172.21.15.115:0/2397209223 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v244) v1 ==== 74+0+17435 (secure 0 0 0) 0x7f12d814ac20 con 0x7f12d815e2f0 2023-12-01T03:12:19.909 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:19.909 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":244,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:17.226038+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:11:49.229904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:11:50.056441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6835/1837882056":"2023-12-02T03:12:14.201815+0000","172.21.15.142:6834/1837882056":"2023-12-02T03:12:14.201815+0000","172.21.15.115:0/2980649202":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/1025490173":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2408987063":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3619877240":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2405923893":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/957514002":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/638241265":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1722505219":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3713299174":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/4124642007":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6824/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/3863453309":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1160795001":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/988288848":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/524060219":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1801977880":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/203535105":"2023-12-02T03:12:02.677778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.909+0000 7f12deb89640 1 -- 172.21.15.115:0/2397209223 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f12bc06a670 msgr2=0x7f12bc06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.909+0000 7f12deb89640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f12bc06a670 0x7f12bc06cb30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f12cc036000 tx=0x7f12cc030000 comp rx=0 tx=0).stop 2023-12-01T03:12:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.909+0000 7f12deb89640 1 -- 172.21.15.115:0/2397209223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 msgr2=0x7f12d80c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.909+0000 7f12deb89640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 0x7f12d80c8200 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f12d000cc30 tx=0x7f12d000a5c0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.910+0000 7f12deb89640 1 -- 172.21.15.115:0/2397209223 shutdown_connections 2023-12-01T03:12:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.910+0000 7f12deb89640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f12bc06a670 0x7f12bc06cb30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.910+0000 7f12deb89640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12d80adf10 0x7f12d80c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.910+0000 7f12deb89640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12d815e2f0 0x7f12d80c8200 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.910+0000 7f12deb89640 1 --2- 172.21.15.115:0/2397209223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12d806f2f0 0x7f12d80acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.910+0000 7f12deb89640 1 -- 172.21.15.115:0/2397209223 >> 172.21.15.115:0/2397209223 conn(0x7f12d80ce880 msgr2=0x7f12d815d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.910+0000 7f12deb89640 1 -- 172.21.15.115:0/2397209223 shutdown_connections 2023-12-01T03:12:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:19.910+0000 7f12deb89640 1 -- 172.21.15.115:0/2397209223 wait complete. 2023-12-01T03:12:19.929 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-12-01T03:12:19.930 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:12:20.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.140+0000 7f7f4b0ce640 1 Processor -- start 2023-12-01T03:12:20.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.140+0000 7f7f4b0ce640 1 -- start start 2023-12-01T03:12:20.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.141+0000 7f7f4b0ce640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7f44159dd0 0x7f7f440c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.142+0000 7f7f4b0ce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4406f8b0 con 0x7f7f44070d30 2023-12-01T03:12:20.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.142+0000 7f7f4b0ce640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f440adcb0 con 0x7f7f4406b6f0 2023-12-01T03:12:20.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.142+0000 7f7f4b0ce640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f440cd270 con 0x7f7f44159dd0 2023-12-01T03:12:20.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.142+0000 7f7f48e6a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7f44159dd0 0x7f7f440c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:20.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.142+0000 7f7f48e6a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7f44159dd0 0x7f7f440c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54174/0 (socket says 172.21.15.115:54174) 2023-12-01T03:12:20.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.142+0000 7f7f48e6a640 1 -- 172.21.15.115:0/2586523707 learned_addr learned my addr 172.21.15.115:0/2586523707 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:20.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.142+0000 7f7f4966b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7f44070d30 0x7f7f440ccb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37772/0 (socket says 172.21.15.115:37772) 2023-12-01T03:12:20.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.142+0000 7f7f48e6a640 1 -- 172.21.15.115:0/2586523707 >> v1:172.21.15.142:6789/0 conn(0x7f7f4406b6f0 legacy=0x7f7f440cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:20.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.142+0000 7f7f48e6a640 1 -- 172.21.15.115:0/2586523707 >> v1:172.21.15.115:6789/0 conn(0x7f7f44070d30 legacy=0x7f7f440ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.142+0000 7f7f48e6a640 1 -- 172.21.15.115:0/2586523707 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440ad9d0 con 0x7f7f44159dd0 2023-12-01T03:12:20.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.143+0000 7f7f48e6a640 1 --2- 172.21.15.115:0/2586523707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44159dd0 0x7f7f440c97e0 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009870 tx=0x7f7f3802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd5d3b31726884 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:20.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.144+0000 7f7f437fe640 1 -- 172.21.15.115:0/2586523707 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f380317e0 con 0x7f7f44159dd0 2023-12-01T03:12:20.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.144+0000 7f7f437fe640 1 -- 172.21.15.115:0/2586523707 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f38031940 con 0x7f7f44159dd0 2023-12-01T03:12:20.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.144+0000 7f7f437fe640 1 -- 172.21.15.115:0/2586523707 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38031cd0 con 0x7f7f44159dd0 2023-12-01T03:12:20.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.144+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/2586523707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44159dd0 msgr2=0x7f7f440c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.144+0000 7f7f4b0ce640 1 --2- 172.21.15.115:0/2586523707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44159dd0 0x7f7f440c97e0 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009870 tx=0x7f7f3802b880 comp rx=0 tx=0).stop 2023-12-01T03:12:20.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.144+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/2586523707 shutdown_connections 2023-12-01T03:12:20.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.144+0000 7f7f4b0ce640 1 --2- 172.21.15.115:0/2586523707 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44159dd0 0x7f7f440c97e0 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.144+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/2586523707 >> 172.21.15.115:0/2586523707 conn(0x7f7f440ce880 msgr2=0x7f7f440cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:20.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.145+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/2586523707 shutdown_connections 2023-12-01T03:12:20.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.145+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/2586523707 wait complete. 2023-12-01T03:12:20.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.146+0000 7f7f4b0ce640 1 Processor -- start 2023-12-01T03:12:20.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.146+0000 7f7f4b0ce640 1 -- start start 2023-12-01T03:12:20.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.147+0000 7f7f4b0ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4406b6f0 0x7f7f4414ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.147+0000 7f7f4b0ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44070d30 0x7f7f44156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.147+0000 7f7f4b0ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f44159dd0 0x7f7f44156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.147+0000 7f7f4b0ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440c7b30 con 0x7f7f44070d30 2023-12-01T03:12:20.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.147+0000 7f7f4b0ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440c7e10 con 0x7f7f4406b6f0 2023-12-01T03:12:20.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.147+0000 7f7f4b0ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440c80f0 con 0x7f7f44159dd0 2023-12-01T03:12:20.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.147+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44070d30 0x7f7f44156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:20.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44070d30 0x7f7f44156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54186/0 (socket says 172.21.15.115:54186) 2023-12-01T03:12:20.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f43fff640 1 -- 172.21.15.115:0/3658430527 learned_addr learned my addr 172.21.15.115:0/3658430527 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:20.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f43fff640 1 -- 172.21.15.115:0/3658430527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f44159dd0 msgr2=0x7f7f44156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f43fff640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f44159dd0 0x7f7f44156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f43fff640 1 -- 172.21.15.115:0/3658430527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4406b6f0 msgr2=0x7f7f4414ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f43fff640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4406b6f0 0x7f7f4414ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f43fff640 1 -- 172.21.15.115:0/3658430527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380094d0 con 0x7f7f44070d30 2023-12-01T03:12:20.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f43fff640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44070d30 0x7f7f44156810 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b3f0 tx=0x7f7f3000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:20.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f417fa640 1 -- 172.21.15.115:0/3658430527 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30011830 con 0x7f7f44070d30 2023-12-01T03:12:20.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f417fa640 1 -- 172.21.15.115:0/3658430527 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f30011990 con 0x7f7f44070d30 2023-12-01T03:12:20.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.148+0000 7f7f417fa640 1 -- 172.21.15.115:0/3658430527 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30011d40 con 0x7f7f44070d30 2023-12-01T03:12:20.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.149+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/3658430527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f440c83d0 con 0x7f7f44070d30 2023-12-01T03:12:20.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.149+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/3658430527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4414bf90 con 0x7f7f44070d30 2023-12-01T03:12:20.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.150+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/3658430527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f440c88f0 con 0x7f7f44070d30 2023-12-01T03:12:20.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.151+0000 7f7f417fa640 1 -- 172.21.15.115:0/3658430527 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f7f3001a400 con 0x7f7f44070d30 2023-12-01T03:12:20.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.152+0000 7f7f417fa640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f2806a670 0x7f7f2806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.152+0000 7f7f417fa640 1 -- 172.21.15.115:0/3658430527 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6625+0+0 (secure 0 0 0) 0x7f7f30089090 con 0x7f7f44070d30 2023-12-01T03:12:20.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.152+0000 7f7f48e6a640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f2806a670 0x7f7f2806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:20.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.153+0000 7f7f48e6a640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f2806a670 0x7f7f2806cb30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f38036000 tx=0x7f7f38030000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:20.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.156+0000 7f7f417fa640 1 -- 172.21.15.115:0/3658430527 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7f440c88f0 con 0x7f7f44070d30 2023-12-01T03:12:20.386 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.384+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/3658430527 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f440c88f0 con 0x7f7f44070d30 2023-12-01T03:12:20.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.385+0000 7f7f417fa640 1 -- 172.21.15.115:0/3658430527 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4658 (secure 0 0 0) 0x7f7f440c88f0 con 0x7f7f44070d30 2023-12-01T03:12:20.387 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:20.387 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":118,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/993295369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":993295369},{"type":"v1","addr":"172.21.15.142:6835","nonce":993295369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:51.037525+0000","modified":"2023-12-01T03:11:54.055433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7732":{"gid":7732,"name":"b","rank":0,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-01T03:12:20.387 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 118 2023-12-01T03:12:20.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/3658430527 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f2806a670 msgr2=0x7f7f2806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f2806a670 0x7f7f2806cb30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f38036000 tx=0x7f7f38030000 comp rx=0 tx=0).stop 2023-12-01T03:12:20.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/3658430527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44070d30 msgr2=0x7f7f44156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44070d30 0x7f7f44156810 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b3f0 tx=0x7f7f3000b8b0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/3658430527 shutdown_connections 2023-12-01T03:12:20.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f2806a670 0x7f7f2806cb30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f44159dd0 0x7f7f44156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f44070d30 0x7f7f44156810 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 --2- 172.21.15.115:0/3658430527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f4406b6f0 0x7f7f4414ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/3658430527 >> 172.21.15.115:0/3658430527 conn(0x7f7f440ce880 msgr2=0x7f7f440ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:20.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/3658430527 shutdown_connections 2023-12-01T03:12:20.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.388+0000 7f7f4b0ce640 1 -- 172.21.15.115:0/3658430527 wait complete. 2023-12-01T03:12:20.409 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:12:20.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.625+0000 7feb00755640 1 Processor -- start 2023-12-01T03:12:20.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.625+0000 7feb00755640 1 -- start start 2023-12-01T03:12:20.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feb00755640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8159dd0 0x7feaf80c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feb00755640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7feaf8070d30 0x7feaf80c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feb00755640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf806f8b0 con 0x7feaf806b6f0 2023-12-01T03:12:20.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feb00755640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf80adcb0 con 0x7feaf8070d30 2023-12-01T03:12:20.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feb00755640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80ad9d0 con 0x7feaf8159dd0 2023-12-01T03:12:20.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feafecf2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7feaf8070d30 0x7feaf80c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:20.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feafecf2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7feaf8070d30 0x7feaf80c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54198/0 (socket says 172.21.15.115:54198) 2023-12-01T03:12:20.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feafecf2640 1 -- 172.21.15.115:0/491494005 learned_addr learned my addr 172.21.15.115:0/491494005 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:20.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feafe4f1640 1 --2- 172.21.15.115:0/491494005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8159dd0 0x7feaf80c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:20.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feafecf2640 1 -- 172.21.15.115:0/491494005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8159dd0 msgr2=0x7feaf80c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:20.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feafecf2640 1 --2- 172.21.15.115:0/491494005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8159dd0 0x7feaf80c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.627+0000 7feafecf2640 1 -- 172.21.15.115:0/491494005 >> v1:172.21.15.142:6789/0 conn(0x7feaf806b6f0 legacy=0x7feaf80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:20.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.628+0000 7feafecf2640 1 -- 172.21.15.115:0/491494005 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf80c9000 con 0x7feaf8070d30 2023-12-01T03:12:20.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.629+0000 7feafecf2640 1 --2- 172.21.15.115:0/491494005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8070d30 0x7feaf80c9d20 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009e50 tx=0x7feaf002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6854bcbb854b981 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:20.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.629+0000 7feafd4ef640 1 -- 172.21.15.115:0/491494005 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf002c950 con 0x7feaf8070d30 2023-12-01T03:12:20.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.629+0000 7feafd4ef640 1 -- 172.21.15.115:0/491494005 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf002cab0 con 0x7feaf8070d30 2023-12-01T03:12:20.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.629+0000 7feafd4ef640 1 -- 172.21.15.115:0/491494005 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf002ce40 con 0x7feaf8070d30 2023-12-01T03:12:20.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.630+0000 7feb00755640 1 -- 172.21.15.115:0/491494005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8070d30 msgr2=0x7feaf80c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.630+0000 7feb00755640 1 --2- 172.21.15.115:0/491494005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8070d30 0x7feaf80c9d20 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009e50 tx=0x7feaf002bdb0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.630+0000 7feb00755640 1 -- 172.21.15.115:0/491494005 shutdown_connections 2023-12-01T03:12:20.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.630+0000 7feb00755640 1 --2- 172.21.15.115:0/491494005 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8070d30 0x7feaf80c9d20 unknown :-1 s=CLOSED pgs=2893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.630+0000 7feb00755640 1 --2- 172.21.15.115:0/491494005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8159dd0 0x7feaf80c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.630+0000 7feb00755640 1 -- 172.21.15.115:0/491494005 >> 172.21.15.115:0/491494005 conn(0x7feaf80ce880 msgr2=0x7feaf80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:20.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.630+0000 7feb00755640 1 -- 172.21.15.115:0/491494005 shutdown_connections 2023-12-01T03:12:20.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.630+0000 7feb00755640 1 -- 172.21.15.115:0/491494005 wait complete. 2023-12-01T03:12:20.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.632+0000 7feb00755640 1 Processor -- start 2023-12-01T03:12:20.633 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.632+0000 7feb00755640 1 -- start start 2023-12-01T03:12:20.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feb00755640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf806b6f0 0x7feaf8152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feb00755640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8070d30 0x7feaf81f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feb00755640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8159dd0 0x7feaf81f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feb00755640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8151810 con 0x7feaf8159dd0 2023-12-01T03:12:20.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feb00755640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf806ccd0 con 0x7feaf806b6f0 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feb00755640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8150f70 con 0x7feaf8070d30 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feafecf2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8159dd0 0x7feaf81f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feafecf2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8159dd0 0x7feaf81f3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54200/0 (socket says 172.21.15.115:54200) 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feafecf2640 1 -- 172.21.15.115:0/964003375 learned_addr learned my addr 172.21.15.115:0/964003375 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feafe4f1640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf806b6f0 0x7feaf8152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.633+0000 7feafdcf0640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8070d30 0x7feaf81f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feafecf2640 1 -- 172.21.15.115:0/964003375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8070d30 msgr2=0x7feaf81f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feafecf2640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8070d30 0x7feaf81f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feafecf2640 1 -- 172.21.15.115:0/964003375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf806b6f0 msgr2=0x7feaf8152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feafecf2640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf806b6f0 0x7feaf8152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feafecf2640 1 -- 172.21.15.115:0/964003375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf0009b30 con 0x7feaf8159dd0 2023-12-01T03:12:20.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feafdcf0640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8070d30 0x7feaf81f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:20.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feafecf2640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8159dd0 0x7feaf81f3f80 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7feaf0036040 tx=0x7feaf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:20.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feaeeffd640 1 -- 172.21.15.115:0/964003375 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf003e450 con 0x7feaf8159dd0 2023-12-01T03:12:20.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feaeeffd640 1 -- 172.21.15.115:0/964003375 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf003e5d0 con 0x7feaf8159dd0 2023-12-01T03:12:20.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feaeeffd640 1 -- 172.21.15.115:0/964003375 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf003e980 con 0x7feaf8159dd0 2023-12-01T03:12:20.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feb00755640 1 -- 172.21.15.115:0/964003375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf8151250 con 0x7feaf8159dd0 2023-12-01T03:12:20.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.634+0000 7feb00755640 1 -- 172.21.15.115:0/964003375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf8151fe0 con 0x7feaf8159dd0 2023-12-01T03:12:20.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.636+0000 7feb00755640 1 -- 172.21.15.115:0/964003375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf8152170 con 0x7feaf8159dd0 2023-12-01T03:12:20.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.637+0000 7feaeeffd640 1 -- 172.21.15.115:0/964003375 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7feaf003eae0 con 0x7feaf8159dd0 2023-12-01T03:12:20.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.637+0000 7feaeeffd640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feadc06a670 0x7feadc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:20.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.637+0000 7feaeeffd640 1 -- 172.21.15.115:0/964003375 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6625+0+0 (secure 0 0 0) 0x7feaf00abf90 con 0x7feaf8159dd0 2023-12-01T03:12:20.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.638+0000 7feafe4f1640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feadc06a670 0x7feadc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:20.640 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.638+0000 7feafe4f1640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feadc06a670 0x7feadc06cb30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feaf4006fd0 tx=0x7feaf4008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:20.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.642+0000 7feaeeffd640 1 -- 172.21.15.115:0/964003375 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7feaf0076f20 con 0x7feaf8159dd0 2023-12-01T03:12:20.873 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.871+0000 7feb00755640 1 -- 172.21.15.115:0/964003375 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feaf804e890 con 0x7feaf8159dd0 2023-12-01T03:12:20.874 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.872+0000 7feaeeffd640 1 -- 172.21.15.115:0/964003375 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4658 (secure 0 0 0) 0x7feaf007a540 con 0x7feaf8159dd0 2023-12-01T03:12:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:20.874 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":118,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/993295369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":993295369},{"type":"v1","addr":"172.21.15.142:6835","nonce":993295369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:11:51.037525+0000","modified":"2023-12-01T03:11:54.055433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7732":{"gid":7732,"name":"b","rank":0,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.142:6833/3596468657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3596468657},{"type":"v1","addr":"172.21.15.142:6833","nonce":3596468657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-01T03:12:20.874 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 118 2023-12-01T03:12:20.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 -- 172.21.15.115:0/964003375 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feadc06a670 msgr2=0x7feadc06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feadc06a670 0x7feadc06cb30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feaf4006fd0 tx=0x7feaf4008040 comp rx=0 tx=0).stop 2023-12-01T03:12:20.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 -- 172.21.15.115:0/964003375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8159dd0 msgr2=0x7feaf81f3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:20.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8159dd0 0x7feaf81f3f80 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7feaf0036040 tx=0x7feaf0004a40 comp rx=0 tx=0).stop 2023-12-01T03:12:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 -- 172.21.15.115:0/964003375 shutdown_connections 2023-12-01T03:12:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feadc06a670 0x7feadc06cb30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feaf8070d30 0x7feaf81f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feaf8159dd0 0x7feaf81f3f80 unknown :-1 s=CLOSED pgs=2894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 --2- 172.21.15.115:0/964003375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf806b6f0 0x7feaf8152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 -- 172.21.15.115:0/964003375 >> 172.21.15.115:0/964003375 conn(0x7feaf80ce880 msgr2=0x7feaf80cbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 -- 172.21.15.115:0/964003375 shutdown_connections 2023-12-01T03:12:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:20.875+0000 7feb00755640 1 -- 172.21.15.115:0/964003375 wait complete. 2023-12-01T03:12:20.895 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:21.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.105+0000 7fe0f5809640 1 Processor -- start 2023-12-01T03:12:21.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.106+0000 7fe0f5809640 1 -- start start 2023-12-01T03:12:21.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.107+0000 7fe0f5809640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe0f015cf90 0x7fe0f00d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.107+0000 7fe0f5809640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f015d370 con 0x7fe0f00b8610 2023-12-01T03:12:21.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.107+0000 7fe0f5809640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f00d19a0 con 0x7fe0f00ce0d0 2023-12-01T03:12:21.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.107+0000 7fe0f5809640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f00d1c80 con 0x7fe0f015cf90 2023-12-01T03:12:21.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.107+0000 7fe0e7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fe0f00b8610 0x7fe0f00cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37774/0 (socket says 172.21.15.115:37774) 2023-12-01T03:12:21.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.107+0000 7fe0e7fff640 1 -- 172.21.15.115:0/1624099018 learned_addr learned my addr 172.21.15.115:0/1624099018 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:21.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0eeffd640 1 --2- 172.21.15.115:0/1624099018 >> v2:172.21.15.142:3300/0 conn(0x7fe0f015cf90 0x7fe0f00d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335603450 0 0) 0x7fe0f015d370 con 0x7fe0f00b8610 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0dc003520 con 0x7fe0f00b8610 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 594368339 0 0) 0x7fe0dc003520 con 0x7fe0f00b8610 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 >> v2:172.21.15.142:3300/0 conn(0x7fe0f015cf90 msgr2=0x7fe0f00d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 --2- 172.21.15.115:0/1624099018 >> v2:172.21.15.142:3300/0 conn(0x7fe0f015cf90 0x7fe0f00d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 >> v1:172.21.15.142:6789/0 conn(0x7fe0f00ce0d0 legacy=0x7fe0f00b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f00d1f60 con 0x7fe0f00b8610 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fe0d8001670 con 0x7fe0f00b8610 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 >> v1:172.21.15.115:6789/0 conn(0x7fe0f00b8610 legacy=0x7fe0f00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0dc0039e0 0x7fe0dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc0047c0 0x7fe0dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0dc005180 0x7fe0dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.108+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f00d1c80 con 0x7fe0dc005180 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc00c0b0 con 0x7fe0dc0047c0 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc00c270 con 0x7fe0dc0039e0 2023-12-01T03:12:21.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0ef7fe640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0dc005180 0x7fe0dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0e7fff640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc0047c0 0x7fe0dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0eeffd640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0dc0039e0 0x7fe0dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0ef7fe640 1 -- 172.21.15.115:0/1624099018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0dc0039e0 msgr2=0x7fe0dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:21.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0ef7fe640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0dc0039e0 0x7fe0dc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0ef7fe640 1 -- 172.21.15.115:0/1624099018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc0047c0 msgr2=0x7fe0dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:21.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0ef7fe640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc0047c0 0x7fe0dc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0ef7fe640 1 -- 172.21.15.115:0/1624099018 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc003520 con 0x7fe0dc005180 2023-12-01T03:12:21.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.109+0000 7fe0eeffd640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0dc0039e0 0x7fe0dc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:21.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.110+0000 7fe0ef7fe640 1 -- 172.21.15.115:0/1624099018 reap_dead start 2023-12-01T03:12:21.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.110+0000 7fe0ef7fe640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0dc005180 0x7fe0dc00b980 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8001e50 tx=0x7fe0dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ff029b460db035 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:21.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.111+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8003b70 con 0x7fe0dc005180 2023-12-01T03:12:21.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.111+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e8003d10 con 0x7fe0dc005180 2023-12-01T03:12:21.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.111+0000 7fe0ee7fc640 1 -- 172.21.15.115:0/1624099018 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8005310 con 0x7fe0dc005180 2023-12-01T03:12:21.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.111+0000 7fe0f5809640 1 -- 172.21.15.115:0/1624099018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0dc005180 msgr2=0x7fe0dc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.111+0000 7fe0f5809640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0dc005180 0x7fe0dc00b980 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8001e50 tx=0x7fe0dc0057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.111+0000 7fe0f5809640 1 -- 172.21.15.115:0/1624099018 shutdown_connections 2023-12-01T03:12:21.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.111+0000 7fe0f5809640 1 --2- 172.21.15.115:0/1624099018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0dc005180 0x7fe0dc00b980 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.111+0000 7fe0f5809640 1 -- 172.21.15.115:0/1624099018 >> 172.21.15.115:0/1624099018 conn(0x7fe0f0065960 msgr2=0x7fe0f0065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:21.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.111+0000 7fe0f5809640 1 -- 172.21.15.115:0/1624099018 shutdown_connections 2023-12-01T03:12:21.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.111+0000 7fe0f5809640 1 -- 172.21.15.115:0/1624099018 wait complete. 2023-12-01T03:12:21.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.113+0000 7fe0f5809640 1 Processor -- start 2023-12-01T03:12:21.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.113+0000 7fe0f5809640 1 -- start start 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.114+0000 7fe0f5809640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc005180 0x7fe0f01f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.114+0000 7fe0f5809640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0f015e590 0x7fe0f015ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.114+0000 7fe0f5809640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f014a840 0x7fe0f014acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.114+0000 7fe0f5809640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f014b200 con 0x7fe0f014a840 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.114+0000 7fe0f5809640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f015f030 con 0x7fe0dc005180 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.114+0000 7fe0f5809640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f015f310 con 0x7fe0f015e590 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0ef7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f014a840 0x7fe0f014acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0ef7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f014a840 0x7fe0f014acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54224/0 (socket says 172.21.15.115:54224) 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0ef7fe640 1 -- 172.21.15.115:0/561172754 learned_addr learned my addr 172.21.15.115:0/561172754 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0eeffd640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc005180 0x7fe0f01f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0e7fff640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0f015e590 0x7fe0f015ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0ef7fe640 1 -- 172.21.15.115:0/561172754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0f015e590 msgr2=0x7fe0f015ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0ef7fe640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0f015e590 0x7fe0f015ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0ef7fe640 1 -- 172.21.15.115:0/561172754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc005180 msgr2=0x7fe0f01f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0ef7fe640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc005180 0x7fe0f01f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0ef7fe640 1 -- 172.21.15.115:0/561172754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f015f5f0 con 0x7fe0f014a840 2023-12-01T03:12:21.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0e7fff640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0f015e590 0x7fe0f015ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:21.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.115+0000 7fe0ef7fe640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f014a840 0x7fe0f014acc0 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8001f70 tx=0x7fe0e80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:21.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.116+0000 7fe0ecff9640 1 -- 172.21.15.115:0/561172754 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8006d50 con 0x7fe0f014a840 2023-12-01T03:12:21.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.116+0000 7fe0ecff9640 1 -- 172.21.15.115:0/561172754 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e8003b80 con 0x7fe0f014a840 2023-12-01T03:12:21.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.116+0000 7fe0ecff9640 1 -- 172.21.15.115:0/561172754 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80396c0 con 0x7fe0f014a840 2023-12-01T03:12:21.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.116+0000 7fe0f5809640 1 -- 172.21.15.115:0/561172754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f014b200 con 0x7fe0f014a840 2023-12-01T03:12:21.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.116+0000 7fe0f5809640 1 -- 172.21.15.115:0/561172754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f015fbb0 con 0x7fe0f014a840 2023-12-01T03:12:21.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.117+0000 7fe0f5809640 1 -- 172.21.15.115:0/561172754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f015fdd0 con 0x7fe0f014a840 2023-12-01T03:12:21.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.118+0000 7fe0ecff9640 1 -- 172.21.15.115:0/561172754 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fe0e8039820 con 0x7fe0f014a840 2023-12-01T03:12:21.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.119+0000 7fe0ecff9640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe0d006a670 0x7fe0d006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.119+0000 7fe0ecff9640 1 -- 172.21.15.115:0/561172754 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6625+0+0 (secure 0 0 0) 0x7fe0e80ab060 con 0x7fe0f014a840 2023-12-01T03:12:21.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.119+0000 7fe0eeffd640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe0d006a670 0x7fe0d006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.120+0000 7fe0eeffd640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe0d006a670 0x7fe0d006cb30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009aa0 tx=0x7fe0e0008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:21.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.123+0000 7fe0ecff9640 1 -- 172.21.15.115:0/561172754 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe0e8075ed0 con 0x7fe0f014a840 2023-12-01T03:12:21.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.318+0000 7fe0f5809640 1 -- 172.21.15.115:0/561172754 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0f004e890 con 0x7fe0f014a840 2023-12-01T03:12:21.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.321+0000 7fe0ecff9640 1 -- 172.21.15.115:0/561172754 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v244) v1 ==== 74+0+17435 (secure 0 0 0) 0x7fe0e8048090 con 0x7fe0f014a840 2023-12-01T03:12:21.323 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:21.323 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":244,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:17.226038+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-01T03:11:49.229904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-01T03:11:50.056441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"239","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6835/1837882056":"2023-12-02T03:12:14.201815+0000","172.21.15.142:6834/1837882056":"2023-12-02T03:12:14.201815+0000","172.21.15.115:0/2980649202":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/1025490173":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2408987063":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3619877240":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/2405923893":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/957514002":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/638241265":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1722505219":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/3713299174":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/4124642007":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6824/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/3863453309":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1160795001":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/988288848":"2023-12-02T03:12:09.014426+0000","172.21.15.115:0/524060219":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/1801977880":"2023-12-02T03:12:02.677778+0000","172.21.15.115:0/203535105":"2023-12-02T03:12:02.677778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:21.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 -- 172.21.15.115:0/561172754 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe0d006a670 msgr2=0x7fe0d006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe0d006a670 0x7fe0d006cb30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009aa0 tx=0x7fe0e0008040 comp rx=0 tx=0).stop 2023-12-01T03:12:21.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 -- 172.21.15.115:0/561172754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f014a840 msgr2=0x7fe0f014acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f014a840 0x7fe0f014acc0 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8001f70 tx=0x7fe0e80014a0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 -- 172.21.15.115:0/561172754 shutdown_connections 2023-12-01T03:12:21.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe0d006a670 0x7fe0d006cb30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe0f015e590 0x7fe0f015ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f014a840 0x7fe0f014acc0 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 --2- 172.21.15.115:0/561172754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0dc005180 0x7fe0f01f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 -- 172.21.15.115:0/561172754 >> 172.21.15.115:0/561172754 conn(0x7fe0f0065960 msgr2=0x7fe0f00c7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 -- 172.21.15.115:0/561172754 shutdown_connections 2023-12-01T03:12:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.324+0000 7fe0f5809640 1 -- 172.21.15.115:0/561172754 wait complete. 2023-12-01T03:12:21.345 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-12-01T03:12:21.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.556+0000 7f5cbb20b640 1 Processor -- start 2023-12-01T03:12:21.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.557+0000 7f5cbb20b640 1 -- start start 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.558+0000 7f5cbb20b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb415d220 0x7f5cb40cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.558+0000 7f5cbb20b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5cb40ad2b0 0x7f5cb40ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.558+0000 7f5cbb20b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5cb40ade80 0x7f5cb406b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.558+0000 7f5cbb20b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb406cc10 con 0x7f5cb40ade80 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.558+0000 7f5cbb20b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb406c720 con 0x7f5cb40ad2b0 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.558+0000 7f5cbb20b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb415d600 con 0x7f5cb415d220 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.558+0000 7f5cb97a8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5cb40ade80 0x7f5cb406b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.559+0000 7f5cb97a8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5cb40ade80 0x7f5cb406b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54240/0 (socket says 172.21.15.115:54240) 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.559+0000 7f5cb97a8640 1 -- 172.21.15.115:0/3598690842 learned_addr learned my addr 172.21.15.115:0/3598690842 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.559+0000 7f5cb8fa7640 1 --2- 172.21.15.115:0/3598690842 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb415d220 0x7f5cb40cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.559+0000 7f5cb97a8640 1 -- 172.21.15.115:0/3598690842 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb415d220 msgr2=0x7f5cb40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.559+0000 7f5cb97a8640 1 --2- 172.21.15.115:0/3598690842 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb415d220 0x7f5cb40cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.559+0000 7f5cb3fff640 1 --2- 172.21.15.115:0/3598690842 >> v2:172.21.15.142:3300/0 conn(0x7f5cb40ad2b0 0x7f5cb40ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.559+0000 7f5cb97a8640 1 -- 172.21.15.115:0/3598690842 >> v2:172.21.15.142:3300/0 conn(0x7f5cb40ad2b0 msgr2=0x7f5cb40ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:21.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.559+0000 7f5cb97a8640 1 --2- 172.21.15.115:0/3598690842 >> v2:172.21.15.142:3300/0 conn(0x7f5cb40ad2b0 0x7f5cb40ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.559+0000 7f5cb97a8640 1 -- 172.21.15.115:0/3598690842 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb406c4b0 con 0x7f5cb40ade80 2023-12-01T03:12:21.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.560+0000 7f5cb97a8640 1 --2- 172.21.15.115:0/3598690842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb40ade80 0x7f5cb406b350 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40b1370 tx=0x7f5cac02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f51390f5de9afb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:21.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.560+0000 7f5cb37fe640 1 -- 172.21.15.115:0/3598690842 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac02c9f0 con 0x7f5cb40ade80 2023-12-01T03:12:21.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.560+0000 7f5cb37fe640 1 -- 172.21.15.115:0/3598690842 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac02cb50 con 0x7f5cb40ade80 2023-12-01T03:12:21.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.560+0000 7f5cb37fe640 1 -- 172.21.15.115:0/3598690842 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0343f0 con 0x7f5cb40ade80 2023-12-01T03:12:21.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.561+0000 7f5cbb20b640 1 -- 172.21.15.115:0/3598690842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb40ade80 msgr2=0x7f5cb406b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.561+0000 7f5cbb20b640 1 --2- 172.21.15.115:0/3598690842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb40ade80 0x7f5cb406b350 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40b1370 tx=0x7f5cac02be50 comp rx=0 tx=0).stop 2023-12-01T03:12:21.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.561+0000 7f5cbb20b640 1 -- 172.21.15.115:0/3598690842 shutdown_connections 2023-12-01T03:12:21.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.561+0000 7f5cbb20b640 1 --2- 172.21.15.115:0/3598690842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb40ade80 0x7f5cb406b350 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.561+0000 7f5cbb20b640 1 --2- 172.21.15.115:0/3598690842 >> v2:172.21.15.142:3300/0 conn(0x7f5cb40ad2b0 0x7f5cb40ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.561+0000 7f5cbb20b640 1 --2- 172.21.15.115:0/3598690842 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb415d220 0x7f5cb40cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.561+0000 7f5cbb20b640 1 -- 172.21.15.115:0/3598690842 >> 172.21.15.115:0/3598690842 conn(0x7f5cb40ce880 msgr2=0x7f5cb40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:21.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.561+0000 7f5cbb20b640 1 -- 172.21.15.115:0/3598690842 shutdown_connections 2023-12-01T03:12:21.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.561+0000 7f5cbb20b640 1 -- 172.21.15.115:0/3598690842 wait complete. 2023-12-01T03:12:21.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.563+0000 7f5cbb20b640 1 Processor -- start 2023-12-01T03:12:21.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.563+0000 7f5cbb20b640 1 -- start start 2023-12-01T03:12:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.564+0000 7f5cbb20b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cb40ad2b0 0x7f5cb4153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.564+0000 7f5cbb20b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb40ade80 0x7f5cb40ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.564+0000 7f5cbb20b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb415d220 0x7f5cb40c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.564+0000 7f5cbb20b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb40c9470 con 0x7f5cb415d220 2023-12-01T03:12:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.564+0000 7f5cbb20b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4151f90 con 0x7f5cb40ad2b0 2023-12-01T03:12:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.564+0000 7f5cbb20b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4152270 con 0x7f5cb40ade80 2023-12-01T03:12:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.564+0000 7f5cb97a8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb415d220 0x7f5cb40c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.564+0000 7f5cb97a8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb415d220 0x7f5cb40c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54256/0 (socket says 172.21.15.115:54256) 2023-12-01T03:12:21.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.564+0000 7f5cb97a8640 1 -- 172.21.15.115:0/1847884249 learned_addr learned my addr 172.21.15.115:0/1847884249 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb3fff640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb40ade80 0x7f5cb40ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb8fa7640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cb40ad2b0 0x7f5cb4153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb97a8640 1 -- 172.21.15.115:0/1847884249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb40ade80 msgr2=0x7f5cb40ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb97a8640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb40ade80 0x7f5cb40ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb97a8640 1 -- 172.21.15.115:0/1847884249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cb40ad2b0 msgr2=0x7f5cb4153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb97a8640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cb40ad2b0 0x7f5cb4153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb97a8640 1 -- 172.21.15.115:0/1847884249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac009b30 con 0x7f5cb415d220 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb3fff640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb40ade80 0x7f5cb40ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb97a8640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb415d220 0x7f5cb40c4c90 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7f5cac036040 tx=0x7f5cac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb17fa640 1 -- 172.21.15.115:0/1847884249 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac009510 con 0x7f5cb415d220 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb17fa640 1 -- 172.21.15.115:0/1847884249 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac03e450 con 0x7f5cb415d220 2023-12-01T03:12:21.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cbb20b640 1 -- 172.21.15.115:0/1847884249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb4152550 con 0x7f5cb415d220 2023-12-01T03:12:21.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.565+0000 7f5cb17fa640 1 -- 172.21.15.115:0/1847884249 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac03e800 con 0x7f5cb415d220 2023-12-01T03:12:21.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.566+0000 7f5cbb20b640 1 -- 172.21.15.115:0/1847884249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb4152b10 con 0x7f5cb415d220 2023-12-01T03:12:21.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.567+0000 7f5cbb20b640 1 -- 172.21.15.115:0/1847884249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb404e890 con 0x7f5cb415d220 2023-12-01T03:12:21.569 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.568+0000 7f5cb17fa640 1 -- 172.21.15.115:0/1847884249 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f5cac03e960 con 0x7f5cb415d220 2023-12-01T03:12:21.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.569+0000 7f5cb17fa640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ca006a670 0x7f5ca006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:21.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.569+0000 7f5cb17fa640 1 -- 172.21.15.115:0/1847884249 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6625+0+0 (secure 0 0 0) 0x7f5cac0abf50 con 0x7f5cb415d220 2023-12-01T03:12:21.570 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.569+0000 7f5cb8fa7640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ca006a670 0x7f5ca006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:21.571 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.569+0000 7f5cb8fa7640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ca006a670 0x7f5ca006cb30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8006fd0 tx=0x7f5ca8008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:21.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.573+0000 7f5cb17fa640 1 -- 172.21.15.115:0/1847884249 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5cac076da0 con 0x7f5cb415d220 2023-12-01T03:12:21.801 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.799+0000 7f5cbb20b640 1 -- 172.21.15.115:0/1847884249 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7f5cb4070470 con 0x7f5cb415d220 2023-12-01T03:12:21.802 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T03:12:21.801+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T03:12:21.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.816+0000 7f5cb17fa640 1 -- 172.21.15.115:0/1847884249 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v119) v1 ==== 167+0+0 (secure 0 0 0) 0x7f5cac07a3c0 con 0x7f5cb415d220 2023-12-01T03:12:21.818 INFO:teuthology.orchestra.run.smithi115.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T03:12:21.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 -- 172.21.15.115:0/1847884249 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ca006a670 msgr2=0x7f5ca006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ca006a670 0x7f5ca006cb30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8006fd0 tx=0x7f5ca8008040 comp rx=0 tx=0).stop 2023-12-01T03:12:21.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 -- 172.21.15.115:0/1847884249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb415d220 msgr2=0x7f5cb40c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:21.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb415d220 0x7f5cb40c4c90 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7f5cac036040 tx=0x7f5cac004a40 comp rx=0 tx=0).stop 2023-12-01T03:12:21.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 -- 172.21.15.115:0/1847884249 shutdown_connections 2023-12-01T03:12:21.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5ca006a670 0x7f5ca006cb30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cb40ade80 0x7f5cb40ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cb415d220 0x7f5cb40c4c90 unknown :-1 s=CLOSED pgs=2899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 --2- 172.21.15.115:0/1847884249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cb40ad2b0 0x7f5cb4153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:21.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 -- 172.21.15.115:0/1847884249 >> 172.21.15.115:0/1847884249 conn(0x7f5cb40ce880 msgr2=0x7f5cb41e3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:21.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 -- 172.21.15.115:0/1847884249 shutdown_connections 2023-12-01T03:12:21.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:21.819+0000 7f5cbb20b640 1 -- 172.21.15.115:0/1847884249 wait complete. 2023-12-01T03:12:21.837 INFO:tasks.ceph.mds.b.smithi142.stderr: -1753> 2023-12-01T03:11:52.615+0000 7f6916e75640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:12:21.839 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-12-01T03:12:21.921 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T03:12:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.047+0000 7f7c2f93c640 1 Processor -- start 2023-12-01T03:12:22.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.047+0000 7f7c2f93c640 1 -- start start 2023-12-01T03:12:22.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.048+0000 7f7c2f93c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c28142870 0x7f7c280ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:22.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.048+0000 7f7c2f93c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c280cd470 con 0x7f7c280b1e20 2023-12-01T03:12:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.048+0000 7f7c2f93c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c280b2f30 con 0x7f7c280cd010 2023-12-01T03:12:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.048+0000 7f7c2f93c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280b3ae0 con 0x7f7c28142870 2023-12-01T03:12:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.049+0000 7f7c2ded9640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7c280b1e20 0x7f7c2815a0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37784/0 (socket says 172.21.15.115:37784) 2023-12-01T03:12:22.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.049+0000 7f7c2ded9640 1 -- 172.21.15.115:0/4197181235 learned_addr learned my addr 172.21.15.115:0/4197181235 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:22.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.049+0000 7f7c2d6d8640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c28142870 0x7f7c280ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:22.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.049+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110189797 0 0) 0x7f7c280cd470 con 0x7f7c280b1e20 2023-12-01T03:12:22.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.049+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c10003520 con 0x7f7c280b1e20 2023-12-01T03:12:22.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1822904318 0 0) 0x7f7c10003520 con 0x7f7c280b1e20 2023-12-01T03:12:22.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c28142870 msgr2=0x7f7c280ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c28142870 0x7f7c280ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 >> v1:172.21.15.142:6789/0 conn(0x7f7c280cd010 legacy=0x7f7c280b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c280b2320 con 0x7f7c280b1e20 2023-12-01T03:12:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f7c20001670 con 0x7f7c280b1e20 2023-12-01T03:12:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 >> v1:172.21.15.115:6789/0 conn(0x7f7c280b1e20 legacy=0x7f7c2815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c100039a0 0x7f7c10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c28142870 0x7f7c100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c10004d00 0x7f7c10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280b3ae0 con 0x7f7c10004d00 2023-12-01T03:12:22.055 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1000ba30 con 0x7f7c100039a0 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1000bbf0 con 0x7f7c28142870 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c2ded9640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c10004d00 0x7f7c10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c2ced7640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c28142870 0x7f7c100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c2d6d8640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c100039a0 0x7f7c10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c2ded9640 1 -- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c28142870 msgr2=0x7f7c100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c2ded9640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c28142870 0x7f7c100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c2ded9640 1 -- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c100039a0 msgr2=0x7f7c10003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c2ded9640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c100039a0 0x7f7c10003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c2ded9640 1 -- 172.21.15.115:0/4197181235 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c10003520 con 0x7f7c10004d00 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.050+0000 7f7c2ced7640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c28142870 0x7f7c100047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.051+0000 7f7c2d6d8640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c100039a0 0x7f7c10003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:22.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c2ded9640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c10004d00 0x7f7c10005180 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f7c280b2250 tx=0x7f7c100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e643af073226b6f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:22.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20005ad0 con 0x7f7c10004d00 2023-12-01T03:12:22.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c20005c30 con 0x7f7c10004d00 2023-12-01T03:12:22.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c27fff640 1 -- 172.21.15.115:0/4197181235 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2002c850 con 0x7f7c10004d00 2023-12-01T03:12:22.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c2f93c640 1 -- 172.21.15.115:0/4197181235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c10004d00 msgr2=0x7f7c10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:22.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c2f93c640 1 --2- 172.21.15.115:0/4197181235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c10004d00 0x7f7c10005180 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f7c280b2250 tx=0x7f7c100057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:22.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c2ded9640 1 -- 172.21.15.115:0/4197181235 reap_dead start 2023-12-01T03:12:22.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c2f93c640 1 -- 172.21.15.115:0/4197181235 shutdown_connections 2023-12-01T03:12:22.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c2f93c640 1 -- 172.21.15.115:0/4197181235 >> 172.21.15.115:0/4197181235 conn(0x7f7c28065960 msgr2=0x7f7c28065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:22.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c2f93c640 1 -- 172.21.15.115:0/4197181235 shutdown_connections 2023-12-01T03:12:22.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.052+0000 7f7c2f93c640 1 -- 172.21.15.115:0/4197181235 wait complete. 2023-12-01T03:12:22.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.054+0000 7f7c2f93c640 1 Processor -- start 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.054+0000 7f7c2f93c640 1 -- start start 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.055+0000 7f7c2f93c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c10004d00 0x7f7c281349a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.055+0000 7f7c2f93c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c28157450 0x7f7c28134ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.055+0000 7f7c2f93c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c280cee10 0x7f7c28158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.055+0000 7f7c2f93c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c281528f0 con 0x7f7c280cee10 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.055+0000 7f7c2f93c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c28152bd0 con 0x7f7c28157450 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.055+0000 7f7c2f93c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c28152eb0 con 0x7f7c10004d00 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.055+0000 7f7c2ded9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c280cee10 0x7f7c28158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.055+0000 7f7c2ded9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c280cee10 0x7f7c28158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54270/0 (socket says 172.21.15.115:54270) 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.055+0000 7f7c2ded9640 1 -- 172.21.15.115:0/2300951266 learned_addr learned my addr 172.21.15.115:0/2300951266 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c2d6d8640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c10004d00 0x7f7c281349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:22.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c2ded9640 1 -- 172.21.15.115:0/2300951266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c10004d00 msgr2=0x7f7c281349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c2ced7640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c28157450 0x7f7c28134ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c2ded9640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c10004d00 0x7f7c281349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c2ded9640 1 -- 172.21.15.115:0/2300951266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c28157450 msgr2=0x7f7c28134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c2ded9640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c28157450 0x7f7c28134ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c2ded9640 1 -- 172.21.15.115:0/2300951266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c28153190 con 0x7f7c280cee10 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c2d6d8640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c10004d00 0x7f7c281349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c2ded9640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c280cee10 0x7f7c28158ac0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000c300 tx=0x7f7c2002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c25ffb640 1 -- 172.21.15.115:0/2300951266 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2003c040 con 0x7f7c280cee10 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c25ffb640 1 -- 172.21.15.115:0/2300951266 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c20001760 con 0x7f7c280cee10 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c25ffb640 1 -- 172.21.15.115:0/2300951266 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2002ce60 con 0x7f7c280cee10 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.056+0000 7f7c2f93c640 1 -- 172.21.15.115:0/2300951266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c28153470 con 0x7f7c280cee10 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.057+0000 7f7c2f93c640 1 -- 172.21.15.115:0/2300951266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c28152480 con 0x7f7c280cee10 2023-12-01T03:12:22.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.058+0000 7f7c2f93c640 1 -- 172.21.15.115:0/2300951266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c281526a0 con 0x7f7c280cee10 2023-12-01T03:12:22.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.059+0000 7f7c25ffb640 1 -- 172.21.15.115:0/2300951266 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f7c20032070 con 0x7f7c280cee10 2023-12-01T03:12:22.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.059+0000 7f7c25ffb640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c1406a670 0x7f7c1406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:22.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.060+0000 7f7c25ffb640 1 -- 172.21.15.115:0/2300951266 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6711+0+0 (secure 0 0 0) 0x7f7c200aa0a0 con 0x7f7c280cee10 2023-12-01T03:12:22.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.060+0000 7f7c2d6d8640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c1406a670 0x7f7c1406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:22.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.060+0000 7f7c2d6d8640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c1406a670 0x7f7c1406cb30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c18007520 tx=0x7f7c1800d040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:22.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.064+0000 7f7c25ffb640 1 -- 172.21.15.115:0/2300951266 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7c20074f70 con 0x7f7c280cee10 2023-12-01T03:12:22.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.292+0000 7f7c2f93c640 1 -- 172.21.15.115:0/2300951266 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7c2804e890 con 0x7f7c280cee10 2023-12-01T03:12:22.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.822+0000 7f7c25ffb640 1 -- 172.21.15.115:0/2300951266 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v120) v1 ==== 107+0+0 (secure 0 0 0) 0x7f7c200493a0 con 0x7f7c280cee10 2023-12-01T03:12:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.825+0000 7f7c2f93c640 1 -- 172.21.15.115:0/2300951266 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c1406a670 msgr2=0x7f7c1406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.825+0000 7f7c2f93c640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c1406a670 0x7f7c1406cb30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c18007520 tx=0x7f7c1800d040 comp rx=0 tx=0).stop 2023-12-01T03:12:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.825+0000 7f7c2f93c640 1 -- 172.21.15.115:0/2300951266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c280cee10 msgr2=0x7f7c28158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.825+0000 7f7c2f93c640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c280cee10 0x7f7c28158ac0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000c300 tx=0x7f7c2002e070 comp rx=0 tx=0).stop 2023-12-01T03:12:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.826+0000 7f7c2f93c640 1 -- 172.21.15.115:0/2300951266 shutdown_connections 2023-12-01T03:12:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.826+0000 7f7c2f93c640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c1406a670 0x7f7c1406cb30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.826+0000 7f7c2f93c640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c280cee10 0x7f7c28158ac0 unknown :-1 s=CLOSED pgs=2902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.826+0000 7f7c2f93c640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c28157450 0x7f7c28134ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.826+0000 7f7c2f93c640 1 --2- 172.21.15.115:0/2300951266 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c10004d00 0x7f7c281349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:22.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.826+0000 7f7c2f93c640 1 -- 172.21.15.115:0/2300951266 >> 172.21.15.115:0/2300951266 conn(0x7f7c28065960 msgr2=0x7f7c28141ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:22.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.826+0000 7f7c2f93c640 1 -- 172.21.15.115:0/2300951266 shutdown_connections 2023-12-01T03:12:22.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:22.826+0000 7f7c2f93c640 1 -- 172.21.15.115:0/2300951266 wait complete. 2023-12-01T03:12:22.848 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-12-01T03:12:23.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.070+0000 7fc5590e0640 1 Processor -- start 2023-12-01T03:12:23.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.070+0000 7fc5590e0640 1 -- start start 2023-12-01T03:12:23.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.072+0000 7fc5590e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5540b4d50 0x7fc5540b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:23.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.072+0000 7fc5590e0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc5540d2730 0x7fc5540d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:23.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.072+0000 7fc5590e0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5540d1460 con 0x7fc55415ce40 2023-12-01T03:12:23.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.072+0000 7fc5590e0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55415d220 con 0x7fc5540d2730 2023-12-01T03:12:23.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.072+0000 7fc5590e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540d1690 con 0x7fc5540b4d50 2023-12-01T03:12:23.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.072+0000 7fc55359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc5540d2730 0x7fc5540d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:23.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.072+0000 7fc55359e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc5540d2730 0x7fc5540d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54274/0 (socket says 172.21.15.115:54274) 2023-12-01T03:12:23.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.072+0000 7fc55359e640 1 -- 172.21.15.115:0/3182704692 learned_addr learned my addr 172.21.15.115:0/3182704692 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:23.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.072+0000 7fc54a59c640 1 --2- 172.21.15.115:0/3182704692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5540b4d50 0x7fc5540b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:23.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.073+0000 7fc55359e640 1 -- 172.21.15.115:0/3182704692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5540b4d50 msgr2=0x7fc5540b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:23.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.073+0000 7fc55359e640 1 --2- 172.21.15.115:0/3182704692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5540b4d50 0x7fc5540b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:23.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.073+0000 7fc55359e640 1 -- 172.21.15.115:0/3182704692 >> v1:172.21.15.142:6789/0 conn(0x7fc55415ce40 legacy=0x7fc5540b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:23.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.073+0000 7fc55359e640 1 -- 172.21.15.115:0/3182704692 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5540d1970 con 0x7fc5540d2730 2023-12-01T03:12:23.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.074+0000 7fc55359e640 1 --2- 172.21.15.115:0/3182704692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 0x7fc5540d0f20 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009e50 tx=0x7fc54c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfc3615269607ab server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:23.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.074+0000 7fc55259c640 1 -- 172.21.15.115:0/3182704692 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c02c9e0 con 0x7fc5540d2730 2023-12-01T03:12:23.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.074+0000 7fc55259c640 1 -- 172.21.15.115:0/3182704692 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c02cb40 con 0x7fc5540d2730 2023-12-01T03:12:23.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.074+0000 7fc55259c640 1 -- 172.21.15.115:0/3182704692 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c0343f0 con 0x7fc5540d2730 2023-12-01T03:12:23.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.075+0000 7fc5590e0640 1 -- 172.21.15.115:0/3182704692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 msgr2=0x7fc5540d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:23.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.075+0000 7fc5590e0640 1 --2- 172.21.15.115:0/3182704692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 0x7fc5540d0f20 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009e50 tx=0x7fc54c02be40 comp rx=0 tx=0).stop 2023-12-01T03:12:23.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.075+0000 7fc5590e0640 1 -- 172.21.15.115:0/3182704692 shutdown_connections 2023-12-01T03:12:23.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.075+0000 7fc5590e0640 1 --2- 172.21.15.115:0/3182704692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 0x7fc5540d0f20 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:23.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.075+0000 7fc5590e0640 1 --2- 172.21.15.115:0/3182704692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5540b4d50 0x7fc5540b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:23.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.075+0000 7fc5590e0640 1 -- 172.21.15.115:0/3182704692 >> 172.21.15.115:0/3182704692 conn(0x7fc554065f90 msgr2=0x7fc5540cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:23.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.075+0000 7fc5590e0640 1 -- 172.21.15.115:0/3182704692 shutdown_connections 2023-12-01T03:12:23.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.075+0000 7fc5590e0640 1 -- 172.21.15.115:0/3182704692 wait complete. 2023-12-01T03:12:23.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.077+0000 7fc5590e0640 1 Processor -- start 2023-12-01T03:12:23.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.077+0000 7fc5590e0640 1 -- start start 2023-12-01T03:12:23.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.078+0000 7fc5590e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5540b4d50 0x7fc5541f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:23.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.078+0000 7fc5590e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 0x7fc55415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:23.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.078+0000 7fc5590e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55415ce40 0x7fc5540cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:23.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.078+0000 7fc5590e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541f6d10 con 0x7fc5540d2730 2023-12-01T03:12:23.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.078+0000 7fc5590e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541f6470 con 0x7fc55415ce40 2023-12-01T03:12:23.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.078+0000 7fc5590e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541f6750 con 0x7fc5540b4d50 2023-12-01T03:12:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.078+0000 7fc54a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 0x7fc55415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.078+0000 7fc54a59c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 0x7fc55415e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54290/0 (socket says 172.21.15.115:54290) 2023-12-01T03:12:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.078+0000 7fc54a59c640 1 -- 172.21.15.115:0/4259508108 learned_addr learned my addr 172.21.15.115:0/4259508108 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc55359e640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55415ce40 0x7fc5540cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc552d9d640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5540b4d50 0x7fc5541f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc54a59c640 1 -- 172.21.15.115:0/4259508108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5540b4d50 msgr2=0x7fc5541f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc54a59c640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5540b4d50 0x7fc5541f5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc54a59c640 1 -- 172.21.15.115:0/4259508108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55415ce40 msgr2=0x7fc5540cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc54a59c640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55415ce40 0x7fc5540cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:23.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc54a59c640 1 -- 172.21.15.115:0/4259508108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c009b30 con 0x7fc5540d2730 2023-12-01T03:12:23.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc54a59c640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 0x7fc55415e620 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00d5f0 tx=0x7fc53c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:23.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc54bfff640 1 -- 172.21.15.115:0/4259508108 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0125f0 con 0x7fc5540d2730 2023-12-01T03:12:23.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc54bfff640 1 -- 172.21.15.115:0/4259508108 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c012750 con 0x7fc5540d2730 2023-12-01T03:12:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc54bfff640 1 -- 172.21.15.115:0/4259508108 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c012b00 con 0x7fc5540d2730 2023-12-01T03:12:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc5590e0640 1 -- 172.21.15.115:0/4259508108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5541f6a30 con 0x7fc5540d2730 2023-12-01T03:12:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.079+0000 7fc5590e0640 1 -- 172.21.15.115:0/4259508108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5540c9c90 con 0x7fc5540d2730 2023-12-01T03:12:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.081+0000 7fc5590e0640 1 -- 172.21.15.115:0/4259508108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc554074210 con 0x7fc5540d2730 2023-12-01T03:12:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.082+0000 7fc54bfff640 1 -- 172.21.15.115:0/4259508108 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fc53c012c60 con 0x7fc5540d2730 2023-12-01T03:12:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.082+0000 7fc54bfff640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc53406a6a0 0x7fc53406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.082+0000 7fc54bfff640 1 -- 172.21.15.115:0/4259508108 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6711+0+0 (secure 0 0 0) 0x7fc53c089550 con 0x7fc5540d2730 2023-12-01T03:12:23.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.083+0000 7fc552d9d640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc53406a6a0 0x7fc53406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:23.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.083+0000 7fc552d9d640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc53406a6a0 0x7fc53406cb60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc544003d90 tx=0x7fc544009040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:23.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.087+0000 7fc54bfff640 1 -- 172.21.15.115:0/4259508108 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc53c054420 con 0x7fc5540d2730 2023-12-01T03:12:23.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:23.277+0000 7fc5590e0640 1 -- 172.21.15.115:0/4259508108 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc554158530 con 0x7fc5540d2730 2023-12-01T03:12:24.281 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.279+0000 7fc54bfff640 1 -- 172.21.15.115:0/4259508108 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v246) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc53c057a40 con 0x7fc5540d2730 2023-12-01T03:12:24.281 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'backup_fs_metadata' removed 2023-12-01T03:12:24.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.282+0000 7fc5590e0640 1 -- 172.21.15.115:0/4259508108 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc53406a6a0 msgr2=0x7fc53406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:24.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.282+0000 7fc5590e0640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc53406a6a0 0x7fc53406cb60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc544003d90 tx=0x7fc544009040 comp rx=0 tx=0).stop 2023-12-01T03:12:24.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.282+0000 7fc5590e0640 1 -- 172.21.15.115:0/4259508108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 msgr2=0x7fc55415e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:24.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.282+0000 7fc5590e0640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 0x7fc55415e620 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00d5f0 tx=0x7fc53c00dab0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.282+0000 7fc5590e0640 1 -- 172.21.15.115:0/4259508108 shutdown_connections 2023-12-01T03:12:24.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.282+0000 7fc5590e0640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc53406a6a0 0x7fc53406cb60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.282+0000 7fc5590e0640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc55415ce40 0x7fc5540cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.282+0000 7fc5590e0640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5540d2730 0x7fc55415e620 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.282+0000 7fc5590e0640 1 --2- 172.21.15.115:0/4259508108 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5540b4d50 0x7fc5541f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.282+0000 7fc5590e0640 1 -- 172.21.15.115:0/4259508108 >> 172.21.15.115:0/4259508108 conn(0x7fc554065f90 msgr2=0x7fc5541541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:24.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.283+0000 7fc5590e0640 1 -- 172.21.15.115:0/4259508108 shutdown_connections 2023-12-01T03:12:24.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.283+0000 7fc5590e0640 1 -- 172.21.15.115:0/4259508108 wait complete. 2023-12-01T03:12:24.305 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-12-01T03:12:24.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.518+0000 7fb59ab30640 1 Processor -- start 2023-12-01T03:12:24.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.518+0000 7fb59ab30640 1 -- start start 2023-12-01T03:12:24.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.520+0000 7fb59ab30640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940cb850 0x7fb5940ae400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:24.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.520+0000 7fb59ab30640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5940ca030 con 0x7fb5941646b0 2023-12-01T03:12:24.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.520+0000 7fb59ab30640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5940c9140 con 0x7fb5940cab60 2023-12-01T03:12:24.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.520+0000 7fb59ab30640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb594070050 con 0x7fb5940cb850 2023-12-01T03:12:24.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.520+0000 7fb599b2e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb5941646b0 0x7fb5940cb140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37788/0 (socket says 172.21.15.115:37788) 2023-12-01T03:12:24.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.520+0000 7fb599b2e640 1 -- 172.21.15.115:0/338546311 learned_addr learned my addr 172.21.15.115:0/338546311 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:24.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.520+0000 7fb59932d640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940cb850 0x7fb5940ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:24.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580501213 0 0) 0x7fb5940ca030 con 0x7fb5941646b0 2023-12-01T03:12:24.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb580003520 con 0x7fb5941646b0 2023-12-01T03:12:24.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2584075623 0 0) 0x7fb580003520 con 0x7fb5941646b0 2023-12-01T03:12:24.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940cb850 msgr2=0x7fb5940ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:24.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940cb850 0x7fb5940ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 >> v1:172.21.15.142:6789/0 conn(0x7fb5940cab60 legacy=0x7fb5940ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:24.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940c9420 con 0x7fb5941646b0 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb590001670 con 0x7fb5941646b0 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 >> v1:172.21.15.115:6789/0 conn(0x7fb5941646b0 legacy=0x7fb5940cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5800039a0 0x7fb580003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940cb850 0x7fb5800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb580004d00 0x7fb580005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb594070050 con 0x7fb5800039a0 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58000bab0 con 0x7fb580004d00 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58000bc70 con 0x7fb5940cb850 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.521+0000 7fb599b2e640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5800039a0 0x7fb580003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.522+0000 7fb59932d640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940cb850 0x7fb5800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.522+0000 7fb59a32f640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb580004d00 0x7fb580005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.522+0000 7fb599b2e640 1 -- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940cb850 msgr2=0x7fb5800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.522+0000 7fb599b2e640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5940cb850 0x7fb5800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.522+0000 7fb599b2e640 1 -- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb580004d00 msgr2=0x7fb580005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:24.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.522+0000 7fb599b2e640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb580004d00 0x7fb580005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.522+0000 7fb599b2e640 1 -- 172.21.15.115:0/338546311 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb580003520 con 0x7fb5800039a0 2023-12-01T03:12:24.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.522+0000 7fb59a32f640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb580004d00 0x7fb580005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.523+0000 7fb599b2e640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5800039a0 0x7fb580003dc0 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fb59000c240 tx=0x7fb5800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf80948e2756538 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.523+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590005ad0 con 0x7fb5800039a0 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.523+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb590005c30 con 0x7fb5800039a0 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.523+0000 7fb598b2c640 1 -- 172.21.15.115:0/338546311 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59002c810 con 0x7fb5800039a0 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.524+0000 7fb59ab30640 1 -- 172.21.15.115:0/338546311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5800039a0 msgr2=0x7fb580003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.524+0000 7fb59ab30640 1 --2- 172.21.15.115:0/338546311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5800039a0 0x7fb580003dc0 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fb59000c240 tx=0x7fb5800057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.524+0000 7fb59a32f640 1 -- 172.21.15.115:0/338546311 reap_dead start 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.524+0000 7fb59ab30640 1 -- 172.21.15.115:0/338546311 shutdown_connections 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.524+0000 7fb59ab30640 1 -- 172.21.15.115:0/338546311 >> 172.21.15.115:0/338546311 conn(0x7fb594065960 msgr2=0x7fb594065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.524+0000 7fb59ab30640 1 -- 172.21.15.115:0/338546311 shutdown_connections 2023-12-01T03:12:24.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.524+0000 7fb59ab30640 1 -- 172.21.15.115:0/338546311 wait complete. 2023-12-01T03:12:24.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.526+0000 7fb59ab30640 1 Processor -- start 2023-12-01T03:12:24.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.526+0000 7fb59ab30640 1 -- start start 2023-12-01T03:12:24.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb59ab30640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5800039a0 0x7fb594162c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:24.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb59ab30640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5940c0b20 0x7fb5940c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:24.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb59ab30640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5940c0fa0 0x7fb5940c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:24.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb59ab30640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940c80f0 con 0x7fb5940c0b20 2023-12-01T03:12:24.529 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb59ab30640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940c61e0 con 0x7fb5940c0fa0 2023-12-01T03:12:24.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb59ab30640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940c6460 con 0x7fb5800039a0 2023-12-01T03:12:24.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb59932d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5940c0b20 0x7fb5940c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:24.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb59932d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5940c0b20 0x7fb5940c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54318/0 (socket says 172.21.15.115:54318) 2023-12-01T03:12:24.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb59932d640 1 -- 172.21.15.115:0/1766563882 learned_addr learned my addr 172.21.15.115:0/1766563882 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:24.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb59a32f640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5940c0fa0 0x7fb5940c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:24.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.527+0000 7fb599b2e640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5800039a0 0x7fb594162c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:24.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb59932d640 1 -- 172.21.15.115:0/1766563882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5800039a0 msgr2=0x7fb594162c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:24.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb59932d640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5800039a0 0x7fb594162c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb59932d640 1 -- 172.21.15.115:0/1766563882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5940c0fa0 msgr2=0x7fb5940c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:24.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb59932d640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5940c0fa0 0x7fb5940c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:24.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb59932d640 1 -- 172.21.15.115:0/1766563882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940c6740 con 0x7fb5940c0b20 2023-12-01T03:12:24.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb59932d640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5940c0b20 0x7fb5940c7460 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fb5840115d0 tx=0x7fb584011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:24.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb58a7fc640 1 -- 172.21.15.115:0/1766563882 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58400f8f0 con 0x7fb5940c0b20 2023-12-01T03:12:24.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb58a7fc640 1 -- 172.21.15.115:0/1766563882 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58400fa50 con 0x7fb5940c0b20 2023-12-01T03:12:24.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb58a7fc640 1 -- 172.21.15.115:0/1766563882 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58400fdc0 con 0x7fb5940c0b20 2023-12-01T03:12:24.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb59ab30640 1 -- 172.21.15.115:0/1766563882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5940c6a20 con 0x7fb5940c0b20 2023-12-01T03:12:24.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.528+0000 7fb59ab30640 1 -- 172.21.15.115:0/1766563882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5940afd10 con 0x7fb5940c0b20 2023-12-01T03:12:24.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.530+0000 7fb59ab30640 1 -- 172.21.15.115:0/1766563882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5940618d0 con 0x7fb5940c0b20 2023-12-01T03:12:24.532 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.531+0000 7fb58a7fc640 1 -- 172.21.15.115:0/1766563882 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fb584015460 con 0x7fb5940c0b20 2023-12-01T03:12:24.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.532+0000 7fb58a7fc640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb57806a6a0 0x7fb57806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:24.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.532+0000 7fb58a7fc640 1 -- 172.21.15.115:0/1766563882 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6267+0+0 (secure 0 0 0) 0x7fb584089370 con 0x7fb5940c0b20 2023-12-01T03:12:24.533 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.532+0000 7fb599b2e640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb57806a6a0 0x7fb57806cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:24.534 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.533+0000 7fb599b2e640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb57806a6a0 0x7fb57806cb60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb590034000 tx=0x7fb590009030 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:24.537 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.536+0000 7fb58a7fc640 1 -- 172.21.15.115:0/1766563882 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb5840543c0 con 0x7fb5940c0b20 2023-12-01T03:12:24.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:24.727+0000 7fb59ab30640 1 -- 172.21.15.115:0/1766563882 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb5940c3100 con 0x7fb5940c0b20 2023-12-01T03:12:25.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.285+0000 7fb58a7fc640 1 -- 172.21.15.115:0/1766563882 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v247) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb5840579e0 con 0x7fb5940c0b20 2023-12-01T03:12:25.288 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'backup_fs_data' removed 2023-12-01T03:12:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 -- 172.21.15.115:0/1766563882 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb57806a6a0 msgr2=0x7fb57806cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb57806a6a0 0x7fb57806cb60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb590034000 tx=0x7fb590009030 comp rx=0 tx=0).stop 2023-12-01T03:12:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 -- 172.21.15.115:0/1766563882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5940c0b20 msgr2=0x7fb5940c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5940c0b20 0x7fb5940c7460 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fb5840115d0 tx=0x7fb584011a90 comp rx=0 tx=0).stop 2023-12-01T03:12:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 -- 172.21.15.115:0/1766563882 shutdown_connections 2023-12-01T03:12:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb57806a6a0 0x7fb57806cb60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5940c0fa0 0x7fb5940c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5940c0b20 0x7fb5940c7460 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 --2- 172.21.15.115:0/1766563882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5800039a0 0x7fb594162c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 -- 172.21.15.115:0/1766563882 >> 172.21.15.115:0/1766563882 conn(0x7fb594065960 msgr2=0x7fb594163c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 -- 172.21.15.115:0/1766563882 shutdown_connections 2023-12-01T03:12:25.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.290+0000 7fb59ab30640 1 -- 172.21.15.115:0/1766563882 wait complete. 2023-12-01T03:12:25.317 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T03:12:25.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.508+0000 7f8772111640 1 Processor -- start 2023-12-01T03:12:25.510 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.508+0000 7f8772111640 1 -- start start 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.510+0000 7f8772111640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f876c0b0e60 0x7f876c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.510+0000 7f8772111640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0cc9a0 con 0x7f876c0ce1b0 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.510+0000 7f8772111640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0cbd90 con 0x7f876c15ac80 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.510+0000 7f8772111640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876c15b060 con 0x7f876c0b0e60 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.510+0000 7f8763fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f876c0ce1b0 0x7f876c06b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37798/0 (socket says 172.21.15.115:37798) 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.510+0000 7f8763fff640 1 -- 172.21.15.115:0/2288795432 learned_addr learned my addr 172.21.15.115:0/2288795432 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.510+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833626501 0 0) 0x7f876c0cc9a0 con 0x7f876c0ce1b0 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.510+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8758003520 con 0x7f876c0ce1b0 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.510+0000 7f876bfff640 1 --2- 172.21.15.115:0/2288795432 >> v2:172.21.15.142:3300/0 conn(0x7f876c0b0e60 0x7f876c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2604225930 0 0) 0x7f8758003520 con 0x7f876c0ce1b0 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 >> v2:172.21.15.142:3300/0 conn(0x7f876c0b0e60 msgr2=0x7f876c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 --2- 172.21.15.115:0/2288795432 >> v2:172.21.15.142:3300/0 conn(0x7f876c0b0e60 0x7f876c0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 >> v1:172.21.15.142:6789/0 conn(0x7f876c15ac80 legacy=0x7f876c0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:25.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c0cc070 con 0x7f876c0ce1b0 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f8754001670 con 0x7f876c0ce1b0 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 >> v1:172.21.15.115:6789/0 conn(0x7f876c0ce1b0 legacy=0x7f876c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8758003970 0x7f8758003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87580047c0 0x7f8758004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8758005180 0x7f875800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c15b060 con 0x7f8758003970 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875800c0b0 con 0x7f87580047c0 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875800c270 con 0x7f8758005180 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f876b7fe640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8758003970 0x7f8758003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.511+0000 7f8763fff640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87580047c0 0x7f8758004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.512+0000 7f876b7fe640 1 -- 172.21.15.115:0/2288795432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8758005180 msgr2=0x7f875800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.512+0000 7f876bfff640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8758005180 0x7f875800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.512+0000 7f876b7fe640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8758005180 0x7f875800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.512+0000 7f876b7fe640 1 -- 172.21.15.115:0/2288795432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87580047c0 msgr2=0x7f8758004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:25.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.512+0000 7f876b7fe640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87580047c0 0x7f8758004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.512+0000 7f876b7fe640 1 -- 172.21.15.115:0/2288795432 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758003520 con 0x7f8758003970 2023-12-01T03:12:25.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.512+0000 7f8763fff640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87580047c0 0x7f8758004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:25.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.512+0000 7f876bfff640 1 -- 172.21.15.115:0/2288795432 reap_dead start 2023-12-01T03:12:25.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.513+0000 7f876b7fe640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8758003970 0x7f8758003d70 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f875c003d80 tx=0x7f87580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e08a4569a98669c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:25.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.513+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c032600 con 0x7f8758003970 2023-12-01T03:12:25.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.513+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c02e050 con 0x7f8758003970 2023-12-01T03:12:25.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.513+0000 7f876affd640 1 -- 172.21.15.115:0/2288795432 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0329e0 con 0x7f8758003970 2023-12-01T03:12:25.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.513+0000 7f8772111640 1 -- 172.21.15.115:0/2288795432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8758003970 msgr2=0x7f8758003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.513+0000 7f8772111640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8758003970 0x7f8758003d70 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f875c003d80 tx=0x7f87580057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.514+0000 7f8772111640 1 -- 172.21.15.115:0/2288795432 shutdown_connections 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.514+0000 7f8772111640 1 --2- 172.21.15.115:0/2288795432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8758003970 0x7f8758003d70 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.514+0000 7f8772111640 1 -- 172.21.15.115:0/2288795432 >> 172.21.15.115:0/2288795432 conn(0x7f876c065960 msgr2=0x7f876c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.514+0000 7f8772111640 1 -- 172.21.15.115:0/2288795432 shutdown_connections 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.514+0000 7f8772111640 1 -- 172.21.15.115:0/2288795432 wait complete. 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.515+0000 7f8772111640 1 Processor -- start 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.516+0000 7f8772111640 1 -- start start 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.516+0000 7f8772111640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8758003970 0x7f876c1503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.516+0000 7f8772111640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f876c06b500 0x7f876c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.516+0000 7f8772111640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c1532c0 0x7f876c153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.516+0000 7f8772111640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c153f30 con 0x7f876c06b500 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.516+0000 7f8772111640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c06bec0 con 0x7f876c1532c0 2023-12-01T03:12:25.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.516+0000 7f8772111640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c1f46d0 con 0x7f8758003970 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.517+0000 7f8763fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f876c06b500 0x7f876c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.517+0000 7f8763fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f876c06b500 0x7f876c06b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54334/0 (socket says 172.21.15.115:54334) 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.517+0000 7f8763fff640 1 -- 172.21.15.115:0/598776526 learned_addr learned my addr 172.21.15.115:0/598776526 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.517+0000 7f876b7fe640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8758003970 0x7f876c1503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.517+0000 7f8763fff640 1 -- 172.21.15.115:0/598776526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8758003970 msgr2=0x7f876c1503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.517+0000 7f8763fff640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8758003970 0x7f876c1503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.517+0000 7f8763fff640 1 -- 172.21.15.115:0/598776526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c1532c0 msgr2=0x7f876c153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.517+0000 7f8763fff640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c1532c0 0x7f876c153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.517+0000 7f8763fff640 1 -- 172.21.15.115:0/598776526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c1f0d30 con 0x7f876c06b500 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.517+0000 7f8763fff640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f876c06b500 0x7f876c06b980 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f875400d840 tx=0x7f875400dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.518+0000 7f8768ff9640 1 -- 172.21.15.115:0/598776526 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540074e0 con 0x7f876c06b500 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.518+0000 7f8768ff9640 1 -- 172.21.15.115:0/598776526 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8754007640 con 0x7f876c06b500 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.518+0000 7f8768ff9640 1 -- 172.21.15.115:0/598776526 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540049e0 con 0x7f876c06b500 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.518+0000 7f8772111640 1 -- 172.21.15.115:0/598776526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c1f01a0 con 0x7f876c06b500 2023-12-01T03:12:25.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.518+0000 7f8772111640 1 -- 172.21.15.115:0/598776526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c153c80 con 0x7f876c06b500 2023-12-01T03:12:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.519+0000 7f8772111640 1 -- 172.21.15.115:0/598776526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c1f0ec0 con 0x7f876c06b500 2023-12-01T03:12:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.520+0000 7f8768ff9640 1 -- 172.21.15.115:0/598776526 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f8754004b40 con 0x7f876c06b500 2023-12-01T03:12:25.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.521+0000 7f8768ff9640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f874c06a6a0 0x7f874c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.521+0000 7f8768ff9640 1 -- 172.21.15.115:0/598776526 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f8754088940 con 0x7f876c06b500 2023-12-01T03:12:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.521+0000 7f876b7fe640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f874c06a6a0 0x7f874c06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.521+0000 7f876b7fe640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f874c06a6a0 0x7f874c06cb60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f875c034000 tx=0x7f875c009030 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:25.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.525+0000 7f8768ff9640 1 -- 172.21.15.115:0/598776526 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f87540539a0 con 0x7f876c06b500 2023-12-01T03:12:25.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.714+0000 7f8772111640 1 -- 172.21.15.115:0/598776526 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f876c04e890 con 0x7f876c06b500 2023-12-01T03:12:25.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.723+0000 7f8768ff9640 1 -- 172.21.15.115:0/598776526 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7f87540190a0 con 0x7f876c06b500 2023-12-01T03:12:25.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.725+0000 7f8772111640 1 -- 172.21.15.115:0/598776526 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f874c06a6a0 msgr2=0x7f874c06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.725+0000 7f8772111640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f874c06a6a0 0x7f874c06cb60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f875c034000 tx=0x7f875c009030 comp rx=0 tx=0).stop 2023-12-01T03:12:25.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.725+0000 7f8772111640 1 -- 172.21.15.115:0/598776526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f876c06b500 msgr2=0x7f876c06b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.725+0000 7f8772111640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f876c06b500 0x7f876c06b980 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f875400d840 tx=0x7f875400dd00 comp rx=0 tx=0).stop 2023-12-01T03:12:25.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.726+0000 7f8772111640 1 -- 172.21.15.115:0/598776526 shutdown_connections 2023-12-01T03:12:25.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.726+0000 7f8772111640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f874c06a6a0 0x7f874c06cb60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.726+0000 7f8772111640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f876c1532c0 0x7f876c153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.726+0000 7f8772111640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f876c06b500 0x7f876c06b980 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.726+0000 7f8772111640 1 --2- 172.21.15.115:0/598776526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8758003970 0x7f876c1503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.726+0000 7f8772111640 1 -- 172.21.15.115:0/598776526 >> 172.21.15.115:0/598776526 conn(0x7f876c065960 msgr2=0x7f876c15d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:25.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.726+0000 7f8772111640 1 -- 172.21.15.115:0/598776526 shutdown_connections 2023-12-01T03:12:25.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.726+0000 7f8772111640 1 -- 172.21.15.115:0/598776526 wait complete. 2023-12-01T03:12:25.745 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T03:12:25.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.969+0000 7f829838b640 1 Processor -- start 2023-12-01T03:12:25.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.969+0000 7f829838b640 1 -- start start 2023-12-01T03:12:25.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f829838b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f829015cd90 0x7f82900cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f829838b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82900b5fa0 0x7f8290161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f829838b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82900cf650 0x7f82900cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f829838b640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82900d0ec0 con 0x7f82900cf650 2023-12-01T03:12:25.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f829838b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82900ce5e0 con 0x7f82900b5fa0 2023-12-01T03:12:25.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f829838b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8290161950 con 0x7f829015cd90 2023-12-01T03:12:25.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f8296928640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82900cf650 0x7f82900cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f8296928640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f82900cf650 0x7f82900cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54346/0 (socket says 172.21.15.115:54346) 2023-12-01T03:12:25.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f8296928640 1 -- 172.21.15.115:0/2832978549 learned_addr learned my addr 172.21.15.115:0/2832978549 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:25.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f8296127640 1 --2- 172.21.15.115:0/2832978549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f829015cd90 0x7f82900cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.971+0000 7f8295926640 1 --2- 172.21.15.115:0/2832978549 >> v2:172.21.15.142:3300/0 conn(0x7f82900b5fa0 0x7f8290161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.972+0000 7f8296928640 1 -- 172.21.15.115:0/2832978549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f829015cd90 msgr2=0x7f82900cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:25.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.972+0000 7f8296928640 1 --2- 172.21.15.115:0/2832978549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f829015cd90 0x7f82900cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.972+0000 7f8296928640 1 -- 172.21.15.115:0/2832978549 >> v2:172.21.15.142:3300/0 conn(0x7f82900b5fa0 msgr2=0x7f8290161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:25.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.972+0000 7f8296928640 1 --2- 172.21.15.115:0/2832978549 >> v2:172.21.15.142:3300/0 conn(0x7f82900b5fa0 0x7f8290161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.972+0000 7f8296928640 1 -- 172.21.15.115:0/2832978549 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82900d0000 con 0x7f82900cf650 2023-12-01T03:12:25.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.973+0000 7f8296928640 1 --2- 172.21.15.115:0/2832978549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900cf650 0x7f82900cfac0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f82900b5880 tx=0x7f828802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf70c73b4cb7385 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:25.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.973+0000 7f8295125640 1 -- 172.21.15.115:0/2832978549 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828802ca70 con 0x7f82900cf650 2023-12-01T03:12:25.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.973+0000 7f8295125640 1 -- 172.21.15.115:0/2832978549 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828802cbd0 con 0x7f82900cf650 2023-12-01T03:12:25.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.973+0000 7f8295125640 1 -- 172.21.15.115:0/2832978549 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82880343f0 con 0x7f82900cf650 2023-12-01T03:12:25.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.974+0000 7f829838b640 1 -- 172.21.15.115:0/2832978549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900cf650 msgr2=0x7f82900cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.974+0000 7f829838b640 1 --2- 172.21.15.115:0/2832978549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900cf650 0x7f82900cfac0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f82900b5880 tx=0x7f828802bed0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.974+0000 7f829838b640 1 -- 172.21.15.115:0/2832978549 shutdown_connections 2023-12-01T03:12:25.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.974+0000 7f829838b640 1 --2- 172.21.15.115:0/2832978549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900cf650 0x7f82900cfac0 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.974+0000 7f829838b640 1 --2- 172.21.15.115:0/2832978549 >> v2:172.21.15.142:3300/0 conn(0x7f82900b5fa0 0x7f8290161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.974+0000 7f829838b640 1 --2- 172.21.15.115:0/2832978549 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f829015cd90 0x7f82900cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.974+0000 7f829838b640 1 -- 172.21.15.115:0/2832978549 >> 172.21.15.115:0/2832978549 conn(0x7f8290065960 msgr2=0x7f8290065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:25.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.974+0000 7f829838b640 1 -- 172.21.15.115:0/2832978549 shutdown_connections 2023-12-01T03:12:25.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.974+0000 7f829838b640 1 -- 172.21.15.115:0/2832978549 wait complete. 2023-12-01T03:12:25.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.976+0000 7f829838b640 1 Processor -- start 2023-12-01T03:12:25.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.976+0000 7f829838b640 1 -- start start 2023-12-01T03:12:25.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.977+0000 7f829838b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900b5fa0 0x7f829015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.977+0000 7f829838b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82900cf650 0x7f82900c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.977+0000 7f829838b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f829015cd90 0x7f82901f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.977+0000 7f829838b640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829015b9a0 con 0x7f82900b5fa0 2023-12-01T03:12:25.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.977+0000 7f829838b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82900c9460 con 0x7f829015cd90 2023-12-01T03:12:25.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.977+0000 7f829838b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82901585c0 con 0x7f82900cf650 2023-12-01T03:12:25.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.977+0000 7f8296127640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900b5fa0 0x7f829015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.977+0000 7f8296127640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900b5fa0 0x7f829015b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54354/0 (socket says 172.21.15.115:54354) 2023-12-01T03:12:25.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.977+0000 7f8296127640 1 -- 172.21.15.115:0/1621631046 learned_addr learned my addr 172.21.15.115:0/1621631046 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f8296928640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f829015cd90 0x7f82901f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f8295926640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82900cf650 0x7f82900c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f8296127640 1 -- 172.21.15.115:0/1621631046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82900cf650 msgr2=0x7f82900c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f8296127640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82900cf650 0x7f82900c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f8296127640 1 -- 172.21.15.115:0/1621631046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f829015cd90 msgr2=0x7f82901f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f8296127640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f829015cd90 0x7f82901f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f8296127640 1 -- 172.21.15.115:0/1621631046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288009b30 con 0x7f82900b5fa0 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f8295926640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82900cf650 0x7f82900c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f8296127640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900b5fa0 0x7f829015b460 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f828000d800 tx=0x7f828000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f828effd640 1 -- 172.21.15.115:0/1621631046 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828000b6c0 con 0x7f82900b5fa0 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.978+0000 7f828effd640 1 -- 172.21.15.115:0/1621631046 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828000b820 con 0x7f82900b5fa0 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.979+0000 7f828effd640 1 -- 172.21.15.115:0/1621631046 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828000bbb0 con 0x7f82900b5fa0 2023-12-01T03:12:25.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.979+0000 7f829838b640 1 -- 172.21.15.115:0/1621631046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82901588a0 con 0x7f82900b5fa0 2023-12-01T03:12:25.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.979+0000 7f829838b640 1 -- 172.21.15.115:0/1621631046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8290158e60 con 0x7f82900b5fa0 2023-12-01T03:12:25.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.980+0000 7f829838b640 1 -- 172.21.15.115:0/1621631046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82900d18a0 con 0x7f82900b5fa0 2023-12-01T03:12:25.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.981+0000 7f828effd640 1 -- 172.21.15.115:0/1621631046 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f828000bd10 con 0x7f82900b5fa0 2023-12-01T03:12:25.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.981+0000 7f828effd640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f827c06a670 0x7f827c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:25.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.982+0000 7f828effd640 1 -- 172.21.15.115:0/1621631046 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f8280089250 con 0x7f82900b5fa0 2023-12-01T03:12:25.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.982+0000 7f8295926640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f827c06a670 0x7f827c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:25.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.982+0000 7f8295926640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f827c06a670 0x7f827c06cb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8284006fd0 tx=0x7f8284008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:25.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:25.986+0000 7f828effd640 1 -- 172.21.15.115:0/1621631046 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f82800544e0 con 0x7f82900b5fa0 2023-12-01T03:12:26.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.179+0000 7f829838b640 1 -- 172.21.15.115:0/1621631046 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f82901606b0 con 0x7f82900b5fa0 2023-12-01T03:12:26.189 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.188+0000 7f828effd640 1 -- 172.21.15.115:0/1621631046 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v80) v1 ==== 120+0+0 (secure 0 0 0) 0x7f8280057b00 con 0x7f82900b5fa0 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 -- 172.21.15.115:0/1621631046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f827c06a670 msgr2=0x7f827c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f827c06a670 0x7f827c06cb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8284006fd0 tx=0x7f8284008040 comp rx=0 tx=0).stop 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 -- 172.21.15.115:0/1621631046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900b5fa0 msgr2=0x7f829015b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900b5fa0 0x7f829015b460 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f828000d800 tx=0x7f828000dcc0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 -- 172.21.15.115:0/1621631046 shutdown_connections 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f827c06a670 0x7f827c06cb30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82900cf650 0x7f82900c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f829015cd90 0x7f82901f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 --2- 172.21.15.115:0/1621631046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82900b5fa0 0x7f829015b460 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 -- 172.21.15.115:0/1621631046 >> 172.21.15.115:0/1621631046 conn(0x7f8290065960 msgr2=0x7f82900d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:26.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 -- 172.21.15.115:0/1621631046 shutdown_connections 2023-12-01T03:12:26.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.191+0000 7f829838b640 1 -- 172.21.15.115:0/1621631046 wait complete. 2023-12-01T03:12:26.212 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T03:12:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.431+0000 7f3f97939640 1 Processor -- start 2023-12-01T03:12:26.433 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.431+0000 7f3f97939640 1 -- start start 2023-12-01T03:12:26.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.433+0000 7f3f97939640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f900b0e60 0x7f3f900cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.433+0000 7f3f97939640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f900cc9a0 con 0x7f3f900ce1b0 2023-12-01T03:12:26.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.433+0000 7f3f97939640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f900cbd90 con 0x7f3f9015ac80 2023-12-01T03:12:26.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.433+0000 7f3f97939640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9015b060 con 0x7f3f900b0e60 2023-12-01T03:12:26.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.433+0000 7f3f94ed4640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3f900ce1b0 0x7f3f9006b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37804/0 (socket says 172.21.15.115:37804) 2023-12-01T03:12:26.435 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.433+0000 7f3f94ed4640 1 -- 172.21.15.115:0/2021145752 learned_addr learned my addr 172.21.15.115:0/2021145752 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:26.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.433+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836566784 0 0) 0x7f3f900cc9a0 con 0x7f3f900ce1b0 2023-12-01T03:12:26.437 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f95ed6640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f900b0e60 0x7f3f900cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f7c003520 con 0x7f3f900ce1b0 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1340090972 0 0) 0x7f3f7c003520 con 0x7f3f900ce1b0 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f900b0e60 msgr2=0x7f3f900cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f900b0e60 0x7f3f900cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 >> v1:172.21.15.142:6789/0 conn(0x7f3f9015ac80 legacy=0x7f3f900cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f900cc070 con 0x7f3f900ce1b0 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3f80001670 con 0x7f3f900ce1b0 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 >> v1:172.21.15.115:6789/0 conn(0x7f3f900ce1b0 legacy=0x7f3f9006b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f7c0039a0 0x7f3f7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f900b0e60 0x7f3f7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f7c004d00 0x7f3f7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9015b060 con 0x7f3f7c0039a0 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c00ba60 con 0x7f3f7c004d00 2023-12-01T03:12:26.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c00bc20 con 0x7f3f900b0e60 2023-12-01T03:12:26.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.434+0000 7f3f956d5640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f7c0039a0 0x7f3f7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.435+0000 7f3f94ed4640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f900b0e60 0x7f3f7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.435+0000 7f3f956d5640 1 -- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f900b0e60 msgr2=0x7f3f7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:26.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.435+0000 7f3f956d5640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f900b0e60 0x7f3f7c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.435+0000 7f3f956d5640 1 -- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f7c004d00 msgr2=0x7f3f7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:26.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.435+0000 7f3f956d5640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f7c004d00 0x7f3f7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.435+0000 7f3f956d5640 1 -- 172.21.15.115:0/2021145752 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c003520 con 0x7f3f7c0039a0 2023-12-01T03:12:26.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.435+0000 7f3f94ed4640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f900b0e60 0x7f3f7c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:26.439 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.436+0000 7f3f956d5640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f7c0039a0 0x7f3f7c003dc0 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c001720 tx=0x7f3f7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cef7b29894a883 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.436+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c0052d0 con 0x7f3f7c0039a0 2023-12-01T03:12:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.436+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c02e050 con 0x7f3f7c0039a0 2023-12-01T03:12:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.436+0000 7f3f87fff640 1 -- 172.21.15.115:0/2021145752 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c0035f0 con 0x7f3f7c0039a0 2023-12-01T03:12:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.437+0000 7f3f97939640 1 -- 172.21.15.115:0/2021145752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f7c0039a0 msgr2=0x7f3f7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.440 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.437+0000 7f3f97939640 1 --2- 172.21.15.115:0/2021145752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f7c0039a0 0x7f3f7c003dc0 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c001720 tx=0x7f3f7c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.437+0000 7f3f95ed6640 1 -- 172.21.15.115:0/2021145752 reap_dead start 2023-12-01T03:12:26.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.437+0000 7f3f97939640 1 -- 172.21.15.115:0/2021145752 shutdown_connections 2023-12-01T03:12:26.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.437+0000 7f3f97939640 1 -- 172.21.15.115:0/2021145752 >> 172.21.15.115:0/2021145752 conn(0x7f3f90065960 msgr2=0x7f3f90065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:26.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.437+0000 7f3f97939640 1 -- 172.21.15.115:0/2021145752 shutdown_connections 2023-12-01T03:12:26.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.437+0000 7f3f97939640 1 -- 172.21.15.115:0/2021145752 wait complete. 2023-12-01T03:12:26.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.439+0000 7f3f97939640 1 Processor -- start 2023-12-01T03:12:26.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.439+0000 7f3f97939640 1 -- start start 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f97939640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f7c0039a0 0x7f3f9014bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f97939640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90150490 0x7f3f90151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f97939640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f901522f0 0x7f3f90152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f97939640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9015b4f0 con 0x7f3f90150490 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f97939640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90154320 con 0x7f3f901522f0 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f97939640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90152cb0 con 0x7f3f7c0039a0 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f94ed4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90150490 0x7f3f90151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f94ed4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90150490 0x7f3f90151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54378/0 (socket says 172.21.15.115:54378) 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f94ed4640 1 -- 172.21.15.115:0/1356934965 learned_addr learned my addr 172.21.15.115:0/1356934965 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f956d5640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f7c0039a0 0x7f3f9014bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.443 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f94ed4640 1 -- 172.21.15.115:0/1356934965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f7c0039a0 msgr2=0x7f3f9014bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f94ed4640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f7c0039a0 0x7f3f9014bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f94ed4640 1 -- 172.21.15.115:0/1356934965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f901522f0 msgr2=0x7f3f90152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f94ed4640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f901522f0 0x7f3f90152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.440+0000 7f3f94ed4640 1 -- 172.21.15.115:0/1356934965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9015b1a0 con 0x7f3f90150490 2023-12-01T03:12:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.441+0000 7f3f94ed4640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90150490 0x7f3f90151ac0 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000d840 tx=0x7f3f8000dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.441+0000 7f3f85ffb640 1 -- 172.21.15.115:0/1356934965 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f800074e0 con 0x7f3f90150490 2023-12-01T03:12:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.441+0000 7f3f85ffb640 1 -- 172.21.15.115:0/1356934965 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f80007640 con 0x7f3f90150490 2023-12-01T03:12:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.441+0000 7f3f97939640 1 -- 172.21.15.115:0/1356934965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9014c4b0 con 0x7f3f90150490 2023-12-01T03:12:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.441+0000 7f3f85ffb640 1 -- 172.21.15.115:0/1356934965 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80004a00 con 0x7f3f90150490 2023-12-01T03:12:26.444 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.441+0000 7f3f97939640 1 -- 172.21.15.115:0/1356934965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f90150cc0 con 0x7f3f90150490 2023-12-01T03:12:26.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.442+0000 7f3f97939640 1 -- 172.21.15.115:0/1356934965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9015b630 con 0x7f3f90150490 2023-12-01T03:12:26.445 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.443+0000 7f3f85ffb640 1 -- 172.21.15.115:0/1356934965 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f3f80004b60 con 0x7f3f90150490 2023-12-01T03:12:26.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.444+0000 7f3f85ffb640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7406a670 0x7f3f7406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.444+0000 7f3f85ffb640 1 -- 172.21.15.115:0/1356934965 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f3f80088b30 con 0x7f3f90150490 2023-12-01T03:12:26.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.444+0000 7f3f956d5640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7406a670 0x7f3f7406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.446 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.445+0000 7f3f956d5640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7406a670 0x7f3f7406cb30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c033000 tx=0x7f3f8c009030 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:26.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.449+0000 7f3f85ffb640 1 -- 172.21.15.115:0/1356934965 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3f9015b630 con 0x7f3f90150490 2023-12-01T03:12:26.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.637+0000 7f3f97939640 1 -- 172.21.15.115:0/1356934965 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f3f9015b630 con 0x7f3f90150490 2023-12-01T03:12:26.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.646+0000 7f3f85ffb640 1 -- 172.21.15.115:0/1356934965 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v81) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3f9015b630 con 0x7f3f90150490 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 -- 172.21.15.115:0/1356934965 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7406a670 msgr2=0x7f3f7406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7406a670 0x7f3f7406cb30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c033000 tx=0x7f3f8c009030 comp rx=0 tx=0).stop 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 -- 172.21.15.115:0/1356934965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90150490 msgr2=0x7f3f90151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90150490 0x7f3f90151ac0 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000d840 tx=0x7f3f8000dd00 comp rx=0 tx=0).stop 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 -- 172.21.15.115:0/1356934965 shutdown_connections 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3f7406a670 0x7f3f7406cb30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f901522f0 0x7f3f90152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f90150490 0x7f3f90151ac0 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 --2- 172.21.15.115:0/1356934965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3f7c0039a0 0x7f3f9014bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 -- 172.21.15.115:0/1356934965 >> 172.21.15.115:0/1356934965 conn(0x7f3f90065960 msgr2=0x7f3f9006b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:26.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 -- 172.21.15.115:0/1356934965 shutdown_connections 2023-12-01T03:12:26.652 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.649+0000 7f3f97939640 1 -- 172.21.15.115:0/1356934965 wait complete. 2023-12-01T03:12:26.670 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-12-01T03:12:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.877+0000 7fd9d0cf3640 1 Processor -- start 2023-12-01T03:12:26.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.879+0000 7fd9d0cf3640 1 -- start start 2023-12-01T03:12:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.880+0000 7fd9d0cf3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc145620 0x7fd9cc0ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.880+0000 7fd9d0cf3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd9cc06b980 0x7fd9cc0c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.880+0000 7fd9d0cf3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9cc0ad7e0 con 0x7fd9cc0cdd80 2023-12-01T03:12:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.880+0000 7fd9d0cf3640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9cc0cc8e0 con 0x7fd9cc06b980 2023-12-01T03:12:26.881 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.880+0000 7fd9d0cf3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc0ce260 con 0x7fd9cc145620 2023-12-01T03:12:26.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.880+0000 7fd9cb7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc145620 0x7fd9cc0ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.880+0000 7fd9cb7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc145620 0x7fd9cc0ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:59452/0 (socket says 172.21.15.115:59452) 2023-12-01T03:12:26.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.880+0000 7fd9cb7fe640 1 -- 172.21.15.115:0/1912561851 learned_addr learned my addr 172.21.15.115:0/1912561851 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:26.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.880+0000 7fd9cbfff640 1 --2- 172.21.15.115:0/1912561851 >> v2:172.21.15.142:3300/0 conn(0x7fd9cc06b980 0x7fd9cc0c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.881+0000 7fd9cb7fe640 1 -- 172.21.15.115:0/1912561851 >> v2:172.21.15.142:3300/0 conn(0x7fd9cc06b980 msgr2=0x7fd9cc0c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:26.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.881+0000 7fd9cb7fe640 1 --2- 172.21.15.115:0/1912561851 >> v2:172.21.15.142:3300/0 conn(0x7fd9cc06b980 0x7fd9cc0c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.881+0000 7fd9cb7fe640 1 -- 172.21.15.115:0/1912561851 >> v1:172.21.15.142:6789/0 conn(0x7fd9cc0cdd80 legacy=0x7fd9cc06b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.883 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.881+0000 7fd9cb7fe640 1 -- 172.21.15.115:0/1912561851 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0cc320 con 0x7fd9cc145620 2023-12-01T03:12:26.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.882+0000 7fd9cb7fe640 1 --2- 172.21.15.115:0/1912561851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc145620 0x7fd9cc0ad2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009870 tx=0x7fd9bc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b8eda2150eaf293 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:26.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.883+0000 7fd9caffd640 1 -- 172.21.15.115:0/1912561851 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc030840 con 0x7fd9cc145620 2023-12-01T03:12:26.884 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.883+0000 7fd9caffd640 1 -- 172.21.15.115:0/1912561851 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc0309a0 con 0x7fd9cc145620 2023-12-01T03:12:26.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.883+0000 7fd9caffd640 1 -- 172.21.15.115:0/1912561851 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc030d30 con 0x7fd9cc145620 2023-12-01T03:12:26.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.883+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/1912561851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc145620 msgr2=0x7fd9cc0ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.883+0000 7fd9d0cf3640 1 --2- 172.21.15.115:0/1912561851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc145620 0x7fd9cc0ad2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009870 tx=0x7fd9bc02b980 comp rx=0 tx=0).stop 2023-12-01T03:12:26.885 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.884+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/1912561851 shutdown_connections 2023-12-01T03:12:26.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.884+0000 7fd9d0cf3640 1 --2- 172.21.15.115:0/1912561851 >> v2:172.21.15.142:3300/0 conn(0x7fd9cc06b980 0x7fd9cc0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.884+0000 7fd9d0cf3640 1 --2- 172.21.15.115:0/1912561851 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc145620 0x7fd9cc0ad2a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.884+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/1912561851 >> 172.21.15.115:0/1912561851 conn(0x7fd9cc0ce640 msgr2=0x7fd9cc0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:26.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.884+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/1912561851 shutdown_connections 2023-12-01T03:12:26.886 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.884+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/1912561851 wait complete. 2023-12-01T03:12:26.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.886+0000 7fd9d0cf3640 1 Processor -- start 2023-12-01T03:12:26.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.886+0000 7fd9d0cf3640 1 -- start start 2023-12-01T03:12:26.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.887+0000 7fd9d0cf3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9cc06b980 0x7fd9cc1314e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.887+0000 7fd9d0cf3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc0cdd80 0x7fd9cc1f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.887+0000 7fd9d0cf3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc145620 0x7fd9cc1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.887+0000 7fd9d0cf3640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc12ea40 con 0x7fd9cc145620 2023-12-01T03:12:26.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.887+0000 7fd9d0cf3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc131a20 con 0x7fd9cc06b980 2023-12-01T03:12:26.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.887+0000 7fd9d0cf3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc12ec50 con 0x7fd9cc0cdd80 2023-12-01T03:12:26.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.888+0000 7fd9cbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc145620 0x7fd9cc1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.888+0000 7fd9cbfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc145620 0x7fd9cc1425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54392/0 (socket says 172.21.15.115:54392) 2023-12-01T03:12:26.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.888+0000 7fd9cbfff640 1 -- 172.21.15.115:0/2285587518 learned_addr learned my addr 172.21.15.115:0/2285587518 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:26.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.888+0000 7fd9cb7fe640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9cc06b980 0x7fd9cc1314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.888+0000 7fd9c3fff640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc0cdd80 0x7fd9cc1f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.888+0000 7fd9cbfff640 1 -- 172.21.15.115:0/2285587518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc0cdd80 msgr2=0x7fd9cc1f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.888+0000 7fd9cbfff640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc0cdd80 0x7fd9cc1f1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.888+0000 7fd9cbfff640 1 -- 172.21.15.115:0/2285587518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9cc06b980 msgr2=0x7fd9cc1314e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:26.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.888+0000 7fd9cbfff640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9cc06b980 0x7fd9cc1314e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:26.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.888+0000 7fd9cbfff640 1 -- 172.21.15.115:0/2285587518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc0094d0 con 0x7fd9cc145620 2023-12-01T03:12:26.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.889+0000 7fd9cbfff640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc145620 0x7fd9cc1425a0 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400efc0 tx=0x7fd9c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:26.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.889+0000 7fd9c8ff9640 1 -- 172.21.15.115:0/2285587518 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4014480 con 0x7fd9cc145620 2023-12-01T03:12:26.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.889+0000 7fd9c8ff9640 1 -- 172.21.15.115:0/2285587518 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c400f040 con 0x7fd9cc145620 2023-12-01T03:12:26.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.889+0000 7fd9c8ff9640 1 -- 172.21.15.115:0/2285587518 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4014820 con 0x7fd9cc145620 2023-12-01T03:12:26.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.889+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/2285587518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9cc12ef30 con 0x7fd9cc145620 2023-12-01T03:12:26.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.889+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/2285587518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9cc12f4f0 con 0x7fd9cc145620 2023-12-01T03:12:26.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.890+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/2285587518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9cc0673a0 con 0x7fd9cc145620 2023-12-01T03:12:26.893 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.891+0000 7fd9c8ff9640 1 -- 172.21.15.115:0/2285587518 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fd9c4014980 con 0x7fd9cc145620 2023-12-01T03:12:26.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.892+0000 7fd9c8ff9640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd9ac06a670 0x7fd9ac06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:26.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.892+0000 7fd9c8ff9640 1 -- 172.21.15.115:0/2285587518 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5847+0+0 (secure 0 0 0) 0x7fd9c4089b20 con 0x7fd9cc145620 2023-12-01T03:12:26.894 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.892+0000 7fd9cb7fe640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd9ac06a670 0x7fd9ac06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:26.895 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.893+0000 7fd9cb7fe640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd9ac06a670 0x7fd9ac06cb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc036000 tx=0x7fd9bc03b040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:26.898 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:26.897+0000 7fd9c8ff9640 1 -- 172.21.15.115:0/2285587518 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd9c4054b70 con 0x7fd9cc145620 2023-12-01T03:12:27.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.156+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/2285587518 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7fd9cc0ad9e0 con 0x7fd9cc145620 2023-12-01T03:12:27.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.309+0000 7fd9c8ff9640 1 -- 172.21.15.115:0/2285587518 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3170) v1 ==== 187+0+0 (secure 0 0 0) 0x7fd9c4018100 con 0x7fd9cc145620 2023-12-01T03:12:27.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/2285587518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd9ac06a670 msgr2=0x7fd9ac06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:27.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd9ac06a670 0x7fd9ac06cb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc036000 tx=0x7fd9bc03b040 comp rx=0 tx=0).stop 2023-12-01T03:12:27.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/2285587518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc145620 msgr2=0x7fd9cc1425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:27.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc145620 0x7fd9cc1425a0 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400efc0 tx=0x7fd9c400c490 comp rx=0 tx=0).stop 2023-12-01T03:12:27.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/2285587518 shutdown_connections 2023-12-01T03:12:27.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd9ac06a670 0x7fd9ac06cb30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:27.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9cc0cdd80 0x7fd9cc1f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:27.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9cc145620 0x7fd9cc1425a0 unknown :-1 s=CLOSED pgs=2916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:27.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 --2- 172.21.15.115:0/2285587518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9cc06b980 0x7fd9cc1314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:27.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/2285587518 >> 172.21.15.115:0/2285587518 conn(0x7fd9cc0ce640 msgr2=0x7fd9cc0acc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:27.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/2285587518 shutdown_connections 2023-12-01T03:12:27.315 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.313+0000 7fd9d0cf3640 1 -- 172.21.15.115:0/2285587518 wait complete. 2023-12-01T03:12:27.335 INFO:tasks.cephfs_test_runner:Test command fails without --fs argument in presence of multiple fs ... ok 2023-12-01T03:12:27.335 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-12-01T03:12:27.335 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-12-01T03:12:27.336 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-12-01T03:12:27.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.550+0000 7fe956bab640 1 Processor -- start 2023-12-01T03:12:27.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.550+0000 7fe956bab640 1 -- start start 2023-12-01T03:12:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.551+0000 7fe956bab640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe95015d220 0x7fe9500cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:27.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.551+0000 7fe956bab640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe9500ad2b0 0x7fe9500ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.551+0000 7fe956bab640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95015d600 con 0x7fe9500ade80 2023-12-01T03:12:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.551+0000 7fe956bab640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95006cc10 con 0x7fe95015d220 2023-12-01T03:12:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.551+0000 7fe956bab640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95006c720 con 0x7fe9500ad2b0 2023-12-01T03:12:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.552+0000 7fe954947640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe95015d220 0x7fe9500cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.552+0000 7fe954947640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe95015d220 0x7fe9500cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54396/0 (socket says 172.21.15.115:54396) 2023-12-01T03:12:27.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.552+0000 7fe954947640 1 -- 172.21.15.115:0/1958311731 learned_addr learned my addr 172.21.15.115:0/1958311731 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:27.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.552+0000 7fe954947640 1 -- 172.21.15.115:0/1958311731 >> v2:172.21.15.142:3300/0 conn(0x7fe9500ad2b0 msgr2=0x7fe9500ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:27.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.552+0000 7fe954947640 1 --2- 172.21.15.115:0/1958311731 >> v2:172.21.15.142:3300/0 conn(0x7fe9500ad2b0 0x7fe9500ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:27.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.552+0000 7fe954947640 1 -- 172.21.15.115:0/1958311731 >> v1:172.21.15.142:6789/0 conn(0x7fe9500ade80 legacy=0x7fe95006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:27.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.552+0000 7fe954947640 1 -- 172.21.15.115:0/1958311731 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95006c4b0 con 0x7fe95015d220 2023-12-01T03:12:27.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.553+0000 7fe954947640 1 --2- 172.21.15.115:0/1958311731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe95015d220 0x7fe9500cc570 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7fe9500cc4e0 tx=0x7fe94402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2f505ff9d233a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.553+0000 7fe94f7fe640 1 -- 172.21.15.115:0/1958311731 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9440317e0 con 0x7fe95015d220 2023-12-01T03:12:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.554+0000 7fe94f7fe640 1 -- 172.21.15.115:0/1958311731 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe944031940 con 0x7fe95015d220 2023-12-01T03:12:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.554+0000 7fe94f7fe640 1 -- 172.21.15.115:0/1958311731 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944031cd0 con 0x7fe95015d220 2023-12-01T03:12:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.554+0000 7fe956bab640 1 -- 172.21.15.115:0/1958311731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe95015d220 msgr2=0x7fe9500cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.554+0000 7fe956bab640 1 --2- 172.21.15.115:0/1958311731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe95015d220 0x7fe9500cc570 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7fe9500cc4e0 tx=0x7fe94402b880 comp rx=0 tx=0).stop 2023-12-01T03:12:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.554+0000 7fe956bab640 1 -- 172.21.15.115:0/1958311731 shutdown_connections 2023-12-01T03:12:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.554+0000 7fe956bab640 1 --2- 172.21.15.115:0/1958311731 >> v2:172.21.15.142:3300/0 conn(0x7fe9500ad2b0 0x7fe9500ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:27.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.554+0000 7fe956bab640 1 --2- 172.21.15.115:0/1958311731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe95015d220 0x7fe9500cc570 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.554+0000 7fe956bab640 1 -- 172.21.15.115:0/1958311731 >> 172.21.15.115:0/1958311731 conn(0x7fe9500ce880 msgr2=0x7fe9500cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.554+0000 7fe956bab640 1 -- 172.21.15.115:0/1958311731 shutdown_connections 2023-12-01T03:12:27.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.554+0000 7fe956bab640 1 -- 172.21.15.115:0/1958311731 wait complete. 2023-12-01T03:12:27.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.556+0000 7fe956bab640 1 Processor -- start 2023-12-01T03:12:27.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.556+0000 7fe956bab640 1 -- start start 2023-12-01T03:12:27.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.557+0000 7fe956bab640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9500ad2b0 0x7fe950157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.557+0000 7fe956bab640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9500ade80 0x7fe9500c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.557+0000 7fe956bab640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe95015d220 0x7fe9500b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.557+0000 7fe956bab640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500c9db0 con 0x7fe9500ade80 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.557+0000 7fe956bab640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500ca030 con 0x7fe9500ad2b0 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.557+0000 7fe956bab640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500c3b70 con 0x7fe95015d220 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.557+0000 7fe94ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9500ade80 0x7fe9500c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.557+0000 7fe94ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9500ade80 0x7fe9500c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:54410/0 (socket says 172.21.15.115:54410) 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.557+0000 7fe94ffff640 1 -- 172.21.15.115:0/1827892118 learned_addr learned my addr 172.21.15.115:0/1827892118 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.558+0000 7fe94ffff640 1 -- 172.21.15.115:0/1827892118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe95015d220 msgr2=0x7fe9500b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.558+0000 7fe94ffff640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe95015d220 0x7fe9500b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.558+0000 7fe94ffff640 1 -- 172.21.15.115:0/1827892118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9500ad2b0 msgr2=0x7fe950157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.558+0000 7fe954947640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9500ad2b0 0x7fe950157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.558+0000 7fe94ffff640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9500ad2b0 0x7fe950157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.558+0000 7fe94ffff640 1 -- 172.21.15.115:0/1827892118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9440094d0 con 0x7fe9500ade80 2023-12-01T03:12:27.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.558+0000 7fe954947640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9500ad2b0 0x7fe950157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:27.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.558+0000 7fe94ffff640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9500ade80 0x7fe9500c9870 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00b410 tx=0x7fe93c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:27.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.558+0000 7fe94d7fa640 1 -- 172.21.15.115:0/1827892118 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c011830 con 0x7fe9500ade80 2023-12-01T03:12:27.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.559+0000 7fe94d7fa640 1 -- 172.21.15.115:0/1827892118 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c0119b0 con 0x7fe9500ade80 2023-12-01T03:12:27.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.559+0000 7fe956bab640 1 -- 172.21.15.115:0/1827892118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9500c3e50 con 0x7fe9500ade80 2023-12-01T03:12:27.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.559+0000 7fe94d7fa640 1 -- 172.21.15.115:0/1827892118 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c011d60 con 0x7fe9500ade80 2023-12-01T03:12:27.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.559+0000 7fe956bab640 1 -- 172.21.15.115:0/1827892118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9500c3840 con 0x7fe9500ade80 2023-12-01T03:12:27.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.560+0000 7fe956bab640 1 -- 172.21.15.115:0/1827892118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9500c4370 con 0x7fe9500ade80 2023-12-01T03:12:27.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.561+0000 7fe94d7fa640 1 -- 172.21.15.115:0/1827892118 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fe93c01a400 con 0x7fe9500ade80 2023-12-01T03:12:27.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.562+0000 7fe94d7fa640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe93406a670 0x7fe93406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:27.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.562+0000 7fe94d7fa640 1 -- 172.21.15.115:0/1827892118 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5847+0+0 (secure 0 0 0) 0x7fe93c088fb0 con 0x7fe9500ade80 2023-12-01T03:12:27.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.562+0000 7fe954947640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe93406a670 0x7fe93406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:27.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.563+0000 7fe954947640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe93406a670 0x7fe93406cb30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe944036000 tx=0x7fe94403d040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:27.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.566+0000 7fe94d7fa640 1 -- 172.21.15.115:0/1827892118 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe9500c4370 con 0x7fe9500ade80 2023-12-01T03:12:27.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:27.830+0000 7fe956bab640 1 -- 172.21.15.115:0/1827892118 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fe9500c4370 con 0x7fe9500ade80 2023-12-01T03:12:28.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.313+0000 7fe94d7fa640 1 -- 172.21.15.115:0/1827892118 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3171) v1 ==== 158+0+0 (secure 0 0 0) 0x7fe9500c4370 con 0x7fe9500ade80 2023-12-01T03:12:28.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.316+0000 7fe956bab640 1 -- 172.21.15.115:0/1827892118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe93406a670 msgr2=0x7fe93406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.316+0000 7fe956bab640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe93406a670 0x7fe93406cb30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe944036000 tx=0x7fe94403d040 comp rx=0 tx=0).stop 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.316+0000 7fe956bab640 1 -- 172.21.15.115:0/1827892118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9500ade80 msgr2=0x7fe9500c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.316+0000 7fe956bab640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9500ade80 0x7fe9500c9870 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00b410 tx=0x7fe93c00b8d0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.316+0000 7fe956bab640 1 -- 172.21.15.115:0/1827892118 shutdown_connections 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.316+0000 7fe956bab640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe93406a670 0x7fe93406cb30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.317+0000 7fe956bab640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe95015d220 0x7fe9500b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.317+0000 7fe956bab640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9500ade80 0x7fe9500c9870 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.317+0000 7fe956bab640 1 --2- 172.21.15.115:0/1827892118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9500ad2b0 0x7fe950157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.317+0000 7fe956bab640 1 -- 172.21.15.115:0/1827892118 >> 172.21.15.115:0/1827892118 conn(0x7fe9500ce880 msgr2=0x7fe9500cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.317+0000 7fe956bab640 1 -- 172.21.15.115:0/1827892118 shutdown_connections 2023-12-01T03:12:28.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.317+0000 7fe956bab640 1 -- 172.21.15.115:0/1827892118 wait complete. 2023-12-01T03:12:28.336 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-01T03:12:28.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.553+0000 7fce2ee43640 1 Processor -- start 2023-12-01T03:12:28.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.553+0000 7fce2ee43640 1 -- start start 2023-12-01T03:12:28.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.554+0000 7fce2ee43640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce280cd470 0x7fce280b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:28.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.554+0000 7fce2ee43640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280b2040 0x7fce2815e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:28.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.554+0000 7fce2ee43640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce280cc2b0 con 0x7fce280cd850 2023-12-01T03:12:28.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.555+0000 7fce2ee43640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce280ce0b0 con 0x7fce280cd470 2023-12-01T03:12:28.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.555+0000 7fce2ee43640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce280cddd0 con 0x7fce280b2040 2023-12-01T03:12:28.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.555+0000 7fce27fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280b2040 0x7fce2815e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:28.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.555+0000 7fce2cbdf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce280cd470 0x7fce280b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:28.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.555+0000 7fce27fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280b2040 0x7fce2815e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:33940/0 (socket says 172.21.15.115:33940) 2023-12-01T03:12:28.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.555+0000 7fce27fff640 1 -- 172.21.15.115:0/4214650074 learned_addr learned my addr 172.21.15.115:0/4214650074 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:28.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.555+0000 7fce2d3e0640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fce280cd850 0x7fce280c7c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:33114/0 (socket says 172.21.15.115:33114) 2023-12-01T03:12:28.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.556+0000 7fce2cbdf640 1 -- 172.21.15.115:0/4214650074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280b2040 msgr2=0x7fce2815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:28.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.556+0000 7fce2cbdf640 1 --2- 172.21.15.115:0/4214650074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280b2040 0x7fce2815e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.556+0000 7fce2cbdf640 1 -- 172.21.15.115:0/4214650074 >> v1:172.21.15.142:6789/0 conn(0x7fce280cd850 legacy=0x7fce280c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:28.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.556+0000 7fce2cbdf640 1 -- 172.21.15.115:0/4214650074 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce280d1780 con 0x7fce280cd470 2023-12-01T03:12:28.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.556+0000 7fce277fe640 1 -- 172.21.15.115:0/4214650074 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570542907 0 0) 0x7fce280cc2b0 con 0x7fce280cd850 2023-12-01T03:12:28.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.558+0000 7fce2cbdf640 1 --2- 172.21.15.115:0/4214650074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce280cd470 0x7fce280b1a70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fce18009d90 tx=0x7fce1802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a1e92de9200a909 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:28.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.558+0000 7fce277fe640 1 -- 172.21.15.115:0/4214650074 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1802cc50 con 0x7fce280cd470 2023-12-01T03:12:28.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.558+0000 7fce277fe640 1 -- 172.21.15.115:0/4214650074 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1802cdb0 con 0x7fce280cd470 2023-12-01T03:12:28.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.558+0000 7fce277fe640 1 -- 172.21.15.115:0/4214650074 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18034660 con 0x7fce280cd470 2023-12-01T03:12:28.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.559+0000 7fce2ee43640 1 -- 172.21.15.115:0/4214650074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce280cd470 msgr2=0x7fce280b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:28.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.559+0000 7fce2ee43640 1 --2- 172.21.15.115:0/4214650074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce280cd470 0x7fce280b1a70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fce18009d90 tx=0x7fce1802bea0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.559+0000 7fce2ee43640 1 -- 172.21.15.115:0/4214650074 shutdown_connections 2023-12-01T03:12:28.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.559+0000 7fce2ee43640 1 --2- 172.21.15.115:0/4214650074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280b2040 0x7fce2815e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.559+0000 7fce2ee43640 1 --2- 172.21.15.115:0/4214650074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce280cd470 0x7fce280b1a70 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.559+0000 7fce2ee43640 1 -- 172.21.15.115:0/4214650074 >> 172.21.15.115:0/4214650074 conn(0x7fce280d0ba0 msgr2=0x7fce280d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:28.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.559+0000 7fce2ee43640 1 -- 172.21.15.115:0/4214650074 shutdown_connections 2023-12-01T03:12:28.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.559+0000 7fce2ee43640 1 -- 172.21.15.115:0/4214650074 wait complete. 2023-12-01T03:12:28.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.561+0000 7fce2ee43640 1 Processor -- start 2023-12-01T03:12:28.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.561+0000 7fce2ee43640 1 -- start start 2023-12-01T03:12:28.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.562+0000 7fce2ee43640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce280b2040 0x7fce280c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:28.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.562+0000 7fce2ee43640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce280cd470 0x7fce28158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.562+0000 7fce2ee43640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280cd850 0x7fce280ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.562+0000 7fce2ee43640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce280b4aa0 con 0x7fce280cd470 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.562+0000 7fce2ee43640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce280b4610 con 0x7fce280b2040 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.562+0000 7fce2ee43640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce28158980 con 0x7fce280cd850 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.562+0000 7fce27fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce280cd470 0x7fce28158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.562+0000 7fce27fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce280cd470 0x7fce28158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47940/0 (socket says 172.21.15.115:47940) 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.562+0000 7fce27fff640 1 -- 172.21.15.115:0/127234244 learned_addr learned my addr 172.21.15.115:0/127234244 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce2d3e0640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280cd850 0x7fce280ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce27fff640 1 -- 172.21.15.115:0/127234244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280cd850 msgr2=0x7fce280ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce2cbdf640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce280b2040 0x7fce280c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce27fff640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280cd850 0x7fce280ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce27fff640 1 -- 172.21.15.115:0/127234244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce280b2040 msgr2=0x7fce280c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:28.564 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce27fff640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce280b2040 0x7fce280c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce27fff640 1 -- 172.21.15.115:0/127234244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce18009a70 con 0x7fce280cd470 2023-12-01T03:12:28.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce2d3e0640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280cd850 0x7fce280ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:28.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce27fff640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce280cd470 0x7fce28158440 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7fce1000e900 tx=0x7fce1000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:28.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce257fa640 1 -- 172.21.15.115:0/127234244 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000ca00 con 0x7fce280cd470 2023-12-01T03:12:28.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce257fa640 1 -- 172.21.15.115:0/127234244 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1000cb60 con 0x7fce280cd470 2023-12-01T03:12:28.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce2ee43640 1 -- 172.21.15.115:0/127234244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce280b3720 con 0x7fce280cd470 2023-12-01T03:12:28.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce257fa640 1 -- 172.21.15.115:0/127234244 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10013470 con 0x7fce280cd470 2023-12-01T03:12:28.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.563+0000 7fce2ee43640 1 -- 172.21.15.115:0/127234244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce280b3ce0 con 0x7fce280cd470 2023-12-01T03:12:28.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.565+0000 7fce2ee43640 1 -- 172.21.15.115:0/127234244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce28067850 con 0x7fce280cd470 2023-12-01T03:12:28.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.565+0000 7fce257fa640 1 -- 172.21.15.115:0/127234244 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fce100135d0 con 0x7fce280cd470 2023-12-01T03:12:28.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.566+0000 7fce257fa640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce0806a670 0x7fce0806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:28.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.566+0000 7fce257fa640 1 -- 172.21.15.115:0/127234244 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5847+0+0 (secure 0 0 0) 0x7fce100890e0 con 0x7fce280cd470 2023-12-01T03:12:28.567 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.567+0000 7fce2cbdf640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce0806a670 0x7fce0806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:28.568 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.567+0000 7fce2cbdf640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce0806a670 0x7fce0806cb30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fce18036000 tx=0x7fce18030000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:28.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.576+0000 7fce257fa640 1 -- 172.21.15.115:0/127234244 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fce10054240 con 0x7fce280cd470 2023-12-01T03:12:28.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.769+0000 7fce2ee43640 1 -- 172.21.15.115:0/127234244 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fce2804e890 con 0x7fce280cd470 2023-12-01T03:12:28.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.778+0000 7fce257fa640 1 -- 172.21.15.115:0/127234244 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v82)=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7fce10057860 con 0x7fce280cd470 2023-12-01T03:12:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 -- 172.21.15.115:0/127234244 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce0806a670 msgr2=0x7fce0806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce0806a670 0x7fce0806cb30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fce18036000 tx=0x7fce18030000 comp rx=0 tx=0).stop 2023-12-01T03:12:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 -- 172.21.15.115:0/127234244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce280cd470 msgr2=0x7fce28158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:28.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce280cd470 0x7fce28158440 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7fce1000e900 tx=0x7fce1000edc0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 -- 172.21.15.115:0/127234244 shutdown_connections 2023-12-01T03:12:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce0806a670 0x7fce0806cb30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce280cd850 0x7fce280ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce280cd470 0x7fce28158440 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 --2- 172.21.15.115:0/127234244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce280b2040 0x7fce280c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 -- 172.21.15.115:0/127234244 >> 172.21.15.115:0/127234244 conn(0x7fce280d0ba0 msgr2=0x7fce280c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 -- 172.21.15.115:0/127234244 shutdown_connections 2023-12-01T03:12:28.783 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:28.781+0000 7fce2ee43640 1 -- 172.21.15.115:0/127234244 wait complete. 2023-12-01T03:12:28.802 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T03:12:28.802 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:12:28.916 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:12:29.048 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T03:12:29.048 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:12:29.048 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:29.108 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:12:29.108 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:12:29.108 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:29.170 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:12:29.171 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:12:29.171 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:29.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:12:29.231 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:12:29.231 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:29.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:12:29.291 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-01T03:12:29.291 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:12:29.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.511+0000 7f7cbd2c4640 1 Processor -- start 2023-12-01T03:12:29.512 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.512+0000 7f7cbd2c4640 1 -- start start 2023-12-01T03:12:29.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.513+0000 7f7cbd2c4640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7cb815cd20 0x7f7cb80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:29.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.513+0000 7f7cbd2c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cb80d1230 0x7f7cb80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:29.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.513+0000 7f7cbd2c4640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb80d1b90 con 0x7f7cb80b5d80 2023-12-01T03:12:29.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.513+0000 7f7cbd2c4640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb80b2160 con 0x7f7cb815cd20 2023-12-01T03:12:29.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.513+0000 7f7cbd2c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb80b8c80 con 0x7f7cb80d1230 2023-12-01T03:12:29.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.513+0000 7f7cb6ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7cb815cd20 0x7f7cb80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:29.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.513+0000 7f7cb67fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7cb80b5d80 0x7f7cb80b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35100/0 (socket says 172.21.15.115:35100) 2023-12-01T03:12:29.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.513+0000 7f7cb67fc640 1 -- 172.21.15.115:0/1649196978 learned_addr learned my addr 172.21.15.115:0/1649196978 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.514+0000 7f7cb6ffd640 1 -- 172.21.15.115:0/1649196978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cb80d1230 msgr2=0x7f7cb80d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.514+0000 7f7cb77fe640 1 --2- 172.21.15.115:0/1649196978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cb80d1230 0x7f7cb80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.514+0000 7f7cb6ffd640 1 --2- 172.21.15.115:0/1649196978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cb80d1230 0x7f7cb80d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.514+0000 7f7cb6ffd640 1 -- 172.21.15.115:0/1649196978 >> v1:172.21.15.115:6789/0 conn(0x7f7cb80b5d80 legacy=0x7f7cb80b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:29.515 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.514+0000 7f7cb5ffb640 1 -- 172.21.15.115:0/1649196978 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655552783 0 0) 0x7f7ca0001110 con 0x7f7cb80b5d80 2023-12-01T03:12:29.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.514+0000 7f7cb6ffd640 1 -- 172.21.15.115:0/1649196978 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb80ce2e0 con 0x7f7cb815cd20 2023-12-01T03:12:29.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.515+0000 7f7cb6ffd640 1 --2- 172.21.15.115:0/1649196978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb815cd20 0x7f7cb80d24d0 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80097a0 tx=0x7f7ca802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70915244e71cd2a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.515+0000 7f7cb5ffb640 1 -- 172.21.15.115:0/1649196978 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8031780 con 0x7f7cb815cd20 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.515+0000 7f7cb5ffb640 1 -- 172.21.15.115:0/1649196978 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca80318e0 con 0x7f7cb815cd20 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.515+0000 7f7cb5ffb640 1 -- 172.21.15.115:0/1649196978 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8031c70 con 0x7f7cb815cd20 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.516+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/1649196978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb815cd20 msgr2=0x7f7cb80d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.516+0000 7f7cbd2c4640 1 --2- 172.21.15.115:0/1649196978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb815cd20 0x7f7cb80d24d0 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80097a0 tx=0x7f7ca802b820 comp rx=0 tx=0).stop 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.516+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/1649196978 shutdown_connections 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.516+0000 7f7cbd2c4640 1 --2- 172.21.15.115:0/1649196978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cb80d1230 0x7f7cb80d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.516+0000 7f7cbd2c4640 1 --2- 172.21.15.115:0/1649196978 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb815cd20 0x7f7cb80d24d0 unknown :-1 s=CLOSED pgs=2921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.516+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/1649196978 >> 172.21.15.115:0/1649196978 conn(0x7f7cb806a0e0 msgr2=0x7f7cb806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.516+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/1649196978 shutdown_connections 2023-12-01T03:12:29.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.516+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/1649196978 wait complete. 2023-12-01T03:12:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.518+0000 7f7cbd2c4640 1 Processor -- start 2023-12-01T03:12:29.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.518+0000 7f7cbd2c4640 1 -- start start 2023-12-01T03:12:29.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cbd2c4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cb80b5d80 0x7f7cb815e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:29.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cbd2c4640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb80d1230 0x7f7cb80b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:29.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cbd2c4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cb815cd20 0x7f7cb80b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:29.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cbd2c4640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb815fab0 con 0x7f7cb80d1230 2023-12-01T03:12:29.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cbd2c4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb815ff40 con 0x7f7cb815cd20 2023-12-01T03:12:29.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cbd2c4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81f2470 con 0x7f7cb80b5d80 2023-12-01T03:12:29.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cb67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb80d1230 0x7f7cb80b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:29.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cb67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb80d1230 0x7f7cb80b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:47966/0 (socket says 172.21.15.115:47966) 2023-12-01T03:12:29.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cb67fc640 1 -- 172.21.15.115:0/888101499 learned_addr learned my addr 172.21.15.115:0/888101499 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:29.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cb67fc640 1 -- 172.21.15.115:0/888101499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cb80b5d80 msgr2=0x7f7cb815e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:29.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cb67fc640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cb80b5d80 0x7f7cb815e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:29.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cb67fc640 1 -- 172.21.15.115:0/888101499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cb815cd20 msgr2=0x7f7cb80b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:29.521 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cb67fc640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cb815cd20 0x7f7cb80b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:29.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.519+0000 7f7cb67fc640 1 -- 172.21.15.115:0/888101499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca8009400 con 0x7f7cb80d1230 2023-12-01T03:12:29.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.520+0000 7f7cb67fc640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb80d1230 0x7f7cb80b6ac0 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c430 tx=0x7f7ca000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:29.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.520+0000 7f7caf7fe640 1 -- 172.21.15.115:0/888101499 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00074e0 con 0x7f7cb80d1230 2023-12-01T03:12:29.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.520+0000 7f7caf7fe640 1 -- 172.21.15.115:0/888101499 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0007660 con 0x7f7cb80d1230 2023-12-01T03:12:29.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.520+0000 7f7caf7fe640 1 -- 172.21.15.115:0/888101499 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0004a20 con 0x7f7cb80d1230 2023-12-01T03:12:29.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.520+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/888101499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb815eea0 con 0x7f7cb80d1230 2023-12-01T03:12:29.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.520+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/888101499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb815f460 con 0x7f7cb80d1230 2023-12-01T03:12:29.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.521+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/888101499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb804e890 con 0x7f7cb80d1230 2023-12-01T03:12:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.522+0000 7f7caf7fe640 1 -- 172.21.15.115:0/888101499 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f7ca0004b80 con 0x7f7cb80d1230 2023-12-01T03:12:29.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.523+0000 7f7caf7fe640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c9806a670 0x7f7c9806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.523+0000 7f7caf7fe640 1 -- 172.21.15.115:0/888101499 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f7ca0088a50 con 0x7f7cb80d1230 2023-12-01T03:12:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.523+0000 7f7cb6ffd640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c9806a670 0x7f7c9806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:29.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.524+0000 7f7cb6ffd640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c9806a670 0x7f7c9806cb30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8036000 tx=0x7f7ca8030000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:29.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.527+0000 7f7caf7fe640 1 -- 172.21.15.115:0/888101499 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f7ca0053c40 con 0x7f7cb80d1230 2023-12-01T03:12:29.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.753+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/888101499 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cb80d1b90 con 0x7f7cb80d1230 2023-12-01T03:12:29.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.754+0000 7f7caf7fe640 1 -- 172.21.15.115:0/888101499 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+3531 (secure 0 0 0) 0x7f7cb80d1b90 con 0x7f7cb80d1230 2023-12-01T03:12:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:29.755 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/993295369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":993295369},{"type":"v1","addr":"172.21.15.142:6835","nonce":993295369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":7085,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/2628767413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":2628767413},{"type":"v1","addr":"172.21.15.142:6833","nonce":2628767413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":7726,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/89104161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":89104161},{"type":"v1","addr":"172.21.15.115:6835","nonce":89104161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":7729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/56938976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":56938976},{"type":"v1","addr":"172.21.15.115:6837","nonce":56938976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[]} 2023-12-01T03:12:29.755 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 120 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.756+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/888101499 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c9806a670 msgr2=0x7f7c9806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.756+0000 7f7cbd2c4640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c9806a670 0x7f7c9806cb30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8036000 tx=0x7f7ca8030000 comp rx=0 tx=0).stop 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.757+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/888101499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb80d1230 msgr2=0x7f7cb80b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.757+0000 7f7cbd2c4640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb80d1230 0x7f7cb80b6ac0 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c430 tx=0x7f7ca000c8f0 comp rx=0 tx=0).stop 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.757+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/888101499 shutdown_connections 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.757+0000 7f7cbd2c4640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7c9806a670 0x7f7c9806cb30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.757+0000 7f7cbd2c4640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cb815cd20 0x7f7cb80b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.757+0000 7f7cbd2c4640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cb80d1230 0x7f7cb80b6ac0 unknown :-1 s=CLOSED pgs=2922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.757+0000 7f7cbd2c4640 1 --2- 172.21.15.115:0/888101499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cb80b5d80 0x7f7cb815e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.757+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/888101499 >> 172.21.15.115:0/888101499 conn(0x7f7cb806a0e0 msgr2=0x7f7cb815bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.757+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/888101499 shutdown_connections 2023-12-01T03:12:29.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:29.757+0000 7f7cbd2c4640 1 -- 172.21.15.115:0/888101499 wait complete. 2023-12-01T03:12:29.778 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-01T03:12:29.778 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-01T03:12:29.779 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T03:12:29.779 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:12:29.779 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-01T03:12:29.779 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-01T03:12:29.779 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T03:12:29.779 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:12:29.779 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-01T03:12:29.779 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-01T03:12:29.780 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T03:12:29.780 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:12:29.780 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-01T03:12:29.780 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-01T03:12:29.780 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T03:12:29.780 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:12:29.799 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T03:12:29.799 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-01T03:12:29.802 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T03:12:29.802 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-01T03:12:29.804 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T03:12:29.804 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-01T03:12:29.809 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T03:12:29.810 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-01T03:12:29.853 INFO:tasks.ceph.mds.c:Started 2023-12-01T03:12:29.853 DEBUG:teuthology.parallel:result is None 2023-12-01T03:12:29.858 INFO:tasks.ceph.mds.d:Started 2023-12-01T03:12:29.859 DEBUG:teuthology.parallel:result is None 2023-12-01T03:12:29.860 INFO:tasks.ceph.mds.a:Started 2023-12-01T03:12:29.861 DEBUG:teuthology.parallel:result is None 2023-12-01T03:12:29.866 INFO:tasks.ceph.mds.b:Started 2023-12-01T03:12:29.866 DEBUG:teuthology.parallel:result is None 2023-12-01T03:12:29.866 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-01T03:12:30.011 INFO:tasks.ceph.mds.c.smithi115.stdout:starting mds.c at 2023-12-01T03:12:30.020 INFO:tasks.ceph.mds.a.smithi115.stdout:starting mds.a at 2023-12-01T03:12:30.028 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T03:12:30.029 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2023-12-01T03:12:30.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.060+0000 7fce73fd9640 1 Processor -- start 2023-12-01T03:12:30.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.060+0000 7fce73fd9640 1 -- start start 2023-12-01T03:12:30.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.061+0000 7fce73fd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c15cf90 0x7fce6c0d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:30.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.061+0000 7fce73fd9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fce6c0b8610 0x7fce6c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:30.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.062+0000 7fce73fd9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6c15d370 con 0x7fce6c0cdf00 2023-12-01T03:12:30.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.062+0000 7fce73fd9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6c0ce390 con 0x7fce6c0b8610 2023-12-01T03:12:30.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.062+0000 7fce73fd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0d19a0 con 0x7fce6c15cf90 2023-12-01T03:12:30.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.062+0000 7fce72576640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fce6c0cdf00 0x7fce6c0b4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35106/0 (socket says 172.21.15.115:35106) 2023-12-01T03:12:30.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.062+0000 7fce72576640 1 -- 172.21.15.115:0/1083152975 learned_addr learned my addr 172.21.15.115:0/1083152975 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:30.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.062+0000 7fce71d75640 1 --2- 172.21.15.115:0/1083152975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c15cf90 0x7fce6c0d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:30.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.062+0000 7fce71574640 1 --2- 172.21.15.115:0/1083152975 >> v2:172.21.15.115:3300/0 conn(0x7fce6c0b8610 0x7fce6c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:30.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.062+0000 7fce70d73640 1 -- 172.21.15.115:0/1083152975 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511107264 0 0) 0x7fce6c15d370 con 0x7fce6c0cdf00 2023-12-01T03:12:30.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.062+0000 7fce70d73640 1 -- 172.21.15.115:0/1083152975 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce58003520 con 0x7fce6c0cdf00 2023-12-01T03:12:30.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.063+0000 7fce71d75640 1 -- 172.21.15.115:0/1083152975 >> v2:172.21.15.115:3300/0 conn(0x7fce6c0b8610 msgr2=0x7fce6c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:30.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.063+0000 7fce71d75640 1 --2- 172.21.15.115:0/1083152975 >> v2:172.21.15.115:3300/0 conn(0x7fce6c0b8610 0x7fce6c0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:30.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.063+0000 7fce71d75640 1 -- 172.21.15.115:0/1083152975 >> v1:172.21.15.115:6789/0 conn(0x7fce6c0cdf00 legacy=0x7fce6c0b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:30.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.063+0000 7fce71d75640 1 -- 172.21.15.115:0/1083152975 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0d1c80 con 0x7fce6c15cf90 2023-12-01T03:12:30.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.064+0000 7fce71d75640 1 --2- 172.21.15.115:0/1083152975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c15cf90 0x7fce6c0d1030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce680097a0 tx=0x7fce6802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7faed04b5d20a38a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:30.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.065+0000 7fce70d73640 1 -- 172.21.15.115:0/1083152975 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6802d040 con 0x7fce6c15cf90 2023-12-01T03:12:30.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.065+0000 7fce70d73640 1 -- 172.21.15.115:0/1083152975 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce68031810 con 0x7fce6c15cf90 2023-12-01T03:12:30.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.065+0000 7fce70d73640 1 -- 172.21.15.115:0/1083152975 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68031ba0 con 0x7fce6c15cf90 2023-12-01T03:12:30.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.066+0000 7fce73fd9640 1 -- 172.21.15.115:0/1083152975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c15cf90 msgr2=0x7fce6c0d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:30.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.066+0000 7fce73fd9640 1 --2- 172.21.15.115:0/1083152975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c15cf90 0x7fce6c0d1030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce680097a0 tx=0x7fce6802b8b0 comp rx=0 tx=0).stop 2023-12-01T03:12:30.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.066+0000 7fce73fd9640 1 -- 172.21.15.115:0/1083152975 shutdown_connections 2023-12-01T03:12:30.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.066+0000 7fce73fd9640 1 --2- 172.21.15.115:0/1083152975 >> v2:172.21.15.115:3300/0 conn(0x7fce6c0b8610 0x7fce6c0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:30.067 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.066+0000 7fce73fd9640 1 --2- 172.21.15.115:0/1083152975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c15cf90 0x7fce6c0d1030 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:30.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.066+0000 7fce73fd9640 1 -- 172.21.15.115:0/1083152975 >> 172.21.15.115:0/1083152975 conn(0x7fce6c065960 msgr2=0x7fce6c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:30.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.066+0000 7fce73fd9640 1 -- 172.21.15.115:0/1083152975 shutdown_connections 2023-12-01T03:12:30.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.067+0000 7fce73fd9640 1 -- 172.21.15.115:0/1083152975 wait complete. 2023-12-01T03:12:30.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.068+0000 7fce73fd9640 1 Processor -- start 2023-12-01T03:12:30.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.069+0000 7fce73fd9640 1 -- start start 2023-12-01T03:12:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.070+0000 7fce73fd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6c0b8610 0x7fce6c1f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.070+0000 7fce73fd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c0cdf00 0x7fce6c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.070+0000 7fce73fd9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6c15cf90 0x7fce6c1f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.070+0000 7fce73fd9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0ccaf0 con 0x7fce6c0b8610 2023-12-01T03:12:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.070+0000 7fce73fd9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0cc530 con 0x7fce6c15cf90 2023-12-01T03:12:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.070+0000 7fce73fd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0cc810 con 0x7fce6c0cdf00 2023-12-01T03:12:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.070+0000 7fce71d75640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6c0b8610 0x7fce6c1f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.070+0000 7fce71d75640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6c0b8610 0x7fce6c1f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48008/0 (socket says 172.21.15.115:48008) 2023-12-01T03:12:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.071+0000 7fce71d75640 1 -- 172.21.15.115:0/178046289 learned_addr learned my addr 172.21.15.115:0/178046289 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:30.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.071+0000 7fce71574640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c0cdf00 0x7fce6c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:30.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.071+0000 7fce72576640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6c15cf90 0x7fce6c1f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:30.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.071+0000 7fce71d75640 1 -- 172.21.15.115:0/178046289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c0cdf00 msgr2=0x7fce6c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:30.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.071+0000 7fce71d75640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c0cdf00 0x7fce6c0cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:30.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.071+0000 7fce71d75640 1 -- 172.21.15.115:0/178046289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6c15cf90 msgr2=0x7fce6c1f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:30.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.071+0000 7fce71d75640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6c15cf90 0x7fce6c1f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:30.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.071+0000 7fce71d75640 1 -- 172.21.15.115:0/178046289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68009400 con 0x7fce6c0b8610 2023-12-01T03:12:30.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.071+0000 7fce71d75640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6c0b8610 0x7fce6c1f64d0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fce680098c0 tx=0x7fce680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:30.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.071+0000 7fce627fc640 1 -- 172.21.15.115:0/178046289 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6803ad60 con 0x7fce6c0b8610 2023-12-01T03:12:30.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.072+0000 7fce627fc640 1 -- 172.21.15.115:0/178046289 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce68044400 con 0x7fce6c0b8610 2023-12-01T03:12:30.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.072+0000 7fce627fc640 1 -- 172.21.15.115:0/178046289 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce680445b0 con 0x7fce6c0b8610 2023-12-01T03:12:30.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.073+0000 7fce73fd9640 1 -- 172.21.15.115:0/178046289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c1f5cd0 con 0x7fce6c0b8610 2023-12-01T03:12:30.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.073+0000 7fce73fd9640 1 -- 172.21.15.115:0/178046289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c1f6a10 con 0x7fce6c0b8610 2023-12-01T03:12:30.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.074+0000 7fce73fd9640 1 -- 172.21.15.115:0/178046289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c1f61f0 con 0x7fce6c0b8610 2023-12-01T03:12:30.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.075+0000 7fce627fc640 1 -- 172.21.15.115:0/178046289 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fce6804d440 con 0x7fce6c0b8610 2023-12-01T03:12:30.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.076+0000 7fce627fc640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce5006a670 0x7fce5006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:30.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.076+0000 7fce627fc640 1 -- 172.21.15.115:0/178046289 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5847+0+0 (secure 0 0 0) 0x7fce680ab140 con 0x7fce6c0b8610 2023-12-01T03:12:30.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.076+0000 7fce71574640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce5006a670 0x7fce5006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:30.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.077+0000 7fce71574640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce5006a670 0x7fce5006cb30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0cbba0 tx=0x7fce5c008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:30.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.081+0000 7fce627fc640 1 -- 172.21.15.115:0/178046289 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fce68076340 con 0x7fce6c0b8610 2023-12-01T03:12:30.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:30.289+0000 7fce73fd9640 1 -- 172.21.15.115:0/178046289 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fce6c04e890 con 0x7fce6c0b8610 2023-12-01T03:12:31.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.075+0000 7fce627fc640 1 -- 172.21.15.115:0/178046289 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v252) v1 ==== 97+0+0 (secure 0 0 0) 0x7fce68079960 con 0x7fce6c0b8610 2023-12-01T03:12:31.076 INFO:teuthology.orchestra.run.smithi115.stderr: removed all blocklist entries 2023-12-01T03:12:31.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 -- 172.21.15.115:0/178046289 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce5006a670 msgr2=0x7fce5006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:31.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce5006a670 0x7fce5006cb30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0cbba0 tx=0x7fce5c008040 comp rx=0 tx=0).stop 2023-12-01T03:12:31.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 -- 172.21.15.115:0/178046289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6c0b8610 msgr2=0x7fce6c1f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:31.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6c0b8610 0x7fce6c1f64d0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fce680098c0 tx=0x7fce680046c0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 -- 172.21.15.115:0/178046289 shutdown_connections 2023-12-01T03:12:31.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fce5006a670 0x7fce5006cb30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce6c0cdf00 0x7fce6c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce6c15cf90 0x7fce6c1f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 --2- 172.21.15.115:0/178046289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce6c0b8610 0x7fce6c1f64d0 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 -- 172.21.15.115:0/178046289 >> 172.21.15.115:0/178046289 conn(0x7fce6c065960 msgr2=0x7fce6c0b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:31.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 -- 172.21.15.115:0/178046289 shutdown_connections 2023-12-01T03:12:31.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.078+0000 7fce73fd9640 1 -- 172.21.15.115:0/178046289 wait complete. 2023-12-01T03:12:31.102 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-01T03:12:31.124 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:12:31.157 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:12:31.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.301+0000 7f279a451640 1 Processor -- start 2023-12-01T03:12:31.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.301+0000 7f279a451640 1 -- start start 2023-12-01T03:12:31.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.302+0000 7f279a451640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2794159dd0 0x7f27940c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.302+0000 7f279a451640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2794070d30 0x7f27940c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.302+0000 7f279a451640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279406f8b0 con 0x7f279406b6f0 2023-12-01T03:12:31.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.302+0000 7f279a451640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27940adcb0 con 0x7f2794159dd0 2023-12-01T03:12:31.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.303+0000 7f279a451640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27940ad9d0 con 0x7f2794070d30 2023-12-01T03:12:31.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.303+0000 7f2793fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2794159dd0 0x7f27940c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:31.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.303+0000 7f2793fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2794159dd0 0x7f27940c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48014/0 (socket says 172.21.15.115:48014) 2023-12-01T03:12:31.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.303+0000 7f2793fff640 1 -- 172.21.15.115:0/1145929385 learned_addr learned my addr 172.21.15.115:0/1145929385 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:31.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.303+0000 7f2793fff640 1 -- 172.21.15.115:0/1145929385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2794070d30 msgr2=0x7f27940c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:31.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.303+0000 7f2793fff640 1 --2- 172.21.15.115:0/1145929385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2794070d30 0x7f27940c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.303+0000 7f2793fff640 1 -- 172.21.15.115:0/1145929385 >> v1:172.21.15.142:6789/0 conn(0x7f279406b6f0 legacy=0x7f27940cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:31.305 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.303+0000 7f2793fff640 1 -- 172.21.15.115:0/1145929385 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940c9000 con 0x7f2794159dd0 2023-12-01T03:12:31.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.304+0000 7f2793fff640 1 --2- 172.21.15.115:0/1145929385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2794159dd0 0x7f27940c97e0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f2784009870 tx=0x7f278402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30566818307b159 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:31.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.304+0000 7f27937fe640 1 -- 172.21.15.115:0/1145929385 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784031740 con 0x7f2794159dd0 2023-12-01T03:12:31.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.305+0000 7f27937fe640 1 -- 172.21.15.115:0/1145929385 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27840318a0 con 0x7f2794159dd0 2023-12-01T03:12:31.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.305+0000 7f27937fe640 1 -- 172.21.15.115:0/1145929385 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784031c30 con 0x7f2794159dd0 2023-12-01T03:12:31.306 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.305+0000 7f279a451640 1 -- 172.21.15.115:0/1145929385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2794159dd0 msgr2=0x7f27940c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:31.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.305+0000 7f279a451640 1 --2- 172.21.15.115:0/1145929385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2794159dd0 0x7f27940c97e0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f2784009870 tx=0x7f278402b7e0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.305+0000 7f279a451640 1 -- 172.21.15.115:0/1145929385 shutdown_connections 2023-12-01T03:12:31.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.305+0000 7f279a451640 1 --2- 172.21.15.115:0/1145929385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2794070d30 0x7f27940c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.305+0000 7f279a451640 1 --2- 172.21.15.115:0/1145929385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2794159dd0 0x7f27940c97e0 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.305+0000 7f279a451640 1 -- 172.21.15.115:0/1145929385 >> 172.21.15.115:0/1145929385 conn(0x7f27940ce880 msgr2=0x7f27940cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:31.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.305+0000 7f279a451640 1 -- 172.21.15.115:0/1145929385 shutdown_connections 2023-12-01T03:12:31.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.305+0000 7f279a451640 1 -- 172.21.15.115:0/1145929385 wait complete. 2023-12-01T03:12:31.307 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.307+0000 7f279a451640 1 Processor -- start 2023-12-01T03:12:31.308 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.307+0000 7f279a451640 1 -- start start 2023-12-01T03:12:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.308+0000 7f279a451640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f279406b6f0 0x7f2794145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.308+0000 7f279a451640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2794070d30 0x7f27940c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.308+0000 7f279a451640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2794159dd0 0x7f27940c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.308+0000 7f279a451640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279414f250 con 0x7f279406b6f0 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.308+0000 7f279a451640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279414e070 con 0x7f2794070d30 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.308+0000 7f279a451640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2794146220 con 0x7f2794159dd0 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.308+0000 7f2793fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f279406b6f0 0x7f2794145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.308+0000 7f2793fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f279406b6f0 0x7f2794145ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48024/0 (socket says 172.21.15.115:48024) 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.308+0000 7f2793fff640 1 -- 172.21.15.115:0/1977770017 learned_addr learned my addr 172.21.15.115:0/1977770017 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f27989ee640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2794159dd0 0x7f27940c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f2793fff640 1 -- 172.21.15.115:0/1977770017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2794159dd0 msgr2=0x7f27940c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f2793fff640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2794159dd0 0x7f27940c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f2793fff640 1 -- 172.21.15.115:0/1977770017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2794070d30 msgr2=0x7f27940c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f278bfff640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2794070d30 0x7f27940c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:31.311 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f2793fff640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2794070d30 0x7f27940c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f2793fff640 1 -- 172.21.15.115:0/1977770017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840094d0 con 0x7f279406b6f0 2023-12-01T03:12:31.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f2793fff640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f279406b6f0 0x7f2794145ce0 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f2784009870 tx=0x7f27840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:31.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f27917fa640 1 -- 172.21.15.115:0/1977770017 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784033c70 con 0x7f279406b6f0 2023-12-01T03:12:31.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f27917fa640 1 -- 172.21.15.115:0/1977770017 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2784033df0 con 0x7f279406b6f0 2023-12-01T03:12:31.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f27917fa640 1 -- 172.21.15.115:0/1977770017 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278403d6b0 con 0x7f279406b6f0 2023-12-01T03:12:31.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.309+0000 7f279a451640 1 -- 172.21.15.115:0/1977770017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27940c71b0 con 0x7f279406b6f0 2023-12-01T03:12:31.312 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.310+0000 7f279a451640 1 -- 172.21.15.115:0/1977770017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27940c7770 con 0x7f279406b6f0 2023-12-01T03:12:31.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.311+0000 7f279a451640 1 -- 172.21.15.115:0/1977770017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279406cba0 con 0x7f279406b6f0 2023-12-01T03:12:31.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.311+0000 7f27917fa640 1 -- 172.21.15.115:0/1977770017 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f278403a040 con 0x7f279406b6f0 2023-12-01T03:12:31.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.312+0000 7f27917fa640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f277406a670 0x7f277406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.312+0000 7f27917fa640 1 -- 172.21.15.115:0/1977770017 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f27840aab50 con 0x7f279406b6f0 2023-12-01T03:12:31.313 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.312+0000 7f278bfff640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f277406a670 0x7f277406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:31.314 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.313+0000 7f278bfff640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f277406a670 0x7f277406cb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f27940ccfd0 tx=0x7f277c008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:31.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.317+0000 7f27917fa640 1 -- 172.21.15.115:0/1977770017 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2784075fe0 con 0x7f279406b6f0 2023-12-01T03:12:31.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.556+0000 7f279a451640 1 -- 172.21.15.115:0/1977770017 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f279404e890 con 0x7f279406b6f0 2023-12-01T03:12:31.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.557+0000 7f27917fa640 1 -- 172.21.15.115:0/1977770017 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v50) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f278402f090 con 0x7f279406b6f0 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_dump": [ 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.a", 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllTccVBxAAZSi7gdbFsi6eH8XT2UtxXQ==", 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:12:31.558 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.b", 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllHLEgKhAAy96AHFCqHwjALohzMdI0wg==", 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.559 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.c", 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBCOGllXMkxGBAABmLp51J7Y1uuaYWSSGYAxw==", 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mds.d", 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllhCICABAAfOrv3k3pY9nHq2kI/brr5Q==", 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:12:31.560 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.0", 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBHOGll9q+1OBAAq2+BSnXkH858kOCJQBcsPw==", 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.1", 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBLOGllr1AcCRAAh7qVP6S8aaM4noZkFabtig==", 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.2", 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBOOGllK9T5IRAANkQAW9r34vX7GG0aNvz3/A==", 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:12:31.562 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.3", 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBROGllBH+gKxAA3ONEsE0+Z/UmhdAQmzIjdg==", 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.4", 2023-12-01T03:12:31.563 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBYOGllfxAtMhAAMkb+6Fd96yFiYJaN347gMg==", 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.5", 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBcOGllNw7uAxAAiZzDF+F09C7K2Ez+wIFuUA==", 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.6", 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBfOGllgQcvHhAAQhNgxGpe/pGvxW5p542pkw==", 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "osd.7", 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBiOGllqJkENxAAvnLZFbHOF0Cym2/A+/9X4w==", 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow profile osd", 2023-12-01T03:12:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile osd", 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.0", 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllVOSbDRAABQeSJ5tT0ZHm0/ufr/fyvg==", 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow r", 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.1", 2023-12-01T03:12:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllU2HZJBAA7Qp7DAXL0T3El5BhH7lX9A==", 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.2", 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGlltcDyGBAA1pUuZRZZ/cGdyAE402u09g==", 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:12:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.3", 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBDOGllTSlcLxAAXH45GcPTwwYdQ3drr28T3g==", 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow", 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow r", 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow rw", 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow rwx" 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "client.admin", 2023-12-01T03:12:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBAOGllAa7iMxAAwZjt4zUC5aQYJQOgdHRuQw==", 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": "allow *", 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow *", 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.x", 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllAZPONhAAyjBzpYLp9sOvV2QPiYl+Yg==", 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "entity": "mgr.y", 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "AQBBOGllGoBLKhAAO3mytwZgIGfdj0lowj45qA==", 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "caps": { 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "mds": "allow *", 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "mon": "allow profile mgr", 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": "allow *" 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2023-12-01T03:12:31.570 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 -- 172.21.15.115:0/1977770017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f277406a670 msgr2=0x7f277406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f277406a670 0x7f277406cb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f27940ccfd0 tx=0x7f277c008040 comp rx=0 tx=0).stop 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 -- 172.21.15.115:0/1977770017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f279406b6f0 msgr2=0x7f2794145ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f279406b6f0 0x7f2794145ce0 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f2784009870 tx=0x7f27840046c0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 -- 172.21.15.115:0/1977770017 shutdown_connections 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f277406a670 0x7f277406cb30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2794159dd0 0x7f27940c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2794070d30 0x7f27940c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 --2- 172.21.15.115:0/1977770017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f279406b6f0 0x7f2794145ce0 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 -- 172.21.15.115:0/1977770017 >> 172.21.15.115:0/1977770017 conn(0x7f27940ce880 msgr2=0x7f279406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:31.572 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 -- 172.21.15.115:0/1977770017 shutdown_connections 2023-12-01T03:12:31.573 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.559+0000 7f279a451640 1 -- 172.21.15.115:0/1977770017 wait complete. 2023-12-01T03:12:31.577 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.819+0000 7fbf49abd640 1 Processor -- start 2023-12-01T03:12:31.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.819+0000 7fbf49abd640 1 -- start start 2023-12-01T03:12:31.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.821+0000 7fbf49abd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbf44159dd0 0x7fbf440c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.821+0000 7fbf49abd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbf4406b6f0 0x7fbf440c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.821+0000 7fbf49abd640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4406f8b0 con 0x7fbf440ade80 2023-12-01T03:12:31.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.821+0000 7fbf49abd640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf440adcb0 con 0x7fbf4406b6f0 2023-12-01T03:12:31.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.821+0000 7fbf49abd640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf440ad9d0 con 0x7fbf44159dd0 2023-12-01T03:12:31.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.821+0000 7fbf3bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbf4406b6f0 0x7fbf440c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:31.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.821+0000 7fbf3bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbf4406b6f0 0x7fbf440c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48028/0 (socket says 172.21.15.115:48028) 2023-12-01T03:12:31.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.821+0000 7fbf3bfff640 1 -- 172.21.15.115:0/2719045427 learned_addr learned my addr 172.21.15.115:0/2719045427 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:31.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.822+0000 7fbf437fe640 1 --2- 172.21.15.115:0/2719045427 >> v2:172.21.15.142:3300/0 conn(0x7fbf44159dd0 0x7fbf440c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:31.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.822+0000 7fbf3bfff640 1 -- 172.21.15.115:0/2719045427 >> v2:172.21.15.142:3300/0 conn(0x7fbf44159dd0 msgr2=0x7fbf440c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:31.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.822+0000 7fbf3bfff640 1 --2- 172.21.15.115:0/2719045427 >> v2:172.21.15.142:3300/0 conn(0x7fbf44159dd0 0x7fbf440c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.822+0000 7fbf3bfff640 1 -- 172.21.15.115:0/2719045427 >> v1:172.21.15.142:6789/0 conn(0x7fbf440ade80 legacy=0x7fbf440cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:31.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.822+0000 7fbf3bfff640 1 -- 172.21.15.115:0/2719045427 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf440c9000 con 0x7fbf4406b6f0 2023-12-01T03:12:31.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.822+0000 7fbf437fe640 1 --2- 172.21.15.115:0/2719045427 >> v2:172.21.15.142:3300/0 conn(0x7fbf44159dd0 0x7fbf440c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.823+0000 7fbf3bfff640 1 --2- 172.21.15.115:0/2719045427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf4406b6f0 0x7fbf440c9d20 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fbf440cf4e0 tx=0x7fbf2c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6329d9779462b101 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.823+0000 7fbf42ffd640 1 -- 172.21.15.115:0/2719045427 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c031750 con 0x7fbf4406b6f0 2023-12-01T03:12:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.823+0000 7fbf42ffd640 1 -- 172.21.15.115:0/2719045427 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c0318b0 con 0x7fbf4406b6f0 2023-12-01T03:12:31.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.823+0000 7fbf42ffd640 1 -- 172.21.15.115:0/2719045427 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c031c40 con 0x7fbf4406b6f0 2023-12-01T03:12:31.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.824+0000 7fbf49abd640 1 -- 172.21.15.115:0/2719045427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf4406b6f0 msgr2=0x7fbf440c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:31.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.824+0000 7fbf49abd640 1 --2- 172.21.15.115:0/2719045427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf4406b6f0 0x7fbf440c9d20 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fbf440cf4e0 tx=0x7fbf2c02b7f0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.824+0000 7fbf49abd640 1 -- 172.21.15.115:0/2719045427 shutdown_connections 2023-12-01T03:12:31.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.824+0000 7fbf49abd640 1 --2- 172.21.15.115:0/2719045427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf4406b6f0 0x7fbf440c9d20 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.824+0000 7fbf49abd640 1 --2- 172.21.15.115:0/2719045427 >> v2:172.21.15.142:3300/0 conn(0x7fbf44159dd0 0x7fbf440c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.824+0000 7fbf49abd640 1 -- 172.21.15.115:0/2719045427 >> 172.21.15.115:0/2719045427 conn(0x7fbf440ce880 msgr2=0x7fbf440cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:31.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.824+0000 7fbf49abd640 1 -- 172.21.15.115:0/2719045427 shutdown_connections 2023-12-01T03:12:31.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.824+0000 7fbf49abd640 1 -- 172.21.15.115:0/2719045427 wait complete. 2023-12-01T03:12:31.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.826+0000 7fbf49abd640 1 Processor -- start 2023-12-01T03:12:31.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.826+0000 7fbf49abd640 1 -- start start 2023-12-01T03:12:31.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf49abd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf4406b6f0 0x7fbf4414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf49abd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440ade80 0x7fbf440ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf49abd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf44159dd0 0x7fbf440acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf49abd640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf441f69b0 con 0x7fbf440ade80 2023-12-01T03:12:31.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf49abd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf441f28c0 con 0x7fbf4406b6f0 2023-12-01T03:12:31.828 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf49abd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf441f2ba0 con 0x7fbf44159dd0 2023-12-01T03:12:31.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf3bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440ade80 0x7fbf440ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:31.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf3bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440ade80 0x7fbf440ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48042/0 (socket says 172.21.15.115:48042) 2023-12-01T03:12:31.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf3bfff640 1 -- 172.21.15.115:0/1461453737 learned_addr learned my addr 172.21.15.115:0/1461453737 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:31.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf3bfff640 1 -- 172.21.15.115:0/1461453737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf44159dd0 msgr2=0x7fbf440acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:31.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf3bfff640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf44159dd0 0x7fbf440acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf3bfff640 1 -- 172.21.15.115:0/1461453737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf4406b6f0 msgr2=0x7fbf4414dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:31.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf3bfff640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf4406b6f0 0x7fbf4414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:31.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.827+0000 7fbf3bfff640 1 -- 172.21.15.115:0/1461453737 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c0094d0 con 0x7fbf440ade80 2023-12-01T03:12:31.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.828+0000 7fbf3bfff640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440ade80 0x7fbf440ac8a0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c000c00 tx=0x7fbf2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:31.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.828+0000 7fbf40ff9640 1 -- 172.21.15.115:0/1461453737 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c033c80 con 0x7fbf440ade80 2023-12-01T03:12:31.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.828+0000 7fbf40ff9640 1 -- 172.21.15.115:0/1461453737 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c033de0 con 0x7fbf440ade80 2023-12-01T03:12:31.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.828+0000 7fbf40ff9640 1 -- 172.21.15.115:0/1461453737 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c03c620 con 0x7fbf440ade80 2023-12-01T03:12:31.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.828+0000 7fbf49abd640 1 -- 172.21.15.115:0/1461453737 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf441f2e80 con 0x7fbf440ade80 2023-12-01T03:12:31.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.828+0000 7fbf49abd640 1 -- 172.21.15.115:0/1461453737 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4414e1c0 con 0x7fbf440ade80 2023-12-01T03:12:31.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.829+0000 7fbf49abd640 1 -- 172.21.15.115:0/1461453737 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf441f30c0 con 0x7fbf440ade80 2023-12-01T03:12:31.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.830+0000 7fbf40ff9640 1 -- 172.21.15.115:0/1461453737 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fbf2c03a040 con 0x7fbf440ade80 2023-12-01T03:12:31.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.831+0000 7fbf40ff9640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2406a670 0x7fbf2406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:31.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.831+0000 7fbf40ff9640 1 -- 172.21.15.115:0/1461453737 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fbf2c0aa950 con 0x7fbf440ade80 2023-12-01T03:12:31.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.831+0000 7fbf437fe640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2406a670 0x7fbf2406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:31.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.832+0000 7fbf437fe640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2406a670 0x7fbf2406cb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006fd0 tx=0x7fbf34008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:31.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:31.835+0000 7fbf40ff9640 1 -- 172.21.15.115:0/1461453737 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbf441f30c0 con 0x7fbf440ade80 2023-12-01T03:12:32.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.025+0000 7fbf49abd640 1 -- 172.21.15.115:0/1461453737 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf441f30c0 con 0x7fbf440ade80 2023-12-01T03:12:32.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.026+0000 7fbf40ff9640 1 -- 172.21.15.115:0/1461453737 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v252) v1 ==== 74+0+12517 (secure 0 0 0) 0x7fbf441f30c0 con 0x7fbf440ade80 2023-12-01T03:12:32.027 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:32.028 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":252,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:31.066011+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":237,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.029+0000 7fbf49abd640 1 -- 172.21.15.115:0/1461453737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2406a670 msgr2=0x7fbf2406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.029+0000 7fbf49abd640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2406a670 0x7fbf2406cb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006fd0 tx=0x7fbf34008040 comp rx=0 tx=0).stop 2023-12-01T03:12:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.029+0000 7fbf49abd640 1 -- 172.21.15.115:0/1461453737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440ade80 msgr2=0x7fbf440ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.029+0000 7fbf49abd640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440ade80 0x7fbf440ac8a0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c000c00 tx=0x7fbf2c0046c0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.030+0000 7fbf49abd640 1 -- 172.21.15.115:0/1461453737 shutdown_connections 2023-12-01T03:12:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.030+0000 7fbf49abd640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbf2406a670 0x7fbf2406cb30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.030+0000 7fbf49abd640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf44159dd0 0x7fbf440acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.030+0000 7fbf49abd640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf440ade80 0x7fbf440ac8a0 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.030+0000 7fbf49abd640 1 --2- 172.21.15.115:0/1461453737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf4406b6f0 0x7fbf4414dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.030+0000 7fbf49abd640 1 -- 172.21.15.115:0/1461453737 >> 172.21.15.115:0/1461453737 conn(0x7fbf440ce880 msgr2=0x7fbf440705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:32.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.030+0000 7fbf49abd640 1 -- 172.21.15.115:0/1461453737 shutdown_connections 2023-12-01T03:12:32.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.030+0000 7fbf49abd640 1 -- 172.21.15.115:0/1461453737 wait complete. 2023-12-01T03:12:32.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:12:32.050 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:12:32.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.266+0000 7f4f68dae640 1 Processor -- start 2023-12-01T03:12:32.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.267+0000 7f4f68dae640 1 -- start start 2023-12-01T03:12:32.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.268+0000 7f4f68dae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6415cd90 0x7f4f640cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.268+0000 7f4f68dae640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4f640b5fa0 0x7f4f64161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.268+0000 7f4f68dae640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f640ce5e0 con 0x7f4f640cf650 2023-12-01T03:12:32.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.268+0000 7f4f68dae640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f640d0ec0 con 0x7f4f640b5fa0 2023-12-01T03:12:32.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.268+0000 7f4f68dae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64161950 con 0x7f4f6415cd90 2023-12-01T03:12:32.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.268+0000 7f4f62d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f4f640cf650 0x7f4f640b4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35118/0 (socket says 172.21.15.115:35118) 2023-12-01T03:12:32.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.269+0000 7f4f62d9d640 1 -- 172.21.15.115:0/2564423885 learned_addr learned my addr 172.21.15.115:0/2564423885 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:32.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.269+0000 7f4f61d9b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f4f640b5fa0 0x7f4f64161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:32.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.269+0000 7f4f6259c640 1 --2- 172.21.15.115:0/2564423885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6415cd90 0x7f4f640cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:32.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.269+0000 7f4f6159a640 1 -- 172.21.15.115:0/2564423885 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320192838 0 0) 0x7f4f5c001110 con 0x7f4f640cf650 2023-12-01T03:12:32.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.269+0000 7f4f61d9b640 1 -- 172.21.15.115:0/2564423885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6415cd90 msgr2=0x7f4f640cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:32.270 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.269+0000 7f4f61d9b640 1 --2- 172.21.15.115:0/2564423885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6415cd90 0x7f4f640cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.269+0000 7f4f61d9b640 1 -- 172.21.15.115:0/2564423885 >> v1:172.21.15.115:6789/0 conn(0x7f4f640cf650 legacy=0x7f4f640b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.269+0000 7f4f61d9b640 1 -- 172.21.15.115:0/2564423885 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f640ce300 con 0x7f4f640b5fa0 2023-12-01T03:12:32.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.270+0000 7f4f61d9b640 1 --2- 172.21.15.115:0/2564423885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640b5fa0 0x7f4f64161410 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0097a0 tx=0x7f4f4c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a287f28c8c6cb605 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:32.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f6159a640 1 -- 172.21.15.115:0/2564423885 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c031830 con 0x7f4f640b5fa0 2023-12-01T03:12:32.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f6159a640 1 -- 172.21.15.115:0/2564423885 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c031990 con 0x7f4f640b5fa0 2023-12-01T03:12:32.271 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f6159a640 1 -- 172.21.15.115:0/2564423885 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c031d20 con 0x7f4f640b5fa0 2023-12-01T03:12:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f68dae640 1 -- 172.21.15.115:0/2564423885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640b5fa0 msgr2=0x7f4f64161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f68dae640 1 --2- 172.21.15.115:0/2564423885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640b5fa0 0x7f4f64161410 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0097a0 tx=0x7f4f4c02b8d0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f68dae640 1 -- 172.21.15.115:0/2564423885 shutdown_connections 2023-12-01T03:12:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f68dae640 1 --2- 172.21.15.115:0/2564423885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640b5fa0 0x7f4f64161410 unknown :-1 s=CLOSED pgs=2938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f68dae640 1 --2- 172.21.15.115:0/2564423885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f6415cd90 0x7f4f640cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f68dae640 1 -- 172.21.15.115:0/2564423885 >> 172.21.15.115:0/2564423885 conn(0x7f4f64065960 msgr2=0x7f4f64065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f68dae640 1 -- 172.21.15.115:0/2564423885 shutdown_connections 2023-12-01T03:12:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.271+0000 7f4f68dae640 1 -- 172.21.15.115:0/2564423885 wait complete. 2023-12-01T03:12:32.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.273+0000 7f4f68dae640 1 Processor -- start 2023-12-01T03:12:32.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.273+0000 7f4f68dae640 1 -- start start 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.274+0000 7f4f68dae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f640b5fa0 0x7f4f641502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.274+0000 7f4f68dae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640cf650 0x7f4f641f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.274+0000 7f4f68dae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6415cd90 0x7f4f64155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.274+0000 7f4f68dae640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640cbd50 con 0x7f4f640cf650 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.274+0000 7f4f68dae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f641f6990 con 0x7f4f6415cd90 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.274+0000 7f4f68dae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640ccc30 con 0x7f4f640b5fa0 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f61d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640cf650 0x7f4f641f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f61d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640cf650 0x7f4f641f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48064/0 (socket says 172.21.15.115:48064) 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f61d9b640 1 -- 172.21.15.115:0/3967782164 learned_addr learned my addr 172.21.15.115:0/3967782164 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f6259c640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f640b5fa0 0x7f4f641502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f61d9b640 1 -- 172.21.15.115:0/3967782164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f640b5fa0 msgr2=0x7f4f641502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f62d9d640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6415cd90 0x7f4f64155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:32.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f61d9b640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f640b5fa0 0x7f4f641502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f61d9b640 1 -- 172.21.15.115:0/3967782164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6415cd90 msgr2=0x7f4f64155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f61d9b640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6415cd90 0x7f4f64155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f61d9b640 1 -- 172.21.15.115:0/3967782164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c009400 con 0x7f4f640cf650 2023-12-01T03:12:32.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f6259c640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f640b5fa0 0x7f4f641502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.275+0000 7f4f61d9b640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640cf650 0x7f4f641f6450 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c02bee0 tx=0x7f4f4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.276+0000 7f4f5affd640 1 -- 172.21.15.115:0/3967782164 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c03a6f0 con 0x7f4f640cf650 2023-12-01T03:12:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.276+0000 7f4f5affd640 1 -- 172.21.15.115:0/3967782164 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c03a870 con 0x7f4f640cf650 2023-12-01T03:12:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.276+0000 7f4f68dae640 1 -- 172.21.15.115:0/3967782164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f640cba70 con 0x7f4f640cf650 2023-12-01T03:12:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.276+0000 7f4f5affd640 1 -- 172.21.15.115:0/3967782164 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c03ac20 con 0x7f4f640cf650 2023-12-01T03:12:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.276+0000 7f4f68dae640 1 -- 172.21.15.115:0/3967782164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f64150ef0 con 0x7f4f640cf650 2023-12-01T03:12:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.277+0000 7f4f68dae640 1 -- 172.21.15.115:0/3967782164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f64150a70 con 0x7f4f640cf650 2023-12-01T03:12:32.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.278+0000 7f4f5affd640 1 -- 172.21.15.115:0/3967782164 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f4f4c04c3b0 con 0x7f4f640cf650 2023-12-01T03:12:32.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.279+0000 7f4f5affd640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f4406a670 0x7f4f4406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.279+0000 7f4f5affd640 1 -- 172.21.15.115:0/3967782164 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 4901+0+0 (secure 0 0 0) 0x7f4f4c0aa8d0 con 0x7f4f640cf650 2023-12-01T03:12:32.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.279+0000 7f4f6259c640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f4406a670 0x7f4f4406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:32.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.280+0000 7f4f6259c640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f4406a670 0x7f4f4406cb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009f50 tx=0x7f4f5400b040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:32.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.283+0000 7f4f5affd640 1 -- 172.21.15.115:0/3967782164 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4f64150a70 con 0x7f4f640cf650 2023-12-01T03:12:32.471 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.470+0000 7f4f68dae640 1 -- 172.21.15.115:0/3967782164 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f64150a70 con 0x7f4f640cf650 2023-12-01T03:12:32.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.472+0000 7f4f5affd640 1 -- 172.21.15.115:0/3967782164 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v252) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f64150a70 con 0x7f4f640cf650 2023-12-01T03:12:32.472 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:12:32.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.474+0000 7f4f68dae640 1 -- 172.21.15.115:0/3967782164 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f4406a670 msgr2=0x7f4f4406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.474+0000 7f4f68dae640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f4406a670 0x7f4f4406cb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009f50 tx=0x7f4f5400b040 comp rx=0 tx=0).stop 2023-12-01T03:12:32.475 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.474+0000 7f4f68dae640 1 -- 172.21.15.115:0/3967782164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640cf650 msgr2=0x7f4f641f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.474+0000 7f4f68dae640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640cf650 0x7f4f641f6450 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c02bee0 tx=0x7f4f4c0046c0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.474+0000 7f4f68dae640 1 -- 172.21.15.115:0/3967782164 shutdown_connections 2023-12-01T03:12:32.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.474+0000 7f4f68dae640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f4f4406a670 0x7f4f4406cb30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.474+0000 7f4f68dae640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6415cd90 0x7f4f64155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.474+0000 7f4f68dae640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f640cf650 0x7f4f641f6450 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.474+0000 7f4f68dae640 1 --2- 172.21.15.115:0/3967782164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f640b5fa0 0x7f4f641502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.474+0000 7f4f68dae640 1 -- 172.21.15.115:0/3967782164 >> 172.21.15.115:0/3967782164 conn(0x7f4f64065960 msgr2=0x7f4f640d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:32.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.475+0000 7f4f68dae640 1 -- 172.21.15.115:0/3967782164 shutdown_connections 2023-12-01T03:12:32.476 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.475+0000 7f4f68dae640 1 -- 172.21.15.115:0/3967782164 wait complete. 2023-12-01T03:12:32.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T03:12:32.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.705+0000 7fc768331640 1 Processor -- start 2023-12-01T03:12:32.706 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.705+0000 7fc768331640 1 -- start start 2023-12-01T03:12:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.707+0000 7fc768331640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc76015e2f0 0x7fc7600ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.707+0000 7fc768331640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc7600adf10 0x7fc7600caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.707+0000 7fc768331640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7600add10 con 0x7fc76006f2f0 2023-12-01T03:12:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.707+0000 7fc768331640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7600c5760 con 0x7fc76015e2f0 2023-12-01T03:12:32.707 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.707+0000 7fc768331640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76015e6d0 con 0x7fc7600adf10 2023-12-01T03:12:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.707+0000 7fc7658cc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc76006f2f0 0x7fc7600c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35124/0 (socket says 172.21.15.115:35124) 2023-12-01T03:12:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.707+0000 7fc7658cc640 1 -- 172.21.15.115:0/38304408 learned_addr learned my addr 172.21.15.115:0/38304408 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.707+0000 7fc7660cd640 1 --2- 172.21.15.115:0/38304408 >> v2:172.21.15.115:3300/0 conn(0x7fc76015e2f0 0x7fc7600ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.707+0000 7fc7668ce640 1 --2- 172.21.15.115:0/38304408 >> v2:172.21.15.142:3300/0 conn(0x7fc7600adf10 0x7fc7600caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.708+0000 7fc7650cb640 1 -- 172.21.15.115:0/38304408 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550338392 0 0) 0x7fc7600add10 con 0x7fc76006f2f0 2023-12-01T03:12:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.708+0000 7fc7650cb640 1 -- 172.21.15.115:0/38304408 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc74c003520 con 0x7fc76006f2f0 2023-12-01T03:12:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.708+0000 7fc7660cd640 1 -- 172.21.15.115:0/38304408 >> v2:172.21.15.142:3300/0 conn(0x7fc7600adf10 msgr2=0x7fc7600caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.708+0000 7fc7660cd640 1 --2- 172.21.15.115:0/38304408 >> v2:172.21.15.142:3300/0 conn(0x7fc7600adf10 0x7fc7600caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.708+0000 7fc7660cd640 1 -- 172.21.15.115:0/38304408 >> v1:172.21.15.115:6789/0 conn(0x7fc76006f2f0 legacy=0x7fc7600c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.709 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.708+0000 7fc7660cd640 1 -- 172.21.15.115:0/38304408 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7600cf7b0 con 0x7fc76015e2f0 2023-12-01T03:12:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.709+0000 7fc7660cd640 1 --2- 172.21.15.115:0/38304408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76015e2f0 0x7fc7600ca660 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7fc7600c8e80 tx=0x7fc75c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be58567639ddade server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc7650cb640 1 -- 172.21.15.115:0/38304408 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c031710 con 0x7fc76015e2f0 2023-12-01T03:12:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc7650cb640 1 -- 172.21.15.115:0/38304408 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c02f050 con 0x7fc76015e2f0 2023-12-01T03:12:32.710 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc7650cb640 1 -- 172.21.15.115:0/38304408 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c031a90 con 0x7fc76015e2f0 2023-12-01T03:12:32.711 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc768331640 1 -- 172.21.15.115:0/38304408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76015e2f0 msgr2=0x7fc7600ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc768331640 1 --2- 172.21.15.115:0/38304408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76015e2f0 0x7fc7600ca660 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7fc7600c8e80 tx=0x7fc75c02b7b0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc768331640 1 -- 172.21.15.115:0/38304408 shutdown_connections 2023-12-01T03:12:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc768331640 1 --2- 172.21.15.115:0/38304408 >> v2:172.21.15.142:3300/0 conn(0x7fc7600adf10 0x7fc7600caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc768331640 1 --2- 172.21.15.115:0/38304408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76015e2f0 0x7fc7600ca660 unknown :-1 s=CLOSED pgs=2941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc768331640 1 -- 172.21.15.115:0/38304408 >> 172.21.15.115:0/38304408 conn(0x7fc7600ce880 msgr2=0x7fc7600cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc768331640 1 -- 172.21.15.115:0/38304408 shutdown_connections 2023-12-01T03:12:32.712 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.710+0000 7fc768331640 1 -- 172.21.15.115:0/38304408 wait complete. 2023-12-01T03:12:32.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.712+0000 7fc768331640 1 Processor -- start 2023-12-01T03:12:32.713 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.712+0000 7fc768331640 1 -- start start 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.713+0000 7fc768331640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76006f2f0 0x7fc760152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.713+0000 7fc768331640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7600adf10 0x7fc7600c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.713+0000 7fc768331640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc76015e2f0 0x7fc7600b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.713+0000 7fc768331640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600c5d10 con 0x7fc76006f2f0 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.713+0000 7fc768331640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600c23b0 con 0x7fc7600adf10 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.713+0000 7fc768331640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600c8160 con 0x7fc76015e2f0 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.713+0000 7fc7660cd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76006f2f0 0x7fc760152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.713+0000 7fc7660cd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76006f2f0 0x7fc760152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48090/0 (socket says 172.21.15.115:48090) 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.713+0000 7fc7660cd640 1 -- 172.21.15.115:0/1702363302 learned_addr learned my addr 172.21.15.115:0/1702363302 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.714+0000 7fc7660cd640 1 -- 172.21.15.115:0/1702363302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc76015e2f0 msgr2=0x7fc7600b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.714+0000 7fc7660cd640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc76015e2f0 0x7fc7600b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.714+0000 7fc7660cd640 1 -- 172.21.15.115:0/1702363302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7600adf10 msgr2=0x7fc7600c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.714+0000 7fc7660cd640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7600adf10 0x7fc7600c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.714+0000 7fc7660cd640 1 -- 172.21.15.115:0/1702363302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c009400 con 0x7fc76006f2f0 2023-12-01T03:12:32.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.714+0000 7fc7660cd640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76006f2f0 0x7fc760152e60 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7fc75c000c00 tx=0x7fc75c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:32.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.714+0000 7fc756ffd640 1 -- 172.21.15.115:0/1702363302 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c03c810 con 0x7fc76006f2f0 2023-12-01T03:12:32.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.715+0000 7fc756ffd640 1 -- 172.21.15.115:0/1702363302 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c03c970 con 0x7fc76006f2f0 2023-12-01T03:12:32.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.715+0000 7fc768331640 1 -- 172.21.15.115:0/1702363302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7600c8400 con 0x7fc76006f2f0 2023-12-01T03:12:32.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.715+0000 7fc756ffd640 1 -- 172.21.15.115:0/1702363302 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c03cd20 con 0x7fc76006f2f0 2023-12-01T03:12:32.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.715+0000 7fc768331640 1 -- 172.21.15.115:0/1702363302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7600c2600 con 0x7fc76006f2f0 2023-12-01T03:12:32.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.716+0000 7fc768331640 1 -- 172.21.15.115:0/1702363302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76006fda0 con 0x7fc76006f2f0 2023-12-01T03:12:32.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.717+0000 7fc756ffd640 1 -- 172.21.15.115:0/1702363302 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fc75c03a040 con 0x7fc76006f2f0 2023-12-01T03:12:32.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.718+0000 7fc756ffd640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc74406a670 0x7fc74406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:32.718 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.718+0000 7fc756ffd640 1 -- 172.21.15.115:0/1702363302 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fc75c0a9de0 con 0x7fc76006f2f0 2023-12-01T03:12:32.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.718+0000 7fc7658cc640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc74406a670 0x7fc74406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:32.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.719+0000 7fc7658cc640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc74406a670 0x7fc74406cb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc7600c4530 tx=0x7fc750007930 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:32.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.722+0000 7fc756ffd640 1 -- 172.21.15.115:0/1702363302 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc75c075340 con 0x7fc76006f2f0 2023-12-01T03:12:32.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.915+0000 7fc768331640 1 -- 172.21.15.115:0/1702363302 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc7600b43a0 con 0x7fc76006f2f0 2023-12-01T03:12:32.916 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.916+0000 7fc756ffd640 1 -- 172.21.15.115:0/1702363302 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v252) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc75c006e10 con 0x7fc76006f2f0 2023-12-01T03:12:32.916 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:32.917 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T03:12:32.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:32.917 INFO:teuthology.orchestra.run.smithi115.stdout: "poolnum": 1, 2023-12-01T03:12:32.917 INFO:teuthology.orchestra.run.smithi115.stdout: "poolname": ".mgr" 2023-12-01T03:12:32.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:32.917 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T03:12:32.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 -- 172.21.15.115:0/1702363302 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc74406a670 msgr2=0x7fc74406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc74406a670 0x7fc74406cb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc7600c4530 tx=0x7fc750007930 comp rx=0 tx=0).stop 2023-12-01T03:12:32.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 -- 172.21.15.115:0/1702363302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76006f2f0 msgr2=0x7fc760152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:32.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76006f2f0 0x7fc760152e60 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7fc75c000c00 tx=0x7fc75c0046c0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 -- 172.21.15.115:0/1702363302 shutdown_connections 2023-12-01T03:12:32.920 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc74406a670 0x7fc74406cb30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc76015e2f0 0x7fc7600b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7600adf10 0x7fc7600c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 --2- 172.21.15.115:0/1702363302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc76006f2f0 0x7fc760152e60 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:32.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 -- 172.21.15.115:0/1702363302 >> 172.21.15.115:0/1702363302 conn(0x7fc7600ce880 msgr2=0x7fc7600ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:32.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 -- 172.21.15.115:0/1702363302 shutdown_connections 2023-12-01T03:12:32.921 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:32.918+0000 7fc768331640 1 -- 172.21.15.115:0/1702363302 wait complete. 2023-12-01T03:12:32.938 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T03:12:32.938 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T03:12:33.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.146+0000 7fa46db99640 1 Processor -- start 2023-12-01T03:12:33.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.146+0000 7fa46db99640 1 -- start start 2023-12-01T03:12:33.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.147+0000 7fa46db99640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa46815cfd0 0x7fa4680b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:33.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.147+0000 7fa46db99640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4680d1280 con 0x7fa4680d2a90 2023-12-01T03:12:33.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.147+0000 7fa46db99640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4680d0070 con 0x7fa4680cfa50 2023-12-01T03:12:33.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.147+0000 7fa46db99640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4680d0d00 con 0x7fa46815cfd0 2023-12-01T03:12:33.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.147+0000 7fa466ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fa4680d2a90 0x7fa4680cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35132/0 (socket says 172.21.15.115:35132) 2023-12-01T03:12:33.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.147+0000 7fa466ffd640 1 -- 172.21.15.115:0/1983327329 learned_addr learned my addr 172.21.15.115:0/1983327329 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232652732 0 0) 0x7fa4680d1280 con 0x7fa4680d2a90 2023-12-01T03:12:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa454003520 con 0x7fa4680d2a90 2023-12-01T03:12:33.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4677fe640 1 --2- 172.21.15.115:0/1983327329 >> v2:172.21.15.142:3300/0 conn(0x7fa46815cfd0 0x7fa4680b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2813320873 0 0) 0x7fa454003520 con 0x7fa4680d2a90 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 >> v2:172.21.15.142:3300/0 conn(0x7fa46815cfd0 msgr2=0x7fa4680b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 --2- 172.21.15.115:0/1983327329 >> v2:172.21.15.142:3300/0 conn(0x7fa46815cfd0 0x7fa4680b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 >> v1:172.21.15.142:6789/0 conn(0x7fa4680cfa50 legacy=0x7fa4680d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4680d0350 con 0x7fa4680d2a90 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fa4500019f0 con 0x7fa4680d2a90 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 >> v1:172.21.15.115:6789/0 conn(0x7fa4680d2a90 legacy=0x7fa4680cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4540039c0 0x7fa454003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.148+0000 7fa4667fc640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540047c0 0x7fa454004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa4667fc640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4540051d0 0x7fa45400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680d0d00 con 0x7fa4540039c0 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45400c100 con 0x7fa4540051d0 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45400c2c0 con 0x7fa4540047c0 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa4677fe640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4540039c0 0x7fa454003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:33.151 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa466ffd640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540047c0 0x7fa454004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:33.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa467fff640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4540051d0 0x7fa45400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:33.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa4677fe640 1 -- 172.21.15.115:0/1983327329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540047c0 msgr2=0x7fa454004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:33.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa4677fe640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540047c0 0x7fa454004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa4677fe640 1 -- 172.21.15.115:0/1983327329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4540051d0 msgr2=0x7fa45400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:33.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa4677fe640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4540051d0 0x7fa45400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa4677fe640 1 -- 172.21.15.115:0/1983327329 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa454003520 con 0x7fa4540039c0 2023-12-01T03:12:33.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa466ffd640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540047c0 0x7fa454004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:33.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa467fff640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4540051d0 0x7fa45400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:33.152 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.149+0000 7fa467fff640 1 -- 172.21.15.115:0/1983327329 reap_dead start 2023-12-01T03:12:33.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.151+0000 7fa4677fe640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4540039c0 0x7fa454003dc0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7fa45800ba00 tx=0x7fa4540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb81a5038aee51a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:33.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.151+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458032620 con 0x7fa4540039c0 2023-12-01T03:12:33.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.151+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa458032780 con 0x7fa4540039c0 2023-12-01T03:12:33.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.151+0000 7fa4667fc640 1 -- 172.21.15.115:0/1983327329 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458032b10 con 0x7fa4540039c0 2023-12-01T03:12:33.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.152+0000 7fa46db99640 1 -- 172.21.15.115:0/1983327329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4540039c0 msgr2=0x7fa454003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:33.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.152+0000 7fa46db99640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4540039c0 0x7fa454003dc0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7fa45800ba00 tx=0x7fa4540057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.152+0000 7fa46db99640 1 -- 172.21.15.115:0/1983327329 shutdown_connections 2023-12-01T03:12:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.152+0000 7fa46db99640 1 --2- 172.21.15.115:0/1983327329 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa4540039c0 0x7fa454003dc0 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.152+0000 7fa46db99640 1 -- 172.21.15.115:0/1983327329 >> 172.21.15.115:0/1983327329 conn(0x7fa468065960 msgr2=0x7fa468065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.152+0000 7fa46db99640 1 -- 172.21.15.115:0/1983327329 shutdown_connections 2023-12-01T03:12:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.152+0000 7fa46db99640 1 -- 172.21.15.115:0/1983327329 wait complete. 2023-12-01T03:12:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.154+0000 7fa46db99640 1 Processor -- start 2023-12-01T03:12:33.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.154+0000 7fa46db99640 1 -- start start 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa46db99640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540039c0 0x7fa4681f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa46db99640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa46815e0f0 0x7fa46815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa46db99640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa46815fdf0 0x7fa46814ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa46db99640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680caf60 con 0x7fa46815fdf0 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa46db99640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4681517b0 con 0x7fa46815e0f0 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa46db99640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680cb200 con 0x7fa4540039c0 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa467fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa46815fdf0 0x7fa46814ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa467fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa46815fdf0 0x7fa46814ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48108/0 (socket says 172.21.15.115:48108) 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa467fff640 1 -- 172.21.15.115:0/4157633624 learned_addr learned my addr 172.21.15.115:0/4157633624 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa4677fe640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540039c0 0x7fa4681f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa466ffd640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa46815e0f0 0x7fa46815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa467fff640 1 -- 172.21.15.115:0/4157633624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540039c0 msgr2=0x7fa4681f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa467fff640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540039c0 0x7fa4681f2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa467fff640 1 -- 172.21.15.115:0/4157633624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa46815e0f0 msgr2=0x7fa46815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa467fff640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa46815e0f0 0x7fa46815f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa467fff640 1 -- 172.21.15.115:0/4157633624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46815d530 con 0x7fa46815fdf0 2023-12-01T03:12:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.155+0000 7fa4677fe640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540039c0 0x7fa4681f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.156+0000 7fa467fff640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa46815fdf0 0x7fa46814ea50 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7fa4680b4730 tx=0x7fa46000d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.156+0000 7fa45ffff640 1 -- 172.21.15.115:0/4157633624 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460017070 con 0x7fa46815fdf0 2023-12-01T03:12:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.156+0000 7fa45ffff640 1 -- 172.21.15.115:0/4157633624 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46000de60 con 0x7fa46815fdf0 2023-12-01T03:12:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.156+0000 7fa46db99640 1 -- 172.21.15.115:0/4157633624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4680cb4e0 con 0x7fa46815fdf0 2023-12-01T03:12:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.156+0000 7fa45ffff640 1 -- 172.21.15.115:0/4157633624 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460005260 con 0x7fa46815fdf0 2023-12-01T03:12:33.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.156+0000 7fa46db99640 1 -- 172.21.15.115:0/4157633624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa468151a90 con 0x7fa46815fdf0 2023-12-01T03:12:33.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.157+0000 7fa46db99640 1 -- 172.21.15.115:0/4157633624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4680cb720 con 0x7fa46815fdf0 2023-12-01T03:12:33.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.158+0000 7fa45ffff640 1 -- 172.21.15.115:0/4157633624 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fa4600053c0 con 0x7fa46815fdf0 2023-12-01T03:12:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.159+0000 7fa45ffff640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa44806a670 0x7fa44806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.160+0000 7fa45ffff640 1 -- 172.21.15.115:0/4157633624 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 4901+0+0 (secure 0 0 0) 0x7fa460087fd0 con 0x7fa46815fdf0 2023-12-01T03:12:33.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.160+0000 7fa4677fe640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa44806a670 0x7fa44806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:33.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.160+0000 7fa4677fe640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa44806a670 0x7fa44806cb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4540057a0 tx=0x7fa45803d040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:33.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.164+0000 7fa45ffff640 1 -- 172.21.15.115:0/4157633624 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa4680cb720 con 0x7fa46815fdf0 2023-12-01T03:12:33.356 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.355+0000 7fa46db99640 1 -- 172.21.15.115:0/4157633624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa4680702e0 con 0x7fa46815fdf0 2023-12-01T03:12:33.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.817+0000 7fa45ffff640 1 -- 172.21.15.115:0/4157633624 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v253) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa460053460 con 0x7fa46815fdf0 2023-12-01T03:12:33.819 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' created 2023-12-01T03:12:33.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.821+0000 7fa46db99640 1 -- 172.21.15.115:0/4157633624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa44806a670 msgr2=0x7fa44806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:33.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.821+0000 7fa46db99640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa44806a670 0x7fa44806cb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4540057a0 tx=0x7fa45803d040 comp rx=0 tx=0).stop 2023-12-01T03:12:33.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.822+0000 7fa46db99640 1 -- 172.21.15.115:0/4157633624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa46815fdf0 msgr2=0x7fa46814ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:33.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.822+0000 7fa46db99640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa46815fdf0 0x7fa46814ea50 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7fa4680b4730 tx=0x7fa46000d660 comp rx=0 tx=0).stop 2023-12-01T03:12:33.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.822+0000 7fa46db99640 1 -- 172.21.15.115:0/4157633624 shutdown_connections 2023-12-01T03:12:33.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.822+0000 7fa46db99640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa44806a670 0x7fa44806cb30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.822+0000 7fa46db99640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa46815fdf0 0x7fa46814ea50 unknown :-1 s=CLOSED pgs=2945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.822+0000 7fa46db99640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa46815e0f0 0x7fa46815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.822+0000 7fa46db99640 1 --2- 172.21.15.115:0/4157633624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4540039c0 0x7fa4681f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:33.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.822+0000 7fa46db99640 1 -- 172.21.15.115:0/4157633624 >> 172.21.15.115:0/4157633624 conn(0x7fa468065960 msgr2=0x7fa4680b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:33.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.822+0000 7fa46db99640 1 -- 172.21.15.115:0/4157633624 shutdown_connections 2023-12-01T03:12:33.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:33.822+0000 7fa46db99640 1 -- 172.21.15.115:0/4157633624 wait complete. 2023-12-01T03:12:33.848 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T03:12:34.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.065+0000 7f613c547640 1 Processor -- start 2023-12-01T03:12:34.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.065+0000 7f613c547640 1 -- start start 2023-12-01T03:12:34.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.067+0000 7f613c547640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f61340af820 0x7f61340afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:34.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.067+0000 7f613c547640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613406b280 con 0x7f61340b19c0 2023-12-01T03:12:34.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.067+0000 7f613c547640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613406b520 con 0x7f6134157250 2023-12-01T03:12:34.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.067+0000 7f613c547640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61340b0210 con 0x7f61340af820 2023-12-01T03:12:34.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.067+0000 7f613aae4640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f61340b19c0 0x7f61340b1e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35148/0 (socket says 172.21.15.115:35148) 2023-12-01T03:12:34.068 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.067+0000 7f613aae4640 1 -- 172.21.15.115:0/135723925 learned_addr learned my addr 172.21.15.115:0/135723925 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:34.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f6139ae2640 1 --2- 172.21.15.115:0/135723925 >> v2:172.21.15.142:3300/0 conn(0x7f61340af820 0x7f61340afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:34.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627539460 0 0) 0x7f613406b280 con 0x7f61340b19c0 2023-12-01T03:12:34.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6120003520 con 0x7f61340b19c0 2023-12-01T03:12:34.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3889104929 0 0) 0x7f6120003520 con 0x7f61340b19c0 2023-12-01T03:12:34.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 >> v2:172.21.15.142:3300/0 conn(0x7f61340af820 msgr2=0x7f61340afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:34.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 --2- 172.21.15.115:0/135723925 >> v2:172.21.15.142:3300/0 conn(0x7f61340af820 0x7f61340afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 >> v1:172.21.15.142:6789/0 conn(0x7f6134157250 legacy=0x7f613415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:34.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613406b800 con 0x7f61340b19c0 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f612c0019f0 con 0x7f61340b19c0 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 >> v1:172.21.15.115:6789/0 conn(0x7f61340b19c0 legacy=0x7f61340b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6120003970 0x7f6120003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6120004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6120005180 0x7f612000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61340b0210 con 0x7f61200047c0 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612000c020 con 0x7f6120005180 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.068+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612000c1e0 con 0x7f6120003970 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.069+0000 7f6139ae2640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6120004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.069+0000 7f613aae4640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6120005180 0x7f612000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.069+0000 7f613a2e3640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6120003970 0x7f6120003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.069+0000 7f6139ae2640 1 -- 172.21.15.115:0/135723925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6120003970 msgr2=0x7f6120003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.069+0000 7f6139ae2640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6120003970 0x7f6120003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.069+0000 7f6139ae2640 1 -- 172.21.15.115:0/135723925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6120005180 msgr2=0x7f612000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:34.071 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.069+0000 7f6139ae2640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6120005180 0x7f612000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.069+0000 7f6139ae2640 1 -- 172.21.15.115:0/135723925 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6120003520 con 0x7f61200047c0 2023-12-01T03:12:34.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.069+0000 7f613aae4640 1 -- 172.21.15.115:0/135723925 reap_dead start 2023-12-01T03:12:34.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.069+0000 7f613a2e3640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6120003970 0x7f6120003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.070+0000 7f6139ae2640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6120004c40 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f6124009ac0 tx=0x7f61200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990e7f0e29378e3c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.071+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612402c6b0 con 0x7f61200047c0 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.071+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612402c810 con 0x7f61200047c0 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.071+0000 7f61392e1640 1 -- 172.21.15.115:0/135723925 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612402cba0 con 0x7f61200047c0 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.071+0000 7f613c547640 1 -- 172.21.15.115:0/135723925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 msgr2=0x7f6120004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.071+0000 7f613c547640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6120004c40 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f6124009ac0 tx=0x7f61200057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.071+0000 7f613c547640 1 -- 172.21.15.115:0/135723925 shutdown_connections 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.071+0000 7f613c547640 1 --2- 172.21.15.115:0/135723925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6120004c40 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.071+0000 7f613c547640 1 -- 172.21.15.115:0/135723925 >> 172.21.15.115:0/135723925 conn(0x7f6134065960 msgr2=0x7f6134065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.071+0000 7f613c547640 1 -- 172.21.15.115:0/135723925 shutdown_connections 2023-12-01T03:12:34.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.071+0000 7f613c547640 1 -- 172.21.15.115:0/135723925 wait complete. 2023-12-01T03:12:34.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.073+0000 7f613c547640 1 Processor -- start 2023-12-01T03:12:34.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.073+0000 7f613c547640 1 -- start start 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.074+0000 7f613c547640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6134142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.074+0000 7f613c547640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6134159f90 0x7f613415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.074+0000 7f613c547640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6134154b70 0x7f6134155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.074+0000 7f613c547640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6134155560 con 0x7f61200047c0 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.074+0000 7f613c547640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61340c8bf0 con 0x7f6134154b70 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.074+0000 7f613c547640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6134142c50 con 0x7f6134159f90 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.074+0000 7f613a2e3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6134142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.075+0000 7f613a2e3640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6134142710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48122/0 (socket says 172.21.15.115:48122) 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.075+0000 7f613a2e3640 1 -- 172.21.15.115:0/86370110 learned_addr learned my addr 172.21.15.115:0/86370110 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.075+0000 7f6139ae2640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6134159f90 0x7f613415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.075+0000 7f613a2e3640 1 -- 172.21.15.115:0/86370110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6134159f90 msgr2=0x7f613415a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.075+0000 7f613a2e3640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6134159f90 0x7f613415a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.075+0000 7f613a2e3640 1 -- 172.21.15.115:0/86370110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6134154b70 msgr2=0x7f6134155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:34.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.075+0000 7f613a2e3640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6134154b70 0x7f6134155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.075+0000 7f613a2e3640 1 -- 172.21.15.115:0/86370110 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61340c9c50 con 0x7f61200047c0 2023-12-01T03:12:34.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.075+0000 7f613a2e3640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6134142710 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f613000d4a0 tx=0x7f613000d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:34.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.075+0000 7f612affd640 1 -- 172.21.15.115:0/86370110 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130005480 con 0x7f61200047c0 2023-12-01T03:12:34.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.076+0000 7f612affd640 1 -- 172.21.15.115:0/86370110 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6130007030 con 0x7f61200047c0 2023-12-01T03:12:34.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.076+0000 7f613c547640 1 -- 172.21.15.115:0/86370110 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61340c8e90 con 0x7f61200047c0 2023-12-01T03:12:34.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.076+0000 7f612affd640 1 -- 172.21.15.115:0/86370110 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130007400 con 0x7f61200047c0 2023-12-01T03:12:34.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.076+0000 7f613c547640 1 -- 172.21.15.115:0/86370110 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61340c9450 con 0x7f61200047c0 2023-12-01T03:12:34.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.077+0000 7f613c547640 1 -- 172.21.15.115:0/86370110 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6134065f70 con 0x7f61200047c0 2023-12-01T03:12:34.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.079+0000 7f612affd640 1 -- 172.21.15.115:0/86370110 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f6130007560 con 0x7f61200047c0 2023-12-01T03:12:34.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.080+0000 7f612affd640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f611806a670 0x7f611806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:34.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.080+0000 7f612affd640 1 -- 172.21.15.115:0/86370110 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 5271+0+0 (secure 0 0 0) 0x7f6130088980 con 0x7f61200047c0 2023-12-01T03:12:34.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.080+0000 7f6139ae2640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f611806a670 0x7f611806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:34.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.081+0000 7f6139ae2640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f611806a670 0x7f611806cb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6124036000 tx=0x7f6124030000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:34.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.084+0000 7f612affd640 1 -- 172.21.15.115:0/86370110 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6130053d20 con 0x7f61200047c0 2023-12-01T03:12:34.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.275+0000 7f613c547640 1 -- 172.21.15.115:0/86370110 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f613406bd90 con 0x7f61200047c0 2023-12-01T03:12:34.837 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.836+0000 7f612affd640 1 -- 172.21.15.115:0/86370110 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v254) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6130057340 con 0x7f61200047c0 2023-12-01T03:12:34.838 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' created 2023-12-01T03:12:34.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.842+0000 7f613c547640 1 -- 172.21.15.115:0/86370110 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f611806a670 msgr2=0x7f611806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:34.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.842+0000 7f613c547640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f611806a670 0x7f611806cb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6124036000 tx=0x7f6124030000 comp rx=0 tx=0).stop 2023-12-01T03:12:34.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.842+0000 7f613c547640 1 -- 172.21.15.115:0/86370110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 msgr2=0x7f6134142710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:34.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.842+0000 7f613c547640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6134142710 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f613000d4a0 tx=0x7f613000d960 comp rx=0 tx=0).stop 2023-12-01T03:12:34.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.842+0000 7f613c547640 1 -- 172.21.15.115:0/86370110 shutdown_connections 2023-12-01T03:12:34.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.842+0000 7f613c547640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f611806a670 0x7f611806cb30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.842+0000 7f613c547640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6134159f90 0x7f613415a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.842+0000 7f613c547640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6134154b70 0x7f6134155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.842+0000 7f613c547640 1 --2- 172.21.15.115:0/86370110 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f61200047c0 0x7f6134142710 unknown :-1 s=CLOSED pgs=2948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:34.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.842+0000 7f613c547640 1 -- 172.21.15.115:0/86370110 >> 172.21.15.115:0/86370110 conn(0x7f6134065960 msgr2=0x7f61340cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:34.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.843+0000 7f613c547640 1 -- 172.21.15.115:0/86370110 shutdown_connections 2023-12-01T03:12:34.844 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:34.843+0000 7f613c547640 1 -- 172.21.15.115:0/86370110 wait complete. 2023-12-01T03:12:34.875 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T03:12:35.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.069+0000 7fb33d5d2640 1 Processor -- start 2023-12-01T03:12:35.070 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.069+0000 7fb33d5d2640 1 -- start start 2023-12-01T03:12:35.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.071+0000 7fb33d5d2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb33815ac80 0x7fb3380cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.071+0000 7fb33d5d2640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb3380b0e60 0x7fb3380b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.071+0000 7fb33d5d2640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3380cc9a0 con 0x7fb3380cdaa0 2023-12-01T03:12:35.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.071+0000 7fb33d5d2640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33815b060 con 0x7fb3380b0e60 2023-12-01T03:12:35.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.071+0000 7fb33d5d2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3380b0ce0 con 0x7fb33815ac80 2023-12-01T03:12:35.072 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.072+0000 7fb32ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fb3380b0e60 0x7fb3380b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.072+0000 7fb3377fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb3380cdaa0 0x7fb33806b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35158/0 (socket says 172.21.15.115:35158) 2023-12-01T03:12:35.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.072+0000 7fb3377fe640 1 -- 172.21.15.115:0/3286554722 learned_addr learned my addr 172.21.15.115:0/3286554722 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:35.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.072+0000 7fb336ffd640 1 --2- 172.21.15.115:0/3286554722 >> v2:172.21.15.142:3300/0 conn(0x7fb33815ac80 0x7fb3380cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.072+0000 7fb32ffff640 1 -- 172.21.15.115:0/3286554722 >> v2:172.21.15.142:3300/0 conn(0x7fb33815ac80 msgr2=0x7fb3380cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:35.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.072+0000 7fb32ffff640 1 --2- 172.21.15.115:0/3286554722 >> v2:172.21.15.142:3300/0 conn(0x7fb33815ac80 0x7fb3380cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.073 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.072+0000 7fb32ffff640 1 -- 172.21.15.115:0/3286554722 >> v1:172.21.15.115:6789/0 conn(0x7fb3380cdaa0 legacy=0x7fb33806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.072+0000 7fb32ffff640 1 -- 172.21.15.115:0/3286554722 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380cbd90 con 0x7fb3380b0e60 2023-12-01T03:12:35.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.072+0000 7fb336ffd640 1 --2- 172.21.15.115:0/3286554722 >> v2:172.21.15.142:3300/0 conn(0x7fb33815ac80 0x7fb3380cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.073+0000 7fb32ffff640 1 --2- 172.21.15.115:0/3286554722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 0x7fb3380b07a0 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7fb3380b3cc0 tx=0x7fb32002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85e887deb2f5cda server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.073+0000 7fb3367fc640 1 -- 172.21.15.115:0/3286554722 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320031830 con 0x7fb3380b0e60 2023-12-01T03:12:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.074+0000 7fb3367fc640 1 -- 172.21.15.115:0/3286554722 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb320031990 con 0x7fb3380b0e60 2023-12-01T03:12:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.074+0000 7fb3367fc640 1 -- 172.21.15.115:0/3286554722 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320031d20 con 0x7fb3380b0e60 2023-12-01T03:12:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.074+0000 7fb33d5d2640 1 -- 172.21.15.115:0/3286554722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 msgr2=0x7fb3380b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.074+0000 7fb33d5d2640 1 --2- 172.21.15.115:0/3286554722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 0x7fb3380b07a0 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7fb3380b3cc0 tx=0x7fb32002b8d0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.074+0000 7fb33d5d2640 1 -- 172.21.15.115:0/3286554722 shutdown_connections 2023-12-01T03:12:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.074+0000 7fb33d5d2640 1 --2- 172.21.15.115:0/3286554722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 0x7fb3380b07a0 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.074+0000 7fb33d5d2640 1 --2- 172.21.15.115:0/3286554722 >> v2:172.21.15.142:3300/0 conn(0x7fb33815ac80 0x7fb3380cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.074+0000 7fb33d5d2640 1 -- 172.21.15.115:0/3286554722 >> 172.21.15.115:0/3286554722 conn(0x7fb338065960 msgr2=0x7fb338065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.074+0000 7fb33d5d2640 1 -- 172.21.15.115:0/3286554722 shutdown_connections 2023-12-01T03:12:35.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.074+0000 7fb33d5d2640 1 -- 172.21.15.115:0/3286554722 wait complete. 2023-12-01T03:12:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.076+0000 7fb33d5d2640 1 Processor -- start 2023-12-01T03:12:35.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.076+0000 7fb33d5d2640 1 -- start start 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.077+0000 7fb33d5d2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 0x7fb33814b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.077+0000 7fb33d5d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3380cdaa0 0x7fb3381581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.077+0000 7fb33d5d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb33815ac80 0x7fb33814c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.077+0000 7fb33d5d2640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33814beb0 con 0x7fb3380b0e60 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.077+0000 7fb33d5d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb338151c20 con 0x7fb33815ac80 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.077+0000 7fb33d5d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb338157140 con 0x7fb3380cdaa0 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.078+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 0x7fb33814b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.078+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 0x7fb33814b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48128/0 (socket says 172.21.15.115:48128) 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.078+0000 7fb336ffd640 1 -- 172.21.15.115:0/251700576 learned_addr learned my addr 172.21.15.115:0/251700576 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.078+0000 7fb336ffd640 1 -- 172.21.15.115:0/251700576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3380cdaa0 msgr2=0x7fb3381581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.078+0000 7fb336ffd640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3380cdaa0 0x7fb3381581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.078+0000 7fb336ffd640 1 -- 172.21.15.115:0/251700576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb33815ac80 msgr2=0x7fb33814c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.078+0000 7fb336ffd640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb33815ac80 0x7fb33814c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.078+0000 7fb336ffd640 1 -- 172.21.15.115:0/251700576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb320009400 con 0x7fb3380b0e60 2023-12-01T03:12:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.078+0000 7fb336ffd640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 0x7fb33814b970 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fb32800ce00 tx=0x7fb32800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:35.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.078+0000 7fb32f7fe640 1 -- 172.21.15.115:0/251700576 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32800f890 con 0x7fb3380b0e60 2023-12-01T03:12:35.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.079+0000 7fb32f7fe640 1 -- 172.21.15.115:0/251700576 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32800f9f0 con 0x7fb3380b0e60 2023-12-01T03:12:35.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.079+0000 7fb33d5d2640 1 -- 172.21.15.115:0/251700576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb338157420 con 0x7fb3380b0e60 2023-12-01T03:12:35.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.079+0000 7fb32f7fe640 1 -- 172.21.15.115:0/251700576 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32800fdc0 con 0x7fb3380b0e60 2023-12-01T03:12:35.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.079+0000 7fb33d5d2640 1 -- 172.21.15.115:0/251700576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3381579e0 con 0x7fb3380b0e60 2023-12-01T03:12:35.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.080+0000 7fb33d5d2640 1 -- 172.21.15.115:0/251700576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33804e890 con 0x7fb3380b0e60 2023-12-01T03:12:35.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.081+0000 7fb32f7fe640 1 -- 172.21.15.115:0/251700576 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fb32800b040 con 0x7fb3380b0e60 2023-12-01T03:12:35.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.082+0000 7fb32f7fe640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb31806a670 0x7fb31806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.082+0000 7fb32f7fe640 1 -- 172.21.15.115:0/251700576 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 5653+0+0 (secure 0 0 0) 0x7fb328089820 con 0x7fb3380b0e60 2023-12-01T03:12:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.082+0000 7fb32ffff640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb31806a670 0x7fb31806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.083+0000 7fb32ffff640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb31806a670 0x7fb31806cb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb320036000 tx=0x7fb320030000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:35.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.088+0000 7fb32f7fe640 1 -- 172.21.15.115:0/251700576 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb3280549d0 con 0x7fb3380b0e60 2023-12-01T03:12:35.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.315+0000 7fb33d5d2640 1 -- 172.21.15.115:0/251700576 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb33804e040 con 0x7fb3380b0e60 2023-12-01T03:12:35.330 INFO:teuthology.orchestra.run.smithi115.stderr:new fs with metadata pool 32 and data pool 33 2023-12-01T03:12:35.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.329+0000 7fb32f7fe640 1 -- 172.21.15.115:0/251700576 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 32 and data pool 33 v122) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb32801d070 con 0x7fb3380b0e60 2023-12-01T03:12:35.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.333+0000 7fb33d5d2640 1 -- 172.21.15.115:0/251700576 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb31806a670 msgr2=0x7fb31806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.333+0000 7fb33d5d2640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb31806a670 0x7fb31806cb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb320036000 tx=0x7fb320030000 comp rx=0 tx=0).stop 2023-12-01T03:12:35.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.333+0000 7fb33d5d2640 1 -- 172.21.15.115:0/251700576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 msgr2=0x7fb33814b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.333+0000 7fb33d5d2640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 0x7fb33814b970 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fb32800ce00 tx=0x7fb32800a650 comp rx=0 tx=0).stop 2023-12-01T03:12:35.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.333+0000 7fb33d5d2640 1 -- 172.21.15.115:0/251700576 shutdown_connections 2023-12-01T03:12:35.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.333+0000 7fb33d5d2640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb31806a670 0x7fb31806cb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.333+0000 7fb33d5d2640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3380cdaa0 0x7fb3381581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.333+0000 7fb33d5d2640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb33815ac80 0x7fb33814c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.333+0000 7fb33d5d2640 1 --2- 172.21.15.115:0/251700576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3380b0e60 0x7fb33814b970 secure :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fb32800ce00 tx=0x7fb32800a650 comp rx=0 tx=0).stop 2023-12-01T03:12:35.338 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.333+0000 7fb33d5d2640 1 -- 172.21.15.115:0/251700576 >> 172.21.15.115:0/251700576 conn(0x7fb338065960 msgr2=0x7fb3380ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:35.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.339+0000 7fb33d5d2640 1 -- 172.21.15.115:0/251700576 shutdown_connections 2023-12-01T03:12:35.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.339+0000 7fb33d5d2640 1 -- 172.21.15.115:0/251700576 wait complete. 2023-12-01T03:12:35.370 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.548+0000 7f57ad518640 1 Processor -- start 2023-12-01T03:12:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.548+0000 7f57ad518640 1 -- start start 2023-12-01T03:12:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.549+0000 7f57ad518640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f57a80ccc80 0x7f57a806fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.550+0000 7f57ad518640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a815d600 con 0x7f57a80ad2b0 2023-12-01T03:12:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.550+0000 7f57ad518640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a806c4b0 con 0x7f57a815d220 2023-12-01T03:12:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.550+0000 7f57ad518640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a806cc10 con 0x7f57a80ccc80 2023-12-01T03:12:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.550+0000 7f57a77fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f57a80ad2b0 0x7f57a806fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35162/0 (socket says 172.21.15.115:35162) 2023-12-01T03:12:35.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.550+0000 7f57a77fe640 1 -- 172.21.15.115:0/3230041446 learned_addr learned my addr 172.21.15.115:0/3230041446 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:35.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.550+0000 7f57a67fc640 1 --2- 172.21.15.115:0/3230041446 >> v2:172.21.15.142:3300/0 conn(0x7f57a80ccc80 0x7f57a806fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.550+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249279753 0 0) 0x7f57a0001490 con 0x7f57a80ad2b0 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.550+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5790003520 con 0x7f57a80ad2b0 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848733692 0 0) 0x7f5790003520 con 0x7f57a80ad2b0 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 >> v2:172.21.15.142:3300/0 conn(0x7f57a80ccc80 msgr2=0x7f57a806fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 --2- 172.21.15.115:0/3230041446 >> v2:172.21.15.142:3300/0 conn(0x7f57a80ccc80 0x7f57a806fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 >> v1:172.21.15.142:6789/0 conn(0x7f57a815d220 legacy=0x7f57a80cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a806f2f0 con 0x7f57a80ad2b0 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f57a0001b80 con 0x7f57a80ad2b0 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 >> v1:172.21.15.115:6789/0 conn(0x7f57a80ad2b0 legacy=0x7f57a806fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57900039e0 0x7f5790003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57900047c0 0x7f5790004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900051f0 0x7f579000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a806cc10 con 0x7f57900047c0 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579000c150 con 0x7f57900051f0 2023-12-01T03:12:35.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579000c310 con 0x7f57900039e0 2023-12-01T03:12:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.551+0000 7f57a67fc640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57900047c0 0x7f5790004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.552+0000 7f57a77fe640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900051f0 0x7f579000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.552+0000 7f57a67fc640 1 -- 172.21.15.115:0/3230041446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57900039e0 msgr2=0x7f5790003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.552+0000 7f57a67fc640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57900039e0 0x7f5790003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.552+0000 7f57a67fc640 1 -- 172.21.15.115:0/3230041446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900051f0 msgr2=0x7f579000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.552+0000 7f57a67fc640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900051f0 0x7f579000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.552+0000 7f57a67fc640 1 -- 172.21.15.115:0/3230041446 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5790003520 con 0x7f57900047c0 2023-12-01T03:12:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.552+0000 7f57a77fe640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900051f0 0x7f579000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:35.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.552+0000 7f57a77fe640 1 -- 172.21.15.115:0/3230041446 reap_dead start 2023-12-01T03:12:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.553+0000 7f57a67fc640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57900047c0 0x7f5790004c40 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f579c009ac0 tx=0x7f57900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace0e7c60fa00fc1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.553+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c02c6b0 con 0x7f57900047c0 2023-12-01T03:12:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.553+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c02c810 con 0x7f57900047c0 2023-12-01T03:12:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.553+0000 7f57a5ffb640 1 -- 172.21.15.115:0/3230041446 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c02cba0 con 0x7f57900047c0 2023-12-01T03:12:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.553+0000 7f57ad518640 1 -- 172.21.15.115:0/3230041446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57900047c0 msgr2=0x7f5790004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.553+0000 7f57ad518640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57900047c0 0x7f5790004c40 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f579c009ac0 tx=0x7f57900057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.554+0000 7f57ad518640 1 -- 172.21.15.115:0/3230041446 shutdown_connections 2023-12-01T03:12:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.554+0000 7f57ad518640 1 --2- 172.21.15.115:0/3230041446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57900047c0 0x7f5790004c40 unknown :-1 s=CLOSED pgs=2953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.554+0000 7f57ad518640 1 -- 172.21.15.115:0/3230041446 >> 172.21.15.115:0/3230041446 conn(0x7f57a80ce880 msgr2=0x7f57a80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.554+0000 7f57ad518640 1 -- 172.21.15.115:0/3230041446 shutdown_connections 2023-12-01T03:12:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.554+0000 7f57ad518640 1 -- 172.21.15.115:0/3230041446 wait complete. 2023-12-01T03:12:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.555+0000 7f57ad518640 1 Processor -- start 2023-12-01T03:12:35.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.555+0000 7f57ad518640 1 -- start start 2023-12-01T03:12:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.556+0000 7f57ad518640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900047c0 0x7f57a80c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.556+0000 7f57ad518640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a80ca970 0x7f57a80c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.556+0000 7f57ad518640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a8158d00 0x7f57a8159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.556+0000 7f57ad518640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81596c0 con 0x7f57a80ca970 2023-12-01T03:12:35.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.556+0000 7f57ad518640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80c5e90 con 0x7f57900047c0 2023-12-01T03:12:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.556+0000 7f57ad518640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81f2640 con 0x7f57a8158d00 2023-12-01T03:12:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a80ca970 0x7f57a80c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a80ca970 0x7f57a80c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48148/0 (socket says 172.21.15.115:48148) 2023-12-01T03:12:35.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a67fc640 1 -- 172.21.15.115:0/3046110775 learned_addr learned my addr 172.21.15.115:0/3046110775 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a6ffd640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900047c0 0x7f57a80c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a77fe640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a8158d00 0x7f57a8159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a67fc640 1 -- 172.21.15.115:0/3046110775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a8158d00 msgr2=0x7f57a8159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a67fc640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a8158d00 0x7f57a8159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a67fc640 1 -- 172.21.15.115:0/3046110775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900047c0 msgr2=0x7f57a80c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a67fc640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900047c0 0x7f57a80c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a67fc640 1 -- 172.21.15.115:0/3046110775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a81f2920 con 0x7f57a80ca970 2023-12-01T03:12:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.557+0000 7f57a6ffd640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900047c0 0x7f57a80c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:35.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.558+0000 7f57a67fc640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a80ca970 0x7f57a80c4e50 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7f579c037040 tx=0x7f579c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.558+0000 7f578f7fe640 1 -- 172.21.15.115:0/3046110775 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c003810 con 0x7f57a80ca970 2023-12-01T03:12:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.558+0000 7f578f7fe640 1 -- 172.21.15.115:0/3046110775 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c003970 con 0x7f57a80ca970 2023-12-01T03:12:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.558+0000 7f578f7fe640 1 -- 172.21.15.115:0/3046110775 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c003d00 con 0x7f57a80ca970 2023-12-01T03:12:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.558+0000 7f57ad518640 1 -- 172.21.15.115:0/3046110775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a81f2c00 con 0x7f57a80ca970 2023-12-01T03:12:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.558+0000 7f57ad518640 1 -- 172.21.15.115:0/3046110775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a80cbb60 con 0x7f57a80ca970 2023-12-01T03:12:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.559+0000 7f57ad518640 1 -- 172.21.15.115:0/3046110775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a804e890 con 0x7f57a80ca970 2023-12-01T03:12:35.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.560+0000 7f578f7fe640 1 -- 172.21.15.115:0/3046110775 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f579c047450 con 0x7f57a80ca970 2023-12-01T03:12:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.561+0000 7f578f7fe640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f579406a6a0 0x7f579406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.562+0000 7f578f7fe640 1 -- 172.21.15.115:0/3046110775 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f579c0ab040 con 0x7f57a80ca970 2023-12-01T03:12:35.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.562+0000 7f57a6ffd640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f579406a6a0 0x7f579406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.563 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.562+0000 7f57a6ffd640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f579406a6a0 0x7f579406cb60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f57980016d0 tx=0x7f5798009040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:35.566 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.566+0000 7f578f7fe640 1 -- 172.21.15.115:0/3046110775 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f579c076350 con 0x7f57a80ca970 2023-12-01T03:12:35.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.754+0000 7f57ad518640 1 -- 172.21.15.115:0/3046110775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57a806a2f0 con 0x7f57a80ca970 2023-12-01T03:12:35.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.755+0000 7f578f7fe640 1 -- 172.21.15.115:0/3046110775 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v255) v1 ==== 74+0+16186 (secure 0 0 0) 0x7f579c079970 con 0x7f57a80ca970 2023-12-01T03:12:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:35.757 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":255,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:35.316461+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:12:33.411870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-01T03:12:34.335941+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 -- 172.21.15.115:0/3046110775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f579406a6a0 msgr2=0x7f579406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f579406a6a0 0x7f579406cb60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f57980016d0 tx=0x7f5798009040 comp rx=0 tx=0).stop 2023-12-01T03:12:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 -- 172.21.15.115:0/3046110775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a80ca970 msgr2=0x7f57a80c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a80ca970 0x7f57a80c4e50 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7f579c037040 tx=0x7f579c004730 comp rx=0 tx=0).stop 2023-12-01T03:12:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 -- 172.21.15.115:0/3046110775 shutdown_connections 2023-12-01T03:12:35.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f579406a6a0 0x7f579406cb60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a8158d00 0x7f57a8159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57a80ca970 0x7f57a80c4e50 unknown :-1 s=CLOSED pgs=2954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 --2- 172.21.15.115:0/3046110775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57900047c0 0x7f57a80c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 -- 172.21.15.115:0/3046110775 >> 172.21.15.115:0/3046110775 conn(0x7f57a80ce880 msgr2=0x7f57a80b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 -- 172.21.15.115:0/3046110775 shutdown_connections 2023-12-01T03:12:35.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.758+0000 7f57ad518640 1 -- 172.21.15.115:0/3046110775 wait complete. 2023-12-01T03:12:35.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:35.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.978+0000 7f8158666640 1 Processor -- start 2023-12-01T03:12:35.981 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.978+0000 7f8158666640 1 -- start start 2023-12-01T03:12:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.979+0000 7f8158666640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81500b5d80 0x7f81500d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.979+0000 7f8158666640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81500d1230 0x7f81500d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.979+0000 7f8158666640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81500d1b90 con 0x7f815015cd20 2023-12-01T03:12:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.979+0000 7f8158666640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81500b2160 con 0x7f81500b5d80 2023-12-01T03:12:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.979+0000 7f8158666640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81500b8c80 con 0x7f81500d1230 2023-12-01T03:12:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.979+0000 7f8155c01640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81500b5d80 0x7f81500d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.979+0000 7f8155c01640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f81500b5d80 0x7f81500d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48156/0 (socket says 172.21.15.115:48156) 2023-12-01T03:12:35.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.979+0000 7f8155c01640 1 -- 172.21.15.115:0/466746552 learned_addr learned my addr 172.21.15.115:0/466746552 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.980+0000 7f8155c01640 1 -- 172.21.15.115:0/466746552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81500d1230 msgr2=0x7f81500d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.980+0000 7f8155c01640 1 --2- 172.21.15.115:0/466746552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81500d1230 0x7f81500d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.980+0000 7f8155c01640 1 -- 172.21.15.115:0/466746552 >> v1:172.21.15.142:6789/0 conn(0x7f815015cd20 legacy=0x7f81500b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.980+0000 7f8155c01640 1 -- 172.21.15.115:0/466746552 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81500ce2e0 con 0x7f81500b5d80 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.981+0000 7f8155c01640 1 --2- 172.21.15.115:0/466746552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 0x7f81500d24d0 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7f813c009870 tx=0x7f813c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf644e384720c46 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.981+0000 7f8155400640 1 -- 172.21.15.115:0/466746552 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c0317c0 con 0x7f81500b5d80 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.981+0000 7f8155400640 1 -- 172.21.15.115:0/466746552 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c031920 con 0x7f81500b5d80 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.981+0000 7f8155400640 1 -- 172.21.15.115:0/466746552 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c031cb0 con 0x7f81500b5d80 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.982+0000 7f8158666640 1 -- 172.21.15.115:0/466746552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 msgr2=0x7f81500d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.982+0000 7f8158666640 1 --2- 172.21.15.115:0/466746552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 0x7f81500d24d0 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7f813c009870 tx=0x7f813c02b860 comp rx=0 tx=0).stop 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.982+0000 7f8158666640 1 -- 172.21.15.115:0/466746552 shutdown_connections 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.982+0000 7f8158666640 1 --2- 172.21.15.115:0/466746552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81500d1230 0x7f81500d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.982+0000 7f8158666640 1 --2- 172.21.15.115:0/466746552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 0x7f81500d24d0 unknown :-1 s=CLOSED pgs=2955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.982+0000 7f8158666640 1 -- 172.21.15.115:0/466746552 >> 172.21.15.115:0/466746552 conn(0x7f815006a0e0 msgr2=0x7f815006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.982+0000 7f8158666640 1 -- 172.21.15.115:0/466746552 shutdown_connections 2023-12-01T03:12:35.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.982+0000 7f8158666640 1 -- 172.21.15.115:0/466746552 wait complete. 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.984+0000 7f8158666640 1 Processor -- start 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.984+0000 7f8158666640 1 -- start start 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8158666640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 0x7f8150155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8158666640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81500d1230 0x7f815015f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8158666640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f815015cd20 0x7f815015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8158666640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500cbaf0 con 0x7f81500b5d80 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8158666640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500cbdd0 con 0x7f815015cd20 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8158666640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81500cc0b0 con 0x7f81500d1230 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8156402640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 0x7f8150155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8156402640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 0x7f8150155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48166/0 (socket says 172.21.15.115:48166) 2023-12-01T03:12:35.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8156402640 1 -- 172.21.15.115:0/4046056685 learned_addr learned my addr 172.21.15.115:0/4046056685 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8156402640 1 -- 172.21.15.115:0/4046056685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81500d1230 msgr2=0x7f815015f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8156402640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81500d1230 0x7f815015f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8156402640 1 -- 172.21.15.115:0/4046056685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f815015cd20 msgr2=0x7f815015f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8156402640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f815015cd20 0x7f815015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.985+0000 7f8156402640 1 -- 172.21.15.115:0/4046056685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0094d0 con 0x7f81500b5d80 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.986+0000 7f8156402640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 0x7f8150155280 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f814400ba50 tx=0x7f814400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.986+0000 7f814effd640 1 -- 172.21.15.115:0/4046056685 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814400cc10 con 0x7f81500b5d80 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.986+0000 7f814effd640 1 -- 172.21.15.115:0/4046056685 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814400cd70 con 0x7f81500b5d80 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.986+0000 7f814effd640 1 -- 172.21.15.115:0/4046056685 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144014680 con 0x7f81500b5d80 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.986+0000 7f8158666640 1 -- 172.21.15.115:0/4046056685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81500cc390 con 0x7f81500b5d80 2023-12-01T03:12:35.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.986+0000 7f8158666640 1 -- 172.21.15.115:0/4046056685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81500cb780 con 0x7f81500b5d80 2023-12-01T03:12:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.987+0000 7f8158666640 1 -- 172.21.15.115:0/4046056685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81500c7c60 con 0x7f81500b5d80 2023-12-01T03:12:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.988+0000 7f814effd640 1 -- 172.21.15.115:0/4046056685 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f814401d3b0 con 0x7f81500b5d80 2023-12-01T03:12:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.989+0000 7f814effd640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f813406a670 0x7f813406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:35.989 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.989+0000 7f814effd640 1 -- 172.21.15.115:0/4046056685 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f8144088e00 con 0x7f81500b5d80 2023-12-01T03:12:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.989+0000 7f8155c01640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f813406a670 0x7f813406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:35.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.990+0000 7f8155c01640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f813406a670 0x7f813406cb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f813c036000 tx=0x7f813c030000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:35.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:35.993+0000 7f814effd640 1 -- 172.21.15.115:0/4046056685 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f81500c7c60 con 0x7f81500b5d80 2023-12-01T03:12:36.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.169+0000 7f8158666640 1 -- 172.21.15.115:0/4046056685 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81500c7c60 con 0x7f81500b5d80 2023-12-01T03:12:36.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.171+0000 7f814effd640 1 -- 172.21.15.115:0/4046056685 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v255) v1 ==== 74+0+16186 (secure 0 0 0) 0x7f8144019020 con 0x7f81500b5d80 2023-12-01T03:12:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:36.173 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":255,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:35.316461+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:12:33.411870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-01T03:12:34.335941+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:36.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 -- 172.21.15.115:0/4046056685 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f813406a670 msgr2=0x7f813406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:36.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f813406a670 0x7f813406cb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f813c036000 tx=0x7f813c030000 comp rx=0 tx=0).stop 2023-12-01T03:12:36.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 -- 172.21.15.115:0/4046056685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 msgr2=0x7f8150155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:36.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 0x7f8150155280 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f814400ba50 tx=0x7f814400bf10 comp rx=0 tx=0).stop 2023-12-01T03:12:36.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 -- 172.21.15.115:0/4046056685 shutdown_connections 2023-12-01T03:12:36.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f813406a670 0x7f813406cb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:36.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f81500d1230 0x7f815015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:36.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f815015cd20 0x7f815015f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:36.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 --2- 172.21.15.115:0/4046056685 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81500b5d80 0x7f8150155280 unknown :-1 s=CLOSED pgs=2956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:36.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 -- 172.21.15.115:0/4046056685 >> 172.21.15.115:0/4046056685 conn(0x7f815006a0e0 msgr2=0x7f81501587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:36.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 -- 172.21.15.115:0/4046056685 shutdown_connections 2023-12-01T03:12:36.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.174+0000 7f8158666640 1 -- 172.21.15.115:0/4046056685 wait complete. 2023-12-01T03:12:36.193 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T03:12:36.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.410+0000 7fbb2759e640 1 Processor -- start 2023-12-01T03:12:36.411 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.411+0000 7fbb2759e640 1 -- start start 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.412+0000 7fbb2759e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbb280b04c0 0x7fbb280cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.412+0000 7fbb2759e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbb280cb630 0x7fbb2806fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.412+0000 7fbb2759e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb280cbaa0 con 0x7fbb2815ac00 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.412+0000 7fbb2759e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb280b12b0 con 0x7fbb280cb630 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.412+0000 7fbb2759e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2815afe0 con 0x7fbb280b04c0 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.413+0000 7fbb26d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbb280cb630 0x7fbb2806fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.413+0000 7fbb26d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbb280cb630 0x7fbb2806fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48174/0 (socket says 172.21.15.115:48174) 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.413+0000 7fbb26d9d640 1 -- 172.21.15.115:0/2656188834 learned_addr learned my addr 172.21.15.115:0/2656188834 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.413+0000 7fbb1ffff640 1 --2- 172.21.15.115:0/2656188834 >> v2:172.21.15.142:3300/0 conn(0x7fbb280b04c0 0x7fbb280cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.413+0000 7fbb26d9d640 1 -- 172.21.15.115:0/2656188834 >> v2:172.21.15.142:3300/0 conn(0x7fbb280b04c0 msgr2=0x7fbb280cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:36.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.413+0000 7fbb26d9d640 1 --2- 172.21.15.115:0/2656188834 >> v2:172.21.15.142:3300/0 conn(0x7fbb280b04c0 0x7fbb280cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:36.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.413+0000 7fbb26d9d640 1 -- 172.21.15.115:0/2656188834 >> v1:172.21.15.142:6789/0 conn(0x7fbb2815ac00 legacy=0x7fbb280ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:36.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.413+0000 7fbb26d9d640 1 -- 172.21.15.115:0/2656188834 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280b0a10 con 0x7fbb280cb630 2023-12-01T03:12:36.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.414+0000 7fbb26d9d640 1 --2- 172.21.15.115:0/2656188834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb280cb630 0x7fbb2806fcd0 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7fbb280aff50 tx=0x7fbb2002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c9afcbf942daed server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb25d9b640 1 -- 172.21.15.115:0/2656188834 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2002ca80 con 0x7fbb280cb630 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb25d9b640 1 -- 172.21.15.115:0/2656188834 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2002cbe0 con 0x7fbb280cb630 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb25d9b640 1 -- 172.21.15.115:0/2656188834 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb200343f0 con 0x7fbb280cb630 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb2759e640 1 -- 172.21.15.115:0/2656188834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb280cb630 msgr2=0x7fbb2806fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb2759e640 1 --2- 172.21.15.115:0/2656188834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb280cb630 0x7fbb2806fcd0 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7fbb280aff50 tx=0x7fbb2002bee0 comp rx=0 tx=0).stop 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb2759e640 1 -- 172.21.15.115:0/2656188834 shutdown_connections 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb2759e640 1 --2- 172.21.15.115:0/2656188834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb280cb630 0x7fbb2806fcd0 unknown :-1 s=CLOSED pgs=2957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb2759e640 1 --2- 172.21.15.115:0/2656188834 >> v2:172.21.15.142:3300/0 conn(0x7fbb280b04c0 0x7fbb280cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb2759e640 1 -- 172.21.15.115:0/2656188834 >> 172.21.15.115:0/2656188834 conn(0x7fbb28065960 msgr2=0x7fbb28065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb2759e640 1 -- 172.21.15.115:0/2656188834 shutdown_connections 2023-12-01T03:12:36.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.415+0000 7fbb2759e640 1 -- 172.21.15.115:0/2656188834 wait complete. 2023-12-01T03:12:36.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.417+0000 7fbb2759e640 1 Processor -- start 2023-12-01T03:12:36.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.417+0000 7fbb2759e640 1 -- start start 2023-12-01T03:12:36.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.418+0000 7fbb2759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb280b04c0 0x7fbb280c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:36.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.418+0000 7fbb2759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb280cb630 0x7fbb28158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:36.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.418+0000 7fbb2759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb2815ac00 0x7fbb280ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:36.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.418+0000 7fbb2759e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280aef10 con 0x7fbb2815ac00 2023-12-01T03:12:36.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.418+0000 7fbb2759e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280c3530 con 0x7fbb280cb630 2023-12-01T03:12:36.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.418+0000 7fbb2759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28158d40 con 0x7fbb280b04c0 2023-12-01T03:12:36.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.418+0000 7fbb26d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb2815ac00 0x7fbb280ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:36.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.418+0000 7fbb26d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb2815ac00 0x7fbb280ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48186/0 (socket says 172.21.15.115:48186) 2023-12-01T03:12:36.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.418+0000 7fbb26d9d640 1 -- 172.21.15.115:0/1365229394 learned_addr learned my addr 172.21.15.115:0/1365229394 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:36.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.419+0000 7fbb1ffff640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb280cb630 0x7fbb28158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.419+0000 7fbb26d9d640 1 -- 172.21.15.115:0/1365229394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb280b04c0 msgr2=0x7fbb280c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.419+0000 7fbb26d9d640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb280b04c0 0x7fbb280c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.419+0000 7fbb26d9d640 1 -- 172.21.15.115:0/1365229394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb280cb630 msgr2=0x7fbb28158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.419+0000 7fbb26d9d640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb280cb630 0x7fbb28158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.419+0000 7fbb26d9d640 1 -- 172.21.15.115:0/1365229394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb20009b30 con 0x7fbb2815ac00 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.419+0000 7fbb26d9d640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb2815ac00 0x7fbb280ac8a0 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009f40 tx=0x7fbb20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.419+0000 7fbb1effd640 1 -- 172.21.15.115:0/1365229394 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20009510 con 0x7fbb2815ac00 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.419+0000 7fbb1effd640 1 -- 172.21.15.115:0/1365229394 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2003e450 con 0x7fbb2815ac00 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.419+0000 7fbb2759e640 1 -- 172.21.15.115:0/1365229394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb280aeb00 con 0x7fbb2815ac00 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.420+0000 7fbb1effd640 1 -- 172.21.15.115:0/1365229394 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2003e800 con 0x7fbb2815ac00 2023-12-01T03:12:36.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.420+0000 7fbb2759e640 1 -- 172.21.15.115:0/1365229394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb280af160 con 0x7fbb2815ac00 2023-12-01T03:12:36.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.421+0000 7fbb2759e640 1 -- 172.21.15.115:0/1365229394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb280618d0 con 0x7fbb2815ac00 2023-12-01T03:12:36.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.422+0000 7fbb1effd640 1 -- 172.21.15.115:0/1365229394 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fbb2002cd50 con 0x7fbb2815ac00 2023-12-01T03:12:36.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.422+0000 7fbb1effd640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbb0806a670 0x7fbb0806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:36.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.423+0000 7fbb1effd640 1 -- 172.21.15.115:0/1365229394 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fbb200ab7b0 con 0x7fbb2815ac00 2023-12-01T03:12:36.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.423+0000 7fbb2659c640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbb0806a670 0x7fbb0806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:36.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.424+0000 7fbb2659c640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbb0806a670 0x7fbb0806cb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000f60 tx=0x7fbb180079b0 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:36.427 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.427+0000 7fbb1effd640 1 -- 172.21.15.115:0/1365229394 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbb20076aa0 con 0x7fbb2815ac00 2023-12-01T03:12:36.641 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:36.640+0000 7fbb2759e640 1 -- 172.21.15.115:0/1365229394 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbb280c3670 con 0x7fbb2815ac00 2023-12-01T03:12:37.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.339+0000 7fbb1effd640 1 -- 172.21.15.115:0/1365229394 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v125) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbb2007a0c0 con 0x7fbb2815ac00 2023-12-01T03:12:37.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.343+0000 7fbb2759e640 1 -- 172.21.15.115:0/1365229394 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbb0806a670 msgr2=0x7fbb0806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:37.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.343+0000 7fbb2759e640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbb0806a670 0x7fbb0806cb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000f60 tx=0x7fbb180079b0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.343+0000 7fbb2759e640 1 -- 172.21.15.115:0/1365229394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb2815ac00 msgr2=0x7fbb280ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:37.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.343+0000 7fbb2759e640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb2815ac00 0x7fbb280ac8a0 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009f40 tx=0x7fbb20004a40 comp rx=0 tx=0).stop 2023-12-01T03:12:37.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.343+0000 7fbb2759e640 1 -- 172.21.15.115:0/1365229394 shutdown_connections 2023-12-01T03:12:37.344 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.343+0000 7fbb2759e640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbb0806a670 0x7fbb0806cb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.343+0000 7fbb2759e640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbb2815ac00 0x7fbb280ac8a0 unknown :-1 s=CLOSED pgs=2958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.343+0000 7fbb2759e640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb280cb630 0x7fbb28158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.343+0000 7fbb2759e640 1 --2- 172.21.15.115:0/1365229394 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb280b04c0 0x7fbb280c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.343+0000 7fbb2759e640 1 -- 172.21.15.115:0/1365229394 >> 172.21.15.115:0/1365229394 conn(0x7fbb28065960 msgr2=0x7fbb280ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:37.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.344+0000 7fbb2759e640 1 -- 172.21.15.115:0/1365229394 shutdown_connections 2023-12-01T03:12:37.345 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.344+0000 7fbb2759e640 1 -- 172.21.15.115:0/1365229394 wait complete. 2023-12-01T03:12:37.367 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-01T03:12:37.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-01T03:12:37.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.587+0000 7efd86181640 1 Processor -- start 2023-12-01T03:12:37.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.587+0000 7efd86181640 1 -- start start 2023-12-01T03:12:37.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.588+0000 7efd86181640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd80153c30 0x7efd800ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:37.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.588+0000 7efd86181640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efd800cfb80 0x7efd80160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:37.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.588+0000 7efd86181640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd800cdcd0 con 0x7efd80154010 2023-12-01T03:12:37.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.588+0000 7efd86181640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd80161830 con 0x7efd800cfb80 2023-12-01T03:12:37.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.588+0000 7efd86181640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd800cd3a0 con 0x7efd80153c30 2023-12-01T03:12:37.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd8497e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7efd80154010 0x7efd800cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:35166/0 (socket says 172.21.15.115:35166) 2023-12-01T03:12:37.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd8497e640 1 -- 172.21.15.115:0/2780561247 learned_addr learned my addr 172.21.15.115:0/2780561247 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:37.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd85980640 1 --2- 172.21.15.115:0/2780561247 >> v2:172.21.15.142:3300/0 conn(0x7efd800cfb80 0x7efd80160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:37.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd8517f640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd80153c30 0x7efd800ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909562207 0 0) 0x7efd800cdcd0 con 0x7efd80154010 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd6c003520 con 0x7efd80154010 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4090055529 0 0) 0x7efd6c003520 con 0x7efd80154010 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd80153c30 msgr2=0x7efd800ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd77fff640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd80153c30 0x7efd800ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 >> v2:172.21.15.142:3300/0 conn(0x7efd800cfb80 msgr2=0x7efd80160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.589+0000 7efd77fff640 1 --2- 172.21.15.115:0/2780561247 >> v2:172.21.15.142:3300/0 conn(0x7efd800cfb80 0x7efd80160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800cd680 con 0x7efd80154010 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7efd70001670 con 0x7efd80154010 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 >> v1:172.21.15.115:6789/0 conn(0x7efd80154010 legacy=0x7efd800cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd77fff640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd6c003970 0x7efd6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd77fff640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd6c0047c0 0x7efd6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd77fff640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd80153c30 0x7efd6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd80161830 con 0x7efd6c003970 2023-12-01T03:12:37.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800cd3a0 con 0x7efd6c0047c0 2023-12-01T03:12:37.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c00b980 con 0x7efd80153c30 2023-12-01T03:12:37.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd8517f640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd6c003970 0x7efd6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:37.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd8497e640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd6c0047c0 0x7efd6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:37.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd8517f640 1 -- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd80153c30 msgr2=0x7efd6c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:37.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd8517f640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd80153c30 0x7efd6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd8517f640 1 -- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd6c0047c0 msgr2=0x7efd6c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:37.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd8517f640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd6c0047c0 0x7efd6c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.590+0000 7efd8517f640 1 -- 172.21.15.115:0/2780561247 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c003520 con 0x7efd6c003970 2023-12-01T03:12:37.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.591+0000 7efd8497e640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd6c0047c0 0x7efd6c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:37.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.592+0000 7efd8517f640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd6c003970 0x7efd6c003d90 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009ac0 tx=0x7efd6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c7d540cfe44d1e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:37.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.592+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c02c710 con 0x7efd6c003970 2023-12-01T03:12:37.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.592+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c02c870 con 0x7efd6c003970 2023-12-01T03:12:37.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.592+0000 7efd77fff640 1 -- 172.21.15.115:0/2780561247 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c02cc00 con 0x7efd6c003970 2023-12-01T03:12:37.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.593+0000 7efd86181640 1 -- 172.21.15.115:0/2780561247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd6c003970 msgr2=0x7efd6c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:37.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.593+0000 7efd86181640 1 --2- 172.21.15.115:0/2780561247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd6c003970 0x7efd6c003d90 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009ac0 tx=0x7efd6c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.593+0000 7efd85980640 1 -- 172.21.15.115:0/2780561247 reap_dead start 2023-12-01T03:12:37.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.593+0000 7efd86181640 1 -- 172.21.15.115:0/2780561247 shutdown_connections 2023-12-01T03:12:37.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.593+0000 7efd86181640 1 -- 172.21.15.115:0/2780561247 >> 172.21.15.115:0/2780561247 conn(0x7efd8006a0b0 msgr2=0x7efd8006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:37.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.593+0000 7efd86181640 1 -- 172.21.15.115:0/2780561247 shutdown_connections 2023-12-01T03:12:37.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.593+0000 7efd86181640 1 -- 172.21.15.115:0/2780561247 wait complete. 2023-12-01T03:12:37.595 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.595+0000 7efd86181640 1 Processor -- start 2023-12-01T03:12:37.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.595+0000 7efd86181640 1 -- start start 2023-12-01T03:12:37.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd86181640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd6c003970 0x7efd800cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:37.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd86181640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd800cc7a0 0x7efd800d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:37.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd86181640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd800d0c40 0x7efd800d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:37.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd86181640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800d1600 con 0x7efd800d0c40 2023-12-01T03:12:37.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd86181640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800cbf80 con 0x7efd6c003970 2023-12-01T03:12:37.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd86181640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd800c87a0 con 0x7efd800cc7a0 2023-12-01T03:12:37.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd85980640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd800d0c40 0x7efd800d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:37.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd85980640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd800d0c40 0x7efd800d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:48192/0 (socket says 172.21.15.115:48192) 2023-12-01T03:12:37.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd85980640 1 -- 172.21.15.115:0/60167648 learned_addr learned my addr 172.21.15.115:0/60167648 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:37.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd8497e640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd800cc7a0 0x7efd800d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:37.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd85980640 1 -- 172.21.15.115:0/60167648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd800cc7a0 msgr2=0x7efd800d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:37.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd85980640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd800cc7a0 0x7efd800d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd8517f640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd6c003970 0x7efd800cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:37.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd85980640 1 -- 172.21.15.115:0/60167648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd6c003970 msgr2=0x7efd800cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:37.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd85980640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd6c003970 0x7efd800cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:37.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.596+0000 7efd85980640 1 -- 172.21.15.115:0/60167648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800c8a80 con 0x7efd800d0c40 2023-12-01T03:12:37.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.597+0000 7efd85980640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd800d0c40 0x7efd800d10c0 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7efd800d04a0 tx=0x7efd7800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:37.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.597+0000 7efd75ffb640 1 -- 172.21.15.115:0/60167648 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78010040 con 0x7efd800d0c40 2023-12-01T03:12:37.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.597+0000 7efd75ffb640 1 -- 172.21.15.115:0/60167648 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7800bae0 con 0x7efd800d0c40 2023-12-01T03:12:37.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.597+0000 7efd75ffb640 1 -- 172.21.15.115:0/60167648 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7800be70 con 0x7efd800d0c40 2023-12-01T03:12:37.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.597+0000 7efd86181640 1 -- 172.21.15.115:0/60167648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd800c8d60 con 0x7efd800d0c40 2023-12-01T03:12:37.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.597+0000 7efd86181640 1 -- 172.21.15.115:0/60167648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd800c9320 con 0x7efd800d0c40 2023-12-01T03:12:37.599 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.598+0000 7efd86181640 1 -- 172.21.15.115:0/60167648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd800618d0 con 0x7efd800d0c40 2023-12-01T03:12:37.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.599+0000 7efd75ffb640 1 -- 172.21.15.115:0/60167648 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7efd7801d400 con 0x7efd800d0c40 2023-12-01T03:12:37.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.600+0000 7efd75ffb640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a670 0x7efd6406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:37.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.600+0000 7efd75ffb640 1 -- 172.21.15.115:0/60167648 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7efd78089490 con 0x7efd800d0c40 2023-12-01T03:12:37.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.601+0000 7efd8517f640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a670 0x7efd6406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:37.602 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.601+0000 7efd8517f640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a670 0x7efd6406cb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd7c036000 tx=0x7efd7c030000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:37.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.605+0000 7efd75ffb640 1 -- 172.21.15.115:0/60167648 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efd780545d0 con 0x7efd800d0c40 2023-12-01T03:12:37.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:37.827+0000 7efd86181640 1 -- 172.21.15.115:0/60167648 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efd8006a5f0 con 0x7efd800d0c40 2023-12-01T03:12:38.484 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.482+0000 7efd75ffb640 1 -- 172.21.15.115:0/60167648 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v126) v1 ==== 122+0+0 (secure 0 0 0) 0x7efd78057bf0 con 0x7efd800d0c40 2023-12-01T03:12:38.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.485+0000 7efd86181640 1 -- 172.21.15.115:0/60167648 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a670 msgr2=0x7efd6406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:38.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.485+0000 7efd86181640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a670 0x7efd6406cb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd7c036000 tx=0x7efd7c030000 comp rx=0 tx=0).stop 2023-12-01T03:12:38.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.485+0000 7efd86181640 1 -- 172.21.15.115:0/60167648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd800d0c40 msgr2=0x7efd800d10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:38.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.485+0000 7efd86181640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd800d0c40 0x7efd800d10c0 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7efd800d04a0 tx=0x7efd7800b480 comp rx=0 tx=0).stop 2023-12-01T03:12:38.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.485+0000 7efd86181640 1 -- 172.21.15.115:0/60167648 shutdown_connections 2023-12-01T03:12:38.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.485+0000 7efd86181640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efd6406a670 0x7efd6406cb30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.485+0000 7efd86181640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd800cc7a0 0x7efd800d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.485+0000 7efd86181640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd800d0c40 0x7efd800d10c0 unknown :-1 s=CLOSED pgs=2961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.485+0000 7efd86181640 1 --2- 172.21.15.115:0/60167648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd6c003970 0x7efd800cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.485+0000 7efd86181640 1 -- 172.21.15.115:0/60167648 >> 172.21.15.115:0/60167648 conn(0x7efd8006a0b0 msgr2=0x7efd8015d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:38.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.486+0000 7efd86181640 1 -- 172.21.15.115:0/60167648 shutdown_connections 2023-12-01T03:12:38.488 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.486+0000 7efd86181640 1 -- 172.21.15.115:0/60167648 wait complete. 2023-12-01T03:12:38.509 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-01T03:12:38.509 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:12:38.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.713+0000 7fbdb68af640 1 Processor -- start 2023-12-01T03:12:38.715 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.713+0000 7fbdb68af640 1 -- start start 2023-12-01T03:12:38.716 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.714+0000 7fbdb68af640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbdb015e2f0 0x7fbdb00ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:38.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.715+0000 7fbdb68af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb00adf10 0x7fbdb00caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:38.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.715+0000 7fbdb68af640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb00add10 con 0x7fbdb006f2f0 2023-12-01T03:12:38.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.715+0000 7fbdb68af640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb00c5760 con 0x7fbdb015e2f0 2023-12-01T03:12:38.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.715+0000 7fbdb68af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb015e6d0 con 0x7fbdb00adf10 2023-12-01T03:12:38.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.715+0000 7fbdaf7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fbdb006f2f0 0x7fbdb00c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37704/0 (socket says 172.21.15.115:37704) 2023-12-01T03:12:38.717 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.715+0000 7fbdaf7fe640 1 -- 172.21.15.115:0/1113916966 learned_addr learned my addr 172.21.15.115:0/1113916966 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.715+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126410164 0 0) 0x7fbdb00add10 con 0x7fbdb006f2f0 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.715+0000 7fbdb4e4c640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb00adf10 0x7fbdb00caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.715+0000 7fbdaffff640 1 --2- 172.21.15.115:0/1113916966 >> v2:172.21.15.142:3300/0 conn(0x7fbdb015e2f0 0x7fbdb00ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.715+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd9c003520 con 0x7fbdb006f2f0 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1980890165 0 0) 0x7fbd9c003520 con 0x7fbdb006f2f0 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb00adf10 msgr2=0x7fbdb00caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb00adf10 0x7fbdb00caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 >> v2:172.21.15.142:3300/0 conn(0x7fbdb015e2f0 msgr2=0x7fbdb00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 --2- 172.21.15.115:0/1113916966 >> v2:172.21.15.142:3300/0 conn(0x7fbdb015e2f0 0x7fbdb00ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb00cf7b0 con 0x7fbdb006f2f0 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fbd98001a10 con 0x7fbdb006f2f0 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 >> v1:172.21.15.115:6789/0 conn(0x7fbdb006f2f0 legacy=0x7fbdb00c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb00adf10 0x7fbd9c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:38.719 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd9c004060 0x7fbd9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd9c004d00 0x7fbd9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00c5760 con 0x7fbd9c004d00 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb015e6d0 con 0x7fbd9c004060 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0100f0 con 0x7fbdb00adf10 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdb4e4c640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd9c004d00 0x7fbd9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaf7fe640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd9c004060 0x7fbd9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdaffff640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb00adf10 0x7fbd9c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdb4e4c640 1 -- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb00adf10 msgr2=0x7fbd9c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.716+0000 7fbdb4e4c640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb00adf10 0x7fbd9c0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.717+0000 7fbdb4e4c640 1 -- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd9c004060 msgr2=0x7fbd9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.717+0000 7fbdb4e4c640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd9c004060 0x7fbd9c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.717+0000 7fbdb4e4c640 1 -- 172.21.15.115:0/1113916966 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c003520 con 0x7fbd9c004d00 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.718+0000 7fbdb4e4c640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd9c004d00 0x7fbd9c005180 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7fbda800ba30 tx=0x7fbd9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea8002bf4a95110 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.718+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8032640 con 0x7fbd9c004d00 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.718+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda80327a0 con 0x7fbd9c004d00 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.718+0000 7fbdaeffd640 1 -- 172.21.15.115:0/1113916966 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8032b30 con 0x7fbd9c004d00 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.718+0000 7fbdb68af640 1 -- 172.21.15.115:0/1113916966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd9c004d00 msgr2=0x7fbd9c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.718+0000 7fbdb68af640 1 --2- 172.21.15.115:0/1113916966 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd9c004d00 0x7fbd9c005180 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7fbda800ba30 tx=0x7fbd9c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.718+0000 7fbdb4e4c640 1 -- 172.21.15.115:0/1113916966 reap_dead start 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.718+0000 7fbdb68af640 1 -- 172.21.15.115:0/1113916966 shutdown_connections 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.718+0000 7fbdb68af640 1 -- 172.21.15.115:0/1113916966 >> 172.21.15.115:0/1113916966 conn(0x7fbdb00ce880 msgr2=0x7fbdb00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.719+0000 7fbdb68af640 1 -- 172.21.15.115:0/1113916966 shutdown_connections 2023-12-01T03:12:38.722 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.719+0000 7fbdb68af640 1 -- 172.21.15.115:0/1113916966 wait complete. 2023-12-01T03:12:38.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.720+0000 7fbdb68af640 1 Processor -- start 2023-12-01T03:12:38.723 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.720+0000 7fbdb68af640 1 -- start start 2023-12-01T03:12:38.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.721+0000 7fbdb68af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd9c004d00 0x7fbdb00c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:38.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.721+0000 7fbdb68af640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdb00c23b0 0x7fbdb0158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:38.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.721+0000 7fbdb68af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb006b4c0 0x7fbdb006b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.721+0000 7fbdb68af640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00c59c0 con 0x7fbdb00c23b0 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.721+0000 7fbdb68af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb006be80 con 0x7fbd9c004d00 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.721+0000 7fbdb68af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb00c4640 con 0x7fbdb006b4c0 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.721+0000 7fbdaf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdb00c23b0 0x7fbdb0158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdaf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdb00c23b0 0x7fbdb0158300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51736/0 (socket says 172.21.15.115:51736) 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdaf7fe640 1 -- 172.21.15.115:0/2657164307 learned_addr learned my addr 172.21.15.115:0/2657164307 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdaf7fe640 1 -- 172.21.15.115:0/2657164307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb006b4c0 msgr2=0x7fbdb006b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdaf7fe640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb006b4c0 0x7fbdb006b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdaf7fe640 1 -- 172.21.15.115:0/2657164307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd9c004d00 msgr2=0x7fbdb00c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdaf7fe640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd9c004d00 0x7fbdb00c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdaf7fe640 1 -- 172.21.15.115:0/2657164307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb00c2970 con 0x7fbdb00c23b0 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdaf7fe640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdb00c23b0 0x7fbdb0158300 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7fbd98007a10 tx=0x7fbd9800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdacff9640 1 -- 172.21.15.115:0/2657164307 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98007b00 con 0x7fbdb00c23b0 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdacff9640 1 -- 172.21.15.115:0/2657164307 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9800edd0 con 0x7fbdb00c23b0 2023-12-01T03:12:38.725 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdacff9640 1 -- 172.21.15.115:0/2657164307 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98013660 con 0x7fbdb00c23b0 2023-12-01T03:12:38.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.722+0000 7fbdb68af640 1 -- 172.21.15.115:0/2657164307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb00c2c50 con 0x7fbdb00c23b0 2023-12-01T03:12:38.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.723+0000 7fbdb68af640 1 -- 172.21.15.115:0/2657164307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb00c3210 con 0x7fbdb00c23b0 2023-12-01T03:12:38.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.724+0000 7fbdb68af640 1 -- 172.21.15.115:0/2657164307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb006fda0 con 0x7fbdb00c23b0 2023-12-01T03:12:38.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.724+0000 7fbdacff9640 1 -- 172.21.15.115:0/2657164307 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fbd980137c0 con 0x7fbdb00c23b0 2023-12-01T03:12:38.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.725+0000 7fbdacff9640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbd9006a6a0 0x7fbd9006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:38.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.725+0000 7fbdacff9640 1 -- 172.21.15.115:0/2657164307 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fbd980892f0 con 0x7fbdb00c23b0 2023-12-01T03:12:38.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.725+0000 7fbdaffff640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbd9006a6a0 0x7fbd9006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:38.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.726+0000 7fbdaffff640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbd9006a6a0 0x7fbd9006cb60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbda0006e80 tx=0x7fbda0006e10 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:38.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.730+0000 7fbdacff9640 1 -- 172.21.15.115:0/2657164307 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbd98054610 con 0x7fbdb00c23b0 2023-12-01T03:12:38.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.938+0000 7fbdb68af640 1 -- 172.21.15.115:0/2657164307 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdb004e890 con 0x7fbdb00c23b0 2023-12-01T03:12:38.940 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.939+0000 7fbdacff9640 1 -- 172.21.15.115:0/2657164307 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4672 (secure 0 0 0) 0x7fbd98057c30 con 0x7fbdb00c23b0 2023-12-01T03:12:38.941 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:38.941 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":126,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2042114371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2042114371},{"type":"v1","addr":"172.21.15.142:6835","nonce":2042114371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/727621046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":727621046},{"type":"v1","addr":"172.21.15.115:6835","nonce":727621046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3018042052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3018042052},{"type":"v1","addr":"172.21.15.115:6837","nonce":3018042052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:12:35.317383+0000","modified":"2023-12-01T03:12:38.475270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12246":{"gid":12246,"name":"b","rank":0,"incarnation":122,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/1356734431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1356734431},{"type":"v1","addr":"172.21.15.142:6834","nonce":1356734431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-01T03:12:38.941 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 126 2023-12-01T03:12:38.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.941+0000 7fbdb68af640 1 -- 172.21.15.115:0/2657164307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbd9006a6a0 msgr2=0x7fbd9006cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:38.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.941+0000 7fbdb68af640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbd9006a6a0 0x7fbd9006cb60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbda0006e80 tx=0x7fbda0006e10 comp rx=0 tx=0).stop 2023-12-01T03:12:38.943 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.941+0000 7fbdb68af640 1 -- 172.21.15.115:0/2657164307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdb00c23b0 msgr2=0x7fbdb0158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:38.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.941+0000 7fbdb68af640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdb00c23b0 0x7fbdb0158300 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7fbd98007a10 tx=0x7fbd9800e530 comp rx=0 tx=0).stop 2023-12-01T03:12:38.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.941+0000 7fbdb68af640 1 -- 172.21.15.115:0/2657164307 shutdown_connections 2023-12-01T03:12:38.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.941+0000 7fbdb68af640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbd9006a6a0 0x7fbd9006cb60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.941+0000 7fbdb68af640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbdb006b4c0 0x7fbdb006b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.941+0000 7fbdb68af640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbdb00c23b0 0x7fbdb0158300 unknown :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.941+0000 7fbdb68af640 1 --2- 172.21.15.115:0/2657164307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd9c004d00 0x7fbdb00c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:38.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.941+0000 7fbdb68af640 1 -- 172.21.15.115:0/2657164307 >> 172.21.15.115:0/2657164307 conn(0x7fbdb00ce880 msgr2=0x7fbdb00cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:38.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.942+0000 7fbdb68af640 1 -- 172.21.15.115:0/2657164307 shutdown_connections 2023-12-01T03:12:38.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:38.942+0000 7fbdb68af640 1 -- 172.21.15.115:0/2657164307 wait complete. 2023-12-01T03:12:38.961 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:39.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.162+0000 7eff23038640 1 Processor -- start 2023-12-01T03:12:39.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.162+0000 7eff23038640 1 -- start start 2023-12-01T03:12:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.164+0000 7eff23038640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7eff1c0adac0 0x7eff1c0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.164+0000 7eff23038640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7eff1c0c8ed0 0x7eff1c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.164+0000 7eff23038640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1c0ce150 con 0x7eff1c0cced0 2023-12-01T03:12:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.164+0000 7eff23038640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1c0cd320 con 0x7eff1c0c8ed0 2023-12-01T03:12:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.164+0000 7eff23038640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1c0cf3a0 con 0x7eff1c0adac0 2023-12-01T03:12:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.164+0000 7eff1bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7eff1c0c8ed0 0x7eff1c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.164+0000 7eff215d5640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7eff1c0cced0 0x7eff1c161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37712/0 (socket says 172.21.15.115:37712) 2023-12-01T03:12:39.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.164+0000 7eff215d5640 1 -- 172.21.15.115:0/592113780 learned_addr learned my addr 172.21.15.115:0/592113780 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.164+0000 7eff20dd4640 1 --2- 172.21.15.115:0/592113780 >> v2:172.21.15.142:3300/0 conn(0x7eff1c0adac0 0x7eff1c0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.165+0000 7eff1bfff640 1 -- 172.21.15.115:0/592113780 >> v2:172.21.15.142:3300/0 conn(0x7eff1c0adac0 msgr2=0x7eff1c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.165+0000 7eff1bfff640 1 --2- 172.21.15.115:0/592113780 >> v2:172.21.15.142:3300/0 conn(0x7eff1c0adac0 0x7eff1c0cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.165+0000 7eff1bfff640 1 -- 172.21.15.115:0/592113780 >> v1:172.21.15.115:6789/0 conn(0x7eff1c0cced0 legacy=0x7eff1c161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.165+0000 7eff1bfff640 1 -- 172.21.15.115:0/592113780 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c06d620 con 0x7eff1c0c8ed0 2023-12-01T03:12:39.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.165+0000 7eff1b7fe640 1 -- 172.21.15.115:0/592113780 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827540892 0 0) 0x7eff1c0ce150 con 0x7eff1c0cced0 2023-12-01T03:12:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.166+0000 7eff1bfff640 1 --2- 172.21.15.115:0/592113780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0c8ed0 0x7eff1c0cc990 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0cb260 tx=0x7eff0802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d0f1cf9025cabb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.166+0000 7eff1b7fe640 1 -- 172.21.15.115:0/592113780 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08031830 con 0x7eff1c0c8ed0 2023-12-01T03:12:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.166+0000 7eff1b7fe640 1 -- 172.21.15.115:0/592113780 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff08031990 con 0x7eff1c0c8ed0 2023-12-01T03:12:39.168 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.166+0000 7eff1b7fe640 1 -- 172.21.15.115:0/592113780 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08031d20 con 0x7eff1c0c8ed0 2023-12-01T03:12:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.167+0000 7eff23038640 1 -- 172.21.15.115:0/592113780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0c8ed0 msgr2=0x7eff1c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.167+0000 7eff23038640 1 --2- 172.21.15.115:0/592113780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0c8ed0 0x7eff1c0cc990 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0cb260 tx=0x7eff0802b8d0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.167+0000 7eff23038640 1 -- 172.21.15.115:0/592113780 shutdown_connections 2023-12-01T03:12:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.167+0000 7eff23038640 1 --2- 172.21.15.115:0/592113780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0c8ed0 0x7eff1c0cc990 unknown :-1 s=CLOSED pgs=2966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.167+0000 7eff23038640 1 --2- 172.21.15.115:0/592113780 >> v2:172.21.15.142:3300/0 conn(0x7eff1c0adac0 0x7eff1c0cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.167+0000 7eff23038640 1 -- 172.21.15.115:0/592113780 >> 172.21.15.115:0/592113780 conn(0x7eff1c0ce880 msgr2=0x7eff1c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.167+0000 7eff23038640 1 -- 172.21.15.115:0/592113780 shutdown_connections 2023-12-01T03:12:39.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.167+0000 7eff23038640 1 -- 172.21.15.115:0/592113780 wait complete. 2023-12-01T03:12:39.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.169+0000 7eff23038640 1 Processor -- start 2023-12-01T03:12:39.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.169+0000 7eff23038640 1 -- start start 2023-12-01T03:12:39.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff23038640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff1c0adac0 0x7eff1c0c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff23038640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c0c8ed0 0x7eff1c0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff23038640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0cced0 0x7eff1c0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff23038640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c0c8570 con 0x7eff1c0cced0 2023-12-01T03:12:39.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff23038640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c0c7fb0 con 0x7eff1c0c8ed0 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff23038640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c0c8290 con 0x7eff1c0adac0 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff215d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0cced0 0x7eff1c0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff215d5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0cced0 0x7eff1c0c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51754/0 (socket says 172.21.15.115:51754) 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff215d5640 1 -- 172.21.15.115:0/4250701411 learned_addr learned my addr 172.21.15.115:0/4250701411 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff20dd4640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff1c0adac0 0x7eff1c0c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff215d5640 1 -- 172.21.15.115:0/4250701411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff1c0adac0 msgr2=0x7eff1c0c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff215d5640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff1c0adac0 0x7eff1c0c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff215d5640 1 -- 172.21.15.115:0/4250701411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c0c8ed0 msgr2=0x7eff1c0c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff215d5640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c0c8ed0 0x7eff1c0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff215d5640 1 -- 172.21.15.115:0/4250701411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff08009400 con 0x7eff1c0cced0 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.170+0000 7eff20dd4640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff1c0adac0 0x7eff1c0c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:39.173 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.171+0000 7eff215d5640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0cced0 0x7eff1c0c3400 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7eff1400c7e0 tx=0x7eff1400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:39.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.171+0000 7eff197fa640 1 -- 172.21.15.115:0/4250701411 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400a640 con 0x7eff1c0cced0 2023-12-01T03:12:39.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.171+0000 7eff197fa640 1 -- 172.21.15.115:0/4250701411 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1400a7c0 con 0x7eff1c0cced0 2023-12-01T03:12:39.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.171+0000 7eff23038640 1 -- 172.21.15.115:0/4250701411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c0b1530 con 0x7eff1c0cced0 2023-12-01T03:12:39.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.171+0000 7eff197fa640 1 -- 172.21.15.115:0/4250701411 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400ab70 con 0x7eff1c0cced0 2023-12-01T03:12:39.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.171+0000 7eff23038640 1 -- 172.21.15.115:0/4250701411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c0c7a20 con 0x7eff1c0cced0 2023-12-01T03:12:39.174 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.172+0000 7eff23038640 1 -- 172.21.15.115:0/4250701411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c0b1a50 con 0x7eff1c0cced0 2023-12-01T03:12:39.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.173+0000 7eff197fa640 1 -- 172.21.15.115:0/4250701411 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7eff1400acd0 con 0x7eff1c0cced0 2023-12-01T03:12:39.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.174+0000 7eff197fa640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7eff0006a670 0x7eff0006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.175+0000 7eff197fa640 1 -- 172.21.15.115:0/4250701411 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7eff14088640 con 0x7eff1c0cced0 2023-12-01T03:12:39.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.175+0000 7eff20dd4640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7eff0006a670 0x7eff0006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.175+0000 7eff20dd4640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7eff0006a670 0x7eff0006cb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff10006fd0 tx=0x7eff10008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:39.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.178+0000 7eff197fa640 1 -- 172.21.15.115:0/4250701411 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7eff1c0b1a50 con 0x7eff1c0cced0 2023-12-01T03:12:39.370 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.368+0000 7eff23038640 1 -- 172.21.15.115:0/4250701411 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff1c0b1a50 con 0x7eff1c0cced0 2023-12-01T03:12:39.372 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.370+0000 7eff197fa640 1 -- 172.21.15.115:0/4250701411 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v256) v1 ==== 74+0+16173 (secure 0 0 0) 0x7eff1c0b1a50 con 0x7eff1c0cced0 2023-12-01T03:12:39.372 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:39.372 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":256,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:36.322308+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:12:33.411870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-01T03:12:34.335941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.372+0000 7eff23038640 1 -- 172.21.15.115:0/4250701411 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7eff0006a670 msgr2=0x7eff0006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.372+0000 7eff23038640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7eff0006a670 0x7eff0006cb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff10006fd0 tx=0x7eff10008040 comp rx=0 tx=0).stop 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.372+0000 7eff23038640 1 -- 172.21.15.115:0/4250701411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0cced0 msgr2=0x7eff1c0c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.372+0000 7eff23038640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0cced0 0x7eff1c0c3400 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7eff1400c7e0 tx=0x7eff1400cca0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.373+0000 7eff23038640 1 -- 172.21.15.115:0/4250701411 shutdown_connections 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.373+0000 7eff23038640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7eff0006a670 0x7eff0006cb30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.373+0000 7eff23038640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff1c0cced0 0x7eff1c0c3400 unknown :-1 s=CLOSED pgs=2967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.373+0000 7eff23038640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff1c0c8ed0 0x7eff1c0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.373+0000 7eff23038640 1 --2- 172.21.15.115:0/4250701411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff1c0adac0 0x7eff1c0c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.373+0000 7eff23038640 1 -- 172.21.15.115:0/4250701411 >> 172.21.15.115:0/4250701411 conn(0x7eff1c0ce880 msgr2=0x7eff1c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.373+0000 7eff23038640 1 -- 172.21.15.115:0/4250701411 shutdown_connections 2023-12-01T03:12:39.375 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.373+0000 7eff23038640 1 -- 172.21.15.115:0/4250701411 wait complete. 2023-12-01T03:12:39.393 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T03:12:39.394 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:12:39.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.603+0000 7fedc52b9640 1 Processor -- start 2023-12-01T03:12:39.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.603+0000 7fedc52b9640 1 -- start start 2023-12-01T03:12:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.604+0000 7fedc52b9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc015e2f0 0x7fedc00ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.604+0000 7fedc52b9640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fedc006f2f0 0x7fedc00caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedc52b9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc00add10 con 0x7fedc00adf10 2023-12-01T03:12:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedc52b9640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc00c5760 con 0x7fedc006f2f0 2023-12-01T03:12:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedc52b9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc015e6d0 con 0x7fedc015e2f0 2023-12-01T03:12:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedbf7fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fedc00adf10 0x7fedc00c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37726/0 (socket says 172.21.15.115:37726) 2023-12-01T03:12:39.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedbf7fe640 1 -- 172.21.15.115:0/266342066 learned_addr learned my addr 172.21.15.115:0/266342066 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:39.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedbe7fc640 1 --2- 172.21.15.115:0/266342066 >> v2:172.21.15.115:3300/0 conn(0x7fedc006f2f0 0x7fedc00caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedbeffd640 1 --2- 172.21.15.115:0/266342066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc015e2f0 0x7fedc00ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedbdffb640 1 -- 172.21.15.115:0/266342066 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278259385 0 0) 0x7fedc00add10 con 0x7fedc00adf10 2023-12-01T03:12:39.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedbe7fc640 1 -- 172.21.15.115:0/266342066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc015e2f0 msgr2=0x7fedc00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:39.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedbe7fc640 1 --2- 172.21.15.115:0/266342066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc015e2f0 0x7fedc00ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedbe7fc640 1 -- 172.21.15.115:0/266342066 >> v1:172.21.15.115:6789/0 conn(0x7fedc00adf10 legacy=0x7fedc00c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.605+0000 7fedbe7fc640 1 -- 172.21.15.115:0/266342066 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00cf7b0 con 0x7fedc006f2f0 2023-12-01T03:12:39.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.606+0000 7fedbe7fc640 1 --2- 172.21.15.115:0/266342066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc006f2f0 0x7fedc00caba0 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009d90 tx=0x7fedb002bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87df9bf73374d150 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:39.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.607+0000 7fedbdffb640 1 -- 172.21.15.115:0/266342066 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb002cb30 con 0x7fedc006f2f0 2023-12-01T03:12:39.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.607+0000 7fedbdffb640 1 -- 172.21.15.115:0/266342066 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb002cc90 con 0x7fedc006f2f0 2023-12-01T03:12:39.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.607+0000 7fedbdffb640 1 -- 172.21.15.115:0/266342066 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0034440 con 0x7fedc006f2f0 2023-12-01T03:12:39.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.607+0000 7fedc52b9640 1 -- 172.21.15.115:0/266342066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc006f2f0 msgr2=0x7fedc00caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.607+0000 7fedc52b9640 1 --2- 172.21.15.115:0/266342066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc006f2f0 0x7fedc00caba0 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009d90 tx=0x7fedb002bd80 comp rx=0 tx=0).stop 2023-12-01T03:12:39.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.607+0000 7fedc52b9640 1 -- 172.21.15.115:0/266342066 shutdown_connections 2023-12-01T03:12:39.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.607+0000 7fedc52b9640 1 --2- 172.21.15.115:0/266342066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc006f2f0 0x7fedc00caba0 unknown :-1 s=CLOSED pgs=2969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.607+0000 7fedc52b9640 1 --2- 172.21.15.115:0/266342066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc015e2f0 0x7fedc00ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.607+0000 7fedc52b9640 1 -- 172.21.15.115:0/266342066 >> 172.21.15.115:0/266342066 conn(0x7fedc00ce880 msgr2=0x7fedc00cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:39.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.608+0000 7fedc52b9640 1 -- 172.21.15.115:0/266342066 shutdown_connections 2023-12-01T03:12:39.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.608+0000 7fedc52b9640 1 -- 172.21.15.115:0/266342066 wait complete. 2023-12-01T03:12:39.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.609+0000 7fedc52b9640 1 Processor -- start 2023-12-01T03:12:39.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.609+0000 7fedc52b9640 1 -- start start 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.610+0000 7fedc52b9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc006f2f0 0x7fedc00b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.610+0000 7fedc52b9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc00adf10 0x7fedc00c4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.610+0000 7fedc52b9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc015e2f0 0x7fedc014e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.610+0000 7fedc52b9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00b3390 con 0x7fedc015e2f0 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.610+0000 7fedc52b9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00b2780 con 0x7fedc006f2f0 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.610+0000 7fedc52b9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00b2a60 con 0x7fedc00adf10 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc015e2f0 0x7fedc014e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc015e2f0 0x7fedc014e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51784/0 (socket says 172.21.15.115:51784) 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbf7fe640 1 -- 172.21.15.115:0/558219000 learned_addr learned my addr 172.21.15.115:0/558219000 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbeffd640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc006f2f0 0x7fedc00b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbe7fc640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc00adf10 0x7fedc00c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbf7fe640 1 -- 172.21.15.115:0/558219000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc00adf10 msgr2=0x7fedc00c4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbf7fe640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc00adf10 0x7fedc00c4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbf7fe640 1 -- 172.21.15.115:0/558219000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc006f2f0 msgr2=0x7fedc00b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbf7fe640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc006f2f0 0x7fedc00b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbf7fe640 1 -- 172.21.15.115:0/558219000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb0009a70 con 0x7fedc015e2f0 2023-12-01T03:12:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedbf7fe640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc015e2f0 0x7fedc014e980 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c410 tx=0x7fedb800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:39.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.611+0000 7fedb77fe640 1 -- 172.21.15.115:0/558219000 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb80074e0 con 0x7fedc015e2f0 2023-12-01T03:12:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.612+0000 7fedb77fe640 1 -- 172.21.15.115:0/558219000 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8007640 con 0x7fedc015e2f0 2023-12-01T03:12:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.612+0000 7fedb77fe640 1 -- 172.21.15.115:0/558219000 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8004a20 con 0x7fedc015e2f0 2023-12-01T03:12:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.612+0000 7fedc52b9640 1 -- 172.21.15.115:0/558219000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc00b2d40 con 0x7fedc015e2f0 2023-12-01T03:12:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.612+0000 7fedc52b9640 1 -- 172.21.15.115:0/558219000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc00b2470 con 0x7fedc015e2f0 2023-12-01T03:12:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.613+0000 7fedc52b9640 1 -- 172.21.15.115:0/558219000 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc006fda0 con 0x7fedc015e2f0 2023-12-01T03:12:39.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.614+0000 7fedb77fe640 1 -- 172.21.15.115:0/558219000 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fedb8004b80 con 0x7fedc015e2f0 2023-12-01T03:12:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.614+0000 7fedb77fe640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feda006a670 0x7feda006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.615+0000 7fedb77fe640 1 -- 172.21.15.115:0/558219000 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fedb8088b70 con 0x7fedc015e2f0 2023-12-01T03:12:39.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.615+0000 7fedbeffd640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feda006a670 0x7feda006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:39.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.615+0000 7fedbeffd640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feda006a670 0x7feda006cb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feda8006fd0 tx=0x7feda8008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:39.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.619+0000 7fedb77fe640 1 -- 172.21.15.115:0/558219000 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fedb8053e80 con 0x7fedc015e2f0 2023-12-01T03:12:39.809 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.807+0000 7fedc52b9640 1 -- 172.21.15.115:0/558219000 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fedc00b43d0 con 0x7feda006a670 2023-12-01T03:12:39.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.822+0000 7fedb77fe640 1 -- 172.21.15.115:0/558219000 <== mgr.11877 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284271 (secure 0 0 0) 0x7feda800e4c0 con 0x7feda006a670 2023-12-01T03:12:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:39.830 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:12:39.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 -- 172.21.15.115:0/558219000 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feda006a670 msgr2=0x7feda006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feda006a670 0x7feda006cb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feda8006fd0 tx=0x7feda8008040 comp rx=0 tx=0).stop 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 -- 172.21.15.115:0/558219000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc015e2f0 msgr2=0x7fedc014e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc015e2f0 0x7fedc014e980 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c410 tx=0x7fedb800c8d0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 -- 172.21.15.115:0/558219000 shutdown_connections 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feda006a670 0x7feda006cb30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedc00adf10 0x7fedc00c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fedc015e2f0 0x7fedc014e980 unknown :-1 s=CLOSED pgs=2970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 --2- 172.21.15.115:0/558219000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedc006f2f0 0x7fedc00b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 -- 172.21.15.115:0/558219000 >> 172.21.15.115:0/558219000 conn(0x7fedc00ce880 msgr2=0x7fedc00cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 -- 172.21.15.115:0/558219000 shutdown_connections 2023-12-01T03:12:39.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:39.829+0000 7fedc52b9640 1 -- 172.21.15.115:0/558219000 wait complete. 2023-12-01T03:12:39.849 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-12-01T03:12:39.049245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":443456,"kb_used_data":26112,"kb_used_omap":0,"kb_used_meta":417280,"kb_avail":754531264,"statfs":{"total":773094113280,"available":772640014336,"internally_reserved":0,"allocated":26738688,"data_stored":13761142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":427294720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.236540"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851842+0000","last_change":"2023-12-01T03:12:34.851842+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851842+0000","last_undegraded":"2023-12-01T03:12:34.851842+0000","last_fullsized":"2023-12-01T03:12:34.851842+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:21:23.692942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849451+0000","last_change":"2023-12-01T03:12:34.849451+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849451+0000","last_undegraded":"2023-12-01T03:12:34.849451+0000","last_fullsized":"2023-12-01T03:12:34.849451+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845514+0000","last_change":"2023-12-01T03:12:34.845514+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845514+0000","last_undegraded":"2023-12-01T03:12:34.845514+0000","last_fullsized":"2023-12-01T03:12:34.845514+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850690+0000","last_change":"2023-12-01T03:12:34.850690+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850690+0000","last_undegraded":"2023-12-01T03:12:34.850690+0000","last_fullsized":"2023-12-01T03:12:34.850690+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:49.741119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850902+0000","last_change":"2023-12-01T03:12:34.850902+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850902+0000","last_undegraded":"2023-12-01T03:12:34.850902+0000","last_fullsized":"2023-12-01T03:12:34.850902+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.848923+0000","last_change":"2023-12-01T03:12:34.848923+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.848923+0000","last_undegraded":"2023-12-01T03:12:34.848923+0000","last_fullsized":"2023-12-01T03:12:34.848923+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:04.481100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.855117+0000","last_change":"2023-12-01T03:12:34.855117+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.855117+0000","last_undegraded":"2023-12-01T03:12:34.855117+0000","last_fullsized":"2023-12-01T03:12:34.855117+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846440+0000","last_change":"2023-12-01T03:12:34.846440+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846440+0000","last_undegraded":"2023-12-01T03:12:34.846440+0000","last_fullsized":"2023-12-01T03:12:34.846440+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:27.055115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.855652+0000","last_change":"2023-12-01T03:12:34.855652+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.855652+0000","last_undegraded":"2023-12-01T03:12:34.855652+0000","last_fullsized":"2023-12-01T03:12:34.855652+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:17:30.864087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.848420+0000","last_change":"2023-12-01T03:12:34.848420+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.848420+0000","last_undegraded":"2023-12-01T03:12:34.848420+0000","last_fullsized":"2023-12-01T03:12:34.848420+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852653+0000","last_change":"2023-12-01T03:12:34.852653+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852653+0000","last_undegraded":"2023-12-01T03:12:34.852653+0000","last_fullsized":"2023-12-01T03:12:34.852653+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845433+0000","last_change":"2023-12-01T03:12:34.845433+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845433+0000","last_undegraded":"2023-12-01T03:12:34.845433+0000","last_fullsized":"2023-12-01T03:12:34.845433+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852346+0000","last_change":"2023-12-01T03:12:34.852346+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852346+0000","last_undegraded":"2023-12-01T03:12:34.852346+0000","last_fullsized":"2023-12-01T03:12:34.852346+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:54:17.936247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854378+0000","last_change":"2023-12-01T03:12:34.854378+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854378+0000","last_undegraded":"2023-12-01T03:12:34.854378+0000","last_fullsized":"2023-12-01T03:12:34.854378+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:59:25.105386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854306+0000","last_change":"2023-12-01T03:12:34.854306+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854306+0000","last_undegraded":"2023-12-01T03:12:34.854306+0000","last_fullsized":"2023-12-01T03:12:34.854306+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845305+0000","last_change":"2023-12-01T03:12:34.845305+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845305+0000","last_undegraded":"2023-12-01T03:12:34.845305+0000","last_fullsized":"2023-12-01T03:12:34.845305+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.853111+0000","last_change":"2023-12-01T03:12:34.853111+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.853111+0000","last_undegraded":"2023-12-01T03:12:34.853111+0000","last_fullsized":"2023-12-01T03:12:34.853111+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:21.877813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846554+0000","last_change":"2023-12-01T03:12:34.846554+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846554+0000","last_undegraded":"2023-12-01T03:12:34.846554+0000","last_fullsized":"2023-12-01T03:12:34.846554+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844698+0000","last_change":"2023-12-01T03:12:34.844698+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844698+0000","last_undegraded":"2023-12-01T03:12:34.844698+0000","last_fullsized":"2023-12-01T03:12:34.844698+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846725+0000","last_change":"2023-12-01T03:12:34.846725+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846725+0000","last_undegraded":"2023-12-01T03:12:34.846725+0000","last_fullsized":"2023-12-01T03:12:34.846725+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845156+0000","last_change":"2023-12-01T03:12:34.845156+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845156+0000","last_undegraded":"2023-12-01T03:12:34.845156+0000","last_fullsized":"2023-12-01T03:12:34.845156+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:31.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846691+0000","last_change":"2023-12-01T03:12:34.846691+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846691+0000","last_undegraded":"2023-12-01T03:12:34.846691+0000","last_fullsized":"2023-12-01T03:12:34.846691+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:59:04.995090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.843637+0000","last_change":"2023-12-01T03:12:34.843637+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.843637+0000","last_undegraded":"2023-12-01T03:12:34.843637+0000","last_fullsized":"2023-12-01T03:12:34.843637+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844221+0000","last_change":"2023-12-01T03:12:34.844221+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844221+0000","last_undegraded":"2023-12-01T03:12:34.844221+0000","last_fullsized":"2023-12-01T03:12:34.844221+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:00:57.853346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.855001+0000","last_change":"2023-12-01T03:12:34.855001+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.855001+0000","last_undegraded":"2023-12-01T03:12:34.855001+0000","last_fullsized":"2023-12-01T03:12:34.855001+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:34.404254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850115+0000","last_change":"2023-12-01T03:12:34.850115+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850115+0000","last_undegraded":"2023-12-01T03:12:34.850115+0000","last_fullsized":"2023-12-01T03:12:34.850115+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:00.577497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846472+0000","last_change":"2023-12-01T03:12:34.846472+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846472+0000","last_undegraded":"2023-12-01T03:12:34.846472+0000","last_fullsized":"2023-12-01T03:12:34.846472+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:09:42.224008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850472+0000","last_change":"2023-12-01T03:12:34.850472+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850472+0000","last_undegraded":"2023-12-01T03:12:34.850472+0000","last_fullsized":"2023-12-01T03:12:34.850472+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:53:08.535342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.847309+0000","last_change":"2023-12-01T03:12:34.847309+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.847309+0000","last_undegraded":"2023-12-01T03:12:34.847309+0000","last_fullsized":"2023-12-01T03:12:34.847309+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:50:37.148015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.853876+0000","last_change":"2023-12-01T03:12:34.853876+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.853876+0000","last_undegraded":"2023-12-01T03:12:34.853876+0000","last_fullsized":"2023-12-01T03:12:34.853876+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:25.025253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854643+0000","last_change":"2023-12-01T03:12:34.854643+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854643+0000","last_undegraded":"2023-12-01T03:12:34.854643+0000","last_fullsized":"2023-12-01T03:12:34.854643+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849276+0000","last_change":"2023-12-01T03:12:34.849276+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849276+0000","last_undegraded":"2023-12-01T03:12:34.849276+0000","last_fullsized":"2023-12-01T03:12:34.849276+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:10:45.146806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.869182+0000","last_change":"2023-12-01T03:12:34.867899+0000","last_active":"2023-12-01T03:12:34.869182+0000","last_peered":"2023-12-01T03:12:34.869182+0000","last_clean":"2023-12-01T03:12:34.869182+0000","last_became_active":"2023-12-01T03:12:34.867559+0000","last_became_peered":"2023-12-01T03:12:34.867559+0000","last_unstale":"2023-12-01T03:12:34.869182+0000","last_undegraded":"2023-12-01T03:12:34.869182+0000","last_fullsized":"2023-12-01T03:12:34.869182+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:49.066648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852053+0000","last_change":"2023-12-01T03:12:34.852053+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852053+0000","last_undegraded":"2023-12-01T03:12:34.852053+0000","last_fullsized":"2023-12-01T03:12:34.852053+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:59.927209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.869257+0000","last_change":"2023-12-01T03:12:34.868270+0000","last_active":"2023-12-01T03:12:34.869257+0000","last_peered":"2023-12-01T03:12:34.869257+0000","last_clean":"2023-12-01T03:12:34.869257+0000","last_became_active":"2023-12-01T03:12:34.867687+0000","last_became_peered":"2023-12-01T03:12:34.867687+0000","last_unstale":"2023-12-01T03:12:34.869257+0000","last_undegraded":"2023-12-01T03:12:34.869257+0000","last_fullsized":"2023-12-01T03:12:34.869257+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:03:51.996656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844945+0000","last_change":"2023-12-01T03:12:34.844945+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844945+0000","last_undegraded":"2023-12-01T03:12:34.844945+0000","last_fullsized":"2023-12-01T03:12:34.844945+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.870274+0000","last_change":"2023-12-01T03:12:34.869563+0000","last_active":"2023-12-01T03:12:34.870274+0000","last_peered":"2023-12-01T03:12:34.870274+0000","last_clean":"2023-12-01T03:12:34.870274+0000","last_became_active":"2023-12-01T03:12:34.869160+0000","last_became_peered":"2023-12-01T03:12:34.869160+0000","last_unstale":"2023-12-01T03:12:34.870274+0000","last_undegraded":"2023-12-01T03:12:34.870274+0000","last_fullsized":"2023-12-01T03:12:34.870274+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:19.315026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852093+0000","last_change":"2023-12-01T03:12:34.852093+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852093+0000","last_undegraded":"2023-12-01T03:12:34.852093+0000","last_fullsized":"2023-12-01T03:12:34.852093+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867132+0000","last_change":"2023-12-01T03:12:34.865447+0000","last_active":"2023-12-01T03:12:34.867132+0000","last_peered":"2023-12-01T03:12:34.867132+0000","last_clean":"2023-12-01T03:12:34.867132+0000","last_became_active":"2023-12-01T03:12:34.865092+0000","last_became_peered":"2023-12-01T03:12:34.865092+0000","last_unstale":"2023-12-01T03:12:34.867132+0000","last_undegraded":"2023-12-01T03:12:34.867132+0000","last_fullsized":"2023-12-01T03:12:34.867132+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:55:07.934979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850269+0000","last_change":"2023-12-01T03:12:34.850269+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850269+0000","last_undegraded":"2023-12-01T03:12:34.850269+0000","last_fullsized":"2023-12-01T03:12:34.850269+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864777+0000","last_change":"2023-12-01T03:12:34.857676+0000","last_active":"2023-12-01T03:12:34.864777+0000","last_peered":"2023-12-01T03:12:34.864777+0000","last_clean":"2023-12-01T03:12:34.864777+0000","last_became_active":"2023-12-01T03:12:34.857365+0000","last_became_peered":"2023-12-01T03:12:34.857365+0000","last_unstale":"2023-12-01T03:12:34.864777+0000","last_undegraded":"2023-12-01T03:12:34.864777+0000","last_fullsized":"2023-12-01T03:12:34.864777+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:22.088313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844881+0000","last_change":"2023-12-01T03:12:34.844881+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844881+0000","last_undegraded":"2023-12-01T03:12:34.844881+0000","last_fullsized":"2023-12-01T03:12:34.844881+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:13.355768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863077+0000","last_change":"2023-12-01T03:12:34.860204+0000","last_active":"2023-12-01T03:12:34.863077+0000","last_peered":"2023-12-01T03:12:34.863077+0000","last_clean":"2023-12-01T03:12:34.863077+0000","last_became_active":"2023-12-01T03:12:34.858034+0000","last_became_peered":"2023-12-01T03:12:34.858034+0000","last_unstale":"2023-12-01T03:12:34.863077+0000","last_undegraded":"2023-12-01T03:12:34.863077+0000","last_fullsized":"2023-12-01T03:12:34.863077+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:23:38.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852657+0000","last_change":"2023-12-01T03:12:34.852657+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852657+0000","last_undegraded":"2023-12-01T03:12:34.852657+0000","last_fullsized":"2023-12-01T03:12:34.852657+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864589+0000","last_change":"2023-12-01T03:12:34.863171+0000","last_active":"2023-12-01T03:12:34.864589+0000","last_peered":"2023-12-01T03:12:34.864589+0000","last_clean":"2023-12-01T03:12:34.864589+0000","last_became_active":"2023-12-01T03:12:34.862887+0000","last_became_peered":"2023-12-01T03:12:34.862887+0000","last_unstale":"2023-12-01T03:12:34.864589+0000","last_undegraded":"2023-12-01T03:12:34.864589+0000","last_fullsized":"2023-12-01T03:12:34.864589+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:51:11.709287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854072+0000","last_change":"2023-12-01T03:12:34.854072+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854072+0000","last_undegraded":"2023-12-01T03:12:34.854072+0000","last_fullsized":"2023-12-01T03:12:34.854072+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.865690+0000","last_change":"2023-12-01T03:12:34.864288+0000","last_active":"2023-12-01T03:12:34.865690+0000","last_peered":"2023-12-01T03:12:34.865690+0000","last_clean":"2023-12-01T03:12:34.865690+0000","last_became_active":"2023-12-01T03:12:34.863960+0000","last_became_peered":"2023-12-01T03:12:34.863960+0000","last_unstale":"2023-12-01T03:12:34.865690+0000","last_undegraded":"2023-12-01T03:12:34.865690+0000","last_fullsized":"2023-12-01T03:12:34.865690+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:02.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850169+0000","last_change":"2023-12-01T03:12:34.850169+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850169+0000","last_undegraded":"2023-12-01T03:12:34.850169+0000","last_fullsized":"2023-12-01T03:12:34.850169+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:19.476510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868695+0000","last_change":"2023-12-01T03:12:34.867539+0000","last_active":"2023-12-01T03:12:34.868695+0000","last_peered":"2023-12-01T03:12:34.868695+0000","last_clean":"2023-12-01T03:12:34.868695+0000","last_became_active":"2023-12-01T03:12:34.866362+0000","last_became_peered":"2023-12-01T03:12:34.866362+0000","last_unstale":"2023-12-01T03:12:34.868695+0000","last_undegraded":"2023-12-01T03:12:34.868695+0000","last_fullsized":"2023-12-01T03:12:34.868695+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:52.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852349+0000","last_change":"2023-12-01T03:12:34.852349+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852349+0000","last_undegraded":"2023-12-01T03:12:34.852349+0000","last_fullsized":"2023-12-01T03:12:34.852349+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.862751+0000","last_change":"2023-12-01T03:12:34.859136+0000","last_active":"2023-12-01T03:12:34.862751+0000","last_peered":"2023-12-01T03:12:34.862751+0000","last_clean":"2023-12-01T03:12:34.862751+0000","last_became_active":"2023-12-01T03:12:34.857840+0000","last_became_peered":"2023-12-01T03:12:34.857840+0000","last_unstale":"2023-12-01T03:12:34.862751+0000","last_undegraded":"2023-12-01T03:12:34.862751+0000","last_fullsized":"2023-12-01T03:12:34.862751+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:36.726453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849661+0000","last_change":"2023-12-01T03:12:34.849661+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849661+0000","last_undegraded":"2023-12-01T03:12:34.849661+0000","last_fullsized":"2023-12-01T03:12:34.849661+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863076+0000","last_change":"2023-12-01T03:12:34.860533+0000","last_active":"2023-12-01T03:12:34.863076+0000","last_peered":"2023-12-01T03:12:34.863076+0000","last_clean":"2023-12-01T03:12:34.863076+0000","last_became_active":"2023-12-01T03:12:34.858865+0000","last_became_peered":"2023-12-01T03:12:34.858865+0000","last_unstale":"2023-12-01T03:12:34.863076+0000","last_undegraded":"2023-12-01T03:12:34.863076+0000","last_fullsized":"2023-12-01T03:12:34.863076+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:22.088313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849924+0000","last_change":"2023-12-01T03:12:34.849924+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849924+0000","last_undegraded":"2023-12-01T03:12:34.849924+0000","last_fullsized":"2023-12-01T03:12:34.849924+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.859521+0000","last_change":"2023-12-01T03:12:34.857964+0000","last_active":"2023-12-01T03:12:34.859521+0000","last_peered":"2023-12-01T03:12:34.859521+0000","last_clean":"2023-12-01T03:12:34.859521+0000","last_became_active":"2023-12-01T03:12:34.856582+0000","last_became_peered":"2023-12-01T03:12:34.856582+0000","last_unstale":"2023-12-01T03:12:34.859521+0000","last_undegraded":"2023-12-01T03:12:34.859521+0000","last_fullsized":"2023-12-01T03:12:34.859521+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:48.550869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.853000+0000","last_change":"2023-12-01T03:12:34.853000+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.853000+0000","last_undegraded":"2023-12-01T03:12:34.853000+0000","last_fullsized":"2023-12-01T03:12:34.853000+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:23.725637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864341+0000","last_change":"2023-12-01T03:12:34.863097+0000","last_active":"2023-12-01T03:12:34.864341+0000","last_peered":"2023-12-01T03:12:34.864341+0000","last_clean":"2023-12-01T03:12:34.864341+0000","last_became_active":"2023-12-01T03:12:34.862657+0000","last_became_peered":"2023-12-01T03:12:34.862657+0000","last_unstale":"2023-12-01T03:12:34.864341+0000","last_undegraded":"2023-12-01T03:12:34.864341+0000","last_fullsized":"2023-12-01T03:12:34.864341+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:22.709559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845680+0000","last_change":"2023-12-01T03:12:34.845680+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845680+0000","last_undegraded":"2023-12-01T03:12:34.845680+0000","last_fullsized":"2023-12-01T03:12:34.845680+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863230+0000","last_change":"2023-12-01T03:12:34.860804+0000","last_active":"2023-12-01T03:12:34.863230+0000","last_peered":"2023-12-01T03:12:34.863230+0000","last_clean":"2023-12-01T03:12:34.863230+0000","last_became_active":"2023-12-01T03:12:34.859477+0000","last_became_peered":"2023-12-01T03:12:34.859477+0000","last_unstale":"2023-12-01T03:12:34.863230+0000","last_undegraded":"2023-12-01T03:12:34.863230+0000","last_fullsized":"2023-12-01T03:12:34.863230+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:00.153556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852165+0000","last_change":"2023-12-01T03:12:34.852165+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852165+0000","last_undegraded":"2023-12-01T03:12:34.852165+0000","last_fullsized":"2023-12-01T03:12:34.852165+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:04:09.892305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863339+0000","last_change":"2023-12-01T03:12:34.860578+0000","last_active":"2023-12-01T03:12:34.863339+0000","last_peered":"2023-12-01T03:12:34.863339+0000","last_clean":"2023-12-01T03:12:34.863339+0000","last_became_active":"2023-12-01T03:12:34.859107+0000","last_became_peered":"2023-12-01T03:12:34.859107+0000","last_unstale":"2023-12-01T03:12:34.863339+0000","last_undegraded":"2023-12-01T03:12:34.863339+0000","last_fullsized":"2023-12-01T03:12:34.863339+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:23:38.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846877+0000","last_change":"2023-12-01T03:12:34.846877+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846877+0000","last_undegraded":"2023-12-01T03:12:34.846877+0000","last_fullsized":"2023-12-01T03:12:34.846877+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868029+0000","last_change":"2023-12-01T03:12:34.866244+0000","last_active":"2023-12-01T03:12:34.868029+0000","last_peered":"2023-12-01T03:12:34.868029+0000","last_clean":"2023-12-01T03:12:34.868029+0000","last_became_active":"2023-12-01T03:12:34.865944+0000","last_became_peered":"2023-12-01T03:12:34.865944+0000","last_unstale":"2023-12-01T03:12:34.868029+0000","last_undegraded":"2023-12-01T03:12:34.868029+0000","last_fullsized":"2023-12-01T03:12:34.868029+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:28.385262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.847970+0000","last_change":"2023-12-01T03:12:34.847970+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.847970+0000","last_undegraded":"2023-12-01T03:12:34.847970+0000","last_fullsized":"2023-12-01T03:12:34.847970+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.866062+0000","last_change":"2023-12-01T03:12:34.864936+0000","last_active":"2023-12-01T03:12:34.866062+0000","last_peered":"2023-12-01T03:12:34.866062+0000","last_clean":"2023-12-01T03:12:34.866062+0000","last_became_active":"2023-12-01T03:12:34.864646+0000","last_became_peered":"2023-12-01T03:12:34.864646+0000","last_unstale":"2023-12-01T03:12:34.866062+0000","last_undegraded":"2023-12-01T03:12:34.866062+0000","last_fullsized":"2023-12-01T03:12:34.866062+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:00.153556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846620+0000","last_change":"2023-12-01T03:12:34.846620+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846620+0000","last_undegraded":"2023-12-01T03:12:34.846620+0000","last_fullsized":"2023-12-01T03:12:34.846620+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868225+0000","last_change":"2023-12-01T03:12:34.866465+0000","last_active":"2023-12-01T03:12:34.868225+0000","last_peered":"2023-12-01T03:12:34.868225+0000","last_clean":"2023-12-01T03:12:34.868225+0000","last_became_active":"2023-12-01T03:12:34.865451+0000","last_became_peered":"2023-12-01T03:12:34.865451+0000","last_unstale":"2023-12-01T03:12:34.868225+0000","last_undegraded":"2023-12-01T03:12:34.868225+0000","last_fullsized":"2023-12-01T03:12:34.868225+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:17:29.848009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852931+0000","last_change":"2023-12-01T03:12:34.852931+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852931+0000","last_undegraded":"2023-12-01T03:12:34.852931+0000","last_fullsized":"2023-12-01T03:12:34.852931+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:03:37.465541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867729+0000","last_change":"2023-12-01T03:12:34.866633+0000","last_active":"2023-12-01T03:12:34.867729+0000","last_peered":"2023-12-01T03:12:34.867729+0000","last_clean":"2023-12-01T03:12:34.867729+0000","last_became_active":"2023-12-01T03:12:34.865565+0000","last_became_peered":"2023-12-01T03:12:34.865565+0000","last_unstale":"2023-12-01T03:12:34.867729+0000","last_undegraded":"2023-12-01T03:12:34.867729+0000","last_fullsized":"2023-12-01T03:12:34.867729+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:37.392214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851102+0000","last_change":"2023-12-01T03:12:34.851102+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851102+0000","last_undegraded":"2023-12-01T03:12:34.851102+0000","last_fullsized":"2023-12-01T03:12:34.851102+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.862760+0000","last_change":"2023-12-01T03:12:34.858480+0000","last_active":"2023-12-01T03:12:34.862760+0000","last_peered":"2023-12-01T03:12:34.862760+0000","last_clean":"2023-12-01T03:12:34.862760+0000","last_became_active":"2023-12-01T03:12:34.857292+0000","last_became_peered":"2023-12-01T03:12:34.857292+0000","last_unstale":"2023-12-01T03:12:34.862760+0000","last_undegraded":"2023-12-01T03:12:34.862760+0000","last_fullsized":"2023-12-01T03:12:34.862760+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:02.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.848035+0000","last_change":"2023-12-01T03:12:34.848035+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.848035+0000","last_undegraded":"2023-12-01T03:12:34.848035+0000","last_fullsized":"2023-12-01T03:12:34.848035+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863422+0000","last_change":"2023-12-01T03:12:34.860490+0000","last_active":"2023-12-01T03:12:34.863422+0000","last_peered":"2023-12-01T03:12:34.863422+0000","last_clean":"2023-12-01T03:12:34.863422+0000","last_became_active":"2023-12-01T03:12:34.860228+0000","last_became_peered":"2023-12-01T03:12:34.860228+0000","last_unstale":"2023-12-01T03:12:34.863422+0000","last_undegraded":"2023-12-01T03:12:34.863422+0000","last_fullsized":"2023-12-01T03:12:34.863422+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:54:16.920168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846992+0000","last_change":"2023-12-01T03:12:34.846992+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846992+0000","last_undegraded":"2023-12-01T03:12:34.846992+0000","last_fullsized":"2023-12-01T03:12:34.846992+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.866877+0000","last_change":"2023-12-01T03:12:34.864610+0000","last_active":"2023-12-01T03:12:34.866877+0000","last_peered":"2023-12-01T03:12:34.866877+0000","last_clean":"2023-12-01T03:12:34.866877+0000","last_became_active":"2023-12-01T03:12:34.864320+0000","last_became_peered":"2023-12-01T03:12:34.864320+0000","last_unstale":"2023-12-01T03:12:34.866877+0000","last_undegraded":"2023-12-01T03:12:34.866877+0000","last_fullsized":"2023-12-01T03:12:34.866877+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:41.361208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.843616+0000","last_change":"2023-12-01T03:12:34.843616+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.843616+0000","last_undegraded":"2023-12-01T03:12:34.843616+0000","last_fullsized":"2023-12-01T03:12:34.843616+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867104+0000","last_change":"2023-12-01T03:12:34.864988+0000","last_active":"2023-12-01T03:12:34.867104+0000","last_peered":"2023-12-01T03:12:34.867104+0000","last_clean":"2023-12-01T03:12:34.867104+0000","last_became_active":"2023-12-01T03:12:34.864626+0000","last_became_peered":"2023-12-01T03:12:34.864626+0000","last_unstale":"2023-12-01T03:12:34.867104+0000","last_undegraded":"2023-12-01T03:12:34.867104+0000","last_fullsized":"2023-12-01T03:12:34.867104+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:35.500459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845514+0000","last_change":"2023-12-01T03:12:34.845514+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845514+0000","last_undegraded":"2023-12-01T03:12:34.845514+0000","last_fullsized":"2023-12-01T03:12:34.845514+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.862837+0000","last_change":"2023-12-01T03:12:34.859322+0000","last_active":"2023-12-01T03:12:34.862837+0000","last_peered":"2023-12-01T03:12:34.862837+0000","last_clean":"2023-12-01T03:12:34.862837+0000","last_became_active":"2023-12-01T03:12:34.859042+0000","last_became_peered":"2023-12-01T03:12:34.859042+0000","last_unstale":"2023-12-01T03:12:34.862837+0000","last_undegraded":"2023-12-01T03:12:34.862837+0000","last_fullsized":"2023-12-01T03:12:34.862837+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:08.980764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846826+0000","last_change":"2023-12-01T03:12:34.846826+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846826+0000","last_undegraded":"2023-12-01T03:12:34.846826+0000","last_fullsized":"2023-12-01T03:12:34.846826+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:55:08.951057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863024+0000","last_change":"2023-12-01T03:12:34.860198+0000","last_active":"2023-12-01T03:12:34.863024+0000","last_peered":"2023-12-01T03:12:34.863024+0000","last_clean":"2023-12-01T03:12:34.863024+0000","last_became_active":"2023-12-01T03:12:34.859894+0000","last_became_peered":"2023-12-01T03:12:34.859894+0000","last_unstale":"2023-12-01T03:12:34.863024+0000","last_undegraded":"2023-12-01T03:12:34.863024+0000","last_fullsized":"2023-12-01T03:12:34.863024+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:36.882914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851163+0000","last_change":"2023-12-01T03:12:34.851163+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851163+0000","last_undegraded":"2023-12-01T03:12:34.851163+0000","last_fullsized":"2023-12-01T03:12:34.851163+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:20.331105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868997+0000","last_change":"2023-12-01T03:12:34.868249+0000","last_active":"2023-12-01T03:12:34.868997+0000","last_peered":"2023-12-01T03:12:34.868997+0000","last_clean":"2023-12-01T03:12:34.868997+0000","last_became_active":"2023-12-01T03:12:34.867861+0000","last_became_peered":"2023-12-01T03:12:34.867861+0000","last_unstale":"2023-12-01T03:12:34.868997+0000","last_undegraded":"2023-12-01T03:12:34.868997+0000","last_fullsized":"2023-12-01T03:12:34.868997+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:00.274064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844493+0000","last_change":"2023-12-01T03:12:34.844493+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844493+0000","last_undegraded":"2023-12-01T03:12:34.844493+0000","last_fullsized":"2023-12-01T03:12:34.844493+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.865132+0000","last_change":"2023-12-01T03:12:34.857807+0000","last_active":"2023-12-01T03:12:34.865132+0000","last_peered":"2023-12-01T03:12:34.865132+0000","last_clean":"2023-12-01T03:12:34.865132+0000","last_became_active":"2023-12-01T03:12:34.857501+0000","last_became_peered":"2023-12-01T03:12:34.857501+0000","last_unstale":"2023-12-01T03:12:34.865132+0000","last_undegraded":"2023-12-01T03:12:34.865132+0000","last_fullsized":"2023-12-01T03:12:34.865132+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:36.882914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845661+0000","last_change":"2023-12-01T03:12:34.845661+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845661+0000","last_undegraded":"2023-12-01T03:12:34.845661+0000","last_fullsized":"2023-12-01T03:12:34.845661+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:45.819776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867080+0000","last_change":"2023-12-01T03:12:34.865848+0000","last_active":"2023-12-01T03:12:34.867080+0000","last_peered":"2023-12-01T03:12:34.867080+0000","last_clean":"2023-12-01T03:12:34.867080+0000","last_became_active":"2023-12-01T03:12:34.865157+0000","last_became_peered":"2023-12-01T03:12:34.865157+0000","last_unstale":"2023-12-01T03:12:34.867080+0000","last_undegraded":"2023-12-01T03:12:34.867080+0000","last_fullsized":"2023-12-01T03:12:34.867080+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:10.788196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849450+0000","last_change":"2023-12-01T03:12:34.849450+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849450+0000","last_undegraded":"2023-12-01T03:12:34.849450+0000","last_fullsized":"2023-12-01T03:12:34.849450+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:01.535641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.870973+0000","last_change":"2023-12-01T03:12:34.870482+0000","last_active":"2023-12-01T03:12:34.870973+0000","last_peered":"2023-12-01T03:12:34.870973+0000","last_clean":"2023-12-01T03:12:34.870973+0000","last_became_active":"2023-12-01T03:12:34.870111+0000","last_became_peered":"2023-12-01T03:12:34.870111+0000","last_unstale":"2023-12-01T03:12:34.870973+0000","last_undegraded":"2023-12-01T03:12:34.870973+0000","last_fullsized":"2023-12-01T03:12:34.870973+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:24.009174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.843521+0000","last_change":"2023-12-01T03:12:34.843521+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.843521+0000","last_undegraded":"2023-12-01T03:12:34.843521+0000","last_fullsized":"2023-12-01T03:12:34.843521+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864342+0000","last_change":"2023-12-01T03:12:34.863164+0000","last_active":"2023-12-01T03:12:34.864342+0000","last_peered":"2023-12-01T03:12:34.864342+0000","last_clean":"2023-12-01T03:12:34.864342+0000","last_became_active":"2023-12-01T03:12:34.862454+0000","last_became_peered":"2023-12-01T03:12:34.862454+0000","last_unstale":"2023-12-01T03:12:34.864342+0000","last_undegraded":"2023-12-01T03:12:34.864342+0000","last_fullsized":"2023-12-01T03:12:34.864342+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:12.944465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851703+0000","last_change":"2023-12-01T03:12:34.851703+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851703+0000","last_undegraded":"2023-12-01T03:12:34.851703+0000","last_fullsized":"2023-12-01T03:12:34.851703+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863195+0000","last_change":"2023-12-01T03:12:34.860753+0000","last_active":"2023-12-01T03:12:34.863195+0000","last_peered":"2023-12-01T03:12:34.863195+0000","last_clean":"2023-12-01T03:12:34.863195+0000","last_became_active":"2023-12-01T03:12:34.858035+0000","last_became_peered":"2023-12-01T03:12:34.858035+0000","last_unstale":"2023-12-01T03:12:34.863195+0000","last_undegraded":"2023-12-01T03:12:34.863195+0000","last_fullsized":"2023-12-01T03:12:34.863195+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:40.965606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845635+0000","last_change":"2023-12-01T03:12:34.845635+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845635+0000","last_undegraded":"2023-12-01T03:12:34.845635+0000","last_fullsized":"2023-12-01T03:12:34.845635+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:08:34.191802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868184+0000","last_change":"2023-12-01T03:12:34.866607+0000","last_active":"2023-12-01T03:12:34.868184+0000","last_peered":"2023-12-01T03:12:34.868184+0000","last_clean":"2023-12-01T03:12:34.868184+0000","last_became_active":"2023-12-01T03:12:34.865003+0000","last_became_peered":"2023-12-01T03:12:34.865003+0000","last_unstale":"2023-12-01T03:12:34.868184+0000","last_undegraded":"2023-12-01T03:12:34.868184+0000","last_fullsized":"2023-12-01T03:12:34.868184+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:29:11.426576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849489+0000","last_change":"2023-12-01T03:12:34.849489+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849489+0000","last_undegraded":"2023-12-01T03:12:34.849489+0000","last_fullsized":"2023-12-01T03:12:34.849489+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"241'359","reported_seq":1270,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.842152+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:12:34.842152+0000","last_peered":"2023-12-01T03:12:34.842152+0000","last_clean":"2023-12-01T03:12:34.842152+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:12:34.842152+0000","last_undegraded":"2023-12-01T03:12:34.842152+0000","last_fullsized":"2023-12-01T03:12:34.842152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60404,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311436,"statfs":{"total":96636764160,"available":96574910464,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0620000000000001,"15min":1.0760000000000001},"min":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.05},{"interface":"front","average":{"1min":1.343,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":1.169}]},{"osd":1,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":0.83999999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":1.399},{"interface":"front","average":{"1min":1.363,"5min":1.3069999999999999,"15min":1.327},"min":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.2849999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.167},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.95,"5min":1.95,"15min":2.1179999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.254,"15min":1.264},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.216}]},{"osd":3,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2729999999999999,"15min":1.302},"min":{"1min":0.78400000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.3959999999999999,"15min":1.425},"min":{"1min":0.84599999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":0.879}]},{"osd":4,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3140000000000001,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7989999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.341},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5029999999999999,"15min":1.508},"min":{"1min":1.1599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":1.3149999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.337,"15min":1.327},"min":{"1min":0.79600000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.369},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.5840000000000001,"15min":1.597},"min":{"1min":1.2430000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.8380000000000001,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":1.4630000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.476,"15min":1.454},"min":{"1min":0.89400000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.8579999999999997,"5min":4.8579999999999997,"15min":4.8579999999999997},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4319999999999999,"15min":1.4550000000000001},"min":{"1min":0.81799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":2.762},"last":1.113}]}]},{"osd":6,"up_from":13,"seq":55834576001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60416,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311424,"statfs":{"total":96636764160,"available":96574898176,"internally_reserved":0,"allocated":3203072,"data_stored":1577290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.036,"15min":1.0980000000000001},"min":{"1min":0.53600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4810000000000001,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.093,"15min":1.1519999999999999},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6439999999999999,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.373}]},{"osd":1,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.127,"15min":1.202},"min":{"1min":0.57399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7010000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.542,"5min":1.694,"15min":1.835},"last":1.0920000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5369999999999999,"5min":1.742,"15min":1.976},"last":1.51},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.272},"min":{"1min":0.71599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.637}]},{"osd":3,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.262,"15min":1.306},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7929999999999999,"5min":1.927,"15min":1.927},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.377,"15min":1.415},"min":{"1min":0.68700000000000006,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.1739999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2809999999999999,"15min":1.27},"min":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.661,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.4199999999999999,"15min":1.3660000000000001},"min":{"1min":0.64100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1019999999999999,"5min":2.48,"15min":2.48},"last":1.2210000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3859999999999999,"15min":1.407},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.651,"5min":2.262,"15min":2.262},"last":1.8200000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.518,"15min":1.492},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0030000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.3360000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5069999999999999,"15min":1.526},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.865,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.905},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0539999999999998,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.2709999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52792,"kb_used_data":3696,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319048,"statfs":{"total":96636764160,"available":96582705152,"internally_reserved":0,"allocated":3784704,"data_stored":2161339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0840000000000001,"15min":1.119},"min":{"1min":0.68200000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.476,"5min":1.776,"15min":1.776},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.996,"5min":1.133,"15min":1.151},"min":{"1min":0.72699999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3879999999999999,"5min":1.746,"15min":1.752},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.147,"15min":1.155},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.52,"5min":1.6519999999999999,"15min":1.696},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1850000000000001,"15min":1.226},"min":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.589,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":1.2410000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.21,"15min":1.246},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.5820000000000001,"5min":1.901,"15min":1.901},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.234,"15min":1.2490000000000001},"min":{"1min":0.90600000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5609999999999999,"5min":1.899,"15min":1.9219999999999999},"last":1.149}]},{"osd":3,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3029999999999999,"15min":1.3140000000000001},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.508,"5min":1.889,"15min":1.8979999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.3169999999999999,"15min":1.3540000000000001},"min":{"1min":1.0649999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.54,"5min":1.97,"15min":2.1859999999999999},"last":1.46}]},{"osd":4,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2669999999999999,"15min":1.256},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.764,"5min":1.869,"15min":2.6040000000000001},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3660000000000001,"15min":1.371},"min":{"1min":0.96599999999999997,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.5449999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.373}]},{"osd":6,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3839999999999999,"15min":1.375},"min":{"1min":0.97699999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.643,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4239999999999999,"15min":1.419},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.9850000000000001,"5min":2.032,"15min":2.577},"last":1.504}]},{"osd":7,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.5349999999999999,"15min":1.536},"min":{"1min":1.294,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.8779999999999999,"5min":2.077,"15min":3.1709999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.5780000000000001,"15min":1.5760000000000001},"min":{"1min":1.353,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.8819999999999999,"5min":2.101,"15min":2.8740000000000001},"last":1.76}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52212,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319628,"statfs":{"total":96636764160,"available":96583299072,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.65200000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.68600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.641,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.034}]},{"osd":1,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.216,"15min":1.2470000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5920000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.151,"15min":1.1699999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6930000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":0.86699999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.3100000000000001,"15min":1.238},"min":{"1min":0.81399999999999995,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.4630000000000001,"5min":11.311,"15min":11.311},"last":1.431},{"interface":"front","average":{"1min":1.159,"5min":1.2629999999999999,"15min":1.3200000000000001},"min":{"1min":0.86199999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.675,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2789999999999999,"15min":1.2829999999999999},"min":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5129999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.363,"15min":1.3320000000000001},"min":{"1min":0.90000000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2559999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.6100000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.375,"15min":1.329},"min":{"1min":1.028,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.897,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.125},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3580000000000001,"15min":1.3149999999999999},"min":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.74,"5min":3.71,"15min":3.71},"last":1.097}]},{"osd":6,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.3899999999999999,"15min":1.3740000000000001},"min":{"1min":1.0289999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.161,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":1.25,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.028,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.23}]},{"osd":7,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5089999999999999,"15min":1.4850000000000001},"min":{"1min":0.82399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.794,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5920000000000001,"15min":1.577},"min":{"1min":1.139,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3250000000000002,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.7350000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56308,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315532,"statfs":{"total":96636764160,"available":96579104768,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.306,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0449999999999999,"15min":1.0609999999999999},"min":{"1min":0.39500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.573,"5min":1.95,"15min":1.95},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.083,"15min":1.1479999999999999},"min":{"1min":0.56200000000000006,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.417,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.94399999999999995,"15min":0.98499999999999999},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6970000000000001,"5min":3.863,"15min":3.863},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.038,"15min":1.085},"min":{"1min":0.49099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.224,"5min":3.597,"15min":3.597},"last":0.629},{"interface":"front","average":{"1min":1.052,"5min":1.175,"15min":1.228},"min":{"1min":0.54700000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.647,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.78300000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.264,"15min":1.286},"min":{"1min":0.86099999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5269999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.248,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.79500000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9970000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.353}]},{"osd":5,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3109999999999999,"15min":1.325},"min":{"1min":0.92200000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.208,"5min":1.3420000000000001,"15min":1.3600000000000001},"min":{"1min":0.82499999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6699999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.22}]},{"osd":6,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.375,"15min":1.4370000000000001},"min":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.29,"5min":1.456,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.6879999999999999,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":1.149}]},{"osd":7,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4750000000000001,"15min":1.514},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.788,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4770000000000001,"15min":1.522},"min":{"1min":1.0549999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.913,"5min":3.004,"15min":3.004},"last":1.1930000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48116,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323724,"statfs":{"total":96636764160,"available":96587493376,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.81100000000000005,"15min":0.87},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92100000000000004,"5min":1.411,"15min":1.4730000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98899999999999999,"15min":1.024},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.216,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.97299999999999998,"15min":1.0309999999999999},"min":{"1min":0.57899999999999996,"5min":0.378,"15min":0.378},"max":{"1min":1.512,"5min":1.512,"15min":1.698},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.04,"15min":1.113},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5229999999999999,"5min":1.635,"15min":1.6599999999999999},"last":1.006}]},{"osd":3,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.97499999999999998,"15min":1.022},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2889999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.097,"15min":1.181},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.4970000000000001,"5min":1.615,"15min":1.79},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2749999999999999,"15min":1.3120000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.621,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.276},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4730000000000001,"5min":1.714,"15min":1.835},"last":1.1299999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.3109999999999999,"15min":1.3680000000000001},"min":{"1min":0.86599999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.639,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.347,"15min":1.393},"min":{"1min":0.79000000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.573,"5min":1.8740000000000001,"15min":2.016},"last":1.23}]},{"osd":6,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.351,"15min":1.367},"min":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.865,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.651},{"interface":"front","average":{"1min":1.268,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.86599999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7090000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4370000000000001,"15min":1.4630000000000001},"min":{"1min":0.94299999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5740000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.464},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.4350000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.8100000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.762}]}]},{"osd":1,"up_from":13,"seq":55834575998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52792,"kb_used_data":3696,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319048,"statfs":{"total":96636764160,"available":96582705152,"internally_reserved":0,"allocated":3784704,"data_stored":2161339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82899999999999996,"15min":0.88800000000000001},"min":{"1min":0.38500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.5569999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.94399999999999995,"15min":1.0249999999999999},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.167,"5min":1.512,"15min":1.681},"last":1.0409999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.86299999999999999,"15min":0.92300000000000004},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97699999999999998,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.28},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0349999999999999,"15min":1.1220000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.476,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":1.3819999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.1359999999999999,"15min":1.268},"min":{"1min":0.52400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.788},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.046,"15min":1.1120000000000001},"min":{"1min":0.41399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.615,"5min":1.714,"15min":1.714},"last":1.1799999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.177,"15min":1.2090000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6000000000000001,"5min":1.623,"15min":1.6859999999999999},"last":1.091},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.329,"15min":1.391},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7450000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.482}]},{"osd":5,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2969999999999999,"15min":1.3240000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8169999999999999,"5min":1.9399999999999999,"15min":2.0379999999999998},"last":1.302},{"interface":"front","average":{"1min":1.2,"5min":1.2569999999999999,"15min":1.319},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.7470000000000001,"5min":1.927,"15min":1.927},"last":1.3480000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.373,"15min":1.429},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.885,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3759999999999999,"15min":1.4019999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.901,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.159}]},{"osd":7,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.431,"15min":1.5069999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.46,"15min":1.5009999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.954,"5min":2.1389999999999998,"15min":2.2069999999999999},"last":1.516}]}]},{"osd":0,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60416,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311424,"statfs":{"total":96636764160,"available":96574898176,"internally_reserved":0,"allocated":3203072,"data_stored":1577290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.40100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94499999999999995,"15min":1.04},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.0249999999999999,"15min":1.0660000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.48699999999999999},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.1429999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.0269999999999999,"15min":1.107},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6619999999999999,"5min":1.726,"15min":1.792},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.99099999999999999,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.306,"5min":1.536,"15min":1.8839999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.194,"5min":1.165,"15min":1.222},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.27,"15min":1.337},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.246,"15min":1.2949999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.536,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1060000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.839,"5min":1.899,"15min":2.7719999999999998},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.365,"15min":1.423},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.484,"5min":2.484,"15min":2.5880000000000001},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.327,"15min":1.371},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.835,"5min":1.835,"15min":2.2029999999999998},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.444,"15min":1.4810000000000001},"min":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":1.9239999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.351}]},{"osd":7,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.4419999999999999,"15min":1.506},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.411,"5min":2.411,"15min":2.7040000000000002},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.456,"15min":1.4770000000000001},"min":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.718,"5min":2.1190000000000002,"15min":2.431},"last":1.143}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:12:39.855 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:12:40.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.106+0000 7efefefbb640 1 Processor -- start 2023-12-01T03:12:40.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.106+0000 7efefefbb640 1 -- start start 2023-12-01T03:12:40.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.108+0000 7efefefbb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efef80adac0 0x7efef80cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.108+0000 7efefefbb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efef806b350 0x7efef806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.108+0000 7efefefbb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef80ce150 con 0x7efef80c8ed0 2023-12-01T03:12:40.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.108+0000 7efefefbb640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef80cf3a0 con 0x7efef80adac0 2023-12-01T03:12:40.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.108+0000 7efefefbb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef80ce5e0 con 0x7efef806b350 2023-12-01T03:12:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.108+0000 7efefcd57640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efef80adac0 0x7efef80cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.108+0000 7efefcd57640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7efef80adac0 0x7efef80cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51798/0 (socket says 172.21.15.115:51798) 2023-12-01T03:12:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.108+0000 7efefcd57640 1 -- 172.21.15.115:0/1268466770 learned_addr learned my addr 172.21.15.115:0/1268466770 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.109+0000 7efefd558640 1 --2- 172.21.15.115:0/1268466770 >> v2:172.21.15.142:3300/0 conn(0x7efef806b350 0x7efef806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.109+0000 7efefcd57640 1 -- 172.21.15.115:0/1268466770 >> v2:172.21.15.142:3300/0 conn(0x7efef806b350 msgr2=0x7efef806b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.109+0000 7efefcd57640 1 --2- 172.21.15.115:0/1268466770 >> v2:172.21.15.142:3300/0 conn(0x7efef806b350 0x7efef806b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.109+0000 7efefcd57640 1 -- 172.21.15.115:0/1268466770 >> v1:172.21.15.142:6789/0 conn(0x7efef80c8ed0 legacy=0x7efef80cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:40.111 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.109+0000 7efefcd57640 1 -- 172.21.15.115:0/1268466770 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef806d620 con 0x7efef80adac0 2023-12-01T03:12:40.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.111+0000 7efefcd57640 1 --2- 172.21.15.115:0/1268466770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef80adac0 0x7efef80cc450 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7efef80c8e60 tx=0x7efeec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6744dff218ac0a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.111+0000 7efef77fe640 1 -- 172.21.15.115:0/1268466770 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec030860 con 0x7efef80adac0 2023-12-01T03:12:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.111+0000 7efef77fe640 1 -- 172.21.15.115:0/1268466770 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec0309c0 con 0x7efef80adac0 2023-12-01T03:12:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.111+0000 7efef77fe640 1 -- 172.21.15.115:0/1268466770 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec030d50 con 0x7efef80adac0 2023-12-01T03:12:40.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.112+0000 7efefefbb640 1 -- 172.21.15.115:0/1268466770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef80adac0 msgr2=0x7efef80cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.112+0000 7efefefbb640 1 --2- 172.21.15.115:0/1268466770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef80adac0 0x7efef80cc450 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7efef80c8e60 tx=0x7efeec02b9a0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.112+0000 7efefefbb640 1 -- 172.21.15.115:0/1268466770 shutdown_connections 2023-12-01T03:12:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.112+0000 7efefefbb640 1 --2- 172.21.15.115:0/1268466770 >> v2:172.21.15.142:3300/0 conn(0x7efef806b350 0x7efef806b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.112+0000 7efefefbb640 1 --2- 172.21.15.115:0/1268466770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef80adac0 0x7efef80cc450 unknown :-1 s=CLOSED pgs=2971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.112+0000 7efefefbb640 1 -- 172.21.15.115:0/1268466770 >> 172.21.15.115:0/1268466770 conn(0x7efef80ce880 msgr2=0x7efef80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.112+0000 7efefefbb640 1 -- 172.21.15.115:0/1268466770 shutdown_connections 2023-12-01T03:12:40.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.112+0000 7efefefbb640 1 -- 172.21.15.115:0/1268466770 wait complete. 2023-12-01T03:12:40.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.114+0000 7efefefbb640 1 Processor -- start 2023-12-01T03:12:40.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.114+0000 7efefefbb640 1 -- start start 2023-12-01T03:12:40.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefefbb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef806b350 0x7efef815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefefbb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef80adac0 0x7efef806f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefefbb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efef80c8ed0 0x7efef80c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefefbb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef8070100 con 0x7efef806b350 2023-12-01T03:12:40.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefefbb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80703e0 con 0x7efef80adac0 2023-12-01T03:12:40.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefefbb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef80706c0 con 0x7efef80c8ed0 2023-12-01T03:12:40.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefcd57640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef806b350 0x7efef815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefcd57640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef806b350 0x7efef815ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51802/0 (socket says 172.21.15.115:51802) 2023-12-01T03:12:40.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefcd57640 1 -- 172.21.15.115:0/322779370 learned_addr learned my addr 172.21.15.115:0/322779370 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:40.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efef7fff640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef80adac0 0x7efef806f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefcd57640 1 -- 172.21.15.115:0/322779370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efef80c8ed0 msgr2=0x7efef80c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefd558640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efef80c8ed0 0x7efef80c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefcd57640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efef80c8ed0 0x7efef80c5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefcd57640 1 -- 172.21.15.115:0/322779370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef80adac0 msgr2=0x7efef806f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefcd57640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef80adac0 0x7efef806f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efefcd57640 1 -- 172.21.15.115:0/322779370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec0094d0 con 0x7efef806b350 2023-12-01T03:12:40.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.115+0000 7efef7fff640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef80adac0 0x7efef806f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:40.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.116+0000 7efefcd57640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef806b350 0x7efef815ea60 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7efeec02b9e0 tx=0x7efeec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:40.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.116+0000 7efef57fa640 1 -- 172.21.15.115:0/322779370 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec039d60 con 0x7efef806b350 2023-12-01T03:12:40.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.116+0000 7efef57fa640 1 -- 172.21.15.115:0/322779370 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec043400 con 0x7efef806b350 2023-12-01T03:12:40.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.116+0000 7efef57fa640 1 -- 172.21.15.115:0/322779370 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec0437b0 con 0x7efef806b350 2023-12-01T03:12:40.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.116+0000 7efefefbb640 1 -- 172.21.15.115:0/322779370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef80709a0 con 0x7efef806b350 2023-12-01T03:12:40.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.116+0000 7efefefbb640 1 -- 172.21.15.115:0/322779370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef815f270 con 0x7efef806b350 2023-12-01T03:12:40.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.117+0000 7efefefbb640 1 -- 172.21.15.115:0/322779370 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef804e890 con 0x7efef806b350 2023-12-01T03:12:40.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.118+0000 7efef57fa640 1 -- 172.21.15.115:0/322779370 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7efeec043910 con 0x7efef806b350 2023-12-01T03:12:40.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.119+0000 7efef57fa640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efedc06a6a0 0x7efedc06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.119+0000 7efef57fa640 1 -- 172.21.15.115:0/322779370 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7efeec03e070 con 0x7efef806b350 2023-12-01T03:12:40.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.119+0000 7efef7fff640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efedc06a6a0 0x7efedc06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.120+0000 7efef7fff640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efedc06a6a0 0x7efedc06cb60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efef80c5100 tx=0x7efee4009040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:40.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.123+0000 7efef57fa640 1 -- 172.21.15.115:0/322779370 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7efeec077180 con 0x7efef806b350 2023-12-01T03:12:40.310 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.308+0000 7efefefbb640 1 -- 172.21.15.115:0/322779370 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efef80cf5b0 con 0x7efedc06a6a0 2023-12-01T03:12:40.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.330+0000 7efef57fa640 1 -- 172.21.15.115:0/322779370 <== mgr.11877 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284271 (secure 0 0 0) 0x7efee40013e0 con 0x7efedc06a6a0 2023-12-01T03:12:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:40.337 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:12:40.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.336+0000 7efefefbb640 1 -- 172.21.15.115:0/322779370 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efedc06a6a0 msgr2=0x7efedc06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.336+0000 7efefefbb640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efedc06a6a0 0x7efedc06cb60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efef80c5100 tx=0x7efee4009040 comp rx=0 tx=0).stop 2023-12-01T03:12:40.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.336+0000 7efefefbb640 1 -- 172.21.15.115:0/322779370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef806b350 msgr2=0x7efef815ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.336+0000 7efefefbb640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef806b350 0x7efef815ea60 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7efeec02b9e0 tx=0x7efeec004730 comp rx=0 tx=0).stop 2023-12-01T03:12:40.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.337+0000 7efefefbb640 1 -- 172.21.15.115:0/322779370 shutdown_connections 2023-12-01T03:12:40.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.337+0000 7efefefbb640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7efedc06a6a0 0x7efedc06cb60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.337+0000 7efefefbb640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efef80c8ed0 0x7efef80c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.339 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.337+0000 7efefefbb640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efef80adac0 0x7efef806f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.337+0000 7efefefbb640 1 --2- 172.21.15.115:0/322779370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efef806b350 0x7efef815ea60 unknown :-1 s=CLOSED pgs=2972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.337+0000 7efefefbb640 1 -- 172.21.15.115:0/322779370 >> 172.21.15.115:0/322779370 conn(0x7efef80ce880 msgr2=0x7efef8160dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:40.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.337+0000 7efefefbb640 1 -- 172.21.15.115:0/322779370 shutdown_connections 2023-12-01T03:12:40.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.337+0000 7efefefbb640 1 -- 172.21.15.115:0/322779370 wait complete. 2023-12-01T03:12:40.359 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-12-01T03:12:39.049245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":443456,"kb_used_data":26112,"kb_used_omap":0,"kb_used_meta":417280,"kb_avail":754531264,"statfs":{"total":773094113280,"available":772640014336,"internally_reserved":0,"allocated":26738688,"data_stored":13761142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":427294720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.236540"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851842+0000","last_change":"2023-12-01T03:12:34.851842+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851842+0000","last_undegraded":"2023-12-01T03:12:34.851842+0000","last_fullsized":"2023-12-01T03:12:34.851842+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:21:23.692942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849451+0000","last_change":"2023-12-01T03:12:34.849451+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849451+0000","last_undegraded":"2023-12-01T03:12:34.849451+0000","last_fullsized":"2023-12-01T03:12:34.849451+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845514+0000","last_change":"2023-12-01T03:12:34.845514+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845514+0000","last_undegraded":"2023-12-01T03:12:34.845514+0000","last_fullsized":"2023-12-01T03:12:34.845514+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850690+0000","last_change":"2023-12-01T03:12:34.850690+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850690+0000","last_undegraded":"2023-12-01T03:12:34.850690+0000","last_fullsized":"2023-12-01T03:12:34.850690+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:49.741119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850902+0000","last_change":"2023-12-01T03:12:34.850902+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850902+0000","last_undegraded":"2023-12-01T03:12:34.850902+0000","last_fullsized":"2023-12-01T03:12:34.850902+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.848923+0000","last_change":"2023-12-01T03:12:34.848923+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.848923+0000","last_undegraded":"2023-12-01T03:12:34.848923+0000","last_fullsized":"2023-12-01T03:12:34.848923+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:04.481100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.855117+0000","last_change":"2023-12-01T03:12:34.855117+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.855117+0000","last_undegraded":"2023-12-01T03:12:34.855117+0000","last_fullsized":"2023-12-01T03:12:34.855117+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846440+0000","last_change":"2023-12-01T03:12:34.846440+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846440+0000","last_undegraded":"2023-12-01T03:12:34.846440+0000","last_fullsized":"2023-12-01T03:12:34.846440+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:27.055115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.855652+0000","last_change":"2023-12-01T03:12:34.855652+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.855652+0000","last_undegraded":"2023-12-01T03:12:34.855652+0000","last_fullsized":"2023-12-01T03:12:34.855652+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:17:30.864087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.848420+0000","last_change":"2023-12-01T03:12:34.848420+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.848420+0000","last_undegraded":"2023-12-01T03:12:34.848420+0000","last_fullsized":"2023-12-01T03:12:34.848420+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852653+0000","last_change":"2023-12-01T03:12:34.852653+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852653+0000","last_undegraded":"2023-12-01T03:12:34.852653+0000","last_fullsized":"2023-12-01T03:12:34.852653+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845433+0000","last_change":"2023-12-01T03:12:34.845433+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845433+0000","last_undegraded":"2023-12-01T03:12:34.845433+0000","last_fullsized":"2023-12-01T03:12:34.845433+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852346+0000","last_change":"2023-12-01T03:12:34.852346+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852346+0000","last_undegraded":"2023-12-01T03:12:34.852346+0000","last_fullsized":"2023-12-01T03:12:34.852346+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:54:17.936247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854378+0000","last_change":"2023-12-01T03:12:34.854378+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854378+0000","last_undegraded":"2023-12-01T03:12:34.854378+0000","last_fullsized":"2023-12-01T03:12:34.854378+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:59:25.105386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854306+0000","last_change":"2023-12-01T03:12:34.854306+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854306+0000","last_undegraded":"2023-12-01T03:12:34.854306+0000","last_fullsized":"2023-12-01T03:12:34.854306+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845305+0000","last_change":"2023-12-01T03:12:34.845305+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845305+0000","last_undegraded":"2023-12-01T03:12:34.845305+0000","last_fullsized":"2023-12-01T03:12:34.845305+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.853111+0000","last_change":"2023-12-01T03:12:34.853111+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.853111+0000","last_undegraded":"2023-12-01T03:12:34.853111+0000","last_fullsized":"2023-12-01T03:12:34.853111+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:21.877813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846554+0000","last_change":"2023-12-01T03:12:34.846554+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846554+0000","last_undegraded":"2023-12-01T03:12:34.846554+0000","last_fullsized":"2023-12-01T03:12:34.846554+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844698+0000","last_change":"2023-12-01T03:12:34.844698+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844698+0000","last_undegraded":"2023-12-01T03:12:34.844698+0000","last_fullsized":"2023-12-01T03:12:34.844698+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846725+0000","last_change":"2023-12-01T03:12:34.846725+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846725+0000","last_undegraded":"2023-12-01T03:12:34.846725+0000","last_fullsized":"2023-12-01T03:12:34.846725+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845156+0000","last_change":"2023-12-01T03:12:34.845156+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845156+0000","last_undegraded":"2023-12-01T03:12:34.845156+0000","last_fullsized":"2023-12-01T03:12:34.845156+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:31.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846691+0000","last_change":"2023-12-01T03:12:34.846691+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846691+0000","last_undegraded":"2023-12-01T03:12:34.846691+0000","last_fullsized":"2023-12-01T03:12:34.846691+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:59:04.995090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.843637+0000","last_change":"2023-12-01T03:12:34.843637+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.843637+0000","last_undegraded":"2023-12-01T03:12:34.843637+0000","last_fullsized":"2023-12-01T03:12:34.843637+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844221+0000","last_change":"2023-12-01T03:12:34.844221+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844221+0000","last_undegraded":"2023-12-01T03:12:34.844221+0000","last_fullsized":"2023-12-01T03:12:34.844221+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:00:57.853346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.855001+0000","last_change":"2023-12-01T03:12:34.855001+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.855001+0000","last_undegraded":"2023-12-01T03:12:34.855001+0000","last_fullsized":"2023-12-01T03:12:34.855001+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:34.404254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850115+0000","last_change":"2023-12-01T03:12:34.850115+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850115+0000","last_undegraded":"2023-12-01T03:12:34.850115+0000","last_fullsized":"2023-12-01T03:12:34.850115+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:00.577497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846472+0000","last_change":"2023-12-01T03:12:34.846472+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846472+0000","last_undegraded":"2023-12-01T03:12:34.846472+0000","last_fullsized":"2023-12-01T03:12:34.846472+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:09:42.224008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850472+0000","last_change":"2023-12-01T03:12:34.850472+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850472+0000","last_undegraded":"2023-12-01T03:12:34.850472+0000","last_fullsized":"2023-12-01T03:12:34.850472+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:53:08.535342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.847309+0000","last_change":"2023-12-01T03:12:34.847309+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.847309+0000","last_undegraded":"2023-12-01T03:12:34.847309+0000","last_fullsized":"2023-12-01T03:12:34.847309+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:50:37.148015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.853876+0000","last_change":"2023-12-01T03:12:34.853876+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.853876+0000","last_undegraded":"2023-12-01T03:12:34.853876+0000","last_fullsized":"2023-12-01T03:12:34.853876+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:25.025253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854643+0000","last_change":"2023-12-01T03:12:34.854643+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854643+0000","last_undegraded":"2023-12-01T03:12:34.854643+0000","last_fullsized":"2023-12-01T03:12:34.854643+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849276+0000","last_change":"2023-12-01T03:12:34.849276+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849276+0000","last_undegraded":"2023-12-01T03:12:34.849276+0000","last_fullsized":"2023-12-01T03:12:34.849276+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:10:45.146806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.869182+0000","last_change":"2023-12-01T03:12:34.867899+0000","last_active":"2023-12-01T03:12:34.869182+0000","last_peered":"2023-12-01T03:12:34.869182+0000","last_clean":"2023-12-01T03:12:34.869182+0000","last_became_active":"2023-12-01T03:12:34.867559+0000","last_became_peered":"2023-12-01T03:12:34.867559+0000","last_unstale":"2023-12-01T03:12:34.869182+0000","last_undegraded":"2023-12-01T03:12:34.869182+0000","last_fullsized":"2023-12-01T03:12:34.869182+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:49.066648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852053+0000","last_change":"2023-12-01T03:12:34.852053+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852053+0000","last_undegraded":"2023-12-01T03:12:34.852053+0000","last_fullsized":"2023-12-01T03:12:34.852053+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:59.927209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.869257+0000","last_change":"2023-12-01T03:12:34.868270+0000","last_active":"2023-12-01T03:12:34.869257+0000","last_peered":"2023-12-01T03:12:34.869257+0000","last_clean":"2023-12-01T03:12:34.869257+0000","last_became_active":"2023-12-01T03:12:34.867687+0000","last_became_peered":"2023-12-01T03:12:34.867687+0000","last_unstale":"2023-12-01T03:12:34.869257+0000","last_undegraded":"2023-12-01T03:12:34.869257+0000","last_fullsized":"2023-12-01T03:12:34.869257+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:03:51.996656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844945+0000","last_change":"2023-12-01T03:12:34.844945+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844945+0000","last_undegraded":"2023-12-01T03:12:34.844945+0000","last_fullsized":"2023-12-01T03:12:34.844945+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.870274+0000","last_change":"2023-12-01T03:12:34.869563+0000","last_active":"2023-12-01T03:12:34.870274+0000","last_peered":"2023-12-01T03:12:34.870274+0000","last_clean":"2023-12-01T03:12:34.870274+0000","last_became_active":"2023-12-01T03:12:34.869160+0000","last_became_peered":"2023-12-01T03:12:34.869160+0000","last_unstale":"2023-12-01T03:12:34.870274+0000","last_undegraded":"2023-12-01T03:12:34.870274+0000","last_fullsized":"2023-12-01T03:12:34.870274+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:19.315026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852093+0000","last_change":"2023-12-01T03:12:34.852093+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852093+0000","last_undegraded":"2023-12-01T03:12:34.852093+0000","last_fullsized":"2023-12-01T03:12:34.852093+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867132+0000","last_change":"2023-12-01T03:12:34.865447+0000","last_active":"2023-12-01T03:12:34.867132+0000","last_peered":"2023-12-01T03:12:34.867132+0000","last_clean":"2023-12-01T03:12:34.867132+0000","last_became_active":"2023-12-01T03:12:34.865092+0000","last_became_peered":"2023-12-01T03:12:34.865092+0000","last_unstale":"2023-12-01T03:12:34.867132+0000","last_undegraded":"2023-12-01T03:12:34.867132+0000","last_fullsized":"2023-12-01T03:12:34.867132+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:55:07.934979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850269+0000","last_change":"2023-12-01T03:12:34.850269+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850269+0000","last_undegraded":"2023-12-01T03:12:34.850269+0000","last_fullsized":"2023-12-01T03:12:34.850269+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864777+0000","last_change":"2023-12-01T03:12:34.857676+0000","last_active":"2023-12-01T03:12:34.864777+0000","last_peered":"2023-12-01T03:12:34.864777+0000","last_clean":"2023-12-01T03:12:34.864777+0000","last_became_active":"2023-12-01T03:12:34.857365+0000","last_became_peered":"2023-12-01T03:12:34.857365+0000","last_unstale":"2023-12-01T03:12:34.864777+0000","last_undegraded":"2023-12-01T03:12:34.864777+0000","last_fullsized":"2023-12-01T03:12:34.864777+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:22.088313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844881+0000","last_change":"2023-12-01T03:12:34.844881+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844881+0000","last_undegraded":"2023-12-01T03:12:34.844881+0000","last_fullsized":"2023-12-01T03:12:34.844881+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:13.355768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863077+0000","last_change":"2023-12-01T03:12:34.860204+0000","last_active":"2023-12-01T03:12:34.863077+0000","last_peered":"2023-12-01T03:12:34.863077+0000","last_clean":"2023-12-01T03:12:34.863077+0000","last_became_active":"2023-12-01T03:12:34.858034+0000","last_became_peered":"2023-12-01T03:12:34.858034+0000","last_unstale":"2023-12-01T03:12:34.863077+0000","last_undegraded":"2023-12-01T03:12:34.863077+0000","last_fullsized":"2023-12-01T03:12:34.863077+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:23:38.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852657+0000","last_change":"2023-12-01T03:12:34.852657+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852657+0000","last_undegraded":"2023-12-01T03:12:34.852657+0000","last_fullsized":"2023-12-01T03:12:34.852657+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864589+0000","last_change":"2023-12-01T03:12:34.863171+0000","last_active":"2023-12-01T03:12:34.864589+0000","last_peered":"2023-12-01T03:12:34.864589+0000","last_clean":"2023-12-01T03:12:34.864589+0000","last_became_active":"2023-12-01T03:12:34.862887+0000","last_became_peered":"2023-12-01T03:12:34.862887+0000","last_unstale":"2023-12-01T03:12:34.864589+0000","last_undegraded":"2023-12-01T03:12:34.864589+0000","last_fullsized":"2023-12-01T03:12:34.864589+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:51:11.709287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854072+0000","last_change":"2023-12-01T03:12:34.854072+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854072+0000","last_undegraded":"2023-12-01T03:12:34.854072+0000","last_fullsized":"2023-12-01T03:12:34.854072+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.865690+0000","last_change":"2023-12-01T03:12:34.864288+0000","last_active":"2023-12-01T03:12:34.865690+0000","last_peered":"2023-12-01T03:12:34.865690+0000","last_clean":"2023-12-01T03:12:34.865690+0000","last_became_active":"2023-12-01T03:12:34.863960+0000","last_became_peered":"2023-12-01T03:12:34.863960+0000","last_unstale":"2023-12-01T03:12:34.865690+0000","last_undegraded":"2023-12-01T03:12:34.865690+0000","last_fullsized":"2023-12-01T03:12:34.865690+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:02.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850169+0000","last_change":"2023-12-01T03:12:34.850169+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850169+0000","last_undegraded":"2023-12-01T03:12:34.850169+0000","last_fullsized":"2023-12-01T03:12:34.850169+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:19.476510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868695+0000","last_change":"2023-12-01T03:12:34.867539+0000","last_active":"2023-12-01T03:12:34.868695+0000","last_peered":"2023-12-01T03:12:34.868695+0000","last_clean":"2023-12-01T03:12:34.868695+0000","last_became_active":"2023-12-01T03:12:34.866362+0000","last_became_peered":"2023-12-01T03:12:34.866362+0000","last_unstale":"2023-12-01T03:12:34.868695+0000","last_undegraded":"2023-12-01T03:12:34.868695+0000","last_fullsized":"2023-12-01T03:12:34.868695+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:52.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852349+0000","last_change":"2023-12-01T03:12:34.852349+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852349+0000","last_undegraded":"2023-12-01T03:12:34.852349+0000","last_fullsized":"2023-12-01T03:12:34.852349+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.862751+0000","last_change":"2023-12-01T03:12:34.859136+0000","last_active":"2023-12-01T03:12:34.862751+0000","last_peered":"2023-12-01T03:12:34.862751+0000","last_clean":"2023-12-01T03:12:34.862751+0000","last_became_active":"2023-12-01T03:12:34.857840+0000","last_became_peered":"2023-12-01T03:12:34.857840+0000","last_unstale":"2023-12-01T03:12:34.862751+0000","last_undegraded":"2023-12-01T03:12:34.862751+0000","last_fullsized":"2023-12-01T03:12:34.862751+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:36.726453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849661+0000","last_change":"2023-12-01T03:12:34.849661+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849661+0000","last_undegraded":"2023-12-01T03:12:34.849661+0000","last_fullsized":"2023-12-01T03:12:34.849661+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863076+0000","last_change":"2023-12-01T03:12:34.860533+0000","last_active":"2023-12-01T03:12:34.863076+0000","last_peered":"2023-12-01T03:12:34.863076+0000","last_clean":"2023-12-01T03:12:34.863076+0000","last_became_active":"2023-12-01T03:12:34.858865+0000","last_became_peered":"2023-12-01T03:12:34.858865+0000","last_unstale":"2023-12-01T03:12:34.863076+0000","last_undegraded":"2023-12-01T03:12:34.863076+0000","last_fullsized":"2023-12-01T03:12:34.863076+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:22.088313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849924+0000","last_change":"2023-12-01T03:12:34.849924+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849924+0000","last_undegraded":"2023-12-01T03:12:34.849924+0000","last_fullsized":"2023-12-01T03:12:34.849924+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.859521+0000","last_change":"2023-12-01T03:12:34.857964+0000","last_active":"2023-12-01T03:12:34.859521+0000","last_peered":"2023-12-01T03:12:34.859521+0000","last_clean":"2023-12-01T03:12:34.859521+0000","last_became_active":"2023-12-01T03:12:34.856582+0000","last_became_peered":"2023-12-01T03:12:34.856582+0000","last_unstale":"2023-12-01T03:12:34.859521+0000","last_undegraded":"2023-12-01T03:12:34.859521+0000","last_fullsized":"2023-12-01T03:12:34.859521+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:48.550869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.853000+0000","last_change":"2023-12-01T03:12:34.853000+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.853000+0000","last_undegraded":"2023-12-01T03:12:34.853000+0000","last_fullsized":"2023-12-01T03:12:34.853000+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:23.725637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864341+0000","last_change":"2023-12-01T03:12:34.863097+0000","last_active":"2023-12-01T03:12:34.864341+0000","last_peered":"2023-12-01T03:12:34.864341+0000","last_clean":"2023-12-01T03:12:34.864341+0000","last_became_active":"2023-12-01T03:12:34.862657+0000","last_became_peered":"2023-12-01T03:12:34.862657+0000","last_unstale":"2023-12-01T03:12:34.864341+0000","last_undegraded":"2023-12-01T03:12:34.864341+0000","last_fullsized":"2023-12-01T03:12:34.864341+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:22.709559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845680+0000","last_change":"2023-12-01T03:12:34.845680+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845680+0000","last_undegraded":"2023-12-01T03:12:34.845680+0000","last_fullsized":"2023-12-01T03:12:34.845680+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863230+0000","last_change":"2023-12-01T03:12:34.860804+0000","last_active":"2023-12-01T03:12:34.863230+0000","last_peered":"2023-12-01T03:12:34.863230+0000","last_clean":"2023-12-01T03:12:34.863230+0000","last_became_active":"2023-12-01T03:12:34.859477+0000","last_became_peered":"2023-12-01T03:12:34.859477+0000","last_unstale":"2023-12-01T03:12:34.863230+0000","last_undegraded":"2023-12-01T03:12:34.863230+0000","last_fullsized":"2023-12-01T03:12:34.863230+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:00.153556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852165+0000","last_change":"2023-12-01T03:12:34.852165+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852165+0000","last_undegraded":"2023-12-01T03:12:34.852165+0000","last_fullsized":"2023-12-01T03:12:34.852165+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:04:09.892305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863339+0000","last_change":"2023-12-01T03:12:34.860578+0000","last_active":"2023-12-01T03:12:34.863339+0000","last_peered":"2023-12-01T03:12:34.863339+0000","last_clean":"2023-12-01T03:12:34.863339+0000","last_became_active":"2023-12-01T03:12:34.859107+0000","last_became_peered":"2023-12-01T03:12:34.859107+0000","last_unstale":"2023-12-01T03:12:34.863339+0000","last_undegraded":"2023-12-01T03:12:34.863339+0000","last_fullsized":"2023-12-01T03:12:34.863339+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:23:38.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846877+0000","last_change":"2023-12-01T03:12:34.846877+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846877+0000","last_undegraded":"2023-12-01T03:12:34.846877+0000","last_fullsized":"2023-12-01T03:12:34.846877+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868029+0000","last_change":"2023-12-01T03:12:34.866244+0000","last_active":"2023-12-01T03:12:34.868029+0000","last_peered":"2023-12-01T03:12:34.868029+0000","last_clean":"2023-12-01T03:12:34.868029+0000","last_became_active":"2023-12-01T03:12:34.865944+0000","last_became_peered":"2023-12-01T03:12:34.865944+0000","last_unstale":"2023-12-01T03:12:34.868029+0000","last_undegraded":"2023-12-01T03:12:34.868029+0000","last_fullsized":"2023-12-01T03:12:34.868029+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:28.385262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.847970+0000","last_change":"2023-12-01T03:12:34.847970+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.847970+0000","last_undegraded":"2023-12-01T03:12:34.847970+0000","last_fullsized":"2023-12-01T03:12:34.847970+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.866062+0000","last_change":"2023-12-01T03:12:34.864936+0000","last_active":"2023-12-01T03:12:34.866062+0000","last_peered":"2023-12-01T03:12:34.866062+0000","last_clean":"2023-12-01T03:12:34.866062+0000","last_became_active":"2023-12-01T03:12:34.864646+0000","last_became_peered":"2023-12-01T03:12:34.864646+0000","last_unstale":"2023-12-01T03:12:34.866062+0000","last_undegraded":"2023-12-01T03:12:34.866062+0000","last_fullsized":"2023-12-01T03:12:34.866062+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:00.153556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846620+0000","last_change":"2023-12-01T03:12:34.846620+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846620+0000","last_undegraded":"2023-12-01T03:12:34.846620+0000","last_fullsized":"2023-12-01T03:12:34.846620+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868225+0000","last_change":"2023-12-01T03:12:34.866465+0000","last_active":"2023-12-01T03:12:34.868225+0000","last_peered":"2023-12-01T03:12:34.868225+0000","last_clean":"2023-12-01T03:12:34.868225+0000","last_became_active":"2023-12-01T03:12:34.865451+0000","last_became_peered":"2023-12-01T03:12:34.865451+0000","last_unstale":"2023-12-01T03:12:34.868225+0000","last_undegraded":"2023-12-01T03:12:34.868225+0000","last_fullsized":"2023-12-01T03:12:34.868225+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:17:29.848009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852931+0000","last_change":"2023-12-01T03:12:34.852931+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852931+0000","last_undegraded":"2023-12-01T03:12:34.852931+0000","last_fullsized":"2023-12-01T03:12:34.852931+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:03:37.465541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867729+0000","last_change":"2023-12-01T03:12:34.866633+0000","last_active":"2023-12-01T03:12:34.867729+0000","last_peered":"2023-12-01T03:12:34.867729+0000","last_clean":"2023-12-01T03:12:34.867729+0000","last_became_active":"2023-12-01T03:12:34.865565+0000","last_became_peered":"2023-12-01T03:12:34.865565+0000","last_unstale":"2023-12-01T03:12:34.867729+0000","last_undegraded":"2023-12-01T03:12:34.867729+0000","last_fullsized":"2023-12-01T03:12:34.867729+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:37.392214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851102+0000","last_change":"2023-12-01T03:12:34.851102+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851102+0000","last_undegraded":"2023-12-01T03:12:34.851102+0000","last_fullsized":"2023-12-01T03:12:34.851102+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.862760+0000","last_change":"2023-12-01T03:12:34.858480+0000","last_active":"2023-12-01T03:12:34.862760+0000","last_peered":"2023-12-01T03:12:34.862760+0000","last_clean":"2023-12-01T03:12:34.862760+0000","last_became_active":"2023-12-01T03:12:34.857292+0000","last_became_peered":"2023-12-01T03:12:34.857292+0000","last_unstale":"2023-12-01T03:12:34.862760+0000","last_undegraded":"2023-12-01T03:12:34.862760+0000","last_fullsized":"2023-12-01T03:12:34.862760+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:02.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.848035+0000","last_change":"2023-12-01T03:12:34.848035+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.848035+0000","last_undegraded":"2023-12-01T03:12:34.848035+0000","last_fullsized":"2023-12-01T03:12:34.848035+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863422+0000","last_change":"2023-12-01T03:12:34.860490+0000","last_active":"2023-12-01T03:12:34.863422+0000","last_peered":"2023-12-01T03:12:34.863422+0000","last_clean":"2023-12-01T03:12:34.863422+0000","last_became_active":"2023-12-01T03:12:34.860228+0000","last_became_peered":"2023-12-01T03:12:34.860228+0000","last_unstale":"2023-12-01T03:12:34.863422+0000","last_undegraded":"2023-12-01T03:12:34.863422+0000","last_fullsized":"2023-12-01T03:12:34.863422+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:54:16.920168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846992+0000","last_change":"2023-12-01T03:12:34.846992+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846992+0000","last_undegraded":"2023-12-01T03:12:34.846992+0000","last_fullsized":"2023-12-01T03:12:34.846992+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.866877+0000","last_change":"2023-12-01T03:12:34.864610+0000","last_active":"2023-12-01T03:12:34.866877+0000","last_peered":"2023-12-01T03:12:34.866877+0000","last_clean":"2023-12-01T03:12:34.866877+0000","last_became_active":"2023-12-01T03:12:34.864320+0000","last_became_peered":"2023-12-01T03:12:34.864320+0000","last_unstale":"2023-12-01T03:12:34.866877+0000","last_undegraded":"2023-12-01T03:12:34.866877+0000","last_fullsized":"2023-12-01T03:12:34.866877+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:41.361208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.843616+0000","last_change":"2023-12-01T03:12:34.843616+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.843616+0000","last_undegraded":"2023-12-01T03:12:34.843616+0000","last_fullsized":"2023-12-01T03:12:34.843616+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867104+0000","last_change":"2023-12-01T03:12:34.864988+0000","last_active":"2023-12-01T03:12:34.867104+0000","last_peered":"2023-12-01T03:12:34.867104+0000","last_clean":"2023-12-01T03:12:34.867104+0000","last_became_active":"2023-12-01T03:12:34.864626+0000","last_became_peered":"2023-12-01T03:12:34.864626+0000","last_unstale":"2023-12-01T03:12:34.867104+0000","last_undegraded":"2023-12-01T03:12:34.867104+0000","last_fullsized":"2023-12-01T03:12:34.867104+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:35.500459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845514+0000","last_change":"2023-12-01T03:12:34.845514+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845514+0000","last_undegraded":"2023-12-01T03:12:34.845514+0000","last_fullsized":"2023-12-01T03:12:34.845514+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.862837+0000","last_change":"2023-12-01T03:12:34.859322+0000","last_active":"2023-12-01T03:12:34.862837+0000","last_peered":"2023-12-01T03:12:34.862837+0000","last_clean":"2023-12-01T03:12:34.862837+0000","last_became_active":"2023-12-01T03:12:34.859042+0000","last_became_peered":"2023-12-01T03:12:34.859042+0000","last_unstale":"2023-12-01T03:12:34.862837+0000","last_undegraded":"2023-12-01T03:12:34.862837+0000","last_fullsized":"2023-12-01T03:12:34.862837+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:08.980764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846826+0000","last_change":"2023-12-01T03:12:34.846826+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846826+0000","last_undegraded":"2023-12-01T03:12:34.846826+0000","last_fullsized":"2023-12-01T03:12:34.846826+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:55:08.951057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863024+0000","last_change":"2023-12-01T03:12:34.860198+0000","last_active":"2023-12-01T03:12:34.863024+0000","last_peered":"2023-12-01T03:12:34.863024+0000","last_clean":"2023-12-01T03:12:34.863024+0000","last_became_active":"2023-12-01T03:12:34.859894+0000","last_became_peered":"2023-12-01T03:12:34.859894+0000","last_unstale":"2023-12-01T03:12:34.863024+0000","last_undegraded":"2023-12-01T03:12:34.863024+0000","last_fullsized":"2023-12-01T03:12:34.863024+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:36.882914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851163+0000","last_change":"2023-12-01T03:12:34.851163+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851163+0000","last_undegraded":"2023-12-01T03:12:34.851163+0000","last_fullsized":"2023-12-01T03:12:34.851163+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:20.331105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868997+0000","last_change":"2023-12-01T03:12:34.868249+0000","last_active":"2023-12-01T03:12:34.868997+0000","last_peered":"2023-12-01T03:12:34.868997+0000","last_clean":"2023-12-01T03:12:34.868997+0000","last_became_active":"2023-12-01T03:12:34.867861+0000","last_became_peered":"2023-12-01T03:12:34.867861+0000","last_unstale":"2023-12-01T03:12:34.868997+0000","last_undegraded":"2023-12-01T03:12:34.868997+0000","last_fullsized":"2023-12-01T03:12:34.868997+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:00.274064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844493+0000","last_change":"2023-12-01T03:12:34.844493+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844493+0000","last_undegraded":"2023-12-01T03:12:34.844493+0000","last_fullsized":"2023-12-01T03:12:34.844493+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.865132+0000","last_change":"2023-12-01T03:12:34.857807+0000","last_active":"2023-12-01T03:12:34.865132+0000","last_peered":"2023-12-01T03:12:34.865132+0000","last_clean":"2023-12-01T03:12:34.865132+0000","last_became_active":"2023-12-01T03:12:34.857501+0000","last_became_peered":"2023-12-01T03:12:34.857501+0000","last_unstale":"2023-12-01T03:12:34.865132+0000","last_undegraded":"2023-12-01T03:12:34.865132+0000","last_fullsized":"2023-12-01T03:12:34.865132+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:36.882914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845661+0000","last_change":"2023-12-01T03:12:34.845661+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845661+0000","last_undegraded":"2023-12-01T03:12:34.845661+0000","last_fullsized":"2023-12-01T03:12:34.845661+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:45.819776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867080+0000","last_change":"2023-12-01T03:12:34.865848+0000","last_active":"2023-12-01T03:12:34.867080+0000","last_peered":"2023-12-01T03:12:34.867080+0000","last_clean":"2023-12-01T03:12:34.867080+0000","last_became_active":"2023-12-01T03:12:34.865157+0000","last_became_peered":"2023-12-01T03:12:34.865157+0000","last_unstale":"2023-12-01T03:12:34.867080+0000","last_undegraded":"2023-12-01T03:12:34.867080+0000","last_fullsized":"2023-12-01T03:12:34.867080+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:10.788196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849450+0000","last_change":"2023-12-01T03:12:34.849450+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849450+0000","last_undegraded":"2023-12-01T03:12:34.849450+0000","last_fullsized":"2023-12-01T03:12:34.849450+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:01.535641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.870973+0000","last_change":"2023-12-01T03:12:34.870482+0000","last_active":"2023-12-01T03:12:34.870973+0000","last_peered":"2023-12-01T03:12:34.870973+0000","last_clean":"2023-12-01T03:12:34.870973+0000","last_became_active":"2023-12-01T03:12:34.870111+0000","last_became_peered":"2023-12-01T03:12:34.870111+0000","last_unstale":"2023-12-01T03:12:34.870973+0000","last_undegraded":"2023-12-01T03:12:34.870973+0000","last_fullsized":"2023-12-01T03:12:34.870973+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:24.009174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.843521+0000","last_change":"2023-12-01T03:12:34.843521+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.843521+0000","last_undegraded":"2023-12-01T03:12:34.843521+0000","last_fullsized":"2023-12-01T03:12:34.843521+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864342+0000","last_change":"2023-12-01T03:12:34.863164+0000","last_active":"2023-12-01T03:12:34.864342+0000","last_peered":"2023-12-01T03:12:34.864342+0000","last_clean":"2023-12-01T03:12:34.864342+0000","last_became_active":"2023-12-01T03:12:34.862454+0000","last_became_peered":"2023-12-01T03:12:34.862454+0000","last_unstale":"2023-12-01T03:12:34.864342+0000","last_undegraded":"2023-12-01T03:12:34.864342+0000","last_fullsized":"2023-12-01T03:12:34.864342+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:12.944465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851703+0000","last_change":"2023-12-01T03:12:34.851703+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851703+0000","last_undegraded":"2023-12-01T03:12:34.851703+0000","last_fullsized":"2023-12-01T03:12:34.851703+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863195+0000","last_change":"2023-12-01T03:12:34.860753+0000","last_active":"2023-12-01T03:12:34.863195+0000","last_peered":"2023-12-01T03:12:34.863195+0000","last_clean":"2023-12-01T03:12:34.863195+0000","last_became_active":"2023-12-01T03:12:34.858035+0000","last_became_peered":"2023-12-01T03:12:34.858035+0000","last_unstale":"2023-12-01T03:12:34.863195+0000","last_undegraded":"2023-12-01T03:12:34.863195+0000","last_fullsized":"2023-12-01T03:12:34.863195+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:40.965606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845635+0000","last_change":"2023-12-01T03:12:34.845635+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845635+0000","last_undegraded":"2023-12-01T03:12:34.845635+0000","last_fullsized":"2023-12-01T03:12:34.845635+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:08:34.191802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868184+0000","last_change":"2023-12-01T03:12:34.866607+0000","last_active":"2023-12-01T03:12:34.868184+0000","last_peered":"2023-12-01T03:12:34.868184+0000","last_clean":"2023-12-01T03:12:34.868184+0000","last_became_active":"2023-12-01T03:12:34.865003+0000","last_became_peered":"2023-12-01T03:12:34.865003+0000","last_unstale":"2023-12-01T03:12:34.868184+0000","last_undegraded":"2023-12-01T03:12:34.868184+0000","last_fullsized":"2023-12-01T03:12:34.868184+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:29:11.426576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849489+0000","last_change":"2023-12-01T03:12:34.849489+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849489+0000","last_undegraded":"2023-12-01T03:12:34.849489+0000","last_fullsized":"2023-12-01T03:12:34.849489+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"241'359","reported_seq":1270,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.842152+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:12:34.842152+0000","last_peered":"2023-12-01T03:12:34.842152+0000","last_clean":"2023-12-01T03:12:34.842152+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:12:34.842152+0000","last_undegraded":"2023-12-01T03:12:34.842152+0000","last_fullsized":"2023-12-01T03:12:34.842152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60404,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311436,"statfs":{"total":96636764160,"available":96574910464,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0620000000000001,"15min":1.0760000000000001},"min":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.05},{"interface":"front","average":{"1min":1.343,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":1.169}]},{"osd":1,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":0.83999999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":1.399},{"interface":"front","average":{"1min":1.363,"5min":1.3069999999999999,"15min":1.327},"min":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.2849999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.167},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.95,"5min":1.95,"15min":2.1179999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.254,"15min":1.264},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.216}]},{"osd":3,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2729999999999999,"15min":1.302},"min":{"1min":0.78400000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.3959999999999999,"15min":1.425},"min":{"1min":0.84599999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":0.879}]},{"osd":4,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3140000000000001,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7989999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.341},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5029999999999999,"15min":1.508},"min":{"1min":1.1599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":1.3149999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.337,"15min":1.327},"min":{"1min":0.79600000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.369},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.5840000000000001,"15min":1.597},"min":{"1min":1.2430000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.8380000000000001,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":1.4630000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.476,"15min":1.454},"min":{"1min":0.89400000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.8579999999999997,"5min":4.8579999999999997,"15min":4.8579999999999997},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4319999999999999,"15min":1.4550000000000001},"min":{"1min":0.81799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":2.762},"last":1.113}]}]},{"osd":6,"up_from":13,"seq":55834576001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60416,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311424,"statfs":{"total":96636764160,"available":96574898176,"internally_reserved":0,"allocated":3203072,"data_stored":1577290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.036,"15min":1.0980000000000001},"min":{"1min":0.53600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4810000000000001,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.093,"15min":1.1519999999999999},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6439999999999999,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.373}]},{"osd":1,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.127,"15min":1.202},"min":{"1min":0.57399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7010000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.542,"5min":1.694,"15min":1.835},"last":1.0920000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5369999999999999,"5min":1.742,"15min":1.976},"last":1.51},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.272},"min":{"1min":0.71599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.637}]},{"osd":3,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.262,"15min":1.306},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7929999999999999,"5min":1.927,"15min":1.927},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.377,"15min":1.415},"min":{"1min":0.68700000000000006,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.1739999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2809999999999999,"15min":1.27},"min":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.661,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.4199999999999999,"15min":1.3660000000000001},"min":{"1min":0.64100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1019999999999999,"5min":2.48,"15min":2.48},"last":1.2210000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3859999999999999,"15min":1.407},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.651,"5min":2.262,"15min":2.262},"last":1.8200000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.518,"15min":1.492},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0030000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.3360000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5069999999999999,"15min":1.526},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.865,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.905},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0539999999999998,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.2709999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52792,"kb_used_data":3696,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319048,"statfs":{"total":96636764160,"available":96582705152,"internally_reserved":0,"allocated":3784704,"data_stored":2161339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0840000000000001,"15min":1.119},"min":{"1min":0.68200000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.476,"5min":1.776,"15min":1.776},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.996,"5min":1.133,"15min":1.151},"min":{"1min":0.72699999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3879999999999999,"5min":1.746,"15min":1.752},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.147,"15min":1.155},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.52,"5min":1.6519999999999999,"15min":1.696},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1850000000000001,"15min":1.226},"min":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.589,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":1.2410000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.21,"15min":1.246},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.5820000000000001,"5min":1.901,"15min":1.901},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.234,"15min":1.2490000000000001},"min":{"1min":0.90600000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5609999999999999,"5min":1.899,"15min":1.9219999999999999},"last":1.149}]},{"osd":3,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3029999999999999,"15min":1.3140000000000001},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.508,"5min":1.889,"15min":1.8979999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.3169999999999999,"15min":1.3540000000000001},"min":{"1min":1.0649999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.54,"5min":1.97,"15min":2.1859999999999999},"last":1.46}]},{"osd":4,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2669999999999999,"15min":1.256},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.764,"5min":1.869,"15min":2.6040000000000001},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3660000000000001,"15min":1.371},"min":{"1min":0.96599999999999997,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.5449999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.373}]},{"osd":6,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3839999999999999,"15min":1.375},"min":{"1min":0.97699999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.643,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4239999999999999,"15min":1.419},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.9850000000000001,"5min":2.032,"15min":2.577},"last":1.504}]},{"osd":7,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.5349999999999999,"15min":1.536},"min":{"1min":1.294,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.8779999999999999,"5min":2.077,"15min":3.1709999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.5780000000000001,"15min":1.5760000000000001},"min":{"1min":1.353,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.8819999999999999,"5min":2.101,"15min":2.8740000000000001},"last":1.76}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52212,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319628,"statfs":{"total":96636764160,"available":96583299072,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.65200000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.68600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.641,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.034}]},{"osd":1,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.216,"15min":1.2470000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5920000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.151,"15min":1.1699999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6930000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":0.86699999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.3100000000000001,"15min":1.238},"min":{"1min":0.81399999999999995,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.4630000000000001,"5min":11.311,"15min":11.311},"last":1.431},{"interface":"front","average":{"1min":1.159,"5min":1.2629999999999999,"15min":1.3200000000000001},"min":{"1min":0.86199999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.675,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2789999999999999,"15min":1.2829999999999999},"min":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5129999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.363,"15min":1.3320000000000001},"min":{"1min":0.90000000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2559999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.6100000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.375,"15min":1.329},"min":{"1min":1.028,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.897,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.125},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3580000000000001,"15min":1.3149999999999999},"min":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.74,"5min":3.71,"15min":3.71},"last":1.097}]},{"osd":6,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.3899999999999999,"15min":1.3740000000000001},"min":{"1min":1.0289999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.161,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":1.25,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.028,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.23}]},{"osd":7,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5089999999999999,"15min":1.4850000000000001},"min":{"1min":0.82399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.794,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5920000000000001,"15min":1.577},"min":{"1min":1.139,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3250000000000002,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.7350000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56308,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315532,"statfs":{"total":96636764160,"available":96579104768,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.306,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0449999999999999,"15min":1.0609999999999999},"min":{"1min":0.39500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.573,"5min":1.95,"15min":1.95},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.083,"15min":1.1479999999999999},"min":{"1min":0.56200000000000006,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.417,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.94399999999999995,"15min":0.98499999999999999},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6970000000000001,"5min":3.863,"15min":3.863},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.038,"15min":1.085},"min":{"1min":0.49099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.224,"5min":3.597,"15min":3.597},"last":0.629},{"interface":"front","average":{"1min":1.052,"5min":1.175,"15min":1.228},"min":{"1min":0.54700000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.647,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.78300000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.264,"15min":1.286},"min":{"1min":0.86099999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5269999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.248,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.79500000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9970000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.353}]},{"osd":5,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3109999999999999,"15min":1.325},"min":{"1min":0.92200000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.208,"5min":1.3420000000000001,"15min":1.3600000000000001},"min":{"1min":0.82499999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6699999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.22}]},{"osd":6,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.375,"15min":1.4370000000000001},"min":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.29,"5min":1.456,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.6879999999999999,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":1.149}]},{"osd":7,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4750000000000001,"15min":1.514},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.788,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4770000000000001,"15min":1.522},"min":{"1min":1.0549999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.913,"5min":3.004,"15min":3.004},"last":1.1930000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48116,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323724,"statfs":{"total":96636764160,"available":96587493376,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.81100000000000005,"15min":0.87},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92100000000000004,"5min":1.411,"15min":1.4730000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98899999999999999,"15min":1.024},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.216,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.97299999999999998,"15min":1.0309999999999999},"min":{"1min":0.57899999999999996,"5min":0.378,"15min":0.378},"max":{"1min":1.512,"5min":1.512,"15min":1.698},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.04,"15min":1.113},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5229999999999999,"5min":1.635,"15min":1.6599999999999999},"last":1.006}]},{"osd":3,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.97499999999999998,"15min":1.022},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2889999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.097,"15min":1.181},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.4970000000000001,"5min":1.615,"15min":1.79},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2749999999999999,"15min":1.3120000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.621,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.276},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4730000000000001,"5min":1.714,"15min":1.835},"last":1.1299999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.3109999999999999,"15min":1.3680000000000001},"min":{"1min":0.86599999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.639,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.347,"15min":1.393},"min":{"1min":0.79000000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.573,"5min":1.8740000000000001,"15min":2.016},"last":1.23}]},{"osd":6,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.351,"15min":1.367},"min":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.865,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.651},{"interface":"front","average":{"1min":1.268,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.86599999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7090000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4370000000000001,"15min":1.4630000000000001},"min":{"1min":0.94299999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5740000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.464},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.4350000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.8100000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.762}]}]},{"osd":1,"up_from":13,"seq":55834575998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52792,"kb_used_data":3696,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319048,"statfs":{"total":96636764160,"available":96582705152,"internally_reserved":0,"allocated":3784704,"data_stored":2161339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82899999999999996,"15min":0.88800000000000001},"min":{"1min":0.38500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.5569999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.94399999999999995,"15min":1.0249999999999999},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.167,"5min":1.512,"15min":1.681},"last":1.0409999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.86299999999999999,"15min":0.92300000000000004},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97699999999999998,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.28},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0349999999999999,"15min":1.1220000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.476,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":1.3819999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.1359999999999999,"15min":1.268},"min":{"1min":0.52400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.788},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.046,"15min":1.1120000000000001},"min":{"1min":0.41399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.615,"5min":1.714,"15min":1.714},"last":1.1799999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.177,"15min":1.2090000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6000000000000001,"5min":1.623,"15min":1.6859999999999999},"last":1.091},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.329,"15min":1.391},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7450000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.482}]},{"osd":5,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2969999999999999,"15min":1.3240000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8169999999999999,"5min":1.9399999999999999,"15min":2.0379999999999998},"last":1.302},{"interface":"front","average":{"1min":1.2,"5min":1.2569999999999999,"15min":1.319},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.7470000000000001,"5min":1.927,"15min":1.927},"last":1.3480000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.373,"15min":1.429},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.885,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3759999999999999,"15min":1.4019999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.901,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.159}]},{"osd":7,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.431,"15min":1.5069999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.46,"15min":1.5009999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.954,"5min":2.1389999999999998,"15min":2.2069999999999999},"last":1.516}]}]},{"osd":0,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60416,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311424,"statfs":{"total":96636764160,"available":96574898176,"internally_reserved":0,"allocated":3203072,"data_stored":1577290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.40100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94499999999999995,"15min":1.04},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.0249999999999999,"15min":1.0660000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.48699999999999999},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.1429999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.0269999999999999,"15min":1.107},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6619999999999999,"5min":1.726,"15min":1.792},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.99099999999999999,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.306,"5min":1.536,"15min":1.8839999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.194,"5min":1.165,"15min":1.222},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.27,"15min":1.337},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.246,"15min":1.2949999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.536,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1060000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.839,"5min":1.899,"15min":2.7719999999999998},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.365,"15min":1.423},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.484,"5min":2.484,"15min":2.5880000000000001},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.327,"15min":1.371},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.835,"5min":1.835,"15min":2.2029999999999998},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.444,"15min":1.4810000000000001},"min":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":1.9239999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.351}]},{"osd":7,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.4419999999999999,"15min":1.506},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.411,"5min":2.411,"15min":2.7040000000000002},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.456,"15min":1.4770000000000001},"min":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.718,"5min":2.1190000000000002,"15min":2.431},"last":1.143}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:12:40.364 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:12:40.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.616+0000 7f66ad2ce640 1 Processor -- start 2023-12-01T03:12:40.618 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.616+0000 7f66ad2ce640 1 -- start start 2023-12-01T03:12:40.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.618+0000 7f66ad2ce640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f66a8159dd0 0x7f66a80c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.618+0000 7f66ad2ce640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f66a8070d30 0x7f66a80c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.618+0000 7f66ad2ce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a806f8b0 con 0x7f66a806b6f0 2023-12-01T03:12:40.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.618+0000 7f66ad2ce640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a80adcb0 con 0x7f66a8159dd0 2023-12-01T03:12:40.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.618+0000 7f66ad2ce640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a80ad9d0 con 0x7f66a8070d30 2023-12-01T03:12:40.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.618+0000 7f66a6ffd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f66a8159dd0 0x7f66a80c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.618+0000 7f669e7fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f66a806b6f0 0x7f66a80cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37732/0 (socket says 172.21.15.115:37732) 2023-12-01T03:12:40.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.618+0000 7f669e7fc640 1 -- 172.21.15.115:0/2592523789 learned_addr learned my addr 172.21.15.115:0/2592523789 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:40.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.619+0000 7f66a77fe640 1 --2- 172.21.15.115:0/2592523789 >> v2:172.21.15.142:3300/0 conn(0x7f66a8070d30 0x7f66a80c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.619+0000 7f66a6ffd640 1 -- 172.21.15.115:0/2592523789 >> v2:172.21.15.142:3300/0 conn(0x7f66a8070d30 msgr2=0x7f66a80c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:40.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.619+0000 7f66a6ffd640 1 --2- 172.21.15.115:0/2592523789 >> v2:172.21.15.142:3300/0 conn(0x7f66a8070d30 0x7f66a80c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.619+0000 7f66a6ffd640 1 -- 172.21.15.115:0/2592523789 >> v1:172.21.15.115:6789/0 conn(0x7f66a806b6f0 legacy=0x7f66a80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.619+0000 7f66a6ffd640 1 -- 172.21.15.115:0/2592523789 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a80c9000 con 0x7f66a8159dd0 2023-12-01T03:12:40.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.620+0000 7f66a6ffd640 1 --2- 172.21.15.115:0/2592523789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 0x7f66a80c97e0 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7f66a806f280 tx=0x7f669802b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40a7b384817fba4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:40.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.620+0000 7f66a67fc640 1 -- 172.21.15.115:0/2592523789 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698031520 con 0x7f66a8159dd0 2023-12-01T03:12:40.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.620+0000 7f66a67fc640 1 -- 172.21.15.115:0/2592523789 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6698031680 con 0x7f66a8159dd0 2023-12-01T03:12:40.622 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.620+0000 7f66a67fc640 1 -- 172.21.15.115:0/2592523789 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698031a10 con 0x7f66a8159dd0 2023-12-01T03:12:40.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.621+0000 7f66ad2ce640 1 -- 172.21.15.115:0/2592523789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 msgr2=0x7f66a80c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.621+0000 7f66ad2ce640 1 --2- 172.21.15.115:0/2592523789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 0x7f66a80c97e0 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7f66a806f280 tx=0x7f669802b7c0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.621+0000 7f66ad2ce640 1 -- 172.21.15.115:0/2592523789 shutdown_connections 2023-12-01T03:12:40.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.621+0000 7f66ad2ce640 1 --2- 172.21.15.115:0/2592523789 >> v2:172.21.15.142:3300/0 conn(0x7f66a8070d30 0x7f66a80c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.623 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.621+0000 7f66ad2ce640 1 --2- 172.21.15.115:0/2592523789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 0x7f66a80c97e0 unknown :-1 s=CLOSED pgs=2974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.621+0000 7f66ad2ce640 1 -- 172.21.15.115:0/2592523789 >> 172.21.15.115:0/2592523789 conn(0x7f66a80ce880 msgr2=0x7f66a80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:40.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.621+0000 7f66ad2ce640 1 -- 172.21.15.115:0/2592523789 shutdown_connections 2023-12-01T03:12:40.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.621+0000 7f66ad2ce640 1 -- 172.21.15.115:0/2592523789 wait complete. 2023-12-01T03:12:40.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.623+0000 7f66ad2ce640 1 Processor -- start 2023-12-01T03:12:40.624 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.623+0000 7f66ad2ce640 1 -- start start 2023-12-01T03:12:40.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.624+0000 7f66ad2ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a806b6f0 0x7f66a8167740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.624+0000 7f66ad2ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66a8070d30 0x7f66a81483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.624+0000 7f66ad2ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 0x7f66a80cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.624+0000 7f66ad2ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a815c720 con 0x7f66a8159dd0 2023-12-01T03:12:40.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.624+0000 7f66ad2ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a815ca00 con 0x7f66a806b6f0 2023-12-01T03:12:40.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.624+0000 7f66ad2ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a815cce0 con 0x7f66a8070d30 2023-12-01T03:12:40.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.624+0000 7f66a77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 0x7f66a80cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.624+0000 7f66a77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 0x7f66a80cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51816/0 (socket says 172.21.15.115:51816) 2023-12-01T03:12:40.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.624+0000 7f66a77fe640 1 -- 172.21.15.115:0/1280750846 learned_addr learned my addr 172.21.15.115:0/1280750846 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:40.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f66a6ffd640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a806b6f0 0x7f66a8167740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.627 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f66a77fe640 1 -- 172.21.15.115:0/1280750846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66a8070d30 msgr2=0x7f66a81483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f66a77fe640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66a8070d30 0x7f66a81483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f66a77fe640 1 -- 172.21.15.115:0/1280750846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a806b6f0 msgr2=0x7f66a8167740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f66a77fe640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a806b6f0 0x7f66a8167740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f66a77fe640 1 -- 172.21.15.115:0/1280750846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6698009400 con 0x7f66a8159dd0 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f66a77fe640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 0x7f66a80cd5f0 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c7f0 tx=0x7f66a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f669ffff640 1 -- 172.21.15.115:0/1280750846 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0004220 con 0x7f66a8159dd0 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f669ffff640 1 -- 172.21.15.115:0/1280750846 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a0004380 con 0x7f66a8159dd0 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f669ffff640 1 -- 172.21.15.115:0/1280750846 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0007860 con 0x7f66a8159dd0 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f66ad2ce640 1 -- 172.21.15.115:0/1280750846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a815cfc0 con 0x7f66a8159dd0 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.625+0000 7f66ad2ce640 1 -- 172.21.15.115:0/1280750846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a8148910 con 0x7f66a8159dd0 2023-12-01T03:12:40.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.626+0000 7f66ad2ce640 1 -- 172.21.15.115:0/1280750846 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a814cdb0 con 0x7f66a8159dd0 2023-12-01T03:12:40.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.627+0000 7f669ffff640 1 -- 172.21.15.115:0/1280750846 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f66a0007bd0 con 0x7f66a8159dd0 2023-12-01T03:12:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.629+0000 7f669ffff640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f668806a670 0x7f668806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:40.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.629+0000 7f669ffff640 1 -- 172.21.15.115:0/1280750846 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f66a0088e70 con 0x7f66a8159dd0 2023-12-01T03:12:40.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.629+0000 7f66a6ffd640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f668806a670 0x7f668806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:40.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.630+0000 7f66a6ffd640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f668806a670 0x7f668806cb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6698035000 tx=0x7f6698030000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:40.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.633+0000 7f669ffff640 1 -- 172.21.15.115:0/1280750846 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f66a0053fc0 con 0x7f66a8159dd0 2023-12-01T03:12:40.797 INFO:tasks.ceph.mds.b.smithi142.stderr:2023-12-01T03:12:40.796+0000 7fc99fb29640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:12:40.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.818+0000 7f66ad2ce640 1 -- 172.21.15.115:0/1280750846 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66a806d0d0 con 0x7f668806a670 2023-12-01T03:12:40.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.830+0000 7f669ffff640 1 -- 172.21.15.115:0/1280750846 <== mgr.11877 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284271 (secure 0 0 0) 0x7f66980319c0 con 0x7f668806a670 2023-12-01T03:12:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:40.837 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:12:40.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 -- 172.21.15.115:0/1280750846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f668806a670 msgr2=0x7f668806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f668806a670 0x7f668806cb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6698035000 tx=0x7f6698030000 comp rx=0 tx=0).stop 2023-12-01T03:12:40.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 -- 172.21.15.115:0/1280750846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 msgr2=0x7f66a80cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:40.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 0x7f66a80cd5f0 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c7f0 tx=0x7f66a000ccb0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 -- 172.21.15.115:0/1280750846 shutdown_connections 2023-12-01T03:12:40.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f668806a670 0x7f668806cb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66a8070d30 0x7f66a81483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66a8159dd0 0x7f66a80cd5f0 unknown :-1 s=CLOSED pgs=2975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 --2- 172.21.15.115:0/1280750846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66a806b6f0 0x7f66a8167740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:40.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 -- 172.21.15.115:0/1280750846 >> 172.21.15.115:0/1280750846 conn(0x7f66a80ce880 msgr2=0x7f66a814fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:40.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 -- 172.21.15.115:0/1280750846 shutdown_connections 2023-12-01T03:12:40.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:40.836+0000 7f66ad2ce640 1 -- 172.21.15.115:0/1280750846 wait complete. 2023-12-01T03:12:40.858 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-12-01T03:12:39.049245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":443456,"kb_used_data":26112,"kb_used_omap":0,"kb_used_meta":417280,"kb_avail":754531264,"statfs":{"total":773094113280,"available":772640014336,"internally_reserved":0,"allocated":26738688,"data_stored":13761142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":427294720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.236540"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851842+0000","last_change":"2023-12-01T03:12:34.851842+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851842+0000","last_undegraded":"2023-12-01T03:12:34.851842+0000","last_fullsized":"2023-12-01T03:12:34.851842+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:21:23.692942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849451+0000","last_change":"2023-12-01T03:12:34.849451+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849451+0000","last_undegraded":"2023-12-01T03:12:34.849451+0000","last_fullsized":"2023-12-01T03:12:34.849451+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845514+0000","last_change":"2023-12-01T03:12:34.845514+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845514+0000","last_undegraded":"2023-12-01T03:12:34.845514+0000","last_fullsized":"2023-12-01T03:12:34.845514+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850690+0000","last_change":"2023-12-01T03:12:34.850690+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850690+0000","last_undegraded":"2023-12-01T03:12:34.850690+0000","last_fullsized":"2023-12-01T03:12:34.850690+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:49.741119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850902+0000","last_change":"2023-12-01T03:12:34.850902+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850902+0000","last_undegraded":"2023-12-01T03:12:34.850902+0000","last_fullsized":"2023-12-01T03:12:34.850902+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.848923+0000","last_change":"2023-12-01T03:12:34.848923+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.848923+0000","last_undegraded":"2023-12-01T03:12:34.848923+0000","last_fullsized":"2023-12-01T03:12:34.848923+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:04.481100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.855117+0000","last_change":"2023-12-01T03:12:34.855117+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.855117+0000","last_undegraded":"2023-12-01T03:12:34.855117+0000","last_fullsized":"2023-12-01T03:12:34.855117+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846440+0000","last_change":"2023-12-01T03:12:34.846440+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846440+0000","last_undegraded":"2023-12-01T03:12:34.846440+0000","last_fullsized":"2023-12-01T03:12:34.846440+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:27.055115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.855652+0000","last_change":"2023-12-01T03:12:34.855652+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.855652+0000","last_undegraded":"2023-12-01T03:12:34.855652+0000","last_fullsized":"2023-12-01T03:12:34.855652+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:17:30.864087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.848420+0000","last_change":"2023-12-01T03:12:34.848420+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.848420+0000","last_undegraded":"2023-12-01T03:12:34.848420+0000","last_fullsized":"2023-12-01T03:12:34.848420+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852653+0000","last_change":"2023-12-01T03:12:34.852653+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852653+0000","last_undegraded":"2023-12-01T03:12:34.852653+0000","last_fullsized":"2023-12-01T03:12:34.852653+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845433+0000","last_change":"2023-12-01T03:12:34.845433+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845433+0000","last_undegraded":"2023-12-01T03:12:34.845433+0000","last_fullsized":"2023-12-01T03:12:34.845433+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852346+0000","last_change":"2023-12-01T03:12:34.852346+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852346+0000","last_undegraded":"2023-12-01T03:12:34.852346+0000","last_fullsized":"2023-12-01T03:12:34.852346+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:54:17.936247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854378+0000","last_change":"2023-12-01T03:12:34.854378+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854378+0000","last_undegraded":"2023-12-01T03:12:34.854378+0000","last_fullsized":"2023-12-01T03:12:34.854378+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:59:25.105386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854306+0000","last_change":"2023-12-01T03:12:34.854306+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854306+0000","last_undegraded":"2023-12-01T03:12:34.854306+0000","last_fullsized":"2023-12-01T03:12:34.854306+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845305+0000","last_change":"2023-12-01T03:12:34.845305+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845305+0000","last_undegraded":"2023-12-01T03:12:34.845305+0000","last_fullsized":"2023-12-01T03:12:34.845305+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.853111+0000","last_change":"2023-12-01T03:12:34.853111+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.853111+0000","last_undegraded":"2023-12-01T03:12:34.853111+0000","last_fullsized":"2023-12-01T03:12:34.853111+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:21.877813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846554+0000","last_change":"2023-12-01T03:12:34.846554+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846554+0000","last_undegraded":"2023-12-01T03:12:34.846554+0000","last_fullsized":"2023-12-01T03:12:34.846554+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844698+0000","last_change":"2023-12-01T03:12:34.844698+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844698+0000","last_undegraded":"2023-12-01T03:12:34.844698+0000","last_fullsized":"2023-12-01T03:12:34.844698+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846725+0000","last_change":"2023-12-01T03:12:34.846725+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846725+0000","last_undegraded":"2023-12-01T03:12:34.846725+0000","last_fullsized":"2023-12-01T03:12:34.846725+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845156+0000","last_change":"2023-12-01T03:12:34.845156+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845156+0000","last_undegraded":"2023-12-01T03:12:34.845156+0000","last_fullsized":"2023-12-01T03:12:34.845156+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:31.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846691+0000","last_change":"2023-12-01T03:12:34.846691+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846691+0000","last_undegraded":"2023-12-01T03:12:34.846691+0000","last_fullsized":"2023-12-01T03:12:34.846691+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:59:04.995090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.843637+0000","last_change":"2023-12-01T03:12:34.843637+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.843637+0000","last_undegraded":"2023-12-01T03:12:34.843637+0000","last_fullsized":"2023-12-01T03:12:34.843637+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844221+0000","last_change":"2023-12-01T03:12:34.844221+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844221+0000","last_undegraded":"2023-12-01T03:12:34.844221+0000","last_fullsized":"2023-12-01T03:12:34.844221+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:00:57.853346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.855001+0000","last_change":"2023-12-01T03:12:34.855001+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.855001+0000","last_undegraded":"2023-12-01T03:12:34.855001+0000","last_fullsized":"2023-12-01T03:12:34.855001+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:34.404254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850115+0000","last_change":"2023-12-01T03:12:34.850115+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850115+0000","last_undegraded":"2023-12-01T03:12:34.850115+0000","last_fullsized":"2023-12-01T03:12:34.850115+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:00.577497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846472+0000","last_change":"2023-12-01T03:12:34.846472+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846472+0000","last_undegraded":"2023-12-01T03:12:34.846472+0000","last_fullsized":"2023-12-01T03:12:34.846472+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:09:42.224008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850472+0000","last_change":"2023-12-01T03:12:34.850472+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850472+0000","last_undegraded":"2023-12-01T03:12:34.850472+0000","last_fullsized":"2023-12-01T03:12:34.850472+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:53:08.535342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.847309+0000","last_change":"2023-12-01T03:12:34.847309+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.847309+0000","last_undegraded":"2023-12-01T03:12:34.847309+0000","last_fullsized":"2023-12-01T03:12:34.847309+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:50:37.148015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.853876+0000","last_change":"2023-12-01T03:12:34.853876+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.853876+0000","last_undegraded":"2023-12-01T03:12:34.853876+0000","last_fullsized":"2023-12-01T03:12:34.853876+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:25.025253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854643+0000","last_change":"2023-12-01T03:12:34.854643+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854643+0000","last_undegraded":"2023-12-01T03:12:34.854643+0000","last_fullsized":"2023-12-01T03:12:34.854643+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849276+0000","last_change":"2023-12-01T03:12:34.849276+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849276+0000","last_undegraded":"2023-12-01T03:12:34.849276+0000","last_fullsized":"2023-12-01T03:12:34.849276+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:10:45.146806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.869182+0000","last_change":"2023-12-01T03:12:34.867899+0000","last_active":"2023-12-01T03:12:34.869182+0000","last_peered":"2023-12-01T03:12:34.869182+0000","last_clean":"2023-12-01T03:12:34.869182+0000","last_became_active":"2023-12-01T03:12:34.867559+0000","last_became_peered":"2023-12-01T03:12:34.867559+0000","last_unstale":"2023-12-01T03:12:34.869182+0000","last_undegraded":"2023-12-01T03:12:34.869182+0000","last_fullsized":"2023-12-01T03:12:34.869182+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:49.066648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852053+0000","last_change":"2023-12-01T03:12:34.852053+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852053+0000","last_undegraded":"2023-12-01T03:12:34.852053+0000","last_fullsized":"2023-12-01T03:12:34.852053+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:59.927209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.869257+0000","last_change":"2023-12-01T03:12:34.868270+0000","last_active":"2023-12-01T03:12:34.869257+0000","last_peered":"2023-12-01T03:12:34.869257+0000","last_clean":"2023-12-01T03:12:34.869257+0000","last_became_active":"2023-12-01T03:12:34.867687+0000","last_became_peered":"2023-12-01T03:12:34.867687+0000","last_unstale":"2023-12-01T03:12:34.869257+0000","last_undegraded":"2023-12-01T03:12:34.869257+0000","last_fullsized":"2023-12-01T03:12:34.869257+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:03:51.996656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844945+0000","last_change":"2023-12-01T03:12:34.844945+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844945+0000","last_undegraded":"2023-12-01T03:12:34.844945+0000","last_fullsized":"2023-12-01T03:12:34.844945+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.870274+0000","last_change":"2023-12-01T03:12:34.869563+0000","last_active":"2023-12-01T03:12:34.870274+0000","last_peered":"2023-12-01T03:12:34.870274+0000","last_clean":"2023-12-01T03:12:34.870274+0000","last_became_active":"2023-12-01T03:12:34.869160+0000","last_became_peered":"2023-12-01T03:12:34.869160+0000","last_unstale":"2023-12-01T03:12:34.870274+0000","last_undegraded":"2023-12-01T03:12:34.870274+0000","last_fullsized":"2023-12-01T03:12:34.870274+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:19.315026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852093+0000","last_change":"2023-12-01T03:12:34.852093+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852093+0000","last_undegraded":"2023-12-01T03:12:34.852093+0000","last_fullsized":"2023-12-01T03:12:34.852093+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867132+0000","last_change":"2023-12-01T03:12:34.865447+0000","last_active":"2023-12-01T03:12:34.867132+0000","last_peered":"2023-12-01T03:12:34.867132+0000","last_clean":"2023-12-01T03:12:34.867132+0000","last_became_active":"2023-12-01T03:12:34.865092+0000","last_became_peered":"2023-12-01T03:12:34.865092+0000","last_unstale":"2023-12-01T03:12:34.867132+0000","last_undegraded":"2023-12-01T03:12:34.867132+0000","last_fullsized":"2023-12-01T03:12:34.867132+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:55:07.934979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850269+0000","last_change":"2023-12-01T03:12:34.850269+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850269+0000","last_undegraded":"2023-12-01T03:12:34.850269+0000","last_fullsized":"2023-12-01T03:12:34.850269+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864777+0000","last_change":"2023-12-01T03:12:34.857676+0000","last_active":"2023-12-01T03:12:34.864777+0000","last_peered":"2023-12-01T03:12:34.864777+0000","last_clean":"2023-12-01T03:12:34.864777+0000","last_became_active":"2023-12-01T03:12:34.857365+0000","last_became_peered":"2023-12-01T03:12:34.857365+0000","last_unstale":"2023-12-01T03:12:34.864777+0000","last_undegraded":"2023-12-01T03:12:34.864777+0000","last_fullsized":"2023-12-01T03:12:34.864777+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:22.088313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844881+0000","last_change":"2023-12-01T03:12:34.844881+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844881+0000","last_undegraded":"2023-12-01T03:12:34.844881+0000","last_fullsized":"2023-12-01T03:12:34.844881+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:13.355768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863077+0000","last_change":"2023-12-01T03:12:34.860204+0000","last_active":"2023-12-01T03:12:34.863077+0000","last_peered":"2023-12-01T03:12:34.863077+0000","last_clean":"2023-12-01T03:12:34.863077+0000","last_became_active":"2023-12-01T03:12:34.858034+0000","last_became_peered":"2023-12-01T03:12:34.858034+0000","last_unstale":"2023-12-01T03:12:34.863077+0000","last_undegraded":"2023-12-01T03:12:34.863077+0000","last_fullsized":"2023-12-01T03:12:34.863077+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:23:38.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852657+0000","last_change":"2023-12-01T03:12:34.852657+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852657+0000","last_undegraded":"2023-12-01T03:12:34.852657+0000","last_fullsized":"2023-12-01T03:12:34.852657+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864589+0000","last_change":"2023-12-01T03:12:34.863171+0000","last_active":"2023-12-01T03:12:34.864589+0000","last_peered":"2023-12-01T03:12:34.864589+0000","last_clean":"2023-12-01T03:12:34.864589+0000","last_became_active":"2023-12-01T03:12:34.862887+0000","last_became_peered":"2023-12-01T03:12:34.862887+0000","last_unstale":"2023-12-01T03:12:34.864589+0000","last_undegraded":"2023-12-01T03:12:34.864589+0000","last_fullsized":"2023-12-01T03:12:34.864589+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:51:11.709287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.854072+0000","last_change":"2023-12-01T03:12:34.854072+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.854072+0000","last_undegraded":"2023-12-01T03:12:34.854072+0000","last_fullsized":"2023-12-01T03:12:34.854072+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.865690+0000","last_change":"2023-12-01T03:12:34.864288+0000","last_active":"2023-12-01T03:12:34.865690+0000","last_peered":"2023-12-01T03:12:34.865690+0000","last_clean":"2023-12-01T03:12:34.865690+0000","last_became_active":"2023-12-01T03:12:34.863960+0000","last_became_peered":"2023-12-01T03:12:34.863960+0000","last_unstale":"2023-12-01T03:12:34.865690+0000","last_undegraded":"2023-12-01T03:12:34.865690+0000","last_fullsized":"2023-12-01T03:12:34.865690+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:02.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.850169+0000","last_change":"2023-12-01T03:12:34.850169+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.850169+0000","last_undegraded":"2023-12-01T03:12:34.850169+0000","last_fullsized":"2023-12-01T03:12:34.850169+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:19.476510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868695+0000","last_change":"2023-12-01T03:12:34.867539+0000","last_active":"2023-12-01T03:12:34.868695+0000","last_peered":"2023-12-01T03:12:34.868695+0000","last_clean":"2023-12-01T03:12:34.868695+0000","last_became_active":"2023-12-01T03:12:34.866362+0000","last_became_peered":"2023-12-01T03:12:34.866362+0000","last_unstale":"2023-12-01T03:12:34.868695+0000","last_undegraded":"2023-12-01T03:12:34.868695+0000","last_fullsized":"2023-12-01T03:12:34.868695+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:52.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852349+0000","last_change":"2023-12-01T03:12:34.852349+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852349+0000","last_undegraded":"2023-12-01T03:12:34.852349+0000","last_fullsized":"2023-12-01T03:12:34.852349+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.862751+0000","last_change":"2023-12-01T03:12:34.859136+0000","last_active":"2023-12-01T03:12:34.862751+0000","last_peered":"2023-12-01T03:12:34.862751+0000","last_clean":"2023-12-01T03:12:34.862751+0000","last_became_active":"2023-12-01T03:12:34.857840+0000","last_became_peered":"2023-12-01T03:12:34.857840+0000","last_unstale":"2023-12-01T03:12:34.862751+0000","last_undegraded":"2023-12-01T03:12:34.862751+0000","last_fullsized":"2023-12-01T03:12:34.862751+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:36.726453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849661+0000","last_change":"2023-12-01T03:12:34.849661+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849661+0000","last_undegraded":"2023-12-01T03:12:34.849661+0000","last_fullsized":"2023-12-01T03:12:34.849661+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863076+0000","last_change":"2023-12-01T03:12:34.860533+0000","last_active":"2023-12-01T03:12:34.863076+0000","last_peered":"2023-12-01T03:12:34.863076+0000","last_clean":"2023-12-01T03:12:34.863076+0000","last_became_active":"2023-12-01T03:12:34.858865+0000","last_became_peered":"2023-12-01T03:12:34.858865+0000","last_unstale":"2023-12-01T03:12:34.863076+0000","last_undegraded":"2023-12-01T03:12:34.863076+0000","last_fullsized":"2023-12-01T03:12:34.863076+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:22.088313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849924+0000","last_change":"2023-12-01T03:12:34.849924+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849924+0000","last_undegraded":"2023-12-01T03:12:34.849924+0000","last_fullsized":"2023-12-01T03:12:34.849924+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.859521+0000","last_change":"2023-12-01T03:12:34.857964+0000","last_active":"2023-12-01T03:12:34.859521+0000","last_peered":"2023-12-01T03:12:34.859521+0000","last_clean":"2023-12-01T03:12:34.859521+0000","last_became_active":"2023-12-01T03:12:34.856582+0000","last_became_peered":"2023-12-01T03:12:34.856582+0000","last_unstale":"2023-12-01T03:12:34.859521+0000","last_undegraded":"2023-12-01T03:12:34.859521+0000","last_fullsized":"2023-12-01T03:12:34.859521+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:48.550869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.853000+0000","last_change":"2023-12-01T03:12:34.853000+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.853000+0000","last_undegraded":"2023-12-01T03:12:34.853000+0000","last_fullsized":"2023-12-01T03:12:34.853000+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:23.725637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864341+0000","last_change":"2023-12-01T03:12:34.863097+0000","last_active":"2023-12-01T03:12:34.864341+0000","last_peered":"2023-12-01T03:12:34.864341+0000","last_clean":"2023-12-01T03:12:34.864341+0000","last_became_active":"2023-12-01T03:12:34.862657+0000","last_became_peered":"2023-12-01T03:12:34.862657+0000","last_unstale":"2023-12-01T03:12:34.864341+0000","last_undegraded":"2023-12-01T03:12:34.864341+0000","last_fullsized":"2023-12-01T03:12:34.864341+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:22.709559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845680+0000","last_change":"2023-12-01T03:12:34.845680+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845680+0000","last_undegraded":"2023-12-01T03:12:34.845680+0000","last_fullsized":"2023-12-01T03:12:34.845680+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863230+0000","last_change":"2023-12-01T03:12:34.860804+0000","last_active":"2023-12-01T03:12:34.863230+0000","last_peered":"2023-12-01T03:12:34.863230+0000","last_clean":"2023-12-01T03:12:34.863230+0000","last_became_active":"2023-12-01T03:12:34.859477+0000","last_became_peered":"2023-12-01T03:12:34.859477+0000","last_unstale":"2023-12-01T03:12:34.863230+0000","last_undegraded":"2023-12-01T03:12:34.863230+0000","last_fullsized":"2023-12-01T03:12:34.863230+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:00.153556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852165+0000","last_change":"2023-12-01T03:12:34.852165+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852165+0000","last_undegraded":"2023-12-01T03:12:34.852165+0000","last_fullsized":"2023-12-01T03:12:34.852165+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:04:09.892305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863339+0000","last_change":"2023-12-01T03:12:34.860578+0000","last_active":"2023-12-01T03:12:34.863339+0000","last_peered":"2023-12-01T03:12:34.863339+0000","last_clean":"2023-12-01T03:12:34.863339+0000","last_became_active":"2023-12-01T03:12:34.859107+0000","last_became_peered":"2023-12-01T03:12:34.859107+0000","last_unstale":"2023-12-01T03:12:34.863339+0000","last_undegraded":"2023-12-01T03:12:34.863339+0000","last_fullsized":"2023-12-01T03:12:34.863339+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:23:38.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846877+0000","last_change":"2023-12-01T03:12:34.846877+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846877+0000","last_undegraded":"2023-12-01T03:12:34.846877+0000","last_fullsized":"2023-12-01T03:12:34.846877+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868029+0000","last_change":"2023-12-01T03:12:34.866244+0000","last_active":"2023-12-01T03:12:34.868029+0000","last_peered":"2023-12-01T03:12:34.868029+0000","last_clean":"2023-12-01T03:12:34.868029+0000","last_became_active":"2023-12-01T03:12:34.865944+0000","last_became_peered":"2023-12-01T03:12:34.865944+0000","last_unstale":"2023-12-01T03:12:34.868029+0000","last_undegraded":"2023-12-01T03:12:34.868029+0000","last_fullsized":"2023-12-01T03:12:34.868029+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:28.385262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.847970+0000","last_change":"2023-12-01T03:12:34.847970+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.847970+0000","last_undegraded":"2023-12-01T03:12:34.847970+0000","last_fullsized":"2023-12-01T03:12:34.847970+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.866062+0000","last_change":"2023-12-01T03:12:34.864936+0000","last_active":"2023-12-01T03:12:34.866062+0000","last_peered":"2023-12-01T03:12:34.866062+0000","last_clean":"2023-12-01T03:12:34.866062+0000","last_became_active":"2023-12-01T03:12:34.864646+0000","last_became_peered":"2023-12-01T03:12:34.864646+0000","last_unstale":"2023-12-01T03:12:34.866062+0000","last_undegraded":"2023-12-01T03:12:34.866062+0000","last_fullsized":"2023-12-01T03:12:34.866062+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:00.153556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846620+0000","last_change":"2023-12-01T03:12:34.846620+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846620+0000","last_undegraded":"2023-12-01T03:12:34.846620+0000","last_fullsized":"2023-12-01T03:12:34.846620+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868225+0000","last_change":"2023-12-01T03:12:34.866465+0000","last_active":"2023-12-01T03:12:34.868225+0000","last_peered":"2023-12-01T03:12:34.868225+0000","last_clean":"2023-12-01T03:12:34.868225+0000","last_became_active":"2023-12-01T03:12:34.865451+0000","last_became_peered":"2023-12-01T03:12:34.865451+0000","last_unstale":"2023-12-01T03:12:34.868225+0000","last_undegraded":"2023-12-01T03:12:34.868225+0000","last_fullsized":"2023-12-01T03:12:34.868225+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:17:29.848009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.852931+0000","last_change":"2023-12-01T03:12:34.852931+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.852931+0000","last_undegraded":"2023-12-01T03:12:34.852931+0000","last_fullsized":"2023-12-01T03:12:34.852931+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:03:37.465541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867729+0000","last_change":"2023-12-01T03:12:34.866633+0000","last_active":"2023-12-01T03:12:34.867729+0000","last_peered":"2023-12-01T03:12:34.867729+0000","last_clean":"2023-12-01T03:12:34.867729+0000","last_became_active":"2023-12-01T03:12:34.865565+0000","last_became_peered":"2023-12-01T03:12:34.865565+0000","last_unstale":"2023-12-01T03:12:34.867729+0000","last_undegraded":"2023-12-01T03:12:34.867729+0000","last_fullsized":"2023-12-01T03:12:34.867729+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:37.392214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851102+0000","last_change":"2023-12-01T03:12:34.851102+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851102+0000","last_undegraded":"2023-12-01T03:12:34.851102+0000","last_fullsized":"2023-12-01T03:12:34.851102+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.862760+0000","last_change":"2023-12-01T03:12:34.858480+0000","last_active":"2023-12-01T03:12:34.862760+0000","last_peered":"2023-12-01T03:12:34.862760+0000","last_clean":"2023-12-01T03:12:34.862760+0000","last_became_active":"2023-12-01T03:12:34.857292+0000","last_became_peered":"2023-12-01T03:12:34.857292+0000","last_unstale":"2023-12-01T03:12:34.862760+0000","last_undegraded":"2023-12-01T03:12:34.862760+0000","last_fullsized":"2023-12-01T03:12:34.862760+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:02.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.848035+0000","last_change":"2023-12-01T03:12:34.848035+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.848035+0000","last_undegraded":"2023-12-01T03:12:34.848035+0000","last_fullsized":"2023-12-01T03:12:34.848035+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863422+0000","last_change":"2023-12-01T03:12:34.860490+0000","last_active":"2023-12-01T03:12:34.863422+0000","last_peered":"2023-12-01T03:12:34.863422+0000","last_clean":"2023-12-01T03:12:34.863422+0000","last_became_active":"2023-12-01T03:12:34.860228+0000","last_became_peered":"2023-12-01T03:12:34.860228+0000","last_unstale":"2023-12-01T03:12:34.863422+0000","last_undegraded":"2023-12-01T03:12:34.863422+0000","last_fullsized":"2023-12-01T03:12:34.863422+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:54:16.920168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846992+0000","last_change":"2023-12-01T03:12:34.846992+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846992+0000","last_undegraded":"2023-12-01T03:12:34.846992+0000","last_fullsized":"2023-12-01T03:12:34.846992+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.866877+0000","last_change":"2023-12-01T03:12:34.864610+0000","last_active":"2023-12-01T03:12:34.866877+0000","last_peered":"2023-12-01T03:12:34.866877+0000","last_clean":"2023-12-01T03:12:34.866877+0000","last_became_active":"2023-12-01T03:12:34.864320+0000","last_became_peered":"2023-12-01T03:12:34.864320+0000","last_unstale":"2023-12-01T03:12:34.866877+0000","last_undegraded":"2023-12-01T03:12:34.866877+0000","last_fullsized":"2023-12-01T03:12:34.866877+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:41.361208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.843616+0000","last_change":"2023-12-01T03:12:34.843616+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.843616+0000","last_undegraded":"2023-12-01T03:12:34.843616+0000","last_fullsized":"2023-12-01T03:12:34.843616+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867104+0000","last_change":"2023-12-01T03:12:34.864988+0000","last_active":"2023-12-01T03:12:34.867104+0000","last_peered":"2023-12-01T03:12:34.867104+0000","last_clean":"2023-12-01T03:12:34.867104+0000","last_became_active":"2023-12-01T03:12:34.864626+0000","last_became_peered":"2023-12-01T03:12:34.864626+0000","last_unstale":"2023-12-01T03:12:34.867104+0000","last_undegraded":"2023-12-01T03:12:34.867104+0000","last_fullsized":"2023-12-01T03:12:34.867104+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:35.500459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845514+0000","last_change":"2023-12-01T03:12:34.845514+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845514+0000","last_undegraded":"2023-12-01T03:12:34.845514+0000","last_fullsized":"2023-12-01T03:12:34.845514+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.862837+0000","last_change":"2023-12-01T03:12:34.859322+0000","last_active":"2023-12-01T03:12:34.862837+0000","last_peered":"2023-12-01T03:12:34.862837+0000","last_clean":"2023-12-01T03:12:34.862837+0000","last_became_active":"2023-12-01T03:12:34.859042+0000","last_became_peered":"2023-12-01T03:12:34.859042+0000","last_unstale":"2023-12-01T03:12:34.862837+0000","last_undegraded":"2023-12-01T03:12:34.862837+0000","last_fullsized":"2023-12-01T03:12:34.862837+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:08.980764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.846826+0000","last_change":"2023-12-01T03:12:34.846826+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.846826+0000","last_undegraded":"2023-12-01T03:12:34.846826+0000","last_fullsized":"2023-12-01T03:12:34.846826+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:55:08.951057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863024+0000","last_change":"2023-12-01T03:12:34.860198+0000","last_active":"2023-12-01T03:12:34.863024+0000","last_peered":"2023-12-01T03:12:34.863024+0000","last_clean":"2023-12-01T03:12:34.863024+0000","last_became_active":"2023-12-01T03:12:34.859894+0000","last_became_peered":"2023-12-01T03:12:34.859894+0000","last_unstale":"2023-12-01T03:12:34.863024+0000","last_undegraded":"2023-12-01T03:12:34.863024+0000","last_fullsized":"2023-12-01T03:12:34.863024+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:36.882914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851163+0000","last_change":"2023-12-01T03:12:34.851163+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851163+0000","last_undegraded":"2023-12-01T03:12:34.851163+0000","last_fullsized":"2023-12-01T03:12:34.851163+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:20.331105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868997+0000","last_change":"2023-12-01T03:12:34.868249+0000","last_active":"2023-12-01T03:12:34.868997+0000","last_peered":"2023-12-01T03:12:34.868997+0000","last_clean":"2023-12-01T03:12:34.868997+0000","last_became_active":"2023-12-01T03:12:34.867861+0000","last_became_peered":"2023-12-01T03:12:34.867861+0000","last_unstale":"2023-12-01T03:12:34.868997+0000","last_undegraded":"2023-12-01T03:12:34.868997+0000","last_fullsized":"2023-12-01T03:12:34.868997+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:00.274064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.844493+0000","last_change":"2023-12-01T03:12:34.844493+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.844493+0000","last_undegraded":"2023-12-01T03:12:34.844493+0000","last_fullsized":"2023-12-01T03:12:34.844493+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.865132+0000","last_change":"2023-12-01T03:12:34.857807+0000","last_active":"2023-12-01T03:12:34.865132+0000","last_peered":"2023-12-01T03:12:34.865132+0000","last_clean":"2023-12-01T03:12:34.865132+0000","last_became_active":"2023-12-01T03:12:34.857501+0000","last_became_peered":"2023-12-01T03:12:34.857501+0000","last_unstale":"2023-12-01T03:12:34.865132+0000","last_undegraded":"2023-12-01T03:12:34.865132+0000","last_fullsized":"2023-12-01T03:12:34.865132+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:36.882914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845661+0000","last_change":"2023-12-01T03:12:34.845661+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845661+0000","last_undegraded":"2023-12-01T03:12:34.845661+0000","last_fullsized":"2023-12-01T03:12:34.845661+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:45.819776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.867080+0000","last_change":"2023-12-01T03:12:34.865848+0000","last_active":"2023-12-01T03:12:34.867080+0000","last_peered":"2023-12-01T03:12:34.867080+0000","last_clean":"2023-12-01T03:12:34.867080+0000","last_became_active":"2023-12-01T03:12:34.865157+0000","last_became_peered":"2023-12-01T03:12:34.865157+0000","last_unstale":"2023-12-01T03:12:34.867080+0000","last_undegraded":"2023-12-01T03:12:34.867080+0000","last_fullsized":"2023-12-01T03:12:34.867080+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:10.788196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849450+0000","last_change":"2023-12-01T03:12:34.849450+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849450+0000","last_undegraded":"2023-12-01T03:12:34.849450+0000","last_fullsized":"2023-12-01T03:12:34.849450+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:01.535641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.870973+0000","last_change":"2023-12-01T03:12:34.870482+0000","last_active":"2023-12-01T03:12:34.870973+0000","last_peered":"2023-12-01T03:12:34.870973+0000","last_clean":"2023-12-01T03:12:34.870973+0000","last_became_active":"2023-12-01T03:12:34.870111+0000","last_became_peered":"2023-12-01T03:12:34.870111+0000","last_unstale":"2023-12-01T03:12:34.870973+0000","last_undegraded":"2023-12-01T03:12:34.870973+0000","last_fullsized":"2023-12-01T03:12:34.870973+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:24.009174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.843521+0000","last_change":"2023-12-01T03:12:34.843521+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.843521+0000","last_undegraded":"2023-12-01T03:12:34.843521+0000","last_fullsized":"2023-12-01T03:12:34.843521+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.864342+0000","last_change":"2023-12-01T03:12:34.863164+0000","last_active":"2023-12-01T03:12:34.864342+0000","last_peered":"2023-12-01T03:12:34.864342+0000","last_clean":"2023-12-01T03:12:34.864342+0000","last_became_active":"2023-12-01T03:12:34.862454+0000","last_became_peered":"2023-12-01T03:12:34.862454+0000","last_unstale":"2023-12-01T03:12:34.864342+0000","last_undegraded":"2023-12-01T03:12:34.864342+0000","last_fullsized":"2023-12-01T03:12:34.864342+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:12.944465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.851703+0000","last_change":"2023-12-01T03:12:34.851703+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.851703+0000","last_undegraded":"2023-12-01T03:12:34.851703+0000","last_fullsized":"2023-12-01T03:12:34.851703+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.863195+0000","last_change":"2023-12-01T03:12:34.860753+0000","last_active":"2023-12-01T03:12:34.863195+0000","last_peered":"2023-12-01T03:12:34.863195+0000","last_clean":"2023-12-01T03:12:34.863195+0000","last_became_active":"2023-12-01T03:12:34.858035+0000","last_became_peered":"2023-12-01T03:12:34.858035+0000","last_unstale":"2023-12-01T03:12:34.863195+0000","last_undegraded":"2023-12-01T03:12:34.863195+0000","last_fullsized":"2023-12-01T03:12:34.863195+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:40.965606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.845635+0000","last_change":"2023-12-01T03:12:34.845635+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.845635+0000","last_undegraded":"2023-12-01T03:12:34.845635+0000","last_fullsized":"2023-12-01T03:12:34.845635+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:08:34.191802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.868184+0000","last_change":"2023-12-01T03:12:34.866607+0000","last_active":"2023-12-01T03:12:34.868184+0000","last_peered":"2023-12-01T03:12:34.868184+0000","last_clean":"2023-12-01T03:12:34.868184+0000","last_became_active":"2023-12-01T03:12:34.865003+0000","last_became_peered":"2023-12-01T03:12:34.865003+0000","last_unstale":"2023-12-01T03:12:34.868184+0000","last_undegraded":"2023-12-01T03:12:34.868184+0000","last_fullsized":"2023-12-01T03:12:34.868184+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:29:11.426576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":1,"reported_epoch":254,"state":"creating+peering","last_fresh":"2023-12-01T03:12:34.849489+0000","last_change":"2023-12-01T03:12:34.849489+0000","last_active":"2023-12-01T03:12:34.812524+0000","last_peered":"2023-12-01T03:12:34.812524+0000","last_clean":"2023-12-01T03:12:34.812524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-01T03:12:34.849489+0000","last_undegraded":"2023-12-01T03:12:34.849489+0000","last_fullsized":"2023-12-01T03:12:34.849489+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"241'359","reported_seq":1270,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-01T03:12:34.842152+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:12:34.842152+0000","last_peered":"2023-12-01T03:12:34.842152+0000","last_clean":"2023-12-01T03:12:34.842152+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:12:34.842152+0000","last_undegraded":"2023-12-01T03:12:34.842152+0000","last_fullsized":"2023-12-01T03:12:34.842152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60404,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311436,"statfs":{"total":96636764160,"available":96574910464,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0620000000000001,"15min":1.0760000000000001},"min":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.05},{"interface":"front","average":{"1min":1.343,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":1.169}]},{"osd":1,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":0.83999999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":1.399},{"interface":"front","average":{"1min":1.363,"5min":1.3069999999999999,"15min":1.327},"min":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.2849999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.167},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.95,"5min":1.95,"15min":2.1179999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.254,"15min":1.264},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.216}]},{"osd":3,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2729999999999999,"15min":1.302},"min":{"1min":0.78400000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.3959999999999999,"15min":1.425},"min":{"1min":0.84599999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":0.879}]},{"osd":4,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3140000000000001,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7989999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.341},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5029999999999999,"15min":1.508},"min":{"1min":1.1599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":1.3149999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.337,"15min":1.327},"min":{"1min":0.79600000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.369},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.5840000000000001,"15min":1.597},"min":{"1min":1.2430000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.8380000000000001,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":1.4630000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.476,"15min":1.454},"min":{"1min":0.89400000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.8579999999999997,"5min":4.8579999999999997,"15min":4.8579999999999997},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4319999999999999,"15min":1.4550000000000001},"min":{"1min":0.81799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":2.762},"last":1.113}]}]},{"osd":6,"up_from":13,"seq":55834576001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60416,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311424,"statfs":{"total":96636764160,"available":96574898176,"internally_reserved":0,"allocated":3203072,"data_stored":1577290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.036,"15min":1.0980000000000001},"min":{"1min":0.53600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4810000000000001,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.093,"15min":1.1519999999999999},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6439999999999999,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.373}]},{"osd":1,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.127,"15min":1.202},"min":{"1min":0.57399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7010000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.542,"5min":1.694,"15min":1.835},"last":1.0920000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5369999999999999,"5min":1.742,"15min":1.976},"last":1.51},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.272},"min":{"1min":0.71599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.637}]},{"osd":3,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.262,"15min":1.306},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7929999999999999,"5min":1.927,"15min":1.927},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.377,"15min":1.415},"min":{"1min":0.68700000000000006,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.1739999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2809999999999999,"15min":1.27},"min":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.661,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.4199999999999999,"15min":1.3660000000000001},"min":{"1min":0.64100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1019999999999999,"5min":2.48,"15min":2.48},"last":1.2210000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3859999999999999,"15min":1.407},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.651,"5min":2.262,"15min":2.262},"last":1.8200000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.518,"15min":1.492},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0030000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.3360000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5069999999999999,"15min":1.526},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.865,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.905},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0539999999999998,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.2709999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52792,"kb_used_data":3696,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319048,"statfs":{"total":96636764160,"available":96582705152,"internally_reserved":0,"allocated":3784704,"data_stored":2161339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0840000000000001,"15min":1.119},"min":{"1min":0.68200000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.476,"5min":1.776,"15min":1.776},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.996,"5min":1.133,"15min":1.151},"min":{"1min":0.72699999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3879999999999999,"5min":1.746,"15min":1.752},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.147,"15min":1.155},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.52,"5min":1.6519999999999999,"15min":1.696},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1850000000000001,"15min":1.226},"min":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.589,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":1.2410000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.21,"15min":1.246},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.5820000000000001,"5min":1.901,"15min":1.901},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.234,"15min":1.2490000000000001},"min":{"1min":0.90600000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5609999999999999,"5min":1.899,"15min":1.9219999999999999},"last":1.149}]},{"osd":3,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3029999999999999,"15min":1.3140000000000001},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.508,"5min":1.889,"15min":1.8979999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.3169999999999999,"15min":1.3540000000000001},"min":{"1min":1.0649999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.54,"5min":1.97,"15min":2.1859999999999999},"last":1.46}]},{"osd":4,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2669999999999999,"15min":1.256},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.764,"5min":1.869,"15min":2.6040000000000001},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3660000000000001,"15min":1.371},"min":{"1min":0.96599999999999997,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.5449999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.373}]},{"osd":6,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3839999999999999,"15min":1.375},"min":{"1min":0.97699999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.643,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4239999999999999,"15min":1.419},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.9850000000000001,"5min":2.032,"15min":2.577},"last":1.504}]},{"osd":7,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.5349999999999999,"15min":1.536},"min":{"1min":1.294,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.8779999999999999,"5min":2.077,"15min":3.1709999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.5780000000000001,"15min":1.5760000000000001},"min":{"1min":1.353,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.8819999999999999,"5min":2.101,"15min":2.8740000000000001},"last":1.76}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52212,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319628,"statfs":{"total":96636764160,"available":96583299072,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.65200000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.68600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.641,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.034}]},{"osd":1,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.216,"15min":1.2470000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5920000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.151,"15min":1.1699999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6930000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":0.86699999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.3100000000000001,"15min":1.238},"min":{"1min":0.81399999999999995,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.4630000000000001,"5min":11.311,"15min":11.311},"last":1.431},{"interface":"front","average":{"1min":1.159,"5min":1.2629999999999999,"15min":1.3200000000000001},"min":{"1min":0.86199999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.675,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2789999999999999,"15min":1.2829999999999999},"min":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5129999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.363,"15min":1.3320000000000001},"min":{"1min":0.90000000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2559999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.6100000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.375,"15min":1.329},"min":{"1min":1.028,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.897,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.125},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3580000000000001,"15min":1.3149999999999999},"min":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.74,"5min":3.71,"15min":3.71},"last":1.097}]},{"osd":6,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.3899999999999999,"15min":1.3740000000000001},"min":{"1min":1.0289999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.161,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":1.25,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.028,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.23}]},{"osd":7,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5089999999999999,"15min":1.4850000000000001},"min":{"1min":0.82399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.794,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5920000000000001,"15min":1.577},"min":{"1min":1.139,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3250000000000002,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.7350000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56308,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315532,"statfs":{"total":96636764160,"available":96579104768,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.306,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0449999999999999,"15min":1.0609999999999999},"min":{"1min":0.39500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.573,"5min":1.95,"15min":1.95},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.083,"15min":1.1479999999999999},"min":{"1min":0.56200000000000006,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.417,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.94399999999999995,"15min":0.98499999999999999},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6970000000000001,"5min":3.863,"15min":3.863},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.038,"15min":1.085},"min":{"1min":0.49099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.224,"5min":3.597,"15min":3.597},"last":0.629},{"interface":"front","average":{"1min":1.052,"5min":1.175,"15min":1.228},"min":{"1min":0.54700000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.647,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.78300000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.264,"15min":1.286},"min":{"1min":0.86099999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5269999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.248,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.79500000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9970000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.353}]},{"osd":5,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3109999999999999,"15min":1.325},"min":{"1min":0.92200000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.208,"5min":1.3420000000000001,"15min":1.3600000000000001},"min":{"1min":0.82499999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6699999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.22}]},{"osd":6,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.375,"15min":1.4370000000000001},"min":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.29,"5min":1.456,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.6879999999999999,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":1.149}]},{"osd":7,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4750000000000001,"15min":1.514},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.788,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4770000000000001,"15min":1.522},"min":{"1min":1.0549999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.913,"5min":3.004,"15min":3.004},"last":1.1930000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48116,"kb_used_data":3116,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323724,"statfs":{"total":96636764160,"available":96587493376,"internally_reserved":0,"allocated":3190784,"data_stored":1570971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.81100000000000005,"15min":0.87},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92100000000000004,"5min":1.411,"15min":1.4730000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98899999999999999,"15min":1.024},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.216,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.97299999999999998,"15min":1.0309999999999999},"min":{"1min":0.57899999999999996,"5min":0.378,"15min":0.378},"max":{"1min":1.512,"5min":1.512,"15min":1.698},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.04,"15min":1.113},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5229999999999999,"5min":1.635,"15min":1.6599999999999999},"last":1.006}]},{"osd":3,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.97499999999999998,"15min":1.022},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2889999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.097,"15min":1.181},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.4970000000000001,"5min":1.615,"15min":1.79},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2749999999999999,"15min":1.3120000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.621,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.276},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4730000000000001,"5min":1.714,"15min":1.835},"last":1.1299999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.3109999999999999,"15min":1.3680000000000001},"min":{"1min":0.86599999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.639,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.347,"15min":1.393},"min":{"1min":0.79000000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.573,"5min":1.8740000000000001,"15min":2.016},"last":1.23}]},{"osd":6,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.351,"15min":1.367},"min":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.865,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.651},{"interface":"front","average":{"1min":1.268,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.86599999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7090000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4370000000000001,"15min":1.4630000000000001},"min":{"1min":0.94299999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5740000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.464},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.4350000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.8100000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.762}]}]},{"osd":1,"up_from":13,"seq":55834575998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52792,"kb_used_data":3696,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319048,"statfs":{"total":96636764160,"available":96582705152,"internally_reserved":0,"allocated":3784704,"data_stored":2161339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82899999999999996,"15min":0.88800000000000001},"min":{"1min":0.38500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.5569999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.94399999999999995,"15min":1.0249999999999999},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.167,"5min":1.512,"15min":1.681},"last":1.0409999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.86299999999999999,"15min":0.92300000000000004},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97699999999999998,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":1.28},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0349999999999999,"15min":1.1220000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.476,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":1.3819999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.1359999999999999,"15min":1.268},"min":{"1min":0.52400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.788},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.046,"15min":1.1120000000000001},"min":{"1min":0.41399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.615,"5min":1.714,"15min":1.714},"last":1.1799999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.177,"15min":1.2090000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6000000000000001,"5min":1.623,"15min":1.6859999999999999},"last":1.091},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.329,"15min":1.391},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7450000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.482}]},{"osd":5,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2969999999999999,"15min":1.3240000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8169999999999999,"5min":1.9399999999999999,"15min":2.0379999999999998},"last":1.302},{"interface":"front","average":{"1min":1.2,"5min":1.2569999999999999,"15min":1.319},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.7470000000000001,"5min":1.927,"15min":1.927},"last":1.3480000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.373,"15min":1.429},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.885,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3759999999999999,"15min":1.4019999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.901,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.159}]},{"osd":7,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.431,"15min":1.5069999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.46,"15min":1.5009999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.954,"5min":2.1389999999999998,"15min":2.2069999999999999},"last":1.516}]}]},{"osd":0,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60416,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311424,"statfs":{"total":96636764160,"available":96574898176,"internally_reserved":0,"allocated":3203072,"data_stored":1577290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.40100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94499999999999995,"15min":1.04},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.0249999999999999,"15min":1.0660000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.48699999999999999},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.1429999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.0269999999999999,"15min":1.107},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6619999999999999,"5min":1.726,"15min":1.792},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.99099999999999999,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.306,"5min":1.536,"15min":1.8839999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.194,"5min":1.165,"15min":1.222},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.27,"15min":1.337},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.246,"15min":1.2949999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.536,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1060000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.839,"5min":1.899,"15min":2.7719999999999998},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.365,"15min":1.423},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.484,"5min":2.484,"15min":2.5880000000000001},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.327,"15min":1.371},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.835,"5min":1.835,"15min":2.2029999999999998},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.444,"15min":1.4810000000000001},"min":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":1.9239999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.351}]},{"osd":7,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.4419999999999999,"15min":1.506},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.411,"5min":2.411,"15min":2.7040000000000002},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.456,"15min":1.4770000000000001},"min":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.718,"5min":2.1190000000000002,"15min":2.431},"last":1.143}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:12:40.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3f is not active+clean 2023-12-01T03:12:40.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.851842+0000', 'last_change': '2023-12-01T03:12:34.851842+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.851842+0000', 'last_undegraded': '2023-12-01T03:12:34.851842+0000', 'last_fullsized': '2023-12-01T03:12:34.851842+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:21:23.692942+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:12:40.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3e is not active+clean 2023-12-01T03:12:40.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.849451+0000', 'last_change': '2023-12-01T03:12:34.849451+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.849451+0000', 'last_undegraded': '2023-12-01T03:12:34.849451+0000', 'last_fullsized': '2023-12-01T03:12:34.849451+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T03:12:40.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3d is not active+clean 2023-12-01T03:12:40.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.845514+0000', 'last_change': '2023-12-01T03:12:34.845514+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.845514+0000', 'last_undegraded': '2023-12-01T03:12:34.845514+0000', 'last_fullsized': '2023-12-01T03:12:34.845514+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T03:12:40.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3c is not active+clean 2023-12-01T03:12:40.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.850690+0000', 'last_change': '2023-12-01T03:12:34.850690+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.850690+0000', 'last_undegraded': '2023-12-01T03:12:34.850690+0000', 'last_fullsized': '2023-12-01T03:12:34.850690+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:08:49.741119+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3b is not active+clean 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.850902+0000', 'last_change': '2023-12-01T03:12:34.850902+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.850902+0000', 'last_undegraded': '2023-12-01T03:12:34.850902+0000', 'last_fullsized': '2023-12-01T03:12:34.850902+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3a is not active+clean 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.848923+0000', 'last_change': '2023-12-01T03:12:34.848923+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.848923+0000', 'last_undegraded': '2023-12-01T03:12:34.848923+0000', 'last_fullsized': '2023-12-01T03:12:34.848923+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:40:04.481100+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.39 is not active+clean 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.855117+0000', 'last_change': '2023-12-01T03:12:34.855117+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.855117+0000', 'last_undegraded': '2023-12-01T03:12:34.855117+0000', 'last_fullsized': '2023-12-01T03:12:34.855117+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.38 is not active+clean 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.846440+0000', 'last_change': '2023-12-01T03:12:34.846440+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.846440+0000', 'last_undegraded': '2023-12-01T03:12:34.846440+0000', 'last_fullsized': '2023-12-01T03:12:34.846440+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:12:27.055115+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.37 is not active+clean 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.855652+0000', 'last_change': '2023-12-01T03:12:34.855652+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.855652+0000', 'last_undegraded': '2023-12-01T03:12:34.855652+0000', 'last_fullsized': '2023-12-01T03:12:34.855652+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:17:30.864087+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.36 is not active+clean 2023-12-01T03:12:40.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.848420+0000', 'last_change': '2023-12-01T03:12:34.848420+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.848420+0000', 'last_undegraded': '2023-12-01T03:12:34.848420+0000', 'last_fullsized': '2023-12-01T03:12:34.848420+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.35 is not active+clean 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.852653+0000', 'last_change': '2023-12-01T03:12:34.852653+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.852653+0000', 'last_undegraded': '2023-12-01T03:12:34.852653+0000', 'last_fullsized': '2023-12-01T03:12:34.852653+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.34 is not active+clean 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.845433+0000', 'last_change': '2023-12-01T03:12:34.845433+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.845433+0000', 'last_undegraded': '2023-12-01T03:12:34.845433+0000', 'last_fullsized': '2023-12-01T03:12:34.845433+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.33 is not active+clean 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.852346+0000', 'last_change': '2023-12-01T03:12:34.852346+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.852346+0000', 'last_undegraded': '2023-12-01T03:12:34.852346+0000', 'last_fullsized': '2023-12-01T03:12:34.852346+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:54:17.936247+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.32 is not active+clean 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.854378+0000', 'last_change': '2023-12-01T03:12:34.854378+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.854378+0000', 'last_undegraded': '2023-12-01T03:12:34.854378+0000', 'last_fullsized': '2023-12-01T03:12:34.854378+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:59:25.105386+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.31 is not active+clean 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.854306+0000', 'last_change': '2023-12-01T03:12:34.854306+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.854306+0000', 'last_undegraded': '2023-12-01T03:12:34.854306+0000', 'last_fullsized': '2023-12-01T03:12:34.854306+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.30 is not active+clean 2023-12-01T03:12:40.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.845305+0000', 'last_change': '2023-12-01T03:12:34.845305+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.845305+0000', 'last_undegraded': '2023-12-01T03:12:34.845305+0000', 'last_fullsized': '2023-12-01T03:12:34.845305+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2f is not active+clean 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.853111+0000', 'last_change': '2023-12-01T03:12:34.853111+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.853111+0000', 'last_undegraded': '2023-12-01T03:12:34.853111+0000', 'last_fullsized': '2023-12-01T03:12:34.853111+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:21:21.877813+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2e is not active+clean 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.846554+0000', 'last_change': '2023-12-01T03:12:34.846554+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.846554+0000', 'last_undegraded': '2023-12-01T03:12:34.846554+0000', 'last_fullsized': '2023-12-01T03:12:34.846554+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2d is not active+clean 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.844698+0000', 'last_change': '2023-12-01T03:12:34.844698+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.844698+0000', 'last_undegraded': '2023-12-01T03:12:34.844698+0000', 'last_fullsized': '2023-12-01T03:12:34.844698+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2c is not active+clean 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.846725+0000', 'last_change': '2023-12-01T03:12:34.846725+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.846725+0000', 'last_undegraded': '2023-12-01T03:12:34.846725+0000', 'last_fullsized': '2023-12-01T03:12:34.846725+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2b is not active+clean 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.845156+0000', 'last_change': '2023-12-01T03:12:34.845156+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.845156+0000', 'last_undegraded': '2023-12-01T03:12:34.845156+0000', 'last_fullsized': '2023-12-01T03:12:34.845156+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:16:31.038196+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:12:40.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2a is not active+clean 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.846691+0000', 'last_change': '2023-12-01T03:12:34.846691+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.846691+0000', 'last_undegraded': '2023-12-01T03:12:34.846691+0000', 'last_fullsized': '2023-12-01T03:12:34.846691+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:59:04.995090+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.29 is not active+clean 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.843637+0000', 'last_change': '2023-12-01T03:12:34.843637+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.843637+0000', 'last_undegraded': '2023-12-01T03:12:34.843637+0000', 'last_fullsized': '2023-12-01T03:12:34.843637+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.28 is not active+clean 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.844221+0000', 'last_change': '2023-12-01T03:12:34.844221+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.844221+0000', 'last_undegraded': '2023-12-01T03:12:34.844221+0000', 'last_fullsized': '2023-12-01T03:12:34.844221+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:00:57.853346+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.27 is not active+clean 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.855001+0000', 'last_change': '2023-12-01T03:12:34.855001+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.855001+0000', 'last_undegraded': '2023-12-01T03:12:34.855001+0000', 'last_fullsized': '2023-12-01T03:12:34.855001+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:19:34.404254+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.26 is not active+clean 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.850115+0000', 'last_change': '2023-12-01T03:12:34.850115+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.850115+0000', 'last_undegraded': '2023-12-01T03:12:34.850115+0000', 'last_fullsized': '2023-12-01T03:12:34.850115+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:17:00.577497+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.25 is not active+clean 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.846472+0000', 'last_change': '2023-12-01T03:12:34.846472+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.846472+0000', 'last_undegraded': '2023-12-01T03:12:34.846472+0000', 'last_fullsized': '2023-12-01T03:12:34.846472+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:09:42.224008+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:12:40.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.24 is not active+clean 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.850472+0000', 'last_change': '2023-12-01T03:12:34.850472+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.850472+0000', 'last_undegraded': '2023-12-01T03:12:34.850472+0000', 'last_fullsized': '2023-12-01T03:12:34.850472+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T04:53:08.535342+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.23 is not active+clean 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.847309+0000', 'last_change': '2023-12-01T03:12:34.847309+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.847309+0000', 'last_undegraded': '2023-12-01T03:12:34.847309+0000', 'last_fullsized': '2023-12-01T03:12:34.847309+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:50:37.148015+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.22 is not active+clean 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.853876+0000', 'last_change': '2023-12-01T03:12:34.853876+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.853876+0000', 'last_undegraded': '2023-12-01T03:12:34.853876+0000', 'last_fullsized': '2023-12-01T03:12:34.853876+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:37:25.025253+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.21 is not active+clean 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.854643+0000', 'last_change': '2023-12-01T03:12:34.854643+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.854643+0000', 'last_undegraded': '2023-12-01T03:12:34.854643+0000', 'last_fullsized': '2023-12-01T03:12:34.854643+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1e is not active+clean 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.849276+0000', 'last_change': '2023-12-01T03:12:34.849276+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.849276+0000', 'last_undegraded': '2023-12-01T03:12:34.849276+0000', 'last_fullsized': '2023-12-01T03:12:34.849276+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:10:45.146806+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1f is not active+clean 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.852053+0000', 'last_change': '2023-12-01T03:12:34.852053+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.852053+0000', 'last_undegraded': '2023-12-01T03:12:34.852053+0000', 'last_fullsized': '2023-12-01T03:12:34.852053+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:13:59.927209+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:12:40.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1c is not active+clean 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.844945+0000', 'last_change': '2023-12-01T03:12:34.844945+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.844945+0000', 'last_undegraded': '2023-12-01T03:12:34.844945+0000', 'last_fullsized': '2023-12-01T03:12:34.844945+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1d is not active+clean 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.852093+0000', 'last_change': '2023-12-01T03:12:34.852093+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.852093+0000', 'last_undegraded': '2023-12-01T03:12:34.852093+0000', 'last_fullsized': '2023-12-01T03:12:34.852093+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1a is not active+clean 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.850269+0000', 'last_change': '2023-12-01T03:12:34.850269+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.850269+0000', 'last_undegraded': '2023-12-01T03:12:34.850269+0000', 'last_fullsized': '2023-12-01T03:12:34.850269+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1b is not active+clean 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.844881+0000', 'last_change': '2023-12-01T03:12:34.844881+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.844881+0000', 'last_undegraded': '2023-12-01T03:12:34.844881+0000', 'last_fullsized': '2023-12-01T03:12:34.844881+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:46:13.355768+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.18 is not active+clean 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.852657+0000', 'last_change': '2023-12-01T03:12:34.852657+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.852657+0000', 'last_undegraded': '2023-12-01T03:12:34.852657+0000', 'last_fullsized': '2023-12-01T03:12:34.852657+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.19 is not active+clean 2023-12-01T03:12:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.854072+0000', 'last_change': '2023-12-01T03:12:34.854072+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.854072+0000', 'last_undegraded': '2023-12-01T03:12:34.854072+0000', 'last_fullsized': '2023-12-01T03:12:34.854072+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.16 is not active+clean 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.850169+0000', 'last_change': '2023-12-01T03:12:34.850169+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.850169+0000', 'last_undegraded': '2023-12-01T03:12:34.850169+0000', 'last_fullsized': '2023-12-01T03:12:34.850169+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:47:19.476510+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.17 is not active+clean 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.852349+0000', 'last_change': '2023-12-01T03:12:34.852349+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.852349+0000', 'last_undegraded': '2023-12-01T03:12:34.852349+0000', 'last_fullsized': '2023-12-01T03:12:34.852349+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.14 is not active+clean 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.849661+0000', 'last_change': '2023-12-01T03:12:34.849661+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.849661+0000', 'last_undegraded': '2023-12-01T03:12:34.849661+0000', 'last_fullsized': '2023-12-01T03:12:34.849661+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.15 is not active+clean 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.849924+0000', 'last_change': '2023-12-01T03:12:34.849924+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.849924+0000', 'last_undegraded': '2023-12-01T03:12:34.849924+0000', 'last_fullsized': '2023-12-01T03:12:34.849924+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.12 is not active+clean 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.853000+0000', 'last_change': '2023-12-01T03:12:34.853000+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.853000+0000', 'last_undegraded': '2023-12-01T03:12:34.853000+0000', 'last_fullsized': '2023-12-01T03:12:34.853000+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:44:23.725637+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.13 is not active+clean 2023-12-01T03:12:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.845680+0000', 'last_change': '2023-12-01T03:12:34.845680+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.845680+0000', 'last_undegraded': '2023-12-01T03:12:34.845680+0000', 'last_fullsized': '2023-12-01T03:12:34.845680+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.10 is not active+clean 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.852165+0000', 'last_change': '2023-12-01T03:12:34.852165+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.852165+0000', 'last_undegraded': '2023-12-01T03:12:34.852165+0000', 'last_fullsized': '2023-12-01T03:12:34.852165+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:04:09.892305+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.11 is not active+clean 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.846877+0000', 'last_change': '2023-12-01T03:12:34.846877+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.846877+0000', 'last_undegraded': '2023-12-01T03:12:34.846877+0000', 'last_fullsized': '2023-12-01T03:12:34.846877+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.e is not active+clean 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.847970+0000', 'last_change': '2023-12-01T03:12:34.847970+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.847970+0000', 'last_undegraded': '2023-12-01T03:12:34.847970+0000', 'last_fullsized': '2023-12-01T03:12:34.847970+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.f is not active+clean 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.846620+0000', 'last_change': '2023-12-01T03:12:34.846620+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.846620+0000', 'last_undegraded': '2023-12-01T03:12:34.846620+0000', 'last_fullsized': '2023-12-01T03:12:34.846620+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.c is not active+clean 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.852931+0000', 'last_change': '2023-12-01T03:12:34.852931+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.852931+0000', 'last_undegraded': '2023-12-01T03:12:34.852931+0000', 'last_fullsized': '2023-12-01T03:12:34.852931+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T14:03:37.465541+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:12:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.d is not active+clean 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.851102+0000', 'last_change': '2023-12-01T03:12:34.851102+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.851102+0000', 'last_undegraded': '2023-12-01T03:12:34.851102+0000', 'last_fullsized': '2023-12-01T03:12:34.851102+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.a is not active+clean 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.848035+0000', 'last_change': '2023-12-01T03:12:34.848035+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.848035+0000', 'last_undegraded': '2023-12-01T03:12:34.848035+0000', 'last_fullsized': '2023-12-01T03:12:34.848035+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.b is not active+clean 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.846992+0000', 'last_change': '2023-12-01T03:12:34.846992+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.846992+0000', 'last_undegraded': '2023-12-01T03:12:34.846992+0000', 'last_fullsized': '2023-12-01T03:12:34.846992+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.8 is not active+clean 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.843616+0000', 'last_change': '2023-12-01T03:12:34.843616+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.843616+0000', 'last_undegraded': '2023-12-01T03:12:34.843616+0000', 'last_fullsized': '2023-12-01T03:12:34.843616+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.9 is not active+clean 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.845514+0000', 'last_change': '2023-12-01T03:12:34.845514+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.845514+0000', 'last_undegraded': '2023-12-01T03:12:34.845514+0000', 'last_fullsized': '2023-12-01T03:12:34.845514+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.6 is not active+clean 2023-12-01T03:12:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.846826+0000', 'last_change': '2023-12-01T03:12:34.846826+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.846826+0000', 'last_undegraded': '2023-12-01T03:12:34.846826+0000', 'last_fullsized': '2023-12-01T03:12:34.846826+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T08:55:08.951057+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.7 is not active+clean 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.851163+0000', 'last_change': '2023-12-01T03:12:34.851163+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.851163+0000', 'last_undegraded': '2023-12-01T03:12:34.851163+0000', 'last_fullsized': '2023-12-01T03:12:34.851163+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T05:24:20.331105+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.4 is not active+clean 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.844493+0000', 'last_change': '2023-12-01T03:12:34.844493+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.844493+0000', 'last_undegraded': '2023-12-01T03:12:34.844493+0000', 'last_fullsized': '2023-12-01T03:12:34.844493+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.5 is not active+clean 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.845661+0000', 'last_change': '2023-12-01T03:12:34.845661+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.845661+0000', 'last_undegraded': '2023-12-01T03:12:34.845661+0000', 'last_fullsized': '2023-12-01T03:12:34.845661+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T09:35:45.819776+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.2 is not active+clean 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.849450+0000', 'last_change': '2023-12-01T03:12:34.849450+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.849450+0000', 'last_undegraded': '2023-12-01T03:12:34.849450+0000', 'last_fullsized': '2023-12-01T03:12:34.849450+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T13:29:01.535641+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.3 is not active+clean 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.843521+0000', 'last_change': '2023-12-01T03:12:34.843521+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.843521+0000', 'last_undegraded': '2023-12-01T03:12:34.843521+0000', 'last_fullsized': '2023-12-01T03:12:34.843521+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.0 is not active+clean 2023-12-01T03:12:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.851703+0000', 'last_change': '2023-12-01T03:12:34.851703+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.851703+0000', 'last_undegraded': '2023-12-01T03:12:34.851703+0000', 'last_fullsized': '2023-12-01T03:12:34.851703+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-01T03:12:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.1 is not active+clean 2023-12-01T03:12:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.845635+0000', 'last_change': '2023-12-01T03:12:34.845635+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.845635+0000', 'last_undegraded': '2023-12-01T03:12:34.845635+0000', 'last_fullsized': '2023-12-01T03:12:34.845635+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T15:08:34.191802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-01T03:12:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 33.20 is not active+clean 2023-12-01T03:12:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '33.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 254, 'state': 'creating+peering', 'last_fresh': '2023-12-01T03:12:34.849489+0000', 'last_change': '2023-12-01T03:12:34.849489+0000', 'last_active': '2023-12-01T03:12:34.812524+0000', 'last_peered': '2023-12-01T03:12:34.812524+0000', 'last_clean': '2023-12-01T03:12:34.812524+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-01T03:12:34.849489+0000', 'last_undegraded': '2023-12-01T03:12:34.849489+0000', 'last_fullsized': '2023-12-01T03:12:34.849489+0000', 'mapping_epoch': 254, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 254, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'last_clean_scrub_stamp': '2023-12-01T03:12:34.812524+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-01T03:12:40.874 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-01T03:12:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.116+0000 7fc0ad9f0640 1 Processor -- start 2023-12-01T03:12:41.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.116+0000 7fc0ad9f0640 1 -- start start 2023-12-01T03:12:41.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.118+0000 7fc0ad9f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0a80cad70 0x7fc0a806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.118+0000 7fc0ad9f0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a80add10 con 0x7fc0a806f2f0 2023-12-01T03:12:41.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.118+0000 7fc0ad9f0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a80c5760 con 0x7fc0a815e2f0 2023-12-01T03:12:41.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.118+0000 7fc0ad9f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a815e6d0 con 0x7fc0a80cad70 2023-12-01T03:12:41.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.118+0000 7fc0a77fe640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc0a806f2f0 0x7fc0a806bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37736/0 (socket says 172.21.15.115:37736) 2023-12-01T03:12:41.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.118+0000 7fc0a77fe640 1 -- 172.21.15.115:0/2601814488 learned_addr learned my addr 172.21.15.115:0/2601814488 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.118+0000 7fc0a67fc640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0a80cad70 0x7fc0a806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.118+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1094987874 0 0) 0x7fc0a80add10 con 0x7fc0a806f2f0 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc090003520 con 0x7fc0a806f2f0 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2893963169 0 0) 0x7fc090003520 con 0x7fc0a806f2f0 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0a80cad70 msgr2=0x7fc0a806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0a80cad70 0x7fc0a806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 >> v1:172.21.15.142:6789/0 conn(0x7fc0a815e2f0 legacy=0x7fc0a80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a80cf7b0 con 0x7fc0a806f2f0 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc0a00019f0 con 0x7fc0a806f2f0 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 >> v1:172.21.15.115:6789/0 conn(0x7fc0a806f2f0 legacy=0x7fc0a806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc090003a10 0x7fc090003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0900047c0 0x7fc090004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0a80cad70 0x7fc09000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a815e6d0 con 0x7fc090003a10 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc090005510 con 0x7fc0900047c0 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09000bec0 con 0x7fc0a80cad70 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.119+0000 7fc0a6ffd640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc090003a10 0x7fc090003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.120+0000 7fc0a77fe640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0a80cad70 0x7fc09000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.120+0000 7fc0a6ffd640 1 -- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0a80cad70 msgr2=0x7fc09000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.120+0000 7fc0a6ffd640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0a80cad70 0x7fc09000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.120+0000 7fc0a6ffd640 1 -- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0900047c0 msgr2=0x7fc090004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.120+0000 7fc0a67fc640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0900047c0 0x7fc090004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.120+0000 7fc0a6ffd640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0900047c0 0x7fc090004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.120+0000 7fc0a6ffd640 1 -- 172.21.15.115:0/2601814488 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc090003520 con 0x7fc090003a10 2023-12-01T03:12:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.121+0000 7fc0a6ffd640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc090003a10 0x7fc090003e30 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7fc098001750 tx=0x7fc0900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffcdf29c54bb4aa server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.121+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0980052d0 con 0x7fc090003a10 2023-12-01T03:12:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.121+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09802e050 con 0x7fc090003a10 2023-12-01T03:12:41.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.121+0000 7fc0a5ffb640 1 -- 172.21.15.115:0/2601814488 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0980035f0 con 0x7fc090003a10 2023-12-01T03:12:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.122+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/2601814488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc090003a10 msgr2=0x7fc090003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.122+0000 7fc0ad9f0640 1 --2- 172.21.15.115:0/2601814488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc090003a10 0x7fc090003e30 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7fc098001750 tx=0x7fc0900057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.122+0000 7fc0a77fe640 1 -- 172.21.15.115:0/2601814488 reap_dead start 2023-12-01T03:12:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.122+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/2601814488 shutdown_connections 2023-12-01T03:12:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.122+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/2601814488 >> 172.21.15.115:0/2601814488 conn(0x7fc0a80ce880 msgr2=0x7fc0a80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.122+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/2601814488 shutdown_connections 2023-12-01T03:12:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.122+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/2601814488 wait complete. 2023-12-01T03:12:41.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.124+0000 7fc0ad9f0640 1 Processor -- start 2023-12-01T03:12:41.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.124+0000 7fc0ad9f0640 1 -- start start 2023-12-01T03:12:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.125+0000 7fc0ad9f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc090003a10 0x7fc0a815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.125+0000 7fc0ad9f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0a80c8450 0x7fc0a80c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.125+0000 7fc0ad9f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a81f5b70 0x7fc0a81f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.125+0000 7fc0ad9f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a81f6530 con 0x7fc0a80c8450 2023-12-01T03:12:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.125+0000 7fc0ad9f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a80c25c0 con 0x7fc0a81f5b70 2023-12-01T03:12:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.125+0000 7fc0ad9f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a815cdb0 con 0x7fc090003a10 2023-12-01T03:12:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.125+0000 7fc0a67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0a80c8450 0x7fc0a80c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.125+0000 7fc0a67fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0a80c8450 0x7fc0a80c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51842/0 (socket says 172.21.15.115:51842) 2023-12-01T03:12:41.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.125+0000 7fc0a67fc640 1 -- 172.21.15.115:0/4264332945 learned_addr learned my addr 172.21.15.115:0/4264332945 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:41.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc0a67fc640 1 -- 172.21.15.115:0/4264332945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc090003a10 msgr2=0x7fc0a815c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc0a67fc640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc090003a10 0x7fc0a815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc0a67fc640 1 -- 172.21.15.115:0/4264332945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a81f5b70 msgr2=0x7fc0a81f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc0a77fe640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a81f5b70 0x7fc0a81f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc0a67fc640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a81f5b70 0x7fc0a81f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc0a67fc640 1 -- 172.21.15.115:0/4264332945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a80c6bb0 con 0x7fc0a80c8450 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc0a67fc640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0a80c8450 0x7fc0a80c6360 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0108c0 tx=0x7fc09c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc08ffff640 1 -- 172.21.15.115:0/4264332945 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c009610 con 0x7fc0a80c8450 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc08ffff640 1 -- 172.21.15.115:0/4264332945 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c00eb60 con 0x7fc0a80c8450 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/4264332945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a80c6e90 con 0x7fc0a80c8450 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.126+0000 7fc08ffff640 1 -- 172.21.15.115:0/4264332945 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c01c400 con 0x7fc0a80c8450 2023-12-01T03:12:41.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.127+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/4264332945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a80c68a0 con 0x7fc0a80c8450 2023-12-01T03:12:41.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.128+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/4264332945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a806fda0 con 0x7fc0a80c8450 2023-12-01T03:12:41.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.129+0000 7fc08ffff640 1 -- 172.21.15.115:0/4264332945 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fc09c01c560 con 0x7fc0a80c8450 2023-12-01T03:12:41.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.129+0000 7fc08ffff640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc09406a670 0x7fc09406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.130+0000 7fc08ffff640 1 -- 172.21.15.115:0/4264332945 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fc09c089260 con 0x7fc0a80c8450 2023-12-01T03:12:41.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.130+0000 7fc0a6ffd640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc09406a670 0x7fc09406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.131+0000 7fc0a6ffd640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc09406a670 0x7fc09406cb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc098033000 tx=0x7fc09802f000 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:41.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.134+0000 7fc08ffff640 1 -- 172.21.15.115:0/4264332945 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc09c054570 con 0x7fc0a80c8450 2023-12-01T03:12:41.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.392+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/4264332945 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc0a80c73b0 con 0x7fc0a80c8450 2023-12-01T03:12:41.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.393+0000 7fc08ffff640 1 -- 172.21.15.115:0/4264332945 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3235 (secure 0 0 0) 0x7fc09c021cc0 con 0x7fc0a80c8450 2023-12-01T03:12:41.395 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:41.395 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5816,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":256,"num_osds":8,"num_up_osds":8,"osd_up_since":1701394558,"num_in_osds":8,"osd_in_since":1701394549,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":454098944,"bytes_avail":772640014336,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":126,"id":16,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":16,"rank":0,"name":"b","status":"up:active","gid":12246}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2023-12-01T03:12:15.042242+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-01T03:12:41.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.396+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/4264332945 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc09406a670 msgr2=0x7fc09406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.396+0000 7fc0ad9f0640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc09406a670 0x7fc09406cb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc098033000 tx=0x7fc09802f000 comp rx=0 tx=0).stop 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.396+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/4264332945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0a80c8450 msgr2=0x7fc0a80c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.396+0000 7fc0ad9f0640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0a80c8450 0x7fc0a80c6360 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0108c0 tx=0x7fc09c010d80 comp rx=0 tx=0).stop 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.396+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/4264332945 shutdown_connections 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.396+0000 7fc0ad9f0640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc09406a670 0x7fc09406cb30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.396+0000 7fc0ad9f0640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0a81f5b70 0x7fc0a81f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.396+0000 7fc0ad9f0640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0a80c8450 0x7fc0a80c6360 unknown :-1 s=CLOSED pgs=2978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.396+0000 7fc0ad9f0640 1 --2- 172.21.15.115:0/4264332945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc090003a10 0x7fc0a815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.396+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/4264332945 >> 172.21.15.115:0/4264332945 conn(0x7fc0a80ce880 msgr2=0x7fc0a806b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.397+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/4264332945 shutdown_connections 2023-12-01T03:12:41.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.397+0000 7fc0ad9f0640 1 -- 172.21.15.115:0/4264332945 wait complete. 2023-12-01T03:12:41.417 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-01T03:12:41.418 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:12:41.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.624+0000 7f36a508c640 1 Processor -- start 2023-12-01T03:12:41.626 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.624+0000 7f36a508c640 1 -- start start 2023-12-01T03:12:41.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.626+0000 7f36a508c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 0x7f36a00cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.626+0000 7f36a508c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f36a015afe0 0x7f36a006b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.626+0000 7f36a508c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a00cc180 con 0x7f36a006b7b0 2023-12-01T03:12:41.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.626+0000 7f36a508c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a006bc40 con 0x7f36a015afe0 2023-12-01T03:12:41.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.626+0000 7f36a508c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a015b550 con 0x7f36a01584b0 2023-12-01T03:12:41.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.626+0000 7f36977fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f36a015afe0 0x7f36a006b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.626+0000 7f36a488b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f36a006b7b0 0x7f36a015e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37744/0 (socket says 172.21.15.115:37744) 2023-12-01T03:12:41.628 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.626+0000 7f36a488b640 1 -- 172.21.15.115:0/2955934543 learned_addr learned my addr 172.21.15.115:0/2955934543 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:41.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.627+0000 7f369ffff640 1 --2- 172.21.15.115:0/2955934543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 0x7f36a00cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.627+0000 7f36977fe640 1 -- 172.21.15.115:0/2955934543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 msgr2=0x7f36a00cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:41.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.627+0000 7f36977fe640 1 --2- 172.21.15.115:0/2955934543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 0x7f36a00cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.629 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.627+0000 7f36977fe640 1 -- 172.21.15.115:0/2955934543 >> v1:172.21.15.115:6789/0 conn(0x7f36a006b7b0 legacy=0x7f36a015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.627+0000 7f36977fe640 1 -- 172.21.15.115:0/2955934543 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a015b830 con 0x7f36a015afe0 2023-12-01T03:12:41.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.627+0000 7f369f7fe640 1 -- 172.21.15.115:0/2955934543 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725730065 0 0) 0x7f36a00cc180 con 0x7f36a006b7b0 2023-12-01T03:12:41.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.627+0000 7f369ffff640 1 --2- 172.21.15.115:0/2955934543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 0x7f36a00cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:41.630 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.628+0000 7f36977fe640 1 --2- 172.21.15.115:0/2955934543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a015afe0 0x7f36a006b270 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f36880097a0 tx=0x7f368802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4b7d1bd8da4d74 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:41.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.628+0000 7f369f7fe640 1 -- 172.21.15.115:0/2955934543 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368802d040 con 0x7f36a015afe0 2023-12-01T03:12:41.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.628+0000 7f369f7fe640 1 -- 172.21.15.115:0/2955934543 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3688031810 con 0x7f36a015afe0 2023-12-01T03:12:41.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.628+0000 7f369f7fe640 1 -- 172.21.15.115:0/2955934543 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688031ba0 con 0x7f36a015afe0 2023-12-01T03:12:41.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.629+0000 7f36a508c640 1 -- 172.21.15.115:0/2955934543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a015afe0 msgr2=0x7f36a006b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.629+0000 7f36a508c640 1 --2- 172.21.15.115:0/2955934543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a015afe0 0x7f36a006b270 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f36880097a0 tx=0x7f368802b8b0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.629+0000 7f36a508c640 1 -- 172.21.15.115:0/2955934543 shutdown_connections 2023-12-01T03:12:41.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.629+0000 7f36a508c640 1 --2- 172.21.15.115:0/2955934543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a015afe0 0x7f36a006b270 unknown :-1 s=CLOSED pgs=2980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.629+0000 7f36a508c640 1 --2- 172.21.15.115:0/2955934543 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 0x7f36a00cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.629+0000 7f36a508c640 1 -- 172.21.15.115:0/2955934543 >> 172.21.15.115:0/2955934543 conn(0x7f36a00ce640 msgr2=0x7f36a00cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:41.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.629+0000 7f36a508c640 1 -- 172.21.15.115:0/2955934543 shutdown_connections 2023-12-01T03:12:41.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.629+0000 7f36a508c640 1 -- 172.21.15.115:0/2955934543 wait complete. 2023-12-01T03:12:41.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.631+0000 7f36a508c640 1 Processor -- start 2023-12-01T03:12:41.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.631+0000 7f36a508c640 1 -- start start 2023-12-01T03:12:41.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f36a508c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a006b7b0 0x7f36a01494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f36a508c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 0x7f36a00c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f36a508c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36a015afe0 0x7f36a00ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f36a508c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a01499e0 con 0x7f36a006b7b0 2023-12-01T03:12:41.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f36a508c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a00c6920 con 0x7f36a015afe0 2023-12-01T03:12:41.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f36a508c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a00cabe0 con 0x7f36a01584b0 2023-12-01T03:12:41.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f369ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a006b7b0 0x7f36a01494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f369ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a006b7b0 0x7f36a01494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51858/0 (socket says 172.21.15.115:51858) 2023-12-01T03:12:41.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f369ffff640 1 -- 172.21.15.115:0/2540849036 learned_addr learned my addr 172.21.15.115:0/2540849036 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f369ffff640 1 -- 172.21.15.115:0/2540849036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 msgr2=0x7f36a00c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f36977fe640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 0x7f36a00c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.635 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.632+0000 7f369ffff640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 0x7f36a00c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.633+0000 7f369ffff640 1 -- 172.21.15.115:0/2540849036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36a015afe0 msgr2=0x7f36a00ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.633+0000 7f369ffff640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36a015afe0 0x7f36a00ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.633+0000 7f369ffff640 1 -- 172.21.15.115:0/2540849036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3688009400 con 0x7f36a006b7b0 2023-12-01T03:12:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.633+0000 7f369ffff640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a006b7b0 0x7f36a01494a0 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7f369000cf60 tx=0x7f369000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.633+0000 7f369d7fa640 1 -- 172.21.15.115:0/2540849036 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369000d840 con 0x7f36a006b7b0 2023-12-01T03:12:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.633+0000 7f369d7fa640 1 -- 172.21.15.115:0/2540849036 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369000d9a0 con 0x7f36a006b7b0 2023-12-01T03:12:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.633+0000 7f36a508c640 1 -- 172.21.15.115:0/2540849036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a00c6b30 con 0x7f36a006b7b0 2023-12-01T03:12:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.633+0000 7f369d7fa640 1 -- 172.21.15.115:0/2540849036 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369000dd30 con 0x7f36a006b7b0 2023-12-01T03:12:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.633+0000 7f36a508c640 1 -- 172.21.15.115:0/2540849036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a00c70f0 con 0x7f36a006b7b0 2023-12-01T03:12:41.636 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.634+0000 7f36a508c640 1 -- 172.21.15.115:0/2540849036 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a00673a0 con 0x7f36a006b7b0 2023-12-01T03:12:41.637 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.635+0000 7f369d7fa640 1 -- 172.21.15.115:0/2540849036 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f3690015440 con 0x7f36a006b7b0 2023-12-01T03:12:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.636+0000 7f369d7fa640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f368006a670 0x7f368006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.636+0000 7f369d7fa640 1 -- 172.21.15.115:0/2540849036 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f3690089760 con 0x7f36a006b7b0 2023-12-01T03:12:41.638 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.637+0000 7f36977fe640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f368006a670 0x7f368006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:41.639 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.637+0000 7f36977fe640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f368006a670 0x7f368006cb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3688036000 tx=0x7f368803c040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:41.642 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.641+0000 7f369d7fa640 1 -- 172.21.15.115:0/2540849036 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3690054a70 con 0x7f36a006b7b0 2023-12-01T03:12:41.816 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.815+0000 7f36a508c640 1 -- 172.21.15.115:0/2540849036 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36a015bdf0 con 0x7f368006a670 2023-12-01T03:12:41.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.837+0000 7f369d7fa640 1 -- 172.21.15.115:0/2540849036 <== mgr.11877 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287189 (secure 0 0 0) 0x7f3688031610 con 0x7f368006a670 2023-12-01T03:12:41.842 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:41.845 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:12:41.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.844+0000 7f36a508c640 1 -- 172.21.15.115:0/2540849036 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f368006a670 msgr2=0x7f368006cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.844+0000 7f36a508c640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f368006a670 0x7f368006cb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3688036000 tx=0x7f368803c040 comp rx=0 tx=0).stop 2023-12-01T03:12:41.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.844+0000 7f36a508c640 1 -- 172.21.15.115:0/2540849036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a006b7b0 msgr2=0x7f36a01494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:41.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.844+0000 7f36a508c640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a006b7b0 0x7f36a01494a0 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7f369000cf60 tx=0x7f369000cf90 comp rx=0 tx=0).stop 2023-12-01T03:12:41.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.844+0000 7f36a508c640 1 -- 172.21.15.115:0/2540849036 shutdown_connections 2023-12-01T03:12:41.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.844+0000 7f36a508c640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f368006a670 0x7f368006cb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.844+0000 7f36a508c640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36a01584b0 0x7f36a00c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.844+0000 7f36a508c640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36a015afe0 0x7f36a00ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.844+0000 7f36a508c640 1 --2- 172.21.15.115:0/2540849036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36a006b7b0 0x7f36a01494a0 unknown :-1 s=CLOSED pgs=2981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:41.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.844+0000 7f36a508c640 1 -- 172.21.15.115:0/2540849036 >> 172.21.15.115:0/2540849036 conn(0x7f36a00ce640 msgr2=0x7f36a01541e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:41.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.845+0000 7f36a508c640 1 -- 172.21.15.115:0/2540849036 shutdown_connections 2023-12-01T03:12:41.847 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:41.845+0000 7f36a508c640 1 -- 172.21.15.115:0/2540849036 wait complete. 2023-12-01T03:12:41.865 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-12-01T03:12:41.049598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":644,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":398,"ondisk_log_size":398,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":443784,"kb_used_data":26440,"kb_used_omap":0,"kb_used_meta":417280,"kb_avail":754530936,"statfs":{"total":773094113280,"available":772639678464,"internally_reserved":0,"allocated":27074560,"data_stored":13909084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":427294720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.216727"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.356633+0000","last_change":"2023-12-01T03:12:35.346245+0000","last_active":"2023-12-01T03:12:35.356633+0000","last_peered":"2023-12-01T03:12:35.356633+0000","last_clean":"2023-12-01T03:12:35.356633+0000","last_became_active":"2023-12-01T03:12:35.345959+0000","last_became_peered":"2023-12-01T03:12:35.345959+0000","last_unstale":"2023-12-01T03:12:35.356633+0000","last_undegraded":"2023-12-01T03:12:35.356633+0000","last_fullsized":"2023-12-01T03:12:35.356633+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:21:23.692942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.347898+0000","last_change":"2023-12-01T03:12:35.346575+0000","last_active":"2023-12-01T03:12:35.347898+0000","last_peered":"2023-12-01T03:12:35.347898+0000","last_clean":"2023-12-01T03:12:35.347898+0000","last_became_active":"2023-12-01T03:12:35.346315+0000","last_became_peered":"2023-12-01T03:12:35.346315+0000","last_unstale":"2023-12-01T03:12:35.347898+0000","last_undegraded":"2023-12-01T03:12:35.347898+0000","last_fullsized":"2023-12-01T03:12:35.347898+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357412+0000","last_change":"2023-12-01T03:12:35.349364+0000","last_active":"2023-12-01T03:12:35.357412+0000","last_peered":"2023-12-01T03:12:35.357412+0000","last_clean":"2023-12-01T03:12:35.357412+0000","last_became_active":"2023-12-01T03:12:35.347941+0000","last_became_peered":"2023-12-01T03:12:35.347941+0000","last_unstale":"2023-12-01T03:12:35.357412+0000","last_undegraded":"2023-12-01T03:12:35.357412+0000","last_fullsized":"2023-12-01T03:12:35.357412+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358161+0000","last_change":"2023-12-01T03:12:35.353122+0000","last_active":"2023-12-01T03:12:35.358161+0000","last_peered":"2023-12-01T03:12:35.358161+0000","last_clean":"2023-12-01T03:12:35.358161+0000","last_became_active":"2023-12-01T03:12:35.352880+0000","last_became_peered":"2023-12-01T03:12:35.352880+0000","last_unstale":"2023-12-01T03:12:35.358161+0000","last_undegraded":"2023-12-01T03:12:35.358161+0000","last_fullsized":"2023-12-01T03:12:35.358161+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:49.741119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358802+0000","last_change":"2023-12-01T03:12:35.356178+0000","last_active":"2023-12-01T03:12:35.358802+0000","last_peered":"2023-12-01T03:12:35.358802+0000","last_clean":"2023-12-01T03:12:35.358802+0000","last_became_active":"2023-12-01T03:12:35.354822+0000","last_became_peered":"2023-12-01T03:12:35.354822+0000","last_unstale":"2023-12-01T03:12:35.358802+0000","last_undegraded":"2023-12-01T03:12:35.358802+0000","last_fullsized":"2023-12-01T03:12:35.358802+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.354894+0000","last_change":"2023-12-01T03:12:35.350584+0000","last_active":"2023-12-01T03:12:35.354894+0000","last_peered":"2023-12-01T03:12:35.354894+0000","last_clean":"2023-12-01T03:12:35.354894+0000","last_became_active":"2023-12-01T03:12:35.346833+0000","last_became_peered":"2023-12-01T03:12:35.346833+0000","last_unstale":"2023-12-01T03:12:35.354894+0000","last_undegraded":"2023-12-01T03:12:35.354894+0000","last_fullsized":"2023-12-01T03:12:35.354894+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:04.481100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358611+0000","last_change":"2023-12-01T03:12:35.355731+0000","last_active":"2023-12-01T03:12:35.358611+0000","last_peered":"2023-12-01T03:12:35.358611+0000","last_clean":"2023-12-01T03:12:35.358611+0000","last_became_active":"2023-12-01T03:12:35.355381+0000","last_became_peered":"2023-12-01T03:12:35.355381+0000","last_unstale":"2023-12-01T03:12:35.358611+0000","last_undegraded":"2023-12-01T03:12:35.358611+0000","last_fullsized":"2023-12-01T03:12:35.358611+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360133+0000","last_change":"2023-12-01T03:12:35.355271+0000","last_active":"2023-12-01T03:12:35.360133+0000","last_peered":"2023-12-01T03:12:35.360133+0000","last_clean":"2023-12-01T03:12:35.360133+0000","last_became_active":"2023-12-01T03:12:35.355020+0000","last_became_peered":"2023-12-01T03:12:35.355020+0000","last_unstale":"2023-12-01T03:12:35.360133+0000","last_undegraded":"2023-12-01T03:12:35.360133+0000","last_fullsized":"2023-12-01T03:12:35.360133+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:27.055115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358363+0000","last_change":"2023-12-01T03:12:35.353249+0000","last_active":"2023-12-01T03:12:35.358363+0000","last_peered":"2023-12-01T03:12:35.358363+0000","last_clean":"2023-12-01T03:12:35.358363+0000","last_became_active":"2023-12-01T03:12:35.352969+0000","last_became_peered":"2023-12-01T03:12:35.352969+0000","last_unstale":"2023-12-01T03:12:35.358363+0000","last_undegraded":"2023-12-01T03:12:35.358363+0000","last_fullsized":"2023-12-01T03:12:35.358363+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:17:30.864087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357144+0000","last_change":"2023-12-01T03:12:35.355534+0000","last_active":"2023-12-01T03:12:35.357144+0000","last_peered":"2023-12-01T03:12:35.357144+0000","last_clean":"2023-12-01T03:12:35.357144+0000","last_became_active":"2023-12-01T03:12:35.354708+0000","last_became_peered":"2023-12-01T03:12:35.354708+0000","last_unstale":"2023-12-01T03:12:35.357144+0000","last_undegraded":"2023-12-01T03:12:35.357144+0000","last_fullsized":"2023-12-01T03:12:35.357144+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358112+0000","last_change":"2023-12-01T03:12:35.351076+0000","last_active":"2023-12-01T03:12:35.358112+0000","last_peered":"2023-12-01T03:12:35.358112+0000","last_clean":"2023-12-01T03:12:35.358112+0000","last_became_active":"2023-12-01T03:12:35.350834+0000","last_became_peered":"2023-12-01T03:12:35.350834+0000","last_unstale":"2023-12-01T03:12:35.358112+0000","last_undegraded":"2023-12-01T03:12:35.358112+0000","last_fullsized":"2023-12-01T03:12:35.358112+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358471+0000","last_change":"2023-12-01T03:12:35.350756+0000","last_active":"2023-12-01T03:12:35.358471+0000","last_peered":"2023-12-01T03:12:35.358471+0000","last_clean":"2023-12-01T03:12:35.358471+0000","last_became_active":"2023-12-01T03:12:35.350503+0000","last_became_peered":"2023-12-01T03:12:35.350503+0000","last_unstale":"2023-12-01T03:12:35.358471+0000","last_undegraded":"2023-12-01T03:12:35.358471+0000","last_fullsized":"2023-12-01T03:12:35.358471+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.361001+0000","last_change":"2023-12-01T03:12:35.358627+0000","last_active":"2023-12-01T03:12:35.361001+0000","last_peered":"2023-12-01T03:12:35.361001+0000","last_clean":"2023-12-01T03:12:35.361001+0000","last_became_active":"2023-12-01T03:12:35.357120+0000","last_became_peered":"2023-12-01T03:12:35.357120+0000","last_unstale":"2023-12-01T03:12:35.361001+0000","last_undegraded":"2023-12-01T03:12:35.361001+0000","last_fullsized":"2023-12-01T03:12:35.361001+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:54:17.936247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.361020+0000","last_change":"2023-12-01T03:12:35.356909+0000","last_active":"2023-12-01T03:12:35.361020+0000","last_peered":"2023-12-01T03:12:35.361020+0000","last_clean":"2023-12-01T03:12:35.361020+0000","last_became_active":"2023-12-01T03:12:35.356405+0000","last_became_peered":"2023-12-01T03:12:35.356405+0000","last_unstale":"2023-12-01T03:12:35.361020+0000","last_undegraded":"2023-12-01T03:12:35.361020+0000","last_fullsized":"2023-12-01T03:12:35.361020+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:59:25.105386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.353415+0000","last_change":"2023-12-01T03:12:35.348321+0000","last_active":"2023-12-01T03:12:35.353415+0000","last_peered":"2023-12-01T03:12:35.353415+0000","last_clean":"2023-12-01T03:12:35.353415+0000","last_became_active":"2023-12-01T03:12:35.344564+0000","last_became_peered":"2023-12-01T03:12:35.344564+0000","last_unstale":"2023-12-01T03:12:35.353415+0000","last_undegraded":"2023-12-01T03:12:35.353415+0000","last_fullsized":"2023-12-01T03:12:35.353415+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350752+0000","last_change":"2023-12-01T03:12:35.349640+0000","last_active":"2023-12-01T03:12:35.350752+0000","last_peered":"2023-12-01T03:12:35.350752+0000","last_clean":"2023-12-01T03:12:35.350752+0000","last_became_active":"2023-12-01T03:12:35.348717+0000","last_became_peered":"2023-12-01T03:12:35.348717+0000","last_unstale":"2023-12-01T03:12:35.350752+0000","last_undegraded":"2023-12-01T03:12:35.350752+0000","last_fullsized":"2023-12-01T03:12:35.350752+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350655+0000","last_change":"2023-12-01T03:12:35.343916+0000","last_active":"2023-12-01T03:12:35.350655+0000","last_peered":"2023-12-01T03:12:35.350655+0000","last_clean":"2023-12-01T03:12:35.350655+0000","last_became_active":"2023-12-01T03:12:35.341639+0000","last_became_peered":"2023-12-01T03:12:35.341639+0000","last_unstale":"2023-12-01T03:12:35.350655+0000","last_undegraded":"2023-12-01T03:12:35.350655+0000","last_fullsized":"2023-12-01T03:12:35.350655+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:21.877813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355343+0000","last_change":"2023-12-01T03:12:35.349529+0000","last_active":"2023-12-01T03:12:35.355343+0000","last_peered":"2023-12-01T03:12:35.355343+0000","last_clean":"2023-12-01T03:12:35.355343+0000","last_became_active":"2023-12-01T03:12:35.349233+0000","last_became_peered":"2023-12-01T03:12:35.349233+0000","last_unstale":"2023-12-01T03:12:35.355343+0000","last_undegraded":"2023-12-01T03:12:35.355343+0000","last_fullsized":"2023-12-01T03:12:35.355343+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357431+0000","last_change":"2023-12-01T03:12:35.348866+0000","last_active":"2023-12-01T03:12:35.357431+0000","last_peered":"2023-12-01T03:12:35.357431+0000","last_clean":"2023-12-01T03:12:35.357431+0000","last_became_active":"2023-12-01T03:12:35.346590+0000","last_became_peered":"2023-12-01T03:12:35.346590+0000","last_unstale":"2023-12-01T03:12:35.357431+0000","last_undegraded":"2023-12-01T03:12:35.357431+0000","last_fullsized":"2023-12-01T03:12:35.357431+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.345236+0000","last_change":"2023-12-01T03:12:35.343338+0000","last_active":"2023-12-01T03:12:35.345236+0000","last_peered":"2023-12-01T03:12:35.345236+0000","last_clean":"2023-12-01T03:12:35.345236+0000","last_became_active":"2023-12-01T03:12:35.343036+0000","last_became_peered":"2023-12-01T03:12:35.343036+0000","last_unstale":"2023-12-01T03:12:35.345236+0000","last_undegraded":"2023-12-01T03:12:35.345236+0000","last_fullsized":"2023-12-01T03:12:35.345236+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357492+0000","last_change":"2023-12-01T03:12:35.352676+0000","last_active":"2023-12-01T03:12:35.357492+0000","last_peered":"2023-12-01T03:12:35.357492+0000","last_clean":"2023-12-01T03:12:35.357492+0000","last_became_active":"2023-12-01T03:12:35.352425+0000","last_became_peered":"2023-12-01T03:12:35.352425+0000","last_unstale":"2023-12-01T03:12:35.357492+0000","last_undegraded":"2023-12-01T03:12:35.357492+0000","last_fullsized":"2023-12-01T03:12:35.357492+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:31.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355176+0000","last_change":"2023-12-01T03:12:35.347876+0000","last_active":"2023-12-01T03:12:35.355176+0000","last_peered":"2023-12-01T03:12:35.355176+0000","last_clean":"2023-12-01T03:12:35.355176+0000","last_became_active":"2023-12-01T03:12:35.347100+0000","last_became_peered":"2023-12-01T03:12:35.347100+0000","last_unstale":"2023-12-01T03:12:35.355176+0000","last_undegraded":"2023-12-01T03:12:35.355176+0000","last_fullsized":"2023-12-01T03:12:35.355176+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:59:04.995090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357520+0000","last_change":"2023-12-01T03:12:35.349609+0000","last_active":"2023-12-01T03:12:35.357520+0000","last_peered":"2023-12-01T03:12:35.357520+0000","last_clean":"2023-12-01T03:12:35.357520+0000","last_became_active":"2023-12-01T03:12:35.349088+0000","last_became_peered":"2023-12-01T03:12:35.349088+0000","last_unstale":"2023-12-01T03:12:35.357520+0000","last_undegraded":"2023-12-01T03:12:35.357520+0000","last_fullsized":"2023-12-01T03:12:35.357520+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.349549+0000","last_change":"2023-12-01T03:12:35.343936+0000","last_active":"2023-12-01T03:12:35.349549+0000","last_peered":"2023-12-01T03:12:35.349549+0000","last_clean":"2023-12-01T03:12:35.349549+0000","last_became_active":"2023-12-01T03:12:35.343638+0000","last_became_peered":"2023-12-01T03:12:35.343638+0000","last_unstale":"2023-12-01T03:12:35.349549+0000","last_undegraded":"2023-12-01T03:12:35.349549+0000","last_fullsized":"2023-12-01T03:12:35.349549+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:00:57.853346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355218+0000","last_change":"2023-12-01T03:12:35.347714+0000","last_active":"2023-12-01T03:12:35.355218+0000","last_peered":"2023-12-01T03:12:35.355218+0000","last_clean":"2023-12-01T03:12:35.355218+0000","last_became_active":"2023-12-01T03:12:35.346230+0000","last_became_peered":"2023-12-01T03:12:35.346230+0000","last_unstale":"2023-12-01T03:12:35.355218+0000","last_undegraded":"2023-12-01T03:12:35.355218+0000","last_fullsized":"2023-12-01T03:12:35.355218+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:34.404254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350731+0000","last_change":"2023-12-01T03:12:35.348505+0000","last_active":"2023-12-01T03:12:35.350731+0000","last_peered":"2023-12-01T03:12:35.350731+0000","last_clean":"2023-12-01T03:12:35.350731+0000","last_became_active":"2023-12-01T03:12:35.345808+0000","last_became_peered":"2023-12-01T03:12:35.345808+0000","last_unstale":"2023-12-01T03:12:35.350731+0000","last_undegraded":"2023-12-01T03:12:35.350731+0000","last_fullsized":"2023-12-01T03:12:35.350731+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:00.577497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.351196+0000","last_change":"2023-12-01T03:12:35.348639+0000","last_active":"2023-12-01T03:12:35.351196+0000","last_peered":"2023-12-01T03:12:35.351196+0000","last_clean":"2023-12-01T03:12:35.351196+0000","last_became_active":"2023-12-01T03:12:35.348394+0000","last_became_peered":"2023-12-01T03:12:35.348394+0000","last_unstale":"2023-12-01T03:12:35.351196+0000","last_undegraded":"2023-12-01T03:12:35.351196+0000","last_fullsized":"2023-12-01T03:12:35.351196+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:09:42.224008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.361185+0000","last_change":"2023-12-01T03:12:35.356530+0000","last_active":"2023-12-01T03:12:35.361185+0000","last_peered":"2023-12-01T03:12:35.361185+0000","last_clean":"2023-12-01T03:12:35.361185+0000","last_became_active":"2023-12-01T03:12:35.356231+0000","last_became_peered":"2023-12-01T03:12:35.356231+0000","last_unstale":"2023-12-01T03:12:35.361185+0000","last_undegraded":"2023-12-01T03:12:35.361185+0000","last_fullsized":"2023-12-01T03:12:35.361185+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:53:08.535342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.352646+0000","last_change":"2023-12-01T03:12:35.348165+0000","last_active":"2023-12-01T03:12:35.352646+0000","last_peered":"2023-12-01T03:12:35.352646+0000","last_clean":"2023-12-01T03:12:35.352646+0000","last_became_active":"2023-12-01T03:12:35.347680+0000","last_became_peered":"2023-12-01T03:12:35.347680+0000","last_unstale":"2023-12-01T03:12:35.352646+0000","last_undegraded":"2023-12-01T03:12:35.352646+0000","last_fullsized":"2023-12-01T03:12:35.352646+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:50:37.148015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.361040+0000","last_change":"2023-12-01T03:12:35.357739+0000","last_active":"2023-12-01T03:12:35.361040+0000","last_peered":"2023-12-01T03:12:35.361040+0000","last_clean":"2023-12-01T03:12:35.361040+0000","last_became_active":"2023-12-01T03:12:35.355379+0000","last_became_peered":"2023-12-01T03:12:35.355379+0000","last_unstale":"2023-12-01T03:12:35.361040+0000","last_undegraded":"2023-12-01T03:12:35.361040+0000","last_fullsized":"2023-12-01T03:12:35.361040+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:25.025253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357725+0000","last_change":"2023-12-01T03:12:35.352593+0000","last_active":"2023-12-01T03:12:35.357725+0000","last_peered":"2023-12-01T03:12:35.357725+0000","last_clean":"2023-12-01T03:12:35.357725+0000","last_became_active":"2023-12-01T03:12:35.350140+0000","last_became_peered":"2023-12-01T03:12:35.350140+0000","last_unstale":"2023-12-01T03:12:35.357725+0000","last_undegraded":"2023-12-01T03:12:35.357725+0000","last_fullsized":"2023-12-01T03:12:35.357725+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350697+0000","last_change":"2023-12-01T03:12:35.346395+0000","last_active":"2023-12-01T03:12:35.350697+0000","last_peered":"2023-12-01T03:12:35.350697+0000","last_clean":"2023-12-01T03:12:35.350697+0000","last_became_active":"2023-12-01T03:12:35.344072+0000","last_became_peered":"2023-12-01T03:12:35.344072+0000","last_unstale":"2023-12-01T03:12:35.350697+0000","last_undegraded":"2023-12-01T03:12:35.350697+0000","last_fullsized":"2023-12-01T03:12:35.350697+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:10:45.146806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1f","version":"255'4","reported_seq":15,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.365834+0000","last_change":"2023-12-01T03:12:34.867899+0000","last_active":"2023-12-01T03:12:35.365834+0000","last_peered":"2023-12-01T03:12:35.365834+0000","last_clean":"2023-12-01T03:12:35.365834+0000","last_became_active":"2023-12-01T03:12:34.867559+0000","last_became_peered":"2023-12-01T03:12:34.867559+0000","last_unstale":"2023-12-01T03:12:35.365834+0000","last_undegraded":"2023-12-01T03:12:35.365834+0000","last_fullsized":"2023-12-01T03:12:35.365834+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:49.066648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350673+0000","last_change":"2023-12-01T03:12:35.348505+0000","last_active":"2023-12-01T03:12:35.350673+0000","last_peered":"2023-12-01T03:12:35.350673+0000","last_clean":"2023-12-01T03:12:35.350673+0000","last_became_active":"2023-12-01T03:12:35.347836+0000","last_became_peered":"2023-12-01T03:12:35.347836+0000","last_unstale":"2023-12-01T03:12:35.350673+0000","last_undegraded":"2023-12-01T03:12:35.350673+0000","last_fullsized":"2023-12-01T03:12:35.350673+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:59.927209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.1e","version":"255'1","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.367023+0000","last_change":"2023-12-01T03:12:34.868270+0000","last_active":"2023-12-01T03:12:35.367023+0000","last_peered":"2023-12-01T03:12:35.367023+0000","last_clean":"2023-12-01T03:12:35.367023+0000","last_became_active":"2023-12-01T03:12:34.867687+0000","last_became_peered":"2023-12-01T03:12:34.867687+0000","last_unstale":"2023-12-01T03:12:35.367023+0000","last_undegraded":"2023-12-01T03:12:35.367023+0000","last_fullsized":"2023-12-01T03:12:35.367023+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:03:51.996656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.359356+0000","last_change":"2023-12-01T03:12:35.355354+0000","last_active":"2023-12-01T03:12:35.359356+0000","last_peered":"2023-12-01T03:12:35.359356+0000","last_clean":"2023-12-01T03:12:35.359356+0000","last_became_active":"2023-12-01T03:12:35.354183+0000","last_became_peered":"2023-12-01T03:12:35.354183+0000","last_unstale":"2023-12-01T03:12:35.359356+0000","last_undegraded":"2023-12-01T03:12:35.359356+0000","last_fullsized":"2023-12-01T03:12:35.359356+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1d","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.364743+0000","last_change":"2023-12-01T03:12:34.869563+0000","last_active":"2023-12-01T03:12:35.364743+0000","last_peered":"2023-12-01T03:12:35.364743+0000","last_clean":"2023-12-01T03:12:35.364743+0000","last_became_active":"2023-12-01T03:12:34.869160+0000","last_became_peered":"2023-12-01T03:12:34.869160+0000","last_unstale":"2023-12-01T03:12:35.364743+0000","last_undegraded":"2023-12-01T03:12:35.364743+0000","last_fullsized":"2023-12-01T03:12:35.364743+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:19.315026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.354616+0000","last_change":"2023-12-01T03:12:35.351477+0000","last_active":"2023-12-01T03:12:35.354616+0000","last_peered":"2023-12-01T03:12:35.354616+0000","last_clean":"2023-12-01T03:12:35.354616+0000","last_became_active":"2023-12-01T03:12:35.351219+0000","last_became_peered":"2023-12-01T03:12:35.351219+0000","last_unstale":"2023-12-01T03:12:35.354616+0000","last_undegraded":"2023-12-01T03:12:35.354616+0000","last_fullsized":"2023-12-01T03:12:35.354616+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1c","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.364480+0000","last_change":"2023-12-01T03:12:34.865447+0000","last_active":"2023-12-01T03:12:35.364480+0000","last_peered":"2023-12-01T03:12:35.364480+0000","last_clean":"2023-12-01T03:12:35.364480+0000","last_became_active":"2023-12-01T03:12:34.865092+0000","last_became_peered":"2023-12-01T03:12:34.865092+0000","last_unstale":"2023-12-01T03:12:35.364480+0000","last_undegraded":"2023-12-01T03:12:35.364480+0000","last_fullsized":"2023-12-01T03:12:35.364480+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:55:07.934979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355407+0000","last_change":"2023-12-01T03:12:35.352631+0000","last_active":"2023-12-01T03:12:35.355407+0000","last_peered":"2023-12-01T03:12:35.355407+0000","last_clean":"2023-12-01T03:12:35.355407+0000","last_became_active":"2023-12-01T03:12:35.351948+0000","last_became_peered":"2023-12-01T03:12:35.351948+0000","last_unstale":"2023-12-01T03:12:35.355407+0000","last_undegraded":"2023-12-01T03:12:35.355407+0000","last_fullsized":"2023-12-01T03:12:35.355407+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1b","version":"255'1","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358726+0000","last_change":"2023-12-01T03:12:34.857676+0000","last_active":"2023-12-01T03:12:35.358726+0000","last_peered":"2023-12-01T03:12:35.358726+0000","last_clean":"2023-12-01T03:12:35.358726+0000","last_became_active":"2023-12-01T03:12:34.857365+0000","last_became_peered":"2023-12-01T03:12:34.857365+0000","last_unstale":"2023-12-01T03:12:35.358726+0000","last_undegraded":"2023-12-01T03:12:35.358726+0000","last_fullsized":"2023-12-01T03:12:35.358726+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:22.088313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.349025+0000","last_change":"2023-12-01T03:12:35.345935+0000","last_active":"2023-12-01T03:12:35.349025+0000","last_peered":"2023-12-01T03:12:35.349025+0000","last_clean":"2023-12-01T03:12:35.349025+0000","last_became_active":"2023-12-01T03:12:35.344888+0000","last_became_peered":"2023-12-01T03:12:35.344888+0000","last_unstale":"2023-12-01T03:12:35.349025+0000","last_undegraded":"2023-12-01T03:12:35.349025+0000","last_fullsized":"2023-12-01T03:12:35.349025+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:13.355768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1a","version":"255'1","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.363324+0000","last_change":"2023-12-01T03:12:34.860204+0000","last_active":"2023-12-01T03:12:35.363324+0000","last_peered":"2023-12-01T03:12:35.363324+0000","last_clean":"2023-12-01T03:12:35.363324+0000","last_became_active":"2023-12-01T03:12:34.858034+0000","last_became_peered":"2023-12-01T03:12:34.858034+0000","last_unstale":"2023-12-01T03:12:35.363324+0000","last_undegraded":"2023-12-01T03:12:35.363324+0000","last_fullsized":"2023-12-01T03:12:35.363324+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:23:38.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350169+0000","last_change":"2023-12-01T03:12:35.342560+0000","last_active":"2023-12-01T03:12:35.350169+0000","last_peered":"2023-12-01T03:12:35.350169+0000","last_clean":"2023-12-01T03:12:35.350169+0000","last_became_active":"2023-12-01T03:12:35.342293+0000","last_became_peered":"2023-12-01T03:12:35.342293+0000","last_unstale":"2023-12-01T03:12:35.350169+0000","last_undegraded":"2023-12-01T03:12:35.350169+0000","last_fullsized":"2023-12-01T03:12:35.350169+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.338718+0000","last_change":"2023-12-01T03:12:34.863171+0000","last_active":"2023-12-01T03:12:35.338718+0000","last_peered":"2023-12-01T03:12:35.338718+0000","last_clean":"2023-12-01T03:12:35.338718+0000","last_became_active":"2023-12-01T03:12:34.862887+0000","last_became_peered":"2023-12-01T03:12:34.862887+0000","last_unstale":"2023-12-01T03:12:35.338718+0000","last_undegraded":"2023-12-01T03:12:35.338718+0000","last_fullsized":"2023-12-01T03:12:35.338718+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:51:11.709287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357903+0000","last_change":"2023-12-01T03:12:35.354236+0000","last_active":"2023-12-01T03:12:35.357903+0000","last_peered":"2023-12-01T03:12:35.357903+0000","last_clean":"2023-12-01T03:12:35.357903+0000","last_became_active":"2023-12-01T03:12:35.350478+0000","last_became_peered":"2023-12-01T03:12:35.350478+0000","last_unstale":"2023-12-01T03:12:35.357903+0000","last_undegraded":"2023-12-01T03:12:35.357903+0000","last_fullsized":"2023-12-01T03:12:35.357903+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.336460+0000","last_change":"2023-12-01T03:12:34.864288+0000","last_active":"2023-12-01T03:12:35.336460+0000","last_peered":"2023-12-01T03:12:35.336460+0000","last_clean":"2023-12-01T03:12:35.336460+0000","last_became_active":"2023-12-01T03:12:34.863960+0000","last_became_peered":"2023-12-01T03:12:34.863960+0000","last_unstale":"2023-12-01T03:12:35.336460+0000","last_undegraded":"2023-12-01T03:12:35.336460+0000","last_fullsized":"2023-12-01T03:12:35.336460+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:02.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357796+0000","last_change":"2023-12-01T03:12:35.352373+0000","last_active":"2023-12-01T03:12:35.357796+0000","last_peered":"2023-12-01T03:12:35.357796+0000","last_clean":"2023-12-01T03:12:35.357796+0000","last_became_active":"2023-12-01T03:12:35.352063+0000","last_became_peered":"2023-12-01T03:12:35.352063+0000","last_unstale":"2023-12-01T03:12:35.357796+0000","last_undegraded":"2023-12-01T03:12:35.357796+0000","last_fullsized":"2023-12-01T03:12:35.357796+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:19.476510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.329796+0000","last_change":"2023-12-01T03:12:34.867539+0000","last_active":"2023-12-01T03:12:35.329796+0000","last_peered":"2023-12-01T03:12:35.329796+0000","last_clean":"2023-12-01T03:12:35.329796+0000","last_became_active":"2023-12-01T03:12:34.866362+0000","last_became_peered":"2023-12-01T03:12:34.866362+0000","last_unstale":"2023-12-01T03:12:35.329796+0000","last_undegraded":"2023-12-01T03:12:35.329796+0000","last_fullsized":"2023-12-01T03:12:35.329796+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:52.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360293+0000","last_change":"2023-12-01T03:12:35.359264+0000","last_active":"2023-12-01T03:12:35.360293+0000","last_peered":"2023-12-01T03:12:35.360293+0000","last_clean":"2023-12-01T03:12:35.360293+0000","last_became_active":"2023-12-01T03:12:35.358987+0000","last_became_peered":"2023-12-01T03:12:35.358987+0000","last_unstale":"2023-12-01T03:12:35.360293+0000","last_undegraded":"2023-12-01T03:12:35.360293+0000","last_fullsized":"2023-12-01T03:12:35.360293+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.340265+0000","last_change":"2023-12-01T03:12:34.859136+0000","last_active":"2023-12-01T03:12:35.340265+0000","last_peered":"2023-12-01T03:12:35.340265+0000","last_clean":"2023-12-01T03:12:35.340265+0000","last_became_active":"2023-12-01T03:12:34.857840+0000","last_became_peered":"2023-12-01T03:12:34.857840+0000","last_unstale":"2023-12-01T03:12:35.340265+0000","last_undegraded":"2023-12-01T03:12:35.340265+0000","last_fullsized":"2023-12-01T03:12:35.340265+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:36.726453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.349687+0000","last_change":"2023-12-01T03:12:35.346099+0000","last_active":"2023-12-01T03:12:35.349687+0000","last_peered":"2023-12-01T03:12:35.349687+0000","last_clean":"2023-12-01T03:12:35.349687+0000","last_became_active":"2023-12-01T03:12:35.345823+0000","last_became_peered":"2023-12-01T03:12:35.345823+0000","last_unstale":"2023-12-01T03:12:35.349687+0000","last_undegraded":"2023-12-01T03:12:35.349687+0000","last_fullsized":"2023-12-01T03:12:35.349687+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.15","version":"255'3","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.362807+0000","last_change":"2023-12-01T03:12:34.860533+0000","last_active":"2023-12-01T03:12:35.362807+0000","last_peered":"2023-12-01T03:12:35.362807+0000","last_clean":"2023-12-01T03:12:35.362807+0000","last_became_active":"2023-12-01T03:12:34.858865+0000","last_became_peered":"2023-12-01T03:12:34.858865+0000","last_unstale":"2023-12-01T03:12:35.362807+0000","last_undegraded":"2023-12-01T03:12:35.362807+0000","last_fullsized":"2023-12-01T03:12:35.362807+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:22.088313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360472+0000","last_change":"2023-12-01T03:12:35.354831+0000","last_active":"2023-12-01T03:12:35.360472+0000","last_peered":"2023-12-01T03:12:35.360472+0000","last_clean":"2023-12-01T03:12:35.360472+0000","last_became_active":"2023-12-01T03:12:35.351242+0000","last_became_peered":"2023-12-01T03:12:35.351242+0000","last_unstale":"2023-12-01T03:12:35.360472+0000","last_undegraded":"2023-12-01T03:12:35.360472+0000","last_fullsized":"2023-12-01T03:12:35.360472+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.14","version":"255'6","reported_seq":17,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.366096+0000","last_change":"2023-12-01T03:12:34.857964+0000","last_active":"2023-12-01T03:12:35.366096+0000","last_peered":"2023-12-01T03:12:35.366096+0000","last_clean":"2023-12-01T03:12:35.366096+0000","last_became_active":"2023-12-01T03:12:34.856582+0000","last_became_peered":"2023-12-01T03:12:34.856582+0000","last_unstale":"2023-12-01T03:12:35.366096+0000","last_undegraded":"2023-12-01T03:12:35.366096+0000","last_fullsized":"2023-12-01T03:12:35.366096+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:48.550869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360788+0000","last_change":"2023-12-01T03:12:35.356681+0000","last_active":"2023-12-01T03:12:35.360788+0000","last_peered":"2023-12-01T03:12:35.360788+0000","last_clean":"2023-12-01T03:12:35.360788+0000","last_became_active":"2023-12-01T03:12:35.354973+0000","last_became_peered":"2023-12-01T03:12:35.354973+0000","last_unstale":"2023-12-01T03:12:35.360788+0000","last_undegraded":"2023-12-01T03:12:35.360788+0000","last_fullsized":"2023-12-01T03:12:35.360788+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:23.725637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.13","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.365830+0000","last_change":"2023-12-01T03:12:34.863097+0000","last_active":"2023-12-01T03:12:35.365830+0000","last_peered":"2023-12-01T03:12:35.365830+0000","last_clean":"2023-12-01T03:12:35.365830+0000","last_became_active":"2023-12-01T03:12:34.862657+0000","last_became_peered":"2023-12-01T03:12:34.862657+0000","last_unstale":"2023-12-01T03:12:35.365830+0000","last_undegraded":"2023-12-01T03:12:35.365830+0000","last_fullsized":"2023-12-01T03:12:35.365830+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:22.709559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.352624+0000","last_change":"2023-12-01T03:12:35.347992+0000","last_active":"2023-12-01T03:12:35.352624+0000","last_peered":"2023-12-01T03:12:35.352624+0000","last_clean":"2023-12-01T03:12:35.352624+0000","last_became_active":"2023-12-01T03:12:35.347364+0000","last_became_peered":"2023-12-01T03:12:35.347364+0000","last_unstale":"2023-12-01T03:12:35.352624+0000","last_undegraded":"2023-12-01T03:12:35.352624+0000","last_fullsized":"2023-12-01T03:12:35.352624+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.337682+0000","last_change":"2023-12-01T03:12:34.860804+0000","last_active":"2023-12-01T03:12:35.337682+0000","last_peered":"2023-12-01T03:12:35.337682+0000","last_clean":"2023-12-01T03:12:35.337682+0000","last_became_active":"2023-12-01T03:12:34.859477+0000","last_became_peered":"2023-12-01T03:12:34.859477+0000","last_unstale":"2023-12-01T03:12:35.337682+0000","last_undegraded":"2023-12-01T03:12:35.337682+0000","last_fullsized":"2023-12-01T03:12:35.337682+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:00.153556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355384+0000","last_change":"2023-12-01T03:12:35.351909+0000","last_active":"2023-12-01T03:12:35.355384+0000","last_peered":"2023-12-01T03:12:35.355384+0000","last_clean":"2023-12-01T03:12:35.355384+0000","last_became_active":"2023-12-01T03:12:35.351434+0000","last_became_peered":"2023-12-01T03:12:35.351434+0000","last_unstale":"2023-12-01T03:12:35.355384+0000","last_undegraded":"2023-12-01T03:12:35.355384+0000","last_fullsized":"2023-12-01T03:12:35.355384+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:04:09.892305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.336924+0000","last_change":"2023-12-01T03:12:34.860578+0000","last_active":"2023-12-01T03:12:35.336924+0000","last_peered":"2023-12-01T03:12:35.336924+0000","last_clean":"2023-12-01T03:12:35.336924+0000","last_became_active":"2023-12-01T03:12:34.859107+0000","last_became_peered":"2023-12-01T03:12:34.859107+0000","last_unstale":"2023-12-01T03:12:35.336924+0000","last_undegraded":"2023-12-01T03:12:35.336924+0000","last_fullsized":"2023-12-01T03:12:35.336924+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:23:38.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360834+0000","last_change":"2023-12-01T03:12:35.354180+0000","last_active":"2023-12-01T03:12:35.360834+0000","last_peered":"2023-12-01T03:12:35.360834+0000","last_clean":"2023-12-01T03:12:35.360834+0000","last_became_active":"2023-12-01T03:12:35.353835+0000","last_became_peered":"2023-12-01T03:12:35.353835+0000","last_unstale":"2023-12-01T03:12:35.360834+0000","last_undegraded":"2023-12-01T03:12:35.360834+0000","last_fullsized":"2023-12-01T03:12:35.360834+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.10","version":"255'8","reported_seq":19,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.367485+0000","last_change":"2023-12-01T03:12:34.866244+0000","last_active":"2023-12-01T03:12:35.367485+0000","last_peered":"2023-12-01T03:12:35.367485+0000","last_clean":"2023-12-01T03:12:35.367485+0000","last_became_active":"2023-12-01T03:12:34.865944+0000","last_became_peered":"2023-12-01T03:12:34.865944+0000","last_unstale":"2023-12-01T03:12:35.367485+0000","last_undegraded":"2023-12-01T03:12:35.367485+0000","last_fullsized":"2023-12-01T03:12:35.367485+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:28.385262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.356768+0000","last_change":"2023-12-01T03:12:35.351685+0000","last_active":"2023-12-01T03:12:35.356768+0000","last_peered":"2023-12-01T03:12:35.356768+0000","last_clean":"2023-12-01T03:12:35.356768+0000","last_became_active":"2023-12-01T03:12:35.349403+0000","last_became_peered":"2023-12-01T03:12:35.349403+0000","last_unstale":"2023-12-01T03:12:35.356768+0000","last_undegraded":"2023-12-01T03:12:35.356768+0000","last_fullsized":"2023-12-01T03:12:35.356768+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.f","version":"255'1","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360105+0000","last_change":"2023-12-01T03:12:34.864936+0000","last_active":"2023-12-01T03:12:35.360105+0000","last_peered":"2023-12-01T03:12:35.360105+0000","last_clean":"2023-12-01T03:12:35.360105+0000","last_became_active":"2023-12-01T03:12:34.864646+0000","last_became_peered":"2023-12-01T03:12:34.864646+0000","last_unstale":"2023-12-01T03:12:35.360105+0000","last_undegraded":"2023-12-01T03:12:35.360105+0000","last_fullsized":"2023-12-01T03:12:35.360105+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:00.153556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355867+0000","last_change":"2023-12-01T03:12:35.352911+0000","last_active":"2023-12-01T03:12:35.355867+0000","last_peered":"2023-12-01T03:12:35.355867+0000","last_clean":"2023-12-01T03:12:35.355867+0000","last_became_active":"2023-12-01T03:12:35.352639+0000","last_became_peered":"2023-12-01T03:12:35.352639+0000","last_unstale":"2023-12-01T03:12:35.355867+0000","last_undegraded":"2023-12-01T03:12:35.355867+0000","last_fullsized":"2023-12-01T03:12:35.355867+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.e","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.364684+0000","last_change":"2023-12-01T03:12:34.866465+0000","last_active":"2023-12-01T03:12:35.364684+0000","last_peered":"2023-12-01T03:12:35.364684+0000","last_clean":"2023-12-01T03:12:35.364684+0000","last_became_active":"2023-12-01T03:12:34.865451+0000","last_became_peered":"2023-12-01T03:12:34.865451+0000","last_unstale":"2023-12-01T03:12:35.364684+0000","last_undegraded":"2023-12-01T03:12:35.364684+0000","last_fullsized":"2023-12-01T03:12:35.364684+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:17:29.848009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.349085+0000","last_change":"2023-12-01T03:12:35.346831+0000","last_active":"2023-12-01T03:12:35.349085+0000","last_peered":"2023-12-01T03:12:35.349085+0000","last_clean":"2023-12-01T03:12:35.349085+0000","last_became_active":"2023-12-01T03:12:35.346545+0000","last_became_peered":"2023-12-01T03:12:35.346545+0000","last_unstale":"2023-12-01T03:12:35.349085+0000","last_undegraded":"2023-12-01T03:12:35.349085+0000","last_fullsized":"2023-12-01T03:12:35.349085+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:03:37.465541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.d","version":"255'1","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.361836+0000","last_change":"2023-12-01T03:12:34.866633+0000","last_active":"2023-12-01T03:12:35.361836+0000","last_peered":"2023-12-01T03:12:35.361836+0000","last_clean":"2023-12-01T03:12:35.361836+0000","last_became_active":"2023-12-01T03:12:34.865565+0000","last_became_peered":"2023-12-01T03:12:34.865565+0000","last_unstale":"2023-12-01T03:12:35.361836+0000","last_undegraded":"2023-12-01T03:12:35.361836+0000","last_fullsized":"2023-12-01T03:12:35.361836+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:37.392214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358091+0000","last_change":"2023-12-01T03:12:35.348060+0000","last_active":"2023-12-01T03:12:35.358091+0000","last_peered":"2023-12-01T03:12:35.358091+0000","last_clean":"2023-12-01T03:12:35.358091+0000","last_became_active":"2023-12-01T03:12:35.347816+0000","last_became_peered":"2023-12-01T03:12:35.347816+0000","last_unstale":"2023-12-01T03:12:35.358091+0000","last_undegraded":"2023-12-01T03:12:35.358091+0000","last_fullsized":"2023-12-01T03:12:35.358091+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.338075+0000","last_change":"2023-12-01T03:12:34.858480+0000","last_active":"2023-12-01T03:12:35.338075+0000","last_peered":"2023-12-01T03:12:35.338075+0000","last_clean":"2023-12-01T03:12:35.338075+0000","last_became_active":"2023-12-01T03:12:34.857292+0000","last_became_peered":"2023-12-01T03:12:34.857292+0000","last_unstale":"2023-12-01T03:12:35.338075+0000","last_undegraded":"2023-12-01T03:12:35.338075+0000","last_fullsized":"2023-12-01T03:12:35.338075+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:02.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350714+0000","last_change":"2023-12-01T03:12:35.342072+0000","last_active":"2023-12-01T03:12:35.350714+0000","last_peered":"2023-12-01T03:12:35.350714+0000","last_clean":"2023-12-01T03:12:35.350714+0000","last_became_active":"2023-12-01T03:12:35.341786+0000","last_became_peered":"2023-12-01T03:12:35.341786+0000","last_unstale":"2023-12-01T03:12:35.350714+0000","last_undegraded":"2023-12-01T03:12:35.350714+0000","last_fullsized":"2023-12-01T03:12:35.350714+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.b","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.362890+0000","last_change":"2023-12-01T03:12:34.860490+0000","last_active":"2023-12-01T03:12:35.362890+0000","last_peered":"2023-12-01T03:12:35.362890+0000","last_clean":"2023-12-01T03:12:35.362890+0000","last_became_active":"2023-12-01T03:12:34.860228+0000","last_became_peered":"2023-12-01T03:12:34.860228+0000","last_unstale":"2023-12-01T03:12:35.362890+0000","last_undegraded":"2023-12-01T03:12:35.362890+0000","last_fullsized":"2023-12-01T03:12:35.362890+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:54:16.920168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350003+0000","last_change":"2023-12-01T03:12:35.345216+0000","last_active":"2023-12-01T03:12:35.350003+0000","last_peered":"2023-12-01T03:12:35.350003+0000","last_clean":"2023-12-01T03:12:35.350003+0000","last_became_active":"2023-12-01T03:12:35.344046+0000","last_became_peered":"2023-12-01T03:12:35.344046+0000","last_unstale":"2023-12-01T03:12:35.350003+0000","last_undegraded":"2023-12-01T03:12:35.350003+0000","last_fullsized":"2023-12-01T03:12:35.350003+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.334144+0000","last_change":"2023-12-01T03:12:34.864610+0000","last_active":"2023-12-01T03:12:35.334144+0000","last_peered":"2023-12-01T03:12:35.334144+0000","last_clean":"2023-12-01T03:12:35.334144+0000","last_became_active":"2023-12-01T03:12:34.864320+0000","last_became_peered":"2023-12-01T03:12:34.864320+0000","last_unstale":"2023-12-01T03:12:35.334144+0000","last_undegraded":"2023-12-01T03:12:35.334144+0000","last_fullsized":"2023-12-01T03:12:35.334144+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:41.361208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358575+0000","last_change":"2023-12-01T03:12:35.352877+0000","last_active":"2023-12-01T03:12:35.358575+0000","last_peered":"2023-12-01T03:12:35.358575+0000","last_clean":"2023-12-01T03:12:35.358575+0000","last_became_active":"2023-12-01T03:12:35.352593+0000","last_became_peered":"2023-12-01T03:12:35.352593+0000","last_unstale":"2023-12-01T03:12:35.358575+0000","last_undegraded":"2023-12-01T03:12:35.358575+0000","last_fullsized":"2023-12-01T03:12:35.358575+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.334961+0000","last_change":"2023-12-01T03:12:34.864988+0000","last_active":"2023-12-01T03:12:35.334961+0000","last_peered":"2023-12-01T03:12:35.334961+0000","last_clean":"2023-12-01T03:12:35.334961+0000","last_became_active":"2023-12-01T03:12:34.864626+0000","last_became_peered":"2023-12-01T03:12:34.864626+0000","last_unstale":"2023-12-01T03:12:35.334961+0000","last_undegraded":"2023-12-01T03:12:35.334961+0000","last_fullsized":"2023-12-01T03:12:35.334961+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:35.500459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.354839+0000","last_change":"2023-12-01T03:12:35.351460+0000","last_active":"2023-12-01T03:12:35.354839+0000","last_peered":"2023-12-01T03:12:35.354839+0000","last_clean":"2023-12-01T03:12:35.354839+0000","last_became_active":"2023-12-01T03:12:35.351166+0000","last_became_peered":"2023-12-01T03:12:35.351166+0000","last_unstale":"2023-12-01T03:12:35.354839+0000","last_undegraded":"2023-12-01T03:12:35.354839+0000","last_fullsized":"2023-12-01T03:12:35.354839+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.338056+0000","last_change":"2023-12-01T03:12:34.859322+0000","last_active":"2023-12-01T03:12:35.338056+0000","last_peered":"2023-12-01T03:12:35.338056+0000","last_clean":"2023-12-01T03:12:35.338056+0000","last_became_active":"2023-12-01T03:12:34.859042+0000","last_became_peered":"2023-12-01T03:12:34.859042+0000","last_unstale":"2023-12-01T03:12:35.338056+0000","last_undegraded":"2023-12-01T03:12:35.338056+0000","last_fullsized":"2023-12-01T03:12:35.338056+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:08.980764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355264+0000","last_change":"2023-12-01T03:12:35.348542+0000","last_active":"2023-12-01T03:12:35.355264+0000","last_peered":"2023-12-01T03:12:35.355264+0000","last_clean":"2023-12-01T03:12:35.355264+0000","last_became_active":"2023-12-01T03:12:35.348272+0000","last_became_peered":"2023-12-01T03:12:35.348272+0000","last_unstale":"2023-12-01T03:12:35.355264+0000","last_undegraded":"2023-12-01T03:12:35.355264+0000","last_fullsized":"2023-12-01T03:12:35.355264+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:55:08.951057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.7","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.363839+0000","last_change":"2023-12-01T03:12:34.860198+0000","last_active":"2023-12-01T03:12:35.363839+0000","last_peered":"2023-12-01T03:12:35.363839+0000","last_clean":"2023-12-01T03:12:35.363839+0000","last_became_active":"2023-12-01T03:12:34.859894+0000","last_became_peered":"2023-12-01T03:12:34.859894+0000","last_unstale":"2023-12-01T03:12:35.363839+0000","last_undegraded":"2023-12-01T03:12:35.363839+0000","last_fullsized":"2023-12-01T03:12:35.363839+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:36.882914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.356357+0000","last_change":"2023-12-01T03:12:35.351385+0000","last_active":"2023-12-01T03:12:35.356357+0000","last_peered":"2023-12-01T03:12:35.356357+0000","last_clean":"2023-12-01T03:12:35.356357+0000","last_became_active":"2023-12-01T03:12:35.351119+0000","last_became_peered":"2023-12-01T03:12:35.351119+0000","last_unstale":"2023-12-01T03:12:35.356357+0000","last_undegraded":"2023-12-01T03:12:35.356357+0000","last_fullsized":"2023-12-01T03:12:35.356357+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:20.331105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.6","version":"256'1","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-12-01T03:12:36.337500+0000","last_change":"2023-12-01T03:12:34.868249+0000","last_active":"2023-12-01T03:12:36.337500+0000","last_peered":"2023-12-01T03:12:36.337500+0000","last_clean":"2023-12-01T03:12:36.337500+0000","last_became_active":"2023-12-01T03:12:34.867861+0000","last_became_peered":"2023-12-01T03:12:34.867861+0000","last_unstale":"2023-12-01T03:12:36.337500+0000","last_undegraded":"2023-12-01T03:12:36.337500+0000","last_fullsized":"2023-12-01T03:12:36.337500+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:00.274064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.344055+0000","last_change":"2023-12-01T03:12:35.342473+0000","last_active":"2023-12-01T03:12:35.344055+0000","last_peered":"2023-12-01T03:12:35.344055+0000","last_clean":"2023-12-01T03:12:35.344055+0000","last_became_active":"2023-12-01T03:12:35.342193+0000","last_became_peered":"2023-12-01T03:12:35.342193+0000","last_unstale":"2023-12-01T03:12:35.344055+0000","last_undegraded":"2023-12-01T03:12:35.344055+0000","last_fullsized":"2023-12-01T03:12:35.344055+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.340708+0000","last_change":"2023-12-01T03:12:34.857807+0000","last_active":"2023-12-01T03:12:35.340708+0000","last_peered":"2023-12-01T03:12:35.340708+0000","last_clean":"2023-12-01T03:12:35.340708+0000","last_became_active":"2023-12-01T03:12:34.857501+0000","last_became_peered":"2023-12-01T03:12:34.857501+0000","last_unstale":"2023-12-01T03:12:35.340708+0000","last_undegraded":"2023-12-01T03:12:35.340708+0000","last_fullsized":"2023-12-01T03:12:35.340708+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:36.882914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358403+0000","last_change":"2023-12-01T03:12:35.351958+0000","last_active":"2023-12-01T03:12:35.358403+0000","last_peered":"2023-12-01T03:12:35.358403+0000","last_clean":"2023-12-01T03:12:35.358403+0000","last_became_active":"2023-12-01T03:12:35.351724+0000","last_became_peered":"2023-12-01T03:12:35.351724+0000","last_unstale":"2023-12-01T03:12:35.358403+0000","last_undegraded":"2023-12-01T03:12:35.358403+0000","last_fullsized":"2023-12-01T03:12:35.358403+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:45.819776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.334149+0000","last_change":"2023-12-01T03:12:34.865848+0000","last_active":"2023-12-01T03:12:35.334149+0000","last_peered":"2023-12-01T03:12:35.334149+0000","last_clean":"2023-12-01T03:12:35.334149+0000","last_became_active":"2023-12-01T03:12:34.865157+0000","last_became_peered":"2023-12-01T03:12:34.865157+0000","last_unstale":"2023-12-01T03:12:35.334149+0000","last_undegraded":"2023-12-01T03:12:35.334149+0000","last_fullsized":"2023-12-01T03:12:35.334149+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:10.788196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.349087+0000","last_change":"2023-12-01T03:12:35.347141+0000","last_active":"2023-12-01T03:12:35.349087+0000","last_peered":"2023-12-01T03:12:35.349087+0000","last_clean":"2023-12-01T03:12:35.349087+0000","last_became_active":"2023-12-01T03:12:35.345505+0000","last_became_peered":"2023-12-01T03:12:35.345505+0000","last_unstale":"2023-12-01T03:12:35.349087+0000","last_undegraded":"2023-12-01T03:12:35.349087+0000","last_fullsized":"2023-12-01T03:12:35.349087+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:01.535641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.327346+0000","last_change":"2023-12-01T03:12:34.870482+0000","last_active":"2023-12-01T03:12:35.327346+0000","last_peered":"2023-12-01T03:12:35.327346+0000","last_clean":"2023-12-01T03:12:35.327346+0000","last_became_active":"2023-12-01T03:12:34.870111+0000","last_became_peered":"2023-12-01T03:12:34.870111+0000","last_unstale":"2023-12-01T03:12:35.327346+0000","last_undegraded":"2023-12-01T03:12:35.327346+0000","last_fullsized":"2023-12-01T03:12:35.327346+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:24.009174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355639+0000","last_change":"2023-12-01T03:12:35.349891+0000","last_active":"2023-12-01T03:12:35.355639+0000","last_peered":"2023-12-01T03:12:35.355639+0000","last_clean":"2023-12-01T03:12:35.355639+0000","last_became_active":"2023-12-01T03:12:35.345336+0000","last_became_peered":"2023-12-01T03:12:35.345336+0000","last_unstale":"2023-12-01T03:12:35.355639+0000","last_undegraded":"2023-12-01T03:12:35.355639+0000","last_fullsized":"2023-12-01T03:12:35.355639+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.335015+0000","last_change":"2023-12-01T03:12:34.863164+0000","last_active":"2023-12-01T03:12:35.335015+0000","last_peered":"2023-12-01T03:12:35.335015+0000","last_clean":"2023-12-01T03:12:35.335015+0000","last_became_active":"2023-12-01T03:12:34.862454+0000","last_became_peered":"2023-12-01T03:12:34.862454+0000","last_unstale":"2023-12-01T03:12:35.335015+0000","last_undegraded":"2023-12-01T03:12:35.335015+0000","last_fullsized":"2023-12-01T03:12:35.335015+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:12.944465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350189+0000","last_change":"2023-12-01T03:12:35.343968+0000","last_active":"2023-12-01T03:12:35.350189+0000","last_peered":"2023-12-01T03:12:35.350189+0000","last_clean":"2023-12-01T03:12:35.350189+0000","last_became_active":"2023-12-01T03:12:35.343680+0000","last_became_peered":"2023-12-01T03:12:35.343680+0000","last_unstale":"2023-12-01T03:12:35.350189+0000","last_undegraded":"2023-12-01T03:12:35.350189+0000","last_fullsized":"2023-12-01T03:12:35.350189+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.338645+0000","last_change":"2023-12-01T03:12:34.860753+0000","last_active":"2023-12-01T03:12:35.338645+0000","last_peered":"2023-12-01T03:12:35.338645+0000","last_clean":"2023-12-01T03:12:35.338645+0000","last_became_active":"2023-12-01T03:12:34.858035+0000","last_became_peered":"2023-12-01T03:12:34.858035+0000","last_unstale":"2023-12-01T03:12:35.338645+0000","last_undegraded":"2023-12-01T03:12:35.338645+0000","last_fullsized":"2023-12-01T03:12:35.338645+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:40.965606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.356341+0000","last_change":"2023-12-01T03:12:35.351762+0000","last_active":"2023-12-01T03:12:35.356341+0000","last_peered":"2023-12-01T03:12:35.356341+0000","last_clean":"2023-12-01T03:12:35.356341+0000","last_became_active":"2023-12-01T03:12:35.351508+0000","last_became_peered":"2023-12-01T03:12:35.351508+0000","last_unstale":"2023-12-01T03:12:35.356341+0000","last_undegraded":"2023-12-01T03:12:35.356341+0000","last_fullsized":"2023-12-01T03:12:35.356341+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:08:34.191802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.335259+0000","last_change":"2023-12-01T03:12:34.866607+0000","last_active":"2023-12-01T03:12:35.335259+0000","last_peered":"2023-12-01T03:12:35.335259+0000","last_clean":"2023-12-01T03:12:35.335259+0000","last_became_active":"2023-12-01T03:12:34.865003+0000","last_became_peered":"2023-12-01T03:12:34.865003+0000","last_unstale":"2023-12-01T03:12:35.335259+0000","last_undegraded":"2023-12-01T03:12:35.335259+0000","last_fullsized":"2023-12-01T03:12:35.335259+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:29:11.426576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357675+0000","last_change":"2023-12-01T03:12:35.351337+0000","last_active":"2023-12-01T03:12:35.357675+0000","last_peered":"2023-12-01T03:12:35.357675+0000","last_clean":"2023-12-01T03:12:35.357675+0000","last_became_active":"2023-12-01T03:12:35.351044+0000","last_became_peered":"2023-12-01T03:12:35.351044+0000","last_unstale":"2023-12-01T03:12:35.357675+0000","last_undegraded":"2023-12-01T03:12:35.357675+0000","last_fullsized":"2023-12-01T03:12:35.357675+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"241'359","reported_seq":1272,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.334911+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:12:35.334911+0000","last_peered":"2023-12-01T03:12:35.334911+0000","last_clean":"2023-12-01T03:12:35.334911+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:12:35.334911+0000","last_undegraded":"2023-12-01T03:12:35.334911+0000","last_fullsized":"2023-12-01T03:12:35.334911+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60452,"kb_used_data":3164,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311388,"statfs":{"total":96636764160,"available":96574861312,"internally_reserved":0,"allocated":3239936,"data_stored":1591139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0620000000000001,"15min":1.0760000000000001},"min":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.095},{"interface":"front","average":{"1min":1.343,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":1.1779999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":0.83999999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":1.397},{"interface":"front","average":{"1min":1.363,"5min":1.3069999999999999,"15min":1.327},"min":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.3720000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.167},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.95,"5min":1.95,"15min":2.1179999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.349,"5min":1.254,"15min":1.264},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.246}]},{"osd":3,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2729999999999999,"15min":1.302},"min":{"1min":0.78400000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.3959999999999999,"15min":1.425},"min":{"1min":0.84599999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":1.4870000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3140000000000001,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7989999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5029999999999999,"15min":1.508},"min":{"1min":1.1599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":1.571}]},{"osd":5,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.337,"15min":1.327},"min":{"1min":0.79600000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.5840000000000001,"15min":1.597},"min":{"1min":1.2430000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.8380000000000001,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":1.6699999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.476,"15min":1.454},"min":{"1min":0.89400000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.8579999999999997,"5min":4.8579999999999997,"15min":4.8579999999999997},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4319999999999999,"15min":1.4550000000000001},"min":{"1min":0.81799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":2.762},"last":1.423}]}]},{"osd":6,"up_from":13,"seq":55834576002,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60444,"kb_used_data":3156,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311396,"statfs":{"total":96636764160,"available":96574869504,"internally_reserved":0,"allocated":3231744,"data_stored":1590543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.036,"15min":1.0980000000000001},"min":{"1min":0.53600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4810000000000001,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.093,"15min":1.1519999999999999},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6439999999999999,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.181}]},{"osd":1,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.127,"15min":1.202},"min":{"1min":0.57399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7010000000000001,"5min":1.976,"15min":2.0449999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.542,"5min":1.694,"15min":1.835},"last":0.78700000000000003}]},{"osd":2,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5369999999999999,"5min":1.742,"15min":1.976},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.272},"min":{"1min":0.71599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.0209999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.262,"15min":1.306},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7929999999999999,"5min":1.927,"15min":1.927},"last":1.28},{"interface":"front","average":{"1min":1.304,"5min":1.377,"15min":1.415},"min":{"1min":0.68700000000000006,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.3049999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2809999999999999,"15min":1.27},"min":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.661,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.232},{"interface":"front","average":{"1min":1.355,"5min":1.4199999999999999,"15min":1.3660000000000001},"min":{"1min":0.64100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1019999999999999,"5min":2.48,"15min":2.48},"last":1.002}]},{"osd":5,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3859999999999999,"15min":1.407},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.651,"5min":2.262,"15min":2.262},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.518,"15min":1.492},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0030000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.129}]},{"osd":7,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5069999999999999,"15min":1.526},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.865,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0539999999999998,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.421}]}]},{"osd":5,"up_from":13,"seq":55834576001,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52828,"kb_used_data":3732,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319012,"statfs":{"total":96636764160,"available":96582668288,"internally_reserved":0,"allocated":3821568,"data_stored":2180821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0840000000000001,"15min":1.119},"min":{"1min":0.68200000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.476,"5min":1.776,"15min":1.776},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.996,"5min":1.133,"15min":1.151},"min":{"1min":0.72699999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3879999999999999,"5min":1.746,"15min":1.752},"last":1.323}]},{"osd":1,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.147,"15min":1.155},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.52,"5min":1.6519999999999999,"15min":1.696},"last":1.272},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1850000000000001,"15min":1.226},"min":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.589,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":1.0409999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.21,"15min":1.246},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.5820000000000001,"5min":1.901,"15min":1.901},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.234,"15min":1.2490000000000001},"min":{"1min":0.90600000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5609999999999999,"5min":1.899,"15min":1.9219999999999999},"last":0.998}]},{"osd":3,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3029999999999999,"15min":1.3140000000000001},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.508,"5min":1.889,"15min":1.8979999999999999},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3169999999999999,"15min":1.3540000000000001},"min":{"1min":1.0649999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.54,"5min":1.97,"15min":2.1859999999999999},"last":1.1180000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2669999999999999,"15min":1.256},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.764,"5min":1.869,"15min":2.6040000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3660000000000001,"15min":1.371},"min":{"1min":0.96599999999999997,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.5449999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.484}]},{"osd":6,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3839999999999999,"15min":1.375},"min":{"1min":0.97699999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.643,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4239999999999999,"15min":1.419},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.9850000000000001,"5min":2.032,"15min":2.577},"last":1.6000000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.5349999999999999,"15min":1.536},"min":{"1min":1.294,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.8779999999999999,"5min":2.077,"15min":3.1709999999999998},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.5780000000000001,"15min":1.5760000000000001},"min":{"1min":1.353,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.8819999999999999,"5min":2.101,"15min":2.8740000000000001},"last":1.552}]}]},{"osd":4,"up_from":13,"seq":55834576001,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52264,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319576,"statfs":{"total":96636764160,"available":96583245824,"internally_reserved":0,"allocated":3244032,"data_stored":1592599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.65200000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.68600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.641,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.0129999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.216,"15min":1.2470000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5920000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.119},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.151,"15min":1.1699999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6930000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.2949999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.3100000000000001,"15min":1.238},"min":{"1min":0.81399999999999995,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.4630000000000001,"5min":11.311,"15min":11.311},"last":1.054},{"interface":"front","average":{"1min":1.159,"5min":1.2629999999999999,"15min":1.3200000000000001},"min":{"1min":0.86199999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.675,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.204}]},{"osd":3,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2789999999999999,"15min":1.2829999999999999},"min":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5129999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.155},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.363,"15min":1.3320000000000001},"min":{"1min":0.90000000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2559999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.244}]},{"osd":5,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.375,"15min":1.329},"min":{"1min":1.028,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.897,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.448},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3580000000000001,"15min":1.3149999999999999},"min":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.74,"5min":3.71,"15min":3.71},"last":1.411}]},{"osd":6,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.3899999999999999,"15min":1.3740000000000001},"min":{"1min":1.0289999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.161,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.48,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":1.25,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.028,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.343}]},{"osd":7,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5089999999999999,"15min":1.4850000000000001},"min":{"1min":0.82399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.794,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.468},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5920000000000001,"15min":1.577},"min":{"1min":1.139,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3250000000000002,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.3819999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576000,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56348,"kb_used_data":3156,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315492,"statfs":{"total":96636764160,"available":96579063808,"internally_reserved":0,"allocated":3231744,"data_stored":1590487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.306,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":1.377},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0449999999999999,"15min":1.0609999999999999},"min":{"1min":0.39500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.573,"5min":1.95,"15min":1.95},"last":0.88100000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.083,"15min":1.1479999999999999},"min":{"1min":0.56200000000000006,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.417,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.94399999999999995,"15min":0.98499999999999999},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6970000000000001,"5min":3.863,"15min":3.863},"last":0.94699999999999995}]},{"osd":2,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.038,"15min":1.085},"min":{"1min":0.49099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.224,"5min":3.597,"15min":3.597},"last":1.044},{"interface":"front","average":{"1min":1.052,"5min":1.175,"15min":1.228},"min":{"1min":0.54700000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.647,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.278}]},{"osd":4,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.264,"15min":1.286},"min":{"1min":0.86099999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5269999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.79500000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9970000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.3160000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3109999999999999,"15min":1.325},"min":{"1min":0.92200000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":1.486},{"interface":"front","average":{"1min":1.208,"5min":1.3420000000000001,"15min":1.3600000000000001},"min":{"1min":0.82499999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6699999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.4490000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.375,"15min":1.4370000000000001},"min":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":1.595},{"interface":"front","average":{"1min":1.29,"5min":1.456,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.6879999999999999,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":1.528}]},{"osd":7,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4750000000000001,"15min":1.514},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.788,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4770000000000001,"15min":1.522},"min":{"1min":1.0549999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.913,"5min":3.004,"15min":3.004},"last":1.6930000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576000,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48168,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323672,"statfs":{"total":96636764160,"available":96587440128,"internally_reserved":0,"allocated":3244032,"data_stored":1592039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.81100000000000005,"15min":0.87},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92100000000000004,"5min":1.411,"15min":1.4730000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98899999999999999,"15min":1.024},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.216,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.107}]},{"osd":1,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.97299999999999998,"15min":1.0309999999999999},"min":{"1min":0.57899999999999996,"5min":0.378,"15min":0.378},"max":{"1min":1.512,"5min":1.512,"15min":1.698},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.04,"15min":1.113},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5229999999999999,"5min":1.635,"15min":1.6599999999999999},"last":1.22}]},{"osd":3,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.97499999999999998,"15min":1.022},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2889999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.097,"15min":1.181},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.4970000000000001,"5min":1.615,"15min":1.79},"last":1.0589999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2749999999999999,"15min":1.3120000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.621,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.609},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.276},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4730000000000001,"5min":1.714,"15min":1.835},"last":1.2589999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.3109999999999999,"15min":1.3680000000000001},"min":{"1min":0.86599999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.639,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.347,"15min":1.393},"min":{"1min":0.79000000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.573,"5min":1.8740000000000001,"15min":2.016},"last":1.4870000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.351,"15min":1.367},"min":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.865,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.86599999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7090000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.2969999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4370000000000001,"15min":1.4630000000000001},"min":{"1min":0.94299999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5740000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.4350000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.8100000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.429}]}]},{"osd":1,"up_from":13,"seq":55834575999,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52836,"kb_used_data":3740,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319004,"statfs":{"total":96636764160,"available":96582660096,"internally_reserved":0,"allocated":3829760,"data_stored":2180913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82899999999999996,"15min":0.88800000000000001},"min":{"1min":0.38500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.5569999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.94399999999999995,"15min":1.0249999999999999},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.167,"5min":1.512,"15min":1.681},"last":1.353}]},{"osd":2,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.86299999999999999,"15min":0.92300000000000004},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97699999999999998,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0349999999999999,"15min":1.1220000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.476,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":1.2010000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.1359999999999999,"15min":1.268},"min":{"1min":0.52400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.788},"last":1.2869999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.046,"15min":1.1120000000000001},"min":{"1min":0.41399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.615,"5min":1.714,"15min":1.714},"last":1.083}]},{"osd":4,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.177,"15min":1.2090000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6000000000000001,"5min":1.623,"15min":1.6859999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.329,"15min":1.391},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7450000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.9219999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2969999999999999,"15min":1.3240000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8169999999999999,"5min":1.9399999999999999,"15min":2.0379999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.2569999999999999,"15min":1.319},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.7470000000000001,"5min":1.927,"15min":1.927},"last":1.1619999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.373,"15min":1.429},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.885,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3759999999999999,"15min":1.4019999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.901,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.25}]},{"osd":7,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.431,"15min":1.5069999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":2.0009999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.46,"15min":1.5009999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.954,"5min":2.1389999999999998,"15min":2.2069999999999999},"last":1.407}]}]},{"osd":0,"up_from":13,"seq":55834576001,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60444,"kb_used_data":3156,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311396,"statfs":{"total":96636764160,"available":96574869504,"internally_reserved":0,"allocated":3231744,"data_stored":1590543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.40100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94499999999999995,"15min":1.04},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.131}]},{"osd":2,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.0249999999999999,"15min":1.0660000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.48699999999999999},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.1429999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.002,"5min":1.0269999999999999,"15min":1.107},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6619999999999999,"5min":1.726,"15min":1.792},"last":1.4770000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.99099999999999999,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.306,"5min":1.536,"15min":1.8839999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.165,"15min":1.222},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.629}]},{"osd":4,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.27,"15min":1.337},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":1.258},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.246,"15min":1.2949999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.536,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.208}]},{"osd":5,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.839,"5min":1.899,"15min":2.7719999999999998},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.365,"15min":1.423},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.484,"5min":2.484,"15min":2.5880000000000001},"last":1.393}]},{"osd":6,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.327,"15min":1.371},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.835,"5min":1.835,"15min":2.2029999999999998},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.444,"15min":1.4810000000000001},"min":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":1.9239999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.236}]},{"osd":7,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.4419999999999999,"15min":1.506},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.411,"5min":2.411,"15min":2.7040000000000002},"last":1.704},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.456,"15min":1.4770000000000001},"min":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.718,"5min":2.1190000000000002,"15min":2.431},"last":1.4950000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:12:44.870 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:12:45.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.094+0000 7f1dfba43640 1 Processor -- start 2023-12-01T03:12:45.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.095+0000 7f1dfba43640 1 -- start start 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.096+0000 7f1dfba43640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1df40b5d80 0x7f1df40d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.096+0000 7f1dfba43640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1df40d1230 0x7f1df40d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.096+0000 7f1dfba43640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df40d1b90 con 0x7f1df415cd20 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.096+0000 7f1dfba43640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df40b2160 con 0x7f1df40b5d80 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.096+0000 7f1dfba43640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df40b8c80 con 0x7f1df40d1230 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.096+0000 7f1df8fde640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1df40b5d80 0x7f1df40d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.096+0000 7f1df8fde640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1df40b5d80 0x7f1df40d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51872/0 (socket says 172.21.15.115:51872) 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.096+0000 7f1df8fde640 1 -- 172.21.15.115:0/1554067223 learned_addr learned my addr 172.21.15.115:0/1554067223 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.097+0000 7f1df9fe0640 1 --2- 172.21.15.115:0/1554067223 >> v2:172.21.15.142:3300/0 conn(0x7f1df40d1230 0x7f1df40d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.097+0000 7f1df8fde640 1 -- 172.21.15.115:0/1554067223 >> v2:172.21.15.142:3300/0 conn(0x7f1df40d1230 msgr2=0x7f1df40d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.097+0000 7f1df8fde640 1 --2- 172.21.15.115:0/1554067223 >> v2:172.21.15.142:3300/0 conn(0x7f1df40d1230 0x7f1df40d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.097+0000 7f1df8fde640 1 -- 172.21.15.115:0/1554067223 >> v1:172.21.15.142:6789/0 conn(0x7f1df415cd20 legacy=0x7f1df40b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.097+0000 7f1df8fde640 1 -- 172.21.15.115:0/1554067223 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40ce2e0 con 0x7f1df40b5d80 2023-12-01T03:12:45.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.097+0000 7f1df9fe0640 1 --2- 172.21.15.115:0/1554067223 >> v2:172.21.15.142:3300/0 conn(0x7f1df40d1230 0x7f1df40d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:45.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.098+0000 7f1df8fde640 1 --2- 172.21.15.115:0/1554067223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40b5d80 0x7f1df40d24d0 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7f1df40c6720 tx=0x7f1df002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4759297d400244ea server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:45.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.098+0000 7f1de7fff640 1 -- 172.21.15.115:0/1554067223 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df002ca80 con 0x7f1df40b5d80 2023-12-01T03:12:45.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.098+0000 7f1de7fff640 1 -- 172.21.15.115:0/1554067223 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df002cbe0 con 0x7f1df40b5d80 2023-12-01T03:12:45.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.098+0000 7f1de7fff640 1 -- 172.21.15.115:0/1554067223 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df00343f0 con 0x7f1df40b5d80 2023-12-01T03:12:45.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.099+0000 7f1dfba43640 1 -- 172.21.15.115:0/1554067223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40b5d80 msgr2=0x7f1df40d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.099+0000 7f1dfba43640 1 --2- 172.21.15.115:0/1554067223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40b5d80 0x7f1df40d24d0 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7f1df40c6720 tx=0x7f1df002bee0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.099+0000 7f1dfba43640 1 -- 172.21.15.115:0/1554067223 shutdown_connections 2023-12-01T03:12:45.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.099+0000 7f1dfba43640 1 --2- 172.21.15.115:0/1554067223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40b5d80 0x7f1df40d24d0 unknown :-1 s=CLOSED pgs=2982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.099+0000 7f1dfba43640 1 --2- 172.21.15.115:0/1554067223 >> v2:172.21.15.142:3300/0 conn(0x7f1df40d1230 0x7f1df40d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.099+0000 7f1dfba43640 1 -- 172.21.15.115:0/1554067223 >> 172.21.15.115:0/1554067223 conn(0x7f1df406a0e0 msgr2=0x7f1df406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:45.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.099+0000 7f1dfba43640 1 -- 172.21.15.115:0/1554067223 shutdown_connections 2023-12-01T03:12:45.102 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.099+0000 7f1dfba43640 1 -- 172.21.15.115:0/1554067223 wait complete. 2023-12-01T03:12:45.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.101+0000 7f1dfba43640 1 Processor -- start 2023-12-01T03:12:45.103 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.101+0000 7f1dfba43640 1 -- start start 2023-12-01T03:12:45.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.102+0000 7f1dfba43640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df40b5d80 0x7f1df41f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.102+0000 7f1dfba43640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40d1230 0x7f1df40b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.102+0000 7f1dfba43640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1df415cd20 0x7f1df40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.102+0000 7f1dfba43640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41569d0 con 0x7f1df40d1230 2023-12-01T03:12:45.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.102+0000 7f1dfba43640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40cae10 con 0x7f1df40b5d80 2023-12-01T03:12:45.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.102+0000 7f1dfba43640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41f6bc0 con 0x7f1df415cd20 2023-12-01T03:12:45.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.102+0000 7f1df8fde640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40d1230 0x7f1df40b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:45.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.102+0000 7f1df8fde640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40d1230 0x7f1df40b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51884/0 (socket says 172.21.15.115:51884) 2023-12-01T03:12:45.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.102+0000 7f1df8fde640 1 -- 172.21.15.115:0/622924698 learned_addr learned my addr 172.21.15.115:0/622924698 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:45.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.103+0000 7f1df8fde640 1 -- 172.21.15.115:0/622924698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1df415cd20 msgr2=0x7f1df40b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.103+0000 7f1df8fde640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1df415cd20 0x7f1df40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.103+0000 7f1df8fde640 1 -- 172.21.15.115:0/622924698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df40b5d80 msgr2=0x7f1df41f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.103+0000 7f1df8fde640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df40b5d80 0x7f1df41f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.103+0000 7f1df8fde640 1 -- 172.21.15.115:0/622924698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df0009b30 con 0x7f1df40d1230 2023-12-01T03:12:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.103+0000 7f1df8fde640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40d1230 0x7f1df40b25b0 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7f1df0036040 tx=0x7f1df0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.103+0000 7f1de5ffb640 1 -- 172.21.15.115:0/622924698 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df003e450 con 0x7f1df40d1230 2023-12-01T03:12:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.103+0000 7f1de5ffb640 1 -- 172.21.15.115:0/622924698 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df003e5f0 con 0x7f1df40d1230 2023-12-01T03:12:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.103+0000 7f1de5ffb640 1 -- 172.21.15.115:0/622924698 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df003e9a0 con 0x7f1df40d1230 2023-12-01T03:12:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.103+0000 7f1dfba43640 1 -- 172.21.15.115:0/622924698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df41f2d00 con 0x7f1df40d1230 2023-12-01T03:12:45.106 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.104+0000 7f1dfba43640 1 -- 172.21.15.115:0/622924698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df41f32c0 con 0x7f1df40d1230 2023-12-01T03:12:45.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.105+0000 7f1dfba43640 1 -- 172.21.15.115:0/622924698 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df40cafe0 con 0x7f1df40d1230 2023-12-01T03:12:45.107 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.106+0000 7f1de5ffb640 1 -- 172.21.15.115:0/622924698 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f1df002cd50 con 0x7f1df40d1230 2023-12-01T03:12:45.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.106+0000 7f1de5ffb640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1dd806a670 0x7f1dd806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.107+0000 7f1de5ffb640 1 -- 172.21.15.115:0/622924698 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f1df00ab9f0 con 0x7f1df40d1230 2023-12-01T03:12:45.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.107+0000 7f1df97df640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1dd806a670 0x7f1dd806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:45.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.108+0000 7f1df97df640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1dd806a670 0x7f1dd806cb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1de8000f60 tx=0x7f1de80079b0 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:45.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.111+0000 7f1de5ffb640 1 -- 172.21.15.115:0/622924698 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1df40cafe0 con 0x7f1df40d1230 2023-12-01T03:12:45.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.297+0000 7f1dfba43640 1 -- 172.21.15.115:0/622924698 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1df40ce000 con 0x7f1dd806a670 2023-12-01T03:12:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.307+0000 7f1de5ffb640 1 -- 172.21.15.115:0/622924698 <== mgr.11877 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287101 (secure 0 0 0) 0x7f1de8005080 con 0x7f1dd806a670 2023-12-01T03:12:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:45.315 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.313+0000 7f1dfba43640 1 -- 172.21.15.115:0/622924698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1dd806a670 msgr2=0x7f1dd806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.313+0000 7f1dfba43640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1dd806a670 0x7f1dd806cb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1de8000f60 tx=0x7f1de80079b0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.314+0000 7f1dfba43640 1 -- 172.21.15.115:0/622924698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40d1230 msgr2=0x7f1df40b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.314+0000 7f1dfba43640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40d1230 0x7f1df40b25b0 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7f1df0036040 tx=0x7f1df0004a40 comp rx=0 tx=0).stop 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.314+0000 7f1dfba43640 1 -- 172.21.15.115:0/622924698 shutdown_connections 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.314+0000 7f1dfba43640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1dd806a670 0x7f1dd806cb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.314+0000 7f1dfba43640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1df415cd20 0x7f1df40b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.314+0000 7f1dfba43640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1df40d1230 0x7f1df40b25b0 unknown :-1 s=CLOSED pgs=2983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.314+0000 7f1dfba43640 1 --2- 172.21.15.115:0/622924698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1df40b5d80 0x7f1df41f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.314+0000 7f1dfba43640 1 -- 172.21.15.115:0/622924698 >> 172.21.15.115:0/622924698 conn(0x7f1df406a0e0 msgr2=0x7f1df415ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.314+0000 7f1dfba43640 1 -- 172.21.15.115:0/622924698 shutdown_connections 2023-12-01T03:12:45.317 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.314+0000 7f1dfba43640 1 -- 172.21.15.115:0/622924698 wait complete. 2023-12-01T03:12:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-12-01T03:12:45.050961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":644,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":398,"ondisk_log_size":398,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":443784,"kb_used_data":26440,"kb_used_omap":0,"kb_used_meta":417280,"kb_avail":754530936,"statfs":{"total":773094113280,"available":772639678464,"internally_reserved":0,"allocated":27074560,"data_stored":13909084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":427294720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.726621"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.356633+0000","last_change":"2023-12-01T03:12:35.346245+0000","last_active":"2023-12-01T03:12:35.356633+0000","last_peered":"2023-12-01T03:12:35.356633+0000","last_clean":"2023-12-01T03:12:35.356633+0000","last_became_active":"2023-12-01T03:12:35.345959+0000","last_became_peered":"2023-12-01T03:12:35.345959+0000","last_unstale":"2023-12-01T03:12:35.356633+0000","last_undegraded":"2023-12-01T03:12:35.356633+0000","last_fullsized":"2023-12-01T03:12:35.356633+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:21:23.692942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.347898+0000","last_change":"2023-12-01T03:12:35.346575+0000","last_active":"2023-12-01T03:12:35.347898+0000","last_peered":"2023-12-01T03:12:35.347898+0000","last_clean":"2023-12-01T03:12:35.347898+0000","last_became_active":"2023-12-01T03:12:35.346315+0000","last_became_peered":"2023-12-01T03:12:35.346315+0000","last_unstale":"2023-12-01T03:12:35.347898+0000","last_undegraded":"2023-12-01T03:12:35.347898+0000","last_fullsized":"2023-12-01T03:12:35.347898+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357412+0000","last_change":"2023-12-01T03:12:35.349364+0000","last_active":"2023-12-01T03:12:35.357412+0000","last_peered":"2023-12-01T03:12:35.357412+0000","last_clean":"2023-12-01T03:12:35.357412+0000","last_became_active":"2023-12-01T03:12:35.347941+0000","last_became_peered":"2023-12-01T03:12:35.347941+0000","last_unstale":"2023-12-01T03:12:35.357412+0000","last_undegraded":"2023-12-01T03:12:35.357412+0000","last_fullsized":"2023-12-01T03:12:35.357412+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358161+0000","last_change":"2023-12-01T03:12:35.353122+0000","last_active":"2023-12-01T03:12:35.358161+0000","last_peered":"2023-12-01T03:12:35.358161+0000","last_clean":"2023-12-01T03:12:35.358161+0000","last_became_active":"2023-12-01T03:12:35.352880+0000","last_became_peered":"2023-12-01T03:12:35.352880+0000","last_unstale":"2023-12-01T03:12:35.358161+0000","last_undegraded":"2023-12-01T03:12:35.358161+0000","last_fullsized":"2023-12-01T03:12:35.358161+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:08:49.741119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358802+0000","last_change":"2023-12-01T03:12:35.356178+0000","last_active":"2023-12-01T03:12:35.358802+0000","last_peered":"2023-12-01T03:12:35.358802+0000","last_clean":"2023-12-01T03:12:35.358802+0000","last_became_active":"2023-12-01T03:12:35.354822+0000","last_became_peered":"2023-12-01T03:12:35.354822+0000","last_unstale":"2023-12-01T03:12:35.358802+0000","last_undegraded":"2023-12-01T03:12:35.358802+0000","last_fullsized":"2023-12-01T03:12:35.358802+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.354894+0000","last_change":"2023-12-01T03:12:35.350584+0000","last_active":"2023-12-01T03:12:35.354894+0000","last_peered":"2023-12-01T03:12:35.354894+0000","last_clean":"2023-12-01T03:12:35.354894+0000","last_became_active":"2023-12-01T03:12:35.346833+0000","last_became_peered":"2023-12-01T03:12:35.346833+0000","last_unstale":"2023-12-01T03:12:35.354894+0000","last_undegraded":"2023-12-01T03:12:35.354894+0000","last_fullsized":"2023-12-01T03:12:35.354894+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:04.481100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358611+0000","last_change":"2023-12-01T03:12:35.355731+0000","last_active":"2023-12-01T03:12:35.358611+0000","last_peered":"2023-12-01T03:12:35.358611+0000","last_clean":"2023-12-01T03:12:35.358611+0000","last_became_active":"2023-12-01T03:12:35.355381+0000","last_became_peered":"2023-12-01T03:12:35.355381+0000","last_unstale":"2023-12-01T03:12:35.358611+0000","last_undegraded":"2023-12-01T03:12:35.358611+0000","last_fullsized":"2023-12-01T03:12:35.358611+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360133+0000","last_change":"2023-12-01T03:12:35.355271+0000","last_active":"2023-12-01T03:12:35.360133+0000","last_peered":"2023-12-01T03:12:35.360133+0000","last_clean":"2023-12-01T03:12:35.360133+0000","last_became_active":"2023-12-01T03:12:35.355020+0000","last_became_peered":"2023-12-01T03:12:35.355020+0000","last_unstale":"2023-12-01T03:12:35.360133+0000","last_undegraded":"2023-12-01T03:12:35.360133+0000","last_fullsized":"2023-12-01T03:12:35.360133+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:27.055115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358363+0000","last_change":"2023-12-01T03:12:35.353249+0000","last_active":"2023-12-01T03:12:35.358363+0000","last_peered":"2023-12-01T03:12:35.358363+0000","last_clean":"2023-12-01T03:12:35.358363+0000","last_became_active":"2023-12-01T03:12:35.352969+0000","last_became_peered":"2023-12-01T03:12:35.352969+0000","last_unstale":"2023-12-01T03:12:35.358363+0000","last_undegraded":"2023-12-01T03:12:35.358363+0000","last_fullsized":"2023-12-01T03:12:35.358363+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:17:30.864087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357144+0000","last_change":"2023-12-01T03:12:35.355534+0000","last_active":"2023-12-01T03:12:35.357144+0000","last_peered":"2023-12-01T03:12:35.357144+0000","last_clean":"2023-12-01T03:12:35.357144+0000","last_became_active":"2023-12-01T03:12:35.354708+0000","last_became_peered":"2023-12-01T03:12:35.354708+0000","last_unstale":"2023-12-01T03:12:35.357144+0000","last_undegraded":"2023-12-01T03:12:35.357144+0000","last_fullsized":"2023-12-01T03:12:35.357144+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358112+0000","last_change":"2023-12-01T03:12:35.351076+0000","last_active":"2023-12-01T03:12:35.358112+0000","last_peered":"2023-12-01T03:12:35.358112+0000","last_clean":"2023-12-01T03:12:35.358112+0000","last_became_active":"2023-12-01T03:12:35.350834+0000","last_became_peered":"2023-12-01T03:12:35.350834+0000","last_unstale":"2023-12-01T03:12:35.358112+0000","last_undegraded":"2023-12-01T03:12:35.358112+0000","last_fullsized":"2023-12-01T03:12:35.358112+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358471+0000","last_change":"2023-12-01T03:12:35.350756+0000","last_active":"2023-12-01T03:12:35.358471+0000","last_peered":"2023-12-01T03:12:35.358471+0000","last_clean":"2023-12-01T03:12:35.358471+0000","last_became_active":"2023-12-01T03:12:35.350503+0000","last_became_peered":"2023-12-01T03:12:35.350503+0000","last_unstale":"2023-12-01T03:12:35.358471+0000","last_undegraded":"2023-12-01T03:12:35.358471+0000","last_fullsized":"2023-12-01T03:12:35.358471+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.361001+0000","last_change":"2023-12-01T03:12:35.358627+0000","last_active":"2023-12-01T03:12:35.361001+0000","last_peered":"2023-12-01T03:12:35.361001+0000","last_clean":"2023-12-01T03:12:35.361001+0000","last_became_active":"2023-12-01T03:12:35.357120+0000","last_became_peered":"2023-12-01T03:12:35.357120+0000","last_unstale":"2023-12-01T03:12:35.361001+0000","last_undegraded":"2023-12-01T03:12:35.361001+0000","last_fullsized":"2023-12-01T03:12:35.361001+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:54:17.936247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.361020+0000","last_change":"2023-12-01T03:12:35.356909+0000","last_active":"2023-12-01T03:12:35.361020+0000","last_peered":"2023-12-01T03:12:35.361020+0000","last_clean":"2023-12-01T03:12:35.361020+0000","last_became_active":"2023-12-01T03:12:35.356405+0000","last_became_peered":"2023-12-01T03:12:35.356405+0000","last_unstale":"2023-12-01T03:12:35.361020+0000","last_undegraded":"2023-12-01T03:12:35.361020+0000","last_fullsized":"2023-12-01T03:12:35.361020+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:59:25.105386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.353415+0000","last_change":"2023-12-01T03:12:35.348321+0000","last_active":"2023-12-01T03:12:35.353415+0000","last_peered":"2023-12-01T03:12:35.353415+0000","last_clean":"2023-12-01T03:12:35.353415+0000","last_became_active":"2023-12-01T03:12:35.344564+0000","last_became_peered":"2023-12-01T03:12:35.344564+0000","last_unstale":"2023-12-01T03:12:35.353415+0000","last_undegraded":"2023-12-01T03:12:35.353415+0000","last_fullsized":"2023-12-01T03:12:35.353415+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350752+0000","last_change":"2023-12-01T03:12:35.349640+0000","last_active":"2023-12-01T03:12:35.350752+0000","last_peered":"2023-12-01T03:12:35.350752+0000","last_clean":"2023-12-01T03:12:35.350752+0000","last_became_active":"2023-12-01T03:12:35.348717+0000","last_became_peered":"2023-12-01T03:12:35.348717+0000","last_unstale":"2023-12-01T03:12:35.350752+0000","last_undegraded":"2023-12-01T03:12:35.350752+0000","last_fullsized":"2023-12-01T03:12:35.350752+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350655+0000","last_change":"2023-12-01T03:12:35.343916+0000","last_active":"2023-12-01T03:12:35.350655+0000","last_peered":"2023-12-01T03:12:35.350655+0000","last_clean":"2023-12-01T03:12:35.350655+0000","last_became_active":"2023-12-01T03:12:35.341639+0000","last_became_peered":"2023-12-01T03:12:35.341639+0000","last_unstale":"2023-12-01T03:12:35.350655+0000","last_undegraded":"2023-12-01T03:12:35.350655+0000","last_fullsized":"2023-12-01T03:12:35.350655+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:21:21.877813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355343+0000","last_change":"2023-12-01T03:12:35.349529+0000","last_active":"2023-12-01T03:12:35.355343+0000","last_peered":"2023-12-01T03:12:35.355343+0000","last_clean":"2023-12-01T03:12:35.355343+0000","last_became_active":"2023-12-01T03:12:35.349233+0000","last_became_peered":"2023-12-01T03:12:35.349233+0000","last_unstale":"2023-12-01T03:12:35.355343+0000","last_undegraded":"2023-12-01T03:12:35.355343+0000","last_fullsized":"2023-12-01T03:12:35.355343+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357431+0000","last_change":"2023-12-01T03:12:35.348866+0000","last_active":"2023-12-01T03:12:35.357431+0000","last_peered":"2023-12-01T03:12:35.357431+0000","last_clean":"2023-12-01T03:12:35.357431+0000","last_became_active":"2023-12-01T03:12:35.346590+0000","last_became_peered":"2023-12-01T03:12:35.346590+0000","last_unstale":"2023-12-01T03:12:35.357431+0000","last_undegraded":"2023-12-01T03:12:35.357431+0000","last_fullsized":"2023-12-01T03:12:35.357431+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.345236+0000","last_change":"2023-12-01T03:12:35.343338+0000","last_active":"2023-12-01T03:12:35.345236+0000","last_peered":"2023-12-01T03:12:35.345236+0000","last_clean":"2023-12-01T03:12:35.345236+0000","last_became_active":"2023-12-01T03:12:35.343036+0000","last_became_peered":"2023-12-01T03:12:35.343036+0000","last_unstale":"2023-12-01T03:12:35.345236+0000","last_undegraded":"2023-12-01T03:12:35.345236+0000","last_fullsized":"2023-12-01T03:12:35.345236+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357492+0000","last_change":"2023-12-01T03:12:35.352676+0000","last_active":"2023-12-01T03:12:35.357492+0000","last_peered":"2023-12-01T03:12:35.357492+0000","last_clean":"2023-12-01T03:12:35.357492+0000","last_became_active":"2023-12-01T03:12:35.352425+0000","last_became_peered":"2023-12-01T03:12:35.352425+0000","last_unstale":"2023-12-01T03:12:35.357492+0000","last_undegraded":"2023-12-01T03:12:35.357492+0000","last_fullsized":"2023-12-01T03:12:35.357492+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:16:31.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355176+0000","last_change":"2023-12-01T03:12:35.347876+0000","last_active":"2023-12-01T03:12:35.355176+0000","last_peered":"2023-12-01T03:12:35.355176+0000","last_clean":"2023-12-01T03:12:35.355176+0000","last_became_active":"2023-12-01T03:12:35.347100+0000","last_became_peered":"2023-12-01T03:12:35.347100+0000","last_unstale":"2023-12-01T03:12:35.355176+0000","last_undegraded":"2023-12-01T03:12:35.355176+0000","last_fullsized":"2023-12-01T03:12:35.355176+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:59:04.995090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357520+0000","last_change":"2023-12-01T03:12:35.349609+0000","last_active":"2023-12-01T03:12:35.357520+0000","last_peered":"2023-12-01T03:12:35.357520+0000","last_clean":"2023-12-01T03:12:35.357520+0000","last_became_active":"2023-12-01T03:12:35.349088+0000","last_became_peered":"2023-12-01T03:12:35.349088+0000","last_unstale":"2023-12-01T03:12:35.357520+0000","last_undegraded":"2023-12-01T03:12:35.357520+0000","last_fullsized":"2023-12-01T03:12:35.357520+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.349549+0000","last_change":"2023-12-01T03:12:35.343936+0000","last_active":"2023-12-01T03:12:35.349549+0000","last_peered":"2023-12-01T03:12:35.349549+0000","last_clean":"2023-12-01T03:12:35.349549+0000","last_became_active":"2023-12-01T03:12:35.343638+0000","last_became_peered":"2023-12-01T03:12:35.343638+0000","last_unstale":"2023-12-01T03:12:35.349549+0000","last_undegraded":"2023-12-01T03:12:35.349549+0000","last_fullsized":"2023-12-01T03:12:35.349549+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:00:57.853346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355218+0000","last_change":"2023-12-01T03:12:35.347714+0000","last_active":"2023-12-01T03:12:35.355218+0000","last_peered":"2023-12-01T03:12:35.355218+0000","last_clean":"2023-12-01T03:12:35.355218+0000","last_became_active":"2023-12-01T03:12:35.346230+0000","last_became_peered":"2023-12-01T03:12:35.346230+0000","last_unstale":"2023-12-01T03:12:35.355218+0000","last_undegraded":"2023-12-01T03:12:35.355218+0000","last_fullsized":"2023-12-01T03:12:35.355218+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:34.404254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350731+0000","last_change":"2023-12-01T03:12:35.348505+0000","last_active":"2023-12-01T03:12:35.350731+0000","last_peered":"2023-12-01T03:12:35.350731+0000","last_clean":"2023-12-01T03:12:35.350731+0000","last_became_active":"2023-12-01T03:12:35.345808+0000","last_became_peered":"2023-12-01T03:12:35.345808+0000","last_unstale":"2023-12-01T03:12:35.350731+0000","last_undegraded":"2023-12-01T03:12:35.350731+0000","last_fullsized":"2023-12-01T03:12:35.350731+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:17:00.577497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.351196+0000","last_change":"2023-12-01T03:12:35.348639+0000","last_active":"2023-12-01T03:12:35.351196+0000","last_peered":"2023-12-01T03:12:35.351196+0000","last_clean":"2023-12-01T03:12:35.351196+0000","last_became_active":"2023-12-01T03:12:35.348394+0000","last_became_peered":"2023-12-01T03:12:35.348394+0000","last_unstale":"2023-12-01T03:12:35.351196+0000","last_undegraded":"2023-12-01T03:12:35.351196+0000","last_fullsized":"2023-12-01T03:12:35.351196+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:09:42.224008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.361185+0000","last_change":"2023-12-01T03:12:35.356530+0000","last_active":"2023-12-01T03:12:35.361185+0000","last_peered":"2023-12-01T03:12:35.361185+0000","last_clean":"2023-12-01T03:12:35.361185+0000","last_became_active":"2023-12-01T03:12:35.356231+0000","last_became_peered":"2023-12-01T03:12:35.356231+0000","last_unstale":"2023-12-01T03:12:35.361185+0000","last_undegraded":"2023-12-01T03:12:35.361185+0000","last_fullsized":"2023-12-01T03:12:35.361185+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:53:08.535342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.352646+0000","last_change":"2023-12-01T03:12:35.348165+0000","last_active":"2023-12-01T03:12:35.352646+0000","last_peered":"2023-12-01T03:12:35.352646+0000","last_clean":"2023-12-01T03:12:35.352646+0000","last_became_active":"2023-12-01T03:12:35.347680+0000","last_became_peered":"2023-12-01T03:12:35.347680+0000","last_unstale":"2023-12-01T03:12:35.352646+0000","last_undegraded":"2023-12-01T03:12:35.352646+0000","last_fullsized":"2023-12-01T03:12:35.352646+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:50:37.148015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.361040+0000","last_change":"2023-12-01T03:12:35.357739+0000","last_active":"2023-12-01T03:12:35.361040+0000","last_peered":"2023-12-01T03:12:35.361040+0000","last_clean":"2023-12-01T03:12:35.361040+0000","last_became_active":"2023-12-01T03:12:35.355379+0000","last_became_peered":"2023-12-01T03:12:35.355379+0000","last_unstale":"2023-12-01T03:12:35.361040+0000","last_undegraded":"2023-12-01T03:12:35.361040+0000","last_fullsized":"2023-12-01T03:12:35.361040+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:25.025253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357725+0000","last_change":"2023-12-01T03:12:35.352593+0000","last_active":"2023-12-01T03:12:35.357725+0000","last_peered":"2023-12-01T03:12:35.357725+0000","last_clean":"2023-12-01T03:12:35.357725+0000","last_became_active":"2023-12-01T03:12:35.350140+0000","last_became_peered":"2023-12-01T03:12:35.350140+0000","last_unstale":"2023-12-01T03:12:35.357725+0000","last_undegraded":"2023-12-01T03:12:35.357725+0000","last_fullsized":"2023-12-01T03:12:35.357725+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350697+0000","last_change":"2023-12-01T03:12:35.346395+0000","last_active":"2023-12-01T03:12:35.350697+0000","last_peered":"2023-12-01T03:12:35.350697+0000","last_clean":"2023-12-01T03:12:35.350697+0000","last_became_active":"2023-12-01T03:12:35.344072+0000","last_became_peered":"2023-12-01T03:12:35.344072+0000","last_unstale":"2023-12-01T03:12:35.350697+0000","last_undegraded":"2023-12-01T03:12:35.350697+0000","last_fullsized":"2023-12-01T03:12:35.350697+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:10:45.146806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1f","version":"255'4","reported_seq":15,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.365834+0000","last_change":"2023-12-01T03:12:34.867899+0000","last_active":"2023-12-01T03:12:35.365834+0000","last_peered":"2023-12-01T03:12:35.365834+0000","last_clean":"2023-12-01T03:12:35.365834+0000","last_became_active":"2023-12-01T03:12:34.867559+0000","last_became_peered":"2023-12-01T03:12:34.867559+0000","last_unstale":"2023-12-01T03:12:35.365834+0000","last_undegraded":"2023-12-01T03:12:35.365834+0000","last_fullsized":"2023-12-01T03:12:35.365834+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:49.066648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350673+0000","last_change":"2023-12-01T03:12:35.348505+0000","last_active":"2023-12-01T03:12:35.350673+0000","last_peered":"2023-12-01T03:12:35.350673+0000","last_clean":"2023-12-01T03:12:35.350673+0000","last_became_active":"2023-12-01T03:12:35.347836+0000","last_became_peered":"2023-12-01T03:12:35.347836+0000","last_unstale":"2023-12-01T03:12:35.350673+0000","last_undegraded":"2023-12-01T03:12:35.350673+0000","last_fullsized":"2023-12-01T03:12:35.350673+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:13:59.927209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.1e","version":"255'1","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.367023+0000","last_change":"2023-12-01T03:12:34.868270+0000","last_active":"2023-12-01T03:12:35.367023+0000","last_peered":"2023-12-01T03:12:35.367023+0000","last_clean":"2023-12-01T03:12:35.367023+0000","last_became_active":"2023-12-01T03:12:34.867687+0000","last_became_peered":"2023-12-01T03:12:34.867687+0000","last_unstale":"2023-12-01T03:12:35.367023+0000","last_undegraded":"2023-12-01T03:12:35.367023+0000","last_fullsized":"2023-12-01T03:12:35.367023+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:03:51.996656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.359356+0000","last_change":"2023-12-01T03:12:35.355354+0000","last_active":"2023-12-01T03:12:35.359356+0000","last_peered":"2023-12-01T03:12:35.359356+0000","last_clean":"2023-12-01T03:12:35.359356+0000","last_became_active":"2023-12-01T03:12:35.354183+0000","last_became_peered":"2023-12-01T03:12:35.354183+0000","last_unstale":"2023-12-01T03:12:35.359356+0000","last_undegraded":"2023-12-01T03:12:35.359356+0000","last_fullsized":"2023-12-01T03:12:35.359356+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1d","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.364743+0000","last_change":"2023-12-01T03:12:34.869563+0000","last_active":"2023-12-01T03:12:35.364743+0000","last_peered":"2023-12-01T03:12:35.364743+0000","last_clean":"2023-12-01T03:12:35.364743+0000","last_became_active":"2023-12-01T03:12:34.869160+0000","last_became_peered":"2023-12-01T03:12:34.869160+0000","last_unstale":"2023-12-01T03:12:35.364743+0000","last_undegraded":"2023-12-01T03:12:35.364743+0000","last_fullsized":"2023-12-01T03:12:35.364743+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:19.315026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.354616+0000","last_change":"2023-12-01T03:12:35.351477+0000","last_active":"2023-12-01T03:12:35.354616+0000","last_peered":"2023-12-01T03:12:35.354616+0000","last_clean":"2023-12-01T03:12:35.354616+0000","last_became_active":"2023-12-01T03:12:35.351219+0000","last_became_peered":"2023-12-01T03:12:35.351219+0000","last_unstale":"2023-12-01T03:12:35.354616+0000","last_undegraded":"2023-12-01T03:12:35.354616+0000","last_fullsized":"2023-12-01T03:12:35.354616+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1c","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.364480+0000","last_change":"2023-12-01T03:12:34.865447+0000","last_active":"2023-12-01T03:12:35.364480+0000","last_peered":"2023-12-01T03:12:35.364480+0000","last_clean":"2023-12-01T03:12:35.364480+0000","last_became_active":"2023-12-01T03:12:34.865092+0000","last_became_peered":"2023-12-01T03:12:34.865092+0000","last_unstale":"2023-12-01T03:12:35.364480+0000","last_undegraded":"2023-12-01T03:12:35.364480+0000","last_fullsized":"2023-12-01T03:12:35.364480+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:55:07.934979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355407+0000","last_change":"2023-12-01T03:12:35.352631+0000","last_active":"2023-12-01T03:12:35.355407+0000","last_peered":"2023-12-01T03:12:35.355407+0000","last_clean":"2023-12-01T03:12:35.355407+0000","last_became_active":"2023-12-01T03:12:35.351948+0000","last_became_peered":"2023-12-01T03:12:35.351948+0000","last_unstale":"2023-12-01T03:12:35.355407+0000","last_undegraded":"2023-12-01T03:12:35.355407+0000","last_fullsized":"2023-12-01T03:12:35.355407+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1b","version":"255'1","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358726+0000","last_change":"2023-12-01T03:12:34.857676+0000","last_active":"2023-12-01T03:12:35.358726+0000","last_peered":"2023-12-01T03:12:35.358726+0000","last_clean":"2023-12-01T03:12:35.358726+0000","last_became_active":"2023-12-01T03:12:34.857365+0000","last_became_peered":"2023-12-01T03:12:34.857365+0000","last_unstale":"2023-12-01T03:12:35.358726+0000","last_undegraded":"2023-12-01T03:12:35.358726+0000","last_fullsized":"2023-12-01T03:12:35.358726+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:22.088313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.349025+0000","last_change":"2023-12-01T03:12:35.345935+0000","last_active":"2023-12-01T03:12:35.349025+0000","last_peered":"2023-12-01T03:12:35.349025+0000","last_clean":"2023-12-01T03:12:35.349025+0000","last_became_active":"2023-12-01T03:12:35.344888+0000","last_became_peered":"2023-12-01T03:12:35.344888+0000","last_unstale":"2023-12-01T03:12:35.349025+0000","last_undegraded":"2023-12-01T03:12:35.349025+0000","last_fullsized":"2023-12-01T03:12:35.349025+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:46:13.355768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1a","version":"255'1","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.363324+0000","last_change":"2023-12-01T03:12:34.860204+0000","last_active":"2023-12-01T03:12:35.363324+0000","last_peered":"2023-12-01T03:12:35.363324+0000","last_clean":"2023-12-01T03:12:35.363324+0000","last_became_active":"2023-12-01T03:12:34.858034+0000","last_became_peered":"2023-12-01T03:12:34.858034+0000","last_unstale":"2023-12-01T03:12:35.363324+0000","last_undegraded":"2023-12-01T03:12:35.363324+0000","last_fullsized":"2023-12-01T03:12:35.363324+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:23:38.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350169+0000","last_change":"2023-12-01T03:12:35.342560+0000","last_active":"2023-12-01T03:12:35.350169+0000","last_peered":"2023-12-01T03:12:35.350169+0000","last_clean":"2023-12-01T03:12:35.350169+0000","last_became_active":"2023-12-01T03:12:35.342293+0000","last_became_peered":"2023-12-01T03:12:35.342293+0000","last_unstale":"2023-12-01T03:12:35.350169+0000","last_undegraded":"2023-12-01T03:12:35.350169+0000","last_fullsized":"2023-12-01T03:12:35.350169+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:41:55.348205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.338718+0000","last_change":"2023-12-01T03:12:34.863171+0000","last_active":"2023-12-01T03:12:35.338718+0000","last_peered":"2023-12-01T03:12:35.338718+0000","last_clean":"2023-12-01T03:12:35.338718+0000","last_became_active":"2023-12-01T03:12:34.862887+0000","last_became_peered":"2023-12-01T03:12:34.862887+0000","last_unstale":"2023-12-01T03:12:35.338718+0000","last_undegraded":"2023-12-01T03:12:35.338718+0000","last_fullsized":"2023-12-01T03:12:35.338718+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:51:11.709287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357903+0000","last_change":"2023-12-01T03:12:35.354236+0000","last_active":"2023-12-01T03:12:35.357903+0000","last_peered":"2023-12-01T03:12:35.357903+0000","last_clean":"2023-12-01T03:12:35.357903+0000","last_became_active":"2023-12-01T03:12:35.350478+0000","last_became_peered":"2023-12-01T03:12:35.350478+0000","last_unstale":"2023-12-01T03:12:35.357903+0000","last_undegraded":"2023-12-01T03:12:35.357903+0000","last_fullsized":"2023-12-01T03:12:35.357903+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.336460+0000","last_change":"2023-12-01T03:12:34.864288+0000","last_active":"2023-12-01T03:12:35.336460+0000","last_peered":"2023-12-01T03:12:35.336460+0000","last_clean":"2023-12-01T03:12:35.336460+0000","last_became_active":"2023-12-01T03:12:34.863960+0000","last_became_peered":"2023-12-01T03:12:34.863960+0000","last_unstale":"2023-12-01T03:12:35.336460+0000","last_undegraded":"2023-12-01T03:12:35.336460+0000","last_fullsized":"2023-12-01T03:12:35.336460+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:02.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357796+0000","last_change":"2023-12-01T03:12:35.352373+0000","last_active":"2023-12-01T03:12:35.357796+0000","last_peered":"2023-12-01T03:12:35.357796+0000","last_clean":"2023-12-01T03:12:35.357796+0000","last_became_active":"2023-12-01T03:12:35.352063+0000","last_became_peered":"2023-12-01T03:12:35.352063+0000","last_unstale":"2023-12-01T03:12:35.357796+0000","last_undegraded":"2023-12-01T03:12:35.357796+0000","last_fullsized":"2023-12-01T03:12:35.357796+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:47:19.476510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.329796+0000","last_change":"2023-12-01T03:12:34.867539+0000","last_active":"2023-12-01T03:12:35.329796+0000","last_peered":"2023-12-01T03:12:35.329796+0000","last_clean":"2023-12-01T03:12:35.329796+0000","last_became_active":"2023-12-01T03:12:34.866362+0000","last_became_peered":"2023-12-01T03:12:34.866362+0000","last_unstale":"2023-12-01T03:12:35.329796+0000","last_undegraded":"2023-12-01T03:12:35.329796+0000","last_fullsized":"2023-12-01T03:12:35.329796+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:41:52.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360293+0000","last_change":"2023-12-01T03:12:35.359264+0000","last_active":"2023-12-01T03:12:35.360293+0000","last_peered":"2023-12-01T03:12:35.360293+0000","last_clean":"2023-12-01T03:12:35.360293+0000","last_became_active":"2023-12-01T03:12:35.358987+0000","last_became_peered":"2023-12-01T03:12:35.358987+0000","last_unstale":"2023-12-01T03:12:35.360293+0000","last_undegraded":"2023-12-01T03:12:35.360293+0000","last_fullsized":"2023-12-01T03:12:35.360293+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.340265+0000","last_change":"2023-12-01T03:12:34.859136+0000","last_active":"2023-12-01T03:12:35.340265+0000","last_peered":"2023-12-01T03:12:35.340265+0000","last_clean":"2023-12-01T03:12:35.340265+0000","last_became_active":"2023-12-01T03:12:34.857840+0000","last_became_peered":"2023-12-01T03:12:34.857840+0000","last_unstale":"2023-12-01T03:12:35.340265+0000","last_undegraded":"2023-12-01T03:12:35.340265+0000","last_fullsized":"2023-12-01T03:12:35.340265+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:34:36.726453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.349687+0000","last_change":"2023-12-01T03:12:35.346099+0000","last_active":"2023-12-01T03:12:35.349687+0000","last_peered":"2023-12-01T03:12:35.349687+0000","last_clean":"2023-12-01T03:12:35.349687+0000","last_became_active":"2023-12-01T03:12:35.345823+0000","last_became_peered":"2023-12-01T03:12:35.345823+0000","last_unstale":"2023-12-01T03:12:35.349687+0000","last_undegraded":"2023-12-01T03:12:35.349687+0000","last_fullsized":"2023-12-01T03:12:35.349687+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:33:02.156364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.15","version":"255'3","reported_seq":14,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.362807+0000","last_change":"2023-12-01T03:12:34.860533+0000","last_active":"2023-12-01T03:12:35.362807+0000","last_peered":"2023-12-01T03:12:35.362807+0000","last_clean":"2023-12-01T03:12:35.362807+0000","last_became_active":"2023-12-01T03:12:34.858865+0000","last_became_peered":"2023-12-01T03:12:34.858865+0000","last_unstale":"2023-12-01T03:12:35.362807+0000","last_undegraded":"2023-12-01T03:12:35.362807+0000","last_fullsized":"2023-12-01T03:12:35.362807+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:40:22.088313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360472+0000","last_change":"2023-12-01T03:12:35.354831+0000","last_active":"2023-12-01T03:12:35.360472+0000","last_peered":"2023-12-01T03:12:35.360472+0000","last_clean":"2023-12-01T03:12:35.360472+0000","last_became_active":"2023-12-01T03:12:35.351242+0000","last_became_peered":"2023-12-01T03:12:35.351242+0000","last_unstale":"2023-12-01T03:12:35.360472+0000","last_undegraded":"2023-12-01T03:12:35.360472+0000","last_fullsized":"2023-12-01T03:12:35.360472+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:22:12.850905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.14","version":"255'6","reported_seq":17,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.366096+0000","last_change":"2023-12-01T03:12:34.857964+0000","last_active":"2023-12-01T03:12:35.366096+0000","last_peered":"2023-12-01T03:12:35.366096+0000","last_clean":"2023-12-01T03:12:35.366096+0000","last_became_active":"2023-12-01T03:12:34.856582+0000","last_became_peered":"2023-12-01T03:12:34.856582+0000","last_unstale":"2023-12-01T03:12:35.366096+0000","last_undegraded":"2023-12-01T03:12:35.366096+0000","last_fullsized":"2023-12-01T03:12:35.366096+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:46:48.550869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360788+0000","last_change":"2023-12-01T03:12:35.356681+0000","last_active":"2023-12-01T03:12:35.360788+0000","last_peered":"2023-12-01T03:12:35.360788+0000","last_clean":"2023-12-01T03:12:35.360788+0000","last_became_active":"2023-12-01T03:12:35.354973+0000","last_became_peered":"2023-12-01T03:12:35.354973+0000","last_unstale":"2023-12-01T03:12:35.360788+0000","last_undegraded":"2023-12-01T03:12:35.360788+0000","last_fullsized":"2023-12-01T03:12:35.360788+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:23.725637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.13","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.365830+0000","last_change":"2023-12-01T03:12:34.863097+0000","last_active":"2023-12-01T03:12:35.365830+0000","last_peered":"2023-12-01T03:12:35.365830+0000","last_clean":"2023-12-01T03:12:35.365830+0000","last_became_active":"2023-12-01T03:12:34.862657+0000","last_became_peered":"2023-12-01T03:12:34.862657+0000","last_unstale":"2023-12-01T03:12:35.365830+0000","last_undegraded":"2023-12-01T03:12:35.365830+0000","last_fullsized":"2023-12-01T03:12:35.365830+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:44:22.709559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.352624+0000","last_change":"2023-12-01T03:12:35.347992+0000","last_active":"2023-12-01T03:12:35.352624+0000","last_peered":"2023-12-01T03:12:35.352624+0000","last_clean":"2023-12-01T03:12:35.352624+0000","last_became_active":"2023-12-01T03:12:35.347364+0000","last_became_peered":"2023-12-01T03:12:35.347364+0000","last_unstale":"2023-12-01T03:12:35.352624+0000","last_undegraded":"2023-12-01T03:12:35.352624+0000","last_fullsized":"2023-12-01T03:12:35.352624+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:40:26.458289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.337682+0000","last_change":"2023-12-01T03:12:34.860804+0000","last_active":"2023-12-01T03:12:35.337682+0000","last_peered":"2023-12-01T03:12:35.337682+0000","last_clean":"2023-12-01T03:12:35.337682+0000","last_became_active":"2023-12-01T03:12:34.859477+0000","last_became_peered":"2023-12-01T03:12:34.859477+0000","last_unstale":"2023-12-01T03:12:35.337682+0000","last_undegraded":"2023-12-01T03:12:35.337682+0000","last_fullsized":"2023-12-01T03:12:35.337682+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:00.153556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355384+0000","last_change":"2023-12-01T03:12:35.351909+0000","last_active":"2023-12-01T03:12:35.355384+0000","last_peered":"2023-12-01T03:12:35.355384+0000","last_clean":"2023-12-01T03:12:35.355384+0000","last_became_active":"2023-12-01T03:12:35.351434+0000","last_became_peered":"2023-12-01T03:12:35.351434+0000","last_unstale":"2023-12-01T03:12:35.355384+0000","last_undegraded":"2023-12-01T03:12:35.355384+0000","last_fullsized":"2023-12-01T03:12:35.355384+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:04:09.892305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.336924+0000","last_change":"2023-12-01T03:12:34.860578+0000","last_active":"2023-12-01T03:12:35.336924+0000","last_peered":"2023-12-01T03:12:35.336924+0000","last_clean":"2023-12-01T03:12:35.336924+0000","last_became_active":"2023-12-01T03:12:34.859107+0000","last_became_peered":"2023-12-01T03:12:34.859107+0000","last_unstale":"2023-12-01T03:12:35.336924+0000","last_undegraded":"2023-12-01T03:12:35.336924+0000","last_fullsized":"2023-12-01T03:12:35.336924+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:23:38.172533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360834+0000","last_change":"2023-12-01T03:12:35.354180+0000","last_active":"2023-12-01T03:12:35.360834+0000","last_peered":"2023-12-01T03:12:35.360834+0000","last_clean":"2023-12-01T03:12:35.360834+0000","last_became_active":"2023-12-01T03:12:35.353835+0000","last_became_peered":"2023-12-01T03:12:35.353835+0000","last_unstale":"2023-12-01T03:12:35.360834+0000","last_undegraded":"2023-12-01T03:12:35.360834+0000","last_fullsized":"2023-12-01T03:12:35.360834+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:15:18.210277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.10","version":"255'8","reported_seq":19,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.367485+0000","last_change":"2023-12-01T03:12:34.866244+0000","last_active":"2023-12-01T03:12:35.367485+0000","last_peered":"2023-12-01T03:12:35.367485+0000","last_clean":"2023-12-01T03:12:35.367485+0000","last_became_active":"2023-12-01T03:12:34.865944+0000","last_became_peered":"2023-12-01T03:12:34.865944+0000","last_unstale":"2023-12-01T03:12:35.367485+0000","last_undegraded":"2023-12-01T03:12:35.367485+0000","last_fullsized":"2023-12-01T03:12:35.367485+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:48:28.385262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.356768+0000","last_change":"2023-12-01T03:12:35.351685+0000","last_active":"2023-12-01T03:12:35.356768+0000","last_peered":"2023-12-01T03:12:35.356768+0000","last_clean":"2023-12-01T03:12:35.356768+0000","last_became_active":"2023-12-01T03:12:35.349403+0000","last_became_peered":"2023-12-01T03:12:35.349403+0000","last_unstale":"2023-12-01T03:12:35.356768+0000","last_undegraded":"2023-12-01T03:12:35.356768+0000","last_fullsized":"2023-12-01T03:12:35.356768+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:42:08.326648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.f","version":"255'1","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.360105+0000","last_change":"2023-12-01T03:12:34.864936+0000","last_active":"2023-12-01T03:12:35.360105+0000","last_peered":"2023-12-01T03:12:35.360105+0000","last_clean":"2023-12-01T03:12:35.360105+0000","last_became_active":"2023-12-01T03:12:34.864646+0000","last_became_peered":"2023-12-01T03:12:34.864646+0000","last_unstale":"2023-12-01T03:12:35.360105+0000","last_undegraded":"2023-12-01T03:12:35.360105+0000","last_fullsized":"2023-12-01T03:12:35.360105+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:10:00.153556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355867+0000","last_change":"2023-12-01T03:12:35.352911+0000","last_active":"2023-12-01T03:12:35.355867+0000","last_peered":"2023-12-01T03:12:35.355867+0000","last_clean":"2023-12-01T03:12:35.355867+0000","last_became_active":"2023-12-01T03:12:35.352639+0000","last_became_peered":"2023-12-01T03:12:35.352639+0000","last_unstale":"2023-12-01T03:12:35.355867+0000","last_undegraded":"2023-12-01T03:12:35.355867+0000","last_fullsized":"2023-12-01T03:12:35.355867+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:20:49.518896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.e","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.364684+0000","last_change":"2023-12-01T03:12:34.866465+0000","last_active":"2023-12-01T03:12:35.364684+0000","last_peered":"2023-12-01T03:12:35.364684+0000","last_clean":"2023-12-01T03:12:35.364684+0000","last_became_active":"2023-12-01T03:12:34.865451+0000","last_became_peered":"2023-12-01T03:12:34.865451+0000","last_unstale":"2023-12-01T03:12:35.364684+0000","last_undegraded":"2023-12-01T03:12:35.364684+0000","last_fullsized":"2023-12-01T03:12:35.364684+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:17:29.848009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.349085+0000","last_change":"2023-12-01T03:12:35.346831+0000","last_active":"2023-12-01T03:12:35.349085+0000","last_peered":"2023-12-01T03:12:35.349085+0000","last_clean":"2023-12-01T03:12:35.349085+0000","last_became_active":"2023-12-01T03:12:35.346545+0000","last_became_peered":"2023-12-01T03:12:35.346545+0000","last_unstale":"2023-12-01T03:12:35.349085+0000","last_undegraded":"2023-12-01T03:12:35.349085+0000","last_fullsized":"2023-12-01T03:12:35.349085+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T14:03:37.465541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.d","version":"255'1","reported_seq":12,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.361836+0000","last_change":"2023-12-01T03:12:34.866633+0000","last_active":"2023-12-01T03:12:35.361836+0000","last_peered":"2023-12-01T03:12:35.361836+0000","last_clean":"2023-12-01T03:12:35.361836+0000","last_became_active":"2023-12-01T03:12:34.865565+0000","last_became_peered":"2023-12-01T03:12:34.865565+0000","last_unstale":"2023-12-01T03:12:35.361836+0000","last_undegraded":"2023-12-01T03:12:35.361836+0000","last_fullsized":"2023-12-01T03:12:35.361836+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:38:37.392214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358091+0000","last_change":"2023-12-01T03:12:35.348060+0000","last_active":"2023-12-01T03:12:35.358091+0000","last_peered":"2023-12-01T03:12:35.358091+0000","last_clean":"2023-12-01T03:12:35.358091+0000","last_became_active":"2023-12-01T03:12:35.347816+0000","last_became_peered":"2023-12-01T03:12:35.347816+0000","last_unstale":"2023-12-01T03:12:35.358091+0000","last_undegraded":"2023-12-01T03:12:35.358091+0000","last_fullsized":"2023-12-01T03:12:35.358091+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.338075+0000","last_change":"2023-12-01T03:12:34.858480+0000","last_active":"2023-12-01T03:12:35.338075+0000","last_peered":"2023-12-01T03:12:35.338075+0000","last_clean":"2023-12-01T03:12:35.338075+0000","last_became_active":"2023-12-01T03:12:34.857292+0000","last_became_peered":"2023-12-01T03:12:34.857292+0000","last_unstale":"2023-12-01T03:12:35.338075+0000","last_undegraded":"2023-12-01T03:12:35.338075+0000","last_fullsized":"2023-12-01T03:12:35.338075+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:55:02.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350714+0000","last_change":"2023-12-01T03:12:35.342072+0000","last_active":"2023-12-01T03:12:35.350714+0000","last_peered":"2023-12-01T03:12:35.350714+0000","last_clean":"2023-12-01T03:12:35.350714+0000","last_became_active":"2023-12-01T03:12:35.341786+0000","last_became_peered":"2023-12-01T03:12:35.341786+0000","last_unstale":"2023-12-01T03:12:35.350714+0000","last_undegraded":"2023-12-01T03:12:35.350714+0000","last_fullsized":"2023-12-01T03:12:35.350714+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.b","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.362890+0000","last_change":"2023-12-01T03:12:34.860490+0000","last_active":"2023-12-01T03:12:35.362890+0000","last_peered":"2023-12-01T03:12:35.362890+0000","last_clean":"2023-12-01T03:12:35.362890+0000","last_became_active":"2023-12-01T03:12:34.860228+0000","last_became_peered":"2023-12-01T03:12:34.860228+0000","last_unstale":"2023-12-01T03:12:35.362890+0000","last_undegraded":"2023-12-01T03:12:35.362890+0000","last_fullsized":"2023-12-01T03:12:35.362890+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:54:16.920168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350003+0000","last_change":"2023-12-01T03:12:35.345216+0000","last_active":"2023-12-01T03:12:35.350003+0000","last_peered":"2023-12-01T03:12:35.350003+0000","last_clean":"2023-12-01T03:12:35.350003+0000","last_became_active":"2023-12-01T03:12:35.344046+0000","last_became_peered":"2023-12-01T03:12:35.344046+0000","last_unstale":"2023-12-01T03:12:35.350003+0000","last_undegraded":"2023-12-01T03:12:35.350003+0000","last_fullsized":"2023-12-01T03:12:35.350003+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:19:43.777998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.334144+0000","last_change":"2023-12-01T03:12:34.864610+0000","last_active":"2023-12-01T03:12:35.334144+0000","last_peered":"2023-12-01T03:12:35.334144+0000","last_clean":"2023-12-01T03:12:35.334144+0000","last_became_active":"2023-12-01T03:12:34.864320+0000","last_became_peered":"2023-12-01T03:12:34.864320+0000","last_unstale":"2023-12-01T03:12:35.334144+0000","last_undegraded":"2023-12-01T03:12:35.334144+0000","last_fullsized":"2023-12-01T03:12:35.334144+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:25:41.361208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358575+0000","last_change":"2023-12-01T03:12:35.352877+0000","last_active":"2023-12-01T03:12:35.358575+0000","last_peered":"2023-12-01T03:12:35.358575+0000","last_clean":"2023-12-01T03:12:35.358575+0000","last_became_active":"2023-12-01T03:12:35.352593+0000","last_became_peered":"2023-12-01T03:12:35.352593+0000","last_unstale":"2023-12-01T03:12:35.358575+0000","last_undegraded":"2023-12-01T03:12:35.358575+0000","last_fullsized":"2023-12-01T03:12:35.358575+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:48:48.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.334961+0000","last_change":"2023-12-01T03:12:34.864988+0000","last_active":"2023-12-01T03:12:35.334961+0000","last_peered":"2023-12-01T03:12:35.334961+0000","last_clean":"2023-12-01T03:12:35.334961+0000","last_became_active":"2023-12-01T03:12:34.864626+0000","last_became_peered":"2023-12-01T03:12:34.864626+0000","last_unstale":"2023-12-01T03:12:35.334961+0000","last_undegraded":"2023-12-01T03:12:35.334961+0000","last_fullsized":"2023-12-01T03:12:35.334961+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:15:35.500459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.354839+0000","last_change":"2023-12-01T03:12:35.351460+0000","last_active":"2023-12-01T03:12:35.354839+0000","last_peered":"2023-12-01T03:12:35.354839+0000","last_clean":"2023-12-01T03:12:35.354839+0000","last_became_active":"2023-12-01T03:12:35.351166+0000","last_became_peered":"2023-12-01T03:12:35.351166+0000","last_unstale":"2023-12-01T03:12:35.354839+0000","last_undegraded":"2023-12-01T03:12:35.354839+0000","last_fullsized":"2023-12-01T03:12:35.354839+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:43:19.046826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.338056+0000","last_change":"2023-12-01T03:12:34.859322+0000","last_active":"2023-12-01T03:12:35.338056+0000","last_peered":"2023-12-01T03:12:35.338056+0000","last_clean":"2023-12-01T03:12:35.338056+0000","last_became_active":"2023-12-01T03:12:34.859042+0000","last_became_peered":"2023-12-01T03:12:34.859042+0000","last_unstale":"2023-12-01T03:12:35.338056+0000","last_undegraded":"2023-12-01T03:12:35.338056+0000","last_fullsized":"2023-12-01T03:12:35.338056+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:26:08.980764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355264+0000","last_change":"2023-12-01T03:12:35.348542+0000","last_active":"2023-12-01T03:12:35.355264+0000","last_peered":"2023-12-01T03:12:35.355264+0000","last_clean":"2023-12-01T03:12:35.355264+0000","last_became_active":"2023-12-01T03:12:35.348272+0000","last_became_peered":"2023-12-01T03:12:35.348272+0000","last_unstale":"2023-12-01T03:12:35.355264+0000","last_undegraded":"2023-12-01T03:12:35.355264+0000","last_fullsized":"2023-12-01T03:12:35.355264+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:55:08.951057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.7","version":"255'2","reported_seq":13,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.363839+0000","last_change":"2023-12-01T03:12:34.860198+0000","last_active":"2023-12-01T03:12:35.363839+0000","last_peered":"2023-12-01T03:12:35.363839+0000","last_clean":"2023-12-01T03:12:35.363839+0000","last_became_active":"2023-12-01T03:12:34.859894+0000","last_became_peered":"2023-12-01T03:12:34.859894+0000","last_unstale":"2023-12-01T03:12:35.363839+0000","last_undegraded":"2023-12-01T03:12:35.363839+0000","last_fullsized":"2023-12-01T03:12:35.363839+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:36.882914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.356357+0000","last_change":"2023-12-01T03:12:35.351385+0000","last_active":"2023-12-01T03:12:35.356357+0000","last_peered":"2023-12-01T03:12:35.356357+0000","last_clean":"2023-12-01T03:12:35.356357+0000","last_became_active":"2023-12-01T03:12:35.351119+0000","last_became_peered":"2023-12-01T03:12:35.351119+0000","last_unstale":"2023-12-01T03:12:35.356357+0000","last_undegraded":"2023-12-01T03:12:35.356357+0000","last_fullsized":"2023-12-01T03:12:35.356357+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:24:20.331105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.6","version":"256'1","reported_seq":12,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-12-01T03:12:36.337500+0000","last_change":"2023-12-01T03:12:34.868249+0000","last_active":"2023-12-01T03:12:36.337500+0000","last_peered":"2023-12-01T03:12:36.337500+0000","last_clean":"2023-12-01T03:12:36.337500+0000","last_became_active":"2023-12-01T03:12:34.867861+0000","last_became_peered":"2023-12-01T03:12:34.867861+0000","last_unstale":"2023-12-01T03:12:36.337500+0000","last_undegraded":"2023-12-01T03:12:36.337500+0000","last_fullsized":"2023-12-01T03:12:36.337500+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:00.274064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.344055+0000","last_change":"2023-12-01T03:12:35.342473+0000","last_active":"2023-12-01T03:12:35.344055+0000","last_peered":"2023-12-01T03:12:35.344055+0000","last_clean":"2023-12-01T03:12:35.344055+0000","last_became_active":"2023-12-01T03:12:35.342193+0000","last_became_peered":"2023-12-01T03:12:35.342193+0000","last_unstale":"2023-12-01T03:12:35.344055+0000","last_undegraded":"2023-12-01T03:12:35.344055+0000","last_fullsized":"2023-12-01T03:12:35.344055+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:00.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.340708+0000","last_change":"2023-12-01T03:12:34.857807+0000","last_active":"2023-12-01T03:12:35.340708+0000","last_peered":"2023-12-01T03:12:35.340708+0000","last_clean":"2023-12-01T03:12:35.340708+0000","last_became_active":"2023-12-01T03:12:34.857501+0000","last_became_peered":"2023-12-01T03:12:34.857501+0000","last_unstale":"2023-12-01T03:12:35.340708+0000","last_undegraded":"2023-12-01T03:12:35.340708+0000","last_fullsized":"2023-12-01T03:12:35.340708+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:36.882914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.358403+0000","last_change":"2023-12-01T03:12:35.351958+0000","last_active":"2023-12-01T03:12:35.358403+0000","last_peered":"2023-12-01T03:12:35.358403+0000","last_clean":"2023-12-01T03:12:35.358403+0000","last_became_active":"2023-12-01T03:12:35.351724+0000","last_became_peered":"2023-12-01T03:12:35.351724+0000","last_unstale":"2023-12-01T03:12:35.358403+0000","last_undegraded":"2023-12-01T03:12:35.358403+0000","last_fullsized":"2023-12-01T03:12:35.358403+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:45.819776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.334149+0000","last_change":"2023-12-01T03:12:34.865848+0000","last_active":"2023-12-01T03:12:35.334149+0000","last_peered":"2023-12-01T03:12:35.334149+0000","last_clean":"2023-12-01T03:12:35.334149+0000","last_became_active":"2023-12-01T03:12:34.865157+0000","last_became_peered":"2023-12-01T03:12:34.865157+0000","last_unstale":"2023-12-01T03:12:35.334149+0000","last_undegraded":"2023-12-01T03:12:35.334149+0000","last_fullsized":"2023-12-01T03:12:35.334149+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:37:10.788196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.349087+0000","last_change":"2023-12-01T03:12:35.347141+0000","last_active":"2023-12-01T03:12:35.349087+0000","last_peered":"2023-12-01T03:12:35.349087+0000","last_clean":"2023-12-01T03:12:35.349087+0000","last_became_active":"2023-12-01T03:12:35.345505+0000","last_became_peered":"2023-12-01T03:12:35.345505+0000","last_unstale":"2023-12-01T03:12:35.349087+0000","last_undegraded":"2023-12-01T03:12:35.349087+0000","last_fullsized":"2023-12-01T03:12:35.349087+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:29:01.535641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.327346+0000","last_change":"2023-12-01T03:12:34.870482+0000","last_active":"2023-12-01T03:12:35.327346+0000","last_peered":"2023-12-01T03:12:35.327346+0000","last_clean":"2023-12-01T03:12:35.327346+0000","last_became_active":"2023-12-01T03:12:34.870111+0000","last_became_peered":"2023-12-01T03:12:34.870111+0000","last_unstale":"2023-12-01T03:12:35.327346+0000","last_undegraded":"2023-12-01T03:12:35.327346+0000","last_fullsized":"2023-12-01T03:12:35.327346+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:37:24.009174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.355639+0000","last_change":"2023-12-01T03:12:35.349891+0000","last_active":"2023-12-01T03:12:35.355639+0000","last_peered":"2023-12-01T03:12:35.355639+0000","last_clean":"2023-12-01T03:12:35.355639+0000","last_became_active":"2023-12-01T03:12:35.345336+0000","last_became_peered":"2023-12-01T03:12:35.345336+0000","last_unstale":"2023-12-01T03:12:35.355639+0000","last_undegraded":"2023-12-01T03:12:35.355639+0000","last_fullsized":"2023-12-01T03:12:35.355639+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:53:10.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.335015+0000","last_change":"2023-12-01T03:12:34.863164+0000","last_active":"2023-12-01T03:12:35.335015+0000","last_peered":"2023-12-01T03:12:35.335015+0000","last_clean":"2023-12-01T03:12:35.335015+0000","last_became_active":"2023-12-01T03:12:34.862454+0000","last_became_peered":"2023-12-01T03:12:34.862454+0000","last_unstale":"2023-12-01T03:12:35.335015+0000","last_undegraded":"2023-12-01T03:12:35.335015+0000","last_fullsized":"2023-12-01T03:12:35.335015+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:06:12.944465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.350189+0000","last_change":"2023-12-01T03:12:35.343968+0000","last_active":"2023-12-01T03:12:35.350189+0000","last_peered":"2023-12-01T03:12:35.350189+0000","last_clean":"2023-12-01T03:12:35.350189+0000","last_became_active":"2023-12-01T03:12:35.343680+0000","last_became_peered":"2023-12-01T03:12:35.343680+0000","last_unstale":"2023-12-01T03:12:35.350189+0000","last_undegraded":"2023-12-01T03:12:35.350189+0000","last_fullsized":"2023-12-01T03:12:35.350189+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:21.113177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.338645+0000","last_change":"2023-12-01T03:12:34.860753+0000","last_active":"2023-12-01T03:12:35.338645+0000","last_peered":"2023-12-01T03:12:35.338645+0000","last_clean":"2023-12-01T03:12:35.338645+0000","last_became_active":"2023-12-01T03:12:34.858035+0000","last_became_peered":"2023-12-01T03:12:34.858035+0000","last_unstale":"2023-12-01T03:12:35.338645+0000","last_undegraded":"2023-12-01T03:12:35.338645+0000","last_fullsized":"2023-12-01T03:12:35.338645+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:17:40.965606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.356341+0000","last_change":"2023-12-01T03:12:35.351762+0000","last_active":"2023-12-01T03:12:35.356341+0000","last_peered":"2023-12-01T03:12:35.356341+0000","last_clean":"2023-12-01T03:12:35.356341+0000","last_became_active":"2023-12-01T03:12:35.351508+0000","last_became_peered":"2023-12-01T03:12:35.351508+0000","last_unstale":"2023-12-01T03:12:35.356341+0000","last_undegraded":"2023-12-01T03:12:35.356341+0000","last_fullsized":"2023-12-01T03:12:35.356341+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T15:08:34.191802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":11,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.335259+0000","last_change":"2023-12-01T03:12:34.866607+0000","last_active":"2023-12-01T03:12:35.335259+0000","last_peered":"2023-12-01T03:12:35.335259+0000","last_clean":"2023-12-01T03:12:35.335259+0000","last_became_active":"2023-12-01T03:12:34.865003+0000","last_became_peered":"2023-12-01T03:12:34.865003+0000","last_unstale":"2023-12-01T03:12:35.335259+0000","last_undegraded":"2023-12-01T03:12:35.335259+0000","last_fullsized":"2023-12-01T03:12:35.335259+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:33.796446+0000","last_clean_scrub_stamp":"2023-12-01T03:12:33.796446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:29:11.426576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":10,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.357675+0000","last_change":"2023-12-01T03:12:35.351337+0000","last_active":"2023-12-01T03:12:35.357675+0000","last_peered":"2023-12-01T03:12:35.357675+0000","last_clean":"2023-12-01T03:12:35.357675+0000","last_became_active":"2023-12-01T03:12:35.351044+0000","last_became_peered":"2023-12-01T03:12:35.351044+0000","last_unstale":"2023-12-01T03:12:35.357675+0000","last_undegraded":"2023-12-01T03:12:35.357675+0000","last_fullsized":"2023-12-01T03:12:35.357675+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T03:12:34.812524+0000","last_clean_scrub_stamp":"2023-12-01T03:12:34.812524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:56:01.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"241'359","reported_seq":1272,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-01T03:12:35.334911+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:12:35.334911+0000","last_peered":"2023-12-01T03:12:35.334911+0000","last_clean":"2023-12-01T03:12:35.334911+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:12:35.334911+0000","last_undegraded":"2023-12-01T03:12:35.334911+0000","last_fullsized":"2023-12-01T03:12:35.334911+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2228,"num_read_kb":4632,"num_write":599,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60452,"kb_used_data":3164,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311388,"statfs":{"total":96636764160,"available":96574861312,"internally_reserved":0,"allocated":3239936,"data_stored":1591139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0620000000000001,"15min":1.0760000000000001},"min":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.095},{"interface":"front","average":{"1min":1.343,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":1.1779999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":0.83999999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":1.397},{"interface":"front","average":{"1min":1.363,"5min":1.3069999999999999,"15min":1.327},"min":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.3720000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.167},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.95,"5min":1.95,"15min":2.1179999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.349,"5min":1.254,"15min":1.264},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.246}]},{"osd":3,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2729999999999999,"15min":1.302},"min":{"1min":0.78400000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.3959999999999999,"15min":1.425},"min":{"1min":0.84599999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":1.4870000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3140000000000001,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7989999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5029999999999999,"15min":1.508},"min":{"1min":1.1599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":1.571}]},{"osd":5,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.337,"15min":1.327},"min":{"1min":0.79600000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.5840000000000001,"15min":1.597},"min":{"1min":1.2430000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.8380000000000001,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":1.6699999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.476,"15min":1.454},"min":{"1min":0.89400000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.8579999999999997,"5min":4.8579999999999997,"15min":4.8579999999999997},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4319999999999999,"15min":1.4550000000000001},"min":{"1min":0.81799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":2.762},"last":1.423}]}]},{"osd":6,"up_from":13,"seq":55834576003,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60444,"kb_used_data":3156,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311396,"statfs":{"total":96636764160,"available":96574869504,"internally_reserved":0,"allocated":3231744,"data_stored":1590543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.036,"15min":1.0980000000000001},"min":{"1min":0.53600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4810000000000001,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.093,"15min":1.1519999999999999},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6439999999999999,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.2270000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.127,"15min":1.202},"min":{"1min":0.57399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7010000000000001,"5min":1.976,"15min":2.0449999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.542,"5min":1.694,"15min":1.835},"last":1.012}]},{"osd":2,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5369999999999999,"5min":1.742,"15min":1.976},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.272},"min":{"1min":0.71599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.262,"15min":1.306},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7929999999999999,"5min":1.927,"15min":1.927},"last":1.423},{"interface":"front","average":{"1min":1.304,"5min":1.377,"15min":1.415},"min":{"1min":0.68700000000000006,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.119}]},{"osd":4,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2809999999999999,"15min":1.27},"min":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.661,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.464},{"interface":"front","average":{"1min":1.355,"5min":1.4199999999999999,"15min":1.3660000000000001},"min":{"1min":0.64100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1019999999999999,"5min":2.48,"15min":2.48},"last":1.2050000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3859999999999999,"15min":1.407},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.651,"5min":2.262,"15min":2.262},"last":1.163},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.518,"15min":1.492},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0030000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.5429999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5069999999999999,"15min":1.526},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.865,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.079},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0539999999999998,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.335}]}]},{"osd":5,"up_from":13,"seq":55834576002,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52828,"kb_used_data":3732,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319012,"statfs":{"total":96636764160,"available":96582668288,"internally_reserved":0,"allocated":3821568,"data_stored":2180821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0840000000000001,"15min":1.119},"min":{"1min":0.68200000000000005,"5min":0.66100000000000003,"15min":0.61299999999999999},"max":{"1min":1.476,"5min":1.776,"15min":1.776},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.996,"5min":1.133,"15min":1.151},"min":{"1min":0.72699999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.3879999999999999,"5min":1.746,"15min":1.752},"last":1.0860000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.147,"15min":1.155},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.52,"5min":1.6519999999999999,"15min":1.696},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1850000000000001,"15min":1.226},"min":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.589,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":1.1439999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.21,"15min":1.246},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.5820000000000001,"5min":1.901,"15min":1.901},"last":1.258},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.234,"15min":1.2490000000000001},"min":{"1min":0.90600000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5609999999999999,"5min":1.899,"15min":1.9219999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3029999999999999,"15min":1.3140000000000001},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.508,"5min":1.889,"15min":1.8979999999999999},"last":1.494},{"interface":"front","average":{"1min":1.294,"5min":1.3169999999999999,"15min":1.3540000000000001},"min":{"1min":1.0649999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.54,"5min":1.97,"15min":2.1859999999999999},"last":1.196}]},{"osd":4,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2669999999999999,"15min":1.256},"min":{"1min":0.82899999999999996,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.764,"5min":1.869,"15min":2.6040000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3660000000000001,"15min":1.371},"min":{"1min":0.96599999999999997,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.5449999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.3879999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3839999999999999,"15min":1.375},"min":{"1min":0.97699999999999998,"5min":0.93200000000000005,"15min":0.88800000000000001},"max":{"1min":1.643,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4239999999999999,"15min":1.419},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.9850000000000001,"5min":2.032,"15min":2.577},"last":1.427}]},{"osd":7,"last update":"Fri Dec 1 03:12:05 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.5349999999999999,"15min":1.536},"min":{"1min":1.294,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.8779999999999999,"5min":2.077,"15min":3.1709999999999998},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.5780000000000001,"15min":1.5760000000000001},"min":{"1min":1.353,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.8819999999999999,"5min":2.101,"15min":2.8740000000000001},"last":1.6639999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576001,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52264,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319576,"statfs":{"total":96636764160,"available":96583245824,"internally_reserved":0,"allocated":3244032,"data_stored":1592599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.65200000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6850000000000001,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.68600000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.641,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.0129999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.216,"15min":1.2470000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5920000000000001,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.119},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.151,"15min":1.1699999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6930000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.2949999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.3100000000000001,"15min":1.238},"min":{"1min":0.81399999999999995,"5min":0.68999999999999995,"15min":0.60499999999999998},"max":{"1min":1.4630000000000001,"5min":11.311,"15min":11.311},"last":1.054},{"interface":"front","average":{"1min":1.159,"5min":1.2629999999999999,"15min":1.3200000000000001},"min":{"1min":0.86199999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.675,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.204}]},{"osd":3,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2789999999999999,"15min":1.2829999999999999},"min":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5129999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.155},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.363,"15min":1.3320000000000001},"min":{"1min":0.90000000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2559999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.244}]},{"osd":5,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.375,"15min":1.329},"min":{"1min":1.028,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.897,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.448},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3580000000000001,"15min":1.3149999999999999},"min":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.74,"5min":3.71,"15min":3.71},"last":1.411}]},{"osd":6,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.3899999999999999,"15min":1.3740000000000001},"min":{"1min":1.0289999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.161,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.48,"5min":1.494,"15min":1.4770000000000001},"min":{"1min":1.25,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.028,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.343}]},{"osd":7,"last update":"Fri Dec 1 03:11:58 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5089999999999999,"15min":1.4850000000000001},"min":{"1min":0.82399999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.794,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.468},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5920000000000001,"15min":1.577},"min":{"1min":1.139,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3250000000000002,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.3819999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576001,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56348,"kb_used_data":3156,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315492,"statfs":{"total":96636764160,"available":96579063808,"internally_reserved":0,"allocated":3231744,"data_stored":1590487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.306,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0449999999999999,"15min":1.0609999999999999},"min":{"1min":0.39500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.573,"5min":1.95,"15min":1.95},"last":1.0409999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.083,"15min":1.1479999999999999},"min":{"1min":0.56200000000000006,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.417,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":1.635},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.94399999999999995,"15min":0.98499999999999999},"min":{"1min":0.45200000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6970000000000001,"5min":3.863,"15min":3.863},"last":1.532}]},{"osd":2,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.038,"15min":1.085},"min":{"1min":0.49099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.224,"5min":3.597,"15min":3.597},"last":1.103},{"interface":"front","average":{"1min":1.052,"5min":1.175,"15min":1.228},"min":{"1min":0.54700000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.647,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":1.5609999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.264,"15min":1.286},"min":{"1min":0.86099999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.5269999999999999,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2569999999999999,"15min":1.2929999999999999},"min":{"1min":0.79500000000000004,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9970000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.26}]},{"osd":5,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.3109999999999999,"15min":1.325},"min":{"1min":0.92200000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":1.341},{"interface":"front","average":{"1min":1.208,"5min":1.3420000000000001,"15min":1.3600000000000001},"min":{"1min":0.82499999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6699999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.306}]},{"osd":6,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.375,"15min":1.4370000000000001},"min":{"1min":0.83199999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8320000000000001,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.456,"15min":1.4990000000000001},"min":{"1min":0.92200000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.6879999999999999,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":1.218}]},{"osd":7,"last update":"Fri Dec 1 03:11:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4750000000000001,"15min":1.514},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.788,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4770000000000001,"15min":1.522},"min":{"1min":1.0549999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.913,"5min":3.004,"15min":3.004},"last":1.5940000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576000,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48168,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323672,"statfs":{"total":96636764160,"available":96587440128,"internally_reserved":0,"allocated":3244032,"data_stored":1592039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.81100000000000005,"15min":0.87},"min":{"1min":0.311,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.92100000000000004,"5min":1.411,"15min":1.4730000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.98899999999999999,"15min":1.024},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.216,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.107}]},{"osd":1,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.97299999999999998,"15min":1.0309999999999999},"min":{"1min":0.57899999999999996,"5min":0.378,"15min":0.378},"max":{"1min":1.512,"5min":1.512,"15min":1.698},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.04,"15min":1.113},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5229999999999999,"5min":1.635,"15min":1.6599999999999999},"last":1.22}]},{"osd":3,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.97499999999999998,"15min":1.022},"min":{"1min":0.497,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2889999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":1.097,"15min":1.181},"min":{"1min":0.48699999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.4970000000000001,"5min":1.615,"15min":1.79},"last":1.0589999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2749999999999999,"15min":1.3120000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.621,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.609},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.276},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4730000000000001,"5min":1.714,"15min":1.835},"last":1.2589999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.3109999999999999,"15min":1.3680000000000001},"min":{"1min":0.86599999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.639,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.347,"15min":1.393},"min":{"1min":0.79000000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.573,"5min":1.8740000000000001,"15min":2.016},"last":1.4870000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.351,"15min":1.367},"min":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.865,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.86599999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7090000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.2969999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4370000000000001,"15min":1.4630000000000001},"min":{"1min":0.94299999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5740000000000001,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3859999999999999,"15min":1.4350000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.8100000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.429}]}]},{"osd":1,"up_from":13,"seq":55834576000,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52836,"kb_used_data":3740,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319004,"statfs":{"total":96636764160,"available":96582660096,"internally_reserved":0,"allocated":3829760,"data_stored":2180913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82899999999999996,"15min":0.88800000000000001},"min":{"1min":0.38500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.5569999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.94399999999999995,"15min":1.0249999999999999},"min":{"1min":0.47299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.167,"5min":1.512,"15min":1.681},"last":1.3520000000000001}]},{"osd":2,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.86299999999999999,"15min":0.92300000000000004},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97699999999999998,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0349999999999999,"15min":1.1220000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.476,"5min":1.6080000000000001,"15min":1.7430000000000001},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.1359999999999999,"15min":1.268},"min":{"1min":0.52400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.593,"5min":1.593,"15min":1.788},"last":1.276},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.046,"15min":1.1120000000000001},"min":{"1min":0.41399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.615,"5min":1.714,"15min":1.714},"last":1.3049999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.177,"15min":1.2090000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6000000000000001,"5min":1.623,"15min":1.6859999999999999},"last":1.125},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.329,"15min":1.391},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7450000000000001,"5min":1.8620000000000001,"15min":2.2109999999999999},"last":1.4259999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2969999999999999,"15min":1.3240000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8169999999999999,"5min":1.9399999999999999,"15min":2.0379999999999998},"last":1.391},{"interface":"front","average":{"1min":1.2,"5min":1.2569999999999999,"15min":1.319},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.7470000000000001,"5min":1.927,"15min":1.927},"last":1.252}]},{"osd":6,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.373,"15min":1.429},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.885,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3759999999999999,"15min":1.4019999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.901,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.5680000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:12:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.431,"15min":1.5069999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.9970000000000001,"15min":2.1190000000000002},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.46,"15min":1.5009999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.954,"5min":2.1389999999999998,"15min":2.2069999999999999},"last":1.494}]}]},{"osd":0,"up_from":13,"seq":55834576001,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60444,"kb_used_data":3156,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311396,"statfs":{"total":96636764160,"available":96574869504,"internally_reserved":0,"allocated":3231744,"data_stored":1590543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.89100000000000001,"15min":0.89600000000000002},"min":{"1min":0.40100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94499999999999995,"15min":1.04},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.131}]},{"osd":2,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.0249999999999999,"15min":1.0660000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.48699999999999999},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.1429999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.002,"5min":1.0269999999999999,"15min":1.107},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6619999999999999,"5min":1.726,"15min":1.792},"last":1.4770000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.99099999999999999,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.306,"5min":1.536,"15min":1.8839999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.165,"15min":1.222},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.629}]},{"osd":4,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.27,"15min":1.337},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":1.258},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.246,"15min":1.2949999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.536,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.208}]},{"osd":5,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.839,"5min":1.899,"15min":2.7719999999999998},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.365,"15min":1.423},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.484,"5min":2.484,"15min":2.5880000000000001},"last":1.393}]},{"osd":6,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.327,"15min":1.371},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.835,"5min":1.835,"15min":2.2029999999999998},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.444,"15min":1.4810000000000001},"min":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":1.9239999999999999,"5min":2.0569999999999999,"15min":2.8759999999999999},"last":1.236}]},{"osd":7,"last update":"Fri Dec 1 03:11:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.4419999999999999,"15min":1.506},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.411,"5min":2.411,"15min":2.7040000000000002},"last":1.704},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.456,"15min":1.4770000000000001},"min":{"1min":0.94999999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.718,"5min":2.1190000000000002,"15min":2.431},"last":1.4950000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:12:45.341 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T03:12:45.341 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-01T03:12:45.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.602+0000 7fabaf5ce640 1 Processor -- start 2023-12-01T03:12:45.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.602+0000 7fabaf5ce640 1 -- start start 2023-12-01T03:12:45.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.604+0000 7fabaf5ce640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7faba80cee70 0x7faba8165740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.604+0000 7fabaf5ce640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba806b400 con 0x7faba81552b0 2023-12-01T03:12:45.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.604+0000 7fabaf5ce640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba806b6e0 con 0x7faba806c1c0 2023-12-01T03:12:45.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.604+0000 7fabaf5ce640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba8155690 con 0x7faba80cee70 2023-12-01T03:12:45.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.604+0000 7fabad36a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7faba81552b0 0x7faba80ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37750/0 (socket says 172.21.15.115:37750) 2023-12-01T03:12:45.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.604+0000 7fabad36a640 1 -- 172.21.15.115:0/562242666 learned_addr learned my addr 172.21.15.115:0/562242666 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:45.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.604+0000 7fabadb6b640 1 --2- 172.21.15.115:0/562242666 >> v2:172.21.15.115:3300/0 conn(0x7faba80cee70 0x7faba8165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:45.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.604+0000 7fab9ffff640 1 -- 172.21.15.115:0/562242666 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411994460 0 0) 0x7faba806b400 con 0x7faba81552b0 2023-12-01T03:12:45.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.604+0000 7fab9ffff640 1 -- 172.21.15.115:0/562242666 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab94003520 con 0x7faba81552b0 2023-12-01T03:12:45.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.605+0000 7fabadb6b640 1 -- 172.21.15.115:0/562242666 >> v1:172.21.15.142:6789/0 conn(0x7faba806c1c0 legacy=0x7faba80af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.605+0000 7fabadb6b640 1 -- 172.21.15.115:0/562242666 >> v1:172.21.15.115:6789/0 conn(0x7faba81552b0 legacy=0x7faba80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.605+0000 7fabadb6b640 1 -- 172.21.15.115:0/562242666 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba806b9c0 con 0x7faba80cee70 2023-12-01T03:12:45.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.606+0000 7fabadb6b640 1 --2- 172.21.15.115:0/562242666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba80cee70 0x7faba8165740 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 crypto rx=0x7faba0009d90 tx=0x7faba002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b034c1cfee70380f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:45.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.606+0000 7fab9ffff640 1 -- 172.21.15.115:0/562242666 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba002ccf0 con 0x7faba80cee70 2023-12-01T03:12:45.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.606+0000 7fab9ffff640 1 -- 172.21.15.115:0/562242666 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba002ce50 con 0x7faba80cee70 2023-12-01T03:12:45.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.606+0000 7fab9ffff640 1 -- 172.21.15.115:0/562242666 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0034660 con 0x7faba80cee70 2023-12-01T03:12:45.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.607+0000 7fabaf5ce640 1 -- 172.21.15.115:0/562242666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba80cee70 msgr2=0x7faba8165740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.607+0000 7fabaf5ce640 1 --2- 172.21.15.115:0/562242666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba80cee70 0x7faba8165740 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 crypto rx=0x7faba0009d90 tx=0x7faba002bf40 comp rx=0 tx=0).stop 2023-12-01T03:12:45.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.607+0000 7fabaf5ce640 1 -- 172.21.15.115:0/562242666 shutdown_connections 2023-12-01T03:12:45.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.607+0000 7fabaf5ce640 1 --2- 172.21.15.115:0/562242666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba80cee70 0x7faba8165740 unknown :-1 s=CLOSED pgs=2985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.607+0000 7fabaf5ce640 1 -- 172.21.15.115:0/562242666 >> 172.21.15.115:0/562242666 conn(0x7faba8065960 msgr2=0x7faba8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:45.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.607+0000 7fabaf5ce640 1 -- 172.21.15.115:0/562242666 shutdown_connections 2023-12-01T03:12:45.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.607+0000 7fabaf5ce640 1 -- 172.21.15.115:0/562242666 wait complete. 2023-12-01T03:12:45.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.609+0000 7fabaf5ce640 1 Processor -- start 2023-12-01T03:12:45.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.609+0000 7fabaf5ce640 1 -- start start 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.610+0000 7fabaf5ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faba806c1c0 0x7faba80cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.610+0000 7fabaf5ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faba80cee70 0x7faba8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.610+0000 7fabaf5ce640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba81552b0 0x7faba8157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.610+0000 7fabaf5ce640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba806bda0 con 0x7faba81552b0 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.610+0000 7fabaf5ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba80cc4c0 con 0x7faba806c1c0 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.610+0000 7fabaf5ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba8156da0 con 0x7faba80cee70 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.610+0000 7fabadb6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba81552b0 0x7faba8157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.610+0000 7fabadb6b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba81552b0 0x7faba8157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51894/0 (socket says 172.21.15.115:51894) 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.610+0000 7fabadb6b640 1 -- 172.21.15.115:0/560691924 learned_addr learned my addr 172.21.15.115:0/560691924 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fabad36a640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faba806c1c0 0x7faba80cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fabadb6b640 1 -- 172.21.15.115:0/560691924 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faba80cee70 msgr2=0x7faba8150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fabacb69640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faba80cee70 0x7faba8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fabadb6b640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faba80cee70 0x7faba8150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fabadb6b640 1 -- 172.21.15.115:0/560691924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faba806c1c0 msgr2=0x7faba80cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fabadb6b640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faba806c1c0 0x7faba80cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fabadb6b640 1 -- 172.21.15.115:0/560691924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba0009a70 con 0x7faba81552b0 2023-12-01T03:12:45.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fabadb6b640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba81552b0 0x7faba8157e40 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7faba0037040 tx=0x7faba0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:45.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fab9dffb640 1 -- 172.21.15.115:0/560691924 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba003d600 con 0x7faba81552b0 2023-12-01T03:12:45.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fab9dffb640 1 -- 172.21.15.115:0/560691924 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba003d760 con 0x7faba81552b0 2023-12-01T03:12:45.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fabaf5ce640 1 -- 172.21.15.115:0/560691924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba8157080 con 0x7faba81552b0 2023-12-01T03:12:45.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.611+0000 7fab9dffb640 1 -- 172.21.15.115:0/560691924 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba003db10 con 0x7faba81552b0 2023-12-01T03:12:45.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.612+0000 7fabaf5ce640 1 -- 172.21.15.115:0/560691924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba8157640 con 0x7faba81552b0 2023-12-01T03:12:45.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.613+0000 7fabaf5ce640 1 -- 172.21.15.115:0/560691924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba804e890 con 0x7faba81552b0 2023-12-01T03:12:45.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.614+0000 7fab9dffb640 1 -- 172.21.15.115:0/560691924 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7faba003b040 con 0x7faba81552b0 2023-12-01T03:12:45.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.614+0000 7fab9dffb640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fab8c06a670 0x7fab8c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:45.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.614+0000 7fab9dffb640 1 -- 172.21.15.115:0/560691924 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7faba00ab250 con 0x7faba81552b0 2023-12-01T03:12:45.616 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.615+0000 7fabad36a640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fab8c06a670 0x7fab8c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:45.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.615+0000 7fabad36a640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fab8c06a670 0x7fab8c06cb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faba40011e0 tx=0x7faba4009040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:45.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.619+0000 7fab9dffb640 1 -- 172.21.15.115:0/560691924 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faba00764b0 con 0x7faba81552b0 2023-12-01T03:12:45.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.877+0000 7fabaf5ce640 1 -- 172.21.15.115:0/560691924 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faba804e040 con 0x7faba81552b0 2023-12-01T03:12:45.887 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.885+0000 7fab9dffb640 1 -- 172.21.15.115:0/560691924 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v51) v1 ==== 190+0+0 (secure 0 0 0) 0x7faba0009450 con 0x7faba81552b0 2023-12-01T03:12:45.887 INFO:teuthology.orchestra.run.smithi115.stderr:updated caps for client.0 2023-12-01T03:12:45.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 -- 172.21.15.115:0/560691924 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fab8c06a670 msgr2=0x7fab8c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fab8c06a670 0x7fab8c06cb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faba40011e0 tx=0x7faba4009040 comp rx=0 tx=0).stop 2023-12-01T03:12:45.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 -- 172.21.15.115:0/560691924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba81552b0 msgr2=0x7faba8157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:45.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba81552b0 0x7faba8157e40 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7faba0037040 tx=0x7faba0004a40 comp rx=0 tx=0).stop 2023-12-01T03:12:45.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 -- 172.21.15.115:0/560691924 shutdown_connections 2023-12-01T03:12:45.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fab8c06a670 0x7fab8c06cb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.890 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faba80cee70 0x7faba8150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faba81552b0 0x7faba8157e40 unknown :-1 s=CLOSED pgs=2986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 --2- 172.21.15.115:0/560691924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faba806c1c0 0x7faba80cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:45.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 -- 172.21.15.115:0/560691924 >> 172.21.15.115:0/560691924 conn(0x7faba8065960 msgr2=0x7faba806f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:45.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 -- 172.21.15.115:0/560691924 shutdown_connections 2023-12-01T03:12:45.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:45.888+0000 7fabaf5ce640 1 -- 172.21.15.115:0/560691924 wait complete. 2023-12-01T03:12:45.907 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:12:46.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.118+0000 7f33482fb640 1 Processor -- start 2023-12-01T03:12:46.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.118+0000 7f33482fb640 1 -- start start 2023-12-01T03:12:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.119+0000 7f33482fb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f334015d220 0x7f33400cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:46.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.120+0000 7f33482fb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334015d600 con 0x7f3340070ac0 2023-12-01T03:12:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.120+0000 7f33482fb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334006c170 con 0x7f33400ad2b0 2023-12-01T03:12:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.120+0000 7f33482fb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334006cc10 con 0x7f334015d220 2023-12-01T03:12:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.120+0000 7f3346898640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3340070ac0 0x7f334006ba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37760/0 (socket says 172.21.15.115:37760) 2023-12-01T03:12:46.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.120+0000 7f3346898640 1 -- 172.21.15.115:0/795800694 learned_addr learned my addr 172.21.15.115:0/795800694 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.120+0000 7f3346097640 1 --2- 172.21.15.115:0/795800694 >> v2:172.21.15.142:3300/0 conn(0x7f334015d220 0x7f33400cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.120+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897501249 0 0) 0x7f3338001490 con 0x7f3340070ac0 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f332c003520 con 0x7f3340070ac0 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2234878061 0 0) 0x7f332c003520 con 0x7f3340070ac0 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 >> v2:172.21.15.142:3300/0 conn(0x7f334015d220 msgr2=0x7f33400cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 --2- 172.21.15.115:0/795800694 >> v2:172.21.15.142:3300/0 conn(0x7f334015d220 0x7f33400cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 >> v1:172.21.15.142:6789/0 conn(0x7f33400ad2b0 legacy=0x7f334006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334006c4b0 con 0x7f3340070ac0 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f3338001b80 con 0x7f3340070ac0 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 >> v1:172.21.15.115:6789/0 conn(0x7f3340070ac0 legacy=0x7f334006ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 --2- 172.21.15.115:0/795800694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c003a30 0x7f332c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:46.124 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 --2- 172.21.15.115:0/795800694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f332c0047c0 0x7f332c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 --2- 172.21.15.115:0/795800694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f332c0052a0 0x7f332c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334006cc10 con 0x7f332c0047c0 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c00c170 con 0x7f332c003a30 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c00c330 con 0x7f332c0052a0 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.121+0000 7f3345896640 1 --2- 172.21.15.115:0/795800694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f332c0047c0 0x7f332c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.122+0000 7f3345896640 1 -- 172.21.15.115:0/795800694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f332c0052a0 msgr2=0x7f332c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.122+0000 7f3345896640 1 --2- 172.21.15.115:0/795800694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f332c0052a0 0x7f332c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.122+0000 7f3345896640 1 -- 172.21.15.115:0/795800694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c003a30 msgr2=0x7f332c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.122+0000 7f3345896640 1 --2- 172.21.15.115:0/795800694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c003a30 0x7f332c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.122+0000 7f3345896640 1 -- 172.21.15.115:0/795800694 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c003520 con 0x7f332c0047c0 2023-12-01T03:12:46.125 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.122+0000 7f3346898640 1 -- 172.21.15.115:0/795800694 reap_dead start 2023-12-01T03:12:46.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.123+0000 7f3345896640 1 --2- 172.21.15.115:0/795800694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f332c0047c0 0x7f332c004c40 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f3330001750 tx=0x7f332c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5b4eae97a90abf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:46.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.123+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300052d0 con 0x7f332c0047c0 2023-12-01T03:12:46.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.123+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333002e050 con 0x7f332c0047c0 2023-12-01T03:12:46.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.123+0000 7f3345095640 1 -- 172.21.15.115:0/795800694 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330003830 con 0x7f332c0047c0 2023-12-01T03:12:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.124+0000 7f33482fb640 1 -- 172.21.15.115:0/795800694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f332c0047c0 msgr2=0x7f332c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.124+0000 7f33482fb640 1 --2- 172.21.15.115:0/795800694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f332c0047c0 0x7f332c004c40 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f3330001750 tx=0x7f332c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.124+0000 7f33482fb640 1 -- 172.21.15.115:0/795800694 shutdown_connections 2023-12-01T03:12:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.124+0000 7f33482fb640 1 --2- 172.21.15.115:0/795800694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f332c0047c0 0x7f332c004c40 unknown :-1 s=CLOSED pgs=2988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.124+0000 7f33482fb640 1 -- 172.21.15.115:0/795800694 >> 172.21.15.115:0/795800694 conn(0x7f33400ce880 msgr2=0x7f33400cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.124+0000 7f33482fb640 1 -- 172.21.15.115:0/795800694 shutdown_connections 2023-12-01T03:12:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.124+0000 7f33482fb640 1 -- 172.21.15.115:0/795800694 wait complete. 2023-12-01T03:12:46.127 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.126+0000 7f33482fb640 1 Processor -- start 2023-12-01T03:12:46.128 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.126+0000 7f33482fb640 1 -- start start 2023-12-01T03:12:46.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f33482fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c0047c0 0x7f3340148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:46.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f33482fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33400c6400 0x7f33401f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:46.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f33482fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33400cc570 0x7f33400cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:46.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f33482fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3340149270 con 0x7f33400c6400 2023-12-01T03:12:46.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f33482fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400ccf30 con 0x7f332c0047c0 2023-12-01T03:12:46.129 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f33482fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33400caf90 con 0x7f33400cc570 2023-12-01T03:12:46.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f3345896640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33400c6400 0x7f33401f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:46.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f3345896640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33400c6400 0x7f33401f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51916/0 (socket says 172.21.15.115:51916) 2023-12-01T03:12:46.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f3345896640 1 -- 172.21.15.115:0/2752409191 learned_addr learned my addr 172.21.15.115:0/2752409191 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:46.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f3346097640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c0047c0 0x7f3340148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:46.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f3345896640 1 -- 172.21.15.115:0/2752409191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33400cc570 msgr2=0x7f33400cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:46.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f3345896640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33400cc570 0x7f33400cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f3345896640 1 -- 172.21.15.115:0/2752409191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c0047c0 msgr2=0x7f3340148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:46.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f3345896640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c0047c0 0x7f3340148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.127+0000 7f3345896640 1 -- 172.21.15.115:0/2752409191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33401f6810 con 0x7f33400c6400 2023-12-01T03:12:46.130 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.128+0000 7f3345896640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33400c6400 0x7f33401f62d0 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7f3330005040 tx=0x7f3330001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.128+0000 7f3336ffd640 1 -- 172.21.15.115:0/2752409191 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300076c0 con 0x7f33400c6400 2023-12-01T03:12:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.128+0000 7f3336ffd640 1 -- 172.21.15.115:0/2752409191 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33300069e0 con 0x7f33400c6400 2023-12-01T03:12:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.128+0000 7f3336ffd640 1 -- 172.21.15.115:0/2752409191 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330006d90 con 0x7f33400c6400 2023-12-01T03:12:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.128+0000 7f33482fb640 1 -- 172.21.15.115:0/2752409191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33400c8970 con 0x7f33400c6400 2023-12-01T03:12:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.128+0000 7f33482fb640 1 -- 172.21.15.115:0/2752409191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33400c8690 con 0x7f33400c6400 2023-12-01T03:12:46.131 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.129+0000 7f33482fb640 1 -- 172.21.15.115:0/2752409191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33400c6270 con 0x7f33400c6400 2023-12-01T03:12:46.132 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.130+0000 7f3336ffd640 1 -- 172.21.15.115:0/2752409191 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f33300035f0 con 0x7f33400c6400 2023-12-01T03:12:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.131+0000 7f3336ffd640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f332406a670 0x7f332406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.131+0000 7f3336ffd640 1 -- 172.21.15.115:0/2752409191 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f33300aab30 con 0x7f33400c6400 2023-12-01T03:12:46.133 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.131+0000 7f3346097640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f332406a670 0x7f332406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:46.134 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.132+0000 7f3346097640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f332406a670 0x7f332406cb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f333c0099f0 tx=0x7f333c008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:46.137 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.135+0000 7f3336ffd640 1 -- 172.21.15.115:0/2752409191 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3330075e80 con 0x7f33400c6400 2023-12-01T03:12:46.346 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.344+0000 7f33482fb640 1 -- 172.21.15.115:0/2752409191 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f334004e890 con 0x7f33400c6400 2023-12-01T03:12:46.347 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.345+0000 7f3336ffd640 1 -- 172.21.15.115:0/2752409191 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f333004b020 con 0x7f33400c6400 2023-12-01T03:12:46.347 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:46.348 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":126,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2042114371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2042114371},{"type":"v1","addr":"172.21.15.142:6835","nonce":2042114371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/727621046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":727621046},{"type":"v1","addr":"172.21.15.115:6835","nonce":727621046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3018042052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3018042052},{"type":"v1","addr":"172.21.15.115:6837","nonce":3018042052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:12:35.317383+0000","modified":"2023-12-01T03:12:38.475270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12246":{"gid":12246,"name":"b","rank":0,"incarnation":122,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/1356734431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1356734431},{"type":"v1","addr":"172.21.15.142:6834","nonce":1356734431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-01T03:12:46.348 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 126 2023-12-01T03:12:46.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 -- 172.21.15.115:0/2752409191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f332406a670 msgr2=0x7f332406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f332406a670 0x7f332406cb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f333c0099f0 tx=0x7f333c008040 comp rx=0 tx=0).stop 2023-12-01T03:12:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 -- 172.21.15.115:0/2752409191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33400c6400 msgr2=0x7f33401f62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33400c6400 0x7f33401f62d0 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7f3330005040 tx=0x7f3330001120 comp rx=0 tx=0).stop 2023-12-01T03:12:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 -- 172.21.15.115:0/2752409191 shutdown_connections 2023-12-01T03:12:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f332406a670 0x7f332406cb30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f33400cc570 0x7f33400cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33400c6400 0x7f33401f62d0 unknown :-1 s=CLOSED pgs=2989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 --2- 172.21.15.115:0/2752409191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f332c0047c0 0x7f3340148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 -- 172.21.15.115:0/2752409191 >> 172.21.15.115:0/2752409191 conn(0x7f33400ce880 msgr2=0x7f33400ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:46.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 -- 172.21.15.115:0/2752409191 shutdown_connections 2023-12-01T03:12:46.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.348+0000 7f33482fb640 1 -- 172.21.15.115:0/2752409191 wait complete. 2023-12-01T03:12:46.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 126, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-01T03:12:35.317383+0000', 'modified': '2023-12-01T03:12:38.475270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12246}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12246': {'gid': 12246, 'name': 'b', 'rank': 0, 'incarnation': 122, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6834/1356734431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 1356734431}, {'type': 'v1', 'addr': '172.21.15.142:6834', 'nonce': 1356734431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [33], 'metadata_pool': 32, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T03:12:46.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-01T03:12:46.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-01T03:12:46.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.993+0000 7f9fbbbfa640 1 Processor -- start 2023-12-01T03:12:46.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.994+0000 7f9fbbbfa640 1 -- start start 2023-12-01T03:12:46.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.995+0000 7f9fbbbfa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9fb415cd20 0x7f9fb40d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.995+0000 7f9fbbbfa640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40ce2e0 con 0x7f9fb40b5d80 2023-12-01T03:12:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.995+0000 7f9fbbbfa640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40ce000 con 0x7f9fb40d1230 2023-12-01T03:12:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.995+0000 7f9fbbbfa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb40b2160 con 0x7f9fb415cd20 2023-12-01T03:12:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.995+0000 7f9fb9195640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f9fb40b5d80 0x7f9fb40b8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37768/0 (socket says 172.21.15.115:37768) 2023-12-01T03:12:46.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.995+0000 7f9fb9195640 1 -- 172.21.15.115:0/3283608651 learned_addr learned my addr 172.21.15.115:0/3283608651 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124499303 0 0) 0x7f9fb40ce2e0 con 0x7f9fb40b5d80 2023-12-01T03:12:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa4003520 con 0x7f9fb40b5d80 2023-12-01T03:12:46.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb9996640 1 --2- 172.21.15.115:0/3283608651 >> v2:172.21.15.142:3300/0 conn(0x7f9fb415cd20 0x7f9fb40d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3042685097 0 0) 0x7f9fa4003520 con 0x7f9fb40b5d80 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 >> v2:172.21.15.142:3300/0 conn(0x7f9fb415cd20 msgr2=0x7f9fb40d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 --2- 172.21.15.115:0/3283608651 >> v2:172.21.15.142:3300/0 conn(0x7f9fb415cd20 0x7f9fb40d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 >> v1:172.21.15.142:6789/0 conn(0x7f9fb40d1230 legacy=0x7f9fb40d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40cd3d0 con 0x7f9fb40b5d80 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f9fa0001670 con 0x7f9fb40b5d80 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 >> v1:172.21.15.115:6789/0 conn(0x7f9fb40b5d80 legacy=0x7f9fb40b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4003a30 0x7f9fa4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fa40047c0 0x7f9fa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fa4005380 0x7f9fa400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa400c1f0 con 0x7f9fa4005380 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa400c360 con 0x7f9fa4003a30 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.996+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa400c520 con 0x7f9fa40047c0 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.997+0000 7f9fba197640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fa4005380 0x7f9fa400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.997+0000 7f9fb9996640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4003a30 0x7f9fa4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.997+0000 7f9fb9195640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fa40047c0 0x7f9fa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.997+0000 7f9fba197640 1 -- 172.21.15.115:0/3283608651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fa40047c0 msgr2=0x7f9fa4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.997+0000 7f9fba197640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fa40047c0 0x7f9fa4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.997+0000 7f9fba197640 1 -- 172.21.15.115:0/3283608651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4003a30 msgr2=0x7f9fa4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.997+0000 7f9fba197640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4003a30 0x7f9fa4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.997+0000 7f9fba197640 1 -- 172.21.15.115:0/3283608651 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa4003520 con 0x7f9fa4005380 2023-12-01T03:12:47.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.997+0000 7f9fb9996640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4003a30 0x7f9fa4003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:47.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.998+0000 7f9fba197640 1 -- 172.21.15.115:0/3283608651 reap_dead start 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.998+0000 7f9fba197640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fa4005380 0x7f9fa400b980 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7f9fac001750 tx=0x7f9fa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b2e919ff037ba0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.998+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0052d0 con 0x7f9fa4005380 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.998+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac02e050 con 0x7f9fa4005380 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.998+0000 7f9fb8994640 1 -- 172.21.15.115:0/3283608651 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac003830 con 0x7f9fa4005380 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.999+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3283608651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fa4005380 msgr2=0x7f9fa400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.999+0000 7f9fbbbfa640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fa4005380 0x7f9fa400b980 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7f9fac001750 tx=0x7f9fa40057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.999+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3283608651 shutdown_connections 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.999+0000 7f9fbbbfa640 1 --2- 172.21.15.115:0/3283608651 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fa4005380 0x7f9fa400b980 unknown :-1 s=CLOSED pgs=2991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.999+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3283608651 >> 172.21.15.115:0/3283608651 conn(0x7f9fb406a0e0 msgr2=0x7f9fb406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.999+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3283608651 shutdown_connections 2023-12-01T03:12:47.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:46.999+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3283608651 wait complete. 2023-12-01T03:12:47.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.001+0000 7f9fbbbfa640 1 Processor -- start 2023-12-01T03:12:47.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.001+0000 7f9fbbbfa640 1 -- start start 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.002+0000 7f9fbbbfa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4005380 0x7f9fb414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.002+0000 7f9fbbbfa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fb41f3a50 0x7f9fb415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.002+0000 7f9fbbbfa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb41f3ed0 0x7f9fb415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.002+0000 7f9fbbbfa640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb41f3350 con 0x7f9fb41f3ed0 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.002+0000 7f9fbbbfa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb41f3630 con 0x7f9fa4005380 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.002+0000 7f9fbbbfa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb415d4f0 con 0x7f9fb41f3a50 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.002+0000 7f9fba197640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb41f3ed0 0x7f9fb415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.002+0000 7f9fba197640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb41f3ed0 0x7f9fb415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51928/0 (socket says 172.21.15.115:51928) 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fba197640 1 -- 172.21.15.115:0/3032281625 learned_addr learned my addr 172.21.15.115:0/3032281625 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fb9996640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4005380 0x7f9fb414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fb9195640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fb41f3a50 0x7f9fb415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fba197640 1 -- 172.21.15.115:0/3032281625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fb41f3a50 msgr2=0x7f9fb415b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fba197640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fb41f3a50 0x7f9fb415b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fba197640 1 -- 172.21.15.115:0/3032281625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4005380 msgr2=0x7f9fb414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fba197640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4005380 0x7f9fb414fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fba197640 1 -- 172.21.15.115:0/3032281625 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb415d7d0 con 0x7f9fb41f3ed0 2023-12-01T03:12:47.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fb9996640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4005380 0x7f9fb414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:47.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fba197640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb41f3ed0 0x7f9fb415bcd0 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7f9fac001870 tx=0x7f9fac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:47.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.003+0000 7f9fb27fc640 1 -- 172.21.15.115:0/3032281625 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac039070 con 0x7f9fb41f3ed0 2023-12-01T03:12:47.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.004+0000 7f9fb27fc640 1 -- 172.21.15.115:0/3032281625 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac007a30 con 0x7f9fb41f3ed0 2023-12-01T03:12:47.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.004+0000 7f9fb27fc640 1 -- 172.21.15.115:0/3032281625 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0054e0 con 0x7f9fb41f3ed0 2023-12-01T03:12:47.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.004+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3032281625 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb40b47c0 con 0x7f9fb41f3ed0 2023-12-01T03:12:47.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.004+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3032281625 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb41f28d0 con 0x7f9fb41f3ed0 2023-12-01T03:12:47.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.007+0000 7f9fb27fc640 1 -- 172.21.15.115:0/3032281625 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f9fac03c020 con 0x7f9fb41f3ed0 2023-12-01T03:12:47.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.007+0000 7f9fb27fc640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9f9806a670 0x7f9f9806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.008+0000 7f9fb27fc640 1 -- 172.21.15.115:0/3032281625 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f9fac0a9070 con 0x7f9fb41f3ed0 2023-12-01T03:12:47.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.008+0000 7f9fb9996640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9f9806a670 0x7f9f9806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.008+0000 7f9fbe768740 1 Processor -- start 2023-12-01T03:12:47.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.008+0000 7f9fbe768740 1 -- start start 2023-12-01T03:12:47.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.009+0000 7f9fb9996640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9f9806a670 0x7f9f9806cb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009540 tx=0x7f9fa800b040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.009+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f8907d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.009+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88d78a0 0x55a4f8904b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.009+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f89050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.009+0000 7f9fbe768740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f88b32f0 con 0x55a4f8907920 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.009+0000 7f9fbe768740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f8847f80 con 0x55a4f88d78a0 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.009+0000 7f9fbe768740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4f8845710 con 0x55a4f88db4a0 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.010+0000 7f9fb9996640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f8907d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.010+0000 7f9fb9996640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f8907d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51932/0 (socket says 172.21.15.115:51932) 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.010+0000 7f9fb9996640 1 -- 172.21.15.115:0/2391760225 learned_addr learned my addr 172.21.15.115:0/2391760225 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.010+0000 7f9fb9996640 1 -- 172.21.15.115:0/2391760225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 msgr2=0x55a4f89050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.010+0000 7f9fb9996640 1 --2- 172.21.15.115:0/2391760225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f89050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.010+0000 7f9fb9996640 1 -- 172.21.15.115:0/2391760225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88d78a0 msgr2=0x55a4f8904b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.010+0000 7f9fb9996640 1 --2- 172.21.15.115:0/2391760225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88d78a0 0x55a4f8904b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.010+0000 7f9fb9996640 1 -- 172.21.15.115:0/2391760225 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4f8808690 con 0x55a4f8907920 2023-12-01T03:12:47.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.010+0000 7f9fb9996640 1 --2- 172.21.15.115:0/2391760225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f8907d00 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8015780 tx=0x7f9fa8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25eb5f62680d2f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9f977fe640 1 -- 172.21.15.115:0/2391760225 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800e050 con 0x55a4f8907920 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9f977fe640 1 -- 172.21.15.115:0/2391760225 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa801a070 con 0x55a4f8907920 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9f977fe640 1 -- 172.21.15.115:0/2391760225 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8016a20 con 0x55a4f8907920 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9fbe768740 1 -- 172.21.15.115:0/2391760225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 msgr2=0x55a4f8907d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9fbe768740 1 --2- 172.21.15.115:0/2391760225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f8907d00 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8015780 tx=0x7f9fa8009f90 comp rx=0 tx=0).stop 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9fbe768740 1 -- 172.21.15.115:0/2391760225 shutdown_connections 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9fbe768740 1 --2- 172.21.15.115:0/2391760225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f89050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9fbe768740 1 --2- 172.21.15.115:0/2391760225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88d78a0 0x55a4f8904b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9fbe768740 1 --2- 172.21.15.115:0/2391760225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f8907d00 unknown :-1 s=CLOSED pgs=2993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9fbe768740 1 -- 172.21.15.115:0/2391760225 >> 172.21.15.115:0/2391760225 conn(0x55a4f88d9bb0 msgr2=0x55a4f8922730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9fbe768740 1 -- 172.21.15.115:0/2391760225 shutdown_connections 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.011+0000 7f9fbe768740 1 -- 172.21.15.115:0/2391760225 wait complete. 2023-12-01T03:12:47.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.012+0000 7f9fbe768740 1 -- start start 2023-12-01T03:12:47.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.012+0000 7f9fbe768740 1 Processor -- start 2023-12-01T03:12:47.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.013+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f88fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.013+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f88fec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.013+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88fcc70 0x55a4f88ff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.013+0000 7f9fbe768740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f8841000 con 0x55a4f8907920 2023-12-01T03:12:47.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.013+0000 7f9fbe768740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f87e27d0 con 0x55a4f88fcc70 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.013+0000 7f9fbe768740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4f87e2470 con 0x55a4f88db4a0 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fbe768740 10 client.0 _ll_register_callbacks cb 0x55a4f88311b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fb9195640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f88fec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fb9195640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f88fec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51934/0 (socket says 172.21.15.115:51934) 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fb9195640 1 -- 172.21.15.115:0/1816590049 learned_addr learned my addr 172.21.15.115:0/1816590049 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fb9996640 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f88fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fb9195640 1 -- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 msgr2=0x55a4f88fe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fb9195640 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f88fe6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fba197640 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88fcc70 0x55a4f88ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fb9195640 1 -- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88fcc70 msgr2=0x55a4f88ff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fb9195640 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88fcc70 0x55a4f88ff150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fb9195640 1 -- 172.21.15.115:0/1816590049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa8015460 con 0x55a4f8907920 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9fb9195640 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f88fec10 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0007fb0 tx=0x7f9fa000e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75640a1b20187d76 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.014+0000 7f9f94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:12:47.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1816590049 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa00072d0 con 0x55a4f8907920 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1816590049 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f9fa0007430 con 0x55a4f8907920 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9fbe768740 1 -- 172.21.15.115:0/1816590049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4f886b550 con 0x55a4f8907920 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1816590049 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fa00075f0 con 0x55a4f8907920 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9fbe768740 10 client.12420 fetch_fsmap learned FSMap version 126 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9fbe768740 1 -- 172.21.15.115:0/1816590049 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) v3 -- 0x55a4f883f680 con 0x55a4f8907920 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1816590049 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a4f886b550 con 0x55a4f8907920 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1816590049 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 126) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9fa00049f0 con 0x55a4f8907920 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9fbe768740 10 client.12420 fetch_fsmap finished waiting for FSMap version 126 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9fbe768740 10 client.12420 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9fbe768740 20 client.12420 populate_metadata read hostname 'smithi115' 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:6832/1356734431,v1:172.21.15.142:6834/1356734431] conn(0x55a4f893d8f0 0x55a4f893fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9fbe768740 4 client.12420 mds_command: new command op to 12246 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T03:12:47.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.015+0000 7f9fbe768740 1 -- 172.21.15.115:0/1816590049 --> [v2:172.21.15.142:6832/1356734431,v1:172.21.15.142:6834/1356734431] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4f8833b50 con 0x55a4f893d8f0 2023-12-01T03:12:47.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.016+0000 7f9fb9996640 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:6832/1356734431,v1:172.21.15.142:6834/1356734431] conn(0x55a4f893d8f0 0x55a4f893fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.017+0000 7f9fb9996640 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:6832/1356734431,v1:172.21.15.142:6834/1356734431] conn(0x55a4f893d8f0 0x55a4f893fce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add7d81c33fd8b9a server_cookie=418efe579f7d95a3 in_seq=0 out_seq=0 2023-12-01T03:12:47.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.017+0000 7f9f94ff9640 10 client.12420 ms_handle_connect on v2:172.21.15.142:6832/1356734431 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.019+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1816590049 <== mds.0 v2:172.21.15.142:6832/1356734431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a4f8833b50 con 0x55a4f893d8f0 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.019+0000 7f9f94ff9640 10 client.12420 handle_command_reply: tid=0 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.019+0000 7f9fbe768740 1 client.12420 shutdown 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 1 -- 172.21.15.115:0/1816590049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 msgr2=0x55a4f88fec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f88fec10 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0007fb0 tx=0x7f9fa000e650 comp rx=0 tx=0).stop 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 1 -- 172.21.15.115:0/1816590049 shutdown_connections 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:6832/1356734431,v1:172.21.15.142:6834/1356734431] conn(0x55a4f893d8f0 0x55a4f893fce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88fcc70 0x55a4f88ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8907920 0x55a4f88fec10 unknown :-1 s=CLOSED pgs=2994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1816590049 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f88fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9f94ff9640 0 client.12420 ms_handle_reset on v2:172.21.15.142:6832/1356734431 2023-12-01T03:12:47.022 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 1 -- 172.21.15.115:0/1816590049 >> 172.21.15.115:0/1816590049 conn(0x55a4f88d78a0 msgr2=0x55a4f88dbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:47.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 1 -- 172.21.15.115:0/1816590049 shutdown_connections 2023-12-01T03:12:47.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 1 -- 172.21.15.115:0/1816590049 wait complete. 2023-12-01T03:12:47.023 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.020+0000 7f9fbe768740 20 client.12420 trim_cache size 0 max 16384 2023-12-01T03:12:47.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.036+0000 7f9fbe768740 1 Processor -- start 2023-12-01T03:12:47.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.036+0000 7f9fbe768740 1 -- start start 2023-12-01T03:12:47.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.037+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f88fbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.037+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f88fcc70 0x55a4f88ff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.037+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f8944350 0x55a4f88ff990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.037+0000 7f9fbe768740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f85a8f20 con 0x55a4f88fcc70 2023-12-01T03:12:47.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.037+0000 7f9fbe768740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f88d8420 con 0x55a4f8944350 2023-12-01T03:12:47.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.037+0000 7f9fbe768740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4f88db1d0 con 0x55a4f88db4a0 2023-12-01T03:12:47.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.038+0000 7f9fb9195640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f88fcc70 0x55a4f88ff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.038+0000 7f9fb9195640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f88fcc70 0x55a4f88ff450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51936/0 (socket says 172.21.15.115:51936) 2023-12-01T03:12:47.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.038+0000 7f9fb9195640 1 -- 172.21.15.115:0/982314426 learned_addr learned my addr 172.21.15.115:0/982314426 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:47.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.038+0000 7f9fb9195640 1 -- 172.21.15.115:0/982314426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 msgr2=0x55a4f88fbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.038+0000 7f9fb9195640 1 --2- 172.21.15.115:0/982314426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f88fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.038+0000 7f9fba197640 1 --2- 172.21.15.115:0/982314426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f8944350 0x55a4f88ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.038+0000 7f9fb9195640 1 -- 172.21.15.115:0/982314426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f8944350 msgr2=0x55a4f88ff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.038+0000 7f9fb9195640 1 --2- 172.21.15.115:0/982314426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f8944350 0x55a4f88ff990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.038+0000 7f9fb9195640 1 -- 172.21.15.115:0/982314426 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4f87e19e0 con 0x55a4f88fcc70 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.038+0000 7f9fb9195640 1 --2- 172.21.15.115:0/982314426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f88fcc70 0x55a4f88ff450 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0001550 tx=0x7f9fa0006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72a319389939835 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9f97fff640 1 -- 172.21.15.115:0/982314426 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa000eb50 con 0x55a4f88fcc70 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9f97fff640 1 -- 172.21.15.115:0/982314426 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa000d040 con 0x55a4f88fcc70 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9f97fff640 1 -- 172.21.15.115:0/982314426 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa001edb0 con 0x55a4f88fcc70 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9fbe768740 1 -- 172.21.15.115:0/982314426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f88fcc70 msgr2=0x55a4f88ff450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9fbe768740 1 --2- 172.21.15.115:0/982314426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f88fcc70 0x55a4f88ff450 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0001550 tx=0x7f9fa0006000 comp rx=0 tx=0).stop 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9fbe768740 1 -- 172.21.15.115:0/982314426 shutdown_connections 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9fbe768740 1 --2- 172.21.15.115:0/982314426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f8944350 0x55a4f88ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9fbe768740 1 --2- 172.21.15.115:0/982314426 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f88fcc70 0x55a4f88ff450 unknown :-1 s=CLOSED pgs=2995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9fbe768740 1 --2- 172.21.15.115:0/982314426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f88fbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9fbe768740 1 -- 172.21.15.115:0/982314426 >> 172.21.15.115:0/982314426 conn(0x55a4f88d78a0 msgr2=0x55a4f8934e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9fbe768740 1 -- 172.21.15.115:0/982314426 shutdown_connections 2023-12-01T03:12:47.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.039+0000 7f9fbe768740 1 -- 172.21.15.115:0/982314426 wait complete. 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.040+0000 7f9fbe768740 1 -- start start 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.040+0000 7f9fbe768740 1 Processor -- start 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.041+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f8988830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.041+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88fcc70 0x55a4f88fac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.041+0000 7f9fbe768740 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8944350 0x55a4f893fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.041+0000 7f9fbe768740 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f88dd400 con 0x55a4f8944350 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.041+0000 7f9fbe768740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f88dd0f0 con 0x55a4f88fcc70 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.041+0000 7f9fbe768740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4f8988d70 con 0x55a4f88db4a0 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9fbe768740 10 client.0 _ll_register_callbacks cb 0x7f9fa8009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9fba197640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8944350 0x55a4f893fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9fba197640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8944350 0x55a4f893fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51940/0 (socket says 172.21.15.115:51940) 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9fba197640 1 -- 172.21.15.115:0/1666629028 learned_addr learned my addr 172.21.15.115:0/1666629028 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:47.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9fba197640 1 -- 172.21.15.115:0/1666629028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 msgr2=0x55a4f8988830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9fba197640 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f8988830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9fba197640 1 -- 172.21.15.115:0/1666629028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88fcc70 msgr2=0x55a4f88fac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9fba197640 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88fcc70 0x55a4f88fac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9fba197640 1 -- 172.21.15.115:0/1666629028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4f883f680 con 0x55a4f8944350 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9fba197640 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8944350 0x55a4f893fce0 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f9fac07fa00 tx=0x7f9fac078710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64117fffe1906d3e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9f94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.115:3300/0 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1666629028 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac077640 con 0x55a4f8944350 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.042+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1666629028 <== mon.0 v2:172.21.15.115:3300/0 2 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f9fac07ff30 con 0x55a4f8944350 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9fbe768740 1 -- 172.21.15.115:0/1666629028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fa801a070 con 0x55a4f8944350 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1666629028 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fac07faa0 con 0x55a4f8944350 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9fbe768740 10 client.12426 fetch_fsmap learned FSMap version 126 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9fbe768740 1 -- 172.21.15.115:0/1666629028 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) v3 -- 0x55a4f8806ef0 con 0x55a4f8944350 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1666629028 <== mon.0 v2:172.21.15.115:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa801a070 con 0x55a4f8944350 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1666629028 <== mon.0 v2:172.21.15.115:3300/0 5 ==== fsmap(e 126) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9fac0a7440 con 0x55a4f8944350 2023-12-01T03:12:47.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9fbe768740 10 client.12426 fetch_fsmap finished waiting for FSMap version 126 2023-12-01T03:12:47.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9fbe768740 10 client.12426 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-01T03:12:47.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9fbe768740 20 client.12426 populate_metadata read hostname 'smithi115' 2023-12-01T03:12:47.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.142:6832/1356734431,v1:172.21.15.142:6834/1356734431] conn(0x55a4f88d6a30 0x55a4f88d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9fbe768740 4 client.12426 mds_command: new command op to 12246 tid=0[{"prefix": "status"}] 2023-12-01T03:12:47.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.043+0000 7f9fbe768740 1 -- 172.21.15.115:0/1666629028 --> [v2:172.21.15.142:6832/1356734431,v1:172.21.15.142:6834/1356734431] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a4f883a080 con 0x55a4f88d6a30 2023-12-01T03:12:47.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.044+0000 7f9fb9996640 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.142:6832/1356734431,v1:172.21.15.142:6834/1356734431] conn(0x55a4f88d6a30 0x55a4f88d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.045+0000 7f9fb9996640 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.142:6832/1356734431,v1:172.21.15.142:6834/1356734431] conn(0x55a4f88d6a30 0x55a4f88d6e10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63670a8d7631239c server_cookie=b89d8d927cffae68 in_seq=0 out_seq=0 2023-12-01T03:12:47.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.045+0000 7f9f94ff9640 10 client.12426 ms_handle_connect on v2:172.21.15.142:6832/1356734431 2023-12-01T03:12:47.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.045+0000 7f9f94ff9640 1 -- 172.21.15.115:0/1666629028 <== mds.0 v2:172.21.15.142:6832/1356734431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55a4f883a080 con 0x55a4f88d6a30 2023-12-01T03:12:47.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.045+0000 7f9f94ff9640 10 client.12426 handle_command_reply: tid=0 2023-12-01T03:12:47.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.045+0000 7f9fbe768740 1 client.12426 shutdown 2023-12-01T03:12:47.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 1 -- 172.21.15.115:0/1666629028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8944350 msgr2=0x55a4f893fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8944350 0x55a4f893fce0 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f9fac07fa00 tx=0x7f9fac078710 comp rx=0 tx=0).stop 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 1 -- 172.21.15.115:0/1666629028 shutdown_connections 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.142:6832/1356734431,v1:172.21.15.142:6834/1356734431] conn(0x55a4f88d6a30 0x55a4f88d6e10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x55a4f8944350 0x55a4f893fce0 unknown :-1 s=CLOSED pgs=2996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4f88fcc70 0x55a4f88fac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 1 --2- 172.21.15.115:0/1666629028 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a4f88db4a0 0x55a4f8988830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 1 -- 172.21.15.115:0/1666629028 >> 172.21.15.115:0/1666629028 conn(0x55a4f88d78a0 msgr2=0x55a4f88d8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9f94ff9640 0 client.12426 ms_handle_reset on v2:172.21.15.142:6832/1356734431 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 1 -- 172.21.15.115:0/1666629028 shutdown_connections 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 1 -- 172.21.15.115:0/1666629028 wait complete. 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbe768740 20 client.12426 trim_cache size 0 max 16384 2023-12-01T03:12:47.049 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_fsid": "5ba3726b-7abf-4a38-892b-ec2cd5ba09fc", 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "whoami": 0, 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "id": 12246, 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "want_state": "up:active", 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "state": "up:active", 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "fs_name": "cephfs", 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "rank_uptime": 11.722744281000001, 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "mdsmap_epoch": 126, 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch": 256, 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "osdmap_epoch_barrier": 256, 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout: "uptime": 17.014037634000001 2023-12-01T03:12:47.050 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:12:47.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3032281625 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9f9806a670 msgr2=0x7f9f9806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbbbfa640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9f9806a670 0x7f9f9806cb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009540 tx=0x7f9fa800b040 comp rx=0 tx=0).stop 2023-12-01T03:12:47.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3032281625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb41f3ed0 msgr2=0x7f9fb415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.046+0000 7f9fbbbfa640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb41f3ed0 0x7f9fb415bcd0 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7f9fac001870 tx=0x7f9fac001120 comp rx=0 tx=0).stop 2023-12-01T03:12:47.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.047+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3032281625 shutdown_connections 2023-12-01T03:12:47.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.047+0000 7f9fbbbfa640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f9f9806a670 0x7f9f9806cb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.047+0000 7f9fbbbfa640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9fb41f3a50 0x7f9fb415b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.047+0000 7f9fbbbfa640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fb41f3ed0 0x7f9fb415bcd0 unknown :-1 s=CLOSED pgs=2992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.047+0000 7f9fbbbfa640 1 --2- 172.21.15.115:0/3032281625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fa4005380 0x7f9fb414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.047+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3032281625 >> 172.21.15.115:0/3032281625 conn(0x7f9fb406a0e0 msgr2=0x7f9fb40ca210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:47.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.047+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3032281625 shutdown_connections 2023-12-01T03:12:47.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.047+0000 7f9fbbbfa640 1 -- 172.21.15.115:0/3032281625 wait complete. 2023-12-01T03:12:47.066 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-01T03:12:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.272+0000 7f261b111640 1 Processor -- start 2023-12-01T03:12:47.274 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.272+0000 7f261b111640 1 -- start start 2023-12-01T03:12:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.273+0000 7f261b111640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f261415e2f0 0x7f26140ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.274+0000 7f261b111640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140add10 con 0x7f26140adf10 2023-12-01T03:12:47.275 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.274+0000 7f261b111640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140c5760 con 0x7f261406f2f0 2023-12-01T03:12:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.274+0000 7f261b111640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26140c9cf0 con 0x7f261415e2f0 2023-12-01T03:12:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.274+0000 7f26196ae640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f26140adf10 0x7f26140c95e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37782/0 (socket says 172.21.15.115:37782) 2023-12-01T03:12:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.274+0000 7f26196ae640 1 -- 172.21.15.115:0/2650485544 learned_addr learned my addr 172.21.15.115:0/2650485544 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.274+0000 7f2618ead640 1 --2- 172.21.15.115:0/2650485544 >> v2:172.21.15.115:3300/0 conn(0x7f261415e2f0 0x7f26140ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.274+0000 7f2618ead640 1 -- 172.21.15.115:0/2650485544 >> v1:172.21.15.142:6789/0 conn(0x7f261406f2f0 legacy=0x7f26140c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.274+0000 7f2618ead640 1 -- 172.21.15.115:0/2650485544 >> v1:172.21.15.115:6789/0 conn(0x7f26140adf10 legacy=0x7f26140c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.274+0000 7f2618ead640 1 -- 172.21.15.115:0/2650485544 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261415e6d0 con 0x7f261415e2f0 2023-12-01T03:12:47.276 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.274+0000 7f26137fe640 1 -- 172.21.15.115:0/2650485544 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780528088 0 0) 0x7f26140add10 con 0x7f26140adf10 2023-12-01T03:12:47.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.275+0000 7f2618ead640 1 --2- 172.21.15.115:0/2650485544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261415e2f0 0x7f26140ca660 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 crypto rx=0x7f26080097a0 tx=0x7f260802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6577a6ed8f3add9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:47.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.276+0000 7f26137fe640 1 -- 172.21.15.115:0/2650485544 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260802f050 con 0x7f261415e2f0 2023-12-01T03:12:47.277 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.276+0000 7f26137fe640 1 -- 172.21.15.115:0/2650485544 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608031700 con 0x7f261415e2f0 2023-12-01T03:12:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.276+0000 7f26137fe640 1 -- 172.21.15.115:0/2650485544 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608031a90 con 0x7f261415e2f0 2023-12-01T03:12:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.276+0000 7f261b111640 1 -- 172.21.15.115:0/2650485544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261415e2f0 msgr2=0x7f26140ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.276+0000 7f261b111640 1 --2- 172.21.15.115:0/2650485544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261415e2f0 0x7f26140ca660 secure :-1 s=READY pgs=2998 cs=0 l=1 rev1=1 crypto rx=0x7f26080097a0 tx=0x7f260802b7a0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.276+0000 7f261b111640 1 -- 172.21.15.115:0/2650485544 shutdown_connections 2023-12-01T03:12:47.278 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.276+0000 7f261b111640 1 --2- 172.21.15.115:0/2650485544 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261415e2f0 0x7f26140ca660 unknown :-1 s=CLOSED pgs=2998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.277+0000 7f261b111640 1 -- 172.21.15.115:0/2650485544 >> 172.21.15.115:0/2650485544 conn(0x7f26140ce880 msgr2=0x7f26140cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:47.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.277+0000 7f261b111640 1 -- 172.21.15.115:0/2650485544 shutdown_connections 2023-12-01T03:12:47.279 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.277+0000 7f261b111640 1 -- 172.21.15.115:0/2650485544 wait complete. 2023-12-01T03:12:47.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.278+0000 7f261b111640 1 Processor -- start 2023-12-01T03:12:47.280 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.278+0000 7f261b111640 1 -- start start 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.279+0000 7f261b111640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261406f2f0 0x7f26140c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.279+0000 7f261b111640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26140adf10 0x7f26140c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.279+0000 7f261b111640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f261415e2f0 0x7f26140ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.279+0000 7f261b111640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140c4ab0 con 0x7f261406f2f0 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.279+0000 7f261b111640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140c3070 con 0x7f26140adf10 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.279+0000 7f261b111640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26140c44a0 con 0x7f261415e2f0 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.279+0000 7f2618ead640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261406f2f0 0x7f26140c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.280+0000 7f2618ead640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261406f2f0 0x7f26140c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:51956/0 (socket says 172.21.15.115:51956) 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.280+0000 7f2618ead640 1 -- 172.21.15.115:0/4039775523 learned_addr learned my addr 172.21.15.115:0/4039775523 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.280+0000 7f2618ead640 1 -- 172.21.15.115:0/4039775523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f261415e2f0 msgr2=0x7f26140ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.280+0000 7f2618ead640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f261415e2f0 0x7f26140ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.282 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.280+0000 7f2618ead640 1 -- 172.21.15.115:0/4039775523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26140adf10 msgr2=0x7f26140c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.280+0000 7f2618ead640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26140adf10 0x7f26140c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.280+0000 7f2618ead640 1 -- 172.21.15.115:0/4039775523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608009400 con 0x7f261406f2f0 2023-12-01T03:12:47.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.280+0000 7f2618ead640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261406f2f0 0x7f26140c23b0 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7f26080097a0 tx=0x7f26080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:47.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.280+0000 7f26117fa640 1 -- 172.21.15.115:0/4039775523 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608033c30 con 0x7f261406f2f0 2023-12-01T03:12:47.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.280+0000 7f26117fa640 1 -- 172.21.15.115:0/4039775523 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2608033d90 con 0x7f261406f2f0 2023-12-01T03:12:47.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.281+0000 7f261b111640 1 -- 172.21.15.115:0/4039775523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26140cdf40 con 0x7f261406f2f0 2023-12-01T03:12:47.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.281+0000 7f26117fa640 1 -- 172.21.15.115:0/4039775523 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260803d6b0 con 0x7f261406f2f0 2023-12-01T03:12:47.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.281+0000 7f261b111640 1 -- 172.21.15.115:0/4039775523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26140c2d90 con 0x7f261406f2f0 2023-12-01T03:12:47.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.282+0000 7f261b111640 1 -- 172.21.15.115:0/4039775523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261406fda0 con 0x7f261406f2f0 2023-12-01T03:12:47.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.283+0000 7f26117fa640 1 -- 172.21.15.115:0/4039775523 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f260803a040 con 0x7f261406f2f0 2023-12-01T03:12:47.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.284+0000 7f26117fa640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f25f806a670 0x7f25f806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:47.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.284+0000 7f26117fa640 1 -- 172.21.15.115:0/4039775523 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f26080aad30 con 0x7f261406f2f0 2023-12-01T03:12:47.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.284+0000 7f2613fff640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f25f806a670 0x7f25f806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:47.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.285+0000 7f2613fff640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f25f806a670 0x7f25f806cb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2600006fd0 tx=0x7f2600006bc0 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:47.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.288+0000 7f26117fa640 1 -- 172.21.15.115:0/4039775523 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2608075df0 con 0x7f261406f2f0 2023-12-01T03:12:47.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.520+0000 7f261b111640 1 -- 172.21.15.115:0/4039775523 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f261404e890 con 0x7f261406f2f0 2023-12-01T03:12:47.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.523+0000 7f26117fa640 1 -- 172.21.15.115:0/4039775523 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v126) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2608041070 con 0x7f261406f2f0 2023-12-01T03:12:47.525 INFO:teuthology.orchestra.run.smithi115.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T03:12:47.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.525+0000 7f261b111640 1 -- 172.21.15.115:0/4039775523 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f25f806a670 msgr2=0x7f25f806cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.525+0000 7f261b111640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f25f806a670 0x7f25f806cb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2600006fd0 tx=0x7f2600006bc0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.525+0000 7f261b111640 1 -- 172.21.15.115:0/4039775523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261406f2f0 msgr2=0x7f26140c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:47.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.525+0000 7f261b111640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261406f2f0 0x7f26140c23b0 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7f26080097a0 tx=0x7f26080046c0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.525+0000 7f261b111640 1 -- 172.21.15.115:0/4039775523 shutdown_connections 2023-12-01T03:12:47.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.525+0000 7f261b111640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f25f806a670 0x7f25f806cb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.525+0000 7f261b111640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f261415e2f0 0x7f26140ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.525+0000 7f261b111640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26140adf10 0x7f26140c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.525+0000 7f261b111640 1 --2- 172.21.15.115:0/4039775523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f261406f2f0 0x7f26140c23b0 unknown :-1 s=CLOSED pgs=2999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:47.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.525+0000 7f261b111640 1 -- 172.21.15.115:0/4039775523 >> 172.21.15.115:0/4039775523 conn(0x7f26140ce880 msgr2=0x7f26140cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:47.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.526+0000 7f261b111640 1 -- 172.21.15.115:0/4039775523 shutdown_connections 2023-12-01T03:12:47.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:47.526+0000 7f261b111640 1 -- 172.21.15.115:0/4039775523 wait complete. 2023-12-01T03:12:47.544 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-01T03:12:47.544 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T03:12:47.544 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2023-12-01T03:12:47.544 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T03:12:47.544 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T03:12:47.545 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-01T03:12:47.545 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-01T03:12:47.545 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T03:12:47.545 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:12:47.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:47.545 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2023-12-01T03:12:47.595 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T03:12:47.595 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T03:12:47.595 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T03:12:47.595 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:12:47.595 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2023-12-01T03:12:47.595 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:12:47.595 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:12:47.595 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2023-12-01T03:12:47.595 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f0 2023-12-01T03:12:47.595 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: altname enp7s0f1 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 36589sec preferred_lft 36589sec 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-01T03:12:47.596 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:12:47.597 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::8426:88ff:fe67:2ad0/64 scope link 2023-12-01T03:12:47.597 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:12:47.597 INFO:teuthology.orchestra.run.smithi115.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T03:12:47.597 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether a2:b0:85:66:e6:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T03:12:47.597 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::a0b0:85ff:fe66:e623/64 scope link 2023-12-01T03:12:47.597 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:12:47.597 INFO:teuthology.orchestra.run.smithi115.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-01T03:12:47.597 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 1a:a6:cd:62:a5:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T03:12:47.597 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::18a6:cdff:fe62:a5af/64 scope link 2023-12-01T03:12:47.597 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2023-12-01T03:12:47.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:47.598 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T03:12:47.678 INFO:teuthology.orchestra.run.smithi115.stdout:1 2023-12-01T03:12:47.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:47.682 DEBUG:teuthology.orchestra.run.smithi115:> route 2023-12-01T03:12:47.737 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2023-12-01T03:12:47.758 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T03:12:47.758 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T03:12:47.758 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T03:12:47.758 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T03:12:47.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:47.758 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2023-12-01T03:12:47.758 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2023-12-01T03:12:47.758 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T03:12:47.759 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T03:12:47.759 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T03:12:47.759 DEBUG:teuthology.orchestra.run.smithi115:> ') 2023-12-01T03:12:47.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:47.947 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T03:12:47.995 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T03:12:47.995 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T03:12:47.996 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:12:47.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:47.996 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2023-12-01T03:12:48.090 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T03:12:48.090 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T03:12:48.142 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T03:12:48.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T03:12:48.142 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T03:12:48.192 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-12-01T03:12:48.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:48.192 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:12:48.329 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:12:48.329 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:12:48.329 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:12:48.350 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:12:48.350 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:12:48.350 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:12:48.350 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T03:12:48.350 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T03:12:48.375 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-12-01T03:12:48.375 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-12-01T03:12:48.376 INFO:teuthology.orchestra.run.smithi115.stdout:invalid new device string format 2023-12-01T03:12:48.376 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:3300,172.21.15.142:3300,172.21.15.142:3301" 2023-12-01T03:12:48.376 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: switching to using v1 address with old syntax 2023-12-01T03:12:48.376 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: resolved to: "172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790" 2023-12-01T03:12:48.376 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.115:6789,172.21.15.142:6789,172.21.15.142:6790:/ 2023-12-01T03:12:48.376 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=5ba3726b-7abf-4a38-892b-ec2cd5ba09fc" will pass to kernel 2023-12-01T03:12:48.377 INFO:teuthology.orchestra.run.smithi115.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-12-01T03:12:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: You just mounted a file system that supports labels which does not 2023-12-01T03:12:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-12-01T03:12:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: applications will generate AVC messages and not be allowed access to 2023-12-01T03:12:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-12-01T03:12:48.381 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T03:12:48.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:48.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T03:12:48.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:48.467 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T03:12:48.551 INFO:teuthology.orchestra.run.smithi115.stdout:client12438 2023-12-01T03:12:48.551 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:12:48.551 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:48.606 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-01T03:12:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.819+0000 7fe96fa2c640 1 Processor -- start 2023-12-01T03:12:48.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.820+0000 7fe96fa2c640 1 -- start start 2023-12-01T03:12:48.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.821+0000 7fe96fa2c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe968159dd0 0x7fe9680c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:48.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.821+0000 7fe96fa2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe96806b6f0 0x7fe9680c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:48.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.821+0000 7fe96fa2c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96806f8b0 con 0x7fe9680ade80 2023-12-01T03:12:48.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.821+0000 7fe96fa2c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9680adcb0 con 0x7fe968159dd0 2023-12-01T03:12:48.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.821+0000 7fe96fa2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680ad9d0 con 0x7fe96806b6f0 2023-12-01T03:12:48.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.821+0000 7fe96d7c8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe968159dd0 0x7fe9680c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:48.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.821+0000 7fe96d7c8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe968159dd0 0x7fe9680c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34038/0 (socket says 172.21.15.115:34038) 2023-12-01T03:12:48.823 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.821+0000 7fe96d7c8640 1 -- 172.21.15.115:0/1081971804 learned_addr learned my addr 172.21.15.115:0/1081971804 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:48.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.822+0000 7fe96d7c8640 1 -- 172.21.15.115:0/1081971804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe96806b6f0 msgr2=0x7fe9680c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:48.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.822+0000 7fe96cfc7640 1 --2- 172.21.15.115:0/1081971804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe96806b6f0 0x7fe9680c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:48.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.822+0000 7fe96d7c8640 1 --2- 172.21.15.115:0/1081971804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe96806b6f0 0x7fe9680c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:48.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.822+0000 7fe96d7c8640 1 -- 172.21.15.115:0/1081971804 >> v1:172.21.15.142:6789/0 conn(0x7fe9680ade80 legacy=0x7fe9680cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:48.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.822+0000 7fe96d7c8640 1 -- 172.21.15.115:0/1081971804 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9680c9000 con 0x7fe968159dd0 2023-12-01T03:12:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.823+0000 7fe96d7c8640 1 --2- 172.21.15.115:0/1081971804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe968159dd0 0x7fe9680c97e0 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7fe964009870 tx=0x7fe96402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7693713b3991ef93 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.823+0000 7fe95ffff640 1 -- 172.21.15.115:0/1081971804 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964031740 con 0x7fe968159dd0 2023-12-01T03:12:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.823+0000 7fe95ffff640 1 -- 172.21.15.115:0/1081971804 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9640318a0 con 0x7fe968159dd0 2023-12-01T03:12:48.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.823+0000 7fe95ffff640 1 -- 172.21.15.115:0/1081971804 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964031c30 con 0x7fe968159dd0 2023-12-01T03:12:48.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.823+0000 7fe96fa2c640 1 -- 172.21.15.115:0/1081971804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe968159dd0 msgr2=0x7fe9680c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:48.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.824+0000 7fe96fa2c640 1 --2- 172.21.15.115:0/1081971804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe968159dd0 0x7fe9680c97e0 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7fe964009870 tx=0x7fe96402b7e0 comp rx=0 tx=0).stop 2023-12-01T03:12:48.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.824+0000 7fe96fa2c640 1 -- 172.21.15.115:0/1081971804 shutdown_connections 2023-12-01T03:12:48.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.824+0000 7fe96fa2c640 1 --2- 172.21.15.115:0/1081971804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe96806b6f0 0x7fe9680c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:48.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.824+0000 7fe96fa2c640 1 --2- 172.21.15.115:0/1081971804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe968159dd0 0x7fe9680c97e0 unknown :-1 s=CLOSED pgs=3001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:48.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.824+0000 7fe96fa2c640 1 -- 172.21.15.115:0/1081971804 >> 172.21.15.115:0/1081971804 conn(0x7fe9680ce880 msgr2=0x7fe9680cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:48.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.824+0000 7fe96fa2c640 1 -- 172.21.15.115:0/1081971804 shutdown_connections 2023-12-01T03:12:48.826 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.824+0000 7fe96fa2c640 1 -- 172.21.15.115:0/1081971804 wait complete. 2023-12-01T03:12:48.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.825+0000 7fe96fa2c640 1 Processor -- start 2023-12-01T03:12:48.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.826+0000 7fe96fa2c640 1 -- start start 2023-12-01T03:12:48.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96fa2c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96806b6f0 0x7fe968151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:48.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96fa2c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9680ade80 0x7fe9681516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:48.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96fa2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe968159dd0 0x7fe9681e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:48.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96fa2c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe968151c10 con 0x7fe9680ade80 2023-12-01T03:12:48.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96fa2c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681e48c0 con 0x7fe96806b6f0 2023-12-01T03:12:48.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96fa2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9681e4b80 con 0x7fe968159dd0 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96cfc7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9680ade80 0x7fe9681516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96cfc7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9680ade80 0x7fe9681516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34048/0 (socket says 172.21.15.115:34048) 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96cfc7640 1 -- 172.21.15.115:0/2219917008 learned_addr learned my addr 172.21.15.115:0/2219917008 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96dfc9640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe968159dd0 0x7fe9681e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96d7c8640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96806b6f0 0x7fe968151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96cfc7640 1 -- 172.21.15.115:0/2219917008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe968159dd0 msgr2=0x7fe9681e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96cfc7640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe968159dd0 0x7fe9681e5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96cfc7640 1 -- 172.21.15.115:0/2219917008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96806b6f0 msgr2=0x7fe968151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96cfc7640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96806b6f0 0x7fe968151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96cfc7640 1 -- 172.21.15.115:0/2219917008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640094d0 con 0x7fe9680ade80 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96dfc9640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe968159dd0 0x7fe9681e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.827+0000 7fe96d7c8640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96806b6f0 0x7fe968151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:48.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.828+0000 7fe96cfc7640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9680ade80 0x7fe9681516d0 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c7f0 tx=0x7fe95800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:48.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.828+0000 7fe95dffb640 1 -- 172.21.15.115:0/2219917008 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958004220 con 0x7fe9680ade80 2023-12-01T03:12:48.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.828+0000 7fe95dffb640 1 -- 172.21.15.115:0/2219917008 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9580043a0 con 0x7fe9680ade80 2023-12-01T03:12:48.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.828+0000 7fe95dffb640 1 -- 172.21.15.115:0/2219917008 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958007860 con 0x7fe9680ade80 2023-12-01T03:12:48.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.828+0000 7fe96fa2c640 1 -- 172.21.15.115:0/2219917008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9681e4e60 con 0x7fe9680ade80 2023-12-01T03:12:48.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.828+0000 7fe96fa2c640 1 -- 172.21.15.115:0/2219917008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9681e5420 con 0x7fe9680ade80 2023-12-01T03:12:48.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.829+0000 7fe96fa2c640 1 -- 172.21.15.115:0/2219917008 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96804e890 con 0x7fe9680ade80 2023-12-01T03:12:48.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.830+0000 7fe95dffb640 1 -- 172.21.15.115:0/2219917008 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7fe958007bd0 con 0x7fe9680ade80 2023-12-01T03:12:48.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.831+0000 7fe95dffb640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe94c06a670 0x7fe94c06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:48.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.831+0000 7fe95dffb640 1 -- 172.21.15.115:0/2219917008 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fe958088e40 con 0x7fe9680ade80 2023-12-01T03:12:48.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.831+0000 7fe96d7c8640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe94c06a670 0x7fe94c06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:48.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.832+0000 7fe96d7c8640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe94c06a670 0x7fe94c06cb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe964035000 tx=0x7fe964008030 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:48.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:48.836+0000 7fe95dffb640 1 -- 172.21.15.115:0/2219917008 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe958053f90 con 0x7fe9680ade80 2023-12-01T03:12:49.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.049+0000 7fe96fa2c640 1 -- 172.21.15.115:0/2219917008 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe96806d0d0 con 0x7fe94c06a670 2023-12-01T03:12:49.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.053+0000 7fe95dffb640 1 -- 172.21.15.115:0/2219917008 <== mgr.11877 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe964031b90 con 0x7fe94c06a670 2023-12-01T03:12:49.054 INFO:teuthology.orchestra.run.smithi115.stdout:[ 2023-12-01T03:12:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2023-12-01T03:12:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "cephfs" 2023-12-01T03:12:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2023-12-01T03:12:49.055 INFO:teuthology.orchestra.run.smithi115.stdout:] 2023-12-01T03:12:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.055+0000 7fe96fa2c640 1 -- 172.21.15.115:0/2219917008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe94c06a670 msgr2=0x7fe94c06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.055+0000 7fe96fa2c640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe94c06a670 0x7fe94c06cb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe964035000 tx=0x7fe964008030 comp rx=0 tx=0).stop 2023-12-01T03:12:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.055+0000 7fe96fa2c640 1 -- 172.21.15.115:0/2219917008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9680ade80 msgr2=0x7fe9681516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.055+0000 7fe96fa2c640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9680ade80 0x7fe9681516d0 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c7f0 tx=0x7fe95800ccb0 comp rx=0 tx=0).stop 2023-12-01T03:12:49.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.055+0000 7fe96fa2c640 1 -- 172.21.15.115:0/2219917008 shutdown_connections 2023-12-01T03:12:49.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.055+0000 7fe96fa2c640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe94c06a670 0x7fe94c06cb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:49.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.055+0000 7fe96fa2c640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe968159dd0 0x7fe9681e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:49.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.055+0000 7fe96fa2c640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9680ade80 0x7fe9681516d0 unknown :-1 s=CLOSED pgs=3002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:49.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.056+0000 7fe96fa2c640 1 --2- 172.21.15.115:0/2219917008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe96806b6f0 0x7fe968151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:49.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.056+0000 7fe96fa2c640 1 -- 172.21.15.115:0/2219917008 >> 172.21.15.115:0/2219917008 conn(0x7fe9680ce880 msgr2=0x7fe9680afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:49.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.056+0000 7fe96fa2c640 1 -- 172.21.15.115:0/2219917008 shutdown_connections 2023-12-01T03:12:49.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.056+0000 7fe96fa2c640 1 -- 172.21.15.115:0/2219917008 wait complete. 2023-12-01T03:12:49.075 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-01T03:12:49.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.282+0000 7f07df682640 1 Processor -- start 2023-12-01T03:12:49.284 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.282+0000 7f07df682640 1 -- start start 2023-12-01T03:12:49.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.283+0000 7f07df682640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f07d8142870 0x7f07d80ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:49.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.283+0000 7f07df682640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80cd010 0x7f07d80b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:49.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.283+0000 7f07df682640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d80cd470 con 0x7f07d80b1c50 2023-12-01T03:12:49.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.283+0000 7f07df682640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d80b3ae0 con 0x7f07d8142870 2023-12-01T03:12:49.285 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.283+0000 7f07df682640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8142c50 con 0x7f07d80cd010 2023-12-01T03:12:49.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.284+0000 7f07dcc1d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80cd010 0x7f07d80b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:49.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.284+0000 7f07dd41e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f07d8142870 0x7f07d80ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:49.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.284+0000 7f07dcc1d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80cd010 0x7f07d80b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.115:50372/0 (socket says 172.21.15.115:50372) 2023-12-01T03:12:49.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.284+0000 7f07dcc1d640 1 -- 172.21.15.115:0/403004882 learned_addr learned my addr 172.21.15.115:0/403004882 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:49.286 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.285+0000 7f07dcc1d640 1 -- 172.21.15.115:0/403004882 >> v2:172.21.15.142:3300/0 conn(0x7f07d8142870 msgr2=0x7f07d80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:49.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.285+0000 7f07dcc1d640 1 --2- 172.21.15.115:0/403004882 >> v2:172.21.15.142:3300/0 conn(0x7f07d8142870 0x7f07d80ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:49.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.285+0000 7f07dcc1d640 1 -- 172.21.15.115:0/403004882 >> v1:172.21.15.142:6789/0 conn(0x7f07d80b1c50 legacy=0x7f07d815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:49.287 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.285+0000 7f07dcc1d640 1 -- 172.21.15.115:0/403004882 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d80b20e0 con 0x7f07d80cd010 2023-12-01T03:12:49.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.286+0000 7f07dcc1d640 1 --2- 172.21.15.115:0/403004882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80cd010 0x7f07d80b1710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009870 tx=0x7f07c802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8531da250555c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:49.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.287+0000 7f07cffff640 1 -- 172.21.15.115:0/403004882 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8030840 con 0x7f07d80cd010 2023-12-01T03:12:49.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.287+0000 7f07cffff640 1 -- 172.21.15.115:0/403004882 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c80309a0 con 0x7f07d80cd010 2023-12-01T03:12:49.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.287+0000 7f07cffff640 1 -- 172.21.15.115:0/403004882 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8030d30 con 0x7f07d80cd010 2023-12-01T03:12:49.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.288+0000 7f07df682640 1 -- 172.21.15.115:0/403004882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80cd010 msgr2=0x7f07d80b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:49.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.288+0000 7f07df682640 1 --2- 172.21.15.115:0/403004882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80cd010 0x7f07d80b1710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009870 tx=0x7f07c802b980 comp rx=0 tx=0).stop 2023-12-01T03:12:49.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.288+0000 7f07df682640 1 -- 172.21.15.115:0/403004882 shutdown_connections 2023-12-01T03:12:49.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.288+0000 7f07df682640 1 --2- 172.21.15.115:0/403004882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80cd010 0x7f07d80b1710 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:49.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.288+0000 7f07df682640 1 --2- 172.21.15.115:0/403004882 >> v2:172.21.15.142:3300/0 conn(0x7f07d8142870 0x7f07d80ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:49.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.288+0000 7f07df682640 1 -- 172.21.15.115:0/403004882 >> 172.21.15.115:0/403004882 conn(0x7f07d8065960 msgr2=0x7f07d8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:49.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.288+0000 7f07df682640 1 -- 172.21.15.115:0/403004882 shutdown_connections 2023-12-01T03:12:49.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.288+0000 7f07df682640 1 -- 172.21.15.115:0/403004882 wait complete. 2023-12-01T03:12:49.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.290+0000 7f07df682640 1 Processor -- start 2023-12-01T03:12:49.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.290+0000 7f07df682640 1 -- start start 2023-12-01T03:12:49.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.291+0000 7f07df682640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80b1c50 0x7f07d813d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:49.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.291+0000 7f07df682640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d80cd010 0x7f07d81e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:49.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.291+0000 7f07df682640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07d8142870 0x7f07d80c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.291+0000 7f07df682640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d81e8af0 con 0x7f07d8142870 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.291+0000 7f07df682640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d81e9450 con 0x7f07d80cd010 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.291+0000 7f07df682640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8158af0 con 0x7f07d80b1c50 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.291+0000 7f07ddc1f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07d8142870 0x7f07d80c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.291+0000 7f07ddc1f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07d8142870 0x7f07d80c4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34064/0 (socket says 172.21.15.115:34064) 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.291+0000 7f07ddc1f640 1 -- 172.21.15.115:0/1829338475 learned_addr learned my addr 172.21.15.115:0/1829338475 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07dcc1d640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d80cd010 0x7f07d81e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07dd41e640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80b1c50 0x7f07d813d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07ddc1f640 1 -- 172.21.15.115:0/1829338475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80b1c50 msgr2=0x7f07d813d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07ddc1f640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80b1c50 0x7f07d813d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07ddc1f640 1 -- 172.21.15.115:0/1829338475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d80cd010 msgr2=0x7f07d81e8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07ddc1f640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d80cd010 0x7f07d81e8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07ddc1f640 1 -- 172.21.15.115:0/1829338475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c80094d0 con 0x7f07d8142870 2023-12-01T03:12:49.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07dcc1d640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d80cd010 0x7f07d81e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:49.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07ddc1f640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07d8142870 0x7f07d80c4600 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7f07d000c7a0 tx=0x7f07d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:49.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07cdffb640 1 -- 172.21.15.115:0/1829338475 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0003b70 con 0x7f07d8142870 2023-12-01T03:12:49.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07cdffb640 1 -- 172.21.15.115:0/1829338475 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d0003cf0 con 0x7f07d8142870 2023-12-01T03:12:49.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07cdffb640 1 -- 172.21.15.115:0/1829338475 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0003ea0 con 0x7f07d8142870 2023-12-01T03:12:49.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.292+0000 7f07df682640 1 -- 172.21.15.115:0/1829338475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d81e7e70 con 0x7f07d8142870 2023-12-01T03:12:49.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.293+0000 7f07df682640 1 -- 172.21.15.115:0/1829338475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d81e8430 con 0x7f07d8142870 2023-12-01T03:12:49.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.294+0000 7f07df682640 1 -- 172.21.15.115:0/1829338475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d81e8c30 con 0x7f07d8142870 2023-12-01T03:12:49.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.295+0000 7f07cdffb640 1 -- 172.21.15.115:0/1829338475 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 142) v1 ==== 88811+0+0 (secure 0 0 0) 0x7f07d000aa90 con 0x7f07d8142870 2023-12-01T03:12:49.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.296+0000 7f07cdffb640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f07bc06a6a0 0x7f07bc06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:49.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.296+0000 7f07cdffb640 1 -- 172.21.15.115:0/1829338475 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f07d0088b90 con 0x7f07d8142870 2023-12-01T03:12:49.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.296+0000 7f07dd41e640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f07bc06a6a0 0x7f07bc06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:49.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.297+0000 7f07dd41e640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f07bc06a6a0 0x7f07bc06cb60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07d80c4040 tx=0x7f07d4008040 comp rx=0 tx=0).ready entity=mgr.11877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:49.302 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.301+0000 7f07cdffb640 1 -- 172.21.15.115:0/1829338475 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f07d81e8c30 con 0x7f07d8142870 2023-12-01T03:12:49.545 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:49.543+0000 7f07df682640 1 -- 172.21.15.115:0/1829338475 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f07d81e8c30 con 0x7f07d8142870 2023-12-01T03:12:50.556 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.554+0000 7f07cdffb640 1 -- 172.21.15.115:0/1829338475 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 143) v1 ==== 88828+0+0 (secure 0 0 0) 0x7f07d000ad40 con 0x7f07d8142870 2023-12-01T03:12:50.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.556+0000 7f07cdffb640 1 -- 172.21.15.115:0/1829338475 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v143) v1 ==== 92+0+0 (secure 0 0 0) 0x7f07d0072170 con 0x7f07d8142870 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.558+0000 7f07df682640 1 -- 172.21.15.115:0/1829338475 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f07bc06a6a0 msgr2=0x7f07bc06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.558+0000 7f07df682640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f07bc06a6a0 0x7f07bc06cb60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07d80c4040 tx=0x7f07d4008040 comp rx=0 tx=0).stop 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.558+0000 7f07df682640 1 -- 172.21.15.115:0/1829338475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07d8142870 msgr2=0x7f07d80c4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.558+0000 7f07df682640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07d8142870 0x7f07d80c4600 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7f07d000c7a0 tx=0x7f07d000cc60 comp rx=0 tx=0).stop 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.558+0000 7f07df682640 1 -- 172.21.15.115:0/1829338475 shutdown_connections 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.558+0000 7f07df682640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f07bc06a6a0 0x7f07bc06cb60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.558+0000 7f07df682640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07d8142870 0x7f07d80c4600 unknown :-1 s=CLOSED pgs=3003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.558+0000 7f07df682640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07d80cd010 0x7f07d81e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.558+0000 7f07df682640 1 --2- 172.21.15.115:0/1829338475 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07d80b1c50 0x7f07d813d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.559+0000 7f07df682640 1 -- 172.21.15.115:0/1829338475 >> 172.21.15.115:0/1829338475 conn(0x7f07d8065960 msgr2=0x7f07d8141a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.559+0000 7f07df682640 1 -- 172.21.15.115:0/1829338475 shutdown_connections 2023-12-01T03:12:50.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.559+0000 7f07df682640 1 -- 172.21.15.115:0/1829338475 wait complete. 2023-12-01T03:12:50.581 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-01T03:12:50.770 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.769+0000 7fd529b88640 1 Processor -- start 2023-12-01T03:12:50.771 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.769+0000 7fd529b88640 1 -- start start 2023-12-01T03:12:50.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.770+0000 7fd529b88640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5240cfd60 0x7fd5240cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:50.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.770+0000 7fd529b88640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd5240b5fa0 0x7fd524161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:50.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.770+0000 7fd529b88640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5240ce5e0 con 0x7fd52415cd90 2023-12-01T03:12:50.772 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.770+0000 7fd529b88640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5240d0ec0 con 0x7fd5240b5fa0 2023-12-01T03:12:50.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.770+0000 7fd529b88640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd524161950 con 0x7fd5240cfd60 2023-12-01T03:12:50.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.770+0000 7fd523fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd5240b5fa0 0x7fd524161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:50.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.770+0000 7fd523fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fd5240b5fa0 0x7fd524161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34096/0 (socket says 172.21.15.115:34096) 2023-12-01T03:12:50.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.770+0000 7fd523fff640 1 -- 172.21.15.115:0/1632872438 learned_addr learned my addr 172.21.15.115:0/1632872438 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:50.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.771+0000 7fd522ffd640 1 --2- 172.21.15.115:0/1632872438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5240cfd60 0x7fd5240cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:50.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.771+0000 7fd523fff640 1 -- 172.21.15.115:0/1632872438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5240cfd60 msgr2=0x7fd5240cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:50.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.771+0000 7fd523fff640 1 --2- 172.21.15.115:0/1632872438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5240cfd60 0x7fd5240cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.771+0000 7fd523fff640 1 -- 172.21.15.115:0/1632872438 >> v1:172.21.15.142:6789/0 conn(0x7fd52415cd90 legacy=0x7fd5240cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:50.773 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.771+0000 7fd523fff640 1 -- 172.21.15.115:0/1632872438 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240ce300 con 0x7fd5240b5fa0 2023-12-01T03:12:50.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.772+0000 7fd523fff640 1 --2- 172.21.15.115:0/1632872438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 0x7fd524161410 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009e50 tx=0x7fd51c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f708e29074853ecd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:50.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.772+0000 7fd5227fc640 1 -- 172.21.15.115:0/1632872438 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c02c9e0 con 0x7fd5240b5fa0 2023-12-01T03:12:50.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.772+0000 7fd5227fc640 1 -- 172.21.15.115:0/1632872438 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c02cb40 con 0x7fd5240b5fa0 2023-12-01T03:12:50.774 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.772+0000 7fd5227fc640 1 -- 172.21.15.115:0/1632872438 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0343f0 con 0x7fd5240b5fa0 2023-12-01T03:12:50.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.773+0000 7fd529b88640 1 -- 172.21.15.115:0/1632872438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 msgr2=0x7fd524161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:50.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.773+0000 7fd529b88640 1 --2- 172.21.15.115:0/1632872438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 0x7fd524161410 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009e50 tx=0x7fd51c02be40 comp rx=0 tx=0).stop 2023-12-01T03:12:50.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.773+0000 7fd529b88640 1 -- 172.21.15.115:0/1632872438 shutdown_connections 2023-12-01T03:12:50.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.773+0000 7fd529b88640 1 --2- 172.21.15.115:0/1632872438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 0x7fd524161410 unknown :-1 s=CLOSED pgs=3008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.773+0000 7fd529b88640 1 --2- 172.21.15.115:0/1632872438 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5240cfd60 0x7fd5240cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.773+0000 7fd529b88640 1 -- 172.21.15.115:0/1632872438 >> 172.21.15.115:0/1632872438 conn(0x7fd524065960 msgr2=0x7fd524065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:50.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.773+0000 7fd529b88640 1 -- 172.21.15.115:0/1632872438 shutdown_connections 2023-12-01T03:12:50.775 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.773+0000 7fd529b88640 1 -- 172.21.15.115:0/1632872438 wait complete. 2023-12-01T03:12:50.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.775+0000 7fd529b88640 1 Processor -- start 2023-12-01T03:12:50.776 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.775+0000 7fd529b88640 1 -- start start 2023-12-01T03:12:50.778 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.776+0000 7fd529b88640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 0x7fd5240b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.776+0000 7fd529b88640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5240cfd60 0x7fd524154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.776+0000 7fd529b88640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd52415cd90 0x7fd524154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.776+0000 7fd529b88640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240ca3e0 con 0x7fd5240b5fa0 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.776+0000 7fd529b88640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd524160430 con 0x7fd5240cfd60 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.776+0000 7fd529b88640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52415dd60 con 0x7fd52415cd90 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.776+0000 7fd5237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 0x7fd5240b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.776+0000 7fd5237fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 0x7fd5240b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34106/0 (socket says 172.21.15.115:34106) 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.776+0000 7fd5237fe640 1 -- 172.21.15.115:0/3855828862 learned_addr learned my addr 172.21.15.115:0/3855828862 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.776+0000 7fd522ffd640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5240cfd60 0x7fd524154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd523fff640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd52415cd90 0x7fd524154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd5237fe640 1 -- 172.21.15.115:0/3855828862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd52415cd90 msgr2=0x7fd524154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd5237fe640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd52415cd90 0x7fd524154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.779 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd5237fe640 1 -- 172.21.15.115:0/3855828862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5240cfd60 msgr2=0x7fd524154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:50.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd5237fe640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5240cfd60 0x7fd524154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd5237fe640 1 -- 172.21.15.115:0/3855828862 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c009b30 con 0x7fd5240b5fa0 2023-12-01T03:12:50.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd5237fe640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 0x7fd5240b4e90 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7fd51400c430 tx=0x7fd51400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:50.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd50bfff640 1 -- 172.21.15.115:0/3855828862 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514011790 con 0x7fd5240b5fa0 2023-12-01T03:12:50.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd50bfff640 1 -- 172.21.15.115:0/3855828862 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5140118f0 con 0x7fd5240b5fa0 2023-12-01T03:12:50.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd50bfff640 1 -- 172.21.15.115:0/3855828862 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514011ce0 con 0x7fd5240b5fa0 2023-12-01T03:12:50.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.777+0000 7fd529b88640 1 -- 172.21.15.115:0/3855828862 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52415e040 con 0x7fd5240b5fa0 2023-12-01T03:12:50.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.778+0000 7fd529b88640 1 -- 172.21.15.115:0/3855828862 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5240ca070 con 0x7fd5240b5fa0 2023-12-01T03:12:50.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.779+0000 7fd529b88640 1 -- 172.21.15.115:0/3855828862 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5240ca230 con 0x7fd5240b5fa0 2023-12-01T03:12:50.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.780+0000 7fd50bfff640 1 -- 172.21.15.115:0/3855828862 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 143) v1 ==== 88828+0+0 (secure 0 0 0) 0x7fd51401c460 con 0x7fd5240b5fa0 2023-12-01T03:12:50.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.780+0000 7fd50bfff640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd51006a6c0 0x7fd51006cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:50.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.781+0000 7fd50bfff640 1 -- 172.21.15.115:0/3855828862 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fd514088cd0 con 0x7fd5240b5fa0 2023-12-01T03:12:50.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.781+0000 7fd522ffd640 1 -- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd51006a6c0 msgr2=0x7fd51006cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:50.782 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.781+0000 7fd522ffd640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd51006a6c0 0x7fd51006cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:50.786 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.785+0000 7fd50bfff640 1 -- 172.21.15.115:0/3855828862 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fd514054000 con 0x7fd5240b5fa0 2023-12-01T03:12:50.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.980+0000 7fd529b88640 1 -- 172.21.15.115:0/3855828862 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fd52404e890 con 0x7fd5240b5fa0 2023-12-01T03:12:50.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.982+0000 7fd522ffd640 1 -- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd51006a6c0 msgr2=0x7fd51006cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:50.984 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.982+0000 7fd522ffd640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd51006a6c0 0x7fd51006cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:12:50.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.986+0000 7fd50bfff640 1 -- 172.21.15.115:0/3855828862 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v83)=0 v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd514057620 con 0x7fd5240b5fa0 2023-12-01T03:12:50.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 -- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd51006a6c0 msgr2=0x7fd51006cb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:50.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd51006a6c0 0x7fd51006cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 -- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 msgr2=0x7fd5240b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:50.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 0x7fd5240b4e90 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7fd51400c430 tx=0x7fd51400c8f0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 -- 172.21.15.115:0/3855828862 shutdown_connections 2023-12-01T03:12:50.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fd51006a6c0 0x7fd51006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd52415cd90 0x7fd524154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5240cfd60 0x7fd524154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 --2- 172.21.15.115:0/3855828862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5240b5fa0 0x7fd5240b4e90 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:50.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 -- 172.21.15.115:0/3855828862 >> 172.21.15.115:0/3855828862 conn(0x7fd524065960 msgr2=0x7fd5240cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:50.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 -- 172.21.15.115:0/3855828862 shutdown_connections 2023-12-01T03:12:50.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:50.988+0000 7fd529b88640 1 -- 172.21.15.115:0/3855828862 wait complete. 2023-12-01T03:12:51.009 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-01T03:12:51.073 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:51.072+0000 7f93af06f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:12:51.089 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:51.087+0000 7f5743411040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:12:51.126 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:51.125+0000 7f93af06f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:12:51.144 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:51.143+0000 7f5743411040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:12:51.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.216+0000 7f2d8c1f0640 1 Processor -- start 2023-12-01T03:12:51.218 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.216+0000 7f2d8c1f0640 1 -- start start 2023-12-01T03:12:51.220 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.218+0000 7f2d8c1f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d8415cf70 0x7f2d840b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.218+0000 7f2d8c1f0640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2d840b18f0 0x7f2d840b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.218+0000 7f2d8c1f0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d840b7760 con 0x7f2d840cfb50 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.218+0000 7f2d8c1f0640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d840b2170 con 0x7f2d840b18f0 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.218+0000 7f2d8c1f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840b8b70 con 0x7f2d8415cf70 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.218+0000 7f2d8978b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2d840b18f0 0x7f2d840b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.218+0000 7f2d8978b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2d840b18f0 0x7f2d840b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34122/0 (socket says 172.21.15.115:34122) 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.218+0000 7f2d8978b640 1 -- 172.21.15.115:0/154222539 learned_addr learned my addr 172.21.15.115:0/154222539 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.219+0000 7f2d89f8c640 1 --2- 172.21.15.115:0/154222539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d8415cf70 0x7f2d840b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.219+0000 7f2d8978b640 1 -- 172.21.15.115:0/154222539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d8415cf70 msgr2=0x7f2d840b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.219+0000 7f2d8978b640 1 --2- 172.21.15.115:0/154222539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d8415cf70 0x7f2d840b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.219+0000 7f2d8978b640 1 -- 172.21.15.115:0/154222539 >> v1:172.21.15.142:6789/0 conn(0x7f2d840cfb50 legacy=0x7f2d840d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.219+0000 7f2d8978b640 1 -- 172.21.15.115:0/154222539 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d840b6e30 con 0x7f2d840b18f0 2023-12-01T03:12:51.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.219+0000 7f2d89f8c640 1 --2- 172.21.15.115:0/154222539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d8415cf70 0x7f2d840b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:51.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.220+0000 7f2d8978b640 1 --2- 172.21.15.115:0/154222539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840b18f0 0x7f2d840b7bf0 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009870 tx=0x7f2d7402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63494d3b35e9b6f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:51.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.220+0000 7f2d88f8a640 1 -- 172.21.15.115:0/154222539 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74030840 con 0x7f2d840b18f0 2023-12-01T03:12:51.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.220+0000 7f2d88f8a640 1 -- 172.21.15.115:0/154222539 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d740309a0 con 0x7f2d840b18f0 2023-12-01T03:12:51.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.220+0000 7f2d88f8a640 1 -- 172.21.15.115:0/154222539 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74030d30 con 0x7f2d840b18f0 2023-12-01T03:12:51.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.221+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/154222539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840b18f0 msgr2=0x7f2d840b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:51.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.221+0000 7f2d8c1f0640 1 --2- 172.21.15.115:0/154222539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840b18f0 0x7f2d840b7bf0 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009870 tx=0x7f2d7402b980 comp rx=0 tx=0).stop 2023-12-01T03:12:51.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.221+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/154222539 shutdown_connections 2023-12-01T03:12:51.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.221+0000 7f2d8c1f0640 1 --2- 172.21.15.115:0/154222539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840b18f0 0x7f2d840b7bf0 unknown :-1 s=CLOSED pgs=3010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.221+0000 7f2d8c1f0640 1 --2- 172.21.15.115:0/154222539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d8415cf70 0x7f2d840b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.221+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/154222539 >> 172.21.15.115:0/154222539 conn(0x7f2d8406a0e0 msgr2=0x7f2d8406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:51.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.221+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/154222539 shutdown_connections 2023-12-01T03:12:51.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.221+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/154222539 wait complete. 2023-12-01T03:12:51.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.223+0000 7f2d8c1f0640 1 Processor -- start 2023-12-01T03:12:51.225 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.223+0000 7f2d8c1f0640 1 -- start start 2023-12-01T03:12:51.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.224+0000 7f2d8c1f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d840b18f0 0x7f2d8415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.224+0000 7f2d8c1f0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840cfb50 0x7f2d8415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.224+0000 7f2d8c1f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d8415cf70 0x7f2d841f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.224+0000 7f2d8c1f0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8415a5d0 con 0x7f2d840cfb50 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.224+0000 7f2d8c1f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d84154fe0 con 0x7f2d8415cf70 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.224+0000 7f2d8c1f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841552c0 con 0x7f2d840b18f0 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.224+0000 7f2d8978b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840cfb50 0x7f2d8415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.224+0000 7f2d8978b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840cfb50 0x7f2d8415acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34132/0 (socket says 172.21.15.115:34132) 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.224+0000 7f2d8978b640 1 -- 172.21.15.115:0/1010140197 learned_addr learned my addr 172.21.15.115:0/1010140197 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.225+0000 7f2d89f8c640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d840b18f0 0x7f2d8415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.225+0000 7f2d8a78d640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d8415cf70 0x7f2d841f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.225+0000 7f2d8978b640 1 -- 172.21.15.115:0/1010140197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d840b18f0 msgr2=0x7f2d8415a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.225+0000 7f2d8978b640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d840b18f0 0x7f2d8415a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.225+0000 7f2d8978b640 1 -- 172.21.15.115:0/1010140197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d8415cf70 msgr2=0x7f2d841f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.225+0000 7f2d8978b640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d8415cf70 0x7f2d841f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.225+0000 7f2d8978b640 1 -- 172.21.15.115:0/1010140197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d740094d0 con 0x7f2d840cfb50 2023-12-01T03:12:51.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.225+0000 7f2d8978b640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840cfb50 0x7f2d8415acf0 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7f2d74000c00 tx=0x7f2d74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:51.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.226+0000 7f2d7a7fc640 1 -- 172.21.15.115:0/1010140197 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74030810 con 0x7f2d840cfb50 2023-12-01T03:12:51.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.226+0000 7f2d7a7fc640 1 -- 172.21.15.115:0/1010140197 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74043400 con 0x7f2d840cfb50 2023-12-01T03:12:51.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.226+0000 7f2d7a7fc640 1 -- 172.21.15.115:0/1010140197 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d740437d0 con 0x7f2d840cfb50 2023-12-01T03:12:51.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.226+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/1010140197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d841555a0 con 0x7f2d840cfb50 2023-12-01T03:12:51.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.226+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/1010140197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d84155b60 con 0x7f2d840cfb50 2023-12-01T03:12:51.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.227+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/1010140197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d84155d50 con 0x7f2d840cfb50 2023-12-01T03:12:51.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.228+0000 7f2d7a7fc640 1 -- 172.21.15.115:0/1010140197 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 143) v1 ==== 88828+0+0 (secure 0 0 0) 0x7f2d74043930 con 0x7f2d840cfb50 2023-12-01T03:12:51.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.228+0000 7f2d7a7fc640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2d6806a6c0 0x7f2d6806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.229+0000 7f2d7a7fc640 1 -- 172.21.15.115:0/1010140197 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7f2d7403e070 con 0x7f2d840cfb50 2023-12-01T03:12:51.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.229+0000 7f2d89f8c640 1 -- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2d6806a6c0 msgr2=0x7f2d6806cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:51.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.229+0000 7f2d89f8c640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2d6806a6c0 0x7f2d6806cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:51.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.233+0000 7f2d7a7fc640 1 -- 172.21.15.115:0/1010140197 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2d74077270 con 0x7f2d840cfb50 2023-12-01T03:12:51.430 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.429+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/1010140197 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f2d8404e890 con 0x7f2d840cfb50 2023-12-01T03:12:51.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.430+0000 7f2d89f8c640 1 -- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2d6806a6c0 msgr2=0x7f2d6806cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:51.431 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.430+0000 7f2d89f8c640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2d6806a6c0 0x7f2d6806cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:12:51.438 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.437+0000 7f2d7a7fc640 1 -- 172.21.15.115:0/1010140197 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v84)=0 v84) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2d7407a890 con 0x7f2d840cfb50 2023-12-01T03:12:51.441 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.439+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2d6806a6c0 msgr2=0x7f2d6806cb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.439+0000 7f2d8c1f0640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2d6806a6c0 0x7f2d6806cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.439+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840cfb50 msgr2=0x7f2d8415acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.439+0000 7f2d8c1f0640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840cfb50 0x7f2d8415acf0 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7f2d74000c00 tx=0x7f2d74004730 comp rx=0 tx=0).stop 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.440+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/1010140197 shutdown_connections 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.440+0000 7f2d8c1f0640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2d6806a6c0 0x7f2d6806cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.440+0000 7f2d8c1f0640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d8415cf70 0x7f2d841f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.440+0000 7f2d8c1f0640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d840cfb50 0x7f2d8415acf0 unknown :-1 s=CLOSED pgs=3011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.440+0000 7f2d8c1f0640 1 --2- 172.21.15.115:0/1010140197 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d840b18f0 0x7f2d8415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.440+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/1010140197 >> 172.21.15.115:0/1010140197 conn(0x7f2d8406a0e0 msgr2=0x7f2d840cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.440+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/1010140197 shutdown_connections 2023-12-01T03:12:51.442 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.440+0000 7f2d8c1f0640 1 -- 172.21.15.115:0/1010140197 wait complete. 2023-12-01T03:12:51.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:51.460 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-01T03:12:51.515 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-01T03:12:51.579 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:51.578+0000 7f93af06f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:12:51.621 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:51.619+0000 7f5743411040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:12:51.632 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:51.631+0000 7f93af06f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:12:51.675 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:51.673+0000 7f5743411040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:12:51.682 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:51.681+0000 7f93af06f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:12:51.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.722+0000 7fb1a6157640 1 Processor -- start 2023-12-01T03:12:51.724 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.723+0000 7fb1a6157640 1 -- start start 2023-12-01T03:12:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.724+0000 7fb1a6157640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a00cfb80 0x7fb1a00ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.724+0000 7fb1a6157640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb1a0154010 0x7fb1a0160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.724+0000 7fb1a6157640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a00cdcd0 con 0x7fb1a0153c30 2023-12-01T03:12:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.724+0000 7fb1a6157640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a0161830 con 0x7fb1a0154010 2023-12-01T03:12:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.724+0000 7fb1a6157640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00cd3a0 con 0x7fb1a00cfb80 2023-12-01T03:12:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.724+0000 7fb1a5155640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb1a0153c30 0x7fb1a00cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40336/0 (socket says 172.21.15.115:40336) 2023-12-01T03:12:51.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.724+0000 7fb1a5155640 1 -- 172.21.15.115:0/3836548242 learned_addr learned my addr 172.21.15.115:0/3836548242 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:51.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb1a5956640 1 --2- 172.21.15.115:0/3836548242 >> v2:172.21.15.142:3300/0 conn(0x7fb1a0154010 0x7fb1a0160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351118376 0 0) 0x7fb1a00cdcd0 con 0x7fb1a0153c30 2023-12-01T03:12:51.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb18c003520 con 0x7fb1a0153c30 2023-12-01T03:12:51.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb1a4954640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a00cfb80 0x7fb1a00ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.728 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:51.725+0000 7f5743411040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:12:51.729 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1663179628 0 0) 0x7fb18c003520 con 0x7fb1a0153c30 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a00cfb80 msgr2=0x7fb1a00ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a00cfb80 0x7fb1a00ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 >> v2:172.21.15.142:3300/0 conn(0x7fb1a0154010 msgr2=0x7fb1a0160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 --2- 172.21.15.115:0/3836548242 >> v2:172.21.15.142:3300/0 conn(0x7fb1a0154010 0x7fb1a0160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a00cd680 con 0x7fb1a0153c30 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb19c001670 con 0x7fb1a0153c30 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb1a4954640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a00cfb80 0x7fb1a00ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 >> v1:172.21.15.115:6789/0 conn(0x7fb1a0153c30 legacy=0x7fb1a00cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a00cfb80 0x7fb18c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18c003f00 0x7fb18c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c004d00 0x7fb18c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0161830 con 0x7fb18c003f00 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00cd3a0 con 0x7fb18c004d00 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.725+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c00ba00 con 0x7fb1a00cfb80 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.726+0000 7fb1a4954640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18c003f00 0x7fb18c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.726+0000 7fb1a5155640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a00cfb80 0x7fb18c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.726+0000 7fb1a5956640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c004d00 0x7fb18c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.726+0000 7fb1a4954640 1 -- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a00cfb80 msgr2=0x7fb18c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.726+0000 7fb1a4954640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb1a00cfb80 0x7fb18c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.726+0000 7fb1a4954640 1 -- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c004d00 msgr2=0x7fb18c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.726+0000 7fb1a4954640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb18c004d00 0x7fb18c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.726+0000 7fb1a4954640 1 -- 172.21.15.115:0/3836548242 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c003520 con 0x7fb18c003f00 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.728+0000 7fb1a4954640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18c003f00 0x7fb18c0047c0 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7fb1900098a0 tx=0x7fb18c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3006c5c7d92b3860 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.728+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190004230 con 0x7fb18c003f00 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.728+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb190004390 con 0x7fb18c003f00 2023-12-01T03:12:51.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.728+0000 7fb197fff640 1 -- 172.21.15.115:0/3836548242 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190003830 con 0x7fb18c003f00 2023-12-01T03:12:51.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.729+0000 7fb1a6157640 1 -- 172.21.15.115:0/3836548242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18c003f00 msgr2=0x7fb18c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:51.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.729+0000 7fb1a6157640 1 --2- 172.21.15.115:0/3836548242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb18c003f00 0x7fb18c0047c0 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7fb1900098a0 tx=0x7fb18c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.729+0000 7fb1a5956640 1 -- 172.21.15.115:0/3836548242 reap_dead start 2023-12-01T03:12:51.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.729+0000 7fb1a6157640 1 -- 172.21.15.115:0/3836548242 shutdown_connections 2023-12-01T03:12:51.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.729+0000 7fb1a6157640 1 -- 172.21.15.115:0/3836548242 >> 172.21.15.115:0/3836548242 conn(0x7fb1a006a0b0 msgr2=0x7fb1a006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:51.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.729+0000 7fb1a6157640 1 -- 172.21.15.115:0/3836548242 shutdown_connections 2023-12-01T03:12:51.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.729+0000 7fb1a6157640 1 -- 172.21.15.115:0/3836548242 wait complete. 2023-12-01T03:12:51.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.731+0000 7fb1a6157640 1 Processor -- start 2023-12-01T03:12:51.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.731+0000 7fb1a6157640 1 -- start start 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a6157640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb18c003f00 0x7fb1a00c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a6157640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a0154420 0x7fb1a01548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a6157640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a01566a0 0x7fb1a0156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a6157640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0155ba0 con 0x7fb1a01566a0 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a6157640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00c5cc0 con 0x7fb1a0154420 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a6157640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0157060 con 0x7fb18c003f00 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a5956640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a01566a0 0x7fb1a0156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a5956640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a01566a0 0x7fb1a0156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34142/0 (socket says 172.21.15.115:34142) 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a5956640 1 -- 172.21.15.115:0/3153269935 learned_addr learned my addr 172.21.15.115:0/3153269935 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a4954640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a0154420 0x7fb1a01548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a5155640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb18c003f00 0x7fb1a00c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:51.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a5956640 1 -- 172.21.15.115:0/3153269935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb18c003f00 msgr2=0x7fb1a00c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a5956640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb18c003f00 0x7fb1a00c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a5956640 1 -- 172.21.15.115:0/3153269935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a0154420 msgr2=0x7fb1a01548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a5956640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a0154420 0x7fb1a01548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.732+0000 7fb1a5956640 1 -- 172.21.15.115:0/3153269935 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a0154f90 con 0x7fb1a01566a0 2023-12-01T03:12:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.733+0000 7fb1a5956640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a01566a0 0x7fb1a0156b20 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00d0010 tx=0x7fb1980115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.733+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19800e070 con 0x7fb1a01566a0 2023-12-01T03:12:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.733+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb198011e10 con 0x7fb1a01566a0 2023-12-01T03:12:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.733+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198003990 con 0x7fb1a01566a0 2023-12-01T03:12:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.733+0000 7fb1a6157640 1 -- 172.21.15.115:0/3153269935 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a0155270 con 0x7fb1a01566a0 2023-12-01T03:12:51.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.733+0000 7fb1a6157640 1 -- 172.21.15.115:0/3153269935 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a0155830 con 0x7fb1a01566a0 2023-12-01T03:12:51.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.734+0000 7fb1a6157640 1 -- 172.21.15.115:0/3153269935 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a01559f0 con 0x7fb1a01566a0 2023-12-01T03:12:51.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.735+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 143) v1 ==== 88828+0+0 (secure 0 0 0) 0x7fb198003af0 con 0x7fb1a01566a0 2023-12-01T03:12:51.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.736+0000 7fb195ffb640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:51.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.736+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5753+0+0 (secure 0 0 0) 0x7fb198089670 con 0x7fb1a01566a0 2023-12-01T03:12:51.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.736+0000 7fb1a5155640 1 -- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 msgr2=0x7fb18406cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:51.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.736+0000 7fb1a5155640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:51.743 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:51.742+0000 7f93af06f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:12:51.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.744+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb1980547c0 con 0x7fb1a01566a0 2023-12-01T03:12:51.793 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:51.792+0000 7f93af06f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:12:51.795 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:51.793+0000 7f5743411040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:12:51.845 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:51.843+0000 7f5743411040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:12:51.903 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:51.902+0000 7f93af06f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:12:51.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.937+0000 7fb1a5155640 1 -- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 msgr2=0x7fb18406cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:51.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.937+0000 7fb1a5155640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:12:51.958 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:51.957+0000 7f5743411040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:12:51.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:51.960+0000 7fb1a6157640 1 -- 172.21.15.115:0/3153269935 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1a00cd930 con 0x7fb18406a6c0 2023-12-01T03:12:51.965 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:51.964+0000 7f93af06f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:12:52.014 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:52.013+0000 7f93af06f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:12:52.021 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:52.019+0000 7f5743411040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:12:52.071 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:52.070+0000 7f93af06f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:12:52.071 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:52.069+0000 7f5743411040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:12:52.122 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:52.121+0000 7f93af06f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:12:52.128 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:52.127+0000 7f5743411040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:12:52.179 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:52.177+0000 7f5743411040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:12:52.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:52.338+0000 7fb1a5155640 1 -- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 msgr2=0x7fb18406cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:52.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:52.338+0000 7fb1a5155640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T03:12:52.511 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:52.510+0000 7f93af06f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:12:52.560 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:52.558+0000 7f5743411040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:12:52.670 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:52.669+0000 7f93af06f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:12:52.720 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:52.718+0000 7f5743411040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:12:52.778 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:52.777+0000 7f93af06f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:12:52.827 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:52.825+0000 7f5743411040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:12:52.828 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:52.827+0000 7f93af06f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:12:52.877 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:52.875+0000 7f5743411040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:12:52.935 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:52.934+0000 7f93af06f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:52.983 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:52.981+0000 7f5743411040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:53.042 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:53.041+0000 7f93af06f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:12:53.091 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:53.089+0000 7f5743411040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:12:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:53.139+0000 7fb1a5155640 1 -- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 msgr2=0x7fb18406cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:53.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:53.139+0000 7fb1a5155640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T03:12:53.198 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:53.197+0000 7f93af06f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:53.247 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:53.245+0000 7f5743411040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:12:53.247 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:53.246+0000 7f93af06f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:12:53.296 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:53.294+0000 7f5743411040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:12:53.366 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:53.365+0000 7f93af06f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:12:53.414 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:53.411+0000 7f5743411040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:12:54.199 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:54.198+0000 7f93af06f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:12:54.245 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.243+0000 7f5743411040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:12:54.493 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:54.492+0000 7f93af06f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:12:54.541 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.539+0000 7f5743411040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:12:54.549 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:54.548+0000 7f93af06f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:12:54.599 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.597+0000 7f5743411040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:12:54.661 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:54.661+0000 7f93af06f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:12:54.713 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.711+0000 7f5743411040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:12:54.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:54.730+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 144) v1 ==== 88335+0+0 (secure 0 0 0) 0x7fb198054b90 con 0x7fb1a01566a0 2023-12-01T03:12:54.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:54.731+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 msgr2=0x7fb18406cb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:54.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:54.731+0000 7fb195ffb640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:54.897 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.894+0000 7f56bfc94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:54.897 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.894+0000 7f56bfc94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:54.897 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.894+0000 7f56bfc94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:54.897 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.894+0000 7f56bfc94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:54.897 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.894+0000 7f56bfc94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:54.897 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.895+0000 7f56c2c9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:54.897 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.895+0000 7f56c2c9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:54.897 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.895+0000 7f56c2c9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:54.897 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.895+0000 7f56c2c9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:54.897 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:54.895+0000 7f56c2c9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:12:55.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.756+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 145) v1 ==== 88462+0+0 (secure 0 0 0) 0x7fb198014350 con 0x7fb1a01566a0 2023-12-01T03:12:55.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.756+0000 7fb195ffb640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:55.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.756+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1a00cd930 con 0x7fb18406a6c0 2023-12-01T03:12:55.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.759+0000 7fb1a5155640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:55.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.761+0000 7fb1a5155640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb19c001eb0 tx=0x7fb19c001e40 comp rx=0 tx=0).ready entity=mgr.12456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:55.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.899+0000 7fb195ffb640 1 -- 172.21.15.115:0/3153269935 <== mgr.12456 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb1a00cd930 con 0x7fb18406a6c0 2023-12-01T03:12:55.902 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule set for path snap_test_dir1 2023-12-01T03:12:55.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 -- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 msgr2=0x7fb18406cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:55.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb19c001eb0 tx=0x7fb19c001e40 comp rx=0 tx=0).stop 2023-12-01T03:12:55.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 -- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a01566a0 msgr2=0x7fb1a0156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:55.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a01566a0 0x7fb1a0156b20 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00d0010 tx=0x7fb1980115a0 comp rx=0 tx=0).stop 2023-12-01T03:12:55.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 -- 172.21.15.115:0/3153269935 shutdown_connections 2023-12-01T03:12:55.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb18406a6c0 0x7fb18406cb00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:55.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1a01566a0 0x7fb1a0156b20 unknown :-1 s=CLOSED pgs=3014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:55.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1a0154420 0x7fb1a01548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:55.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 --2- 172.21.15.115:0/3153269935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb18c003f00 0x7fb1a00c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:55.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 -- 172.21.15.115:0/3153269935 >> 172.21.15.115:0/3153269935 conn(0x7fb1a006a0b0 msgr2=0x7fb1a00d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:55.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.903+0000 7fb1a6157640 1 -- 172.21.15.115:0/3153269935 shutdown_connections 2023-12-01T03:12:55.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:55.904+0000 7fb1a6157640 1 -- 172.21.15.115:0/3153269935 wait complete. 2023-12-01T03:12:55.927 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-01T03:12:55.927 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1M 2023-12-01T03:12:56.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.134+0000 7ff78bb0a640 1 Processor -- start 2023-12-01T03:12:56.136 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.134+0000 7ff78bb0a640 1 -- start start 2023-12-01T03:12:56.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.136+0000 7ff78bb0a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff78415ac70 0x7ff7840cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.136+0000 7ff78bb0a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff7840b4070 0x7ff7840cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.136+0000 7ff78bb0a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78406c860 con 0x7ff7840adca0 2023-12-01T03:12:56.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.136+0000 7ff78bb0a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78415b050 con 0x7ff7840b4070 2023-12-01T03:12:56.138 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.136+0000 7ff78bb0a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840ce0b0 con 0x7ff78415ac70 2023-12-01T03:12:56.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.136+0000 7ff78a0a7640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7ff7840adca0 0x7ff78406b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40350/0 (socket says 172.21.15.115:40350) 2023-12-01T03:12:56.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.136+0000 7ff78a0a7640 1 -- 172.21.15.115:0/3159143908 learned_addr learned my addr 172.21.15.115:0/3159143908 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:56.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7890a5640 1 --2- 172.21.15.115:0/3159143908 >> v2:172.21.15.142:3300/0 conn(0x7ff7840b4070 0x7ff7840cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7898a6640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff78415ac70 0x7ff7840cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789469226 0 0) 0x7ff78406c860 con 0x7ff7840adca0 2023-12-01T03:12:56.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff770003520 con 0x7ff7840adca0 2023-12-01T03:12:56.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1667715266 0 0) 0x7ff770003520 con 0x7ff7840adca0 2023-12-01T03:12:56.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff78415ac70 msgr2=0x7ff7840cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:56.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7888a4640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff78415ac70 0x7ff7840cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.140 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 >> v2:172.21.15.142:3300/0 conn(0x7ff7840b4070 msgr2=0x7ff7840cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7888a4640 1 --2- 172.21.15.115:0/3159143908 >> v2:172.21.15.142:3300/0 conn(0x7ff7840b4070 0x7ff7840cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7840ce390 con 0x7ff7840adca0 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7ff77c0019f0 con 0x7ff7840adca0 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.137+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 >> v1:172.21.15.115:6789/0 conn(0x7ff7840adca0 legacy=0x7ff78406b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff7888a4640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770003970 0x7ff770003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff7888a4640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff78415ac70 0x7ff7700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff7888a4640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff770004d00 0x7ff770005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78415b050 con 0x7ff770004d00 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840ce0b0 con 0x7ff770003970 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77000b980 con 0x7ff78415ac70 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff78a0a7640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff770004d00 0x7ff770005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff7898a6640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770003970 0x7ff770003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff78a0a7640 1 -- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff78415ac70 msgr2=0x7ff7700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff7890a5640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff78415ac70 0x7ff7700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff78a0a7640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff78415ac70 0x7ff7700047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.141 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff78a0a7640 1 -- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770003970 msgr2=0x7ff770003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:56.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff78a0a7640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770003970 0x7ff770003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff78a0a7640 1 -- 172.21.15.115:0/3159143908 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770003520 con 0x7ff770004d00 2023-12-01T03:12:56.142 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.138+0000 7ff7898a6640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770003970 0x7ff770003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.139+0000 7ff78a0a7640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff770004d00 0x7ff770005180 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00a860 tx=0x7ff7700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6293a48c8be32379 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.140+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c034630 con 0x7ff770004d00 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.140+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c02f070 con 0x7ff770004d00 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.140+0000 7ff7888a4640 1 -- 172.21.15.115:0/3159143908 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0349f0 con 0x7ff770004d00 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.140+0000 7ff78bb0a640 1 -- 172.21.15.115:0/3159143908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff770004d00 msgr2=0x7ff770005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.140+0000 7ff78bb0a640 1 --2- 172.21.15.115:0/3159143908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff770004d00 0x7ff770005180 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00a860 tx=0x7ff7700057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.140+0000 7ff78a0a7640 1 -- 172.21.15.115:0/3159143908 reap_dead start 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.140+0000 7ff78bb0a640 1 -- 172.21.15.115:0/3159143908 shutdown_connections 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.140+0000 7ff78bb0a640 1 -- 172.21.15.115:0/3159143908 >> 172.21.15.115:0/3159143908 conn(0x7ff7840662c0 msgr2=0x7ff7840b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.140+0000 7ff78bb0a640 1 -- 172.21.15.115:0/3159143908 shutdown_connections 2023-12-01T03:12:56.143 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.140+0000 7ff78bb0a640 1 -- 172.21.15.115:0/3159143908 wait complete. 2023-12-01T03:12:56.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.142+0000 7ff78bb0a640 1 Processor -- start 2023-12-01T03:12:56.144 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.142+0000 7ff78bb0a640 1 -- start start 2023-12-01T03:12:56.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.143+0000 7ff78bb0a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770004d00 0x7ff78414fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.143+0000 7ff78bb0a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7840c8430 0x7ff78415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.143+0000 7ff78bb0a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7840b2b80 0x7ff7840b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.145 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.143+0000 7ff78bb0a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840b3540 con 0x7ff7840b2b80 2023-12-01T03:12:56.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.143+0000 7ff78bb0a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840c2d10 con 0x7ff770004d00 2023-12-01T03:12:56.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.143+0000 7ff78bb0a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78415d6e0 con 0x7ff7840c8430 2023-12-01T03:12:56.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff78a0a7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7840b2b80 0x7ff7840b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff78a0a7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7840b2b80 0x7ff7840b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34286/0 (socket says 172.21.15.115:34286) 2023-12-01T03:12:56.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff78a0a7640 1 -- 172.21.15.115:0/2476387670 learned_addr learned my addr 172.21.15.115:0/2476387670 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:56.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff78a0a7640 1 -- 172.21.15.115:0/2476387670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7840c8430 msgr2=0x7ff78415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.147 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff7898a6640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770004d00 0x7ff78414fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff78a0a7640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7840c8430 0x7ff78415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff78a0a7640 1 -- 172.21.15.115:0/2476387670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770004d00 msgr2=0x7ff78414fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff78a0a7640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770004d00 0x7ff78414fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff78a0a7640 1 -- 172.21.15.115:0/2476387670 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7840c23e0 con 0x7ff7840b2b80 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff7898a6640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770004d00 0x7ff78414fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.144+0000 7ff78a0a7640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7840b2b80 0x7ff7840b3000 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00a860 tx=0x7ff77c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.145+0000 7ff77a7fc640 1 -- 172.21.15.115:0/2476387670 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c001fd0 con 0x7ff7840b2b80 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.145+0000 7ff77a7fc640 1 -- 172.21.15.115:0/2476387670 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c034640 con 0x7ff7840b2b80 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.145+0000 7ff78bb0a640 1 -- 172.21.15.115:0/2476387670 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7840c26c0 con 0x7ff7840b2b80 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.145+0000 7ff77a7fc640 1 -- 172.21.15.115:0/2476387670 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0349b0 con 0x7ff7840b2b80 2023-12-01T03:12:56.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.145+0000 7ff78bb0a640 1 -- 172.21.15.115:0/2476387670 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78415d2b0 con 0x7ff7840b2b80 2023-12-01T03:12:56.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.146+0000 7ff78bb0a640 1 -- 172.21.15.115:0/2476387670 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78415d440 con 0x7ff7840b2b80 2023-12-01T03:12:56.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.147+0000 7ff77a7fc640 1 -- 172.21.15.115:0/2476387670 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 145) v1 ==== 88462+0+0 (secure 0 0 0) 0x7ff77c031030 con 0x7ff7840b2b80 2023-12-01T03:12:56.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.148+0000 7ff77a7fc640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff76806a2d0 0x7ff76806c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.148+0000 7ff77a7fc640 1 -- 172.21.15.115:0/2476387670 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7ff77c0aa9a0 con 0x7ff7840b2b80 2023-12-01T03:12:56.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.148+0000 7ff7898a6640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff76806a2d0 0x7ff76806c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.149+0000 7ff7898a6640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff76806a2d0 0x7ff76806c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff780009aa0 tx=0x7ff780008040 comp rx=0 tx=0).ready entity=mgr.12456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:56.154 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.152+0000 7ff77a7fc640 1 -- 172.21.15.115:0/2476387670 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff77c076020 con 0x7ff7840b2b80 2023-12-01T03:12:56.367 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.366+0000 7ff78bb0a640 1 -- 172.21.15.115:0/2476387670 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff78406cb00 con 0x7ff76806a2d0 2023-12-01T03:12:56.483 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.481+0000 7ff77a7fc640 1 -- 172.21.15.115:0/2476387670 <== mgr.12456 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7ff7800119a0 con 0x7ff76806a2d0 2023-12-01T03:12:56.483 INFO:teuthology.orchestra.run.smithi115.stdout:Retention added to path snap_test_dir1 2023-12-01T03:12:56.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.484+0000 7ff78bb0a640 1 -- 172.21.15.115:0/2476387670 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff76806a2d0 msgr2=0x7ff76806c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.484+0000 7ff78bb0a640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff76806a2d0 0x7ff76806c790 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff780009aa0 tx=0x7ff780008040 comp rx=0 tx=0).stop 2023-12-01T03:12:56.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.484+0000 7ff78bb0a640 1 -- 172.21.15.115:0/2476387670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7840b2b80 msgr2=0x7ff7840b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.486 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.484+0000 7ff78bb0a640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7840b2b80 0x7ff7840b3000 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00a860 tx=0x7ff77c001d80 comp rx=0 tx=0).stop 2023-12-01T03:12:56.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.484+0000 7ff78bb0a640 1 -- 172.21.15.115:0/2476387670 shutdown_connections 2023-12-01T03:12:56.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.484+0000 7ff78bb0a640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff76806a2d0 0x7ff76806c790 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.484+0000 7ff78bb0a640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7840c8430 0x7ff78415cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.484+0000 7ff78bb0a640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7840b2b80 0x7ff7840b3000 unknown :-1 s=CLOSED pgs=3030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.484+0000 7ff78bb0a640 1 --2- 172.21.15.115:0/2476387670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff770004d00 0x7ff78414fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.484+0000 7ff78bb0a640 1 -- 172.21.15.115:0/2476387670 >> 172.21.15.115:0/2476387670 conn(0x7ff7840662c0 msgr2=0x7ff7840707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:56.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.485+0000 7ff78bb0a640 1 -- 172.21.15.115:0/2476387670 shutdown_connections 2023-12-01T03:12:56.487 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.485+0000 7ff78bb0a640 1 -- 172.21.15.115:0/2476387670 wait complete. 2023-12-01T03:12:56.508 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-12-01T03:12:56.512 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-01-03_13_00 in ~4s... 2023-12-01T03:12:56.512 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-12-01T03:12:56.512 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-01T03:12:56.726 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.725+0000 7fcc83c5f640 1 Processor -- start 2023-12-01T03:12:56.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.725+0000 7fcc83c5f640 1 -- start start 2023-12-01T03:12:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.726+0000 7fcc83c5f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcc7c142870 0x7fcc7c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.726+0000 7fcc83c5f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7c0b2f30 con 0x7fcc7c0b1e20 2023-12-01T03:12:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.726+0000 7fcc83c5f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7c0b2320 con 0x7fcc7c0cd010 2023-12-01T03:12:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.726+0000 7fcc83c5f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7c0b3ae0 con 0x7fcc7c142870 2023-12-01T03:12:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.726+0000 7fcc821fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fcc7c0b1e20 0x7fcc7c15a0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40362/0 (socket says 172.21.15.115:40362) 2023-12-01T03:12:56.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.726+0000 7fcc821fc640 1 -- 172.21.15.115:0/3665416304 learned_addr learned my addr 172.21.15.115:0/3665416304 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:56.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc819fb640 1 --2- 172.21.15.115:0/3665416304 >> v2:172.21.15.142:3300/0 conn(0x7fcc7c142870 0x7fcc7c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753221280 0 0) 0x7fcc7c0b2f30 con 0x7fcc7c0b1e20 2023-12-01T03:12:56.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc68003520 con 0x7fcc7c0b1e20 2023-12-01T03:12:56.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1986496007 0 0) 0x7fcc68003520 con 0x7fcc7c0b1e20 2023-12-01T03:12:56.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 >> v2:172.21.15.142:3300/0 conn(0x7fcc7c142870 msgr2=0x7fcc7c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 --2- 172.21.15.115:0/3665416304 >> v2:172.21.15.142:3300/0 conn(0x7fcc7c142870 0x7fcc7c0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 >> v1:172.21.15.142:6789/0 conn(0x7fcc7c0cd010 legacy=0x7fcc7c0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0b2600 con 0x7fcc7c0b1e20 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fcc740019f0 con 0x7fcc7c0b1e20 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 >> v1:172.21.15.115:6789/0 conn(0x7fcc7c0b1e20 legacy=0x7fcc7c15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc680039c0 0x7fcc68003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.727+0000 7fcc809f9640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc809f9640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc68005180 0x7fcc6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c0b3ae0 con 0x7fcc680047c0 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6800c080 con 0x7fcc68005180 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6800e450 con 0x7fcc680039c0 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc811fa640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc819fb640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc680039c0 0x7fcc68003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc821fc640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc68005180 0x7fcc6800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc811fa640 1 -- 172.21.15.115:0/3665416304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc680039c0 msgr2=0x7fcc68003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:56.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc811fa640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc680039c0 0x7fcc68003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc811fa640 1 -- 172.21.15.115:0/3665416304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc68005180 msgr2=0x7fcc6800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc811fa640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc68005180 0x7fcc6800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc821fc640 1 -- 172.21.15.115:0/3665416304 reap_dead start 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.728+0000 7fcc811fa640 1 -- 172.21.15.115:0/3665416304 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68003520 con 0x7fcc680047c0 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.729+0000 7fcc811fa640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc68004c40 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c001750 tx=0x7fcc680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9ffe76c576e2d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.730+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0052d0 con 0x7fcc680047c0 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.730+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c02e050 con 0x7fcc680047c0 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.730+0000 7fcc809f9640 1 -- 172.21.15.115:0/3665416304 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c003830 con 0x7fcc680047c0 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.730+0000 7fcc83c5f640 1 -- 172.21.15.115:0/3665416304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 msgr2=0x7fcc68004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.730+0000 7fcc83c5f640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc68004c40 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c001750 tx=0x7fcc680057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.730+0000 7fcc83c5f640 1 -- 172.21.15.115:0/3665416304 shutdown_connections 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.730+0000 7fcc83c5f640 1 --2- 172.21.15.115:0/3665416304 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc68004c40 unknown :-1 s=CLOSED pgs=3032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.730+0000 7fcc83c5f640 1 -- 172.21.15.115:0/3665416304 >> 172.21.15.115:0/3665416304 conn(0x7fcc7c065960 msgr2=0x7fcc7c065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:56.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.730+0000 7fcc83c5f640 1 -- 172.21.15.115:0/3665416304 shutdown_connections 2023-12-01T03:12:56.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.730+0000 7fcc83c5f640 1 -- 172.21.15.115:0/3665416304 wait complete. 2023-12-01T03:12:56.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.732+0000 7fcc83c5f640 1 Processor -- start 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.732+0000 7fcc83c5f640 1 -- start start 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.733+0000 7fcc83c5f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc7c0abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.733+0000 7fcc83c5f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc7c13f200 0x7fcc7c13f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.733+0000 7fcc83c5f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc7c1ee3a0 0x7fcc7c1ee830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.733+0000 7fcc83c5f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c13fba0 con 0x7fcc680047c0 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.733+0000 7fcc83c5f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c1efc50 con 0x7fcc7c1ee3a0 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.733+0000 7fcc83c5f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c1eed70 con 0x7fcc7c13f200 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.733+0000 7fcc819fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc7c0abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.733+0000 7fcc819fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc7c0abeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34296/0 (socket says 172.21.15.115:34296) 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.733+0000 7fcc819fb640 1 -- 172.21.15.115:0/2305561220 learned_addr learned my addr 172.21.15.115:0/2305561220 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.734+0000 7fcc819fb640 1 -- 172.21.15.115:0/2305561220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc7c13f200 msgr2=0x7fcc7c13f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.734+0000 7fcc819fb640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc7c13f200 0x7fcc7c13f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.734+0000 7fcc819fb640 1 -- 172.21.15.115:0/2305561220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc7c1ee3a0 msgr2=0x7fcc7c1ee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.734+0000 7fcc819fb640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc7c1ee3a0 0x7fcc7c1ee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.734+0000 7fcc819fb640 1 -- 172.21.15.115:0/2305561220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c15db70 con 0x7fcc680047c0 2023-12-01T03:12:56.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.734+0000 7fcc819fb640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc7c0abeb0 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7fcc78010910 tx=0x7fcc78010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:56.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.734+0000 7fcc727fc640 1 -- 172.21.15.115:0/2305561220 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78009690 con 0x7fcc680047c0 2023-12-01T03:12:56.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.734+0000 7fcc727fc640 1 -- 172.21.15.115:0/2305561220 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7800ec20 con 0x7fcc680047c0 2023-12-01T03:12:56.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.734+0000 7fcc83c5f640 1 -- 172.21.15.115:0/2305561220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7c1eeff0 con 0x7fcc680047c0 2023-12-01T03:12:56.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.734+0000 7fcc727fc640 1 -- 172.21.15.115:0/2305561220 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7801c400 con 0x7fcc680047c0 2023-12-01T03:12:56.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.735+0000 7fcc83c5f640 1 -- 172.21.15.115:0/2305561220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c1ef8e0 con 0x7fcc680047c0 2023-12-01T03:12:56.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.736+0000 7fcc83c5f640 1 -- 172.21.15.115:0/2305561220 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c04e890 con 0x7fcc680047c0 2023-12-01T03:12:56.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.737+0000 7fcc727fc640 1 -- 172.21.15.115:0/2305561220 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 145) v1 ==== 88462+0+0 (secure 0 0 0) 0x7fcc7801c560 con 0x7fcc680047c0 2023-12-01T03:12:56.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.738+0000 7fcc727fc640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc6006a300 0x7fcc6006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:56.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.738+0000 7fcc727fc640 1 -- 172.21.15.115:0/2305561220 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fcc78088df0 con 0x7fcc680047c0 2023-12-01T03:12:56.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.738+0000 7fcc811fa640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc6006a300 0x7fcc6006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:56.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.739+0000 7fcc811fa640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc6006a300 0x7fcc6006c7c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c033000 tx=0x7fcc6c02f000 comp rx=0 tx=0).ready entity=mgr.12456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:56.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.742+0000 7fcc727fc640 1 -- 172.21.15.115:0/2305561220 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcc78054520 con 0x7fcc680047c0 2023-12-01T03:12:56.792 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.790+0000 7fcc727fc640 1 -- 172.21.15.115:0/2305561220 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 146) v1 ==== 88727+0+0 (secure 0 0 0) 0x7fcc780548f0 con 0x7fcc680047c0 2023-12-01T03:12:56.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.968+0000 7fcc83c5f640 1 -- 172.21.15.115:0/2305561220 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc7c0b28e0 con 0x7fcc6006a300 2023-12-01T03:12:56.983 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.981+0000 7fcc727fc640 1 -- 172.21.15.115:0/2305561220 <== mgr.12456 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7fcc6c005300 con 0x7fcc6006a300 2023-12-01T03:12:56.983 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:56.983 INFO:teuthology.orchestra.run.smithi115.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-01T03:12:56.985 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 -- 172.21.15.115:0/2305561220 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc6006a300 msgr2=0x7fcc6006c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc6006a300 0x7fcc6006c7c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c033000 tx=0x7fcc6c02f000 comp rx=0 tx=0).stop 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 -- 172.21.15.115:0/2305561220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 msgr2=0x7fcc7c0abeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc7c0abeb0 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7fcc78010910 tx=0x7fcc78010dd0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 -- 172.21.15.115:0/2305561220 shutdown_connections 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcc6006a300 0x7fcc6006c7c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc7c13f200 0x7fcc7c13f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc7c1ee3a0 0x7fcc7c1ee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 --2- 172.21.15.115:0/2305561220 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc680047c0 0x7fcc7c0abeb0 unknown :-1 s=CLOSED pgs=3033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 -- 172.21.15.115:0/2305561220 >> 172.21.15.115:0/2305561220 conn(0x7fcc7c065960 msgr2=0x7fcc7c13e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 -- 172.21.15.115:0/2305561220 shutdown_connections 2023-12-01T03:12:56.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:56.984+0000 7fcc83c5f640 1 -- 172.21.15.115:0/2305561220 wait complete. 2023-12-01T03:12:57.006 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-01T03:12:57.006 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-12-01T03:12:57.006 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-01T03:12:57.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.220+0000 7f437cfd9640 1 Processor -- start 2023-12-01T03:12:57.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.220+0000 7f437cfd9640 1 -- start start 2023-12-01T03:12:57.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.222+0000 7f437cfd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437815ac80 0x7f43780cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.222+0000 7f437cfd9640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43780cc9a0 con 0x7f43780ce1b0 2023-12-01T03:12:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.222+0000 7f437cfd9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43780cbd90 con 0x7f43780b0e60 2023-12-01T03:12:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.222+0000 7f437cfd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437815b060 con 0x7f437815ac80 2023-12-01T03:12:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.222+0000 7f4376d9d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f43780ce1b0 0x7f437806b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:40370/0 (socket says 172.21.15.115:40370) 2023-12-01T03:12:57.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.222+0000 7f4376d9d640 1 -- 172.21.15.115:0/3524932933 learned_addr learned my addr 172.21.15.115:0/3524932933 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.222+0000 7f437659c640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437815ac80 0x7f43780cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451733159 0 0) 0x7f43780cc9a0 con 0x7f43780ce1b0 2023-12-01T03:12:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4364003520 con 0x7f43780ce1b0 2023-12-01T03:12:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174967869 0 0) 0x7f4364003520 con 0x7f43780ce1b0 2023-12-01T03:12:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437815ac80 msgr2=0x7f43780cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437815ac80 0x7f43780cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 >> v1:172.21.15.142:6789/0 conn(0x7f43780b0e60 legacy=0x7f43780cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780cc070 con 0x7f43780ce1b0 2023-12-01T03:12:57.226 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f43700019f0 con 0x7f43780ce1b0 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f437659c640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437815ac80 0x7f43780cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 >> v1:172.21.15.115:6789/0 conn(0x7f43780ce1b0 legacy=0x7f437806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43640039a0 0x7f4364003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43640047c0 0x7f4364004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437815ac80 0x7f4364005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437815b060 con 0x7f43640047c0 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436400ba80 con 0x7f43640039a0 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436400bc40 con 0x7f437815ac80 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.223+0000 7f436ffff640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43640047c0 0x7f4364004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.224+0000 7f437659c640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43640039a0 0x7f4364003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.224+0000 7f436ffff640 1 -- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437815ac80 msgr2=0x7f4364005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.224+0000 7f436ffff640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f437815ac80 0x7f4364005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.224+0000 7f436ffff640 1 -- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43640039a0 msgr2=0x7f4364003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:57.227 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.224+0000 7f436ffff640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43640039a0 0x7f4364003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.224+0000 7f436ffff640 1 -- 172.21.15.115:0/3524932933 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364003520 con 0x7f43640047c0 2023-12-01T03:12:57.228 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.224+0000 7f437659c640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43640039a0 0x7f4364003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.225+0000 7f436ffff640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43640047c0 0x7f4364004c40 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7f4360001750 tx=0x7f43640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69041101701cdac server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.225+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43600052d0 con 0x7f43640047c0 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.225+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436002e050 con 0x7f43640047c0 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.225+0000 7f4375d9b640 1 -- 172.21.15.115:0/3524932933 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43600035f0 con 0x7f43640047c0 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.226+0000 7f437cfd9640 1 -- 172.21.15.115:0/3524932933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43640047c0 msgr2=0x7f4364004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.226+0000 7f437cfd9640 1 --2- 172.21.15.115:0/3524932933 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43640047c0 0x7f4364004c40 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7f4360001750 tx=0x7f43640057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.226+0000 7f4376d9d640 1 -- 172.21.15.115:0/3524932933 reap_dead start 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.226+0000 7f437cfd9640 1 -- 172.21.15.115:0/3524932933 shutdown_connections 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.226+0000 7f437cfd9640 1 -- 172.21.15.115:0/3524932933 >> 172.21.15.115:0/3524932933 conn(0x7f4378065960 msgr2=0x7f4378065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.226+0000 7f437cfd9640 1 -- 172.21.15.115:0/3524932933 shutdown_connections 2023-12-01T03:12:57.229 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.226+0000 7f437cfd9640 1 -- 172.21.15.115:0/3524932933 wait complete. 2023-12-01T03:12:57.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.228+0000 7f437cfd9640 1 Processor -- start 2023-12-01T03:12:57.230 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.228+0000 7f437cfd9640 1 -- start start 2023-12-01T03:12:57.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.229+0000 7f437cfd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43640047c0 0x7f43781430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.229+0000 7f437cfd9640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43781500c0 0x7f437815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.229+0000 7f437cfd9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f437815b6e0 0x7f437815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.229+0000 7f437cfd9640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4378159fd0 con 0x7f43781500c0 2023-12-01T03:12:57.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.229+0000 7f437cfd9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43781597e0 con 0x7f437815b6e0 2023-12-01T03:12:57.231 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.229+0000 7f437cfd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4378152160 con 0x7f43640047c0 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f436ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43781500c0 0x7f437815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f436ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43781500c0 0x7f437815b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:34348/0 (socket says 172.21.15.115:34348) 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f436ffff640 1 -- 172.21.15.115:0/475170793 learned_addr learned my addr 172.21.15.115:0/475170793 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f437659c640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43640047c0 0x7f43781430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f436ffff640 1 -- 172.21.15.115:0/475170793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43640047c0 msgr2=0x7f43781430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f4376d9d640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f437815b6e0 0x7f437815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f436ffff640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43640047c0 0x7f43781430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f436ffff640 1 -- 172.21.15.115:0/475170793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f437815b6e0 msgr2=0x7f437815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f436ffff640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f437815b6e0 0x7f437815bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f436ffff640 1 -- 172.21.15.115:0/475170793 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4378152400 con 0x7f43781500c0 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.230+0000 7f437659c640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43640047c0 0x7f43781430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:57.233 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.231+0000 7f436ffff640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43781500c0 0x7f437815b1a0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7f4360005040 tx=0x7f4360001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.231+0000 7f436effd640 1 -- 172.21.15.115:0/475170793 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43600076c0 con 0x7f43781500c0 2023-12-01T03:12:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.231+0000 7f436effd640 1 -- 172.21.15.115:0/475170793 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360007a30 con 0x7f43781500c0 2023-12-01T03:12:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.231+0000 7f437cfd9640 1 -- 172.21.15.115:0/475170793 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4378143620 con 0x7f43781500c0 2023-12-01T03:12:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.231+0000 7f436effd640 1 -- 172.21.15.115:0/475170793 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360006be0 con 0x7f43781500c0 2023-12-01T03:12:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.231+0000 7f437cfd9640 1 -- 172.21.15.115:0/475170793 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4378159cf0 con 0x7f43781500c0 2023-12-01T03:12:57.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.232+0000 7f437cfd9640 1 -- 172.21.15.115:0/475170793 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437804e890 con 0x7f43781500c0 2023-12-01T03:12:57.235 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.233+0000 7f436effd640 1 -- 172.21.15.115:0/475170793 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 146) v1 ==== 88727+0+0 (secure 0 0 0) 0x7f4360006d40 con 0x7f43781500c0 2023-12-01T03:12:57.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.234+0000 7f436effd640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f435806a5d0 0x7f435806ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.235+0000 7f436effd640 1 -- 172.21.15.115:0/475170793 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f43600aab10 con 0x7f43781500c0 2023-12-01T03:12:57.236 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.235+0000 7f437659c640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f435806a5d0 0x7f435806ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.237 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.235+0000 7f437659c640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f435806a5d0 0x7f435806ca90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f436800ba20 tx=0x7f4368008040 comp rx=0 tx=0).ready entity=mgr.12456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:57.240 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.239+0000 7f436effd640 1 -- 172.21.15.115:0/475170793 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f4360075cf0 con 0x7f43781500c0 2023-12-01T03:12:57.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.454+0000 7f437cfd9640 1 -- 172.21.15.115:0/475170793 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f43780cc630 con 0x7f435806a5d0 2023-12-01T03:12:57.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.592+0000 7f436effd640 1 -- 172.21.15.115:0/475170793 <== mgr.12456 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f436800c040 con 0x7f435806a5d0 2023-12-01T03:12:57.594 INFO:teuthology.orchestra.run.smithi115.stdout:Schedule removed for path snap_test_dir1 2023-12-01T03:12:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.595+0000 7f437cfd9640 1 -- 172.21.15.115:0/475170793 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f435806a5d0 msgr2=0x7f435806ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f435806a5d0 0x7f435806ca90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f436800ba20 tx=0x7f4368008040 comp rx=0 tx=0).stop 2023-12-01T03:12:57.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 -- 172.21.15.115:0/475170793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43781500c0 msgr2=0x7f437815b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43781500c0 0x7f437815b1a0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7f4360005040 tx=0x7f4360001120 comp rx=0 tx=0).stop 2023-12-01T03:12:57.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 -- 172.21.15.115:0/475170793 shutdown_connections 2023-12-01T03:12:57.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f435806a5d0 0x7f435806ca90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f437815b6e0 0x7f437815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43781500c0 0x7f437815b1a0 unknown :-1 s=CLOSED pgs=3038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 --2- 172.21.15.115:0/475170793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43640047c0 0x7f43781430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 -- 172.21.15.115:0/475170793 >> 172.21.15.115:0/475170793 conn(0x7f4378065960 msgr2=0x7f437806c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:57.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 -- 172.21.15.115:0/475170793 shutdown_connections 2023-12-01T03:12:57.598 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.596+0000 7f437cfd9640 1 -- 172.21.15.115:0/475170793 wait complete. 2023-12-01T03:12:57.618 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-01T03:12:57.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:57.618 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-01T03:12:57.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:12:57.677 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-01T03:12:57.738 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-01T03:12:57.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.948+0000 7fb358dbf640 1 Processor -- start 2023-12-01T03:12:57.950 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.949+0000 7fb358dbf640 1 -- start start 2023-12-01T03:12:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.950+0000 7fb358dbf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3540addc0 0x7fb35415f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.950+0000 7fb358dbf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb35415bdf0 0x7fb3540cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.950+0000 7fb358dbf640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35415b040 con 0x7fb3540c97a0 2023-12-01T03:12:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.950+0000 7fb358dbf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35415f5d0 con 0x7fb35415bdf0 2023-12-01T03:12:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.950+0000 7fb358dbf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540ca020 con 0x7fb3540addc0 2023-12-01T03:12:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.950+0000 7fb353fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fb3540c97a0 0x7fb35406b370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37148/0 (socket says 172.21.15.115:37148) 2023-12-01T03:12:57.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.950+0000 7fb353fff640 1 -- 172.21.15.115:0/1503423031 learned_addr learned my addr 172.21.15.115:0/1503423031 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb34bfff640 1 --2- 172.21.15.115:0/1503423031 >> v2:172.21.15.142:3300/0 conn(0x7fb35415bdf0 0x7fb3540cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177574629 0 0) 0x7fb35415b040 con 0x7fb3540c97a0 2023-12-01T03:12:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb340003520 con 0x7fb3540c97a0 2023-12-01T03:12:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb3537fe640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3540addc0 0x7fb35415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2133961607 0 0) 0x7fb340003520 con 0x7fb3540c97a0 2023-12-01T03:12:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3540addc0 msgr2=0x7fb35415f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3540addc0 0x7fb35415f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.955 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 >> v2:172.21.15.142:3300/0 conn(0x7fb35415bdf0 msgr2=0x7fb3540cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 --2- 172.21.15.115:0/1503423031 >> v2:172.21.15.142:3300/0 conn(0x7fb35415bdf0 0x7fb3540cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3540ae1a0 con 0x7fb3540c97a0 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fb34c0019f0 con 0x7fb3540c97a0 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb34bfff640 1 --2- 172.21.15.115:0/1503423031 >> v2:172.21.15.142:3300/0 conn(0x7fb35415bdf0 0x7fb3540cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 >> v1:172.21.15.115:6789/0 conn(0x7fb3540c97a0 legacy=0x7fb35406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb340003970 0x7fb340003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3540addc0 0x7fb3400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 0x7fb340005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35415f5d0 con 0x7fb340004d00 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540ca020 con 0x7fb340003970 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.951+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34000ba30 con 0x7fb3540addc0 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.952+0000 7fb353fff640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 0x7fb340005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.952+0000 7fb3537fe640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb340003970 0x7fb340003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.952+0000 7fb353fff640 1 -- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3540addc0 msgr2=0x7fb3400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:57.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.952+0000 7fb34bfff640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3540addc0 0x7fb3400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.952+0000 7fb353fff640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3540addc0 0x7fb3400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.952+0000 7fb353fff640 1 -- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb340003970 msgr2=0x7fb340003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.952+0000 7fb353fff640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb340003970 0x7fb340003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.952+0000 7fb353fff640 1 -- 172.21.15.115:0/1503423031 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb340003520 con 0x7fb340004d00 2023-12-01T03:12:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.952+0000 7fb34bfff640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3540addc0 0x7fb3400047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:12:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.953+0000 7fb353fff640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 0x7fb340005180 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00a890 tx=0x7fb3400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebc3baad4f58773 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.953+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c034630 con 0x7fb340004d00 2023-12-01T03:12:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.953+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c02f070 con 0x7fb340004d00 2023-12-01T03:12:57.957 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.953+0000 7fb352ffd640 1 -- 172.21.15.115:0/1503423031 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0349f0 con 0x7fb340004d00 2023-12-01T03:12:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.954+0000 7fb358dbf640 1 -- 172.21.15.115:0/1503423031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 msgr2=0x7fb340005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.954+0000 7fb358dbf640 1 --2- 172.21.15.115:0/1503423031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 0x7fb340005180 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00a890 tx=0x7fb3400057f0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.954+0000 7fb353fff640 1 -- 172.21.15.115:0/1503423031 reap_dead start 2023-12-01T03:12:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.954+0000 7fb358dbf640 1 -- 172.21.15.115:0/1503423031 shutdown_connections 2023-12-01T03:12:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.954+0000 7fb358dbf640 1 -- 172.21.15.115:0/1503423031 >> 172.21.15.115:0/1503423031 conn(0x7fb354065960 msgr2=0x7fb354065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.954+0000 7fb358dbf640 1 -- 172.21.15.115:0/1503423031 shutdown_connections 2023-12-01T03:12:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.954+0000 7fb358dbf640 1 -- 172.21.15.115:0/1503423031 wait complete. 2023-12-01T03:12:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.956+0000 7fb358dbf640 1 Processor -- start 2023-12-01T03:12:57.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.956+0000 7fb358dbf640 1 -- start start 2023-12-01T03:12:57.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.957+0000 7fb358dbf640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 0x7fb35414f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.957+0000 7fb358dbf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb35406f6b0 0x7fb35406fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.957+0000 7fb358dbf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3540b3c20 0x7fb3540b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.957+0000 7fb358dbf640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540b4610 con 0x7fb340004d00 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.957+0000 7fb358dbf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540c35e0 con 0x7fb3540b3c20 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.957+0000 7fb358dbf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35414f940 con 0x7fb35406f6b0 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.957+0000 7fb3537fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 0x7fb35414f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.957+0000 7fb3537fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 0x7fb35414f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50512/0 (socket says 172.21.15.115:50512) 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.957+0000 7fb3537fe640 1 -- 172.21.15.115:0/1003674727 learned_addr learned my addr 172.21.15.115:0/1003674727 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb3537fe640 1 -- 172.21.15.115:0/1003674727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb35406f6b0 msgr2=0x7fb35406fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb3537fe640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb35406f6b0 0x7fb35406fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb3537fe640 1 -- 172.21.15.115:0/1003674727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3540b3c20 msgr2=0x7fb3540b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb3537fe640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3540b3c20 0x7fb3540b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb3537fe640 1 -- 172.21.15.115:0/1003674727 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3540c37f0 con 0x7fb340004d00 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb3537fe640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 0x7fb35414f400 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7fb34400f580 tx=0x7fb34400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:57.961 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb350ff9640 1 -- 172.21.15.115:0/1003674727 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb344013810 con 0x7fb340004d00 2023-12-01T03:12:57.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb350ff9640 1 -- 172.21.15.115:0/1003674727 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb344013970 con 0x7fb340004d00 2023-12-01T03:12:57.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb350ff9640 1 -- 172.21.15.115:0/1003674727 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb344013ce0 con 0x7fb340004d00 2023-12-01T03:12:57.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb358dbf640 1 -- 172.21.15.115:0/1003674727 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3540c3ad0 con 0x7fb340004d00 2023-12-01T03:12:57.962 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.958+0000 7fb358dbf640 1 -- 172.21.15.115:0/1003674727 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3540c4090 con 0x7fb340004d00 2023-12-01T03:12:57.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.960+0000 7fb358dbf640 1 -- 172.21.15.115:0/1003674727 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3540c4250 con 0x7fb340004d00 2023-12-01T03:12:57.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.960+0000 7fb350ff9640 1 -- 172.21.15.115:0/1003674727 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 146) v1 ==== 88727+0+0 (secure 0 0 0) 0x7fb34401f460 con 0x7fb340004d00 2023-12-01T03:12:57.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.961+0000 7fb350ff9640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb33406a5d0 0x7fb33406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:57.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.961+0000 7fb350ff9640 1 -- 172.21.15.115:0/1003674727 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fb344088fe0 con 0x7fb340004d00 2023-12-01T03:12:57.963 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.962+0000 7fb34bfff640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb33406a5d0 0x7fb33406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:57.964 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.962+0000 7fb34bfff640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb33406a5d0 0x7fb33406ca90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb33c006e80 tx=0x7fb33c006df0 comp rx=0 tx=0).ready entity=mgr.12456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:57.967 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:57.966+0000 7fb350ff9640 1 -- 172.21.15.115:0/1003674727 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fb3440542f0 con 0x7fb340004d00 2023-12-01T03:12:58.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.207+0000 7fb358dbf640 1 -- 172.21.15.115:0/1003674727 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb35415b2c0 con 0x7fb340004d00 2023-12-01T03:12:58.814 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.812+0000 7fb350ff9640 1 -- 172.21.15.115:0/1003674727 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 147) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fb3440546c0 con 0x7fb340004d00 2023-12-01T03:12:58.815 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.813+0000 7fb350ff9640 1 -- 172.21.15.115:0/1003674727 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v147) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb3440542f0 con 0x7fb340004d00 2023-12-01T03:12:58.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.816+0000 7fb358dbf640 1 -- 172.21.15.115:0/1003674727 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb33406a5d0 msgr2=0x7fb33406ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:58.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.816+0000 7fb358dbf640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb33406a5d0 0x7fb33406ca90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb33c006e80 tx=0x7fb33c006df0 comp rx=0 tx=0).stop 2023-12-01T03:12:58.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.816+0000 7fb358dbf640 1 -- 172.21.15.115:0/1003674727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 msgr2=0x7fb35414f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:58.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.816+0000 7fb358dbf640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 0x7fb35414f400 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7fb34400f580 tx=0x7fb34400fa40 comp rx=0 tx=0).stop 2023-12-01T03:12:58.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.817+0000 7fb358dbf640 1 -- 172.21.15.115:0/1003674727 shutdown_connections 2023-12-01T03:12:58.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.817+0000 7fb358dbf640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fb33406a5d0 0x7fb33406ca90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:58.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.817+0000 7fb358dbf640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb35406f6b0 0x7fb35406fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:58.819 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.817+0000 7fb358dbf640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3540b3c20 0x7fb3540b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:58.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.817+0000 7fb358dbf640 1 --2- 172.21.15.115:0/1003674727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb340004d00 0x7fb35414f400 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:58.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.817+0000 7fb358dbf640 1 -- 172.21.15.115:0/1003674727 >> 172.21.15.115:0/1003674727 conn(0x7fb354065960 msgr2=0x7fb3540cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:58.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.817+0000 7fb358dbf640 1 -- 172.21.15.115:0/1003674727 shutdown_connections 2023-12-01T03:12:58.820 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:58.817+0000 7fb358dbf640 1 -- 172.21.15.115:0/1003674727 wait complete. 2023-12-01T03:12:58.841 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-01T03:12:58.841 DEBUG:teuthology.orchestra.run.smithi115:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:12:58.950 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-01T03:12:59.049 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-01T03:12:59.049 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:12:59.049 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:59.104 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:12:59.104 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:12:59.198 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:12:59.326 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:59.324+0000 7f64f4841040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:12:59.337 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:59.336+0000 7ffbace32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T03:12:59.385 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:59.383+0000 7f64f4841040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:12:59.395 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:59.394+0000 7ffbace32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T03:12:59.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.396+0000 7f76ef9fa640 1 Processor -- start 2023-12-01T03:12:59.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.396+0000 7f76ef9fa640 1 -- start start 2023-12-01T03:12:59.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.397+0000 7f76ef9fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e815cd20 0x7f76e80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.397+0000 7f76ef9fa640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f76e80d1230 0x7f76e80d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.397+0000 7f76ef9fa640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e80ce2e0 con 0x7f76e80b5d80 2023-12-01T03:12:59.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.397+0000 7f76ef9fa640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e80b2160 con 0x7f76e80d1230 2023-12-01T03:12:59.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.397+0000 7f76ef9fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80d1be0 con 0x7f76e815cd20 2023-12-01T03:12:59.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.398+0000 7f76edf97640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f76e80d1230 0x7f76e80d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:59.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.398+0000 7f76edf97640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f76e80d1230 0x7f76e80d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50526/0 (socket says 172.21.15.115:50526) 2023-12-01T03:12:59.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.398+0000 7f76edf97640 1 -- 172.21.15.115:0/2625116850 learned_addr learned my addr 172.21.15.115:0/2625116850 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:59.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.398+0000 7f76ed796640 1 --2- 172.21.15.115:0/2625116850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e815cd20 0x7f76e80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:59.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.398+0000 7f76edf97640 1 -- 172.21.15.115:0/2625116850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e815cd20 msgr2=0x7f76e80d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:59.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.398+0000 7f76edf97640 1 --2- 172.21.15.115:0/2625116850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e815cd20 0x7f76e80d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.398+0000 7f76edf97640 1 -- 172.21.15.115:0/2625116850 >> v1:172.21.15.142:6789/0 conn(0x7f76e80b5d80 legacy=0x7f76e80b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:12:59.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.398+0000 7f76edf97640 1 -- 172.21.15.115:0/2625116850 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e80ce000 con 0x7f76e80d1230 2023-12-01T03:12:59.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.398+0000 7f76ed796640 1 --2- 172.21.15.115:0/2625116850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e815cd20 0x7f76e80d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:12:59.401 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.400+0000 7f76edf97640 1 --2- 172.21.15.115:0/2625116850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 0x7f76e80d16a0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009e50 tx=0x7f76e002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ae8dfc0d2008 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:59.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.400+0000 7f76dffff640 1 -- 172.21.15.115:0/2625116850 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e002c9e0 con 0x7f76e80d1230 2023-12-01T03:12:59.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.400+0000 7f76dffff640 1 -- 172.21.15.115:0/2625116850 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e002cb40 con 0x7f76e80d1230 2023-12-01T03:12:59.402 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.400+0000 7f76dffff640 1 -- 172.21.15.115:0/2625116850 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e00343f0 con 0x7f76e80d1230 2023-12-01T03:12:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.401+0000 7f76ef9fa640 1 -- 172.21.15.115:0/2625116850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 msgr2=0x7f76e80d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.401+0000 7f76ef9fa640 1 --2- 172.21.15.115:0/2625116850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 0x7f76e80d16a0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009e50 tx=0x7f76e002be40 comp rx=0 tx=0).stop 2023-12-01T03:12:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.401+0000 7f76ef9fa640 1 -- 172.21.15.115:0/2625116850 shutdown_connections 2023-12-01T03:12:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.401+0000 7f76ef9fa640 1 --2- 172.21.15.115:0/2625116850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 0x7f76e80d16a0 unknown :-1 s=CLOSED pgs=3045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.401+0000 7f76ef9fa640 1 --2- 172.21.15.115:0/2625116850 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e815cd20 0x7f76e80d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.401+0000 7f76ef9fa640 1 -- 172.21.15.115:0/2625116850 >> 172.21.15.115:0/2625116850 conn(0x7f76e806a0e0 msgr2=0x7f76e806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.401+0000 7f76ef9fa640 1 -- 172.21.15.115:0/2625116850 shutdown_connections 2023-12-01T03:12:59.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.401+0000 7f76ef9fa640 1 -- 172.21.15.115:0/2625116850 wait complete. 2023-12-01T03:12:59.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.402+0000 7f76ef9fa640 1 Processor -- start 2023-12-01T03:12:59.404 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.403+0000 7f76ef9fa640 1 -- start start 2023-12-01T03:12:59.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ef9fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e80b5d80 0x7f76e8153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ef9fa640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 0x7f76e80c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ef9fa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e815cd20 0x7f76e81f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ef9fa640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80c7760 con 0x7f76e80d1230 2023-12-01T03:12:59.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ef9fa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80c6d90 con 0x7f76e815cd20 2023-12-01T03:12:59.407 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ef9fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80c6780 con 0x7f76e80b5d80 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ecf95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 0x7f76e80c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ecf95640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 0x7f76e80c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50532/0 (socket says 172.21.15.115:50532) 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ecf95640 1 -- 172.21.15.115:0/1525204689 learned_addr learned my addr 172.21.15.115:0/1525204689 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ed796640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e80b5d80 0x7f76e8153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76edf97640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e815cd20 0x7f76e81f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ecf95640 1 -- 172.21.15.115:0/1525204689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e80b5d80 msgr2=0x7f76e8153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ecf95640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e80b5d80 0x7f76e8153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ecf95640 1 -- 172.21.15.115:0/1525204689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e815cd20 msgr2=0x7f76e81f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ecf95640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e815cd20 0x7f76e81f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ecf95640 1 -- 172.21.15.115:0/1525204689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0009b30 con 0x7f76e80d1230 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.404+0000 7f76ed796640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e80b5d80 0x7f76e8153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.405+0000 7f76ecf95640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 0x7f76e80c7220 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7f76d800c400 tx=0x7f76d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:59.408 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.405+0000 7f76ddffb640 1 -- 172.21.15.115:0/1525204689 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8011790 con 0x7f76e80d1230 2023-12-01T03:12:59.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.405+0000 7f76ddffb640 1 -- 172.21.15.115:0/1525204689 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8011930 con 0x7f76e80d1230 2023-12-01T03:12:59.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.405+0000 7f76ddffb640 1 -- 172.21.15.115:0/1525204689 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8011ce0 con 0x7f76e80d1230 2023-12-01T03:12:59.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.405+0000 7f76ef9fa640 1 -- 172.21.15.115:0/1525204689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e81f2ff0 con 0x7f76e80d1230 2023-12-01T03:12:59.409 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.405+0000 7f76ef9fa640 1 -- 172.21.15.115:0/1525204689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e80c6a20 con 0x7f76e80d1230 2023-12-01T03:12:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.406+0000 7f76ef9fa640 1 -- 172.21.15.115:0/1525204689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e80c6f40 con 0x7f76e80d1230 2023-12-01T03:12:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.407+0000 7f76ddffb640 1 -- 172.21.15.115:0/1525204689 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 147) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f76d801c460 con 0x7f76e80d1230 2023-12-01T03:12:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.408+0000 7f76ddffb640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f76cc06a610 0x7f76cc06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.408+0000 7f76ddffb640 1 -- 172.21.15.115:0/1525204689 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f76d8088d00 con 0x7f76e80d1230 2023-12-01T03:12:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.408+0000 7f76ed796640 1 -- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f76cc06a610 msgr2=0x7f76cc06cad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:59.410 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.408+0000 7f76ed796640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f76cc06a610 0x7f76cc06cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:59.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.416+0000 7f76ddffb640 1 -- 172.21.15.115:0/1525204689 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f76e80c6f40 con 0x7f76e80d1230 2023-12-01T03:12:59.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.608+0000 7f76ef9fa640 1 -- 172.21.15.115:0/1525204689 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76e80c6f00 con 0x7f76e80d1230 2023-12-01T03:12:59.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.609+0000 7f76ed796640 1 -- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f76cc06a610 msgr2=0x7f76cc06cad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:59.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.609+0000 7f76ed796640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f76cc06a610 0x7f76cc06cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:12:59.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.610+0000 7f76ddffb640 1 -- 172.21.15.115:0/1525204689 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) v1 ==== 74+0+16799 (secure 0 0 0) 0x7f76d8053e90 con 0x7f76e80d1230 2023-12-01T03:12:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:12:59.613 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":257,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:54.714825+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:12:33.411870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-01T03:12:34.335941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/138932":"2023-12-02T03:12:54.714747+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2457056129":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2634160496":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2258893627":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2547954996":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2121389099":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/3223789397":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/534295842":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2854101523":"2023-12-02T03:12:54.714747+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 -- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f76cc06a610 msgr2=0x7f76cc06cad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f76cc06a610 0x7f76cc06cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 -- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 msgr2=0x7f76e80c7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 0x7f76e80c7220 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 crypto rx=0x7f76d800c400 tx=0x7f76d800c8c0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 -- 172.21.15.115:0/1525204689 shutdown_connections 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f76cc06a610 0x7f76cc06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76e815cd20 0x7f76e81f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76e80d1230 0x7f76e80c7220 unknown :-1 s=CLOSED pgs=3046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 --2- 172.21.15.115:0/1525204689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76e80b5d80 0x7f76e8153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 -- 172.21.15.115:0/1525204689 >> 172.21.15.115:0/1525204689 conn(0x7f76e806a0e0 msgr2=0x7f76e8157590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 -- 172.21.15.115:0/1525204689 shutdown_connections 2023-12-01T03:12:59.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.613+0000 7f76ef9fa640 1 -- 172.21.15.115:0/1525204689 wait complete. 2023-12-01T03:12:59.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T03:12:33.411870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '255', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T03:12:34.335941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '256', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:12:59.633 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:12:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.827+0000 7f756dad5640 1 Processor -- start 2023-12-01T03:12:59.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.827+0000 7f756dad5640 1 -- start start 2023-12-01T03:12:59.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.829+0000 7f756dad5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75680ce160 0x7f7568154010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.829+0000 7f756dad5640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75680cdcd0 con 0x7f7568153c30 2023-12-01T03:12:59.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.829+0000 7f756dad5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7568161830 con 0x7f75680cfb80 2023-12-01T03:12:59.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.829+0000 7f756dad5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75680cd3a0 con 0x7f75680ce160 2023-12-01T03:12:59.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.829+0000 7f756cad3640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f7568153c30 0x7f75680cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37172/0 (socket says 172.21.15.115:37172) 2023-12-01T03:12:59.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.829+0000 7f756cad3640 1 -- 172.21.15.115:0/1364852952 learned_addr learned my addr 172.21.15.115:0/1364852952 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:12:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.829+0000 7f756d2d4640 1 --2- 172.21.15.115:0/1364852952 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75680ce160 0x7f7568154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.830+0000 7f75677fe640 1 -- 172.21.15.115:0/1364852952 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859825981 0 0) 0x7f75680cdcd0 con 0x7f7568153c30 2023-12-01T03:12:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.830+0000 7f75677fe640 1 -- 172.21.15.115:0/1364852952 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7558003520 con 0x7f7568153c30 2023-12-01T03:12:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.830+0000 7f756d2d4640 1 -- 172.21.15.115:0/1364852952 >> v1:172.21.15.142:6789/0 conn(0x7f75680cfb80 legacy=0x7f7568160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.830+0000 7f756d2d4640 1 -- 172.21.15.115:0/1364852952 >> v1:172.21.15.115:6789/0 conn(0x7f7568153c30 legacy=0x7f75680cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:59.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.830+0000 7f756d2d4640 1 -- 172.21.15.115:0/1364852952 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680cd680 con 0x7f75680ce160 2023-12-01T03:12:59.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.831+0000 7f756d2d4640 1 --2- 172.21.15.115:0/1364852952 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75680ce160 0x7f7568154010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7560009dc0 tx=0x7f756002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f89103140ebfbf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:59.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.832+0000 7f75677fe640 1 -- 172.21.15.115:0/1364852952 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756002ccd0 con 0x7f75680ce160 2023-12-01T03:12:59.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.832+0000 7f75677fe640 1 -- 172.21.15.115:0/1364852952 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756002ce30 con 0x7f75680ce160 2023-12-01T03:12:59.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.832+0000 7f75677fe640 1 -- 172.21.15.115:0/1364852952 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560034640 con 0x7f75680ce160 2023-12-01T03:12:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.832+0000 7f756dad5640 1 -- 172.21.15.115:0/1364852952 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75680ce160 msgr2=0x7f7568154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.832+0000 7f756dad5640 1 --2- 172.21.15.115:0/1364852952 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75680ce160 0x7f7568154010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7560009dc0 tx=0x7f756002bf20 comp rx=0 tx=0).stop 2023-12-01T03:12:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.832+0000 7f756dad5640 1 -- 172.21.15.115:0/1364852952 shutdown_connections 2023-12-01T03:12:59.834 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.832+0000 7f756dad5640 1 --2- 172.21.15.115:0/1364852952 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75680ce160 0x7f7568154010 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.832+0000 7f756dad5640 1 -- 172.21.15.115:0/1364852952 >> 172.21.15.115:0/1364852952 conn(0x7f756806a0b0 msgr2=0x7f756806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:12:59.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.833+0000 7f756dad5640 1 -- 172.21.15.115:0/1364852952 shutdown_connections 2023-12-01T03:12:59.835 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.833+0000 7f756dad5640 1 -- 172.21.15.115:0/1364852952 wait complete. 2023-12-01T03:12:59.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.834+0000 7f756dad5640 1 Processor -- start 2023-12-01T03:12:59.836 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.835+0000 7f756dad5640 1 -- start start 2023-12-01T03:12:59.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f756dad5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75680ce160 0x7f75681f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f756dad5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75680cfb80 0x7f75681f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f756dad5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7568153c30 0x7f75680c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f756dad5640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75681f32a0 con 0x7f75680cfb80 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f756dad5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75680ca150 con 0x7f75680ce160 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f756dad5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75680ca3f0 con 0x7f7568153c30 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f7567fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75680cfb80 0x7f75681f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f7567fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75680cfb80 0x7f75681f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50544/0 (socket says 172.21.15.115:50544) 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f7567fff640 1 -- 172.21.15.115:0/72137368 learned_addr learned my addr 172.21.15.115:0/72137368 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f756d2d4640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7568153c30 0x7f75680c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f756cad3640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75680ce160 0x7f75681f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f7567fff640 1 -- 172.21.15.115:0/72137368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7568153c30 msgr2=0x7f75680c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f7567fff640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7568153c30 0x7f75680c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f7567fff640 1 -- 172.21.15.115:0/72137368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75680ce160 msgr2=0x7f75681f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f7567fff640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75680ce160 0x7f75681f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.836+0000 7f7567fff640 1 -- 172.21.15.115:0/72137368 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7560009a70 con 0x7f75680cfb80 2023-12-01T03:12:59.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.837+0000 7f7567fff640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75680cfb80 0x7f75681f2d60 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f755400c440 tx=0x7f7554001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:12:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.837+0000 7f75657fa640 1 -- 172.21.15.115:0/72137368 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75540052d0 con 0x7f75680cfb80 2023-12-01T03:12:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.837+0000 7f75657fa640 1 -- 172.21.15.115:0/72137368 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7554005430 con 0x7f75680cfb80 2023-12-01T03:12:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.837+0000 7f75657fa640 1 -- 172.21.15.115:0/72137368 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554007260 con 0x7f75680cfb80 2023-12-01T03:12:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.837+0000 7f756dad5640 1 -- 172.21.15.115:0/72137368 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75680ca6d0 con 0x7f75680cfb80 2023-12-01T03:12:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.837+0000 7f756dad5640 1 -- 172.21.15.115:0/72137368 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75680cac90 con 0x7f75680cfb80 2023-12-01T03:12:59.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.838+0000 7f756dad5640 1 -- 172.21.15.115:0/72137368 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7568074210 con 0x7f75680cfb80 2023-12-01T03:12:59.841 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.839+0000 7f75657fa640 1 -- 172.21.15.115:0/72137368 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 147) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f75540073c0 con 0x7f75680cfb80 2023-12-01T03:12:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.840+0000 7f75657fa640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f754c06a5e0 0x7f754c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:12:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.840+0000 7f75657fa640 1 -- 172.21.15.115:0/72137368 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f7554088230 con 0x7f75680cfb80 2023-12-01T03:12:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.840+0000 7f756cad3640 1 -- 172.21.15.115:0/72137368 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f754c06a5e0 msgr2=0x7f754c06caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:12:59.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.840+0000 7f756cad3640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f754c06a5e0 0x7f754c06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:12:59.845 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:59.844+0000 7ffbace32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:12:59.846 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:12:59.845+0000 7f75657fa640 1 -- 172.21.15.115:0/72137368 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f75540533f0 con 0x7f75680cfb80 2023-12-01T03:12:59.860 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:59.859+0000 7f64f4841040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T03:12:59.897 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:59.897+0000 7ffbace32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:12:59.915 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:59.913+0000 7f64f4841040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T03:12:59.948 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:12:59.946+0000 7ffbace32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:12:59.966 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:12:59.964+0000 7f64f4841040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T03:13:00.008 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:00.007+0000 7ffbace32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:13:00.029 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:00.027+0000 7f64f4841040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T03:13:00.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.031+0000 7f756dad5640 1 -- 172.21.15.115:0/72137368 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75680cd930 con 0x7f75680cfb80 2023-12-01T03:13:00.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.032+0000 7f75657fa640 1 -- 172.21.15.115:0/72137368 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v257) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7554056a10 con 0x7f75680cfb80 2023-12-01T03:13:00.034 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.034+0000 7f756dad5640 1 -- 172.21.15.115:0/72137368 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f754c06a5e0 msgr2=0x7f754c06caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.034+0000 7f756dad5640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f754c06a5e0 0x7f754c06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.034+0000 7f756dad5640 1 -- 172.21.15.115:0/72137368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75680cfb80 msgr2=0x7f75681f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.034+0000 7f756dad5640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75680cfb80 0x7f75681f2d60 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f755400c440 tx=0x7f7554001e70 comp rx=0 tx=0).stop 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.035+0000 7f756dad5640 1 -- 172.21.15.115:0/72137368 shutdown_connections 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.035+0000 7f756dad5640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f754c06a5e0 0x7f754c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.035+0000 7f756dad5640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7568153c30 0x7f75680c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.035+0000 7f756dad5640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75680cfb80 0x7f75681f2d60 unknown :-1 s=CLOSED pgs=3048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.035+0000 7f756dad5640 1 --2- 172.21.15.115:0/72137368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75680ce160 0x7f75681f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.035+0000 7f756dad5640 1 -- 172.21.15.115:0/72137368 >> 172.21.15.115:0/72137368 conn(0x7f756806a0b0 msgr2=0x7f756815e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.035+0000 7f756dad5640 1 -- 172.21.15.115:0/72137368 shutdown_connections 2023-12-01T03:13:00.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.035+0000 7f756dad5640 1 -- 172.21.15.115:0/72137368 wait complete. 2023-12-01T03:13:00.055 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:13:00.060 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:00.058+0000 7ffbace32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:13:00.081 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:00.079+0000 7f64f4841040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T03:13:00.170 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:00.169+0000 7ffbace32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:13:00.198 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:00.196+0000 7f64f4841040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T03:13:00.232 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:00.231+0000 7ffbace32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:13:00.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.255+0000 7f2aa24c7640 1 Processor -- start 2023-12-01T03:13:00.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.256+0000 7f2aa24c7640 1 -- start start 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2aa24c7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a9c15cd90 0x7f2a9c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2aa24c7640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2a9c0b5fa0 0x7f2a9c161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2aa24c7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9c0cf650 0x7f2a9c0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2aa24c7640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c0d0ec0 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2aa24c7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c0ce5e0 con 0x7f2a9c15cd90 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2aa24c7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c161950 con 0x7f2a9c0cf650 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2a9b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2a9c0b5fa0 0x7f2a9c161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2a9b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2a9c0b5fa0 0x7f2a9c161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50546/0 (socket says 172.21.15.115:50546) 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2a9b7fe640 1 -- 172.21.15.115:0/1505303378 learned_addr learned my addr 172.21.15.115:0/1505303378 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2aa0a64640 1 --2- 172.21.15.115:0/1505303378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9c0cf650 0x7f2a9c0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.257+0000 7f2a9bfff640 1 --2- 172.21.15.115:0/1505303378 >> v2:172.21.15.142:3300/0 conn(0x7f2a9c15cd90 0x7f2a9c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.258+0000 7f2a9b7fe640 1 -- 172.21.15.115:0/1505303378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9c0cf650 msgr2=0x7f2a9c0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:00.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.258+0000 7f2a9b7fe640 1 --2- 172.21.15.115:0/1505303378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9c0cf650 0x7f2a9c0cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.258+0000 7f2a9b7fe640 1 -- 172.21.15.115:0/1505303378 >> v2:172.21.15.142:3300/0 conn(0x7f2a9c15cd90 msgr2=0x7f2a9c0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:00.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.258+0000 7f2a9b7fe640 1 --2- 172.21.15.115:0/1505303378 >> v2:172.21.15.142:3300/0 conn(0x7f2a9c15cd90 0x7f2a9c0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.258+0000 7f2a9b7fe640 1 -- 172.21.15.115:0/1505303378 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c0cffc0 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.258+0000 7f2a9bfff640 1 --2- 172.21.15.115:0/1505303378 >> v2:172.21.15.142:3300/0 conn(0x7f2a9c15cd90 0x7f2a9c0cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:13:00.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.259+0000 7f2a9b7fe640 1 --2- 172.21.15.115:0/1505303378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 0x7f2a9c161410 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c06bd10 tx=0x7f2a9002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362629932e4f39fb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:00.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.259+0000 7f2a9affd640 1 -- 172.21.15.115:0/1505303378 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a90031870 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.259+0000 7f2a9affd640 1 -- 172.21.15.115:0/1505303378 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a900319d0 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.259+0000 7f2a9affd640 1 -- 172.21.15.115:0/1505303378 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a90031d60 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.260+0000 7f2aa24c7640 1 -- 172.21.15.115:0/1505303378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 msgr2=0x7f2a9c161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.260+0000 7f2aa24c7640 1 --2- 172.21.15.115:0/1505303378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 0x7f2a9c161410 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c06bd10 tx=0x7f2a9002b910 comp rx=0 tx=0).stop 2023-12-01T03:13:00.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.260+0000 7f2aa24c7640 1 -- 172.21.15.115:0/1505303378 shutdown_connections 2023-12-01T03:13:00.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.260+0000 7f2aa24c7640 1 --2- 172.21.15.115:0/1505303378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9c0cf650 0x7f2a9c0cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.260+0000 7f2aa24c7640 1 --2- 172.21.15.115:0/1505303378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 0x7f2a9c161410 unknown :-1 s=CLOSED pgs=3049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.260+0000 7f2aa24c7640 1 --2- 172.21.15.115:0/1505303378 >> v2:172.21.15.142:3300/0 conn(0x7f2a9c15cd90 0x7f2a9c0cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.260+0000 7f2aa24c7640 1 -- 172.21.15.115:0/1505303378 >> 172.21.15.115:0/1505303378 conn(0x7f2a9c065960 msgr2=0x7f2a9c065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:00.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.260+0000 7f2aa24c7640 1 -- 172.21.15.115:0/1505303378 shutdown_connections 2023-12-01T03:13:00.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.260+0000 7f2aa24c7640 1 -- 172.21.15.115:0/1505303378 wait complete. 2023-12-01T03:13:00.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.261+0000 7f2aa24c7640 1 Processor -- start 2023-12-01T03:13:00.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.262+0000 7f2aa24c7640 1 -- start start 2023-12-01T03:13:00.263 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:00.262+0000 7f64f4841040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T03:13:00.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2aa24c7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 0x7f2a9c15f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2aa24c7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c0cf650 0x7f2a9c153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2aa24c7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9c15cd90 0x7f2a9c15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2aa24c7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c1f39c0 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2aa24c7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c1f3610 con 0x7f2a9c0cf650 2023-12-01T03:13:00.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2aa24c7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c15dd00 con 0x7f2a9c15cd90 2023-12-01T03:13:00.265 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 0x7f2a9c15f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 0x7f2a9c15f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50548/0 (socket says 172.21.15.115:50548) 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2a9bfff640 1 -- 172.21.15.115:0/765550832 learned_addr learned my addr 172.21.15.115:0/765550832 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2aa0a64640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9c15cd90 0x7f2a9c15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2a9b7fe640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c0cf650 0x7f2a9c153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2a9bfff640 1 -- 172.21.15.115:0/765550832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9c15cd90 msgr2=0x7f2a9c15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2a9bfff640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9c15cd90 0x7f2a9c15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2a9bfff640 1 -- 172.21.15.115:0/765550832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c0cf650 msgr2=0x7f2a9c153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2a9bfff640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c0cf650 0x7f2a9c153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2a9bfff640 1 -- 172.21.15.115:0/765550832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a900094d0 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.263+0000 7f2a9b7fe640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c0cf650 0x7f2a9c153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:00.266 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.264+0000 7f2a9bfff640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 0x7f2a9c15f7f0 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00d7d0 tx=0x7f2a8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:00.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.264+0000 7f2a98ff9640 1 -- 172.21.15.115:0/765550832 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c004030 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.264+0000 7f2a98ff9640 1 -- 172.21.15.115:0/765550832 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c004190 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.264+0000 7f2a98ff9640 1 -- 172.21.15.115:0/765550832 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c004520 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.264+0000 7f2aa24c7640 1 -- 172.21.15.115:0/765550832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c15dfe0 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.267 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.264+0000 7f2aa24c7640 1 -- 172.21.15.115:0/765550832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c0b6610 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.265+0000 7f2aa24c7640 1 -- 172.21.15.115:0/765550832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c1f3c30 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.266+0000 7f2a98ff9640 1 -- 172.21.15.115:0/765550832 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 147) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f2a8c0096a0 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.267+0000 7f2a98ff9640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a8806a610 0x7f2a8806cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.267+0000 7f2a98ff9640 1 -- 172.21.15.115:0/765550832 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f2a8c088eb0 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.267+0000 7f2a9b7fe640 1 -- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a8806a610 msgr2=0x7f2a8806cad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:13:00.269 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.267+0000 7f2a9b7fe640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a8806a610 0x7f2a8806cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:13:00.273 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.271+0000 7f2a98ff9640 1 -- 172.21.15.115:0/765550832 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f2a8c0540d0 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.283 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:00.282+0000 7ffbace32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:13:00.317 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:00.315+0000 7f64f4841040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T03:13:00.341 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:00.340+0000 7ffbace32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:13:00.376 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:00.375+0000 7f64f4841040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T03:13:00.392 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:00.391+0000 7ffbace32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:13:00.428 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:00.426+0000 7f64f4841040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T03:13:00.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.466+0000 7f2aa24c7640 1 -- 172.21.15.115:0/765550832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a9c0d1060 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.467+0000 7f2a98ff9640 1 -- 172.21.15.115:0/765550832 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v257) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a8c0576f0 con 0x7f2a9c0b5fa0 2023-12-01T03:13:00.469 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:13:00.469 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.468+0000 7f2a9b7fe640 1 -- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a8806a610 msgr2=0x7f2a8806cad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:13:00.470 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.468+0000 7f2a9b7fe640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a8806a610 0x7f2a8806cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 -- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a8806a610 msgr2=0x7f2a8806cad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a8806a610 0x7f2a8806cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 -- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 msgr2=0x7f2a9c15f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 0x7f2a9c15f7f0 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00d7d0 tx=0x7f2a8c00dc90 comp rx=0 tx=0).stop 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 -- 172.21.15.115:0/765550832 shutdown_connections 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f2a8806a610 0x7f2a8806cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a9c15cd90 0x7f2a9c15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a9c0cf650 0x7f2a9c153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 --2- 172.21.15.115:0/765550832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a9c0b5fa0 0x7f2a9c15f7f0 unknown :-1 s=CLOSED pgs=3050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 -- 172.21.15.115:0/765550832 >> 172.21.15.115:0/765550832 conn(0x7f2a9c065960 msgr2=0x7f2a9c15a410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 -- 172.21.15.115:0/765550832 shutdown_connections 2023-12-01T03:13:00.472 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.470+0000 7f2aa24c7640 1 -- 172.21.15.115:0/765550832 wait complete. 2023-12-01T03:13:00.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:13:00.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.738+0000 7f74c593c640 1 Processor -- start 2023-12-01T03:13:00.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.738+0000 7f74c593c640 1 -- start start 2023-12-01T03:13:00.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.739+0000 7f74c593c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f74c006c200 0x7f74c00cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.739+0000 7f74c593c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c00cac30 con 0x7f74c00b1710 2023-12-01T03:13:00.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.739+0000 7f74c593c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c00b2140 con 0x7f74c01590b0 2023-12-01T03:13:00.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.739+0000 7f74c593c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c00b2420 con 0x7f74c006c200 2023-12-01T03:13:00.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74b7fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f74c00b1710 0x7f74c006baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37186/0 (socket says 172.21.15.115:37186) 2023-12-01T03:13:00.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74b7fff640 1 -- 172.21.15.115:0/3025489214 learned_addr learned my addr 172.21.15.115:0/3025489214 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74bf7fe640 1 --2- 172.21.15.115:0/3025489214 >> v2:172.21.15.142:3300/0 conn(0x7f74c006c200 0x7f74c00cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484859678 0 0) 0x7f74c00cac30 con 0x7f74c00b1710 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ac003520 con 0x7f74c00b1710 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1329581339 0 0) 0x7f74ac003520 con 0x7f74c00b1710 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 >> v2:172.21.15.142:3300/0 conn(0x7f74c006c200 msgr2=0x7f74c00cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74be7fc640 1 --2- 172.21.15.115:0/3025489214 >> v2:172.21.15.142:3300/0 conn(0x7f74c006c200 0x7f74c00cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 >> v1:172.21.15.142:6789/0 conn(0x7f74c01590b0 legacy=0x7f74c0159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c00caed0 con 0x7f74c00b1710 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f74a8001670 con 0x7f74c00b1710 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 >> v1:172.21.15.115:6789/0 conn(0x7f74c00b1710 legacy=0x7f74c006baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.740+0000 7f74be7fc640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac003970 0x7f74ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74be7fc640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0047c0 0x7f74ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74be7fc640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac005180 0x7f74ac00ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c00b2420 con 0x7f74ac003970 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac00d1e0 con 0x7f74ac0047c0 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac0059d0 con 0x7f74ac005180 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74bf7fe640 1 --2- 172.21.15.115:0/3025489214 >> v2:172.21.15.142:3300/0 conn(0x7f74c006c200 0x7f74c00cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74beffd640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac003970 0x7f74ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74b7fff640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0047c0 0x7f74ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74bf7fe640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac005180 0x7f74ac00ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74beffd640 1 -- 172.21.15.115:0/3025489214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac005180 msgr2=0x7f74ac00ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74beffd640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74ac005180 0x7f74ac00ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74beffd640 1 -- 172.21.15.115:0/3025489214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0047c0 msgr2=0x7f74ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74beffd640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac0047c0 0x7f74ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74beffd640 1 -- 172.21.15.115:0/3025489214 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac003520 con 0x7f74ac003970 2023-12-01T03:13:00.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.741+0000 7f74bf7fe640 1 -- 172.21.15.115:0/3025489214 reap_dead start 2023-12-01T03:13:00.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.742+0000 7f74beffd640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac003970 0x7f74ac003d90 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7f74b0001750 tx=0x7f74ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fa4355b2cdec2a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.743+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b00052d0 con 0x7f74ac003970 2023-12-01T03:13:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.743+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b002e050 con 0x7f74ac003970 2023-12-01T03:13:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.743+0000 7f74be7fc640 1 -- 172.21.15.115:0/3025489214 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0003830 con 0x7f74ac003970 2023-12-01T03:13:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.743+0000 7f74c593c640 1 -- 172.21.15.115:0/3025489214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac003970 msgr2=0x7f74ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.743+0000 7f74c593c640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac003970 0x7f74ac003d90 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7f74b0001750 tx=0x7f74ac0057f0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.743+0000 7f74c593c640 1 -- 172.21.15.115:0/3025489214 shutdown_connections 2023-12-01T03:13:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.743+0000 7f74c593c640 1 --2- 172.21.15.115:0/3025489214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74ac003970 0x7f74ac003d90 unknown :-1 s=CLOSED pgs=3052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.743+0000 7f74c593c640 1 -- 172.21.15.115:0/3025489214 >> 172.21.15.115:0/3025489214 conn(0x7f74c0065960 msgr2=0x7f74c0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.743+0000 7f74c593c640 1 -- 172.21.15.115:0/3025489214 shutdown_connections 2023-12-01T03:13:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.743+0000 7f74c593c640 1 -- 172.21.15.115:0/3025489214 wait complete. 2023-12-01T03:13:00.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.745+0000 7f74c593c640 1 Processor -- start 2023-12-01T03:13:00.748 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.745+0000 7f74c593c640 1 -- start start 2023-12-01T03:13:00.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.746+0000 7f74c593c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac003970 0x7f74c01555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.746+0000 7f74c593c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74c0147670 0x7f74c014b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.746+0000 7f74c593c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c014b6e0 0x7f74c0148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.746+0000 7f74c593c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c014c1e0 con 0x7f74c014b6e0 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.746+0000 7f74c593c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0154130 con 0x7f74ac003970 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.746+0000 7f74c593c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0154410 con 0x7f74c0147670 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.746+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c014b6e0 0x7f74c0148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.746+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c014b6e0 0x7f74c0148040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50566/0 (socket says 172.21.15.115:50566) 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.746+0000 7f74bf7fe640 1 -- 172.21.15.115:0/1801351056 learned_addr learned my addr 172.21.15.115:0/1801351056 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74b7fff640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74c0147670 0x7f74c014b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74bf7fe640 1 -- 172.21.15.115:0/1801351056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74c0147670 msgr2=0x7f74c014b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74bf7fe640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74c0147670 0x7f74c014b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.750 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74bf7fe640 1 -- 172.21.15.115:0/1801351056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac003970 msgr2=0x7f74c01555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74bf7fe640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac003970 0x7f74c01555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74bf7fe640 1 -- 172.21.15.115:0/1801351056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c014be70 con 0x7f74c014b6e0 2023-12-01T03:13:00.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74b7fff640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74c0147670 0x7f74c014b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:13:00.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74bf7fe640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c014b6e0 0x7f74c0148040 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7f74c00ca920 tx=0x7f74b8004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:00.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74bcff9640 1 -- 172.21.15.115:0/1801351056 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8007db0 con 0x7f74c014b6e0 2023-12-01T03:13:00.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74bcff9640 1 -- 172.21.15.115:0/1801351056 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b800f050 con 0x7f74c014b6e0 2023-12-01T03:13:00.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74bcff9640 1 -- 172.21.15.115:0/1801351056 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8014630 con 0x7f74c014b6e0 2023-12-01T03:13:00.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.747+0000 7f74c593c640 1 -- 172.21.15.115:0/1801351056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c0155af0 con 0x7f74c014b6e0 2023-12-01T03:13:00.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.748+0000 7f74c593c640 1 -- 172.21.15.115:0/1801351056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c014c670 con 0x7f74c014b6e0 2023-12-01T03:13:00.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.749+0000 7f74c593c640 1 -- 172.21.15.115:0/1801351056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c004e890 con 0x7f74c014b6e0 2023-12-01T03:13:00.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.750+0000 7f74bcff9640 1 -- 172.21.15.115:0/1801351056 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 147) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f74b8014790 con 0x7f74c014b6e0 2023-12-01T03:13:00.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.751+0000 7f74bcff9640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74a006a610 0x7f74a006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:00.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.751+0000 7f74bcff9640 1 -- 172.21.15.115:0/1801351056 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f74b8089660 con 0x7f74c014b6e0 2023-12-01T03:13:00.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.751+0000 7f74beffd640 1 -- 172.21.15.115:0/1801351056 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74a006a610 msgr2=0x7f74a006cad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:13:00.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.751+0000 7f74beffd640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74a006a610 0x7f74a006cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:13:00.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.756+0000 7f74bcff9640 1 -- 172.21.15.115:0/1801351056 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f74b80546e0 con 0x7f74c014b6e0 2023-12-01T03:13:00.773 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:00.772+0000 7ffbace32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:13:00.829 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:00.827+0000 7f64f4841040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T03:13:00.932 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:00.931+0000 7ffbace32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:13:00.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.939+0000 7f74c593c640 1 -- 172.21.15.115:0/1801351056 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74c00b26d0 con 0x7f74c014b6e0 2023-12-01T03:13:00.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.940+0000 7f74bcff9640 1 -- 172.21.15.115:0/1801351056 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v257) v1 ==== 100+0+11 (secure 0 0 0) 0x7f74b8057d00 con 0x7f74c014b6e0 2023-12-01T03:13:00.942 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.943+0000 7f74c593c640 1 -- 172.21.15.115:0/1801351056 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74a006a610 msgr2=0x7f74a006cad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.943+0000 7f74c593c640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74a006a610 0x7f74a006cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.943+0000 7f74c593c640 1 -- 172.21.15.115:0/1801351056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c014b6e0 msgr2=0x7f74c0148040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.943+0000 7f74c593c640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c014b6e0 0x7f74c0148040 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7f74c00ca920 tx=0x7f74b8004840 comp rx=0 tx=0).stop 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.943+0000 7f74c593c640 1 -- 172.21.15.115:0/1801351056 shutdown_connections 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.944+0000 7f74c593c640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f74a006a610 0x7f74a006cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.944+0000 7f74c593c640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f74c0147670 0x7f74c014b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.944+0000 7f74c593c640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74c014b6e0 0x7f74c0148040 unknown :-1 s=CLOSED pgs=3053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.944+0000 7f74c593c640 1 --2- 172.21.15.115:0/1801351056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac003970 0x7f74c01555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.944+0000 7f74c593c640 1 -- 172.21.15.115:0/1801351056 >> 172.21.15.115:0/1801351056 conn(0x7f74c0065960 msgr2=0x7f74c01521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.944+0000 7f74c593c640 1 -- 172.21.15.115:0/1801351056 shutdown_connections 2023-12-01T03:13:00.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:00.944+0000 7f74c593c640 1 -- 172.21.15.115:0/1801351056 wait complete. 2023-12-01T03:13:00.963 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T03:13:00.998 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:00.994+0000 7f64f4841040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T03:13:01.040 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:01.039+0000 7ffbace32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:13:01.090 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:01.088+0000 7ffbace32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:13:01.109 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:01.107+0000 7f64f4841040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T03:13:01.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.158+0000 7fcf1ad70640 1 Processor -- start 2023-12-01T03:13:01.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.158+0000 7fcf1ad70640 1 -- start start 2023-12-01T03:13:01.161 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:01.160+0000 7f64f4841040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T03:13:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.160+0000 7fcf1ad70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140701b0 0x7fcf14070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.160+0000 7fcf1ad70640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1406da60 con 0x7fcf14159af0 2023-12-01T03:13:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.160+0000 7fcf1ad70640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf14070ad0 con 0x7fcf140cac50 2023-12-01T03:13:01.162 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.160+0000 7fcf1ad70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140cd9c0 con 0x7fcf140701b0 2023-12-01T03:13:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.160+0000 7fcf1956d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fcf14159af0 0x7fcf140cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37196/0 (socket says 172.21.15.115:37196) 2023-12-01T03:13:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.160+0000 7fcf1956d640 1 -- 172.21.15.115:0/2859792797 learned_addr learned my addr 172.21.15.115:0/2859792797 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.160+0000 7fcf19d6e640 1 --2- 172.21.15.115:0/2859792797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140701b0 0x7fcf14070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.160+0000 7fcf18d6c640 1 -- 172.21.15.115:0/2859792797 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377797302 0 0) 0x7fcf1406da60 con 0x7fcf14159af0 2023-12-01T03:13:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.161+0000 7fcf18d6c640 1 -- 172.21.15.115:0/2859792797 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf00005730 con 0x7fcf14159af0 2023-12-01T03:13:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.161+0000 7fcf19d6e640 1 -- 172.21.15.115:0/2859792797 >> v1:172.21.15.142:6789/0 conn(0x7fcf140cac50 legacy=0x7fcf1406def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.161+0000 7fcf19d6e640 1 -- 172.21.15.115:0/2859792797 >> v1:172.21.15.115:6789/0 conn(0x7fcf14159af0 legacy=0x7fcf140cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:01.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.161+0000 7fcf19d6e640 1 -- 172.21.15.115:0/2859792797 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1406ce50 con 0x7fcf140701b0 2023-12-01T03:13:01.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.162+0000 7fcf19d6e640 1 --2- 172.21.15.115:0/2859792797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140701b0 0x7fcf14070590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcf100097d0 tx=0x7fcf1002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce81c58a19b22a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:01.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.162+0000 7fcf18d6c640 1 -- 172.21.15.115:0/2859792797 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10031890 con 0x7fcf140701b0 2023-12-01T03:13:01.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.162+0000 7fcf18d6c640 1 -- 172.21.15.115:0/2859792797 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf100319f0 con 0x7fcf140701b0 2023-12-01T03:13:01.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.162+0000 7fcf18d6c640 1 -- 172.21.15.115:0/2859792797 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10031d60 con 0x7fcf140701b0 2023-12-01T03:13:01.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.163+0000 7fcf1ad70640 1 -- 172.21.15.115:0/2859792797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140701b0 msgr2=0x7fcf14070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:01.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.163+0000 7fcf1ad70640 1 --2- 172.21.15.115:0/2859792797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140701b0 0x7fcf14070590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcf100097d0 tx=0x7fcf1002b930 comp rx=0 tx=0).stop 2023-12-01T03:13:01.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.163+0000 7fcf1ad70640 1 -- 172.21.15.115:0/2859792797 shutdown_connections 2023-12-01T03:13:01.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.163+0000 7fcf1ad70640 1 --2- 172.21.15.115:0/2859792797 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140701b0 0x7fcf14070590 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.163+0000 7fcf1ad70640 1 -- 172.21.15.115:0/2859792797 >> 172.21.15.115:0/2859792797 conn(0x7fcf140cc160 msgr2=0x7fcf140cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:01.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.163+0000 7fcf1ad70640 1 -- 172.21.15.115:0/2859792797 shutdown_connections 2023-12-01T03:13:01.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.163+0000 7fcf1ad70640 1 -- 172.21.15.115:0/2859792797 wait complete. 2023-12-01T03:13:01.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.165+0000 7fcf1ad70640 1 Processor -- start 2023-12-01T03:13:01.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.165+0000 7fcf1ad70640 1 -- start start 2023-12-01T03:13:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1ad70640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf140701b0 0x7fcf14153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1ad70640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140cac50 0x7fcf1415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1ad70640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf14159af0 0x7fcf140b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1ad70640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1415b5b0 con 0x7fcf14159af0 2023-12-01T03:13:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1ad70640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140b4a20 con 0x7fcf140701b0 2023-12-01T03:13:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1ad70640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140b4740 con 0x7fcf140cac50 2023-12-01T03:13:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1a56f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf14159af0 0x7fcf140b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1a56f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf14159af0 0x7fcf140b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50580/0 (socket says 172.21.15.115:50580) 2023-12-01T03:13:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1a56f640 1 -- 172.21.15.115:0/594873616 learned_addr learned my addr 172.21.15.115:0/594873616 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1956d640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140cac50 0x7fcf1415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf19d6e640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf140701b0 0x7fcf14153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1a56f640 1 -- 172.21.15.115:0/594873616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140cac50 msgr2=0x7fcf1415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1a56f640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140cac50 0x7fcf1415b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.166+0000 7fcf1a56f640 1 -- 172.21.15.115:0/594873616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf140701b0 msgr2=0x7fcf14153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.167+0000 7fcf1a56f640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf140701b0 0x7fcf14153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.167+0000 7fcf1a56f640 1 -- 172.21.15.115:0/594873616 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10009400 con 0x7fcf14159af0 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.167+0000 7fcf1a56f640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf14159af0 0x7fcf140b3c80 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c7d0 tx=0x7fcf0c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.167+0000 7fcf0a7fc640 1 -- 172.21.15.115:0/594873616 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c003b70 con 0x7fcf14159af0 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.167+0000 7fcf0a7fc640 1 -- 172.21.15.115:0/594873616 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c003cf0 con 0x7fcf14159af0 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.167+0000 7fcf0a7fc640 1 -- 172.21.15.115:0/594873616 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00aa90 con 0x7fcf14159af0 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.167+0000 7fcf1ad70640 1 -- 172.21.15.115:0/594873616 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1415aef0 con 0x7fcf14159af0 2023-12-01T03:13:01.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.167+0000 7fcf1ad70640 1 -- 172.21.15.115:0/594873616 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf14154200 con 0x7fcf14159af0 2023-12-01T03:13:01.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.169+0000 7fcf1ad70640 1 -- 172.21.15.115:0/594873616 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1415b6f0 con 0x7fcf14159af0 2023-12-01T03:13:01.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.169+0000 7fcf0a7fc640 1 -- 172.21.15.115:0/594873616 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 147) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fcf0c00abf0 con 0x7fcf14159af0 2023-12-01T03:13:01.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.170+0000 7fcf0a7fc640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcef806a5e0 0x7fcef806caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.170+0000 7fcf19d6e640 1 -- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcef806a5e0 msgr2=0x7fcef806caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:13:01.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.170+0000 7fcf0a7fc640 1 -- 172.21.15.115:0/594873616 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fcf0c088af0 con 0x7fcf14159af0 2023-12-01T03:13:01.172 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.170+0000 7fcf19d6e640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcef806a5e0 0x7fcef806caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:13:01.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.174+0000 7fcf0a7fc640 1 -- 172.21.15.115:0/594873616 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fcf1415b6f0 con 0x7fcf14159af0 2023-12-01T03:13:01.202 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:01.197+0000 7ffbace32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:13:01.275 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:01.273+0000 7f64f4841040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:13:01.306 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:01.304+0000 7ffbace32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:13:01.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.371+0000 7fcf19d6e640 1 -- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcef806a5e0 msgr2=0x7fcef806caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:13:01.373 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.371+0000 7fcf19d6e640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcef806a5e0 0x7fcef806caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:13:01.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.377+0000 7fcf1ad70640 1 -- 172.21.15.115:0/594873616 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcf1415b6f0 con 0x7fcf14159af0 2023-12-01T03:13:01.379 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.378+0000 7fcf0a7fc640 1 -- 172.21.15.115:0/594873616 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v257) v1 ==== 81+0+587 (secure 0 0 0) 0x7fcf1415b6f0 con 0x7fcf14159af0 2023-12-01T03:13:01.380 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6824/138932 2023-12-02T03:12:54.714747+0000 2023-12-01T03:13:01.380 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:6822/138932 2023-12-02T03:12:54.714747+0000 2023-12-01T03:13:01.380 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2457056129 2023-12-02T03:12:54.714747+0000 2023-12-01T03:13:01.380 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2634160496 2023-12-02T03:12:54.714747+0000 2023-12-01T03:13:01.380 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2258893627 2023-12-02T03:12:54.714747+0000 2023-12-01T03:13:01.380 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2547954996 2023-12-02T03:12:54.714747+0000 2023-12-01T03:13:01.380 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2121389099 2023-12-02T03:12:54.714747+0000 2023-12-01T03:13:01.380 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/3223789397 2023-12-02T03:12:54.714747+0000 2023-12-01T03:13:01.380 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/534295842 2023-12-02T03:12:54.714747+0000 2023-12-01T03:13:01.380 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115:0/2854101523 2023-12-02T03:12:54.714747+0000 2023-12-01T03:13:01.381 INFO:teuthology.orchestra.run.smithi115.stderr:listed 10 entries 2023-12-01T03:13:01.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.380+0000 7fcf1ad70640 1 -- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcef806a5e0 msgr2=0x7fcef806caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:13:01.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.380+0000 7fcf1ad70640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcef806a5e0 0x7fcef806caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.382 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.380+0000 7fcf1ad70640 1 -- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf14159af0 msgr2=0x7fcf140b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:01.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.380+0000 7fcf1ad70640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf14159af0 0x7fcf140b3c80 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c7d0 tx=0x7fcf0c00cc90 comp rx=0 tx=0).stop 2023-12-01T03:13:01.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.380+0000 7fcf1ad70640 1 -- 172.21.15.115:0/594873616 shutdown_connections 2023-12-01T03:13:01.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.380+0000 7fcf1ad70640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fcef806a5e0 0x7fcef806caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.380+0000 7fcf1ad70640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf140cac50 0x7fcf1415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.380+0000 7fcf1ad70640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf14159af0 0x7fcf140b3c80 unknown :-1 s=CLOSED pgs=3055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.380+0000 7fcf1ad70640 1 --2- 172.21.15.115:0/594873616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf140701b0 0x7fcf14153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.380+0000 7fcf1ad70640 1 -- 172.21.15.115:0/594873616 >> 172.21.15.115:0/594873616 conn(0x7fcf140cc160 msgr2=0x7fcf140ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:01.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.381+0000 7fcf1ad70640 1 -- 172.21.15.115:0/594873616 shutdown_connections 2023-12-01T03:13:01.383 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.381+0000 7fcf1ad70640 1 -- 172.21.15.115:0/594873616 wait complete. 2023-12-01T03:13:01.386 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:01.385+0000 7f64f4841040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T03:13:01.399 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T03:13:01.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:01.400 DEBUG:teuthology.orchestra.run.smithi115:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T03:13:01.462 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:01.460+0000 7ffbace32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:13:01.511 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:01.509+0000 7ffbace32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:13:01.538 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T03:13:01.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:01.538 DEBUG:teuthology.orchestra.run.smithi115:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T03:13:01.553 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:01.551+0000 7f64f4841040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T03:13:01.574 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:13:01.578 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:13:01.583 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:13:01.583 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:13:01.606 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:01.605+0000 7f64f4841040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T03:13:01.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:13:01.627 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:13:01.627 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:13:01.642 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:01.640+0000 7ffbace32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:13:01.676 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:13:01.676 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:13:01.676 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:13:01.723 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:13:01.723 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:13:01.738 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:01.732+0000 7f64f4841040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T03:13:01.936 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.934+0000 7f28ee911640 1 Processor -- start 2023-12-01T03:13:01.937 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.935+0000 7f28ee911640 1 -- start start 2023-12-01T03:13:01.938 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.936+0000 7f28ee911640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28e815db90 0x7f28e80cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.936+0000 7f28ee911640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f28e80cddc0 0x7f28e80d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.936+0000 7f28ee911640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e80cd3f0 con 0x7f28e815d7b0 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.936+0000 7f28ee911640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e815dff0 con 0x7f28e80cddc0 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.936+0000 7f28ee911640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80cd0f0 con 0x7f28e815db90 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.936+0000 7f28ed90f640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f28e815d7b0 0x7f28e8160670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37212/0 (socket says 172.21.15.115:37212) 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.936+0000 7f28ed90f640 1 -- 172.21.15.115:0/753097166 learned_addr learned my addr 172.21.15.115:0/753097166 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.936+0000 7f28ee110640 1 --2- 172.21.15.115:0/753097166 >> v2:172.21.15.115:3300/0 conn(0x7f28e80cddc0 0x7f28e80d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.937+0000 7f28ed10e640 1 --2- 172.21.15.115:0/753097166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28e815db90 0x7f28e80cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.937+0000 7f28ee110640 1 -- 172.21.15.115:0/753097166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28e815db90 msgr2=0x7f28e80cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.937+0000 7f28ee110640 1 --2- 172.21.15.115:0/753097166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28e815db90 0x7f28e80cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.937+0000 7f28ee110640 1 -- 172.21.15.115:0/753097166 >> v1:172.21.15.115:6789/0 conn(0x7f28e815d7b0 legacy=0x7f28e8160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.937+0000 7f28ee110640 1 -- 172.21.15.115:0/753097166 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e8160ec0 con 0x7f28e80cddc0 2023-12-01T03:13:01.939 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.937+0000 7f28ec90d640 1 -- 172.21.15.115:0/753097166 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499429738 0 0) 0x7f28e80cd3f0 con 0x7f28e815d7b0 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.938+0000 7f28ee110640 1 --2- 172.21.15.115:0/753097166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 0x7f28e80d05d0 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009d90 tx=0x7f28e002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef461e730bdd6dd0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.938+0000 7f28ec90d640 1 -- 172.21.15.115:0/753097166 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e002cce0 con 0x7f28e80cddc0 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.938+0000 7f28ec90d640 1 -- 172.21.15.115:0/753097166 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e002ce40 con 0x7f28e80cddc0 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.938+0000 7f28ec90d640 1 -- 172.21.15.115:0/753097166 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0034660 con 0x7f28e80cddc0 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.939+0000 7f28ee911640 1 -- 172.21.15.115:0/753097166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 msgr2=0x7f28e80d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.939+0000 7f28ee911640 1 --2- 172.21.15.115:0/753097166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 0x7f28e80d05d0 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009d90 tx=0x7f28e002bf30 comp rx=0 tx=0).stop 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.939+0000 7f28ee911640 1 -- 172.21.15.115:0/753097166 shutdown_connections 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.939+0000 7f28ee911640 1 --2- 172.21.15.115:0/753097166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 0x7f28e80d05d0 unknown :-1 s=CLOSED pgs=3057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.939+0000 7f28ee911640 1 --2- 172.21.15.115:0/753097166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28e815db90 0x7f28e80cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.939+0000 7f28ee911640 1 -- 172.21.15.115:0/753097166 >> 172.21.15.115:0/753097166 conn(0x7f28e806a0e0 msgr2=0x7f28e806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:01.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.939+0000 7f28ee911640 1 -- 172.21.15.115:0/753097166 shutdown_connections 2023-12-01T03:13:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.939+0000 7f28ee911640 1 -- 172.21.15.115:0/753097166 wait complete. 2023-12-01T03:13:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.940+0000 7f28ee911640 1 Processor -- start 2023-12-01T03:13:01.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.941+0000 7f28ee911640 1 -- start start 2023-12-01T03:13:01.944 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.941+0000 7f28ee911640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 0x7f28e81f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.941+0000 7f28ee911640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28e815d7b0 0x7f28e815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ee911640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e815db90 0x7f28e80b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ee911640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80b6a10 con 0x7f28e80cddc0 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ee911640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e81f36c0 con 0x7f28e815db90 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ee911640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80cc850 con 0x7f28e815d7b0 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ed90f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 0x7f28e81f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ed90f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 0x7f28e81f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50594/0 (socket says 172.21.15.115:50594) 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ed90f640 1 -- 172.21.15.115:0/8570832 learned_addr learned my addr 172.21.15.115:0/8570832 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ed10e640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28e815d7b0 0x7f28e815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ed90f640 1 -- 172.21.15.115:0/8570832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28e815d7b0 msgr2=0x7f28e815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ed90f640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28e815d7b0 0x7f28e815be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ed90f640 1 -- 172.21.15.115:0/8570832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e815db90 msgr2=0x7f28e80b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ee110640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e815db90 0x7f28e80b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:01.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ed90f640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e815db90 0x7f28e80b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:01.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ed90f640 1 -- 172.21.15.115:0/8570832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0009a70 con 0x7f28e80cddc0 2023-12-01T03:13:01.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.942+0000 7f28ee110640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e815db90 0x7f28e80b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:01.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.943+0000 7f28ed90f640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 0x7f28e81f3180 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c4d0 tx=0x7f28e400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:01.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.943+0000 7f28de7fc640 1 -- 172.21.15.115:0/8570832 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e40074e0 con 0x7f28e80cddc0 2023-12-01T03:13:01.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.943+0000 7f28de7fc640 1 -- 172.21.15.115:0/8570832 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e4007660 con 0x7f28e80cddc0 2023-12-01T03:13:01.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.943+0000 7f28de7fc640 1 -- 172.21.15.115:0/8570832 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4004a20 con 0x7f28e80cddc0 2023-12-01T03:13:01.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.943+0000 7f28ee911640 1 -- 172.21.15.115:0/8570832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e80b5e00 con 0x7f28e80cddc0 2023-12-01T03:13:01.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.943+0000 7f28ee911640 1 -- 172.21.15.115:0/8570832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e80b63c0 con 0x7f28e80cddc0 2023-12-01T03:13:01.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.944+0000 7f28ee911640 1 -- 172.21.15.115:0/8570832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e80d0ac0 con 0x7f28e80cddc0 2023-12-01T03:13:01.947 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.945+0000 7f28de7fc640 1 -- 172.21.15.115:0/8570832 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 147) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f28e4004b80 con 0x7f28e80cddc0 2023-12-01T03:13:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.946+0000 7f28de7fc640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f28cc06a5e0 0x7f28cc06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.947+0000 7f28ed10e640 1 -- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f28cc06a5e0 msgr2=0x7f28cc06caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:13:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.947+0000 7f28de7fc640 1 -- 172.21.15.115:0/8570832 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f28e40880c0 con 0x7f28e80cddc0 2023-12-01T03:13:01.948 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.947+0000 7f28ed10e640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f28cc06a5e0 0x7f28cc06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:13:01.952 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:01.951+0000 7f28de7fc640 1 -- 172.21.15.115:0/8570832 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f28e40531d0 con 0x7f28e80cddc0 2023-12-01T03:13:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.148+0000 7f28ed10e640 1 -- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f28cc06a5e0 msgr2=0x7f28cc06caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:13:02.150 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.148+0000 7f28ed10e640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f28cc06a5e0 0x7f28cc06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T03:13:02.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.178+0000 7f28ee911640 1 -- 172.21.15.115:0/8570832 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28e80673a0 con 0x7f28e80cddc0 2023-12-01T03:13:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.178+0000 7f28de7fc640 1 -- 172.21.15.115:0/8570832 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f28e40567f0 con 0x7f28e80cddc0 2023-12-01T03:13:02.180 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:02.180 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":126,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2042114371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2042114371},{"type":"v1","addr":"172.21.15.142:6835","nonce":2042114371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/727621046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":727621046},{"type":"v1","addr":"172.21.15.115:6835","nonce":727621046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3018042052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3018042052},{"type":"v1","addr":"172.21.15.115:6837","nonce":3018042052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:12:35.317383+0000","modified":"2023-12-01T03:12:38.475270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12246":{"gid":12246,"name":"b","rank":0,"incarnation":122,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/1356734431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1356734431},{"type":"v1","addr":"172.21.15.142:6834","nonce":1356734431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-01T03:13:02.180 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 126 2023-12-01T03:13:02.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 -- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f28cc06a5e0 msgr2=0x7f28cc06caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:13:02.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f28cc06a5e0 0x7f28cc06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 -- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 msgr2=0x7f28e81f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:02.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 0x7f28e81f3180 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c4d0 tx=0x7f28e400c990 comp rx=0 tx=0).stop 2023-12-01T03:13:02.183 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 -- 172.21.15.115:0/8570832 shutdown_connections 2023-12-01T03:13:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f28cc06a5e0 0x7f28cc06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28e815d7b0 0x7f28e815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28e815db90 0x7f28e80b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 --2- 172.21.15.115:0/8570832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28e80cddc0 0x7f28e81f3180 unknown :-1 s=CLOSED pgs=3058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 -- 172.21.15.115:0/8570832 >> 172.21.15.115:0/8570832 conn(0x7f28e806a0e0 msgr2=0x7f28e806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 -- 172.21.15.115:0/8570832 shutdown_connections 2023-12-01T03:13:02.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.181+0000 7f28ee911640 1 -- 172.21.15.115:0/8570832 wait complete. 2023-12-01T03:13:02.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:13:02.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.385+0000 7fe22dd57640 1 Processor -- start 2023-12-01T03:13:02.387 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.385+0000 7fe22dd57640 1 -- start start 2023-12-01T03:13:02.388 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.386+0000 7fe22dd57640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe228159b80 0x7fe22806ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.386+0000 7fe22dd57640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe22806bfa0 0x7fe228159f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.386+0000 7fe22dd57640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe22815a4a0 0x7fe22815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.386+0000 7fe22dd57640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22806ce10 con 0x7fe22806bfa0 2023-12-01T03:13:02.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe22dd57640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22806fce0 con 0x7fe22815a4a0 2023-12-01T03:13:02.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe22dd57640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22806c400 con 0x7fe228159b80 2023-12-01T03:13:02.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe227fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe22806bfa0 0x7fe228159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe227fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fe22806bfa0 0x7fe228159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50602/0 (socket says 172.21.15.115:50602) 2023-12-01T03:13:02.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe227fff640 1 -- 172.21.15.115:0/3836383921 learned_addr learned my addr 172.21.15.115:0/3836383921 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:02.389 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe22d556640 1 --2- 172.21.15.115:0/3836383921 >> v2:172.21.15.142:3300/0 conn(0x7fe22815a4a0 0x7fe22815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe22cd55640 1 --2- 172.21.15.115:0/3836383921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe228159b80 0x7fe22806ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe227fff640 1 -- 172.21.15.115:0/3836383921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe228159b80 msgr2=0x7fe22806ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:02.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe227fff640 1 --2- 172.21.15.115:0/3836383921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe228159b80 0x7fe22806ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe227fff640 1 -- 172.21.15.115:0/3836383921 >> v2:172.21.15.142:3300/0 conn(0x7fe22815a4a0 msgr2=0x7fe22815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:02.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe227fff640 1 --2- 172.21.15.115:0/3836383921 >> v2:172.21.15.142:3300/0 conn(0x7fe22815a4a0 0x7fe22815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.390 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe227fff640 1 -- 172.21.15.115:0/3836383921 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22815b060 con 0x7fe22806bfa0 2023-12-01T03:13:02.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.387+0000 7fe22cd55640 1 --2- 172.21.15.115:0/3836383921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe228159b80 0x7fe22806ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:02.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.389+0000 7fe227fff640 1 --2- 172.21.15.115:0/3836383921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22806bfa0 0x7fe228159f60 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7fe2280c94b0 tx=0x7fe21402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e9e409054ff183 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:02.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.389+0000 7fe2277fe640 1 -- 172.21.15.115:0/3836383921 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21402ee90 con 0x7fe22806bfa0 2023-12-01T03:13:02.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.389+0000 7fe2277fe640 1 -- 172.21.15.115:0/3836383921 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214036440 con 0x7fe22806bfa0 2023-12-01T03:13:02.391 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.389+0000 7fe2277fe640 1 -- 172.21.15.115:0/3836383921 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140367d0 con 0x7fe22806bfa0 2023-12-01T03:13:02.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.390+0000 7fe22dd57640 1 -- 172.21.15.115:0/3836383921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22806bfa0 msgr2=0x7fe228159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:02.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.390+0000 7fe22dd57640 1 --2- 172.21.15.115:0/3836383921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22806bfa0 0x7fe228159f60 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7fe2280c94b0 tx=0x7fe21402e0e0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.390+0000 7fe22dd57640 1 -- 172.21.15.115:0/3836383921 shutdown_connections 2023-12-01T03:13:02.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.390+0000 7fe22dd57640 1 --2- 172.21.15.115:0/3836383921 >> v2:172.21.15.142:3300/0 conn(0x7fe22815a4a0 0x7fe22815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.390+0000 7fe22dd57640 1 --2- 172.21.15.115:0/3836383921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22806bfa0 0x7fe228159f60 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.392 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.390+0000 7fe22dd57640 1 --2- 172.21.15.115:0/3836383921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe228159b80 0x7fe22806ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.390+0000 7fe22dd57640 1 -- 172.21.15.115:0/3836383921 >> 172.21.15.115:0/3836383921 conn(0x7fe2280ceed0 msgr2=0x7fe2280cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:02.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.391+0000 7fe22dd57640 1 -- 172.21.15.115:0/3836383921 shutdown_connections 2023-12-01T03:13:02.393 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.391+0000 7fe22dd57640 1 -- 172.21.15.115:0/3836383921 wait complete. 2023-12-01T03:13:02.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.392+0000 7fe22dd57640 1 Processor -- start 2023-12-01T03:13:02.394 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.392+0000 7fe22dd57640 1 -- start start 2023-12-01T03:13:02.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.393+0000 7fe22dd57640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe22806bfa0 0x7fe2281f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.393+0000 7fe22dd57640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe228159b80 0x7fe228150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.393+0000 7fe22dd57640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22815a4a0 0x7fe2280cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.393+0000 7fe22dd57640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280b2bc0 con 0x7fe228159b80 2023-12-01T03:13:02.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.393+0000 7fe22dd57640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2281f2bd0 con 0x7fe22815a4a0 2023-12-01T03:13:02.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.393+0000 7fe22dd57640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe228151490 con 0x7fe22806bfa0 2023-12-01T03:13:02.395 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe227fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe228159b80 0x7fe228150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe227fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe228159b80 0x7fe228150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50612/0 (socket says 172.21.15.115:50612) 2023-12-01T03:13:02.396 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe227fff640 1 -- 172.21.15.115:0/2962220379 learned_addr learned my addr 172.21.15.115:0/2962220379 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe22d556640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22815a4a0 0x7fe2280cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe22cd55640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe22806bfa0 0x7fe2281f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe227fff640 1 -- 172.21.15.115:0/2962220379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe22806bfa0 msgr2=0x7fe2281f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe227fff640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe22806bfa0 0x7fe2281f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe227fff640 1 -- 172.21.15.115:0/2962220379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22815a4a0 msgr2=0x7fe2280cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe227fff640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22815a4a0 0x7fe2280cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe227fff640 1 -- 172.21.15.115:0/2962220379 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21400b770 con 0x7fe228159b80 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.394+0000 7fe227fff640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe228159b80 0x7fe228150f50 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7fe21402e120 tx=0x7fe2140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.395+0000 7fe2257fa640 1 -- 172.21.15.115:0/2962220379 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140348d0 con 0x7fe228159b80 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.395+0000 7fe2257fa640 1 -- 172.21.15.115:0/2962220379 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214034a30 con 0x7fe228159b80 2023-12-01T03:13:02.397 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.395+0000 7fe2257fa640 1 -- 172.21.15.115:0/2962220379 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214034de0 con 0x7fe228159b80 2023-12-01T03:13:02.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.395+0000 7fe22dd57640 1 -- 172.21.15.115:0/2962220379 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2280b1fb0 con 0x7fe228159b80 2023-12-01T03:13:02.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.395+0000 7fe22dd57640 1 -- 172.21.15.115:0/2962220379 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2280b2570 con 0x7fe228159b80 2023-12-01T03:13:02.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.396+0000 7fe22dd57640 1 -- 172.21.15.115:0/2962220379 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2280673a0 con 0x7fe228159b80 2023-12-01T03:13:02.398 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.397+0000 7fe2257fa640 1 -- 172.21.15.115:0/2962220379 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 147) v1 ==== 88766+0+0 (secure 0 0 0) 0x7fe21403d040 con 0x7fe228159b80 2023-12-01T03:13:02.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.398+0000 7fe2257fa640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe20c06a610 0x7fe20c06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.399 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.398+0000 7fe2257fa640 1 -- 172.21.15.115:0/2962220379 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fe2140adad0 con 0x7fe228159b80 2023-12-01T03:13:02.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.398+0000 7fe22cd55640 1 -- 172.21.15.115:0/2962220379 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe20c06a610 msgr2=0x7fe20c06cad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:13:02.400 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.398+0000 7fe22cd55640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe20c06a610 0x7fe20c06cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:13:02.403 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.402+0000 7fe2257fa640 1 -- 172.21.15.115:0/2962220379 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fe214078cf0 con 0x7fe228159b80 2023-12-01T03:13:02.471 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:02.469+0000 7ffbace32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:13:02.581 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:02.579+0000 7f64f4841040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T03:13:02.596 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.594+0000 7fe22dd57640 1 -- 172.21.15.115:0/2962220379 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe22815a180 con 0x7fe228159b80 2023-12-01T03:13:02.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.596+0000 7fe2257fa640 1 -- 172.21.15.115:0/2962220379 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) v1 ==== 74+0+16799 (secure 0 0 0) 0x7fe21407c310 con 0x7fe228159b80 2023-12-01T03:13:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:02.598 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":257,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:12:54.714825+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:12:33.411870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-01T03:12:34.335941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:6824/138932":"2023-12-02T03:12:54.714747+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2457056129":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2634160496":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2258893627":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2547954996":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2121389099":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/3223789397":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/534295842":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2854101523":"2023-12-02T03:12:54.714747+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:13:02.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 -- 172.21.15.115:0/2962220379 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe20c06a610 msgr2=0x7fe20c06cad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:13:02.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe20c06a610 0x7fe20c06cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 -- 172.21.15.115:0/2962220379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe228159b80 msgr2=0x7fe228150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:02.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe228159b80 0x7fe228150f50 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7fe21402e120 tx=0x7fe2140046c0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 -- 172.21.15.115:0/2962220379 shutdown_connections 2023-12-01T03:13:02.600 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fe20c06a610 0x7fe20c06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe22815a4a0 0x7fe2280cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe228159b80 0x7fe228150f50 unknown :-1 s=CLOSED pgs=3060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 --2- 172.21.15.115:0/2962220379 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe22806bfa0 0x7fe2281f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 -- 172.21.15.115:0/2962220379 >> 172.21.15.115:0/2962220379 conn(0x7fe2280ceed0 msgr2=0x7fe2280cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:02.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 -- 172.21.15.115:0/2962220379 shutdown_connections 2023-12-01T03:13:02.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.598+0000 7fe22dd57640 1 -- 172.21.15.115:0/2962220379 wait complete. 2023-12-01T03:13:02.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:36:00.229588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T03:12:33.411870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '255', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T03:12:34.335941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '256', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T03:13:02.618 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T03:13:02.769 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:02.767+0000 7ffbace32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:13:02.824 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.823+0000 7f689bfff640 1 Processor -- start 2023-12-01T03:13:02.825 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.823+0000 7f689bfff640 1 -- start start 2023-12-01T03:13:02.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.824+0000 7f689bfff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f689c0acd00 0x7f689c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.825+0000 7f689bfff640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c0c98a0 con 0x7f689c146440 2023-12-01T03:13:02.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.825+0000 7f689bfff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c0adbb0 con 0x7f689c0cd120 2023-12-01T03:13:02.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.825+0000 7f689bfff640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689c0ade90 con 0x7f689c0acd00 2023-12-01T03:13:02.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.825+0000 7f6893fff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f689c146440 0x7f689c0cca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37228/0 (socket says 172.21.15.115:37228) 2023-12-01T03:13:02.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.825+0000 7f6893fff640 1 -- 172.21.15.115:0/1394548722 learned_addr learned my addr 172.21.15.115:0/1394548722 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:02.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.825+0000 7f689affd640 1 --2- 172.21.15.115:0/1394548722 >> v2:172.21.15.142:3300/0 conn(0x7f689c0acd00 0x7f689c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.825+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335932004 0 0) 0x7f689c0c98a0 con 0x7f689c146440 2023-12-01T03:13:02.827 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.825+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6888003520 con 0x7f689c146440 2023-12-01T03:13:02.828 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:02.825+0000 7ffbace32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:13:02.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2380678824 0 0) 0x7f6888003520 con 0x7f689c146440 2023-12-01T03:13:02.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 >> v2:172.21.15.142:3300/0 conn(0x7f689c0acd00 msgr2=0x7f689c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:02.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 --2- 172.21.15.115:0/1394548722 >> v2:172.21.15.142:3300/0 conn(0x7f689c0acd00 0x7f689c0ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.829 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 >> v1:172.21.15.142:6789/0 conn(0x7f689c0cd120 legacy=0x7f689c0c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c0ae170 con 0x7f689c146440 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6884001670 con 0x7f689c146440 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 >> v1:172.21.15.115:6789/0 conn(0x7f689c146440 legacy=0x7f689c0cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f6888003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68880047c0 0x7f6888004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6888005180 0x7f688800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c0ade90 con 0x7f68880039c0 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688800c080 con 0x7f6888005180 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688800c240 con 0x7f68880047c0 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689affd640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f6888003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.826+0000 7f689b7fe640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6888005180 0x7f688800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.827+0000 7f6893fff640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68880047c0 0x7f6888004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.830 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.827+0000 7f689affd640 1 -- 172.21.15.115:0/1394548722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68880047c0 msgr2=0x7f6888004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:02.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.827+0000 7f689affd640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68880047c0 0x7f6888004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.827+0000 7f689affd640 1 -- 172.21.15.115:0/1394548722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6888005180 msgr2=0x7f688800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:02.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.827+0000 7f689affd640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6888005180 0x7f688800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.827+0000 7f689affd640 1 -- 172.21.15.115:0/1394548722 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888003520 con 0x7f68880039c0 2023-12-01T03:13:02.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.827+0000 7f689b7fe640 1 -- 172.21.15.115:0/1394548722 reap_dead start 2023-12-01T03:13:02.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.827+0000 7f6893fff640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68880047c0 0x7f6888004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:02.831 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.828+0000 7f689affd640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f6888003de0 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f688c009ac0 tx=0x7f68880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d41dad2e80502b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:02.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.828+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c02c6b0 con 0x7f68880039c0 2023-12-01T03:13:02.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.828+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c02c810 con 0x7f68880039c0 2023-12-01T03:13:02.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.828+0000 7f689a7fc640 1 -- 172.21.15.115:0/1394548722 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c02cba0 con 0x7f68880039c0 2023-12-01T03:13:02.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.829+0000 7f689bfff640 1 -- 172.21.15.115:0/1394548722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 msgr2=0x7f6888003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:02.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.829+0000 7f689bfff640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f6888003de0 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f688c009ac0 tx=0x7f68880057f0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.829+0000 7f689bfff640 1 -- 172.21.15.115:0/1394548722 shutdown_connections 2023-12-01T03:13:02.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.829+0000 7f689bfff640 1 --2- 172.21.15.115:0/1394548722 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f6888003de0 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.829+0000 7f689bfff640 1 -- 172.21.15.115:0/1394548722 >> 172.21.15.115:0/1394548722 conn(0x7f689c0cee40 msgr2=0x7f689c0cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:02.832 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.829+0000 7f689bfff640 1 -- 172.21.15.115:0/1394548722 shutdown_connections 2023-12-01T03:13:02.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.829+0000 7f689bfff640 1 -- 172.21.15.115:0/1394548722 wait complete. 2023-12-01T03:13:02.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.830+0000 7f689bfff640 1 Processor -- start 2023-12-01T03:13:02.833 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.831+0000 7f689bfff640 1 -- start start 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.831+0000 7f689bfff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f689c0c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.831+0000 7f689bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689c0b3ea0 0x7f689c15ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.831+0000 7f689bfff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f689c0b4320 0x7f689c15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.831+0000 7f689bfff640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c0c6bc0 con 0x7f68880039c0 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.831+0000 7f689bfff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c0c5fb0 con 0x7f689c0b4320 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.831+0000 7f689bfff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689c0c6290 con 0x7f689c0b3ea0 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f689affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f689c0c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f689affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f689c0c2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50620/0 (socket says 172.21.15.115:50620) 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f689affd640 1 -- 172.21.15.115:0/4063621420 learned_addr learned my addr 172.21.15.115:0/4063621420 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f6893fff640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689c0b3ea0 0x7f689c15ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:02.838 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f689affd640 1 -- 172.21.15.115:0/4063621420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689c0b3ea0 msgr2=0x7f689c15ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f689affd640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689c0b3ea0 0x7f689c15ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f689affd640 1 -- 172.21.15.115:0/4063621420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f689c0b4320 msgr2=0x7f689c15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f689affd640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f689c0b4320 0x7f689c15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f689affd640 1 -- 172.21.15.115:0/4063621420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c0c6570 con 0x7f68880039c0 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f6893fff640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689c0b3ea0 0x7f689c15ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.832+0000 7f689affd640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f689c0c2970 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7f688c02c250 tx=0x7f688c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.833+0000 7f68937fe640 1 -- 172.21.15.115:0/4063621420 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c003810 con 0x7f68880039c0 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.833+0000 7f68937fe640 1 -- 172.21.15.115:0/4063621420 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c003970 con 0x7f68880039c0 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.833+0000 7f689bfff640 1 -- 172.21.15.115:0/4063621420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689c0c6bc0 con 0x7f68880039c0 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.833+0000 7f68937fe640 1 -- 172.21.15.115:0/4063621420 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c003d00 con 0x7f68880039c0 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.833+0000 7f689bfff640 1 -- 172.21.15.115:0/4063621420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689c0c2eb0 con 0x7f68880039c0 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.834+0000 7f689bfff640 1 -- 172.21.15.115:0/4063621420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689c0618d0 con 0x7f68880039c0 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.835+0000 7f68937fe640 1 -- 172.21.15.115:0/4063621420 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 147) v1 ==== 88766+0+0 (secure 0 0 0) 0x7f688c047450 con 0x7f68880039c0 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.836+0000 7f68937fe640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f687c06a5e0 0x7f687c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:02.839 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.836+0000 7f68937fe640 1 -- 172.21.15.115:0/4063621420 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f688c0ab030 con 0x7f68880039c0 2023-12-01T03:13:02.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.836+0000 7f6893fff640 1 -- 172.21.15.115:0/4063621420 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f687c06a5e0 msgr2=0x7f687c06caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.115:6822/138932 2023-12-01T03:13:02.840 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.836+0000 7f6893fff640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f687c06a5e0 0x7f687c06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T03:13:02.842 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:02.840+0000 7f68937fe640 1 -- 172.21.15.115:0/4063621420 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f688c076270 con 0x7f68880039c0 2023-12-01T03:13:02.899 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:02.897+0000 7f64f4841040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T03:13:02.942 INFO:tasks.ceph.mgr.x.smithi142.stderr:2023-12-01T03:13:02.940+0000 7ffbace32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:13:02.958 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:02.956+0000 7f64f4841040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T03:13:03.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.029+0000 7f689bfff640 1 -- 172.21.15.115:0/4063621420 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f689c073270 con 0x7f68880039c0 2023-12-01T03:13:03.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.030+0000 7f68937fe640 1 -- 172.21.15.115:0/4063621420 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v257) v1 ==== 93+0+10 (secure 0 0 0) 0x7f688c003e60 con 0x7f68880039c0 2023-12-01T03:13:03.032 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2023-12-01T03:13:03.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 -- 172.21.15.115:0/4063621420 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f687c06a5e0 msgr2=0x7f687c06caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T03:13:03.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f687c06a5e0 0x7f687c06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 -- 172.21.15.115:0/4063621420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 msgr2=0x7f689c0c2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:03.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f689c0c2970 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7f688c02c250 tx=0x7f688c004730 comp rx=0 tx=0).stop 2023-12-01T03:13:03.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 -- 172.21.15.115:0/4063621420 shutdown_connections 2023-12-01T03:13:03.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f687c06a5e0 0x7f687c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689c0b3ea0 0x7f689c15ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f689c0b4320 0x7f689c15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 --2- 172.21.15.115:0/4063621420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68880039c0 0x7f689c0c2970 unknown :-1 s=CLOSED pgs=3063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 -- 172.21.15.115:0/4063621420 >> 172.21.15.115:0/4063621420 conn(0x7f689c0cee40 msgr2=0x7f689c0c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:03.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 -- 172.21.15.115:0/4063621420 shutdown_connections 2023-12-01T03:13:03.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.033+0000 7f689bfff640 1 -- 172.21.15.115:0/4063621420 wait complete. 2023-12-01T03:13:03.054 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T03:13:03.078 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.076+0000 7f64f4841040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T03:13:03.154 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.152+0000 7f6479c0f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:13:03.154 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.152+0000 7f6479c0f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:13:03.155 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.152+0000 7f6479c0f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:13:03.155 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.152+0000 7f6479c0f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:13:03.155 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.152+0000 7f6479c0f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:13:03.155 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.153+0000 7f6476c09640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:13:03.155 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.153+0000 7f6476c09640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:13:03.155 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.153+0000 7f6476c09640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:13:03.155 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.153+0000 7f6476c09640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:13:03.155 INFO:tasks.ceph.mgr.y.smithi115.stderr:2023-12-01T03:13:03.153+0000 7f6476c09640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T03:13:03.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.251+0000 7f659f573640 1 Processor -- start 2023-12-01T03:13:03.253 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.251+0000 7f659f573640 1 -- start start 2023-12-01T03:13:03.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.252+0000 7f659f573640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65980aef60 0x7f65980b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.252+0000 7f659f573640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f659815e3c0 0x7f65980cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.252+0000 7f659f573640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f65980b3140 0x7f65980cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.252+0000 7f659f573640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65980af340 con 0x7f65980b3140 2023-12-01T03:13:03.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.252+0000 7f659f573640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65980b38d0 con 0x7f659815e3c0 2023-12-01T03:13:03.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.252+0000 7f659f573640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65980b35f0 con 0x7f65980aef60 2023-12-01T03:13:03.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659db10640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f65980b3140 0x7f65980cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:03.255 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659db10640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f65980b3140 0x7f65980cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50734/0 (socket says 172.21.15.115:50734) 2023-12-01T03:13:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659db10640 1 -- 172.21.15.115:0/2624627731 learned_addr learned my addr 172.21.15.115:0/2624627731 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659d30f640 1 --2- 172.21.15.115:0/2624627731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65980aef60 0x7f65980b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659cb0e640 1 --2- 172.21.15.115:0/2624627731 >> v2:172.21.15.142:3300/0 conn(0x7f659815e3c0 0x7f65980cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659db10640 1 -- 172.21.15.115:0/2624627731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65980aef60 msgr2=0x7f65980b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659db10640 1 --2- 172.21.15.115:0/2624627731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65980aef60 0x7f65980b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659db10640 1 -- 172.21.15.115:0/2624627731 >> v2:172.21.15.142:3300/0 conn(0x7f659815e3c0 msgr2=0x7f65980cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659db10640 1 --2- 172.21.15.115:0/2624627731 >> v2:172.21.15.142:3300/0 conn(0x7f659815e3c0 0x7f65980cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659db10640 1 -- 172.21.15.115:0/2624627731 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6598070570 con 0x7f65980b3140 2023-12-01T03:13:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659d30f640 1 --2- 172.21.15.115:0/2624627731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65980aef60 0x7f65980b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:03.256 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.253+0000 7f659cb0e640 1 --2- 172.21.15.115:0/2624627731 >> v2:172.21.15.142:3300/0 conn(0x7f659815e3c0 0x7f65980cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:03.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.254+0000 7f659db10640 1 --2- 172.21.15.115:0/2624627731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 0x7f65980cc8e0 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 crypto rx=0x7f6598073ad0 tx=0x7f659002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcbd68bcbb80576 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:03.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.254+0000 7f658ffff640 1 -- 172.21.15.115:0/2624627731 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659002ea80 con 0x7f65980b3140 2023-12-01T03:13:03.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.254+0000 7f658ffff640 1 -- 172.21.15.115:0/2624627731 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659002ebe0 con 0x7f65980b3140 2023-12-01T03:13:03.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.254+0000 7f658ffff640 1 -- 172.21.15.115:0/2624627731 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590009600 con 0x7f65980b3140 2023-12-01T03:13:03.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.255+0000 7f659f573640 1 -- 172.21.15.115:0/2624627731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 msgr2=0x7f65980cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:03.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.255+0000 7f659f573640 1 --2- 172.21.15.115:0/2624627731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 0x7f65980cc8e0 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 crypto rx=0x7f6598073ad0 tx=0x7f659002e0e0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.255+0000 7f659f573640 1 -- 172.21.15.115:0/2624627731 shutdown_connections 2023-12-01T03:13:03.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.255+0000 7f659f573640 1 --2- 172.21.15.115:0/2624627731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 0x7f65980cc8e0 unknown :-1 s=CLOSED pgs=3075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.255+0000 7f659f573640 1 --2- 172.21.15.115:0/2624627731 >> v2:172.21.15.142:3300/0 conn(0x7f659815e3c0 0x7f65980cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.255+0000 7f659f573640 1 --2- 172.21.15.115:0/2624627731 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65980aef60 0x7f65980b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.255+0000 7f659f573640 1 -- 172.21.15.115:0/2624627731 >> 172.21.15.115:0/2624627731 conn(0x7f65980b1720 msgr2=0x7f65980b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:03.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.255+0000 7f659f573640 1 -- 172.21.15.115:0/2624627731 shutdown_connections 2023-12-01T03:13:03.258 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.255+0000 7f659f573640 1 -- 172.21.15.115:0/2624627731 wait complete. 2023-12-01T03:13:03.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.257+0000 7f659f573640 1 Processor -- start 2023-12-01T03:13:03.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.257+0000 7f659f573640 1 -- start start 2023-12-01T03:13:03.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.258+0000 7f659f573640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65980aef60 0x7f65981f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.258+0000 7f659f573640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 0x7f659814a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.258+0000 7f659f573640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f659815e3c0 0x7f65980a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.258+0000 7f659f573640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65980afd90 con 0x7f65980b3140 2023-12-01T03:13:03.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.258+0000 7f659f573640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65980aa5f0 con 0x7f659815e3c0 2023-12-01T03:13:03.260 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.258+0000 7f659f573640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65981f1490 con 0x7f65980aef60 2023-12-01T03:13:03.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.258+0000 7f659cb0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 0x7f659814a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:03.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.259+0000 7f659cb0e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 0x7f659814a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50740/0 (socket says 172.21.15.115:50740) 2023-12-01T03:13:03.261 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.259+0000 7f659cb0e640 1 -- 172.21.15.115:0/75240843 learned_addr learned my addr 172.21.15.115:0/75240843 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.259+0000 7f659db10640 1 --2- 172.21.15.115:0/75240843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f659815e3c0 0x7f65980a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.259+0000 7f659cb0e640 1 -- 172.21.15.115:0/75240843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65980aef60 msgr2=0x7f65981f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.259+0000 7f659cb0e640 1 --2- 172.21.15.115:0/75240843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65980aef60 0x7f65981f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.259+0000 7f659cb0e640 1 -- 172.21.15.115:0/75240843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f659815e3c0 msgr2=0x7f65980a9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.259+0000 7f659cb0e640 1 --2- 172.21.15.115:0/75240843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f659815e3c0 0x7f65980a9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.259+0000 7f659cb0e640 1 -- 172.21.15.115:0/75240843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659000bd40 con 0x7f65980b3140 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.259+0000 7f659cb0e640 1 --2- 172.21.15.115:0/75240843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 0x7f659814a110 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 crypto rx=0x7f658800c7f0 tx=0x7f658800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.259+0000 7f658dffb640 1 -- 172.21.15.115:0/75240843 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588007820 con 0x7f65980b3140 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.260+0000 7f658dffb640 1 -- 172.21.15.115:0/75240843 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6588007980 con 0x7f65980b3140 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.260+0000 7f658dffb640 1 -- 172.21.15.115:0/75240843 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588007cf0 con 0x7f65980b3140 2023-12-01T03:13:03.262 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.260+0000 7f659f573640 1 -- 172.21.15.115:0/75240843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65981f1770 con 0x7f65980b3140 2023-12-01T03:13:03.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.260+0000 7f659f573640 1 -- 172.21.15.115:0/75240843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65980aa2e0 con 0x7f65980b3140 2023-12-01T03:13:03.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.261+0000 7f659f573640 1 -- 172.21.15.115:0/75240843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659804e890 con 0x7f65980b3140 2023-12-01T03:13:03.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.262+0000 7f658dffb640 1 -- 172.21.15.115:0/75240843 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 148) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f65880113f0 con 0x7f65980b3140 2023-12-01T03:13:03.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.263+0000 7f658dffb640 1 -- 172.21.15.115:0/75240843 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f6588088ad0 con 0x7f65980b3140 2023-12-01T03:13:03.268 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.267+0000 7f658dffb640 1 -- 172.21.15.115:0/75240843 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6588054110 con 0x7f65980b3140 2023-12-01T03:13:03.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.453+0000 7f659f573640 1 -- 172.21.15.115:0/75240843 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65980ccc00 con 0x7f65980b3140 2023-12-01T03:13:03.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.454+0000 7f658dffb640 1 -- 172.21.15.115:0/75240843 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v258) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6588057730 con 0x7f65980b3140 2023-12-01T03:13:03.456 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 32 2023-12-01T03:13:03.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.457+0000 7f659f573640 1 -- 172.21.15.115:0/75240843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 msgr2=0x7f659814a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:03.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.457+0000 7f659f573640 1 --2- 172.21.15.115:0/75240843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 0x7f659814a110 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 crypto rx=0x7f658800c7f0 tx=0x7f658800ccb0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.457+0000 7f659f573640 1 -- 172.21.15.115:0/75240843 shutdown_connections 2023-12-01T03:13:03.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.457+0000 7f659f573640 1 --2- 172.21.15.115:0/75240843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f659815e3c0 0x7f65980a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.457+0000 7f659f573640 1 --2- 172.21.15.115:0/75240843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65980b3140 0x7f659814a110 unknown :-1 s=CLOSED pgs=3076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.457+0000 7f659f573640 1 --2- 172.21.15.115:0/75240843 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65980aef60 0x7f65981f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.457+0000 7f659f573640 1 -- 172.21.15.115:0/75240843 >> 172.21.15.115:0/75240843 conn(0x7f65980b1720 msgr2=0x7f65980b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:03.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.457+0000 7f659f573640 1 -- 172.21.15.115:0/75240843 shutdown_connections 2023-12-01T03:13:03.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.457+0000 7f659f573640 1 -- 172.21.15.115:0/75240843 wait complete. 2023-12-01T03:13:03.476 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T03:13:03.735 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.734+0000 7f9dc5118640 1 Processor -- start 2023-12-01T03:13:03.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.734+0000 7f9dc5118640 1 -- start start 2023-12-01T03:13:03.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.735+0000 7f9dc5118640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9dc015ab10 0x7f9dc015e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.735+0000 7f9dc5118640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 0x7f9dc00cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.735+0000 7f9dc5118640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc015ec60 con 0x7f9dc00cba50 2023-12-01T03:13:03.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.735+0000 7f9dc5118640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc015f730 con 0x7f9dc015ab10 2023-12-01T03:13:03.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.735+0000 7f9dc5118640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00cbee0 con 0x7f9dc006ff00 2023-12-01T03:13:03.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.736+0000 7f9dbed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9dc015ab10 0x7f9dc015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:03.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.736+0000 7f9dbed9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f9dc015ab10 0x7f9dc015e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50746/0 (socket says 172.21.15.115:50746) 2023-12-01T03:13:03.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.736+0000 7f9dbed9d640 1 -- 172.21.15.115:0/3705414279 learned_addr learned my addr 172.21.15.115:0/3705414279 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:03.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.736+0000 7f9db7fff640 1 --2- 172.21.15.115:0/3705414279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 0x7f9dc00cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:03.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.736+0000 7f9dbed9d640 1 -- 172.21.15.115:0/3705414279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 msgr2=0x7f9dc00cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:03.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.736+0000 7f9dbed9d640 1 --2- 172.21.15.115:0/3705414279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 0x7f9dc00cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.736+0000 7f9dbed9d640 1 -- 172.21.15.115:0/3705414279 >> v1:172.21.15.142:6789/0 conn(0x7f9dc00cba50 legacy=0x7f9dc006b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:03.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.736+0000 7f9dbed9d640 1 -- 172.21.15.115:0/3705414279 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00ae6d0 con 0x7f9dc015ab10 2023-12-01T03:13:03.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.736+0000 7f9db7fff640 1 --2- 172.21.15.115:0/3705414279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 0x7f9dc00cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:03.739 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.737+0000 7f9dbed9d640 1 --2- 172.21.15.115:0/3705414279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 0x7f9dc015e720 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009870 tx=0x7f9db002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3e0cbe287a9ce server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:03.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.737+0000 7f9dbe59c640 1 -- 172.21.15.115:0/3705414279 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0031640 con 0x7f9dc015ab10 2023-12-01T03:13:03.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.738+0000 7f9dbe59c640 1 -- 172.21.15.115:0/3705414279 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db00317a0 con 0x7f9dc015ab10 2023-12-01T03:13:03.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.738+0000 7f9dbe59c640 1 -- 172.21.15.115:0/3705414279 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0031b30 con 0x7f9dc015ab10 2023-12-01T03:13:03.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.738+0000 7f9dc5118640 1 -- 172.21.15.115:0/3705414279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 msgr2=0x7f9dc015e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:03.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.738+0000 7f9dc5118640 1 --2- 172.21.15.115:0/3705414279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 0x7f9dc015e720 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009870 tx=0x7f9db002b8f0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.738+0000 7f9dc5118640 1 -- 172.21.15.115:0/3705414279 shutdown_connections 2023-12-01T03:13:03.740 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.738+0000 7f9dc5118640 1 --2- 172.21.15.115:0/3705414279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 0x7f9dc00cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.738+0000 7f9dc5118640 1 --2- 172.21.15.115:0/3705414279 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 0x7f9dc015e720 unknown :-1 s=CLOSED pgs=3077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.738+0000 7f9dc5118640 1 -- 172.21.15.115:0/3705414279 >> 172.21.15.115:0/3705414279 conn(0x7f9dc0065b40 msgr2=0x7f9dc0065f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:03.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.738+0000 7f9dc5118640 1 -- 172.21.15.115:0/3705414279 shutdown_connections 2023-12-01T03:13:03.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.738+0000 7f9dc5118640 1 -- 172.21.15.115:0/3705414279 wait complete. 2023-12-01T03:13:03.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.740+0000 7f9dc5118640 1 Processor -- start 2023-12-01T03:13:03.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.740+0000 7f9dc5118640 1 -- start start 2023-12-01T03:13:03.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.741+0000 7f9dc5118640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 0x7f9dc01f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.741+0000 7f9dc5118640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dc00cba50 0x7f9dc00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.741+0000 7f9dc5118640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 0x7f9dc00ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.741+0000 7f9dc5118640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc01f6bd0 con 0x7f9dc015ab10 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.741+0000 7f9dc5118640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00b2d90 con 0x7f9dc00cba50 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.741+0000 7f9dc5118640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00b1370 con 0x7f9dc006ff00 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.741+0000 7f9dbf59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 0x7f9dc00ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.741+0000 7f9dbf59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 0x7f9dc00ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50752/0 (socket says 172.21.15.115:50752) 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.741+0000 7f9dbf59e640 1 -- 172.21.15.115:0/273697767 learned_addr learned my addr 172.21.15.115:0/273697767 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9dbed9d640 1 --2- 172.21.15.115:0/273697767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 0x7f9dc01f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9dbf59e640 1 -- 172.21.15.115:0/273697767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 msgr2=0x7f9dc01f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9dbf59e640 1 --2- 172.21.15.115:0/273697767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 0x7f9dc01f6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9dbf59e640 1 -- 172.21.15.115:0/273697767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dc00cba50 msgr2=0x7f9dc00ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9dbf59e640 1 --2- 172.21.15.115:0/273697767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dc00cba50 0x7f9dc00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9dbf59e640 1 -- 172.21.15.115:0/273697767 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db00094d0 con 0x7f9dc015ab10 2023-12-01T03:13:03.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9dbf59e640 1 --2- 172.21.15.115:0/273697767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 0x7f9dc00ace00 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c780 tx=0x7f9db800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:03.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9db77fe640 1 -- 172.21.15.115:0/273697767 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8004b60 con 0x7f9dc015ab10 2023-12-01T03:13:03.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9db77fe640 1 -- 172.21.15.115:0/273697767 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db800ce90 con 0x7f9dc015ab10 2023-12-01T03:13:03.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9dc5118640 1 -- 172.21.15.115:0/273697767 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc01474f0 con 0x7f9dc015ab10 2023-12-01T03:13:03.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9db77fe640 1 -- 172.21.15.115:0/273697767 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db800a850 con 0x7f9dc015ab10 2023-12-01T03:13:03.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.742+0000 7f9dc5118640 1 -- 172.21.15.115:0/273697767 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc0147ab0 con 0x7f9dc015ab10 2023-12-01T03:13:03.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.744+0000 7f9dc5118640 1 -- 172.21.15.115:0/273697767 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc00b2f50 con 0x7f9dc015ab10 2023-12-01T03:13:03.746 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.745+0000 7f9db77fe640 1 -- 172.21.15.115:0/273697767 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 148) v1 ==== 88318+0+0 (secure 0 0 0) 0x7f9db800a9b0 con 0x7f9dc015ab10 2023-12-01T03:13:03.747 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.746+0000 7f9db77fe640 1 -- 172.21.15.115:0/273697767 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f9db8088800 con 0x7f9dc015ab10 2023-12-01T03:13:03.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.750+0000 7f9db77fe640 1 -- 172.21.15.115:0/273697767 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f9db8053f30 con 0x7f9dc015ab10 2023-12-01T03:13:03.941 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.940+0000 7f9dc5118640 1 -- 172.21.15.115:0/273697767 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9dc004e890 con 0x7f9dc015ab10 2023-12-01T03:13:03.942 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.941+0000 7f9db77fe640 1 -- 172.21.15.115:0/273697767 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v258) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9db8057550 con 0x7f9dc015ab10 2023-12-01T03:13:03.942 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 64 2023-12-01T03:13:03.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.943+0000 7f9dc5118640 1 -- 172.21.15.115:0/273697767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 msgr2=0x7f9dc00ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:03.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.943+0000 7f9dc5118640 1 --2- 172.21.15.115:0/273697767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 0x7f9dc00ace00 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c780 tx=0x7f9db800cc40 comp rx=0 tx=0).stop 2023-12-01T03:13:03.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.943+0000 7f9dc5118640 1 -- 172.21.15.115:0/273697767 shutdown_connections 2023-12-01T03:13:03.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.943+0000 7f9dc5118640 1 --2- 172.21.15.115:0/273697767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dc015ab10 0x7f9dc00ace00 unknown :-1 s=CLOSED pgs=3078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.943+0000 7f9dc5118640 1 --2- 172.21.15.115:0/273697767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dc00cba50 0x7f9dc00ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.943+0000 7f9dc5118640 1 --2- 172.21.15.115:0/273697767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9dc006ff00 0x7f9dc01f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:03.945 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.943+0000 7f9dc5118640 1 -- 172.21.15.115:0/273697767 >> 172.21.15.115:0/273697767 conn(0x7f9dc0065b40 msgr2=0x7f9dc00c4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:03.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.943+0000 7f9dc5118640 1 -- 172.21.15.115:0/273697767 shutdown_connections 2023-12-01T03:13:03.946 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:03.943+0000 7f9dc5118640 1 -- 172.21.15.115:0/273697767 wait complete. 2023-12-01T03:13:03.963 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:13:04.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.173+0000 7f8528900640 1 Processor -- start 2023-12-01T03:13:04.175 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.173+0000 7f8528900640 1 -- start start 2023-12-01T03:13:04.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.174+0000 7f8528900640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8524159dd0 0x7f85240c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.174+0000 7f8528900640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8524070d30 0x7f85240c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.174+0000 7f8528900640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852406f8b0 con 0x7f852406b6f0 2023-12-01T03:13:04.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.174+0000 7f8528900640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85240adcb0 con 0x7f8524070d30 2023-12-01T03:13:04.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.174+0000 7f8528900640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85240ad9d0 con 0x7f8524159dd0 2023-12-01T03:13:04.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.174+0000 7f8522d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8524070d30 0x7f85240c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:04.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.175+0000 7f8522d9d640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8524070d30 0x7f85240c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50758/0 (socket says 172.21.15.115:50758) 2023-12-01T03:13:04.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.175+0000 7f8522d9d640 1 -- 172.21.15.115:0/4147819508 learned_addr learned my addr 172.21.15.115:0/4147819508 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:04.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.175+0000 7f852259c640 1 --2- 172.21.15.115:0/4147819508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8524159dd0 0x7f85240c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:04.177 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.175+0000 7f8522d9d640 1 -- 172.21.15.115:0/4147819508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8524159dd0 msgr2=0x7f85240c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:04.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.175+0000 7f8522d9d640 1 --2- 172.21.15.115:0/4147819508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8524159dd0 0x7f85240c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.175+0000 7f8522d9d640 1 -- 172.21.15.115:0/4147819508 >> v1:172.21.15.142:6789/0 conn(0x7f852406b6f0 legacy=0x7f85240cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:04.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.175+0000 7f8522d9d640 1 -- 172.21.15.115:0/4147819508 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85240c9000 con 0x7f8524070d30 2023-12-01T03:13:04.178 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.176+0000 7f8522d9d640 1 --2- 172.21.15.115:0/4147819508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524070d30 0x7f85240c9d20 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 crypto rx=0x7f851c009870 tx=0x7f851c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c220956ec9f8cd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:04.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.176+0000 7f8521d9b640 1 -- 172.21.15.115:0/4147819508 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c031740 con 0x7f8524070d30 2023-12-01T03:13:04.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.176+0000 7f8521d9b640 1 -- 172.21.15.115:0/4147819508 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c0318a0 con 0x7f8524070d30 2023-12-01T03:13:04.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.176+0000 7f8521d9b640 1 -- 172.21.15.115:0/4147819508 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c031c30 con 0x7f8524070d30 2023-12-01T03:13:04.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.177+0000 7f8528900640 1 -- 172.21.15.115:0/4147819508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524070d30 msgr2=0x7f85240c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.177+0000 7f8528900640 1 --2- 172.21.15.115:0/4147819508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524070d30 0x7f85240c9d20 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 crypto rx=0x7f851c009870 tx=0x7f851c02b7e0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.177+0000 7f8528900640 1 -- 172.21.15.115:0/4147819508 shutdown_connections 2023-12-01T03:13:04.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.177+0000 7f8528900640 1 --2- 172.21.15.115:0/4147819508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524070d30 0x7f85240c9d20 unknown :-1 s=CLOSED pgs=3079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.177+0000 7f8528900640 1 --2- 172.21.15.115:0/4147819508 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8524159dd0 0x7f85240c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.177+0000 7f8528900640 1 -- 172.21.15.115:0/4147819508 >> 172.21.15.115:0/4147819508 conn(0x7f85240ce880 msgr2=0x7f85240cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:04.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.177+0000 7f8528900640 1 -- 172.21.15.115:0/4147819508 shutdown_connections 2023-12-01T03:13:04.180 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.177+0000 7f8528900640 1 -- 172.21.15.115:0/4147819508 wait complete. 2023-12-01T03:13:04.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.179+0000 7f8528900640 1 Processor -- start 2023-12-01T03:13:04.181 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.179+0000 7f8528900640 1 -- start start 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.180+0000 7f8528900640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f852406b6f0 0x7f85240c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.180+0000 7f8528900640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524070d30 0x7f85240c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.180+0000 7f8528900640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524159dd0 0x7f85241f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.180+0000 7f8528900640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85240c27e0 con 0x7f8524159dd0 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.180+0000 7f8528900640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85240c5e90 con 0x7f8524070d30 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.180+0000 7f8528900640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85240c8190 con 0x7f852406b6f0 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.180+0000 7f8522d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524159dd0 0x7f85241f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f8522d9d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524159dd0 0x7f85241f20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50770/0 (socket says 172.21.15.115:50770) 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f8522d9d640 1 -- 172.21.15.115:0/1464412775 learned_addr learned my addr 172.21.15.115:0/1464412775 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f8519d9b640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524070d30 0x7f85240c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f852259c640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f852406b6f0 0x7f85240c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:04.184 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f8522d9d640 1 -- 172.21.15.115:0/1464412775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f852406b6f0 msgr2=0x7f85240c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f8522d9d640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f852406b6f0 0x7f85240c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f8522d9d640 1 -- 172.21.15.115:0/1464412775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524070d30 msgr2=0x7f85240c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f8522d9d640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524070d30 0x7f85240c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f8522d9d640 1 -- 172.21.15.115:0/1464412775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0094d0 con 0x7f8524159dd0 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f8519d9b640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524070d30 0x7f85240c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.181+0000 7f8522d9d640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524159dd0 0x7f85241f20f0 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 crypto rx=0x7f851c009870 tx=0x7f851c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.182+0000 7f851b7fe640 1 -- 172.21.15.115:0/1464412775 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c033c70 con 0x7f8524159dd0 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.182+0000 7f851b7fe640 1 -- 172.21.15.115:0/1464412775 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c033dd0 con 0x7f8524159dd0 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.182+0000 7f8528900640 1 -- 172.21.15.115:0/1464412775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8524143020 con 0x7f8524159dd0 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.182+0000 7f851b7fe640 1 -- 172.21.15.115:0/1464412775 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c03c640 con 0x7f8524159dd0 2023-12-01T03:13:04.185 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.182+0000 7f8528900640 1 -- 172.21.15.115:0/1464412775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85241432c0 con 0x7f8524159dd0 2023-12-01T03:13:04.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.183+0000 7f8528900640 1 -- 172.21.15.115:0/1464412775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85240738e0 con 0x7f8524159dd0 2023-12-01T03:13:04.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.184+0000 7f851b7fe640 1 -- 172.21.15.115:0/1464412775 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 149) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f851c03a040 con 0x7f8524159dd0 2023-12-01T03:13:04.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.185+0000 7f851b7fe640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f850406a280 0x7f850406c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.186 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.185+0000 7f851b7fe640 1 -- 172.21.15.115:0/1464412775 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f851c0aa2b0 con 0x7f8524159dd0 2023-12-01T03:13:04.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.185+0000 7f852259c640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f850406a280 0x7f850406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:04.187 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.186+0000 7f852259c640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f850406a280 0x7f850406c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8514009e90 tx=0x7f851400b040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:04.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.189+0000 7f851b7fe640 1 -- 172.21.15.115:0/1464412775 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f851c0758d0 con 0x7f8524159dd0 2023-12-01T03:13:04.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.417+0000 7f8528900640 1 -- 172.21.15.115:0/1464412775 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f852404e890 con 0x7f8524159dd0 2023-12-01T03:13:04.420 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.418+0000 7f851b7fe640 1 -- 172.21.15.115:0/1464412775 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f851c02f090 con 0x7f8524159dd0 2023-12-01T03:13:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:04.421 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":126,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2042114371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2042114371},{"type":"v1","addr":"172.21.15.142:6835","nonce":2042114371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/727621046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":727621046},{"type":"v1","addr":"172.21.15.115:6835","nonce":727621046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3018042052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3018042052},{"type":"v1","addr":"172.21.15.115:6837","nonce":3018042052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:12:35.317383+0000","modified":"2023-12-01T03:12:38.475270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12246":{"gid":12246,"name":"b","rank":0,"incarnation":122,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/1356734431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1356734431},{"type":"v1","addr":"172.21.15.142:6834","nonce":1356734431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-01T03:13:04.421 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 126 2023-12-01T03:13:04.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 -- 172.21.15.115:0/1464412775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f850406a280 msgr2=0x7f850406c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f850406a280 0x7f850406c740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8514009e90 tx=0x7f851400b040 comp rx=0 tx=0).stop 2023-12-01T03:13:04.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 -- 172.21.15.115:0/1464412775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524159dd0 msgr2=0x7f85241f20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524159dd0 0x7f85241f20f0 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 crypto rx=0x7f851c009870 tx=0x7f851c0046c0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 -- 172.21.15.115:0/1464412775 shutdown_connections 2023-12-01T03:13:04.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f850406a280 0x7f850406c740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8524159dd0 0x7f85241f20f0 unknown :-1 s=CLOSED pgs=3080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8524070d30 0x7f85240c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 --2- 172.21.15.115:0/1464412775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f852406b6f0 0x7f85240c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 -- 172.21.15.115:0/1464412775 >> 172.21.15.115:0/1464412775 conn(0x7f85240ce880 msgr2=0x7f852415cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:04.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 -- 172.21.15.115:0/1464412775 shutdown_connections 2023-12-01T03:13:04.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.421+0000 7f8528900640 1 -- 172.21.15.115:0/1464412775 wait complete. 2023-12-01T03:13:04.442 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:13:04.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.641+0000 7f102f99c640 1 Processor -- start 2023-12-01T03:13:04.643 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.641+0000 7f102f99c640 1 -- start start 2023-12-01T03:13:04.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.643+0000 7f102f99c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f102815e2f0 0x7f10280ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.643+0000 7f102f99c640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280add10 con 0x7f102806f2f0 2023-12-01T03:13:04.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.643+0000 7f102f99c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280c5760 con 0x7f10280adf10 2023-12-01T03:13:04.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.643+0000 7f102f99c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10280c9cf0 con 0x7f102815e2f0 2023-12-01T03:13:04.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.643+0000 7f102d738640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f102815e2f0 0x7f10280ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:04.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.643+0000 7f102d738640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f102815e2f0 0x7f10280ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50784/0 (socket says 172.21.15.115:50784) 2023-12-01T03:13:04.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.643+0000 7f102cf37640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f102806f2f0 0x7f10280c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37234/0 (socket says 172.21.15.115:37234) 2023-12-01T03:13:04.645 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.643+0000 7f102d738640 1 -- 172.21.15.115:0/2926570046 learned_addr learned my addr 172.21.15.115:0/2926570046 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:04.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.644+0000 7f102d738640 1 -- 172.21.15.115:0/2926570046 >> v1:172.21.15.142:6789/0 conn(0x7f10280adf10 legacy=0x7f10280c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:04.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.644+0000 7f102d738640 1 -- 172.21.15.115:0/2926570046 >> v1:172.21.15.115:6789/0 conn(0x7f102806f2f0 legacy=0x7f10280c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.644+0000 7f102d738640 1 -- 172.21.15.115:0/2926570046 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102815e6d0 con 0x7f102815e2f0 2023-12-01T03:13:04.646 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.644+0000 7f101bfff640 1 -- 172.21.15.115:0/2926570046 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001088590 0 0) 0x7f10280add10 con 0x7f102806f2f0 2023-12-01T03:13:04.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.645+0000 7f102d738640 1 --2- 172.21.15.115:0/2926570046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f102815e2f0 0x7f10280ca660 secure :-1 s=READY pgs=3082 cs=0 l=1 rev1=1 crypto rx=0x7f1024009870 tx=0x7f102402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c189bb9f6cef94d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:04.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.646+0000 7f101bfff640 1 -- 172.21.15.115:0/2926570046 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10240317e0 con 0x7f102815e2f0 2023-12-01T03:13:04.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.646+0000 7f101bfff640 1 -- 172.21.15.115:0/2926570046 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1024031940 con 0x7f102815e2f0 2023-12-01T03:13:04.647 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.646+0000 7f101bfff640 1 -- 172.21.15.115:0/2926570046 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024031cd0 con 0x7f102815e2f0 2023-12-01T03:13:04.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.646+0000 7f102f99c640 1 -- 172.21.15.115:0/2926570046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f102815e2f0 msgr2=0x7f10280ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.646+0000 7f102f99c640 1 --2- 172.21.15.115:0/2926570046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f102815e2f0 0x7f10280ca660 secure :-1 s=READY pgs=3082 cs=0 l=1 rev1=1 crypto rx=0x7f1024009870 tx=0x7f102402b880 comp rx=0 tx=0).stop 2023-12-01T03:13:04.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.646+0000 7f102f99c640 1 -- 172.21.15.115:0/2926570046 shutdown_connections 2023-12-01T03:13:04.648 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.646+0000 7f102f99c640 1 --2- 172.21.15.115:0/2926570046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f102815e2f0 0x7f10280ca660 unknown :-1 s=CLOSED pgs=3082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.646+0000 7f102f99c640 1 -- 172.21.15.115:0/2926570046 >> 172.21.15.115:0/2926570046 conn(0x7f10280ce880 msgr2=0x7f10280cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:04.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.646+0000 7f102f99c640 1 -- 172.21.15.115:0/2926570046 shutdown_connections 2023-12-01T03:13:04.649 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.646+0000 7f102f99c640 1 -- 172.21.15.115:0/2926570046 wait complete. 2023-12-01T03:13:04.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.648+0000 7f102f99c640 1 Processor -- start 2023-12-01T03:13:04.650 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.648+0000 7f102f99c640 1 -- start start 2023-12-01T03:13:04.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.649+0000 7f102f99c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f102806f2f0 0x7f1028155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.649+0000 7f102f99c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280adf10 0x7f102814d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.649+0000 7f102f99c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f102815e2f0 0x7f10280ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.649+0000 7f102f99c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028156490 con 0x7f10280adf10 2023-12-01T03:13:04.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.649+0000 7f102f99c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028156700 con 0x7f102815e2f0 2023-12-01T03:13:04.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.649+0000 7f102f99c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10281569e0 con 0x7f102806f2f0 2023-12-01T03:13:04.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.649+0000 7f102cf37640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280adf10 0x7f102814d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:04.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.649+0000 7f102cf37640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280adf10 0x7f102814d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50788/0 (socket says 172.21.15.115:50788) 2023-12-01T03:13:04.651 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.649+0000 7f102cf37640 1 -- 172.21.15.115:0/429675799 learned_addr learned my addr 172.21.15.115:0/429675799 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f102df39640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f102815e2f0 0x7f10280ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f102cf37640 1 -- 172.21.15.115:0/429675799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f102806f2f0 msgr2=0x7f1028155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f102cf37640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f102806f2f0 0x7f1028155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f102cf37640 1 -- 172.21.15.115:0/429675799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f102815e2f0 msgr2=0x7f10280ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f102cf37640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f102815e2f0 0x7f10280ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f102cf37640 1 -- 172.21.15.115:0/429675799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240094d0 con 0x7f10280adf10 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f102cf37640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280adf10 0x7f102814d330 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f101c00c430 tx=0x7f101c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f1019ffb640 1 -- 172.21.15.115:0/429675799 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c0074e0 con 0x7f10280adf10 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f1019ffb640 1 -- 172.21.15.115:0/429675799 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c007660 con 0x7f10280adf10 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f1019ffb640 1 -- 172.21.15.115:0/429675799 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c004a20 con 0x7f10280adf10 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.650+0000 7f102f99c640 1 -- 172.21.15.115:0/429675799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1028156cc0 con 0x7f10280adf10 2023-12-01T03:13:04.653 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.651+0000 7f102f99c640 1 -- 172.21.15.115:0/429675799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1028157280 con 0x7f10280adf10 2023-12-01T03:13:04.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.652+0000 7f102f99c640 1 -- 172.21.15.115:0/429675799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f102806fda0 con 0x7f10280adf10 2023-12-01T03:13:04.654 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.653+0000 7f1019ffb640 1 -- 172.21.15.115:0/429675799 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 149) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f101c004b80 con 0x7f10280adf10 2023-12-01T03:13:04.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.653+0000 7f1019ffb640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f100c06a280 0x7f100c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:04.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.654+0000 7f1019ffb640 1 -- 172.21.15.115:0/429675799 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f101c088680 con 0x7f10280adf10 2023-12-01T03:13:04.655 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.654+0000 7f102d738640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f100c06a280 0x7f100c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:04.656 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.654+0000 7f102d738640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f100c06a280 0x7f100c06c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1024036000 tx=0x7f1024030000 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:04.659 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.658+0000 7f1019ffb640 1 -- 172.21.15.115:0/429675799 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f101c053ca0 con 0x7f10280adf10 2023-12-01T03:13:04.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.848+0000 7f102f99c640 1 -- 172.21.15.115:0/429675799 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f102804e890 con 0x7f10280adf10 2023-12-01T03:13:04.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.849+0000 7f1019ffb640 1 -- 172.21.15.115:0/429675799 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v258) v1 ==== 74+0+17238 (secure 0 0 0) 0x7f101c0572c0 con 0x7f10280adf10 2023-12-01T03:13:04.851 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:04.851 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":258,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:13:03.079539+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:12:33.411870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-01T03:12:34.335941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/797535177":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/1950965948":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/1670255025":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/954404245":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/1517453095":"2023-12-02T03:13:03.079506+0000","172.21.15.115:6824/138932":"2023-12-02T03:12:54.714747+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2180551612":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/2457056129":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2634160496":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2258893627":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2137126608":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/2547954996":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2121389099":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/3223789397":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/534295842":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2854101523":"2023-12-02T03:12:54.714747+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:13:04.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.851+0000 7f102f99c640 1 -- 172.21.15.115:0/429675799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f100c06a280 msgr2=0x7f100c06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.851+0000 7f102f99c640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f100c06a280 0x7f100c06c740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1024036000 tx=0x7f1024030000 comp rx=0 tx=0).stop 2023-12-01T03:13:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.851+0000 7f102f99c640 1 -- 172.21.15.115:0/429675799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280adf10 msgr2=0x7f102814d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.851+0000 7f102f99c640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280adf10 0x7f102814d330 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f101c00c430 tx=0x7f101c00c8f0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.851+0000 7f102f99c640 1 -- 172.21.15.115:0/429675799 shutdown_connections 2023-12-01T03:13:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.851+0000 7f102f99c640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f100c06a280 0x7f100c06c740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.851+0000 7f102f99c640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f102815e2f0 0x7f10280ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.851+0000 7f102f99c640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10280adf10 0x7f102814d330 unknown :-1 s=CLOSED pgs=3083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.851+0000 7f102f99c640 1 --2- 172.21.15.115:0/429675799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f102806f2f0 0x7f1028155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.852+0000 7f102f99c640 1 -- 172.21.15.115:0/429675799 >> 172.21.15.115:0/429675799 conn(0x7f10280ce880 msgr2=0x7f10280c95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.852+0000 7f102f99c640 1 -- 172.21.15.115:0/429675799 shutdown_connections 2023-12-01T03:13:04.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:04.852+0000 7f102f99c640 1 -- 172.21.15.115:0/429675799 wait complete. 2023-12-01T03:13:04.871 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T03:13:04.871 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:13:05.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.083+0000 7f6a3d0e0640 1 Processor -- start 2023-12-01T03:13:05.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.083+0000 7f6a3d0e0640 1 -- start start 2023-12-01T03:13:05.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.084+0000 7f6a3d0e0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6a380adac0 0x7f6a380cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.084+0000 7f6a3d0e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a3806b350 0x7f6a3806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.085+0000 7f6a3d0e0640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a380ce150 con 0x7f6a380c8ed0 2023-12-01T03:13:05.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.085+0000 7f6a3d0e0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a380cf3a0 con 0x7f6a380adac0 2023-12-01T03:13:05.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.085+0000 7f6a3d0e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380ce5e0 con 0x7f6a3806b350 2023-12-01T03:13:05.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.085+0000 7f6a2ffff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6a380c8ed0 0x7f6a380cc990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37250/0 (socket says 172.21.15.115:37250) 2023-12-01T03:13:05.087 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.085+0000 7f6a2ffff640 1 -- 172.21.15.115:0/3359187757 learned_addr learned my addr 172.21.15.115:0/3359187757 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.085+0000 7f6a36d9d640 1 --2- 172.21.15.115:0/3359187757 >> v2:172.21.15.142:3300/0 conn(0x7f6a380adac0 0x7f6a380cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.085+0000 7f6a3759e640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a3806b350 0x7f6a3806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.085+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432821917 0 0) 0x7f6a380ce150 con 0x7f6a380c8ed0 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.085+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a24003520 con 0x7f6a380c8ed0 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3703686883 0 0) 0x7f6a24003520 con 0x7f6a380c8ed0 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a3806b350 msgr2=0x7f6a3806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a3806b350 0x7f6a3806b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 >> v2:172.21.15.142:3300/0 conn(0x7f6a380adac0 msgr2=0x7f6a380cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 --2- 172.21.15.115:0/3359187757 >> v2:172.21.15.142:3300/0 conn(0x7f6a380adac0 0x7f6a380cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3806d620 con 0x7f6a380c8ed0 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f6a20001670 con 0x7f6a380c8ed0 2023-12-01T03:13:05.089 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 >> v1:172.21.15.115:6789/0 conn(0x7f6a380c8ed0 legacy=0x7f6a380cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a240039e0 0x7f6a24003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a3806b350 0x7f6a240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a24004d00 0x7f6a24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380cf3a0 con 0x7f6a240039e0 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380ce5e0 con 0x7f6a24004d00 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2400b980 con 0x7f6a3806b350 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.086+0000 7f6a36d9d640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a240039e0 0x7f6a24003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.087+0000 7f6a36d9d640 1 -- 172.21.15.115:0/3359187757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a3806b350 msgr2=0x7f6a240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.087+0000 7f6a36d9d640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a3806b350 0x7f6a240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.087+0000 7f6a36d9d640 1 -- 172.21.15.115:0/3359187757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a24004d00 msgr2=0x7f6a24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.087+0000 7f6a3759e640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a24004d00 0x7f6a24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.087+0000 7f6a36d9d640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a24004d00 0x7f6a24005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.087+0000 7f6a36d9d640 1 -- 172.21.15.115:0/3359187757 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a24003520 con 0x7f6a240039e0 2023-12-01T03:13:05.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.088+0000 7f6a36d9d640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a240039e0 0x7f6a24003e00 secure :-1 s=READY pgs=3085 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009ac0 tx=0x7f6a240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8bf7191fea4877 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:05.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.088+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2802c710 con 0x7f6a240039e0 2023-12-01T03:13:05.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.088+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2802c870 con 0x7f6a240039e0 2023-12-01T03:13:05.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.088+0000 7f6a3659c640 1 -- 172.21.15.115:0/3359187757 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2802cc00 con 0x7f6a240039e0 2023-12-01T03:13:05.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.089+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/3359187757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a240039e0 msgr2=0x7f6a24003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.089+0000 7f6a3d0e0640 1 --2- 172.21.15.115:0/3359187757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a240039e0 0x7f6a24003e00 secure :-1 s=READY pgs=3085 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009ac0 tx=0x7f6a240057f0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.089+0000 7f6a3759e640 1 -- 172.21.15.115:0/3359187757 reap_dead start 2023-12-01T03:13:05.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.089+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/3359187757 shutdown_connections 2023-12-01T03:13:05.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.089+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/3359187757 >> 172.21.15.115:0/3359187757 conn(0x7f6a380ce880 msgr2=0x7f6a380cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:05.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.089+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/3359187757 shutdown_connections 2023-12-01T03:13:05.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.089+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/3359187757 wait complete. 2023-12-01T03:13:05.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.091+0000 7f6a3d0e0640 1 Processor -- start 2023-12-01T03:13:05.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.091+0000 7f6a3d0e0640 1 -- start start 2023-12-01T03:13:05.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3d0e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a240039e0 0x7f6a380c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3d0e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a3806bd60 0x7f6a3806c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3d0e0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a380c46d0 0x7f6a380c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3d0e0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a381faff0 con 0x7f6a380c46d0 2023-12-01T03:13:05.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3d0e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380c5440 con 0x7f6a3806bd60 2023-12-01T03:13:05.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3d0e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3806c720 con 0x7f6a240039e0 2023-12-01T03:13:05.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a380c46d0 0x7f6a380c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a380c46d0 0x7f6a380c4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50794/0 (socket says 172.21.15.115:50794) 2023-12-01T03:13:05.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3759e640 1 -- 172.21.15.115:0/2522820881 learned_addr learned my addr 172.21.15.115:0/2522820881 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:05.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a36d9d640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a240039e0 0x7f6a380c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a2ffff640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a3806bd60 0x7f6a3806c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3759e640 1 -- 172.21.15.115:0/2522820881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a240039e0 msgr2=0x7f6a380c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3759e640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a240039e0 0x7f6a380c29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3759e640 1 -- 172.21.15.115:0/2522820881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a3806bd60 msgr2=0x7f6a3806c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3759e640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a3806bd60 0x7f6a3806c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a3759e640 1 -- 172.21.15.115:0/2522820881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a381fab60 con 0x7f6a380c46d0 2023-12-01T03:13:05.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a36d9d640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a240039e0 0x7f6a380c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:05.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.092+0000 7f6a2ffff640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a3806bd60 0x7f6a3806c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:05.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.093+0000 7f6a3759e640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a380c46d0 0x7f6a380c4b80 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7f6a3806ed00 tx=0x7f6a3000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:05.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.094+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/2522820881 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3000fe60 con 0x7f6a380c46d0 2023-12-01T03:13:05.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.094+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/2522820881 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a30011040 con 0x7f6a380c46d0 2023-12-01T03:13:05.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.094+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/2522820881 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30003bd0 con 0x7f6a380c46d0 2023-12-01T03:13:05.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.094+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/2522820881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a381fa230 con 0x7f6a380c46d0 2023-12-01T03:13:05.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.094+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/2522820881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a381fa7f0 con 0x7f6a380c46d0 2023-12-01T03:13:05.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.096+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/2522820881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a381facf0 con 0x7f6a380c46d0 2023-12-01T03:13:05.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.096+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/2522820881 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 149) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f6a30003d30 con 0x7f6a380c46d0 2023-12-01T03:13:05.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.098+0000 7f6a2f7fe640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2b0 0x7f6a1806c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.098+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/2522820881 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f6a30088b20 con 0x7f6a380c46d0 2023-12-01T03:13:05.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.098+0000 7f6a36d9d640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2b0 0x7f6a1806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.099+0000 7f6a36d9d640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2b0 0x7f6a1806c770 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a28036000 tx=0x7f6a28030000 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:05.104 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.102+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/2522820881 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6a30054140 con 0x7f6a380c46d0 2023-12-01T03:13:05.336 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.334+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/2522820881 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a3804e890 con 0x7f6a380c46d0 2023-12-01T03:13:05.337 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.335+0000 7f6a2f7fe640 1 -- 172.21.15.115:0/2522820881 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f6a30057760 con 0x7f6a380c46d0 2023-12-01T03:13:05.337 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:05.337 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":126,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2042114371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2042114371},{"type":"v1","addr":"172.21.15.142:6835","nonce":2042114371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/727621046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":727621046},{"type":"v1","addr":"172.21.15.115:6835","nonce":727621046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3018042052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3018042052},{"type":"v1","addr":"172.21.15.115:6837","nonce":3018042052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:12:35.317383+0000","modified":"2023-12-01T03:12:38.475270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12246":{"gid":12246,"name":"b","rank":0,"incarnation":122,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/1356734431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1356734431},{"type":"v1","addr":"172.21.15.142:6834","nonce":1356734431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-01T03:13:05.337 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 126 2023-12-01T03:13:05.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/2522820881 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2b0 msgr2=0x7f6a1806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2b0 0x7f6a1806c770 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a28036000 tx=0x7f6a28030000 comp rx=0 tx=0).stop 2023-12-01T03:13:05.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/2522820881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a380c46d0 msgr2=0x7f6a380c4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a380c46d0 0x7f6a380c4b80 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7f6a3806ed00 tx=0x7f6a3000f4a0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/2522820881 shutdown_connections 2023-12-01T03:13:05.340 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6a1806a2b0 0x7f6a1806c770 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a380c46d0 0x7f6a380c4b80 unknown :-1 s=CLOSED pgs=3086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a3806bd60 0x7f6a3806c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 --2- 172.21.15.115:0/2522820881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a240039e0 0x7f6a380c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/2522820881 >> 172.21.15.115:0/2522820881 conn(0x7f6a380ce880 msgr2=0x7f6a38160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:05.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.338+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/2522820881 shutdown_connections 2023-12-01T03:13:05.341 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.339+0000 7f6a3d0e0640 1 -- 172.21.15.115:0/2522820881 wait complete. 2023-12-01T03:13:05.359 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:13:05.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.574+0000 7f6da2373640 1 Processor -- start 2023-12-01T03:13:05.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.574+0000 7f6da2373640 1 -- start start 2023-12-01T03:13:05.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.575+0000 7f6da2373640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6d9c0cad70 0x7f6d9c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.575+0000 7f6da2373640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f6d9c06f2f0 0x7f6d9c06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.575+0000 7f6da2373640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9c0add10 con 0x7f6d9c15e2f0 2023-12-01T03:13:05.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.575+0000 7f6da2373640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9c0c5760 con 0x7f6d9c06f2f0 2023-12-01T03:13:05.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.575+0000 7f6da2373640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9c15e6d0 con 0x7f6d9c0cad70 2023-12-01T03:13:05.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.575+0000 7f6d9bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f6d9c15e2f0 0x7f6d9c0ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37262/0 (socket says 172.21.15.115:37262) 2023-12-01T03:13:05.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.576+0000 7f6d9bfff640 1 -- 172.21.15.115:0/3635754272 learned_addr learned my addr 172.21.15.115:0/3635754272 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:05.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.576+0000 7f6da0910640 1 --2- 172.21.15.115:0/3635754272 >> v2:172.21.15.115:3300/0 conn(0x7f6d9c06f2f0 0x7f6d9c06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.576+0000 7f6d9affd640 1 -- 172.21.15.115:0/3635754272 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784617343 0 0) 0x7f6d9c0add10 con 0x7f6d9c15e2f0 2023-12-01T03:13:05.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.576+0000 7f6d9b7fe640 1 --2- 172.21.15.115:0/3635754272 >> v2:172.21.15.142:3300/0 conn(0x7f6d9c0cad70 0x7f6d9c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.576+0000 7f6d9affd640 1 -- 172.21.15.115:0/3635754272 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d84003520 con 0x7f6d9c15e2f0 2023-12-01T03:13:05.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.576+0000 7f6da0910640 1 -- 172.21.15.115:0/3635754272 >> v2:172.21.15.142:3300/0 conn(0x7f6d9c0cad70 msgr2=0x7f6d9c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:05.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.576+0000 7f6da0910640 1 --2- 172.21.15.115:0/3635754272 >> v2:172.21.15.142:3300/0 conn(0x7f6d9c0cad70 0x7f6d9c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.576+0000 7f6da0910640 1 -- 172.21.15.115:0/3635754272 >> v1:172.21.15.115:6789/0 conn(0x7f6d9c15e2f0 legacy=0x7f6d9c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.576+0000 7f6da0910640 1 -- 172.21.15.115:0/3635754272 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0cf7b0 con 0x7f6d9c06f2f0 2023-12-01T03:13:05.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.576+0000 7f6d9b7fe640 1 --2- 172.21.15.115:0/3635754272 >> v2:172.21.15.142:3300/0 conn(0x7f6d9c0cad70 0x7f6d9c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:05.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.577+0000 7f6da0910640 1 --2- 172.21.15.115:0/3635754272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c06f2f0 0x7f6d9c06bb00 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009d90 tx=0x7f6d9402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699aee35ceaff088 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:05.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.577+0000 7f6d9affd640 1 -- 172.21.15.115:0/3635754272 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9402cb40 con 0x7f6d9c06f2f0 2023-12-01T03:13:05.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.577+0000 7f6d9affd640 1 -- 172.21.15.115:0/3635754272 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9402cca0 con 0x7f6d9c06f2f0 2023-12-01T03:13:05.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.577+0000 7f6d9affd640 1 -- 172.21.15.115:0/3635754272 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94034660 con 0x7f6d9c06f2f0 2023-12-01T03:13:05.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.578+0000 7f6da2373640 1 -- 172.21.15.115:0/3635754272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c06f2f0 msgr2=0x7f6d9c06bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.578+0000 7f6da2373640 1 --2- 172.21.15.115:0/3635754272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c06f2f0 0x7f6d9c06bb00 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009d90 tx=0x7f6d9402bd90 comp rx=0 tx=0).stop 2023-12-01T03:13:05.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.578+0000 7f6da2373640 1 -- 172.21.15.115:0/3635754272 shutdown_connections 2023-12-01T03:13:05.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.578+0000 7f6da2373640 1 --2- 172.21.15.115:0/3635754272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c06f2f0 0x7f6d9c06bb00 unknown :-1 s=CLOSED pgs=3090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.578+0000 7f6da2373640 1 --2- 172.21.15.115:0/3635754272 >> v2:172.21.15.142:3300/0 conn(0x7f6d9c0cad70 0x7f6d9c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.578+0000 7f6da2373640 1 -- 172.21.15.115:0/3635754272 >> 172.21.15.115:0/3635754272 conn(0x7f6d9c0ce880 msgr2=0x7f6d9c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:05.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.578+0000 7f6da2373640 1 -- 172.21.15.115:0/3635754272 shutdown_connections 2023-12-01T03:13:05.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.578+0000 7f6da2373640 1 -- 172.21.15.115:0/3635754272 wait complete. 2023-12-01T03:13:05.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.580+0000 7f6da2373640 1 Processor -- start 2023-12-01T03:13:05.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.580+0000 7f6da2373640 1 -- start start 2023-12-01T03:13:05.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.581+0000 7f6da2373640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9c06f2f0 0x7f6d9c14bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.581+0000 7f6da2373640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c0cad70 0x7f6d9c0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.581+0000 7f6da2373640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9c15e2f0 0x7f6d9c0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.581+0000 7f6da2373640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0c62f0 con 0x7f6d9c0cad70 2023-12-01T03:13:05.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.581+0000 7f6da2373640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0c7bc0 con 0x7f6d9c15e2f0 2023-12-01T03:13:05.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.581+0000 7f6da2373640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0c5950 con 0x7f6d9c06f2f0 2023-12-01T03:13:05.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.581+0000 7f6d9b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c0cad70 0x7f6d9c0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.581+0000 7f6d9b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c0cad70 0x7f6d9c0c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50824/0 (socket says 172.21.15.115:50824) 2023-12-01T03:13:05.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.581+0000 7f6d9b7fe640 1 -- 172.21.15.115:0/101159566 learned_addr learned my addr 172.21.15.115:0/101159566 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:05.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6d9bfff640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9c06f2f0 0x7f6d9c14bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6da0910640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9c15e2f0 0x7f6d9c0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6d9b7fe640 1 -- 172.21.15.115:0/101159566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9c06f2f0 msgr2=0x7f6d9c14bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6d9b7fe640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9c06f2f0 0x7f6d9c14bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6d9b7fe640 1 -- 172.21.15.115:0/101159566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9c15e2f0 msgr2=0x7f6d9c0c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6d9b7fe640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9c15e2f0 0x7f6d9c0c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6d9b7fe640 1 -- 172.21.15.115:0/101159566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d94009a70 con 0x7f6d9c0cad70 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6d9b7fe640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c0cad70 0x7f6d9c0c71e0 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c7f0 tx=0x7f6d9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6d98ff9640 1 -- 172.21.15.115:0/101159566 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90007820 con 0x7f6d9c0cad70 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6d98ff9640 1 -- 172.21.15.115:0/101159566 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d90007980 con 0x7f6d9c0cad70 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6d98ff9640 1 -- 172.21.15.115:0/101159566 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90007cf0 con 0x7f6d9c0cad70 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6da2373640 1 -- 172.21.15.115:0/101159566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9c0c5c30 con 0x7f6d9c0cad70 2023-12-01T03:13:05.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.582+0000 7f6da2373640 1 -- 172.21.15.115:0/101159566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c0c9440 con 0x7f6d9c0cad70 2023-12-01T03:13:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.583+0000 7f6da2373640 1 -- 172.21.15.115:0/101159566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9c06fda0 con 0x7f6d9c0cad70 2023-12-01T03:13:05.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.584+0000 7f6d98ff9640 1 -- 172.21.15.115:0/101159566 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 149) v1 ==== 88445+0+0 (secure 0 0 0) 0x7f6d900113f0 con 0x7f6d9c0cad70 2023-12-01T03:13:05.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.585+0000 7f6d98ff9640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6d8806a280 0x7f6d8806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:05.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.585+0000 7f6d98ff9640 1 -- 172.21.15.115:0/101159566 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f6d90088c40 con 0x7f6d9c0cad70 2023-12-01T03:13:05.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.586+0000 7f6d9bfff640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6d8806a280 0x7f6d8806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:05.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.586+0000 7f6d9bfff640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6d8806a280 0x7f6d8806c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000f60 tx=0x7f6d8c009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:05.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.589+0000 7f6d98ff9640 1 -- 172.21.15.115:0/101159566 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6d90054190 con 0x7f6d9c0cad70 2023-12-01T03:13:05.817 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.815+0000 7f6da2373640 1 -- 172.21.15.115:0/101159566 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d9c0c6430 con 0x7f6d9c0cad70 2023-12-01T03:13:05.818 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.817+0000 7f6d98ff9640 1 -- 172.21.15.115:0/101159566 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+4672 (secure 0 0 0) 0x7f6d9c0c6430 con 0x7f6d9c0cad70 2023-12-01T03:13:05.819 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:05.819 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":126,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2042114371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2042114371},{"type":"v1","addr":"172.21.15.142:6835","nonce":2042114371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/727621046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":727621046},{"type":"v1","addr":"172.21.15.115:6835","nonce":727621046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3018042052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3018042052},{"type":"v1","addr":"172.21.15.115:6837","nonce":3018042052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-01T03:12:35.317383+0000","modified":"2023-12-01T03:12:38.475270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12246":{"gid":12246,"name":"b","rank":0,"incarnation":122,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6834/1356734431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1356734431},{"type":"v1","addr":"172.21.15.142:6834","nonce":1356734431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-01T03:13:05.819 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 126 2023-12-01T03:13:05.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.819+0000 7f6da2373640 1 -- 172.21.15.115:0/101159566 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6d8806a280 msgr2=0x7f6d8806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.819+0000 7f6da2373640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6d8806a280 0x7f6d8806c740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000f60 tx=0x7f6d8c009040 comp rx=0 tx=0).stop 2023-12-01T03:13:05.821 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.819+0000 7f6da2373640 1 -- 172.21.15.115:0/101159566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c0cad70 msgr2=0x7f6d9c0c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.819+0000 7f6da2373640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c0cad70 0x7f6d9c0c71e0 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c7f0 tx=0x7f6d9000ccb0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.820+0000 7f6da2373640 1 -- 172.21.15.115:0/101159566 shutdown_connections 2023-12-01T03:13:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.820+0000 7f6da2373640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f6d8806a280 0x7f6d8806c740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.820+0000 7f6da2373640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d9c15e2f0 0x7f6d9c0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.820+0000 7f6da2373640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d9c0cad70 0x7f6d9c0c71e0 unknown :-1 s=CLOSED pgs=3091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.820+0000 7f6da2373640 1 --2- 172.21.15.115:0/101159566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d9c06f2f0 0x7f6d9c14bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.820+0000 7f6da2373640 1 -- 172.21.15.115:0/101159566 >> 172.21.15.115:0/101159566 conn(0x7f6d9c0ce880 msgr2=0x7f6d9c0cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.820+0000 7f6da2373640 1 -- 172.21.15.115:0/101159566 shutdown_connections 2023-12-01T03:13:05.822 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:05.820+0000 7f6da2373640 1 -- 172.21.15.115:0/101159566 wait complete. 2023-12-01T03:13:05.841 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:13:06.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.046+0000 7f5d42500640 1 Processor -- start 2023-12-01T03:13:06.048 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.046+0000 7f5d42500640 1 -- start start 2023-12-01T03:13:06.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d42500640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5d3c0cad70 0x7f5d3c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d42500640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c0add10 con 0x7f5d3c15e2f0 2023-12-01T03:13:06.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d42500640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c0c5760 con 0x7f5d3c06f2f0 2023-12-01T03:13:06.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d42500640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3c15e6d0 con 0x7f5d3c0cad70 2023-12-01T03:13:06.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d3b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5d3c0cad70 0x7f5d3c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d3bfff640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f5d3c15e2f0 0x7f5d3c0ca660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37266/0 (socket says 172.21.15.115:37266) 2023-12-01T03:13:06.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d3b7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f5d3c0cad70 0x7f5d3c06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50840/0 (socket says 172.21.15.115:50840) 2023-12-01T03:13:06.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d3bfff640 1 -- 172.21.15.115:0/1668664956 learned_addr learned my addr 172.21.15.115:0/1668664956 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:06.050 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d3b7fe640 1 -- 172.21.15.115:0/1668664956 >> v1:172.21.15.142:6789/0 conn(0x7f5d3c06f2f0 legacy=0x7f5d3c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:06.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d3b7fe640 1 -- 172.21.15.115:0/1668664956 >> v1:172.21.15.115:6789/0 conn(0x7f5d3c15e2f0 legacy=0x7f5d3c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.048+0000 7f5d3b7fe640 1 -- 172.21.15.115:0/1668664956 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0cf7b0 con 0x7f5d3c0cad70 2023-12-01T03:13:06.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.049+0000 7f5d3b7fe640 1 --2- 172.21.15.115:0/1668664956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 0x7f5d3c06b5c0 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 crypto rx=0x7f5d300097d0 tx=0x7f5d3002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d920726c616471 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:06.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.050+0000 7f5d3affd640 1 -- 172.21.15.115:0/1668664956 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d300316f0 con 0x7f5d3c0cad70 2023-12-01T03:13:06.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.050+0000 7f5d3affd640 1 -- 172.21.15.115:0/1668664956 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d30031850 con 0x7f5d3c0cad70 2023-12-01T03:13:06.051 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.050+0000 7f5d3affd640 1 -- 172.21.15.115:0/1668664956 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30031be0 con 0x7f5d3c0cad70 2023-12-01T03:13:06.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.050+0000 7f5d42500640 1 -- 172.21.15.115:0/1668664956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 msgr2=0x7f5d3c06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.050+0000 7f5d42500640 1 --2- 172.21.15.115:0/1668664956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 0x7f5d3c06b5c0 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 crypto rx=0x7f5d300097d0 tx=0x7f5d3002b790 comp rx=0 tx=0).stop 2023-12-01T03:13:06.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.050+0000 7f5d42500640 1 -- 172.21.15.115:0/1668664956 shutdown_connections 2023-12-01T03:13:06.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.051+0000 7f5d42500640 1 --2- 172.21.15.115:0/1668664956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 0x7f5d3c06b5c0 unknown :-1 s=CLOSED pgs=3093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.052 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.051+0000 7f5d42500640 1 -- 172.21.15.115:0/1668664956 >> 172.21.15.115:0/1668664956 conn(0x7f5d3c0ce880 msgr2=0x7f5d3c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:06.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.051+0000 7f5d42500640 1 -- 172.21.15.115:0/1668664956 shutdown_connections 2023-12-01T03:13:06.053 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.051+0000 7f5d42500640 1 -- 172.21.15.115:0/1668664956 wait complete. 2023-12-01T03:13:06.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.053+0000 7f5d42500640 1 Processor -- start 2023-12-01T03:13:06.054 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.053+0000 7f5d42500640 1 -- start start 2023-12-01T03:13:06.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.054+0000 7f5d42500640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d3c06f2f0 0x7f5d3c157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.054+0000 7f5d42500640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 0x7f5d3c0c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.054+0000 7f5d42500640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d3c15e2f0 0x7f5d3c0cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.054+0000 7f5d42500640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c157fd0 con 0x7f5d3c0cad70 2023-12-01T03:13:06.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.054+0000 7f5d42500640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0c59e0 con 0x7f5d3c06f2f0 2023-12-01T03:13:06.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.054+0000 7f5d42500640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1f3da0 con 0x7f5d3c15e2f0 2023-12-01T03:13:06.056 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.054+0000 7f5d3b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 0x7f5d3c0c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.054+0000 7f5d3b7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 0x7f5d3c0c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50848/0 (socket says 172.21.15.115:50848) 2023-12-01T03:13:06.057 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.054+0000 7f5d3b7fe640 1 -- 172.21.15.115:0/815745525 learned_addr learned my addr 172.21.15.115:0/815745525 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.055+0000 7f5d3b7fe640 1 -- 172.21.15.115:0/815745525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d3c15e2f0 msgr2=0x7f5d3c0cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.055+0000 7f5d3b7fe640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d3c15e2f0 0x7f5d3c0cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.055+0000 7f5d3bfff640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d3c06f2f0 0x7f5d3c157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.055+0000 7f5d3b7fe640 1 -- 172.21.15.115:0/815745525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d3c06f2f0 msgr2=0x7f5d3c157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.055+0000 7f5d3b7fe640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d3c06f2f0 0x7f5d3c157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.055+0000 7f5d3b7fe640 1 -- 172.21.15.115:0/815745525 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d30009400 con 0x7f5d3c0cad70 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.055+0000 7f5d3b7fe640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 0x7f5d3c0c4c00 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 crypto rx=0x7f5d300098f0 tx=0x7f5d300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.055+0000 7f5d38ff9640 1 -- 172.21.15.115:0/815745525 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30034890 con 0x7f5d3c0cad70 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.055+0000 7f5d38ff9640 1 -- 172.21.15.115:0/815745525 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d30031f00 con 0x7f5d3c0cad70 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.055+0000 7f5d42500640 1 -- 172.21.15.115:0/815745525 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c1f4080 con 0x7f5d3c0cad70 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.056+0000 7f5d38ff9640 1 -- 172.21.15.115:0/815745525 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30034c50 con 0x7f5d3c0cad70 2023-12-01T03:13:06.058 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.056+0000 7f5d42500640 1 -- 172.21.15.115:0/815745525 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c1f4640 con 0x7f5d3c0cad70 2023-12-01T03:13:06.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.057+0000 7f5d42500640 1 -- 172.21.15.115:0/815745525 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c06fda0 con 0x7f5d3c0cad70 2023-12-01T03:13:06.059 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.058+0000 7f5d38ff9640 1 -- 172.21.15.115:0/815745525 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 150) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f5d3003b040 con 0x7f5d3c0cad70 2023-12-01T03:13:06.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.058+0000 7f5d38ff9640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5d2806a4c0 0x7f5d2806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.059+0000 7f5d38ff9640 1 -- 172.21.15.115:0/815745525 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f5d300aaba0 con 0x7f5d3c0cad70 2023-12-01T03:13:06.060 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.059+0000 7f5d3bfff640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5d2806a4c0 0x7f5d2806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.059+0000 7f5d3bfff640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5d2806a4c0 0x7f5d2806c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c000f60 tx=0x7f5d2c009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:06.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.063+0000 7f5d38ff9640 1 -- 172.21.15.115:0/815745525 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f5d30075ea0 con 0x7f5d3c0cad70 2023-12-01T03:13:06.257 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.255+0000 7f5d42500640 1 -- 172.21.15.115:0/815745525 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d3c04e890 con 0x7f5d3c0cad70 2023-12-01T03:13:06.259 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.258+0000 7f5d38ff9640 1 -- 172.21.15.115:0/815745525 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v258) v1 ==== 74+0+17238 (secure 0 0 0) 0x7f5d300794c0 con 0x7f5d3c0cad70 2023-12-01T03:13:06.260 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:06.260 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":258,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:13:03.079539+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-01T03:12:33.411870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-01T03:12:34.335941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/797535177":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/1950965948":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/1670255025":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/954404245":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/1517453095":"2023-12-02T03:13:03.079506+0000","172.21.15.115:6824/138932":"2023-12-02T03:12:54.714747+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2180551612":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/2457056129":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2634160496":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2258893627":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2137126608":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/2547954996":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2121389099":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/3223789397":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/534295842":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2854101523":"2023-12-02T03:12:54.714747+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:13:06.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 -- 172.21.15.115:0/815745525 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5d2806a4c0 msgr2=0x7f5d2806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5d2806a4c0 0x7f5d2806c980 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c000f60 tx=0x7f5d2c009040 comp rx=0 tx=0).stop 2023-12-01T03:13:06.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 -- 172.21.15.115:0/815745525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 msgr2=0x7f5d3c0c4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 0x7f5d3c0c4c00 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 crypto rx=0x7f5d300098f0 tx=0x7f5d300046c0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 -- 172.21.15.115:0/815745525 shutdown_connections 2023-12-01T03:13:06.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f5d2806a4c0 0x7f5d2806c980 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.263 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d3c15e2f0 0x7f5d3c0cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d3c0cad70 0x7f5d3c0c4c00 unknown :-1 s=CLOSED pgs=3094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 --2- 172.21.15.115:0/815745525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d3c06f2f0 0x7f5d3c157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 -- 172.21.15.115:0/815745525 >> 172.21.15.115:0/815745525 conn(0x7f5d3c0ce880 msgr2=0x7f5d3c0c1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:06.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 -- 172.21.15.115:0/815745525 shutdown_connections 2023-12-01T03:13:06.264 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.261+0000 7f5d42500640 1 -- 172.21.15.115:0/815745525 wait complete. 2023-12-01T03:13:06.281 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T03:13:06.490 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.489+0000 7f69316f7640 1 Processor -- start 2023-12-01T03:13:06.491 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.489+0000 7f69316f7640 1 -- start start 2023-12-01T03:13:06.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.490+0000 7f69316f7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f692c16a140 0x7f692c16c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.491+0000 7f69316f7640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692c0ce150 con 0x7f692c0adac0 2023-12-01T03:13:06.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.491+0000 7f69316f7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692c0cd270 con 0x7f692c0c8ed0 2023-12-01T03:13:06.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.491+0000 7f69316f7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692c0cf3a0 con 0x7f692c16a140 2023-12-01T03:13:06.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.491+0000 7f692affd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f692c0adac0 0x7f692c0cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37272/0 (socket says 172.21.15.115:37272) 2023-12-01T03:13:06.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.491+0000 7f692affd640 1 -- 172.21.15.115:0/2226550667 learned_addr learned my addr 172.21.15.115:0/2226550667 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.491+0000 7f692b7fe640 1 --2- 172.21.15.115:0/2226550667 >> v2:172.21.15.142:3300/0 conn(0x7f692c16a140 0x7f692c16c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.491+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777240948 0 0) 0x7f692c0ce150 con 0x7f692c0adac0 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.491+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6918003520 con 0x7f692c0adac0 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 86710242 0 0) 0x7f6918003520 con 0x7f692c0adac0 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 >> v2:172.21.15.142:3300/0 conn(0x7f692c16a140 msgr2=0x7f692c16c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 --2- 172.21.15.115:0/2226550667 >> v2:172.21.15.142:3300/0 conn(0x7f692c16a140 0x7f692c16c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 >> v1:172.21.15.142:6789/0 conn(0x7f692c0c8ed0 legacy=0x7f692c0ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c06d620 con 0x7f692c0adac0 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f691c001670 con 0x7f692c0adac0 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 >> v1:172.21.15.115:6789/0 conn(0x7f692c0adac0 legacy=0x7f692c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 --2- 172.21.15.115:0/2226550667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69180039e0 0x7f6918003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 --2- 172.21.15.115:0/2226550667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69180047c0 0x7f6918004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 --2- 172.21.15.115:0/2226550667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6918005180 0x7f691800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0cf3a0 con 0x7f69180047c0 2023-12-01T03:13:06.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691800c0b0 con 0x7f69180039e0 2023-12-01T03:13:06.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691800c270 con 0x7f6918005180 2023-12-01T03:13:06.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.492+0000 7f69227fc640 1 --2- 172.21.15.115:0/2226550667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69180047c0 0x7f6918004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.493+0000 7f692affd640 1 --2- 172.21.15.115:0/2226550667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69180039e0 0x7f6918003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.493+0000 7f69227fc640 1 -- 172.21.15.115:0/2226550667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6918005180 msgr2=0x7f691800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:06.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.493+0000 7f69227fc640 1 --2- 172.21.15.115:0/2226550667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6918005180 0x7f691800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.493+0000 7f69227fc640 1 -- 172.21.15.115:0/2226550667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69180039e0 msgr2=0x7f6918003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:06.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.493+0000 7f69227fc640 1 --2- 172.21.15.115:0/2226550667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69180039e0 0x7f6918003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.493+0000 7f69227fc640 1 -- 172.21.15.115:0/2226550667 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6918003520 con 0x7f69180047c0 2023-12-01T03:13:06.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.493+0000 7f692b7fe640 1 -- 172.21.15.115:0/2226550667 reap_dead start 2023-12-01T03:13:06.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.494+0000 7f69227fc640 1 --2- 172.21.15.115:0/2226550667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69180047c0 0x7f6918004c40 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7f6914001750 tx=0x7f69180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15af84d315ddff5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:06.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.494+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69140052d0 con 0x7f69180047c0 2023-12-01T03:13:06.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.494+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691402e050 con 0x7f69180047c0 2023-12-01T03:13:06.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.494+0000 7f692a7fc640 1 -- 172.21.15.115:0/2226550667 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914003830 con 0x7f69180047c0 2023-12-01T03:13:06.497 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.494+0000 7f69316f7640 1 -- 172.21.15.115:0/2226550667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69180047c0 msgr2=0x7f6918004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.494+0000 7f69316f7640 1 --2- 172.21.15.115:0/2226550667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69180047c0 0x7f6918004c40 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7f6914001750 tx=0x7f69180057f0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.495+0000 7f69316f7640 1 -- 172.21.15.115:0/2226550667 shutdown_connections 2023-12-01T03:13:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.495+0000 7f69316f7640 1 --2- 172.21.15.115:0/2226550667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69180047c0 0x7f6918004c40 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.495+0000 7f69316f7640 1 -- 172.21.15.115:0/2226550667 >> 172.21.15.115:0/2226550667 conn(0x7f692c0ce880 msgr2=0x7f692c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.495+0000 7f69316f7640 1 -- 172.21.15.115:0/2226550667 shutdown_connections 2023-12-01T03:13:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.495+0000 7f69316f7640 1 -- 172.21.15.115:0/2226550667 wait complete. 2023-12-01T03:13:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.496+0000 7f69316f7640 1 Processor -- start 2023-12-01T03:13:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.497+0000 7f69316f7640 1 -- start start 2023-12-01T03:13:06.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.497+0000 7f69316f7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69180047c0 0x7f692c146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.497+0000 7f69316f7640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c147130 0x7f692c149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69316f7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692c1475b0 0x7f692c06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69316f7640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c15b240 con 0x7f692c147130 2023-12-01T03:13:06.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69316f7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c15b7c0 con 0x7f692c1475b0 2023-12-01T03:13:06.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69316f7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c149c80 con 0x7f69180047c0 2023-12-01T03:13:06.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69227fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c147130 0x7f692c149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69227fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c147130 0x7f692c149190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50860/0 (socket says 172.21.15.115:50860) 2023-12-01T03:13:06.500 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69227fc640 1 -- 172.21.15.115:0/100046346 learned_addr learned my addr 172.21.15.115:0/100046346 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:06.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f692affd640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69180047c0 0x7f692c146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f692b7fe640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692c1475b0 0x7f692c06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69227fc640 1 -- 172.21.15.115:0/100046346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69180047c0 msgr2=0x7f692c146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69227fc640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69180047c0 0x7f692c146850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69227fc640 1 -- 172.21.15.115:0/100046346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692c1475b0 msgr2=0x7f692c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69227fc640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692c1475b0 0x7f692c06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f69227fc640 1 -- 172.21.15.115:0/100046346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c149f60 con 0x7f692c147130 2023-12-01T03:13:06.501 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.498+0000 7f692b7fe640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692c1475b0 0x7f692c06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:06.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.499+0000 7f69227fc640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c147130 0x7f692c149190 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 crypto rx=0x7f691402bc00 tx=0x7f6914001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:06.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.499+0000 7f6923fff640 1 -- 172.21.15.115:0/100046346 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69140076c0 con 0x7f692c147130 2023-12-01T03:13:06.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.499+0000 7f6923fff640 1 -- 172.21.15.115:0/100046346 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6914007a30 con 0x7f692c147130 2023-12-01T03:13:06.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.499+0000 7f69316f7640 1 -- 172.21.15.115:0/100046346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c15b450 con 0x7f692c147130 2023-12-01T03:13:06.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.499+0000 7f6923fff640 1 -- 172.21.15.115:0/100046346 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914006be0 con 0x7f692c147130 2023-12-01T03:13:06.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.499+0000 7f69316f7640 1 -- 172.21.15.115:0/100046346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c146d90 con 0x7f692c147130 2023-12-01T03:13:06.502 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.500+0000 7f69316f7640 1 -- 172.21.15.115:0/100046346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c04e890 con 0x7f692c147130 2023-12-01T03:13:06.503 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.501+0000 7f6923fff640 1 -- 172.21.15.115:0/100046346 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 150) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f6914006d40 con 0x7f692c147130 2023-12-01T03:13:06.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.502+0000 7f6923fff640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f690c06a4c0 0x7f690c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.502+0000 7f6923fff640 1 -- 172.21.15.115:0/100046346 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6484+0+0 (secure 0 0 0) 0x7f69140aa940 con 0x7f692c147130 2023-12-01T03:13:06.504 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.502+0000 7f692affd640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f690c06a4c0 0x7f690c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.505 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.503+0000 7f692affd640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f690c06a4c0 0x7f690c06c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f691c001050 tx=0x7f691c009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:06.508 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.507+0000 7f6923fff640 1 -- 172.21.15.115:0/100046346 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f6914075b20 con 0x7f692c147130 2023-12-01T03:13:06.734 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.732+0000 7f69316f7640 1 -- 172.21.15.115:0/100046346 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f692c14a150 con 0x7f692c147130 2023-12-01T03:13:06.735 INFO:tasks.ceph.mon.a.smithi115.stderr:2023-12-01T03:13:06.734+0000 7f7e24575640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T03:13:06.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.749+0000 7f6923fff640 1 -- 172.21.15.115:0/100046346 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v127) v1 ==== 161+0+0 (secure 0 0 0) 0x7f692c14a150 con 0x7f692c147130 2023-12-01T03:13:06.751 INFO:teuthology.orchestra.run.smithi115.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T03:13:06.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.752+0000 7f69316f7640 1 -- 172.21.15.115:0/100046346 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f690c06a4c0 msgr2=0x7f690c06c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.752+0000 7f69316f7640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f690c06a4c0 0x7f690c06c980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f691c001050 tx=0x7f691c009040 comp rx=0 tx=0).stop 2023-12-01T03:13:06.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.752+0000 7f69316f7640 1 -- 172.21.15.115:0/100046346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c147130 msgr2=0x7f692c149190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.752+0000 7f69316f7640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c147130 0x7f692c149190 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 crypto rx=0x7f691402bc00 tx=0x7f6914001120 comp rx=0 tx=0).stop 2023-12-01T03:13:06.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.752+0000 7f69316f7640 1 -- 172.21.15.115:0/100046346 shutdown_connections 2023-12-01T03:13:06.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.752+0000 7f69316f7640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f690c06a4c0 0x7f690c06c980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.752+0000 7f69316f7640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f692c1475b0 0x7f692c06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.752+0000 7f69316f7640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c147130 0x7f692c149190 unknown :-1 s=CLOSED pgs=3097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.752+0000 7f69316f7640 1 --2- 172.21.15.115:0/100046346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f69180047c0 0x7f692c146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.752+0000 7f69316f7640 1 -- 172.21.15.115:0/100046346 >> 172.21.15.115:0/100046346 conn(0x7f692c0ce880 msgr2=0x7f692c0cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:06.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.753+0000 7f69316f7640 1 -- 172.21.15.115:0/100046346 shutdown_connections 2023-12-01T03:13:06.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.753+0000 7f69316f7640 1 -- 172.21.15.115:0/100046346 wait complete. 2023-12-01T03:13:06.774 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T03:13:06.777 INFO:tasks.ceph.mds.b.smithi142.stderr: -3994> 2023-12-01T03:12:40.796+0000 7fc99fb29640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T03:13:06.869 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2023-12-01T03:13:06.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.984+0000 7f0ce517e640 1 Processor -- start 2023-12-01T03:13:06.986 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.984+0000 7f0ce517e640 1 -- start start 2023-12-01T03:13:06.987 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0ce517e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0ce0159970 0x7f0ce00ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0ce517e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce015abf0 0x7f0ce00cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0ce517e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce015a760 con 0x7f0ce0159d50 2023-12-01T03:13:06.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0ce517e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce015a1d0 con 0x7f0ce0159970 2023-12-01T03:13:06.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0ce517e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0070ee0 con 0x7f0ce015abf0 2023-12-01T03:13:06.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0cde59c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0ce0159d50 0x7f0ce00707d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:37288/0 (socket says 172.21.15.115:37288) 2023-12-01T03:13:06.988 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0cde59c640 1 -- 172.21.15.115:0/1519050569 learned_addr learned my addr 172.21.15.115:0/1519050569 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0cded9d640 1 --2- 172.21.15.115:0/1519050569 >> v2:172.21.15.142:3300/0 conn(0x7f0ce0159970 0x7f0ce00ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3695592577 0 0) 0x7f0ce015a760 con 0x7f0ce0159d50 2023-12-01T03:13:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0cdf59e640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce015abf0 0x7f0ce00cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.986+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cc8003520 con 0x7f0ce0159d50 2023-12-01T03:13:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 369211043 0 0) 0x7f0cc8003520 con 0x7f0ce0159d50 2023-12-01T03:13:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce015abf0 msgr2=0x7f0ce00cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce015abf0 0x7f0ce00cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 >> v2:172.21.15.142:3300/0 conn(0x7f0ce0159970 msgr2=0x7f0ce00ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 --2- 172.21.15.115:0/1519050569 >> v2:172.21.15.142:3300/0 conn(0x7f0ce0159970 0x7f0ce00ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce015b060 con 0x7f0ce0159d50 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f0cd4001670 con 0x7f0ce0159d50 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 >> v1:172.21.15.115:6789/0 conn(0x7f0ce0159d50 legacy=0x7f0ce00707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80039a0 0x7f0cc8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc80047c0 0x7f0cc8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce015abf0 0x7f0cc8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce015a1d0 con 0x7f0cc80047c0 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0070ee0 con 0x7f0cc80039a0 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc800dbc0 con 0x7f0ce015abf0 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.987+0000 7f0cde59c640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc80047c0 0x7f0cc8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.988+0000 7f0cdf59e640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce015abf0 0x7f0cc8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.988+0000 7f0cded9d640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80039a0 0x7f0cc8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.988+0000 7f0cde59c640 1 -- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce015abf0 msgr2=0x7f0cc8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.988+0000 7f0cde59c640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce015abf0 0x7f0cc8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.988+0000 7f0cde59c640 1 -- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80039a0 msgr2=0x7f0cc8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:06.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.988+0000 7f0cde59c640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80039a0 0x7f0cc8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.988+0000 7f0cde59c640 1 -- 172.21.15.115:0/1519050569 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc8003520 con 0x7f0cc80047c0 2023-12-01T03:13:06.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.988+0000 7f0cded9d640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80039a0 0x7f0cc8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:06.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.989+0000 7f0cde59c640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc80047c0 0x7f0cc8004c40 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400c240 tx=0x7f0cc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90474fe1327f9993 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:06.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.989+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4005ad0 con 0x7f0cc80047c0 2023-12-01T03:13:06.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.989+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd4005c50 con 0x7f0cc80047c0 2023-12-01T03:13:06.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.989+0000 7f0cddd9b640 1 -- 172.21.15.115:0/1519050569 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd402c8d0 con 0x7f0cc80047c0 2023-12-01T03:13:06.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.990+0000 7f0ce517e640 1 -- 172.21.15.115:0/1519050569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc80047c0 msgr2=0x7f0cc8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.990+0000 7f0ce517e640 1 --2- 172.21.15.115:0/1519050569 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0cc80047c0 0x7f0cc8004c40 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400c240 tx=0x7f0cc80057f0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.990+0000 7f0cdf59e640 1 -- 172.21.15.115:0/1519050569 reap_dead start 2023-12-01T03:13:06.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.990+0000 7f0ce517e640 1 -- 172.21.15.115:0/1519050569 shutdown_connections 2023-12-01T03:13:06.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.990+0000 7f0ce517e640 1 -- 172.21.15.115:0/1519050569 >> 172.21.15.115:0/1519050569 conn(0x7f0ce0065b40 msgr2=0x7f0ce0065f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:06.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.990+0000 7f0ce517e640 1 -- 172.21.15.115:0/1519050569 shutdown_connections 2023-12-01T03:13:06.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.990+0000 7f0ce517e640 1 -- 172.21.15.115:0/1519050569 wait complete. 2023-12-01T03:13:06.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.992+0000 7f0ce517e640 1 Processor -- start 2023-12-01T03:13:06.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.992+0000 7f0ce517e640 1 -- start start 2023-12-01T03:13:06.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0ce517e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80047c0 0x7f0ce0153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0ce517e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce00ca410 0x7f0ce00ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0ce517e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce0156900 0x7f0ce0156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0ce517e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0158ad0 con 0x7f0ce0156900 2023-12-01T03:13:06.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0ce517e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00cadd0 con 0x7f0cc80047c0 2023-12-01T03:13:06.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0ce517e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0146d00 con 0x7f0ce00ca410 2023-12-01T03:13:06.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0cdf59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce0156900 0x7f0ce0156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0cdf59e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce0156900 0x7f0ce0156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:50868/0 (socket says 172.21.15.115:50868) 2023-12-01T03:13:06.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0cdf59e640 1 -- 172.21.15.115:0/1855830095 learned_addr learned my addr 172.21.15.115:0/1855830095 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:06.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0cded9d640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80047c0 0x7f0ce0153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0cde59c640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce00ca410 0x7f0ce00ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:06.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0cdf59e640 1 -- 172.21.15.115:0/1855830095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce00ca410 msgr2=0x7f0ce00ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0cdf59e640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce00ca410 0x7f0ce00ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0cdf59e640 1 -- 172.21.15.115:0/1855830095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80047c0 msgr2=0x7f0ce0153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:06.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.993+0000 7f0cdf59e640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80047c0 0x7f0ce0153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:06.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.994+0000 7f0cdf59e640 1 -- 172.21.15.115:0/1855830095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce01575d0 con 0x7f0ce0156900 2023-12-01T03:13:06.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.994+0000 7f0cdf59e640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce0156900 0x7f0ce0156d80 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 crypto rx=0x7f0ce006efd0 tx=0x7f0cd8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:06.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.994+0000 7f0cc77fe640 1 -- 172.21.15.115:0/1855830095 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8007cf0 con 0x7f0ce0156900 2023-12-01T03:13:06.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.994+0000 7f0cc77fe640 1 -- 172.21.15.115:0/1855830095 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd800f050 con 0x7f0ce0156900 2023-12-01T03:13:06.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.994+0000 7f0cc77fe640 1 -- 172.21.15.115:0/1855830095 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8014650 con 0x7f0ce0156900 2023-12-01T03:13:06.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.994+0000 7f0ce517e640 1 -- 172.21.15.115:0/1855830095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce01578b0 con 0x7f0ce0156900 2023-12-01T03:13:06.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.994+0000 7f0ce517e640 1 -- 172.21.15.115:0/1855830095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce0157e70 con 0x7f0ce0156900 2023-12-01T03:13:06.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.995+0000 7f0ce517e640 1 -- 172.21.15.115:0/1855830095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce004e890 con 0x7f0ce0156900 2023-12-01T03:13:06.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.996+0000 7f0cc77fe640 1 -- 172.21.15.115:0/1855830095 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 150) v1 ==== 88653+0+0 (secure 0 0 0) 0x7f0cd80147b0 con 0x7f0ce0156900 2023-12-01T03:13:06.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.997+0000 7f0cc77fe640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0ccc06a4f0 0x7f0ccc06c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:06.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.997+0000 7f0cc77fe640 1 -- 172.21.15.115:0/1855830095 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6570+0+0 (secure 0 0 0) 0x7f0cd80890c0 con 0x7f0ce0156900 2023-12-01T03:13:06.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.997+0000 7f0cded9d640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0ccc06a4f0 0x7f0ccc06c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:07.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:06.998+0000 7f0cded9d640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0ccc06a4f0 0x7f0ccc06c9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009b20 tx=0x7f0cd0008040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:07.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.001+0000 7f0cc77fe640 1 -- 172.21.15.115:0/1855830095 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0cd8054250 con 0x7f0ce0156900 2023-12-01T03:13:07.234 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.232+0000 7f0ce517e640 1 -- 172.21.15.115:0/1855830095 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0ce00cd910 con 0x7f0ce0156900 2023-12-01T03:13:07.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.846+0000 7f0cc77fe640 1 -- 172.21.15.115:0/1855830095 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0cd8054620 con 0x7f0ce0156900 2023-12-01T03:13:07.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.848+0000 7f0cc77fe640 1 -- 172.21.15.115:0/1855830095 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v128) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0cd8086860 con 0x7f0ce0156900 2023-12-01T03:13:07.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.852+0000 7f0ce517e640 1 -- 172.21.15.115:0/1855830095 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0ccc06a4f0 msgr2=0x7f0ccc06c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:07.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.852+0000 7f0ce517e640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0ccc06a4f0 0x7f0ccc06c9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009b20 tx=0x7f0cd0008040 comp rx=0 tx=0).stop 2023-12-01T03:13:07.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.852+0000 7f0ce517e640 1 -- 172.21.15.115:0/1855830095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce0156900 msgr2=0x7f0ce0156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:07.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.852+0000 7f0ce517e640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce0156900 0x7f0ce0156d80 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 crypto rx=0x7f0ce006efd0 tx=0x7f0cd8009cd0 comp rx=0 tx=0).stop 2023-12-01T03:13:07.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.852+0000 7f0ce517e640 1 -- 172.21.15.115:0/1855830095 shutdown_connections 2023-12-01T03:13:07.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.852+0000 7f0ce517e640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0ccc06a4f0 0x7f0ccc06c9b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:07.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.852+0000 7f0ce517e640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ce00ca410 0x7f0ce00ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:07.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.852+0000 7f0ce517e640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce0156900 0x7f0ce0156d80 unknown :-1 s=CLOSED pgs=3101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:07.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.852+0000 7f0ce517e640 1 --2- 172.21.15.115:0/1855830095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80047c0 0x7f0ce0153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:07.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.852+0000 7f0ce517e640 1 -- 172.21.15.115:0/1855830095 >> 172.21.15.115:0/1855830095 conn(0x7f0ce0065b40 msgr2=0x7f0ce006bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:07.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.853+0000 7f0ce517e640 1 -- 172.21.15.115:0/1855830095 shutdown_connections 2023-12-01T03:13:07.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:07.853+0000 7f0ce517e640 1 -- 172.21.15.115:0/1855830095 wait complete. 2023-12-01T03:13:07.874 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T03:13:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.086+0000 7fc8fdd2e640 1 Processor -- start 2023-12-01T03:13:08.088 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.087+0000 7fc8fdd2e640 1 -- start start 2023-12-01T03:13:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.088+0000 7fc8fdd2e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 0x7fc8f815e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.088+0000 7fc8fdd2e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc8f80ca7c0 0x7fc8f80cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.088+0000 7fc8fdd2e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc8f80cb6f0 0x7fc8f806f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.088+0000 7fc8fdd2e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f8159fd0 con 0x7fc8f80ca7c0 2023-12-01T03:13:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.088+0000 7fc8fdd2e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f80cbb80 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.088+0000 7fc8fdd2e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f815e570 con 0x7fc8f8159bf0 2023-12-01T03:13:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.088+0000 7fc8effff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc8f80ca7c0 0x7fc8f80cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:08.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.088+0000 7fc8effff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc8f80ca7c0 0x7fc8f80cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52580/0 (socket says 172.21.15.115:52580) 2023-12-01T03:13:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.088+0000 7fc8effff640 1 -- 172.21.15.115:0/2606812000 learned_addr learned my addr 172.21.15.115:0/2606812000 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.089+0000 7fc8f77fe640 1 --2- 172.21.15.115:0/2606812000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 0x7fc8f815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.089+0000 7fc8effff640 1 -- 172.21.15.115:0/2606812000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 msgr2=0x7fc8f815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.089+0000 7fc8effff640 1 --2- 172.21.15.115:0/2606812000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 0x7fc8f815e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.091 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.089+0000 7fc8effff640 1 -- 172.21.15.115:0/2606812000 >> v2:172.21.15.142:3300/0 conn(0x7fc8f80cb6f0 msgr2=0x7fc8f806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.089+0000 7fc8f7fff640 1 --2- 172.21.15.115:0/2606812000 >> v2:172.21.15.142:3300/0 conn(0x7fc8f80cb6f0 0x7fc8f806f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.089+0000 7fc8effff640 1 --2- 172.21.15.115:0/2606812000 >> v2:172.21.15.142:3300/0 conn(0x7fc8f80cb6f0 0x7fc8f806f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.089+0000 7fc8effff640 1 -- 172.21.15.115:0/2606812000 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f80c9440 con 0x7fc8f80ca7c0 2023-12-01T03:13:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.089+0000 7fc8f77fe640 1 --2- 172.21.15.115:0/2606812000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 0x7fc8f815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:08.092 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.089+0000 7fc8f7fff640 1 --2- 172.21.15.115:0/2606812000 >> v2:172.21.15.142:3300/0 conn(0x7fc8f80cb6f0 0x7fc8f806f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.090+0000 7fc8effff640 1 --2- 172.21.15.115:0/2606812000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80ca7c0 0x7fc8f80cb1b0 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 crypto rx=0x7fc8f806f420 tx=0x7fc8e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5869aacfa367ef server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.090+0000 7fc8f6ffd640 1 -- 172.21.15.115:0/2606812000 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0031870 con 0x7fc8f80ca7c0 2023-12-01T03:13:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.090+0000 7fc8f6ffd640 1 -- 172.21.15.115:0/2606812000 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e00319d0 con 0x7fc8f80ca7c0 2023-12-01T03:13:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.090+0000 7fc8f6ffd640 1 -- 172.21.15.115:0/2606812000 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0031d60 con 0x7fc8f80ca7c0 2023-12-01T03:13:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.091+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/2606812000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80ca7c0 msgr2=0x7fc8f80cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.091+0000 7fc8fdd2e640 1 --2- 172.21.15.115:0/2606812000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80ca7c0 0x7fc8f80cb1b0 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 crypto rx=0x7fc8f806f420 tx=0x7fc8e002b910 comp rx=0 tx=0).stop 2023-12-01T03:13:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.091+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/2606812000 shutdown_connections 2023-12-01T03:13:08.093 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.091+0000 7fc8fdd2e640 1 --2- 172.21.15.115:0/2606812000 >> v2:172.21.15.142:3300/0 conn(0x7fc8f80cb6f0 0x7fc8f806f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.091+0000 7fc8fdd2e640 1 --2- 172.21.15.115:0/2606812000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80ca7c0 0x7fc8f80cb1b0 unknown :-1 s=CLOSED pgs=3102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.091+0000 7fc8fdd2e640 1 --2- 172.21.15.115:0/2606812000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 0x7fc8f815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.091+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/2606812000 >> 172.21.15.115:0/2606812000 conn(0x7fc8f80662c0 msgr2=0x7fc8f80b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.091+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/2606812000 shutdown_connections 2023-12-01T03:13:08.094 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.091+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/2606812000 wait complete. 2023-12-01T03:13:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.093+0000 7fc8fdd2e640 1 Processor -- start 2023-12-01T03:13:08.095 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.093+0000 7fc8fdd2e640 1 -- start start 2023-12-01T03:13:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8fdd2e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f80ca7c0 0x7fc8f80c3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8fdd2e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80cb6f0 0x7fc8f8155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8fdd2e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 0x7fc8f81559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:08.096 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8fdd2e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80c41d0 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8fdd2e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80c5030 con 0x7fc8f80ca7c0 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8fdd2e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80c4420 con 0x7fc8f8159bf0 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8effff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80cb6f0 0x7fc8f8155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8effff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80cb6f0 0x7fc8f8155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52586/0 (socket says 172.21.15.115:52586) 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8effff640 1 -- 172.21.15.115:0/3496633869 learned_addr learned my addr 172.21.15.115:0/3496633869 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8f77fe640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f80ca7c0 0x7fc8f80c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.094+0000 7fc8f7fff640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 0x7fc8f81559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8effff640 1 -- 172.21.15.115:0/3496633869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 msgr2=0x7fc8f81559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8effff640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 0x7fc8f81559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8effff640 1 -- 172.21.15.115:0/3496633869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f80ca7c0 msgr2=0x7fc8f80c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8effff640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f80ca7c0 0x7fc8f80c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.097 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8effff640 1 -- 172.21.15.115:0/3496633869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e00094d0 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8effff640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80cb6f0 0x7fc8f8155490 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000c00 tx=0x7fc8e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8f4ff9640 1 -- 172.21.15.115:0/3496633869 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0031860 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8f4ff9640 1 -- 172.21.15.115:0/3496633869 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e00443f0 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.098 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8f4ff9640 1 -- 172.21.15.115:0/3496633869 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00447a0 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/3496633869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f80c4700 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.095+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/3496633869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f80c4cc0 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.096+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/3496633869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f8074210 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.099 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.098+0000 7fc8f4ff9640 1 -- 172.21.15.115:0/3496633869 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc8e0044900 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.100 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.099+0000 7fc8f4ff9640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc8d806a550 0x7fc8d806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.099+0000 7fc8f4ff9640 1 -- 172.21.15.115:0/3496633869 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6570+0+0 (secure 0 0 0) 0x7fc8e002d040 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.099+0000 7fc8f77fe640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc8d806a550 0x7fc8d806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:08.101 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.100+0000 7fc8f77fe640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc8d806a550 0x7fc8d806ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80b45f0 tx=0x7fc8e8008040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:08.105 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.104+0000 7fc8f4ff9640 1 -- 172.21.15.115:0/3496633869 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc8e00769e0 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.294+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/3496633869 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8f81ee0d0 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.843 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.841+0000 7fc8f4ff9640 1 -- 172.21.15.115:0/3496633869 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v260) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc8e00350d0 con 0x7fc8f80cb6f0 2023-12-01T03:13:08.844 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_metadata' removed 2023-12-01T03:13:08.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.846+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/3496633869 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc8d806a550 msgr2=0x7fc8d806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:08.848 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.846+0000 7fc8fdd2e640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc8d806a550 0x7fc8d806ca10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80b45f0 tx=0x7fc8e8008040 comp rx=0 tx=0).stop 2023-12-01T03:13:08.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.846+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/3496633869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80cb6f0 msgr2=0x7fc8f8155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:08.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.846+0000 7fc8fdd2e640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80cb6f0 0x7fc8f8155490 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000c00 tx=0x7fc8e00046c0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.847+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/3496633869 shutdown_connections 2023-12-01T03:13:08.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.847+0000 7fc8fdd2e640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc8d806a550 0x7fc8d806ca10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.847+0000 7fc8fdd2e640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8f8159bf0 0x7fc8f81559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.847+0000 7fc8fdd2e640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8f80cb6f0 0x7fc8f8155490 secure :-1 s=CLOSED pgs=3103 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0000c00 tx=0x7fc8e00046c0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.847+0000 7fc8fdd2e640 1 --2- 172.21.15.115:0/3496633869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8f80ca7c0 0x7fc8f80c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:08.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.847+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/3496633869 >> 172.21.15.115:0/3496633869 conn(0x7fc8f80662c0 msgr2=0x7fc8f80cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:08.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.848+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/3496633869 shutdown_connections 2023-12-01T03:13:08.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:08.848+0000 7fc8fdd2e640 1 -- 172.21.15.115:0/3496633869 wait complete. 2023-12-01T03:13:08.875 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T03:13:09.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.072+0000 7fc111320640 1 Processor -- start 2023-12-01T03:13:09.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.072+0000 7fc111320640 1 -- start start 2023-12-01T03:13:09.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc111320640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc10c15ce50 0x7fc10c0ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:09.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc111320640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc10c06caa0 0x7fc10c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:09.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc111320640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10c0ca120 con 0x7fc10c0cb6f0 2023-12-01T03:13:09.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc111320640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10c0cbad0 con 0x7fc10c15ce50 2023-12-01T03:13:09.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc111320640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc10c06d2f0 con 0x7fc10c06caa0 2023-12-01T03:13:09.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc10affd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc10c15ce50 0x7fc10c0ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:09.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc10affd640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fc10c15ce50 0x7fc10c0ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52594/0 (socket says 172.21.15.115:52594) 2023-12-01T03:13:09.076 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc10affd640 1 -- 172.21.15.115:0/2778774325 learned_addr learned my addr 172.21.15.115:0/2778774325 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:09.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc10a7fc640 1 --2- 172.21.15.115:0/2778774325 >> v2:172.21.15.142:3300/0 conn(0x7fc10c06caa0 0x7fc10c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:09.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc10affd640 1 -- 172.21.15.115:0/2778774325 >> v2:172.21.15.142:3300/0 conn(0x7fc10c06caa0 msgr2=0x7fc10c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:09.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.074+0000 7fc10affd640 1 --2- 172.21.15.115:0/2778774325 >> v2:172.21.15.142:3300/0 conn(0x7fc10c06caa0 0x7fc10c0cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.075+0000 7fc10affd640 1 -- 172.21.15.115:0/2778774325 >> v1:172.21.15.115:6789/0 conn(0x7fc10c0cb6f0 legacy=0x7fc10c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:09.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.075+0000 7fc10affd640 1 -- 172.21.15.115:0/2778774325 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0c9380 con 0x7fc10c15ce50 2023-12-01T03:13:09.077 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.075+0000 7fc109ffb640 1 -- 172.21.15.115:0/2778774325 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460709727 0 0) 0x7fc10c0ca120 con 0x7fc10c0cb6f0 2023-12-01T03:13:09.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.076+0000 7fc10affd640 1 --2- 172.21.15.115:0/2778774325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c15ce50 0x7fc10c0ccdf0 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0cbff0 tx=0x7fc0fc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3120b3eb6f863a44 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:09.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.076+0000 7fc109ffb640 1 -- 172.21.15.115:0/2778774325 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc030860 con 0x7fc10c15ce50 2023-12-01T03:13:09.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.076+0000 7fc109ffb640 1 -- 172.21.15.115:0/2778774325 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc0309c0 con 0x7fc10c15ce50 2023-12-01T03:13:09.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.076+0000 7fc109ffb640 1 -- 172.21.15.115:0/2778774325 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc030d50 con 0x7fc10c15ce50 2023-12-01T03:13:09.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.076+0000 7fc111320640 1 -- 172.21.15.115:0/2778774325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c15ce50 msgr2=0x7fc10c0ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:09.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.076+0000 7fc111320640 1 --2- 172.21.15.115:0/2778774325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c15ce50 0x7fc10c0ccdf0 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0cbff0 tx=0x7fc0fc02b9a0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.079 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.077+0000 7fc111320640 1 -- 172.21.15.115:0/2778774325 shutdown_connections 2023-12-01T03:13:09.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.077+0000 7fc111320640 1 --2- 172.21.15.115:0/2778774325 >> v2:172.21.15.142:3300/0 conn(0x7fc10c06caa0 0x7fc10c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.077+0000 7fc111320640 1 --2- 172.21.15.115:0/2778774325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c15ce50 0x7fc10c0ccdf0 unknown :-1 s=CLOSED pgs=3105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.077+0000 7fc111320640 1 -- 172.21.15.115:0/2778774325 >> 172.21.15.115:0/2778774325 conn(0x7fc10c0662c0 msgr2=0x7fc10c0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:09.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.077+0000 7fc111320640 1 -- 172.21.15.115:0/2778774325 shutdown_connections 2023-12-01T03:13:09.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.077+0000 7fc111320640 1 -- 172.21.15.115:0/2778774325 wait complete. 2023-12-01T03:13:09.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.078+0000 7fc111320640 1 Processor -- start 2023-12-01T03:13:09.080 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.079+0000 7fc111320640 1 -- start start 2023-12-01T03:13:09.081 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc111320640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c06caa0 0x7fc10c1f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc111320640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10c0cb6f0 0x7fc10c15e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc111320640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc10c15ce50 0x7fc10c15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc111320640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c1f5c50 con 0x7fc10c06caa0 2023-12-01T03:13:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc111320640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c15e8e0 con 0x7fc10c15ce50 2023-12-01T03:13:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc111320640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c1f5600 con 0x7fc10c0cb6f0 2023-12-01T03:13:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc10affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c06caa0 0x7fc10c1f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc10affd640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c06caa0 0x7fc10c1f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52596/0 (socket says 172.21.15.115:52596) 2023-12-01T03:13:09.082 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc10affd640 1 -- 172.21.15.115:0/1894284606 learned_addr learned my addr 172.21.15.115:0/1894284606 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:09.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc10affd640 1 -- 172.21.15.115:0/1894284606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10c0cb6f0 msgr2=0x7fc10c15e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:09.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc10affd640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10c0cb6f0 0x7fc10c15e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc10affd640 1 -- 172.21.15.115:0/1894284606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc10c15ce50 msgr2=0x7fc10c15d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:09.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc10affd640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc10c15ce50 0x7fc10c15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.083 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.080+0000 7fc10affd640 1 -- 172.21.15.115:0/1894284606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc0094d0 con 0x7fc10c06caa0 2023-12-01T03:13:09.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.081+0000 7fc10affd640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c06caa0 0x7fc10c1f60e0 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc000c00 tx=0x7fc0fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:09.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.081+0000 7fc1037fe640 1 -- 172.21.15.115:0/1894284606 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc03e070 con 0x7fc10c06caa0 2023-12-01T03:13:09.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.081+0000 7fc1037fe640 1 -- 172.21.15.115:0/1894284606 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc004030 con 0x7fc10c06caa0 2023-12-01T03:13:09.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.081+0000 7fc1037fe640 1 -- 172.21.15.115:0/1894284606 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0043e0 con 0x7fc10c06caa0 2023-12-01T03:13:09.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.081+0000 7fc111320640 1 -- 172.21.15.115:0/1894284606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c1f58e0 con 0x7fc10c06caa0 2023-12-01T03:13:09.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.081+0000 7fc111320640 1 -- 172.21.15.115:0/1894284606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c15ebc0 con 0x7fc10c06caa0 2023-12-01T03:13:09.084 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.082+0000 7fc1017fa640 1 -- 172.21.15.115:0/1894284606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e4005250 con 0x7fc10c06caa0 2023-12-01T03:13:09.085 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.083+0000 7fc1037fe640 1 -- 172.21.15.115:0/1894284606 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc0fc042070 con 0x7fc10c06caa0 2023-12-01T03:13:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.084+0000 7fc1037fe640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ec06a550 0x7fc0ec06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.084+0000 7fc1037fe640 1 -- 172.21.15.115:0/1894284606 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 6132+0+0 (secure 0 0 0) 0x7fc0fc0aaed0 con 0x7fc10c06caa0 2023-12-01T03:13:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.084+0000 7fc10a7fc640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ec06a550 0x7fc0ec06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:09.086 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.085+0000 7fc10a7fc640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ec06a550 0x7fc0ec06ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc10c1519c0 tx=0x7fc0f40073b0 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:09.090 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.088+0000 7fc1037fe640 1 -- 172.21.15.115:0/1894284606 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fc0fc076050 con 0x7fc10c06caa0 2023-12-01T03:13:09.283 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.281+0000 7fc1017fa640 1 -- 172.21.15.115:0/1894284606 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0e4005500 con 0x7fc10c06caa0 2023-12-01T03:13:09.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.869+0000 7fc1037fe640 1 -- 172.21.15.115:0/1894284606 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v261) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc0fc034020 con 0x7fc10c06caa0 2023-12-01T03:13:09.871 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'cephfs_data' removed 2023-12-01T03:13:09.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.874+0000 7fc1017fa640 1 -- 172.21.15.115:0/1894284606 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ec06a550 msgr2=0x7fc0ec06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:09.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.874+0000 7fc1017fa640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ec06a550 0x7fc0ec06ca10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc10c1519c0 tx=0x7fc0f40073b0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.874+0000 7fc1017fa640 1 -- 172.21.15.115:0/1894284606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c06caa0 msgr2=0x7fc10c1f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:09.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.874+0000 7fc1017fa640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c06caa0 0x7fc10c1f60e0 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc000c00 tx=0x7fc0fc0046c0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.874+0000 7fc1017fa640 1 -- 172.21.15.115:0/1894284606 shutdown_connections 2023-12-01T03:13:09.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.874+0000 7fc1017fa640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc0ec06a550 0x7fc0ec06ca10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.874+0000 7fc1017fa640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10c0cb6f0 0x7fc10c15e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.874+0000 7fc1017fa640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc10c15ce50 0x7fc10c15d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.874+0000 7fc1017fa640 1 --2- 172.21.15.115:0/1894284606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc10c06caa0 0x7fc10c1f60e0 unknown :-1 s=CLOSED pgs=3106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:09.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.875+0000 7fc1017fa640 1 -- 172.21.15.115:0/1894284606 >> 172.21.15.115:0/1894284606 conn(0x7fc10c0662c0 msgr2=0x7fc10c0ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:09.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.875+0000 7fc1017fa640 1 -- 172.21.15.115:0/1894284606 shutdown_connections 2023-12-01T03:13:09.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:09.875+0000 7fc1017fa640 1 -- 172.21.15.115:0/1894284606 wait complete. 2023-12-01T03:13:09.898 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-01T03:13:10.108 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.107+0000 7f048eefb640 1 Processor -- start 2023-12-01T03:13:10.109 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.107+0000 7f048eefb640 1 -- start start 2023-12-01T03:13:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.108+0000 7f048eefb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f04880ce1b0 0x7f04880cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.108+0000 7f048eefb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04880b0e60 0x7f04880b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.108+0000 7f048eefb640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04880cc9a0 con 0x7f048815ac80 2023-12-01T03:13:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.108+0000 7f048eefb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048815b060 con 0x7f04880ce1b0 2023-12-01T03:13:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.108+0000 7f048eefb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04880b0ce0 con 0x7f04880b0e60 2023-12-01T03:13:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.108+0000 7f048cc97640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f048815ac80 0x7f04880cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58840/0 (socket says 172.21.15.115:58840) 2023-12-01T03:13:10.110 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.108+0000 7f048cc97640 1 -- 172.21.15.115:0/2770053877 learned_addr learned my addr 172.21.15.115:0/2770053877 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:10.112 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480707617 0 0) 0x7f04880cc9a0 con 0x7f048815ac80 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f048d498640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04880b0e60 0x7f04880b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0478003520 con 0x7f048815ac80 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f0487fff640 1 --2- 172.21.15.115:0/2770053877 >> v2:172.21.15.142:3300/0 conn(0x7f04880ce1b0 0x7f04880cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4200278090 0 0) 0x7f0478003520 con 0x7f048815ac80 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04880b0e60 msgr2=0x7f04880b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f04877fe640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04880b0e60 0x7f04880b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 >> v2:172.21.15.142:3300/0 conn(0x7f04880ce1b0 msgr2=0x7f04880cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f04877fe640 1 --2- 172.21.15.115:0/2770053877 >> v2:172.21.15.142:3300/0 conn(0x7f04880ce1b0 0x7f04880cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04880cbd90 con 0x7f048815ac80 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.109+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f047c001670 con 0x7f048815ac80 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 >> v1:172.21.15.115:6789/0 conn(0x7f048815ac80 legacy=0x7f04880cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f04877fe640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04880b0e60 0x7f0478003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f04877fe640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 0x7f04780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.113 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f04877fe640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0478004d00 0x7f0478005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048815b060 con 0x7f0478003ef0 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880b0ce0 con 0x7f0478004d00 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047800b9d0 con 0x7f04880b0e60 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f0487fff640 1 --2- 172.21.15.115:0/2770053877 >> v2:172.21.15.142:3300/0 conn(0x7f04880ce1b0 0x7f04880cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f0487fff640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 0x7f04780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f048cc97640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04880b0e60 0x7f0478003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.110+0000 7f048d498640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0478004d00 0x7f0478005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.111+0000 7f0487fff640 1 -- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04880b0e60 msgr2=0x7f0478003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.111+0000 7f0487fff640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04880b0e60 0x7f0478003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.111+0000 7f0487fff640 1 -- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0478004d00 msgr2=0x7f0478005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.111+0000 7f0487fff640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0478004d00 0x7f0478005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.114 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.111+0000 7f0487fff640 1 -- 172.21.15.115:0/2770053877 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0478003520 con 0x7f0478003ef0 2023-12-01T03:13:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.112+0000 7f0487fff640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 0x7f04780047c0 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 crypto rx=0x7f04740098a0 tx=0x7f04780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e988182fa5bb0de9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:10.115 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.112+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0474037600 con 0x7f0478003ef0 2023-12-01T03:13:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.112+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0474037760 con 0x7f0478003ef0 2023-12-01T03:13:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.112+0000 7f04877fe640 1 -- 172.21.15.115:0/2770053877 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0474037ad0 con 0x7f0478003ef0 2023-12-01T03:13:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.113+0000 7f048eefb640 1 -- 172.21.15.115:0/2770053877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 msgr2=0x7f04780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.113+0000 7f048eefb640 1 --2- 172.21.15.115:0/2770053877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 0x7f04780047c0 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 crypto rx=0x7f04740098a0 tx=0x7f04780057f0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.113+0000 7f048d498640 1 -- 172.21.15.115:0/2770053877 reap_dead start 2023-12-01T03:13:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.113+0000 7f048eefb640 1 -- 172.21.15.115:0/2770053877 shutdown_connections 2023-12-01T03:13:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.113+0000 7f048eefb640 1 -- 172.21.15.115:0/2770053877 >> 172.21.15.115:0/2770053877 conn(0x7f0488065960 msgr2=0x7f0488065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.113+0000 7f048eefb640 1 -- 172.21.15.115:0/2770053877 shutdown_connections 2023-12-01T03:13:10.116 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.113+0000 7f048eefb640 1 -- 172.21.15.115:0/2770053877 wait complete. 2023-12-01T03:13:10.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.115+0000 7f048eefb640 1 Processor -- start 2023-12-01T03:13:10.117 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.115+0000 7f048eefb640 1 -- start start 2023-12-01T03:13:10.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.116+0000 7f048eefb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 0x7f04881f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.116+0000 7f048eefb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488154ed0 0x7f0488155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.116+0000 7f048eefb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f048815c4a0 0x7f048815c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.116+0000 7f048eefb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048815de50 con 0x7f0478003ef0 2023-12-01T03:13:10.118 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.116+0000 7f048eefb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0488155890 con 0x7f0488154ed0 2023-12-01T03:13:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.116+0000 7f048eefb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048815d1d0 con 0x7f048815c4a0 2023-12-01T03:13:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.116+0000 7f048cc97640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 0x7f04881f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.116+0000 7f048cc97640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 0x7f04881f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52614/0 (socket says 172.21.15.115:52614) 2023-12-01T03:13:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.116+0000 7f048cc97640 1 -- 172.21.15.115:0/4115711255 learned_addr learned my addr 172.21.15.115:0/4115711255 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f0487fff640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488154ed0 0x7f0488155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f048cc97640 1 -- 172.21.15.115:0/4115711255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f048815c4a0 msgr2=0x7f048815c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f048cc97640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f048815c4a0 0x7f048815c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f048cc97640 1 -- 172.21.15.115:0/4115711255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488154ed0 msgr2=0x7f0488155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f048cc97640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488154ed0 0x7f0488155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.119 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f048cc97640 1 -- 172.21.15.115:0/4115711255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048815d4b0 con 0x7f0478003ef0 2023-12-01T03:13:10.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f048cc97640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 0x7f04881f5110 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e7f0 tx=0x7f047c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:10.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f04857fa640 1 -- 172.21.15.115:0/4115711255 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c00c7f0 con 0x7f0478003ef0 2023-12-01T03:13:10.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f04857fa640 1 -- 172.21.15.115:0/4115711255 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c00c950 con 0x7f0478003ef0 2023-12-01T03:13:10.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f04857fa640 1 -- 172.21.15.115:0/4115711255 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c00cd00 con 0x7f0478003ef0 2023-12-01T03:13:10.120 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f048eefb640 1 -- 172.21.15.115:0/4115711255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048815d790 con 0x7f0478003ef0 2023-12-01T03:13:10.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.117+0000 7f048eefb640 1 -- 172.21.15.115:0/4115711255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04881f5650 con 0x7f0478003ef0 2023-12-01T03:13:10.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.119+0000 7f048eefb640 1 -- 172.21.15.115:0/4115711255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04880cc330 con 0x7f0478003ef0 2023-12-01T03:13:10.121 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.119+0000 7f04857fa640 1 -- 172.21.15.115:0/4115711255 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f047c01d460 con 0x7f0478003ef0 2023-12-01T03:13:10.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.120+0000 7f04857fa640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f046c06a550 0x7f046c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.120+0000 7f04857fa640 1 -- 172.21.15.115:0/4115711255 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f047c088a40 con 0x7f0478003ef0 2023-12-01T03:13:10.122 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.121+0000 7f0487fff640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f046c06a550 0x7f046c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.123 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.121+0000 7f0487fff640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f046c06a550 0x7f046c06ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0474036000 tx=0x7f0474031000 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:10.126 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.124+0000 7f04857fa640 1 -- 172.21.15.115:0/4115711255 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f04880cc330 con 0x7f0478003ef0 2023-12-01T03:13:10.316 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.314+0000 7f048eefb640 1 -- 172.21.15.115:0/4115711255 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f04880cc300 con 0x7f0478003ef0 2023-12-01T03:13:10.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.323+0000 7f04857fa640 1 -- 172.21.15.115:0/4115711255 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f047c053f00 con 0x7f0478003ef0 2023-12-01T03:13:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.325+0000 7f048eefb640 1 -- 172.21.15.115:0/4115711255 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f046c06a550 msgr2=0x7f046c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.325+0000 7f048eefb640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f046c06a550 0x7f046c06ca10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0474036000 tx=0x7f0474031000 comp rx=0 tx=0).stop 2023-12-01T03:13:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.325+0000 7f048eefb640 1 -- 172.21.15.115:0/4115711255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 msgr2=0x7f04881f5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.325+0000 7f048eefb640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 0x7f04881f5110 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e7f0 tx=0x7f047c00ecb0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.326+0000 7f048eefb640 1 -- 172.21.15.115:0/4115711255 shutdown_connections 2023-12-01T03:13:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.326+0000 7f048eefb640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f046c06a550 0x7f046c06ca10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.326+0000 7f048eefb640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f048815c4a0 0x7f048815c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.326+0000 7f048eefb640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0488154ed0 0x7f0488155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.326+0000 7f048eefb640 1 --2- 172.21.15.115:0/4115711255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0478003ef0 0x7f04881f5110 unknown :-1 s=CLOSED pgs=3109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.326+0000 7f048eefb640 1 -- 172.21.15.115:0/4115711255 >> 172.21.15.115:0/4115711255 conn(0x7f0488065960 msgr2=0x7f04880c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.326+0000 7f048eefb640 1 -- 172.21.15.115:0/4115711255 shutdown_connections 2023-12-01T03:13:10.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.326+0000 7f048eefb640 1 -- 172.21.15.115:0/4115711255 wait complete. 2023-12-01T03:13:10.345 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-01T03:13:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.547+0000 7f3a0364a640 1 Processor -- start 2023-12-01T03:13:10.549 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.547+0000 7f3a0364a640 1 -- start start 2023-12-01T03:13:10.550 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.548+0000 7f3a0364a640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f39fc15ac80 0x7f39fc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.549+0000 7f3a0364a640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc0cc9a0 con 0x7f39fc0b0e60 2023-12-01T03:13:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.549+0000 7f3a0364a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc0cbd90 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.549+0000 7f3a0364a640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39fc15b060 con 0x7f39fc15ac80 2023-12-01T03:13:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.549+0000 7f3a00be5640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f39fc0b0e60 0x7f39fc0cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58848/0 (socket says 172.21.15.115:58848) 2023-12-01T03:13:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.549+0000 7f3a00be5640 1 -- 172.21.15.115:0/3385302104 learned_addr learned my addr 172.21.15.115:0/3385302104 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:10.551 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.549+0000 7f3a013e6640 1 --2- 172.21.15.115:0/3385302104 >> v2:172.21.15.115:3300/0 conn(0x7f39fc15ac80 0x7f39fc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.549+0000 7f39f3fff640 1 -- 172.21.15.115:0/3385302104 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459590042 0 0) 0x7f39fc0cc9a0 con 0x7f39fc0b0e60 2023-12-01T03:13:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.550+0000 7f39f3fff640 1 -- 172.21.15.115:0/3385302104 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e8003520 con 0x7f39fc0b0e60 2023-12-01T03:13:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.550+0000 7f3a013e6640 1 -- 172.21.15.115:0/3385302104 >> v1:172.21.15.142:6789/0 conn(0x7f39fc0ce1b0 legacy=0x7f39fc06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.550+0000 7f3a013e6640 1 -- 172.21.15.115:0/3385302104 >> v1:172.21.15.115:6789/0 conn(0x7f39fc0b0e60 legacy=0x7f39fc0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.550+0000 7f3a013e6640 1 -- 172.21.15.115:0/3385302104 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0cc070 con 0x7f39fc15ac80 2023-12-01T03:13:10.552 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.551+0000 7f3a013e6640 1 --2- 172.21.15.115:0/3385302104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc15ac80 0x7f39fc0cce30 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 crypto rx=0x7f39f80097a0 tx=0x7f39f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc7a571f961ea73 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:10.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.551+0000 7f39f3fff640 1 -- 172.21.15.115:0/3385302104 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8031830 con 0x7f39fc15ac80 2023-12-01T03:13:10.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.551+0000 7f39f3fff640 1 -- 172.21.15.115:0/3385302104 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f8031990 con 0x7f39fc15ac80 2023-12-01T03:13:10.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.551+0000 7f39f3fff640 1 -- 172.21.15.115:0/3385302104 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8031d00 con 0x7f39fc15ac80 2023-12-01T03:13:10.553 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.551+0000 7f3a0364a640 1 -- 172.21.15.115:0/3385302104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc15ac80 msgr2=0x7f39fc0cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.551+0000 7f3a0364a640 1 --2- 172.21.15.115:0/3385302104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc15ac80 0x7f39fc0cce30 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 crypto rx=0x7f39f80097a0 tx=0x7f39f802b8d0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.552+0000 7f3a0364a640 1 -- 172.21.15.115:0/3385302104 shutdown_connections 2023-12-01T03:13:10.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.552+0000 7f3a0364a640 1 --2- 172.21.15.115:0/3385302104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc15ac80 0x7f39fc0cce30 unknown :-1 s=CLOSED pgs=3111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.552+0000 7f3a0364a640 1 -- 172.21.15.115:0/3385302104 >> 172.21.15.115:0/3385302104 conn(0x7f39fc065960 msgr2=0x7f39fc065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:10.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.552+0000 7f3a0364a640 1 -- 172.21.15.115:0/3385302104 shutdown_connections 2023-12-01T03:13:10.554 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.552+0000 7f3a0364a640 1 -- 172.21.15.115:0/3385302104 wait complete. 2023-12-01T03:13:10.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.554+0000 7f3a0364a640 1 Processor -- start 2023-12-01T03:13:10.555 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.554+0000 7f3a0364a640 1 -- start start 2023-12-01T03:13:10.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a0364a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39fc0b0e60 0x7f39fc150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a0364a640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc0ce1b0 0x7f39fc13fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a0364a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39fc15ac80 0x7f39fc0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a0364a640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc14fe70 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a0364a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc1435c0 con 0x7f39fc0b0e60 2023-12-01T03:13:10.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a0364a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc1506f0 con 0x7f39fc15ac80 2023-12-01T03:13:10.557 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a00be5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc0ce1b0 0x7f39fc13fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a00be5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc0ce1b0 0x7f39fc13fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52628/0 (socket says 172.21.15.115:52628) 2023-12-01T03:13:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a00be5640 1 -- 172.21.15.115:0/4242571506 learned_addr learned my addr 172.21.15.115:0/4242571506 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a013e6640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39fc0b0e60 0x7f39fc150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a00be5640 1 -- 172.21.15.115:0/4242571506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39fc15ac80 msgr2=0x7f39fc0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a00be5640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39fc15ac80 0x7f39fc0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a00be5640 1 -- 172.21.15.115:0/4242571506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39fc0b0e60 msgr2=0x7f39fc150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a00be5640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39fc0b0e60 0x7f39fc150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.555+0000 7f3a00be5640 1 -- 172.21.15.115:0/4242571506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f8009400 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.558 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.556+0000 7f3a00be5640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc0ce1b0 0x7f39fc13fd90 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c460 tx=0x7f39ec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.556+0000 7f39f1ffb640 1 -- 172.21.15.115:0/4242571506 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec005ce0 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.556+0000 7f39f1ffb640 1 -- 172.21.15.115:0/4242571506 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec005e40 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.556+0000 7f3a0364a640 1 -- 172.21.15.115:0/4242571506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc1402d0 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.556+0000 7f39f1ffb640 1 -- 172.21.15.115:0/4242571506 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec007290 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.556+0000 7f3a0364a640 1 -- 172.21.15.115:0/4242571506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc1500a0 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.557+0000 7f3a0364a640 1 -- 172.21.15.115:0/4242571506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc04e890 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.558+0000 7f39f1ffb640 1 -- 172.21.15.115:0/4242571506 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f39ec0073f0 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.559+0000 7f39f1ffb640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f39e006a550 0x7f39e006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.559+0000 7f39f1ffb640 1 -- 172.21.15.115:0/4242571506 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f39ec0880e0 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.561 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.559+0000 7f3a013e6640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f39e006a550 0x7f39e006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:10.562 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.560+0000 7f3a013e6640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f39e006a550 0x7f39e006ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f39f8036000 tx=0x7f39f803c040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.563+0000 7f39f1ffb640 1 -- 172.21.15.115:0/4242571506 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f39ec053520 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.751+0000 7f3a0364a640 1 -- 172.21.15.115:0/4242571506 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f39fc150830 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.761+0000 7f39f1ffb640 1 -- 172.21.15.115:0/4242571506 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v86) v1 ==== 120+0+0 (secure 0 0 0) 0x7f39ec0170b0 con 0x7f39fc0ce1b0 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.763+0000 7f3a0364a640 1 -- 172.21.15.115:0/4242571506 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f39e006a550 msgr2=0x7f39e006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f39e006a550 0x7f39e006ca10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f39f8036000 tx=0x7f39f803c040 comp rx=0 tx=0).stop 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 -- 172.21.15.115:0/4242571506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc0ce1b0 msgr2=0x7f39fc13fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc0ce1b0 0x7f39fc13fd90 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c460 tx=0x7f39ec00c920 comp rx=0 tx=0).stop 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 -- 172.21.15.115:0/4242571506 shutdown_connections 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f39e006a550 0x7f39e006ca10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f39fc15ac80 0x7f39fc0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39fc0ce1b0 0x7f39fc13fd90 unknown :-1 s=CLOSED pgs=3112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 --2- 172.21.15.115:0/4242571506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39fc0b0e60 0x7f39fc150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 -- 172.21.15.115:0/4242571506 >> 172.21.15.115:0/4242571506 conn(0x7f39fc065960 msgr2=0x7f39fc06b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:10.766 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 -- 172.21.15.115:0/4242571506 shutdown_connections 2023-12-01T03:13:10.767 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.764+0000 7f3a0364a640 1 -- 172.21.15.115:0/4242571506 wait complete. 2023-12-01T03:13:10.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-01T03:13:10.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.997+0000 7f92d4bf0640 1 Processor -- start 2023-12-01T03:13:10.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.997+0000 7f92d4bf0640 1 -- start start 2023-12-01T03:13:11.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.998+0000 7f92d4bf0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d00ce1b0 0x7f92d00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.998+0000 7f92d4bf0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f92d00b0e60 0x7f92d00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.999+0000 7f92d4bf0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d00cc9a0 con 0x7f92d015ac80 2023-12-01T03:13:11.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.999+0000 7f92d4bf0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d015b060 con 0x7f92d00b0e60 2023-12-01T03:13:11.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.999+0000 7f92d4bf0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00b0ce0 con 0x7f92d00ce1b0 2023-12-01T03:13:11.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.999+0000 7f92ced9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f92d00b0e60 0x7f92d00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.999+0000 7f92ced9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f92d00b0e60 0x7f92d00b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:59258/0 (socket says 172.21.15.115:59258) 2023-12-01T03:13:11.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:10.999+0000 7f92ced9d640 1 -- 172.21.15.115:0/3657322387 learned_addr learned my addr 172.21.15.115:0/3657322387 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:11.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.000+0000 7f92c5d9b640 1 --2- 172.21.15.115:0/3657322387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d00ce1b0 0x7f92d00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.000+0000 7f92ced9d640 1 -- 172.21.15.115:0/3657322387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d00ce1b0 msgr2=0x7f92d00cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:11.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.000+0000 7f92ced9d640 1 --2- 172.21.15.115:0/3657322387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d00ce1b0 0x7f92d00cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.000+0000 7f92ced9d640 1 -- 172.21.15.115:0/3657322387 >> v1:172.21.15.142:6789/0 conn(0x7f92d015ac80 legacy=0x7f92d00cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.000+0000 7f92ced9d640 1 -- 172.21.15.115:0/3657322387 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d00cbd90 con 0x7f92d00b0e60 2023-12-01T03:13:11.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.002+0000 7f92ced9d640 1 --2- 172.21.15.115:0/3657322387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d00b0e60 0x7f92d00b07a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009870 tx=0x7f92c802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2707aaf534b37527 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:11.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.002+0000 7f92cdd9b640 1 -- 172.21.15.115:0/3657322387 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8030840 con 0x7f92d00b0e60 2023-12-01T03:13:11.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.002+0000 7f92cdd9b640 1 -- 172.21.15.115:0/3657322387 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c80309a0 con 0x7f92d00b0e60 2023-12-01T03:13:11.004 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.002+0000 7f92cdd9b640 1 -- 172.21.15.115:0/3657322387 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8030d30 con 0x7f92d00b0e60 2023-12-01T03:13:11.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.003+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3657322387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d00b0e60 msgr2=0x7f92d00b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.003+0000 7f92d4bf0640 1 --2- 172.21.15.115:0/3657322387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d00b0e60 0x7f92d00b07a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009870 tx=0x7f92c802b980 comp rx=0 tx=0).stop 2023-12-01T03:13:11.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.003+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3657322387 shutdown_connections 2023-12-01T03:13:11.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.003+0000 7f92d4bf0640 1 --2- 172.21.15.115:0/3657322387 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d00ce1b0 0x7f92d00cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.003+0000 7f92d4bf0640 1 --2- 172.21.15.115:0/3657322387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d00b0e60 0x7f92d00b07a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.003+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3657322387 >> 172.21.15.115:0/3657322387 conn(0x7f92d0065960 msgr2=0x7f92d0065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:11.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.003+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3657322387 shutdown_connections 2023-12-01T03:13:11.005 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.003+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3657322387 wait complete. 2023-12-01T03:13:11.006 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.005+0000 7f92d4bf0640 1 Processor -- start 2023-12-01T03:13:11.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.005+0000 7f92d4bf0640 1 -- start start 2023-12-01T03:13:11.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.006+0000 7f92d4bf0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d00b0e60 0x7f92d014b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.006+0000 7f92d4bf0640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00ce1b0 0x7f92d00a9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.006+0000 7f92d4bf0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d015ac80 0x7f92d01590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.006+0000 7f92d4bf0640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00a8480 con 0x7f92d00ce1b0 2023-12-01T03:13:11.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.006+0000 7f92d4bf0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d014b7d0 con 0x7f92d00b0e60 2023-12-01T03:13:11.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.006+0000 7f92d4bf0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d0155a50 con 0x7f92d015ac80 2023-12-01T03:13:11.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.006+0000 7f92c5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00ce1b0 0x7f92d00a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.008 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92c5d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00ce1b0 0x7f92d00a9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52640/0 (socket says 172.21.15.115:52640) 2023-12-01T03:13:11.009 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92c5d9b640 1 -- 172.21.15.115:0/3971489105 learned_addr learned my addr 172.21.15.115:0/3971489105 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:11.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92ced9d640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d015ac80 0x7f92d01590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92ce59c640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d00b0e60 0x7f92d014b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92c5d9b640 1 -- 172.21.15.115:0/3971489105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d015ac80 msgr2=0x7f92d01590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92c5d9b640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d015ac80 0x7f92d01590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92c5d9b640 1 -- 172.21.15.115:0/3971489105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d00b0e60 msgr2=0x7f92d014b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92c5d9b640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d00b0e60 0x7f92d014b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92c5d9b640 1 -- 172.21.15.115:0/3971489105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c80094d0 con 0x7f92d00ce1b0 2023-12-01T03:13:11.010 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92ced9d640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d015ac80 0x7f92d01590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:11.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92ce59c640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d00b0e60 0x7f92d014b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:11.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.007+0000 7f92c5d9b640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00ce1b0 0x7f92d00a9140 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7f92b800e9b0 tx=0x7f92b800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:11.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.008+0000 7f92c77fe640 1 -- 172.21.15.115:0/3971489105 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8009670 con 0x7f92d00ce1b0 2023-12-01T03:13:11.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.008+0000 7f92c77fe640 1 -- 172.21.15.115:0/3971489105 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b800c9a0 con 0x7f92d00ce1b0 2023-12-01T03:13:11.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.008+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3971489105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d00b3630 con 0x7f92d00ce1b0 2023-12-01T03:13:11.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.008+0000 7f92c77fe640 1 -- 172.21.15.115:0/3971489105 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b800cd50 con 0x7f92d00ce1b0 2023-12-01T03:13:11.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.008+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3971489105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d00a8940 con 0x7f92d00ce1b0 2023-12-01T03:13:11.011 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.009+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3971489105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d004e890 con 0x7f92d00ce1b0 2023-12-01T03:13:11.012 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.010+0000 7f92c77fe640 1 -- 172.21.15.115:0/3971489105 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f92b8013470 con 0x7f92d00ce1b0 2023-12-01T03:13:11.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.011+0000 7f92c77fe640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f92b006a550 0x7f92b006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.011+0000 7f92c77fe640 1 -- 172.21.15.115:0/3971489105 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f92b8088d30 con 0x7f92d00ce1b0 2023-12-01T03:13:11.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.011+0000 7f92ce59c640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f92b006a550 0x7f92b006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.013 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.012+0000 7f92ce59c640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f92b006a550 0x7f92b006ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f92d00a9cc0 tx=0x7f92c0009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:11.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.015+0000 7f92c77fe640 1 -- 172.21.15.115:0/3971489105 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f92b8054200 con 0x7f92d00ce1b0 2023-12-01T03:13:11.209 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.207+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3971489105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f92d00cf480 con 0x7f92d00ce1b0 2023-12-01T03:13:11.217 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.216+0000 7f92c77fe640 1 -- 172.21.15.115:0/3971489105 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v87) v1 ==== 115+0+0 (secure 0 0 0) 0x7f92b80170b0 con 0x7f92d00ce1b0 2023-12-01T03:13:11.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.218+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3971489105 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f92b006a550 msgr2=0x7f92b006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.218+0000 7f92d4bf0640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f92b006a550 0x7f92b006ca10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f92d00a9cc0 tx=0x7f92c0009040 comp rx=0 tx=0).stop 2023-12-01T03:13:11.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.218+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3971489105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00ce1b0 msgr2=0x7f92d00a9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.218+0000 7f92d4bf0640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00ce1b0 0x7f92d00a9140 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7f92b800e9b0 tx=0x7f92b800ee70 comp rx=0 tx=0).stop 2023-12-01T03:13:11.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.219+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3971489105 shutdown_connections 2023-12-01T03:13:11.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.219+0000 7f92d4bf0640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f92b006a550 0x7f92b006ca10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.219+0000 7f92d4bf0640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92d015ac80 0x7f92d01590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.219+0000 7f92d4bf0640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00ce1b0 0x7f92d00a9140 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.219+0000 7f92d4bf0640 1 --2- 172.21.15.115:0/3971489105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92d00b0e60 0x7f92d014b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.219+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3971489105 >> 172.21.15.115:0/3971489105 conn(0x7f92d0065960 msgr2=0x7f92d00708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:11.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.219+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3971489105 shutdown_connections 2023-12-01T03:13:11.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.219+0000 7f92d4bf0640 1 -- 172.21.15.115:0/3971489105 wait complete. 2023-12-01T03:13:11.239 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-12-01T03:13:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.450+0000 7fec74099640 1 Processor -- start 2023-12-01T03:13:11.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.450+0000 7fec74099640 1 -- start start 2023-12-01T03:13:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.451+0000 7fec74099640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fec6c159dd0 0x7fec6c0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.451+0000 7fec74099640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c070d30 0x7fec6c0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec74099640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c06f8b0 con 0x7fec6c06b6f0 2023-12-01T03:13:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec74099640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec6c0adcb0 con 0x7fec6c159dd0 2023-12-01T03:13:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec74099640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0ad9d0 con 0x7fec6c070d30 2023-12-01T03:13:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec71e35640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fec6c159dd0 0x7fec6c0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec71e35640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fec6c159dd0 0x7fec6c0c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52646/0 (socket says 172.21.15.115:52646) 2023-12-01T03:13:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec71e35640 1 -- 172.21.15.115:0/3086856101 learned_addr learned my addr 172.21.15.115:0/3086856101 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec72636640 1 --2- 172.21.15.115:0/3086856101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c070d30 0x7fec6c0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec71e35640 1 -- 172.21.15.115:0/3086856101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c070d30 msgr2=0x7fec6c0c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec71e35640 1 --2- 172.21.15.115:0/3086856101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c070d30 0x7fec6c0c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec71e35640 1 -- 172.21.15.115:0/3086856101 >> v1:172.21.15.142:6789/0 conn(0x7fec6c06b6f0 legacy=0x7fec6c0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.452+0000 7fec71e35640 1 -- 172.21.15.115:0/3086856101 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0c9000 con 0x7fec6c159dd0 2023-12-01T03:13:11.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.453+0000 7fec71e35640 1 --2- 172.21.15.115:0/3086856101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c159dd0 0x7fec6c0c97e0 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7fec68009870 tx=0x7fec6802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbfed915c0cd0bc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:11.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.454+0000 7fec70e33640 1 -- 172.21.15.115:0/3086856101 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68031740 con 0x7fec6c159dd0 2023-12-01T03:13:11.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.454+0000 7fec70e33640 1 -- 172.21.15.115:0/3086856101 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec680318a0 con 0x7fec6c159dd0 2023-12-01T03:13:11.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.454+0000 7fec70e33640 1 -- 172.21.15.115:0/3086856101 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68031c30 con 0x7fec6c159dd0 2023-12-01T03:13:11.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.454+0000 7fec74099640 1 -- 172.21.15.115:0/3086856101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c159dd0 msgr2=0x7fec6c0c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.454+0000 7fec74099640 1 --2- 172.21.15.115:0/3086856101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c159dd0 0x7fec6c0c97e0 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7fec68009870 tx=0x7fec6802b7e0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.455+0000 7fec74099640 1 -- 172.21.15.115:0/3086856101 shutdown_connections 2023-12-01T03:13:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.455+0000 7fec74099640 1 --2- 172.21.15.115:0/3086856101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c070d30 0x7fec6c0c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.455+0000 7fec74099640 1 --2- 172.21.15.115:0/3086856101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c159dd0 0x7fec6c0c97e0 unknown :-1 s=CLOSED pgs=3114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.455+0000 7fec74099640 1 -- 172.21.15.115:0/3086856101 >> 172.21.15.115:0/3086856101 conn(0x7fec6c0ce880 msgr2=0x7fec6c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.455+0000 7fec74099640 1 -- 172.21.15.115:0/3086856101 shutdown_connections 2023-12-01T03:13:11.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.455+0000 7fec74099640 1 -- 172.21.15.115:0/3086856101 wait complete. 2023-12-01T03:13:11.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.456+0000 7fec74099640 1 Processor -- start 2023-12-01T03:13:11.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.457+0000 7fec74099640 1 -- start start 2023-12-01T03:13:11.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.457+0000 7fec74099640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c06b6f0 0x7fec6c15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec74099640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec6c070d30 0x7fec6c157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec74099640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c159dd0 0x7fec6c158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec74099640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c15b220 con 0x7fec6c06b6f0 2023-12-01T03:13:11.460 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec74099640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c15b500 con 0x7fec6c070d30 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec74099640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c15b7e0 con 0x7fec6c159dd0 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec71e35640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c06b6f0 0x7fec6c15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec71e35640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c06b6f0 0x7fec6c15c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52660/0 (socket says 172.21.15.115:52660) 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec71e35640 1 -- 172.21.15.115:0/3074615051 learned_addr learned my addr 172.21.15.115:0/3074615051 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec72636640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c159dd0 0x7fec6c158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec71e35640 1 -- 172.21.15.115:0/3074615051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c159dd0 msgr2=0x7fec6c158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec71e35640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c159dd0 0x7fec6c158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec71e35640 1 -- 172.21.15.115:0/3074615051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec6c070d30 msgr2=0x7fec6c157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec71634640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec6c070d30 0x7fec6c157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec71e35640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec6c070d30 0x7fec6c157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.458+0000 7fec71e35640 1 -- 172.21.15.115:0/3074615051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680094d0 con 0x7fec6c06b6f0 2023-12-01T03:13:11.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.459+0000 7fec71e35640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c06b6f0 0x7fec6c15c7c0 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 crypto rx=0x7fec68009870 tx=0x7fec680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.459+0000 7fec5e7fc640 1 -- 172.21.15.115:0/3074615051 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68033c70 con 0x7fec6c06b6f0 2023-12-01T03:13:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.459+0000 7fec5e7fc640 1 -- 172.21.15.115:0/3074615051 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec68033dd0 con 0x7fec6c06b6f0 2023-12-01T03:13:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.459+0000 7fec5e7fc640 1 -- 172.21.15.115:0/3074615051 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6803c640 con 0x7fec6c06b6f0 2023-12-01T03:13:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.459+0000 7fec74099640 1 -- 172.21.15.115:0/3074615051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6c15bac0 con 0x7fec6c06b6f0 2023-12-01T03:13:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.459+0000 7fec74099640 1 -- 172.21.15.115:0/3074615051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c15ae80 con 0x7fec6c06b6f0 2023-12-01T03:13:11.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.460+0000 7fec74099640 1 -- 172.21.15.115:0/3074615051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6c1e78f0 con 0x7fec6c06b6f0 2023-12-01T03:13:11.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.461+0000 7fec5e7fc640 1 -- 172.21.15.115:0/3074615051 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fec6803a040 con 0x7fec6c06b6f0 2023-12-01T03:13:11.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.462+0000 7fec5e7fc640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec5006a550 0x7fec5006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:11.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.463+0000 7fec5e7fc640 1 -- 172.21.15.115:0/3074615051 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7fec680aa510 con 0x7fec6c06b6f0 2023-12-01T03:13:11.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.463+0000 7fec71634640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec5006a550 0x7fec5006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:11.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.463+0000 7fec71634640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec5006a550 0x7fec5006ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0c1840 tx=0x7fec60009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:11.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.466+0000 7fec5e7fc640 1 -- 172.21.15.115:0/3074615051 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fec68075940 con 0x7fec6c06b6f0 2023-12-01T03:13:11.728 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.726+0000 7fec74099640 1 -- 172.21.15.115:0/3074615051 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fec6c15bfe0 con 0x7fec6c06b6f0 2023-12-01T03:13:11.888 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.886+0000 7fec5e7fc640 1 -- 172.21.15.115:0/3074615051 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3207) v1 ==== 155+0+0 (secure 0 0 0) 0x7fec6807f030 con 0x7fec6c06b6f0 2023-12-01T03:13:11.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 -- 172.21.15.115:0/3074615051 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec5006a550 msgr2=0x7fec5006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec5006a550 0x7fec5006ca10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0c1840 tx=0x7fec60009040 comp rx=0 tx=0).stop 2023-12-01T03:13:11.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 -- 172.21.15.115:0/3074615051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c06b6f0 msgr2=0x7fec6c15c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:11.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c06b6f0 0x7fec6c15c7c0 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 crypto rx=0x7fec68009870 tx=0x7fec680046c0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 -- 172.21.15.115:0/3074615051 shutdown_connections 2023-12-01T03:13:11.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fec5006a550 0x7fec5006ca10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.891 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c159dd0 0x7fec6c158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec6c070d30 0x7fec6c157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 --2- 172.21.15.115:0/3074615051 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec6c06b6f0 0x7fec6c15c7c0 unknown :-1 s=CLOSED pgs=3115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:11.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 -- 172.21.15.115:0/3074615051 >> 172.21.15.115:0/3074615051 conn(0x7fec6c0ce880 msgr2=0x7fec6c158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:11.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 -- 172.21.15.115:0/3074615051 shutdown_connections 2023-12-01T03:13:11.892 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:11.889+0000 7fec74099640 1 -- 172.21.15.115:0/3074615051 wait complete. 2023-12-01T03:13:11.912 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-12-01T03:13:11.912 INFO:tasks.cephfs_test_runner: 2023-12-01T03:13:11.912 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-01T03:13:11.912 INFO:tasks.cephfs_test_runner:Ran 13 tests in 5776.779s 2023-12-01T03:13:11.912 INFO:tasks.cephfs_test_runner: 2023-12-01T03:13:11.912 INFO:tasks.cephfs_test_runner:OK 2023-12-01T03:13:11.913 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-12-01T03:13:11.922 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-01T03:13:11.937 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-01T03:13:11.938 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:13:11.938 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:13:11.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:13:11.994 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2023-12-01T03:13:11.994 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:13:12.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-01T03:13:12.046 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:13:12.046 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:13:12.107 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:13:12.107 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2023-12-01T03:13:12.107 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T03:13:12.163 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-01T03:13:12.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:12.163 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T03:13:12.223 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T03:13:12.223 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-01T03:13:12.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:12.224 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T03:13:12.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:12.322 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-01T03:13:12.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:12.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link delete ceph-brx 2023-12-01T03:13:12.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:12.559 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2023-12-01T03:13:12.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:12.606 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link delete ceph-brx 2023-12-01T03:13:12.737 INFO:teuthology.orchestra.run.smithi115.stderr:Cannot find device "ceph-brx" 2023-12-01T03:13:12.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T03:13:12.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:12.741 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2023-12-01T03:13:12.801 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-01T03:13:12.802 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-01T03:13:12.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:12.803 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-01T03:13:12.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:12.885 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-01T03:13:12.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:12.969 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2023-12-01T03:13:13.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:13.097 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2023-12-01T03:13:13.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T03:13:13.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2023-12-01T03:13:13.279 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2023-12-01T03:13:13.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T03:13:13.282 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-12-01T03:13:13.292 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-01T03:13:13.292 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:13:13.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.512+0000 7fa31ec2c640 1 Processor -- start 2023-12-01T03:13:13.514 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.512+0000 7fa31ec2c640 1 -- start start 2023-12-01T03:13:13.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.514+0000 7fa31ec2c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa31815d220 0x7fa3180cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.514+0000 7fa31ec2c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa318070ac0 0x7fa3180ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.514+0000 7fa31ec2c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31815d600 con 0x7fa3180ad2b0 2023-12-01T03:13:13.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.514+0000 7fa31ec2c640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31806cc10 con 0x7fa31815d220 2023-12-01T03:13:13.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.514+0000 7fa31ec2c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31806c720 con 0x7fa318070ac0 2023-12-01T03:13:13.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.514+0000 7fa31c9c8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa31815d220 0x7fa3180cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.514+0000 7fa31c9c8640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fa31815d220 0x7fa3180cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52676/0 (socket says 172.21.15.115:52676) 2023-12-01T03:13:13.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.514+0000 7fa31c9c8640 1 -- 172.21.15.115:0/2100380221 learned_addr learned my addr 172.21.15.115:0/2100380221 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:13.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.514+0000 7fa31d1c9640 1 --2- 172.21.15.115:0/2100380221 >> v2:172.21.15.142:3300/0 conn(0x7fa318070ac0 0x7fa3180ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.515+0000 7fa31c9c8640 1 -- 172.21.15.115:0/2100380221 >> v2:172.21.15.142:3300/0 conn(0x7fa318070ac0 msgr2=0x7fa3180ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:13.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.515+0000 7fa31c9c8640 1 --2- 172.21.15.115:0/2100380221 >> v2:172.21.15.142:3300/0 conn(0x7fa318070ac0 0x7fa3180ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.515+0000 7fa31c9c8640 1 -- 172.21.15.115:0/2100380221 >> v1:172.21.15.142:6789/0 conn(0x7fa3180ad2b0 legacy=0x7fa31806b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:13.517 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.515+0000 7fa31c9c8640 1 -- 172.21.15.115:0/2100380221 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31806c4b0 con 0x7fa31815d220 2023-12-01T03:13:13.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.516+0000 7fa31c9c8640 1 --2- 172.21.15.115:0/2100380221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa31815d220 0x7fa3180cc570 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 crypto rx=0x7fa3180c8e60 tx=0x7fa30c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc6b497e9daae31 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:13.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.516+0000 7fa3177fe640 1 -- 172.21.15.115:0/2100380221 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0317e0 con 0x7fa31815d220 2023-12-01T03:13:13.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.516+0000 7fa3177fe640 1 -- 172.21.15.115:0/2100380221 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c031940 con 0x7fa31815d220 2023-12-01T03:13:13.518 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.516+0000 7fa3177fe640 1 -- 172.21.15.115:0/2100380221 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c031cd0 con 0x7fa31815d220 2023-12-01T03:13:13.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.516+0000 7fa31ec2c640 1 -- 172.21.15.115:0/2100380221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa31815d220 msgr2=0x7fa3180cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:13.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.516+0000 7fa31ec2c640 1 --2- 172.21.15.115:0/2100380221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa31815d220 0x7fa3180cc570 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 crypto rx=0x7fa3180c8e60 tx=0x7fa30c02b880 comp rx=0 tx=0).stop 2023-12-01T03:13:13.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.517+0000 7fa31ec2c640 1 -- 172.21.15.115:0/2100380221 shutdown_connections 2023-12-01T03:13:13.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.517+0000 7fa31ec2c640 1 --2- 172.21.15.115:0/2100380221 >> v2:172.21.15.142:3300/0 conn(0x7fa318070ac0 0x7fa3180ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.517+0000 7fa31ec2c640 1 --2- 172.21.15.115:0/2100380221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa31815d220 0x7fa3180cc570 unknown :-1 s=CLOSED pgs=3116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.517+0000 7fa31ec2c640 1 -- 172.21.15.115:0/2100380221 >> 172.21.15.115:0/2100380221 conn(0x7fa3180ce880 msgr2=0x7fa3180cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:13.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.517+0000 7fa31ec2c640 1 -- 172.21.15.115:0/2100380221 shutdown_connections 2023-12-01T03:13:13.519 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.517+0000 7fa31ec2c640 1 -- 172.21.15.115:0/2100380221 wait complete. 2023-12-01T03:13:13.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.519+0000 7fa31ec2c640 1 Processor -- start 2023-12-01T03:13:13.520 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.519+0000 7fa31ec2c640 1 -- start start 2023-12-01T03:13:13.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31ec2c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318070ac0 0x7fa3181e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31ec2c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3180ad2b0 0x7fa31815b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31ec2c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31815d220 0x7fa3181576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31ec2c640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3181f26f0 con 0x7fa318070ac0 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31ec2c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3181e6810 con 0x7fa31815d220 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31ec2c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3181f61c0 con 0x7fa3180ad2b0 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31c9c8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318070ac0 0x7fa3181e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31c9c8640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318070ac0 0x7fa3181e62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52682/0 (socket says 172.21.15.115:52682) 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31c9c8640 1 -- 172.21.15.115:0/663833489 learned_addr learned my addr 172.21.15.115:0/663833489 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31c9c8640 1 -- 172.21.15.115:0/663833489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3180ad2b0 msgr2=0x7fa31815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa317fff640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3180ad2b0 0x7fa31815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31d1c9640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31815d220 0x7fa3181576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31c9c8640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3180ad2b0 0x7fa31815b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.520+0000 7fa31c9c8640 1 -- 172.21.15.115:0/663833489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31815d220 msgr2=0x7fa3181576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.521+0000 7fa31c9c8640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31815d220 0x7fa3181576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.521+0000 7fa31c9c8640 1 -- 172.21.15.115:0/663833489 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c0094d0 con 0x7fa318070ac0 2023-12-01T03:13:13.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.521+0000 7fa317fff640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3180ad2b0 0x7fa31815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:13.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.521+0000 7fa31c9c8640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318070ac0 0x7fa3181e62d0 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7fa30c02b8c0 tx=0x7fa30c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:13.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.521+0000 7fa3157fa640 1 -- 172.21.15.115:0/663833489 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0329c0 con 0x7fa318070ac0 2023-12-01T03:13:13.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.521+0000 7fa3157fa640 1 -- 172.21.15.115:0/663833489 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c032b40 con 0x7fa318070ac0 2023-12-01T03:13:13.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.521+0000 7fa31ec2c640 1 -- 172.21.15.115:0/663833489 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3181f64a0 con 0x7fa318070ac0 2023-12-01T03:13:13.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.521+0000 7fa3157fa640 1 -- 172.21.15.115:0/663833489 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0453f0 con 0x7fa318070ac0 2023-12-01T03:13:13.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.521+0000 7fa31ec2c640 1 -- 172.21.15.115:0/663833489 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3181f6a60 con 0x7fa318070ac0 2023-12-01T03:13:13.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.522+0000 7fa31ec2c640 1 -- 172.21.15.115:0/663833489 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31804e890 con 0x7fa318070ac0 2023-12-01T03:13:13.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.523+0000 7fa3157fa640 1 -- 172.21.15.115:0/663833489 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fa30c04e450 con 0x7fa318070ac0 2023-12-01T03:13:13.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.524+0000 7fa3157fa640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2fc06a550 0x7fa2fc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.524+0000 7fa3157fa640 1 -- 172.21.15.115:0/663833489 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7fa30c0aad80 con 0x7fa318070ac0 2023-12-01T03:13:13.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.525+0000 7fa317fff640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2fc06a550 0x7fa2fc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.525+0000 7fa317fff640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2fc06a550 0x7fa2fc06ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa3180b47f0 tx=0x7fa304008040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:13.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.529+0000 7fa3157fa640 1 -- 172.21.15.115:0/663833489 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fa30c076260 con 0x7fa318070ac0 2023-12-01T03:13:13.720 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.718+0000 7fa31ec2c640 1 -- 172.21.15.115:0/663833489 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa31806ce10 con 0x7fa2fc06a550 2023-12-01T03:13:13.727 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.725+0000 7fa3157fa640 1 -- 172.21.15.115:0/663833489 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48520 (secure 0 0 0) 0x7fa30400ad60 con 0x7fa2fc06a550 2023-12-01T03:13:13.727 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:13.728 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:13:13.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 -- 172.21.15.115:0/663833489 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2fc06a550 msgr2=0x7fa2fc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:13.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2fc06a550 0x7fa2fc06ca10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa3180b47f0 tx=0x7fa304008040 comp rx=0 tx=0).stop 2023-12-01T03:13:13.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 -- 172.21.15.115:0/663833489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318070ac0 msgr2=0x7fa3181e62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:13.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318070ac0 0x7fa3181e62d0 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 crypto rx=0x7fa30c02b8c0 tx=0x7fa30c0046c0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 -- 172.21.15.115:0/663833489 shutdown_connections 2023-12-01T03:13:13.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fa2fc06a550 0x7fa2fc06ca10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa3180ad2b0 0x7fa31815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31815d220 0x7fa3181576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 --2- 172.21.15.115:0/663833489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa318070ac0 0x7fa3181e62d0 unknown :-1 s=CLOSED pgs=3117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 -- 172.21.15.115:0/663833489 >> 172.21.15.115:0/663833489 conn(0x7fa3180ce880 msgr2=0x7fa3180afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:13.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 -- 172.21.15.115:0/663833489 shutdown_connections 2023-12-01T03:13:13.731 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.728+0000 7fa31ec2c640 1 -- 172.21.15.115:0/663833489 wait complete. 2023-12-01T03:13:13.748 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-12-01T03:13:13.101035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":444152,"kb_used_data":26808,"kb_used_omap":0,"kb_used_meta":417280,"kb_avail":754530568,"statfs":{"total":773094113280,"available":772639301632,"internally_reserved":0,"allocated":27451392,"data_stored":14131411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":427294720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668411,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-261,"num_write_kb":-2055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.363049"},"pg_stats":[{"pgid":"1.0","version":"258'381","reported_seq":1354,"reported_epoch":259,"state":"active+clean","last_fresh":"2023-12-01T03:13:06.806193+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:13:06.806193+0000","last_peered":"2023-12-01T03:13:06.806193+0000","last_clean":"2023-12-01T03:13:06.806193+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:13:06.806193+0000","last_undegraded":"2023-12-01T03:13:06.806193+0000","last_fullsized":"2023-12-01T03:13:06.806193+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576006,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3280896,"data_stored":1617971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0620000000000001,"15min":1.0760000000000001},"min":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.752},{"interface":"front","average":{"1min":1.343,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.95899999999999996}]},{"osd":1,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":0.83999999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.3069999999999999,"15min":1.327},"min":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.6679999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.167},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.95,"5min":1.95,"15min":2.1179999999999999},"last":1.526},{"interface":"front","average":{"1min":1.349,"5min":1.254,"15min":1.264},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.2070000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2729999999999999,"15min":1.302},"min":{"1min":0.78400000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":1.127},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.3959999999999999,"15min":1.425},"min":{"1min":0.84599999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":1.488}]},{"osd":4,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3140000000000001,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7989999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5029999999999999,"15min":1.508},"min":{"1min":1.1599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":1.9930000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.337,"15min":1.327},"min":{"1min":0.79600000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.5840000000000001,"15min":1.597},"min":{"1min":1.2430000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.8380000000000001,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":1.8560000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.476,"15min":1.454},"min":{"1min":0.89400000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.8579999999999997,"5min":4.8579999999999997,"15min":4.8579999999999997},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4319999999999999,"15min":1.4550000000000001},"min":{"1min":0.81799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":2.762},"last":1.3120000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576007,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3280896,"data_stored":1617971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.036,"15min":1.0980000000000001},"min":{"1min":0.53600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4810000000000001,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.093,"15min":1.1519999999999999},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6439999999999999,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.0609999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.127,"15min":1.202},"min":{"1min":0.57399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7010000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.542,"5min":1.694,"15min":1.835},"last":1.1279999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5369999999999999,"5min":1.742,"15min":1.976},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.272},"min":{"1min":0.71599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.3680000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.262,"15min":1.306},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7929999999999999,"5min":1.927,"15min":1.927},"last":1.417},{"interface":"front","average":{"1min":1.304,"5min":1.377,"15min":1.415},"min":{"1min":0.68700000000000006,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.339}]},{"osd":4,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2809999999999999,"15min":1.27},"min":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.661,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.4199999999999999,"15min":1.3660000000000001},"min":{"1min":0.64100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1019999999999999,"5min":2.48,"15min":2.48},"last":0.97799999999999998}]},{"osd":5,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3859999999999999,"15min":1.407},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.651,"5min":2.262,"15min":2.262},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.518,"15min":1.492},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0030000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.23}]},{"osd":7,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5069999999999999,"15min":1.526},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.865,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0539999999999998,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.292}]}]},{"osd":5,"up_from":13,"seq":55834576006,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52880,"kb_used_data":3784,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94318960,"statfs":{"total":96636764160,"available":96582615040,"internally_reserved":0,"allocated":3874816,"data_stored":2208339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.089,"15min":1.113},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.61299999999999999},"max":{"1min":1.4339999999999999,"5min":1.776,"15min":1.776},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1299999999999999,"15min":1.151},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.90400000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.1319999999999999,"15min":1.145},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":2.1349999999999998},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.179,"15min":1.218},"min":{"1min":0.74199999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5329999999999999,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":0.97999999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1970000000000001,"15min":1.2410000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.6499999999999999,"5min":1.901,"15min":1.901},"last":1.073},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.224,"15min":1.244},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.546,"5min":1.899,"15min":1.9219999999999999},"last":3.1579999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2989999999999999,"15min":1.3109999999999999},"min":{"1min":0.92700000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.5740000000000001,"5min":1.889,"15min":1.8979999999999999},"last":1.014},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.294,"15min":1.3460000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5960000000000001,"5min":1.97,"15min":2.1859999999999999},"last":2.0600000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3,"15min":1.258},"min":{"1min":0.90700000000000003,"5min":0.82899999999999996,"15min":0.78000000000000003},"max":{"1min":1.681,"5min":1.869,"15min":2.6040000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.399,"5min":1.369,"15min":1.3720000000000001},"min":{"1min":1.0349999999999999,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.6659999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.6850000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3899999999999999,"15min":1.3740000000000001},"min":{"1min":0.97399999999999998,"5min":0.93300000000000005,"15min":0.88800000000000001},"max":{"1min":1.6399999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.978},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.47,"15min":1.4259999999999999},"min":{"1min":1.3149999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7529999999999999,"5min":2.032,"15min":2.577},"last":2.1970000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.5489999999999999,"15min":1.5389999999999999},"min":{"1min":1.294,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.9550000000000001,"5min":2.077,"15min":3.1709999999999998},"last":1.752},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.591,"15min":1.5760000000000001},"min":{"1min":1.2669999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.883,"5min":2.101,"15min":2.8740000000000001},"last":2.621}]}]},{"osd":4,"up_from":13,"seq":55834576006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52312,"kb_used_data":3216,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319528,"statfs":{"total":96636764160,"available":96583196672,"internally_reserved":0,"allocated":3293184,"data_stored":1625026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.0549999999999999,"15min":1.1080000000000001},"min":{"1min":0.63200000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.343,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.179},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1850000000000001,"15min":1.173},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5589999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.109}]},{"osd":1,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.159,"15min":1.2250000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.617,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.997,"5min":1.1140000000000001,"15min":1.163},"min":{"1min":0.66000000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4550000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.375}]},{"osd":2,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2869999999999999,"15min":1.2330000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.482,"5min":11.311,"15min":11.311},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.236,"15min":1.3160000000000001},"min":{"1min":0.82899999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.212}]},{"osd":3,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2470000000000001,"15min":1.27},"min":{"1min":0.88800000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.877,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.27},{"interface":"front","average":{"1min":1.258,"5min":1.347,"15min":1.3300000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.667,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.2989999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.377,"15min":1.3300000000000001},"min":{"1min":0.751,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.716,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.327},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.349,"15min":1.3160000000000001},"min":{"1min":0.70399999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.0249999999999999,"5min":3.71,"15min":3.71},"last":1.4830000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3899999999999999,"15min":1.3720000000000001},"min":{"1min":0.84299999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8759999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.4890000000000001,"15min":1.47},"min":{"1min":0.86499999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1120000000000001,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.4410000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.486,"15min":1.4750000000000001},"min":{"1min":0.995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0289999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5840000000000001,"15min":1.573},"min":{"1min":0.95699999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.1480000000000001,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.681}]}]},{"osd":3,"up_from":13,"seq":55834576005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56388,"kb_used_data":3196,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315452,"statfs":{"total":96636764160,"available":96579022848,"internally_reserved":0,"allocated":3272704,"data_stored":1611328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.51600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.581,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":1.0589999999999999},"min":{"1min":0.54000000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.534,"5min":1.95,"15min":1.95},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.04,"15min":1.1279999999999999},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.635,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.92400000000000004,"15min":0.97699999999999998},"min":{"1min":0.57199999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.532,"5min":3.863,"15min":3.863},"last":0.627}]},{"osd":2,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1,"15min":1.0620000000000001},"min":{"1min":0.52000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3149999999999999,"5min":3.597,"15min":3.597},"last":0.51300000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.139,"15min":1.2090000000000001},"min":{"1min":0.60499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5609999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2549999999999999,"15min":1.282},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.383,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.004},{"interface":"front","average":{"1min":1.232,"5min":1.2529999999999999,"15min":1.2849999999999999},"min":{"1min":0.85599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9890000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2889999999999999,"15min":1.3180000000000001},"min":{"1min":0.82299999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8979999999999999,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.359,"5min":1.343,"15min":1.353},"min":{"1min":0.81799999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.0830000000000002,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.747}]},{"osd":6,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3580000000000001,"15min":1.4259999999999999},"min":{"1min":0.83499999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.3079999999999998,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":1.125},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.448,"15min":1.488},"min":{"1min":0.997,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4870000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.452,"15min":1.502},"min":{"1min":1.0429999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.0470000000000002,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.462,"15min":1.516},"min":{"1min":0.96199999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.2440000000000002,"5min":3.004,"15min":3.004},"last":1.028}]}]},{"osd":2,"up_from":13,"seq":55834576005,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48216,"kb_used_data":3216,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323624,"statfs":{"total":96636764160,"available":96587390976,"internally_reserved":0,"allocated":3293184,"data_stored":1624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79700000000000004,"15min":0.86099999999999999},"min":{"1min":0.503,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.90800000000000003,"5min":1.411,"15min":1.4730000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.91900000000000004,"15min":1.0149999999999999},"min":{"1min":0.54400000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.198,"5min":1.5640000000000001,"15min":1.7310000000000001},"last":1.3660000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":1.02},"min":{"1min":0.64600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.2569999999999999,"5min":1.512,"15min":1.698},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.97899999999999998,"15min":1.103},"min":{"1min":0.54000000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4399999999999999,"5min":1.5800000000000001,"15min":1.6599999999999999},"last":1.26}]},{"osd":3,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95299999999999996,"15min":1.014},"min":{"1min":0.45500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.502,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.0609999999999999,"15min":1.171},"min":{"1min":0.63900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.367,"5min":1.615,"15min":1.79},"last":1.1619999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.22,"15min":1.302},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.609,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.1759999999999999,"15min":1.2609999999999999},"min":{"1min":0.64300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.508,"5min":1.714,"15min":1.835},"last":1.3080000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.282,"15min":1.3660000000000001},"min":{"1min":0.97599999999999998,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7509999999999999,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.296,"15min":1.387},"min":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5569999999999999,"5min":1.8740000000000001,"15min":2.016},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3149999999999999,"15min":1.3640000000000001},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7190000000000001,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3779999999999999,"15min":1.4690000000000001},"min":{"1min":1.024,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.8740000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.5880000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3819999999999999,"15min":1.4590000000000001},"min":{"1min":0.89400000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.9099999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.3500000000000001,"15min":1.4239999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9610000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.544}]}]},{"osd":1,"up_from":13,"seq":55834576004,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52880,"kb_used_data":3784,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94318960,"statfs":{"total":96636764160,"available":96582615040,"internally_reserved":0,"allocated":3874816,"data_stored":2208339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.81100000000000005,"15min":0.877},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.093,"5min":1.5569999999999999,"15min":1.6850000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88800000000000001,"15min":1.006},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.353,"5min":1.512,"15min":1.681},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82099999999999995,"15min":0.91300000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.335,"5min":1.6599999999999999,"15min":1.7030000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.98199999999999998,"15min":1.101},"min":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3819999999999999,"5min":1.581,"15min":1.7430000000000001},"last":0.80500000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.069,"15min":1.244},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.3799999999999999,"5min":1.593,"15min":1.788},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.998,"15min":1.093},"min":{"1min":0.48699999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3600000000000001,"5min":1.714,"15min":1.714},"last":0.85199999999999998}]},{"osd":4,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1519999999999999,"15min":1.202},"min":{"1min":0.61099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.302,"15min":1.381},"min":{"1min":0.76300000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.2109999999999999},"last":0.998}]},{"osd":5,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.29,"15min":1.3200000000000001},"min":{"1min":0.81499999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":1.083},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.2749999999999999,"15min":1.327},"min":{"1min":0.66100000000000003,"5min":0.496,"15min":0.496},"max":{"1min":3.75,"5min":3.75,"15min":3.75},"last":1.278}]},{"osd":6,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.327,"15min":1.4079999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5580000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.452},{"interface":"front","average":{"1min":1.28,"5min":1.345,"15min":1.393},"min":{"1min":0.70499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8700000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.5389999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.4219999999999999,"15min":1.502},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.5350000000000001,"5min":3.5350000000000001,"15min":3.5350000000000001},"last":1.123},{"interface":"front","average":{"1min":1.268,"5min":1.413,"15min":1.482},"min":{"1min":0.80400000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.617,"5min":2.1389999999999998,"15min":2.2069999999999999},"last":1.2150000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576006,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3280896,"data_stored":1617971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.54500000000000004,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.175,"5min":3.472,"15min":3.472},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91400000000000003,"15min":1.026},"min":{"1min":0.58899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1950000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.002,"15min":1.056},"min":{"1min":0.5,"5min":0.5,"15min":0.48699999999999999},"max":{"1min":1.2849999999999999,"5min":2.0179999999999998,"15min":2.1429999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98399999999999999,"15min":1.097},"min":{"1min":0.61299999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4770000000000001,"5min":1.726,"15min":1.792},"last":0.84699999999999998}]},{"osd":3,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93899999999999995,"15min":1.0369999999999999},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.262,"5min":1.536,"15min":1.8839999999999999},"last":0.437},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.135,"15min":1.21},"min":{"1min":0.629,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5369999999999999,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.875}]},{"osd":4,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.2769999999999999,"15min":1.3420000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.206,"5min":1.2170000000000001,"15min":1.284},"min":{"1min":0.67600000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.752,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1220000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3300000000000001,"15min":1.393},"min":{"1min":0.81100000000000005,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.986,"5min":1.986,"15min":2.7719999999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.3400000000000001,"15min":1.413},"min":{"1min":0.84499999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7969999999999999,"5min":2.484,"15min":2.5880000000000001},"last":1.266}]},{"osd":6,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.298,"15min":1.363},"min":{"1min":0.94399999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.835,"15min":2.2029999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4359999999999999,"15min":1.4770000000000001},"min":{"1min":0.84799999999999998,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.8759999999999999},"last":1.226}]},{"osd":7,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4330000000000001,"15min":1.502},"min":{"1min":1.0960000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2069999999999999,"5min":2.411,"15min":2.7040000000000002},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":0.92300000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":0.91200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:13:13.750 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:13:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.964+0000 7f1cce01e640 1 Processor -- start 2023-12-01T03:13:13.966 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.964+0000 7f1cce01e640 1 -- start start 2023-12-01T03:13:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.965+0000 7f1cce01e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cc80c9ef0 0x7f1cc806fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.965+0000 7f1cce01e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1cc806b6f0 0x7f1cc8070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.965+0000 7f1cce01e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc806f8b0 con 0x7f1cc8159dd0 2023-12-01T03:13:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.965+0000 7f1cce01e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc80adcb0 con 0x7f1cc806b6f0 2023-12-01T03:13:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.965+0000 7f1cce01e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80ad9d0 con 0x7f1cc80c9ef0 2023-12-01T03:13:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.966+0000 7f1cc7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1cc806b6f0 0x7f1cc8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.966+0000 7f1cc7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f1cc806b6f0 0x7f1cc8070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52692/0 (socket says 172.21.15.115:52692) 2023-12-01T03:13:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.966+0000 7f1cc7fff640 1 -- 172.21.15.115:0/2762748798 learned_addr learned my addr 172.21.15.115:0/2762748798 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.966+0000 7f1cbffff640 1 --2- 172.21.15.115:0/2762748798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cc80c9ef0 0x7f1cc806fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.968 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.966+0000 7f1cc7fff640 1 -- 172.21.15.115:0/2762748798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cc80c9ef0 msgr2=0x7f1cc806fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.966+0000 7f1cc7fff640 1 --2- 172.21.15.115:0/2762748798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cc80c9ef0 0x7f1cc806fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.966+0000 7f1cc7fff640 1 -- 172.21.15.115:0/2762748798 >> v1:172.21.15.142:6789/0 conn(0x7f1cc8159dd0 legacy=0x7f1cc80c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.966+0000 7f1cc7fff640 1 -- 172.21.15.115:0/2762748798 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80c9000 con 0x7f1cc806b6f0 2023-12-01T03:13:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.967+0000 7f1cc7fff640 1 --2- 172.21.15.115:0/2762748798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 0x7f1cc8070260 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009e50 tx=0x7f1cc002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde0fcf8247f6b5b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:13.969 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.968+0000 7f1cc6ffd640 1 -- 172.21.15.115:0/2762748798 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc002c950 con 0x7f1cc806b6f0 2023-12-01T03:13:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.968+0000 7f1cc6ffd640 1 -- 172.21.15.115:0/2762748798 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc002cab0 con 0x7f1cc806b6f0 2023-12-01T03:13:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.968+0000 7f1cc6ffd640 1 -- 172.21.15.115:0/2762748798 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc002ce40 con 0x7f1cc806b6f0 2023-12-01T03:13:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.968+0000 7f1cce01e640 1 -- 172.21.15.115:0/2762748798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 msgr2=0x7f1cc8070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:13.970 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.968+0000 7f1cce01e640 1 --2- 172.21.15.115:0/2762748798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 0x7f1cc8070260 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009e50 tx=0x7f1cc002bdb0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.969+0000 7f1cce01e640 1 -- 172.21.15.115:0/2762748798 shutdown_connections 2023-12-01T03:13:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.969+0000 7f1cce01e640 1 --2- 172.21.15.115:0/2762748798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 0x7f1cc8070260 unknown :-1 s=CLOSED pgs=3118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.969+0000 7f1cce01e640 1 --2- 172.21.15.115:0/2762748798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cc80c9ef0 0x7f1cc806fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.969+0000 7f1cce01e640 1 -- 172.21.15.115:0/2762748798 >> 172.21.15.115:0/2762748798 conn(0x7f1cc80ce880 msgr2=0x7f1cc80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.969+0000 7f1cce01e640 1 -- 172.21.15.115:0/2762748798 shutdown_connections 2023-12-01T03:13:13.971 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.969+0000 7f1cce01e640 1 -- 172.21.15.115:0/2762748798 wait complete. 2023-12-01T03:13:13.972 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.970+0000 7f1cce01e640 1 Processor -- start 2023-12-01T03:13:13.973 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.971+0000 7f1cce01e640 1 -- start start 2023-12-01T03:13:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.971+0000 7f1cce01e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 0x7f1cc80accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.971+0000 7f1cce01e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc80c9ef0 0x7f1cc81e5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cce01e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cc8159dd0 0x7f1cc80c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cce01e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8148420 con 0x7f1cc806b6f0 2023-12-01T03:13:13.974 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cce01e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80ad200 con 0x7f1cc80c9ef0 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cce01e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc81e4dc0 con 0x7f1cc8159dd0 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 0x7f1cc80accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 0x7f1cc80accc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52702/0 (socket says 172.21.15.115:52702) 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cc77fe640 1 -- 172.21.15.115:0/2525332183 learned_addr learned my addr 172.21.15.115:0/2525332183 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cc7fff640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cc8159dd0 0x7f1cc80c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cbffff640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc80c9ef0 0x7f1cc81e5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cc77fe640 1 -- 172.21.15.115:0/2525332183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cc8159dd0 msgr2=0x7f1cc80c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cc77fe640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cc8159dd0 0x7f1cc80c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cc77fe640 1 -- 172.21.15.115:0/2525332183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc80c9ef0 msgr2=0x7f1cc81e5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cc77fe640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc80c9ef0 0x7f1cc81e5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.972+0000 7f1cc77fe640 1 -- 172.21.15.115:0/2525332183 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc0009b30 con 0x7f1cc806b6f0 2023-12-01T03:13:13.975 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.973+0000 7f1cc77fe640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 0x7f1cc80accc0 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800b3f0 tx=0x7f1cb800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:13.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.973+0000 7f1cc4ff9640 1 -- 172.21.15.115:0/2525332183 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8011830 con 0x7f1cc806b6f0 2023-12-01T03:13:13.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.973+0000 7f1cc4ff9640 1 -- 172.21.15.115:0/2525332183 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb8011990 con 0x7f1cc806b6f0 2023-12-01T03:13:13.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.973+0000 7f1cc4ff9640 1 -- 172.21.15.115:0/2525332183 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8011d80 con 0x7f1cc806b6f0 2023-12-01T03:13:13.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.973+0000 7f1cce01e640 1 -- 172.21.15.115:0/2525332183 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc81e44d0 con 0x7f1cc806b6f0 2023-12-01T03:13:13.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.973+0000 7f1cce01e640 1 -- 172.21.15.115:0/2525332183 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc8148140 con 0x7f1cc806b6f0 2023-12-01T03:13:13.976 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.974+0000 7f1cce01e640 1 -- 172.21.15.115:0/2525332183 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc804e890 con 0x7f1cc806b6f0 2023-12-01T03:13:13.977 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.975+0000 7f1cc4ff9640 1 -- 172.21.15.115:0/2525332183 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f1cb801a400 con 0x7f1cc806b6f0 2023-12-01T03:13:13.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.976+0000 7f1cc4ff9640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1cac06a550 0x7f1cac06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:13.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.976+0000 7f1cc4ff9640 1 -- 172.21.15.115:0/2525332183 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f1cb80889c0 con 0x7f1cc806b6f0 2023-12-01T03:13:13.978 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.976+0000 7f1cbffff640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1cac06a550 0x7f1cac06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:13.979 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.977+0000 7f1cbffff640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1cac06a550 0x7f1cac06ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1cc81e4260 tx=0x7f1cb0008040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:13.982 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:13.980+0000 7f1cc4ff9640 1 -- 172.21.15.115:0/2525332183 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f1cb8053d60 con 0x7f1cc806b6f0 2023-12-01T03:13:14.170 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.168+0000 7f1cce01e640 1 -- 172.21.15.115:0/2525332183 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1cc806d0d0 con 0x7f1cac06a550 2023-12-01T03:13:14.176 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.174+0000 7f1cc4ff9640 1 -- 172.21.15.115:0/2525332183 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48520 (secure 0 0 0) 0x7f1cb000b200 con 0x7f1cac06a550 2023-12-01T03:13:14.176 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:14.177 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 -- 172.21.15.115:0/2525332183 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1cac06a550 msgr2=0x7f1cac06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1cac06a550 0x7f1cac06ca10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1cc81e4260 tx=0x7f1cb0008040 comp rx=0 tx=0).stop 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 -- 172.21.15.115:0/2525332183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 msgr2=0x7f1cc80accc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 0x7f1cc80accc0 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800b3f0 tx=0x7f1cb800b8b0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 -- 172.21.15.115:0/2525332183 shutdown_connections 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f1cac06a550 0x7f1cac06ca10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1cc8159dd0 0x7f1cc80c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc80c9ef0 0x7f1cc81e5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 --2- 172.21.15.115:0/2525332183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cc806b6f0 0x7f1cc80accc0 unknown :-1 s=CLOSED pgs=3119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 -- 172.21.15.115:0/2525332183 >> 172.21.15.115:0/2525332183 conn(0x7f1cc80ce880 msgr2=0x7f1cc8158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 -- 172.21.15.115:0/2525332183 shutdown_connections 2023-12-01T03:13:14.179 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.177+0000 7f1cce01e640 1 -- 172.21.15.115:0/2525332183 wait complete. 2023-12-01T03:13:14.198 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-12-01T03:13:13.101035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":444152,"kb_used_data":26808,"kb_used_omap":0,"kb_used_meta":417280,"kb_avail":754530568,"statfs":{"total":773094113280,"available":772639301632,"internally_reserved":0,"allocated":27451392,"data_stored":14131411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":427294720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668411,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-261,"num_write_kb":-2055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.363049"},"pg_stats":[{"pgid":"1.0","version":"258'381","reported_seq":1354,"reported_epoch":259,"state":"active+clean","last_fresh":"2023-12-01T03:13:06.806193+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:13:06.806193+0000","last_peered":"2023-12-01T03:13:06.806193+0000","last_clean":"2023-12-01T03:13:06.806193+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:13:06.806193+0000","last_undegraded":"2023-12-01T03:13:06.806193+0000","last_fullsized":"2023-12-01T03:13:06.806193+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576006,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3280896,"data_stored":1617971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0620000000000001,"15min":1.0760000000000001},"min":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.752},{"interface":"front","average":{"1min":1.343,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.95899999999999996}]},{"osd":1,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":0.83999999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.3069999999999999,"15min":1.327},"min":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.6679999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.167},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.95,"5min":1.95,"15min":2.1179999999999999},"last":1.526},{"interface":"front","average":{"1min":1.349,"5min":1.254,"15min":1.264},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.2070000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2729999999999999,"15min":1.302},"min":{"1min":0.78400000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":1.127},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.3959999999999999,"15min":1.425},"min":{"1min":0.84599999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":1.488}]},{"osd":4,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3140000000000001,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7989999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5029999999999999,"15min":1.508},"min":{"1min":1.1599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":1.9930000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.337,"15min":1.327},"min":{"1min":0.79600000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.5840000000000001,"15min":1.597},"min":{"1min":1.2430000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.8380000000000001,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":1.8560000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.476,"15min":1.454},"min":{"1min":0.89400000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.8579999999999997,"5min":4.8579999999999997,"15min":4.8579999999999997},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4319999999999999,"15min":1.4550000000000001},"min":{"1min":0.81799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":2.762},"last":1.3120000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576007,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3280896,"data_stored":1617971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.036,"15min":1.0980000000000001},"min":{"1min":0.53600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4810000000000001,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.093,"15min":1.1519999999999999},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6439999999999999,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.0609999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.127,"15min":1.202},"min":{"1min":0.57399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7010000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.542,"5min":1.694,"15min":1.835},"last":1.1279999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5369999999999999,"5min":1.742,"15min":1.976},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.272},"min":{"1min":0.71599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.3680000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.262,"15min":1.306},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7929999999999999,"5min":1.927,"15min":1.927},"last":1.417},{"interface":"front","average":{"1min":1.304,"5min":1.377,"15min":1.415},"min":{"1min":0.68700000000000006,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.339}]},{"osd":4,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2809999999999999,"15min":1.27},"min":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.661,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.4199999999999999,"15min":1.3660000000000001},"min":{"1min":0.64100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1019999999999999,"5min":2.48,"15min":2.48},"last":0.97799999999999998}]},{"osd":5,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3859999999999999,"15min":1.407},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.651,"5min":2.262,"15min":2.262},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.518,"15min":1.492},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0030000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.23}]},{"osd":7,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5069999999999999,"15min":1.526},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.865,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0539999999999998,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.292}]}]},{"osd":5,"up_from":13,"seq":55834576006,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52880,"kb_used_data":3784,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94318960,"statfs":{"total":96636764160,"available":96582615040,"internally_reserved":0,"allocated":3874816,"data_stored":2208339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.089,"15min":1.113},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.61299999999999999},"max":{"1min":1.4339999999999999,"5min":1.776,"15min":1.776},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1299999999999999,"15min":1.151},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.90400000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.1319999999999999,"15min":1.145},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":2.1349999999999998},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.179,"15min":1.218},"min":{"1min":0.74199999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5329999999999999,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":0.97999999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1970000000000001,"15min":1.2410000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.6499999999999999,"5min":1.901,"15min":1.901},"last":1.073},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.224,"15min":1.244},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.546,"5min":1.899,"15min":1.9219999999999999},"last":3.1579999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2989999999999999,"15min":1.3109999999999999},"min":{"1min":0.92700000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.5740000000000001,"5min":1.889,"15min":1.8979999999999999},"last":1.014},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.294,"15min":1.3460000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5960000000000001,"5min":1.97,"15min":2.1859999999999999},"last":2.0600000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3,"15min":1.258},"min":{"1min":0.90700000000000003,"5min":0.82899999999999996,"15min":0.78000000000000003},"max":{"1min":1.681,"5min":1.869,"15min":2.6040000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.399,"5min":1.369,"15min":1.3720000000000001},"min":{"1min":1.0349999999999999,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.6659999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.6850000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3899999999999999,"15min":1.3740000000000001},"min":{"1min":0.97399999999999998,"5min":0.93300000000000005,"15min":0.88800000000000001},"max":{"1min":1.6399999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.978},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.47,"15min":1.4259999999999999},"min":{"1min":1.3149999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7529999999999999,"5min":2.032,"15min":2.577},"last":2.1970000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.5489999999999999,"15min":1.5389999999999999},"min":{"1min":1.294,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.9550000000000001,"5min":2.077,"15min":3.1709999999999998},"last":1.752},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.591,"15min":1.5760000000000001},"min":{"1min":1.2669999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.883,"5min":2.101,"15min":2.8740000000000001},"last":2.621}]}]},{"osd":4,"up_from":13,"seq":55834576006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52312,"kb_used_data":3216,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319528,"statfs":{"total":96636764160,"available":96583196672,"internally_reserved":0,"allocated":3293184,"data_stored":1625026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.0549999999999999,"15min":1.1080000000000001},"min":{"1min":0.63200000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.343,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.179},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1850000000000001,"15min":1.173},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5589999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.109}]},{"osd":1,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.159,"15min":1.2250000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.617,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.997,"5min":1.1140000000000001,"15min":1.163},"min":{"1min":0.66000000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4550000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.375}]},{"osd":2,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2869999999999999,"15min":1.2330000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.482,"5min":11.311,"15min":11.311},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.236,"15min":1.3160000000000001},"min":{"1min":0.82899999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.212}]},{"osd":3,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2470000000000001,"15min":1.27},"min":{"1min":0.88800000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.877,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.27},{"interface":"front","average":{"1min":1.258,"5min":1.347,"15min":1.3300000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.667,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.2989999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.377,"15min":1.3300000000000001},"min":{"1min":0.751,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.716,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.327},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.349,"15min":1.3160000000000001},"min":{"1min":0.70399999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.0249999999999999,"5min":3.71,"15min":3.71},"last":1.4830000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3899999999999999,"15min":1.3720000000000001},"min":{"1min":0.84299999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8759999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.4890000000000001,"15min":1.47},"min":{"1min":0.86499999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1120000000000001,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.4410000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.486,"15min":1.4750000000000001},"min":{"1min":0.995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0289999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5840000000000001,"15min":1.573},"min":{"1min":0.95699999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.1480000000000001,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.681}]}]},{"osd":3,"up_from":13,"seq":55834576005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56388,"kb_used_data":3196,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315452,"statfs":{"total":96636764160,"available":96579022848,"internally_reserved":0,"allocated":3272704,"data_stored":1611328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.51600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.581,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":1.0589999999999999},"min":{"1min":0.54000000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.534,"5min":1.95,"15min":1.95},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.04,"15min":1.1279999999999999},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.635,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.92400000000000004,"15min":0.97699999999999998},"min":{"1min":0.57199999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.532,"5min":3.863,"15min":3.863},"last":0.627}]},{"osd":2,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1,"15min":1.0620000000000001},"min":{"1min":0.52000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3149999999999999,"5min":3.597,"15min":3.597},"last":0.51300000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.139,"15min":1.2090000000000001},"min":{"1min":0.60499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5609999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2549999999999999,"15min":1.282},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.383,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.004},{"interface":"front","average":{"1min":1.232,"5min":1.2529999999999999,"15min":1.2849999999999999},"min":{"1min":0.85599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9890000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2889999999999999,"15min":1.3180000000000001},"min":{"1min":0.82299999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8979999999999999,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.359,"5min":1.343,"15min":1.353},"min":{"1min":0.81799999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.0830000000000002,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.747}]},{"osd":6,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3580000000000001,"15min":1.4259999999999999},"min":{"1min":0.83499999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.3079999999999998,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":1.125},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.448,"15min":1.488},"min":{"1min":0.997,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4870000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.452,"15min":1.502},"min":{"1min":1.0429999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.0470000000000002,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.462,"15min":1.516},"min":{"1min":0.96199999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.2440000000000002,"5min":3.004,"15min":3.004},"last":1.028}]}]},{"osd":2,"up_from":13,"seq":55834576005,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48216,"kb_used_data":3216,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323624,"statfs":{"total":96636764160,"available":96587390976,"internally_reserved":0,"allocated":3293184,"data_stored":1624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79700000000000004,"15min":0.86099999999999999},"min":{"1min":0.503,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.90800000000000003,"5min":1.411,"15min":1.4730000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.91900000000000004,"15min":1.0149999999999999},"min":{"1min":0.54400000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.198,"5min":1.5640000000000001,"15min":1.7310000000000001},"last":1.3660000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":1.02},"min":{"1min":0.64600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.2569999999999999,"5min":1.512,"15min":1.698},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.97899999999999998,"15min":1.103},"min":{"1min":0.54000000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4399999999999999,"5min":1.5800000000000001,"15min":1.6599999999999999},"last":1.26}]},{"osd":3,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95299999999999996,"15min":1.014},"min":{"1min":0.45500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.502,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.0609999999999999,"15min":1.171},"min":{"1min":0.63900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.367,"5min":1.615,"15min":1.79},"last":1.1619999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.22,"15min":1.302},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.609,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.1759999999999999,"15min":1.2609999999999999},"min":{"1min":0.64300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.508,"5min":1.714,"15min":1.835},"last":1.3080000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.282,"15min":1.3660000000000001},"min":{"1min":0.97599999999999998,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7509999999999999,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.296,"15min":1.387},"min":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5569999999999999,"5min":1.8740000000000001,"15min":2.016},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3149999999999999,"15min":1.3640000000000001},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7190000000000001,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3779999999999999,"15min":1.4690000000000001},"min":{"1min":1.024,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.8740000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.5880000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3819999999999999,"15min":1.4590000000000001},"min":{"1min":0.89400000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.9099999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.3500000000000001,"15min":1.4239999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9610000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.544}]}]},{"osd":1,"up_from":13,"seq":55834576004,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52880,"kb_used_data":3784,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94318960,"statfs":{"total":96636764160,"available":96582615040,"internally_reserved":0,"allocated":3874816,"data_stored":2208339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.81100000000000005,"15min":0.877},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.093,"5min":1.5569999999999999,"15min":1.6850000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88800000000000001,"15min":1.006},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.353,"5min":1.512,"15min":1.681},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82099999999999995,"15min":0.91300000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.335,"5min":1.6599999999999999,"15min":1.7030000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.98199999999999998,"15min":1.101},"min":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3819999999999999,"5min":1.581,"15min":1.7430000000000001},"last":0.80500000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.069,"15min":1.244},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.3799999999999999,"5min":1.593,"15min":1.788},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.998,"15min":1.093},"min":{"1min":0.48699999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3600000000000001,"5min":1.714,"15min":1.714},"last":0.85199999999999998}]},{"osd":4,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1519999999999999,"15min":1.202},"min":{"1min":0.61099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.302,"15min":1.381},"min":{"1min":0.76300000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.2109999999999999},"last":0.998}]},{"osd":5,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.29,"15min":1.3200000000000001},"min":{"1min":0.81499999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":1.083},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.2749999999999999,"15min":1.327},"min":{"1min":0.66100000000000003,"5min":0.496,"15min":0.496},"max":{"1min":3.75,"5min":3.75,"15min":3.75},"last":1.278}]},{"osd":6,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.327,"15min":1.4079999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5580000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.452},{"interface":"front","average":{"1min":1.28,"5min":1.345,"15min":1.393},"min":{"1min":0.70499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8700000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.5389999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.4219999999999999,"15min":1.502},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.5350000000000001,"5min":3.5350000000000001,"15min":3.5350000000000001},"last":1.123},{"interface":"front","average":{"1min":1.268,"5min":1.413,"15min":1.482},"min":{"1min":0.80400000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.617,"5min":2.1389999999999998,"15min":2.2069999999999999},"last":1.2150000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576006,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3280896,"data_stored":1617971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.54500000000000004,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.175,"5min":3.472,"15min":3.472},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91400000000000003,"15min":1.026},"min":{"1min":0.58899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1950000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.002,"15min":1.056},"min":{"1min":0.5,"5min":0.5,"15min":0.48699999999999999},"max":{"1min":1.2849999999999999,"5min":2.0179999999999998,"15min":2.1429999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98399999999999999,"15min":1.097},"min":{"1min":0.61299999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4770000000000001,"5min":1.726,"15min":1.792},"last":0.84699999999999998}]},{"osd":3,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93899999999999995,"15min":1.0369999999999999},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.262,"5min":1.536,"15min":1.8839999999999999},"last":0.437},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.135,"15min":1.21},"min":{"1min":0.629,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5369999999999999,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.875}]},{"osd":4,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.2769999999999999,"15min":1.3420000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.206,"5min":1.2170000000000001,"15min":1.284},"min":{"1min":0.67600000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.752,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1220000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3300000000000001,"15min":1.393},"min":{"1min":0.81100000000000005,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.986,"5min":1.986,"15min":2.7719999999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.3400000000000001,"15min":1.413},"min":{"1min":0.84499999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7969999999999999,"5min":2.484,"15min":2.5880000000000001},"last":1.266}]},{"osd":6,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.298,"15min":1.363},"min":{"1min":0.94399999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.835,"15min":2.2029999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4359999999999999,"15min":1.4770000000000001},"min":{"1min":0.84799999999999998,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.8759999999999999},"last":1.226}]},{"osd":7,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4330000000000001,"15min":1.502},"min":{"1min":1.0960000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2069999999999999,"5min":2.411,"15min":2.7040000000000002},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":0.92300000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":0.91200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:13:14.200 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-01T03:13:14.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:13:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.410+0000 7f0c32eee640 1 Processor -- start 2023-12-01T03:13:14.412 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.410+0000 7f0c32eee640 1 -- start start 2023-12-01T03:13:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.412+0000 7f0c32eee640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0c2c0c9ef0 0x7f0c2c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.412+0000 7f0c32eee640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0c2c06b6f0 0x7f0c2c070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.412+0000 7f0c32eee640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2c06f8b0 con 0x7f0c2c159dd0 2023-12-01T03:13:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.412+0000 7f0c32eee640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c0adcb0 con 0x7f0c2c0c9ef0 2023-12-01T03:13:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.412+0000 7f0c32eee640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c0ad9d0 con 0x7f0c2c06b6f0 2023-12-01T03:13:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.412+0000 7f0c2bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0c2c0c9ef0 0x7f0c2c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.412+0000 7f0c30c8a640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f0c2c159dd0 0x7f0c2c0c97e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58854/0 (socket says 172.21.15.115:58854) 2023-12-01T03:13:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.412+0000 7f0c2bfff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f0c2c0c9ef0 0x7f0c2c06fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52704/0 (socket says 172.21.15.115:52704) 2023-12-01T03:13:14.414 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.412+0000 7f0c30c8a640 1 -- 172.21.15.115:0/3267422661 learned_addr learned my addr 172.21.15.115:0/3267422661 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.413+0000 7f0c3148b640 1 --2- 172.21.15.115:0/3267422661 >> v2:172.21.15.142:3300/0 conn(0x7f0c2c06b6f0 0x7f0c2c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.413+0000 7f0c2bfff640 1 -- 172.21.15.115:0/3267422661 >> v2:172.21.15.142:3300/0 conn(0x7f0c2c06b6f0 msgr2=0x7f0c2c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.413+0000 7f0c2bfff640 1 --2- 172.21.15.115:0/3267422661 >> v2:172.21.15.142:3300/0 conn(0x7f0c2c06b6f0 0x7f0c2c070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.413+0000 7f0c2bfff640 1 -- 172.21.15.115:0/3267422661 >> v1:172.21.15.115:6789/0 conn(0x7f0c2c159dd0 legacy=0x7f0c2c0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.413+0000 7f0c2b7fe640 1 -- 172.21.15.115:0/3267422661 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207565654 0 0) 0x7f0c2c06f8b0 con 0x7f0c2c159dd0 2023-12-01T03:13:14.415 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.413+0000 7f0c2bfff640 1 -- 172.21.15.115:0/3267422661 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c0c9000 con 0x7f0c2c0c9ef0 2023-12-01T03:13:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.414+0000 7f0c2bfff640 1 --2- 172.21.15.115:0/3267422661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c0c9ef0 0x7f0c2c06fd20 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c06d470 tx=0x7f0c1802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6823438c277167 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.415+0000 7f0c2b7fe640 1 -- 172.21.15.115:0/3267422661 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18031710 con 0x7f0c2c0c9ef0 2023-12-01T03:13:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.415+0000 7f0c2b7fe640 1 -- 172.21.15.115:0/3267422661 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1802f050 con 0x7f0c2c0c9ef0 2023-12-01T03:13:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.415+0000 7f0c2b7fe640 1 -- 172.21.15.115:0/3267422661 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18031a90 con 0x7f0c2c0c9ef0 2023-12-01T03:13:14.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.415+0000 7f0c32eee640 1 -- 172.21.15.115:0/3267422661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c0c9ef0 msgr2=0x7f0c2c06fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.415+0000 7f0c32eee640 1 --2- 172.21.15.115:0/3267422661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c0c9ef0 0x7f0c2c06fd20 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c06d470 tx=0x7f0c1802b7b0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.415+0000 7f0c32eee640 1 -- 172.21.15.115:0/3267422661 shutdown_connections 2023-12-01T03:13:14.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.415+0000 7f0c32eee640 1 --2- 172.21.15.115:0/3267422661 >> v2:172.21.15.142:3300/0 conn(0x7f0c2c06b6f0 0x7f0c2c070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.417 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.415+0000 7f0c32eee640 1 --2- 172.21.15.115:0/3267422661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c0c9ef0 0x7f0c2c06fd20 unknown :-1 s=CLOSED pgs=3121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.415+0000 7f0c32eee640 1 -- 172.21.15.115:0/3267422661 >> 172.21.15.115:0/3267422661 conn(0x7f0c2c0ce880 msgr2=0x7f0c2c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:14.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.416+0000 7f0c32eee640 1 -- 172.21.15.115:0/3267422661 shutdown_connections 2023-12-01T03:13:14.418 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.416+0000 7f0c32eee640 1 -- 172.21.15.115:0/3267422661 wait complete. 2023-12-01T03:13:14.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.417+0000 7f0c32eee640 1 Processor -- start 2023-12-01T03:13:14.419 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.417+0000 7f0c32eee640 1 -- start start 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.418+0000 7f0c32eee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c06b6f0 0x7f0c2c06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.418+0000 7f0c32eee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c0c9ef0 0x7f0c2c0c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.418+0000 7f0c32eee640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c159dd0 0x7f0c2c14cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.418+0000 7f0c32eee640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0c5e90 con 0x7f0c2c159dd0 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.418+0000 7f0c32eee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0702b0 con 0x7f0c2c0c9ef0 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.418+0000 7f0c32eee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c070590 con 0x7f0c2c06b6f0 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c3148b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c159dd0 0x7f0c2c14cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c3148b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c159dd0 0x7f0c2c14cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52706/0 (socket says 172.21.15.115:52706) 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c3148b640 1 -- 172.21.15.115:0/907467907 learned_addr learned my addr 172.21.15.115:0/907467907 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c2bfff640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c0c9ef0 0x7f0c2c0c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.421 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c30c8a640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c06b6f0 0x7f0c2c06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c3148b640 1 -- 172.21.15.115:0/907467907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c06b6f0 msgr2=0x7f0c2c06fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c3148b640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c06b6f0 0x7f0c2c06fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c3148b640 1 -- 172.21.15.115:0/907467907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c0c9ef0 msgr2=0x7f0c2c0c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c3148b640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c0c9ef0 0x7f0c2c0c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c3148b640 1 -- 172.21.15.115:0/907467907 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18009400 con 0x7f0c2c159dd0 2023-12-01T03:13:14.422 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.419+0000 7f0c30c8a640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c06b6f0 0x7f0c2c06fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:14.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.420+0000 7f0c3148b640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c159dd0 0x7f0c2c14cbd0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7f0c24007e40 tx=0x7f0c2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:14.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.420+0000 7f0c297fa640 1 -- 172.21.15.115:0/907467907 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c24014620 con 0x7f0c2c159dd0 2023-12-01T03:13:14.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.420+0000 7f0c297fa640 1 -- 172.21.15.115:0/907467907 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2400ee90 con 0x7f0c2c159dd0 2023-12-01T03:13:14.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.420+0000 7f0c297fa640 1 -- 172.21.15.115:0/907467907 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c240149e0 con 0x7f0c2c159dd0 2023-12-01T03:13:14.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.420+0000 7f0c32eee640 1 -- 172.21.15.115:0/907467907 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c070870 con 0x7f0c2c159dd0 2023-12-01T03:13:14.423 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.420+0000 7f0c32eee640 1 -- 172.21.15.115:0/907467907 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c14d110 con 0x7f0c2c159dd0 2023-12-01T03:13:14.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.421+0000 7f0c32eee640 1 -- 172.21.15.115:0/907467907 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c04e890 con 0x7f0c2c159dd0 2023-12-01T03:13:14.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.422+0000 7f0c297fa640 1 -- 172.21.15.115:0/907467907 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f0c24014b40 con 0x7f0c2c159dd0 2023-12-01T03:13:14.424 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.423+0000 7f0c297fa640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c1006a550 0x7f0c1006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.423+0000 7f0c297fa640 1 -- 172.21.15.115:0/907467907 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f0c240894f0 con 0x7f0c2c159dd0 2023-12-01T03:13:14.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.423+0000 7f0c30c8a640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c1006a550 0x7f0c1006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.425 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.424+0000 7f0c30c8a640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c1006a550 0x7f0c1006ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c20000f60 tx=0x7f0c20009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:14.429 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.427+0000 7f0c297fa640 1 -- 172.21.15.115:0/907467907 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f0c24054920 con 0x7f0c2c159dd0 2023-12-01T03:13:14.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.612+0000 7f0c32eee640 1 -- 172.21.15.115:0/907467907 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c2c14d2a0 con 0x7f0c1006a550 2023-12-01T03:13:14.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.615+0000 7f0c297fa640 1 -- 172.21.15.115:0/907467907 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48520 (secure 0 0 0) 0x7f0c20001c90 con 0x7f0c1006a550 2023-12-01T03:13:14.618 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:14.618 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:13:14.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 -- 172.21.15.115:0/907467907 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c1006a550 msgr2=0x7f0c1006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c1006a550 0x7f0c1006ca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c20000f60 tx=0x7f0c20009040 comp rx=0 tx=0).stop 2023-12-01T03:13:14.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 -- 172.21.15.115:0/907467907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c159dd0 msgr2=0x7f0c2c14cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c159dd0 0x7f0c2c14cbd0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7f0c24007e40 tx=0x7f0c2400e4b0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 -- 172.21.15.115:0/907467907 shutdown_connections 2023-12-01T03:13:14.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f0c1006a550 0x7f0c1006ca10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.620 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c2c159dd0 0x7f0c2c14cbd0 unknown :-1 s=CLOSED pgs=3122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c2c0c9ef0 0x7f0c2c0c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 --2- 172.21.15.115:0/907467907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c2c06b6f0 0x7f0c2c06fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 -- 172.21.15.115:0/907467907 >> 172.21.15.115:0/907467907 conn(0x7f0c2c0ce880 msgr2=0x7f0c2c15c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:14.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 -- 172.21.15.115:0/907467907 shutdown_connections 2023-12-01T03:13:14.621 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.618+0000 7f0c32eee640 1 -- 172.21.15.115:0/907467907 wait complete. 2023-12-01T03:13:14.640 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-12-01T03:13:13.101035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":444152,"kb_used_data":26808,"kb_used_omap":0,"kb_used_meta":417280,"kb_avail":754530568,"statfs":{"total":773094113280,"available":772639301632,"internally_reserved":0,"allocated":27451392,"data_stored":14131411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":427294720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668411,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-261,"num_write_kb":-2055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.363049"},"pg_stats":[{"pgid":"1.0","version":"258'381","reported_seq":1354,"reported_epoch":259,"state":"active+clean","last_fresh":"2023-12-01T03:13:06.806193+0000","last_change":"2023-12-01T01:36:02.280136+0000","last_active":"2023-12-01T03:13:06.806193+0000","last_peered":"2023-12-01T03:13:06.806193+0000","last_clean":"2023-12-01T03:13:06.806193+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:13:06.806193+0000","last_undegraded":"2023-12-01T03:13:06.806193+0000","last_fullsized":"2023-12-01T03:13:06.806193+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:36:00.878357+0000","last_clean_scrub_stamp":"2023-12-01T01:36:00.878357+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:24.839716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576006,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3280896,"data_stored":1617971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.0620000000000001,"15min":1.0760000000000001},"min":{"1min":0.59499999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.752},{"interface":"front","average":{"1min":1.343,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.95899999999999996}]},{"osd":1,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":0.83999999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.1930000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.3069999999999999,"15min":1.327},"min":{"1min":0.83799999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.6679999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.167},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.95,"5min":1.95,"15min":2.1179999999999999},"last":1.526},{"interface":"front","average":{"1min":1.349,"5min":1.254,"15min":1.264},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.2070000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2729999999999999,"15min":1.302},"min":{"1min":0.78400000000000003,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":1.127},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.3959999999999999,"15min":1.425},"min":{"1min":0.84599999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":1.488}]},{"osd":4,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3140000000000001,"15min":1.329},"min":{"1min":0.92900000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7989999999999999,"5min":1.901,"15min":2.5899999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.7110000000000001,"5min":1.5029999999999999,"15min":1.508},"min":{"1min":1.1599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":1.9930000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.337,"15min":1.327},"min":{"1min":0.79600000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":4.2709999999999999,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.5840000000000001,"15min":1.597},"min":{"1min":1.2430000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.8380000000000001,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":1.8560000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:12:15 2023","interfaces":[{"interface":"back","average":{"1min":1.728,"5min":1.476,"15min":1.454},"min":{"1min":0.89400000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.8579999999999997,"5min":4.8579999999999997,"15min":4.8579999999999997},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4319999999999999,"15min":1.4550000000000001},"min":{"1min":0.81799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":2.762},"last":1.3120000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576007,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3280896,"data_stored":1617971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.036,"15min":1.0980000000000001},"min":{"1min":0.53600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4810000000000001,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.093,"15min":1.1519999999999999},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6439999999999999,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.0609999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.127,"15min":1.202},"min":{"1min":0.57399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7010000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.542,"5min":1.694,"15min":1.835},"last":1.1279999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5369999999999999,"5min":1.742,"15min":1.976},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.272},"min":{"1min":0.71599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.3680000000000001}]},{"osd":3,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.262,"15min":1.306},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7929999999999999,"5min":1.927,"15min":1.927},"last":1.417},{"interface":"front","average":{"1min":1.304,"5min":1.377,"15min":1.415},"min":{"1min":0.68700000000000006,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.339}]},{"osd":4,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2809999999999999,"15min":1.27},"min":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.661,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.4199999999999999,"15min":1.3660000000000001},"min":{"1min":0.64100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1019999999999999,"5min":2.48,"15min":2.48},"last":0.97799999999999998}]},{"osd":5,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3859999999999999,"15min":1.407},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.651,"5min":2.262,"15min":2.262},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.518,"15min":1.492},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0030000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.23}]},{"osd":7,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5069999999999999,"15min":1.526},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.865,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0539999999999998,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.292}]}]},{"osd":5,"up_from":13,"seq":55834576006,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52880,"kb_used_data":3784,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94318960,"statfs":{"total":96636764160,"available":96582615040,"internally_reserved":0,"allocated":3874816,"data_stored":2208339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.089,"15min":1.113},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.61299999999999999},"max":{"1min":1.4339999999999999,"5min":1.776,"15min":1.776},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1299999999999999,"15min":1.151},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.90400000000000003}]},{"osd":1,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.1319999999999999,"15min":1.145},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":2.1349999999999998},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.179,"15min":1.218},"min":{"1min":0.74199999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5329999999999999,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":0.97999999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1970000000000001,"15min":1.2410000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.6499999999999999,"5min":1.901,"15min":1.901},"last":1.073},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.224,"15min":1.244},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.546,"5min":1.899,"15min":1.9219999999999999},"last":3.1579999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2989999999999999,"15min":1.3109999999999999},"min":{"1min":0.92700000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.5740000000000001,"5min":1.889,"15min":1.8979999999999999},"last":1.014},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.294,"15min":1.3460000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5960000000000001,"5min":1.97,"15min":2.1859999999999999},"last":2.0600000000000001}]},{"osd":4,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3,"15min":1.258},"min":{"1min":0.90700000000000003,"5min":0.82899999999999996,"15min":0.78000000000000003},"max":{"1min":1.681,"5min":1.869,"15min":2.6040000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.399,"5min":1.369,"15min":1.3720000000000001},"min":{"1min":1.0349999999999999,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.6659999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.6850000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3899999999999999,"15min":1.3740000000000001},"min":{"1min":0.97399999999999998,"5min":0.93300000000000005,"15min":0.88800000000000001},"max":{"1min":1.6399999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.978},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.47,"15min":1.4259999999999999},"min":{"1min":1.3149999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7529999999999999,"5min":2.032,"15min":2.577},"last":2.1970000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.5489999999999999,"15min":1.5389999999999999},"min":{"1min":1.294,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.9550000000000001,"5min":2.077,"15min":3.1709999999999998},"last":1.752},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.591,"15min":1.5760000000000001},"min":{"1min":1.2669999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.883,"5min":2.101,"15min":2.8740000000000001},"last":2.621}]}]},{"osd":4,"up_from":13,"seq":55834576006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52312,"kb_used_data":3216,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319528,"statfs":{"total":96636764160,"available":96583196672,"internally_reserved":0,"allocated":3293184,"data_stored":1625026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.0549999999999999,"15min":1.1080000000000001},"min":{"1min":0.63200000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.343,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.179},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1850000000000001,"15min":1.173},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5589999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.109}]},{"osd":1,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.159,"15min":1.2250000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.617,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.997,"5min":1.1140000000000001,"15min":1.163},"min":{"1min":0.66000000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4550000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.375}]},{"osd":2,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2869999999999999,"15min":1.2330000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.482,"5min":11.311,"15min":11.311},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.236,"15min":1.3160000000000001},"min":{"1min":0.82899999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.212}]},{"osd":3,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2470000000000001,"15min":1.27},"min":{"1min":0.88800000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.877,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.27},{"interface":"front","average":{"1min":1.258,"5min":1.347,"15min":1.3300000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.667,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1.2989999999999999}]},{"osd":5,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.377,"15min":1.3300000000000001},"min":{"1min":0.751,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.716,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.327},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.349,"15min":1.3160000000000001},"min":{"1min":0.70399999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.0249999999999999,"5min":3.71,"15min":3.71},"last":1.4830000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3899999999999999,"15min":1.3720000000000001},"min":{"1min":0.84299999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8759999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.4890000000000001,"15min":1.47},"min":{"1min":0.86499999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1120000000000001,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.4410000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.486,"15min":1.4750000000000001},"min":{"1min":0.995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0289999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5840000000000001,"15min":1.573},"min":{"1min":0.95699999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.1480000000000001,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.681}]}]},{"osd":3,"up_from":13,"seq":55834576005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56388,"kb_used_data":3196,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315452,"statfs":{"total":96636764160,"available":96579022848,"internally_reserved":0,"allocated":3272704,"data_stored":1611328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.51600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.581,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":1.0589999999999999},"min":{"1min":0.54000000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.534,"5min":1.95,"15min":1.95},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.04,"15min":1.1279999999999999},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.635,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.92400000000000004,"15min":0.97699999999999998},"min":{"1min":0.57199999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.532,"5min":3.863,"15min":3.863},"last":0.627}]},{"osd":2,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1,"15min":1.0620000000000001},"min":{"1min":0.52000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3149999999999999,"5min":3.597,"15min":3.597},"last":0.51300000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.139,"15min":1.2090000000000001},"min":{"1min":0.60499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5609999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2549999999999999,"15min":1.282},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.383,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.004},{"interface":"front","average":{"1min":1.232,"5min":1.2529999999999999,"15min":1.2849999999999999},"min":{"1min":0.85599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9890000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2889999999999999,"15min":1.3180000000000001},"min":{"1min":0.82299999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8979999999999999,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.359,"5min":1.343,"15min":1.353},"min":{"1min":0.81799999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.0830000000000002,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.747}]},{"osd":6,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3580000000000001,"15min":1.4259999999999999},"min":{"1min":0.83499999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.3079999999999998,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":1.125},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.448,"15min":1.488},"min":{"1min":0.997,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4870000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.452,"15min":1.502},"min":{"1min":1.0429999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.0470000000000002,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.462,"15min":1.516},"min":{"1min":0.96199999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.2440000000000002,"5min":3.004,"15min":3.004},"last":1.028}]}]},{"osd":2,"up_from":13,"seq":55834576005,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48216,"kb_used_data":3216,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323624,"statfs":{"total":96636764160,"available":96587390976,"internally_reserved":0,"allocated":3293184,"data_stored":1624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79700000000000004,"15min":0.86099999999999999},"min":{"1min":0.503,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.90800000000000003,"5min":1.411,"15min":1.4730000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.91900000000000004,"15min":1.0149999999999999},"min":{"1min":0.54400000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.198,"5min":1.5640000000000001,"15min":1.7310000000000001},"last":1.3660000000000001}]},{"osd":1,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":1.02},"min":{"1min":0.64600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.2569999999999999,"5min":1.512,"15min":1.698},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.97899999999999998,"15min":1.103},"min":{"1min":0.54000000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4399999999999999,"5min":1.5800000000000001,"15min":1.6599999999999999},"last":1.26}]},{"osd":3,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95299999999999996,"15min":1.014},"min":{"1min":0.45500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.502,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.0609999999999999,"15min":1.171},"min":{"1min":0.63900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.367,"5min":1.615,"15min":1.79},"last":1.1619999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.22,"15min":1.302},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.609,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.1759999999999999,"15min":1.2609999999999999},"min":{"1min":0.64300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.508,"5min":1.714,"15min":1.835},"last":1.3080000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.282,"15min":1.3660000000000001},"min":{"1min":0.97599999999999998,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7509999999999999,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.296,"15min":1.387},"min":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5569999999999999,"5min":1.8740000000000001,"15min":2.016},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3149999999999999,"15min":1.3640000000000001},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7190000000000001,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3779999999999999,"15min":1.4690000000000001},"min":{"1min":1.024,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.8740000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.5880000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3819999999999999,"15min":1.4590000000000001},"min":{"1min":0.89400000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.9099999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.3500000000000001,"15min":1.4239999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9610000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.544}]}]},{"osd":1,"up_from":13,"seq":55834576004,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52880,"kb_used_data":3784,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94318960,"statfs":{"total":96636764160,"available":96582615040,"internally_reserved":0,"allocated":3874816,"data_stored":2208339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.81100000000000005,"15min":0.877},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.093,"5min":1.5569999999999999,"15min":1.6850000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88800000000000001,"15min":1.006},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.353,"5min":1.512,"15min":1.681},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82099999999999995,"15min":0.91300000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.335,"5min":1.6599999999999999,"15min":1.7030000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.98199999999999998,"15min":1.101},"min":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3819999999999999,"5min":1.581,"15min":1.7430000000000001},"last":0.80500000000000005}]},{"osd":3,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.069,"15min":1.244},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.3799999999999999,"5min":1.593,"15min":1.788},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.998,"15min":1.093},"min":{"1min":0.48699999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3600000000000001,"5min":1.714,"15min":1.714},"last":0.85199999999999998}]},{"osd":4,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1519999999999999,"15min":1.202},"min":{"1min":0.61099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.302,"15min":1.381},"min":{"1min":0.76300000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.2109999999999999},"last":0.998}]},{"osd":5,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.29,"15min":1.3200000000000001},"min":{"1min":0.81499999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":1.083},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.2749999999999999,"15min":1.327},"min":{"1min":0.66100000000000003,"5min":0.496,"15min":0.496},"max":{"1min":3.75,"5min":3.75,"15min":3.75},"last":1.278}]},{"osd":6,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.327,"15min":1.4079999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5580000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.452},{"interface":"front","average":{"1min":1.28,"5min":1.345,"15min":1.393},"min":{"1min":0.70499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8700000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.5389999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.4219999999999999,"15min":1.502},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.5350000000000001,"5min":3.5350000000000001,"15min":3.5350000000000001},"last":1.123},{"interface":"front","average":{"1min":1.268,"5min":1.413,"15min":1.482},"min":{"1min":0.80400000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.617,"5min":2.1389999999999998,"15min":2.2069999999999999},"last":1.2150000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576006,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60492,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311348,"statfs":{"total":96636764160,"available":96574820352,"internally_reserved":0,"allocated":3280896,"data_stored":1617971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.54500000000000004,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.175,"5min":3.472,"15min":3.472},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91400000000000003,"15min":1.026},"min":{"1min":0.58899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1950000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.002,"15min":1.056},"min":{"1min":0.5,"5min":0.5,"15min":0.48699999999999999},"max":{"1min":1.2849999999999999,"5min":2.0179999999999998,"15min":2.1429999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98399999999999999,"15min":1.097},"min":{"1min":0.61299999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4770000000000001,"5min":1.726,"15min":1.792},"last":0.84699999999999998}]},{"osd":3,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93899999999999995,"15min":1.0369999999999999},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.262,"5min":1.536,"15min":1.8839999999999999},"last":0.437},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.135,"15min":1.21},"min":{"1min":0.629,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5369999999999999,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.875}]},{"osd":4,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.2769999999999999,"15min":1.3420000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.206,"5min":1.2170000000000001,"15min":1.284},"min":{"1min":0.67600000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.752,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":1.1220000000000001}]},{"osd":5,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3300000000000001,"15min":1.393},"min":{"1min":0.81100000000000005,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.986,"5min":1.986,"15min":2.7719999999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.3400000000000001,"15min":1.413},"min":{"1min":0.84499999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7969999999999999,"5min":2.484,"15min":2.5880000000000001},"last":1.266}]},{"osd":6,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.298,"15min":1.363},"min":{"1min":0.94399999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.835,"15min":2.2029999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4359999999999999,"15min":1.4770000000000001},"min":{"1min":0.84799999999999998,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.8759999999999999},"last":1.226}]},{"osd":7,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4330000000000001,"15min":1.502},"min":{"1min":1.0960000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2069999999999999,"5min":2.411,"15min":2.7040000000000002},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":0.92300000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":0.91200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:13:14.642 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T03:13:14.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.847+0000 7faf3e7fb640 1 Processor -- start 2023-12-01T03:13:14.849 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.847+0000 7faf3e7fb640 1 -- start start 2023-12-01T03:13:14.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.849+0000 7faf3e7fb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faf3815ada0 0x7faf380ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.850 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.849+0000 7faf3e7fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 0x7faf380cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.849+0000 7faf3e7fb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf380cfa20 con 0x7faf380cf5b0 2023-12-01T03:13:14.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.849+0000 7faf3e7fb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf380cc9b0 con 0x7faf3815ada0 2023-12-01T03:13:14.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.849+0000 7faf3e7fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380cc110 con 0x7faf380d15b0 2023-12-01T03:13:14.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.849+0000 7faf377fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 0x7faf380cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.850+0000 7faf37fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faf3815ada0 0x7faf380ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.851 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.850+0000 7faf3cd98640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7faf380cf5b0 0x7faf380cd0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:44360/0 (socket says 172.21.15.115:44360) 2023-12-01T03:13:14.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.850+0000 7faf3cd98640 1 -- 172.21.15.115:0/855892351 learned_addr learned my addr 172.21.15.115:0/855892351 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:14.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.851+0000 7faf377fe640 1 -- 172.21.15.115:0/855892351 >> v2:172.21.15.142:3300/0 conn(0x7faf3815ada0 msgr2=0x7faf380ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:14.852 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.851+0000 7faf377fe640 1 --2- 172.21.15.115:0/855892351 >> v2:172.21.15.142:3300/0 conn(0x7faf3815ada0 0x7faf380ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.851+0000 7faf377fe640 1 -- 172.21.15.115:0/855892351 >> v1:172.21.15.142:6789/0 conn(0x7faf380cf5b0 legacy=0x7faf380cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.851+0000 7faf377fe640 1 -- 172.21.15.115:0/855892351 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380cc3f0 con 0x7faf380d15b0 2023-12-01T03:13:14.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.851+0000 7faf36ffd640 1 -- 172.21.15.115:0/855892351 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769384079 0 0) 0x7faf380cfa20 con 0x7faf380cf5b0 2023-12-01T03:13:14.853 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.851+0000 7faf37fff640 1 --2- 172.21.15.115:0/855892351 >> v2:172.21.15.142:3300/0 conn(0x7faf3815ada0 0x7faf380ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T03:13:14.854 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.852+0000 7faf377fe640 1 --2- 172.21.15.115:0/855892351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 0x7faf380cf070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf200097a0 tx=0x7faf2002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99fbc531d8b1fff2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:14.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.853+0000 7faf36ffd640 1 -- 172.21.15.115:0/855892351 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf200316f0 con 0x7faf380d15b0 2023-12-01T03:13:14.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.853+0000 7faf36ffd640 1 -- 172.21.15.115:0/855892351 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf20031850 con 0x7faf380d15b0 2023-12-01T03:13:14.855 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.853+0000 7faf36ffd640 1 -- 172.21.15.115:0/855892351 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20031be0 con 0x7faf380d15b0 2023-12-01T03:13:14.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.854+0000 7faf3e7fb640 1 -- 172.21.15.115:0/855892351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 msgr2=0x7faf380cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.854+0000 7faf3e7fb640 1 --2- 172.21.15.115:0/855892351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 0x7faf380cf070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf200097a0 tx=0x7faf2002b790 comp rx=0 tx=0).stop 2023-12-01T03:13:14.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.854+0000 7faf3e7fb640 1 -- 172.21.15.115:0/855892351 shutdown_connections 2023-12-01T03:13:14.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.854+0000 7faf3e7fb640 1 --2- 172.21.15.115:0/855892351 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 0x7faf380cf070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.854+0000 7faf3e7fb640 1 --2- 172.21.15.115:0/855892351 >> v2:172.21.15.142:3300/0 conn(0x7faf3815ada0 0x7faf380ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.854+0000 7faf3e7fb640 1 -- 172.21.15.115:0/855892351 >> 172.21.15.115:0/855892351 conn(0x7faf380d0a90 msgr2=0x7faf380d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:14.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.854+0000 7faf3e7fb640 1 -- 172.21.15.115:0/855892351 shutdown_connections 2023-12-01T03:13:14.856 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.854+0000 7faf3e7fb640 1 -- 172.21.15.115:0/855892351 wait complete. 2023-12-01T03:13:14.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.856+0000 7faf3e7fb640 1 Processor -- start 2023-12-01T03:13:14.858 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.857+0000 7faf3e7fb640 1 -- start start 2023-12-01T03:13:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.858+0000 7faf3e7fb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380cf5b0 0x7faf381583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.858+0000 7faf3e7fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 0x7faf38069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.858+0000 7faf3e7fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf3815ada0 0x7faf380c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.858+0000 7faf3e7fb640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38158900 con 0x7faf380cf5b0 2023-12-01T03:13:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.858+0000 7faf3e7fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3806ad00 con 0x7faf3815ada0 2023-12-01T03:13:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.858+0000 7faf3e7fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380b3cd0 con 0x7faf380d15b0 2023-12-01T03:13:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.858+0000 7faf37fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380cf5b0 0x7faf381583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.860 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.858+0000 7faf37fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380cf5b0 0x7faf381583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52716/0 (socket says 172.21.15.115:52716) 2023-12-01T03:13:14.861 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.858+0000 7faf37fff640 1 -- 172.21.15.115:0/756735055 learned_addr learned my addr 172.21.15.115:0/756735055 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:14.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.859+0000 7faf377fe640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 0x7faf38069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.859+0000 7faf37fff640 1 -- 172.21.15.115:0/756735055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 msgr2=0x7faf38069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.859+0000 7faf37fff640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 0x7faf38069cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.859+0000 7faf37fff640 1 -- 172.21.15.115:0/756735055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf3815ada0 msgr2=0x7faf380c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:14.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.859+0000 7faf37fff640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf3815ada0 0x7faf380c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:14.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.859+0000 7faf37fff640 1 -- 172.21.15.115:0/756735055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf28009a70 con 0x7faf380cf5b0 2023-12-01T03:13:14.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.859+0000 7faf37fff640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380cf5b0 0x7faf381583c0 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7faf2800efc0 tx=0x7faf2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:14.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.860+0000 7faf34ff9640 1 -- 172.21.15.115:0/756735055 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28014680 con 0x7faf380cf5b0 2023-12-01T03:13:14.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.860+0000 7faf34ff9640 1 -- 172.21.15.115:0/756735055 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2800f040 con 0x7faf380cf5b0 2023-12-01T03:13:14.862 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.860+0000 7faf34ff9640 1 -- 172.21.15.115:0/756735055 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28014a00 con 0x7faf380cf5b0 2023-12-01T03:13:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.860+0000 7faf3e7fb640 1 -- 172.21.15.115:0/756735055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf20009400 con 0x7faf380cf5b0 2023-12-01T03:13:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.860+0000 7faf3e7fb640 1 -- 172.21.15.115:0/756735055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3806a200 con 0x7faf380cf5b0 2023-12-01T03:13:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.861+0000 7faf3e7fb640 1 -- 172.21.15.115:0/756735055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3804e890 con 0x7faf380cf5b0 2023-12-01T03:13:14.863 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.862+0000 7faf34ff9640 1 -- 172.21.15.115:0/756735055 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7faf28014b60 con 0x7faf380cf5b0 2023-12-01T03:13:14.864 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.863+0000 7faf34ff9640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1806a550 0x7faf1806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:14.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.863+0000 7faf34ff9640 1 -- 172.21.15.115:0/756735055 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7faf28089b20 con 0x7faf380cf5b0 2023-12-01T03:13:14.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.863+0000 7faf377fe640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1806a550 0x7faf1806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:14.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.864+0000 7faf377fe640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1806a550 0x7faf1806ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf20036000 tx=0x7faf20008030 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:14.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:14.867+0000 7faf34ff9640 1 -- 172.21.15.115:0/756735055 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7faf28054f70 con 0x7faf380cf5b0 2023-12-01T03:13:15.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.059+0000 7faf3e7fb640 1 -- 172.21.15.115:0/756735055 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf380d0fe0 con 0x7faf380cf5b0 2023-12-01T03:13:15.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.061+0000 7faf34ff9640 1 -- 172.21.15.115:0/756735055 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v261) v1 ==== 74+0+13714 (secure 0 0 0) 0x7faf28058590 con 0x7faf380cf5b0 2023-12-01T03:13:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:15.063 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":261,"fsid":"5ba3726b-7abf-4a38-892b-ec2cd5ba09fc","created":"2023-12-01T01:35:45.132784+0000","modified":"2023-12-01T03:13:09.849753+0000","last_up_change":"2023-12-01T01:35:58.663239+0000","last_in_change":"2023-12-01T01:35:49.376922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:36:00.229588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aaf68732-34d8-41ee-9104-1789636cd3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":88579},{"type":"v1","addr":"172.21.15.115:6817","nonce":88579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":88579},{"type":"v1","addr":"172.21.15.115:6819","nonce":88579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6823","nonce":88579},{"type":"v1","addr":"172.21.15.115:6825","nonce":88579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":88579},{"type":"v1","addr":"172.21.15.115:6821","nonce":88579}]},"public_addr":"172.21.15.115:6817/88579","cluster_addr":"172.21.15.115:6819/88579","heartbeat_back_addr":"172.21.15.115:6825/88579","heartbeat_front_addr":"172.21.15.115:6821/88579","state":["exists","up"]},{"osd":1,"uuid":"ae0caaf6-7427-4951-85a2-68c07f744762","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":88580},{"type":"v1","addr":"172.21.15.115:6809","nonce":88580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":88580},{"type":"v1","addr":"172.21.15.115:6811","nonce":88580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":88580},{"type":"v1","addr":"172.21.15.115:6816","nonce":88580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":88580},{"type":"v1","addr":"172.21.15.115:6813","nonce":88580}]},"public_addr":"172.21.15.115:6809/88580","cluster_addr":"172.21.15.115:6811/88580","heartbeat_back_addr":"172.21.15.115:6816/88580","heartbeat_front_addr":"172.21.15.115:6813/88580","state":["exists","up"]},{"osd":2,"uuid":"5ba3cc79-ca5f-4da6-adcb-8a99c60652ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":88578},{"type":"v1","addr":"172.21.15.115:6801","nonce":88578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":88578},{"type":"v1","addr":"172.21.15.115:6803","nonce":88578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":88578},{"type":"v1","addr":"172.21.15.115:6807","nonce":88578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":88578},{"type":"v1","addr":"172.21.15.115:6805","nonce":88578}]},"public_addr":"172.21.15.115:6801/88578","cluster_addr":"172.21.15.115:6803/88578","heartbeat_back_addr":"172.21.15.115:6807/88578","heartbeat_front_addr":"172.21.15.115:6805/88578","state":["exists","up"]},{"osd":3,"uuid":"abcdd200-4c30-4703-815f-8f8adbe1dd08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":88584},{"type":"v1","addr":"172.21.15.115:6827","nonce":88584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":88584},{"type":"v1","addr":"172.21.15.115:6829","nonce":88584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":88584},{"type":"v1","addr":"172.21.15.115:6833","nonce":88584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":88584},{"type":"v1","addr":"172.21.15.115:6831","nonce":88584}]},"public_addr":"172.21.15.115:6827/88584","cluster_addr":"172.21.15.115:6829/88584","heartbeat_back_addr":"172.21.15.115:6833/88584","heartbeat_front_addr":"172.21.15.115:6831/88584","state":["exists","up"]},{"osd":4,"uuid":"ce369f10-36d4-4573-bd78-b67c31ceb9bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":88936},{"type":"v1","addr":"172.21.15.142:6801","nonce":88936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":88936},{"type":"v1","addr":"172.21.15.142:6803","nonce":88936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":88936},{"type":"v1","addr":"172.21.15.142:6807","nonce":88936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":88936},{"type":"v1","addr":"172.21.15.142:6805","nonce":88936}]},"public_addr":"172.21.15.142:6801/88936","cluster_addr":"172.21.15.142:6803/88936","heartbeat_back_addr":"172.21.15.142:6807/88936","heartbeat_front_addr":"172.21.15.142:6805/88936","state":["exists","up"]},{"osd":5,"uuid":"ec117800-c03d-4c2a-bdb2-b5c96726712a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":88938},{"type":"v1","addr":"172.21.15.142:6818","nonce":88938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":88938},{"type":"v1","addr":"172.21.15.142:6822","nonce":88938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6827","nonce":88938},{"type":"v1","addr":"172.21.15.142:6829","nonce":88938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6823","nonce":88938},{"type":"v1","addr":"172.21.15.142:6825","nonce":88938}]},"public_addr":"172.21.15.142:6818/88938","cluster_addr":"172.21.15.142:6822/88938","heartbeat_back_addr":"172.21.15.142:6829/88938","heartbeat_front_addr":"172.21.15.142:6825/88938","state":["exists","up"]},{"osd":6,"uuid":"c9536043-c132-4a02-a76a-2cbef11be908","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":88937},{"type":"v1","addr":"172.21.15.142:6809","nonce":88937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":88937},{"type":"v1","addr":"172.21.15.142:6811","nonce":88937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":88937},{"type":"v1","addr":"172.21.15.142:6815","nonce":88937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":88937},{"type":"v1","addr":"172.21.15.142:6813","nonce":88937}]},"public_addr":"172.21.15.142:6809/88937","cluster_addr":"172.21.15.142:6811/88937","heartbeat_back_addr":"172.21.15.142:6815/88937","heartbeat_front_addr":"172.21.15.142:6813/88937","state":["exists","up"]},{"osd":7,"uuid":"f333b559-012d-4988-a29f-21609f533af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":254,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6817","nonce":88947},{"type":"v1","addr":"172.21.15.142:6819","nonce":88947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":88947},{"type":"v1","addr":"172.21.15.142:6824","nonce":88947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":88947},{"type":"v1","addr":"172.21.15.142:6831","nonce":88947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":88947},{"type":"v1","addr":"172.21.15.142:6828","nonce":88947}]},"public_addr":"172.21.15.142:6819/88947","cluster_addr":"172.21.15.142:6824/88947","heartbeat_back_addr":"172.21.15.142:6831/88947","heartbeat_front_addr":"172.21.15.142:6828/88947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.932909+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.693940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.665985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.757634+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.737004+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.670938+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:35:54.747238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6832/1356734431":"2023-12-02T03:13:06.734516+0000","172.21.15.115:0/797535177":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/1950965948":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/1670255025":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/954404245":"2023-12-02T03:13:03.079506+0000","172.21.15.142:6834/1356734431":"2023-12-02T03:13:06.734516+0000","172.21.15.115:0/1517453095":"2023-12-02T03:13:03.079506+0000","172.21.15.115:6824/138932":"2023-12-02T03:12:54.714747+0000","172.21.15.115:6822/138932":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2180551612":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/2457056129":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2634160496":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2258893627":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2137126608":"2023-12-02T03:13:03.079506+0000","172.21.15.115:0/2547954996":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2121389099":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/3223789397":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/534295842":"2023-12-02T03:12:54.714747+0000","172.21.15.115:0/2854101523":"2023-12-02T03:12:54.714747+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 -- 172.21.15.115:0/756735055 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1806a550 msgr2=0x7faf1806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1806a550 0x7faf1806ca10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf20036000 tx=0x7faf20008030 comp rx=0 tx=0).stop 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 -- 172.21.15.115:0/756735055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380cf5b0 msgr2=0x7faf381583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380cf5b0 0x7faf381583c0 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7faf2800efc0 tx=0x7faf2800c490 comp rx=0 tx=0).stop 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 -- 172.21.15.115:0/756735055 shutdown_connections 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7faf1806a550 0x7faf1806ca10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf380d15b0 0x7faf38069cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf3815ada0 0x7faf380c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 --2- 172.21.15.115:0/756735055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf380cf5b0 0x7faf381583c0 unknown :-1 s=CLOSED pgs=3123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 -- 172.21.15.115:0/756735055 >> 172.21.15.115:0/756735055 conn(0x7faf380d0a90 msgr2=0x7faf38159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:15.066 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 -- 172.21.15.115:0/756735055 shutdown_connections 2023-12-01T03:13:15.069 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:15.064+0000 7faf3e7fb640 1 -- 172.21.15.115:0/756735055 wait complete. 2023-12-01T03:13:16.085 INFO:tasks.ceph:Scrubbing osd.0 2023-12-01T03:13:16.086 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-12-01T03:13:16.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.318+0000 7fc5bb72e640 1 Processor -- start 2023-12-01T03:13:16.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.318+0000 7fc5bb72e640 1 -- start start 2023-12-01T03:13:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.319+0000 7fc5bb72e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc5b40cb8c0 0x7fc5b406f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.319+0000 7fc5bb72e640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b40c9440 con 0x7fc5b4159bf0 2023-12-01T03:13:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.319+0000 7fc5bb72e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b40c9720 con 0x7fc5b40ca7c0 2023-12-01T03:13:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.319+0000 7fc5bb72e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b4159fd0 con 0x7fc5b40cb8c0 2023-12-01T03:13:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.320+0000 7fc5b94ca640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fc5b4159bf0 0x7fc5b415e030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58866/0 (socket says 172.21.15.115:58866) 2023-12-01T03:13:16.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.320+0000 7fc5b94ca640 1 -- 172.21.15.115:0/3112735344 learned_addr learned my addr 172.21.15.115:0/3112735344 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:16.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.320+0000 7fc5b9ccb640 1 --2- 172.21.15.115:0/3112735344 >> v2:172.21.15.142:3300/0 conn(0x7fc5b40cb8c0 0x7fc5b406f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.320+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310707074 0 0) 0x7fc5b40c9440 con 0x7fc5b4159bf0 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.320+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a0003520 con 0x7fc5b4159bf0 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.320+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2575425583 0 0) 0x7fc5a0003520 con 0x7fc5b4159bf0 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.320+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 >> v2:172.21.15.142:3300/0 conn(0x7fc5b40cb8c0 msgr2=0x7fc5b406f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.320+0000 7fc5a7fff640 1 --2- 172.21.15.115:0/3112735344 >> v2:172.21.15.142:3300/0 conn(0x7fc5b40cb8c0 0x7fc5b406f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.320+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 >> v1:172.21.15.142:6789/0 conn(0x7fc5b40ca7c0 legacy=0x7fc5b40cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b40c9a00 con 0x7fc5b4159bf0 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fc5a8001670 con 0x7fc5b4159bf0 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 >> v1:172.21.15.115:6789/0 conn(0x7fc5b4159bf0 legacy=0x7fc5b415e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5a7fff640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a0003970 0x7fc5a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5a7fff640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5a00047c0 0x7fc5a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5a7fff640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a0005180 0x7fc5a000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a000e200 con 0x7fc5a00047c0 2023-12-01T03:13:16.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a000e370 con 0x7fc5a0003970 2023-12-01T03:13:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a000e530 con 0x7fc5a0005180 2023-12-01T03:13:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5b8cc9640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5a00047c0 0x7fc5a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5b94ca640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a0003970 0x7fc5a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5b9ccb640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a0005180 0x7fc5a000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5b8cc9640 1 -- 172.21.15.115:0/3112735344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a0005180 msgr2=0x7fc5a000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5b8cc9640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5a0005180 0x7fc5a000db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5b8cc9640 1 -- 172.21.15.115:0/3112735344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a0003970 msgr2=0x7fc5a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5b8cc9640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a0003970 0x7fc5a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5b8cc9640 1 -- 172.21.15.115:0/3112735344 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0003520 con 0x7fc5a00047c0 2023-12-01T03:13:16.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.321+0000 7fc5b9ccb640 1 -- 172.21.15.115:0/3112735344 reap_dead start 2023-12-01T03:13:16.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.322+0000 7fc5b8cc9640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5a00047c0 0x7fc5a0004c40 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0001750 tx=0x7fc5a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d267876bcc3cd188 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:16.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.323+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00052d0 con 0x7fc5a00047c0 2023-12-01T03:13:16.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.323+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b002e050 con 0x7fc5a00047c0 2023-12-01T03:13:16.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.323+0000 7fc5a7fff640 1 -- 172.21.15.115:0/3112735344 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0003830 con 0x7fc5a00047c0 2023-12-01T03:13:16.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.323+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3112735344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5a00047c0 msgr2=0x7fc5a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.326 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.323+0000 7fc5bb72e640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5a00047c0 0x7fc5a0004c40 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0001750 tx=0x7fc5a00057f0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.323+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3112735344 shutdown_connections 2023-12-01T03:13:16.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.323+0000 7fc5bb72e640 1 --2- 172.21.15.115:0/3112735344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5a00047c0 0x7fc5a0004c40 unknown :-1 s=CLOSED pgs=3125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.323+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3112735344 >> 172.21.15.115:0/3112735344 conn(0x7fc5b40662c0 msgr2=0x7fc5b40b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:16.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.323+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3112735344 shutdown_connections 2023-12-01T03:13:16.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.323+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3112735344 wait complete. 2023-12-01T03:13:16.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.325+0000 7fc5bb72e640 1 Processor -- start 2023-12-01T03:13:16.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.325+0000 7fc5bb72e640 1 -- start start 2023-12-01T03:13:16.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.326+0000 7fc5bb72e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a00047c0 0x7fc5b40c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.326+0000 7fc5bb72e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5b40c8110 0x7fc5b4154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.326+0000 7fc5bb72e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b40ac8a0 0x7fc5b40acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.326+0000 7fc5bb72e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40ad290 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.326+0000 7fc5bb72e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40c6f70 con 0x7fc5a00047c0 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.326+0000 7fc5bb72e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40b1620 con 0x7fc5b40c8110 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b9ccb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b40ac8a0 0x7fc5b40acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b9ccb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b40ac8a0 0x7fc5b40acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52726/0 (socket says 172.21.15.115:52726) 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b9ccb640 1 -- 172.21.15.115:0/3052223746 learned_addr learned my addr 172.21.15.115:0/3052223746 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b94ca640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a00047c0 0x7fc5b40c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b8cc9640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5b40c8110 0x7fc5b4154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b9ccb640 1 -- 172.21.15.115:0/3052223746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5b40c8110 msgr2=0x7fc5b4154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b9ccb640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5b40c8110 0x7fc5b4154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b9ccb640 1 -- 172.21.15.115:0/3052223746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a00047c0 msgr2=0x7fc5b40c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b9ccb640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5a00047c0 0x7fc5b40c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b9ccb640 1 -- 172.21.15.115:0/3052223746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b40c7780 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5b9ccb640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b40ac8a0 0x7fc5b40acd50 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7fc5b406b4d0 tx=0x7fc5ac0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.327+0000 7fc5a5ffb640 1 -- 172.21.15.115:0/3052223746 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00e070 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.328+0000 7fc5a5ffb640 1 -- 172.21.15.115:0/3052223746 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac011e40 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.328+0000 7fc5a5ffb640 1 -- 172.21.15.115:0/3052223746 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac007ee0 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.328+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b40c67c0 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.328+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b40c71c0 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.329+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc5b40c9fc0 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.330+0000 7fc5a5ffb640 1 -- 172.21.15.115:0/3052223746 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fc5ac003990 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.331+0000 7fc5a5ffb640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc59806a550 0x7fc59806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.331+0000 7fc5a5ffb640 1 -- 172.21.15.115:0/3052223746 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7fc5ac0893c0 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.331+0000 7fc5b94ca640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc59806a550 0x7fc59806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.331+0000 7fc5a5ffb640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] conn(0x7fc598070a20 0x7fc598072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.331+0000 7fc5a5ffb640 1 -- 172.21.15.115:0/3052223746 --> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5980734f0 con 0x7fc598070a20 2023-12-01T03:13:16.334 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.331+0000 7fc5a5ffb640 1 -- 172.21.15.115:0/3052223746 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5ac08b9a0 con 0x7fc5b40ac8a0 2023-12-01T03:13:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.331+0000 7fc5b8cc9640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] conn(0x7fc598070a20 0x7fc598072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.332+0000 7fc5b94ca640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc59806a550 0x7fc59806ca10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8001eb0 tx=0x7fc5a8009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.332+0000 7fc5b8cc9640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] conn(0x7fc598070a20 0x7fc598072e40 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:16.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.333+0000 7fc5a5ffb640 1 -- 172.21.15.115:0/3052223746 <== osd.0 v2:172.21.15.115:6815/88579 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc5980734f0 con 0x7fc598070a20 2023-12-01T03:13:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.346+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 --> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc5b40c9ce0 con 0x7fc598070a20 2023-12-01T03:13:16.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.347+0000 7fc5a5ffb640 1 -- 172.21.15.115:0/3052223746 <== osd.0 v2:172.21.15.115:6815/88579 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc5b0005480 con 0x7fc598070a20 2023-12-01T03:13:16.349 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:13:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-01T03:13:16.349 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.347+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] conn(0x7fc598070a20 msgr2=0x7fc598072e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.347+0000 7fc5bb72e640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:6815/88579,v1:172.21.15.115:6817/88579] conn(0x7fc598070a20 0x7fc598072e40 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.348+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc59806a550 msgr2=0x7fc59806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.348+0000 7fc5bb72e640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fc59806a550 0x7fc59806ca10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8001eb0 tx=0x7fc5a8009040 comp rx=0 tx=0).stop 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.348+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b40ac8a0 msgr2=0x7fc5b40acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.348+0000 7fc5bb72e640 1 --2- 172.21.15.115:0/3052223746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5b40ac8a0 0x7fc5b40acd50 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7fc5b406b4d0 tx=0x7fc5ac0115a0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.348+0000 7fc5b9ccb640 1 -- 172.21.15.115:0/3052223746 reap_dead start 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.348+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 shutdown_connections 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.348+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 >> 172.21.15.115:0/3052223746 conn(0x7fc5b40662c0 msgr2=0x7fc5b4070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.348+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 shutdown_connections 2023-12-01T03:13:16.350 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.348+0000 7fc5bb72e640 1 -- 172.21.15.115:0/3052223746 wait complete. 2023-12-01T03:13:16.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-12-01T03:13:16.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.577+0000 7fae6d969640 1 Processor -- start 2023-12-01T03:13:16.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.578+0000 7fae6d969640 1 -- start start 2023-12-01T03:13:16.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.580+0000 7fae6d969640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae680c8e60 0x7fae680c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.580+0000 7fae6d969640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6815dbb0 con 0x7fae68156c90 2023-12-01T03:13:16.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.580+0000 7fae6d969640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae68157070 con 0x7fae680afdc0 2023-12-01T03:13:16.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.580+0000 7fae6d969640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae680c9810 con 0x7fae680c8e60 2023-12-01T03:13:16.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.580+0000 7fae66ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fae68156c90 0x7fae681596d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58870/0 (socket says 172.21.15.115:58870) 2023-12-01T03:13:16.582 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.580+0000 7fae66ffd640 1 -- 172.21.15.115:0/3615796265 learned_addr learned my addr 172.21.15.115:0/3615796265 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:16.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.580+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720313912 0 0) 0x7fae6815dbb0 con 0x7fae68156c90 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.580+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae54003520 con 0x7fae68156c90 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.580+0000 7fae677fe640 1 --2- 172.21.15.115:0/3615796265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae680c8e60 0x7fae680c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 20876939 0 0) 0x7fae54003520 con 0x7fae68156c90 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae680c8e60 msgr2=0x7fae680c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 --2- 172.21.15.115:0/3615796265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae680c8e60 0x7fae680c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 >> v1:172.21.15.142:6789/0 conn(0x7fae680afdc0 legacy=0x7fae6815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6815ccc0 con 0x7fae68156c90 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7fae58001690 con 0x7fae68156c90 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 >> v1:172.21.15.115:6789/0 conn(0x7fae68156c90 legacy=0x7fae681596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 --2- 172.21.15.115:0/3615796265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae540039a0 0x7fae54003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 --2- 172.21.15.115:0/3615796265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae540047c0 0x7fae54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 --2- 172.21.15.115:0/3615796265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae680c8e60 0x7fae540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae680c9810 con 0x7fae540047c0 2023-12-01T03:13:16.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5400bab0 con 0x7fae540039a0 2023-12-01T03:13:16.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5400bc70 con 0x7fae680c8e60 2023-12-01T03:13:16.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.581+0000 7fae5e7fc640 1 --2- 172.21.15.115:0/3615796265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae540047c0 0x7fae54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.582+0000 7fae5e7fc640 1 -- 172.21.15.115:0/3615796265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae680c8e60 msgr2=0x7fae540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:16.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.582+0000 7fae5e7fc640 1 --2- 172.21.15.115:0/3615796265 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae680c8e60 0x7fae540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.582+0000 7fae5e7fc640 1 -- 172.21.15.115:0/3615796265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae540039a0 msgr2=0x7fae54003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:16.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.582+0000 7fae5e7fc640 1 --2- 172.21.15.115:0/3615796265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae540039a0 0x7fae54003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.582+0000 7fae5e7fc640 1 -- 172.21.15.115:0/3615796265 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae54003520 con 0x7fae540047c0 2023-12-01T03:13:16.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.583+0000 7fae5e7fc640 1 --2- 172.21.15.115:0/3615796265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae540047c0 0x7fae54004c40 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7fae50001750 tx=0x7fae540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8405368ed46f03a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:16.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.583+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae500052d0 con 0x7fae540047c0 2023-12-01T03:13:16.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.583+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5002e050 con 0x7fae540047c0 2023-12-01T03:13:16.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.583+0000 7fae667fc640 1 -- 172.21.15.115:0/3615796265 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae500035f0 con 0x7fae540047c0 2023-12-01T03:13:16.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.584+0000 7fae6d969640 1 -- 172.21.15.115:0/3615796265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae540047c0 msgr2=0x7fae54004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.584+0000 7fae6d969640 1 --2- 172.21.15.115:0/3615796265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae540047c0 0x7fae54004c40 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 crypto rx=0x7fae50001750 tx=0x7fae540057f0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.584+0000 7fae677fe640 1 -- 172.21.15.115:0/3615796265 reap_dead start 2023-12-01T03:13:16.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.584+0000 7fae6d969640 1 -- 172.21.15.115:0/3615796265 shutdown_connections 2023-12-01T03:13:16.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.584+0000 7fae6d969640 1 -- 172.21.15.115:0/3615796265 >> 172.21.15.115:0/3615796265 conn(0x7fae68065b40 msgr2=0x7fae68065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:16.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.584+0000 7fae6d969640 1 -- 172.21.15.115:0/3615796265 shutdown_connections 2023-12-01T03:13:16.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.584+0000 7fae6d969640 1 -- 172.21.15.115:0/3615796265 wait complete. 2023-12-01T03:13:16.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.586+0000 7fae6d969640 1 Processor -- start 2023-12-01T03:13:16.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.586+0000 7fae6d969640 1 -- start start 2023-12-01T03:13:16.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae6d969640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae540047c0 0x7fae68155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae6d969640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae680b4180 0x7fae68158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.589 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae6d969640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae681f6120 0x7fae681f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae6d969640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae681543b0 con 0x7fae680b4180 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae6d969640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae68155410 con 0x7fae540047c0 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae6d969640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae681564b0 con 0x7fae681f6120 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae5e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae680b4180 0x7fae68158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae5e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae680b4180 0x7fae68158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52740/0 (socket says 172.21.15.115:52740) 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae5e7fc640 1 -- 172.21.15.115:0/1015422421 learned_addr learned my addr 172.21.15.115:0/1015422421 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae66ffd640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae540047c0 0x7fae68155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.587+0000 7fae677fe640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae681f6120 0x7fae681f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae5e7fc640 1 -- 172.21.15.115:0/1015422421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae681f6120 msgr2=0x7fae681f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae5e7fc640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae681f6120 0x7fae681f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae5e7fc640 1 -- 172.21.15.115:0/1015422421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae540047c0 msgr2=0x7fae68155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae5e7fc640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae540047c0 0x7fae68155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae5e7fc640 1 -- 172.21.15.115:0/1015422421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae68154650 con 0x7fae680b4180 2023-12-01T03:13:16.590 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae5e7fc640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae680b4180 0x7fae68158530 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7fae50008fd0 tx=0x7fae50001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:16.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae64ff9640 1 -- 172.21.15.115:0/1015422421 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae500076c0 con 0x7fae680b4180 2023-12-01T03:13:16.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae64ff9640 1 -- 172.21.15.115:0/1015422421 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae50007a30 con 0x7fae680b4180 2023-12-01T03:13:16.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae6d969640 1 -- 172.21.15.115:0/1015422421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae68154930 con 0x7fae680b4180 2023-12-01T03:13:16.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae64ff9640 1 -- 172.21.15.115:0/1015422421 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50006c00 con 0x7fae680b4180 2023-12-01T03:13:16.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.588+0000 7fae6d969640 1 -- 172.21.15.115:0/1015422421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae68154ef0 con 0x7fae680b4180 2023-12-01T03:13:16.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.589+0000 7fae6d969640 1 -- 172.21.15.115:0/1015422421 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6804e890 con 0x7fae680b4180 2023-12-01T03:13:16.592 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.590+0000 7fae64ff9640 1 -- 172.21.15.115:0/1015422421 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fae500035f0 con 0x7fae680b4180 2023-12-01T03:13:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.591+0000 7fae64ff9640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fae4806a580 0x7fae4806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.591+0000 7fae64ff9640 1 -- 172.21.15.115:0/1015422421 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7fae500aac20 con 0x7fae680b4180 2023-12-01T03:13:16.593 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.592+0000 7fae66ffd640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fae4806a580 0x7fae4806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:16.594 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.592+0000 7fae66ffd640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fae4806a580 0x7fae4806ca40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fae580017c0 tx=0x7fae58001610 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:16.597 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.596+0000 7fae64ff9640 1 -- 172.21.15.115:0/1015422421 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fae50079b80 con 0x7fae680b4180 2023-12-01T03:13:16.787 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.785+0000 7fae6d969640 1 -- 172.21.15.115:0/1015422421 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fae681543b0 con 0x7fae4806a580 2023-12-01T03:13:16.789 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.786+0000 7fae64ff9640 1 -- 172.21.15.115:0/1015422421 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fae681543b0 con 0x7fae4806a580 2023-12-01T03:13:16.789 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 0 to deep-scrub 2023-12-01T03:13:16.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 -- 172.21.15.115:0/1015422421 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fae4806a580 msgr2=0x7fae4806ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fae4806a580 0x7fae4806ca40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fae580017c0 tx=0x7fae58001610 comp rx=0 tx=0).stop 2023-12-01T03:13:16.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 -- 172.21.15.115:0/1015422421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae680b4180 msgr2=0x7fae68158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:16.790 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae680b4180 0x7fae68158530 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7fae50008fd0 tx=0x7fae50001120 comp rx=0 tx=0).stop 2023-12-01T03:13:16.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 -- 172.21.15.115:0/1015422421 shutdown_connections 2023-12-01T03:13:16.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fae4806a580 0x7fae4806ca40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fae681f6120 0x7fae681f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae680b4180 0x7fae68158530 unknown :-1 s=CLOSED pgs=3129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 --2- 172.21.15.115:0/1015422421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae540047c0 0x7fae68155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:16.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 -- 172.21.15.115:0/1015422421 >> 172.21.15.115:0/1015422421 conn(0x7fae68065b40 msgr2=0x7fae680cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:16.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 -- 172.21.15.115:0/1015422421 shutdown_connections 2023-12-01T03:13:16.791 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:16.789+0000 7fae6d969640 1 -- 172.21.15.115:0/1015422421 wait complete. 2023-12-01T03:13:16.808 INFO:tasks.ceph:Scrubbing osd.1 2023-12-01T03:13:16.809 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-12-01T03:13:17.031 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.029+0000 7fbcbd217640 1 Processor -- start 2023-12-01T03:13:17.032 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.030+0000 7fbcbd217640 1 -- start start 2023-12-01T03:13:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.031+0000 7fbcbd217640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbcb815ce40 0x7fbcb80b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.031+0000 7fbcbd217640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb80b4d50 0x7fbcb80d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.031+0000 7fbcbd217640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb80d1460 con 0x7fbcb80d2730 2023-12-01T03:13:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.031+0000 7fbcbd217640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb815d220 con 0x7fbcb815ce40 2023-12-01T03:13:17.033 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.031+0000 7fbcbd217640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80d1690 con 0x7fbcb80b4d50 2023-12-01T03:13:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.032+0000 7fbcb6d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbcb815ce40 0x7fbcb80b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.032+0000 7fbcb759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb80b4d50 0x7fbcb80d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.032+0000 7fbcb6d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbcb815ce40 0x7fbcb80b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.115:59402/0 (socket says 172.21.15.115:59402) 2023-12-01T03:13:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.032+0000 7fbcb6d9d640 1 -- 172.21.15.115:0/2838031572 learned_addr learned my addr 172.21.15.115:0/2838031572 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:17.034 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.032+0000 7fbcb659c640 1 --1- 172.21.15.115:0/2838031572 >> v1:172.21.15.142:6789/0 conn(0x7fbcb80d2730 0x7fbcb80b4640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.115:44382/0 (socket says 172.21.15.115:44382) 2023-12-01T03:13:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.033+0000 7fbcb759e640 1 -- 172.21.15.115:0/2838031572 >> v2:172.21.15.142:3300/0 conn(0x7fbcb815ce40 msgr2=0x7fbcb80b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.033+0000 7fbcb759e640 1 --2- 172.21.15.115:0/2838031572 >> v2:172.21.15.142:3300/0 conn(0x7fbcb815ce40 0x7fbcb80b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.033+0000 7fbcb759e640 1 -- 172.21.15.115:0/2838031572 >> v1:172.21.15.142:6789/0 conn(0x7fbcb80d2730 legacy=0x7fbcb80b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.035 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.033+0000 7fbcb759e640 1 -- 172.21.15.115:0/2838031572 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb80d1970 con 0x7fbcb80b4d50 2023-12-01T03:13:17.036 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.035+0000 7fbcb759e640 1 --2- 172.21.15.115:0/2838031572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb80b4d50 0x7fbcb80d0f20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00097a0 tx=0x7fbcb002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5623e966c6d28bbb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.035+0000 7fbcb5d9b640 1 -- 172.21.15.115:0/2838031572 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0031780 con 0x7fbcb80b4d50 2023-12-01T03:13:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.036+0000 7fbcb5d9b640 1 -- 172.21.15.115:0/2838031572 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb00318e0 con 0x7fbcb80b4d50 2023-12-01T03:13:17.037 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.036+0000 7fbcb5d9b640 1 -- 172.21.15.115:0/2838031572 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0031c70 con 0x7fbcb80b4d50 2023-12-01T03:13:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.036+0000 7fbcbd217640 1 -- 172.21.15.115:0/2838031572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb80b4d50 msgr2=0x7fbcb80d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.036+0000 7fbcbd217640 1 --2- 172.21.15.115:0/2838031572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb80b4d50 0x7fbcb80d0f20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00097a0 tx=0x7fbcb002b820 comp rx=0 tx=0).stop 2023-12-01T03:13:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.036+0000 7fbcbd217640 1 -- 172.21.15.115:0/2838031572 shutdown_connections 2023-12-01T03:13:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.036+0000 7fbcbd217640 1 --2- 172.21.15.115:0/2838031572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb80b4d50 0x7fbcb80d0f20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.036+0000 7fbcbd217640 1 --2- 172.21.15.115:0/2838031572 >> v2:172.21.15.142:3300/0 conn(0x7fbcb815ce40 0x7fbcb80b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.038 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.036+0000 7fbcbd217640 1 -- 172.21.15.115:0/2838031572 >> 172.21.15.115:0/2838031572 conn(0x7fbcb8065f90 msgr2=0x7fbcb80cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.036+0000 7fbcbd217640 1 -- 172.21.15.115:0/2838031572 shutdown_connections 2023-12-01T03:13:17.039 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.037+0000 7fbcbd217640 1 -- 172.21.15.115:0/2838031572 wait complete. 2023-12-01T03:13:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.038+0000 7fbcbd217640 1 Processor -- start 2023-12-01T03:13:17.040 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.038+0000 7fbcbd217640 1 -- start start 2023-12-01T03:13:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.039+0000 7fbcbd217640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb80b4d50 0x7fbcb815fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.039+0000 7fbcbd217640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb80d2730 0x7fbcb81e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.039+0000 7fbcbd217640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb815ce40 0x7fbcb81e5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.039+0000 7fbcbd217640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8148c90 con 0x7fbcb80d2730 2023-12-01T03:13:17.041 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.039+0000 7fbcbd217640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb815f900 con 0x7fbcb815ce40 2023-12-01T03:13:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.039+0000 7fbcbd217640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80c8ec0 con 0x7fbcb80b4d50 2023-12-01T03:13:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.040+0000 7fbcb659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb80d2730 0x7fbcb81e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.040+0000 7fbcb659c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb80d2730 0x7fbcb81e67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52750/0 (socket says 172.21.15.115:52750) 2023-12-01T03:13:17.042 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.040+0000 7fbcb659c640 1 -- 172.21.15.115:0/678377349 learned_addr learned my addr 172.21.15.115:0/678377349 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.040+0000 7fbcb759e640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb815ce40 0x7fbcb81e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.040+0000 7fbcb6d9d640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb80b4d50 0x7fbcb815fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.040+0000 7fbcb659c640 1 -- 172.21.15.115:0/678377349 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb80b4d50 msgr2=0x7fbcb815fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.040+0000 7fbcb659c640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbcb80b4d50 0x7fbcb815fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.040+0000 7fbcb659c640 1 -- 172.21.15.115:0/678377349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb815ce40 msgr2=0x7fbcb81e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.040+0000 7fbcb659c640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb815ce40 0x7fbcb81e5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.043 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.040+0000 7fbcb659c640 1 -- 172.21.15.115:0/678377349 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb0009400 con 0x7fbcb80d2730 2023-12-01T03:13:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.041+0000 7fbcb659c640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb80d2730 0x7fbcb81e67c0 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7fbca000c430 tx=0x7fbca000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.041+0000 7fbcaf7fe640 1 -- 172.21.15.115:0/678377349 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca00074e0 con 0x7fbcb80d2730 2023-12-01T03:13:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.041+0000 7fbcaf7fe640 1 -- 172.21.15.115:0/678377349 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca0007640 con 0x7fbcb80d2730 2023-12-01T03:13:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.041+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb80c91a0 con 0x7fbcb80d2730 2023-12-01T03:13:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.041+0000 7fbcaf7fe640 1 -- 172.21.15.115:0/678377349 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0004a20 con 0x7fbcb80d2730 2023-12-01T03:13:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.041+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb815f590 con 0x7fbcb80d2730 2023-12-01T03:13:17.044 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.042+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbcb815f410 con 0x7fbcb80d2730 2023-12-01T03:13:17.045 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.043+0000 7fbcaf7fe640 1 -- 172.21.15.115:0/678377349 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fbca0004b80 con 0x7fbcb80d2730 2023-12-01T03:13:17.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.044+0000 7fbcaf7fe640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbc9806a550 0x7fbc9806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.044+0000 7fbcaf7fe640 1 -- 172.21.15.115:0/678377349 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7fbca00884f0 con 0x7fbcb80d2730 2023-12-01T03:13:17.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.044+0000 7fbcb6d9d640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbc9806a550 0x7fbc9806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.044+0000 7fbcaf7fe640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] conn(0x7fbc98070a20 0x7fbc98072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.044+0000 7fbcaf7fe640 1 -- 172.21.15.115:0/678377349 --> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc980734f0 con 0x7fbc98070a20 2023-12-01T03:13:17.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.044+0000 7fbcaf7fe640 1 -- 172.21.15.115:0/678377349 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbca008abb0 con 0x7fbcb80d2730 2023-12-01T03:13:17.046 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.045+0000 7fbcb759e640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] conn(0x7fbc98070a20 0x7fbc98072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.045+0000 7fbcb6d9d640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbc9806a550 0x7fbc9806ca10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009d80 tx=0x7fbca800b040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.045+0000 7fbcb759e640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] conn(0x7fbc98070a20 0x7fbc98072e40 crc :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.047 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.046+0000 7fbcaf7fe640 1 -- 172.21.15.115:0/678377349 <== osd.1 v2:172.21.15.115:6808/88580 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbc980734f0 con 0x7fbc98070a20 2023-12-01T03:13:17.061 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.060+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 --> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbcb80d1f30 con 0x7fbc98070a20 2023-12-01T03:13:17.062 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.060+0000 7fbcaf7fe640 1 -- 172.21.15.115:0/678377349 <== osd.1 v2:172.21.15.115:6808/88580 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbcb0031a50 con 0x7fbc98070a20 2023-12-01T03:13:17.062 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:13:17.062 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-01T03:13:17.062 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:13:17.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] conn(0x7fbc98070a20 msgr2=0x7fbc98072e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.063 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcbd217640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:6808/88580,v1:172.21.15.115:6809/88580] conn(0x7fbc98070a20 0x7fbc98072e40 crc :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbc9806a550 msgr2=0x7fbc9806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcbd217640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbc9806a550 0x7fbc9806ca10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009d80 tx=0x7fbca800b040 comp rx=0 tx=0).stop 2023-12-01T03:13:17.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb80d2730 msgr2=0x7fbcb81e67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcbd217640 1 --2- 172.21.15.115:0/678377349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb80d2730 0x7fbcb81e67c0 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7fbca000c430 tx=0x7fbca000c8f0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcb759e640 1 -- 172.21.15.115:0/678377349 reap_dead start 2023-12-01T03:13:17.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 shutdown_connections 2023-12-01T03:13:17.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 >> 172.21.15.115:0/678377349 conn(0x7fbcb8065f90 msgr2=0x7fbcb815a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:17.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 shutdown_connections 2023-12-01T03:13:17.064 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.061+0000 7fbcbd217640 1 -- 172.21.15.115:0/678377349 wait complete. 2023-12-01T03:13:17.079 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-12-01T03:13:17.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.286+0000 7f8d8d209640 1 Processor -- start 2023-12-01T03:13:17.288 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.286+0000 7f8d8d209640 1 -- start start 2023-12-01T03:13:17.289 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d8d209640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8d880cf160 0x7f8d880c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d8d209640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d880c9070 con 0x7f8d880b0050 2023-12-01T03:13:17.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d8d209640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d880c9350 con 0x7f8d880afc70 2023-12-01T03:13:17.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d8d209640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d880c8e60 con 0x7f8d880cf160 2023-12-01T03:13:17.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d7ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8d880cf160 0x7f8d880c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d7ffff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f8d880cf160 0x7f8d880c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52754/0 (socket says 172.21.15.115:52754) 2023-12-01T03:13:17.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d8759e640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f8d880b0050 0x7f8d8815d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:58880/0 (socket says 172.21.15.115:58880) 2023-12-01T03:13:17.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d7ffff640 1 -- 172.21.15.115:0/1883962307 learned_addr learned my addr 172.21.15.115:0/1883962307 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:17.290 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d7ffff640 1 -- 172.21.15.115:0/1883962307 >> v1:172.21.15.142:6789/0 conn(0x7f8d880afc70 legacy=0x7f8d880cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:17.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d7ffff640 1 -- 172.21.15.115:0/1883962307 >> v1:172.21.15.115:6789/0 conn(0x7f8d880b0050 legacy=0x7f8d8815d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.288+0000 7f8d7ffff640 1 -- 172.21.15.115:0/1883962307 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880c9630 con 0x7f8d880cf160 2023-12-01T03:13:17.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.290+0000 7f8d7ffff640 1 --2- 172.21.15.115:0/1883962307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880cf160 0x7f8d880c9e30 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009870 tx=0x7f8d7002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da6c595a951a1e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.291 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.290+0000 7f8d8659c640 1 -- 172.21.15.115:0/1883962307 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70030860 con 0x7f8d880cf160 2023-12-01T03:13:17.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.290+0000 7f8d8659c640 1 -- 172.21.15.115:0/1883962307 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d700309c0 con 0x7f8d880cf160 2023-12-01T03:13:17.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.290+0000 7f8d8659c640 1 -- 172.21.15.115:0/1883962307 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70030d50 con 0x7f8d880cf160 2023-12-01T03:13:17.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.290+0000 7f8d8d209640 1 -- 172.21.15.115:0/1883962307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880cf160 msgr2=0x7f8d880c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.292 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.290+0000 7f8d8d209640 1 --2- 172.21.15.115:0/1883962307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880cf160 0x7f8d880c9e30 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009870 tx=0x7f8d7002b9a0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.291+0000 7f8d8d209640 1 -- 172.21.15.115:0/1883962307 shutdown_connections 2023-12-01T03:13:17.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.291+0000 7f8d8d209640 1 --2- 172.21.15.115:0/1883962307 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880cf160 0x7f8d880c9e30 unknown :-1 s=CLOSED pgs=3132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.291+0000 7f8d8d209640 1 -- 172.21.15.115:0/1883962307 >> 172.21.15.115:0/1883962307 conn(0x7f8d88065b40 msgr2=0x7f8d88065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:17.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.291+0000 7f8d8d209640 1 -- 172.21.15.115:0/1883962307 shutdown_connections 2023-12-01T03:13:17.293 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.291+0000 7f8d8d209640 1 -- 172.21.15.115:0/1883962307 wait complete. 2023-12-01T03:13:17.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.292+0000 7f8d8d209640 1 Processor -- start 2023-12-01T03:13:17.294 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.292+0000 7f8d8d209640 1 -- start start 2023-12-01T03:13:17.295 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.293+0000 7f8d8d209640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d880afc70 0x7f8d881502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.293+0000 7f8d8d209640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880b0050 0x7f8d8815c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.293+0000 7f8d8d209640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880cf160 0x7f8d8815c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.293+0000 7f8d8d209640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88153cd0 con 0x7f8d880b0050 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.293+0000 7f8d8d209640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880ae580 con 0x7f8d880cf160 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.293+0000 7f8d8d209640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880ae860 con 0x7f8d880afc70 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.294+0000 7f8d7ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880b0050 0x7f8d8815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.294+0000 7f8d7ffff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880b0050 0x7f8d8815c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:52770/0 (socket says 172.21.15.115:52770) 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.294+0000 7f8d7ffff640 1 -- 172.21.15.115:0/2342357753 learned_addr learned my addr 172.21.15.115:0/2342357753 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.294+0000 7f8d7ffff640 1 -- 172.21.15.115:0/2342357753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d880afc70 msgr2=0x7f8d881502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.294+0000 7f8d7ffff640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d880afc70 0x7f8d881502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.294+0000 7f8d7ffff640 1 -- 172.21.15.115:0/2342357753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880cf160 msgr2=0x7f8d8815c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.294+0000 7f8d7ffff640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880cf160 0x7f8d8815c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.296 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.294+0000 7f8d7ffff640 1 -- 172.21.15.115:0/2342357753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700094d0 con 0x7f8d880b0050 2023-12-01T03:13:17.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.294+0000 7f8d7ffff640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880b0050 0x7f8d8815c110 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009870 tx=0x7f8d70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.295+0000 7f8d7f7fe640 1 -- 172.21.15.115:0/2342357753 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70030810 con 0x7f8d880b0050 2023-12-01T03:13:17.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.295+0000 7f8d7f7fe640 1 -- 172.21.15.115:0/2342357753 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d70043400 con 0x7f8d880b0050 2023-12-01T03:13:17.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.295+0000 7f8d7f7fe640 1 -- 172.21.15.115:0/2342357753 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700437b0 con 0x7f8d880b0050 2023-12-01T03:13:17.297 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.295+0000 7f8d8d209640 1 -- 172.21.15.115:0/2342357753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d880aeb40 con 0x7f8d880b0050 2023-12-01T03:13:17.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.295+0000 7f8d8d209640 1 -- 172.21.15.115:0/2342357753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d88150820 con 0x7f8d880b0050 2023-12-01T03:13:17.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.296+0000 7f8d8d209640 1 -- 172.21.15.115:0/2342357753 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d880af470 con 0x7f8d880b0050 2023-12-01T03:13:17.298 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.297+0000 7f8d7f7fe640 1 -- 172.21.15.115:0/2342357753 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f8d70043910 con 0x7f8d880b0050 2023-12-01T03:13:17.299 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.298+0000 7f8d7f7fe640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8d6806a550 0x7f8d6806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.298+0000 7f8d7f7fe640 1 -- 172.21.15.115:0/2342357753 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f8d7003e070 con 0x7f8d880b0050 2023-12-01T03:13:17.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.298+0000 7f8d86d9d640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8d6806a550 0x7f8d6806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.300 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.299+0000 7f8d86d9d640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8d6806a550 0x7f8d6806ca10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d78006fd0 tx=0x7f8d78006bc0 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.304 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.302+0000 7f8d7f7fe640 1 -- 172.21.15.115:0/2342357753 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f8d70077270 con 0x7f8d880b0050 2023-12-01T03:13:17.492 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.490+0000 7f8d8d209640 1 -- 172.21.15.115:0/2342357753 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d880c9910 con 0x7f8d6806a550 2023-12-01T03:13:17.493 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.491+0000 7f8d7f7fe640 1 -- 172.21.15.115:0/2342357753 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8d78004020 con 0x7f8d6806a550 2023-12-01T03:13:17.493 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 1 to deep-scrub 2023-12-01T03:13:17.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.493+0000 7f8d8d209640 1 -- 172.21.15.115:0/2342357753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8d6806a550 msgr2=0x7f8d6806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.493+0000 7f8d8d209640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8d6806a550 0x7f8d6806ca10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d78006fd0 tx=0x7f8d78006bc0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.495 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.493+0000 7f8d8d209640 1 -- 172.21.15.115:0/2342357753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880b0050 msgr2=0x7f8d8815c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.493+0000 7f8d8d209640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880b0050 0x7f8d8815c110 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009870 tx=0x7f8d70004730 comp rx=0 tx=0).stop 2023-12-01T03:13:17.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.493+0000 7f8d8d209640 1 -- 172.21.15.115:0/2342357753 shutdown_connections 2023-12-01T03:13:17.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.493+0000 7f8d8d209640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f8d6806a550 0x7f8d6806ca10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.493+0000 7f8d8d209640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d880cf160 0x7f8d8815c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.493+0000 7f8d8d209640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880b0050 0x7f8d8815c110 unknown :-1 s=CLOSED pgs=3133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.493+0000 7f8d8d209640 1 --2- 172.21.15.115:0/2342357753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d880afc70 0x7f8d881502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.493+0000 7f8d8d209640 1 -- 172.21.15.115:0/2342357753 >> 172.21.15.115:0/2342357753 conn(0x7f8d88065b40 msgr2=0x7f8d880c9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:17.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.494+0000 7f8d8d209640 1 -- 172.21.15.115:0/2342357753 shutdown_connections 2023-12-01T03:13:17.496 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.494+0000 7f8d8d209640 1 -- 172.21.15.115:0/2342357753 wait complete. 2023-12-01T03:13:17.515 INFO:tasks.ceph:Scrubbing osd.2 2023-12-01T03:13:17.515 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-12-01T03:13:17.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.728+0000 7feb8d720640 1 Processor -- start 2023-12-01T03:13:17.730 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.728+0000 7feb8d720640 1 -- start start 2023-12-01T03:13:17.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.730+0000 7feb8d720640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7feb880b4070 0x7feb880cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.732 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.730+0000 7feb8d720640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8806c860 con 0x7feb880cb8c0 2023-12-01T03:13:17.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.730+0000 7feb8d720640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8815b050 con 0x7feb8815ac70 2023-12-01T03:13:17.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.730+0000 7feb8d720640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb880ce0b0 con 0x7feb880b4070 2023-12-01T03:13:17.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.730+0000 7feb7e7fc640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7feb880cb8c0 0x7feb880cc370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59170/0 (socket says 172.21.15.115:59170) 2023-12-01T03:13:17.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.730+0000 7feb7e7fc640 1 -- 172.21.15.115:0/2138959082 learned_addr learned my addr 172.21.15.115:0/2138959082 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:17.733 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.730+0000 7feb877fe640 1 --2- 172.21.15.115:0/2138959082 >> v2:172.21.15.115:3300/0 conn(0x7feb880b4070 0x7feb880cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.731+0000 7feb877fe640 1 -- 172.21.15.115:0/2138959082 >> v1:172.21.15.142:6789/0 conn(0x7feb8815ac70 legacy=0x7feb880cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:17.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.731+0000 7feb877fe640 1 -- 172.21.15.115:0/2138959082 >> v1:172.21.15.115:6789/0 conn(0x7feb880cb8c0 legacy=0x7feb880cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.731+0000 7feb877fe640 1 -- 172.21.15.115:0/2138959082 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb880ce390 con 0x7feb880b4070 2023-12-01T03:13:17.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.731+0000 7feb867fc640 1 -- 172.21.15.115:0/2138959082 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841722251 0 0) 0x7feb8806c860 con 0x7feb880cb8c0 2023-12-01T03:13:17.736 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.732+0000 7feb877fe640 1 --2- 172.21.15.115:0/2138959082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb880b4070 0x7feb880cca80 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7feb80009d90 tx=0x7feb8002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5c5f16775d6557 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.732+0000 7feb867fc640 1 -- 172.21.15.115:0/2138959082 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8002cbd0 con 0x7feb880b4070 2023-12-01T03:13:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.732+0000 7feb867fc640 1 -- 172.21.15.115:0/2138959082 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8002cd30 con 0x7feb880b4070 2023-12-01T03:13:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.732+0000 7feb867fc640 1 -- 172.21.15.115:0/2138959082 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8002cea0 con 0x7feb880b4070 2023-12-01T03:13:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.733+0000 7feb8d720640 1 -- 172.21.15.115:0/2138959082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb880b4070 msgr2=0x7feb880cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.733+0000 7feb8d720640 1 --2- 172.21.15.115:0/2138959082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb880b4070 0x7feb880cca80 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7feb80009d90 tx=0x7feb8002be20 comp rx=0 tx=0).stop 2023-12-01T03:13:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.733+0000 7feb8d720640 1 -- 172.21.15.115:0/2138959082 shutdown_connections 2023-12-01T03:13:17.737 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.733+0000 7feb8d720640 1 --2- 172.21.15.115:0/2138959082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb880b4070 0x7feb880cca80 unknown :-1 s=CLOSED pgs=3135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.733+0000 7feb8d720640 1 -- 172.21.15.115:0/2138959082 >> 172.21.15.115:0/2138959082 conn(0x7feb880662c0 msgr2=0x7feb880b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:17.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.733+0000 7feb8d720640 1 -- 172.21.15.115:0/2138959082 shutdown_connections 2023-12-01T03:13:17.738 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.733+0000 7feb8d720640 1 -- 172.21.15.115:0/2138959082 wait complete. 2023-12-01T03:13:17.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.735+0000 7feb8d720640 1 Processor -- start 2023-12-01T03:13:17.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.735+0000 7feb8d720640 1 -- start start 2023-12-01T03:13:17.741 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.736+0000 7feb8d720640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb880b4070 0x7feb881550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.736+0000 7feb8d720640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb880cb8c0 0x7feb880c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.736+0000 7feb8d720640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8815ac70 0x7feb880c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.736+0000 7feb8d720640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb880cabf0 con 0x7feb880cb8c0 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.736+0000 7feb8d720640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb880ca600 con 0x7feb8815ac70 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.736+0000 7feb8d720640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb880b1460 con 0x7feb880b4070 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.736+0000 7feb7e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb880cb8c0 0x7feb880c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.736+0000 7feb7e7fc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb880cb8c0 0x7feb880c2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60906/0 (socket says 172.21.15.115:60906) 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.736+0000 7feb7e7fc640 1 -- 172.21.15.115:0/10617063 learned_addr learned my addr 172.21.15.115:0/10617063 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.737+0000 7feb877fe640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8815ac70 0x7feb880c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.737+0000 7feb7e7fc640 1 -- 172.21.15.115:0/10617063 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb880b4070 msgr2=0x7feb881550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.737+0000 7feb86ffd640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb880b4070 0x7feb881550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.737+0000 7feb7e7fc640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb880b4070 0x7feb881550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.742 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.737+0000 7feb7e7fc640 1 -- 172.21.15.115:0/10617063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8815ac70 msgr2=0x7feb880c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.737+0000 7feb7e7fc640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8815ac70 0x7feb880c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.737+0000 7feb7e7fc640 1 -- 172.21.15.115:0/10617063 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb80009a70 con 0x7feb880cb8c0 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.737+0000 7feb877fe640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb8815ac70 0x7feb880c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.737+0000 7feb7e7fc640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb880cb8c0 0x7feb880c2ce0 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7feb7000c4d0 tx=0x7feb7000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.738+0000 7feb7ffff640 1 -- 172.21.15.115:0/10617063 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700074e0 con 0x7feb880cb8c0 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.738+0000 7feb7ffff640 1 -- 172.21.15.115:0/10617063 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb70007660 con 0x7feb880cb8c0 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.738+0000 7feb7ffff640 1 -- 172.21.15.115:0/10617063 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70004a20 con 0x7feb880cb8c0 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.738+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb880b1740 con 0x7feb880cb8c0 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.738+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb880ca810 con 0x7feb880cb8c0 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.739+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb8806cde0 con 0x7feb880cb8c0 2023-12-01T03:13:17.743 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.740+0000 7feb7ffff640 1 -- 172.21.15.115:0/10617063 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7feb70004b80 con 0x7feb880cb8c0 2023-12-01T03:13:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.741+0000 7feb7ffff640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb6806a550 0x7feb6806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.741+0000 7feb7ffff640 1 -- 172.21.15.115:0/10617063 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7feb70087c60 con 0x7feb880cb8c0 2023-12-01T03:13:17.744 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.741+0000 7feb7ffff640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] conn(0x7feb68070a20 0x7feb68072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.741+0000 7feb7ffff640 1 -- 172.21.15.115:0/10617063 --> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb680734f0 con 0x7feb68070a20 2023-12-01T03:13:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.741+0000 7feb7ffff640 1 -- 172.21.15.115:0/10617063 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb7008c2a0 con 0x7feb880cb8c0 2023-12-01T03:13:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.741+0000 7feb86ffd640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb6806a550 0x7feb6806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.741+0000 7feb877fe640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] conn(0x7feb68070a20 0x7feb68072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.742+0000 7feb86ffd640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb6806a550 0x7feb6806ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feb78003d90 tx=0x7feb78009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.742+0000 7feb877fe640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] conn(0x7feb68070a20 0x7feb68072e40 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.745 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.743+0000 7feb7ffff640 1 -- 172.21.15.115:0/10617063 <== osd.2 v2:172.21.15.115:6800/88578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7feb680734f0 con 0x7feb68070a20 2023-12-01T03:13:17.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.756+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 --> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feb8806cb00 con 0x7feb68070a20 2023-12-01T03:13:17.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.757+0000 7feb7ffff640 1 -- 172.21.15.115:0/10617063 <== osd.2 v2:172.21.15.115:6800/88578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7feb80034590 con 0x7feb68070a20 2023-12-01T03:13:17.759 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:13:17.759 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-01T03:13:17.759 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:13:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.757+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] conn(0x7feb68070a20 msgr2=0x7feb68072e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.757+0000 7feb8d720640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:6800/88578,v1:172.21.15.115:6801/88578] conn(0x7feb68070a20 0x7feb68072e40 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.758+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb6806a550 msgr2=0x7feb6806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.758+0000 7feb8d720640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7feb6806a550 0x7feb6806ca10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feb78003d90 tx=0x7feb78009040 comp rx=0 tx=0).stop 2023-12-01T03:13:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.758+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb880cb8c0 msgr2=0x7feb880c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.758+0000 7feb8d720640 1 --2- 172.21.15.115:0/10617063 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb880cb8c0 0x7feb880c2ce0 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7feb7000c4d0 tx=0x7feb7000c990 comp rx=0 tx=0).stop 2023-12-01T03:13:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.758+0000 7feb877fe640 1 -- 172.21.15.115:0/10617063 reap_dead start 2023-12-01T03:13:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.758+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 shutdown_connections 2023-12-01T03:13:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.758+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 >> 172.21.15.115:0/10617063 conn(0x7feb880662c0 msgr2=0x7feb880b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.758+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 shutdown_connections 2023-12-01T03:13:17.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.758+0000 7feb8d720640 1 -- 172.21.15.115:0/10617063 wait complete. 2023-12-01T03:13:17.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-12-01T03:13:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.988+0000 7f05a8877640 1 Processor -- start 2023-12-01T03:13:17.990 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.988+0000 7f05a8877640 1 -- start start 2023-12-01T03:13:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.990+0000 7f05a8877640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f05a4160370 0x7f05a40cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.990+0000 7f05a8877640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a40b6090 0x7f05a41613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.990+0000 7f05a8877640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a40d2c10 con 0x7f05a40cf740 2023-12-01T03:13:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.990+0000 7f05a8877640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a40ce660 con 0x7f05a4160370 2023-12-01T03:13:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.990+0000 7f05a8877640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41618f0 con 0x7f05a40b6090 2023-12-01T03:13:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.990+0000 7f05a259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f05a4160370 0x7f05a40cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.990+0000 7f05a259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f05a4160370 0x7f05a40cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60920/0 (socket says 172.21.15.115:60920) 2023-12-01T03:13:17.992 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.990+0000 7f05a259c640 1 -- 172.21.15.115:0/3793904094 learned_addr learned my addr 172.21.15.115:0/3793904094 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.991+0000 7f05a259c640 1 -- 172.21.15.115:0/3793904094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a40b6090 msgr2=0x7f05a41613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.991+0000 7f05a259c640 1 --2- 172.21.15.115:0/3793904094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a40b6090 0x7f05a41613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.991+0000 7f05a259c640 1 -- 172.21.15.115:0/3793904094 >> v1:172.21.15.142:6789/0 conn(0x7f05a40cf740 legacy=0x7f05a40b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:17.993 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.991+0000 7f05a259c640 1 -- 172.21.15.115:0/3793904094 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a4160750 con 0x7f05a4160370 2023-12-01T03:13:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.992+0000 7f05a259c640 1 --2- 172.21.15.115:0/3793904094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a4160370 0x7f05a40cdcf0 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 crypto rx=0x7f0594009870 tx=0x7f059402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980fbedc033a271d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.992+0000 7f05a159a640 1 -- 172.21.15.115:0/3793904094 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594031640 con 0x7f05a4160370 2023-12-01T03:13:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.992+0000 7f05a159a640 1 -- 172.21.15.115:0/3793904094 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05940317a0 con 0x7f05a4160370 2023-12-01T03:13:17.994 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.992+0000 7f05a159a640 1 -- 172.21.15.115:0/3793904094 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594031b30 con 0x7f05a4160370 2023-12-01T03:13:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.993+0000 7f05a8877640 1 -- 172.21.15.115:0/3793904094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a4160370 msgr2=0x7f05a40cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.993+0000 7f05a8877640 1 --2- 172.21.15.115:0/3793904094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a4160370 0x7f05a40cdcf0 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 crypto rx=0x7f0594009870 tx=0x7f059402b8f0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.993+0000 7f05a8877640 1 -- 172.21.15.115:0/3793904094 shutdown_connections 2023-12-01T03:13:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.993+0000 7f05a8877640 1 --2- 172.21.15.115:0/3793904094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a40b6090 0x7f05a41613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.993+0000 7f05a8877640 1 --2- 172.21.15.115:0/3793904094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a4160370 0x7f05a40cdcf0 unknown :-1 s=CLOSED pgs=3137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.993+0000 7f05a8877640 1 -- 172.21.15.115:0/3793904094 >> 172.21.15.115:0/3793904094 conn(0x7f05a4065960 msgr2=0x7f05a4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.993+0000 7f05a8877640 1 -- 172.21.15.115:0/3793904094 shutdown_connections 2023-12-01T03:13:17.995 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.993+0000 7f05a8877640 1 -- 172.21.15.115:0/3793904094 wait complete. 2023-12-01T03:13:17.996 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.995+0000 7f05a8877640 1 Processor -- start 2023-12-01T03:13:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.995+0000 7f05a8877640 1 -- start start 2023-12-01T03:13:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.996+0000 7f05a8877640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a40b6090 0x7f05a40c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.996+0000 7f05a8877640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a40cf740 0x7f05a415c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.996+0000 7f05a8877640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05a4160370 0x7f05a415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:17.998 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.996+0000 7f05a8877640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a414bb20 con 0x7f05a40cf740 2023-12-01T03:13:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.996+0000 7f05a8877640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40ca920 con 0x7f05a4160370 2023-12-01T03:13:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.996+0000 7f05a8877640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40cac00 con 0x7f05a40b6090 2023-12-01T03:13:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.996+0000 7f05a1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a40cf740 0x7f05a415c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.997+0000 7f05a1d9b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a40cf740 0x7f05a415c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60928/0 (socket says 172.21.15.115:60928) 2023-12-01T03:13:17.999 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.997+0000 7f05a1d9b640 1 -- 172.21.15.115:0/1648993439 learned_addr learned my addr 172.21.15.115:0/1648993439 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.997+0000 7f05a1d9b640 1 -- 172.21.15.115:0/1648993439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a40b6090 msgr2=0x7f05a40c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.997+0000 7f05a1d9b640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a40b6090 0x7f05a40c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.997+0000 7f05a1d9b640 1 -- 172.21.15.115:0/1648993439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05a4160370 msgr2=0x7f05a415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.997+0000 7f05a1d9b640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05a4160370 0x7f05a415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.997+0000 7f05a1d9b640 1 -- 172.21.15.115:0/1648993439 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05940094d0 con 0x7f05a40cf740 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.997+0000 7f05a1d9b640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a40cf740 0x7f05a415c7a0 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7f058c00b9c0 tx=0x7f058c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.998+0000 7f059affd640 1 -- 172.21.15.115:0/1648993439 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00cba0 con 0x7f05a40cf740 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.998+0000 7f059affd640 1 -- 172.21.15.115:0/1648993439 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c00cd20 con 0x7f05a40cf740 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.998+0000 7f059affd640 1 -- 172.21.15.115:0/1648993439 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c014680 con 0x7f05a40cf740 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.998+0000 7f05a8877640 1 -- 172.21.15.115:0/1648993439 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a40caee0 con 0x7f05a40cf740 2023-12-01T03:13:18.000 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.998+0000 7f05a8877640 1 -- 172.21.15.115:0/1648993439 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a40cb4a0 con 0x7f05a40cf740 2023-12-01T03:13:18.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:17.999+0000 7f05a8877640 1 -- 172.21.15.115:0/1648993439 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a40cb6c0 con 0x7f05a40cf740 2023-12-01T03:13:18.001 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.000+0000 7f059affd640 1 -- 172.21.15.115:0/1648993439 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f058c01d3b0 con 0x7f05a40cf740 2023-12-01T03:13:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.001+0000 7f059affd640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f058806a550 0x7f058806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.002 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.001+0000 7f059affd640 1 -- 172.21.15.115:0/1648993439 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f058c088a60 con 0x7f05a40cf740 2023-12-01T03:13:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.001+0000 7f05a259c640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f058806a550 0x7f058806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.003 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.002+0000 7f05a259c640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f058806a550 0x7f058806ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0594036000 tx=0x7f0594030000 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:18.007 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.006+0000 7f059affd640 1 -- 172.21.15.115:0/1648993439 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f058c053f30 con 0x7f05a40cf740 2023-12-01T03:13:18.199 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.198+0000 7f05a8877640 1 -- 172.21.15.115:0/1648993439 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f05a40d1120 con 0x7f058806a550 2023-12-01T03:13:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.199+0000 7f059affd640 1 -- 172.21.15.115:0/1648993439 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0594031a10 con 0x7f058806a550 2023-12-01T03:13:18.200 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 2 to deep-scrub 2023-12-01T03:13:18.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 -- 172.21.15.115:0/1648993439 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f058806a550 msgr2=0x7f058806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f058806a550 0x7f058806ca10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0594036000 tx=0x7f0594030000 comp rx=0 tx=0).stop 2023-12-01T03:13:18.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 -- 172.21.15.115:0/1648993439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a40cf740 msgr2=0x7f05a415c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a40cf740 0x7f05a415c7a0 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7f058c00b9c0 tx=0x7f058c00be80 comp rx=0 tx=0).stop 2023-12-01T03:13:18.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 -- 172.21.15.115:0/1648993439 shutdown_connections 2023-12-01T03:13:18.203 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f058806a550 0x7f058806ca10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05a4160370 0x7f05a415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05a40cf740 0x7f05a415c7a0 unknown :-1 s=CLOSED pgs=3138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 --2- 172.21.15.115:0/1648993439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f05a40b6090 0x7f05a40c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 -- 172.21.15.115:0/1648993439 >> 172.21.15.115:0/1648993439 conn(0x7f05a4065960 msgr2=0x7f05a40d3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 -- 172.21.15.115:0/1648993439 shutdown_connections 2023-12-01T03:13:18.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.201+0000 7f05a8877640 1 -- 172.21.15.115:0/1648993439 wait complete. 2023-12-01T03:13:18.226 INFO:tasks.ceph:Scrubbing osd.3 2023-12-01T03:13:18.226 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-12-01T03:13:18.450 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.449+0000 7f9392e2d640 1 Processor -- start 2023-12-01T03:13:18.516 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.449+0000 7f9392e2d640 1 -- start start 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.450+0000 7f9392e2d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f938c0ce9c0 0x7f938c0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.450+0000 7f9392e2d640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938c0cc5d0 con 0x7f938c15abb0 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.450+0000 7f9392e2d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938c0cf7f0 con 0x7f938c0cf3a0 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.450+0000 7f9392e2d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c0cc810 con 0x7f938c0ce9c0 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.450+0000 7f9391e2b640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f938c15abb0 0x7f938c0cbec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59184/0 (socket says 172.21.15.115:59184) 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.450+0000 7f9391e2b640 1 -- 172.21.15.115:0/1047761909 learned_addr learned my addr 172.21.15.115:0/1047761909 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.451+0000 7f939162a640 1 --2- 172.21.15.115:0/1047761909 >> v2:172.21.15.142:3300/0 conn(0x7f938c0ce9c0 0x7f938c0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.451+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725803409 0 0) 0x7f9380001610 con 0x7f938c15abb0 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.451+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f937c003520 con 0x7f938c15abb0 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.451+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1779441174 0 0) 0x7f937c003520 con 0x7f938c15abb0 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.451+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 >> v2:172.21.15.142:3300/0 conn(0x7f938c0ce9c0 msgr2=0x7f938c0cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.451+0000 7f9390e29640 1 --2- 172.21.15.115:0/1047761909 >> v2:172.21.15.142:3300/0 conn(0x7f938c0ce9c0 0x7f938c0cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.522 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.451+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 >> v1:172.21.15.142:6789/0 conn(0x7f938c0cf3a0 legacy=0x7f938c0b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.451+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c0b0330 con 0x7f938c15abb0 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1980110602 0 0) 0x7f9380001d00 con 0x7f938c15abb0 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 >> v1:172.21.15.115:6789/0 conn(0x7f938c15abb0 legacy=0x7f938c0cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9390e29640 1 --2- 172.21.15.115:0/1047761909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f937c0039c0 0x7f937c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9390e29640 1 --2- 172.21.15.115:0/1047761909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f937c0047c0 0x7f937c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9390e29640 1 --2- 172.21.15.115:0/1047761909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f937c005180 0x7f937c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0cc810 con 0x7f937c0039c0 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c00c080 con 0x7f937c005180 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937c00c240 con 0x7f937c0047c0 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9391e2b640 1 --2- 172.21.15.115:0/1047761909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f937c0039c0 0x7f937c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9391e2b640 1 -- 172.21.15.115:0/1047761909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f937c0047c0 msgr2=0x7f937c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9391e2b640 1 --2- 172.21.15.115:0/1047761909 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f937c0047c0 0x7f937c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9391e2b640 1 -- 172.21.15.115:0/1047761909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f937c005180 msgr2=0x7f937c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:18.523 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9391e2b640 1 --2- 172.21.15.115:0/1047761909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f937c005180 0x7f937c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f9391e2b640 1 -- 172.21.15.115:0/1047761909 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c003520 con 0x7f937c0039c0 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.452+0000 7f939262c640 1 -- 172.21.15.115:0/1047761909 reap_dead start 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.453+0000 7f9391e2b640 1 --2- 172.21.15.115:0/1047761909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f937c0039c0 0x7f937c003de0 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7f938000c410 tx=0x7f937c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0821f8d7bda407 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.454+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938002ce00 con 0x7f937c0039c0 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.454+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9380005b20 con 0x7f937c0039c0 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.454+0000 7f9390e29640 1 -- 172.21.15.115:0/1047761909 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93800070d0 con 0x7f937c0039c0 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.454+0000 7f9392e2d640 1 -- 172.21.15.115:0/1047761909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f937c0039c0 msgr2=0x7f937c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.454+0000 7f9392e2d640 1 --2- 172.21.15.115:0/1047761909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f937c0039c0 0x7f937c003de0 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7f938000c410 tx=0x7f937c0057f0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.454+0000 7f9392e2d640 1 -- 172.21.15.115:0/1047761909 shutdown_connections 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.454+0000 7f9392e2d640 1 --2- 172.21.15.115:0/1047761909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f937c0039c0 0x7f937c003de0 unknown :-1 s=CLOSED pgs=3140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.454+0000 7f9392e2d640 1 -- 172.21.15.115:0/1047761909 >> 172.21.15.115:0/1047761909 conn(0x7f938c099a20 msgr2=0x7f938c0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.454+0000 7f9392e2d640 1 -- 172.21.15.115:0/1047761909 shutdown_connections 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.454+0000 7f9392e2d640 1 -- 172.21.15.115:0/1047761909 wait complete. 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.456+0000 7f9392e2d640 1 Processor -- start 2023-12-01T03:13:18.524 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.456+0000 7f9392e2d640 1 -- start start 2023-12-01T03:13:18.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.457+0000 7f9392e2d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f937c0039c0 0x7f938c15af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.457+0000 7f9392e2d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f938c144c10 0x7f938c156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.457+0000 7f9392e2d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f938c145090 0x7f938c1568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.457+0000 7f9392e2d640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c156df0 con 0x7f938c145090 2023-12-01T03:13:18.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.457+0000 7f9392e2d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1586c0 con 0x7f937c0039c0 2023-12-01T03:13:18.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.457+0000 7f9392e2d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1589a0 con 0x7f938c144c10 2023-12-01T03:13:18.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.457+0000 7f939262c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f938c145090 0x7f938c1568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.525 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.457+0000 7f939262c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f938c145090 0x7f938c1568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60948/0 (socket says 172.21.15.115:60948) 2023-12-01T03:13:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.457+0000 7f939262c640 1 -- 172.21.15.115:0/503451157 learned_addr learned my addr 172.21.15.115:0/503451157 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.458+0000 7f939262c640 1 -- 172.21.15.115:0/503451157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f938c144c10 msgr2=0x7f938c156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.458+0000 7f939262c640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f938c144c10 0x7f938c156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.458+0000 7f939262c640 1 -- 172.21.15.115:0/503451157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f937c0039c0 msgr2=0x7f938c15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.458+0000 7f939262c640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f937c0039c0 0x7f938c15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.458+0000 7f939262c640 1 -- 172.21.15.115:0/503451157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c158c80 con 0x7f938c145090 2023-12-01T03:13:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.458+0000 7f939262c640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f938c145090 0x7f938c1568b0 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 crypto rx=0x7f938c06fa60 tx=0x7f938400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.458+0000 7f938a7fc640 1 -- 172.21.15.115:0/503451157 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93840072d0 con 0x7f938c145090 2023-12-01T03:13:18.526 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.458+0000 7f938a7fc640 1 -- 172.21.15.115:0/503451157 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384007450 con 0x7f938c145090 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.458+0000 7f938a7fc640 1 -- 172.21.15.115:0/503451157 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93840057c0 con 0x7f938c145090 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.458+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c158f60 con 0x7f938c145090 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.459+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c15b4d0 con 0x7f938c145090 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.459+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f938c158830 con 0x7f938c145090 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.460+0000 7f938a7fc640 1 -- 172.21.15.115:0/503451157 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f9384005920 con 0x7f938c145090 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.461+0000 7f938a7fc640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f937006a550 0x7f937006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.461+0000 7f938a7fc640 1 -- 172.21.15.115:0/503451157 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f9384087bc0 con 0x7f938c145090 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.462+0000 7f9391e2b640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f937006a550 0x7f937006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.462+0000 7f938a7fc640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] conn(0x7f9370070a20 0x7f9370072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.462+0000 7f938a7fc640 1 -- 172.21.15.115:0/503451157 --> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93700734f0 con 0x7f9370070a20 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.462+0000 7f938a7fc640 1 -- 172.21.15.115:0/503451157 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938408c280 con 0x7f938c145090 2023-12-01T03:13:18.527 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.462+0000 7f939162a640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] conn(0x7f9370070a20 0x7f9370072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.463+0000 7f9391e2b640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f937006a550 0x7f937006ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9380034000 tx=0x7f9380006000 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.463+0000 7f939162a640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] conn(0x7f9370070a20 0x7f9370072e40 crc :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.464+0000 7f938a7fc640 1 -- 172.21.15.115:0/503451157 <== osd.3 v2:172.21.15.115:6826/88584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f93700734f0 con 0x7f9370070a20 2023-12-01T03:13:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.477+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 --> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f938c0b08f0 con 0x7f9370070a20 2023-12-01T03:13:18.528 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.478+0000 7f938a7fc640 1 -- 172.21.15.115:0/503451157 <== osd.3 v2:172.21.15.115:6826/88584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f93780095c0 con 0x7f9370070a20 2023-12-01T03:13:18.528 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:13:18.528 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-01T03:13:18.528 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:13:18.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.478+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] conn(0x7f9370070a20 msgr2=0x7f9370072e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.478+0000 7f9392e2d640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:6826/88584,v1:172.21.15.115:6827/88584] conn(0x7f9370070a20 0x7f9370072e40 crc :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.478+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f937006a550 msgr2=0x7f937006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.478+0000 7f9392e2d640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f937006a550 0x7f937006ca10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9380034000 tx=0x7f9380006000 comp rx=0 tx=0).stop 2023-12-01T03:13:18.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.478+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f938c145090 msgr2=0x7f938c1568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.478+0000 7f9392e2d640 1 --2- 172.21.15.115:0/503451157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f938c145090 0x7f938c1568b0 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 crypto rx=0x7f938c06fa60 tx=0x7f938400c820 comp rx=0 tx=0).stop 2023-12-01T03:13:18.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.478+0000 7f939262c640 1 -- 172.21.15.115:0/503451157 reap_dead start 2023-12-01T03:13:18.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.478+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 shutdown_connections 2023-12-01T03:13:18.530 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.478+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 >> 172.21.15.115:0/503451157 conn(0x7f938c099a20 msgr2=0x7f938c0cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:18.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.479+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 shutdown_connections 2023-12-01T03:13:18.531 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.479+0000 7f9392e2d640 1 -- 172.21.15.115:0/503451157 wait complete. 2023-12-01T03:13:18.531 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-12-01T03:13:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.747+0000 7fbad7638640 1 Processor -- start 2023-12-01T03:13:18.749 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.747+0000 7fbad7638640 1 -- start start 2023-12-01T03:13:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.749+0000 7fbad7638640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbad0070820 0x7fbad00af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.749+0000 7fbad7638640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbad00cea50 0x7fbad00ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.749+0000 7fbad7638640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad00cf3e0 con 0x7fbad015acf0 2023-12-01T03:13:18.751 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.749+0000 7fbad7638640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad00b01a0 con 0x7fbad0070820 2023-12-01T03:13:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.749+0000 7fbad7638640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad015f670 con 0x7fbad00cea50 2023-12-01T03:13:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.749+0000 7fbad4bd3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbad0070820 0x7fbad00af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.749+0000 7fbad4bd3640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fbad0070820 0x7fbad00af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60952/0 (socket says 172.21.15.115:60952) 2023-12-01T03:13:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.749+0000 7fbad4bd3640 1 -- 172.21.15.115:0/1791136986 learned_addr learned my addr 172.21.15.115:0/1791136986 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.750+0000 7fbad5bd5640 1 --2- 172.21.15.115:0/1791136986 >> v2:172.21.15.142:3300/0 conn(0x7fbad00cea50 0x7fbad00ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.750+0000 7fbad4bd3640 1 -- 172.21.15.115:0/1791136986 >> v2:172.21.15.142:3300/0 conn(0x7fbad00cea50 msgr2=0x7fbad00ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.750+0000 7fbad4bd3640 1 --2- 172.21.15.115:0/1791136986 >> v2:172.21.15.142:3300/0 conn(0x7fbad00cea50 0x7fbad00ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.750+0000 7fbad4bd3640 1 -- 172.21.15.115:0/1791136986 >> v1:172.21.15.115:6789/0 conn(0x7fbad015acf0 legacy=0x7fbad015ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.750+0000 7fbad4bd3640 1 -- 172.21.15.115:0/1791136986 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0070e90 con 0x7fbad0070820 2023-12-01T03:13:18.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.751+0000 7fbad4bd3640 1 --2- 172.21.15.115:0/1791136986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0070820 0x7fbad00af790 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7fbad006d7d0 tx=0x7fbac002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ca67f009b4312e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:18.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.751+0000 7fbac7fff640 1 -- 172.21.15.115:0/1791136986 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0030860 con 0x7fbad0070820 2023-12-01T03:13:18.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.751+0000 7fbac7fff640 1 -- 172.21.15.115:0/1791136986 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac00309c0 con 0x7fbad0070820 2023-12-01T03:13:18.753 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.751+0000 7fbac7fff640 1 -- 172.21.15.115:0/1791136986 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0030d50 con 0x7fbad0070820 2023-12-01T03:13:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.752+0000 7fbad7638640 1 -- 172.21.15.115:0/1791136986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0070820 msgr2=0x7fbad00af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.752+0000 7fbad7638640 1 --2- 172.21.15.115:0/1791136986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0070820 0x7fbad00af790 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7fbad006d7d0 tx=0x7fbac002b9a0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.752+0000 7fbad7638640 1 -- 172.21.15.115:0/1791136986 shutdown_connections 2023-12-01T03:13:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.752+0000 7fbad7638640 1 --2- 172.21.15.115:0/1791136986 >> v2:172.21.15.142:3300/0 conn(0x7fbad00cea50 0x7fbad00ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.752+0000 7fbad7638640 1 --2- 172.21.15.115:0/1791136986 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad0070820 0x7fbad00af790 unknown :-1 s=CLOSED pgs=3143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.752+0000 7fbad7638640 1 -- 172.21.15.115:0/1791136986 >> 172.21.15.115:0/1791136986 conn(0x7fbad0065b40 msgr2=0x7fbad0065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.752+0000 7fbad7638640 1 -- 172.21.15.115:0/1791136986 shutdown_connections 2023-12-01T03:13:18.754 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.752+0000 7fbad7638640 1 -- 172.21.15.115:0/1791136986 wait complete. 2023-12-01T03:13:18.755 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.754+0000 7fbad7638640 1 Processor -- start 2023-12-01T03:13:18.756 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.754+0000 7fbad7638640 1 -- start start 2023-12-01T03:13:18.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.755+0000 7fbad7638640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbad0070820 0x7fbad015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.757 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.755+0000 7fbad7638640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad00cea50 0x7fbad00c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.755+0000 7fbad7638640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad015acf0 0x7fbad00b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.755+0000 7fbad7638640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00c6ed0 con 0x7fbad015acf0 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.755+0000 7fbad7638640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00c6250 con 0x7fbad00cea50 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.755+0000 7fbad7638640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00c6530 con 0x7fbad0070820 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.755+0000 7fbad5bd5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad015acf0 0x7fbad00b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.755+0000 7fbad5bd5640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad015acf0 0x7fbad00b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60968/0 (socket says 172.21.15.115:60968) 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.755+0000 7fbad5bd5640 1 -- 172.21.15.115:0/3964077653 learned_addr learned my addr 172.21.15.115:0/3964077653 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.755+0000 7fbad53d4640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbad0070820 0x7fbad015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbad4bd3640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad00cea50 0x7fbad00c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbad5bd5640 1 -- 172.21.15.115:0/3964077653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbad0070820 msgr2=0x7fbad015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbad5bd5640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbad0070820 0x7fbad015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbad5bd5640 1 -- 172.21.15.115:0/3964077653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad00cea50 msgr2=0x7fbad00c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.758 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbad5bd5640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad00cea50 0x7fbad00c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbad5bd5640 1 -- 172.21.15.115:0/3964077653 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac00094d0 con 0x7fbad015acf0 2023-12-01T03:13:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbad5bd5640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad015acf0 0x7fbad00b25a0 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7fbac8007e40 tx=0x7fbac800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbac5ffb640 1 -- 172.21.15.115:0/3964077653 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8014620 con 0x7fbad015acf0 2023-12-01T03:13:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbac5ffb640 1 -- 172.21.15.115:0/3964077653 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac800ee90 con 0x7fbad015acf0 2023-12-01T03:13:18.759 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbad7638640 1 -- 172.21.15.115:0/3964077653 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad00c6810 con 0x7fbad015acf0 2023-12-01T03:13:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbac5ffb640 1 -- 172.21.15.115:0/3964077653 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac80149e0 con 0x7fbad015acf0 2023-12-01T03:13:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.756+0000 7fbad7638640 1 -- 172.21.15.115:0/3964077653 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad015e030 con 0x7fbad015acf0 2023-12-01T03:13:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.758+0000 7fbad7638640 1 -- 172.21.15.115:0/3964077653 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad00c7040 con 0x7fbad015acf0 2023-12-01T03:13:18.760 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.758+0000 7fbac5ffb640 1 -- 172.21.15.115:0/3964077653 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fbac8014b40 con 0x7fbad015acf0 2023-12-01T03:13:18.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.759+0000 7fbac5ffb640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbab406a550 0x7fbab406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:18.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.759+0000 7fbac5ffb640 1 -- 172.21.15.115:0/3964077653 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7fbac8089460 con 0x7fbad015acf0 2023-12-01T03:13:18.761 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.760+0000 7fbad53d4640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbab406a550 0x7fbab406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:18.762 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.760+0000 7fbad53d4640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbab406a550 0x7fbab406ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbad01551b0 tx=0x7fbacc009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:18.765 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.764+0000 7fbac5ffb640 1 -- 172.21.15.115:0/3964077653 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fbac8054920 con 0x7fbad015acf0 2023-12-01T03:13:18.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.953+0000 7fbad7638640 1 -- 172.21.15.115:0/3964077653 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbad00c6d30 con 0x7fbab406a550 2023-12-01T03:13:18.956 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.954+0000 7fbac5ffb640 1 -- 172.21.15.115:0/3964077653 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbacc001c70 con 0x7fbab406a550 2023-12-01T03:13:18.956 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 3 to deep-scrub 2023-12-01T03:13:18.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 -- 172.21.15.115:0/3964077653 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbab406a550 msgr2=0x7fbab406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbab406a550 0x7fbab406ca10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbad01551b0 tx=0x7fbacc009040 comp rx=0 tx=0).stop 2023-12-01T03:13:18.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 -- 172.21.15.115:0/3964077653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad015acf0 msgr2=0x7fbad00b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:18.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad015acf0 0x7fbad00b25a0 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7fbac8007e40 tx=0x7fbac800e4b0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 -- 172.21.15.115:0/3964077653 shutdown_connections 2023-12-01T03:13:18.959 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fbab406a550 0x7fbab406ca10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbad015acf0 0x7fbad00b25a0 unknown :-1 s=CLOSED pgs=3144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad00cea50 0x7fbad00c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 --2- 172.21.15.115:0/3964077653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbad0070820 0x7fbad015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:18.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 -- 172.21.15.115:0/3964077653 >> 172.21.15.115:0/3964077653 conn(0x7fbad0065b40 msgr2=0x7fbad00c36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:18.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 -- 172.21.15.115:0/3964077653 shutdown_connections 2023-12-01T03:13:18.960 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:18.957+0000 7fbad7638640 1 -- 172.21.15.115:0/3964077653 wait complete. 2023-12-01T03:13:18.978 INFO:tasks.ceph:Scrubbing osd.4 2023-12-01T03:13:18.978 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-12-01T03:13:19.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.190+0000 7f89d13ae640 1 Processor -- start 2023-12-01T03:13:19.191 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.190+0000 7f89d13ae640 1 -- start start 2023-12-01T03:13:19.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.191+0000 7f89d13ae640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f89cc0cb8c0 0x7f89cc0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.191+0000 7f89d13ae640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f89cc0b4070 0x7f89cc0cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.191+0000 7f89d13ae640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89cc06c860 con 0x7f89cc15ac70 2023-12-01T03:13:19.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.191+0000 7f89d13ae640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc15b050 con 0x7f89cc0b4070 2023-12-01T03:13:19.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.191+0000 7f89d13ae640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc0ce0b0 con 0x7f89cc0cb8c0 2023-12-01T03:13:19.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.191+0000 7f89cb7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f89cc0b4070 0x7f89cc0cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.191+0000 7f89cb7fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f89cc0b4070 0x7f89cc0cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60976/0 (socket says 172.21.15.115:60976) 2023-12-01T03:13:19.193 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.191+0000 7f89cb7fe640 1 -- 172.21.15.115:0/1966616422 learned_addr learned my addr 172.21.15.115:0/1966616422 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.192+0000 7f89ca7fc640 1 --2- 172.21.15.115:0/1966616422 >> v2:172.21.15.142:3300/0 conn(0x7f89cc0cb8c0 0x7f89cc0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.192+0000 7f89cb7fe640 1 -- 172.21.15.115:0/1966616422 >> v2:172.21.15.142:3300/0 conn(0x7f89cc0cb8c0 msgr2=0x7f89cc0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.192+0000 7f89cb7fe640 1 --2- 172.21.15.115:0/1966616422 >> v2:172.21.15.142:3300/0 conn(0x7f89cc0cb8c0 0x7f89cc0cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.192+0000 7f89cb7fe640 1 -- 172.21.15.115:0/1966616422 >> v1:172.21.15.142:6789/0 conn(0x7f89cc15ac70 legacy=0x7f89cc0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.192+0000 7f89cb7fe640 1 -- 172.21.15.115:0/1966616422 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0ce390 con 0x7f89cc0b4070 2023-12-01T03:13:19.194 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.192+0000 7f89ca7fc640 1 --2- 172.21.15.115:0/1966616422 >> v2:172.21.15.142:3300/0 conn(0x7f89cc0cb8c0 0x7f89cc0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.193+0000 7f89cb7fe640 1 --2- 172.21.15.115:0/1966616422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89cc0b4070 0x7f89cc0cc8b0 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0cbe60 tx=0x7f89c402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff838adf2c1a5b56 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.193+0000 7f89c9ffb640 1 -- 172.21.15.115:0/1966616422 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c402c9f0 con 0x7f89cc0b4070 2023-12-01T03:13:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.193+0000 7f89c9ffb640 1 -- 172.21.15.115:0/1966616422 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c402cb50 con 0x7f89cc0b4070 2023-12-01T03:13:19.195 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.193+0000 7f89c9ffb640 1 -- 172.21.15.115:0/1966616422 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c40343f0 con 0x7f89cc0b4070 2023-12-01T03:13:19.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.194+0000 7f89d13ae640 1 -- 172.21.15.115:0/1966616422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89cc0b4070 msgr2=0x7f89cc0cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.194+0000 7f89d13ae640 1 --2- 172.21.15.115:0/1966616422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89cc0b4070 0x7f89cc0cc8b0 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0cbe60 tx=0x7f89c402be50 comp rx=0 tx=0).stop 2023-12-01T03:13:19.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.194+0000 7f89d13ae640 1 -- 172.21.15.115:0/1966616422 shutdown_connections 2023-12-01T03:13:19.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.194+0000 7f89d13ae640 1 --2- 172.21.15.115:0/1966616422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89cc0b4070 0x7f89cc0cc8b0 unknown :-1 s=CLOSED pgs=3145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.194+0000 7f89d13ae640 1 --2- 172.21.15.115:0/1966616422 >> v2:172.21.15.142:3300/0 conn(0x7f89cc0cb8c0 0x7f89cc0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.194+0000 7f89d13ae640 1 -- 172.21.15.115:0/1966616422 >> 172.21.15.115:0/1966616422 conn(0x7f89cc0662c0 msgr2=0x7f89cc0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:19.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.194+0000 7f89d13ae640 1 -- 172.21.15.115:0/1966616422 shutdown_connections 2023-12-01T03:13:19.196 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.194+0000 7f89d13ae640 1 -- 172.21.15.115:0/1966616422 wait complete. 2023-12-01T03:13:19.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.196+0000 7f89d13ae640 1 Processor -- start 2023-12-01T03:13:19.198 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.196+0000 7f89d13ae640 1 -- start start 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.197+0000 7f89d13ae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89cc0b4070 0x7f89cc1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.197+0000 7f89d13ae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89cc0cb8c0 0x7f89cc0c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.197+0000 7f89d13ae640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89cc15ac70 0x7f89cc0c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.197+0000 7f89d13ae640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc158c00 con 0x7f89cc15ac70 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.197+0000 7f89d13ae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc15d030 con 0x7f89cc0b4070 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.197+0000 7f89d13ae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc15d2d0 con 0x7f89cc0cb8c0 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.198+0000 7f89cb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89cc15ac70 0x7f89cc0c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.198+0000 7f89cb7fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89cc15ac70 0x7f89cc0c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60988/0 (socket says 172.21.15.115:60988) 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.198+0000 7f89cb7fe640 1 -- 172.21.15.115:0/1815211240 learned_addr learned my addr 172.21.15.115:0/1815211240 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.198+0000 7f89caffd640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89cc0b4070 0x7f89cc1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.200 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.198+0000 7f89cb7fe640 1 -- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89cc0cb8c0 msgr2=0x7f89cc0c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.198+0000 7f89ca7fc640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89cc0cb8c0 0x7f89cc0c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.198+0000 7f89cb7fe640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89cc0cb8c0 0x7f89cc0c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.198+0000 7f89cb7fe640 1 -- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89cc0b4070 msgr2=0x7f89cc1586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.198+0000 7f89cb7fe640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89cc0b4070 0x7f89cc1586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.201 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.198+0000 7f89cb7fe640 1 -- 172.21.15.115:0/1815211240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c4009b30 con 0x7f89cc15ac70 2023-12-01T03:13:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.199+0000 7f89cb7fe640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89cc15ac70 0x7f89cc0c7a40 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7f89c4036040 tx=0x7f89c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.199+0000 7f89b37fe640 1 -- 172.21.15.115:0/1815211240 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c403e450 con 0x7f89cc15ac70 2023-12-01T03:13:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.199+0000 7f89b37fe640 1 -- 172.21.15.115:0/1815211240 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c403e5d0 con 0x7f89cc15ac70 2023-12-01T03:13:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.199+0000 7f89b37fe640 1 -- 172.21.15.115:0/1815211240 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c403e980 con 0x7f89cc15ac70 2023-12-01T03:13:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.199+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc15d5b0 con 0x7f89cc15ac70 2023-12-01T03:13:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.199+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc15b190 con 0x7f89cc15ac70 2023-12-01T03:13:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.200+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89cc06cde0 con 0x7f89cc15ac70 2023-12-01T03:13:19.202 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.201+0000 7f89b37fe640 1 -- 172.21.15.115:0/1815211240 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f89c403eae0 con 0x7f89cc15ac70 2023-12-01T03:13:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.202+0000 7f89b37fe640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89b806a550 0x7f89b806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.202+0000 7f89b37fe640 1 -- 172.21.15.115:0/1815211240 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f89c40ab980 con 0x7f89cc15ac70 2023-12-01T03:13:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.202+0000 7f89caffd640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89b806a550 0x7f89b806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.202+0000 7f89b37fe640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] conn(0x7f89b8070a20 0x7f89b8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.202+0000 7f89b37fe640 1 -- 172.21.15.115:0/1815211240 --> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89b80734f0 con 0x7f89b8070a20 2023-12-01T03:13:19.204 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.202+0000 7f89b37fe640 1 -- 172.21.15.115:0/1815211240 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89c40b02a0 con 0x7f89cc15ac70 2023-12-01T03:13:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.203+0000 7f89caffd640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89b806a550 0x7f89b806ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89bc007fd0 tx=0x7f89bc009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.203+0000 7f89ca7fc640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] conn(0x7f89b8070a20 0x7f89b8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.205 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.204+0000 7f89ca7fc640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] conn(0x7f89b8070a20 0x7f89b8072e40 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.207 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.207+0000 7f89b37fe640 1 -- 172.21.15.115:0/1815211240 <== osd.4 v2:172.21.15.142:6800/88936 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f89b80734f0 con 0x7f89b8070a20 2023-12-01T03:13:19.221 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.220+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 --> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f89cc06cb00 con 0x7f89b8070a20 2023-12-01T03:13:19.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.220+0000 7f89b37fe640 1 -- 172.21.15.115:0/1815211240 <== osd.4 v2:172.21.15.142:6800/88936 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f89c000c040 con 0x7f89b8070a20 2023-12-01T03:13:19.222 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:13:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-01T03:13:19.222 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:13:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] conn(0x7f89b8070a20 msgr2=0x7f89b8072e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89d13ae640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.142:6800/88936,v1:172.21.15.142:6801/88936] conn(0x7f89b8070a20 0x7f89b8072e40 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89b806a550 msgr2=0x7f89b806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89d13ae640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89b806a550 0x7f89b806ca10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f89bc007fd0 tx=0x7f89bc009040 comp rx=0 tx=0).stop 2023-12-01T03:13:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89cc15ac70 msgr2=0x7f89cc0c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89d13ae640 1 --2- 172.21.15.115:0/1815211240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89cc15ac70 0x7f89cc0c7a40 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7f89c4036040 tx=0x7f89c4004a40 comp rx=0 tx=0).stop 2023-12-01T03:13:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89cb7fe640 1 -- 172.21.15.115:0/1815211240 reap_dead start 2023-12-01T03:13:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 shutdown_connections 2023-12-01T03:13:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 >> 172.21.15.115:0/1815211240 conn(0x7f89cc0662c0 msgr2=0x7f89cc0b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 shutdown_connections 2023-12-01T03:13:19.224 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.221+0000 7f89d13ae640 1 -- 172.21.15.115:0/1815211240 wait complete. 2023-12-01T03:13:19.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-12-01T03:13:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.449+0000 7ff3f8c1e640 1 Processor -- start 2023-12-01T03:13:19.451 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.449+0000 7ff3f8c1e640 1 -- start start 2023-12-01T03:13:19.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f8c1e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3f415cda0 0x7ff3f406c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f8c1e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff3f40cea50 0x7ff3f40ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f8c1e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f40cd3b0 con 0x7ff3f4160970 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f8c1e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f40cf420 con 0x7ff3f415cda0 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f8c1e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f406c970 con 0x7ff3f40cea50 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3f415cda0 0x7ff3f406c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f259c640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff3f415cda0 0x7ff3f406c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:60996/0 (socket says 172.21.15.115:60996) 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f259c640 1 -- 172.21.15.115:0/3635985720 learned_addr learned my addr 172.21.15.115:0/3635985720 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f1d9b640 1 --2- 172.21.15.115:0/3635985720 >> v2:172.21.15.142:3300/0 conn(0x7ff3f40cea50 0x7ff3f40ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f259c640 1 -- 172.21.15.115:0/3635985720 >> v2:172.21.15.142:3300/0 conn(0x7ff3f40cea50 msgr2=0x7ff3f40ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f259c640 1 --2- 172.21.15.115:0/3635985720 >> v2:172.21.15.142:3300/0 conn(0x7ff3f40cea50 0x7ff3f40ceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f259c640 1 -- 172.21.15.115:0/3635985720 >> v1:172.21.15.142:6789/0 conn(0x7ff3f4160970 legacy=0x7ff3f4160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:19.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.452+0000 7ff3f259c640 1 -- 172.21.15.115:0/3635985720 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f40cb530 con 0x7ff3f415cda0 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.453+0000 7ff3f259c640 1 --2- 172.21.15.115:0/3635985720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f415cda0 0x7ff3f406c430 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40cc250 tx=0x7ff3e402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348f7f7f9fb9d291 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.453+0000 7ff3f159a640 1 -- 172.21.15.115:0/3635985720 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40317e0 con 0x7ff3f415cda0 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.453+0000 7ff3f159a640 1 -- 172.21.15.115:0/3635985720 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e4031940 con 0x7ff3f415cda0 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.453+0000 7ff3f159a640 1 -- 172.21.15.115:0/3635985720 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4031cd0 con 0x7ff3f415cda0 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.454+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/3635985720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f415cda0 msgr2=0x7ff3f406c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.454+0000 7ff3f8c1e640 1 --2- 172.21.15.115:0/3635985720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f415cda0 0x7ff3f406c430 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40cc250 tx=0x7ff3e402b880 comp rx=0 tx=0).stop 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.454+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/3635985720 shutdown_connections 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.454+0000 7ff3f8c1e640 1 --2- 172.21.15.115:0/3635985720 >> v2:172.21.15.142:3300/0 conn(0x7ff3f40cea50 0x7ff3f40ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.454+0000 7ff3f8c1e640 1 --2- 172.21.15.115:0/3635985720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f415cda0 0x7ff3f406c430 unknown :-1 s=CLOSED pgs=3147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.454+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/3635985720 >> 172.21.15.115:0/3635985720 conn(0x7ff3f4065b40 msgr2=0x7ff3f4065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.454+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/3635985720 shutdown_connections 2023-12-01T03:13:19.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.454+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/3635985720 wait complete. 2023-12-01T03:13:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.456+0000 7ff3f8c1e640 1 Processor -- start 2023-12-01T03:13:19.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.456+0000 7ff3f8c1e640 1 -- start start 2023-12-01T03:13:19.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.457+0000 7ff3f8c1e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f40cea50 0x7ff3f40af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.457+0000 7ff3f8c1e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3f415cda0 0x7ff3f41f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.457+0000 7ff3f8c1e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f4160970 0x7ff3f40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.457+0000 7ff3f8c1e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40ac9a0 con 0x7ff3f40cea50 2023-12-01T03:13:19.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.457+0000 7ff3f8c1e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40c58e0 con 0x7ff3f4160970 2023-12-01T03:13:19.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.457+0000 7ff3f8c1e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f41f3030 con 0x7ff3f415cda0 2023-12-01T03:13:19.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.457+0000 7ff3f259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f40cea50 0x7ff3f40af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.457+0000 7ff3f259c640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f40cea50 0x7ff3f40af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32772/0 (socket says 172.21.15.115:32772) 2023-12-01T03:13:19.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.457+0000 7ff3f259c640 1 -- 172.21.15.115:0/360568131 learned_addr learned my addr 172.21.15.115:0/360568131 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3f2d9d640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f4160970 0x7ff3f40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3f1d9b640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3f415cda0 0x7ff3f41f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3f259c640 1 -- 172.21.15.115:0/360568131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3f415cda0 msgr2=0x7ff3f41f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3f259c640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3f415cda0 0x7ff3f41f2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3f259c640 1 -- 172.21.15.115:0/360568131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f4160970 msgr2=0x7ff3f40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3f259c640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f4160970 0x7ff3f40b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3f259c640 1 -- 172.21.15.115:0/360568131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e40094d0 con 0x7ff3f40cea50 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3f2d9d640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f4160970 0x7ff3f40b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3f259c640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f40cea50 0x7ff3f40af700 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000c00 tx=0x7ff3e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3e2ffd640 1 -- 172.21.15.115:0/360568131 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40329c0 con 0x7ff3f40cea50 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3e2ffd640 1 -- 172.21.15.115:0/360568131 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e4032b40 con 0x7ff3f40cea50 2023-12-01T03:13:19.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3e2ffd640 1 -- 172.21.15.115:0/360568131 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40453f0 con 0x7ff3f40cea50 2023-12-01T03:13:19.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.458+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/360568131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f41f3310 con 0x7ff3f40cea50 2023-12-01T03:13:19.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.459+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/360568131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f40c5b80 con 0x7ff3f40cea50 2023-12-01T03:13:19.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.460+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/360568131 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f404e890 con 0x7ff3f40cea50 2023-12-01T03:13:19.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.461+0000 7ff3e2ffd640 1 -- 172.21.15.115:0/360568131 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff3e404d3f0 con 0x7ff3f40cea50 2023-12-01T03:13:19.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.462+0000 7ff3e2ffd640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3d406a550 0x7ff3d406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.462+0000 7ff3e2ffd640 1 -- 172.21.15.115:0/360568131 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7ff3e40aab60 con 0x7ff3f40cea50 2023-12-01T03:13:19.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.462+0000 7ff3f1d9b640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3d406a550 0x7ff3d406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.463+0000 7ff3f1d9b640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3d406a550 0x7ff3d406ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8006fd0 tx=0x7ff3e8008040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.467 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.467+0000 7ff3e2ffd640 1 -- 172.21.15.115:0/360568131 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7ff3e4075f40 con 0x7ff3f40cea50 2023-12-01T03:13:19.658 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.656+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/360568131 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3f40cca80 con 0x7ff3d406a550 2023-12-01T03:13:19.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.657+0000 7ff3e2ffd640 1 -- 172.21.15.115:0/360568131 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff3e800c040 con 0x7ff3d406a550 2023-12-01T03:13:19.661 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 4 to deep-scrub 2023-12-01T03:13:19.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/360568131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3d406a550 msgr2=0x7ff3d406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3d406a550 0x7ff3d406ca10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8006fd0 tx=0x7ff3e8008040 comp rx=0 tx=0).stop 2023-12-01T03:13:19.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/360568131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f40cea50 msgr2=0x7ff3f40af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f40cea50 0x7ff3f40af700 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000c00 tx=0x7ff3e40046c0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.662 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/360568131 shutdown_connections 2023-12-01T03:13:19.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff3d406a550 0x7ff3d406ca10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff3f415cda0 0x7ff3f41f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3f4160970 0x7ff3f40b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 --2- 172.21.15.115:0/360568131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3f40cea50 0x7ff3f40af700 unknown :-1 s=CLOSED pgs=3148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/360568131 >> 172.21.15.115:0/360568131 conn(0x7ff3f4065b40 msgr2=0x7ff3f415c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:19.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/360568131 shutdown_connections 2023-12-01T03:13:19.663 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.660+0000 7ff3f8c1e640 1 -- 172.21.15.115:0/360568131 wait complete. 2023-12-01T03:13:19.681 INFO:tasks.ceph:Scrubbing osd.5 2023-12-01T03:13:19.681 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-12-01T03:13:19.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.900+0000 7f7fab12e640 1 Processor -- start 2023-12-01T03:13:19.901 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.900+0000 7f7fab12e640 1 -- start start 2023-12-01T03:13:19.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.901+0000 7f7fab12e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa40ca7c0 0x7f7fa40cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.901+0000 7f7fab12e640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7fa40cb6f0 0x7f7fa406f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.901+0000 7f7fab12e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40c9440 con 0x7f7fa4159bf0 2023-12-01T03:13:19.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.901+0000 7f7fab12e640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa4159fd0 con 0x7f7fa40cb6f0 2023-12-01T03:13:19.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.901+0000 7f7fab12e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40cbb80 con 0x7f7fa40ca7c0 2023-12-01T03:13:19.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.901+0000 7f7fa96cb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7fa40cb6f0 0x7f7fa406f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.901+0000 7f7fa96cb640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f7fa40cb6f0 0x7f7fa406f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32784/0 (socket says 172.21.15.115:32784) 2023-12-01T03:13:19.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.901+0000 7f7fa96cb640 1 -- 172.21.15.115:0/2406212011 learned_addr learned my addr 172.21.15.115:0/2406212011 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:19.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.902+0000 7f7fa3fff640 1 --2- 172.21.15.115:0/2406212011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa40ca7c0 0x7f7fa40cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.902+0000 7f7fa96cb640 1 -- 172.21.15.115:0/2406212011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa40ca7c0 msgr2=0x7f7fa40cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:19.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.902+0000 7f7fa96cb640 1 --2- 172.21.15.115:0/2406212011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa40ca7c0 0x7f7fa40cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.902+0000 7f7fa96cb640 1 -- 172.21.15.115:0/2406212011 >> v1:172.21.15.142:6789/0 conn(0x7f7fa4159bf0 legacy=0x7f7fa415e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.902+0000 7f7fa96cb640 1 -- 172.21.15.115:0/2406212011 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40c9720 con 0x7f7fa40cb6f0 2023-12-01T03:13:19.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.903+0000 7f7fa96cb640 1 --2- 172.21.15.115:0/2406212011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40cb6f0 0x7f7fa406f650 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009e50 tx=0x7f7f9c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee07d6d4d15b8aa server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.903+0000 7f7fa37fe640 1 -- 172.21.15.115:0/2406212011 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c02cb00 con 0x7f7fa40cb6f0 2023-12-01T03:13:19.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.903+0000 7f7fa37fe640 1 -- 172.21.15.115:0/2406212011 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c02cc60 con 0x7f7fa40cb6f0 2023-12-01T03:13:19.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.903+0000 7f7fa37fe640 1 -- 172.21.15.115:0/2406212011 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0343f0 con 0x7f7fa40cb6f0 2023-12-01T03:13:19.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.904+0000 7f7fab12e640 1 -- 172.21.15.115:0/2406212011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40cb6f0 msgr2=0x7f7fa406f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.904+0000 7f7fab12e640 1 --2- 172.21.15.115:0/2406212011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40cb6f0 0x7f7fa406f650 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009e50 tx=0x7f7f9c02bf60 comp rx=0 tx=0).stop 2023-12-01T03:13:19.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.904+0000 7f7fab12e640 1 -- 172.21.15.115:0/2406212011 shutdown_connections 2023-12-01T03:13:19.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.904+0000 7f7fab12e640 1 --2- 172.21.15.115:0/2406212011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa40cb6f0 0x7f7fa406f650 unknown :-1 s=CLOSED pgs=3149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.904+0000 7f7fab12e640 1 --2- 172.21.15.115:0/2406212011 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa40ca7c0 0x7f7fa40cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.904+0000 7f7fab12e640 1 -- 172.21.15.115:0/2406212011 >> 172.21.15.115:0/2406212011 conn(0x7f7fa40662c0 msgr2=0x7f7fa40b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:19.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.904+0000 7f7fab12e640 1 -- 172.21.15.115:0/2406212011 shutdown_connections 2023-12-01T03:13:19.907 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.904+0000 7f7fab12e640 1 -- 172.21.15.115:0/2406212011 wait complete. 2023-12-01T03:13:19.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.906+0000 7f7fab12e640 1 Processor -- start 2023-12-01T03:13:19.908 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.906+0000 7f7fab12e640 1 -- start start 2023-12-01T03:13:19.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.907+0000 7f7fab12e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa40ca7c0 0x7f7fa40c2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.907+0000 7f7fab12e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa40cb6f0 0x7f7fa40c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.907+0000 7f7fab12e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa4159bf0 0x7f7fa40c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.907+0000 7f7fab12e640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40c4da0 con 0x7f7fa4159bf0 2023-12-01T03:13:19.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.907+0000 7f7fab12e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40c5080 con 0x7f7fa40ca7c0 2023-12-01T03:13:19.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.907+0000 7f7fab12e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40c61d0 con 0x7f7fa40cb6f0 2023-12-01T03:13:19.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.907+0000 7f7fa96cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa4159bf0 0x7f7fa40c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.907+0000 7f7fa96cb640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa4159bf0 0x7f7fa40c6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32786/0 (socket says 172.21.15.115:32786) 2023-12-01T03:13:19.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.907+0000 7f7fa96cb640 1 -- 172.21.15.115:0/1185360681 learned_addr learned my addr 172.21.15.115:0/1185360681 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.908+0000 7f7fa96cb640 1 -- 172.21.15.115:0/1185360681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa40cb6f0 msgr2=0x7f7fa40c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.908+0000 7f7fa96cb640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa40cb6f0 0x7f7fa40c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.908+0000 7f7fa96cb640 1 -- 172.21.15.115:0/1185360681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa40ca7c0 msgr2=0x7f7fa40c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.908+0000 7f7fa96cb640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fa40ca7c0 0x7f7fa40c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.908+0000 7f7fa96cb640 1 -- 172.21.15.115:0/1185360681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c009b30 con 0x7f7fa4159bf0 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.908+0000 7f7fa96cb640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa4159bf0 0x7f7fa40c6cb0 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c036040 tx=0x7f7f9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.908+0000 7f7fa17fa640 1 -- 172.21.15.115:0/1185360681 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c009510 con 0x7f7fa4159bf0 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.908+0000 7f7fa17fa640 1 -- 172.21.15.115:0/1185360681 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c03e450 con 0x7f7fa4159bf0 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.908+0000 7f7fa17fa640 1 -- 172.21.15.115:0/1185360681 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c03e7e0 con 0x7f7fa4159bf0 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.908+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa40c64b0 con 0x7f7fa4159bf0 2023-12-01T03:13:19.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.909+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa414df20 con 0x7f7fa4159bf0 2023-12-01T03:13:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.909+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7fa40c9ce0 con 0x7f7fa4159bf0 2023-12-01T03:13:19.912 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.911+0000 7f7fa17fa640 1 -- 172.21.15.115:0/1185360681 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f7f9c03e940 con 0x7f7fa4159bf0 2023-12-01T03:13:19.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.911+0000 7f7fa17fa640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f8806a550 0x7f7f8806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.912+0000 7f7fa17fa640 1 -- 172.21.15.115:0/1185360681 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f7f9c0ab870 con 0x7f7fa4159bf0 2023-12-01T03:13:19.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.912+0000 7f7fa8eca640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f8806a550 0x7f7f8806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.912+0000 7f7fa17fa640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] conn(0x7f7f88070a20 0x7f7f88072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:19.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.912+0000 7f7fa17fa640 1 -- 172.21.15.115:0/1185360681 --> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f880734f0 con 0x7f7f88070a20 2023-12-01T03:13:19.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.912+0000 7f7fa17fa640 1 -- 172.21.15.115:0/1185360681 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f9c0b02c0 con 0x7f7fa4159bf0 2023-12-01T03:13:19.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.912+0000 7f7fa8eca640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f8806a550 0x7f7f8806ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7f98000f60 tx=0x7f7f980079b0 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.914 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.913+0000 7f7fa3fff640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] conn(0x7f7f88070a20 0x7f7f88072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:19.915 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.914+0000 7f7fa3fff640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] conn(0x7f7f88070a20 0x7f7f88072e40 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:19.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.916+0000 7f7fa17fa640 1 -- 172.21.15.115:0/1185360681 <== osd.5 v2:172.21.15.142:6816/88938 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7f880734f0 con 0x7f7f88070a20 2023-12-01T03:13:19.932 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.930+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 --> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7fa40c9a00 con 0x7f7f88070a20 2023-12-01T03:13:19.933 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.931+0000 7f7fa17fa640 1 -- 172.21.15.115:0/1185360681 <== osd.5 v2:172.21.15.142:6816/88938 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7f90009a20 con 0x7f7f88070a20 2023-12-01T03:13:19.933 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:13:19.933 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-01T03:13:19.934 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 >> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] conn(0x7f7f88070a20 msgr2=0x7f7f88072e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fab12e640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.142:6816/88938,v1:172.21.15.142:6818/88938] conn(0x7f7f88070a20 0x7f7f88072e40 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f8806a550 msgr2=0x7f7f8806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fab12e640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f7f8806a550 0x7f7f8806ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7f98000f60 tx=0x7f7f980079b0 comp rx=0 tx=0).stop 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa4159bf0 msgr2=0x7f7fa40c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fab12e640 1 --2- 172.21.15.115:0/1185360681 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7fa4159bf0 0x7f7fa40c6cb0 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c036040 tx=0x7f7f9c004a40 comp rx=0 tx=0).stop 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fa96cb640 1 -- 172.21.15.115:0/1185360681 reap_dead start 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 shutdown_connections 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 >> 172.21.15.115:0/1185360681 conn(0x7f7fa40662c0 msgr2=0x7f7fa4158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 shutdown_connections 2023-12-01T03:13:19.935 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:19.933+0000 7f7fab12e640 1 -- 172.21.15.115:0/1185360681 wait complete. 2023-12-01T03:13:19.951 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-12-01T03:13:20.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.152+0000 7f789b615640 1 Processor -- start 2023-12-01T03:13:20.153 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.153+0000 7f789b615640 1 -- start start 2023-12-01T03:13:20.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.154+0000 7f789b615640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f789415acf0 0x7f7894070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.154+0000 7f789b615640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940af790 0x7f789415ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.154+0000 7f789b615640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7894070e90 con 0x7f78940cea50 2023-12-01T03:13:20.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.154+0000 7f789b615640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78940b01a0 con 0x7f789415acf0 2023-12-01T03:13:20.155 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.154+0000 7f789b615640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7894071170 con 0x7f78940af790 2023-12-01T03:13:20.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.155+0000 7f78993b1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f789415acf0 0x7f7894070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.155+0000 7f78993b1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f789415acf0 0x7f7894070820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32792/0 (socket says 172.21.15.115:32792) 2023-12-01T03:13:20.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.155+0000 7f78993b1640 1 -- 172.21.15.115:0/3311379798 learned_addr learned my addr 172.21.15.115:0/3311379798 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:20.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.155+0000 7f7898bb0640 1 --2- 172.21.15.115:0/3311379798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940af790 0x7f789415ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.155+0000 7f78993b1640 1 -- 172.21.15.115:0/3311379798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940af790 msgr2=0x7f789415ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:20.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.155+0000 7f78993b1640 1 --2- 172.21.15.115:0/3311379798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940af790 0x7f789415ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.156 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.155+0000 7f78993b1640 1 -- 172.21.15.115:0/3311379798 >> v1:172.21.15.142:6789/0 conn(0x7f78940cea50 legacy=0x7f78940c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:20.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.155+0000 7f78993b1640 1 -- 172.21.15.115:0/3311379798 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789415e760 con 0x7f789415acf0 2023-12-01T03:13:20.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.156+0000 7f78993b1640 1 --2- 172.21.15.115:0/3311379798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 0x7f7894070820 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 crypto rx=0x7f7890009870 tx=0x7f789002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943f79c855ad3111 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:20.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.157+0000 7f788bfff640 1 -- 172.21.15.115:0/3311379798 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890031640 con 0x7f789415acf0 2023-12-01T03:13:20.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.157+0000 7f788bfff640 1 -- 172.21.15.115:0/3311379798 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78900317a0 con 0x7f789415acf0 2023-12-01T03:13:20.157 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.157+0000 7f788bfff640 1 -- 172.21.15.115:0/3311379798 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890031b30 con 0x7f789415acf0 2023-12-01T03:13:20.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.157+0000 7f789b615640 1 -- 172.21.15.115:0/3311379798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 msgr2=0x7f7894070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.157+0000 7f789b615640 1 --2- 172.21.15.115:0/3311379798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 0x7f7894070820 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 crypto rx=0x7f7890009870 tx=0x7f789002b8f0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.157+0000 7f789b615640 1 -- 172.21.15.115:0/3311379798 shutdown_connections 2023-12-01T03:13:20.158 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.157+0000 7f789b615640 1 --2- 172.21.15.115:0/3311379798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940af790 0x7f789415ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.157+0000 7f789b615640 1 --2- 172.21.15.115:0/3311379798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 0x7f7894070820 unknown :-1 s=CLOSED pgs=3151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.157+0000 7f789b615640 1 -- 172.21.15.115:0/3311379798 >> 172.21.15.115:0/3311379798 conn(0x7f7894065b40 msgr2=0x7f7894065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:20.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.157+0000 7f789b615640 1 -- 172.21.15.115:0/3311379798 shutdown_connections 2023-12-01T03:13:20.159 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.158+0000 7f789b615640 1 -- 172.21.15.115:0/3311379798 wait complete. 2023-12-01T03:13:20.160 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.160+0000 7f789b615640 1 Processor -- start 2023-12-01T03:13:20.161 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.160+0000 7f789b615640 1 -- start start 2023-12-01T03:13:20.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.161+0000 7f789b615640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78940af790 0x7f78940c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.161+0000 7f789b615640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940cea50 0x7f78940addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.161+0000 7f789b615640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 0x7f78940c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.161+0000 7f789b615640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940c71c0 con 0x7f789415acf0 2023-12-01T03:13:20.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.161+0000 7f789b615640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940c74a0 con 0x7f78940af790 2023-12-01T03:13:20.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.161+0000 7f789b615640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78940c7780 con 0x7f78940cea50 2023-12-01T03:13:20.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.161+0000 7f7899bb2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 0x7f78940c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.163 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.161+0000 7f7899bb2640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 0x7f78940c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32798/0 (socket says 172.21.15.115:32798) 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.161+0000 7f7899bb2640 1 -- 172.21.15.115:0/333918160 learned_addr learned my addr 172.21.15.115:0/333918160 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.161+0000 7f78993b1640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78940af790 0x7f78940c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7898bb0640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940cea50 0x7f78940addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7899bb2640 1 -- 172.21.15.115:0/333918160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940cea50 msgr2=0x7f78940addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7899bb2640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940cea50 0x7f78940addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7899bb2640 1 -- 172.21.15.115:0/333918160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78940af790 msgr2=0x7f78940c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7899bb2640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78940af790 0x7f78940c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7899bb2640 1 -- 172.21.15.115:0/333918160 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78900094d0 con 0x7f789415acf0 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7898bb0640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940cea50 0x7f78940addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7899bb2640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 0x7f78940c9fb0 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 crypto rx=0x7f788c00c6b0 tx=0x7f788c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7889ffb640 1 -- 172.21.15.115:0/333918160 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c003b70 con 0x7f789415acf0 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7889ffb640 1 -- 172.21.15.115:0/333918160 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c003cf0 con 0x7f789415acf0 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f789b615640 1 -- 172.21.15.115:0/333918160 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78940c7a60 con 0x7f789415acf0 2023-12-01T03:13:20.164 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f7889ffb640 1 -- 172.21.15.115:0/333918160 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c003ea0 con 0x7f789415acf0 2023-12-01T03:13:20.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.162+0000 7f789b615640 1 -- 172.21.15.115:0/333918160 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78940c6e70 con 0x7f789415acf0 2023-12-01T03:13:20.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.164+0000 7f789b615640 1 -- 172.21.15.115:0/333918160 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78940c7f80 con 0x7f789415acf0 2023-12-01T03:13:20.165 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.165+0000 7f7889ffb640 1 -- 172.21.15.115:0/333918160 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f788c005310 con 0x7f789415acf0 2023-12-01T03:13:20.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.165+0000 7f7889ffb640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f787806a550 0x7f787806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.166+0000 7f7889ffb640 1 -- 172.21.15.115:0/333918160 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f788c088840 con 0x7f789415acf0 2023-12-01T03:13:20.166 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.166+0000 7f78993b1640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f787806a550 0x7f787806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.167 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.167+0000 7f78993b1640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f787806a550 0x7f787806ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7890036000 tx=0x7f789003c040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:20.171 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.171+0000 7f7889ffb640 1 -- 172.21.15.115:0/333918160 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f78940c7f80 con 0x7f789415acf0 2023-12-01T03:13:20.361 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.360+0000 7f789b615640 1 -- 172.21.15.115:0/333918160 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f78940afbf0 con 0x7f787806a550 2023-12-01T03:13:20.362 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.361+0000 7f7889ffb640 1 -- 172.21.15.115:0/333918160 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7890031990 con 0x7f787806a550 2023-12-01T03:13:20.362 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 5 to deep-scrub 2023-12-01T03:13:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.363+0000 7f789b615640 1 -- 172.21.15.115:0/333918160 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f787806a550 msgr2=0x7f787806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.363+0000 7f789b615640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f787806a550 0x7f787806ca10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7890036000 tx=0x7f789003c040 comp rx=0 tx=0).stop 2023-12-01T03:13:20.364 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.363+0000 7f789b615640 1 -- 172.21.15.115:0/333918160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 msgr2=0x7f78940c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.363+0000 7f789b615640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 0x7f78940c9fb0 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 crypto rx=0x7f788c00c6b0 tx=0x7f788c00cb70 comp rx=0 tx=0).stop 2023-12-01T03:13:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.364+0000 7f789b615640 1 -- 172.21.15.115:0/333918160 shutdown_connections 2023-12-01T03:13:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.364+0000 7f789b615640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f787806a550 0x7f787806ca10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.364+0000 7f789b615640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78940cea50 0x7f78940addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.364+0000 7f789b615640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f789415acf0 0x7f78940c9fb0 unknown :-1 s=CLOSED pgs=3152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.364+0000 7f789b615640 1 --2- 172.21.15.115:0/333918160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78940af790 0x7f78940c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.364+0000 7f789b615640 1 -- 172.21.15.115:0/333918160 >> 172.21.15.115:0/333918160 conn(0x7f7894065b40 msgr2=0x7f789406be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.364+0000 7f789b615640 1 -- 172.21.15.115:0/333918160 shutdown_connections 2023-12-01T03:13:20.365 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.364+0000 7f789b615640 1 -- 172.21.15.115:0/333918160 wait complete. 2023-12-01T03:13:20.383 INFO:tasks.ceph:Scrubbing osd.6 2023-12-01T03:13:20.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-12-01T03:13:20.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.600+0000 7f2a11349640 1 Processor -- start 2023-12-01T03:13:20.601 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.600+0000 7f2a11349640 1 -- start start 2023-12-01T03:13:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.602+0000 7f2a11349640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a0c15d910 0x7f2a0c15dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.602+0000 7f2a11349640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2a0c0d30e0 0x7f2a0c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.602+0000 7f2a11349640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0c0b2300 con 0x7f2a0c0cdaa0 2023-12-01T03:13:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.602+0000 7f2a11349640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c1612b0 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.603 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.602+0000 7f2a11349640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c1616d0 con 0x7f2a0c15d910 2023-12-01T03:13:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.602+0000 7f2a10b48640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2a0c0d30e0 0x7f2a0c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.602+0000 7f2a10b48640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f2a0c0d30e0 0x7f2a0c0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32810/0 (socket says 172.21.15.115:32810) 2023-12-01T03:13:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.603+0000 7f2a10b48640 1 -- 172.21.15.115:0/319076702 learned_addr learned my addr 172.21.15.115:0/319076702 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.603+0000 7f2a0bfff640 1 --2- 172.21.15.115:0/319076702 >> v2:172.21.15.142:3300/0 conn(0x7f2a0c15d910 0x7f2a0c15dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.603+0000 7f2a10b48640 1 -- 172.21.15.115:0/319076702 >> v2:172.21.15.142:3300/0 conn(0x7f2a0c15d910 msgr2=0x7f2a0c15dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.603+0000 7f2a10b48640 1 --2- 172.21.15.115:0/319076702 >> v2:172.21.15.142:3300/0 conn(0x7f2a0c15d910 0x7f2a0c15dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.604 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.603+0000 7f2a10b48640 1 -- 172.21.15.115:0/319076702 >> v1:172.21.15.142:6789/0 conn(0x7f2a0c0cdaa0 legacy=0x7f2a0c0d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.603+0000 7f2a10b48640 1 -- 172.21.15.115:0/319076702 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c0ce260 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.604+0000 7f2a10b48640 1 --2- 172.21.15.115:0/319076702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0d30e0 0x7f2a0c0d0e10 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0d0de0 tx=0x7f2a0402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46359d1fbb132ed server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.605+0000 7f2a0b7fe640 1 -- 172.21.15.115:0/319076702 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0402c9f0 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.605+0000 7f2a0b7fe640 1 -- 172.21.15.115:0/319076702 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0402cb50 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.605 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.605+0000 7f2a0b7fe640 1 -- 172.21.15.115:0/319076702 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a040343f0 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.605+0000 7f2a11349640 1 -- 172.21.15.115:0/319076702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0d30e0 msgr2=0x7f2a0c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.605+0000 7f2a11349640 1 --2- 172.21.15.115:0/319076702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0d30e0 0x7f2a0c0d0e10 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0d0de0 tx=0x7f2a0402be50 comp rx=0 tx=0).stop 2023-12-01T03:13:20.606 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.605+0000 7f2a11349640 1 -- 172.21.15.115:0/319076702 shutdown_connections 2023-12-01T03:13:20.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.605+0000 7f2a11349640 1 --2- 172.21.15.115:0/319076702 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0d30e0 0x7f2a0c0d0e10 unknown :-1 s=CLOSED pgs=3153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.605+0000 7f2a11349640 1 --2- 172.21.15.115:0/319076702 >> v2:172.21.15.142:3300/0 conn(0x7f2a0c15d910 0x7f2a0c15dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.605+0000 7f2a11349640 1 -- 172.21.15.115:0/319076702 >> 172.21.15.115:0/319076702 conn(0x7f2a0c06a0b0 msgr2=0x7f2a0c06a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:20.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.606+0000 7f2a11349640 1 -- 172.21.15.115:0/319076702 shutdown_connections 2023-12-01T03:13:20.607 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.606+0000 7f2a11349640 1 -- 172.21.15.115:0/319076702 wait complete. 2023-12-01T03:13:20.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.607+0000 7f2a11349640 1 Processor -- start 2023-12-01T03:13:20.608 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.607+0000 7f2a11349640 1 -- start start 2023-12-01T03:13:20.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.608+0000 7f2a11349640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a0c0cdaa0 0x7f2a0c0c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.609 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.608+0000 7f2a11349640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0d30e0 0x7f2a0c158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.608+0000 7f2a11349640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c15d910 0x7f2a0c0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.608+0000 7f2a11349640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0c74c0 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.608+0000 7f2a11349640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0c7740 con 0x7f2a0c15d910 2023-12-01T03:13:20.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.608+0000 7f2a11349640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0c7a20 con 0x7f2a0c0cdaa0 2023-12-01T03:13:20.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a03fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0d30e0 0x7f2a0c158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a03fff640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0d30e0 0x7f2a0c158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32814/0 (socket says 172.21.15.115:32814) 2023-12-01T03:13:20.610 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a03fff640 1 -- 172.21.15.115:0/3411030062 learned_addr learned my addr 172.21.15.115:0/3411030062 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:20.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a10b48640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c15d910 0x7f2a0c0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.611 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a03fff640 1 -- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a0c0cdaa0 msgr2=0x7f2a0c0c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a0bfff640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a0c0cdaa0 0x7f2a0c0c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a03fff640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a0c0cdaa0 0x7f2a0c0c6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a03fff640 1 -- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c15d910 msgr2=0x7f2a0c0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a03fff640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c15d910 0x7f2a0c0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a03fff640 1 -- 172.21.15.115:0/3411030062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04009b30 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.609+0000 7f2a10b48640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a0c15d910 0x7f2a0c0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.610+0000 7f2a03fff640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0d30e0 0x7f2a0c158060 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c430 tx=0x7f29f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.610+0000 7f2a097fa640 1 -- 172.21.15.115:0/3411030062 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4011790 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.610+0000 7f2a097fa640 1 -- 172.21.15.115:0/3411030062 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f40118f0 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.610+0000 7f2a097fa640 1 -- 172.21.15.115:0/3411030062 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4011ce0 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.610+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c0c7d00 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.610+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c0c9bb0 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.612 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.611+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a0c160bf0 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.612+0000 7f2a097fa640 1 -- 172.21.15.115:0/3411030062 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f29f401c460 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.613 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.613+0000 7f2a097fa640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f29ec06a580 0x7f29ec06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.613+0000 7f2a097fa640 1 -- 172.21.15.115:0/3411030062 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f29f4088a60 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.613+0000 7f2a097fa640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] conn(0x7f29ec070a50 0x7f29ec072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.613+0000 7f2a0bfff640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f29ec06a580 0x7f29ec06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.613+0000 7f2a097fa640 1 -- 172.21.15.115:0/3411030062 --> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29ec073520 con 0x7f29ec070a50 2023-12-01T03:13:20.614 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.613+0000 7f2a097fa640 1 -- 172.21.15.115:0/3411030062 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29f408d2a0 con 0x7f2a0c0d30e0 2023-12-01T03:13:20.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.614+0000 7f2a0bfff640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f29ec06a580 0x7f29ec06ca40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006fd0 tx=0x7f29fc008040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:20.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.614+0000 7f2a10b48640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] conn(0x7f29ec070a50 0x7f29ec072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.615 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.615+0000 7f2a10b48640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] conn(0x7f29ec070a50 0x7f29ec072e70 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:20.617 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.617+0000 7f2a097fa640 1 -- 172.21.15.115:0/3411030062 <== osd.6 v2:172.21.15.142:6808/88937 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f29ec073520 con 0x7f29ec070a50 2023-12-01T03:13:20.631 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.631+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 --> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2a0c160910 con 0x7f29ec070a50 2023-12-01T03:13:20.632 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.632+0000 7f2a097fa640 1 -- 172.21.15.115:0/3411030062 <== osd.6 v2:172.21.15.142:6808/88937 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2a04040410 con 0x7f29ec070a50 2023-12-01T03:13:20.632 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:13:20.633 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-01T03:13:20.633 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.632+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] conn(0x7f29ec070a50 msgr2=0x7f29ec072e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.632+0000 7f2a11349640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.142:6808/88937,v1:172.21.15.142:6809/88937] conn(0x7f29ec070a50 0x7f29ec072e70 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.632+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f29ec06a580 msgr2=0x7f29ec06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.632+0000 7f2a11349640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f29ec06a580 0x7f29ec06ca40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006fd0 tx=0x7f29fc008040 comp rx=0 tx=0).stop 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.632+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0d30e0 msgr2=0x7f2a0c158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.632+0000 7f2a11349640 1 --2- 172.21.15.115:0/3411030062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0d30e0 0x7f2a0c158060 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c430 tx=0x7f29f400c8f0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.632+0000 7f2a10b48640 1 -- 172.21.15.115:0/3411030062 reap_dead start 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.632+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 shutdown_connections 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.632+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 >> 172.21.15.115:0/3411030062 conn(0x7f2a0c06a0b0 msgr2=0x7f2a0c15ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.633+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 shutdown_connections 2023-12-01T03:13:20.634 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.633+0000 7f2a11349640 1 -- 172.21.15.115:0/3411030062 wait complete. 2023-12-01T03:13:20.653 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-12-01T03:13:20.865 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.864+0000 7f89ff9b1640 1 Processor -- start 2023-12-01T03:13:20.866 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.865+0000 7f89ff9b1640 1 -- start start 2023-12-01T03:13:20.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.866+0000 7f89ff9b1640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f89f80ae260 0x7f89f80c5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.866+0000 7f89ff9b1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f89f80c5990 0x7f89f80c9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.868 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.866+0000 7f89ff9b1640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f80c5da0 con 0x7f89f81605f0 2023-12-01T03:13:20.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.866+0000 7f89ff9b1640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f80ad4c0 con 0x7f89f80ae260 2023-12-01T03:13:20.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.866+0000 7f89ff9b1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f80ae700 con 0x7f89f80c5990 2023-12-01T03:13:20.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.867+0000 7f89fd74d640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f89f81605f0 0x7f89f80ac890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59206/0 (socket says 172.21.15.115:59206) 2023-12-01T03:13:20.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.867+0000 7f89fd74d640 1 -- 172.21.15.115:0/306949962 learned_addr learned my addr 172.21.15.115:0/306949962 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:20.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.867+0000 7f89fcf4c640 1 --2- 172.21.15.115:0/306949962 >> v2:172.21.15.115:3300/0 conn(0x7f89f80ae260 0x7f89f80c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.867+0000 7f89f7fff640 1 -- 172.21.15.115:0/306949962 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435327346 0 0) 0x7f89f80c5da0 con 0x7f89f81605f0 2023-12-01T03:13:20.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.867+0000 7f89f7fff640 1 -- 172.21.15.115:0/306949962 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e8003520 con 0x7f89f81605f0 2023-12-01T03:13:20.869 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.867+0000 7f89fcf4c640 1 -- 172.21.15.115:0/306949962 >> v2:172.21.15.142:3300/0 conn(0x7f89f80c5990 msgr2=0x7f89f80c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:20.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.867+0000 7f89fcf4c640 1 --2- 172.21.15.115:0/306949962 >> v2:172.21.15.142:3300/0 conn(0x7f89f80c5990 0x7f89f80c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.870 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.867+0000 7f89fcf4c640 1 -- 172.21.15.115:0/306949962 >> v1:172.21.15.115:6789/0 conn(0x7f89f81605f0 legacy=0x7f89f80ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.871 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.867+0000 7f89fcf4c640 1 -- 172.21.15.115:0/306949962 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f806dbb0 con 0x7f89f80ae260 2023-12-01T03:13:20.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.868+0000 7f89fcf4c640 1 --2- 172.21.15.115:0/306949962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 0x7f89f80c5450 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 crypto rx=0x7f89f80c96c0 tx=0x7f89e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4eda1054628b71 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:20.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89f7fff640 1 -- 172.21.15.115:0/306949962 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4031830 con 0x7f89f80ae260 2023-12-01T03:13:20.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89f7fff640 1 -- 172.21.15.115:0/306949962 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e4031990 con 0x7f89f80ae260 2023-12-01T03:13:20.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89f7fff640 1 -- 172.21.15.115:0/306949962 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4031d20 con 0x7f89f80ae260 2023-12-01T03:13:20.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89ff9b1640 1 -- 172.21.15.115:0/306949962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 msgr2=0x7f89f80c5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.875 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89ff9b1640 1 --2- 172.21.15.115:0/306949962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 0x7f89f80c5450 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 crypto rx=0x7f89f80c96c0 tx=0x7f89e402b8d0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89ff9b1640 1 -- 172.21.15.115:0/306949962 shutdown_connections 2023-12-01T03:13:20.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89ff9b1640 1 --2- 172.21.15.115:0/306949962 >> v2:172.21.15.142:3300/0 conn(0x7f89f80c5990 0x7f89f80c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89ff9b1640 1 --2- 172.21.15.115:0/306949962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 0x7f89f80c5450 unknown :-1 s=CLOSED pgs=3156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89ff9b1640 1 -- 172.21.15.115:0/306949962 >> 172.21.15.115:0/306949962 conn(0x7f89f80cbb60 msgr2=0x7f89f80cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:20.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89ff9b1640 1 -- 172.21.15.115:0/306949962 shutdown_connections 2023-12-01T03:13:20.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.869+0000 7f89ff9b1640 1 -- 172.21.15.115:0/306949962 wait complete. 2023-12-01T03:13:20.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.871+0000 7f89ff9b1640 1 Processor -- start 2023-12-01T03:13:20.876 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.871+0000 7f89ff9b1640 1 -- start start 2023-12-01T03:13:20.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.872+0000 7f89ff9b1640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 0x7f89f80c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.872+0000 7f89ff9b1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89f80c5990 0x7f89f815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.872+0000 7f89ff9b1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f81605f0 0x7f89f8149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.872+0000 7f89ff9b1640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f815baf0 con 0x7f89f80ae260 2023-12-01T03:13:20.877 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.872+0000 7f89ff9b1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80ce5f0 con 0x7f89f81605f0 2023-12-01T03:13:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.872+0000 7f89ff9b1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80ce820 con 0x7f89f80c5990 2023-12-01T03:13:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.872+0000 7f89fd74d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 0x7f89f80c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89fd74d640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 0x7f89f80c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32820/0 (socket says 172.21.15.115:32820) 2023-12-01T03:13:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89fd74d640 1 -- 172.21.15.115:0/3467486305 learned_addr learned my addr 172.21.15.115:0/3467486305 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89fd74d640 1 -- 172.21.15.115:0/3467486305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89f80c5990 msgr2=0x7f89f815d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89fd74d640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89f80c5990 0x7f89f815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89fd74d640 1 -- 172.21.15.115:0/3467486305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f81605f0 msgr2=0x7f89f8149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89fd74d640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f81605f0 0x7f89f8149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89fd74d640 1 -- 172.21.15.115:0/3467486305 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4009400 con 0x7f89f80ae260 2023-12-01T03:13:20.878 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89fd74d640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 0x7f89f80c61b0 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00cbc0 tx=0x7f89ec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:20.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89f5ffb640 1 -- 172.21.15.115:0/3467486305 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00ad40 con 0x7f89f80ae260 2023-12-01T03:13:20.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89f5ffb640 1 -- 172.21.15.115:0/3467486305 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec012400 con 0x7f89f80ae260 2023-12-01T03:13:20.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.873+0000 7f89ff9b1640 1 -- 172.21.15.115:0/3467486305 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f80ceb00 con 0x7f89f80ae260 2023-12-01T03:13:20.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.874+0000 7f89f5ffb640 1 -- 172.21.15.115:0/3467486305 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00aea0 con 0x7f89f80ae260 2023-12-01T03:13:20.879 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.874+0000 7f89ff9b1640 1 -- 172.21.15.115:0/3467486305 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f815b710 con 0x7f89f80ae260 2023-12-01T03:13:20.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.875+0000 7f89ff9b1640 1 -- 172.21.15.115:0/3467486305 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f804e890 con 0x7f89f80ae260 2023-12-01T03:13:20.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.876+0000 7f89f5ffb640 1 -- 172.21.15.115:0/3467486305 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f89ec007070 con 0x7f89f80ae260 2023-12-01T03:13:20.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.877+0000 7f89f5ffb640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89dc06a550 0x7f89dc06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:20.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.877+0000 7f89f5ffb640 1 -- 172.21.15.115:0/3467486305 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f89ec088ab0 con 0x7f89f80ae260 2023-12-01T03:13:20.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.877+0000 7f89fcf4c640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89dc06a550 0x7f89dc06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:20.880 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.878+0000 7f89fcf4c640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89dc06a550 0x7f89dc06ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89e4036000 tx=0x7f89e4030000 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:20.882 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:20.881+0000 7f89f5ffb640 1 -- 172.21.15.115:0/3467486305 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f89ec053f80 con 0x7f89f80ae260 2023-12-01T03:13:21.074 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.073+0000 7f89ff9b1640 1 -- 172.21.15.115:0/3467486305 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f89f806e170 con 0x7f89dc06a550 2023-12-01T03:13:21.075 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.074+0000 7f89f5ffb640 1 -- 172.21.15.115:0/3467486305 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f89e4031630 con 0x7f89dc06a550 2023-12-01T03:13:21.075 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 6 to deep-scrub 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.076+0000 7f89ff9b1640 1 -- 172.21.15.115:0/3467486305 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89dc06a550 msgr2=0x7f89dc06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.076+0000 7f89ff9b1640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89dc06a550 0x7f89dc06ca10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89e4036000 tx=0x7f89e4030000 comp rx=0 tx=0).stop 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.076+0000 7f89ff9b1640 1 -- 172.21.15.115:0/3467486305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 msgr2=0x7f89f80c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.076+0000 7f89ff9b1640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 0x7f89f80c61b0 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00cbc0 tx=0x7f89ec00a430 comp rx=0 tx=0).stop 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.076+0000 7f89ff9b1640 1 -- 172.21.15.115:0/3467486305 shutdown_connections 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.076+0000 7f89ff9b1640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f89dc06a550 0x7f89dc06ca10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.076+0000 7f89ff9b1640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f89f80c5990 0x7f89f815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.076+0000 7f89ff9b1640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f81605f0 0x7f89f8149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.076+0000 7f89ff9b1640 1 --2- 172.21.15.115:0/3467486305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f89f80ae260 0x7f89f80c61b0 unknown :-1 s=CLOSED pgs=3157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.077+0000 7f89ff9b1640 1 -- 172.21.15.115:0/3467486305 >> 172.21.15.115:0/3467486305 conn(0x7f89f80cbb60 msgr2=0x7f89f815ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.077+0000 7f89ff9b1640 1 -- 172.21.15.115:0/3467486305 shutdown_connections 2023-12-01T03:13:21.078 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.077+0000 7f89ff9b1640 1 -- 172.21.15.115:0/3467486305 wait complete. 2023-12-01T03:13:21.097 INFO:tasks.ceph:Scrubbing osd.7 2023-12-01T03:13:21.097 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-12-01T03:13:21.318 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.318+0000 7ff5dc849640 1 Processor -- start 2023-12-01T03:13:21.319 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.318+0000 7ff5dc849640 1 -- start start 2023-12-01T03:13:21.320 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.319+0000 7ff5dc849640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5d815cb60 0x7ff5d80caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.319+0000 7ff5dc849640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5d80ca2d0 0x7ff5d80ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.319+0000 7ff5dc849640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d80cb8c0 con 0x7ff5d80cb430 2023-12-01T03:13:21.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.319+0000 7ff5dc849640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d80ca740 con 0x7ff5d815cb60 2023-12-01T03:13:21.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.319+0000 7ff5dc849640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d8070000 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5d77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5d815cb60 0x7ff5d80caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5d77fe640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7ff5d815cb60 0x7ff5d80caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32826/0 (socket says 172.21.15.115:32826) 2023-12-01T03:13:21.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5d77fe640 1 -- 172.21.15.115:0/364193836 learned_addr learned my addr 172.21.15.115:0/364193836 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:21.321 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5cffff640 1 --1- 172.21.15.115:0/364193836 >> v1:172.21.15.115:6789/0 conn(0x7ff5d80cb430 0x7ff5d80c9bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59216/0 (socket says 172.21.15.115:59216) 2023-12-01T03:13:21.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5d7fff640 1 --2- 172.21.15.115:0/364193836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5d80ca2d0 0x7ff5d80ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5d77fe640 1 -- 172.21.15.115:0/364193836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5d80ca2d0 msgr2=0x7ff5d80ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:21.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5d77fe640 1 --2- 172.21.15.115:0/364193836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5d80ca2d0 0x7ff5d80ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5d77fe640 1 -- 172.21.15.115:0/364193836 >> v1:172.21.15.115:6789/0 conn(0x7ff5d80cb430 legacy=0x7ff5d80c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5d77fe640 1 -- 172.21.15.115:0/364193836 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d80702e0 con 0x7ff5d815cb60 2023-12-01T03:13:21.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5d7fff640 1 --2- 172.21.15.115:0/364193836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5d80ca2d0 0x7ff5d80ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:21.322 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.320+0000 7ff5d6ffd640 1 -- 172.21.15.115:0/364193836 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857095796 0 0) 0x7ff5d80cb8c0 con 0x7ff5d80cb430 2023-12-01T03:13:21.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.322+0000 7ff5d77fe640 1 --2- 172.21.15.115:0/364193836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5d815cb60 0x7ff5d80caef0 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009870 tx=0x7ff5c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a2f56f42b5cb50 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:21.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.322+0000 7ff5d6ffd640 1 -- 172.21.15.115:0/364193836 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8030840 con 0x7ff5d815cb60 2023-12-01T03:13:21.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.322+0000 7ff5d6ffd640 1 -- 172.21.15.115:0/364193836 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c80309a0 con 0x7ff5d815cb60 2023-12-01T03:13:21.323 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.322+0000 7ff5d6ffd640 1 -- 172.21.15.115:0/364193836 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8030d30 con 0x7ff5d815cb60 2023-12-01T03:13:21.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.323+0000 7ff5dc849640 1 -- 172.21.15.115:0/364193836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5d815cb60 msgr2=0x7ff5d80caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.323+0000 7ff5dc849640 1 --2- 172.21.15.115:0/364193836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5d815cb60 0x7ff5d80caef0 secure :-1 s=READY pgs=3159 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009870 tx=0x7ff5c802b980 comp rx=0 tx=0).stop 2023-12-01T03:13:21.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.323+0000 7ff5dc849640 1 -- 172.21.15.115:0/364193836 shutdown_connections 2023-12-01T03:13:21.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.323+0000 7ff5dc849640 1 --2- 172.21.15.115:0/364193836 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5d80ca2d0 0x7ff5d80ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.323+0000 7ff5dc849640 1 --2- 172.21.15.115:0/364193836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5d815cb60 0x7ff5d80caef0 unknown :-1 s=CLOSED pgs=3159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.323+0000 7ff5dc849640 1 -- 172.21.15.115:0/364193836 >> 172.21.15.115:0/364193836 conn(0x7ff5d8065960 msgr2=0x7ff5d8065d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:21.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.323+0000 7ff5dc849640 1 -- 172.21.15.115:0/364193836 shutdown_connections 2023-12-01T03:13:21.324 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.323+0000 7ff5dc849640 1 -- 172.21.15.115:0/364193836 wait complete. 2023-12-01T03:13:21.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.325+0000 7ff5dc849640 1 Processor -- start 2023-12-01T03:13:21.325 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.325+0000 7ff5dc849640 1 -- start start 2023-12-01T03:13:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5dc849640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5d80ca2d0 0x7ff5d80c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5dc849640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5d80cb430 0x7ff5d81416a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5dc849640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5d815cb60 0x7ff5d80cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5dc849640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d8141be0 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5dc849640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d80cdac0 con 0x7ff5d815cb60 2023-12-01T03:13:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5dc849640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d815d6a0 con 0x7ff5d80cb430 2023-12-01T03:13:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5d77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5d80ca2d0 0x7ff5d80c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5d77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5d80ca2d0 0x7ff5d80c5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32830/0 (socket says 172.21.15.115:32830) 2023-12-01T03:13:21.327 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5d77fe640 1 -- 172.21.15.115:0/2606509535 learned_addr learned my addr 172.21.15.115:0/2606509535 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:21.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5cffff640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5d80cb430 0x7ff5d81416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.326+0000 7ff5d77fe640 1 -- 172.21.15.115:0/2606509535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5d80cb430 msgr2=0x7ff5d81416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.327+0000 7ff5d77fe640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff5d80cb430 0x7ff5d81416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.327+0000 7ff5d77fe640 1 -- 172.21.15.115:0/2606509535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5d815cb60 msgr2=0x7ff5d80cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.328 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.327+0000 7ff5d77fe640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5d815cb60 0x7ff5d80cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.327+0000 7ff5d77fe640 1 -- 172.21.15.115:0/2606509535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80094d0 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.327+0000 7ff5d77fe640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5d80ca2d0 0x7ff5d80c5800 secure :-1 s=READY pgs=3160 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8000c00 tx=0x7ff5c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:21.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.327+0000 7ff5d4ff9640 1 -- 172.21.15.115:0/2606509535 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c803e070 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.327+0000 7ff5d4ff9640 1 -- 172.21.15.115:0/2606509535 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c8004030 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.327+0000 7ff5d4ff9640 1 -- 172.21.15.115:0/2606509535 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80043e0 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.327+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d80c5e90 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.327+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d815d3c0 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.329 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.328+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5d80708a0 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.330 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.329+0000 7ff5d4ff9640 1 -- 172.21.15.115:0/2606509535 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7ff5c8042070 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.330+0000 7ff5d4ff9640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff5bc06a580 0x7ff5bc06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.330+0000 7ff5d4ff9640 1 -- 172.21.15.115:0/2606509535 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7ff5c80aaa90 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.331+0000 7ff5cffff640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff5bc06a580 0x7ff5bc06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.331+0000 7ff5d4ff9640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] conn(0x7ff5bc070a50 0x7ff5bc072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.331+0000 7ff5d4ff9640 1 -- 172.21.15.115:0/2606509535 --> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5bc073520 con 0x7ff5bc070a50 2023-12-01T03:13:21.331 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.331+0000 7ff5d4ff9640 1 -- 172.21.15.115:0/2606509535 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_get_version_reply(handle=1 version=261) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5c80af2c0 con 0x7ff5d80ca2d0 2023-12-01T03:13:21.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.331+0000 7ff5cffff640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff5bc06a580 0x7ff5bc06ca40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0007fd0 tx=0x7ff5c0009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:21.332 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.331+0000 7ff5d7fff640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] conn(0x7ff5bc070a50 0x7ff5bc072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.333 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.332+0000 7ff5d7fff640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] conn(0x7ff5bc070a50 0x7ff5bc072e70 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:21.335 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.334+0000 7ff5d4ff9640 1 -- 172.21.15.115:0/2606509535 <== osd.7 v2:172.21.15.142:6817/88947 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff5bc073520 con 0x7ff5bc070a50 2023-12-01T03:13:21.348 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.348+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 --> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff5d80705c0 con 0x7ff5bc070a50 2023-12-01T03:13:21.349 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.349+0000 7ff5d4ff9640 1 -- 172.21.15.115:0/2606509535 <== osd.7 v2:172.21.15.142:6817/88947 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff5d0003990 con 0x7ff5bc070a50 2023-12-01T03:13:21.350 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2023-12-01T03:13:21.350 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-01T03:13:21.350 INFO:teuthology.orchestra.run.smithi115.stdout:} 2023-12-01T03:13:21.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.349+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 >> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] conn(0x7ff5bc070a50 msgr2=0x7ff5bc072e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.349+0000 7ff5dc849640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.142:6817/88947,v1:172.21.15.142:6819/88947] conn(0x7ff5bc070a50 0x7ff5bc072e70 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.351 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.349+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff5bc06a580 msgr2=0x7ff5bc06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.349+0000 7ff5dc849640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7ff5bc06a580 0x7ff5bc06ca40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0007fd0 tx=0x7ff5c0009040 comp rx=0 tx=0).stop 2023-12-01T03:13:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.349+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5d80ca2d0 msgr2=0x7ff5d80c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.349+0000 7ff5dc849640 1 --2- 172.21.15.115:0/2606509535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff5d80ca2d0 0x7ff5d80c5800 secure :-1 s=READY pgs=3160 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8000c00 tx=0x7ff5c80046c0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.349+0000 7ff5d7fff640 1 -- 172.21.15.115:0/2606509535 reap_dead start 2023-12-01T03:13:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.350+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 shutdown_connections 2023-12-01T03:13:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.350+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 >> 172.21.15.115:0/2606509535 conn(0x7ff5d8065960 msgr2=0x7ff5d80c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.350+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 shutdown_connections 2023-12-01T03:13:21.352 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.350+0000 7ff5dc849640 1 -- 172.21.15.115:0/2606509535 wait complete. 2023-12-01T03:13:21.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-12-01T03:13:21.574 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.574+0000 7f3f02edc640 1 Processor -- start 2023-12-01T03:13:21.575 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.574+0000 7f3f02edc640 1 -- start start 2023-12-01T03:13:21.576 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.575+0000 7f3f02edc640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3efc153fb0 0x7f3efc0c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.575+0000 7f3f02edc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0cc1c0 0x7f3efc0cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.575+0000 7f3f02edc640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc0cbd30 con 0x7f3efc0c4ac0 2023-12-01T03:13:21.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.575+0000 7f3f02edc640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3efc0cb160 con 0x7f3efc153fb0 2023-12-01T03:13:21.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.575+0000 7f3f02edc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc154390 con 0x7f3efc0cc1c0 2023-12-01T03:13:21.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.576+0000 7f3f01eda640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3efc153fb0 0x7f3efc0c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.576+0000 7f3f01eda640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3efc153fb0 0x7f3efc0c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32842/0 (socket says 172.21.15.115:32842) 2023-12-01T03:13:21.577 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.576+0000 7f3f01eda640 1 -- 172.21.15.115:0/1123642610 learned_addr learned my addr 172.21.15.115:0/1123642610 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:21.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.576+0000 7f3f01eda640 1 -- 172.21.15.115:0/1123642610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0cc1c0 msgr2=0x7f3efc0cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:21.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.576+0000 7f3f01eda640 1 --2- 172.21.15.115:0/1123642610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0cc1c0 0x7f3efc0cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.576+0000 7f3f01eda640 1 -- 172.21.15.115:0/1123642610 >> v1:172.21.15.115:6789/0 conn(0x7f3efc0c4ac0 legacy=0x7f3efc06cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:21.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.576+0000 7f3f01eda640 1 -- 172.21.15.115:0/1123642610 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc0cb400 con 0x7f3efc153fb0 2023-12-01T03:13:21.578 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.577+0000 7f3f01eda640 1 --2- 172.21.15.115:0/1123642610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 0x7f3efc0c4580 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009e50 tx=0x7f3eec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c23385fa809d774 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:21.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.578+0000 7f3f00ed8640 1 -- 172.21.15.115:0/1123642610 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec02ca70 con 0x7f3efc153fb0 2023-12-01T03:13:21.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.578+0000 7f3f00ed8640 1 -- 172.21.15.115:0/1123642610 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec02cbd0 con 0x7f3efc153fb0 2023-12-01T03:13:21.579 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.578+0000 7f3f00ed8640 1 -- 172.21.15.115:0/1123642610 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0343f0 con 0x7f3efc153fb0 2023-12-01T03:13:21.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.578+0000 7f3f02edc640 1 -- 172.21.15.115:0/1123642610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 msgr2=0x7f3efc0c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.578+0000 7f3f02edc640 1 --2- 172.21.15.115:0/1123642610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 0x7f3efc0c4580 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009e50 tx=0x7f3eec02bed0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.578+0000 7f3f02edc640 1 -- 172.21.15.115:0/1123642610 shutdown_connections 2023-12-01T03:13:21.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.578+0000 7f3f02edc640 1 --2- 172.21.15.115:0/1123642610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0cc1c0 0x7f3efc0cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.578+0000 7f3f02edc640 1 --2- 172.21.15.115:0/1123642610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 0x7f3efc0c4580 unknown :-1 s=CLOSED pgs=3162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.578+0000 7f3f02edc640 1 -- 172.21.15.115:0/1123642610 >> 172.21.15.115:0/1123642610 conn(0x7f3efc06e750 msgr2=0x7f3efc069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:21.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.579+0000 7f3f02edc640 1 -- 172.21.15.115:0/1123642610 shutdown_connections 2023-12-01T03:13:21.580 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.579+0000 7f3f02edc640 1 -- 172.21.15.115:0/1123642610 wait complete. 2023-12-01T03:13:21.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.580+0000 7f3f02edc640 1 Processor -- start 2023-12-01T03:13:21.581 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.580+0000 7f3f02edc640 1 -- start start 2023-12-01T03:13:21.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.581+0000 7f3f02edc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc0c4ac0 0x7f3efc14df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.581+0000 7f3f02edc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0cc1c0 0x7f3efc149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.581+0000 7f3f02edc640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 0x7f3efc14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.581+0000 7f3f02edc640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc151500 con 0x7f3efc153fb0 2023-12-01T03:13:21.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.581+0000 7f3f02edc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc14ba70 con 0x7f3efc0c4ac0 2023-12-01T03:13:21.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.581+0000 7f3f02edc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc14bcf0 con 0x7f3efc0cc1c0 2023-12-01T03:13:21.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f026db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 0x7f3efc14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f026db640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 0x7f3efc14d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32856/0 (socket says 172.21.15.115:32856) 2023-12-01T03:13:21.583 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f026db640 1 -- 172.21.15.115:0/1777012946 learned_addr learned my addr 172.21.15.115:0/1777012946 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:21.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f01eda640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc0c4ac0 0x7f3efc14df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f016d9640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0cc1c0 0x7f3efc149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f026db640 1 -- 172.21.15.115:0/1777012946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0cc1c0 msgr2=0x7f3efc149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f026db640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0cc1c0 0x7f3efc149d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f026db640 1 -- 172.21.15.115:0/1777012946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc0c4ac0 msgr2=0x7f3efc14df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f026db640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc0c4ac0 0x7f3efc14df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f026db640 1 -- 172.21.15.115:0/1777012946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec009b30 con 0x7f3efc153fb0 2023-12-01T03:13:21.584 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.582+0000 7f3f016d9640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0cc1c0 0x7f3efc149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.583+0000 7f3f026db640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 0x7f3efc14d130 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400b410 tx=0x7f3ef400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.583+0000 7f3efa7fc640 1 -- 172.21.15.115:0/1777012946 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4011830 con 0x7f3efc153fb0 2023-12-01T03:13:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.583+0000 7f3efa7fc640 1 -- 172.21.15.115:0/1777012946 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef40119b0 con 0x7f3efc153fb0 2023-12-01T03:13:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.583+0000 7f3efa7fc640 1 -- 172.21.15.115:0/1777012946 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4011d60 con 0x7f3efc153fb0 2023-12-01T03:13:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.583+0000 7f3f02edc640 1 -- 172.21.15.115:0/1777012946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc14c030 con 0x7f3efc153fb0 2023-12-01T03:13:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.583+0000 7f3f02edc640 1 -- 172.21.15.115:0/1777012946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc14a2a0 con 0x7f3efc153fb0 2023-12-01T03:13:21.585 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.584+0000 7f3f02edc640 1 -- 172.21.15.115:0/1777012946 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc0cb9a0 con 0x7f3efc153fb0 2023-12-01T03:13:21.586 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.585+0000 7f3efa7fc640 1 -- 172.21.15.115:0/1777012946 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3ef401a400 con 0x7f3efc153fb0 2023-12-01T03:13:21.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.586+0000 7f3efa7fc640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3ee806a550 0x7f3ee806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:21.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.586+0000 7f3efa7fc640 1 -- 172.21.15.115:0/1777012946 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f3ef4088c10 con 0x7f3efc153fb0 2023-12-01T03:13:21.587 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.586+0000 7f3f01eda640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3ee806a550 0x7f3ee806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:21.588 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.587+0000 7f3f01eda640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3ee806a550 0x7f3ee806ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3eec035000 tx=0x7f3eec030000 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:21.591 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.590+0000 7f3efa7fc640 1 -- 172.21.15.115:0/1777012946 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3efc0cb9a0 con 0x7f3efc153fb0 2023-12-01T03:13:21.780 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.779+0000 7f3f02edc640 1 -- 172.21.15.115:0/1777012946 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f3efc14beb0 con 0x7f3ee806a550 2023-12-01T03:13:21.781 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.780+0000 7f3efa7fc640 1 -- 172.21.15.115:0/1777012946 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3eec009510 con 0x7f3ee806a550 2023-12-01T03:13:21.781 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 7 to deep-scrub 2023-12-01T03:13:21.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 -- 172.21.15.115:0/1777012946 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3ee806a550 msgr2=0x7f3ee806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3ee806a550 0x7f3ee806ca10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3eec035000 tx=0x7f3eec030000 comp rx=0 tx=0).stop 2023-12-01T03:13:21.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 -- 172.21.15.115:0/1777012946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 msgr2=0x7f3efc14d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:21.784 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 0x7f3efc14d130 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400b410 tx=0x7f3ef400b8d0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 -- 172.21.15.115:0/1777012946 shutdown_connections 2023-12-01T03:13:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3ee806a550 0x7f3ee806ca10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3efc0cc1c0 0x7f3efc149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3efc153fb0 0x7f3efc14d130 unknown :-1 s=CLOSED pgs=3163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 --2- 172.21.15.115:0/1777012946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3efc0c4ac0 0x7f3efc14df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 -- 172.21.15.115:0/1777012946 >> 172.21.15.115:0/1777012946 conn(0x7f3efc06e750 msgr2=0x7f3efc15b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 -- 172.21.15.115:0/1777012946 shutdown_connections 2023-12-01T03:13:21.785 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:21.783+0000 7f3f02edc640 1 -- 172.21.15.115:0/1777012946 wait complete. 2023-12-01T03:13:21.803 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T03:13:22.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.013+0000 7f75bca42640 1 Processor -- start 2023-12-01T03:13:22.014 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.013+0000 7f75bca42640 1 -- start start 2023-12-01T03:13:22.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.014+0000 7f75bca42640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f75b815afe0 0x7f75b806b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75bca42640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75b806b7b0 0x7f75b80cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75bca42640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b80ce4a0 con 0x7f75b81584b0 2023-12-01T03:13:22.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75bca42640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b806bc40 con 0x7f75b815afe0 2023-12-01T03:13:22.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75bca42640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b80cc180 con 0x7f75b806b7b0 2023-12-01T03:13:22.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75affff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f75b815afe0 0x7f75b806b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75affff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f75b815afe0 0x7f75b806b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32860/0 (socket says 172.21.15.115:32860) 2023-12-01T03:13:22.016 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75affff640 1 -- 172.21.15.115:0/3694050181 learned_addr learned my addr 172.21.15.115:0/3694050181 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75b759e640 1 --2- 172.21.15.115:0/3694050181 >> v2:172.21.15.142:3300/0 conn(0x7f75b806b7b0 0x7f75b80cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75affff640 1 -- 172.21.15.115:0/3694050181 >> v2:172.21.15.142:3300/0 conn(0x7f75b806b7b0 msgr2=0x7f75b80cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75affff640 1 --2- 172.21.15.115:0/3694050181 >> v2:172.21.15.142:3300/0 conn(0x7f75b806b7b0 0x7f75b80cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75affff640 1 -- 172.21.15.115:0/3694050181 >> v1:172.21.15.115:6789/0 conn(0x7f75b81584b0 legacy=0x7f75b80cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.015+0000 7f75affff640 1 -- 172.21.15.115:0/3694050181 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b815b550 con 0x7f75b815afe0 2023-12-01T03:13:22.017 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.017+0000 7f75affff640 1 --2- 172.21.15.115:0/3694050181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 0x7f75b806b270 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 crypto rx=0x7f75b815b440 tx=0x7f75a002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed436e8897119277 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.017+0000 7f75b659c640 1 -- 172.21.15.115:0/3694050181 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0004230 con 0x7f75b815afe0 2023-12-01T03:13:22.018 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.017+0000 7f75b659c640 1 -- 172.21.15.115:0/3694050181 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a002e050 con 0x7f75b815afe0 2023-12-01T03:13:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.017+0000 7f75b659c640 1 -- 172.21.15.115:0/3694050181 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0003610 con 0x7f75b815afe0 2023-12-01T03:13:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.017+0000 7f75bca42640 1 -- 172.21.15.115:0/3694050181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 msgr2=0x7f75b806b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.017+0000 7f75bca42640 1 --2- 172.21.15.115:0/3694050181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 0x7f75b806b270 secure :-1 s=READY pgs=3165 cs=0 l=1 rev1=1 crypto rx=0x7f75b815b440 tx=0x7f75a002b7e0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.018+0000 7f75bca42640 1 -- 172.21.15.115:0/3694050181 shutdown_connections 2023-12-01T03:13:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.018+0000 7f75bca42640 1 --2- 172.21.15.115:0/3694050181 >> v2:172.21.15.142:3300/0 conn(0x7f75b806b7b0 0x7f75b80cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.018+0000 7f75bca42640 1 --2- 172.21.15.115:0/3694050181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 0x7f75b806b270 unknown :-1 s=CLOSED pgs=3165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.018+0000 7f75bca42640 1 -- 172.21.15.115:0/3694050181 >> 172.21.15.115:0/3694050181 conn(0x7f75b80ce640 msgr2=0x7f75b80cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.018+0000 7f75bca42640 1 -- 172.21.15.115:0/3694050181 shutdown_connections 2023-12-01T03:13:22.019 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.018+0000 7f75bca42640 1 -- 172.21.15.115:0/3694050181 wait complete. 2023-12-01T03:13:22.020 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.020+0000 7f75bca42640 1 Processor -- start 2023-12-01T03:13:22.021 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.020+0000 7f75bca42640 1 -- start start 2023-12-01T03:13:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75bca42640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b806b7b0 0x7f75b80c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75bca42640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b81584b0 0x7f75b8159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75bca42640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 0x7f75b8152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75bca42640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b80cab90 con 0x7f75b815afe0 2023-12-01T03:13:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75bca42640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b80c6970 con 0x7f75b81584b0 2023-12-01T03:13:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75bca42640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b80c6c30 con 0x7f75b806b7b0 2023-12-01T03:13:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75b759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 0x7f75b8152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.024 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75b759e640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 0x7f75b8152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32866/0 (socket says 172.21.15.115:32866) 2023-12-01T03:13:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75b759e640 1 -- 172.21.15.115:0/4144404115 learned_addr learned my addr 172.21.15.115:0/4144404115 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75affff640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b81584b0 0x7f75b8159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.021+0000 7f75b6d9d640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b806b7b0 0x7f75b80c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75b759e640 1 -- 172.21.15.115:0/4144404115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b806b7b0 msgr2=0x7f75b80c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75b759e640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b806b7b0 0x7f75b80c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75b759e640 1 -- 172.21.15.115:0/4144404115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b81584b0 msgr2=0x7f75b8159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75b759e640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b81584b0 0x7f75b8159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.025 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75b759e640 1 -- 172.21.15.115:0/4144404115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00094d0 con 0x7f75b815afe0 2023-12-01T03:13:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75b759e640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 0x7f75b8152dd0 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 crypto rx=0x7f75b000cd70 tx=0x7f75b000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75af7fe640 1 -- 172.21.15.115:0/4144404115 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0003b90 con 0x7f75b815afe0 2023-12-01T03:13:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75af7fe640 1 -- 172.21.15.115:0/4144404115 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b0003cf0 con 0x7f75b815afe0 2023-12-01T03:13:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75af7fe640 1 -- 172.21.15.115:0/4144404115 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0004240 con 0x7f75b815afe0 2023-12-01T03:13:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75bca42640 1 -- 172.21.15.115:0/4144404115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b80c6f10 con 0x7f75b815afe0 2023-12-01T03:13:22.026 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.022+0000 7f75bca42640 1 -- 172.21.15.115:0/4144404115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b80c74d0 con 0x7f75b815afe0 2023-12-01T03:13:22.027 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.024+0000 7f75bca42640 1 -- 172.21.15.115:0/4144404115 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b80673a0 con 0x7f75b815afe0 2023-12-01T03:13:22.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.024+0000 7f75af7fe640 1 -- 172.21.15.115:0/4144404115 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f75b00043a0 con 0x7f75b815afe0 2023-12-01T03:13:22.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.025+0000 7f75af7fe640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f759806a550 0x7f759806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.025+0000 7f75af7fe640 1 -- 172.21.15.115:0/4144404115 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f75b0012070 con 0x7f75b815afe0 2023-12-01T03:13:22.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.026+0000 7f75b6d9d640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f759806a550 0x7f759806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.028 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.026+0000 7f75b6d9d640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f759806a550 0x7f759806ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f75a8000f60 tx=0x7f75a8009040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:22.030 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.029+0000 7f75af7fe640 1 -- 172.21.15.115:0/4144404115 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f75b00549e0 con 0x7f75b815afe0 2023-12-01T03:13:22.215 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.214+0000 7f75bca42640 1 -- 172.21.15.115:0/4144404115 --> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75b815bb10 con 0x7f759806a550 2023-12-01T03:13:22.219 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.219+0000 7f75af7fe640 1 -- 172.21.15.115:0/4144404115 <== mgr.12555 v2:172.21.15.115:6822/138932 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48256 (secure 0 0 0) 0x7f75a8001710 con 0x7f759806a550 2023-12-01T03:13:22.220 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:22.221 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2023-12-01T03:13:22.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.221+0000 7f75bca42640 1 -- 172.21.15.115:0/4144404115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f759806a550 msgr2=0x7f759806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.221+0000 7f75bca42640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f759806a550 0x7f759806ca10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f75a8000f60 tx=0x7f75a8009040 comp rx=0 tx=0).stop 2023-12-01T03:13:22.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.221+0000 7f75bca42640 1 -- 172.21.15.115:0/4144404115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 msgr2=0x7f75b8152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.221+0000 7f75bca42640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 0x7f75b8152dd0 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 crypto rx=0x7f75b000cd70 tx=0x7f75b000a660 comp rx=0 tx=0).stop 2023-12-01T03:13:22.222 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.221+0000 7f75bca42640 1 -- 172.21.15.115:0/4144404115 shutdown_connections 2023-12-01T03:13:22.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.221+0000 7f75bca42640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f759806a550 0x7f759806ca10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.221+0000 7f75bca42640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b815afe0 0x7f75b8152dd0 unknown :-1 s=CLOSED pgs=3166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.221+0000 7f75bca42640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b81584b0 0x7f75b8159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.221+0000 7f75bca42640 1 --2- 172.21.15.115:0/4144404115 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f75b806b7b0 0x7f75b80c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.221+0000 7f75bca42640 1 -- 172.21.15.115:0/4144404115 >> 172.21.15.115:0/4144404115 conn(0x7f75b80ce640 msgr2=0x7f75b8154c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:22.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.222+0000 7f75bca42640 1 -- 172.21.15.115:0/4144404115 shutdown_connections 2023-12-01T03:13:22.223 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.222+0000 7f75bca42640 1 -- 172.21.15.115:0/4144404115 wait complete. 2023-12-01T03:13:22.241 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2023-12-01T03:13:21.102794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":382,"ondisk_log_size":382,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":444264,"kb_used_data":26920,"kb_used_omap":0,"kb_used_meta":417280,"kb_avail":754530456,"statfs":{"total":773094113280,"available":772639186944,"internally_reserved":0,"allocated":27566080,"data_stored":14175224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":427294720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.235105"},"pg_stats":[{"pgid":"1.0","version":"261'382","reported_seq":1368,"reported_epoch":261,"state":"active+clean","last_fresh":"2023-12-01T03:13:17.969464+0000","last_change":"2023-12-01T03:13:17.969691+0000","last_active":"2023-12-01T03:13:17.969464+0000","last_peered":"2023-12-01T03:13:17.969464+0000","last_clean":"2023-12-01T03:13:17.969464+0000","last_became_active":"2023-12-01T01:36:02.277916+0000","last_became_peered":"2023-12-01T01:36:02.277916+0000","last_unstale":"2023-12-01T03:13:17.969464+0000","last_undegraded":"2023-12-01T03:13:17.969464+0000","last_fullsized":"2023-12-01T03:13:17.969464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'382","last_scrub_stamp":"2023-12-01T03:13:17.969413+0000","last_deep_scrub":"261'382","last_deep_scrub_stamp":"2023-12-01T03:13:17.969413+0000","last_clean_scrub_stamp":"2023-12-01T03:13:17.969413+0000","objects_scrubbed":2,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:36:22.776205+0000","scrub_duration":0.011147432000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2420,"num_read_kb":5044,"num_write":639,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":382,"ondisk_log_size":382,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60508,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311332,"statfs":{"total":96636764160,"available":96574803968,"internally_reserved":0,"allocated":3297280,"data_stored":1624311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0429999999999999,"15min":1.071},"min":{"1min":0.752,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.696,"5min":1.986,"15min":1.986},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.1220000000000001,"15min":1.121},"min":{"1min":0.68300000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.512,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":1.111}]},{"osd":1,"last update":"Fri Dec 1 03:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.232,"15min":1.2889999999999999},"min":{"1min":0.79800000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.8640000000000001,"5min":3.1930000000000001,"15min":3.1930000000000001},"last":1.274},{"interface":"front","average":{"1min":1.282,"5min":1.2909999999999999,"15min":1.3180000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.8109999999999999,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.4259999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1040000000000001,"15min":1.1579999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.526,"5min":1.95,"15min":2.1179999999999999},"last":1.052},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.278,"15min":1.27},"min":{"1min":0.60399999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.861,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":1.492}]},{"osd":3,"last update":"Fri Dec 1 03:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2430000000000001,"15min":1.2889999999999999},"min":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.611,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3859999999999999,"15min":1.427},"min":{"1min":0.72399999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.0259999999999998,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":1.0069999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.319,"15min":1.327},"min":{"1min":0.63500000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.5899999999999999},"last":1.623},{"interface":"front","average":{"1min":1.611,"5min":1.5109999999999999,"15min":1.5169999999999999},"min":{"1min":0.82599999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.2450000000000001,"5min":3.7200000000000002,"15min":3.7200000000000002},"last":1.645}]},{"osd":5,"last update":"Fri Dec 1 03:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3540000000000001,"15min":1.3340000000000001},"min":{"1min":0.90200000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.133,"5min":4.2709999999999999,"15min":4.2709999999999999},"last":1.591},{"interface":"front","average":{"1min":1.704,"5min":1.6020000000000001,"15min":1.609},"min":{"1min":0.77000000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.3079999999999998,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":1.712}]},{"osd":6,"last update":"Fri Dec 1 03:13:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.478,"15min":1.46},"min":{"1min":0.76900000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.258,"5min":4.8579999999999997,"15min":4.8579999999999997},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.431,"15min":1.4550000000000001},"min":{"1min":0.74399999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.762},"last":1.5569999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60508,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311332,"statfs":{"total":96636764160,"available":96574803968,"internally_reserved":0,"allocated":3297280,"data_stored":1624311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.036,"15min":1.0980000000000001},"min":{"1min":0.53600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4810000000000001,"5min":1.8799999999999999,"15min":1.9219999999999999},"last":1.054},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.093,"15min":1.1519999999999999},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6439999999999999,"5min":1.7310000000000001,"15min":2.0619999999999998},"last":1.425}]},{"osd":1,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.127,"15min":1.202},"min":{"1min":0.57399999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7010000000000001,"5min":1.976,"15min":2.0449999999999999},"last":1.222},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.179,"15min":1.2170000000000001},"min":{"1min":0.58599999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.542,"5min":1.694,"15min":1.835},"last":0.93799999999999994}]},{"osd":2,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.149,"15min":1.2},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5369999999999999,"5min":1.742,"15min":1.976},"last":1.087},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.272},"min":{"1min":0.71599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4510000000000001,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.98699999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.262,"15min":1.306},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7929999999999999,"5min":1.927,"15min":1.927},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.377,"15min":1.415},"min":{"1min":0.68700000000000006,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.361}]},{"osd":4,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2809999999999999,"15min":1.27},"min":{"1min":0.61499999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.661,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.391},{"interface":"front","average":{"1min":1.355,"5min":1.4199999999999999,"15min":1.3660000000000001},"min":{"1min":0.64100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1019999999999999,"5min":2.48,"15min":2.48},"last":1.54}]},{"osd":5,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3859999999999999,"15min":1.407},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.651,"5min":2.262,"15min":2.262},"last":1.323},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.518,"15min":1.492},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0030000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:12:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5069999999999999,"15min":1.526},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.865,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.5},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.0539999999999998,"5min":2.2999999999999998,"15min":2.7730000000000001},"last":1.4630000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576008,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52896,"kb_used_data":3800,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94318944,"statfs":{"total":96636764160,"available":96582598656,"internally_reserved":0,"allocated":3891200,"data_stored":2214679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.089,"15min":1.113},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.61299999999999999},"max":{"1min":1.4339999999999999,"5min":1.776,"15min":1.776},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1299999999999999,"15min":1.151},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.97699999999999998}]},{"osd":1,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.1319999999999999,"15min":1.145},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.179,"15min":1.218},"min":{"1min":0.74199999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5329999999999999,"5min":1.7010000000000001,"15min":1.7649999999999999},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1970000000000001,"15min":1.2410000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.6499999999999999,"5min":1.901,"15min":1.901},"last":1.109},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.224,"15min":1.244},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.546,"5min":1.899,"15min":1.9219999999999999},"last":1.242}]},{"osd":3,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2989999999999999,"15min":1.3109999999999999},"min":{"1min":0.92700000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.5740000000000001,"5min":1.889,"15min":1.8979999999999999},"last":1.512},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.294,"15min":1.3460000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5960000000000001,"5min":1.97,"15min":2.1859999999999999},"last":1.331}]},{"osd":4,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3,"15min":1.258},"min":{"1min":0.90700000000000003,"5min":0.82899999999999996,"15min":0.78000000000000003},"max":{"1min":1.681,"5min":1.869,"15min":2.6040000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.369,"15min":1.3720000000000001},"min":{"1min":1.0349999999999999,"5min":0.85799999999999998,"15min":0.84599999999999997},"max":{"1min":1.6659999999999999,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.7210000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3899999999999999,"15min":1.3740000000000001},"min":{"1min":0.97399999999999998,"5min":0.93300000000000005,"15min":0.88800000000000001},"max":{"1min":1.6399999999999999,"5min":2.0129999999999999,"15min":2.5099999999999998},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.47,"15min":1.4259999999999999},"min":{"1min":1.3149999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7529999999999999,"5min":2.032,"15min":2.577},"last":1.4710000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:13:05 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.5489999999999999,"15min":1.5389999999999999},"min":{"1min":1.294,"5min":1.0940000000000001,"15min":0.92800000000000005},"max":{"1min":1.9550000000000001,"5min":2.077,"15min":3.1709999999999998},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.591,"15min":1.5760000000000001},"min":{"1min":1.2669999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.883,"5min":2.101,"15min":2.8740000000000001},"last":1.4399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52316,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319524,"statfs":{"total":96636764160,"available":96583192576,"internally_reserved":0,"allocated":3297280,"data_stored":1624311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.0549999999999999,"15min":1.1080000000000001},"min":{"1min":0.63200000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.343,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1850000000000001,"15min":1.173},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5589999999999999,"5min":4.3970000000000002,"15min":4.3970000000000002},"last":1.3819999999999999}]},{"osd":1,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.159,"15min":1.2250000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.617,"5min":2.9380000000000002,"15min":2.9380000000000002},"last":1.296},{"interface":"front","average":{"1min":0.997,"5min":1.1140000000000001,"15min":1.163},"min":{"1min":0.66000000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4550000000000001,"5min":3.4169999999999998,"15min":3.4169999999999998},"last":1.2569999999999999}]},{"osd":2,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2869999999999999,"15min":1.2330000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.482,"5min":11.311,"15min":11.311},"last":0.871},{"interface":"front","average":{"1min":1.252,"5min":1.236,"15min":1.3160000000000001},"min":{"1min":0.82899999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.486}]},{"osd":3,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.2470000000000001,"15min":1.27},"min":{"1min":0.88800000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.877,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.258,"5min":1.347,"15min":1.3300000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.667,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":1}]},{"osd":5,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.377,"15min":1.3300000000000001},"min":{"1min":0.751,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.716,"5min":4.3010000000000002,"15min":4.3010000000000002},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.349,"15min":1.3160000000000001},"min":{"1min":0.70399999999999996,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.0249999999999999,"5min":3.71,"15min":3.71},"last":1.1140000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3899999999999999,"15min":1.3720000000000001},"min":{"1min":0.84299999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8759999999999999,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.4890000000000001,"15min":1.47},"min":{"1min":0.86499999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.1120000000000001,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.5529999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.486,"15min":1.4750000000000001},"min":{"1min":0.995,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0289999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.51},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5840000000000001,"15min":1.573},"min":{"1min":0.95699999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.1480000000000001,"5min":4.1390000000000002,"15min":4.1390000000000002},"last":1.4410000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56412,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315428,"statfs":{"total":96636764160,"available":96578998272,"internally_reserved":0,"allocated":3297280,"data_stored":1624311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.51600000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.581,"5min":3.4260000000000002,"15min":3.4260000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":1.0589999999999999},"min":{"1min":0.54000000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.534,"5min":1.95,"15min":1.95},"last":0.56399999999999995}]},{"osd":1,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.04,"15min":1.1279999999999999},"min":{"1min":0.59599999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.635,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.624},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.92400000000000004,"15min":0.97699999999999998},"min":{"1min":0.57199999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.532,"5min":3.863,"15min":3.863},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1,"15min":1.0620000000000001},"min":{"1min":0.52000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3149999999999999,"5min":3.597,"15min":3.597},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.028,"5min":1.139,"15min":1.2090000000000001},"min":{"1min":0.60499999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5609999999999999,"5min":3.0790000000000002,"15min":3.0790000000000002},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2549999999999999,"15min":1.282},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.383,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.232,"5min":1.2529999999999999,"15min":1.2849999999999999},"min":{"1min":0.85599999999999998,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9890000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2889999999999999,"15min":1.3180000000000001},"min":{"1min":0.82299999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8979999999999999,"5min":3.1240000000000001,"15min":3.1240000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.359,"5min":1.343,"15min":1.353},"min":{"1min":0.81799999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.0830000000000002,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.02}]},{"osd":6,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3580000000000001,"15min":1.4259999999999999},"min":{"1min":0.83499999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.3079999999999998,"5min":2.5649999999999999,"15min":3.0960000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.448,"15min":1.488},"min":{"1min":0.997,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4870000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.98899999999999999}]},{"osd":7,"last update":"Fri Dec 1 03:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.452,"15min":1.502},"min":{"1min":1.0429999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.0470000000000002,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.202},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.462,"15min":1.516},"min":{"1min":0.96199999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.2440000000000002,"5min":3.004,"15min":3.004},"last":0.95599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834576007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48220,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323620,"statfs":{"total":96636764160,"available":96587386880,"internally_reserved":0,"allocated":3297280,"data_stored":1624311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79700000000000004,"15min":0.86099999999999999},"min":{"1min":0.503,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.90800000000000003,"5min":1.411,"15min":1.4730000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.91900000000000004,"15min":1.0149999999999999},"min":{"1min":0.54400000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.198,"5min":1.5640000000000001,"15min":1.7310000000000001},"last":0.96199999999999997}]},{"osd":1,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":1.02},"min":{"1min":0.64600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.2569999999999999,"5min":1.512,"15min":1.698},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.97899999999999998,"15min":1.103},"min":{"1min":0.54000000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4399999999999999,"5min":1.5800000000000001,"15min":1.6599999999999999},"last":0.85999999999999999}]},{"osd":3,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.95299999999999996,"15min":1.014},"min":{"1min":0.45500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.502,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.03},{"interface":"front","average":{"1min":1.022,"5min":1.0609999999999999,"15min":1.171},"min":{"1min":0.63900000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.367,"5min":1.615,"15min":1.79},"last":1.3879999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.22,"15min":1.302},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.609,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1759999999999999,"15min":1.2609999999999999},"min":{"1min":0.64300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.508,"5min":1.714,"15min":1.835},"last":1.175}]},{"osd":5,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.282,"15min":1.3660000000000001},"min":{"1min":0.97599999999999998,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7509999999999999,"5min":1.7989999999999999,"15min":2.2040000000000002},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.296,"15min":1.387},"min":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5569999999999999,"5min":1.8740000000000001,"15min":2.016},"last":1.2270000000000001}]},{"osd":6,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3149999999999999,"15min":1.3640000000000001},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7190000000000001,"5min":1.9079999999999999,"15min":2.0289999999999999},"last":1.319},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3779999999999999,"15min":1.4690000000000001},"min":{"1min":1.024,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.8740000000000001,"5min":2.0150000000000001,"15min":2.3010000000000002},"last":1.589}]},{"osd":7,"last update":"Fri Dec 1 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3819999999999999,"15min":1.4590000000000001},"min":{"1min":0.89400000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.9099999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.3500000000000001,"15min":1.4239999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9610000000000001,"5min":2.0670000000000002,"15min":2.1379999999999999},"last":1.478}]}]},{"osd":1,"up_from":13,"seq":55834576006,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52896,"kb_used_data":3800,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94318944,"statfs":{"total":96636764160,"available":96582598656,"internally_reserved":0,"allocated":3891200,"data_stored":2214679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.81100000000000005,"15min":0.877},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.093,"5min":1.5569999999999999,"15min":1.6850000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88800000000000001,"15min":1.006},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.353,"5min":1.512,"15min":1.681},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82099999999999995,"15min":0.91300000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.335,"5min":1.6599999999999999,"15min":1.7030000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.98199999999999998,"15min":1.101},"min":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3819999999999999,"5min":1.581,"15min":1.7430000000000001},"last":1.212}]},{"osd":3,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.069,"15min":1.244},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.3799999999999999,"5min":1.593,"15min":1.788},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.998,"15min":1.093},"min":{"1min":0.48699999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3600000000000001,"5min":1.714,"15min":1.714},"last":1.0269999999999999}]},{"osd":4,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1519999999999999,"15min":1.202},"min":{"1min":0.61099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.302,"15min":1.381},"min":{"1min":0.76300000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":2.2109999999999999},"last":1.349}]},{"osd":5,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.29,"15min":1.3200000000000001},"min":{"1min":0.81499999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.2749999999999999,"15min":1.327},"min":{"1min":0.66100000000000003,"5min":0.496,"15min":0.496},"max":{"1min":3.75,"5min":3.75,"15min":3.75},"last":1.5289999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.327,"15min":1.4079999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5580000000000001,"5min":2.0680000000000001,"15min":2.2599999999999998},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.345,"15min":1.393},"min":{"1min":0.70499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8700000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.4670000000000001}]},{"osd":7,"last update":"Fri Dec 1 03:13:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.4219999999999999,"15min":1.502},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.5350000000000001,"5min":3.5350000000000001,"15min":3.5350000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.413,"15min":1.482},"min":{"1min":0.80400000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.617,"5min":2.1389999999999998,"15min":2.2069999999999999},"last":1.165}]}]},{"osd":0,"up_from":13,"seq":55834576008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60508,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311332,"statfs":{"total":96636764160,"available":96574803968,"internally_reserved":0,"allocated":3297280,"data_stored":1624311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86799999999999999,"15min":0.89000000000000001},"min":{"1min":0.54500000000000004,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.175,"5min":3.472,"15min":3.472},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91400000000000003,"15min":1.026},"min":{"1min":0.58899999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1950000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.47499999999999998}]},{"osd":2,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.002,"15min":1.056},"min":{"1min":0.5,"5min":0.5,"15min":0.48699999999999999},"max":{"1min":1.2849999999999999,"5min":2.0179999999999998,"15min":2.1429999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98399999999999999,"15min":1.097},"min":{"1min":0.61299999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4770000000000001,"5min":1.726,"15min":1.792},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93899999999999995,"15min":1.0369999999999999},"min":{"1min":0.47799999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.262,"5min":1.536,"15min":1.8839999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.135,"15min":1.21},"min":{"1min":0.629,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5369999999999999,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.54300000000000004}]},{"osd":4,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.2769999999999999,"15min":1.3420000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.2170000000000001,"15min":1.284},"min":{"1min":0.67600000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.752,"5min":2.0259999999999998,"15min":2.3210000000000002},"last":0.97399999999999998}]},{"osd":5,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3300000000000001,"15min":1.393},"min":{"1min":0.81100000000000005,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.986,"5min":1.986,"15min":2.7719999999999998},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.3400000000000001,"15min":1.413},"min":{"1min":0.84499999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7969999999999999,"5min":2.484,"15min":2.5880000000000001},"last":1.1619999999999999}]},{"osd":6,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.298,"15min":1.363},"min":{"1min":0.94399999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.633,"5min":1.835,"15min":2.2029999999999998},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4359999999999999,"15min":1.4770000000000001},"min":{"1min":0.84799999999999998,"5min":0.77600000000000002,"15min":0.75800000000000001},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.8759999999999999},"last":0.95399999999999996}]},{"osd":7,"last update":"Fri Dec 1 03:12:59 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4330000000000001,"15min":1.502},"min":{"1min":1.0960000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.2069999999999999,"5min":2.411,"15min":2.7040000000000002},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4379999999999999,"15min":1.472},"min":{"1min":0.92300000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":3.25,"5min":3.25,"15min":3.25},"last":1.008}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T03:13:22.245 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-12-01T03:13:22.452 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.451+0000 7f3bcf79f640 1 Processor -- start 2023-12-01T03:13:22.453 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.452+0000 7f3bcf79f640 1 -- start start 2023-12-01T03:13:22.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.453+0000 7f3bcf79f640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3bc8065960 0x7f3bc8161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.453+0000 7f3bcf79f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3bc806d510 0x7f3bc8065d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.453+0000 7f3bcf79f640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc8070700 con 0x7f3bc8070b90 2023-12-01T03:13:22.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.453+0000 7f3bcf79f640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc81618a0 con 0x7f3bc8065960 2023-12-01T03:13:22.454 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.453+0000 7f3bcf79f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc80d0850 con 0x7f3bc806d510 2023-12-01T03:13:22.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.454+0000 7f3bcd53b640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7f3bc8065960 0x7f3bc8161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.454+0000 7f3bcdd3c640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7f3bc8070b90 0x7f3bc80aca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59248/0 (socket says 172.21.15.115:59248) 2023-12-01T03:13:22.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.454+0000 7f3bcdd3c640 1 -- 172.21.15.115:0/2907011290 learned_addr learned my addr 172.21.15.115:0/2907011290 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:22.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.454+0000 7f3bccd3a640 1 --2- 172.21.15.115:0/2907011290 >> v2:172.21.15.142:3300/0 conn(0x7f3bc806d510 0x7f3bc8065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.454+0000 7f3bc7fff640 1 -- 172.21.15.115:0/2907011290 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912125319 0 0) 0x7f3bc8070700 con 0x7f3bc8070b90 2023-12-01T03:13:22.455 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.454+0000 7f3bc7fff640 1 -- 172.21.15.115:0/2907011290 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bb8003520 con 0x7f3bc8070b90 2023-12-01T03:13:22.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.454+0000 7f3bcd53b640 1 -- 172.21.15.115:0/2907011290 >> v2:172.21.15.142:3300/0 conn(0x7f3bc806d510 msgr2=0x7f3bc8065d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:22.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.454+0000 7f3bcd53b640 1 --2- 172.21.15.115:0/2907011290 >> v2:172.21.15.142:3300/0 conn(0x7f3bc806d510 0x7f3bc8065d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.454+0000 7f3bcd53b640 1 -- 172.21.15.115:0/2907011290 >> v1:172.21.15.115:6789/0 conn(0x7f3bc8070b90 legacy=0x7f3bc80aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.454+0000 7f3bcd53b640 1 -- 172.21.15.115:0/2907011290 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8066280 con 0x7f3bc8065960 2023-12-01T03:13:22.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.455+0000 7f3bcd53b640 1 --2- 172.21.15.115:0/2907011290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 0x7f3bc8161360 secure :-1 s=READY pgs=3168 cs=0 l=1 rev1=1 crypto rx=0x7f3bc806ffe0 tx=0x7f3bbc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10aa93a01b151bd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:22.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.456+0000 7f3bc7fff640 1 -- 172.21.15.115:0/2907011290 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc02cc80 con 0x7f3bc8065960 2023-12-01T03:13:22.456 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.456+0000 7f3bc7fff640 1 -- 172.21.15.115:0/2907011290 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc02cde0 con 0x7f3bc8065960 2023-12-01T03:13:22.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.456+0000 7f3bc7fff640 1 -- 172.21.15.115:0/2907011290 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc034660 con 0x7f3bc8065960 2023-12-01T03:13:22.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.456+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2907011290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 msgr2=0x7f3bc8161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.456+0000 7f3bcf79f640 1 --2- 172.21.15.115:0/2907011290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 0x7f3bc8161360 secure :-1 s=READY pgs=3168 cs=0 l=1 rev1=1 crypto rx=0x7f3bc806ffe0 tx=0x7f3bbc02bed0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.457 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.456+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2907011290 shutdown_connections 2023-12-01T03:13:22.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.456+0000 7f3bcf79f640 1 --2- 172.21.15.115:0/2907011290 >> v2:172.21.15.142:3300/0 conn(0x7f3bc806d510 0x7f3bc8065d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.456+0000 7f3bcf79f640 1 --2- 172.21.15.115:0/2907011290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 0x7f3bc8161360 unknown :-1 s=CLOSED pgs=3168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.456+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2907011290 >> 172.21.15.115:0/2907011290 conn(0x7f3bc80cf8e0 msgr2=0x7f3bc80cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:22.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.457+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2907011290 shutdown_connections 2023-12-01T03:13:22.458 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.457+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2907011290 wait complete. 2023-12-01T03:13:22.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.458+0000 7f3bcf79f640 1 Processor -- start 2023-12-01T03:13:22.459 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.459+0000 7f3bcf79f640 1 -- start start 2023-12-01T03:13:22.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.459+0000 7f3bcf79f640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 0x7f3bc81f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.459+0000 7f3bcf79f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc806d510 0x7f3bc80c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcf79f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc8070b90 0x7f3bc80c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcf79f640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc81f2930 con 0x7f3bc8065960 2023-12-01T03:13:22.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcf79f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80cda10 con 0x7f3bc8070b90 2023-12-01T03:13:22.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcf79f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc81f2b20 con 0x7f3bc806d510 2023-12-01T03:13:22.461 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcd53b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 0x7f3bc81f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcd53b640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 0x7f3bc81f23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32878/0 (socket says 172.21.15.115:32878) 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcd53b640 1 -- 172.21.15.115:0/2348038510 learned_addr learned my addr 172.21.15.115:0/2348038510 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcdd3c640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc8070b90 0x7f3bc80c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bccd3a640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc806d510 0x7f3bc80c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcd53b640 1 -- 172.21.15.115:0/2348038510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc806d510 msgr2=0x7f3bc80c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcd53b640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc806d510 0x7f3bc80c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcd53b640 1 -- 172.21.15.115:0/2348038510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc8070b90 msgr2=0x7f3bc80c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcd53b640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc8070b90 0x7f3bc80c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcd53b640 1 -- 172.21.15.115:0/2348038510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc009a70 con 0x7f3bc8065960 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.460+0000 7f3bcdd3c640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc8070b90 0x7f3bc80c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T03:13:22.462 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.461+0000 7f3bcd53b640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 0x7f3bc81f23f0 secure :-1 s=READY pgs=3169 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc037040 tx=0x7f3bbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:22.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.461+0000 7f3bc5ffb640 1 -- 172.21.15.115:0/2348038510 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc040650 con 0x7f3bc8065960 2023-12-01T03:13:22.463 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.461+0000 7f3bc5ffb640 1 -- 172.21.15.115:0/2348038510 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc0407b0 con 0x7f3bc8065960 2023-12-01T03:13:22.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.461+0000 7f3bc5ffb640 1 -- 172.21.15.115:0/2348038510 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc040b60 con 0x7f3bc8065960 2023-12-01T03:13:22.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.461+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2348038510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc81f2e00 con 0x7f3bc8065960 2023-12-01T03:13:22.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.461+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2348038510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc81f33c0 con 0x7f3bc8065960 2023-12-01T03:13:22.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.462+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2348038510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc804e890 con 0x7f3bc8065960 2023-12-01T03:13:22.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.463+0000 7f3bc5ffb640 1 -- 172.21.15.115:0/2348038510 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7f3bbc040cc0 con 0x7f3bc8065960 2023-12-01T03:13:22.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.464+0000 7f3bc5ffb640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bac06a550 0x7f3bac06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.464 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.464+0000 7f3bc5ffb640 1 -- 172.21.15.115:0/2348038510 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7f3bbc0ab920 con 0x7f3bc8065960 2023-12-01T03:13:22.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.464+0000 7f3bccd3a640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bac06a550 0x7f3bac06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.465 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.465+0000 7f3bccd3a640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bac06a550 0x7f3bac06ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006fd0 tx=0x7f3bb4008040 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:22.468 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.468+0000 7f3bc5ffb640 1 -- 172.21.15.115:0/2348038510 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7f3bbc076d80 con 0x7f3bc8065960 2023-12-01T03:13:22.661 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.660+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2348038510 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f3bc81f3580 con 0x7f3bc8065960 2023-12-01T03:13:22.669 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.669+0000 7f3bc5ffb640 1 -- 172.21.15.115:0/2348038510 <== mon.0 v2:172.21.15.115:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3bbc076d80 con 0x7f3bc8065960 2023-12-01T03:13:22.670 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.669+0000 7f3bc5ffb640 1 -- 172.21.15.115:0/2348038510 <== mon.0 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v88)=0 v88) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3bbc009450 con 0x7f3bc8065960 2023-12-01T03:13:22.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.672+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2348038510 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bac06a550 msgr2=0x7f3bac06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.672+0000 7f3bcf79f640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bac06a550 0x7f3bac06ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006fd0 tx=0x7f3bb4008040 comp rx=0 tx=0).stop 2023-12-01T03:13:22.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.672+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2348038510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 msgr2=0x7f3bc81f23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.672+0000 7f3bcf79f640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 0x7f3bc81f23f0 secure :-1 s=READY pgs=3169 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc037040 tx=0x7f3bbc004a40 comp rx=0 tx=0).stop 2023-12-01T03:13:22.673 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.672+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2348038510 shutdown_connections 2023-12-01T03:13:22.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.672+0000 7f3bcf79f640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7f3bac06a550 0x7f3bac06ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.672+0000 7f3bcf79f640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3bc806d510 0x7f3bc80c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.672+0000 7f3bcf79f640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bc8070b90 0x7f3bc80c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.672+0000 7f3bcf79f640 1 --2- 172.21.15.115:0/2348038510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bc8065960 0x7f3bc81f23f0 unknown :-1 s=CLOSED pgs=3169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.672+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2348038510 >> 172.21.15.115:0/2348038510 conn(0x7f3bc80cf8e0 msgr2=0x7f3bc80ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:22.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.673+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2348038510 shutdown_connections 2023-12-01T03:13:22.674 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.673+0000 7f3bcf79f640 1 -- 172.21.15.115:0/2348038510 wait complete. 2023-12-01T03:13:22.693 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T03:13:22.693 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T03:13:22.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.899+0000 7fccee096640 1 Processor -- start 2023-12-01T03:13:22.900 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.899+0000 7fccee096640 1 -- start start 2023-12-01T03:13:22.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.901+0000 7fccee096640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcce806f2f0 0x7fcce806bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.901+0000 7fccee096640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce80add10 con 0x7fcce80cad70 2023-12-01T03:13:22.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.901+0000 7fccee096640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce80c5760 con 0x7fcce815e2f0 2023-12-01T03:13:22.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.901+0000 7fccee096640 1 -- --> v2:172.21.15.115:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce815e6d0 con 0x7fcce806f2f0 2023-12-01T03:13:22.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.901+0000 7fcce7fff640 1 --2- >> v2:172.21.15.115:3300/0 conn(0x7fcce806f2f0 0x7fcce806bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.901+0000 7fcce6ffd640 1 --1- >> v1:172.21.15.115:6789/0 conn(0x7fcce80cad70 0x7fcce806b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.115:6789/0 says I am v1:172.21.15.115:59262/0 (socket says 172.21.15.115:59262) 2023-12-01T03:13:22.902 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.901+0000 7fcce6ffd640 1 -- 172.21.15.115:0/3734959106 learned_addr learned my addr 172.21.15.115:0/3734959106 (peer_addr_for_me v1:172.21.15.115:0/0) 2023-12-01T03:13:22.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.902+0000 7fcce7fff640 1 -- 172.21.15.115:0/3734959106 >> v1:172.21.15.142:6789/0 conn(0x7fcce815e2f0 legacy=0x7fcce80ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T03:13:22.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.902+0000 7fcce7fff640 1 -- 172.21.15.115:0/3734959106 >> v1:172.21.15.115:6789/0 conn(0x7fcce80cad70 legacy=0x7fcce806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.902+0000 7fcce7fff640 1 -- 172.21.15.115:0/3734959106 --> v2:172.21.15.115:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce80cf7b0 con 0x7fcce806f2f0 2023-12-01T03:13:22.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.902+0000 7fcce67fc640 1 -- 172.21.15.115:0/3734959106 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338026316 0 0) 0x7fcce80add10 con 0x7fcce80cad70 2023-12-01T03:13:22.903 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.903+0000 7fcce7fff640 1 --2- 172.21.15.115:0/3734959106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 0x7fcce806bcd0 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009dc0 tx=0x7fcce002be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871454f3735b7efd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:22.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.903+0000 7fcce67fc640 1 -- 172.21.15.115:0/3734959106 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce002cbb0 con 0x7fcce806f2f0 2023-12-01T03:13:22.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.903+0000 7fcce67fc640 1 -- 172.21.15.115:0/3734959106 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcce002cd10 con 0x7fcce806f2f0 2023-12-01T03:13:22.904 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.903+0000 7fcce67fc640 1 -- 172.21.15.115:0/3734959106 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0034680 con 0x7fcce806f2f0 2023-12-01T03:13:22.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.904+0000 7fccee096640 1 -- 172.21.15.115:0/3734959106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 msgr2=0x7fcce806bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.904+0000 7fccee096640 1 --2- 172.21.15.115:0/3734959106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 0x7fcce806bcd0 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009dc0 tx=0x7fcce002be00 comp rx=0 tx=0).stop 2023-12-01T03:13:22.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.904+0000 7fccee096640 1 -- 172.21.15.115:0/3734959106 shutdown_connections 2023-12-01T03:13:22.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.904+0000 7fccee096640 1 --2- 172.21.15.115:0/3734959106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 0x7fcce806bcd0 unknown :-1 s=CLOSED pgs=3171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.904+0000 7fccee096640 1 -- 172.21.15.115:0/3734959106 >> 172.21.15.115:0/3734959106 conn(0x7fcce80ce880 msgr2=0x7fcce80cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:22.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.904+0000 7fccee096640 1 -- 172.21.15.115:0/3734959106 shutdown_connections 2023-12-01T03:13:22.905 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.904+0000 7fccee096640 1 -- 172.21.15.115:0/3734959106 wait complete. 2023-12-01T03:13:22.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.906+0000 7fccee096640 1 Processor -- start 2023-12-01T03:13:22.906 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.906+0000 7fccee096640 1 -- start start 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fccee096640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 0x7fcce8147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fccee096640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce80cad70 0x7fcce80c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fccee096640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcce815e2f0 0x7fcce815c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fccee096640 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce80c32a0 con 0x7fcce806f2f0 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fccee096640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce80c95e0 con 0x7fcce80cad70 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fccee096640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce80c23b0 con 0x7fcce815e2f0 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fcce77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 0x7fcce8147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fcce77fe640 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 0x7fcce8147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:32898/0 (socket says 172.21.15.115:32898) 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fcce77fe640 1 -- 172.21.15.115:0/3451305581 learned_addr learned my addr 172.21.15.115:0/3451305581 (peer_addr_for_me v2:172.21.15.115:0/0) 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fcce77fe640 1 -- 172.21.15.115:0/3451305581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcce815e2f0 msgr2=0x7fcce815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fcce77fe640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcce815e2f0 0x7fcce815c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.909 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fcce77fe640 1 -- 172.21.15.115:0/3451305581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce80cad70 msgr2=0x7fcce80c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:22.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fcce77fe640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce80cad70 0x7fcce80c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:22.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.907+0000 7fcce77fe640 1 -- 172.21.15.115:0/3451305581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce0009a70 con 0x7fcce806f2f0 2023-12-01T03:13:22.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.908+0000 7fcce77fe640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 0x7fcce8147c40 secure :-1 s=READY pgs=3172 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c450 tx=0x7fccd800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:22.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.908+0000 7fcccffff640 1 -- 172.21.15.115:0/3451305581 <== mon.0 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80052d0 con 0x7fcce806f2f0 2023-12-01T03:13:22.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.908+0000 7fcccffff640 1 -- 172.21.15.115:0/3451305581 <== mon.0 v2:172.21.15.115:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fccd8005430 con 0x7fcce806f2f0 2023-12-01T03:13:22.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.908+0000 7fcccffff640 1 -- 172.21.15.115:0/3451305581 <== mon.0 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8012890 con 0x7fcce806f2f0 2023-12-01T03:13:22.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.908+0000 7fccee096640 1 -- 172.21.15.115:0/3451305581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce80c2690 con 0x7fcce806f2f0 2023-12-01T03:13:22.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.908+0000 7fccee096640 1 -- 172.21.15.115:0/3451305581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce80c2c50 con 0x7fcce806f2f0 2023-12-01T03:13:22.910 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.910+0000 7fccee096640 1 -- 172.21.15.115:0/3451305581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce806fda0 con 0x7fcce806f2f0 2023-12-01T03:13:22.911 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.910+0000 7fcccffff640 1 -- 172.21.15.115:0/3451305581 <== mon.0 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 151) v1 ==== 88709+0+0 (secure 0 0 0) 0x7fccd80129f0 con 0x7fcce806f2f0 2023-12-01T03:13:22.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.912+0000 7fcccffff640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fccd406a550 0x7fccd406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T03:13:22.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.912+0000 7fcccffff640 1 -- 172.21.15.115:0/3451305581 <== mon.0 v2:172.21.15.115:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 5718+0+0 (secure 0 0 0) 0x7fccd8088710 con 0x7fcce806f2f0 2023-12-01T03:13:22.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.912+0000 7fcce6ffd640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fccd406a550 0x7fccd406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T03:13:22.913 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.913+0000 7fcce6ffd640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fccd406a550 0x7fccd406ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fccdc000c00 tx=0x7fccdc007a60 comp rx=0 tx=0).ready entity=mgr.12555 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T03:13:22.917 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:22.917+0000 7fcccffff640 1 -- 172.21.15.115:0/3451305581 <== mon.0 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179298 (secure 0 0 0) 0x7fccd8053b50 con 0x7fcce806f2f0 2023-12-01T03:13:23.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.143+0000 7fccee096640 1 -- 172.21.15.115:0/3451305581 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcce804e890 con 0x7fcce806f2f0 2023-12-01T03:13:23.146 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.145+0000 7fcccffff640 1 -- 172.21.15.115:0/3451305581 <== mon.0 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+3545 (secure 0 0 0) 0x7fccd8057170 con 0x7fcce806f2f0 2023-12-01T03:13:23.146 INFO:teuthology.orchestra.run.smithi115.stdout: 2023-12-01T03:13:23.147 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":128,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2042114371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6833","nonce":2042114371},{"type":"v1","addr":"172.21.15.142:6835","nonce":2042114371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":8266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6834/1215693160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":1215693160},{"type":"v1","addr":"172.21.15.142:6834","nonce":1215693160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":12234,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6835/727621046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6834","nonce":727621046},{"type":"v1","addr":"172.21.15.115:6835","nonce":727621046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121},{"gid":12243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.115:6837/3018042052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6836","nonce":3018042052},{"type":"v1","addr":"172.21.15.115:6837","nonce":3018042052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[]} 2023-12-01T03:13:23.147 INFO:teuthology.orchestra.run.smithi115.stderr:dumped fsmap epoch 128 2023-12-01T03:13:23.148 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.147+0000 7fccee096640 1 -- 172.21.15.115:0/3451305581 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fccd406a550 msgr2=0x7fccd406ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.147+0000 7fccee096640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fccd406a550 0x7fccd406ca10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fccdc000c00 tx=0x7fccdc007a60 comp rx=0 tx=0).stop 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.147+0000 7fccee096640 1 -- 172.21.15.115:0/3451305581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 msgr2=0x7fcce8147c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.147+0000 7fccee096640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 0x7fcce8147c40 secure :-1 s=READY pgs=3172 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c450 tx=0x7fccd800c910 comp rx=0 tx=0).stop 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.147+0000 7fccee096640 1 -- 172.21.15.115:0/3451305581 shutdown_connections 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.147+0000 7fccee096640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.115:6822/138932,v1:172.21.15.115:6824/138932] conn(0x7fccd406a550 0x7fccd406ca10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.147+0000 7fccee096640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcce815e2f0 0x7fcce815c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.147+0000 7fccee096640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcce80cad70 0x7fcce80c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.147+0000 7fccee096640 1 --2- 172.21.15.115:0/3451305581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcce806f2f0 0x7fcce8147c40 unknown :-1 s=CLOSED pgs=3172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.147+0000 7fccee096640 1 -- 172.21.15.115:0/3451305581 >> 172.21.15.115:0/3451305581 conn(0x7fcce80ce880 msgr2=0x7fcce80c1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.148+0000 7fccee096640 1 -- 172.21.15.115:0/3451305581 shutdown_connections 2023-12-01T03:13:23.149 INFO:teuthology.orchestra.run.smithi115.stderr:2023-12-01T03:13:23.148+0000 7fccee096640 1 -- 172.21.15.115:0/3451305581 wait complete. 2023-12-01T03:13:23.166 DEBUG:tasks.cephfs.filesystem:already dead... 2023-12-01T03:13:23.167 INFO:teuthology.misc:Shutting down mds daemons... 2023-12-01T03:13:23.167 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-01T03:13:23.167 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.183 INFO:tasks.ceph.mds.a:Stopped 2023-12-01T03:13:23.183 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-01T03:13:23.183 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.252 INFO:tasks.ceph.mds.c:Stopped 2023-12-01T03:13:23.252 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-01T03:13:23.252 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.283 INFO:tasks.ceph.mds.b:Stopped 2023-12-01T03:13:23.283 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-01T03:13:23.283 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.349 INFO:tasks.ceph.mds.d:Stopped 2023-12-01T03:13:23.349 INFO:teuthology.misc:Shutting down osd daemons... 2023-12-01T03:13:23.350 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-12-01T03:13:23.350 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.419 INFO:tasks.ceph.osd.0:Stopped 2023-12-01T03:13:23.420 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-12-01T03:13:23.420 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.489 INFO:tasks.ceph.osd.1:Stopped 2023-12-01T03:13:23.489 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-12-01T03:13:23.489 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.558 INFO:tasks.ceph.osd.2:Stopped 2023-12-01T03:13:23.558 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-12-01T03:13:23.558 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.635 INFO:tasks.ceph.osd.3:Stopped 2023-12-01T03:13:23.635 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-12-01T03:13:23.635 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.701 INFO:tasks.ceph.osd.4:Stopped 2023-12-01T03:13:23.701 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-12-01T03:13:23.701 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.791 INFO:tasks.ceph.osd.5:Stopped 2023-12-01T03:13:23.791 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-12-01T03:13:23.791 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.853 INFO:tasks.ceph.osd.6:Stopped 2023-12-01T03:13:23.853 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-12-01T03:13:23.854 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:23.922 INFO:tasks.ceph.osd.7:Stopped 2023-12-01T03:13:23.923 INFO:teuthology.misc:Shutting down mgr daemons... 2023-12-01T03:13:23.923 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-01T03:13:23.923 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:24.011 INFO:tasks.ceph.mgr.y:Stopped 2023-12-01T03:13:24.011 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-01T03:13:24.011 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:27.225 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-12-01T03:13:30.113 INFO:tasks.ceph.mgr.x:Stopped 2023-12-01T03:13:30.113 INFO:teuthology.misc:Shutting down mon daemons... 2023-12-01T03:13:30.113 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-01T03:13:30.113 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:30.196 INFO:tasks.ceph.mon.a:Stopped 2023-12-01T03:13:30.196 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-01T03:13:30.197 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:30.259 INFO:tasks.ceph.mon.b:Stopped 2023-12-01T03:13:30.259 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-01T03:13:30.259 INFO:teuthology.orchestra.run:waiting for 300 2023-12-01T03:13:30.302 INFO:tasks.ceph.mon.c:Stopped 2023-12-01T03:13:30.302 INFO:tasks.ceph:Checking cluster log for badness... 2023-12-01T03:13:30.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-01T03:13:30.418 INFO:teuthology.orchestra.run.smithi115.stdout:2023-12-01T01:38:40.360718+0000 mon.a (mon.0) 476 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 4 pgs peering (PG_AVAILABILITY) 2023-12-01T03:13:30.419 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-12-01T03:13:30.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-01T03:13:30.506 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-01T03:13:30.591 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-01T03:13:30.673 INFO:teuthology.orchestra.run.smithi115.stdout:2023-12-01T01:38:40.360718+0000 mon.a (mon.0) 476 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 4 pgs peering (PG_AVAILABILITY) 2023-12-01T03:13:30.674 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T03:13:30.674 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-12-01T03:13:31.009 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T03:13:31.009 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-12-01T03:13:31.182 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T03:13:31.182 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-12-01T03:13:31.324 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T03:13:31.325 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-12-01T03:13:31.488 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T03:13:31.489 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-12-01T03:13:31.774 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:13:31.784 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:13:31.820 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T03:13:31.821 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-12-01T03:13:32.028 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T03:13:32.028 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-12-01T03:13:32.193 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T03:13:32.194 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-12-01T03:13:32.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-12-01T03:13:32.360 INFO:tasks.ceph:Archiving mon data... 2023-12-01T03:13:32.361 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199/data/mon.a.tgz 2023-12-01T03:13:32.361 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2023-12-01T03:13:32.412 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.0IO6bkQuiV 2023-12-01T03:13:32.413 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.0IO6bkQuiV 2023-12-01T03:13:33.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.0IO6bkQuiV 2023-12-01T03:13:33.260 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.0IO6bkQuiV is 4MB 2023-12-01T03:13:33.427 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.0IO6bkQuiV 2023-12-01T03:13:33.489 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199/data/mon.b.tgz 2023-12-01T03:13:33.489 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2023-12-01T03:13:33.548 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.k9Y0AUorwg 2023-12-01T03:13:33.548 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.k9Y0AUorwg 2023-12-01T03:13:34.293 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.k9Y0AUorwg 2023-12-01T03:13:34.427 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.k9Y0AUorwg is 4MB 2023-12-01T03:13:34.601 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.k9Y0AUorwg 2023-12-01T03:13:34.663 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199/data/mon.c.tgz 2023-12-01T03:13:34.664 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2023-12-01T03:13:34.718 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.1wglWuygJ4 2023-12-01T03:13:34.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.1wglWuygJ4 2023-12-01T03:13:35.453 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.1wglWuygJ4 2023-12-01T03:13:35.595 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.1wglWuygJ4 is 4MB 2023-12-01T03:13:35.763 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.1wglWuygJ4 2023-12-01T03:13:35.823 INFO:tasks.ceph:Cleaning ceph cluster... 2023-12-01T03:13:35.823 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-01T03:13:35.832 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-01T03:13:35.927 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-12-01T03:13:35.928 DEBUG:teuthology.orchestra.run.smithi115:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-01T03:13:35.937 DEBUG:teuthology.orchestra.run.smithi142:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-01T03:13:36.028 INFO:teuthology.orchestra.run.smithi115.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-01T03:13:36.036 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-01T03:13:36.048 INFO:tasks.ceph:Archiving crash dumps... 2023-12-01T03:13:36.050 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199/remote/smithi115/crash 2023-12-01T03:13:36.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-01T03:13:36.130 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199/remote/smithi142/crash 2023-12-01T03:13:36.131 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-01T03:13:36.229 INFO:tasks.ceph:Shutting down logrotate 2023-12-01T03:13:36.229 DEBUG:teuthology.orchestra.run.smithi115:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:13:36.238 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-01T03:13:36.361 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-01T03:13:36.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-01T03:13:36.555 INFO:tasks.ceph:Compressing logs... 2023-12-01T03:13:36.556 DEBUG:teuthology.orchestra.run.smithi115:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T03:13:36.565 DEBUG:teuthology.orchestra.run.smithi142:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T03:13:56.569 INFO:tasks.ceph:Archiving logs... 2023-12-01T03:13:56.569 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199/remote/smithi115/log 2023-12-01T03:13:56.571 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-01T03:13:59.758 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199/remote/smithi142/log 2023-12-01T03:13:59.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-01T03:14:01.622 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-01T03:14:01.632 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-01T03:14:01.632 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-01T03:14:01.642 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-01T03:14:01.798 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-01T03:14:01.807 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-01T03:14:01.808 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T03:14:01.817 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T03:14:01.866 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2023-12-01T03:14:01.874 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2023-12-01T03:14:01.875 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-01T03:14:01.875 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2023-12-01T03:14:01.875 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 111 -198us[ -205us] +/- 98ms 2023-12-01T03:14:01.875 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 +234us[ +230us] +/- 68ms 2023-12-01T03:14:01.876 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 108 -31us[ -35us] +/- 68ms 2023-12-01T03:14:01.876 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T03:14:01.884 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-01T03:14:01.884 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2023-12-01T03:14:01.884 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 112 -212us[ -213us] +/- 98ms 2023-12-01T03:14:01.884 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 +203us[ +201us] +/- 68ms 2023-12-01T03:14:01.884 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 104 -49us[ -51us] +/- 68ms 2023-12-01T03:14:01.884 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T03:14:01.885 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-01T03:14:01.898 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-01T03:14:01.899 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-01T03:14:01.911 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-01T03:14:02.094 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-01T03:14:02.359 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-01T03:14:02.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T03:14:02.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-01T03:14:02.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T03:14:02.573 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-01T03:14:02.583 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-01T03:14:02.604 INFO:teuthology.task.internal:Duration was 6679.735821 seconds 2023-12-01T03:14:02.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-01T03:14:02.617 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-01T03:14:02.617 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-01T03:14:02.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-01T03:14:02.790 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-01T03:14:02.797 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-01T03:14:03.267 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-01T03:14:03.267 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2023-12-01T03:14:03.268 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-01T03:14:03.341 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2023-12-01T03:14:03.341 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-01T03:14:03.419 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-01T03:14:03.419 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T03:14:03.428 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T03:14:03.519 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-01T03:14:03.519 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-01T03:14:03.525 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-01T03:14:03.778 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-01T03:14:03.789 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-01T03:14:03.789 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-01T03:14:03.879 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-01T03:14:03.982 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-01T03:14:03.992 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-01T03:14:04.001 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-01T03:14:04.106 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2023-12-01T03:14:04.113 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2023-12-01T03:14:04.165 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-01T03:14:04.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T03:14:04.215 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-01T03:14:04.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T03:14:04.273 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-01T03:14:04.284 INFO:teuthology.task.internal:Transferring archived files... 2023-12-01T03:14:04.284 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199/remote/smithi115 2023-12-01T03:14:04.284 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-01T03:14:04.390 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199/remote/smithi142 2023-12-01T03:14:04.390 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-01T03:14:04.500 INFO:teuthology.task.internal:Removing archive directory... 2023-12-01T03:14:04.500 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-01T03:14:04.509 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-01T03:14:04.566 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-01T03:14:04.577 INFO:teuthology.task.internal:Not uploading archives. 2023-12-01T03:14:04.577 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-01T03:14:04.590 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-01T03:14:04.590 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-01T03:14:04.597 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-01T03:14:04.647 INFO:teuthology.orchestra.run.smithi115.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 1 03:14 /home/ubuntu/cephtest 2023-12-01T03:14:04.655 INFO:teuthology.orchestra.run.smithi142.stdout: 524696 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 1 03:14 /home/ubuntu/cephtest 2023-12-01T03:14:04.658 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-01T03:14:04.670 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-01T03:14:04.692 INFO:teuthology.nuke:Checking targets against current locks 2023-12-01T03:14:04.718 DEBUG:teuthology.nuke:shortname: smithi115 2023-12-01T03:14:04.719 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-01T03:14:04.742 DEBUG:teuthology.nuke:shortname: smithi142 2023-12-01T03:14:04.742 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-01T03:14:04.746 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 01:13:49.534310', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAG1ZN4VL+XG0I899Gyv9++bmYkxtxRfIGXaOnv+Y5GEYm47zUgmZY1Nos8WsFkWxPJZYnlnIPcgDlygBFmRhMI='} 2023-12-01T03:14:04.768 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 01:13:49.532312', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrWw7VN9mng46aNqeugrNH8siuKXNZ4jODSxOlLGm+vYJVed2n0zJzYcvmZ9uTx05qQgz+NtqqnVuk7wDuI/G8='} 2023-12-01T03:14:04.801 INFO:teuthology.orchestra.console.smithi115:Power off 2023-12-01T03:14:04.801 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-01T03:14:04.816 INFO:teuthology.orchestra.console.smithi142:Power off 2023-12-01T03:14:04.816 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-01T03:14:04.824 DEBUG:teuthology.orchestra.console.smithi115:power off output: Chassis Power Control: Down/Off 2023-12-01T03:14:04.824 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T03:14:04.838 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2023-12-01T03:14:04.838 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T03:14:04.847 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2023-12-01T03:14:04.860 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2023-12-01T03:14:08.848 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T03:14:08.861 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T03:14:08.959 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2023-12-01T03:14:08.971 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2023-12-01T03:14:12.961 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T03:14:12.973 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T03:14:13.074 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on 2023-12-01T03:14:13.084 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2023-12-01T03:14:17.078 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T03:14:17.087 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T03:14:17.190 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off 2023-12-01T03:14:17.196 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2023-12-01T03:14:17.290 INFO:teuthology.orchestra.console.smithi115:Power off completed 2023-12-01T03:14:17.297 INFO:teuthology.orchestra.console.smithi142:Power off completed 2023-12-01T03:14:17.402 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 client.1-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6679.735821247101 failure_reason: '"2023-12-01T01:38:40.360718+0000 mon.a (mon.0) 476 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 4 pgs peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-12-01T03:14:17.402 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-01T03:14:17.485 INFO:teuthology.run:FAIL